Merge tag 'acpi-5.17-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm
Pull ACPI updates from Rafael Wysocki:
"These are usual ACPICA code updates (although there are more of them
than in the last few releases), a noticeable EC driver update (which
mostly consists of cleanups, though), the device enumeration quirks
handling rework from Hans, some updates eliminating unnecessary CPU
cache flushing in some places (processor idle and system-wide PM code)
and a bunch of assorted cleanups and fixes.
Specifics:
- Update ACPICA code in the kernel to the 20211217 upstream release
including the following changes:
- iASL/Disassembler: Additional support for NHLT table (Bob
Moore).
- Change a return_ACPI_STATUS (AE_BAD_PARAMETER) (Bob Moore).
- Fix a couple of warnings under MSVC (Bob Moore).
- iASL: Add TDEL table to both compiler/disassembler (Bob Moore).
- iASL/NHLT table: "Specific Data" field support (Bob Moore).
- Use original data_table_region pointer for accesses (Jessica
Clarke).
- Use original pointer for virtual origin tables (Jessica Clarke).
- Macros: Remove ACPI_PHYSADDR_TO_PTR (Jessica Clarke).
- Avoid subobject buffer overflow when validating RSDP signature
(Jessica Clarke).
- iASL: Add suppport for AGDI table (Ilkka Koskinen).
- Hardware: Do not flush CPU cache when entering S4 and S5 (Kirill
A. Shutemov).
- Expand the ACPI_ACCESS_ definitions (Mark Langsdorf).
- Utilities: Avoid deleting the same object twice in a row (Rafael
Wysocki).
- Executer: Fix REFCLASS_REFOF case in acpi_ex_opcode_1A_0T_1R()
(Rafael Wysocki).
- Fix AEST Processor generic resource substructure data field byte
length (Shuuichirou Ishii).
- Fix wrong interpretation of PCC address (Sudeep Holla).
- Add support for PCC Opregion special context data (Sudeep
Holla).
- Implement OperationRegion handler for PCC Type 3 subtype (Sudeep
Holla).
- Introduce acpi_fetch_acpi_dev() as a replacement for
acpi_bus_get_device() and use it in the ACPI subsystem (Rafael
Wysocki).
- Avoid using _CID for device enumaration if _HID is missing or
invalid (Rafael Wysocki).
- Rework quirk handling during ACPI device enumeration and add some
new quirks for known broken platforms (Hans de Goede).
- Avoid unnecessary or redundant CPU cache flushing during system PM
transitions (Kirill A. Shutemov).
- Add PM debug messages related to power resources (Rafael Wysocki).
- Fix kernel-doc comment in the PCI host bridge ACPI driver (Yang
Li).
- Rework flushing of EC work while suspended to idle and clean up the
handling of events in the ACPI EC driver (Rafael Wysocki).
- Prohibit ec_sys module parameter write_support from being used when
the system is locked down (Hans de Goede).
- Make the ACPI processor thermal driver use cpufreq_cpu_get() to
check for presence of cpufreq policy (Manfred Spraul).
- Avoid unnecessary CPU cache flushing in the ACPI processor idle
driver (Kirill A. Shutemov).
- Replace kernel.h with the necessary inclusions in the ACPI
processor driver (Andy Shevchenko).
- Use swap() instead of open coding it in the ACPI processor idle
driver (Guo Zhengkui).
- Fix the handling of defective LPAT in the ACPI xpower PMIC driver
and clean up some definitions of PMIC data structures (Hans de
Goede).
- Fix outdated comment in the ACPI DPTF driver (Sumeet Pawnikar).
- Add AEST to the list of known ACPI table signatures (Shuuichirou
Ishii).
- Make ACPI NUMA code take hotpluggable memblocks into account when
CONFIG_MEMORY_HOTPLUG is not set (Vitaly Kuznetsov).
- Use default_groups in kobj_type in the ACPI sysfs code (Greg
Kroah-Hartman).
- Rearrange _CPC structure documentation (Andy Shevchenko).
- Drop an always true check from the ACPI thermal driver (Adam
Borowski).
- Add new "not charging" quirk for Lenovo ThinkPads to the ACPI
battery driver (Thomas Weißschuh)"
* tag 'acpi-5.17-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm: (64 commits)
ACPI: PCC: Implement OperationRegion handler for the PCC Type 3 subtype
ACPI / x86: Skip AC and battery devices on x86 Android tablets with broken DSDTs
ACPI / x86: Introduce an acpi_quirk_skip_acpi_ac_and_battery() helper
ACPI: processor: thermal: avoid cpufreq_get_policy()
serdev: Do not instantiate serdevs on boards with known bogus DSDT entries
i2c: acpi: Do not instantiate I2C-clients on boards with known bogus DSDT entries
ACPI / x86: Add acpi_quirk_skip_[i2c_client|serdev]_enumeration() helpers
ACPI: scan: Create platform device for BCM4752 and LNV4752 ACPI nodes
PCI/ACPI: Fix acpi_pci_osc_control_set() kernel-doc comment
ACPI: battery: Add the ThinkPad "Not Charging" quirk
ACPI: sysfs: use default_groups in kobj_type
ACPICA: Update version to 20211217
ACPICA: iASL/NHLT table: "Specific Data" field support
ACPICA: iASL: Add suppport for AGDI table
ACPICA: iASL: Add TDEL table to both compiler/disassembler
ACPICA: Fixed a couple of warnings under MSVC
ACPICA: Change a return_ACPI_STATUS (AE_BAD_PARAMETER)
ACPICA: Hardware: Do not flush CPU cache when entering S4 and S5
ACPICA: Add support for PCC Opregion special context data
ACPICA: Fix wrong interpretation of PCC address
...
diff --git a/.mailmap b/.mailmap
index b344067..a835999 100644
--- a/.mailmap
+++ b/.mailmap
@@ -10,10 +10,12 @@
# Please keep this list dictionary sorted.
#
Aaron Durbin <adurbin@google.com>
+Abhinav Kumar <quic_abhinavk@quicinc.com> <abhinavk@codeaurora.org>
Adam Oldham <oldhamca@gmail.com>
Adam Radford <aradford@gmail.com>
Adriana Reus <adi.reus@gmail.com> <adriana.reus@intel.com>
Adrian Bunk <bunk@stusta.de>
+Akhil P Oommen <quic_akhilpo@quicinc.com> <akhilpo@codeaurora.org>
Alan Cox <alan@lxorguk.ukuu.org.uk>
Alan Cox <root@hraefn.swansea.linux.org.uk>
Aleksandar Markovic <aleksandar.markovic@mips.com> <aleksandar.markovic@imgtec.com>
@@ -42,6 +44,7 @@
Andrey Konovalov <andreyknvl@gmail.com> <andreyknvl@google.com>
Andrey Ryabinin <ryabinin.a.a@gmail.com> <a.ryabinin@samsung.com>
Andrey Ryabinin <ryabinin.a.a@gmail.com> <aryabinin@virtuozzo.com>
+Andrzej Hajda <andrzej.hajda@intel.com> <a.hajda@samsung.com>
Andy Adamson <andros@citi.umich.edu>
Antoine Tenart <atenart@kernel.org> <antoine.tenart@bootlin.com>
Antoine Tenart <atenart@kernel.org> <antoine.tenart@free-electrons.com>
@@ -172,6 +175,7 @@
Jens Axboe <axboe@suse.de>
Jens Osterkamp <Jens.Osterkamp@de.ibm.com>
Jernej Skrabec <jernej.skrabec@gmail.com> <jernej.skrabec@siol.net>
+Jessica Zhang <quic_jesszhan@quicinc.com> <jesszhan@codeaurora.org>
Jiri Slaby <jirislaby@kernel.org> <jirislaby@gmail.com>
Jiri Slaby <jirislaby@kernel.org> <jslaby@novell.com>
Jiri Slaby <jirislaby@kernel.org> <jslaby@suse.com>
@@ -191,6 +195,7 @@
Juha Yrjola <juha.yrjola@nokia.com>
Juha Yrjola <juha.yrjola@solidboot.com>
Julien Thierry <julien.thierry.kdev@gmail.com> <julien.thierry@arm.com>
+Kalyan Thota <quic_kalyant@quicinc.com> <kalyan_t@codeaurora.org>
Kay Sievers <kay.sievers@vrfy.org>
Kees Cook <keescook@chromium.org> <kees.cook@canonical.com>
Kees Cook <keescook@chromium.org> <keescook@google.com>
@@ -202,9 +207,11 @@
Konstantin Khlebnikov <koct9i@gmail.com> <khlebnikov@yandex-team.ru>
Konstantin Khlebnikov <koct9i@gmail.com> <k.khlebnikov@samsung.com>
Koushik <raghavendra.koushik@neterion.com>
+Krishna Manikandan <quic_mkrishn@quicinc.com> <mkrishn@codeaurora.org>
Krzysztof Kozlowski <krzk@kernel.org> <k.kozlowski.k@gmail.com>
Krzysztof Kozlowski <krzk@kernel.org> <k.kozlowski@samsung.com>
Kuninori Morimoto <kuninori.morimoto.gx@renesas.com>
+Kuogee Hsieh <quic_khsieh@quicinc.com> <khsieh@codeaurora.org>
Leonardo Bras <leobras.c@gmail.com> <leonardo@linux.ibm.com>
Leonid I Ananiev <leonid.i.ananiev@intel.com>
Leon Romanovsky <leon@kernel.org> <leon@leon.nu>
@@ -311,6 +318,7 @@
Quentin Monnet <quentin@isovalent.com> <quentin.monnet@netronome.com>
Quentin Perret <qperret@qperret.net> <quentin.perret@arm.com>
Rafael J. Wysocki <rjw@rjwysocki.net> <rjw@sisk.pl>
+Rajeev Nandan <quic_rajeevny@quicinc.com> <rajeevny@codeaurora.org>
Rajesh Shah <rajesh.shah@intel.com>
Ralf Baechle <ralf@linux-mips.org>
Ralf Wildenhues <Ralf.Wildenhues@gmx.de>
@@ -325,6 +333,7 @@
Sachin P Sant <ssant@in.ibm.com>
Sakari Ailus <sakari.ailus@linux.intel.com> <sakari.ailus@iki.fi>
Sam Ravnborg <sam@mars.ravnborg.org>
+Sankeerth Billakanti <quic_sbillaka@quicinc.com> <sbillaka@codeaurora.org>
Santosh Shilimkar <santosh.shilimkar@oracle.org>
Santosh Shilimkar <ssantosh@kernel.org>
Sarangdhar Joshi <spjoshi@codeaurora.org>
diff --git a/Documentation/ABI/stable/sysfs-devices-node b/Documentation/ABI/stable/sysfs-devices-node
index 484fc04..8db67aa 100644
--- a/Documentation/ABI/stable/sysfs-devices-node
+++ b/Documentation/ABI/stable/sysfs-devices-node
@@ -176,3 +176,9 @@
Description:
The cache write policy: 0 for write-back, 1 for write-through,
other or unknown.
+
+What: /sys/devices/system/node/nodeX/x86/sgx_total_bytes
+Date: November 2021
+Contact: Jarkko Sakkinen <jarkko@kernel.org>
+Description:
+ The total amount of SGX physical memory in bytes.
diff --git a/Documentation/admin-guide/cifs/usage.rst b/Documentation/admin-guide/cifs/usage.rst
index f170d88..3766bf8 100644
--- a/Documentation/admin-guide/cifs/usage.rst
+++ b/Documentation/admin-guide/cifs/usage.rst
@@ -734,10 +734,9 @@
using weaker password hashes is 0x37037 (lanman,
plaintext, ntlm, ntlmv2, signing allowed). Some
SecurityFlags require the corresponding menuconfig
- options to be enabled (lanman and plaintext require
- CONFIG_CIFS_WEAK_PW_HASH for example). Enabling
- plaintext authentication currently requires also
- enabling lanman authentication in the security flags
+ options to be enabled. Enabling plaintext
+ authentication currently requires also enabling
+ lanman authentication in the security flags
because the cifs module only supports sending
laintext passwords using the older lanman dialect
form of the session setup SMB. (e.g. for authentication
diff --git a/Documentation/admin-guide/devices.txt b/Documentation/admin-guide/devices.txt
index 922c23b..c07dc0e 100644
--- a/Documentation/admin-guide/devices.txt
+++ b/Documentation/admin-guide/devices.txt
@@ -2339,13 +2339,7 @@
disks (see major number 3) except that the limit on
partitions is 31.
- 162 char Raw block device interface
- 0 = /dev/rawctl Raw I/O control device
- 1 = /dev/raw/raw1 First raw I/O device
- 2 = /dev/raw/raw2 Second raw I/O device
- ...
- max minor number of raw device is set by kernel config
- MAX_RAW_DEVS or raw module parameter 'max_raw_devs'
+ 162 char Used for (now removed) raw block device interface
163 char
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 2fba824..f8a663f 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -2940,7 +2940,7 @@
both parameters are enabled, hugetlb_free_vmemmap takes
precedence over memory_hotplug.memmap_on_memory.
- memtest= [KNL,X86,ARM,PPC,RISCV] Enable memtest
+ memtest= [KNL,X86,ARM,M68K,PPC,RISCV] Enable memtest
Format: <integer>
default : 0 <disable>
Specifies the number of memtest passes to be
@@ -3551,6 +3551,13 @@
shutdown the other cpus. Instead use the REBOOT_VECTOR
irq.
+ nomodeset Disable kernel modesetting. DRM drivers will not perform
+ display-mode changes or accelerated rendering. Only the
+ system framebuffer will be available for use if this was
+ set-up by the firmware or boot loader.
+
+ Useful as fallback, or for testing and debugging.
+
nomodule Disable module load
nopat [X86] Disable PAT (page attribute table extension of
diff --git a/Documentation/admin-guide/media/platform-cardlist.rst b/Documentation/admin-guide/media/platform-cardlist.rst
index 261e777..ac73c41 100644
--- a/Documentation/admin-guide/media/platform-cardlist.rst
+++ b/Documentation/admin-guide/media/platform-cardlist.rst
@@ -60,6 +60,7 @@
sh_veu SuperH VEU mem2mem video processing
sh_vou SuperH VOU video output
stm32-dcmi STM32 Digital Camera Memory Interface (DCMI)
+stm32-dma2d STM32 Chrom-Art Accelerator Unit
sun4i-csi Allwinner A10 CMOS Sensor Interface Support
sun6i-csi Allwinner V3s Camera Sensor Interface
sun8i-di Allwinner Deinterlace
diff --git a/Documentation/admin-guide/perf/hisi-pcie-pmu.rst b/Documentation/admin-guide/perf/hisi-pcie-pmu.rst
new file mode 100644
index 0000000..294ebbd
--- /dev/null
+++ b/Documentation/admin-guide/perf/hisi-pcie-pmu.rst
@@ -0,0 +1,106 @@
+================================================
+HiSilicon PCIe Performance Monitoring Unit (PMU)
+================================================
+
+On Hip09, HiSilicon PCIe Performance Monitoring Unit (PMU) could monitor
+bandwidth, latency, bus utilization and buffer occupancy data of PCIe.
+
+Each PCIe Core has a PMU to monitor multi Root Ports of this PCIe Core and
+all Endpoints downstream these Root Ports.
+
+
+HiSilicon PCIe PMU driver
+=========================
+
+The PCIe PMU driver registers a perf PMU with the name of its sicl-id and PCIe
+Core id.::
+
+ /sys/bus/event_source/hisi_pcie<sicl>_<core>
+
+PMU driver provides description of available events and filter options in sysfs,
+see /sys/bus/event_source/devices/hisi_pcie<sicl>_<core>.
+
+The "format" directory describes all formats of the config (events) and config1
+(filter options) fields of the perf_event_attr structure. The "events" directory
+describes all documented events shown in perf list.
+
+The "identifier" sysfs file allows users to identify the version of the
+PMU hardware device.
+
+The "bus" sysfs file allows users to get the bus number of Root Ports
+monitored by PMU.
+
+Example usage of perf::
+
+ $# perf list
+ hisi_pcie0_0/rx_mwr_latency/ [kernel PMU event]
+ hisi_pcie0_0/rx_mwr_cnt/ [kernel PMU event]
+ ------------------------------------------
+
+ $# perf stat -e hisi_pcie0_0/rx_mwr_latency/
+ $# perf stat -e hisi_pcie0_0/rx_mwr_cnt/
+ $# perf stat -g -e hisi_pcie0_0/rx_mwr_latency/ -e hisi_pcie0_0/rx_mwr_cnt/
+
+The current driver does not support sampling. So "perf record" is unsupported.
+Also attach to a task is unsupported for PCIe PMU.
+
+Filter options
+--------------
+
+1. Target filter
+PMU could only monitor the performance of traffic downstream target Root Ports
+or downstream target Endpoint. PCIe PMU driver support "port" and "bdf"
+interfaces for users, and these two interfaces aren't supported at the same
+time.
+
+-port
+"port" filter can be used in all PCIe PMU events, target Root Port can be
+selected by configuring the 16-bits-bitmap "port". Multi ports can be selected
+for AP-layer-events, and only one port can be selected for TL/DL-layer-events.
+
+For example, if target Root Port is 0000:00:00.0 (x8 lanes), bit0 of bitmap
+should be set, port=0x1; if target Root Port is 0000:00:04.0 (x4 lanes),
+bit8 is set, port=0x100; if these two Root Ports are both monitored, port=0x101.
+
+Example usage of perf::
+
+ $# perf stat -e hisi_pcie0_0/rx_mwr_latency,port=0x1/ sleep 5
+
+-bdf
+
+"bdf" filter can only be used in bandwidth events, target Endpoint is selected
+by configuring BDF to "bdf". Counter only counts the bandwidth of message
+requested by target Endpoint.
+
+For example, "bdf=0x3900" means BDF of target Endpoint is 0000:39:00.0.
+
+Example usage of perf::
+
+ $# perf stat -e hisi_pcie0_0/rx_mrd_flux,bdf=0x3900/ sleep 5
+
+2. Trigger filter
+Event statistics start when the first time TLP length is greater/smaller
+than trigger condition. You can set the trigger condition by writing "trig_len",
+and set the trigger mode by writing "trig_mode". This filter can only be used
+in bandwidth events.
+
+For example, "trig_len=4" means trigger condition is 2^4 DW, "trig_mode=0"
+means statistics start when TLP length > trigger condition, "trig_mode=1"
+means start when TLP length < condition.
+
+Example usage of perf::
+
+ $# perf stat -e hisi_pcie0_0/rx_mrd_flux,trig_len=0x4,trig_mode=1/ sleep 5
+
+3. Threshold filter
+Counter counts when TLP length within the specified range. You can set the
+threshold by writing "thr_len", and set the threshold mode by writing
+"thr_mode". This filter can only be used in bandwidth events.
+
+For example, "thr_len=4" means threshold is 2^4 DW, "thr_mode=0" means
+counter counts when TLP length >= threshold, and "thr_mode=1" means counts
+when TLP length < threshold.
+
+Example usage of perf::
+
+ $# perf stat -e hisi_pcie0_0/rx_mrd_flux,thr_len=0x4,thr_mode=1/ sleep 5
diff --git a/Documentation/admin-guide/sysctl/kernel.rst b/Documentation/admin-guide/sysctl/kernel.rst
index 0e486f4..d359bcf 100644
--- a/Documentation/admin-guide/sysctl/kernel.rst
+++ b/Documentation/admin-guide/sysctl/kernel.rst
@@ -905,6 +905,17 @@
The default value is 8.
+perf_user_access (arm64 only)
+=================================
+
+Controls user space access for reading perf event counters. When set to 1,
+user space can read performance monitor counter registers directly.
+
+The default value is 0 (access disabled).
+
+See Documentation/arm64/perf.rst for more information.
+
+
pid_max
=======
diff --git a/Documentation/arm64/cpu-feature-registers.rst b/Documentation/arm64/cpu-feature-registers.rst
index 9f9b8fd..749ae97 100644
--- a/Documentation/arm64/cpu-feature-registers.rst
+++ b/Documentation/arm64/cpu-feature-registers.rst
@@ -275,6 +275,23 @@
| SVEVer | [3-0] | y |
+------------------------------+---------+---------+
+ 8) ID_AA64MMFR1_EL1 - Memory model feature register 1
+
+ +------------------------------+---------+---------+
+ | Name | bits | visible |
+ +------------------------------+---------+---------+
+ | AFP | [47-44] | y |
+ +------------------------------+---------+---------+
+
+ 9) ID_AA64ISAR2_EL1 - Instruction set attribute register 2
+
+ +------------------------------+---------+---------+
+ | Name | bits | visible |
+ +------------------------------+---------+---------+
+ | RPRES | [7-4] | y |
+ +------------------------------+---------+---------+
+
+
Appendix I: Example
-------------------
diff --git a/Documentation/arm64/elf_hwcaps.rst b/Documentation/arm64/elf_hwcaps.rst
index af106af..b72ff17 100644
--- a/Documentation/arm64/elf_hwcaps.rst
+++ b/Documentation/arm64/elf_hwcaps.rst
@@ -251,6 +251,14 @@
Functionality implied by ID_AA64MMFR0_EL1.ECV == 0b0001.
+HWCAP2_AFP
+
+ Functionality implied by ID_AA64MFR1_EL1.AFP == 0b0001.
+
+HWCAP2_RPRES
+
+ Functionality implied by ID_AA64ISAR2_EL1.RPRES == 0b0001.
+
4. Unused AT_HWCAP bits
-----------------------
diff --git a/Documentation/arm64/perf.rst b/Documentation/arm64/perf.rst
index b567f17..1f87b57 100644
--- a/Documentation/arm64/perf.rst
+++ b/Documentation/arm64/perf.rst
@@ -2,7 +2,10 @@
.. _perf_index:
-=====================
+====
+Perf
+====
+
Perf Event Attributes
=====================
@@ -88,3 +91,76 @@
window at the guest entry/exit where host events are not captured.
On VHE systems there are no blackout windows.
+
+Perf Userspace PMU Hardware Counter Access
+==========================================
+
+Overview
+--------
+The perf userspace tool relies on the PMU to monitor events. It offers an
+abstraction layer over the hardware counters since the underlying
+implementation is cpu-dependent.
+Arm64 allows userspace tools to have access to the registers storing the
+hardware counters' values directly.
+
+This targets specifically self-monitoring tasks in order to reduce the overhead
+by directly accessing the registers without having to go through the kernel.
+
+How-to
+------
+The focus is set on the armv8 PMUv3 which makes sure that the access to the pmu
+registers is enabled and that the userspace has access to the relevant
+information in order to use them.
+
+In order to have access to the hardware counters, the global sysctl
+kernel/perf_user_access must first be enabled:
+
+.. code-block:: sh
+
+ echo 1 > /proc/sys/kernel/perf_user_access
+
+It is necessary to open the event using the perf tool interface with config1:1
+attr bit set: the sys_perf_event_open syscall returns a fd which can
+subsequently be used with the mmap syscall in order to retrieve a page of memory
+containing information about the event. The PMU driver uses this page to expose
+to the user the hardware counter's index and other necessary data. Using this
+index enables the user to access the PMU registers using the `mrs` instruction.
+Access to the PMU registers is only valid while the sequence lock is unchanged.
+In particular, the PMSELR_EL0 register is zeroed each time the sequence lock is
+changed.
+
+The userspace access is supported in libperf using the perf_evsel__mmap()
+and perf_evsel__read() functions. See `tools/lib/perf/tests/test-evsel.c`_ for
+an example.
+
+About heterogeneous systems
+---------------------------
+On heterogeneous systems such as big.LITTLE, userspace PMU counter access can
+only be enabled when the tasks are pinned to a homogeneous subset of cores and
+the corresponding PMU instance is opened by specifying the 'type' attribute.
+The use of generic event types is not supported in this case.
+
+Have a look at `tools/perf/arch/arm64/tests/user-events.c`_ for an example. It
+can be run using the perf tool to check that the access to the registers works
+correctly from userspace:
+
+.. code-block:: sh
+
+ perf test -v user
+
+About chained events and counter sizes
+--------------------------------------
+The user can request either a 32-bit (config1:0 == 0) or 64-bit (config1:0 == 1)
+counter along with userspace access. The sys_perf_event_open syscall will fail
+if a 64-bit counter is requested and the hardware doesn't support 64-bit
+counters. Chained events are not supported in conjunction with userspace counter
+access. If a 32-bit counter is requested on hardware with 64-bit counters, then
+userspace must treat the upper 32-bits read from the counter as UNKNOWN. The
+'pmc_width' field in the user page will indicate the valid width of the counter
+and should be used to mask the upper bits as needed.
+
+.. Links
+.. _tools/perf/arch/arm64/tests/user-events.c:
+ https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/perf/arch/arm64/tests/user-events.c
+.. _tools/lib/perf/tests/test-evsel.c:
+ https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/lib/perf/tests/test-evsel.c
diff --git a/Documentation/arm64/sve.rst b/Documentation/arm64/sve.rst
index 0313715..9d9a4de 100644
--- a/Documentation/arm64/sve.rst
+++ b/Documentation/arm64/sve.rst
@@ -255,7 +255,7 @@
vector length change (which would only normally be the case between a
fork() or vfork() and the corresponding execve() in typical use).
- To extract the vector length from the result, and it with
+ To extract the vector length from the result, bitwise and it with
PR_SVE_VL_LEN_MASK.
Return value: a nonnegative value on success, or a negative value on error:
diff --git a/Documentation/arm64/tagged-address-abi.rst b/Documentation/arm64/tagged-address-abi.rst
index 0c9120e..540a1d4 100644
--- a/Documentation/arm64/tagged-address-abi.rst
+++ b/Documentation/arm64/tagged-address-abi.rst
@@ -49,7 +49,7 @@
- ``brk()``, ``mmap()`` and the ``new_address`` argument to
``mremap()`` as these have the potential to alias with existing
- user addresses.
+ user addresses.
NOTE: This behaviour changed in v5.6 and so some earlier kernels may
incorrectly accept valid tagged pointers for the ``brk()``,
diff --git a/Documentation/bpf/btf.rst b/Documentation/bpf/btf.rst
index 9ad4218..1ebf4c5 100644
--- a/Documentation/bpf/btf.rst
+++ b/Documentation/bpf/btf.rst
@@ -3,7 +3,7 @@
=====================
1. Introduction
-***************
+===============
BTF (BPF Type Format) is the metadata format which encodes the debug info
related to BPF program/map. The name BTF was used initially to describe data
@@ -30,7 +30,7 @@
.. _BTF_Type_String:
2. BTF Type and String Encoding
-*******************************
+===============================
The file ``include/uapi/linux/btf.h`` provides high-level definition of how
types/strings are encoded.
@@ -57,13 +57,13 @@
generated.
2.1 String Encoding
-===================
+-------------------
The first string in the string section must be a null string. The rest of
string table is a concatenation of other null-terminated strings.
2.2 Type Encoding
-=================
+-----------------
The type id ``0`` is reserved for ``void`` type. The type section is parsed
sequentially and type id is assigned to each recognized type starting from id
@@ -86,6 +86,7 @@
#define BTF_KIND_DATASEC 15 /* Section */
#define BTF_KIND_FLOAT 16 /* Floating point */
#define BTF_KIND_DECL_TAG 17 /* Decl Tag */
+ #define BTF_KIND_TYPE_TAG 18 /* Type Tag */
Note that the type section encodes debug info, not just pure types.
``BTF_KIND_FUNC`` is not a type, and it represents a defined subprogram.
@@ -107,7 +108,7 @@
* "size" tells the size of the type it is describing.
*
* "type" is used by PTR, TYPEDEF, VOLATILE, CONST, RESTRICT,
- * FUNC, FUNC_PROTO and DECL_TAG.
+ * FUNC, FUNC_PROTO, DECL_TAG and TYPE_TAG.
* "type" is a type_id referring to another type.
*/
union {
@@ -492,8 +493,18 @@
a ``func`` argument, and ``btf_decl_tag.component_idx`` should be a
valid index (starting from 0) pointing to a member or an argument.
+2.2.17 BTF_KIND_TYPE_TAG
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+``struct btf_type`` encoding requirement:
+ * ``name_off``: offset to a non-empty string
+ * ``info.kind_flag``: 0
+ * ``info.kind``: BTF_KIND_TYPE_TAG
+ * ``info.vlen``: 0
+ * ``type``: the type with ``btf_type_tag`` attribute
+
3. BTF Kernel API
-*****************
+=================
The following bpf syscall command involves BTF:
* BPF_BTF_LOAD: load a blob of BTF data into kernel
@@ -536,14 +547,14 @@
3.1 BPF_BTF_LOAD
-================
+----------------
Load a blob of BTF data into kernel. A blob of data, described in
:ref:`BTF_Type_String`, can be directly loaded into the kernel. A ``btf_fd``
is returned to a userspace.
3.2 BPF_MAP_CREATE
-==================
+------------------
A map can be created with ``btf_fd`` and specified key/value type id.::
@@ -570,7 +581,7 @@
.. _BPF_Prog_Load:
3.3 BPF_PROG_LOAD
-=================
+-----------------
During prog_load, func_info and line_info can be passed to kernel with proper
values for the following attributes:
@@ -620,7 +631,7 @@
#define BPF_LINE_INFO_LINE_COL(line_col) ((line_col) & 0x3ff)
3.4 BPF_{PROG,MAP}_GET_NEXT_ID
-==============================
+------------------------------
In kernel, every loaded program, map or btf has a unique id. The id won't
change during the lifetime of a program, map, or btf.
@@ -630,13 +641,13 @@
inspection tool can inspect all programs and maps.
3.5 BPF_{PROG,MAP}_GET_FD_BY_ID
-===============================
+-------------------------------
An introspection tool cannot use id to get details about program or maps.
A file descriptor needs to be obtained first for reference-counting purpose.
3.6 BPF_OBJ_GET_INFO_BY_FD
-==========================
+--------------------------
Once a program/map fd is acquired, an introspection tool can get the detailed
information from kernel about this fd, some of which are BTF-related. For
@@ -645,7 +656,7 @@
bpf byte codes, and jited_line_info.
3.7 BPF_BTF_GET_FD_BY_ID
-========================
+------------------------
With ``btf_id`` obtained in ``bpf_map_info`` and ``bpf_prog_info``, bpf
syscall command BPF_BTF_GET_FD_BY_ID can retrieve a btf fd. Then, with
@@ -657,10 +668,10 @@
func signatures and line info, along with byte/jit codes.
4. ELF File Format Interface
-****************************
+============================
4.1 .BTF section
-================
+----------------
The .BTF section contains type and string data. The format of this section is
same as the one describe in :ref:`BTF_Type_String`.
@@ -668,7 +679,7 @@
.. _BTF_Ext_Section:
4.2 .BTF.ext section
-====================
+--------------------
The .BTF.ext section encodes func_info and line_info which needs loader
manipulation before loading into the kernel.
@@ -732,7 +743,7 @@
beginning of section (``btf_ext_info_sec->sec_name_off``).
4.2 .BTF_ids section
-====================
+--------------------
The .BTF_ids section encodes BTF ID values that are used within the kernel.
@@ -793,10 +804,10 @@
resolved during the linking phase of kernel build by ``resolve_btfids`` tool.
5. Using BTF
-************
+============
5.1 bpftool map pretty print
-============================
+----------------------------
With BTF, the map key/value can be printed based on fields rather than simply
raw bytes. This is especially valuable for large structure or if your data
@@ -838,7 +849,7 @@
]
5.2 bpftool prog dump
-=====================
+---------------------
The following is an example showing how func_info and line_info can help prog
dump with better kernel symbol names, function prototypes and line
@@ -872,7 +883,7 @@
[...]
5.3 Verifier Log
-================
+----------------
The following is an example of how line_info can help debugging verification
failure.::
@@ -898,7 +909,7 @@
R2 offset is outside of the packet
6. BTF Generation
-*****************
+=================
You need latest pahole
@@ -1005,6 +1016,6 @@
.long 8206 # Line 8 Col 14
7. Testing
-**********
+==========
Kernel bpf selftest `test_btf.c` provides extensive set of BTF-related tests.
diff --git a/Documentation/bpf/classic_vs_extended.rst b/Documentation/bpf/classic_vs_extended.rst
new file mode 100644
index 0000000..2f81a81
--- /dev/null
+++ b/Documentation/bpf/classic_vs_extended.rst
@@ -0,0 +1,376 @@
+
+===================
+Classic BPF vs eBPF
+===================
+
+eBPF is designed to be JITed with one to one mapping, which can also open up
+the possibility for GCC/LLVM compilers to generate optimized eBPF code through
+an eBPF backend that performs almost as fast as natively compiled code.
+
+Some core changes of the eBPF format from classic BPF:
+
+- Number of registers increase from 2 to 10:
+
+ The old format had two registers A and X, and a hidden frame pointer. The
+ new layout extends this to be 10 internal registers and a read-only frame
+ pointer. Since 64-bit CPUs are passing arguments to functions via registers
+ the number of args from eBPF program to in-kernel function is restricted
+ to 5 and one register is used to accept return value from an in-kernel
+ function. Natively, x86_64 passes first 6 arguments in registers, aarch64/
+ sparcv9/mips64 have 7 - 8 registers for arguments; x86_64 has 6 callee saved
+ registers, and aarch64/sparcv9/mips64 have 11 or more callee saved registers.
+
+ Thus, all eBPF registers map one to one to HW registers on x86_64, aarch64,
+ etc, and eBPF calling convention maps directly to ABIs used by the kernel on
+ 64-bit architectures.
+
+ On 32-bit architectures JIT may map programs that use only 32-bit arithmetic
+ and may let more complex programs to be interpreted.
+
+ R0 - R5 are scratch registers and eBPF program needs spill/fill them if
+ necessary across calls. Note that there is only one eBPF program (== one
+ eBPF main routine) and it cannot call other eBPF functions, it can only
+ call predefined in-kernel functions, though.
+
+- Register width increases from 32-bit to 64-bit:
+
+ Still, the semantics of the original 32-bit ALU operations are preserved
+ via 32-bit subregisters. All eBPF registers are 64-bit with 32-bit lower
+ subregisters that zero-extend into 64-bit if they are being written to.
+ That behavior maps directly to x86_64 and arm64 subregister definition, but
+ makes other JITs more difficult.
+
+ 32-bit architectures run 64-bit eBPF programs via interpreter.
+ Their JITs may convert BPF programs that only use 32-bit subregisters into
+ native instruction set and let the rest being interpreted.
+
+ Operation is 64-bit, because on 64-bit architectures, pointers are also
+ 64-bit wide, and we want to pass 64-bit values in/out of kernel functions,
+ so 32-bit eBPF registers would otherwise require to define register-pair
+ ABI, thus, there won't be able to use a direct eBPF register to HW register
+ mapping and JIT would need to do combine/split/move operations for every
+ register in and out of the function, which is complex, bug prone and slow.
+ Another reason is the use of atomic 64-bit counters.
+
+- Conditional jt/jf targets replaced with jt/fall-through:
+
+ While the original design has constructs such as ``if (cond) jump_true;
+ else jump_false;``, they are being replaced into alternative constructs like
+ ``if (cond) jump_true; /* else fall-through */``.
+
+- Introduces bpf_call insn and register passing convention for zero overhead
+ calls from/to other kernel functions:
+
+ Before an in-kernel function call, the eBPF program needs to
+ place function arguments into R1 to R5 registers to satisfy calling
+ convention, then the interpreter will take them from registers and pass
+ to in-kernel function. If R1 - R5 registers are mapped to CPU registers
+ that are used for argument passing on given architecture, the JIT compiler
+ doesn't need to emit extra moves. Function arguments will be in the correct
+ registers and BPF_CALL instruction will be JITed as single 'call' HW
+ instruction. This calling convention was picked to cover common call
+ situations without performance penalty.
+
+ After an in-kernel function call, R1 - R5 are reset to unreadable and R0 has
+ a return value of the function. Since R6 - R9 are callee saved, their state
+ is preserved across the call.
+
+ For example, consider three C functions::
+
+ u64 f1() { return (*_f2)(1); }
+ u64 f2(u64 a) { return f3(a + 1, a); }
+ u64 f3(u64 a, u64 b) { return a - b; }
+
+ GCC can compile f1, f3 into x86_64::
+
+ f1:
+ movl $1, %edi
+ movq _f2(%rip), %rax
+ jmp *%rax
+ f3:
+ movq %rdi, %rax
+ subq %rsi, %rax
+ ret
+
+ Function f2 in eBPF may look like::
+
+ f2:
+ bpf_mov R2, R1
+ bpf_add R1, 1
+ bpf_call f3
+ bpf_exit
+
+ If f2 is JITed and the pointer stored to ``_f2``. The calls f1 -> f2 -> f3 and
+ returns will be seamless. Without JIT, __bpf_prog_run() interpreter needs to
+ be used to call into f2.
+
+ For practical reasons all eBPF programs have only one argument 'ctx' which is
+ already placed into R1 (e.g. on __bpf_prog_run() startup) and the programs
+ can call kernel functions with up to 5 arguments. Calls with 6 or more arguments
+ are currently not supported, but these restrictions can be lifted if necessary
+ in the future.
+
+ On 64-bit architectures all register map to HW registers one to one. For
+ example, x86_64 JIT compiler can map them as ...
+
+ ::
+
+ R0 - rax
+ R1 - rdi
+ R2 - rsi
+ R3 - rdx
+ R4 - rcx
+ R5 - r8
+ R6 - rbx
+ R7 - r13
+ R8 - r14
+ R9 - r15
+ R10 - rbp
+
+ ... since x86_64 ABI mandates rdi, rsi, rdx, rcx, r8, r9 for argument passing
+ and rbx, r12 - r15 are callee saved.
+
+ Then the following eBPF pseudo-program::
+
+ bpf_mov R6, R1 /* save ctx */
+ bpf_mov R2, 2
+ bpf_mov R3, 3
+ bpf_mov R4, 4
+ bpf_mov R5, 5
+ bpf_call foo
+ bpf_mov R7, R0 /* save foo() return value */
+ bpf_mov R1, R6 /* restore ctx for next call */
+ bpf_mov R2, 6
+ bpf_mov R3, 7
+ bpf_mov R4, 8
+ bpf_mov R5, 9
+ bpf_call bar
+ bpf_add R0, R7
+ bpf_exit
+
+ After JIT to x86_64 may look like::
+
+ push %rbp
+ mov %rsp,%rbp
+ sub $0x228,%rsp
+ mov %rbx,-0x228(%rbp)
+ mov %r13,-0x220(%rbp)
+ mov %rdi,%rbx
+ mov $0x2,%esi
+ mov $0x3,%edx
+ mov $0x4,%ecx
+ mov $0x5,%r8d
+ callq foo
+ mov %rax,%r13
+ mov %rbx,%rdi
+ mov $0x6,%esi
+ mov $0x7,%edx
+ mov $0x8,%ecx
+ mov $0x9,%r8d
+ callq bar
+ add %r13,%rax
+ mov -0x228(%rbp),%rbx
+ mov -0x220(%rbp),%r13
+ leaveq
+ retq
+
+ Which is in this example equivalent in C to::
+
+ u64 bpf_filter(u64 ctx)
+ {
+ return foo(ctx, 2, 3, 4, 5) + bar(ctx, 6, 7, 8, 9);
+ }
+
+ In-kernel functions foo() and bar() with prototype: u64 (*)(u64 arg1, u64
+ arg2, u64 arg3, u64 arg4, u64 arg5); will receive arguments in proper
+ registers and place their return value into ``%rax`` which is R0 in eBPF.
+ Prologue and epilogue are emitted by JIT and are implicit in the
+ interpreter. R0-R5 are scratch registers, so eBPF program needs to preserve
+ them across the calls as defined by calling convention.
+
+ For example the following program is invalid::
+
+ bpf_mov R1, 1
+ bpf_call foo
+ bpf_mov R0, R1
+ bpf_exit
+
+ After the call the registers R1-R5 contain junk values and cannot be read.
+ An in-kernel verifier.rst is used to validate eBPF programs.
+
+Also in the new design, eBPF is limited to 4096 insns, which means that any
+program will terminate quickly and will only call a fixed number of kernel
+functions. Original BPF and eBPF are two operand instructions,
+which helps to do one-to-one mapping between eBPF insn and x86 insn during JIT.
+
+The input context pointer for invoking the interpreter function is generic,
+its content is defined by a specific use case. For seccomp register R1 points
+to seccomp_data, for converted BPF filters R1 points to a skb.
+
+A program, that is translated internally consists of the following elements::
+
+ op:16, jt:8, jf:8, k:32 ==> op:8, dst_reg:4, src_reg:4, off:16, imm:32
+
+So far 87 eBPF instructions were implemented. 8-bit 'op' opcode field
+has room for new instructions. Some of them may use 16/24/32 byte encoding. New
+instructions must be multiple of 8 bytes to preserve backward compatibility.
+
+eBPF is a general purpose RISC instruction set. Not every register and
+every instruction are used during translation from original BPF to eBPF.
+For example, socket filters are not using ``exclusive add`` instruction, but
+tracing filters may do to maintain counters of events, for example. Register R9
+is not used by socket filters either, but more complex filters may be running
+out of registers and would have to resort to spill/fill to stack.
+
+eBPF can be used as a generic assembler for last step performance
+optimizations, socket filters and seccomp are using it as assembler. Tracing
+filters may use it as assembler to generate code from kernel. In kernel usage
+may not be bounded by security considerations, since generated eBPF code
+may be optimizing internal code path and not being exposed to the user space.
+Safety of eBPF can come from the verifier.rst. In such use cases as
+described, it may be used as safe instruction set.
+
+Just like the original BPF, eBPF runs within a controlled environment,
+is deterministic and the kernel can easily prove that. The safety of the program
+can be determined in two steps: first step does depth-first-search to disallow
+loops and other CFG validation; second step starts from the first insn and
+descends all possible paths. It simulates execution of every insn and observes
+the state change of registers and stack.
+
+opcode encoding
+===============
+
+eBPF is reusing most of the opcode encoding from classic to simplify conversion
+of classic BPF to eBPF.
+
+For arithmetic and jump instructions the 8-bit 'code' field is divided into three
+parts::
+
+ +----------------+--------+--------------------+
+ | 4 bits | 1 bit | 3 bits |
+ | operation code | source | instruction class |
+ +----------------+--------+--------------------+
+ (MSB) (LSB)
+
+Three LSB bits store instruction class which is one of:
+
+ =================== ===============
+ Classic BPF classes eBPF classes
+ =================== ===============
+ BPF_LD 0x00 BPF_LD 0x00
+ BPF_LDX 0x01 BPF_LDX 0x01
+ BPF_ST 0x02 BPF_ST 0x02
+ BPF_STX 0x03 BPF_STX 0x03
+ BPF_ALU 0x04 BPF_ALU 0x04
+ BPF_JMP 0x05 BPF_JMP 0x05
+ BPF_RET 0x06 BPF_JMP32 0x06
+ BPF_MISC 0x07 BPF_ALU64 0x07
+ =================== ===============
+
+The 4th bit encodes the source operand ...
+
+ ::
+
+ BPF_K 0x00
+ BPF_X 0x08
+
+ * in classic BPF, this means::
+
+ BPF_SRC(code) == BPF_X - use register X as source operand
+ BPF_SRC(code) == BPF_K - use 32-bit immediate as source operand
+
+ * in eBPF, this means::
+
+ BPF_SRC(code) == BPF_X - use 'src_reg' register as source operand
+ BPF_SRC(code) == BPF_K - use 32-bit immediate as source operand
+
+... and four MSB bits store operation code.
+
+If BPF_CLASS(code) == BPF_ALU or BPF_ALU64 [ in eBPF ], BPF_OP(code) is one of::
+
+ BPF_ADD 0x00
+ BPF_SUB 0x10
+ BPF_MUL 0x20
+ BPF_DIV 0x30
+ BPF_OR 0x40
+ BPF_AND 0x50
+ BPF_LSH 0x60
+ BPF_RSH 0x70
+ BPF_NEG 0x80
+ BPF_MOD 0x90
+ BPF_XOR 0xa0
+ BPF_MOV 0xb0 /* eBPF only: mov reg to reg */
+ BPF_ARSH 0xc0 /* eBPF only: sign extending shift right */
+ BPF_END 0xd0 /* eBPF only: endianness conversion */
+
+If BPF_CLASS(code) == BPF_JMP or BPF_JMP32 [ in eBPF ], BPF_OP(code) is one of::
+
+ BPF_JA 0x00 /* BPF_JMP only */
+ BPF_JEQ 0x10
+ BPF_JGT 0x20
+ BPF_JGE 0x30
+ BPF_JSET 0x40
+ BPF_JNE 0x50 /* eBPF only: jump != */
+ BPF_JSGT 0x60 /* eBPF only: signed '>' */
+ BPF_JSGE 0x70 /* eBPF only: signed '>=' */
+ BPF_CALL 0x80 /* eBPF BPF_JMP only: function call */
+ BPF_EXIT 0x90 /* eBPF BPF_JMP only: function return */
+ BPF_JLT 0xa0 /* eBPF only: unsigned '<' */
+ BPF_JLE 0xb0 /* eBPF only: unsigned '<=' */
+ BPF_JSLT 0xc0 /* eBPF only: signed '<' */
+ BPF_JSLE 0xd0 /* eBPF only: signed '<=' */
+
+So BPF_ADD | BPF_X | BPF_ALU means 32-bit addition in both classic BPF
+and eBPF. There are only two registers in classic BPF, so it means A += X.
+In eBPF it means dst_reg = (u32) dst_reg + (u32) src_reg; similarly,
+BPF_XOR | BPF_K | BPF_ALU means A ^= imm32 in classic BPF and analogous
+src_reg = (u32) src_reg ^ (u32) imm32 in eBPF.
+
+Classic BPF is using BPF_MISC class to represent A = X and X = A moves.
+eBPF is using BPF_MOV | BPF_X | BPF_ALU code instead. Since there are no
+BPF_MISC operations in eBPF, the class 7 is used as BPF_ALU64 to mean
+exactly the same operations as BPF_ALU, but with 64-bit wide operands
+instead. So BPF_ADD | BPF_X | BPF_ALU64 means 64-bit addition, i.e.:
+dst_reg = dst_reg + src_reg
+
+Classic BPF wastes the whole BPF_RET class to represent a single ``ret``
+operation. Classic BPF_RET | BPF_K means copy imm32 into return register
+and perform function exit. eBPF is modeled to match CPU, so BPF_JMP | BPF_EXIT
+in eBPF means function exit only. The eBPF program needs to store return
+value into register R0 before doing a BPF_EXIT. Class 6 in eBPF is used as
+BPF_JMP32 to mean exactly the same operations as BPF_JMP, but with 32-bit wide
+operands for the comparisons instead.
+
+For load and store instructions the 8-bit 'code' field is divided as::
+
+ +--------+--------+-------------------+
+ | 3 bits | 2 bits | 3 bits |
+ | mode | size | instruction class |
+ +--------+--------+-------------------+
+ (MSB) (LSB)
+
+Size modifier is one of ...
+
+::
+
+ BPF_W 0x00 /* word */
+ BPF_H 0x08 /* half word */
+ BPF_B 0x10 /* byte */
+ BPF_DW 0x18 /* eBPF only, double word */
+
+... which encodes size of load/store operation::
+
+ B - 1 byte
+ H - 2 byte
+ W - 4 byte
+ DW - 8 byte (eBPF only)
+
+Mode modifier is one of::
+
+ BPF_IMM 0x00 /* used for 32-bit mov in classic BPF and 64-bit in eBPF */
+ BPF_ABS 0x20
+ BPF_IND 0x40
+ BPF_MEM 0x60
+ BPF_LEN 0x80 /* classic BPF only, reserved in eBPF */
+ BPF_MSH 0xa0 /* classic BPF only, reserved in eBPF */
+ BPF_ATOMIC 0xc0 /* eBPF only, atomic operations */
diff --git a/Documentation/bpf/faq.rst b/Documentation/bpf/faq.rst
new file mode 100644
index 0000000..a622602
--- /dev/null
+++ b/Documentation/bpf/faq.rst
@@ -0,0 +1,11 @@
+================================
+Frequently asked questions (FAQ)
+================================
+
+Two sets of Questions and Answers (Q&A) are maintained.
+
+.. toctree::
+ :maxdepth: 1
+
+ bpf_design_QA
+ bpf_devel_QA
diff --git a/Documentation/bpf/helpers.rst b/Documentation/bpf/helpers.rst
new file mode 100644
index 0000000..c4ee0cc
--- /dev/null
+++ b/Documentation/bpf/helpers.rst
@@ -0,0 +1,7 @@
+Helper functions
+================
+
+* `bpf-helpers(7)`_ maintains a list of helpers available to eBPF programs.
+
+.. Links
+.. _bpf-helpers(7): https://man7.org/linux/man-pages/man7/bpf-helpers.7.html
\ No newline at end of file
diff --git a/Documentation/bpf/index.rst b/Documentation/bpf/index.rst
index 610450f..ef5c996 100644
--- a/Documentation/bpf/index.rst
+++ b/Documentation/bpf/index.rst
@@ -5,104 +5,33 @@
This directory contains documentation for the BPF (Berkeley Packet
Filter) facility, with a focus on the extended BPF version (eBPF).
-This kernel side documentation is still work in progress. The main
-textual documentation is (for historical reasons) described in
-:ref:`networking-filter`, which describe both classical and extended
-BPF instruction-set.
+This kernel side documentation is still work in progress.
The Cilium project also maintains a `BPF and XDP Reference Guide`_
that goes into great technical depth about the BPF Architecture.
-libbpf
-======
-
-Documentation/bpf/libbpf/index.rst is a userspace library for loading and interacting with bpf programs.
-
-BPF Type Format (BTF)
-=====================
-
.. toctree::
:maxdepth: 1
+ instruction-set
+ verifier
+ libbpf/index
btf
-
-
-Frequently asked questions (FAQ)
-================================
-
-Two sets of Questions and Answers (Q&A) are maintained.
-
-.. toctree::
- :maxdepth: 1
-
- bpf_design_QA
- bpf_devel_QA
-
-Syscall API
-===========
-
-The primary info for the bpf syscall is available in the `man-pages`_
-for `bpf(2)`_. For more information about the userspace API, see
-Documentation/userspace-api/ebpf/index.rst.
-
-Helper functions
-================
-
-* `bpf-helpers(7)`_ maintains a list of helpers available to eBPF programs.
-
-
-Program types
-=============
-
-.. toctree::
- :maxdepth: 1
-
- prog_cgroup_sockopt
- prog_cgroup_sysctl
- prog_flow_dissector
- bpf_lsm
- prog_sk_lookup
-
-
-Map types
-=========
-
-.. toctree::
- :maxdepth: 1
-
- map_cgroup_storage
-
-
-Testing and debugging BPF
-=========================
-
-.. toctree::
- :maxdepth: 1
-
- drgn
- s390
-
-
-Licensing
-=========
-
-.. toctree::
- :maxdepth: 1
-
+ faq
+ syscall_api
+ helpers
+ programs
+ maps
+ classic_vs_extended.rst
bpf_licensing
+ test_debug
+ other
+.. only:: subproject and html
-Other
-=====
+ Indices
+ =======
-.. toctree::
- :maxdepth: 1
-
- ringbuf
- llvm_reloc
+ * :ref:`genindex`
.. Links:
-.. _networking-filter: ../networking/filter.rst
-.. _man-pages: https://www.kernel.org/doc/man-pages/
-.. _bpf(2): https://man7.org/linux/man-pages/man2/bpf.2.html
-.. _bpf-helpers(7): https://man7.org/linux/man-pages/man7/bpf-helpers.7.html
.. _BPF and XDP Reference Guide: https://docs.cilium.io/en/latest/bpf/
diff --git a/Documentation/bpf/instruction-set.rst b/Documentation/bpf/instruction-set.rst
new file mode 100644
index 0000000..3704836
--- /dev/null
+++ b/Documentation/bpf/instruction-set.rst
@@ -0,0 +1,279 @@
+
+====================
+eBPF Instruction Set
+====================
+
+Registers and calling convention
+================================
+
+eBPF has 10 general purpose registers and a read-only frame pointer register,
+all of which are 64-bits wide.
+
+The eBPF calling convention is defined as:
+
+ * R0: return value from function calls, and exit value for eBPF programs
+ * R1 - R5: arguments for function calls
+ * R6 - R9: callee saved registers that function calls will preserve
+ * R10: read-only frame pointer to access stack
+
+R0 - R5 are scratch registers and eBPF programs needs to spill/fill them if
+necessary across calls.
+
+Instruction encoding
+====================
+
+eBPF uses 64-bit instructions with the following encoding:
+
+ ============= ======= =============== ==================== ============
+ 32 bits (MSB) 16 bits 4 bits 4 bits 8 bits (LSB)
+ ============= ======= =============== ==================== ============
+ immediate offset source register destination register opcode
+ ============= ======= =============== ==================== ============
+
+Note that most instructions do not use all of the fields.
+Unused fields shall be cleared to zero.
+
+Instruction classes
+-------------------
+
+The three LSB bits of the 'opcode' field store the instruction class:
+
+ ========= ===== ===============================
+ class value description
+ ========= ===== ===============================
+ BPF_LD 0x00 non-standard load operations
+ BPF_LDX 0x01 load into register operations
+ BPF_ST 0x02 store from immediate operations
+ BPF_STX 0x03 store from register operations
+ BPF_ALU 0x04 32-bit arithmetic operations
+ BPF_JMP 0x05 64-bit jump operations
+ BPF_JMP32 0x06 32-bit jump operations
+ BPF_ALU64 0x07 64-bit arithmetic operations
+ ========= ===== ===============================
+
+Arithmetic and jump instructions
+================================
+
+For arithmetic and jump instructions (BPF_ALU, BPF_ALU64, BPF_JMP and
+BPF_JMP32), the 8-bit 'opcode' field is divided into three parts:
+
+ ============== ====== =================
+ 4 bits (MSB) 1 bit 3 bits (LSB)
+ ============== ====== =================
+ operation code source instruction class
+ ============== ====== =================
+
+The 4th bit encodes the source operand:
+
+ ====== ===== ========================================
+ source value description
+ ====== ===== ========================================
+ BPF_K 0x00 use 32-bit immediate as source operand
+ BPF_X 0x08 use 'src_reg' register as source operand
+ ====== ===== ========================================
+
+The four MSB bits store the operation code.
+
+
+Arithmetic instructions
+-----------------------
+
+BPF_ALU uses 32-bit wide operands while BPF_ALU64 uses 64-bit wide operands for
+otherwise identical operations.
+The code field encodes the operation as below:
+
+ ======== ===== ==========================
+ code value description
+ ======== ===== ==========================
+ BPF_ADD 0x00 dst += src
+ BPF_SUB 0x10 dst -= src
+ BPF_MUL 0x20 dst \*= src
+ BPF_DIV 0x30 dst /= src
+ BPF_OR 0x40 dst \|= src
+ BPF_AND 0x50 dst &= src
+ BPF_LSH 0x60 dst <<= src
+ BPF_RSH 0x70 dst >>= src
+ BPF_NEG 0x80 dst = ~src
+ BPF_MOD 0x90 dst %= src
+ BPF_XOR 0xa0 dst ^= src
+ BPF_MOV 0xb0 dst = src
+ BPF_ARSH 0xc0 sign extending shift right
+ BPF_END 0xd0 endianness conversion
+ ======== ===== ==========================
+
+BPF_ADD | BPF_X | BPF_ALU means::
+
+ dst_reg = (u32) dst_reg + (u32) src_reg;
+
+BPF_ADD | BPF_X | BPF_ALU64 means::
+
+ dst_reg = dst_reg + src_reg
+
+BPF_XOR | BPF_K | BPF_ALU means::
+
+ src_reg = (u32) src_reg ^ (u32) imm32
+
+BPF_XOR | BPF_K | BPF_ALU64 means::
+
+ src_reg = src_reg ^ imm32
+
+
+Jump instructions
+-----------------
+
+BPF_JMP32 uses 32-bit wide operands while BPF_JMP uses 64-bit wide operands for
+otherwise identical operations.
+The code field encodes the operation as below:
+
+ ======== ===== ========================= ============
+ code value description notes
+ ======== ===== ========================= ============
+ BPF_JA 0x00 PC += off BPF_JMP only
+ BPF_JEQ 0x10 PC += off if dst == src
+ BPF_JGT 0x20 PC += off if dst > src unsigned
+ BPF_JGE 0x30 PC += off if dst >= src unsigned
+ BPF_JSET 0x40 PC += off if dst & src
+ BPF_JNE 0x50 PC += off if dst != src
+ BPF_JSGT 0x60 PC += off if dst > src signed
+ BPF_JSGE 0x70 PC += off if dst >= src signed
+ BPF_CALL 0x80 function call
+ BPF_EXIT 0x90 function / program return BPF_JMP only
+ BPF_JLT 0xa0 PC += off if dst < src unsigned
+ BPF_JLE 0xb0 PC += off if dst <= src unsigned
+ BPF_JSLT 0xc0 PC += off if dst < src signed
+ BPF_JSLE 0xd0 PC += off if dst <= src signed
+ ======== ===== ========================= ============
+
+The eBPF program needs to store the return value into register R0 before doing a
+BPF_EXIT.
+
+
+Load and store instructions
+===========================
+
+For load and store instructions (BPF_LD, BPF_LDX, BPF_ST and BPF_STX), the
+8-bit 'opcode' field is divided as:
+
+ ============ ====== =================
+ 3 bits (MSB) 2 bits 3 bits (LSB)
+ ============ ====== =================
+ mode size instruction class
+ ============ ====== =================
+
+The size modifier is one of:
+
+ ============= ===== =====================
+ size modifier value description
+ ============= ===== =====================
+ BPF_W 0x00 word (4 bytes)
+ BPF_H 0x08 half word (2 bytes)
+ BPF_B 0x10 byte
+ BPF_DW 0x18 double word (8 bytes)
+ ============= ===== =====================
+
+The mode modifier is one of:
+
+ ============= ===== ====================================
+ mode modifier value description
+ ============= ===== ====================================
+ BPF_IMM 0x00 used for 64-bit mov
+ BPF_ABS 0x20 legacy BPF packet access
+ BPF_IND 0x40 legacy BPF packet access
+ BPF_MEM 0x60 all normal load and store operations
+ BPF_ATOMIC 0xc0 atomic operations
+ ============= ===== ====================================
+
+BPF_MEM | <size> | BPF_STX means::
+
+ *(size *) (dst_reg + off) = src_reg
+
+BPF_MEM | <size> | BPF_ST means::
+
+ *(size *) (dst_reg + off) = imm32
+
+BPF_MEM | <size> | BPF_LDX means::
+
+ dst_reg = *(size *) (src_reg + off)
+
+Where size is one of: BPF_B or BPF_H or BPF_W or BPF_DW.
+
+Atomic operations
+-----------------
+
+eBPF includes atomic operations, which use the immediate field for extra
+encoding::
+
+ .imm = BPF_ADD, .code = BPF_ATOMIC | BPF_W | BPF_STX: lock xadd *(u32 *)(dst_reg + off16) += src_reg
+ .imm = BPF_ADD, .code = BPF_ATOMIC | BPF_DW | BPF_STX: lock xadd *(u64 *)(dst_reg + off16) += src_reg
+
+The basic atomic operations supported are::
+
+ BPF_ADD
+ BPF_AND
+ BPF_OR
+ BPF_XOR
+
+Each having equivalent semantics with the ``BPF_ADD`` example, that is: the
+memory location addresed by ``dst_reg + off`` is atomically modified, with
+``src_reg`` as the other operand. If the ``BPF_FETCH`` flag is set in the
+immediate, then these operations also overwrite ``src_reg`` with the
+value that was in memory before it was modified.
+
+The more special operations are::
+
+ BPF_XCHG
+
+This atomically exchanges ``src_reg`` with the value addressed by ``dst_reg +
+off``. ::
+
+ BPF_CMPXCHG
+
+This atomically compares the value addressed by ``dst_reg + off`` with
+``R0``. If they match it is replaced with ``src_reg``. In either case, the
+value that was there before is zero-extended and loaded back to ``R0``.
+
+Note that 1 and 2 byte atomic operations are not supported.
+
+Clang can generate atomic instructions by default when ``-mcpu=v3`` is
+enabled. If a lower version for ``-mcpu`` is set, the only atomic instruction
+Clang can generate is ``BPF_ADD`` *without* ``BPF_FETCH``. If you need to enable
+the atomics features, while keeping a lower ``-mcpu`` version, you can use
+``-Xclang -target-feature -Xclang +alu32``.
+
+You may encounter ``BPF_XADD`` - this is a legacy name for ``BPF_ATOMIC``,
+referring to the exclusive-add operation encoded when the immediate field is
+zero.
+
+16-byte instructions
+--------------------
+
+eBPF has one 16-byte instruction: ``BPF_LD | BPF_DW | BPF_IMM`` which consists
+of two consecutive ``struct bpf_insn`` 8-byte blocks and interpreted as single
+instruction that loads 64-bit immediate value into a dst_reg.
+
+Packet access instructions
+--------------------------
+
+eBPF has two non-generic instructions: (BPF_ABS | <size> | BPF_LD) and
+(BPF_IND | <size> | BPF_LD) which are used to access packet data.
+
+They had to be carried over from classic BPF to have strong performance of
+socket filters running in eBPF interpreter. These instructions can only
+be used when interpreter context is a pointer to ``struct sk_buff`` and
+have seven implicit operands. Register R6 is an implicit input that must
+contain pointer to sk_buff. Register R0 is an implicit output which contains
+the data fetched from the packet. Registers R1-R5 are scratch registers
+and must not be used to store the data across BPF_ABS | BPF_LD or
+BPF_IND | BPF_LD instructions.
+
+These instructions have implicit program exit condition as well. When
+eBPF program is trying to access the data beyond the packet boundary,
+the interpreter will abort the execution of the program. JIT compilers
+therefore must preserve this property. src_reg and imm32 fields are
+explicit inputs to these instructions.
+
+For example, BPF_IND | BPF_W | BPF_LD means::
+
+ R0 = ntohl(*(u32 *) (((struct sk_buff *) R6)->data + src_reg + imm32))
+
+and R1 - R5 are clobbered.
diff --git a/Documentation/bpf/libbpf/index.rst b/Documentation/bpf/libbpf/index.rst
index 4f8adfc..4e8c656 100644
--- a/Documentation/bpf/libbpf/index.rst
+++ b/Documentation/bpf/libbpf/index.rst
@@ -3,8 +3,6 @@
libbpf
======
-For API documentation see the `versioned API documentation site <https://libbpf.readthedocs.io/en/latest/api.html>`_.
-
.. toctree::
:maxdepth: 1
@@ -14,6 +12,8 @@
This is documentation for libbpf, a userspace library for loading and
interacting with bpf programs.
+For API documentation see the `versioned API documentation site <https://libbpf.readthedocs.io/en/latest/api.html>`_.
+
All general BPF questions, including kernel functionality, libbpf APIs and
their application, should be sent to bpf@vger.kernel.org mailing list.
You can `subscribe <http://vger.kernel.org/vger-lists.html#bpf>`_ to the
diff --git a/Documentation/bpf/maps.rst b/Documentation/bpf/maps.rst
new file mode 100644
index 0000000..f41619e
--- /dev/null
+++ b/Documentation/bpf/maps.rst
@@ -0,0 +1,52 @@
+
+=========
+eBPF maps
+=========
+
+'maps' is a generic storage of different types for sharing data between kernel
+and userspace.
+
+The maps are accessed from user space via BPF syscall, which has commands:
+
+- create a map with given type and attributes
+ ``map_fd = bpf(BPF_MAP_CREATE, union bpf_attr *attr, u32 size)``
+ using attr->map_type, attr->key_size, attr->value_size, attr->max_entries
+ returns process-local file descriptor or negative error
+
+- lookup key in a given map
+ ``err = bpf(BPF_MAP_LOOKUP_ELEM, union bpf_attr *attr, u32 size)``
+ using attr->map_fd, attr->key, attr->value
+ returns zero and stores found elem into value or negative error
+
+- create or update key/value pair in a given map
+ ``err = bpf(BPF_MAP_UPDATE_ELEM, union bpf_attr *attr, u32 size)``
+ using attr->map_fd, attr->key, attr->value
+ returns zero or negative error
+
+- find and delete element by key in a given map
+ ``err = bpf(BPF_MAP_DELETE_ELEM, union bpf_attr *attr, u32 size)``
+ using attr->map_fd, attr->key
+
+- to delete map: close(fd)
+ Exiting process will delete maps automatically
+
+userspace programs use this syscall to create/access maps that eBPF programs
+are concurrently updating.
+
+maps can have different types: hash, array, bloom filter, radix-tree, etc.
+
+The map is defined by:
+
+ - type
+ - max number of elements
+ - key size in bytes
+ - value size in bytes
+
+Map Types
+=========
+
+.. toctree::
+ :maxdepth: 1
+ :glob:
+
+ map_*
\ No newline at end of file
diff --git a/Documentation/bpf/other.rst b/Documentation/bpf/other.rst
new file mode 100644
index 0000000..3d61963
--- /dev/null
+++ b/Documentation/bpf/other.rst
@@ -0,0 +1,9 @@
+=====
+Other
+=====
+
+.. toctree::
+ :maxdepth: 1
+
+ ringbuf
+ llvm_reloc
\ No newline at end of file
diff --git a/Documentation/bpf/bpf_lsm.rst b/Documentation/bpf/prog_lsm.rst
similarity index 100%
rename from Documentation/bpf/bpf_lsm.rst
rename to Documentation/bpf/prog_lsm.rst
diff --git a/Documentation/bpf/programs.rst b/Documentation/bpf/programs.rst
new file mode 100644
index 0000000..620eb66
--- /dev/null
+++ b/Documentation/bpf/programs.rst
@@ -0,0 +1,9 @@
+=============
+Program Types
+=============
+
+.. toctree::
+ :maxdepth: 1
+ :glob:
+
+ prog_*
diff --git a/Documentation/bpf/syscall_api.rst b/Documentation/bpf/syscall_api.rst
new file mode 100644
index 0000000..f0a1dff
--- /dev/null
+++ b/Documentation/bpf/syscall_api.rst
@@ -0,0 +1,11 @@
+===========
+Syscall API
+===========
+
+The primary info for the bpf syscall is available in the `man-pages`_
+for `bpf(2)`_. For more information about the userspace API, see
+Documentation/userspace-api/ebpf/index.rst.
+
+.. Links:
+.. _man-pages: https://www.kernel.org/doc/man-pages/
+.. _bpf(2): https://man7.org/linux/man-pages/man2/bpf.2.html
\ No newline at end of file
diff --git a/Documentation/bpf/test_debug.rst b/Documentation/bpf/test_debug.rst
new file mode 100644
index 0000000..ebf0cac
--- /dev/null
+++ b/Documentation/bpf/test_debug.rst
@@ -0,0 +1,9 @@
+=========================
+Testing and debugging BPF
+=========================
+
+.. toctree::
+ :maxdepth: 1
+
+ drgn
+ s390
diff --git a/Documentation/bpf/verifier.rst b/Documentation/bpf/verifier.rst
new file mode 100644
index 0000000..fae5f62
--- /dev/null
+++ b/Documentation/bpf/verifier.rst
@@ -0,0 +1,529 @@
+
+=============
+eBPF verifier
+=============
+
+The safety of the eBPF program is determined in two steps.
+
+First step does DAG check to disallow loops and other CFG validation.
+In particular it will detect programs that have unreachable instructions.
+(though classic BPF checker allows them)
+
+Second step starts from the first insn and descends all possible paths.
+It simulates execution of every insn and observes the state change of
+registers and stack.
+
+At the start of the program the register R1 contains a pointer to context
+and has type PTR_TO_CTX.
+If verifier sees an insn that does R2=R1, then R2 has now type
+PTR_TO_CTX as well and can be used on the right hand side of expression.
+If R1=PTR_TO_CTX and insn is R2=R1+R1, then R2=SCALAR_VALUE,
+since addition of two valid pointers makes invalid pointer.
+(In 'secure' mode verifier will reject any type of pointer arithmetic to make
+sure that kernel addresses don't leak to unprivileged users)
+
+If register was never written to, it's not readable::
+
+ bpf_mov R0 = R2
+ bpf_exit
+
+will be rejected, since R2 is unreadable at the start of the program.
+
+After kernel function call, R1-R5 are reset to unreadable and
+R0 has a return type of the function.
+
+Since R6-R9 are callee saved, their state is preserved across the call.
+
+::
+
+ bpf_mov R6 = 1
+ bpf_call foo
+ bpf_mov R0 = R6
+ bpf_exit
+
+is a correct program. If there was R1 instead of R6, it would have
+been rejected.
+
+load/store instructions are allowed only with registers of valid types, which
+are PTR_TO_CTX, PTR_TO_MAP, PTR_TO_STACK. They are bounds and alignment checked.
+For example::
+
+ bpf_mov R1 = 1
+ bpf_mov R2 = 2
+ bpf_xadd *(u32 *)(R1 + 3) += R2
+ bpf_exit
+
+will be rejected, since R1 doesn't have a valid pointer type at the time of
+execution of instruction bpf_xadd.
+
+At the start R1 type is PTR_TO_CTX (a pointer to generic ``struct bpf_context``)
+A callback is used to customize verifier to restrict eBPF program access to only
+certain fields within ctx structure with specified size and alignment.
+
+For example, the following insn::
+
+ bpf_ld R0 = *(u32 *)(R6 + 8)
+
+intends to load a word from address R6 + 8 and store it into R0
+If R6=PTR_TO_CTX, via is_valid_access() callback the verifier will know
+that offset 8 of size 4 bytes can be accessed for reading, otherwise
+the verifier will reject the program.
+If R6=PTR_TO_STACK, then access should be aligned and be within
+stack bounds, which are [-MAX_BPF_STACK, 0). In this example offset is 8,
+so it will fail verification, since it's out of bounds.
+
+The verifier will allow eBPF program to read data from stack only after
+it wrote into it.
+
+Classic BPF verifier does similar check with M[0-15] memory slots.
+For example::
+
+ bpf_ld R0 = *(u32 *)(R10 - 4)
+ bpf_exit
+
+is invalid program.
+Though R10 is correct read-only register and has type PTR_TO_STACK
+and R10 - 4 is within stack bounds, there were no stores into that location.
+
+Pointer register spill/fill is tracked as well, since four (R6-R9)
+callee saved registers may not be enough for some programs.
+
+Allowed function calls are customized with bpf_verifier_ops->get_func_proto()
+The eBPF verifier will check that registers match argument constraints.
+After the call register R0 will be set to return type of the function.
+
+Function calls is a main mechanism to extend functionality of eBPF programs.
+Socket filters may let programs to call one set of functions, whereas tracing
+filters may allow completely different set.
+
+If a function made accessible to eBPF program, it needs to be thought through
+from safety point of view. The verifier will guarantee that the function is
+called with valid arguments.
+
+seccomp vs socket filters have different security restrictions for classic BPF.
+Seccomp solves this by two stage verifier: classic BPF verifier is followed
+by seccomp verifier. In case of eBPF one configurable verifier is shared for
+all use cases.
+
+See details of eBPF verifier in kernel/bpf/verifier.c
+
+Register value tracking
+=======================
+
+In order to determine the safety of an eBPF program, the verifier must track
+the range of possible values in each register and also in each stack slot.
+This is done with ``struct bpf_reg_state``, defined in include/linux/
+bpf_verifier.h, which unifies tracking of scalar and pointer values. Each
+register state has a type, which is either NOT_INIT (the register has not been
+written to), SCALAR_VALUE (some value which is not usable as a pointer), or a
+pointer type. The types of pointers describe their base, as follows:
+
+
+ PTR_TO_CTX
+ Pointer to bpf_context.
+ CONST_PTR_TO_MAP
+ Pointer to struct bpf_map. "Const" because arithmetic
+ on these pointers is forbidden.
+ PTR_TO_MAP_VALUE
+ Pointer to the value stored in a map element.
+ PTR_TO_MAP_VALUE_OR_NULL
+ Either a pointer to a map value, or NULL; map accesses
+ (see maps.rst) return this type, which becomes a
+ PTR_TO_MAP_VALUE when checked != NULL. Arithmetic on
+ these pointers is forbidden.
+ PTR_TO_STACK
+ Frame pointer.
+ PTR_TO_PACKET
+ skb->data.
+ PTR_TO_PACKET_END
+ skb->data + headlen; arithmetic forbidden.
+ PTR_TO_SOCKET
+ Pointer to struct bpf_sock_ops, implicitly refcounted.
+ PTR_TO_SOCKET_OR_NULL
+ Either a pointer to a socket, or NULL; socket lookup
+ returns this type, which becomes a PTR_TO_SOCKET when
+ checked != NULL. PTR_TO_SOCKET is reference-counted,
+ so programs must release the reference through the
+ socket release function before the end of the program.
+ Arithmetic on these pointers is forbidden.
+
+However, a pointer may be offset from this base (as a result of pointer
+arithmetic), and this is tracked in two parts: the 'fixed offset' and 'variable
+offset'. The former is used when an exactly-known value (e.g. an immediate
+operand) is added to a pointer, while the latter is used for values which are
+not exactly known. The variable offset is also used in SCALAR_VALUEs, to track
+the range of possible values in the register.
+
+The verifier's knowledge about the variable offset consists of:
+
+* minimum and maximum values as unsigned
+* minimum and maximum values as signed
+
+* knowledge of the values of individual bits, in the form of a 'tnum': a u64
+ 'mask' and a u64 'value'. 1s in the mask represent bits whose value is unknown;
+ 1s in the value represent bits known to be 1. Bits known to be 0 have 0 in both
+ mask and value; no bit should ever be 1 in both. For example, if a byte is read
+ into a register from memory, the register's top 56 bits are known zero, while
+ the low 8 are unknown - which is represented as the tnum (0x0; 0xff). If we
+ then OR this with 0x40, we get (0x40; 0xbf), then if we add 1 we get (0x0;
+ 0x1ff), because of potential carries.
+
+Besides arithmetic, the register state can also be updated by conditional
+branches. For instance, if a SCALAR_VALUE is compared > 8, in the 'true' branch
+it will have a umin_value (unsigned minimum value) of 9, whereas in the 'false'
+branch it will have a umax_value of 8. A signed compare (with BPF_JSGT or
+BPF_JSGE) would instead update the signed minimum/maximum values. Information
+from the signed and unsigned bounds can be combined; for instance if a value is
+first tested < 8 and then tested s> 4, the verifier will conclude that the value
+is also > 4 and s< 8, since the bounds prevent crossing the sign boundary.
+
+PTR_TO_PACKETs with a variable offset part have an 'id', which is common to all
+pointers sharing that same variable offset. This is important for packet range
+checks: after adding a variable to a packet pointer register A, if you then copy
+it to another register B and then add a constant 4 to A, both registers will
+share the same 'id' but the A will have a fixed offset of +4. Then if A is
+bounds-checked and found to be less than a PTR_TO_PACKET_END, the register B is
+now known to have a safe range of at least 4 bytes. See 'Direct packet access',
+below, for more on PTR_TO_PACKET ranges.
+
+The 'id' field is also used on PTR_TO_MAP_VALUE_OR_NULL, common to all copies of
+the pointer returned from a map lookup. This means that when one copy is
+checked and found to be non-NULL, all copies can become PTR_TO_MAP_VALUEs.
+As well as range-checking, the tracked information is also used for enforcing
+alignment of pointer accesses. For instance, on most systems the packet pointer
+is 2 bytes after a 4-byte alignment. If a program adds 14 bytes to that to jump
+over the Ethernet header, then reads IHL and addes (IHL * 4), the resulting
+pointer will have a variable offset known to be 4n+2 for some n, so adding the 2
+bytes (NET_IP_ALIGN) gives a 4-byte alignment and so word-sized accesses through
+that pointer are safe.
+The 'id' field is also used on PTR_TO_SOCKET and PTR_TO_SOCKET_OR_NULL, common
+to all copies of the pointer returned from a socket lookup. This has similar
+behaviour to the handling for PTR_TO_MAP_VALUE_OR_NULL->PTR_TO_MAP_VALUE, but
+it also handles reference tracking for the pointer. PTR_TO_SOCKET implicitly
+represents a reference to the corresponding ``struct sock``. To ensure that the
+reference is not leaked, it is imperative to NULL-check the reference and in
+the non-NULL case, and pass the valid reference to the socket release function.
+
+Direct packet access
+====================
+
+In cls_bpf and act_bpf programs the verifier allows direct access to the packet
+data via skb->data and skb->data_end pointers.
+Ex::
+
+ 1: r4 = *(u32 *)(r1 +80) /* load skb->data_end */
+ 2: r3 = *(u32 *)(r1 +76) /* load skb->data */
+ 3: r5 = r3
+ 4: r5 += 14
+ 5: if r5 > r4 goto pc+16
+ R1=ctx R3=pkt(id=0,off=0,r=14) R4=pkt_end R5=pkt(id=0,off=14,r=14) R10=fp
+ 6: r0 = *(u16 *)(r3 +12) /* access 12 and 13 bytes of the packet */
+
+this 2byte load from the packet is safe to do, since the program author
+did check ``if (skb->data + 14 > skb->data_end) goto err`` at insn #5 which
+means that in the fall-through case the register R3 (which points to skb->data)
+has at least 14 directly accessible bytes. The verifier marks it
+as R3=pkt(id=0,off=0,r=14).
+id=0 means that no additional variables were added to the register.
+off=0 means that no additional constants were added.
+r=14 is the range of safe access which means that bytes [R3, R3 + 14) are ok.
+Note that R5 is marked as R5=pkt(id=0,off=14,r=14). It also points
+to the packet data, but constant 14 was added to the register, so
+it now points to ``skb->data + 14`` and accessible range is [R5, R5 + 14 - 14)
+which is zero bytes.
+
+More complex packet access may look like::
+
+
+ R0=inv1 R1=ctx R3=pkt(id=0,off=0,r=14) R4=pkt_end R5=pkt(id=0,off=14,r=14) R10=fp
+ 6: r0 = *(u8 *)(r3 +7) /* load 7th byte from the packet */
+ 7: r4 = *(u8 *)(r3 +12)
+ 8: r4 *= 14
+ 9: r3 = *(u32 *)(r1 +76) /* load skb->data */
+ 10: r3 += r4
+ 11: r2 = r1
+ 12: r2 <<= 48
+ 13: r2 >>= 48
+ 14: r3 += r2
+ 15: r2 = r3
+ 16: r2 += 8
+ 17: r1 = *(u32 *)(r1 +80) /* load skb->data_end */
+ 18: if r2 > r1 goto pc+2
+ R0=inv(id=0,umax_value=255,var_off=(0x0; 0xff)) R1=pkt_end R2=pkt(id=2,off=8,r=8) R3=pkt(id=2,off=0,r=8) R4=inv(id=0,umax_value=3570,var_off=(0x0; 0xfffe)) R5=pkt(id=0,off=14,r=14) R10=fp
+ 19: r1 = *(u8 *)(r3 +4)
+
+The state of the register R3 is R3=pkt(id=2,off=0,r=8)
+id=2 means that two ``r3 += rX`` instructions were seen, so r3 points to some
+offset within a packet and since the program author did
+``if (r3 + 8 > r1) goto err`` at insn #18, the safe range is [R3, R3 + 8).
+The verifier only allows 'add'/'sub' operations on packet registers. Any other
+operation will set the register state to 'SCALAR_VALUE' and it won't be
+available for direct packet access.
+
+Operation ``r3 += rX`` may overflow and become less than original skb->data,
+therefore the verifier has to prevent that. So when it sees ``r3 += rX``
+instruction and rX is more than 16-bit value, any subsequent bounds-check of r3
+against skb->data_end will not give us 'range' information, so attempts to read
+through the pointer will give "invalid access to packet" error.
+
+Ex. after insn ``r4 = *(u8 *)(r3 +12)`` (insn #7 above) the state of r4 is
+R4=inv(id=0,umax_value=255,var_off=(0x0; 0xff)) which means that upper 56 bits
+of the register are guaranteed to be zero, and nothing is known about the lower
+8 bits. After insn ``r4 *= 14`` the state becomes
+R4=inv(id=0,umax_value=3570,var_off=(0x0; 0xfffe)), since multiplying an 8-bit
+value by constant 14 will keep upper 52 bits as zero, also the least significant
+bit will be zero as 14 is even. Similarly ``r2 >>= 48`` will make
+R2=inv(id=0,umax_value=65535,var_off=(0x0; 0xffff)), since the shift is not sign
+extending. This logic is implemented in adjust_reg_min_max_vals() function,
+which calls adjust_ptr_min_max_vals() for adding pointer to scalar (or vice
+versa) and adjust_scalar_min_max_vals() for operations on two scalars.
+
+The end result is that bpf program author can access packet directly
+using normal C code as::
+
+ void *data = (void *)(long)skb->data;
+ void *data_end = (void *)(long)skb->data_end;
+ struct eth_hdr *eth = data;
+ struct iphdr *iph = data + sizeof(*eth);
+ struct udphdr *udp = data + sizeof(*eth) + sizeof(*iph);
+
+ if (data + sizeof(*eth) + sizeof(*iph) + sizeof(*udp) > data_end)
+ return 0;
+ if (eth->h_proto != htons(ETH_P_IP))
+ return 0;
+ if (iph->protocol != IPPROTO_UDP || iph->ihl != 5)
+ return 0;
+ if (udp->dest == 53 || udp->source == 9)
+ ...;
+
+which makes such programs easier to write comparing to LD_ABS insn
+and significantly faster.
+
+Pruning
+=======
+
+The verifier does not actually walk all possible paths through the program. For
+each new branch to analyse, the verifier looks at all the states it's previously
+been in when at this instruction. If any of them contain the current state as a
+subset, the branch is 'pruned' - that is, the fact that the previous state was
+accepted implies the current state would be as well. For instance, if in the
+previous state, r1 held a packet-pointer, and in the current state, r1 holds a
+packet-pointer with a range as long or longer and at least as strict an
+alignment, then r1 is safe. Similarly, if r2 was NOT_INIT before then it can't
+have been used by any path from that point, so any value in r2 (including
+another NOT_INIT) is safe. The implementation is in the function regsafe().
+Pruning considers not only the registers but also the stack (and any spilled
+registers it may hold). They must all be safe for the branch to be pruned.
+This is implemented in states_equal().
+
+Understanding eBPF verifier messages
+====================================
+
+The following are few examples of invalid eBPF programs and verifier error
+messages as seen in the log:
+
+Program with unreachable instructions::
+
+ static struct bpf_insn prog[] = {
+ BPF_EXIT_INSN(),
+ BPF_EXIT_INSN(),
+ };
+
+Error:
+
+ unreachable insn 1
+
+Program that reads uninitialized register::
+
+ BPF_MOV64_REG(BPF_REG_0, BPF_REG_2),
+ BPF_EXIT_INSN(),
+
+Error::
+
+ 0: (bf) r0 = r2
+ R2 !read_ok
+
+Program that doesn't initialize R0 before exiting::
+
+ BPF_MOV64_REG(BPF_REG_2, BPF_REG_1),
+ BPF_EXIT_INSN(),
+
+Error::
+
+ 0: (bf) r2 = r1
+ 1: (95) exit
+ R0 !read_ok
+
+Program that accesses stack out of bounds::
+
+ BPF_ST_MEM(BPF_DW, BPF_REG_10, 8, 0),
+ BPF_EXIT_INSN(),
+
+Error::
+
+ 0: (7a) *(u64 *)(r10 +8) = 0
+ invalid stack off=8 size=8
+
+Program that doesn't initialize stack before passing its address into function::
+
+ BPF_MOV64_REG(BPF_REG_2, BPF_REG_10),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -8),
+ BPF_LD_MAP_FD(BPF_REG_1, 0),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_map_lookup_elem),
+ BPF_EXIT_INSN(),
+
+Error::
+
+ 0: (bf) r2 = r10
+ 1: (07) r2 += -8
+ 2: (b7) r1 = 0x0
+ 3: (85) call 1
+ invalid indirect read from stack off -8+0 size 8
+
+Program that uses invalid map_fd=0 while calling to map_lookup_elem() function::
+
+ BPF_ST_MEM(BPF_DW, BPF_REG_10, -8, 0),
+ BPF_MOV64_REG(BPF_REG_2, BPF_REG_10),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -8),
+ BPF_LD_MAP_FD(BPF_REG_1, 0),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_map_lookup_elem),
+ BPF_EXIT_INSN(),
+
+Error::
+
+ 0: (7a) *(u64 *)(r10 -8) = 0
+ 1: (bf) r2 = r10
+ 2: (07) r2 += -8
+ 3: (b7) r1 = 0x0
+ 4: (85) call 1
+ fd 0 is not pointing to valid bpf_map
+
+Program that doesn't check return value of map_lookup_elem() before accessing
+map element::
+
+ BPF_ST_MEM(BPF_DW, BPF_REG_10, -8, 0),
+ BPF_MOV64_REG(BPF_REG_2, BPF_REG_10),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -8),
+ BPF_LD_MAP_FD(BPF_REG_1, 0),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_map_lookup_elem),
+ BPF_ST_MEM(BPF_DW, BPF_REG_0, 0, 0),
+ BPF_EXIT_INSN(),
+
+Error::
+
+ 0: (7a) *(u64 *)(r10 -8) = 0
+ 1: (bf) r2 = r10
+ 2: (07) r2 += -8
+ 3: (b7) r1 = 0x0
+ 4: (85) call 1
+ 5: (7a) *(u64 *)(r0 +0) = 0
+ R0 invalid mem access 'map_value_or_null'
+
+Program that correctly checks map_lookup_elem() returned value for NULL, but
+accesses the memory with incorrect alignment::
+
+ BPF_ST_MEM(BPF_DW, BPF_REG_10, -8, 0),
+ BPF_MOV64_REG(BPF_REG_2, BPF_REG_10),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -8),
+ BPF_LD_MAP_FD(BPF_REG_1, 0),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_map_lookup_elem),
+ BPF_JMP_IMM(BPF_JEQ, BPF_REG_0, 0, 1),
+ BPF_ST_MEM(BPF_DW, BPF_REG_0, 4, 0),
+ BPF_EXIT_INSN(),
+
+Error::
+
+ 0: (7a) *(u64 *)(r10 -8) = 0
+ 1: (bf) r2 = r10
+ 2: (07) r2 += -8
+ 3: (b7) r1 = 1
+ 4: (85) call 1
+ 5: (15) if r0 == 0x0 goto pc+1
+ R0=map_ptr R10=fp
+ 6: (7a) *(u64 *)(r0 +4) = 0
+ misaligned access off 4 size 8
+
+Program that correctly checks map_lookup_elem() returned value for NULL and
+accesses memory with correct alignment in one side of 'if' branch, but fails
+to do so in the other side of 'if' branch::
+
+ BPF_ST_MEM(BPF_DW, BPF_REG_10, -8, 0),
+ BPF_MOV64_REG(BPF_REG_2, BPF_REG_10),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -8),
+ BPF_LD_MAP_FD(BPF_REG_1, 0),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_map_lookup_elem),
+ BPF_JMP_IMM(BPF_JEQ, BPF_REG_0, 0, 2),
+ BPF_ST_MEM(BPF_DW, BPF_REG_0, 0, 0),
+ BPF_EXIT_INSN(),
+ BPF_ST_MEM(BPF_DW, BPF_REG_0, 0, 1),
+ BPF_EXIT_INSN(),
+
+Error::
+
+ 0: (7a) *(u64 *)(r10 -8) = 0
+ 1: (bf) r2 = r10
+ 2: (07) r2 += -8
+ 3: (b7) r1 = 1
+ 4: (85) call 1
+ 5: (15) if r0 == 0x0 goto pc+2
+ R0=map_ptr R10=fp
+ 6: (7a) *(u64 *)(r0 +0) = 0
+ 7: (95) exit
+
+ from 5 to 8: R0=imm0 R10=fp
+ 8: (7a) *(u64 *)(r0 +0) = 1
+ R0 invalid mem access 'imm'
+
+Program that performs a socket lookup then sets the pointer to NULL without
+checking it::
+
+ BPF_MOV64_IMM(BPF_REG_2, 0),
+ BPF_STX_MEM(BPF_W, BPF_REG_10, BPF_REG_2, -8),
+ BPF_MOV64_REG(BPF_REG_2, BPF_REG_10),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -8),
+ BPF_MOV64_IMM(BPF_REG_3, 4),
+ BPF_MOV64_IMM(BPF_REG_4, 0),
+ BPF_MOV64_IMM(BPF_REG_5, 0),
+ BPF_EMIT_CALL(BPF_FUNC_sk_lookup_tcp),
+ BPF_MOV64_IMM(BPF_REG_0, 0),
+ BPF_EXIT_INSN(),
+
+Error::
+
+ 0: (b7) r2 = 0
+ 1: (63) *(u32 *)(r10 -8) = r2
+ 2: (bf) r2 = r10
+ 3: (07) r2 += -8
+ 4: (b7) r3 = 4
+ 5: (b7) r4 = 0
+ 6: (b7) r5 = 0
+ 7: (85) call bpf_sk_lookup_tcp#65
+ 8: (b7) r0 = 0
+ 9: (95) exit
+ Unreleased reference id=1, alloc_insn=7
+
+Program that performs a socket lookup but does not NULL-check the returned
+value::
+
+ BPF_MOV64_IMM(BPF_REG_2, 0),
+ BPF_STX_MEM(BPF_W, BPF_REG_10, BPF_REG_2, -8),
+ BPF_MOV64_REG(BPF_REG_2, BPF_REG_10),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -8),
+ BPF_MOV64_IMM(BPF_REG_3, 4),
+ BPF_MOV64_IMM(BPF_REG_4, 0),
+ BPF_MOV64_IMM(BPF_REG_5, 0),
+ BPF_EMIT_CALL(BPF_FUNC_sk_lookup_tcp),
+ BPF_EXIT_INSN(),
+
+Error::
+
+ 0: (b7) r2 = 0
+ 1: (63) *(u32 *)(r10 -8) = r2
+ 2: (bf) r2 = r10
+ 3: (07) r2 += -8
+ 4: (b7) r3 = 4
+ 5: (b7) r4 = 0
+ 6: (b7) r5 = 0
+ 7: (85) call bpf_sk_lookup_tcp#65
+ 8: (95) exit
+ Unreleased reference id=1, alloc_insn=7
diff --git a/Documentation/dev-tools/kunit/api/index.rst b/Documentation/dev-tools/kunit/api/index.rst
index b33ad72..3006cad 100644
--- a/Documentation/dev-tools/kunit/api/index.rst
+++ b/Documentation/dev-tools/kunit/api/index.rst
@@ -12,5 +12,4 @@
Documentation/dev-tools/kunit/api/test.rst
- - documents all of the standard testing API excluding mocking
- or mocking related features.
+ - documents all of the standard testing API
diff --git a/Documentation/dev-tools/kunit/api/test.rst b/Documentation/dev-tools/kunit/api/test.rst
index aaa97f1..c5eca42 100644
--- a/Documentation/dev-tools/kunit/api/test.rst
+++ b/Documentation/dev-tools/kunit/api/test.rst
@@ -4,8 +4,7 @@
Test API
========
-This file documents all of the standard testing API excluding mocking or mocking
-related features.
+This file documents all of the standard testing API.
.. kernel-doc:: include/kunit/test.h
:internal:
diff --git a/Documentation/dev-tools/kunit/index.rst b/Documentation/dev-tools/kunit/index.rst
index cacb35e..7af7dec 100644
--- a/Documentation/dev-tools/kunit/index.rst
+++ b/Documentation/dev-tools/kunit/index.rst
@@ -19,7 +19,7 @@
What is KUnit?
==============
-KUnit is a lightweight unit testing and mocking framework for the Linux kernel.
+KUnit is a lightweight unit testing framework for the Linux kernel.
KUnit is heavily inspired by JUnit, Python's unittest.mock, and
Googletest/Googlemock for C++. KUnit provides facilities for defining unit test
diff --git a/Documentation/dev-tools/kunit/start.rst b/Documentation/dev-tools/kunit/start.rst
index 1e00f92..0a5e655 100644
--- a/Documentation/dev-tools/kunit/start.rst
+++ b/Documentation/dev-tools/kunit/start.rst
@@ -50,10 +50,10 @@
using.
.. note::
- Note that removing something from the ``.kunitconfig`` will not trigger a
- rebuild of the ``.config`` file: the configuration is only updated if the
- ``.kunitconfig`` is not a subset of ``.config``. This means that you can use
- other tools (such as make menuconfig) to adjust other config options.
+ If you change the ``.kunitconfig``, kunit.py will trigger a rebuild of the
+ ``.config`` file. But you can edit the ``.config`` file directly or with
+ tools like ``make menuconfig O=.kunit``. As long as its a superset of
+ ``.kunitconfig``, kunit.py won't overwrite your changes.
Running the tests (KUnit Wrapper)
diff --git a/Documentation/devicetree/bindings/arm/apple.yaml b/Documentation/devicetree/bindings/arm/apple.yaml
index 1e772c8..8d93e8a 100644
--- a/Documentation/devicetree/bindings/arm/apple.yaml
+++ b/Documentation/devicetree/bindings/arm/apple.yaml
@@ -12,12 +12,19 @@
description: |
ARM platforms using SoCs designed by Apple Inc., branded "Apple Silicon".
- This currently includes devices based on the "M1" SoC, starting with the
- three Mac models released in late 2020:
+ This currently includes devices based on the "M1" SoC:
- Mac mini (M1, 2020)
- MacBook Pro (13-inch, M1, 2020)
- MacBook Air (M1, 2020)
+ - iMac (24-inch, M1, 2021)
+
+ And devices based on the "M1 Pro" and "M1 Max" SoCs:
+
+ - MacBook Pro (14-inch, M1 Pro, 2021)
+ - MacBook Pro (14-inch, M1 Max, 2021)
+ - MacBook Pro (16-inch, M1 Pro, 2021)
+ - MacBook Pro (16-inch, M1 Max, 2021)
The compatible property should follow this format:
@@ -56,8 +63,24 @@
- apple,j274 # Mac mini (M1, 2020)
- apple,j293 # MacBook Pro (13-inch, M1, 2020)
- apple,j313 # MacBook Air (M1, 2020)
+ - apple,j456 # iMac (24-inch, 4x USB-C, M1, 2021)
+ - apple,j457 # iMac (24-inch, 2x USB-C, M1, 2021)
- const: apple,t8103
- const: apple,arm-platform
+ - description: Apple M1 Pro SoC based platforms
+ items:
+ - enum:
+ - apple,j314s # MacBook Pro (14-inch, M1 Pro, 2021)
+ - apple,j316s # MacBook Pro (16-inch, M1 Pro, 2021)
+ - const: apple,t6000
+ - const: apple,arm-platform
+ - description: Apple M1 Max SoC based platforms
+ items:
+ - enum:
+ - apple,j314c # MacBook Pro (14-inch, M1 Max, 2021)
+ - apple,j316c # MacBook Pro (16-inch, M1 Max, 2021)
+ - const: apple,t6001
+ - const: apple,arm-platform
additionalProperties: true
diff --git a/Documentation/devicetree/bindings/arm/apple/apple,pmgr.yaml b/Documentation/devicetree/bindings/arm/apple/apple,pmgr.yaml
new file mode 100644
index 0000000..b6b5d3a
--- /dev/null
+++ b/Documentation/devicetree/bindings/arm/apple/apple,pmgr.yaml
@@ -0,0 +1,134 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/arm/apple/apple,pmgr.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Apple SoC Power Manager (PMGR)
+
+maintainers:
+ - Hector Martin <marcan@marcan.st>
+
+description: |
+ Apple SoCs include PMGR blocks responsible for power management,
+ which can control various clocks, resets, power states, and
+ performance features. This node represents the PMGR as a syscon,
+ with sub-nodes representing individual features.
+
+properties:
+ $nodename:
+ pattern: "^power-management@[0-9a-f]+$"
+
+ compatible:
+ items:
+ - enum:
+ - apple,t8103-pmgr
+ - apple,t6000-pmgr
+ - const: apple,pmgr
+ - const: syscon
+ - const: simple-mfd
+
+ reg:
+ maxItems: 1
+
+ "#address-cells":
+ const: 1
+
+ "#size-cells":
+ const: 1
+
+patternProperties:
+ "power-controller@[0-9a-f]+$":
+ description:
+ The individual power management domains within this controller
+ type: object
+ $ref: /power/apple,pmgr-pwrstate.yaml#
+
+required:
+ - compatible
+ - reg
+
+additionalProperties: false
+
+examples:
+ - |
+ soc {
+ #address-cells = <2>;
+ #size-cells = <2>;
+
+ power-management@23b700000 {
+ compatible = "apple,t8103-pmgr", "apple,pmgr", "syscon", "simple-mfd";
+ #address-cells = <1>;
+ #size-cells = <1>;
+ reg = <0x2 0x3b700000 0x0 0x14000>;
+
+ ps_sio: power-controller@1c0 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x1c0 8>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "sio";
+ apple,always-on;
+ };
+
+ ps_uart_p: power-controller@220 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x220 8>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "uart_p";
+ power-domains = <&ps_sio>;
+ };
+
+ ps_uart0: power-controller@270 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x270 8>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "uart0";
+ power-domains = <&ps_uart_p>;
+ };
+ };
+
+ power-management@23d280000 {
+ compatible = "apple,t8103-pmgr", "apple,pmgr", "syscon", "simple-mfd";
+ #address-cells = <1>;
+ #size-cells = <1>;
+ reg = <0x2 0x3d280000 0x0 0xc000>;
+
+ ps_aop_filter: power-controller@4000 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x4000 8>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "aop_filter";
+ };
+
+ ps_aop_base: power-controller@4010 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x4010 8>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "aop_base";
+ power-domains = <&ps_aop_filter>;
+ };
+
+ ps_aop_shim: power-controller@4038 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x4038 8>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "aop_shim";
+ power-domains = <&ps_aop_base>;
+ };
+
+ ps_aop_uart0: power-controller@4048 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x4048 8>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "aop_uart0";
+ power-domains = <&ps_aop_shim>;
+ };
+ };
+ };
diff --git a/Documentation/devicetree/bindings/arm/aspeed/aspeed,sbc.yaml b/Documentation/devicetree/bindings/arm/aspeed/aspeed,sbc.yaml
new file mode 100644
index 0000000..c72aab7
--- /dev/null
+++ b/Documentation/devicetree/bindings/arm/aspeed/aspeed,sbc.yaml
@@ -0,0 +1,37 @@
+# SPDX-License-Identifier: (GPL-2.0-or-later OR BSD-2-Clause)
+# Copyright 2021 Joel Stanley, IBM Corp.
+%YAML 1.2
+---
+$id: "http://devicetree.org/schemas/arm/aspeed/aspeed,sbc.yaml#"
+$schema: "http://devicetree.org/meta-schemas/core.yaml#"
+
+title: ASPEED Secure Boot Controller
+
+maintainers:
+ - Joel Stanley <joel@jms.id.au>
+ - Andrew Jeffery <andrew@aj.id.au>
+
+description: |
+ The ASPEED SoCs have a register bank for interacting with the secure boot
+ controller.
+
+properties:
+ compatible:
+ items:
+ - const: aspeed,ast2600-sbc
+
+ reg:
+ maxItems: 1
+
+required:
+ - compatible
+ - reg
+
+additionalProperties: false
+
+examples:
+ - |
+ sbc: secure-boot-controller@1e6f2000 {
+ compatible = "aspeed,ast2600-sbc";
+ reg = <0x1e6f2000 0x1000>;
+ };
diff --git a/Documentation/devicetree/bindings/arm/bcm/brcm,bcm4908.yaml b/Documentation/devicetree/bindings/arm/bcm/brcm,bcm4908.yaml
index 2cd4e4a..9b74553 100644
--- a/Documentation/devicetree/bindings/arm/bcm/brcm,bcm4908.yaml
+++ b/Documentation/devicetree/bindings/arm/bcm/brcm,bcm4908.yaml
@@ -29,6 +29,7 @@
items:
- enum:
- asus,gt-ac5300
+ - netgear,raxe500
- const: brcm,bcm4908
- description: BCM49408 based boards
diff --git a/Documentation/devicetree/bindings/arm/cpus.yaml b/Documentation/devicetree/bindings/arm/cpus.yaml
index f2ab642..dfa28e3 100644
--- a/Documentation/devicetree/bindings/arm/cpus.yaml
+++ b/Documentation/devicetree/bindings/arm/cpus.yaml
@@ -174,6 +174,7 @@
- qcom,kryo560
- qcom,kryo570
- qcom,kryo685
+ - qcom,kryo780
- qcom,scorpion
enable-method:
diff --git a/Documentation/devicetree/bindings/arm/firmware/linaro,optee-tz.yaml b/Documentation/devicetree/bindings/arm/firmware/linaro,optee-tz.yaml
index c24047c..9a42611 100644
--- a/Documentation/devicetree/bindings/arm/firmware/linaro,optee-tz.yaml
+++ b/Documentation/devicetree/bindings/arm/firmware/linaro,optee-tz.yaml
@@ -24,6 +24,12 @@
compatible:
const: linaro,optee-tz
+ interrupts:
+ maxItems: 1
+ description: |
+ This interrupt which is used to signal an event by the secure world
+ software is expected to be edge-triggered.
+
method:
enum: [smc, hvc]
description: |
@@ -42,10 +48,12 @@
examples:
- |
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
firmware {
optee {
compatible = "linaro,optee-tz";
method = "smc";
+ interrupts = <GIC_SPI 187 IRQ_TYPE_EDGE_RISING>;
};
};
diff --git a/Documentation/devicetree/bindings/arm/fsl.yaml b/Documentation/devicetree/bindings/arm/fsl.yaml
index 0b595b2..97f6eeb 100644
--- a/Documentation/devicetree/bindings/arm/fsl.yaml
+++ b/Documentation/devicetree/bindings/arm/fsl.yaml
@@ -240,6 +240,7 @@
- uniwest,imx6q-evi # Uniwest Evi
- variscite,dt6customboard
- wand,imx6q-wandboard # Wandboard i.MX6 Quad Board
+ - ysoft,imx6q-yapp4-crux # i.MX6 Quad Y Soft IOTA Crux board
- zealz,imx6q-gk802 # Zealz GK802
- zii,imx6q-zii-rdu2 # ZII RDU2 Board
- const: fsl,imx6q
@@ -323,6 +324,20 @@
- const: toradex,apalis_imx6q
- const: fsl,imx6q
+ - description: TQ-Systems TQMa6Q SoM (variant A) on MBa6x
+ items:
+ - const: tq,imx6q-mba6x-a
+ - const: tq,mba6a # Expected by bootloader, to be removed in the future
+ - const: tq,imx6q-tqma6q-a
+ - const: fsl,imx6q
+
+ - description: TQ-Systems TQMa6Q SoM (variant B) on MBa6x
+ items:
+ - const: tq,imx6q-mba6x-b
+ - const: tq,mba6b # Expected by bootloader, to be removed in the future
+ - const: tq,imx6q-tqma6q-b
+ - const: fsl,imx6q
+
- description: i.MX6QP based Boards
items:
- enum:
@@ -334,6 +349,7 @@
- kvg,vicutp # Kverneland UT1P board
- prt,prtwd3 # Protonic WD3 board
- wand,imx6qp-wandboard # Wandboard i.MX6 QuadPlus Board
+ - ysoft,imx6qp-yapp4-crux-plus # i.MX6 Quad Plus Y Soft IOTA Crux+ board
- zii,imx6qp-zii-rdu2 # ZII RDU2+ Board
- const: fsl,imx6qp
@@ -344,6 +360,13 @@
- const: phytec,imx6qdl-pcm058 # PHYTEC phyCORE-i.MX6
- const: fsl,imx6qp
+ - description: TQ-Systems TQMa6QP SoM on MBa6x
+ items:
+ - const: tq,imx6qp-mba6x-b
+ - const: tq,mba6b # Expected by bootloader, to be removed in the future
+ - const: tq,imx6qp-tqma6qp-b
+ - const: fsl,imx6qp
+
- description: i.MX6DL based Boards
items:
- enum:
@@ -482,6 +505,20 @@
- const: dh,imx6s-dhcom-som
- const: fsl,imx6dl
+ - description: TQ-Systems TQMa6DL SoM (variant A) on MBa6x
+ items:
+ - const: tq,imx6dl-mba6x-a
+ - const: tq,mba6a # Expected by bootloader, to be removed in the future
+ - const: tq,imx6dl-tqma6dl-a
+ - const: fsl,imx6dl
+
+ - description: TQ-Systems TQMa6DL SoM (variant B) on MBa6x
+ items:
+ - const: tq,imx6dl-mba6x-b
+ - const: tq,mba6b # Expected by bootloader, to be removed in the future
+ - const: tq,imx6dl-tqma6dl-b
+ - const: fsl,imx6dl
+
- description: i.MX6SL based Boards
items:
- enum:
@@ -580,6 +617,7 @@
items:
- enum:
- fsl,imx6ull-14x14-evk # i.MX6 UltraLiteLite 14x14 EVK Board
+ - joz,jozacp # JOZ Access Point
- kontron,imx6ull-n6411-som # Kontron N6411 SOM
- myir,imx6ull-mys-6ulx-eval # MYiR Tech iMX6ULL Evaluation Board
- toradex,colibri-imx6ull # Colibri iMX6ULL Modules
@@ -632,6 +670,7 @@
- description: i.MX6ULZ based Boards
items:
- enum:
+ - bsh,imx6ulz-bsh-smm-m2 # i.MX6 ULZ BSH SystemMaster
- fsl,imx6ulz-14x14-evk # i.MX6 ULZ 14x14 EVK Board
- const: fsl,imx6ull # This seems odd. Should be last?
- const: fsl,imx6ulz
@@ -754,10 +793,23 @@
- const: variscite,var-som-mx8mm
- const: fsl,imx8mm
+ - description:
+ TQMa8MxML is a series of SOM featuring NXP i.MX8MM system-on-chip
+ variants. It is designed to be soldered on different carrier boards.
+ All variants (TQMa8M[Q,D,S][L]ML) use the same device tree, hence only
+ one compatible is needed.
+ items:
+ - enum:
+ - tq,imx8mm-tqma8mqml-mba8mx # TQ-Systems GmbH i.MX8MM TQMa8MQML SOM on MBa8Mx
+ - const: tq,imx8mm-tqma8mqml # TQ-Systems GmbH i.MX8MM TQMa8MQML SOM
+ - const: fsl,imx8mm
+
- description: i.MX8MN based Boards
items:
- enum:
- beacon,imx8mn-beacon-kit # i.MX8MN Beacon Development Kit
+ - bsh,imx8mn-bsh-smm-s2 # i.MX8MN BSH SystemMaster S2
+ - bsh,imx8mn-bsh-smm-s2pro # i.MX8MN BSH SystemMaster S2 PRO
- fsl,imx8mn-ddr4-evk # i.MX8MN DDR4 EVK Board
- fsl,imx8mn-evk # i.MX8MN LPDDR4 EVK Board
- gw,imx8mn-gw7902 # i.MX8MM Gateworks Board
@@ -769,6 +821,17 @@
- const: variscite,var-som-mx8mn
- const: fsl,imx8mn
+ - description:
+ TQMa8MxNL is a series of SOM featuring NXP i.MX8MN system-on-chip
+ variants. It is designed to be soldered on different carrier boards.
+ All variants (TQMa8M[Q,D,S][L]NL) use the same device tree, hence only
+ one compatible is needed.
+ items:
+ - enum:
+ - tq,imx8mn-tqma8mqnl-mba8mx # TQ-Systems GmbH i.MX8MN TQMa8MQNL SOM on MBa8Mx
+ - const: tq,imx8mn-tqma8mqnl # TQ-Systems GmbH i.MX8MN TQMa8MQNL SOM
+ - const: fsl,imx8mn
+
- description: i.MX8MP based Boards
items:
- enum:
@@ -805,6 +868,15 @@
- const: purism,librem5
- const: fsl,imx8mq
+ - description:
+ TQMa8Mx is a series of SOM featuring NXP i.MX8MQ system-on-chip
+ variants. It is designed to be clicked on different carrier boards.
+ items:
+ - enum:
+ - tq,imx8mq-tqma8mq-mba8mx # TQ-Systems GmbH i.MX8MQ TQMa8Mx SOM on MBa8Mx
+ - const: tq,imx8mq-tqma8mq # TQ-Systems GmbH i.MX8MQ TQMa8Mx SOM
+ - const: fsl,imx8mq
+
- description: Zodiac Inflight Innovations Ultra Boards
items:
- enum:
@@ -834,6 +906,12 @@
- const: toradex,colibri-imx8x
- const: fsl,imx8qxp
+ - description: i.MX8ULP based Boards
+ items:
+ - enum:
+ - fsl,imx8ulp-evk # i.MX8ULP EVK Board
+ - const: fsl,imx8ulp
+
- description:
Freescale Vybrid Platform Device Tree Bindings
diff --git a/Documentation/devicetree/bindings/arm/mediatek.yaml b/Documentation/devicetree/bindings/arm/mediatek.yaml
index 0fa5549..0ffe1ac 100644
--- a/Documentation/devicetree/bindings/arm/mediatek.yaml
+++ b/Documentation/devicetree/bindings/arm/mediatek.yaml
@@ -79,6 +79,14 @@
- const: mediatek,mt7629
- items:
- enum:
+ - mediatek,mt7986a-rfb
+ - const: mediatek,mt7986a
+ - items:
+ - enum:
+ - mediatek,mt7986b-rfb
+ - const: mediatek,mt7986b
+ - items:
+ - enum:
- mediatek,mt8127-moose
- const: mediatek,mt8127
- items:
@@ -134,6 +142,10 @@
- google,krane-sku176
- const: google,krane
- const: mediatek,mt8183
+ - description: Google Cozmo (Acer Chromebook 314)
+ items:
+ - const: google,cozmo
+ - const: mediatek,mt8183
- description: Google Damu (ASUS Chromebook Flip CM3)
items:
- const: google,damu
@@ -143,7 +155,9 @@
- enum:
- google,fennel-sku0
- google,fennel-sku1
+ - google,fennel-sku2
- google,fennel-sku6
+ - google,fennel-sku7
- const: google,fennel
- const: mediatek,mt8183
- description: Google Juniper (Acer Chromebook Spin 311) / Kenzo (Acer Chromebook 311)
@@ -159,6 +173,12 @@
- const: google,kakadu-rev2
- const: google,kakadu
- const: mediatek,mt8183
+ - description: Google Kakadu (ASUS Chromebook Detachable CM3)
+ items:
+ - const: google,kakadu-rev3-sku22
+ - const: google,kakadu-rev2-sku22
+ - const: google,kakadu
+ - const: mediatek,mt8183
- description: Google Kappa (HP Chromebook 11a)
items:
- const: google,kappa
diff --git a/Documentation/devicetree/bindings/arm/msm/qcom,llcc.yaml b/Documentation/devicetree/bindings/arm/msm/qcom,llcc.yaml
index 62fcbd8..03882aa 100644
--- a/Documentation/devicetree/bindings/arm/msm/qcom,llcc.yaml
+++ b/Documentation/devicetree/bindings/arm/msm/qcom,llcc.yaml
@@ -24,6 +24,7 @@
- qcom,sc7180-llcc
- qcom,sc7280-llcc
- qcom,sdm845-llcc
+ - qcom,sm6350-llcc
- qcom,sm8150-llcc
- qcom,sm8250-llcc
@@ -44,7 +45,6 @@
- compatible
- reg
- reg-names
- - interrupts
additionalProperties: false
diff --git a/Documentation/devicetree/bindings/arm/qcom.yaml b/Documentation/devicetree/bindings/arm/qcom.yaml
index c8808e0..370aab2 100644
--- a/Documentation/devicetree/bindings/arm/qcom.yaml
+++ b/Documentation/devicetree/bindings/arm/qcom.yaml
@@ -48,8 +48,10 @@
sdx65
sm7225
sm8150
+ sdx65
sm8250
sm8350
+ sm8450
The 'board' element must be one of the following strings:
@@ -201,8 +203,10 @@
- items:
- enum:
+ - qcom,sc7280-crd
- qcom,sc7280-idp
- qcom,sc7280-idp2
+ - google,hoglin
- google,piglin
- google,senor
- const: qcom,sc7280
@@ -226,6 +230,11 @@
- items:
- enum:
+ - qcom,sdx65-mtp
+ - const: qcom,sdx65
+
+ - items:
+ - enum:
- qcom,ipq6018-cp01
- qcom,ipq6018-cp01-c1
- const: qcom,ipq6018
@@ -257,6 +266,11 @@
- qcom,sm8350-mtp
- const: qcom,sm8350
+ - items:
+ - enum:
+ - qcom,sm8450-qrd
+ - const: qcom,sm8450
+
additionalProperties: true
...
diff --git a/Documentation/devicetree/bindings/arm/renesas.yaml b/Documentation/devicetree/bindings/arm/renesas.yaml
index 5172065..6a9350e 100644
--- a/Documentation/devicetree/bindings/arm/renesas.yaml
+++ b/Documentation/devicetree/bindings/arm/renesas.yaml
@@ -315,6 +315,18 @@
- const: renesas,falcon-cpu
- const: renesas,r8a779a0
+ - description: R-Car S4-8 (R8A779F0)
+ items:
+ - enum:
+ - renesas,spider-cpu # Spider CPU board (RTP8A779F0ASKB0SC2S)
+ - const: renesas,r8a779f0
+
+ - items:
+ - enum:
+ - renesas,spider-breakout # Spider BreakOut board (RTP8A779F0ASKB0SB0S)
+ - const: renesas,spider-cpu
+ - const: renesas,r8a779f0
+
- description: R-Car H3e (R8A779M0)
items:
- enum:
diff --git a/Documentation/devicetree/bindings/arm/samsung/exynos-chipid.yaml b/Documentation/devicetree/bindings/arm/samsung/exynos-chipid.yaml
deleted file mode 100644
index bfc352a..0000000
--- a/Documentation/devicetree/bindings/arm/samsung/exynos-chipid.yaml
+++ /dev/null
@@ -1,41 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0
-%YAML 1.2
----
-$id: http://devicetree.org/schemas/arm/samsung/exynos-chipid.yaml#
-$schema: http://devicetree.org/meta-schemas/core.yaml#
-
-title: Samsung Exynos SoC series Chipid driver
-
-maintainers:
- - Krzysztof Kozlowski <krzk@kernel.org>
-
-properties:
- compatible:
- enum:
- - samsung,exynos4210-chipid
- - samsung,exynos850-chipid
-
- reg:
- maxItems: 1
-
- samsung,asv-bin:
- description:
- Adaptive Supply Voltage bin selection. This can be used
- to determine the ASV bin of an SoC if respective information
- is missing in the CHIPID registers or in the OTP memory.
- $ref: /schemas/types.yaml#/definitions/uint32
- enum: [0, 1, 2, 3]
-
-required:
- - compatible
- - reg
-
-additionalProperties: false
-
-examples:
- - |
- chipid@10000000 {
- compatible = "samsung,exynos4210-chipid";
- reg = <0x10000000 0x100>;
- samsung,asv-bin = <2>;
- };
diff --git a/Documentation/devicetree/bindings/arm/samsung/pmu.yaml b/Documentation/devicetree/bindings/arm/samsung/pmu.yaml
deleted file mode 100644
index 17678d9..0000000
--- a/Documentation/devicetree/bindings/arm/samsung/pmu.yaml
+++ /dev/null
@@ -1,128 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0
-%YAML 1.2
----
-$id: http://devicetree.org/schemas/arm/samsung/pmu.yaml#
-$schema: http://devicetree.org/meta-schemas/core.yaml#
-
-title: Samsung Exynos SoC series Power Management Unit (PMU)
-
-maintainers:
- - Krzysztof Kozlowski <krzk@kernel.org>
-
-# Custom select to avoid matching all nodes with 'syscon'
-select:
- properties:
- compatible:
- contains:
- enum:
- - samsung,exynos3250-pmu
- - samsung,exynos4210-pmu
- - samsung,exynos4412-pmu
- - samsung,exynos5250-pmu
- - samsung,exynos5260-pmu
- - samsung,exynos5410-pmu
- - samsung,exynos5420-pmu
- - samsung,exynos5433-pmu
- - samsung,exynos7-pmu
- - samsung-s5pv210-pmu
- required:
- - compatible
-
-properties:
- compatible:
- items:
- - enum:
- - samsung,exynos3250-pmu
- - samsung,exynos4210-pmu
- - samsung,exynos4412-pmu
- - samsung,exynos5250-pmu
- - samsung,exynos5260-pmu
- - samsung,exynos5410-pmu
- - samsung,exynos5420-pmu
- - samsung,exynos5433-pmu
- - samsung,exynos7-pmu
- - samsung-s5pv210-pmu
- - const: syscon
-
- reg:
- maxItems: 1
-
- assigned-clock-parents: true
- assigned-clocks: true
-
- '#clock-cells':
- const: 1
-
- clock-names:
- description:
- List of clock names for particular CLKOUT mux inputs
- minItems: 1
- maxItems: 32
- items:
- pattern: '^clkout([0-9]|[12][0-9]|3[0-1])$'
-
- clocks:
- minItems: 1
- maxItems: 32
-
- interrupt-controller:
- description:
- Some PMUs are capable of behaving as an interrupt controller (mostly
- to wake up a suspended PMU).
-
- '#interrupt-cells':
- description:
- Must be identical to the that of the parent interrupt controller.
- const: 3
-
- syscon-poweroff:
- $ref: "../../power/reset/syscon-poweroff.yaml#"
- type: object
- description:
- Node for power off method
-
- syscon-reboot:
- $ref: "../../power/reset/syscon-reboot.yaml#"
- type: object
- description:
- Node for reboot method
-
-required:
- - compatible
- - reg
-
-additionalProperties: false
-
-allOf:
- - if:
- properties:
- compatible:
- contains:
- enum:
- - samsung,exynos3250-pmu
- - samsung,exynos4210-pmu
- - samsung,exynos4412-pmu
- - samsung,exynos5250-pmu
- - samsung,exynos5410-pmu
- - samsung,exynos5420-pmu
- - samsung,exynos5433-pmu
- then:
- required:
- - '#clock-cells'
- - clock-names
- - clocks
-
-examples:
- - |
- #include <dt-bindings/clock/exynos5250.h>
-
- pmu_system_controller: system-controller@10040000 {
- compatible = "samsung,exynos5250-pmu", "syscon";
- reg = <0x10040000 0x5000>;
- interrupt-controller;
- #interrupt-cells = <3>;
- interrupt-parent = <&gic>;
- #clock-cells = <1>;
- clock-names = "clkout16";
- clocks = <&clock CLK_FIN_PLL>;
- };
diff --git a/Documentation/devicetree/bindings/arm/samsung/samsung-boards.yaml b/Documentation/devicetree/bindings/arm/samsung/samsung-boards.yaml
index ef6dc14..052cd94 100644
--- a/Documentation/devicetree/bindings/arm/samsung/samsung-boards.yaml
+++ b/Documentation/devicetree/bindings/arm/samsung/samsung-boards.yaml
@@ -199,6 +199,18 @@
- samsung,exynos7-espresso # Samsung Exynos7 Espresso
- const: samsung,exynos7
+ - description: Exynos7885 based boards
+ items:
+ - enum:
+ - samsung,jackpotlte # Samsung Galaxy A8 (2018)
+ - const: samsung,exynos7885
+
+ - description: Exynos850 based boards
+ items:
+ - enum:
+ - winlink,e850-96 # WinLink E850-96
+ - const: samsung,exynos850
+
- description: Exynos Auto v9 based boards
items:
- enum:
diff --git a/Documentation/devicetree/bindings/arm/stm32/stm32.yaml b/Documentation/devicetree/bindings/arm/stm32/stm32.yaml
index bcaf7be..b07720e 100644
--- a/Documentation/devicetree/bindings/arm/stm32/stm32.yaml
+++ b/Documentation/devicetree/bindings/arm/stm32/stm32.yaml
@@ -77,6 +77,7 @@
items:
- enum:
- engicam,icore-stm32mp1-ctouch2 # STM32MP1 Engicam i.Core STM32MP1 C.TOUCH 2.0
+ - engicam,icore-stm32mp1-ctouch2-of10 # STM32MP1 Engicam i.Core STM32MP1 C.TOUCH 2.0 10.1" OF
- engicam,icore-stm32mp1-edimm2.2 # STM32MP1 Engicam i.Core STM32MP1 EDIMM2.2 Starter Kit
- const: engicam,icore-stm32mp1 # STM32MP1 Engicam i.Core STM32MP1 SoM
- const: st,stm32mp157
diff --git a/Documentation/devicetree/bindings/arm/sunxi.yaml b/Documentation/devicetree/bindings/arm/sunxi.yaml
index 889128a..c8a3102 100644
--- a/Documentation/devicetree/bindings/arm/sunxi.yaml
+++ b/Documentation/devicetree/bindings/arm/sunxi.yaml
@@ -808,6 +808,11 @@
- const: oranth,tanix-tx6
- const: allwinner,sun50i-h6
+ - description: Tanix TX6 mini
+ items:
+ - const: oranth,tanix-tx6-mini
+ - const: allwinner,sun50i-h6
+
- description: TBS A711 Tablet
items:
- const: tbs-biometrics,a711
diff --git a/Documentation/devicetree/bindings/arm/sunxi/allwinner,sun4i-a10-mbus.yaml b/Documentation/devicetree/bindings/arm/sunxi/allwinner,sun4i-a10-mbus.yaml
index 29c9961..8eee312 100644
--- a/Documentation/devicetree/bindings/arm/sunxi/allwinner,sun4i-a10-mbus.yaml
+++ b/Documentation/devicetree/bindings/arm/sunxi/allwinner,sun4i-a10-mbus.yaml
@@ -32,12 +32,38 @@
- allwinner,sun8i-h3-mbus
- allwinner,sun8i-r40-mbus
- allwinner,sun50i-a64-mbus
+ - allwinner,sun50i-h5-mbus
reg:
- maxItems: 1
+ minItems: 1
+ items:
+ - description: MBUS interconnect/bandwidth limit/PMU registers
+ - description: DRAM controller/PHY registers
+
+ reg-names:
+ minItems: 1
+ items:
+ - const: mbus
+ - const: dram
clocks:
+ minItems: 1
+ items:
+ - description: MBUS interconnect module clock
+ - description: DRAM controller/PHY module clock
+ - description: Register bus clock, shared by MBUS and DRAM
+
+ clock-names:
+ minItems: 1
+ items:
+ - const: mbus
+ - const: dram
+ - const: bus
+
+ interrupts:
maxItems: 1
+ description:
+ MBUS PMU activity interrupt.
dma-ranges:
description:
@@ -54,13 +80,55 @@
- clocks
- dma-ranges
+if:
+ properties:
+ compatible:
+ contains:
+ enum:
+ - allwinner,sun8i-h3-mbus
+ - allwinner,sun50i-a64-mbus
+ - allwinner,sun50i-h5-mbus
+
+then:
+ properties:
+ reg:
+ minItems: 2
+
+ reg-names:
+ minItems: 2
+
+ clocks:
+ minItems: 3
+
+ clock-names:
+ minItems: 3
+
+ required:
+ - reg-names
+ - clock-names
+
+else:
+ properties:
+ reg:
+ maxItems: 1
+
+ reg-names:
+ maxItems: 1
+
+ clocks:
+ maxItems: 1
+
+ clock-names:
+ maxItems: 1
+
additionalProperties: false
examples:
- |
- #include <dt-bindings/clock/sun5i-ccu.h>
+ #include <dt-bindings/clock/sun50i-a64-ccu.h>
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
- mbus: dram-controller@1c01000 {
+ dram-controller@1c01000 {
compatible = "allwinner,sun5i-a13-mbus";
reg = <0x01c01000 0x1000>;
clocks = <&ccu CLK_MBUS>;
@@ -70,4 +138,21 @@
#interconnect-cells = <1>;
};
+ - |
+ dram-controller@1c62000 {
+ compatible = "allwinner,sun50i-a64-mbus";
+ reg = <0x01c62000 0x1000>,
+ <0x01c63000 0x1000>;
+ reg-names = "mbus", "dram";
+ clocks = <&ccu CLK_MBUS>,
+ <&ccu CLK_DRAM>,
+ <&ccu CLK_BUS_DRAM>;
+ clock-names = "mbus", "dram", "bus";
+ interrupts = <GIC_SPI 69 IRQ_TYPE_LEVEL_HIGH>;
+ #address-cells = <1>;
+ #size-cells = <1>;
+ dma-ranges = <0x00000000 0x40000000 0xc0000000>;
+ #interconnect-cells = <1>;
+ };
+
...
diff --git a/Documentation/devicetree/bindings/arm/tegra.yaml b/Documentation/devicetree/bindings/arm/tegra.yaml
index d79d36a..49841ca 100644
--- a/Documentation/devicetree/bindings/arm/tegra.yaml
+++ b/Documentation/devicetree/bindings/arm/tegra.yaml
@@ -37,6 +37,9 @@
- const: toradex,colibri_t20
- const: nvidia,tegra20
- items:
+ - const: asus,tf101
+ - const: nvidia,tegra20
+ - items:
- const: acer,picasso
- const: nvidia,tegra20
- items:
@@ -50,6 +53,18 @@
- const: nvidia,cardhu
- const: nvidia,tegra30
- items:
+ - const: asus,tf201
+ - const: nvidia,tegra30
+ - items:
+ - const: asus,tf300t
+ - const: nvidia,tegra30
+ - items:
+ - const: asus,tf300tg
+ - const: nvidia,tegra30
+ - items:
+ - const: asus,tf700t
+ - const: nvidia,tegra30
+ - items:
- const: toradex,apalis_t30-eval
- const: toradex,apalis_t30
- const: nvidia,tegra30
@@ -75,7 +90,11 @@
- const: ouya,ouya
- const: nvidia,tegra30
- items:
+ - const: pegatron,chagall
+ - const: nvidia,tegra30
+ - items:
- enum:
+ - asus,tf701t
- nvidia,dalmore
- nvidia,roth
- nvidia,tn7
@@ -108,14 +127,17 @@
- nvidia,p2571
- nvidia,p2894-0050-a08
- const: nvidia,tegra210
- - items:
- - enum:
- - nvidia,p2771-0000
- - nvidia,p3509-0000+p3636-0001
+ - description: Jetson TX2 Developer Kit
+ items:
+ - const: nvidia,p2771-0000
- const: nvidia,tegra186
- - items:
- - enum:
- - nvidia,p2972-0000
+ - description: Jetson TX2 NX Developer Kit
+ items:
+ - const: nvidia,p3509-0000+p3636-0001
+ - const: nvidia,tegra186
+ - description: Jetson AGX Xavier Developer Kit
+ items:
+ - const: nvidia,p2972-0000
- const: nvidia,tegra194
- description: Jetson Xavier NX
items:
@@ -134,8 +156,16 @@
- const: nvidia,p3509-0000+p3668-0001
- const: nvidia,tegra194
- items:
- - enum:
- - nvidia,tegra234-vdk
+ - const: nvidia,tegra234-vdk
+ - const: nvidia,tegra234
+ - description: Jetson AGX Orin
+ items:
+ - const: nvidia,p3701-0000
+ - const: nvidia,tegra234
+ - description: Jetson AGX Orin Developer Kit
+ items:
+ - const: nvidia,p3737-0000+p3701-0000
+ - const: nvidia,p3701-0000
- const: nvidia,tegra234
additionalProperties: true
diff --git a/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra186-pmc.txt b/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra186-pmc.txt
deleted file mode 100644
index 576462f..0000000
--- a/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra186-pmc.txt
+++ /dev/null
@@ -1,133 +0,0 @@
-NVIDIA Tegra Power Management Controller (PMC)
-
-Required properties:
-- compatible: Should contain one of the following:
- - "nvidia,tegra186-pmc": for Tegra186
- - "nvidia,tegra194-pmc": for Tegra194
- - "nvidia,tegra234-pmc": for Tegra234
-- reg: Must contain an (offset, length) pair of the register set for each
- entry in reg-names.
-- reg-names: Must include the following entries:
- - "pmc"
- - "wake"
- - "aotag"
- - "scratch"
- - "misc" (Only for Tegra194 and later)
-
-Optional properties:
-- nvidia,invert-interrupt: If present, inverts the PMU interrupt signal.
-- interrupt-controller: Identifies the node as an interrupt controller.
-- #interrupt-cells: Specifies the number of cells needed to encode an
- interrupt source. The value must be 2.
-
-Example:
-
-SoC DTSI:
-
- pmc@c3600000 {
- compatible = "nvidia,tegra186-pmc";
- reg = <0 0x0c360000 0 0x10000>,
- <0 0x0c370000 0 0x10000>,
- <0 0x0c380000 0 0x10000>,
- <0 0x0c390000 0 0x10000>;
- reg-names = "pmc", "wake", "aotag", "scratch";
- };
-
-Board DTS:
-
- pmc@c360000 {
- nvidia,invert-interrupt;
- };
-
-== Pad Control ==
-
-On Tegra SoCs a pad is a set of pins which are configured as a group.
-The pin grouping is a fixed attribute of the hardware. The PMC can be
-used to set pad power state and signaling voltage. A pad can be either
-in active or power down mode. The support for power state and signaling
-voltage configuration varies depending on the pad in question. 3.3 V and
-1.8 V signaling voltages are supported on pins where software
-controllable signaling voltage switching is available.
-
-Pad configurations are described with pin configuration nodes which
-are placed under the pmc node and they are referred to by the pinctrl
-client properties. For more information see
-Documentation/devicetree/bindings/pinctrl/pinctrl-bindings.txt.
-
-The following pads are present on Tegra186:
-csia csib dsi mipi-bias
-pex-clk-bias pex-clk3 pex-clk2 pex-clk1
-usb0 usb1 usb2 usb-bias
-uart audio hsic dbg
-hdmi-dp0 hdmi-dp1 pex-cntrl sdmmc2-hv
-sdmmc4 cam dsib dsic
-dsid csic csid csie
-dsif spi ufs dmic-hv
-edp sdmmc1-hv sdmmc3-hv conn
-audio-hv ao-hv
-
-Required pin configuration properties:
- - pins: A list of strings, each of which contains the name of a pad
- to be configured.
-
-Optional pin configuration properties:
- - low-power-enable: Configure the pad into power down mode
- - low-power-disable: Configure the pad into active mode
- - power-source: Must contain either TEGRA_IO_PAD_VOLTAGE_1V8 or
- TEGRA_IO_PAD_VOLTAGE_3V3 to select between signaling voltages.
- The values are defined in
- include/dt-bindings/pinctrl/pinctrl-tegra-io-pad.h.
-
-Note: The power state can be configured on all of the above pads except
- for ao-hv. Following pads have software configurable signaling
- voltages: sdmmc2-hv, dmic-hv, sdmmc1-hv, sdmmc3-hv, audio-hv,
- ao-hv.
-
-Pad configuration state example:
- pmc: pmc@7000e400 {
- compatible = "nvidia,tegra186-pmc";
- reg = <0 0x0c360000 0 0x10000>,
- <0 0x0c370000 0 0x10000>,
- <0 0x0c380000 0 0x10000>,
- <0 0x0c390000 0 0x10000>;
- reg-names = "pmc", "wake", "aotag", "scratch";
-
- ...
-
- sdmmc1_3v3: sdmmc1-3v3 {
- pins = "sdmmc1-hv";
- power-source = <TEGRA_IO_PAD_VOLTAGE_3V3>;
- };
-
- sdmmc1_1v8: sdmmc1-1v8 {
- pins = "sdmmc1-hv";
- power-source = <TEGRA_IO_PAD_VOLTAGE_1V8>;
- };
-
- hdmi_off: hdmi-off {
- pins = "hdmi";
- low-power-enable;
- }
-
- hdmi_on: hdmi-on {
- pins = "hdmi";
- low-power-disable;
- }
- };
-
-Pinctrl client example:
- sdmmc1: sdhci@3400000 {
- ...
- pinctrl-names = "sdmmc-3v3", "sdmmc-1v8";
- pinctrl-0 = <&sdmmc1_3v3>;
- pinctrl-1 = <&sdmmc1_1v8>;
- };
-
- ...
-
- sor0: sor@15540000 {
- ...
- pinctrl-0 = <&hdmi_off>;
- pinctrl-1 = <&hdmi_on>;
- pinctrl-names = "hdmi-on", "hdmi-off";
- };
diff --git a/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra186-pmc.yaml b/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra186-pmc.yaml
new file mode 100644
index 0000000..0faa403
--- /dev/null
+++ b/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra186-pmc.yaml
@@ -0,0 +1,198 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/arm/tegra/nvidia,tegra186-pmc.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: NVIDIA Tegra Power Management Controller (PMC)
+
+maintainers:
+ - Thierry Reding <thierry.reding@gmail.com>
+ - Jon Hunter <jonathanh@nvidia.com>
+
+properties:
+ compatible:
+ enum:
+ - nvidia,tegra186-pmc
+ - nvidia,tegra194-pmc
+ - nvidia,tegra234-pmc
+
+ reg:
+ minItems: 4
+ maxItems: 5
+
+ reg-names:
+ minItems: 4
+ items:
+ - const: pmc
+ - const: wake
+ - const: aotag
+ - const: scratch
+ - const: misc
+
+ interrupt-controller: true
+
+ "#interrupt-cells":
+ description: Specifies the number of cells needed to encode an
+ interrupt source. The value must be 2.
+ const: 2
+
+ nvidia,invert-interrupt:
+ description: If present, inverts the PMU interrupt signal.
+ $ref: /schemas/types.yaml#/definitions/flag
+
+if:
+ properties:
+ compatible:
+ contains:
+ const: nvidia,tegra186-pmc
+then:
+ properties:
+ reg:
+ maxItems: 4
+
+ reg-names:
+ maxItems: 4
+else:
+ properties:
+ reg:
+ minItems: 5
+
+ reg-names:
+ minItems: 5
+
+patternProperties:
+ "^[a-z0-9]+-[a-z0-9]+$":
+ if:
+ type: object
+ then:
+ description: |
+ These are pad configuration nodes. On Tegra SoCs a pad is a set of
+ pins which are configured as a group. The pin grouping is a fixed
+ attribute of the hardware. The PMC can be used to set pad power
+ state and signaling voltage. A pad can be either in active or
+ power down mode. The support for power state and signaling voltage
+ configuration varies depending on the pad in question. 3.3 V and
+ 1.8 V signaling voltages are supported on pins where software
+ controllable signaling voltage switching is available.
+
+ Pad configurations are described with pin configuration nodes
+ which are placed under the pmc node and they are referred to by
+ the pinctrl client properties. For more information see
+
+ Documentation/devicetree/bindings/pinctrl/pinctrl-bindings.txt
+
+ The following pads are present on Tegra186:
+
+ csia, csib, dsi, mipi-bias, pex-clk-bias, pex-clk3, pex-clk2,
+ pex-clk1, usb0, usb1, usb2, usb-bias, uart, audio, hsic, dbg,
+ hdmi-dp0, hdmi-dp1, pex-cntrl, sdmmc2-hv, sdmmc4, cam, dsib,
+ dsic, dsid, csic, csid, csie, dsif, spi, ufs, dmic-hv, edp,
+ sdmmc1-hv, sdmmc3-hv, conn, audio-hv, ao-hv
+
+ The following pads are present on Tegra194:
+
+ csia, csib, mipi-bias, pex-clk-bias, pex-clk3, pex-clk2,
+ pex-clk1, eqos, pex-clk-2-bias, pex-clk-2, dap3, dap5, uart,
+ pwr-ctl, soc-gpio53, audio, gp-pwm2, gp-pwm3, soc-gpio12,
+ soc-gpio13, soc-gpio10, uart4, uart5, dbg, hdmi-dp3, hdmi-dp2,
+ hdmi-dp0, hdmi-dp1, pex-cntrl, pex-ctl2, pex-l0-rst,
+ pex-l1-rst, sdmmc4, pex-l5-rst, cam, csic, csid, csie, csif,
+ spi, ufs, csig, csih, edp, sdmmc1-hv, sdmmc3-hv, conn,
+ audio-hv, ao-hv
+
+ properties:
+ pins:
+ $ref: /schemas/types.yaml#/definitions/string
+ description: Must contain the name of the pad(s) to be
+ configured.
+
+ low-power-enable:
+ description: Configure the pad into power down mode.
+ $ref: /schemas/types.yaml#/definitions/flag
+
+ low-power-disable:
+ description: Configure the pad into active mode.
+ $ref: /schemas/types.yaml#/definitions/flag
+
+ power-source:
+ $ref: /schemas/types.yaml#/definitions/uint32
+ description: |
+ Must contain either TEGRA_IO_PAD_VOLTAGE_1V8 or
+ TEGRA_IO_PAD_VOLTAGE_3V3 to select between signalling
+ voltages.
+
+ The values are defined in
+
+ include/dt-bindings/pinctrl/pinctrl-tegra-io-pad.h
+
+ The power state can be configured on all of the above pads
+ except for ao-hv. Following pads have software configurable
+ signaling voltages: sdmmc2-hv, dmic-hv, sdmmc1-hv, sdmmc3-hv,
+ audio-hv, ao-hv.
+
+ phandle: true
+
+ required:
+ - pins
+
+ additionalProperties: false
+
+required:
+ - compatible
+ - reg
+ - reg-names
+
+additionalProperties: false
+
+dependencies:
+ interrupt-controller: ['#interrupt-cells']
+ "#interrupt-cells":
+ required:
+ - interrupt-controller
+
+examples:
+ - |
+ #include <dt-bindings/clock/tegra186-clock.h>
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+ #include <dt-bindings/pinctrl/pinctrl-tegra-io-pad.h>
+ #include <dt-bindings/memory/tegra186-mc.h>
+ #include <dt-bindings/reset/tegra186-reset.h>
+
+ pmc@c3600000 {
+ compatible = "nvidia,tegra186-pmc";
+ reg = <0x0c360000 0x10000>,
+ <0x0c370000 0x10000>,
+ <0x0c380000 0x10000>,
+ <0x0c390000 0x10000>;
+ reg-names = "pmc", "wake", "aotag", "scratch";
+ nvidia,invert-interrupt;
+
+ sdmmc1_3v3: sdmmc1-3v3 {
+ pins = "sdmmc1-hv";
+ power-source = <TEGRA_IO_PAD_VOLTAGE_3V3>;
+ };
+
+ sdmmc1_1v8: sdmmc1-1v8 {
+ pins = "sdmmc1-hv";
+ power-source = <TEGRA_IO_PAD_VOLTAGE_1V8>;
+ };
+ };
+
+ sdmmc1: mmc@3400000 {
+ compatible = "nvidia,tegra186-sdhci";
+ reg = <0x03400000 0x10000>;
+ interrupts = <GIC_SPI 62 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&bpmp TEGRA186_CLK_SDMMC1>,
+ <&bpmp TEGRA186_CLK_SDMMC_LEGACY_TM>;
+ clock-names = "sdhci", "tmclk";
+ resets = <&bpmp TEGRA186_RESET_SDMMC1>;
+ reset-names = "sdhci";
+ interconnects = <&mc TEGRA186_MEMORY_CLIENT_SDMMCRA &emc>,
+ <&mc TEGRA186_MEMORY_CLIENT_SDMMCWA &emc>;
+ interconnect-names = "dma-mem", "write";
+ iommus = <&smmu TEGRA186_SID_SDMMC1>;
+ pinctrl-names = "sdmmc-3v3", "sdmmc-1v8";
+ pinctrl-0 = <&sdmmc1_3v3>;
+ pinctrl-1 = <&sdmmc1_1v8>;
+ };
diff --git a/Documentation/devicetree/bindings/arm/ti/k3.yaml b/Documentation/devicetree/bindings/arm/ti/k3.yaml
index cf32723..b03c10f 100644
--- a/Documentation/devicetree/bindings/arm/ti/k3.yaml
+++ b/Documentation/devicetree/bindings/arm/ti/k3.yaml
@@ -53,6 +53,12 @@
- ti,am642-sk
- const: ti,am642
+ - description: K3 J721s2 SoC
+ items:
+ - enum:
+ - ti,j721s2-evm
+ - const: ti,j721s2
+
additionalProperties: true
...
diff --git a/Documentation/devicetree/bindings/bus/fsl,spba-bus.yaml b/Documentation/devicetree/bindings/bus/fsl,spba-bus.yaml
new file mode 100644
index 0000000..d42dbb0
--- /dev/null
+++ b/Documentation/devicetree/bindings/bus/fsl,spba-bus.yaml
@@ -0,0 +1,68 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/bus/fsl,spba-bus.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Shared Peripherals Bus Interface
+
+maintainers:
+ - Shawn Guo <shawnguo@kernel.org>
+
+description: |
+ A simple bus enabling access to shared peripherals.
+
+ The "spba-bus" follows the "simple-bus" set of properties, as
+ specified in the Devicetree Specification. It is an extension of
+ "simple-bus" because the SDMA controller uses this compatible flag to
+ determine which peripherals are available to it and the range over which
+ the SDMA can access. There are no special clocks for the bus, because
+ the SDMA controller itself has its interrupt and clock assignments.
+
+select:
+ properties:
+ compatible:
+ contains:
+ const: fsl,spba-bus
+ required:
+ - compatible
+
+properties:
+ $nodename:
+ pattern: "^spba-bus(@[0-9a-f]+)?$"
+
+ compatible:
+ items:
+ - const: fsl,spba-bus
+ - const: simple-bus
+
+ '#address-cells':
+ enum: [ 1, 2 ]
+
+ '#size-cells':
+ enum: [ 1, 2 ]
+
+ reg:
+ maxItems: 1
+
+ ranges: true
+
+required:
+ - compatible
+ - '#address-cells'
+ - '#size-cells'
+ - reg
+ - ranges
+
+additionalProperties:
+ type: object
+
+examples:
+ - |
+ spba-bus@30000000 {
+ compatible = "fsl,spba-bus", "simple-bus";
+ #address-cells = <1>;
+ #size-cells = <1>;
+ reg = <0x30000000 0x100000>;
+ ranges;
+ };
diff --git a/Documentation/devicetree/bindings/bus/imx-weim.txt b/Documentation/devicetree/bindings/bus/imx-weim.txt
index 1b1d1c5..e7f5020 100644
--- a/Documentation/devicetree/bindings/bus/imx-weim.txt
+++ b/Documentation/devicetree/bindings/bus/imx-weim.txt
@@ -48,6 +48,11 @@
devices, the presence of this property indicates that
the weim bus should operate in Burst Clock Mode.
+ - fsl,continuous-burst-clk Make Burst Clock to output continuous clock.
+ Without this option Burst Clock will output clock
+ only when necessary. This takes effect only if
+ "fsl,burst-clk-enable" is set.
+
Timing property for child nodes. It is mandatory, not optional.
- fsl,weim-cs-timing: The timing array, contains timing values for the
diff --git a/Documentation/devicetree/bindings/clock/nvidia,tegra20-car.yaml b/Documentation/devicetree/bindings/clock/nvidia,tegra20-car.yaml
index 459d2a52..f832abb 100644
--- a/Documentation/devicetree/bindings/clock/nvidia,tegra20-car.yaml
+++ b/Documentation/devicetree/bindings/clock/nvidia,tegra20-car.yaml
@@ -42,6 +42,36 @@
"#reset-cells":
const: 1
+patternProperties:
+ "^(sclk)|(pll-[cem])$":
+ type: object
+ properties:
+ compatible:
+ enum:
+ - nvidia,tegra20-sclk
+ - nvidia,tegra30-sclk
+ - nvidia,tegra30-pllc
+ - nvidia,tegra30-plle
+ - nvidia,tegra30-pllm
+
+ operating-points-v2: true
+
+ clocks:
+ items:
+ - description: node's clock
+
+ power-domains:
+ maxItems: 1
+ description: phandle to the core SoC power domain
+
+ required:
+ - compatible
+ - operating-points-v2
+ - clocks
+ - power-domains
+
+ additionalProperties: false
+
required:
- compatible
- reg
@@ -59,6 +89,13 @@
reg = <0x60006000 0x1000>;
#clock-cells = <1>;
#reset-cells = <1>;
+
+ sclk {
+ compatible = "nvidia,tegra20-sclk";
+ operating-points-v2 = <&opp_table>;
+ clocks = <&tegra_car TEGRA20_CLK_SCLK>;
+ power-domains = <&domain>;
+ };
};
usb-controller@c5004000 {
diff --git a/Documentation/devicetree/bindings/clock/qcom,gcc-sdx65.yaml b/Documentation/devicetree/bindings/clock/qcom,gcc-sdx65.yaml
new file mode 100644
index 0000000..16c4cdc
--- /dev/null
+++ b/Documentation/devicetree/bindings/clock/qcom,gcc-sdx65.yaml
@@ -0,0 +1,80 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/clock/qcom,gcc-sdx65.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Qualcomm Global Clock & Reset Controller Binding for SDX65
+
+maintainers:
+ - Vamsi krishna Lanka <quic_vamslank@quicinc.com>
+
+description: |
+ Qualcomm global clock control module which supports the clocks, resets and
+ power domains on SDX65
+
+ See also:
+ - dt-bindings/clock/qcom,gcc-sdx65.h
+
+properties:
+ compatible:
+ const: qcom,gcc-sdx65
+
+ reg:
+ maxItems: 1
+
+ clocks:
+ items:
+ - description: Board XO source
+ - description: Board active XO source
+ - description: Sleep clock source
+ - description: PCIE Pipe clock source
+ - description: USB3 phy wrapper pipe clock source
+ - description: PLL test clock source (Optional clock)
+ minItems: 5
+
+ clock-names:
+ items:
+ - const: bi_tcxo
+ - const: bi_tcxo_ao
+ - const: sleep_clk
+ - const: pcie_pipe_clk
+ - const: usb3_phy_wrapper_gcc_usb30_pipe_clk
+ - const: core_bi_pll_test_se # Optional clock
+ minItems: 5
+
+ '#clock-cells':
+ const: 1
+
+ '#reset-cells':
+ const: 1
+
+ '#power-domain-cells':
+ const: 1
+
+required:
+ - compatible
+ - reg
+ - clocks
+ - clock-names
+ - '#clock-cells'
+ - '#reset-cells'
+ - '#power-domain-cells'
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/clock/qcom,rpmh.h>
+ clock-controller@100000 {
+ compatible = "qcom,gcc-sdx65";
+ reg = <0x100000 0x1f7400>;
+ clocks = <&rpmhcc RPMH_CXO_CLK>, <&rpmhcc RPMH_CXO_CLK_A>, <&sleep_clk>,
+ <&pcie_pipe_clk>, <&usb3_phy_wrapper_gcc_usb30_pipe_clk>, <&pll_test_clk>;
+ clock-names = "bi_tcxo", "bi_tcxo_ao", "sleep_clk",
+ "pcie_pipe_clk", "usb3_phy_wrapper_gcc_usb30_pipe_clk", "core_bi_pll_test_se";
+ #clock-cells = <1>;
+ #reset-cells = <1>;
+ #power-domain-cells = <1>;
+ };
+...
diff --git a/Documentation/devicetree/bindings/clock/qcom,gcc-sm8450.yaml b/Documentation/devicetree/bindings/clock/qcom,gcc-sm8450.yaml
new file mode 100644
index 0000000..58d98a7
--- /dev/null
+++ b/Documentation/devicetree/bindings/clock/qcom,gcc-sm8450.yaml
@@ -0,0 +1,85 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/clock/qcom,gcc-sm8450.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Qualcomm Global Clock & Reset Controller Binding for SM8450
+
+maintainers:
+ - Vinod Koul <vkoul@kernel.org>
+
+description: |
+ Qualcomm global clock control module which supports the clocks, resets and
+ power domains on SM8450
+
+ See also:
+ - dt-bindings/clock/qcom,gcc-sm8450.h
+
+properties:
+ compatible:
+ const: qcom,gcc-sm8450
+
+ clocks:
+ items:
+ - description: Board XO source
+ - description: Sleep clock source
+ - description: PCIE 0 Pipe clock source (Optional clock)
+ - description: PCIE 1 Pipe clock source (Optional clock)
+ - description: PCIE 1 Phy Auxillary clock source (Optional clock)
+ - description: UFS Phy Rx symbol 0 clock source (Optional clock)
+ - description: UFS Phy Rx symbol 1 clock source (Optional clock)
+ - description: UFS Phy Tx symbol 0 clock source (Optional clock)
+ - description: USB3 Phy wrapper pipe clock source (Optional clock)
+ minItems: 2
+
+ clock-names:
+ items:
+ - const: bi_tcxo
+ - const: sleep_clk
+ - const: pcie_0_pipe_clk # Optional clock
+ - const: pcie_1_pipe_clk # Optional clock
+ - const: pcie_1_phy_aux_clk # Optional clock
+ - const: ufs_phy_rx_symbol_0_clk # Optional clock
+ - const: ufs_phy_rx_symbol_1_clk # Optional clock
+ - const: ufs_phy_tx_symbol_0_clk # Optional clock
+ - const: usb3_phy_wrapper_gcc_usb30_pipe_clk # Optional clock
+ minItems: 2
+
+ '#clock-cells':
+ const: 1
+
+ '#reset-cells':
+ const: 1
+
+ '#power-domain-cells':
+ const: 1
+
+ reg:
+ maxItems: 1
+
+required:
+ - compatible
+ - reg
+ - clocks
+ - clock-names
+ - '#clock-cells'
+ - '#reset-cells'
+ - '#power-domain-cells'
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/clock/qcom,rpmh.h>
+ clock-controller@100000 {
+ compatible = "qcom,gcc-sm8450";
+ reg = <0x00100000 0x001f4200>;
+ clocks = <&rpmhcc RPMH_CXO_CLK>, <&sleep_clk>;
+ clock-names = "bi_tcxo", "sleep_clk";
+ #clock-cells = <1>;
+ #reset-cells = <1>;
+ #power-domain-cells = <1>;
+ };
+
+...
diff --git a/Documentation/devicetree/bindings/clock/starfive,jh7100-clkgen.yaml b/Documentation/devicetree/bindings/clock/starfive,jh7100-clkgen.yaml
new file mode 100644
index 0000000..12f17b6
--- /dev/null
+++ b/Documentation/devicetree/bindings/clock/starfive,jh7100-clkgen.yaml
@@ -0,0 +1,56 @@
+# SPDX-License-Identifier: GPL-2.0-only OR BSD-2-Clause
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/clock/starfive,jh7100-clkgen.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: StarFive JH7100 Clock Generator
+
+maintainers:
+ - Geert Uytterhoeven <geert@linux-m68k.org>
+ - Emil Renner Berthing <kernel@esmil.dk>
+
+properties:
+ compatible:
+ const: starfive,jh7100-clkgen
+
+ reg:
+ maxItems: 1
+
+ clocks:
+ items:
+ - description: Main clock source (25 MHz)
+ - description: Application-specific clock source (12-27 MHz)
+ - description: RMII reference clock (50 MHz)
+ - description: RGMII RX clock (125 MHz)
+
+ clock-names:
+ items:
+ - const: osc_sys
+ - const: osc_aud
+ - const: gmac_rmii_ref
+ - const: gmac_gr_mii_rxclk
+
+ '#clock-cells':
+ const: 1
+ description:
+ See <dt-bindings/clock/starfive-jh7100.h> for valid indices.
+
+required:
+ - compatible
+ - reg
+ - clocks
+ - clock-names
+ - '#clock-cells'
+
+additionalProperties: false
+
+examples:
+ - |
+ clock-controller@11800000 {
+ compatible = "starfive,jh7100-clkgen";
+ reg = <0x11800000 0x10000>;
+ clocks = <&osc_sys>, <&osc_aud>, <&gmac_rmii_ref>, <&gmac_gr_mii_rxclk>;
+ clock-names = "osc_sys", "osc_aud", "gmac_rmii_ref", "gmac_gr_mii_rxclk";
+ #clock-cells = <1>;
+ };
diff --git a/Documentation/devicetree/bindings/crypto/allwinner,sun4i-a10-crypto.yaml b/Documentation/devicetree/bindings/crypto/allwinner,sun4i-a10-crypto.yaml
index 0429fb7..dedc99e 100644
--- a/Documentation/devicetree/bindings/crypto/allwinner,sun4i-a10-crypto.yaml
+++ b/Documentation/devicetree/bindings/crypto/allwinner,sun4i-a10-crypto.yaml
@@ -44,6 +44,16 @@
- const: ahb
- const: mod
+ dmas:
+ items:
+ - description: RX DMA Channel
+ - description: TX DMA Channel
+
+ dma-names:
+ items:
+ - const: rx
+ - const: tx
+
resets:
maxItems: 1
diff --git a/Documentation/devicetree/bindings/display/bridge/analogix,anx7625.yaml b/Documentation/devicetree/bindings/display/bridge/analogix,anx7625.yaml
index ab48ab2f..1d3e88d 100644
--- a/Documentation/devicetree/bindings/display/bridge/analogix,anx7625.yaml
+++ b/Documentation/devicetree/bindings/display/bridge/analogix,anx7625.yaml
@@ -43,14 +43,70 @@
vdd33-supply:
description: Regulator that provides the supply 3.3V power.
+ analogix,lane0-swing:
+ $ref: /schemas/types.yaml#/definitions/uint8-array
+ minItems: 1
+ maxItems: 20
+ description:
+ an array of swing register setting for DP tx lane0 PHY.
+ Registers 0~9 are Swing0_Pre0, Swing1_Pre0, Swing2_Pre0,
+ Swing3_Pre0, Swing0_Pre1, Swing1_Pre1, Swing2_Pre1, Swing0_Pre2,
+ Swing1_Pre2, Swing0_Pre3, they are for [Boost control] and
+ [Swing control] setting.
+ Registers 0~9, bit 3:0 is [Boost control], these bits control
+ post cursor manual, increase the [Boost control] to increase
+ Pre-emphasis value.
+ Registers 0~9, bit 6:4 is [Swing control], these bits control
+ swing manual, increase [Swing control] setting to add Vp-p value
+ for each Swing, Pre.
+ Registers 10~19 are Swing0_Pre0, Swing1_Pre0, Swing2_Pre0,
+ Swing3_Pre0, Swing0_Pre1, Swing1_Pre1, Swing2_Pre1, Swing0_Pre2,
+ Swing1_Pre2, Swing0_Pre3, they are for [R select control] and
+ [R Termination control] setting.
+ Registers 10~19, bit 4:0 is [R select control], these bits are
+ compensation manual, increase it can enhance IO driven strength
+ and Vp-p.
+ Registers 10~19, bit 5:6 is [R termination control], these bits
+ adjust 50ohm impedance of DP tx termination. 00:55 ohm,
+ 01:50 ohm(default), 10:45 ohm, 11:40 ohm.
+
+ analogix,lane1-swing:
+ $ref: /schemas/types.yaml#/definitions/uint8-array
+ minItems: 1
+ maxItems: 20
+ description:
+ an array of swing register setting for DP tx lane1 PHY.
+ DP TX lane1 swing register setting same with lane0
+ swing, please refer lane0-swing property description.
+
+ analogix,audio-enable:
+ type: boolean
+ description: let the driver enable audio HDMI codec function or not.
+
ports:
$ref: /schemas/graph.yaml#/properties/ports
properties:
port@0:
- $ref: /schemas/graph.yaml#/properties/port
+ $ref: /schemas/graph.yaml#/$defs/port-base
+ unevaluatedProperties: false
description:
- Video port for MIPI DSI input.
+ MIPI DSI/DPI input.
+
+ properties:
+ endpoint:
+ $ref: /schemas/media/video-interfaces.yaml#
+ type: object
+ additionalProperties: false
+
+ properties:
+ remote-endpoint: true
+
+ bus-type:
+ enum: [1, 5]
+ default: 1
+
+ data-lanes: true
port@1:
$ref: /schemas/graph.yaml#/properties/port
@@ -87,6 +143,9 @@
vdd10-supply = <&pp1000_mipibrdg>;
vdd18-supply = <&pp1800_mipibrdg>;
vdd33-supply = <&pp3300_mipibrdg>;
+ analogix,audio-enable;
+ analogix,lane0-swing = /bits/ 8 <0x14 0x54 0x64 0x74>;
+ analogix,lane1-swing = /bits/ 8 <0x14 0x54 0x64 0x74>;
ports {
#address-cells = <1>;
@@ -96,6 +155,8 @@
reg = <0>;
anx7625_in: endpoint {
remote-endpoint = <&mipi_dsi>;
+ bus-type = <5>;
+ data-lanes = <0 1 2 3>;
};
};
diff --git a/Documentation/devicetree/bindings/display/bridge/lvds-codec.yaml b/Documentation/devicetree/bindings/display/bridge/lvds-codec.yaml
index 1faae3e..5079c1c 100644
--- a/Documentation/devicetree/bindings/display/bridge/lvds-codec.yaml
+++ b/Documentation/devicetree/bindings/display/bridge/lvds-codec.yaml
@@ -79,6 +79,14 @@
- port@0
- port@1
+ pclk-sample:
+ description:
+ Data sampling on rising or falling edge.
+ enum:
+ - 0 # Falling edge
+ - 1 # Rising edge
+ default: 0
+
powerdown-gpios:
description:
The GPIO used to control the power down line of this device.
@@ -86,21 +94,32 @@
power-supply: true
-if:
- not:
- properties:
- compatible:
- contains:
- const: lvds-decoder
-then:
- properties:
- ports:
+allOf:
+ - if:
+ not:
+ properties:
+ compatible:
+ contains:
+ const: lvds-decoder
+ then:
properties:
- port@0:
+ ports:
properties:
- endpoint:
+ port@0:
properties:
- data-mapping: false
+ endpoint:
+ properties:
+ data-mapping: false
+
+ - if:
+ not:
+ properties:
+ compatible:
+ contains:
+ const: lvds-encoder
+ then:
+ properties:
+ pclk-sample: false
required:
- compatible
diff --git a/Documentation/devicetree/bindings/display/bridge/nxp,ptn3460.yaml b/Documentation/devicetree/bindings/display/bridge/nxp,ptn3460.yaml
new file mode 100644
index 0000000..107dd13
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/bridge/nxp,ptn3460.yaml
@@ -0,0 +1,106 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/display/bridge/nxp,ptn3460.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: NXP PTN3460 eDP to LVDS bridge
+
+maintainers:
+ - Sean Paul <seanpaul@chromium.org>
+
+properties:
+ compatible:
+ const: nxp,ptn3460
+
+ reg:
+ description: I2C address of the bridge
+ maxItems: 1
+
+ edid-emulation:
+ $ref: "/schemas/types.yaml#/definitions/uint32"
+ description:
+ The EDID emulation entry to use
+ Value Resolution Description
+ 0 1024x768 NXP Generic
+ 1 1920x1080 NXP Generic
+ 2 1920x1080 NXP Generic
+ 3 1600x900 Samsung LTM200KT
+ 4 1920x1080 Samsung LTM230HT
+ 5 1366x768 NXP Generic
+ 6 1600x900 ChiMei M215HGE
+ enum: [0, 1, 2, 3, 4, 5, 6]
+
+ powerdown-gpios:
+ description: GPIO connected to the PD_N signal.
+ maxItems: 1
+
+ reset-gpios:
+ description: GPIO connected to the RST_N signal.
+ maxItems: 1
+
+ ports:
+ $ref: /schemas/graph.yaml#/properties/ports
+
+ properties:
+ port@0:
+ $ref: /schemas/graph.yaml#/properties/port
+ description:
+ Video port for LVDS output
+
+ port@1:
+ $ref: /schemas/graph.yaml#/properties/port
+ description:
+ Video port for eDP input
+
+ required:
+ - port@0
+ - port@1
+
+required:
+ - compatible
+ - reg
+ - edid-emulation
+ - powerdown-gpios
+ - reset-gpios
+ - ports
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/gpio/gpio.h>
+
+ i2c1 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ bridge@20 {
+ compatible = "nxp,ptn3460";
+ reg = <0x20>;
+ edid-emulation = <5>;
+ powerdown-gpios = <&gpy2 5 GPIO_ACTIVE_HIGH>;
+ reset-gpios = <&gpx1 5 GPIO_ACTIVE_LOW>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ bridge_out: endpoint {
+ remote-endpoint = <&panel_in>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+ bridge_in: endpoint {
+ remote-endpoint = <&dp_out>;
+ };
+ };
+ };
+ };
+ };
+
+...
diff --git a/Documentation/devicetree/bindings/display/bridge/ptn3460.txt b/Documentation/devicetree/bindings/display/bridge/ptn3460.txt
deleted file mode 100644
index 361971b..0000000
--- a/Documentation/devicetree/bindings/display/bridge/ptn3460.txt
+++ /dev/null
@@ -1,39 +0,0 @@
-ptn3460 bridge bindings
-
-Required properties:
- - compatible: "nxp,ptn3460"
- - reg: i2c address of the bridge
- - powerdown-gpio: OF device-tree gpio specification for PD_N pin.
- - reset-gpio: OF device-tree gpio specification for RST_N pin.
- - edid-emulation: The EDID emulation entry to use
- +-------+------------+------------------+
- | Value | Resolution | Description |
- | 0 | 1024x768 | NXP Generic |
- | 1 | 1920x1080 | NXP Generic |
- | 2 | 1920x1080 | NXP Generic |
- | 3 | 1600x900 | Samsung LTM200KT |
- | 4 | 1920x1080 | Samsung LTM230HT |
- | 5 | 1366x768 | NXP Generic |
- | 6 | 1600x900 | ChiMei M215HGE |
- +-------+------------+------------------+
-
- - video interfaces: Device node can contain video interface port
- nodes for panel according to [1].
-
-[1]: Documentation/devicetree/bindings/media/video-interfaces.txt
-
-Example:
- lvds-bridge@20 {
- compatible = "nxp,ptn3460";
- reg = <0x20>;
- powerdown-gpio = <&gpy2 5 1 0 0>;
- reset-gpio = <&gpx1 5 1 0 0>;
- edid-emulation = <5>;
- ports {
- port@0 {
- bridge_out: endpoint {
- remote-endpoint = <&panel_in>;
- };
- };
- };
- };
diff --git a/Documentation/devicetree/bindings/display/bridge/renesas,dsi-csi2-tx.yaml b/Documentation/devicetree/bindings/display/bridge/renesas,dsi-csi2-tx.yaml
new file mode 100644
index 0000000..afeeb96
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/bridge/renesas,dsi-csi2-tx.yaml
@@ -0,0 +1,118 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/display/bridge/renesas,dsi-csi2-tx.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Renesas R-Car MIPI DSI/CSI-2 Encoder
+
+maintainers:
+ - Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
+
+description: |
+ This binding describes the MIPI DSI/CSI-2 encoder embedded in the Renesas
+ R-Car V3U SoC. The encoder can operate in either DSI or CSI-2 mode, with up
+ to four data lanes.
+
+properties:
+ compatible:
+ enum:
+ - renesas,r8a779a0-dsi-csi2-tx # for V3U
+
+ reg:
+ maxItems: 1
+
+ clocks:
+ items:
+ - description: Functional clock
+ - description: DSI (and CSI-2) functional clock
+ - description: PLL reference clock
+
+ clock-names:
+ items:
+ - const: fck
+ - const: dsi
+ - const: pll
+
+ power-domains:
+ maxItems: 1
+
+ resets:
+ maxItems: 1
+
+ ports:
+ $ref: /schemas/graph.yaml#/properties/ports
+
+ properties:
+ port@0:
+ $ref: /schemas/graph.yaml#/properties/port
+ description: Parallel input port
+
+ port@1:
+ $ref: /schemas/graph.yaml#/$defs/port-base
+ unevaluatedProperties: false
+ description: DSI/CSI-2 output port
+
+ properties:
+ endpoint:
+ $ref: /schemas/media/video-interfaces.yaml#
+ unevaluatedProperties: false
+
+ properties:
+ data-lanes:
+ minItems: 1
+ maxItems: 4
+
+ required:
+ - data-lanes
+
+ required:
+ - port@0
+ - port@1
+
+required:
+ - compatible
+ - reg
+ - clocks
+ - power-domains
+ - resets
+ - ports
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/clock/r8a779a0-cpg-mssr.h>
+ #include <dt-bindings/power/r8a779a0-sysc.h>
+
+ dsi0: dsi-encoder@fed80000 {
+ compatible = "renesas,r8a779a0-dsi-csi2-tx";
+ reg = <0xfed80000 0x10000>;
+ power-domains = <&sysc R8A779A0_PD_ALWAYS_ON>;
+ clocks = <&cpg CPG_MOD 415>,
+ <&cpg CPG_CORE R8A779A0_CLK_DSI>,
+ <&cpg CPG_CORE R8A779A0_CLK_CP>;
+ clock-names = "fck", "dsi", "pll";
+ resets = <&cpg 415>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ dsi0_in: endpoint {
+ remote-endpoint = <&du_out_dsi0>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+ dsi0_out: endpoint {
+ data-lanes = <1 2>;
+ remote-endpoint = <&sn65dsi86_in>;
+ };
+ };
+ };
+ };
+...
diff --git a/Documentation/devicetree/bindings/display/bridge/sii9234.txt b/Documentation/devicetree/bindings/display/bridge/sii9234.txt
deleted file mode 100644
index a55bf77..0000000
--- a/Documentation/devicetree/bindings/display/bridge/sii9234.txt
+++ /dev/null
@@ -1,49 +0,0 @@
-Silicon Image SiI9234 HDMI/MHL bridge bindings
-
-Required properties:
- - compatible : "sil,sii9234".
- - reg : I2C address for TPI interface, use 0x39
- - avcc33-supply : MHL/USB Switch Supply Voltage (3.3V)
- - iovcc18-supply : I/O Supply Voltage (1.8V)
- - avcc12-supply : TMDS Analog Supply Voltage (1.2V)
- - cvcc12-supply : Digital Core Supply Voltage (1.2V)
- - interrupts: interrupt specifier of INT pin
- - reset-gpios: gpio specifier of RESET pin (active low)
- - video interfaces: Device node can contain two video interface port
- nodes for HDMI encoder and connector according to [1].
- - port@0 - MHL to HDMI
- - port@1 - MHL to connector
-
-[1]: Documentation/devicetree/bindings/media/video-interfaces.txt
-
-
-Example:
- sii9234@39 {
- compatible = "sil,sii9234";
- reg = <0x39>;
- avcc33-supply = <&vcc33mhl>;
- iovcc18-supply = <&vcc18mhl>;
- avcc12-supply = <&vsil12>;
- cvcc12-supply = <&vsil12>;
- reset-gpios = <&gpf3 4 GPIO_ACTIVE_LOW>;
- interrupt-parent = <&gpf3>;
- interrupts = <5 IRQ_TYPE_LEVEL_HIGH>;
-
- ports {
- #address-cells = <1>;
- #size-cells = <0>;
-
- port@0 {
- reg = <0>;
- mhl_to_hdmi: endpoint {
- remote-endpoint = <&hdmi_to_mhl>;
- };
- };
- port@1 {
- reg = <1>;
- mhl_to_connector: endpoint {
- remote-endpoint = <&connector_to_mhl>;
- };
- };
- };
- };
diff --git a/Documentation/devicetree/bindings/display/bridge/sil,sii9234.yaml b/Documentation/devicetree/bindings/display/bridge/sil,sii9234.yaml
new file mode 100644
index 0000000..f88ddfe
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/bridge/sil,sii9234.yaml
@@ -0,0 +1,110 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/display/bridge/sil,sii9234.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Silicon Image SiI9234 HDMI/MHL bridge
+
+maintainers:
+ - Maciej Purski <m.purski@samsung.com>
+
+properties:
+ compatible:
+ const: sil,sii9234
+
+ reg:
+ description: I2C address for TPI interface
+ maxItems: 1
+
+ avcc12-supply:
+ description: TMDS Analog Supply Voltage, 1.2V
+
+ avcc33-supply:
+ description: MHL/USB Switch Supply Voltage, 3.3V
+
+ cvcc12-supply:
+ description: Digital Core Supply Voltage, 1.2V
+
+ iovcc18-supply:
+ description: I/O voltage supply, 1.8V
+
+ interrupts:
+ maxItems: 1
+
+ reset-gpios:
+ description: GPIO connected to the reset pin.
+ maxItems: 1
+
+ ports:
+ $ref: /schemas/graph.yaml#/properties/ports
+
+ properties:
+ port@0:
+ $ref: /schemas/graph.yaml#/properties/port
+ description:
+ Video port for HDMI (encoder) input
+
+ port@1:
+ $ref: /schemas/graph.yaml#/properties/port
+ description:
+ MHL to connector port
+
+ required:
+ - port@0
+
+required:
+ - compatible
+ - reg
+ - avcc12-supply
+ - avcc33-supply
+ - cvcc12-supply
+ - iovcc18-supply
+ - interrupts
+ - reset-gpios
+ - ports
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/gpio/gpio.h>
+ #include <dt-bindings/interrupt-controller/irq.h>
+
+ i2c1 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ bridge@39 {
+ compatible = "sil,sii9234";
+ reg = <0x39>;
+ avcc12-supply = <&vsil12>;
+ avcc33-supply = <&vcc33mhl>;
+ cvcc12-supply = <&vsil12>;
+ iovcc18-supply = <&vcc18mhl>;
+ interrupt-parent = <&gpf3>;
+ interrupts = <5 IRQ_TYPE_LEVEL_HIGH>;
+ reset-gpios = <&gpf3 4 GPIO_ACTIVE_LOW>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ mhl_to_hdmi: endpoint {
+ remote-endpoint = <&hdmi_to_mhl>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+ mhl_to_connector: endpoint {
+ remote-endpoint = <&connector_to_mhl>;
+ };
+ };
+ };
+ };
+ };
+
+...
diff --git a/Documentation/devicetree/bindings/display/msm/dp-controller.yaml b/Documentation/devicetree/bindings/display/msm/dp-controller.yaml
index 63e585f..5457612 100644
--- a/Documentation/devicetree/bindings/display/msm/dp-controller.yaml
+++ b/Documentation/devicetree/bindings/display/msm/dp-controller.yaml
@@ -17,6 +17,8 @@
compatible:
enum:
- qcom,sc7180-dp
+ - qcom,sc7280-dp
+ - qcom,sc7280-edp
- qcom,sc8180x-dp
- qcom,sc8180x-edp
diff --git a/Documentation/devicetree/bindings/display/msm/edp.txt b/Documentation/devicetree/bindings/display/msm/edp.txt
deleted file mode 100644
index eff9daf..0000000
--- a/Documentation/devicetree/bindings/display/msm/edp.txt
+++ /dev/null
@@ -1,56 +0,0 @@
-Qualcomm Technologies Inc. adreno/snapdragon eDP output
-
-Required properties:
-- compatible:
- * "qcom,mdss-edp"
-- reg: Physical base address and length of the registers of controller and PLL
-- reg-names: The names of register regions. The following regions are required:
- * "edp"
- * "pll_base"
-- interrupts: The interrupt signal from the eDP block.
-- power-domains: Should be <&mmcc MDSS_GDSC>.
-- clocks: device clocks
- See Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
-- clock-names: the following clocks are required:
- * "core"
- * "iface"
- * "mdp_core"
- * "pixel"
- * "link"
-- #clock-cells: The value should be 1.
-- vdda-supply: phandle to vdda regulator device node
-- lvl-vdd-supply: phandle to regulator device node which is used to supply power
- to HPD receiving chip
-- panel-en-gpios: GPIO pin to supply power to panel.
-- panel-hpd-gpios: GPIO pin used for eDP hpd.
-
-
-Example:
- mdss_edp: qcom,mdss_edp@fd923400 {
- compatible = "qcom,mdss-edp";
- reg-names =
- "edp",
- "pll_base";
- reg = <0xfd923400 0x700>,
- <0xfd923a00 0xd4>;
- interrupt-parent = <&mdss_mdp>;
- interrupts = <12 0>;
- power-domains = <&mmcc MDSS_GDSC>;
- clock-names =
- "core",
- "pixel",
- "iface",
- "link",
- "mdp_core";
- clocks =
- <&mmcc MDSS_EDPAUX_CLK>,
- <&mmcc MDSS_EDPPIXEL_CLK>,
- <&mmcc MDSS_AHB_CLK>,
- <&mmcc MDSS_EDPLINK_CLK>,
- <&mmcc MDSS_MDP_CLK>;
- #clock-cells = <1>;
- vdda-supply = <&pma8084_l12>;
- lvl-vdd-supply = <&lvl_vreg>;
- panel-en-gpios = <&tlmm 137 0>;
- panel-hpd-gpios = <&tlmm 103 0>;
- };
diff --git a/Documentation/devicetree/bindings/display/panel/boe,bf060y8m-aj0.yaml b/Documentation/devicetree/bindings/display/panel/boe,bf060y8m-aj0.yaml
new file mode 100644
index 0000000..a8f3afa
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/panel/boe,bf060y8m-aj0.yaml
@@ -0,0 +1,81 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/display/panel/boe,bf060y8m-aj0.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: BOE BF060Y8M-AJ0 5.99" 1080x2160 AMOLED Panel
+
+maintainers:
+ - AngeloGioacchino Del Regno <angelogioacchino.delregno@somainline.org>
+
+description: |
+ This is a 5.99" 1080x2160 16.7M Color active matrix AMOLED
+ video mode panel module on MIPI-DSI 4-Lane interface, GGRB
+ pixel arrangement, 63 micrometers pitch, with an active
+ area of 68.04 x 136.08 millimeters.
+ Each pixel is divided into red and green dots, or blue and
+ green dots, and two pixels share red or blue dots which are
+ arranged in vertical stripe.
+ The DriverIC for this panel module is SW43404.
+
+allOf:
+ - $ref: panel-common.yaml#
+
+properties:
+ compatible:
+ const: boe,bf060y8m-aj0
+
+ elvdd-supply:
+ description: EL Driving positive (VDD) supply (4.40-4.80V)
+ elvss-supply:
+ description: EL Driving negative (VSS) supply (-5.00V to -1.40V)
+ vcc-supply:
+ description: Core (TSP) voltage supply (2.70-3.60V)
+ vci-supply:
+ description: DriverIC Operation supply (2.60-3.60V)
+ vddio-supply:
+ description: I/O voltage supply (1.62-1.98V)
+
+ port: true
+ reg: true
+ reset-gpios: true
+
+required:
+ - compatible
+ - elvdd-supply
+ - elvss-supply
+ - vcc-supply
+ - vci-supply
+ - vddio-supply
+ - reg
+ - reset-gpios
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/gpio/gpio.h>
+
+ dsi {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ panel@0 {
+ compatible = "boe,bf060y8m-aj0";
+ reg = <0>;
+
+ reset-gpios = <&tlmm 94 GPIO_ACTIVE_HIGH>;
+
+ vcc-supply = <&disp_vcc_vreg>;
+ vddio-supply = <&disp_vddio_vreg>;
+ vci-supply = <&disp_vci_vreg>;
+ elvdd-supply = <&disp_elvdd_vreg>;
+ elvss-supply = <&disp_elvss_vreg>;
+
+ port {
+ panel_in: endpoint {
+ remote-endpoint = <&dsi0_out>;
+ };
+ };
+ };
+ };
diff --git a/Documentation/devicetree/bindings/display/panel/ilitek,ili9163.yaml b/Documentation/devicetree/bindings/display/panel/ilitek,ili9163.yaml
new file mode 100644
index 0000000..7e7a836
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/panel/ilitek,ili9163.yaml
@@ -0,0 +1,69 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/display/panel/ilitek,ili9163.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Ilitek ILI9163 display panels device tree bindings
+
+maintainers:
+ - Daniel Mack <daniel@zonque.org>
+
+description:
+ This binding is for display panels using an Ilitek ILI9163 controller in SPI
+ mode.
+
+allOf:
+ - $ref: panel-common.yaml#
+
+properties:
+ compatible:
+ items:
+ - enum:
+ - newhaven,1.8-128160EF
+ - const: ilitek,ili9163
+
+ spi-max-frequency:
+ maximum: 32000000
+
+ dc-gpios:
+ maxItems: 1
+ description: Display data/command selection (D/CX)
+
+ backlight: true
+ reg: true
+ reset-gpios: true
+ rotation: true
+
+required:
+ - compatible
+ - reg
+ - dc-gpios
+ - reset-gpios
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/gpio/gpio.h>
+
+ backlight: backlight {
+ compatible = "gpio-backlight";
+ gpios = <&gpio 22 GPIO_ACTIVE_HIGH>;
+ };
+ spi {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ display@0 {
+ compatible = "newhaven,1.8-128160EF", "ilitek,ili9163";
+ reg = <0>;
+ spi-max-frequency = <32000000>;
+ dc-gpios = <&gpio0 24 GPIO_ACTIVE_HIGH>;
+ reset-gpios = <&gpio0 25 GPIO_ACTIVE_HIGH>;
+ rotation = <180>;
+ backlight = <&backlight>;
+ };
+ };
+
+...
diff --git a/Documentation/devicetree/bindings/display/panel/ilitek,ili9881c.yaml b/Documentation/devicetree/bindings/display/panel/ilitek,ili9881c.yaml
index b2fcec4..c5d1df6 100644
--- a/Documentation/devicetree/bindings/display/panel/ilitek,ili9881c.yaml
+++ b/Documentation/devicetree/bindings/display/panel/ilitek,ili9881c.yaml
@@ -9,24 +9,28 @@
maintainers:
- Maxime Ripard <mripard@kernel.org>
+allOf:
+ - $ref: panel-common.yaml#
+
properties:
compatible:
items:
- enum:
- bananapi,lhr050h41
- feixin,k101-im2byl02
+ - wanchanglong,w552946aba
- const: ilitek,ili9881c
backlight: true
power-supply: true
reg: true
reset-gpios: true
+ rotation: true
required:
- compatible
- power-supply
- reg
- - reset-gpios
additionalProperties: false
diff --git a/Documentation/devicetree/bindings/display/panel/novatek,nt35950.yaml b/Documentation/devicetree/bindings/display/panel/novatek,nt35950.yaml
new file mode 100644
index 0000000..377a05d
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/panel/novatek,nt35950.yaml
@@ -0,0 +1,106 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/display/panel/novatek,nt35950.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Novatek NT35950-based display panels
+
+maintainers:
+ - AngeloGioacchino Del Regno <angelogioacchino.delregno@somainline.org>
+
+description: |
+ The nt35950 IC from Novatek is a Driver IC used to drive MIPI-DSI panels,
+ with Static RAM for content retention in command mode and also supports
+ video mode with VESA Frame Buffer Compression or Display Stream Compression
+ on single, or dual dsi port(s).
+ This DDIC is also capable of upscaling an input image to the panel's native
+ resolution, for example it can upscale a 1920x1080 input to 3840x2160 with
+ either bilinear interpolation or pixel duplication.
+
+allOf:
+ - $ref: panel-common.yaml#
+
+properties:
+ compatible:
+ items:
+ - enum:
+ - sharp,ls055d1sx04
+ - const: novatek,nt35950
+ description: This indicates the panel manufacturer of the panel
+ that is in turn using the NT35950 panel driver. The compatible
+ string determines how the NT35950 panel driver shall be configured
+ to work with the indicated panel. The novatek,nt35950 compatible shall
+ always be provided as a fallback.
+
+ reset-gpios:
+ maxItems: 1
+ description: phandle of gpio for reset line - This should be 8mA, gpio
+ can be configured using mux, pinctrl, pinctrl-names (active high)
+
+ avdd-supply:
+ description: positive boost supply regulator
+ avee-supply:
+ description: negative boost supply regulator
+ dvdd-supply:
+ description: regulator that supplies the digital voltage
+ vddio-supply:
+ description: regulator that supplies the I/O voltage
+
+ backlight: true
+ ports: true
+ reg: true
+
+required:
+ - compatible
+ - reg
+ - reset-gpios
+ - avdd-supply
+ - avee-supply
+ - dvdd-supply
+ - vddio-supply
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/gpio/gpio.h>
+
+ dsi0 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ panel@0 {
+ compatible = "sharp,ls055d1sx04", "novatek,nt35950";
+ reg = <0>;
+
+ backlight = <&pmi8998_wled>;
+ reset-gpios = <&tlmm 94 GPIO_ACTIVE_HIGH>;
+
+ avdd-supply = <&lab>;
+ avee-supply = <&ibb>;
+ dvdd-supply = <&disp_dvdd_vreg>;
+ vddio-supply = <&vreg_l14a_1p85>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ panel_in0: endpoint {
+ remote-endpoint = <&dsi0_out>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+ panel_in1: endpoint {
+ remote-endpoint = <&dsi1_out>;
+ };
+ };
+ };
+ };
+ };
+
+...
diff --git a/Documentation/devicetree/bindings/display/panel/panel-simple-dsi.yaml b/Documentation/devicetree/bindings/display/panel/panel-simple-dsi.yaml
index fbd7166..2c00813 100644
--- a/Documentation/devicetree/bindings/display/panel/panel-simple-dsi.yaml
+++ b/Documentation/devicetree/bindings/display/panel/panel-simple-dsi.yaml
@@ -35,6 +35,8 @@
- boe,tv080wum-nl0
# Innolux P079ZCA 7.85" 768x1024 TFT LCD panel
- innolux,p079zca
+ # JDI FHD_R63452 1080x1920 5.2" IPS LCD Panel
+ - jdi,fhd-r63452
# Khadas TS050 5" 1080x1920 LCD panel
- khadas,ts050
# Kingdisplay KD097D04 9.7" 1536x2048 TFT LCD panel
diff --git a/Documentation/devicetree/bindings/display/panel/panel-simple.yaml b/Documentation/devicetree/bindings/display/panel/panel-simple.yaml
index f3c9395..62f5f050 100644
--- a/Documentation/devicetree/bindings/display/panel/panel-simple.yaml
+++ b/Documentation/devicetree/bindings/display/panel/panel-simple.yaml
@@ -290,6 +290,8 @@
- starry,kr070pe2t
# Starry 12.2" (1920x1200 pixels) TFT LCD panel
- starry,kr122ea0sra
+ # Team Source Display Technology TST043015CMHX 4.3" WQVGA TFT LCD panel
+ - team-source-display,tst043015cmhx
# Tianma Micro-electronics TM070JDHG30 7.0" WXGA TFT LCD panel
- tianma,tm070jdhg30
# Tianma Micro-electronics TM070JVHG33 7.0" WXGA TFT LCD panel
diff --git a/Documentation/devicetree/bindings/display/panel/sony,tulip-truly-nt35521.yaml b/Documentation/devicetree/bindings/display/panel/sony,tulip-truly-nt35521.yaml
new file mode 100644
index 0000000..9679729
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/panel/sony,tulip-truly-nt35521.yaml
@@ -0,0 +1,72 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/display/panel/sony,tulip-truly-nt35521.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Sony Tulip Truly NT35521 5.24" 1280x720 MIPI-DSI Panel
+
+maintainers:
+ - Shawn Guo <shawn.guo@linaro.org>
+
+description: |
+ The Sony Tulip Truly NT35521 is a 5.24" 1280x720 MIPI-DSI panel, which
+ can be found no Sony Xperia M4 phone. The panel backlight is managed
+ through DSI link.
+
+allOf:
+ - $ref: panel-common.yaml#
+
+properties:
+ compatible:
+ const: sony,tulip-truly-nt35521
+
+ reg: true
+
+ positive5-supply:
+ description: Positive 5V supply
+
+ negative5-supply:
+ description: Negative 5V supply
+
+ reset-gpios: true
+
+ enable-gpios: true
+
+ port: true
+
+required:
+ - compatible
+ - reg
+ - positive5-supply
+ - negative5-supply
+ - reset-gpios
+ - enable-gpios
+ - port
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/gpio/gpio.h>
+
+ dsi {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ panel@0 {
+ compatible = "sony,tulip-truly-nt35521";
+ reg = <0>;
+ positive5-supply = <&positive5_reg>;
+ negative5-supply = <&negative5_reg>;
+ reset-gpios = <&msmgpio 25 GPIO_ACTIVE_LOW>;
+ enable-gpios = <&msmgpio 10 GPIO_ACTIVE_HIGH>;
+
+ port {
+ panel_in: endpoint {
+ remote-endpoint = <&dsi0_out>;
+ };
+ };
+ };
+ };
+...
diff --git a/Documentation/devicetree/bindings/display/sprd/sprd,display-subsystem.yaml b/Documentation/devicetree/bindings/display/sprd/sprd,display-subsystem.yaml
new file mode 100644
index 0000000..3d107e9
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/sprd/sprd,display-subsystem.yaml
@@ -0,0 +1,64 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/display/sprd/sprd,display-subsystem.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Unisoc DRM master device
+
+maintainers:
+ - Kevin Tang <kevin.tang@unisoc.com>
+
+description: |
+ The Unisoc DRM master device is a virtual device needed to list all
+ DPU devices or other display interface nodes that comprise the
+ graphics subsystem.
+
+ Unisoc's display pipeline have several components as below description,
+ multi display controllers and corresponding physical interfaces.
+ For different display scenarios, dpu0 and dpu1 maybe binding to different
+ encoder.
+
+ E.g:
+ dpu0 and dpu1 both binding to DSI for dual mipi-dsi display;
+ dpu0 binding to DSI for primary display, and dpu1 binding to DP for external display;
+
+ +-----------------------------------------+
+ | |
+ | +---------+ |
+ +----+ | +----+ +---------+ |DPHY/CPHY| | +------+
+ | +----->+dpu0+--->+MIPI|DSI +--->+Combo +----->+Panel0|
+ |AXI | | +----+ +---------+ +---------+ | +------+
+ | | | ^ |
+ | | | | |
+ | | | +-----------+ |
+ | | | | |
+ |APB | | +--+-+ +-----------+ +---+ | +------+
+ | +----->+dpu1+--->+DisplayPort+--->+PHY+--------->+Panel1|
+ | | | +----+ +-----------+ +---+ | +------+
+ +----+ | |
+ +-----------------------------------------+
+
+properties:
+ compatible:
+ const: sprd,display-subsystem
+
+ ports:
+ $ref: /schemas/types.yaml#/definitions/phandle-array
+ description:
+ Should contain a list of phandles pointing to display interface port
+ of DPU devices.
+
+required:
+ - compatible
+ - ports
+
+additionalProperties: false
+
+examples:
+ - |
+ display-subsystem {
+ compatible = "sprd,display-subsystem";
+ ports = <&dpu_out>;
+ };
+
diff --git a/Documentation/devicetree/bindings/display/sprd/sprd,sharkl3-dpu.yaml b/Documentation/devicetree/bindings/display/sprd/sprd,sharkl3-dpu.yaml
new file mode 100644
index 0000000..4ebea60
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/sprd/sprd,sharkl3-dpu.yaml
@@ -0,0 +1,77 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/display/sprd/sprd,sharkl3-dpu.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Unisoc Sharkl3 Display Processor Unit (DPU)
+
+maintainers:
+ - Kevin Tang <kevin.tang@unisoc.com>
+
+description: |
+ DPU (Display Processor Unit) is the Display Controller for the Unisoc SoCs
+ which transfers the image data from a video memory buffer to an internal
+ LCD interface.
+
+properties:
+ compatible:
+ const: sprd,sharkl3-dpu
+
+ reg:
+ maxItems: 1
+
+ interrupts:
+ maxItems: 1
+
+ clocks:
+ minItems: 2
+
+ clock-names:
+ items:
+ - const: clk_src_128m
+ - const: clk_src_384m
+
+ power-domains:
+ maxItems: 1
+
+ iommus:
+ maxItems: 1
+
+ port:
+ type: object
+ description:
+ A port node with endpoint definitions as defined in
+ Documentation/devicetree/bindings/media/video-interfaces.txt.
+ That port should be the output endpoint, usually output to
+ the associated DSI.
+
+required:
+ - compatible
+ - reg
+ - interrupts
+ - clocks
+ - clock-names
+ - port
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+ #include <dt-bindings/clock/sprd,sc9860-clk.h>
+ dpu: dpu@63000000 {
+ compatible = "sprd,sharkl3-dpu";
+ reg = <0x63000000 0x1000>;
+ interrupts = <GIC_SPI 46 IRQ_TYPE_LEVEL_HIGH>;
+ clock-names = "clk_src_128m", "clk_src_384m";
+
+ clocks = <&pll CLK_TWPLL_128M>,
+ <&pll CLK_TWPLL_384M>;
+
+ dpu_port: port {
+ dpu_out: endpoint {
+ remote-endpoint = <&dsi_in>;
+ };
+ };
+ };
diff --git a/Documentation/devicetree/bindings/display/sprd/sprd,sharkl3-dsi-host.yaml b/Documentation/devicetree/bindings/display/sprd/sprd,sharkl3-dsi-host.yaml
new file mode 100644
index 0000000..bc5594d
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/sprd/sprd,sharkl3-dsi-host.yaml
@@ -0,0 +1,88 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/display/sprd/sprd,sharkl3-dsi-host.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Unisoc MIPI DSI Controller
+
+maintainers:
+ - Kevin Tang <kevin.tang@unisoc.com>
+
+properties:
+ compatible:
+ const: sprd,sharkl3-dsi-host
+
+ reg:
+ maxItems: 1
+
+ interrupts:
+ maxItems: 2
+
+ clocks:
+ minItems: 1
+
+ clock-names:
+ items:
+ - const: clk_src_96m
+
+ power-domains:
+ maxItems: 1
+
+ ports:
+ type: object
+
+ properties:
+ "#address-cells":
+ const: 1
+
+ "#size-cells":
+ const: 0
+
+ port@0:
+ type: object
+ description:
+ A port node with endpoint definitions as defined in
+ Documentation/devicetree/bindings/media/video-interfaces.txt.
+ That port should be the input endpoint, usually coming from
+ the associated DPU.
+
+ required:
+ - "#address-cells"
+ - "#size-cells"
+ - port@0
+
+ additionalProperties: false
+
+required:
+ - compatible
+ - reg
+ - interrupts
+ - clocks
+ - clock-names
+ - ports
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+ #include <dt-bindings/clock/sprd,sc9860-clk.h>
+ dsi: dsi@63100000 {
+ compatible = "sprd,sharkl3-dsi-host";
+ reg = <0x63100000 0x1000>;
+ interrupts = <GIC_SPI 48 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 49 IRQ_TYPE_LEVEL_HIGH>;
+ clock-names = "clk_src_96m";
+ clocks = <&pll CLK_TWPLL_96M>;
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ port@0 {
+ reg = <0>;
+ dsi_in: endpoint {
+ remote-endpoint = <&dpu_out>;
+ };
+ };
+ };
+ };
diff --git a/Documentation/devicetree/bindings/display/tegra/nvidia,tegra20-host1x.txt b/Documentation/devicetree/bindings/display/tegra/nvidia,tegra20-host1x.txt
index 8a6d3e1..e61999c 100644
--- a/Documentation/devicetree/bindings/display/tegra/nvidia,tegra20-host1x.txt
+++ b/Documentation/devicetree/bindings/display/tegra/nvidia,tegra20-host1x.txt
@@ -19,6 +19,19 @@
See ../reset/reset.txt for details.
- reset-names: Must include the following entries:
- host1x
+ - mc
+
+Optional properties:
+- operating-points-v2: See ../bindings/opp/opp.txt for details.
+ - power-domains: Phandle to HEG or core power domain.
+
+For each opp entry in 'operating-points-v2' table of host1x and its modules:
+- opp-supported-hw: One bitfield indicating:
+ On Tegra20: SoC process ID mask
+ On Tegra30+: SoC speedo ID mask
+
+ A bitwise AND is performed against the value and if any bit
+ matches, the OPP gets enabled.
Each host1x client module having to perform DMA through the Memory Controller
should have the interconnect endpoints set to the Memory Client and External
@@ -45,6 +58,8 @@
- interconnect-names: Must include name of the interconnect path for each
interconnect entry. Consult TRM documentation for information about
available memory clients, see MEMORY CONTROLLER section.
+ - operating-points-v2: See ../bindings/opp/opp.txt for details.
+ - power-domains: Phandle to MPE power domain.
- vi: video input
@@ -128,6 +143,8 @@
- interconnect-names: Must include name of the interconnect path for each
interconnect entry. Consult TRM documentation for information about
available memory clients, see MEMORY CONTROLLER section.
+ - operating-points-v2: See ../bindings/opp/opp.txt for details.
+ - power-domains: Phandle to VENC power domain.
- epp: encoder pre-processor
@@ -147,6 +164,8 @@
- interconnect-names: Must include name of the interconnect path for each
interconnect entry. Consult TRM documentation for information about
available memory clients, see MEMORY CONTROLLER section.
+ - operating-points-v2: See ../bindings/opp/opp.txt for details.
+ - power-domains: Phandle to HEG or core power domain.
- isp: image signal processor
@@ -166,6 +185,7 @@
- interconnect-names: Must include name of the interconnect path for each
interconnect entry. Consult TRM documentation for information about
available memory clients, see MEMORY CONTROLLER section.
+ - power-domains: Phandle to VENC or core power domain.
- gr2d: 2D graphics engine
@@ -179,12 +199,15 @@
See ../reset/reset.txt for details.
- reset-names: Must include the following entries:
- 2d
+ - mc
Optional properties:
- interconnects: Must contain entry for the GR2D memory clients.
- interconnect-names: Must include name of the interconnect path for each
interconnect entry. Consult TRM documentation for information about
available memory clients, see MEMORY CONTROLLER section.
+ - operating-points-v2: See ../bindings/opp/opp.txt for details.
+ - power-domains: Phandle to HEG or core power domain.
- gr3d: 3D graphics engine
@@ -203,12 +226,16 @@
- reset-names: Must include the following entries:
- 3d
- 3d2 (Only required on SoCs with two 3D clocks)
+ - mc
+ - mc2 (Only required on SoCs with two 3D clocks)
Optional properties:
- interconnects: Must contain entry for the GR3D memory clients.
- interconnect-names: Must include name of the interconnect path for each
interconnect entry. Consult TRM documentation for information about
available memory clients, see MEMORY CONTROLLER section.
+ - operating-points-v2: See ../bindings/opp/opp.txt for details.
+ - power-domains: Phandles to 3D or core power domain.
- dc: display controller
@@ -241,6 +268,8 @@
- interconnect-names: Must include name of the interconnect path for each
interconnect entry. Consult TRM documentation for information about
available memory clients, see MEMORY CONTROLLER section.
+ - operating-points-v2: See ../bindings/opp/opp.txt for details.
+ - power-domains: Phandle to core power domain.
- hdmi: High Definition Multimedia Interface
@@ -267,6 +296,7 @@
- nvidia,hpd-gpio: specifies a GPIO used for hotplug detection
- nvidia,edid: supplies a binary EDID blob
- nvidia,panel: phandle of a display panel
+ - operating-points-v2: See ../bindings/opp/opp.txt for details.
- tvo: TV encoder output
@@ -277,6 +307,10 @@
- clocks: Must contain one entry, for the module clock.
See ../clocks/clock-bindings.txt for details.
+ Optional properties:
+ - operating-points-v2: See ../bindings/opp/opp.txt for details.
+ - power-domains: Phandle to core power domain.
+
- dsi: display serial interface
Required properties:
@@ -305,6 +339,7 @@
- nvidia,panel: phandle of a display panel
- nvidia,ganged-mode: contains a phandle to a second DSI controller to gang
up with in order to support up to 8 data lanes
+ - operating-points-v2: See ../bindings/opp/opp.txt for details.
- sor: serial output resource
@@ -408,6 +443,8 @@
clocks = <&tegra_car TEGRA20_CLK_HOST1X>;
resets = <&tegra_car 28>;
reset-names = "host1x";
+ operating-points-v2 = <&dvfs_opp_table>;
+ power-domains = <&domain>;
#address-cells = <1>;
#size-cells = <1>;
@@ -421,6 +458,8 @@
clocks = <&tegra_car TEGRA20_CLK_MPE>;
resets = <&tegra_car 60>;
reset-names = "mpe";
+ operating-points-v2 = <&dvfs_opp_table>;
+ power-domains = <&domain>;
};
vi@54080000 {
@@ -429,6 +468,7 @@
interrupts = <GIC_SPI 69 IRQ_TYPE_LEVEL_HIGH>;
assigned-clocks = <&tegra_car TEGRA210_CLK_VI>;
assigned-clock-parents = <&tegra_car TEGRA210_CLK_PLL_C4_OUT0>;
+ operating-points-v2 = <&dvfs_opp_table>;
clocks = <&tegra_car TEGRA210_CLK_VI>;
power-domains = <&pd_venc>;
@@ -510,6 +550,8 @@
clocks = <&tegra_car TEGRA20_CLK_EPP>;
resets = <&tegra_car 19>;
reset-names = "epp";
+ operating-points-v2 = <&dvfs_opp_table>;
+ power-domains = <&domain>;
};
isp {
@@ -528,6 +570,8 @@
clocks = <&tegra_car TEGRA20_CLK_GR2D>;
resets = <&tegra_car 21>;
reset-names = "2d";
+ operating-points-v2 = <&dvfs_opp_table>;
+ power-domains = <&domain>;
};
gr3d {
@@ -536,6 +580,8 @@
clocks = <&tegra_car TEGRA20_CLK_GR3D>;
resets = <&tegra_car 24>;
reset-names = "3d";
+ operating-points-v2 = <&dvfs_opp_table>;
+ power-domains = <&domain>;
};
dc@54200000 {
@@ -547,6 +593,8 @@
clock-names = "dc", "parent";
resets = <&tegra_car 27>;
reset-names = "dc";
+ operating-points-v2 = <&dvfs_opp_table>;
+ power-domains = <&domain>;
interconnects = <&mc TEGRA20_MC_DISPLAY0A &emc>,
<&mc TEGRA20_MC_DISPLAY0B &emc>,
@@ -571,6 +619,8 @@
clock-names = "dc", "parent";
resets = <&tegra_car 26>;
reset-names = "dc";
+ operating-points-v2 = <&dvfs_opp_table>;
+ power-domains = <&domain>;
interconnects = <&mc TEGRA20_MC_DISPLAY0AB &emc>,
<&mc TEGRA20_MC_DISPLAY0BB &emc>,
@@ -596,6 +646,7 @@
resets = <&tegra_car 51>;
reset-names = "hdmi";
status = "disabled";
+ operating-points-v2 = <&dvfs_opp_table>;
};
tvo {
@@ -604,6 +655,7 @@
interrupts = <0 76 0x04>;
clocks = <&tegra_car TEGRA20_CLK_TVO>;
status = "disabled";
+ operating-points-v2 = <&dvfs_opp_table>;
};
dsi {
@@ -615,6 +667,7 @@
resets = <&tegra_car 48>;
reset-names = "dsi";
status = "disabled";
+ operating-points-v2 = <&dvfs_opp_table>;
};
};
diff --git a/Documentation/devicetree/bindings/firmware/intel,ixp4xx-network-processing-engine.yaml b/Documentation/devicetree/bindings/firmware/intel,ixp4xx-network-processing-engine.yaml
index c435c9f..9a785bb 100644
--- a/Documentation/devicetree/bindings/firmware/intel,ixp4xx-network-processing-engine.yaml
+++ b/Documentation/devicetree/bindings/firmware/intel,ixp4xx-network-processing-engine.yaml
@@ -37,6 +37,20 @@
should be named with the instance number of the NPE engine used for
the crypto engine.
+ "#address-cells":
+ const: 1
+
+ "#size-cells":
+ const: 0
+
+patternProperties:
+ hss@[0-9]+$:
+ $ref: /schemas/net/intel,ixp4xx-hss.yaml#
+ type: object
+ description: Optional node for the High Speed Serial link (HSS), the
+ node should be named with the instance number of the NPE engine
+ used for the HSS.
+
required:
- compatible
- reg
@@ -45,9 +59,30 @@
examples:
- |
+ #include <dt-bindings/gpio/gpio.h>
+
npe: npe@c8006000 {
compatible = "intel,ixp4xx-network-processing-engine";
reg = <0xc8006000 0x1000>, <0xc8007000 0x1000>, <0xc8008000 0x1000>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ hss@0 {
+ compatible = "intel,ixp4xx-hss";
+ reg = <0>;
+ intel,npe-handle = <&npe 0>;
+ intel,queue-chl-rxtrig = <&qmgr 12>;
+ intel,queue-chl-txready = <&qmgr 34>;
+ intel,queue-pkt-rx = <&qmgr 13>;
+ intel,queue-pkt-tx = <&qmgr 14>, <&qmgr 15>, <&qmgr 16>, <&qmgr 17>;
+ intel,queue-pkt-rxfree = <&qmgr 18>, <&qmgr 19>, <&qmgr 20>, <&qmgr 21>;
+ intel,queue-pkt-txdone = <&qmgr 22>;
+ cts-gpios = <&gpio0 10 GPIO_ACTIVE_LOW>;
+ rts-gpios = <&gpio0 14 GPIO_ACTIVE_LOW>;
+ dcd-gpios = <&gpio0 6 GPIO_ACTIVE_LOW>;
+ dtr-gpios = <&gpio_74 2 GPIO_ACTIVE_LOW>;
+ clk-internal-gpios = <&gpio_74 0 GPIO_ACTIVE_HIGH>;
+ };
crypto {
compatible = "intel,ixp4xx-crypto";
diff --git a/Documentation/devicetree/bindings/firmware/nvidia,tegra186-bpmp.txt b/Documentation/devicetree/bindings/firmware/nvidia,tegra186-bpmp.txt
deleted file mode 100644
index e44a13b..0000000
--- a/Documentation/devicetree/bindings/firmware/nvidia,tegra186-bpmp.txt
+++ /dev/null
@@ -1,107 +0,0 @@
-NVIDIA Tegra Boot and Power Management Processor (BPMP)
-
-The BPMP is a specific processor in Tegra chip, which is designed for
-booting process handling and offloading the power management, clock
-management, and reset control tasks from the CPU. The binding document
-defines the resources that would be used by the BPMP firmware driver,
-which can create the interprocessor communication (IPC) between the CPU
-and BPMP.
-
-Required properties:
-- compatible
- Array of strings
- One of:
- - "nvidia,tegra186-bpmp"
-- mboxes : The phandle of mailbox controller and the mailbox specifier.
-- shmem : List of the phandle of the TX and RX shared memory area that
- the IPC between CPU and BPMP is based on.
-- #clock-cells : Should be 1.
-- #power-domain-cells : Should be 1.
-- #reset-cells : Should be 1.
-
-This node is a mailbox consumer. See the following files for details of
-the mailbox subsystem, and the specifiers implemented by the relevant
-provider(s):
-
-- .../mailbox/mailbox.txt
-- .../mailbox/nvidia,tegra186-hsp.txt
-
-This node is a clock, power domain, and reset provider. See the following
-files for general documentation of those features, and the specifiers
-implemented by this node:
-
-- .../clock/clock-bindings.txt
-- <dt-bindings/clock/tegra186-clock.h>
-- ../power/power-domain.yaml
-- <dt-bindings/power/tegra186-powergate.h>
-- .../reset/reset.txt
-- <dt-bindings/reset/tegra186-reset.h>
-
-The BPMP implements some services which must be represented by separate nodes.
-For example, it can provide access to certain I2C controllers, and the I2C
-bindings represent each I2C controller as a device tree node. Such nodes should
-be nested directly inside the main BPMP node.
-
-Software can determine whether a child node of the BPMP node represents a device
-by checking for a compatible property. Any node with a compatible property
-represents a device that can be instantiated. Nodes without a compatible
-property may be used to provide configuration information regarding the BPMP
-itself, although no such configuration nodes are currently defined by this
-binding.
-
-The BPMP firmware defines no single global name-/numbering-space for such
-services. Put another way, the numbering scheme for I2C buses is distinct from
-the numbering scheme for any other service the BPMP may provide (e.g. a future
-hypothetical SPI bus service). As such, child device nodes will have no reg
-property, and the BPMP node will have no #address-cells or #size-cells property.
-
-The shared memory bindings for BPMP
------------------------------------
-
-The shared memory area for the IPC TX and RX between CPU and BPMP are
-predefined and work on top of sysram, which is an SRAM inside the chip.
-
-See ".../sram/sram.txt" for the bindings.
-
-Example:
-
-hsp_top0: hsp@3c00000 {
- ...
- #mbox-cells = <2>;
-};
-
-sysram@30000000 {
- compatible = "nvidia,tegra186-sysram", "mmio-sram";
- reg = <0x0 0x30000000 0x0 0x50000>;
- #address-cells = <2>;
- #size-cells = <2>;
- ranges = <0 0x0 0x0 0x30000000 0x0 0x50000>;
-
- cpu_bpmp_tx: shmem@4e000 {
- compatible = "nvidia,tegra186-bpmp-shmem";
- reg = <0x0 0x4e000 0x0 0x1000>;
- label = "cpu-bpmp-tx";
- pool;
- };
-
- cpu_bpmp_rx: shmem@4f000 {
- compatible = "nvidia,tegra186-bpmp-shmem";
- reg = <0x0 0x4f000 0x0 0x1000>;
- label = "cpu-bpmp-rx";
- pool;
- };
-};
-
-bpmp {
- compatible = "nvidia,tegra186-bpmp";
- mboxes = <&hsp_top0 TEGRA_HSP_MBOX_TYPE_DB TEGRA_HSP_DB_MASTER_BPMP>;
- shmem = <&cpu_bpmp_tx &cpu_bpmp_rx>;
- #clock-cells = <1>;
- #power-domain-cells = <1>;
- #reset-cells = <1>;
-
- i2c {
- compatible = "...";
- ...
- };
-};
diff --git a/Documentation/devicetree/bindings/firmware/nvidia,tegra186-bpmp.yaml b/Documentation/devicetree/bindings/firmware/nvidia,tegra186-bpmp.yaml
new file mode 100644
index 0000000..833c07f
--- /dev/null
+++ b/Documentation/devicetree/bindings/firmware/nvidia,tegra186-bpmp.yaml
@@ -0,0 +1,186 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/firmware/nvidia,tegra186-bpmp.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: NVIDIA Tegra Boot and Power Management Processor (BPMP)
+
+maintainers:
+ - Thierry Reding <thierry.reding@gmail.com>
+ - Jon Hunter <jonathanh@nvidia.com>
+
+description: |
+ The BPMP is a specific processor in Tegra chip, which is designed for
+ booting process handling and offloading the power management, clock
+ management, and reset control tasks from the CPU. The binding document
+ defines the resources that would be used by the BPMP firmware driver,
+ which can create the interprocessor communication (IPC) between the
+ CPU and BPMP.
+
+ This node is a mailbox consumer. See the following files for details
+ of the mailbox subsystem, and the specifiers implemented by the
+ relevant provider(s):
+
+ - .../mailbox/mailbox.txt
+ - .../mailbox/nvidia,tegra186-hsp.yaml
+
+ This node is a clock, power domain, and reset provider. See the
+ following files for general documentation of those features, and the
+ specifiers implemented by this node:
+
+ - .../clock/clock-bindings.txt
+ - <dt-bindings/clock/tegra186-clock.h>
+ - ../power/power-domain.yaml
+ - <dt-bindings/power/tegra186-powergate.h>
+ - .../reset/reset.txt
+ - <dt-bindings/reset/tegra186-reset.h>
+
+ The BPMP implements some services which must be represented by
+ separate nodes. For example, it can provide access to certain I2C
+ controllers, and the I2C bindings represent each I2C controller as a
+ device tree node. Such nodes should be nested directly inside the main
+ BPMP node.
+
+ Software can determine whether a child node of the BPMP node
+ represents a device by checking for a compatible property. Any node
+ with a compatible property represents a device that can be
+ instantiated. Nodes without a compatible property may be used to
+ provide configuration information regarding the BPMP itself, although
+ no such configuration nodes are currently defined by this binding.
+
+ The BPMP firmware defines no single global name-/numbering-space for
+ such services. Put another way, the numbering scheme for I2C buses is
+ distinct from the numbering scheme for any other service the BPMP may
+ provide (e.g. a future hypothetical SPI bus service). As such, child
+ device nodes will have no reg property, and the BPMP node will have no
+ "#address-cells" or "#size-cells" property.
+
+ The shared memory area for the IPC TX and RX between CPU and BPMP are
+ predefined and work on top of sysram, which is an SRAM inside the
+ chip. See ".../sram/sram.yaml" for the bindings.
+
+properties:
+ compatible:
+ oneOf:
+ - items:
+ - enum:
+ - nvidia,tegra194-bpmp
+ - nvidia,tegra234-bpmp
+ - const: nvidia,tegra186-bpmp
+ - const: nvidia,tegra186-bpmp
+
+ mboxes:
+ description: A phandle and channel specifier for the mailbox used to
+ communicate with the BPMP.
+ maxItems: 1
+
+ shmem:
+ description: List of the phandle to the TX and RX shared memory area
+ that the IPC between CPU and BPMP is based on.
+ minItems: 2
+ maxItems: 2
+
+ "#clock-cells":
+ const: 1
+
+ "#power-domain-cells":
+ const: 1
+
+ "#reset-cells":
+ const: 1
+
+ interconnects:
+ items:
+ - description: memory read client
+ - description: memory write client
+ - description: DMA read client
+ - description: DMA write client
+
+ interconnect-names:
+ items:
+ - const: read
+ - const: write
+ - const: dma-mem # dma-read
+ - const: dma-write
+
+ iommus:
+ maxItems: 1
+
+ i2c:
+ type: object
+
+ thermal:
+ type: object
+
+additionalProperties: false
+
+required:
+ - compatible
+ - mboxes
+ - shmem
+ - "#clock-cells"
+ - "#power-domain-cells"
+ - "#reset-cells"
+
+examples:
+ - |
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+ #include <dt-bindings/mailbox/tegra186-hsp.h>
+ #include <dt-bindings/memory/tegra186-mc.h>
+
+ hsp_top0: hsp@3c00000 {
+ compatible = "nvidia,tegra186-hsp";
+ reg = <0x03c00000 0xa0000>;
+ interrupts = <GIC_SPI 176 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "doorbell";
+ #mbox-cells = <2>;
+ };
+
+ sram@30000000 {
+ compatible = "nvidia,tegra186-sysram", "mmio-sram";
+ reg = <0x30000000 0x50000>;
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges = <0x0 0x30000000 0x50000>;
+
+ cpu_bpmp_tx: sram@4e000 {
+ reg = <0x4e000 0x1000>;
+ label = "cpu-bpmp-tx";
+ pool;
+ };
+
+ cpu_bpmp_rx: sram@4f000 {
+ reg = <0x4f000 0x1000>;
+ label = "cpu-bpmp-rx";
+ pool;
+ };
+ };
+
+ bpmp {
+ compatible = "nvidia,tegra186-bpmp";
+ interconnects = <&mc TEGRA186_MEMORY_CLIENT_BPMPR &emc>,
+ <&mc TEGRA186_MEMORY_CLIENT_BPMPW &emc>,
+ <&mc TEGRA186_MEMORY_CLIENT_BPMPDMAR &emc>,
+ <&mc TEGRA186_MEMORY_CLIENT_BPMPDMAW &emc>;
+ interconnect-names = "read", "write", "dma-mem", "dma-write";
+ iommus = <&smmu TEGRA186_SID_BPMP>;
+ mboxes = <&hsp_top0 TEGRA_HSP_MBOX_TYPE_DB
+ TEGRA_HSP_DB_MASTER_BPMP>;
+ shmem = <&cpu_bpmp_tx>, <&cpu_bpmp_rx>;
+ #clock-cells = <1>;
+ #power-domain-cells = <1>;
+ #reset-cells = <1>;
+
+ i2c {
+ compatible = "nvidia,tegra186-bpmp-i2c";
+ nvidia,bpmp-bus-id = <5>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ };
+
+ thermal {
+ compatible = "nvidia,tegra186-bpmp-thermal";
+ #thermal-sensor-cells = <1>;
+ };
+ };
diff --git a/Documentation/devicetree/bindings/firmware/qcom,scm.txt b/Documentation/devicetree/bindings/firmware/qcom,scm.txt
index d7e3cda..b1cd4ad 100644
--- a/Documentation/devicetree/bindings/firmware/qcom,scm.txt
+++ b/Documentation/devicetree/bindings/firmware/qcom,scm.txt
@@ -26,9 +26,11 @@
* "qcom,scm-sc7280"
* "qcom,scm-sdm845"
* "qcom,scm-sdx55"
+ * "qcom,scm-sm6350"
* "qcom,scm-sm8150"
* "qcom,scm-sm8250"
* "qcom,scm-sm8350"
+ * "qcom,scm-sm8450"
and:
* "qcom,scm"
- clocks: Specifies clocks needed by the SCM interface, if any:
diff --git a/Documentation/devicetree/bindings/fuse/nvidia,tegra20-fuse.txt b/Documentation/devicetree/bindings/fuse/nvidia,tegra20-fuse.txt
deleted file mode 100644
index b109911..0000000
--- a/Documentation/devicetree/bindings/fuse/nvidia,tegra20-fuse.txt
+++ /dev/null
@@ -1,42 +0,0 @@
-NVIDIA Tegra20/Tegra30/Tegr114/Tegra124 fuse block.
-
-Required properties:
-- compatible : For Tegra20, must contain "nvidia,tegra20-efuse". For Tegra30,
- must contain "nvidia,tegra30-efuse". For Tegra114, must contain
- "nvidia,tegra114-efuse". For Tegra124, must contain "nvidia,tegra124-efuse".
- For Tegra132 must contain "nvidia,tegra132-efuse", "nvidia,tegra124-efuse".
- For Tegra210 must contain "nvidia,tegra210-efuse". For Tegra186 must contain
- "nvidia,tegra186-efuse". For Tegra194 must contain "nvidia,tegra194-efuse".
- For Tegra234 must contain "nvidia,tegra234-efuse".
- Details:
- nvidia,tegra20-efuse: Tegra20 requires using APB DMA to read the fuse data
- due to a hardware bug. Tegra20 also lacks certain information which is
- available in later generations such as fab code, lot code, wafer id,..
- nvidia,tegra30-efuse, nvidia,tegra114-efuse and nvidia,tegra124-efuse:
- The differences between these SoCs are the size of the efuse array,
- the location of the spare (OEM programmable) bits and the location of
- the speedo data.
-- reg: Should contain 1 entry: the entry gives the physical address and length
- of the fuse registers.
-- clocks: Must contain an entry for each entry in clock-names.
- See ../clocks/clock-bindings.txt for details.
-- clock-names: Must include the following entries:
- - fuse
-- resets: Must contain an entry for each entry in reset-names.
- See ../reset/reset.txt for details.
-- reset-names: Must include the following entries:
- - fuse
-
-Example:
-
- fuse@7000f800 {
- compatible = "nvidia,tegra20-efuse";
- reg = <0x7000f800 0x400>,
- <0x70000000 0x400>;
- clocks = <&tegra_car TEGRA20_CLK_FUSE>;
- clock-names = "fuse";
- resets = <&tegra_car 39>;
- reset-names = "fuse";
- };
-
-
diff --git a/Documentation/devicetree/bindings/fuse/nvidia,tegra20-fuse.yaml b/Documentation/devicetree/bindings/fuse/nvidia,tegra20-fuse.yaml
new file mode 100644
index 0000000..4819012
--- /dev/null
+++ b/Documentation/devicetree/bindings/fuse/nvidia,tegra20-fuse.yaml
@@ -0,0 +1,89 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/fuse/nvidia,tegra20-fuse.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: NVIDIA Tegra FUSE block
+
+maintainers:
+ - Thierry Reding <thierry.reding@gmail.com>
+ - Jon Hunter <jonathanh@nvidia.com>
+
+properties:
+ compatible:
+ oneOf:
+ - enum:
+ - nvidia,tegra20-efuse
+ - nvidia,tegra30-efuse
+ - nvidia,tegra114-efuse
+ - nvidia,tegra124-efuse
+ - nvidia,tegra210-efuse
+ - nvidia,tegra186-efuse
+ - nvidia,tegra194-efuse
+ - nvidia,tegra234-efuse
+
+ - items:
+ - const: nvidia,tegra132-efuse
+ - const: nvidia,tegra124-efuse
+
+ reg:
+ maxItems: 1
+
+ clocks:
+ maxItems: 1
+
+ clock-names:
+ items:
+ - const: fuse
+
+ resets:
+ maxItems: 1
+
+ reset-names:
+ items:
+ - const: fuse
+
+ operating-points-v2:
+ $ref: "/schemas/types.yaml#/definitions/phandle"
+
+ power-domains:
+ items:
+ - description: phandle to the core power domain
+
+additionalProperties: false
+
+required:
+ - compatible
+ - reg
+ - clocks
+ - clock-names
+
+if:
+ properties:
+ compatible:
+ contains:
+ enum:
+ - nvidia,tegra20-efuse
+ - nvidia,tegra30-efuse
+ - nvidia,tegra114-efuse
+ - nvidia,tegra124-efuse
+ - nvidia,tegra132-efuse
+ - nvidia,tegra210-efuse
+then:
+ required:
+ - resets
+ - reset-names
+
+examples:
+ - |
+ #include <dt-bindings/clock/tegra20-car.h>
+
+ fuse@7000f800 {
+ compatible = "nvidia,tegra20-efuse";
+ reg = <0x7000f800 0x400>;
+ clocks = <&tegra_car TEGRA20_CLK_FUSE>;
+ clock-names = "fuse";
+ resets = <&tegra_car 39>;
+ reset-names = "fuse";
+ };
diff --git a/Documentation/devicetree/bindings/gpu/host1x/nvidia,tegra210-nvenc.yaml b/Documentation/devicetree/bindings/gpu/host1x/nvidia,tegra210-nvenc.yaml
new file mode 100644
index 0000000..e63ae1a
--- /dev/null
+++ b/Documentation/devicetree/bindings/gpu/host1x/nvidia,tegra210-nvenc.yaml
@@ -0,0 +1,135 @@
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: "http://devicetree.org/schemas/gpu/host1x/nvidia,tegra210-nvenc.yaml#"
+$schema: "http://devicetree.org/meta-schemas/core.yaml#"
+
+title: Device tree binding for NVIDIA Tegra NVENC
+
+description: |
+ NVENC is the hardware video encoder present on NVIDIA Tegra210
+ and newer chips. It is located on the Host1x bus and typically
+ programmed through Host1x channels.
+
+maintainers:
+ - Thierry Reding <treding@gmail.com>
+ - Mikko Perttunen <mperttunen@nvidia.com>
+
+properties:
+ $nodename:
+ pattern: "^nvenc@[0-9a-f]*$"
+
+ compatible:
+ enum:
+ - nvidia,tegra210-nvenc
+ - nvidia,tegra186-nvenc
+ - nvidia,tegra194-nvenc
+
+ reg:
+ maxItems: 1
+
+ clocks:
+ maxItems: 1
+
+ clock-names:
+ items:
+ - const: nvenc
+
+ resets:
+ maxItems: 1
+
+ reset-names:
+ items:
+ - const: nvenc
+
+ power-domains:
+ maxItems: 1
+
+ iommus:
+ maxItems: 1
+
+ dma-coherent: true
+
+ interconnects:
+ minItems: 2
+ maxItems: 3
+
+ interconnect-names:
+ minItems: 2
+ maxItems: 3
+
+ nvidia,host1x-class:
+ description: |
+ Host1x class of the engine, used to specify the targeted engine
+ when programming the engine through Host1x channels or when
+ configuring engine-specific behavior in Host1x.
+ default: 0x21
+ $ref: /schemas/types.yaml#/definitions/uint32
+
+required:
+ - compatible
+ - reg
+ - clocks
+ - clock-names
+ - resets
+ - reset-names
+ - power-domains
+
+allOf:
+ - if:
+ properties:
+ compatible:
+ enum:
+ - nvidia,tegra210-nvenc
+ - nvidia,tegra186-nvenc
+ then:
+ properties:
+ interconnects:
+ items:
+ - description: DMA read memory client
+ - description: DMA write memory client
+ interconnect-names:
+ items:
+ - const: dma-mem
+ - const: write
+ - if:
+ properties:
+ compatible:
+ enum:
+ - nvidia,tegra194-nvenc
+ then:
+ properties:
+ interconnects:
+ items:
+ - description: DMA read memory client
+ - description: DMA read 2 memory client
+ - description: DMA write memory client
+ interconnect-names:
+ items:
+ - const: dma-mem
+ - const: read-1
+ - const: write
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/clock/tegra186-clock.h>
+ #include <dt-bindings/memory/tegra186-mc.h>
+ #include <dt-bindings/power/tegra186-powergate.h>
+ #include <dt-bindings/reset/tegra186-reset.h>
+
+ nvenc@154c0000 {
+ compatible = "nvidia,tegra186-nvenc";
+ reg = <0x154c0000 0x40000>;
+ clocks = <&bpmp TEGRA186_CLK_NVENC>;
+ clock-names = "nvenc";
+ resets = <&bpmp TEGRA186_RESET_NVENC>;
+ reset-names = "nvenc";
+
+ power-domains = <&bpmp TEGRA186_POWER_DOMAIN_MPE>;
+ interconnects = <&mc TEGRA186_MEMORY_CLIENT_NVENCSRD &emc>,
+ <&mc TEGRA186_MEMORY_CLIENT_NVENCSWR &emc>;
+ interconnect-names = "dma-mem", "write";
+ iommus = <&smmu TEGRA186_SID_NVENC>;
+ };
diff --git a/Documentation/devicetree/bindings/gpu/host1x/nvidia,tegra210-nvjpg.yaml b/Documentation/devicetree/bindings/gpu/host1x/nvidia,tegra210-nvjpg.yaml
new file mode 100644
index 0000000..8647404
--- /dev/null
+++ b/Documentation/devicetree/bindings/gpu/host1x/nvidia,tegra210-nvjpg.yaml
@@ -0,0 +1,94 @@
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: "http://devicetree.org/schemas/gpu/host1x/nvidia,tegra210-nvjpg.yaml#"
+$schema: "http://devicetree.org/meta-schemas/core.yaml#"
+
+title: Device tree binding for NVIDIA Tegra NVJPG
+
+description: |
+ NVJPG is the hardware JPEG decoder and encoder present on NVIDIA Tegra210
+ and newer chips. It is located on the Host1x bus and typically programmed
+ through Host1x channels.
+
+maintainers:
+ - Thierry Reding <treding@gmail.com>
+ - Mikko Perttunen <mperttunen@nvidia.com>
+
+properties:
+ $nodename:
+ pattern: "^nvjpg@[0-9a-f]*$"
+
+ compatible:
+ enum:
+ - nvidia,tegra210-nvjpg
+ - nvidia,tegra186-nvjpg
+ - nvidia,tegra194-nvjpg
+
+ reg:
+ maxItems: 1
+
+ clocks:
+ maxItems: 1
+
+ clock-names:
+ items:
+ - const: nvjpg
+
+ resets:
+ maxItems: 1
+
+ reset-names:
+ items:
+ - const: nvjpg
+
+ power-domains:
+ maxItems: 1
+
+ iommus:
+ maxItems: 1
+
+ dma-coherent: true
+
+ interconnects:
+ items:
+ - description: DMA read memory client
+ - description: DMA write memory client
+
+ interconnect-names:
+ items:
+ - const: dma-mem
+ - const: write
+
+required:
+ - compatible
+ - reg
+ - clocks
+ - clock-names
+ - resets
+ - reset-names
+ - power-domains
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/clock/tegra186-clock.h>
+ #include <dt-bindings/memory/tegra186-mc.h>
+ #include <dt-bindings/power/tegra186-powergate.h>
+ #include <dt-bindings/reset/tegra186-reset.h>
+
+ nvjpg@15380000 {
+ compatible = "nvidia,tegra186-nvjpg";
+ reg = <0x15380000 0x40000>;
+ clocks = <&bpmp TEGRA186_CLK_NVJPG>;
+ clock-names = "nvjpg";
+ resets = <&bpmp TEGRA186_RESET_NVJPG>;
+ reset-names = "nvjpg";
+
+ power-domains = <&bpmp TEGRA186_POWER_DOMAIN_NVJPG>;
+ interconnects = <&mc TEGRA186_MEMORY_CLIENT_NVJPGSRD &emc>,
+ <&mc TEGRA186_MEMORY_CLIENT_NVJPGSWR &emc>;
+ interconnect-names = "dma-mem", "write";
+ iommus = <&smmu TEGRA186_SID_NVJPG>;
+ };
diff --git a/Documentation/devicetree/bindings/i2c/apple,i2c.yaml b/Documentation/devicetree/bindings/i2c/apple,i2c.yaml
index 82b9531..4ac61fe 100644
--- a/Documentation/devicetree/bindings/i2c/apple,i2c.yaml
+++ b/Documentation/devicetree/bindings/i2c/apple,i2c.yaml
@@ -21,7 +21,9 @@
properties:
compatible:
items:
- - const: apple,t8103-i2c
+ - enum:
+ - apple,t8103-i2c
+ - apple,t6000-i2c
- const: apple,i2c
reg:
@@ -40,6 +42,9 @@
used. This frequency is generated by dividing the reference clock.
Allowed values are between ref_clk/(16*4) and ref_clk/(16*255).
+ power-domains:
+ maxItems: 1
+
required:
- compatible
- reg
diff --git a/Documentation/devicetree/bindings/interrupt-controller/apple,aic.yaml b/Documentation/devicetree/bindings/interrupt-controller/apple,aic.yaml
index cf6c091..9735902 100644
--- a/Documentation/devicetree/bindings/interrupt-controller/apple,aic.yaml
+++ b/Documentation/devicetree/bindings/interrupt-controller/apple,aic.yaml
@@ -65,6 +65,9 @@
Specifies base physical address and size of the AIC registers.
maxItems: 1
+ power-domains:
+ maxItems: 1
+
required:
- compatible
- '#interrupt-cells'
diff --git a/Documentation/devicetree/bindings/interrupt-controller/sifive,plic-1.0.0.yaml b/Documentation/devicetree/bindings/interrupt-controller/sifive,plic-1.0.0.yaml
index 08d5a57..28b6b17 100644
--- a/Documentation/devicetree/bindings/interrupt-controller/sifive,plic-1.0.0.yaml
+++ b/Documentation/devicetree/bindings/interrupt-controller/sifive,plic-1.0.0.yaml
@@ -45,6 +45,7 @@
items:
- enum:
- sifive,fu540-c000-plic
+ - starfive,jh7100-plic
- canaan,k210-plic
- const: sifive,plic-1.0.0
diff --git a/Documentation/devicetree/bindings/iommu/apple,dart.yaml b/Documentation/devicetree/bindings/iommu/apple,dart.yaml
index 94aa9e9..82ad669 100644
--- a/Documentation/devicetree/bindings/iommu/apple,dart.yaml
+++ b/Documentation/devicetree/bindings/iommu/apple,dart.yaml
@@ -41,6 +41,9 @@
Has to be one. The single cell describes the stream id emitted by
a master to the IOMMU.
+ power-domains:
+ maxItems: 1
+
required:
- compatible
- reg
diff --git a/Documentation/devicetree/bindings/ipmi/aspeed,ast2400-ibt-bmc.txt b/Documentation/devicetree/bindings/ipmi/aspeed,ast2400-ibt-bmc.txt
index c9902fd..25f86da 100644
--- a/Documentation/devicetree/bindings/ipmi/aspeed,ast2400-ibt-bmc.txt
+++ b/Documentation/devicetree/bindings/ipmi/aspeed,ast2400-ibt-bmc.txt
@@ -11,6 +11,7 @@
"aspeed,ast2500-ibt-bmc"
"aspeed,ast2600-ibt-bmc"
- reg: physical address and size of the registers
+- clocks: clock for the device
Optional properties:
@@ -23,4 +24,5 @@
compatible = "aspeed,ast2400-ibt-bmc";
reg = <0x1e789140 0x18>;
interrupts = <8>;
+ clocks = <&syscon ASPEED_CLK_GATE_LCLK>;
};
diff --git a/Documentation/devicetree/bindings/mailbox/apple,mailbox.yaml b/Documentation/devicetree/bindings/mailbox/apple,mailbox.yaml
index 2c1704b..c4255f4 100644
--- a/Documentation/devicetree/bindings/mailbox/apple,mailbox.yaml
+++ b/Documentation/devicetree/bindings/mailbox/apple,mailbox.yaml
@@ -56,6 +56,9 @@
"#mbox-cells":
const: 0
+ power-domains:
+ maxItems: 1
+
required:
- compatible
- reg
diff --git a/Documentation/devicetree/bindings/mailbox/nvidia,tegra186-hsp.txt b/Documentation/devicetree/bindings/mailbox/nvidia,tegra186-hsp.txt
deleted file mode 100644
index ff3eafc..0000000
--- a/Documentation/devicetree/bindings/mailbox/nvidia,tegra186-hsp.txt
+++ /dev/null
@@ -1,72 +0,0 @@
-NVIDIA Tegra Hardware Synchronization Primitives (HSP)
-
-The HSP modules are used for the processors to share resources and communicate
-together. It provides a set of hardware synchronization primitives for
-interprocessor communication. So the interprocessor communication (IPC)
-protocols can use hardware synchronization primitives, when operating between
-two processors not in an SMP relationship.
-
-The features that HSP supported are shared mailboxes, shared semaphores,
-arbitrated semaphores and doorbells.
-
-Required properties:
-- name : Should be hsp
-- compatible
- Array of strings.
- one of:
- - "nvidia,tegra186-hsp"
- - "nvidia,tegra194-hsp", "nvidia,tegra186-hsp"
-- reg : Offset and length of the register set for the device.
-- interrupt-names
- Array of strings.
- Contains a list of names for the interrupts described by the interrupt
- property. May contain the following entries, in any order:
- - "doorbell"
- - "sharedN", where 'N' is a number from zero up to the number of
- external interrupts supported by the HSP instance minus one.
- Users of this binding MUST look up entries in the interrupt property
- by name, using this interrupt-names property to do so.
-- interrupts
- Array of interrupt specifiers.
- Must contain one entry per entry in the interrupt-names property,
- in a matching order.
-- #mbox-cells : Should be 2.
-
-The mbox specifier of the "mboxes" property in the client node should contain
-two cells. The first cell determines the HSP type and the second cell is used
-to identify the mailbox that the client is going to use.
-
-For doorbells, the second cell specifies the index of the doorbell to use.
-
-For shared mailboxes, the second cell is composed of two fields:
-- bits 31..24:
- A bit mask of flags that further specify how the shared mailbox will be
- used. Valid flags are:
- - bit 31:
- Defines the direction of the mailbox. If set, the mailbox will be used
- as a producer (i.e. used to send data). If cleared, the mailbox is the
- consumer of data sent by a producer.
-
-- bits 23.. 0:
- The index of the shared mailbox to use. The number of available mailboxes
- may vary by instance of the HSP block and SoC generation.
-
-The following file contains definitions that can be used to construct mailbox
-specifiers:
-
- <dt-bindings/mailbox/tegra186-hsp.h>
-
-Example:
-
-hsp_top0: hsp@3c00000 {
- compatible = "nvidia,tegra186-hsp";
- reg = <0x0 0x03c00000 0x0 0xa0000>;
- interrupts = <GIC_SPI 176 IRQ_TYPE_LEVEL_HIGH>;
- interrupt-names = "doorbell";
- #mbox-cells = <2>;
-};
-
-client {
- ...
- mboxes = <&hsp_top0 TEGRA_HSP_MBOX_TYPE_DB TEGRA_HSP_DB_MASTER_XXX>;
-};
diff --git a/Documentation/devicetree/bindings/mailbox/nvidia,tegra186-hsp.yaml b/Documentation/devicetree/bindings/mailbox/nvidia,tegra186-hsp.yaml
new file mode 100644
index 0000000..9f7a729
--- /dev/null
+++ b/Documentation/devicetree/bindings/mailbox/nvidia,tegra186-hsp.yaml
@@ -0,0 +1,114 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/mailbox/nvidia,tegra186-hsp.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: NVIDIA Tegra Hardware Synchronization Primitives (HSP)
+
+maintainers:
+ - Thierry Reding <thierry.reding@gmail.com>
+ - Jon Hunter <jonathanh@nvidia.com>
+
+description: |
+ The HSP modules are used for the processors to share resources and
+ communicate together. It provides a set of hardware synchronization
+ primitives for interprocessor communication. So the interprocessor
+ communication (IPC) protocols can use hardware synchronization
+ primitives, when operating between two processors not in an SMP
+ relationship.
+
+ The features that HSP supported are shared mailboxes, shared
+ semaphores, arbitrated semaphores and doorbells.
+
+ The mbox specifier of the "mboxes" property in the client node should
+ contain two cells. The first cell determines the HSP type and the
+ second cell is used to identify the mailbox that the client is going
+ to use.
+
+ For doorbells, the second cell specifies the index of the doorbell to
+ use.
+
+ For shared mailboxes, the second cell is composed of two fields:
+ - bits 31..24:
+ A bit mask of flags that further specify how the shared mailbox
+ will be used. Valid flags are:
+ - bit 31:
+ Defines the direction of the mailbox. If set, the mailbox
+ will be used as a producer (i.e. used to send data). If
+ cleared, the mailbox is the consumer of data sent by a
+ producer.
+
+ - bits 23..0:
+ The index of the shared mailbox to use. The number of available
+ mailboxes may vary by instance of the HSP block and SoC
+ generation.
+
+ The following file contains definitions that can be used to
+ construct mailbox specifiers:
+
+ <dt-bindings/mailbox/tegra186-hsp.h>
+
+properties:
+ $nodename:
+ pattern: "^hsp@[0-9a-f]+$"
+
+ compatible:
+ oneOf:
+ - const: nvidia,tegra186-hsp
+ - const: nvidia,tegra194-hsp
+ - items:
+ - const: nvidia,tegra234-hsp
+ - const: nvidia,tegra194-hsp
+
+ reg:
+ maxItems: 1
+
+ interrupts:
+ minItems: 1
+ maxItems: 9
+
+ interrupt-names:
+ oneOf:
+ # shared interrupts are optional
+ - items:
+ - const: doorbell
+
+ - items:
+ - const: doorbell
+ - pattern: "^shared[0-7]$"
+ - pattern: "^shared[0-7]$"
+ - pattern: "^shared[0-7]$"
+ - pattern: "^shared[0-7]$"
+ - pattern: "^shared[0-7]$"
+ - pattern: "^shared[0-7]$"
+ - pattern: "^shared[0-7]$"
+ - pattern: "^shared[0-7]$"
+
+ - items:
+ - pattern: "^shared[0-7]$"
+ - pattern: "^shared[0-7]$"
+ - pattern: "^shared[0-7]$"
+ - pattern: "^shared[0-7]$"
+
+ "#mbox-cells":
+ const: 2
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+ #include <dt-bindings/mailbox/tegra186-hsp.h>
+
+ hsp_top0: hsp@3c00000 {
+ compatible = "nvidia,tegra186-hsp";
+ reg = <0x03c00000 0xa0000>;
+ interrupts = <GIC_SPI 176 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "doorbell";
+ #mbox-cells = <2>;
+ };
+
+ client {
+ mboxes = <&hsp_top0 TEGRA_HSP_MBOX_TYPE_DB TEGRA_HSP_DB_MASTER_CCPLEX>;
+ };
diff --git a/Documentation/devicetree/bindings/media/allwinner,sun4i-a10-video-engine.yaml b/Documentation/devicetree/bindings/media/allwinner,sun4i-a10-video-engine.yaml
index c34303b..c3de96d 100644
--- a/Documentation/devicetree/bindings/media/allwinner,sun4i-a10-video-engine.yaml
+++ b/Documentation/devicetree/bindings/media/allwinner,sun4i-a10-video-engine.yaml
@@ -20,6 +20,7 @@
- allwinner,sun8i-h3-video-engine
- allwinner,sun8i-v3s-video-engine
- allwinner,sun8i-r40-video-engine
+ - allwinner,sun20i-d1-video-engine
- allwinner,sun50i-a64-video-engine
- allwinner,sun50i-h5-video-engine
- allwinner,sun50i-h6-video-engine
diff --git a/Documentation/devicetree/bindings/media/allwinner,sun50i-h6-vpu-g2.yaml b/Documentation/devicetree/bindings/media/allwinner,sun50i-h6-vpu-g2.yaml
new file mode 100644
index 0000000..24d7bf2
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/allwinner,sun50i-h6-vpu-g2.yaml
@@ -0,0 +1,64 @@
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+
+%YAML 1.2
+---
+$id: "http://devicetree.org/schemas/media/allwinner,sun50i-h6-vpu-g2.yaml#"
+$schema: "http://devicetree.org/meta-schemas/core.yaml#"
+
+title: Hantro G2 VPU codec implemented on Allwinner H6 SoC
+
+maintainers:
+ - Jernej Skrabec <jernej.skrabec@gmail.com>
+
+description:
+ Hantro G2 video decode accelerator present on Allwinner H6 SoC.
+
+properties:
+ compatible:
+ const: allwinner,sun50i-h6-vpu-g2
+
+ reg:
+ maxItems: 1
+
+ interrupts:
+ maxItems: 1
+
+ clocks:
+ items:
+ - description: Bus Clock
+ - description: Module Clock
+
+ clock-names:
+ items:
+ - const: bus
+ - const: mod
+
+ resets:
+ maxItems: 1
+
+required:
+ - compatible
+ - reg
+ - interrupts
+ - clocks
+ - clock-names
+ - resets
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+ #include <dt-bindings/clock/sun50i-h6-ccu.h>
+ #include <dt-bindings/reset/sun50i-h6-ccu.h>
+
+ video-codec-g2@1c00000 {
+ compatible = "allwinner,sun50i-h6-vpu-g2";
+ reg = <0x01c00000 0x1000>;
+ interrupts = <GIC_SPI 90 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&ccu CLK_BUS_VP9>, <&ccu CLK_VP9>;
+ clock-names = "bus", "mod";
+ resets = <&ccu RST_BUS_VP9>;
+ };
+
+...
diff --git a/Documentation/devicetree/bindings/media/i2c/adv748x.txt b/Documentation/devicetree/bindings/media/i2c/adv748x.txt
deleted file mode 100644
index 4f91686..0000000
--- a/Documentation/devicetree/bindings/media/i2c/adv748x.txt
+++ /dev/null
@@ -1,116 +0,0 @@
-* Analog Devices ADV748X video decoder with HDMI receiver
-
-The ADV7481 and ADV7482 are multi format video decoders with an integrated
-HDMI receiver. They can output CSI-2 on two independent outputs TXA and TXB
-from three input sources HDMI, analog and TTL.
-
-Required Properties:
-
- - compatible: Must contain one of the following
- - "adi,adv7481" for the ADV7481
- - "adi,adv7482" for the ADV7482
-
- - reg: I2C slave addresses
- The ADV748x has up to twelve 256-byte maps that can be accessed via the
- main I2C ports. Each map has it own I2C address and acts as a standard
- slave device on the I2C bus. The main address is mandatory, others are
- optional and remain at default values if not specified.
-
-Optional Properties:
-
- - interrupt-names: Should specify the interrupts as "intrq1", "intrq2" and/or
- "intrq3". All interrupts are optional. The "intrq3" interrupt
- is only available on the adv7481
- - interrupts: Specify the interrupt lines for the ADV748x
- - reg-names : Names of maps with programmable addresses.
- It shall contain all maps needing a non-default address.
- Possible map names are:
- "main", "dpll", "cp", "hdmi", "edid", "repeater",
- "infoframe", "cbus", "cec", "sdp", "txa", "txb"
-
-The device node must contain one 'port' child node per device input and output
-port, in accordance with the video interface bindings defined in
-Documentation/devicetree/bindings/media/video-interfaces.txt. The port nodes
-are numbered as follows.
-
- Name Type Port
- ---------------------------------------
- AIN0 sink 0
- AIN1 sink 1
- AIN2 sink 2
- AIN3 sink 3
- AIN4 sink 4
- AIN5 sink 5
- AIN6 sink 6
- AIN7 sink 7
- HDMI sink 8
- TTL sink 9
- TXA source 10
- TXB source 11
-
-The digital output port nodes, when present, shall contain at least one
-endpoint. Each of those endpoints shall contain the data-lanes property as
-described in video-interfaces.txt.
-
-Required source endpoint properties:
- - data-lanes: an array of physical data lane indexes
- The accepted value(s) for this property depends on which of the two
- sources are described. For TXA 1, 2 or 4 data lanes can be described
- while for TXB only 1 data lane is valid. See video-interfaces.txt
- for detailed description.
-
-Ports are optional if they are not connected to anything at the hardware level.
-
-Example:
-
- video-receiver@70 {
- compatible = "adi,adv7482";
- reg = <0x70 0x71 0x72 0x73 0x74 0x75
- 0x60 0x61 0x62 0x63 0x64 0x65>;
- reg-names = "main", "dpll", "cp", "hdmi", "edid", "repeater",
- "infoframe", "cbus", "cec", "sdp", "txa", "txb";
-
- #address-cells = <1>;
- #size-cells = <0>;
-
- interrupt-parent = <&gpio6>;
- interrupt-names = "intrq1", "intrq2";
- interrupts = <30 IRQ_TYPE_LEVEL_LOW>,
- <31 IRQ_TYPE_LEVEL_LOW>;
-
- port@7 {
- reg = <7>;
-
- adv7482_ain7: endpoint {
- remote-endpoint = <&cvbs_in>;
- };
- };
-
- port@8 {
- reg = <8>;
-
- adv7482_hdmi: endpoint {
- remote-endpoint = <&hdmi_in>;
- };
- };
-
- port@a {
- reg = <10>;
-
- adv7482_txa: endpoint {
- clock-lanes = <0>;
- data-lanes = <1 2 3 4>;
- remote-endpoint = <&csi40_in>;
- };
- };
-
- port@b {
- reg = <11>;
-
- adv7482_txb: endpoint {
- clock-lanes = <0>;
- data-lanes = <1>;
- remote-endpoint = <&csi20_in>;
- };
- };
- };
diff --git a/Documentation/devicetree/bindings/media/i2c/adv748x.yaml b/Documentation/devicetree/bindings/media/i2c/adv748x.yaml
new file mode 100644
index 0000000..d635308
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/i2c/adv748x.yaml
@@ -0,0 +1,212 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/media/i2c/adv748x.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Analog Devices ADV748X video decoder with HDMI receiver
+
+maintainers:
+ - Kieran Bingham <kieran.bingham@ideasonboard.com>
+ - Niklas Söderlund <niklas.soderlund@ragnatech.se>
+
+description:
+ The ADV7481 and ADV7482 are multi format video decoders with an integrated
+ HDMI receiver. They can output CSI-2 on two independent outputs TXA and TXB
+ from three input sources HDMI, analog and TTL.
+
+properties:
+ compatible:
+ items:
+ - enum:
+ - adi,adv7481
+ - adi,adv7482
+
+ reg:
+ minItems: 1
+ maxItems: 12
+ description:
+ The ADV748x has up to twelve 256-byte maps that can be accessed via the
+ main I2C ports. Each map has it own I2C address and acts as a standard
+ slave device on the I2C bus. The main address is mandatory, others are
+ optional and remain at default values if not specified.
+
+ reg-names:
+ minItems: 1
+ items:
+ - const: main
+ - enum: [ dpll, cp, hdmi, edid, repeater, infoframe, cbus, cec, sdp, txa, txb ]
+ - enum: [ dpll, cp, hdmi, edid, repeater, infoframe, cbus, cec, sdp, txa, txb ]
+ - enum: [ dpll, cp, hdmi, edid, repeater, infoframe, cbus, cec, sdp, txa, txb ]
+ - enum: [ dpll, cp, hdmi, edid, repeater, infoframe, cbus, cec, sdp, txa, txb ]
+ - enum: [ dpll, cp, hdmi, edid, repeater, infoframe, cbus, cec, sdp, txa, txb ]
+ - enum: [ dpll, cp, hdmi, edid, repeater, infoframe, cbus, cec, sdp, txa, txb ]
+ - enum: [ dpll, cp, hdmi, edid, repeater, infoframe, cbus, cec, sdp, txa, txb ]
+ - enum: [ dpll, cp, hdmi, edid, repeater, infoframe, cbus, cec, sdp, txa, txb ]
+ - enum: [ dpll, cp, hdmi, edid, repeater, infoframe, cbus, cec, sdp, txa, txb ]
+ - enum: [ dpll, cp, hdmi, edid, repeater, infoframe, cbus, cec, sdp, txa, txb ]
+ - enum: [ dpll, cp, hdmi, edid, repeater, infoframe, cbus, cec, sdp, txa, txb ]
+
+ interrupts: true
+
+ interrupt-names: true
+
+ ports:
+ $ref: /schemas/graph.yaml#/properties/ports
+
+ patternProperties:
+ "^port@[0-7]$":
+ $ref: /schemas/graph.yaml#/properties/port
+ description: Input port nodes for analog inputs AIN[0-7].
+
+ properties:
+ port@8:
+ $ref: /schemas/graph.yaml#/properties/port
+ description: Input port node for HDMI.
+
+ port@9:
+ $ref: /schemas/graph.yaml#/properties/port
+ description: Input port node for TTL.
+
+ port@a:
+ $ref: /schemas/graph.yaml#/$defs/port-base
+ unevaluatedProperties: false
+ description:
+ Output port node, single endpoint describing the CSI-2 transmitter TXA.
+
+ properties:
+ endpoint:
+ $ref: /schemas/media/video-interfaces.yaml#
+ unevaluatedProperties: false
+
+ properties:
+ clock-lanes:
+ maxItems: 1
+
+ data-lanes:
+ minItems: 1
+ maxItems: 4
+
+ required:
+ - clock-lanes
+ - data-lanes
+
+ port@b:
+ $ref: /schemas/graph.yaml#/$defs/port-base
+ unevaluatedProperties: false
+ description:
+ Output port node, single endpoint describing the CSI-2 transmitter TXB.
+
+ properties:
+ endpoint:
+ $ref: /schemas/media/video-interfaces.yaml#
+ unevaluatedProperties: false
+
+ properties:
+ clock-lanes:
+ maxItems: 1
+
+ data-lanes:
+ maxItems: 1
+
+ required:
+ - clock-lanes
+ - data-lanes
+
+allOf:
+ - if:
+ properties:
+ compatible:
+ contains:
+ const: adi,adv7481
+ then:
+ properties:
+ interrupts:
+ minItems: 1
+ maxItems: 3
+
+ interrupt-names:
+ minItems: 1
+ maxItems: 3
+ items:
+ enum: [ intrq1, intrq2, intrq3 ]
+ else:
+ properties:
+ interrupts:
+ minItems: 1
+ maxItems: 2
+
+ interrupt-names:
+ minItems: 1
+ maxItems: 2
+ items:
+ enum: [ intrq1, intrq2 ]
+
+additionalProperties: false
+
+required:
+ - compatible
+ - reg
+ - ports
+
+examples:
+ - |
+ #include <dt-bindings/interrupt-controller/irq.h>
+
+ i2c {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ video-receiver@70 {
+ compatible = "adi,adv7482";
+ reg = <0x70 0x71 0x72 0x73 0x74 0x75
+ 0x60 0x61 0x62 0x63 0x64 0x65>;
+ reg-names = "main", "dpll", "cp", "hdmi", "edid", "repeater",
+ "infoframe", "cbus", "cec", "sdp", "txa", "txb";
+
+ interrupt-parent = <&gpio6>;
+ interrupts = <30 IRQ_TYPE_LEVEL_LOW>, <31 IRQ_TYPE_LEVEL_LOW>;
+ interrupt-names = "intrq1", "intrq2";
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@7 {
+ reg = <7>;
+
+ adv7482_ain7: endpoint {
+ remote-endpoint = <&cvbs_in>;
+ };
+ };
+
+ port@8 {
+ reg = <8>;
+
+ adv7482_hdmi: endpoint {
+ remote-endpoint = <&hdmi_in>;
+ };
+ };
+
+ port@a {
+ reg = <10>;
+
+ adv7482_txa: endpoint {
+ clock-lanes = <0>;
+ data-lanes = <1 2 3 4>;
+ remote-endpoint = <&csi40_in>;
+ };
+ };
+
+ port@b {
+ reg = <11>;
+
+ adv7482_txb: endpoint {
+ clock-lanes = <0>;
+ data-lanes = <1>;
+ remote-endpoint = <&csi20_in>;
+ };
+ };
+ };
+ };
+ };
diff --git a/Documentation/devicetree/bindings/media/nvidia,tegra-vde.txt b/Documentation/devicetree/bindings/media/nvidia,tegra-vde.txt
deleted file mode 100644
index 602169b..0000000
--- a/Documentation/devicetree/bindings/media/nvidia,tegra-vde.txt
+++ /dev/null
@@ -1,64 +0,0 @@
-NVIDIA Tegra Video Decoder Engine
-
-Required properties:
-- compatible : Must contain one of the following values:
- - "nvidia,tegra20-vde"
- - "nvidia,tegra30-vde"
- - "nvidia,tegra114-vde"
- - "nvidia,tegra124-vde"
- - "nvidia,tegra132-vde"
-- reg : Must contain an entry for each entry in reg-names.
-- reg-names : Must include the following entries:
- - sxe
- - bsev
- - mbe
- - ppe
- - mce
- - tfe
- - ppb
- - vdma
- - frameid
-- iram : Must contain phandle to the mmio-sram device node that represents
- IRAM region used by VDE.
-- interrupts : Must contain an entry for each entry in interrupt-names.
-- interrupt-names : Must include the following entries:
- - sync-token
- - bsev
- - sxe
-- clocks : Must include the following entries:
- - vde
-- resets : Must contain an entry for each entry in reset-names.
-- reset-names : Should include the following entries:
- - vde
-
-Optional properties:
-- resets : Must contain an entry for each entry in reset-names.
-- reset-names : Must include the following entries:
- - mc
-- iommus: Must contain phandle to the IOMMU device node.
-
-Example:
-
-video-codec@6001a000 {
- compatible = "nvidia,tegra20-vde";
- reg = <0x6001a000 0x1000 /* Syntax Engine */
- 0x6001b000 0x1000 /* Video Bitstream Engine */
- 0x6001c000 0x100 /* Macroblock Engine */
- 0x6001c200 0x100 /* Post-processing Engine */
- 0x6001c400 0x100 /* Motion Compensation Engine */
- 0x6001c600 0x100 /* Transform Engine */
- 0x6001c800 0x100 /* Pixel prediction block */
- 0x6001ca00 0x100 /* Video DMA */
- 0x6001d800 0x300 /* Video frame controls */>;
- reg-names = "sxe", "bsev", "mbe", "ppe", "mce",
- "tfe", "ppb", "vdma", "frameid";
- iram = <&vde_pool>; /* IRAM region */
- interrupts = <GIC_SPI 9 IRQ_TYPE_LEVEL_HIGH>, /* Sync token interrupt */
- <GIC_SPI 10 IRQ_TYPE_LEVEL_HIGH>, /* BSE-V interrupt */
- <GIC_SPI 12 IRQ_TYPE_LEVEL_HIGH>; /* SXE interrupt */
- interrupt-names = "sync-token", "bsev", "sxe";
- clocks = <&tegra_car TEGRA20_CLK_VDE>;
- reset-names = "vde", "mc";
- resets = <&tegra_car 61>, <&mc TEGRA20_MC_RESET_VDE>;
- iommus = <&mc TEGRA_SWGROUP_VDE>;
-};
diff --git a/Documentation/devicetree/bindings/media/nvidia,tegra-vde.yaml b/Documentation/devicetree/bindings/media/nvidia,tegra-vde.yaml
new file mode 100644
index 0000000..4ecdee1
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/nvidia,tegra-vde.yaml
@@ -0,0 +1,119 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/media/nvidia,tegra-vde.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: NVIDIA Tegra Video Decoder Engine
+
+maintainers:
+ - Dmitry Osipenko <digetx@gmail.com>
+ - Jon Hunter <jonathanh@nvidia.com>
+ - Thierry Reding <thierry.reding@gmail.com>
+
+properties:
+ compatible:
+ oneOf:
+ - items:
+ - enum:
+ - nvidia,tegra132-vde
+ - nvidia,tegra124-vde
+ - nvidia,tegra114-vde
+ - items:
+ - const: nvidia,tegra30-vde
+ - const: nvidia,tegra20-vde
+ - items:
+ - const: nvidia,tegra20-vde
+
+ reg:
+ maxItems: 9
+
+ reg-names:
+ items:
+ - const: sxe
+ - const: bsev
+ - const: mbe
+ - const: ppe
+ - const: mce
+ - const: tfe
+ - const: ppb
+ - const: vdma
+ - const: frameid
+
+ clocks:
+ maxItems: 1
+
+ resets:
+ maxItems: 2
+
+ reset-names:
+ items:
+ - const: vde
+ - const: mc
+
+ interrupts:
+ maxItems: 3
+
+ interrupt-names:
+ items:
+ - const: sync-token
+ - const: bsev
+ - const: sxe
+
+ iommus:
+ maxItems: 1
+
+ iram:
+ $ref: /schemas/types.yaml#/definitions/phandle
+ description:
+ Phandle of the SRAM MMIO node.
+
+ operating-points-v2:
+ description:
+ Should contain freqs and voltages and opp-supported-hw property,
+ which is a bitfield indicating SoC speedo or process ID mask.
+
+ power-domains:
+ maxItems: 1
+ description:
+ Phandle to the SoC core power domain.
+
+required:
+ - compatible
+ - reg
+ - reg-names
+ - clocks
+ - resets
+ - reset-names
+ - interrupts
+ - interrupt-names
+
+additionalProperties: false
+
+examples:
+ - |
+ video-codec@6001a000 {
+ compatible = "nvidia,tegra20-vde";
+ reg = <0x6001a000 0x1000>, /* Syntax Engine */
+ <0x6001b000 0x1000>, /* Video Bitstream Engine */
+ <0x6001c000 0x100>, /* Macroblock Engine */
+ <0x6001c200 0x100>, /* Post-processing Engine */
+ <0x6001c400 0x100>, /* Motion Compensation Engine */
+ <0x6001c600 0x100>, /* Transform Engine */
+ <0x6001c800 0x100>, /* Pixel prediction block */
+ <0x6001ca00 0x100>, /* Video DMA */
+ <0x6001d800 0x300>; /* Video frame controls */
+ reg-names = "sxe", "bsev", "mbe", "ppe", "mce",
+ "tfe", "ppb", "vdma", "frameid";
+ iram = <&iram>; /* IRAM MMIO region */
+ interrupts = <0 9 4>, /* Sync token */
+ <0 10 4>, /* BSE-V */
+ <0 12 4>; /* SXE */
+ interrupt-names = "sync-token", "bsev", "sxe";
+ clocks = <&clk 61>;
+ reset-names = "vde", "mc";
+ resets = <&rst 61>, <&mem 13>;
+ iommus = <&mem 15>;
+ operating-points-v2 = <&dvfs_opp_table>;
+ power-domains = <&domain>;
+ };
diff --git a/Documentation/devicetree/bindings/media/renesas,jpu.txt b/Documentation/devicetree/bindings/media/renesas,jpu.txt
deleted file mode 100644
index d3436e5..0000000
--- a/Documentation/devicetree/bindings/media/renesas,jpu.txt
+++ /dev/null
@@ -1,25 +0,0 @@
-* Renesas JPEG Processing Unit
-
-The JPEG processing unit (JPU) incorporates the JPEG codec with an encoding
-and decoding function conforming to the JPEG baseline process, so that the JPU
-can encode image data and decode JPEG data quickly.
-
-Required properties:
-- compatible: "renesas,jpu-<soctype>", "renesas,rcar-gen2-jpu" as fallback.
- Examples with soctypes are:
- - "renesas,jpu-r8a7790" for R-Car H2
- - "renesas,jpu-r8a7791" for R-Car M2-W
- - "renesas,jpu-r8a7792" for R-Car V2H
- - "renesas,jpu-r8a7793" for R-Car M2-N
-
- - reg: Base address and length of the registers block for the JPU.
- - interrupts: JPU interrupt specifier.
- - clocks: A phandle + clock-specifier pair for the JPU functional clock.
-
-Example: R8A7790 (R-Car H2) JPU node
- jpeg-codec@fe980000 {
- compatible = "renesas,jpu-r8a7790", "renesas,rcar-gen2-jpu";
- reg = <0 0xfe980000 0 0x10300>;
- interrupts = <0 272 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&mstp1_clks R8A7790_CLK_JPU>;
- };
diff --git a/Documentation/devicetree/bindings/media/renesas,jpu.yaml b/Documentation/devicetree/bindings/media/renesas,jpu.yaml
new file mode 100644
index 0000000..2f55968
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/renesas,jpu.yaml
@@ -0,0 +1,65 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/media/renesas,jpu.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Renesas JPEG Processing Unit
+
+maintainers:
+ - Mikhail Ulyanov <mikhail.ulyanov@cogentembedded.com>
+
+description:
+ The JPEG processing unit (JPU) incorporates the JPEG codec with an encoding
+ and decoding function conforming to the JPEG baseline process, so that the
+ JPU can encode image data and decode JPEG data quickly.
+
+properties:
+ compatible:
+ items:
+ - enum:
+ - renesas,jpu-r8a7790 # R-Car H2
+ - renesas,jpu-r8a7791 # R-Car M2-W
+ - renesas,jpu-r8a7792 # R-Car V2H
+ - renesas,jpu-r8a7793 # R-Car M2-N
+ - const: renesas,rcar-gen2-jpu # R-Car Gen2
+
+ reg:
+ maxItems: 1
+
+ interrupts:
+ maxItems: 1
+
+ clocks:
+ maxItems: 1
+
+ power-domains:
+ maxItems: 1
+
+ resets:
+ maxItems: 1
+
+required:
+ - compatible
+ - reg
+ - interrupts
+ - clocks
+ - power-domains
+ - resets
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/clock/r8a7790-cpg-mssr.h>
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+ #include <dt-bindings/power/r8a7790-sysc.h>
+
+ jpeg-codec@fe980000 {
+ compatible = "renesas,jpu-r8a7790", "renesas,rcar-gen2-jpu";
+ reg = <0xfe980000 0x10300>;
+ interrupts = <GIC_SPI 272 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&cpg CPG_MOD 106>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
+ resets = <&cpg 106>;
+ };
diff --git a/Documentation/devicetree/bindings/media/st,stm32-dma2d.yaml b/Documentation/devicetree/bindings/media/st,stm32-dma2d.yaml
new file mode 100644
index 0000000..f97b4a2
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/st,stm32-dma2d.yaml
@@ -0,0 +1,71 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/media/st,stm32-dma2d.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: STMicroelectronics STM32 Chrom-Art Accelerator DMA2D binding
+
+description:
+ Chrom-ART Accelerator(DMA2D), graphical hardware accelerator
+ enabling enhanced graphical user interface with minimum CPU load
+
+ It can perform the following operations.
+
+ - Filling a part or the whole of a destination image with a specific color.
+ - Copying a part or the whole of a source image into a part or the whole of
+ a destination image.
+ - Copying a part or the whole of a source image into a part or the whole of
+ a destination image with a pixel format conversion.
+ - Blending a part and/or two complete source images with different pixel
+ format and copy the result into a part or the whole of a destination image
+ with a different color format. (TODO)
+
+
+maintainers:
+ - Dillon Min <dillon.minfei@gmail.com>
+
+properties:
+ compatible:
+ const: st,stm32-dma2d
+
+ reg:
+ maxItems: 1
+
+ interrupts:
+ maxItems: 1
+
+ clocks:
+ maxItems: 1
+
+ clock-names:
+ items:
+ - const: dma2d
+
+ resets:
+ maxItems: 1
+
+required:
+ - compatible
+ - reg
+ - interrupts
+ - clocks
+ - clock-names
+ - resets
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/clock/stm32fx-clock.h>
+ #include <dt-bindings/mfd/stm32f4-rcc.h>
+ dma2d: dma2d@4002b000 {
+ compatible = "st,stm32-dma2d";
+ reg = <0x4002b000 0xc00>;
+ interrupts = <90>;
+ resets = <&rcc STM32F4_AHB1_RESET(DMA2D)>;
+ clocks = <&rcc 0 STM32F4_AHB1_CLOCK(DMA2D)>;
+ clock-names = "dma2d";
+ };
+
+...
diff --git a/Documentation/devicetree/bindings/memory-controllers/nvidia,tegra186-mc.yaml b/Documentation/devicetree/bindings/memory-controllers/nvidia,tegra186-mc.yaml
index 611bda3..13c4c82 100644
--- a/Documentation/devicetree/bindings/memory-controllers/nvidia,tegra186-mc.yaml
+++ b/Documentation/devicetree/bindings/memory-controllers/nvidia,tegra186-mc.yaml
@@ -31,12 +31,15 @@
- enum:
- nvidia,tegra186-mc
- nvidia,tegra194-mc
+ - nvidia,tegra234-mc
reg:
- maxItems: 1
+ minItems: 1
+ maxItems: 3
interrupts:
- maxItems: 1
+ items:
+ - description: MC general interrupt
"#address-cells":
const: 2
@@ -48,6 +51,9 @@
dma-ranges: true
+ "#interconnect-cells":
+ const: 1
+
patternProperties:
"^external-memory-controller@[0-9a-f]+$":
description:
@@ -63,12 +69,15 @@
- enum:
- nvidia,tegra186-emc
- nvidia,tegra194-emc
+ - nvidia,tegra234-emc
reg:
- maxItems: 1
+ minItems: 1
+ maxItems: 2
interrupts:
- maxItems: 1
+ items:
+ - description: EMC general interrupt
clocks:
items:
@@ -78,11 +87,83 @@
items:
- const: emc
+ "#interconnect-cells":
+ const: 0
+
nvidia,bpmp:
$ref: /schemas/types.yaml#/definitions/phandle
description:
phandle of the node representing the BPMP
+ allOf:
+ - if:
+ properties:
+ compatible:
+ const: nvidia,tegra186-emc
+ then:
+ properties:
+ reg:
+ maxItems: 1
+
+ - if:
+ properties:
+ compatible:
+ const: nvidia,tegra194-emc
+ then:
+ properties:
+ reg:
+ minItems: 2
+
+ - if:
+ properties:
+ compatible:
+ const: nvidia,tegra234-emc
+ then:
+ properties:
+ reg:
+ minItems: 2
+
+ additionalProperties: false
+
+ required:
+ - compatible
+ - reg
+ - interrupts
+ - clocks
+ - clock-names
+ - "#interconnect-cells"
+ - nvidia,bpmp
+
+allOf:
+ - if:
+ properties:
+ compatible:
+ const: nvidia,tegra186-mc
+ then:
+ properties:
+ reg:
+ maxItems: 1
+
+ - if:
+ properties:
+ compatible:
+ const: nvidia,tegra194-mc
+ then:
+ properties:
+ reg:
+ minItems: 3
+
+ - if:
+ properties:
+ compatible:
+ const: nvidia,tegra234-mc
+ then:
+ properties:
+ reg:
+ minItems: 3
+
+additionalProperties: false
+
required:
- compatible
- reg
@@ -90,8 +171,6 @@
- "#address-cells"
- "#size-cells"
-additionalProperties: false
-
examples:
- |
#include <dt-bindings/clock/tegra186-clock.h>
@@ -124,12 +203,9 @@
clocks = <&bpmp TEGRA186_CLK_EMC>;
clock-names = "emc";
+ #interconnect-cells = <0>;
+
nvidia,bpmp = <&bpmp>;
};
};
};
-
- bpmp: bpmp {
- compatible = "nvidia,tegra186-bpmp";
- #clock-cells = <1>;
- };
diff --git a/Documentation/devicetree/bindings/memory-controllers/renesas,rpc-if.yaml b/Documentation/devicetree/bindings/memory-controllers/renesas,rpc-if.yaml
index 9da80e8..294f103 100644
--- a/Documentation/devicetree/bindings/memory-controllers/renesas,rpc-if.yaml
+++ b/Documentation/devicetree/bindings/memory-controllers/renesas,rpc-if.yaml
@@ -24,17 +24,23 @@
properties:
compatible:
- items:
- - enum:
- - renesas,r8a774a1-rpc-if # RZ/G2M
- - renesas,r8a774b1-rpc-if # RZ/G2N
- - renesas,r8a774c0-rpc-if # RZ/G2E
- - renesas,r8a774e1-rpc-if # RZ/G2H
- - renesas,r8a77970-rpc-if # R-Car V3M
- - renesas,r8a77980-rpc-if # R-Car V3H
- - renesas,r8a77995-rpc-if # R-Car D3
- - renesas,r8a779a0-rpc-if # R-Car V3U
- - const: renesas,rcar-gen3-rpc-if # a generic R-Car gen3 or RZ/G2 device
+ oneOf:
+ - items:
+ - enum:
+ - renesas,r8a774a1-rpc-if # RZ/G2M
+ - renesas,r8a774b1-rpc-if # RZ/G2N
+ - renesas,r8a774c0-rpc-if # RZ/G2E
+ - renesas,r8a774e1-rpc-if # RZ/G2H
+ - renesas,r8a77970-rpc-if # R-Car V3M
+ - renesas,r8a77980-rpc-if # R-Car V3H
+ - renesas,r8a77995-rpc-if # R-Car D3
+ - renesas,r8a779a0-rpc-if # R-Car V3U
+ - const: renesas,rcar-gen3-rpc-if # a generic R-Car gen3 or RZ/G2{E,H,M,N} device
+
+ - items:
+ - enum:
+ - renesas,r9a07g044-rpc-if # RZ/G2{L,LC}
+ - const: renesas,rzg2l-rpc-if # RZ/G2L family
reg:
items:
@@ -48,7 +54,9 @@
- const: dirmap
- const: wbuf
- clocks:
+ clocks: true
+
+ interrupts:
maxItems: 1
power-domains:
@@ -67,8 +75,6 @@
- cfi-flash
- jedec,spi-nor
-unevaluatedProperties: false
-
required:
- compatible
- reg
@@ -79,6 +85,26 @@
- '#address-cells'
- '#size-cells'
+if:
+ properties:
+ compatible:
+ contains:
+ enum:
+ - renesas,rzg2l-rpc-if
+then:
+ properties:
+ clocks:
+ items:
+ - description: SPI Multi IO Register access clock (SPI_CLK2)
+ - description: SPI Multi IO Main clock (SPI_CLK).
+
+else:
+ properties:
+ clocks:
+ maxItems: 1
+
+unevaluatedProperties: false
+
examples:
- |
#include <dt-bindings/clock/renesas-cpg-mssr.h>
diff --git a/Documentation/devicetree/bindings/memory-controllers/synopsys,ddrc-ecc.yaml b/Documentation/devicetree/bindings/memory-controllers/synopsys,ddrc-ecc.yaml
index a245884..fb7ae38 100644
--- a/Documentation/devicetree/bindings/memory-controllers/synopsys,ddrc-ecc.yaml
+++ b/Documentation/devicetree/bindings/memory-controllers/synopsys,ddrc-ecc.yaml
@@ -26,6 +26,7 @@
enum:
- xlnx,zynq-ddrc-a05
- xlnx,zynqmp-ddrc-2.40a
+ - snps,ddrc-3.80a
interrupts:
maxItems: 1
diff --git a/Documentation/devicetree/bindings/misc/nvidia,tegra186-misc.txt b/Documentation/devicetree/bindings/misc/nvidia,tegra186-misc.txt
deleted file mode 100644
index 43d777e..0000000
--- a/Documentation/devicetree/bindings/misc/nvidia,tegra186-misc.txt
+++ /dev/null
@@ -1,14 +0,0 @@
-NVIDIA Tegra186 (and later) MISC register block
-
-The MISC register block found on Tegra186 and later SoCs contains registers
-that can be used to identify a given chip and various strapping options.
-
-Required properties:
-- compatible: Must be:
- - Tegra186: "nvidia,tegra186-misc"
- - Tegra194: "nvidia,tegra194-misc"
- - Tegra234: "nvidia,tegra234-misc"
-- reg: Should contain 2 entries: The first entry gives the physical address
- and length of the register region which contains revision and debug
- features. The second entry specifies the physical address and length
- of the register region indicating the strapping options.
diff --git a/Documentation/devicetree/bindings/misc/nvidia,tegra186-misc.yaml b/Documentation/devicetree/bindings/misc/nvidia,tegra186-misc.yaml
new file mode 100644
index 0000000..cacb845
--- /dev/null
+++ b/Documentation/devicetree/bindings/misc/nvidia,tegra186-misc.yaml
@@ -0,0 +1,43 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/misc/nvidia,tegra186-misc.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: NVIDIA Tegra186 (and later) MISC register block
+
+maintainers:
+ - Thierry Reding <thierry.reding@gmail.com>
+ - Jon Hunter <jonathanh@nvidia.com>
+
+description: The MISC register block found on Tegra186 and later SoCs contains
+ registers that can be used to identify a given chip and various strapping
+ options.
+
+properties:
+ compatible:
+ enum:
+ - nvidia,tegra186-misc
+ - nvidia,tegra194-misc
+ - nvidia,tegra234-misc
+
+ reg:
+ items:
+ - description: physical address and length of the registers which
+ contain revision and debug features
+ - description: physical address and length of the registers which
+ indicate strapping options
+
+additionalProperties: false
+
+required:
+ - compatible
+ - reg
+
+examples:
+ - |
+ misc@100000 {
+ compatible = "nvidia,tegra186-misc";
+ reg = <0x00100000 0xf000>,
+ <0x0010f000 0x1000>;
+ };
diff --git a/Documentation/devicetree/bindings/misc/nvidia,tegra20-apbmisc.txt b/Documentation/devicetree/bindings/misc/nvidia,tegra20-apbmisc.txt
deleted file mode 100644
index 83f6a25..0000000
--- a/Documentation/devicetree/bindings/misc/nvidia,tegra20-apbmisc.txt
+++ /dev/null
@@ -1,17 +0,0 @@
-NVIDIA Tegra APBMISC block
-
-Required properties:
-- compatible: Must be:
- - Tegra20: "nvidia,tegra20-apbmisc"
- - Tegra30: "nvidia,tegra30-apbmisc", "nvidia,tegra20-apbmisc"
- - Tegra114: "nvidia,tegra114-apbmisc", "nvidia,tegra20-apbmisc"
- - Tegra124: "nvidia,tegra124-apbmisc", "nvidia,tegra20-apbmisc"
- - Tegra132: "nvidia,tegra124-apbmisc", "nvidia,tegra20-apbmisc"
- - Tegra210: "nvidia,tegra210-apbmisc", "nvidia,tegra20-apbmisc"
-- reg: Should contain 2 entries: the first entry gives the physical address
- and length of the registers which contain revision and debug features.
- The second entry gives the physical address and length of the
- registers indicating the strapping options.
-
-Optional properties:
-- nvidia,long-ram-code: If present, the RAM code is long (4 bit). If not, short (2 bit).
diff --git a/Documentation/devicetree/bindings/misc/nvidia,tegra20-apbmisc.yaml b/Documentation/devicetree/bindings/misc/nvidia,tegra20-apbmisc.yaml
new file mode 100644
index 0000000..6f504fa
--- /dev/null
+++ b/Documentation/devicetree/bindings/misc/nvidia,tegra20-apbmisc.yaml
@@ -0,0 +1,51 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/misc/nvidia,tegra20-apbmisc.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: NVIDIA Tegra APBMISC block
+
+maintainers:
+ - Thierry Reding <thierry.reding@gmail.com>
+ - Jon Hunter <jonathanh@nvidia.com>
+
+properties:
+ compatible:
+ oneOf:
+ - items:
+ - enum:
+ - nvidia,tegra210-apbmisc
+ - nvidia,tegra124-apbmisc
+ - nvidia,tegra114-apbmisc
+ - nvidia,tegra30-apbmisc
+ - const: nvidia,tegra20-apbmisc
+
+ - items:
+ - const: nvidia,tegra20-apbmisc
+
+ reg:
+ items:
+ - description: physical address and length of the registers which
+ contain revision and debug features
+ - description: physical address and length of the registers which
+ indicate strapping options
+
+ nvidia,long-ram-code:
+ description: If present, the RAM code is long (4 bit). If not, short
+ (2 bit).
+ type: boolean
+
+additionalProperties: false
+
+required:
+ - compatible
+ - reg
+
+examples:
+ - |
+ apbmisc@70000800 {
+ compatible = "nvidia,tegra20-apbmisc";
+ reg = <0x70000800 0x64>, /* Chip revision */
+ <0x70000008 0x04>; /* Strapping options */
+ };
diff --git a/Documentation/devicetree/bindings/mmc/nvidia,tegra20-sdhci.txt b/Documentation/devicetree/bindings/mmc/nvidia,tegra20-sdhci.txt
deleted file mode 100644
index 96c0b14..0000000
--- a/Documentation/devicetree/bindings/mmc/nvidia,tegra20-sdhci.txt
+++ /dev/null
@@ -1,143 +0,0 @@
-* NVIDIA Tegra Secure Digital Host Controller
-
-This controller on Tegra family SoCs provides an interface for MMC, SD,
-and SDIO types of memory cards.
-
-This file documents differences between the core properties described
-by mmc.txt and the properties used by the sdhci-tegra driver.
-
-Required properties:
-- compatible : should be one of:
- - "nvidia,tegra20-sdhci": for Tegra20
- - "nvidia,tegra30-sdhci": for Tegra30
- - "nvidia,tegra114-sdhci": for Tegra114
- - "nvidia,tegra124-sdhci": for Tegra124 and Tegra132
- - "nvidia,tegra210-sdhci": for Tegra210
- - "nvidia,tegra186-sdhci": for Tegra186
- - "nvidia,tegra194-sdhci": for Tegra194
-- clocks: For Tegra210, Tegra186 and Tegra194 must contain two entries.
- One for the module clock and one for the timeout clock.
- For all other Tegra devices, must contain a single entry for
- the module clock. See ../clocks/clock-bindings.txt for details.
-- clock-names: For Tegra210, Tegra186 and Tegra194 must contain the
- strings 'sdhci' and 'tmclk' to represent the module and
- the timeout clocks, respectively.
- For all other Tegra devices must contain the string 'sdhci'
- to represent the module clock.
-- resets : Must contain an entry for each entry in reset-names.
- See ../reset/reset.txt for details.
-- reset-names : Must include the following entries:
- - sdhci
-
-Optional properties:
-- power-gpios : Specify GPIOs for power control
-
-Example:
-
-sdhci@c8000200 {
- compatible = "nvidia,tegra20-sdhci";
- reg = <0xc8000200 0x200>;
- interrupts = <47>;
- clocks = <&tegra_car 14>;
- resets = <&tegra_car 14>;
- reset-names = "sdhci";
- cd-gpios = <&gpio 69 0>; /* gpio PI5 */
- wp-gpios = <&gpio 57 0>; /* gpio PH1 */
- power-gpios = <&gpio 155 0>; /* gpio PT3 */
- bus-width = <8>;
-};
-
-Optional properties for Tegra210, Tegra186 and Tegra194:
-- pinctrl-names, pinctrl-0, pinctrl-1 : Specify pad voltage
- configurations. Valid pinctrl-names are "sdmmc-3v3" and "sdmmc-1v8"
- for controllers supporting multiple voltage levels. The order of names
- should correspond to the pin configuration states in pinctrl-0 and
- pinctrl-1.
-- pinctrl-names : "sdmmc-3v3-drv" and "sdmmc-1v8-drv" are applicable for
- Tegra210 where pad config registers are in the pinmux register domain
- for pull-up-strength and pull-down-strength values configuration when
- using pads at 3V3 and 1V8 levels.
-- nvidia,only-1-8-v : The presence of this property indicates that the
- controller operates at a 1.8 V fixed I/O voltage.
-- nvidia,pad-autocal-pull-up-offset-3v3,
- nvidia,pad-autocal-pull-down-offset-3v3 : Specify drive strength
- calibration offsets for 3.3 V signaling modes.
-- nvidia,pad-autocal-pull-up-offset-1v8,
- nvidia,pad-autocal-pull-down-offset-1v8 : Specify drive strength
- calibration offsets for 1.8 V signaling modes.
-- nvidia,pad-autocal-pull-up-offset-3v3-timeout,
- nvidia,pad-autocal-pull-down-offset-3v3-timeout : Specify drive
- strength used as a fallback in case the automatic calibration times
- out on a 3.3 V signaling mode.
-- nvidia,pad-autocal-pull-up-offset-1v8-timeout,
- nvidia,pad-autocal-pull-down-offset-1v8-timeout : Specify drive
- strength used as a fallback in case the automatic calibration times
- out on a 1.8 V signaling mode.
-- nvidia,pad-autocal-pull-up-offset-sdr104,
- nvidia,pad-autocal-pull-down-offset-sdr104 : Specify drive strength
- calibration offsets for SDR104 mode.
-- nvidia,pad-autocal-pull-up-offset-hs400,
- nvidia,pad-autocal-pull-down-offset-hs400 : Specify drive strength
- calibration offsets for HS400 mode.
-- nvidia,default-tap : Specify the default inbound sampling clock
- trimmer value for non-tunable modes.
-- nvidia,default-trim : Specify the default outbound clock trimmer
- value.
-- nvidia,dqs-trim : Specify DQS trim value for HS400 timing
-
- Notes on the pad calibration pull up and pulldown offset values:
- - The property values are drive codes which are programmed into the
- PD_OFFSET and PU_OFFSET sections of the
- SDHCI_TEGRA_AUTO_CAL_CONFIG register.
- - A higher value corresponds to higher drive strength. Please refer
- to the reference manual of the SoC for correct values.
- - The SDR104 and HS400 timing specific values are used in
- corresponding modes if specified.
-
- Notes on tap and trim values:
- - The values are used for compensating trace length differences
- by adjusting the sampling point.
- - The values are programmed to the Vendor Clock Control Register.
- Please refer to the reference manual of the SoC for correct
- values.
- - The DQS trim values are only used on controllers which support
- HS400 timing. Only SDMMC4 on Tegra210 and Tegra 186 supports
- HS400.
-
-Example:
-sdhci@700b0000 {
- compatible = "nvidia,tegra124-sdhci";
- reg = <0x0 0x700b0000 0x0 0x200>;
- interrupts = <GIC_SPI 14 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&tegra_car TEGRA210_CLK_SDMMC1>;
- clock-names = "sdhci";
- resets = <&tegra_car 14>;
- reset-names = "sdhci";
- pinctrl-names = "sdmmc-3v3", "sdmmc-1v8";
- pinctrl-0 = <&sdmmc1_3v3>;
- pinctrl-1 = <&sdmmc1_1v8>;
- nvidia,pad-autocal-pull-up-offset-3v3 = <0x00>;
- nvidia,pad-autocal-pull-down-offset-3v3 = <0x7d>;
- nvidia,pad-autocal-pull-up-offset-1v8 = <0x7b>;
- nvidia,pad-autocal-pull-down-offset-1v8 = <0x7b>;
- status = "disabled";
-};
-
-sdhci@700b0000 {
- compatible = "nvidia,tegra210-sdhci";
- reg = <0x0 0x700b0000 0x0 0x200>;
- interrupts = <GIC_SPI 14 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&tegra_car TEGRA210_CLK_SDMMC1>,
- <&tegra_car TEGRA210_CLK_SDMMC_LEGACY>;
- clock-names = "sdhci", "tmclk";
- resets = <&tegra_car 14>;
- reset-names = "sdhci";
- pinctrl-names = "sdmmc-3v3", "sdmmc-1v8";
- pinctrl-0 = <&sdmmc1_3v3>;
- pinctrl-1 = <&sdmmc1_1v8>;
- nvidia,pad-autocal-pull-up-offset-3v3 = <0x00>;
- nvidia,pad-autocal-pull-down-offset-3v3 = <0x7d>;
- nvidia,pad-autocal-pull-up-offset-1v8 = <0x7b>;
- nvidia,pad-autocal-pull-down-offset-1v8 = <0x7b>;
- status = "disabled";
-};
diff --git a/Documentation/devicetree/bindings/mmc/nvidia,tegra20-sdhci.yaml b/Documentation/devicetree/bindings/mmc/nvidia,tegra20-sdhci.yaml
new file mode 100644
index 0000000..ce64b34
--- /dev/null
+++ b/Documentation/devicetree/bindings/mmc/nvidia,tegra20-sdhci.yaml
@@ -0,0 +1,317 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/mmc/nvidia,tegra20-sdhci.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: NVIDIA Tegra Secure Digital Host Controller
+
+maintainers:
+ - Thierry Reding <thierry.reding@gmail.com>
+ - Jon Hunter <jonathanh@nvidia.com>
+
+description: |
+ This controller on Tegra family SoCs provides an interface for MMC, SD, and
+ SDIO types of memory cards.
+
+ This file documents differences between the core properties described by
+ mmc-controller.yaml and the properties for the Tegra SDHCI controller.
+
+properties:
+ compatible:
+ oneOf:
+ - enum:
+ - nvidia,tegra20-sdhci
+ - nvidia,tegra30-sdhci
+ - nvidia,tegra114-sdhci
+ - nvidia,tegra124-sdhci
+ - nvidia,tegra210-sdhci
+ - nvidia,tegra186-sdhci
+ - nvidia,tegra194-sdhci
+
+ - items:
+ - const: nvidia,tegra132-sdhci
+ - const: nvidia,tegra124-sdhci
+
+ - items:
+ - enum:
+ - nvidia,tegra194-sdhci
+ - nvidia,tegra234-sdhci
+ - const: nvidia,tegra186-sdhci
+
+ reg:
+ maxItems: 1
+
+ interrupts:
+ maxItems: 1
+
+ assigned-clocks: true
+ assigned-clock-parents: true
+ assigned-clock-rates: true
+
+ clocks:
+ minItems: 1
+ maxItems: 2
+
+ clock-names:
+ minItems: 1
+ maxItems: 2
+
+ resets:
+ items:
+ - description: module reset
+
+ reset-names:
+ items:
+ - const: sdhci
+
+ power-gpios:
+ description: specify GPIOs for power control
+ maxItems: 1
+
+ interconnects:
+ items:
+ - description: memory read client
+ - description: memory write client
+
+ interconnect-names:
+ items:
+ - const: dma-mem # read
+ - const: write
+
+ iommus:
+ maxItems: 1
+
+ operating-points-v2:
+ $ref: "/schemas/types.yaml#/definitions/phandle"
+
+ power-domains:
+ items:
+ - description: phandle to the core power domain
+
+ nvidia,default-tap:
+ description: Specify the default inbound sampling clock trimmer value for
+ non-tunable modes.
+
+ The values are used for compensating trace length differences by
+ adjusting the sampling point. The values are programmed to the Vendor
+ Clock Control Register. Please refer to the reference manual of the SoC
+ for correct values.
+
+ The DQS trim values are only used on controllers which support HS400
+ timing. Only SDMMC4 on Tegra210 and Tegra186 supports HS400.
+ $ref: "/schemas/types.yaml#/definitions/uint32"
+
+ nvidia,default-trim:
+ description: Specify the default outbound clock trimmer value.
+ $ref: "/schemas/types.yaml#/definitions/uint32"
+
+ nvidia,dqs-trim:
+ description: Specify DQS trim value for HS400 timing.
+ $ref: "/schemas/types.yaml#/definitions/uint32"
+
+ nvidia,pad-autocal-pull-down-offset-1v8:
+ description: Specify drive strength calibration offsets for 1.8 V
+ signaling modes.
+ $ref: "/schemas/types.yaml#/definitions/uint32"
+
+ nvidia,pad-autocal-pull-down-offset-1v8-timeout:
+ description: Specify drive strength used as a fallback in case the
+ automatic calibration times out on a 1.8 V signaling mode.
+ $ref: "/schemas/types.yaml#/definitions/uint32"
+
+ nvidia,pad-autocal-pull-down-offset-3v3:
+ description: Specify drive strength calibration offsets for 3.3 V
+ signaling modes.
+ $ref: "/schemas/types.yaml#/definitions/uint32"
+
+ nvidia,pad-autocal-pull-down-offset-3v3-timeout:
+ description: Specify drive strength used as a fallback in case the
+ automatic calibration times out on a 3.3 V signaling mode.
+ $ref: "/schemas/types.yaml#/definitions/uint32"
+
+ nvidia,pad-autocal-pull-down-offset-sdr104:
+ description: Specify drive strength calibration offsets for SDR104 mode.
+ $ref: "/schemas/types.yaml#/definitions/uint32"
+
+ nvidia,pad-autocal-pull-down-offset-hs400:
+ description: Specify drive strength calibration offsets for HS400 mode.
+ $ref: "/schemas/types.yaml#/definitions/uint32"
+
+ nvidia,pad-autocal-pull-up-offset-1v8:
+ description: Specify drive strength calibration offsets for 1.8 V
+ signaling modes.
+ $ref: "/schemas/types.yaml#/definitions/uint32"
+
+ nvidia,pad-autocal-pull-up-offset-1v8-timeout:
+ description: Specify drive strength used as a fallback in case the
+ automatic calibration times out on a 1.8 V signaling mode.
+ $ref: "/schemas/types.yaml#/definitions/uint32"
+
+ nvidia,pad-autocal-pull-up-offset-3v3:
+ description: Specify drive strength calibration offsets for 3.3 V
+ signaling modes.
+
+ The property values are drive codes which are programmed into the
+ PD_OFFSET and PU_OFFSET sections of the SDHCI_TEGRA_AUTO_CAL_CONFIG
+ register. A higher value corresponds to higher drive strength. Please
+ refer to the reference manual of the SoC for correct values. The SDR104
+ and HS400 timing specific values are used in corresponding modes if
+ specified.
+ $ref: "/schemas/types.yaml#/definitions/uint32"
+
+ nvidia,pad-autocal-pull-up-offset-3v3-timeout:
+ description: Specify drive strength used as a fallback in case the
+ automatic calibration times out on a 3.3 V signaling mode.
+ $ref: "/schemas/types.yaml#/definitions/uint32"
+
+ nvidia,pad-autocal-pull-up-offset-sdr104:
+ description: Specify drive strength calibration offsets for SDR104 mode.
+ $ref: "/schemas/types.yaml#/definitions/uint32"
+
+ nvidia,pad-autocal-pull-up-offset-hs400:
+ description: Specify drive strength calibration offsets for HS400 mode.
+ $ref: "/schemas/types.yaml#/definitions/uint32"
+
+ nvidia,only-1-8v:
+ description: The presence of this property indicates that the controller
+ operates at a 1.8 V fixed I/O voltage.
+ $ref: "/schemas/types.yaml#/definitions/flag"
+
+required:
+ - compatible
+ - reg
+ - interrupts
+ - clocks
+ - resets
+ - reset-names
+
+allOf:
+ - $ref: "mmc-controller.yaml"
+ - if:
+ properties:
+ compatible:
+ contains:
+ enum:
+ - nvidia,tegra20-sdhci
+ - nvidia,tegra30-sdhci
+ - nvidia,tegra114-sdhci
+ - nvidia,tegra124-sdhci
+ clocks:
+ items:
+ - description: module clock
+ minItems: 1
+ maxItems: 1
+ else:
+ properties:
+ clocks:
+ items:
+ - description: module clock
+ - description: timeout clock
+ minItems: 2
+ maxItems: 2
+ clock-names:
+ items:
+ - const: sdhci
+ - const: tmclk
+ minItems: 2
+ maxItems: 2
+ required:
+ - clock-names
+
+ - if:
+ properties:
+ compatible:
+ contains:
+ const: nvidia,tegra210-sdhci
+ then:
+ properties:
+ pinctrl-names:
+ oneOf:
+ - items:
+ - const: sdmmc-3v3
+ description: pad configuration for 3.3 V
+ - const: sdmmc-1v8
+ description: pad configuration for 1.8 V
+ - const: sdmmc-3v3-drv
+ description: pull-up/down configuration for 3.3 V
+ - const: sdmmc-1v8-drv
+ description: pull-up/down configuration for 1.8 V
+ - items:
+ - const: sdmmc-3v3-drv
+ description: pull-up/down configuration for 3.3 V
+ - const: sdmmc-1v8-drv
+ description: pull-up/down configuration for 1.8 V
+ - items:
+ - const: sdmmc-1v8-drv
+ description: pull-up/down configuration for 1.8 V
+ required:
+ - clock-names
+ - if:
+ properties:
+ compatible:
+ contains:
+ enum:
+ - nvidia,tegra186-sdhci
+ - nvidia,tegra194-sdhci
+ then:
+ properties:
+ pinctrl-names:
+ items:
+ - const: sdmmc-3v3
+ description: pad configuration for 3.3 V
+ - const: sdmmc-1v8
+ description: pad configuration for 1.8 V
+ required:
+ - clock-names
+
+unevaluatedProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+
+ mmc@c8000200 {
+ compatible = "nvidia,tegra20-sdhci";
+ reg = <0xc8000200 0x200>;
+ interrupts = <47>;
+ clocks = <&tegra_car 14>;
+ resets = <&tegra_car 14>;
+ reset-names = "sdhci";
+ cd-gpios = <&gpio 69 0>; /* gpio PI5 */
+ wp-gpios = <&gpio 57 0>; /* gpio PH1 */
+ power-gpios = <&gpio 155 0>; /* gpio PT3 */
+ bus-width = <8>;
+ };
+
+ - |
+ #include <dt-bindings/clock/tegra210-car.h>
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+
+ mmc@700b0000 {
+ compatible = "nvidia,tegra210-sdhci";
+ reg = <0x700b0000 0x200>;
+ interrupts = <GIC_SPI 14 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&tegra_car TEGRA210_CLK_SDMMC1>,
+ <&tegra_car TEGRA210_CLK_SDMMC_LEGACY>;
+ clock-names = "sdhci", "tmclk";
+ resets = <&tegra_car 14>;
+ reset-names = "sdhci";
+ pinctrl-names = "sdmmc-3v3", "sdmmc-1v8",
+ "sdmmc-3v3-drv", "sdmmc-1v8-drv";
+ pinctrl-0 = <&sdmmc1_3v3>;
+ pinctrl-1 = <&sdmmc1_1v8>;
+ pinctrl-2 = <&sdmmc1_3v3_drv>;
+ pinctrl-3 = <&sdmmc1_1v8_drv>;
+ nvidia,pad-autocal-pull-up-offset-3v3 = <0x00>;
+ nvidia,pad-autocal-pull-down-offset-3v3 = <0x7d>;
+ nvidia,pad-autocal-pull-up-offset-1v8 = <0x7b>;
+ nvidia,pad-autocal-pull-down-offset-1v8 = <0x7b>;
+ nvidia,default-tap = <0x2>;
+ nvidia,default-trim = <0x4>;
+ assigned-clocks = <&tegra_car TEGRA210_CLK_SDMMC4>,
+ <&tegra_car TEGRA210_CLK_PLL_C4_OUT0>,
+ <&tegra_car TEGRA210_CLK_PLL_C4>;
+ assigned-clock-parents = <&tegra_car TEGRA210_CLK_PLL_C4_OUT0>;
+ assigned-clock-rates = <200000000>, <1000000000>, <1000000000>;
+ };
diff --git a/Documentation/devicetree/bindings/mmc/renesas,sdhi.yaml b/Documentation/devicetree/bindings/mmc/renesas,sdhi.yaml
index 9f1e709..9ce6e06 100644
--- a/Documentation/devicetree/bindings/mmc/renesas,sdhi.yaml
+++ b/Documentation/devicetree/bindings/mmc/renesas,sdhi.yaml
@@ -113,31 +113,51 @@
clocks:
items:
- description: IMCLK, SDHI channel main clock1.
+ - description: CLK_HS, SDHI channel High speed clock which operates
+ 4 times that of SDHI channel main clock1.
- description: IMCLK2, SDHI channel main clock2. When this clock is
turned off, external SD card detection cannot be
detected.
- - description: CLK_HS, SDHI channel High speed clock which operates
- 4 times that of SDHI channel main clock1.
- description: ACLK, SDHI channel bus clock.
clock-names:
items:
- - const: imclk
- - const: imclk2
- - const: clk_hs
+ - const: core
+ - const: clkh
+ - const: cd
- const: aclk
required:
- clock-names
- resets
else:
- properties:
- clocks:
- minItems: 1
- maxItems: 2
- clock-names:
- minItems: 1
- items:
- - const: core
- - const: cd
+ if:
+ properties:
+ compatible:
+ contains:
+ enum:
+ - renesas,rcar-gen2-sdhi
+ - renesas,rcar-gen3-sdhi
+ then:
+ properties:
+ clocks:
+ minItems: 1
+ maxItems: 3
+ clock-names:
+ minItems: 1
+ uniqueItems: true
+ items:
+ - const: core
+ - enum: [ clkh, cd ]
+ - const: cd
+ else:
+ properties:
+ clocks:
+ minItems: 1
+ maxItems: 2
+ clock-names:
+ minItems: 1
+ items:
+ - const: core
+ - const: cd
- if:
properties:
diff --git a/Documentation/devicetree/bindings/net/can/allwinner,sun4i-a10-can.yaml b/Documentation/devicetree/bindings/net/can/allwinner,sun4i-a10-can.yaml
index a95960e..c93fe9d 100644
--- a/Documentation/devicetree/bindings/net/can/allwinner,sun4i-a10-can.yaml
+++ b/Documentation/devicetree/bindings/net/can/allwinner,sun4i-a10-can.yaml
@@ -17,6 +17,7 @@
- const: allwinner,sun7i-a20-can
- const: allwinner,sun4i-a10-can
- const: allwinner,sun4i-a10-can
+ - const: allwinner,sun8i-r40-can
reg:
maxItems: 1
@@ -27,6 +28,19 @@
clocks:
maxItems: 1
+ resets:
+ maxItems: 1
+
+if:
+ properties:
+ compatible:
+ contains:
+ const: allwinner,sun8i-r40-can
+
+then:
+ required:
+ - resets
+
required:
- compatible
- reg
@@ -47,5 +61,15 @@
interrupts = <GIC_SPI 26 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&ccu CLK_APB1_CAN>;
};
+ - |
+ #define RST_BUS_CAN 68
+ #define CLK_BUS_CAN 91
+ can1: can@1c2bc00 {
+ compatible = "allwinner,sun8i-r40-can";
+ reg = <0x01c2bc00 0x400>;
+ interrupts = <GIC_SPI 26 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&ccu CLK_BUS_CAN>;
+ resets = <&ccu RST_BUS_CAN>;
+ };
...
diff --git a/Documentation/devicetree/bindings/net/dsa/dsa-port.yaml b/Documentation/devicetree/bindings/net/dsa/dsa-port.yaml
new file mode 100644
index 0000000..702df84
--- /dev/null
+++ b/Documentation/devicetree/bindings/net/dsa/dsa-port.yaml
@@ -0,0 +1,77 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/net/dsa/dsa-port.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Ethernet Switch port Device Tree Bindings
+
+maintainers:
+ - Andrew Lunn <andrew@lunn.ch>
+ - Florian Fainelli <f.fainelli@gmail.com>
+ - Vivien Didelot <vivien.didelot@gmail.com>
+
+description:
+ Ethernet switch port Description
+
+allOf:
+ - $ref: "http://devicetree.org/schemas/net/ethernet-controller.yaml#"
+
+properties:
+ reg:
+ description: Port number
+
+ label:
+ description:
+ Describes the label associated with this port, which will become
+ the netdev name
+ $ref: /schemas/types.yaml#/definitions/string
+
+ link:
+ description:
+ Should be a list of phandles to other switch's DSA port. This
+ port is used as the outgoing port towards the phandle ports. The
+ full routing information must be given, not just the one hop
+ routes to neighbouring switches
+ $ref: /schemas/types.yaml#/definitions/phandle-array
+
+ ethernet:
+ description:
+ Should be a phandle to a valid Ethernet device node. This host
+ device is what the switch port is connected to
+ $ref: /schemas/types.yaml#/definitions/phandle
+
+ dsa-tag-protocol:
+ description:
+ Instead of the default, the switch will use this tag protocol if
+ possible. Useful when a device supports multiple protocols and
+ the default is incompatible with the Ethernet device.
+ enum:
+ - dsa
+ - edsa
+ - ocelot
+ - ocelot-8021q
+ - seville
+
+ phy-handle: true
+
+ phy-mode: true
+
+ fixed-link: true
+
+ mac-address: true
+
+ sfp: true
+
+ managed: true
+
+ rx-internal-delay-ps: true
+
+ tx-internal-delay-ps: true
+
+required:
+ - reg
+
+additionalProperties: true
+
+...
diff --git a/Documentation/devicetree/bindings/net/dsa/dsa.yaml b/Documentation/devicetree/bindings/net/dsa/dsa.yaml
index 2ad7f79..b9d48e3 100644
--- a/Documentation/devicetree/bindings/net/dsa/dsa.yaml
+++ b/Documentation/devicetree/bindings/net/dsa/dsa.yaml
@@ -46,65 +46,9 @@
type: object
description: Ethernet switch ports
- allOf:
- - $ref: "http://devicetree.org/schemas/net/ethernet-controller.yaml#"
+ $ref: dsa-port.yaml#
- properties:
- reg:
- description: Port number
-
- label:
- description:
- Describes the label associated with this port, which will become
- the netdev name
- $ref: /schemas/types.yaml#/definitions/string
-
- link:
- description:
- Should be a list of phandles to other switch's DSA port. This
- port is used as the outgoing port towards the phandle ports. The
- full routing information must be given, not just the one hop
- routes to neighbouring switches
- $ref: /schemas/types.yaml#/definitions/phandle-array
-
- ethernet:
- description:
- Should be a phandle to a valid Ethernet device node. This host
- device is what the switch port is connected to
- $ref: /schemas/types.yaml#/definitions/phandle
-
- dsa-tag-protocol:
- description:
- Instead of the default, the switch will use this tag protocol if
- possible. Useful when a device supports multiple protocols and
- the default is incompatible with the Ethernet device.
- enum:
- - dsa
- - edsa
- - ocelot
- - ocelot-8021q
- - seville
-
- phy-handle: true
-
- phy-mode: true
-
- fixed-link: true
-
- mac-address: true
-
- sfp: true
-
- managed: true
-
- rx-internal-delay-ps: true
-
- tx-internal-delay-ps: true
-
- required:
- - reg
-
- additionalProperties: false
+ unevaluatedProperties: false
oneOf:
- required:
diff --git a/Documentation/devicetree/bindings/net/dsa/qca8k.yaml b/Documentation/devicetree/bindings/net/dsa/qca8k.yaml
index 48de0ac..89c21b2 100644
--- a/Documentation/devicetree/bindings/net/dsa/qca8k.yaml
+++ b/Documentation/devicetree/bindings/net/dsa/qca8k.yaml
@@ -99,40 +99,9 @@
type: object
description: Ethernet switch ports
+ $ref: dsa-port.yaml#
+
properties:
- reg:
- description: Port number
-
- label:
- description:
- Describes the label associated with this port, which will become
- the netdev name
- $ref: /schemas/types.yaml#/definitions/string
-
- link:
- description:
- Should be a list of phandles to other switch's DSA port. This
- port is used as the outgoing port towards the phandle ports. The
- full routing information must be given, not just the one hop
- routes to neighbouring switches
- $ref: /schemas/types.yaml#/definitions/phandle-array
-
- ethernet:
- description:
- Should be a phandle to a valid Ethernet device node. This host
- device is what the switch port is connected to
- $ref: /schemas/types.yaml#/definitions/phandle
-
- phy-handle: true
-
- phy-mode: true
-
- fixed-link: true
-
- mac-address: true
-
- sfp: true
-
qca,sgmii-rxclk-falling-edge:
$ref: /schemas/types.yaml#/definitions/flag
description:
@@ -154,10 +123,7 @@
SGMII on the QCA8337, it is advised to set this unless a communication
issue is observed.
- required:
- - reg
-
- additionalProperties: false
+ unevaluatedProperties: false
oneOf:
- required:
diff --git a/Documentation/devicetree/bindings/net/engleder,tsnep.yaml b/Documentation/devicetree/bindings/net/engleder,tsnep.yaml
new file mode 100644
index 0000000..d0e1476
--- /dev/null
+++ b/Documentation/devicetree/bindings/net/engleder,tsnep.yaml
@@ -0,0 +1,79 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/net/engleder,tsnep.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: TSN endpoint Ethernet MAC binding
+
+maintainers:
+ - Gerhard Engleder <gerhard@engleder-embedded.com>
+
+allOf:
+ - $ref: ethernet-controller.yaml#
+
+properties:
+ compatible:
+ const: engleder,tsnep
+
+ reg:
+ maxItems: 1
+
+ interrupts:
+ maxItems: 1
+
+ local-mac-address: true
+
+ mac-address: true
+
+ nvmem-cells: true
+
+ nvmem-cells-names: true
+
+ phy-connection-type:
+ enum:
+ - mii
+ - gmii
+ - rgmii
+ - rgmii-id
+
+ phy-mode: true
+
+ phy-handle: true
+
+ mdio:
+ type: object
+ $ref: "mdio.yaml#"
+ description: optional node for embedded MDIO controller
+
+required:
+ - compatible
+ - reg
+ - interrupts
+
+additionalProperties: false
+
+examples:
+ - |
+ axi {
+ #address-cells = <2>;
+ #size-cells = <2>;
+ tnsep0: ethernet@a0000000 {
+ compatible = "engleder,tsnep";
+ reg = <0x0 0xa0000000 0x0 0x10000>;
+ interrupts = <0 89 1>;
+ interrupt-parent = <&gic>;
+ local-mac-address = [00 00 00 00 00 00];
+ phy-mode = "rgmii";
+ phy-handle = <&phy0>;
+ mdio {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ suppress-preamble;
+ phy0: ethernet-phy@1 {
+ reg = <1>;
+ rxc-skew-ps = <1080>;
+ };
+ };
+ };
+ };
diff --git a/Documentation/devicetree/bindings/net/intel,ixp4xx-hss.yaml b/Documentation/devicetree/bindings/net/intel,ixp4xx-hss.yaml
new file mode 100644
index 0000000..4dcd53c
--- /dev/null
+++ b/Documentation/devicetree/bindings/net/intel,ixp4xx-hss.yaml
@@ -0,0 +1,100 @@
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+# Copyright 2021 Linaro Ltd.
+%YAML 1.2
+---
+$id: "http://devicetree.org/schemas/net/intel,ixp4xx-hss.yaml#"
+$schema: "http://devicetree.org/meta-schemas/core.yaml#"
+
+title: Intel IXP4xx V.35 WAN High Speed Serial Link (HSS)
+
+maintainers:
+ - Linus Walleij <linus.walleij@linaro.org>
+
+description: |
+ The Intel IXP4xx HSS makes use of the IXP4xx NPE (Network
+ Processing Engine) and the IXP4xx Queue Manager to process
+ V.35 Wideband Modem (WAN) links.
+
+properties:
+ compatible:
+ const: intel,ixp4xx-hss
+
+ reg:
+ maxItems: 1
+ description: The HSS instance
+
+ intel,npe-handle:
+ $ref: '/schemas/types.yaml#/definitions/phandle-array'
+ maxItems: 1
+ description: phandle to the NPE this HSS instance is using
+ and the instance to use in the second cell
+
+ intel,queue-chl-rxtrig:
+ $ref: '/schemas/types.yaml#/definitions/phandle-array'
+ maxItems: 1
+ description: phandle to the RX trigger queue on the NPE
+
+ intel,queue-chl-txready:
+ $ref: '/schemas/types.yaml#/definitions/phandle-array'
+ maxItems: 1
+ description: phandle to the TX ready queue on the NPE
+
+ intel,queue-pkt-rx:
+ $ref: '/schemas/types.yaml#/definitions/phandle-array'
+ maxItems: 1
+ description: phandle to the packet RX queue on the NPE
+
+ intel,queue-pkt-tx:
+ $ref: '/schemas/types.yaml#/definitions/phandle-array'
+ maxItems: 4
+ description: phandle to the packet TX0, TX1, TX2 and TX3 queues on the NPE
+
+ intel,queue-pkt-rxfree:
+ $ref: '/schemas/types.yaml#/definitions/phandle-array'
+ maxItems: 4
+ description: phandle to the packet RXFREE0, RXFREE1, RXFREE2 and
+ RXFREE3 queues on the NPE
+
+ intel,queue-pkt-txdone:
+ $ref: '/schemas/types.yaml#/definitions/phandle-array'
+ maxItems: 1
+ description: phandle to the packet TXDONE queue on the NPE
+
+ cts-gpios:
+ maxItems: 1
+ description: Clear To Send (CTS) GPIO line
+
+ rts-gpios:
+ maxItems: 1
+ description: Ready To Send (RTS) GPIO line
+
+ dcd-gpios:
+ maxItems: 1
+ description: Data Carrier Detect (DCD) GPIO line
+
+ dtr-gpios:
+ maxItems: 1
+ description: Data Terminal Ready (DTR) GPIO line
+
+ clk-internal-gpios:
+ maxItems: 1
+ description: Clock internal GPIO line, driving this high will make the HSS
+ use internal clocking as opposed to external clocking
+
+required:
+ - compatible
+ - reg
+ - intel,npe-handle
+ - intel,queue-chl-rxtrig
+ - intel,queue-chl-txready
+ - intel,queue-pkt-rx
+ - intel,queue-pkt-tx
+ - intel,queue-pkt-rxfree
+ - intel,queue-pkt-txdone
+ - cts-gpios
+ - rts-gpios
+ - dcd-gpios
+ - dtr-gpios
+ - clk-internal-gpios
+
+additionalProperties: false
diff --git a/Documentation/devicetree/bindings/net/microchip,lan966x-switch.yaml b/Documentation/devicetree/bindings/net/microchip,lan966x-switch.yaml
new file mode 100644
index 0000000..e79e4e1
--- /dev/null
+++ b/Documentation/devicetree/bindings/net/microchip,lan966x-switch.yaml
@@ -0,0 +1,169 @@
+# SPDX-License-Identifier: GPL-2.0-only OR BSD-2-Clause
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/net/microchip,lan966x-switch.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Microchip Lan966x Ethernet switch controller
+
+maintainers:
+ - Horatiu Vultur <horatiu.vultur@microchip.com>
+
+description: |
+ The lan966x switch is a multi-port Gigabit AVB/TSN Ethernet Switch with
+ two integrated 10/100/1000Base-T PHYs. In addition to the integrated PHYs,
+ it supports up to 2RGMII/RMII, up to 3BASE-X/SERDES/2.5GBASE-X and up to
+ 2 Quad-SGMII/Quad-USGMII interfaces.
+
+properties:
+ $nodename:
+ pattern: "^switch@[0-9a-f]+$"
+
+ compatible:
+ const: microchip,lan966x-switch
+
+ reg:
+ items:
+ - description: cpu target
+ - description: general control block target
+
+ reg-names:
+ items:
+ - const: cpu
+ - const: gcb
+
+ interrupts:
+ minItems: 1
+ items:
+ - description: register based extraction
+ - description: frame dma based extraction
+ - description: analyzer interrupt
+
+ interrupt-names:
+ minItems: 1
+ items:
+ - const: xtr
+ - const: fdma
+ - const: ana
+
+ resets:
+ items:
+ - description: Reset controller used for switch core reset (soft reset)
+ - description: Reset controller used for releasing the phy from reset
+
+ reset-names:
+ items:
+ - const: switch
+ - const: phy
+
+ ethernet-ports:
+ type: object
+
+ properties:
+ '#address-cells':
+ const: 1
+ '#size-cells':
+ const: 0
+
+ additionalProperties: false
+
+ patternProperties:
+ "^port@[0-9a-f]+$":
+ type: object
+
+ $ref: "/schemas/net/ethernet-controller.yaml#"
+ unevaluatedProperties: false
+
+ properties:
+ '#address-cells':
+ const: 1
+ '#size-cells':
+ const: 0
+
+ reg:
+ description:
+ Switch port number
+
+ phys:
+ description:
+ Phandle of a Ethernet SerDes PHY
+
+ phy-mode:
+ description:
+ This specifies the interface used by the Ethernet SerDes towards
+ the PHY or SFP.
+ enum:
+ - gmii
+ - sgmii
+ - qsgmii
+ - 1000base-x
+ - 2500base-x
+
+ phy-handle:
+ description:
+ Phandle of a Ethernet PHY.
+
+ sfp:
+ description:
+ Phandle of an SFP.
+
+ managed: true
+
+ required:
+ - reg
+ - phys
+ - phy-mode
+
+ oneOf:
+ - required:
+ - phy-handle
+ - required:
+ - sfp
+ - managed
+
+required:
+ - compatible
+ - reg
+ - reg-names
+ - interrupts
+ - interrupt-names
+ - resets
+ - reset-names
+ - ethernet-ports
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+ switch: switch@e0000000 {
+ compatible = "microchip,lan966x-switch";
+ reg = <0xe0000000 0x0100000>,
+ <0xe2000000 0x0800000>;
+ reg-names = "cpu", "gcb";
+ interrupts = <GIC_SPI 30 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "xtr";
+ resets = <&switch_reset 0>, <&phy_reset 0>;
+ reset-names = "switch", "phy";
+ ethernet-ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port0: port@0 {
+ reg = <0>;
+ phy-handle = <&phy0>;
+ phys = <&serdes 0 0>;
+ phy-mode = "gmii";
+ };
+
+ port1: port@1 {
+ reg = <1>;
+ sfp = <&sfp_eth1>;
+ managed = "in-band-status";
+ phys = <&serdes 2 4>;
+ phy-mode = "sgmii";
+ };
+ };
+ };
+
+...
diff --git a/Documentation/devicetree/bindings/net/qcom,bam-dmux.yaml b/Documentation/devicetree/bindings/net/qcom,bam-dmux.yaml
new file mode 100644
index 0000000..b305444
--- /dev/null
+++ b/Documentation/devicetree/bindings/net/qcom,bam-dmux.yaml
@@ -0,0 +1,92 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/net/qcom,bam-dmux.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Qualcomm BAM Data Multiplexer
+
+maintainers:
+ - Stephan Gerhold <stephan@gerhold.net>
+
+description: |
+ The BAM Data Multiplexer provides access to the network data channels
+ of modems integrated into many older Qualcomm SoCs, e.g. Qualcomm MSM8916
+ or MSM8974. It is built using a simple protocol layer on top of a DMA engine
+ (Qualcomm BAM DMA) and bidirectional interrupts to coordinate power control.
+
+ Note that this schema does not directly describe a hardware block but rather
+ a firmware convention that combines several other hardware blocks (such as the
+ DMA engine). As such it is specific to a firmware version, not a particular
+ SoC or hardware version.
+
+properties:
+ compatible:
+ const: qcom,bam-dmux
+
+ interrupts:
+ description:
+ Interrupts used by the modem to signal the AP.
+ Both interrupts must be declared as IRQ_TYPE_EDGE_BOTH.
+ items:
+ - description: Power control
+ - description: Power control acknowledgment
+
+ interrupt-names:
+ items:
+ - const: pc
+ - const: pc-ack
+
+ qcom,smem-states:
+ $ref: /schemas/types.yaml#/definitions/phandle-array
+ description: State bits used by the AP to signal the modem.
+ items:
+ - description: Power control
+ - description: Power control acknowledgment
+
+ qcom,smem-state-names:
+ description: Names for the state bits used by the AP to signal the modem.
+ items:
+ - const: pc
+ - const: pc-ack
+
+ dmas:
+ items:
+ - description: TX DMA channel phandle
+ - description: RX DMA channel phandle
+
+ dma-names:
+ items:
+ - const: tx
+ - const: rx
+
+required:
+ - compatible
+ - interrupts
+ - interrupt-names
+ - qcom,smem-states
+ - qcom,smem-state-names
+ - dmas
+ - dma-names
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/interrupt-controller/irq.h>
+
+ mpss: remoteproc {
+ bam-dmux {
+ compatible = "qcom,bam-dmux";
+
+ interrupt-parent = <&modem_smsm>;
+ interrupts = <1 IRQ_TYPE_EDGE_BOTH>, <11 IRQ_TYPE_EDGE_BOTH>;
+ interrupt-names = "pc", "pc-ack";
+
+ qcom,smem-states = <&apps_smsm 1>, <&apps_smsm 11>;
+ qcom,smem-state-names = "pc", "pc-ack";
+
+ dmas = <&bam_dmux_dma 4>, <&bam_dmux_dma 5>;
+ dma-names = "tx", "rx";
+ };
+ };
diff --git a/Documentation/devicetree/bindings/net/vertexcom-mse102x.yaml b/Documentation/devicetree/bindings/net/vertexcom-mse102x.yaml
new file mode 100644
index 0000000..8156a9a
--- /dev/null
+++ b/Documentation/devicetree/bindings/net/vertexcom-mse102x.yaml
@@ -0,0 +1,71 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: "http://devicetree.org/schemas/net/vertexcom-mse102x.yaml#"
+$schema: "http://devicetree.org/meta-schemas/core.yaml#"
+
+title: The Vertexcom MSE102x (SPI) Device Tree Bindings
+
+maintainers:
+ - Stefan Wahren <stefan.wahren@in-tech.com>
+
+description:
+ Vertexcom's MSE102x are a family of HomePlug GreenPHY chips.
+ They can be connected either via RGMII, RMII or SPI to a host CPU.
+
+ In order to use a MSE102x chip as SPI device, it must be defined as
+ a child of an SPI master device in the device tree.
+
+ More information can be found at
+ http://www.vertexcom.com/doc/MSE1022%20Product%20Brief.pdf
+
+allOf:
+ - $ref: ethernet-controller.yaml#
+
+properties:
+ compatible:
+ enum:
+ - vertexcom,mse1021
+ - vertexcom,mse1022
+
+ reg:
+ maxItems: 1
+
+ interrupts:
+ maxItems: 1
+
+ spi-cpha: true
+
+ spi-cpol: true
+
+ spi-max-frequency:
+ minimum: 6000000
+ maximum: 7142857
+
+required:
+ - compatible
+ - reg
+ - interrupts
+ - spi-cpha
+ - spi-cpol
+ - spi-max-frequency
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/interrupt-controller/irq.h>
+ spi0 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ ethernet@0 {
+ compatible = "vertexcom,mse1021";
+ reg = <0>;
+ interrupt-parent = <&gpio>;
+ interrupts = <23 IRQ_TYPE_EDGE_RISING>;
+ spi-cpha;
+ spi-cpol;
+ spi-max-frequency = <7142857>;
+ };
+ };
diff --git a/Documentation/devicetree/bindings/net/wireless/microchip,wilc1000.yaml b/Documentation/devicetree/bindings/net/wireless/microchip,wilc1000.yaml
index 6c35682..60de78f 100644
--- a/Documentation/devicetree/bindings/net/wireless/microchip,wilc1000.yaml
+++ b/Documentation/devicetree/bindings/net/wireless/microchip,wilc1000.yaml
@@ -32,6 +32,21 @@
clock-names:
const: rtc
+ enable-gpios:
+ maxItems: 1
+ description: Used by wilc1000-spi to determine the GPIO line
+ connected to the ENABLE line. If specified, reset-gpios
+ must be specified as well as otherwise the driver cannot
+ ensure the timing required between asserting ENABLE
+ and deasserting RESET. This should be declared as an
+ active-high signal.
+
+ reset-gpios:
+ maxItems: 1
+ description: Used by wilc1000-spi to determine the GPIO line
+ connected to the RESET line. This should be declared as an
+ active-low signal.
+
required:
- compatible
- interrupts
@@ -40,6 +55,8 @@
examples:
- |
+ #include <dt-bindings/gpio/gpio.h>
+
spi {
#address-cells = <1>;
#size-cells = <0>;
@@ -51,6 +68,8 @@
interrupts = <27 0>;
clocks = <&pck1>;
clock-names = "rtc";
+ enable-gpios = <&pioA 5 GPIO_ACTIVE_HIGH>;
+ reset-gpios = <&pioA 6 GPIO_ACTIVE_LOW>;
};
};
diff --git a/Documentation/devicetree/bindings/net/wireless/qcom,ath11k.yaml b/Documentation/devicetree/bindings/net/wireless/qcom,ath11k.yaml
index 85c2f69..cdf7b87 100644
--- a/Documentation/devicetree/bindings/net/wireless/qcom,ath11k.yaml
+++ b/Documentation/devicetree/bindings/net/wireless/qcom,ath11k.yaml
@@ -150,6 +150,12 @@
string to uniquely identify variant of the calibration data in the
board-2.bin for designs with colliding bus and device specific ids
+ memory-region:
+ maxItems: 1
+ description:
+ phandle to a node describing reserved memory (System RAM memory)
+ used by ath11k firmware (see bindings/reserved-memory/reserved-memory.txt)
+
required:
- compatible
- reg
@@ -279,3 +285,27 @@
"tcl2host-status-ring";
qcom,rproc = <&q6v5_wcss>;
};
+
+ - |
+
+ reserved-memory {
+ #address-cells = <2>;
+ #size-cells = <2>;
+
+ qcn9074_0: qcn9074_0@51100000 {
+ no-map;
+ reg = <0x0 0x51100000 0x0 0x03500000>;
+ };
+ };
+
+ pci {
+ pcie0 {
+ #size-cells = <2>;
+ #address-cells = <3>;
+
+ wifi_0: wifi@0 {
+ reg = <0 0 0 0 0>;
+ memory-region = <&qcn9074_0>;
+ };
+ };
+ };
diff --git a/Documentation/devicetree/bindings/pci/apple,pcie.yaml b/Documentation/devicetree/bindings/pci/apple,pcie.yaml
index ef1d424..7f01e15 100644
--- a/Documentation/devicetree/bindings/pci/apple,pcie.yaml
+++ b/Documentation/devicetree/bindings/pci/apple,pcie.yaml
@@ -28,19 +28,17 @@
distributed over the root ports as the OS sees fit by programming
the PCIe controller's port registers.
-allOf:
- - $ref: /schemas/pci/pci-bus.yaml#
- - $ref: /schemas/interrupt-controller/msi-controller.yaml#
-
properties:
compatible:
items:
- - const: apple,t8103-pcie
+ - enum:
+ - apple,t8103-pcie
+ - apple,t6000-pcie
- const: apple,pcie
reg:
minItems: 3
- maxItems: 5
+ maxItems: 6
reg-names:
minItems: 3
@@ -50,6 +48,7 @@
- const: port0
- const: port1
- const: port2
+ - const: port3
ranges:
minItems: 2
@@ -59,7 +58,7 @@
description:
Interrupt specifiers, one for each root port.
minItems: 1
- maxItems: 3
+ maxItems: 4
msi-parent: true
@@ -81,6 +80,21 @@
unevaluatedProperties: false
+allOf:
+ - $ref: /schemas/pci/pci-bus.yaml#
+ - $ref: /schemas/interrupt-controller/msi-controller.yaml#
+ - if:
+ properties:
+ compatible:
+ contains:
+ const: apple,t8103-pcie
+ then:
+ properties:
+ reg:
+ maxItems: 5
+ interrupts:
+ maxItems: 3
+
examples:
- |
#include <dt-bindings/interrupt-controller/apple-aic.h>
diff --git a/Documentation/devicetree/bindings/perf/arm,cmn.yaml b/Documentation/devicetree/bindings/perf/arm,cmn.yaml
index 42424cc..2d4219e 100644
--- a/Documentation/devicetree/bindings/perf/arm,cmn.yaml
+++ b/Documentation/devicetree/bindings/perf/arm,cmn.yaml
@@ -12,12 +12,14 @@
properties:
compatible:
- const: arm,cmn-600
+ enum:
+ - arm,cmn-600
+ - arm,ci-700
reg:
items:
- description: Physical address of the base (PERIPHBASE) and
- size (up to 64MB) of the configuration address space.
+ size of the configuration address space.
interrupts:
minItems: 1
@@ -31,14 +33,23 @@
arm,root-node:
$ref: /schemas/types.yaml#/definitions/uint32
- description: Offset from PERIPHBASE of the configuration
- discovery node (see TRM definition of ROOTNODEBASE).
+ description: Offset from PERIPHBASE of CMN-600's configuration
+ discovery node (see TRM definition of ROOTNODEBASE). Not
+ relevant for newer CMN/CI products.
required:
- compatible
- reg
- interrupts
- - arm,root-node
+
+if:
+ properties:
+ compatible:
+ contains:
+ const: arm,cmn-600
+then:
+ required:
+ - arm,root-node
additionalProperties: false
diff --git a/Documentation/devicetree/bindings/perf/arm,smmu-v3-pmcg.yaml b/Documentation/devicetree/bindings/perf/arm,smmu-v3-pmcg.yaml
new file mode 100644
index 0000000..a4b53a6
--- /dev/null
+++ b/Documentation/devicetree/bindings/perf/arm,smmu-v3-pmcg.yaml
@@ -0,0 +1,70 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/perf/arm,smmu-v3-pmcg.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Arm SMMUv3 Performance Monitor Counter Group
+
+maintainers:
+ - Will Deacon <will@kernel.org>
+ - Robin Murphy <robin.murphy@arm.com>
+
+description: |
+ An SMMUv3 may have several Performance Monitor Counter Group (PMCG).
+ They are standalone performance monitoring units that support both
+ architected and IMPLEMENTATION DEFINED event counters.
+
+properties:
+ $nodename:
+ pattern: "^pmu@[0-9a-f]*"
+ compatible:
+ oneOf:
+ - items:
+ - const: arm,mmu-600-pmcg
+ - const: arm,smmu-v3-pmcg
+ - const: arm,smmu-v3-pmcg
+
+ reg:
+ items:
+ - description: Register page 0
+ - description: Register page 1, if SMMU_PMCG_CFGR.RELOC_CTRS = 1
+ minItems: 1
+
+ interrupts:
+ maxItems: 1
+
+ msi-parent: true
+
+required:
+ - compatible
+ - reg
+
+anyOf:
+ - required:
+ - interrupts
+ - required:
+ - msi-parent
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+ #include <dt-bindings/interrupt-controller/irq.h>
+
+ pmu@2b420000 {
+ compatible = "arm,smmu-v3-pmcg";
+ reg = <0x2b420000 0x1000>,
+ <0x2b430000 0x1000>;
+ interrupts = <GIC_SPI 80 IRQ_TYPE_EDGE_RISING>;
+ msi-parent = <&its 0xff0000>;
+ };
+
+ pmu@2b440000 {
+ compatible = "arm,smmu-v3-pmcg";
+ reg = <0x2b440000 0x1000>,
+ <0x2b450000 0x1000>;
+ interrupts = <GIC_SPI 81 IRQ_TYPE_EDGE_RISING>;
+ msi-parent = <&its 0xff0000>;
+ };
diff --git a/Documentation/devicetree/bindings/perf/marvell-cn10k-tad.yaml b/Documentation/devicetree/bindings/perf/marvell-cn10k-tad.yaml
new file mode 100644
index 0000000..3621422
--- /dev/null
+++ b/Documentation/devicetree/bindings/perf/marvell-cn10k-tad.yaml
@@ -0,0 +1,63 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/perf/marvell-cn10k-tad.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Marvell CN10K LLC-TAD performance monitor
+
+maintainers:
+ - Bhaskara Budiredla <bbudiredla@marvell.com>
+
+description: |
+ The Tag-and-Data units (TADs) maintain coherence and contain CN10K
+ shared on-chip last level cache (LLC). The tad pmu measures the
+ performance of last-level cache. Each tad pmu supports up to eight
+ counters.
+
+ The DT setup comprises of number of tad blocks, the sizes of pmu
+ regions, tad blocks and overall base address of the HW.
+
+properties:
+ compatible:
+ const: marvell,cn10k-tad-pmu
+
+ reg:
+ maxItems: 1
+
+ marvell,tad-cnt:
+ description: specifies the number of tads on the soc
+ $ref: /schemas/types.yaml#/definitions/uint32
+
+ marvell,tad-page-size:
+ description: specifies the size of each tad page
+ $ref: /schemas/types.yaml#/definitions/uint32
+
+ marvell,tad-pmu-page-size:
+ description: specifies the size of page that the pmu uses
+ $ref: /schemas/types.yaml#/definitions/uint32
+
+required:
+ - compatible
+ - reg
+ - marvell,tad-cnt
+ - marvell,tad-page-size
+ - marvell,tad-pmu-page-size
+
+additionalProperties: false
+
+examples:
+ - |
+
+ tad {
+ #address-cells = <2>;
+ #size-cells = <2>;
+
+ tad_pmu@80000000 {
+ compatible = "marvell,cn10k-tad-pmu";
+ reg = <0x87e2 0x80000000 0x0 0x1000>;
+ marvell,tad-cnt = <1>;
+ marvell,tad-page-size = <0x1000>;
+ marvell,tad-pmu-page-size = <0x1000>;
+ };
+ };
diff --git a/Documentation/devicetree/bindings/pinctrl/apple,pinctrl.yaml b/Documentation/devicetree/bindings/pinctrl/apple,pinctrl.yaml
index 07b00de..572923d 100644
--- a/Documentation/devicetree/bindings/pinctrl/apple,pinctrl.yaml
+++ b/Documentation/devicetree/bindings/pinctrl/apple,pinctrl.yaml
@@ -17,7 +17,9 @@
properties:
compatible:
items:
- - const: apple,t8103-pinctrl
+ - enum:
+ - apple,t8103-pinctrl
+ - apple,t6000-pinctrl
- const: apple,pinctrl
reg:
@@ -50,6 +52,9 @@
'#interrupt-cells':
const: 2
+ power-domains:
+ maxItems: 1
+
patternProperties:
'-pins$':
type: object
diff --git a/Documentation/devicetree/bindings/pinctrl/starfive,jh7100-pinctrl.yaml b/Documentation/devicetree/bindings/pinctrl/starfive,jh7100-pinctrl.yaml
new file mode 100644
index 0000000..9296360
--- /dev/null
+++ b/Documentation/devicetree/bindings/pinctrl/starfive,jh7100-pinctrl.yaml
@@ -0,0 +1,307 @@
+# SPDX-License-Identifier: GPL-2.0-only OR BSD-2-Clause
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/pinctrl/starfive,jh7100-pinctrl.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: StarFive JH7100 Pin Controller Device Tree Bindings
+
+description: |
+ Bindings for the JH7100 RISC-V SoC from StarFive Ltd.
+
+ Out of the SoC's many pins only the ones named PAD_GPIO[0] to PAD_GPIO[63]
+ and PAD_FUNC_SHARE[0] to PAD_FUNC_SHARE[141] can be multiplexed and have
+ configurable bias, drive strength, schmitt trigger etc. The SoC has an
+ interesting 2-layered approach to pin muxing best illustrated by the diagram
+ below.
+
+ Signal group 0, 1, ... or 6
+ ___|___
+ | |
+ LCD output -----------------| |
+ CMOS Camera interface ------| |--- PAD_GPIO[0]
+ Ethernet PHY interface -----| MUX |--- PAD_GPIO[1]
+ ... | | ...
+ | |--- PAD_GPIO[63]
+ -------- GPIO0 ------------| |
+ | -------|-- GPIO1 --------| |--- PAD_FUNC_SHARE[0]
+ | | | | | |--- PAD_FUNC_SHARE[1]
+ | | | | ... | | ...
+ | | | | | |--- PAD_FUNC_SHARE[141]
+ | | -----|---|-- GPIO63 ---| |
+ | | | | | | -------
+ UART0 UART1 --
+
+
+ The big MUX in the diagram only has 7 different ways of mapping peripherals
+ on the left to pins on the right. StarFive calls the 7 configurations "signal
+ groups".
+ However some peripherals have their I/O go through the 64 "GPIOs". The
+ diagram only shows UART0 and UART1, but this also includes a number of other
+ UARTs, I2Cs, SPIs, PWMs etc. All these peripherals are connected to all 64
+ GPIOs such that any GPIO can be set up to be controlled by any of the
+ peripherals.
+ Note that signal group 0 doesn't map any of the GPIOs to pins, and only
+ signal group 1 maps the GPIOs to the pins named PAD_GPIO[0] to PAD_GPIO[63].
+
+maintainers:
+ - Emil Renner Berthing <kernel@esmil.dk>
+ - Drew Fustini <drew@beagleboard.org>
+
+properties:
+ compatible:
+ const: starfive,jh7100-pinctrl
+
+ reg:
+ minItems: 2
+ maxItems: 2
+
+ reg-names:
+ items:
+ - const: gpio
+ - const: padctl
+
+ clocks:
+ maxItems: 1
+
+ resets:
+ maxItems: 1
+
+ gpio-controller: true
+
+ "#gpio-cells":
+ const: 2
+
+ interrupts:
+ maxItems: 1
+ description: The GPIO parent interrupt.
+
+ interrupt-controller: true
+
+ "#interrupt-cells":
+ const: 2
+
+ starfive,signal-group:
+ description: |
+ Select one of the 7 signal groups. If this property is not set it
+ defaults to the configuration already chosen by the earlier boot stages.
+ $ref: /schemas/types.yaml#/definitions/uint32
+ enum: [0, 1, 2, 3, 4, 5, 6]
+
+required:
+ - compatible
+ - reg
+ - reg-names
+ - clocks
+ - gpio-controller
+ - "#gpio-cells"
+ - interrupts
+ - interrupt-controller
+ - "#interrupt-cells"
+
+patternProperties:
+ '-[0-9]+$':
+ type: object
+ patternProperties:
+ '-pins$':
+ type: object
+ description: |
+ A pinctrl node should contain at least one subnode representing the
+ pinctrl groups available on the machine. Each subnode will list the
+ pins it needs, and how they should be configured, with regard to
+ muxer configuration, bias, input enable/disable, input schmitt
+ trigger enable/disable, slew-rate and drive strength.
+ $ref: "/schemas/pinctrl/pincfg-node.yaml"
+
+ properties:
+ pins:
+ description: |
+ The list of pin identifiers that properties in the node apply to.
+ This should be set using either the PAD_GPIO or PAD_FUNC_SHARE
+ macros.
+ Either this or "pinmux" has to be specified, but not both.
+ $ref: "/schemas/pinctrl/pinmux-node.yaml#/properties/pins"
+
+ pinmux:
+ description: |
+ The list of GPIOs and their mux settings that properties in the
+ node apply to. This should be set using the GPIOMUX macro.
+ Either this or "pins" has to be specified, but not both.
+ $ref: "/schemas/pinctrl/pinmux-node.yaml#/properties/pinmux"
+
+ bias-disable: true
+
+ bias-pull-up:
+ type: boolean
+
+ bias-pull-down:
+ type: boolean
+
+ drive-strength:
+ enum: [ 14, 21, 28, 35, 42, 49, 56, 63 ]
+
+ input-enable: true
+
+ input-disable: true
+
+ input-schmitt-enable: true
+
+ input-schmitt-disable: true
+
+ slew-rate:
+ maximum: 7
+
+ starfive,strong-pull-up:
+ description: enable strong pull-up.
+ type: boolean
+
+ additionalProperties: false
+
+ additionalProperties: false
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/clock/starfive-jh7100.h>
+ #include <dt-bindings/reset/starfive-jh7100.h>
+ #include <dt-bindings/pinctrl/pinctrl-starfive.h>
+
+ soc {
+ #address-cells = <2>;
+ #size-cells = <2>;
+
+ pinctrl@11910000 {
+ compatible = "starfive,jh7100-pinctrl";
+ reg = <0x0 0x11910000 0x0 0x10000>,
+ <0x0 0x11858000 0x0 0x1000>;
+ reg-names = "gpio", "padctl";
+ clocks = <&clkgen JH7100_CLK_GPIO_APB>;
+ resets = <&clkgen JH7100_RSTN_GPIO_APB>;
+ interrupts = <32>;
+ gpio-controller;
+ #gpio-cells = <2>;
+ interrupt-controller;
+ #interrupt-cells = <2>;
+ starfive,signal-group = <6>;
+
+ gmac_pins_default: gmac-0 {
+ gtxclk-pins {
+ pins = <PAD_FUNC_SHARE(115)>;
+ bias-pull-up;
+ drive-strength = <35>;
+ input-enable;
+ input-schmitt-enable;
+ slew-rate = <0>;
+ };
+ miitxclk-pins {
+ pins = <PAD_FUNC_SHARE(116)>;
+ bias-pull-up;
+ drive-strength = <14>;
+ input-enable;
+ input-schmitt-disable;
+ slew-rate = <0>;
+ };
+ tx-pins {
+ pins = <PAD_FUNC_SHARE(117)>,
+ <PAD_FUNC_SHARE(119)>,
+ <PAD_FUNC_SHARE(120)>,
+ <PAD_FUNC_SHARE(121)>,
+ <PAD_FUNC_SHARE(122)>,
+ <PAD_FUNC_SHARE(123)>,
+ <PAD_FUNC_SHARE(124)>,
+ <PAD_FUNC_SHARE(125)>,
+ <PAD_FUNC_SHARE(126)>;
+ bias-disable;
+ drive-strength = <35>;
+ input-disable;
+ input-schmitt-disable;
+ slew-rate = <0>;
+ };
+ rxclk-pins {
+ pins = <PAD_FUNC_SHARE(127)>;
+ bias-pull-up;
+ drive-strength = <14>;
+ input-enable;
+ input-schmitt-disable;
+ slew-rate = <6>;
+ };
+ rxer-pins {
+ pins = <PAD_FUNC_SHARE(129)>;
+ bias-pull-up;
+ drive-strength = <14>;
+ input-enable;
+ input-schmitt-disable;
+ slew-rate = <0>;
+ };
+ rx-pins {
+ pins = <PAD_FUNC_SHARE(128)>,
+ <PAD_FUNC_SHARE(130)>,
+ <PAD_FUNC_SHARE(131)>,
+ <PAD_FUNC_SHARE(132)>,
+ <PAD_FUNC_SHARE(133)>,
+ <PAD_FUNC_SHARE(134)>,
+ <PAD_FUNC_SHARE(135)>,
+ <PAD_FUNC_SHARE(136)>,
+ <PAD_FUNC_SHARE(137)>,
+ <PAD_FUNC_SHARE(138)>,
+ <PAD_FUNC_SHARE(139)>,
+ <PAD_FUNC_SHARE(140)>,
+ <PAD_FUNC_SHARE(141)>;
+ bias-pull-up;
+ drive-strength = <14>;
+ input-enable;
+ input-schmitt-enable;
+ slew-rate = <0>;
+ };
+ };
+
+ i2c0_pins_default: i2c0-0 {
+ i2c-pins {
+ pinmux = <GPIOMUX(62, GPO_LOW,
+ GPO_I2C0_PAD_SCK_OEN,
+ GPI_I2C0_PAD_SCK_IN)>,
+ <GPIOMUX(61, GPO_LOW,
+ GPO_I2C0_PAD_SDA_OEN,
+ GPI_I2C0_PAD_SDA_IN)>;
+ bias-disable; /* external pull-up */
+ input-enable;
+ input-schmitt-enable;
+ };
+ };
+
+ uart3_pins_default: uart3-0 {
+ rx-pins {
+ pinmux = <GPIOMUX(13, GPO_LOW, GPO_DISABLE,
+ GPI_UART3_PAD_SIN)>;
+ bias-pull-up;
+ input-enable;
+ input-schmitt-enable;
+ };
+ tx-pins {
+ pinmux = <GPIOMUX(14, GPO_UART3_PAD_SOUT,
+ GPO_ENABLE, GPI_NONE)>;
+ bias-disable;
+ input-disable;
+ input-schmitt-disable;
+ };
+ };
+ };
+
+ gmac {
+ pinctrl-0 = <&gmac_pins_default>;
+ pinctrl-names = "default";
+ };
+
+ i2c0 {
+ pinctrl-0 = <&i2c0_pins_default>;
+ pinctrl-names = "default";
+ };
+
+ uart3 {
+ pinctrl-0 = <&uart3_pins_default>;
+ pinctrl-names = "default";
+ };
+ };
+
+...
diff --git a/Documentation/devicetree/bindings/power/apple,pmgr-pwrstate.yaml b/Documentation/devicetree/bindings/power/apple,pmgr-pwrstate.yaml
new file mode 100644
index 0000000..19a1949
--- /dev/null
+++ b/Documentation/devicetree/bindings/power/apple,pmgr-pwrstate.yaml
@@ -0,0 +1,79 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/power/apple,pmgr-pwrstate.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Apple SoC PMGR Power States
+
+maintainers:
+ - Hector Martin <marcan@marcan.st>
+
+allOf:
+ - $ref: "power-domain.yaml#"
+
+description: |
+ Apple SoCs include PMGR blocks responsible for power management,
+ which can control various clocks, resets, power states, and
+ performance features. This binding describes the device power
+ state registers, which control power states and resets.
+
+ Each instance of a power controller within the PMGR syscon node
+ represents a generic power domain provider, as documented in
+ Documentation/devicetree/bindings/power/power-domain.yaml.
+ The provider controls a single SoC block. The power hierarchy is
+ represented via power-domains relationships between these nodes.
+
+ See Documentation/devicetree/bindings/arm/apple/apple,pmgr.yaml
+ for the top-level PMGR node documentation.
+
+properties:
+ compatible:
+ items:
+ - enum:
+ - apple,t8103-pmgr-pwrstate
+ - apple,t6000-pmgr-pwrstate
+ - const: apple,pmgr-pwrstate
+
+ reg:
+ maxItems: 1
+
+ "#power-domain-cells":
+ const: 0
+
+ "#reset-cells":
+ const: 0
+
+ power-domains:
+ description:
+ Reference to parent power domains. A domain may have multiple parents,
+ and all will be powered up when it is powered.
+ minItems: 1
+ maxItems: 8 # Arbitrary, should be enough
+
+ label:
+ description:
+ Specifies the name of the SoC domain being controlled. This is used to
+ name the power/reset domains.
+
+ apple,always-on:
+ description:
+ Forces this power domain to always be powered up.
+ type: boolean
+
+ apple,min-state:
+ description:
+ Specifies the minimum power state for auto-PM.
+ 0 = power gated, 4 = clock gated, 15 = on.
+ $ref: /schemas/types.yaml#/definitions/uint32
+ minimum: 0
+ maximum: 15
+
+required:
+ - compatible
+ - reg
+ - "#power-domain-cells"
+ - "#reset-cells"
+ - label
+
+additionalProperties: false
diff --git a/Documentation/devicetree/bindings/power/qcom,rpmpd.yaml b/Documentation/devicetree/bindings/power/qcom,rpmpd.yaml
index e810480..f48bc41 100644
--- a/Documentation/devicetree/bindings/power/qcom,rpmpd.yaml
+++ b/Documentation/devicetree/bindings/power/qcom,rpmpd.yaml
@@ -24,6 +24,7 @@
- qcom,msm8994-rpmpd
- qcom,msm8996-rpmpd
- qcom,msm8998-rpmpd
+ - qcom,qcm2290-rpmpd
- qcom,qcs404-rpmpd
- qcom,sdm660-rpmpd
- qcom,sc7180-rpmhpd
@@ -32,10 +33,12 @@
- qcom,sdm845-rpmhpd
- qcom,sdx55-rpmhpd
- qcom,sm6115-rpmpd
+ - qcom,sm6125-rpmpd
- qcom,sm6350-rpmhpd
- qcom,sm8150-rpmhpd
- qcom,sm8250-rpmhpd
- qcom,sm8350-rpmhpd
+ - qcom,sm8450-rpmhpd
'#power-domain-cells':
const: 1
diff --git a/Documentation/devicetree/bindings/power/renesas,rcar-sysc.yaml b/Documentation/devicetree/bindings/power/renesas,rcar-sysc.yaml
index 99e8042..62a49ca 100644
--- a/Documentation/devicetree/bindings/power/renesas,rcar-sysc.yaml
+++ b/Documentation/devicetree/bindings/power/renesas,rcar-sysc.yaml
@@ -41,6 +41,7 @@
- renesas,r8a77990-sysc # R-Car E3
- renesas,r8a77995-sysc # R-Car D3
- renesas,r8a779a0-sysc # R-Car V3U
+ - renesas,r8a779f0-sysc # R-Car S4-8
reg:
maxItems: 1
diff --git a/Documentation/devicetree/bindings/reset/renesas,rst.yaml b/Documentation/devicetree/bindings/reset/renesas,rst.yaml
index 620cd05..bbe313b 100644
--- a/Documentation/devicetree/bindings/reset/renesas,rst.yaml
+++ b/Documentation/devicetree/bindings/reset/renesas,rst.yaml
@@ -48,6 +48,7 @@
- renesas,r8a77990-rst # R-Car E3
- renesas,r8a77995-rst # R-Car D3
- renesas,r8a779a0-rst # R-Car V3U
+ - renesas,r8a779f0-rst # R-Car S4-8
reg:
maxItems: 1
diff --git a/Documentation/devicetree/bindings/reset/starfive,jh7100-reset.yaml b/Documentation/devicetree/bindings/reset/starfive,jh7100-reset.yaml
new file mode 100644
index 0000000..300359a
--- /dev/null
+++ b/Documentation/devicetree/bindings/reset/starfive,jh7100-reset.yaml
@@ -0,0 +1,38 @@
+# SPDX-License-Identifier: GPL-2.0-only OR BSD-2-Clause
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/reset/starfive,jh7100-reset.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: StarFive JH7100 SoC Reset Controller Device Tree Bindings
+
+maintainers:
+ - Emil Renner Berthing <kernel@esmil.dk>
+
+properties:
+ compatible:
+ enum:
+ - starfive,jh7100-reset
+
+ reg:
+ maxItems: 1
+
+ "#reset-cells":
+ const: 1
+
+required:
+ - compatible
+ - reg
+ - "#reset-cells"
+
+additionalProperties: false
+
+examples:
+ - |
+ reset-controller@11840000 {
+ compatible = "starfive,jh7100-reset";
+ reg = <0x11840000 0x10000>;
+ #reset-cells = <1>;
+ };
+
+...
diff --git a/Documentation/devicetree/bindings/rtc/nvidia,tegra20-rtc.txt b/Documentation/devicetree/bindings/rtc/nvidia,tegra20-rtc.txt
deleted file mode 100644
index b7d98ed..0000000
--- a/Documentation/devicetree/bindings/rtc/nvidia,tegra20-rtc.txt
+++ /dev/null
@@ -1,24 +0,0 @@
-NVIDIA Tegra20 real-time clock
-
-The Tegra RTC maintains seconds and milliseconds counters, and five alarm
-registers. The alarms and other interrupts may wake the system from low-power
-state.
-
-Required properties:
-
-- compatible : For Tegra20, must contain "nvidia,tegra20-rtc". Otherwise,
- must contain '"nvidia,<chip>-rtc", "nvidia,tegra20-rtc"', where <chip>
- can be tegra30, tegra114, tegra124, or tegra132.
-- reg : Specifies base physical address and size of the registers.
-- interrupts : A single interrupt specifier.
-- clocks : Must contain one entry, for the module clock.
- See ../clocks/clock-bindings.txt for details.
-
-Example:
-
-timer {
- compatible = "nvidia,tegra20-rtc";
- reg = <0x7000e000 0x100>;
- interrupts = <0 2 0x04>;
- clocks = <&tegra_car 4>;
-};
diff --git a/Documentation/devicetree/bindings/rtc/nvidia,tegra20-rtc.yaml b/Documentation/devicetree/bindings/rtc/nvidia,tegra20-rtc.yaml
new file mode 100644
index 0000000..17d6280
--- /dev/null
+++ b/Documentation/devicetree/bindings/rtc/nvidia,tegra20-rtc.yaml
@@ -0,0 +1,61 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/rtc/nvidia,tegra20-rtc.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: NVIDIA Tegra real-time clock
+
+maintainers:
+ - Thierry Reding <thierry.reding@gmail.com>
+ - Jon Hunter <jonathanh@nvidia.com>
+
+description: |
+ The Tegra RTC maintains seconds and milliseconds counters, and five
+ alarm registers. The alarms and other interrupts may wake the system
+ from low-power state.
+
+properties:
+ compatible:
+ oneOf:
+ - const: nvidia,tegra20-rtc
+ - items:
+ - enum:
+ - nvidia,tegra30-rtc
+ - nvidia,tegra114-rtc
+ - nvidia,tegra124-rtc
+ - nvidia,tegra210-rtc
+ - nvidia,tegra186-rtc
+ - nvidia,tegra194-rtc
+ - nvidia,tegra234-rtc
+ - const: nvidia,tegra20-rtc
+
+ reg:
+ maxItems: 1
+
+ interrupts:
+ maxItems: 1
+
+ clocks:
+ maxItems: 1
+
+ clock-names:
+ items:
+ - const: rtc
+
+additionalProperties: false
+
+required:
+ - compatible
+ - reg
+ - interrupts
+ - clocks
+
+examples:
+ - |
+ timer@7000e000 {
+ compatible = "nvidia,tegra20-rtc";
+ reg = <0x7000e000 0x100>;
+ interrupts = <0 2 0x04>;
+ clocks = <&tegra_car 4>;
+ };
diff --git a/Documentation/devicetree/bindings/serial/8250.yaml b/Documentation/devicetree/bindings/serial/8250.yaml
index fa76744..3bab2f2 100644
--- a/Documentation/devicetree/bindings/serial/8250.yaml
+++ b/Documentation/devicetree/bindings/serial/8250.yaml
@@ -113,9 +113,10 @@
- nvidia,tegra30-uart
- nvidia,tegra114-uart
- nvidia,tegra124-uart
+ - nvidia,tegra210-uart
- nvidia,tegra186-uart
- nvidia,tegra194-uart
- - nvidia,tegra210-uart
+ - nvidia,tegra234-uart
- const: nvidia,tegra20-uart
reg:
diff --git a/Documentation/devicetree/bindings/serial/nvidia,tegra194-tcu.txt b/Documentation/devicetree/bindings/serial/nvidia,tegra194-tcu.txt
deleted file mode 100644
index 085a859..0000000
--- a/Documentation/devicetree/bindings/serial/nvidia,tegra194-tcu.txt
+++ /dev/null
@@ -1,35 +0,0 @@
-NVIDIA Tegra Combined UART (TCU)
-
-The TCU is a system for sharing a hardware UART instance among multiple
-systems within the Tegra SoC. It is implemented through a mailbox-
-based protocol where each "virtual UART" has a pair of mailboxes, one
-for transmitting and one for receiving, that is used to communicate
-with the hardware implementing the TCU.
-
-Required properties:
-- name : Should be tcu
-- compatible
- Array of strings
- One of:
- - "nvidia,tegra194-tcu"
-- mbox-names:
- "rx" - Mailbox for receiving data from hardware UART
- "tx" - Mailbox for transmitting data to hardware UART
-- mboxes: Mailboxes corresponding to the mbox-names.
-
-This node is a mailbox consumer. See the following files for details of
-the mailbox subsystem, and the specifiers implemented by the relevant
-provider(s):
-
-- .../mailbox/mailbox.txt
-- .../mailbox/nvidia,tegra186-hsp.txt
-
-Example bindings:
------------------
-
-tcu: tcu {
- compatible = "nvidia,tegra194-tcu";
- mboxes = <&hsp_top0 TEGRA_HSP_MBOX_TYPE_SM 0>,
- <&hsp_aon TEGRA_HSP_MBOX_TYPE_SM 1>;
- mbox-names = "rx", "tx";
-};
diff --git a/Documentation/devicetree/bindings/serial/nvidia,tegra194-tcu.yaml b/Documentation/devicetree/bindings/serial/nvidia,tegra194-tcu.yaml
new file mode 100644
index 0000000..e2d111b
--- /dev/null
+++ b/Documentation/devicetree/bindings/serial/nvidia,tegra194-tcu.yaml
@@ -0,0 +1,61 @@
+# SPDX-License-Identifier: GPL-2.0
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/serial/nvidia,tegra194-tcu.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: NVIDIA Tegra Combined UART (TCU)
+
+maintainers:
+ - Thierry Reding <thierry.reding@gmail.com>
+ - Jonathan Hunter <jonathanh@nvidia.com>
+
+description:
+ The TCU is a system for sharing a hardware UART instance among multiple
+ systems within the Tegra SoC. It is implemented through a mailbox-
+ based protocol where each "virtual UART" has a pair of mailboxes, one
+ for transmitting and one for receiving, that is used to communicate
+ with the hardware implementing the TCU.
+
+properties:
+ $nodename:
+ pattern: "^serial(@.*)?$"
+
+ compatible:
+ oneOf:
+ - const: nvidia,tegra194-tcu
+ - items:
+ - enum:
+ - nvidia,tegra234-tcu
+ - const: nvidia,tegra194-tcu
+
+ mbox-names:
+ items:
+ - const: rx
+ - const: tx
+
+ mboxes:
+ description: |
+ List of phandles to mailbox channels used for receiving and
+ transmitting data from and to the hardware UART.
+ items:
+ - description: mailbox for receiving data from hardware UART
+ - description: mailbox for transmitting data to hardware UART
+
+required:
+ - compatible
+ - mbox-names
+ - mboxes
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/mailbox/tegra186-hsp.h>
+
+ tcu: serial {
+ compatible = "nvidia,tegra194-tcu";
+ mboxes = <&hsp_top0 TEGRA_HSP_MBOX_TYPE_SM 0>,
+ <&hsp_aon TEGRA_HSP_MBOX_TYPE_SM 1>;
+ mbox-names = "rx", "tx";
+ };
diff --git a/Documentation/devicetree/bindings/serial/snps-dw-apb-uart.yaml b/Documentation/devicetree/bindings/serial/snps-dw-apb-uart.yaml
index b49fda5..12137fe 100644
--- a/Documentation/devicetree/bindings/serial/snps-dw-apb-uart.yaml
+++ b/Documentation/devicetree/bindings/serial/snps-dw-apb-uart.yaml
@@ -40,6 +40,11 @@
- brcm,bcm11351-dw-apb-uart
- brcm,bcm21664-dw-apb-uart
- const: snps,dw-apb-uart
+ - items:
+ - enum:
+ - starfive,jh7100-hsuart
+ - starfive,jh7100-uart
+ - const: snps,dw-apb-uart
- const: snps,dw-apb-uart
reg:
diff --git a/Documentation/devicetree/bindings/soc/bcm/brcm,bcm2835-vchiq.txt b/Documentation/devicetree/bindings/soc/bcm/brcm,bcm2835-vchiq.txt
deleted file mode 100644
index f331316..0000000
--- a/Documentation/devicetree/bindings/soc/bcm/brcm,bcm2835-vchiq.txt
+++ /dev/null
@@ -1,17 +0,0 @@
-Broadcom VCHIQ firmware services
-
-Required properties:
-
-- compatible: Should be "brcm,bcm2835-vchiq" on BCM2835, otherwise
- "brcm,bcm2836-vchiq".
-- reg: Physical base address and length of the doorbell register pair
-- interrupts: The interrupt number
- See bindings/interrupt-controller/brcm,bcm2835-armctrl-ic.txt
-
-Example:
-
-mailbox@7e00b840 {
- compatible = "brcm,bcm2835-vchiq";
- reg = <0x7e00b840 0xf>;
- interrupts = <0 2>;
-};
diff --git a/Documentation/devicetree/bindings/soc/bcm/brcm,bcm2835-vchiq.yaml b/Documentation/devicetree/bindings/soc/bcm/brcm,bcm2835-vchiq.yaml
new file mode 100644
index 0000000..e04439b
--- /dev/null
+++ b/Documentation/devicetree/bindings/soc/bcm/brcm,bcm2835-vchiq.yaml
@@ -0,0 +1,53 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/soc/bcm/brcm,bcm2835-vchiq.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Broadcom VCHIQ firmware services
+
+maintainers:
+ - Nicolas Saenz Julienne <nsaenz@kernel.org>
+
+description:
+ The VCHIQ communication channel can be provided by BCM283x and Capri SoCs,
+ to communicate with the VPU-side OS services.
+
+properties:
+ compatible:
+ oneOf:
+ - description: BCM2835 based boards
+ items:
+ - enum:
+ - brcm,bcm2835-vchiq
+
+ - description: BCM2836/BCM2837 based boards
+ items:
+ - enum:
+ - brcm,bcm2836-vchiq
+ - const: brcm,bcm2835-vchiq
+
+ reg:
+ description: Physical base address and length of the doorbell register pair
+ minItems: 1
+
+ interrupts:
+ description: Interrupt number of the doorbell interrupt
+ minItems: 1
+
+required:
+ - compatible
+ - reg
+ - interrupts
+
+additionalProperties: false
+
+examples:
+ - |
+ mailbox@7e00b840 {
+ compatible = "brcm,bcm2835-vchiq";
+ reg = <0x7e00b840 0xf>;
+ interrupts = <0 2>;
+ };
+
+...
diff --git a/Documentation/devicetree/bindings/soc/imx/fsl,imx8mn-disp-blk-ctrl.yaml b/Documentation/devicetree/bindings/soc/imx/fsl,imx8mn-disp-blk-ctrl.yaml
new file mode 100644
index 0000000..fbeaac3
--- /dev/null
+++ b/Documentation/devicetree/bindings/soc/imx/fsl,imx8mn-disp-blk-ctrl.yaml
@@ -0,0 +1,97 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/soc/imx/fsl,imx8mn-disp-blk-ctrl.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: NXP i.MX8MN DISP blk-ctrl
+
+maintainers:
+ - Lucas Stach <l.stach@pengutronix.de>
+
+description:
+ The i.MX8MN DISP blk-ctrl is a top-level peripheral providing access to
+ the NoC and ensuring proper power sequencing of the display and MIPI CSI
+ peripherals located in the DISP domain of the SoC.
+
+properties:
+ compatible:
+ items:
+ - const: fsl,imx8mn-disp-blk-ctrl
+ - const: syscon
+
+ reg:
+ maxItems: 1
+
+ '#power-domain-cells':
+ const: 1
+
+ power-domains:
+ minItems: 5
+ maxItems: 5
+
+ power-domain-names:
+ items:
+ - const: bus
+ - const: isi
+ - const: lcdif
+ - const: mipi-dsi
+ - const: mipi-csi
+
+ clocks:
+ minItems: 11
+ maxItems: 11
+
+ clock-names:
+ items:
+ - const: disp_axi
+ - const: disp_apb
+ - const: disp_axi_root
+ - const: disp_apb_root
+ - const: lcdif-axi
+ - const: lcdif-apb
+ - const: lcdif-pix
+ - const: dsi-pclk
+ - const: dsi-ref
+ - const: csi-aclk
+ - const: csi-pclk
+
+required:
+ - compatible
+ - reg
+ - power-domains
+ - power-domain-names
+ - clocks
+ - clock-names
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/clock/imx8mn-clock.h>
+ #include <dt-bindings/power/imx8mn-power.h>
+
+ disp_blk_ctl: blk_ctrl@32e28000 {
+ compatible = "fsl,imx8mn-disp-blk-ctrl", "syscon";
+ reg = <0x32e28000 0x100>;
+ power-domains = <&pgc_dispmix>, <&pgc_dispmix>,
+ <&pgc_dispmix>, <&pgc_mipi>,
+ <&pgc_mipi>;
+ power-domain-names = "bus", "isi", "lcdif", "mipi-dsi",
+ "mipi-csi";
+ clocks = <&clk IMX8MN_CLK_DISP_AXI>,
+ <&clk IMX8MN_CLK_DISP_APB>,
+ <&clk IMX8MN_CLK_DISP_AXI_ROOT>,
+ <&clk IMX8MN_CLK_DISP_APB_ROOT>,
+ <&clk IMX8MN_CLK_DISP_AXI_ROOT>,
+ <&clk IMX8MN_CLK_DISP_APB_ROOT>,
+ <&clk IMX8MN_CLK_DISP_PIXEL_ROOT>,
+ <&clk IMX8MN_CLK_DSI_CORE>,
+ <&clk IMX8MN_CLK_DSI_PHY_REF>,
+ <&clk IMX8MN_CLK_CSI1_PHY_REF>,
+ <&clk IMX8MN_CLK_CAMERA_PIXEL_ROOT>;
+ clock-names = "disp_axi", "disp_apb", "disp_axi_root", "disp_apb_root",
+ "lcdif-axi", "lcdif-apb", "lcdif-pix", "dsi-pclk",
+ "dsi-ref", "csi-aclk", "csi-pclk";
+ #power-domain-cells = <1>;
+ };
diff --git a/Documentation/devicetree/bindings/soc/qcom/qcom-stats.yaml b/Documentation/devicetree/bindings/soc/qcom/qcom-stats.yaml
index 99dff7d..473adca 100644
--- a/Documentation/devicetree/bindings/soc/qcom/qcom-stats.yaml
+++ b/Documentation/devicetree/bindings/soc/qcom/qcom-stats.yaml
@@ -21,6 +21,11 @@
enum:
- qcom,rpmh-stats
- qcom,rpm-stats
+ # For older RPM firmware versions with fixed offset for the sleep stats
+ - qcom,apq8084-rpm-stats
+ - qcom,msm8226-rpm-stats
+ - qcom,msm8916-rpm-stats
+ - qcom,msm8974-rpm-stats
reg:
maxItems: 1
diff --git a/Documentation/devicetree/bindings/soc/samsung/exynos-chipid.yaml b/Documentation/devicetree/bindings/soc/samsung/exynos-chipid.yaml
new file mode 100644
index 0000000..4bb8efb
--- /dev/null
+++ b/Documentation/devicetree/bindings/soc/samsung/exynos-chipid.yaml
@@ -0,0 +1,41 @@
+# SPDX-License-Identifier: GPL-2.0
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/soc/samsung/exynos-chipid.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Samsung Exynos SoC series Chipid driver
+
+maintainers:
+ - Krzysztof Kozlowski <krzk@kernel.org>
+
+properties:
+ compatible:
+ enum:
+ - samsung,exynos4210-chipid
+ - samsung,exynos850-chipid
+
+ reg:
+ maxItems: 1
+
+ samsung,asv-bin:
+ description:
+ Adaptive Supply Voltage bin selection. This can be used
+ to determine the ASV bin of an SoC if respective information
+ is missing in the CHIPID registers or in the OTP memory.
+ $ref: /schemas/types.yaml#/definitions/uint32
+ enum: [0, 1, 2, 3]
+
+required:
+ - compatible
+ - reg
+
+additionalProperties: false
+
+examples:
+ - |
+ chipid@10000000 {
+ compatible = "samsung,exynos4210-chipid";
+ reg = <0x10000000 0x100>;
+ samsung,asv-bin = <2>;
+ };
diff --git a/Documentation/devicetree/bindings/soc/samsung/exynos-pmu.yaml b/Documentation/devicetree/bindings/soc/samsung/exynos-pmu.yaml
new file mode 100644
index 0000000..c30a643
--- /dev/null
+++ b/Documentation/devicetree/bindings/soc/samsung/exynos-pmu.yaml
@@ -0,0 +1,130 @@
+# SPDX-License-Identifier: GPL-2.0
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/soc/samsung/exynos-pmu.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Samsung Exynos SoC series Power Management Unit (PMU)
+
+maintainers:
+ - Krzysztof Kozlowski <krzk@kernel.org>
+
+# Custom select to avoid matching all nodes with 'syscon'
+select:
+ properties:
+ compatible:
+ contains:
+ enum:
+ - samsung,exynos3250-pmu
+ - samsung,exynos4210-pmu
+ - samsung,exynos4412-pmu
+ - samsung,exynos5250-pmu
+ - samsung,exynos5260-pmu
+ - samsung,exynos5410-pmu
+ - samsung,exynos5420-pmu
+ - samsung,exynos5433-pmu
+ - samsung,exynos7-pmu
+ - samsung,exynos850-pmu
+ - samsung-s5pv210-pmu
+ required:
+ - compatible
+
+properties:
+ compatible:
+ items:
+ - enum:
+ - samsung,exynos3250-pmu
+ - samsung,exynos4210-pmu
+ - samsung,exynos4412-pmu
+ - samsung,exynos5250-pmu
+ - samsung,exynos5260-pmu
+ - samsung,exynos5410-pmu
+ - samsung,exynos5420-pmu
+ - samsung,exynos5433-pmu
+ - samsung,exynos7-pmu
+ - samsung,exynos850-pmu
+ - samsung-s5pv210-pmu
+ - const: syscon
+
+ reg:
+ maxItems: 1
+
+ assigned-clock-parents: true
+ assigned-clocks: true
+
+ '#clock-cells':
+ const: 1
+
+ clock-names:
+ description:
+ List of clock names for particular CLKOUT mux inputs
+ minItems: 1
+ maxItems: 32
+ items:
+ pattern: '^clkout([0-9]|[12][0-9]|3[0-1])$'
+
+ clocks:
+ minItems: 1
+ maxItems: 32
+
+ interrupt-controller:
+ description:
+ Some PMUs are capable of behaving as an interrupt controller (mostly
+ to wake up a suspended PMU).
+
+ '#interrupt-cells':
+ description:
+ Must be identical to the that of the parent interrupt controller.
+ const: 3
+
+ syscon-poweroff:
+ $ref: "../../power/reset/syscon-poweroff.yaml#"
+ type: object
+ description:
+ Node for power off method
+
+ syscon-reboot:
+ $ref: "../../power/reset/syscon-reboot.yaml#"
+ type: object
+ description:
+ Node for reboot method
+
+required:
+ - compatible
+ - reg
+
+additionalProperties: false
+
+allOf:
+ - if:
+ properties:
+ compatible:
+ contains:
+ enum:
+ - samsung,exynos3250-pmu
+ - samsung,exynos4210-pmu
+ - samsung,exynos4412-pmu
+ - samsung,exynos5250-pmu
+ - samsung,exynos5410-pmu
+ - samsung,exynos5420-pmu
+ - samsung,exynos5433-pmu
+ then:
+ required:
+ - '#clock-cells'
+ - clock-names
+ - clocks
+
+examples:
+ - |
+ #include <dt-bindings/clock/exynos5250.h>
+
+ pmu_system_controller: system-controller@10040000 {
+ compatible = "samsung,exynos5250-pmu", "syscon";
+ reg = <0x10040000 0x5000>;
+ interrupt-controller;
+ #interrupt-cells = <3>;
+ interrupt-parent = <&gic>;
+ #clock-cells = <1>;
+ clock-names = "clkout16";
+ clocks = <&clock CLK_FIN_PLL>;
+ };
diff --git a/Documentation/devicetree/bindings/soc/samsung/exynos-usi.yaml b/Documentation/devicetree/bindings/soc/samsung/exynos-usi.yaml
new file mode 100644
index 0000000..273f2d9
--- /dev/null
+++ b/Documentation/devicetree/bindings/soc/samsung/exynos-usi.yaml
@@ -0,0 +1,159 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/soc/samsung/exynos-usi.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Samsung's Exynos USI (Universal Serial Interface) binding
+
+maintainers:
+ - Sam Protsenko <semen.protsenko@linaro.org>
+ - Krzysztof Kozlowski <krzysztof.kozlowski@canonical.com>
+
+description: |
+ USI IP-core provides selectable serial protocol (UART, SPI or High-Speed I2C).
+ USI shares almost all internal circuits within each protocol, so only one
+ protocol can be chosen at a time. USI is modeled as a node with zero or more
+ child nodes, each representing a serial sub-node device. The mode setting
+ selects which particular function will be used.
+
+ Refer to next bindings documentation for information on protocol subnodes that
+ can exist under USI node:
+
+ [1] Documentation/devicetree/bindings/serial/samsung_uart.yaml
+ [2] Documentation/devicetree/bindings/i2c/i2c-exynos5.txt
+ [3] Documentation/devicetree/bindings/spi/spi-samsung.txt
+
+properties:
+ $nodename:
+ pattern: "^usi@[0-9a-f]+$"
+
+ compatible:
+ enum:
+ - samsung,exynos850-usi # for USIv2 (Exynos850, ExynosAutoV9)
+
+ reg: true
+
+ clocks: true
+
+ clock-names: true
+
+ ranges: true
+
+ "#address-cells":
+ const: 1
+
+ "#size-cells":
+ const: 1
+
+ samsung,sysreg:
+ $ref: /schemas/types.yaml#/definitions/phandle-array
+ description:
+ Should be phandle/offset pair. The phandle to System Register syscon node
+ (for the same domain where this USI controller resides) and the offset
+ of SW_CONF register for this USI controller.
+
+ samsung,mode:
+ $ref: /schemas/types.yaml#/definitions/uint32
+ description:
+ Selects USI function (which serial protocol to use). Refer to
+ <include/dt-bindings/soc/samsung,exynos-usi.h> for valid USI mode values.
+
+ samsung,clkreq-on:
+ type: boolean
+ description:
+ Enable this property if underlying protocol requires the clock to be
+ continuously provided without automatic gating. As suggested by SoC
+ manual, it should be set in case of SPI/I2C slave, UART Rx and I2C
+ multi-master mode. Usually this property is needed if USI mode is set
+ to "UART".
+
+ This property is optional.
+
+patternProperties:
+ # All other properties should be child nodes
+ "^(serial|spi|i2c)@[0-9a-f]+$":
+ type: object
+ description: Child node describing underlying USI serial protocol
+
+required:
+ - compatible
+ - ranges
+ - "#address-cells"
+ - "#size-cells"
+ - samsung,sysreg
+ - samsung,mode
+
+if:
+ properties:
+ compatible:
+ contains:
+ enum:
+ - samsung,exynos850-usi
+
+then:
+ properties:
+ reg:
+ maxItems: 1
+
+ clocks:
+ items:
+ - description: Bus (APB) clock
+ - description: Operating clock for UART/SPI/I2C protocol
+
+ clock-names:
+ items:
+ - const: pclk
+ - const: ipclk
+
+ required:
+ - reg
+ - clocks
+ - clock-names
+
+else:
+ properties:
+ reg: false
+ clocks: false
+ clock-names: false
+ samsung,clkreq-on: false
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+ #include <dt-bindings/soc/samsung,exynos-usi.h>
+
+ usi0: usi@138200c0 {
+ compatible = "samsung,exynos850-usi";
+ reg = <0x138200c0 0x20>;
+ samsung,sysreg = <&sysreg_peri 0x1010>;
+ samsung,mode = <USI_V2_UART>;
+ samsung,clkreq-on; /* needed for UART mode */
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges;
+ clocks = <&cmu_peri 32>, <&cmu_peri 31>;
+ clock-names = "pclk", "ipclk";
+
+ serial_0: serial@13820000 {
+ compatible = "samsung,exynos850-uart";
+ reg = <0x13820000 0xc0>;
+ interrupts = <GIC_SPI 227 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&cmu_peri 32>, <&cmu_peri 31>;
+ clock-names = "uart", "clk_uart_baud0";
+ status = "disabled";
+ };
+
+ hsi2c_0: i2c@13820000 {
+ compatible = "samsung,exynosautov9-hsi2c";
+ reg = <0x13820000 0xc0>;
+ interrupts = <GIC_SPI 227 IRQ_TYPE_LEVEL_HIGH>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ clocks = <&cmu_peri 31>, <&cmu_peri 32>;
+ clock-names = "hsi2c", "hsi2c_pclk";
+ status = "disabled";
+ };
+ };
diff --git a/Documentation/devicetree/bindings/spi/cdns,qspi-nor.yaml b/Documentation/devicetree/bindings/spi/cdns,qspi-nor.yaml
index ca155ab..037f41f 100644
--- a/Documentation/devicetree/bindings/spi/cdns,qspi-nor.yaml
+++ b/Documentation/devicetree/bindings/spi/cdns,qspi-nor.yaml
@@ -29,6 +29,7 @@
- ti,am654-ospi
- intel,lgm-qspi
- xlnx,versal-ospi-1.0
+ - intel,socfpga-qspi
- const: cdns,qspi-nor
- const: cdns,qspi-nor
diff --git a/Documentation/devicetree/bindings/sram/sram.yaml b/Documentation/devicetree/bindings/sram/sram.yaml
index d4e418b..668a9a4 100644
--- a/Documentation/devicetree/bindings/sram/sram.yaml
+++ b/Documentation/devicetree/bindings/sram/sram.yaml
@@ -31,6 +31,9 @@
- amlogic,meson-gxbb-sram
- arm,juno-sram-ns
- atmel,sama5d2-securam
+ - nvidia,tegra186-sysram
+ - nvidia,tegra194-sysram
+ - nvidia,tegra234-sysram
- qcom,rpm-msg-ram
- rockchip,rk3288-pmu-sram
diff --git a/Documentation/devicetree/bindings/thermal/nvidia,tegra186-bpmp-thermal.txt b/Documentation/devicetree/bindings/thermal/nvidia,tegra186-bpmp-thermal.txt
deleted file mode 100644
index fc87f6a..0000000
--- a/Documentation/devicetree/bindings/thermal/nvidia,tegra186-bpmp-thermal.txt
+++ /dev/null
@@ -1,33 +0,0 @@
-NVIDIA Tegra186 BPMP thermal sensor
-
-In Tegra186, the BPMP (Boot and Power Management Processor) implements an
-interface that is used to read system temperatures, including CPU cluster
-and GPU temperatures. This binding describes the thermal sensor that is
-exposed by BPMP.
-
-The BPMP thermal node must be located directly inside the main BPMP node. See
-../firmware/nvidia,tegra186-bpmp.txt for details of the BPMP binding.
-
-This node represents a thermal sensor. See Documentation/devicetree/bindings/thermal/thermal-sensor.yaml for details of the
-core thermal binding.
-
-Required properties:
-- compatible:
- Array of strings.
- One of:
- - "nvidia,tegra186-bpmp-thermal"
- - "nvidia,tegra194-bpmp-thermal"
-- #thermal-sensor-cells: Cell for sensor index.
- Single-cell integer.
- Must be <1>.
-
-Example:
-
-bpmp {
- ...
-
- bpmp_thermal: thermal {
- compatible = "nvidia,tegra186-bpmp-thermal";
- #thermal-sensor-cells = <1>;
- };
-};
diff --git a/Documentation/devicetree/bindings/thermal/nvidia,tegra186-bpmp-thermal.yaml b/Documentation/devicetree/bindings/thermal/nvidia,tegra186-bpmp-thermal.yaml
new file mode 100644
index 0000000..c91fd07
--- /dev/null
+++ b/Documentation/devicetree/bindings/thermal/nvidia,tegra186-bpmp-thermal.yaml
@@ -0,0 +1,42 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/thermal/nvidia,tegra186-bpmp-thermal.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: NVIDIA Tegra186 (and later) BPMP thermal sensor
+
+maintainers:
+ - Thierry Reding <thierry.reding@gmail.com>
+ - Jon Hunter <jonathanh@nvidia.com>
+
+description: |
+ In Tegra186, the BPMP (Boot and Power Management Processor) implements
+ an interface that is used to read system temperatures, including CPU
+ cluster and GPU temperatures. This binding describes the thermal
+ sensor that is exposed by BPMP.
+
+ The BPMP thermal node must be located directly inside the main BPMP
+ node. See ../firmware/nvidia,tegra186-bpmp.yaml for details of the
+ BPMP binding.
+
+ This node represents a thermal sensor. See
+
+ Documentation/devicetree/bindings/thermal/thermal-sensor.yaml
+
+ for details of the core thermal binding.
+
+properties:
+ compatible:
+ enum:
+ - nvidia,tegra186-bpmp-thermal
+ - nvidia,tegra194-bpmp-thermal
+
+ '#thermal-sensor-cells':
+ $ref: /schemas/types.yaml#/definitions/uint32
+ description: Number of cells needed in the phandle specifier to
+ identify a given sensor. Must be 1 and the single cell specifies
+ the sensor index.
+ const: 1
+
+additionalProperties: false
diff --git a/Documentation/devicetree/bindings/timer/sifive,clint.yaml b/Documentation/devicetree/bindings/timer/sifive,clint.yaml
index a35952f..8d5f468 100644
--- a/Documentation/devicetree/bindings/timer/sifive,clint.yaml
+++ b/Documentation/devicetree/bindings/timer/sifive,clint.yaml
@@ -25,6 +25,7 @@
items:
- enum:
- sifive,fu540-c000-clint
+ - starfive,jh7100-clint
- canaan,k210-clint
- const: sifive,clint0
diff --git a/Documentation/devicetree/bindings/usb/nvidia,tegra-xudc.yaml b/Documentation/devicetree/bindings/usb/nvidia,tegra-xudc.yaml
index 8428415..a39c76b 100644
--- a/Documentation/devicetree/bindings/usb/nvidia,tegra-xudc.yaml
+++ b/Documentation/devicetree/bindings/usb/nvidia,tegra-xudc.yaml
@@ -59,6 +59,19 @@
- const: fs_src
- const: hs_src
+ interconnects:
+ items:
+ - description: memory read client
+ - description: memory write client
+
+ interconnect-names:
+ items:
+ - const: dma-mem # read
+ - const: write
+
+ iommus:
+ maxItems: 1
+
power-domains:
items:
- description: XUSBB(device) power-domain
diff --git a/Documentation/devicetree/bindings/vendor-prefixes.yaml b/Documentation/devicetree/bindings/vendor-prefixes.yaml
index 66d6432..ae2aaab 100644
--- a/Documentation/devicetree/bindings/vendor-prefixes.yaml
+++ b/Documentation/devicetree/bindings/vendor-prefixes.yaml
@@ -187,6 +187,8 @@
description: Shanghai Broadmobi Communication Technology Co.,Ltd.
"^brcm,.*":
description: Broadcom Corporation
+ "^bsh,.*":
+ description: BSH Hausgeraete GmbH
"^buffalo,.*":
description: Buffalo, Inc.
"^bur,.*":
@@ -379,6 +381,8 @@
description: Silicon Laboratories (formerly Energy Micro AS)
"^engicam,.*":
description: Engicam S.r.l.
+ "^engleder,.*":
+ description: Engleder
"^epcos,.*":
description: EPCOS AG
"^epfl,.*":
@@ -593,6 +597,8 @@
description: JetHome (IP Sokolov P.A.)
"^jianda,.*":
description: Jiandangjing Technology Co., Ltd.
+ "^joz,.*":
+ description: JOZ BV
"^kam,.*":
description: Kamstrup A/S
"^karo,.*":
@@ -1236,6 +1242,8 @@
description: Truly Semiconductors Limited
"^visionox,.*":
description: Visionox
+ "^team-source-display,.*":
+ description: Shenzhen Team Source Display Technology Co., Ltd. (TSD)
"^tsd,.*":
description: Theobroma Systems Design und Consulting GmbH
"^tyan,.*":
@@ -1272,6 +1280,8 @@
description: Variscite Ltd.
"^vdl,.*":
description: Van der Laan b.v.
+ "^vertexcom,.*":
+ description: Vertexcom Technologies, Inc.
"^via,.*":
description: VIA Technologies, Inc.
"^videostrong,.*":
@@ -1318,6 +1328,8 @@
description: Wiligear, Ltd.
"^winbond,.*":
description: Winbond Electronics corp.
+ "^winlink,.*":
+ description: WinLink Co., Ltd
"^winstar,.*":
description: Winstar Display Corp.
"^wits,.*":
@@ -1328,6 +1340,8 @@
description: Wondermedia Technologies, Inc.
"^wobo,.*":
description: Wobo
+ "^wanchanglong,.*":
+ description: Wanchanglong Electronics Technology（SHENZHEN）Co.，Ltd.
"^x-powers,.*":
description: X-Powers
"^xes,.*":
@@ -1350,6 +1364,8 @@
description: Shenzhen Xunlong Software CO.,Limited
"^xylon,.*":
description: Xylon
+ "^yadro,.*":
+ description: YADRO
"^yamaha,.*":
description: Yamaha Corporation
"^yes-optoelectronics,.*":
diff --git a/Documentation/devicetree/bindings/watchdog/apple,wdt.yaml b/Documentation/devicetree/bindings/watchdog/apple,wdt.yaml
new file mode 100644
index 0000000..e58c56a
--- /dev/null
+++ b/Documentation/devicetree/bindings/watchdog/apple,wdt.yaml
@@ -0,0 +1,52 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/watchdog/apple,wdt.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Apple SoC Watchdog
+
+allOf:
+ - $ref: "watchdog.yaml#"
+
+maintainers:
+ - Sven Peter <sven@svenpeter.dev>
+
+properties:
+ compatible:
+ items:
+ - enum:
+ - apple,t8103-wdt
+ - apple,t6000-wdt
+ - const: apple,wdt
+
+ reg:
+ maxItems: 1
+
+ clocks:
+ maxItems: 1
+
+ interrupts:
+ maxItems: 1
+
+required:
+ - compatible
+ - reg
+ - clocks
+ - interrupts
+
+unevaluatedProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/interrupt-controller/apple-aic.h>
+ #include <dt-bindings/interrupt-controller/irq.h>
+
+ wdt: watchdog@50000000 {
+ compatible = "apple,t8103-wdt", "apple,wdt";
+ reg = <0x50000000 0x4000>;
+ clocks = <&clk>;
+ interrupts = <AIC_IRQ 123 IRQ_TYPE_LEVEL_HIGH>;
+ };
+
+...
diff --git a/Documentation/driver-api/media/drivers/index.rst b/Documentation/driver-api/media/drivers/index.rst
index 426cda6..3240649 100644
--- a/Documentation/driver-api/media/drivers/index.rst
+++ b/Documentation/driver-api/media/drivers/index.rst
@@ -21,6 +21,7 @@
pvrusb2
pxa_camera
radiotrack
+ rkisp1
saa7134-devel
sh_mobile_ceu_camera
tuners
diff --git a/Documentation/firmware-guide/acpi/apei/einj.rst b/Documentation/firmware-guide/acpi/apei/einj.rst
index c042176..55e2331 100644
--- a/Documentation/firmware-guide/acpi/apei/einj.rst
+++ b/Documentation/firmware-guide/acpi/apei/einj.rst
@@ -181,5 +181,24 @@
[22715.834759] EDAC sbridge MC3: PROCESSOR 0:306e7 TIME 1422553404 SOCKET 0 APIC 0
[22716.616173] EDAC MC3: 1 CE memory read error on CPU_SrcID#0_Channel#0_DIMM#0 (channel:0 slot:0 page:0x12345 offset:0x0 grain:32 syndrome:0x0 - area:DRAM err_code:0001:0090 socket:0 channel_mask:1 rank:0)
+Special notes for injection into SGX enclaves:
+
+There may be a separate BIOS setup option to enable SGX injection.
+
+The injection process consists of setting some special memory controller
+trigger that will inject the error on the next write to the target
+address. But the h/w prevents any software outside of an SGX enclave
+from accessing enclave pages (even BIOS SMM mode).
+
+The following sequence can be used:
+ 1) Determine physical address of enclave page
+ 2) Use "notrigger=1" mode to inject (this will setup
+ the injection address, but will not actually inject)
+ 3) Enter the enclave
+ 4) Store data to the virtual address matching physical address from step 1
+ 5) Execute CLFLUSH for that virtual address
+ 6) Spin delay for 250ms
+ 7) Read from the virtual address. This will trigger the error
+
For more information about EINJ, please refer to ACPI specification
version 4.0, section 17.5 and ACPI 5.0, section 18.6.
diff --git a/Documentation/gpu/amdgpu-dc.rst b/Documentation/gpu/amdgpu-dc.rst
deleted file mode 100644
index f7ff7e1..0000000
--- a/Documentation/gpu/amdgpu-dc.rst
+++ /dev/null
@@ -1,74 +0,0 @@
-===================================
-drm/amd/display - Display Core (DC)
-===================================
-
-*placeholder - general description of supported platforms, what dc is, etc.*
-
-Because it is partially shared with other operating systems, the Display Core
-Driver is divided in two pieces.
-
-1. **Display Core (DC)** contains the OS-agnostic components. Things like
- hardware programming and resource management are handled here.
-2. **Display Manager (DM)** contains the OS-dependent components. Hooks to the
- amdgpu base driver and DRM are implemented here.
-
-It doesn't help that the entire package is frequently referred to as DC. But
-with the context in mind, it should be clear.
-
-When CONFIG_DRM_AMD_DC is enabled, DC will be initialized by default for
-supported ASICs. To force disable, set `amdgpu.dc=0` on kernel command line.
-Likewise, to force enable on unsupported ASICs, set `amdgpu.dc=1`.
-
-To determine if DC is loaded, search dmesg for the following entry:
-
-``Display Core initialized with <version number here>``
-
-AMDgpu Display Manager
-======================
-
-.. kernel-doc:: drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
- :doc: overview
-
-.. kernel-doc:: drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h
- :internal:
-
-Lifecycle
----------
-
-.. kernel-doc:: drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
- :doc: DM Lifecycle
-
-.. kernel-doc:: drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
- :functions: dm_hw_init dm_hw_fini
-
-Interrupts
-----------
-
-.. kernel-doc:: drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq.c
- :doc: overview
-
-.. kernel-doc:: drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq.c
- :internal:
-
-.. kernel-doc:: drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
- :functions: register_hpd_handlers dm_crtc_high_irq dm_pflip_high_irq
-
-Atomic Implementation
----------------------
-
-.. kernel-doc:: drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
- :doc: atomic
-
-.. kernel-doc:: drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
- :functions: amdgpu_dm_atomic_check amdgpu_dm_atomic_commit_tail
-
-Display Core
-============
-
-**WIP**
-
-FreeSync Video
---------------
-
-.. kernel-doc:: drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
- :doc: FreeSync Video
diff --git a/Documentation/gpu/amdgpu.rst b/Documentation/gpu/amdgpu.rst
deleted file mode 100644
index 8ba72e8..0000000
--- a/Documentation/gpu/amdgpu.rst
+++ /dev/null
@@ -1,324 +0,0 @@
-=========================
- drm/amdgpu AMDgpu driver
-=========================
-
-The drm/amdgpu driver supports all AMD Radeon GPUs based on the Graphics Core
-Next (GCN) architecture.
-
-Module Parameters
-=================
-
-The amdgpu driver supports the following module parameters:
-
-.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
-
-Core Driver Infrastructure
-==========================
-
-This section covers core driver infrastructure.
-
-.. _amdgpu_memory_domains:
-
-Memory Domains
---------------
-
-.. kernel-doc:: include/uapi/drm/amdgpu_drm.h
- :doc: memory domains
-
-Buffer Objects
---------------
-
-.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
- :doc: amdgpu_object
-
-.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
- :internal:
-
-PRIME Buffer Sharing
---------------------
-
-.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
- :doc: PRIME Buffer Sharing
-
-.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
- :internal:
-
-MMU Notifier
-------------
-
-.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
- :doc: MMU Notifier
-
-.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
- :internal:
-
-AMDGPU Virtual Memory
----------------------
-
-.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
- :doc: GPUVM
-
-.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
- :internal:
-
-Interrupt Handling
-------------------
-
-.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
- :doc: Interrupt Handling
-
-.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
- :internal:
-
-IP Blocks
-------------------
-
-.. kernel-doc:: drivers/gpu/drm/amd/include/amd_shared.h
- :doc: IP Blocks
-
-.. kernel-doc:: drivers/gpu/drm/amd/include/amd_shared.h
- :identifiers: amd_ip_block_type amd_ip_funcs
-
-AMDGPU XGMI Support
-===================
-
-.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_xgmi.c
-
-AMDGPU RAS Support
-==================
-
-The AMDGPU RAS interfaces are exposed via sysfs (for informational queries) and
-debugfs (for error injection).
-
-RAS debugfs/sysfs Control and Error Injection Interfaces
---------------------------------------------------------
-
-.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
- :doc: AMDGPU RAS debugfs control interface
-
-RAS Reboot Behavior for Unrecoverable Errors
---------------------------------------------------------
-
-.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
- :doc: AMDGPU RAS Reboot Behavior for Unrecoverable Errors
-
-RAS Error Count sysfs Interface
--------------------------------
-
-.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
- :doc: AMDGPU RAS sysfs Error Count Interface
-
-RAS EEPROM debugfs Interface
-----------------------------
-
-.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
- :doc: AMDGPU RAS debugfs EEPROM table reset interface
-
-RAS VRAM Bad Pages sysfs Interface
-----------------------------------
-
-.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
- :doc: AMDGPU RAS sysfs gpu_vram_bad_pages Interface
-
-Sample Code
------------
-Sample code for testing error injection can be found here:
-https://cgit.freedesktop.org/mesa/drm/tree/tests/amdgpu/ras_tests.c
-
-This is part of the libdrm amdgpu unit tests which cover several areas of the GPU.
-There are four sets of tests:
-
-RAS Basic Test
-
-The test verifies the RAS feature enabled status and makes sure the necessary sysfs and debugfs files
-are present.
-
-RAS Query Test
-
-This test checks the RAS availability and enablement status for each supported IP block as well as
-the error counts.
-
-RAS Inject Test
-
-This test injects errors for each IP.
-
-RAS Disable Test
-
-This test tests disabling of RAS features for each IP block.
-
-
-GPU Power/Thermal Controls and Monitoring
-=========================================
-
-This section covers hwmon and power/thermal controls.
-
-HWMON Interfaces
-----------------
-
-.. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c
- :doc: hwmon
-
-GPU sysfs Power State Interfaces
---------------------------------
-
-GPU power controls are exposed via sysfs files.
-
-power_dpm_state
-~~~~~~~~~~~~~~~
-
-.. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c
- :doc: power_dpm_state
-
-power_dpm_force_performance_level
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-.. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c
- :doc: power_dpm_force_performance_level
-
-pp_table
-~~~~~~~~
-
-.. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c
- :doc: pp_table
-
-pp_od_clk_voltage
-~~~~~~~~~~~~~~~~~
-
-.. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c
- :doc: pp_od_clk_voltage
-
-pp_dpm_*
-~~~~~~~~
-
-.. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c
- :doc: pp_dpm_sclk pp_dpm_mclk pp_dpm_socclk pp_dpm_fclk pp_dpm_dcefclk pp_dpm_pcie
-
-pp_power_profile_mode
-~~~~~~~~~~~~~~~~~~~~~
-
-.. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c
- :doc: pp_power_profile_mode
-
-\*_busy_percent
-~~~~~~~~~~~~~~~
-
-.. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c
- :doc: gpu_busy_percent
-
-.. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c
- :doc: mem_busy_percent
-
-gpu_metrics
-~~~~~~~~~~~~~~~~~~~~~
-
-.. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c
- :doc: gpu_metrics
-
-GPU Product Information
-=======================
-
-Information about the GPU can be obtained on certain cards
-via sysfs
-
-product_name
-------------
-
-.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
- :doc: product_name
-
-product_number
---------------
-
-.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
- :doc: product_name
-
-serial_number
--------------
-
-.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
- :doc: serial_number
-
-unique_id
----------
-
-.. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c
- :doc: unique_id
-
-GPU Memory Usage Information
-============================
-
-Various memory accounting can be accessed via sysfs
-
-mem_info_vram_total
--------------------
-
-.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c
- :doc: mem_info_vram_total
-
-mem_info_vram_used
-------------------
-
-.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c
- :doc: mem_info_vram_used
-
-mem_info_vis_vram_total
------------------------
-
-.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c
- :doc: mem_info_vis_vram_total
-
-mem_info_vis_vram_used
-----------------------
-
-.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c
- :doc: mem_info_vis_vram_used
-
-mem_info_gtt_total
-------------------
-
-.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c
- :doc: mem_info_gtt_total
-
-mem_info_gtt_used
------------------
-
-.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c
- :doc: mem_info_gtt_used
-
-PCIe Accounting Information
-===========================
-
-pcie_bw
--------
-
-.. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c
- :doc: pcie_bw
-
-pcie_replay_count
------------------
-
-.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
- :doc: pcie_replay_count
-
-GPU SmartShift Information
-==========================
-
-GPU SmartShift information via sysfs
-
-smartshift_apu_power
---------------------
-
-.. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c
- :doc: smartshift_apu_power
-
-smartshift_dgpu_power
----------------------
-
-.. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c
- :doc: smartshift_dgpu_power
-
-smartshift_bias
----------------
-
-.. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c
- :doc: smartshift_bias
diff --git a/Documentation/gpu/amdgpu/amdgpu-glossary.rst b/Documentation/gpu/amdgpu/amdgpu-glossary.rst
new file mode 100644
index 0000000..859dcec
--- /dev/null
+++ b/Documentation/gpu/amdgpu/amdgpu-glossary.rst
@@ -0,0 +1,87 @@
+===============
+AMDGPU Glossary
+===============
+
+Here you can find some generic acronyms used in the amdgpu driver. Notice that
+we have a dedicated glossary for Display Core at
+'Documentation/gpu/amdgpu/display/dc-glossary.rst'.
+
+.. glossary::
+
+ CP
+ Command Processor
+
+ CPLIB
+ Content Protection Library
+
+ DFS
+ Digital Frequency Synthesizer
+
+ ECP
+ Enhanced Content Protection
+
+ EOP
+ End Of Pipe/Pipeline
+
+ GC
+ Graphics and Compute
+
+ GMC
+ Graphic Memory Controller
+
+ IH
+ Interrupt Handler
+
+ HQD
+ Hardware Queue Descriptor
+
+ IB
+ Indirect Buffer
+
+ IP
+ Intellectual Property blocks
+
+ KCQ
+ Kernel Compute Queue
+
+ KGQ
+ Kernel Graphics Queue
+
+ KIQ
+ Kernel Interface Queue
+
+ MEC
+ MicroEngine Compute
+
+ MES
+ MicroEngine Scheduler
+
+ MMHUB
+ Multi-Media HUB
+
+ MQD
+ Memory Queue Descriptor
+
+ PPLib
+ PowerPlay Library - PowerPlay is the power management component.
+
+ PSP
+ Platform Security Processor
+
+ RCL
+ RunList Controller
+
+ SDMA
+ System DMA
+
+ SMU
+ System Management Unit
+
+ SS
+ Spread Spectrum
+
+ VCE
+ Video Compression Engine
+
+ VCN
+ Video Codec Next
diff --git a/Documentation/gpu/amdgpu/display/config_example.svg b/Documentation/gpu/amdgpu/display/config_example.svg
new file mode 100644
index 0000000..cdac985
--- /dev/null
+++ b/Documentation/gpu/amdgpu/display/config_example.svg
@@ -0,0 +1,414 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ xmlns:dc="http://purl.org/dc/elements/1.1/"
+ xmlns:cc="http://creativecommons.org/ns#"
+ xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+ xmlns:svg="http://www.w3.org/2000/svg"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ width="144.63406mm"
+ height="66.596054mm"
+ viewBox="0 0 144.15195 66.596054"
+ version="1.1"
+ id="svg8"
+ inkscape:version="0.92.5 (2060ec1f9f, 2020-04-08)"
+ sodipodi:docname="config_example.svg">
+ <defs
+ id="defs2">
+ <marker
+ inkscape:stockid="Arrow1Mend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Mend"
+ style="overflow:visible"
+ inkscape:isstock="true">
+ <path
+ id="path4547"
+ d="M 0,0 5,-5 -12.5,0 5,5 Z"
+ style="fill:#ff0000;fill-opacity:1;fill-rule:evenodd;stroke:#ff0000;stroke-width:1.00000003pt;stroke-opacity:1"
+ transform="matrix(-0.4,0,0,-0.4,-4,0)"
+ inkscape:connector-curvature="0" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Mend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Mend-3"
+ style="overflow:visible"
+ inkscape:isstock="true">
+ <path
+ inkscape:connector-curvature="0"
+ id="path4547-6"
+ d="M 0,0 5,-5 -12.5,0 5,5 Z"
+ style="fill:#ff0000;fill-opacity:1;fill-rule:evenodd;stroke:#ff0000;stroke-width:1.00000003pt;stroke-opacity:1"
+ transform="matrix(-0.4,0,0,-0.4,-4,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Mend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Mend-3-5"
+ style="overflow:visible"
+ inkscape:isstock="true">
+ <path
+ inkscape:connector-curvature="0"
+ id="path4547-6-3"
+ d="M 0,0 5,-5 -12.5,0 5,5 Z"
+ style="fill:#ff0000;fill-opacity:1;fill-rule:evenodd;stroke:#ff0000;stroke-width:1.00000003pt;stroke-opacity:1"
+ transform="matrix(-0.4,0,0,-0.4,-4,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Mend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Mend-3-5-0"
+ style="overflow:visible"
+ inkscape:isstock="true">
+ <path
+ inkscape:connector-curvature="0"
+ id="path4547-6-3-6"
+ d="M 0,0 5,-5 -12.5,0 5,5 Z"
+ style="fill:#ff0000;fill-opacity:1;fill-rule:evenodd;stroke:#ff0000;stroke-width:1.00000003pt;stroke-opacity:1"
+ transform="matrix(-0.4,0,0,-0.4,-4,0)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Mend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Mend-3-5-7"
+ style="overflow:visible"
+ inkscape:isstock="true">
+ <path
+ inkscape:connector-curvature="0"
+ id="path4547-6-3-3"
+ d="M 0,0 5,-5 -12.5,0 5,5 Z"
+ style="fill:#ff0000;fill-opacity:1;fill-rule:evenodd;stroke:#ff0000;stroke-width:1.00000003pt;stroke-opacity:1"
+ transform="matrix(-0.4,0,0,-0.4,-4,0)" />
+ </marker>
+ </defs>
+ <sodipodi:namedview
+ id="base"
+ pagecolor="#ffffff"
+ bordercolor="#666666"
+ borderopacity="1.0"
+ inkscape:pageopacity="0.0"
+ inkscape:pageshadow="2"
+ inkscape:zoom="0.98994949"
+ inkscape:cx="518.91791"
+ inkscape:cy="172.50112"
+ inkscape:document-units="mm"
+ inkscape:current-layer="layer1"
+ showgrid="true"
+ viewbox-width="209.3"
+ inkscape:window-width="3840"
+ inkscape:window-height="1136"
+ inkscape:window-x="0"
+ inkscape:window-y="27"
+ inkscape:window-maximized="1"
+ fit-margin-top="0"
+ fit-margin-left="0"
+ fit-margin-right="0"
+ fit-margin-bottom="0">
+ <inkscape:grid
+ type="xygrid"
+ id="grid817"
+ originx="4.390216"
+ originy="-208.88856" />
+ </sodipodi:namedview>
+ <metadata
+ id="metadata5">
+ <rdf:RDF>
+ <cc:Work
+ rdf:about="">
+ <dc:format>image/svg+xml</dc:format>
+ <dc:type
+ rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
+ <dc:title></dc:title>
+ </cc:Work>
+ </rdf:RDF>
+ </metadata>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1"
+ transform="translate(4.4048992,-21.515392)">
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.26458335px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
+ d="m 20.816662,35.062492 h 23.8125 v -5.291667 h 5.291667 v 5.291667 h 10.583334 v -5.291667 h 5.291667 v 5.291667 h 2.645833 v -5.291667 h 5.291667 v 5.291667 h 66.14583"
+ id="path4522"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.26458335px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
+ d="m 20.816662,48.291659 h 7.9375 v -5.291667 h 5.291667 v 5.291667 h 58.208335 v -5.291667 h 5.291666 v 5.291667 h 42.33333"
+ id="path4524"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.26458335px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
+ d="m 20.816662,61.520826 h 26.458334 v -5.291667 h 44.979168 v 5.291667 h 47.624996"
+ id="path4526"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.26458335px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
+ d="M 20.816662,72.104159 H 139.87916"
+ id="path4528"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.26458335px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
+ d="M 20.816662,77.395826 H 139.87916"
+ id="path4530"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.26458335px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
+ d="M 20.816662,82.687493 H 139.87916"
+ id="path4532"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.26458335px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
+ d="M 20.816662,87.97916 H 139.87916"
+ id="path4534"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#ff0000;stroke-width:0.5291667;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.52916668, 0.52916668;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#Arrow1Mend)"
+ d="m 47.274996,29.770826 c 3.836215,14.933158 3.472151,27.586643 0.264583,41.010418"
+ id="path4536"
+ inkscape:connector-curvature="0"
+ sodipodi:nodetypes="cc" />
+ <path
+ style="fill:none;stroke:#ff0000;stroke-width:0.5291667;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.52916669, 0.52916669;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#Arrow1Mend-3)"
+ d="m 63.149996,29.770826 c 3.836214,14.933158 5.059652,27.586642 1.852084,41.010418"
+ id="path4536-7"
+ inkscape:connector-curvature="0"
+ sodipodi:nodetypes="cc" />
+ <path
+ style="fill:none;stroke:#ff0000;stroke-width:0.5291667;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.5291667, 0.5291667;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#Arrow1Mend-3-5)"
+ d="m 71.087496,29.770825 c 3.836214,14.933158 5.059652,27.586643 1.852084,41.010419"
+ id="path4536-7-5"
+ inkscape:connector-curvature="0"
+ sodipodi:nodetypes="cc" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:10.58333397px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458335"
+ x="59.359009"
+ y="24.195677"
+ id="text6572"><tspan
+ sodipodi:role="line"
+ x="59.359009"
+ y="24.195677"
+ style="font-size:3.52777791px;line-height:5.39999962;text-align:center;text-anchor:middle;stroke-width:0.26458335"
+ id="tspan6574">Configurations</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:10.58333397px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458335"
+ x="46.825508"
+ y="28.542402"
+ id="text6572-6"><tspan
+ sodipodi:role="line"
+ x="46.825508"
+ y="28.542402"
+ style="font-size:3.52777815px;line-height:5.39999962;text-align:center;text-anchor:middle;fill:#008000;stroke-width:0.26458335"
+ id="tspan6574-2">A</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:10.58333397px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458335"
+ x="62.8895"
+ y="28.825886"
+ id="text6572-6-2"><tspan
+ sodipodi:role="line"
+ x="62.8895"
+ y="28.825886"
+ style="font-size:3.52777839px;line-height:5.39999962;text-align:center;text-anchor:middle;fill:#0000ff;stroke-width:0.26458335"
+ id="tspan6574-2-7">B</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:10.58333397px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458335"
+ x="70.827003"
+ y="29.109362"
+ id="text6572-6-2-3"><tspan
+ sodipodi:role="line"
+ x="70.827003"
+ y="29.109362"
+ style="font-size:3.52777863px;line-height:5.39999962;text-align:center;text-anchor:middle;fill:#c87137;stroke-width:0.26458335"
+ id="tspan6574-2-7-6">C</tspan></text>
+ <path
+ style="fill:none;stroke:#ff0000;stroke-width:0.5291667;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.52916671, 0.52916671;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#Arrow1Mend-3-5-0)"
+ d="m 92.254164,42.999993 c 9.142136,12.745655 4.411987,28.608461 0.529167,38.364584"
+ id="path4536-7-5-2"
+ inkscape:connector-curvature="0"
+ sodipodi:nodetypes="cc" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.5291667;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
+ d="m 47.274996,72.104159 v 5.291667"
+ id="path8053"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.5291667;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
+ d="m 64.472913,72.10416 v 5.291667"
+ id="path8053-6"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.5291667;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
+ d="m 72.410413,72.10416 v 5.291667"
+ id="path8053-6-1"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.5291667;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
+ d="m 92.254164,82.687494 v 5.291667"
+ id="path8053-6-1-8"
+ inkscape:connector-curvature="0" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:10.58333397px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458335"
+ x="55.802444"
+ y="76.167412"
+ id="text6572-6-7"><tspan
+ sodipodi:role="line"
+ x="55.802444"
+ y="76.167412"
+ style="font-size:3.52777839px;line-height:5.39999962;text-align:center;text-anchor:middle;fill:#008000;stroke-width:0.26458335"
+ id="tspan6574-2-9">A</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:10.58333397px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458335"
+ x="68.559143"
+ y="75.883926"
+ id="text6572-6-2-2"><tspan
+ sodipodi:role="line"
+ x="68.559143"
+ y="75.883926"
+ style="font-size:3.52777863px;line-height:5.39999962;text-align:center;text-anchor:middle;fill:#0000ff;stroke-width:0.26458335"
+ id="tspan6574-2-7-0">B</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:10.58333397px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458335"
+ x="84.812119"
+ y="75.883911"
+ id="text6572-6-2-3-2"><tspan
+ sodipodi:role="line"
+ x="84.812119"
+ y="75.883911"
+ style="font-size:3.52777863px;line-height:5.39999962;text-align:center;text-anchor:middle;fill:#c87137;stroke-width:0.26458335"
+ id="tspan6574-2-7-6-3">C</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:10.58333397px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458335"
+ x="98.513756"
+ y="86.845222"
+ id="text6572-6-2-3-2-7"><tspan
+ sodipodi:role="line"
+ x="98.513756"
+ y="86.845222"
+ style="font-size:3.52777863px;line-height:5.39999962;text-align:center;text-anchor:middle;fill:#c87137;stroke-width:0.26458335"
+ id="tspan6574-2-7-6-3-5">C</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:10.58333397px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458335"
+ x="35.452015"
+ y="75.694931"
+ id="text6572-9"><tspan
+ sodipodi:role="line"
+ x="35.452015"
+ y="75.694931"
+ style="font-size:3.52777815px;line-height:5.39999962;text-align:center;text-anchor:middle;stroke-width:0.26458335"
+ id="tspan6574-22">Old config</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:10.58333397px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458335"
+ x="55.484753"
+ y="86.656235"
+ id="text6572-9-8"><tspan
+ sodipodi:role="line"
+ x="55.484753"
+ y="86.656235"
+ style="font-size:3.52777839px;line-height:5.39999962;text-align:center;text-anchor:middle;stroke-width:0.26458335"
+ id="tspan6574-22-9">Old config</tspan></text>
+ <path
+ style="fill:none;stroke:#ff0000;stroke-width:0.5291667;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.52916671, 0.52916671;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#Arrow1Mend-3-5-7)"
+ d="m 92.254164,42.999993 c 4.233333,4.7625 2.645833,13.229167 0.79375,17.197917"
+ inkscape:connector-curvature="0"
+ sodipodi:nodetypes="cc" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:10.58333397px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458335"
+ x="3.7020128"
+ y="33.550579"
+ id="text6572-1"><tspan
+ sodipodi:role="line"
+ x="3.7020128"
+ y="42.914349"
+ style="font-size:3.52777815px;line-height:5.39999962;text-align:center;text-anchor:middle;stroke-width:0.26458335"
+ id="tspan15310" /></text>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:3.17500019px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458335"
+ x="13.366468"
+ y="46.590767"
+ id="text15316"><tspan
+ sodipodi:role="line"
+ x="13.366468"
+ y="46.590767"
+ style="text-align:center;text-anchor:middle;stroke-width:0.26458335"
+ id="tspan15318">VUpdate</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:3.17500043px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458335"
+ x="14.45245"
+ y="29.676321"
+ id="text15316-3"><tspan
+ sodipodi:role="line"
+ id="tspan15314-1"
+ x="14.45245"
+ y="29.676321"
+ style="text-align:center;text-anchor:middle;stroke-width:0.26458335">Update</tspan><tspan
+ sodipodi:role="line"
+ x="14.45245"
+ y="33.645073"
+ style="text-align:center;text-anchor:middle;stroke-width:0.26458335"
+ id="tspan15318-9">Lock</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:3.17500043px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458335"
+ x="7.5676007"
+ y="56.985115"
+ id="text15316-4"><tspan
+ sodipodi:role="line"
+ x="7.5676007"
+ y="56.985115"
+ style="text-align:center;text-anchor:middle;stroke-width:0.26458335"
+ id="tspan15318-7">Register update</tspan><tspan
+ sodipodi:role="line"
+ x="7.5676007"
+ y="60.953865"
+ style="text-align:center;text-anchor:middle;stroke-width:0.26458335"
+ id="tspan15361">Pending Status</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:3.17500043px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458335"
+ x="16.074829"
+ y="76.167404"
+ id="text15316-8"><tspan
+ sodipodi:role="line"
+ x="16.074829"
+ y="76.167404"
+ style="text-align:center;text-anchor:middle;stroke-width:0.26458335"
+ id="tspan15318-4">Buf 0</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:3.17500067px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458335"
+ x="16.156994"
+ y="86.089279"
+ id="text15316-8-5"><tspan
+ sodipodi:role="line"
+ x="16.156994"
+ y="86.089279"
+ style="text-align:center;text-anchor:middle;stroke-width:0.26458335"
+ id="tspan15318-4-0">Buf 1</tspan></text>
+ </g>
+</svg>
diff --git a/Documentation/gpu/amdgpu/display/dc-debug.rst b/Documentation/gpu/amdgpu/display/dc-debug.rst
new file mode 100644
index 0000000..40c55a6
--- /dev/null
+++ b/Documentation/gpu/amdgpu/display/dc-debug.rst
@@ -0,0 +1,77 @@
+========================
+Display Core Debug tools
+========================
+
+DC Visual Confirmation
+======================
+
+Display core provides a feature named visual confirmation, which is a set of
+bars added at the scanout time by the driver to convey some specific
+information. In general, you can enable this debug option by using::
+
+ echo <N> > /sys/kernel/debug/dri/0/amdgpu_dm_visual_confirm
+
+Where `N` is an integer number for some specific scenarios that the developer
+wants to enable, you will see some of these debug cases in the following
+subsection.
+
+Multiple Planes Debug
+---------------------
+
+If you want to enable or debug multiple planes in a specific user-space
+application, you can leverage a debug feature named visual confirm. For
+enabling it, you will need::
+
+ echo 1 > /sys/kernel/debug/dri/0/amdgpu_dm_visual_confirm
+
+You need to reload your GUI to see the visual confirmation. When the plane
+configuration changes or a full update occurs there will be a colored bar at
+the bottom of each hardware plane being drawn on the screen.
+
+* The color indicates the format - For example, red is AR24 and green is NV12
+* The height of the bar indicates the index of the plane
+* Pipe split can be observed if there are two bars with a difference in height
+ covering the same plane
+
+Consider the video playback case in which a video is played in a specific
+plane, and the desktop is drawn in another plane. The video plane should
+feature one or two green bars at the bottom of the video depending on pipe
+split configuration.
+
+* There should **not** be any visual corruption
+* There should **not** be any underflow or screen flashes
+* There should **not** be any black screens
+* There should **not** be any cursor corruption
+* Multiple plane **may** be briefly disabled during window transitions or
+ resizing but should come back after the action has finished
+
+Pipe Split Debug
+----------------
+
+Sometimes we need to debug if DCN is splitting pipes correctly, and visual
+confirmation is also handy for this case. Similar to the MPO case, you can use
+the below command to enable visual confirmation::
+
+ echo 1 > /sys/kernel/debug/dri/0/amdgpu_dm_visual_confirm
+
+In this case, if you have a pipe split, you will see one small red bar at the
+bottom of the display covering the entire display width and another bar
+covering the second pipe. In other words, you will see a bit high bar in the
+second pipe.
+
+DTN Debug
+=========
+
+DC (DCN) provides an extensive log that dumps multiple details from our
+hardware configuration. Via debugfs, you can capture those status values by
+using Display Test Next (DTN) log, which can be captured via debugfs by using::
+
+ cat /sys/kernel/debug/dri/0/amdgpu_dm_dtn_log
+
+Since this log is updated accordingly with DCN status, you can also follow the
+change in real-time by using something like::
+
+ sudo watch -d cat /sys/kernel/debug/dri/0/amdgpu_dm_dtn_log
+
+When reporting a bug related to DC, consider attaching this log before and
+after you reproduce the bug.
diff --git a/Documentation/gpu/amdgpu/display/dc-glossary.rst b/Documentation/gpu/amdgpu/display/dc-glossary.rst
new file mode 100644
index 0000000..116f5f0
--- /dev/null
+++ b/Documentation/gpu/amdgpu/display/dc-glossary.rst
@@ -0,0 +1,237 @@
+===========
+DC Glossary
+===========
+
+On this page, we try to keep track of acronyms related to the display
+component. If you do not find what you are looking for, look at the
+'Documentation/gpu/amdgpu/amdgpu-glossary.rst'; if you cannot find it anywhere,
+consider asking in the amdgfx and update this page.
+
+.. glossary::
+
+ ABM
+ Adaptive Backlight Modulation
+
+ APU
+ Accelerated Processing Unit
+
+ ASIC
+ Application-Specific Integrated Circuit
+
+ ASSR
+ Alternate Scrambler Seed Reset
+
+ AZ
+ Azalia (HD audio DMA engine)
+
+ BPC
+ Bits Per Colour/Component
+
+ BPP
+ Bits Per Pixel
+
+ Clocks
+ * PCLK: Pixel Clock
+ * SYMCLK: Symbol Clock
+ * SOCCLK: GPU Engine Clock
+ * DISPCLK: Display Clock
+ * DPPCLK: DPP Clock
+ * DCFCLK: Display Controller Fabric Clock
+ * REFCLK: Real Time Reference Clock
+ * PPLL: Pixel PLL
+ * FCLK: Fabric Clock
+ * MCLK: Memory Clock
+
+ CRC
+ Cyclic Redundancy Check
+
+ CRTC
+ Cathode Ray Tube Controller - commonly called "Controller" - Generates
+ raw stream of pixels, clocked at pixel clock
+
+ CVT
+ Coordinated Video Timings
+
+ DAL
+ Display Abstraction layer
+
+ DC (Software)
+ Display Core
+
+ DC (Hardware)
+ Display Controller
+
+ DCC
+ Delta Colour Compression
+
+ DCE
+ Display Controller Engine
+
+ DCHUB
+ Display Controller HUB
+
+ ARB
+ Arbiter
+
+ VTG
+ Vertical Timing Generator
+
+ DCN
+ Display Core Next
+
+ DCCG
+ Display Clock Generator block
+
+ DDC
+ Display Data Channel
+
+ DIO
+ Display IO
+
+ DPP
+ Display Pipes and Planes
+
+ DSC
+ Display Stream Compression (Reduce the amount of bits to represent pixel
+ count while at the same pixel clock)
+
+ dGPU
+ discrete GPU
+
+ DMIF
+ Display Memory Interface
+
+ DML
+ Display Mode Library
+
+ DMCU
+ Display Micro-Controller Unit
+
+ DMCUB
+ Display Micro-Controller Unit, version B
+
+ DPCD
+ DisplayPort Configuration Data
+
+ DPM(S)
+ Display Power Management (Signaling)
+
+ DRR
+ Dynamic Refresh Rate
+
+ DWB
+ Display Writeback
+
+ FB
+ Frame Buffer
+
+ FBC
+ Frame Buffer Compression
+
+ FEC
+ Forward Error Correction
+
+ FRL
+ Fixed Rate Link
+
+ GCO
+ Graphical Controller Object
+
+ GSL
+ Global Swap Lock
+
+ iGPU
+ integrated GPU
+
+ ISR
+ Interrupt Service Request
+
+ ISV
+ Independent Software Vendor
+
+ KMD
+ Kernel Mode Driver
+
+ LB
+ Line Buffer
+
+ LFC
+ Low Framerate Compensation
+
+ LTTPR
+ Link Training Tunable Phy Repeater
+
+ LUT
+ Lookup Table
+
+ MALL
+ Memory Access at Last Level
+
+ MC
+ Memory Controller
+
+ MPC
+ Multiple pipes and plane combine
+
+ MPO
+ Multi Plane Overlay
+
+ MST
+ Multi Stream Transport
+
+ NBP State
+ Northbridge Power State
+
+ NBIO
+ North Bridge Input/Output
+
+ ODM
+ Output Data Mapping
+
+ OPM
+ Output Protection Manager
+
+ OPP
+ Output Plane Processor
+
+ OPTC
+ Output Pipe Timing Combiner
+
+ OTG
+ Output Timing Generator
+
+ PCON
+ Power Controller
+
+ PGFSM
+ Power Gate Finite State Machine
+
+ PSR
+ Panel Self Refresh
+
+ SCL
+ Scaler
+
+ SDP
+ Scalable Data Port
+
+ SLS
+ Single Large Surface
+
+ SST
+ Single Stream Transport
+
+ TMDS
+ Transition-Minimized Differential Signaling
+
+ TMZ
+ Trusted Memory Zone
+
+ TTU
+ Time to Underflow
+
+ VRR
+ Variable Refresh Rate
+
+ UVD
+ Unified Video Decoder
diff --git a/Documentation/gpu/amdgpu/display/dc_pipeline_overview.svg b/Documentation/gpu/amdgpu/display/dc_pipeline_overview.svg
new file mode 100644
index 0000000..9adeceb
--- /dev/null
+++ b/Documentation/gpu/amdgpu/display/dc_pipeline_overview.svg
@@ -0,0 +1,1125 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ xmlns:dc="http://purl.org/dc/elements/1.1/"
+ xmlns:cc="http://creativecommons.org/ns#"
+ xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+ xmlns:svg="http://www.w3.org/2000/svg"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ width="1296.7491"
+ height="741.97845"
+ viewBox="0 0 343.0982 196.31514"
+ version="1.1"
+ id="svg8"
+ inkscape:version="0.92.5 (2060ec1f9f, 2020-04-08)"
+ sodipodi:docname="dc_pipeline_overview.svg">
+ <defs
+ id="defs2">
+ <marker
+ inkscape:stockid="Arrow2Mend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="marker8858"
+ style="overflow:visible"
+ inkscape:isstock="true">
+ <path
+ id="path8616"
+ style="fill:#aa00d4;fill-opacity:1;fill-rule:evenodd;stroke:#aa00d4;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1"
+ d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z"
+ transform="scale(-0.6)"
+ inkscape:connector-curvature="0" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow2Send"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow2Send"
+ style="overflow:visible"
+ inkscape:isstock="true">
+ <path
+ id="path8622"
+ style="fill:#ff0000;fill-opacity:1;fill-rule:evenodd;stroke:#ff0000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1"
+ d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z"
+ transform="matrix(-0.3,0,0,-0.3,0.69,0)"
+ inkscape:connector-curvature="0" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend"
+ style="overflow:visible"
+ inkscape:isstock="true">
+ <path
+ id="path8592"
+ d="M 0,0 5,-5 -12.5,0 5,5 Z"
+ style="fill:#ff0000;fill-opacity:1;fill-rule:evenodd;stroke:#ff0000;stroke-width:1.00000003pt;stroke-opacity:1"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)"
+ inkscape:connector-curvature="0" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow2Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow2Lend"
+ style="overflow:visible"
+ inkscape:isstock="true">
+ <path
+ id="path8610"
+ style="fill:#ff0000;fill-opacity:1;fill-rule:evenodd;stroke:#ff0000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1"
+ d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z"
+ transform="matrix(-1.1,0,0,-1.1,-1.1,0)"
+ inkscape:connector-curvature="0" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow2Mend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow2Mend"
+ style="overflow:visible"
+ inkscape:isstock="true">
+ <path
+ id="path1200"
+ style="fill:#ff0000;fill-opacity:1;fill-rule:evenodd;stroke:#ff0000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1"
+ d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z"
+ transform="scale(-0.6)"
+ inkscape:connector-curvature="0" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow2Mend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow2Mend-8"
+ style="overflow:visible"
+ inkscape:isstock="true">
+ <path
+ inkscape:connector-curvature="0"
+ id="path1200-9"
+ style="fill:#ff0000;fill-opacity:1;fill-rule:evenodd;stroke:#ff0000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1"
+ d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z"
+ transform="scale(-0.6)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow2Mend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow2Mend-8-3"
+ style="overflow:visible"
+ inkscape:isstock="true">
+ <path
+ inkscape:connector-curvature="0"
+ id="path1200-9-6"
+ style="fill:#ff0000;fill-opacity:1;fill-rule:evenodd;stroke:#ff0000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1"
+ d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z"
+ transform="scale(-0.6)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow2Mend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow2Mend-8-3-2"
+ style="overflow:visible"
+ inkscape:isstock="true">
+ <path
+ inkscape:connector-curvature="0"
+ id="path1200-9-6-9"
+ style="fill:#ff0000;fill-opacity:1;fill-rule:evenodd;stroke:#ff0000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1"
+ d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z"
+ transform="scale(-0.6)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow2Mend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow2Mend-8-3-2-1"
+ style="overflow:visible"
+ inkscape:isstock="true">
+ <path
+ inkscape:connector-curvature="0"
+ id="path1200-9-6-9-9"
+ style="fill:#ff0000;fill-opacity:1;fill-rule:evenodd;stroke:#ff0000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1"
+ d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z"
+ transform="scale(-0.6)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow2Mend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow2Mend-8-3-2-7"
+ style="overflow:visible"
+ inkscape:isstock="true">
+ <path
+ inkscape:connector-curvature="0"
+ id="path1200-9-6-9-8"
+ style="fill:#008000;fill-opacity:1;fill-rule:evenodd;stroke:#008000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1"
+ d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z"
+ transform="scale(-0.6)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow2Mend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow2Mend-8-3-4"
+ style="overflow:visible"
+ inkscape:isstock="true">
+ <path
+ inkscape:connector-curvature="0"
+ id="path1200-9-6-5"
+ style="fill:#008000;fill-opacity:1;fill-rule:evenodd;stroke:#008000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1"
+ d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z"
+ transform="scale(-0.6)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow2Mend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow2Mend-8-0"
+ style="overflow:visible"
+ inkscape:isstock="true">
+ <path
+ inkscape:connector-curvature="0"
+ id="path1200-9-3"
+ style="fill:#008000;fill-opacity:1;fill-rule:evenodd;stroke:#008000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1"
+ d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z"
+ transform="scale(-0.6)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow2Mend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow2Mend-6"
+ style="overflow:visible"
+ inkscape:isstock="true">
+ <path
+ inkscape:connector-curvature="0"
+ id="path1200-1"
+ style="fill:#008000;fill-opacity:1;fill-rule:evenodd;stroke:#008000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1"
+ d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z"
+ transform="scale(-0.6)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow2Mend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow2Mend-8-3-2-6"
+ style="overflow:visible"
+ inkscape:isstock="true">
+ <path
+ inkscape:connector-curvature="0"
+ id="path1200-9-6-9-1"
+ style="fill:#ff0000;fill-opacity:1;fill-rule:evenodd;stroke:#ff0000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1"
+ d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z"
+ transform="scale(-0.6)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow2Mend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow2Mend-8-0-7"
+ style="overflow:visible"
+ inkscape:isstock="true">
+ <path
+ inkscape:connector-curvature="0"
+ id="path1200-9-3-4"
+ style="fill:#008000;fill-opacity:1;fill-rule:evenodd;stroke:#008000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1"
+ d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z"
+ transform="scale(-0.6)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow2Mend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow2Mend-6-3"
+ style="overflow:visible"
+ inkscape:isstock="true">
+ <path
+ inkscape:connector-curvature="0"
+ id="path1200-1-0"
+ style="fill:#008000;fill-opacity:1;fill-rule:evenodd;stroke:#008000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1"
+ d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z"
+ transform="scale(-0.6)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow2Mend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow2Mend-8-3-2-8"
+ style="overflow:visible"
+ inkscape:isstock="true">
+ <path
+ inkscape:connector-curvature="0"
+ id="path1200-9-6-9-6"
+ style="fill:#ff0000;fill-opacity:1;fill-rule:evenodd;stroke:#ff0000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1"
+ d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z"
+ transform="scale(-0.6)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow2Mend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow2Mend-3"
+ style="overflow:visible"
+ inkscape:isstock="true">
+ <path
+ id="path1200-6"
+ style="fill:#ff0000;fill-opacity:1;fill-rule:evenodd;stroke:#ff0000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1"
+ d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z"
+ transform="scale(-0.6)"
+ inkscape:connector-curvature="0" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow2Mend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="marker8858-3"
+ style="overflow:visible"
+ inkscape:isstock="true">
+ <path
+ id="path8616-5"
+ style="fill:#00ffcc;fill-opacity:1;fill-rule:evenodd;stroke:#00ffcc;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1"
+ d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z"
+ transform="scale(-0.6)"
+ inkscape:connector-curvature="0" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow2Mend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow2Mend-8-3-3"
+ style="overflow:visible"
+ inkscape:isstock="true">
+ <path
+ inkscape:connector-curvature="0"
+ id="path1200-9-6-56"
+ style="fill:#ff0000;fill-opacity:1;fill-rule:evenodd;stroke:#ff0000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1"
+ d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z"
+ transform="scale(-0.6)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow2Mend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow2Mend-8-0-2"
+ style="overflow:visible"
+ inkscape:isstock="true">
+ <path
+ inkscape:connector-curvature="0"
+ id="path1200-9-3-9"
+ style="fill:#008000;fill-opacity:1;fill-rule:evenodd;stroke:#008000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1"
+ d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z"
+ transform="scale(-0.6)" />
+ </marker>
+ </defs>
+ <sodipodi:namedview
+ id="base"
+ pagecolor="#ffffff"
+ bordercolor="#666666"
+ borderopacity="1.0"
+ inkscape:pageopacity="0.0"
+ inkscape:pageshadow="2"
+ inkscape:zoom="2.8"
+ inkscape:cx="603.80172"
+ inkscape:cy="404.14319"
+ inkscape:document-units="mm"
+ inkscape:current-layer="layer1"
+ showgrid="false"
+ inkscape:window-width="3840"
+ inkscape:window-height="2096"
+ inkscape:window-x="0"
+ inkscape:window-y="27"
+ inkscape:window-maximized="1"
+ showguides="false"
+ fit-margin-top="0"
+ fit-margin-left="0"
+ fit-margin-right="0"
+ fit-margin-bottom="0"
+ units="px"
+ inkscape:snap-global="false" />
+ <metadata
+ id="metadata5">
+ <rdf:RDF>
+ <cc:Work
+ rdf:about="">
+ <dc:format>image/svg+xml</dc:format>
+ <dc:type
+ rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
+ <dc:title />
+ </cc:Work>
+ </rdf:RDF>
+ </metadata>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1"
+ transform="translate(419.79645,20.103767)">
+ <path
+ style="fill:#008000;stroke:#008000;stroke-width:0.59275198;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#Arrow2Mend-8-3-2-7)"
+ d="m -340.37552,57.5332 h -14.81024"
+ id="path1171-7-1-3-0"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:#008000;stroke:#008000;stroke-width:0.59715915;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#Arrow2Mend-8-3-4)"
+ d="m -293.23443,57.5332 h -15.03129"
+ id="path1171-7-1-32"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:#008000;stroke:#008000;stroke-width:0.59275198;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#Arrow2Mend-8-0)"
+ d="M -246.45946,57.5332 H -261.2697"
+ id="path1171-7-6"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:#008000;stroke:#008000;stroke-width:0.59275198;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#Arrow2Mend-6)"
+ d="m -151.28623,57.5332 h -14.81024"
+ id="path1171-0"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#ff0000;stroke-width:0.98222464;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#Arrow2Mend-8-3-2-6)"
+ d="m -310.11621,-10.988713 h -35.41856"
+ id="path1171-7-1-3-5"
+ inkscape:connector-curvature="0"
+ sodipodi:nodetypes="cc" />
+ <path
+ style="fill:none;stroke:#ff0000;stroke-width:1.33745635;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#Arrow2Mend-8-3-2-1)"
+ d="M -174.42569,48.441117 V -10.963061 L -277.26548,-11.45916"
+ id="path1171-7-1-3-4"
+ inkscape:connector-curvature="0"
+ sodipodi:nodetypes="ccc" />
+ <path
+ style="fill:none;stroke:#ff0000;stroke-width:0.95872593;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#Arrow2Mend-8-3)"
+ d="m -262.79442,87.935594 h 14.32069"
+ id="path1171-7-1"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#ff0000;stroke-width:0.97006679;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#Arrow2Mend-8)"
+ d="m -309.80088,87.935594 h 14.44587"
+ id="path1171-7"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#ff0000;stroke-width:0.96187615;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#Arrow2Mend)"
+ d="m -356.45657,87.935594 h 14.20296"
+ id="path1171"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#ff0000;stroke-width:0.96061862;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#Arrow2Mend-8-3-2)"
+ d="m -167.44556,87.935594 h 14.16584"
+ id="path1171-7-1-3"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#008000;stroke-width:0.87091714;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#Arrow2Mend-8-0-7)"
+ d="M -193.82812,48.312503 V 14.168502 l -84.03577,-0.467726"
+ id="path1171-7-6-4"
+ inkscape:connector-curvature="0"
+ sodipodi:nodetypes="ccc" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.81852055;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:4.91112339, 0.81852057;stroke-dashoffset:0;stroke-opacity:1"
+ d="m -133.33998,42.989657 v 5.457081"
+ id="path7149-3-7"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.81852055;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:4.91112339, 0.81852057;stroke-dashoffset:0;stroke-opacity:1"
+ d="m -298.69506,162.44998 v 13.31197"
+ id="path7149"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#008080;stroke-width:0.81852055;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.81852057, 0.81852057;stroke-dashoffset:0;stroke-opacity:1"
+ d="m -242.80131,107.00907 v 9.60171"
+ id="path7040-5-4-7-5-6-9"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#008080;stroke-width:0.81852055;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.81852057, 0.81852057;stroke-dashoffset:0;stroke-opacity:1"
+ d="m -300.34873,107.17445 v 9.6017"
+ id="path7040-5-4-7-5-6"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#008080;stroke-width:0.81852055;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.81852057, 0.81852057;stroke-dashoffset:0;stroke-opacity:1"
+ d="m -359.26293,106.99745 v 9.60171"
+ id="path7040-5-4-7-5"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#008080;stroke-width:0.81852055;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.81852057, 0.81852057;stroke-dashoffset:0;stroke-opacity:1"
+ d="M -369.74543,25.114933 V 37.991587"
+ id="path7040-5-4-7-6"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#008080;stroke-width:0.91136348;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.91136346, 0.91136346;stroke-dashoffset:0;stroke-opacity:1"
+ d="M -135.17034,93.582486 V 107.10642 H -403.93077 V 37.882965 h 109.60575 V 25.225991"
+ id="path7038"
+ inkscape:connector-curvature="0"
+ sodipodi:nodetypes="cccccc" />
+ <path
+ style="fill:none;stroke:#008080;stroke-width:0.81852055;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.81852057, 0.81852057;stroke-dashoffset:0;stroke-opacity:1"
+ d="M -231.106,94.010086 V 106.96943"
+ id="path7040"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#008080;stroke-width:0.81852055;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.81852057, 0.81852057;stroke-dashoffset:0;stroke-opacity:1"
+ d="M -278.50224,93.844719 V 106.80406"
+ id="path7040-5"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#008080;stroke-width:0.81852055;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.81852057, 0.81852057;stroke-dashoffset:0;stroke-opacity:1"
+ d="M -325.89848,93.701083 V 106.99115"
+ id="path7040-5-4"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#008080;stroke-width:0.81852055;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.81852057, 0.81852057;stroke-dashoffset:0;stroke-opacity:1"
+ d="M -373.29471,93.899037 V 107.27179"
+ id="path7040-5-4-7"
+ inkscape:connector-curvature="0" />
+ <g
+ id="g934"
+ transform="matrix(0.61872421,0,0,0.61872421,-154.16506,-3.5724799)">
+ <rect
+ ry="2.1052283e-06"
+ y="84.280701"
+ x="-376.383"
+ height="72.786827"
+ width="49.352299"
+ id="rect834"
+ style="fill:none;stroke:#000000;stroke-width:1.16258347;stroke-linecap:round;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:none" />
+ <text
+ id="text838"
+ y="95.916664"
+ x="-371.17261"
+ style="font-style:normal;font-weight:normal;font-size:10.58333302px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332"
+ xml:space="preserve"><tspan
+ style="stroke-width:0.26458332"
+ y="95.916664"
+ x="-371.17261"
+ id="tspan836"
+ sodipodi:role="line">DCHUB</tspan></text>
+ <text
+ id="text846"
+ y="121.99702"
+ x="-352.74997"
+ style="font-style:normal;font-weight:normal;font-size:10.58333302px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332"
+ xml:space="preserve"><tspan
+ style="text-align:center;text-anchor:middle;stroke-width:0.26458332"
+ y="121.99702"
+ x="-352.74997"
+ id="tspan844"
+ sodipodi:role="line">HUBP</tspan><tspan
+ id="tspan863"
+ style="text-align:center;text-anchor:middle;stroke-width:0.26458332"
+ y="135.22618"
+ x="-352.74997"
+ sodipodi:role="line">(n)</tspan></text>
+ </g>
+ <g
+ id="g942"
+ transform="matrix(0.61872421,0,0,0.61872421,-158.40385,-3.2216813)">
+ <text
+ id="text838-5"
+ y="116.65257"
+ x="-269.45752"
+ style="font-style:normal;font-weight:normal;font-size:10.58333302px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332"
+ xml:space="preserve"><tspan
+ style="text-align:center;text-anchor:middle;stroke-width:0.26458332"
+ y="116.65257"
+ x="-269.45752"
+ id="tspan836-3"
+ sodipodi:role="line">DPP</tspan><tspan
+ id="tspan936"
+ style="text-align:center;text-anchor:middle;stroke-width:0.26458332"
+ y="129.88174"
+ x="-269.45752"
+ sodipodi:role="line">(n)</tspan></text>
+ <rect
+ ry="2.1052283e-06"
+ y="83.71373"
+ x="-293.7952"
+ height="72.786827"
+ width="49.352303"
+ id="rect834-5"
+ style="fill:none;stroke:#000000;stroke-width:1.16258347;stroke-linecap:round;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:none" />
+ </g>
+ <g
+ id="g1158"
+ transform="matrix(0.61872421,0,0,0.61872421,-154.34048,-6.2618995)">
+ <text
+ id="text838-5-2"
+ y="128.87331"
+ x="-200.18195"
+ style="font-style:normal;font-weight:normal;font-size:10.58333302px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332"
+ xml:space="preserve"><tspan
+ id="tspan936-1"
+ style="text-align:center;text-anchor:middle;stroke-width:0.26458332"
+ y="128.87331"
+ x="-200.18195"
+ sodipodi:role="line">MPC</tspan></text>
+ <rect
+ ry="2.1052283e-06"
+ y="88.627419"
+ x="-224.62555"
+ height="72.786827"
+ width="49.352303"
+ id="rect834-5-2"
+ style="fill:none;stroke:#000000;stroke-width:1.16258347;stroke-linecap:round;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:none" />
+ </g>
+ <g
+ id="g1153"
+ transform="matrix(0.61872421,0,0,0.61872421,-108.51628,-6.4957668)">
+ <text
+ id="text838-5-2-7"
+ y="129.2513"
+ x="-120.96272"
+ style="font-style:normal;font-weight:normal;font-size:10.58333302px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332"
+ xml:space="preserve"><tspan
+ id="tspan936-1-0"
+ style="text-align:center;text-anchor:middle;stroke-width:0.26458332"
+ y="129.2513"
+ x="-120.96272"
+ sodipodi:role="line">OPTC</tspan></text>
+ <rect
+ ry="2.1052283e-06"
+ y="89.005402"
+ x="-145.62854"
+ height="72.786827"
+ width="49.352306"
+ id="rect834-5-2-9"
+ style="fill:none;stroke:#000000;stroke-width:1.16258347;stroke-linecap:round;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:none" />
+ </g>
+ <g
+ id="g1148"
+ transform="matrix(0.61872421,0,0,0.61872421,-105.25474,-7.6650796)">
+ <text
+ id="text838-5-2-7-3"
+ y="131.14117"
+ x="-48.981136"
+ style="font-style:normal;font-weight:normal;font-size:10.58333302px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332"
+ xml:space="preserve"><tspan
+ id="tspan936-1-0-6"
+ style="text-align:center;text-anchor:middle;stroke-width:0.26458332"
+ y="131.14117"
+ x="-48.981136"
+ sodipodi:role="line">DIO</tspan></text>
+ <rect
+ ry="2.1052283e-06"
+ y="90.895279"
+ x="-73.435081"
+ height="72.786827"
+ width="49.352306"
+ id="rect834-5-2-9-0"
+ style="fill:none;stroke:#000000;stroke-width:1.16258347;stroke-linecap:round;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:none" />
+ </g>
+ <g
+ id="g1133"
+ transform="matrix(0.61872421,0,0,0.61872421,-181.52704,-7.6650796)">
+ <text
+ id="text838-5-2-6"
+ y="241.13223"
+ x="-286.96921"
+ style="font-style:normal;font-weight:normal;font-size:10.58333302px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332"
+ xml:space="preserve"><tspan
+ id="tspan936-1-2"
+ style="text-align:center;text-anchor:middle;stroke-width:0.26458332"
+ y="241.13223"
+ x="-286.96921"
+ sodipodi:role="line">DCCG</tspan></text>
+ <rect
+ ry="2.1052283e-06"
+ y="200.88634"
+ x="-311.56009"
+ height="72.786827"
+ width="49.352306"
+ id="rect834-5-2-6"
+ style="fill:none;stroke:#000000;stroke-width:1.16258347;stroke-linecap:round;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:none" />
+ </g>
+ <g
+ id="g1138"
+ transform="matrix(0.61872421,0,0,0.61872421,-181.52704,-7.6650796)">
+ <text
+ id="text838-5-2-6-1"
+ y="241.81844"
+ x="-190.55942"
+ style="font-style:normal;font-weight:normal;font-size:10.58333302px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332"
+ xml:space="preserve"><tspan
+ id="tspan936-1-2-8"
+ style="text-align:center;text-anchor:middle;stroke-width:0.26458332"
+ y="241.81844"
+ x="-190.55942"
+ sodipodi:role="line">DMU</tspan></text>
+ <rect
+ ry="2.1052283e-06"
+ y="201.6423"
+ x="-215.17615"
+ height="72.786827"
+ width="49.352306"
+ id="rect834-5-2-6-7"
+ style="fill:none;stroke:#000000;stroke-width:1.16258347;stroke-linecap:round;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:none" />
+ </g>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:6.54816437px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.1637041"
+ x="-297.75696"
+ y="109.44505"
+ id="text1063"><tspan
+ sodipodi:role="line"
+ id="tspan1061"
+ x="-297.75696"
+ y="115.23865"
+ style="stroke-width:0.1637041" /></text>
+ <g
+ id="g1143"
+ transform="matrix(0.61872421,0,0,0.61872421,-181.52704,-8.9747125)">
+ <text
+ id="text838-5-2-6-1-9"
+ y="243.02728"
+ x="-99.967323"
+ style="font-style:normal;font-weight:normal;font-size:10.58333302px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332"
+ xml:space="preserve"><tspan
+ id="tspan936-1-2-8-2"
+ style="text-align:center;text-anchor:middle;stroke-width:0.26458332"
+ y="243.02728"
+ x="-99.967323"
+ sodipodi:role="line">AZ</tspan></text>
+ <rect
+ ry="2.1052283e-06"
+ y="202.77623"
+ x="-124.83984"
+ height="72.786827"
+ width="49.352306"
+ id="rect834-5-2-6-7-0"
+ style="fill:none;stroke:#000000;stroke-width:1.16258347;stroke-linecap:round;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:none" />
+ </g>
+ <g
+ id="g1169"
+ transform="matrix(0.61872421,0,0,0.61872421,-154.16506,1.4555785)">
+ <text
+ id="text838-5-2-6-2"
+ y="5.9612885"
+ x="-348.74365"
+ style="font-style:normal;font-weight:normal;font-size:10.58333302px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332"
+ xml:space="preserve"><tspan
+ id="tspan936-1-2-3"
+ style="text-align:center;text-anchor:middle;stroke-width:0.26458332"
+ y="5.9612885"
+ x="-348.74365"
+ sodipodi:role="line">MMHUBBUB</tspan></text>
+ <rect
+ ry="2.1010696e-06"
+ y="-34.142948"
+ x="-384.64743"
+ height="72.643044"
+ width="72.096924"
+ id="rect834-5-2-6-75"
+ style="fill:none;stroke:#000000;stroke-width:1.40378118;stroke-linecap:round;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:none" />
+ </g>
+ <g
+ id="g1164"
+ transform="matrix(0.61872421,0,0,0.61872421,-154.16506,-7.6650796)">
+ <text
+ id="text838-5-2-6-9"
+ y="13.465075"
+ x="-227.30836"
+ style="font-style:normal;font-weight:normal;font-size:10.58333302px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332"
+ xml:space="preserve"><tspan
+ id="tspan936-1-2-2"
+ style="text-align:center;text-anchor:middle;stroke-width:0.26458332"
+ y="13.465075"
+ x="-227.30836"
+ sodipodi:role="line">DWB</tspan><tspan
+ id="tspan1128"
+ style="text-align:center;text-anchor:middle;stroke-width:0.26458332"
+ y="26.694241"
+ x="-227.30836"
+ sodipodi:role="line">(n)</tspan></text>
+ <rect
+ ry="2.1052283e-06"
+ y="-19.473768"
+ x="-251.83983"
+ height="72.786827"
+ width="49.352306"
+ id="rect834-5-2-6-2"
+ style="fill:none;stroke:#000000;stroke-width:1.16258347;stroke-linecap:round;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:none" />
+ </g>
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.91371936;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:5.48231601, 0.91371934;stroke-dashoffset:0;stroke-opacity:1"
+ d="m -358.95963,161.63019 v 14.12431 h 250.20395 V 43.149938 H -361.845 V 25.478973"
+ id="path7147"
+ inkscape:connector-curvature="0"
+ sodipodi:nodetypes="cccccc" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.81852055;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:4.91112339, 0.81852057;stroke-dashoffset:0;stroke-opacity:1"
+ d="m -242.92533,161.58513 v 14.05612"
+ id="path7149-3"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.81852055;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:4.91112339, 0.81852057;stroke-dashoffset:0;stroke-opacity:1"
+ d="m -184.37695,42.955607 v 5.457082"
+ id="path7149-3-7-4"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.81852055;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:4.91112339, 0.81852057;stroke-dashoffset:0;stroke-opacity:1"
+ d="m -277.36283,43.141644 v 5.457082"
+ id="path7149-3-7-5"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.81852055;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:4.91112339, 0.81852057;stroke-dashoffset:0;stroke-opacity:1"
+ d="M -325.48437,42.976278 V 48.43336"
+ id="path7149-3-7-2"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.81852055;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:4.91112339, 0.81852057;stroke-dashoffset:0;stroke-opacity:1"
+ d="m -361.86492,43.141644 v 5.457083"
+ id="path7149-3-7-54"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:#008000;stroke:#008000;stroke-width:0.46329758;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#Arrow2Mend-6-3)"
+ d="m -147.58542,-8.2978166 h -9.04766"
+ id="path1171-0-7"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#ff0000;stroke-width:0.98222464;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#Arrow2Mend-8-3-2-8)"
+ d="m -157.13421,-1.6500501 h 8.66407"
+ id="path1171-7-1-3-8"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.74503672;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:4.47022031, 0.74503672;stroke-dashoffset:0;stroke-opacity:1"
+ d="m -148.50314,4.9845652 h -7.91265"
+ id="path7149-3-7-8"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#008080;stroke-width:0.81852055;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.81852057, 0.81852057;stroke-dashoffset:0;stroke-opacity:1"
+ d="m -157.59442,11.623513 h 10.26991"
+ id="path7040-4"
+ inkscape:connector-curvature="0" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:6.54816437px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.1637041"
+ x="-142.72867"
+ y="-6.9685979"
+ id="text12079"><tspan
+ sodipodi:role="line"
+ id="tspan12077"
+ x="-142.72867"
+ y="-6.9685979"
+ style="font-size:4.80198765px;stroke-width:0.1637041">Global sync</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:6.54816437px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.1637041"
+ x="-142.93031"
+ y="0.13578746"
+ id="text12079-3"><tspan
+ sodipodi:role="line"
+ id="tspan12077-1"
+ x="-142.93031"
+ y="0.13578746"
+ style="font-size:4.80198765px;stroke-width:0.1637041">Pixel data</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:6.54816437px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.1637041"
+ x="-142.77556"
+ y="6.3093324"
+ id="text12079-3-4"><tspan
+ sodipodi:role="line"
+ id="tspan12077-1-9"
+ x="-142.77556"
+ y="6.3093324"
+ style="font-size:4.80198765px;stroke-width:0.1637041">Sideband signal</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:6.54816437px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.1637041"
+ x="-142.72867"
+ y="12.948278"
+ id="text12079-3-4-2"><tspan
+ sodipodi:role="line"
+ id="tspan12077-1-9-0"
+ x="-142.72867"
+ y="12.948278"
+ style="font-size:4.80198765px;stroke-width:0.1637041">Config. Bus</tspan></text>
+ <path
+ style="fill:none;stroke:#aa00d4;stroke-width:1.32291663;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#marker8858)"
+ d="m -406.68795,73.185276 h 14.20296"
+ id="path1171-75"
+ inkscape:connector-curvature="0" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332"
+ x="-420.21503"
+ y="75.065918"
+ id="text8862"><tspan
+ sodipodi:role="line"
+ id="tspan8860"
+ x="-420.21503"
+ y="75.065918"
+ style="font-size:6.3499999px;stroke-width:0.26458332">SDP</tspan></text>
+ <path
+ style="fill:none;stroke:#00ffcc;stroke-width:1.25980031;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#marker8858-3)"
+ d="m -119.19923,72.243805 h 12.88004"
+ id="path1171-75-6"
+ inkscape:connector-curvature="0" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332"
+ x="-104.87327"
+ y="74.54258"
+ id="text8862-2"><tspan
+ sodipodi:role="line"
+ id="tspan8860-9"
+ x="-104.87327"
+ y="74.54258"
+ style="font-size:6.3499999px;stroke-width:0.26458332">Monitor</tspan></text>
+ <g
+ id="g6280"
+ transform="translate(-133.43389,-37.35791)">
+ <text
+ id="text838-5-2-7-6"
+ y="110.67171"
+ x="-97.4758"
+ style="font-style:normal;font-weight:normal;font-size:6.54816437px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.16370411"
+ xml:space="preserve"><tspan
+ id="tspan936-1-0-7"
+ style="text-align:center;text-anchor:middle;stroke-width:0.16370411"
+ y="110.67171"
+ x="-97.4758"
+ sodipodi:role="line">OPP</tspan></text>
+ <rect
+ ry="1.3025557e-06"
+ y="85.770599"
+ x="-112.73714"
+ height="45.034973"
+ width="30.535467"
+ id="rect834-5-2-9-5"
+ style="fill:none;stroke:#000000;stroke-width:0.71931857;stroke-linecap:round;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:none" />
+ </g>
+ <path
+ style="fill:#008000;stroke:#008000;stroke-width:0.59275198;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#Arrow2Mend-8-0-2)"
+ d="m -199.6735,57.600919 h -14.81024"
+ id="path1171-7-6-1"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#ff0000;stroke-width:0.95872593;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#Arrow2Mend-8-3-3)"
+ d="m -214.95012,88.003315 h 14.32069"
+ id="path1171-7-1-2"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#008080;stroke-width:0.81852055;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.81852058, 0.81852058;stroke-dashoffset:0;stroke-opacity:1"
+ d="M -182.99565,94.057598 V 107.01694"
+ id="path7040-7"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.81852055;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:4.91112344, 0.81852058;stroke-dashoffset:0;stroke-opacity:1"
+ d="m -231.7616,43.563759 v 5.457082"
+ id="path7149-3-7-4-0"
+ inkscape:connector-curvature="0" />
+ <g
+ aria-label="["
+ transform="matrix(0,-1,0.74237844,0,14.567595,39.540924)"
+ style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332"
+ id="text6872">
+ <path
+ d="m -65.936923,-548.78511 h 8.816294 v 2.79112 h -6.82247 v 176.34952 h 6.82247 v 2.41314 h -8.816294 z"
+ style="font-size:50.79999924px;stroke-width:0.26458332"
+ id="path6874"
+ inkscape:connector-curvature="0"
+ sodipodi:nodetypes="ccccccccc" />
+ </g>
+ <g
+ aria-label="["
+ transform="rotate(-90,182.49521,-144.01791)"
+ style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332"
+ id="text6872-3">
+ <path
+ d="m -63.825546,-623.34091 h 7.228794 v 2.26195 h -5.764137 l 0,127.08032 h 5.764137 v 1.88397 h -7.228794 z"
+ style="font-size:50.79999924px;stroke-width:0.26458332"
+ id="path6874-6"
+ inkscape:connector-curvature="0"
+ sodipodi:nodetypes="ccccccccc" />
+ </g>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332"
+ x="-359.80389"
+ y="99.104233"
+ id="text6929"><tspan
+ sodipodi:role="line"
+ id="tspan6927"
+ x="-359.80389"
+ y="99.104233"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444447px;font-family:sans-serif;-inkscape-font-specification:'sans-serif Bold';stroke-width:0.26458332">dc_plane</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332"
+ x="-223.56163"
+ y="99.142021"
+ id="text6933"><tspan
+ sodipodi:role="line"
+ id="tspan6931"
+ x="-223.56163"
+ y="99.142021"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444447px;font-family:sans-serif;-inkscape-font-specification:'sans-serif Bold';stroke-width:0.26458332">dc_stream</tspan></text>
+ <g
+ aria-label="["
+ transform="matrix(0,1,1,0,153.30551,96.566025)"
+ style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332"
+ id="text6872-35">
+ <path
+ d="m -65.936923,-545.95029 h 8.816294 v 2.79112 h -6.898066 v 271.78851 h 6.898066 v 2.41314 h -8.816294 z"
+ style="font-size:50.79999924px;stroke-width:0.26458332"
+ id="path6874-62"
+ inkscape:connector-curvature="0"
+ sodipodi:nodetypes="ccccccccc" />
+ </g>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332"
+ x="-267.43958"
+ y="28.5028"
+ id="text6933-9"><tspan
+ sodipodi:role="line"
+ id="tspan6931-1"
+ x="-267.43958"
+ y="28.5028"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444447px;font-family:sans-serif;-inkscape-font-specification:'sans-serif Bold';stroke-width:0.26458332">dc_state</tspan></text>
+ <g
+ aria-label="["
+ transform="matrix(0,0.98158883,-1.0187565,0,0,-7.4835468)"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:6.62759447px;line-height:1.25;font-family:sans-serif;-inkscape-font-specification:'sans-serif Bold';letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.5522995"
+ id="text6973">
+ <path
+ d="m 23.679381,144.30265 h 3.028123 v 1.29445 h -1.820839 v 7.91629 h 1.820839 v 1.29445 h -3.028123 z"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:11.78239059px;font-family:sans-serif;-inkscape-font-specification:'sans-serif Bold';stroke-width:0.5522995"
+ id="path6975"
+ inkscape:connector-curvature="0"
+ sodipodi:nodetypes="ccccccccc" />
+ </g>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332"
+ x="-142.71655"
+ y="18.955769"
+ id="text6980"><tspan
+ sodipodi:role="line"
+ id="tspan6978"
+ x="-142.71655"
+ y="18.955769"
+ style="font-size:4.58611107px;stroke-width:0.26458332">Code struct</tspan></text>
+ <g
+ aria-label="["
+ transform="rotate(-90,94.826273,-58.762727)"
+ style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332"
+ id="text6872-35-2">
+ <path
+ d="m -66.881863,-308.95922 h 7.115401 l 0,1.69499 h -5.197173 v 42.03568 h 5.197173 v 1.78948 h -7.115401 z"
+ style="font-size:50.79999924px;stroke-width:0.26458332"
+ id="path6874-62-7"
+ inkscape:connector-curvature="0"
+ sodipodi:nodetypes="ccccccccc" />
+ </g>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332"
+ x="-134.09625"
+ y="99.354439"
+ id="text6933-9-0"><tspan
+ sodipodi:role="line"
+ id="tspan6931-1-9"
+ x="-134.09625"
+ y="99.354439"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444447px;font-family:sans-serif;-inkscape-font-specification:'sans-serif Bold';stroke-width:0.26458332">dc_link</tspan></text>
+ <g
+ aria-label="}"
+ transform="rotate(90,-145.27371,-140.09832)"
+ style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#3771c8;fill-opacity:1;stroke:none;stroke-width:0.26458332"
+ id="text1003">
+ <path
+ d="m 102.58571,58.211269 h 0.86816 c 1.15755,0 3.11267,-0.177767 3.45994,-0.5333 0.35553,-0.355534 1.28925,-1.124479 1.28925,-2.306836 V -61.475482 c 0,-1.289844 0.18603,-2.228288 0.5581,-2.815332 0.37207,-0.587044 0.26105,-0.992187 1.17882,-1.215429 -0.91777,-0.206706 -0.80675,-0.603581 -1.17882,-1.190625 -0.37207,-0.587045 -0.5581,-1.529623 -0.5581,-2.827735 v -3.075781 c 0,-1.174088 -0.93372,-1.938899 -1.28925,-2.294433 -0.34727,-0.363802 -2.30239,-0.545703 -3.45994,-0.545703 h -0.86816 v -1.773536 h 0.78134 c 2.05879,0 4.63403,0.305924 5.32029,0.917774 0.69453,0.60358 1.0418,1.81901 1.0418,3.646289 v 2.976562 c 0,1.231966 0.22324,2.087728 0.66973,2.567285 0.44648,0.471289 5.80035,0.706934 6.97444,0.706934 h 0.76894 v 1.773535 h -0.76894 c -1.17409,0 -6.52796,0.239778 -6.97444,0.719336 -0.44649,0.479557 -0.66973,1.343587 -0.66973,2.59209 V 55.420742 c 0,1.827279 -0.34727,3.046842 -1.0418,3.658691 -0.68626,0.611849 -3.2615,0.917774 -5.32029,0.917774 h -0.78134 z"
+ style="font-size:25.39999962px;fill:#3771c8;stroke-width:0.26458332"
+ id="path1005"
+ inkscape:connector-curvature="0"
+ sodipodi:nodetypes="cscsscccsscsccscsscsccscsscscc" />
+ </g>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332"
+ x="-200.59984"
+ y="129.60852"
+ id="text1010"><tspan
+ sodipodi:role="line"
+ id="tspan1008"
+ x="-200.59984"
+ y="129.60852"
+ style="font-style:italic;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:6.3499999px;font-family:sans-serif;-inkscape-font-specification:'sans-serif Bold Italic';text-align:center;text-anchor:middle;stroke-width:0.26458332">Floating point</tspan><tspan
+ sodipodi:role="line"
+ x="-200.59984"
+ y="137.54602"
+ style="font-size:6.3499999px;text-align:center;text-anchor:middle;stroke-width:0.26458332"
+ id="tspan1059">calculation</tspan></text>
+ <g
+ aria-label="}"
+ transform="rotate(90,-94.294068,-92.593178)"
+ style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#3771c8;fill-opacity:1;stroke:none;stroke-width:0.26458332"
+ id="text1003-5">
+ <path
+ d="m 102.58571,58.211269 h 0.86816 c 1.15755,0 3.11267,-0.177767 3.45994,-0.5333 0.35553,-0.355534 1.10026,-1.124479 1.10026,-2.306836 V -44.637502 c 0,-1.289844 0.18603,-2.228288 0.5581,-2.815332 0.37207,-0.587044 0.45004,-0.992187 1.36781,-1.215429 -0.91777,-0.206706 -0.99574,-0.603581 -1.36781,-1.190625 -0.37207,-0.587045 -0.5581,-1.529623 -0.5581,-2.827735 v -19.913761 c 0,-1.174088 -0.74473,-1.938899 -1.10026,-2.294433 -0.34727,-0.363802 -2.30239,-0.545703 -3.45994,-0.545703 h -0.86816 v -1.773536 h 0.78134 c 2.05879,0 4.63403,0.305924 5.32029,0.917774 0.69453,0.60358 1.0418,1.81901 1.0418,3.646289 v 19.814542 c 0,1.231966 0.22324,2.087728 0.66973,2.567285 0.44648,0.471289 1.25677,0.706934 2.43086,0.706934 h 0.76894 v 1.773535 h -0.76894 c -1.17409,0 -1.98438,0.239778 -2.43086,0.719336 -0.44649,0.479557 -0.66973,1.343587 -0.66973,2.59209 v 99.897013 c 0,1.827279 -0.34727,3.046842 -1.0418,3.658691 -0.68626,0.611849 -3.2615,0.917774 -5.32029,0.917774 h -0.78134 z"
+ style="font-size:25.39999962px;fill:#3771c8;stroke-width:0.26458332"
+ id="path1005-3"
+ inkscape:connector-curvature="0"
+ sodipodi:nodetypes="cscsscccsscsccscsscsccscsscscc" />
+ </g>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332"
+ x="-137.43764"
+ y="122.46283"
+ id="text1010-5"><tspan
+ sodipodi:role="line"
+ id="tspan1008-6"
+ x="-137.43764"
+ y="122.46283"
+ style="font-size:6.3499999px;text-align:center;text-anchor:middle;stroke-width:0.26458332">bit-depth</tspan><tspan
+ sodipodi:role="line"
+ x="-137.43764"
+ y="130.40033"
+ style="font-size:6.3499999px;text-align:center;text-anchor:middle;stroke-width:0.26458332"
+ id="tspan1057">reduction/dither</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#3771c8;fill-opacity:1;stroke:none;stroke-width:0.26458332;"
+ x="21.087883"
+ y="155.64751"
+ id="text1064"
+ transform="rotate(90)"><tspan
+ sodipodi:role="line"
+ id="tspan1062"
+ x="21.087883"
+ y="155.64751"
+ style="font-size:9.87777805px;stroke-width:0.26458332;fill:#3771c8;">}</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332"
+ x="-142.71655"
+ y="25.939869"
+ id="text6980-9"><tspan
+ sodipodi:role="line"
+ id="tspan6978-1"
+ x="-142.71655"
+ y="25.939869"
+ style="font-size:4.58611107px;stroke-width:0.26458332">Notes</tspan></text>
+ </g>
+</svg>
diff --git a/Documentation/gpu/amdgpu/display/dcn-overview.rst b/Documentation/gpu/amdgpu/display/dcn-overview.rst
new file mode 100644
index 0000000..f98624d
--- /dev/null
+++ b/Documentation/gpu/amdgpu/display/dcn-overview.rst
@@ -0,0 +1,171 @@
+=======================
+Display Core Next (DCN)
+=======================
+
+To equip our readers with the basic knowledge of how AMD Display Core Next
+(DCN) works, we need to start with an overview of the hardware pipeline. Below
+you can see a picture that provides a DCN overview, keep in mind that this is a
+generic diagram, and we have variations per ASIC.
+
+.. kernel-figure:: dc_pipeline_overview.svg
+
+Based on this diagram, we can pass through each block and briefly describe
+them:
+
+* **Display Controller Hub (DCHUB)**: This is the gateway between the Scalable
+ Data Port (SDP) and DCN. This component has multiple features, such as memory
+ arbitration, rotation, and cursor manipulation.
+
+* **Display Pipe and Plane (DPP)**: This block provides pre-blend pixel
+ processing such as color space conversion, linearization of pixel data, tone
+ mapping, and gamut mapping.
+
+* **Multiple Pipe/Plane Combined (MPC)**: This component performs blending of
+ multiple planes, using global or per-pixel alpha.
+
+* **Output Pixel Processing (OPP)**: Process and format pixels to be sent to
+ the display.
+
+* **Output Pipe Timing Combiner (OPTC)**: It generates time output to combine
+ streams or divide capabilities. CRC values are generated in this block.
+
+* **Display Output (DIO)**: Codify the output to the display connected to our
+ GPU.
+
+* **Display Writeback (DWB)**: It provides the ability to write the output of
+ the display pipe back to memory as video frames.
+
+* **Multi-Media HUB (MMHUBBUB)**: Memory controller interface for DMCUB and DWB
+ (Note that DWB is not hooked yet).
+
+* **DCN Management Unit (DMU)**: It provides registers with access control and
+ interrupts the controller to the SOC host interrupt unit. This block includes
+ the Display Micro-Controller Unit - version B (DMCUB), which is handled via
+ firmware.
+
+* **DCN Clock Generator Block (DCCG)**: It provides the clocks and resets
+ for all of the display controller clock domains.
+
+* **Azalia (AZ)**: Audio engine.
+
+The above diagram is an architecture generalization of DCN, which means that
+every ASIC has variations around this base model. Notice that the display
+pipeline is connected to the Scalable Data Port (SDP) via DCHUB; you can see
+the SDP as the element from our Data Fabric that feeds the display pipe.
+
+Always approach the DCN architecture as something flexible that can be
+configured and reconfigured in multiple ways; in other words, each block can be
+setup or ignored accordingly with userspace demands. For example, if we
+want to drive an 8k@60Hz with a DSC enabled, our DCN may require 4 DPP and 2
+OPP. It is DC's responsibility to drive the best configuration for each
+specific scenario. Orchestrate all of these components together requires a
+sophisticated communication interface which is highlighted in the diagram by
+the edges that connect each block; from the chart, each connection between
+these blocks represents:
+
+1. Pixel data interface (red): Represents the pixel data flow;
+2. Global sync signals (green): It is a set of synchronization signals composed
+ by VStartup, VUpdate, and VReady;
+3. Config interface: Responsible to configure blocks;
+4. Sideband signals: All other signals that do not fit the previous one.
+
+These signals are essential and play an important role in DCN. Nevertheless,
+the Global Sync deserves an extra level of detail described in the next
+section.
+
+All of these components are represented by a data structure named dc_state.
+From DCHUB to MPC, we have a representation called dc_plane; from MPC to OPTC,
+we have dc_stream, and the output (DIO) is handled by dc_link. Keep in mind
+that HUBP accesses a surface using a specific format read from memory, and our
+dc_plane should work to convert all pixels in the plane to something that can
+be sent to the display via dc_stream and dc_link.
+
+Front End and Back End
+----------------------
+
+Display pipeline can be broken down into two components that are usually
+referred as **Front End (FE)** and **Back End (BE)**, where FE consists of:
+
+* DCHUB (Mainly referring to a subcomponent named HUBP)
+* DPP
+* MPC
+
+On the other hand, BE consist of
+
+* OPP
+* OPTC
+* DIO (DP/HDMI stream encoder and link encoder)
+
+OPP and OPTC are two joining blocks between FE and BE. On a side note, this is
+a one-to-one mapping of the link encoder to PHY, but we can configure the DCN
+to choose which link encoder to connect to which PHY. FE's main responsibility
+is to change, blend and compose pixel data, while BE's job is to frame a
+generic pixel stream to a specific display's pixel stream.
+
+Data Flow
+---------
+
+Initially, data is passed in from VRAM through Data Fabric (DF) in native pixel
+formats. Such data format stays through till HUBP in DCHUB, where HUBP unpacks
+different pixel formats and outputs them to DPP in uniform streams through 4
+channels (1 for alpha + 3 for colors).
+
+The Converter and Cursor (CNVC) in DPP would then normalize the data
+representation and convert them to a DCN specific floating-point format (i.e.,
+different from the IEEE floating-point format). In the process, CNVC also
+applies a degamma function to transform the data from non-linear to linear
+space to relax the floating-point calculations following. Data would stay in
+this floating-point format from DPP to OPP.
+
+Starting OPP, because color transformation and blending have been completed
+(i.e alpha can be dropped), and the end sinks do not require the precision and
+dynamic range that floating points provide (i.e. all displays are in integer
+depth format), bit-depth reduction/dithering would kick in. In OPP, we would
+also apply a regamma function to introduce the gamma removed earlier back.
+Eventually, we output data in integer format at DIO.
+
+Global Sync
+-----------
+
+Many DCN registers are double buffered, most importantly the surface address.
+This allows us to update DCN hardware atomically for page flips, as well as
+for most other updates that don't require enabling or disabling of new pipes.
+
+(Note: There are many scenarios when DC will decide to reserve extra pipes
+in order to support outputs that need a very high pixel clock, or for
+power saving purposes.)
+
+These atomic register updates are driven by global sync signals in DCN. In
+order to understand how atomic updates interact with DCN hardware, and how DCN
+signals page flip and vblank events it is helpful to understand how global sync
+is programmed.
+
+Global sync consists of three signals, VSTARTUP, VUPDATE, and VREADY. These are
+calculated by the Display Mode Library - DML (drivers/gpu/drm/amd/display/dc/dml)
+based on a large number of parameters and ensure our hardware is able to feed
+the DCN pipeline without underflows or hangs in any given system configuration.
+The global sync signals always happen during VBlank, are independent from the
+VSync signal, and do not overlap each other.
+
+VUPDATE is the only signal that is of interest to the rest of the driver stack
+or userspace clients as it signals the point at which hardware latches to
+atomically programmed (i.e. double buffered) registers. Even though it is
+independent of the VSync signal we use VUPDATE to signal the VSync event as it
+provides the best indication of how atomic commits and hardware interact.
+
+Since DCN hardware is double-buffered the DC driver is able to program the
+hardware at any point during the frame.
+
+The below picture illustrates the global sync signals:
+
+.. kernel-figure:: global_sync_vblank.svg
+
+These signals affect core DCN behavior. Programming them incorrectly will lead
+to a number of negative consequences, most of them quite catastrophic.
+
+The following picture shows how global sync allows for a mailbox style of
+updates, i.e. it allows for multiple re-configurations between VUpdate
+events where only the last configuration programmed before the VUpdate signal
+becomes effective.
+
+.. kernel-figure:: config_example.svg
diff --git a/Documentation/gpu/amdgpu/display/display-manager.rst b/Documentation/gpu/amdgpu/display/display-manager.rst
new file mode 100644
index 0000000..7ce31f8
--- /dev/null
+++ b/Documentation/gpu/amdgpu/display/display-manager.rst
@@ -0,0 +1,42 @@
+======================
+AMDgpu Display Manager
+======================
+
+.. contents:: Table of Contents
+ :depth: 3
+
+.. kernel-doc:: drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
+ :doc: overview
+
+.. kernel-doc:: drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h
+ :internal:
+
+Lifecycle
+=========
+
+.. kernel-doc:: drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
+ :doc: DM Lifecycle
+
+.. kernel-doc:: drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
+ :functions: dm_hw_init dm_hw_fini
+
+Interrupts
+==========
+
+.. kernel-doc:: drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq.c
+ :doc: overview
+
+.. kernel-doc:: drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq.c
+ :internal:
+
+.. kernel-doc:: drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
+ :functions: register_hpd_handlers dm_crtc_high_irq dm_pflip_high_irq
+
+Atomic Implementation
+=====================
+
+.. kernel-doc:: drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
+ :doc: atomic
+
+.. kernel-doc:: drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
+ :functions: amdgpu_dm_atomic_check amdgpu_dm_atomic_commit_tail
diff --git a/Documentation/gpu/amdgpu/display/global_sync_vblank.svg b/Documentation/gpu/amdgpu/display/global_sync_vblank.svg
new file mode 100644
index 0000000..48f5dc4
--- /dev/null
+++ b/Documentation/gpu/amdgpu/display/global_sync_vblank.svg
@@ -0,0 +1,485 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ xmlns:dc="http://purl.org/dc/elements/1.1/"
+ xmlns:cc="http://creativecommons.org/ns#"
+ xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+ xmlns:svg="http://www.w3.org/2000/svg"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ width="232.24133mm"
+ height="96.174995mm"
+ viewBox="0 0 232.24133 96.174995"
+ version="1.1"
+ id="svg8"
+ inkscape:version="0.92.5 (2060ec1f9f, 2020-04-08)"
+ sodipodi:docname="global_sync_vblank.svg">
+ <defs
+ id="defs2">
+ <marker
+ inkscape:stockid="Arrow2Mend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow2Mend"
+ style="overflow:visible"
+ inkscape:isstock="true">
+ <path
+ id="path862"
+ style="fill:#800080;fill-opacity:1;fill-rule:evenodd;stroke:#800080;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1"
+ d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z"
+ transform="scale(-0.6)"
+ inkscape:connector-curvature="0" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow2Send"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow2Send"
+ style="overflow:visible"
+ inkscape:isstock="true">
+ <path
+ id="path868"
+ style="fill:#ff00ff;fill-opacity:1;fill-rule:evenodd;stroke:#ff00ff;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1"
+ d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z"
+ transform="matrix(-0.3,0,0,-0.3,0.69,0)"
+ inkscape:connector-curvature="0" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow2Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow2Lend"
+ style="overflow:visible"
+ inkscape:isstock="true">
+ <path
+ id="path856"
+ style="fill:#ff00ff;fill-opacity:1;fill-rule:evenodd;stroke:#ff00ff;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1"
+ d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z"
+ transform="matrix(-1.1,0,0,-1.1,-1.1,0)"
+ inkscape:connector-curvature="0" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="marker1719"
+ style="overflow:visible"
+ inkscape:isstock="true">
+ <path
+ id="path1717"
+ d="M 0,0 5,-5 -12.5,0 5,5 Z"
+ style="fill:#ff00ff;fill-opacity:1;fill-rule:evenodd;stroke:#ff00ff;stroke-width:1.00000003pt;stroke-opacity:1"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)"
+ inkscape:connector-curvature="0" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="marker1661"
+ style="overflow:visible"
+ inkscape:isstock="true">
+ <path
+ id="path1659"
+ d="M 0,0 5,-5 -12.5,0 5,5 Z"
+ style="fill:#ff00ff;fill-opacity:1;fill-rule:evenodd;stroke:#ff00ff;stroke-width:1.00000003pt;stroke-opacity:1"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)"
+ inkscape:connector-curvature="0" />
+ </marker>
+ <marker
+ inkscape:isstock="true"
+ style="overflow:visible"
+ id="marker1311"
+ refX="0"
+ refY="0"
+ orient="auto"
+ inkscape:stockid="Arrow1Lend"
+ inkscape:collect="always">
+ <path
+ transform="matrix(-0.8,0,0,-0.8,-10,0)"
+ style="fill:#ff0000;fill-opacity:1;fill-rule:evenodd;stroke:#ff0000;stroke-width:1.00000003pt;stroke-opacity:1"
+ d="M 0,0 5,-5 -12.5,0 5,5 Z"
+ id="path1309"
+ inkscape:connector-curvature="0" />
+ </marker>
+ <marker
+ inkscape:isstock="true"
+ style="overflow:visible"
+ id="marker1253"
+ refX="0"
+ refY="0"
+ orient="auto"
+ inkscape:stockid="Arrow1Lstart">
+ <path
+ transform="matrix(0.8,0,0,0.8,10,0)"
+ style="fill:#ff0000;fill-opacity:1;fill-rule:evenodd;stroke:#ff0000;stroke-width:1.00000003pt;stroke-opacity:1"
+ d="M 0,0 5,-5 -12.5,0 5,5 Z"
+ id="path1251"
+ inkscape:connector-curvature="0" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lend"
+ style="overflow:visible"
+ inkscape:isstock="true"
+ inkscape:collect="always">
+ <path
+ id="path838"
+ d="M 0,0 5,-5 -12.5,0 5,5 Z"
+ style="fill:#ff0000;fill-opacity:1;fill-rule:evenodd;stroke:#ff0000;stroke-width:1.00000003pt;stroke-opacity:1"
+ transform="matrix(-0.8,0,0,-0.8,-10,0)"
+ inkscape:connector-curvature="0" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Lstart"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Lstart"
+ style="overflow:visible"
+ inkscape:isstock="true"
+ inkscape:collect="always">
+ <path
+ id="path835"
+ d="M 0,0 5,-5 -12.5,0 5,5 Z"
+ style="fill:#ff0000;fill-opacity:1;fill-rule:evenodd;stroke:#ff0000;stroke-width:1.00000003pt;stroke-opacity:1"
+ transform="matrix(0.8,0,0,0.8,10,0)"
+ inkscape:connector-curvature="0" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow1Send"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow1Send"
+ style="overflow:visible"
+ inkscape:isstock="true">
+ <path
+ id="path850"
+ d="M 0,0 5,-5 -12.5,0 5,5 Z"
+ style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;stroke-opacity:1"
+ transform="matrix(-0.2,0,0,-0.2,-1.2,0)"
+ inkscape:connector-curvature="0" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow2Sstart"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow2Sstart"
+ style="overflow:visible"
+ inkscape:isstock="true">
+ <path
+ id="path865"
+ style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1"
+ d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z"
+ transform="matrix(0.3,0,0,0.3,-0.69,0)"
+ inkscape:connector-curvature="0" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow2Mend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow2Mend-2"
+ style="overflow:visible"
+ inkscape:isstock="true">
+ <path
+ inkscape:connector-curvature="0"
+ id="path862-3"
+ style="fill:#800080;fill-opacity:1;fill-rule:evenodd;stroke:#800080;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1"
+ d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z"
+ transform="scale(-0.6)" />
+ </marker>
+ <marker
+ inkscape:stockid="Arrow2Mend"
+ orient="auto"
+ refY="0"
+ refX="0"
+ id="Arrow2Mend-2-5"
+ style="overflow:visible"
+ inkscape:isstock="true">
+ <path
+ inkscape:connector-curvature="0"
+ id="path862-3-9"
+ style="fill:#800080;fill-opacity:1;fill-rule:evenodd;stroke:#800080;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1"
+ d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z"
+ transform="scale(-0.6)" />
+ </marker>
+ </defs>
+ <sodipodi:namedview
+ id="base"
+ pagecolor="#ffffff"
+ bordercolor="#666666"
+ borderopacity="1.0"
+ inkscape:pageopacity="0.0"
+ inkscape:pageshadow="2"
+ inkscape:zoom="1.979899"
+ inkscape:cx="747.52324"
+ inkscape:cy="319.84503"
+ inkscape:document-units="mm"
+ inkscape:current-layer="layer1"
+ showgrid="true"
+ inkscape:window-width="3840"
+ inkscape:window-height="2096"
+ inkscape:window-x="0"
+ inkscape:window-y="27"
+ inkscape:window-maximized="1"
+ fit-margin-top="0"
+ fit-margin-left="0"
+ fit-margin-right="0"
+ fit-margin-bottom="0">
+ <inkscape:grid
+ type="xygrid"
+ id="grid815"
+ originx="15.282997"
+ originy="-184.54792" />
+ </sodipodi:namedview>
+ <metadata
+ id="metadata5">
+ <rdf:RDF>
+ <cc:Work
+ rdf:about="">
+ <dc:format>image/svg+xml</dc:format>
+ <dc:type
+ rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
+ <dc:title />
+ </cc:Work>
+ </rdf:RDF>
+ </metadata>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1"
+ transform="translate(15.282998,-16.277083)">
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.5291667;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
+ d="M 15.875,27.125001 V 16.541666 H 26.458333 V 27.125001 H 177.27084 V 16.541666 h 10.58333 v 10.583335 h 29.10416"
+ id="path817"
+ inkscape:connector-curvature="0"
+ sodipodi:nodetypes="ccccccccc" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.5291667;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
+ d="M 15.875,37.708334 H 44.979166 V 48.291667 H 100.54167 V 37.708334 H 206.375 v 10.583333 h 10.58333"
+ id="path819"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.5291667;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
+ d="m 15.875,66.8125 h 97.89583 V 56.229167 h 7.9375 V 66.8125 h 92.60417"
+ id="path821"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.5291667;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
+ d="m 15.875,85.333334 c 0,0 132.29166,0 132.29166,0 V 74.75 h 15.875 v 10.583334 h 47.625"
+ id="path823"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.5291667;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
+ d="M 15.875,101.20833 H 187.85416 V 90.625 h 10.58334 v 10.58333 h 10.58333"
+ id="path825"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.5291667;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.52916667, 0.52916667;stroke-dashoffset:0;stroke-opacity:1"
+ d="M 100.54167,48.291667 V 111.79167"
+ id="path827"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.5291667;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.52916667, 0.52916667;stroke-dashoffset:0;stroke-opacity:1"
+ d="m 113.77083,66.8125 v 44.97917"
+ id="path829"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#000000;stroke-width:0.5291667;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.52916667, 0.52916667;stroke-dashoffset:0;stroke-opacity:1"
+ d="M 206.375,48.291667 V 109.14583"
+ id="path831"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#ff0000;stroke-width:0.26458332px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-start:url(#Arrow1Lstart);marker-end:url(#Arrow1Lend)"
+ d="m 100.54167,106.5 h 13.22916"
+ id="path833"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;stroke:#ff0000;stroke-width:0.26458332px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-start:url(#marker1253);marker-end:url(#marker1311)"
+ d="M 113.77083,106.5 H 206.375"
+ id="path1243"
+ inkscape:connector-curvature="0" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332"
+ x="105.83333"
+ y="111.79166"
+ id="text1405"><tspan
+ sodipodi:role="line"
+ id="tspan1403"
+ x="105.83333"
+ y="111.79166"
+ style="stroke-width:0.26458332">To</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332"
+ x="145.52083"
+ y="111.79166"
+ id="text1409"><tspan
+ sodipodi:role="line"
+ id="tspan1407"
+ x="145.52083"
+ y="111.79166"
+ style="stroke-width:0.26458332">VStartup Period</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332"
+ x="156.01123"
+ y="78.71875"
+ id="text1413"><tspan
+ sodipodi:role="line"
+ x="156.01123"
+ y="78.71875"
+ style="font-weight:bold;text-align:center;text-anchor:middle;stroke-width:0.26458332"
+ id="tspan1415">VUpdate</tspan><tspan
+ sodipodi:role="line"
+ x="156.01123"
+ y="82.6875"
+ style="font-weight:bold;text-align:center;text-anchor:middle;stroke-width:0.26458332"
+ id="tspan1440">Width</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332"
+ x="173.77611"
+ y="92.703873"
+ id="text1413-3"><tspan
+ sodipodi:role="line"
+ id="tspan1411-6"
+ x="173.77611"
+ y="92.703873"
+ style="font-weight:bold;text-align:center;text-anchor:middle;stroke-width:0.26458332">VReady</tspan><tspan
+ sodipodi:role="line"
+ x="173.77611"
+ y="96.672623"
+ style="font-weight:bold;text-align:center;text-anchor:middle;stroke-width:0.26458332"
+ id="tspan1415-7">Offset</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332"
+ x="135.78951"
+ y="70.78125"
+ id="text1413-5"><tspan
+ sodipodi:role="line"
+ x="135.78951"
+ y="70.78125"
+ style="font-weight:bold;text-align:center;text-anchor:middle;stroke-width:0.26458332"
+ id="tspan1440-5">VUpdate</tspan><tspan
+ sodipodi:role="line"
+ x="135.78951"
+ y="74.75"
+ style="font-weight:bold;text-align:center;text-anchor:middle;stroke-width:0.26458332"
+ id="tspan1465">Offset</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332"
+ x="137.39433"
+ y="48.291664"
+ id="text1479"><tspan
+ sodipodi:role="line"
+ id="tspan1477"
+ x="137.39433"
+ y="48.291664"
+ style="font-weight:bold;stroke-width:0.26458332">VSTARTUP_START</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332"
+ x="-5.4806676"
+ y="22.778271"
+ id="text1479-1"><tspan
+ sodipodi:role="line"
+ id="tspan1477-2"
+ x="-5.4806676"
+ y="22.778271"
+ style="font-weight:bold;font-size:4.93888903px;stroke-width:0.26458332">VSYNC</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332"
+ x="-9.3767252"
+ y="45.64584"
+ id="text1479-1-7"><tspan
+ sodipodi:role="line"
+ id="tspan1477-2-0"
+ x="-9.3767252"
+ y="45.64584"
+ style="font-weight:bold;font-size:5.64444447px;stroke-width:0.26458332">VBlank</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332"
+ x="-15.310558"
+ y="64.92263"
+ id="text1479-1-7-9"><tspan
+ sodipodi:role="line"
+ id="tspan1477-2-0-3"
+ x="-15.310558"
+ y="64.92263"
+ style="font-weight:bold;font-size:5.64444447px;stroke-width:0.26458332">VStartup</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332"
+ x="-14.17781"
+ y="85.144356"
+ id="text1479-1-7-9-6"><tspan
+ sodipodi:role="line"
+ id="tspan1477-2-0-3-0"
+ x="-14.17781"
+ y="85.144356"
+ style="font-weight:bold;font-size:5.64444447px;stroke-width:0.26458332">VUpdate</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332"
+ x="-11.052421"
+ y="101.39733"
+ id="text1479-1-7-9-6-6"><tspan
+ sodipodi:role="line"
+ id="tspan1477-2-0-3-0-2"
+ x="-11.052421"
+ y="101.39733"
+ style="font-weight:bold;font-size:5.64444447px;stroke-width:0.26458332">VReady</tspan></text>
+ <g
+ id="g5189"
+ transform="translate(269.875,-14.287499)">
+ <path
+ sodipodi:nodetypes="cc"
+ inkscape:connector-curvature="0"
+ id="path5143"
+ d="m -202.40625,45.645828 3.96875,-7.9375"
+ style="fill:none;stroke:#000000;stroke-width:0.5291667;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" />
+ <path
+ sodipodi:nodetypes="cc"
+ inkscape:connector-curvature="0"
+ id="path5143-2"
+ d="m -199.76042,45.645828 3.96874,-7.937499"
+ style="fill:none;stroke:#000000;stroke-width:0.52916676;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" />
+ </g>
+ <g
+ id="g5189-3"
+ transform="translate(268.55209,7.9375003)">
+ <path
+ sodipodi:nodetypes="cc"
+ inkscape:connector-curvature="0"
+ id="path5143-6"
+ d="m -202.40625,45.645828 3.96875,-7.9375"
+ style="fill:none;stroke:#000000;stroke-width:0.5291667;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" />
+ <path
+ sodipodi:nodetypes="cc"
+ inkscape:connector-curvature="0"
+ id="path5143-2-1"
+ d="m -199.76042,45.645828 3.96874,-7.937499"
+ style="fill:none;stroke:#000000;stroke-width:0.52916676;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" />
+ </g>
+ </g>
+</svg>
diff --git a/Documentation/gpu/amdgpu/display/index.rst b/Documentation/gpu/amdgpu/display/index.rst
new file mode 100644
index 0000000..c1fb2fb
--- /dev/null
+++ b/Documentation/gpu/amdgpu/display/index.rst
@@ -0,0 +1,31 @@
+.. _amdgpu-display-core:
+
+===================================
+drm/amd/display - Display Core (DC)
+===================================
+
+AMD display engine is partially shared with other operating systems; for this
+reason, our Display Core Driver is divided into two pieces:
+
+1. **Display Core (DC)** contains the OS-agnostic components. Things like
+ hardware programming and resource management are handled here.
+2. **Display Manager (DM)** contains the OS-dependent components. Hooks to the
+ amdgpu base driver and DRM are implemented here.
+
+The display pipe is responsible for "scanning out" a rendered frame from the
+GPU memory (also called VRAM, FrameBuffer, etc.) to a display. In other words,
+it would:
+
+1. Read frame information from memory;
+2. Perform required transformation;
+3. Send pixel data to sink devices.
+
+If you want to learn more about our driver details, take a look at the below
+table of content:
+
+.. toctree::
+
+ display-manager.rst
+ dc-debug.rst
+ dcn-overview.rst
+ dc-glossary.rst
diff --git a/Documentation/gpu/amdgpu/driver-core.rst b/Documentation/gpu/amdgpu/driver-core.rst
new file mode 100644
index 0000000..ebf5932
--- /dev/null
+++ b/Documentation/gpu/amdgpu/driver-core.rst
@@ -0,0 +1,182 @@
+============================
+ Core Driver Infrastructure
+============================
+
+GPU Hardware Structure
+======================
+
+Each ASIC is a collection of hardware blocks. We refer to them as
+"IPs" (Intellectual Property blocks). Each IP encapsulates certain
+functionality. IPs are versioned and can also be mixed and matched.
+E.g., you might have two different ASICs that both have System DMA (SDMA) 5.x IPs.
+The driver is arranged by IPs. There are driver components to handle
+the initialization and operation of each IP. There are also a bunch
+of smaller IPs that don't really need much if any driver interaction.
+Those end up getting lumped into the common stuff in the soc files.
+The soc files (e.g., vi.c, soc15.c nv.c) contain code for aspects of
+the SoC itself rather than specific IPs. E.g., things like GPU resets
+and register access functions are SoC dependent.
+
+An APU contains more than just CPU and GPU, it also contains all of
+the platform stuff (audio, usb, gpio, etc.). Also, a lot of
+components are shared between the CPU, platform, and the GPU (e.g.,
+SMU, PSP, etc.). Specific components (CPU, GPU, etc.) usually have
+their interface to interact with those common components. For things
+like S0i3 there is a ton of coordination required across all the
+components, but that is probably a bit beyond the scope of this
+section.
+
+With respect to the GPU, we have the following major IPs:
+
+GMC (Graphics Memory Controller)
+ This was a dedicated IP on older pre-vega chips, but has since
+ become somewhat decentralized on vega and newer chips. They now
+ have dedicated memory hubs for specific IPs or groups of IPs. We
+ still treat it as a single component in the driver however since
+ the programming model is still pretty similar. This is how the
+ different IPs on the GPU get the memory (VRAM or system memory).
+ It also provides the support for per process GPU virtual address
+ spaces.
+
+IH (Interrupt Handler)
+ This is the interrupt controller on the GPU. All of the IPs feed
+ their interrupts into this IP and it aggregates them into a set of
+ ring buffers that the driver can parse to handle interrupts from
+ different IPs.
+
+PSP (Platform Security Processor)
+ This handles security policy for the SoC and executes trusted
+ applications, and validates and loads firmwares for other blocks.
+
+SMU (System Management Unit)
+ This is the power management microcontroller. It manages the entire
+ SoC. The driver interacts with it to control power management
+ features like clocks, voltages, power rails, etc.
+
+DCN (Display Controller Next)
+ This is the display controller. It handles the display hardware.
+ It is described in more details in :ref:`Display Core <amdgpu-display-core>`.
+
+SDMA (System DMA)
+ This is a multi-purpose DMA engine. The kernel driver uses it for
+ various things including paging and GPU page table updates. It's also
+ exposed to userspace for use by user mode drivers (OpenGL, Vulkan,
+ etc.)
+
+GC (Graphics and Compute)
+ This is the graphics and compute engine, i.e., the block that
+ encompasses the 3D pipeline and and shader blocks. This is by far the
+ largest block on the GPU. The 3D pipeline has tons of sub-blocks. In
+ addition to that, it also contains the CP microcontrollers (ME, PFP,
+ CE, MEC) and the RLC microcontroller. It's exposed to userspace for
+ user mode drivers (OpenGL, Vulkan, OpenCL, etc.)
+
+VCN (Video Core Next)
+ This is the multi-media engine. It handles video and image encode and
+ decode. It's exposed to userspace for user mode drivers (VA-API,
+ OpenMAX, etc.)
+
+Graphics and Compute Microcontrollers
+-------------------------------------
+
+CP (Command Processor)
+ The name for the hardware block that encompasses the front end of the
+ GFX/Compute pipeline. Consists mainly of a bunch of microcontrollers
+ (PFP, ME, CE, MEC). The firmware that runs on these microcontrollers
+ provides the driver interface to interact with the GFX/Compute engine.
+
+ MEC (MicroEngine Compute)
+ This is the microcontroller that controls the compute queues on the
+ GFX/compute engine.
+
+ MES (MicroEngine Scheduler)
+ This is a new engine for managing queues. This is currently unused.
+
+RLC (RunList Controller)
+ This is another microcontroller in the GFX/Compute engine. It handles
+ power management related functionality within the GFX/Compute engine.
+ The name is a vestige of old hardware where it was originally added
+ and doesn't really have much relation to what the engine does now.
+
+Driver Structure
+================
+
+In general, the driver has a list of all of the IPs on a particular
+SoC and for things like init/fini/suspend/resume, more or less just
+walks the list and handles each IP.
+
+Some useful constructs:
+
+KIQ (Kernel Interface Queue)
+ This is a control queue used by the kernel driver to manage other gfx
+ and compute queues on the GFX/compute engine. You can use it to
+ map/unmap additional queues, etc.
+
+IB (Indirect Buffer)
+ A command buffer for a particular engine. Rather than writing
+ commands directly to the queue, you can write the commands into a
+ piece of memory and then put a pointer to the memory into the queue.
+ The hardware will then follow the pointer and execute the commands in
+ the memory, then returning to the rest of the commands in the ring.
+
+.. _amdgpu_memory_domains:
+
+Memory Domains
+==============
+
+.. kernel-doc:: include/uapi/drm/amdgpu_drm.h
+ :doc: memory domains
+
+Buffer Objects
+==============
+
+.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
+ :doc: amdgpu_object
+
+.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
+ :internal:
+
+PRIME Buffer Sharing
+====================
+
+.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
+ :doc: PRIME Buffer Sharing
+
+.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
+ :internal:
+
+MMU Notifier
+============
+
+.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
+ :doc: MMU Notifier
+
+.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
+ :internal:
+
+AMDGPU Virtual Memory
+=====================
+
+.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+ :doc: GPUVM
+
+.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+ :internal:
+
+Interrupt Handling
+==================
+
+.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
+ :doc: Interrupt Handling
+
+.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
+ :internal:
+
+IP Blocks
+=========
+
+.. kernel-doc:: drivers/gpu/drm/amd/include/amd_shared.h
+ :doc: IP Blocks
+
+.. kernel-doc:: drivers/gpu/drm/amd/include/amd_shared.h
+ :identifiers: amd_ip_block_type amd_ip_funcs
diff --git a/Documentation/gpu/amdgpu/driver-misc.rst b/Documentation/gpu/amdgpu/driver-misc.rst
new file mode 100644
index 0000000..e3d6b2f
--- /dev/null
+++ b/Documentation/gpu/amdgpu/driver-misc.rst
@@ -0,0 +1,112 @@
+================================
+ Misc AMDGPU driver information
+================================
+
+GPU Product Information
+=======================
+
+Information about the GPU can be obtained on certain cards
+via sysfs
+
+product_name
+------------
+
+.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+ :doc: product_name
+
+product_number
+--------------
+
+.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+ :doc: product_name
+
+serial_number
+-------------
+
+.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+ :doc: serial_number
+
+unique_id
+---------
+
+.. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c
+ :doc: unique_id
+
+GPU Memory Usage Information
+============================
+
+Various memory accounting can be accessed via sysfs
+
+mem_info_vram_total
+-------------------
+
+.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c
+ :doc: mem_info_vram_total
+
+mem_info_vram_used
+------------------
+
+.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c
+ :doc: mem_info_vram_used
+
+mem_info_vis_vram_total
+-----------------------
+
+.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c
+ :doc: mem_info_vis_vram_total
+
+mem_info_vis_vram_used
+----------------------
+
+.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c
+ :doc: mem_info_vis_vram_used
+
+mem_info_gtt_total
+------------------
+
+.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c
+ :doc: mem_info_gtt_total
+
+mem_info_gtt_used
+-----------------
+
+.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c
+ :doc: mem_info_gtt_used
+
+PCIe Accounting Information
+===========================
+
+pcie_bw
+-------
+
+.. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c
+ :doc: pcie_bw
+
+pcie_replay_count
+-----------------
+
+.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+ :doc: pcie_replay_count
+
+GPU SmartShift Information
+==========================
+
+GPU SmartShift information via sysfs
+
+smartshift_apu_power
+--------------------
+
+.. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c
+ :doc: smartshift_apu_power
+
+smartshift_dgpu_power
+---------------------
+
+.. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c
+ :doc: smartshift_dgpu_power
+
+smartshift_bias
+---------------
+
+.. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c
+ :doc: smartshift_bias
diff --git a/Documentation/gpu/amdgpu/index.rst b/Documentation/gpu/amdgpu/index.rst
new file mode 100644
index 0000000..a24e1cf
--- /dev/null
+++ b/Documentation/gpu/amdgpu/index.rst
@@ -0,0 +1,17 @@
+==========================
+ drm/amdgpu AMDgpu driver
+==========================
+
+The drm/amdgpu driver supports all AMD Radeon GPUs based on the Graphics Core
+Next (GCN) architecture.
+
+.. toctree::
+
+ module-parameters
+ driver-core
+ display/index
+ xgmi
+ ras
+ thermal
+ driver-misc
+ amdgpu-glossary
diff --git a/Documentation/gpu/amdgpu/module-parameters.rst b/Documentation/gpu/amdgpu/module-parameters.rst
new file mode 100644
index 0000000..ea538c8
--- /dev/null
+++ b/Documentation/gpu/amdgpu/module-parameters.rst
@@ -0,0 +1,7 @@
+===================
+ Module Parameters
+===================
+
+The amdgpu driver supports the following module parameters:
+
+.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
diff --git a/Documentation/gpu/amdgpu/ras.rst b/Documentation/gpu/amdgpu/ras.rst
new file mode 100644
index 0000000..047f76e
--- /dev/null
+++ b/Documentation/gpu/amdgpu/ras.rst
@@ -0,0 +1,62 @@
+====================
+ AMDGPU RAS Support
+====================
+
+The AMDGPU RAS interfaces are exposed via sysfs (for informational queries) and
+debugfs (for error injection).
+
+RAS debugfs/sysfs Control and Error Injection Interfaces
+========================================================
+
+.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
+ :doc: AMDGPU RAS debugfs control interface
+
+RAS Reboot Behavior for Unrecoverable Errors
+============================================
+
+.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
+ :doc: AMDGPU RAS Reboot Behavior for Unrecoverable Errors
+
+RAS Error Count sysfs Interface
+===============================
+
+.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
+ :doc: AMDGPU RAS sysfs Error Count Interface
+
+RAS EEPROM debugfs Interface
+============================
+
+.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
+ :doc: AMDGPU RAS debugfs EEPROM table reset interface
+
+RAS VRAM Bad Pages sysfs Interface
+==================================
+
+.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
+ :doc: AMDGPU RAS sysfs gpu_vram_bad_pages Interface
+
+Sample Code
+===========
+Sample code for testing error injection can be found here:
+https://cgit.freedesktop.org/mesa/drm/tree/tests/amdgpu/ras_tests.c
+
+This is part of the libdrm amdgpu unit tests which cover several areas of the GPU.
+There are four sets of tests:
+
+RAS Basic Test
+
+The test verifies the RAS feature enabled status and makes sure the necessary sysfs and debugfs files
+are present.
+
+RAS Query Test
+
+This test checks the RAS availability and enablement status for each supported IP block as well as
+the error counts.
+
+RAS Inject Test
+
+This test injects errors for each IP.
+
+RAS Disable Test
+
+This test tests disabling of RAS features for each IP block.
diff --git a/Documentation/gpu/amdgpu/thermal.rst b/Documentation/gpu/amdgpu/thermal.rst
new file mode 100644
index 0000000..8aeb018
--- /dev/null
+++ b/Documentation/gpu/amdgpu/thermal.rst
@@ -0,0 +1,65 @@
+===========================================
+ GPU Power/Thermal Controls and Monitoring
+===========================================
+
+HWMON Interfaces
+================
+
+.. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c
+ :doc: hwmon
+
+GPU sysfs Power State Interfaces
+================================
+
+GPU power controls are exposed via sysfs files.
+
+power_dpm_state
+---------------
+
+.. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c
+ :doc: power_dpm_state
+
+power_dpm_force_performance_level
+---------------------------------
+
+.. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c
+ :doc: power_dpm_force_performance_level
+
+pp_table
+--------
+
+.. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c
+ :doc: pp_table
+
+pp_od_clk_voltage
+-----------------
+
+.. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c
+ :doc: pp_od_clk_voltage
+
+pp_dpm_*
+--------
+
+.. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c
+ :doc: pp_dpm_sclk pp_dpm_mclk pp_dpm_socclk pp_dpm_fclk pp_dpm_dcefclk pp_dpm_pcie
+
+pp_power_profile_mode
+---------------------
+
+.. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c
+ :doc: pp_power_profile_mode
+
+\*_busy_percent
+---------------
+
+.. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c
+ :doc: gpu_busy_percent
+
+.. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c
+ :doc: mem_busy_percent
+
+gpu_metrics
+-----------
+
+.. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c
+ :doc: gpu_metrics
diff --git a/Documentation/gpu/amdgpu/xgmi.rst b/Documentation/gpu/amdgpu/xgmi.rst
new file mode 100644
index 0000000..23f2856
--- /dev/null
+++ b/Documentation/gpu/amdgpu/xgmi.rst
@@ -0,0 +1,5 @@
+=====================
+ AMDGPU XGMI Support
+=====================
+
+.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_xgmi.c
diff --git a/Documentation/gpu/drivers.rst b/Documentation/gpu/drivers.rst
index b4a0ed3..3a52f48 100644
--- a/Documentation/gpu/drivers.rst
+++ b/Documentation/gpu/drivers.rst
@@ -4,8 +4,7 @@
.. toctree::
- amdgpu
- amdgpu-dc
+ amdgpu/index
i915
mcde
meson
diff --git a/Documentation/gpu/drm-kms-helpers.rst b/Documentation/gpu/drm-kms-helpers.rst
index ec2f65b..5bb55ec 100644
--- a/Documentation/gpu/drm-kms-helpers.rst
+++ b/Documentation/gpu/drm-kms-helpers.rst
@@ -435,3 +435,18 @@
.. kernel-doc:: drivers/gpu/drm/drm_crtc_helper.c
:export:
+
+Privacy-screen class
+====================
+
+.. kernel-doc:: drivers/gpu/drm/drm_privacy_screen.c
+ :doc: overview
+
+.. kernel-doc:: include/drm/drm_privacy_screen_driver.h
+ :internal:
+
+.. kernel-doc:: include/drm/drm_privacy_screen_machine.h
+ :internal:
+
+.. kernel-doc:: drivers/gpu/drm/drm_privacy_screen.c
+ :export:
diff --git a/Documentation/gpu/drm-kms.rst b/Documentation/gpu/drm-kms.rst
index 1ef7951..d14bf1c 100644
--- a/Documentation/gpu/drm-kms.rst
+++ b/Documentation/gpu/drm-kms.rst
@@ -506,6 +506,8 @@
.. kernel-doc:: drivers/gpu/drm/drm_property.c
:export:
+.. _standard_connector_properties:
+
Standard Connector Properties
-----------------------------
diff --git a/Documentation/gpu/todo.rst b/Documentation/gpu/todo.rst
index 60d1d7e..da138dd 100644
--- a/Documentation/gpu/todo.rst
+++ b/Documentation/gpu/todo.rst
@@ -268,17 +268,6 @@
Level: Intermediate
-Clean up mmap forwarding
-------------------------
-
-A lot of drivers forward gem mmap calls to dma-buf mmap for imported buffers.
-And also a lot of them forward dma-buf mmap to the gem mmap implementations.
-There's drm_gem_prime_mmap() for this now, but still needs to be rolled out.
-
-Contact: Daniel Vetter
-
-Level: Intermediate
-
Generic fbdev defio support
---------------------------
@@ -463,6 +452,21 @@
Level: Intermediate
+Review all drivers for setting struct drm_mode_config.{max_width,max_height} correctly
+--------------------------------------------------------------------------------------
+
+The values in struct drm_mode_config.{max_width,max_height} describe the
+maximum supported framebuffer size. It's the virtual screen size, but many
+drivers treat it like limitations of the physical resolution.
+
+The maximum width depends on the hardware's maximum scanline pitch. The
+maximum height depends on the amount of addressable video memory. Review all
+drivers to initialize the fields to the correct values.
+
+Contact: Thomas Zimmermann <tzimmermann@suse.de>
+
+Level: Intermediate
+
Core refactorings
=================
@@ -642,6 +646,17 @@
Contact: Harry Wentland, Alex Deucher
+vmwgfx: Replace hashtable with Linux' implementation
+----------------------------------------------------
+
+The vmwgfx driver uses its own hashtable implementation. Replace the
+code with Linux' implementation and update the callers. It's mostly a
+refactoring task, but the interfaces are different.
+
+Contact: Zack Rusin, Thomas Zimmermann <tzimmermann@suse.de>
+
+Level: Intermediate
+
Bootsplash
==========
diff --git a/Documentation/i2c/summary.rst b/Documentation/i2c/summary.rst
index 136c4e3..786c618 100644
--- a/Documentation/i2c/summary.rst
+++ b/Documentation/i2c/summary.rst
@@ -11,9 +11,11 @@
and so are not advertised as being I2C but come under different names,
e.g. TWI (Two Wire Interface), IIC.
-The official I2C specification is the `"I2C-bus specification and user
-manual" (UM10204) <https://www.nxp.com/docs/en/user-guide/UM10204.pdf>`_
-published by NXP Semiconductors.
+The latest official I2C specification is the `"I2C-bus specification and user
+manual" (UM10204) <https://www.nxp.com/webapp/Download?colCode=UM10204>`_
+published by NXP Semiconductors. However, you need to log-in to the site to
+access the PDF. An older version of the specification (revision 6) is archived
+`here <https://web.archive.org/web/20210813122132/https://www.nxp.com/docs/en/user-guide/UM10204.pdf>`_.
SMBus (System Management Bus) is based on the I2C protocol, and is mostly
a subset of I2C protocols and signaling. Many I2C devices will work on an
diff --git a/Documentation/memory-barriers.txt b/Documentation/memory-barriers.txt
index 7367ada..b12df91 100644
--- a/Documentation/memory-barriers.txt
+++ b/Documentation/memory-barriers.txt
@@ -1950,6 +1950,14 @@
For load from persistent memory, existing read memory barriers are sufficient
to ensure read ordering.
+ (*) io_stop_wc();
+
+ For memory accesses with write-combining attributes (e.g. those returned
+ by ioremap_wc(), the CPU may wait for prior accesses to be merged with
+ subsequent ones. io_stop_wc() can be used to prevent the merging of
+ write-combining memory accesses before this macro with those after it when
+ such wait has performance implications.
+
===============================
IMPLICIT KERNEL MEMORY BARRIERS
===============================
diff --git a/Documentation/networking/bonding.rst b/Documentation/networking/bonding.rst
index c0a789b..ab98373 100644
--- a/Documentation/networking/bonding.rst
+++ b/Documentation/networking/bonding.rst
@@ -422,6 +422,17 @@
consider the slave up only when all of the arp_ip_targets
are reachable
+arp_missed_max
+
+ Specifies the number of arp_interval monitor checks that must
+ fail in order for an interface to be marked down by the ARP monitor.
+
+ In order to provide orderly failover semantics, backup interfaces
+ are permitted an extra monitor check (i.e., they must fail
+ arp_missed_max + 1 times before being marked down).
+
+ The default value is 2, and the allowable range is 1 - 255.
+
downdelay
Specifies the time, in milliseconds, to wait before disabling
diff --git a/Documentation/networking/device_drivers/can/freescale/flexcan.rst b/Documentation/networking/device_drivers/can/freescale/flexcan.rst
new file mode 100644
index 0000000..4e3eec6
--- /dev/null
+++ b/Documentation/networking/device_drivers/can/freescale/flexcan.rst
@@ -0,0 +1,54 @@
+.. SPDX-License-Identifier: GPL-2.0+
+
+=============================
+Flexcan CAN Controller driver
+=============================
+
+Authors: Marc Kleine-Budde <mkl@pengutronix.de>,
+Dario Binacchi <dario.binacchi@amarula.solutions.com>
+
+On/off RTR frames reception
+===========================
+
+For most flexcan IP cores the driver supports 2 RX modes:
+
+- FIFO
+- mailbox
+
+The older flexcan cores (integrated into the i.MX25, i.MX28, i.MX35
+and i.MX53 SOCs) only receive RTR frames if the controller is
+configured for RX-FIFO mode.
+
+The RX FIFO mode uses a hardware FIFO with a depth of 6 CAN frames,
+while the mailbox mode uses a software FIFO with a depth of up to 62
+CAN frames. With the help of the bigger buffer, the mailbox mode
+performs better under high system load situations.
+
+As reception of RTR frames is part of the CAN standard, all flexcan
+cores come up in a mode where RTR reception is possible.
+
+With the "rx-rtr" private flag the ability to receive RTR frames can
+be waived at the expense of losing the ability to receive RTR
+messages. This trade off is beneficial in certain use cases.
+
+"rx-rtr" on
+ Receive RTR frames. (default)
+
+ The CAN controller can and will receive RTR frames.
+
+ On some IP cores the controller cannot receive RTR frames in the
+ more performant "RX mailbox" mode and will use "RX FIFO" mode
+ instead.
+
+"rx-rtr" off
+
+ Waive ability to receive RTR frames. (not supported on all IP cores)
+
+ This mode activates the "RX mailbox mode" for better performance, on
+ some IP cores RTR frames cannot be received anymore.
+
+The setting can only be changed if the interface is down::
+
+ ip link set dev can0 down
+ ethtool --set-priv-flags can0 rx-rtr {off|on}
+ ip link set dev can0 up
diff --git a/Documentation/networking/device_drivers/can/index.rst b/Documentation/networking/device_drivers/can/index.rst
new file mode 100644
index 0000000..58b6e0a
--- /dev/null
+++ b/Documentation/networking/device_drivers/can/index.rst
@@ -0,0 +1,20 @@
+.. SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+
+Controller Area Network (CAN) Device Drivers
+============================================
+
+Device drivers for CAN devices.
+
+Contents:
+
+.. toctree::
+ :maxdepth: 2
+
+ freescale/flexcan
+
+.. only:: subproject and html
+
+ Indices
+ =======
+
+ * :ref:`genindex`
diff --git a/Documentation/networking/device_drivers/ethernet/amazon/ena.rst b/Documentation/networking/device_drivers/ethernet/amazon/ena.rst
index 01b2a69..8bcb173 100644
--- a/Documentation/networking/device_drivers/ethernet/amazon/ena.rst
+++ b/Documentation/networking/device_drivers/ethernet/amazon/ena.rst
@@ -135,7 +135,7 @@
- **Low Latency Queue (LLQ) mode or "push-mode":**
In this mode the driver pushes the transmit descriptors and the
- first 128 bytes of the packet directly to the ENA device memory
+ first 96 bytes of the packet directly to the ENA device memory
space. The rest of the packet payload is fetched by the
device. For this operation mode, the driver uses a dedicated PCI
device memory BAR, which is mapped with write-combine capability.
diff --git a/Documentation/networking/device_drivers/index.rst b/Documentation/networking/device_drivers/index.rst
index 3a5a1d4..5f5cfdb 100644
--- a/Documentation/networking/device_drivers/index.rst
+++ b/Documentation/networking/device_drivers/index.rst
@@ -11,6 +11,7 @@
appletalk/index
atm/index
cable/index
+ can/index
cellular/index
ethernet/index
fddi/index
diff --git a/Documentation/networking/devlink/devlink-params.rst b/Documentation/networking/devlink/devlink-params.rst
index 4878907..4e01dc3 100644
--- a/Documentation/networking/devlink/devlink-params.rst
+++ b/Documentation/networking/devlink/devlink-params.rst
@@ -109,14 +109,19 @@
- Boolean
- When enabled, the device driver will instantiate VDPA networking
specific auxiliary device of the devlink device.
+ * - ``enable_iwarp``
+ - Boolean
+ - Enable handling of iWARP traffic in the device.
* - ``internal_err_reset``
- Boolean
- When enabled, the device driver will reset the device on internal
errors.
* - ``max_macs``
- u32
- - Specifies the maximum number of MAC addresses per ethernet port of
- this device.
+ - Typically macvlan, vlan net devices mac are also programmed in their
+ parent netdevice's Function rx filter. This parameter limit the
+ maximum number of unicast mac address filters to receive traffic from
+ per ethernet port of this device.
* - ``region_snapshot_enable``
- Boolean
- Enable capture of ``devlink-region`` snapshots.
@@ -126,3 +131,9 @@
will NACK any attempt of other host to reset the device. This parameter
is useful for setups where a device is shared by different hosts, such
as multi-host setup.
+ * - ``io_eq_size``
+ - u32
+ - Control the size of I/O completion EQs.
+ * - ``event_eq_size``
+ - u32
+ - Control the size of asynchronous control events EQ.
diff --git a/Documentation/networking/devlink/ice.rst b/Documentation/networking/devlink/ice.rst
index 59c78e9..8c082b1 100644
--- a/Documentation/networking/devlink/ice.rst
+++ b/Documentation/networking/devlink/ice.rst
@@ -26,8 +26,10 @@
* - ``fw.mgmt``
- running
- 2.1.7
- - 3-digit version number of the management firmware that controls the
- PHY, link, etc.
+ - 3-digit version number of the management firmware running on the
+ Embedded Management Processor of the device. It controls the PHY,
+ link, access to device resources, etc. Intel documentation refers to
+ this as the EMP firmware.
* - ``fw.mgmt.api``
- running
- 1.5.1
@@ -119,6 +121,24 @@
own will be rejected. If no overwrite mask is provided, the firmware will be
instructed to preserve all settings and identifying fields when updating.
+Reload
+======
+
+The ``ice`` driver supports activating new firmware after a flash update
+using ``DEVLINK_CMD_RELOAD`` with the ``DEVLINK_RELOAD_ACTION_FW_ACTIVATE``
+action.
+
+.. code:: shell
+
+ $ devlink dev reload pci/0000:01:00.0 reload action fw_activate
+
+The new firmware is activated by issuing a device specific Embedded
+Management Processor reset which requests the device to reset and reload the
+EMP firmware image.
+
+The driver does not currently support reloading the driver via
+``DEVLINK_RELOAD_ACTION_DRIVER_REINIT``.
+
Regions
=======
diff --git a/Documentation/networking/devlink/mlx5.rst b/Documentation/networking/devlink/mlx5.rst
index 4e4b97f..29ad304 100644
--- a/Documentation/networking/devlink/mlx5.rst
+++ b/Documentation/networking/devlink/mlx5.rst
@@ -14,8 +14,19 @@
* - Name
- Mode
+ - Validation
* - ``enable_roce``
- driverinit
+ - Type: Boolean
+ * - ``io_eq_size``
+ - driverinit
+ - The range is between 64 and 4096.
+ * - ``event_eq_size``
+ - driverinit
+ - The range is between 64 and 4096.
+ * - ``max_macs``
+ - driverinit
+ - The range is between 1 and 2^31. Only power of 2 values are supported.
The ``mlx5`` driver also implements the following driver-specific
parameters.
diff --git a/Documentation/networking/ethtool-netlink.rst b/Documentation/networking/ethtool-netlink.rst
index 7b598c7..9d98e05 100644
--- a/Documentation/networking/ethtool-netlink.rst
+++ b/Documentation/networking/ethtool-netlink.rst
@@ -849,7 +849,7 @@
Kernel response contents:
- ==================================== ====== ==========================
+ ==================================== ====== ===========================
``ETHTOOL_A_RINGS_HEADER`` nested reply header
``ETHTOOL_A_RINGS_RX_MAX`` u32 max size of RX ring
``ETHTOOL_A_RINGS_RX_MINI_MAX`` u32 max size of RX mini ring
@@ -859,7 +859,8 @@
``ETHTOOL_A_RINGS_RX_MINI`` u32 size of RX mini ring
``ETHTOOL_A_RINGS_RX_JUMBO`` u32 size of RX jumbo ring
``ETHTOOL_A_RINGS_TX`` u32 size of TX ring
- ==================================== ====== ==========================
+ ``ETHTOOL_A_RINGS_RX_BUF_LEN`` u32 size of buffers on the ring
+ ==================================== ====== ===========================
RINGS_SET
@@ -869,13 +870,14 @@
Request contents:
- ==================================== ====== ==========================
+ ==================================== ====== ===========================
``ETHTOOL_A_RINGS_HEADER`` nested reply header
``ETHTOOL_A_RINGS_RX`` u32 size of RX ring
``ETHTOOL_A_RINGS_RX_MINI`` u32 size of RX mini ring
``ETHTOOL_A_RINGS_RX_JUMBO`` u32 size of RX jumbo ring
``ETHTOOL_A_RINGS_TX`` u32 size of TX ring
- ==================================== ====== ==========================
+ ``ETHTOOL_A_RINGS_RX_BUF_LEN`` u32 size of buffers on the ring
+ ==================================== ====== ===========================
Kernel checks that requested ring sizes do not exceed limits reported by
driver. Driver may impose additional constraints and may not suspport all
diff --git a/Documentation/networking/filter.rst b/Documentation/networking/filter.rst
index ce2b8e8..43cdc4d 100644
--- a/Documentation/networking/filter.rst
+++ b/Documentation/networking/filter.rst
@@ -6,6 +6,13 @@
Linux Socket Filtering aka Berkeley Packet Filter (BPF)
=======================================================
+Notice
+------
+
+This file used to document the eBPF format and mechanisms even when not
+related to socket filtering. The ../bpf/index.rst has more details
+on eBPF.
+
Introduction
------------
@@ -617,15 +624,11 @@
paragraphs is being used. However, the instruction set format is modelled
closer to the underlying architecture to mimic native instruction sets, so
that a better performance can be achieved (more details later). This new
-ISA is called 'eBPF' or 'internal BPF' interchangeably. (Note: eBPF which
+ISA is called eBPF. See the ../bpf/index.rst for details. (Note: eBPF which
originates from [e]xtended BPF is not the same as BPF extensions! While
eBPF is an ISA, BPF extensions date back to classic BPF's 'overloading'
of BPF_LD | BPF_{B,H,W} | BPF_ABS instruction.)
-It is designed to be JITed with one to one mapping, which can also open up
-the possibility for GCC/LLVM compilers to generate optimized eBPF code through
-an eBPF backend that performs almost as fast as natively compiled code.
-
The new instruction set was originally designed with the possible goal in
mind to write programs in "restricted C" and compile into eBPF with a optional
GCC/LLVM backend, so that it can just-in-time map to modern 64-bit CPUs with
@@ -650,1032 +653,11 @@
sparc64, arm32, riscv64, riscv32 perform JIT compilation from eBPF
instruction set.
-Some core changes of the new internal format:
-
-- Number of registers increase from 2 to 10:
-
- The old format had two registers A and X, and a hidden frame pointer. The
- new layout extends this to be 10 internal registers and a read-only frame
- pointer. Since 64-bit CPUs are passing arguments to functions via registers
- the number of args from eBPF program to in-kernel function is restricted
- to 5 and one register is used to accept return value from an in-kernel
- function. Natively, x86_64 passes first 6 arguments in registers, aarch64/
- sparcv9/mips64 have 7 - 8 registers for arguments; x86_64 has 6 callee saved
- registers, and aarch64/sparcv9/mips64 have 11 or more callee saved registers.
-
- Therefore, eBPF calling convention is defined as:
-
- * R0 - return value from in-kernel function, and exit value for eBPF program
- * R1 - R5 - arguments from eBPF program to in-kernel function
- * R6 - R9 - callee saved registers that in-kernel function will preserve
- * R10 - read-only frame pointer to access stack
-
- Thus, all eBPF registers map one to one to HW registers on x86_64, aarch64,
- etc, and eBPF calling convention maps directly to ABIs used by the kernel on
- 64-bit architectures.
-
- On 32-bit architectures JIT may map programs that use only 32-bit arithmetic
- and may let more complex programs to be interpreted.
-
- R0 - R5 are scratch registers and eBPF program needs spill/fill them if
- necessary across calls. Note that there is only one eBPF program (== one
- eBPF main routine) and it cannot call other eBPF functions, it can only
- call predefined in-kernel functions, though.
-
-- Register width increases from 32-bit to 64-bit:
-
- Still, the semantics of the original 32-bit ALU operations are preserved
- via 32-bit subregisters. All eBPF registers are 64-bit with 32-bit lower
- subregisters that zero-extend into 64-bit if they are being written to.
- That behavior maps directly to x86_64 and arm64 subregister definition, but
- makes other JITs more difficult.
-
- 32-bit architectures run 64-bit internal BPF programs via interpreter.
- Their JITs may convert BPF programs that only use 32-bit subregisters into
- native instruction set and let the rest being interpreted.
-
- Operation is 64-bit, because on 64-bit architectures, pointers are also
- 64-bit wide, and we want to pass 64-bit values in/out of kernel functions,
- so 32-bit eBPF registers would otherwise require to define register-pair
- ABI, thus, there won't be able to use a direct eBPF register to HW register
- mapping and JIT would need to do combine/split/move operations for every
- register in and out of the function, which is complex, bug prone and slow.
- Another reason is the use of atomic 64-bit counters.
-
-- Conditional jt/jf targets replaced with jt/fall-through:
-
- While the original design has constructs such as ``if (cond) jump_true;
- else jump_false;``, they are being replaced into alternative constructs like
- ``if (cond) jump_true; /* else fall-through */``.
-
-- Introduces bpf_call insn and register passing convention for zero overhead
- calls from/to other kernel functions:
-
- Before an in-kernel function call, the internal BPF program needs to
- place function arguments into R1 to R5 registers to satisfy calling
- convention, then the interpreter will take them from registers and pass
- to in-kernel function. If R1 - R5 registers are mapped to CPU registers
- that are used for argument passing on given architecture, the JIT compiler
- doesn't need to emit extra moves. Function arguments will be in the correct
- registers and BPF_CALL instruction will be JITed as single 'call' HW
- instruction. This calling convention was picked to cover common call
- situations without performance penalty.
-
- After an in-kernel function call, R1 - R5 are reset to unreadable and R0 has
- a return value of the function. Since R6 - R9 are callee saved, their state
- is preserved across the call.
-
- For example, consider three C functions::
-
- u64 f1() { return (*_f2)(1); }
- u64 f2(u64 a) { return f3(a + 1, a); }
- u64 f3(u64 a, u64 b) { return a - b; }
-
- GCC can compile f1, f3 into x86_64::
-
- f1:
- movl $1, %edi
- movq _f2(%rip), %rax
- jmp *%rax
- f3:
- movq %rdi, %rax
- subq %rsi, %rax
- ret
-
- Function f2 in eBPF may look like::
-
- f2:
- bpf_mov R2, R1
- bpf_add R1, 1
- bpf_call f3
- bpf_exit
-
- If f2 is JITed and the pointer stored to ``_f2``. The calls f1 -> f2 -> f3 and
- returns will be seamless. Without JIT, __bpf_prog_run() interpreter needs to
- be used to call into f2.
-
- For practical reasons all eBPF programs have only one argument 'ctx' which is
- already placed into R1 (e.g. on __bpf_prog_run() startup) and the programs
- can call kernel functions with up to 5 arguments. Calls with 6 or more arguments
- are currently not supported, but these restrictions can be lifted if necessary
- in the future.
-
- On 64-bit architectures all register map to HW registers one to one. For
- example, x86_64 JIT compiler can map them as ...
-
- ::
-
- R0 - rax
- R1 - rdi
- R2 - rsi
- R3 - rdx
- R4 - rcx
- R5 - r8
- R6 - rbx
- R7 - r13
- R8 - r14
- R9 - r15
- R10 - rbp
-
- ... since x86_64 ABI mandates rdi, rsi, rdx, rcx, r8, r9 for argument passing
- and rbx, r12 - r15 are callee saved.
-
- Then the following internal BPF pseudo-program::
-
- bpf_mov R6, R1 /* save ctx */
- bpf_mov R2, 2
- bpf_mov R3, 3
- bpf_mov R4, 4
- bpf_mov R5, 5
- bpf_call foo
- bpf_mov R7, R0 /* save foo() return value */
- bpf_mov R1, R6 /* restore ctx for next call */
- bpf_mov R2, 6
- bpf_mov R3, 7
- bpf_mov R4, 8
- bpf_mov R5, 9
- bpf_call bar
- bpf_add R0, R7
- bpf_exit
-
- After JIT to x86_64 may look like::
-
- push %rbp
- mov %rsp,%rbp
- sub $0x228,%rsp
- mov %rbx,-0x228(%rbp)
- mov %r13,-0x220(%rbp)
- mov %rdi,%rbx
- mov $0x2,%esi
- mov $0x3,%edx
- mov $0x4,%ecx
- mov $0x5,%r8d
- callq foo
- mov %rax,%r13
- mov %rbx,%rdi
- mov $0x6,%esi
- mov $0x7,%edx
- mov $0x8,%ecx
- mov $0x9,%r8d
- callq bar
- add %r13,%rax
- mov -0x228(%rbp),%rbx
- mov -0x220(%rbp),%r13
- leaveq
- retq
-
- Which is in this example equivalent in C to::
-
- u64 bpf_filter(u64 ctx)
- {
- return foo(ctx, 2, 3, 4, 5) + bar(ctx, 6, 7, 8, 9);
- }
-
- In-kernel functions foo() and bar() with prototype: u64 (*)(u64 arg1, u64
- arg2, u64 arg3, u64 arg4, u64 arg5); will receive arguments in proper
- registers and place their return value into ``%rax`` which is R0 in eBPF.
- Prologue and epilogue are emitted by JIT and are implicit in the
- interpreter. R0-R5 are scratch registers, so eBPF program needs to preserve
- them across the calls as defined by calling convention.
-
- For example the following program is invalid::
-
- bpf_mov R1, 1
- bpf_call foo
- bpf_mov R0, R1
- bpf_exit
-
- After the call the registers R1-R5 contain junk values and cannot be read.
- An in-kernel eBPF verifier is used to validate internal BPF programs.
-
-Also in the new design, eBPF is limited to 4096 insns, which means that any
-program will terminate quickly and will only call a fixed number of kernel
-functions. Original BPF and the new format are two operand instructions,
-which helps to do one-to-one mapping between eBPF insn and x86 insn during JIT.
-
-The input context pointer for invoking the interpreter function is generic,
-its content is defined by a specific use case. For seccomp register R1 points
-to seccomp_data, for converted BPF filters R1 points to a skb.
-
-A program, that is translated internally consists of the following elements::
-
- op:16, jt:8, jf:8, k:32 ==> op:8, dst_reg:4, src_reg:4, off:16, imm:32
-
-So far 87 internal BPF instructions were implemented. 8-bit 'op' opcode field
-has room for new instructions. Some of them may use 16/24/32 byte encoding. New
-instructions must be multiple of 8 bytes to preserve backward compatibility.
-
-Internal BPF is a general purpose RISC instruction set. Not every register and
-every instruction are used during translation from original BPF to new format.
-For example, socket filters are not using ``exclusive add`` instruction, but
-tracing filters may do to maintain counters of events, for example. Register R9
-is not used by socket filters either, but more complex filters may be running
-out of registers and would have to resort to spill/fill to stack.
-
-Internal BPF can be used as a generic assembler for last step performance
-optimizations, socket filters and seccomp are using it as assembler. Tracing
-filters may use it as assembler to generate code from kernel. In kernel usage
-may not be bounded by security considerations, since generated internal BPF code
-may be optimizing internal code path and not being exposed to the user space.
-Safety of internal BPF can come from a verifier (TBD). In such use cases as
-described, it may be used as safe instruction set.
-
-Just like the original BPF, the new format runs within a controlled environment,
-is deterministic and the kernel can easily prove that. The safety of the program
-can be determined in two steps: first step does depth-first-search to disallow
-loops and other CFG validation; second step starts from the first insn and
-descends all possible paths. It simulates execution of every insn and observes
-the state change of registers and stack.
-
-eBPF opcode encoding
---------------------
-
-eBPF is reusing most of the opcode encoding from classic to simplify conversion
-of classic BPF to eBPF. For arithmetic and jump instructions the 8-bit 'code'
-field is divided into three parts::
-
- +----------------+--------+--------------------+
- | 4 bits | 1 bit | 3 bits |
- | operation code | source | instruction class |
- +----------------+--------+--------------------+
- (MSB) (LSB)
-
-Three LSB bits store instruction class which is one of:
-
- =================== ===============
- Classic BPF classes eBPF classes
- =================== ===============
- BPF_LD 0x00 BPF_LD 0x00
- BPF_LDX 0x01 BPF_LDX 0x01
- BPF_ST 0x02 BPF_ST 0x02
- BPF_STX 0x03 BPF_STX 0x03
- BPF_ALU 0x04 BPF_ALU 0x04
- BPF_JMP 0x05 BPF_JMP 0x05
- BPF_RET 0x06 BPF_JMP32 0x06
- BPF_MISC 0x07 BPF_ALU64 0x07
- =================== ===============
-
-When BPF_CLASS(code) == BPF_ALU or BPF_JMP, 4th bit encodes source operand ...
-
- ::
-
- BPF_K 0x00
- BPF_X 0x08
-
- * in classic BPF, this means::
-
- BPF_SRC(code) == BPF_X - use register X as source operand
- BPF_SRC(code) == BPF_K - use 32-bit immediate as source operand
-
- * in eBPF, this means::
-
- BPF_SRC(code) == BPF_X - use 'src_reg' register as source operand
- BPF_SRC(code) == BPF_K - use 32-bit immediate as source operand
-
-... and four MSB bits store operation code.
-
-If BPF_CLASS(code) == BPF_ALU or BPF_ALU64 [ in eBPF ], BPF_OP(code) is one of::
-
- BPF_ADD 0x00
- BPF_SUB 0x10
- BPF_MUL 0x20
- BPF_DIV 0x30
- BPF_OR 0x40
- BPF_AND 0x50
- BPF_LSH 0x60
- BPF_RSH 0x70
- BPF_NEG 0x80
- BPF_MOD 0x90
- BPF_XOR 0xa0
- BPF_MOV 0xb0 /* eBPF only: mov reg to reg */
- BPF_ARSH 0xc0 /* eBPF only: sign extending shift right */
- BPF_END 0xd0 /* eBPF only: endianness conversion */
-
-If BPF_CLASS(code) == BPF_JMP or BPF_JMP32 [ in eBPF ], BPF_OP(code) is one of::
-
- BPF_JA 0x00 /* BPF_JMP only */
- BPF_JEQ 0x10
- BPF_JGT 0x20
- BPF_JGE 0x30
- BPF_JSET 0x40
- BPF_JNE 0x50 /* eBPF only: jump != */
- BPF_JSGT 0x60 /* eBPF only: signed '>' */
- BPF_JSGE 0x70 /* eBPF only: signed '>=' */
- BPF_CALL 0x80 /* eBPF BPF_JMP only: function call */
- BPF_EXIT 0x90 /* eBPF BPF_JMP only: function return */
- BPF_JLT 0xa0 /* eBPF only: unsigned '<' */
- BPF_JLE 0xb0 /* eBPF only: unsigned '<=' */
- BPF_JSLT 0xc0 /* eBPF only: signed '<' */
- BPF_JSLE 0xd0 /* eBPF only: signed '<=' */
-
-So BPF_ADD | BPF_X | BPF_ALU means 32-bit addition in both classic BPF
-and eBPF. There are only two registers in classic BPF, so it means A += X.
-In eBPF it means dst_reg = (u32) dst_reg + (u32) src_reg; similarly,
-BPF_XOR | BPF_K | BPF_ALU means A ^= imm32 in classic BPF and analogous
-src_reg = (u32) src_reg ^ (u32) imm32 in eBPF.
-
-Classic BPF is using BPF_MISC class to represent A = X and X = A moves.
-eBPF is using BPF_MOV | BPF_X | BPF_ALU code instead. Since there are no
-BPF_MISC operations in eBPF, the class 7 is used as BPF_ALU64 to mean
-exactly the same operations as BPF_ALU, but with 64-bit wide operands
-instead. So BPF_ADD | BPF_X | BPF_ALU64 means 64-bit addition, i.e.:
-dst_reg = dst_reg + src_reg
-
-Classic BPF wastes the whole BPF_RET class to represent a single ``ret``
-operation. Classic BPF_RET | BPF_K means copy imm32 into return register
-and perform function exit. eBPF is modeled to match CPU, so BPF_JMP | BPF_EXIT
-in eBPF means function exit only. The eBPF program needs to store return
-value into register R0 before doing a BPF_EXIT. Class 6 in eBPF is used as
-BPF_JMP32 to mean exactly the same operations as BPF_JMP, but with 32-bit wide
-operands for the comparisons instead.
-
-For load and store instructions the 8-bit 'code' field is divided as::
-
- +--------+--------+-------------------+
- | 3 bits | 2 bits | 3 bits |
- | mode | size | instruction class |
- +--------+--------+-------------------+
- (MSB) (LSB)
-
-Size modifier is one of ...
-
-::
-
- BPF_W 0x00 /* word */
- BPF_H 0x08 /* half word */
- BPF_B 0x10 /* byte */
- BPF_DW 0x18 /* eBPF only, double word */
-
-... which encodes size of load/store operation::
-
- B - 1 byte
- H - 2 byte
- W - 4 byte
- DW - 8 byte (eBPF only)
-
-Mode modifier is one of::
-
- BPF_IMM 0x00 /* used for 32-bit mov in classic BPF and 64-bit in eBPF */
- BPF_ABS 0x20
- BPF_IND 0x40
- BPF_MEM 0x60
- BPF_LEN 0x80 /* classic BPF only, reserved in eBPF */
- BPF_MSH 0xa0 /* classic BPF only, reserved in eBPF */
- BPF_ATOMIC 0xc0 /* eBPF only, atomic operations */
-
-eBPF has two non-generic instructions: (BPF_ABS | <size> | BPF_LD) and
-(BPF_IND | <size> | BPF_LD) which are used to access packet data.
-
-They had to be carried over from classic to have strong performance of
-socket filters running in eBPF interpreter. These instructions can only
-be used when interpreter context is a pointer to ``struct sk_buff`` and
-have seven implicit operands. Register R6 is an implicit input that must
-contain pointer to sk_buff. Register R0 is an implicit output which contains
-the data fetched from the packet. Registers R1-R5 are scratch registers
-and must not be used to store the data across BPF_ABS | BPF_LD or
-BPF_IND | BPF_LD instructions.
-
-These instructions have implicit program exit condition as well. When
-eBPF program is trying to access the data beyond the packet boundary,
-the interpreter will abort the execution of the program. JIT compilers
-therefore must preserve this property. src_reg and imm32 fields are
-explicit inputs to these instructions.
-
-For example::
-
- BPF_IND | BPF_W | BPF_LD means:
-
- R0 = ntohl(*(u32 *) (((struct sk_buff *) R6)->data + src_reg + imm32))
- and R1 - R5 were scratched.
-
-Unlike classic BPF instruction set, eBPF has generic load/store operations::
-
- BPF_MEM | <size> | BPF_STX: *(size *) (dst_reg + off) = src_reg
- BPF_MEM | <size> | BPF_ST: *(size *) (dst_reg + off) = imm32
- BPF_MEM | <size> | BPF_LDX: dst_reg = *(size *) (src_reg + off)
-
-Where size is one of: BPF_B or BPF_H or BPF_W or BPF_DW.
-
-It also includes atomic operations, which use the immediate field for extra
-encoding::
-
- .imm = BPF_ADD, .code = BPF_ATOMIC | BPF_W | BPF_STX: lock xadd *(u32 *)(dst_reg + off16) += src_reg
- .imm = BPF_ADD, .code = BPF_ATOMIC | BPF_DW | BPF_STX: lock xadd *(u64 *)(dst_reg + off16) += src_reg
-
-The basic atomic operations supported are::
-
- BPF_ADD
- BPF_AND
- BPF_OR
- BPF_XOR
-
-Each having equivalent semantics with the ``BPF_ADD`` example, that is: the
-memory location addresed by ``dst_reg + off`` is atomically modified, with
-``src_reg`` as the other operand. If the ``BPF_FETCH`` flag is set in the
-immediate, then these operations also overwrite ``src_reg`` with the
-value that was in memory before it was modified.
-
-The more special operations are::
-
- BPF_XCHG
-
-This atomically exchanges ``src_reg`` with the value addressed by ``dst_reg +
-off``. ::
-
- BPF_CMPXCHG
-
-This atomically compares the value addressed by ``dst_reg + off`` with
-``R0``. If they match it is replaced with ``src_reg``. In either case, the
-value that was there before is zero-extended and loaded back to ``R0``.
-
-Note that 1 and 2 byte atomic operations are not supported.
-
-Clang can generate atomic instructions by default when ``-mcpu=v3`` is
-enabled. If a lower version for ``-mcpu`` is set, the only atomic instruction
-Clang can generate is ``BPF_ADD`` *without* ``BPF_FETCH``. If you need to enable
-the atomics features, while keeping a lower ``-mcpu`` version, you can use
-``-Xclang -target-feature -Xclang +alu32``.
-
-You may encounter ``BPF_XADD`` - this is a legacy name for ``BPF_ATOMIC``,
-referring to the exclusive-add operation encoded when the immediate field is
-zero.
-
-eBPF has one 16-byte instruction: ``BPF_LD | BPF_DW | BPF_IMM`` which consists
-of two consecutive ``struct bpf_insn`` 8-byte blocks and interpreted as single
-instruction that loads 64-bit immediate value into a dst_reg.
-Classic BPF has similar instruction: ``BPF_LD | BPF_W | BPF_IMM`` which loads
-32-bit immediate value into a register.
-
-eBPF verifier
--------------
-The safety of the eBPF program is determined in two steps.
-
-First step does DAG check to disallow loops and other CFG validation.
-In particular it will detect programs that have unreachable instructions.
-(though classic BPF checker allows them)
-
-Second step starts from the first insn and descends all possible paths.
-It simulates execution of every insn and observes the state change of
-registers and stack.
-
-At the start of the program the register R1 contains a pointer to context
-and has type PTR_TO_CTX.
-If verifier sees an insn that does R2=R1, then R2 has now type
-PTR_TO_CTX as well and can be used on the right hand side of expression.
-If R1=PTR_TO_CTX and insn is R2=R1+R1, then R2=SCALAR_VALUE,
-since addition of two valid pointers makes invalid pointer.
-(In 'secure' mode verifier will reject any type of pointer arithmetic to make
-sure that kernel addresses don't leak to unprivileged users)
-
-If register was never written to, it's not readable::
-
- bpf_mov R0 = R2
- bpf_exit
-
-will be rejected, since R2 is unreadable at the start of the program.
-
-After kernel function call, R1-R5 are reset to unreadable and
-R0 has a return type of the function.
-
-Since R6-R9 are callee saved, their state is preserved across the call.
-
-::
-
- bpf_mov R6 = 1
- bpf_call foo
- bpf_mov R0 = R6
- bpf_exit
-
-is a correct program. If there was R1 instead of R6, it would have
-been rejected.
-
-load/store instructions are allowed only with registers of valid types, which
-are PTR_TO_CTX, PTR_TO_MAP, PTR_TO_STACK. They are bounds and alignment checked.
-For example::
-
- bpf_mov R1 = 1
- bpf_mov R2 = 2
- bpf_xadd *(u32 *)(R1 + 3) += R2
- bpf_exit
-
-will be rejected, since R1 doesn't have a valid pointer type at the time of
-execution of instruction bpf_xadd.
-
-At the start R1 type is PTR_TO_CTX (a pointer to generic ``struct bpf_context``)
-A callback is used to customize verifier to restrict eBPF program access to only
-certain fields within ctx structure with specified size and alignment.
-
-For example, the following insn::
-
- bpf_ld R0 = *(u32 *)(R6 + 8)
-
-intends to load a word from address R6 + 8 and store it into R0
-If R6=PTR_TO_CTX, via is_valid_access() callback the verifier will know
-that offset 8 of size 4 bytes can be accessed for reading, otherwise
-the verifier will reject the program.
-If R6=PTR_TO_STACK, then access should be aligned and be within
-stack bounds, which are [-MAX_BPF_STACK, 0). In this example offset is 8,
-so it will fail verification, since it's out of bounds.
-
-The verifier will allow eBPF program to read data from stack only after
-it wrote into it.
-
-Classic BPF verifier does similar check with M[0-15] memory slots.
-For example::
-
- bpf_ld R0 = *(u32 *)(R10 - 4)
- bpf_exit
-
-is invalid program.
-Though R10 is correct read-only register and has type PTR_TO_STACK
-and R10 - 4 is within stack bounds, there were no stores into that location.
-
-Pointer register spill/fill is tracked as well, since four (R6-R9)
-callee saved registers may not be enough for some programs.
-
-Allowed function calls are customized with bpf_verifier_ops->get_func_proto()
-The eBPF verifier will check that registers match argument constraints.
-After the call register R0 will be set to return type of the function.
-
-Function calls is a main mechanism to extend functionality of eBPF programs.
-Socket filters may let programs to call one set of functions, whereas tracing
-filters may allow completely different set.
-
-If a function made accessible to eBPF program, it needs to be thought through
-from safety point of view. The verifier will guarantee that the function is
-called with valid arguments.
-
-seccomp vs socket filters have different security restrictions for classic BPF.
-Seccomp solves this by two stage verifier: classic BPF verifier is followed
-by seccomp verifier. In case of eBPF one configurable verifier is shared for
-all use cases.
-
-See details of eBPF verifier in kernel/bpf/verifier.c
-
-Register value tracking
------------------------
-In order to determine the safety of an eBPF program, the verifier must track
-the range of possible values in each register and also in each stack slot.
-This is done with ``struct bpf_reg_state``, defined in include/linux/
-bpf_verifier.h, which unifies tracking of scalar and pointer values. Each
-register state has a type, which is either NOT_INIT (the register has not been
-written to), SCALAR_VALUE (some value which is not usable as a pointer), or a
-pointer type. The types of pointers describe their base, as follows:
-
-
- PTR_TO_CTX
- Pointer to bpf_context.
- CONST_PTR_TO_MAP
- Pointer to struct bpf_map. "Const" because arithmetic
- on these pointers is forbidden.
- PTR_TO_MAP_VALUE
- Pointer to the value stored in a map element.
- PTR_TO_MAP_VALUE_OR_NULL
- Either a pointer to a map value, or NULL; map accesses
- (see section 'eBPF maps', below) return this type,
- which becomes a PTR_TO_MAP_VALUE when checked != NULL.
- Arithmetic on these pointers is forbidden.
- PTR_TO_STACK
- Frame pointer.
- PTR_TO_PACKET
- skb->data.
- PTR_TO_PACKET_END
- skb->data + headlen; arithmetic forbidden.
- PTR_TO_SOCKET
- Pointer to struct bpf_sock_ops, implicitly refcounted.
- PTR_TO_SOCKET_OR_NULL
- Either a pointer to a socket, or NULL; socket lookup
- returns this type, which becomes a PTR_TO_SOCKET when
- checked != NULL. PTR_TO_SOCKET is reference-counted,
- so programs must release the reference through the
- socket release function before the end of the program.
- Arithmetic on these pointers is forbidden.
-
-However, a pointer may be offset from this base (as a result of pointer
-arithmetic), and this is tracked in two parts: the 'fixed offset' and 'variable
-offset'. The former is used when an exactly-known value (e.g. an immediate
-operand) is added to a pointer, while the latter is used for values which are
-not exactly known. The variable offset is also used in SCALAR_VALUEs, to track
-the range of possible values in the register.
-
-The verifier's knowledge about the variable offset consists of:
-
-* minimum and maximum values as unsigned
-* minimum and maximum values as signed
-
-* knowledge of the values of individual bits, in the form of a 'tnum': a u64
- 'mask' and a u64 'value'. 1s in the mask represent bits whose value is unknown;
- 1s in the value represent bits known to be 1. Bits known to be 0 have 0 in both
- mask and value; no bit should ever be 1 in both. For example, if a byte is read
- into a register from memory, the register's top 56 bits are known zero, while
- the low 8 are unknown - which is represented as the tnum (0x0; 0xff). If we
- then OR this with 0x40, we get (0x40; 0xbf), then if we add 1 we get (0x0;
- 0x1ff), because of potential carries.
-
-Besides arithmetic, the register state can also be updated by conditional
-branches. For instance, if a SCALAR_VALUE is compared > 8, in the 'true' branch
-it will have a umin_value (unsigned minimum value) of 9, whereas in the 'false'
-branch it will have a umax_value of 8. A signed compare (with BPF_JSGT or
-BPF_JSGE) would instead update the signed minimum/maximum values. Information
-from the signed and unsigned bounds can be combined; for instance if a value is
-first tested < 8 and then tested s> 4, the verifier will conclude that the value
-is also > 4 and s< 8, since the bounds prevent crossing the sign boundary.
-
-PTR_TO_PACKETs with a variable offset part have an 'id', which is common to all
-pointers sharing that same variable offset. This is important for packet range
-checks: after adding a variable to a packet pointer register A, if you then copy
-it to another register B and then add a constant 4 to A, both registers will
-share the same 'id' but the A will have a fixed offset of +4. Then if A is
-bounds-checked and found to be less than a PTR_TO_PACKET_END, the register B is
-now known to have a safe range of at least 4 bytes. See 'Direct packet access',
-below, for more on PTR_TO_PACKET ranges.
-
-The 'id' field is also used on PTR_TO_MAP_VALUE_OR_NULL, common to all copies of
-the pointer returned from a map lookup. This means that when one copy is
-checked and found to be non-NULL, all copies can become PTR_TO_MAP_VALUEs.
-As well as range-checking, the tracked information is also used for enforcing
-alignment of pointer accesses. For instance, on most systems the packet pointer
-is 2 bytes after a 4-byte alignment. If a program adds 14 bytes to that to jump
-over the Ethernet header, then reads IHL and addes (IHL * 4), the resulting
-pointer will have a variable offset known to be 4n+2 for some n, so adding the 2
-bytes (NET_IP_ALIGN) gives a 4-byte alignment and so word-sized accesses through
-that pointer are safe.
-The 'id' field is also used on PTR_TO_SOCKET and PTR_TO_SOCKET_OR_NULL, common
-to all copies of the pointer returned from a socket lookup. This has similar
-behaviour to the handling for PTR_TO_MAP_VALUE_OR_NULL->PTR_TO_MAP_VALUE, but
-it also handles reference tracking for the pointer. PTR_TO_SOCKET implicitly
-represents a reference to the corresponding ``struct sock``. To ensure that the
-reference is not leaked, it is imperative to NULL-check the reference and in
-the non-NULL case, and pass the valid reference to the socket release function.
-
-Direct packet access
---------------------
-In cls_bpf and act_bpf programs the verifier allows direct access to the packet
-data via skb->data and skb->data_end pointers.
-Ex::
-
- 1: r4 = *(u32 *)(r1 +80) /* load skb->data_end */
- 2: r3 = *(u32 *)(r1 +76) /* load skb->data */
- 3: r5 = r3
- 4: r5 += 14
- 5: if r5 > r4 goto pc+16
- R1=ctx R3=pkt(id=0,off=0,r=14) R4=pkt_end R5=pkt(id=0,off=14,r=14) R10=fp
- 6: r0 = *(u16 *)(r3 +12) /* access 12 and 13 bytes of the packet */
-
-this 2byte load from the packet is safe to do, since the program author
-did check ``if (skb->data + 14 > skb->data_end) goto err`` at insn #5 which
-means that in the fall-through case the register R3 (which points to skb->data)
-has at least 14 directly accessible bytes. The verifier marks it
-as R3=pkt(id=0,off=0,r=14).
-id=0 means that no additional variables were added to the register.
-off=0 means that no additional constants were added.
-r=14 is the range of safe access which means that bytes [R3, R3 + 14) are ok.
-Note that R5 is marked as R5=pkt(id=0,off=14,r=14). It also points
-to the packet data, but constant 14 was added to the register, so
-it now points to ``skb->data + 14`` and accessible range is [R5, R5 + 14 - 14)
-which is zero bytes.
-
-More complex packet access may look like::
-
-
- R0=inv1 R1=ctx R3=pkt(id=0,off=0,r=14) R4=pkt_end R5=pkt(id=0,off=14,r=14) R10=fp
- 6: r0 = *(u8 *)(r3 +7) /* load 7th byte from the packet */
- 7: r4 = *(u8 *)(r3 +12)
- 8: r4 *= 14
- 9: r3 = *(u32 *)(r1 +76) /* load skb->data */
- 10: r3 += r4
- 11: r2 = r1
- 12: r2 <<= 48
- 13: r2 >>= 48
- 14: r3 += r2
- 15: r2 = r3
- 16: r2 += 8
- 17: r1 = *(u32 *)(r1 +80) /* load skb->data_end */
- 18: if r2 > r1 goto pc+2
- R0=inv(id=0,umax_value=255,var_off=(0x0; 0xff)) R1=pkt_end R2=pkt(id=2,off=8,r=8) R3=pkt(id=2,off=0,r=8) R4=inv(id=0,umax_value=3570,var_off=(0x0; 0xfffe)) R5=pkt(id=0,off=14,r=14) R10=fp
- 19: r1 = *(u8 *)(r3 +4)
-
-The state of the register R3 is R3=pkt(id=2,off=0,r=8)
-id=2 means that two ``r3 += rX`` instructions were seen, so r3 points to some
-offset within a packet and since the program author did
-``if (r3 + 8 > r1) goto err`` at insn #18, the safe range is [R3, R3 + 8).
-The verifier only allows 'add'/'sub' operations on packet registers. Any other
-operation will set the register state to 'SCALAR_VALUE' and it won't be
-available for direct packet access.
-
-Operation ``r3 += rX`` may overflow and become less than original skb->data,
-therefore the verifier has to prevent that. So when it sees ``r3 += rX``
-instruction and rX is more than 16-bit value, any subsequent bounds-check of r3
-against skb->data_end will not give us 'range' information, so attempts to read
-through the pointer will give "invalid access to packet" error.
-
-Ex. after insn ``r4 = *(u8 *)(r3 +12)`` (insn #7 above) the state of r4 is
-R4=inv(id=0,umax_value=255,var_off=(0x0; 0xff)) which means that upper 56 bits
-of the register are guaranteed to be zero, and nothing is known about the lower
-8 bits. After insn ``r4 *= 14`` the state becomes
-R4=inv(id=0,umax_value=3570,var_off=(0x0; 0xfffe)), since multiplying an 8-bit
-value by constant 14 will keep upper 52 bits as zero, also the least significant
-bit will be zero as 14 is even. Similarly ``r2 >>= 48`` will make
-R2=inv(id=0,umax_value=65535,var_off=(0x0; 0xffff)), since the shift is not sign
-extending. This logic is implemented in adjust_reg_min_max_vals() function,
-which calls adjust_ptr_min_max_vals() for adding pointer to scalar (or vice
-versa) and adjust_scalar_min_max_vals() for operations on two scalars.
-
-The end result is that bpf program author can access packet directly
-using normal C code as::
-
- void *data = (void *)(long)skb->data;
- void *data_end = (void *)(long)skb->data_end;
- struct eth_hdr *eth = data;
- struct iphdr *iph = data + sizeof(*eth);
- struct udphdr *udp = data + sizeof(*eth) + sizeof(*iph);
-
- if (data + sizeof(*eth) + sizeof(*iph) + sizeof(*udp) > data_end)
- return 0;
- if (eth->h_proto != htons(ETH_P_IP))
- return 0;
- if (iph->protocol != IPPROTO_UDP || iph->ihl != 5)
- return 0;
- if (udp->dest == 53 || udp->source == 9)
- ...;
-
-which makes such programs easier to write comparing to LD_ABS insn
-and significantly faster.
-
-eBPF maps
----------
-'maps' is a generic storage of different types for sharing data between kernel
-and userspace.
-
-The maps are accessed from user space via BPF syscall, which has commands:
-
-- create a map with given type and attributes
- ``map_fd = bpf(BPF_MAP_CREATE, union bpf_attr *attr, u32 size)``
- using attr->map_type, attr->key_size, attr->value_size, attr->max_entries
- returns process-local file descriptor or negative error
-
-- lookup key in a given map
- ``err = bpf(BPF_MAP_LOOKUP_ELEM, union bpf_attr *attr, u32 size)``
- using attr->map_fd, attr->key, attr->value
- returns zero and stores found elem into value or negative error
-
-- create or update key/value pair in a given map
- ``err = bpf(BPF_MAP_UPDATE_ELEM, union bpf_attr *attr, u32 size)``
- using attr->map_fd, attr->key, attr->value
- returns zero or negative error
-
-- find and delete element by key in a given map
- ``err = bpf(BPF_MAP_DELETE_ELEM, union bpf_attr *attr, u32 size)``
- using attr->map_fd, attr->key
-
-- to delete map: close(fd)
- Exiting process will delete maps automatically
-
-userspace programs use this syscall to create/access maps that eBPF programs
-are concurrently updating.
-
-maps can have different types: hash, array, bloom filter, radix-tree, etc.
-
-The map is defined by:
-
- - type
- - max number of elements
- - key size in bytes
- - value size in bytes
-
-Pruning
--------
-The verifier does not actually walk all possible paths through the program. For
-each new branch to analyse, the verifier looks at all the states it's previously
-been in when at this instruction. If any of them contain the current state as a
-subset, the branch is 'pruned' - that is, the fact that the previous state was
-accepted implies the current state would be as well. For instance, if in the
-previous state, r1 held a packet-pointer, and in the current state, r1 holds a
-packet-pointer with a range as long or longer and at least as strict an
-alignment, then r1 is safe. Similarly, if r2 was NOT_INIT before then it can't
-have been used by any path from that point, so any value in r2 (including
-another NOT_INIT) is safe. The implementation is in the function regsafe().
-Pruning considers not only the registers but also the stack (and any spilled
-registers it may hold). They must all be safe for the branch to be pruned.
-This is implemented in states_equal().
-
-Understanding eBPF verifier messages
-------------------------------------
-
-The following are few examples of invalid eBPF programs and verifier error
-messages as seen in the log:
-
-Program with unreachable instructions::
-
- static struct bpf_insn prog[] = {
- BPF_EXIT_INSN(),
- BPF_EXIT_INSN(),
- };
-
-Error:
-
- unreachable insn 1
-
-Program that reads uninitialized register::
-
- BPF_MOV64_REG(BPF_REG_0, BPF_REG_2),
- BPF_EXIT_INSN(),
-
-Error::
-
- 0: (bf) r0 = r2
- R2 !read_ok
-
-Program that doesn't initialize R0 before exiting::
-
- BPF_MOV64_REG(BPF_REG_2, BPF_REG_1),
- BPF_EXIT_INSN(),
-
-Error::
-
- 0: (bf) r2 = r1
- 1: (95) exit
- R0 !read_ok
-
-Program that accesses stack out of bounds::
-
- BPF_ST_MEM(BPF_DW, BPF_REG_10, 8, 0),
- BPF_EXIT_INSN(),
-
-Error::
-
- 0: (7a) *(u64 *)(r10 +8) = 0
- invalid stack off=8 size=8
-
-Program that doesn't initialize stack before passing its address into function::
-
- BPF_MOV64_REG(BPF_REG_2, BPF_REG_10),
- BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -8),
- BPF_LD_MAP_FD(BPF_REG_1, 0),
- BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_map_lookup_elem),
- BPF_EXIT_INSN(),
-
-Error::
-
- 0: (bf) r2 = r10
- 1: (07) r2 += -8
- 2: (b7) r1 = 0x0
- 3: (85) call 1
- invalid indirect read from stack off -8+0 size 8
-
-Program that uses invalid map_fd=0 while calling to map_lookup_elem() function::
-
- BPF_ST_MEM(BPF_DW, BPF_REG_10, -8, 0),
- BPF_MOV64_REG(BPF_REG_2, BPF_REG_10),
- BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -8),
- BPF_LD_MAP_FD(BPF_REG_1, 0),
- BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_map_lookup_elem),
- BPF_EXIT_INSN(),
-
-Error::
-
- 0: (7a) *(u64 *)(r10 -8) = 0
- 1: (bf) r2 = r10
- 2: (07) r2 += -8
- 3: (b7) r1 = 0x0
- 4: (85) call 1
- fd 0 is not pointing to valid bpf_map
-
-Program that doesn't check return value of map_lookup_elem() before accessing
-map element::
-
- BPF_ST_MEM(BPF_DW, BPF_REG_10, -8, 0),
- BPF_MOV64_REG(BPF_REG_2, BPF_REG_10),
- BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -8),
- BPF_LD_MAP_FD(BPF_REG_1, 0),
- BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_map_lookup_elem),
- BPF_ST_MEM(BPF_DW, BPF_REG_0, 0, 0),
- BPF_EXIT_INSN(),
-
-Error::
-
- 0: (7a) *(u64 *)(r10 -8) = 0
- 1: (bf) r2 = r10
- 2: (07) r2 += -8
- 3: (b7) r1 = 0x0
- 4: (85) call 1
- 5: (7a) *(u64 *)(r0 +0) = 0
- R0 invalid mem access 'map_value_or_null'
-
-Program that correctly checks map_lookup_elem() returned value for NULL, but
-accesses the memory with incorrect alignment::
-
- BPF_ST_MEM(BPF_DW, BPF_REG_10, -8, 0),
- BPF_MOV64_REG(BPF_REG_2, BPF_REG_10),
- BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -8),
- BPF_LD_MAP_FD(BPF_REG_1, 0),
- BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_map_lookup_elem),
- BPF_JMP_IMM(BPF_JEQ, BPF_REG_0, 0, 1),
- BPF_ST_MEM(BPF_DW, BPF_REG_0, 4, 0),
- BPF_EXIT_INSN(),
-
-Error::
-
- 0: (7a) *(u64 *)(r10 -8) = 0
- 1: (bf) r2 = r10
- 2: (07) r2 += -8
- 3: (b7) r1 = 1
- 4: (85) call 1
- 5: (15) if r0 == 0x0 goto pc+1
- R0=map_ptr R10=fp
- 6: (7a) *(u64 *)(r0 +4) = 0
- misaligned access off 4 size 8
-
-Program that correctly checks map_lookup_elem() returned value for NULL and
-accesses memory with correct alignment in one side of 'if' branch, but fails
-to do so in the other side of 'if' branch::
-
- BPF_ST_MEM(BPF_DW, BPF_REG_10, -8, 0),
- BPF_MOV64_REG(BPF_REG_2, BPF_REG_10),
- BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -8),
- BPF_LD_MAP_FD(BPF_REG_1, 0),
- BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_map_lookup_elem),
- BPF_JMP_IMM(BPF_JEQ, BPF_REG_0, 0, 2),
- BPF_ST_MEM(BPF_DW, BPF_REG_0, 0, 0),
- BPF_EXIT_INSN(),
- BPF_ST_MEM(BPF_DW, BPF_REG_0, 0, 1),
- BPF_EXIT_INSN(),
-
-Error::
-
- 0: (7a) *(u64 *)(r10 -8) = 0
- 1: (bf) r2 = r10
- 2: (07) r2 += -8
- 3: (b7) r1 = 1
- 4: (85) call 1
- 5: (15) if r0 == 0x0 goto pc+2
- R0=map_ptr R10=fp
- 6: (7a) *(u64 *)(r0 +0) = 0
- 7: (95) exit
-
- from 5 to 8: R0=imm0 R10=fp
- 8: (7a) *(u64 *)(r0 +0) = 1
- R0 invalid mem access 'imm'
-
-Program that performs a socket lookup then sets the pointer to NULL without
-checking it::
-
- BPF_MOV64_IMM(BPF_REG_2, 0),
- BPF_STX_MEM(BPF_W, BPF_REG_10, BPF_REG_2, -8),
- BPF_MOV64_REG(BPF_REG_2, BPF_REG_10),
- BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -8),
- BPF_MOV64_IMM(BPF_REG_3, 4),
- BPF_MOV64_IMM(BPF_REG_4, 0),
- BPF_MOV64_IMM(BPF_REG_5, 0),
- BPF_EMIT_CALL(BPF_FUNC_sk_lookup_tcp),
- BPF_MOV64_IMM(BPF_REG_0, 0),
- BPF_EXIT_INSN(),
-
-Error::
-
- 0: (b7) r2 = 0
- 1: (63) *(u32 *)(r10 -8) = r2
- 2: (bf) r2 = r10
- 3: (07) r2 += -8
- 4: (b7) r3 = 4
- 5: (b7) r4 = 0
- 6: (b7) r5 = 0
- 7: (85) call bpf_sk_lookup_tcp#65
- 8: (b7) r0 = 0
- 9: (95) exit
- Unreleased reference id=1, alloc_insn=7
-
-Program that performs a socket lookup but does not NULL-check the returned
-value::
-
- BPF_MOV64_IMM(BPF_REG_2, 0),
- BPF_STX_MEM(BPF_W, BPF_REG_10, BPF_REG_2, -8),
- BPF_MOV64_REG(BPF_REG_2, BPF_REG_10),
- BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -8),
- BPF_MOV64_IMM(BPF_REG_3, 4),
- BPF_MOV64_IMM(BPF_REG_4, 0),
- BPF_MOV64_IMM(BPF_REG_5, 0),
- BPF_EMIT_CALL(BPF_FUNC_sk_lookup_tcp),
- BPF_EXIT_INSN(),
-
-Error::
-
- 0: (b7) r2 = 0
- 1: (63) *(u32 *)(r10 -8) = r2
- 2: (bf) r2 = r10
- 3: (07) r2 += -8
- 4: (b7) r3 = 4
- 5: (b7) r4 = 0
- 6: (b7) r5 = 0
- 7: (85) call bpf_sk_lookup_tcp#65
- 8: (95) exit
- Unreleased reference id=1, alloc_insn=7
-
Testing
-------
Next to the BPF toolchain, the kernel also ships a test module that contains
-various test cases for classic and internal BPF that can be executed against
+various test cases for classic and eBPF that can be executed against
the BPF interpreter and JIT compiler. It can be found in lib/test_bpf.c and
enabled via Kconfig::
diff --git a/Documentation/networking/ip-sysctl.rst b/Documentation/networking/ip-sysctl.rst
index c044311..2572eec 100644
--- a/Documentation/networking/ip-sysctl.rst
+++ b/Documentation/networking/ip-sysctl.rst
@@ -25,7 +25,8 @@
ip_no_pmtu_disc - INTEGER
Disable Path MTU Discovery. If enabled in mode 1 and a
fragmentation-required ICMP is received, the PMTU to this
- destination will be set to min_pmtu (see below). You will need
+ destination will be set to the smallest of the old MTU to
+ this destination and min_pmtu (see below). You will need
to raise min_pmtu to the smallest interface MTU on your system
manually if you want to avoid locally generated fragments.
@@ -49,7 +50,8 @@
Default: FALSE
min_pmtu - INTEGER
- default 552 - minimum discovered Path MTU
+ default 552 - minimum Path MTU. Unless this is changed mannually,
+ each cached pmtu will never be lower than this setting.
ip_forward_use_pmtu - BOOLEAN
By default we don't trust protocol path MTUs while forwarding
diff --git a/Documentation/networking/net_failover.rst b/Documentation/networking/net_failover.rst
index e143ab7..3a662f2 100644
--- a/Documentation/networking/net_failover.rst
+++ b/Documentation/networking/net_failover.rst
@@ -35,7 +35,7 @@
feature on the virtio-net interface and assign the same MAC address to both
virtio-net and VF interfaces.
-Here is an example XML snippet that shows such configuration.
+Here is an example libvirt XML snippet that shows such configuration:
::
<interface type='network'>
@@ -45,18 +45,32 @@
<model type='virtio'/>
<driver name='vhost' queues='4'/>
<link state='down'/>
- <address type='pci' domain='0x0000' bus='0x00' slot='0x0a' function='0x0'/>
+ <teaming type='persistent'/>
+ <alias name='ua-backup0'/>
</interface>
<interface type='hostdev' managed='yes'>
<mac address='52:54:00:00:12:53'/>
<source>
<address type='pci' domain='0x0000' bus='0x42' slot='0x02' function='0x5'/>
</source>
- <address type='pci' domain='0x0000' bus='0x00' slot='0x0b' function='0x0'/>
+ <teaming type='transient' persistent='ua-backup0'/>
</interface>
+In this configuration, the first device definition is for the virtio-net
+interface and this acts as the 'persistent' device indicating that this
+interface will always be plugged in. This is specified by the 'teaming' tag with
+required attribute type having value 'persistent'. The link state for the
+virtio-net device is set to 'down' to ensure that the 'failover' netdev prefers
+the VF passthrough device for normal communication. The virtio-net device will
+be brought UP during live migration to allow uninterrupted communication.
+
+The second device definition is for the VF passthrough interface. Here the
+'teaming' tag is provided with type 'transient' indicating that this device may
+periodically be unplugged. A second attribute - 'persistent' is provided and
+points to the alias name declared for the virtio-net device.
+
Booting a VM with the above configuration will result in the following 3
-netdevs created in the VM.
+interfaces created in the VM:
::
4: ens10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
@@ -65,13 +79,36 @@
valid_lft 42482sec preferred_lft 42482sec
inet6 fe80::97d8:db2:8c10:b6d6/64 scope link
valid_lft forever preferred_lft forever
- 5: ens10nsby: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master ens10 state UP group default qlen 1000
+ 5: ens10nsby: <BROADCAST,MULTICAST> mtu 1500 qdisc fq_codel master ens10 state DOWN group default qlen 1000
link/ether 52:54:00:00:12:53 brd ff:ff:ff:ff:ff:ff
7: ens11: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master ens10 state UP group default qlen 1000
link/ether 52:54:00:00:12:53 brd ff:ff:ff:ff:ff:ff
-ens10 is the 'failover' master netdev, ens10nsby and ens11 are the slave
-'standby' and 'primary' netdevs respectively.
+Here, ens10 is the 'failover' master interface, ens10nsby is the slave 'standby'
+virtio-net interface, and ens11 is the slave 'primary' VF passthrough interface.
+
+One point to note here is that some user space network configuration daemons
+like systemd-networkd, ifupdown, etc, do not understand the 'net_failover'
+device; and on the first boot, the VM might end up with both 'failover' device
+and VF accquiring IP addresses (either same or different) from the DHCP server.
+This will result in lack of connectivity to the VM. So some tweaks might be
+needed to these network configuration daemons to make sure that an IP is
+received only on the 'failover' device.
+
+Below is the patch snippet used with 'cloud-ifupdown-helper' script found on
+Debian cloud images:
+
+::
+ @@ -27,6 +27,8 @@ do_setup() {
+ local working="$cfgdir/.$INTERFACE"
+ local final="$cfgdir/$INTERFACE"
+
+ + if [ -d "/sys/class/net/${INTERFACE}/master" ]; then exit 0; fi
+ +
+ if ifup --no-act "$INTERFACE" > /dev/null 2>&1; then
+ # interface is already known to ifupdown, no need to generate cfg
+ log "Skipping configuration generation for $INTERFACE"
+
Live Migration of a VM with SR-IOV VF & virtio-net in STANDBY mode
==================================================================
@@ -80,40 +117,68 @@
with VMs that have direct attached SR-IOV VF devices by automatic failover to
the paravirtual datapath when the VF is unplugged.
-Here is a sample script that shows the steps to initiate live migration on
-the source hypervisor.
+Here is a sample script that shows the steps to initiate live migration from
+the source hypervisor. Note: It is assumed that the VM is connected to a
+software bridge 'br0' which has a single VF attached to it along with the vnet
+device to the VM. This is not the VF that was passthrough'd to the VM (seen in
+the vf.xml file).
::
- # cat vf_xml
+ # cat vf.xml
<interface type='hostdev' managed='yes'>
<mac address='52:54:00:00:12:53'/>
<source>
<address type='pci' domain='0x0000' bus='0x42' slot='0x02' function='0x5'/>
</source>
- <address type='pci' domain='0x0000' bus='0x00' slot='0x0b' function='0x0'/>
+ <teaming type='transient' persistent='ua-backup0'/>
</interface>
- # Source Hypervisor
+ # Source Hypervisor migrate.sh
#!/bin/bash
- DOMAIN=fedora27-tap01
- PF=enp66s0f0
- VF_NUM=5
- TAP_IF=tap01
- VF_XML=
+ DOMAIN=vm-01
+ PF=ens6np0
+ VF=ens6v1 # VF attached to the bridge.
+ VF_NUM=1
+ TAP_IF=vmtap01 # virtio-net interface in the VM.
+ VF_XML=vf.xml
MAC=52:54:00:00:12:53
ZERO_MAC=00:00:00:00:00:00
+ # Set the virtio-net interface up.
virsh domif-setlink $DOMAIN $TAP_IF up
- bridge fdb del $MAC dev $PF master
- virsh detach-device $DOMAIN $VF_XML
+
+ # Remove the VF that was passthrough'd to the VM.
+ virsh detach-device --live --config $DOMAIN $VF_XML
+
ip link set $PF vf $VF_NUM mac $ZERO_MAC
- virsh migrate --live $DOMAIN qemu+ssh://$REMOTE_HOST/system
+ # Add FDB entry for traffic to continue going to the VM via
+ # the VF -> br0 -> vnet interface path.
+ bridge fdb add $MAC dev $VF
+ bridge fdb add $MAC dev $TAP_IF master
- # Destination Hypervisor
+ # Migrate the VM
+ virsh migrate --live --persistent $DOMAIN qemu+ssh://$REMOTE_HOST/system
+
+ # Clean up FDB entries after migration completes.
+ bridge fdb del $MAC dev $VF
+ bridge fdb del $MAC dev $TAP_IF master
+
+On the destination hypervisor, a shared bridge 'br0' is created before migration
+starts, and a VF from the destination PF is added to the bridge. Similarly an
+appropriate FDB entry is added.
+
+The following script is executed on the destination hypervisor once migration
+completes, and it reattaches the VF to the VM and brings down the virtio-net
+interface.
+
+::
+ # reattach-vf.sh
#!/bin/bash
- virsh attach-device $DOMAIN $VF_XML
- virsh domif-setlink $DOMAIN $TAP_IF down
+ bridge fdb del 52:54:00:00:12:53 dev ens36v0
+ bridge fdb del 52:54:00:00:12:53 dev vmtap01 master
+ virsh attach-device --config --live vm01 vf.xml
+ virsh domif-setlink vm01 vmtap01 down
diff --git a/Documentation/networking/phy.rst b/Documentation/networking/phy.rst
index 571ba08..d43da70 100644
--- a/Documentation/networking/phy.rst
+++ b/Documentation/networking/phy.rst
@@ -237,6 +237,11 @@
Some of the interface modes are described below:
+``PHY_INTERFACE_MODE_SMII``
+ This is serial MII, clocked at 125MHz, supporting 100M and 10M speeds.
+ Some details can be found in
+ https://opencores.org/ocsvn/smii/smii/trunk/doc/SMII.pdf
+
``PHY_INTERFACE_MODE_1000BASEX``
This defines the 1000BASE-X single-lane serdes link as defined by the
802.3 standard section 36. The link operates at a fixed bit rate of
diff --git a/Documentation/staging/tee.rst b/Documentation/staging/tee.rst
index 4d4b5f8..3c63d8d 100644
--- a/Documentation/staging/tee.rst
+++ b/Documentation/staging/tee.rst
@@ -184,6 +184,36 @@
application to retrieve a list of Trusted Applications which can be registered
as devices on the TEE bus.
+OP-TEE notifications
+--------------------
+
+There are two kinds of notifications that secure world can use to make
+normal world aware of some event.
+
+1. Synchronous notifications delivered with ``OPTEE_RPC_CMD_NOTIFICATION``
+ using the ``OPTEE_RPC_NOTIFICATION_SEND`` parameter.
+2. Asynchronous notifications delivered with a combination of a non-secure
+ edge-triggered interrupt and a fast call from the non-secure interrupt
+ handler.
+
+Synchronous notifications are limited by depending on RPC for delivery,
+this is only usable when secure world is entered with a yielding call via
+``OPTEE_SMC_CALL_WITH_ARG``. This excludes such notifications from secure
+world interrupt handlers.
+
+An asynchronous notification is delivered via a non-secure edge-triggered
+interrupt to an interrupt handler registered in the OP-TEE driver. The
+actual notification value are retrieved with the fast call
+``OPTEE_SMC_GET_ASYNC_NOTIF_VALUE``. Note that one interrupt can represent
+multiple notifications.
+
+One notification value ``OPTEE_SMC_ASYNC_NOTIF_VALUE_DO_BOTTOM_HALF`` has a
+special meaning. When this value is received it means that normal world is
+supposed to make a yielding call ``OPTEE_MSG_CMD_DO_BOTTOM_HALF``. This
+call is done from the thread assisting the interrupt handler. This is a
+building block for OP-TEE OS in secure world to implement the top half and
+bottom half style of device drivers.
+
AMD-TEE driver
==============
diff --git a/Documentation/userspace-api/media/cec/cec-ioc-receive.rst b/Documentation/userspace-api/media/cec/cec-ioc-receive.rst
index bd7f7e7..364938a 100644
--- a/Documentation/userspace-api/media/cec/cec-ioc-receive.rst
+++ b/Documentation/userspace-api/media/cec/cec-ioc-receive.rst
@@ -48,9 +48,12 @@
A received message can be:
1. a message received from another CEC device (the ``sequence`` field will
- be 0).
-2. the result of an earlier non-blocking transmit (the ``sequence`` field will
- be non-zero).
+ be 0, ``tx_status`` will be 0 and ``rx_status`` will be non-zero).
+2. the transmit result of an earlier non-blocking transmit (the ``sequence``
+ field will be non-zero, ``tx_status`` will be non-zero and ``rx_status``
+ will be 0).
+3. the reply to an earlier non-blocking transmit (the ``sequence`` field will
+ be non-zero, ``tx_status`` will be 0 and ``rx_status`` will be non-zero).
To send a CEC message the application has to fill in the struct
:c:type:`cec_msg` and pass it to :ref:`ioctl CEC_TRANSMIT <CEC_TRANSMIT>`.
@@ -64,12 +67,11 @@
If the file descriptor is in non-blocking mode then the transmit will
return 0 and the result of the transmit will be available via
-:ref:`ioctl CEC_RECEIVE <CEC_RECEIVE>` once the transmit has finished
-(including waiting for a reply, if requested).
-
-The ``sequence`` field is filled in for every transmit and this can be
-checked against the received messages to find the corresponding transmit
-result.
+:ref:`ioctl CEC_RECEIVE <CEC_RECEIVE>` once the transmit has finished.
+If a non-blocking transmit also specified waiting for a reply, then
+the reply will arrive in a later message. The ``sequence`` field can
+be used to associate both transmit results and replies with the original
+transmit.
Normally calling :ref:`ioctl CEC_TRANSMIT <CEC_TRANSMIT>` when the physical
address is invalid (due to e.g. a disconnect) will return ``ENONET``.
@@ -123,18 +125,17 @@
- ``sequence``
- A non-zero sequence number is automatically assigned by the CEC framework
for all transmitted messages. It is used by the CEC framework when it queues
- the transmit result (when transmit was called in non-blocking mode). This
- allows the application to associate the received message with the original
- transmit.
+ the transmit result for a non-blocking transmit. This allows the application
+ to associate the received message with the original transmit.
+
+ In addition, if a non-blocking transmit will wait for a reply (ii.e. ``timeout``
+ was not 0), then the ``sequence`` field of the reply will be set to the sequence
+ value of the original transmit. This allows the application to associate the
+ received message with the original transmit.
* - __u32
- ``flags``
- Flags. See :ref:`cec-msg-flags` for a list of available flags.
* - __u8
- - ``tx_status``
- - The status bits of the transmitted message. See
- :ref:`cec-tx-status` for the possible status values. It is 0 if
- this message was received, not transmitted.
- * - __u8
- ``msg[16]``
- The message payload. For :ref:`ioctl CEC_TRANSMIT <CEC_TRANSMIT>` this is filled in by the
application. The driver will fill this in for :ref:`ioctl CEC_RECEIVE <CEC_RECEIVE>`.
@@ -162,15 +163,17 @@
* - __u8
- ``rx_status``
- The status bits of the received message. See
- :ref:`cec-rx-status` for the possible status values. It is 0 if
- this message was transmitted, not received, unless this is the
- reply to a transmitted message. In that case both ``rx_status``
- and ``tx_status`` are set.
+ :ref:`cec-rx-status` for the possible status values.
* - __u8
- ``tx_status``
- The status bits of the transmitted message. See
- :ref:`cec-tx-status` for the possible status values. It is 0 if
- this message was received, not transmitted.
+ :ref:`cec-tx-status` for the possible status values.
+ When calling :ref:`ioctl CEC_TRANSMIT <CEC_TRANSMIT>` in non-blocking mode,
+ this field will be 0 if the transmit started, or non-0 if the transmit
+ result is known immediately. The latter would be the case when attempting
+ to transmit a Poll message to yourself. That results in a
+ :ref:`CEC_TX_STATUS_NACK <CEC-TX-STATUS-NACK>` without ever actually
+ transmitting the Poll message.
* - __u8
- ``tx_arb_lost_cnt``
- A counter of the number of transmit attempts that resulted in the
diff --git a/Documentation/userspace-api/media/lirc.h.rst.exceptions b/Documentation/userspace-api/media/lirc.h.rst.exceptions
index e74b73c..ec86e82 100644
--- a/Documentation/userspace-api/media/lirc.h.rst.exceptions
+++ b/Documentation/userspace-api/media/lirc.h.rst.exceptions
@@ -33,6 +33,7 @@
# Obsolete ioctls
ignore ioctl LIRC_GET_LENGTH
+ignore ioctl LIRC_SET_REC_TIMEOUT_REPORTS
# rc protocols
@@ -73,6 +74,7 @@
ignore define LIRC_MODE2_SPACE
ignore define LIRC_MODE2_PULSE
+ignore define LIRC_MODE2_TIMEOUT
ignore define LIRC_VALUE_MASK
ignore define LIRC_MODE2_MASK
diff --git a/Documentation/userspace-api/media/rc/lirc-dev-intro.rst b/Documentation/userspace-api/media/rc/lirc-dev-intro.rst
index c889737..9a5e5f0 100644
--- a/Documentation/userspace-api/media/rc/lirc-dev-intro.rst
+++ b/Documentation/userspace-api/media/rc/lirc-dev-intro.rst
@@ -117,11 +117,9 @@
``LIRC_MODE2_TIMEOUT``
- If timeout reports are enabled with
- :ref:`lirc_set_rec_timeout_reports`, when the timeout set with
- :ref:`lirc_set_rec_timeout` expires due to no IR being detected,
- this packet will be sent, with the number of microseconds with
- no IR.
+ When the timeout set with :ref:`lirc_set_rec_timeout` expires due
+ to no IR being detected, this packet will be sent, with the number
+ of microseconds with no IR.
.. _lirc-mode-pulse:
diff --git a/Documentation/userspace-api/media/rc/lirc-func.rst b/Documentation/userspace-api/media/rc/lirc-func.rst
index 793f295..5c84888 100644
--- a/Documentation/userspace-api/media/rc/lirc-func.rst
+++ b/Documentation/userspace-api/media/rc/lirc-func.rst
@@ -22,6 +22,5 @@
lirc-set-rec-carrier-range
lirc-set-send-carrier
lirc-set-transmitter-mask
- lirc-set-rec-timeout-reports
lirc-set-measure-carrier-mode
lirc-set-wideband-receiver
diff --git a/Documentation/userspace-api/media/rc/lirc-set-rec-timeout-reports.rst b/Documentation/userspace-api/media/rc/lirc-set-rec-timeout-reports.rst
deleted file mode 100644
index aebe810..0000000
--- a/Documentation/userspace-api/media/rc/lirc-set-rec-timeout-reports.rst
+++ /dev/null
@@ -1,49 +0,0 @@
-.. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later
-.. c:namespace:: RC
-
-.. _lirc_set_rec_timeout_reports:
-
-**********************************
-ioctl LIRC_SET_REC_TIMEOUT_REPORTS
-**********************************
-
-Name
-====
-
-LIRC_SET_REC_TIMEOUT_REPORTS - enable or disable timeout reports for IR receive
-
-Synopsis
-========
-
-.. c:macro:: LIRC_SET_REC_TIMEOUT_REPORTS
-
-``int ioctl(int fd, LIRC_SET_REC_TIMEOUT_REPORTS, __u32 *enable)``
-
-Arguments
-=========
-
-``fd``
- File descriptor returned by open().
-
-``enable``
- enable = 1 means enable timeout report, enable = 0 means disable timeout
- reports.
-
-Description
-===========
-
-.. _lirc-mode2-timeout:
-
-Enable or disable timeout reports for IR receive. By default, timeout reports
-should be turned off.
-
-.. note::
-
- This ioctl is only valid for :ref:`LIRC_MODE_MODE2 <lirc-mode-mode2>`.
-
-Return Value
-============
-
-On success 0 is returned, on error -1 and the ``errno`` variable is set
-appropriately. The generic error codes are described at the
-:ref:`Generic Error Codes <gen-errors>` chapter.
diff --git a/Documentation/userspace-api/media/v4l/biblio.rst b/Documentation/userspace-api/media/v4l/biblio.rst
index 7b8e673..9cd18c1 100644
--- a/Documentation/userspace-api/media/v4l/biblio.rst
+++ b/Documentation/userspace-api/media/v4l/biblio.rst
@@ -417,3 +417,13 @@
:title: RFC 6386: "VP8 Data Format and Decoding Guide"
:author: J. Bankoski et al.
+
+.. _vp9:
+
+VP9
+===
+
+
+:title: VP9 Bitstream & Decoding Process Specification
+
+:author: Adrian Grange (Google), Peter de Rivaz (Argon Design), Jonathan Hunt (Argon Design)
diff --git a/Documentation/userspace-api/media/v4l/capture.c.rst b/Documentation/userspace-api/media/v4l/capture.c.rst
index ccbd52c..eef6772 100644
--- a/Documentation/userspace-api/media/v4l/capture.c.rst
+++ b/Documentation/userspace-api/media/v4l/capture.c.rst
@@ -56,7 +56,7 @@
static void errno_exit(const char *s)
{
- fprintf(stderr, "%s error %d, %s\\n", s, errno, strerror(errno));
+ fprintf(stderr, "%s error %d, %s\n", s, errno, strerror(errno));
exit(EXIT_FAILURE);
}
@@ -201,7 +201,7 @@
}
if (0 == r) {
- fprintf(stderr, "select timeout\\n");
+ fprintf(stderr, "select timeout\n");
exit(EXIT_FAILURE);
}
@@ -307,7 +307,7 @@
buffers = calloc(1, sizeof(*buffers));
if (!buffers) {
- fprintf(stderr, "Out of memory\\n");
+ fprintf(stderr, "Out of memory\n");
exit(EXIT_FAILURE);
}
@@ -315,7 +315,7 @@
buffers[0].start = malloc(buffer_size);
if (!buffers[0].start) {
- fprintf(stderr, "Out of memory\\n");
+ fprintf(stderr, "Out of memory\n");
exit(EXIT_FAILURE);
}
}
@@ -341,7 +341,7 @@
}
if (req.count < 2) {
- fprintf(stderr, "Insufficient buffer memory on %s\\n",
+ fprintf(stderr, "Insufficient buffer memory on %s\n",
dev_name);
exit(EXIT_FAILURE);
}
@@ -349,7 +349,7 @@
buffers = calloc(req.count, sizeof(*buffers));
if (!buffers) {
- fprintf(stderr, "Out of memory\\n");
+ fprintf(stderr, "Out of memory\n");
exit(EXIT_FAILURE);
}
@@ -401,7 +401,7 @@
buffers = calloc(4, sizeof(*buffers));
if (!buffers) {
- fprintf(stderr, "Out of memory\\n");
+ fprintf(stderr, "Out of memory\n");
exit(EXIT_FAILURE);
}
@@ -410,7 +410,7 @@
buffers[n_buffers].start = malloc(buffer_size);
if (!buffers[n_buffers].start) {
- fprintf(stderr, "Out of memory\\n");
+ fprintf(stderr, "Out of memory\n");
exit(EXIT_FAILURE);
}
}
@@ -426,7 +426,7 @@
if (-1 == xioctl(fd, VIDIOC_QUERYCAP, &cap)) {
if (EINVAL == errno) {
- fprintf(stderr, "%s is no V4L2 device\\n",
+ fprintf(stderr, "%s is no V4L2 device\n",
dev_name);
exit(EXIT_FAILURE);
} else {
@@ -435,7 +435,7 @@
}
if (!(cap.capabilities & V4L2_CAP_VIDEO_CAPTURE)) {
- fprintf(stderr, "%s is no video capture device\\n",
+ fprintf(stderr, "%s is no video capture device\n",
dev_name);
exit(EXIT_FAILURE);
}
@@ -443,7 +443,7 @@
switch (io) {
case IO_METHOD_READ:
if (!(cap.capabilities & V4L2_CAP_READWRITE)) {
- fprintf(stderr, "%s does not support read i/o\\n",
+ fprintf(stderr, "%s does not support read i/o\n",
dev_name);
exit(EXIT_FAILURE);
}
@@ -452,7 +452,7 @@
case IO_METHOD_MMAP:
case IO_METHOD_USERPTR:
if (!(cap.capabilities & V4L2_CAP_STREAMING)) {
- fprintf(stderr, "%s does not support streaming i/o\\n",
+ fprintf(stderr, "%s does not support streaming i/o\n",
dev_name);
exit(EXIT_FAILURE);
}
@@ -541,7 +541,7 @@
struct stat st;
if (-1 == stat(dev_name, &st)) {
- fprintf(stderr, "Cannot identify '%s': %d, %s\\n",
+ fprintf(stderr, "Cannot identify '%s': %d, %s\n",
dev_name, errno, strerror(errno));
exit(EXIT_FAILURE);
}
@@ -554,7 +554,7 @@
fd = open(dev_name, O_RDWR /* required */ | O_NONBLOCK, 0);
if (-1 == fd) {
- fprintf(stderr, "Cannot open '%s': %d, %s\\n",
+ fprintf(stderr, "Cannot open '%s': %d, %s\n",
dev_name, errno, strerror(errno));
exit(EXIT_FAILURE);
}
@@ -563,17 +563,17 @@
static void usage(FILE *fp, int argc, char **argv)
{
fprintf(fp,
- "Usage: %s [options]\\n\\n"
- "Version 1.3\\n"
- "Options:\\n"
- "-d | --device name Video device name [%s]n"
- "-h | --help Print this messagen"
- "-m | --mmap Use memory mapped buffers [default]n"
- "-r | --read Use read() callsn"
- "-u | --userp Use application allocated buffersn"
- "-o | --output Outputs stream to stdoutn"
- "-f | --format Force format to 640x480 YUYVn"
- "-c | --count Number of frames to grab [%i]n"
+ "Usage: %s [options]\n\n"
+ "Version 1.3\n"
+ "Options:\n"
+ "-d | --device name Video device name [%s]\n"
+ "-h | --help Print this message\n"
+ "-m | --mmap Use memory mapped buffers [default]\n"
+ "-r | --read Use read() calls\n"
+ "-u | --userp Use application allocated buffers\n"
+ "-o | --output Outputs stream to stdout\n"
+ "-f | --format Force format to 640x480 YUYV\n"
+ "-c | --count Number of frames to grab [%i]\n"
"",
argv[0], dev_name, frame_count);
}
@@ -659,6 +659,6 @@
stop_capturing();
uninit_device();
close_device();
- fprintf(stderr, "\\n");
+ fprintf(stderr, "\n");
return 0;
}
diff --git a/Documentation/userspace-api/media/v4l/control.rst b/Documentation/userspace-api/media/v4l/control.rst
index f8d0b92..3eec651 100644
--- a/Documentation/userspace-api/media/v4l/control.rst
+++ b/Documentation/userspace-api/media/v4l/control.rst
@@ -242,8 +242,17 @@
* - ``V4L2_COLORFX_SET_CBCR``
- The Cb and Cr chroma components are replaced by fixed coefficients
determined by ``V4L2_CID_COLORFX_CBCR`` control.
+ * - ``V4L2_COLORFX_SET_RGB``
+ - The RGB components are replaced by the fixed RGB components determined
+ by ``V4L2_CID_COLORFX_RGB`` control.
+``V4L2_CID_COLORFX_RGB`` ``(integer)``
+ Determines the Red, Green, and Blue coefficients for
+ ``V4L2_COLORFX_SET_RGB`` color effect.
+ Bits [7:0] of the supplied 32 bit value are interpreted as Blue component,
+ bits [15:8] as Green component, bits [23:16] as Red component, and
+ bits [31:24] must be zero.
``V4L2_CID_COLORFX_CBCR`` ``(integer)``
Determines the Cb and Cr coefficients for ``V4L2_COLORFX_SET_CBCR``
diff --git a/Documentation/userspace-api/media/v4l/dev-decoder.rst b/Documentation/userspace-api/media/v4l/dev-decoder.rst
index 5b9b83f..3cf2b49 100644
--- a/Documentation/userspace-api/media/v4l/dev-decoder.rst
+++ b/Documentation/userspace-api/media/v4l/dev-decoder.rst
@@ -752,6 +752,23 @@
buffers are out-of-order compared to the ``OUTPUT`` buffers): ``CAPTURE``
timestamps will not retain the order of ``OUTPUT`` timestamps.
+.. note::
+
+ The backing memory of ``CAPTURE`` buffers that are used as reference frames
+ by the stream may be read by the hardware even after they are dequeued.
+ Consequently, the client should avoid writing into this memory while the
+ ``CAPTURE`` queue is streaming. Failure to observe this may result in
+ corruption of decoded frames.
+
+ Similarly, when using a memory type other than ``V4L2_MEMORY_MMAP``, the
+ client should make sure that each ``CAPTURE`` buffer is always queued with
+ the same backing memory for as long as the ``CAPTURE`` queue is streaming.
+ The reason for this is that V4L2 buffer indices can be used by drivers to
+ identify frames. Thus, if the backing memory of a reference frame is
+ submitted under a different buffer ID, the driver may misidentify it and
+ decode a new frame into it while it is still in use, resulting in corruption
+ of the following frames.
+
During the decoding, the decoder may initiate one of the special sequences, as
listed below. The sequences will result in the decoder returning all the
``CAPTURE`` buffers that originated from all the ``OUTPUT`` buffers processed
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
index 72f5e85..cc080c4 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
@@ -1458,3 +1458,576 @@
.. raw:: latex
\normalsize
+
+.. _v4l2-codec-stateless-vp9:
+
+``V4L2_CID_STATELESS_VP9_COMPRESSED_HDR (struct)``
+ Stores VP9 probabilities updates as parsed from the current compressed frame
+ header. A value of zero in an array element means no update of the relevant
+ probability. Motion vector-related updates contain a new value or zero. All
+ other updates contain values translated with inv_map_table[] (see 6.3.5 in
+ :ref:`vp9`).
+
+.. c:type:: v4l2_ctrl_vp9_compressed_hdr
+
+.. tabularcolumns:: |p{1cm}|p{4.8cm}|p{11.4cm}|
+
+.. cssclass:: longtable
+
+.. flat-table:: struct v4l2_ctrl_vp9_compressed_hdr
+ :header-rows: 0
+ :stub-columns: 0
+ :widths: 1 1 2
+
+ * - __u8
+ - ``tx_mode``
+ - Specifies the TX mode. See :ref:`TX Mode <vp9_tx_mode>` for more details.
+ * - __u8
+ - ``tx8[2][1]``
+ - TX 8x8 probabilities delta.
+ * - __u8
+ - ``tx16[2][2]``
+ - TX 16x16 probabilities delta.
+ * - __u8
+ - ``tx32[2][3]``
+ - TX 32x32 probabilities delta.
+ * - __u8
+ - ``coef[4][2][2][6][6][3]``
+ - Coefficient probabilities delta.
+ * - __u8
+ - ``skip[3]``
+ - Skip probabilities delta.
+ * - __u8
+ - ``inter_mode[7][3]``
+ - Inter prediction mode probabilities delta.
+ * - __u8
+ - ``interp_filter[4][2]``
+ - Interpolation filter probabilities delta.
+ * - __u8
+ - ``is_inter[4]``
+ - Is inter-block probabilities delta.
+ * - __u8
+ - ``comp_mode[5]``
+ - Compound prediction mode probabilities delta.
+ * - __u8
+ - ``single_ref[5][2]``
+ - Single reference probabilities delta.
+ * - __u8
+ - ``comp_ref[5]``
+ - Compound reference probabilities delta.
+ * - __u8
+ - ``y_mode[4][9]``
+ - Y prediction mode probabilities delta.
+ * - __u8
+ - ``uv_mode[10][9]``
+ - UV prediction mode probabilities delta.
+ * - __u8
+ - ``partition[16][3]``
+ - Partition probabilities delta.
+ * - __u8
+ - ``mv.joint[3]``
+ - Motion vector joint probabilities delta.
+ * - __u8
+ - ``mv.sign[2]``
+ - Motion vector sign probabilities delta.
+ * - __u8
+ - ``mv.classes[2][10]``
+ - Motion vector class probabilities delta.
+ * - __u8
+ - ``mv.class0_bit[2]``
+ - Motion vector class0 bit probabilities delta.
+ * - __u8
+ - ``mv.bits[2][10]``
+ - Motion vector bits probabilities delta.
+ * - __u8
+ - ``mv.class0_fr[2][2][3]``
+ - Motion vector class0 fractional bit probabilities delta.
+ * - __u8
+ - ``mv.fr[2][3]``
+ - Motion vector fractional bit probabilities delta.
+ * - __u8
+ - ``mv.class0_hp[2]``
+ - Motion vector class0 high precision fractional bit probabilities delta.
+ * - __u8
+ - ``mv.hp[2]``
+ - Motion vector high precision fractional bit probabilities delta.
+
+.. _vp9_tx_mode:
+
+``TX Mode``
+
+.. tabularcolumns:: |p{6.5cm}|p{0.5cm}|p{10.3cm}|
+
+.. flat-table::
+ :header-rows: 0
+ :stub-columns: 0
+ :widths: 1 1 2
+
+ * - ``V4L2_VP9_TX_MODE_ONLY_4X4``
+ - 0
+ - Transform size is 4x4.
+ * - ``V4L2_VP9_TX_MODE_ALLOW_8X8``
+ - 1
+ - Transform size can be up to 8x8.
+ * - ``V4L2_VP9_TX_MODE_ALLOW_16X16``
+ - 2
+ - Transform size can be up to 16x16.
+ * - ``V4L2_VP9_TX_MODE_ALLOW_32X32``
+ - 3
+ - transform size can be up to 32x32.
+ * - ``V4L2_VP9_TX_MODE_SELECT``
+ - 4
+ - Bitstream contains the transform size for each block.
+
+See section '7.3.1 Tx mode semantics' of the :ref:`vp9` specification for more details.
+
+``V4L2_CID_STATELESS_VP9_FRAME (struct)``
+ Specifies the frame parameters for the associated VP9 frame decode request.
+ This includes the necessary parameters for configuring a stateless hardware
+ decoding pipeline for VP9. The bitstream parameters are defined according
+ to :ref:`vp9`.
+
+.. c:type:: v4l2_ctrl_vp9_frame
+
+.. raw:: latex
+
+ \small
+
+.. tabularcolumns:: |p{4.7cm}|p{5.5cm}|p{7.1cm}|
+
+.. cssclass:: longtable
+
+.. flat-table:: struct v4l2_ctrl_vp9_frame
+ :header-rows: 0
+ :stub-columns: 0
+ :widths: 1 1 2
+
+ * - struct :c:type:`v4l2_vp9_loop_filter`
+ - ``lf``
+ - Loop filter parameters. See struct :c:type:`v4l2_vp9_loop_filter` for more details.
+ * - struct :c:type:`v4l2_vp9_quantization`
+ - ``quant``
+ - Quantization parameters. See :c:type:`v4l2_vp9_quantization` for more details.
+ * - struct :c:type:`v4l2_vp9_segmentation`
+ - ``seg``
+ - Segmentation parameters. See :c:type:`v4l2_vp9_segmentation` for more details.
+ * - __u32
+ - ``flags``
+ - Combination of V4L2_VP9_FRAME_FLAG_* flags. See :ref:`Frame Flags<vp9_frame_flags>`.
+ * - __u16
+ - ``compressed_header_size``
+ - Compressed header size in bytes.
+ * - __u16
+ - ``uncompressed_header_size``
+ - Uncompressed header size in bytes.
+ * - __u16
+ - ``frame_width_minus_1``
+ - Add 1 to get the frame width expressed in pixels. See section 7.2.3 in :ref:`vp9`.
+ * - __u16
+ - ``frame_height_minus_1``
+ - Add 1 to get the frame height expressed in pixels. See section 7.2.3 in :ref:`vp9`.
+ * - __u16
+ - ``render_width_minus_1``
+ - Add 1 to get the expected render width expressed in pixels. This is
+ not used during the decoding process but might be used by HW scalers to
+ prepare a frame that's ready for scanout. See section 7.2.4 in :ref:`vp9`.
+ * - __u16
+ - render_height_minus_1
+ - Add 1 to get the expected render height expressed in pixels. This is
+ not used during the decoding process but might be used by HW scalers to
+ prepare a frame that's ready for scanout. See section 7.2.4 in :ref:`vp9`.
+ * - __u64
+ - ``last_frame_ts``
+ - "last" reference buffer timestamp.
+ The timestamp refers to the ``timestamp`` field in
+ struct :c:type:`v4l2_buffer`. Use the :c:func:`v4l2_timeval_to_ns()`
+ function to convert the struct :c:type:`timeval` in struct
+ :c:type:`v4l2_buffer` to a __u64.
+ * - __u64
+ - ``golden_frame_ts``
+ - "golden" reference buffer timestamp.
+ The timestamp refers to the ``timestamp`` field in
+ struct :c:type:`v4l2_buffer`. Use the :c:func:`v4l2_timeval_to_ns()`
+ function to convert the struct :c:type:`timeval` in struct
+ :c:type:`v4l2_buffer` to a __u64.
+ * - __u64
+ - ``alt_frame_ts``
+ - "alt" reference buffer timestamp.
+ The timestamp refers to the ``timestamp`` field in
+ struct :c:type:`v4l2_buffer`. Use the :c:func:`v4l2_timeval_to_ns()`
+ function to convert the struct :c:type:`timeval` in struct
+ :c:type:`v4l2_buffer` to a __u64.
+ * - __u8
+ - ``ref_frame_sign_bias``
+ - a bitfield specifying whether the sign bias is set for a given
+ reference frame. See :ref:`Reference Frame Sign Bias<vp9_ref_frame_sign_bias>`
+ for more details.
+ * - __u8
+ - ``reset_frame_context``
+ - specifies whether the frame context should be reset to default values. See
+ :ref:`Reset Frame Context<vp9_reset_frame_context>` for more details.
+ * - __u8
+ - ``frame_context_idx``
+ - Frame context that should be used/updated.
+ * - __u8
+ - ``profile``
+ - VP9 profile. Can be 0, 1, 2 or 3.
+ * - __u8
+ - ``bit_depth``
+ - Component depth in bits. Can be 8, 10 or 12. Note that not all profiles
+ support 10 and/or 12 bits depths.
+ * - __u8
+ - ``interpolation_filter``
+ - Specifies the filter selection used for performing inter prediction. See
+ :ref:`Interpolation Filter<vp9_interpolation_filter>` for more details.
+ * - __u8
+ - ``tile_cols_log2``
+ - Specifies the base 2 logarithm of the width of each tile (where the
+ width is measured in units of 8x8 blocks). Shall be less than or equal
+ to 6.
+ * - __u8
+ - ``tile_rows_log2``
+ - Specifies the base 2 logarithm of the height of each tile (where the
+ height is measured in units of 8x8 blocks).
+ * - __u8
+ - ``reference_mode``
+ - Specifies the type of inter prediction to be used. See
+ :ref:`Reference Mode<vp9_reference_mode>` for more details.
+ * - __u8
+ - ``reserved[7]``
+ - Applications and drivers must set this to zero.
+
+.. raw:: latex
+
+ \normalsize
+
+.. _vp9_frame_flags:
+
+``Frame Flags``
+
+.. tabularcolumns:: |p{10.0cm}|p{1.2cm}|p{6.1cm}|
+
+.. flat-table::
+ :header-rows: 0
+ :stub-columns: 0
+ :widths: 1 1 2
+
+ * - ``V4L2_VP9_FRAME_FLAG_KEY_FRAME``
+ - 0x001
+ - The frame is a key frame.
+ * - ``V4L2_VP9_FRAME_FLAG_SHOW_FRAME``
+ - 0x002
+ - The frame should be displayed.
+ * - ``V4L2_VP9_FRAME_FLAG_ERROR_RESILIENT``
+ - 0x004
+ - The decoding should be error resilient.
+ * - ``V4L2_VP9_FRAME_FLAG_INTRA_ONLY``
+ - 0x008
+ - The frame does not reference other frames.
+ * - ``V4L2_VP9_FRAME_FLAG_ALLOW_HIGH_PREC_MV``
+ - 0x010
+ - The frame can use high precision motion vectors.
+ * - ``V4L2_VP9_FRAME_FLAG_REFRESH_FRAME_CTX``
+ - 0x020
+ - Frame context should be updated after decoding.
+ * - ``V4L2_VP9_FRAME_FLAG_PARALLEL_DEC_MODE``
+ - 0x040
+ - Parallel decoding is used.
+ * - ``V4L2_VP9_FRAME_FLAG_X_SUBSAMPLING``
+ - 0x080
+ - Vertical subsampling is enabled.
+ * - ``V4L2_VP9_FRAME_FLAG_Y_SUBSAMPLING``
+ - 0x100
+ - Horizontal subsampling is enabled.
+ * - ``V4L2_VP9_FRAME_FLAG_COLOR_RANGE_FULL_SWING``
+ - 0x200
+ - The full UV range is used.
+
+.. _vp9_ref_frame_sign_bias:
+
+``Reference Frame Sign Bias``
+
+.. tabularcolumns:: |p{7.0cm}|p{1.2cm}|p{9.1cm}|
+
+.. flat-table::
+ :header-rows: 0
+ :stub-columns: 0
+ :widths: 1 1 2
+
+ * - ``V4L2_VP9_SIGN_BIAS_LAST``
+ - 0x1
+ - Sign bias is set for the last reference frame.
+ * - ``V4L2_VP9_SIGN_BIAS_GOLDEN``
+ - 0x2
+ - Sign bias is set for the golden reference frame.
+ * - ``V4L2_VP9_SIGN_BIAS_ALT``
+ - 0x2
+ - Sign bias is set for the alt reference frame.
+
+.. _vp9_reset_frame_context:
+
+``Reset Frame Context``
+
+.. tabularcolumns:: |p{7.0cm}|p{1.2cm}|p{9.1cm}|
+
+.. flat-table::
+ :header-rows: 0
+ :stub-columns: 0
+ :widths: 1 1 2
+
+ * - ``V4L2_VP9_RESET_FRAME_CTX_NONE``
+ - 0
+ - Do not reset any frame context.
+ * - ``V4L2_VP9_RESET_FRAME_CTX_SPEC``
+ - 1
+ - Reset the frame context pointed to by
+ :c:type:`v4l2_ctrl_vp9_frame`.frame_context_idx.
+ * - ``V4L2_VP9_RESET_FRAME_CTX_ALL``
+ - 2
+ - Reset all frame contexts.
+
+See section '7.2 Uncompressed header semantics' of the :ref:`vp9` specification
+for more details.
+
+.. _vp9_interpolation_filter:
+
+``Interpolation Filter``
+
+.. tabularcolumns:: |p{9.0cm}|p{1.2cm}|p{7.1cm}|
+
+.. flat-table::
+ :header-rows: 0
+ :stub-columns: 0
+ :widths: 1 1 2
+
+ * - ``V4L2_VP9_INTERP_FILTER_EIGHTTAP``
+ - 0
+ - Eight tap filter.
+ * - ``V4L2_VP9_INTERP_FILTER_EIGHTTAP_SMOOTH``
+ - 1
+ - Eight tap smooth filter.
+ * - ``V4L2_VP9_INTERP_FILTER_EIGHTTAP_SHARP``
+ - 2
+ - Eeight tap sharp filter.
+ * - ``V4L2_VP9_INTERP_FILTER_BILINEAR``
+ - 3
+ - Bilinear filter.
+ * - ``V4L2_VP9_INTERP_FILTER_SWITCHABLE``
+ - 4
+ - Filter selection is signaled at the block level.
+
+See section '7.2.7 Interpolation filter semantics' of the :ref:`vp9` specification
+for more details.
+
+.. _vp9_reference_mode:
+
+``Reference Mode``
+
+.. tabularcolumns:: |p{9.6cm}|p{0.5cm}|p{7.2cm}|
+
+.. flat-table::
+ :header-rows: 0
+ :stub-columns: 0
+ :widths: 1 1 2
+
+ * - ``V4L2_VP9_REFERENCE_MODE_SINGLE_REFERENCE``
+ - 0
+ - Indicates that all the inter blocks use only a single reference frame
+ to generate motion compensated prediction.
+ * - ``V4L2_VP9_REFERENCE_MODE_COMPOUND_REFERENCE``
+ - 1
+ - Requires all the inter blocks to use compound mode. Single reference
+ frame prediction is not allowed.
+ * - ``V4L2_VP9_REFERENCE_MODE_SELECT``
+ - 2
+ - Allows each individual inter block to select between single and
+ compound prediction modes.
+
+See section '7.3.6 Frame reference mode semantics' of the :ref:`vp9` specification for more details.
+
+.. c:type:: v4l2_vp9_segmentation
+
+Encodes the quantization parameters. See section '7.2.10 Segmentation
+params syntax' of the :ref:`vp9` specification for more details.
+
+.. tabularcolumns:: |p{0.8cm}|p{5cm}|p{11.4cm}|
+
+.. cssclass:: longtable
+
+.. flat-table:: struct v4l2_vp9_segmentation
+ :header-rows: 0
+ :stub-columns: 0
+ :widths: 1 1 2
+
+ * - __u8
+ - ``feature_data[8][4]``
+ - Data attached to each feature. Data entry is only valid if the feature
+ is enabled. The array shall be indexed with segment number as the first dimension
+ (0..7) and one of V4L2_VP9_SEG_* as the second dimension.
+ See :ref:`Segment Feature IDs<vp9_segment_feature>`.
+ * - __u8
+ - ``feature_enabled[8]``
+ - Bitmask defining which features are enabled in each segment. The value for each
+ segment is a combination of V4L2_VP9_SEGMENT_FEATURE_ENABLED(id) values where id is
+ one of V4L2_VP9_SEG_*. See :ref:`Segment Feature IDs<vp9_segment_feature>`.
+ * - __u8
+ - ``tree_probs[7]``
+ - Specifies the probability values to be used when decoding a Segment-ID.
+ See '5.15. Segmentation map' section of :ref:`vp9` for more details.
+ * - __u8
+ - ``pred_probs[3]``
+ - Specifies the probability values to be used when decoding a
+ Predicted-Segment-ID. See '6.4.14. Get segment id syntax'
+ section of :ref:`vp9` for more details.
+ * - __u8
+ - ``flags``
+ - Combination of V4L2_VP9_SEGMENTATION_FLAG_* flags. See
+ :ref:`Segmentation Flags<vp9_segmentation_flags>`.
+ * - __u8
+ - ``reserved[5]``
+ - Applications and drivers must set this to zero.
+
+.. _vp9_segment_feature:
+
+``Segment feature IDs``
+
+.. tabularcolumns:: |p{6.0cm}|p{1cm}|p{10.3cm}|
+
+.. flat-table::
+ :header-rows: 0
+ :stub-columns: 0
+ :widths: 1 1 2
+
+ * - ``V4L2_VP9_SEG_LVL_ALT_Q``
+ - 0
+ - Quantizer segment feature.
+ * - ``V4L2_VP9_SEG_LVL_ALT_L``
+ - 1
+ - Loop filter segment feature.
+ * - ``V4L2_VP9_SEG_LVL_REF_FRAME``
+ - 2
+ - Reference frame segment feature.
+ * - ``V4L2_VP9_SEG_LVL_SKIP``
+ - 3
+ - Skip segment feature.
+ * - ``V4L2_VP9_SEG_LVL_MAX``
+ - 4
+ - Number of segment features.
+
+.. _vp9_segmentation_flags:
+
+``Segmentation Flags``
+
+.. tabularcolumns:: |p{10.6cm}|p{0.8cm}|p{5.9cm}|
+
+.. flat-table::
+ :header-rows: 0
+ :stub-columns: 0
+ :widths: 1 1 2
+
+ * - ``V4L2_VP9_SEGMENTATION_FLAG_ENABLED``
+ - 0x01
+ - Indicates that this frame makes use of the segmentation tool.
+ * - ``V4L2_VP9_SEGMENTATION_FLAG_UPDATE_MAP``
+ - 0x02
+ - Indicates that the segmentation map should be updated during the
+ decoding of this frame.
+ * - ``V4L2_VP9_SEGMENTATION_FLAG_TEMPORAL_UPDATE``
+ - 0x04
+ - Indicates that the updates to the segmentation map are coded
+ relative to the existing segmentation map.
+ * - ``V4L2_VP9_SEGMENTATION_FLAG_UPDATE_DATA``
+ - 0x08
+ - Indicates that new parameters are about to be specified for each
+ segment.
+ * - ``V4L2_VP9_SEGMENTATION_FLAG_ABS_OR_DELTA_UPDATE``
+ - 0x10
+ - Indicates that the segmentation parameters represent the actual values
+ to be used.
+
+.. c:type:: v4l2_vp9_quantization
+
+Encodes the quantization parameters. See section '7.2.9 Quantization params
+syntax' of the VP9 specification for more details.
+
+.. tabularcolumns:: |p{0.8cm}|p{4cm}|p{12.4cm}|
+
+.. cssclass:: longtable
+
+.. flat-table:: struct v4l2_vp9_quantization
+ :header-rows: 0
+ :stub-columns: 0
+ :widths: 1 1 2
+
+ * - __u8
+ - ``base_q_idx``
+ - Indicates the base frame qindex.
+ * - __s8
+ - ``delta_q_y_dc``
+ - Indicates the Y DC quantizer relative to base_q_idx.
+ * - __s8
+ - ``delta_q_uv_dc``
+ - Indicates the UV DC quantizer relative to base_q_idx.
+ * - __s8
+ - ``delta_q_uv_ac``
+ - Indicates the UV AC quantizer relative to base_q_idx.
+ * - __u8
+ - ``reserved[4]``
+ - Applications and drivers must set this to zero.
+
+.. c:type:: v4l2_vp9_loop_filter
+
+This structure contains all loop filter related parameters. See sections
+'7.2.8 Loop filter semantics' of the :ref:`vp9` specification for more details.
+
+.. tabularcolumns:: |p{0.8cm}|p{4cm}|p{12.4cm}|
+
+.. cssclass:: longtable
+
+.. flat-table:: struct v4l2_vp9_loop_filter
+ :header-rows: 0
+ :stub-columns: 0
+ :widths: 1 1 2
+
+ * - __s8
+ - ``ref_deltas[4]``
+ - Contains the adjustment needed for the filter level based on the chosen
+ reference frame.
+ * - __s8
+ - ``mode_deltas[2]``
+ - Contains the adjustment needed for the filter level based on the chosen
+ mode.
+ * - __u8
+ - ``level``
+ - Indicates the loop filter strength.
+ * - __u8
+ - ``sharpness``
+ - Indicates the sharpness level.
+ * - __u8
+ - ``flags``
+ - Combination of V4L2_VP9_LOOP_FILTER_FLAG_* flags.
+ See :ref:`Loop Filter Flags <vp9_loop_filter_flags>`.
+ * - __u8
+ - ``reserved[7]``
+ - Applications and drivers must set this to zero.
+
+
+.. _vp9_loop_filter_flags:
+
+``Loop Filter Flags``
+
+.. tabularcolumns:: |p{9.6cm}|p{0.5cm}|p{7.2cm}|
+
+.. flat-table::
+ :header-rows: 0
+ :stub-columns: 0
+ :widths: 1 1 2
+
+ * - ``V4L2_VP9_LOOP_FILTER_FLAG_DELTA_ENABLED``
+ - 0x1
+ - When set, the filter level depends on the mode and reference frame used
+ to predict a block.
+ * - ``V4L2_VP9_LOOP_FILTER_FLAG_DELTA_UPDATE``
+ - 0x2
+ - When set, the bitstream contains additional syntax elements that
+ specify which mode and reference frame deltas are to be updated.
diff --git a/Documentation/userspace-api/media/v4l/libv4l-introduction.rst b/Documentation/userspace-api/media/v4l/libv4l-introduction.rst
index 05690f2..9021531 100644
--- a/Documentation/userspace-api/media/v4l/libv4l-introduction.rst
+++ b/Documentation/userspace-api/media/v4l/libv4l-introduction.rst
@@ -26,7 +26,7 @@
It currently accepts the following V4L2 driver formats:
:ref:`V4L2_PIX_FMT_BGR24 <V4L2-PIX-FMT-BGR24>`,
-:ref:`V4L2_PIX_FMT_HM12 <V4L2-PIX-FMT-HM12>`,
+:ref:`V4L2_PIX_FMT_NV12_16L16 <V4L2-PIX-FMT-NV12-16L16>`,
:ref:`V4L2_PIX_FMT_JPEG <V4L2-PIX-FMT-JPEG>`,
:ref:`V4L2_PIX_FMT_MJPEG <V4L2-PIX-FMT-MJPEG>`,
:ref:`V4L2_PIX_FMT_MR97310A <V4L2-PIX-FMT-MR97310A>`,
diff --git a/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst b/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst
index 0ede399..967fc80 100644
--- a/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst
+++ b/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst
@@ -172,6 +172,21 @@
- VP9 compressed video frame. The encoder generates one
compressed frame per buffer, and the decoder requires one
compressed frame per buffer.
+ * .. _V4L2-PIX-FMT-VP9-FRAME:
+
+ - ``V4L2_PIX_FMT_VP9_FRAME``
+ - 'VP9F'
+ - VP9 parsed frame, including the frame header, as extracted from the container.
+ This format is adapted for stateless video decoders that implement a
+ VP9 pipeline with the :ref:`stateless_decoder`.
+ Metadata associated with the frame to decode is required to be passed
+ through the ``V4L2_CID_STATELESS_VP9_FRAME`` and
+ the ``V4L2_CID_STATELESS_VP9_COMPRESSED_HDR`` controls.
+ See the :ref:`associated Codec Control IDs <v4l2-codec-stateless-vp9>`.
+ Exactly one output and one capture buffer must be provided for use with
+ this pixel format. The output buffer must contain the appropriate number
+ of macroblocks to decode a full corresponding frame to the matching
+ capture buffer.
* .. _V4L2-PIX-FMT-HEVC:
- ``V4L2_PIX_FMT_HEVC``
diff --git a/Documentation/userspace-api/media/v4l/subdev-formats.rst b/Documentation/userspace-api/media/v4l/subdev-formats.rst
index bd68588..0cbc045d 100644
--- a/Documentation/userspace-api/media/v4l/subdev-formats.rst
+++ b/Documentation/userspace-api/media/v4l/subdev-formats.rst
@@ -7833,7 +7833,7 @@
.. raw:: latex
- \normalsize
+ \endgroup
JPEG Compressed Formats
diff --git a/Documentation/userspace-api/media/v4l/v4l2grab.c.rst b/Documentation/userspace-api/media/v4l/v4l2grab.c.rst
index eaa0f95..b38f661 100644
--- a/Documentation/userspace-api/media/v4l/v4l2grab.c.rst
+++ b/Documentation/userspace-api/media/v4l/v4l2grab.c.rst
@@ -46,7 +46,7 @@
} while (r == -1 && ((errno == EINTR) || (errno == EAGAIN)));
if (r == -1) {
- fprintf(stderr, "error %d, %s\\n", errno, strerror(errno));
+ fprintf(stderr, "error %d, %s\n", errno, strerror(errno));
exit(EXIT_FAILURE);
}
}
@@ -80,11 +80,11 @@
fmt.fmt.pix.field = V4L2_FIELD_INTERLACED;
xioctl(fd, VIDIOC_S_FMT, &fmt);
if (fmt.fmt.pix.pixelformat != V4L2_PIX_FMT_RGB24) {
- printf("Libv4l didn't accept RGB24 format. Can't proceed.\\n");
+ printf("Libv4l didn't accept RGB24 format. Can't proceed.\n");
exit(EXIT_FAILURE);
}
if ((fmt.fmt.pix.width != 640) || (fmt.fmt.pix.height != 480))
- printf("Warning: driver is sending image at %dx%d\\n",
+ printf("Warning: driver is sending image at %dx%d\n",
fmt.fmt.pix.width, fmt.fmt.pix.height);
CLEAR(req);
@@ -151,7 +151,7 @@
perror("Cannot open image");
exit(EXIT_FAILURE);
}
- fprintf(fout, "P6\\n%d %d 255\\n",
+ fprintf(fout, "P6\n%d %d 255\n",
fmt.fmt.pix.width, fmt.fmt.pix.height);
fwrite(buffers[buf.index].start, buf.bytesused, 1, fout);
fclose(fout);
diff --git a/Documentation/userspace-api/media/v4l/vidioc-g-ext-ctrls.rst b/Documentation/userspace-api/media/v4l/vidioc-g-ext-ctrls.rst
index fdde0ae..29971a4 100644
--- a/Documentation/userspace-api/media/v4l/vidioc-g-ext-ctrls.rst
+++ b/Documentation/userspace-api/media/v4l/vidioc-g-ext-ctrls.rst
@@ -233,6 +233,14 @@
- ``p_mpeg2_quantisation``
- A pointer to a struct :c:type:`v4l2_ctrl_mpeg2_quantisation`. Valid if this control is
of type ``V4L2_CTRL_TYPE_MPEG2_QUANTISATION``.
+ * - struct :c:type:`v4l2_ctrl_vp9_compressed_hdr` *
+ - ``p_vp9_compressed_hdr_probs``
+ - A pointer to a struct :c:type:`v4l2_ctrl_vp9_compressed_hdr`. Valid if this
+ control is of type ``V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR``.
+ * - struct :c:type:`v4l2_ctrl_vp9_frame` *
+ - ``p_vp9_frame``
+ - A pointer to a struct :c:type:`v4l2_ctrl_vp9_frame`. Valid if this
+ control is of type ``V4L2_CTRL_TYPE_VP9_FRAME``.
* - struct :c:type:`v4l2_ctrl_hdr10_cll_info` *
- ``p_hdr10_cll``
- A pointer to a struct :c:type:`v4l2_ctrl_hdr10_cll_info`. Valid if this control is
diff --git a/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst b/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
index 2f491c1..88f6302 100644
--- a/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
+++ b/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
@@ -513,6 +513,18 @@
- n/a
- A struct :c:type:`v4l2_ctrl_hevc_decode_params`, containing HEVC
decoding parameters for stateless video decoders.
+ * - ``V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR``
+ - n/a
+ - n/a
+ - n/a
+ - A struct :c:type:`v4l2_ctrl_vp9_compressed_hdr`, containing VP9
+ probabilities updates for stateless video decoders.
+ * - ``V4L2_CTRL_TYPE_VP9_FRAME``
+ - n/a
+ - n/a
+ - n/a
+ - A struct :c:type:`v4l2_ctrl_vp9_frame`, containing VP9
+ frame decode parameters for stateless video decoders.
.. raw:: latex
diff --git a/Documentation/userspace-api/media/videodev2.h.rst.exceptions b/Documentation/userspace-api/media/videodev2.h.rst.exceptions
index eb0b1cd..9cbb7a0 100644
--- a/Documentation/userspace-api/media/videodev2.h.rst.exceptions
+++ b/Documentation/userspace-api/media/videodev2.h.rst.exceptions
@@ -149,6 +149,8 @@
replace symbol V4L2_CTRL_TYPE_AREA :c:type:`v4l2_ctrl_type`
replace symbol V4L2_CTRL_TYPE_FWHT_PARAMS :c:type:`v4l2_ctrl_type`
replace symbol V4L2_CTRL_TYPE_VP8_FRAME :c:type:`v4l2_ctrl_type`
+replace symbol V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR :c:type:`v4l2_ctrl_type`
+replace symbol V4L2_CTRL_TYPE_VP9_FRAME :c:type:`v4l2_ctrl_type`
replace symbol V4L2_CTRL_TYPE_HDR10_CLL_INFO :c:type:`v4l2_ctrl_type`
replace symbol V4L2_CTRL_TYPE_HDR10_MASTERING_DISPLAY :c:type:`v4l2_ctrl_type`
diff --git a/Documentation/x86/sgx.rst b/Documentation/x86/sgx.rst
index a608f66..265568a 100644
--- a/Documentation/x86/sgx.rst
+++ b/Documentation/x86/sgx.rst
@@ -10,7 +10,7 @@
Software Guard eXtensions (SGX) hardware enables for user space applications
to set aside private memory regions of code and data:
-* Privileged (ring-0) ENCLS functions orchestrate the construction of the.
+* Privileged (ring-0) ENCLS functions orchestrate the construction of the
regions.
* Unprivileged (ring-3) ENCLU functions allow an application to enter and
execute inside the regions.
@@ -91,7 +91,7 @@
separate enclave “build” process. Enclaves must be built before they can be
executed (entered). The first step in building an enclave is opening the
**/dev/sgx_enclave** device. Since enclave memory is protected from direct
-access, special privileged instructions are Then used to copy data into enclave
+access, special privileged instructions are then used to copy data into enclave
pages and establish enclave page permissions.
.. kernel-doc:: arch/x86/kernel/cpu/sgx/ioctl.c
@@ -126,13 +126,13 @@
ksgxd
=====
-SGX support includes a kernel thread called *ksgxwapd*.
+SGX support includes a kernel thread called *ksgxd*.
EPC sanitization
----------------
ksgxd is started when SGX initializes. Enclave memory is typically ready
-For use when the processor powers on or resets. However, if SGX has been in
+for use when the processor powers on or resets. However, if SGX has been in
use since the reset, enclave pages may be in an inconsistent state. This might
occur after a crash and kexec() cycle, for instance. At boot, ksgxd
reinitializes all enclave pages so that they can be allocated and re-used.
@@ -147,7 +147,7 @@
Similar to the core kswapd, ksgxd, is responsible for managing the
overcommitment of enclave memory. If the system runs out of enclave memory,
-*ksgxwapd* “swaps” enclave memory to normal memory.
+*ksgxd* “swaps” enclave memory to normal memory.
Launch Control
==============
@@ -156,7 +156,7 @@
copied, kernel executes EINIT function, which initializes the enclave. Only after
this the CPU can execute inside the enclave.
-ENIT function takes an RSA-3072 signature of the enclave measurement. The function
+EINIT function takes an RSA-3072 signature of the enclave measurement. The function
checks that the measurement is correct and signature is signed with the key
hashed to the four **IA32_SGXLEPUBKEYHASH{0, 1, 2, 3}** MSRs representing the
SHA256 of a public key.
@@ -184,7 +184,7 @@
MEE. TME-based SGX implementations do not have an integrity Merkle tree, which
means integrity and replay-attacks are not mitigated. B, it includes
additional changes to prevent cipher text from being returned and SW memory
-aliases from being Created.
+aliases from being created.
DMA to enclave memory is blocked by range registers on both MEE and TME systems
(SDM section 41.10).
diff --git a/MAINTAINERS b/MAINTAINERS
index fb18ce7..aeb2863 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -966,6 +966,7 @@
F: drivers/gpu/drm/amd/include/v9_structs.h
F: drivers/gpu/drm/amd/include/vi_structs.h
F: include/uapi/linux/kfd_ioctl.h
+F: include/uapi/linux/kfd_sysfs.h
AMD SPI DRIVER
M: Sanjay R Mehta <sanju.mehta@amd.com>
@@ -1139,6 +1140,7 @@
M: Kieran Bingham <kieran.bingham@ideasonboard.com>
L: linux-media@vger.kernel.org
S: Maintained
+F: Documentation/devicetree/bindings/media/i2c/adv748x.yaml
F: drivers/media/i2c/adv748x/*
ANALOG DEVICES INC ADV7511 DRIVER
@@ -1745,17 +1747,21 @@
C: irc://irc.oftc.net/asahi-dev
T: git https://github.com/AsahiLinux/linux.git
F: Documentation/devicetree/bindings/arm/apple.yaml
+F: Documentation/devicetree/bindings/arm/apple/*
F: Documentation/devicetree/bindings/i2c/apple,i2c.yaml
F: Documentation/devicetree/bindings/interrupt-controller/apple,aic.yaml
F: Documentation/devicetree/bindings/mailbox/apple,mailbox.yaml
F: Documentation/devicetree/bindings/pci/apple,pcie.yaml
F: Documentation/devicetree/bindings/pinctrl/apple,pinctrl.yaml
+F: Documentation/devicetree/bindings/power/apple*
+F: Documentation/devicetree/bindings/watchdog/apple,wdt.yaml
F: arch/arm64/boot/dts/apple/
F: drivers/i2c/busses/i2c-pasemi-core.c
F: drivers/i2c/busses/i2c-pasemi-platform.c
F: drivers/irqchip/irq-apple-aic.c
F: drivers/mailbox/apple-mailbox.c
F: drivers/pinctrl/pinctrl-apple-gpio.c
+F: drivers/soc/apple/*
F: include/dt-bindings/interrupt-controller/apple-aic.h
F: include/dt-bindings/pinctrl/apple.h
F: include/linux/apple-mailbox.h
@@ -2551,6 +2557,7 @@
F: Documentation/arm/samsung/
F: Documentation/devicetree/bindings/arm/samsung/
F: Documentation/devicetree/bindings/power/pd-samsung.yaml
+F: Documentation/devicetree/bindings/soc/samsung/
F: arch/arm/boot/dts/exynos*
F: arch/arm/boot/dts/s3c*
F: arch/arm/boot/dts/s5p*
@@ -2577,7 +2584,7 @@
N: s5pv210
ARM/SAMSUNG S5P SERIES 2D GRAPHICS ACCELERATION (G2D) SUPPORT
-M: Andrzej Hajda <a.hajda@samsung.com>
+M: Łukasz Stelmach <l.stelmach@samsung.com>
L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
L: linux-media@vger.kernel.org
S: Maintained
@@ -2601,7 +2608,8 @@
F: drivers/media/platform/s5p-jpeg/
ARM/SAMSUNG S5P SERIES Multi Format Codec (MFC) SUPPORT
-M: Andrzej Hajda <a.hajda@samsung.com>
+M: Marek Szyprowski <m.szyprowski@samsung.com>
+M: Andrzej Hajda <andrzej.hajda@intel.com>
L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
L: linux-media@vger.kernel.org
S: Maintained
@@ -3569,7 +3577,7 @@
R: Brendan Jackman <jackmanb@chromium.org>
L: bpf@vger.kernel.org
S: Maintained
-F: Documentation/bpf/bpf_lsm.rst
+F: Documentation/bpf/prog_lsm.rst
F: include/linux/bpf_lsm.h
F: kernel/bpf/bpf_lsm.c
F: security/bpf/
@@ -3636,6 +3644,7 @@
F: drivers/net/ethernet/broadcom/unimac.h
BROADCOM BCM5301X ARM ARCHITECTURE
+M: Florian Fainelli <f.fainelli@gmail.com>
M: Hauke Mehrtens <hauke@hauke-m.de>
M: Rafał Miłecki <zajec5@gmail.com>
M: bcm-kernel-feedback-list@broadcom.com
@@ -3647,6 +3656,7 @@
F: arch/arm/mach-bcm/bcm_5301x.c
BROADCOM BCM53573 ARM ARCHITECTURE
+M: Florian Fainelli <f.fainelli@gmail.com>
M: Rafał Miłecki <rafal@milecki.pl>
L: bcm-kernel-feedback-list@broadcom.com
L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
@@ -3769,7 +3779,8 @@
F: drivers/net/wireless/broadcom/brcm80211/
BROADCOM BRCMSTB GPIO DRIVER
-M: Gregory Fong <gregory.0xf0@gmail.com>
+M: Doug Berger <opendmb@gmail.com>
+M: Florian Fainelli <f.fainelli@gmail.com>
L: bcm-kernel-feedback-list@broadcom.com
S: Supported
F: Documentation/devicetree/bindings/gpio/brcm,brcmstb-gpio.txt
@@ -6049,6 +6060,7 @@
DRM DRIVER FOR MSM ADRENO GPU
M: Rob Clark <robdclark@gmail.com>
M: Sean Paul <sean@poorly.run>
+R: Abhinav Kumar <quic_abhinavk@quicinc.com>
L: linux-arm-msm@vger.kernel.org
L: dri-devel@lists.freedesktop.org
L: freedreno@lists.freedesktop.org
@@ -6074,10 +6086,17 @@
DRM DRIVER FOR NVIDIA GEFORCE/QUADRO GPUS
M: Ben Skeggs <bskeggs@redhat.com>
+M: Karol Herbst <kherbst@redhat.com>
+M: Lyude Paul <lyude@redhat.com>
L: dri-devel@lists.freedesktop.org
L: nouveau@lists.freedesktop.org
S: Supported
-T: git git://github.com/skeggsb/linux
+W: https://nouveau.freedesktop.org/
+Q: https://patchwork.freedesktop.org/project/nouveau/
+Q: https://gitlab.freedesktop.org/drm/nouveau/-/merge_requests
+B: https://gitlab.freedesktop.org/drm/nouveau/-/issues
+C: irc://irc.oftc.net/nouveau
+T: git https://gitlab.freedesktop.org/drm/nouveau.git
F: drivers/gpu/drm/nouveau/
F: include/uapi/drm/nouveau_drm.h
@@ -6310,7 +6329,7 @@
F: drivers/gpu/drm/atmel-hlcdc/
DRM DRIVERS FOR BRIDGE CHIPS
-M: Andrzej Hajda <a.hajda@samsung.com>
+M: Andrzej Hajda <andrzej.hajda@intel.com>
M: Neil Armstrong <narmstrong@baylibre.com>
M: Robert Foss <robert.foss@linaro.org>
R: Laurent Pinchart <Laurent.pinchart@ideasonboard.com>
@@ -6417,6 +6436,7 @@
L: linux-renesas-soc@vger.kernel.org
S: Supported
T: git git://linuxtv.org/pinchartl/media drm/du/next
+F: Documentation/devicetree/bindings/display/bridge/renesas,dsi-csi2-tx.yaml
F: Documentation/devicetree/bindings/display/bridge/renesas,dw-hdmi.yaml
F: Documentation/devicetree/bindings/display/bridge/renesas,lvds.yaml
F: Documentation/devicetree/bindings/display/renesas,du.yaml
@@ -6535,6 +6555,14 @@
F: drivers/gpu/drm/panel/
F: include/drm/drm_panel.h
+DRM PRIVACY-SCREEN CLASS
+M: Hans de Goede <hdegoede@redhat.com>
+L: dri-devel@lists.freedesktop.org
+S: Maintained
+T: git git://anongit.freedesktop.org/drm/drm-misc
+F: drivers/gpu/drm/drm_privacy_screen*
+F: include/drm/drm_privacy_screen*
+
DRM TTM SUBSYSTEM
M: Christian Koenig <christian.koenig@amd.com>
M: Huang Rui <ray.huang@amd.com>
@@ -8607,8 +8635,10 @@
HISILICON PMU DRIVER
M: Shaokun Zhang <zhangshaokun@hisilicon.com>
+M: Qi Liu <liuqi115@huawei.com>
S: Supported
W: http://www.hisilicon.com
+F: Documentation/admin-guide/perf/hisi-pcie-pmu.rst
F: Documentation/admin-guide/perf/hisi-pmu.rst
F: drivers/perf/hisilicon
@@ -9489,6 +9519,7 @@
M: Jani Nikula <jani.nikula@linux.intel.com>
M: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
M: Rodrigo Vivi <rodrigo.vivi@intel.com>
+M: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com>
L: intel-gfx@lists.freedesktop.org
S: Supported
W: https://01.org/linuxgraphics/
@@ -11533,6 +11564,12 @@
F: Documentation/devicetree/bindings/media/i2c/maxim,max9286.yaml
F: drivers/media/i2c/max9286.c
+MAX96712 QUAD GMSL2 DESERIALIZER DRIVER
+M: Niklas Söderlund <niklas.soderlund@ragnatech.se>
+L: linux-media@vger.kernel.org
+S: Maintained
+F: drivers/staging/media/max96712/max96712.c
+
MAX9860 MONO AUDIO VOICE CODEC DRIVER
M: Peter Rosin <peda@axentia.se>
L: alsa-devel@alsa-project.org (moderated for non-subscribers)
@@ -12523,6 +12560,13 @@
S: Maintained
F: drivers/net/ethernet/microchip/lan743x_*
+MICROCHIP LAN966X ETHERNET DRIVER
+M: Horatiu Vultur <horatiu.vultur@microchip.com>
+M: UNGLinuxDriver@microchip.com
+L: netdev@vger.kernel.org
+S: Maintained
+F: drivers/net/ethernet/microchip/lan966x/*
+
MICROCHIP LCDFB DRIVER
M: Nicolas Ferre <nicolas.ferre@microchip.com>
L: linux-fbdev@vger.kernel.org
@@ -14097,7 +14141,6 @@
OMNIVISION OV5670 SENSOR DRIVER
M: Chiranjeevi Rapolu <chiranjeevi.rapolu@intel.com>
-M: Hyungwoo Yang <hyungwoo.yang@intel.com>
L: linux-media@vger.kernel.org
S: Maintained
T: git git://linuxtv.org/media_tree.git
@@ -14110,6 +14153,13 @@
T: git git://linuxtv.org/media_tree.git
F: drivers/media/i2c/ov5675.c
+OMNIVISION OV5693 SENSOR DRIVER
+M: Daniel Scally <djrscally@gmail.com>
+L: linux-media@vger.kernel.org
+S: Maintained
+T: git git://linuxtv.org/media_tree.git
+F: drivers/media/i2c/ov5693.c
+
OMNIVISION OV5695 SENSOR DRIVER
M: Shunqian Zheng <zhengsq@rock-chips.com>
L: linux-media@vger.kernel.org
@@ -15726,6 +15776,14 @@
F: Documentation/devicetree/bindings/net/wireless/qca,ath9k.yaml
F: drivers/net/wireless/ath/ath9k/
+QUALCOMM BAM-DMUX WWAN NETWORK DRIVER
+M: Stephan Gerhold <stephan@gerhold.net>
+L: netdev@vger.kernel.org
+L: linux-arm-msm@vger.kernel.org
+S: Maintained
+F: Documentation/devicetree/bindings/net/qcom,bam-dmux.yaml
+F: drivers/net/wwan/qcom_bam_dmux.c
+
QUALCOMM CAMERA SUBSYSTEM DRIVER
M: Robert Foss <robert.foss@linaro.org>
M: Todor Tomov <todor.too@gmail.com>
@@ -15988,6 +16046,7 @@
RANDOM NUMBER DRIVER
M: "Theodore Ts'o" <tytso@mit.edu>
M: Jason A. Donenfeld <Jason@zx2c4.com>
+T: git https://git.kernel.org/pub/scm/linux/kernel/git/crng/random.git
S: Maintained
F: drivers/char/random.c
@@ -16836,13 +16895,15 @@
F: drivers/nfc/s3fwrn5
SAMSUNG S5C73M3 CAMERA DRIVER
-M: Andrzej Hajda <a.hajda@samsung.com>
+M: Sylwester Nawrocki <s.nawrocki@samsung.com>
+M: Andrzej Hajda <andrzej.hajda@intel.com>
L: linux-media@vger.kernel.org
S: Supported
F: drivers/media/i2c/s5c73m3/*
SAMSUNG S5K5BAF CAMERA DRIVER
-M: Andrzej Hajda <a.hajda@samsung.com>
+M: Sylwester Nawrocki <s.nawrocki@samsung.com>
+M: Andrzej Hajda <andrzej.hajda@intel.com>
L: linux-media@vger.kernel.org
S: Supported
F: drivers/media/i2c/s5k5baf.c
@@ -18156,6 +18217,28 @@
S: Odd Fixes
F: drivers/net/ethernet/adaptec/starfire*
+STARFIVE JH7100 CLOCK DRIVER
+M: Emil Renner Berthing <kernel@esmil.dk>
+S: Maintained
+F: Documentation/devicetree/bindings/clock/starfive,jh7100-clkgen.yaml
+F: drivers/clk/starfive/clk-starfive-jh7100.c
+F: include/dt-bindings/clock/starfive-jh7100.h
+
+STARFIVE JH7100 PINCTRL DRIVER
+M: Emil Renner Berthing <kernel@esmil.dk>
+L: linux-gpio@vger.kernel.org
+S: Maintained
+F: Documentation/devicetree/bindings/pinctrl/starfive,jh7100-pinctrl.yaml
+F: drivers/pinctrl/pinctrl-starfive.c
+F: include/dt-bindings/pinctrl/pinctrl-starfive.h
+
+STARFIVE JH7100 RESET CONTROLLER DRIVER
+M: Emil Renner Berthing <kernel@esmil.dk>
+S: Maintained
+F: Documentation/devicetree/bindings/reset/starfive,jh7100-reset.yaml
+F: drivers/reset/reset-starfive-jh7100.c
+F: include/dt-bindings/reset/starfive-jh7100.h
+
STATIC BRANCH/CALL
M: Peter Zijlstra <peterz@infradead.org>
M: Josh Poimboeuf <jpoimboe@redhat.com>
@@ -20195,6 +20278,8 @@
VIRTIO GPU DRIVER
M: David Airlie <airlied@linux.ie>
M: Gerd Hoffmann <kraxel@redhat.com>
+R: Gurchetan Singh <gurchetansingh@chromium.org>
+R: Chia-I Wu <olvaffe@gmail.com>
L: dri-devel@lists.freedesktop.org
L: virtualization@lists.linux-foundation.org
S: Maintained
diff --git a/Makefile b/Makefile
index 17b4319..0851023 100644
--- a/Makefile
+++ b/Makefile
@@ -2,7 +2,7 @@
VERSION = 5
PATCHLEVEL = 16
SUBLEVEL = 0
-EXTRAVERSION = -rc7
+EXTRAVERSION =
NAME = Gobble Gobble
# *DOCUMENTATION*
diff --git a/arch/Kconfig b/arch/Kconfig
index d3c4ab2..847fde3 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -1312,6 +1312,10 @@
config DYNAMIC_SIGFRAME
bool
+# Select, if arch has a named attribute group bound to NUMA device nodes.
+config HAVE_ARCH_NODE_DEV_GROUP
+ bool
+
source "kernel/gcov/Kconfig"
source "scripts/gcc-plugins/Kconfig"
diff --git a/arch/alpha/kernel/signal.c b/arch/alpha/kernel/signal.c
index bc077ba..d8ed71d 100644
--- a/arch/alpha/kernel/signal.c
+++ b/arch/alpha/kernel/signal.c
@@ -535,6 +535,6 @@ do_work_pending(struct pt_regs *regs, unsigned long thread_flags,
}
}
local_irq_disable();
- thread_flags = current_thread_info()->flags;
+ thread_flags = read_thread_flags();
} while (thread_flags & _TIF_WORK_MASK);
}
diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index c2724d9..796fc80 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -479,7 +479,6 @@
select GPIO_SAMSUNG
select GPIOLIB
select GENERIC_IRQ_MULTI_HANDLER
- select HAVE_S3C2410_I2C if I2C
select NEED_MACH_IO_H
select S3C2410_WATCHDOG
select SAMSUNG_ATAGS
diff --git a/arch/arm/Kconfig.debug b/arch/arm/Kconfig.debug
index 9843670..644875d 100644
--- a/arch/arm/Kconfig.debug
+++ b/arch/arm/Kconfig.debug
@@ -410,12 +410,12 @@
Say Y here if you want kernel low-level debugging support
on i.MX25.
- config DEBUG_IMX21_IMX27_UART
- bool "i.MX21 and i.MX27 Debug UART"
- depends on SOC_IMX21 || SOC_IMX27
+ config DEBUG_IMX27_UART
+ bool "i.MX27 Debug UART"
+ depends on SOC_IMX27
help
Say Y here if you want kernel low-level debugging support
- on i.MX21 or i.MX27.
+ on i.MX27.
config DEBUG_IMX28_UART
bool "i.MX28 Debug UART"
@@ -1481,7 +1481,7 @@
int "i.MX Debug UART Port Selection"
depends on DEBUG_IMX1_UART || \
DEBUG_IMX25_UART || \
- DEBUG_IMX21_IMX27_UART || \
+ DEBUG_IMX27_UART || \
DEBUG_IMX31_UART || \
DEBUG_IMX35_UART || \
DEBUG_IMX50_UART || \
@@ -1540,12 +1540,12 @@
default "debug/icedcc.S" if DEBUG_ICEDCC
default "debug/imx.S" if DEBUG_IMX1_UART || \
DEBUG_IMX25_UART || \
- DEBUG_IMX21_IMX27_UART || \
+ DEBUG_IMX27_UART || \
DEBUG_IMX31_UART || \
DEBUG_IMX35_UART || \
DEBUG_IMX50_UART || \
DEBUG_IMX51_UART || \
- DEBUG_IMX53_UART ||\
+ DEBUG_IMX53_UART || \
DEBUG_IMX6Q_UART || \
DEBUG_IMX6SL_UART || \
DEBUG_IMX6SX_UART || \
diff --git a/arch/arm/boot/dts/Makefile b/arch/arm/boot/dts/Makefile
index 0de64f2..235ad55 100644
--- a/arch/arm/boot/dts/Makefile
+++ b/arch/arm/boot/dts/Makefile
@@ -61,6 +61,7 @@
at91-sama5d2_icp.dtb \
at91-sama5d2_ptc_ek.dtb \
at91-sama5d2_xplained.dtb \
+ at91-sama5d3_ksz9477_evb.dtb \
at91-sama5d3_xplained.dtb \
at91-dvk_som60.dtb \
at91-gatwick.dtb \
@@ -263,12 +264,14 @@
intel-ixp46x-ixdp465.dtb \
intel-ixp42x-adi-coyote.dtb \
intel-ixp42x-ixdpg425.dtb \
+ intel-ixp42x-goramo-multilink.dtb \
intel-ixp42x-iomega-nas100d.dtb \
intel-ixp42x-dlink-dsm-g600.dtb \
intel-ixp42x-gateworks-gw2348.dtb \
intel-ixp43x-gateworks-gw2358.dtb \
intel-ixp42x-netgear-wg302v2.dtb \
- intel-ixp42x-arcom-vulcan.dtb
+ intel-ixp42x-arcom-vulcan.dtb \
+ intel-ixp42x-gateway-7001.dtb
dtb-$(CONFIG_ARCH_KEYSTONE) += \
keystone-k2hk-evm.dtb \
keystone-k2l-evm.dtb \
@@ -483,6 +486,8 @@
imx6dl-icore-rqs.dtb \
imx6dl-lanmcu.dtb \
imx6dl-mamoj.dtb \
+ imx6dl-mba6a.dtb \
+ imx6dl-mba6b.dtb \
imx6dl-nit6xlite.dtb \
imx6dl-nitrogen6x.dtb \
imx6dl-phytec-mira-rdk-nand.dtb \
@@ -584,6 +589,8 @@
imx6q-kp-tpc.dtb \
imx6q-logicpd.dtb \
imx6q-marsboard.dtb \
+ imx6q-mba6a.dtb \
+ imx6q-mba6b.dtb \
imx6q-mccmon6.dtb \
imx6q-nitrogen6x.dtb \
imx6q-nitrogen6_max.dtb \
@@ -628,7 +635,9 @@
imx6q-wandboard.dtb \
imx6q-wandboard-revb1.dtb \
imx6q-wandboard-revd1.dtb \
+ imx6q-yapp4-crux.dtb \
imx6q-zii-rdu2.dtb \
+ imx6qp-mba6b.dtb \
imx6qp-nitrogen6_max.dtb \
imx6qp-nitrogen6_som2.dtb \
imx6qp-phytec-mira-rdk-nand.dtb \
@@ -641,6 +650,7 @@
imx6qp-tx6qp-8137-mb7.dtb \
imx6qp-vicutp.dtb \
imx6qp-wandboard-revd1.dtb \
+ imx6qp-yapp4-crux-plus.dtb \
imx6qp-zii-rdu2.dtb \
imx6s-dhcom-drc02.dtb
dtb-$(CONFIG_SOC_IMX6SL) += \
@@ -688,12 +698,14 @@
imx6ull-colibri-emmc-eval-v3.dtb \
imx6ull-colibri-eval-v3.dtb \
imx6ull-colibri-wifi-eval-v3.dtb \
+ imx6ull-jozacp.dtb \
imx6ull-myir-mys-6ulx-eval.dtb \
imx6ull-opos6uldev.dtb \
imx6ull-phytec-segin-ff-rdk-nand.dtb \
imx6ull-phytec-segin-ff-rdk-emmc.dtb \
imx6ull-phytec-segin-lc-rdk-nand.dtb \
- imx6ulz-14x14-evk.dtb
+ imx6ulz-14x14-evk.dtb \
+ imx6ulz-bsh-smm-m2.dtb
dtb-$(CONFIG_SOC_IMX7D) += \
imx7d-cl-som-imx7.dtb \
imx7d-colibri-aster.dtb \
@@ -954,6 +966,7 @@
ox810se-wd-mbwe.dtb \
ox820-cloudengines-pogoplug-series-3.dtb
dtb-$(CONFIG_ARCH_QCOM) += \
+ qcom-apq8016-sbc.dtb \
qcom-apq8026-lg-lenok.dtb \
qcom-apq8060-dragonboard.dtb \
qcom-apq8064-cm-qs600.dtb \
@@ -986,7 +999,8 @@
qcom-mdm9615-wp8548-mangoh-green.dtb \
qcom-sdx55-mtp.dtb \
qcom-sdx55-t55.dtb \
- qcom-sdx55-telit-fn980-tlb.dtb
+ qcom-sdx55-telit-fn980-tlb.dtb \
+ qcom-sdx65-mtp.dtb
dtb-$(CONFIG_ARCH_RDA) += \
rda8810pl-orangepi-2g-iot.dtb \
rda8810pl-orangepi-i96.dtb
@@ -1140,6 +1154,7 @@
stm32mp157a-microgea-stm32mp1-microdev2.0.dtb \
stm32mp157a-microgea-stm32mp1-microdev2.0-of7.dtb \
stm32mp157a-icore-stm32mp1-ctouch2.dtb \
+ stm32mp157a-icore-stm32mp1-ctouch2-of10.dtb \
stm32mp157a-icore-stm32mp1-edimm2.2.dtb \
stm32mp157a-stinger96.dtb \
stm32mp157c-dhcom-pdk2.dtb \
@@ -1304,6 +1319,7 @@
suniv-f1c100s-licheepi-nano.dtb
dtb-$(CONFIG_ARCH_TEGRA_2x_SOC) += \
tegra20-acer-a500-picasso.dtb \
+ tegra20-asus-tf101.dtb \
tegra20-harmony.dtb \
tegra20-colibri-eval-v3.dtb \
tegra20-colibri-iris.dtb \
@@ -1320,12 +1336,18 @@
tegra30-asus-nexus7-grouper-PM269.dtb \
tegra30-asus-nexus7-grouper-E1565.dtb \
tegra30-asus-nexus7-tilapia-E1565.dtb \
+ tegra30-asus-tf201.dtb \
+ tegra30-asus-tf300t.dtb \
+ tegra30-asus-tf300tg.dtb \
+ tegra30-asus-tf700t.dtb \
tegra30-beaver.dtb \
tegra30-cardhu-a02.dtb \
tegra30-cardhu-a04.dtb \
tegra30-colibri-eval-v3.dtb \
- tegra30-ouya.dtb
+ tegra30-ouya.dtb \
+ tegra30-pegatron-chagall.dtb
dtb-$(CONFIG_ARCH_TEGRA_114_SOC) += \
+ tegra114-asus-tf701t.dtb \
tegra114-dalmore.dtb \
tegra114-roth.dtb \
tegra114-tn7.dtb
@@ -1334,6 +1356,7 @@
tegra124-apalis-v1.2-eval.dtb \
tegra124-jetson-tk1.dtb \
tegra124-nyan-big.dtb \
+ tegra124-nyan-big-fhd.dtb \
tegra124-nyan-blaze.dtb \
tegra124-venice2.dtb
dtb-$(CONFIG_ARCH_U8500) += \
@@ -1457,6 +1480,7 @@
mt2701-evb.dtb \
mt6580-evbp1.dtb \
mt6589-aquaris5.dtb \
+ mt6589-fairphone-fp1.dtb \
mt6592-evb.dtb \
mt7623a-rfb-emmc.dtb \
mt7623a-rfb-nand.dtb \
@@ -1482,6 +1506,7 @@
aspeed-bmc-arm-stardragon4800-rep2.dtb \
aspeed-bmc-asrock-e3c246d4i.dtb \
aspeed-bmc-bytedance-g220a.dtb \
+ aspeed-bmc-facebook-bletchley.dtb \
aspeed-bmc-facebook-cloudripper.dtb \
aspeed-bmc-facebook-cmm.dtb \
aspeed-bmc-facebook-elbert.dtb \
@@ -1519,4 +1544,8 @@
aspeed-bmc-quanta-q71l.dtb \
aspeed-bmc-supermicro-x11spi.dtb \
aspeed-bmc-inventec-transformers.dtb \
- aspeed-bmc-tyan-s7106.dtb
+ aspeed-bmc-tyan-s7106.dtb \
+ aspeed-bmc-tyan-s8036.dtb \
+ aspeed-bmc-vegman-n110.dtb \
+ aspeed-bmc-vegman-rx20.dtb \
+ aspeed-bmc-vegman-sx20.dtb
diff --git a/arch/arm/boot/dts/am335x-bone-common.dtsi b/arch/arm/boot/dts/am335x-bone-common.dtsi
index 0ccdc7c..56ae509 100644
--- a/arch/arm/boot/dts/am335x-bone-common.dtsi
+++ b/arch/arm/boot/dts/am335x-bone-common.dtsi
@@ -399,6 +399,7 @@ &sham {
&rtc {
clocks = <&clk_32768_ck>, <&clk_24mhz_clkctrl AM3_CLK_24MHZ_CLKDIV32K_CLKCTRL 0>;
clock-names = "ext-clk", "int-clk";
+ system-power-controller;
};
&pruss_tm {
diff --git a/arch/arm/boot/dts/am335x-boneblack-common.dtsi b/arch/arm/boot/dts/am335x-boneblack-common.dtsi
index 10494c4..a7a8c61 100644
--- a/arch/arm/boot/dts/am335x-boneblack-common.dtsi
+++ b/arch/arm/boot/dts/am335x-boneblack-common.dtsi
@@ -22,10 +22,6 @@ &mmc2 {
non-removable;
};
-&rtc {
- system-power-controller;
-};
-
/ {
memory@80000000 {
device_type = "memory";
diff --git a/arch/arm/boot/dts/am335x-boneblue.dts b/arch/arm/boot/dts/am335x-boneblue.dts
index c6bb325..147c00d 100644
--- a/arch/arm/boot/dts/am335x-boneblue.dts
+++ b/arch/arm/boot/dts/am335x-boneblue.dts
@@ -341,7 +341,7 @@ i2c-gate {
#address-cells = <1>;
#size-cells = <0>;
ax8975@c {
- compatible = "ak,ak8975";
+ compatible = "asahi-kasei,ak8975";
reg = <0x0c>;
};
};
diff --git a/arch/arm/boot/dts/am335x-icev2.dts b/arch/arm/boot/dts/am335x-icev2.dts
index e5ce89c..5835c0c 100644
--- a/arch/arm/boot/dts/am335x-icev2.dts
+++ b/arch/arm/boot/dts/am335x-icev2.dts
@@ -512,3 +512,7 @@ ethphy1: ethernet-phy@3 {
&pruss_tm {
status = "okay";
};
+
+&rtc {
+ system-power-controller;
+};
diff --git a/arch/arm/boot/dts/am335x-osd3358-sm-red.dts b/arch/arm/boot/dts/am335x-osd3358-sm-red.dts
index 605b2a4..b2846cd 100644
--- a/arch/arm/boot/dts/am335x-osd3358-sm-red.dts
+++ b/arch/arm/boot/dts/am335x-osd3358-sm-red.dts
@@ -84,7 +84,7 @@ i2c-gate {
#address-cells = <1>;
#size-cells = <0>;
ax8975@c {
- compatible = "ak,ak8975";
+ compatible = "asahi-kasei,ak8975";
reg = <0x0c>;
};
};
diff --git a/arch/arm/boot/dts/am437x-cm-t43.dts b/arch/arm/boot/dts/am437x-cm-t43.dts
index 5ce8e68..3e33547 100644
--- a/arch/arm/boot/dts/am437x-cm-t43.dts
+++ b/arch/arm/boot/dts/am437x-cm-t43.dts
@@ -399,7 +399,7 @@ &tscadc {
tsc {
ti,wires = <4>;
ti,x-plate-resistance = <200>;
- ti,coordiante-readouts = <5>;
+ ti,coordinate-readouts = <5>;
ti,wire-config = <0x00 0x11 0x22 0x33>;
};
diff --git a/arch/arm/boot/dts/am437x-gp-evm.dts b/arch/arm/boot/dts/am437x-gp-evm.dts
index c2e4896..4416ddb 100644
--- a/arch/arm/boot/dts/am437x-gp-evm.dts
+++ b/arch/arm/boot/dts/am437x-gp-evm.dts
@@ -775,6 +775,14 @@ adc {
};
};
+&magadc {
+ status = "okay";
+
+ adc {
+ ti,adc-channels = <0 1 2 3 4 5 6 7>;
+ };
+};
+
&ecap0 {
status = "okay";
pinctrl-names = "default";
diff --git a/arch/arm/boot/dts/am437x-l4.dtsi b/arch/arm/boot/dts/am437x-l4.dtsi
index ba58e6b..8f2268c 100644
--- a/arch/arm/boot/dts/am437x-l4.dtsi
+++ b/arch/arm/boot/dts/am437x-l4.dtsi
@@ -2378,11 +2378,38 @@ hdq: hdq@0 {
};
target-module@4c000 { /* 0x4834c000, ap 114 72.0 */
- compatible = "ti,sysc";
- status = "disabled";
+ compatible = "ti,sysc-omap4", "ti,sysc";
+ reg = <0x4c000 0x4>,
+ <0x4c010 0x4>;
+ reg-names = "rev", "sysc";
+ ti,sysc-sidle = <SYSC_IDLE_FORCE>,
+ <SYSC_IDLE_NO>,
+ <SYSC_IDLE_SMART>;
+ clocks = <&l3s_clkctrl AM4_L3S_ADC1_CLKCTRL 0>;
+ clock-names = "fck";
#address-cells = <1>;
#size-cells = <1>;
ranges = <0x0 0x4c000 0x2000>;
+
+ magadc: magadc@0 {
+ compatible = "ti,am4372-magadc";
+ reg = <0x0 0x2000>;
+ interrupts = <GIC_SPI 115 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&adc_mag_fck>;
+ clock-names = "fck";
+ dmas = <&edma 54 0>, <&edma 55 0>;
+ dma-names = "fifo0", "fifo1";
+ status = "disabled";
+
+ mag {
+ compatible = "ti,am4372-mag";
+ };
+
+ adc {
+ #io-channel-cells = <1>;
+ compatible ="ti,am4372-adc";
+ };
+ };
};
target-module@80000 { /* 0x48380000, ap 123 42.0 */
diff --git a/arch/arm/boot/dts/am43xx-clocks.dtsi b/arch/arm/boot/dts/am43xx-clocks.dtsi
index 314fc59..66e892f 100644
--- a/arch/arm/boot/dts/am43xx-clocks.dtsi
+++ b/arch/arm/boot/dts/am43xx-clocks.dtsi
@@ -444,6 +444,13 @@ wdt1_fck: wdt1_fck@422c {
reg = <0x422c>;
};
+ adc_mag_fck: adc_mag_fck@424c {
+ #clock-cells = <0>;
+ compatible = "ti,mux-clock";
+ clocks = <&sys_clkin_ck>, <&dpll_per_m2_ck>;
+ reg = <0x424c>;
+ };
+
l3_gclk: l3_gclk {
#clock-cells = <0>;
compatible = "fixed-factor-clock";
diff --git a/arch/arm/boot/dts/armada-38x.dtsi b/arch/arm/boot/dts/armada-38x.dtsi
index 9b1a24c..df3c8d1 100644
--- a/arch/arm/boot/dts/armada-38x.dtsi
+++ b/arch/arm/boot/dts/armada-38x.dtsi
@@ -168,7 +168,7 @@ i2c1: i2c@11100 {
};
uart0: serial@12000 {
- compatible = "marvell,armada-38x-uart";
+ compatible = "marvell,armada-38x-uart", "ns16550a";
reg = <0x12000 0x100>;
reg-shift = <2>;
interrupts = <GIC_SPI 12 IRQ_TYPE_LEVEL_HIGH>;
@@ -178,7 +178,7 @@ uart0: serial@12000 {
};
uart1: serial@12100 {
- compatible = "marvell,armada-38x-uart";
+ compatible = "marvell,armada-38x-uart", "ns16550a";
reg = <0x12100 0x100>;
reg-shift = <2>;
interrupts = <GIC_SPI 13 IRQ_TYPE_LEVEL_HIGH>;
diff --git a/arch/arm/boot/dts/aspeed-bmc-ampere-mtjade.dts b/arch/arm/boot/dts/aspeed-bmc-ampere-mtjade.dts
index 3515d55..4c3c3f1 100644
--- a/arch/arm/boot/dts/aspeed-bmc-ampere-mtjade.dts
+++ b/arch/arm/boot/dts/aspeed-bmc-ampere-mtjade.dts
@@ -7,6 +7,50 @@ / {
model = "Ampere Mt. Jade BMC";
compatible = "ampere,mtjade-bmc", "aspeed,ast2500";
+ aliases {
+ /*
+ * i2c bus 50-57 assigned to NVMe slot 0-7
+ */
+ i2c50 = &nvmeslot_0;
+ i2c51 = &nvmeslot_1;
+ i2c52 = &nvmeslot_2;
+ i2c53 = &nvmeslot_3;
+ i2c54 = &nvmeslot_4;
+ i2c55 = &nvmeslot_5;
+ i2c56 = &nvmeslot_6;
+ i2c57 = &nvmeslot_7;
+
+ /*
+ * i2c bus 60-67 assigned to NVMe slot 8-15
+ */
+ i2c60 = &nvmeslot_8;
+ i2c61 = &nvmeslot_9;
+ i2c62 = &nvmeslot_10;
+ i2c63 = &nvmeslot_11;
+ i2c64 = &nvmeslot_12;
+ i2c65 = &nvmeslot_13;
+ i2c66 = &nvmeslot_14;
+ i2c67 = &nvmeslot_15;
+
+ /*
+ * i2c bus 70-77 assigned to NVMe slot 16-23
+ */
+ i2c70 = &nvmeslot_16;
+ i2c71 = &nvmeslot_17;
+ i2c72 = &nvmeslot_18;
+ i2c73 = &nvmeslot_19;
+ i2c74 = &nvmeslot_20;
+ i2c75 = &nvmeslot_21;
+ i2c76 = &nvmeslot_22;
+ i2c77 = &nvmeslot_23;
+
+ /*
+ * i2c bus 80-81 assigned to NVMe M2 slot 0-1
+ */
+ i2c80 = &nvme_m2_0;
+ i2c81 = &nvme_m2_1;
+ };
+
chosen {
stdout-path = &uart5;
bootargs = "console=ttyS4,115200 earlycon";
@@ -330,6 +374,15 @@ flash@0 {
m25p,fast-read;
label = "pnor";
/* spi-max-frequency = <100000000>; */
+ partitions {
+ compatible = "fixed-partitions";
+ #address-cells = <1>;
+ #size-cells = <1>;
+ uefi@400000 {
+ reg = <0x400000 0x1C00000>;
+ label = "pnor-uefi";
+ };
+ };
};
};
@@ -445,6 +498,220 @@ rtc@51 {
&i2c5 {
status = "okay";
+ i2c-mux@70 {
+ compatible = "nxp,pca9548";
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x70>;
+ i2c-mux-idle-disconnect;
+
+ nvmeslot_0_7: i2c@3 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x3>;
+ };
+ };
+
+ i2c-mux@71 {
+ compatible = "nxp,pca9548";
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x71>;
+ i2c-mux-idle-disconnect;
+
+ nvmeslot_8_15: i2c@4 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x4>;
+ };
+
+ nvmeslot_16_23: i2c@3 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x3>;
+ };
+
+ };
+
+ i2c-mux@72 {
+ compatible = "nxp,pca9545";
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x72>;
+ i2c-mux-idle-disconnect;
+
+ nvme_m2_0: i2c@0 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x0>;
+ };
+
+ nvme_m2_1: i2c@1 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x1>;
+ };
+ };
+};
+
+&nvmeslot_0_7 {
+ status = "okay";
+
+ i2c-mux@75 {
+ compatible = "nxp,pca9548";
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x75>;
+ i2c-mux-idle-disconnect;
+
+ nvmeslot_0: i2c@0 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x0>;
+ };
+ nvmeslot_1: i2c@1 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x1>;
+ };
+ nvmeslot_2: i2c@2 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x2>;
+ };
+ nvmeslot_3: i2c@3 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x3>;
+ };
+ nvmeslot_4: i2c@4 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x4>;
+ };
+ nvmeslot_5: i2c@5 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x5>;
+ };
+ nvmeslot_6: i2c@6 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x6>;
+ };
+ nvmeslot_7: i2c@7 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x7>;
+ };
+
+ };
+};
+
+&nvmeslot_8_15 {
+ status = "okay";
+
+ i2c-mux@75 {
+ compatible = "nxp,pca9548";
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x75>;
+ i2c-mux-idle-disconnect;
+
+ nvmeslot_8: i2c@0 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x0>;
+ };
+ nvmeslot_9: i2c@1 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x1>;
+ };
+ nvmeslot_10: i2c@2 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x2>;
+ };
+ nvmeslot_11: i2c@3 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x3>;
+ };
+ nvmeslot_12: i2c@4 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x4>;
+ };
+ nvmeslot_13: i2c@5 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x5>;
+ };
+ nvmeslot_14: i2c@6 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x6>;
+ };
+ nvmeslot_15: i2c@7 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x7>;
+ };
+ };
+};
+
+&nvmeslot_16_23 {
+ status = "okay";
+
+ i2c-mux@75 {
+ compatible = "nxp,pca9548";
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x75>;
+ i2c-mux-idle-disconnect;
+
+ nvmeslot_16: i2c@0 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x0>;
+ };
+ nvmeslot_17: i2c@1 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x1>;
+ };
+ nvmeslot_18: i2c@2 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x2>;
+ };
+ nvmeslot_19: i2c@3 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x3>;
+ };
+ nvmeslot_20: i2c@4 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x4>;
+ };
+ nvmeslot_21: i2c@5 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x5>;
+ };
+ nvmeslot_22: i2c@6 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x6>;
+ };
+ nvmeslot_23: i2c@7 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x7>;
+ };
+ };
};
&i2c6 {
diff --git a/arch/arm/boot/dts/aspeed-bmc-bytedance-g220a.dts b/arch/arm/boot/dts/aspeed-bmc-bytedance-g220a.dts
index 01dace8..0d1fb5c 100644
--- a/arch/arm/boot/dts/aspeed-bmc-bytedance-g220a.dts
+++ b/arch/arm/boot/dts/aspeed-bmc-bytedance-g220a.dts
@@ -260,6 +260,13 @@ flash@0 {
spi-max-frequency = <50000000>;
#include "openbmc-flash-layout-64.dtsi"
};
+ flash@1 {
+ status = "okay";
+ label = "alt-bmc";
+ m25p,fast-read;
+ spi-max-frequency = <50000000>;
+#include "openbmc-flash-layout-64-alt.dtsi"
+ };
};
&spi1 {
@@ -278,6 +285,11 @@ &adc {
status = "okay";
};
+&wdt2 {
+ status = "okay";
+ aspeed,alt-boot;
+};
+
&gpio {
status = "okay";
gpio-line-names =
diff --git a/arch/arm/boot/dts/aspeed-bmc-facebook-bletchley.dts b/arch/arm/boot/dts/aspeed-bmc-facebook-bletchley.dts
new file mode 100644
index 0000000..f973ea8
--- /dev/null
+++ b/arch/arm/boot/dts/aspeed-bmc-facebook-bletchley.dts
@@ -0,0 +1,756 @@
+// SPDX-License-Identifier: GPL-2.0+
+// Copyright (c) 2021 Facebook Inc.
+/dts-v1/;
+
+#include "aspeed-g6.dtsi"
+#include <dt-bindings/gpio/aspeed-gpio.h>
+#include <dt-bindings/usb/pd.h>
+
+/ {
+ model = "Facebook Bletchley BMC";
+ compatible = "facebook,bletchley-bmc", "aspeed,ast2600";
+
+ aliases {
+ serial4 = &uart5;
+ };
+
+ chosen {
+ bootargs = "console=ttyS4,57600n8";
+ };
+
+ memory@80000000 {
+ device_type = "memory";
+ reg = <0x80000000 0x80000000>;
+ };
+
+ iio-hwmon {
+ compatible = "iio-hwmon";
+ io-channels = <&adc0 0>, <&adc0 1>, <&adc0 2>, <&adc0 3>,
+ <&adc0 4>, <&adc0 5>, <&adc0 6>, <&adc0 7>,
+ <&adc1 0>, <&adc1 1>, <&adc1 2>, <&adc1 3>,
+ <&adc1 4>, <&adc1 5>, <&adc1 6>, <&adc1 7>;
+ };
+
+ spi_gpio: spi-gpio {
+ compatible = "spi-gpio";
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ gpio-sck = <&gpio0 ASPEED_GPIO(Z, 3) GPIO_ACTIVE_HIGH>;
+ gpio-mosi = <&gpio0 ASPEED_GPIO(Z, 4) GPIO_ACTIVE_HIGH>;
+ gpio-miso = <&gpio0 ASPEED_GPIO(Z, 5) GPIO_ACTIVE_HIGH>;
+ num-chipselects = <1>;
+ cs-gpios = <&gpio0 ASPEED_GPIO(Z, 0) GPIO_ACTIVE_LOW>;
+
+ tpmdev@0 {
+ compatible = "tcg,tpm_tis-spi";
+ spi-max-frequency = <33000000>;
+ reg = <0>;
+ };
+ };
+
+ switchphy: ethernet-phy@0 {
+ // Fixed link
+ };
+
+ leds {
+ compatible = "gpio-leds";
+
+ sys_log_id {
+ retain-state-shutdown;
+ default-state = "keep";
+ gpios = <&front_leds 0 GPIO_ACTIVE_HIGH>;
+ };
+ fan0_blue {
+ retain-state-shutdown;
+ default-state = "on";
+ gpios = <&fan_ioexp 8 GPIO_ACTIVE_HIGH>;
+ };
+ fan1_blue {
+ retain-state-shutdown;
+ default-state = "on";
+ gpios = <&fan_ioexp 9 GPIO_ACTIVE_HIGH>;
+ };
+ fan2_blue {
+ retain-state-shutdown;
+ default-state = "on";
+ gpios = <&fan_ioexp 10 GPIO_ACTIVE_HIGH>;
+ };
+ fan3_blue {
+ retain-state-shutdown;
+ default-state = "on";
+ gpios = <&fan_ioexp 11 GPIO_ACTIVE_HIGH>;
+ };
+ fan0_amber {
+ retain-state-shutdown;
+ default-state = "off";
+ gpios = <&fan_ioexp 12 GPIO_ACTIVE_HIGH>;
+ };
+ fan1_amber {
+ retain-state-shutdown;
+ default-state = "off";
+ gpios = <&fan_ioexp 13 GPIO_ACTIVE_HIGH>;
+ };
+ fan2_amber {
+ retain-state-shutdown;
+ default-state = "off";
+ gpios = <&fan_ioexp 14 GPIO_ACTIVE_HIGH>;
+ };
+ fan3_amber {
+ retain-state-shutdown;
+ default-state = "off";
+ gpios = <&fan_ioexp 15 GPIO_ACTIVE_HIGH>;
+ };
+ sled0_amber {
+ retain-state-shutdown;
+ default-state = "off";
+ gpios = <&sled0_leds 0 GPIO_ACTIVE_LOW>;
+ };
+ sled0_blue {
+ retain-state-shutdown;
+ default-state = "off";
+ gpios = <&sled0_leds 1 GPIO_ACTIVE_LOW>;
+ };
+ sled1_amber {
+ retain-state-shutdown;
+ default-state = "off";
+ gpios = <&sled1_leds 0 GPIO_ACTIVE_LOW>;
+ };
+ sled1_blue {
+ retain-state-shutdown;
+ default-state = "off";
+ gpios = <&sled1_leds 1 GPIO_ACTIVE_LOW>;
+ };
+ sled2_amber {
+ retain-state-shutdown;
+ default-state = "off";
+ gpios = <&sled2_leds 0 GPIO_ACTIVE_LOW>;
+ };
+ sled2_blue {
+ retain-state-shutdown;
+ default-state = "off";
+ gpios = <&sled2_leds 1 GPIO_ACTIVE_LOW>;
+ };
+ sled3_amber {
+ retain-state-shutdown;
+ default-state = "off";
+ gpios = <&sled3_leds 0 GPIO_ACTIVE_LOW>;
+ };
+ sled3_blue {
+ retain-state-shutdown;
+ default-state = "off";
+ gpios = <&sled3_leds 1 GPIO_ACTIVE_LOW>;
+ };
+ sled4_amber {
+ retain-state-shutdown;
+ default-state = "off";
+ gpios = <&sled4_leds 0 GPIO_ACTIVE_LOW>;
+ };
+ sled4_blue {
+ retain-state-shutdown;
+ default-state = "off";
+ gpios = <&sled4_leds 1 GPIO_ACTIVE_LOW>;
+ };
+ sled5_amber {
+ retain-state-shutdown;
+ default-state = "off";
+ gpios = <&sled5_leds 0 GPIO_ACTIVE_LOW>;
+ };
+ sled5_blue {
+ retain-state-shutdown;
+ default-state = "off";
+ gpios = <&sled5_leds 1 GPIO_ACTIVE_LOW>;
+ };
+ };
+};
+
+&mac2 {
+ status = "okay";
+ phy-mode = "rgmii";
+ phy-handle = <&switchphy>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_rgmii3_default>;
+
+ fixed-link {
+ speed = <1000>;
+ full-duplex;
+ };
+};
+
+&rtc {
+ status = "okay";
+};
+
+&fmc {
+ status = "okay";
+ flash@0 {
+ status = "okay";
+ m25p,fast-read;
+ label = "bmc";
+ spi-max-frequency = <50000000>;
+#include "openbmc-flash-layout-128.dtsi"
+ };
+};
+
+&spi2 {
+ status = "okay";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_spi2_default>;
+
+ flash@0 {
+ status = "okay";
+ m25p,fast-read;
+ label = "pnor";
+ spi-max-frequency = <100000000>;
+ };
+};
+
+&i2c0 {
+ status = "okay";
+ /* TODO: Add ADC INA230 */
+
+ mp5023@40 {
+ compatible = "mps,mp5023";
+ reg = <0x40>;
+ };
+
+ tmp421@4f {
+ compatible = "ti,tmp421";
+ reg = <0x4f>;
+ };
+
+ sled0_ioexp: pca9539@76 {
+ compatible = "nxp,pca9539";
+ reg = <0x76>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ gpio-controller;
+ #gpio-cells = <2>;
+
+ gpio-line-names =
+ "SLED0_MS_DETECT1","SLED0_VBUS_BMC_EN","SLED0_INA230_ALERT","SLED0_P12V_STBY_ALERT",
+ "SLED0_SSD_ALERT","SLED0_MS_DETECT0","SLED0_RST_CCG5","SLED0_FUSB302_INT",
+ "SLED0_MD_STBY_RESET","SLED0_MD_IOEXP_EN_FAULT","SLED0_MD_DIR","SLED0_MD_DECAY",
+ "SLED0_MD_MODE1","SLED0_MD_MODE2","SLED0_MD_MODE3","power-host0";
+ };
+
+ sled0_leds: pca9552@67 {
+ compatible = "nxp,pca9552";
+ reg = <0x67>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ gpio-controller;
+ #gpio-cells = <2>;
+
+ gpio-line-names =
+ "led-sled0-amber","led-sled0-blue","SLED0_RST_IOEXP","",
+ "","","","",
+ "","","","",
+ "","","","";
+ };
+
+ sled0_fusb302: typec-portc@22 {
+ compatible = "fcs,fusb302";
+ reg = <0x22>;
+
+ connector {
+ compatible = "usb-c-connector";
+ label = "USB-C";
+ power-role = "dual";
+ try-power-role = "sink";
+ data-role = "dual";
+ source-pdos = <PDO_FIXED(5000, 3000, PDO_FIXED_USB_COMM)>;
+ sink-pdos = <PDO_FIXED(5000, 3000, PDO_FIXED_USB_COMM)
+ PDO_VAR(3000, 12000, 3000)
+ PDO_PPS_APDO(3000, 11000, 3000)>;
+ op-sink-microwatt = <10000000>;
+ };
+ };
+};
+
+&i2c1 {
+ status = "okay";
+ /* TODO: Add ADC INA230 */
+
+ mp5023@40 {
+ compatible = "mps,mp5023";
+ reg = <0x40>;
+ };
+
+ tmp421@4f {
+ compatible = "ti,tmp421";
+ reg = <0x4f>;
+ };
+
+ sled1_ioexp: pca9539@76 {
+ compatible = "nxp,pca9539";
+ reg = <0x76>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ gpio-controller;
+ #gpio-cells = <2>;
+
+ gpio-line-names =
+ "SLED1_MS_DETECT1","SLED1_VBUS_BMC_EN","SLED1_INA230_ALERT","SLED1_P12V_STBY_ALERT",
+ "SLED1_SSD_ALERT","SLED1_MS_DETECT0","SLED1_RST_CCG5","SLED1_FUSB302_INT",
+ "SLED1_MD_STBY_RESET","SLED1_MD_IOEXP_EN_FAULT","SLED1_MD_DIR","SLED1_MD_DECAY",
+ "SLED1_MD_MODE1","SLED1_MD_MODE2","SLED1_MD_MODE3","power-host1";
+ };
+
+ sled1_leds: pca9552@67 {
+ compatible = "nxp,pca9552";
+ reg = <0x67>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ gpio-controller;
+ #gpio-cells = <2>;
+
+ gpio-line-names =
+ "led-sled1-amber","led-sled1-blue","SLED1_RST_IOEXP","",
+ "","","","",
+ "","","","",
+ "","","","";
+ };
+
+ sled1_fusb302: typec-portc@22 {
+ compatible = "fcs,fusb302";
+ reg = <0x22>;
+
+ connector {
+ compatible = "usb-c-connector";
+ label = "USB-C";
+ power-role = "dual";
+ try-power-role = "sink";
+ data-role = "dual";
+ source-pdos = <PDO_FIXED(5000, 3000, PDO_FIXED_USB_COMM)>;
+ sink-pdos = <PDO_FIXED(5000, 3000, PDO_FIXED_USB_COMM)
+ PDO_VAR(3000, 12000, 3000)
+ PDO_PPS_APDO(3000, 11000, 3000)>;
+ op-sink-microwatt = <10000000>;
+ };
+ };
+};
+
+&i2c1 {
+ status = "okay";
+};
+
+&i2c2 {
+ status = "okay";
+ /* TODO: Add ADC INA230 */
+
+ mp5023@40 {
+ compatible = "mps,mp5023";
+ reg = <0x40>;
+ };
+
+ tmp421@4f {
+ compatible = "ti,tmp421";
+ reg = <0x4f>;
+ };
+
+ sled2_ioexp: pca9539@76 {
+ compatible = "nxp,pca9539";
+ reg = <0x76>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ gpio-controller;
+ #gpio-cells = <2>;
+
+ gpio-line-names =
+ "SLED2_MS_DETECT1","SLED2_VBUS_BMC_EN","SLED2_INA230_ALERT","SLED2_P12V_STBY_ALERT",
+ "SLED2_SSD_ALERT","SLED2_MS_DETECT0","SLED2_RST_CCG5","SLED2_FUSB302_INT",
+ "SLED2_MD_STBY_RESET","SLED2_MD_IOEXP_EN_FAULT","SLED2_MD_DIR","SLED2_MD_DECAY",
+ "SLED2_MD_MODE1","SLED2_MD_MODE2","SLED2_MD_MODE3","power-host2";
+ };
+
+ sled2_leds: pca9552@67 {
+ compatible = "nxp,pca9552";
+ reg = <0x67>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ gpio-controller;
+ #gpio-cells = <2>;
+
+ gpio-line-names =
+ "led-sled2-amber","led-sled2-blue","SLED2_RST_IOEXP","",
+ "","","","",
+ "","","","",
+ "","","","";
+ };
+
+ sled2_fusb302: typec-portc@22 {
+ compatible = "fcs,fusb302";
+ reg = <0x22>;
+
+ connector {
+ compatible = "usb-c-connector";
+ label = "USB-C";
+ power-role = "dual";
+ try-power-role = "sink";
+ data-role = "dual";
+ source-pdos = <PDO_FIXED(5000, 3000, PDO_FIXED_USB_COMM)>;
+ sink-pdos = <PDO_FIXED(5000, 3000, PDO_FIXED_USB_COMM)
+ PDO_VAR(3000, 12000, 3000)
+ PDO_PPS_APDO(3000, 11000, 3000)>;
+ op-sink-microwatt = <10000000>;
+ };
+ };
+};
+
+&i2c3 {
+ status = "okay";
+ /* TODO: Add ADC INA230 */
+
+ mp5023@40 {
+ compatible = "mps,mp5023";
+ reg = <0x40>;
+ };
+
+ tmp421@4f {
+ compatible = "ti,tmp421";
+ reg = <0x4f>;
+ };
+
+ sled3_ioexp: pca9539@76 {
+ compatible = "nxp,pca9539";
+ reg = <0x76>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ gpio-controller;
+ #gpio-cells = <2>;
+
+ gpio-line-names =
+ "SLED3_MS_DETECT1","SLED3_VBUS_BMC_EN","SLED3_INA230_ALERT","SLED3_P12V_STBY_ALERT",
+ "SLED3_SSD_ALERT","SLED3_MS_DETECT0","SLED3_RST_CCG5","SLED3_FUSB302_INT",
+ "SLED3_MD_STBY_RESET","SLED3_MD_IOEXP_EN_FAULT","SLED3_MD_DIR","SLED3_MD_DECAY",
+ "SLED3_MD_MODE1","SLED3_MD_MODE2","SLED3_MD_MODE3","power-host3";
+ };
+
+ sled3_leds: pca9552@67 {
+ compatible = "nxp,pca9552";
+ reg = <0x67>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ gpio-controller;
+ #gpio-cells = <2>;
+
+ gpio-line-names =
+ "led-sled3-amber","led-sled3-blue","SLED3_RST_IOEXP","",
+ "","","","",
+ "","","","",
+ "","","","";
+ };
+
+ sled3_fusb302: typec-portc@22 {
+ compatible = "fcs,fusb302";
+ reg = <0x22>;
+
+ connector {
+ compatible = "usb-c-connector";
+ label = "USB-C";
+ power-role = "dual";
+ try-power-role = "sink";
+ data-role = "dual";
+ source-pdos = <PDO_FIXED(5000, 3000, PDO_FIXED_USB_COMM)>;
+ sink-pdos = <PDO_FIXED(5000, 3000, PDO_FIXED_USB_COMM)
+ PDO_VAR(3000, 12000, 3000)
+ PDO_PPS_APDO(3000, 11000, 3000)>;
+ op-sink-microwatt = <10000000>;
+ };
+ };
+};
+
+&i2c4 {
+ status = "okay";
+ /* TODO: Add ADC INA230 */
+
+ mp5023@40 {
+ compatible = "mps,mp5023";
+ reg = <0x40>;
+ };
+
+ tmp421@4f {
+ compatible = "ti,tmp421";
+ reg = <0x4f>;
+ };
+
+ sled4_ioexp: pca9539@76 {
+ compatible = "nxp,pca9539";
+ reg = <0x76>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ gpio-controller;
+ #gpio-cells = <2>;
+
+ gpio-line-names =
+ "SLED4_MS_DETECT1","SLED4_VBUS_BMC_EN","SLED4_INA230_ALERT","SLED4_P12V_STBY_ALERT",
+ "SLED4_SSD_ALERT","SLED4_MS_DETECT0","SLED4_RST_CCG5","SLED4_FUSB302_INT",
+ "SLED4_MD_STBY_RESET","SLED4_MD_IOEXP_EN_FAULT","SLED4_MD_DIR","SLED4_MD_DECAY",
+ "SLED4_MD_MODE1","SLED4_MD_MODE2","SLED4_MD_MODE3","power-host4";
+ };
+
+ sled4_leds: pca9552@67 {
+ compatible = "nxp,pca9552";
+ reg = <0x67>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ gpio-controller;
+ #gpio-cells = <2>;
+
+ gpio-line-names =
+ "led-sled4-amber","led-sled4-blue","SLED4_RST_IOEXP","",
+ "","","","",
+ "","","","",
+ "","","","";
+ };
+
+ sled4_fusb302: typec-portc@22 {
+ compatible = "fcs,fusb302";
+ reg = <0x22>;
+
+ connector {
+ compatible = "usb-c-connector";
+ label = "USB-C";
+ power-role = "dual";
+ try-power-role = "sink";
+ data-role = "dual";
+ source-pdos = <PDO_FIXED(5000, 3000, PDO_FIXED_USB_COMM)>;
+ sink-pdos = <PDO_FIXED(5000, 3000, PDO_FIXED_USB_COMM)
+ PDO_VAR(3000, 12000, 3000)
+ PDO_PPS_APDO(3000, 11000, 3000)>;
+ op-sink-microwatt = <10000000>;
+ };
+ };
+};
+
+&i2c5 {
+ status = "okay";
+ /* TODO: Add ADC INA230 */
+
+ mp5023@40 {
+ compatible = "mps,mp5023";
+ reg = <0x40>;
+ };
+
+ tmp421@4f {
+ compatible = "ti,tmp421";
+ reg = <0x4f>;
+ };
+
+ sled5_ioexp: pca9539@76 {
+ compatible = "nxp,pca9539";
+ reg = <0x76>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ gpio-controller;
+ #gpio-cells = <2>;
+
+ gpio-line-names =
+ "SLED5_MS_DETECT1","SLED5_VBUS_BMC_EN","SLED5_INA230_ALERT","SLED5_P12V_STBY_ALERT",
+ "SLED5_SSD_ALERT","SLED5_MS_DETECT0","SLED5_RST_CCG5","SLED5_FUSB302_INT",
+ "SLED5_MD_STBY_RESET","SLED5_MD_IOEXP_EN_FAULT","SLED5_MD_DIR","SLED5_MD_DECAY",
+ "SLED5_MD_MODE1","SLED5_MD_MODE2","SLED5_MD_MODE3","power-host5";
+ };
+
+ sled5_leds: pca9552@67 {
+ compatible = "nxp,pca9552";
+ reg = <0x67>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ gpio-controller;
+ #gpio-cells = <2>;
+
+ gpio-line-names =
+ "led-sled5-amber","led-sled5-blue","SLED5_RST_IOEXP","",
+ "","","","",
+ "","","","",
+ "","","","";
+ };
+
+ sled5_fusb302: typec-portc@22 {
+ compatible = "fcs,fusb302";
+ reg = <0x22>;
+
+ connector {
+ compatible = "usb-c-connector";
+ label = "USB-C";
+ power-role = "dual";
+ try-power-role = "sink";
+ data-role = "dual";
+ source-pdos = <PDO_FIXED(5000, 3000, PDO_FIXED_USB_COMM)>;
+ sink-pdos = <PDO_FIXED(5000, 3000, PDO_FIXED_USB_COMM)
+ PDO_VAR(3000, 12000, 3000)
+ PDO_PPS_APDO(3000, 11000, 3000)>;
+ op-sink-microwatt = <10000000>;
+ };
+ };
+};
+
+&i2c6 {
+ status = "okay";
+
+ eeprom@56 {
+ compatible = "atmel,24c64";
+ reg = <0x56>;
+ };
+
+ rtc@51 {
+ compatible = "nxp,pcf85263";
+ reg = <0x51>;
+ };
+};
+
+&i2c7 {
+ status = "okay";
+
+ eeprom@54 {
+ compatible = "atmel,24c64";
+ reg = <0x54>;
+ };
+};
+
+&i2c9 {
+ status = "okay";
+
+ tmp421@4f {
+ compatible = "ti,tmp421";
+ reg = <0x4f>;
+ };
+};
+
+&i2c10 {
+ status = "okay";
+
+ tmp421@4f {
+ compatible = "ti,tmp421";
+ reg = <0x4f>;
+ };
+
+ hdc1080@40 {
+ compatible = "ti,hdc1080";
+ reg = <0x40>;
+ };
+
+ front_leds: pca9552@67 {
+ compatible = "nxp,pca9552";
+ reg = <0x67>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ gpio-controller;
+ #gpio-cells = <2>;
+
+ gpio-line-names =
+ "led-fault-identify","power-p5v-stby-good",
+ "power-p1v0-dvdd-good","power-p1v0-avdd-good",
+ "","","","",
+ "","","","",
+ "","","","";
+ };
+};
+
+&i2c12 {
+ status = "okay";
+
+ adm1278@11 {
+ compatible = "adi,adm1278";
+ reg = <0x11>;
+ };
+
+ tmp421@4c {
+ compatible = "ti,tmp421";
+ reg = <0x4c>;
+ };
+
+ tmp421@4d {
+ compatible = "ti,tmp421";
+ reg = <0x4d>;
+ };
+
+ fan_ioexp: pca9552@67 {
+ compatible = "nxp,pca9552";
+ reg = <0x67>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ gpio-controller;
+ #gpio-cells = <2>;
+
+ gpio-line-names =
+ "presence-fan0","presence-fan1",
+ "presence-fan2","presence-fan3",
+ "power-fan0-good","power-fan1-good",
+ "power-fan2-good","power-fan3-good",
+ "","","","",
+ "","","","";
+ };
+};
+
+&i2c13 {
+ multi-master;
+ aspeed,hw-timeout-ms = <1000>;
+ status = "okay";
+};
+
+&gpio0 {
+ gpio-line-names =
+ /*A0-A7*/ "","","","","","","","",
+ /*B0-B7*/ "","","SEL_SPI2_MUX","SPI2_MUX1",
+ "SPI2_MUX2","SPI2_MUX3","","",
+ /*C0-C7*/ "","","","","","","","",
+ /*D0-D7*/ "","","","","","","","",
+ /*E0-E7*/ "","","","","","","","",
+ /*F0-F7*/ "","","","","","","","",
+ /*G0-G7*/ "","SWITCH_FRU_MUX","","","","","","",
+ /*H0-H7*/ "presence-riser1","presence-riser2",
+ "presence-sled0","presence-sled1",
+ "presence-sled2","presence-sled3",
+ "presence-sled4","presence-sled5",
+ /*I0-I7*/ "REV_ID0","","REV_ID1","REV_ID2",
+ "","","","",
+ /*J0-J7*/ "","","","","","","","",
+ /*K0-K7*/ "","","","","","","","",
+ /*L0-L7*/ "","","","","","","","",
+ /*M0-M7*/ "ALERT_SLED0","ALERT_SLED1",
+ "ALERT_SLED2","ALERT_SLED3",
+ "ALERT_SLED4","ALERT_SLED5",
+ "P12V_AUX_ALERT1","",
+ /*N0-N7*/ "","","","","","","","",
+ /*O0-O7*/ "","","","",
+ "","BOARD_ID0","BOARD_ID1","BOARD_ID2",
+ /*P0-P7*/ "","","","","","","","",
+ /*Q0-Q7*/ "","","","","","","","",
+ /*R0-R7*/ "","","","","","","","",
+ /*S0-S7*/ "","","","BAT_DETECT",
+ "BMC_BT_WP0","BMC_BT_WP1","","",
+ /*T0-T7*/ "","","","","","","","",
+ /*U0-U7*/ "","","","","","","","",
+ /*V0-V7*/ "","RST_BMC_MVL","","",
+ "USB2_SEL0_A","USB2_SEL1_A",
+ "USB2_SEL0_B","USB2_SEL1_B",
+ /*W0-W7*/ "RST_FRONT_IOEXP","","","","","","","",
+ /*X0-X7*/ "","","","","","","","",
+ /*Y0-Y7*/ "","","BSM_FLASH_LATCH","","","","","",
+ /*Z0-Z7*/ "","","","","","","","";
+};
+
+&adc0 {
+ vref = <1800>;
+ status = "okay";
+
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_adc0_default &pinctrl_adc1_default
+ &pinctrl_adc2_default &pinctrl_adc3_default
+ &pinctrl_adc4_default &pinctrl_adc5_default
+ &pinctrl_adc6_default &pinctrl_adc7_default>;
+};
+
+&adc1 {
+ vref = <2500>;
+ status = "okay";
+
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_adc8_default &pinctrl_adc9_default
+ &pinctrl_adc10_default &pinctrl_adc11_default
+ &pinctrl_adc12_default &pinctrl_adc13_default
+ &pinctrl_adc14_default &pinctrl_adc15_default>;
+};
diff --git a/arch/arm/boot/dts/aspeed-bmc-ibm-everest.dts b/arch/arm/boot/dts/aspeed-bmc-ibm-everest.dts
index f42e2d7..22c06ff 100644
--- a/arch/arm/boot/dts/aspeed-bmc-ibm-everest.dts
+++ b/arch/arm/boot/dts/aspeed-bmc-ibm-everest.dts
@@ -280,7 +280,7 @@ &gpio0 {
/*L0-L7*/ "","","","","","","","",
/*M0-M7*/ "","","","","","","","",
/*N0-N7*/ "","","","","","","","",
- /*O0-O7*/ "","","","","","","","",
+ /*O0-O7*/ "","","","usb-power","","","","",
/*P0-P7*/ "","","","","led-pcieslot-power","","","",
/*Q0-Q7*/ "","","regulator-standby-faulted","","","","","",
/*R0-R7*/ "bmc-tpm-reset","power-chassis-control","power-chassis-good","","","I2C_FLASH_MICRO_N","","",
@@ -292,6 +292,12 @@ &gpio0 {
/*X0-X7*/ "","","","","","","","",
/*Y0-Y7*/ "","","","","","","","",
/*Z0-Z7*/ "","","","","","","","";
+
+ usb_power {
+ gpio-hog;
+ gpios = <ASPEED_GPIO(O, 3) GPIO_ACTIVE_LOW>;
+ output-high;
+ };
};
&i2c0 {
@@ -1881,6 +1887,11 @@ eeprom@52 {
&i2c12 {
status = "okay";
+
+ tpm@2e {
+ compatible = "nuvoton,npct75x", "tcg,tpm-tis-i2c";
+ reg = <0x2e>;
+ };
};
&i2c13 {
@@ -2375,6 +2386,10 @@ i2c15mux2chn3: i2c@3 {
};
};
+&ehci0 {
+ status = "okay";
+};
+
&ehci1 {
status = "okay";
};
diff --git a/arch/arm/boot/dts/aspeed-bmc-ibm-rainier.dts b/arch/arm/boot/dts/aspeed-bmc-ibm-rainier.dts
index 866f32c..c479742 100644
--- a/arch/arm/boot/dts/aspeed-bmc-ibm-rainier.dts
+++ b/arch/arm/boot/dts/aspeed-bmc-ibm-rainier.dts
@@ -275,6 +275,12 @@ i2c3_mux_oe_n {
output-high;
line-name = "I2C3_MUX_OE_N";
};
+
+ usb_power {
+ gpio-hog;
+ gpios = <ASPEED_GPIO(O, 3) GPIO_ACTIVE_LOW>;
+ output-high;
+ };
};
&emmc_controller {
@@ -2061,6 +2067,11 @@ eeprom@51 {
&i2c12 {
status = "okay";
+ tpm@2e {
+ compatible = "nuvoton,npct75x";
+ reg = <0x2e>;
+ };
+
eeprom@50 {
compatible = "atmel,24c64";
reg = <0x50>;
diff --git a/arch/arm/boot/dts/aspeed-bmc-tyan-s7106.dts b/arch/arm/boot/dts/aspeed-bmc-tyan-s7106.dts
index 68f332e..aff27c1 100644
--- a/arch/arm/boot/dts/aspeed-bmc-tyan-s7106.dts
+++ b/arch/arm/boot/dts/aspeed-bmc-tyan-s7106.dts
@@ -3,6 +3,7 @@
#include "aspeed-g5.dtsi"
#include <dt-bindings/gpio/aspeed-gpio.h>
+#include <dt-bindings/interrupt-controller/irq.h>
/ {
model = "Tyan S7106 BMC";
@@ -127,8 +128,23 @@ &uart5 {
status = "okay";
};
+&uart_routing {
+ status = "okay";
+};
+
&vuart {
status = "okay";
+
+ /* We enable the VUART here, but leave it in a state that does
+ * not interfere with the SuperIO. The goal is to have both the
+ * VUART and the SuperIO available and decide at runtime whether
+ * the VUART should actually be used. For that reason, configure
+ * an "invalid" IO address and an IRQ that is not used by the
+ * BMC.
+ */
+
+ aspeed,lpc-io-reg = <0xffff>;
+ aspeed,lpc-interrupts = <15 IRQ_TYPE_LEVEL_HIGH>;
};
&lpc_ctrl {
@@ -213,6 +229,30 @@ &i2c0 {
nct7802@28 {
compatible = "nuvoton,nct7802";
reg = <0x28>;
+
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ channel@0 { /* LTD */
+ reg = <0>;
+ };
+
+ channel@1 { /* RTD1 */
+ reg = <1>;
+ sensor-type = "temperature";
+ temperature-mode = "thermistor";
+ };
+
+ channel@2 { /* RTD2 */
+ reg = <2>;
+ sensor-type = "temperature";
+ temperature-mode = "thermistor";
+ };
+
+ channel@3 { /* RTD3 */
+ reg = <3>;
+ sensor-type = "temperature";
+ };
};
/* Also connected to:
diff --git a/arch/arm/boot/dts/aspeed-bmc-tyan-s8036.dts b/arch/arm/boot/dts/aspeed-bmc-tyan-s8036.dts
new file mode 100644
index 0000000..708ee78
--- /dev/null
+++ b/arch/arm/boot/dts/aspeed-bmc-tyan-s8036.dts
@@ -0,0 +1,470 @@
+// SPDX-License-Identifier: GPL-2.0+
+/dts-v1/;
+
+#include "aspeed-g5.dtsi"
+#include <dt-bindings/gpio/aspeed-gpio.h>
+#include <dt-bindings/interrupt-controller/irq.h>
+
+/ {
+ model = "Tyan S8036 BMC";
+ compatible = "tyan,s8036-bmc", "aspeed,ast2500";
+
+ chosen {
+ stdout-path = &uart5;
+ bootargs = "console=ttyS4,115200 earlycon";
+ };
+
+ memory@80000000 {
+ device_type = "memory";
+ reg = <0x80000000 0x20000000>;
+ };
+
+ reserved-memory {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges;
+
+ p2a_memory: region@987f0000 {
+ no-map;
+ reg = <0x987f0000 0x00010000>; /* 64KB */
+ };
+
+ vga_memory: framebuffer@9f000000 {
+ no-map;
+ reg = <0x9f000000 0x01000000>; /* 16M */
+ };
+
+ gfx_memory: framebuffer {
+ size = <0x01000000>; /* 16M */
+ alignment = <0x01000000>;
+ compatible = "shared-dma-pool";
+ reusable;
+ };
+ };
+
+ leds {
+ compatible = "gpio-leds";
+
+ identify {
+ gpios = <&gpio ASPEED_GPIO(A, 2) GPIO_ACTIVE_LOW>;
+ };
+
+ heartbeat {
+ gpios = <&gpio ASPEED_GPIO(E, 7) GPIO_ACTIVE_LOW>;
+ };
+ };
+
+ iio-hwmon {
+ compatible = "iio-hwmon";
+ io-channels = <&adc 0>, <&adc 1>, <&adc 2>, <&adc 3>,
+ <&adc 4>, <&adc 5>, <&adc 6>, <&adc 7>,
+ <&adc 8>, <&adc 9>, <&adc 10>, <&adc 11>,
+ <&adc 12>, <&adc 13>, <&adc 14>;
+ };
+
+ iio-hwmon-battery {
+ compatible = "iio-hwmon";
+ io-channels = <&adc 15>;
+ };
+};
+
+&fmc {
+ status = "okay";
+ flash@0 {
+ label = "bmc";
+ status = "okay";
+ m25p,fast-read;
+#include "openbmc-flash-layout.dtsi"
+ };
+};
+
+&spi1 {
+ status = "okay";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_spi1_default>;
+
+ flash@0 {
+ status = "okay";
+ label = "pnor";
+ m25p,fast-read;
+ };
+};
+
+&uart1 {
+ /* Rear RS-232 connector */
+ status = "okay";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_txd1_default
+ &pinctrl_rxd1_default>;
+};
+
+&uart2 {
+ /* RS-232 connector on header */
+ status = "okay";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_txd2_default
+ &pinctrl_rxd2_default>;
+};
+
+&uart3 {
+ /* Alternative to vuart to internally connect (route) to uart1
+ * when vuart cannot be used due to BIOS limitations.
+ */
+ status = "okay";
+};
+
+&uart4 {
+ /* Alternative to vuart to internally connect (route) to the
+ * external port usually used by uart1 when vuart cannot be
+ * used due to BIOS limitations.
+ */
+ status = "okay";
+};
+
+&uart5 {
+ /* BMC "debug" (console) UART; connected to RS-232 connector
+ * on header; selectable via jumpers as alternative to uart2
+ */
+ status = "okay";
+};
+
+&uart_routing {
+ status = "okay";
+};
+
+&vuart {
+ status = "okay";
+
+ /* We enable the VUART here, but leave it in a state that does
+ * not interfere with the SuperIO. The goal is to have both the
+ * VUART and the SuperIO available and decide at runtime whether
+ * the VUART should actually be used. For that reason, configure
+ * an "invalid" IO address and an IRQ that is not used by the
+ * BMC.
+ */
+ aspeed,lpc-io-reg = <0xffff>;
+ aspeed,lpc-interrupts = <15 IRQ_TYPE_LEVEL_HIGH>;
+};
+
+&lpc_ctrl {
+ status = "okay";
+};
+
+&p2a {
+ status = "okay";
+ memory-region = <&p2a_memory>;
+};
+
+&lpc_snoop {
+ status = "okay";
+ snoop-ports = <0x80>;
+};
+
+&adc {
+ status = "okay";
+};
+
+&vhub {
+ status = "okay";
+};
+
+&pwm_tacho {
+ status = "okay";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_pwm0_default
+ &pinctrl_pwm1_default
+ &pinctrl_pwm3_default
+ &pinctrl_pwm4_default>;
+
+ /* CPU fan */
+ fan@0 {
+ reg = <0x00>;
+ aspeed,fan-tach-ch = /bits/ 8 <0x00>;
+ };
+
+ /* PWM group for chassis fans #1, #2, #3 and #4 */
+ fan@2 {
+ reg = <0x03>;
+ aspeed,fan-tach-ch = /bits/ 8 <0x02>;
+ };
+
+ fan@3 {
+ reg = <0x03>;
+ aspeed,fan-tach-ch = /bits/ 8 <0x03>;
+ };
+
+ fan@4 {
+ reg = <0x03>;
+ aspeed,fan-tach-ch = /bits/ 8 <0x04>;
+ };
+
+ fan@5 {
+ reg = <0x03>;
+ aspeed,fan-tach-ch = /bits/ 8 <0x05>;
+ };
+
+ /* PWM group for chassis fans #5 and #6 */
+ fan@6 {
+ reg = <0x04>;
+ aspeed,fan-tach-ch = /bits/ 8 <0x06>;
+ };
+
+ fan@7 {
+ reg = <0x04>;
+ aspeed,fan-tach-ch = /bits/ 8 <0x07>;
+ };
+};
+
+&i2c0 {
+ /* Directly connected to Sideband-Temperature Sensor Interface (APML) */
+ status = "okay";
+};
+
+&i2c1 {
+ /* Directly connected to IPMB HDR. */
+ status = "okay";
+};
+
+&i2c2 {
+ status = "okay";
+
+ /* BMC EEPROM, incl. mainboard FRU */
+ eeprom@50 {
+ compatible = "atmel,24c256";
+ reg = <0x50>;
+ };
+ /* Also connected to:
+ * - BCM5720
+ * - FPGA
+ * - FAN HDR
+ * - FPIO HDR
+ */
+};
+
+&i2c3 {
+ status = "okay";
+
+ /* PSU1 FRU @ 0xA0 */
+ eeprom@50 {
+ compatible = "atmel,24c02";
+ reg = <0x50>;
+ };
+
+ /* PSU2 FRU @ 0xA2 */
+ eeprom@51 {
+ compatible = "atmel,24c02";
+ reg = <0x51>;
+ };
+
+ /* PSU1 @ 0xB0 */
+ power-supply@58 {
+ compatible = "pmbus";
+ reg = <0x58>;
+ };
+
+ /* PSU2 @ 0xB2 */
+ power-supply@59 {
+ compatible = "pmbus";
+ reg = <0x59>;
+ };
+
+};
+
+&i2c4 {
+ status = "okay";
+};
+
+&i2c5 {
+ status = "okay";
+ /* Hardware monitor with temperature sensors */
+ nct7802@28 {
+ compatible = "nuvoton,nct7802";
+ reg = <0x28>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ channel@0 { /* LTD */
+ reg = <0>;
+ status = "okay";
+ };
+
+ channel@1 { /* RTD1 */
+ reg = <1>;
+ status = "okay";
+ sensor-type = "temperature";
+ temperature-mode = "thermistor";
+ };
+
+ channel@2 { /* RTD2 */
+ reg = <2>;
+ status = "okay";
+ sensor-type = "temperature";
+ temperature-mode = "thermistor";
+ };
+
+ channel@3 { /* RTD3 */
+ reg = <3>;
+ status = "okay";
+ sensor-type = "temperature";
+ };
+ };
+
+ /* Also connected to:
+ * - PCA9544
+ * - CLK BUFF
+ * - OCP FRU
+ */
+};
+
+&i2c6 {
+ status = "okay";
+ /* Connected to:
+ * - PCA9548 @0xE0
+ * - PCA9548 @0xE2
+ * - PCA9544 @0xE4
+ */
+};
+
+&i2c7 {
+ status = "okay";
+
+ /* Connected to:
+ * - PCH SMBUS #4
+ */
+};
+
+&i2c8 {
+ status = "okay";
+
+ /* Not connected */
+};
+
+&mac0 {
+ status = "okay";
+ use-ncsi;
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_rmii1_default>;
+};
+
+&mac1 {
+ status = "okay";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_rgmii2_default &pinctrl_mdio2_default>;
+};
+
+&ibt {
+ status = "okay";
+};
+
+&kcs1 {
+ status = "okay";
+ aspeed,lpc-io-reg = <0xca8>;
+};
+
+&kcs3 {
+ status = "okay";
+ aspeed,lpc-io-reg = <0xca2>;
+};
+
+/* Enable BMC VGA output to show an early (pre-BIOS) boot screen */
+&gfx {
+ status = "okay";
+ memory-region = <&gfx_memory>;
+};
+
+/* We're following the GPIO naming as defined at
+ * https://github.com/openbmc/docs/blob/master/designs/device-tree-gpio-naming.md.
+ *
+ * Notes on led-identify and id-button:
+ * - A physical button is connected to id-button which
+ * triggers the clock on a D flip-flop. The /Q output of the
+ * flip-flop drives its D input.
+ * - The flip-flop's Q output drives led-identify which is
+ * connected to LEDs.
+ * - With that, every button press toggles the LED between on and off.
+ *
+ * Notes on power-, reset- and nmi- button and control:
+ * - The -button signals can be used to monitor physical buttons.
+ * - The -control signals can be used to actuate the specific
+ * operation.
+ * - In hardware, the -button signals are connected to the -control
+ * signals through drivers with the -control signals being
+ * protected through diodes.
+ */
+&gpio {
+ status = "okay";
+ gpio-line-names =
+ /*A0*/ "",
+ /*A1*/ "",
+ /*A2*/ "led-identify", /* in/out: BMC_CHASSIS_ID_LED_L */
+ /*A3*/ "",
+ /*A4*/ "",
+ /*A5*/ "",
+ /*A6*/ "",
+ /*A7*/ "",
+ /*B0-B7*/ "","","","","","","","",
+ /*C0-C7*/ "","","","","","","","",
+ /*D0*/ "",
+ /*D1*/ "",
+ /*D2*/ "power-chassis-good", /* in: PWR_GOOD_LED -- Check if this is Z3?*/
+ /*D3*/ "platform-reset", /* in: RESET_LED_L */
+ /*D4*/ "",
+ /*D5*/ "",
+ /*D6*/ "",
+ /*D7*/ "",
+ /*E0*/ "power-button", /* in: BMC_SYS_MON_PWR_BTN_L */
+ /*E1*/ "power-chassis-control", /* out: BMC_ASSERT_PWR_BTN */
+ /*E2*/ "reset-button", /* in: BMC_SYS_MOS_RST_BTN_L*/
+ /*E3*/ "reset-control", /* out: BMC_ASSERT_RST_BTN */
+ /*E4*/ "nmi-button", /* in: BMC_SYS_MON_NMI_BTN_L */
+ /*E5*/ "nmi-control", /* out: BMC_ASSERT_NMI_BTN */
+ /*E6*/ "TSI_RESERT",
+ /*E7*/ "led-heartbeat", /* out: BMC_GPIOE7 */
+ /*F0*/ "",
+ /*F1*/ "clear-cmos-control", /* out: BMC_ASSERT_CLR_CMOS_L */
+ /*F2*/ "",
+ /*F3*/ "",
+ /*F4*/ "led-fault", /* out: BMC_HWM_FAULT_LED_L */
+ /*F5*/ "BMC_SYS_FAULT_LED_L",
+ /*F6*/ "BMC_ASSERT_BIOS_WP_L",
+ /*F7*/ "",
+ /*G0-G7*/ "","","","","","","","",
+ /*H0-H7*/ "","","","","","","","",
+ /*I0-I7*/ "","","","","","","","",
+ /*J0-J7*/ "","","","","","","","",
+ /*K0-K7*/ "","","","","","","","",
+ /*L0-L7*/ "","","","","","","","",
+ /*M0-M7*/ "","","","","","","","",
+ /*N0-N7*/ "","","","","","","","",
+ /*O0-O7*/ "","","","","","","","",
+ /*P0-P7*/ "","","","","","","","",
+ /*Q0*/ "",
+ /*Q1*/ "",
+ /*Q2*/ "",
+ /*Q3*/ "",
+ /*Q4*/ "",
+ /*Q5*/ "",
+ /*Q6*/ "id-button", /* in: BMC_CHASSIS_ID_BTN_L */
+ /*Q7*/ "",
+ /*R0-R7*/ "","","","","","","","",
+ /*S0-S7*/ "","","","","","","","",
+ /*T0-T7*/ "","","","","","","","",
+ /*U0-U7*/ "","","","","","","","",
+ /*V0-V7*/ "","","","","","","","",
+ /*W0-W7*/ "","","","","","","","",
+ /*X0-X7*/ "","","","","","","","",
+ /*Y0-Y7*/ "","","","","","","","",
+ /*Z0-Z2*/ "","","",
+ /*Z3*/ "post-complete", /* BMC_SYS_MON_PWROK */
+ /*Z4-Z7*/ "","","","",
+ /*AA0*/ "",
+ /*AA1*/ "",
+ /*AA2*/ "",
+ /*AA3*/ "",
+ /*AA4*/ "",
+ /*AA5*/ "",
+ /*AA6*/ "",
+ /*AA7*/ "BMC_ASSERT_BMC_READY",
+ /*AB0*/ "BMC_SPD_SEL",
+ /*AB1-AB7*/ "","","","","","","";
+};
diff --git a/arch/arm/boot/dts/aspeed-bmc-vegman-n110.dts b/arch/arm/boot/dts/aspeed-bmc-vegman-n110.dts
new file mode 100644
index 0000000..2431926
--- /dev/null
+++ b/arch/arm/boot/dts/aspeed-bmc-vegman-n110.dts
@@ -0,0 +1,149 @@
+// SPDX-License-Identifier: GPL-2.0+
+// Copyright (C) 2021 YADRO
+/dts-v1/;
+
+#include "aspeed-bmc-vegman.dtsi"
+
+/ {
+ model = "YADRO VEGMAN N110 BMC";
+ compatible = "yadro,vegman-n110-bmc", "aspeed,ast2500";
+};
+
+&gpio {
+ status = "okay";
+ gpio-line-names =
+ /*A0-A7*/ "CHASSIS_INTRUSION","CASE_OPEN_FAULT_RST","","","SPEAKER_BMC","FM_FORCE_BMC_UPDATE","","",
+ /*B0-B7*/ "","","","","","","","",
+ /*C0-C7*/ "","","","","","","","",
+ /*D0-D7*/ "","","","","","","","",
+ /*E0-E7*/ "RESET_BUTTON","RESET_OUT","POWER_BUTTON","POWER_OUT","","","","",
+ /*F0-F7*/ "NMI_OUT","PCIE_NIC_ALERT","","","SKT0_FAULT_LED","","RST_RGMII_PHYRST_DNP","",
+ /*G0-G7*/ "CPU_ERR2","CPU_CATERR","PCH_BMC_THERMTRIP","","IRQ_NMI_EVENT","","","",
+ /*H0-H7*/ "PWRGD_P3V3_RISER1","PWRGD_P3V3_RISER2","PWRGD_P3V3_RISER3","","MIO_BIOS_SEL","_SPI_FLASH_HOLD","_SPI_FLASH_WP","FM_240VA_STATUS",
+ /*I0-I7*/ "","","","","","","","",
+ /*J0-J7*/ "","","","","","","","",
+ /*K0-K7*/ "","","","","","","","",
+ /*L0-L7*/ "","","","","","","","",
+ /*M0-M7*/ "","","","","","","","",
+ /*N0-N7*/ "","","","","","","","",
+ /*O0-O7*/ "","","","","","","","_SPI2_BMC_CS_SEL",
+ /*P0-P7*/ "","","","","","","","",
+ /*Q0-Q7*/ "","","","","","","","",
+ /*R0-R7*/ "_SPI_RMM4_LITE_CS","","","","","","","",
+ /*S0-S7*/ "_SPI2_BMC_CS1","","","IRQ_SML0_ALERT_MUX","FP_LED_STATUS_GREEN","FP_LED_STATUS_AMBER","FP_ID_LED","",
+ /*T0-T7*/ "","","","","","","","",
+ /*U0-U7*/ "","","","","","","","",
+ /*V0-V7*/ "","","","","","","","",
+ /*W0-W7*/ "","","","","","","","",
+ /*X0-X7*/ "","","","","","","","",
+ /*Y0-Y7*/ "SIO_S3","SIO_S5","","SIO_ONCONTROL","","","","",
+ /*Z0-Z7*/ "FM_BMC_PWR_BTN","SIO_POWER_GOOD","FM_BMC_PWRBTN_OUT","FM_BMC_PCH_SCI_LPC","","","","",
+ /*AA0-AA7*/ "","IRQ_SML1_PMBUS_ALERT","FM_PVCCIN_CPU0_PWR_IN_ALERT","FM_PVCCIN_CPU1_PWR_IN_ALERT","BMC_SYS_PWR_FAULT","BMC_SYS_PWR_OK","SMI","POST_COMPLETE",
+ /*AB0-AB7*/ "FM_CPU_BMCINIT","NMI_BUTTON","ID_BUTTON","PS_PWROK","","","","",
+ /*AC0-AC7*/ "","","","","","","","";
+};
+
+&sgpio {
+ ngpios = <80>;
+ bus-frequency = <2000000>;
+ status = "okay";
+ /* SGPIO lines. even: input, odd: output */
+ gpio-line-names =
+ /*A0-A7*/ "CPU1_PRESENCE","","CPU1_THERMTRIP","","CPU1_VRHOT","","CPU1_FIVR_FAULT","","CPU1_MEM_ABCD_VRHOT","","CPU1_MEM_EFGH_VRHOT","","","","","",
+ /*B0-B7*/ "CPU1_MISMATCH","","CPU1_MEM_THERM_EVENT","","CPU2_PRESENCE","","CPU2_THERMTRIP","","CPU2_VRHOT","","CPU2_FIVR_FAULT","","CPU2_MEM_ABCD_VRHOT","","CPU2_MEM_EFGH_VRHOT","",
+ /*C0-C7*/ "","","","","CPU2_MISMATCH","","CPU2_MEM_THERM_EVENT","","","","","","","","","",
+ /*D0-D7*/ "","","","","","","","","","","","","","","","",
+ /*E0-E7*/ "","","","","","","","","","","","","","","","",
+ /*F0-F7*/ "SGPIO_PLD_MINOR_REV_BIT0","","SGPIO_PLD_MINOR_REV_BIT1","","SGPIO_PLD_MINOR_REV_BIT2","","SGPIO_PLD_MINOR_REV_BIT3","","SGPIO_PLD_MAJOR_REV_BIT0","","SGPIO_PLD_MAJOR_REV_BIT1","","SGPIO_PLD_MAJOR_REV_BIT2","","SGPIO_PLD_MAJOR_REV_BIT3","",
+ /*G0-G7*/ "MAIN_PLD_MINOR_REV_BIT0","","MAIN_PLD_MINOR_REV_BIT1","","MAIN_PLD_MINOR_REV_BIT2","","MAIN_PLD_MINOR_REV_BIT3","","MAIN_PLD_MAJOR_REV_BIT0","","MAIN_PLD_MAJOR_REV_BIT1","","MAIN_PLD_MAJOR_REV_BIT2","","MAIN_PLD_MAJOR_REV_BIT3","",
+ /*H0-H7*/ "","","","","","","","","","","","","","","","",
+ /*I0-I7*/ "","","","","","","","","","","","","","","","",
+ /*J0-J7*/ "","","","","","","","","","","","","","","","";
+};
+
+&i2c11 {
+ /* SMB_BMC_MGMT_LVC3 */
+ gpio@21 {
+ compatible = "nxp,pcal9535";
+ reg = <0x21>;
+ gpio-controller;
+ #gpio-cells = <2>;
+ gpio-line-names =
+ /*IO0.0-0.7*/ "", "", "", "", "", "", "PE_PCH_SCR_CLKREQ", "",
+ /*IO1.0-1.7*/ "", "PE_PCH_MEZ_PRSNT", "PE_PCH_MEZ_PRSNT_", "NIC_4_PE_PRSNT", "NIC_3_PE_PRSNT", "NIC_2_PE_PRSNT", "NIC_1_PE_PRSNT", "";
+ };
+ gpio@27 {
+ compatible = "nxp,pca9698";
+ reg = <0x27>;
+ gpio-controller;
+ #gpio-cells = <2>;
+ gpio-line-names =
+ /*IO0.0-0.7*/ "PWRGD_PS_PWROK", "PWRGD_DSW_PWROK", "PWRGD_P5V_AUX", "PWRGD_P3V3_AUX", "PWRGD_P5V", "PWRGD_P3V3", "PWRGD_P1V8_PCH_AUX", "PWRGD_PCH_PVNN_AUX",
+ /*IO1.0-1.7*/ "PWRGD_P1V05_PCH_AUX", "PWRGD_PCH_AUX_VRS", "PWRGD_PVCCIN_CPU0", "PWRGD_PVCCSA_CPU0", "PWRGD_PVCCIO_CPU0", "PWRGD_PVMCP_CPU0", "PWRGD_P1V0_CPU0", "PWRGD_PVDDQ_ABC_CPU0",
+ /*IO2.0-2.7*/ "PWRGD_PVPP_ABC_CPU0", "PWRGD_PVTT_ABC_CPU0", "PWRGD_PVDDQ_DEF_CPU0", "PWRGD_PVPP_DEF_CPU0", "PWRGD_PVTT_DEF_CPU0", "", "", "",
+ /*IO3.0-3.7*/ "", "", "", "", "", "", "", "",
+ /*IO4.0-4.7*/ "", "", "", "", "", "", "", "";
+ };
+};
+
+&i2c13 {
+ /* SMB_PCIE2_STBY_LVC3 */
+ mux-expa@73 {
+ compatible = "nxp,pca9545";
+ reg = <0x73>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ i2c-mux-idle-disconnect;
+ };
+ mux-sata@71 {
+ compatible = "nxp,pca9543";
+ reg = <0x71>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ i2c-mux-idle-disconnect;
+ };
+};
+
+&i2c2 {
+ /* SMB_PCIE_STBY_LVC3 */
+ mux-expb@71 {
+ compatible = "nxp,pca9545";
+ reg = <0x71>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ i2c-mux-idle-disconnect;
+ };
+};
+
+&pwm_tacho {
+ status = "okay";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_pwm0_default &pinctrl_pwm1_default
+ &pinctrl_pwm2_default &pinctrl_pwm3_default
+ &pinctrl_pwm4_default &pinctrl_pwm5_default>;
+
+ fan@0 {
+ reg = <0x00>;
+ aspeed,fan-tach-ch = /bits/ 8 <0x00 0x06>;
+ };
+ fan@1 {
+ reg = <0x01>;
+ aspeed,fan-tach-ch = /bits/ 8 <0x01 0x08>;
+ };
+ fan@2 {
+ reg = <0x02>;
+ aspeed,fan-tach-ch = /bits/ 8 <0x02 0x09>;
+ };
+ fan@3 {
+ reg = <0x03>;
+ aspeed,fan-tach-ch = /bits/ 8 <0x03 0x0A>;
+ };
+ fan@4 {
+ reg = <0x04>;
+ aspeed,fan-tach-ch = /bits/ 8 <0x04 0x0B>;
+ };
+ fan@5 {
+ reg = <0x05>;
+ aspeed,fan-tach-ch = /bits/ 8 <0x05>;
+ };
+};
diff --git a/arch/arm/boot/dts/aspeed-bmc-vegman-rx20.dts b/arch/arm/boot/dts/aspeed-bmc-vegman-rx20.dts
new file mode 100644
index 0000000..ebbb68b
--- /dev/null
+++ b/arch/arm/boot/dts/aspeed-bmc-vegman-rx20.dts
@@ -0,0 +1,255 @@
+// SPDX-License-Identifier: GPL-2.0+
+// Copyright (C) 2021 YADRO
+/dts-v1/;
+
+#include "aspeed-bmc-vegman.dtsi"
+
+/ {
+ model = "YADRO VEGMAN Rx20 BMC";
+ compatible = "yadro,vegman-rx20-bmc", "aspeed,ast2500";
+
+ leds {
+ compatible = "gpio-leds";
+
+ temp_alarm {
+ label = "temp:red:status";
+ default-state = "off";
+ gpios = <&gpio ASPEED_GPIO(E, 4) GPIO_ACTIVE_LOW>;
+ };
+
+ temp_ok {
+ label = "temp:green:status";
+ default-state = "off";
+ gpios = <&gpio ASPEED_GPIO(E, 5) GPIO_ACTIVE_LOW>;
+ };
+
+ psu_fault {
+ label = "psu:red:status";
+ default-state = "off";
+ gpios = <&gpio ASPEED_GPIO(E, 6) GPIO_ACTIVE_LOW>;
+ };
+
+ psu_ok {
+ label = "psu:green:status";
+ default-state = "off";
+ gpios = <&gpio ASPEED_GPIO(E, 7) GPIO_ACTIVE_LOW>;
+ };
+ };
+};
+
+&gpio {
+ status = "okay";
+ gpio-line-names =
+ /*A0-A7*/ "CASE_OPEN_DNP","CASE_OPEN_FAULT_RST_DNP","BEZEL_ON_PWR_P3V3","PWM_PWRGD_EXP_EN","SPEAKER_BMC","FM_FORCE_BMC_UPDATE","","",
+ /*B0-B7*/ "","","","","","","","",
+ /*C0-C7*/ "","","","","","","","",
+ /*D0-D7*/ "","","","","","","","",
+ /*E0-E7*/ "RESET_BUTTON","RESET_OUT","POWER_BUTTON","POWER_OUT","LED_TEMP_STATUS_R","LED_TEMP_STATUS_G","LED_PWR_STATUS_R","LED_PWR_STATUS_G",
+ /*F0-F7*/ "NMI_OUT","CPU1_DISABLE_COD","","","SKT0_FAULT_LED_DNP","SKT1_FAULT_LED_DNP","RST_RGMII_PHYRST_DNP","",
+ /*G0-G7*/ "CPU_ERR2","CPU_CATERR","PCH_BMC_THERMTRIP","SPI_BMC_BOOT_HD","IRQ_NMI_EVENT","SPI_BMC_BOOT_WP","SPI_BMC_BOOT_WP1","",
+ /*H0-H7*/ "PWRGD_P3V3_RISER1","PWRGD_P3V3_RISER2","PWRGD_P3V3_RISER3","","MIO_BIOS_SEL","_SPI_FLASH_HOLD","_SPI_FLASH_WP","FM_240VA_STATUS",
+ /*I0-I7*/ "","","","","","","","",
+ /*J0-J7*/ "","","","","","","","",
+ /*K0-K7*/ "","","","","","","","",
+ /*L0-L7*/ "","","","","","","","",
+ /*M0-M7*/ "SEL_FLASH_SOFT","STATUS_SEL_BMC","","","BMC_WDT_P","ID_BUTTON","PS_PWROK","",
+ /*N0-N7*/ "","","","","","","","",
+ /*O0-O7*/ "","","","","","","","",
+ /*P0-P7*/ "","","","","","","SPI_BIOS_ACTIVE_FLASH_SEL","STATUS_SEL_BIOS",
+ /*Q0-Q7*/ "","","","","","","","",
+ /*R0-R7*/ "_SPI_BMC_BOOT_CS1","","","","","","","",
+ /*S0-S7*/ "_SPI2_BMC_CS1","RSR_A_SMBEXP_RST_INT","RSR_B_SMBEXP_RST_INT","IRQ_SML0_ALERT_MUX","FP_LED_STATUS_GREEN","FP_LED_STATUS_AMBER","FP_ID_LED","",
+ /*T0-T7*/ "","","","","","","","",
+ /*U0-U7*/ "","","","","","","","",
+ /*V0-V7*/ "","","","","","","","",
+ /*W0-W7*/ "","","","","","","","",
+ /*X0-X7*/ "","","","","","","","",
+ /*Y0-Y7*/ "SIO_S3","SIO_S5","","SIO_ONCONTROL","","","","",
+ /*Z0-Z7*/ "FM_BMC_PWR_BTN","SIO_POWER_GOOD","FM_BMC_PWRBTN_OUT","FM_BMC_PCH_SCI_LPC","","","","",
+ /*AA0-AA7*/ "CPU_CLK_MUX_SEL","IRQ_SML1_PMBUS_ALERT","FM_PVCCIN_CPU0_PWR_IN_ALERT","FM_PVCCIN_CPU1_PWR_IN_ALERT","BMC_SYS_PWR_FAULT","BMC_SYS_PWR_OK","SMI","POST_COMPLETE",
+ /*AB0-AB7*/ "FM_CPU_BMCINIT","NMI_BUTTON","BMC_WDT_RST1","BMC_WDT_RST2","","","","",
+ /*AC0-AC7*/ "","","","","","","","";
+};
+
+&sgpio {
+ ngpios = <80>;
+ bus-frequency = <2000000>;
+ status = "okay";
+ /* SGPIO lines. even: input, odd: output */
+ gpio-line-names =
+ /*A0-A7*/ "CPU1_PRESENCE","","CPU1_THERMTRIP","","CPU1_VRHOT","","CPU1_FIVR_FAULT","","CPU1_MEM_ABCD_VRHOT","","CPU1_MEM_EFGH_VRHOT","","","","","",
+ /*B0-B7*/ "CPU1_MISMATCH","","CPU1_MEM_THERM_EVENT","","CPU2_PRESENCE","","CPU2_THERMTRIP","","CPU2_VRHOT","","CPU2_FIVR_FAULT","","CPU2_MEM_ABCD_VRHOT","","CPU2_MEM_EFGH_VRHOT","",
+ /*C0-C7*/ "","","","","CPU2_MISMATCH","","CPU2_MEM_THERM_EVENT","","","","","","","","","",
+ /*D0-D7*/ "","","","","","","","","","","","","","","","",
+ /*E0-E7*/ "","","","","","","","","","","","","","","","",
+ /*F0-F7*/ "SGPIO_PLD_MINOR_REV_BIT0","","SGPIO_PLD_MINOR_REV_BIT1","","SGPIO_PLD_MINOR_REV_BIT2","","SGPIO_PLD_MINOR_REV_BIT3","","SGPIO_PLD_MAJOR_REV_BIT0","","SGPIO_PLD_MAJOR_REV_BIT1","","SGPIO_PLD_MAJOR_REV_BIT2","","SGPIO_PLD_MAJOR_REV_BIT3","",
+ /*G0-G7*/ "MAIN_PLD_MINOR_REV_BIT0","","MAIN_PLD_MINOR_REV_BIT1","","MAIN_PLD_MINOR_REV_BIT2","","MAIN_PLD_MINOR_REV_BIT3","","MAIN_PLD_MAJOR_REV_BIT0","","MAIN_PLD_MAJOR_REV_BIT1","","MAIN_PLD_MAJOR_REV_BIT2","","MAIN_PLD_MAJOR_REV_BIT3","",
+ /*H0-H7*/ "","","","","","","","","","","","","","","","",
+ /*I0-I7*/ "","","","","","","","","","","","","","","","",
+ /*J0-J7*/ "","","","","","","","","","","","","","","","";
+};
+
+&i2c11 {
+ /* SMB_BMC_MGMT_LVC3 */
+ gpio@21 {
+ compatible = "nxp,pcal9535";
+ reg = <0x21>;
+ gpio-controller;
+ #gpio-cells = <2>;
+ gpio-line-names =
+ /*IO0.0-0.7*/ "ETH3_CLK_REQ", "ETH2_CLK_REQ", "RSR_A_PCIE_X16_2_PRSNT", "RSR_B_PCIE_X16_2_PRSNT", "", "RSR_B_PCIE_X8_3_PRSNT", "RSR_B_PCIE_X8_4_PRSNT", "RSR_B_PCIE_X16_PRSNT_N",
+ /*IO1.0-1.7*/ "RSR_B_PCIE_X8_2_PRSNT", "RSR_B_PCIE_X8_1_PRSNT", "NIC_1_PE_BUF_PRSNT", "RSR_A_PCIE_X16_PRSNT", "RSR_A_PCIE_X8_3_PRSNT", "RSR_A_PCIE_X8_2_PRSNT", "RSR_A_PCIE_X8_1_PRSNT_N", "";
+ };
+ gpio@23 {
+ compatible = "nxp,pcal9535";
+ reg = <0x23>;
+ gpio-controller;
+ #gpio-cells = <2>;
+ gpio-line-names =
+ /*IO0.0-0.7*/ "FM_LINK_WIDTH_ID0", "FM_LINK_WIDTH_ID0", "FM_LINK_WIDTH_ID0", "FM_LINK_WIDTH_ID0", "FM_LINK_WIDTH_ID0", "", "", "",
+ /*IO1.0-1.7*/ "", "", "", "", "", "", "", "";
+ };
+ gpio@27 {
+ compatible = "nxp,pca9698";
+ reg = <0x27>;
+ gpio-controller;
+ #gpio-cells = <2>;
+ gpio-line-names =
+ /*IO0.0-0.7*/ "PWRGD_PS_PWROK", "PWRGD_DSW_PWROK", "PWRGD_P5V_AUX", "PWRGD_P3V3_AUX", "PWRGD_P5V", "PWRGD_P3V3", "PWRGD_P1V8_PCH_AUX", "PWRGD_PCH_PVNN_AUX",
+ /*IO1.0-1.7*/ "PWRGD_P1V05_PCH_AUX", "PWRGD_PCH_AUX_VRS", "PWRGD_PVCCIN_CPU0", "PWRGD_PVCCSA_CPU0", "PWRGD_PVCCIO_CPU0", "PWRGD_PVMCP_CPU0", "PWRGD_P1V0_CPU0", "PWRGD_PVDDQ_ABC_CPU0",
+ /*IO2.0-2.7*/ "PWRGD_PVPP_ABC_CPU0", "PWRGD_PVTT_ABC_CPU0", "PWRGD_PVDDQ_DEF_CPU0", "PWRGD_PVPP_DEF_CPU0", "PWRGD_PVTT_DEF_CPU0", "PWRGD_PVCCIN_CPU1", "PWRGD_PVCCSA_CPU1", "PWRGD_PVCCIO_CPU1",
+ /*IO3.0-3.7*/ "PWRGD_PVMCP_CPU1", "PWRGD_P1V0_CPU1", "PWRGD_PVDDQ_GHJ_CPU1", "PWRGD_PVPP_GHJ_CPU1", "PWRGD_PVTT_GHJ_CPU1", "PWRGD_PVDDQ_KLM_CPU1", "PWRGD_PVPP_KLM_CPU1", "PWRGD_PVTT_KLM_CPU1",
+ /*IO4.0-4.7*/ "PCH_PWR_RESET_N", "FM_BOARD_SKU_ID0", "FM_BOARD_SKU_ID1", "FM_BOARD_SKU_ID2", "FM_BOARD_SKU_ID3", "FM_BOARD_SKU_ID4", "FM_BOARD_REV_ID0", "FM_BOARD_REV_ID1";
+ };
+ gpio@39 {
+ compatible = "nxp,pca9554";
+ reg = <0x39>;
+ gpio-controller;
+ #gpio-cells = <2>;
+ gpio-line-names =
+ /*IO0.0-0.7*/ "FAN_FAULT_0", "FAN_FAULT_1", "FAN_FAULT_2", "FAN_FAULT_3", "FAN_FAULT_4", "FAN_FAULT_5", "FAN_FAULT_6", "";
+ };
+};
+
+&i2c13 {
+ /* SMB_PCIE2_STBY_LVC3 */
+ mux-expa@70 {
+ compatible = "nxp,pca9548";
+ reg = <0x70>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ i2c-mux-idle-disconnect;
+
+ i2c@2 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <2>;
+ rsra-mux@72 {
+ compatible = "nxp,pca9548";
+ reg = <0x72>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ i2c@7 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <7>;
+ at24@50 {
+ compatible = "atmel,24c64";
+ reg = <0x50>;
+ pagesize = <32>;
+ size = <8192>;
+ address-width = <16>;
+ };
+ };
+ };
+ };
+ };
+ mux-sata@71 {
+ compatible = "nxp,pca9543";
+ reg = <0x71>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ i2c-mux-idle-disconnect;
+ };
+};
+
+&i2c2 {
+ /* SMB_PCIE_STBY_LVC3 */
+ mux-expb@71 {
+ compatible = "nxp,pca9548";
+ reg = <0x71>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ i2c-mux-idle-disconnect;
+
+ i2c@0 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0>;
+ rsrb-mux@72 {
+ compatible = "nxp,pca9548";
+ reg = <0x72>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ i2c@7 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <7>;
+ at24@50 {
+ compatible = "atmel,24c64";
+ reg = <0x50>;
+ pagesize = <32>;
+ size = <8192>;
+ address-width = <16>;
+ };
+ };
+ };
+ at24@50 {
+ compatible = "atmel,24c64";
+ reg = <0x50>;
+ pagesize = <32>;
+ size = <8192>;
+ address-width = <16>;
+ };
+ };
+ };
+};
+
+&pwm_tacho {
+ status = "okay";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_pwm0_default &pinctrl_pwm1_default
+ &pinctrl_pwm2_default &pinctrl_pwm3_default
+ &pinctrl_pwm4_default &pinctrl_pwm5_default
+ &pinctrl_pwm6_default>;
+
+ fan@0 {
+ reg = <0x00>;
+ aspeed,fan-tach-ch = /bits/ 8 <0x00 0x07>;
+ };
+ fan@1 {
+ reg = <0x01>;
+ aspeed,fan-tach-ch = /bits/ 8 <0x01 0x08>;
+ };
+ fan@2 {
+ reg = <0x02>;
+ aspeed,fan-tach-ch = /bits/ 8 <0x02 0x09>;
+ };
+ fan@3 {
+ reg = <0x03>;
+ aspeed,fan-tach-ch = /bits/ 8 <0x03 0x0A>;
+ };
+ fan@4 {
+ reg = <0x04>;
+ aspeed,fan-tach-ch = /bits/ 8 <0x04 0x0B>;
+ };
+ fan@5 {
+ reg = <0x05>;
+ aspeed,fan-tach-ch = /bits/ 8 <0x05 0x0C>;
+ };
+ fan@6 {
+ reg = <0x06>;
+ aspeed,fan-tach-ch = /bits/ 8 <0x06 0x0D>;
+ };
+};
diff --git a/arch/arm/boot/dts/aspeed-bmc-vegman-sx20.dts b/arch/arm/boot/dts/aspeed-bmc-vegman-sx20.dts
new file mode 100644
index 0000000..e36ee47
--- /dev/null
+++ b/arch/arm/boot/dts/aspeed-bmc-vegman-sx20.dts
@@ -0,0 +1,154 @@
+// SPDX-License-Identifier: GPL-2.0+
+// Copyright (C) 2021 YADRO
+/dts-v1/;
+
+#include "aspeed-bmc-vegman.dtsi"
+
+/ {
+ model = "YADRO VEGMAN Sx20 BMC";
+ compatible = "yadro,vegman-sx20-bmc", "aspeed,ast2500";
+};
+
+&gpio {
+ status = "okay";
+ gpio-line-names =
+ /*A0-A7*/ "CHASSIS_INTRUSION","CASE_OPEN_FAULT_RST","","","SPEAKER_BMC","FM_FORCE_BMC_UPDATE","","",
+ /*B0-B7*/ "","","","","","","","",
+ /*C0-C7*/ "","","","","","","","",
+ /*D0-D7*/ "","","","","","","","",
+ /*E0-E7*/ "RESET_BUTTON","RESET_OUT","POWER_BUTTON","POWER_OUT","","","","",
+ /*F0-F7*/ "NMI_OUT","CPU1_DISABLE_COD","","","SKT0_FAULT_LED","SKT1_FAULT_LED","RST_RGMII_PHYRST_DNP","",
+ /*G0-G7*/ "CPU_ERR2","CPU_CATERR","PCH_BMC_THERMTRIP","","IRQ_NMI_EVENT","","","",
+ /*H0-H7*/ "PWRGD_P3V3_RISER1","PWRGD_P3V3_RISER2","PWRGD_P3V3_RISER3","","MIO_BIOS_SEL","_SPI_FLASH_HOLD","_SPI_FLASH_WP","FM_240VA_STATUS",
+ /*I0-I7*/ "","","","","","","","",
+ /*J0-J7*/ "","","","","","","","",
+ /*K0-K7*/ "","","","","","","","",
+ /*L0-L7*/ "","","","","","","","",
+ /*M0-M7*/ "","","","","BMC_GPU_RISER_ID1","BMC_GPU_RISER_ID0","","",
+ /*N0-N7*/ "","","","","","","","",
+ /*O0-O7*/ "","","","","","","","_SPI2_BMC_CS_SEL",
+ /*P0-P7*/ "","P12V_HDDS_A_EN","P12V_HDDS_B_EN","P5V_HDDS_A_EN","PWRGD_P5V_HDDS_A","P5V_HDDS_B_EN","PWRGD_P5V_HDDS_B","",
+ /*Q0-Q7*/ "","","","","","","","",
+ /*R0-R7*/ "_SPI_RMM4_LITE_CS","","","","","","","",
+ /*S0-S7*/ "_SPI2_BMC_CS1","","","IRQ_SML0_ALERT_MUX","FP_LED_STATUS_GREEN","FP_LED_STATUS_AMBER","FP_ID_LED","",
+ /*T0-T7*/ "","","","","","","","",
+ /*U0-U7*/ "","","","","","","","",
+ /*V0-V7*/ "","","","","","","","",
+ /*W0-W7*/ "","","","","","","","",
+ /*X0-X7*/ "","","","","","","","",
+ /*Y0-Y7*/ "SIO_S3","SIO_S5","","SIO_ONCONTROL","","","","",
+ /*Z0-Z7*/ "FM_BMC_PWR_BTN","SIO_POWER_GOOD","FM_BMC_PWRBTN_OUT","FM_BMC_PCH_SCI_LPC","","","","",
+ /*AA0-AA7*/ "CPU_CLK_MUX_SEL","IRQ_SML1_PMBUS_ALERT","FM_PVCCIN_CPU0_PWR_IN_ALERT","FM_PVCCIN_CPU1_PWR_IN_ALERT","BMC_SYS_PWR_FAULT","BMC_SYS_PWR_OK","SMI","POST_COMPLETE",
+ /*AB0-AB7*/ "FM_CPU_BMCINIT","NMI_BUTTON","ID_BUTTON","PS_PWROK","","","","",
+ /*AC0-AC7*/ "","","","","","","","";
+};
+
+&sgpio {
+ ngpios = <80>;
+ bus-frequency = <2000000>;
+ status = "okay";
+ /* SGPIO lines. even: input, odd: output */
+ gpio-line-names =
+ /*A0-A7*/ "CPU1_PRESENCE","","CPU1_THERMTRIP","","CPU1_VRHOT","","CPU1_FIVR_FAULT","","CPU1_MEM_ABCD_VRHOT","","CPU1_MEM_EFGH_VRHOT","","","","","",
+ /*B0-B7*/ "CPU1_MISMATCH","","CPU1_MEM_THERM_EVENT","","CPU2_PRESENCE","","CPU2_THERMTRIP","","CPU2_VRHOT","","CPU2_FIVR_FAULT","","CPU2_MEM_ABCD_VRHOT","","CPU2_MEM_EFGH_VRHOT","",
+ /*C0-C7*/ "","","","","CPU2_MISMATCH","","CPU2_MEM_THERM_EVENT","","","","","","","","","",
+ /*D0-D7*/ "","","","","","","","","","","","","","","","",
+ /*E0-E7*/ "","","","","","","","","","","","","","","","",
+ /*F0-F7*/ "SGPIO_PLD_MINOR_REV_BIT0","","SGPIO_PLD_MINOR_REV_BIT1","","SGPIO_PLD_MINOR_REV_BIT2","","SGPIO_PLD_MINOR_REV_BIT3","","SGPIO_PLD_MAJOR_REV_BIT0","","SGPIO_PLD_MAJOR_REV_BIT1","","SGPIO_PLD_MAJOR_REV_BIT2","","SGPIO_PLD_MAJOR_REV_BIT3","",
+ /*G0-G7*/ "MAIN_PLD_MINOR_REV_BIT0","","MAIN_PLD_MINOR_REV_BIT1","","MAIN_PLD_MINOR_REV_BIT2","","MAIN_PLD_MINOR_REV_BIT3","","MAIN_PLD_MAJOR_REV_BIT0","","MAIN_PLD_MAJOR_REV_BIT1","","MAIN_PLD_MAJOR_REV_BIT2","","MAIN_PLD_MAJOR_REV_BIT3","",
+ /*H0-H7*/ "","","","","","","","","","","","","","","","",
+ /*I0-I7*/ "","","","","","","","","","","","","","","","",
+ /*J0-J7*/ "","","","","","","","","","","","","","","","";
+};
+
+&i2c11 {
+ /* SMB_BMC_MGMT_LVC3 */
+ gpio@21 {
+ compatible = "nxp,pcal9535";
+ reg = <0x21>;
+ gpio-controller;
+ #gpio-cells = <2>;
+ gpio-line-names =
+ /*IO0.0-0.7*/ "", "", "CPU1_PE3_0_SLOT_PRSNT", "", "CPU1_PE1_GPU_PRSNT", "CPU1_PE3_1_SLOT_PRSNT", "PE_PCH_MEZ_PRSNT", "CPU0_PE3_1_SLOT_PRSNT",
+ /*IO1.0-1.7*/ "CPU0_PE1_GPU_PRSNT", "CPU0_PE2_NVME2_PRSNT", "CPU1_PE2_NVME3_PRSNT", "CPU1_PE2_SLOT_PRSNT", "CPU1_PE2_NVME4_PRSNT", "", "CPU0_PE2_NVME1_PRSNT", "CPU0_PE3_0_RAID_PRSNT";
+ };
+ gpio@27 {
+ compatible = "nxp,pca9698";
+ reg = <0x27>;
+ gpio-controller;
+ #gpio-cells = <2>;
+ gpio-line-names =
+ /*IO0.0-0.7*/ "PWRGD_PS_PWROK", "PWRGD_DSW_PWROK", "PWRGD_P5V_AUX", "PWRGD_P3V3_AUX", "PWRGD_P5V", "PWRGD_P3V3", "PWRGD_P1V8_PCH_AUX", "PWRGD_PCH_PVNN_AUX",
+ /*IO1.0-1.7*/ "PWRGD_P1V05_PCH_AUX", "PWRGD_PCH_AUX_VRS", "PWRGD_PVCCIN_CPU0", "PWRGD_PVCCSA_CPU0", "PWRGD_PVCCIO_CPU0", "PWRGD_PVMCP_CPU0", "PWRGD_P1V0_CPU0", "PWRGD_PVDDQ_ABC_CPU0",
+ /*IO2.0-2.7*/ "PWRGD_PVPP_ABC_CPU0", "PWRGD_PVTT_ABC_CPU0", "PWRGD_PVDDQ_DEF_CPU0", "PWRGD_PVPP_DEF_CPU0", "PWRGD_PVTT_DEF_CPU0", "PWRGD_PVCCIN_CPU1", "PWRGD_PVCCSA_CPU1", "PWRGD_PVCCIO_CPU1",
+ /*IO3.0-3.7*/ "PWRGD_PVMCP_CPU1", "PWRGD_P1V0_CPU1", "PWRGD_PVDDQ_GHJ_CPU1", "PWRGD_PVPP_GHJ_CPU1", "PWRGD_PVTT_GHJ_CPU1", "PWRGD_PVDDQ_KLM_CPU1", "PWRGD_PVPP_KLM_CPU1", "PWRGD_PVTT_KLM_CPU1",
+ /*IO4.0-4.7*/ "PWRGD_P5V_HDDS_A_R", "PWRGD_P5V_HDDS_B_R", "", "", "", "", "", "";
+ };
+};
+
+&i2c13 {
+ /* SMB_PCIE2_STBY_LVC3 */
+ mux-expa@73 {
+ compatible = "nxp,pca9545";
+ reg = <0x73>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ i2c-mux-idle-disconnect;
+ };
+ mux-sata@71 {
+ compatible = "nxp,pca9543";
+ reg = <0x71>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ i2c-mux-idle-disconnect;
+ };
+};
+
+&i2c2 {
+ /* SMB_PCIE_STBY_LVC3 */
+ mux-expb@71 {
+ compatible = "nxp,pca9545";
+ reg = <0x71>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ i2c-mux-idle-disconnect;
+ };
+};
+
+&pwm_tacho {
+ status = "okay";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_pwm0_default &pinctrl_pwm1_default
+ &pinctrl_pwm2_default &pinctrl_pwm3_default
+ &pinctrl_pwm4_default &pinctrl_pwm5_default
+ &pinctrl_pwm6_default>;
+
+ fan@0 {
+ reg = <0x00>;
+ aspeed,fan-tach-ch = /bits/ 8 <0x00>;
+ };
+ fan@1 {
+ reg = <0x01>;
+ aspeed,fan-tach-ch = /bits/ 8 <0x01>;
+ };
+ fan@2 {
+ reg = <0x02>;
+ aspeed,fan-tach-ch = /bits/ 8 <0x02>;
+ };
+ fan@3 {
+ reg = <0x03>;
+ aspeed,fan-tach-ch = /bits/ 8 <0x03>;
+ };
+ fan@4 {
+ reg = <0x04>;
+ aspeed,fan-tach-ch = /bits/ 8 <0x04>;
+ };
+ fan@5 {
+ reg = <0x05>;
+ aspeed,fan-tach-ch = /bits/ 8 <0x05>;
+ };
+ fan@6 {
+ reg = <0x06>;
+ aspeed,fan-tach-ch = /bits/ 8 <0x06>;
+ };
+};
diff --git a/arch/arm/boot/dts/aspeed-bmc-vegman.dtsi b/arch/arm/boot/dts/aspeed-bmc-vegman.dtsi
new file mode 100644
index 0000000..1a5b25b
--- /dev/null
+++ b/arch/arm/boot/dts/aspeed-bmc-vegman.dtsi
@@ -0,0 +1,311 @@
+// SPDX-License-Identifier: GPL-2.0+
+// Copyright (C) 2021 YADRO
+
+#include "aspeed-g5.dtsi"
+#include <dt-bindings/gpio/aspeed-gpio.h>
+
+/ {
+ aliases {
+ serial4 = &uart5;
+ };
+
+ chosen {
+ stdout-path = &uart5;
+ bootargs = "console=ttyS4,115200 earlyprintk";
+ };
+
+ memory@80000000 {
+ reg = <0x80000000 0x20000000>;
+ };
+
+ reserved-memory {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges;
+
+ video_engine_memory: jpegbuffer {
+ size = <0x02000000>; /* 32M */
+ alignment = <0x01000000>;
+ compatible = "shared-dma-pool";
+ reusable;
+ };
+
+ ramoops@9eff0000{
+ compatible = "ramoops";
+ reg = <0x9eff0000 0x10000>;
+ record-size = <0x2000>;
+ console-size = <0x2000>;
+ };
+ };
+
+ iio-hwmon {
+ compatible = "iio-hwmon";
+ io-channels = <&adc 0>, <&adc 1>, <&adc 2>, <&adc 3>,
+ <&adc 4>, <&adc 5>, <&adc 6>, <&adc 7>,
+ <&adc 8>, <&adc 9>, <&adc 10>, <&adc 11>,
+ <&adc 12>, <&adc 13>, <&adc 14>, <&adc 15>;
+ };
+
+ leds {
+ compatible = "gpio-leds";
+
+ identify {
+ label = "platform:blue:indicator";
+ linux,default-trigger = "heartbeat";
+ gpios = <&gpio ASPEED_GPIO(S, 6) GPIO_ACTIVE_LOW>;
+ };
+
+ status_amber {
+ label = "platform:red:status";
+ default-state = "off";
+ gpios = <&gpio ASPEED_GPIO(S, 5) GPIO_ACTIVE_LOW>;
+ };
+
+ status_green {
+ label = "platform:green:status";
+ default-state = "off";
+ gpios = <&gpio ASPEED_GPIO(S, 4) GPIO_ACTIVE_LOW>;
+ };
+
+ power_fault {
+ label = "platform:red:power";
+ default-state = "off";
+ gpios = <&gpio ASPEED_GPIO(AA, 4) GPIO_ACTIVE_LOW>;
+ };
+
+ power_ok {
+ label = "platform:green:power";
+ default-state = "off";
+ gpios = <&gpio ASPEED_GPIO(AA, 5) GPIO_ACTIVE_LOW>;
+ };
+ };
+
+ beeper {
+ compatible = "pwm-beeper";
+ pwms = <&timer 5 1000000 0>;
+ };
+};
+
+&fmc {
+ status = "okay";
+ flash@0 {
+ status = "okay";
+ label = "bmc";
+ m25p,fast-read;
+#include "openbmc-flash-layout-64.dtsi"
+ };
+};
+
+&spi2 {
+ status = "okay";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_spi2ck_default
+ &pinctrl_spi2miso_default
+ &pinctrl_spi2mosi_default
+ &pinctrl_spi2cs0_default>;
+ flash@0 {
+ status = "okay";
+ label = "bios";
+ m25p,fast-read;
+ };
+};
+
+&mac0 {
+ status = "okay";
+ use-ncsi;
+
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_rmii1_default>;
+};
+
+&mac1 {
+ status = "okay";
+
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_rgmii2_default &pinctrl_mdio2_default>;
+
+ phy-mode = "rgmii";
+ phy-handle = <&phy>;
+ mdio {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ phy: ethernet-phy@1 {
+ /* KSZ9131 */
+ compatible = "ethernet-phy-id0022.1640";
+ reg = <1>;
+
+ micrel,led-mode = <0>;
+ };
+ };
+};
+
+&vhub {
+ status = "okay";
+};
+
+&adc {
+ status = "okay";
+};
+
+&video {
+ status = "okay";
+ memory-region = <&video_engine_memory>;
+};
+
+&sdmmc {
+ status = "okay";
+};
+
+&sdhci1 {
+ status = "okay";
+
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_sd2_default>;
+ disable-wp;
+};
+
+&timer {
+ fttmr010,pwm-outputs = <5>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_timer5_default>;
+ #pwm-cells = <3>;
+ status = "okay";
+};
+
+&uart1 {
+ status = "okay";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_txd1_default
+ &pinctrl_rxd1_default
+ &pinctrl_nrts1_default
+ &pinctrl_ndtr1_default
+ &pinctrl_ndsr1_default
+ &pinctrl_ncts1_default
+ &pinctrl_ndcd1_default
+ &pinctrl_nri1_default>;
+};
+
+&uart5 {
+ status = "okay";
+};
+
+&vuart {
+ status = "okay";
+};
+
+&kcs3 {
+ aspeed,lpc-io-reg = <0xCA2>;
+ status = "okay";
+};
+
+&kcs4 {
+ aspeed,lpc-io-reg = <0xCA4>;
+ status = "okay";
+};
+
+&lpc_snoop {
+ snoop-ports = <0x80>;
+ status = "okay";
+};
+
+&uart_routing {
+ status = "okay";
+};
+
+&uart2 {
+ status = "okay";
+ pinctrl-names = "default";
+ pinctrl-0 = <>;
+};
+
+&uart3 {
+ status = "okay";
+ pinctrl-names = "default";
+ pinctrl-0 = <>;
+};
+
+&uart4 {
+ status = "okay";
+ pinctrl-names = "default";
+ pinctrl-0 = <>;
+};
+
+&i2c0 {
+ /* SMB_IPMB_STBY_LVC3 */
+ multi-master;
+ status = "okay";
+};
+
+&i2c1 {
+ /* SMB_CHASSENSOR_STBY_LVC3 */
+ status = "okay";
+};
+
+&i2c2 {
+ /* SMB_PCIE_STBY_LVC3 */
+ status = "okay";
+};
+
+&i2c3 {
+ /* SMB_HOST_STBY_LVC3 */
+ multi-master;
+ status = "okay";
+};
+
+&i2c4 {
+ /* BMC_PMBUS2_STBY */
+ status = "okay";
+};
+
+&i2c5 {
+ /* SMB_SMLINK0_STBY_LVC3 */
+ bus-frequency = <1000000>;
+ multi-master;
+ status = "okay";
+};
+
+&i2c6 {
+ /* SMB_TEMPSENSOR_STBY_LVC3 */
+ multi-master;
+ status = "okay";
+};
+
+&i2c7 {
+ /* SMB_SM_PMB1_SML1_STBY_LVC3 */
+ multi-master;
+ status = "okay";
+};
+
+&i2c9 {
+ /* SMB_BMC_ETH3_LVC3 */
+ status = "okay";
+};
+
+&i2c10 {
+ /* SMB_BMC_ETH2_LVC3 */
+ status = "okay";
+};
+
+&i2c11 {
+ /* SMB_BMC_MGMT_LVC3 */
+ status = "okay";
+
+ at24@50 {
+ compatible = "atmel,24c64";
+ reg = <0x50>;
+ pagesize = <32>;
+ size = <8192>;
+ address-width = <16>;
+ };
+};
+
+&i2c12 {
+ /* SMB_BMC_FAULT_EXP_LVC3 */
+ status = "okay";
+};
+
+&i2c13 {
+ /* SMB_PCIE2_STBY_LVC3 */
+ status = "okay";
+};
diff --git a/arch/arm/boot/dts/aspeed-g4.dtsi b/arch/arm/boot/dts/aspeed-g4.dtsi
index b313a1c..f14dace 100644
--- a/arch/arm/boot/dts/aspeed-g4.dtsi
+++ b/arch/arm/boot/dts/aspeed-g4.dtsi
@@ -381,6 +381,7 @@ ibt: ibt@140 {
compatible = "aspeed,ast2400-ibt-bmc";
reg = <0x140 0x18>;
interrupts = <8>;
+ clocks = <&syscon ASPEED_CLK_GATE_LCLK>;
status = "disabled";
};
diff --git a/arch/arm/boot/dts/aspeed-g5.dtsi b/arch/arm/boot/dts/aspeed-g5.dtsi
index c704945..7495f93 100644
--- a/arch/arm/boot/dts/aspeed-g5.dtsi
+++ b/arch/arm/boot/dts/aspeed-g5.dtsi
@@ -446,6 +446,7 @@ kcs1: kcs@24 {
compatible = "aspeed,ast2500-kcs-bmc-v2";
reg = <0x24 0x1>, <0x30 0x1>, <0x3c 0x1>;
interrupts = <8>;
+ clocks = <&syscon ASPEED_CLK_GATE_LCLK>;
status = "disabled";
};
@@ -453,6 +454,7 @@ kcs2: kcs@28 {
compatible = "aspeed,ast2500-kcs-bmc-v2";
reg = <0x28 0x1>, <0x34 0x1>, <0x40 0x1>;
interrupts = <8>;
+ clocks = <&syscon ASPEED_CLK_GATE_LCLK>;
status = "disabled";
};
@@ -460,6 +462,7 @@ kcs3: kcs@2c {
compatible = "aspeed,ast2500-kcs-bmc-v2";
reg = <0x2c 0x1>, <0x38 0x1>, <0x44 0x1>;
interrupts = <8>;
+ clocks = <&syscon ASPEED_CLK_GATE_LCLK>;
status = "disabled";
};
@@ -467,6 +470,7 @@ kcs4: kcs@114 {
compatible = "aspeed,ast2500-kcs-bmc-v2";
reg = <0x114 0x1>, <0x118 0x1>, <0x11c 0x1>;
interrupts = <8>;
+ clocks = <&syscon ASPEED_CLK_GATE_LCLK>;
status = "disabled";
};
@@ -507,6 +511,7 @@ ibt: ibt@140 {
compatible = "aspeed,ast2500-ibt-bmc";
reg = <0x140 0x18>;
interrupts = <8>;
+ clocks = <&syscon ASPEED_CLK_GATE_LCLK>;
status = "disabled";
};
};
diff --git a/arch/arm/boot/dts/aspeed-g6.dtsi b/arch/arm/boot/dts/aspeed-g6.dtsi
index 5106a42..c32e87f 100644
--- a/arch/arm/boot/dts/aspeed-g6.dtsi
+++ b/arch/arm/boot/dts/aspeed-g6.dtsi
@@ -384,6 +384,11 @@ adc1: adc@1e6e9100 {
status = "disabled";
};
+ sbc: secure-boot-controller@1e6f2000 {
+ compatible = "aspeed,ast2600-sbc";
+ reg = <0x1e6f2000 0x1000>;
+ };
+
gpio0: gpio@1e780000 {
#gpio-cells = <2>;
gpio-controller;
@@ -520,6 +525,7 @@ kcs1: kcs@24 {
compatible = "aspeed,ast2500-kcs-bmc-v2";
reg = <0x24 0x1>, <0x30 0x1>, <0x3c 0x1>;
interrupts = <GIC_SPI 138 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&syscon ASPEED_CLK_GATE_LCLK>;
kcs_chan = <1>;
status = "disabled";
};
@@ -528,6 +534,7 @@ kcs2: kcs@28 {
compatible = "aspeed,ast2500-kcs-bmc-v2";
reg = <0x28 0x1>, <0x34 0x1>, <0x40 0x1>;
interrupts = <GIC_SPI 139 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&syscon ASPEED_CLK_GATE_LCLK>;
status = "disabled";
};
@@ -535,6 +542,7 @@ kcs3: kcs@2c {
compatible = "aspeed,ast2500-kcs-bmc-v2";
reg = <0x2c 0x1>, <0x38 0x1>, <0x44 0x1>;
interrupts = <GIC_SPI 140 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&syscon ASPEED_CLK_GATE_LCLK>;
status = "disabled";
};
@@ -542,6 +550,7 @@ kcs4: kcs@114 {
compatible = "aspeed,ast2500-kcs-bmc-v2";
reg = <0x114 0x1>, <0x118 0x1>, <0x11c 0x1>;
interrupts = <GIC_SPI 141 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&syscon ASPEED_CLK_GATE_LCLK>;
status = "disabled";
};
@@ -581,6 +590,7 @@ ibt: ibt@140 {
compatible = "aspeed,ast2600-ibt-bmc";
reg = <0x140 0x18>;
interrupts = <GIC_SPI 143 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&syscon ASPEED_CLK_GATE_LCLK>;
status = "disabled";
};
};
diff --git a/arch/arm/boot/dts/at91-q5xr5.dts b/arch/arm/boot/dts/at91-q5xr5.dts
index 5827383..47a0006 100644
--- a/arch/arm/boot/dts/at91-q5xr5.dts
+++ b/arch/arm/boot/dts/at91-q5xr5.dts
@@ -147,12 +147,6 @@ uboot@8000 {
reg = <0x8000 0x3E000>;
};
};
-
- spidev@1 {
- compatible = "spidev";
- spi-max-frequency = <2000000>;
- reg = <1>;
- };
};
&spi1 {
@@ -160,18 +154,6 @@ &spi1 {
pinctrl-0 = <&pinctrl_spi1 &pinctrl_spi1_npcs0 &pinctrl_spi1_npcs1>;
cs-gpios = <&pioB 3 GPIO_ACTIVE_HIGH>, <&pioC 5 GPIO_ACTIVE_LOW>, <0>, <0>;
status = "okay";
-
- spidev@0 {
- compatible = "spidev";
- spi-max-frequency = <2000000>;
- reg = <0>;
- };
-
- spidev@1 {
- compatible = "spidev";
- spi-max-frequency = <2000000>;
- reg = <1>;
- };
};
&usart0 {
diff --git a/arch/arm/boot/dts/at91-sama5d2_xplained.dts b/arch/arm/boot/dts/at91-sama5d2_xplained.dts
index b1e854f..9bf2ec0 100644
--- a/arch/arm/boot/dts/at91-sama5d2_xplained.dts
+++ b/arch/arm/boot/dts/at91-sama5d2_xplained.dts
@@ -66,7 +66,7 @@ sdmmc0: sdio-host@a0000000 {
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_sdmmc0_default>;
non-removable;
- mmc-ddr-1_8v;
+ mmc-ddr-3_3v;
status = "okay";
};
@@ -619,10 +619,9 @@ cmd_data {
bias-disable;
};
- ck_cd_rstn_vddsel {
+ ck_cd_rstn {
pinmux = <PIN_PA0__SDMMC0_CK>,
<PIN_PA10__SDMMC0_RSTN>,
- <PIN_PA11__SDMMC0_VDDSEL>,
<PIN_PA13__SDMMC0_CD>;
bias-disable;
};
diff --git a/arch/arm/boot/dts/at91-sama5d3_ksz9477_evb.dts b/arch/arm/boot/dts/at91-sama5d3_ksz9477_evb.dts
new file mode 100644
index 0000000..443e8b0
--- /dev/null
+++ b/arch/arm/boot/dts/at91-sama5d3_ksz9477_evb.dts
@@ -0,0 +1,209 @@
+// SPDX-License-Identifier: GPL-2.0-or-later OR MIT
+/*
+ * Copyright (c) 2021 Oleksij Rempel <kernel@pengutronix.de>, Pengutronix
+ */
+/dts-v1/;
+#include "sama5d36.dtsi"
+
+/ {
+ model = "EVB-KSZ9477";
+ compatible = "microchip,sama5d3-ksz9477-evb", "atmel,sama5d36",
+ "atmel,sama5d3", "atmel,sama5";
+
+ chosen {
+ stdout-path = &dbgu;
+ };
+
+ reg_3v3: regulator-3v3 {
+ compatible = "regulator-fixed";
+ regulator-name = "3v3";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ };
+
+ reg_vcc_mmc0: regulator-mmc0 {
+ compatible = "regulator-fixed";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_mcc0_vcc>;
+ regulator-name = "mmc0-vcc";
+ regulator-max-microvolt = <3300000>;
+ regulator-min-microvolt = <3300000>;
+ gpio = <&pioE 2 GPIO_ACTIVE_LOW>;
+ };
+};
+
+&dbgu {
+ status = "okay";
+};
+
+&ebi {
+ pinctrl-0 = <&pinctrl_ebi_nand_addr>;
+ pinctrl-names = "default";
+ status = "okay";
+};
+
+&i2c0 {
+ pinctrl-0 = <&pinctrl_i2c0_pu>;
+ status = "okay";
+};
+
+&macb0 {
+ phy-mode = "rgmii";
+ status = "okay";
+
+ fixed-link {
+ speed = <1000>;
+ full-duplex;
+ };
+};
+
+&main_xtal {
+ clock-frequency = <12000000>;
+};
+
+&mmc0 {
+ pinctrl-0 = <&pinctrl_mmc0_clk_cmd_dat0 &pinctrl_mmc0_dat1_3
+ &pinctrl_mmc0_dat4_7 &pinctrl_mmc0_cd>;
+ status = "okay";
+
+ slot@0 {
+ reg = <0>;
+ bus-width = <8>;
+ cd-gpios = <&pioE 0 GPIO_ACTIVE_LOW>;
+ disable-wp;
+ vmmc-supply = <®_vcc_mmc0>;
+ vqmmc-supply = <®_3v3>;
+ };
+};
+
+&nand_controller {
+ status = "okay";
+
+ nand@3 {
+ reg = <0x3 0x0 0x2>;
+ atmel,rb = <0>;
+ nand-bus-width = <8>;
+ nand-ecc-mode = "hw";
+ nand-ecc-strength = <4>;
+ nand-ecc-step-size = <512>;
+ nand-on-flash-bbt;
+ label = "atmel_nand";
+ };
+};
+
+&slow_xtal {
+ clock-frequency = <32768>;
+};
+
+&spi0 {
+ cs-gpios = <&pioD 13 GPIO_ACTIVE_LOW>, <0>, <0>,
+ <&pioD 16 GPIO_ACTIVE_LOW>;
+ status = "okay";
+};
+
+&spi1 {
+ pinctrl-0 = <&pinctrl_spi_ksz>;
+ cs-gpios = <&pioC 25 GPIO_ACTIVE_LOW>;
+ status = "okay";
+
+ switch@0 {
+ compatible = "microchip,ksz9477";
+ reg = <0>;
+ spi-max-frequency = <1000000>;
+ spi-cpha;
+ spi-cpol;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ label = "lan1";
+ };
+
+ port@1 {
+ reg = <1>;
+ label = "lan2";
+ };
+
+ port@2 {
+ reg = <2>;
+ label = "lan3";
+ };
+
+ port@3 {
+ reg = <3>;
+ label = "lan4";
+ };
+
+ port@4 {
+ reg = <4>;
+ label = "lan5";
+ };
+
+ port@5 {
+ reg = <5>;
+ label = "cpu";
+ ethernet = <&macb0>;
+ phy-mode = "rgmii-txid";
+
+ fixed-link {
+ speed = <1000>;
+ full-duplex;
+ };
+ };
+ };
+ };
+};
+
+&usb0 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_usba_vbus>;
+ atmel,vbus-gpio = <&pioE 9 GPIO_ACTIVE_HIGH>;
+ status = "okay";
+};
+
+&pinctrl {
+ board {
+ pinctrl_i2c0_pu: i2c0-pu {
+ atmel,pins =
+ <AT91_PIOA 30 AT91_PERIPH_A AT91_PINCTRL_PULL_UP>,
+ <AT91_PIOA 31 AT91_PERIPH_A AT91_PINCTRL_PULL_UP>;
+ };
+
+ pinctrl_mmc0_cd: mmc0-cd {
+ atmel,pins = <AT91_PIOE 0 AT91_PERIPH_GPIO AT91_PINCTRL_NONE>;
+ };
+
+ pinctrl_mcc0_vcc: mmc0-vcc {
+ atmel,pins = <AT91_PIOE 2 AT91_PERIPH_GPIO AT91_PINCTRL_NONE>;
+ };
+
+ pinctrl_spi_ksz: spi-ksz {
+ atmel,pins =
+ <
+ /* SPI1_MISO */
+ AT91_PIOC 22 AT91_PERIPH_A AT91_PINCTRL_PULL_UP
+ /* SPI1_MOSI */
+ AT91_PIOC 23 AT91_PERIPH_A AT91_PINCTRL_NONE
+ /* SPI1_SPCK */
+ AT91_PIOC 24 AT91_PERIPH_A AT91_PINCTRL_NONE
+
+ /* SPI CS */
+ AT91_PIOC 25 AT91_PERIPH_GPIO AT91_PINCTRL_NONE
+ /* switch IRQ */
+ AT91_PIOB 28 AT91_PERIPH_GPIO AT91_PINCTRL_DEGLITCH
+ /* switch PME_N, SoC IN */
+ AT91_PIOC 30 AT91_PERIPH_GPIO AT91_PINCTRL_PULL_UP
+ /* switch RST */
+ AT91_PIOC 31 AT91_PERIPH_GPIO AT91_PINCTRL_PULL_UP_DEGLITCH
+ >;
+ };
+
+ pinctrl_usba_vbus: usba-vbus {
+ atmel,pins =
+ <AT91_PIOE 9 AT91_PERIPH_GPIO AT91_PINCTRL_DEGLITCH>;
+ };
+ };
+};
diff --git a/arch/arm/boot/dts/at91-sama7g5ek.dts b/arch/arm/boot/dts/at91-sama7g5ek.dts
index 0e1975c..ccf9e22 100644
--- a/arch/arm/boot/dts/at91-sama7g5ek.dts
+++ b/arch/arm/boot/dts/at91-sama7g5ek.dts
@@ -13,6 +13,7 @@
#include "sama7g5.dtsi"
#include <dt-bindings/mfd/atmel-flexcom.h>
#include <dt-bindings/input/input.h>
+#include <dt-bindings/pinctrl/at91.h>
/ {
model = "Microchip SAMA7G5-EK";
@@ -134,6 +135,59 @@ &cpu0 {
cpu-supply = <&vddcpu>;
};
+&qspi0 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_qspi>;
+ status = "okay";
+
+ flash@0 {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ compatible = "jedec,spi-nor";
+ reg = <0>;
+ spi-max-frequency = <133000000>;
+ spi-tx-bus-width = <8>;
+ spi-rx-bus-width = <8>;
+ m25p,fast-read;
+
+ at91bootstrap@0 {
+ label = "ospi: at91bootstrap";
+ reg = <0x0 0x40000>;
+ };
+
+ bootloader@40000 {
+ label = "ospi: bootloader";
+ reg = <0x40000 0xc0000>;
+ };
+
+ bootloaderenvred@100000 {
+ label = "ospi: bootloader env redundant";
+ reg = <0x100000 0x40000>;
+ };
+
+ bootloaderenv@140000 {
+ label = "ospi: bootloader env";
+ reg = <0x140000 0x40000>;
+ };
+
+ dtb@180000 {
+ label = "ospi: device tree";
+ reg = <0x180000 0x80000>;
+ };
+
+ kernel@200000 {
+ label = "ospi: kernel";
+ reg = <0x200000 0x600000>;
+ };
+
+ rootfs@800000 {
+ label = "ospi: rootfs";
+ reg = <0x800000 0x7800000>;
+ };
+
+ };
+};
+
&dma0 {
status = "okay";
};
@@ -555,6 +609,25 @@ pinctrl_mikrobus1_spi: mikrobus1_spi {
bias-disable;
};
+ pinctrl_qspi: qspi {
+ pinmux = <PIN_PB12__QSPI0_IO0>,
+ <PIN_PB11__QSPI0_IO1>,
+ <PIN_PB10__QSPI0_IO2>,
+ <PIN_PB9__QSPI0_IO3>,
+ <PIN_PB16__QSPI0_IO4>,
+ <PIN_PB17__QSPI0_IO5>,
+ <PIN_PB18__QSPI0_IO6>,
+ <PIN_PB19__QSPI0_IO7>,
+ <PIN_PB13__QSPI0_CS>,
+ <PIN_PB14__QSPI0_SCK>,
+ <PIN_PB15__QSPI0_SCKN>,
+ <PIN_PB20__QSPI0_DQS>,
+ <PIN_PB21__QSPI0_INT>;
+ bias-disable;
+ slew-rate = <0>;
+ atmel,drive-strength = <ATMEL_PIO_DRVSTR_HI>;
+ };
+
pinctrl_sdmmc0_default: sdmmc0_default {
cmd_data {
pinmux = <PIN_PA1__SDMMC0_CMD>,
diff --git a/arch/arm/boot/dts/at91-wb50n.dts b/arch/arm/boot/dts/at91-wb50n.dts
index a5e45bb..89f0f71 100644
--- a/arch/arm/boot/dts/at91-wb50n.dts
+++ b/arch/arm/boot/dts/at91-wb50n.dts
@@ -90,12 +90,6 @@ &i2c0 {
&spi1 {
status = "okay";
-
- spidev@0 {
- compatible = "spidev";
- reg = <0>;
- spi-max-frequency = <8000000>;
- };
};
&usb0 {
diff --git a/arch/arm/boot/dts/bcm-cygnus.dtsi b/arch/arm/boot/dts/bcm-cygnus.dtsi
index 8ecb786..ad65be8 100644
--- a/arch/arm/boot/dts/bcm-cygnus.dtsi
+++ b/arch/arm/boot/dts/bcm-cygnus.dtsi
@@ -112,18 +112,18 @@ otp: otp@301c800 {
status = "disabled";
};
- pcie_phy: phy@301d0a0 {
+ pcie_phy: pcie_phy@301d0a0 {
compatible = "brcm,cygnus-pcie-phy";
reg = <0x0301d0a0 0x14>;
#address-cells = <1>;
#size-cells = <0>;
- pcie0_phy: phy@0 {
+ pcie0_phy: pcie-phy@0 {
reg = <0>;
#phy-cells = <0>;
};
- pcie1_phy: phy@1 {
+ pcie1_phy: pcie-phy@1 {
reg = <1>;
#phy-cells = <0>;
};
@@ -274,8 +274,8 @@ pcie0: pcie@18012000 {
#address-cells = <3>;
#size-cells = <2>;
device_type = "pci";
- ranges = <0x81000000 0 0 0x28000000 0 0x00010000
- 0x82000000 0 0x20000000 0x20000000 0 0x04000000>;
+ ranges = <0x81000000 0 0 0x28000000 0 0x00010000>,
+ <0x82000000 0 0x20000000 0x20000000 0 0x04000000>;
phys = <&pcie0_phy>;
phy-names = "pcie-phy";
@@ -283,7 +283,7 @@ pcie0: pcie@18012000 {
status = "disabled";
msi-parent = <&msi0>;
- msi0: msi-controller {
+ msi0: msi {
compatible = "brcm,iproc-msi";
msi-controller;
interrupt-parent = <&gic>;
@@ -309,8 +309,8 @@ pcie1: pcie@18013000 {
#address-cells = <3>;
#size-cells = <2>;
device_type = "pci";
- ranges = <0x81000000 0 0 0x48000000 0 0x00010000
- 0x82000000 0 0x40000000 0x40000000 0 0x04000000>;
+ ranges = <0x81000000 0 0 0x48000000 0 0x00010000>,
+ <0x82000000 0 0x40000000 0x40000000 0 0x04000000>;
phys = <&pcie1_phy>;
phy-names = "pcie-phy";
@@ -318,7 +318,7 @@ pcie1: pcie@18013000 {
status = "disabled";
msi-parent = <&msi1>;
- msi1: msi-controller {
+ msi1: msi {
compatible = "brcm,iproc-msi";
msi-controller;
interrupt-parent = <&gic>;
diff --git a/arch/arm/boot/dts/bcm-hr2.dtsi b/arch/arm/boot/dts/bcm-hr2.dtsi
index 84cda16..33e6ba6 100644
--- a/arch/arm/boot/dts/bcm-hr2.dtsi
+++ b/arch/arm/boot/dts/bcm-hr2.dtsi
@@ -318,7 +318,7 @@ pcie0: pcie@18012000 {
status = "disabled";
msi-parent = <&msi0>;
- msi0: msi-controller {
+ msi0: msi {
compatible = "brcm,iproc-msi";
msi-controller;
interrupt-parent = <&gic>;
@@ -354,7 +354,7 @@ pcie1: pcie@18013000 {
status = "disabled";
msi-parent = <&msi1>;
- msi1: msi-controller {
+ msi1: msi {
compatible = "brcm,iproc-msi";
msi-controller;
interrupt-parent = <&gic>;
diff --git a/arch/arm/boot/dts/bcm-nsp.dtsi b/arch/arm/boot/dts/bcm-nsp.dtsi
index 1c08daa..5b1dc58 100644
--- a/arch/arm/boot/dts/bcm-nsp.dtsi
+++ b/arch/arm/boot/dts/bcm-nsp.dtsi
@@ -534,7 +534,7 @@ sata_phy1: sata-phy@1 {
};
};
- sata: ahci@41000 {
+ sata: sata@41000 {
compatible = "brcm,bcm-nsp-ahci";
reg-names = "ahci", "top-ctrl";
reg = <0x41000 0x1000>, <0x40020 0x1c>;
@@ -587,7 +587,7 @@ pcie0: pcie@18012000 {
status = "disabled";
msi-parent = <&msi0>;
- msi0: msi-controller {
+ msi0: msi {
compatible = "brcm,iproc-msi";
msi-controller;
interrupt-parent = <&gic>;
@@ -624,7 +624,7 @@ pcie1: pcie@18013000 {
status = "disabled";
msi-parent = <&msi1>;
- msi1: msi-controller {
+ msi1: msi {
compatible = "brcm,iproc-msi";
msi-controller;
interrupt-parent = <&gic>;
@@ -661,7 +661,7 @@ pcie2: pcie@18014000 {
status = "disabled";
msi-parent = <&msi2>;
- msi2: msi-controller {
+ msi2: msi {
compatible = "brcm,iproc-msi";
msi-controller;
interrupt-parent = <&gic>;
diff --git a/arch/arm/boot/dts/bcm2711-rpi-4-b.dts b/arch/arm/boot/dts/bcm2711-rpi-4-b.dts
index 631dd5b..4432412 100644
--- a/arch/arm/boot/dts/bcm2711-rpi-4-b.dts
+++ b/arch/arm/boot/dts/bcm2711-rpi-4-b.dts
@@ -65,12 +65,12 @@ &ddc1 {
};
&expgpio {
- gpio-line-names = "BT_ON",
+ gpio-line-names = "BT_ON", /* 0 */
"WL_ON",
"PWR_LED_OFF",
"GLOBAL_RESET",
"VDD_SD_IO_SEL",
- "CAM_GPIO",
+ "CAM_GPIO", /* 5 */
"SD_PWR_ON",
"";
};
@@ -84,66 +84,66 @@ &gpio {
* "FOO" = GPIO line named "FOO" on the schematic
* "FOO_N" = GPIO line named "FOO" on schematic, active low
*/
- gpio-line-names = "ID_SDA",
+ gpio-line-names = "ID_SDA", /* 0 */
"ID_SCL",
"SDA1",
"SCL1",
"GPIO_GCLK",
- "GPIO5",
+ "GPIO5", /* 5 */
"GPIO6",
"SPI_CE1_N",
"SPI_CE0_N",
"SPI_MISO",
- "SPI_MOSI",
+ "SPI_MOSI", /* 10 */
"SPI_SCLK",
"GPIO12",
"GPIO13",
/* Serial port */
"TXD1",
- "RXD1",
+ "RXD1", /* 15 */
"GPIO16",
"GPIO17",
"GPIO18",
"GPIO19",
- "GPIO20",
+ "GPIO20", /* 20 */
"GPIO21",
"GPIO22",
"GPIO23",
"GPIO24",
- "GPIO25",
+ "GPIO25", /* 25 */
"GPIO26",
"GPIO27",
"RGMII_MDIO",
"RGMIO_MDC",
/* Used by BT module */
- "CTS0",
+ "CTS0", /* 30 */
"RTS0",
"TXD0",
"RXD0",
/* Used by Wifi */
"SD1_CLK",
- "SD1_CMD",
+ "SD1_CMD", /* 35 */
"SD1_DATA0",
"SD1_DATA1",
"SD1_DATA2",
"SD1_DATA3",
/* Shared with SPI flash */
- "PWM0_MISO",
+ "PWM0_MISO", /* 40 */
"PWM1_MOSI",
"STATUS_LED_G_CLK",
"SPIFLASH_CE_N",
"SDA0",
- "SCL0",
+ "SCL0", /* 45 */
"RGMII_RXCLK",
"RGMII_RXCTL",
"RGMII_RXD0",
"RGMII_RXD1",
- "RGMII_RXD2",
+ "RGMII_RXD2", /* 50 */
"RGMII_RXD3",
"RGMII_TXCLK",
"RGMII_TXCTL",
"RGMII_TXD0",
- "RGMII_TXD1",
+ "RGMII_TXD1", /* 55 */
"RGMII_TXD2",
"RGMII_TXD3";
};
diff --git a/arch/arm/boot/dts/bcm2711.dtsi b/arch/arm/boot/dts/bcm2711.dtsi
index 9e01dbc..dff18fc 100644
--- a/arch/arm/boot/dts/bcm2711.dtsi
+++ b/arch/arm/boot/dts/bcm2711.dtsi
@@ -582,6 +582,8 @@ &gpio {
<GIC_SPI 115 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 116 IRQ_TYPE_LEVEL_HIGH>;
+ gpio-ranges = <&gpio 0 0 58>;
+
gpclk0_gpio49: gpclk0_gpio49 {
pin-gpclk {
pins = "gpio49";
diff --git a/arch/arm/boot/dts/bcm283x.dtsi b/arch/arm/boot/dts/bcm283x.dtsi
index a3e06b6..c113661 100644
--- a/arch/arm/boot/dts/bcm283x.dtsi
+++ b/arch/arm/boot/dts/bcm283x.dtsi
@@ -126,6 +126,8 @@ gpio: gpio@7e200000 {
interrupt-controller;
#interrupt-cells = <2>;
+ gpio-ranges = <&gpio 0 0 54>;
+
/* Defines common pin muxing groups
*
* While each pin can have its mux selected
diff --git a/arch/arm/boot/dts/bcm47094-asus-rt-ac88u.dts b/arch/arm/boot/dts/bcm47094-asus-rt-ac88u.dts
index 4480605..249476f 100644
--- a/arch/arm/boot/dts/bcm47094-asus-rt-ac88u.dts
+++ b/arch/arm/boot/dts/bcm47094-asus-rt-ac88u.dts
@@ -68,8 +68,6 @@ wps {
gpio-keys {
compatible = "gpio-keys";
- #address-cells = <1>;
- #size-cells = <0>;
wps {
label = "WPS";
@@ -138,6 +136,7 @@ sw0_p5: port@5 {
fixed-link {
speed = <1000>;
full-duplex;
+ pause;
};
};
diff --git a/arch/arm/boot/dts/bcm53016-meraki-mr32.dts b/arch/arm/boot/dts/bcm53016-meraki-mr32.dts
index 64f973e..66c64a6 100644
--- a/arch/arm/boot/dts/bcm53016-meraki-mr32.dts
+++ b/arch/arm/boot/dts/bcm53016-meraki-mr32.dts
@@ -84,40 +84,6 @@ blue {
max-brightness = <255>;
};
};
-
- i2c {
- /*
- * The platform provided I2C does not budge.
- * This is a replacement until I can figure
- * out what are the missing bits...
- */
-
- compatible = "i2c-gpio";
- sda-gpios = <&chipcommon 5 GPIO_ACTIVE_HIGH>;
- scl-gpios = <&chipcommon 4 GPIO_ACTIVE_HIGH>;
- i2c-gpio,delay-us = <10>; /* close to 100 kHz */
- #address-cells = <1>;
- #size-cells = <0>;
-
- current_sense: ina219@45 {
- compatible = "ti,ina219";
- reg = <0x45>;
- shunt-resistor = <60000>; /* = 60 mOhms */
- };
-
- eeprom: eeprom@50 {
- compatible = "atmel,24c64";
- reg = <0x50>;
- pagesize = <32>;
- read-only;
- #address-cells = <1>;
- #size-cells = <1>;
-
- mac_address: mac-address@66 {
- reg = <0x66 0x6>;
- };
- };
- };
};
&uart0 {
@@ -228,3 +194,31 @@ fixed-link {
};
};
};
+
+&i2c0 {
+ status = "okay";
+
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinmux_i2c>;
+
+ clock-frequency = <100000>;
+
+ current_sense: ina219@45 {
+ compatible = "ti,ina219";
+ reg = <0x45>;
+ shunt-resistor = <60000>; /* = 60 mOhms */
+ };
+
+ eeprom: eeprom@50 {
+ compatible = "atmel,24c64";
+ reg = <0x50>;
+ pagesize = <32>;
+ read-only;
+ #address-cells = <1>;
+ #size-cells = <1>;
+
+ mac_address: mac-address@66 {
+ reg = <0x66 0x6>;
+ };
+ };
+};
diff --git a/arch/arm/boot/dts/bcm5301x.dtsi b/arch/arm/boot/dts/bcm5301x.dtsi
index f69d2af..603c700 100644
--- a/arch/arm/boot/dts/bcm5301x.dtsi
+++ b/arch/arm/boot/dts/bcm5301x.dtsi
@@ -148,15 +148,6 @@ periph_clk: periph_clk {
};
};
- usb2_phy: usb2-phy@1800c000 {
- compatible = "brcm,ns-usb2-phy";
- reg = <0x1800c000 0x1000>;
- reg-names = "dmu";
- #phy-cells = <0>;
- clocks = <&genpll BCM_NSP_GENPLL_USB_PHY_REF_CLK>;
- clock-names = "phy-ref-clk";
- };
-
axi@18000000 {
compatible = "brcm,bus-axi";
reg = <0x18000000 0x1000>;
@@ -423,14 +414,14 @@ dmu-bus@1800c000 {
#address-cells = <1>;
#size-cells = <1>;
- cru@100 {
- compatible = "simple-bus";
+ cru-bus@100 {
+ compatible = "brcm,ns-cru", "simple-mfd";
reg = <0x100 0x1a4>;
ranges;
#address-cells = <1>;
#size-cells = <1>;
- lcpll0: lcpll0@100 {
+ lcpll0: clock-controller@100 {
#clock-cells = <1>;
compatible = "brcm,nsp-lcpll0";
reg = <0x100 0x14>;
@@ -439,7 +430,7 @@ lcpll0: lcpll0@100 {
"sdio", "ddr_phy";
};
- genpll: genpll@140 {
+ genpll: clock-controller@140 {
#clock-cells = <1>;
compatible = "brcm,nsp-genpll";
reg = <0x140 0x24>;
@@ -450,6 +441,20 @@ genpll: genpll@140 {
"sata1", "sata2";
};
+ usb2_phy: phy@164 {
+ compatible = "brcm,ns-usb2-phy";
+ reg = <0x164 0x4>;
+ brcm,syscon-clkset = <&cru_clkset>;
+ clocks = <&genpll BCM_NSP_GENPLL_USB_PHY_REF_CLK>;
+ clock-names = "phy-ref-clk";
+ #phy-cells = <0>;
+ };
+
+ cru_clkset: syscon@180 {
+ compatible = "brcm,cru-clkset", "syscon";
+ reg = <0x180 0x4>;
+ };
+
pinctrl: pin-controller@1c0 {
compatible = "brcm,bcm4708-pinmux";
reg = <0x1c0 0x24>;
diff --git a/arch/arm/boot/dts/bcm958625-meraki-alamo.dtsi b/arch/arm/boot/dts/bcm958625-meraki-alamo.dtsi
index 102acd8..ba01054 100644
--- a/arch/arm/boot/dts/bcm958625-meraki-alamo.dtsi
+++ b/arch/arm/boot/dts/bcm958625-meraki-alamo.dtsi
@@ -118,6 +118,8 @@ port@0 {
reg = <0>;
ethernet = <&sgmii1>;
phy-mode = "sgmii";
+ qca,sgmii-enable-pll;
+ qca,sgmii-txclk-falling-edge;
fixed-link {
speed = <1000>;
full-duplex;
@@ -194,6 +196,8 @@ port@0 {
reg = <0>;
ethernet = <&sgmii0>;
phy-mode = "sgmii";
+ qca,sgmii-enable-pll;
+ qca,sgmii-txclk-falling-edge;
fixed-link {
speed = <1000>;
full-duplex;
diff --git a/arch/arm/boot/dts/elpida_ecb240abacn.dtsi b/arch/arm/boot/dts/elpida_ecb240abacn.dtsi
index d87ee47..9698801 100644
--- a/arch/arm/boot/dts/elpida_ecb240abacn.dtsi
+++ b/arch/arm/boot/dts/elpida_ecb240abacn.dtsi
@@ -5,7 +5,7 @@
/ {
elpida_ECB240ABACN: lpddr2 {
- compatible = "Elpida,ECB240ABACN","jedec,lpddr2-s4";
+ compatible = "elpida,ECB240ABACN","jedec,lpddr2-s4";
density = <2048>;
io-width = <32>;
diff --git a/arch/arm/boot/dts/exynos4210-i9100.dts b/arch/arm/boot/dts/exynos4210-i9100.dts
index 19bb7dc..3389405 100644
--- a/arch/arm/boot/dts/exynos4210-i9100.dts
+++ b/arch/arm/boot/dts/exynos4210-i9100.dts
@@ -828,9 +828,12 @@ bluetooth {
compatible = "brcm,bcm4330-bt";
shutdown-gpios = <&gpl0 4 GPIO_ACTIVE_HIGH>;
- reset-gpios = <&gpl1 0 GPIO_ACTIVE_HIGH>;
+ reset-gpios = <&gpl1 0 GPIO_ACTIVE_LOW>;
device-wakeup-gpios = <&gpx3 1 GPIO_ACTIVE_HIGH>;
- host-wakeup-gpios = <&gpx2 6 GPIO_ACTIVE_HIGH>;
+
+ interrupt-parent = <&gpx2>;
+ interrupts = <6 IRQ_TYPE_EDGE_FALLING>;
+ interrupt-names = "host-wakeup";
};
};
diff --git a/arch/arm/boot/dts/exynos5260.dtsi b/arch/arm/boot/dts/exynos5260.dtsi
index 52fa211..524d244 100644
--- a/arch/arm/boot/dts/exynos5260.dtsi
+++ b/arch/arm/boot/dts/exynos5260.dtsi
@@ -368,7 +368,7 @@ mmc_2: mmc@12160000 {
status = "disabled";
};
- hsi2c_0: hsi2c@12da0000 {
+ hsi2c_0: i2c@12da0000 {
compatible = "samsung,exynos5260-hsi2c";
reg = <0x12DA0000 0x1000>;
interrupts = <GIC_SPI 108 IRQ_TYPE_LEVEL_HIGH>;
@@ -381,7 +381,7 @@ hsi2c_0: hsi2c@12da0000 {
status = "disabled";
};
- hsi2c_1: hsi2c@12db0000 {
+ hsi2c_1: i2c@12db0000 {
compatible = "samsung,exynos5260-hsi2c";
reg = <0x12DB0000 0x1000>;
interrupts = <GIC_SPI 109 IRQ_TYPE_LEVEL_HIGH>;
@@ -394,7 +394,7 @@ hsi2c_1: hsi2c@12db0000 {
status = "disabled";
};
- hsi2c_2: hsi2c@12dc0000 {
+ hsi2c_2: i2c@12dc0000 {
compatible = "samsung,exynos5260-hsi2c";
reg = <0x12DC0000 0x1000>;
interrupts = <GIC_SPI 110 IRQ_TYPE_LEVEL_HIGH>;
@@ -407,7 +407,7 @@ hsi2c_2: hsi2c@12dc0000 {
status = "disabled";
};
- hsi2c_3: hsi2c@12dd0000 {
+ hsi2c_3: i2c@12dd0000 {
compatible = "samsung,exynos5260-hsi2c";
reg = <0x12DD0000 0x1000>;
interrupts = <GIC_SPI 111 IRQ_TYPE_LEVEL_HIGH>;
diff --git a/arch/arm/boot/dts/gemini-nas4220b.dts b/arch/arm/boot/dts/gemini-nas4220b.dts
index 13112a8..6544c73 100644
--- a/arch/arm/boot/dts/gemini-nas4220b.dts
+++ b/arch/arm/boot/dts/gemini-nas4220b.dts
@@ -84,7 +84,7 @@ flash@30000000 {
partitions {
compatible = "redboot-fis";
/* Eraseblock at 0xfe0000 */
- fis-index-block = <0x1fc>;
+ fis-index-block = <0x7f>;
};
};
diff --git a/arch/arm/boot/dts/imx1-pinfunc.h b/arch/arm/boot/dts/imx1-pinfunc.h
index 050a1fc..bd2e679 100644
--- a/arch/arm/boot/dts/imx1-pinfunc.h
+++ b/arch/arm/boot/dts/imx1-pinfunc.h
@@ -26,9 +26,9 @@
* 2 - 0
* 3 - 1
*
- * 'pin' is an integer between 0 and 0xbf. i.MX1 has 4 ports with 32 configurable
- * configurable pins each. 'pin' is PORT * 32 + PORT_PIN, PORT_PIN is the pin
- * number on the specific port (between 0 and 31).
+ * 'pin' is an integer between 0 and 0xbf. i.MX1 has 4 ports with 32
+ * configurable pins each. 'pin' is PORT * 32 + PORT_PIN, PORT_PIN is
+ * the pin number on the specific port (between 0 and 31).
*/
#define MX1_PAD_A24__A24 0x00 0x004
diff --git a/arch/arm/boot/dts/imx1.dtsi b/arch/arm/boot/dts/imx1.dtsi
index 9b94098..e312f1e 100644
--- a/arch/arm/boot/dts/imx1.dtsi
+++ b/arch/arm/boot/dts/imx1.dtsi
@@ -55,7 +55,7 @@ cpu@0 {
clocks {
clk32 {
- compatible = "fsl,imx-clk32", "fixed-clock";
+ compatible = "fixed-clock";
#clock-cells = <0>;
clock-frequency = <32000>;
};
diff --git a/arch/arm/boot/dts/imx25.dtsi b/arch/arm/boot/dts/imx25.dtsi
index fdcca82..fa8044c 100644
--- a/arch/arm/boot/dts/imx25.dtsi
+++ b/arch/arm/boot/dts/imx25.dtsi
@@ -62,7 +62,7 @@ asic: asic-interrupt-controller@68000000 {
clocks {
osc {
- compatible = "fsl,imx-osc", "fixed-clock";
+ compatible = "fixed-clock";
#clock-cells = <0>;
clock-frequency = <24000000>;
};
@@ -200,7 +200,7 @@ audmux: audmux@43fb0000 {
};
};
- spba@50000000 {
+ spba-bus@50000000 {
compatible = "fsl,spba-bus", "simple-bus";
#address-cells = <1>;
#size-cells = <1>;
diff --git a/arch/arm/boot/dts/imx27-pinfunc.h b/arch/arm/boot/dts/imx27-pinfunc.h
index 1514d80..75aea0c 100644
--- a/arch/arm/boot/dts/imx27-pinfunc.h
+++ b/arch/arm/boot/dts/imx27-pinfunc.h
@@ -26,9 +26,9 @@
* 2 - 0
* 3 - 1
*
- * 'pin' is an integer between 0 and 0xbf. imx27 has 6 ports with 32 configurable
- * configurable pins each. 'pin' is PORT * 32 + PORT_PIN, PORT_PIN is the pin
- * number on the specific port (between 0 and 31).
+ * 'pin' is an integer between 0 and 0xbf. imx27 has 6 ports with 32
+ * configurable pins each. 'pin' is PORT * 32 + PORT_PIN, PORT_PIN is
+ * the pin number on the specific port (between 0 and 31).
*/
#define MX27_PAD_USBH2_CLK__USBH2_CLK 0x00 0x000
diff --git a/arch/arm/boot/dts/imx28-tx28.dts b/arch/arm/boot/dts/imx28-tx28.dts
index 164254c..9e5651c 100644
--- a/arch/arm/boot/dts/imx28-tx28.dts
+++ b/arch/arm/boot/dts/imx28-tx28.dts
@@ -239,24 +239,6 @@ &gpio3 9 GPIO_ACTIVE_LOW
>;
/* enable this and disable ssp3 below, if you need full duplex SPI transfer */
status = "disabled";
-
- spi@0 {
- compatible = "spidev";
- reg = <0>;
- spi-max-frequency = <57600000>;
- };
-
- spi@1 {
- compatible = "spidev";
- reg = <1>;
- spi-max-frequency = <57600000>;
- };
-
- spi@2 {
- compatible = "spidev";
- reg = <2>;
- spi-max-frequency = <57600000>;
- };
};
};
@@ -700,24 +682,6 @@ &ssp3 {
pinctrl-0 = <&spi3_pins_a>;
clock-frequency = <57600000>;
status = "okay";
-
- spi@0 {
- compatible = "spidev";
- reg = <0>;
- spi-max-frequency = <57600000>;
- };
-
- spi@1 {
- compatible = "spidev";
- reg = <1>;
- spi-max-frequency = <57600000>;
- };
-
- spi@2 {
- compatible = "spidev";
- reg = <2>;
- spi-max-frequency = <57600000>;
- };
};
&usb0 {
diff --git a/arch/arm/boot/dts/imx31.dtsi b/arch/arm/boot/dts/imx31.dtsi
index 948d2a5..2adb923 100644
--- a/arch/arm/boot/dts/imx31.dtsi
+++ b/arch/arm/boot/dts/imx31.dtsi
@@ -166,7 +166,7 @@ uart5: serial@43fb4000 {
};
};
- spba@50000000 {
+ spba-bus@50000000 {
compatible = "fsl,spba-bus", "simple-bus";
#address-cells = <1>;
#size-cells = <1>;
diff --git a/arch/arm/boot/dts/imx50.dtsi b/arch/arm/boot/dts/imx50.dtsi
index a969f33..be0de0f 100644
--- a/arch/arm/boot/dts/imx50.dtsi
+++ b/arch/arm/boot/dts/imx50.dtsi
@@ -62,25 +62,25 @@ tzic: tz-interrupt-controller@fffc000 {
clocks {
ckil {
- compatible = "fsl,imx-ckil", "fixed-clock";
+ compatible = "fixed-clock";
#clock-cells = <0>;
clock-frequency = <32768>;
};
ckih1 {
- compatible = "fsl,imx-ckih1", "fixed-clock";
+ compatible = "fixed-clock";
#clock-cells = <0>;
clock-frequency = <22579200>;
};
ckih2 {
- compatible = "fsl,imx-ckih2", "fixed-clock";
+ compatible = "fixed-clock";
#clock-cells = <0>;
clock-frequency = <0>;
};
osc {
- compatible = "fsl,imx-osc", "fixed-clock";
+ compatible = "fixed-clock";
#clock-cells = <0>;
clock-frequency = <24000000>;
};
@@ -108,7 +108,7 @@ bus@50000000 { /* AIPS1 */
reg = <0x50000000 0x10000000>;
ranges;
- spba@50000000 {
+ spba-bus@50000000 {
compatible = "fsl,spba-bus", "simple-bus";
#address-cells = <1>;
#size-cells = <1>;
diff --git a/arch/arm/boot/dts/imx51.dtsi b/arch/arm/boot/dts/imx51.dtsi
index 01cfcbe..56c8d87 100644
--- a/arch/arm/boot/dts/imx51.dtsi
+++ b/arch/arm/boot/dts/imx51.dtsi
@@ -48,25 +48,25 @@ tzic: tz-interrupt-controller@e0000000 {
clocks {
ckil {
- compatible = "fsl,imx-ckil", "fixed-clock";
+ compatible = "fixed-clock";
#clock-cells = <0>;
clock-frequency = <32768>;
};
ckih1 {
- compatible = "fsl,imx-ckih1", "fixed-clock";
+ compatible = "fixed-clock";
#clock-cells = <0>;
clock-frequency = <0>;
};
ckih2 {
- compatible = "fsl,imx-ckih2", "fixed-clock";
+ compatible = "fixed-clock";
#clock-cells = <0>;
clock-frequency = <0>;
};
osc {
- compatible = "fsl,imx-osc", "fixed-clock";
+ compatible = "fixed-clock";
#clock-cells = <0>;
clock-frequency = <24000000>;
};
@@ -178,7 +178,7 @@ bus@70000000 { /* AIPS1 */
reg = <0x70000000 0x10000000>;
ranges;
- spba@70000000 {
+ spba-bus@70000000 {
compatible = "fsl,spba-bus", "simple-bus";
#address-cells = <1>;
#size-cells = <1>;
diff --git a/arch/arm/boot/dts/imx53-tx53.dtsi b/arch/arm/boot/dts/imx53-tx53.dtsi
index 7c9730f..81c2726 100644
--- a/arch/arm/boot/dts/imx53-tx53.dtsi
+++ b/arch/arm/boot/dts/imx53-tx53.dtsi
@@ -192,17 +192,6 @@ &gpio2 30 GPIO_ACTIVE_HIGH
&gpio3 19 GPIO_ACTIVE_HIGH
>;
- spidev0: spi@0 {
- compatible = "spidev";
- reg = <0>;
- spi-max-frequency = <54000000>;
- };
-
- spidev1: spi@1 {
- compatible = "spidev";
- reg = <1>;
- spi-max-frequency = <54000000>;
- };
};
&esdhc1 {
diff --git a/arch/arm/boot/dts/imx53.dtsi b/arch/arm/boot/dts/imx53.dtsi
index 2cf3909..67487f3 100644
--- a/arch/arm/boot/dts/imx53.dtsi
+++ b/arch/arm/boot/dts/imx53.dtsi
@@ -86,25 +86,25 @@ tzic: tz-interrupt-controller@fffc000 {
clocks {
ckil {
- compatible = "fsl,imx-ckil", "fixed-clock";
+ compatible = "fixed-clock";
#clock-cells = <0>;
clock-frequency = <32768>;
};
ckih1 {
- compatible = "fsl,imx-ckih1", "fixed-clock";
+ compatible = "fixed-clock";
#clock-cells = <0>;
clock-frequency = <22579200>;
};
ckih2 {
- compatible = "fsl,imx-ckih2", "fixed-clock";
+ compatible = "fixed-clock";
#clock-cells = <0>;
clock-frequency = <0>;
};
osc {
- compatible = "fsl,imx-osc", "fixed-clock";
+ compatible = "fixed-clock";
#clock-cells = <0>;
clock-frequency = <24000000>;
};
@@ -229,7 +229,7 @@ bus@50000000 { /* AIPS1 */
reg = <0x50000000 0x10000000>;
ranges;
- spba@50000000 {
+ spba-bus@50000000 {
compatible = "fsl,spba-bus", "simple-bus";
#address-cells = <1>;
#size-cells = <1>;
diff --git a/arch/arm/boot/dts/imx6dl-mba6.dtsi b/arch/arm/boot/dts/imx6dl-mba6.dtsi
new file mode 100644
index 0000000..b749b42
--- /dev/null
+++ b/arch/arm/boot/dts/imx6dl-mba6.dtsi
@@ -0,0 +1,22 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright 2013 Sascha Hauer, Pengutronix
+ *
+ * Copyright 2013-2021 TQ-Systems GmbH
+ * Author: Markus Niebel <Markus.Niebel@tq-group.com>
+ */
+
+ðphy {
+ rxdv-skew-ps = <180>;
+ txen-skew-ps = <0>;
+ rxd3-skew-ps = <180>;
+ rxd2-skew-ps = <180>;
+ rxd1-skew-ps = <180>;
+ rxd0-skew-ps = <180>;
+ txd3-skew-ps = <120>;
+ txd2-skew-ps = <0>;
+ txd1-skew-ps = <300>;
+ txd0-skew-ps = <120>;
+ txc-skew-ps = <1860>;
+ rxc-skew-ps = <1860>;
+};
diff --git a/arch/arm/boot/dts/imx6dl-mba6a.dts b/arch/arm/boot/dts/imx6dl-mba6a.dts
new file mode 100644
index 0000000..df0a96b
--- /dev/null
+++ b/arch/arm/boot/dts/imx6dl-mba6a.dts
@@ -0,0 +1,21 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright 2013 Sascha Hauer, Pengutronix
+ *
+ * Copyright 2013-2021 TQ-Systems GmbH
+ * Author: Markus Niebel <Markus.Niebel@tq-group.com>
+ */
+
+/dts-v1/;
+
+#include <dt-bindings/gpio/gpio.h>
+#include "imx6dl-tqma6a.dtsi"
+#include "imx6qdl-mba6.dtsi"
+#include "imx6qdl-mba6a.dtsi"
+#include "imx6dl-mba6.dtsi"
+
+/ {
+ model = "TQ TQMa6S/DL on MBa6x";
+ compatible = "tq,imx6dl-mba6x-a", "tq,mba6a",
+ "tq,imx6dl-tqma6dl-a", "fsl,imx6dl";
+};
diff --git a/arch/arm/boot/dts/imx6dl-mba6b.dts b/arch/arm/boot/dts/imx6dl-mba6b.dts
new file mode 100644
index 0000000..610b19d
--- /dev/null
+++ b/arch/arm/boot/dts/imx6dl-mba6b.dts
@@ -0,0 +1,21 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright 2013 Sascha Hauer, Pengutronix
+ *
+ * Copyright 2013-2021 TQ-Systems GmbH
+ * Author: Markus Niebel <Markus.Niebel@tq-group.com>
+ */
+
+/dts-v1/;
+
+#include <dt-bindings/gpio/gpio.h>
+#include "imx6dl-tqma6b.dtsi"
+#include "imx6qdl-mba6.dtsi"
+#include "imx6qdl-mba6b.dtsi"
+#include "imx6dl-mba6.dtsi"
+
+/ {
+ model = "TQ TQMa6S/DL on MBa6x";
+ compatible = "tq,imx6dl-mba6x-b", "tq,mba6b",
+ "tq,imx6dl-tqma6dl-b", "fsl,imx6dl";
+};
diff --git a/arch/arm/boot/dts/imx6dl-phytec-mira-rdk-nand.dts b/arch/arm/boot/dts/imx6dl-phytec-mira-rdk-nand.dts
index 9f7f9f9..d906a7f 100644
--- a/arch/arm/boot/dts/imx6dl-phytec-mira-rdk-nand.dts
+++ b/arch/arm/boot/dts/imx6dl-phytec-mira-rdk-nand.dts
@@ -8,6 +8,9 @@
#include "imx6dl.dtsi"
#include "imx6qdl-phytec-phycore-som.dtsi"
#include "imx6qdl-phytec-mira.dtsi"
+#include "imx6qdl-phytec-mira-peb-eval-01.dtsi"
+#include "imx6qdl-phytec-mira-peb-av-02.dtsi"
+#include "imx6qdl-phytec-mira-peb-wlbt-05.dtsi"
/ {
model = "PHYTEC phyBOARD-Mira DualLite/Solo Carrier-Board with NAND";
diff --git a/arch/arm/boot/dts/imx6q-mba6.dtsi b/arch/arm/boot/dts/imx6q-mba6.dtsi
new file mode 100644
index 0000000..0d7be45
--- /dev/null
+++ b/arch/arm/boot/dts/imx6q-mba6.dtsi
@@ -0,0 +1,44 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright 2013 Sascha Hauer, Pengutronix
+ *
+ * Copyright 2013-2021 TQ-Systems GmbH
+ * Author: Markus Niebel <Markus.Niebel@tq-group.com>
+ */
+
+&ecspi5 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_ecspi5_mba6x>;
+ cs-gpios = <&gpio1 17 GPIO_ACTIVE_LOW>;
+};
+
+ðphy {
+ rxdv-skew-ps = <180>;
+ txen-skew-ps = <120>;
+ rxd3-skew-ps = <180>;
+ rxd2-skew-ps = <180>;
+ rxd1-skew-ps = <180>;
+ rxd0-skew-ps = <180>;
+ txd3-skew-ps = <120>;
+ txd2-skew-ps = <0>;
+ txd1-skew-ps = <180>;
+ txd0-skew-ps = <360>;
+ txc-skew-ps = <1860>;
+ rxc-skew-ps = <1860>;
+};
+
+&sata {
+ status = "okay";
+};
+
+&iomuxc {
+ pinctrl_ecspi5_mba6x: ecspi5grp-mba6x {
+ fsl,pins = <
+ /* HYS, SPEED = MED, 100k up, DSE = 011, SRE_FAST */
+ MX6QDL_PAD_SD1_DAT0__ECSPI5_MISO 0x1b099
+ MX6QDL_PAD_SD1_CMD__ECSPI5_MOSI 0xb099
+ MX6QDL_PAD_SD1_CLK__ECSPI5_SCLK 0xb099
+ MX6QDL_PAD_SD1_DAT1__GPIO1_IO17 0xb099 /* eCSPI5 SS0 */
+ >;
+ };
+};
diff --git a/arch/arm/boot/dts/imx6q-mba6a.dts b/arch/arm/boot/dts/imx6q-mba6a.dts
new file mode 100644
index 0000000..349a086
--- /dev/null
+++ b/arch/arm/boot/dts/imx6q-mba6a.dts
@@ -0,0 +1,20 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright 2013 Sascha Hauer, Pengutronix
+ *
+ * Copyright 2013-2021 TQ-Systems GmbH
+ * Author: Markus Niebel <Markus.Niebel@tq-group.com>
+ */
+
+/dts-v1/;
+
+#include "imx6q-tqma6a.dtsi"
+#include "imx6qdl-mba6.dtsi"
+#include "imx6qdl-mba6a.dtsi"
+#include "imx6q-mba6.dtsi"
+
+/ {
+ model = "TQ TQMa6Q on MBa6x";
+ compatible = "tq,imx6q-mba6x-a", "tq,mba6a",
+ "tq,imx6q-tqma6q-a", "fsl,imx6q";
+};
diff --git a/arch/arm/boot/dts/imx6q-mba6b.dts b/arch/arm/boot/dts/imx6q-mba6b.dts
new file mode 100644
index 0000000..02c9f3e
--- /dev/null
+++ b/arch/arm/boot/dts/imx6q-mba6b.dts
@@ -0,0 +1,20 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright 2013 Sascha Hauer, Pengutronix
+ *
+ * Copyright 2013-2021 TQ-Systems GmbH
+ * Author: Markus Niebel <Markus.Niebel@tq-group.com>
+ */
+
+/dts-v1/;
+
+#include "imx6q-tqma6b.dtsi"
+#include "imx6qdl-mba6.dtsi"
+#include "imx6qdl-mba6b.dtsi"
+#include "imx6q-mba6.dtsi"
+
+/ {
+ model = "TQ TQMa6Q on MBa6x";
+ compatible = "tq,imx6q-mba6x-b", "tq,mba6b",
+ "tq,imx6q-tqma6q-b", "fsl,imx6q";
+};
diff --git a/arch/arm/boot/dts/imx6q-phytec-mira-rdk-emmc.dts b/arch/arm/boot/dts/imx6q-phytec-mira-rdk-emmc.dts
index 2e70ea5..322f071d 100644
--- a/arch/arm/boot/dts/imx6q-phytec-mira-rdk-emmc.dts
+++ b/arch/arm/boot/dts/imx6q-phytec-mira-rdk-emmc.dts
@@ -8,6 +8,9 @@
#include "imx6q.dtsi"
#include "imx6qdl-phytec-phycore-som.dtsi"
#include "imx6qdl-phytec-mira.dtsi"
+#include "imx6qdl-phytec-mira-peb-eval-01.dtsi"
+#include "imx6qdl-phytec-mira-peb-av-02.dtsi"
+#include "imx6qdl-phytec-mira-peb-wlbt-05.dtsi"
/ {
model = "PHYTEC phyBOARD-Mira Quad Carrier-Board with eMMC";
diff --git a/arch/arm/boot/dts/imx6q-phytec-mira-rdk-nand.dts b/arch/arm/boot/dts/imx6q-phytec-mira-rdk-nand.dts
index 65d2e48..3f13726 100644
--- a/arch/arm/boot/dts/imx6q-phytec-mira-rdk-nand.dts
+++ b/arch/arm/boot/dts/imx6q-phytec-mira-rdk-nand.dts
@@ -8,6 +8,9 @@
#include "imx6q.dtsi"
#include "imx6qdl-phytec-phycore-som.dtsi"
#include "imx6qdl-phytec-mira.dtsi"
+#include "imx6qdl-phytec-mira-peb-eval-01.dtsi"
+#include "imx6qdl-phytec-mira-peb-av-02.dtsi"
+#include "imx6qdl-phytec-mira-peb-wlbt-05.dtsi"
/ {
model = "PHYTEC phyBOARD-Mira Quad Carrier-Board with NAND";
diff --git a/arch/arm/boot/dts/imx6q-yapp4-crux.dts b/arch/arm/boot/dts/imx6q-yapp4-crux.dts
new file mode 100644
index 0000000..15f4824
--- /dev/null
+++ b/arch/arm/boot/dts/imx6q-yapp4-crux.dts
@@ -0,0 +1,54 @@
+// SPDX-License-Identifier: GPL-2.0
+//
+// Copyright (C) 2021 Y Soft Corporation, a.s.
+
+/dts-v1/;
+
+#include "imx6q.dtsi"
+#include "imx6dl-yapp4-common.dtsi"
+
+/ {
+ model = "Y Soft IOTA Crux i.MX6Quad board";
+ compatible = "ysoft,imx6q-yapp4-crux", "fsl,imx6q";
+
+ memory@10000000 {
+ device_type = "memory";
+ reg = <0x10000000 0xf0000000>;
+ };
+};
+
+&gpio_oled {
+ status = "okay";
+};
+
+&leds {
+ status = "okay";
+};
+
+&oled_1305 {
+ status = "okay";
+};
+
+&oled_1309 {
+ status = "okay";
+};
+
+®_usb_h1_vbus {
+ status = "okay";
+};
+
+&touchkeys {
+ status = "okay";
+};
+
+&uart2 {
+ status = "disabled";
+};
+
+&usbh1 {
+ status = "okay";
+};
+
+&usbphy2 {
+ status = "okay";
+};
diff --git a/arch/arm/boot/dts/imx6qdl-dhcom-drc02.dtsi b/arch/arm/boot/dts/imx6qdl-dhcom-drc02.dtsi
index 3d0a50a..702cd4a 100644
--- a/arch/arm/boot/dts/imx6qdl-dhcom-drc02.dtsi
+++ b/arch/arm/boot/dts/imx6qdl-dhcom-drc02.dtsi
@@ -95,6 +95,10 @@ &uart5 {
rts-gpios = <&gpio7 13 GPIO_ACTIVE_HIGH>; /* GPIO P */
};
+&usbh1 {
+ disable-over-current;
+};
+
&usdhc2 { /* SD card */
status = "okay";
};
diff --git a/arch/arm/boot/dts/imx6qdl-dhcom-pdk2.dtsi b/arch/arm/boot/dts/imx6qdl-dhcom-pdk2.dtsi
index dc21853..dc89b55 100644
--- a/arch/arm/boot/dts/imx6qdl-dhcom-pdk2.dtsi
+++ b/arch/arm/boot/dts/imx6qdl-dhcom-pdk2.dtsi
@@ -260,6 +260,10 @@ &ssi1 {
status = "okay";
};
+&usbh1 {
+ disable-over-current;
+};
+
&usdhc2 { /* SD card */
status = "okay";
};
diff --git a/arch/arm/boot/dts/imx6qdl-dhcom-som.dtsi b/arch/arm/boot/dts/imx6qdl-dhcom-som.dtsi
index 5d10c40..5befbe1 100644
--- a/arch/arm/boot/dts/imx6qdl-dhcom-som.dtsi
+++ b/arch/arm/boot/dts/imx6qdl-dhcom-som.dtsi
@@ -132,14 +132,15 @@ mdio {
#size-cells = <0>;
ethphy0: ethernet-phy@0 { /* SMSC LAN8710Ai */
- compatible = "ethernet-phy-ieee802.3-c22";
+ compatible = "ethernet-phy-id0007.c0f0",
+ "ethernet-phy-ieee802.3-c22";
interrupt-parent = <&gpio4>;
interrupts = <15 IRQ_TYPE_LEVEL_LOW>;
pinctrl-0 = <&pinctrl_ethphy0>;
pinctrl-names = "default";
reg = <0>;
- reset-assert-us = <1000>;
- reset-deassert-us = <1000>;
+ reset-assert-us = <500>;
+ reset-deassert-us = <500>;
reset-gpios = <&gpio5 0 GPIO_ACTIVE_LOW>;
smsc,disable-energy-detect; /* Make plugin detection reliable */
};
@@ -728,6 +729,7 @@ MX6QDL_PAD_CSI0_DAT19__UART5_CTS_B 0x4001b0b1
pinctrl_usbh1: usbh1-grp {
fsl,pins = <
MX6QDL_PAD_EIM_D31__GPIO3_IO31 0x120b0
+ MX6QDL_PAD_EIM_D30__USB_H1_OC 0x1b0b1
>;
};
diff --git a/arch/arm/boot/dts/imx6qdl-mba6.dtsi b/arch/arm/boot/dts/imx6qdl-mba6.dtsi
new file mode 100644
index 0000000..daf7634
--- /dev/null
+++ b/arch/arm/boot/dts/imx6qdl-mba6.dtsi
@@ -0,0 +1,526 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright 2013 Sascha Hauer, Pengutronix
+ *
+ * Copyright 2013-2021 TQ-Systems GmbH
+ * Author: Markus Niebel <Markus.Niebel@tq-group.com>
+ */
+
+#include <dt-bindings/clock/imx6qdl-clock.h>
+#include <dt-bindings/gpio/gpio.h>
+#include <dt-bindings/input/input.h>
+#include <dt-bindings/sound/fsl-imx-audmux.h>
+
+/ {
+ aliases {
+ mmc0 = &usdhc3;
+ mmc1 = &usdhc2;
+ /delete-property/ mmc2;
+ /delete-property/ mmc3;
+ };
+
+ chosen {
+ stdout-path = &uart2;
+ };
+
+ beeper: gpio-beeper {
+ compatible = "gpio-beeper";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_gpiobeeper>;
+ gpios = <&gpio1 21 GPIO_ACTIVE_HIGH>;
+ };
+
+ gpio_buttons: gpio-buttons {
+ compatible = "gpio-keys";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_gpiobuttons>;
+
+ button1 {
+ label = "s6";
+ linux,code = <KEY_F6>;
+ gpios = <&gpio7 13 GPIO_ACTIVE_LOW>;
+ };
+
+ button2 {
+ label = "s7";
+ linux,code = <KEY_F7>;
+ gpios = <&gpio7 12 GPIO_ACTIVE_LOW>;
+ };
+
+ button3 {
+ label = "s8";
+ linux,code = <KEY_F8>;
+ gpios = <&gpio1 8 GPIO_ACTIVE_LOW>;
+ };
+ };
+
+ gpio-leds {
+ compatible = "gpio-leds";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_gpioled>;
+
+ led1 {
+ label = "led1";
+ gpios = <&gpio6 16 GPIO_ACTIVE_HIGH>;
+ linux,default-trigger = "default-on";
+ };
+
+ led2 {
+ label = "led2";
+ gpios = <&gpio6 31 GPIO_ACTIVE_HIGH>;
+ linux,default-trigger = "heartbeat";
+ };
+ };
+
+ reg_mba6_3p3v: regulator-mba6-3p3v {
+ compatible = "regulator-fixed";
+ regulator-name = "supply-mba6-3p3v";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-always-on;
+ };
+
+ reg_pcie: regulator-pcie {
+ compatible = "regulator-fixed";
+ regulator-name = "supply-pcie";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ /* PCIE.PWR_EN */
+ gpio = <&gpio2 0 GPIO_ACTIVE_HIGH>;
+ enable-active-high;
+ regulator-always-on;
+ vin-supply = <®_mba6_3p3v>;
+ };
+
+ reg_vcc3v3_audio: regulator-vcc3v3-audio {
+ compatible = "regulator-fixed";
+ regulator-name = "vcc3v3-audio";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ vin-supply = <®_mba6_3p3v>;
+ };
+
+ sound {
+ compatible = "fsl,imx-audio-tlv320aic32x4";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_audmux>;
+ model = "imx-audio-tlv320aic32x4";
+ ssi-controller = <&ssi1>;
+ audio-codec = <&tlv320aic32x4>;
+ audio-asrc = <&asrc>;
+ audio-routing =
+ "IN3_L", "Mic Jack",
+ "Mic Jack", "Mic Bias",
+ "IN1_L", "Line In Jack",
+ "IN1_R", "Line In Jack",
+ "Line Out Jack", "LOL",
+ "Line Out Jack", "LOR";
+ mux-int-port = <1>;
+ mux-ext-port = <3>;
+ };
+};
+
+&audmux {
+ status = "okay";
+
+ ssi0 {
+ fsl,audmux-port = <MX31_AUDMUX_PORT1_SSI0>;
+ fsl,port-config = <
+ (IMX_AUDMUX_V2_PTCR_SYN |
+ IMX_AUDMUX_V2_PTCR_TFSDIR |
+ IMX_AUDMUX_V2_PTCR_TFSEL(MX31_AUDMUX_PORT3_SSI_PINS_3) |
+ IMX_AUDMUX_V2_PTCR_TCLKDIR |
+ IMX_AUDMUX_V2_PTCR_TCSEL(MX31_AUDMUX_PORT3_SSI_PINS_3))
+ IMX_AUDMUX_V2_PDCR_RXDSEL(MX31_AUDMUX_PORT3_SSI_PINS_3)
+ >;
+ };
+
+ aud3 {
+ fsl,audmux-port = <MX31_AUDMUX_PORT3_SSI_PINS_3>;
+ fsl,port-config = <
+ IMX_AUDMUX_V2_PTCR_SYN
+ IMX_AUDMUX_V2_PDCR_RXDSEL(MX31_AUDMUX_PORT1_SSI0)
+ >;
+ };
+};
+
+&can1 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_can1>;
+ status = "okay";
+};
+
+&can2 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_can2>;
+ status = "okay";
+};
+
+&ecspi1 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_ecspi1>, <&pinctrl_ecspi1_mba6>;
+ cs-gpios = <&gpio3 19 0>, <&gpio3 24 0>;
+};
+
+&fec {
+ phy-mode = "rgmii-id";
+ phy-handle = <ðphy>;
+ mac-address = [00 00 00 00 00 00];
+ status = "okay";
+
+ mdio {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ ethphy: ethernet-phy@3 {
+ compatible = "ethernet-phy-ieee802.3-c22";
+ reg = <3>;
+ interrupt-parent = <&gpio1>;
+ interrupts = <28 IRQ_TYPE_LEVEL_LOW>;
+ reset-gpios = <&gpio1 25 GPIO_ACTIVE_LOW>;
+ reset-assert-us = <1000>;
+ reset-deassert-us = <100000>;
+ micrel,force-master;
+ max-speed = <1000>;
+ };
+ };
+};
+
+&i2c1 {
+ tlv320aic32x4: audio-codec@18 {
+ compatible = "ti,tlv320aic32x4";
+ reg = <0x18>;
+ clocks = <&clks IMX6QDL_CLK_CKO>;
+ clock-names = "mclk";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_codec>;
+ ldoin-supply = <®_vcc3v3_audio>;
+ iov-supply = <®_mba6_3p3v>;
+ };
+};
+
+&pcie {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_pcie>;
+ reset-gpio = <&gpio6 7 GPIO_ACTIVE_LOW>;
+ status = "okay";
+};
+
+&pwm1 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_pwm1>;
+ status = "okay";
+};
+
+&pwm3 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_pwm3>;
+ status = "okay";
+};
+
+&pwm4 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_pwm4>;
+ status = "okay";
+};
+
+&snvs_poweroff {
+ status = "okay";
+};
+
+&ssi1 {
+ status = "okay";
+};
+
+&uart2 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_uart2>;
+ status = "okay";
+};
+
+
+&uart3 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_uart3>;
+ uart-has-rtscts;
+ status = "okay";
+};
+
+&uart4 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_uart4>;
+ uart-has-rtscts;
+ linux,rs485-enabled-at-boot-time;
+ rs485-rts-active-low;
+ rs485-rx-during-tx;
+ status = "okay";
+};
+
+&uart5 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_uart5>;
+ uart-has-rtscts;
+ status = "okay";
+};
+
+&usbh1 {
+ disable-over-current;
+ status = "okay";
+};
+
+&usbotg {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_usbotg>;
+ power-active-high;
+ over-current-active-low;
+ srp-disable;
+ hnp-disable;
+ adp-disable;
+ dr_mode = "otg";
+ status = "okay";
+};
+
+/* SD card slot */
+&usdhc2 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_usdhc2>;
+ vmmc-supply = <®_mba6_3p3v>;
+ bus-width = <4>;
+ no-1-8-v;
+ no-mmc;
+ no-sdio;
+ cd-gpios = <&gpio1 4 GPIO_ACTIVE_LOW>;
+ wp-gpios = <&gpio1 2 GPIO_ACTIVE_HIGH>;
+ status = "okay";
+};
+
+&wdog1 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_wdog1>;
+ /* does not work on unmodified starter kit */
+ /* fsl,ext-reset-output; */
+ status = "okay";
+};
+
+&iomuxc {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_hog>;
+
+ pinctrl_audmux: audmuxgrp {
+ fsl,pins = <
+ MX6QDL_PAD_CSI0_DAT7__AUD3_RXD 0x1b0b0
+ MX6QDL_PAD_CSI0_DAT4__AUD3_TXC 0x1b0b0
+ MX6QDL_PAD_CSI0_DAT5__AUD3_TXD 0x1b0b0
+ MX6QDL_PAD_CSI0_DAT6__AUD3_TXFS 0x1b0b0
+ >;
+ };
+
+ pinctrl_can1: can1grp {
+ fsl,pins = <
+ MX6QDL_PAD_KEY_COL2__FLEXCAN1_TX 0xb099
+ MX6QDL_PAD_KEY_ROW2__FLEXCAN1_RX 0xb099
+ >;
+ };
+
+ pinctrl_can2: can2grp {
+ fsl,pins = <
+ MX6QDL_PAD_KEY_COL4__FLEXCAN2_TX 0xb099
+ MX6QDL_PAD_KEY_ROW4__FLEXCAN2_RX 0xb099
+ >;
+ };
+
+ pinctrl_codec: codecgrp {
+ fsl,pins = <
+ MX6QDL_PAD_CSI0_MCLK__CCM_CLKO1 0xb0 /* CLK */
+ >;
+ };
+
+ pinctrl_ecspi1_mba6: ecspimba6grp {
+ fsl,pins = <
+ MX6QDL_PAD_EIM_D24__GPIO3_IO24 0xb099 /* eCSPI1 SS2 */
+ >;
+ };
+
+ pinctrl_enet: enetgrp {
+ fsl,pins = <
+ /* FEC phy IRQ */
+ MX6QDL_PAD_ENET_TX_EN__GPIO1_IO28 0x00011008
+ /* FEC phy reset */
+ MX6QDL_PAD_ENET_CRS_DV__GPIO1_IO25 0x1b099
+ /* DSE = 100, 100k up, SPEED = MED */
+ MX6QDL_PAD_ENET_MDIO__ENET_MDIO 0xb0a0
+ MX6QDL_PAD_ENET_MDC__ENET_MDC 0xb0a0
+ /* DSE = 111, pull 100k up */
+ MX6QDL_PAD_RGMII_TXC__RGMII_TXC 0xb038
+ MX6QDL_PAD_RGMII_TD0__RGMII_TD0 0xb038
+ MX6QDL_PAD_RGMII_TD1__RGMII_TD1 0xb038
+ MX6QDL_PAD_RGMII_TD2__RGMII_TD2 0xb038
+ MX6QDL_PAD_RGMII_TD3__RGMII_TD3 0xb038
+ MX6QDL_PAD_RGMII_TX_CTL__RGMII_TX_CTL 0xb038
+ /* DSE = 111, pull external */
+ MX6QDL_PAD_RGMII_RXC__RGMII_RXC 0x0038
+ MX6QDL_PAD_RGMII_RD0__RGMII_RD0 0x0038
+ MX6QDL_PAD_RGMII_RD1__RGMII_RD1 0x0038
+ MX6QDL_PAD_RGMII_RD2__RGMII_RD2 0x0038
+ MX6QDL_PAD_RGMII_RD3__RGMII_RD3 0x0038
+ MX6QDL_PAD_RGMII_RX_CTL__RGMII_RX_CTL 0x0038
+ /* HYS = 1, DSE = 111, 100k up, SPEED = HIGH */
+ MX6QDL_PAD_ENET_REF_CLK__ENET_TX_CLK 0x1b0f0
+ >;
+ };
+
+ pinctrl_gpiobeeper: gpiobeepergrp {
+ fsl,pins = <
+ MX6QDL_PAD_SD1_DAT3__GPIO1_IO21 0xb099
+ >;
+ };
+
+ pinctrl_gpiobuttons: gpiobuttongrp {
+ fsl,pins = <
+ MX6QDL_PAD_GPIO_17__GPIO7_IO12 0x0001b099
+ MX6QDL_PAD_GPIO_18__GPIO7_IO13 0x0001b099
+ MX6QDL_PAD_GPIO_8__GPIO1_IO08 0x0001b099
+ >;
+ };
+
+ pinctrl_gpioled: gpioledgrp {
+ fsl,pins = <
+ MX6QDL_PAD_NANDF_CS3__GPIO6_IO16 0xb099 /* LED V15 */
+ MX6QDL_PAD_EIM_BCLK__GPIO6_IO31 0xb099 /* LED V16 */
+ >;
+ };
+
+ pinctrl_hog: hoggrp {
+ fsl,pins = <
+ /* LCD.CONTRAST -> Rev 0100 only, not used on Rev.0200*/
+ MX6QDL_PAD_DI0_PIN4__GPIO4_IO20 0x0001b099
+
+ MX6QDL_PAD_ENET_RXD1__GPIO1_IO26 0x0001b099
+ MX6QDL_PAD_ENET_TXD1__GPIO1_IO29 0x0001b099
+ MX6QDL_PAD_ENET_TXD0__GPIO1_IO30 0x0001b099
+
+ MX6QDL_PAD_NANDF_D1__GPIO2_IO01 0x0001b099
+ MX6QDL_PAD_NANDF_D2__GPIO2_IO02 0x0001b099
+ MX6QDL_PAD_NANDF_D3__GPIO2_IO03 0x0001b099
+ MX6QDL_PAD_SD4_DAT0__GPIO2_IO08 0x0001b099
+ MX6QDL_PAD_EIM_CS0__GPIO2_IO23 0x0001b099
+ MX6QDL_PAD_EIM_CS1__GPIO2_IO24 0x0001b099
+ MX6QDL_PAD_EIM_OE__GPIO2_IO25 0x0001b099
+
+ MX6QDL_PAD_EIM_D20__GPIO3_IO20 0x0001b099
+ MX6QDL_PAD_EIM_D26__GPIO3_IO26 0x0001b099
+ MX6QDL_PAD_EIM_D27__GPIO3_IO27 0x0001b099
+ MX6QDL_PAD_EIM_D28__GPIO3_IO28 0x0001b099
+ MX6QDL_PAD_EIM_D29__GPIO3_IO29 0x0001b099
+
+ MX6QDL_PAD_KEY_COL0__GPIO4_IO06 0x0001b099
+ MX6QDL_PAD_KEY_ROW0__GPIO4_IO07 0x0001b099
+ MX6QDL_PAD_KEY_COL1__GPIO4_IO08 0x0001b099
+ MX6QDL_PAD_KEY_ROW1__GPIO4_IO09 0x0001b099
+
+ MX6QDL_PAD_CSI0_PIXCLK__GPIO5_IO18 0x0001b099
+ MX6QDL_PAD_CSI0_DATA_EN__GPIO5_IO20 0x0001b099
+ MX6QDL_PAD_CSI0_VSYNC__GPIO5_IO21 0x0001b099
+
+ MX6QDL_PAD_NANDF_ALE__GPIO6_IO08 0x0001b099
+ MX6QDL_PAD_NANDF_CS1__GPIO6_IO14 0x0001b099
+ >;
+ };
+
+ pinctrl_pcie: pciegrp {
+ fsl,pins = <
+ /* HYS = 1, DSE = 110, 100k up, SPEED = HIGH (11)*/
+ MX6QDL_PAD_SD4_DAT3__GPIO2_IO11 0x001b0f0 /* #PCIE.WAKE */
+ MX6QDL_PAD_NANDF_CLE__GPIO6_IO07 0x001b0f0 /* #PCIE.RST */
+ MX6QDL_PAD_NANDF_CS0__GPIO6_IO11 0x001b0f0 /* #PCIE.DIS */
+ /* HYS = 1, DSE = 110, PUE+PKE, SPEED = HIGH (11)*/
+ MX6QDL_PAD_NANDF_D0__GPIO2_IO00 0x00130f0 /* PCIE.PWR_EN */
+ >;
+ };
+
+ pinctrl_pwm1: pwm1grp {
+ fsl,pins = <
+ /* 100 k PD, DSE 120 OHM, SPPEED LO */
+ MX6QDL_PAD_GPIO_9__PWM1_OUT 0x00003050
+ >;
+ };
+
+ pinctrl_pwm3: pwm3grp {
+ fsl,pins = <
+ /* 100 k PD, DSE 120 OHM, SPPEED LO */
+ MX6QDL_PAD_SD4_DAT1__PWM3_OUT 0x00003050
+ >;
+ };
+
+ pinctrl_pwm4: pwm4grp {
+ fsl,pins = <
+ /* 100 k PD, DSE 120 OHM, SPPEED LO */
+ MX6QDL_PAD_SD4_DAT2__PWM4_OUT 0x00003050
+ >;
+ };
+
+ pinctrl_uart2: uart2grp {
+ fsl,pins = <
+ MX6QDL_PAD_SD4_DAT4__UART2_RX_DATA 0x1b099
+ MX6QDL_PAD_SD4_DAT7__UART2_TX_DATA 0x1b099
+ >;
+ };
+
+ pinctrl_uart3: uart3grp {
+ fsl,pins = <
+ MX6QDL_PAD_SD4_CLK__UART3_RX_DATA 0x1b0b1
+ MX6QDL_PAD_SD4_CMD__UART3_TX_DATA 0x1b0b1
+ MX6QDL_PAD_EIM_D30__UART3_CTS_B 0x1b0b1
+ MX6QDL_PAD_EIM_D31__UART3_RTS_B 0x1b0b1
+ >;
+ };
+
+ pinctrl_uart4: uart4grp {
+ fsl,pins = <
+ MX6QDL_PAD_CSI0_DAT12__UART4_TX_DATA 0x1b0b1
+ MX6QDL_PAD_CSI0_DAT13__UART4_RX_DATA 0x1b0b1
+ MX6QDL_PAD_CSI0_DAT16__UART4_RTS_B 0x1b0b1
+ MX6QDL_PAD_CSI0_DAT17__UART4_CTS_B 0x1b0b1
+ >;
+ };
+
+ pinctrl_uart5: uart5grp {
+ fsl,pins = <
+ MX6QDL_PAD_CSI0_DAT14__UART5_TX_DATA 0x1b0b1
+ MX6QDL_PAD_CSI0_DAT15__UART5_RX_DATA 0x1b0b1
+ MX6QDL_PAD_CSI0_DAT18__UART5_RTS_B 0x1b0b1
+ MX6QDL_PAD_CSI0_DAT19__UART5_CTS_B 0x1b0b1
+ >;
+ };
+
+ pinctrl_usdhc2: usdhc2grp {
+ fsl,pins = <
+ /* CLK: 47k Pup SPD_LOW DSE 40Ohm SRE_FAST HYS */
+ MX6QDL_PAD_SD2_CLK__SD2_CLK 0x00017071
+ /* SD2: 47k Pup SPD_LOW DSE 80Ohm SRE_FAST HYS */
+ MX6QDL_PAD_SD2_CMD__SD2_CMD 0x00017059
+ MX6QDL_PAD_SD2_DAT0__SD2_DATA0 0x00017059
+ MX6QDL_PAD_SD2_DAT1__SD2_DATA1 0x00017059
+ MX6QDL_PAD_SD2_DAT2__SD2_DATA2 0x00017059
+ MX6QDL_PAD_SD2_DAT3__SD2_DATA3 0x00017059
+
+ MX6QDL_PAD_GPIO_4__GPIO1_IO04 0x0001b099 /* usdhc2 CD */
+ MX6QDL_PAD_GPIO_2__GPIO1_IO02 0x0001b099 /* usdhc2 WP */
+ >;
+ };
+
+ pinctrl_usbotg: usbotggrp {
+ fsl,pins = <
+ MX6QDL_PAD_EIM_D21__USB_OTG_OC 0x0001b0b0
+ MX6QDL_PAD_GPIO_1__USB_OTG_ID 0x00017059
+ MX6QDL_PAD_EIM_D22__USB_OTG_PWR 0x0001b099
+ >;
+ };
+
+ pinctrl_wdog1: wdog1grp {
+ fsl,pins = <
+ /* Watchdog out */
+ MX6QDL_PAD_SD1_DAT2__WDOG1_B 0x0000b099
+ >;
+ };
+};
diff --git a/arch/arm/boot/dts/imx6qdl-mba6a.dtsi b/arch/arm/boot/dts/imx6qdl-mba6a.dtsi
new file mode 100644
index 0000000..a61f270
--- /dev/null
+++ b/arch/arm/boot/dts/imx6qdl-mba6a.dtsi
@@ -0,0 +1,36 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright 2013 Sascha Hauer, Pengutronix
+ *
+ * Copyright 2013-2021 TQ-Systems GmbH
+ * Author: Markus Niebel <Markus.Niebel@tq-group.com>
+ */
+
+/ {
+ aliases {
+ rtc0 = &rtc0;
+ };
+};
+
+&fec {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_enet>, <&pinctrl_enet_fix>;
+};
+
+&i2c1 {
+ lm75: temperature-sensor@49 {
+ compatible = "national,lm75";
+ reg = <0x49>;
+ };
+
+ m24c64_57: eeprom@57 {
+ compatible = "atmel,24c64";
+ reg = <0x57>;
+ pagesize = <32>;
+ };
+
+ rtc0: rtc@68 {
+ compatible = "dallas,ds1339";
+ reg = <0x68>;
+ };
+};
diff --git a/arch/arm/boot/dts/imx6qdl-mba6b.dtsi b/arch/arm/boot/dts/imx6qdl-mba6b.dtsi
new file mode 100644
index 0000000..9f9f703
--- /dev/null
+++ b/arch/arm/boot/dts/imx6qdl-mba6b.dtsi
@@ -0,0 +1,52 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright 2013 Sascha Hauer, Pengutronix
+ *
+ * Copyright 2013-2021 TQ-Systems GmbH
+ * Author: Markus Niebel <Markus.Niebel@tq-group.com>
+ */
+
+/ {
+ aliases {
+ rtc0 = &rtc0;
+ };
+};
+
+&fec {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_enet>;
+};
+
+&i2c1 {
+ clock-frequency = <100000>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_i2c1>;
+ status = "okay";
+};
+
+&i2c3 {
+ lm75: temperature-sensor@49 {
+ compatible = "national,lm75";
+ reg = <0x49>;
+ };
+
+ m24c64_57: eeprom@57 {
+ compatible = "atmel,24c64";
+ reg = <0x57>;
+ pagesize = <32>;
+ };
+
+ rtc0: rtc@68 {
+ compatible = "dallas,ds1339";
+ reg = <0x68>;
+ };
+};
+
+&iomuxc {
+ pinctrl_i2c1: i2c1grp {
+ fsl,pins = <
+ MX6QDL_PAD_CSI0_DAT8__I2C1_SDA 0x4001b899
+ MX6QDL_PAD_CSI0_DAT9__I2C1_SCL 0x4001b899
+ >;
+ };
+};
diff --git a/arch/arm/boot/dts/imx6qdl-phytec-mira-peb-av-02.dtsi b/arch/arm/boot/dts/imx6qdl-phytec-mira-peb-av-02.dtsi
new file mode 100644
index 0000000..393475c
--- /dev/null
+++ b/arch/arm/boot/dts/imx6qdl-phytec-mira-peb-av-02.dtsi
@@ -0,0 +1,119 @@
+// SPDX-License-Identifier: (GPL-2.0+ OR MIT)
+/*
+ * Copyright (C) 2018 PHYTEC Messtechnik
+ * Author: Christian Hemp <c.hemp@phytec.de>
+ */
+
+/ {
+ display: display0 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ compatible = "fsl,imx-parallel-display";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_disp0>;
+ interface-pix-fmt = "rgb24";
+ status = "disabled";
+
+ port@0 {
+ reg = <0>;
+
+ display0_in: endpoint {
+ remote-endpoint = <&ipu1_di0_disp0>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+
+ display0_out: endpoint {
+ remote-endpoint = <&peb_panel_lcd_in>;
+ };
+ };
+ };
+
+ panel-lcd {
+ compatible = "edt,etm0700g0edh6";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_disp0_pwr>;
+ power-supply = <®_display>;
+ enable-gpios = <&gpio3 22 GPIO_ACTIVE_HIGH>;
+ backlight = <&backlight>;
+ status = "disabled";
+
+ port {
+ peb_panel_lcd_in: endpoint {
+ remote-endpoint = <&display0_out>;
+ };
+ };
+ };
+
+ reg_display: regulator-peb-display {
+ compatible = "regulator-fixed";
+ regulator-name = "peb-display";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ };
+};
+
+&i2c1 {
+ edt_ft5x06: touchscreen@38 {
+ compatible = "edt,edt-ft5406";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_edt_ft5x06>;
+ reg = <0x38>;
+ interrupt-parent = <&gpio3>;
+ interrupts = <2 IRQ_TYPE_NONE>;
+ status = "disabled";
+ };
+};
+
+&ipu1_di0_disp0 {
+ remote-endpoint = <&display0_in>;
+};
+
+&iomuxc {
+ pinctrl_disp0: disp0grp {
+ fsl,pins = <
+ MX6QDL_PAD_DI0_DISP_CLK__IPU1_DI0_DISP_CLK 0x10
+ MX6QDL_PAD_DI0_PIN2__IPU1_DI0_PIN02 0x10
+ MX6QDL_PAD_DI0_PIN3__IPU1_DI0_PIN03 0x10
+ MX6QDL_PAD_DI0_PIN15__IPU1_DI0_PIN15 0x1b080
+ MX6QDL_PAD_DISP0_DAT0__IPU1_DISP0_DATA00 0x10
+ MX6QDL_PAD_DISP0_DAT1__IPU1_DISP0_DATA01 0x10
+ MX6QDL_PAD_DISP0_DAT2__IPU1_DISP0_DATA02 0x10
+ MX6QDL_PAD_DISP0_DAT3__IPU1_DISP0_DATA03 0x10
+ MX6QDL_PAD_DISP0_DAT4__IPU1_DISP0_DATA04 0x10
+ MX6QDL_PAD_DISP0_DAT5__IPU1_DISP0_DATA05 0x10
+ MX6QDL_PAD_DISP0_DAT6__IPU1_DISP0_DATA06 0x10
+ MX6QDL_PAD_DISP0_DAT7__IPU1_DISP0_DATA07 0x10
+ MX6QDL_PAD_DISP0_DAT8__IPU1_DISP0_DATA08 0x10
+ MX6QDL_PAD_DISP0_DAT9__IPU1_DISP0_DATA09 0x10
+ MX6QDL_PAD_DISP0_DAT10__IPU1_DISP0_DATA10 0x10
+ MX6QDL_PAD_DISP0_DAT11__IPU1_DISP0_DATA11 0x10
+ MX6QDL_PAD_DISP0_DAT12__IPU1_DISP0_DATA12 0x10
+ MX6QDL_PAD_DISP0_DAT13__IPU1_DISP0_DATA13 0x10
+ MX6QDL_PAD_DISP0_DAT14__IPU1_DISP0_DATA14 0x10
+ MX6QDL_PAD_DISP0_DAT15__IPU1_DISP0_DATA15 0x10
+ MX6QDL_PAD_DISP0_DAT16__IPU1_DISP0_DATA16 0x10
+ MX6QDL_PAD_DISP0_DAT17__IPU1_DISP0_DATA17 0x10
+ MX6QDL_PAD_DISP0_DAT18__IPU1_DISP0_DATA18 0x10
+ MX6QDL_PAD_DISP0_DAT19__IPU1_DISP0_DATA19 0x10
+ MX6QDL_PAD_DISP0_DAT20__IPU1_DISP0_DATA20 0x10
+ MX6QDL_PAD_DISP0_DAT21__IPU1_DISP0_DATA21 0x10
+ MX6QDL_PAD_DISP0_DAT22__IPU1_DISP0_DATA22 0x10
+ MX6QDL_PAD_DISP0_DAT23__IPU1_DISP0_DATA23 0x10
+ >;
+ };
+
+ pinctrl_disp0_pwr: disp0pwrgrp {
+ fsl,pins = <
+ MX6QDL_PAD_EIM_D22__GPIO3_IO22 0x1b0b0
+ >;
+ };
+
+ pinctrl_edt_ft5x06: edtft5x06grp {
+ fsl,pins = <
+ MX6QDL_PAD_EIM_DA2__GPIO3_IO02 0xb0b1
+ >;
+ };
+};
diff --git a/arch/arm/boot/dts/imx6qdl-phytec-mira-peb-eval-01.dtsi b/arch/arm/boot/dts/imx6qdl-phytec-mira-peb-eval-01.dtsi
new file mode 100644
index 0000000..037b601
--- /dev/null
+++ b/arch/arm/boot/dts/imx6qdl-phytec-mira-peb-eval-01.dtsi
@@ -0,0 +1,71 @@
+// SPDX-License-Identifier: (GPL-2.0+ OR MIT)
+/*
+ * Copyright (C) 2018 PHYTEC Messtechnik
+ * Author: Christian Hemp <c.hemp@phytec.de>
+ */
+
+#include <dt-bindings/input/input.h>
+
+/ {
+ gpio-keys {
+ compatible = "gpio-keys";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_gpio_keys>;
+ status = "disabled";
+
+ power {
+ label = "Power Button";
+ gpios = <&gpio5 28 GPIO_ACTIVE_LOW>;
+ linux,code = <KEY_WAKEUP>;
+ wakeup-source;
+ };
+
+ sleep {
+ label = "Sleep Button";
+ gpios = <&gpio6 18 GPIO_ACTIVE_LOW>;
+ linux,code = <KEY_SLEEP>;
+ };
+ };
+
+ user_leds: user-leds {
+ compatible = "gpio-leds";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_user_leds>;
+ status = "disabled";
+
+ user-led1 {
+ gpios = <&gpio7 1 GPIO_ACTIVE_HIGH>;
+ linux,default-trigger = "gpio";
+ default-state = "on";
+ };
+
+ user-led2 {
+ gpios = <&gpio7 0 GPIO_ACTIVE_HIGH>;
+ linux,default-trigger = "gpio";
+ default-state = "on";
+ };
+
+ user-led3 {
+ gpios = <&gpio5 29 GPIO_ACTIVE_HIGH>;
+ linux,default-trigger = "gpio";
+ default-state = "on";
+ };
+ };
+};
+
+&iomuxc {
+ pinctrl_gpio_keys: gpiokeysgrp {
+ fsl,pins = <
+ MX6QDL_PAD_SD3_DAT6__GPIO6_IO18 0x1b0b0
+ MX6QDL_PAD_CSI0_DAT10__GPIO5_IO28 0x1b0b0
+ >;
+ };
+
+ pinctrl_user_leds: userledsgrp {
+ fsl,pins = <
+ MX6QDL_PAD_SD3_DAT4__GPIO7_IO01 0x1b0b0
+ MX6QDL_PAD_SD3_DAT5__GPIO7_IO00 0x1b0b0
+ MX6QDL_PAD_CSI0_DAT11__GPIO5_IO29 0x1b0b0
+ >;
+ };
+};
diff --git a/arch/arm/boot/dts/imx6qdl-phytec-mira-peb-wlbt-05.dtsi b/arch/arm/boot/dts/imx6qdl-phytec-mira-peb-wlbt-05.dtsi
new file mode 100644
index 0000000..84f884d
--- /dev/null
+++ b/arch/arm/boot/dts/imx6qdl-phytec-mira-peb-wlbt-05.dtsi
@@ -0,0 +1,85 @@
+// SPDX-License-Identifier: (GPL-2.0+ OR MIT)
+/*
+ * Copyright (C) 2021 PHYTEC Messtechnik GmbH
+ * Author: Yunus Bas <y.bas@phytec.de>
+ */
+
+#include <dt-bindings/gpio/gpio.h>
+#include <dt-bindings/interrupt-controller/irq.h>
+
+/ {
+ reg_wl_en: regulator-wl-en {
+ compatible = "regulator-fixed";
+ regulator-name = "wlan_en";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_wl>;
+ gpio = <&gpio5 2 GPIO_ACTIVE_HIGH>;
+ enable-active-high;
+ startup-delay-us = <100>;
+ status = "disabled";
+ };
+};
+
+&uart3 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_uart3_bt>;
+ uart-has-rtscts;
+
+ bluetooth {
+ compatible = "brcm,bcm43438-bt";
+ shutdown-gpios = <&gpio7 0 GPIO_ACTIVE_HIGH>;
+ device-wakeup-gpios = <&gpio7 1 GPIO_ACTIVE_HIGH>;
+ host-wakeup-gpios = <&gpio5 26 GPIO_ACTIVE_HIGH>;
+ status = "disabled";
+ };
+};
+
+&usdhc3 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_usdhc3_wl>;
+ vmmc-supply = <®_wl_en>;
+ bus-width = <4>;
+ non-removable;
+ no-1-8-v;
+ status = "disabled";
+
+ brmcf: wifi@1 {
+ compatible = "brcm,bcm4329-fmac";
+ reg = <1>;
+ };
+};
+
+&iomuxc {
+ pinctrl_uart3_bt: uart3grp-bt {
+ fsl,pins = <
+ MX6QDL_PAD_EIM_D25__UART3_RX_DATA 0x1b0b1
+ MX6QDL_PAD_EIM_D24__UART3_TX_DATA 0x1b0b1
+ MX6QDL_PAD_EIM_EB3__UART3_RTS_B 0x1b0b1
+ MX6QDL_PAD_EIM_D23__UART3_CTS_B 0x1b0b1
+ MX6QDL_PAD_SD3_DAT5__GPIO7_IO00 0xb0b1 /* BT ENABLE */
+ MX6QDL_PAD_SD3_DAT4__GPIO7_IO01 0xb0b1 /* DEV WAKEUP */
+ MX6QDL_PAD_CSI0_DAT8__GPIO5_IO26 0xb0b1 /* HOST WAKEUP */
+ >;
+ };
+
+ pinctrl_usdhc3_wl: usdhc3grp-wl {
+ fsl,pins = <
+ MX6QDL_PAD_SD3_CMD__SD3_CMD 0x17059
+ MX6QDL_PAD_SD3_CLK__SD3_CLK 0x10059
+ MX6QDL_PAD_SD3_DAT0__SD3_DATA0 0x17059
+ MX6QDL_PAD_SD3_DAT1__SD3_DATA1 0x17059
+ MX6QDL_PAD_SD3_DAT2__SD3_DATA2 0x17059
+ MX6QDL_PAD_SD3_DAT3__SD3_DATA3 0x17059
+ >;
+ };
+
+ pinctrl_wl: wlgrp {
+ fsl,pins = <
+ MX6QDL_PAD_EIM_A25__GPIO5_IO02 0xb0b1 /* WLAN ENABLE */
+ >;
+ };
+};
diff --git a/arch/arm/boot/dts/imx6qdl-tqma6a.dtsi b/arch/arm/boot/dts/imx6qdl-tqma6a.dtsi
index b679bec..bfb67da 100644
--- a/arch/arm/boot/dts/imx6qdl-tqma6a.dtsi
+++ b/arch/arm/boot/dts/imx6qdl-tqma6a.dtsi
@@ -4,6 +4,12 @@
* Copyright 2013-2017 Markus Niebel <Markus.Niebel@tq-group.com>
*/
+&fec {
+ interrupts-extended = <&gpio1 6 IRQ_TYPE_LEVEL_HIGH>,
+ <&intc 0 119 IRQ_TYPE_LEVEL_HIGH>;
+ fsl,err006687-workaround-present;
+};
+
&i2c1 {
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_i2c1>;
@@ -26,3 +32,17 @@ eeprom@50 {
pagesize = <32>;
};
};
+
+&iomuxc {
+ /*
+ * This pinmuxing is required for the ERR006687 workaround. Board
+ * DTS files that enable the FEC controller with
+ * fsl,err006687-workaround-present must include this group.
+ */
+ pinctrl_enet_fix: enetfixgrp {
+ fsl,pins = <
+ /* ENET ping patch */
+ MX6QDL_PAD_GPIO_6__ENET_IRQ 0x000b1
+ >;
+ };
+};
diff --git a/arch/arm/boot/dts/imx6qdl-tx6.dtsi b/arch/arm/boot/dts/imx6qdl-tx6.dtsi
index 362e65c..bcc5bbc 100644
--- a/arch/arm/boot/dts/imx6qdl-tx6.dtsi
+++ b/arch/arm/boot/dts/imx6qdl-tx6.dtsi
@@ -259,18 +259,6 @@ &gpio2 30 GPIO_ACTIVE_HIGH
&gpio3 19 GPIO_ACTIVE_HIGH
>;
status = "disabled";
-
- spidev0: spi@0 {
- compatible = "spidev";
- reg = <0>;
- spi-max-frequency = <54000000>;
- };
-
- spidev1: spi@1 {
- compatible = "spidev";
- reg = <1>;
- spi-max-frequency = <54000000>;
- };
};
&fec {
diff --git a/arch/arm/boot/dts/imx6qdl.dtsi b/arch/arm/boot/dts/imx6qdl.dtsi
index f5de5de..d27beb4 100644
--- a/arch/arm/boot/dts/imx6qdl.dtsi
+++ b/arch/arm/boot/dts/imx6qdl.dtsi
@@ -55,19 +55,19 @@ aliases {
clocks {
ckil {
- compatible = "fsl,imx-ckil", "fixed-clock";
+ compatible = "fixed-clock";
#clock-cells = <0>;
clock-frequency = <32768>;
};
ckih1 {
- compatible = "fsl,imx-ckih1", "fixed-clock";
+ compatible = "fixed-clock";
#clock-cells = <0>;
clock-frequency = <0>;
};
osc {
- compatible = "fsl,imx-osc", "fixed-clock";
+ compatible = "fixed-clock";
#clock-cells = <0>;
clock-frequency = <24000000>;
};
@@ -481,7 +481,7 @@ asrc: asrc@2034000 {
status = "okay";
};
- spba@203c000 {
+ spba-bus@203c000 {
reg = <0x0203c000 0x4000>;
};
};
diff --git a/arch/arm/boot/dts/imx6qp-mba6b.dts b/arch/arm/boot/dts/imx6qp-mba6b.dts
new file mode 100644
index 0000000..eee2e09
--- /dev/null
+++ b/arch/arm/boot/dts/imx6qp-mba6b.dts
@@ -0,0 +1,18 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright 2015-2021 TQ-Systems GmbH
+ * Author: Markus Niebel <Markus.Niebel@tq-group.com>
+ */
+/dts-v1/;
+
+#include <dt-bindings/gpio/gpio.h>
+#include "imx6qp-tqma6b.dtsi"
+#include "imx6qdl-mba6.dtsi"
+#include "imx6qdl-mba6b.dtsi"
+#include "imx6q-mba6.dtsi"
+
+/ {
+ model = "TQ TQMa6QP on MBa6x";
+ compatible = "tq,imx6qp-mba6x-b", "tq,mba6b",
+ "tq,imx6qp-tqma6qp-b", "fsl,imx6qp";
+};
diff --git a/arch/arm/boot/dts/imx6qp-phytec-mira-rdk-nand.dts b/arch/arm/boot/dts/imx6qp-phytec-mira-rdk-nand.dts
index f27d7ab..a182665 100644
--- a/arch/arm/boot/dts/imx6qp-phytec-mira-rdk-nand.dts
+++ b/arch/arm/boot/dts/imx6qp-phytec-mira-rdk-nand.dts
@@ -8,6 +8,9 @@
#include "imx6qp.dtsi"
#include "imx6qdl-phytec-phycore-som.dtsi"
#include "imx6qdl-phytec-mira.dtsi"
+#include "imx6qdl-phytec-mira-peb-eval-01.dtsi"
+#include "imx6qdl-phytec-mira-peb-av-02.dtsi"
+#include "imx6qdl-phytec-mira-peb-wlbt-05.dtsi"
/ {
model = "PHYTEC phyBOARD-Mira QuadPlus Carrier-Board with NAND";
diff --git a/arch/arm/boot/dts/imx6qp-yapp4-crux-plus.dts b/arch/arm/boot/dts/imx6qp-yapp4-crux-plus.dts
new file mode 100644
index 0000000..cea165f
--- /dev/null
+++ b/arch/arm/boot/dts/imx6qp-yapp4-crux-plus.dts
@@ -0,0 +1,54 @@
+// SPDX-License-Identifier: GPL-2.0
+//
+// Copyright (C) 2021 Y Soft Corporation, a.s.
+
+/dts-v1/;
+
+#include "imx6qp.dtsi"
+#include "imx6dl-yapp4-common.dtsi"
+
+/ {
+ model = "Y Soft IOTA Crux+ i.MX6QuadPlus board";
+ compatible = "ysoft,imx6qp-yapp4-crux-plus", "fsl,imx6qp";
+
+ memory@10000000 {
+ device_type = "memory";
+ reg = <0x10000000 0xf0000000>;
+ };
+};
+
+&gpio_oled {
+ status = "okay";
+};
+
+&leds {
+ status = "okay";
+};
+
+&oled_1305 {
+ status = "okay";
+};
+
+&oled_1309 {
+ status = "okay";
+};
+
+®_usb_h1_vbus {
+ status = "okay";
+};
+
+&touchkeys {
+ status = "okay";
+};
+
+&uart2 {
+ status = "disabled";
+};
+
+&usbh1 {
+ status = "okay";
+};
+
+&usbphy2 {
+ status = "okay";
+};
diff --git a/arch/arm/boot/dts/imx6ul-imx6ull-opos6uldev.dtsi b/arch/arm/boot/dts/imx6ul-imx6ull-opos6uldev.dtsi
index 935a77d..18cac19 100644
--- a/arch/arm/boot/dts/imx6ul-imx6ull-opos6uldev.dtsi
+++ b/arch/arm/boot/dts/imx6ul-imx6ull-opos6uldev.dtsi
@@ -114,18 +114,6 @@ &ecspi4 {
pinctrl-0 = <&pinctrl_ecspi4>;
cs-gpios = <&gpio4 9 GPIO_ACTIVE_LOW>, <&gpio4 3 GPIO_ACTIVE_LOW>;
status = "okay";
-
- spidev0: spi@0 {
- compatible = "spidev";
- reg = <0>;
- spi-max-frequency = <5000000>;
- };
-
- spidev1: spi@1 {
- compatible = "spidev";
- reg = <1>;
- spi-max-frequency = <5000000>;
- };
};
&i2c1 {
diff --git a/arch/arm/boot/dts/imx6ul-phytec-segin-ff-rdk-nand.dts b/arch/arm/boot/dts/imx6ul-phytec-segin-ff-rdk-nand.dts
index bff98e6..607eddc 100644
--- a/arch/arm/boot/dts/imx6ul-phytec-segin-ff-rdk-nand.dts
+++ b/arch/arm/boot/dts/imx6ul-phytec-segin-ff-rdk-nand.dts
@@ -10,6 +10,7 @@
#include "imx6ul-phytec-segin.dtsi"
#include "imx6ul-phytec-segin-peb-eval-01.dtsi"
#include "imx6ul-phytec-segin-peb-av-02.dtsi"
+#include "imx6ul-phytec-segin-peb-wlbt-05.dtsi"
/ {
model = "PHYTEC phyBOARD-Segin i.MX6 UltraLite Full Featured with NAND";
diff --git a/arch/arm/boot/dts/imx6ul-phytec-segin-peb-wlbt-05.dtsi b/arch/arm/boot/dts/imx6ul-phytec-segin-peb-wlbt-05.dtsi
new file mode 100644
index 0000000..04477fd
--- /dev/null
+++ b/arch/arm/boot/dts/imx6ul-phytec-segin-peb-wlbt-05.dtsi
@@ -0,0 +1,90 @@
+// SPDX-License-Identifier: (GPL-2.0+ OR MIT)
+/*
+ * Copyright (C) 2021 PHYTEC Messtechnik GmbH
+ * Author: Yunus Bas <y.bas@phytec.de>
+ */
+
+#include <dt-bindings/gpio/gpio.h>
+#include <dt-bindings/interrupt-controller/irq.h>
+
+/ {
+ reg_wl_en: regulator-wl-en {
+ compatible = "regulator-fixed";
+ regulator-name = "wlan_en";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_wl>;
+ gpio = <&gpio5 9 GPIO_ACTIVE_HIGH>;
+ enable-active-high;
+ startup-delay-us = <100>;
+ status = "disabled";
+ };
+};
+
+&iomuxc {
+ pinctrl_bt: btgrp {
+ fsl,pins = <
+ MX6UL_PAD_GPIO1_IO01__GPIO1_IO01 0x3031 /* BT ENABLE */
+ MX6UL_PAD_GPIO1_IO03__GPIO1_IO03 0x3031 /* HOST WAKEUP */
+ MX6UL_PAD_JTAG_MOD__GPIO1_IO10 0x3031 /* DEV WAKEUP */
+ >;
+ };
+
+ pinctrl_uart2_bt: uart2grp-bt {
+ fsl,pins = <
+ MX6UL_PAD_UART2_TX_DATA__UART2_DCE_TX 0x17059
+ MX6UL_PAD_UART2_RX_DATA__UART2_DCE_RX 0x17059
+ MX6UL_PAD_UART2_CTS_B__UART2_DCE_CTS 0x17059
+ MX6UL_PAD_UART2_RTS_B__UART2_DCE_RTS 0x17059
+ >;
+ };
+
+ pinctrl_usdhc2_wl: usdhc2grp-wl {
+ fsl,pins = <
+ MX6UL_PAD_LCD_DATA18__USDHC2_CMD 0x10051
+ MX6UL_PAD_LCD_DATA19__USDHC2_CLK 0x10061
+ MX6UL_PAD_LCD_DATA20__USDHC2_DATA0 0x10051
+ MX6UL_PAD_LCD_DATA21__USDHC2_DATA1 0x10051
+ MX6UL_PAD_LCD_DATA22__USDHC2_DATA2 0x10051
+ MX6UL_PAD_LCD_DATA23__USDHC2_DATA3 0x10051
+ >;
+ };
+
+ pinctrl_wl: wlgrp {
+ fsl,pins = <
+ MX6UL_PAD_SNVS_TAMPER9__GPIO5_IO09 0x3031 /* WLAN ENABLE */
+ >;
+ };
+};
+
+&uart2 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_uart2_bt &pinctrl_bt>;
+ uart-has-rtscts;
+ status = "disabled";
+
+ bluetooth {
+ compatible = "brcm,bcm43438-bt";
+ shutdown-gpios = <&gpio1 1 GPIO_ACTIVE_HIGH>;
+ device-wakeup-gpios = <&gpio1 10 GPIO_ACTIVE_HIGH>;
+ host-wakeup-gpios = <&gpio1 3 GPIO_ACTIVE_HIGH>;
+ };
+};
+
+&usdhc2 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_usdhc2_wl>;
+ vmmc-supply = <®_wl_en>;
+ bus-width = <4>;
+ non-removable;
+ no-1-8-v;
+ status = "disabled";
+
+ brmcf: wifi@1 {
+ compatible = "brcm,bcm4329-fmac";
+ reg = <1>;
+ };
+};
diff --git a/arch/arm/boot/dts/imx6ul-prti6g.dts b/arch/arm/boot/dts/imx6ul-prti6g.dts
index d620157..c18390f 100644
--- a/arch/arm/boot/dts/imx6ul-prti6g.dts
+++ b/arch/arm/boot/dts/imx6ul-prti6g.dts
@@ -78,12 +78,6 @@ &ecspi2 {
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_ecspi2>;
status = "okay";
-
- spi@0 {
- compatible = "spidev";
- reg = <0>;
- spi-max-frequency = <1000000>;
- };
};
&fec1 {
diff --git a/arch/arm/boot/dts/imx6ul-tx6ul.dtsi b/arch/arm/boot/dts/imx6ul-tx6ul.dtsi
index 938a32c..c485d05 100644
--- a/arch/arm/boot/dts/imx6ul-tx6ul.dtsi
+++ b/arch/arm/boot/dts/imx6ul-tx6ul.dtsi
@@ -227,18 +227,6 @@ &gpio1 29 GPIO_ACTIVE_HIGH
&gpio1 10 GPIO_ACTIVE_HIGH
>;
status = "disabled";
-
- spi@0 {
- compatible = "spidev";
- reg = <0>;
- spi-max-frequency = <660000>;
- };
-
- spi@1 {
- compatible = "spidev";
- reg = <1>;
- spi-max-frequency = <660000>;
- };
};
sound {
@@ -290,18 +278,6 @@ &gpio1 29 GPIO_ACTIVE_HIGH
&gpio1 10 GPIO_ACTIVE_HIGH
>;
status = "disabled";
-
- spidev0: spi@0 {
- compatible = "spidev";
- reg = <0>;
- spi-max-frequency = <60000000>;
- };
-
- spidev1: spi@1 {
- compatible = "spidev";
- reg = <1>;
- spi-max-frequency = <60000000>;
- };
};
&fec1 {
diff --git a/arch/arm/boot/dts/imx6ull-jozacp.dts b/arch/arm/boot/dts/imx6ull-jozacp.dts
new file mode 100644
index 0000000..a152eeb
--- /dev/null
+++ b/arch/arm/boot/dts/imx6ull-jozacp.dts
@@ -0,0 +1,456 @@
+// SPDX-License-Identifier: GPL-2.0-or-later OR MIT
+/*
+ * Copyright (c) 2020 Protonic Holland
+ * Copyright (c) 2020 Oleksij Rempel <kernel@pengutronix.de>, Pengutronix
+ */
+
+/dts-v1/;
+
+#include <dt-bindings/gpio/gpio.h>
+#include <dt-bindings/leds/common.h>
+#include "imx6ull.dtsi"
+
+/ {
+ model = "JOZ Access Point";
+ compatible = "joz,jozacp", "fsl,imx6ull";
+
+ chosen {
+ stdout-path = &uart1;
+ };
+
+ /* On board name LED_RGB1 */
+ led-controller-1 {
+ compatible = "pwm-leds";
+
+ led-0 {
+ color = <LED_COLOR_ID_RED>;
+ function = LED_FUNCTION_INDICATOR;
+ function-enumerator = <0>;
+ pwms = <&pwm1 0 10000000 0>;
+ max-brightness = <255>;
+ };
+
+ led-1 {
+ color = <LED_COLOR_ID_GREEN>;
+ function = LED_FUNCTION_INDICATOR;
+ function-enumerator = <1>;
+ pwms = <&pwm3 0 10000000 0>;
+ max-brightness = <255>;
+ };
+
+ led-2 {
+ color = <LED_COLOR_ID_BLUE>;
+ function = LED_FUNCTION_INDICATOR;
+ function-enumerator = <2>;
+ pwms = <&pwm5 0 10000000 0>;
+ max-brightness = <255>;
+ linux,default-trigger = "heartbeat";
+ };
+ };
+
+ /* On board name LED_RGB2 */
+ led-controller-2 {
+ compatible = "pwm-leds";
+
+ led-3 {
+ color = <LED_COLOR_ID_RED>;
+ function = LED_FUNCTION_INDICATOR;
+ function-enumerator = <3>;
+ pwms = <&pwm2 0 10000000 0>;
+ max-brightness = <255>;
+ };
+
+ led-4 {
+ color = <LED_COLOR_ID_GREEN>;
+ function = LED_FUNCTION_INDICATOR;
+ function-enumerator = <4>;
+ pwms = <&pwm4 0 10000000 0>;
+ max-brightness = <255>;
+ };
+
+ led-5 {
+ color = <LED_COLOR_ID_BLUE>;
+ function = LED_FUNCTION_INDICATOR;
+ function-enumerator = <5>;
+ pwms = <&pwm6 0 10000000 0>;
+ max-brightness = <255>;
+ };
+ };
+
+ reg_3v3: regulator-3v3 {
+ compatible = "regulator-fixed";
+ regulator-name = "3v3";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ vin-supply = <®_5v0>;
+ };
+
+ reg_5v0: regulator-5v0 {
+ compatible = "regulator-fixed";
+ regulator-name = "5v0";
+ regulator-min-microvolt = <5000000>;
+ regulator-max-microvolt = <5000000>;
+ };
+
+ reg_vbus: regulator-vbus {
+ compatible = "regulator-fixed";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_vbus>;
+ regulator-name = "vbus";
+ regulator-min-microvolt = <5000000>;
+ regulator-max-microvolt = <5000000>;
+ vin-supply = <®_5v0>;
+ gpio = <&gpio1 2 GPIO_ACTIVE_HIGH>;
+ enable-active-high;
+ };
+
+ usdhc2_wifi_pwrseq: usdhc2-wifi-pwrseq {
+ compatible = "mmc-pwrseq-simple";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_wifi_npd>;
+ reset-gpios = <&gpio4 25 GPIO_ACTIVE_LOW>;
+ };
+};
+
+&can1 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_can1>;
+ status = "okay";
+};
+
+&cpu0 {
+ clock-frequency = <792000000>;
+};
+
+&fec1 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_enet1>;
+ phy-mode = "rmii";
+ phy-handle = <ðphy0>;
+ status = "okay";
+
+ mdio {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ ethphy0: ethernet-phy@0 {
+ reg = <0>;
+ clocks = <&clks IMX6UL_CLK_ENET_REF>;
+ clock-names = "rmii-ref";
+ interrupts-extended = <&gpio1 29 IRQ_TYPE_LEVEL_LOW>;
+ reset-gpios = <&gpio1 28 GPIO_ACTIVE_LOW>;
+ reset-assert-us = <10000>;
+ reset-deassert-us = <300>;
+ };
+ };
+};
+
+&i2c1 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_i2c1>;
+ clock-frequency = <100000>;
+ status = "okay";
+};
+
+&i2c2 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_i2c2>;
+ clock-frequency = <100000>;
+ status = "okay";
+
+ rtc@51 {
+ compatible = "nxp,pcf8563";
+ reg = <0x51>;
+ };
+};
+
+&pwm1 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_pwm1>;
+ status = "okay";
+};
+
+&pwm2 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_pwm2>;
+ status = "okay";
+};
+
+&pwm3 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_pwm3>;
+ status = "okay";
+};
+
+&pwm4 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_pwm4>;
+ status = "okay";
+};
+
+&pwm5 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_pwm5>;
+ status = "okay";
+};
+
+&pwm6 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_pwm6>;
+ status = "okay";
+};
+
+&snvs_rtc {
+ status = "disabled";
+};
+
+&uart1 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_uart1>;
+ status = "okay";
+};
+
+&uart4 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_uart4>;
+ dtr-gpios = <&gpio3 4 GPIO_ACTIVE_LOW>;
+ uart-has-rtscts;
+ status = "okay";
+};
+
+&usbotg1 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_usbotg1>;
+ vbus-supply = <®_vbus>;
+ dr_mode = "host";
+ over-current-active-low;
+ status = "okay";
+};
+
+&usdhc1 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_usdhc1>;
+ vmmc-supply = <®_3v3>;
+ bus-width = <8>;
+ no-1-8-v;
+ non-removable;
+ cap-mmc-hw-reset;
+ no-sd;
+ no-sdio;
+ status = "okay";
+};
+
+&usdhc2 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_usdhc2>;
+ mmc-pwrseq = <&usdhc2_wifi_pwrseq>;
+ bus-width = <4>;
+ no-1-8-v;
+ no-mmc;
+ no-sd;
+ non-removable;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "okay";
+
+ wifi@1 {
+ compatible = "brcm,bcm4329-fmac";
+ reg = <1>;
+ };
+};
+
+&iomuxc {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_hog>;
+
+ pinctrl_can1: can1grp {
+ fsl,pins = <
+ MX6UL_PAD_UART3_CTS_B__FLEXCAN1_TX 0x1b0b0
+ MX6UL_PAD_UART3_RTS_B__FLEXCAN1_RX 0x1b0b0
+ >;
+ };
+
+ pinctrl_enet1: enet1grp {
+ fsl,pins = <
+ MX6UL_PAD_GPIO1_IO07__ENET1_MDC 0x1b0b0
+ MX6UL_PAD_GPIO1_IO06__ENET1_MDIO 0x1b0b0
+ MX6UL_PAD_ENET1_RX_EN__ENET1_RX_EN 0x1b0b0
+ MX6UL_PAD_ENET1_RX_ER__ENET1_RX_ER 0x1b0b0
+ MX6UL_PAD_ENET1_RX_DATA0__ENET1_RDATA00 0x1b0b0
+ MX6UL_PAD_ENET1_RX_DATA1__ENET1_RDATA01 0x1b0b0
+ MX6UL_PAD_ENET1_TX_EN__ENET1_TX_EN 0x1b0b0
+ MX6UL_PAD_ENET1_TX_DATA0__ENET1_TDATA00 0x1b0b0
+ MX6UL_PAD_ENET1_TX_DATA1__ENET1_TDATA01 0x1b0b0
+ MX6UL_PAD_ENET1_TX_CLK__ENET1_REF_CLK1 0x4001b031
+
+ MX6UL_PAD_UART4_TX_DATA__GPIO1_IO28 0x038b0
+ MX6UL_PAD_UART4_RX_DATA__GPIO1_IO29 0x170b0
+ >;
+ };
+
+ pinctrl_hog: hoggrp {
+ fsl,pins = <
+ /* HW Revision */
+ MX6UL_PAD_ENET2_RX_DATA0__GPIO2_IO08 0x1b0b0
+ MX6UL_PAD_ENET2_RX_DATA1__GPIO2_IO09 0x1b0b0
+ MX6UL_PAD_ENET2_RX_EN__GPIO2_IO10 0x1b0b0
+
+ /* HW ID */
+ MX6UL_PAD_ENET2_TX_DATA0__GPIO2_IO11 0x1b0b0
+ MX6UL_PAD_ENET2_TX_DATA1__GPIO2_IO12 0x1b0b0
+ MX6UL_PAD_ENET2_TX_EN__GPIO2_IO13 0x1b0b0
+ MX6UL_PAD_ENET2_TX_CLK__GPIO2_IO14 0x1b0b0
+ MX6UL_PAD_ENET2_RX_ER__GPIO2_IO15 0x1b0b0
+
+ /* Digital inputs */
+ MX6UL_PAD_GPIO1_IO03__GPIO1_IO03 0x11000
+ MX6UL_PAD_GPIO1_IO04__GPIO1_IO04 0x11000
+ MX6UL_PAD_GPIO1_IO05__GPIO1_IO05 0x11000
+ MX6UL_PAD_GPIO1_IO08__GPIO1_IO08 0x11000
+ MX6UL_PAD_GPIO1_IO09__GPIO1_IO09 0x11000
+
+ /* Isolated outputs */
+ MX6UL_PAD_UART2_TX_DATA__GPIO1_IO20 0x01020
+ MX6UL_PAD_UART2_RX_DATA__GPIO1_IO21 0x01020
+ MX6UL_PAD_UART2_RTS_B__GPIO1_IO23 0x01020
+ MX6UL_PAD_UART3_TX_DATA__GPIO1_IO24 0x01020
+ MX6UL_PAD_UART3_RX_DATA__GPIO1_IO25 0x01020
+ >;
+ };
+
+ pinctrl_i2c1: i2c1grp {
+ fsl,pins = <
+ MX6UL_PAD_CSI_MCLK__I2C1_SDA 0x4001f8b1
+ MX6UL_PAD_CSI_PIXCLK__I2C1_SCL 0x4001f8b1
+ >;
+ };
+
+ pinctrl_i2c2: i2c2grp {
+ fsl,pins = <
+ MX6UL_PAD_UART5_RX_DATA__I2C2_SDA 0x4001f8b1
+ MX6UL_PAD_UART5_TX_DATA__I2C2_SCL 0x4001f8b1
+ >;
+ };
+
+ pinctrl_pwm1: pwm1grp {
+ fsl,pins = <
+ MX6UL_PAD_LCD_DATA00__PWM1_OUT 0x01010
+ >;
+ };
+
+ pinctrl_pwm2: pwm2grp {
+ fsl,pins = <
+ MX6UL_PAD_LCD_DATA01__PWM2_OUT 0x01010
+ >;
+ };
+
+ pinctrl_pwm3: pwm3grp {
+ fsl,pins = <
+ MX6UL_PAD_LCD_DATA02__PWM3_OUT 0x01010
+ >;
+ };
+
+ pinctrl_pwm4: pwm4grp {
+ fsl,pins = <
+ MX6UL_PAD_LCD_DATA03__PWM4_OUT 0x01010
+ >;
+ };
+
+ pinctrl_pwm5: pwm5grp {
+ fsl,pins = <
+ MX6UL_PAD_LCD_DATA18__PWM5_OUT 0x01010
+ >;
+ };
+
+ pinctrl_pwm6: pwm6grp {
+ fsl,pins = <
+ MX6UL_PAD_LCD_DATA19__PWM6_OUT 0x01010
+ >;
+ };
+
+ pinctrl_uart1: uart1grp {
+ fsl,pins = <
+ MX6UL_PAD_UART1_TX_DATA__UART1_DCE_TX 0x1b0b1
+ MX6UL_PAD_UART1_RX_DATA__UART1_DCE_RX 0x1b0b1
+ >;
+ };
+
+ pinctrl_uart4: uart4grp {
+ fsl,pins = <
+ MX6UL_PAD_LCD_CLK__UART4_DCE_TX 0x1b0b0
+ MX6UL_PAD_LCD_ENABLE__UART4_DCE_RX 0x1b0b0
+ MX6UL_PAD_LCD_HSYNC__UART4_DCE_CTS 0x1b0b0
+ MX6UL_PAD_LCD_VSYNC__UART4_DCE_RTS 0x1b0b0
+ MX6UL_PAD_LCD_RESET__GPIO3_IO04 0x1b0b0
+ >;
+ };
+
+ pinctrl_usbotg1: usbotg1grp {
+ fsl,pins = <
+ MX6UL_PAD_GPIO1_IO01__USB_OTG1_OC 0x1b0b0
+ >;
+ };
+
+ pinctrl_usdhc1: usdhc1grp {
+ fsl,pins = <
+ MX6UL_PAD_NAND_WP_B__USDHC1_RESET_B 0x17099
+ MX6UL_PAD_SD1_CMD__USDHC1_CMD 0x1f099
+ MX6UL_PAD_SD1_CLK__USDHC1_CLK 0x10099
+ MX6UL_PAD_SD1_DATA0__USDHC1_DATA0 0x17099
+ MX6UL_PAD_SD1_DATA1__USDHC1_DATA1 0x17099
+ MX6UL_PAD_SD1_DATA2__USDHC1_DATA2 0x17099
+ MX6UL_PAD_SD1_DATA3__USDHC1_DATA3 0x17099
+ MX6UL_PAD_NAND_READY_B__USDHC1_DATA4 0x17099
+ MX6UL_PAD_NAND_CE0_B__USDHC1_DATA5 0x17099
+ MX6UL_PAD_NAND_CE1_B__USDHC1_DATA6 0x17099
+ MX6UL_PAD_NAND_CLE__USDHC1_DATA7 0x17099
+ >;
+ };
+
+ pinctrl_usdhc2: usdhc2grp {
+ fsl,pins = <
+ MX6UL_PAD_CSI_VSYNC__USDHC2_CLK 0x100b9
+ MX6UL_PAD_CSI_HSYNC__USDHC2_CMD 0x170b9
+ MX6UL_PAD_CSI_DATA00__USDHC2_DATA0 0x170b9
+ MX6UL_PAD_CSI_DATA01__USDHC2_DATA1 0x170b9
+ MX6UL_PAD_CSI_DATA02__USDHC2_DATA2 0x170b9
+ MX6UL_PAD_CSI_DATA03__USDHC2_DATA3 0x170b9
+ >;
+ };
+
+ pinctrl_vbus: vbus0grp {
+ fsl,pins = <
+ MX6UL_PAD_GPIO1_IO02__GPIO1_IO02 0x030b0
+ >;
+ };
+
+ pinctrl_wifi_npd: wifigrp {
+ fsl,pins = <
+ /* WL_REG_ON */
+ MX6UL_PAD_CSI_DATA04__GPIO4_IO25 0x03020
+ >;
+ };
+};
+
+&iomuxc_snvs {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_snvs_hog>;
+
+ pinctrl_snvs_hog: snvs-hog-grp {
+ fsl,pins = <
+ /* Digital outputs */
+ MX6ULL_PAD_SNVS_TAMPER2__GPIO5_IO02 0x00020
+ MX6ULL_PAD_SNVS_TAMPER3__GPIO5_IO03 0x00020
+ MX6ULL_PAD_SNVS_TAMPER4__GPIO5_IO04 0x00020
+ MX6ULL_PAD_SNVS_TAMPER5__GPIO5_IO05 0x00020
+ MX6ULL_PAD_SNVS_TAMPER6__GPIO5_IO06 0x00020
+
+ /* Digital outputs fault feedback */
+ MX6ULL_PAD_SNVS_TAMPER0__GPIO5_IO00 0x17000
+ MX6ULL_PAD_SNVS_TAMPER1__GPIO5_IO01 0x17000
+ MX6ULL_PAD_SNVS_TAMPER7__GPIO5_IO07 0x17000
+ MX6ULL_PAD_SNVS_TAMPER8__GPIO5_IO08 0x17000
+ MX6ULL_PAD_SNVS_TAMPER9__GPIO5_IO09 0x17000
+ >;
+ };
+};
diff --git a/arch/arm/boot/dts/imx6ull-phytec-segin-ff-rdk-nand.dts b/arch/arm/boot/dts/imx6ull-phytec-segin-ff-rdk-nand.dts
index c8d3eff..1d7362b 100644
--- a/arch/arm/boot/dts/imx6ull-phytec-segin-ff-rdk-nand.dts
+++ b/arch/arm/boot/dts/imx6ull-phytec-segin-ff-rdk-nand.dts
@@ -10,6 +10,7 @@
#include "imx6ull-phytec-segin.dtsi"
#include "imx6ull-phytec-segin-peb-eval-01.dtsi"
#include "imx6ull-phytec-segin-peb-av-02.dtsi"
+#include "imx6ull-phytec-segin-peb-wlbt-05.dtsi"
/ {
model = "PHYTEC phyBOARD-Segin i.MX6 ULL Full Featured with NAND";
diff --git a/arch/arm/boot/dts/imx6ull-phytec-segin-lc-rdk-nand.dts b/arch/arm/boot/dts/imx6ull-phytec-segin-lc-rdk-nand.dts
index e168494..4bcbae0 100644
--- a/arch/arm/boot/dts/imx6ull-phytec-segin-lc-rdk-nand.dts
+++ b/arch/arm/boot/dts/imx6ull-phytec-segin-lc-rdk-nand.dts
@@ -9,6 +9,7 @@
#include "imx6ull-phytec-phycore-som.dtsi"
#include "imx6ull-phytec-segin.dtsi"
#include "imx6ull-phytec-segin-peb-eval-01.dtsi"
+#include "imx6ull-phytec-segin-peb-wlbt-05.dtsi"
/ {
model = "PHYTEC phyBOARD-Segin i.MX6 ULL Low Cost with NAND";
diff --git a/arch/arm/boot/dts/imx6ull-phytec-segin-peb-wlbt-05.dtsi b/arch/arm/boot/dts/imx6ull-phytec-segin-peb-wlbt-05.dtsi
new file mode 100644
index 0000000..df25814
--- /dev/null
+++ b/arch/arm/boot/dts/imx6ull-phytec-segin-peb-wlbt-05.dtsi
@@ -0,0 +1,19 @@
+// SPDX-License-Identifier: (GPL-2.0+ OR MIT)
+/*
+ * Copyright (C) 2021 PHYTEC Messtechnik GmbH
+ * Author: Yunus Bas <y.bas@phytec.de>
+ */
+
+#include "imx6ul-phytec-segin-peb-wlbt-05.dtsi"
+
+&iomuxc {
+ /delete-node/ wlgrp;
+};
+
+&iomuxc_snvs {
+ pinctrl_wl: wlgrp {
+ fsl,pins = <
+ MX6ULL_PAD_SNVS_TAMPER9__GPIO5_IO09 0x3031
+ >;
+ };
+};
diff --git a/arch/arm/boot/dts/imx6ulz-bsh-smm-m2.dts b/arch/arm/boot/dts/imx6ulz-bsh-smm-m2.dts
new file mode 100644
index 0000000..59bcfc9
--- /dev/null
+++ b/arch/arm/boot/dts/imx6ulz-bsh-smm-m2.dts
@@ -0,0 +1,146 @@
+// SPDX-License-Identifier: (GPL-2.0 OR MIT)
+/*
+ * Copyright (C) 2021 BSH Hausgeraete GmbH
+ */
+
+/dts-v1/;
+
+#include <dt-bindings/input/input.h>
+#include "imx6ulz.dtsi"
+
+/ {
+ model = "BSH SMM M2";
+ compatible = "bsh,imx6ulz-bsh-smm-m2", "fsl,imx6ull", "fsl,imx6ulz";
+
+ chosen {
+ stdout-path = &uart4;
+ };
+
+ usdhc2_pwrseq: usdhc2-pwrseq {
+ compatible = "mmc-pwrseq-simple";
+ reset-gpios = <&gpio2 21 GPIO_ACTIVE_LOW>;
+ };
+};
+
+&gpmi {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_gpmi_nand>;
+ nand-on-flash-bbt;
+ status = "okay";
+};
+
+&uart3 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_uart3>;
+ uart-has-rtscts;
+ status = "okay";
+
+ bluetooth {
+ compatible = "brcm,bcm4330-bt";
+ max-speed = <3000000>;
+ shutdown-gpios = <&gpio1 1 GPIO_ACTIVE_HIGH>;
+ device-wakeup-gpios = <&gpio2 17 GPIO_ACTIVE_HIGH>;
+ host-wakeup-gpios = <&gpio2 13 GPIO_ACTIVE_HIGH>;
+ };
+};
+
+&uart4 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_uart4>;
+ status = "okay";
+};
+
+&usbotg1 {
+ dr_mode = "peripheral";
+ srp-disable;
+ hnp-disable;
+ adp-disable;
+ status = "okay";
+};
+
+&usbphy1 {
+ fsl,tx-d-cal = <106>;
+};
+
+&usdhc2 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_wlan>;
+ bus-width = <4>;
+ no-1-8-v;
+ non-removable;
+ cap-power-off-card;
+ keep-power-in-suspend;
+ cap-sdio-irq;
+ mmc-pwrseq = <&usdhc2_pwrseq>;
+ status = "okay";
+
+ brcmf: wifi@1 {
+ reg = <1>;
+ compatible = "brcm,bcm4329-fmac";
+ interrupt-parent = <&gpio1>;
+ interrupts = <18 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "host-wake";
+ };
+};
+
+&wdog1 {
+ status = "okay";
+};
+
+&iomuxc {
+ pinctrl_gpmi_nand: gpmi-nand {
+ fsl,pins = <
+ MX6UL_PAD_NAND_CLE__RAWNAND_CLE 0xb0b1
+ MX6UL_PAD_NAND_ALE__RAWNAND_ALE 0xb0b1
+ MX6UL_PAD_NAND_WP_B__RAWNAND_WP_B 0xb0b1
+ MX6UL_PAD_NAND_READY_B__RAWNAND_READY_B 0xb000
+ MX6UL_PAD_NAND_CE0_B__RAWNAND_CE0_B 0xb0b1
+ MX6UL_PAD_NAND_RE_B__RAWNAND_RE_B 0xb0b1
+ MX6UL_PAD_NAND_WE_B__RAWNAND_WE_B 0xb0b1
+ MX6UL_PAD_NAND_DATA00__RAWNAND_DATA00 0xb0b1
+ MX6UL_PAD_NAND_DATA01__RAWNAND_DATA01 0xb0b1
+ MX6UL_PAD_NAND_DATA02__RAWNAND_DATA02 0xb0b1
+ MX6UL_PAD_NAND_DATA03__RAWNAND_DATA03 0xb0b1
+ MX6UL_PAD_NAND_DATA04__RAWNAND_DATA04 0xb0b1
+ MX6UL_PAD_NAND_DATA05__RAWNAND_DATA05 0xb0b1
+ MX6UL_PAD_NAND_DATA06__RAWNAND_DATA06 0xb0b1
+ MX6UL_PAD_NAND_DATA07__RAWNAND_DATA07 0xb0b1
+ >;
+ };
+
+ pinctrl_uart3: uart3grp {
+ fsl,pins = <
+ MX6UL_PAD_UART3_TX_DATA__UART3_DCE_TX 0x1b0b1
+ MX6UL_PAD_UART3_RX_DATA__UART3_DCE_RX 0x1b099
+ MX6UL_PAD_UART3_RTS_B__UART3_DCE_RTS 0x1b0b1
+ MX6UL_PAD_UART3_CTS_B__UART3_DCE_CTS 0x1b099
+ MX6UL_PAD_GPIO1_IO01__GPIO1_IO01 0x79 /* BT_REG_ON */
+ MX6UL_PAD_SD1_CLK__GPIO2_IO17 0x100b1 /* BT_DEV_WAKE out */
+ MX6UL_PAD_ENET2_TX_EN__GPIO2_IO13 0x1b0b0 /* BT_HOST_WAKE in */
+ >;
+ };
+
+ pinctrl_uart4: uart4grp {
+ fsl,pins = <
+ MX6UL_PAD_UART4_TX_DATA__UART4_DCE_TX 0x1b0b1
+ MX6UL_PAD_UART4_RX_DATA__UART4_DCE_RX 0x1b0b1
+ >;
+ };
+
+ pinctrl_wlan: wlangrp {
+ fsl,pins = <
+ MX6UL_PAD_CSI_HSYNC__USDHC2_CMD 0x17059
+ MX6UL_PAD_CSI_VSYNC__USDHC2_CLK 0x10059
+ MX6UL_PAD_CSI_DATA00__USDHC2_DATA0 0x17059
+ MX6UL_PAD_CSI_DATA01__USDHC2_DATA1 0x17059
+ MX6UL_PAD_CSI_DATA02__USDHC2_DATA2 0x17059
+ MX6UL_PAD_CSI_DATA03__USDHC2_DATA3 0x17059
+ MX6UL_PAD_SD1_DATA3__GPIO2_IO21 0x79 /* WL_REG_ON */
+ MX6UL_PAD_UART2_CTS_B__GPIO1_IO22 0x100b1 /* WL_DEV_WAKE - WiFi_GPIO_4 - WiFi FW UART */
+ MX6UL_PAD_UART1_CTS_B__GPIO1_IO18 0x1b0b1 /* WL_HOST_WAKE - WIFI_GPIO_0 - OOB IRQ */
+ MX6UL_PAD_ENET1_RX_EN__OSC32K_32K_OUT 0x4001b031 /* OSC 32Khz wifi clk in */
+ >;
+ };
+};
diff --git a/arch/arm/boot/dts/imx7d-remarkable2.dts b/arch/arm/boot/dts/imx7d-remarkable2.dts
index 89cbf13..a2a91bf 100644
--- a/arch/arm/boot/dts/imx7d-remarkable2.dts
+++ b/arch/arm/boot/dts/imx7d-remarkable2.dts
@@ -34,6 +34,19 @@ reg_brcm: regulator-brcm {
startup-delay-us = <150>;
};
+ reg_digitizer: regulator-digitizer {
+ compatible = "regulator-fixed";
+ regulator-name = "VDD_3V3_DIGITIZER";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ pinctrl-names = "default", "sleep";
+ pinctrl-0 = <&pinctrl_digitizer_reg>;
+ pinctrl-1 = <&pinctrl_digitizer_reg>;
+ gpio = <&gpio1 6 GPIO_ACTIVE_HIGH>;
+ enable-active-high;
+ startup-delay-us = <100000>; /* 100 ms */
+ };
+
wifi_pwrseq: wifi_pwrseq {
compatible = "mmc-pwrseq-simple";
pinctrl-names = "default";
@@ -51,6 +64,26 @@ &clks {
assigned-clock-rates = <0>, <32768>;
};
+&i2c1 {
+ clock-frequency = <400000>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_i2c1>;
+ status = "okay";
+
+ wacom_digitizer: digitizer@9 {
+ compatible = "hid-over-i2c";
+ reg = <0x09>;
+ hid-descr-addr = <0x01>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_wacom>;
+ interrupt-parent = <&gpio1>;
+ interrupts = <1 IRQ_TYPE_LEVEL_LOW>;
+ touchscreen-inverted-x;
+ touchscreen-inverted-y;
+ vdd-supply = <®_digitizer>;
+ };
+};
+
&snvs_pwrkey {
status = "okay";
};
@@ -117,6 +150,25 @@ &wdog1 {
fsl,ext-reset-output;
};
+&iomuxc_lpsr {
+ pinctrl_digitizer_reg: digitizerreggrp {
+ fsl,pins = <
+ /* DIGITIZER_PWR_EN */
+ MX7D_PAD_LPSR_GPIO1_IO06__GPIO1_IO6 0x14
+ >;
+ };
+
+ pinctrl_wacom: wacomgrp {
+ fsl,pins = <
+ /*MX7D_PAD_LPSR_GPIO1_IO05__GPIO1_IO5 0x00000014 FWE */
+ MX7D_PAD_LPSR_GPIO1_IO04__GPIO1_IO4 0x00000074 /* PDCTB */
+ MX7D_PAD_LPSR_GPIO1_IO01__GPIO1_IO1 0x00000034 /* WACOM INT */
+ /*MX7D_PAD_LPSR_GPIO1_IO06__GPIO1_IO6 0x00000014 WACOM PWR ENABLE */
+ /*MX7D_PAD_LPSR_GPIO1_IO00__GPIO1_IO0 0x00000074 WACOM RESET */
+ >;
+ };
+};
+
&iomuxc {
pinctrl_brcm_reg: brcmreggrp {
fsl,pins = <
@@ -125,6 +177,13 @@ MX7D_PAD_SAI1_TX_BCLK__GPIO6_IO13 0x14
>;
};
+ pinctrl_i2c1: i2c1grp {
+ fsl,pins = <
+ MX7D_PAD_I2C1_SDA__I2C1_SDA 0x4000007f
+ MX7D_PAD_I2C1_SCL__I2C1_SCL 0x4000007f
+ >;
+ };
+
pinctrl_uart1: uart1grp {
fsl,pins = <
MX7D_PAD_UART1_TX_DATA__UART1_DCE_TX 0x79
diff --git a/arch/arm/boot/dts/imx7s-warp.dts b/arch/arm/boot/dts/imx7s-warp.dts
index 569bbd8..4f1edef 100644
--- a/arch/arm/boot/dts/imx7s-warp.dts
+++ b/arch/arm/boot/dts/imx7s-warp.dts
@@ -248,17 +248,17 @@ mpl3115@60 {
&mipi_csi {
clock-frequency = <166000000>;
- fsl,csis-hs-settle = <3>;
status = "okay";
- port@0 {
- reg = <0>;
+ ports {
+ port@0 {
+ reg = <0>;
- mipi_from_sensor: endpoint {
- remote-endpoint = <&ov2680_to_mipi>;
- data-lanes = <1>;
+ mipi_from_sensor: endpoint {
+ remote-endpoint = <&ov2680_to_mipi>;
+ data-lanes = <1>;
+ };
};
-
};
};
diff --git a/arch/arm/boot/dts/imx7s.dtsi b/arch/arm/boot/dts/imx7s.dtsi
index 1843fc0..52a9aee 100644
--- a/arch/arm/boot/dts/imx7s.dtsi
+++ b/arch/arm/boot/dts/imx7s.dtsi
@@ -809,8 +809,6 @@ lcdif: lcdif@30730000 {
mipi_csi: mipi-csi@30750000 {
compatible = "fsl,imx7-mipi-csi2";
reg = <0x30750000 0x10000>;
- #address-cells = <1>;
- #size-cells = <0>;
interrupts = <GIC_SPI 25 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&clks IMX7D_IPG_ROOT_CLK>,
<&clks IMX7D_MIPI_CSI_ROOT_CLK>,
@@ -819,18 +817,22 @@ mipi_csi: mipi-csi@30750000 {
power-domains = <&pgc_mipi_phy>;
phy-supply = <®_1p0d>;
resets = <&src IMX7_RESET_MIPI_PHY_MRST>;
- reset-names = "mrst";
status = "disabled";
- port@0 {
- reg = <0>;
- };
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
- port@1 {
- reg = <1>;
+ port@0 {
+ reg = <0>;
+ };
- mipi_vc0_to_csi_mux: endpoint {
- remote-endpoint = <&csi_mux_from_mipi_vc0>;
+ port@1 {
+ reg = <1>;
+
+ mipi_vc0_to_csi_mux: endpoint {
+ remote-endpoint = <&csi_mux_from_mipi_vc0>;
+ };
};
};
};
diff --git a/arch/arm/boot/dts/intel-ixp42x-freecom-fsg-3.dts b/arch/arm/boot/dts/intel-ixp42x-freecom-fsg-3.dts
index 598586f..b740403 100644
--- a/arch/arm/boot/dts/intel-ixp42x-freecom-fsg-3.dts
+++ b/arch/arm/boot/dts/intel-ixp42x-freecom-fsg-3.dts
@@ -97,6 +97,65 @@ partitions {
fis-index-block = <0x1f>;
};
};
+
+ /* Small syscon with some LEDs at CS2 */
+ syscon@2,0 {
+ compatible = "freecom,fsg-cs2-system-controller", "syscon";
+ reg = <2 0x0 0x200>;
+ reg-io-width = <2>;
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges = <2 0x0 0x0 0x200>;
+
+ led@0,0 {
+ compatible = "register-bit-led";
+ reg = <0x00 0x02>;
+ mask = <0x01>;
+ label = "fsg:blue:wlan";
+ linux,default-trigger = "wlan";
+ default-state = "on";
+ };
+ led@0,1 {
+ compatible = "register-bit-led";
+ reg = <0x00 0x02>;
+ mask = <0x02>;
+ label = "fsg:blue:wan";
+ linux,default-trigger = "";
+ default-state = "on";
+ };
+ led@0,2 {
+ compatible = "register-bit-led";
+ reg = <0x00 0x02>;
+ mask = <0x04>;
+ label = "fsg:blue:sata";
+ linux,default-trigger = "";
+ default-state = "on";
+ };
+ led@0,3 {
+ compatible = "register-bit-led";
+ reg = <0x00 0x02>;
+ mask = <0x04>;
+ label = "fsg:blue:usb";
+ linux,default-trigger = "";
+ default-state = "on";
+ };
+ led@0,4 {
+ compatible = "register-bit-led";
+ reg = <0x00 0x02>;
+ mask = <0x08>;
+ label = "fsg:blue:sync";
+ linux,default-trigger = "";
+ default-state = "on";
+ };
+ led@0,5 {
+ compatible = "register-bit-led";
+ reg = <0x00 0x02>;
+ mask = <0x10>;
+ label = "fsg:blue:ring";
+ linux,default-trigger = "";
+ default-state = "on";
+ };
+ };
};
pci@c0000000 {
diff --git a/arch/arm/boot/dts/intel-ixp42x-gateway-7001.dts b/arch/arm/boot/dts/intel-ixp42x-gateway-7001.dts
new file mode 100644
index 0000000..a1c03c9
--- /dev/null
+++ b/arch/arm/boot/dts/intel-ixp42x-gateway-7001.dts
@@ -0,0 +1,111 @@
+// SPDX-License-Identifier: ISC
+/*
+ * Device Tree file for Gateway 7001 AP
+ * Derived from boardfiles written by Imre Kaloz
+ */
+
+/dts-v1/;
+
+#include "intel-ixp42x.dtsi"
+#include <dt-bindings/input/input.h>
+
+/ {
+ model = "Gateway 7001 AP";
+ compatible = "gateway,7001", "intel,ixp42x";
+ #address-cells = <1>;
+ #size-cells = <1>;
+
+ memory@0 {
+ /* 32 MB SDRAM */
+ device_type = "memory";
+ reg = <0x00000000 0x2000000>;
+ };
+
+ chosen {
+ bootargs = "console=ttyS0,115200n8";
+ stdout-path = "uart1:115200n8";
+ };
+
+ aliases {
+ /* second UART is the primary console */
+ serial0 = &uart1;
+ serial1 = &uart0;
+ };
+
+ soc {
+ bus@c4000000 {
+ flash@0,0 {
+ compatible = "intel,ixp4xx-flash", "cfi-flash";
+ bank-width = <2>;
+ /*
+ * 8 MB of flash
+ */
+ reg = <0 0x00000000 0x800000>;
+
+ /* Configure expansion bus to allow writes */
+ intel,ixp4xx-eb-write-enable = <1>;
+
+ partitions {
+ compatible = "redboot-fis";
+ /* Eraseblock at 0x7e0000 */
+ fis-index-block = <0x3f>;
+ };
+ };
+ };
+
+ pci@c0000000 {
+ status = "ok";
+
+ /*
+ * Taken from Gateway 7001 PCI boardfile (gateway7001-pci.c)
+ * We have slots (IDSEL) 1 and 2 with one assigned IRQ
+ * each handling all IRQs.
+ */
+ interrupt-map =
+ /* IDSEL 1 */
+ <0x0800 0 0 1 &gpio0 11 IRQ_TYPE_LEVEL_LOW>, /* INT A on slot 1 is irq 11 */
+ <0x0800 0 0 2 &gpio0 11 IRQ_TYPE_LEVEL_LOW>, /* INT B on slot 1 is irq 11 */
+ <0x0800 0 0 3 &gpio0 11 IRQ_TYPE_LEVEL_LOW>, /* INT C on slot 1 is irq 11 */
+ <0x0800 0 0 4 &gpio0 11 IRQ_TYPE_LEVEL_LOW>, /* INT D on slot 1 is irq 11 */
+ /* IDSEL 2 */
+ <0x1000 0 0 1 &gpio0 10 IRQ_TYPE_LEVEL_LOW>, /* INT A on slot 2 is irq 10 */
+ <0x1000 0 0 2 &gpio0 10 IRQ_TYPE_LEVEL_LOW>, /* INT B on slot 2 is irq 10 */
+ <0x1000 0 0 3 &gpio0 10 IRQ_TYPE_LEVEL_LOW>, /* INT C on slot 2 is irq 10 */
+ <0x1000 0 0 4 &gpio0 10 IRQ_TYPE_LEVEL_LOW>; /* INT D on slot 2 is irq 10 */
+ };
+
+ ethernet@c8009000 {
+ status = "ok";
+ queue-rx = <&qmgr 3>;
+ queue-txready = <&qmgr 20>;
+ phy-mode = "rgmii";
+ phy-handle = <&phy1>;
+
+ mdio {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ phy1: ethernet-phy@1 {
+ reg = <1>;
+ };
+ };
+ };
+
+ ethernet@c800a000 {
+ status = "ok";
+ queue-rx = <&qmgr 4>;
+ queue-txready = <&qmgr 21>;
+ phy-mode = "rgmii";
+ phy-handle = <&phy2>;
+
+ mdio {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ phy2: ethernet-phy@2 {
+ reg = <2>;
+ };
+ };
+ };
+ };
+};
diff --git a/arch/arm/boot/dts/intel-ixp42x-goramo-multilink.dts b/arch/arm/boot/dts/intel-ixp42x-goramo-multilink.dts
new file mode 100644
index 0000000..f80388b
--- /dev/null
+++ b/arch/arm/boot/dts/intel-ixp42x-goramo-multilink.dts
@@ -0,0 +1,180 @@
+// SPDX-License-Identifier: ISC
+/*
+ * Device Tree file for the Goramo MultiLink Router
+ * There are two variants:
+ * - MultiLink Basic (a box)
+ * - MultiLink Max (19" rack mount)
+ * This device tree supports MultiLink Basic.
+ * This machine is based on IXP425.
+ * This is one of the few devices supporting the IXP4xx High-Speed Serial
+ * (HSS) link for a V.35 WAN interface.
+ * The hardware originates in Poland.
+ */
+
+/dts-v1/;
+
+#include "intel-ixp42x.dtsi"
+#include <dt-bindings/input/input.h>
+
+/ {
+ model = "Goramo MultiLink Router";
+ compatible = "goramo,multilink-router", "intel,ixp42x";
+ #address-cells = <1>;
+ #size-cells = <1>;
+
+ memory@0 {
+ /*
+ * 64 MB of RAM according to the manual. The MultiLink
+ * Max has 128 MB.
+ */
+ device_type = "memory";
+ reg = <0x00000000 0x4000000>;
+ };
+
+ chosen {
+ bootargs = "console=ttyS0,115200n8";
+ stdout-path = "uart0:115200n8";
+ };
+
+ aliases {
+ serial0 = &uart0;
+ serial1 = &uart1;
+ };
+
+ /*
+ * 74HC4094 which is used as a rudimentary GPIO expander
+ * FIXME:
+ * - Create device tree bindings for this as GPIO expander
+ * - Write a pure DT GPIO driver using these bindings
+ * - Support cascading in the style of gpio-74x164.c (cannot be reused, very different)
+ */
+ gpio_74: gpio-74hc4094 {
+ compatible = "nxp,74hc4094";
+ cp-gpios = <&gpio0 0 GPIO_ACTIVE_HIGH>;
+ d-gpios = <&gpio0 1 GPIO_ACTIVE_HIGH>;
+ str-gpios = <&gpio0 2 GPIO_ACTIVE_HIGH>;
+ /* oe-gpios is optional */
+ gpio-controller;
+ #gpio-cells = <2>;
+ /* We are not cascaded */
+ registers-number = <1>;
+ gpio-line-names = "CONTROL_HSS0_CLK_INT", "CONTROL_HSS1_CLK_INT", "CONTROL_HSS0_DTR_N",
+ "CONTROL_HSS1_DTR_N", "CONTROL_EXT", "CONTROL_AUTO_RESET",
+ "CONTROL_PCI_RESET_N", "CONTROL_EEPROM_WC_N";
+ };
+
+ soc {
+ bus@c4000000 {
+ flash@0,0 {
+ compatible = "intel,ixp4xx-flash", "cfi-flash";
+ bank-width = <2>;
+ /* Enable writes on the expansion bus */
+ intel,ixp4xx-eb-write-enable = <1>;
+ /* 16 MB of Flash mapped in at CS0 */
+ reg = <0 0x00000000 0x1000000>;
+
+ partitions {
+ compatible = "redboot-fis";
+ /* Eraseblock at 0x0fe0000 */
+ fis-index-block = <0x7f>;
+ };
+ };
+ };
+
+ pci@c0000000 {
+ status = "ok";
+
+ /*
+ * The device has 4 slots (IDSEL) with one dedicated IRQ per slot.
+ * The slots have Ethernet, Ethernet, NEC and MPCI.
+ * The IDSELs are 11, 12, 13, 14.
+ */
+ interrupt-map =
+ /* IDSEL 11 - Ethernet A */
+ <0x5800 0 0 1 &gpio0 4 IRQ_TYPE_LEVEL_LOW>, /* INT A on slot 11 is irq 4 */
+ <0x5800 0 0 2 &gpio0 4 IRQ_TYPE_LEVEL_LOW>, /* INT B on slot 11 is irq 4 */
+ <0x5800 0 0 3 &gpio0 4 IRQ_TYPE_LEVEL_LOW>, /* INT C on slot 11 is irq 4 */
+ <0x5800 0 0 4 &gpio0 4 IRQ_TYPE_LEVEL_LOW>, /* INT D on slot 11 is irq 4 */
+ /* IDSEL 12 - Ethernet B */
+ <0x6000 0 0 1 &gpio0 5 IRQ_TYPE_LEVEL_LOW>, /* INT A on slot 12 is irq 5 */
+ <0x6000 0 0 2 &gpio0 5 IRQ_TYPE_LEVEL_LOW>, /* INT B on slot 12 is irq 5 */
+ <0x6000 0 0 3 &gpio0 5 IRQ_TYPE_LEVEL_LOW>, /* INT C on slot 12 is irq 5 */
+ <0x6000 0 0 4 &gpio0 5 IRQ_TYPE_LEVEL_LOW>, /* INT D on slot 12 is irq 5 */
+ /* IDSEL 13 - MPCI */
+ <0x6800 0 0 1 &gpio0 12 IRQ_TYPE_LEVEL_LOW>, /* INT A on slot 13 is irq 12 */
+ <0x6800 0 0 2 &gpio0 12 IRQ_TYPE_LEVEL_LOW>, /* INT B on slot 13 is irq 12 */
+ <0x6800 0 0 3 &gpio0 12 IRQ_TYPE_LEVEL_LOW>, /* INT C on slot 13 is irq 12 */
+ <0x6800 0 0 4 &gpio0 12 IRQ_TYPE_LEVEL_LOW>, /* INT D on slot 13 is irq 12 */
+ /* IDSEL 14 - NEC */
+ <0x7000 0 0 1 &gpio0 3 IRQ_TYPE_LEVEL_LOW>, /* INT A on slot 14 is irq 3 */
+ <0x7000 0 0 2 &gpio0 3 IRQ_TYPE_LEVEL_LOW>, /* INT B on slot 14 is irq 3 */
+ <0x7000 0 0 3 &gpio0 3 IRQ_TYPE_LEVEL_LOW>, /* INT C on slot 14 is irq 3 */
+ <0x7000 0 0 4 &gpio0 3 IRQ_TYPE_LEVEL_LOW>; /* INT D on slot 14 is irq 3 */
+ };
+
+ /* HSS links */
+ npe@c8006000 {
+ hss@0 {
+ status = "okay";
+ intel,queue-chl-rxtrig = <&qmgr 12>;
+ intel,queue-chl-txready = <&qmgr 34>;
+ intel,queue-pkt-rx = <&qmgr 13>;
+ intel,queue-pkt-tx = <&qmgr 14>, <&qmgr 15>, <&qmgr 16>, <&qmgr 17>;
+ intel,queue-pkt-rxfree = <&qmgr 18>, <&qmgr 19>, <&qmgr 20>, <&qmgr 21>;
+ intel,queue-pkt-txdone = <&qmgr 22>;
+ /* The Goramo GPIO-based clock etc control */
+ cts-gpios = <&gpio0 10 GPIO_ACTIVE_LOW>;
+ rts-gpios = <&gpio0 14 GPIO_ACTIVE_LOW>;
+ dcd-gpios = <&gpio0 6 GPIO_ACTIVE_LOW>;
+ dtr-gpios = <&gpio_74 2 GPIO_ACTIVE_LOW>;
+ clk-internal-gpios = <&gpio_74 0 GPIO_ACTIVE_HIGH>;
+ };
+ hss@1 {
+ status = "okay";
+ intel,queue-chl-rxtrig = <&qmgr 10>;
+ intel,queue-chl-txready = <&qmgr 35>;
+ intel,queue-pkt-rx = <&qmgr 0>;
+ intel,queue-pkt-tx = <&qmgr 5>, <&qmgr 6>, <&qmgr 7>, <&qmgr 8>;
+ intel,queue-pkt-rxfree = <&qmgr 1>, <&qmgr 2>, <&qmgr 3>, <&qmgr 4>;
+ intel,queue-pkt-txdone = <&qmgr 9>;
+ /* The Goramo GPIO-based clock etc control */
+ cts-gpios = <&gpio0 11 GPIO_ACTIVE_LOW>;
+ rts-gpios = <&gpio0 13 GPIO_ACTIVE_LOW>;
+ dcd-gpios = <&gpio0 7 GPIO_ACTIVE_LOW>;
+ dtr-gpios = <&gpio_74 3 GPIO_ACTIVE_LOW>;
+ clk-internal-gpios = <&gpio_74 1 GPIO_ACTIVE_HIGH>;
+ };
+ };
+
+ /* EthB */
+ ethernet@c8009000 {
+ status = "ok";
+ queue-rx = <&qmgr 3>;
+ queue-txready = <&qmgr 32>;
+ phy-mode = "rgmii";
+ phy-handle = <&phy0>;
+
+ mdio {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ phy0: ethernet-phy@0 {
+ reg = <0>;
+ };
+
+ phy1: ethernet-phy@1 {
+ reg = <1>;
+ };
+ };
+ };
+
+ /* EthC */
+ ethernet@c800a000 {
+ status = "ok";
+ queue-rx = <&qmgr 4>;
+ queue-txready = <&qmgr 33>;
+ phy-mode = "rgmii";
+ phy-handle = <&phy1>;
+ };
+ };
+};
diff --git a/arch/arm/boot/dts/intel-ixp4xx.dtsi b/arch/arm/boot/dts/intel-ixp4xx.dtsi
index 46fede0..51a716c 100644
--- a/arch/arm/boot/dts/intel-ixp4xx.dtsi
+++ b/arch/arm/boot/dts/intel-ixp4xx.dtsi
@@ -139,6 +139,23 @@ timer@c8005000 {
npe: npe@c8006000 {
compatible = "intel,ixp4xx-network-processing-engine";
reg = <0xc8006000 0x1000>, <0xc8007000 0x1000>, <0xc8008000 0x1000>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ /* NPE-A contains two high-speed serial links */
+ hss@0 {
+ compatible = "intel,ixp4xx-hss";
+ reg = <0>;
+ intel,npe-handle = <&npe 0>;
+ status = "disabled";
+ };
+
+ hss@1 {
+ compatible = "intel,ixp4xx-hss";
+ reg = <1>;
+ intel,npe-handle = <&npe 0>;
+ status = "disabled";
+ };
/* NPE-C contains a crypto accelerator */
crypto {
diff --git a/arch/arm/boot/dts/milbeaut-m10v.dtsi b/arch/arm/boot/dts/milbeaut-m10v.dtsi
index aa7c6ca..75f0c0a 100644
--- a/arch/arm/boot/dts/milbeaut-m10v.dtsi
+++ b/arch/arm/boot/dts/milbeaut-m10v.dtsi
@@ -65,10 +65,18 @@ gic: interrupt-controller@1d000000 {
<0x1d002000 0x1000>; /* CPU I/f base and size */
};
+ clk: clock-ctrl@1d021000 {
+ compatible = "socionext,milbeaut-m10v-ccu";
+ #clock-cells = <1>;
+ reg = <0x1d021000 0x1000>;
+ clocks = <&uclk40xi>;
+ };
+
timer@1e000050 { /* 32-bit Reload Timers */
compatible = "socionext,milbeaut-timer";
reg = <0x1e000050 0x20>;
interrupts = <0 91 4>;
+ clocks = <&clk 4>;
};
uart1: serial@1e700010 { /* PE4, PE5 */
@@ -77,6 +85,7 @@ uart1: serial@1e700010 { /* PE4, PE5 */
reg = <0x1e700010 0x10>;
interrupts = <0 141 0x4>, <0 149 0x4>;
interrupt-names = "rx", "tx";
+ clocks = <&clk 2>;
};
};
diff --git a/arch/arm/boot/dts/motorola-mapphone-common.dtsi b/arch/arm/boot/dts/motorola-mapphone-common.dtsi
index a4423ff..c7a1f3f 100644
--- a/arch/arm/boot/dts/motorola-mapphone-common.dtsi
+++ b/arch/arm/boot/dts/motorola-mapphone-common.dtsi
@@ -310,7 +310,7 @@ &mmc3 {
#address-cells = <1>;
#size-cells = <0>;
wlcore: wlcore@2 {
- compatible = "ti,wl1285", "ti,wl1283";
+ compatible = "ti,wl1285";
reg = <2>;
/* gpio_100 with gpmc_wait2 pad as wakeirq */
interrupts-extended = <&gpio4 4 IRQ_TYPE_LEVEL_HIGH>,
diff --git a/arch/arm/boot/dts/mt6589-fairphone-fp1.dts b/arch/arm/boot/dts/mt6589-fairphone-fp1.dts
new file mode 100644
index 0000000..c952347
--- /dev/null
+++ b/arch/arm/boot/dts/mt6589-fairphone-fp1.dts
@@ -0,0 +1,30 @@
+// SPDX-License-Identifier: BSD-3-Clause
+/*
+ * Copyright (c) 2021, Luca Weiss <luca@z3ntu.xyz>
+ */
+
+/dts-v1/;
+#include "mt6589.dtsi"
+
+/ {
+ model = "Fairphone 1";
+ compatible = "fairphone,fp1", "mediatek,mt6589";
+
+ chosen {
+ stdout-path = &uart3;
+ };
+
+ memory@80000000 {
+ device_type = "memory";
+ reg = <0x80000000 0x40000000>;
+ };
+};
+
+&cpus {
+ /* SMP is not stable on this board, makes the kernel panic */
+ /delete-property/ enable-method;
+};
+
+&uart3 {
+ status = "okay";
+};
diff --git a/arch/arm/boot/dts/mt6589.dtsi b/arch/arm/boot/dts/mt6589.dtsi
index 70df00a..c6babc8 100644
--- a/arch/arm/boot/dts/mt6589.dtsi
+++ b/arch/arm/boot/dts/mt6589.dtsi
@@ -14,7 +14,7 @@ / {
compatible = "mediatek,mt6589";
interrupt-parent = <&sysirq>;
- cpus {
+ cpus: cpus {
#address-cells = <1>;
#size-cells = <0>;
enable-method = "mediatek,mt6589-smp";
diff --git a/arch/arm/boot/dts/omap3-tao3530.dtsi b/arch/arm/boot/dts/omap3-tao3530.dtsi
index 580bfa1..7f440d1 100644
--- a/arch/arm/boot/dts/omap3-tao3530.dtsi
+++ b/arch/arm/boot/dts/omap3-tao3530.dtsi
@@ -197,25 +197,11 @@ &i2c3 {
&mcspi1 {
pinctrl-names = "default";
pinctrl-0 = <&mcspi1_pins>;
-
- spidev@0 {
- compatible = "spidev";
- spi-max-frequency = <48000000>;
- reg = <0>;
- spi-cpha;
- };
};
&mcspi3 {
pinctrl-names = "default";
pinctrl-0 = <&mcspi3_pins>;
-
- spidev@0 {
- compatible = "spidev";
- spi-max-frequency = <48000000>;
- reg = <0>;
- spi-cpha;
- };
};
#include "twl4030.dtsi"
diff --git a/arch/arm/boot/dts/openbmc-flash-layout-64-alt.dtsi b/arch/arm/boot/dts/openbmc-flash-layout-64-alt.dtsi
new file mode 100644
index 0000000..6505258
--- /dev/null
+++ b/arch/arm/boot/dts/openbmc-flash-layout-64-alt.dtsi
@@ -0,0 +1,35 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Copyright (C) 2020 Bytedance.
+ */
+
+partitions {
+ compatible = "fixed-partitions";
+ #address-cells = <1>;
+ #size-cells = <1>;
+
+ u-boot@0 {
+ reg = <0x0 0xe0000>; // 896KB
+ label = "alt-u-boot";
+ };
+
+ u-boot-env@e0000 {
+ reg = <0xe0000 0x20000>; // 128KB
+ label = "alt-u-boot-env";
+ };
+
+ kernel@100000 {
+ reg = <0x100000 0x900000>; // 9MB
+ label = "alt-kernel";
+ };
+
+ rofs@a00000 {
+ reg = <0xa00000 0x2000000>; // 32MB
+ label = "alt-rofs";
+ };
+
+ rwfs@6000000 {
+ reg = <0x2a00000 0x1600000>; // 22MB
+ label = "alt-rwfs";
+ };
+};
diff --git a/arch/arm/boot/dts/qcom-apq8016-sbc.dts b/arch/arm/boot/dts/qcom-apq8016-sbc.dts
new file mode 100644
index 0000000..4ccd2dc
--- /dev/null
+++ b/arch/arm/boot/dts/qcom-apq8016-sbc.dts
@@ -0,0 +1,2 @@
+// SPDX-License-Identifier: GPL-2.0-only
+#include "arm64/qcom/apq8016-sbc.dts"
diff --git a/arch/arm/boot/dts/qcom-apq8064-sony-xperia-yuga.dts b/arch/arm/boot/dts/qcom-apq8064-sony-xperia-yuga.dts
index f8c97ef..0cee62c 100644
--- a/arch/arm/boot/dts/qcom-apq8064-sony-xperia-yuga.dts
+++ b/arch/arm/boot/dts/qcom-apq8064-sony-xperia-yuga.dts
@@ -19,7 +19,6 @@ chosen {
gpio-keys {
compatible = "gpio-keys";
- input-name = "gpio-keys";
pinctrl-names = "default";
pinctrl-0 = <&gpio_keys_pin_a>;
diff --git a/arch/arm/boot/dts/qcom-ipq4019.dtsi b/arch/arm/boot/dts/qcom-ipq4019.dtsi
index ff1bdb10..7dec055 100644
--- a/arch/arm/boot/dts/qcom-ipq4019.dtsi
+++ b/arch/arm/boot/dts/qcom-ipq4019.dtsi
@@ -637,7 +637,7 @@ usb3_hs_phy: hsphy@a6000 {
};
usb3: usb3@8af8800 {
- compatible = "qcom,dwc3";
+ compatible = "qcom,ipq4019-dwc3", "qcom,dwc3";
reg = <0x8af8800 0x100>;
#address-cells = <1>;
#size-cells = <1>;
@@ -669,7 +669,7 @@ usb2_hs_phy: hsphy@a8000 {
};
usb2: usb2@60f8800 {
- compatible = "qcom,dwc3";
+ compatible = "qcom,ipq4019-dwc3", "qcom,dwc3";
reg = <0x60f8800 0x100>;
#address-cells = <1>;
#size-cells = <1>;
diff --git a/arch/arm/boot/dts/qcom-ipq8064.dtsi b/arch/arm/boot/dts/qcom-ipq8064.dtsi
index 1148131..996f445 100644
--- a/arch/arm/boot/dts/qcom-ipq8064.dtsi
+++ b/arch/arm/boot/dts/qcom-ipq8064.dtsi
@@ -1080,7 +1080,7 @@ ss_phy_0: phy@100f8830 {
};
usb3_0: usb3@100f8800 {
- compatible = "qcom,dwc3", "syscon";
+ compatible = "qcom,ipq8064-dwc3", "qcom,dwc3";
#address-cells = <1>;
#size-cells = <1>;
reg = <0x100f8800 0x8000>;
@@ -1122,7 +1122,7 @@ ss_phy_1: phy@110f8830 {
};
usb3_1: usb3@110f8800 {
- compatible = "qcom,dwc3", "syscon";
+ compatible = "qcom,ipq8064-dwc3", "qcom,dwc3";
#address-cells = <1>;
#size-cells = <1>;
reg = <0x110f8800 0x8000>;
diff --git a/arch/arm/boot/dts/qcom-mdm9615-wp8548-mangoh-green.dts b/arch/arm/boot/dts/qcom-mdm9615-wp8548-mangoh-green.dts
index 942e3a2..0827de5 100644
--- a/arch/arm/boot/dts/qcom-mdm9615-wp8548-mangoh-green.dts
+++ b/arch/arm/boot/dts/qcom-mdm9615-wp8548-mangoh-green.dts
@@ -103,7 +103,7 @@ pins {
&gsbi3_spi {
spi@0 {
- compatible = "swir,mangoh-iotport-spi", "spidev";
+ compatible = "swir,mangoh-iotport-spi";
spi-max-frequency = <24000000>;
reg = <0>;
};
diff --git a/arch/arm/boot/dts/qcom-msm8974-fairphone-fp2.dts b/arch/arm/boot/dts/qcom-msm8974-fairphone-fp2.dts
index ea15b64..6d77e0f8 100644
--- a/arch/arm/boot/dts/qcom-msm8974-fairphone-fp2.dts
+++ b/arch/arm/boot/dts/qcom-msm8974-fairphone-fp2.dts
@@ -20,7 +20,6 @@ chosen {
gpio-keys {
compatible = "gpio-keys";
- input-name = "gpio-keys";
pinctrl-names = "default";
pinctrl-0 = <&gpio_keys_pin_a>;
diff --git a/arch/arm/boot/dts/qcom-msm8974-lge-nexus5-hammerhead.dts b/arch/arm/boot/dts/qcom-msm8974-lge-nexus5-hammerhead.dts
index 30ee913..0691361 100644
--- a/arch/arm/boot/dts/qcom-msm8974-lge-nexus5-hammerhead.dts
+++ b/arch/arm/boot/dts/qcom-msm8974-lge-nexus5-hammerhead.dts
@@ -450,7 +450,6 @@ bcrmf@1 {
gpio-keys {
compatible = "gpio-keys";
- input-name = "gpio-keys";
pinctrl-names = "default";
pinctrl-0 = <&gpio_keys_pin_a>;
diff --git a/arch/arm/boot/dts/qcom-msm8974-samsung-klte.dts b/arch/arm/boot/dts/qcom-msm8974-samsung-klte.dts
index 003f0fa9..96e1c97 100644
--- a/arch/arm/boot/dts/qcom-msm8974-samsung-klte.dts
+++ b/arch/arm/boot/dts/qcom-msm8974-samsung-klte.dts
@@ -349,7 +349,6 @@ bluetooth {
gpio-keys {
compatible = "gpio-keys";
- input-name = "gpio-keys";
pinctrl-names = "default";
pinctrl-0 = <&gpio_keys_pin_a>;
diff --git a/arch/arm/boot/dts/qcom-msm8974-sony-xperia-amami.dts b/arch/arm/boot/dts/qcom-msm8974-sony-xperia-amami.dts
index 398a3ea..79e2cfb 100644
--- a/arch/arm/boot/dts/qcom-msm8974-sony-xperia-amami.dts
+++ b/arch/arm/boot/dts/qcom-msm8974-sony-xperia-amami.dts
@@ -20,7 +20,6 @@ chosen {
gpio-keys {
compatible = "gpio-keys";
- input-name = "gpio-keys";
pinctrl-names = "default";
pinctrl-0 = <&gpio_keys_pin_a>;
diff --git a/arch/arm/boot/dts/qcom-msm8974-sony-xperia-castor.dts b/arch/arm/boot/dts/qcom-msm8974-sony-xperia-castor.dts
index b4dd85b..e66937e 100644
--- a/arch/arm/boot/dts/qcom-msm8974-sony-xperia-castor.dts
+++ b/arch/arm/boot/dts/qcom-msm8974-sony-xperia-castor.dts
@@ -20,7 +20,6 @@ chosen {
gpio-keys {
compatible = "gpio-keys";
- input-name = "gpio-keys";
pinctrl-names = "default";
pinctrl-0 = <&gpio_keys_pin_a>;
diff --git a/arch/arm/boot/dts/qcom-msm8974-sony-xperia-honami.dts b/arch/arm/boot/dts/qcom-msm8974-sony-xperia-honami.dts
index 9743bee..a62e5c2 100644
--- a/arch/arm/boot/dts/qcom-msm8974-sony-xperia-honami.dts
+++ b/arch/arm/boot/dts/qcom-msm8974-sony-xperia-honami.dts
@@ -20,7 +20,6 @@ chosen {
gpio-keys {
compatible = "gpio-keys";
- input-name = "gpio-keys";
pinctrl-names = "default";
pinctrl-0 = <&gpio_keys_pin_a>;
diff --git a/arch/arm/boot/dts/qcom-sdx55-t55.dts b/arch/arm/boot/dts/qcom-sdx55-t55.dts
index 2ffcd08..7ed8feb 100644
--- a/arch/arm/boot/dts/qcom-sdx55-t55.dts
+++ b/arch/arm/boot/dts/qcom-sdx55-t55.dts
@@ -236,6 +236,12 @@ &blsp1_uart3 {
status = "ok";
};
+&ipa {
+ status = "okay";
+
+ memory-region = <&ipa_fw_mem>;
+};
+
&qpic_bam {
status = "ok";
};
diff --git a/arch/arm/boot/dts/qcom-sdx55-telit-fn980-tlb.dts b/arch/arm/boot/dts/qcom-sdx55-telit-fn980-tlb.dts
index 80c40da..a4fa468 100644
--- a/arch/arm/boot/dts/qcom-sdx55-telit-fn980-tlb.dts
+++ b/arch/arm/boot/dts/qcom-sdx55-telit-fn980-tlb.dts
@@ -236,6 +236,27 @@ &blsp1_uart3 {
status = "ok";
};
+&ipa {
+ status = "okay";
+
+ memory-region = <&ipa_fw_mem>;
+};
+
+&pcie0_phy {
+ status = "okay";
+
+ vdda-phy-supply = <&vreg_l1e_bb_1p2>;
+ vdda-pll-supply = <&vreg_l4e_bb_0p875>;
+};
+
+&pcie_ep {
+ status = "okay";
+
+ pinctrl-names = "default";
+ pinctrl-0 = <&pcie_ep_clkreq_default &pcie_ep_perst_default
+ &pcie_ep_wake_default>;
+};
+
&qpic_bam {
status = "ok";
};
@@ -260,6 +281,44 @@ &remoteproc_mpss {
memory-region = <&mpss_adsp_mem>;
};
+&tlmm {
+ pcie_ep_clkreq_default: pcie_ep_clkreq_default {
+ mux {
+ pins = "gpio56";
+ function = "pcie_clkreq";
+ };
+ config {
+ pins = "gpio56";
+ drive-strength = <2>;
+ bias-disable;
+ };
+ };
+
+ pcie_ep_perst_default: pcie_ep_perst_default {
+ mux {
+ pins = "gpio57";
+ function = "gpio";
+ };
+ config {
+ pins = "gpio57";
+ drive-strength = <2>;
+ bias-pull-down;
+ };
+ };
+
+ pcie_ep_wake_default: pcie_ep_wake_default {
+ mux {
+ pins = "gpio53";
+ function = "gpio";
+ };
+ config {
+ pins = "gpio53";
+ drive-strength = <2>;
+ bias-disable;
+ };
+ };
+};
+
&usb_hsphy {
status = "okay";
vdda-pll-supply = <&vreg_l4e_bb_0p875>;
diff --git a/arch/arm/boot/dts/qcom-sdx55.dtsi b/arch/arm/boot/dts/qcom-sdx55.dtsi
index 44526ad..8ac0492 100644
--- a/arch/arm/boot/dts/qcom-sdx55.dtsi
+++ b/arch/arm/boot/dts/qcom-sdx55.dtsi
@@ -8,6 +8,7 @@
#include <dt-bindings/clock/qcom,gcc-sdx55.h>
#include <dt-bindings/clock/qcom,rpmh.h>
+#include <dt-bindings/gpio/gpio.h>
#include <dt-bindings/interconnect/qcom,sdx55.h>
#include <dt-bindings/interrupt-controller/arm-gic.h>
#include <dt-bindings/power/qcom-rpmpd.h>
@@ -309,6 +310,41 @@ qpic_nand: nand-controller@1b30000 {
status = "disabled";
};
+ pcie0_phy: phy@1c07000 {
+ compatible = "qcom,sdx55-qmp-pcie-phy";
+ reg = <0x01c07000 0x1c4>;
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges;
+ clocks = <&gcc GCC_PCIE_AUX_PHY_CLK_SRC>,
+ <&gcc GCC_PCIE_CFG_AHB_CLK>,
+ <&gcc GCC_PCIE_0_CLKREF_CLK>,
+ <&gcc GCC_PCIE_RCHNG_PHY_CLK>;
+ clock-names = "aux", "cfg_ahb", "ref", "refgen";
+
+ resets = <&gcc GCC_PCIE_PHY_BCR>;
+ reset-names = "phy";
+
+ assigned-clocks = <&gcc GCC_PCIE_RCHNG_PHY_CLK>;
+ assigned-clock-rates = <100000000>;
+
+ status = "disabled";
+
+ pcie0_lane: lanes@1c06000 {
+ reg = <0x01c06000 0x104>, /* tx0 */
+ <0x01c06200 0x328>, /* rx0 */
+ <0x01c07200 0x1e8>, /* pcs */
+ <0x01c06800 0x104>, /* tx1 */
+ <0x01c06a00 0x328>, /* rx1 */
+ <0x01c07600 0x800>; /* pcs_misc */
+ clocks = <&gcc GCC_PCIE_PIPE_CLK>;
+ clock-names = "pipe0";
+
+ #phy-cells = <0>;
+ clock-output-names = "pcie_pipe_clk";
+ };
+ };
+
ipa: ipa@1e40000 {
compatible = "qcom,sdx55-ipa";
@@ -333,12 +369,10 @@ ipa: ipa@1e40000 {
clocks = <&rpmhcc RPMH_IPA_CLK>;
clock-names = "core";
- interconnects = <&system_noc MASTER_IPA &system_noc SLAVE_SNOC_MEM_NOC_GC>,
- <&mem_noc MASTER_SNOC_GC_MEM_NOC &mc_virt SLAVE_EBI_CH0>,
+ interconnects = <&system_noc MASTER_IPA &mc_virt SLAVE_EBI_CH0>,
<&system_noc MASTER_IPA &system_noc SLAVE_OCIMEM>,
<&mem_noc MASTER_AMPSS_M0 &system_noc SLAVE_IPA_CFG>;
- interconnect-names = "memory-a",
- "memory-b",
+ interconnect-names = "memory",
"imem",
"config";
@@ -356,6 +390,11 @@ tcsr_mutex: hwlock@1f40000 {
#hwlock-cells = <1>;
};
+ tcsr: syscon@1fcb000 {
+ compatible = "syscon";
+ reg = <0x01fc0000 0x1000>;
+ };
+
sdhc_1: sdhci@8804000 {
compatible = "qcom,sdx55-sdhci", "qcom,sdhci-msm-v5";
reg = <0x08804000 0x1000>;
@@ -368,6 +407,45 @@ sdhc_1: sdhci@8804000 {
status = "disabled";
};
+ pcie_ep: pcie-ep@40000000 {
+ compatible = "qcom,sdx55-pcie-ep";
+ reg = <0x01c00000 0x3000>,
+ <0x40000000 0xf1d>,
+ <0x40000f20 0xc8>,
+ <0x40001000 0x1000>,
+ <0x40002000 0x10000>,
+ <0x01c03000 0x3000>;
+ reg-names = "parf", "dbi", "elbi", "atu", "addr_space",
+ "mmio";
+
+ qcom,perst-regs = <&tcsr 0xb258 0xb270>;
+
+ clocks = <&gcc GCC_PCIE_AUX_CLK>,
+ <&gcc GCC_PCIE_CFG_AHB_CLK>,
+ <&gcc GCC_PCIE_MSTR_AXI_CLK>,
+ <&gcc GCC_PCIE_SLV_AXI_CLK>,
+ <&gcc GCC_PCIE_SLV_Q2A_AXI_CLK>,
+ <&gcc GCC_PCIE_SLEEP_CLK>,
+ <&gcc GCC_PCIE_0_CLKREF_CLK>;
+ clock-names = "aux", "cfg", "bus_master", "bus_slave",
+ "slave_q2a", "sleep", "ref";
+
+ interrupts = <GIC_SPI 140 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 145 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "global", "doorbell";
+ reset-gpios = <&tlmm 57 GPIO_ACTIVE_LOW>;
+ wake-gpios = <&tlmm 53 GPIO_ACTIVE_LOW>;
+ resets = <&gcc GCC_PCIE_BCR>;
+ reset-names = "core";
+ power-domains = <&gcc PCIE_GDSC>;
+ phys = <&pcie0_lane>;
+ phy-names = "pciephy";
+ max-link-speed = <3>;
+ num-lanes = <2>;
+
+ status = "disabled";
+ };
+
remoteproc_mpss: remoteproc@4080000 {
compatible = "qcom,sdx55-mpss-pas";
reg = <0x04080000 0x4040>;
diff --git a/arch/arm/boot/dts/qcom-sdx65-mtp.dts b/arch/arm/boot/dts/qcom-sdx65-mtp.dts
new file mode 100644
index 0000000..59457da
--- /dev/null
+++ b/arch/arm/boot/dts/qcom-sdx65-mtp.dts
@@ -0,0 +1,25 @@
+// SPDX-License-Identifier: BSD-3-Clause
+/*
+ * Copyright (c) 2021 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+/dts-v1/;
+
+#include "qcom-sdx65.dtsi"
+
+/ {
+ model = "Qualcomm Technologies, Inc. SDX65 MTP";
+ compatible = "qcom,sdx65-mtp", "qcom,sdx65";
+ qcom,board-id = <0x2010008 0x302>;
+
+ aliases {
+ serial0 = &blsp1_uart3;
+ };
+
+ chosen {
+ stdout-path = "serial0:115200n8";
+ };
+};
+
+&blsp1_uart3 {
+ status = "ok";
+};
diff --git a/arch/arm/boot/dts/qcom-sdx65.dtsi b/arch/arm/boot/dts/qcom-sdx65.dtsi
new file mode 100644
index 0000000..796641d
--- /dev/null
+++ b/arch/arm/boot/dts/qcom-sdx65.dtsi
@@ -0,0 +1,222 @@
+// SPDX-License-Identifier: BSD-3-Clause
+/*
+ * SDX65 SoC device tree source
+ *
+ * Copyright (c) 2021 Qualcomm Innovation Center, Inc. All rights reserved.
+ *
+ */
+
+#include <dt-bindings/clock/qcom,gcc-sdx65.h>
+#include <dt-bindings/clock/qcom,rpmh.h>
+#include <dt-bindings/interrupt-controller/arm-gic.h>
+#include <dt-bindings/soc/qcom,rpmh-rsc.h>
+
+/ {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ qcom,msm-id = <458 0x10000>, <483 0x10000>, <509 0x10000>;
+ interrupt-parent = <&intc>;
+
+ memory {
+ device_type = "memory";
+ reg = <0 0>;
+ };
+
+ clocks {
+ xo_board: xo-board {
+ compatible = "fixed-clock";
+ clock-frequency = <76800000>;
+ clock-output-names = "xo_board";
+ #clock-cells = <0>;
+ };
+
+ sleep_clk: sleep-clk {
+ compatible = "fixed-clock";
+ clock-frequency = <32764>;
+ clock-output-names = "sleep_clk";
+ #clock-cells = <0>;
+ };
+ };
+
+ cpus {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ cpu0: cpu@0 {
+ device_type = "cpu";
+ compatible = "arm,cortex-a7";
+ reg = <0x0>;
+ enable-method = "psci";
+ };
+ };
+
+ psci {
+ compatible = "arm,psci-1.0";
+ method = "smc";
+ };
+
+ reserved_memory: reserved-memory {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges;
+
+ cmd_db: reserved-memory@8fee0000 {
+ compatible = "qcom,cmd-db";
+ reg = <0x8fee0000 0x20000>;
+ no-map;
+ };
+ };
+
+ soc: soc {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges;
+ compatible = "simple-bus";
+
+ gcc: clock-controller@100000 {
+ compatible = "qcom,gcc-sdx65";
+ reg = <0x00100000 0x001f7400>;
+ clocks = <&rpmhcc RPMH_CXO_CLK>, <&rpmhcc RPMH_CXO_CLK_A>, <&sleep_clk>;
+ clock-names = "bi_tcxo", "bi_tcxo_ao", "sleep_clk";
+ #clock-cells = <1>;
+ #reset-cells = <1>;
+ };
+
+ blsp1_uart3: serial@831000 {
+ compatible = "qcom,msm-uartdm-v1.4", "qcom,msm-uartdm";
+ reg = <0x00831000 0x200>;
+ interrupts = <GIC_SPI 26 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&gcc GCC_BLSP1_UART3_APPS_CLK>, <&gcc GCC_BLSP1_AHB_CLK>;
+ clock-names = "core", "iface";
+ status = "disabled";
+ };
+
+ tlmm: pinctrl@f100000 {
+ compatible = "qcom,sdx65-tlmm";
+ reg = <0xf100000 0x300000>;
+ interrupts = <GIC_SPI 212 IRQ_TYPE_LEVEL_HIGH>;
+ gpio-controller;
+ #gpio-cells = <2>;
+ gpio-ranges = <&tlmm 0 0 109>;
+ interrupt-controller;
+ interrupt-parent = <&intc>;
+ #interrupt-cells = <2>;
+ };
+
+ pdc: interrupt-controller@b210000 {
+ compatible = "qcom,sdx65-pdc", "qcom,pdc";
+ reg = <0xb210000 0x10000>;
+ qcom,pdc-ranges = <0 147 52>, <52 266 32>;
+ #interrupt-cells = <2>;
+ interrupt-parent = <&intc>;
+ interrupt-controller;
+ };
+
+ intc: interrupt-controller@17800000 {
+ compatible = "qcom,msm-qgic2";
+ interrupt-controller;
+ interrupt-parent = <&intc>;
+ #interrupt-cells = <3>;
+ reg = <0x17800000 0x1000>,
+ <0x17802000 0x1000>;
+ };
+
+ timer@17820000 {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges;
+ compatible = "arm,armv7-timer-mem";
+ reg = <0x17820000 0x1000>;
+ clock-frequency = <19200000>;
+
+ frame@17821000 {
+ frame-number = <0>;
+ interrupts = <GIC_SPI 7 0x4>,
+ <GIC_SPI 6 0x4>;
+ reg = <0x17821000 0x1000>,
+ <0x17822000 0x1000>;
+ };
+
+ frame@17823000 {
+ frame-number = <1>;
+ interrupts = <GIC_SPI 8 0x4>;
+ reg = <0x17823000 0x1000>;
+ status = "disabled";
+ };
+
+ frame@17824000 {
+ frame-number = <2>;
+ interrupts = <GIC_SPI 9 0x4>;
+ reg = <0x17824000 0x1000>;
+ status = "disabled";
+ };
+
+ frame@17825000 {
+ frame-number = <3>;
+ interrupts = <GIC_SPI 10 0x4>;
+ reg = <0x17825000 0x1000>;
+ status = "disabled";
+ };
+
+ frame@17826000 {
+ frame-number = <4>;
+ interrupts = <GIC_SPI 11 0x4>;
+ reg = <0x17826000 0x1000>;
+ status = "disabled";
+ };
+
+ frame@17827000 {
+ frame-number = <5>;
+ interrupts = <GIC_SPI 12 0x4>;
+ reg = <0x17827000 0x1000>;
+ status = "disabled";
+ };
+
+ frame@17828000 {
+ frame-number = <6>;
+ interrupts = <GIC_SPI 13 0x4>;
+ reg = <0x17828000 0x1000>;
+ status = "disabled";
+ };
+
+ frame@17829000 {
+ frame-number = <7>;
+ interrupts = <GIC_SPI 14 0x4>;
+ reg = <0x17829000 0x1000>;
+ status = "disabled";
+ };
+ };
+
+ apps_rsc: rsc@17830000 {
+ label = "apps_rsc";
+ compatible = "qcom,rpmh-rsc";
+ reg = <0x17830000 0x10000>,
+ <0x17840000 0x10000>;
+ reg-names = "drv-0", "drv-1";
+ interrupts = <GIC_SPI 16 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 17 IRQ_TYPE_LEVEL_HIGH>;
+ qcom,tcs-offset = <0xd00>;
+ qcom,drv-id = <1>;
+ qcom,tcs-config = <ACTIVE_TCS 2>,
+ <SLEEP_TCS 2>,
+ <WAKE_TCS 2>,
+ <CONTROL_TCS 1>;
+
+ rpmhcc: clock-controller@1 {
+ compatible = "qcom,sdx65-rpmh-clk";
+ #clock-cells = <1>;
+ clock-names = "xo";
+ clocks = <&xo_board>;
+ };
+ };
+ };
+
+ timer {
+ compatible = "arm,armv7-timer";
+ interrupts = <1 13 0xf08>,
+ <1 12 0xf08>,
+ <1 10 0xf08>,
+ <1 11 0xf08>;
+ clock-frequency = <19200000>;
+ };
+};
diff --git a/arch/arm/boot/dts/r8a7742-iwg21d-q7-dbcm-ca.dts b/arch/arm/boot/dts/r8a7742-iwg21d-q7-dbcm-ca.dts
index 33db593..3c8a7c8 100644
--- a/arch/arm/boot/dts/r8a7742-iwg21d-q7-dbcm-ca.dts
+++ b/arch/arm/boot/dts/r8a7742-iwg21d-q7-dbcm-ca.dts
@@ -44,6 +44,22 @@ mclk_cam4: mclk-cam4 {
#clock-cells = <0>;
clock-frequency = <26000000>;
};
+
+ reg_1p8v: 1p8v {
+ compatible = "regulator-fixed";
+ regulator-name = "1P8V";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ regulator-always-on;
+ };
+
+ reg_2p8v: 2p8v {
+ compatible = "regulator-fixed";
+ regulator-name = "2P8V";
+ regulator-min-microvolt = <2800000>;
+ regulator-max-microvolt = <2800000>;
+ regulator-always-on;
+ };
};
&avb {
diff --git a/arch/arm/boot/dts/r8a7742-iwg21d-q7-dbcm-ov5640-single.dtsi b/arch/arm/boot/dts/r8a7742-iwg21d-q7-dbcm-ov5640-single.dtsi
index 70c72ba..40cef0b 100644
--- a/arch/arm/boot/dts/r8a7742-iwg21d-q7-dbcm-ov5640-single.dtsi
+++ b/arch/arm/boot/dts/r8a7742-iwg21d-q7-dbcm-ov5640-single.dtsi
@@ -17,6 +17,9 @@ ov5640@3c {
reg = <0x3c>;
clocks = <&MCLK_CAM>;
clock-names = "xclk";
+ AVDD-supply = <®_2p8v>;
+ DOVDD-supply = <®_2p8v>;
+ DVDD-supply = <®_1p8v>;
status = "okay";
port {
diff --git a/arch/arm/boot/dts/sama5d2.dtsi b/arch/arm/boot/dts/sama5d2.dtsi
index 801969c..09c741e 100644
--- a/arch/arm/boot/dts/sama5d2.dtsi
+++ b/arch/arm/boot/dts/sama5d2.dtsi
@@ -288,6 +288,7 @@ qspi0: spi@f0020000 {
reg-names = "qspi_base", "qspi_mmap";
interrupts = <52 IRQ_TYPE_LEVEL_HIGH 7>;
clocks = <&pmc PMC_TYPE_PERIPHERAL 52>;
+ clock-names = "pclk";
#address-cells = <1>;
#size-cells = <0>;
status = "disabled";
@@ -299,6 +300,7 @@ qspi1: spi@f0024000 {
reg-names = "qspi_base", "qspi_mmap";
interrupts = <53 IRQ_TYPE_LEVEL_HIGH 7>;
clocks = <&pmc PMC_TYPE_PERIPHERAL 53>;
+ clock-names = "pclk";
#address-cells = <1>;
#size-cells = <0>;
status = "disabled";
diff --git a/arch/arm/boot/dts/sama7g5-pinfunc.h b/arch/arm/boot/dts/sama7g5-pinfunc.h
index 22fe9e5..4eb3044 100644
--- a/arch/arm/boot/dts/sama7g5-pinfunc.h
+++ b/arch/arm/boot/dts/sama7g5-pinfunc.h
@@ -765,7 +765,7 @@
#define PIN_PD20__PCK0 PINMUX_PIN(PIN_PD20, 1, 3)
#define PIN_PD20__FLEXCOM2_IO3 PINMUX_PIN(PIN_PD20, 2, 2)
#define PIN_PD20__PWMH3 PINMUX_PIN(PIN_PD20, 3, 4)
-#define PIN_PD20__CANTX4 PINMUX_PIN(PIN_PD20, 5, 2)
+#define PIN_PD20__CANTX4 PINMUX_PIN(PIN_PD20, 4, 2)
#define PIN_PD20__FLEXCOM5_IO0 PINMUX_PIN(PIN_PD20, 6, 5)
#define PIN_PD21 117
#define PIN_PD21__GPIO PINMUX_PIN(PIN_PD21, 0, 0)
diff --git a/arch/arm/boot/dts/sama7g5.dtsi b/arch/arm/boot/dts/sama7g5.dtsi
index 7039311..eddcfbf 100644
--- a/arch/arm/boot/dts/sama7g5.dtsi
+++ b/arch/arm/boot/dts/sama7g5.dtsi
@@ -181,6 +181,36 @@ tcb1: timer@e0800000 {
clock-names = "t0_clk", "t1_clk", "t2_clk", "slow_clk";
};
+ qspi0: spi@e080c000 {
+ compatible = "microchip,sama7g5-ospi";
+ reg = <0xe080c000 0x400>, <0x20000000 0x10000000>;
+ reg-names = "qspi_base", "qspi_mmap";
+ interrupts = <GIC_SPI 78 IRQ_TYPE_LEVEL_HIGH>;
+ dmas = <&dma0 AT91_XDMAC_DT_PERID(41)>,
+ <&dma0 AT91_XDMAC_DT_PERID(40)>;
+ dma-names = "tx", "rx";
+ clocks = <&pmc PMC_TYPE_PERIPHERAL 78>, <&pmc PMC_TYPE_GCK 78>;
+ clock-names = "pclk", "gclk";
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ qspi1: spi@e0810000 {
+ compatible = "microchip,sama7g5-qspi";
+ reg = <0xe0810000 0x400>, <0x30000000 0x10000000>;
+ reg-names = "qspi_base", "qspi_mmap";
+ interrupts = <GIC_SPI 79 IRQ_TYPE_LEVEL_HIGH>;
+ dmas = <&dma0 AT91_XDMAC_DT_PERID(43)>,
+ <&dma0 AT91_XDMAC_DT_PERID(42)>;
+ dma-names = "tx", "rx";
+ clocks = <&pmc PMC_TYPE_PERIPHERAL 79>, <&pmc PMC_TYPE_GCK 79>;
+ clock-names = "pclk", "gclk";
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
adc: adc@e1000000 {
compatible = "microchip,sama7g5-adc";
reg = <0xe1000000 0x200>;
diff --git a/arch/arm/boot/dts/socfpga.dtsi b/arch/arm/boot/dts/socfpga.dtsi
index 0b021ee..7c1d642 100644
--- a/arch/arm/boot/dts/socfpga.dtsi
+++ b/arch/arm/boot/dts/socfpga.dtsi
@@ -782,7 +782,7 @@ ocram: sram@ffff0000 {
};
qspi: spi@ff705000 {
- compatible = "cdns,qspi-nor";
+ compatible = "intel,socfpga-qspi", "cdns,qspi-nor";
#address-cells = <1>;
#size-cells = <0>;
reg = <0xff705000 0x1000>,
diff --git a/arch/arm/boot/dts/socfpga_arria10.dtsi b/arch/arm/boot/dts/socfpga_arria10.dtsi
index a574ea9..3ba431d 100644
--- a/arch/arm/boot/dts/socfpga_arria10.dtsi
+++ b/arch/arm/boot/dts/socfpga_arria10.dtsi
@@ -756,7 +756,7 @@ usb0-ecc@ff8c8800 {
};
qspi: spi@ff809000 {
- compatible = "cdns,qspi-nor";
+ compatible = "intel,socfpga-qspi", "cdns,qspi-nor";
#address-cells = <1>;
#size-cells = <0>;
reg = <0xff809000 0x100>,
diff --git a/arch/arm/boot/dts/spear1310-evb.dts b/arch/arm/boot/dts/spear1310-evb.dts
index 8fcb6be..4cbadcb 100644
--- a/arch/arm/boot/dts/spear1310-evb.dts
+++ b/arch/arm/boot/dts/spear1310-evb.dts
@@ -395,22 +395,6 @@ m25p80@1 {
pl022,wait-state = <0>;
pl022,duplex = <0>;
};
-
- spidev@2 {
- compatible = "spidev";
- reg = <2>;
- spi-max-frequency = <25000000>;
- spi-cpha;
- pl022,hierarchy = <0>;
- pl022,interface = <0>;
- pl022,slave-tx-disable;
- pl022,com-mode = <0x2>;
- pl022,rx-level-trig = <0>;
- pl022,tx-level-trig = <0>;
- pl022,ctrl-len = <0x11>;
- pl022,wait-state = <0>;
- pl022,duplex = <0>;
- };
};
wdt@ec800620 {
diff --git a/arch/arm/boot/dts/spear1340-evb.dts b/arch/arm/boot/dts/spear1340-evb.dts
index f70ff56..fd194eb 100644
--- a/arch/arm/boot/dts/spear1340-evb.dts
+++ b/arch/arm/boot/dts/spear1340-evb.dts
@@ -489,22 +489,6 @@ stmpe_touchscreen {
ts,i-drive = <1>;
};
};
-
- spidev@2 {
- compatible = "spidev";
- reg = <2>;
- spi-max-frequency = <25000000>;
- spi-cpha;
- pl022,hierarchy = <0>;
- pl022,interface = <0>;
- pl022,slave-tx-disable;
- pl022,com-mode = <0x2>;
- pl022,rx-level-trig = <0>;
- pl022,tx-level-trig = <0>;
- pl022,ctrl-len = <0x11>;
- pl022,wait-state = <0>;
- pl022,duplex = <0>;
- };
};
timer@ec800600 {
diff --git a/arch/arm/boot/dts/spear310.dtsi b/arch/arm/boot/dts/spear310.dtsi
index 8ce751a..7757083 100644
--- a/arch/arm/boot/dts/spear310.dtsi
+++ b/arch/arm/boot/dts/spear310.dtsi
@@ -92,6 +92,7 @@ serial@b2200000 {
gpiopinctrl: gpio@b4000000 {
compatible = "st,spear-plgpio";
reg = <0xb4000000 0x1000>;
+ regmap = <&pinmux>;
#interrupt-cells = <1>;
interrupt-controller;
gpio-controller;
diff --git a/arch/arm/boot/dts/spear320.dtsi b/arch/arm/boot/dts/spear320.dtsi
index 3bc1e93..47ac447 100644
--- a/arch/arm/boot/dts/spear320.dtsi
+++ b/arch/arm/boot/dts/spear320.dtsi
@@ -120,6 +120,7 @@ serial@a4000000 {
gpiopinctrl: gpio@b3000000 {
compatible = "st,spear-plgpio";
reg = <0xb3000000 0x1000>;
+ regmap = <&pinmux>;
#interrupt-cells = <1>;
interrupt-controller;
gpio-controller;
diff --git a/arch/arm/boot/dts/spear320s.dtsi b/arch/arm/boot/dts/spear320s.dtsi
new file mode 100644
index 0000000..133236d
--- /dev/null
+++ b/arch/arm/boot/dts/spear320s.dtsi
@@ -0,0 +1,24 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * DTS file for SPEAr320s SoC
+ *
+ * Copyright 2021 Herve Codina <herve.codina@bootlin.com>
+ */
+
+/include/ "spear320.dtsi"
+
+/ {
+ ahb {
+ apb {
+ gpiopinctrl: gpio@b3000000 {
+ /*
+ * The "RM0321 SPEAr320s address and map
+ * registers" document mentions interrupt 6
+ * (NPGIO_INTR) for the PL_GPIO interrupt.
+ */
+ interrupts = <6>;
+ interrupt-parent = <&shirq>;
+ };
+ };
+ };
+};
diff --git a/arch/arm/boot/dts/ste-dbx5x0.dtsi b/arch/arm/boot/dts/ste-dbx5x0.dtsi
index 68607e4..dc0bcc7 100644
--- a/arch/arm/boot/dts/ste-dbx5x0.dtsi
+++ b/arch/arm/boot/dts/ste-dbx5x0.dtsi
@@ -5,6 +5,7 @@
#include <dt-bindings/interrupt-controller/irq.h>
#include <dt-bindings/interrupt-controller/arm-gic.h>
+#include <dt-bindings/reset/stericsson,db8500-prcc-reset.h>
#include <dt-bindings/mfd/dbx500-prcmu.h>
#include <dt-bindings/arm/ux500_pm_domains.h>
#include <dt-bindings/gpio/gpio.h>
@@ -300,6 +301,10 @@ prcc_kclk: prcc-kernel-clock {
#clock-cells = <2>;
};
+ prcc_reset: prcc-reset-controller {
+ #reset-cells = <2>;
+ };
+
rtc_clk: rtc32k-clock {
#clock-cells = <0>;
};
@@ -662,6 +667,7 @@ i2c0: i2c@80004000 {
clocks = <&prcc_kclk 3 3>, <&prcc_pclk 3 3>;
clock-names = "i2cclk", "apb_pclk";
power-domains = <&pm_domains DOMAIN_VAPE>;
+ resets = <&prcc_reset DB8500_PRCC_3 DB8500_PRCC_3_RESET_I2C0>;
status = "disabled";
};
@@ -680,6 +686,7 @@ i2c1: i2c@80122000 {
clocks = <&prcc_kclk 1 2>, <&prcc_pclk 1 2>;
clock-names = "i2cclk", "apb_pclk";
power-domains = <&pm_domains DOMAIN_VAPE>;
+ resets = <&prcc_reset DB8500_PRCC_1 DB8500_PRCC_1_RESET_I2C1>;
status = "disabled";
};
@@ -698,6 +705,7 @@ i2c2: i2c@80128000 {
clocks = <&prcc_kclk 1 6>, <&prcc_pclk 1 6>;
clock-names = "i2cclk", "apb_pclk";
power-domains = <&pm_domains DOMAIN_VAPE>;
+ resets = <&prcc_reset DB8500_PRCC_1 DB8500_PRCC_1_RESET_I2C2>;
status = "disabled";
};
@@ -716,6 +724,7 @@ i2c3: i2c@80110000 {
clocks = <&prcc_kclk 2 0>, <&prcc_pclk 2 0>;
clock-names = "i2cclk", "apb_pclk";
power-domains = <&pm_domains DOMAIN_VAPE>;
+ resets = <&prcc_reset DB8500_PRCC_2 DB8500_PRCC_2_RESET_I2C3>;
status = "disabled";
};
@@ -734,6 +743,7 @@ i2c4: i2c@8012a000 {
clocks = <&prcc_kclk 1 9>, <&prcc_pclk 1 10>;
clock-names = "i2cclk", "apb_pclk";
power-domains = <&pm_domains DOMAIN_VAPE>;
+ resets = <&prcc_reset DB8500_PRCC_1 DB8500_PRCC_1_RESET_I2C4>;
status = "disabled";
};
@@ -750,6 +760,7 @@ ssp0: spi@80002000 {
<&dma 8 0 0x0>; /* Logical - MemToDev */
dma-names = "rx", "tx";
power-domains = <&pm_domains DOMAIN_VAPE>;
+ resets = <&prcc_reset DB8500_PRCC_3 DB8500_PRCC_3_RESET_SSP0>;
status = "disabled";
};
@@ -766,6 +777,7 @@ ssp1: spi@80003000 {
<&dma 9 0 0x0>; /* Logical - MemToDev */
dma-names = "rx", "tx";
power-domains = <&pm_domains DOMAIN_VAPE>;
+ resets = <&prcc_reset DB8500_PRCC_3 DB8500_PRCC_3_RESET_SSP1>;
status = "disabled";
};
@@ -834,6 +846,7 @@ spi3: spi@80129000 {
<&dma 40 0 0x0>; /* Logical - MemToDev */
dma-names = "rx", "tx";
power-domains = <&pm_domains DOMAIN_VAPE>;
+ resets = <&prcc_reset DB8500_PRCC_1 DB8500_PRCC_1_RESET_SPI3>;
status = "disabled";
};
@@ -849,6 +862,7 @@ serial0: uart@80120000 {
clocks = <&prcc_kclk 1 0>, <&prcc_pclk 1 0>;
clock-names = "uart", "apb_pclk";
+ resets = <&prcc_reset DB8500_PRCC_1 DB8500_PRCC_1_RESET_UART0>;
status = "disabled";
};
@@ -864,6 +878,7 @@ serial1: uart@80121000 {
clocks = <&prcc_kclk 1 1>, <&prcc_pclk 1 1>;
clock-names = "uart", "apb_pclk";
+ resets = <&prcc_reset DB8500_PRCC_1 DB8500_PRCC_1_RESET_UART1>;
status = "disabled";
};
@@ -879,6 +894,7 @@ serial2: uart@80007000 {
clocks = <&prcc_kclk 3 6>, <&prcc_pclk 3 6>;
clock-names = "uart", "apb_pclk";
+ resets = <&prcc_reset DB8500_PRCC_3 DB8500_PRCC_3_RESET_UART2>;
status = "disabled";
};
@@ -895,6 +911,7 @@ mmc@80126000 {
clocks = <&prcc_kclk 1 5>, <&prcc_pclk 1 5>;
clock-names = "sdi", "apb_pclk";
power-domains = <&pm_domains DOMAIN_VAPE>;
+ resets = <&prcc_reset DB8500_PRCC_1 DB8500_PRCC_1_RESET_SDI0>;
status = "disabled";
};
@@ -911,6 +928,7 @@ mmc@80118000 {
clocks = <&prcc_kclk 2 4>, <&prcc_pclk 2 6>;
clock-names = "sdi", "apb_pclk";
power-domains = <&pm_domains DOMAIN_VAPE>;
+ resets = <&prcc_reset DB8500_PRCC_2 DB8500_PRCC_2_RESET_SDI1>;
status = "disabled";
};
@@ -927,6 +945,7 @@ mmc@80005000 {
clocks = <&prcc_kclk 3 4>, <&prcc_pclk 3 4>;
clock-names = "sdi", "apb_pclk";
power-domains = <&pm_domains DOMAIN_VAPE>;
+ resets = <&prcc_reset DB8500_PRCC_3 DB8500_PRCC_3_RESET_SDI2>;
status = "disabled";
};
@@ -943,6 +962,7 @@ mmc@80119000 {
clocks = <&prcc_kclk 2 5>, <&prcc_pclk 2 7>;
clock-names = "sdi", "apb_pclk";
power-domains = <&pm_domains DOMAIN_VAPE>;
+ resets = <&prcc_reset DB8500_PRCC_2 DB8500_PRCC_2_RESET_SDI3>;
status = "disabled";
};
@@ -959,6 +979,7 @@ mmc@80114000 {
clocks = <&prcc_kclk 2 2>, <&prcc_pclk 2 4>;
clock-names = "sdi", "apb_pclk";
power-domains = <&pm_domains DOMAIN_VAPE>;
+ resets = <&prcc_reset DB8500_PRCC_2 DB8500_PRCC_2_RESET_SDI4>;
status = "disabled";
};
@@ -975,6 +996,7 @@ mmc@80008000 {
clocks = <&prcc_kclk 3 7>, <&prcc_pclk 3 7>;
clock-names = "sdi", "apb_pclk";
power-domains = <&pm_domains DOMAIN_VAPE>;
+ resets = <&prcc_reset DB8500_PRCC_3 DB8500_PRCC_3_RESET_SDI5>;
status = "disabled";
};
@@ -996,6 +1018,7 @@ msp0: msp@80123000 {
clocks = <&prcc_kclk 1 3>, <&prcc_pclk 1 3>;
clock-names = "msp", "apb_pclk";
+ resets = <&prcc_reset DB8500_PRCC_1 DB8500_PRCC_1_RESET_MSP0>;
status = "disabled";
};
@@ -1012,6 +1035,7 @@ msp1: msp@80124000 {
clocks = <&prcc_kclk 1 4>, <&prcc_pclk 1 4>;
clock-names = "msp", "apb_pclk";
+ resets = <&prcc_reset DB8500_PRCC_1 DB8500_PRCC_1_RESET_MSP1>;
status = "disabled";
};
@@ -1030,6 +1054,7 @@ HighPrio - Fixed */
clocks = <&prcc_kclk 2 3>, <&prcc_pclk 2 5>;
clock-names = "msp", "apb_pclk";
+ resets = <&prcc_reset DB8500_PRCC_2 DB8500_PRCC_2_RESET_MSP2>;
status = "disabled";
};
@@ -1046,6 +1071,7 @@ msp3: msp@80125000 {
clocks = <&prcc_kclk 1 10>, <&prcc_pclk 1 11>;
clock-names = "msp", "apb_pclk";
+ resets = <&prcc_reset DB8500_PRCC_1 DB8500_PRCC_1_RESET_MSP3>;
status = "disabled";
};
diff --git a/arch/arm/boot/dts/ste-ux500-samsung-gavini.dts b/arch/arm/boot/dts/ste-ux500-samsung-gavini.dts
index 47bbf5a..1c0e5cf 100644
--- a/arch/arm/boot/dts/ste-ux500-samsung-gavini.dts
+++ b/arch/arm/boot/dts/ste-ux500-samsung-gavini.dts
@@ -232,19 +232,18 @@ i2c-gpio-0 {
#address-cells = <1>;
#size-cells = <0>;
- /* TODO: Memsic MMC328 magnetometer */
- magnetometer@30 {
- compatible = "memsic,mmc328";
- reg = <0x30>;
- /* TODO: if you have the schematic, check if both voltages come from AUX2 */
- /* VDA 1.8 V */
- vda-supply = <&ab8500_ldo_aux2_reg>;
- /* VDD 1.8V */
- vdd-supply = <&ab8500_ldo_aux2_reg>;
- /* GPIO204 */
+ /* Yamaha YAS530 magnetometer */
+ magnetometer@2e {
+ compatible = "yamaha,yas530";
+ reg = <0x2e>;
+ /* VDD 3V */
+ vdd-supply = <&ab8500_ldo_aux1_reg>;
+ /* IOVDD 1.8V */
+ iovdd-supply = <&ab8500_ldo_aux2_reg>;
+ /* GPIO204 COMPASS_RST_N */
reset-gpios = <&gpio6 12 GPIO_ACTIVE_LOW>;
pinctrl-names = "default";
- pinctrl-0 = <&mmc328_default>;
+ pinctrl-0 = <&yas530_default>;
};
/* TODO: this should also be used by the NCP6914 Camera power management unit */
};
@@ -645,6 +644,15 @@ gavini_cfg1 {
};
};
};
+ /* Reset line for the Yamaha YAS530 magnetometer */
+ yas530 {
+ yas530_default: yas530_janice {
+ janice_cfg1 {
+ pins = "GPIO204_AF23";
+ ste,config = <&gpio_out_hi>;
+ };
+ };
+ };
/* Flash and torch */
flash {
gpio_flash_default_mode: flash_default {
@@ -747,15 +755,6 @@ gavini_cfg1 {
};
};
};
- /* Reset line for the Memsic MMC328 magnetometer */
- mmc328 {
- mmc328_default: mmc328_gavini {
- gavini_cfg1 {
- pins = "GPIO204_AF23";
- ste,config = <&gpio_out_hi>;
- };
- };
- };
/* Interrupt line for Invensense MPU3050 gyroscope */
mpu3050 {
mpu3050_default: mpu3050 {
diff --git a/arch/arm/boot/dts/stm32f429-disco.dts b/arch/arm/boot/dts/stm32f429-disco.dts
index 075ac57..6435e099 100644
--- a/arch/arm/boot/dts/stm32f429-disco.dts
+++ b/arch/arm/boot/dts/stm32f429-disco.dts
@@ -192,7 +192,7 @@ l3gd20: l3gd20@0 {
display: display@1{
/* Connect panel-ilitek-9341 to ltdc */
- compatible = "st,sf-tc240t-9370-t";
+ compatible = "st,sf-tc240t-9370-t", "ilitek,ili9341";
reg = <1>;
spi-3wire;
spi-max-frequency = <10000000>;
diff --git a/arch/arm/boot/dts/stm32mp15-pinctrl.dtsi b/arch/arm/boot/dts/stm32mp15-pinctrl.dtsi
index 2ebafe2..3b65130 100644
--- a/arch/arm/boot/dts/stm32mp15-pinctrl.dtsi
+++ b/arch/arm/boot/dts/stm32mp15-pinctrl.dtsi
@@ -1718,7 +1718,7 @@ pins2 {
stusb1600_pins_a: stusb1600-0 {
pins {
- pinmux = <STM32_PINMUX('I', 11, ANALOG)>;
+ pinmux = <STM32_PINMUX('I', 11, GPIO)>;
bias-pull-up;
};
};
@@ -1737,20 +1737,20 @@ pins2 {
};
uart4_idle_pins_a: uart4-idle-0 {
- pins1 {
- pinmux = <STM32_PINMUX('G', 11, ANALOG)>; /* UART4_TX */
- };
- pins2 {
- pinmux = <STM32_PINMUX('B', 2, AF8)>; /* UART4_RX */
- bias-disable;
- };
+ pins1 {
+ pinmux = <STM32_PINMUX('G', 11, ANALOG)>; /* UART4_TX */
+ };
+ pins2 {
+ pinmux = <STM32_PINMUX('B', 2, AF8)>; /* UART4_RX */
+ bias-disable;
+ };
};
uart4_sleep_pins_a: uart4-sleep-0 {
- pins {
+ pins {
pinmux = <STM32_PINMUX('G', 11, ANALOG)>, /* UART4_TX */
<STM32_PINMUX('B', 2, ANALOG)>; /* UART4_RX */
- };
+ };
};
uart4_pins_b: uart4-1 {
@@ -1816,7 +1816,7 @@ pins1 {
};
pins2 {
pinmux = <STM32_PINMUX('E', 7, AF7)>; /* UART7_RX */
- bias-disable;
+ bias-pull-up;
};
};
@@ -1826,7 +1826,7 @@ pins1 {
};
pins2 {
pinmux = <STM32_PINMUX('E', 7, AF7)>; /* UART7_RX */
- bias-disable;
+ bias-pull-up;
};
};
@@ -1971,7 +1971,7 @@ pins1 {
pins2 {
pinmux = <STM32_PINMUX('B', 12, AF8)>, /* USART3_RX */
<STM32_PINMUX('I', 10, AF8)>; /* USART3_CTS_NSS */
- bias-disable;
+ bias-pull-up;
};
};
@@ -1988,7 +1988,7 @@ pins2 {
};
pins3 {
pinmux = <STM32_PINMUX('B', 12, AF8)>; /* USART3_RX */
- bias-disable;
+ bias-pull-up;
};
};
@@ -2012,7 +2012,7 @@ pins1 {
pins2 {
pinmux = <STM32_PINMUX('B', 12, AF8)>, /* USART3_RX */
<STM32_PINMUX('B', 13, AF7)>; /* USART3_CTS_NSS */
- bias-disable;
+ bias-pull-up;
};
};
@@ -2029,7 +2029,7 @@ pins2 {
};
pins3 {
pinmux = <STM32_PINMUX('B', 12, AF8)>; /* USART3_RX */
- bias-disable;
+ bias-pull-up;
};
};
diff --git a/arch/arm/boot/dts/stm32mp157a-icore-stm32mp1-ctouch2-of10.dts b/arch/arm/boot/dts/stm32mp157a-icore-stm32mp1-ctouch2-of10.dts
new file mode 100644
index 0000000..2a282928
--- /dev/null
+++ b/arch/arm/boot/dts/stm32mp157a-icore-stm32mp1-ctouch2-of10.dts
@@ -0,0 +1,132 @@
+// SPDX-License-Identifier: (GPL-2.0+ OR BSD-3-Clause)
+/*
+ * Copyright (c) STMicroelectronics 2019 - All Rights Reserved
+ * Copyright (c) 2020 Engicam srl
+ * Copyright (c) 2020 Amarula Solutons(India)
+ */
+
+/dts-v1/;
+#include "stm32mp157.dtsi"
+#include "stm32mp157a-icore-stm32mp1.dtsi"
+#include "stm32mp15-pinctrl.dtsi"
+#include "stm32mp15xxaa-pinctrl.dtsi"
+#include <dt-bindings/gpio/gpio.h>
+
+/ {
+ model = "Engicam i.Core STM32MP1 C.TOUCH 2.0 10.1\" Open Frame";
+ compatible = "engicam,icore-stm32mp1-ctouch2-of10",
+ "engicam,icore-stm32mp1", "st,stm32mp157";
+
+ aliases {
+ serial0 = &uart4;
+ };
+
+ backlight: backlight {
+ compatible = "gpio-backlight";
+ gpios = <&gpiod 13 GPIO_ACTIVE_HIGH>;
+ default-on;
+ };
+
+ chosen {
+ stdout-path = "serial0:115200n8";
+ };
+
+ panel {
+ compatible = "ampire,am-1280800n3tzqw-t00h";
+ backlight = <&backlight>;
+ power-supply = <&v3v3>;
+
+ port {
+ panel_in_lvds: endpoint {
+ remote-endpoint = <&bridge_out>;
+ };
+ };
+ };
+};
+
+&dsi {
+ status = "okay";
+ phy-dsi-supply = <®18>;
+
+ ports {
+ port@0 {
+ reg = <0>;
+ dsi_in: endpoint {
+ remote-endpoint = <<dc_ep0_out>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+ dsi_out: endpoint {
+ remote-endpoint = <&bridge_in>;
+ };
+ };
+ };
+};
+
+&i2c6 {
+ i2c-scl-falling-time-ns = <20>;
+ i2c-scl-rising-time-ns = <185>;
+ pinctrl-names = "default", "sleep";
+ pinctrl-0 = <&i2c6_pins_a>;
+ pinctrl-1 = <&i2c6_sleep_pins_a>;
+ status = "okay";
+
+ bridge@2c {
+ compatible = "ti,sn65dsi84";
+ reg = <0x2c>;
+ enable-gpios = <&gpiof 15 GPIO_ACTIVE_HIGH>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ bridge_in: endpoint {
+ remote-endpoint = <&dsi_out>;
+ data-lanes = <1 2>;
+ };
+ };
+
+ port@2 {
+ reg = <2>;
+ bridge_out: endpoint {
+ remote-endpoint = <&panel_in_lvds>;
+ };
+ };
+ };
+ };
+};
+
+<dc {
+ status = "okay";
+
+ port {
+ ltdc_ep0_out: endpoint@0 {
+ reg = <0>;
+ remote-endpoint = <&dsi_in>;
+ };
+ };
+};
+
+&sdmmc1 {
+ bus-width = <4>;
+ disable-wp;
+ pinctrl-names = "default", "opendrain", "sleep";
+ pinctrl-0 = <&sdmmc1_b4_pins_a>;
+ pinctrl-1 = <&sdmmc1_b4_od_pins_a>;
+ pinctrl-2 = <&sdmmc1_b4_sleep_pins_a>;
+ st,neg-edge;
+ vmmc-supply = <&v3v3>;
+ status = "okay";
+};
+
+&uart4 {
+ pinctrl-names = "default", "sleep", "idle";
+ pinctrl-0 = <&uart4_pins_a>;
+ pinctrl-1 = <&uart4_sleep_pins_a>;
+ pinctrl-2 = <&uart4_idle_pins_a>;
+ status = "okay";
+};
diff --git a/arch/arm/boot/dts/stm32mp157a-icore-stm32mp1-edimm2.2.dts b/arch/arm/boot/dts/stm32mp157a-icore-stm32mp1-edimm2.2.dts
index ec9f1d1c..a797eaa 100644
--- a/arch/arm/boot/dts/stm32mp157a-icore-stm32mp1-edimm2.2.dts
+++ b/arch/arm/boot/dts/stm32mp157a-icore-stm32mp1-edimm2.2.dts
@@ -24,6 +24,91 @@ aliases {
chosen {
stdout-path = "serial0:115200n8";
};
+
+ backlight: backlight {
+ compatible = "gpio-backlight";
+ gpios = <&gpiod 13 GPIO_ACTIVE_HIGH>;
+ default-on;
+ };
+
+ panel {
+ compatible = "yes-optoelectronics,ytc700tlag-05-201c";
+ backlight = <&backlight>;
+ power-supply = <&v3v3>;
+
+ port {
+ panel_out_bridge: endpoint {
+ remote-endpoint = <&bridge_out_panel>;
+ };
+ };
+ };
+};
+
+&dsi {
+ status = "okay";
+ phy-dsi-supply = <®18>;
+
+ ports {
+ port@0 {
+ reg = <0>;
+ dsi_in_ltdc: endpoint {
+ remote-endpoint = <<dc_out_dsi>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+ dsi_out_bridge: endpoint {
+ remote-endpoint = <&bridge_in_dsi>;
+ };
+ };
+ };
+};
+
+&i2c6 {
+ i2c-scl-falling-time-ns = <20>;
+ i2c-scl-rising-time-ns = <185>;
+ pinctrl-names = "default", "sleep";
+ pinctrl-0 = <&i2c6_pins_a>;
+ pinctrl-1 = <&i2c6_sleep_pins_a>;
+ status = "okay";
+
+ bridge@2c {
+ compatible = "ti,sn65dsi84";
+ reg = <0x2c>;
+ enable-gpios = <&gpiof 15 GPIO_ACTIVE_HIGH>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ bridge_in_dsi: endpoint {
+ remote-endpoint = <&dsi_out_bridge>;
+ data-lanes = <1 2>;
+ };
+ };
+
+ port@2 {
+ reg = <2>;
+ bridge_out_panel: endpoint {
+ remote-endpoint = <&panel_out_bridge>;
+ };
+ };
+ };
+ };
+};
+
+<dc {
+ status = "okay";
+
+ port {
+ ltdc_out_dsi: endpoint@0 {
+ reg = <0>;
+ remote-endpoint = <&dsi_in_ltdc>;
+ };
+ };
};
&sdmmc1 {
diff --git a/arch/arm/boot/dts/stm32mp157c-ev1.dts b/arch/arm/boot/dts/stm32mp157c-ev1.dts
index 5c5b1dd..e222d2d 100644
--- a/arch/arm/boot/dts/stm32mp157c-ev1.dts
+++ b/arch/arm/boot/dts/stm32mp157c-ev1.dts
@@ -375,3 +375,25 @@ &usbotg_hs {
&usbphyc {
status = "okay";
};
+
+&usbphyc_port0 {
+ st,tune-hs-dc-level = <2>;
+ st,enable-fs-rftime-tuning;
+ st,enable-hs-rftime-reduction;
+ st,trim-hs-current = <15>;
+ st,trim-hs-impedance = <1>;
+ st,tune-squelch-level = <3>;
+ st,tune-hs-rx-offset = <2>;
+ st,no-lsfs-sc;
+};
+
+&usbphyc_port1 {
+ st,tune-hs-dc-level = <2>;
+ st,enable-fs-rftime-tuning;
+ st,enable-hs-rftime-reduction;
+ st,trim-hs-current = <15>;
+ st,trim-hs-impedance = <1>;
+ st,tune-squelch-level = <3>;
+ st,tune-hs-rx-offset = <2>;
+ st,no-lsfs-sc;
+};
diff --git a/arch/arm/boot/dts/stm32mp15xx-dkx.dtsi b/arch/arm/boot/dts/stm32mp15xx-dkx.dtsi
index 48beed0..6caeb44 100644
--- a/arch/arm/boot/dts/stm32mp15xx-dkx.dtsi
+++ b/arch/arm/boot/dts/stm32mp15xx-dkx.dtsi
@@ -694,10 +694,26 @@ &usbphyc {
&usbphyc_port0 {
phy-supply = <&vdd_usb>;
+ st,tune-hs-dc-level = <2>;
+ st,enable-fs-rftime-tuning;
+ st,enable-hs-rftime-reduction;
+ st,trim-hs-current = <15>;
+ st,trim-hs-impedance = <1>;
+ st,tune-squelch-level = <3>;
+ st,tune-hs-rx-offset = <2>;
+ st,no-lsfs-sc;
};
&usbphyc_port1 {
phy-supply = <&vdd_usb>;
+ st,tune-hs-dc-level = <2>;
+ st,enable-fs-rftime-tuning;
+ st,enable-hs-rftime-reduction;
+ st,trim-hs-current = <15>;
+ st,trim-hs-impedance = <1>;
+ st,tune-squelch-level = <3>;
+ st,tune-hs-rx-offset = <2>;
+ st,no-lsfs-sc;
};
&vrefbuf {
diff --git a/arch/arm/boot/dts/sun8i-h2-plus-bananapi-m2-zero.dts b/arch/arm/boot/dts/sun8i-h2-plus-bananapi-m2-zero.dts
index 8e8634ff2..d5c7b79 100644
--- a/arch/arm/boot/dts/sun8i-h2-plus-bananapi-m2-zero.dts
+++ b/arch/arm/boot/dts/sun8i-h2-plus-bananapi-m2-zero.dts
@@ -52,8 +52,9 @@ gpio_keys {
sw4 {
label = "power";
- linux,code = <BTN_0>;
+ linux,code = <KEY_POWER>;
gpios = <&r_pio 0 3 GPIO_ACTIVE_LOW>;
+ wakeup-source;
};
};
diff --git a/arch/arm/boot/dts/sun8i-h3-beelink-x2.dts b/arch/arm/boot/dts/sun8i-h3-beelink-x2.dts
index f0e591e..cd9f655 100644
--- a/arch/arm/boot/dts/sun8i-h3-beelink-x2.dts
+++ b/arch/arm/boot/dts/sun8i-h3-beelink-x2.dts
@@ -57,6 +57,12 @@ aliases {
ethernet1 = &sdiowifi;
};
+ cec-gpio {
+ compatible = "cec-gpio";
+ cec-gpios = <&pio 0 14 (GPIO_ACTIVE_HIGH|GPIO_OPEN_DRAIN)>; /* PA14 */
+ hdmi-phandle = <&hdmi>;
+ };
+
chosen {
stdout-path = "serial0:115200n8";
};
@@ -87,11 +93,15 @@ led-1 {
};
};
- wifi_pwrseq: wifi_pwrseq {
- compatible = "mmc-pwrseq-simple";
- reset-gpios = <&r_pio 0 7 GPIO_ACTIVE_LOW>; /* PL7 */
- clocks = <&rtc 1>;
- clock-names = "ext_clock";
+ r-gpio-keys {
+ compatible = "gpio-keys";
+
+ power {
+ label = "power";
+ linux,code = <KEY_POWER>;
+ gpios = <&r_pio 0 3 GPIO_ACTIVE_LOW>;
+ wakeup-source;
+ };
};
sound_spdif {
@@ -112,15 +122,11 @@ spdif_out: spdif-out {
compatible = "linux,spdif-dit";
};
- r-gpio-keys {
- compatible = "gpio-keys";
-
- power {
- label = "power";
- linux,code = <KEY_POWER>;
- gpios = <&r_pio 0 3 GPIO_ACTIVE_LOW>;
- wakeup-source;
- };
+ wifi_pwrseq: wifi_pwrseq {
+ compatible = "mmc-pwrseq-simple";
+ reset-gpios = <&r_pio 0 7 GPIO_ACTIVE_LOW>; /* PL7 */
+ clocks = <&rtc 1>;
+ clock-names = "ext_clock";
};
};
diff --git a/arch/arm/boot/dts/sun8i-h3-nanopi.dtsi b/arch/arm/boot/dts/sun8i-h3-nanopi.dtsi
index c7c3e7d..fc45d5a 100644
--- a/arch/arm/boot/dts/sun8i-h3-nanopi.dtsi
+++ b/arch/arm/boot/dts/sun8i-h3-nanopi.dtsi
@@ -81,6 +81,7 @@ k1 {
label = "k1";
linux,code = <KEY_POWER>;
gpios = <&r_pio 0 3 GPIO_ACTIVE_LOW>;
+ wakeup-source;
};
};
};
diff --git a/arch/arm/boot/dts/sun8i-h3-orangepi-2.dts b/arch/arm/boot/dts/sun8i-h3-orangepi-2.dts
index 597c425..9daffd9 100644
--- a/arch/arm/boot/dts/sun8i-h3-orangepi-2.dts
+++ b/arch/arm/boot/dts/sun8i-h3-orangepi-2.dts
@@ -99,8 +99,9 @@ sw2 {
sw4 {
label = "sw4";
- linux,code = <BTN_0>;
+ linux,code = <KEY_POWER>;
gpios = <&r_pio 0 3 GPIO_ACTIVE_LOW>;
+ wakeup-source;
};
};
diff --git a/arch/arm/boot/dts/sun8i-h3-orangepi-pc.dts b/arch/arm/boot/dts/sun8i-h3-orangepi-pc.dts
index 5aff8ec..90f75fa 100644
--- a/arch/arm/boot/dts/sun8i-h3-orangepi-pc.dts
+++ b/arch/arm/boot/dts/sun8i-h3-orangepi-pc.dts
@@ -91,8 +91,9 @@ r_gpio_keys {
sw4 {
label = "sw4";
- linux,code = <BTN_0>;
+ linux,code = <KEY_POWER>;
gpios = <&r_pio 0 3 GPIO_ACTIVE_LOW>;
+ wakeup-source;
};
};
};
diff --git a/arch/arm/boot/dts/sun8i-h3.dtsi b/arch/arm/boot/dts/sun8i-h3.dtsi
index ae4f933..845f2523 100644
--- a/arch/arm/boot/dts/sun8i-h3.dtsi
+++ b/arch/arm/boot/dts/sun8i-h3.dtsi
@@ -282,6 +282,10 @@ &display_clocks {
compatible = "allwinner,sun8i-h3-de2-clk";
};
+&mbus {
+ compatible = "allwinner,sun8i-h3-mbus";
+};
+
&mmc0 {
compatible = "allwinner,sun7i-a20-mmc";
clocks = <&ccu CLK_BUS_MMC0>,
diff --git a/arch/arm/boot/dts/sun8i-r40.dtsi b/arch/arm/boot/dts/sun8i-r40.dtsi
index 1d87fc0..03d3e5f 100644
--- a/arch/arm/boot/dts/sun8i-r40.dtsi
+++ b/arch/arm/boot/dts/sun8i-r40.dtsi
@@ -511,6 +511,16 @@ pio: pinctrl@1c20800 {
#interrupt-cells = <3>;
#gpio-cells = <3>;
+ can_ph_pins: can-ph-pins {
+ pins = "PH20", "PH21";
+ function = "can";
+ };
+
+ can_pa_pins: can-pa-pins {
+ pins = "PA16", "PA17";
+ function = "can";
+ };
+
clk_out_a_pin: clk-out-a-pin {
pins = "PI12";
function = "clk_out_a";
@@ -926,6 +936,15 @@ i2c3: i2c@1c2b800 {
#size-cells = <0>;
};
+ can0: can@1c2bc00 {
+ compatible = "allwinner,sun8i-r40-can";
+ reg = <0x01c2bc00 0x400>;
+ interrupts = <GIC_SPI 26 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&ccu CLK_BUS_CAN>;
+ resets = <&ccu RST_BUS_CAN>;
+ status = "disabled";
+ };
+
i2c4: i2c@1c2c000 {
compatible = "allwinner,sun6i-a31-i2c";
reg = <0x01c2c000 0x400>;
@@ -1212,8 +1231,8 @@ hdmi: hdmi@1ee0000 {
reg-io-width = <1>;
interrupts = <GIC_SPI 58 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&ccu CLK_BUS_HDMI0>, <&ccu CLK_HDMI_SLOW>,
- <&ccu CLK_HDMI>;
- clock-names = "iahb", "isfr", "tmds";
+ <&ccu CLK_HDMI>, <&rtc 0>;
+ clock-names = "iahb", "isfr", "tmds", "cec";
resets = <&ccu RST_BUS_HDMI1>;
reset-names = "ctrl";
phys = <&hdmi_phy>;
diff --git a/arch/arm/boot/dts/sunxi-bananapi-m2-plus.dtsi b/arch/arm/boot/dts/sunxi-bananapi-m2-plus.dtsi
index 7a6af54..d03f585 100644
--- a/arch/arm/boot/dts/sunxi-bananapi-m2-plus.dtsi
+++ b/arch/arm/boot/dts/sunxi-bananapi-m2-plus.dtsi
@@ -82,8 +82,9 @@ gpio_keys {
sw4 {
label = "power";
- linux,code = <BTN_0>;
+ linux,code = <KEY_POWER>;
gpios = <&r_pio 0 3 GPIO_ACTIVE_LOW>;
+ wakeup-source;
};
};
diff --git a/arch/arm/boot/dts/sunxi-h3-h5.dtsi b/arch/arm/boot/dts/sunxi-h3-h5.dtsi
index c7428df..4aeca9e 100644
--- a/arch/arm/boot/dts/sunxi-h3-h5.dtsi
+++ b/arch/arm/boot/dts/sunxi-h3-h5.dtsi
@@ -568,9 +568,14 @@ external_mdio: mdio@2 {
};
mbus: dram-controller@1c62000 {
- compatible = "allwinner,sun8i-h3-mbus";
- reg = <0x01c62000 0x1000>;
- clocks = <&ccu CLK_MBUS>;
+ /* compatible is in per SoC .dtsi file */
+ reg = <0x01c62000 0x1000>,
+ <0x01c63000 0x1000>;
+ reg-names = "mbus", "dram";
+ clocks = <&ccu CLK_MBUS>,
+ <&ccu CLK_DRAM>,
+ <&ccu CLK_BUS_DRAM>;
+ clock-names = "mbus", "dram", "bus";
#address-cells = <1>;
#size-cells = <1>;
dma-ranges = <0x00000000 0x40000000 0xc0000000>;
@@ -813,8 +818,8 @@ hdmi: hdmi@1ee0000 {
reg-io-width = <1>;
interrupts = <GIC_SPI 88 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&ccu CLK_BUS_HDMI>, <&ccu CLK_HDMI_DDC>,
- <&ccu CLK_HDMI>;
- clock-names = "iahb", "isfr", "tmds";
+ <&ccu CLK_HDMI>, <&rtc 0>;
+ clock-names = "iahb", "isfr", "tmds", "cec";
resets = <&ccu RST_BUS_HDMI1>;
reset-names = "ctrl";
phys = <&hdmi_phy>;
diff --git a/arch/arm/boot/dts/sunxi-libretech-all-h3-cc.dtsi b/arch/arm/boot/dts/sunxi-libretech-all-h3-cc.dtsi
index c44fd72..9e14fe5 100644
--- a/arch/arm/boot/dts/sunxi-libretech-all-h3-cc.dtsi
+++ b/arch/arm/boot/dts/sunxi-libretech-all-h3-cc.dtsi
@@ -49,6 +49,7 @@ power {
label = "power";
linux,code = <KEY_POWER>;
gpios = <&r_pio 0 2 GPIO_ACTIVE_LOW>; /* PL2 */
+ wakeup-source;
};
};
diff --git a/arch/arm/boot/dts/tegra114-asus-tf701t.dts b/arch/arm/boot/dts/tegra114-asus-tf701t.dts
new file mode 100644
index 0000000..b791ce9
--- /dev/null
+++ b/arch/arm/boot/dts/tegra114-asus-tf701t.dts
@@ -0,0 +1,807 @@
+// SPDX-License-Identifier: GPL-2.0
+
+/dts-v1/;
+
+#include <dt-bindings/input/gpio-keys.h>
+#include <dt-bindings/input/input.h>
+
+#include "tegra114.dtsi"
+
+/ {
+ model = "Asus Transformer Pad TF701T";
+ compatible = "asus,tf701t", "nvidia,tegra114";
+ chassis-type = "convertible";
+
+ aliases {
+ mmc0 = "/mmc@78000600"; /* eMMC */
+ mmc1 = "/mmc@78000400"; /* uSD slot */
+ mmc2 = "/mmc@78000000"; /* WiFi */
+
+ rtc0 = &palmas;
+ rtc1 = "/rtc@7000e000";
+
+ serial0 = &uartd; /* Console */
+ serial1 = &uartc; /* Bluetooth */
+ serial2 = &uartb; /* GPS */
+ };
+
+ memory@80000000 {
+ reg = <0x80000000 0x80000000>;
+ };
+
+ reserved-memory {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges;
+
+ linux,cma@80000000 {
+ compatible = "shared-dma-pool";
+ alloc-ranges = <0x80000000 0x30000000>;
+ size = <0x10000000>;
+ linux,cma-default;
+ reusable;
+ };
+
+ trustzone@bfe00000 {
+ reg = <0xbfe00000 0x200000>;
+ no-map;
+ };
+ };
+
+ host1x@50000000 {
+ dsi@54300000 {
+ status = "okay";
+
+ avdd-dsi-csi-supply = <&tps65913_ldo2>;
+
+ nvidia,ganged-mode = <&dsib>;
+
+ panel_primary: panel@0 {
+ compatible = "sharp,lq101r1sx01";
+ reg = <0>;
+
+ link2 = <&panel_secondary>;
+
+ power-supply = <&vdd_lcd>;
+ backlight = <&backlight>;
+ };
+ };
+
+ dsi@54400000 {
+ status = "okay";
+
+ avdd-dsi-csi-supply = <&tps65913_ldo2>;
+
+ panel_secondary: panel@0 {
+ compatible = "sharp,lq101r1sx01";
+ reg = <0>;
+ };
+ };
+ };
+
+ pinmux@70000868 {
+ asus_pad_ec_default: asus-pad-ec-default {
+ ec-interrupt {
+ nvidia,pins = "kb_col5_pq5";
+ nvidia,function = "kbc";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ ec-request {
+ nvidia,pins = "kb_col2_pq2";
+ nvidia,function = "kbc";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+ };
+
+ backlight_default: backlight-default {
+ backlight-enable {
+ nvidia,pins = "gmi_ad10_ph2";
+ nvidia,function = "gmi";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+ };
+
+ codec_default: codec-default {
+ ldo1-en {
+ nvidia,pins = "sdmmc1_wp_n_pv3";
+ nvidia,function = "sdmmc1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ interrupt {
+ nvidia,pins = "gpio_w2_aud_pw2",
+ "gpio_w3_aud_pw3";
+ nvidia,function = "spi6";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+ };
+
+ gpio_keys_default: gpio-keys-default {
+ power {
+ nvidia,pins = "kb_col0_pq0";
+ nvidia,function = "kbc";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ volume {
+ nvidia,pins = "kb_row1_pr1",
+ "kb_row2_pr2";
+ nvidia,function = "rsvd2";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+ };
+
+ gpio_hall_sensor_default: gpio-hall-sensor-default {
+ ulpi_data4_po5 {
+ nvidia,pins = "ulpi_data4_po5";
+ nvidia,function = "spi2";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+ };
+
+ hp_det_default: hp-det-default {
+ gmi_iordy_pi5 {
+ nvidia,pins = "kb_row7_pr7";
+ nvidia,function = "rsvd2";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+ };
+
+ imu_default: imu-default {
+ kb_row3_pr3 {
+ nvidia,pins = "kb_row3_pr3";
+ nvidia,function = "rsvd3";
+ nvidia,pull = <TEGRA_PIN_PULL_DOWN>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+ };
+
+ pwm_default: pwm-default {
+ gmi_ad9_ph1 {
+ nvidia,pins = "gmi_ad9_ph1";
+ nvidia,function = "pwm1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+ };
+
+ /* XXX make this something more sensible */
+ pwm_sleep: pwm-sleep {
+ gmi_ad9_ph1 {
+ nvidia,pins = "gmi_ad9_ph1";
+ nvidia,function = "pwm1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+ };
+
+ sdmmc3_default: sdmmc3-default {
+ sdmmc3_clk_pa6 {
+ nvidia,pins = "sdmmc3_clk_pa6";
+ nvidia,function = "sdmmc3";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ sdmmc3_cmd_pa7 {
+ nvidia,pins = "sdmmc3_cmd_pa7",
+ "sdmmc3_dat0_pb7",
+ "sdmmc3_dat1_pb6",
+ "sdmmc3_dat2_pb5",
+ "sdmmc3_dat3_pb4",
+ "kb_col4_pq4",
+ "sdmmc3_clk_lb_out_pee4",
+ "sdmmc3_clk_lb_in_pee5",
+ "sdmmc3_cd_n_pv2";
+ nvidia,function = "sdmmc3";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ drive_sdio3 {
+ nvidia,pins = "drive_sdio3";
+ nvidia,high-speed-mode = <TEGRA_PIN_ENABLE>;
+ nvidia,schmitt = <TEGRA_PIN_DISABLE>;
+ nvidia,pull-down-strength = <22>;
+ nvidia,pull-up-strength = <36>;
+ nvidia,slew-rate-rising = <TEGRA_PIN_SLEW_RATE_FASTEST>;
+ nvidia,slew-rate-falling = <TEGRA_PIN_SLEW_RATE_FASTEST>;
+ };
+ };
+
+ sdmmc3_vdd_default: sdmmc3-vdd-default {
+ gmi_clk_pk1 {
+ nvidia,pins = "gmi_clk_pk1";
+ nvidia,function = "gmi";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+ };
+
+ vdd_lcd_default: vdd-lcd-default {
+ sdmmc4_clk_pcc4 {
+ nvidia,pins = "sdmmc4_clk_pcc4";
+ nvidia,function = "sdmmc4";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+ };
+ };
+
+ serial@70006040 {
+ /* GPS */
+ };
+
+ serial@70006200 {
+ /* Bluetooth */
+ };
+
+ serial@70006300 {
+ status = "okay";
+ };
+
+ pwm@7000a000 {
+ status = "okay";
+
+ pinctrl-names = "default", "sleep";
+ pinctrl-0 = <&pwm_default>;
+ pinctrl-1 = <&pwm_sleep>;
+ };
+
+ i2c@7000c000 {
+ status = "okay";
+ clock-frequency = <100000>;
+
+ magnetometer@c {
+ compatible = "asahi-kasei,ak09911";
+ reg = <0xc>;
+
+ vdd-supply = <&vdd_3v3_sys>;
+ };
+
+ rt5639: audio-codec@1c {
+ compatible = "realtek,rt5639";
+ reg = <0x1c>;
+
+ interrupt-parent = <&gpio>;
+ interrupts = <TEGRA_GPIO(W, 3) IRQ_TYPE_EDGE_FALLING>;
+
+ realtek,ldo1-en-gpios = <&gpio TEGRA_GPIO(V, 3) GPIO_ACTIVE_HIGH>;
+
+ pinctrl-names = "default";
+ pinctrl-0 = <&codec_default>;
+ };
+
+ motion-tracker@68 {
+ compatible = "invensense,mpu6500";
+ reg = <0x68>;
+
+ interrupt-parent = <&gpio>;
+ interrupts = <TEGRA_GPIO(R, 3) IRQ_TYPE_LEVEL_HIGH>;
+
+ mount-matrix = "0", "-1", "0",
+ "1", "0", "0",
+ "0", "0", "1";
+
+ pinctrl-names = "default";
+ pinctrl-0 = <&imu_default>;
+ };
+
+ temp_sensor: temperature-sensor@4c {
+ compatible = "onnn,nct1008";
+ reg = <0x4c>;
+
+ vcc-supply = <&vdd_3v3_sys>;
+ #thermal-sensor-cells = <1>;
+ };
+ };
+
+ i2c@7000c400 {
+ status = "okay";
+ clock-frequency = <100000>;
+
+ power-sensor@44 {
+ compatible = "ti,ina230";
+ reg = <0x44>;
+ };
+ };
+
+ i2c@7000c500 {
+ status = "okay";
+ clock-frequency = <400000>;
+
+ light-sensor@1c {
+ compatible = "dynaimage,al3320a";
+ reg = <0x1c>;
+
+ vdd-supply = <&vdd_3v3_sys>;
+ };
+ };
+
+ i2c@7000c700 {
+ /* HDMI DDC */
+ };
+
+ i2c@7000d000 {
+ status = "okay";
+ clock-frequency = <400000>;
+
+ palmas: pmic@58 {
+ compatible = "ti,tps65913", "ti,palmas";
+ reg = <0x58>;
+ interrupts = <GIC_SPI 86 IRQ_TYPE_LEVEL_HIGH>;
+
+ #interrupt-cells = <2>;
+ interrupt-controller;
+
+ ti,system-power-controller;
+
+ palmas_gpio: gpio {
+ compatible = "ti,palmas-gpio";
+ gpio-controller;
+ #gpio-cells = <2>;
+ };
+
+ pmic {
+ compatible = "ti,tps65913-pmic", "ti,palmas-pmic";
+
+ ldo1-in-supply = <&tps65913_smps7>;
+ ldo2-in-supply = <&tps65913_smps7>;
+ ldo4-in-supply = <&tps65913_smps8>;
+ ldo5-in-supply = <&tps65913_smps9>;
+ ldo6-in-supply = <&tps65913_smps9>;
+ ldo7-in-supply = <&tps65913_smps9>;
+ ldo9-in-supply = <&tps65913_smps9>;
+
+ regulators {
+ tps65913_smps123: smps123 {
+ regulator-name = "vdd-cpu";
+ regulator-min-microvolt = <900000>;
+ regulator-max-microvolt = <1350000>;
+ regulator-always-on;
+ regulator-boot-on;
+ ti,roof-floor = <1>;
+ ti,mode-sleep = <3>;
+ };
+
+ tps65913_smps45: smps45 {
+ regulator-name = "vdd-core";
+ regulator-min-microvolt = <900000>;
+ regulator-max-microvolt = <1400000>;
+ regulator-always-on;
+ regulator-boot-on;
+ ti,roof-floor = <3>;
+ };
+
+ smps6 {
+ regulator-name = "va-lcd-hv";
+ regulator-min-microvolt = <1000000>;
+ regulator-max-microvolt = <1000000>;
+ regulator-always-on;
+ regulator-boot-on;
+ };
+
+ tps65913_smps7: smps7 {
+ regulator-name = "vdd-ddr";
+ regulator-min-microvolt = <1350000>;
+ regulator-max-microvolt = <1350000>;
+ regulator-always-on;
+ regulator-boot-on;
+ };
+
+ tps65913_smps8: smps8 {
+ regulator-name = "vdd-1v8";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ regulator-always-on;
+ regulator-boot-on;
+ };
+
+ tps65913_smps9: smps9 {
+ regulator-name = "vdd-sd";
+ regulator-min-microvolt = <2900000>;
+ regulator-max-microvolt = <2900000>;
+ regulator-always-on;
+ };
+
+ tps65913_smps10_out1: smps10_out1 {
+ regulator-name = "vd-smps10-out1";
+ regulator-min-microvolt = <5000000>;
+ regulator-max-microvolt = <5000000>;
+ regulator-always-on;
+ regulator-boot-on;
+ };
+
+ tps65913_smps10_out2: smps10_out2 {
+ regulator-name = "vd-smps10-out2";
+ regulator-min-microvolt = <5000000>;
+ regulator-max-microvolt = <5000000>;
+ regulator-always-on;
+ regulator-boot-on;
+ };
+
+ tps65913_ldo1: ldo1 {
+ regulator-name = "vdd-hdmi-pll";
+ regulator-min-microvolt = <1050000>;
+ regulator-max-microvolt = <1050000>;
+ regulator-always-on;
+ ti,roof-floor = <3>;
+ };
+
+ tps65913_ldo2: ldo2 {
+ regulator-name = "vdd-2v8-dsi-csi";
+ regulator-min-microvolt = <1200000>;
+ regulator-max-microvolt = <1200000>;
+ regulator-boot-on;
+ };
+
+ ldo3 {
+ regulator-name = "vpp-fuse";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ };
+
+ ldo4 {
+ regulator-name = "vdd-1v2-cam";
+ regulator-min-microvolt = <1200000>;
+ regulator-max-microvolt = <1200000>;
+ };
+
+ ldo5 {
+ regulator-name = "vdd-cam";
+ regulator-min-microvolt = <2800000>;
+ regulator-max-microvolt = <2800000>;
+ };
+
+ ldo6 {
+ regulator-name = "vdd-dev";
+ regulator-min-microvolt = <2850000>;
+ regulator-max-microvolt = <2850000>;
+ regulator-boot-on;
+ };
+
+ ldo7 {
+ regulator-name = "vdd-2v8-cam";
+ regulator-min-microvolt = <2800000>;
+ regulator-max-microvolt = <2800000>;
+ };
+
+ tps65913_ldo8: ldo8 {
+ regulator-name = "vdd-rtc";
+ regulator-min-microvolt = <950000>;
+ regulator-max-microvolt = <950000>;
+ regulator-always-on;
+ regulator-boot-on;
+ ti,enable-ldo8-tracking;
+ };
+
+ tps65913_ldo9: ldo9 {
+ regulator-name = "vdd-sdmmc";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <2900000>;
+ };
+
+ tps65913_ldoln: ldoln {
+ regulator-name = "vdd-hdmi";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ };
+
+ ldousb {
+ regulator-name = "vdd-usb";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-always-on;
+ regulator-boot-on;
+ };
+ };
+ };
+
+ rtc {
+ compatible = "ti,palmas-rtc";
+ interrupt-parent = <&palmas>;
+ interrupts = <8 0>;
+ };
+
+ pinmux {
+ compatible = "ti,tps65913-pinctrl";
+ ti,palmas-enable-dvfs1;
+
+ pinctrl-names = "default";
+ pinctrl-0 = <&palmas_default>;
+
+ palmas_default: pinmux {
+ pin_powergood {
+ pins = "powergood";
+ function = "powergood";
+ };
+
+ pin_vac {
+ pins = "vac";
+ function = "vac";
+ };
+
+ pin_gpio0 {
+ pins = "gpio0";
+ function = "gpio";
+ };
+
+ pin_gpio1 {
+ pins = "gpio1";
+ function = "gpio";
+ };
+
+ pin_gpio2 {
+ pins = "gpio2";
+ function = "gpio";
+ };
+
+ pin_gpio3 {
+ pins = "gpio3";
+ function = "gpio";
+ };
+
+ pin_gpio4 {
+ pins = "gpio4";
+ function = "gpio";
+ };
+
+ pin_gpio5 {
+ pins = "gpio5";
+ function = "gpio";
+ };
+
+ pin_gpio6 {
+ pins = "gpio6";
+ function = "gpio";
+ };
+
+ pin_gpio7 {
+ pins = "gpio7";
+ function = "gpio";
+ };
+ };
+ };
+ };
+ };
+
+ ahub@70080000 {
+ i2s@70080300 {
+ status = "okay";
+ };
+ };
+
+ mmc@78000000 {
+ /* WiFi */
+ };
+
+ /* MicroSD card */
+ mmc@78000400 {
+ status = "okay";
+
+ bus-width = <4>;
+ cd-gpios = <&gpio TEGRA_GPIO(V, 2) GPIO_ACTIVE_LOW>;
+
+ nvidia,default-tap = <0x3>;
+ nvidia,default-trim = <0x3>;
+
+ vmmc-supply = <&vdd_usd>;
+ vqmmc-supply = <&tps65913_ldo9>;
+
+ pinctrl-names = "default";
+ pinctrl-0 = <&sdmmc3_default>;
+ };
+
+ mmc@78000600 {
+ /* eMMC */
+ };
+
+ usb@7d000000 {
+ compatible = "nvidia,tegra114-udc";
+ status = "okay";
+ dr_mode = "peripheral";
+
+ /* Peripheral USB via ASUS connector */
+ };
+
+ usb-phy@7d000000 {
+ status = "okay";
+ };
+
+ usb@7d008000 {
+ status = "okay";
+
+ /* Host USB via dock */
+ };
+
+ usb-phy@7d008000 {
+ status = "okay";
+ vbus-supply = <&vdd_5v0_sys>;
+ };
+
+ backlight: backlight {
+ compatible = "pwm-backlight";
+
+ enable-gpios = <&gpio TEGRA_GPIO(H, 2) GPIO_ACTIVE_HIGH>;
+ power-supply = <&vdd_5v0_sys>;
+ pwms = <&pwm 1 1000000>;
+
+ brightness-levels = <1 255>;
+ num-interpolated-steps = <254>;
+ default-brightness-level = <224>;
+
+ pinctrl-names = "default";
+ pinctrl-0 = <&backlight_default>;
+ };
+
+ /* PMIC has a built-in 32KHz oscillator which is used by PMC */
+ clk32k_in: clock-32k {
+ compatible = "fixed-clock";
+ #clock-cells = <0>;
+ clock-frequency = <32768>;
+ clock-output-names = "pmic-oscillator";
+ };
+
+ firmware {
+ trusted-foundations {
+ compatible = "tlm,trusted-foundations";
+ tlm,version-major = <2>;
+ tlm,version-minor = <8>;
+ };
+ };
+
+ gpio-keys {
+ compatible = "gpio-keys";
+
+ label = "GPIO Buttons";
+
+ pinctrl-names = "default";
+ pinctrl-0 = <&gpio_keys_default>;
+
+ power {
+ label = "Power";
+ gpios = <&gpio TEGRA_GPIO(Q, 0) GPIO_ACTIVE_LOW>;
+ linux,code = <KEY_POWER>;
+ debounce-interval = <10>;
+ wakeup-source;
+ };
+
+ volume-up {
+ label = "Volume Up";
+ gpios = <&gpio TEGRA_GPIO(R, 2) GPIO_ACTIVE_LOW>;
+ linux,code = <KEY_VOLUMEUP>;
+ debounce-interval = <10>;
+ };
+
+ volume-down {
+ label = "Volume Down";
+ gpios = <&gpio TEGRA_GPIO(R, 1) GPIO_ACTIVE_LOW>;
+ linux,code = <KEY_VOLUMEDOWN>;
+ debounce-interval = <10>;
+ };
+ };
+
+ gpio-hall-sensor {
+ compatible = "gpio-keys";
+
+ label = "GPIO Hall Effect Sensor";
+
+ pinctrl-names = "default";
+ pinctrl-0 = <&gpio_hall_sensor_default>;
+
+ hall-sensor {
+ label = "Hall Effect Sensor";
+ gpios = <&gpio TEGRA_GPIO(O, 5) GPIO_ACTIVE_LOW>;
+ linux,input-type = <EV_SW>;
+ linux,code = <SW_LID>;
+ linux,can-disable;
+ wakeup-source;
+ };
+ };
+
+ sound {
+ compatible = "asus,tegra-audio-rt5639-tf701t",
+ "nvidia,tegra-audio-rt5640";
+ nvidia,model = "Asus Transformer Pad TF701T RT5639";
+
+ nvidia,audio-routing =
+ "Headphones", "HPOR",
+ "Headphones", "HPOL",
+ "Speakers", "SPORP",
+ "Speakers", "SPORN",
+ "Speakers", "SPOLP",
+ "Speakers", "SPOLN",
+ "Mic Jack", "MICBIAS1",
+ "IN2P", "Mic Jack";
+
+ nvidia,i2s-controller = <&tegra_i2s0>;
+ nvidia,audio-codec = <&rt5639>;
+
+ nvidia,hp-det-gpios = <&gpio TEGRA_GPIO(R, 7) GPIO_ACTIVE_LOW>;
+
+ clocks = <&tegra_car TEGRA114_CLK_PLL_A>,
+ <&tegra_car TEGRA114_CLK_PLL_A_OUT0>,
+ <&tegra_pmc TEGRA_PMC_CLK_OUT_1>;
+ clock-names = "pll_a", "pll_a_out0", "mclk";
+
+ assigned-clocks = <&tegra_car TEGRA114_CLK_EXTERN1>,
+ <&tegra_pmc TEGRA_PMC_CLK_OUT_1>;
+
+ assigned-clock-parents = <&tegra_car TEGRA114_CLK_PLL_A_OUT0>,
+ <&tegra_car TEGRA114_CLK_EXTERN1>;
+
+ pinctrl-names = "default";
+ pinctrl-0 = <&hp_det_default>;
+ };
+
+ vdd_5v0_sys: regulator-5v0-sys {
+ compatible = "regulator-fixed";
+ regulator-name = "vdd_5v0";
+ regulator-min-microvolt = <5000000>;
+ regulator-max-microvolt = <5000000>;
+ regulator-always-on;
+ regulator-boot-on;
+ };
+
+ vdd_3v3_sys: regulator-3v3-sys {
+ compatible = "regulator-fixed";
+ regulator-name = "vdd_3v3";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-always-on;
+ regulator-boot-on;
+ };
+
+ vdd_lcd: regulator-vdd-lcd {
+ compatible = "regulator-fixed";
+ regulator-name = "vdd_lcd_1v8";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ vin-supply = <&tps65913_smps8>;
+ enable-active-high;
+ gpio = <&gpio TEGRA_GPIO(CC, 4) GPIO_ACTIVE_HIGH>;
+ regulator-boot-on;
+
+ pinctrl-names = "default";
+ pinctrl-0 = <&vdd_lcd_default>;
+ };
+
+ vdd_usd: regulator-vdd-usd {
+ compatible = "regulator-fixed";
+ regulator-name = "vdd_sd_slot";
+ regulator-min-microvolt = <2900000>;
+ regulator-max-microvolt = <2900000>;
+ vin-supply = <&tps65913_smps9>;
+ enable-active-high;
+ gpio = <&gpio TEGRA_GPIO(K, 1) GPIO_ACTIVE_HIGH>;
+
+ pinctrl-names = "default";
+ pinctrl-0 = <&sdmmc3_vdd_default>;
+ };
+};
diff --git a/arch/arm/boot/dts/tegra114-dalmore.dts b/arch/arm/boot/dts/tegra114-dalmore.dts
index 7fd901f..658edfb 100644
--- a/arch/arm/boot/dts/tegra114-dalmore.dts
+++ b/arch/arm/boot/dts/tegra114-dalmore.dts
@@ -894,7 +894,7 @@ ldo2 {
};
palmas: tps65913@58 {
- compatible = "ti,palmas";
+ compatible = "ti,tps65913", "ti,palmas";
reg = <0x58>;
interrupts = <0 86 IRQ_TYPE_LEVEL_HIGH>;
@@ -1084,7 +1084,8 @@ pin_gpio6 {
spi@7000da00 {
status = "okay";
spi-max-frequency = <25000000>;
- spi-flash@0 {
+
+ flash@0 {
compatible = "winbond,w25q32dw", "jedec,spi-nor";
reg = <0>;
spi-max-frequency = <20000000>;
@@ -1151,7 +1152,7 @@ backlight: backlight {
default-brightness-level = <6>;
};
- clk32k_in: clock@0 {
+ clk32k_in: clock-32k {
compatible = "fixed-clock";
clock-frequency = <32768>;
#clock-cells = <0>;
@@ -1186,7 +1187,7 @@ volume_up {
};
};
- vdd_ac_bat_reg: regulator@0 {
+ vdd_ac_bat_reg: regulator-acbat {
compatible = "regulator-fixed";
regulator-name = "vdd_ac_bat";
regulator-min-microvolt = <5000000>;
@@ -1194,7 +1195,7 @@ vdd_ac_bat_reg: regulator@0 {
regulator-always-on;
};
- dvdd_ts_reg: regulator@1 {
+ dvdd_ts_reg: regulator-ts {
compatible = "regulator-fixed";
regulator-name = "dvdd_ts";
regulator-min-microvolt = <1800000>;
@@ -1203,7 +1204,7 @@ dvdd_ts_reg: regulator@1 {
gpio = <&gpio TEGRA_GPIO(H, 5) GPIO_ACTIVE_HIGH>;
};
- usb1_vbus_reg: regulator@3 {
+ usb1_vbus_reg: regulator-usb1 {
compatible = "regulator-fixed";
regulator-name = "usb1_vbus";
regulator-min-microvolt = <5000000>;
@@ -1214,7 +1215,7 @@ usb1_vbus_reg: regulator@3 {
vin-supply = <&tps65090_dcdc1_reg>;
};
- usb3_vbus_reg: regulator@4 {
+ usb3_vbus_reg: regulator-usb3 {
compatible = "regulator-fixed";
regulator-name = "usb2_vbus";
regulator-min-microvolt = <5000000>;
@@ -1225,7 +1226,7 @@ usb3_vbus_reg: regulator@4 {
vin-supply = <&tps65090_dcdc1_reg>;
};
- vdd_hdmi_reg: regulator@5 {
+ vdd_hdmi_reg: regulator-hdmi {
compatible = "regulator-fixed";
regulator-name = "vdd_hdmi_5v0";
regulator-min-microvolt = <5000000>;
@@ -1233,7 +1234,7 @@ vdd_hdmi_reg: regulator@5 {
vin-supply = <&tps65090_dcdc1_reg>;
};
- vdd_cam_1v8_reg: regulator@6 {
+ vdd_cam_1v8_reg: regulator-cam {
compatible = "regulator-fixed";
regulator-name = "vdd_cam_1v8_reg";
regulator-min-microvolt = <1800000>;
@@ -1242,7 +1243,7 @@ vdd_cam_1v8_reg: regulator@6 {
gpio = <&palmas_gpio 6 0>;
};
- vdd_5v0_hdmi: regulator@7 {
+ vdd_5v0_hdmi: regulator-hdmicon {
compatible = "regulator-fixed";
regulator-name = "VDD_5V0_HDMI_CON";
regulator-min-microvolt = <5000000>;
diff --git a/arch/arm/boot/dts/tegra114-roth.dts b/arch/arm/boot/dts/tegra114-roth.dts
index 0796017..2498cf1 100644
--- a/arch/arm/boot/dts/tegra114-roth.dts
+++ b/arch/arm/boot/dts/tegra114-roth.dts
@@ -801,7 +801,7 @@ regulator@43 {
};
palmas: pmic@58 {
- compatible = "ti,palmas";
+ compatible = "ti,tps65913", "ti,palmas";
reg = <0x58>;
interrupts = <GIC_SPI 86 IRQ_TYPE_LEVEL_HIGH>;
@@ -1016,7 +1016,7 @@ backlight: backlight {
enable-gpios = <&gpio TEGRA_GPIO(H, 2) GPIO_ACTIVE_HIGH>;
};
- clk32k_in: clock@0 {
+ clk32k_in: clock-32k {
compatible = "fixed-clock";
clock-frequency = <32768>;
#clock-cells = <0>;
@@ -1045,7 +1045,7 @@ power {
};
};
- lcd_bl_en: regulator@0 {
+ lcd_bl_en: regulator-lcden {
compatible = "regulator-fixed";
regulator-name = "lcd_bl_en";
regulator-min-microvolt = <5000000>;
@@ -1053,7 +1053,7 @@ lcd_bl_en: regulator@0 {
regulator-boot-on;
};
- vdd_lcd: regulator@1 {
+ vdd_lcd: regulator-lcd {
compatible = "regulator-fixed";
regulator-name = "vdd_lcd_1v8";
regulator-min-microvolt = <1800000>;
@@ -1064,7 +1064,7 @@ vdd_lcd: regulator@1 {
regulator-boot-on;
};
- regulator@2 {
+ regulator-1v8ts {
compatible = "regulator-fixed";
regulator-name = "vdd_1v8_ts";
regulator-min-microvolt = <1800000>;
@@ -1073,7 +1073,7 @@ regulator@2 {
regulator-boot-on;
};
- regulator@3 {
+ regulator-3v3ts {
compatible = "regulator-fixed";
regulator-name = "vdd_3v3_ts";
regulator-min-microvolt = <3300000>;
@@ -1083,7 +1083,7 @@ regulator@3 {
regulator-boot-on;
};
- regulator@4 {
+ regulator-1v8com {
compatible = "regulator-fixed";
regulator-name = "vdd_1v8_com";
regulator-min-microvolt = <1800000>;
@@ -1094,7 +1094,7 @@ regulator@4 {
regulator-boot-on;
};
- regulator@5 {
+ regulator-3v3com {
compatible = "regulator-fixed";
regulator-name = "vdd_3v3_com";
regulator-min-microvolt = <3300000>;
diff --git a/arch/arm/boot/dts/tegra114-tn7.dts b/arch/arm/boot/dts/tegra114-tn7.dts
index 745d234b..ef8f722 100644
--- a/arch/arm/boot/dts/tegra114-tn7.dts
+++ b/arch/arm/boot/dts/tegra114-tn7.dts
@@ -62,7 +62,7 @@ i2c@7000d000 {
clock-frequency = <400000>;
palmas: pmic@58 {
- compatible = "ti,palmas";
+ compatible = "ti,tps65913", "ti,palmas";
reg = <0x58>;
interrupts = <GIC_SPI 86 IRQ_TYPE_LEVEL_HIGH>;
@@ -273,7 +273,7 @@ backlight: backlight {
power-supply = <&lcd_bl_en>;
};
- clk32k_in: clock@0 {
+ clk32k_in: clock-32k {
compatible = "fixed-clock";
clock-frequency = <32768>;
#clock-cells = <0>;
@@ -303,7 +303,7 @@ volume_up {
};
/* FIXME: output of BQ24192 */
- vs_sys: regulator@0 {
+ vs_sys: regulator-vs {
compatible = "regulator-fixed";
regulator-name = "VS_SYS";
regulator-min-microvolt = <4200000>;
@@ -312,7 +312,7 @@ vs_sys: regulator@0 {
regulator-boot-on;
};
- lcd_bl_en: regulator@1 {
+ lcd_bl_en: regulator-lcden {
compatible = "regulator-fixed";
regulator-name = "VDD_LCD_BL";
regulator-min-microvolt = <16500000>;
@@ -323,7 +323,7 @@ lcd_bl_en: regulator@1 {
regulator-boot-on;
};
- vdd_lcd: regulator@2 {
+ vdd_lcd: regulator-lcd {
compatible = "regulator-fixed";
regulator-name = "VD_LCD_1V8";
regulator-min-microvolt = <1800000>;
diff --git a/arch/arm/boot/dts/tegra114.dtsi b/arch/arm/boot/dts/tegra114.dtsi
index 546272e..09996ac 100644
--- a/arch/arm/boot/dts/tegra114.dtsi
+++ b/arch/arm/boot/dts/tegra114.dtsi
@@ -17,6 +17,19 @@ memory@80000000 {
reg = <0x80000000 0x0>;
};
+ sram@40000000 {
+ compatible = "mmio-sram";
+ reg = <0x40000000 0x40000>;
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges = <0 0x40000000 0x40000>;
+
+ vde_pool: sram@400 {
+ reg = <0x400 0x3fc00>;
+ pool;
+ };
+ };
+
host1x@50000000 {
compatible = "nvidia,tegra114-host1x";
reg = <0x50000000 0x00028000>;
@@ -25,8 +38,8 @@ host1x@50000000 {
interrupt-names = "syncpt", "host1x";
clocks = <&tegra_car TEGRA114_CLK_HOST1X>;
clock-names = "host1x";
- resets = <&tegra_car 28>;
- reset-names = "host1x";
+ resets = <&tegra_car 28>, <&mc TEGRA114_MC_RESET_HC>;
+ reset-names = "host1x", "mc";
iommus = <&mc TEGRA_SWGROUP_HC>;
#address-cells = <1>;
@@ -39,8 +52,8 @@ gr2d@54140000 {
reg = <0x54140000 0x00040000>;
interrupts = <GIC_SPI 72 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&tegra_car TEGRA114_CLK_GR2D>;
- resets = <&tegra_car 21>;
- reset-names = "2d";
+ resets = <&tegra_car 21>, <&mc TEGRA114_MC_RESET_2D>;
+ reset-names = "2d", "mc";
iommus = <&mc TEGRA_SWGROUP_G2>;
};
@@ -49,8 +62,8 @@ gr3d@54180000 {
compatible = "nvidia,tegra114-gr3d";
reg = <0x54180000 0x00040000>;
clocks = <&tegra_car TEGRA114_CLK_GR3D>;
- resets = <&tegra_car 24>;
- reset-names = "3d";
+ resets = <&tegra_car 24>, <&mc TEGRA114_MC_RESET_3D>;
+ reset-names = "3d", "mc";
iommus = <&mc TEGRA_SWGROUP_NV>;
};
@@ -105,7 +118,7 @@ hdmi@54280000 {
status = "disabled";
};
- dsi@54300000 {
+ dsia: dsi@54300000 {
compatible = "nvidia,tegra114-dsi";
reg = <0x54300000 0x00040000>;
clocks = <&tegra_car TEGRA114_CLK_DSIA>,
@@ -121,7 +134,7 @@ dsi@54300000 {
#size-cells = <0>;
};
- dsi@54400000 {
+ dsib: dsi@54400000 {
compatible = "nvidia,tegra114-dsi";
reg = <0x54400000 0x00040000>;
clocks = <&tegra_car TEGRA114_CLK_DSIB>,
@@ -164,7 +177,7 @@ lic: interrupt-controller@60004000 {
};
timer@60005000 {
- compatible = "nvidia,tegra114-timer", "nvidia,tegra30-timer", "nvidia,tegra20-timer";
+ compatible = "nvidia,tegra114-timer", "nvidia,tegra30-timer";
reg = <0x60005000 0x400>;
interrupts = <GIC_SPI 0 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 1 IRQ_TYPE_LEVEL_HIGH>,
@@ -248,9 +261,31 @@ gpio: gpio@6000d000 {
gpio-controller;
#interrupt-cells = <2>;
interrupt-controller;
- /*
gpio-ranges = <&pinmux 0 0 246>;
- */
+ };
+
+ vde@6001a000 {
+ compatible = "nvidia,tegra114-vde";
+ reg = <0x6001a000 0x1000>, /* Syntax Engine */
+ <0x6001b000 0x1000>, /* Video Bitstream Engine */
+ <0x6001c000 0x100>, /* Macroblock Engine */
+ <0x6001c200 0x100>, /* Post-processing Engine */
+ <0x6001c400 0x100>, /* Motion Compensation Engine */
+ <0x6001c600 0x100>, /* Transform Engine */
+ <0x6001c800 0x100>, /* Pixel prediction block */
+ <0x6001ca00 0x100>, /* Video DMA */
+ <0x6001d800 0x400>; /* Video frame controls */
+ reg-names = "sxe", "bsev", "mbe", "ppe", "mce",
+ "tfe", "ppb", "vdma", "frameid";
+ iram = <&vde_pool>; /* IRAM region */
+ interrupts = <GIC_SPI 9 IRQ_TYPE_LEVEL_HIGH>, /* Sync token interrupt */
+ <GIC_SPI 10 IRQ_TYPE_LEVEL_HIGH>, /* BSE-V interrupt */
+ <GIC_SPI 12 IRQ_TYPE_LEVEL_HIGH>; /* SXE interrupt */
+ interrupt-names = "sync-token", "bsev", "sxe";
+ clocks = <&tegra_car TEGRA114_CLK_VDE>;
+ reset-names = "vde", "mc";
+ resets = <&tegra_car 61>, <&mc TEGRA114_MC_RESET_VDE>;
+ iommus = <&mc TEGRA_SWGROUP_VDE>;
};
apbmisc@70000800 {
@@ -542,6 +577,7 @@ mc: memory-controller@70019000 {
interrupts = <GIC_SPI 77 IRQ_TYPE_LEVEL_HIGH>;
+ #reset-cells = <1>;
#iommu-cells = <1>;
};
diff --git a/arch/arm/boot/dts/tegra124-apalis-emc.dtsi b/arch/arm/boot/dts/tegra124-apalis-emc.dtsi
index a7ac805..7143c6b 100644
--- a/arch/arm/boot/dts/tegra124-apalis-emc.dtsi
+++ b/arch/arm/boot/dts/tegra124-apalis-emc.dtsi
@@ -15,66 +15,77 @@ timing-12750000 {
clocks = <&tegra_car TEGRA124_CLK_PLL_P>;
clock-names = "emc-parent";
};
+
timing-20400000 {
clock-frequency = <20400000>;
nvidia,parent-clock-frequency = <408000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_P>;
clock-names = "emc-parent";
};
+
timing-40800000 {
clock-frequency = <40800000>;
nvidia,parent-clock-frequency = <408000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_P>;
clock-names = "emc-parent";
};
+
timing-68000000 {
clock-frequency = <68000000>;
nvidia,parent-clock-frequency = <408000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_P>;
clock-names = "emc-parent";
};
+
timing-102000000 {
clock-frequency = <102000000>;
nvidia,parent-clock-frequency = <408000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_P>;
clock-names = "emc-parent";
};
+
timing-204000000 {
clock-frequency = <204000000>;
nvidia,parent-clock-frequency = <408000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_P>;
clock-names = "emc-parent";
};
+
timing-300000000 {
clock-frequency = <300000000>;
nvidia,parent-clock-frequency = <600000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_C>;
clock-names = "emc-parent";
};
+
timing-396000000 {
clock-frequency = <396000000>;
nvidia,parent-clock-frequency = <792000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_M>;
clock-names = "emc-parent";
};
+
timing-528000000 {
clock-frequency = <528000000>;
nvidia,parent-clock-frequency = <528000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_M_UD>;
clock-names = "emc-parent";
};
+
timing-600000000 {
clock-frequency = <600000000>;
nvidia,parent-clock-frequency = <600000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_C_UD>;
clock-names = "emc-parent";
};
+
timing-792000000 {
clock-frequency = <792000000>;
nvidia,parent-clock-frequency = <792000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_M_UD>;
clock-names = "emc-parent";
};
+
timing-924000000 {
clock-frequency = <924000000>;
nvidia,parent-clock-frequency = <924000000>;
@@ -84,6 +95,216 @@ timing-924000000 {
};
};
+ memory-controller@70019000 {
+ emc-timings-1 {
+ nvidia,ram-code = <1>;
+
+ timing-12750000 {
+ clock-frequency = <12750000>;
+
+ nvidia,emem-configuration = <
+ 0x40040001 0x8000000a
+ 0x00000001 0x00000001
+ 0x00000002 0x00000000
+ 0x00000002 0x00000001
+ 0x00000003 0x00000008
+ 0x00000003 0x00000002
+ 0x00000003 0x00000006
+ 0x06030203 0x000a0502
+ 0x77e30303 0x70000f03
+ 0x001f0000
+ >;
+ };
+
+ timing-20400000 {
+ clock-frequency = <20400000>;
+
+ nvidia,emem-configuration = <
+ 0x40020001 0x80000012
+ 0x00000001 0x00000001
+ 0x00000002 0x00000000
+ 0x00000002 0x00000001
+ 0x00000003 0x00000008
+ 0x00000003 0x00000002
+ 0x00000003 0x00000006
+ 0x06030203 0x000a0502
+ 0x76230303 0x70000f03
+ 0x001f0000
+ >;
+ };
+
+ timing-40800000 {
+ clock-frequency = <40800000>;
+
+ nvidia,emem-configuration = <
+ 0xa0000001 0x80000017
+ 0x00000001 0x00000001
+ 0x00000002 0x00000000
+ 0x00000002 0x00000001
+ 0x00000003 0x00000008
+ 0x00000003 0x00000002
+ 0x00000003 0x00000006
+ 0x06030203 0x000a0502
+ 0x74a30303 0x70000f03
+ 0x001f0000
+ >;
+ };
+
+ timing-68000000 {
+ clock-frequency = <68000000>;
+
+ nvidia,emem-configuration = <
+ 0x00000001 0x8000001e
+ 0x00000001 0x00000001
+ 0x00000002 0x00000000
+ 0x00000002 0x00000001
+ 0x00000003 0x00000008
+ 0x00000003 0x00000002
+ 0x00000003 0x00000006
+ 0x06030203 0x000a0502
+ 0x74230403 0x70000f03
+ 0x001f0000
+ >;
+ };
+
+ timing-102000000 {
+ clock-frequency = <102000000>;
+
+ nvidia,emem-configuration = <
+ 0x08000001 0x80000026
+ 0x00000001 0x00000001
+ 0x00000003 0x00000000
+ 0x00000002 0x00000001
+ 0x00000003 0x00000008
+ 0x00000003 0x00000002
+ 0x00000003 0x00000006
+ 0x06030203 0x000a0503
+ 0x73c30504 0x70000f03
+ 0x001f0000
+ >;
+ };
+
+ timing-204000000 {
+ clock-frequency = <204000000>;
+
+ nvidia,emem-configuration = <
+ 0x01000003 0x80000040
+ 0x00000001 0x00000001
+ 0x00000004 0x00000002
+ 0x00000003 0x00000001
+ 0x00000003 0x00000008
+ 0x00000003 0x00000002
+ 0x00000004 0x00000006
+ 0x06040203 0x000a0504
+ 0x73840a05 0x70000f03
+ 0x001f0000
+ >;
+ };
+
+ timing-300000000 {
+ clock-frequency = <300000000>;
+
+ nvidia,emem-configuration = <
+ 0x08000004 0x80000040
+ 0x00000001 0x00000002
+ 0x00000007 0x00000004
+ 0x00000004 0x00000001
+ 0x00000002 0x00000007
+ 0x00000002 0x00000002
+ 0x00000004 0x00000006
+ 0x06040202 0x000b0607
+ 0x77450e08 0x70000f03
+ 0x001f0000
+ >;
+ };
+
+ timing-396000000 {
+ clock-frequency = <396000000>;
+
+ nvidia,emem-configuration = <
+ 0x0f000005 0x80000040
+ 0x00000001 0x00000002
+ 0x00000009 0x00000005
+ 0x00000006 0x00000001
+ 0x00000002 0x00000008
+ 0x00000002 0x00000002
+ 0x00000004 0x00000006
+ 0x06040202 0x000d0709
+ 0x7586120a 0x70000f03
+ 0x001f0000
+ >;
+ };
+
+ timing-528000000 {
+ clock-frequency = <528000000>;
+
+ nvidia,emem-configuration = <
+ 0x0f000007 0x80000040
+ 0x00000002 0x00000003
+ 0x0000000c 0x00000007
+ 0x00000008 0x00000001
+ 0x00000002 0x00000009
+ 0x00000002 0x00000002
+ 0x00000005 0x00000006
+ 0x06050202 0x0010090c
+ 0x7428180d 0x70000f03
+ 0x001f0000
+ >;
+ };
+
+ timing-600000000 {
+ clock-frequency = <600000000>;
+
+ nvidia,emem-configuration = <
+ 0x00000009 0x80000040
+ 0x00000003 0x00000004
+ 0x0000000e 0x00000009
+ 0x0000000a 0x00000001
+ 0x00000003 0x0000000b
+ 0x00000002 0x00000002
+ 0x00000005 0x00000007
+ 0x07050202 0x00130b0e
+ 0x73a91b0f 0x70000f03
+ 0x001f0000
+ >;
+ };
+
+ timing-792000000 {
+ clock-frequency = <792000000>;
+
+ nvidia,emem-configuration = <
+ 0x0e00000b 0x80000040
+ 0x00000004 0x00000005
+ 0x00000013 0x0000000c
+ 0x0000000d 0x00000002
+ 0x00000003 0x0000000c
+ 0x00000002 0x00000002
+ 0x00000006 0x00000008
+ 0x08060202 0x00170e13
+ 0x736c2414 0x70000f02
+ 0x001f0000
+ >;
+ };
+
+ timing-924000000 {
+ clock-frequency = <924000000>;
+
+ nvidia,emem-configuration = <
+ 0x0e00000d 0x80000040
+ 0x00000005 0x00000006
+ 0x00000016 0x0000000e
+ 0x0000000f 0x00000002
+ 0x00000004 0x0000000e
+ 0x00000002 0x00000002
+ 0x00000006 0x00000009
+ 0x09060202 0x001a1016
+ 0x734e2a17 0x70000f02
+ 0x001f0000
+ >;
+ };
+ };
+ };
+
external-memory-controller@7001b000 {
emc-timings-1 {
nvidia,ram-code = <1>;
@@ -1251,225 +1472,14 @@ timing-924000000 {
0x00000011
>;
};
-
- };
- };
-
- memory-controller@70019000 {
- emc-timings-1 {
- nvidia,ram-code = <1>;
-
- timing-12750000 {
- clock-frequency = <12750000>;
-
- nvidia,emem-configuration = <
- 0x40040001 0x8000000a
- 0x00000001 0x00000001
- 0x00000002 0x00000000
- 0x00000002 0x00000001
- 0x00000003 0x00000008
- 0x00000003 0x00000002
- 0x00000003 0x00000006
- 0x06030203 0x000a0502
- 0x77e30303 0x70000f03
- 0x001f0000
- >;
- };
-
- timing-20400000 {
- clock-frequency = <20400000>;
-
- nvidia,emem-configuration = <
- 0x40020001 0x80000012
- 0x00000001 0x00000001
- 0x00000002 0x00000000
- 0x00000002 0x00000001
- 0x00000003 0x00000008
- 0x00000003 0x00000002
- 0x00000003 0x00000006
- 0x06030203 0x000a0502
- 0x76230303 0x70000f03
- 0x001f0000
- >;
- };
-
- timing-40800000 {
- clock-frequency = <40800000>;
-
- nvidia,emem-configuration = <
- 0xa0000001 0x80000017
- 0x00000001 0x00000001
- 0x00000002 0x00000000
- 0x00000002 0x00000001
- 0x00000003 0x00000008
- 0x00000003 0x00000002
- 0x00000003 0x00000006
- 0x06030203 0x000a0502
- 0x74a30303 0x70000f03
- 0x001f0000
- >;
- };
-
- timing-68000000 {
- clock-frequency = <68000000>;
-
- nvidia,emem-configuration = <
- 0x00000001 0x8000001e
- 0x00000001 0x00000001
- 0x00000002 0x00000000
- 0x00000002 0x00000001
- 0x00000003 0x00000008
- 0x00000003 0x00000002
- 0x00000003 0x00000006
- 0x06030203 0x000a0502
- 0x74230403 0x70000f03
- 0x001f0000
- >;
- };
-
- timing-102000000 {
- clock-frequency = <102000000>;
-
- nvidia,emem-configuration = <
- 0x08000001 0x80000026
- 0x00000001 0x00000001
- 0x00000003 0x00000000
- 0x00000002 0x00000001
- 0x00000003 0x00000008
- 0x00000003 0x00000002
- 0x00000003 0x00000006
- 0x06030203 0x000a0503
- 0x73c30504 0x70000f03
- 0x001f0000
- >;
- };
-
- timing-204000000 {
- clock-frequency = <204000000>;
-
- nvidia,emem-configuration = <
- 0x01000003 0x80000040
- 0x00000001 0x00000001
- 0x00000004 0x00000002
- 0x00000003 0x00000001
- 0x00000003 0x00000008
- 0x00000003 0x00000002
- 0x00000004 0x00000006
- 0x06040203 0x000a0504
- 0x73840a05 0x70000f03
- 0x001f0000
- >;
- };
-
- timing-300000000 {
- clock-frequency = <300000000>;
-
- nvidia,emem-configuration = <
- 0x08000004 0x80000040
- 0x00000001 0x00000002
- 0x00000007 0x00000004
- 0x00000004 0x00000001
- 0x00000002 0x00000007
- 0x00000002 0x00000002
- 0x00000004 0x00000006
- 0x06040202 0x000b0607
- 0x77450e08 0x70000f03
- 0x001f0000
- >;
- };
-
- timing-396000000 {
- clock-frequency = <396000000>;
-
- nvidia,emem-configuration = <
- 0x0f000005 0x80000040
- 0x00000001 0x00000002
- 0x00000009 0x00000005
- 0x00000006 0x00000001
- 0x00000002 0x00000008
- 0x00000002 0x00000002
- 0x00000004 0x00000006
- 0x06040202 0x000d0709
- 0x7586120a 0x70000f03
- 0x001f0000
- >;
- };
-
- timing-528000000 {
- clock-frequency = <528000000>;
-
- nvidia,emem-configuration = <
- 0x0f000007 0x80000040
- 0x00000002 0x00000003
- 0x0000000c 0x00000007
- 0x00000008 0x00000001
- 0x00000002 0x00000009
- 0x00000002 0x00000002
- 0x00000005 0x00000006
- 0x06050202 0x0010090c
- 0x7428180d 0x70000f03
- 0x001f0000
- >;
- };
-
- timing-600000000 {
- clock-frequency = <600000000>;
-
- nvidia,emem-configuration = <
- 0x00000009 0x80000040
- 0x00000003 0x00000004
- 0x0000000e 0x00000009
- 0x0000000a 0x00000001
- 0x00000003 0x0000000b
- 0x00000002 0x00000002
- 0x00000005 0x00000007
- 0x07050202 0x00130b0e
- 0x73a91b0f 0x70000f03
- 0x001f0000
- >;
- };
-
- timing-792000000 {
- clock-frequency = <792000000>;
-
- nvidia,emem-configuration = <
- 0x0e00000b 0x80000040
- 0x00000004 0x00000005
- 0x00000013 0x0000000c
- 0x0000000d 0x00000002
- 0x00000003 0x0000000c
- 0x00000002 0x00000002
- 0x00000006 0x00000008
- 0x08060202 0x00170e13
- 0x736c2414 0x70000f02
- 0x001f0000
- >;
- };
-
- timing-924000000 {
- clock-frequency = <924000000>;
-
- nvidia,emem-configuration = <
- 0x0e00000d 0x80000040
- 0x00000005 0x00000006
- 0x00000016 0x0000000e
- 0x0000000f 0x00000002
- 0x00000004 0x0000000e
- 0x00000002 0x00000002
- 0x00000006 0x00000009
- 0x09060202 0x001a1016
- 0x734e2a17 0x70000f02
- 0x001f0000
- >;
- };
};
};
};
&emc_icc_dvfs_opp_table {
- /delete-node/ opp@1200000000,1100;
+ /delete-node/ opp-1200000000-1100;
};
&emc_bw_dfs_opp_table {
- /delete-node/ opp@1200000000;
+ /delete-node/ opp-1200000000;
};
diff --git a/arch/arm/boot/dts/tegra124-apalis-eval.dts b/arch/arm/boot/dts/tegra124-apalis-eval.dts
index 28c29b6..3209554 100644
--- a/arch/arm/boot/dts/tegra124-apalis-eval.dts
+++ b/arch/arm/boot/dts/tegra124-apalis-eval.dts
@@ -246,7 +246,7 @@ reg_usbh_vbus: regulator-usbh-vbus {
&gpio {
/* Apalis GPIO7 MXM3 pin 15 PLX PEX 8605 PCIe Switch Reset */
- pex-perst-n {
+ pex-perst-n-hog {
gpio-hog;
gpios = <TEGRA_GPIO(DD, 1) GPIO_ACTIVE_HIGH>;
output-high;
diff --git a/arch/arm/boot/dts/tegra124-apalis-v1.2-eval.dts b/arch/arm/boot/dts/tegra124-apalis-v1.2-eval.dts
index f3afde4..814257c 100644
--- a/arch/arm/boot/dts/tegra124-apalis-v1.2-eval.dts
+++ b/arch/arm/boot/dts/tegra124-apalis-v1.2-eval.dts
@@ -248,7 +248,7 @@ reg_usbh_vbus: regulator-usbh-vbus {
&gpio {
/* Apalis GPIO7 MXM3 pin 15 PLX PEX 8605 PCIe Switch Reset */
- pex-perst-n {
+ pex-perst-n-hog {
gpio-hog;
gpios = <TEGRA_GPIO(DD, 1) GPIO_ACTIVE_HIGH>;
output-high;
diff --git a/arch/arm/boot/dts/tegra124-apalis-v1.2.dtsi b/arch/arm/boot/dts/tegra124-apalis-v1.2.dtsi
index cde9ae8..dbb0da0 100644
--- a/arch/arm/boot/dts/tegra124-apalis-v1.2.dtsi
+++ b/arch/arm/boot/dts/tegra124-apalis-v1.2.dtsi
@@ -57,7 +57,7 @@ hdmi@54280000 {
};
};
- gpu@0,57000000 {
+ gpu@57000000 {
/*
* Node left disabled on purpose - the bootloader will enable
* it after having set the VPR up
@@ -1539,14 +1539,17 @@ sdmmc3-clk-lb-out-pee4 { /* NC */
serial@70006040 {
compatible = "nvidia,tegra124-hsuart", "nvidia,tegra30-hsuart";
+ /delete-property/ reg-shift;
};
serial@70006200 {
compatible = "nvidia,tegra124-hsuart", "nvidia,tegra30-hsuart";
+ /delete-property/ reg-shift;
};
serial@70006300 {
compatible = "nvidia,tegra124-hsuart", "nvidia,tegra30-hsuart";
+ /delete-property/ reg-shift;
};
hdmi_ddc: i2c@7000c700 {
@@ -1885,6 +1888,7 @@ ports {
usb2-0 {
status = "okay";
mode = "otg";
+ usb-role-switch;
vbus-supply = <®_usbo1_vbus>;
};
@@ -2021,7 +2025,7 @@ sound {
};
thermal-zones {
- cpu {
+ cpu-thermal {
trips {
cpu-shutdown-trip {
temperature = <101000>;
@@ -2031,7 +2035,7 @@ cpu-shutdown-trip {
};
};
- mem {
+ mem-thermal {
trips {
mem-shutdown-trip {
temperature = <101000>;
@@ -2041,7 +2045,7 @@ mem-shutdown-trip {
};
};
- gpu {
+ gpu-thermal {
trips {
gpu-shutdown-trip {
temperature = <101000>;
@@ -2055,7 +2059,7 @@ gpu-shutdown-trip {
&gpio {
/* I210 Gigabit Ethernet Controller Reset */
- lan-reset-n {
+ lan-reset-n-hog {
gpio-hog;
gpios = <TEGRA_GPIO(S, 2) GPIO_ACTIVE_HIGH>;
output-high;
@@ -2063,7 +2067,7 @@ lan-reset-n {
};
/* Control MXM3 pin 26 Reset Module Output Carrier Input */
- reset-moci-ctrl {
+ reset-moci-ctrl-hog {
gpio-hog;
gpios = <TEGRA_GPIO(U, 4) GPIO_ACTIVE_HIGH>;
output-high;
diff --git a/arch/arm/boot/dts/tegra124-apalis.dtsi b/arch/arm/boot/dts/tegra124-apalis.dtsi
index a46d9ba..d3f16c1 100644
--- a/arch/arm/boot/dts/tegra124-apalis.dtsi
+++ b/arch/arm/boot/dts/tegra124-apalis.dtsi
@@ -56,7 +56,7 @@ hdmi@54280000 {
};
};
- gpu@0,57000000 {
+ gpu@57000000 {
/*
* Node left disabled on purpose - the bootloader will enable
* it after having set the VPR up
@@ -1532,14 +1532,17 @@ sdmmc3-clk-lb-out-pee4 { /* NC */
serial@70006040 {
compatible = "nvidia,tegra124-hsuart", "nvidia,tegra30-hsuart";
+ /delete-property/ reg-shift;
};
serial@70006200 {
compatible = "nvidia,tegra124-hsuart", "nvidia,tegra30-hsuart";
+ /delete-property/ reg-shift;
};
serial@70006300 {
compatible = "nvidia,tegra124-hsuart", "nvidia,tegra30-hsuart";
+ /delete-property/ reg-shift;
};
hdmi_ddc: i2c@7000c400 {
@@ -1877,6 +1880,7 @@ ports {
usb2-0 {
status = "okay";
mode = "otg";
+ usb-role-switch;
vbus-supply = <®_usbo1_vbus>;
};
@@ -2013,7 +2017,7 @@ sound {
};
thermal-zones {
- cpu {
+ cpu-thermal {
trips {
cpu-shutdown-trip {
temperature = <101000>;
@@ -2023,7 +2027,7 @@ cpu-shutdown-trip {
};
};
- mem {
+ mem-thermal {
trips {
mem-shutdown-trip {
temperature = <101000>;
@@ -2033,7 +2037,7 @@ mem-shutdown-trip {
};
};
- gpu {
+ gpu-thermal {
trips {
gpu-shutdown-trip {
temperature = <101000>;
@@ -2047,7 +2051,7 @@ gpu-shutdown-trip {
&gpio {
/* I210 Gigabit Ethernet Controller Reset */
- lan-reset-n {
+ lan-reset-n-hog {
gpio-hog;
gpios = <TEGRA_GPIO(S, 2) GPIO_ACTIVE_HIGH>;
output-high;
@@ -2055,7 +2059,7 @@ lan-reset-n {
};
/* Control MXM3 pin 26 Reset Module Output Carrier Input */
- reset-moci-ctrl {
+ reset-moci-ctrl-hog {
gpio-hog;
gpios = <TEGRA_GPIO(U, 4) GPIO_ACTIVE_HIGH>;
output-high;
diff --git a/arch/arm/boot/dts/tegra124-jetson-tk1-emc.dtsi b/arch/arm/boot/dts/tegra124-jetson-tk1-emc.dtsi
index df4e463..79e776d 100644
--- a/arch/arm/boot/dts/tegra124-jetson-tk1-emc.dtsi
+++ b/arch/arm/boot/dts/tegra124-jetson-tk1-emc.dtsi
@@ -10,66 +10,77 @@ timing-12750000 {
clocks = <&tegra_car TEGRA124_CLK_PLL_P>;
clock-names = "emc-parent";
};
+
timing-20400000 {
clock-frequency = <20400000>;
nvidia,parent-clock-frequency = <408000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_P>;
clock-names = "emc-parent";
};
+
timing-40800000 {
clock-frequency = <40800000>;
nvidia,parent-clock-frequency = <408000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_P>;
clock-names = "emc-parent";
};
+
timing-68000000 {
clock-frequency = <68000000>;
nvidia,parent-clock-frequency = <408000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_P>;
clock-names = "emc-parent";
};
+
timing-102000000 {
clock-frequency = <102000000>;
nvidia,parent-clock-frequency = <408000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_P>;
clock-names = "emc-parent";
};
+
timing-204000000 {
clock-frequency = <204000000>;
nvidia,parent-clock-frequency = <408000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_P>;
clock-names = "emc-parent";
};
+
timing-300000000 {
clock-frequency = <300000000>;
nvidia,parent-clock-frequency = <600000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_C>;
clock-names = "emc-parent";
};
+
timing-396000000 {
clock-frequency = <396000000>;
nvidia,parent-clock-frequency = <792000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_M>;
clock-names = "emc-parent";
};
+
timing-528000000 {
clock-frequency = <528000000>;
nvidia,parent-clock-frequency = <528000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_M_UD>;
clock-names = "emc-parent";
};
+
timing-600000000 {
clock-frequency = <600000000>;
nvidia,parent-clock-frequency = <600000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_C_UD>;
clock-names = "emc-parent";
};
+
timing-792000000 {
clock-frequency = <792000000>;
nvidia,parent-clock-frequency = <792000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_M_UD>;
clock-names = "emc-parent";
};
+
timing-924000000 {
clock-frequency = <924000000>;
nvidia,parent-clock-frequency = <924000000>;
@@ -79,6 +90,324 @@ timing-924000000 {
};
};
+ memory-controller@70019000 {
+ emc-timings-3 {
+ nvidia,ram-code = <3>;
+
+ timing-12750000 {
+ clock-frequency = <12750000>;
+
+ nvidia,emem-configuration = <
+ 0x40040001
+ 0x8000000a
+ 0x00000001
+ 0x00000001
+ 0x00000002
+ 0x00000000
+ 0x00000002
+ 0x00000001
+ 0x00000003
+ 0x00000008
+ 0x00000003
+ 0x00000002
+ 0x00000003
+ 0x00000006
+ 0x06030203
+ 0x000a0502
+ 0x77e30303
+ 0x70000f03
+ 0x001f0000
+ >;
+ };
+
+ timing-20400000 {
+ clock-frequency = <20400000>;
+
+ nvidia,emem-configuration = <
+ 0x40020001
+ 0x80000012
+ 0x00000001
+ 0x00000001
+ 0x00000002
+ 0x00000000
+ 0x00000002
+ 0x00000001
+ 0x00000003
+ 0x00000008
+ 0x00000003
+ 0x00000002
+ 0x00000003
+ 0x00000006
+ 0x06030203
+ 0x000a0502
+ 0x76230303
+ 0x70000f03
+ 0x001f0000
+ >;
+ };
+
+ timing-40800000 {
+ clock-frequency = <40800000>;
+
+ nvidia,emem-configuration = <
+ 0xa0000001
+ 0x80000017
+ 0x00000001
+ 0x00000001
+ 0x00000002
+ 0x00000000
+ 0x00000002
+ 0x00000001
+ 0x00000003
+ 0x00000008
+ 0x00000003
+ 0x00000002
+ 0x00000003
+ 0x00000006
+ 0x06030203
+ 0x000a0502
+ 0x74a30303
+ 0x70000f03
+ 0x001f0000
+ >;
+ };
+
+ timing-68000000 {
+ clock-frequency = <68000000>;
+
+ nvidia,emem-configuration = <
+ 0x00000001
+ 0x8000001e
+ 0x00000001
+ 0x00000001
+ 0x00000002
+ 0x00000000
+ 0x00000002
+ 0x00000001
+ 0x00000003
+ 0x00000008
+ 0x00000003
+ 0x00000002
+ 0x00000003
+ 0x00000006
+ 0x06030203
+ 0x000a0502
+ 0x74230403
+ 0x70000f03
+ 0x001f0000
+ >;
+ };
+
+ timing-102000000 {
+ clock-frequency = <102000000>;
+
+ nvidia,emem-configuration = <
+ 0x08000001
+ 0x80000026
+ 0x00000001
+ 0x00000001
+ 0x00000003
+ 0x00000000
+ 0x00000002
+ 0x00000001
+ 0x00000003
+ 0x00000008
+ 0x00000003
+ 0x00000002
+ 0x00000003
+ 0x00000006
+ 0x06030203
+ 0x000a0503
+ 0x73c30504
+ 0x70000f03
+ 0x001f0000
+ >;
+ };
+
+ timing-204000000 {
+ clock-frequency = <204000000>;
+
+ nvidia,emem-configuration = <
+ 0x01000003
+ 0x80000040
+ 0x00000001
+ 0x00000001
+ 0x00000004
+ 0x00000002
+ 0x00000003
+ 0x00000001
+ 0x00000003
+ 0x00000008
+ 0x00000003
+ 0x00000002
+ 0x00000004
+ 0x00000006
+ 0x06040203
+ 0x000a0504
+ 0x73840a05
+ 0x70000f03
+ 0x001f0000
+ >;
+ };
+
+ timing-300000000 {
+ clock-frequency = <300000000>;
+
+ nvidia,emem-configuration = <
+ 0x08000004
+ 0x80000040
+ 0x00000001
+ 0x00000002
+ 0x00000007
+ 0x00000004
+ 0x00000004
+ 0x00000001
+ 0x00000002
+ 0x00000007
+ 0x00000002
+ 0x00000002
+ 0x00000004
+ 0x00000006
+ 0x06040202
+ 0x000b0607
+ 0x77450e08
+ 0x70000f03
+ 0x001f0000
+ >;
+ };
+
+ timing-396000000 {
+ clock-frequency = <396000000>;
+
+ nvidia,emem-configuration = <
+ 0x0f000005
+ 0x80000040
+ 0x00000001
+ 0x00000002
+ 0x00000009
+ 0x00000005
+ 0x00000006
+ 0x00000001
+ 0x00000002
+ 0x00000008
+ 0x00000002
+ 0x00000002
+ 0x00000004
+ 0x00000006
+ 0x06040202
+ 0x000d0709
+ 0x7586120a
+ 0x70000f03
+ 0x001f0000
+ >;
+ };
+
+ timing-528000000 {
+ clock-frequency = <528000000>;
+
+ nvidia,emem-configuration = <
+ 0x0f000007
+ 0x80000040
+ 0x00000002
+ 0x00000003
+ 0x0000000c
+ 0x00000007
+ 0x00000008
+ 0x00000001
+ 0x00000002
+ 0x00000009
+ 0x00000002
+ 0x00000002
+ 0x00000005
+ 0x00000006
+ 0x06050202
+ 0x0010090c
+ 0x7428180d
+ 0x70000f03
+ 0x001f0000
+ >;
+ };
+
+ timing-600000000 {
+ clock-frequency = <600000000>;
+
+ nvidia,emem-configuration = <
+ 0x00000009
+ 0x80000040
+ 0x00000003
+ 0x00000004
+ 0x0000000e
+ 0x00000009
+ 0x0000000a
+ 0x00000001
+ 0x00000003
+ 0x0000000b
+ 0x00000002
+ 0x00000002
+ 0x00000005
+ 0x00000007
+ 0x07050202
+ 0x00130b0e
+ 0x73a91b0f
+ 0x70000f03
+ 0x001f0000
+ >;
+ };
+
+ timing-792000000 {
+ clock-frequency = <792000000>;
+
+ nvidia,emem-configuration = <
+ 0x0e00000b
+ 0x80000040
+ 0x00000004
+ 0x00000005
+ 0x00000013
+ 0x0000000c
+ 0x0000000d
+ 0x00000002
+ 0x00000003
+ 0x0000000c
+ 0x00000002
+ 0x00000002
+ 0x00000006
+ 0x00000008
+ 0x08060202
+ 0x00170e13
+ 0x736c2414
+ 0x70000f02
+ 0x001f0000
+ >;
+ };
+
+ timing-924000000 {
+ clock-frequency = <924000000>;
+
+ nvidia,emem-configuration = <
+ 0x0e00000d
+ 0x80000040
+ 0x00000005
+ 0x00000006
+ 0x00000016
+ 0x0000000e
+ 0x0000000f
+ 0x00000002
+ 0x00000004
+ 0x0000000e
+ 0x00000002
+ 0x00000002
+ 0x00000006
+ 0x00000009
+ 0x09060202
+ 0x001a1016
+ 0x734e2a17
+ 0x70000f02
+ 0x001f0000
+ >;
+ };
+ };
+ };
+
external-memory-controller@7001b000 {
emc-timings-3 {
nvidia,ram-code = <3>;
@@ -2098,333 +2427,14 @@ timing-924000000 {
0x00000011
>;
};
-
- };
- };
-
- memory-controller@70019000 {
- emc-timings-3 {
- nvidia,ram-code = <3>;
-
- timing-12750000 {
- clock-frequency = <12750000>;
-
- nvidia,emem-configuration = <
- 0x40040001
- 0x8000000a
- 0x00000001
- 0x00000001
- 0x00000002
- 0x00000000
- 0x00000002
- 0x00000001
- 0x00000003
- 0x00000008
- 0x00000003
- 0x00000002
- 0x00000003
- 0x00000006
- 0x06030203
- 0x000a0502
- 0x77e30303
- 0x70000f03
- 0x001f0000
- >;
- };
-
- timing-20400000 {
- clock-frequency = <20400000>;
-
- nvidia,emem-configuration = <
- 0x40020001
- 0x80000012
- 0x00000001
- 0x00000001
- 0x00000002
- 0x00000000
- 0x00000002
- 0x00000001
- 0x00000003
- 0x00000008
- 0x00000003
- 0x00000002
- 0x00000003
- 0x00000006
- 0x06030203
- 0x000a0502
- 0x76230303
- 0x70000f03
- 0x001f0000
- >;
- };
-
- timing-40800000 {
- clock-frequency = <40800000>;
-
- nvidia,emem-configuration = <
- 0xa0000001
- 0x80000017
- 0x00000001
- 0x00000001
- 0x00000002
- 0x00000000
- 0x00000002
- 0x00000001
- 0x00000003
- 0x00000008
- 0x00000003
- 0x00000002
- 0x00000003
- 0x00000006
- 0x06030203
- 0x000a0502
- 0x74a30303
- 0x70000f03
- 0x001f0000
- >;
- };
-
- timing-68000000 {
- clock-frequency = <68000000>;
-
- nvidia,emem-configuration = <
- 0x00000001
- 0x8000001e
- 0x00000001
- 0x00000001
- 0x00000002
- 0x00000000
- 0x00000002
- 0x00000001
- 0x00000003
- 0x00000008
- 0x00000003
- 0x00000002
- 0x00000003
- 0x00000006
- 0x06030203
- 0x000a0502
- 0x74230403
- 0x70000f03
- 0x001f0000
- >;
- };
-
- timing-102000000 {
- clock-frequency = <102000000>;
-
- nvidia,emem-configuration = <
- 0x08000001
- 0x80000026
- 0x00000001
- 0x00000001
- 0x00000003
- 0x00000000
- 0x00000002
- 0x00000001
- 0x00000003
- 0x00000008
- 0x00000003
- 0x00000002
- 0x00000003
- 0x00000006
- 0x06030203
- 0x000a0503
- 0x73c30504
- 0x70000f03
- 0x001f0000
- >;
- };
-
- timing-204000000 {
- clock-frequency = <204000000>;
-
- nvidia,emem-configuration = <
- 0x01000003
- 0x80000040
- 0x00000001
- 0x00000001
- 0x00000004
- 0x00000002
- 0x00000003
- 0x00000001
- 0x00000003
- 0x00000008
- 0x00000003
- 0x00000002
- 0x00000004
- 0x00000006
- 0x06040203
- 0x000a0504
- 0x73840a05
- 0x70000f03
- 0x001f0000
- >;
- };
-
- timing-300000000 {
- clock-frequency = <300000000>;
-
- nvidia,emem-configuration = <
- 0x08000004
- 0x80000040
- 0x00000001
- 0x00000002
- 0x00000007
- 0x00000004
- 0x00000004
- 0x00000001
- 0x00000002
- 0x00000007
- 0x00000002
- 0x00000002
- 0x00000004
- 0x00000006
- 0x06040202
- 0x000b0607
- 0x77450e08
- 0x70000f03
- 0x001f0000
- >;
- };
-
- timing-396000000 {
- clock-frequency = <396000000>;
-
- nvidia,emem-configuration = <
- 0x0f000005
- 0x80000040
- 0x00000001
- 0x00000002
- 0x00000009
- 0x00000005
- 0x00000006
- 0x00000001
- 0x00000002
- 0x00000008
- 0x00000002
- 0x00000002
- 0x00000004
- 0x00000006
- 0x06040202
- 0x000d0709
- 0x7586120a
- 0x70000f03
- 0x001f0000
- >;
- };
-
- timing-528000000 {
- clock-frequency = <528000000>;
-
- nvidia,emem-configuration = <
- 0x0f000007
- 0x80000040
- 0x00000002
- 0x00000003
- 0x0000000c
- 0x00000007
- 0x00000008
- 0x00000001
- 0x00000002
- 0x00000009
- 0x00000002
- 0x00000002
- 0x00000005
- 0x00000006
- 0x06050202
- 0x0010090c
- 0x7428180d
- 0x70000f03
- 0x001f0000
- >;
- };
-
- timing-600000000 {
- clock-frequency = <600000000>;
-
- nvidia,emem-configuration = <
- 0x00000009
- 0x80000040
- 0x00000003
- 0x00000004
- 0x0000000e
- 0x00000009
- 0x0000000a
- 0x00000001
- 0x00000003
- 0x0000000b
- 0x00000002
- 0x00000002
- 0x00000005
- 0x00000007
- 0x07050202
- 0x00130b0e
- 0x73a91b0f
- 0x70000f03
- 0x001f0000
- >;
- };
-
- timing-792000000 {
- clock-frequency = <792000000>;
-
- nvidia,emem-configuration = <
- 0x0e00000b
- 0x80000040
- 0x00000004
- 0x00000005
- 0x00000013
- 0x0000000c
- 0x0000000d
- 0x00000002
- 0x00000003
- 0x0000000c
- 0x00000002
- 0x00000002
- 0x00000006
- 0x00000008
- 0x08060202
- 0x00170e13
- 0x736c2414
- 0x70000f02
- 0x001f0000
- >;
- };
-
- timing-924000000 {
- clock-frequency = <924000000>;
-
- nvidia,emem-configuration = <
- 0x0e00000d
- 0x80000040
- 0x00000005
- 0x00000006
- 0x00000016
- 0x0000000e
- 0x0000000f
- 0x00000002
- 0x00000004
- 0x0000000e
- 0x00000002
- 0x00000002
- 0x00000006
- 0x00000009
- 0x09060202
- 0x001a1016
- 0x734e2a17
- 0x70000f02
- 0x001f0000
- >;
- };
};
};
};
&emc_icc_dvfs_opp_table {
- /delete-node/ opp@1200000000,1100;
+ /delete-node/ opp-1200000000-1100;
};
&emc_bw_dfs_opp_table {
- /delete-node/ opp@1200000000;
+ /delete-node/ opp-1200000000;
};
diff --git a/arch/arm/boot/dts/tegra124-jetson-tk1.dts b/arch/arm/boot/dts/tegra124-jetson-tk1.dts
index 35ab296..28b889e 100644
--- a/arch/arm/boot/dts/tegra124-jetson-tk1.dts
+++ b/arch/arm/boot/dts/tegra124-jetson-tk1.dts
@@ -72,7 +72,7 @@ cec@70015000 {
status = "okay";
};
- gpu@0,57000000 {
+ gpu@57000000 {
/*
* Node left disabled on purpose - the bootloader will enable
* it after having set the VPR up
@@ -1389,6 +1389,7 @@ dsi_b {
*/
serial@70006000 {
compatible = "nvidia,tegra124-hsuart", "nvidia,tegra30-hsuart";
+ /delete-property/ reg-shift;
status = "okay";
};
@@ -1401,6 +1402,7 @@ serial@70006000 {
*/
serial@70006040 {
compatible = "nvidia,tegra124-hsuart", "nvidia,tegra30-hsuart";
+ /delete-property/ reg-shift;
status = "okay";
};
@@ -1655,7 +1657,8 @@ spi@7000d400 {
spi@7000da00 {
status = "okay";
spi-max-frequency = <25000000>;
- spi-flash@0 {
+
+ flash@0 {
compatible = "winbond,w25q32dw", "jedec,spi-nor";
reg = <0>;
spi-max-frequency = <20000000>;
@@ -1868,7 +1871,7 @@ usb-phy@7d008000 {
vbus-supply = <&vdd_usb3_vbus>;
};
- clk32k_in: clock@0 {
+ clk32k_in: clock-32k {
compatible = "fixed-clock";
clock-frequency = <32768>;
#clock-cells = <0>;
@@ -1892,7 +1895,7 @@ power {
};
};
- vdd_mux: regulator@0 {
+ vdd_mux: regulator-mux {
compatible = "regulator-fixed";
regulator-name = "+VDD_MUX";
regulator-min-microvolt = <12000000>;
@@ -1901,7 +1904,7 @@ vdd_mux: regulator@0 {
regulator-boot-on;
};
- vdd_5v0_sys: regulator@1 {
+ vdd_5v0_sys: regulator-5v0sys {
compatible = "regulator-fixed";
regulator-name = "+5V_SYS";
regulator-min-microvolt = <5000000>;
@@ -1911,7 +1914,7 @@ vdd_5v0_sys: regulator@1 {
vin-supply = <&vdd_mux>;
};
- vdd_3v3_sys: regulator@2 {
+ vdd_3v3_sys: regulator-3v3sys {
compatible = "regulator-fixed";
regulator-name = "+3.3V_SYS";
regulator-min-microvolt = <3300000>;
@@ -1921,7 +1924,7 @@ vdd_3v3_sys: regulator@2 {
vin-supply = <&vdd_mux>;
};
- vdd_3v3_run: regulator@3 {
+ vdd_3v3_run: regulator-3v3run {
compatible = "regulator-fixed";
regulator-name = "+3.3V_RUN";
regulator-min-microvolt = <3300000>;
@@ -1933,7 +1936,7 @@ vdd_3v3_run: regulator@3 {
vin-supply = <&vdd_3v3_sys>;
};
- vdd_3v3_hdmi: regulator@4 {
+ vdd_3v3_hdmi: regulator-3v3hdmi {
compatible = "regulator-fixed";
regulator-name = "+3.3V_AVDD_HDMI_AP_GATED";
regulator-min-microvolt = <3300000>;
@@ -1941,7 +1944,7 @@ vdd_3v3_hdmi: regulator@4 {
vin-supply = <&vdd_3v3_run>;
};
- vdd_usb1_vbus: regulator@5 {
+ vdd_usb1_vbus: regulator-usb1 {
compatible = "regulator-fixed";
regulator-name = "+USB0_VBUS_SW";
regulator-min-microvolt = <5000000>;
@@ -1952,7 +1955,7 @@ vdd_usb1_vbus: regulator@5 {
vin-supply = <&vdd_5v0_sys>;
};
- vdd_usb3_vbus: regulator@6 {
+ vdd_usb3_vbus: regulator-usb3 {
compatible = "regulator-fixed";
regulator-name = "+5V_USB_HS";
regulator-min-microvolt = <5000000>;
@@ -1963,7 +1966,7 @@ vdd_usb3_vbus: regulator@6 {
vin-supply = <&vdd_5v0_sys>;
};
- vdd_3v3_lp0: regulator@7 {
+ vdd_3v3_lp0: regulator-lp0 {
compatible = "regulator-fixed";
regulator-name = "+3.3V_LP0";
regulator-min-microvolt = <3300000>;
@@ -1975,7 +1978,7 @@ vdd_3v3_lp0: regulator@7 {
vin-supply = <&vdd_3v3_sys>;
};
- vdd_hdmi_pll: regulator@8 {
+ vdd_hdmi_pll: regulator-hdmipll {
compatible = "regulator-fixed";
regulator-name = "+1.05V_RUN_AVDD_HDMI_PLL";
regulator-min-microvolt = <1050000>;
@@ -1984,7 +1987,7 @@ vdd_hdmi_pll: regulator@8 {
vin-supply = <&vdd_1v05_run>;
};
- vdd_5v0_hdmi: regulator@9 {
+ vdd_5v0_hdmi: regulator-hdmicon {
compatible = "regulator-fixed";
regulator-name = "+5V_HDMI_CON";
regulator-min-microvolt = <5000000>;
@@ -1995,7 +1998,7 @@ vdd_5v0_hdmi: regulator@9 {
};
/* Molex power connector */
- vdd_5v0_sata: regulator@10 {
+ vdd_5v0_sata: regulator-5v0sata {
compatible = "regulator-fixed";
regulator-name = "+5V_SATA";
regulator-min-microvolt = <5000000>;
@@ -2005,7 +2008,7 @@ vdd_5v0_sata: regulator@10 {
vin-supply = <&vdd_5v0_sys>;
};
- vdd_12v0_sata: regulator@11 {
+ vdd_12v0_sata: regulator-12v0sata {
compatible = "regulator-fixed";
regulator-name = "+12V_SATA";
regulator-min-microvolt = <12000000>;
@@ -2044,7 +2047,7 @@ sound {
};
thermal-zones {
- cpu {
+ cpu-thermal {
trips {
cpu-shutdown-trip {
temperature = <101000>;
@@ -2054,7 +2057,7 @@ cpu-shutdown-trip {
};
};
- mem {
+ mem-thermal {
trips {
mem-shutdown-trip {
temperature = <101000>;
@@ -2064,7 +2067,7 @@ mem-shutdown-trip {
};
};
- gpu {
+ gpu-thermal {
trips {
gpu-shutdown-trip {
temperature = <101000>;
diff --git a/arch/arm/boot/dts/tegra124-nyan-big-emc.dtsi b/arch/arm/boot/dts/tegra124-nyan-big-emc.dtsi
index a0f56cc..31b2e26 100644
--- a/arch/arm/boot/dts/tegra124-nyan-big-emc.dtsi
+++ b/arch/arm/boot/dts/tegra124-nyan-big-emc.dtsi
@@ -14,60 +14,70 @@ timing-12750000 {
clocks = <&tegra_car TEGRA124_CLK_PLL_P>;
clock-names = "emc-parent";
};
+
timing-20400000 {
clock-frequency = <20400000>;
nvidia,parent-clock-frequency = <408000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_P>;
clock-names = "emc-parent";
};
+
timing-40800000 {
clock-frequency = <40800000>;
nvidia,parent-clock-frequency = <408000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_P>;
clock-names = "emc-parent";
};
+
timing-68000000 {
clock-frequency = <68000000>;
nvidia,parent-clock-frequency = <408000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_P>;
clock-names = "emc-parent";
};
+
timing-102000000 {
clock-frequency = <102000000>;
nvidia,parent-clock-frequency = <408000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_P>;
clock-names = "emc-parent";
};
+
timing-204000000 {
clock-frequency = <204000000>;
nvidia,parent-clock-frequency = <408000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_P>;
clock-names = "emc-parent";
};
+
timing-300000000 {
clock-frequency = <300000000>;
nvidia,parent-clock-frequency = <600000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_C>;
clock-names = "emc-parent";
};
+
timing-396000000 {
clock-frequency = <396000000>;
nvidia,parent-clock-frequency = <792000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_M>;
clock-names = "emc-parent";
};
+
timing-528000000 {
clock-frequency = <528000000>;
nvidia,parent-clock-frequency = <528000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_M_UD>;
clock-names = "emc-parent";
};
+
timing-600000000 {
clock-frequency = <600000000>;
nvidia,parent-clock-frequency = <600000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_C_UD>;
clock-names = "emc-parent";
};
+
timing-792000000 {
clock-frequency = <792000000>;
nvidia,parent-clock-frequency = <792000000>;
@@ -85,60 +95,70 @@ timing-12750000 {
clocks = <&tegra_car TEGRA124_CLK_PLL_P>;
clock-names = "emc-parent";
};
+
timing-20400000 {
clock-frequency = <20400000>;
nvidia,parent-clock-frequency = <408000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_P>;
clock-names = "emc-parent";
};
+
timing-40800000 {
clock-frequency = <40800000>;
nvidia,parent-clock-frequency = <408000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_P>;
clock-names = "emc-parent";
};
+
timing-68000000 {
clock-frequency = <68000000>;
nvidia,parent-clock-frequency = <408000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_P>;
clock-names = "emc-parent";
};
+
timing-102000000 {
clock-frequency = <102000000>;
nvidia,parent-clock-frequency = <408000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_P>;
clock-names = "emc-parent";
};
+
timing-204000000 {
clock-frequency = <204000000>;
nvidia,parent-clock-frequency = <408000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_P>;
clock-names = "emc-parent";
};
+
timing-300000000 {
clock-frequency = <300000000>;
nvidia,parent-clock-frequency = <600000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_C>;
clock-names = "emc-parent";
};
+
timing-396000000 {
clock-frequency = <396000000>;
nvidia,parent-clock-frequency = <792000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_M>;
clock-names = "emc-parent";
};
+
timing-528000000 {
clock-frequency = <528000000>;
nvidia,parent-clock-frequency = <528000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_M_UD>;
clock-names = "emc-parent";
};
+
timing-600000000 {
clock-frequency = <600000000>;
nvidia,parent-clock-frequency = <600000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_C_UD>;
clock-names = "emc-parent";
};
+
timing-792000000 {
clock-frequency = <792000000>;
nvidia,parent-clock-frequency = <792000000>;
@@ -156,60 +176,70 @@ timing-12750000 {
clocks = <&tegra_car TEGRA124_CLK_PLL_P>;
clock-names = "emc-parent";
};
+
timing-20400000 {
clock-frequency = <20400000>;
nvidia,parent-clock-frequency = <408000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_P>;
clock-names = "emc-parent";
};
+
timing-40800000 {
clock-frequency = <40800000>;
nvidia,parent-clock-frequency = <408000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_P>;
clock-names = "emc-parent";
};
+
timing-68000000 {
clock-frequency = <68000000>;
nvidia,parent-clock-frequency = <408000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_P>;
clock-names = "emc-parent";
};
+
timing-102000000 {
clock-frequency = <102000000>;
nvidia,parent-clock-frequency = <408000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_P>;
clock-names = "emc-parent";
};
+
timing-204000000 {
clock-frequency = <204000000>;
nvidia,parent-clock-frequency = <408000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_P>;
clock-names = "emc-parent";
};
+
timing-300000000 {
clock-frequency = <300000000>;
nvidia,parent-clock-frequency = <600000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_C>;
clock-names = "emc-parent";
};
+
timing-396000000 {
clock-frequency = <396000000>;
nvidia,parent-clock-frequency = <792000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_M>;
clock-names = "emc-parent";
};
+
timing-528000000 {
clock-frequency = <528000000>;
nvidia,parent-clock-frequency = <528000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_M_UD>;
clock-names = "emc-parent";
};
+
timing-600000000 {
clock-frequency = <600000000>;
nvidia,parent-clock-frequency = <600000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_C_UD>;
clock-names = "emc-parent";
};
+
timing-792000000 {
clock-frequency = <792000000>;
nvidia,parent-clock-frequency = <792000000>;
@@ -219,6 +249,878 @@ timing-792000000 {
};
};
+ memory-controller@70019000 {
+ emc-timings-1 {
+ nvidia,ram-code = <1>;
+
+ timing-12750000 {
+ clock-frequency = <12750000>;
+
+ nvidia,emem-configuration = <
+ 0x40040001 /* MC_EMEM_ARB_CFG */
+ 0x8000000a /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06030203 /* MC_EMEM_ARB_DA_TURNS */
+ 0x000a0402 /* MC_EMEM_ARB_DA_COVERS */
+ 0x77e30303 /* MC_EMEM_ARB_MISC0 */
+ 0x70000f03 /* MC_EMEM_ARB_MISC1 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-20400000 {
+ clock-frequency = <20400000>;
+
+ nvidia,emem-configuration = <
+ 0x40020001 /* MC_EMEM_ARB_CFG */
+ 0x80000012 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06030203 /* MC_EMEM_ARB_DA_TURNS */
+ 0x000a0402 /* MC_EMEM_ARB_DA_COVERS */
+ 0x76230303 /* MC_EMEM_ARB_MISC0 */
+ 0x70000f03 /* MC_EMEM_ARB_MISC1 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-40800000 {
+ clock-frequency = <40800000>;
+
+ nvidia,emem-configuration = <
+ 0xa0000001 /* MC_EMEM_ARB_CFG */
+ 0x80000017 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06030203 /* MC_EMEM_ARB_DA_TURNS */
+ 0x000a0402 /* MC_EMEM_ARB_DA_COVERS */
+ 0x74a30303 /* MC_EMEM_ARB_MISC0 */
+ 0x70000f03 /* MC_EMEM_ARB_MISC1 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-68000000 {
+ clock-frequency = <68000000>;
+
+ nvidia,emem-configuration = <
+ 0x00000001 /* MC_EMEM_ARB_CFG */
+ 0x8000001e /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06030203 /* MC_EMEM_ARB_DA_TURNS */
+ 0x000a0402 /* MC_EMEM_ARB_DA_COVERS */
+ 0x74230403 /* MC_EMEM_ARB_MISC0 */
+ 0x70000f03 /* MC_EMEM_ARB_MISC1 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-102000000 {
+ clock-frequency = <102000000>;
+
+ nvidia,emem-configuration = <
+ 0x08000001 /* MC_EMEM_ARB_CFG */
+ 0x80000026 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06030203 /* MC_EMEM_ARB_DA_TURNS */
+ 0x000a0403 /* MC_EMEM_ARB_DA_COVERS */
+ 0x73c30504 /* MC_EMEM_ARB_MISC0 */
+ 0x70000f03 /* MC_EMEM_ARB_MISC1 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-204000000 {
+ clock-frequency = <204000000>;
+
+ nvidia,emem-configuration = <
+ 0x01000003 /* MC_EMEM_ARB_CFG */
+ 0x80000040 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000005 /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x00000004 /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000004 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06040203 /* MC_EMEM_ARB_DA_TURNS */
+ 0x000a0405 /* MC_EMEM_ARB_DA_COVERS */
+ 0x73840a06 /* MC_EMEM_ARB_MISC0 */
+ 0x70000f03 /* MC_EMEM_ARB_MISC1 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-300000000 {
+ clock-frequency = <300000000>;
+
+ nvidia,emem-configuration = <
+ 0x08000004 /* MC_EMEM_ARB_CFG */
+ 0x80000040 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000007 /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000004 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x00000005 /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000007 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000004 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06040202 /* MC_EMEM_ARB_DA_TURNS */
+ 0x000b0607 /* MC_EMEM_ARB_DA_COVERS */
+ 0x77450e08 /* MC_EMEM_ARB_MISC0 */
+ 0x70000f03 /* MC_EMEM_ARB_MISC1 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-396000000 {
+ clock-frequency = <396000000>;
+
+ nvidia,emem-configuration = <
+ 0x0f000005 /* MC_EMEM_ARB_CFG */
+ 0x80000040 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000009 /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000005 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x00000007 /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000004 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06040202 /* MC_EMEM_ARB_DA_TURNS */
+ 0x000d0709 /* MC_EMEM_ARB_DA_COVERS */
+ 0x7586120a /* MC_EMEM_ARB_MISC0 */
+ 0x70000f03 /* MC_EMEM_ARB_MISC1 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-528000000 {
+ clock-frequency = <528000000>;
+
+ nvidia,emem-configuration = <
+ 0x0f000007 /* MC_EMEM_ARB_CFG */
+ 0x80000040 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_RP */
+ 0x0000000d /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x0000000a /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000009 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000005 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06050202 /* MC_EMEM_ARB_DA_TURNS */
+ 0x0010090d /* MC_EMEM_ARB_DA_COVERS */
+ 0x7428180e /* MC_EMEM_ARB_MISC0 */
+ 0x70000f03 /* MC_EMEM_ARB_MISC1 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-600000000 {
+ clock-frequency = <600000000>;
+
+ nvidia,emem-configuration = <
+ 0x00000009 /* MC_EMEM_ARB_CFG */
+ 0x80000040 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000004 /* MC_EMEM_ARB_TIMING_RP */
+ 0x0000000e /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000009 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x0000000b /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x0000000b /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000005 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000007 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x07050202 /* MC_EMEM_ARB_DA_TURNS */
+ 0x00130b0e /* MC_EMEM_ARB_DA_COVERS */
+ 0x73a91b0f /* MC_EMEM_ARB_MISC0 */
+ 0x70000f03 /* MC_EMEM_ARB_MISC1 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-792000000 {
+ clock-frequency = <792000000>;
+
+ nvidia,emem-configuration = <
+ 0x0e00000b /* MC_EMEM_ARB_CFG */
+ 0x80000040 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000004 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000005 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000013 /* MC_EMEM_ARB_TIMING_RC */
+ 0x0000000c /* MC_EMEM_ARB_TIMING_RAS */
+ 0x0000000f /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x0000000c /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x08060202 /* MC_EMEM_ARB_DA_TURNS */
+ 0x00160d13 /* MC_EMEM_ARB_DA_COVERS */
+ 0x734c2414 /* MC_EMEM_ARB_MISC0 */
+ 0x70000f02 /* MC_EMEM_ARB_MISC1 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+ };
+
+ emc-timings-4 {
+ nvidia,ram-code = <4>;
+
+ timing-12750000 {
+ clock-frequency = <12750000>;
+
+ nvidia,emem-configuration = <
+ 0x40040001 /* MC_EMEM_ARB_CFG */
+ 0x8000000a /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06030203 /* MC_EMEM_ARB_DA_TURNS */
+ 0x000a0502 /* MC_EMEM_ARB_DA_COVERS */
+ 0x77e30303 /* MC_EMEM_ARB_MISC0 */
+ 0x70000f03 /* MC_EMEM_ARB_MISC1 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-20400000 {
+ clock-frequency = <20400000>;
+
+ nvidia,emem-configuration = <
+ 0x40020001 /* MC_EMEM_ARB_CFG */
+ 0x80000012 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06030203 /* MC_EMEM_ARB_DA_TURNS */
+ 0x000a0502 /* MC_EMEM_ARB_DA_COVERS */
+ 0x77430303 /* MC_EMEM_ARB_MISC0 */
+ 0x70000f03 /* MC_EMEM_ARB_MISC1 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-40800000 {
+ clock-frequency = <40800000>;
+
+ nvidia,emem-configuration = <
+ 0xa0000001 /* MC_EMEM_ARB_CFG */
+ 0x80000017 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06030203 /* MC_EMEM_ARB_DA_TURNS */
+ 0x000a0502 /* MC_EMEM_ARB_DA_COVERS */
+ 0x75e30303 /* MC_EMEM_ARB_MISC0 */
+ 0x70000f03 /* MC_EMEM_ARB_MISC1 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-68000000 {
+ clock-frequency = <68000000>;
+
+ nvidia,emem-configuration = <
+ 0x00000001 /* MC_EMEM_ARB_CFG */
+ 0x8000001e /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06030203 /* MC_EMEM_ARB_DA_TURNS */
+ 0x000a0502 /* MC_EMEM_ARB_DA_COVERS */
+ 0x75430403 /* MC_EMEM_ARB_MISC0 */
+ 0x70000f03 /* MC_EMEM_ARB_MISC1 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-102000000 {
+ clock-frequency = <102000000>;
+
+ nvidia,emem-configuration = <
+ 0x08000001 /* MC_EMEM_ARB_CFG */
+ 0x80000026 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06030203 /* MC_EMEM_ARB_DA_TURNS */
+ 0x000a0503 /* MC_EMEM_ARB_DA_COVERS */
+ 0x74e30504 /* MC_EMEM_ARB_MISC0 */
+ 0x70000f03 /* MC_EMEM_ARB_MISC1 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-204000000 {
+ clock-frequency = <204000000>;
+
+ nvidia,emem-configuration = <
+ 0x01000003 /* MC_EMEM_ARB_CFG */
+ 0x80000040 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000004 /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x00000004 /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000004 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06040203 /* MC_EMEM_ARB_DA_TURNS */
+ 0x000a0504 /* MC_EMEM_ARB_DA_COVERS */
+ 0x74a40a05 /* MC_EMEM_ARB_MISC0 */
+ 0x70000f03 /* MC_EMEM_ARB_MISC1 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-300000000 {
+ clock-frequency = <300000000>;
+
+ nvidia,emem-configuration = <
+ 0x08000004 /* MC_EMEM_ARB_CFG */
+ 0x80000040 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000007 /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000004 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x00000005 /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000007 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000004 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06040202 /* MC_EMEM_ARB_DA_TURNS */
+ 0x000b0607 /* MC_EMEM_ARB_DA_COVERS */
+ 0x77450e08 /* MC_EMEM_ARB_MISC0 */
+ 0x70000f03 /* MC_EMEM_ARB_MISC1 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-396000000 {
+ clock-frequency = <396000000>;
+
+ nvidia,emem-configuration = <
+ 0x0f000005 /* MC_EMEM_ARB_CFG */
+ 0x80000040 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000009 /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000005 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x00000007 /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000004 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06040202 /* MC_EMEM_ARB_DA_TURNS */
+ 0x000d0709 /* MC_EMEM_ARB_DA_COVERS */
+ 0x7586120a /* MC_EMEM_ARB_MISC0 */
+ 0x70000f03 /* MC_EMEM_ARB_MISC1 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-528000000 {
+ clock-frequency = <528000000>;
+
+ nvidia,emem-configuration = <
+ 0x0f000007 /* MC_EMEM_ARB_CFG */
+ 0x80000040 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_RP */
+ 0x0000000c /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000007 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x0000000a /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000009 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000005 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06050202 /* MC_EMEM_ARB_DA_TURNS */
+ 0x0010090c /* MC_EMEM_ARB_DA_COVERS */
+ 0x7488180d /* MC_EMEM_ARB_MISC0 */
+ 0x70000f03 /* MC_EMEM_ARB_MISC1 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-600000000 {
+ clock-frequency = <600000000>;
+
+ nvidia,emem-configuration = <
+ 0x00000009 /* MC_EMEM_ARB_CFG */
+ 0x80000040 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000004 /* MC_EMEM_ARB_TIMING_RP */
+ 0x0000000e /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000009 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x0000000b /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x0000000b /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000005 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000007 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x07050202 /* MC_EMEM_ARB_DA_TURNS */
+ 0x00130b0e /* MC_EMEM_ARB_DA_COVERS */
+ 0x74691b0f /* MC_EMEM_ARB_MISC0 */
+ 0x70000f03 /* MC_EMEM_ARB_MISC1 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-792000000 {
+ clock-frequency = <792000000>;
+
+ nvidia,emem-configuration = <
+ 0x0e00000b /* MC_EMEM_ARB_CFG */
+ 0x80000040 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000004 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000005 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000013 /* MC_EMEM_ARB_TIMING_RC */
+ 0x0000000c /* MC_EMEM_ARB_TIMING_RAS */
+ 0x0000000f /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x0000000c /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x08060202 /* MC_EMEM_ARB_DA_TURNS */
+ 0x00170e13 /* MC_EMEM_ARB_DA_COVERS */
+ 0x746c2414 /* MC_EMEM_ARB_MISC0 */
+ 0x70000f02 /* MC_EMEM_ARB_MISC1 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+ };
+
+ emc-timings-6 {
+ nvidia,ram-code = <6>;
+
+ timing-12750000 {
+ clock-frequency = <12750000>;
+
+ nvidia,emem-configuration = <
+ 0x40040001 /* MC_EMEM_ARB_CFG */
+ 0x8000000a /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06030203 /* MC_EMEM_ARB_DA_TURNS */
+ 0x000a0402 /* MC_EMEM_ARB_DA_COVERS */
+ 0x77e30303 /* MC_EMEM_ARB_MISC0 */
+ 0x70000f03 /* MC_EMEM_ARB_MISC1 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-20400000 {
+ clock-frequency = <20400000>;
+
+ nvidia,emem-configuration = <
+ 0x40020001 /* MC_EMEM_ARB_CFG */
+ 0x80000012 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06030203 /* MC_EMEM_ARB_DA_TURNS */
+ 0x000a0402 /* MC_EMEM_ARB_DA_COVERS */
+ 0x76230303 /* MC_EMEM_ARB_MISC0 */
+ 0x70000f03 /* MC_EMEM_ARB_MISC1 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-40800000 {
+ clock-frequency = <40800000>;
+
+ nvidia,emem-configuration = <
+ 0xa0000001 /* MC_EMEM_ARB_CFG */
+ 0x80000017 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06030203 /* MC_EMEM_ARB_DA_TURNS */
+ 0x000a0402 /* MC_EMEM_ARB_DA_COVERS */
+ 0x74a30303 /* MC_EMEM_ARB_MISC0 */
+ 0x70000f03 /* MC_EMEM_ARB_MISC1 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-68000000 {
+ clock-frequency = <68000000>;
+
+ nvidia,emem-configuration = <
+ 0x00000001 /* MC_EMEM_ARB_CFG */
+ 0x8000001e /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06030203 /* MC_EMEM_ARB_DA_TURNS */
+ 0x000a0402 /* MC_EMEM_ARB_DA_COVERS */
+ 0x74230403 /* MC_EMEM_ARB_MISC0 */
+ 0x70000f03 /* MC_EMEM_ARB_MISC1 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-102000000 {
+ clock-frequency = <102000000>;
+
+ nvidia,emem-configuration = <
+ 0x08000001 /* MC_EMEM_ARB_CFG */
+ 0x80000026 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06030203 /* MC_EMEM_ARB_DA_TURNS */
+ 0x000a0403 /* MC_EMEM_ARB_DA_COVERS */
+ 0x73c30504 /* MC_EMEM_ARB_MISC0 */
+ 0x70000f03 /* MC_EMEM_ARB_MISC1 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-204000000 {
+ clock-frequency = <204000000>;
+
+ nvidia,emem-configuration = <
+ 0x01000003 /* MC_EMEM_ARB_CFG */
+ 0x80000040 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000005 /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x00000004 /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000004 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06040203 /* MC_EMEM_ARB_DA_TURNS */
+ 0x000a0405 /* MC_EMEM_ARB_DA_COVERS */
+ 0x73840a06 /* MC_EMEM_ARB_MISC0 */
+ 0x70000f03 /* MC_EMEM_ARB_MISC1 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-300000000 {
+ clock-frequency = <300000000>;
+
+ nvidia,emem-configuration = <
+ 0x08000004 /* MC_EMEM_ARB_CFG */
+ 0x80000040 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000007 /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000004 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x00000005 /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000007 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000004 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06040202 /* MC_EMEM_ARB_DA_TURNS */
+ 0x000b0607 /* MC_EMEM_ARB_DA_COVERS */
+ 0x77450e08 /* MC_EMEM_ARB_MISC0 */
+ 0x70000f03 /* MC_EMEM_ARB_MISC1 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-396000000 {
+ clock-frequency = <396000000>;
+
+ nvidia,emem-configuration = <
+ 0x0f000005 /* MC_EMEM_ARB_CFG */
+ 0x80000040 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000009 /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000005 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x00000007 /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000004 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06040202 /* MC_EMEM_ARB_DA_TURNS */
+ 0x000d0709 /* MC_EMEM_ARB_DA_COVERS */
+ 0x7586120a /* MC_EMEM_ARB_MISC0 */
+ 0x70000f03 /* MC_EMEM_ARB_MISC1 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-528000000 {
+ clock-frequency = <528000000>;
+
+ nvidia,emem-configuration = <
+ 0x0f000007 /* MC_EMEM_ARB_CFG */
+ 0x80000040 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_RP */
+ 0x0000000d /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x0000000a /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000009 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000005 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06050202 /* MC_EMEM_ARB_DA_TURNS */
+ 0x0010090d /* MC_EMEM_ARB_DA_COVERS */
+ 0x7428180e /* MC_EMEM_ARB_MISC0 */
+ 0x70000f03 /* MC_EMEM_ARB_MISC1 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-600000000 {
+ clock-frequency = <600000000>;
+
+ nvidia,emem-configuration = <
+ 0x00000009 /* MC_EMEM_ARB_CFG */
+ 0x80000040 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000004 /* MC_EMEM_ARB_TIMING_RP */
+ 0x0000000e /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000009 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x0000000b /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x0000000b /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000005 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000007 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x07050202 /* MC_EMEM_ARB_DA_TURNS */
+ 0x00130b0e /* MC_EMEM_ARB_DA_COVERS */
+ 0x73a91b0f /* MC_EMEM_ARB_MISC0 */
+ 0x70000f03 /* MC_EMEM_ARB_MISC1 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-792000000 {
+ clock-frequency = <792000000>;
+
+ nvidia,emem-configuration = <
+ 0x0e00000b /* MC_EMEM_ARB_CFG */
+ 0x80000040 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000004 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000005 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000013 /* MC_EMEM_ARB_TIMING_RC */
+ 0x0000000c /* MC_EMEM_ARB_TIMING_RAS */
+ 0x0000000f /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x0000000c /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x08060202 /* MC_EMEM_ARB_DA_TURNS */
+ 0x00160d13 /* MC_EMEM_ARB_DA_COVERS */
+ 0x734c2414 /* MC_EMEM_ARB_MISC0 */
+ 0x70000f02 /* MC_EMEM_ARB_MISC1 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+ };
+ };
+
external-memory-controller@7001b000 {
emc-timings-1 {
nvidia,ram-code = <1>;
@@ -5776,886 +6678,14 @@ timing-792000000 {
};
};
};
-
- memory-controller@70019000 {
- emc-timings-1 {
- nvidia,ram-code = <1>;
-
- timing-12750000 {
- clock-frequency = <12750000>;
-
- nvidia,emem-configuration = <
- 0x40040001 /* MC_EMEM_ARB_CFG */
- 0x8000000a /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RC */
- 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
- 0x00000002 /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000003 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000003 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06030203 /* MC_EMEM_ARB_DA_TURNS */
- 0x000a0402 /* MC_EMEM_ARB_DA_COVERS */
- 0x77e30303 /* MC_EMEM_ARB_MISC0 */
- 0x70000f03 /* MC_EMEM_ARB_MISC1 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
-
- timing-20400000 {
- clock-frequency = <20400000>;
-
- nvidia,emem-configuration = <
- 0x40020001 /* MC_EMEM_ARB_CFG */
- 0x80000012 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RC */
- 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
- 0x00000002 /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000003 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000003 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06030203 /* MC_EMEM_ARB_DA_TURNS */
- 0x000a0402 /* MC_EMEM_ARB_DA_COVERS */
- 0x76230303 /* MC_EMEM_ARB_MISC0 */
- 0x70000f03 /* MC_EMEM_ARB_MISC1 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
-
- timing-40800000 {
- clock-frequency = <40800000>;
-
- nvidia,emem-configuration = <
- 0xa0000001 /* MC_EMEM_ARB_CFG */
- 0x80000017 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RC */
- 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
- 0x00000002 /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000003 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000003 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06030203 /* MC_EMEM_ARB_DA_TURNS */
- 0x000a0402 /* MC_EMEM_ARB_DA_COVERS */
- 0x74a30303 /* MC_EMEM_ARB_MISC0 */
- 0x70000f03 /* MC_EMEM_ARB_MISC1 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
-
- timing-68000000 {
- clock-frequency = <68000000>;
-
- nvidia,emem-configuration = <
- 0x00000001 /* MC_EMEM_ARB_CFG */
- 0x8000001e /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RC */
- 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
- 0x00000002 /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000003 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000003 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06030203 /* MC_EMEM_ARB_DA_TURNS */
- 0x000a0402 /* MC_EMEM_ARB_DA_COVERS */
- 0x74230403 /* MC_EMEM_ARB_MISC0 */
- 0x70000f03 /* MC_EMEM_ARB_MISC1 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
-
- timing-102000000 {
- clock-frequency = <102000000>;
-
- nvidia,emem-configuration = <
- 0x08000001 /* MC_EMEM_ARB_CFG */
- 0x80000026 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000003 /* MC_EMEM_ARB_TIMING_RC */
- 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
- 0x00000002 /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000003 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000003 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06030203 /* MC_EMEM_ARB_DA_TURNS */
- 0x000a0403 /* MC_EMEM_ARB_DA_COVERS */
- 0x73c30504 /* MC_EMEM_ARB_MISC0 */
- 0x70000f03 /* MC_EMEM_ARB_MISC1 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
-
- timing-204000000 {
- clock-frequency = <204000000>;
-
- nvidia,emem-configuration = <
- 0x01000003 /* MC_EMEM_ARB_CFG */
- 0x80000040 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000005 /* MC_EMEM_ARB_TIMING_RC */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RAS */
- 0x00000004 /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000003 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000004 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06040203 /* MC_EMEM_ARB_DA_TURNS */
- 0x000a0405 /* MC_EMEM_ARB_DA_COVERS */
- 0x73840a06 /* MC_EMEM_ARB_MISC0 */
- 0x70000f03 /* MC_EMEM_ARB_MISC1 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
-
- timing-300000000 {
- clock-frequency = <300000000>;
-
- nvidia,emem-configuration = <
- 0x08000004 /* MC_EMEM_ARB_CFG */
- 0x80000040 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000007 /* MC_EMEM_ARB_TIMING_RC */
- 0x00000004 /* MC_EMEM_ARB_TIMING_RAS */
- 0x00000005 /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000007 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000004 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06040202 /* MC_EMEM_ARB_DA_TURNS */
- 0x000b0607 /* MC_EMEM_ARB_DA_COVERS */
- 0x77450e08 /* MC_EMEM_ARB_MISC0 */
- 0x70000f03 /* MC_EMEM_ARB_MISC1 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
-
- timing-396000000 {
- clock-frequency = <396000000>;
-
- nvidia,emem-configuration = <
- 0x0f000005 /* MC_EMEM_ARB_CFG */
- 0x80000040 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000009 /* MC_EMEM_ARB_TIMING_RC */
- 0x00000005 /* MC_EMEM_ARB_TIMING_RAS */
- 0x00000007 /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000004 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06040202 /* MC_EMEM_ARB_DA_TURNS */
- 0x000d0709 /* MC_EMEM_ARB_DA_COVERS */
- 0x7586120a /* MC_EMEM_ARB_MISC0 */
- 0x70000f03 /* MC_EMEM_ARB_MISC1 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
-
- timing-528000000 {
- clock-frequency = <528000000>;
-
- nvidia,emem-configuration = <
- 0x0f000007 /* MC_EMEM_ARB_CFG */
- 0x80000040 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000003 /* MC_EMEM_ARB_TIMING_RP */
- 0x0000000d /* MC_EMEM_ARB_TIMING_RC */
- 0x00000008 /* MC_EMEM_ARB_TIMING_RAS */
- 0x0000000a /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000009 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000005 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06050202 /* MC_EMEM_ARB_DA_TURNS */
- 0x0010090d /* MC_EMEM_ARB_DA_COVERS */
- 0x7428180e /* MC_EMEM_ARB_MISC0 */
- 0x70000f03 /* MC_EMEM_ARB_MISC1 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
-
- timing-600000000 {
- clock-frequency = <600000000>;
-
- nvidia,emem-configuration = <
- 0x00000009 /* MC_EMEM_ARB_CFG */
- 0x80000040 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000003 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000004 /* MC_EMEM_ARB_TIMING_RP */
- 0x0000000e /* MC_EMEM_ARB_TIMING_RC */
- 0x00000009 /* MC_EMEM_ARB_TIMING_RAS */
- 0x0000000b /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x0000000b /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000005 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000007 /* MC_EMEM_ARB_TIMING_W2R */
- 0x07050202 /* MC_EMEM_ARB_DA_TURNS */
- 0x00130b0e /* MC_EMEM_ARB_DA_COVERS */
- 0x73a91b0f /* MC_EMEM_ARB_MISC0 */
- 0x70000f03 /* MC_EMEM_ARB_MISC1 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
-
- timing-792000000 {
- clock-frequency = <792000000>;
-
- nvidia,emem-configuration = <
- 0x0e00000b /* MC_EMEM_ARB_CFG */
- 0x80000040 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000004 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000005 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000013 /* MC_EMEM_ARB_TIMING_RC */
- 0x0000000c /* MC_EMEM_ARB_TIMING_RAS */
- 0x0000000f /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x0000000c /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000008 /* MC_EMEM_ARB_TIMING_W2R */
- 0x08060202 /* MC_EMEM_ARB_DA_TURNS */
- 0x00160d13 /* MC_EMEM_ARB_DA_COVERS */
- 0x734c2414 /* MC_EMEM_ARB_MISC0 */
- 0x70000f02 /* MC_EMEM_ARB_MISC1 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
- };
-
- emc-timings-4 {
- nvidia,ram-code = <4>;
-
- timing-12750000 {
- clock-frequency = <12750000>;
-
- nvidia,emem-configuration = <
- 0x40040001 /* MC_EMEM_ARB_CFG */
- 0x8000000a /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RC */
- 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
- 0x00000002 /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000003 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000003 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06030203 /* MC_EMEM_ARB_DA_TURNS */
- 0x000a0502 /* MC_EMEM_ARB_DA_COVERS */
- 0x77e30303 /* MC_EMEM_ARB_MISC0 */
- 0x70000f03 /* MC_EMEM_ARB_MISC1 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
-
- timing-20400000 {
- clock-frequency = <20400000>;
-
- nvidia,emem-configuration = <
- 0x40020001 /* MC_EMEM_ARB_CFG */
- 0x80000012 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RC */
- 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
- 0x00000002 /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000003 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000003 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06030203 /* MC_EMEM_ARB_DA_TURNS */
- 0x000a0502 /* MC_EMEM_ARB_DA_COVERS */
- 0x77430303 /* MC_EMEM_ARB_MISC0 */
- 0x70000f03 /* MC_EMEM_ARB_MISC1 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
-
- timing-40800000 {
- clock-frequency = <40800000>;
-
- nvidia,emem-configuration = <
- 0xa0000001 /* MC_EMEM_ARB_CFG */
- 0x80000017 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RC */
- 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
- 0x00000002 /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000003 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000003 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06030203 /* MC_EMEM_ARB_DA_TURNS */
- 0x000a0502 /* MC_EMEM_ARB_DA_COVERS */
- 0x75e30303 /* MC_EMEM_ARB_MISC0 */
- 0x70000f03 /* MC_EMEM_ARB_MISC1 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
-
- timing-68000000 {
- clock-frequency = <68000000>;
-
- nvidia,emem-configuration = <
- 0x00000001 /* MC_EMEM_ARB_CFG */
- 0x8000001e /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RC */
- 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
- 0x00000002 /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000003 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000003 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06030203 /* MC_EMEM_ARB_DA_TURNS */
- 0x000a0502 /* MC_EMEM_ARB_DA_COVERS */
- 0x75430403 /* MC_EMEM_ARB_MISC0 */
- 0x70000f03 /* MC_EMEM_ARB_MISC1 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
-
- timing-102000000 {
- clock-frequency = <102000000>;
-
- nvidia,emem-configuration = <
- 0x08000001 /* MC_EMEM_ARB_CFG */
- 0x80000026 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000003 /* MC_EMEM_ARB_TIMING_RC */
- 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
- 0x00000002 /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000003 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000003 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06030203 /* MC_EMEM_ARB_DA_TURNS */
- 0x000a0503 /* MC_EMEM_ARB_DA_COVERS */
- 0x74e30504 /* MC_EMEM_ARB_MISC0 */
- 0x70000f03 /* MC_EMEM_ARB_MISC1 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
-
- timing-204000000 {
- clock-frequency = <204000000>;
-
- nvidia,emem-configuration = <
- 0x01000003 /* MC_EMEM_ARB_CFG */
- 0x80000040 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000004 /* MC_EMEM_ARB_TIMING_RC */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RAS */
- 0x00000004 /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000003 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000004 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06040203 /* MC_EMEM_ARB_DA_TURNS */
- 0x000a0504 /* MC_EMEM_ARB_DA_COVERS */
- 0x74a40a05 /* MC_EMEM_ARB_MISC0 */
- 0x70000f03 /* MC_EMEM_ARB_MISC1 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
-
- timing-300000000 {
- clock-frequency = <300000000>;
-
- nvidia,emem-configuration = <
- 0x08000004 /* MC_EMEM_ARB_CFG */
- 0x80000040 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000007 /* MC_EMEM_ARB_TIMING_RC */
- 0x00000004 /* MC_EMEM_ARB_TIMING_RAS */
- 0x00000005 /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000007 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000004 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06040202 /* MC_EMEM_ARB_DA_TURNS */
- 0x000b0607 /* MC_EMEM_ARB_DA_COVERS */
- 0x77450e08 /* MC_EMEM_ARB_MISC0 */
- 0x70000f03 /* MC_EMEM_ARB_MISC1 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
-
- timing-396000000 {
- clock-frequency = <396000000>;
-
- nvidia,emem-configuration = <
- 0x0f000005 /* MC_EMEM_ARB_CFG */
- 0x80000040 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000009 /* MC_EMEM_ARB_TIMING_RC */
- 0x00000005 /* MC_EMEM_ARB_TIMING_RAS */
- 0x00000007 /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000004 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06040202 /* MC_EMEM_ARB_DA_TURNS */
- 0x000d0709 /* MC_EMEM_ARB_DA_COVERS */
- 0x7586120a /* MC_EMEM_ARB_MISC0 */
- 0x70000f03 /* MC_EMEM_ARB_MISC1 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
-
- timing-528000000 {
- clock-frequency = <528000000>;
-
- nvidia,emem-configuration = <
- 0x0f000007 /* MC_EMEM_ARB_CFG */
- 0x80000040 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000003 /* MC_EMEM_ARB_TIMING_RP */
- 0x0000000c /* MC_EMEM_ARB_TIMING_RC */
- 0x00000007 /* MC_EMEM_ARB_TIMING_RAS */
- 0x0000000a /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000009 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000005 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06050202 /* MC_EMEM_ARB_DA_TURNS */
- 0x0010090c /* MC_EMEM_ARB_DA_COVERS */
- 0x7488180d /* MC_EMEM_ARB_MISC0 */
- 0x70000f03 /* MC_EMEM_ARB_MISC1 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
-
- timing-600000000 {
- clock-frequency = <600000000>;
-
- nvidia,emem-configuration = <
- 0x00000009 /* MC_EMEM_ARB_CFG */
- 0x80000040 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000003 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000004 /* MC_EMEM_ARB_TIMING_RP */
- 0x0000000e /* MC_EMEM_ARB_TIMING_RC */
- 0x00000009 /* MC_EMEM_ARB_TIMING_RAS */
- 0x0000000b /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x0000000b /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000005 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000007 /* MC_EMEM_ARB_TIMING_W2R */
- 0x07050202 /* MC_EMEM_ARB_DA_TURNS */
- 0x00130b0e /* MC_EMEM_ARB_DA_COVERS */
- 0x74691b0f /* MC_EMEM_ARB_MISC0 */
- 0x70000f03 /* MC_EMEM_ARB_MISC1 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
-
- timing-792000000 {
- clock-frequency = <792000000>;
-
- nvidia,emem-configuration = <
- 0x0e00000b /* MC_EMEM_ARB_CFG */
- 0x80000040 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000004 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000005 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000013 /* MC_EMEM_ARB_TIMING_RC */
- 0x0000000c /* MC_EMEM_ARB_TIMING_RAS */
- 0x0000000f /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x0000000c /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000008 /* MC_EMEM_ARB_TIMING_W2R */
- 0x08060202 /* MC_EMEM_ARB_DA_TURNS */
- 0x00170e13 /* MC_EMEM_ARB_DA_COVERS */
- 0x746c2414 /* MC_EMEM_ARB_MISC0 */
- 0x70000f02 /* MC_EMEM_ARB_MISC1 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
- };
-
- emc-timings-6 {
- nvidia,ram-code = <6>;
-
- timing-12750000 {
- clock-frequency = <12750000>;
-
- nvidia,emem-configuration = <
- 0x40040001 /* MC_EMEM_ARB_CFG */
- 0x8000000a /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RC */
- 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
- 0x00000002 /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000003 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000003 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06030203 /* MC_EMEM_ARB_DA_TURNS */
- 0x000a0402 /* MC_EMEM_ARB_DA_COVERS */
- 0x77e30303 /* MC_EMEM_ARB_MISC0 */
- 0x70000f03 /* MC_EMEM_ARB_MISC1 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
-
- timing-20400000 {
- clock-frequency = <20400000>;
-
- nvidia,emem-configuration = <
- 0x40020001 /* MC_EMEM_ARB_CFG */
- 0x80000012 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RC */
- 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
- 0x00000002 /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000003 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000003 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06030203 /* MC_EMEM_ARB_DA_TURNS */
- 0x000a0402 /* MC_EMEM_ARB_DA_COVERS */
- 0x76230303 /* MC_EMEM_ARB_MISC0 */
- 0x70000f03 /* MC_EMEM_ARB_MISC1 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
-
- timing-40800000 {
- clock-frequency = <40800000>;
-
- nvidia,emem-configuration = <
- 0xa0000001 /* MC_EMEM_ARB_CFG */
- 0x80000017 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RC */
- 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
- 0x00000002 /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000003 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000003 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06030203 /* MC_EMEM_ARB_DA_TURNS */
- 0x000a0402 /* MC_EMEM_ARB_DA_COVERS */
- 0x74a30303 /* MC_EMEM_ARB_MISC0 */
- 0x70000f03 /* MC_EMEM_ARB_MISC1 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
-
- timing-68000000 {
- clock-frequency = <68000000>;
-
- nvidia,emem-configuration = <
- 0x00000001 /* MC_EMEM_ARB_CFG */
- 0x8000001e /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RC */
- 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
- 0x00000002 /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000003 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000003 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06030203 /* MC_EMEM_ARB_DA_TURNS */
- 0x000a0402 /* MC_EMEM_ARB_DA_COVERS */
- 0x74230403 /* MC_EMEM_ARB_MISC0 */
- 0x70000f03 /* MC_EMEM_ARB_MISC1 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
-
- timing-102000000 {
- clock-frequency = <102000000>;
-
- nvidia,emem-configuration = <
- 0x08000001 /* MC_EMEM_ARB_CFG */
- 0x80000026 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000003 /* MC_EMEM_ARB_TIMING_RC */
- 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
- 0x00000002 /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000003 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000003 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06030203 /* MC_EMEM_ARB_DA_TURNS */
- 0x000a0403 /* MC_EMEM_ARB_DA_COVERS */
- 0x73c30504 /* MC_EMEM_ARB_MISC0 */
- 0x70000f03 /* MC_EMEM_ARB_MISC1 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
-
- timing-204000000 {
- clock-frequency = <204000000>;
-
- nvidia,emem-configuration = <
- 0x01000003 /* MC_EMEM_ARB_CFG */
- 0x80000040 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000005 /* MC_EMEM_ARB_TIMING_RC */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RAS */
- 0x00000004 /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000003 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000004 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06040203 /* MC_EMEM_ARB_DA_TURNS */
- 0x000a0405 /* MC_EMEM_ARB_DA_COVERS */
- 0x73840a06 /* MC_EMEM_ARB_MISC0 */
- 0x70000f03 /* MC_EMEM_ARB_MISC1 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
-
- timing-300000000 {
- clock-frequency = <300000000>;
-
- nvidia,emem-configuration = <
- 0x08000004 /* MC_EMEM_ARB_CFG */
- 0x80000040 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000007 /* MC_EMEM_ARB_TIMING_RC */
- 0x00000004 /* MC_EMEM_ARB_TIMING_RAS */
- 0x00000005 /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000007 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000004 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06040202 /* MC_EMEM_ARB_DA_TURNS */
- 0x000b0607 /* MC_EMEM_ARB_DA_COVERS */
- 0x77450e08 /* MC_EMEM_ARB_MISC0 */
- 0x70000f03 /* MC_EMEM_ARB_MISC1 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
-
- timing-396000000 {
- clock-frequency = <396000000>;
-
- nvidia,emem-configuration = <
- 0x0f000005 /* MC_EMEM_ARB_CFG */
- 0x80000040 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000009 /* MC_EMEM_ARB_TIMING_RC */
- 0x00000005 /* MC_EMEM_ARB_TIMING_RAS */
- 0x00000007 /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000004 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06040202 /* MC_EMEM_ARB_DA_TURNS */
- 0x000d0709 /* MC_EMEM_ARB_DA_COVERS */
- 0x7586120a /* MC_EMEM_ARB_MISC0 */
- 0x70000f03 /* MC_EMEM_ARB_MISC1 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
-
- timing-528000000 {
- clock-frequency = <528000000>;
-
- nvidia,emem-configuration = <
- 0x0f000007 /* MC_EMEM_ARB_CFG */
- 0x80000040 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000003 /* MC_EMEM_ARB_TIMING_RP */
- 0x0000000d /* MC_EMEM_ARB_TIMING_RC */
- 0x00000008 /* MC_EMEM_ARB_TIMING_RAS */
- 0x0000000a /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000009 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000005 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06050202 /* MC_EMEM_ARB_DA_TURNS */
- 0x0010090d /* MC_EMEM_ARB_DA_COVERS */
- 0x7428180e /* MC_EMEM_ARB_MISC0 */
- 0x70000f03 /* MC_EMEM_ARB_MISC1 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
-
- timing-600000000 {
- clock-frequency = <600000000>;
-
- nvidia,emem-configuration = <
- 0x00000009 /* MC_EMEM_ARB_CFG */
- 0x80000040 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000003 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000004 /* MC_EMEM_ARB_TIMING_RP */
- 0x0000000e /* MC_EMEM_ARB_TIMING_RC */
- 0x00000009 /* MC_EMEM_ARB_TIMING_RAS */
- 0x0000000b /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x0000000b /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000005 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000007 /* MC_EMEM_ARB_TIMING_W2R */
- 0x07050202 /* MC_EMEM_ARB_DA_TURNS */
- 0x00130b0e /* MC_EMEM_ARB_DA_COVERS */
- 0x73a91b0f /* MC_EMEM_ARB_MISC0 */
- 0x70000f03 /* MC_EMEM_ARB_MISC1 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
-
- timing-792000000 {
- clock-frequency = <792000000>;
-
- nvidia,emem-configuration = <
- 0x0e00000b /* MC_EMEM_ARB_CFG */
- 0x80000040 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000004 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000005 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000013 /* MC_EMEM_ARB_TIMING_RC */
- 0x0000000c /* MC_EMEM_ARB_TIMING_RAS */
- 0x0000000f /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x0000000c /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000008 /* MC_EMEM_ARB_TIMING_W2R */
- 0x08060202 /* MC_EMEM_ARB_DA_TURNS */
- 0x00160d13 /* MC_EMEM_ARB_DA_COVERS */
- 0x734c2414 /* MC_EMEM_ARB_MISC0 */
- 0x70000f02 /* MC_EMEM_ARB_MISC1 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
- };
- };
};
&emc_icc_dvfs_opp_table {
- /delete-node/ opp@924000000,1100;
- /delete-node/ opp@1200000000,1100;
+ /delete-node/ opp-924000000-1100;
+ /delete-node/ opp-1200000000-1100;
};
&emc_bw_dfs_opp_table {
- /delete-node/ opp@924000000;
- /delete-node/ opp@1200000000;
+ /delete-node/ opp-924000000;
+ /delete-node/ opp-1200000000;
};
diff --git a/arch/arm/boot/dts/tegra124-nyan-big-fhd.dts b/arch/arm/boot/dts/tegra124-nyan-big-fhd.dts
new file mode 100644
index 0000000..d35fb79
--- /dev/null
+++ b/arch/arm/boot/dts/tegra124-nyan-big-fhd.dts
@@ -0,0 +1,11 @@
+// SPDX-License-Identifier: GPL-2.0
+/dts-v1/;
+
+#include "tegra124-nyan-big.dts"
+
+/ {
+ /* Version of Nyan Big with 1080p panel */
+ panel {
+ compatible = "auo,b133htn01";
+ };
+};
diff --git a/arch/arm/boot/dts/tegra124-nyan-blaze-emc.dtsi b/arch/arm/boot/dts/tegra124-nyan-blaze-emc.dtsi
index 35c9873..2ce1b12 100644
--- a/arch/arm/boot/dts/tegra124-nyan-blaze-emc.dtsi
+++ b/arch/arm/boot/dts/tegra124-nyan-blaze-emc.dtsi
@@ -10,55 +10,65 @@ timing-12750000 {
clocks = <&tegra_car TEGRA124_CLK_PLL_P>;
clock-names = "emc-parent";
};
+
timing-20400000 {
clock-frequency = <20400000>;
nvidia,parent-clock-frequency = <408000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_P>;
clock-names = "emc-parent";
};
+
timing-40800000 {
clock-frequency = <40800000>;
nvidia,parent-clock-frequency = <408000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_P>;
clock-names = "emc-parent";
};
+
timing-68000000 {
clock-frequency = <68000000>;
nvidia,parent-clock-frequency = <408000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_P>;
clock-names = "emc-parent";
};
+
timing-102000000 {
clock-frequency = <102000000>;
nvidia,parent-clock-frequency = <408000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_P>;
clock-names = "emc-parent";
};
+
timing-204000000 {
clock-frequency = <204000000>;
nvidia,parent-clock-frequency = <408000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_P>;
clock-names = "emc-parent";
};
+
timing-300000000 {
clock-frequency = <300000000>;
nvidia,parent-clock-frequency = <600000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_C>;
clock-names = "emc-parent";
};
+
timing-396000000 {
clock-frequency = <396000000>;
nvidia,parent-clock-frequency = <792000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_M>;
clock-names = "emc-parent";
};
+
/* TODO: Add 528MHz frequency */
+
timing-600000000 {
clock-frequency = <600000000>;
nvidia,parent-clock-frequency = <600000000>;
clocks = <&tegra_car TEGRA124_CLK_PLL_C_UD>;
clock-names = "emc-parent";
};
+
timing-792000000 {
clock-frequency = <792000000>;
nvidia,parent-clock-frequency = <792000000>;
@@ -68,6 +78,298 @@ timing-792000000 {
};
};
+ memory-controller@70019000 {
+ emc-timings-1 {
+ nvidia,ram-code = <1>;
+
+ timing-12750000 {
+ clock-frequency = <12750000>;
+
+ nvidia,emem-configuration = <
+ 0x40040001
+ 0x8000000a
+ 0x00000001
+ 0x00000001
+ 0x00000002
+ 0x00000000
+ 0x00000002
+ 0x00000001
+ 0x00000002
+ 0x00000008
+ 0x00000003
+ 0x00000002
+ 0x00000003
+ 0x00000006
+ 0x06030203
+ 0x000a0402
+ 0x77e30303
+ 0x70000f03
+ 0x001f0000
+ >;
+ };
+
+ timing-20400000 {
+ clock-frequency = <20400000>;
+
+ nvidia,emem-configuration = <
+ 0x40020001
+ 0x80000012
+ 0x00000001
+ 0x00000001
+ 0x00000002
+ 0x00000000
+ 0x00000002
+ 0x00000001
+ 0x00000002
+ 0x00000008
+ 0x00000003
+ 0x00000002
+ 0x00000003
+ 0x00000006
+ 0x06030203
+ 0x000a0402
+ 0x76230303
+ 0x70000f03
+ 0x001f0000
+ >;
+ };
+
+ timing-40800000 {
+ clock-frequency = <40800000>;
+
+ nvidia,emem-configuration = <
+ 0xa0000001
+ 0x80000017
+ 0x00000001
+ 0x00000001
+ 0x00000002
+ 0x00000000
+ 0x00000002
+ 0x00000001
+ 0x00000002
+ 0x00000008
+ 0x00000003
+ 0x00000002
+ 0x00000003
+ 0x00000006
+ 0x06030203
+ 0x000a0402
+ 0x74a30303
+ 0x70000f03
+ 0x001f0000
+ >;
+ };
+
+ timing-68000000 {
+ clock-frequency = <68000000>;
+
+ nvidia,emem-configuration = <
+ 0x00000001
+ 0x8000001e
+ 0x00000001
+ 0x00000001
+ 0x00000002
+ 0x00000000
+ 0x00000002
+ 0x00000001
+ 0x00000002
+ 0x00000008
+ 0x00000003
+ 0x00000002
+ 0x00000003
+ 0x00000006
+ 0x06030203
+ 0x000a0402
+ 0x74230403
+ 0x70000f03
+ 0x001f0000
+ >;
+ };
+
+ timing-102000000 {
+ clock-frequency = <102000000>;
+
+ nvidia,emem-configuration = <
+ 0x08000001
+ 0x80000026
+ 0x00000001
+ 0x00000001
+ 0x00000003
+ 0x00000000
+ 0x00000002
+ 0x00000001
+ 0x00000002
+ 0x00000008
+ 0x00000003
+ 0x00000002
+ 0x00000003
+ 0x00000006
+ 0x06030203
+ 0x000a0403
+ 0x73c30504
+ 0x70000f03
+ 0x001f0000
+ >;
+ };
+
+ timing-204000000 {
+ clock-frequency = <204000000>;
+
+ nvidia,emem-configuration = <
+ 0x01000003
+ 0x80000040
+ 0x00000001
+ 0x00000001
+ 0x00000005
+ 0x00000002
+ 0x00000004
+ 0x00000001
+ 0x00000002
+ 0x00000008
+ 0x00000003
+ 0x00000002
+ 0x00000004
+ 0x00000006
+ 0x06040203
+ 0x000a0405
+ 0x73840a06
+ 0x70000f03
+ 0x001f0000
+ >;
+ };
+
+ timing-300000000 {
+ clock-frequency = <300000000>;
+
+ nvidia,emem-configuration = <
+ 0x08000004
+ 0x80000040
+ 0x00000001
+ 0x00000002
+ 0x00000007
+ 0x00000004
+ 0x00000005
+ 0x00000001
+ 0x00000002
+ 0x00000007
+ 0x00000002
+ 0x00000002
+ 0x00000004
+ 0x00000006
+ 0x06040202
+ 0x000b0607
+ 0x77450e08
+ 0x70000f03
+ 0x001f0000
+ >;
+ };
+
+ timing-396000000 {
+ clock-frequency = <396000000>;
+
+ nvidia,emem-configuration = <
+ 0x0f000005
+ 0x80000040
+ 0x00000001
+ 0x00000002
+ 0x00000009
+ 0x00000005
+ 0x00000007
+ 0x00000001
+ 0x00000002
+ 0x00000008
+ 0x00000002
+ 0x00000002
+ 0x00000004
+ 0x00000006
+ 0x06040202
+ 0x000d0709
+ 0x7586120a
+ 0x70000f03
+ 0x001f0000
+ >;
+ };
+
+ timing-528000000 {
+ clock-frequency = <528000000>;
+
+ nvidia,emem-configuration = <
+ 0x0f000007
+ 0x80000040
+ 0x00000002
+ 0x00000003
+ 0x0000000d
+ 0x00000008
+ 0x0000000a
+ 0x00000001
+ 0x00000002
+ 0x00000009
+ 0x00000002
+ 0x00000002
+ 0x00000005
+ 0x00000006
+ 0x06050202
+ 0x0010090d
+ 0x7428180e
+ 0x70000f03
+ 0x001f0000
+ >;
+ };
+
+ timing-600000000 {
+ clock-frequency = <600000000>;
+
+ nvidia,emem-configuration = <
+ 0x00000009
+ 0x80000040
+ 0x00000003
+ 0x00000004
+ 0x0000000e
+ 0x00000009
+ 0x0000000b
+ 0x00000001
+ 0x00000003
+ 0x0000000b
+ 0x00000002
+ 0x00000002
+ 0x00000005
+ 0x00000007
+ 0x07050202
+ 0x00130b0e
+ 0x73a91b0f
+ 0x70000f03
+ 0x001f0000
+ >;
+ };
+
+ timing-792000000 {
+ clock-frequency = <792000000>;
+
+ nvidia,emem-configuration = <
+ 0x0e00000b
+ 0x80000040
+ 0x00000004
+ 0x00000005
+ 0x00000013
+ 0x0000000c
+ 0x0000000f
+ 0x00000002
+ 0x00000003
+ 0x0000000c
+ 0x00000002
+ 0x00000002
+ 0x00000006
+ 0x00000008
+ 0x08060202
+ 0x00160d13
+ 0x734c2414
+ 0x70000f02
+ 0x001f0000
+ >;
+ };
+ };
+ };
+
external-memory-controller@7001b000 {
emc-timings-1 {
nvidia,ram-code = <1>;
@@ -1751,310 +2053,16 @@ timing-792000000 {
0x0000000f
>;
};
-
- };
- };
-
- memory-controller@70019000 {
- emc-timings-1 {
- nvidia,ram-code = <1>;
-
-
- timing-12750000 {
- clock-frequency = <12750000>;
-
- nvidia,emem-configuration = <
- 0x40040001
- 0x8000000a
- 0x00000001
- 0x00000001
- 0x00000002
- 0x00000000
- 0x00000002
- 0x00000001
- 0x00000002
- 0x00000008
- 0x00000003
- 0x00000002
- 0x00000003
- 0x00000006
- 0x06030203
- 0x000a0402
- 0x77e30303
- 0x70000f03
- 0x001f0000
- >;
- };
-
- timing-20400000 {
- clock-frequency = <20400000>;
-
- nvidia,emem-configuration = <
- 0x40020001
- 0x80000012
- 0x00000001
- 0x00000001
- 0x00000002
- 0x00000000
- 0x00000002
- 0x00000001
- 0x00000002
- 0x00000008
- 0x00000003
- 0x00000002
- 0x00000003
- 0x00000006
- 0x06030203
- 0x000a0402
- 0x76230303
- 0x70000f03
- 0x001f0000
- >;
- };
-
- timing-40800000 {
- clock-frequency = <40800000>;
-
- nvidia,emem-configuration = <
- 0xa0000001
- 0x80000017
- 0x00000001
- 0x00000001
- 0x00000002
- 0x00000000
- 0x00000002
- 0x00000001
- 0x00000002
- 0x00000008
- 0x00000003
- 0x00000002
- 0x00000003
- 0x00000006
- 0x06030203
- 0x000a0402
- 0x74a30303
- 0x70000f03
- 0x001f0000
- >;
- };
-
- timing-68000000 {
- clock-frequency = <68000000>;
-
- nvidia,emem-configuration = <
- 0x00000001
- 0x8000001e
- 0x00000001
- 0x00000001
- 0x00000002
- 0x00000000
- 0x00000002
- 0x00000001
- 0x00000002
- 0x00000008
- 0x00000003
- 0x00000002
- 0x00000003
- 0x00000006
- 0x06030203
- 0x000a0402
- 0x74230403
- 0x70000f03
- 0x001f0000
- >;
- };
-
- timing-102000000 {
- clock-frequency = <102000000>;
-
- nvidia,emem-configuration = <
- 0x08000001
- 0x80000026
- 0x00000001
- 0x00000001
- 0x00000003
- 0x00000000
- 0x00000002
- 0x00000001
- 0x00000002
- 0x00000008
- 0x00000003
- 0x00000002
- 0x00000003
- 0x00000006
- 0x06030203
- 0x000a0403
- 0x73c30504
- 0x70000f03
- 0x001f0000
- >;
- };
-
- timing-204000000 {
- clock-frequency = <204000000>;
-
- nvidia,emem-configuration = <
- 0x01000003
- 0x80000040
- 0x00000001
- 0x00000001
- 0x00000005
- 0x00000002
- 0x00000004
- 0x00000001
- 0x00000002
- 0x00000008
- 0x00000003
- 0x00000002
- 0x00000004
- 0x00000006
- 0x06040203
- 0x000a0405
- 0x73840a06
- 0x70000f03
- 0x001f0000
- >;
- };
-
- timing-300000000 {
- clock-frequency = <300000000>;
-
- nvidia,emem-configuration = <
- 0x08000004
- 0x80000040
- 0x00000001
- 0x00000002
- 0x00000007
- 0x00000004
- 0x00000005
- 0x00000001
- 0x00000002
- 0x00000007
- 0x00000002
- 0x00000002
- 0x00000004
- 0x00000006
- 0x06040202
- 0x000b0607
- 0x77450e08
- 0x70000f03
- 0x001f0000
- >;
- };
-
- timing-396000000 {
- clock-frequency = <396000000>;
-
- nvidia,emem-configuration = <
- 0x0f000005
- 0x80000040
- 0x00000001
- 0x00000002
- 0x00000009
- 0x00000005
- 0x00000007
- 0x00000001
- 0x00000002
- 0x00000008
- 0x00000002
- 0x00000002
- 0x00000004
- 0x00000006
- 0x06040202
- 0x000d0709
- 0x7586120a
- 0x70000f03
- 0x001f0000
- >;
- };
-
- timing-528000000 {
- clock-frequency = <528000000>;
-
- nvidia,emem-configuration = <
- 0x0f000007
- 0x80000040
- 0x00000002
- 0x00000003
- 0x0000000d
- 0x00000008
- 0x0000000a
- 0x00000001
- 0x00000002
- 0x00000009
- 0x00000002
- 0x00000002
- 0x00000005
- 0x00000006
- 0x06050202
- 0x0010090d
- 0x7428180e
- 0x70000f03
- 0x001f0000
- >;
- };
-
- timing-600000000 {
- clock-frequency = <600000000>;
-
- nvidia,emem-configuration = <
- 0x00000009
- 0x80000040
- 0x00000003
- 0x00000004
- 0x0000000e
- 0x00000009
- 0x0000000b
- 0x00000001
- 0x00000003
- 0x0000000b
- 0x00000002
- 0x00000002
- 0x00000005
- 0x00000007
- 0x07050202
- 0x00130b0e
- 0x73a91b0f
- 0x70000f03
- 0x001f0000
- >;
- };
-
- timing-792000000 {
- clock-frequency = <792000000>;
-
- nvidia,emem-configuration = <
- 0x0e00000b
- 0x80000040
- 0x00000004
- 0x00000005
- 0x00000013
- 0x0000000c
- 0x0000000f
- 0x00000002
- 0x00000003
- 0x0000000c
- 0x00000002
- 0x00000002
- 0x00000006
- 0x00000008
- 0x08060202
- 0x00160d13
- 0x734c2414
- 0x70000f02
- 0x001f0000
- >;
- };
};
};
};
&emc_icc_dvfs_opp_table {
- /delete-node/ opp@924000000,1100;
- /delete-node/ opp@1200000000,1100;
+ /delete-node/ opp-924000000-1100;
+ /delete-node/ opp-1200000000-1100;
};
&emc_bw_dfs_opp_table {
- /delete-node/ opp@924000000;
- /delete-node/ opp@1200000000;
+ /delete-node/ opp-924000000;
+ /delete-node/ opp-1200000000;
};
diff --git a/arch/arm/boot/dts/tegra124-nyan.dtsi b/arch/arm/boot/dts/tegra124-nyan.dtsi
index 63a8127..a93cfb4 100644
--- a/arch/arm/boot/dts/tegra124-nyan.dtsi
+++ b/arch/arm/boot/dts/tegra124-nyan.dtsi
@@ -1,5 +1,6 @@
// SPDX-License-Identifier: GPL-2.0
#include <dt-bindings/input/input.h>
+#include <dt-bindings/thermal/thermal.h>
#include "tegra124.dtsi"
/ {
@@ -61,7 +62,7 @@ dpaux@545c0000 {
};
};
- gpu@0,57000000 {
+ gpu@57000000 {
status = "okay";
vdd-supply = <&vdd_gpu>;
@@ -87,7 +88,7 @@ acodec: audio-codec@10 {
interrupts = <TEGRA_GPIO(H, 4) IRQ_TYPE_EDGE_FALLING>;
};
- temperature-sensor@4c {
+ tmp451: temperature-sensor@4c {
compatible = "ti,tmp451";
reg = <0x4c>;
interrupt-parent = <&gpio>;
@@ -390,6 +391,10 @@ pmc@7000e400 {
nvidia,sys-clock-req-active-high;
};
+ cec@70015000 {
+ status = "okay";
+ };
+
hda@70030000 {
status = "okay";
};
@@ -466,6 +471,7 @@ usb2-0 {
vbus-supply = <&vdd_usb1_vbus>;
status = "okay";
mode = "otg";
+ usb-role-switch;
};
usb2-1 {
@@ -527,7 +533,7 @@ mmc@700b0600 { /* eMMC on this bus */
/* CPU DFLL clock */
clock@70110000 {
- status = "disabled";
+ status = "okay";
vdd-cpu-supply = <&vdd_cpu>;
nvidia,i2c-fs-rate = <400000>;
};
@@ -582,7 +588,7 @@ backlight: backlight {
256>;
};
- clk32k_in: clock@0 {
+ clk32k_in: clock-32k {
compatible = "fixed-clock";
clock-frequency = <32768>;
#clock-cells = <0>;
@@ -615,7 +621,7 @@ power {
};
};
- vdd_mux: regulator@0 {
+ vdd_mux: regulator-mux {
compatible = "regulator-fixed";
regulator-name = "+VDD_MUX";
regulator-min-microvolt = <12000000>;
@@ -624,7 +630,7 @@ vdd_mux: regulator@0 {
regulator-boot-on;
};
- vdd_5v0_sys: regulator@1 {
+ vdd_5v0_sys: regulator-5v0sys {
compatible = "regulator-fixed";
regulator-name = "+5V_SYS";
regulator-min-microvolt = <5000000>;
@@ -634,7 +640,7 @@ vdd_5v0_sys: regulator@1 {
vin-supply = <&vdd_mux>;
};
- vdd_3v3_sys: regulator@2 {
+ vdd_3v3_sys: regulator-3v3sys {
compatible = "regulator-fixed";
regulator-name = "+3.3V_SYS";
regulator-min-microvolt = <3300000>;
@@ -644,7 +650,7 @@ vdd_3v3_sys: regulator@2 {
vin-supply = <&vdd_mux>;
};
- vdd_3v3_run: regulator@3 {
+ vdd_3v3_run: regulator-3v3run {
compatible = "regulator-fixed";
regulator-name = "+3.3V_RUN";
regulator-min-microvolt = <3300000>;
@@ -656,7 +662,7 @@ vdd_3v3_run: regulator@3 {
vin-supply = <&vdd_3v3_sys>;
};
- vdd_3v3_hdmi: regulator@4 {
+ vdd_3v3_hdmi: regulator-3v3hdmi {
compatible = "regulator-fixed";
regulator-name = "+3.3V_AVDD_HDMI_AP_GATED";
regulator-min-microvolt = <3300000>;
@@ -664,7 +670,7 @@ vdd_3v3_hdmi: regulator@4 {
vin-supply = <&vdd_3v3_run>;
};
- vdd_led: regulator@5 {
+ vdd_led: regulator-led {
compatible = "regulator-fixed";
regulator-name = "+VDD_LED";
gpio = <&gpio TEGRA_GPIO(P, 2) GPIO_ACTIVE_HIGH>;
@@ -672,7 +678,7 @@ vdd_led: regulator@5 {
vin-supply = <&vdd_mux>;
};
- vdd_5v0_ts: regulator@6 {
+ vdd_5v0_ts: regulator-ts {
compatible = "regulator-fixed";
regulator-name = "+5V_VDD_TS_SW";
regulator-min-microvolt = <5000000>;
@@ -683,7 +689,7 @@ vdd_5v0_ts: regulator@6 {
vin-supply = <&vdd_5v0_sys>;
};
- vdd_usb1_vbus: regulator@7 {
+ vdd_usb1_vbus: regulator-usb1 {
compatible = "regulator-fixed";
regulator-name = "+5V_USB_HS";
regulator-min-microvolt = <5000000>;
@@ -694,7 +700,7 @@ vdd_usb1_vbus: regulator@7 {
vin-supply = <&vdd_5v0_sys>;
};
- vdd_usb3_vbus: regulator@8 {
+ vdd_usb3_vbus: regulator-usb3 {
compatible = "regulator-fixed";
regulator-name = "+5V_USB_SS";
regulator-min-microvolt = <5000000>;
@@ -705,7 +711,7 @@ vdd_usb3_vbus: regulator@8 {
vin-supply = <&vdd_5v0_sys>;
};
- vdd_3v3_panel: regulator@9 {
+ vdd_3v3_panel: regulator-panel {
compatible = "regulator-fixed";
regulator-name = "+3.3V_PANEL";
regulator-min-microvolt = <3300000>;
@@ -715,7 +721,7 @@ vdd_3v3_panel: regulator@9 {
vin-supply = <&vdd_3v3_run>;
};
- vdd_3v3_lp0: regulator@10 {
+ vdd_3v3_lp0: regulator-lp0 {
compatible = "regulator-fixed";
regulator-name = "+3.3V_LP0";
regulator-min-microvolt = <3300000>;
@@ -730,7 +736,7 @@ vdd_3v3_lp0: regulator@10 {
vin-supply = <&vdd_3v3_sys>;
};
- vdd_hdmi_pll: regulator@11 {
+ vdd_hdmi_pll: regulator-hdmipll {
compatible = "regulator-fixed";
regulator-name = "+1.05V_RUN_AVDD_HDMI_PLL";
regulator-min-microvolt = <1050000>;
@@ -739,7 +745,7 @@ vdd_hdmi_pll: regulator@11 {
vin-supply = <&vdd_1v05_run>;
};
- vdd_5v0_hdmi: regulator@12 {
+ vdd_5v0_hdmi: regulator-hdmicon {
compatible = "regulator-fixed";
regulator-name = "+5V_HDMI_CON";
regulator-min-microvolt = <5000000>;
@@ -784,6 +790,52 @@ gpio-restart {
gpios = <&gpio TEGRA_GPIO(I, 5) GPIO_ACTIVE_LOW>;
priority = <200>;
};
+
+ cpus {
+ cpu0: cpu@0 {
+ #cooling-cells = <2>;
+ };
+
+ cpu1: cpu@1 {
+ #cooling-cells = <2>;
+ };
+
+ cpu2: cpu@2 {
+ #cooling-cells = <2>;
+ };
+
+ cpu3: cpu@3 {
+ #cooling-cells = <2>;
+ };
+ };
+
+ thermal-zones {
+ cpu-skin-thermal {
+ polling-delay-passive = <1000>; /* milliseconds */
+ polling-delay = <5000>; /* milliseconds */
+
+ thermal-sensors = <&tmp451 0>;
+
+ trips {
+ cpu_passive_trip: cpu-alert0 {
+ /* throttle at 70C until temperature drops to 69.8C */
+ temperature = <70000>;
+ hysteresis = <200>;
+ type = "passive";
+ };
+ };
+
+ cooling-maps {
+ map0 {
+ trip = <&cpu_passive_trip>;
+ cooling-device = <&cpu0 THERMAL_NO_LIMIT THERMAL_NO_LIMIT>,
+ <&cpu1 THERMAL_NO_LIMIT THERMAL_NO_LIMIT>,
+ <&cpu2 THERMAL_NO_LIMIT THERMAL_NO_LIMIT>,
+ <&cpu3 THERMAL_NO_LIMIT THERMAL_NO_LIMIT>;
+ };
+ };
+ };
+ };
};
#include "cros-ec-keyboard.dtsi"
diff --git a/arch/arm/boot/dts/tegra124-peripherals-opp.dtsi b/arch/arm/boot/dts/tegra124-peripherals-opp.dtsi
index 781ac860..b262c12 100644
--- a/arch/arm/boot/dts/tegra124-peripherals-opp.dtsi
+++ b/arch/arm/boot/dts/tegra124-peripherals-opp.dtsi
@@ -1,421 +1,421 @@
// SPDX-License-Identifier: GPL-2.0
/ {
- emc_icc_dvfs_opp_table: emc-dvfs-opp-table {
+ emc_icc_dvfs_opp_table: opp-table-emc {
compatible = "operating-points-v2";
- opp@12750000,800 {
+ opp-12750000-800 {
opp-microvolt = <800000 800000 1150000>;
opp-hz = /bits/ 64 <12750000>;
opp-supported-hw = <0x0003>;
};
- opp@12750000,950 {
+ opp-12750000-950 {
opp-microvolt = <950000 950000 1150000>;
opp-hz = /bits/ 64 <12750000>;
opp-supported-hw = <0x0008>;
};
- opp@12750000,1050 {
+ opp-12750000-1050 {
opp-microvolt = <1050000 1050000 1150000>;
opp-hz = /bits/ 64 <12750000>;
opp-supported-hw = <0x0010>;
};
- opp@12750000,1110 {
+ opp-12750000-1110 {
opp-microvolt = <1110000 1110000 1150000>;
opp-hz = /bits/ 64 <12750000>;
opp-supported-hw = <0x0004>;
};
- opp@20400000,800 {
+ opp-20400000-800 {
opp-microvolt = <800000 800000 1150000>;
opp-hz = /bits/ 64 <20400000>;
opp-supported-hw = <0x0003>;
};
- opp@20400000,950 {
+ opp-20400000-950 {
opp-microvolt = <950000 950000 1150000>;
opp-hz = /bits/ 64 <20400000>;
opp-supported-hw = <0x0008>;
};
- opp@20400000,1050 {
+ opp-20400000-1050 {
opp-microvolt = <1050000 1050000 1150000>;
opp-hz = /bits/ 64 <20400000>;
opp-supported-hw = <0x0010>;
};
- opp@20400000,1110 {
+ opp-20400000-1110 {
opp-microvolt = <1110000 1110000 1150000>;
opp-hz = /bits/ 64 <20400000>;
opp-supported-hw = <0x0004>;
};
- opp@40800000,800 {
+ opp-40800000-800 {
opp-microvolt = <800000 800000 1150000>;
opp-hz = /bits/ 64 <40800000>;
opp-supported-hw = <0x0003>;
};
- opp@40800000,950 {
+ opp-40800000-950 {
opp-microvolt = <950000 950000 1150000>;
opp-hz = /bits/ 64 <40800000>;
opp-supported-hw = <0x0008>;
};
- opp@40800000,1050 {
+ opp-40800000-1050 {
opp-microvolt = <1050000 1050000 1150000>;
opp-hz = /bits/ 64 <40800000>;
opp-supported-hw = <0x0010>;
};
- opp@40800000,1110 {
+ opp-40800000-1110 {
opp-microvolt = <1110000 1110000 1150000>;
opp-hz = /bits/ 64 <40800000>;
opp-supported-hw = <0x0004>;
};
- opp@68000000,800 {
+ opp-68000000-800 {
opp-microvolt = <800000 800000 1150000>;
opp-hz = /bits/ 64 <68000000>;
opp-supported-hw = <0x0003>;
};
- opp@68000000,950 {
+ opp-68000000-950 {
opp-microvolt = <950000 950000 1150000>;
opp-hz = /bits/ 64 <68000000>;
opp-supported-hw = <0x0008>;
};
- opp@68000000,1050 {
+ opp-68000000-1050 {
opp-microvolt = <1050000 1050000 1150000>;
opp-hz = /bits/ 64 <68000000>;
opp-supported-hw = <0x0010>;
};
- opp@68000000,1110 {
+ opp-68000000-1110 {
opp-microvolt = <1110000 1110000 1150000>;
opp-hz = /bits/ 64 <68000000>;
opp-supported-hw = <0x0004>;
};
- opp@102000000,800 {
+ opp-102000000-800 {
opp-microvolt = <800000 800000 1150000>;
opp-hz = /bits/ 64 <102000000>;
opp-supported-hw = <0x0003>;
};
- opp@102000000,950 {
+ opp-102000000-950 {
opp-microvolt = <950000 950000 1150000>;
opp-hz = /bits/ 64 <102000000>;
opp-supported-hw = <0x0008>;
};
- opp@102000000,1050 {
+ opp-102000000-1050 {
opp-microvolt = <1050000 1050000 1150000>;
opp-hz = /bits/ 64 <102000000>;
opp-supported-hw = <0x0010>;
};
- opp@102000000,1110 {
+ opp-102000000-1110 {
opp-microvolt = <1110000 1110000 1150000>;
opp-hz = /bits/ 64 <102000000>;
opp-supported-hw = <0x0004>;
};
- opp@204000000,800 {
+ opp-204000000-800 {
opp-microvolt = <800000 800000 1150000>;
opp-hz = /bits/ 64 <204000000>;
opp-supported-hw = <0x0003>;
opp-suspend;
};
- opp@204000000,950 {
+ opp-204000000-950 {
opp-microvolt = <950000 950000 1150000>;
opp-hz = /bits/ 64 <204000000>;
opp-supported-hw = <0x0008>;
opp-suspend;
};
- opp@204000000,1050 {
+ opp-204000000-1050 {
opp-microvolt = <1050000 1050000 1150000>;
opp-hz = /bits/ 64 <204000000>;
opp-supported-hw = <0x0010>;
opp-suspend;
};
- opp@204000000,1110 {
+ opp-204000000-1110 {
opp-microvolt = <1110000 1110000 1150000>;
opp-hz = /bits/ 64 <204000000>;
opp-supported-hw = <0x0004>;
opp-suspend;
};
- opp@264000000,800 {
+ opp-264000000-800 {
opp-microvolt = <800000 800000 1150000>;
opp-hz = /bits/ 64 <264000000>;
opp-supported-hw = <0x0003>;
};
- opp@264000000,950 {
+ opp-264000000-950 {
opp-microvolt = <950000 950000 1150000>;
opp-hz = /bits/ 64 <264000000>;
opp-supported-hw = <0x0008>;
};
- opp@264000000,1050 {
+ opp-264000000-1050 {
opp-microvolt = <1050000 1050000 1150000>;
opp-hz = /bits/ 64 <264000000>;
opp-supported-hw = <0x0010>;
};
- opp@264000000,1110 {
+ opp-264000000-1110 {
opp-microvolt = <1110000 1110000 1150000>;
opp-hz = /bits/ 64 <264000000>;
opp-supported-hw = <0x0004>;
};
- opp@300000000,850 {
+ opp-300000000-850 {
opp-microvolt = <850000 850000 1150000>;
opp-hz = /bits/ 64 <300000000>;
opp-supported-hw = <0x0003>;
};
- opp@300000000,950 {
+ opp-300000000-950 {
opp-microvolt = <950000 950000 1150000>;
opp-hz = /bits/ 64 <300000000>;
opp-supported-hw = <0x0008>;
};
- opp@300000000,1050 {
+ opp-300000000-1050 {
opp-microvolt = <1050000 1050000 1150000>;
opp-hz = /bits/ 64 <300000000>;
opp-supported-hw = <0x0010>;
};
- opp@300000000,1110 {
+ opp-300000000-1110 {
opp-microvolt = <1110000 1110000 1150000>;
opp-hz = /bits/ 64 <300000000>;
opp-supported-hw = <0x0004>;
};
- opp@348000000,850 {
+ opp-348000000-850 {
opp-microvolt = <850000 850000 1150000>;
opp-hz = /bits/ 64 <348000000>;
opp-supported-hw = <0x0003>;
};
- opp@348000000,950 {
+ opp-348000000-950 {
opp-microvolt = <950000 950000 1150000>;
opp-hz = /bits/ 64 <348000000>;
opp-supported-hw = <0x0008>;
};
- opp@348000000,1050 {
+ opp-348000000-1050 {
opp-microvolt = <1050000 1050000 1150000>;
opp-hz = /bits/ 64 <348000000>;
opp-supported-hw = <0x0010>;
};
- opp@348000000,1110 {
+ opp-348000000-1110 {
opp-microvolt = <1110000 1110000 1150000>;
opp-hz = /bits/ 64 <348000000>;
opp-supported-hw = <0x0004>;
};
- opp@396000000,950 {
+ opp-396000000-950 {
opp-microvolt = <950000 950000 1150000>;
opp-hz = /bits/ 64 <396000000>;
opp-supported-hw = <0x0008>;
};
- opp@396000000,1000 {
+ opp-396000000-1000 {
opp-microvolt = <1000000 1000000 1150000>;
opp-hz = /bits/ 64 <396000000>;
opp-supported-hw = <0x0003>;
};
- opp@396000000,1050 {
+ opp-396000000-1050 {
opp-microvolt = <1050000 1050000 1150000>;
opp-hz = /bits/ 64 <396000000>;
opp-supported-hw = <0x0010>;
};
- opp@396000000,1110 {
+ opp-396000000-1110 {
opp-microvolt = <1110000 1110000 1150000>;
opp-hz = /bits/ 64 <396000000>;
opp-supported-hw = <0x0004>;
};
- opp@528000000,950 {
+ opp-528000000-950 {
opp-microvolt = <950000 950000 1150000>;
opp-hz = /bits/ 64 <528000000>;
opp-supported-hw = <0x0008>;
};
- opp@528000000,1000 {
+ opp-528000000-1000 {
opp-microvolt = <1000000 1000000 1150000>;
opp-hz = /bits/ 64 <528000000>;
opp-supported-hw = <0x0003>;
};
- opp@528000000,1050 {
+ opp-528000000-1050 {
opp-microvolt = <1050000 1050000 1150000>;
opp-hz = /bits/ 64 <528000000>;
opp-supported-hw = <0x0010>;
};
- opp@528000000,1110 {
+ opp-528000000-1110 {
opp-microvolt = <1110000 1110000 1150000>;
opp-hz = /bits/ 64 <528000000>;
opp-supported-hw = <0x0004>;
};
- opp@600000000,950 {
+ opp-600000000-950 {
opp-microvolt = <950000 950000 1150000>;
opp-hz = /bits/ 64 <600000000>;
opp-supported-hw = <0x0008>;
};
- opp@600000000,1000 {
+ opp-600000000-1000 {
opp-microvolt = <1000000 1000000 1150000>;
opp-hz = /bits/ 64 <600000000>;
opp-supported-hw = <0x0003>;
};
- opp@600000000,1050 {
+ opp-600000000-1050 {
opp-microvolt = <1050000 1050000 1150000>;
opp-hz = /bits/ 64 <600000000>;
opp-supported-hw = <0x0010>;
};
- opp@600000000,1110 {
+ opp-600000000-1110 {
opp-microvolt = <1110000 1110000 1150000>;
opp-hz = /bits/ 64 <600000000>;
opp-supported-hw = <0x0004>;
};
- opp@792000000,1000 {
+ opp-792000000-1000 {
opp-microvolt = <1000000 1000000 1150000>;
opp-hz = /bits/ 64 <792000000>;
opp-supported-hw = <0x000B>;
};
- opp@792000000,1050 {
+ opp-792000000-1050 {
opp-microvolt = <1050000 1050000 1150000>;
opp-hz = /bits/ 64 <792000000>;
opp-supported-hw = <0x0010>;
};
- opp@792000000,1110 {
+ opp-792000000-1110 {
opp-microvolt = <1110000 1110000 1150000>;
opp-hz = /bits/ 64 <792000000>;
opp-supported-hw = <0x0004>;
};
- opp@924000000,1100 {
+ opp-924000000-1100 {
opp-microvolt = <1100000 1100000 1150000>;
opp-hz = /bits/ 64 <924000000>;
opp-supported-hw = <0x0013>;
};
- opp@1200000000,1100 {
+ opp-1200000000-1100 {
opp-microvolt = <1100000 1100000 1150000>;
opp-hz = /bits/ 64 <1200000000>;
opp-supported-hw = <0x0003>;
};
};
- emc_bw_dfs_opp_table: emc-bandwidth-opp-table {
+ emc_bw_dfs_opp_table: opp-table-actmon {
compatible = "operating-points-v2";
- opp@12750000 {
+ opp-12750000 {
opp-hz = /bits/ 64 <12750000>;
opp-supported-hw = <0x001F>;
opp-peak-kBps = <204000>;
};
- opp@20400000 {
+ opp-20400000 {
opp-hz = /bits/ 64 <20400000>;
opp-supported-hw = <0x001F>;
opp-peak-kBps = <326400>;
};
- opp@40800000 {
+ opp-40800000 {
opp-hz = /bits/ 64 <40800000>;
opp-supported-hw = <0x001F>;
opp-peak-kBps = <652800>;
};
- opp@68000000 {
+ opp-68000000 {
opp-hz = /bits/ 64 <68000000>;
opp-supported-hw = <0x001F>;
opp-peak-kBps = <1088000>;
};
- opp@102000000 {
+ opp-102000000 {
opp-hz = /bits/ 64 <102000000>;
opp-supported-hw = <0x001F>;
opp-peak-kBps = <1632000>;
};
- opp@204000000 {
+ opp-204000000 {
opp-hz = /bits/ 64 <204000000>;
opp-supported-hw = <0x001F>;
opp-peak-kBps = <3264000>;
opp-suspend;
};
- opp@264000000 {
+ opp-264000000 {
opp-hz = /bits/ 64 <264000000>;
opp-supported-hw = <0x001F>;
opp-peak-kBps = <4224000>;
};
- opp@300000000 {
+ opp-300000000 {
opp-hz = /bits/ 64 <300000000>;
opp-supported-hw = <0x001F>;
opp-peak-kBps = <4800000>;
};
- opp@348000000 {
+ opp-348000000 {
opp-hz = /bits/ 64 <348000000>;
opp-supported-hw = <0x001F>;
opp-peak-kBps = <5568000>;
};
- opp@396000000 {
+ opp-396000000 {
opp-hz = /bits/ 64 <396000000>;
opp-supported-hw = <0x001F>;
opp-peak-kBps = <6336000>;
};
- opp@528000000 {
+ opp-528000000 {
opp-hz = /bits/ 64 <528000000>;
opp-supported-hw = <0x001F>;
opp-peak-kBps = <8448000>;
};
- opp@600000000 {
+ opp-600000000 {
opp-hz = /bits/ 64 <600000000>;
opp-supported-hw = <0x001F>;
opp-peak-kBps = <9600000>;
};
- opp@792000000 {
+ opp-792000000 {
opp-hz = /bits/ 64 <792000000>;
opp-supported-hw = <0x001F>;
opp-peak-kBps = <12672000>;
};
- opp@924000000 {
+ opp-924000000 {
opp-hz = /bits/ 64 <924000000>;
opp-supported-hw = <0x0013>;
opp-peak-kBps = <14784000>;
};
- opp@1200000000 {
+ opp-1200000000 {
opp-hz = /bits/ 64 <1200000000>;
opp-supported-hw = <0x0003>;
opp-peak-kBps = <19200000>;
diff --git a/arch/arm/boot/dts/tegra124-venice2.dts b/arch/arm/boot/dts/tegra124-venice2.dts
index e6b54ac..232c906 100644
--- a/arch/arm/boot/dts/tegra124-venice2.dts
+++ b/arch/arm/boot/dts/tegra124-venice2.dts
@@ -51,7 +51,7 @@ dpaux@545c0000 {
};
};
- gpu@0,57000000 {
+ gpu@57000000 {
/*
* Node left disabled on purpose - the bootloader will enable
* it after having set the VPR up
@@ -881,7 +881,8 @@ battery: sbs-battery@b {
spi@7000da00 {
status = "okay";
spi-max-frequency = <25000000>;
- spi-flash@0 {
+
+ flash@0 {
compatible = "winbond,w25q32dw", "jedec,spi-nor";
reg = <0>;
spi-max-frequency = <20000000>;
@@ -972,7 +973,7 @@ ports {
usb2-0 {
status = "okay";
mode = "otg";
-
+ usb-role-switch;
vbus-supply = <&vdd_usb1_vbus>;
};
@@ -1061,7 +1062,7 @@ backlight: backlight {
default-brightness-level = <6>;
};
- clk32k_in: clock@0 {
+ clk32k_in: clock-32k {
compatible = "fixed-clock";
clock-frequency = <32768>;
#clock-cells = <0>;
@@ -1086,7 +1087,7 @@ panel: panel {
ddc-i2c-bus = <&dpaux>;
};
- vdd_mux: regulator@0 {
+ vdd_mux: regulator-mux {
compatible = "regulator-fixed";
regulator-name = "+VDD_MUX";
regulator-min-microvolt = <12000000>;
@@ -1095,7 +1096,7 @@ vdd_mux: regulator@0 {
regulator-boot-on;
};
- vdd_5v0_sys: regulator@1 {
+ vdd_5v0_sys: regulator-5v0sys {
compatible = "regulator-fixed";
regulator-name = "+5V_SYS";
regulator-min-microvolt = <5000000>;
@@ -1105,7 +1106,7 @@ vdd_5v0_sys: regulator@1 {
vin-supply = <&vdd_mux>;
};
- vdd_3v3_sys: regulator@2 {
+ vdd_3v3_sys: regulator-3v3sys {
compatible = "regulator-fixed";
regulator-name = "+3.3V_SYS";
regulator-min-microvolt = <3300000>;
@@ -1115,7 +1116,7 @@ vdd_3v3_sys: regulator@2 {
vin-supply = <&vdd_mux>;
};
- vdd_3v3_run: regulator@3 {
+ vdd_3v3_run: regulator-3v3run {
compatible = "regulator-fixed";
regulator-name = "+3.3V_RUN";
regulator-min-microvolt = <3300000>;
@@ -1127,7 +1128,7 @@ vdd_3v3_run: regulator@3 {
vin-supply = <&vdd_3v3_sys>;
};
- vdd_3v3_hdmi: regulator@4 {
+ vdd_3v3_hdmi: regulator-hdmi {
compatible = "regulator-fixed";
regulator-name = "+3.3V_AVDD_HDMI_AP_GATED";
regulator-min-microvolt = <3300000>;
@@ -1135,7 +1136,7 @@ vdd_3v3_hdmi: regulator@4 {
vin-supply = <&vdd_3v3_run>;
};
- vdd_led: regulator@5 {
+ vdd_led: regulator-led {
compatible = "regulator-fixed";
regulator-name = "+VDD_LED";
regulator-min-microvolt = <3300000>;
@@ -1145,7 +1146,7 @@ vdd_led: regulator@5 {
vin-supply = <&vdd_mux>;
};
- vdd_5v0_ts: regulator@6 {
+ vdd_5v0_ts: regulator-ts {
compatible = "regulator-fixed";
regulator-name = "+5V_VDD_TS_SW";
regulator-min-microvolt = <5000000>;
@@ -1156,7 +1157,7 @@ vdd_5v0_ts: regulator@6 {
vin-supply = <&vdd_5v0_sys>;
};
- vdd_usb1_vbus: regulator@7 {
+ vdd_usb1_vbus: regulator-usb1 {
compatible = "regulator-fixed";
regulator-name = "+5V_USB_HS";
regulator-min-microvolt = <5000000>;
@@ -1167,7 +1168,7 @@ vdd_usb1_vbus: regulator@7 {
vin-supply = <&vdd_5v0_sys>;
};
- vdd_usb3_vbus: regulator@8 {
+ vdd_usb3_vbus: regulator-usb3 {
compatible = "regulator-fixed";
regulator-name = "+5V_USB_SS";
regulator-min-microvolt = <5000000>;
@@ -1178,7 +1179,7 @@ vdd_usb3_vbus: regulator@8 {
vin-supply = <&vdd_5v0_sys>;
};
- vdd_3v3_panel: regulator@9 {
+ vdd_3v3_panel: regulator-panel {
compatible = "regulator-fixed";
regulator-name = "+3.3V_PANEL";
regulator-min-microvolt = <3300000>;
@@ -1188,7 +1189,7 @@ vdd_3v3_panel: regulator@9 {
vin-supply = <&vdd_3v3_run>;
};
- vdd_3v3_lp0: regulator@10 {
+ vdd_3v3_lp0: regulator-lp0 {
compatible = "regulator-fixed";
regulator-name = "+3.3V_LP0";
regulator-min-microvolt = <3300000>;
@@ -1203,7 +1204,7 @@ vdd_3v3_lp0: regulator@10 {
vin-supply = <&vdd_3v3_sys>;
};
- vdd_hdmi_pll: regulator@11 {
+ vdd_hdmi_pll: regulator-hdmipll {
compatible = "regulator-fixed";
regulator-name = "+1.05V_RUN_AVDD_HDMI_PLL";
regulator-min-microvolt = <1050000>;
@@ -1212,7 +1213,7 @@ vdd_hdmi_pll: regulator@11 {
vin-supply = <&vdd_1v05_run>;
};
- vdd_5v0_hdmi: regulator@12 {
+ vdd_5v0_hdmi: regulator-hdmicon {
compatible = "regulator-fixed";
regulator-name = "+5V_HDMI_CON";
regulator-min-microvolt = <5000000>;
diff --git a/arch/arm/boot/dts/tegra124.dtsi b/arch/arm/boot/dts/tegra124.dtsi
index 63a6417..a9ab548 100644
--- a/arch/arm/boot/dts/tegra124.dtsi
+++ b/arch/arm/boot/dts/tegra124.dtsi
@@ -94,8 +94,8 @@ host1x@50000000 {
interrupt-names = "syncpt", "host1x";
clocks = <&tegra_car TEGRA124_CLK_HOST1X>;
clock-names = "host1x";
- resets = <&tegra_car 28>;
- reset-names = "host1x";
+ resets = <&tegra_car 28>, <&mc TEGRA124_MC_RESET_HC>;
+ reset-names = "host1x", "mc";
iommus = <&mc TEGRA_SWGROUP_HC>;
#address-cells = <2>;
@@ -223,12 +223,7 @@ gic: interrupt-controller@50041000 {
interrupt-parent = <&gic>;
};
- /*
- * Please keep the following 0, notation in place as a former mainline
- * U-Boot version was looking for that particular notation in order to
- * perform required fix-ups on that GPU node.
- */
- gpu@0,57000000 {
+ gpu@57000000 {
compatible = "nvidia,gk20a";
reg = <0x0 0x57000000 0x0 0x01000000>,
<0x0 0x58000000 0x0 0x01000000>;
@@ -259,7 +254,7 @@ lic: interrupt-controller@60004000 {
};
timer@60005000 {
- compatible = "nvidia,tegra124-timer", "nvidia,tegra30-timer", "nvidia,tegra20-timer";
+ compatible = "nvidia,tegra124-timer", "nvidia,tegra30-timer";
reg = <0x0 0x60005000 0x0 0x400>;
interrupts = <GIC_SPI 0 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 1 IRQ_TYPE_LEVEL_HIGH>,
@@ -313,9 +308,7 @@ gpio: gpio@6000d000 {
gpio-controller;
#interrupt-cells = <2>;
interrupt-controller;
- /*
gpio-ranges = <&pinmux 0 0 251>;
- */
};
apbdma: dma@60020000 {
@@ -443,7 +436,7 @@ pwm: pwm@7000a000 {
};
i2c@7000c000 {
- compatible = "nvidia,tegra124-i2c", "nvidia,tegra114-i2c";
+ compatible = "nvidia,tegra124-i2c";
reg = <0x0 0x7000c000 0x0 0x100>;
interrupts = <GIC_SPI 38 IRQ_TYPE_LEVEL_HIGH>;
#address-cells = <1>;
@@ -458,7 +451,7 @@ i2c@7000c000 {
};
i2c@7000c400 {
- compatible = "nvidia,tegra124-i2c", "nvidia,tegra114-i2c";
+ compatible = "nvidia,tegra124-i2c";
reg = <0x0 0x7000c400 0x0 0x100>;
interrupts = <GIC_SPI 84 IRQ_TYPE_LEVEL_HIGH>;
#address-cells = <1>;
@@ -473,7 +466,7 @@ i2c@7000c400 {
};
i2c@7000c500 {
- compatible = "nvidia,tegra124-i2c", "nvidia,tegra114-i2c";
+ compatible = "nvidia,tegra124-i2c";
reg = <0x0 0x7000c500 0x0 0x100>;
interrupts = <GIC_SPI 92 IRQ_TYPE_LEVEL_HIGH>;
#address-cells = <1>;
@@ -488,7 +481,7 @@ i2c@7000c500 {
};
i2c@7000c700 {
- compatible = "nvidia,tegra124-i2c", "nvidia,tegra114-i2c";
+ compatible = "nvidia,tegra124-i2c";
reg = <0x0 0x7000c700 0x0 0x100>;
interrupts = <GIC_SPI 120 IRQ_TYPE_LEVEL_HIGH>;
#address-cells = <1>;
@@ -503,7 +496,7 @@ i2c@7000c700 {
};
i2c@7000d000 {
- compatible = "nvidia,tegra124-i2c", "nvidia,tegra114-i2c";
+ compatible = "nvidia,tegra124-i2c";
reg = <0x0 0x7000d000 0x0 0x100>;
interrupts = <GIC_SPI 53 IRQ_TYPE_LEVEL_HIGH>;
#address-cells = <1>;
@@ -518,7 +511,7 @@ i2c@7000d000 {
};
i2c@7000d100 {
- compatible = "nvidia,tegra124-i2c", "nvidia,tegra114-i2c";
+ compatible = "nvidia,tegra124-i2c";
reg = <0x0 0x7000d100 0x0 0x100>;
interrupts = <GIC_SPI 63 IRQ_TYPE_LEVEL_HIGH>;
#address-cells = <1>;
@@ -677,10 +670,8 @@ sata@70020000 {
<0x0 0x70020000 0x0 0x7000>; /* SATA */
interrupts = <GIC_SPI 23 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&tegra_car TEGRA124_CLK_SATA>,
- <&tegra_car TEGRA124_CLK_SATA_OOB>,
- <&tegra_car TEGRA124_CLK_CML1>,
- <&tegra_car TEGRA124_CLK_PLL_E>;
- clock-names = "sata", "sata-oob", "cml1", "pll_e";
+ <&tegra_car TEGRA124_CLK_SATA_OOB>;
+ clock-names = "sata", "sata-oob";
resets = <&tegra_car 124>,
<&tegra_car 129>,
<&tegra_car 123>;
@@ -717,8 +708,8 @@ usb@70090000 {
<&tegra_car TEGRA124_CLK_XUSB_HOST_SRC>,
<&tegra_car TEGRA124_CLK_XUSB_FALCON_SRC>,
<&tegra_car TEGRA124_CLK_XUSB_SS>,
- <&tegra_car TEGRA124_CLK_XUSB_SS_SRC>,
<&tegra_car TEGRA124_CLK_XUSB_SS_DIV2>,
+ <&tegra_car TEGRA124_CLK_XUSB_SS_SRC>,
<&tegra_car TEGRA124_CLK_XUSB_HS_SRC>,
<&tegra_car TEGRA124_CLK_XUSB_FS_SRC>,
<&tegra_car TEGRA124_CLK_PLL_U_480M>,
@@ -726,7 +717,7 @@ usb@70090000 {
<&tegra_car TEGRA124_CLK_PLL_E>;
clock-names = "xusb_host", "xusb_host_src",
"xusb_falcon_src", "xusb_ss",
- "xusb_ss_src", "xusb_ss_div2",
+ "xusb_ss_div2", "xusb_ss_src",
"xusb_hs_src", "xusb_fs_src",
"pll_u_480m", "clk_m", "pll_e";
resets = <&tegra_car 89>, <&tegra_car 156>,
@@ -1247,7 +1238,7 @@ pmu {
};
thermal-zones {
- cpu {
+ cpu-thermal {
polling-delay-passive = <1000>;
polling-delay = <1000>;
@@ -1275,7 +1266,7 @@ map0 {
};
};
- mem {
+ mem-thermal {
polling-delay-passive = <1000>;
polling-delay = <1000>;
@@ -1303,7 +1294,7 @@ cooling-maps {
};
};
- gpu {
+ gpu-thermal {
polling-delay-passive = <1000>;
polling-delay = <1000>;
@@ -1331,7 +1322,7 @@ map0 {
};
};
- pllx {
+ pllx-thermal {
polling-delay-passive = <1000>;
polling-delay = <1000>;
diff --git a/arch/arm/boot/dts/tegra20-acer-a500-picasso.dts b/arch/arm/boot/dts/tegra20-acer-a500-picasso.dts
index 23d3f8d..a170a4b 100644
--- a/arch/arm/boot/dts/tegra20-acer-a500-picasso.dts
+++ b/arch/arm/boot/dts/tegra20-acer-a500-picasso.dts
@@ -376,17 +376,27 @@ pta {
};
};
+ tegra_spdif: spdif@70002400 {
+ status = "okay";
+
+ nvidia,fixed-parent-rate;
+ };
+
tegra_i2s1: i2s@70002800 {
status = "okay";
+
+ nvidia,fixed-parent-rate;
};
uartb: serial@70006040 {
compatible = "nvidia,tegra20-hsuart";
+ /delete-property/ reg-shift;
/* GPS BCM4751 */
};
uartc: serial@70006200 {
compatible = "nvidia,tegra20-hsuart";
+ /delete-property/ reg-shift;
status = "okay";
/* Azurewave AW-NH665 BCM4329B1 */
@@ -716,6 +726,7 @@ pmc@7000e400 {
nvidia,core-pwr-good-time = <3845 3845>;
nvidia,core-pwr-off-time = <458>;
nvidia,sys-clock-req-active-high;
+ core-supply = <&vdd_core>;
};
usb@c5000000 {
@@ -827,7 +838,7 @@ bat1010: battery-2s1p {
};
/* PMIC has a built-in 32KHz oscillator which is used by PMC */
- clk32k_in: clock@0 {
+ clk32k_in: clock-32k-in {
compatible = "fixed-clock";
#clock-cells = <0>;
clock-frequency = <32768>;
@@ -839,7 +850,7 @@ clk32k_in: clock@0 {
* oscillator is used as a reference clock-source by the
* Azurewave WiFi/BT module.
*/
- rtc_32k_wifi: clock@1 {
+ rtc_32k_wifi: clock-32k-wifi {
compatible = "fixed-clock";
#clock-cells = <0>;
clock-frequency = <32768>;
@@ -964,7 +975,7 @@ lvds_encoder_output: endpoint {
};
};
- vdd_5v0_sys: regulator@0 {
+ vdd_5v0_sys: regulator-5v0 {
compatible = "regulator-fixed";
regulator-name = "vdd_5v0";
regulator-min-microvolt = <5000000>;
@@ -972,7 +983,7 @@ vdd_5v0_sys: regulator@0 {
regulator-always-on;
};
- vdd_3v3_sys: regulator@1 {
+ vdd_3v3_sys: regulator-3v3 {
compatible = "regulator-fixed";
regulator-name = "vdd_3v3_vs";
regulator-min-microvolt = <3300000>;
@@ -981,7 +992,7 @@ vdd_3v3_sys: regulator@1 {
vin-supply = <&vdd_5v0_sys>;
};
- vdd_1v8_sys: regulator@2 {
+ vdd_1v8_sys: regulator-1v8 {
compatible = "regulator-fixed";
regulator-name = "vdd_1v8_vs";
regulator-min-microvolt = <1800000>;
@@ -990,7 +1001,7 @@ vdd_1v8_sys: regulator@2 {
vin-supply = <&vdd_5v0_sys>;
};
- vdd_pnl: regulator@3 {
+ vdd_pnl: regulator-panel {
compatible = "regulator-fixed";
regulator-name = "vdd_panel";
regulator-min-microvolt = <3300000>;
@@ -1506,6 +1517,6 @@ emc-table@300000 {
};
&emc_icc_dvfs_opp_table {
- /delete-node/ opp@666000000;
- /delete-node/ opp@760000000;
+ /delete-node/ opp-666000000;
+ /delete-node/ opp-760000000;
};
diff --git a/arch/arm/boot/dts/tegra20-asus-tf101.dts b/arch/arm/boot/dts/tegra20-asus-tf101.dts
new file mode 100644
index 0000000..020172e
--- /dev/null
+++ b/arch/arm/boot/dts/tegra20-asus-tf101.dts
@@ -0,0 +1,1280 @@
+// SPDX-License-Identifier: GPL-2.0
+/dts-v1/;
+
+#include <dt-bindings/input/atmel-maxtouch.h>
+#include <dt-bindings/input/gpio-keys.h>
+#include <dt-bindings/input/input.h>
+#include <dt-bindings/thermal/thermal.h>
+
+#include "tegra20.dtsi"
+#include "tegra20-cpu-opp.dtsi"
+#include "tegra20-cpu-opp-microvolt.dtsi"
+
+/ {
+ model = "ASUS EeePad Transformer TF101";
+ compatible = "asus,tf101", "nvidia,tegra20";
+ chassis-type = "convertible";
+
+ aliases {
+ mmc0 = &sdmmc4; /* eMMC */
+ mmc1 = &sdmmc3; /* MicroSD */
+ mmc2 = &sdmmc1; /* WiFi */
+
+ rtc0 = &pmic;
+ rtc1 = "/rtc@7000e000";
+
+ serial0 = &uartd;
+ serial1 = &uartc; /* Bluetooth */
+ serial2 = &uartb; /* GPS */
+ };
+
+ /*
+ * The decompressor and also some bootloaders rely on a
+ * pre-existing /chosen node to be available to insert the
+ * command line and merge other ATAGS info.
+ */
+ chosen {};
+
+ memory@0 {
+ reg = <0x00000000 0x40000000>;
+ };
+
+ reserved-memory {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges;
+
+ ramoops@2ffe0000 {
+ compatible = "ramoops";
+ reg = <0x2ffe0000 0x10000>; /* 64kB */
+ console-size = <0x8000>; /* 32kB */
+ record-size = <0x400>; /* 1kB */
+ ecc-size = <16>;
+ };
+
+ linux,cma@30000000 {
+ compatible = "shared-dma-pool";
+ alloc-ranges = <0x30000000 0x10000000>;
+ size = <0x10000000>; /* 256MiB */
+ linux,cma-default;
+ reusable;
+ };
+ };
+
+ host1x@50000000 {
+ dc@54200000 {
+ rgb {
+ status = "okay";
+
+ port@0 {
+ lcd_output: endpoint {
+ remote-endpoint = <&lvds_encoder_input>;
+ bus-width = <18>;
+ };
+ };
+ };
+ };
+
+ hdmi@54280000 {
+ status = "okay";
+
+ vdd-supply = <&hdmi_vdd_reg>;
+ pll-supply = <&hdmi_pll_reg>;
+ hdmi-supply = <&vdd_hdmi_en>;
+
+ nvidia,ddc-i2c-bus = <&hdmi_ddc>;
+ nvidia,hpd-gpio = <&gpio TEGRA_GPIO(N, 7)
+ GPIO_ACTIVE_HIGH>;
+ };
+ };
+
+ gpio@6000d000 {
+ charging-enable-hog {
+ gpio-hog;
+ gpios = <TEGRA_GPIO(R, 6) GPIO_ACTIVE_HIGH>;
+ output-low;
+ };
+ };
+
+ pinmux@70000014 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&state_default>;
+
+ state_default: pinmux {
+ ata {
+ nvidia,pins = "ata";
+ nvidia,function = "ide";
+ };
+
+ atb {
+ nvidia,pins = "atb", "gma", "gme";
+ nvidia,function = "sdio4";
+ };
+
+ atc {
+ nvidia,pins = "atc";
+ nvidia,function = "nand";
+ };
+
+ atd {
+ nvidia,pins = "atd", "ate", "gmb", "spia",
+ "spib", "spic";
+ nvidia,function = "gmi";
+ };
+
+ cdev1 {
+ nvidia,pins = "cdev1";
+ nvidia,function = "plla_out";
+ };
+
+ cdev2 {
+ nvidia,pins = "cdev2";
+ nvidia,function = "pllp_out4";
+ };
+
+ crtp {
+ nvidia,pins = "crtp";
+ nvidia,function = "crt";
+ };
+
+ lm1 {
+ nvidia,pins = "lm1";
+ nvidia,function = "rsvd3";
+ };
+
+ csus {
+ nvidia,pins = "csus";
+ nvidia,function = "vi_sensor_clk";
+ };
+
+ dap1 {
+ nvidia,pins = "dap1";
+ nvidia,function = "dap1";
+ };
+
+ dap2 {
+ nvidia,pins = "dap2";
+ nvidia,function = "dap2";
+ };
+
+ dap3 {
+ nvidia,pins = "dap3";
+ nvidia,function = "dap3";
+ };
+
+ dap4 {
+ nvidia,pins = "dap4";
+ nvidia,function = "dap4";
+ };
+
+ dta {
+ nvidia,pins = "dta", "dtb", "dtc", "dtd", "dte";
+ nvidia,function = "vi";
+ };
+
+ dtf {
+ nvidia,pins = "dtf";
+ nvidia,function = "i2c3";
+ };
+
+ gmc {
+ nvidia,pins = "gmc";
+ nvidia,function = "uartd";
+ };
+
+ gmd {
+ nvidia,pins = "gmd";
+ nvidia,function = "sflash";
+ };
+
+ gpu {
+ nvidia,pins = "gpu";
+ nvidia,function = "pwm";
+ };
+
+ gpu7 {
+ nvidia,pins = "gpu7";
+ nvidia,function = "rtck";
+ };
+
+ gpv {
+ nvidia,pins = "gpv", "slxa";
+ nvidia,function = "pcie";
+ };
+
+ hdint {
+ nvidia,pins = "hdint";
+ nvidia,function = "hdmi";
+ };
+
+ i2cp {
+ nvidia,pins = "i2cp";
+ nvidia,function = "i2cp";
+ };
+
+ irrx {
+ nvidia,pins = "irrx", "irtx";
+ nvidia,function = "uartb";
+ };
+
+ kbca {
+ nvidia,pins = "kbca", "kbcb", "kbcc", "kbcd",
+ "kbce", "kbcf";
+ nvidia,function = "kbc";
+ };
+
+ lcsn {
+ nvidia,pins = "lcsn", "ldc", "lm0", "lpw1",
+ "lsdi", "lvp0";
+ nvidia,function = "rsvd4";
+ };
+
+ ld0 {
+ nvidia,pins = "ld0", "ld1", "ld2", "ld3", "ld4",
+ "ld5", "ld6", "ld7", "ld8", "ld9",
+ "ld10", "ld11", "ld12", "ld13", "ld14",
+ "ld15", "ld16", "ld17", "ldi", "lhp0",
+ "lhp1", "lhp2", "lhs", "lpp", "lpw0",
+ "lpw2", "lsc0", "lsc1", "lsck", "lsda",
+ "lspi", "lvp1", "lvs";
+ nvidia,function = "displaya";
+ };
+
+ owc {
+ nvidia,pins = "owc", "spdi", "spdo", "uac";
+ nvidia,function = "rsvd2";
+ };
+
+ pmc {
+ nvidia,pins = "pmc";
+ nvidia,function = "pwr_on";
+ };
+
+ rm {
+ nvidia,pins = "rm";
+ nvidia,function = "i2c1";
+ };
+
+ sdb {
+ nvidia,pins = "sdb", "sdc", "sdd", "slxc", "slxk";
+ nvidia,function = "sdio3";
+ };
+
+ sdio1 {
+ nvidia,pins = "sdio1";
+ nvidia,function = "sdio1";
+ };
+
+ slxd {
+ nvidia,pins = "slxd";
+ nvidia,function = "spdif";
+ };
+
+ spid {
+ nvidia,pins = "spid", "spie", "spif";
+ nvidia,function = "spi1";
+ };
+
+ spig {
+ nvidia,pins = "spig", "spih";
+ nvidia,function = "spi2_alt";
+ };
+
+ uaa {
+ nvidia,pins = "uaa", "uab", "uda";
+ nvidia,function = "ulpi";
+ };
+
+ uad {
+ nvidia,pins = "uad";
+ nvidia,function = "irda";
+ };
+
+ uca {
+ nvidia,pins = "uca", "ucb";
+ nvidia,function = "uartc";
+ };
+
+ conf_ata {
+ nvidia,pins = "ata", "atb", "atc", "atd",
+ "cdev1", "cdev2", "dap1", "dap4",
+ "dte", "ddc", "dtf", "gma", "gmc",
+ "gme", "gpu", "gpu7", "gpv", "i2cp",
+ "irrx", "irtx", "pta", "rm", "sdc",
+ "sdd", "slxc", "slxd", "slxk", "spdi",
+ "spdo", "uac", "uad",
+ "uda", "csus";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ };
+
+ conf_ate {
+ nvidia,pins = "ate", "dap2", "dap3", "gmb", "gmd",
+ "owc", "spia", "spib", "spic",
+ "spid", "spie", "spig", "slxa";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ };
+
+ conf_ck32 {
+ nvidia,pins = "ck32", "ddrc", "pmca", "pmcb",
+ "pmcc", "pmcd", "pmce", "xm2c", "xm2d";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ };
+
+ conf_crtp {
+ nvidia,pins = "crtp", "spih";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ };
+
+ conf_dta {
+ nvidia,pins = "dta", "dtb", "dtc", "dtd";
+ nvidia,pull = <TEGRA_PIN_PULL_DOWN>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ };
+
+ conf_spif {
+ nvidia,pins = "spif";
+ nvidia,pull = <TEGRA_PIN_PULL_DOWN>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ };
+
+ conf_hdint {
+ nvidia,pins = "hdint", "lcsn", "ldc", "lm1",
+ "lpw1", "lsck", "lsda", "lsdi", "lvp0";
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ };
+
+ conf_kbca {
+ nvidia,pins = "kbca", "kbcb", "kbcc", "kbcd",
+ "kbce", "kbcf", "sdio1", "uaa", "uab",
+ "uca", "ucb";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ };
+
+ conf_lc {
+ nvidia,pins = "lc", "ls";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ };
+
+ conf_ld0 {
+ nvidia,pins = "ld0", "ld1", "ld2", "ld3", "ld4",
+ "ld5", "ld6", "ld7", "ld8", "ld9",
+ "ld10", "ld11", "ld12", "ld13", "ld14",
+ "ld15", "ld16", "ld17", "ldi", "lhp0",
+ "lhp1", "lhp2", "lhs", "lm0", "lpp",
+ "lpw0", "lpw2", "lsc0", "lsc1", "lspi",
+ "lvp1", "lvs", "pmc", "sdb";
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ };
+
+ conf_ld17_0 {
+ nvidia,pins = "ld17_0", "ld19_18", "ld21_20",
+ "ld23_22";
+ nvidia,pull = <TEGRA_PIN_PULL_DOWN>;
+ };
+
+ drive_sdio1 {
+ nvidia,pins = "drive_sdio1", "drive_ddc", "drive_vi1";
+ nvidia,high-speed-mode = <TEGRA_PIN_DISABLE>;
+ nvidia,schmitt = <TEGRA_PIN_ENABLE>;
+ nvidia,low-power-mode = <TEGRA_PIN_LP_DRIVE_DIV_1>;
+ nvidia,pull-down-strength = <31>;
+ nvidia,pull-up-strength = <31>;
+ nvidia,slew-rate-rising = <TEGRA_PIN_SLEW_RATE_SLOWEST>;
+ nvidia,slew-rate-falling = <TEGRA_PIN_SLEW_RATE_SLOWEST>;
+ };
+
+ drive_csus {
+ nvidia,pins = "drive_csus";
+ nvidia,high-speed-mode = <TEGRA_PIN_DISABLE>;
+ nvidia,schmitt = <TEGRA_PIN_DISABLE>;
+ nvidia,low-power-mode = <TEGRA_PIN_LP_DRIVE_DIV_1>;
+ nvidia,pull-down-strength = <31>;
+ nvidia,pull-up-strength = <31>;
+ nvidia,slew-rate-rising = <TEGRA_PIN_SLEW_RATE_SLOWEST>;
+ nvidia,slew-rate-falling = <TEGRA_PIN_SLEW_RATE_SLOWEST>;
+ };
+ };
+
+ state_i2cmux_ddc: pinmux_i2cmux_ddc {
+ ddc {
+ nvidia,pins = "ddc";
+ nvidia,function = "i2c2";
+ };
+
+ pta {
+ nvidia,pins = "pta";
+ nvidia,function = "rsvd4";
+ };
+ };
+
+ state_i2cmux_pta: pinmux_i2cmux_pta {
+ ddc {
+ nvidia,pins = "ddc";
+ nvidia,function = "rsvd4";
+ };
+
+ pta {
+ nvidia,pins = "pta";
+ nvidia,function = "i2c2";
+ };
+ };
+
+ state_i2cmux_idle: pinmux_i2cmux_idle {
+ ddc {
+ nvidia,pins = "ddc";
+ nvidia,function = "rsvd4";
+ };
+
+ pta {
+ nvidia,pins = "pta";
+ nvidia,function = "rsvd4";
+ };
+ };
+ };
+
+ i2s@70002800 {
+ status = "okay";
+ };
+
+ serial@70006040 {
+ compatible = "nvidia,tegra20-hsuart";
+ /* GPS BCM4751 */
+ };
+
+ serial@70006200 {
+ compatible = "nvidia,tegra20-hsuart";
+ status = "okay";
+
+ /* Azurewave AW-NH615 BCM4329B1 */
+ bluetooth {
+ compatible = "brcm,bcm4329-bt";
+
+ interrupt-parent = <&gpio>;
+ interrupts = <TEGRA_GPIO(U, 6) IRQ_TYPE_EDGE_RISING>;
+ interrupt-names = "host-wakeup";
+
+ /* PLLP 216MHz / 16 / 4 */
+ max-speed = <3375000>;
+
+ clocks = <&tegra_pmc TEGRA_PMC_CLK_BLINK>;
+ clock-names = "txco";
+
+ vbat-supply = <&vdd_3v3_sys>;
+ vddio-supply = <&vdd_1v8_sys>;
+
+ device-wakeup-gpios = <&gpio TEGRA_GPIO(U, 1) GPIO_ACTIVE_HIGH>;
+ shutdown-gpios = <&gpio TEGRA_GPIO(U, 0) GPIO_ACTIVE_HIGH>;
+ };
+ };
+
+ serial@70006300 {
+ status = "okay";
+ };
+
+ pwm@7000a000 {
+ status = "okay";
+ };
+
+ i2c@7000c000 {
+ status = "okay";
+ clock-frequency = <400000>;
+
+ /* Aichi AMI306 digital compass */
+ magnetometer@e {
+ compatible = "asahi-kasei,ak8974";
+ reg = <0xe>;
+
+ avdd-supply = <&vdd_3v3_sys>;
+ dvdd-supply = <&vdd_1v8_sys>;
+
+ mount-matrix = "-1", "0", "0",
+ "0", "1", "0",
+ "0", "0", "-1";
+ };
+
+ wm8903: audio-codec@1a {
+ compatible = "wlf,wm8903";
+ reg = <0x1a>;
+
+ interrupt-parent = <&gpio>;
+ interrupts = <TEGRA_GPIO(X, 1) IRQ_TYPE_EDGE_BOTH>;
+
+ gpio-controller;
+ #gpio-cells = <2>;
+
+ micdet-cfg = <0x83>;
+ micdet-delay = <100>;
+
+ gpio-cfg = <
+ 0xffffffff /* don't touch */
+ 0xffffffff /* don't touch */
+ 0x00000000 /* Speaker-enable GPIO, output, low */
+ 0x00000400 /* Mic bias current detect */
+ 0xffffffff /* don't touch */
+ >;
+
+ AVDD-supply = <&vdd_1v8_sys>;
+ CPVDD-supply = <&vdd_1v8_sys>;
+ DBVDD-supply = <&vdd_1v8_sys>;
+ DCVDD-supply = <&vdd_1v8_sys>;
+ };
+
+ /* Atmel MXT1386 Touchscreen */
+ touchscreen@5b {
+ compatible = "atmel,maxtouch";
+ reg = <0x5b>;
+
+ interrupt-parent = <&gpio>;
+ interrupts = <TEGRA_GPIO(V, 6) IRQ_TYPE_LEVEL_LOW>;
+
+ reset-gpios = <&gpio TEGRA_GPIO(Q, 7) GPIO_ACTIVE_LOW>;
+
+ vdda-supply = <&vdd_3v3_sys>;
+ vdd-supply = <&vdd_3v3_sys>;
+
+ atmel,wakeup-method = <ATMEL_MXT_WAKEUP_I2C_SCL>;
+ };
+
+ gyroscope@68 {
+ compatible = "invensense,mpu3050";
+ reg = <0x68>;
+
+ interrupt-parent = <&gpio>;
+ interrupts = <TEGRA_GPIO(Z, 4) IRQ_TYPE_EDGE_RISING>;
+
+ vdd-supply = <&vdd_3v3_sys>;
+ vlogic-supply = <&vdd_1v8_sys>;
+
+ mount-matrix = "0", "1", "0",
+ "-1", "0", "0",
+ "0", "0", "1";
+
+ i2c-gate {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ accelerometer@f {
+ compatible = "kionix,kxtf9";
+ reg = <0xf>;
+
+ interrupt-parent = <&gpio>;
+ interrupts = <TEGRA_GPIO(N, 4) IRQ_TYPE_EDGE_RISING>;
+
+ vdd-supply = <&vdd_1v8_sys>;
+ vddio-supply = <&vdd_1v8_sys>;
+
+ mount-matrix = "1", "0", "0",
+ "0", "1", "0",
+ "0", "0", "1";
+ };
+ };
+ };
+ };
+
+ i2c2: i2c@7000c400 {
+ status = "okay";
+ clock-frequency = <100000>;
+ };
+
+ i2c@7000c500 {
+ status = "okay";
+ clock-frequency = <400000>;
+ };
+
+ i2c@7000d000 {
+ status = "okay";
+ clock-frequency = <400000>;
+
+ pmic: pmic@34 {
+ compatible = "ti,tps6586x";
+ reg = <0x34>;
+ interrupts = <GIC_SPI 86 IRQ_TYPE_LEVEL_HIGH>;
+
+ ti,system-power-controller;
+
+ #gpio-cells = <2>;
+ gpio-controller;
+
+ sys-supply = <&vdd_5v0_sys>;
+ vin-sm0-supply = <&sys_reg>;
+ vin-sm1-supply = <&sys_reg>;
+ vin-sm2-supply = <&sys_reg>;
+ vinldo01-supply = <&sm2_reg>;
+ vinldo23-supply = <&sm2_reg>;
+ vinldo4-supply = <&sm2_reg>;
+ vinldo678-supply = <&sm2_reg>;
+ vinldo9-supply = <&sm2_reg>;
+
+ regulators {
+ sys_reg: sys {
+ regulator-name = "vdd_sys";
+ regulator-always-on;
+ };
+
+ vdd_core: sm0 {
+ regulator-name = "vdd_sm0,vdd_core";
+ regulator-min-microvolt = <950000>;
+ regulator-max-microvolt = <1300000>;
+ regulator-coupled-with = <&rtc_vdd &vdd_cpu>;
+ regulator-coupled-max-spread = <170000 550000>;
+ regulator-always-on;
+ regulator-boot-on;
+
+ nvidia,tegra-core-regulator;
+ };
+
+ vdd_cpu: sm1 {
+ regulator-name = "vdd_sm1,vdd_cpu";
+ regulator-min-microvolt = <750000>;
+ regulator-max-microvolt = <1125000>;
+ regulator-coupled-with = <&vdd_core &rtc_vdd>;
+ regulator-coupled-max-spread = <550000 550000>;
+ regulator-always-on;
+ regulator-boot-on;
+
+ nvidia,tegra-cpu-regulator;
+ };
+
+ sm2_reg: sm2 {
+ regulator-name = "vdd_sm2,vin_ldo*";
+ regulator-min-microvolt = <3700000>;
+ regulator-max-microvolt = <3700000>;
+ regulator-always-on;
+ };
+
+ /* LDO0 is not connected to anything */
+
+ ldo1 {
+ regulator-name = "vdd_ldo1,avdd_pll*";
+ regulator-min-microvolt = <1100000>;
+ regulator-max-microvolt = <1100000>;
+ regulator-always-on;
+ };
+
+ rtc_vdd: ldo2 {
+ regulator-name = "vdd_ldo2,vdd_rtc";
+ regulator-min-microvolt = <950000>;
+ regulator-max-microvolt = <1300000>;
+ regulator-coupled-with = <&vdd_core &vdd_cpu>;
+ regulator-coupled-max-spread = <170000 550000>;
+ regulator-always-on;
+ regulator-boot-on;
+
+ nvidia,tegra-rtc-regulator;
+ };
+
+ ldo3 {
+ regulator-name = "vdd_ldo3,avdd_usb*";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-always-on;
+ };
+
+ ldo4 {
+ regulator-name = "vdd_ldo4,avdd_osc,vddio_sys";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ regulator-always-on;
+ };
+
+ vcore_emmc: ldo5 {
+ regulator-name = "vdd_ldo5,vcore_mmc";
+ regulator-min-microvolt = <2850000>;
+ regulator-max-microvolt = <2850000>;
+ regulator-always-on;
+ };
+
+ ldo6 {
+ regulator-name = "vdd_ldo6,avdd_vdac";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ };
+
+ hdmi_vdd_reg: ldo7 {
+ regulator-name = "vdd_ldo7,avdd_hdmi,vdd_fuse";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ };
+
+ hdmi_pll_reg: ldo8 {
+ regulator-name = "vdd_ldo8,avdd_hdmi_pll";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ };
+
+ ldo9 {
+ regulator-name = "vdd_ldo9,avdd_2v85,vdd_ddr_rx";
+ regulator-min-microvolt = <2850000>;
+ regulator-max-microvolt = <2850000>;
+ regulator-always-on;
+ };
+
+ ldo_rtc {
+ regulator-name = "vdd_rtc_out,vdd_cell";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-always-on;
+ };
+ };
+ };
+
+ nct1008: temperature-sensor@4c {
+ compatible = "onnn,nct1008";
+ reg = <0x4c>;
+ vcc-supply = <&vdd_3v3_sys>;
+
+ interrupt-parent = <&gpio>;
+ interrupts = <TEGRA_GPIO(N, 6) IRQ_TYPE_EDGE_FALLING>;
+
+ #thermal-sensor-cells = <1>;
+ };
+ };
+
+ pmc@7000e400 {
+ nvidia,invert-interrupt;
+ nvidia,suspend-mode = <1>;
+ nvidia,cpu-pwr-good-time = <2000>;
+ nvidia,cpu-pwr-off-time = <100>;
+ nvidia,core-pwr-good-time = <3845 3845>;
+ nvidia,core-pwr-off-time = <458>;
+ nvidia,sys-clock-req-active-high;
+ core-supply = <&vdd_core>;
+ };
+
+ memory-controller@7000f400 {
+ nvidia,use-ram-code;
+
+ emc-tables@3 {
+ reg = <0x3>;
+
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ lpddr2 {
+ compatible = "elpida,B8132B2PB-6D-F", "jedec,lpddr2-s4";
+ revision-id1 = <1>;
+ density = <2048>;
+ io-width = <16>;
+ };
+
+ emc-table@25000 {
+ reg = <25000>;
+ compatible = "nvidia,tegra20-emc-table";
+ clock-frequency = <25000>;
+ nvidia,emc-registers = <0x00000002 0x00000006
+ 0x00000003 0x00000003 0x00000006 0x00000004
+ 0x00000002 0x00000009 0x00000003 0x00000003
+ 0x00000002 0x00000002 0x00000002 0x00000004
+ 0x00000003 0x00000008 0x0000000b 0x0000004d
+ 0x00000000 0x00000003 0x00000003 0x00000003
+ 0x00000008 0x00000001 0x0000000a 0x00000004
+ 0x00000003 0x00000008 0x00000004 0x00000006
+ 0x00000002 0x00000068 0x00000000 0x00000003
+ 0x00000000 0x00000000 0x00000282 0xa0ae04ae
+ 0x00070000 0x00000000 0x00000000 0x00000003
+ 0x00000000 0x00000000 0x00000000 0x00000000>;
+ };
+
+ emc-table@50000 {
+ reg = <50000>;
+ compatible = "nvidia,tegra20-emc-table";
+ clock-frequency = <50000>;
+ nvidia,emc-registers = <0x00000003 0x00000007
+ 0x00000003 0x00000003 0x00000006 0x00000004
+ 0x00000002 0x00000009 0x00000003 0x00000003
+ 0x00000002 0x00000002 0x00000002 0x00000005
+ 0x00000003 0x00000008 0x0000000b 0x0000009f
+ 0x00000000 0x00000003 0x00000003 0x00000003
+ 0x00000008 0x00000001 0x0000000a 0x00000007
+ 0x00000003 0x00000008 0x00000004 0x00000006
+ 0x00000002 0x000000d0 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000282 0xa0ae04ae
+ 0x00070000 0x00000000 0x00000000 0x00000005
+ 0x00000000 0x00000000 0x00000000 0x00000000>;
+ };
+
+ emc-table@75000 {
+ reg = <75000>;
+ compatible = "nvidia,tegra20-emc-table";
+ clock-frequency = <75000>;
+ nvidia,emc-registers = <0x00000005 0x0000000a
+ 0x00000004 0x00000003 0x00000006 0x00000004
+ 0x00000002 0x00000009 0x00000003 0x00000003
+ 0x00000002 0x00000002 0x00000002 0x00000005
+ 0x00000003 0x00000008 0x0000000b 0x000000ff
+ 0x00000000 0x00000003 0x00000003 0x00000003
+ 0x00000008 0x00000001 0x0000000a 0x0000000b
+ 0x00000003 0x00000008 0x00000004 0x00000006
+ 0x00000002 0x00000138 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000282 0xa0ae04ae
+ 0x00070000 0x00000000 0x00000000 0x00000007
+ 0x00000000 0x00000000 0x00000000 0x00000000>;
+ };
+
+ emc-table@150000 {
+ reg = <150000>;
+ compatible = "nvidia,tegra20-emc-table";
+ clock-frequency = <150000>;
+ nvidia,emc-registers = <0x00000009 0x00000014
+ 0x00000007 0x00000003 0x00000006 0x00000004
+ 0x00000002 0x00000009 0x00000003 0x00000003
+ 0x00000002 0x00000002 0x00000002 0x00000005
+ 0x00000003 0x00000008 0x0000000b 0x0000021f
+ 0x00000000 0x00000003 0x00000003 0x00000003
+ 0x00000008 0x00000001 0x0000000a 0x00000015
+ 0x00000003 0x00000008 0x00000004 0x00000006
+ 0x00000002 0x00000270 0x00000000 0x00000001
+ 0x00000000 0x00000000 0x00000282 0xa07c04ae
+ 0x007dc010 0x00000000 0x00000000 0x0000000e
+ 0x00000000 0x00000000 0x00000000 0x00000000>;
+ };
+
+ emc-table@300000 {
+ reg = <300000>;
+ compatible = "nvidia,tegra20-emc-table";
+ clock-frequency = <300000>;
+ nvidia,emc-registers = <0x00000012 0x00000027
+ 0x0000000d 0x00000006 0x00000007 0x00000005
+ 0x00000003 0x00000009 0x00000006 0x00000006
+ 0x00000003 0x00000003 0x00000002 0x00000006
+ 0x00000003 0x00000009 0x0000000c 0x0000045f
+ 0x00000000 0x00000004 0x00000004 0x00000006
+ 0x00000008 0x00000001 0x0000000e 0x0000002a
+ 0x00000003 0x0000000f 0x00000007 0x00000005
+ 0x00000002 0x000004e0 0x00000005 0x00000002
+ 0x00000000 0x00000000 0x00000282 0xe059048b
+ 0x007e0010 0x00000000 0x00000000 0x0000001b
+ 0x00000000 0x00000000 0x00000000 0x00000000>;
+ };
+ };
+ };
+
+ /* Peripheral USB via ASUS connector */
+ usb@c5000000 {
+ compatible = "nvidia,tegra20-udc";
+ status = "okay";
+ dr_mode = "peripheral";
+ };
+
+ usb-phy@c5000000 {
+ status = "okay";
+ dr_mode = "peripheral";
+ nvidia,xcvr-setup-use-fuses;
+ nvidia,xcvr-lsfslew = <2>;
+ nvidia,xcvr-lsrslew = <2>;
+ vbus-supply = <&vdd_5v0_sys>;
+ };
+
+ /* Dock's USB port */
+ usb@c5008000 {
+ status = "okay";
+ };
+
+ usb-phy@c5008000 {
+ status = "okay";
+ nvidia,xcvr-setup-use-fuses;
+ vbus-supply = <&vdd_5v0_sys>;
+ };
+
+ sdmmc1: mmc@c8000000 {
+ status = "okay";
+
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ assigned-clocks = <&tegra_car TEGRA20_CLK_SDMMC1>;
+ assigned-clock-parents = <&tegra_car TEGRA20_CLK_PLL_C>;
+ assigned-clock-rates = <40000000>;
+
+ max-frequency = <40000000>;
+ keep-power-in-suspend;
+ bus-width = <4>;
+ non-removable;
+
+ mmc-pwrseq = <&brcm_wifi_pwrseq>;
+ vmmc-supply = <&vdd_3v3_sys>;
+ vqmmc-supply = <&vdd_3v3_sys>;
+
+ /* Azurewave AW-NH615 BCM4329B1 */
+ wifi@1 {
+ compatible = "brcm,bcm4329-fmac";
+ reg = <1>;
+
+ interrupt-parent = <&gpio>;
+ interrupts = <TEGRA_GPIO(S, 0) IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "host-wake";
+ };
+ };
+
+ sdmmc3: mmc@c8000400 {
+ status = "okay";
+ bus-width = <4>;
+ cd-gpios = <&gpio TEGRA_GPIO(I, 5) GPIO_ACTIVE_LOW>;
+ wp-gpios = <&gpio TEGRA_GPIO(H, 1) GPIO_ACTIVE_HIGH>;
+ power-gpios = <&gpio TEGRA_GPIO(I, 6) GPIO_ACTIVE_HIGH>;
+ vmmc-supply = <&vdd_3v3_sys>;
+ vqmmc-supply = <&vdd_3v3_sys>;
+ };
+
+ sdmmc4: mmc@c8000600 {
+ status = "okay";
+ bus-width = <8>;
+ vmmc-supply = <&vcore_emmc>;
+ vqmmc-supply = <&vdd_3v3_sys>;
+ non-removable;
+ };
+
+ mains: ac-adapter-detect {
+ compatible = "gpio-charger";
+ charger-type = "mains";
+ gpios = <&gpio TEGRA_GPIO(V, 3) GPIO_ACTIVE_LOW>;
+ };
+
+ backlight: backlight {
+ compatible = "pwm-backlight";
+
+ enable-gpios = <&gpio TEGRA_GPIO(D, 4) GPIO_ACTIVE_HIGH>;
+ power-supply = <&vdd_3v3_sys>;
+ pwms = <&pwm 2 4000000>;
+
+ brightness-levels = <7 255>;
+ num-interpolated-steps = <248>;
+ default-brightness-level = <20>;
+ };
+
+ /* PMIC has a built-in 32KHz oscillator which is used by PMC */
+ clk32k_in: clock-32k-in {
+ compatible = "fixed-clock";
+ clock-frequency = <32768>;
+ #clock-cells = <0>;
+ };
+
+ cpus {
+ cpu0: cpu@0 {
+ cpu-supply = <&vdd_cpu>;
+ operating-points-v2 = <&cpu0_opp_table>;
+ #cooling-cells = <2>;
+ };
+
+ cpu1: cpu@1 {
+ cpu-supply = <&vdd_cpu>;
+ operating-points-v2 = <&cpu0_opp_table>;
+ #cooling-cells = <2>;
+ };
+ };
+
+ gpio-keys {
+ compatible = "gpio-keys";
+
+ dock-hall-sensor {
+ label = "Lid";
+ gpios = <&gpio TEGRA_GPIO(S, 4) GPIO_ACTIVE_LOW>;
+ linux,input-type = <EV_SW>;
+ linux,code = <SW_LID>;
+ debounce-interval = <500>;
+ wakeup-event-action = <EV_ACT_ASSERTED>;
+ wakeup-source;
+ };
+
+ power {
+ label = "Power";
+ gpios = <&gpio TEGRA_GPIO(V, 2) GPIO_ACTIVE_LOW>;
+ linux,code = <KEY_POWER>;
+ debounce-interval = <10>;
+ wakeup-event-action = <EV_ACT_ASSERTED>;
+ wakeup-source;
+ };
+
+ volume-up {
+ label = "Volume Up";
+ gpios = <&gpio TEGRA_GPIO(Q, 5) GPIO_ACTIVE_LOW>;
+ linux,code = <KEY_VOLUMEUP>;
+ debounce-interval = <10>;
+ wakeup-event-action = <EV_ACT_ASSERTED>;
+ wakeup-source;
+ };
+
+ volume-down {
+ label = "Volume Down";
+ gpios = <&gpio TEGRA_GPIO(Q, 4) GPIO_ACTIVE_LOW>;
+ linux,code = <KEY_VOLUMEDOWN>;
+ debounce-interval = <10>;
+ wakeup-event-action = <EV_ACT_ASSERTED>;
+ wakeup-source;
+ };
+ };
+
+ display-panel {
+ compatible = "panel-lvds";
+
+ /* AUO B101EW05 using custom timings */
+
+ backlight = <&backlight>;
+ ddc-i2c-bus = <&lvds_ddc>;
+ power-supply = <&vdd_pnl_reg>;
+
+ width-mm = <218>;
+ height-mm = <135>;
+
+ data-mapping = "jeida-18";
+
+ panel-timing {
+ clock-frequency = <71200000>;
+ hactive = <1280>;
+ vactive = <800>;
+ hfront-porch = <8>;
+ hback-porch = <18>;
+ hsync-len = <184>;
+ vsync-len = <3>;
+ vfront-porch = <4>;
+ vback-porch = <8>;
+ };
+
+ port {
+ panel_input: endpoint {
+ remote-endpoint = <&lvds_encoder_output>;
+ };
+ };
+ };
+
+ i2cmux {
+ compatible = "i2c-mux-pinctrl";
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ i2c-parent = <&i2c2>;
+
+ pinctrl-names = "ddc", "pta", "idle";
+ pinctrl-0 = <&state_i2cmux_ddc>;
+ pinctrl-1 = <&state_i2cmux_pta>;
+ pinctrl-2 = <&state_i2cmux_idle>;
+
+ hdmi_ddc: i2c@0 {
+ reg = <0>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ };
+
+ lvds_ddc: i2c@1 {
+ reg = <1>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ smart-battery@b {
+ compatible = "ti,bq20z75", "sbs,sbs-battery";
+ reg = <0xb>;
+ sbs,i2c-retry-count = <2>;
+ sbs,poll-retry-count = <10>;
+ power-supplies = <&mains>;
+ };
+ };
+ };
+
+ lvds-encoder {
+ compatible = "ti,sn75lvds83", "lvds-encoder";
+
+ powerdown-gpios = <&gpio TEGRA_GPIO(B, 2) GPIO_ACTIVE_LOW>;
+ power-supply = <&vdd_3v3_sys>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+
+ lvds_encoder_input: endpoint {
+ remote-endpoint = <&lcd_output>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+
+ lvds_encoder_output: endpoint {
+ remote-endpoint = <&panel_input>;
+ };
+ };
+ };
+ };
+
+ vdd_5v0_sys: regulator-5v0 {
+ compatible = "regulator-fixed";
+ regulator-name = "vdd_5v0";
+ regulator-min-microvolt = <5000000>;
+ regulator-max-microvolt = <5000000>;
+ regulator-always-on;
+ };
+
+ vdd_3v3_sys: regulator-3v3 {
+ compatible = "regulator-fixed";
+ regulator-name = "vdd_3v3_vs";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-always-on;
+ vin-supply = <&vdd_5v0_sys>;
+ };
+
+ regulator-pcie {
+ compatible = "regulator-fixed";
+ regulator-name = "pcie_vdd";
+ regulator-min-microvolt = <1500000>;
+ regulator-max-microvolt = <1500000>;
+ gpio = <&pmic 0 GPIO_ACTIVE_HIGH>;
+ regulator-always-on;
+ };
+
+ vdd_pnl_reg: regulator-panel {
+ compatible = "regulator-fixed";
+ regulator-name = "vdd_pnl";
+ regulator-min-microvolt = <2800000>;
+ regulator-max-microvolt = <2800000>;
+ gpio = <&gpio TEGRA_GPIO(C, 6) GPIO_ACTIVE_HIGH>;
+ enable-active-high;
+ };
+
+ vdd_1v8_sys: regulator-1v8 {
+ compatible = "regulator-fixed";
+ regulator-name = "vdd_1v8_vs";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ regulator-always-on;
+ vin-supply = <&vdd_5v0_sys>;
+ };
+
+ vdd_hdmi_en: regulator-hdmi {
+ compatible = "regulator-fixed";
+ regulator-name = "vdd_5v0_hdmi_en";
+ regulator-min-microvolt = <5000000>;
+ regulator-max-microvolt = <5000000>;
+ regulator-always-on;
+ vin-supply = <&vdd_5v0_sys>;
+ gpio = <&gpio TEGRA_GPIO(V, 5) GPIO_ACTIVE_HIGH>;
+ enable-active-high;
+ };
+
+ sound {
+ compatible = "asus,tegra-audio-wm8903-tf101",
+ "nvidia,tegra-audio-wm8903";
+ nvidia,model = "Asus EeePad Transformer WM8903";
+
+ nvidia,audio-routing =
+ "Headphone Jack", "HPOUTR",
+ "Headphone Jack", "HPOUTL",
+ "Int Spk", "ROP",
+ "Int Spk", "RON",
+ "Int Spk", "LOP",
+ "Int Spk", "LON",
+ "Mic Jack", "MICBIAS",
+ "IN1L", "Mic Jack";
+
+ nvidia,i2s-controller = <&tegra_i2s1>;
+ nvidia,audio-codec = <&wm8903>;
+
+ nvidia,spkr-en-gpios = <&wm8903 2 GPIO_ACTIVE_HIGH>;
+ nvidia,hp-det-gpios = <&gpio TEGRA_GPIO(W, 2) GPIO_ACTIVE_LOW>;
+ nvidia,headset;
+
+ clocks = <&tegra_car TEGRA20_CLK_PLL_A>,
+ <&tegra_car TEGRA20_CLK_PLL_A_OUT0>,
+ <&tegra_car TEGRA20_CLK_CDEV1>;
+ clock-names = "pll_a", "pll_a_out0", "mclk";
+ };
+
+ thermal-zones {
+ /*
+ * NCT1008 has two sensors:
+ *
+ * 0: internal that monitors ambient/skin temperature
+ * 1: external that is connected to the CPU's diode
+ *
+ * Ideally we should use userspace thermal governor,
+ * but it's a much more complex solution. The "skin"
+ * zone is a simpler solution which prevents TF101 from
+ * getting too hot from a user's tactile perspective.
+ * The CPU zone is intended to protect silicon from damage.
+ */
+
+ skin-thermal {
+ polling-delay-passive = <1000>; /* milliseconds */
+ polling-delay = <5000>; /* milliseconds */
+
+ thermal-sensors = <&nct1008 0>;
+
+ trips {
+ trip0: skin-alert {
+ /* start throttling at 60C */
+ temperature = <60000>;
+ hysteresis = <200>;
+ type = "passive";
+ };
+
+ trip1: skin-crit {
+ /* shut down at 70C */
+ temperature = <70000>;
+ hysteresis = <2000>;
+ type = "critical";
+ };
+ };
+
+ cooling-maps {
+ map0 {
+ trip = <&trip0>;
+ cooling-device = <&cpu0 THERMAL_NO_LIMIT THERMAL_NO_LIMIT>,
+ <&cpu1 THERMAL_NO_LIMIT THERMAL_NO_LIMIT>;
+ };
+ };
+ };
+
+ cpu-thermal {
+ polling-delay-passive = <1000>; /* milliseconds */
+ polling-delay = <5000>; /* milliseconds */
+
+ thermal-sensors = <&nct1008 1>;
+
+ trips {
+ trip2: cpu-alert {
+ /* throttle at 85C until temperature drops to 84.8C */
+ temperature = <85000>;
+ hysteresis = <200>;
+ type = "passive";
+ };
+
+ trip3: cpu-crit {
+ /* shut down at 90C */
+ temperature = <90000>;
+ hysteresis = <2000>;
+ type = "critical";
+ };
+ };
+
+ cooling-maps {
+ map1 {
+ trip = <&trip2>;
+ cooling-device = <&cpu0 THERMAL_NO_LIMIT THERMAL_NO_LIMIT>,
+ <&cpu1 THERMAL_NO_LIMIT THERMAL_NO_LIMIT>;
+ };
+ };
+ };
+ };
+
+ brcm_wifi_pwrseq: wifi-pwrseq {
+ compatible = "mmc-pwrseq-simple";
+
+ clocks = <&tegra_pmc TEGRA_PMC_CLK_BLINK>;
+ clock-names = "ext_clock";
+
+ reset-gpios = <&gpio TEGRA_GPIO(K, 6) GPIO_ACTIVE_LOW>;
+ post-power-on-delay-ms = <200>;
+ power-off-delay-us = <200>;
+ };
+};
+
+&emc_icc_dvfs_opp_table {
+ /delete-node/ opp-666000000;
+ /delete-node/ opp-760000000;
+};
diff --git a/arch/arm/boot/dts/tegra20-colibri-eval-v3.dts b/arch/arm/boot/dts/tegra20-colibri-eval-v3.dts
index a05fb38..d2a3bf9 100644
--- a/arch/arm/boot/dts/tegra20-colibri-eval-v3.dts
+++ b/arch/arm/boot/dts/tegra20-colibri-eval-v3.dts
@@ -70,11 +70,11 @@ mmccd {
nvidia,tristate = <TEGRA_PIN_DISABLE>;
};
- pwm-a-b {
+ sdc {
nvidia,tristate = <TEGRA_PIN_DISABLE>;
};
- pwm-c-d {
+ sdb_sdd {
nvidia,tristate = <TEGRA_PIN_DISABLE>;
};
diff --git a/arch/arm/boot/dts/tegra20-colibri-iris.dts b/arch/arm/boot/dts/tegra20-colibri-iris.dts
index 425494b..00ecbbd 100644
--- a/arch/arm/boot/dts/tegra20-colibri-iris.dts
+++ b/arch/arm/boot/dts/tegra20-colibri-iris.dts
@@ -70,11 +70,11 @@ mmccd {
nvidia,tristate = <TEGRA_PIN_DISABLE>;
};
- pwm-a-b {
+ sdc {
nvidia,tristate = <TEGRA_PIN_DISABLE>;
};
- pwm-c-d {
+ sdb_sdd {
nvidia,tristate = <TEGRA_PIN_DISABLE>;
};
diff --git a/arch/arm/boot/dts/tegra20-colibri.dtsi b/arch/arm/boot/dts/tegra20-colibri.dtsi
index 585a5b4..1eefb9e 100644
--- a/arch/arm/boot/dts/tegra20-colibri.dtsi
+++ b/arch/arm/boot/dts/tegra20-colibri.dtsi
@@ -113,7 +113,7 @@ bl-on {
};
/* Colibri Backlight PWM<A>, PWM<B> */
- pwm-a-b {
+ sdc {
nvidia,pins = "sdc";
nvidia,function = "pwm";
nvidia,tristate = <TEGRA_PIN_ENABLE>;
@@ -242,7 +242,7 @@ cif {
};
/* Colibri PWM<C>, PWM<D> */
- pwm-c-d {
+ sdb_sdd {
nvidia,pins = "sdb", "sdd";
nvidia,function = "pwm";
nvidia,tristate = <TEGRA_PIN_ENABLE>;
@@ -428,10 +428,12 @@ tegra_ac97: ac97@70002000 {
serial@70006040 {
compatible = "nvidia,tegra20-hsuart";
+ /delete-property/ reg-shift;
};
serial@70006300 {
compatible = "nvidia,tegra20-hsuart";
+ /delete-property/ reg-shift;
};
nand-controller@70008000 {
@@ -495,7 +497,7 @@ reg_3v3_vsys: sys {
regulator-always-on;
};
- sm0 {
+ vdd_core: sm0 {
regulator-name = "VDD_CORE_1.2V";
regulator-min-microvolt = <1200000>;
regulator-max-microvolt = <1200000>;
@@ -601,6 +603,7 @@ pmc@7000e400 {
nvidia,core-pwr-good-time = <3845 3845>;
nvidia,core-pwr-off-time = <3875>;
nvidia,sys-clock-req-active-high;
+ core-supply = <&vdd_core>;
/* Set SLEEP MODE bit in SUPPLYENE register of TPS658643 PMIC */
i2c-thermtrip {
@@ -689,6 +692,7 @@ usb@c5004000 {
#size-cells = <0>;
asix@1 {
+ compatible = "usbb95,772b";
reg = <1>;
local-mac-address = [00 00 00 00 00 00];
};
@@ -743,11 +747,11 @@ sound {
};
&emc_icc_dvfs_opp_table {
- /delete-node/ opp@760000000;
+ /delete-node/ opp-760000000;
};
&gpio {
- lan-reset-n {
+ lan-reset-n-hog {
gpio-hog;
gpios = <TEGRA_GPIO(V, 4) GPIO_ACTIVE_HIGH>;
output-high;
@@ -755,7 +759,7 @@ lan-reset-n {
};
/* Tri-stating GMI_WR_N on SODIMM pin 99 nPWE */
- npwe {
+ npwe-hog {
gpio-hog;
gpios = <TEGRA_GPIO(T, 5) GPIO_ACTIVE_HIGH>;
output-high;
@@ -763,7 +767,7 @@ npwe {
};
/* Not tri-stating GMI_WR_N on SODIMM pin 93 RDnWR */
- rdnwr {
+ rdnwr-hog {
gpio-hog;
gpios = <TEGRA_GPIO(T, 6) GPIO_ACTIVE_HIGH>;
output-low;
diff --git a/arch/arm/boot/dts/tegra20-cpu-opp-microvolt.dtsi b/arch/arm/boot/dts/tegra20-cpu-opp-microvolt.dtsi
index 6f3e8c5..7330c1b 100644
--- a/arch/arm/boot/dts/tegra20-cpu-opp-microvolt.dtsi
+++ b/arch/arm/boot/dts/tegra20-cpu-opp-microvolt.dtsi
@@ -1,164 +1,164 @@
// SPDX-License-Identifier: GPL-2.0
/ {
- cpu0_opp_table: cpu_opp_table0 {
- opp@216000000,750 {
+ cpu0_opp_table: opp-table-cpu0 {
+ opp-216000000-750 {
opp-microvolt = <750000 750000 1125000>;
};
- opp@216000000,800 {
+ opp-216000000-800 {
opp-microvolt = <800000 800000 1125000>;
};
- opp@312000000,750 {
+ opp-312000000-750 {
opp-microvolt = <750000 750000 1125000>;
};
- opp@312000000,800 {
+ opp-312000000-800 {
opp-microvolt = <800000 800000 1125000>;
};
- opp@456000000,750 {
+ opp-456000000-750 {
opp-microvolt = <750000 750000 1125000>;
};
- opp@456000000,800 {
+ opp-456000000-800 {
opp-microvolt = <800000 800000 1125000>;
};
- opp@456000000,825 {
+ opp-456000000-825 {
opp-microvolt = <825000 825000 1125000>;
};
- opp@608000000,750 {
+ opp-608000000-750 {
opp-microvolt = <750000 750000 1125000>;
};
- opp@608000000,800 {
+ opp-608000000-800 {
opp-microvolt = <800000 800000 1125000>;
};
- opp@608000000,825 {
+ opp-608000000-825 {
opp-microvolt = <825000 825000 1125000>;
};
- opp@608000000,850 {
+ opp-608000000-850 {
opp-microvolt = <850000 850000 1125000>;
};
- opp@608000000,900 {
+ opp-608000000-900 {
opp-microvolt = <900000 900000 1125000>;
};
- opp@760000000,775 {
+ opp-760000000-775 {
opp-microvolt = <775000 775000 1125000>;
};
- opp@760000000,800 {
+ opp-760000000-800 {
opp-microvolt = <800000 800000 1125000>;
};
- opp@760000000,850 {
+ opp-760000000-850 {
opp-microvolt = <850000 850000 1125000>;
};
- opp@760000000,875 {
+ opp-760000000-875 {
opp-microvolt = <875000 875000 1125000>;
};
- opp@760000000,900 {
+ opp-760000000-900 {
opp-microvolt = <900000 900000 1125000>;
};
- opp@760000000,975 {
+ opp-760000000-975 {
opp-microvolt = <975000 975000 1125000>;
};
- opp@816000000,800 {
+ opp-816000000-800 {
opp-microvolt = <800000 800000 1125000>;
};
- opp@816000000,850 {
+ opp-816000000-850 {
opp-microvolt = <850000 850000 1125000>;
};
- opp@816000000,875 {
+ opp-816000000-875 {
opp-microvolt = <875000 875000 1125000>;
};
- opp@816000000,950 {
+ opp-816000000-950 {
opp-microvolt = <950000 950000 1125000>;
};
- opp@816000000,1000 {
+ opp-816000000-1000 {
opp-microvolt = <1000000 1000000 1125000>;
};
- opp@912000000,850 {
+ opp-912000000-850 {
opp-microvolt = <850000 850000 1125000>;
};
- opp@912000000,900 {
+ opp-912000000-900 {
opp-microvolt = <900000 900000 1125000>;
};
- opp@912000000,925 {
+ opp-912000000-925 {
opp-microvolt = <925000 925000 1125000>;
};
- opp@912000000,950 {
+ opp-912000000-950 {
opp-microvolt = <950000 950000 1125000>;
};
- opp@912000000,1000 {
+ opp-912000000-1000 {
opp-microvolt = <1000000 1000000 1125000>;
};
- opp@912000000,1050 {
+ opp-912000000-1050 {
opp-microvolt = <1050000 1050000 1125000>;
};
- opp@1000000000,875 {
+ opp-1000000000-875 {
opp-microvolt = <875000 875000 1125000>;
};
- opp@1000000000,900 {
+ opp-1000000000-900 {
opp-microvolt = <900000 900000 1125000>;
};
- opp@1000000000,950 {
+ opp-1000000000-950 {
opp-microvolt = <950000 950000 1125000>;
};
- opp@1000000000,975 {
+ opp-1000000000-975 {
opp-microvolt = <975000 975000 1125000>;
};
- opp@1000000000,1000 {
+ opp-1000000000-1000 {
opp-microvolt = <1000000 1000000 1125000>;
};
- opp@1000000000,1025 {
+ opp-1000000000-1025 {
opp-microvolt = <1025000 1025000 1125000>;
};
- opp@1000000000,1100 {
+ opp-1000000000-1100 {
opp-microvolt = <1100000 1100000 1125000>;
};
- opp@1200000000,1000 {
+ opp-1200000000-1000 {
opp-microvolt = <1000000 1000000 1125000>;
};
- opp@1200000000,1050 {
+ opp-1200000000-1050 {
opp-microvolt = <1050000 1050000 1125000>;
};
- opp@1200000000,1100 {
+ opp-1200000000-1100 {
opp-microvolt = <1100000 1100000 1125000>;
};
- opp@1200000000,1125 {
+ opp-1200000000-1125 {
opp-microvolt = <1125000 1125000 1125000>;
};
};
diff --git a/arch/arm/boot/dts/tegra20-cpu-opp.dtsi b/arch/arm/boot/dts/tegra20-cpu-opp.dtsi
index 135de31..47c8e78 100644
--- a/arch/arm/boot/dts/tegra20-cpu-opp.dtsi
+++ b/arch/arm/boot/dts/tegra20-cpu-opp.dtsi
@@ -1,250 +1,250 @@
// SPDX-License-Identifier: GPL-2.0
/ {
- cpu0_opp_table: cpu_opp_table0 {
+ cpu0_opp_table: opp-table-cpu0 {
compatible = "operating-points-v2";
opp-shared;
- opp@216000000,750 {
+ opp-216000000-750 {
clock-latency-ns = <400000>;
opp-supported-hw = <0x0F 0x0003>;
opp-hz = /bits/ 64 <216000000>;
opp-suspend;
};
- opp@216000000,800 {
+ opp-216000000-800 {
clock-latency-ns = <400000>;
opp-supported-hw = <0x0F 0x0004>;
opp-hz = /bits/ 64 <216000000>;
opp-suspend;
};
- opp@312000000,750 {
+ opp-312000000-750 {
clock-latency-ns = <400000>;
opp-supported-hw = <0x0F 0x0003>;
opp-hz = /bits/ 64 <312000000>;
};
- opp@312000000,800 {
+ opp-312000000-800 {
clock-latency-ns = <400000>;
opp-supported-hw = <0x0F 0x0004>;
opp-hz = /bits/ 64 <312000000>;
};
- opp@456000000,750 {
+ opp-456000000-750 {
clock-latency-ns = <400000>;
opp-supported-hw = <0x0C 0x0003>;
opp-hz = /bits/ 64 <456000000>;
};
- opp@456000000,800 {
+ opp-456000000-800 {
clock-latency-ns = <400000>;
opp-supported-hw = <0x03 0x0006>, <0x04 0x0004>,
<0x08 0x0004>;
opp-hz = /bits/ 64 <456000000>;
};
- opp@456000000,825 {
+ opp-456000000-825 {
clock-latency-ns = <400000>;
opp-supported-hw = <0x03 0x0001>;
opp-hz = /bits/ 64 <456000000>;
};
- opp@608000000,750 {
+ opp-608000000-750 {
clock-latency-ns = <400000>;
opp-supported-hw = <0x08 0x0003>;
opp-hz = /bits/ 64 <608000000>;
};
- opp@608000000,800 {
+ opp-608000000-800 {
clock-latency-ns = <400000>;
opp-supported-hw = <0x04 0x0006>, <0x08 0x0004>;
opp-hz = /bits/ 64 <608000000>;
};
- opp@608000000,825 {
+ opp-608000000-825 {
clock-latency-ns = <400000>;
opp-supported-hw = <0x04 0x0001>;
opp-hz = /bits/ 64 <608000000>;
};
- opp@608000000,850 {
+ opp-608000000-850 {
clock-latency-ns = <400000>;
opp-supported-hw = <0x03 0x0006>;
opp-hz = /bits/ 64 <608000000>;
};
- opp@608000000,900 {
+ opp-608000000-900 {
clock-latency-ns = <400000>;
opp-supported-hw = <0x03 0x0001>;
opp-hz = /bits/ 64 <608000000>;
};
- opp@760000000,775 {
+ opp-760000000-775 {
clock-latency-ns = <400000>;
opp-supported-hw = <0x08 0x0003>;
opp-hz = /bits/ 64 <760000000>;
};
- opp@760000000,800 {
+ opp-760000000-800 {
clock-latency-ns = <400000>;
opp-supported-hw = <0x08 0x0004>;
opp-hz = /bits/ 64 <760000000>;
};
- opp@760000000,850 {
+ opp-760000000-850 {
clock-latency-ns = <400000>;
opp-supported-hw = <0x04 0x0006>;
opp-hz = /bits/ 64 <760000000>;
};
- opp@760000000,875 {
+ opp-760000000-875 {
clock-latency-ns = <400000>;
opp-supported-hw = <0x04 0x0001>, <0x02 0x0002>,
<0x01 0x0004>, <0x02 0x0004>;
opp-hz = /bits/ 64 <760000000>;
};
- opp@760000000,900 {
+ opp-760000000-900 {
clock-latency-ns = <400000>;
opp-supported-hw = <0x01 0x0002>;
opp-hz = /bits/ 64 <760000000>;
};
- opp@760000000,975 {
+ opp-760000000-975 {
clock-latency-ns = <400000>;
opp-supported-hw = <0x03 0x0001>;
opp-hz = /bits/ 64 <760000000>;
};
- opp@816000000,800 {
+ opp-816000000-800 {
clock-latency-ns = <400000>;
opp-supported-hw = <0x08 0x0007>;
opp-hz = /bits/ 64 <816000000>;
};
- opp@816000000,850 {
+ opp-816000000-850 {
clock-latency-ns = <400000>;
opp-supported-hw = <0x04 0x0002>;
opp-hz = /bits/ 64 <816000000>;
};
- opp@816000000,875 {
+ opp-816000000-875 {
clock-latency-ns = <400000>;
opp-supported-hw = <0x04 0x0005>;
opp-hz = /bits/ 64 <816000000>;
};
- opp@816000000,950 {
+ opp-816000000-950 {
clock-latency-ns = <400000>;
opp-supported-hw = <0x03 0x0006>;
opp-hz = /bits/ 64 <816000000>;
};
- opp@816000000,1000 {
+ opp-816000000-1000 {
clock-latency-ns = <400000>;
opp-supported-hw = <0x03 0x0001>;
opp-hz = /bits/ 64 <816000000>;
};
- opp@912000000,850 {
+ opp-912000000-850 {
clock-latency-ns = <400000>;
opp-supported-hw = <0x08 0x0007>;
opp-hz = /bits/ 64 <912000000>;
};
- opp@912000000,900 {
+ opp-912000000-900 {
clock-latency-ns = <400000>;
opp-supported-hw = <0x04 0x0002>;
opp-hz = /bits/ 64 <912000000>;
};
- opp@912000000,925 {
+ opp-912000000-925 {
clock-latency-ns = <400000>;
opp-supported-hw = <0x04 0x0001>;
opp-hz = /bits/ 64 <912000000>;
};
- opp@912000000,950 {
+ opp-912000000-950 {
clock-latency-ns = <400000>;
opp-supported-hw = <0x02 0x0006>, <0x01 0x0004>,
<0x04 0x0004>;
opp-hz = /bits/ 64 <912000000>;
};
- opp@912000000,1000 {
+ opp-912000000-1000 {
clock-latency-ns = <400000>;
opp-supported-hw = <0x01 0x0002>;
opp-hz = /bits/ 64 <912000000>;
};
- opp@912000000,1050 {
+ opp-912000000-1050 {
clock-latency-ns = <400000>;
opp-supported-hw = <0x03 0x0001>;
opp-hz = /bits/ 64 <912000000>;
};
- opp@1000000000,875 {
+ opp-1000000000-875 {
clock-latency-ns = <400000>;
opp-supported-hw = <0x08 0x0007>;
opp-hz = /bits/ 64 <1000000000>;
};
- opp@1000000000,900 {
+ opp-1000000000-900 {
clock-latency-ns = <400000>;
opp-supported-hw = <0x04 0x0002>;
opp-hz = /bits/ 64 <1000000000>;
};
- opp@1000000000,950 {
+ opp-1000000000-950 {
clock-latency-ns = <400000>;
opp-supported-hw = <0x04 0x0004>;
opp-hz = /bits/ 64 <1000000000>;
};
- opp@1000000000,975 {
+ opp-1000000000-975 {
clock-latency-ns = <400000>;
opp-supported-hw = <0x04 0x0001>;
opp-hz = /bits/ 64 <1000000000>;
};
- opp@1000000000,1000 {
+ opp-1000000000-1000 {
clock-latency-ns = <400000>;
opp-supported-hw = <0x02 0x0006>, <0x01 0x0004>;
opp-hz = /bits/ 64 <1000000000>;
};
- opp@1000000000,1025 {
+ opp-1000000000-1025 {
clock-latency-ns = <400000>;
opp-supported-hw = <0x01 0x0002>;
opp-hz = /bits/ 64 <1000000000>;
};
- opp@1000000000,1100 {
+ opp-1000000000-1100 {
clock-latency-ns = <400000>;
opp-supported-hw = <0x03 0x0001>;
opp-hz = /bits/ 64 <1000000000>;
};
- opp@1200000000,1000 {
+ opp-1200000000-1000 {
clock-latency-ns = <400000>;
opp-supported-hw = <0x08 0x0004>;
opp-hz = /bits/ 64 <1200000000>;
};
- opp@1200000000,1050 {
+ opp-1200000000-1050 {
clock-latency-ns = <400000>;
opp-supported-hw = <0x04 0x0004>;
opp-hz = /bits/ 64 <1200000000>;
};
- opp@1200000000,1100 {
+ opp-1200000000-1100 {
clock-latency-ns = <400000>;
opp-supported-hw = <0x02 0x0004>;
opp-hz = /bits/ 64 <1200000000>;
};
- opp@1200000000,1125 {
+ opp-1200000000-1125 {
clock-latency-ns = <400000>;
opp-supported-hw = <0x01 0x0004>;
opp-hz = /bits/ 64 <1200000000>;
diff --git a/arch/arm/boot/dts/tegra20-harmony.dts b/arch/arm/boot/dts/tegra20-harmony.dts
index ae4312e..79b6b79 100644
--- a/arch/arm/boot/dts/tegra20-harmony.dts
+++ b/arch/arm/boot/dts/tegra20-harmony.dts
@@ -339,7 +339,7 @@ sys_reg: sys {
regulator-always-on;
};
- sm0 {
+ vdd_core: sm0 {
regulator-name = "vdd_sm0,vdd_core";
regulator-min-microvolt = <1200000>;
regulator-max-microvolt = <1200000>;
@@ -565,6 +565,7 @@ pmc@7000e400 {
nvidia,core-pwr-good-time = <3845 3845>;
nvidia,core-pwr-off-time = <3875>;
nvidia,sys-clock-req-active-high;
+ core-supply = <&vdd_core>;
};
pcie@80003000 {
@@ -595,8 +596,6 @@ usb-phy@c5000000 {
usb@c5004000 {
status = "okay";
- nvidia,phy-reset-gpio = <&gpio TEGRA_GPIO(V, 1)
- GPIO_ACTIVE_LOW>;
};
usb-phy@c5004000 {
@@ -640,7 +639,7 @@ backlight: backlight {
default-brightness-level = <6>;
};
- clk32k_in: clock@0 {
+ clk32k_in: clock-32k {
compatible = "fixed-clock";
clock-frequency = <32768>;
#clock-cells = <0>;
@@ -666,7 +665,7 @@ panel: panel {
backlight = <&backlight>;
};
- vdd_5v0_reg: regulator@0 {
+ vdd_5v0_reg: regulator-5v0 {
compatible = "regulator-fixed";
regulator-name = "vdd_5v0";
regulator-min-microvolt = <5000000>;
@@ -674,7 +673,7 @@ vdd_5v0_reg: regulator@0 {
regulator-always-on;
};
- regulator@1 {
+ regulator-1v5 {
compatible = "regulator-fixed";
regulator-name = "vdd_1v5";
regulator-min-microvolt = <1500000>;
@@ -682,7 +681,7 @@ regulator@1 {
gpio = <&pmic 0 GPIO_ACTIVE_HIGH>;
};
- regulator@2 {
+ regulator-1v2 {
compatible = "regulator-fixed";
regulator-name = "vdd_1v2";
regulator-min-microvolt = <1200000>;
@@ -691,7 +690,7 @@ regulator@2 {
enable-active-high;
};
- pci_vdd_reg: regulator@3 {
+ pci_vdd_reg: regulator-1v05 {
compatible = "regulator-fixed";
regulator-name = "vdd_1v05";
regulator-min-microvolt = <1050000>;
@@ -700,7 +699,7 @@ pci_vdd_reg: regulator@3 {
enable-active-high;
};
- vdd_pnl_reg: regulator@4 {
+ vdd_pnl_reg: regulator-pn1 {
compatible = "regulator-fixed";
regulator-name = "vdd_pnl";
regulator-min-microvolt = <2800000>;
@@ -709,7 +708,7 @@ vdd_pnl_reg: regulator@4 {
enable-active-high;
};
- vdd_bl_reg: regulator@5 {
+ vdd_bl_reg: regulator-bl {
compatible = "regulator-fixed";
regulator-name = "vdd_bl";
regulator-min-microvolt = <2800000>;
@@ -718,7 +717,7 @@ vdd_bl_reg: regulator@5 {
enable-active-high;
};
- vdd_5v0_hdmi: regulator@6 {
+ vdd_5v0_hdmi: regulator-hdmi {
compatible = "regulator-fixed";
regulator-name = "VDDIO_HDMI";
regulator-min-microvolt = <5000000>;
diff --git a/arch/arm/boot/dts/tegra20-medcom-wide.dts b/arch/arm/boot/dts/tegra20-medcom-wide.dts
index b31c9bc..f144487c 100644
--- a/arch/arm/boot/dts/tegra20-medcom-wide.dts
+++ b/arch/arm/boot/dts/tegra20-medcom-wide.dts
@@ -54,6 +54,9 @@ backlight: backlight {
brightness-levels = <0 4 8 16 32 64 128 255>;
default-brightness-level = <6>;
+
+ /* close enough */
+ power-supply = <&vdd_3v3_reg>;
};
panel: panel {
@@ -92,7 +95,7 @@ sound {
clock-names = "pll_a", "pll_a_out0", "mclk";
};
- vcc_24v_reg: regulator@100 {
+ vcc_24v_reg: regulator-24v0 {
compatible = "regulator-fixed";
regulator-name = "vcc_24v";
regulator-min-microvolt = <24000000>;
@@ -100,7 +103,7 @@ vcc_24v_reg: regulator@100 {
regulator-always-on;
};
- vdd_5v0_reg: regulator@101 {
+ vdd_5v0_reg: regulator-5v0 {
compatible = "regulator-fixed";
regulator-name = "vdd_5v0";
vin-supply = <&vcc_24v_reg>;
@@ -109,7 +112,7 @@ vdd_5v0_reg: regulator@101 {
regulator-always-on;
};
- vdd_3v3_reg: regulator@102 {
+ vdd_3v3_reg: regulator-3v3 {
compatible = "regulator-fixed";
regulator-name = "vdd_3v3";
vin-supply = <&vcc_24v_reg>;
@@ -118,7 +121,7 @@ vdd_3v3_reg: regulator@102 {
regulator-always-on;
};
- vdd_1v8_reg: regulator@103 {
+ vdd_1v8_reg: regulator-1v8 {
compatible = "regulator-fixed";
regulator-name = "vdd_1v8";
vin-supply = <&vdd_3v3_reg>;
diff --git a/arch/arm/boot/dts/tegra20-paz00.dts b/arch/arm/boot/dts/tegra20-paz00.dts
index 5b38b06..d53a175 100644
--- a/arch/arm/boot/dts/tegra20-paz00.dts
+++ b/arch/arm/boot/dts/tegra20-paz00.dts
@@ -264,8 +264,16 @@ conf_ld17_0 {
};
};
+ spdif@70002400 {
+ status = "okay";
+
+ nvidia,fixed-parent-rate;
+ };
+
i2s@70002800 {
status = "okay";
+
+ nvidia,fixed-parent-rate;
};
serial@70006000 {
@@ -519,6 +527,7 @@ pmc@7000e400 {
nvidia,core-pwr-good-time = <3845 3845>;
nvidia,core-pwr-off-time = <0>;
nvidia,sys-clock-req-active-high;
+ core-supply = <&core_vdd_reg>;
};
usb@c5000000 {
@@ -533,8 +542,6 @@ usb-phy@c5000000 {
usb@c5004000 {
status = "okay";
- nvidia,phy-reset-gpio = <&gpio TEGRA_GPIO(V, 0)
- GPIO_ACTIVE_LOW>;
};
usb-phy@c5004000 {
@@ -573,9 +580,12 @@ backlight: backlight {
brightness-levels = <0 16 32 48 64 80 96 112 128 144 160 176 192 208 224 240 255>;
default-brightness-level = <10>;
+
+ /* close enough */
+ power-supply = <&vdd_pnl_reg>;
};
- clk32k_in: clock@0 {
+ clk32k_in: clock-32k {
compatible = "fixed-clock";
clock-frequency = <32768>;
#clock-cells = <0>;
@@ -612,7 +622,7 @@ panel: panel {
backlight = <&backlight>;
};
- p5valw_reg: regulator@0 {
+ p5valw_reg: regulator-5v0alw {
compatible = "regulator-fixed";
regulator-name = "+5valw";
regulator-min-microvolt = <5000000>;
@@ -620,7 +630,7 @@ p5valw_reg: regulator@0 {
regulator-always-on;
};
- vdd_pnl_reg: regulator@1 {
+ vdd_pnl_reg: regulator-3v0 {
compatible = "regulator-fixed";
regulator-name = "+3VS,vdd_pnl";
regulator-min-microvolt = <3300000>;
@@ -705,5 +715,5 @@ map0 {
};
&emc_icc_dvfs_opp_table {
- /delete-node/ opp@760000000;
+ /delete-node/ opp-760000000;
};
diff --git a/arch/arm/boot/dts/tegra20-peripherals-opp.dtsi b/arch/arm/boot/dts/tegra20-peripherals-opp.dtsi
index ef3ad2e..1b80823 100644
--- a/arch/arm/boot/dts/tegra20-peripherals-opp.dtsi
+++ b/arch/arm/boot/dts/tegra20-peripherals-opp.dtsi
@@ -1,110 +1,1022 @@
// SPDX-License-Identifier: GPL-2.0
/ {
- emc_icc_dvfs_opp_table: emc-dvfs-opp-table {
+ core_opp_table: opp-table-core {
+ compatible = "operating-points-v2";
+ opp-shared;
+
+ core_opp_950: opp-950000 {
+ opp-microvolt = <950000 950000 1300000>;
+ opp-level = <950000>;
+ };
+
+ core_opp_1000: opp-1000000 {
+ opp-microvolt = <1000000 1000000 1300000>;
+ opp-level = <1000000>;
+ };
+
+ core_opp_1100: opp-1100000 {
+ opp-microvolt = <1100000 1100000 1300000>;
+ opp-level = <1100000>;
+ };
+
+ core_opp_1200: opp-1200000 {
+ opp-microvolt = <1200000 1200000 1300000>;
+ opp-level = <1200000>;
+ };
+
+ core_opp_1225: opp-1225000 {
+ opp-microvolt = <1225000 1225000 1300000>;
+ opp-level = <1225000>;
+ };
+
+ core_opp_1275: opp-1275000 {
+ opp-microvolt = <1275000 1275000 1300000>;
+ opp-level = <1275000>;
+ };
+
+ core_opp_1300: opp-1300000 {
+ opp-microvolt = <1300000 1300000 1300000>;
+ opp-level = <1300000>;
+ };
+ };
+
+ emc_icc_dvfs_opp_table: opp-table-emc {
compatible = "operating-points-v2";
- opp@36000000 {
+ opp-36000000 {
opp-microvolt = <950000 950000 1300000>;
opp-hz = /bits/ 64 <36000000>;
opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_950>;
};
- opp@47500000 {
+ opp-47500000 {
opp-microvolt = <950000 950000 1300000>;
opp-hz = /bits/ 64 <47500000>;
opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_950>;
};
- opp@50000000 {
+ opp-50000000 {
opp-microvolt = <950000 950000 1300000>;
opp-hz = /bits/ 64 <50000000>;
opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_950>;
};
- opp@54000000 {
+ opp-54000000 {
opp-microvolt = <950000 950000 1300000>;
opp-hz = /bits/ 64 <54000000>;
opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_950>;
};
- opp@57000000 {
+ opp-57000000 {
opp-microvolt = <950000 950000 1300000>;
opp-hz = /bits/ 64 <57000000>;
opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_950>;
};
- opp@100000000 {
+ opp-100000000 {
opp-microvolt = <1000000 1000000 1300000>;
opp-hz = /bits/ 64 <100000000>;
opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1000>;
};
- opp@108000000 {
+ opp-108000000 {
opp-microvolt = <1000000 1000000 1300000>;
opp-hz = /bits/ 64 <108000000>;
opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1000>;
};
- opp@126666000 {
+ opp-126666000 {
opp-microvolt = <1000000 1000000 1300000>;
opp-hz = /bits/ 64 <126666000>;
opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1000>;
};
- opp@150000000 {
+ opp-150000000 {
opp-microvolt = <1000000 1000000 1300000>;
opp-hz = /bits/ 64 <150000000>;
opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1000>;
};
- opp@190000000 {
+ opp-190000000 {
opp-microvolt = <1000000 1000000 1300000>;
opp-hz = /bits/ 64 <190000000>;
opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1000>;
};
- opp@216000000 {
+ opp-216000000 {
opp-microvolt = <1000000 1000000 1300000>;
opp-hz = /bits/ 64 <216000000>;
opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1000>;
opp-suspend;
};
- opp@300000000 {
+ opp-300000000 {
opp-microvolt = <1000000 1000000 1300000>;
opp-hz = /bits/ 64 <300000000>;
opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1000>;
};
- opp@333000000 {
+ opp-333000000 {
opp-microvolt = <1000000 1000000 1300000>;
opp-hz = /bits/ 64 <333000000>;
opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1000>;
};
- opp@380000000 {
+ opp-380000000 {
opp-microvolt = <1100000 1100000 1300000>;
opp-hz = /bits/ 64 <380000000>;
opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1100>;
};
- opp@600000000 {
+ opp-600000000 {
opp-microvolt = <1200000 1200000 1300000>;
opp-hz = /bits/ 64 <600000000>;
opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1200>;
};
- opp@666000000 {
+ opp-666000000 {
opp-microvolt = <1200000 1200000 1300000>;
opp-hz = /bits/ 64 <666000000>;
opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1200>;
};
- opp@760000000 {
+ opp-760000000 {
opp-microvolt = <1300000 1300000 1300000>;
opp-hz = /bits/ 64 <760000000>;
opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1300>;
+ };
+ };
+
+ host1x_dvfs_opp_table: opp-table-host1x {
+ compatible = "operating-points-v2";
+
+ opp-104500000-950 {
+ opp-microvolt = <950000 950000 1300000>;
+ opp-hz = /bits/ 64 <104500000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_950>;
+ };
+
+ opp-133000000-1000 {
+ opp-microvolt = <1000000 1000000 1300000>;
+ opp-hz = /bits/ 64 <133000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-166000000-1100 {
+ opp-microvolt = <1100000 1100000 1300000>;
+ opp-hz = /bits/ 64 <166000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1100>;
+ };
+ };
+
+ mpe_dvfs_opp_table: opp-table-mpe {
+ compatible = "operating-points-v2";
+
+ opp-104500000-950 {
+ opp-microvolt = <950000 950000 1300000>;
+ opp-hz = /bits/ 64 <104500000>;
+ opp-supported-hw = <0x0001>;
+ required-opps = <&core_opp_950>;
+ };
+
+ opp-142500000-950 {
+ opp-microvolt = <950000 950000 1300000>;
+ opp-hz = /bits/ 64 <142500000>;
+ opp-supported-hw = <0x0002>;
+ required-opps = <&core_opp_950>;
+ };
+
+ opp-152000000-1000 {
+ opp-microvolt = <1000000 1000000 1300000>;
+ opp-hz = /bits/ 64 <152000000>;
+ opp-supported-hw = <0x0001>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-190000000-1000 {
+ opp-microvolt = <1000000 1000000 1300000>;
+ opp-hz = /bits/ 64 <190000000>;
+ opp-supported-hw = <0x0002>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-190000000-950 {
+ opp-microvolt = <950000 950000 1300000>;
+ opp-hz = /bits/ 64 <190000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_950>;
+ };
+
+ opp-228000000-1100 {
+ opp-microvolt = <1100000 1100000 1300000>;
+ opp-hz = /bits/ 64 <228000000>;
+ opp-supported-hw = <0x0001>;
+ required-opps = <&core_opp_1100>;
+ };
+
+ opp-228000000-950 {
+ opp-microvolt = <950000 950000 1300000>;
+ opp-hz = /bits/ 64 <228000000>;
+ opp-supported-hw = <0x0008>;
+ required-opps = <&core_opp_950>;
+ };
+
+ opp-237500000-1000 {
+ opp-microvolt = <1000000 1000000 1300000>;
+ opp-hz = /bits/ 64 <237500000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-266000000-1000 {
+ opp-microvolt = <1000000 1000000 1300000>;
+ opp-hz = /bits/ 64 <266000000>;
+ opp-supported-hw = <0x0008>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-275500000-1100 {
+ opp-microvolt = <1100000 1100000 1300000>;
+ opp-hz = /bits/ 64 <275500000>;
+ opp-supported-hw = <0x0002>;
+ required-opps = <&core_opp_1100>;
+ };
+
+ opp-300000000-1200 {
+ opp-microvolt = <1200000 1200000 1300000>;
+ opp-hz = /bits/ 64 <300000000>;
+ opp-supported-hw = <0x0003>;
+ required-opps = <&core_opp_1200>;
+ };
+
+ opp-300000000-1100 {
+ opp-microvolt = <1100000 1100000 1300000>;
+ opp-hz = /bits/ 64 <300000000>;
+ opp-supported-hw = <0x000C>;
+ required-opps = <&core_opp_1100>;
+ };
+ };
+
+ vi_dvfs_opp_table: opp-table-vi {
+ compatible = "operating-points-v2";
+
+ opp-85000000-950 {
+ opp-microvolt = <950000 950000 1300000>;
+ opp-hz = /bits/ 64 <85000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_950>;
+ };
+
+ opp-100000000-1000 {
+ opp-microvolt = <1000000 1000000 1300000>;
+ opp-hz = /bits/ 64 <100000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-150000000-1100 {
+ opp-microvolt = <1100000 1100000 1300000>;
+ opp-hz = /bits/ 64 <150000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1100>;
+ };
+ };
+
+ epp_dvfs_opp_table: opp-table-epp {
+ compatible = "operating-points-v2";
+
+ opp-133000000-950 {
+ opp-microvolt = <950000 950000 1300000>;
+ opp-hz = /bits/ 64 <133000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_950>;
+ };
+
+ opp-171000000-1000 {
+ opp-microvolt = <1000000 1000000 1300000>;
+ opp-hz = /bits/ 64 <171000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-247000000-1100 {
+ opp-microvolt = <1100000 1100000 1300000>;
+ opp-hz = /bits/ 64 <247000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1100>;
+ };
+
+ opp-300000000-1200 {
+ opp-microvolt = <1200000 1200000 1300000>;
+ opp-hz = /bits/ 64 <300000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1200>;
+ };
+ };
+
+ gr2d_dvfs_opp_table: opp-table-gr2d {
+ compatible = "operating-points-v2";
+
+ opp-133000000-950 {
+ opp-microvolt = <950000 950000 1300000>;
+ opp-hz = /bits/ 64 <133000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_950>;
+ };
+
+ opp-171000000-1000 {
+ opp-microvolt = <1000000 1000000 1300000>;
+ opp-hz = /bits/ 64 <171000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-247000000-1100 {
+ opp-microvolt = <1100000 1100000 1300000>;
+ opp-hz = /bits/ 64 <247000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1100>;
+ };
+
+ opp-300000000-1200 {
+ opp-microvolt = <1200000 1200000 1300000>;
+ opp-hz = /bits/ 64 <300000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1200>;
+ };
+ };
+
+ gr3d_dvfs_opp_table: opp-table-gr3d {
+ compatible = "operating-points-v2";
+
+ opp-114000000-950 {
+ opp-microvolt = <950000 950000 1300000>;
+ opp-hz = /bits/ 64 <114000000>;
+ opp-supported-hw = <0x0001>;
+ required-opps = <&core_opp_950>;
+ };
+
+ opp-161500000-1000 {
+ opp-microvolt = <1000000 1000000 1300000>;
+ opp-hz = /bits/ 64 <161500000>;
+ opp-supported-hw = <0x0001>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-161500000-950 {
+ opp-microvolt = <950000 950000 1300000>;
+ opp-hz = /bits/ 64 <161500000>;
+ opp-supported-hw = <0x0002>;
+ required-opps = <&core_opp_950>;
+ };
+
+ opp-209000000-1000 {
+ opp-microvolt = <1000000 1000000 1300000>;
+ opp-hz = /bits/ 64 <209000000>;
+ opp-supported-hw = <0x0002>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-218500000-950 {
+ opp-microvolt = <950000 950000 1300000>;
+ opp-hz = /bits/ 64 <218500000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_950>;
+ };
+
+ opp-247000000-1100 {
+ opp-microvolt = <1100000 1100000 1300000>;
+ opp-hz = /bits/ 64 <247000000>;
+ opp-supported-hw = <0x0001>;
+ required-opps = <&core_opp_1100>;
+ };
+
+ opp-247000000-950 {
+ opp-microvolt = <950000 950000 1300000>;
+ opp-hz = /bits/ 64 <247000000>;
+ opp-supported-hw = <0x0008>;
+ required-opps = <&core_opp_950>;
+ };
+
+ opp-256500000-1000 {
+ opp-microvolt = <1000000 1000000 1300000>;
+ opp-hz = /bits/ 64 <256500000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-285000000-1100 {
+ opp-microvolt = <1100000 1100000 1300000>;
+ opp-hz = /bits/ 64 <285000000>;
+ opp-supported-hw = <0x0002>;
+ required-opps = <&core_opp_1100>;
+ };
+
+ opp-285000000-1000 {
+ opp-microvolt = <1000000 1000000 1300000>;
+ opp-hz = /bits/ 64 <285000000>;
+ opp-supported-hw = <0x0008>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-304000000-1200 {
+ opp-microvolt = <1200000 1200000 1300000>;
+ opp-hz = /bits/ 64 <304000000>;
+ opp-supported-hw = <0x0001>;
+ required-opps = <&core_opp_1200>;
+ };
+
+ opp-323000000-1100 {
+ opp-microvolt = <1100000 1100000 1300000>;
+ opp-hz = /bits/ 64 <323000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1100>;
+ };
+
+ opp-333500000-1275 {
+ opp-microvolt = <1275000 1275000 1300000>;
+ opp-hz = /bits/ 64 <333500000>;
+ opp-supported-hw = <0x0001>;
+ required-opps = <&core_opp_1275>;
+ };
+
+ opp-333500000-1200 {
+ opp-microvolt = <1200000 1200000 1300000>;
+ opp-hz = /bits/ 64 <333500000>;
+ opp-supported-hw = <0x0002>;
+ required-opps = <&core_opp_1200>;
+ };
+
+ opp-351500000-1100 {
+ opp-microvolt = <1100000 1100000 1300000>;
+ opp-hz = /bits/ 64 <351500000>;
+ opp-supported-hw = <0x0008>;
+ required-opps = <&core_opp_1100>;
+ };
+
+ opp-361000000-1275 {
+ opp-microvolt = <1275000 1275000 1300000>;
+ opp-hz = /bits/ 64 <361000000>;
+ opp-supported-hw = <0x0002>;
+ required-opps = <&core_opp_1275>;
+ };
+
+ opp-380000000-1200 {
+ opp-microvolt = <1200000 1200000 1300000>;
+ opp-hz = /bits/ 64 <380000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1200>;
+ };
+
+ opp-400000000-1275 {
+ opp-microvolt = <1275000 1275000 1300000>;
+ opp-hz = /bits/ 64 <400000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1275>;
+ };
+
+ opp-400000000-1200 {
+ opp-microvolt = <1200000 1200000 1300000>;
+ opp-hz = /bits/ 64 <400000000>;
+ opp-supported-hw = <0x0008>;
+ required-opps = <&core_opp_1200>;
+ };
+ };
+
+ disp1_dvfs_opp_table: opp-table-disp1 {
+ compatible = "operating-points-v2";
+
+ opp-158000000-950 {
+ opp-microvolt = <950000 950000 1300000>;
+ opp-hz = /bits/ 64 <158000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_950>;
+ };
+
+ opp-190000000-1100 {
+ opp-microvolt = <1100000 1100000 1300000>;
+ opp-hz = /bits/ 64 <190000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1100>;
+ };
+ };
+
+ disp2_dvfs_opp_table: opp-table-disp2 {
+ compatible = "operating-points-v2";
+
+ opp-158000000-950 {
+ opp-microvolt = <950000 950000 1300000>;
+ opp-hz = /bits/ 64 <158000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_950>;
+ };
+
+ opp-190000000-1100 {
+ opp-microvolt = <1100000 1100000 1300000>;
+ opp-hz = /bits/ 64 <190000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1100>;
+ };
+ };
+
+ dsi_dvfs_opp_table: opp-table-dsi {
+ compatible = "operating-points-v2";
+
+ opp-100000000-950 {
+ opp-microvolt = <950000 950000 1300000>;
+ opp-hz = /bits/ 64 <100000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_950>;
+ };
+
+ opp-500000000-1200 {
+ opp-microvolt = <1200000 1200000 1300000>;
+ opp-hz = /bits/ 64 <500000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1200>;
+ };
+ };
+
+ hdmi_dvfs_opp_table: opp-table-hdmi {
+ compatible = "operating-points-v2";
+
+ opp-148500000-1200 {
+ opp-microvolt = <1200000 1200000 1300000>;
+ opp-hz = /bits/ 64 <148500000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1200>;
+ };
+ };
+
+ tvo_dvfs_opp_table: opp-table-tvo {
+ compatible = "operating-points-v2";
+
+ opp-250000000-1200 {
+ opp-microvolt = <1200000 1200000 1300000>;
+ opp-hz = /bits/ 64 <250000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1200>;
+ };
+ };
+
+ sclk_dvfs_opp_table: opp-table-sclk {
+ compatible = "operating-points-v2";
+
+ opp-95000000-950 {
+ opp-microvolt = <950000 950000 1300000>;
+ opp-hz = /bits/ 64 <95000000>;
+ opp-supported-hw = <0x0001>;
+ required-opps = <&core_opp_950>;
+ };
+
+ opp-123500000-950 {
+ opp-microvolt = <950000 950000 1300000>;
+ opp-hz = /bits/ 64 <123500000>;
+ opp-supported-hw = <0x0002>;
+ required-opps = <&core_opp_950>;
+ };
+
+ opp-133000000-1000 {
+ opp-microvolt = <1000000 1000000 1300000>;
+ opp-hz = /bits/ 64 <133000000>;
+ opp-supported-hw = <0x0001>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-152000000-950 {
+ opp-microvolt = <950000 950000 1300000>;
+ opp-hz = /bits/ 64 <152000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_950>;
+ };
+
+ opp-159500000-1000 {
+ opp-microvolt = <1000000 1000000 1300000>;
+ opp-hz = /bits/ 64 <159500000>;
+ opp-supported-hw = <0x0002>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-171000000-950 {
+ opp-microvolt = <950000 950000 1300000>;
+ opp-hz = /bits/ 64 <171000000>;
+ opp-supported-hw = <0x0008>;
+ required-opps = <&core_opp_950>;
+ };
+
+ opp-180500000-1000 {
+ opp-microvolt = <1000000 1000000 1300000>;
+ opp-hz = /bits/ 64 <180500000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-190000000-1100 {
+ opp-microvolt = <1100000 1100000 1300000>;
+ opp-hz = /bits/ 64 <190000000>;
+ opp-supported-hw = <0x0001>;
+ required-opps = <&core_opp_1100>;
+ };
+
+ opp-207000000-1100 {
+ opp-microvolt = <1100000 1100000 1300000>;
+ opp-hz = /bits/ 64 <207000000>;
+ opp-supported-hw = <0x0002>;
+ required-opps = <&core_opp_1100>;
+ };
+
+ opp-218500000-1000 {
+ opp-microvolt = <1000000 1000000 1300000>;
+ opp-hz = /bits/ 64 <218500000>;
+ opp-supported-hw = <0x0008>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-222500000-1200 {
+ opp-microvolt = <1200000 1200000 1300000>;
+ opp-hz = /bits/ 64 <222500000>;
+ opp-supported-hw = <0x0001>;
+ required-opps = <&core_opp_1200>;
+ };
+
+ opp-229500000-1100 {
+ opp-microvolt = <1100000 1100000 1300000>;
+ opp-hz = /bits/ 64 <229500000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1100>;
+ };
+
+ opp-240000000-1225 {
+ opp-microvolt = <1225000 1225000 1300000>;
+ opp-hz = /bits/ 64 <240000000>;
+ opp-supported-hw = <0x0001>;
+ required-opps = <&core_opp_1225>;
+ };
+
+ opp-240000000-1200 {
+ opp-microvolt = <1200000 1200000 1300000>;
+ opp-hz = /bits/ 64 <240000000>;
+ opp-supported-hw = <0x0002>;
+ required-opps = <&core_opp_1200>;
+ };
+
+ opp-247000000-1275 {
+ opp-microvolt = <1275000 1275000 1300000>;
+ opp-hz = /bits/ 64 <247000000>;
+ opp-supported-hw = <0x0001>;
+ required-opps = <&core_opp_1275>;
+ };
+
+ opp-256500000-1100 {
+ opp-microvolt = <1100000 1100000 1300000>;
+ opp-hz = /bits/ 64 <256500000>;
+ opp-supported-hw = <0x0008>;
+ required-opps = <&core_opp_1100>;
+ };
+
+ opp-260000000-1200 {
+ opp-microvolt = <1200000 1200000 1300000>;
+ opp-hz = /bits/ 64 <260000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1200>;
+ };
+
+ opp-262000000-1300 {
+ opp-microvolt = <1300000 1300000 1300000>;
+ opp-hz = /bits/ 64 <262000000>;
+ opp-supported-hw = <0x0001>;
+ required-opps = <&core_opp_1300>;
+ };
+
+ opp-264000000-1275 {
+ opp-microvolt = <1275000 1275000 1300000>;
+ opp-hz = /bits/ 64 <264000000>;
+ opp-supported-hw = <0x0002>;
+ required-opps = <&core_opp_1275>;
+ };
+
+ opp-277500000-1300 {
+ opp-microvolt = <1300000 1300000 1300000>;
+ opp-hz = /bits/ 64 <277500000>;
+ opp-supported-hw = <0x0002>;
+ required-opps = <&core_opp_1300>;
+ };
+
+ opp-285000000-1275 {
+ opp-microvolt = <1275000 1275000 1300000>;
+ opp-hz = /bits/ 64 <285000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1275>;
+ };
+
+ opp-292500000-1200 {
+ opp-microvolt = <1200000 1200000 1300000>;
+ opp-hz = /bits/ 64 <292500000>;
+ opp-supported-hw = <0x0008>;
+ required-opps = <&core_opp_1200>;
+ };
+
+ opp-300000000-1300 {
+ opp-microvolt = <1300000 1300000 1300000>;
+ opp-hz = /bits/ 64 <300000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1300>;
+ };
+
+ opp-300000000-1275 {
+ opp-microvolt = <1275000 1275000 1300000>;
+ opp-hz = /bits/ 64 <300000000>;
+ opp-supported-hw = <0x0008>;
+ required-opps = <&core_opp_1275>;
+ };
+ };
+
+ vde_dvfs_opp_table: opp-table-vde {
+ compatible = "operating-points-v2";
+
+ opp-95000000-950 {
+ opp-microvolt = <950000 950000 1300000>;
+ opp-hz = /bits/ 64 <95000000>;
+ opp-supported-hw = <0x0001>;
+ required-opps = <&core_opp_950>;
+ };
+
+ opp-123500000-1000 {
+ opp-microvolt = <1000000 1000000 1300000>;
+ opp-hz = /bits/ 64 <123500000>;
+ opp-supported-hw = <0x0001>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-123500000-950 {
+ opp-microvolt = <950000 950000 1300000>;
+ opp-hz = /bits/ 64 <123500000>;
+ opp-supported-hw = <0x0002>;
+ required-opps = <&core_opp_950>;
+ };
+
+ opp-152000000-1000 {
+ opp-microvolt = <1000000 1000000 1300000>;
+ opp-hz = /bits/ 64 <152000000>;
+ opp-supported-hw = <0x0002>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-152000000-950 {
+ opp-microvolt = <950000 950000 1300000>;
+ opp-hz = /bits/ 64 <152000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_950>;
+ };
+
+ opp-171000000-950 {
+ opp-microvolt = <950000 950000 1300000>;
+ opp-hz = /bits/ 64 <171000000>;
+ opp-supported-hw = <0x0008>;
+ required-opps = <&core_opp_950>;
+ };
+
+ opp-209000000-1100 {
+ opp-microvolt = <1100000 1100000 1300000>;
+ opp-hz = /bits/ 64 <209000000>;
+ opp-supported-hw = <0x0001>;
+ required-opps = <&core_opp_1100>;
+ };
+
+ opp-209000000-1000 {
+ opp-microvolt = <1000000 1000000 1300000>;
+ opp-hz = /bits/ 64 <209000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-218500000-1000 {
+ opp-microvolt = <1000000 1000000 1300000>;
+ opp-hz = /bits/ 64 <218500000>;
+ opp-supported-hw = <0x0008>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-237500000-1100 {
+ opp-microvolt = <1100000 1100000 1300000>;
+ opp-hz = /bits/ 64 <237500000>;
+ opp-supported-hw = <0x0002>;
+ required-opps = <&core_opp_1100>;
+ };
+
+ opp-275500000-1200 {
+ opp-microvolt = <1200000 1200000 1300000>;
+ opp-hz = /bits/ 64 <275500000>;
+ opp-supported-hw = <0x0001>;
+ required-opps = <&core_opp_1200>;
+ };
+
+ opp-285000000-1100 {
+ opp-microvolt = <1100000 1100000 1300000>;
+ opp-hz = /bits/ 64 <285000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1100>;
+ };
+
+ opp-300000000-1275 {
+ opp-microvolt = <1275000 1275000 1300000>;
+ opp-hz = /bits/ 64 <300000000>;
+ opp-supported-hw = <0x0001>;
+ required-opps = <&core_opp_1275>;
+ };
+
+ opp-300000000-1200 {
+ opp-microvolt = <1200000 1200000 1300000>;
+ opp-hz = /bits/ 64 <300000000>;
+ opp-supported-hw = <0x0006>;
+ required-opps = <&core_opp_1200>;
+ };
+
+ opp-300000000-1100 {
+ opp-microvolt = <1100000 1100000 1300000>;
+ opp-hz = /bits/ 64 <300000000>;
+ opp-supported-hw = <0x0008>;
+ required-opps = <&core_opp_1100>;
+ };
+ };
+
+ ndflash_dvfs_opp_table: opp-table-ndflash {
+ compatible = "operating-points-v2";
+
+ opp-130000000-950 {
+ opp-microvolt = <950000 950000 1300000>;
+ opp-hz = /bits/ 64 <130000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_950>;
+ };
+
+ opp-150000000-1000 {
+ opp-microvolt = <1000000 1000000 1300000>;
+ opp-hz = /bits/ 64 <150000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-158000000-1100 {
+ opp-microvolt = <1100000 1100000 1300000>;
+ opp-hz = /bits/ 64 <158000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1100>;
+ };
+
+ opp-164000000-1200 {
+ opp-microvolt = <1200000 1200000 1300000>;
+ opp-hz = /bits/ 64 <164000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1200>;
+ };
+ };
+
+ nor_dvfs_opp_table: opp-table-nor {
+ compatible = "operating-points-v2";
+
+ opp-92000000-1000 {
+ opp-microvolt = <1000000 1000000 1300000>;
+ opp-hz = /bits/ 64 <92000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1000>;
+ };
+ };
+
+ sdmmc1_dvfs_opp_table: opp-table-sdmmc1 {
+ compatible = "operating-points-v2";
+
+ opp-44000000-950 {
+ opp-microvolt = <950000 950000 1300000>;
+ opp-hz = /bits/ 64 <44000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_950>;
+ };
+
+ opp-52000000-1000 {
+ opp-microvolt = <1000000 1000000 1300000>;
+ opp-hz = /bits/ 64 <52000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1000>;
+ };
+ };
+
+ sdmmc2_dvfs_opp_table: opp-table-sdmmc2 {
+ compatible = "operating-points-v2";
+
+ opp-44000000-950 {
+ opp-microvolt = <950000 950000 1300000>;
+ opp-hz = /bits/ 64 <44000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_950>;
+ };
+
+ opp-52000000-1000 {
+ opp-microvolt = <1000000 1000000 1300000>;
+ opp-hz = /bits/ 64 <52000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1000>;
+ };
+ };
+
+ sdmmc3_dvfs_opp_table: opp-table-sdmmc3 {
+ compatible = "operating-points-v2";
+
+ opp-44000000-950 {
+ opp-microvolt = <950000 950000 1300000>;
+ opp-hz = /bits/ 64 <44000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_950>;
+ };
+
+ opp-52000000-1000 {
+ opp-microvolt = <1000000 1000000 1300000>;
+ opp-hz = /bits/ 64 <52000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1000>;
+ };
+ };
+
+ sdmmc4_dvfs_opp_table: opp-table-sdmmc4 {
+ compatible = "operating-points-v2";
+
+ opp-44000000-950 {
+ opp-microvolt = <950000 950000 1300000>;
+ opp-hz = /bits/ 64 <44000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_950>;
+ };
+
+ opp-52000000-1000 {
+ opp-microvolt = <1000000 1000000 1300000>;
+ opp-hz = /bits/ 64 <52000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1000>;
+ };
+ };
+
+ pcie_dvfs_opp_table: opp-table-pcie {
+ compatible = "operating-points-v2";
+
+ opp-250000000-1200 {
+ opp-microvolt = <1200000 1200000 1300000>;
+ opp-hz = /bits/ 64 <250000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1200>;
+ };
+ };
+
+ usbd_dvfs_opp_table: opp-table-usbd {
+ compatible = "operating-points-v2";
+
+ opp-480000000-1100 {
+ opp-microvolt = <1100000 1100000 1300000>;
+ opp-hz = /bits/ 64 <480000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1100>;
+ };
+ };
+
+ usb2_dvfs_opp_table: opp-table-usb2 {
+ compatible = "operating-points-v2";
+
+ opp-480000000-1100 {
+ opp-microvolt = <1100000 1100000 1300000>;
+ opp-hz = /bits/ 64 <480000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1100>;
+ };
+ };
+
+ usb3_dvfs_opp_table: opp-table-usb3 {
+ compatible = "operating-points-v2";
+
+ opp-480000000-1100 {
+ opp-microvolt = <1100000 1100000 1300000>;
+ opp-hz = /bits/ 64 <480000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1100>;
};
};
};
diff --git a/arch/arm/boot/dts/tegra20-plutux.dts b/arch/arm/boot/dts/tegra20-plutux.dts
index 5811b70..71a8236 100644
--- a/arch/arm/boot/dts/tegra20-plutux.dts
+++ b/arch/arm/boot/dts/tegra20-plutux.dts
@@ -60,7 +60,7 @@ sound {
clock-names = "pll_a", "pll_a_out0", "mclk";
};
- vcc_24v_reg: regulator@100 {
+ vcc_24v_reg: regulator-24v0 {
compatible = "regulator-fixed";
regulator-name = "vcc_24v";
regulator-min-microvolt = <24000000>;
@@ -68,7 +68,7 @@ vcc_24v_reg: regulator@100 {
regulator-always-on;
};
- vdd_5v0_reg: regulator@101 {
+ vdd_5v0_reg: regulator-5v0 {
compatible = "regulator-fixed";
regulator-name = "vdd_5v0";
vin-supply = <&vcc_24v_reg>;
@@ -77,7 +77,7 @@ vdd_5v0_reg: regulator@101 {
regulator-always-on;
};
- vdd_3v3_reg: regulator@102 {
+ vdd_3v3_reg: regulator-3v3 {
compatible = "regulator-fixed";
regulator-name = "vdd_3v3";
vin-supply = <&vcc_24v_reg>;
@@ -86,7 +86,7 @@ vdd_3v3_reg: regulator@102 {
regulator-always-on;
};
- vdd_1v8_reg: regulator@103 {
+ vdd_1v8_reg: regulator-1v8 {
compatible = "regulator-fixed";
regulator-name = "vdd_1v8";
vin-supply = <&vdd_3v3_reg>;
diff --git a/arch/arm/boot/dts/tegra20-seaboard.dts b/arch/arm/boot/dts/tegra20-seaboard.dts
index 92d494b..c4a6a6a 100644
--- a/arch/arm/boot/dts/tegra20-seaboard.dts
+++ b/arch/arm/boot/dts/tegra20-seaboard.dts
@@ -358,7 +358,7 @@ isl29018@44 {
};
gyrometer@68 {
- compatible = "invn,mpu3050";
+ compatible = "invensense,mpu3050";
reg = <0x68>;
interrupt-parent = <&gpio>;
interrupts = <TEGRA_GPIO(Z, 4) IRQ_TYPE_LEVEL_HIGH>;
@@ -444,7 +444,7 @@ sys_reg: sys {
regulator-always-on;
};
- sm0 {
+ vdd_core: sm0 {
regulator-name = "vdd_sm0,vdd_core";
regulator-min-microvolt = <1300000>;
regulator-max-microvolt = <1300000>;
@@ -689,6 +689,7 @@ pmc@7000e400 {
nvidia,core-pwr-good-time = <3845 3845>;
nvidia,core-pwr-off-time = <3875>;
nvidia,sys-clock-req-active-high;
+ core-supply = <&vdd_core>;
};
memory-controller@7000f400 {
@@ -742,8 +743,6 @@ usb-phy@c5000000 {
usb@c5004000 {
status = "okay";
- nvidia,phy-reset-gpio = <&gpio TEGRA_GPIO(V, 1)
- GPIO_ACTIVE_LOW>;
};
usb-phy@c5004000 {
@@ -792,7 +791,7 @@ backlight: backlight {
default-brightness-level = <6>;
};
- clk32k_in: clock@0 {
+ clk32k_in: clock-32k {
compatible = "fixed-clock";
clock-frequency = <32768>;
#clock-cells = <0>;
@@ -828,7 +827,7 @@ panel: panel {
ddc-i2c-bus = <&lvds_ddc>;
};
- vdd_5v0_reg: regulator@0 {
+ vdd_5v0_reg: regulator-5v0 {
compatible = "regulator-fixed";
regulator-name = "vdd_5v0";
regulator-min-microvolt = <5000000>;
@@ -836,7 +835,7 @@ vdd_5v0_reg: regulator@0 {
regulator-always-on;
};
- regulator@1 {
+ regulator-1v5 {
compatible = "regulator-fixed";
regulator-name = "vdd_1v5";
regulator-min-microvolt = <1500000>;
@@ -844,7 +843,7 @@ regulator@1 {
gpio = <&pmic 0 GPIO_ACTIVE_HIGH>;
};
- regulator@2 {
+ regulator-1v2 {
compatible = "regulator-fixed";
regulator-name = "vdd_1v2";
regulator-min-microvolt = <1200000>;
@@ -853,7 +852,7 @@ regulator@2 {
enable-active-high;
};
- vbus_reg: regulator@3 {
+ vbus_reg: regulator-vbus {
compatible = "regulator-fixed";
regulator-name = "vdd_vbus_wup1";
regulator-min-microvolt = <5000000>;
@@ -864,7 +863,7 @@ vbus_reg: regulator@3 {
regulator-boot-on;
};
- vdd_pnl_reg: regulator@4 {
+ vdd_pnl_reg: regulator-pnl {
compatible = "regulator-fixed";
regulator-name = "vdd_pnl";
regulator-min-microvolt = <2800000>;
@@ -873,7 +872,7 @@ vdd_pnl_reg: regulator@4 {
enable-active-high;
};
- vdd_bl_reg: regulator@5 {
+ vdd_bl_reg: regulator-bl {
compatible = "regulator-fixed";
regulator-name = "vdd_bl";
regulator-min-microvolt = <2800000>;
@@ -882,7 +881,7 @@ vdd_bl_reg: regulator@5 {
enable-active-high;
};
- vdd_hdmi: regulator@6 {
+ vdd_hdmi: regulator-hdmi {
compatible = "regulator-fixed";
regulator-name = "VDDIO_HDMI";
regulator-min-microvolt = <5000000>;
diff --git a/arch/arm/boot/dts/tegra20-tamonten.dtsi b/arch/arm/boot/dts/tegra20-tamonten.dtsi
index dd4d5066..de39c54 100644
--- a/arch/arm/boot/dts/tegra20-tamonten.dtsi
+++ b/arch/arm/boot/dts/tegra20-tamonten.dtsi
@@ -357,7 +357,7 @@ sys_reg: sys {
regulator-always-on;
};
- sm0 {
+ vdd_core: sm0 {
regulator-name = "vdd_sys_sm0,vdd_core";
regulator-min-microvolt = <1200000>;
regulator-max-microvolt = <1200000>;
@@ -477,6 +477,7 @@ pmc@7000e400 {
nvidia,core-pwr-good-time = <3845 3845>;
nvidia,core-pwr-off-time = <3875>;
nvidia,sys-clock-req-active-high;
+ core-supply = <&vdd_core>;
};
pcie@80003000 {
@@ -502,13 +503,13 @@ mmc@c8000600 {
status = "okay";
};
- clk32k_in: clock@0 {
+ clk32k_in: clock-32k {
compatible = "fixed-clock";
clock-frequency = <32768>;
#clock-cells = <0>;
};
- pci_vdd_reg: regulator@1 {
+ pci_vdd_reg: regulator-1v05 {
compatible = "regulator-fixed";
regulator-name = "vdd_1v05";
regulator-min-microvolt = <1050000>;
diff --git a/arch/arm/boot/dts/tegra20-tec.dts b/arch/arm/boot/dts/tegra20-tec.dts
index 10ff09d..4f41c74 100644
--- a/arch/arm/boot/dts/tegra20-tec.dts
+++ b/arch/arm/boot/dts/tegra20-tec.dts
@@ -69,7 +69,7 @@ sound {
clock-names = "pll_a", "pll_a_out0", "mclk";
};
- vcc_24v_reg: regulator@100 {
+ vcc_24v_reg: regulator-24v {
compatible = "regulator-fixed";
regulator-name = "vcc_24v";
regulator-min-microvolt = <24000000>;
@@ -77,7 +77,7 @@ vcc_24v_reg: regulator@100 {
regulator-always-on;
};
- vdd_5v0_reg: regulator@101 {
+ vdd_5v0_reg: regulator-5v0 {
compatible = "regulator-fixed";
regulator-name = "vdd_5v0";
vin-supply = <&vcc_24v_reg>;
@@ -86,7 +86,7 @@ vdd_5v0_reg: regulator@101 {
regulator-always-on;
};
- vdd_3v3_reg: regulator@102 {
+ vdd_3v3_reg: regulator-3v3 {
compatible = "regulator-fixed";
regulator-name = "vdd_3v3";
vin-supply = <&vcc_24v_reg>;
@@ -95,7 +95,7 @@ vdd_3v3_reg: regulator@102 {
regulator-always-on;
};
- vdd_1v8_reg: regulator@103 {
+ vdd_1v8_reg: regulator-1v8 {
compatible = "regulator-fixed";
regulator-name = "vdd_1v8";
vin-supply = <&vdd_3v3_reg>;
diff --git a/arch/arm/boot/dts/tegra20-trimslice.dts b/arch/arm/boot/dts/tegra20-trimslice.dts
index 4bc87bc..9d0c867 100644
--- a/arch/arm/boot/dts/tegra20-trimslice.dts
+++ b/arch/arm/boot/dts/tegra20-trimslice.dts
@@ -287,7 +287,8 @@ dvi_ddc: i2c@7000c000 {
spi@7000c380 {
status = "okay";
spi-max-frequency = <48000000>;
- spi-flash@0 {
+
+ flash@0 {
compatible = "winbond,w25q80bl", "jedec,spi-nor";
reg = <0>;
spi-max-frequency = <48000000>;
@@ -321,6 +322,7 @@ pmc@7000e400 {
nvidia,core-pwr-good-time = <3845 3845>;
nvidia,core-pwr-off-time = <3875>;
nvidia,sys-clock-req-active-high;
+ core-supply = <&vdd_core>;
};
pcie@80003000 {
@@ -348,8 +350,6 @@ usb-phy@c5000000 {
usb@c5004000 {
status = "okay";
- nvidia,phy-reset-gpio = <&gpio TEGRA_GPIO(V, 0)
- GPIO_ACTIVE_LOW>;
};
usb-phy@c5004000 {
@@ -379,7 +379,7 @@ mmc@c8000600 {
bus-width = <4>;
};
- clk32k_in: clock@0 {
+ clk32k_in: clock-32k {
compatible = "fixed-clock";
clock-frequency = <32768>;
#clock-cells = <0>;
@@ -401,7 +401,7 @@ poweroff {
gpios = <&gpio TEGRA_GPIO(X, 7) GPIO_ACTIVE_LOW>;
};
- hdmi_vdd_reg: regulator@0 {
+ hdmi_vdd_reg: regulator-hdmi {
compatible = "regulator-fixed";
regulator-name = "avdd_hdmi";
regulator-min-microvolt = <3300000>;
@@ -409,7 +409,7 @@ hdmi_vdd_reg: regulator@0 {
regulator-always-on;
};
- hdmi_pll_reg: regulator@1 {
+ hdmi_pll_reg: regulator-hdmipll {
compatible = "regulator-fixed";
regulator-name = "avdd_hdmi_pll";
regulator-min-microvolt = <1800000>;
@@ -417,7 +417,7 @@ hdmi_pll_reg: regulator@1 {
regulator-always-on;
};
- vbus_reg: regulator@2 {
+ vbus_reg: regulator-vbus {
compatible = "regulator-fixed";
regulator-name = "usb1_vbus";
regulator-min-microvolt = <5000000>;
@@ -428,7 +428,7 @@ vbus_reg: regulator@2 {
regulator-boot-on;
};
- pci_clk_reg: regulator@3 {
+ pci_clk_reg: regulator-pciclk {
compatible = "regulator-fixed";
regulator-name = "pci_clk";
regulator-min-microvolt = <3300000>;
@@ -436,7 +436,7 @@ pci_clk_reg: regulator@3 {
regulator-always-on;
};
- pci_vdd_reg: regulator@4 {
+ pci_vdd_reg: regulator-pcivdd {
compatible = "regulator-fixed";
regulator-name = "pci_vdd";
regulator-min-microvolt = <1050000>;
@@ -444,6 +444,14 @@ pci_vdd_reg: regulator@4 {
regulator-always-on;
};
+ vdd_core: regulator-core {
+ compatible = "regulator-fixed";
+ regulator-name = "vdd_core";
+ regulator-min-microvolt = <1300000>;
+ regulator-max-microvolt = <1300000>;
+ regulator-always-on;
+ };
+
sound {
compatible = "nvidia,tegra-audio-trimslice";
nvidia,i2s-controller = <&tegra_i2s1>;
diff --git a/arch/arm/boot/dts/tegra20-ventana.dts b/arch/arm/boot/dts/tegra20-ventana.dts
index 5a2578b..b0a0097 100644
--- a/arch/arm/boot/dts/tegra20-ventana.dts
+++ b/arch/arm/boot/dts/tegra20-ventana.dts
@@ -544,6 +544,7 @@ pmc@7000e400 {
nvidia,core-pwr-good-time = <3845 3845>;
nvidia,core-pwr-off-time = <458>;
nvidia,sys-clock-req-active-high;
+ core-supply = <&vdd_core>;
};
usb@c5000000 {
@@ -556,8 +557,6 @@ usb-phy@c5000000 {
usb@c5004000 {
status = "okay";
- nvidia,phy-reset-gpio = <&gpio TEGRA_GPIO(V, 1)
- GPIO_ACTIVE_LOW>;
};
usb-phy@c5004000 {
@@ -606,7 +605,7 @@ backlight: backlight {
default-brightness-level = <6>;
};
- clk32k_in: clock@0 {
+ clk32k_in: clock-32k {
compatible = "fixed-clock";
clock-frequency = <32768>;
#clock-cells = <0>;
@@ -647,7 +646,7 @@ panel: panel {
ddc-i2c-bus = <&lvds_ddc>;
};
- vdd_5v0_reg: regulator@0 {
+ vdd_5v0_reg: regulator-5v0 {
compatible = "regulator-fixed";
regulator-name = "vdd_5v0";
regulator-min-microvolt = <5000000>;
@@ -655,7 +654,7 @@ vdd_5v0_reg: regulator@0 {
regulator-always-on;
};
- regulator@1 {
+ regulator-1v5 {
compatible = "regulator-fixed";
regulator-name = "vdd_1v5";
regulator-min-microvolt = <1500000>;
@@ -663,7 +662,7 @@ regulator@1 {
gpio = <&pmic 0 GPIO_ACTIVE_HIGH>;
};
- regulator@2 {
+ regulator-1v2 {
compatible = "regulator-fixed";
regulator-name = "vdd_1v2";
regulator-min-microvolt = <1200000>;
@@ -672,7 +671,7 @@ regulator@2 {
enable-active-high;
};
- vdd_pnl_reg: regulator@3 {
+ vdd_pnl_reg: regulator-pnl {
compatible = "regulator-fixed";
regulator-name = "vdd_pnl";
regulator-min-microvolt = <2800000>;
@@ -681,7 +680,7 @@ vdd_pnl_reg: regulator@3 {
enable-active-high;
};
- vdd_bl_reg: regulator@4 {
+ vdd_bl_reg: regulator-bl {
compatible = "regulator-fixed";
regulator-name = "vdd_bl";
regulator-min-microvolt = <2800000>;
diff --git a/arch/arm/boot/dts/tegra20.dtsi b/arch/arm/boot/dts/tegra20.dtsi
index 9508248..62bf0b3 100644
--- a/arch/arm/boot/dts/tegra20.dtsi
+++ b/arch/arm/boot/dts/tegra20.dtsi
@@ -40,8 +40,10 @@ host1x@50000000 {
interrupt-names = "syncpt", "host1x";
clocks = <&tegra_car TEGRA20_CLK_HOST1X>;
clock-names = "host1x";
- resets = <&tegra_car 28>;
- reset-names = "host1x";
+ resets = <&tegra_car 28>, <&mc TEGRA20_MC_RESET_HC>;
+ reset-names = "host1x", "mc";
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&host1x_dvfs_opp_table>;
#address-cells = <1>;
#size-cells = <1>;
@@ -55,6 +57,9 @@ mpe@54040000 {
clocks = <&tegra_car TEGRA20_CLK_MPE>;
resets = <&tegra_car 60>;
reset-names = "mpe";
+ power-domains = <&pd_mpe>;
+ operating-points-v2 = <&mpe_dvfs_opp_table>;
+ status = "disabled";
};
vi@54080000 {
@@ -64,6 +69,9 @@ vi@54080000 {
clocks = <&tegra_car TEGRA20_CLK_VI>;
resets = <&tegra_car 20>;
reset-names = "vi";
+ power-domains = <&pd_venc>;
+ operating-points-v2 = <&vi_dvfs_opp_table>;
+ status = "disabled";
};
epp@540c0000 {
@@ -73,6 +81,9 @@ epp@540c0000 {
clocks = <&tegra_car TEGRA20_CLK_EPP>;
resets = <&tegra_car 19>;
reset-names = "epp";
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&epp_dvfs_opp_table>;
+ status = "disabled";
};
isp@54100000 {
@@ -82,6 +93,8 @@ isp@54100000 {
clocks = <&tegra_car TEGRA20_CLK_ISP>;
resets = <&tegra_car 23>;
reset-names = "isp";
+ power-domains = <&pd_venc>;
+ status = "disabled";
};
gr2d@54140000 {
@@ -89,16 +102,20 @@ gr2d@54140000 {
reg = <0x54140000 0x00040000>;
interrupts = <GIC_SPI 72 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&tegra_car TEGRA20_CLK_GR2D>;
- resets = <&tegra_car 21>;
- reset-names = "2d";
+ resets = <&tegra_car 21>, <&mc TEGRA20_MC_RESET_2D>;
+ reset-names = "2d", "mc";
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&gr2d_dvfs_opp_table>;
};
gr3d@54180000 {
compatible = "nvidia,tegra20-gr3d";
reg = <0x54180000 0x00040000>;
clocks = <&tegra_car TEGRA20_CLK_GR3D>;
- resets = <&tegra_car 24>;
- reset-names = "3d";
+ resets = <&tegra_car 24>, <&mc TEGRA20_MC_RESET_3D>;
+ reset-names = "3d", "mc";
+ power-domains = <&pd_3d>;
+ operating-points-v2 = <&gr3d_dvfs_opp_table>;
};
dc@54200000 {
@@ -110,6 +127,8 @@ dc@54200000 {
clock-names = "dc", "parent";
resets = <&tegra_car 27>;
reset-names = "dc";
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&disp1_dvfs_opp_table>;
nvidia,head = <0>;
@@ -138,6 +157,8 @@ dc@54240000 {
clock-names = "dc", "parent";
resets = <&tegra_car 26>;
reset-names = "dc";
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&disp2_dvfs_opp_table>;
nvidia,head = <1>;
@@ -157,7 +178,7 @@ rgb {
};
};
- hdmi@54280000 {
+ tegra_hdmi: hdmi@54280000 {
compatible = "nvidia,tegra20-hdmi";
reg = <0x54280000 0x00040000>;
interrupts = <GIC_SPI 75 IRQ_TYPE_LEVEL_HIGH>;
@@ -166,6 +187,9 @@ hdmi@54280000 {
clock-names = "hdmi", "parent";
resets = <&tegra_car 51>;
reset-names = "hdmi";
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&hdmi_dvfs_opp_table>;
+ #sound-dai-cells = <0>;
status = "disabled";
};
@@ -174,6 +198,8 @@ tvo@542c0000 {
reg = <0x542c0000 0x00040000>;
interrupts = <GIC_SPI 76 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&tegra_car TEGRA20_CLK_TVO>;
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&tvo_dvfs_opp_table>;
status = "disabled";
};
@@ -185,6 +211,8 @@ dsi@54300000 {
clock-names = "dsi", "parent";
resets = <&tegra_car 48>;
reset-names = "dsi";
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&dsi_dvfs_opp_table>;
status = "disabled";
};
};
@@ -242,6 +270,13 @@ tegra_car: clock@60006000 {
reg = <0x60006000 0x1000>;
#clock-cells = <1>;
#reset-cells = <1>;
+
+ sclk {
+ compatible = "nvidia,tegra20-sclk";
+ clocks = <&tegra_car TEGRA20_CLK_SCLK>;
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&sclk_dvfs_opp_table>;
+ };
};
flow-controller@60007000 {
@@ -293,9 +328,7 @@ gpio: gpio@6000d000 {
gpio-controller;
#interrupt-cells = <2>;
interrupt-controller;
- /*
gpio-ranges = <&pinmux 0 0 224>;
- */
};
vde@6001a000 {
@@ -319,6 +352,8 @@ vde@6001a000 {
clocks = <&tegra_car TEGRA20_CLK_VDE>;
reset-names = "vde", "mc";
resets = <&tegra_car 61>, <&mc TEGRA20_MC_RESET_VDE>;
+ power-domains = <&pd_vde>;
+ operating-points-v2 = <&vde_dvfs_opp_table>;
};
apbmisc@70000800 {
@@ -352,6 +387,23 @@ tegra_ac97: ac97@70002000 {
status = "disabled";
};
+ tegra_spdif: spdif@70002400 {
+ compatible = "nvidia,tegra20-spdif";
+ reg = <0x70002400 0x200>;
+ interrupts = <GIC_SPI 45 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&tegra_car TEGRA20_CLK_SPDIF_OUT>,
+ <&tegra_car TEGRA20_CLK_SPDIF_IN>;
+ clock-names = "out", "in";
+ resets = <&tegra_car 10>;
+ dmas = <&apbdma 3>, <&apbdma 3>;
+ dma-names = "rx", "tx";
+ #sound-dai-cells = <0>;
+ status = "disabled";
+
+ assigned-clocks = <&tegra_car TEGRA20_CLK_SPDIF_OUT>;
+ assigned-clock-parents = <&tegra_car TEGRA20_CLK_PLL_A_OUT0>;
+ };
+
tegra_i2s1: i2s@70002800 {
compatible = "nvidia,tegra20-i2s";
reg = <0x70002800 0x200>;
@@ -460,6 +512,8 @@ nand-controller@70008000 {
reset-names = "nand";
assigned-clocks = <&tegra_car TEGRA20_CLK_NDFLASH>;
assigned-clock-rates = <150000000>;
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&ndflash_dvfs_opp_table>;
status = "disabled";
};
@@ -473,6 +527,8 @@ gmi@70009000 {
clock-names = "gmi";
resets = <&tegra_car 42>;
reset-names = "gmi";
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&nor_dvfs_opp_table>;
status = "disabled";
};
@@ -523,7 +579,7 @@ spi@7000c380 {
status = "disabled";
};
- i2c@7000c400 {
+ i2c2: i2c@7000c400 {
compatible = "nvidia,tegra20-i2c";
reg = <0x7000c400 0x100>;
interrupts = <GIC_SPI 84 IRQ_TYPE_LEVEL_HIGH>;
@@ -643,6 +699,52 @@ tegra_pmc: pmc@7000e400 {
clocks = <&tegra_car TEGRA20_CLK_PCLK>, <&clk32k_in>;
clock-names = "pclk", "clk32k_in";
#clock-cells = <1>;
+
+ pd_core: core-domain {
+ #power-domain-cells = <0>;
+ operating-points-v2 = <&core_opp_table>;
+ };
+
+ powergates {
+ pd_3d: td {
+ clocks = <&tegra_car TEGRA20_CLK_GR3D>;
+ resets = <&mc TEGRA20_MC_RESET_3D>,
+ <&tegra_car TEGRA20_CLK_GR3D>;
+ power-domains = <&pd_core>;
+ #power-domain-cells = <0>;
+ };
+
+ pd_venc: venc {
+ clocks = <&tegra_car TEGRA20_CLK_ISP>,
+ <&tegra_car TEGRA20_CLK_VI>,
+ <&tegra_car TEGRA20_CLK_CSI>;
+ resets = <&mc TEGRA20_MC_RESET_ISP>,
+ <&mc TEGRA20_MC_RESET_VI>,
+ <&tegra_car TEGRA20_CLK_ISP>,
+ <&tegra_car 20 /* VI */>,
+ <&tegra_car TEGRA20_CLK_CSI>;
+ power-domains = <&pd_core>;
+ #power-domain-cells = <0>;
+ };
+
+ pd_vde: vdec {
+ clocks = <&tegra_car TEGRA20_CLK_VDE>;
+ resets = <&mc TEGRA20_MC_RESET_VDE>,
+ <&tegra_car TEGRA20_CLK_VDE>;
+ power-domains = <&pd_core>;
+ #power-domain-cells = <0>;
+ };
+
+ pd_mpe: mpe {
+ clocks = <&tegra_car TEGRA20_CLK_MPE>;
+ resets = <&mc TEGRA20_MC_RESET_MPEA>,
+ <&mc TEGRA20_MC_RESET_MPEB>,
+ <&mc TEGRA20_MC_RESET_MPEC>,
+ <&tegra_car TEGRA20_CLK_MPE>;
+ power-domains = <&pd_core>;
+ #power-domain-cells = <0>;
+ };
+ };
};
mc: memory-controller@7000f000 {
@@ -662,12 +764,13 @@ emc: memory-controller@7000f400 {
reg = <0x7000f400 0x400>;
interrupts = <GIC_SPI 78 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&tegra_car TEGRA20_CLK_EMC>;
+ power-domains = <&pd_core>;
#address-cells = <1>;
#size-cells = <0>;
#interconnect-cells = <0>;
- operating-points-v2 = <&emc_icc_dvfs_opp_table>;
nvidia,memory-controller = <&mc>;
+ operating-points-v2 = <&emc_icc_dvfs_opp_table>;
};
fuse@7000f800 {
@@ -712,6 +815,9 @@ pcie@80003000 {
<&tegra_car 72>,
<&tegra_car 74>;
reset-names = "pex", "afi", "pcie_x";
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&pcie_dvfs_opp_table>;
+
status = "disabled";
pci@1,0 {
@@ -753,6 +859,8 @@ usb@c5000000 {
reset-names = "usb";
nvidia,needs-double-reset;
nvidia,phy = <&phy1>;
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&usbd_dvfs_opp_table>;
status = "disabled";
};
@@ -792,6 +900,8 @@ usb@c5004000 {
resets = <&tegra_car 58>;
reset-names = "usb";
nvidia,phy = <&phy2>;
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&usb2_dvfs_opp_table>;
status = "disabled";
};
@@ -820,6 +930,8 @@ usb@c5008000 {
resets = <&tegra_car 59>;
reset-names = "usb";
nvidia,phy = <&phy3>;
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&usb3_dvfs_opp_table>;
status = "disabled";
};
@@ -856,6 +968,8 @@ mmc@c8000000 {
clock-names = "sdhci";
resets = <&tegra_car 14>;
reset-names = "sdhci";
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&sdmmc1_dvfs_opp_table>;
status = "disabled";
};
@@ -867,6 +981,8 @@ mmc@c8000200 {
clock-names = "sdhci";
resets = <&tegra_car 9>;
reset-names = "sdhci";
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&sdmmc2_dvfs_opp_table>;
status = "disabled";
};
@@ -878,6 +994,8 @@ mmc@c8000400 {
clock-names = "sdhci";
resets = <&tegra_car 69>;
reset-names = "sdhci";
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&sdmmc3_dvfs_opp_table>;
status = "disabled";
};
@@ -889,6 +1007,8 @@ mmc@c8000600 {
clock-names = "sdhci";
resets = <&tegra_car 15>;
reset-names = "sdhci";
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&sdmmc4_dvfs_opp_table>;
status = "disabled";
};
@@ -918,4 +1038,24 @@ pmu {
interrupt-affinity = <&{/cpus/cpu@0}>,
<&{/cpus/cpu@1}>;
};
+
+ sound-hdmi {
+ compatible = "simple-audio-card";
+ simple-audio-card,name = "NVIDIA Tegra20 HDMI";
+
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ simple-audio-card,dai-link@0 {
+ reg = <0>;
+
+ cpu {
+ sound-dai = <&tegra_spdif>;
+ };
+
+ codec {
+ sound-dai = <&tegra_hdmi>;
+ };
+ };
+ };
};
diff --git a/arch/arm/boot/dts/tegra30-apalis-eval.dts b/arch/arm/boot/dts/tegra30-apalis-eval.dts
index 9f653ef..93b83b3 100644
--- a/arch/arm/boot/dts/tegra30-apalis-eval.dts
+++ b/arch/arm/boot/dts/tegra30-apalis-eval.dts
@@ -239,7 +239,7 @@ reg_usbh_vbus: regulator-usbh-vbus {
&gpio {
/* Apalis GPIO7 MXM3 pin 15 PLX PEX 8605 PCIe Switch Reset */
- pex-perst-n {
+ pex-perst-n-hog {
gpio-hog;
gpios = <TEGRA_GPIO(S, 7) GPIO_ACTIVE_HIGH>;
output-high;
diff --git a/arch/arm/boot/dts/tegra30-apalis-v1.1-eval.dts b/arch/arm/boot/dts/tegra30-apalis-v1.1-eval.dts
index 86e138e..fbfa75e 100644
--- a/arch/arm/boot/dts/tegra30-apalis-v1.1-eval.dts
+++ b/arch/arm/boot/dts/tegra30-apalis-v1.1-eval.dts
@@ -257,7 +257,7 @@ reg_vddio_sdmmc3: regulator-vddio-sdmmc3 {
&gpio {
/* Apalis GPIO7 MXM3 pin 15 PLX PEX 8605 PCIe Switch Reset */
- pex-perst-n {
+ pex-perst-n-hog {
gpio-hog;
gpios = <TEGRA_GPIO(S, 7) GPIO_ACTIVE_HIGH>;
output-high;
diff --git a/arch/arm/boot/dts/tegra30-apalis-v1.1.dtsi b/arch/arm/boot/dts/tegra30-apalis-v1.1.dtsi
index 6a3a72f..380f22a 100644
--- a/arch/arm/boot/dts/tegra30-apalis-v1.1.dtsi
+++ b/arch/arm/boot/dts/tegra30-apalis-v1.1.dtsi
@@ -829,14 +829,17 @@ pv0 {
serial@70006040 {
compatible = "nvidia,tegra30-hsuart";
+ /delete-property/ reg-shift;
};
serial@70006200 {
compatible = "nvidia,tegra30-hsuart";
+ /delete-property/ reg-shift;
};
serial@70006300 {
compatible = "nvidia,tegra30-hsuart";
+ /delete-property/ reg-shift;
};
hdmi_ddc: i2c@7000c700 {
@@ -1047,9 +1050,6 @@ regulator@60 {
regulator-max-microvolt = <1400000>;
regulator-boot-on;
regulator-always-on;
- ti,vsel0-state-low;
- /* VSEL1: EN_CORE_DVFS_N low for DVFS */
- ti,vsel1-state-low;
};
};
diff --git a/arch/arm/boot/dts/tegra30-apalis.dtsi b/arch/arm/boot/dts/tegra30-apalis.dtsi
index b2ac51f..9bdc4cb 100644
--- a/arch/arm/boot/dts/tegra30-apalis.dtsi
+++ b/arch/arm/boot/dts/tegra30-apalis.dtsi
@@ -820,14 +820,17 @@ pv0 {
serial@70006040 {
compatible = "nvidia,tegra30-hsuart";
+ /delete-property/ reg-shift;
};
serial@70006200 {
compatible = "nvidia,tegra30-hsuart";
+ /delete-property/ reg-shift;
};
serial@70006300 {
compatible = "nvidia,tegra30-hsuart";
+ /delete-property/ reg-shift;
};
hdmi_ddc: i2c@7000c700 {
@@ -1030,9 +1033,6 @@ regulator@60 {
regulator-max-microvolt = <1400000>;
regulator-boot-on;
regulator-always-on;
- ti,vsel0-state-low;
- /* VSEL1: EN_CORE_DVFS_N low for DVFS */
- ti,vsel1-state-low;
};
};
diff --git a/arch/arm/boot/dts/tegra30-asus-lvds-display.dtsi b/arch/arm/boot/dts/tegra30-asus-lvds-display.dtsi
new file mode 100644
index 0000000..a047abf
--- /dev/null
+++ b/arch/arm/boot/dts/tegra30-asus-lvds-display.dtsi
@@ -0,0 +1,61 @@
+// SPDX-License-Identifier: GPL-2.0
+
+/* This dtsi file describes parts common for Asus T30 devices with a LVDS panel. */
+
+/ {
+ host1x@50000000 {
+ lcd: dc@54200000 {
+ rgb {
+ status = "okay";
+
+ port@0 {
+ dpi_output: endpoint {
+ remote-endpoint = <&bridge_input>;
+ bus-width = <24>;
+ };
+ };
+ };
+ };
+ };
+
+ display-panel {
+ power-supply = <&vdd_pnl>;
+ ddc-i2c-bus = <&lcd_ddc>;
+ backlight = <&backlight>;
+
+ port {
+ panel_input: endpoint {
+ remote-endpoint = <&bridge_output>;
+ };
+ };
+ };
+
+ /* Texas Instruments SN75LVDS83B LVDS Transmitter */
+ lvds-encoder {
+ compatible = "ti,sn75lvds83", "lvds-encoder";
+
+ powerdown-gpios = <&gpio TEGRA_GPIO(N, 6) GPIO_ACTIVE_LOW>;
+ power-supply = <&vdd_3v3_sys>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+
+ bridge_input: endpoint {
+ remote-endpoint = <&dpi_output>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+
+ bridge_output: endpoint {
+ remote-endpoint = <&panel_input>;
+ };
+ };
+ };
+ };
+};
diff --git a/arch/arm/boot/dts/tegra30-asus-nexus7-grouper-common.dtsi b/arch/arm/boot/dts/tegra30-asus-nexus7-grouper-common.dtsi
index 07d4ea1..2c2ad2a 100644
--- a/arch/arm/boot/dts/tegra30-asus-nexus7-grouper-common.dtsi
+++ b/arch/arm/boot/dts/tegra30-asus-nexus7-grouper-common.dtsi
@@ -8,6 +8,7 @@
#include "tegra30.dtsi"
#include "tegra30-cpu-opp.dtsi"
#include "tegra30-cpu-opp-microvolt.dtsi"
+#include "tegra30-asus-lvds-display.dtsi"
/ {
aliases {
@@ -59,21 +60,6 @@ trustzone@bfe00000 {
};
};
- host1x@50000000 {
- dc@54200000 {
- rgb {
- status = "okay";
-
- port@0 {
- lcd_output: endpoint {
- remote-endpoint = <&lvds_encoder_input>;
- bus-width = <24>;
- };
- };
- };
- };
- };
-
gpio@6000d000 {
init-mode-hog {
gpio-hog;
@@ -804,11 +790,13 @@ drive_gma {
uartb: serial@70006040 {
compatible = "nvidia,tegra30-hsuart";
+ /delete-property/ reg-shift;
/* GPS BCM4751 */
};
uartc: serial@70006200 {
compatible = "nvidia,tegra30-hsuart";
+ /delete-property/ reg-shift;
status = "okay";
nvidia,adjust-baud-rates = <0 9600 100>,
@@ -980,6 +968,7 @@ pmc@7000e400 {
nvidia,core-pwr-off-time = <0>;
nvidia,core-power-req-active-high;
nvidia,sys-clock-req-active-high;
+ core-supply = <&vdd_core>;
};
ahub@70080000 {
@@ -1069,7 +1058,7 @@ battery_cell: battery-cell {
};
/* PMIC has a built-in 32KHz oscillator which is used by PMC */
- clk32k_in: clock@0 {
+ clk32k_in: clock-32k {
compatible = "fixed-clock";
#clock-cells = <0>;
clock-frequency = <32768>;
@@ -1114,20 +1103,14 @@ display-panel {
*/
compatible = "panel-lvds";
- power-supply = <&vdd_pnl>;
- backlight = <&backlight>;
-
width-mm = <94>;
height-mm = <150>;
rotation = <180>;
data-mapping = "jeida-24";
- port {
- panel_input: endpoint {
- remote-endpoint = <&lvds_encoder_output>;
- };
- };
+ /* DDC unconnected on Nexus 7 */
+ /delete-property/ ddc-i2c-bus;
};
firmware {
@@ -1179,35 +1162,7 @@ volume-down {
};
};
- lvds-encoder {
- compatible = "ti,sn75lvds83", "lvds-encoder";
-
- powerdown-gpios = <&gpio TEGRA_GPIO(N, 6) GPIO_ACTIVE_LOW>;
- power-supply = <&vdd_3v3_sys>;
-
- ports {
- #address-cells = <1>;
- #size-cells = <0>;
-
- port@0 {
- reg = <0>;
-
- lvds_encoder_input: endpoint {
- remote-endpoint = <&lcd_output>;
- };
- };
-
- port@1 {
- reg = <1>;
-
- lvds_encoder_output: endpoint {
- remote-endpoint = <&panel_input>;
- };
- };
- };
- };
-
- vdd_5v0_sys: regulator@0 {
+ vdd_5v0_sys: regulator-5v0 {
compatible = "regulator-fixed";
regulator-name = "vdd_5v0";
regulator-min-microvolt = <5000000>;
@@ -1216,7 +1171,7 @@ vdd_5v0_sys: regulator@0 {
regulator-boot-on;
};
- vdd_3v3_sys: regulator@1 {
+ vdd_3v3_sys: regulator-3v3 {
compatible = "regulator-fixed";
regulator-name = "vdd_3v3";
regulator-min-microvolt = <3300000>;
@@ -1226,7 +1181,7 @@ vdd_3v3_sys: regulator@1 {
vin-supply = <&vdd_5v0_sys>;
};
- vdd_pnl: regulator@2 {
+ vdd_pnl: regulator-panel {
compatible = "regulator-fixed";
regulator-name = "vdd_panel";
regulator-min-microvolt = <3300000>;
@@ -1237,7 +1192,7 @@ vdd_pnl: regulator@2 {
vin-supply = <&vdd_3v3_sys>;
};
- vcc_3v3_ts: regulator@3 {
+ vcc_3v3_ts: regulator-ts {
compatible = "regulator-fixed";
regulator-name = "ldo_s-1167_3v3";
regulator-min-microvolt = <3300000>;
diff --git a/arch/arm/boot/dts/tegra30-asus-nexus7-grouper-maxim-pmic.dtsi b/arch/arm/boot/dts/tegra30-asus-nexus7-grouper-maxim-pmic.dtsi
index 53966fa..cd28e87 100644
--- a/arch/arm/boot/dts/tegra30-asus-nexus7-grouper-maxim-pmic.dtsi
+++ b/arch/arm/boot/dts/tegra30-asus-nexus7-grouper-maxim-pmic.dtsi
@@ -166,12 +166,12 @@ ldo8 {
};
};
- vdd_3v3_sys: regulator@1 {
+ vdd_3v3_sys: regulator-3v3 {
gpio = <&pmic 3 GPIO_ACTIVE_HIGH>;
enable-active-high;
};
- regulator@4 {
+ regulator-usb {
compatible = "regulator-fixed";
regulator-name = "avdd_usb";
regulator-min-microvolt = <3300000>;
diff --git a/arch/arm/boot/dts/tegra30-asus-nexus7-grouper-memory-timings.dtsi b/arch/arm/boot/dts/tegra30-asus-nexus7-grouper-memory-timings.dtsi
index bcff099..6c229e1 100644
--- a/arch/arm/boot/dts/tegra30-asus-nexus7-grouper-memory-timings.dtsi
+++ b/arch/arm/boot/dts/tegra30-asus-nexus7-grouper-memory-timings.dtsi
@@ -1565,13 +1565,13 @@ timing-667000000 {
};
&emc_icc_dvfs_opp_table {
- /delete-node/ opp@750000000,1300;
- /delete-node/ opp@800000000,1300;
- /delete-node/ opp@900000000,1350;
+ /delete-node/ opp-750000000-1300;
+ /delete-node/ opp-800000000-1300;
+ /delete-node/ opp-900000000-1350;
};
&emc_bw_dfs_opp_table {
- /delete-node/ opp@750000000;
- /delete-node/ opp@800000000;
- /delete-node/ opp@900000000;
+ /delete-node/ opp-750000000;
+ /delete-node/ opp-800000000;
+ /delete-node/ opp-900000000;
};
diff --git a/arch/arm/boot/dts/tegra30-asus-nexus7-grouper-ti-pmic.dtsi b/arch/arm/boot/dts/tegra30-asus-nexus7-grouper-ti-pmic.dtsi
index 9365ae6..ee4a3f48 100644
--- a/arch/arm/boot/dts/tegra30-asus-nexus7-grouper-ti-pmic.dtsi
+++ b/arch/arm/boot/dts/tegra30-asus-nexus7-grouper-ti-pmic.dtsi
@@ -143,7 +143,7 @@ vdd_core: core-regulator@60 {
};
};
- vdd_3v3_sys: regulator@1 {
+ vdd_3v3_sys: regulator-3v3 {
gpio = <&pmic 6 GPIO_ACTIVE_HIGH>;
enable-active-high;
};
diff --git a/arch/arm/boot/dts/tegra30-asus-nexus7-grouper.dtsi b/arch/arm/boot/dts/tegra30-asus-nexus7-grouper.dtsi
index a044dbd..564cfcd 100644
--- a/arch/arm/boot/dts/tegra30-asus-nexus7-grouper.dtsi
+++ b/arch/arm/boot/dts/tegra30-asus-nexus7-grouper.dtsi
@@ -137,7 +137,6 @@ i2c@7000c500 {
nfc@28 {
compatible = "nxp,pn544-i2c";
reg = <0x28>;
- clock-frequency = <100000>;
interrupt-parent = <&gpio>;
interrupts = <TEGRA_GPIO(X, 0) IRQ_TYPE_EDGE_RISING>;
diff --git a/arch/arm/boot/dts/tegra30-asus-nexus7-tilapia.dtsi b/arch/arm/boot/dts/tegra30-asus-nexus7-tilapia.dtsi
index a681ad5..cd63e0e 100644
--- a/arch/arm/boot/dts/tegra30-asus-nexus7-tilapia.dtsi
+++ b/arch/arm/boot/dts/tegra30-asus-nexus7-tilapia.dtsi
@@ -223,8 +223,6 @@ nfc@2a {
compatible = "nxp,pn544-i2c";
reg = <0x2a>;
- clock-frequency = <100000>;
-
interrupt-parent = <&gpio>;
interrupts = <TEGRA_GPIO(S, 7) IRQ_TYPE_EDGE_RISING>;
diff --git a/arch/arm/boot/dts/tegra30-asus-tf201.dts b/arch/arm/boot/dts/tegra30-asus-tf201.dts
new file mode 100644
index 0000000..315c6dc
--- /dev/null
+++ b/arch/arm/boot/dts/tegra30-asus-tf201.dts
@@ -0,0 +1,627 @@
+// SPDX-License-Identifier: GPL-2.0
+/dts-v1/;
+
+#include "tegra30-asus-transformer-common.dtsi"
+#include "tegra30-asus-lvds-display.dtsi"
+
+/ {
+ model = "Asus Transformer Prime TF201";
+ compatible = "asus,tf201", "nvidia,tegra30";
+
+ pinmux@70000868 {
+ state_default: pinmux {
+ lcd_pwr2_pc6 {
+ nvidia,pins = "lcd_pwr2_pc6",
+ "lcd_dc1_pd2";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ pbb3 {
+ nvidia,pins = "pbb3";
+ nvidia,function = "vgp3";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ pbb7 {
+ nvidia,pins = "pbb7";
+ nvidia,function = "i2s4";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ kb_row7_pr7 {
+ nvidia,pins = "kb_row7_pr7";
+ nvidia,function = "kbc";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ gmi_cs4_n_pk2 {
+ nvidia,pins = "gmi_cs4_n_pk2";
+ nvidia,function = "gmi";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+ };
+ };
+
+ serial@70006200 {
+ /* Azurewave AW-NH615 BCM4329B1 */
+ bluetooth {
+ compatible = "brcm,bcm4329-bt";
+ };
+ };
+
+ i2c@7000c400 {
+ /* Atmel MXT768E touchscreen */
+ touchscreen@4d {
+ compatible = "atmel,maxtouch";
+ reg = <0x4d>;
+
+ interrupt-parent = <&gpio>;
+ interrupts = <TEGRA_GPIO(H, 4) IRQ_TYPE_EDGE_FALLING>;
+ reset-gpios = <&gpio TEGRA_GPIO(H, 6) GPIO_ACTIVE_LOW>;
+
+ vdda-supply = <&vdd_3v3_sys>;
+ vdd-supply = <&vdd_3v3_sys>;
+ };
+ };
+
+ i2c@7000c500 {
+ clock-frequency = <100000>;
+
+ magnetometer@e {
+ mount-matrix = "-1", "0", "0",
+ "0", "-1", "0",
+ "0", "0", "-1";
+ };
+
+ gyroscope@68 {
+ mount-matrix = "0", "-1", "0",
+ "-1", "0", "0",
+ "0", "0", "-1";
+
+ /* External I2C interface */
+ i2c-gate {
+ accelerometer@f {
+ mount-matrix = "1", "0", "0",
+ "0", "-1", "0",
+ "0", "0", "1";
+ };
+ };
+ };
+ };
+
+ i2c@7000d000 {
+ /* Realtek ALC5631 audio codec */
+ rt5631: audio-codec@1a {
+ compatible = "realtek,rt5631";
+ reg = <0x1a>;
+ };
+ };
+
+ memory-controller@7000f000 {
+ emc-timings-0 {
+ /* Elpida 1GB EDB8132B2MA-8D-F LPDDR2 400MHz */
+ nvidia,ram-code = <0>;
+
+ timing-25500000 {
+ clock-frequency = <25500000>;
+
+ nvidia,emem-configuration = < 0x00020001 0x80000010
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000003 0x00000001 0x00000002 0x00000004
+ 0x00000001 0x00000000 0x00000002 0x00000002
+ 0x02020001 0x00060402 0x73e30303 0x001f0000 >;
+ };
+
+ timing-51000000 {
+ clock-frequency = <51000000>;
+
+ nvidia,emem-configuration = < 0x00010001 0x80000010
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000003 0x00000001 0x00000002 0x00000004
+ 0x00000001 0x00000000 0x00000002 0x00000002
+ 0x02020001 0x00060402 0x72c30303 0x001f0000 >;
+ };
+
+ timing-102000000 {
+ clock-frequency = <102000000>;
+
+ nvidia,emem-configuration = < 0x00000001 0x80000018
+ 0x00000001 0x00000001 0x00000003 0x00000001
+ 0x00000003 0x00000001 0x00000002 0x00000004
+ 0x00000001 0x00000000 0x00000002 0x00000002
+ 0x02020001 0x00060403 0x72430504 0x001f0000 >;
+ };
+
+ timing-204000000 {
+ clock-frequency = <204000000>;
+
+ nvidia,emem-configuration = < 0x00000003 0x80000025
+ 0x00000001 0x00000001 0x00000006 0x00000003
+ 0x00000005 0x00000001 0x00000002 0x00000004
+ 0x00000001 0x00000000 0x00000003 0x00000002
+ 0x02030001 0x00070506 0x71e40a07 0x001f0000 >;
+ };
+
+ timing-400000000 {
+ clock-frequency = <400000000>;
+
+ nvidia,emem-configuration = < 0x00000006 0x80000048
+ 0x00000002 0x00000003 0x0000000c 0x00000007
+ 0x00000009 0x00000001 0x00000002 0x00000006
+ 0x00000001 0x00000000 0x00000004 0x00000004
+ 0x04040001 0x000d090c 0x71c6120d 0x001f0000 >;
+ };
+ };
+
+ emc-timings-1 {
+ /* TF201 Unknown 1GB LPDDR2 500MHZ */
+ nvidia,ram-code = <1>;
+
+ timing-25500000 {
+ clock-frequency = <25500000>;
+
+ nvidia,emem-configuration = < 0x00020001 0x80000010
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000003 0x00000001 0x00000002 0x00000004
+ 0x00000001 0x00000000 0x00000002 0x00000002
+ 0x02020001 0x00060402 0x73e30303 0x001f0000 >;
+ };
+
+ timing-51000000 {
+ clock-frequency = <51000000>;
+
+ nvidia,emem-configuration = < 0x00010001 0x80000010
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000003 0x00000001 0x00000002 0x00000004
+ 0x00000001 0x00000000 0x00000002 0x00000002
+ 0x02020001 0x00060402 0x72c30303 0x001f0000 >;
+ };
+
+ timing-102000000 {
+ clock-frequency = <102000000>;
+
+ nvidia,emem-configuration = < 0x00000001 0x80000018
+ 0x00000001 0x00000001 0x00000003 0x00000001
+ 0x00000003 0x00000001 0x00000002 0x00000004
+ 0x00000001 0x00000000 0x00000002 0x00000002
+ 0x02020001 0x00060403 0x72430504 0x001f0000 >;
+ };
+
+ timing-204000000 {
+ clock-frequency = <204000000>;
+
+ nvidia,emem-configuration = < 0x00000003 0x80000025
+ 0x00000001 0x00000001 0x00000006 0x00000003
+ 0x00000005 0x00000001 0x00000002 0x00000004
+ 0x00000001 0x00000000 0x00000003 0x00000002
+ 0x02030001 0x00070506 0x71e40a07 0x001f0000 >;
+ };
+
+ timing-500000000 {
+ clock-frequency = <500000000>;
+
+ nvidia,emem-configuration = < 0x00000007 0x8000005a
+ 0x00000003 0x00000004 0x0000000e 0x00000009
+ 0x0000000c 0x00000002 0x00000002 0x00000008
+ 0x00000001 0x00000000 0x00000004 0x00000005
+ 0x05040001 0x00100a0e 0x71c8170f 0x001f0000 >;
+ };
+ };
+ };
+
+ memory-controller@7000f400 {
+ emc-timings-0 {
+ /* Elpida 1GB EDB8132B2MA-8D-F LPDDR2 400MHz */
+ nvidia,ram-code = <0>;
+
+ timing-25500000 {
+ clock-frequency = <25500000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x00010022>;
+ nvidia,emc-mode-2 = <0x00020001>;
+ nvidia,emc-mode-reset = <0x00000000>;
+ nvidia,emc-zcal-cnt-long = <0x00000009>;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000001
+ 0x00000003 0x00000002 0x00000002 0x00000004
+ 0x00000004 0x00000001 0x00000005 0x00000002
+ 0x00000002 0x00000001 0x00000001 0x00000000
+ 0x00000001 0x00000003 0x00000001 0x0000000b
+ 0x0000000a 0x00000060 0x00000000 0x00000018
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000001 0x00000007 0x00000004 0x00000004
+ 0x00000003 0x00000008 0x00000004 0x00000004
+ 0x00000002 0x0000006b 0x00000004 0x00000004
+ 0x00000000 0x00000000 0x00004282 0x00780084
+ 0x00008000 0x00098000 0x00098000 0x00098000
+ 0x00098000 0x00000010 0x00000010 0x00000010
+ 0x00000010 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000008 0x00000008 0x00000008
+ 0x00000008 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00080000 0x00080000 0x00080000
+ 0x00080000 0x00100220 0x0800201c 0x00000000
+ 0x77ffc004 0x01f1f008 0x00000000 0x00000007
+ 0x08000068 0x08000000 0x00000802 0x00000000
+ 0x00000009 0x00090009 0xa0f10000 0x00000000
+ 0x00000000 0x800001c5 0xe0000000 0xff00ff00 >;
+ };
+
+ timing-51000000 {
+ clock-frequency = <51000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x00010022>;
+ nvidia,emc-mode-2 = <0x00020001>;
+ nvidia,emc-mode-reset = <0x00000000>;
+ nvidia,emc-zcal-cnt-long = <0x00000009>;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000003
+ 0x00000006 0x00000002 0x00000002 0x00000004
+ 0x00000004 0x00000001 0x00000005 0x00000002
+ 0x00000002 0x00000001 0x00000001 0x00000000
+ 0x00000001 0x00000003 0x00000001 0x0000000b
+ 0x0000000a 0x000000c0 0x00000000 0x00000030
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000001 0x00000007 0x00000008 0x00000008
+ 0x00000003 0x00000008 0x00000004 0x00000004
+ 0x00000002 0x000000d5 0x00000004 0x00000004
+ 0x00000000 0x00000000 0x00004282 0x00780084
+ 0x00008000 0x00098000 0x00098000 0x00098000
+ 0x00098000 0x00000010 0x00000010 0x00000010
+ 0x00000010 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000018 0x00000018 0x00000018
+ 0x00000018 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00080000 0x00080000 0x00080000
+ 0x00080000 0x00100220 0x0800201c 0x00000000
+ 0x77ffc004 0x01f1f008 0x00000000 0x00000007
+ 0x08000068 0x08000000 0x00000802 0x00000000
+ 0x00000009 0x00090009 0xa0f10000 0x00000000
+ 0x00000000 0x80000287 0xe0000000 0xff00ff00 >;
+ };
+
+ timing-102000000 {
+ clock-frequency = <102000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x00010022>;
+ nvidia,emc-mode-2 = <0x00020001>;
+ nvidia,emc-mode-reset = <0x00000000>;
+ nvidia,emc-zcal-cnt-long = <0x0000000a>;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000006
+ 0x0000000d 0x00000004 0x00000002 0x00000004
+ 0x00000004 0x00000001 0x00000005 0x00000002
+ 0x00000002 0x00000001 0x00000001 0x00000000
+ 0x00000001 0x00000003 0x00000001 0x0000000b
+ 0x0000000a 0x00000181 0x00000000 0x00000060
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000001 0x00000007 0x0000000f 0x0000000f
+ 0x00000003 0x00000008 0x00000004 0x00000004
+ 0x00000002 0x000001a9 0x00000004 0x00000006
+ 0x00000000 0x00000000 0x00004282 0x00780084
+ 0x00008000 0x000a0000 0x000a0000 0x000a0000
+ 0x000a0000 0x00000010 0x00000010 0x00000010
+ 0x00000010 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000008 0x00000008 0x00000008
+ 0x00000008 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00080000 0x00080000 0x00080000
+ 0x00080000 0x00120220 0x0800201c 0x00000000
+ 0x77ffc004 0x01f1f008 0x00000000 0x00000007
+ 0x08000068 0x08000000 0x00000802 0x00000000
+ 0x0000000a 0x00090009 0xa0f10000 0x00000000
+ 0x00000000 0x8000040b 0xe0000000 0xff00ff00 >;
+ };
+
+ timing-204000000 {
+ clock-frequency = <204000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x00010042>;
+ nvidia,emc-mode-2 = <0x00020001>;
+ nvidia,emc-mode-reset = <0x00000000>;
+ nvidia,emc-zcal-cnt-long = <0x00000013>;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x0000000c
+ 0x0000001a 0x00000008 0x00000003 0x00000005
+ 0x00000004 0x00000001 0x00000006 0x00000003
+ 0x00000003 0x00000002 0x00000002 0x00000000
+ 0x00000001 0x00000003 0x00000001 0x0000000c
+ 0x0000000a 0x00000303 0x00000000 0x000000c0
+ 0x00000001 0x00000001 0x00000003 0x00000000
+ 0x00000001 0x00000007 0x0000001d 0x0000001d
+ 0x00000004 0x0000000b 0x00000005 0x00000004
+ 0x00000002 0x00000351 0x00000004 0x00000006
+ 0x00000000 0x00000000 0x00004282 0x00440084
+ 0x00008000 0x00074000 0x00074000 0x00074000
+ 0x00074000 0x00000010 0x00000010 0x00000010
+ 0x00000010 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000018 0x00000018 0x00000018
+ 0x00000018 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00078000 0x00078000 0x00078000
+ 0x00078000 0x00100220 0x0800201c 0x00000000
+ 0x77ffc004 0x01f1f008 0x00000000 0x00000007
+ 0x08000068 0x08000000 0x00000802 0x00000000
+ 0x00000013 0x00090009 0xa0f10000 0x00000000
+ 0x00000000 0x80000713 0xe0000000 0xff00ff00 >;
+ };
+
+ timing-400000000 {
+ clock-frequency = <400000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x00010082>;
+ nvidia,emc-mode-2 = <0x00020004>;
+ nvidia,emc-mode-reset = <0x00000000>;
+ nvidia,emc-zcal-cnt-long = <0x00000024>;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000017
+ 0x00000033 0x00000010 0x00000007 0x00000007
+ 0x00000007 0x00000002 0x0000000a 0x00000007
+ 0x00000007 0x00000003 0x00000002 0x00000000
+ 0x00000003 0x00000007 0x00000004 0x0000000d
+ 0x0000000e 0x000005e9 0x00000000 0x0000017a
+ 0x00000002 0x00000002 0x00000007 0x00000000
+ 0x00000001 0x0000000c 0x00000038 0x00000038
+ 0x00000006 0x00000014 0x00000009 0x00000004
+ 0x00000002 0x00000680 0x00000000 0x00000006
+ 0x00000000 0x00000000 0x00006282 0x001d0084
+ 0x00008000 0x0002c000 0x0002c000 0x0002c000
+ 0x0002c000 0x00000010 0x00000010 0x00000010
+ 0x00000010 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000008 0x00000008 0x00000008
+ 0x00000008 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00048000 0x00048000 0x00048000
+ 0x00048000 0x000c0220 0x0800003d 0x00000000
+ 0x77ffc004 0x01f1f408 0x00000000 0x00000007
+ 0x08000068 0x08000000 0x00000802 0x00000000
+ 0x00000024 0x000c000c 0xa0f10000 0x00000000
+ 0x00000000 0x80000ce6 0xe0000000 0xff00ff88 >;
+ };
+ };
+
+ emc-timings-1 {
+ /* TF201 Unknown 1GB LPDDR2 500MHZ */
+ nvidia,ram-code = <1>;
+
+ timing-25500000 {
+ clock-frequency = <25500000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x00010022>;
+ nvidia,emc-mode-2 = <0x00020001>;
+ nvidia,emc-mode-reset = <0x00000000>;
+ nvidia,emc-zcal-cnt-long = <0x00000009>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000001
+ 0x00000003 0x00000002 0x00000002 0x00000004
+ 0x00000004 0x00000001 0x00000005 0x00000002
+ 0x00000002 0x00000001 0x00000001 0x00000000
+ 0x00000001 0x00000003 0x00000001 0x0000000b
+ 0x00000009 0x00000060 0x00000000 0x00000018
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000001 0x00000007 0x00000004 0x00000004
+ 0x00000003 0x00000008 0x00000004 0x00000004
+ 0x00000002 0x0000006b 0x00000004 0x00000004
+ 0x00000000 0x00000000 0x00004282 0x00780084
+ 0x00008000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00100220 0x0800201c 0x00000000
+ 0x77ffc004 0x01f1f008 0x00000000 0x00000007
+ 0x08000068 0x08000000 0x00000802 0x00064000
+ 0x0000000a 0x00090009 0xa0f10000 0x00000000
+ 0x00000000 0x800001c5 0xe0000000 0xff00ff00 >;
+ };
+
+ timing-51000000 {
+ clock-frequency = <51000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x00010022>;
+ nvidia,emc-mode-2 = <0x00020001>;
+ nvidia,emc-mode-reset = <0x00000000>;
+ nvidia,emc-zcal-cnt-long = <0x00000009>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000003
+ 0x00000006 0x00000002 0x00000002 0x00000004
+ 0x00000004 0x00000001 0x00000005 0x00000002
+ 0x00000002 0x00000001 0x00000001 0x00000000
+ 0x00000001 0x00000003 0x00000001 0x0000000b
+ 0x00000009 0x000000c0 0x00000000 0x00000030
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000001 0x00000007 0x00000008 0x00000008
+ 0x00000003 0x00000008 0x00000004 0x00000004
+ 0x00000002 0x000000d5 0x00000004 0x00000004
+ 0x00000000 0x00000000 0x00004282 0x00780084
+ 0x00008000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00100220 0x0800201c 0x00000000
+ 0x77ffc004 0x01f1f008 0x00000000 0x00000007
+ 0x08000068 0x08000000 0x00000802 0x00064000
+ 0x00000013 0x00090009 0xa0f10000 0x00000000
+ 0x00000000 0x80000287 0xe0000000 0xff00ff00 >;
+ };
+
+ timing-102000000 {
+ clock-frequency = <102000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x00010022>;
+ nvidia,emc-mode-2 = <0x00020001>;
+ nvidia,emc-mode-reset = <0x00000000>;
+ nvidia,emc-zcal-cnt-long = <0x0000000a>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000006
+ 0x0000000d 0x00000004 0x00000002 0x00000004
+ 0x00000004 0x00000001 0x00000005 0x00000002
+ 0x00000002 0x00000001 0x00000001 0x00000000
+ 0x00000001 0x00000003 0x00000001 0x0000000b
+ 0x00000009 0x00000181 0x00000000 0x00000060
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000001 0x00000007 0x0000000f 0x0000000f
+ 0x00000003 0x00000008 0x00000004 0x00000004
+ 0x00000002 0x000001a9 0x00000004 0x00000004
+ 0x00000000 0x00000000 0x00004282 0x00780084
+ 0x00008000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00100220 0x0800201c 0x00000000
+ 0x77ffc004 0x01f1f008 0x00000000 0x00000007
+ 0x08000068 0x08000000 0x00000802 0x00064000
+ 0x00000025 0x00090009 0xa0f10000 0x00000000
+ 0x00000000 0x8000040b 0xe0000000 0xff00ff00 >;
+ };
+
+ timing-204000000 {
+ clock-frequency = <204000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x00010042>;
+ nvidia,emc-mode-2 = <0x00020001>;
+ nvidia,emc-mode-reset = <0x00000000>;
+ nvidia,emc-zcal-cnt-long = <0x00000013>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x0000000c
+ 0x0000001a 0x00000008 0x00000003 0x00000005
+ 0x00000004 0x00000001 0x00000006 0x00000003
+ 0x00000003 0x00000002 0x00000002 0x00000000
+ 0x00000001 0x00000003 0x00000001 0x0000000c
+ 0x0000000a 0x00000303 0x00000000 0x000000c0
+ 0x00000001 0x00000001 0x00000003 0x00000000
+ 0x00000001 0x00000007 0x0000001d 0x0000001d
+ 0x00000004 0x0000000b 0x00000005 0x00000004
+ 0x00000002 0x00000351 0x00000004 0x00000006
+ 0x00000000 0x00000000 0x00004282 0x00440084
+ 0x00008000 0x00060000 0x00060000 0x00060000
+ 0x00060000 0x00072000 0x00072000 0x00072000
+ 0x00072000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x000d0000 0x000d0000 0x000d0000
+ 0x000d0000 0x000e0220 0x0800201c 0x00000000
+ 0x77ffc004 0x01f1f008 0x00000000 0x00000007
+ 0x08000068 0x08000000 0x00000802 0x00064000
+ 0x0000004a 0x00090009 0xa0f10000 0x00000000
+ 0x00000000 0x80000713 0xe0000000 0xff00ff00 >;
+ };
+
+ timing-500000000 {
+ clock-frequency = <500000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x000100c2>;
+ nvidia,emc-mode-2 = <0x00020005>;
+ nvidia,emc-mode-reset = <0x00000000>;
+ nvidia,emc-zcal-cnt-long = <0x0000002d>;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x0000001d
+ 0x00000040 0x00000014 0x00000008 0x00000007
+ 0x00000009 0x00000003 0x0000000d 0x00000008
+ 0x00000008 0x00000004 0x00000002 0x00000000
+ 0x00000004 0x00000008 0x00000005 0x0000000d
+ 0x0000000f 0x00000763 0x00000000 0x000001d8
+ 0x00000003 0x00000003 0x00000008 0x00000000
+ 0x00000001 0x0000000e 0x00000046 0x00000046
+ 0x00000008 0x00000019 0x0000000b 0x00000004
+ 0x00000002 0x00000820 0x00000000 0x00000006
+ 0x00000000 0x00000000 0x00006282 0xf0140091
+ 0x00008000 0x00000008 0x00000008 0x00000008
+ 0x00000008 0x0000000a 0x0000000a 0x0000000a
+ 0x0000000a 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x0000000c 0x0000000c 0x0000000c
+ 0x0000000c 0x00080220 0x0800003d 0x00000000
+ 0x77ffc004 0x01f1f408 0x00000000 0x00000007
+ 0x08000068 0x08000000 0x00000802 0x00064000
+ 0x000000b4 0x000d000d 0xa0f10404 0x00000000
+ 0x00000000 0x80000fde 0xe0000000 0xff00ff88 >;
+ };
+ };
+ };
+
+ usb-phy@7d000000 {
+ /delete-property/ nvidia,xcvr-setup-use-fuses;
+ nvidia,xcvr-setup = <5>; /* Based on TF201 fuse value - 48 */
+ };
+
+ usb-phy@7d008000 {
+ /delete-property/ nvidia,xcvr-setup-use-fuses;
+ nvidia,xcvr-setup = <5>; /* Based on TF201 fuse value - 48 */
+ };
+
+ display-panel {
+ compatible = "hannstar,hsd101pww2";
+ };
+
+ haptic-feedback {
+ compatible = "gpio-vibrator";
+ enable-gpios = <&gpio TEGRA_GPIO(H, 7) GPIO_ACTIVE_HIGH>;
+ vcc-supply = <&vdd_3v3_sys>;
+ };
+};
+
+&emc_icc_dvfs_opp_table {
+ /delete-node/ opp-533000000-1200;
+ /delete-node/ opp-625000000-1200;
+ /delete-node/ opp-625000000-1250;
+ /delete-node/ opp-667000000-1200;
+ /delete-node/ opp-750000000-1300;
+ /delete-node/ opp-800000000-1300;
+ /delete-node/ opp-900000000-1350;
+};
+
+&emc_bw_dfs_opp_table {
+ /delete-node/ opp-533000000;
+ /delete-node/ opp-625000000;
+ /delete-node/ opp-667000000;
+ /delete-node/ opp-750000000;
+ /delete-node/ opp-800000000;
+ /delete-node/ opp-900000000;
+};
diff --git a/arch/arm/boot/dts/tegra30-asus-tf300t.dts b/arch/arm/boot/dts/tegra30-asus-tf300t.dts
new file mode 100644
index 0000000..f474348
--- /dev/null
+++ b/arch/arm/boot/dts/tegra30-asus-tf300t.dts
@@ -0,0 +1,1034 @@
+// SPDX-License-Identifier: GPL-2.0
+/dts-v1/;
+
+#include "tegra30-asus-transformer-common.dtsi"
+#include "tegra30-asus-lvds-display.dtsi"
+
+/ {
+ model = "Asus Transformer Pad TF300T";
+ compatible = "asus,tf300t", "nvidia,tegra30";
+
+ gpio@6000d000 {
+ tf300t-init-hog {
+ gpio-hog;
+ gpios = <TEGRA_GPIO(BB, 5) GPIO_ACTIVE_HIGH>;
+ output-low;
+ };
+ };
+
+ pinmux@70000868 {
+ state_default: pinmux {
+ lcd_pwr2_pc6 {
+ nvidia,pins = "lcd_pwr2_pc6",
+ "lcd_dc1_pd2";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ pbb3 {
+ nvidia,pins = "pbb3";
+ nvidia,function = "vgp3";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ pbb7 {
+ nvidia,pins = "pbb7";
+ nvidia,function = "i2s4";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ kb_row7_pr7 {
+ nvidia,pins = "kb_row7_pr7";
+ nvidia,function = "kbc";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ gmi_cs4_n_pk2 {
+ nvidia,pins = "gmi_cs4_n_pk2";
+ nvidia,function = "gmi";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+ };
+ };
+
+ serial@70006200 {
+ /* Azurewave AW-NH615 BCM4329B1 */
+ bluetooth {
+ compatible = "brcm,bcm4329-bt";
+ };
+ };
+
+ i2c@7000c400 {
+ /* Elantech EKTH1036 touchscreen */
+ touchscreen@10 {
+ compatible = "elan,ektf3624";
+ reg = <0x10>;
+
+ interrupt-parent = <&gpio>;
+ interrupts = <TEGRA_GPIO(H, 4) IRQ_TYPE_EDGE_FALLING>;
+ reset-gpios = <&gpio TEGRA_GPIO(H, 6) GPIO_ACTIVE_LOW>;
+
+ vcc33-supply = <&vdd_3v3_sys>;
+ vccio-supply = <&vdd_3v3_sys>;
+
+ touchscreen-size-x = <2240>;
+ touchscreen-size-y = <1408>;
+ touchscreen-inverted-y;
+ };
+ };
+
+ i2c@7000c500 {
+ clock-frequency = <400000>;
+
+ magnetometer@e {
+ mount-matrix = "0", "-1", "0",
+ "-1", "0", "0",
+ "0", "0", "-1";
+ };
+
+ gyroscope@68 {
+ mount-matrix = "-1", "0", "0",
+ "0", "1", "0",
+ "0", "0", "-1";
+
+ /* External I2C interface */
+ i2c-gate {
+ accelerometer@f {
+ mount-matrix = "0", "-1", "0",
+ "-1", "0", "0",
+ "0", "0", "1";
+ };
+ };
+ };
+ };
+
+ i2c@7000d000 {
+ /* Wolfson Microelectronics WM8903 audio codec */
+ wm8903: audio-codec@1a {
+ compatible = "wlf,wm8903";
+ reg = <0x1a>;
+
+ interrupt-parent = <&gpio>;
+ interrupts = <TEGRA_GPIO(W, 3) IRQ_TYPE_LEVEL_LOW>;
+
+ gpio-controller;
+ #gpio-cells = <2>;
+
+ micdet-cfg = <0>;
+ micdet-delay = <100>;
+
+ gpio-cfg = <
+ 0xffffffff /* don't touch */
+ 0xffffffff /* don't touch */
+ 0x00000000 /* Speaker-enable GPIO, output, low */
+ 0xffffffff /* don't touch */
+ 0xffffffff /* don't touch */
+ >;
+
+ AVDD-supply = <&vdd_1v8_vio>;
+ CPVDD-supply = <&vdd_1v8_vio>;
+ DBVDD-supply = <&vdd_1v8_vio>;
+ DCVDD-supply = <&vdd_1v8_vio>;
+ };
+ };
+
+ memory-controller@7000f000 {
+ emc-timings-0 {
+ /* Elpida 1GB 667MHZ */
+ nvidia,ram-code = <0>;
+
+ timing-25500000 {
+ clock-frequency = <25500000>;
+
+ nvidia,emem-configuration = < 0x00030003 0xc0000020
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000001 0x00000001 0x00000003 0x00000008
+ 0x00000002 0x00000001 0x00000002 0x00000006
+ 0x06020102 0x000a0502 0x74830303 0x001f0000 >;
+ };
+
+ timing-51000000 {
+ clock-frequency = <51000000>;
+
+ nvidia,emem-configuration = < 0x00010003 0xc0000020
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000001 0x00000001 0x00000003 0x00000008
+ 0x00000002 0x00000001 0x00000002 0x00000006
+ 0x06020102 0x000a0502 0x73430303 0x001f0000 >;
+ };
+
+ timing-102000000 {
+ clock-frequency = <102000000>;
+
+ nvidia,emem-configuration = < 0x00000003 0xc0000030
+ 0x00000001 0x00000001 0x00000003 0x00000000
+ 0x00000001 0x00000001 0x00000003 0x00000008
+ 0x00000002 0x00000001 0x00000002 0x00000006
+ 0x06020102 0x000a0503 0x72830504 0x001f0000 >;
+ };
+
+ timing-204000000 {
+ clock-frequency = <204000000>;
+
+ nvidia,emem-configuration = < 0x00000006 0xc0000025
+ 0x00000001 0x00000001 0x00000005 0x00000002
+ 0x00000003 0x00000001 0x00000003 0x00000008
+ 0x00000002 0x00000001 0x00000002 0x00000006
+ 0x06020102 0x000a0505 0x72440a06 0x001f0000 >;
+ };
+
+ timing-333500000 {
+ clock-frequency = <333500000>;
+
+ nvidia,emem-configuration = < 0x0000000a 0xc000003d
+ 0x00000001 0x00000002 0x00000008 0x00000004
+ 0x00000004 0x00000001 0x00000002 0x00000007
+ 0x00000002 0x00000002 0x00000003 0x00000006
+ 0x06030202 0x000b0608 0x70850f09 0x001f0000 >;
+ };
+
+ timing-667000000 {
+ clock-frequency = <667000000>;
+
+ nvidia,emem-configuration = < 0x00000014 0xc0000079
+ 0x00000003 0x00000004 0x00000010 0x0000000b
+ 0x0000000a 0x00000001 0x00000003 0x0000000b
+ 0x00000002 0x00000002 0x00000004 0x00000008
+ 0x08040202 0x00130b10 0x70ea1f11 0x001f0000 >;
+ };
+ };
+
+ emc-timings-1 {
+ /* Hynix 1GB 667MHZ */
+ nvidia,ram-code = <1>;
+
+ timing-25500000 {
+ clock-frequency = <25500000>;
+
+ nvidia,emem-configuration = < 0x00030003 0xc0000020
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000001 0x00000001 0x00000003 0x00000008
+ 0x00000002 0x00000001 0x00000002 0x00000006
+ 0x06020102 0x000a0502 0x74830303 0x001f0000 >;
+ };
+
+ timing-51000000 {
+ clock-frequency = <51000000>;
+
+ nvidia,emem-configuration = < 0x00010003 0xc0000020
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000001 0x00000001 0x00000003 0x00000008
+ 0x00000002 0x00000001 0x00000002 0x00000006
+ 0x06020102 0x000a0502 0x73430303 0x001f0000 >;
+ };
+
+ timing-102000000 {
+ clock-frequency = <102000000>;
+
+ nvidia,emem-configuration = < 0x00000003 0xc0000030
+ 0x00000001 0x00000001 0x00000003 0x00000000
+ 0x00000001 0x00000001 0x00000003 0x00000008
+ 0x00000002 0x00000001 0x00000002 0x00000006
+ 0x06020102 0x000a0503 0x72830504 0x001f0000 >;
+ };
+
+ timing-204000000 {
+ clock-frequency = <204000000>;
+
+ nvidia,emem-configuration = < 0x00000006 0xc0000025
+ 0x00000001 0x00000001 0x00000005 0x00000002
+ 0x00000003 0x00000001 0x00000003 0x00000008
+ 0x00000002 0x00000001 0x00000002 0x00000006
+ 0x06020102 0x000a0605 0x72440a06 0x001f0000 >;
+ };
+
+ timing-333500000 {
+ clock-frequency = <333500000>;
+
+ nvidia,emem-configuration = < 0x0000000a 0xc000003d
+ 0x00000001 0x00000002 0x00000008 0x00000005
+ 0x00000004 0x00000001 0x00000002 0x00000007
+ 0x00000002 0x00000002 0x00000003 0x00000006
+ 0x06030202 0x000b0608 0x70850f09 0x001f0000 >;
+ };
+
+ timing-667000000 {
+ clock-frequency = <667000000>;
+
+ nvidia,emem-configuration = < 0x00000014 0xc0000079
+ 0x00000003 0x00000004 0x00000011 0x0000000b
+ 0x0000000a 0x00000001 0x00000003 0x0000000b
+ 0x00000002 0x00000002 0x00000004 0x00000008
+ 0x08040202 0x00140b11 0x70ea1f12 0x001f0000 >;
+ };
+ };
+
+ emc-timings-2 {
+ /* Micron 1GB 667MHZ */
+ nvidia,ram-code = <2>;
+
+ timing-25500000 {
+ clock-frequency = <25500000>;
+
+ nvidia,emem-configuration = < 0x00020001 0xc0000020
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000001 0x00000001 0x00000003 0x00000008
+ 0x00000002 0x00000001 0x00000002 0x00000006
+ 0x06020102 0x000a0502 0x74830303 0x001f0000 >;
+ };
+
+ timing-51000000 {
+ clock-frequency = <51000000>;
+
+ nvidia,emem-configuration = < 0x00010001 0xc0000020
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000001 0x00000001 0x00000003 0x00000008
+ 0x00000002 0x00000001 0x00000002 0x00000006
+ 0x06020102 0x000a0502 0x73430303 0x001f0000 >;
+ };
+
+ timing-102000000 {
+ clock-frequency = <102000000>;
+
+ nvidia,emem-configuration = < 0x00000001 0xc0000030
+ 0x00000001 0x00000001 0x00000003 0x00000000
+ 0x00000001 0x00000001 0x00000003 0x00000008
+ 0x00000002 0x00000001 0x00000002 0x00000006
+ 0x06020102 0x000a0503 0x72830504 0x001f0000 >;
+ };
+
+ timing-204000000 {
+ clock-frequency = <204000000>;
+
+ nvidia,emem-configuration = < 0x00000003 0xc0000025
+ 0x00000001 0x00000001 0x00000005 0x00000002
+ 0x00000003 0x00000001 0x00000003 0x00000008
+ 0x00000002 0x00000001 0x00000002 0x00000006
+ 0x06020102 0x000a0505 0x72440a06 0x001f0000 >;
+ };
+
+ timing-333500000 {
+ clock-frequency = <333500000>;
+
+ nvidia,emem-configuration = < 0x00000005 0xc000003d
+ 0x00000001 0x00000002 0x00000008 0x00000004
+ 0x00000004 0x00000001 0x00000002 0x00000007
+ 0x00000002 0x00000002 0x00000003 0x00000006
+ 0x06030202 0x000b0608 0x70850f09 0x001f0000 >;
+ };
+
+ timing-667000000 {
+ clock-frequency = <667000000>;
+
+ nvidia,emem-configuration = < 0x0000000a 0xc0000079
+ 0x00000003 0x00000004 0x00000010 0x0000000a
+ 0x0000000a 0x00000001 0x00000003 0x0000000b
+ 0x00000002 0x00000002 0x00000004 0x00000008
+ 0x08040202 0x00140b10 0x70ea1f11 0x001f0000 >;
+ };
+ };
+ };
+
+ memory-controller@7000f400 {
+ emc-timings-0 {
+ /* Elpida 1GB 667MHZ */
+ nvidia,ram-code = <0>;
+
+ timing-25500000 {
+ clock-frequency = <25500000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200008>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000001
+ 0x00000004 0x00000000 0x00000000 0x00000002
+ 0x0000000a 0x00000005 0x0000000b 0x00000000
+ 0x00000000 0x00000003 0x00000001 0x00000000
+ 0x00000005 0x00000005 0x00000004 0x00000009
+ 0x0000000b 0x000000c0 0x00000000 0x00000030
+ 0x00000002 0x00000002 0x00000001 0x00000000
+ 0x00000007 0x0000000f 0x00000005 0x00000005
+ 0x00000004 0x00000001 0x00000000 0x00000004
+ 0x00000005 0x000000c7 0x00000006 0x00000004
+ 0x00000000 0x00000000 0x00004288 0x007800a4
+ 0x00008000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000002a0 0x0800211c 0x00000000
+ 0x77fff884 0x01f1f108 0x05057404 0x54000007
+ 0x08000168 0x08000000 0x00000802 0x00000000
+ 0x00000040 0x000c000c 0xa0f10000 0x00000000
+ 0x00000000 0x80000287 0xe8000000 0xff00ff00 >;
+ };
+
+ timing-51000000 {
+ clock-frequency = <51000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200008>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000002
+ 0x00000008 0x00000001 0x00000000 0x00000002
+ 0x0000000a 0x00000005 0x0000000b 0x00000000
+ 0x00000000 0x00000003 0x00000001 0x00000000
+ 0x00000005 0x00000005 0x00000004 0x00000009
+ 0x0000000b 0x00000181 0x00000000 0x00000060
+ 0x00000002 0x00000002 0x00000001 0x00000000
+ 0x00000007 0x0000000f 0x00000009 0x00000009
+ 0x00000004 0x00000002 0x00000000 0x00000004
+ 0x00000005 0x0000018e 0x00000006 0x00000004
+ 0x00000000 0x00000000 0x00004288 0x007800a4
+ 0x00008000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000002a0 0x0800211c 0x00000000
+ 0x77fff884 0x01f1f108 0x05057404 0x54000007
+ 0x08000168 0x08000000 0x00000802 0x00000000
+ 0x00000040 0x000c000c 0xa0f10000 0x00000000
+ 0x00000000 0x8000040b 0xe8000000 0xff00ff00 >;
+ };
+
+ timing-102000000 {
+ clock-frequency = <102000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200008>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000004
+ 0x00000010 0x00000003 0x00000001 0x00000002
+ 0x0000000a 0x00000005 0x0000000b 0x00000001
+ 0x00000001 0x00000003 0x00000001 0x00000000
+ 0x00000005 0x00000005 0x00000004 0x00000009
+ 0x0000000b 0x00000303 0x00000000 0x000000c0
+ 0x00000002 0x00000002 0x00000001 0x00000000
+ 0x00000007 0x0000000f 0x00000012 0x00000012
+ 0x00000004 0x00000004 0x00000000 0x00000004
+ 0x00000005 0x0000031c 0x00000006 0x00000004
+ 0x00000000 0x00000000 0x00004288 0x007800a4
+ 0x00008000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000002a0 0x0800211c 0x00000000
+ 0x77fff884 0x01f1f108 0x05057404 0x54000007
+ 0x08000168 0x08000000 0x00000802 0x00000000
+ 0x00000040 0x000c000c 0xa0f10000 0x00000000
+ 0x00000000 0x80000713 0xe8000000 0xff00ff00 >;
+ };
+
+ timing-204000000 {
+ clock-frequency = <204000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200008>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x0000000a
+ 0x00000020 0x00000007 0x00000002 0x00000002
+ 0x0000000a 0x00000005 0x0000000b 0x00000002
+ 0x00000002 0x00000003 0x00000001 0x00000000
+ 0x00000005 0x00000005 0x00000004 0x00000009
+ 0x0000000b 0x00000607 0x00000000 0x00000181
+ 0x00000002 0x00000002 0x00000001 0x00000000
+ 0x00000007 0x0000000f 0x00000023 0x00000023
+ 0x00000004 0x00000007 0x00000000 0x00000004
+ 0x00000005 0x00000638 0x00000006 0x00000006
+ 0x00000000 0x00000000 0x00004288 0x004400a4
+ 0x00008000 0x00080000 0x00080000 0x00080000
+ 0x00080000 0x00080000 0x00080000 0x00080000
+ 0x00080000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00080000 0x00080000 0x00080000
+ 0x00080000 0x000002a0 0x0800211c 0x00000000
+ 0x77fff884 0x01f1f108 0x05057404 0x54000007
+ 0x08000168 0x08000000 0x00000802 0x00020000
+ 0x00000100 0x000c000c 0xa0f10000 0x00000000
+ 0x00000000 0x80000d22 0xe8000000 0xff00ff00 >;
+ };
+
+ timing-333500000 {
+ clock-frequency = <333500000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100002>;
+ nvidia,emc-mode-2 = <0x80200000>;
+ nvidia,emc-mode-reset = <0x80000321>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+
+ nvidia,emc-configuration = < 0x0000000f
+ 0x00000034 0x0000000a 0x00000003 0x00000003
+ 0x00000008 0x00000002 0x00000009 0x00000003
+ 0x00000003 0x00000002 0x00000001 0x00000000
+ 0x00000004 0x00000006 0x00000004 0x0000000a
+ 0x0000000c 0x000009e9 0x00000000 0x0000027a
+ 0x00000001 0x00000008 0x00000001 0x00000000
+ 0x00000007 0x0000000e 0x00000039 0x00000200
+ 0x00000004 0x0000000a 0x00000000 0x00000004
+ 0x00000005 0x00000a2a 0x00000000 0x00000004
+ 0x00000000 0x00000000 0x00007088 0x002600a4
+ 0x00008000 0x0003c000 0x0003c000 0x0003c000
+ 0x0003c000 0x00014000 0x00014000 0x00014000
+ 0x00014000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00048000 0x00048000 0x00048000
+ 0x00048000 0x000002a0 0x0800013d 0x00000000
+ 0x77fff884 0x01f1f508 0x05057404 0x54000007
+ 0x080001e8 0x08000021 0x00000802 0x00020000
+ 0x00000100 0x018b000c 0xa0f10000 0x00000000
+ 0x00000000 0x800014d4 0xe8000000 0xff00ff89 >;
+ };
+
+ timing-667000000 {
+ clock-frequency = <667000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100002>;
+ nvidia,emc-mode-2 = <0x80200018>;
+ nvidia,emc-mode-reset = <0x80000b71>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x0000001f
+ 0x00000069 0x00000017 0x00000007 0x00000005
+ 0x0000000c 0x00000003 0x00000011 0x00000007
+ 0x00000007 0x00000002 0x00000001 0x00000000
+ 0x00000007 0x0000000a 0x00000009 0x0000000a
+ 0x00000011 0x00001412 0x00000000 0x00000504
+ 0x00000002 0x0000000e 0x00000001 0x00000000
+ 0x0000000c 0x00000016 0x00000072 0x00000200
+ 0x00000005 0x00000015 0x00000000 0x00000006
+ 0x00000007 0x00001453 0x0000000b 0x00000006
+ 0x00000000 0x00000000 0x00005088 0xf00b0191
+ 0x00008000 0x00000008 0x00000008 0x00000008
+ 0x00000008 0x0000000a 0x0000000a 0x0000000a
+ 0x0000000a 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x0000000a 0x0000000a 0x0000000a
+ 0x0000000a 0x000002a0 0x0800013d 0x22220000
+ 0x77fff884 0x01f1f501 0x07077404 0x54000000
+ 0x080001e8 0x0f000021 0x00000802 0x00020000
+ 0x00000100 0x0156000c 0xa0f10000 0x00000000
+ 0x00000000 0x800028a5 0xe8000000 0xff00ff49 >;
+ };
+ };
+
+ emc-timings-1 {
+ /* Hynix 1GB 667MHZ */
+ nvidia,ram-code = <1>;
+
+ timing-25500000 {
+ clock-frequency = <25500000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200008>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000001
+ 0x00000004 0x00000000 0x00000000 0x00000002
+ 0x0000000a 0x00000005 0x0000000b 0x00000000
+ 0x00000000 0x00000003 0x00000001 0x00000000
+ 0x00000005 0x00000005 0x00000004 0x00000009
+ 0x0000000b 0x000000c0 0x00000000 0x00000030
+ 0x00000002 0x00000002 0x00000001 0x00000000
+ 0x00000007 0x0000000f 0x00000005 0x00000005
+ 0x00000004 0x00000001 0x00000000 0x00000004
+ 0x00000005 0x000000c7 0x00000006 0x00000004
+ 0x00000000 0x00000000 0x00004288 0x007800a4
+ 0x00008000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000002a0 0x0800211c 0x00000000
+ 0x77fff884 0x01f1f108 0x05057404 0x54000007
+ 0x08000168 0x08000000 0x00000802 0x00000000
+ 0x00000040 0x000c000c 0xa0f10000 0x00000000
+ 0x00000000 0x80000287 0xe8000000 0xff00ff00 >;
+ };
+
+ timing-51000000 {
+ clock-frequency = <51000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200008>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000002
+ 0x00000008 0x00000001 0x00000000 0x00000002
+ 0x0000000a 0x00000005 0x0000000b 0x00000000
+ 0x00000000 0x00000003 0x00000001 0x00000000
+ 0x00000005 0x00000005 0x00000004 0x00000009
+ 0x0000000b 0x00000181 0x00000000 0x00000060
+ 0x00000002 0x00000002 0x00000001 0x00000000
+ 0x00000007 0x0000000f 0x00000009 0x00000009
+ 0x00000004 0x00000002 0x00000000 0x00000004
+ 0x00000005 0x0000018e 0x00000006 0x00000004
+ 0x00000000 0x00000000 0x00004288 0x007800a4
+ 0x00008000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000002a0 0x0800211c 0x00000000
+ 0x77fff884 0x01f1f108 0x05057404 0x54000007
+ 0x08000168 0x08000000 0x00000802 0x00000000
+ 0x00000040 0x000c000c 0xa0f10000 0x00000000
+ 0x00000000 0x8000040b 0xe8000000 0xff00ff00 >;
+ };
+
+ timing-102000000 {
+ clock-frequency = <102000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200008>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000005
+ 0x00000010 0x00000003 0x00000001 0x00000002
+ 0x0000000a 0x00000005 0x0000000b 0x00000001
+ 0x00000001 0x00000003 0x00000001 0x00000000
+ 0x00000005 0x00000005 0x00000004 0x00000009
+ 0x0000000b 0x00000303 0x00000000 0x000000c0
+ 0x00000002 0x00000002 0x00000001 0x00000000
+ 0x00000007 0x0000000f 0x00000012 0x00000012
+ 0x00000004 0x00000004 0x00000000 0x00000004
+ 0x00000005 0x0000031c 0x00000006 0x00000004
+ 0x00000000 0x00000000 0x00004288 0x007800a4
+ 0x00008000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000002a0 0x0800211c 0x00000000
+ 0x77fff884 0x01f1f108 0x05057404 0x54000007
+ 0x08000168 0x08000000 0x00000802 0x00000000
+ 0x00000040 0x000c000c 0xa0f10000 0x00000000
+ 0x00000000 0x80000713 0xe8000000 0xff00ff00 >;
+ };
+
+ timing-204000000 {
+ clock-frequency = <204000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200008>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x0000000a
+ 0x00000020 0x00000007 0x00000003 0x00000002
+ 0x0000000a 0x00000005 0x0000000b 0x00000002
+ 0x00000002 0x00000003 0x00000001 0x00000000
+ 0x00000005 0x00000005 0x00000004 0x00000009
+ 0x0000000b 0x00000607 0x00000000 0x00000181
+ 0x00000002 0x00000002 0x00000001 0x00000000
+ 0x00000007 0x0000000f 0x00000023 0x00000023
+ 0x00000004 0x00000007 0x00000000 0x00000004
+ 0x00000005 0x00000638 0x00000006 0x00000006
+ 0x00000000 0x00000000 0x00004288 0x004400a4
+ 0x00008000 0x00080000 0x00080000 0x00080000
+ 0x00080000 0x00080000 0x00080000 0x00080000
+ 0x00080000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00080000 0x00080000 0x00080000
+ 0x00080000 0x000002a0 0x0800211c 0x00000000
+ 0x77fff884 0x01f1f108 0x05057404 0x54000007
+ 0x08000168 0x08000000 0x00000802 0x00020000
+ 0x00000100 0x000c000c 0xa0f10000 0x00000000
+ 0x00000000 0x80000d22 0xe8000000 0xff00ff00 >;
+ };
+
+ timing-333500000 {
+ clock-frequency = <333500000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100002>;
+ nvidia,emc-mode-2 = <0x80200000>;
+ nvidia,emc-mode-reset = <0x80000321>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+
+ nvidia,emc-configuration = < 0x0000000f
+ 0x00000034 0x0000000b 0x00000003 0x00000003
+ 0x00000008 0x00000002 0x00000009 0x00000003
+ 0x00000003 0x00000002 0x00000001 0x00000000
+ 0x00000004 0x00000006 0x00000004 0x0000000a
+ 0x0000000c 0x000009e9 0x00000000 0x0000027a
+ 0x00000001 0x00000008 0x00000001 0x00000000
+ 0x00000007 0x0000000e 0x00000039 0x00000200
+ 0x00000004 0x0000000a 0x00000000 0x00000004
+ 0x00000005 0x00000a2a 0x00000000 0x00000004
+ 0x00000000 0x00000000 0x00007088 0x002600a4
+ 0x00008000 0x0003c000 0x0003c000 0x0003c000
+ 0x0003c000 0x00014000 0x00014000 0x00014000
+ 0x00014000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00048000 0x00048000 0x00048000
+ 0x00048000 0x000002a0 0x0800013d 0x00000000
+ 0x77fff884 0x01f1f508 0x05057404 0x54000007
+ 0x080001e8 0x08000021 0x00000802 0x00020000
+ 0x00000100 0x018b000c 0xa0f10000 0x00000000
+ 0x00000000 0x800014d4 0xe8000000 0xff00ff89 >;
+ };
+
+ timing-667000000 {
+ clock-frequency = <667000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100002>;
+ nvidia,emc-mode-2 = <0x80200018>;
+ nvidia,emc-mode-reset = <0x80000b71>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000020
+ 0x0000006a 0x00000018 0x00000008 0x00000005
+ 0x0000000c 0x00000003 0x00000011 0x00000007
+ 0x00000007 0x00000002 0x00000001 0x00000000
+ 0x00000007 0x0000000a 0x00000009 0x0000000a
+ 0x00000011 0x00001412 0x00000000 0x00000504
+ 0x00000002 0x0000000e 0x00000001 0x00000000
+ 0x0000000c 0x00000016 0x00000072 0x00000200
+ 0x00000005 0x00000015 0x00000000 0x00000006
+ 0x00000007 0x00001453 0x0000000b 0x00000006
+ 0x00000000 0x00000000 0x00005088 0xf00b0191
+ 0x00008000 0x00000008 0x00000008 0x00000008
+ 0x00000008 0x0000000a 0x0000000a 0x0000000a
+ 0x0000000a 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x0000000c 0x0000000c 0x0000000c
+ 0x0000000c 0x000002a0 0x0800013d 0x22220000
+ 0x77fff884 0x01f1f501 0x07077404 0x54000000
+ 0x080001e8 0x08000021 0x00000802 0x00020000
+ 0x00000100 0x0155000c 0xa0f10000 0x00000000
+ 0x00000000 0x800028a5 0xe8000000 0xff00ff49 >;
+ };
+ };
+
+ emc-timings-2 {
+ /* Micron 1GB 667MHZ */
+ nvidia,ram-code = <2>;
+
+ timing-25500000 {
+ clock-frequency = <25500000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200048>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000001
+ 0x00000004 0x00000000 0x00000000 0x00000002
+ 0x0000000a 0x00000005 0x0000000b 0x00000000
+ 0x00000000 0x00000003 0x00000001 0x00000000
+ 0x00000005 0x00000005 0x00000004 0x00000009
+ 0x0000000b 0x000000c0 0x00000000 0x00000030
+ 0x00000002 0x00000002 0x00000001 0x00000000
+ 0x00000007 0x0000000f 0x00000005 0x00000005
+ 0x00000004 0x00000001 0x00000000 0x00000004
+ 0x00000005 0x000000c7 0x00000006 0x00000004
+ 0x00000000 0x00000000 0x00004288 0x007800a4
+ 0x00008000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000002a0 0x0800211c 0x00000000
+ 0x77fff884 0x01f1f108 0x05057404 0x54000007
+ 0x08000168 0x08000000 0x00000802 0x00000000
+ 0x00000040 0x000c000c 0xa0f10000 0x00000000
+ 0x00000000 0x80000287 0xe8000000 0xff00ff00 >;
+ };
+
+ timing-51000000 {
+ clock-frequency = <51000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200008>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000002
+ 0x00000008 0x00000001 0x00000000 0x00000002
+ 0x0000000a 0x00000005 0x0000000b 0x00000000
+ 0x00000000 0x00000003 0x00000001 0x00000000
+ 0x00000005 0x00000005 0x00000004 0x00000009
+ 0x0000000b 0x00000181 0x00000000 0x00000060
+ 0x00000002 0x00000002 0x00000001 0x00000000
+ 0x00000007 0x0000000f 0x00000009 0x00000009
+ 0x00000004 0x00000002 0x00000000 0x00000004
+ 0x00000005 0x0000018e 0x00000006 0x00000004
+ 0x00000000 0x00000000 0x00004288 0x007800a4
+ 0x00008000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000002a0 0x0800211c 0x00000000
+ 0x77fff884 0x01f1f108 0x05057404 0x54000007
+ 0x08000168 0x08000000 0x00000802 0x00000000
+ 0x00000040 0x000c000c 0xa0f10000 0x00000000
+ 0x00000000 0x8000040b 0xe8000000 0xff00ff00 >;
+ };
+
+ timing-102000000 {
+ clock-frequency = <102000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200048>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000004
+ 0x00000010 0x00000003 0x00000001 0x00000002
+ 0x0000000a 0x00000005 0x0000000b 0x00000001
+ 0x00000001 0x00000003 0x00000001 0x00000000
+ 0x00000005 0x00000005 0x00000004 0x0000000a
+ 0x0000000b 0x00000303 0x00000000 0x000000c0
+ 0x00000002 0x00000002 0x00000001 0x00000000
+ 0x00000007 0x0000000f 0x00000012 0x00000012
+ 0x00000004 0x00000004 0x00000000 0x00000004
+ 0x00000005 0x0000031c 0x00000006 0x00000004
+ 0x00000000 0x00000000 0x00004288 0x007800a4
+ 0x00008000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000002a0 0x0800211c 0x00000000
+ 0x77fff884 0x01f1f108 0x05057404 0x54000007
+ 0x08000168 0x08000000 0x00000802 0x00000000
+ 0x00000040 0x000c000c 0xa0f10000 0x00000000
+ 0x00000000 0x80000713 0xd8000000 0xff00ff00 >;
+ };
+
+ timing-204000000 {
+ clock-frequency = <204000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200048>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000009
+ 0x00000020 0x00000007 0x00000002 0x00000002
+ 0x0000000a 0x00000005 0x0000000b 0x00000002
+ 0x00000002 0x00000003 0x00000001 0x00000000
+ 0x00000005 0x00000005 0x00000004 0x0000000a
+ 0x0000000b 0x00000607 0x00000000 0x00000181
+ 0x00000002 0x00000002 0x00000001 0x00000000
+ 0x00000007 0x0000000f 0x00000023 0x00000023
+ 0x00000004 0x00000007 0x00000000 0x00000004
+ 0x00000005 0x00000638 0x00000006 0x00000006
+ 0x00000000 0x00000000 0x00004288 0x004400a4
+ 0x00008000 0x00080000 0x00080000 0x00080000
+ 0x00080000 0x00080000 0x00080000 0x00080000
+ 0x00080000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00080000 0x00080000 0x00080000
+ 0x00080000 0x000002a0 0x0800211c 0x00000000
+ 0x77fff884 0x01f1f108 0x05057404 0x54000007
+ 0x08000168 0x08000000 0x00000802 0x00020000
+ 0x00000100 0x000c000c 0xa0f10000 0x00000000
+ 0x00000000 0x80000d22 0xd8000000 0xff00ff00 >;
+ };
+
+ timing-333500000 {
+ clock-frequency = <333500000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100002>;
+ nvidia,emc-mode-2 = <0x80200040>;
+ nvidia,emc-mode-reset = <0x80000321>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+
+ nvidia,emc-configuration = < 0x0000000f
+ 0x00000034 0x0000000a 0x00000003 0x00000003
+ 0x00000008 0x00000002 0x00000009 0x00000003
+ 0x00000003 0x00000002 0x00000001 0x00000000
+ 0x00000004 0x00000006 0x00000004 0x0000000a
+ 0x0000000c 0x000009e9 0x00000000 0x0000027a
+ 0x00000001 0x00000008 0x00000001 0x00000000
+ 0x00000007 0x0000000e 0x00000039 0x00000200
+ 0x00000004 0x0000000a 0x00000000 0x00000004
+ 0x00000005 0x00000a2a 0x00000000 0x00000004
+ 0x00000000 0x00000000 0x00007088 0x002600a4
+ 0x00008000 0x0003c000 0x0003c000 0x0003c000
+ 0x0003c000 0x00014000 0x00014000 0x00014000
+ 0x00014000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00048000 0x00048000 0x00048000
+ 0x00048000 0x000002a0 0x0800013d 0x00000000
+ 0x77fff884 0x01f1f508 0x05057404 0x54000007
+ 0x080001e8 0x08000021 0x00000802 0x00020000
+ 0x00000100 0x018b000c 0xa0f10000 0x00000000
+ 0x00000000 0x800014d4 0xe8000000 0xff00ff89 >;
+ };
+
+ timing-667000000 {
+ clock-frequency = <667000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100002>;
+ nvidia,emc-mode-2 = <0x80200058>;
+ nvidia,emc-mode-reset = <0x80000b71>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x0000001f
+ 0x00000069 0x00000016 0x00000007 0x00000005
+ 0x0000000c 0x00000003 0x00000011 0x00000008
+ 0x00000008 0x00000002 0x00000001 0x00000000
+ 0x00000007 0x0000000a 0x00000009 0x0000000b
+ 0x00000011 0x00001412 0x00000000 0x00000504
+ 0x00000002 0x0000000e 0x00000001 0x00000000
+ 0x0000000c 0x00000016 0x00000072 0x00000200
+ 0x00000005 0x00000015 0x00000000 0x00000006
+ 0x00000007 0x00001453 0x0000000b 0x00000006
+ 0x00000000 0x00000000 0x00005088 0xf00b0191
+ 0x00008000 0x0000000a 0x0000000a 0x0000000a
+ 0x0000000a 0x0000000a 0x0000000a 0x0000000a
+ 0x0000000a 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x0000000c 0x0000000c 0x0000000c
+ 0x0000000c 0x000002a0 0x0800013d 0x22220000
+ 0x77fff884 0x01f1f501 0x07077404 0x54000000
+ 0x080001e8 0x08000021 0x00000802 0x00020000
+ 0x00000100 0x0156000c 0xa0f10000 0x00000000
+ 0x00000000 0x800028a5 0xf8000000 0xff00ff49 >;
+ };
+ };
+ };
+
+ display-panel {
+ compatible = "innolux,g101ice-l01";
+ };
+
+ sound {
+ compatible = "asus,tegra-audio-wm8903-tf300t",
+ "nvidia,tegra-audio-wm8903";
+ nvidia,model = "Asus Transformer Pad TF300T WM8903";
+
+ nvidia,audio-routing =
+ "Headphone Jack", "HPOUTR",
+ "Headphone Jack", "HPOUTL",
+ "Int Spk", "ROP",
+ "Int Spk", "RON",
+ "Int Spk", "LOP",
+ "Int Spk", "LON",
+ "IN1L", "Mic Jack",
+ "IN2L", "Mic Jack",
+ "DMICDAT", "Int Mic";
+
+ nvidia,audio-codec = <&wm8903>;
+ nvidia,spkr-en-gpios = <&wm8903 2 GPIO_ACTIVE_HIGH>;
+ nvidia,headset;
+ };
+};
+
+&emc_icc_dvfs_opp_table {
+ /delete-node/ opp-750000000-1300;
+ /delete-node/ opp-800000000-1300;
+ /delete-node/ opp-900000000-1350;
+};
+
+&emc_bw_dfs_opp_table {
+ /delete-node/ opp-750000000;
+ /delete-node/ opp-800000000;
+ /delete-node/ opp-900000000;
+};
diff --git a/arch/arm/boot/dts/tegra30-asus-tf300tg.dts b/arch/arm/boot/dts/tegra30-asus-tf300tg.dts
new file mode 100644
index 0000000..96345f8
--- /dev/null
+++ b/arch/arm/boot/dts/tegra30-asus-tf300tg.dts
@@ -0,0 +1,1087 @@
+// SPDX-License-Identifier: GPL-2.0
+/dts-v1/;
+
+#include "tegra30-asus-transformer-common.dtsi"
+#include "tegra30-asus-lvds-display.dtsi"
+
+/ {
+ model = "Asus Transformer Pad 3G TF300TG";
+ compatible = "asus,tf300tg", "nvidia,tegra30";
+
+ gpio@6000d000 {
+ tf300tg-init-hog {
+ gpio-hog;
+ gpios = <TEGRA_GPIO(C, 6) GPIO_ACTIVE_HIGH>,
+ <TEGRA_GPIO(D, 2) GPIO_ACTIVE_HIGH>,
+ <TEGRA_GPIO(P, 1) GPIO_ACTIVE_HIGH>,
+ <TEGRA_GPIO(X, 5) GPIO_ACTIVE_HIGH>,
+ <TEGRA_GPIO(W, 3) GPIO_ACTIVE_HIGH>,
+ <TEGRA_GPIO(R, 3) GPIO_ACTIVE_HIGH>,
+ <TEGRA_GPIO(U, 5) GPIO_ACTIVE_HIGH>,
+ <TEGRA_GPIO(X, 7) GPIO_ACTIVE_HIGH>,
+ <TEGRA_GPIO(X, 0) GPIO_ACTIVE_HIGH>,
+ <TEGRA_GPIO(Y, 2) GPIO_ACTIVE_HIGH>,
+ <TEGRA_GPIO(Y, 3) GPIO_ACTIVE_HIGH>,
+ <TEGRA_GPIO(EE, 1) GPIO_ACTIVE_HIGH>,
+ <TEGRA_GPIO(R, 7) GPIO_ACTIVE_HIGH>,
+ <TEGRA_GPIO(U, 3) GPIO_ACTIVE_HIGH>;
+ output-low;
+ };
+ };
+
+ pinmux@70000868 {
+ state_default: pinmux {
+ pbb3 {
+ nvidia,pins = "pbb3";
+ nvidia,function = "vgp3";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ pbb7 {
+ nvidia,pins = "pbb7";
+ nvidia,function = "i2s4";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ gmi_cs4_n_pk2 {
+ nvidia,pins = "gmi_cs4_n_pk2";
+ nvidia,function = "gmi";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ lcd_pwr2_pc6 {
+ nvidia,pins = "lcd_pwr2_pc6",
+ "lcd_dc1_pd2";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_DOWN>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ kb_row7_pr7 {
+ nvidia,pins = "kb_row7_pr7";
+ nvidia,function = "kbc";
+ nvidia,pull = <TEGRA_PIN_PULL_DOWN>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ spi2_cs2_n_pw3 {
+ nvidia,pins = "spi2_cs2_n_pw3";
+ nvidia,function = "spi2";
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ };
+
+ dap3_din_pp1 {
+ nvidia,pins = "dap3_din_pp1";
+ nvidia,function = "i2s2";
+ nvidia,pull = <TEGRA_PIN_PULL_DOWN>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ spi1_sck_px5 {
+ nvidia,pins = "spi1_sck_px5";
+ nvidia,function = "spi1";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ pu5 {
+ nvidia,pins = "pu5";
+ nvidia,function = "pwm2";
+ nvidia,pull = <TEGRA_PIN_PULL_DOWN>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ spi2_mosi_px0 {
+ nvidia,pins = "spi2_mosi_px0";
+ nvidia,function = "spi2";
+ };
+
+ spi1_miso_px7 {
+ nvidia,pins = "spi1_miso_px7";
+ nvidia,function = "spi1";
+ nvidia,pull = <TEGRA_PIN_PULL_DOWN>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ clk3_req_pee1 {
+ nvidia,pins = "clk3_req_pee1";
+ nvidia,function = "dev3";
+ nvidia,pull = <TEGRA_PIN_PULL_DOWN>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ ulpi_nxt_py2 {
+ nvidia,pins = "ulpi_nxt_py2";
+ nvidia,function = "uartd";
+ nvidia,pull = <TEGRA_PIN_PULL_DOWN>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ ulpi_stp_py3 {
+ nvidia,pins = "ulpi_stp_py3";
+ nvidia,function = "uartd";
+ nvidia,pull = <TEGRA_PIN_PULL_DOWN>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ pu3 {
+ nvidia,pins = "pu3";
+ nvidia,function = "rsvd1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ dap1_din_pn1 {
+ nvidia,pins = "dap1_din_pn1";
+ nvidia,function = "i2s0";
+ nvidia,pull = <TEGRA_PIN_PULL_DOWN>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+ };
+ };
+
+ serial@70006200 {
+ /* Azurewave AW-NH615 BCM4329B1 */
+ bluetooth {
+ compatible = "brcm,bcm4329-bt";
+ };
+ };
+
+ i2c@7000c400 {
+ /* Elantech EKTH1036 touchscreen */
+ touchscreen@10 {
+ compatible = "elan,ektf3624";
+ reg = <0x10>;
+
+ interrupt-parent = <&gpio>;
+ interrupts = <TEGRA_GPIO(H, 4) IRQ_TYPE_EDGE_FALLING>;
+ reset-gpios = <&gpio TEGRA_GPIO(H, 6) GPIO_ACTIVE_LOW>;
+
+ vcc33-supply = <&vdd_3v3_sys>;
+ vccio-supply = <&vdd_3v3_sys>;
+
+ touchscreen-size-x = <2240>;
+ touchscreen-size-y = <1408>;
+ touchscreen-inverted-y;
+ };
+ };
+
+ i2c@7000c500 {
+ clock-frequency = <400000>;
+
+ magnetometer@e {
+ mount-matrix = "1", "0", "0",
+ "0", "-1", "0",
+ "0", "0", "-1";
+ };
+
+ gyroscope@68 {
+ mount-matrix = "-1", "0", "0",
+ "0", "1", "0",
+ "0", "0", "-1";
+
+ /* External I2C interface */
+ i2c-gate {
+ accelerometer@f {
+ mount-matrix = "0", "-1", "0",
+ "-1", "0", "0",
+ "0", "0", "1";
+ };
+ };
+ };
+ };
+
+ i2c@7000d000 {
+ /* Realtek ALC5631 audio codec */
+ rt5631: audio-codec@1a {
+ compatible = "realtek,rt5631";
+ reg = <0x1a>;
+ };
+ };
+
+ memory-controller@7000f000 {
+ emc-timings-0 {
+ /* Elpida 1GB 667MHZ */
+ nvidia,ram-code = <0>;
+
+ timing-25500000 {
+ clock-frequency = <25500000>;
+
+ nvidia,emem-configuration = < 0x00020001 0xc0000020
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000001 0x00000001 0x00000003 0x00000008
+ 0x00000002 0x00000001 0x00000002 0x00000006
+ 0x06020102 0x000a0502 0x74830303 0x001f0000 >;
+ };
+
+ timing-51000000 {
+ clock-frequency = <51000000>;
+
+ nvidia,emem-configuration = < 0x00010001 0xc0000020
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000001 0x00000001 0x00000003 0x00000008
+ 0x00000002 0x00000001 0x00000002 0x00000006
+ 0x06020102 0x000a0502 0x73430303 0x001f0000 >;
+ };
+
+ timing-102000000 {
+ clock-frequency = <102000000>;
+
+ nvidia,emem-configuration = < 0x00000001 0xc0000030
+ 0x00000001 0x00000001 0x00000003 0x00000000
+ 0x00000001 0x00000001 0x00000003 0x00000008
+ 0x00000002 0x00000001 0x00000002 0x00000006
+ 0x06020102 0x000a0503 0x72830504 0x001f0000 >;
+ };
+
+ timing-204000000 {
+ clock-frequency = <204000000>;
+
+ nvidia,emem-configuration = < 0x00000003 0xc0000025
+ 0x00000001 0x00000001 0x00000005 0x00000002
+ 0x00000003 0x00000001 0x00000003 0x00000008
+ 0x00000002 0x00000001 0x00000002 0x00000006
+ 0x06020102 0x000a0505 0x72440a06 0x001f0000 >;
+ };
+
+ timing-333500000 {
+ clock-frequency = <333500000>;
+
+ nvidia,emem-configuration = < 0x00000005 0xc000003d
+ 0x00000001 0x00000002 0x00000008 0x00000004
+ 0x00000004 0x00000001 0x00000002 0x00000007
+ 0x00000002 0x00000002 0x00000003 0x00000006
+ 0x06030202 0x000b0608 0x70850f09 0x001f0000 >;
+ };
+
+ timing-667000000 {
+ clock-frequency = <667000000>;
+
+ nvidia,emem-configuration = < 0x0000000a 0xc0000079
+ 0x00000003 0x00000004 0x00000010 0x0000000b
+ 0x0000000a 0x00000001 0x00000003 0x0000000b
+ 0x00000002 0x00000002 0x00000004 0x00000008
+ 0x08040202 0x00130b10 0x70ea1f11 0x001f0000 >;
+ };
+ };
+
+ emc-timings-1 {
+ /* Hynix 1GB 667MHZ */
+ nvidia,ram-code = <1>;
+
+ timing-25500000 {
+ clock-frequency = <25500000>;
+
+ nvidia,emem-configuration = < 0x00020001 0xc0000020
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000001 0x00000001 0x00000003 0x00000008
+ 0x00000002 0x00000001 0x00000002 0x00000006
+ 0x06020102 0x000a0502 0x74830303 0x001f0000 >;
+ };
+
+ timing-51000000 {
+ clock-frequency = <51000000>;
+
+ nvidia,emem-configuration = < 0x00010001 0xc0000020
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000001 0x00000001 0x00000003 0x00000008
+ 0x00000002 0x00000001 0x00000002 0x00000006
+ 0x06020102 0x000a0502 0x73430303 0x001f0000 >;
+ };
+
+ timing-102000000 {
+ clock-frequency = <102000000>;
+
+ nvidia,emem-configuration = < 0x00000001 0xc0000030
+ 0x00000001 0x00000001 0x00000003 0x00000000
+ 0x00000001 0x00000001 0x00000003 0x00000008
+ 0x00000002 0x00000001 0x00000002 0x00000006
+ 0x06020102 0x000a0503 0x72830504 0x001f0000 >;
+ };
+
+ timing-204000000 {
+ clock-frequency = <204000000>;
+
+ nvidia,emem-configuration = < 0x00000003 0xc0000025
+ 0x00000001 0x00000001 0x00000005 0x00000002
+ 0x00000003 0x00000001 0x00000003 0x00000008
+ 0x00000002 0x00000001 0x00000002 0x00000006
+ 0x06020102 0x000a0505 0x72440a06 0x001f0000 >;
+ };
+
+ timing-333500000 {
+ clock-frequency = <333500000>;
+
+ nvidia,emem-configuration = < 0x00000005 0xc000003d
+ 0x00000001 0x00000002 0x00000008 0x00000004
+ 0x00000004 0x00000001 0x00000002 0x00000007
+ 0x00000002 0x00000002 0x00000003 0x00000006
+ 0x06030202 0x000b0608 0x70850f09 0x001f0000 >;
+ };
+
+ timing-667000000 {
+ clock-frequency = <667000000>;
+
+ nvidia,emem-configuration = < 0x0000000a 0xc0000079
+ 0x00000003 0x00000004 0x00000010 0x0000000b
+ 0x0000000a 0x00000001 0x00000003 0x0000000b
+ 0x00000002 0x00000002 0x00000004 0x00000008
+ 0x08040202 0x00130b10 0x70ea1f11 0x001f0000 >;
+ };
+ };
+
+ emc-timings-2 {
+ /* Micron 1GB 667MHZ */
+ nvidia,ram-code = <2>;
+
+ timing-25500000 {
+ clock-frequency = <25500000>;
+
+ nvidia,emem-configuration = < 0x00020001 0xc0000020
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000001 0x00000001 0x00000003 0x00000008
+ 0x00000002 0x00000001 0x00000002 0x00000006
+ 0x06020102 0x000a0502 0x74830303 0x001f0000 >;
+ };
+
+ timing-51000000 {
+ clock-frequency = <51000000>;
+
+ nvidia,emem-configuration = < 0x00010001 0xc0000020
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000001 0x00000001 0x00000003 0x00000008
+ 0x00000002 0x00000001 0x00000002 0x00000006
+ 0x06020102 0x000a0502 0x73430303 0x001f0000 >;
+ };
+
+ timing-102000000 {
+ clock-frequency = <102000000>;
+
+ nvidia,emem-configuration = < 0x00000001 0xc0000030
+ 0x00000001 0x00000001 0x00000003 0x00000000
+ 0x00000001 0x00000001 0x00000003 0x00000008
+ 0x00000002 0x00000001 0x00000002 0x00000006
+ 0x06020102 0x000a0503 0x72830504 0x001f0000 >;
+ };
+
+ timing-204000000 {
+ clock-frequency = <204000000>;
+
+ nvidia,emem-configuration = < 0x00000003 0xc0000025
+ 0x00000001 0x00000001 0x00000005 0x00000002
+ 0x00000003 0x00000001 0x00000003 0x00000008
+ 0x00000002 0x00000001 0x00000002 0x00000006
+ 0x06020102 0x000a0505 0x72440a06 0x001f0000 >;
+ };
+
+ timing-333500000 {
+ clock-frequency = <333500000>;
+
+ nvidia,emem-configuration = < 0x00000005 0x8000003d
+ 0x00000001 0x00000002 0x00000008 0x00000004
+ 0x00000004 0x00000001 0x00000002 0x00000007
+ 0x00000002 0x00000002 0x00000003 0x00000006
+ 0x06030202 0x000b0608 0x70850f09 0x001f0000 >;
+ };
+
+ timing-667000000 {
+ clock-frequency = <667000000>;
+
+ nvidia,emem-configuration = < 0x0000000a 0x80000079
+ 0x00000003 0x00000004 0x00000010 0x0000000a
+ 0x0000000a 0x00000001 0x00000003 0x0000000b
+ 0x00000002 0x00000002 0x00000004 0x00000008
+ 0x08040202 0x00130b10 0x70ea1f11 0x001f0000 >;
+ };
+ };
+ };
+
+ memory-controller@7000f400 {
+ emc-timings-0 {
+ /* Elpida 1GB 667MHZ */
+ nvidia,ram-code = <0>;
+
+ timing-25500000 {
+ clock-frequency = <25500000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200048>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000001
+ 0x00000004 0x00000000 0x00000000 0x00000002
+ 0x0000000a 0x00000005 0x0000000b 0x00000000
+ 0x00000000 0x00000003 0x00000001 0x00000000
+ 0x00000005 0x00000005 0x00000004 0x0000000a
+ 0x0000000b 0x000000c0 0x00000000 0x00000030
+ 0x00000002 0x00000002 0x00000001 0x00000000
+ 0x00000007 0x0000000f 0x00000005 0x00000005
+ 0x00000004 0x00000001 0x00000000 0x00000004
+ 0x00000005 0x000000c7 0x00000006 0x00000004
+ 0x00000000 0x00000000 0x00004288 0x007800a4
+ 0x00008000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000002a0 0x0800211c 0x00000000
+ 0x77fff884 0x01f1f108 0x05057404 0x54000007
+ 0x08000168 0x08000000 0x00000802 0x00000000
+ 0x00000040 0x000c000c 0xa0f10000 0x00000000
+ 0x00000000 0x80000287 0xe8000000 0xff00ff00 >;
+ };
+
+ timing-51000000 {
+ clock-frequency = <51000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200048>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000002
+ 0x00000008 0x00000001 0x00000000 0x00000002
+ 0x0000000a 0x00000005 0x0000000b 0x00000000
+ 0x00000000 0x00000003 0x00000001 0x00000000
+ 0x00000005 0x00000005 0x00000004 0x0000000a
+ 0x0000000b 0x00000181 0x00000000 0x00000060
+ 0x00000002 0x00000002 0x00000001 0x00000000
+ 0x00000007 0x0000000f 0x00000009 0x00000009
+ 0x00000004 0x00000002 0x00000000 0x00000004
+ 0x00000005 0x0000018e 0x00000006 0x00000004
+ 0x00000000 0x00000000 0x00004288 0x007800a4
+ 0x00008000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000002a0 0x0800211c 0x00000000
+ 0x77fff884 0x01f1f108 0x05057404 0x54000007
+ 0x08000168 0x08000000 0x00000802 0x00000000
+ 0x00000040 0x000c000c 0xa0f10000 0x00000000
+ 0x00000000 0x8000040b 0xe8000000 0xff00ff00 >;
+ };
+
+ timing-102000000 {
+ clock-frequency = <102000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200048>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000005
+ 0x00000010 0x00000003 0x00000001 0x00000002
+ 0x0000000a 0x00000005 0x0000000b 0x00000001
+ 0x00000001 0x00000003 0x00000001 0x00000000
+ 0x00000005 0x00000005 0x00000004 0x0000000a
+ 0x0000000b 0x00000303 0x00000000 0x000000c0
+ 0x00000002 0x00000002 0x00000001 0x00000000
+ 0x00000007 0x0000000f 0x00000012 0x00000012
+ 0x00000004 0x00000004 0x00000000 0x00000004
+ 0x00000005 0x0000031c 0x00000006 0x00000004
+ 0x00000000 0x00000000 0x00004288 0x007800a4
+ 0x00008000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000002a0 0x0800211c 0x00000000
+ 0x77fff884 0x01f1f108 0x05057404 0x54000007
+ 0x08000168 0x08000000 0x00000802 0x00000000
+ 0x00000040 0x000c000c 0xa0f10000 0x00000000
+ 0x00000000 0x80000713 0xe8000000 0xff00ff00 >;
+ };
+
+ timing-204000000 {
+ clock-frequency = <204000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200048>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x0000000a
+ 0x00000020 0x00000007 0x00000002 0x00000002
+ 0x0000000a 0x00000005 0x0000000b 0x00000002
+ 0x00000002 0x00000003 0x00000001 0x00000000
+ 0x00000005 0x00000006 0x00000004 0x0000000a
+ 0x0000000b 0x00000607 0x00000000 0x00000181
+ 0x00000002 0x00000002 0x00000001 0x00000000
+ 0x00000007 0x0000000f 0x00000023 0x00000023
+ 0x00000004 0x00000007 0x00000000 0x00000004
+ 0x00000005 0x00000638 0x00000007 0x00000004
+ 0x00000000 0x00000000 0x00004288 0x004400a4
+ 0x00008000 0x00080000 0x00080000 0x00080000
+ 0x00080000 0x00080000 0x00080000 0x00080000
+ 0x00080000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00080000 0x00080000 0x00080000
+ 0x00080000 0x000002a0 0x0800211c 0x00000000
+ 0x77fff884 0x01f1f108 0x05057404 0x54000007
+ 0x08000168 0x08000000 0x00000802 0x00020000
+ 0x00000100 0x000c000c 0xa0f10000 0x00000000
+ 0x00000000 0x80000d22 0xe8000000 0xff00ff00 >;
+ };
+
+ timing-333500000 {
+ clock-frequency = <333500000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100002>;
+ nvidia,emc-mode-2 = <0x80200040>;
+ nvidia,emc-mode-reset = <0x80000321>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+
+ nvidia,emc-configuration = < 0x0000000f
+ 0x00000034 0x0000000a 0x00000003 0x00000003
+ 0x00000008 0x00000002 0x00000009 0x00000003
+ 0x00000003 0x00000002 0x00000001 0x00000000
+ 0x00000004 0x00000006 0x00000004 0x0000000a
+ 0x0000000c 0x000009e9 0x00000000 0x0000027a
+ 0x00000001 0x00000008 0x00000001 0x00000000
+ 0x00000007 0x0000000e 0x00000039 0x00000200
+ 0x00000004 0x0000000a 0x00000000 0x00000004
+ 0x00000005 0x00000a2a 0x00000000 0x00000004
+ 0x00000000 0x00000000 0x00007088 0x002600a4
+ 0x00008000 0x0003c000 0x0003c000 0x0003c000
+ 0x0003c000 0x00014000 0x00014000 0x00014000
+ 0x00014000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00050000 0x00050000 0x00050000
+ 0x00050000 0x000002a0 0x0800013d 0x00000000
+ 0x77fff884 0x01f1f508 0x05057404 0x54000007
+ 0x080001e8 0x08000021 0x00000802 0x00020000
+ 0x00000100 0x018b000c 0xa0f10000 0x00000000
+ 0x00000000 0x800014d4 0xe8000000 0xff00ff89 >;
+ };
+
+ timing-667000000 {
+ clock-frequency = <667000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100002>;
+ nvidia,emc-mode-2 = <0x80200058>;
+ nvidia,emc-mode-reset = <0x80000b71>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x0000001f
+ 0x00000069 0x00000017 0x00000007 0x00000005
+ 0x0000000c 0x00000003 0x00000011 0x00000007
+ 0x00000007 0x00000002 0x00000001 0x00000000
+ 0x00000007 0x0000000b 0x00000009 0x0000000b
+ 0x00000011 0x00001412 0x00000000 0x00000504
+ 0x00000002 0x0000000e 0x00000001 0x00000000
+ 0x0000000c 0x00000016 0x00000072 0x00000200
+ 0x00000005 0x00000015 0x00000000 0x00000006
+ 0x00000007 0x00001453 0x0000000c 0x00000004
+ 0x00000000 0x00000000 0x00005088 0xf00b0191
+ 0x00008000 0x0000000c 0x0000000c 0x0000000c
+ 0x0000000c 0x0000000a 0x0000000a 0x0000000a
+ 0x0000000a 0x00018000 0x00018000 0x00018000
+ 0x00018000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x0000000c 0x0000000c 0x0000000c
+ 0x0000000c 0x000002a0 0x0a00013d 0x22220000
+ 0x77fff884 0x01f1f501 0x07077404 0x54000000
+ 0x080001e8 0x0a000021 0x00000802 0x00020000
+ 0x00000100 0x0156000c 0xa0f10000 0x00000000
+ 0x00000000 0x800028a5 0xe8000000 0xff00ff49 >;
+ };
+ };
+
+ emc-timings-1 {
+ /* Hynix 1GB 667MHZ */
+ nvidia,ram-code = <1>;
+
+ timing-25500000 {
+ clock-frequency = <25500000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200048>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000001
+ 0x00000004 0x00000000 0x00000000 0x00000002
+ 0x0000000a 0x00000005 0x0000000b 0x00000000
+ 0x00000000 0x00000003 0x00000001 0x00000000
+ 0x00000005 0x00000005 0x00000004 0x0000000a
+ 0x0000000b 0x000000c0 0x00000000 0x00000030
+ 0x00000002 0x00000002 0x00000001 0x00000000
+ 0x00000007 0x0000000f 0x00000005 0x00000005
+ 0x00000004 0x00000001 0x00000000 0x00000004
+ 0x00000005 0x000000c7 0x00000006 0x00000004
+ 0x00000000 0x00000000 0x00004288 0x007800a4
+ 0x00008000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000002a0 0x0800211c 0x00000000
+ 0x77fff884 0x01f1f108 0x05057404 0x54000007
+ 0x08000168 0x08000000 0x00000802 0x00000000
+ 0x00000040 0x000c000c 0xa0f10000 0x00000000
+ 0x00000000 0x80000287 0xe8000000 0xff00ff00 >;
+ };
+
+ timing-51000000 {
+ clock-frequency = <51000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200048>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000002
+ 0x00000008 0x00000001 0x00000000 0x00000002
+ 0x0000000a 0x00000005 0x0000000b 0x00000000
+ 0x00000000 0x00000003 0x00000001 0x00000000
+ 0x00000005 0x00000005 0x00000004 0x0000000a
+ 0x0000000b 0x00000181 0x00000000 0x00000060
+ 0x00000002 0x00000002 0x00000001 0x00000000
+ 0x00000007 0x0000000f 0x00000009 0x00000009
+ 0x00000004 0x00000002 0x00000000 0x00000004
+ 0x00000005 0x0000018e 0x00000006 0x00000004
+ 0x00000000 0x00000000 0x00004288 0x007800a4
+ 0x00008000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000002a0 0x0800211c 0x00000000
+ 0x77fff884 0x01f1f108 0x05057404 0x54000007
+ 0x08000168 0x08000000 0x00000802 0x00000000
+ 0x00000040 0x000c000c 0xa0f10000 0x00000000
+ 0x00000000 0x8000040b 0xe8000000 0xff00ff00 >;
+ };
+
+ timing-102000000 {
+ clock-frequency = <102000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200048>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000005
+ 0x00000010 0x00000003 0x00000001 0x00000002
+ 0x0000000a 0x00000005 0x0000000b 0x00000001
+ 0x00000001 0x00000003 0x00000001 0x00000000
+ 0x00000005 0x00000005 0x00000004 0x0000000a
+ 0x0000000b 0x00000303 0x00000000 0x000000c0
+ 0x00000002 0x00000002 0x00000001 0x00000000
+ 0x00000007 0x0000000f 0x00000012 0x00000012
+ 0x00000004 0x00000004 0x00000000 0x00000004
+ 0x00000005 0x0000031c 0x00000006 0x00000004
+ 0x00000000 0x00000000 0x00004288 0x007800a4
+ 0x00008000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000002a0 0x0800211c 0x00000000
+ 0x77fff884 0x01f1f108 0x05057404 0x54000007
+ 0x08000168 0x08000000 0x00000802 0x00000000
+ 0x00000040 0x000c000c 0xa0f10000 0x00000000
+ 0x00000000 0x80000713 0xe8000000 0xff00ff00 >;
+ };
+
+ timing-204000000 {
+ clock-frequency = <204000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200048>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x0000000a
+ 0x00000020 0x00000007 0x00000002 0x00000002
+ 0x0000000a 0x00000005 0x0000000b 0x00000002
+ 0x00000002 0x00000003 0x00000001 0x00000000
+ 0x00000005 0x00000006 0x00000004 0x0000000a
+ 0x0000000b 0x00000607 0x00000000 0x00000181
+ 0x00000002 0x00000002 0x00000001 0x00000000
+ 0x00000007 0x0000000f 0x00000023 0x00000023
+ 0x00000004 0x00000007 0x00000000 0x00000004
+ 0x00000005 0x00000638 0x00000007 0x00000004
+ 0x00000000 0x00000000 0x00004288 0x004400a4
+ 0x00008000 0x00080000 0x00080000 0x00080000
+ 0x00080000 0x00080000 0x00080000 0x00080000
+ 0x00080000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00080000 0x00080000 0x00080000
+ 0x00080000 0x000002a0 0x0800211c 0x00000000
+ 0x77fff884 0x01f1f108 0x05057404 0x54000007
+ 0x08000168 0x08000000 0x00000802 0x00020000
+ 0x00000100 0x000c000c 0xa0f10000 0x00000000
+ 0x00000000 0x80000d22 0xe8000000 0xff00ff00 >;
+ };
+
+ timing-333500000 {
+ clock-frequency = <333500000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100002>;
+ nvidia,emc-mode-2 = <0x80200040>;
+ nvidia,emc-mode-reset = <0x80000321>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+
+ nvidia,emc-configuration = < 0x0000000f
+ 0x00000034 0x0000000a 0x00000003 0x00000003
+ 0x00000008 0x00000002 0x00000009 0x00000003
+ 0x00000003 0x00000002 0x00000001 0x00000000
+ 0x00000004 0x00000006 0x00000004 0x0000000a
+ 0x0000000c 0x000009e9 0x00000000 0x0000027a
+ 0x00000001 0x00000008 0x00000001 0x00000000
+ 0x00000007 0x0000000e 0x00000039 0x00000200
+ 0x00000004 0x0000000a 0x00000000 0x00000004
+ 0x00000005 0x00000a2a 0x00000000 0x00000004
+ 0x00000000 0x00000000 0x00007088 0x002600a4
+ 0x00008000 0x0003c000 0x0003c000 0x0003c000
+ 0x0003c000 0x00014000 0x00014000 0x00014000
+ 0x00014000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00048000 0x00048000 0x00048000
+ 0x00048000 0x000002a0 0x0800013d 0x00000000
+ 0x77fff884 0x01f1f508 0x05057404 0x54000007
+ 0x080001e8 0x08000021 0x00000802 0x00020000
+ 0x00000100 0x018b000c 0xa0f10000 0x00000000
+ 0x00000000 0x800014d4 0xe8000000 0xff00ff89 >;
+ };
+
+ timing-667000000 {
+ clock-frequency = <667000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100002>;
+ nvidia,emc-mode-2 = <0x80200058>;
+ nvidia,emc-mode-reset = <0x80000b71>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000020
+ 0x00000069 0x00000017 0x00000007 0x00000005
+ 0x0000000c 0x00000003 0x00000011 0x00000007
+ 0x00000007 0x00000002 0x00000001 0x00000000
+ 0x00000007 0x0000000b 0x00000009 0x0000000b
+ 0x00000011 0x00001412 0x00000000 0x00000504
+ 0x00000002 0x0000000e 0x00000001 0x00000000
+ 0x0000000c 0x00000016 0x00000072 0x00000200
+ 0x00000005 0x00000015 0x00000000 0x00000006
+ 0x00000007 0x00001453 0x0000000c 0x00000004
+ 0x00000000 0x00000000 0x00005088 0xf00b0191
+ 0x00008000 0x0000000a 0x0000000a 0x0000000a
+ 0x0000000a 0x0000000a 0x0000000a 0x0000000a
+ 0x0000000a 0x00018000 0x00018000 0x00018000
+ 0x00018000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x0000000a 0x0000000a 0x0000000a
+ 0x0000000a 0x000002a0 0x0800013d 0x22220000
+ 0x77fff884 0x01f1f501 0x07077404 0x54000000
+ 0x080001e8 0x08000021 0x00000802 0x00020000
+ 0x00000100 0x0156000c 0xa0f10000 0x00000000
+ 0x00000000 0x800028a5 0xe8000000 0xff00ff49 >;
+ };
+ };
+
+ emc-timings-2 {
+ /* Micron 1GB 667MHZ */
+ nvidia,ram-code = <2>;
+
+ timing-25500000 {
+ clock-frequency = <25500000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200008>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000001
+ 0x00000004 0x00000000 0x00000000 0x00000002
+ 0x0000000a 0x00000005 0x0000000b 0x00000000
+ 0x00000000 0x00000003 0x00000001 0x00000000
+ 0x00000005 0x00000005 0x00000004 0x0000000a
+ 0x0000000b 0x000000c0 0x00000000 0x00000030
+ 0x00000002 0x00000002 0x00000001 0x00000000
+ 0x00000007 0x0000000f 0x00000005 0x00000005
+ 0x00000004 0x00000001 0x00000000 0x00000004
+ 0x00000005 0x000000c7 0x00000006 0x00000004
+ 0x00000000 0x00000000 0x00004288 0x007800a4
+ 0x00008000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000002a0 0x0800211c 0x00000000
+ 0x77fff884 0x01f1f108 0x05057404 0x54000007
+ 0x08000168 0x08000000 0x00000802 0x00000000
+ 0x00000040 0x000c000c 0xa0f10000 0x00000000
+ 0x00000000 0x80000287 0xe8000000 0xff00ff00 >;
+ };
+
+ timing-51000000 {
+ clock-frequency = <51000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200008>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000002
+ 0x00000008 0x00000001 0x00000000 0x00000002
+ 0x0000000a 0x00000005 0x0000000b 0x00000000
+ 0x00000000 0x00000003 0x00000001 0x00000000
+ 0x00000005 0x00000005 0x00000004 0x0000000a
+ 0x0000000b 0x00000181 0x00000000 0x00000060
+ 0x00000002 0x00000002 0x00000001 0x00000000
+ 0x00000007 0x0000000f 0x00000009 0x00000009
+ 0x00000004 0x00000002 0x00000000 0x00000004
+ 0x00000005 0x0000018e 0x00000006 0x00000004
+ 0x00000000 0x00000000 0x00004288 0x007800a4
+ 0x00008000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000002a0 0x0800211c 0x00000000
+ 0x77fff884 0x01f1f108 0x05057404 0x54000007
+ 0x08000168 0x08000000 0x00000802 0x00000000
+ 0x00000040 0x000c000c 0xa0f10000 0x00000000
+ 0x00000000 0x8000040b 0xe8000000 0xff00ff00 >;
+ };
+
+ timing-102000000 {
+ clock-frequency = <102000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200008>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000004
+ 0x00000010 0x00000003 0x00000001 0x00000002
+ 0x0000000a 0x00000005 0x0000000b 0x00000001
+ 0x00000001 0x00000003 0x00000001 0x00000000
+ 0x00000005 0x00000005 0x00000004 0x0000000a
+ 0x0000000b 0x00000303 0x00000000 0x000000c0
+ 0x00000002 0x00000002 0x00000001 0x00000000
+ 0x00000007 0x0000000f 0x00000012 0x00000012
+ 0x00000004 0x00000004 0x00000000 0x00000004
+ 0x00000005 0x0000031c 0x00000006 0x00000004
+ 0x00000000 0x00000000 0x00004288 0x007800a4
+ 0x00008000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000002a0 0x0800211c 0x00000000
+ 0x77fff884 0x01f1f108 0x05057404 0x54000007
+ 0x08000168 0x08000000 0x00000802 0x00000000
+ 0x00000040 0x000c000c 0xa0f10000 0x00000000
+ 0x00000000 0x80000713 0xe8000000 0xff00ff00 >;
+ };
+
+ timing-204000000 {
+ clock-frequency = <204000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200008>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000009
+ 0x00000020 0x00000007 0x00000002 0x00000002
+ 0x0000000a 0x00000005 0x0000000b 0x00000002
+ 0x00000002 0x00000003 0x00000001 0x00000000
+ 0x00000005 0x00000006 0x00000004 0x0000000a
+ 0x0000000b 0x00000607 0x00000000 0x00000181
+ 0x00000002 0x00000002 0x00000001 0x00000000
+ 0x00000007 0x0000000f 0x00000023 0x00000023
+ 0x00000004 0x00000007 0x00000000 0x00000004
+ 0x00000005 0x00000638 0x00000007 0x00000004
+ 0x00000000 0x00000000 0x00004288 0x004400a4
+ 0x00008000 0x00080000 0x00080000 0x00080000
+ 0x00080000 0x00080000 0x00080000 0x00080000
+ 0x00080000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00080000 0x00080000 0x00080000
+ 0x00080000 0x000002a0 0x0800211c 0x00000000
+ 0x77fff884 0x01f1f108 0x05057404 0x54000007
+ 0x08000168 0x08000000 0x00000802 0x00020000
+ 0x00000100 0x000c000c 0xa0f10000 0x00000000
+ 0x00000000 0x80000d22 0xe8000000 0xff00ff00 >;
+ };
+
+ timing-333500000 {
+ clock-frequency = <333500000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100002>;
+ nvidia,emc-mode-2 = <0x80200000>;
+ nvidia,emc-mode-reset = <0x80000321>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+
+ nvidia,emc-configuration = < 0x0000000f
+ 0x00000034 0x0000000a 0x00000003 0x00000003
+ 0x00000008 0x00000002 0x00000009 0x00000003
+ 0x00000003 0x00000002 0x00000001 0x00000000
+ 0x00000004 0x00000006 0x00000004 0x0000000a
+ 0x0000000c 0x000009e9 0x00000000 0x0000027a
+ 0x00000001 0x00000008 0x00000001 0x00000000
+ 0x00000007 0x0000000e 0x00000039 0x00000200
+ 0x00000004 0x0000000a 0x00000000 0x00000004
+ 0x00000005 0x00000a2a 0x00000000 0x00000004
+ 0x00000000 0x00000000 0x00007088 0x002600a4
+ 0x00008000 0x0003c000 0x0003c000 0x0003c000
+ 0x0003c000 0x00014000 0x00014000 0x00014000
+ 0x00014000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00040000 0x00040000 0x00040000
+ 0x00040000 0x000002a0 0x0800013d 0x00000000
+ 0x77fff884 0x01f1f508 0x05057404 0x54000007
+ 0x080001e8 0x08000021 0x00000802 0x00020000
+ 0x00000100 0x018b000c 0xa0f10000 0x00000000
+ 0x00000000 0x800014d4 0xe8000000 0xff00ff89 >;
+ };
+
+ timing-667000000 {
+ clock-frequency = <667000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100002>;
+ nvidia,emc-mode-2 = <0x80200018>;
+ nvidia,emc-mode-reset = <0x80000b71>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x0000001f
+ 0x00000069 0x00000016 0x00000007 0x00000005
+ 0x0000000c 0x00000003 0x00000011 0x00000007
+ 0x00000007 0x00000002 0x00000001 0x00000000
+ 0x00000007 0x0000000b 0x00000009 0x0000000b
+ 0x00000011 0x00001412 0x00000000 0x00000504
+ 0x00000002 0x0000000e 0x00000001 0x00000000
+ 0x0000000c 0x00000016 0x00000072 0x00000200
+ 0x00000005 0x00000015 0x00000000 0x00000006
+ 0x00000007 0x00001453 0x0000000c 0x00000004
+ 0x00000000 0x00000000 0x00005088 0xf00b0191
+ 0x00008000 0x0000000a 0x0000000a 0x0000000a
+ 0x0000000a 0x0000000a 0x0000000a 0x0000000a
+ 0x0000000a 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x0000000a 0x0000000a 0x0000000a
+ 0x0000000a 0x000002a0 0x0600013d 0x22220000
+ 0x77fff884 0x01f1f501 0x07077404 0x54000000
+ 0x080001e8 0x08000021 0x00000802 0x00020000
+ 0x00000100 0x0156000c 0xa0f10000 0x00000000
+ 0x00000000 0x800028a5 0xe8000000 0xff00ff49 >;
+ };
+ };
+ };
+
+ display-panel {
+ compatible = "innolux,g101ice-l01";
+ };
+};
+
+&emc_icc_dvfs_opp_table {
+ /delete-node/ opp-750000000-1300;
+ /delete-node/ opp-800000000-1300;
+ /delete-node/ opp-900000000-1350;
+};
+
+&emc_bw_dfs_opp_table {
+ /delete-node/ opp-750000000;
+ /delete-node/ opp-800000000;
+ /delete-node/ opp-900000000;
+};
diff --git a/arch/arm/boot/dts/tegra30-asus-tf700t.dts b/arch/arm/boot/dts/tegra30-asus-tf700t.dts
new file mode 100644
index 0000000..18a9bfa
--- /dev/null
+++ b/arch/arm/boot/dts/tegra30-asus-tf700t.dts
@@ -0,0 +1,823 @@
+// SPDX-License-Identifier: GPL-2.0
+/dts-v1/;
+
+#include "tegra30-asus-transformer-common.dtsi"
+
+/ {
+ model = "Asus Transformer Infinity TF700T";
+ compatible = "asus,tf700t", "nvidia,tegra30";
+
+ host1x@50000000 {
+ lcd: dc@54200000 {
+ clocks = <&tegra_car TEGRA30_CLK_DISP1>,
+ <&tegra_car TEGRA30_CLK_PLL_D_OUT0>;
+
+ rgb {
+ status = "okay";
+
+ port@0 {
+ dpi_output: endpoint {
+ remote-endpoint = <&bridge_input>;
+ bus-width = <24>;
+ };
+ };
+ };
+ };
+ };
+
+ pinmux@70000868 {
+ state_default: pinmux {
+ lcd_pwr2_pc6 {
+ nvidia,pins = "lcd_pwr2_pc6",
+ "lcd_dc1_pd2";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ pbb3 {
+ nvidia,pins = "pbb3";
+ nvidia,function = "vgp3";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ spi2_mosi_px0 {
+ nvidia,pins = "spi2_mosi_px0";
+ nvidia,function = "spi2";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ pbb7 {
+ nvidia,pins = "pbb7";
+ nvidia,function = "i2s4";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ kb_row7_pr7 {
+ nvidia,pins = "kb_row7_pr7";
+ nvidia,function = "kbc";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ gmi_cs4_n_pk2 {
+ nvidia,pins = "gmi_cs4_n_pk2";
+ nvidia,function = "gmi";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+ };
+ };
+
+ serial@70006200 {
+ /* Azurewave AW-NH665 BCM4330B1 */
+ bluetooth {
+ compatible = "brcm,bcm4330-bt";
+ };
+ };
+
+ i2c@7000c400 {
+ /* Elantech ELAN-3024-7053 or 5184N FPC-1 REV: 2/3 touchscreen */
+ touchscreen@10 {
+ compatible = "elan,ektf3624";
+ reg = <0x10>;
+
+ interrupt-parent = <&gpio>;
+ interrupts = <TEGRA_GPIO(H, 4) IRQ_TYPE_EDGE_FALLING>;
+ reset-gpios = <&gpio TEGRA_GPIO(H, 6) GPIO_ACTIVE_LOW>;
+
+ vcc33-supply = <&vdd_3v3_sys>;
+ vccio-supply = <&vdd_3v3_sys>;
+
+ touchscreen-size-x = <2944>;
+ touchscreen-size-y = <1856>;
+ touchscreen-inverted-y;
+ };
+ };
+
+ i2c@7000c500 {
+ clock-frequency = <100000>;
+
+ magnetometer@e {
+ mount-matrix = "1", "0", "0",
+ "0", "-1", "0",
+ "0", "0", "-1";
+ };
+
+ gyroscope@68 {
+ mount-matrix = "0", "1", "0",
+ "1", "0", "0",
+ "0", "0", "-1";
+
+ /* External I2C interface */
+ i2c-gate {
+ accelerometer@f {
+ mount-matrix = "0", "-1", "0",
+ "-1", "0", "0",
+ "0", "0", "1";
+ };
+ };
+ };
+ };
+
+ i2c@7000d000 {
+ /* Realtek ALC5631 audio codec */
+ rt5631: audio-codec@1a {
+ compatible = "realtek,rt5631";
+ reg = <0x1a>;
+ };
+ };
+
+ memory-controller@7000f000 {
+ emc-timings-0 {
+ /* Micron 1GB 800MHZ */
+ nvidia,ram-code = <0>;
+
+ timing-25500000 {
+ clock-frequency = <25500000>;
+
+ nvidia,emem-configuration = < 0x00020001 0xc0000020
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000001 0x00000001 0x00000003 0x00000008
+ 0x00000002 0x00000001 0x00000002 0x00000006
+ 0x06020102 0x000a0502 0x75830303 0x001f0000 >;
+ };
+
+ timing-51000000 {
+ clock-frequency = <51000000>;
+
+ nvidia,emem-configuration = < 0x00010001 0xc0000020
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000001 0x00000001 0x00000003 0x00000008
+ 0x00000002 0x00000001 0x00000002 0x00000006
+ 0x06020102 0x000a0502 0x74630303 0x001f0000 >;
+ };
+
+ timing-102000000 {
+ clock-frequency = <102000000>;
+
+ nvidia,emem-configuration = < 0x00000001 0xc0000030
+ 0x00000001 0x00000001 0x00000003 0x00000000
+ 0x00000002 0x00000001 0x00000003 0x00000008
+ 0x00000002 0x00000001 0x00000002 0x00000006
+ 0x06020102 0x000a0503 0x73c30504 0x001f0000 >;
+ };
+
+ timing-204000000 {
+ clock-frequency = <204000000>;
+
+ nvidia,emem-configuration = < 0x00000003 0xc0000025
+ 0x00000001 0x00000001 0x00000005 0x00000002
+ 0x00000004 0x00000001 0x00000003 0x00000008
+ 0x00000002 0x00000001 0x00000002 0x00000006
+ 0x06020102 0x000a0505 0x73840a06 0x001f0000 >;
+ };
+
+ timing-400000000 {
+ clock-frequency = <400000000>;
+
+ nvidia,emem-configuration = < 0x00000006 0xc0000048
+ 0x00000001 0x00000002 0x00000009 0x00000005
+ 0x00000007 0x00000001 0x00000002 0x00000008
+ 0x00000002 0x00000002 0x00000003 0x00000006
+ 0x06030202 0x000d0709 0x7086120a 0x001f0000 >;
+ };
+
+ timing-800000000 {
+ clock-frequency = <800000000>;
+
+ nvidia,emem-configuration = < 0x0000000c 0xc0000090
+ 0x00000004 0x00000005 0x00000013 0x0000000c
+ 0x0000000f 0x00000002 0x00000003 0x0000000c
+ 0x00000002 0x00000002 0x00000004 0x00000008
+ 0x08040202 0x00160d13 0x712c2414 0x001f0000 >;
+ };
+ };
+
+ emc-timings-1 {
+ /* Elpida 1GB 800MHZ */
+ nvidia,ram-code = <1>;
+
+ timing-25500000 {
+ clock-frequency = <25500000>;
+
+ nvidia,emem-configuration = < 0x00020001 0xc0000020
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000001 0x00000001 0x00000003 0x00000008
+ 0x00000002 0x00000001 0x00000002 0x00000006
+ 0x06020102 0x000a0502 0x75830303 0x001f0000 >;
+ };
+
+ timing-51000000 {
+ clock-frequency = <51000000>;
+
+ nvidia,emem-configuration = < 0x00010001 0xc0000020
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000001 0x00000001 0x00000003 0x00000008
+ 0x00000002 0x00000001 0x00000002 0x00000006
+ 0x06020102 0x000a0502 0x74630303 0x001f0000 >;
+ };
+
+ timing-102000000 {
+ clock-frequency = <102000000>;
+
+ nvidia,emem-configuration = < 0x00000001 0xc0000030
+ 0x00000001 0x00000001 0x00000003 0x00000000
+ 0x00000002 0x00000001 0x00000003 0x00000008
+ 0x00000002 0x00000001 0x00000002 0x00000006
+ 0x06020102 0x000a0503 0x73c30504 0x001f0000 >;
+ };
+
+ timing-204000000 {
+ clock-frequency = <204000000>;
+
+ nvidia,emem-configuration = < 0x00000003 0xc0000025
+ 0x00000001 0x00000001 0x00000005 0x00000002
+ 0x00000004 0x00000001 0x00000003 0x00000008
+ 0x00000002 0x00000001 0x00000002 0x00000006
+ 0x06020102 0x000a0505 0x73840a06 0x001f0000 >;
+ };
+
+ timing-400000000 {
+ clock-frequency = <400000000>;
+
+ nvidia,emem-configuration = < 0x00000006 0xc0000048
+ 0x00000001 0x00000002 0x00000009 0x00000005
+ 0x00000007 0x00000001 0x00000002 0x00000008
+ 0x00000002 0x00000002 0x00000003 0x00000006
+ 0x06030202 0x000d0709 0x7086120a 0x001f0000 >;
+ };
+
+ timing-800000000 {
+ clock-frequency = <800000000>;
+
+ nvidia,emem-configuration = < 0x0000000c 0xc0000090
+ 0x00000004 0x00000005 0x00000013 0x0000000c
+ 0x0000000f 0x00000002 0x00000003 0x0000000c
+ 0x00000002 0x00000002 0x00000004 0x00000008
+ 0x08040202 0x00160d13 0x712c2414 0x001f0000 >;
+ };
+ };
+ };
+
+ memory-controller@7000f400 {
+ emc-timings-0 {
+ /* Micron 1GB 800MHZ */
+ nvidia,ram-code = <0>;
+
+ timing-25500000 {
+ clock-frequency = <25500000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200008>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000001
+ 0x00000006 0x00000000 0x00000000 0x00000002
+ 0x0000000a 0x00000005 0x0000000b 0x00000000
+ 0x00000000 0x00000003 0x00000001 0x00000000
+ 0x00000005 0x00000005 0x00000004 0x0000000a
+ 0x0000000b 0x000000c0 0x00000000 0x00000030
+ 0x00000002 0x00000002 0x00000001 0x00000000
+ 0x00000007 0x0000000f 0x00000007 0x00000007
+ 0x00000004 0x00000002 0x00000000 0x00000004
+ 0x00000005 0x000000c7 0x00000006 0x00000004
+ 0x00000000 0x00000000 0x00004288 0x007800a4
+ 0x00008000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000002a0 0x0800211c 0x00000000
+ 0x77fff884 0x01f1f108 0x05057404 0x54000007
+ 0x08000168 0x08000000 0x00000802 0x00000000
+ 0x00000040 0x000c000c 0xa0f10000 0x00000000
+ 0x00000000 0x80000287 0xe8000000 0xff00ff00 >;
+ };
+
+ timing-51000000 {
+ clock-frequency = <51000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200008>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000002
+ 0x0000000d 0x00000001 0x00000000 0x00000002
+ 0x0000000a 0x00000005 0x0000000b 0x00000000
+ 0x00000000 0x00000003 0x00000001 0x00000000
+ 0x00000005 0x00000005 0x00000004 0x0000000a
+ 0x0000000b 0x00000181 0x00000000 0x00000060
+ 0x00000002 0x00000002 0x00000001 0x00000000
+ 0x00000007 0x0000000f 0x0000000e 0x0000000e
+ 0x00000004 0x00000003 0x00000000 0x00000004
+ 0x00000005 0x0000018e 0x00000006 0x00000004
+ 0x00000000 0x00000000 0x00004288 0x007800a4
+ 0x00008000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000002a0 0x0800211c 0x00000000
+ 0x77fff884 0x01f1f108 0x05057404 0x54000007
+ 0x08000168 0x08000000 0x00000802 0x00000000
+ 0x00000040 0x000c000c 0xa0f10000 0x00000000
+ 0x00000000 0x8000040b 0xe8000000 0xff00ff00 >;
+ };
+
+ timing-102000000 {
+ clock-frequency = <102000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200008>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000004
+ 0x0000001a 0x00000003 0x00000001 0x00000002
+ 0x0000000a 0x00000005 0x0000000b 0x00000001
+ 0x00000001 0x00000003 0x00000001 0x00000000
+ 0x00000005 0x00000005 0x00000004 0x0000000a
+ 0x0000000b 0x00000303 0x00000000 0x000000c0
+ 0x00000002 0x00000002 0x00000001 0x00000000
+ 0x00000007 0x0000000f 0x0000001c 0x0000001c
+ 0x00000004 0x00000005 0x00000000 0x00000004
+ 0x00000005 0x0000031c 0x00000006 0x00000004
+ 0x00000000 0x00000000 0x00004288 0x007800a4
+ 0x00008000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000002a0 0x0800211c 0x00000000
+ 0x77fff884 0x01f1f108 0x05057404 0x54000007
+ 0x08000168 0x08000000 0x00000802 0x00000000
+ 0x00000040 0x000c000c 0xa0f10000 0x00000000
+ 0x00000000 0x80000713 0xe8000000 0xff00ff00 >;
+ };
+
+ timing-204000000 {
+ clock-frequency = <204000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200008>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000009
+ 0x00000035 0x00000007 0x00000002 0x00000002
+ 0x0000000a 0x00000005 0x0000000b 0x00000002
+ 0x00000002 0x00000003 0x00000001 0x00000000
+ 0x00000005 0x00000006 0x00000004 0x0000000a
+ 0x0000000b 0x00000607 0x00000000 0x00000181
+ 0x00000002 0x00000002 0x00000001 0x00000000
+ 0x00000007 0x0000000f 0x00000038 0x00000038
+ 0x00000004 0x00000009 0x00000000 0x00000004
+ 0x00000005 0x00000638 0x00000007 0x00000004
+ 0x00000000 0x00000000 0x00004288 0x004400a4
+ 0x00008000 0x00080000 0x00080000 0x00080000
+ 0x00080000 0x00080000 0x00080000 0x00080000
+ 0x00080000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00080000 0x00080000 0x00080000
+ 0x00080000 0x000002a0 0x0800211c 0x00000000
+ 0x77fff884 0x01f1f108 0x05057404 0x54000007
+ 0x08000168 0x08000000 0x00000802 0x00020000
+ 0x00000100 0x000c000c 0xa0f10000 0x00000000
+ 0x00000000 0x80000d22 0xe8000000 0xff00ff00 >;
+ };
+
+ timing-400000000 {
+ clock-frequency = <400000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100002>;
+ nvidia,emc-mode-2 = <0x80200000>;
+ nvidia,emc-mode-reset = <0x80000521>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+
+ nvidia,emc-configuration = < 0x00000012
+ 0x00000066 0x0000000c 0x00000004 0x00000003
+ 0x00000008 0x00000002 0x0000000a 0x00000004
+ 0x00000004 0x00000002 0x00000001 0x00000000
+ 0x00000004 0x00000006 0x00000004 0x0000000a
+ 0x0000000c 0x00000bf0 0x00000000 0x000002fc
+ 0x00000001 0x00000008 0x00000001 0x00000000
+ 0x00000008 0x0000000f 0x0000006c 0x00000200
+ 0x00000004 0x00000010 0x00000000 0x00000004
+ 0x00000005 0x00000c30 0x00000000 0x00000004
+ 0x00000000 0x00000000 0x00007088 0x001d0084
+ 0x00008000 0x00044000 0x00044000 0x00044000
+ 0x00044000 0x00014000 0x00014000 0x00014000
+ 0x00014000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00048000 0x00048000 0x00048000
+ 0x00048000 0x000002a0 0x0600013d 0x00000000
+ 0x77fff884 0x01f1f508 0x05057404 0x54000007
+ 0x080001e8 0x08000021 0x00000802 0x00020000
+ 0x00000100 0x0158000c 0xa0f10000 0x00000000
+ 0x00000000 0x800018c8 0xe8000000 0xff00ff89 >;
+ };
+
+ timing-800000000 {
+ clock-frequency = <800000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100002>;
+ nvidia,emc-mode-2 = <0x80200018>;
+ nvidia,emc-mode-reset = <0x80000d71>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000025
+ 0x000000ce 0x0000001a 0x00000009 0x00000005
+ 0x0000000d 0x00000004 0x00000013 0x00000009
+ 0x00000009 0x00000004 0x00000001 0x00000000
+ 0x00000007 0x0000000a 0x00000009 0x0000000a
+ 0x00000011 0x00001820 0x00000000 0x00000608
+ 0x00000003 0x00000012 0x00000001 0x00000000
+ 0x0000000f 0x00000018 0x000000d8 0x00000200
+ 0x00000005 0x00000020 0x00000000 0x00000007
+ 0x00000008 0x00001860 0x0000000b 0x00000006
+ 0x00000000 0x00000000 0x00005088 0xf0070191
+ 0x00008000 0x0000000a 0x0000000a 0x0000000a
+ 0x0000000a 0x0000000a 0x0000000a 0x0000000a
+ 0x0000000a 0x00018000 0x00018000 0x00018000
+ 0x00018000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x0000000a 0x0000000a 0x0000000a
+ 0x0000000a 0x000002a0 0x0800013d 0x22220000
+ 0x77fff884 0x01f1f501 0x07077404 0x54000000
+ 0x080001e8 0x08000021 0x00000802 0x00020000
+ 0x00000100 0x00f0000c 0xa0f10000 0x00000000
+ 0x00000000 0x8000308c 0xe8000000 0xff00ff49 >;
+ };
+ };
+
+ emc-timings-1 {
+ /* Elpida 1GB 800MHZ */
+ nvidia,ram-code = <1>;
+
+ timing-25500000 {
+ clock-frequency = <25500000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200008>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000001
+ 0x00000006 0x00000000 0x00000000 0x00000002
+ 0x0000000a 0x00000005 0x0000000b 0x00000000
+ 0x00000000 0x00000003 0x00000001 0x00000000
+ 0x00000005 0x00000005 0x00000004 0x0000000a
+ 0x0000000b 0x000000c0 0x00000000 0x00000030
+ 0x00000002 0x00000002 0x00000001 0x00000000
+ 0x00000007 0x0000000f 0x00000007 0x00000007
+ 0x00000004 0x00000002 0x00000000 0x00000004
+ 0x00000005 0x000000c7 0x00000006 0x00000004
+ 0x00000000 0x00000000 0x00004288 0x007800a4
+ 0x00008000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000002a0 0x0800211c 0x00000000
+ 0x77fff884 0x01f1f108 0x05057404 0x54000007
+ 0x08000168 0x08000000 0x00000802 0x00000000
+ 0x00000040 0x000c000c 0xa0f10000 0x00000000
+ 0x00000000 0x80000287 0xe8000000 0xff00ff00 >;
+ };
+
+ timing-51000000 {
+ clock-frequency = <51000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200008>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000002
+ 0x0000000d 0x00000001 0x00000000 0x00000002
+ 0x0000000a 0x00000005 0x0000000b 0x00000000
+ 0x00000000 0x00000003 0x00000001 0x00000000
+ 0x00000005 0x00000005 0x00000004 0x0000000a
+ 0x0000000b 0x00000181 0x00000000 0x00000060
+ 0x00000002 0x00000002 0x00000001 0x00000000
+ 0x00000007 0x0000000f 0x0000000e 0x0000000e
+ 0x00000004 0x00000003 0x00000000 0x00000004
+ 0x00000005 0x0000018e 0x00000006 0x00000004
+ 0x00000000 0x00000000 0x00004288 0x007800a4
+ 0x00008000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000002a0 0x0800211c 0x00000000
+ 0x77fff884 0x01f1f108 0x05057404 0x54000007
+ 0x08000168 0x08000000 0x00000802 0x00000000
+ 0x00000040 0x000c000c 0xa0f10000 0x00000000
+ 0x00000000 0x8000040b 0xe8000000 0xff00ff00 >;
+ };
+
+ timing-102000000 {
+ clock-frequency = <102000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200008>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000004
+ 0x0000001a 0x00000003 0x00000001 0x00000002
+ 0x0000000a 0x00000005 0x0000000b 0x00000001
+ 0x00000001 0x00000003 0x00000001 0x00000000
+ 0x00000005 0x00000005 0x00000004 0x0000000a
+ 0x0000000b 0x00000303 0x00000000 0x000000c0
+ 0x00000002 0x00000002 0x00000001 0x00000000
+ 0x00000007 0x0000000f 0x0000001c 0x0000001c
+ 0x00000004 0x00000005 0x00000000 0x00000004
+ 0x00000005 0x0000031c 0x00000006 0x00000004
+ 0x00000000 0x00000000 0x00004288 0x007800a4
+ 0x00008000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000002a0 0x0800211c 0x00000000
+ 0x77fff884 0x01f1f108 0x05057404 0x54000007
+ 0x08000168 0x08000000 0x00000802 0x00000000
+ 0x00000040 0x000c000c 0xa0f10000 0x00000000
+ 0x00000000 0x80000713 0xe8000000 0xff00ff00 >;
+ };
+
+ timing-204000000 {
+ clock-frequency = <204000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200008>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000009
+ 0x00000035 0x00000007 0x00000002 0x00000002
+ 0x0000000a 0x00000005 0x0000000b 0x00000002
+ 0x00000002 0x00000003 0x00000001 0x00000000
+ 0x00000005 0x00000006 0x00000004 0x0000000a
+ 0x0000000b 0x00000607 0x00000000 0x00000181
+ 0x00000002 0x00000002 0x00000001 0x00000000
+ 0x00000007 0x0000000f 0x00000038 0x00000038
+ 0x00000004 0x00000009 0x00000000 0x00000004
+ 0x00000005 0x00000638 0x00000007 0x00000004
+ 0x00000000 0x00000000 0x00004288 0x004400a4
+ 0x00008000 0x00080000 0x00080000 0x00080000
+ 0x00080000 0x00080000 0x00080000 0x00080000
+ 0x00080000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00080000 0x00080000 0x00080000
+ 0x00080000 0x000002a0 0x0800211c 0x00000000
+ 0x77fff884 0x01f1f108 0x05057404 0x54000007
+ 0x08000168 0x08000000 0x00000802 0x00020000
+ 0x00000100 0x000c000c 0xa0f10000 0x00000000
+ 0x00000000 0x80000d22 0xe8000000 0xff00ff00 >;
+ };
+
+ timing-400000000 {
+ clock-frequency = <400000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100002>;
+ nvidia,emc-mode-2 = <0x80200000>;
+ nvidia,emc-mode-reset = <0x80000521>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+
+ nvidia,emc-configuration = < 0x00000012
+ 0x00000066 0x0000000c 0x00000004 0x00000003
+ 0x00000008 0x00000002 0x0000000a 0x00000004
+ 0x00000004 0x00000002 0x00000001 0x00000000
+ 0x00000004 0x00000006 0x00000004 0x0000000a
+ 0x0000000c 0x00000bf0 0x00000000 0x000002fc
+ 0x00000001 0x00000008 0x00000001 0x00000000
+ 0x00000008 0x0000000f 0x0000006c 0x00000200
+ 0x00000004 0x00000010 0x00000000 0x00000004
+ 0x00000005 0x00000c30 0x00000000 0x00000004
+ 0x00000000 0x00000000 0x00007088 0x001d0084
+ 0x00008000 0x00044000 0x00044000 0x00044000
+ 0x00044000 0x00014000 0x00014000 0x00014000
+ 0x00014000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00048000 0x00048000 0x00048000
+ 0x00048000 0x000002a0 0x0600013d 0x00000000
+ 0x77fff884 0x01f1f508 0x05057404 0x54000007
+ 0x080001e8 0x08000021 0x00000802 0x00020000
+ 0x00000100 0x0158000c 0xa0f10000 0x00000000
+ 0x00000000 0x800018c8 0xe8000000 0xff00ff89 >;
+ };
+
+ timing-800000000 {
+ clock-frequency = <800000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100002>;
+ nvidia,emc-mode-2 = <0x80200018>;
+ nvidia,emc-mode-reset = <0x80000d71>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000025
+ 0x000000ce 0x0000001a 0x00000009 0x00000005
+ 0x0000000d 0x00000004 0x00000013 0x00000009
+ 0x00000009 0x00000004 0x00000001 0x00000000
+ 0x00000007 0x0000000a 0x00000009 0x0000000a
+ 0x00000011 0x00001820 0x00000000 0x00000608
+ 0x00000003 0x00000012 0x00000001 0x00000000
+ 0x0000000f 0x00000018 0x000000d8 0x00000200
+ 0x00000005 0x00000020 0x00000000 0x00000007
+ 0x00000008 0x00001860 0x0000000b 0x00000006
+ 0x00000000 0x00000000 0x00005088 0xf0070191
+ 0x00008000 0x0000000a 0x0000000a 0x0000000a
+ 0x0000000a 0x0000000a 0x0000000a 0x0000000a
+ 0x0000000a 0x00018000 0x00018000 0x00018000
+ 0x00018000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x0000000a 0x0000000a 0x0000000a
+ 0x0000000a 0x000002a0 0x0a00013d 0x22220000
+ 0x77fff884 0x01f1f501 0x07077404 0x54000000
+ 0x080001e8 0x08000021 0x00000802 0x00020000
+ 0x00000100 0x00f0000c 0xa0f10000 0x00000000
+ 0x00000000 0x8000308c 0xe8000000 0xff00ff49 >;
+ };
+ };
+ };
+
+ tc358768_refclk: clock-tc358768 {
+ compatible = "fixed-clock";
+ clock-frequency = <23100000>;
+ clock-accuracy = <100>;
+ #clock-cells = <0>;
+ };
+
+ tc358768_osc: clock-tc358768-osc-gate {
+ compatible = "gpio-gate-clock";
+ enable-gpios = <&gpio TEGRA_GPIO(D, 2) GPIO_ACTIVE_HIGH>;
+ clocks = <&tc358768_refclk>;
+ #clock-cells = <0>;
+ };
+
+ haptic-feedback {
+ compatible = "gpio-vibrator";
+ enable-gpios = <&gpio TEGRA_GPIO(H, 7) GPIO_ACTIVE_HIGH>;
+ vcc-supply = <&vdd_3v3_sys>;
+ };
+
+ i2c-mux {
+ compatible = "i2c-mux-gpio";
+
+ mux-gpios = <&gpio TEGRA_GPIO(X, 0) GPIO_ACTIVE_HIGH>;
+ i2c-parent = <&lcd_ddc>;
+ idle-state = <0x0>;
+
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ i2c@1 {
+ reg = <1>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ dsi-bridge@7 {
+ compatible = "toshiba,tc358768";
+ reg = <0x7>;
+
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ clocks = <&tc358768_osc>;
+ clock-names = "refclk";
+
+ reset-gpios = <&gpio TEGRA_GPIO(N, 6) GPIO_ACTIVE_LOW>;
+
+ vddc-supply = <&vdd_1v2_mipi>;
+ vddio-supply = <&vdd_1v8_vio>;
+ vddmipi-supply = <&vdd_1v2_mipi>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+
+ bridge_input: endpoint {
+ remote-endpoint = <&dpi_output>;
+ data-lines = <24>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+
+ bridge_output: endpoint {
+ remote-endpoint = <&panel_input>;
+ };
+ };
+ };
+
+ /*
+ * Panasonic VVX10F004B00 or HYDIS HV101WU1-1E1
+ * LCD SuperIPS+ Full HD panel.
+ */
+ panel@1 {
+ compatible = "panasonic,vvx10f004b00";
+ reg = <1>;
+
+ power-supply = <&vdd_pnl>;
+ backlight = <&backlight>;
+
+ port {
+ panel_input: endpoint {
+ remote-endpoint = <&bridge_output>;
+ };
+ };
+ };
+ };
+ };
+ };
+
+ vdd_1v2_mipi: regulator-mipi {
+ compatible = "regulator-fixed";
+ regulator-name = "tc358768_1v2_vdd";
+ regulator-min-microvolt = <1200000>;
+ regulator-max-microvolt = <1200000>;
+ regulator-enable-ramp-delay = <10000>;
+ regulator-boot-on;
+ gpio = <&gpio TEGRA_GPIO(BB, 3) GPIO_ACTIVE_HIGH>;
+ enable-active-high;
+ vin-supply = <&vdd_3v3_sys>;
+ };
+};
+
+&emc_icc_dvfs_opp_table {
+ /delete-node/ opp-900000000-1350;
+};
+
+&emc_bw_dfs_opp_table {
+ /delete-node/ opp-900000000;
+};
diff --git a/arch/arm/boot/dts/tegra30-asus-transformer-common.dtsi b/arch/arm/boot/dts/tegra30-asus-transformer-common.dtsi
new file mode 100644
index 0000000..85b43a8
--- /dev/null
+++ b/arch/arm/boot/dts/tegra30-asus-transformer-common.dtsi
@@ -0,0 +1,1787 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <dt-bindings/input/gpio-keys.h>
+#include <dt-bindings/input/input.h>
+#include <dt-bindings/thermal/thermal.h>
+
+#include "tegra30.dtsi"
+#include "tegra30-cpu-opp.dtsi"
+#include "tegra30-cpu-opp-microvolt.dtsi"
+
+/ {
+ chassis-type = "convertible";
+
+ aliases {
+ mmc0 = "/mmc@78000600"; /* eMMC */
+ mmc1 = "/mmc@78000000"; /* uSD slot */
+ mmc2 = "/mmc@78000400"; /* WiFi */
+
+ rtc0 = &pmic;
+ rtc1 = "/rtc@7000e000";
+
+ display0 = &lcd;
+ display1 = &hdmi;
+
+ serial1 = &uartc; /* Bluetooth */
+ serial2 = &uartb; /* GPS */
+ };
+
+ /*
+ * The decompressor and also some bootloaders rely on a
+ * pre-existing /chosen node to be available to insert the
+ * command line and merge other ATAGS info.
+ */
+ chosen {};
+
+ memory@80000000 {
+ reg = <0x80000000 0x40000000>;
+ };
+
+ reserved-memory {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges;
+
+ linux,cma@80000000 {
+ compatible = "shared-dma-pool";
+ alloc-ranges = <0x80000000 0x30000000>;
+ size = <0x10000000>; /* 256MiB */
+ linux,cma-default;
+ reusable;
+ };
+
+ ramoops@beb00000 {
+ compatible = "ramoops";
+ reg = <0xbeb00000 0x10000>; /* 64kB */
+ console-size = <0x8000>; /* 32kB */
+ record-size = <0x400>; /* 1kB */
+ ecc-size = <16>;
+ };
+
+ trustzone@bfe00000 {
+ reg = <0xbfe00000 0x200000>; /* 2MB */
+ no-map;
+ };
+ };
+
+ host1x@50000000 {
+ hdmi: hdmi@54280000 {
+ status = "okay";
+
+ hdmi-supply = <&hdmi_5v0_sys>;
+ pll-supply = <&vdd_1v8_vio>;
+ vdd-supply = <&vdd_3v3_sys>;
+
+ nvidia,hpd-gpio = <&gpio TEGRA_GPIO(N, 7) GPIO_ACTIVE_HIGH>;
+ nvidia,ddc-i2c-bus = <&hdmi_ddc>;
+ };
+ };
+
+ gpio@6000d000 {
+ init-lpm-in-hog {
+ gpio-hog;
+ gpios = <TEGRA_GPIO(I, 6) GPIO_ACTIVE_HIGH>,
+ <TEGRA_GPIO(B, 1) GPIO_ACTIVE_HIGH>;
+ input;
+ };
+
+ init-lpm-out-hog {
+ gpio-hog;
+ gpios = <TEGRA_GPIO(K, 7) GPIO_ACTIVE_HIGH>,
+ <TEGRA_GPIO(R, 0) GPIO_ACTIVE_HIGH>;
+ output-low;
+ };
+
+ usb-charge-limit-hog {
+ gpio-hog;
+ gpios = <TEGRA_GPIO(R, 1) GPIO_ACTIVE_LOW>;
+ output-high;
+ };
+ };
+
+ vde@6001a000 {
+ assigned-clocks = <&tegra_car TEGRA30_CLK_VDE>;
+ assigned-clock-parents = <&tegra_car TEGRA30_CLK_PLL_P>;
+ assigned-clock-rates = <408000000>;
+ };
+
+ pinmux@70000868 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&state_default>;
+
+ state_default: pinmux {
+ /* SDMMC1 pinmux */
+ sdmmc1_clk {
+ nvidia,pins = "sdmmc1_clk_pz0";
+ nvidia,function = "sdmmc1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ sdmmc1_cmd {
+ nvidia,pins = "sdmmc1_dat3_py4",
+ "sdmmc1_dat2_py5",
+ "sdmmc1_dat1_py6",
+ "sdmmc1_dat0_py7",
+ "sdmmc1_cmd_pz1";
+ nvidia,function = "sdmmc1";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ sdmmc1_cd {
+ nvidia,pins = "gmi_iordy_pi5";
+ nvidia,function = "rsvd1";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ sdmmc1_wp {
+ nvidia,pins = "vi_d11_pt3";
+ nvidia,function = "rsvd2";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ /* SDMMC2 pinmux */
+ vi_d1_pd5 {
+ nvidia,pins = "vi_d1_pd5",
+ "vi_d2_pl0",
+ "vi_d3_pl1",
+ "vi_d5_pl3",
+ "vi_d7_pl5";
+ nvidia,function = "sdmmc2";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ vi_d8_pl6 {
+ nvidia,pins = "vi_d8_pl6",
+ "vi_d9_pl7";
+ nvidia,function = "sdmmc2";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ nvidia,lock = <0>;
+ nvidia,ioreset = <0>;
+ };
+
+ /* SDMMC3 pinmux */
+ sdmmc3_clk {
+ nvidia,pins = "sdmmc3_clk_pa6";
+ nvidia,function = "sdmmc3";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ sdmmc3_cmd {
+ nvidia,pins = "sdmmc3_cmd_pa7",
+ "sdmmc3_dat0_pb7",
+ "sdmmc3_dat1_pb6",
+ "sdmmc3_dat2_pb5",
+ "sdmmc3_dat3_pb4",
+ "sdmmc3_dat4_pd1",
+ "sdmmc3_dat5_pd0",
+ "sdmmc3_dat6_pd3",
+ "sdmmc3_dat7_pd4";
+ nvidia,function = "sdmmc3";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ /* SDMMC4 pinmux */
+ sdmmc4_clk {
+ nvidia,pins = "sdmmc4_clk_pcc4";
+ nvidia,function = "sdmmc4";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ sdmmc4_cmd {
+ nvidia,pins = "sdmmc4_cmd_pt7",
+ "sdmmc4_dat0_paa0",
+ "sdmmc4_dat1_paa1",
+ "sdmmc4_dat2_paa2",
+ "sdmmc4_dat3_paa3",
+ "sdmmc4_dat4_paa4",
+ "sdmmc4_dat5_paa5",
+ "sdmmc4_dat6_paa6",
+ "sdmmc4_dat7_paa7";
+ nvidia,function = "sdmmc4";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ sdmmc4_rst_n {
+ nvidia,pins = "sdmmc4_rst_n_pcc3";
+ nvidia,function = "rsvd2";
+ nvidia,pull = <TEGRA_PIN_PULL_DOWN>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ cam_mclk {
+ nvidia,pins = "cam_mclk_pcc0";
+ nvidia,function = "vi_alt3";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ drive_sdmmc4 {
+ nvidia,pins = "drive_gma",
+ "drive_gmb",
+ "drive_gmc",
+ "drive_gmd";
+ nvidia,pull-down-strength = <9>;
+ nvidia,pull-up-strength = <9>;
+ nvidia,slew-rate-rising = <TEGRA_PIN_SLEW_RATE_SLOWEST>;
+ nvidia,slew-rate-falling = <TEGRA_PIN_SLEW_RATE_SLOWEST>;
+ };
+
+ /* I2C pinmux */
+ gen1_i2c {
+ nvidia,pins = "gen1_i2c_scl_pc4",
+ "gen1_i2c_sda_pc5";
+ nvidia,function = "i2c1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ nvidia,open-drain = <TEGRA_PIN_ENABLE>;
+ nvidia,lock = <0>;
+ };
+
+ gen2_i2c {
+ nvidia,pins = "gen2_i2c_scl_pt5",
+ "gen2_i2c_sda_pt6";
+ nvidia,function = "i2c2";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ nvidia,open-drain = <TEGRA_PIN_ENABLE>;
+ nvidia,lock = <0>;
+ };
+
+ cam_i2c {
+ nvidia,pins = "cam_i2c_scl_pbb1",
+ "cam_i2c_sda_pbb2";
+ nvidia,function = "i2c3";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ nvidia,open-drain = <TEGRA_PIN_ENABLE>;
+ nvidia,lock = <0>;
+ };
+
+ ddc_i2c {
+ nvidia,pins = "ddc_scl_pv4",
+ "ddc_sda_pv5";
+ nvidia,function = "i2c4";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ nvidia,lock = <0>;
+ };
+
+ pwr_i2c {
+ nvidia,pins = "pwr_i2c_scl_pz6",
+ "pwr_i2c_sda_pz7";
+ nvidia,function = "i2cpwr";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ nvidia,open-drain = <TEGRA_PIN_ENABLE>;
+ nvidia,lock = <0>;
+ };
+
+ hotplug_i2c {
+ nvidia,pins = "pu4";
+ nvidia,function = "rsvd4";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ /* HDMI pinmux */
+ hdmi_cec {
+ nvidia,pins = "hdmi_cec_pee3";
+ nvidia,function = "cec";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ nvidia,open-drain = <TEGRA_PIN_ENABLE>;
+ nvidia,lock = <0>;
+ };
+
+ hdmi_hpd {
+ nvidia,pins = "hdmi_int_pn7";
+ nvidia,function = "hdmi";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ /* UART-A */
+ ulpi_data0_po1 {
+ nvidia,pins = "ulpi_data0_po1";
+ nvidia,function = "uarta";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ ulpi_data1_po2 {
+ nvidia,pins = "ulpi_data1_po2";
+ nvidia,function = "uarta";
+ nvidia,pull = <TEGRA_PIN_PULL_DOWN>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ ulpi_data5_po6 {
+ nvidia,pins = "ulpi_data5_po6";
+ nvidia,function = "uarta";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ ulpi_data7_po0 {
+ nvidia,pins = "ulpi_data7_po0",
+ "ulpi_data2_po3",
+ "ulpi_data3_po4",
+ "ulpi_data4_po5",
+ "ulpi_data6_po7";
+ nvidia,function = "uarta";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ /* UART-B */
+ uartb_txd_rts {
+ nvidia,pins = "uart2_txd_pc2",
+ "uart2_rts_n_pj6";
+ nvidia,function = "uartb";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ uartb_rxd_cts {
+ nvidia,pins = "uart2_rxd_pc3",
+ "uart2_cts_n_pj5";
+ nvidia,function = "uartb";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ /* UART-C */
+ uartc_rxd_cts {
+ nvidia,pins = "uart3_cts_n_pa1",
+ "uart3_rxd_pw7";
+ nvidia,function = "uartc";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ uartc_txd_rts {
+ nvidia,pins = "uart3_rts_n_pc0",
+ "uart3_txd_pw6";
+ nvidia,function = "uartc";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ /* UART-D */
+ ulpi_nxt_py2 {
+ nvidia,pins = "ulpi_nxt_py2";
+ nvidia,function = "uartd";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ ulpi_clk_py0 {
+ nvidia,pins = "ulpi_clk_py0",
+ "ulpi_dir_py1",
+ "ulpi_stp_py3";
+ nvidia,function = "uartd";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ /* I2S pinmux */
+ dap_i2s0 {
+ nvidia,pins = "dap1_fs_pn0",
+ "dap1_din_pn1",
+ "dap1_dout_pn2",
+ "dap1_sclk_pn3";
+ nvidia,function = "i2s0";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ dap_i2s1 {
+ nvidia,pins = "dap2_fs_pa2",
+ "dap2_sclk_pa3",
+ "dap2_din_pa4",
+ "dap2_dout_pa5";
+ nvidia,function = "i2s1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ dap3_fs {
+ nvidia,pins = "dap3_fs_pp0",
+ "dap3_din_pp1";
+ nvidia,function = "i2s2";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ dap3_dout {
+ nvidia,pins = "dap3_dout_pp2",
+ "dap3_sclk_pp3";
+ nvidia,function = "i2s2";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ dap_i2s3 {
+ nvidia,pins = "dap4_fs_pp4",
+ "dap4_din_pp5",
+ "dap4_dout_pp6",
+ "dap4_sclk_pp7";
+ nvidia,function = "i2s3";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ /* Sensors pinmux */
+ nct_irq {
+ nvidia,pins = "pcc2";
+ nvidia,function = "i2s4";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ /* Asus EC pinmux */
+ ec_irqs {
+ nvidia,pins = "kb_row10_ps2",
+ "kb_row15_ps7";
+ nvidia,function = "kbc";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ ec_reqs {
+ nvidia,pins = "kb_col1_pq1";
+ nvidia,function = "kbc";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ /* Memory type bootstrap */
+ mem_boostraps {
+ nvidia,pins = "gmi_ad4_pg4",
+ "gmi_ad5_pg5";
+ nvidia,function = "nand";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ /* PCI-e pinmux */
+ pex_l2_rst_n {
+ nvidia,pins = "pex_l2_rst_n_pcc6",
+ "pex_l0_rst_n_pdd1",
+ "pex_l1_rst_n_pdd5";
+ nvidia,function = "pcie";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ pex_l2_clkreq_n {
+ nvidia,pins = "pex_l2_clkreq_n_pcc7",
+ "pex_l0_prsnt_n_pdd0",
+ "pex_l0_clkreq_n_pdd2",
+ "pex_wake_n_pdd3",
+ "pex_l1_prsnt_n_pdd4",
+ "pex_l1_clkreq_n_pdd6",
+ "pex_l2_prsnt_n_pdd7";
+ nvidia,function = "pcie";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ /* SPI pinmux */
+ spi1_mosi_px4 {
+ nvidia,pins = "spi1_mosi_px4",
+ "spi1_sck_px5",
+ "spi1_cs0_n_px6",
+ "spi1_miso_px7";
+ nvidia,function = "spi1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ spi2_cs1_n_pw2 {
+ nvidia,pins = "spi2_cs1_n_pw2";
+ nvidia,function = "spi2";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ spi2_sck_px2 {
+ nvidia,pins = "spi2_sck_px2";
+ nvidia,function = "spi2";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ gmi_a17_pb0 {
+ nvidia,pins = "gmi_a17_pb0",
+ "gmi_a16_pj7";
+ nvidia,function = "spi4";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ gmi_a18_pb1 {
+ nvidia,pins = "gmi_a18_pb1";
+ nvidia,function = "spi4";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ gmi_a19_pk7 {
+ nvidia,pins = "gmi_a19_pk7";
+ nvidia,function = "spi4";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ /* Display A pinmux */
+ lcd_pwr0_pb2 {
+ nvidia,pins = "lcd_pwr0_pb2",
+ "lcd_pclk_pb3",
+ "lcd_pwr1_pc1",
+ "lcd_d0_pe0",
+ "lcd_d1_pe1",
+ "lcd_d2_pe2",
+ "lcd_d3_pe3",
+ "lcd_d4_pe4",
+ "lcd_d5_pe5",
+ "lcd_d6_pe6",
+ "lcd_d7_pe7",
+ "lcd_d8_pf0",
+ "lcd_d9_pf1",
+ "lcd_d10_pf2",
+ "lcd_d11_pf3",
+ "lcd_d12_pf4",
+ "lcd_d13_pf5",
+ "lcd_d14_pf6",
+ "lcd_d15_pf7",
+ "lcd_de_pj1",
+ "lcd_hsync_pj3",
+ "lcd_vsync_pj4",
+ "lcd_d16_pm0",
+ "lcd_d17_pm1",
+ "lcd_d18_pm2",
+ "lcd_d19_pm3",
+ "lcd_d20_pm4",
+ "lcd_d21_pm5",
+ "lcd_d22_pm6",
+ "lcd_d23_pm7",
+ "lcd_dc0_pn6",
+ "lcd_sdin_pz2";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ lcd_cs0_n_pn4 {
+ nvidia,pins = "lcd_cs0_n_pn4",
+ "lcd_sdout_pn5",
+ "lcd_wr_n_pz3";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ blink {
+ nvidia,pins = "clk_32k_out_pa0";
+ nvidia,function = "blink";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ /* KBC keys */
+ kb_col0_pq0 {
+ nvidia,pins = "kb_col0_pq0";
+ nvidia,function = "kbc";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ kb_col1_pq1 {
+ nvidia,pins = "kb_row1_pr1",
+ "kb_row3_pr3",
+ "kb_row8_ps0",
+ "kb_row14_ps6";
+ nvidia,function = "kbc";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ kb_col4_pq4 {
+ nvidia,pins = "kb_col4_pq4",
+ "kb_col5_pq5",
+ "kb_col7_pq7",
+ "kb_row2_pr2",
+ "kb_row4_pr4",
+ "kb_row5_pr5",
+ "kb_row12_ps4",
+ "kb_row13_ps5";
+ nvidia,function = "kbc";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ gmi_wp_n_pc7 {
+ nvidia,pins = "gmi_wp_n_pc7",
+ "gmi_wait_pi7",
+ "gmi_cs3_n_pk4";
+ nvidia,function = "rsvd1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ gmi_cs0_n_pj0 {
+ nvidia,pins = "gmi_cs0_n_pj0",
+ "gmi_cs1_n_pj2",
+ "gmi_cs2_n_pk3";
+ nvidia,function = "rsvd1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ vi_pclk_pt0 {
+ nvidia,pins = "vi_pclk_pt0";
+ nvidia,function = "rsvd1";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ nvidia,lock = <0>;
+ nvidia,ioreset = <0>;
+ };
+
+ /* GPIO keys pinmux */
+ power_key {
+ nvidia,pins = "pv0";
+ nvidia,function = "rsvd1";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ vol_keys {
+ nvidia,pins = "kb_col2_pq2",
+ "kb_col3_pq3";
+ nvidia,function = "rsvd4";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ /* Bluetooth */
+ bt_shutdown {
+ nvidia,pins = "pu0";
+ nvidia,function = "rsvd4";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ bt_dev_wake {
+ nvidia,pins = "pu1";
+ nvidia,function = "rsvd1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ bt_host_wake {
+ nvidia,pins = "pu6";
+ nvidia,function = "rsvd4";
+ nvidia,pull = <TEGRA_PIN_PULL_DOWN>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ pu2 {
+ nvidia,pins = "pu2";
+ nvidia,function = "rsvd1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ pu3 {
+ nvidia,pins = "pu3";
+ nvidia,function = "rsvd4";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ pcc1 {
+ nvidia,pins = "pcc1";
+ nvidia,function = "rsvd2";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ pv2 {
+ nvidia,pins = "pv2";
+ nvidia,function = "rsvd2";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ pv3 {
+ nvidia,pins = "pv3";
+ nvidia,function = "rsvd2";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ vi_vsync_pd6 {
+ nvidia,pins = "vi_vsync_pd6",
+ "vi_hsync_pd7";
+ nvidia,function = "rsvd2";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ nvidia,lock = <0>;
+ nvidia,ioreset = <0>;
+ };
+
+ vi_d10_pt2 {
+ nvidia,pins = "vi_d10_pt2",
+ "vi_d0_pt4", "pbb0";
+ nvidia,function = "rsvd2";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ kb_row0_pr0 {
+ nvidia,pins = "kb_row0_pr0";
+ nvidia,function = "rsvd4";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ gmi_ad0_pg0 {
+ nvidia,pins = "gmi_ad0_pg0",
+ "gmi_ad1_pg1",
+ "gmi_ad2_pg2",
+ "gmi_ad3_pg3",
+ "gmi_ad6_pg6",
+ "gmi_ad7_pg7",
+ "gmi_wr_n_pi0",
+ "gmi_oe_n_pi1",
+ "gmi_dqs_pi2",
+ "gmi_adv_n_pk0",
+ "gmi_clk_pk1";
+ nvidia,function = "nand";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ gmi_ad13_ph5 {
+ nvidia,pins = "gmi_ad13_ph5";
+ nvidia,function = "nand";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ gmi_ad10_ph2 {
+ nvidia,pins = "gmi_ad10_ph2",
+ "gmi_ad11_ph3",
+ "gmi_ad14_ph6";
+ nvidia,function = "nand";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ gmi_ad12_ph4 {
+ nvidia,pins = "gmi_ad12_ph4",
+ "gmi_rst_n_pi4",
+ "gmi_cs7_n_pi6";
+ nvidia,function = "nand";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ /* Vibrator control */
+ vibrator {
+ nvidia,pins = "gmi_ad15_ph7";
+ nvidia,function = "nand";
+ nvidia,pull = <TEGRA_PIN_PULL_DOWN>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ /* PWM pimnmux */
+ pwm_0 {
+ nvidia,pins = "gmi_ad8_ph0";
+ nvidia,function = "pwm0";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ pwm_2 {
+ nvidia,pins = "pu5";
+ nvidia,function = "pwm2";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ gmi_cs6_n_pi3 {
+ nvidia,pins = "gmi_cs6_n_pi3";
+ nvidia,function = "gmi";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ /* Spdif pinmux */
+ spdif_out {
+ nvidia,pins = "spdif_out_pk5";
+ nvidia,function = "spdif";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ spdif_in {
+ nvidia,pins = "spdif_in_pk6";
+ nvidia,function = "spdif";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ vi_d4_pl2 {
+ nvidia,pins = "vi_d4_pl2";
+ nvidia,function = "vi";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ vi_d6_pl4 {
+ nvidia,pins = "vi_d6_pl4";
+ nvidia,function = "vi";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ nvidia,lock = <0>;
+ nvidia,ioreset = <0>;
+ };
+
+ vi_mclk_pt1 {
+ nvidia,pins = "vi_mclk_pt1";
+ nvidia,function = "vi";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ jtag_rtck {
+ nvidia,pins = "jtag_rtck_pu7";
+ nvidia,function = "rtck";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ crt_hsync_pv6 {
+ nvidia,pins = "crt_hsync_pv6",
+ "crt_vsync_pv7";
+ nvidia,function = "crt";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ clk1_out {
+ nvidia,pins = "clk1_out_pw4";
+ nvidia,function = "extperiph1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ clk2_out {
+ nvidia,pins = "clk2_out_pw5";
+ nvidia,function = "extperiph2";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ clk3_out {
+ nvidia,pins = "clk3_out_pee0";
+ nvidia,function = "extperiph3";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ sys_clk_req {
+ nvidia,pins = "sys_clk_req_pz5";
+ nvidia,function = "sysclk";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ pbb4 {
+ nvidia,pins = "pbb4";
+ nvidia,function = "vgp4";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ pbb5 {
+ nvidia,pins = "pbb5";
+ nvidia,function = "vgp5";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ pbb6 {
+ nvidia,pins = "pbb6";
+ nvidia,function = "vgp6";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ clk2_req_pcc5 {
+ nvidia,pins = "clk2_req_pcc5",
+ "clk1_req_pee2";
+ nvidia,function = "dap";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ clk3_req_pee1 {
+ nvidia,pins = "clk3_req_pee1";
+ nvidia,function = "dev3";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ owr {
+ nvidia,pins = "owr";
+ nvidia,function = "owr";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ /* GPIO power/drive control */
+ drive_dap1 {
+ nvidia,pins = "drive_dap1",
+ "drive_dap2",
+ "drive_dbg",
+ "drive_at5",
+ "drive_gme",
+ "drive_ddc",
+ "drive_ao1",
+ "drive_uart3";
+ nvidia,high-speed-mode = <TEGRA_PIN_DISABLE>;
+ nvidia,schmitt = <TEGRA_PIN_ENABLE>;
+ nvidia,low-power-mode = <TEGRA_PIN_LP_DRIVE_DIV_1>;
+ nvidia,pull-down-strength = <31>;
+ nvidia,pull-up-strength = <31>;
+ nvidia,slew-rate-rising = <TEGRA_PIN_SLEW_RATE_FASTEST>;
+ nvidia,slew-rate-falling = <TEGRA_PIN_SLEW_RATE_FASTEST>;
+ };
+
+ drive_sdio1 {
+ nvidia,pins = "drive_sdio1",
+ "drive_sdio3";
+ nvidia,high-speed-mode = <TEGRA_PIN_DISABLE>;
+ nvidia,schmitt = <TEGRA_PIN_DISABLE>;
+ nvidia,pull-down-strength = <46>;
+ nvidia,pull-up-strength = <42>;
+ nvidia,slew-rate-rising = <TEGRA_PIN_SLEW_RATE_FAST>;
+ nvidia,slew-rate-falling = <TEGRA_PIN_SLEW_RATE_FAST>;
+ };
+ };
+ };
+
+ serial@70006040 {
+ compatible = "nvidia,tegra30-hsuart";
+ status = "okay";
+
+ /* Broadcom GPS BCM47511 */
+ };
+
+ serial@70006200 {
+ compatible = "nvidia,tegra30-hsuart";
+ status = "okay";
+
+ nvidia,adjust-baud-rates = <0 9600 100>,
+ <9600 115200 200>,
+ <1000000 4000000 136>;
+
+ bluetooth {
+ max-speed = <4000000>;
+
+ clocks = <&tegra_pmc TEGRA_PMC_CLK_BLINK>;
+ clock-names = "txco";
+
+ interrupt-parent = <&gpio>;
+ interrupts = <TEGRA_GPIO(U, 6) IRQ_TYPE_EDGE_RISING>;
+ interrupt-names = "host-wakeup";
+
+ device-wakeup-gpios = <&gpio TEGRA_GPIO(U, 1) GPIO_ACTIVE_HIGH>;
+ shutdown-gpios = <&gpio TEGRA_GPIO(U, 0) GPIO_ACTIVE_HIGH>;
+
+ vbat-supply = <&vdd_3v3_com>;
+ vddio-supply = <&vdd_1v8_vio>;
+ };
+ };
+
+ pwm@7000a000 {
+ status = "okay";
+ };
+
+ lcd_ddc: i2c@7000c000 {
+ status = "okay";
+ clock-frequency = <100000>;
+ };
+
+ i2c@7000c400 {
+ status = "okay";
+ clock-frequency = <400000>;
+ };
+
+ i2c@7000c500 {
+ status = "okay";
+
+ /* Aichi AMI306 digital compass */
+ magnetometer@e {
+ compatible = "asahi-kasei,ak8974";
+ reg = <0x0e>;
+
+ avdd-supply = <&vdd_3v3_sys>;
+ dvdd-supply = <&vdd_1v8_vio>;
+ };
+
+ /* Dynaimage ambient light sensor */
+ light-sensor@1c {
+ compatible = "dynaimage,al3010";
+ reg = <0x1c>;
+
+ interrupt-parent = <&gpio>;
+ interrupts = <TEGRA_GPIO(Z, 2) IRQ_TYPE_LEVEL_HIGH>;
+
+ vdd-supply = <&vdd_3v3_sys>;
+ };
+
+ gyroscope@68 {
+ compatible = "invensense,mpu3050";
+ reg = <0x68>;
+
+ interrupt-parent = <&gpio>;
+ interrupts = <TEGRA_GPIO(X, 1) IRQ_TYPE_EDGE_RISING>;
+
+ vdd-supply = <&vdd_3v3_sys>;
+ vlogic-supply = <&vdd_1v8_vio>;
+
+ i2c-gate {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ accelerometer@f {
+ compatible = "kionix,kxtf9";
+ reg = <0x0f>;
+
+ interrupt-parent = <&gpio>;
+ interrupts = <TEGRA_GPIO(O, 5) IRQ_TYPE_EDGE_RISING>;
+
+ vdd-supply = <&vdd_1v8_vio>;
+ vddio-supply = <&vdd_1v8_vio>;
+ };
+ };
+ };
+ };
+
+ hdmi_ddc: i2c@7000c700 {
+ status = "okay";
+ clock-frequency = <93750>;
+ };
+
+ i2c@7000d000 {
+ status = "okay";
+ clock-frequency = <400000>;
+
+ nct72: temperature-sensor@4c {
+ compatible = "onnn,nct1008";
+ reg = <0x4c>;
+
+ interrupt-parent = <&gpio>;
+ interrupts = <TEGRA_GPIO(CC, 2) IRQ_TYPE_EDGE_FALLING>;
+
+ vcc-supply = <&vdd_3v3_sys>;
+ #thermal-sensor-cells = <1>;
+ };
+
+ /* Texas Instruments TPS659110 PMIC */
+ pmic: pmic@2d {
+ compatible = "ti,tps65911";
+ reg = <0x2d>;
+
+ interrupts = <GIC_SPI 86 IRQ_TYPE_LEVEL_HIGH>;
+ #interrupt-cells = <2>;
+ interrupt-controller;
+ wakeup-source;
+
+ ti,en-gpio-sleep = <0 0 1 0 0 0 0 0 0>;
+ ti,system-power-controller;
+ ti,sleep-keep-ck32k;
+ ti,sleep-enable;
+
+ #gpio-cells = <2>;
+ gpio-controller;
+
+ vcc1-supply = <&vdd_5v0_bat>;
+ vcc2-supply = <&vdd_5v0_bat>;
+ vcc3-supply = <&vdd_1v8_vio>;
+ vcc4-supply = <&vdd_5v0_sys>;
+ vcc5-supply = <&vdd_5v0_bat>;
+ vcc6-supply = <&vdd_3v3_sys>;
+ vcc7-supply = <&vdd_5v0_bat>;
+ vccio-supply = <&vdd_5v0_bat>;
+
+ pmic-sleep-hog {
+ gpio-hog;
+ gpios = <2 GPIO_ACTIVE_HIGH>;
+ output-high;
+ };
+
+ regulators {
+ /* VDD1 is not used by Transformers */
+
+ vddio_ddr: vdd2 {
+ regulator-name = "vddio_ddr";
+ regulator-min-microvolt = <1200000>;
+ regulator-max-microvolt = <1200000>;
+ regulator-always-on;
+ regulator-boot-on;
+ };
+
+ vdd_cpu: vddctrl {
+ regulator-name = "vdd_cpu,vdd_sys";
+ regulator-min-microvolt = <600000>;
+ regulator-max-microvolt = <1400000>;
+ regulator-coupled-with = <&vdd_core>;
+ regulator-coupled-max-spread = <300000>;
+ regulator-max-step-microvolt = <100000>;
+ regulator-always-on;
+ regulator-boot-on;
+ ti,regulator-ext-sleep-control = <1>;
+
+ nvidia,tegra-cpu-regulator;
+ };
+
+ vdd_1v8_vio: vio {
+ regulator-name = "vdd_1v8_gen";
+ /* FIXME: eMMC won't work, if set to 1.8 V */
+ regulator-min-microvolt = <1500000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-always-on;
+ regulator-boot-on;
+ };
+
+ /* eMMC VDD */
+ vcore_emmc: ldo1 {
+ regulator-name = "vdd_emmc_core";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-always-on;
+ };
+
+ /* uSD slot VDD */
+ vdd_usd: ldo2 {
+ regulator-name = "vdd_usd";
+ regulator-min-microvolt = <3100000>;
+ regulator-max-microvolt = <3100000>;
+ /* FIXME: Without this, voltage switching fails */
+ regulator-always-on;
+ };
+
+ /* uSD slot VDDIO */
+ vddio_usd: ldo3 {
+ regulator-name = "vddio_usd";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <3100000>;
+ };
+
+ ldo4 {
+ regulator-name = "vdd_rtc";
+ regulator-min-microvolt = <1200000>;
+ regulator-max-microvolt = <1200000>;
+ regulator-always-on;
+ };
+
+ /* LDO5 is not used by Transformers */
+
+ ldo6 {
+ regulator-name = "avdd_dsi_csi,pwrdet_mipi";
+ regulator-min-microvolt = <1200000>;
+ regulator-max-microvolt = <1200000>;
+ };
+
+ ldo7 {
+ regulator-name = "vdd_pllm,x,u,a_p_c_s";
+ regulator-min-microvolt = <1200000>;
+ regulator-max-microvolt = <1200000>;
+ regulator-always-on;
+ regulator-boot-on;
+ ti,regulator-ext-sleep-control = <8>;
+ };
+
+ ldo8 {
+ regulator-name = "vdd_ddr_hs";
+ regulator-min-microvolt = <1000000>;
+ regulator-max-microvolt = <1000000>;
+ regulator-always-on;
+ ti,regulator-ext-sleep-control = <8>;
+ };
+ };
+ };
+
+ vdd_core: core-regulator@60 {
+ compatible = "ti,tps62361";
+ reg = <0x60>;
+
+ regulator-name = "tps62361-vout";
+ regulator-min-microvolt = <500000>;
+ regulator-max-microvolt = <1770000>;
+ regulator-coupled-with = <&vdd_cpu>;
+ regulator-coupled-max-spread = <300000>;
+ regulator-max-step-microvolt = <100000>;
+ regulator-boot-on;
+ regulator-always-on;
+ ti,enable-vout-discharge;
+ ti,vsel0-state-high;
+ ti,vsel1-state-high;
+
+ nvidia,tegra-core-regulator;
+ };
+ };
+
+ pmc@7000e400 {
+ status = "okay";
+ nvidia,invert-interrupt;
+ /* FIXME: LP1 doesn't work at the moment */
+ nvidia,suspend-mode = <2>;
+ nvidia,cpu-pwr-good-time = <2000>;
+ nvidia,cpu-pwr-off-time = <200>;
+ nvidia,core-pwr-good-time = <3845 3845>;
+ nvidia,core-pwr-off-time = <0>;
+ nvidia,core-power-req-active-high;
+ nvidia,sys-clock-req-active-high;
+ core-supply = <&vdd_core>;
+
+ /* Set DEV_OFF + PWR_OFF_SET bit in DCDC control register of TPS65911 PMIC */
+ i2c-thermtrip {
+ nvidia,i2c-controller-id = <4>;
+ nvidia,bus-addr = <0x2d>;
+ nvidia,reg-addr = <0x3f>;
+ nvidia,reg-data = <0x81>;
+ };
+ };
+
+ hda@70030000 {
+ status = "okay";
+ };
+
+ ahub@70080000 {
+ i2s@70080400 { /* i2s1 */
+ status = "okay";
+ };
+
+ /* BT SCO */
+ i2s@70080600 { /* i2s3 */
+ status = "okay";
+ };
+ };
+
+ mmc@78000000 {
+ status = "okay";
+
+ /* FIXME: Full 208Mhz clock rate doesn't work reliably */
+ max-frequency = <104000000>;
+
+ cd-gpios = <&gpio TEGRA_GPIO(I, 5) GPIO_ACTIVE_LOW>;
+ bus-width = <4>;
+
+ vmmc-supply = <&vdd_usd>; /* ldo2 */
+ vqmmc-supply = <&vddio_usd>; /* ldo3 */
+ };
+
+ mmc@78000400 {
+ status = "okay";
+
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ assigned-clocks = <&tegra_car TEGRA30_CLK_SDMMC3>;
+ assigned-clock-parents = <&tegra_car TEGRA30_CLK_PLL_C>;
+ assigned-clock-rates = <50000000>;
+
+ max-frequency = <50000000>;
+ keep-power-in-suspend;
+ bus-width = <4>;
+ non-removable;
+
+ mmc-pwrseq = <&brcm_wifi_pwrseq>;
+ vmmc-supply = <&vdd_3v3_com>;
+ vqmmc-supply = <&vdd_1v8_vio>;
+
+ /* Azurewave AW-NH615 BCM4329B1 or AW-NH665 BCM4330B1 */
+ wifi@1 {
+ compatible = "brcm,bcm4329-fmac";
+ reg = <1>;
+
+ interrupt-parent = <&gpio>;
+ interrupts = <TEGRA_GPIO(O, 4) IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "host-wake";
+ };
+ };
+
+ mmc@78000600 {
+ status = "okay";
+ bus-width = <8>;
+ vmmc-supply = <&vcore_emmc>;
+ vqmmc-supply = <&vdd_1v8_vio>;
+ mmc-ddr-3_3v;
+ non-removable;
+ };
+
+ /* USB via ASUS connector */
+ usb@7d000000 {
+ compatible = "nvidia,tegra30-udc";
+ status = "okay";
+ dr_mode = "peripheral";
+ };
+
+ usb-phy@7d000000 {
+ status = "okay";
+ dr_mode = "peripheral";
+ nvidia,hssync-start-delay = <0>;
+ nvidia,xcvr-lsfslew = <2>;
+ nvidia,xcvr-lsrslew = <2>;
+ vbus-supply = <&vdd_5v0_sys>;
+ };
+
+ /* Dock's USB port */
+ usb@7d008000 {
+ status = "okay";
+ };
+
+ usb-phy@7d008000 {
+ status = "okay";
+ vbus-supply = <&vdd_5v0_bat>;
+ };
+
+ mains: ac-adapter-detect {
+ compatible = "gpio-charger";
+ charger-type = "mains";
+ gpios = <&gpio TEGRA_GPIO(H, 5) GPIO_ACTIVE_HIGH>;
+ };
+
+ backlight: backlight {
+ compatible = "pwm-backlight";
+
+ enable-gpios = <&gpio TEGRA_GPIO(H, 2) GPIO_ACTIVE_HIGH>;
+ power-supply = <&vdd_5v0_bl>;
+ pwms = <&pwm 0 4000000>;
+
+ brightness-levels = <1 255>;
+ num-interpolated-steps = <254>;
+ default-brightness-level = <40>;
+ };
+
+ /* PMIC has a built-in 32KHz oscillator which is used by PMC */
+ clk32k_in: clock-32k {
+ compatible = "fixed-clock";
+ #clock-cells = <0>;
+ clock-frequency = <32768>;
+ clock-output-names = "pmic-oscillator";
+ };
+
+ cpus {
+ cpu0: cpu@0 {
+ cpu-supply = <&vdd_cpu>;
+ operating-points-v2 = <&cpu0_opp_table>;
+ #cooling-cells = <2>;
+ };
+ cpu1: cpu@1 {
+ cpu-supply = <&vdd_cpu>;
+ operating-points-v2 = <&cpu0_opp_table>;
+ #cooling-cells = <2>;
+ };
+ cpu2: cpu@2 {
+ cpu-supply = <&vdd_cpu>;
+ operating-points-v2 = <&cpu0_opp_table>;
+ #cooling-cells = <2>;
+ };
+ cpu3: cpu@3 {
+ cpu-supply = <&vdd_cpu>;
+ operating-points-v2 = <&cpu0_opp_table>;
+ #cooling-cells = <2>;
+ };
+ };
+
+ extcon-keys {
+ compatible = "gpio-keys";
+ interrupt-parent = <&gpio>;
+
+ dock-hall-sensor {
+ label = "Lid sensor";
+ gpios = <&gpio TEGRA_GPIO(S, 6) GPIO_ACTIVE_LOW>;
+ linux,input-type = <EV_SW>;
+ linux,code = <SW_LID>;
+ debounce-interval = <500>;
+ wakeup-event-action = <EV_ACT_ASSERTED>;
+ wakeup-source;
+ };
+
+ lineout-detect {
+ label = "Audio dock line-out detect";
+ gpios = <&gpio TEGRA_GPIO(X, 3) GPIO_ACTIVE_LOW>;
+ linux,input-type = <EV_SW>;
+ linux,code = <SW_LINEOUT_INSERT>;
+ debounce-interval = <10>;
+ wakeup-event-action = <EV_ACT_ASSERTED>;
+ wakeup-source;
+ };
+ };
+
+ firmware {
+ trusted-foundations {
+ compatible = "tlm,trusted-foundations";
+ tlm,version-major = <2>;
+ tlm,version-minor = <8>;
+ };
+ };
+
+ gpio-keys {
+ compatible = "gpio-keys";
+ interrupt-parent = <&gpio>;
+
+ power {
+ label = "Power";
+ gpios = <&gpio TEGRA_GPIO(V, 0) GPIO_ACTIVE_LOW>;
+ linux,code = <KEY_POWER>;
+ debounce-interval = <10>;
+ wakeup-event-action = <EV_ACT_ASSERTED>;
+ wakeup-source;
+ };
+
+ volume-up {
+ label = "Volume Up";
+ gpios = <&gpio TEGRA_GPIO(Q, 2) GPIO_ACTIVE_LOW>;
+ linux,code = <KEY_VOLUMEUP>;
+ debounce-interval = <10>;
+ wakeup-event-action = <EV_ACT_ASSERTED>;
+ wakeup-source;
+ };
+
+ volume-down {
+ label = "Volume Down";
+ gpios = <&gpio TEGRA_GPIO(Q, 3) GPIO_ACTIVE_LOW>;
+ linux,code = <KEY_VOLUMEDOWN>;
+ debounce-interval = <10>;
+ wakeup-event-action = <EV_ACT_ASSERTED>;
+ wakeup-source;
+ };
+ };
+
+ vdd_5v0_bat: regulator-bat {
+ compatible = "regulator-fixed";
+ regulator-name = "vdd_ac_bat";
+ regulator-min-microvolt = <5000000>;
+ regulator-max-microvolt = <5000000>;
+ regulator-always-on;
+ regulator-boot-on;
+ };
+
+ vdd_5v0_cp: regulator-sby {
+ compatible = "regulator-fixed";
+ regulator-name = "vdd_5v0_sby";
+ regulator-min-microvolt = <5000000>;
+ regulator-max-microvolt = <5000000>;
+ regulator-always-on;
+ regulator-boot-on;
+ gpio = <&pmic 0 GPIO_ACTIVE_HIGH>;
+ enable-active-high;
+ vin-supply = <&vdd_5v0_bat>;
+ };
+
+ vdd_5v0_sys: regulator-5v {
+ compatible = "regulator-fixed";
+ regulator-name = "vdd_5v0_sys";
+ regulator-min-microvolt = <5000000>;
+ regulator-max-microvolt = <5000000>;
+ regulator-always-on;
+ regulator-boot-on;
+ gpio = <&pmic 8 GPIO_ACTIVE_HIGH>;
+ enable-active-high;
+ vin-supply = <&vdd_5v0_bat>;
+ };
+
+ vdd_1v5_ddr: regulator-ddr {
+ compatible = "regulator-fixed";
+ regulator-name = "vdd_ddr";
+ regulator-min-microvolt = <1500000>;
+ regulator-max-microvolt = <1500000>;
+ regulator-always-on;
+ regulator-boot-on;
+ gpio = <&pmic 7 GPIO_ACTIVE_HIGH>;
+ enable-active-high;
+ vin-supply = <&vdd_5v0_bat>;
+ };
+
+ vdd_3v3_sys: regulator-3v {
+ compatible = "regulator-fixed";
+ regulator-name = "vdd_3v3_sys";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-always-on;
+ regulator-boot-on;
+ gpio = <&pmic 6 GPIO_ACTIVE_HIGH>;
+ enable-active-high;
+ vin-supply = <&vdd_5v0_bat>;
+ };
+
+ vdd_pnl: regulator-panel {
+ compatible = "regulator-fixed";
+ regulator-name = "vdd_panel";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-enable-ramp-delay = <20000>;
+ gpio = <&gpio TEGRA_GPIO(W, 1) GPIO_ACTIVE_HIGH>;
+ enable-active-high;
+ vin-supply = <&vdd_3v3_sys>;
+ };
+
+ vdd_3v3_com: regulator-com {
+ compatible = "regulator-fixed";
+ regulator-name = "vdd_3v3_com";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-always-on;
+ gpio = <&gpio TEGRA_GPIO(D, 0) GPIO_ACTIVE_HIGH>;
+ enable-active-high;
+ vin-supply = <&vdd_3v3_sys>;
+ };
+
+ vdd_5v0_bl: regulator-bl {
+ compatible = "regulator-fixed";
+ regulator-name = "vdd_5v0_bl";
+ regulator-min-microvolt = <5000000>;
+ regulator-max-microvolt = <5000000>;
+ regulator-boot-on;
+ gpio = <&gpio TEGRA_GPIO(H, 3) GPIO_ACTIVE_HIGH>;
+ enable-active-high;
+ vin-supply = <&vdd_5v0_bat>;
+ };
+
+ hdmi_5v0_sys: regulator-hdmi {
+ compatible = "regulator-fixed";
+ regulator-name = "hdmi_5v0_sys";
+ regulator-min-microvolt = <5000000>;
+ regulator-max-microvolt = <5000000>;
+ gpio = <&gpio TEGRA_GPIO(P, 2) GPIO_ACTIVE_HIGH>;
+ enable-active-high;
+ vin-supply = <&vdd_5v0_sys>;
+ };
+
+ sound {
+ nvidia,i2s-controller = <&tegra_i2s1>;
+
+ nvidia,hp-det-gpios = <&gpio TEGRA_GPIO(W, 2) GPIO_ACTIVE_LOW>;
+ nvidia,hp-mute-gpios = <&gpio TEGRA_GPIO(X, 2) GPIO_ACTIVE_LOW>;
+
+ clocks = <&tegra_car TEGRA30_CLK_PLL_A>,
+ <&tegra_car TEGRA30_CLK_PLL_A_OUT0>,
+ <&tegra_pmc TEGRA_PMC_CLK_OUT_1>;
+ clock-names = "pll_a", "pll_a_out0", "mclk";
+
+ assigned-clocks = <&tegra_car TEGRA30_CLK_EXTERN1>,
+ <&tegra_pmc TEGRA_PMC_CLK_OUT_1>;
+
+ assigned-clock-parents = <&tegra_car TEGRA30_CLK_PLL_A_OUT0>,
+ <&tegra_car TEGRA30_CLK_EXTERN1>;
+ };
+
+ thermal-zones {
+ /*
+ * NCT72 has two sensors:
+ *
+ * 0: internal that monitors ambient/skin temperature
+ * 1: external that is connected to the CPU's diode
+ *
+ * Ideally we should use userspace thermal governor,
+ * but it's a much more complex solution. The "skin"
+ * zone exists as a simpler solution which prevents
+ * Transformers from getting too hot from a user's
+ * tactile perspective. The CPU zone is intended to
+ * protect silicon from damage.
+ */
+
+ skin-thermal {
+ polling-delay-passive = <1000>; /* milliseconds */
+ polling-delay = <5000>; /* milliseconds */
+
+ thermal-sensors = <&nct72 0>;
+
+ trips {
+ trip0: skin-alert {
+ /* throttle at 57C until temperature drops to 56.8C */
+ temperature = <57000>;
+ hysteresis = <200>;
+ type = "passive";
+ };
+
+ trip1: skin-crit {
+ /* shut down at 65C */
+ temperature = <65000>;
+ hysteresis = <2000>;
+ type = "critical";
+ };
+ };
+
+ cooling-maps {
+ map0 {
+ trip = <&trip0>;
+ cooling-device = <&cpu0 THERMAL_NO_LIMIT THERMAL_NO_LIMIT>,
+ <&cpu1 THERMAL_NO_LIMIT THERMAL_NO_LIMIT>,
+ <&cpu2 THERMAL_NO_LIMIT THERMAL_NO_LIMIT>,
+ <&cpu3 THERMAL_NO_LIMIT THERMAL_NO_LIMIT>,
+ <&actmon THERMAL_NO_LIMIT
+ THERMAL_NO_LIMIT>;
+ };
+ };
+ };
+
+ cpu-thermal {
+ polling-delay-passive = <1000>; /* milliseconds */
+ polling-delay = <5000>; /* milliseconds */
+
+ thermal-sensors = <&nct72 1>;
+
+ trips {
+ trip2: cpu-alert {
+ /* throttle at 75C until temperature drops to 74.8C */
+ temperature = <75000>;
+ hysteresis = <200>;
+ type = "passive";
+ };
+
+ trip3: cpu-crit {
+ /* shut down at 90C */
+ temperature = <90000>;
+ hysteresis = <2000>;
+ type = "critical";
+ };
+ };
+
+ cooling-maps {
+ map1 {
+ trip = <&trip2>;
+ cooling-device = <&cpu0 THERMAL_NO_LIMIT THERMAL_NO_LIMIT>,
+ <&cpu1 THERMAL_NO_LIMIT THERMAL_NO_LIMIT>,
+ <&cpu2 THERMAL_NO_LIMIT THERMAL_NO_LIMIT>,
+ <&cpu3 THERMAL_NO_LIMIT THERMAL_NO_LIMIT>,
+ <&actmon THERMAL_NO_LIMIT
+ THERMAL_NO_LIMIT>;
+ };
+ };
+ };
+ };
+
+ brcm_wifi_pwrseq: wifi-pwrseq {
+ compatible = "mmc-pwrseq-simple";
+
+ clocks = <&tegra_pmc TEGRA_PMC_CLK_BLINK>;
+ clock-names = "ext_clock";
+
+ reset-gpios = <&gpio TEGRA_GPIO(D, 4) GPIO_ACTIVE_LOW>;
+ post-power-on-delay-ms = <300>;
+ power-off-delay-us = <300>;
+ };
+};
diff --git a/arch/arm/boot/dts/tegra30-beaver.dts b/arch/arm/boot/dts/tegra30-beaver.dts
index e159fee..5ad62b5 100644
--- a/arch/arm/boot/dts/tegra30-beaver.dts
+++ b/arch/arm/boot/dts/tegra30-beaver.dts
@@ -1898,7 +1898,8 @@ core_vdd_reg: tps62361@60 {
spi@7000da00 {
status = "okay";
spi-max-frequency = <25000000>;
- spi-flash@1 {
+
+ flash@1 {
compatible = "winbond,w25q32", "jedec,spi-nor";
reg = <1>;
spi-max-frequency = <20000000>;
@@ -1915,6 +1916,7 @@ pmc@7000e400 {
nvidia,core-pwr-off-time = <0>;
nvidia,core-power-req-active-high;
nvidia,sys-clock-req-active-high;
+ core-supply = <&core_vdd_reg>;
};
ahub@70080000 {
@@ -1966,7 +1968,7 @@ usb-phy@7d008000 {
status = "okay";
};
- clk32k_in: clock@0 {
+ clk32k_in: clock-32k {
compatible = "fixed-clock";
clock-frequency = <32768>;
#clock-cells = <0>;
@@ -1985,7 +1987,7 @@ gpled2 {
};
};
- vdd_5v_in_reg: regulator@0 {
+ vdd_5v_in_reg: regulator-5v0 {
compatible = "regulator-fixed";
regulator-name = "vdd_5v_in";
regulator-min-microvolt = <5000000>;
@@ -1993,7 +1995,7 @@ vdd_5v_in_reg: regulator@0 {
regulator-always-on;
};
- chargepump_5v_reg: regulator@1 {
+ chargepump_5v_reg: regulator-chargepump {
compatible = "regulator-fixed";
regulator-name = "chargepump_5v";
regulator-min-microvolt = <5000000>;
@@ -2004,7 +2006,7 @@ chargepump_5v_reg: regulator@1 {
gpio = <&pmic 0 GPIO_ACTIVE_HIGH>;
};
- ddr_reg: regulator@2 {
+ ddr_reg: regulator-ddr {
compatible = "regulator-fixed";
regulator-name = "vdd_ddr";
regulator-min-microvolt = <1500000>;
@@ -2016,7 +2018,7 @@ ddr_reg: regulator@2 {
vin-supply = <&vdd_5v_in_reg>;
};
- vdd_5v_sata_reg: regulator@3 {
+ vdd_5v_sata_reg: regulator-sata {
compatible = "regulator-fixed";
regulator-name = "vdd_5v_sata";
regulator-min-microvolt = <5000000>;
@@ -2028,7 +2030,7 @@ vdd_5v_sata_reg: regulator@3 {
vin-supply = <&vdd_5v_in_reg>;
};
- usb1_vbus_reg: regulator@4 {
+ usb1_vbus_reg: regulator-usb1 {
compatible = "regulator-fixed";
regulator-name = "usb1_vbus";
regulator-min-microvolt = <5000000>;
@@ -2039,7 +2041,7 @@ usb1_vbus_reg: regulator@4 {
vin-supply = <&vdd_5v_in_reg>;
};
- usb3_vbus_reg: regulator@5 {
+ usb3_vbus_reg: regulator-usb3 {
compatible = "regulator-fixed";
regulator-name = "usb3_vbus";
regulator-min-microvolt = <5000000>;
@@ -2050,7 +2052,7 @@ usb3_vbus_reg: regulator@5 {
vin-supply = <&vdd_5v_in_reg>;
};
- sys_3v3_reg: regulator@6 {
+ sys_3v3_reg: regulator-3v3 {
compatible = "regulator-fixed";
regulator-name = "sys_3v3,vdd_3v3_alw";
regulator-min-microvolt = <3300000>;
@@ -2062,7 +2064,7 @@ sys_3v3_reg: regulator@6 {
vin-supply = <&vdd_5v_in_reg>;
};
- sys_3v3_pexs_reg: regulator@7 {
+ sys_3v3_pexs_reg: regulator-pexs {
compatible = "regulator-fixed";
regulator-name = "sys_3v3_pexs";
regulator-min-microvolt = <3300000>;
@@ -2074,7 +2076,7 @@ sys_3v3_pexs_reg: regulator@7 {
vin-supply = <&sys_3v3_reg>;
};
- vdd_5v0_hdmi: regulator@8 {
+ vdd_5v0_hdmi: regulator-hdmi {
compatible = "regulator-fixed";
regulator-name = "+VDD_5V_HDMI";
regulator-min-microvolt = <5000000>;
diff --git a/arch/arm/boot/dts/tegra30-cardhu-a02.dts b/arch/arm/boot/dts/tegra30-cardhu-a02.dts
index 4899e05..2471853 100644
--- a/arch/arm/boot/dts/tegra30-cardhu-a02.dts
+++ b/arch/arm/boot/dts/tegra30-cardhu-a02.dts
@@ -16,7 +16,7 @@ mmc@78000400 {
keep-power-in-suspend;
};
- ddr_reg: regulator@100 {
+ ddr_reg: regulator-ddr {
compatible = "regulator-fixed";
regulator-name = "vdd_ddr";
regulator-min-microvolt = <1500000>;
@@ -27,7 +27,7 @@ ddr_reg: regulator@100 {
gpio = <&pmic 6 GPIO_ACTIVE_HIGH>;
};
- sys_3v3_reg: regulator@101 {
+ sys_3v3_reg: regulator-3v3 {
compatible = "regulator-fixed";
regulator-name = "sys_3v3";
regulator-min-microvolt = <3300000>;
@@ -38,7 +38,7 @@ sys_3v3_reg: regulator@101 {
gpio = <&pmic 7 GPIO_ACTIVE_HIGH>;
};
- usb1_vbus_reg: regulator@102 {
+ usb1_vbus_reg: regulator-usb1 {
compatible = "regulator-fixed";
regulator-name = "usb1_vbus";
regulator-min-microvolt = <5000000>;
@@ -49,7 +49,7 @@ usb1_vbus_reg: regulator@102 {
vin-supply = <&vdd_5v0_reg>;
};
- usb3_vbus_reg: regulator@103 {
+ usb3_vbus_reg: regulator-usb3 {
compatible = "regulator-fixed";
regulator-name = "usb3_vbus";
regulator-min-microvolt = <5000000>;
@@ -60,7 +60,7 @@ usb3_vbus_reg: regulator@103 {
vin-supply = <&vdd_5v0_reg>;
};
- vdd_5v0_reg: regulator@104 {
+ vdd_5v0_reg: regulator-5v0 {
compatible = "regulator-fixed";
regulator-name = "5v0";
regulator-min-microvolt = <5000000>;
@@ -69,7 +69,7 @@ vdd_5v0_reg: regulator@104 {
gpio = <&pmic 2 GPIO_ACTIVE_HIGH>;
};
- vdd_bl_reg: regulator@105 {
+ vdd_bl_reg: regulator-bl {
compatible = "regulator-fixed";
regulator-name = "vdd_bl";
regulator-min-microvolt = <5000000>;
diff --git a/arch/arm/boot/dts/tegra30-cardhu-a04.dts b/arch/arm/boot/dts/tegra30-cardhu-a04.dts
index a11028b..2911f08 100644
--- a/arch/arm/boot/dts/tegra30-cardhu-a04.dts
+++ b/arch/arm/boot/dts/tegra30-cardhu-a04.dts
@@ -16,7 +16,7 @@ mmc@78000400 {
keep-power-in-suspend;
};
- ddr_reg: regulator@100 {
+ ddr_reg: regulator-ddr {
compatible = "regulator-fixed";
regulator-name = "ddr";
regulator-min-microvolt = <1500000>;
@@ -27,7 +27,7 @@ ddr_reg: regulator@100 {
gpio = <&pmic 7 GPIO_ACTIVE_HIGH>;
};
- sys_3v3_reg: regulator@101 {
+ sys_3v3_reg: regulator-3v3 {
compatible = "regulator-fixed";
regulator-name = "sys_3v3";
regulator-min-microvolt = <3300000>;
@@ -38,7 +38,7 @@ sys_3v3_reg: regulator@101 {
gpio = <&pmic 6 GPIO_ACTIVE_HIGH>;
};
- usb1_vbus_reg: regulator@102 {
+ usb1_vbus_reg: regulator-usb1 {
compatible = "regulator-fixed";
regulator-name = "usb1_vbus";
regulator-min-microvolt = <5000000>;
@@ -49,7 +49,7 @@ usb1_vbus_reg: regulator@102 {
vin-supply = <&vdd_5v0_reg>;
};
- usb3_vbus_reg: regulator@103 {
+ usb3_vbus_reg: regulator-usb3 {
compatible = "regulator-fixed";
regulator-name = "usb3_vbus";
regulator-min-microvolt = <5000000>;
@@ -60,7 +60,7 @@ usb3_vbus_reg: regulator@103 {
vin-supply = <&vdd_5v0_reg>;
};
- vdd_5v0_reg: regulator@104 {
+ vdd_5v0_reg: regulator-5v0 {
compatible = "regulator-fixed";
regulator-name = "5v0";
regulator-min-microvolt = <5000000>;
@@ -69,7 +69,7 @@ vdd_5v0_reg: regulator@104 {
gpio = <&pmic 8 GPIO_ACTIVE_HIGH>;
};
- vdd_bl_reg: regulator@105 {
+ vdd_bl_reg: regulator-bl {
compatible = "regulator-fixed";
regulator-name = "vdd_bl";
regulator-min-microvolt = <5000000>;
@@ -80,7 +80,7 @@ vdd_bl_reg: regulator@105 {
gpio = <&gpio TEGRA_GPIO(DD, 2) GPIO_ACTIVE_HIGH>;
};
- vdd_bl2_reg: regulator@106 {
+ vdd_bl2_reg: regulator-bl2 {
compatible = "regulator-fixed";
regulator-name = "vdd_bl2";
regulator-min-microvolt = <5000000>;
diff --git a/arch/arm/boot/dts/tegra30-cardhu.dtsi b/arch/arm/boot/dts/tegra30-cardhu.dtsi
index 448f139..ba257ed 100644
--- a/arch/arm/boot/dts/tegra30-cardhu.dtsi
+++ b/arch/arm/boot/dts/tegra30-cardhu.dtsi
@@ -175,6 +175,7 @@ serial@70006000 {
serial@70006200 {
compatible = "nvidia,tegra30-hsuart";
+ /delete-property/ reg-shift;
status = "okay";
};
@@ -209,7 +210,7 @@ i2cmux@70 {
#address-cells = <1>;
#size-cells = <0>;
reg = <0x70>;
- reset-gpio = <&gpio TEGRA_GPIO(BB, 0) GPIO_ACTIVE_LOW>;
+ reset-gpios = <&gpio TEGRA_GPIO(BB, 0) GPIO_ACTIVE_LOW>;
};
};
@@ -374,7 +375,8 @@ vdd_core: tps62361@60 {
spi@7000da00 {
status = "okay";
spi-max-frequency = <25000000>;
- spi-flash@1 {
+
+ flash@1 {
compatible = "winbond,w25q32", "jedec,spi-nor";
reg = <1>;
spi-max-frequency = <20000000>;
@@ -391,6 +393,7 @@ pmc@7000e400 {
nvidia,core-pwr-off-time = <0>;
nvidia,core-power-req-active-high;
nvidia,sys-clock-req-active-high;
+ core-supply = <&vdd_core>;
};
ahub@70080000 {
@@ -433,7 +436,7 @@ backlight: backlight {
default-brightness-level = <6>;
};
- clk32k_in: clock@0 {
+ clk32k_in: clock-32k {
compatible = "fixed-clock";
clock-frequency = <32768>;
#clock-cells = <0>;
@@ -475,7 +478,7 @@ panel: panel {
backlight = <&backlight>;
};
- vdd_ac_bat_reg: regulator@0 {
+ vdd_ac_bat_reg: regulator-acbat {
compatible = "regulator-fixed";
regulator-name = "vdd_ac_bat";
regulator-min-microvolt = <5000000>;
@@ -483,7 +486,7 @@ vdd_ac_bat_reg: regulator@0 {
regulator-always-on;
};
- cam_1v8_reg: regulator@1 {
+ cam_1v8_reg: regulator-cam {
compatible = "regulator-fixed";
regulator-name = "cam_1v8";
regulator-min-microvolt = <1800000>;
@@ -493,7 +496,7 @@ cam_1v8_reg: regulator@1 {
vin-supply = <&vio_reg>;
};
- cp_5v_reg: regulator@2 {
+ cp_5v_reg: regulator-5v0cp {
compatible = "regulator-fixed";
regulator-name = "cp_5v";
regulator-min-microvolt = <5000000>;
@@ -504,7 +507,7 @@ cp_5v_reg: regulator@2 {
gpio = <&pmic 0 GPIO_ACTIVE_HIGH>;
};
- emmc_3v3_reg: regulator@3 {
+ emmc_3v3_reg: regulator-emmc {
compatible = "regulator-fixed";
regulator-name = "emmc_3v3";
regulator-min-microvolt = <3300000>;
@@ -516,7 +519,7 @@ emmc_3v3_reg: regulator@3 {
vin-supply = <&sys_3v3_reg>;
};
- modem_3v3_reg: regulator@4 {
+ modem_3v3_reg: regulator-modem {
compatible = "regulator-fixed";
regulator-name = "modem_3v3";
regulator-min-microvolt = <3300000>;
@@ -525,7 +528,7 @@ modem_3v3_reg: regulator@4 {
gpio = <&gpio TEGRA_GPIO(D, 6) GPIO_ACTIVE_HIGH>;
};
- pex_hvdd_3v3_reg: regulator@5 {
+ pex_hvdd_3v3_reg: regulator-pex {
compatible = "regulator-fixed";
regulator-name = "pex_hvdd_3v3";
regulator-min-microvolt = <3300000>;
@@ -535,7 +538,7 @@ pex_hvdd_3v3_reg: regulator@5 {
vin-supply = <&sys_3v3_reg>;
};
- vdd_cam1_ldo_reg: regulator@6 {
+ vdd_cam1_ldo_reg: regulator-cam1 {
compatible = "regulator-fixed";
regulator-name = "vdd_cam1_ldo";
regulator-min-microvolt = <2800000>;
@@ -545,7 +548,7 @@ vdd_cam1_ldo_reg: regulator@6 {
vin-supply = <&sys_3v3_reg>;
};
- vdd_cam2_ldo_reg: regulator@7 {
+ vdd_cam2_ldo_reg: regulator-cam2 {
compatible = "regulator-fixed";
regulator-name = "vdd_cam2_ldo";
regulator-min-microvolt = <2800000>;
@@ -555,7 +558,7 @@ vdd_cam2_ldo_reg: regulator@7 {
vin-supply = <&sys_3v3_reg>;
};
- vdd_cam3_ldo_reg: regulator@8 {
+ vdd_cam3_ldo_reg: regulator-cam3 {
compatible = "regulator-fixed";
regulator-name = "vdd_cam3_ldo";
regulator-min-microvolt = <3300000>;
@@ -565,7 +568,7 @@ vdd_cam3_ldo_reg: regulator@8 {
vin-supply = <&sys_3v3_reg>;
};
- vdd_com_reg: regulator@9 {
+ vdd_com_reg: regulator-com {
compatible = "regulator-fixed";
regulator-name = "vdd_com";
regulator-min-microvolt = <3300000>;
@@ -577,7 +580,7 @@ vdd_com_reg: regulator@9 {
vin-supply = <&sys_3v3_reg>;
};
- vdd_fuse_3v3_reg: regulator@10 {
+ vdd_fuse_3v3_reg: regulator-fuse {
compatible = "regulator-fixed";
regulator-name = "vdd_fuse_3v3";
regulator-min-microvolt = <3300000>;
@@ -587,7 +590,7 @@ vdd_fuse_3v3_reg: regulator@10 {
vin-supply = <&sys_3v3_reg>;
};
- vdd_pnl1_reg: regulator@11 {
+ vdd_pnl1_reg: regulator-pnl1 {
compatible = "regulator-fixed";
regulator-name = "vdd_pnl1";
regulator-min-microvolt = <3300000>;
@@ -599,7 +602,7 @@ vdd_pnl1_reg: regulator@11 {
vin-supply = <&sys_3v3_reg>;
};
- vdd_vid_reg: regulator@12 {
+ vdd_vid_reg: regulator-vid {
compatible = "regulator-fixed";
regulator-name = "vddio_vid";
regulator-min-microvolt = <5000000>;
diff --git a/arch/arm/boot/dts/tegra30-colibri.dtsi b/arch/arm/boot/dts/tegra30-colibri.dtsi
index 413e352..be691a1 100644
--- a/arch/arm/boot/dts/tegra30-colibri.dtsi
+++ b/arch/arm/boot/dts/tegra30-colibri.dtsi
@@ -701,10 +701,12 @@ pv0 {
serial@70006040 {
compatible = "nvidia,tegra30-hsuart";
+ /delete-property/ reg-shift;
};
serial@70006300 {
compatible = "nvidia,tegra30-hsuart";
+ /delete-property/ reg-shift;
};
hdmi_ddc: i2c@7000c700 {
@@ -765,9 +767,14 @@ vdd1_reg: vdd1 {
vddctrl_reg: vddctrl {
regulator-name = "+V1.0_VDD_CPU";
- regulator-min-microvolt = <1150000>;
- regulator-max-microvolt = <1150000>;
+ regulator-min-microvolt = <800000>;
+ regulator-max-microvolt = <1250000>;
+ regulator-coupled-with = <&vdd_core>;
+ regulator-coupled-max-spread = <300000>;
+ regulator-max-step-microvolt = <100000>;
regulator-always-on;
+
+ nvidia,tegra-cpu-regulator;
};
reg_1v8_vio: vio {
@@ -890,18 +897,20 @@ temp-sensor@4c {
};
/* SW: +V1.2_VDD_CORE */
- regulator@60 {
+ vdd_core: regulator@60 {
compatible = "ti,tps62362";
reg = <0x60>;
regulator-name = "tps62362-vout";
regulator-min-microvolt = <900000>;
regulator-max-microvolt = <1400000>;
+ regulator-coupled-with = <&vddctrl_reg>;
+ regulator-coupled-max-spread = <300000>;
+ regulator-max-step-microvolt = <100000>;
regulator-boot-on;
regulator-always-on;
- ti,vsel0-state-low;
- /* VSEL1: EN_CORE_DVFS_N low for DVFS */
- ti,vsel1-state-low;
+
+ nvidia,tegra-core-regulator;
};
};
@@ -914,6 +923,7 @@ pmc@7000e400 {
nvidia,core-pwr-off-time = <0>;
nvidia,core-power-req-active-high;
nvidia,sys-clock-req-active-high;
+ core-supply = <&vdd_core>;
/* Set DEV_OFF bit in DCDC control register of TPS65911 PMIC */
i2c-thermtrip {
@@ -951,6 +961,7 @@ usb@7d004000 {
#size-cells = <0>;
asix@1 {
+ compatible = "usbb95,772b";
reg = <1>;
local-mac-address = [00 00 00 00 00 00];
};
@@ -1044,7 +1055,7 @@ sound {
};
&gpio {
- lan-reset-n {
+ lan-reset-n-hog {
gpio-hog;
gpios = <TEGRA_GPIO(DD, 0) GPIO_ACTIVE_HIGH>;
output-high;
diff --git a/arch/arm/boot/dts/tegra30-cpu-opp-microvolt.dtsi b/arch/arm/boot/dts/tegra30-cpu-opp-microvolt.dtsi
index 1be715d..b8e0e91 100644
--- a/arch/arm/boot/dts/tegra30-cpu-opp-microvolt.dtsi
+++ b/arch/arm/boot/dts/tegra30-cpu-opp-microvolt.dtsi
@@ -1,288 +1,288 @@
// SPDX-License-Identifier: GPL-2.0
/ {
- cpu0_opp_table: cpu_opp_table0 {
- opp@51000000,800 {
+ cpu0_opp_table: opp-table-cpu0 {
+ opp-51000000-800 {
opp-microvolt = <800000 800000 1250000>;
};
- opp@51000000,850 {
+ opp-51000000-850 {
opp-microvolt = <850000 850000 1250000>;
};
- opp@51000000,912 {
+ opp-51000000-912 {
opp-microvolt = <912000 912000 1250000>;
};
- opp@102000000,800 {
+ opp-102000000-800 {
opp-microvolt = <800000 800000 1250000>;
};
- opp@102000000,850 {
+ opp-102000000-850 {
opp-microvolt = <850000 850000 1250000>;
};
- opp@102000000,912 {
+ opp-102000000-912 {
opp-microvolt = <912000 912000 1250000>;
};
- opp@204000000,800 {
+ opp-204000000-800 {
opp-microvolt = <800000 800000 1250000>;
};
- opp@204000000,850 {
+ opp-204000000-850 {
opp-microvolt = <850000 850000 1250000>;
};
- opp@204000000,912 {
+ opp-204000000-912 {
opp-microvolt = <912000 912000 1250000>;
};
- opp@312000000,850 {
+ opp-312000000-850 {
opp-microvolt = <850000 850000 1250000>;
};
- opp@312000000,912 {
+ opp-312000000-912 {
opp-microvolt = <912000 912000 1250000>;
};
- opp@340000000,800 {
+ opp-340000000-800 {
opp-microvolt = <800000 800000 1250000>;
};
- opp@340000000,850 {
+ opp-340000000-850 {
opp-microvolt = <850000 850000 1250000>;
};
- opp@370000000,800 {
+ opp-370000000-800 {
opp-microvolt = <800000 800000 1250000>;
};
- opp@456000000,850 {
+ opp-456000000-850 {
opp-microvolt = <850000 850000 1250000>;
};
- opp@456000000,912 {
+ opp-456000000-912 {
opp-microvolt = <912000 912000 1250000>;
};
- opp@475000000,800 {
+ opp-475000000-800 {
opp-microvolt = <800000 800000 1250000>;
};
- opp@475000000,850 {
+ opp-475000000-850 {
opp-microvolt = <850000 850000 1250000>;
};
- opp@608000000,850 {
+ opp-608000000-850 {
opp-microvolt = <850000 850000 1250000>;
};
- opp@608000000,912 {
+ opp-608000000-912 {
opp-microvolt = <912000 912000 1250000>;
};
- opp@620000000,850 {
+ opp-620000000-850 {
opp-microvolt = <850000 850000 1250000>;
};
- opp@640000000,850 {
+ opp-640000000-850 {
opp-microvolt = <850000 850000 1250000>;
};
- opp@640000000,900 {
+ opp-640000000-900 {
opp-microvolt = <900000 900000 1250000>;
};
- opp@760000000,850 {
+ opp-760000000-850 {
opp-microvolt = <850000 850000 1250000>;
};
- opp@760000000,900 {
+ opp-760000000-900 {
opp-microvolt = <900000 900000 1250000>;
};
- opp@760000000,912 {
+ opp-760000000-912 {
opp-microvolt = <912000 912000 1250000>;
};
- opp@760000000,975 {
+ opp-760000000-975 {
opp-microvolt = <975000 975000 1250000>;
};
- opp@816000000,850 {
+ opp-816000000-850 {
opp-microvolt = <850000 850000 1250000>;
};
- opp@816000000,912 {
+ opp-816000000-912 {
opp-microvolt = <912000 912000 1250000>;
};
- opp@860000000,850 {
+ opp-860000000-850 {
opp-microvolt = <850000 850000 1250000>;
};
- opp@860000000,900 {
+ opp-860000000-900 {
opp-microvolt = <900000 900000 1250000>;
};
- opp@860000000,975 {
+ opp-860000000-975 {
opp-microvolt = <975000 975000 1250000>;
};
- opp@860000000,1000 {
+ opp-860000000-1000 {
opp-microvolt = <1000000 1000000 1250000>;
};
- opp@910000000,900 {
+ opp-910000000-900 {
opp-microvolt = <900000 900000 1250000>;
};
- opp@1000000000,900 {
+ opp-1000000000-900 {
opp-microvolt = <900000 900000 1250000>;
};
- opp@1000000000,975 {
+ opp-1000000000-975 {
opp-microvolt = <975000 975000 1250000>;
};
- opp@1000000000,1000 {
+ opp-1000000000-1000 {
opp-microvolt = <1000000 1000000 1250000>;
};
- opp@1000000000,1025 {
+ opp-1000000000-1025 {
opp-microvolt = <1025000 1025000 1250000>;
};
- opp@1100000000,900 {
+ opp-1100000000-900 {
opp-microvolt = <900000 900000 1250000>;
};
- opp@1100000000,975 {
+ opp-1100000000-975 {
opp-microvolt = <975000 975000 1250000>;
};
- opp@1100000000,1000 {
+ opp-1100000000-1000 {
opp-microvolt = <1000000 1000000 1250000>;
};
- opp@1100000000,1025 {
+ opp-1100000000-1025 {
opp-microvolt = <1025000 1025000 1250000>;
};
- opp@1100000000,1075 {
+ opp-1100000000-1075 {
opp-microvolt = <1075000 1075000 1250000>;
};
- opp@1150000000,975 {
+ opp-1150000000-975 {
opp-microvolt = <975000 975000 1250000>;
};
- opp@1200000000,975 {
+ opp-1200000000-975 {
opp-microvolt = <975000 975000 1250000>;
};
- opp@1200000000,1000 {
+ opp-1200000000-1000 {
opp-microvolt = <1000000 1000000 1250000>;
};
- opp@1200000000,1025 {
+ opp-1200000000-1025 {
opp-microvolt = <1025000 1025000 1250000>;
};
- opp@1200000000,1050 {
+ opp-1200000000-1050 {
opp-microvolt = <1050000 1050000 1250000>;
};
- opp@1200000000,1075 {
+ opp-1200000000-1075 {
opp-microvolt = <1075000 1075000 1250000>;
};
- opp@1200000000,1100 {
+ opp-1200000000-1100 {
opp-microvolt = <1100000 1100000 1250000>;
};
- opp@1300000000,1000 {
+ opp-1300000000-1000 {
opp-microvolt = <1000000 1000000 1250000>;
};
- opp@1300000000,1025 {
+ opp-1300000000-1025 {
opp-microvolt = <1025000 1025000 1250000>;
};
- opp@1300000000,1050 {
+ opp-1300000000-1050 {
opp-microvolt = <1050000 1050000 1250000>;
};
- opp@1300000000,1075 {
+ opp-1300000000-1075 {
opp-microvolt = <1075000 1075000 1250000>;
};
- opp@1300000000,1100 {
+ opp-1300000000-1100 {
opp-microvolt = <1100000 1100000 1250000>;
};
- opp@1300000000,1125 {
+ opp-1300000000-1125 {
opp-microvolt = <1125000 1125000 1250000>;
};
- opp@1300000000,1150 {
+ opp-1300000000-1150 {
opp-microvolt = <1150000 1150000 1250000>;
};
- opp@1300000000,1175 {
+ opp-1300000000-1175 {
opp-microvolt = <1175000 1175000 1250000>;
};
- opp@1400000000,1100 {
+ opp-1400000000-1100 {
opp-microvolt = <1100000 1100000 1250000>;
};
- opp@1400000000,1125 {
+ opp-1400000000-1125 {
opp-microvolt = <1125000 1125000 1250000>;
};
- opp@1400000000,1150 {
+ opp-1400000000-1150 {
opp-microvolt = <1150000 1150000 1250000>;
};
- opp@1400000000,1175 {
+ opp-1400000000-1175 {
opp-microvolt = <1175000 1175000 1250000>;
};
- opp@1400000000,1237 {
+ opp-1400000000-1237 {
opp-microvolt = <1237000 1237000 1250000>;
};
- opp@1500000000,1125 {
+ opp-1500000000-1125 {
opp-microvolt = <1125000 1125000 1250000>;
};
- opp@1500000000,1150 {
+ opp-1500000000-1150 {
opp-microvolt = <1150000 1150000 1250000>;
};
- opp@1500000000,1200 {
+ opp-1500000000-1200 {
opp-microvolt = <1200000 1200000 1250000>;
};
- opp@1500000000,1237 {
+ opp-1500000000-1237 {
opp-microvolt = <1237000 1237000 1250000>;
};
- opp@1600000000,1212 {
+ opp-1600000000-1212 {
opp-microvolt = <1212000 1212000 1250000>;
};
- opp@1600000000,1237 {
+ opp-1600000000-1237 {
opp-microvolt = <1237000 1237000 1250000>;
};
- opp@1700000000,1212 {
+ opp-1700000000-1212 {
opp-microvolt = <1212000 1212000 1250000>;
};
- opp@1700000000,1237 {
+ opp-1700000000-1237 {
opp-microvolt = <1237000 1237000 1250000>;
};
};
diff --git a/arch/arm/boot/dts/tegra30-cpu-opp.dtsi b/arch/arm/boot/dts/tegra30-cpu-opp.dtsi
index 72f2fe2..5b9ebb7 100644
--- a/arch/arm/boot/dts/tegra30-cpu-opp.dtsi
+++ b/arch/arm/boot/dts/tegra30-cpu-opp.dtsi
@@ -1,116 +1,116 @@
// SPDX-License-Identifier: GPL-2.0
/ {
- cpu0_opp_table: cpu_opp_table0 {
+ cpu0_opp_table: opp-table-cpu0 {
compatible = "operating-points-v2";
opp-shared;
- opp@51000000,800 {
+ opp-51000000-800 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x1F 0x31FE>;
opp-hz = /bits/ 64 <51000000>;
};
- opp@51000000,850 {
+ opp-51000000-850 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x1F 0x0C01>;
opp-hz = /bits/ 64 <51000000>;
};
- opp@51000000,912 {
+ opp-51000000-912 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x1F 0x0200>;
opp-hz = /bits/ 64 <51000000>;
};
- opp@102000000,800 {
+ opp-102000000-800 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x1F 0x31FE>;
opp-hz = /bits/ 64 <102000000>;
};
- opp@102000000,850 {
+ opp-102000000-850 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x1F 0x0C01>;
opp-hz = /bits/ 64 <102000000>;
};
- opp@102000000,912 {
+ opp-102000000-912 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x1F 0x0200>;
opp-hz = /bits/ 64 <102000000>;
};
- opp@204000000,800 {
+ opp-204000000-800 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x1F 0x31FE>;
opp-hz = /bits/ 64 <204000000>;
opp-suspend;
};
- opp@204000000,850 {
+ opp-204000000-850 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x1F 0x0C01>;
opp-hz = /bits/ 64 <204000000>;
opp-suspend;
};
- opp@204000000,912 {
+ opp-204000000-912 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x1F 0x0200>;
opp-hz = /bits/ 64 <204000000>;
opp-suspend;
};
- opp@312000000,850 {
+ opp-312000000-850 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x1F 0x0C00>;
opp-hz = /bits/ 64 <312000000>;
};
- opp@312000000,912 {
+ opp-312000000-912 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x1F 0x0200>;
opp-hz = /bits/ 64 <312000000>;
};
- opp@340000000,800 {
+ opp-340000000-800 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x1F 0x0192>;
opp-hz = /bits/ 64 <340000000>;
};
- opp@340000000,850 {
+ opp-340000000-850 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x0F 0x0001>;
opp-hz = /bits/ 64 <340000000>;
};
- opp@370000000,800 {
+ opp-370000000-800 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x1E 0x306C>;
opp-hz = /bits/ 64 <370000000>;
};
- opp@456000000,850 {
+ opp-456000000-850 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x1F 0x0C00>;
opp-hz = /bits/ 64 <456000000>;
};
- opp@456000000,912 {
+ opp-456000000-912 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x1F 0x0200>;
opp-hz = /bits/ 64 <456000000>;
};
- opp@475000000,800 {
+ opp-475000000-800 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x1E 0x31FE>;
opp-hz = /bits/ 64 <475000000>;
};
- opp@475000000,850 {
+ opp-475000000-850 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x0F 0x0001>, <0x01 0x0002>,
<0x01 0x0010>, <0x01 0x0080>,
@@ -118,25 +118,25 @@ opp@475000000,850 {
opp-hz = /bits/ 64 <475000000>;
};
- opp@608000000,850 {
+ opp-608000000-850 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x1F 0x0400>;
opp-hz = /bits/ 64 <608000000>;
};
- opp@608000000,912 {
+ opp-608000000-912 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x1F 0x0200>;
opp-hz = /bits/ 64 <608000000>;
};
- opp@620000000,850 {
+ opp-620000000-850 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x1E 0x306C>;
opp-hz = /bits/ 64 <620000000>;
};
- opp@640000000,850 {
+ opp-640000000-850 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x0F 0x0001>, <0x02 0x0002>,
<0x04 0x0002>, <0x08 0x0002>,
@@ -149,13 +149,13 @@ opp@640000000,850 {
opp-hz = /bits/ 64 <640000000>;
};
- opp@640000000,900 {
+ opp-640000000-900 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x01 0x0192>;
opp-hz = /bits/ 64 <640000000>;
};
- opp@760000000,850 {
+ opp-760000000-850 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x1E 0x3461>, <0x08 0x0002>,
<0x08 0x0004>, <0x08 0x0008>,
@@ -165,7 +165,7 @@ opp@760000000,850 {
opp-hz = /bits/ 64 <760000000>;
};
- opp@760000000,900 {
+ opp-760000000-900 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x01 0x0001>, <0x02 0x0002>,
<0x04 0x0002>, <0x02 0x0004>,
@@ -177,37 +177,37 @@ opp@760000000,900 {
opp-hz = /bits/ 64 <760000000>;
};
- opp@760000000,912 {
+ opp-760000000-912 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x1F 0x0200>;
opp-hz = /bits/ 64 <760000000>;
};
- opp@760000000,975 {
+ opp-760000000-975 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x01 0x0192>;
opp-hz = /bits/ 64 <760000000>;
};
- opp@816000000,850 {
+ opp-816000000-850 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x1F 0x0400>;
opp-hz = /bits/ 64 <816000000>;
};
- opp@816000000,912 {
+ opp-816000000-912 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x1F 0x0200>;
opp-hz = /bits/ 64 <816000000>;
};
- opp@860000000,850 {
+ opp-860000000-850 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x0C 0x0001>;
opp-hz = /bits/ 64 <860000000>;
};
- opp@860000000,900 {
+ opp-860000000-900 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x02 0x0001>, <0x04 0x0002>,
<0x08 0x0002>, <0x04 0x0004>,
@@ -220,7 +220,7 @@ opp@860000000,900 {
opp-hz = /bits/ 64 <860000000>;
};
- opp@860000000,975 {
+ opp-860000000-975 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x01 0x0001>, <0x02 0x0002>,
<0x02 0x0004>, <0x02 0x0008>,
@@ -229,25 +229,25 @@ opp@860000000,975 {
opp-hz = /bits/ 64 <860000000>;
};
- opp@860000000,1000 {
+ opp-860000000-1000 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x01 0x0192>;
opp-hz = /bits/ 64 <860000000>;
};
- opp@910000000,900 {
+ opp-910000000-900 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x18 0x3060>;
opp-hz = /bits/ 64 <910000000>;
};
- opp@1000000000,900 {
+ opp-1000000000-900 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x0C 0x0001>;
opp-hz = /bits/ 64 <1000000000>;
};
- opp@1000000000,975 {
+ opp-1000000000-975 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x03 0x0001>, <0x04 0x0002>,
<0x08 0x0002>, <0x04 0x0004>,
@@ -260,25 +260,25 @@ opp@1000000000,975 {
opp-hz = /bits/ 64 <1000000000>;
};
- opp@1000000000,1000 {
+ opp-1000000000-1000 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x02 0x019E>;
opp-hz = /bits/ 64 <1000000000>;
};
- opp@1000000000,1025 {
+ opp-1000000000-1025 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x01 0x0192>;
opp-hz = /bits/ 64 <1000000000>;
};
- opp@1100000000,900 {
+ opp-1100000000-900 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x08 0x0001>;
opp-hz = /bits/ 64 <1100000000>;
};
- opp@1100000000,975 {
+ opp-1100000000-975 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x06 0x0001>, <0x08 0x0002>,
<0x08 0x0004>, <0x08 0x0008>,
@@ -288,7 +288,7 @@ opp@1100000000,975 {
opp-hz = /bits/ 64 <1100000000>;
};
- opp@1100000000,1000 {
+ opp-1100000000-1000 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x01 0x0001>, <0x04 0x0002>,
<0x04 0x0004>, <0x04 0x0008>,
@@ -297,31 +297,31 @@ opp@1100000000,1000 {
opp-hz = /bits/ 64 <1100000000>;
};
- opp@1100000000,1025 {
+ opp-1100000000-1025 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x02 0x019E>;
opp-hz = /bits/ 64 <1100000000>;
};
- opp@1100000000,1075 {
+ opp-1100000000-1075 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x01 0x0192>;
opp-hz = /bits/ 64 <1100000000>;
};
- opp@1150000000,975 {
+ opp-1150000000-975 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x18 0x3060>;
opp-hz = /bits/ 64 <1150000000>;
};
- opp@1200000000,975 {
+ opp-1200000000-975 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x08 0x0001>;
opp-hz = /bits/ 64 <1200000000>;
};
- opp@1200000000,1000 {
+ opp-1200000000-1000 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x04 0x0001>, <0x08 0x0002>,
<0x08 0x0004>, <0x08 0x0008>,
@@ -331,7 +331,7 @@ opp@1200000000,1000 {
opp-hz = /bits/ 64 <1200000000>;
};
- opp@1200000000,1025 {
+ opp-1200000000-1025 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x02 0x0001>, <0x04 0x0002>,
<0x04 0x0004>, <0x04 0x0008>,
@@ -340,39 +340,39 @@ opp@1200000000,1025 {
opp-hz = /bits/ 64 <1200000000>;
};
- opp@1200000000,1050 {
+ opp-1200000000-1050 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x02 0x019E>;
opp-hz = /bits/ 64 <1200000000>;
};
- opp@1200000000,1075 {
+ opp-1200000000-1075 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x01 0x0001>;
opp-hz = /bits/ 64 <1200000000>;
};
- opp@1200000000,1100 {
+ opp-1200000000-1100 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x01 0x0192>;
opp-hz = /bits/ 64 <1200000000>;
};
- opp@1300000000,1000 {
+ opp-1300000000-1000 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x08 0x0001>, <0x10 0x0080>,
<0x10 0x0100>;
opp-hz = /bits/ 64 <1300000000>;
};
- opp@1300000000,1025 {
+ opp-1300000000-1025 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x04 0x0001>, <0x08 0x0002>,
<0x08 0x0080>, <0x08 0x0100>;
opp-hz = /bits/ 64 <1300000000>;
};
- opp@1300000000,1050 {
+ opp-1300000000-1050 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x12 0x3061>, <0x04 0x0002>,
<0x08 0x0004>, <0x08 0x0008>,
@@ -383,68 +383,68 @@ opp@1300000000,1050 {
opp-hz = /bits/ 64 <1300000000>;
};
- opp@1300000000,1075 {
+ opp-1300000000-1075 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x02 0x0182>, <0x04 0x0004>,
<0x04 0x0008>, <0x04 0x0010>;
opp-hz = /bits/ 64 <1300000000>;
};
- opp@1300000000,1100 {
+ opp-1300000000-1100 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x02 0x001C>;
opp-hz = /bits/ 64 <1300000000>;
};
- opp@1300000000,1125 {
+ opp-1300000000-1125 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x01 0x0001>;
opp-hz = /bits/ 64 <1300000000>;
};
- opp@1300000000,1150 {
+ opp-1300000000-1150 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x01 0x0182>;
opp-hz = /bits/ 64 <1300000000>;
};
- opp@1300000000,1175 {
+ opp-1300000000-1175 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x01 0x0010>;
opp-hz = /bits/ 64 <1300000000>;
};
- opp@1400000000,1100 {
+ opp-1400000000-1100 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x18 0x307C>;
opp-hz = /bits/ 64 <1400000000>;
};
- opp@1400000000,1125 {
+ opp-1400000000-1125 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x04 0x000C>;
opp-hz = /bits/ 64 <1400000000>;
};
- opp@1400000000,1150 {
+ opp-1400000000-1150 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x02 0x000C>, <0x04 0x0010>;
opp-hz = /bits/ 64 <1400000000>;
};
- opp@1400000000,1175 {
+ opp-1400000000-1175 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x02 0x0010>;
opp-hz = /bits/ 64 <1400000000>;
};
- opp@1400000000,1237 {
+ opp-1400000000-1237 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x01 0x0010>;
opp-hz = /bits/ 64 <1400000000>;
};
- opp@1500000000,1125 {
+ opp-1500000000-1125 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x08 0x0010>, <0x10 0x0020>,
<0x10 0x0040>, <0x10 0x1000>,
@@ -452,7 +452,7 @@ opp@1500000000,1125 {
opp-hz = /bits/ 64 <1500000000>;
};
- opp@1500000000,1150 {
+ opp-1500000000-1150 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x04 0x0010>, <0x08 0x0020>,
<0x08 0x0040>, <0x08 0x1000>,
@@ -460,37 +460,37 @@ opp@1500000000,1150 {
opp-hz = /bits/ 64 <1500000000>;
};
- opp@1500000000,1200 {
+ opp-1500000000-1200 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x02 0x0010>;
opp-hz = /bits/ 64 <1500000000>;
};
- opp@1500000000,1237 {
+ opp-1500000000-1237 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x01 0x0010>;
opp-hz = /bits/ 64 <1500000000>;
};
- opp@1600000000,1212 {
+ opp-1600000000-1212 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x10 0x3060>;
opp-hz = /bits/ 64 <1600000000>;
};
- opp@1600000000,1237 {
+ opp-1600000000-1237 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x08 0x3060>;
opp-hz = /bits/ 64 <1600000000>;
};
- opp@1700000000,1212 {
+ opp-1700000000-1212 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x10 0x3060>;
opp-hz = /bits/ 64 <1700000000>;
};
- opp@1700000000,1237 {
+ opp-1700000000-1237 {
clock-latency-ns = <100000>;
opp-supported-hw = <0x08 0x3060>;
opp-hz = /bits/ 64 <1700000000>;
diff --git a/arch/arm/boot/dts/tegra30-ouya.dts b/arch/arm/boot/dts/tegra30-ouya.dts
index 4259871..a5cfbab 100644
--- a/arch/arm/boot/dts/tegra30-ouya.dts
+++ b/arch/arm/boot/dts/tegra30-ouya.dts
@@ -68,22 +68,1936 @@ hdmi@54280000 {
};
};
- gpio: gpio@6000d000 {
- gpio-ranges = <&pinmux 0 0 248>;
- #reset-cells = <1>;
- };
-
pinmux@70000868 {
pinctrl-names = "default";
pinctrl-0 = <&state_default>;
+
state_default: pinmux {
- /* located at $state_default below */
+ clk_32k_out_pa0 {
+ nvidia,pins = "clk_32k_out_pa0";
+ nvidia,function = "blink";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ uart3_cts_n_pa1 {
+ nvidia,pins = "uart3_cts_n_pa1";
+ nvidia,function = "uartc";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ dap2_fs_pa2 {
+ nvidia,pins = "dap2_fs_pa2";
+ nvidia,function = "i2s1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ dap2_sclk_pa3 {
+ nvidia,pins = "dap2_sclk_pa3";
+ nvidia,function = "i2s1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ dap2_din_pa4 {
+ nvidia,pins = "dap2_din_pa4";
+ nvidia,function = "i2s1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ dap2_dout_pa5 {
+ nvidia,pins = "dap2_dout_pa5";
+ nvidia,function = "i2s1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ sdmmc3_clk_pa6 {
+ nvidia,pins = "sdmmc3_clk_pa6";
+ nvidia,function = "sdmmc3";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ sdmmc3_cmd_pa7 {
+ nvidia,pins = "sdmmc3_cmd_pa7";
+ nvidia,function = "sdmmc3";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ gmi_a17_pb0 {
+ nvidia,pins = "gmi_a17_pb0";
+ nvidia,function = "spi4";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ gmi_a18_pb1 {
+ nvidia,pins = "gmi_a18_pb1";
+ nvidia,function = "spi4";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ lcd_pwr0_pb2 {
+ nvidia,pins = "lcd_pwr0_pb2";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ lcd_pclk_pb3 {
+ nvidia,pins = "lcd_pclk_pb3";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ sdmmc3_dat3_pb4 {
+ nvidia,pins = "sdmmc3_dat3_pb4";
+ nvidia,function = "sdmmc3";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ sdmmc3_dat2_pb5 {
+ nvidia,pins = "sdmmc3_dat2_pb5";
+ nvidia,function = "sdmmc3";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ sdmmc3_dat1_pb6 {
+ nvidia,pins = "sdmmc3_dat1_pb6";
+ nvidia,function = "sdmmc3";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ sdmmc3_dat0_pb7 {
+ nvidia,pins = "sdmmc3_dat0_pb7";
+ nvidia,function = "sdmmc3";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ uart3_rts_n_pc0 {
+ nvidia,pins = "uart3_rts_n_pc0";
+ nvidia,function = "uartc";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ lcd_pwr1_pc1 {
+ nvidia,pins = "lcd_pwr1_pc1";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ uart2_txd_pc2 {
+ nvidia,pins = "uart2_txd_pc2";
+ nvidia,function = "uartb";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ uart2_rxd_pc3 {
+ nvidia,pins = "uart2_rxd_pc3";
+ nvidia,function = "uartb";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ gen1_i2c_scl_pc4 {
+ nvidia,pins = "gen1_i2c_scl_pc4";
+ nvidia,function = "i2c1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ nvidia,open-drain = <TEGRA_PIN_DISABLE>;
+ };
+
+ gen1_i2c_sda_pc5 {
+ nvidia,pins = "gen1_i2c_sda_pc5";
+ nvidia,function = "i2c1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ nvidia,open-drain = <TEGRA_PIN_DISABLE>;
+ };
+
+ lcd_pwr2_pc6 {
+ nvidia,pins = "lcd_pwr2_pc6";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ gmi_wp_n_pc7 {
+ nvidia,pins = "gmi_wp_n_pc7";
+ nvidia,function = "gmi";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ sdmmc3_dat5_pd0 {
+ nvidia,pins = "sdmmc3_dat5_pd0";
+ nvidia,function = "sdmmc3";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ sdmmc3_dat4_pd1 {
+ nvidia,pins = "sdmmc3_dat4_pd1";
+ nvidia,function = "sdmmc3";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ lcd_dc1_pd2 {
+ nvidia,pins = "lcd_dc1_pd2";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ sdmmc3_dat6_pd3 {
+ nvidia,pins = "sdmmc3_dat6_pd3";
+ nvidia,function = "spi4";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ sdmmc3_dat7_pd4 {
+ nvidia,pins = "sdmmc3_dat7_pd4";
+ nvidia,function = "spi4";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ vi_d1_pd5 {
+ nvidia,pins = "vi_d1_pd5";
+ nvidia,function = "sdmmc2";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ vi_vsync_pd6 {
+ nvidia,pins = "vi_vsync_pd6";
+ nvidia,function = "ddr";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ vi_hsync_pd7 {
+ nvidia,pins = "vi_hsync_pd7";
+ nvidia,function = "ddr";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ lcd_d0_pe0 {
+ nvidia,pins = "lcd_d0_pe0";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ lcd_d1_pe1 {
+ nvidia,pins = "lcd_d1_pe1";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ lcd_d2_pe2 {
+ nvidia,pins = "lcd_d2_pe2";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ lcd_d3_pe3 {
+ nvidia,pins = "lcd_d3_pe3";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ lcd_d4_pe4 {
+ nvidia,pins = "lcd_d4_pe4";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ lcd_d5_pe5 {
+ nvidia,pins = "lcd_d5_pe5";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ lcd_d6_pe6 {
+ nvidia,pins = "lcd_d6_pe6";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ lcd_d7_pe7 {
+ nvidia,pins = "lcd_d7_pe7";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ lcd_d8_pf0 {
+ nvidia,pins = "lcd_d8_pf0";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ lcd_d9_pf1 {
+ nvidia,pins = "lcd_d9_pf1";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ lcd_d10_pf2 {
+ nvidia,pins = "lcd_d10_pf2";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ lcd_d11_pf3 {
+ nvidia,pins = "lcd_d11_pf3";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ lcd_d12_pf4 {
+ nvidia,pins = "lcd_d12_pf4";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ lcd_d13_pf5 {
+ nvidia,pins = "lcd_d13_pf5";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ lcd_d14_pf6 {
+ nvidia,pins = "lcd_d14_pf6";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ lcd_d15_pf7 {
+ nvidia,pins = "lcd_d15_pf7";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ gmi_ad0_pg0 {
+ nvidia,pins = "gmi_ad0_pg0";
+ nvidia,function = "nand";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ gmi_ad1_pg1 {
+ nvidia,pins = "gmi_ad1_pg1";
+ nvidia,function = "nand";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ gmi_ad2_pg2 {
+ nvidia,pins = "gmi_ad2_pg2";
+ nvidia,function = "nand";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ gmi_ad3_pg3 {
+ nvidia,pins = "gmi_ad3_pg3";
+ nvidia,function = "nand";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ gmi_ad4_pg4 {
+ nvidia,pins = "gmi_ad4_pg4";
+ nvidia,function = "nand";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ gmi_ad5_pg5 {
+ nvidia,pins = "gmi_ad5_pg5";
+ nvidia,function = "nand";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ gmi_ad6_pg6 {
+ nvidia,pins = "gmi_ad6_pg6";
+ nvidia,function = "nand";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ gmi_ad7_pg7 {
+ nvidia,pins = "gmi_ad7_pg7";
+ nvidia,function = "nand";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ gmi_ad8_ph0 {
+ nvidia,pins = "gmi_ad8_ph0";
+ nvidia,function = "pwm0";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ gmi_ad9_ph1 {
+ nvidia,pins = "gmi_ad9_ph1";
+ nvidia,function = "pwm1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ gmi_ad10_ph2 {
+ nvidia,pins = "gmi_ad10_ph2";
+ nvidia,function = "pwm2";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ gmi_ad11_ph3 {
+ nvidia,pins = "gmi_ad11_ph3";
+ nvidia,function = "nand";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ gmi_ad12_ph4 {
+ nvidia,pins = "gmi_ad12_ph4";
+ nvidia,function = "nand";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ gmi_ad13_ph5 {
+ nvidia,pins = "gmi_ad13_ph5";
+ nvidia,function = "nand";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ gmi_ad14_ph6 {
+ nvidia,pins = "gmi_ad14_ph6";
+ nvidia,function = "nand";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ gmi_wr_n_pi0 {
+ nvidia,pins = "gmi_wr_n_pi0";
+ nvidia,function = "nand";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ gmi_oe_n_pi1 {
+ nvidia,pins = "gmi_oe_n_pi1";
+ nvidia,function = "nand";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ gmi_dqs_pi2 {
+ nvidia,pins = "gmi_dqs_pi2";
+ nvidia,function = "nand";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ gmi_iordy_pi5 {
+ nvidia,pins = "gmi_iordy_pi5";
+ nvidia,function = "rsvd1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ gmi_cs7_n_pi6 {
+ nvidia,pins = "gmi_cs7_n_pi6";
+ nvidia,function = "nand";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ gmi_wait_pi7 {
+ nvidia,pins = "gmi_wait_pi7";
+ nvidia,function = "nand";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ lcd_de_pj1 {
+ nvidia,pins = "lcd_de_pj1";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ gmi_cs1_n_pj2 {
+ nvidia,pins = "gmi_cs1_n_pj2";
+ nvidia,function = "rsvd1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ lcd_hsync_pj3 {
+ nvidia,pins = "lcd_hsync_pj3";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ lcd_vsync_pj4 {
+ nvidia,pins = "lcd_vsync_pj4";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ uart2_cts_n_pj5 {
+ nvidia,pins = "uart2_cts_n_pj5";
+ nvidia,function = "uartb";
+ nvidia,pull = <TEGRA_PIN_PULL_DOWN>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ uart2_rts_n_pj6 {
+ nvidia,pins = "uart2_rts_n_pj6";
+ nvidia,function = "uartb";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ gmi_a16_pj7 {
+ nvidia,pins = "gmi_a16_pj7";
+ nvidia,function = "spi4";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ gmi_adv_n_pk0 {
+ nvidia,pins = "gmi_adv_n_pk0";
+ nvidia,function = "nand";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ gmi_clk_pk1 {
+ nvidia,pins = "gmi_clk_pk1";
+ nvidia,function = "nand";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ gmi_cs2_n_pk3 {
+ nvidia,pins = "gmi_cs2_n_pk3";
+ nvidia,function = "rsvd1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ gmi_cs3_n_pk4 {
+ nvidia,pins = "gmi_cs3_n_pk4";
+ nvidia,function = "nand";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ spdif_out_pk5 {
+ nvidia,pins = "spdif_out_pk5";
+ nvidia,function = "spdif";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ spdif_in_pk6 {
+ nvidia,pins = "spdif_in_pk6";
+ nvidia,function = "spdif";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ gmi_a19_pk7 {
+ nvidia,pins = "gmi_a19_pk7";
+ nvidia,function = "spi4";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ vi_d2_pl0 {
+ nvidia,pins = "vi_d2_pl0";
+ nvidia,function = "sdmmc2";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ vi_d3_pl1 {
+ nvidia,pins = "vi_d3_pl1";
+ nvidia,function = "sdmmc2";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ vi_d4_pl2 {
+ nvidia,pins = "vi_d4_pl2";
+ nvidia,function = "vi";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ vi_d5_pl3 {
+ nvidia,pins = "vi_d5_pl3";
+ nvidia,function = "sdmmc2";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ vi_d6_pl4 {
+ nvidia,pins = "vi_d6_pl4";
+ nvidia,function = "vi";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ vi_d7_pl5 {
+ nvidia,pins = "vi_d7_pl5";
+ nvidia,function = "sdmmc2";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ vi_d8_pl6 {
+ nvidia,pins = "vi_d8_pl6";
+ nvidia,function = "sdmmc2";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ vi_d9_pl7 {
+ nvidia,pins = "vi_d9_pl7";
+ nvidia,function = "sdmmc2";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ lcd_d16_pm0 {
+ nvidia,pins = "lcd_d16_pm0";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ lcd_d17_pm1 {
+ nvidia,pins = "lcd_d17_pm1";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ lcd_d18_pm2 {
+ nvidia,pins = "lcd_d18_pm2";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ lcd_d19_pm3 {
+ nvidia,pins = "lcd_d19_pm3";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ lcd_d20_pm4 {
+ nvidia,pins = "lcd_d20_pm4";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ lcd_d21_pm5 {
+ nvidia,pins = "lcd_d21_pm5";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ lcd_d22_pm6 {
+ nvidia,pins = "lcd_d22_pm6";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ lcd_d23_pm7 {
+ nvidia,pins = "lcd_d23_pm7";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ dap1_fs_pn0 {
+ nvidia,pins = "dap1_fs_pn0";
+ nvidia,function = "i2s0";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ dap1_din_pn1 {
+ nvidia,pins = "dap1_din_pn1";
+ nvidia,function = "i2s0";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ dap1_dout_pn2 {
+ nvidia,pins = "dap1_dout_pn2";
+ nvidia,function = "i2s0";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ dap1_sclk_pn3 {
+ nvidia,pins = "dap1_sclk_pn3";
+ nvidia,function = "i2s0";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ lcd_cs0_n_pn4 {
+ nvidia,pins = "lcd_cs0_n_pn4";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ lcd_sdout_pn5 {
+ nvidia,pins = "lcd_sdout_pn5";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ lcd_dc0_pn6 {
+ nvidia,pins = "lcd_dc0_pn6";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ hdmi_int_pn7 {
+ nvidia,pins = "hdmi_int_pn7";
+ nvidia,function = "hdmi";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ ulpi_data7_po0 {
+ nvidia,pins = "ulpi_data7_po0";
+ nvidia,function = "uarta";
+ nvidia,pull = <TEGRA_PIN_PULL_DOWN>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ ulpi_data0_po1 {
+ nvidia,pins = "ulpi_data0_po1";
+ nvidia,function = "uarta";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ ulpi_data1_po2 {
+ nvidia,pins = "ulpi_data1_po2";
+ nvidia,function = "uarta";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ ulpi_data2_po3 {
+ nvidia,pins = "ulpi_data2_po3";
+ nvidia,function = "uarta";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ ulpi_data3_po4 {
+ nvidia,pins = "ulpi_data3_po4";
+ nvidia,function = "uarta";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ ulpi_data4_po5 {
+ nvidia,pins = "ulpi_data4_po5";
+ nvidia,function = "uarta";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ ulpi_data5_po6 {
+ nvidia,pins = "ulpi_data5_po6";
+ nvidia,function = "uarta";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ ulpi_data6_po7 {
+ nvidia,pins = "ulpi_data6_po7";
+ nvidia,function = "uarta";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ dap3_fs_pp0 {
+ nvidia,pins = "dap3_fs_pp0";
+ nvidia,function = "i2s2";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ dap3_din_pp1 {
+ nvidia,pins = "dap3_din_pp1";
+ nvidia,function = "i2s2";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ dap3_dout_pp2 {
+ nvidia,pins = "dap3_dout_pp2";
+ nvidia,function = "i2s2";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ dap3_sclk_pp3 {
+ nvidia,pins = "dap3_sclk_pp3";
+ nvidia,function = "i2s2";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ dap4_fs_pp4 {
+ nvidia,pins = "dap4_fs_pp4";
+ nvidia,function = "i2s3";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ dap4_din_pp5 {
+ nvidia,pins = "dap4_din_pp5";
+ nvidia,function = "i2s3";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ dap4_dout_pp6 {
+ nvidia,pins = "dap4_dout_pp6";
+ nvidia,function = "i2s3";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ dap4_sclk_pp7 {
+ nvidia,pins = "dap4_sclk_pp7";
+ nvidia,function = "i2s3";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ kb_col0_pq0 {
+ nvidia,pins = "kb_col0_pq0";
+ nvidia,function = "kbc";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ kb_col1_pq1 {
+ nvidia,pins = "kb_col1_pq1";
+ nvidia,function = "kbc";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ kb_col2_pq2 {
+ nvidia,pins = "kb_col2_pq2";
+ nvidia,function = "kbc";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ kb_col3_pq3 {
+ nvidia,pins = "kb_col3_pq3";
+ nvidia,function = "kbc";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ kb_col4_pq4 {
+ nvidia,pins = "kb_col4_pq4";
+ nvidia,function = "kbc";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ kb_col5_pq5 {
+ nvidia,pins = "kb_col5_pq5";
+ nvidia,function = "kbc";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ kb_col6_pq6 {
+ nvidia,pins = "kb_col6_pq6";
+ nvidia,function = "kbc";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ kb_col7_pq7 {
+ nvidia,pins = "kb_col7_pq7";
+ nvidia,function = "kbc";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ kb_row0_pr0 {
+ nvidia,pins = "kb_row0_pr0";
+ nvidia,function = "kbc";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ kb_row1_pr1 {
+ nvidia,pins = "kb_row1_pr1";
+ nvidia,function = "kbc";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ kb_row2_pr2 {
+ nvidia,pins = "kb_row2_pr2";
+ nvidia,function = "kbc";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ kb_row3_pr3 {
+ nvidia,pins = "kb_row3_pr3";
+ nvidia,function = "kbc";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ kb_row4_pr4 {
+ nvidia,pins = "kb_row4_pr4";
+ nvidia,function = "kbc";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ kb_row5_pr5 {
+ nvidia,pins = "kb_row5_pr5";
+ nvidia,function = "kbc";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ kb_row6_pr6 {
+ nvidia,pins = "kb_row6_pr6";
+ nvidia,function = "kbc";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ kb_row7_pr7 {
+ nvidia,pins = "kb_row7_pr7";
+ nvidia,function = "kbc";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ kb_row8_ps0 {
+ nvidia,pins = "kb_row8_ps0";
+ nvidia,function = "kbc";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ kb_row9_ps1 {
+ nvidia,pins = "kb_row9_ps1";
+ nvidia,function = "kbc";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ kb_row10_ps2 {
+ nvidia,pins = "kb_row10_ps2";
+ nvidia,function = "kbc";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ kb_row11_ps3 {
+ nvidia,pins = "kb_row11_ps3";
+ nvidia,function = "kbc";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ kb_row12_ps4 {
+ nvidia,pins = "kb_row12_ps4";
+ nvidia,function = "kbc";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ kb_row13_ps5 {
+ nvidia,pins = "kb_row13_ps5";
+ nvidia,function = "kbc";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ kb_row14_ps6 {
+ nvidia,pins = "kb_row14_ps6";
+ nvidia,function = "kbc";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ kb_row15_ps7 {
+ nvidia,pins = "kb_row15_ps7";
+ nvidia,function = "kbc";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ vi_pclk_pt0 {
+ nvidia,pins = "vi_pclk_pt0";
+ nvidia,function = "rsvd1";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ vi_mclk_pt1 {
+ nvidia,pins = "vi_mclk_pt1";
+ nvidia,function = "vi";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ vi_d10_pt2 {
+ nvidia,pins = "vi_d10_pt2";
+ nvidia,function = "ddr";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ vi_d11_pt3 {
+ nvidia,pins = "vi_d11_pt3";
+ nvidia,function = "ddr";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ vi_d0_pt4 {
+ nvidia,pins = "vi_d0_pt4";
+ nvidia,function = "ddr";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ gen2_i2c_scl_pt5 {
+ nvidia,pins = "gen2_i2c_scl_pt5";
+ nvidia,function = "i2c2";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ nvidia,open-drain = <TEGRA_PIN_DISABLE>;
+ };
+
+ gen2_i2c_sda_pt6 {
+ nvidia,pins = "gen2_i2c_sda_pt6";
+ nvidia,function = "i2c2";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ nvidia,open-drain = <TEGRA_PIN_DISABLE>;
+ };
+
+ sdmmc4_cmd_pt7 {
+ nvidia,pins = "sdmmc4_cmd_pt7";
+ nvidia,function = "sdmmc4";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ nvidia,io-reset = <TEGRA_PIN_DISABLE>;
+ };
+
+ pu0 {
+ nvidia,pins = "pu0";
+ nvidia,function = "owr";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ pu1 {
+ nvidia,pins = "pu1";
+ nvidia,function = "rsvd1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ pu2 {
+ nvidia,pins = "pu2";
+ nvidia,function = "rsvd1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ pu3 {
+ nvidia,pins = "pu3";
+ nvidia,function = "pwm0";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ pu4 {
+ nvidia,pins = "pu4";
+ nvidia,function = "pwm1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ pu5 {
+ nvidia,pins = "pu5";
+ nvidia,function = "rsvd4";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ pu6 {
+ nvidia,pins = "pu6";
+ nvidia,function = "pwm3";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ jtag_rtck_pu7 {
+ nvidia,pins = "jtag_rtck_pu7";
+ nvidia,function = "rtck";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ pv0 {
+ nvidia,pins = "pv0";
+ nvidia,function = "rsvd1";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ pv1 {
+ nvidia,pins = "pv1";
+ nvidia,function = "rsvd1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ pv2 {
+ nvidia,pins = "pv2";
+ nvidia,function = "owr";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ pv3 {
+ nvidia,pins = "pv3";
+ nvidia,function = "clk_12m_out";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ ddc_scl_pv4 {
+ nvidia,pins = "ddc_scl_pv4";
+ nvidia,function = "i2c4";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ ddc_sda_pv5 {
+ nvidia,pins = "ddc_sda_pv5";
+ nvidia,function = "i2c4";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ crt_hsync_pv6 {
+ nvidia,pins = "crt_hsync_pv6";
+ nvidia,function = "crt";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ crt_vsync_pv7 {
+ nvidia,pins = "crt_vsync_pv7";
+ nvidia,function = "crt";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ lcd_cs1_n_pw0 {
+ nvidia,pins = "lcd_cs1_n_pw0";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ lcd_m1_pw1 {
+ nvidia,pins = "lcd_m1_pw1";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_DOWN>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ spi2_cs1_n_pw2 {
+ nvidia,pins = "spi2_cs1_n_pw2";
+ nvidia,function = "spi2";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ clk1_out_pw4 {
+ nvidia,pins = "clk1_out_pw4";
+ nvidia,function = "extperiph1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ clk2_out_pw5 {
+ nvidia,pins = "clk2_out_pw5";
+ nvidia,function = "extperiph2";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ uart3_txd_pw6 {
+ nvidia,pins = "uart3_txd_pw6";
+ nvidia,function = "uartc";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ uart3_rxd_pw7 {
+ nvidia,pins = "uart3_rxd_pw7";
+ nvidia,function = "uartc";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ spi2_sck_px2 {
+ nvidia,pins = "spi2_sck_px2";
+ nvidia,function = "gmi";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ spi1_mosi_px4 {
+ nvidia,pins = "spi1_mosi_px4";
+ nvidia,function = "spi1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ spi1_sck_px5 {
+ nvidia,pins = "spi1_sck_px5";
+ nvidia,function = "spi1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ spi1_cs0_n_px6 {
+ nvidia,pins = "spi1_cs0_n_px6";
+ nvidia,function = "spi1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ spi1_miso_px7 {
+ nvidia,pins = "spi1_miso_px7";
+ nvidia,function = "spi1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ ulpi_clk_py0 {
+ nvidia,pins = "ulpi_clk_py0";
+ nvidia,function = "uartd";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ ulpi_dir_py1 {
+ nvidia,pins = "ulpi_dir_py1";
+ nvidia,function = "uartd";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ ulpi_nxt_py2 {
+ nvidia,pins = "ulpi_nxt_py2";
+ nvidia,function = "uartd";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ ulpi_stp_py3 {
+ nvidia,pins = "ulpi_stp_py3";
+ nvidia,function = "uartd";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ sdmmc1_dat3_py4 {
+ nvidia,pins = "sdmmc1_dat3_py4";
+ nvidia,function = "sdmmc1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ sdmmc1_dat2_py5 {
+ nvidia,pins = "sdmmc1_dat2_py5";
+ nvidia,function = "sdmmc1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ sdmmc1_dat1_py6 {
+ nvidia,pins = "sdmmc1_dat1_py6";
+ nvidia,function = "sdmmc1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ sdmmc1_dat0_py7 {
+ nvidia,pins = "sdmmc1_dat0_py7";
+ nvidia,function = "sdmmc1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ sdmmc1_clk_pz0 {
+ nvidia,pins = "sdmmc1_clk_pz0";
+ nvidia,function = "sdmmc1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ sdmmc1_cmd_pz1 {
+ nvidia,pins = "sdmmc1_cmd_pz1";
+ nvidia,function = "sdmmc1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ lcd_sdin_pz2 {
+ nvidia,pins = "lcd_sdin_pz2";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ lcd_wr_n_pz3 {
+ nvidia,pins = "lcd_wr_n_pz3";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ lcd_sck_pz4 {
+ nvidia,pins = "lcd_sck_pz4";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ sys_clk_req_pz5 {
+ nvidia,pins = "sys_clk_req_pz5";
+ nvidia,function = "sysclk";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ pwr_i2c_scl_pz6 {
+ nvidia,pins = "pwr_i2c_scl_pz6";
+ nvidia,function = "i2cpwr";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ nvidia,open-drain = <TEGRA_PIN_ENABLE>;
+ };
+
+ pwr_i2c_sda_pz7 {
+ nvidia,pins = "pwr_i2c_sda_pz7";
+ nvidia,function = "i2cpwr";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ nvidia,open-drain = <TEGRA_PIN_ENABLE>;
+ };
+
+ sdmmc4_dat0_paa0 {
+ nvidia,pins = "sdmmc4_dat0_paa0";
+ nvidia,function = "sdmmc4";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ nvidia,io-reset = <TEGRA_PIN_DISABLE>;
+ };
+
+ sdmmc4_dat1_paa1 {
+ nvidia,pins = "sdmmc4_dat1_paa1";
+ nvidia,function = "sdmmc4";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ nvidia,io-reset = <TEGRA_PIN_DISABLE>;
+ };
+
+ sdmmc4_dat2_paa2 {
+ nvidia,pins = "sdmmc4_dat2_paa2";
+ nvidia,function = "sdmmc4";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ nvidia,io-reset = <TEGRA_PIN_DISABLE>;
+ };
+
+ sdmmc4_dat3_paa3 {
+ nvidia,pins = "sdmmc4_dat3_paa3";
+ nvidia,function = "sdmmc4";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ nvidia,io-reset = <TEGRA_PIN_DISABLE>;
+ };
+
+ sdmmc4_dat4_paa4 {
+ nvidia,pins = "sdmmc4_dat4_paa4";
+ nvidia,function = "sdmmc4";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ nvidia,io-reset = <TEGRA_PIN_DISABLE>;
+ };
+
+ sdmmc4_dat5_paa5 {
+ nvidia,pins = "sdmmc4_dat5_paa5";
+ nvidia,function = "sdmmc4";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ nvidia,io-reset = <TEGRA_PIN_DISABLE>;
+ };
+
+ sdmmc4_dat6_paa6 {
+ nvidia,pins = "sdmmc4_dat6_paa6";
+ nvidia,function = "sdmmc4";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ nvidia,io-reset = <TEGRA_PIN_DISABLE>;
+ };
+
+ sdmmc4_dat7_paa7 {
+ nvidia,pins = "sdmmc4_dat7_paa7";
+ nvidia,function = "sdmmc4";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ nvidia,io-reset = <TEGRA_PIN_DISABLE>;
+ };
+
+ pbb0 {
+ nvidia,pins = "pbb0";
+ nvidia,function = "i2s4";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ cam_i2c_scl_pbb1 {
+ nvidia,pins = "cam_i2c_scl_pbb1";
+ nvidia,function = "i2c3";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ nvidia,open-drain = <TEGRA_PIN_DISABLE>;
+ };
+
+ cam_i2c_sda_pbb2 {
+ nvidia,pins = "cam_i2c_sda_pbb2";
+ nvidia,function = "i2c3";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ nvidia,open-drain = <TEGRA_PIN_DISABLE>;
+ };
+
+ pbb3 {
+ nvidia,pins = "pbb3";
+ nvidia,function = "vgp3";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ pbb4 {
+ nvidia,pins = "pbb4";
+ nvidia,function = "vgp4";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ pbb5 {
+ nvidia,pins = "pbb5";
+ nvidia,function = "vgp5";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ pbb6 {
+ nvidia,pins = "pbb6";
+ nvidia,function = "vgp6";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ pbb7 {
+ nvidia,pins = "pbb7";
+ nvidia,function = "i2s4";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ cam_mclk_pcc0 {
+ nvidia,pins = "cam_mclk_pcc0";
+ nvidia,function = "vi_alt3";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ pcc1 {
+ nvidia,pins = "pcc1";
+ nvidia,function = "i2s4";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ pcc2 {
+ nvidia,pins = "pcc2";
+ nvidia,function = "i2s4";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ sdmmc4_rst_n_pcc3 {
+ nvidia,pins = "sdmmc4_rst_n_pcc3";
+ nvidia,function = "sdmmc4";
+ nvidia,pull = <TEGRA_PIN_PULL_DOWN>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ nvidia,io-reset = <TEGRA_PIN_DISABLE>;
+ };
+
+ sdmmc4_clk_pcc4 {
+ nvidia,pins = "sdmmc4_clk_pcc4";
+ nvidia,function = "sdmmc4";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ nvidia,io-reset = <TEGRA_PIN_DISABLE>;
+ };
+
+ clk2_req_pcc5 {
+ nvidia,pins = "clk2_req_pcc5";
+ nvidia,function = "dap";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ pex_l2_rst_n_pcc6 {
+ nvidia,pins = "pex_l2_rst_n_pcc6";
+ nvidia,function = "pcie";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ pex_l2_clkreq_n_pcc7 {
+ nvidia,pins = "pex_l2_clkreq_n_pcc7";
+ nvidia,function = "pcie";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ pex_l0_prsnt_n_pdd0 {
+ nvidia,pins = "pex_l0_prsnt_n_pdd0";
+ nvidia,function = "pcie";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ pex_l0_rst_n_pdd1 {
+ nvidia,pins = "pex_l0_rst_n_pdd1";
+ nvidia,function = "pcie";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ pex_l0_clkreq_n_pdd2 {
+ nvidia,pins = "pex_l0_clkreq_n_pdd2";
+ nvidia,function = "pcie";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ pex_wake_n_pdd3 {
+ nvidia,pins = "pex_wake_n_pdd3";
+ nvidia,function = "pcie";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ pex_l1_prsnt_n_pdd4 {
+ nvidia,pins = "pex_l1_prsnt_n_pdd4";
+ nvidia,function = "pcie";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ pex_l1_rst_n_pdd5 {
+ nvidia,pins = "pex_l1_rst_n_pdd5";
+ nvidia,function = "pcie";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ pex_l1_clkreq_n_pdd6 {
+ nvidia,pins = "pex_l1_clkreq_n_pdd6";
+ nvidia,function = "pcie";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ pex_l2_prsnt_n_pdd7 {
+ nvidia,pins = "pex_l2_prsnt_n_pdd7";
+ nvidia,function = "pcie";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ clk3_out_pee0 {
+ nvidia,pins = "clk3_out_pee0";
+ nvidia,function = "extperiph3";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ clk3_req_pee1 {
+ nvidia,pins = "clk3_req_pee1";
+ nvidia,function = "dev3";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ clk1_req_pee2 {
+ nvidia,pins = "clk1_req_pee2";
+ nvidia,function = "dap";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ hdmi_cec_pee3 {
+ nvidia,pins = "hdmi_cec_pee3";
+ nvidia,function = "cec";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ nvidia,open-drain = <TEGRA_PIN_DISABLE>;
+ };
+
+ owr {
+ nvidia,pins = "owr";
+ nvidia,function = "owr";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ drive_groups {
+ nvidia,pins = "drive_gma",
+ "drive_gmb",
+ "drive_gmc",
+ "drive_gmd";
+ nvidia,pull-down-strength = <9>;
+ nvidia,pull-up-strength = <9>;
+ nvidia,slew-rate-rising = <TEGRA_PIN_SLEW_RATE_SLOWEST>;
+ nvidia,slew-rate-falling = <TEGRA_PIN_SLEW_RATE_SLOWEST>;
+ };
};
};
uartc: serial@70006200 {
- status = "okay";
compatible = "nvidia,tegra30-hsuart";
+ /delete-property/ reg-shift;
+ status = "okay";
nvidia,adjust-baud-rates = <0 9600 100>,
<9600 115200 200>,
@@ -277,14 +2191,2294 @@ pmc@7000e400 {
nvidia,core-pwr-off-time = <458>;
nvidia,core-power-req-active-high;
nvidia,sys-clock-req-active-high;
+ core-supply = <&vdd_core>;
};
- mc_timings: memory-controller@7000f000 {
- /* timings located at &mc_timings below */
+ memory-controller@7000f000 {
+ emc-timings-0 {
+ nvidia,ram-code = <0>; /* Samsung RAM */
+
+ timing-25500000 {
+ clock-frequency = <25500000>;
+ nvidia,emem-configuration = <
+ 0x00030003 /* MC_EMEM_ARB_CFG */
+ 0xc0000010 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06020102 /* MC_EMEM_ARB_DA_TURNS */
+ 0x000a0502 /* MC_EMEM_ARB_DA_COVERS */
+ 0x75830303 /* MC_EMEM_ARB_MISC0 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-51000000 {
+ clock-frequency = <51000000>;
+ nvidia,emem-configuration = <
+ 0x00010003 /* MC_EMEM_ARB_CFG */
+ 0xc0000010 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06020102 /* MC_EMEM_ARB_DA_TURNS */
+ 0x000a0502 /* MC_EMEM_ARB_DA_COVERS */
+ 0x74630303 /* MC_EMEM_ARB_MISC0 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-102000000 {
+ clock-frequency = <102000000>;
+ nvidia,emem-configuration = <
+ 0x00000003 /* MC_EMEM_ARB_CFG */
+ 0xc0000018 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06020102 /* MC_EMEM_ARB_DA_TURNS */
+ 0x000a0503 /* MC_EMEM_ARB_DA_COVERS */
+ 0x73c30504 /* MC_EMEM_ARB_MISC0 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-204000000 {
+ clock-frequency = <204000000>;
+ nvidia,emem-configuration = <
+ 0x00000006 /* MC_EMEM_ARB_CFG */
+ 0xc0000025 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000005 /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x00000004 /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06020102 /* MC_EMEM_ARB_DA_TURNS */
+ 0x000a0505 /* MC_EMEM_ARB_DA_COVERS */
+ 0x73840a06 /* MC_EMEM_ARB_MISC0 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-400000000 {
+ clock-frequency = <400000000>;
+ nvidia,emem-configuration = <
+ 0x0000000c /* MC_EMEM_ARB_CFG */
+ 0xc0000048 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000009 /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000005 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x00000007 /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06030202 /* MC_EMEM_ARB_DA_TURNS */
+ 0x000d0709 /* MC_EMEM_ARB_DA_COVERS */
+ 0x7086120a /* MC_EMEM_ARB_MISC0 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-800000000 {
+ clock-frequency = <800000000>;
+ nvidia,emem-configuration = <
+ 0x00000018 /* MC_EMEM_ARB_CFG */
+ 0xc0000090 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000004 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000005 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000013 /* MC_EMEM_ARB_TIMING_RC */
+ 0x0000000c /* MC_EMEM_ARB_TIMING_RAS */
+ 0x0000000f /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x0000000c /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000004 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x08040202 /* MC_EMEM_ARB_DA_TURNS */
+ 0x00160d13 /* MC_EMEM_ARB_DA_COVERS */
+ 0x712c2414 /* MC_EMEM_ARB_MISC0 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+ };
+
+ emc-timings-1 {
+ nvidia,ram-code = <1>; /* Hynix M RAM */
+
+ timing-25500000 {
+ clock-frequency = <25500000>;
+ nvidia,emem-configuration = <
+ 0x00030003 /* MC_EMEM_ARB_CFG */
+ 0xc0000010 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06020102 /* MC_EMEM_ARB_DA_TURNS */
+ 0x000a0502 /* MC_EMEM_ARB_DA_COVERS */
+ 0x75830303 /* MC_EMEM_ARB_MISC0 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-51000000 {
+ clock-frequency = <51000000>;
+ nvidia,emem-configuration = <
+ 0x00010003 /* MC_EMEM_ARB_CFG */
+ 0xc0000010 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06020102 /* MC_EMEM_ARB_DA_TURNS */
+ 0x000a0502 /* MC_EMEM_ARB_DA_COVERS */
+ 0x74630303 /* MC_EMEM_ARB_MISC0 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-102000000 {
+ clock-frequency = <102000000>;
+ nvidia,emem-configuration = <
+ 0x00000003 /* MC_EMEM_ARB_CFG */
+ 0xc0000018 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06020102 /* MC_EMEM_ARB_DA_TURNS */
+ 0x000a0503 /* MC_EMEM_ARB_DA_COVERS */
+ 0x73c30504 /* MC_EMEM_ARB_MISC0 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-204000000 {
+ clock-frequency = <204000000>;
+ nvidia,emem-configuration = <
+ 0x00000006 /* MC_EMEM_ARB_CFG */
+ 0xc0000025 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000005 /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x00000004 /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06020102 /* MC_EMEM_ARB_DA_TURNS */
+ 0x000a0505 /* MC_EMEM_ARB_DA_COVERS */
+ 0x73840a06 /* MC_EMEM_ARB_MISC0 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-400000000 {
+ clock-frequency = <400000000>;
+ nvidia,emem-configuration = <
+ 0x0000000c /* MC_EMEM_ARB_CFG */
+ 0xc0000048 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000009 /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000005 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x00000007 /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06030202 /* MC_EMEM_ARB_DA_TURNS */
+ 0x000d0709 /* MC_EMEM_ARB_DA_COVERS */
+ 0x7086120a /* MC_EMEM_ARB_MISC0 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-800000000 {
+ clock-frequency = <800000000>;
+ nvidia,emem-configuration = <
+ 0x00000018 /* MC_EMEM_ARB_CFG */
+ 0xc0000090 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000004 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000005 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000013 /* MC_EMEM_ARB_TIMING_RC */
+ 0x0000000c /* MC_EMEM_ARB_TIMING_RAS */
+ 0x0000000f /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x0000000c /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000004 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x08040202 /* MC_EMEM_ARB_DA_TURNS */
+ 0x00160d13 /* MC_EMEM_ARB_DA_COVERS */
+ 0x712c2414 /* MC_EMEM_ARB_MISC0 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+ };
+
+ emc-timings-2 {
+ nvidia,ram-code = <2>; /* Hynix A RAM */
+
+ timing-25500000 {
+ clock-frequency = <25500000>;
+ nvidia,emem-configuration = <
+ 0x00030003 /* MC_EMEM_ARB_CFG */
+ 0xc0000010 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06020102 /* MC_EMEM_ARB_DA_TURNS */
+ 0x000a0502 /* MC_EMEM_ARB_DA_COVERS */
+ 0x75e30303 /* MC_EMEM_ARB_MISC0 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-51000000 {
+ clock-frequency = <51000000>;
+ nvidia,emem-configuration = <
+ 0x00010003 /* MC_EMEM_ARB_CFG */
+ 0xc0000010 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06020102 /* MC_EMEM_ARB_DA_TURNS */
+ 0x000a0502 /* MC_EMEM_ARB_DA_COVERS */
+ 0x74e30303 /* MC_EMEM_ARB_MISC0 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-102000000 {
+ clock-frequency = <102000000>;
+ nvidia,emem-configuration = <
+ 0x00000003 /* MC_EMEM_ARB_CFG */
+ 0xc0000018 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06020102 /* MC_EMEM_ARB_DA_TURNS */
+ 0x000a0503 /* MC_EMEM_ARB_DA_COVERS */
+ 0x74430504 /* MC_EMEM_ARB_MISC0 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-204000000 {
+ clock-frequency = <204000000>;
+ nvidia,emem-configuration = <
+ 0x00000006 /* MC_EMEM_ARB_CFG */
+ 0xc0000025 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000005 /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x00000004 /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06020102 /* MC_EMEM_ARB_DA_TURNS */
+ 0x000a0505 /* MC_EMEM_ARB_DA_COVERS */
+ 0x74040a06 /* MC_EMEM_ARB_MISC0 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-400000000 {
+ clock-frequency = <400000000>;
+ nvidia,emem-configuration = <
+ 0x0000000c /* MC_EMEM_ARB_CFG */
+ 0xc0000048 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000009 /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000005 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x00000007 /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06030202 /* MC_EMEM_ARB_DA_TURNS */
+ 0x000d0709 /* MC_EMEM_ARB_DA_COVERS */
+ 0x7086120a /* MC_EMEM_ARB_MISC0 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+
+ timing-800000000 {
+ clock-frequency = <800000000>;
+ nvidia,emem-configuration = <
+ 0x00000018 /* MC_EMEM_ARB_CFG */
+ 0xc0000090 /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000004 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000005 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000013 /* MC_EMEM_ARB_TIMING_RC */
+ 0x0000000c /* MC_EMEM_ARB_TIMING_RAS */
+ 0x0000000f /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x0000000c /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000004 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x08040202 /* MC_EMEM_ARB_DA_TURNS */
+ 0x00160d13 /* MC_EMEM_ARB_DA_COVERS */
+ 0x712c2414 /* MC_EMEM_ARB_MISC0 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+ };
};
- emc_timings: memory-controller@7000f400 {
- /* timings located at &emc_timings below */
+ memory-controller@7000f400 {
+ emc-timings-0 {
+ nvidia,ram-code = <0>; /* Samsung RAM */
+
+ timing-25500000 {
+ clock-frequency = <25500000>;
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200008>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-periodic-qrst;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-configuration = <
+ 0x00000001 /* EMC_RC */
+ 0x00000006 /* EMC_RFC */
+ 0x00000000 /* EMC_RAS */
+ 0x00000000 /* EMC_RP */
+ 0x00000002 /* EMC_R2W */
+ 0x0000000a /* EMC_W2R */
+ 0x00000005 /* EMC_R2P */
+ 0x0000000b /* EMC_W2P */
+ 0x00000000 /* EMC_RD_RCD */
+ 0x00000000 /* EMC_WR_RCD */
+ 0x00000003 /* EMC_RRD */
+ 0x00000001 /* EMC_REXT */
+ 0x00000000 /* EMC_WEXT */
+ 0x00000005 /* EMC_WDV */
+ 0x00000005 /* EMC_QUSE */
+ 0x00000004 /* EMC_QRST */
+ 0x0000000a /* EMC_QSAFE */
+ 0x0000000b /* EMC_RDV */
+ 0x000000c0 /* EMC_REFRESH */
+ 0x00000000 /* EMC_BURST_REFRESH_NUM */
+ 0x00000030 /* EMC_PRE_REFRESH_REQ_CNT */
+ 0x00000002 /* EMC_PDEX2WR */
+ 0x00000002 /* EMC_PDEX2RD */
+ 0x00000001 /* EMC_PCHG2PDEN */
+ 0x00000000 /* EMC_ACT2PDEN */
+ 0x00000007 /* EMC_AR2PDEN */
+ 0x0000000f /* EMC_RW2PDEN */
+ 0x00000007 /* EMC_TXSR */
+ 0x00000007 /* EMC_TXSRDLL */
+ 0x00000004 /* EMC_TCKE */
+ 0x00000002 /* EMC_TFAW */
+ 0x00000000 /* EMC_TRPAB */
+ 0x00000004 /* EMC_TCLKSTABLE */
+ 0x00000005 /* EMC_TCLKSTOP */
+ 0x000000c7 /* EMC_TREFBW */
+ 0x00000006 /* EMC_QUSE_EXTRA */
+ 0x00000004 /* EMC_FBIO_CFG6 */
+ 0x00000000 /* EMC_ODT_WRITE */
+ 0x00000000 /* EMC_ODT_READ */
+ 0x00004288 /* EMC_FBIO_CFG5 */
+ 0x007800a4 /* EMC_CFG_DIG_DLL */
+ 0x00008000 /* EMC_CFG_DIG_DLL_PERIOD */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS0 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS1 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS2 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS3 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS4 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS5 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS6 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS7 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE0 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE1 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE2 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE3 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE4 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE5 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE6 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE7 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS0 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS1 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS2 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS3 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS4 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS5 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS6 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS7 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQ0 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQ1 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQ2 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQ3 */
+ 0x000002a0 /* EMC_XM2CMDPADCTRL */
+ 0x0800211c /* EMC_XM2DQSPADCTRL2 */
+ 0x00000000 /* EMC_XM2DQPADCTRL2 */
+ 0x77fff884 /* EMC_XM2CLKPADCTRL */
+ 0x01f1f108 /* EMC_XM2COMPPADCTRL */
+ 0x05057404 /* EMC_XM2VTTGENPADCTRL */
+ 0x54000007 /* EMC_XM2VTTGENPADCTRL2 */
+ 0x08000168 /* EMC_XM2QUSEPADCTRL */
+ 0x08000000 /* EMC_XM2DQSPADCTRL3 */
+ 0x00000802 /* EMC_CTT_TERM_CTRL */
+ 0x00000000 /* EMC_ZCAL_INTERVAL */
+ 0x00000040 /* EMC_ZCAL_WAIT_CNT */
+ 0x000c000c /* EMC_MRS_WAIT_CNT */
+ 0xa0f10000 /* EMC_AUTO_CAL_CONFIG */
+ 0x00000000 /* EMC_CTT */
+ 0x00000000 /* EMC_CTT_DURATION */
+ 0x80000287 /* EMC_DYN_SELF_REF_CONTROL */
+ 0xe8000000 /* EMC_FBIO_SPARE */
+ 0xff00ff00 /* EMC_CFG_RSV */
+ >;
+ };
+
+ timing-51000000 {
+ clock-frequency = <51000000>;
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200008>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-periodic-qrst;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-configuration = <
+ 0x00000002 /* EMC_RC */
+ 0x0000000d /* EMC_RFC */
+ 0x00000001 /* EMC_RAS */
+ 0x00000000 /* EMC_RP */
+ 0x00000002 /* EMC_R2W */
+ 0x0000000a /* EMC_W2R */
+ 0x00000005 /* EMC_R2P */
+ 0x0000000b /* EMC_W2P */
+ 0x00000000 /* EMC_RD_RCD */
+ 0x00000000 /* EMC_WR_RCD */
+ 0x00000003 /* EMC_RRD */
+ 0x00000001 /* EMC_REXT */
+ 0x00000000 /* EMC_WEXT */
+ 0x00000005 /* EMC_WDV */
+ 0x00000005 /* EMC_QUSE */
+ 0x00000004 /* EMC_QRST */
+ 0x0000000a /* EMC_QSAFE */
+ 0x0000000b /* EMC_RDV */
+ 0x00000181 /* EMC_REFRESH */
+ 0x00000000 /* EMC_BURST_REFRESH_NUM */
+ 0x00000060 /* EMC_PRE_REFRESH_REQ_CNT */
+ 0x00000002 /* EMC_PDEX2WR */
+ 0x00000002 /* EMC_PDEX2RD */
+ 0x00000001 /* EMC_PCHG2PDEN */
+ 0x00000000 /* EMC_ACT2PDEN */
+ 0x00000007 /* EMC_AR2PDEN */
+ 0x0000000f /* EMC_RW2PDEN */
+ 0x0000000e /* EMC_TXSR */
+ 0x0000000e /* EMC_TXSRDLL */
+ 0x00000004 /* EMC_TCKE */
+ 0x00000003 /* EMC_TFAW */
+ 0x00000000 /* EMC_TRPAB */
+ 0x00000004 /* EMC_TCLKSTABLE */
+ 0x00000005 /* EMC_TCLKSTOP */
+ 0x0000018e /* EMC_TREFBW */
+ 0x00000006 /* EMC_QUSE_EXTRA */
+ 0x00000004 /* EMC_FBIO_CFG6 */
+ 0x00000000 /* EMC_ODT_WRITE */
+ 0x00000000 /* EMC_ODT_READ */
+ 0x00004288 /* EMC_FBIO_CFG5 */
+ 0x007800a4 /* EMC_CFG_DIG_DLL */
+ 0x00008000 /* EMC_CFG_DIG_DLL_PERIOD */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS0 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS1 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS2 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS3 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS4 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS5 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS6 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS7 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE0 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE1 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE2 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE3 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE4 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE5 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE6 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE7 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS0 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS1 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS2 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS3 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS4 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS5 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS6 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS7 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQ0 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQ1 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQ2 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQ3 */
+ 0x000002a0 /* EMC_XM2CMDPADCTRL */
+ 0x0800211c /* EMC_XM2DQSPADCTRL2 */
+ 0x00000000 /* EMC_XM2DQPADCTRL2 */
+ 0x77fff884 /* EMC_XM2CLKPADCTRL */
+ 0x01f1f108 /* EMC_XM2COMPPADCTRL */
+ 0x05057404 /* EMC_XM2VTTGENPADCTRL */
+ 0x54000007 /* EMC_XM2VTTGENPADCTRL2 */
+ 0x08000168 /* EMC_XM2QUSEPADCTRL */
+ 0x08000000 /* EMC_XM2DQSPADCTRL3 */
+ 0x00000802 /* EMC_CTT_TERM_CTRL */
+ 0x00000000 /* EMC_ZCAL_INTERVAL */
+ 0x00000040 /* EMC_ZCAL_WAIT_CNT */
+ 0x000c000c /* EMC_MRS_WAIT_CNT */
+ 0xa0f10000 /* EMC_AUTO_CAL_CONFIG */
+ 0x00000000 /* EMC_CTT */
+ 0x00000000 /* EMC_CTT_DURATION */
+ 0x8000040b /* EMC_DYN_SELF_REF_CONTROL */
+ 0xe8000000 /* EMC_FBIO_SPARE */
+ 0xff00ff00 /* EMC_CFG_RSV */
+ >;
+ };
+
+ timing-102000000 {
+ clock-frequency = <102000000>;
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200008>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-periodic-qrst;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-configuration = <
+ 0x00000004 /* EMC_RC */
+ 0x0000001a /* EMC_RFC */
+ 0x00000003 /* EMC_RAS */
+ 0x00000001 /* EMC_RP */
+ 0x00000002 /* EMC_R2W */
+ 0x0000000a /* EMC_W2R */
+ 0x00000005 /* EMC_R2P */
+ 0x0000000b /* EMC_W2P */
+ 0x00000001 /* EMC_RD_RCD */
+ 0x00000001 /* EMC_WR_RCD */
+ 0x00000003 /* EMC_RRD */
+ 0x00000001 /* EMC_REXT */
+ 0x00000000 /* EMC_WEXT */
+ 0x00000005 /* EMC_WDV */
+ 0x00000005 /* EMC_QUSE */
+ 0x00000004 /* EMC_QRST */
+ 0x0000000a /* EMC_QSAFE */
+ 0x0000000b /* EMC_RDV */
+ 0x00000303 /* EMC_REFRESH */
+ 0x00000000 /* EMC_BURST_REFRESH_NUM */
+ 0x000000c0 /* EMC_PRE_REFRESH_REQ_CNT */
+ 0x00000002 /* EMC_PDEX2WR */
+ 0x00000002 /* EMC_PDEX2RD */
+ 0x00000001 /* EMC_PCHG2PDEN */
+ 0x00000000 /* EMC_ACT2PDEN */
+ 0x00000007 /* EMC_AR2PDEN */
+ 0x0000000f /* EMC_RW2PDEN */
+ 0x0000001c /* EMC_TXSR */
+ 0x0000001c /* EMC_TXSRDLL */
+ 0x00000004 /* EMC_TCKE */
+ 0x00000005 /* EMC_TFAW */
+ 0x00000000 /* EMC_TRPAB */
+ 0x00000004 /* EMC_TCLKSTABLE */
+ 0x00000005 /* EMC_TCLKSTOP */
+ 0x0000031c /* EMC_TREFBW */
+ 0x00000006 /* EMC_QUSE_EXTRA */
+ 0x00000004 /* EMC_FBIO_CFG6 */
+ 0x00000000 /* EMC_ODT_WRITE */
+ 0x00000000 /* EMC_ODT_READ */
+ 0x00004288 /* EMC_FBIO_CFG5 */
+ 0x007800a4 /* EMC_CFG_DIG_DLL */
+ 0x00008000 /* EMC_CFG_DIG_DLL_PERIOD */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS0 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS1 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS2 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS3 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS4 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS5 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS6 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS7 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE0 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE1 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE2 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE3 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE4 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE5 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE6 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE7 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS0 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS1 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS2 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS3 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS4 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS5 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS6 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS7 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQ0 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQ1 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQ2 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQ3 */
+ 0x000002a0 /* EMC_XM2CMDPADCTRL */
+ 0x0800211c /* EMC_XM2DQSPADCTRL2 */
+ 0x00000000 /* EMC_XM2DQPADCTRL2 */
+ 0x77fff884 /* EMC_XM2CLKPADCTRL */
+ 0x01f1f108 /* EMC_XM2COMPPADCTRL */
+ 0x05057404 /* EMC_XM2VTTGENPADCTRL */
+ 0x54000007 /* EMC_XM2VTTGENPADCTRL2 */
+ 0x08000168 /* EMC_XM2QUSEPADCTRL */
+ 0x08000000 /* EMC_XM2DQSPADCTRL3 */
+ 0x00000802 /* EMC_CTT_TERM_CTRL */
+ 0x00000000 /* EMC_ZCAL_INTERVAL */
+ 0x00000040 /* EMC_ZCAL_WAIT_CNT */
+ 0x000c000c /* EMC_MRS_WAIT_CNT */
+ 0xa0f10000 /* EMC_AUTO_CAL_CONFIG */
+ 0x00000000 /* EMC_CTT */
+ 0x00000000 /* EMC_CTT_DURATION */
+ 0x80000713 /* EMC_DYN_SELF_REF_CONTROL */
+ 0xe8000000 /* EMC_FBIO_SPARE */
+ 0xff00ff00 /* EMC_CFG_RSV */
+ >;
+ };
+
+ timing-204000000 {
+ clock-frequency = <204000000>;
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200008>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-periodic-qrst;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-configuration = <
+ 0x00000009 /* EMC_RC */
+ 0x00000035 /* EMC_RFC */
+ 0x00000007 /* EMC_RAS */
+ 0x00000002 /* EMC_RP */
+ 0x00000002 /* EMC_R2W */
+ 0x0000000a /* EMC_W2R */
+ 0x00000005 /* EMC_R2P */
+ 0x0000000b /* EMC_W2P */
+ 0x00000002 /* EMC_RD_RCD */
+ 0x00000002 /* EMC_WR_RCD */
+ 0x00000003 /* EMC_RRD */
+ 0x00000001 /* EMC_REXT */
+ 0x00000000 /* EMC_WEXT */
+ 0x00000005 /* EMC_WDV */
+ 0x00000005 /* EMC_QUSE */
+ 0x00000004 /* EMC_QRST */
+ 0x0000000a /* EMC_QSAFE */
+ 0x0000000b /* EMC_RDV */
+ 0x00000607 /* EMC_REFRESH */
+ 0x00000000 /* EMC_BURST_REFRESH_NUM */
+ 0x00000181 /* EMC_PRE_REFRESH_REQ_CNT */
+ 0x00000002 /* EMC_PDEX2WR */
+ 0x00000002 /* EMC_PDEX2RD */
+ 0x00000001 /* EMC_PCHG2PDEN */
+ 0x00000000 /* EMC_ACT2PDEN */
+ 0x00000007 /* EMC_AR2PDEN */
+ 0x0000000f /* EMC_RW2PDEN */
+ 0x00000038 /* EMC_TXSR */
+ 0x00000038 /* EMC_TXSRDLL */
+ 0x00000004 /* EMC_TCKE */
+ 0x00000009 /* EMC_TFAW */
+ 0x00000000 /* EMC_TRPAB */
+ 0x00000004 /* EMC_TCLKSTABLE */
+ 0x00000005 /* EMC_TCLKSTOP */
+ 0x00000638 /* EMC_TREFBW */
+ 0x00000006 /* EMC_QUSE_EXTRA */
+ 0x00000006 /* EMC_FBIO_CFG6 */
+ 0x00000000 /* EMC_ODT_WRITE */
+ 0x00000000 /* EMC_ODT_READ */
+ 0x00004288 /* EMC_FBIO_CFG5 */
+ 0x004400a4 /* EMC_CFG_DIG_DLL */
+ 0x00008000 /* EMC_CFG_DIG_DLL_PERIOD */
+ 0x00080000 /* EMC_DLL_XFORM_DQS0 */
+ 0x00080000 /* EMC_DLL_XFORM_DQS1 */
+ 0x00080000 /* EMC_DLL_XFORM_DQS2 */
+ 0x00080000 /* EMC_DLL_XFORM_DQS3 */
+ 0x00080000 /* EMC_DLL_XFORM_DQS4 */
+ 0x00080000 /* EMC_DLL_XFORM_DQS5 */
+ 0x00080000 /* EMC_DLL_XFORM_DQS6 */
+ 0x00080000 /* EMC_DLL_XFORM_DQS7 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE0 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE1 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE2 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE3 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE4 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE5 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE6 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE7 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS0 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS1 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS2 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS3 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS4 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS5 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS6 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS7 */
+ 0x00080000 /* EMC_DLL_XFORM_DQ0 */
+ 0x00080000 /* EMC_DLL_XFORM_DQ1 */
+ 0x00080000 /* EMC_DLL_XFORM_DQ2 */
+ 0x00080000 /* EMC_DLL_XFORM_DQ3 */
+ 0x000002a0 /* EMC_XM2CMDPADCTRL */
+ 0x0800211c /* EMC_XM2DQSPADCTRL2 */
+ 0x00000000 /* EMC_XM2DQPADCTRL2 */
+ 0x77fff884 /* EMC_XM2CLKPADCTRL */
+ 0x01f1f108 /* EMC_XM2COMPPADCTRL */
+ 0x05057404 /* EMC_XM2VTTGENPADCTRL */
+ 0x54000007 /* EMC_XM2VTTGENPADCTRL2 */
+ 0x08000168 /* EMC_XM2QUSEPADCTRL */
+ 0x08000000 /* EMC_XM2DQSPADCTRL3 */
+ 0x00000802 /* EMC_CTT_TERM_CTRL */
+ 0x00020000 /* EMC_ZCAL_INTERVAL */
+ 0x00000100 /* EMC_ZCAL_WAIT_CNT */
+ 0x000c000c /* EMC_MRS_WAIT_CNT */
+ 0xa0f10000 /* EMC_AUTO_CAL_CONFIG */
+ 0x00000000 /* EMC_CTT */
+ 0x00000000 /* EMC_CTT_DURATION */
+ 0x80000d22 /* EMC_DYN_SELF_REF_CONTROL */
+ 0xe8000000 /* EMC_FBIO_SPARE */
+ 0xff00ff00 /* EMC_CFG_RSV */
+ >;
+ };
+
+ timing-400000000 {
+ clock-frequency = <400000000>;
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100002>;
+ nvidia,emc-mode-2 = <0x80200000>;
+ nvidia,emc-mode-reset = <0x80000521>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-configuration = <
+ 0x00000012 /* EMC_RC */
+ 0x00000066 /* EMC_RFC */
+ 0x0000000c /* EMC_RAS */
+ 0x00000004 /* EMC_RP */
+ 0x00000003 /* EMC_R2W */
+ 0x00000008 /* EMC_W2R */
+ 0x00000002 /* EMC_R2P */
+ 0x0000000a /* EMC_W2P */
+ 0x00000004 /* EMC_RD_RCD */
+ 0x00000004 /* EMC_WR_RCD */
+ 0x00000002 /* EMC_RRD */
+ 0x00000001 /* EMC_REXT */
+ 0x00000000 /* EMC_WEXT */
+ 0x00000004 /* EMC_WDV */
+ 0x00000006 /* EMC_QUSE */
+ 0x00000004 /* EMC_QRST */
+ 0x0000000a /* EMC_QSAFE */
+ 0x0000000c /* EMC_RDV */
+ 0x00000bf0 /* EMC_REFRESH */
+ 0x00000000 /* EMC_BURST_REFRESH_NUM */
+ 0x000002fc /* EMC_PRE_REFRESH_REQ_CNT */
+ 0x00000001 /* EMC_PDEX2WR */
+ 0x00000008 /* EMC_PDEX2RD */
+ 0x00000001 /* EMC_PCHG2PDEN */
+ 0x00000000 /* EMC_ACT2PDEN */
+ 0x00000008 /* EMC_AR2PDEN */
+ 0x0000000f /* EMC_RW2PDEN */
+ 0x0000006c /* EMC_TXSR */
+ 0x00000200 /* EMC_TXSRDLL */
+ 0x00000004 /* EMC_TCKE */
+ 0x00000010 /* EMC_TFAW */
+ 0x00000000 /* EMC_TRPAB */
+ 0x00000004 /* EMC_TCLKSTABLE */
+ 0x00000005 /* EMC_TCLKSTOP */
+ 0x00000c30 /* EMC_TREFBW */
+ 0x00000000 /* EMC_QUSE_EXTRA */
+ 0x00000004 /* EMC_FBIO_CFG6 */
+ 0x00000000 /* EMC_ODT_WRITE */
+ 0x00000000 /* EMC_ODT_READ */
+ 0x00007088 /* EMC_FBIO_CFG5 */
+ 0x001d0084 /* EMC_CFG_DIG_DLL */
+ 0x00008000 /* EMC_CFG_DIG_DLL_PERIOD */
+ 0x0003c000 /* EMC_DLL_XFORM_DQS0 */
+ 0x0003c000 /* EMC_DLL_XFORM_DQS1 */
+ 0x0003c000 /* EMC_DLL_XFORM_DQS2 */
+ 0x0003c000 /* EMC_DLL_XFORM_DQS3 */
+ 0x0003c000 /* EMC_DLL_XFORM_DQS4 */
+ 0x0003c000 /* EMC_DLL_XFORM_DQS5 */
+ 0x0003c000 /* EMC_DLL_XFORM_DQS6 */
+ 0x0003c000 /* EMC_DLL_XFORM_DQS7 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE0 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE1 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE2 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE3 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE4 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE5 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE6 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE7 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS0 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS1 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS2 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS3 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS4 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS5 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS6 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS7 */
+ 0x00048000 /* EMC_DLL_XFORM_DQ0 */
+ 0x00048000 /* EMC_DLL_XFORM_DQ1 */
+ 0x00048000 /* EMC_DLL_XFORM_DQ2 */
+ 0x00048000 /* EMC_DLL_XFORM_DQ3 */
+ 0x000002a0 /* EMC_XM2CMDPADCTRL */
+ 0x0800013d /* EMC_XM2DQSPADCTRL2 */
+ 0x00000000 /* EMC_XM2DQPADCTRL2 */
+ 0x77fff884 /* EMC_XM2CLKPADCTRL */
+ 0x01f1f508 /* EMC_XM2COMPPADCTRL */
+ 0x05057404 /* EMC_XM2VTTGENPADCTRL */
+ 0x54000007 /* EMC_XM2VTTGENPADCTRL2 */
+ 0x080001e8 /* EMC_XM2QUSEPADCTRL */
+ 0x08000021 /* EMC_XM2DQSPADCTRL3 */
+ 0x00000802 /* EMC_CTT_TERM_CTRL */
+ 0x00020000 /* EMC_ZCAL_INTERVAL */
+ 0x00000100 /* EMC_ZCAL_WAIT_CNT */
+ 0x0158000c /* EMC_MRS_WAIT_CNT */
+ 0xa0f10000 /* EMC_AUTO_CAL_CONFIG */
+ 0x00000000 /* EMC_CTT */
+ 0x00000000 /* EMC_CTT_DURATION */
+ 0x800018c8 /* EMC_DYN_SELF_REF_CONTROL */
+ 0xe8000000 /* EMC_FBIO_SPARE */
+ 0xff00ff89 /* EMC_CFG_RSV */
+ >;
+ };
+
+ timing-800000000 {
+ clock-frequency = <800000000>;
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100002>;
+ nvidia,emc-mode-2 = <0x80200018>;
+ nvidia,emc-mode-reset = <0x80000d71>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-periodic-qrst;
+ nvidia,emc-configuration = <
+ 0x00000025 /* EMC_RC */
+ 0x000000ce /* EMC_RFC */
+ 0x0000001a /* EMC_RAS */
+ 0x00000009 /* EMC_RP */
+ 0x00000005 /* EMC_R2W */
+ 0x0000000d /* EMC_W2R */
+ 0x00000004 /* EMC_R2P */
+ 0x00000013 /* EMC_W2P */
+ 0x00000009 /* EMC_RD_RCD */
+ 0x00000009 /* EMC_WR_RCD */
+ 0x00000004 /* EMC_RRD */
+ 0x00000001 /* EMC_REXT */
+ 0x00000000 /* EMC_WEXT */
+ 0x00000007 /* EMC_WDV */
+ 0x0000000a /* EMC_QUSE */
+ 0x00000009 /* EMC_QRST */
+ 0x0000000b /* EMC_QSAFE */
+ 0x00000011 /* EMC_RDV */
+ 0x00001820 /* EMC_REFRESH */
+ 0x00000000 /* EMC_BURST_REFRESH_NUM */
+ 0x00000608 /* EMC_PRE_REFRESH_REQ_CNT */
+ 0x00000003 /* EMC_PDEX2WR */
+ 0x00000012 /* EMC_PDEX2RD */
+ 0x00000001 /* EMC_PCHG2PDEN */
+ 0x00000000 /* EMC_ACT2PDEN */
+ 0x0000000f /* EMC_AR2PDEN */
+ 0x00000018 /* EMC_RW2PDEN */
+ 0x000000d8 /* EMC_TXSR */
+ 0x00000200 /* EMC_TXSRDLL */
+ 0x00000005 /* EMC_TCKE */
+ 0x00000020 /* EMC_TFAW */
+ 0x00000000 /* EMC_TRPAB */
+ 0x00000007 /* EMC_TCLKSTABLE */
+ 0x00000008 /* EMC_TCLKSTOP */
+ 0x00001860 /* EMC_TREFBW */
+ 0x0000000b /* EMC_QUSE_EXTRA */
+ 0x00000006 /* EMC_FBIO_CFG6 */
+ 0x00000000 /* EMC_ODT_WRITE */
+ 0x00000000 /* EMC_ODT_READ */
+ 0x00005088 /* EMC_FBIO_CFG5 */
+ 0xf0070191 /* EMC_CFG_DIG_DLL */
+ 0x00008000 /* EMC_CFG_DIG_DLL_PERIOD */
+ 0x0000800a /* EMC_DLL_XFORM_DQS0 */
+ 0x0000000a /* EMC_DLL_XFORM_DQS1 */
+ 0x0000000a /* EMC_DLL_XFORM_DQS2 */
+ 0x0000000a /* EMC_DLL_XFORM_DQS3 */
+ 0x0000000a /* EMC_DLL_XFORM_DQS4 */
+ 0x0000000a /* EMC_DLL_XFORM_DQS5 */
+ 0x0000000a /* EMC_DLL_XFORM_DQS6 */
+ 0x0000000a /* EMC_DLL_XFORM_DQS7 */
+ 0x00018000 /* EMC_DLL_XFORM_QUSE0 */
+ 0x00018000 /* EMC_DLL_XFORM_QUSE1 */
+ 0x00018000 /* EMC_DLL_XFORM_QUSE2 */
+ 0x00018000 /* EMC_DLL_XFORM_QUSE3 */
+ 0x00018000 /* EMC_DLL_XFORM_QUSE4 */
+ 0x00018000 /* EMC_DLL_XFORM_QUSE5 */
+ 0x00018000 /* EMC_DLL_XFORM_QUSE6 */
+ 0x00018000 /* EMC_DLL_XFORM_QUSE7 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS0 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS1 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS2 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS3 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS4 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS5 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS6 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS7 */
+ 0x0000000a /* EMC_DLL_XFORM_DQ0 */
+ 0x0000000a /* EMC_DLL_XFORM_DQ1 */
+ 0x0000000a /* EMC_DLL_XFORM_DQ2 */
+ 0x0000000a /* EMC_DLL_XFORM_DQ3 */
+ 0x000002a0 /* EMC_XM2CMDPADCTRL */
+ 0x0600013d /* EMC_XM2DQSPADCTRL2 */
+ 0x22220000 /* EMC_XM2DQPADCTRL2 */
+ 0x77fff884 /* EMC_XM2CLKPADCTRL */
+ 0x01f1f501 /* EMC_XM2COMPPADCTRL */
+ 0x07077404 /* EMC_XM2VTTGENPADCTRL */
+ 0x54000000 /* EMC_XM2VTTGENPADCTRL2 */
+ 0x080001e8 /* EMC_XM2QUSEPADCTRL */
+ 0x08000021 /* EMC_XM2DQSPADCTRL3 */
+ 0x00000802 /* EMC_CTT_TERM_CTRL */
+ 0x00020000 /* EMC_ZCAL_INTERVAL */
+ 0x00000100 /* EMC_ZCAL_WAIT_CNT */
+ 0x00f0000c /* EMC_MRS_WAIT_CNT */
+ 0xa0f10000 /* EMC_AUTO_CAL_CONFIG */
+ 0x00000000 /* EMC_CTT */
+ 0x00000000 /* EMC_CTT_DURATION */
+ 0x8000308c /* EMC_DYN_SELF_REF_CONTROL */
+ 0xe8000000 /* EMC_FBIO_SPARE */
+ 0xff00ff49 /* EMC_CFG_RSV */
+ >;
+ };
+ };
+
+ emc-timings-1 {
+ nvidia,ram-code = <1>; /* Hynix M RAM */
+
+ timing-25500000 {
+ clock-frequency = <25500000>;
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200008>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-periodic-qrst;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-configuration = <
+ 0x00000001 /* EMC_RC */
+ 0x00000006 /* EMC_RFC */
+ 0x00000000 /* EMC_RAS */
+ 0x00000000 /* EMC_RP */
+ 0x00000002 /* EMC_R2W */
+ 0x0000000a /* EMC_W2R */
+ 0x00000005 /* EMC_R2P */
+ 0x0000000b /* EMC_W2P */
+ 0x00000000 /* EMC_RD_RCD */
+ 0x00000000 /* EMC_WR_RCD */
+ 0x00000003 /* EMC_RRD */
+ 0x00000001 /* EMC_REXT */
+ 0x00000000 /* EMC_WEXT */
+ 0x00000005 /* EMC_WDV */
+ 0x00000005 /* EMC_QUSE */
+ 0x00000004 /* EMC_QRST */
+ 0x0000000a /* EMC_QSAFE */
+ 0x0000000b /* EMC_RDV */
+ 0x000000c0 /* EMC_REFRESH */
+ 0x00000000 /* EMC_BURST_REFRESH_NUM */
+ 0x00000030 /* EMC_PRE_REFRESH_REQ_CNT */
+ 0x00000002 /* EMC_PDEX2WR */
+ 0x00000002 /* EMC_PDEX2RD */
+ 0x00000001 /* EMC_PCHG2PDEN */
+ 0x00000000 /* EMC_ACT2PDEN */
+ 0x00000007 /* EMC_AR2PDEN */
+ 0x0000000f /* EMC_RW2PDEN */
+ 0x00000007 /* EMC_TXSR */
+ 0x00000007 /* EMC_TXSRDLL */
+ 0x00000004 /* EMC_TCKE */
+ 0x00000002 /* EMC_TFAW */
+ 0x00000000 /* EMC_TRPAB */
+ 0x00000004 /* EMC_TCLKSTABLE */
+ 0x00000005 /* EMC_TCLKSTOP */
+ 0x000000c7 /* EMC_TREFBW */
+ 0x00000006 /* EMC_QUSE_EXTRA */
+ 0x00000004 /* EMC_FBIO_CFG6 */
+ 0x00000000 /* EMC_ODT_WRITE */
+ 0x00000000 /* EMC_ODT_READ */
+ 0x00004288 /* EMC_FBIO_CFG5 */
+ 0x007800a4 /* EMC_CFG_DIG_DLL */
+ 0x00008000 /* EMC_CFG_DIG_DLL_PERIOD */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS0 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS1 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS2 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS3 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS4 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS5 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS6 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS7 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE0 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE1 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE2 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE3 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE4 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE5 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE6 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE7 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS0 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS1 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS2 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS3 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS4 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS5 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS6 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS7 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQ0 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQ1 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQ2 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQ3 */
+ 0x000002a0 /* EMC_XM2CMDPADCTRL */
+ 0x0800211c /* EMC_XM2DQSPADCTRL2 */
+ 0x00000000 /* EMC_XM2DQPADCTRL2 */
+ 0x77fff884 /* EMC_XM2CLKPADCTRL */
+ 0x01f1f108 /* EMC_XM2COMPPADCTRL */
+ 0x05057404 /* EMC_XM2VTTGENPADCTRL */
+ 0x54000007 /* EMC_XM2VTTGENPADCTRL2 */
+ 0x08000168 /* EMC_XM2QUSEPADCTRL */
+ 0x08000000 /* EMC_XM2DQSPADCTRL3 */
+ 0x00000802 /* EMC_CTT_TERM_CTRL */
+ 0x00000000 /* EMC_ZCAL_INTERVAL */
+ 0x00000040 /* EMC_ZCAL_WAIT_CNT */
+ 0x000c000c /* EMC_MRS_WAIT_CNT */
+ 0xa0f10000 /* EMC_AUTO_CAL_CONFIG */
+ 0x00000000 /* EMC_CTT */
+ 0x00000000 /* EMC_CTT_DURATION */
+ 0x80000287 /* EMC_DYN_SELF_REF_CONTROL */
+ 0xe8000000 /* EMC_FBIO_SPARE */
+ 0xff00ff00 /* EMC_CFG_RSV */
+ >;
+ };
+
+ timing-51000000 {
+ clock-frequency = <51000000>;
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200008>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-periodic-qrst;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-configuration = <
+ 0x00000002 /* EMC_RC */
+ 0x0000000d /* EMC_RFC */
+ 0x00000001 /* EMC_RAS */
+ 0x00000000 /* EMC_RP */
+ 0x00000002 /* EMC_R2W */
+ 0x0000000a /* EMC_W2R */
+ 0x00000005 /* EMC_R2P */
+ 0x0000000b /* EMC_W2P */
+ 0x00000000 /* EMC_RD_RCD */
+ 0x00000000 /* EMC_WR_RCD */
+ 0x00000003 /* EMC_RRD */
+ 0x00000001 /* EMC_REXT */
+ 0x00000000 /* EMC_WEXT */
+ 0x00000005 /* EMC_WDV */
+ 0x00000005 /* EMC_QUSE */
+ 0x00000004 /* EMC_QRST */
+ 0x0000000a /* EMC_QSAFE */
+ 0x0000000b /* EMC_RDV */
+ 0x00000181 /* EMC_REFRESH */
+ 0x00000000 /* EMC_BURST_REFRESH_NUM */
+ 0x00000060 /* EMC_PRE_REFRESH_REQ_CNT */
+ 0x00000002 /* EMC_PDEX2WR */
+ 0x00000002 /* EMC_PDEX2RD */
+ 0x00000001 /* EMC_PCHG2PDEN */
+ 0x00000000 /* EMC_ACT2PDEN */
+ 0x00000007 /* EMC_AR2PDEN */
+ 0x0000000f /* EMC_RW2PDEN */
+ 0x0000000e /* EMC_TXSR */
+ 0x0000000e /* EMC_TXSRDLL */
+ 0x00000004 /* EMC_TCKE */
+ 0x00000003 /* EMC_TFAW */
+ 0x00000000 /* EMC_TRPAB */
+ 0x00000004 /* EMC_TCLKSTABLE */
+ 0x00000005 /* EMC_TCLKSTOP */
+ 0x0000018e /* EMC_TREFBW */
+ 0x00000006 /* EMC_QUSE_EXTRA */
+ 0x00000004 /* EMC_FBIO_CFG6 */
+ 0x00000000 /* EMC_ODT_WRITE */
+ 0x00000000 /* EMC_ODT_READ */
+ 0x00004288 /* EMC_FBIO_CFG5 */
+ 0x007800a4 /* EMC_CFG_DIG_DLL */
+ 0x00008000 /* EMC_CFG_DIG_DLL_PERIOD */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS0 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS1 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS2 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS3 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS4 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS5 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS6 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS7 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE0 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE1 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE2 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE3 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE4 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE5 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE6 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE7 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS0 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS1 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS2 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS3 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS4 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS5 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS6 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS7 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQ0 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQ1 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQ2 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQ3 */
+ 0x000002a0 /* EMC_XM2CMDPADCTRL */
+ 0x0800211c /* EMC_XM2DQSPADCTRL2 */
+ 0x00000000 /* EMC_XM2DQPADCTRL2 */
+ 0x77fff884 /* EMC_XM2CLKPADCTRL */
+ 0x01f1f108 /* EMC_XM2COMPPADCTRL */
+ 0x05057404 /* EMC_XM2VTTGENPADCTRL */
+ 0x54000007 /* EMC_XM2VTTGENPADCTRL2 */
+ 0x08000168 /* EMC_XM2QUSEPADCTRL */
+ 0x08000000 /* EMC_XM2DQSPADCTRL3 */
+ 0x00000802 /* EMC_CTT_TERM_CTRL */
+ 0x00000000 /* EMC_ZCAL_INTERVAL */
+ 0x00000040 /* EMC_ZCAL_WAIT_CNT */
+ 0x000c000c /* EMC_MRS_WAIT_CNT */
+ 0xa0f10000 /* EMC_AUTO_CAL_CONFIG */
+ 0x00000000 /* EMC_CTT */
+ 0x00000000 /* EMC_CTT_DURATION */
+ 0x8000040b /* EMC_DYN_SELF_REF_CONTROL */
+ 0xe8000000 /* EMC_FBIO_SPARE */
+ 0xff00ff00 /* EMC_CFG_RSV */
+ >;
+ };
+
+ timing-102000000 {
+ clock-frequency = <102000000>;
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200008>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-periodic-qrst;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-configuration = <
+ 0x00000004 /* EMC_RC */
+ 0x0000001a /* EMC_RFC */
+ 0x00000003 /* EMC_RAS */
+ 0x00000001 /* EMC_RP */
+ 0x00000002 /* EMC_R2W */
+ 0x0000000a /* EMC_W2R */
+ 0x00000005 /* EMC_R2P */
+ 0x0000000b /* EMC_W2P */
+ 0x00000001 /* EMC_RD_RCD */
+ 0x00000001 /* EMC_WR_RCD */
+ 0x00000003 /* EMC_RRD */
+ 0x00000001 /* EMC_REXT */
+ 0x00000000 /* EMC_WEXT */
+ 0x00000005 /* EMC_WDV */
+ 0x00000005 /* EMC_QUSE */
+ 0x00000004 /* EMC_QRST */
+ 0x0000000a /* EMC_QSAFE */
+ 0x0000000b /* EMC_RDV */
+ 0x00000303 /* EMC_REFRESH */
+ 0x00000000 /* EMC_BURST_REFRESH_NUM */
+ 0x000000c0 /* EMC_PRE_REFRESH_REQ_CNT */
+ 0x00000002 /* EMC_PDEX2WR */
+ 0x00000002 /* EMC_PDEX2RD */
+ 0x00000001 /* EMC_PCHG2PDEN */
+ 0x00000000 /* EMC_ACT2PDEN */
+ 0x00000007 /* EMC_AR2PDEN */
+ 0x0000000f /* EMC_RW2PDEN */
+ 0x0000001c /* EMC_TXSR */
+ 0x0000001c /* EMC_TXSRDLL */
+ 0x00000004 /* EMC_TCKE */
+ 0x00000005 /* EMC_TFAW */
+ 0x00000000 /* EMC_TRPAB */
+ 0x00000004 /* EMC_TCLKSTABLE */
+ 0x00000005 /* EMC_TCLKSTOP */
+ 0x0000031c /* EMC_TREFBW */
+ 0x00000006 /* EMC_QUSE_EXTRA */
+ 0x00000004 /* EMC_FBIO_CFG6 */
+ 0x00000000 /* EMC_ODT_WRITE */
+ 0x00000000 /* EMC_ODT_READ */
+ 0x00004288 /* EMC_FBIO_CFG5 */
+ 0x007800a4 /* EMC_CFG_DIG_DLL */
+ 0x00008000 /* EMC_CFG_DIG_DLL_PERIOD */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS0 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS1 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS2 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS3 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS4 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS5 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS6 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS7 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE0 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE1 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE2 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE3 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE4 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE5 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE6 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE7 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS0 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS1 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS2 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS3 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS4 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS5 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS6 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS7 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQ0 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQ1 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQ2 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQ3 */
+ 0x000002a0 /* EMC_XM2CMDPADCTRL */
+ 0x0800211c /* EMC_XM2DQSPADCTRL2 */
+ 0x00000000 /* EMC_XM2DQPADCTRL2 */
+ 0x77fff884 /* EMC_XM2CLKPADCTRL */
+ 0x01f1f108 /* EMC_XM2COMPPADCTRL */
+ 0x05057404 /* EMC_XM2VTTGENPADCTRL */
+ 0x54000007 /* EMC_XM2VTTGENPADCTRL2 */
+ 0x08000168 /* EMC_XM2QUSEPADCTRL */
+ 0x08000000 /* EMC_XM2DQSPADCTRL3 */
+ 0x00000802 /* EMC_CTT_TERM_CTRL */
+ 0x00000000 /* EMC_ZCAL_INTERVAL */
+ 0x00000040 /* EMC_ZCAL_WAIT_CNT */
+ 0x000c000c /* EMC_MRS_WAIT_CNT */
+ 0xa0f10000 /* EMC_AUTO_CAL_CONFIG */
+ 0x00000000 /* EMC_CTT */
+ 0x00000000 /* EMC_CTT_DURATION */
+ 0x80000713 /* EMC_DYN_SELF_REF_CONTROL */
+ 0xe8000000 /* EMC_FBIO_SPARE */
+ 0xff00ff00 /* EMC_CFG_RSV */
+ >;
+ };
+
+ timing-204000000 {
+ clock-frequency = <204000000>;
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200008>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-periodic-qrst;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-configuration = <
+ 0x00000009 /* EMC_RC */
+ 0x00000035 /* EMC_RFC */
+ 0x00000007 /* EMC_RAS */
+ 0x00000002 /* EMC_RP */
+ 0x00000002 /* EMC_R2W */
+ 0x0000000a /* EMC_W2R */
+ 0x00000005 /* EMC_R2P */
+ 0x0000000b /* EMC_W2P */
+ 0x00000002 /* EMC_RD_RCD */
+ 0x00000002 /* EMC_WR_RCD */
+ 0x00000003 /* EMC_RRD */
+ 0x00000001 /* EMC_REXT */
+ 0x00000000 /* EMC_WEXT */
+ 0x00000005 /* EMC_WDV */
+ 0x00000005 /* EMC_QUSE */
+ 0x00000004 /* EMC_QRST */
+ 0x0000000a /* EMC_QSAFE */
+ 0x0000000b /* EMC_RDV */
+ 0x00000607 /* EMC_REFRESH */
+ 0x00000000 /* EMC_BURST_REFRESH_NUM */
+ 0x00000181 /* EMC_PRE_REFRESH_REQ_CNT */
+ 0x00000002 /* EMC_PDEX2WR */
+ 0x00000002 /* EMC_PDEX2RD */
+ 0x00000001 /* EMC_PCHG2PDEN */
+ 0x00000000 /* EMC_ACT2PDEN */
+ 0x00000007 /* EMC_AR2PDEN */
+ 0x0000000f /* EMC_RW2PDEN */
+ 0x00000038 /* EMC_TXSR */
+ 0x00000038 /* EMC_TXSRDLL */
+ 0x00000004 /* EMC_TCKE */
+ 0x00000009 /* EMC_TFAW */
+ 0x00000000 /* EMC_TRPAB */
+ 0x00000004 /* EMC_TCLKSTABLE */
+ 0x00000005 /* EMC_TCLKSTOP */
+ 0x00000638 /* EMC_TREFBW */
+ 0x00000006 /* EMC_QUSE_EXTRA */
+ 0x00000006 /* EMC_FBIO_CFG6 */
+ 0x00000000 /* EMC_ODT_WRITE */
+ 0x00000000 /* EMC_ODT_READ */
+ 0x00004288 /* EMC_FBIO_CFG5 */
+ 0x004400a4 /* EMC_CFG_DIG_DLL */
+ 0x00008000 /* EMC_CFG_DIG_DLL_PERIOD */
+ 0x00080000 /* EMC_DLL_XFORM_DQS0 */
+ 0x00080000 /* EMC_DLL_XFORM_DQS1 */
+ 0x00080000 /* EMC_DLL_XFORM_DQS2 */
+ 0x00080000 /* EMC_DLL_XFORM_DQS3 */
+ 0x00080000 /* EMC_DLL_XFORM_DQS4 */
+ 0x00080000 /* EMC_DLL_XFORM_DQS5 */
+ 0x00080000 /* EMC_DLL_XFORM_DQS6 */
+ 0x00080000 /* EMC_DLL_XFORM_DQS7 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE0 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE1 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE2 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE3 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE4 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE5 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE6 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE7 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS0 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS1 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS2 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS3 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS4 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS5 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS6 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS7 */
+ 0x00080000 /* EMC_DLL_XFORM_DQ0 */
+ 0x00080000 /* EMC_DLL_XFORM_DQ1 */
+ 0x00080000 /* EMC_DLL_XFORM_DQ2 */
+ 0x00080000 /* EMC_DLL_XFORM_DQ3 */
+ 0x000002a0 /* EMC_XM2CMDPADCTRL */
+ 0x0800211c /* EMC_XM2DQSPADCTRL2 */
+ 0x00000000 /* EMC_XM2DQPADCTRL2 */
+ 0x77fff884 /* EMC_XM2CLKPADCTRL */
+ 0x01f1f108 /* EMC_XM2COMPPADCTRL */
+ 0x05057404 /* EMC_XM2VTTGENPADCTRL */
+ 0x54000007 /* EMC_XM2VTTGENPADCTRL2 */
+ 0x08000168 /* EMC_XM2QUSEPADCTRL */
+ 0x08000000 /* EMC_XM2DQSPADCTRL3 */
+ 0x00000802 /* EMC_CTT_TERM_CTRL */
+ 0x00020000 /* EMC_ZCAL_INTERVAL */
+ 0x00000100 /* EMC_ZCAL_WAIT_CNT */
+ 0x000c000c /* EMC_MRS_WAIT_CNT */
+ 0xa0f10000 /* EMC_AUTO_CAL_CONFIG */
+ 0x00000000 /* EMC_CTT */
+ 0x00000000 /* EMC_CTT_DURATION */
+ 0x80000d22 /* EMC_DYN_SELF_REF_CONTROL */
+ 0xe8000000 /* EMC_FBIO_SPARE */
+ 0xff00ff00 /* EMC_CFG_RSV */
+ >;
+ };
+
+ timing-400000000 {
+ clock-frequency = <400000000>;
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100002>;
+ nvidia,emc-mode-2 = <0x80200000>;
+ nvidia,emc-mode-reset = <0x80000521>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-configuration = <
+ 0x00000012 /* EMC_RC */
+ 0x00000066 /* EMC_RFC */
+ 0x0000000c /* EMC_RAS */
+ 0x00000004 /* EMC_RP */
+ 0x00000003 /* EMC_R2W */
+ 0x00000008 /* EMC_W2R */
+ 0x00000002 /* EMC_R2P */
+ 0x0000000a /* EMC_W2P */
+ 0x00000004 /* EMC_RD_RCD */
+ 0x00000004 /* EMC_WR_RCD */
+ 0x00000002 /* EMC_RRD */
+ 0x00000001 /* EMC_REXT */
+ 0x00000000 /* EMC_WEXT */
+ 0x00000004 /* EMC_WDV */
+ 0x00000006 /* EMC_QUSE */
+ 0x00000004 /* EMC_QRST */
+ 0x0000000a /* EMC_QSAFE */
+ 0x0000000c /* EMC_RDV */
+ 0x00000bf0 /* EMC_REFRESH */
+ 0x00000000 /* EMC_BURST_REFRESH_NUM */
+ 0x000002fc /* EMC_PRE_REFRESH_REQ_CNT */
+ 0x00000001 /* EMC_PDEX2WR */
+ 0x00000008 /* EMC_PDEX2RD */
+ 0x00000001 /* EMC_PCHG2PDEN */
+ 0x00000000 /* EMC_ACT2PDEN */
+ 0x00000008 /* EMC_AR2PDEN */
+ 0x0000000f /* EMC_RW2PDEN */
+ 0x0000006c /* EMC_TXSR */
+ 0x00000200 /* EMC_TXSRDLL */
+ 0x00000004 /* EMC_TCKE */
+ 0x00000010 /* EMC_TFAW */
+ 0x00000000 /* EMC_TRPAB */
+ 0x00000004 /* EMC_TCLKSTABLE */
+ 0x00000005 /* EMC_TCLKSTOP */
+ 0x00000c30 /* EMC_TREFBW */
+ 0x00000000 /* EMC_QUSE_EXTRA */
+ 0x00000004 /* EMC_FBIO_CFG6 */
+ 0x00000000 /* EMC_ODT_WRITE */
+ 0x00000000 /* EMC_ODT_READ */
+ 0x00007088 /* EMC_FBIO_CFG5 */
+ 0x001d0084 /* EMC_CFG_DIG_DLL */
+ 0x00008000 /* EMC_CFG_DIG_DLL_PERIOD */
+ 0x0003c000 /* EMC_DLL_XFORM_DQS0 */
+ 0x0003c000 /* EMC_DLL_XFORM_DQS1 */
+ 0x0003c000 /* EMC_DLL_XFORM_DQS2 */
+ 0x0003c000 /* EMC_DLL_XFORM_DQS3 */
+ 0x0003c000 /* EMC_DLL_XFORM_DQS4 */
+ 0x0003c000 /* EMC_DLL_XFORM_DQS5 */
+ 0x0003c000 /* EMC_DLL_XFORM_DQS6 */
+ 0x0003c000 /* EMC_DLL_XFORM_DQS7 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE0 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE1 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE2 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE3 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE4 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE5 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE6 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE7 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS0 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS1 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS2 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS3 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS4 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS5 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS6 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS7 */
+ 0x00048000 /* EMC_DLL_XFORM_DQ0 */
+ 0x00048000 /* EMC_DLL_XFORM_DQ1 */
+ 0x00048000 /* EMC_DLL_XFORM_DQ2 */
+ 0x00048000 /* EMC_DLL_XFORM_DQ3 */
+ 0x000002a0 /* EMC_XM2CMDPADCTRL */
+ 0x0800013d /* EMC_XM2DQSPADCTRL2 */
+ 0x00000000 /* EMC_XM2DQPADCTRL2 */
+ 0x77fff884 /* EMC_XM2CLKPADCTRL */
+ 0x01f1f508 /* EMC_XM2COMPPADCTRL */
+ 0x05057404 /* EMC_XM2VTTGENPADCTRL */
+ 0x54000007 /* EMC_XM2VTTGENPADCTRL2 */
+ 0x080001e8 /* EMC_XM2QUSEPADCTRL */
+ 0x08000021 /* EMC_XM2DQSPADCTRL3 */
+ 0x00000802 /* EMC_CTT_TERM_CTRL */
+ 0x00020000 /* EMC_ZCAL_INTERVAL */
+ 0x00000100 /* EMC_ZCAL_WAIT_CNT */
+ 0x0158000c /* EMC_MRS_WAIT_CNT */
+ 0xa0f10000 /* EMC_AUTO_CAL_CONFIG */
+ 0x00000000 /* EMC_CTT */
+ 0x00000000 /* EMC_CTT_DURATION */
+ 0x800018c8 /* EMC_DYN_SELF_REF_CONTROL */
+ 0xe8000000 /* EMC_FBIO_SPARE */
+ 0xff00ff89 /* EMC_CFG_RSV */
+ >;
+ };
+
+ timing-800000000 {
+ clock-frequency = <800000000>;
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100002>;
+ nvidia,emc-mode-2 = <0x80200018>;
+ nvidia,emc-mode-reset = <0x80000d71>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-periodic-qrst;
+ nvidia,emc-configuration = <
+ 0x00000025 /* EMC_RC */
+ 0x000000ce /* EMC_RFC */
+ 0x0000001a /* EMC_RAS */
+ 0x00000009 /* EMC_RP */
+ 0x00000005 /* EMC_R2W */
+ 0x0000000d /* EMC_W2R */
+ 0x00000004 /* EMC_R2P */
+ 0x00000013 /* EMC_W2P */
+ 0x00000009 /* EMC_RD_RCD */
+ 0x00000009 /* EMC_WR_RCD */
+ 0x00000004 /* EMC_RRD */
+ 0x00000001 /* EMC_REXT */
+ 0x00000000 /* EMC_WEXT */
+ 0x00000007 /* EMC_WDV */
+ 0x0000000a /* EMC_QUSE */
+ 0x00000009 /* EMC_QRST */
+ 0x0000000b /* EMC_QSAFE */
+ 0x00000011 /* EMC_RDV */
+ 0x00001820 /* EMC_REFRESH */
+ 0x00000000 /* EMC_BURST_REFRESH_NUM */
+ 0x00000608 /* EMC_PRE_REFRESH_REQ_CNT */
+ 0x00000003 /* EMC_PDEX2WR */
+ 0x00000012 /* EMC_PDEX2RD */
+ 0x00000001 /* EMC_PCHG2PDEN */
+ 0x00000000 /* EMC_ACT2PDEN */
+ 0x0000000f /* EMC_AR2PDEN */
+ 0x00000018 /* EMC_RW2PDEN */
+ 0x000000d8 /* EMC_TXSR */
+ 0x00000200 /* EMC_TXSRDLL */
+ 0x00000005 /* EMC_TCKE */
+ 0x00000020 /* EMC_TFAW */
+ 0x00000000 /* EMC_TRPAB */
+ 0x00000007 /* EMC_TCLKSTABLE */
+ 0x00000008 /* EMC_TCLKSTOP */
+ 0x00001860 /* EMC_TREFBW */
+ 0x0000000b /* EMC_QUSE_EXTRA */
+ 0x00000006 /* EMC_FBIO_CFG6 */
+ 0x00000000 /* EMC_ODT_WRITE */
+ 0x00000000 /* EMC_ODT_READ */
+ 0x00005088 /* EMC_FBIO_CFG5 */
+ 0xf0070191 /* EMC_CFG_DIG_DLL */
+ 0x00008000 /* EMC_CFG_DIG_DLL_PERIOD */
+ 0x0000800a /* EMC_DLL_XFORM_DQS0 */
+ 0x0000000a /* EMC_DLL_XFORM_DQS1 */
+ 0x0000000a /* EMC_DLL_XFORM_DQS2 */
+ 0x0000000a /* EMC_DLL_XFORM_DQS3 */
+ 0x0000000a /* EMC_DLL_XFORM_DQS4 */
+ 0x0000000a /* EMC_DLL_XFORM_DQS5 */
+ 0x0000000a /* EMC_DLL_XFORM_DQS6 */
+ 0x0000000a /* EMC_DLL_XFORM_DQS7 */
+ 0x00018000 /* EMC_DLL_XFORM_QUSE0 */
+ 0x00018000 /* EMC_DLL_XFORM_QUSE1 */
+ 0x00018000 /* EMC_DLL_XFORM_QUSE2 */
+ 0x00018000 /* EMC_DLL_XFORM_QUSE3 */
+ 0x00018000 /* EMC_DLL_XFORM_QUSE4 */
+ 0x00018000 /* EMC_DLL_XFORM_QUSE5 */
+ 0x00018000 /* EMC_DLL_XFORM_QUSE6 */
+ 0x00018000 /* EMC_DLL_XFORM_QUSE7 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS0 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS1 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS2 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS3 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS4 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS5 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS6 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS7 */
+ 0x0000000a /* EMC_DLL_XFORM_DQ0 */
+ 0x0000000a /* EMC_DLL_XFORM_DQ1 */
+ 0x0000000a /* EMC_DLL_XFORM_DQ2 */
+ 0x0000000a /* EMC_DLL_XFORM_DQ3 */
+ 0x000002a0 /* EMC_XM2CMDPADCTRL */
+ 0x0600013d /* EMC_XM2DQSPADCTRL2 */
+ 0x22220000 /* EMC_XM2DQPADCTRL2 */
+ 0x77fff884 /* EMC_XM2CLKPADCTRL */
+ 0x01f1f501 /* EMC_XM2COMPPADCTRL */
+ 0x07077404 /* EMC_XM2VTTGENPADCTRL */
+ 0x54000000 /* EMC_XM2VTTGENPADCTRL2 */
+ 0x080001e8 /* EMC_XM2QUSEPADCTRL */
+ 0x08000021 /* EMC_XM2DQSPADCTRL3 */
+ 0x00000802 /* EMC_CTT_TERM_CTRL */
+ 0x00020000 /* EMC_ZCAL_INTERVAL */
+ 0x00000100 /* EMC_ZCAL_WAIT_CNT */
+ 0x00f0000c /* EMC_MRS_WAIT_CNT */
+ 0xa0f10000 /* EMC_AUTO_CAL_CONFIG */
+ 0x00000000 /* EMC_CTT */
+ 0x00000000 /* EMC_CTT_DURATION */
+ 0x8000308c /* EMC_DYN_SELF_REF_CONTROL */
+ 0xe8000000 /* EMC_FBIO_SPARE */
+ 0xff00ff49 /* EMC_CFG_RSV */
+ >;
+ };
+ };
+
+ emc-timings-2 {
+ nvidia,ram-code = <2>; /* Hynix A RAM */
+
+ timing-25500000 {
+ clock-frequency = <25500000>;
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200008>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-periodic-qrst;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-configuration = <
+ 0x00000001 /* EMC_RC */
+ 0x00000007 /* EMC_RFC */
+ 0x00000000 /* EMC_RAS */
+ 0x00000000 /* EMC_RP */
+ 0x00000002 /* EMC_R2W */
+ 0x0000000a /* EMC_W2R */
+ 0x00000005 /* EMC_R2P */
+ 0x0000000b /* EMC_W2P */
+ 0x00000000 /* EMC_RD_RCD */
+ 0x00000000 /* EMC_WR_RCD */
+ 0x00000003 /* EMC_RRD */
+ 0x00000001 /* EMC_REXT */
+ 0x00000000 /* EMC_WEXT */
+ 0x00000005 /* EMC_WDV */
+ 0x00000005 /* EMC_QUSE */
+ 0x00000004 /* EMC_QRST */
+ 0x0000000a /* EMC_QSAFE */
+ 0x0000000b /* EMC_RDV */
+ 0x000000c0 /* EMC_REFRESH */
+ 0x00000000 /* EMC_BURST_REFRESH_NUM */
+ 0x00000030 /* EMC_PRE_REFRESH_REQ_CNT */
+ 0x00000002 /* EMC_PDEX2WR */
+ 0x00000002 /* EMC_PDEX2RD */
+ 0x00000001 /* EMC_PCHG2PDEN */
+ 0x00000000 /* EMC_ACT2PDEN */
+ 0x00000007 /* EMC_AR2PDEN */
+ 0x0000000f /* EMC_RW2PDEN */
+ 0x00000008 /* EMC_TXSR */
+ 0x00000008 /* EMC_TXSRDLL */
+ 0x00000004 /* EMC_TCKE */
+ 0x00000002 /* EMC_TFAW */
+ 0x00000000 /* EMC_TRPAB */
+ 0x00000004 /* EMC_TCLKSTABLE */
+ 0x00000005 /* EMC_TCLKSTOP */
+ 0x000000c7 /* EMC_TREFBW */
+ 0x00000006 /* EMC_QUSE_EXTRA */
+ 0x00000004 /* EMC_FBIO_CFG6 */
+ 0x00000000 /* EMC_ODT_WRITE */
+ 0x00000000 /* EMC_ODT_READ */
+ 0x00004288 /* EMC_FBIO_CFG5 */
+ 0x007800a4 /* EMC_CFG_DIG_DLL */
+ 0x00008000 /* EMC_CFG_DIG_DLL_PERIOD */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS0 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS1 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS2 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS3 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS4 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS5 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS6 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS7 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE0 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE1 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE2 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE3 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE4 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE5 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE6 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE7 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS0 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS1 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS2 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS3 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS4 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS5 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS6 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS7 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQ0 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQ1 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQ2 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQ3 */
+ 0x000002a0 /* EMC_XM2CMDPADCTRL */
+ 0x0800211c /* EMC_XM2DQSPADCTRL2 */
+ 0x00000000 /* EMC_XM2DQPADCTRL2 */
+ 0x77fff884 /* EMC_XM2CLKPADCTRL */
+ 0x01f1f108 /* EMC_XM2COMPPADCTRL */
+ 0x05057404 /* EMC_XM2VTTGENPADCTRL */
+ 0x54000007 /* EMC_XM2VTTGENPADCTRL2 */
+ 0x08000168 /* EMC_XM2QUSEPADCTRL */
+ 0x08000000 /* EMC_XM2DQSPADCTRL3 */
+ 0x00000802 /* EMC_CTT_TERM_CTRL */
+ 0x00000000 /* EMC_ZCAL_INTERVAL */
+ 0x00000040 /* EMC_ZCAL_WAIT_CNT */
+ 0x000c000c /* EMC_MRS_WAIT_CNT */
+ 0xa0f10000 /* EMC_AUTO_CAL_CONFIG */
+ 0x00000000 /* EMC_CTT */
+ 0x00000000 /* EMC_CTT_DURATION */
+ 0x80000287 /* EMC_DYN_SELF_REF_CONTROL */
+ 0xe8000000 /* EMC_FBIO_SPARE */
+ 0xff00ff00 /* EMC_CFG_RSV */
+ >;
+ };
+
+ timing-51000000 {
+ clock-frequency = <51000000>;
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200008>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-periodic-qrst;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-configuration = <
+ 0x00000002 /* EMC_RC */
+ 0x0000000f /* EMC_RFC */
+ 0x00000001 /* EMC_RAS */
+ 0x00000000 /* EMC_RP */
+ 0x00000002 /* EMC_R2W */
+ 0x0000000a /* EMC_W2R */
+ 0x00000005 /* EMC_R2P */
+ 0x0000000b /* EMC_W2P */
+ 0x00000000 /* EMC_RD_RCD */
+ 0x00000000 /* EMC_WR_RCD */
+ 0x00000003 /* EMC_RRD */
+ 0x00000001 /* EMC_REXT */
+ 0x00000000 /* EMC_WEXT */
+ 0x00000005 /* EMC_WDV */
+ 0x00000005 /* EMC_QUSE */
+ 0x00000004 /* EMC_QRST */
+ 0x0000000a /* EMC_QSAFE */
+ 0x0000000b /* EMC_RDV */
+ 0x00000181 /* EMC_REFRESH */
+ 0x00000000 /* EMC_BURST_REFRESH_NUM */
+ 0x00000060 /* EMC_PRE_REFRESH_REQ_CNT */
+ 0x00000002 /* EMC_PDEX2WR */
+ 0x00000002 /* EMC_PDEX2RD */
+ 0x00000001 /* EMC_PCHG2PDEN */
+ 0x00000000 /* EMC_ACT2PDEN */
+ 0x00000007 /* EMC_AR2PDEN */
+ 0x0000000f /* EMC_RW2PDEN */
+ 0x00000010 /* EMC_TXSR */
+ 0x00000010 /* EMC_TXSRDLL */
+ 0x00000004 /* EMC_TCKE */
+ 0x00000003 /* EMC_TFAW */
+ 0x00000000 /* EMC_TRPAB */
+ 0x00000004 /* EMC_TCLKSTABLE */
+ 0x00000005 /* EMC_TCLKSTOP */
+ 0x0000018e /* EMC_TREFBW */
+ 0x00000006 /* EMC_QUSE_EXTRA */
+ 0x00000004 /* EMC_FBIO_CFG6 */
+ 0x00000000 /* EMC_ODT_WRITE */
+ 0x00000000 /* EMC_ODT_READ */
+ 0x00004288 /* EMC_FBIO_CFG5 */
+ 0x007800a4 /* EMC_CFG_DIG_DLL */
+ 0x00008000 /* EMC_CFG_DIG_DLL_PERIOD */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS0 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS1 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS2 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS3 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS4 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS5 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS6 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS7 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE0 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE1 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE2 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE3 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE4 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE5 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE6 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE7 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS0 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS1 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS2 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS3 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS4 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS5 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS6 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS7 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQ0 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQ1 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQ2 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQ3 */
+ 0x000002a0 /* EMC_XM2CMDPADCTRL */
+ 0x0800211c /* EMC_XM2DQSPADCTRL2 */
+ 0x00000000 /* EMC_XM2DQPADCTRL2 */
+ 0x77fff884 /* EMC_XM2CLKPADCTRL */
+ 0x01f1f108 /* EMC_XM2COMPPADCTRL */
+ 0x05057404 /* EMC_XM2VTTGENPADCTRL */
+ 0x54000007 /* EMC_XM2VTTGENPADCTRL2 */
+ 0x08000168 /* EMC_XM2QUSEPADCTRL */
+ 0x08000000 /* EMC_XM2DQSPADCTRL3 */
+ 0x00000802 /* EMC_CTT_TERM_CTRL */
+ 0x00000000 /* EMC_ZCAL_INTERVAL */
+ 0x00000040 /* EMC_ZCAL_WAIT_CNT */
+ 0x000c000c /* EMC_MRS_WAIT_CNT */
+ 0xa0f10000 /* EMC_AUTO_CAL_CONFIG */
+ 0x00000000 /* EMC_CTT */
+ 0x00000000 /* EMC_CTT_DURATION */
+ 0x8000040b /* EMC_DYN_SELF_REF_CONTROL */
+ 0xe8000000 /* EMC_FBIO_SPARE */
+ 0xff00ff00 /* EMC_CFG_RSV */
+ >;
+ };
+
+ timing-102000000 {
+ clock-frequency = <102000000>;
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200008>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-periodic-qrst;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-configuration = <
+ 0x00000004 /* EMC_RC */
+ 0x0000001e /* EMC_RFC */
+ 0x00000003 /* EMC_RAS */
+ 0x00000001 /* EMC_RP */
+ 0x00000002 /* EMC_R2W */
+ 0x0000000a /* EMC_W2R */
+ 0x00000005 /* EMC_R2P */
+ 0x0000000b /* EMC_W2P */
+ 0x00000001 /* EMC_RD_RCD */
+ 0x00000001 /* EMC_WR_RCD */
+ 0x00000003 /* EMC_RRD */
+ 0x00000001 /* EMC_REXT */
+ 0x00000000 /* EMC_WEXT */
+ 0x00000005 /* EMC_WDV */
+ 0x00000005 /* EMC_QUSE */
+ 0x00000004 /* EMC_QRST */
+ 0x0000000a /* EMC_QSAFE */
+ 0x0000000b /* EMC_RDV */
+ 0x00000303 /* EMC_REFRESH */
+ 0x00000000 /* EMC_BURST_REFRESH_NUM */
+ 0x000000c0 /* EMC_PRE_REFRESH_REQ_CNT */
+ 0x00000002 /* EMC_PDEX2WR */
+ 0x00000002 /* EMC_PDEX2RD */
+ 0x00000001 /* EMC_PCHG2PDEN */
+ 0x00000000 /* EMC_ACT2PDEN */
+ 0x00000007 /* EMC_AR2PDEN */
+ 0x0000000f /* EMC_RW2PDEN */
+ 0x00000020 /* EMC_TXSR */
+ 0x00000020 /* EMC_TXSRDLL */
+ 0x00000004 /* EMC_TCKE */
+ 0x00000005 /* EMC_TFAW */
+ 0x00000000 /* EMC_TRPAB */
+ 0x00000004 /* EMC_TCLKSTABLE */
+ 0x00000005 /* EMC_TCLKSTOP */
+ 0x0000031c /* EMC_TREFBW */
+ 0x00000006 /* EMC_QUSE_EXTRA */
+ 0x00000004 /* EMC_FBIO_CFG6 */
+ 0x00000000 /* EMC_ODT_WRITE */
+ 0x00000000 /* EMC_ODT_READ */
+ 0x00004288 /* EMC_FBIO_CFG5 */
+ 0x007800a4 /* EMC_CFG_DIG_DLL */
+ 0x00008000 /* EMC_CFG_DIG_DLL_PERIOD */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS0 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS1 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS2 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS3 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS4 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS5 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS6 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQS7 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE0 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE1 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE2 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE3 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE4 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE5 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE6 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE7 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS0 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS1 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS2 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS3 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS4 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS5 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS6 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS7 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQ0 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQ1 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQ2 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQ3 */
+ 0x000002a0 /* EMC_XM2CMDPADCTRL */
+ 0x0800211c /* EMC_XM2DQSPADCTRL2 */
+ 0x00000000 /* EMC_XM2DQPADCTRL2 */
+ 0x77fff884 /* EMC_XM2CLKPADCTRL */
+ 0x01f1f108 /* EMC_XM2COMPPADCTRL */
+ 0x05057404 /* EMC_XM2VTTGENPADCTRL */
+ 0x54000007 /* EMC_XM2VTTGENPADCTRL2 */
+ 0x08000168 /* EMC_XM2QUSEPADCTRL */
+ 0x08000000 /* EMC_XM2DQSPADCTRL3 */
+ 0x00000802 /* EMC_CTT_TERM_CTRL */
+ 0x00000000 /* EMC_ZCAL_INTERVAL */
+ 0x00000040 /* EMC_ZCAL_WAIT_CNT */
+ 0x000c000c /* EMC_MRS_WAIT_CNT */
+ 0xa0f10000 /* EMC_AUTO_CAL_CONFIG */
+ 0x00000000 /* EMC_CTT */
+ 0x00000000 /* EMC_CTT_DURATION */
+ 0x80000713 /* EMC_DYN_SELF_REF_CONTROL */
+ 0xe8000000 /* EMC_FBIO_SPARE */
+ 0xff00ff00 /* EMC_CFG_RSV */
+ >;
+ };
+
+ timing-204000000 {
+ clock-frequency = <204000000>;
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200008>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-periodic-qrst;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-configuration = <
+ 0x00000009 /* EMC_RC */
+ 0x0000003d /* EMC_RFC */
+ 0x00000007 /* EMC_RAS */
+ 0x00000002 /* EMC_RP */
+ 0x00000002 /* EMC_R2W */
+ 0x0000000a /* EMC_W2R */
+ 0x00000005 /* EMC_R2P */
+ 0x0000000b /* EMC_W2P */
+ 0x00000002 /* EMC_RD_RCD */
+ 0x00000002 /* EMC_WR_RCD */
+ 0x00000003 /* EMC_RRD */
+ 0x00000001 /* EMC_REXT */
+ 0x00000000 /* EMC_WEXT */
+ 0x00000005 /* EMC_WDV */
+ 0x00000005 /* EMC_QUSE */
+ 0x00000004 /* EMC_QRST */
+ 0x0000000a /* EMC_QSAFE */
+ 0x0000000b /* EMC_RDV */
+ 0x00000607 /* EMC_REFRESH */
+ 0x00000000 /* EMC_BURST_REFRESH_NUM */
+ 0x00000181 /* EMC_PRE_REFRESH_REQ_CNT */
+ 0x00000002 /* EMC_PDEX2WR */
+ 0x00000002 /* EMC_PDEX2RD */
+ 0x00000001 /* EMC_PCHG2PDEN */
+ 0x00000000 /* EMC_ACT2PDEN */
+ 0x00000007 /* EMC_AR2PDEN */
+ 0x0000000f /* EMC_RW2PDEN */
+ 0x00000040 /* EMC_TXSR */
+ 0x00000040 /* EMC_TXSRDLL */
+ 0x00000004 /* EMC_TCKE */
+ 0x00000009 /* EMC_TFAW */
+ 0x00000000 /* EMC_TRPAB */
+ 0x00000004 /* EMC_TCLKSTABLE */
+ 0x00000005 /* EMC_TCLKSTOP */
+ 0x00000638 /* EMC_TREFBW */
+ 0x00000006 /* EMC_QUSE_EXTRA */
+ 0x00000006 /* EMC_FBIO_CFG6 */
+ 0x00000000 /* EMC_ODT_WRITE */
+ 0x00000000 /* EMC_ODT_READ */
+ 0x00004288 /* EMC_FBIO_CFG5 */
+ 0x004400a4 /* EMC_CFG_DIG_DLL */
+ 0x00008000 /* EMC_CFG_DIG_DLL_PERIOD */
+ 0x00080000 /* EMC_DLL_XFORM_DQS0 */
+ 0x00080000 /* EMC_DLL_XFORM_DQS1 */
+ 0x00080000 /* EMC_DLL_XFORM_DQS2 */
+ 0x00080000 /* EMC_DLL_XFORM_DQS3 */
+ 0x00080000 /* EMC_DLL_XFORM_DQS4 */
+ 0x00080000 /* EMC_DLL_XFORM_DQS5 */
+ 0x00080000 /* EMC_DLL_XFORM_DQS6 */
+ 0x00080000 /* EMC_DLL_XFORM_DQS7 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE0 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE1 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE2 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE3 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE4 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE5 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE6 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE7 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS0 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS1 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS2 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS3 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS4 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS5 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS6 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS7 */
+ 0x00080000 /* EMC_DLL_XFORM_DQ0 */
+ 0x00080000 /* EMC_DLL_XFORM_DQ1 */
+ 0x00080000 /* EMC_DLL_XFORM_DQ2 */
+ 0x00080000 /* EMC_DLL_XFORM_DQ3 */
+ 0x000002a0 /* EMC_XM2CMDPADCTRL */
+ 0x0800211c /* EMC_XM2DQSPADCTRL2 */
+ 0x00000000 /* EMC_XM2DQPADCTRL2 */
+ 0x77fff884 /* EMC_XM2CLKPADCTRL */
+ 0x01f1f108 /* EMC_XM2COMPPADCTRL */
+ 0x05057404 /* EMC_XM2VTTGENPADCTRL */
+ 0x54000007 /* EMC_XM2VTTGENPADCTRL2 */
+ 0x08000168 /* EMC_XM2QUSEPADCTRL */
+ 0x08000000 /* EMC_XM2DQSPADCTRL3 */
+ 0x00000802 /* EMC_CTT_TERM_CTRL */
+ 0x00020000 /* EMC_ZCAL_INTERVAL */
+ 0x00000100 /* EMC_ZCAL_WAIT_CNT */
+ 0x000c000c /* EMC_MRS_WAIT_CNT */
+ 0xa0f10000 /* EMC_AUTO_CAL_CONFIG */
+ 0x00000000 /* EMC_CTT */
+ 0x00000000 /* EMC_CTT_DURATION */
+ 0x80000d22 /* EMC_DYN_SELF_REF_CONTROL */
+ 0xe8000000 /* EMC_FBIO_SPARE */
+ 0xff00ff00 /* EMC_CFG_RSV */
+ >;
+ };
+
+ timing-400000000 {
+ clock-frequency = <400000000>;
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100002>;
+ nvidia,emc-mode-2 = <0x80200000>;
+ nvidia,emc-mode-reset = <0x80000521>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-configuration = <
+ 0x00000012 /* EMC_RC */
+ 0x00000076 /* EMC_RFC */
+ 0x0000000c /* EMC_RAS */
+ 0x00000004 /* EMC_RP */
+ 0x00000003 /* EMC_R2W */
+ 0x00000008 /* EMC_W2R */
+ 0x00000002 /* EMC_R2P */
+ 0x0000000a /* EMC_W2P */
+ 0x00000004 /* EMC_RD_RCD */
+ 0x00000004 /* EMC_WR_RCD */
+ 0x00000002 /* EMC_RRD */
+ 0x00000001 /* EMC_REXT */
+ 0x00000000 /* EMC_WEXT */
+ 0x00000004 /* EMC_WDV */
+ 0x00000006 /* EMC_QUSE */
+ 0x00000004 /* EMC_QRST */
+ 0x0000000a /* EMC_QSAFE */
+ 0x0000000c /* EMC_RDV */
+ 0x00000bf0 /* EMC_REFRESH */
+ 0x00000000 /* EMC_BURST_REFRESH_NUM */
+ 0x000002fc /* EMC_PRE_REFRESH_REQ_CNT */
+ 0x00000001 /* EMC_PDEX2WR */
+ 0x00000008 /* EMC_PDEX2RD */
+ 0x00000001 /* EMC_PCHG2PDEN */
+ 0x00000000 /* EMC_ACT2PDEN */
+ 0x00000008 /* EMC_AR2PDEN */
+ 0x0000000f /* EMC_RW2PDEN */
+ 0x0000007c /* EMC_TXSR */
+ 0x00000200 /* EMC_TXSRDLL */
+ 0x00000004 /* EMC_TCKE */
+ 0x00000010 /* EMC_TFAW */
+ 0x00000000 /* EMC_TRPAB */
+ 0x00000004 /* EMC_TCLKSTABLE */
+ 0x00000005 /* EMC_TCLKSTOP */
+ 0x00000c30 /* EMC_TREFBW */
+ 0x00000000 /* EMC_QUSE_EXTRA */
+ 0x00000004 /* EMC_FBIO_CFG6 */
+ 0x00000000 /* EMC_ODT_WRITE */
+ 0x00000000 /* EMC_ODT_READ */
+ 0x00007088 /* EMC_FBIO_CFG5 */
+ 0x001d0084 /* EMC_CFG_DIG_DLL */
+ 0x00008000 /* EMC_CFG_DIG_DLL_PERIOD */
+ 0x00044000 /* EMC_DLL_XFORM_DQS0 */
+ 0x00044000 /* EMC_DLL_XFORM_DQS1 */
+ 0x00044000 /* EMC_DLL_XFORM_DQS2 */
+ 0x00044000 /* EMC_DLL_XFORM_DQS3 */
+ 0x00044000 /* EMC_DLL_XFORM_DQS4 */
+ 0x00044000 /* EMC_DLL_XFORM_DQS5 */
+ 0x00044000 /* EMC_DLL_XFORM_DQS6 */
+ 0x00044000 /* EMC_DLL_XFORM_DQS7 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE0 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE1 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE2 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE3 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE4 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE5 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE6 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE7 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS0 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS1 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS2 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS3 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS4 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS5 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS6 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS7 */
+ 0x00058000 /* EMC_DLL_XFORM_DQ0 */
+ 0x00058000 /* EMC_DLL_XFORM_DQ1 */
+ 0x00058000 /* EMC_DLL_XFORM_DQ2 */
+ 0x00058000 /* EMC_DLL_XFORM_DQ3 */
+ 0x000002a0 /* EMC_XM2CMDPADCTRL */
+ 0x0800013d /* EMC_XM2DQSPADCTRL2 */
+ 0x00000000 /* EMC_XM2DQPADCTRL2 */
+ 0x77fff884 /* EMC_XM2CLKPADCTRL */
+ 0x01f1f508 /* EMC_XM2COMPPADCTRL */
+ 0x05057404 /* EMC_XM2VTTGENPADCTRL */
+ 0x54000007 /* EMC_XM2VTTGENPADCTRL2 */
+ 0x080001e8 /* EMC_XM2QUSEPADCTRL */
+ 0x08000021 /* EMC_XM2DQSPADCTRL3 */
+ 0x00000802 /* EMC_CTT_TERM_CTRL */
+ 0x00020000 /* EMC_ZCAL_INTERVAL */
+ 0x00000100 /* EMC_ZCAL_WAIT_CNT */
+ 0x0148000c /* EMC_MRS_WAIT_CNT */
+ 0xa0f10000 /* EMC_AUTO_CAL_CONFIG */
+ 0x00000000 /* EMC_CTT */
+ 0x00000000 /* EMC_CTT_DURATION */
+ 0x800018c8 /* EMC_DYN_SELF_REF_CONTROL */
+ 0xe8000000 /* EMC_FBIO_SPARE */
+ 0xff00ff89 /* EMC_CFG_RSV */
+ >;
+ };
+
+ timing-800000000 {
+ clock-frequency = <800000000>;
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x80100002>;
+ nvidia,emc-mode-2 = <0x80200018>;
+ nvidia,emc-mode-reset = <0x80000d71>;
+ nvidia,emc-zcal-cnt-long = <0x00000040>;
+ nvidia,emc-cfg-periodic-qrst;
+ nvidia,emc-configuration = <
+ 0x00000025 /* EMC_RC */
+ 0x000000ee /* EMC_RFC */
+ 0x0000001a /* EMC_RAS */
+ 0x00000009 /* EMC_RP */
+ 0x00000005 /* EMC_R2W */
+ 0x0000000d /* EMC_W2R */
+ 0x00000004 /* EMC_R2P */
+ 0x00000013 /* EMC_W2P */
+ 0x00000009 /* EMC_RD_RCD */
+ 0x00000009 /* EMC_WR_RCD */
+ 0x00000003 /* EMC_RRD */
+ 0x00000001 /* EMC_REXT */
+ 0x00000000 /* EMC_WEXT */
+ 0x00000007 /* EMC_WDV */
+ 0x0000000a /* EMC_QUSE */
+ 0x00000009 /* EMC_QRST */
+ 0x0000000b /* EMC_QSAFE */
+ 0x00000011 /* EMC_RDV */
+ 0x00001820 /* EMC_REFRESH */
+ 0x00000000 /* EMC_BURST_REFRESH_NUM */
+ 0x00000608 /* EMC_PRE_REFRESH_REQ_CNT */
+ 0x00000003 /* EMC_PDEX2WR */
+ 0x00000012 /* EMC_PDEX2RD */
+ 0x00000001 /* EMC_PCHG2PDEN */
+ 0x00000000 /* EMC_ACT2PDEN */
+ 0x0000000f /* EMC_AR2PDEN */
+ 0x00000018 /* EMC_RW2PDEN */
+ 0x000000f8 /* EMC_TXSR */
+ 0x00000200 /* EMC_TXSRDLL */
+ 0x00000005 /* EMC_TCKE */
+ 0x00000020 /* EMC_TFAW */
+ 0x00000000 /* EMC_TRPAB */
+ 0x00000007 /* EMC_TCLKSTABLE */
+ 0x00000008 /* EMC_TCLKSTOP */
+ 0x00001860 /* EMC_TREFBW */
+ 0x0000000b /* EMC_QUSE_EXTRA */
+ 0x00000006 /* EMC_FBIO_CFG6 */
+ 0x00000000 /* EMC_ODT_WRITE */
+ 0x00000000 /* EMC_ODT_READ */
+ 0x00005088 /* EMC_FBIO_CFG5 */
+ 0xf0070191 /* EMC_CFG_DIG_DLL */
+ 0x00008000 /* EMC_CFG_DIG_DLL_PERIOD */
+ 0x0000000c /* EMC_DLL_XFORM_DQS0 */
+ 0x007fc00a /* EMC_DLL_XFORM_DQS1 */
+ 0x00000008 /* EMC_DLL_XFORM_DQS2 */
+ 0x0000000a /* EMC_DLL_XFORM_DQS3 */
+ 0x0000000a /* EMC_DLL_XFORM_DQS4 */
+ 0x0000000a /* EMC_DLL_XFORM_DQS5 */
+ 0x0000000a /* EMC_DLL_XFORM_DQS6 */
+ 0x0000000a /* EMC_DLL_XFORM_DQS7 */
+ 0x00018000 /* EMC_DLL_XFORM_QUSE0 */
+ 0x00018000 /* EMC_DLL_XFORM_QUSE1 */
+ 0x00018000 /* EMC_DLL_XFORM_QUSE2 */
+ 0x00018000 /* EMC_DLL_XFORM_QUSE3 */
+ 0x00018000 /* EMC_DLL_XFORM_QUSE4 */
+ 0x00018000 /* EMC_DLL_XFORM_QUSE5 */
+ 0x00018000 /* EMC_DLL_XFORM_QUSE6 */
+ 0x00018000 /* EMC_DLL_XFORM_QUSE7 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS0 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS1 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS2 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS3 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS4 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS5 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS6 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS7 */
+ 0x0000000a /* EMC_DLL_XFORM_DQ0 */
+ 0x0000000c /* EMC_DLL_XFORM_DQ1 */
+ 0x0000000a /* EMC_DLL_XFORM_DQ2 */
+ 0x0000000a /* EMC_DLL_XFORM_DQ3 */
+ 0x000002a0 /* EMC_XM2CMDPADCTRL */
+ 0x0600013d /* EMC_XM2DQSPADCTRL2 */
+ 0x22220000 /* EMC_XM2DQPADCTRL2 */
+ 0x77fff884 /* EMC_XM2CLKPADCTRL */
+ 0x01f1f501 /* EMC_XM2COMPPADCTRL */
+ 0x07077404 /* EMC_XM2VTTGENPADCTRL */
+ 0x54000000 /* EMC_XM2VTTGENPADCTRL2 */
+ 0x080001e8 /* EMC_XM2QUSEPADCTRL */
+ 0x0a000021 /* EMC_XM2DQSPADCTRL3 */
+ 0x00000802 /* EMC_CTT_TERM_CTRL */
+ 0x00020000 /* EMC_ZCAL_INTERVAL */
+ 0x00000100 /* EMC_ZCAL_WAIT_CNT */
+ 0x00d0000c /* EMC_MRS_WAIT_CNT */
+ 0xa0f10000 /* EMC_AUTO_CAL_CONFIG */
+ 0x00000000 /* EMC_CTT */
+ 0x00000000 /* EMC_CTT_DURATION */
+ 0x8000308c /* EMC_DYN_SELF_REF_CONTROL */
+ 0xe8000000 /* EMC_FBIO_SPARE */
+ 0xff00ff49 /* EMC_CFG_RSV */
+ >;
+ };
+ };
};
hda@70030000 {
@@ -592,3940 +4786,11 @@ led-power {
};
};
};
-&mc_timings {
- emc-timings-0 {
- nvidia,ram-code = <0>; /* Samsung RAM */
- timing-25500000 {
- clock-frequency = <25500000>;
- nvidia,emem-configuration = <
- 0x00030003 /* MC_EMEM_ARB_CFG */
- 0xc0000010 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RC */
- 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
- 0x00000001 /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000001 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06020102 /* MC_EMEM_ARB_DA_TURNS */
- 0x000a0502 /* MC_EMEM_ARB_DA_COVERS */
- 0x75830303 /* MC_EMEM_ARB_MISC0 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
- timing-51000000 {
- clock-frequency = <51000000>;
- nvidia,emem-configuration = <
- 0x00010003 /* MC_EMEM_ARB_CFG */
- 0xc0000010 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RC */
- 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
- 0x00000001 /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000001 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06020102 /* MC_EMEM_ARB_DA_TURNS */
- 0x000a0502 /* MC_EMEM_ARB_DA_COVERS */
- 0x74630303 /* MC_EMEM_ARB_MISC0 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
- timing-102000000 {
- clock-frequency = <102000000>;
- nvidia,emem-configuration = <
- 0x00000003 /* MC_EMEM_ARB_CFG */
- 0xc0000018 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000003 /* MC_EMEM_ARB_TIMING_RC */
- 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
- 0x00000002 /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000001 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06020102 /* MC_EMEM_ARB_DA_TURNS */
- 0x000a0503 /* MC_EMEM_ARB_DA_COVERS */
- 0x73c30504 /* MC_EMEM_ARB_MISC0 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
- timing-204000000 {
- clock-frequency = <204000000>;
- nvidia,emem-configuration = <
- 0x00000006 /* MC_EMEM_ARB_CFG */
- 0xc0000025 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000005 /* MC_EMEM_ARB_TIMING_RC */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RAS */
- 0x00000004 /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000001 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06020102 /* MC_EMEM_ARB_DA_TURNS */
- 0x000a0505 /* MC_EMEM_ARB_DA_COVERS */
- 0x73840a06 /* MC_EMEM_ARB_MISC0 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
- timing-400000000 {
- clock-frequency = <400000000>;
- nvidia,emem-configuration = <
- 0x0000000c /* MC_EMEM_ARB_CFG */
- 0xc0000048 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000009 /* MC_EMEM_ARB_TIMING_RC */
- 0x00000005 /* MC_EMEM_ARB_TIMING_RAS */
- 0x00000007 /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000003 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06030202 /* MC_EMEM_ARB_DA_TURNS */
- 0x000d0709 /* MC_EMEM_ARB_DA_COVERS */
- 0x7086120a /* MC_EMEM_ARB_MISC0 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
- timing-800000000 {
- clock-frequency = <800000000>;
- nvidia,emem-configuration = <
- 0x00000018 /* MC_EMEM_ARB_CFG */
- 0xc0000090 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000004 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000005 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000013 /* MC_EMEM_ARB_TIMING_RC */
- 0x0000000c /* MC_EMEM_ARB_TIMING_RAS */
- 0x0000000f /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x0000000c /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000004 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000008 /* MC_EMEM_ARB_TIMING_W2R */
- 0x08040202 /* MC_EMEM_ARB_DA_TURNS */
- 0x00160d13 /* MC_EMEM_ARB_DA_COVERS */
- 0x712c2414 /* MC_EMEM_ARB_MISC0 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
- };
- emc-timings-1 {
- nvidia,ram-code = <1>; /* Hynix M RAM */
- timing-25500000 {
- clock-frequency = <25500000>;
- nvidia,emem-configuration = <
- 0x00030003 /* MC_EMEM_ARB_CFG */
- 0xc0000010 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RC */
- 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
- 0x00000001 /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000001 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06020102 /* MC_EMEM_ARB_DA_TURNS */
- 0x000a0502 /* MC_EMEM_ARB_DA_COVERS */
- 0x75830303 /* MC_EMEM_ARB_MISC0 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
- timing-51000000 {
- clock-frequency = <51000000>;
- nvidia,emem-configuration = <
- 0x00010003 /* MC_EMEM_ARB_CFG */
- 0xc0000010 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RC */
- 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
- 0x00000001 /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000001 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06020102 /* MC_EMEM_ARB_DA_TURNS */
- 0x000a0502 /* MC_EMEM_ARB_DA_COVERS */
- 0x74630303 /* MC_EMEM_ARB_MISC0 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
- timing-102000000 {
- clock-frequency = <102000000>;
- nvidia,emem-configuration = <
- 0x00000003 /* MC_EMEM_ARB_CFG */
- 0xc0000018 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000003 /* MC_EMEM_ARB_TIMING_RC */
- 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
- 0x00000002 /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000001 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06020102 /* MC_EMEM_ARB_DA_TURNS */
- 0x000a0503 /* MC_EMEM_ARB_DA_COVERS */
- 0x73c30504 /* MC_EMEM_ARB_MISC0 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
- timing-204000000 {
- clock-frequency = <204000000>;
- nvidia,emem-configuration = <
- 0x00000006 /* MC_EMEM_ARB_CFG */
- 0xc0000025 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000005 /* MC_EMEM_ARB_TIMING_RC */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RAS */
- 0x00000004 /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000001 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06020102 /* MC_EMEM_ARB_DA_TURNS */
- 0x000a0505 /* MC_EMEM_ARB_DA_COVERS */
- 0x73840a06 /* MC_EMEM_ARB_MISC0 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
- timing-400000000 {
- clock-frequency = <400000000>;
- nvidia,emem-configuration = <
- 0x0000000c /* MC_EMEM_ARB_CFG */
- 0xc0000048 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000009 /* MC_EMEM_ARB_TIMING_RC */
- 0x00000005 /* MC_EMEM_ARB_TIMING_RAS */
- 0x00000007 /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000003 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06030202 /* MC_EMEM_ARB_DA_TURNS */
- 0x000d0709 /* MC_EMEM_ARB_DA_COVERS */
- 0x7086120a /* MC_EMEM_ARB_MISC0 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
- timing-800000000 {
- clock-frequency = <800000000>;
- nvidia,emem-configuration = <
- 0x00000018 /* MC_EMEM_ARB_CFG */
- 0xc0000090 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000004 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000005 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000013 /* MC_EMEM_ARB_TIMING_RC */
- 0x0000000c /* MC_EMEM_ARB_TIMING_RAS */
- 0x0000000f /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x0000000c /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000004 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000008 /* MC_EMEM_ARB_TIMING_W2R */
- 0x08040202 /* MC_EMEM_ARB_DA_TURNS */
- 0x00160d13 /* MC_EMEM_ARB_DA_COVERS */
- 0x712c2414 /* MC_EMEM_ARB_MISC0 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
- };
- emc-timings-2 {
- nvidia,ram-code = <2>; /* Hynix A RAM */
- timing-25500000 {
- clock-frequency = <25500000>;
- nvidia,emem-configuration = <
- 0x00030003 /* MC_EMEM_ARB_CFG */
- 0xc0000010 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RC */
- 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
- 0x00000001 /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000001 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06020102 /* MC_EMEM_ARB_DA_TURNS */
- 0x000a0502 /* MC_EMEM_ARB_DA_COVERS */
- 0x75e30303 /* MC_EMEM_ARB_MISC0 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
- timing-51000000 {
- clock-frequency = <51000000>;
- nvidia,emem-configuration = <
- 0x00010003 /* MC_EMEM_ARB_CFG */
- 0xc0000010 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RC */
- 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
- 0x00000001 /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000001 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06020102 /* MC_EMEM_ARB_DA_TURNS */
- 0x000a0502 /* MC_EMEM_ARB_DA_COVERS */
- 0x74e30303 /* MC_EMEM_ARB_MISC0 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
- timing-102000000 {
- clock-frequency = <102000000>;
- nvidia,emem-configuration = <
- 0x00000003 /* MC_EMEM_ARB_CFG */
- 0xc0000018 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000003 /* MC_EMEM_ARB_TIMING_RC */
- 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
- 0x00000002 /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000001 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06020102 /* MC_EMEM_ARB_DA_TURNS */
- 0x000a0503 /* MC_EMEM_ARB_DA_COVERS */
- 0x74430504 /* MC_EMEM_ARB_MISC0 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
- timing-204000000 {
- clock-frequency = <204000000>;
- nvidia,emem-configuration = <
- 0x00000006 /* MC_EMEM_ARB_CFG */
- 0xc0000025 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000005 /* MC_EMEM_ARB_TIMING_RC */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RAS */
- 0x00000004 /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000001 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06020102 /* MC_EMEM_ARB_DA_TURNS */
- 0x000a0505 /* MC_EMEM_ARB_DA_COVERS */
- 0x74040a06 /* MC_EMEM_ARB_MISC0 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
- timing-400000000 {
- clock-frequency = <400000000>;
- nvidia,emem-configuration = <
- 0x0000000c /* MC_EMEM_ARB_CFG */
- 0xc0000048 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000009 /* MC_EMEM_ARB_TIMING_RC */
- 0x00000005 /* MC_EMEM_ARB_TIMING_RAS */
- 0x00000007 /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000003 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06030202 /* MC_EMEM_ARB_DA_TURNS */
- 0x000d0709 /* MC_EMEM_ARB_DA_COVERS */
- 0x7086120a /* MC_EMEM_ARB_MISC0 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
- timing-800000000 {
- clock-frequency = <800000000>;
- nvidia,emem-configuration = <
- 0x00000018 /* MC_EMEM_ARB_CFG */
- 0xc0000090 /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000004 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000005 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000013 /* MC_EMEM_ARB_TIMING_RC */
- 0x0000000c /* MC_EMEM_ARB_TIMING_RAS */
- 0x0000000f /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000003 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x0000000c /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000002 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000004 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000008 /* MC_EMEM_ARB_TIMING_W2R */
- 0x08040202 /* MC_EMEM_ARB_DA_TURNS */
- 0x00160d13 /* MC_EMEM_ARB_DA_COVERS */
- 0x712c2414 /* MC_EMEM_ARB_MISC0 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
- };
-};
-&emc_timings {
- emc-timings-0 {
- nvidia,ram-code = <0>; /* Samsung RAM */
- timing-25500000 {
- clock-frequency = <25500000>;
- nvidia,emc-auto-cal-interval = <0x001fffff>;
- nvidia,emc-mode-1 = <0x80100003>;
- nvidia,emc-mode-2 = <0x80200008>;
- nvidia,emc-mode-reset = <0x80001221>;
- nvidia,emc-zcal-cnt-long = <0x00000040>;
- nvidia,emc-cfg-periodic-qrst;
- nvidia,emc-cfg-dyn-self-ref;
- nvidia,emc-configuration = <
- 0x00000001 /* EMC_RC */
- 0x00000006 /* EMC_RFC */
- 0x00000000 /* EMC_RAS */
- 0x00000000 /* EMC_RP */
- 0x00000002 /* EMC_R2W */
- 0x0000000a /* EMC_W2R */
- 0x00000005 /* EMC_R2P */
- 0x0000000b /* EMC_W2P */
- 0x00000000 /* EMC_RD_RCD */
- 0x00000000 /* EMC_WR_RCD */
- 0x00000003 /* EMC_RRD */
- 0x00000001 /* EMC_REXT */
- 0x00000000 /* EMC_WEXT */
- 0x00000005 /* EMC_WDV */
- 0x00000005 /* EMC_QUSE */
- 0x00000004 /* EMC_QRST */
- 0x0000000a /* EMC_QSAFE */
- 0x0000000b /* EMC_RDV */
- 0x000000c0 /* EMC_REFRESH */
- 0x00000000 /* EMC_BURST_REFRESH_NUM */
- 0x00000030 /* EMC_PRE_REFRESH_REQ_CNT */
- 0x00000002 /* EMC_PDEX2WR */
- 0x00000002 /* EMC_PDEX2RD */
- 0x00000001 /* EMC_PCHG2PDEN */
- 0x00000000 /* EMC_ACT2PDEN */
- 0x00000007 /* EMC_AR2PDEN */
- 0x0000000f /* EMC_RW2PDEN */
- 0x00000007 /* EMC_TXSR */
- 0x00000007 /* EMC_TXSRDLL */
- 0x00000004 /* EMC_TCKE */
- 0x00000002 /* EMC_TFAW */
- 0x00000000 /* EMC_TRPAB */
- 0x00000004 /* EMC_TCLKSTABLE */
- 0x00000005 /* EMC_TCLKSTOP */
- 0x000000c7 /* EMC_TREFBW */
- 0x00000006 /* EMC_QUSE_EXTRA */
- 0x00000004 /* EMC_FBIO_CFG6 */
- 0x00000000 /* EMC_ODT_WRITE */
- 0x00000000 /* EMC_ODT_READ */
- 0x00004288 /* EMC_FBIO_CFG5 */
- 0x007800a4 /* EMC_CFG_DIG_DLL */
- 0x00008000 /* EMC_CFG_DIG_DLL_PERIOD */
- 0x000fc000 /* EMC_DLL_XFORM_DQS0 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS1 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS2 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS3 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS4 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS5 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS6 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS7 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE0 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE1 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE2 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE3 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE4 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE5 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE6 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE7 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS0 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS1 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS2 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS3 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS4 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS5 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS6 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS7 */
- 0x000fc000 /* EMC_DLL_XFORM_DQ0 */
- 0x000fc000 /* EMC_DLL_XFORM_DQ1 */
- 0x000fc000 /* EMC_DLL_XFORM_DQ2 */
- 0x000fc000 /* EMC_DLL_XFORM_DQ3 */
- 0x000002a0 /* EMC_XM2CMDPADCTRL */
- 0x0800211c /* EMC_XM2DQSPADCTRL2 */
- 0x00000000 /* EMC_XM2DQPADCTRL2 */
- 0x77fff884 /* EMC_XM2CLKPADCTRL */
- 0x01f1f108 /* EMC_XM2COMPPADCTRL */
- 0x05057404 /* EMC_XM2VTTGENPADCTRL */
- 0x54000007 /* EMC_XM2VTTGENPADCTRL2 */
- 0x08000168 /* EMC_XM2QUSEPADCTRL */
- 0x08000000 /* EMC_XM2DQSPADCTRL3 */
- 0x00000802 /* EMC_CTT_TERM_CTRL */
- 0x00000000 /* EMC_ZCAL_INTERVAL */
- 0x00000040 /* EMC_ZCAL_WAIT_CNT */
- 0x000c000c /* EMC_MRS_WAIT_CNT */
- 0xa0f10000 /* EMC_AUTO_CAL_CONFIG */
- 0x00000000 /* EMC_CTT */
- 0x00000000 /* EMC_CTT_DURATION */
- 0x80000287 /* EMC_DYN_SELF_REF_CONTROL */
- 0xe8000000 /* EMC_FBIO_SPARE */
- 0xff00ff00 /* EMC_CFG_RSV */
- >;
- };
- timing-51000000 {
- clock-frequency = <51000000>;
- nvidia,emc-auto-cal-interval = <0x001fffff>;
- nvidia,emc-mode-1 = <0x80100003>;
- nvidia,emc-mode-2 = <0x80200008>;
- nvidia,emc-mode-reset = <0x80001221>;
- nvidia,emc-zcal-cnt-long = <0x00000040>;
- nvidia,emc-cfg-periodic-qrst;
- nvidia,emc-cfg-dyn-self-ref;
- nvidia,emc-configuration = <
- 0x00000002 /* EMC_RC */
- 0x0000000d /* EMC_RFC */
- 0x00000001 /* EMC_RAS */
- 0x00000000 /* EMC_RP */
- 0x00000002 /* EMC_R2W */
- 0x0000000a /* EMC_W2R */
- 0x00000005 /* EMC_R2P */
- 0x0000000b /* EMC_W2P */
- 0x00000000 /* EMC_RD_RCD */
- 0x00000000 /* EMC_WR_RCD */
- 0x00000003 /* EMC_RRD */
- 0x00000001 /* EMC_REXT */
- 0x00000000 /* EMC_WEXT */
- 0x00000005 /* EMC_WDV */
- 0x00000005 /* EMC_QUSE */
- 0x00000004 /* EMC_QRST */
- 0x0000000a /* EMC_QSAFE */
- 0x0000000b /* EMC_RDV */
- 0x00000181 /* EMC_REFRESH */
- 0x00000000 /* EMC_BURST_REFRESH_NUM */
- 0x00000060 /* EMC_PRE_REFRESH_REQ_CNT */
- 0x00000002 /* EMC_PDEX2WR */
- 0x00000002 /* EMC_PDEX2RD */
- 0x00000001 /* EMC_PCHG2PDEN */
- 0x00000000 /* EMC_ACT2PDEN */
- 0x00000007 /* EMC_AR2PDEN */
- 0x0000000f /* EMC_RW2PDEN */
- 0x0000000e /* EMC_TXSR */
- 0x0000000e /* EMC_TXSRDLL */
- 0x00000004 /* EMC_TCKE */
- 0x00000003 /* EMC_TFAW */
- 0x00000000 /* EMC_TRPAB */
- 0x00000004 /* EMC_TCLKSTABLE */
- 0x00000005 /* EMC_TCLKSTOP */
- 0x0000018e /* EMC_TREFBW */
- 0x00000006 /* EMC_QUSE_EXTRA */
- 0x00000004 /* EMC_FBIO_CFG6 */
- 0x00000000 /* EMC_ODT_WRITE */
- 0x00000000 /* EMC_ODT_READ */
- 0x00004288 /* EMC_FBIO_CFG5 */
- 0x007800a4 /* EMC_CFG_DIG_DLL */
- 0x00008000 /* EMC_CFG_DIG_DLL_PERIOD */
- 0x000fc000 /* EMC_DLL_XFORM_DQS0 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS1 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS2 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS3 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS4 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS5 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS6 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS7 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE0 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE1 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE2 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE3 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE4 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE5 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE6 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE7 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS0 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS1 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS2 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS3 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS4 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS5 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS6 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS7 */
- 0x000fc000 /* EMC_DLL_XFORM_DQ0 */
- 0x000fc000 /* EMC_DLL_XFORM_DQ1 */
- 0x000fc000 /* EMC_DLL_XFORM_DQ2 */
- 0x000fc000 /* EMC_DLL_XFORM_DQ3 */
- 0x000002a0 /* EMC_XM2CMDPADCTRL */
- 0x0800211c /* EMC_XM2DQSPADCTRL2 */
- 0x00000000 /* EMC_XM2DQPADCTRL2 */
- 0x77fff884 /* EMC_XM2CLKPADCTRL */
- 0x01f1f108 /* EMC_XM2COMPPADCTRL */
- 0x05057404 /* EMC_XM2VTTGENPADCTRL */
- 0x54000007 /* EMC_XM2VTTGENPADCTRL2 */
- 0x08000168 /* EMC_XM2QUSEPADCTRL */
- 0x08000000 /* EMC_XM2DQSPADCTRL3 */
- 0x00000802 /* EMC_CTT_TERM_CTRL */
- 0x00000000 /* EMC_ZCAL_INTERVAL */
- 0x00000040 /* EMC_ZCAL_WAIT_CNT */
- 0x000c000c /* EMC_MRS_WAIT_CNT */
- 0xa0f10000 /* EMC_AUTO_CAL_CONFIG */
- 0x00000000 /* EMC_CTT */
- 0x00000000 /* EMC_CTT_DURATION */
- 0x8000040b /* EMC_DYN_SELF_REF_CONTROL */
- 0xe8000000 /* EMC_FBIO_SPARE */
- 0xff00ff00 /* EMC_CFG_RSV */
- >;
- };
- timing-102000000 {
- clock-frequency = <102000000>;
- nvidia,emc-auto-cal-interval = <0x001fffff>;
- nvidia,emc-mode-1 = <0x80100003>;
- nvidia,emc-mode-2 = <0x80200008>;
- nvidia,emc-mode-reset = <0x80001221>;
- nvidia,emc-zcal-cnt-long = <0x00000040>;
- nvidia,emc-cfg-periodic-qrst;
- nvidia,emc-cfg-dyn-self-ref;
- nvidia,emc-configuration = <
- 0x00000004 /* EMC_RC */
- 0x0000001a /* EMC_RFC */
- 0x00000003 /* EMC_RAS */
- 0x00000001 /* EMC_RP */
- 0x00000002 /* EMC_R2W */
- 0x0000000a /* EMC_W2R */
- 0x00000005 /* EMC_R2P */
- 0x0000000b /* EMC_W2P */
- 0x00000001 /* EMC_RD_RCD */
- 0x00000001 /* EMC_WR_RCD */
- 0x00000003 /* EMC_RRD */
- 0x00000001 /* EMC_REXT */
- 0x00000000 /* EMC_WEXT */
- 0x00000005 /* EMC_WDV */
- 0x00000005 /* EMC_QUSE */
- 0x00000004 /* EMC_QRST */
- 0x0000000a /* EMC_QSAFE */
- 0x0000000b /* EMC_RDV */
- 0x00000303 /* EMC_REFRESH */
- 0x00000000 /* EMC_BURST_REFRESH_NUM */
- 0x000000c0 /* EMC_PRE_REFRESH_REQ_CNT */
- 0x00000002 /* EMC_PDEX2WR */
- 0x00000002 /* EMC_PDEX2RD */
- 0x00000001 /* EMC_PCHG2PDEN */
- 0x00000000 /* EMC_ACT2PDEN */
- 0x00000007 /* EMC_AR2PDEN */
- 0x0000000f /* EMC_RW2PDEN */
- 0x0000001c /* EMC_TXSR */
- 0x0000001c /* EMC_TXSRDLL */
- 0x00000004 /* EMC_TCKE */
- 0x00000005 /* EMC_TFAW */
- 0x00000000 /* EMC_TRPAB */
- 0x00000004 /* EMC_TCLKSTABLE */
- 0x00000005 /* EMC_TCLKSTOP */
- 0x0000031c /* EMC_TREFBW */
- 0x00000006 /* EMC_QUSE_EXTRA */
- 0x00000004 /* EMC_FBIO_CFG6 */
- 0x00000000 /* EMC_ODT_WRITE */
- 0x00000000 /* EMC_ODT_READ */
- 0x00004288 /* EMC_FBIO_CFG5 */
- 0x007800a4 /* EMC_CFG_DIG_DLL */
- 0x00008000 /* EMC_CFG_DIG_DLL_PERIOD */
- 0x000fc000 /* EMC_DLL_XFORM_DQS0 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS1 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS2 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS3 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS4 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS5 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS6 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS7 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE0 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE1 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE2 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE3 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE4 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE5 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE6 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE7 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS0 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS1 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS2 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS3 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS4 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS5 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS6 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS7 */
- 0x000fc000 /* EMC_DLL_XFORM_DQ0 */
- 0x000fc000 /* EMC_DLL_XFORM_DQ1 */
- 0x000fc000 /* EMC_DLL_XFORM_DQ2 */
- 0x000fc000 /* EMC_DLL_XFORM_DQ3 */
- 0x000002a0 /* EMC_XM2CMDPADCTRL */
- 0x0800211c /* EMC_XM2DQSPADCTRL2 */
- 0x00000000 /* EMC_XM2DQPADCTRL2 */
- 0x77fff884 /* EMC_XM2CLKPADCTRL */
- 0x01f1f108 /* EMC_XM2COMPPADCTRL */
- 0x05057404 /* EMC_XM2VTTGENPADCTRL */
- 0x54000007 /* EMC_XM2VTTGENPADCTRL2 */
- 0x08000168 /* EMC_XM2QUSEPADCTRL */
- 0x08000000 /* EMC_XM2DQSPADCTRL3 */
- 0x00000802 /* EMC_CTT_TERM_CTRL */
- 0x00000000 /* EMC_ZCAL_INTERVAL */
- 0x00000040 /* EMC_ZCAL_WAIT_CNT */
- 0x000c000c /* EMC_MRS_WAIT_CNT */
- 0xa0f10000 /* EMC_AUTO_CAL_CONFIG */
- 0x00000000 /* EMC_CTT */
- 0x00000000 /* EMC_CTT_DURATION */
- 0x80000713 /* EMC_DYN_SELF_REF_CONTROL */
- 0xe8000000 /* EMC_FBIO_SPARE */
- 0xff00ff00 /* EMC_CFG_RSV */
- >;
- };
- timing-204000000 {
- clock-frequency = <204000000>;
- nvidia,emc-auto-cal-interval = <0x001fffff>;
- nvidia,emc-mode-1 = <0x80100003>;
- nvidia,emc-mode-2 = <0x80200008>;
- nvidia,emc-mode-reset = <0x80001221>;
- nvidia,emc-zcal-cnt-long = <0x00000040>;
- nvidia,emc-cfg-periodic-qrst;
- nvidia,emc-cfg-dyn-self-ref;
- nvidia,emc-configuration = <
- 0x00000009 /* EMC_RC */
- 0x00000035 /* EMC_RFC */
- 0x00000007 /* EMC_RAS */
- 0x00000002 /* EMC_RP */
- 0x00000002 /* EMC_R2W */
- 0x0000000a /* EMC_W2R */
- 0x00000005 /* EMC_R2P */
- 0x0000000b /* EMC_W2P */
- 0x00000002 /* EMC_RD_RCD */
- 0x00000002 /* EMC_WR_RCD */
- 0x00000003 /* EMC_RRD */
- 0x00000001 /* EMC_REXT */
- 0x00000000 /* EMC_WEXT */
- 0x00000005 /* EMC_WDV */
- 0x00000005 /* EMC_QUSE */
- 0x00000004 /* EMC_QRST */
- 0x0000000a /* EMC_QSAFE */
- 0x0000000b /* EMC_RDV */
- 0x00000607 /* EMC_REFRESH */
- 0x00000000 /* EMC_BURST_REFRESH_NUM */
- 0x00000181 /* EMC_PRE_REFRESH_REQ_CNT */
- 0x00000002 /* EMC_PDEX2WR */
- 0x00000002 /* EMC_PDEX2RD */
- 0x00000001 /* EMC_PCHG2PDEN */
- 0x00000000 /* EMC_ACT2PDEN */
- 0x00000007 /* EMC_AR2PDEN */
- 0x0000000f /* EMC_RW2PDEN */
- 0x00000038 /* EMC_TXSR */
- 0x00000038 /* EMC_TXSRDLL */
- 0x00000004 /* EMC_TCKE */
- 0x00000009 /* EMC_TFAW */
- 0x00000000 /* EMC_TRPAB */
- 0x00000004 /* EMC_TCLKSTABLE */
- 0x00000005 /* EMC_TCLKSTOP */
- 0x00000638 /* EMC_TREFBW */
- 0x00000006 /* EMC_QUSE_EXTRA */
- 0x00000006 /* EMC_FBIO_CFG6 */
- 0x00000000 /* EMC_ODT_WRITE */
- 0x00000000 /* EMC_ODT_READ */
- 0x00004288 /* EMC_FBIO_CFG5 */
- 0x004400a4 /* EMC_CFG_DIG_DLL */
- 0x00008000 /* EMC_CFG_DIG_DLL_PERIOD */
- 0x00080000 /* EMC_DLL_XFORM_DQS0 */
- 0x00080000 /* EMC_DLL_XFORM_DQS1 */
- 0x00080000 /* EMC_DLL_XFORM_DQS2 */
- 0x00080000 /* EMC_DLL_XFORM_DQS3 */
- 0x00080000 /* EMC_DLL_XFORM_DQS4 */
- 0x00080000 /* EMC_DLL_XFORM_DQS5 */
- 0x00080000 /* EMC_DLL_XFORM_DQS6 */
- 0x00080000 /* EMC_DLL_XFORM_DQS7 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE0 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE1 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE2 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE3 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE4 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE5 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE6 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE7 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS0 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS1 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS2 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS3 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS4 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS5 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS6 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS7 */
- 0x00080000 /* EMC_DLL_XFORM_DQ0 */
- 0x00080000 /* EMC_DLL_XFORM_DQ1 */
- 0x00080000 /* EMC_DLL_XFORM_DQ2 */
- 0x00080000 /* EMC_DLL_XFORM_DQ3 */
- 0x000002a0 /* EMC_XM2CMDPADCTRL */
- 0x0800211c /* EMC_XM2DQSPADCTRL2 */
- 0x00000000 /* EMC_XM2DQPADCTRL2 */
- 0x77fff884 /* EMC_XM2CLKPADCTRL */
- 0x01f1f108 /* EMC_XM2COMPPADCTRL */
- 0x05057404 /* EMC_XM2VTTGENPADCTRL */
- 0x54000007 /* EMC_XM2VTTGENPADCTRL2 */
- 0x08000168 /* EMC_XM2QUSEPADCTRL */
- 0x08000000 /* EMC_XM2DQSPADCTRL3 */
- 0x00000802 /* EMC_CTT_TERM_CTRL */
- 0x00020000 /* EMC_ZCAL_INTERVAL */
- 0x00000100 /* EMC_ZCAL_WAIT_CNT */
- 0x000c000c /* EMC_MRS_WAIT_CNT */
- 0xa0f10000 /* EMC_AUTO_CAL_CONFIG */
- 0x00000000 /* EMC_CTT */
- 0x00000000 /* EMC_CTT_DURATION */
- 0x80000d22 /* EMC_DYN_SELF_REF_CONTROL */
- 0xe8000000 /* EMC_FBIO_SPARE */
- 0xff00ff00 /* EMC_CFG_RSV */
- >;
- };
- timing-400000000 {
- clock-frequency = <400000000>;
- nvidia,emc-auto-cal-interval = <0x001fffff>;
- nvidia,emc-mode-1 = <0x80100002>;
- nvidia,emc-mode-2 = <0x80200000>;
- nvidia,emc-mode-reset = <0x80000521>;
- nvidia,emc-zcal-cnt-long = <0x00000040>;
- nvidia,emc-configuration = <
- 0x00000012 /* EMC_RC */
- 0x00000066 /* EMC_RFC */
- 0x0000000c /* EMC_RAS */
- 0x00000004 /* EMC_RP */
- 0x00000003 /* EMC_R2W */
- 0x00000008 /* EMC_W2R */
- 0x00000002 /* EMC_R2P */
- 0x0000000a /* EMC_W2P */
- 0x00000004 /* EMC_RD_RCD */
- 0x00000004 /* EMC_WR_RCD */
- 0x00000002 /* EMC_RRD */
- 0x00000001 /* EMC_REXT */
- 0x00000000 /* EMC_WEXT */
- 0x00000004 /* EMC_WDV */
- 0x00000006 /* EMC_QUSE */
- 0x00000004 /* EMC_QRST */
- 0x0000000a /* EMC_QSAFE */
- 0x0000000c /* EMC_RDV */
- 0x00000bf0 /* EMC_REFRESH */
- 0x00000000 /* EMC_BURST_REFRESH_NUM */
- 0x000002fc /* EMC_PRE_REFRESH_REQ_CNT */
- 0x00000001 /* EMC_PDEX2WR */
- 0x00000008 /* EMC_PDEX2RD */
- 0x00000001 /* EMC_PCHG2PDEN */
- 0x00000000 /* EMC_ACT2PDEN */
- 0x00000008 /* EMC_AR2PDEN */
- 0x0000000f /* EMC_RW2PDEN */
- 0x0000006c /* EMC_TXSR */
- 0x00000200 /* EMC_TXSRDLL */
- 0x00000004 /* EMC_TCKE */
- 0x00000010 /* EMC_TFAW */
- 0x00000000 /* EMC_TRPAB */
- 0x00000004 /* EMC_TCLKSTABLE */
- 0x00000005 /* EMC_TCLKSTOP */
- 0x00000c30 /* EMC_TREFBW */
- 0x00000000 /* EMC_QUSE_EXTRA */
- 0x00000004 /* EMC_FBIO_CFG6 */
- 0x00000000 /* EMC_ODT_WRITE */
- 0x00000000 /* EMC_ODT_READ */
- 0x00007088 /* EMC_FBIO_CFG5 */
- 0x001d0084 /* EMC_CFG_DIG_DLL */
- 0x00008000 /* EMC_CFG_DIG_DLL_PERIOD */
- 0x0003c000 /* EMC_DLL_XFORM_DQS0 */
- 0x0003c000 /* EMC_DLL_XFORM_DQS1 */
- 0x0003c000 /* EMC_DLL_XFORM_DQS2 */
- 0x0003c000 /* EMC_DLL_XFORM_DQS3 */
- 0x0003c000 /* EMC_DLL_XFORM_DQS4 */
- 0x0003c000 /* EMC_DLL_XFORM_DQS5 */
- 0x0003c000 /* EMC_DLL_XFORM_DQS6 */
- 0x0003c000 /* EMC_DLL_XFORM_DQS7 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE0 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE1 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE2 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE3 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE4 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE5 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE6 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE7 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS0 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS1 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS2 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS3 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS4 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS5 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS6 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS7 */
- 0x00048000 /* EMC_DLL_XFORM_DQ0 */
- 0x00048000 /* EMC_DLL_XFORM_DQ1 */
- 0x00048000 /* EMC_DLL_XFORM_DQ2 */
- 0x00048000 /* EMC_DLL_XFORM_DQ3 */
- 0x000002a0 /* EMC_XM2CMDPADCTRL */
- 0x0800013d /* EMC_XM2DQSPADCTRL2 */
- 0x00000000 /* EMC_XM2DQPADCTRL2 */
- 0x77fff884 /* EMC_XM2CLKPADCTRL */
- 0x01f1f508 /* EMC_XM2COMPPADCTRL */
- 0x05057404 /* EMC_XM2VTTGENPADCTRL */
- 0x54000007 /* EMC_XM2VTTGENPADCTRL2 */
- 0x080001e8 /* EMC_XM2QUSEPADCTRL */
- 0x08000021 /* EMC_XM2DQSPADCTRL3 */
- 0x00000802 /* EMC_CTT_TERM_CTRL */
- 0x00020000 /* EMC_ZCAL_INTERVAL */
- 0x00000100 /* EMC_ZCAL_WAIT_CNT */
- 0x0158000c /* EMC_MRS_WAIT_CNT */
- 0xa0f10000 /* EMC_AUTO_CAL_CONFIG */
- 0x00000000 /* EMC_CTT */
- 0x00000000 /* EMC_CTT_DURATION */
- 0x800018c8 /* EMC_DYN_SELF_REF_CONTROL */
- 0xe8000000 /* EMC_FBIO_SPARE */
- 0xff00ff89 /* EMC_CFG_RSV */
- >;
- };
- timing-800000000 {
- clock-frequency = <800000000>;
- nvidia,emc-auto-cal-interval = <0x001fffff>;
- nvidia,emc-mode-1 = <0x80100002>;
- nvidia,emc-mode-2 = <0x80200018>;
- nvidia,emc-mode-reset = <0x80000d71>;
- nvidia,emc-zcal-cnt-long = <0x00000040>;
- nvidia,emc-cfg-periodic-qrst;
- nvidia,emc-configuration = <
- 0x00000025 /* EMC_RC */
- 0x000000ce /* EMC_RFC */
- 0x0000001a /* EMC_RAS */
- 0x00000009 /* EMC_RP */
- 0x00000005 /* EMC_R2W */
- 0x0000000d /* EMC_W2R */
- 0x00000004 /* EMC_R2P */
- 0x00000013 /* EMC_W2P */
- 0x00000009 /* EMC_RD_RCD */
- 0x00000009 /* EMC_WR_RCD */
- 0x00000004 /* EMC_RRD */
- 0x00000001 /* EMC_REXT */
- 0x00000000 /* EMC_WEXT */
- 0x00000007 /* EMC_WDV */
- 0x0000000a /* EMC_QUSE */
- 0x00000009 /* EMC_QRST */
- 0x0000000b /* EMC_QSAFE */
- 0x00000011 /* EMC_RDV */
- 0x00001820 /* EMC_REFRESH */
- 0x00000000 /* EMC_BURST_REFRESH_NUM */
- 0x00000608 /* EMC_PRE_REFRESH_REQ_CNT */
- 0x00000003 /* EMC_PDEX2WR */
- 0x00000012 /* EMC_PDEX2RD */
- 0x00000001 /* EMC_PCHG2PDEN */
- 0x00000000 /* EMC_ACT2PDEN */
- 0x0000000f /* EMC_AR2PDEN */
- 0x00000018 /* EMC_RW2PDEN */
- 0x000000d8 /* EMC_TXSR */
- 0x00000200 /* EMC_TXSRDLL */
- 0x00000005 /* EMC_TCKE */
- 0x00000020 /* EMC_TFAW */
- 0x00000000 /* EMC_TRPAB */
- 0x00000007 /* EMC_TCLKSTABLE */
- 0x00000008 /* EMC_TCLKSTOP */
- 0x00001860 /* EMC_TREFBW */
- 0x0000000b /* EMC_QUSE_EXTRA */
- 0x00000006 /* EMC_FBIO_CFG6 */
- 0x00000000 /* EMC_ODT_WRITE */
- 0x00000000 /* EMC_ODT_READ */
- 0x00005088 /* EMC_FBIO_CFG5 */
- 0xf0070191 /* EMC_CFG_DIG_DLL */
- 0x00008000 /* EMC_CFG_DIG_DLL_PERIOD */
- 0x0000800a /* EMC_DLL_XFORM_DQS0 */
- 0x0000000a /* EMC_DLL_XFORM_DQS1 */
- 0x0000000a /* EMC_DLL_XFORM_DQS2 */
- 0x0000000a /* EMC_DLL_XFORM_DQS3 */
- 0x0000000a /* EMC_DLL_XFORM_DQS4 */
- 0x0000000a /* EMC_DLL_XFORM_DQS5 */
- 0x0000000a /* EMC_DLL_XFORM_DQS6 */
- 0x0000000a /* EMC_DLL_XFORM_DQS7 */
- 0x00018000 /* EMC_DLL_XFORM_QUSE0 */
- 0x00018000 /* EMC_DLL_XFORM_QUSE1 */
- 0x00018000 /* EMC_DLL_XFORM_QUSE2 */
- 0x00018000 /* EMC_DLL_XFORM_QUSE3 */
- 0x00018000 /* EMC_DLL_XFORM_QUSE4 */
- 0x00018000 /* EMC_DLL_XFORM_QUSE5 */
- 0x00018000 /* EMC_DLL_XFORM_QUSE6 */
- 0x00018000 /* EMC_DLL_XFORM_QUSE7 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS0 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS1 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS2 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS3 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS4 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS5 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS6 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS7 */
- 0x0000000a /* EMC_DLL_XFORM_DQ0 */
- 0x0000000a /* EMC_DLL_XFORM_DQ1 */
- 0x0000000a /* EMC_DLL_XFORM_DQ2 */
- 0x0000000a /* EMC_DLL_XFORM_DQ3 */
- 0x000002a0 /* EMC_XM2CMDPADCTRL */
- 0x0600013d /* EMC_XM2DQSPADCTRL2 */
- 0x22220000 /* EMC_XM2DQPADCTRL2 */
- 0x77fff884 /* EMC_XM2CLKPADCTRL */
- 0x01f1f501 /* EMC_XM2COMPPADCTRL */
- 0x07077404 /* EMC_XM2VTTGENPADCTRL */
- 0x54000000 /* EMC_XM2VTTGENPADCTRL2 */
- 0x080001e8 /* EMC_XM2QUSEPADCTRL */
- 0x08000021 /* EMC_XM2DQSPADCTRL3 */
- 0x00000802 /* EMC_CTT_TERM_CTRL */
- 0x00020000 /* EMC_ZCAL_INTERVAL */
- 0x00000100 /* EMC_ZCAL_WAIT_CNT */
- 0x00f0000c /* EMC_MRS_WAIT_CNT */
- 0xa0f10000 /* EMC_AUTO_CAL_CONFIG */
- 0x00000000 /* EMC_CTT */
- 0x00000000 /* EMC_CTT_DURATION */
- 0x8000308c /* EMC_DYN_SELF_REF_CONTROL */
- 0xe8000000 /* EMC_FBIO_SPARE */
- 0xff00ff49 /* EMC_CFG_RSV */
- >;
- };
- };
- emc-timings-1 {
- nvidia,ram-code = <1>; /* Hynix M RAM */
- timing-25500000 {
- clock-frequency = <25500000>;
- nvidia,emc-auto-cal-interval = <0x001fffff>;
- nvidia,emc-mode-1 = <0x80100003>;
- nvidia,emc-mode-2 = <0x80200008>;
- nvidia,emc-mode-reset = <0x80001221>;
- nvidia,emc-zcal-cnt-long = <0x00000040>;
- nvidia,emc-cfg-periodic-qrst;
- nvidia,emc-cfg-dyn-self-ref;
- nvidia,emc-configuration = <
- 0x00000001 /* EMC_RC */
- 0x00000006 /* EMC_RFC */
- 0x00000000 /* EMC_RAS */
- 0x00000000 /* EMC_RP */
- 0x00000002 /* EMC_R2W */
- 0x0000000a /* EMC_W2R */
- 0x00000005 /* EMC_R2P */
- 0x0000000b /* EMC_W2P */
- 0x00000000 /* EMC_RD_RCD */
- 0x00000000 /* EMC_WR_RCD */
- 0x00000003 /* EMC_RRD */
- 0x00000001 /* EMC_REXT */
- 0x00000000 /* EMC_WEXT */
- 0x00000005 /* EMC_WDV */
- 0x00000005 /* EMC_QUSE */
- 0x00000004 /* EMC_QRST */
- 0x0000000a /* EMC_QSAFE */
- 0x0000000b /* EMC_RDV */
- 0x000000c0 /* EMC_REFRESH */
- 0x00000000 /* EMC_BURST_REFRESH_NUM */
- 0x00000030 /* EMC_PRE_REFRESH_REQ_CNT */
- 0x00000002 /* EMC_PDEX2WR */
- 0x00000002 /* EMC_PDEX2RD */
- 0x00000001 /* EMC_PCHG2PDEN */
- 0x00000000 /* EMC_ACT2PDEN */
- 0x00000007 /* EMC_AR2PDEN */
- 0x0000000f /* EMC_RW2PDEN */
- 0x00000007 /* EMC_TXSR */
- 0x00000007 /* EMC_TXSRDLL */
- 0x00000004 /* EMC_TCKE */
- 0x00000002 /* EMC_TFAW */
- 0x00000000 /* EMC_TRPAB */
- 0x00000004 /* EMC_TCLKSTABLE */
- 0x00000005 /* EMC_TCLKSTOP */
- 0x000000c7 /* EMC_TREFBW */
- 0x00000006 /* EMC_QUSE_EXTRA */
- 0x00000004 /* EMC_FBIO_CFG6 */
- 0x00000000 /* EMC_ODT_WRITE */
- 0x00000000 /* EMC_ODT_READ */
- 0x00004288 /* EMC_FBIO_CFG5 */
- 0x007800a4 /* EMC_CFG_DIG_DLL */
- 0x00008000 /* EMC_CFG_DIG_DLL_PERIOD */
- 0x000fc000 /* EMC_DLL_XFORM_DQS0 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS1 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS2 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS3 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS4 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS5 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS6 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS7 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE0 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE1 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE2 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE3 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE4 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE5 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE6 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE7 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS0 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS1 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS2 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS3 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS4 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS5 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS6 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS7 */
- 0x000fc000 /* EMC_DLL_XFORM_DQ0 */
- 0x000fc000 /* EMC_DLL_XFORM_DQ1 */
- 0x000fc000 /* EMC_DLL_XFORM_DQ2 */
- 0x000fc000 /* EMC_DLL_XFORM_DQ3 */
- 0x000002a0 /* EMC_XM2CMDPADCTRL */
- 0x0800211c /* EMC_XM2DQSPADCTRL2 */
- 0x00000000 /* EMC_XM2DQPADCTRL2 */
- 0x77fff884 /* EMC_XM2CLKPADCTRL */
- 0x01f1f108 /* EMC_XM2COMPPADCTRL */
- 0x05057404 /* EMC_XM2VTTGENPADCTRL */
- 0x54000007 /* EMC_XM2VTTGENPADCTRL2 */
- 0x08000168 /* EMC_XM2QUSEPADCTRL */
- 0x08000000 /* EMC_XM2DQSPADCTRL3 */
- 0x00000802 /* EMC_CTT_TERM_CTRL */
- 0x00000000 /* EMC_ZCAL_INTERVAL */
- 0x00000040 /* EMC_ZCAL_WAIT_CNT */
- 0x000c000c /* EMC_MRS_WAIT_CNT */
- 0xa0f10000 /* EMC_AUTO_CAL_CONFIG */
- 0x00000000 /* EMC_CTT */
- 0x00000000 /* EMC_CTT_DURATION */
- 0x80000287 /* EMC_DYN_SELF_REF_CONTROL */
- 0xe8000000 /* EMC_FBIO_SPARE */
- 0xff00ff00 /* EMC_CFG_RSV */
- >;
- };
- timing-51000000 {
- clock-frequency = <51000000>;
- nvidia,emc-auto-cal-interval = <0x001fffff>;
- nvidia,emc-mode-1 = <0x80100003>;
- nvidia,emc-mode-2 = <0x80200008>;
- nvidia,emc-mode-reset = <0x80001221>;
- nvidia,emc-zcal-cnt-long = <0x00000040>;
- nvidia,emc-cfg-periodic-qrst;
- nvidia,emc-cfg-dyn-self-ref;
- nvidia,emc-configuration = <
- 0x00000002 /* EMC_RC */
- 0x0000000d /* EMC_RFC */
- 0x00000001 /* EMC_RAS */
- 0x00000000 /* EMC_RP */
- 0x00000002 /* EMC_R2W */
- 0x0000000a /* EMC_W2R */
- 0x00000005 /* EMC_R2P */
- 0x0000000b /* EMC_W2P */
- 0x00000000 /* EMC_RD_RCD */
- 0x00000000 /* EMC_WR_RCD */
- 0x00000003 /* EMC_RRD */
- 0x00000001 /* EMC_REXT */
- 0x00000000 /* EMC_WEXT */
- 0x00000005 /* EMC_WDV */
- 0x00000005 /* EMC_QUSE */
- 0x00000004 /* EMC_QRST */
- 0x0000000a /* EMC_QSAFE */
- 0x0000000b /* EMC_RDV */
- 0x00000181 /* EMC_REFRESH */
- 0x00000000 /* EMC_BURST_REFRESH_NUM */
- 0x00000060 /* EMC_PRE_REFRESH_REQ_CNT */
- 0x00000002 /* EMC_PDEX2WR */
- 0x00000002 /* EMC_PDEX2RD */
- 0x00000001 /* EMC_PCHG2PDEN */
- 0x00000000 /* EMC_ACT2PDEN */
- 0x00000007 /* EMC_AR2PDEN */
- 0x0000000f /* EMC_RW2PDEN */
- 0x0000000e /* EMC_TXSR */
- 0x0000000e /* EMC_TXSRDLL */
- 0x00000004 /* EMC_TCKE */
- 0x00000003 /* EMC_TFAW */
- 0x00000000 /* EMC_TRPAB */
- 0x00000004 /* EMC_TCLKSTABLE */
- 0x00000005 /* EMC_TCLKSTOP */
- 0x0000018e /* EMC_TREFBW */
- 0x00000006 /* EMC_QUSE_EXTRA */
- 0x00000004 /* EMC_FBIO_CFG6 */
- 0x00000000 /* EMC_ODT_WRITE */
- 0x00000000 /* EMC_ODT_READ */
- 0x00004288 /* EMC_FBIO_CFG5 */
- 0x007800a4 /* EMC_CFG_DIG_DLL */
- 0x00008000 /* EMC_CFG_DIG_DLL_PERIOD */
- 0x000fc000 /* EMC_DLL_XFORM_DQS0 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS1 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS2 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS3 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS4 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS5 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS6 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS7 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE0 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE1 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE2 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE3 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE4 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE5 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE6 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE7 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS0 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS1 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS2 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS3 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS4 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS5 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS6 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS7 */
- 0x000fc000 /* EMC_DLL_XFORM_DQ0 */
- 0x000fc000 /* EMC_DLL_XFORM_DQ1 */
- 0x000fc000 /* EMC_DLL_XFORM_DQ2 */
- 0x000fc000 /* EMC_DLL_XFORM_DQ3 */
- 0x000002a0 /* EMC_XM2CMDPADCTRL */
- 0x0800211c /* EMC_XM2DQSPADCTRL2 */
- 0x00000000 /* EMC_XM2DQPADCTRL2 */
- 0x77fff884 /* EMC_XM2CLKPADCTRL */
- 0x01f1f108 /* EMC_XM2COMPPADCTRL */
- 0x05057404 /* EMC_XM2VTTGENPADCTRL */
- 0x54000007 /* EMC_XM2VTTGENPADCTRL2 */
- 0x08000168 /* EMC_XM2QUSEPADCTRL */
- 0x08000000 /* EMC_XM2DQSPADCTRL3 */
- 0x00000802 /* EMC_CTT_TERM_CTRL */
- 0x00000000 /* EMC_ZCAL_INTERVAL */
- 0x00000040 /* EMC_ZCAL_WAIT_CNT */
- 0x000c000c /* EMC_MRS_WAIT_CNT */
- 0xa0f10000 /* EMC_AUTO_CAL_CONFIG */
- 0x00000000 /* EMC_CTT */
- 0x00000000 /* EMC_CTT_DURATION */
- 0x8000040b /* EMC_DYN_SELF_REF_CONTROL */
- 0xe8000000 /* EMC_FBIO_SPARE */
- 0xff00ff00 /* EMC_CFG_RSV */
- >;
- };
- timing-102000000 {
- clock-frequency = <102000000>;
- nvidia,emc-auto-cal-interval = <0x001fffff>;
- nvidia,emc-mode-1 = <0x80100003>;
- nvidia,emc-mode-2 = <0x80200008>;
- nvidia,emc-mode-reset = <0x80001221>;
- nvidia,emc-zcal-cnt-long = <0x00000040>;
- nvidia,emc-cfg-periodic-qrst;
- nvidia,emc-cfg-dyn-self-ref;
- nvidia,emc-configuration = <
- 0x00000004 /* EMC_RC */
- 0x0000001a /* EMC_RFC */
- 0x00000003 /* EMC_RAS */
- 0x00000001 /* EMC_RP */
- 0x00000002 /* EMC_R2W */
- 0x0000000a /* EMC_W2R */
- 0x00000005 /* EMC_R2P */
- 0x0000000b /* EMC_W2P */
- 0x00000001 /* EMC_RD_RCD */
- 0x00000001 /* EMC_WR_RCD */
- 0x00000003 /* EMC_RRD */
- 0x00000001 /* EMC_REXT */
- 0x00000000 /* EMC_WEXT */
- 0x00000005 /* EMC_WDV */
- 0x00000005 /* EMC_QUSE */
- 0x00000004 /* EMC_QRST */
- 0x0000000a /* EMC_QSAFE */
- 0x0000000b /* EMC_RDV */
- 0x00000303 /* EMC_REFRESH */
- 0x00000000 /* EMC_BURST_REFRESH_NUM */
- 0x000000c0 /* EMC_PRE_REFRESH_REQ_CNT */
- 0x00000002 /* EMC_PDEX2WR */
- 0x00000002 /* EMC_PDEX2RD */
- 0x00000001 /* EMC_PCHG2PDEN */
- 0x00000000 /* EMC_ACT2PDEN */
- 0x00000007 /* EMC_AR2PDEN */
- 0x0000000f /* EMC_RW2PDEN */
- 0x0000001c /* EMC_TXSR */
- 0x0000001c /* EMC_TXSRDLL */
- 0x00000004 /* EMC_TCKE */
- 0x00000005 /* EMC_TFAW */
- 0x00000000 /* EMC_TRPAB */
- 0x00000004 /* EMC_TCLKSTABLE */
- 0x00000005 /* EMC_TCLKSTOP */
- 0x0000031c /* EMC_TREFBW */
- 0x00000006 /* EMC_QUSE_EXTRA */
- 0x00000004 /* EMC_FBIO_CFG6 */
- 0x00000000 /* EMC_ODT_WRITE */
- 0x00000000 /* EMC_ODT_READ */
- 0x00004288 /* EMC_FBIO_CFG5 */
- 0x007800a4 /* EMC_CFG_DIG_DLL */
- 0x00008000 /* EMC_CFG_DIG_DLL_PERIOD */
- 0x000fc000 /* EMC_DLL_XFORM_DQS0 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS1 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS2 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS3 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS4 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS5 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS6 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS7 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE0 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE1 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE2 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE3 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE4 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE5 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE6 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE7 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS0 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS1 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS2 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS3 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS4 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS5 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS6 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS7 */
- 0x000fc000 /* EMC_DLL_XFORM_DQ0 */
- 0x000fc000 /* EMC_DLL_XFORM_DQ1 */
- 0x000fc000 /* EMC_DLL_XFORM_DQ2 */
- 0x000fc000 /* EMC_DLL_XFORM_DQ3 */
- 0x000002a0 /* EMC_XM2CMDPADCTRL */
- 0x0800211c /* EMC_XM2DQSPADCTRL2 */
- 0x00000000 /* EMC_XM2DQPADCTRL2 */
- 0x77fff884 /* EMC_XM2CLKPADCTRL */
- 0x01f1f108 /* EMC_XM2COMPPADCTRL */
- 0x05057404 /* EMC_XM2VTTGENPADCTRL */
- 0x54000007 /* EMC_XM2VTTGENPADCTRL2 */
- 0x08000168 /* EMC_XM2QUSEPADCTRL */
- 0x08000000 /* EMC_XM2DQSPADCTRL3 */
- 0x00000802 /* EMC_CTT_TERM_CTRL */
- 0x00000000 /* EMC_ZCAL_INTERVAL */
- 0x00000040 /* EMC_ZCAL_WAIT_CNT */
- 0x000c000c /* EMC_MRS_WAIT_CNT */
- 0xa0f10000 /* EMC_AUTO_CAL_CONFIG */
- 0x00000000 /* EMC_CTT */
- 0x00000000 /* EMC_CTT_DURATION */
- 0x80000713 /* EMC_DYN_SELF_REF_CONTROL */
- 0xe8000000 /* EMC_FBIO_SPARE */
- 0xff00ff00 /* EMC_CFG_RSV */
- >;
- };
- timing-204000000 {
- clock-frequency = <204000000>;
- nvidia,emc-auto-cal-interval = <0x001fffff>;
- nvidia,emc-mode-1 = <0x80100003>;
- nvidia,emc-mode-2 = <0x80200008>;
- nvidia,emc-mode-reset = <0x80001221>;
- nvidia,emc-zcal-cnt-long = <0x00000040>;
- nvidia,emc-cfg-periodic-qrst;
- nvidia,emc-cfg-dyn-self-ref;
- nvidia,emc-configuration = <
- 0x00000009 /* EMC_RC */
- 0x00000035 /* EMC_RFC */
- 0x00000007 /* EMC_RAS */
- 0x00000002 /* EMC_RP */
- 0x00000002 /* EMC_R2W */
- 0x0000000a /* EMC_W2R */
- 0x00000005 /* EMC_R2P */
- 0x0000000b /* EMC_W2P */
- 0x00000002 /* EMC_RD_RCD */
- 0x00000002 /* EMC_WR_RCD */
- 0x00000003 /* EMC_RRD */
- 0x00000001 /* EMC_REXT */
- 0x00000000 /* EMC_WEXT */
- 0x00000005 /* EMC_WDV */
- 0x00000005 /* EMC_QUSE */
- 0x00000004 /* EMC_QRST */
- 0x0000000a /* EMC_QSAFE */
- 0x0000000b /* EMC_RDV */
- 0x00000607 /* EMC_REFRESH */
- 0x00000000 /* EMC_BURST_REFRESH_NUM */
- 0x00000181 /* EMC_PRE_REFRESH_REQ_CNT */
- 0x00000002 /* EMC_PDEX2WR */
- 0x00000002 /* EMC_PDEX2RD */
- 0x00000001 /* EMC_PCHG2PDEN */
- 0x00000000 /* EMC_ACT2PDEN */
- 0x00000007 /* EMC_AR2PDEN */
- 0x0000000f /* EMC_RW2PDEN */
- 0x00000038 /* EMC_TXSR */
- 0x00000038 /* EMC_TXSRDLL */
- 0x00000004 /* EMC_TCKE */
- 0x00000009 /* EMC_TFAW */
- 0x00000000 /* EMC_TRPAB */
- 0x00000004 /* EMC_TCLKSTABLE */
- 0x00000005 /* EMC_TCLKSTOP */
- 0x00000638 /* EMC_TREFBW */
- 0x00000006 /* EMC_QUSE_EXTRA */
- 0x00000006 /* EMC_FBIO_CFG6 */
- 0x00000000 /* EMC_ODT_WRITE */
- 0x00000000 /* EMC_ODT_READ */
- 0x00004288 /* EMC_FBIO_CFG5 */
- 0x004400a4 /* EMC_CFG_DIG_DLL */
- 0x00008000 /* EMC_CFG_DIG_DLL_PERIOD */
- 0x00080000 /* EMC_DLL_XFORM_DQS0 */
- 0x00080000 /* EMC_DLL_XFORM_DQS1 */
- 0x00080000 /* EMC_DLL_XFORM_DQS2 */
- 0x00080000 /* EMC_DLL_XFORM_DQS3 */
- 0x00080000 /* EMC_DLL_XFORM_DQS4 */
- 0x00080000 /* EMC_DLL_XFORM_DQS5 */
- 0x00080000 /* EMC_DLL_XFORM_DQS6 */
- 0x00080000 /* EMC_DLL_XFORM_DQS7 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE0 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE1 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE2 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE3 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE4 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE5 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE6 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE7 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS0 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS1 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS2 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS3 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS4 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS5 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS6 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS7 */
- 0x00080000 /* EMC_DLL_XFORM_DQ0 */
- 0x00080000 /* EMC_DLL_XFORM_DQ1 */
- 0x00080000 /* EMC_DLL_XFORM_DQ2 */
- 0x00080000 /* EMC_DLL_XFORM_DQ3 */
- 0x000002a0 /* EMC_XM2CMDPADCTRL */
- 0x0800211c /* EMC_XM2DQSPADCTRL2 */
- 0x00000000 /* EMC_XM2DQPADCTRL2 */
- 0x77fff884 /* EMC_XM2CLKPADCTRL */
- 0x01f1f108 /* EMC_XM2COMPPADCTRL */
- 0x05057404 /* EMC_XM2VTTGENPADCTRL */
- 0x54000007 /* EMC_XM2VTTGENPADCTRL2 */
- 0x08000168 /* EMC_XM2QUSEPADCTRL */
- 0x08000000 /* EMC_XM2DQSPADCTRL3 */
- 0x00000802 /* EMC_CTT_TERM_CTRL */
- 0x00020000 /* EMC_ZCAL_INTERVAL */
- 0x00000100 /* EMC_ZCAL_WAIT_CNT */
- 0x000c000c /* EMC_MRS_WAIT_CNT */
- 0xa0f10000 /* EMC_AUTO_CAL_CONFIG */
- 0x00000000 /* EMC_CTT */
- 0x00000000 /* EMC_CTT_DURATION */
- 0x80000d22 /* EMC_DYN_SELF_REF_CONTROL */
- 0xe8000000 /* EMC_FBIO_SPARE */
- 0xff00ff00 /* EMC_CFG_RSV */
- >;
- };
- timing-400000000 {
- clock-frequency = <400000000>;
- nvidia,emc-auto-cal-interval = <0x001fffff>;
- nvidia,emc-mode-1 = <0x80100002>;
- nvidia,emc-mode-2 = <0x80200000>;
- nvidia,emc-mode-reset = <0x80000521>;
- nvidia,emc-zcal-cnt-long = <0x00000040>;
- nvidia,emc-configuration = <
- 0x00000012 /* EMC_RC */
- 0x00000066 /* EMC_RFC */
- 0x0000000c /* EMC_RAS */
- 0x00000004 /* EMC_RP */
- 0x00000003 /* EMC_R2W */
- 0x00000008 /* EMC_W2R */
- 0x00000002 /* EMC_R2P */
- 0x0000000a /* EMC_W2P */
- 0x00000004 /* EMC_RD_RCD */
- 0x00000004 /* EMC_WR_RCD */
- 0x00000002 /* EMC_RRD */
- 0x00000001 /* EMC_REXT */
- 0x00000000 /* EMC_WEXT */
- 0x00000004 /* EMC_WDV */
- 0x00000006 /* EMC_QUSE */
- 0x00000004 /* EMC_QRST */
- 0x0000000a /* EMC_QSAFE */
- 0x0000000c /* EMC_RDV */
- 0x00000bf0 /* EMC_REFRESH */
- 0x00000000 /* EMC_BURST_REFRESH_NUM */
- 0x000002fc /* EMC_PRE_REFRESH_REQ_CNT */
- 0x00000001 /* EMC_PDEX2WR */
- 0x00000008 /* EMC_PDEX2RD */
- 0x00000001 /* EMC_PCHG2PDEN */
- 0x00000000 /* EMC_ACT2PDEN */
- 0x00000008 /* EMC_AR2PDEN */
- 0x0000000f /* EMC_RW2PDEN */
- 0x0000006c /* EMC_TXSR */
- 0x00000200 /* EMC_TXSRDLL */
- 0x00000004 /* EMC_TCKE */
- 0x00000010 /* EMC_TFAW */
- 0x00000000 /* EMC_TRPAB */
- 0x00000004 /* EMC_TCLKSTABLE */
- 0x00000005 /* EMC_TCLKSTOP */
- 0x00000c30 /* EMC_TREFBW */
- 0x00000000 /* EMC_QUSE_EXTRA */
- 0x00000004 /* EMC_FBIO_CFG6 */
- 0x00000000 /* EMC_ODT_WRITE */
- 0x00000000 /* EMC_ODT_READ */
- 0x00007088 /* EMC_FBIO_CFG5 */
- 0x001d0084 /* EMC_CFG_DIG_DLL */
- 0x00008000 /* EMC_CFG_DIG_DLL_PERIOD */
- 0x0003c000 /* EMC_DLL_XFORM_DQS0 */
- 0x0003c000 /* EMC_DLL_XFORM_DQS1 */
- 0x0003c000 /* EMC_DLL_XFORM_DQS2 */
- 0x0003c000 /* EMC_DLL_XFORM_DQS3 */
- 0x0003c000 /* EMC_DLL_XFORM_DQS4 */
- 0x0003c000 /* EMC_DLL_XFORM_DQS5 */
- 0x0003c000 /* EMC_DLL_XFORM_DQS6 */
- 0x0003c000 /* EMC_DLL_XFORM_DQS7 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE0 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE1 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE2 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE3 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE4 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE5 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE6 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE7 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS0 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS1 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS2 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS3 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS4 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS5 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS6 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS7 */
- 0x00048000 /* EMC_DLL_XFORM_DQ0 */
- 0x00048000 /* EMC_DLL_XFORM_DQ1 */
- 0x00048000 /* EMC_DLL_XFORM_DQ2 */
- 0x00048000 /* EMC_DLL_XFORM_DQ3 */
- 0x000002a0 /* EMC_XM2CMDPADCTRL */
- 0x0800013d /* EMC_XM2DQSPADCTRL2 */
- 0x00000000 /* EMC_XM2DQPADCTRL2 */
- 0x77fff884 /* EMC_XM2CLKPADCTRL */
- 0x01f1f508 /* EMC_XM2COMPPADCTRL */
- 0x05057404 /* EMC_XM2VTTGENPADCTRL */
- 0x54000007 /* EMC_XM2VTTGENPADCTRL2 */
- 0x080001e8 /* EMC_XM2QUSEPADCTRL */
- 0x08000021 /* EMC_XM2DQSPADCTRL3 */
- 0x00000802 /* EMC_CTT_TERM_CTRL */
- 0x00020000 /* EMC_ZCAL_INTERVAL */
- 0x00000100 /* EMC_ZCAL_WAIT_CNT */
- 0x0158000c /* EMC_MRS_WAIT_CNT */
- 0xa0f10000 /* EMC_AUTO_CAL_CONFIG */
- 0x00000000 /* EMC_CTT */
- 0x00000000 /* EMC_CTT_DURATION */
- 0x800018c8 /* EMC_DYN_SELF_REF_CONTROL */
- 0xe8000000 /* EMC_FBIO_SPARE */
- 0xff00ff89 /* EMC_CFG_RSV */
- >;
- };
- timing-800000000 {
- clock-frequency = <800000000>;
- nvidia,emc-auto-cal-interval = <0x001fffff>;
- nvidia,emc-mode-1 = <0x80100002>;
- nvidia,emc-mode-2 = <0x80200018>;
- nvidia,emc-mode-reset = <0x80000d71>;
- nvidia,emc-zcal-cnt-long = <0x00000040>;
- nvidia,emc-cfg-periodic-qrst;
- nvidia,emc-configuration = <
- 0x00000025 /* EMC_RC */
- 0x000000ce /* EMC_RFC */
- 0x0000001a /* EMC_RAS */
- 0x00000009 /* EMC_RP */
- 0x00000005 /* EMC_R2W */
- 0x0000000d /* EMC_W2R */
- 0x00000004 /* EMC_R2P */
- 0x00000013 /* EMC_W2P */
- 0x00000009 /* EMC_RD_RCD */
- 0x00000009 /* EMC_WR_RCD */
- 0x00000004 /* EMC_RRD */
- 0x00000001 /* EMC_REXT */
- 0x00000000 /* EMC_WEXT */
- 0x00000007 /* EMC_WDV */
- 0x0000000a /* EMC_QUSE */
- 0x00000009 /* EMC_QRST */
- 0x0000000b /* EMC_QSAFE */
- 0x00000011 /* EMC_RDV */
- 0x00001820 /* EMC_REFRESH */
- 0x00000000 /* EMC_BURST_REFRESH_NUM */
- 0x00000608 /* EMC_PRE_REFRESH_REQ_CNT */
- 0x00000003 /* EMC_PDEX2WR */
- 0x00000012 /* EMC_PDEX2RD */
- 0x00000001 /* EMC_PCHG2PDEN */
- 0x00000000 /* EMC_ACT2PDEN */
- 0x0000000f /* EMC_AR2PDEN */
- 0x00000018 /* EMC_RW2PDEN */
- 0x000000d8 /* EMC_TXSR */
- 0x00000200 /* EMC_TXSRDLL */
- 0x00000005 /* EMC_TCKE */
- 0x00000020 /* EMC_TFAW */
- 0x00000000 /* EMC_TRPAB */
- 0x00000007 /* EMC_TCLKSTABLE */
- 0x00000008 /* EMC_TCLKSTOP */
- 0x00001860 /* EMC_TREFBW */
- 0x0000000b /* EMC_QUSE_EXTRA */
- 0x00000006 /* EMC_FBIO_CFG6 */
- 0x00000000 /* EMC_ODT_WRITE */
- 0x00000000 /* EMC_ODT_READ */
- 0x00005088 /* EMC_FBIO_CFG5 */
- 0xf0070191 /* EMC_CFG_DIG_DLL */
- 0x00008000 /* EMC_CFG_DIG_DLL_PERIOD */
- 0x0000800a /* EMC_DLL_XFORM_DQS0 */
- 0x0000000a /* EMC_DLL_XFORM_DQS1 */
- 0x0000000a /* EMC_DLL_XFORM_DQS2 */
- 0x0000000a /* EMC_DLL_XFORM_DQS3 */
- 0x0000000a /* EMC_DLL_XFORM_DQS4 */
- 0x0000000a /* EMC_DLL_XFORM_DQS5 */
- 0x0000000a /* EMC_DLL_XFORM_DQS6 */
- 0x0000000a /* EMC_DLL_XFORM_DQS7 */
- 0x00018000 /* EMC_DLL_XFORM_QUSE0 */
- 0x00018000 /* EMC_DLL_XFORM_QUSE1 */
- 0x00018000 /* EMC_DLL_XFORM_QUSE2 */
- 0x00018000 /* EMC_DLL_XFORM_QUSE3 */
- 0x00018000 /* EMC_DLL_XFORM_QUSE4 */
- 0x00018000 /* EMC_DLL_XFORM_QUSE5 */
- 0x00018000 /* EMC_DLL_XFORM_QUSE6 */
- 0x00018000 /* EMC_DLL_XFORM_QUSE7 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS0 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS1 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS2 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS3 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS4 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS5 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS6 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS7 */
- 0x0000000a /* EMC_DLL_XFORM_DQ0 */
- 0x0000000a /* EMC_DLL_XFORM_DQ1 */
- 0x0000000a /* EMC_DLL_XFORM_DQ2 */
- 0x0000000a /* EMC_DLL_XFORM_DQ3 */
- 0x000002a0 /* EMC_XM2CMDPADCTRL */
- 0x0600013d /* EMC_XM2DQSPADCTRL2 */
- 0x22220000 /* EMC_XM2DQPADCTRL2 */
- 0x77fff884 /* EMC_XM2CLKPADCTRL */
- 0x01f1f501 /* EMC_XM2COMPPADCTRL */
- 0x07077404 /* EMC_XM2VTTGENPADCTRL */
- 0x54000000 /* EMC_XM2VTTGENPADCTRL2 */
- 0x080001e8 /* EMC_XM2QUSEPADCTRL */
- 0x08000021 /* EMC_XM2DQSPADCTRL3 */
- 0x00000802 /* EMC_CTT_TERM_CTRL */
- 0x00020000 /* EMC_ZCAL_INTERVAL */
- 0x00000100 /* EMC_ZCAL_WAIT_CNT */
- 0x00f0000c /* EMC_MRS_WAIT_CNT */
- 0xa0f10000 /* EMC_AUTO_CAL_CONFIG */
- 0x00000000 /* EMC_CTT */
- 0x00000000 /* EMC_CTT_DURATION */
- 0x8000308c /* EMC_DYN_SELF_REF_CONTROL */
- 0xe8000000 /* EMC_FBIO_SPARE */
- 0xff00ff49 /* EMC_CFG_RSV */
- >;
- };
- };
- emc-timings-2 {
- nvidia,ram-code = <2>; /* Hynix A RAM */
- timing-25500000 {
- clock-frequency = <25500000>;
- nvidia,emc-auto-cal-interval = <0x001fffff>;
- nvidia,emc-mode-1 = <0x80100003>;
- nvidia,emc-mode-2 = <0x80200008>;
- nvidia,emc-mode-reset = <0x80001221>;
- nvidia,emc-zcal-cnt-long = <0x00000040>;
- nvidia,emc-cfg-periodic-qrst;
- nvidia,emc-cfg-dyn-self-ref;
- nvidia,emc-configuration = <
- 0x00000001 /* EMC_RC */
- 0x00000007 /* EMC_RFC */
- 0x00000000 /* EMC_RAS */
- 0x00000000 /* EMC_RP */
- 0x00000002 /* EMC_R2W */
- 0x0000000a /* EMC_W2R */
- 0x00000005 /* EMC_R2P */
- 0x0000000b /* EMC_W2P */
- 0x00000000 /* EMC_RD_RCD */
- 0x00000000 /* EMC_WR_RCD */
- 0x00000003 /* EMC_RRD */
- 0x00000001 /* EMC_REXT */
- 0x00000000 /* EMC_WEXT */
- 0x00000005 /* EMC_WDV */
- 0x00000005 /* EMC_QUSE */
- 0x00000004 /* EMC_QRST */
- 0x0000000a /* EMC_QSAFE */
- 0x0000000b /* EMC_RDV */
- 0x000000c0 /* EMC_REFRESH */
- 0x00000000 /* EMC_BURST_REFRESH_NUM */
- 0x00000030 /* EMC_PRE_REFRESH_REQ_CNT */
- 0x00000002 /* EMC_PDEX2WR */
- 0x00000002 /* EMC_PDEX2RD */
- 0x00000001 /* EMC_PCHG2PDEN */
- 0x00000000 /* EMC_ACT2PDEN */
- 0x00000007 /* EMC_AR2PDEN */
- 0x0000000f /* EMC_RW2PDEN */
- 0x00000008 /* EMC_TXSR */
- 0x00000008 /* EMC_TXSRDLL */
- 0x00000004 /* EMC_TCKE */
- 0x00000002 /* EMC_TFAW */
- 0x00000000 /* EMC_TRPAB */
- 0x00000004 /* EMC_TCLKSTABLE */
- 0x00000005 /* EMC_TCLKSTOP */
- 0x000000c7 /* EMC_TREFBW */
- 0x00000006 /* EMC_QUSE_EXTRA */
- 0x00000004 /* EMC_FBIO_CFG6 */
- 0x00000000 /* EMC_ODT_WRITE */
- 0x00000000 /* EMC_ODT_READ */
- 0x00004288 /* EMC_FBIO_CFG5 */
- 0x007800a4 /* EMC_CFG_DIG_DLL */
- 0x00008000 /* EMC_CFG_DIG_DLL_PERIOD */
- 0x000fc000 /* EMC_DLL_XFORM_DQS0 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS1 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS2 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS3 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS4 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS5 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS6 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS7 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE0 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE1 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE2 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE3 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE4 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE5 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE6 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE7 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS0 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS1 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS2 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS3 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS4 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS5 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS6 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS7 */
- 0x000fc000 /* EMC_DLL_XFORM_DQ0 */
- 0x000fc000 /* EMC_DLL_XFORM_DQ1 */
- 0x000fc000 /* EMC_DLL_XFORM_DQ2 */
- 0x000fc000 /* EMC_DLL_XFORM_DQ3 */
- 0x000002a0 /* EMC_XM2CMDPADCTRL */
- 0x0800211c /* EMC_XM2DQSPADCTRL2 */
- 0x00000000 /* EMC_XM2DQPADCTRL2 */
- 0x77fff884 /* EMC_XM2CLKPADCTRL */
- 0x01f1f108 /* EMC_XM2COMPPADCTRL */
- 0x05057404 /* EMC_XM2VTTGENPADCTRL */
- 0x54000007 /* EMC_XM2VTTGENPADCTRL2 */
- 0x08000168 /* EMC_XM2QUSEPADCTRL */
- 0x08000000 /* EMC_XM2DQSPADCTRL3 */
- 0x00000802 /* EMC_CTT_TERM_CTRL */
- 0x00000000 /* EMC_ZCAL_INTERVAL */
- 0x00000040 /* EMC_ZCAL_WAIT_CNT */
- 0x000c000c /* EMC_MRS_WAIT_CNT */
- 0xa0f10000 /* EMC_AUTO_CAL_CONFIG */
- 0x00000000 /* EMC_CTT */
- 0x00000000 /* EMC_CTT_DURATION */
- 0x80000287 /* EMC_DYN_SELF_REF_CONTROL */
- 0xe8000000 /* EMC_FBIO_SPARE */
- 0xff00ff00 /* EMC_CFG_RSV */
- >;
- };
- timing-51000000 {
- clock-frequency = <51000000>;
- nvidia,emc-auto-cal-interval = <0x001fffff>;
- nvidia,emc-mode-1 = <0x80100003>;
- nvidia,emc-mode-2 = <0x80200008>;
- nvidia,emc-mode-reset = <0x80001221>;
- nvidia,emc-zcal-cnt-long = <0x00000040>;
- nvidia,emc-cfg-periodic-qrst;
- nvidia,emc-cfg-dyn-self-ref;
- nvidia,emc-configuration = <
- 0x00000002 /* EMC_RC */
- 0x0000000f /* EMC_RFC */
- 0x00000001 /* EMC_RAS */
- 0x00000000 /* EMC_RP */
- 0x00000002 /* EMC_R2W */
- 0x0000000a /* EMC_W2R */
- 0x00000005 /* EMC_R2P */
- 0x0000000b /* EMC_W2P */
- 0x00000000 /* EMC_RD_RCD */
- 0x00000000 /* EMC_WR_RCD */
- 0x00000003 /* EMC_RRD */
- 0x00000001 /* EMC_REXT */
- 0x00000000 /* EMC_WEXT */
- 0x00000005 /* EMC_WDV */
- 0x00000005 /* EMC_QUSE */
- 0x00000004 /* EMC_QRST */
- 0x0000000a /* EMC_QSAFE */
- 0x0000000b /* EMC_RDV */
- 0x00000181 /* EMC_REFRESH */
- 0x00000000 /* EMC_BURST_REFRESH_NUM */
- 0x00000060 /* EMC_PRE_REFRESH_REQ_CNT */
- 0x00000002 /* EMC_PDEX2WR */
- 0x00000002 /* EMC_PDEX2RD */
- 0x00000001 /* EMC_PCHG2PDEN */
- 0x00000000 /* EMC_ACT2PDEN */
- 0x00000007 /* EMC_AR2PDEN */
- 0x0000000f /* EMC_RW2PDEN */
- 0x00000010 /* EMC_TXSR */
- 0x00000010 /* EMC_TXSRDLL */
- 0x00000004 /* EMC_TCKE */
- 0x00000003 /* EMC_TFAW */
- 0x00000000 /* EMC_TRPAB */
- 0x00000004 /* EMC_TCLKSTABLE */
- 0x00000005 /* EMC_TCLKSTOP */
- 0x0000018e /* EMC_TREFBW */
- 0x00000006 /* EMC_QUSE_EXTRA */
- 0x00000004 /* EMC_FBIO_CFG6 */
- 0x00000000 /* EMC_ODT_WRITE */
- 0x00000000 /* EMC_ODT_READ */
- 0x00004288 /* EMC_FBIO_CFG5 */
- 0x007800a4 /* EMC_CFG_DIG_DLL */
- 0x00008000 /* EMC_CFG_DIG_DLL_PERIOD */
- 0x000fc000 /* EMC_DLL_XFORM_DQS0 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS1 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS2 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS3 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS4 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS5 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS6 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS7 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE0 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE1 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE2 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE3 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE4 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE5 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE6 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE7 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS0 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS1 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS2 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS3 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS4 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS5 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS6 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS7 */
- 0x000fc000 /* EMC_DLL_XFORM_DQ0 */
- 0x000fc000 /* EMC_DLL_XFORM_DQ1 */
- 0x000fc000 /* EMC_DLL_XFORM_DQ2 */
- 0x000fc000 /* EMC_DLL_XFORM_DQ3 */
- 0x000002a0 /* EMC_XM2CMDPADCTRL */
- 0x0800211c /* EMC_XM2DQSPADCTRL2 */
- 0x00000000 /* EMC_XM2DQPADCTRL2 */
- 0x77fff884 /* EMC_XM2CLKPADCTRL */
- 0x01f1f108 /* EMC_XM2COMPPADCTRL */
- 0x05057404 /* EMC_XM2VTTGENPADCTRL */
- 0x54000007 /* EMC_XM2VTTGENPADCTRL2 */
- 0x08000168 /* EMC_XM2QUSEPADCTRL */
- 0x08000000 /* EMC_XM2DQSPADCTRL3 */
- 0x00000802 /* EMC_CTT_TERM_CTRL */
- 0x00000000 /* EMC_ZCAL_INTERVAL */
- 0x00000040 /* EMC_ZCAL_WAIT_CNT */
- 0x000c000c /* EMC_MRS_WAIT_CNT */
- 0xa0f10000 /* EMC_AUTO_CAL_CONFIG */
- 0x00000000 /* EMC_CTT */
- 0x00000000 /* EMC_CTT_DURATION */
- 0x8000040b /* EMC_DYN_SELF_REF_CONTROL */
- 0xe8000000 /* EMC_FBIO_SPARE */
- 0xff00ff00 /* EMC_CFG_RSV */
- >;
- };
- timing-102000000 {
- clock-frequency = <102000000>;
- nvidia,emc-auto-cal-interval = <0x001fffff>;
- nvidia,emc-mode-1 = <0x80100003>;
- nvidia,emc-mode-2 = <0x80200008>;
- nvidia,emc-mode-reset = <0x80001221>;
- nvidia,emc-zcal-cnt-long = <0x00000040>;
- nvidia,emc-cfg-periodic-qrst;
- nvidia,emc-cfg-dyn-self-ref;
- nvidia,emc-configuration = <
- 0x00000004 /* EMC_RC */
- 0x0000001e /* EMC_RFC */
- 0x00000003 /* EMC_RAS */
- 0x00000001 /* EMC_RP */
- 0x00000002 /* EMC_R2W */
- 0x0000000a /* EMC_W2R */
- 0x00000005 /* EMC_R2P */
- 0x0000000b /* EMC_W2P */
- 0x00000001 /* EMC_RD_RCD */
- 0x00000001 /* EMC_WR_RCD */
- 0x00000003 /* EMC_RRD */
- 0x00000001 /* EMC_REXT */
- 0x00000000 /* EMC_WEXT */
- 0x00000005 /* EMC_WDV */
- 0x00000005 /* EMC_QUSE */
- 0x00000004 /* EMC_QRST */
- 0x0000000a /* EMC_QSAFE */
- 0x0000000b /* EMC_RDV */
- 0x00000303 /* EMC_REFRESH */
- 0x00000000 /* EMC_BURST_REFRESH_NUM */
- 0x000000c0 /* EMC_PRE_REFRESH_REQ_CNT */
- 0x00000002 /* EMC_PDEX2WR */
- 0x00000002 /* EMC_PDEX2RD */
- 0x00000001 /* EMC_PCHG2PDEN */
- 0x00000000 /* EMC_ACT2PDEN */
- 0x00000007 /* EMC_AR2PDEN */
- 0x0000000f /* EMC_RW2PDEN */
- 0x00000020 /* EMC_TXSR */
- 0x00000020 /* EMC_TXSRDLL */
- 0x00000004 /* EMC_TCKE */
- 0x00000005 /* EMC_TFAW */
- 0x00000000 /* EMC_TRPAB */
- 0x00000004 /* EMC_TCLKSTABLE */
- 0x00000005 /* EMC_TCLKSTOP */
- 0x0000031c /* EMC_TREFBW */
- 0x00000006 /* EMC_QUSE_EXTRA */
- 0x00000004 /* EMC_FBIO_CFG6 */
- 0x00000000 /* EMC_ODT_WRITE */
- 0x00000000 /* EMC_ODT_READ */
- 0x00004288 /* EMC_FBIO_CFG5 */
- 0x007800a4 /* EMC_CFG_DIG_DLL */
- 0x00008000 /* EMC_CFG_DIG_DLL_PERIOD */
- 0x000fc000 /* EMC_DLL_XFORM_DQS0 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS1 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS2 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS3 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS4 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS5 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS6 */
- 0x000fc000 /* EMC_DLL_XFORM_DQS7 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE0 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE1 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE2 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE3 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE4 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE5 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE6 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE7 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS0 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS1 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS2 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS3 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS4 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS5 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS6 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS7 */
- 0x000fc000 /* EMC_DLL_XFORM_DQ0 */
- 0x000fc000 /* EMC_DLL_XFORM_DQ1 */
- 0x000fc000 /* EMC_DLL_XFORM_DQ2 */
- 0x000fc000 /* EMC_DLL_XFORM_DQ3 */
- 0x000002a0 /* EMC_XM2CMDPADCTRL */
- 0x0800211c /* EMC_XM2DQSPADCTRL2 */
- 0x00000000 /* EMC_XM2DQPADCTRL2 */
- 0x77fff884 /* EMC_XM2CLKPADCTRL */
- 0x01f1f108 /* EMC_XM2COMPPADCTRL */
- 0x05057404 /* EMC_XM2VTTGENPADCTRL */
- 0x54000007 /* EMC_XM2VTTGENPADCTRL2 */
- 0x08000168 /* EMC_XM2QUSEPADCTRL */
- 0x08000000 /* EMC_XM2DQSPADCTRL3 */
- 0x00000802 /* EMC_CTT_TERM_CTRL */
- 0x00000000 /* EMC_ZCAL_INTERVAL */
- 0x00000040 /* EMC_ZCAL_WAIT_CNT */
- 0x000c000c /* EMC_MRS_WAIT_CNT */
- 0xa0f10000 /* EMC_AUTO_CAL_CONFIG */
- 0x00000000 /* EMC_CTT */
- 0x00000000 /* EMC_CTT_DURATION */
- 0x80000713 /* EMC_DYN_SELF_REF_CONTROL */
- 0xe8000000 /* EMC_FBIO_SPARE */
- 0xff00ff00 /* EMC_CFG_RSV */
- >;
- };
- timing-204000000 {
- clock-frequency = <204000000>;
- nvidia,emc-auto-cal-interval = <0x001fffff>;
- nvidia,emc-mode-1 = <0x80100003>;
- nvidia,emc-mode-2 = <0x80200008>;
- nvidia,emc-mode-reset = <0x80001221>;
- nvidia,emc-zcal-cnt-long = <0x00000040>;
- nvidia,emc-cfg-periodic-qrst;
- nvidia,emc-cfg-dyn-self-ref;
- nvidia,emc-configuration = <
- 0x00000009 /* EMC_RC */
- 0x0000003d /* EMC_RFC */
- 0x00000007 /* EMC_RAS */
- 0x00000002 /* EMC_RP */
- 0x00000002 /* EMC_R2W */
- 0x0000000a /* EMC_W2R */
- 0x00000005 /* EMC_R2P */
- 0x0000000b /* EMC_W2P */
- 0x00000002 /* EMC_RD_RCD */
- 0x00000002 /* EMC_WR_RCD */
- 0x00000003 /* EMC_RRD */
- 0x00000001 /* EMC_REXT */
- 0x00000000 /* EMC_WEXT */
- 0x00000005 /* EMC_WDV */
- 0x00000005 /* EMC_QUSE */
- 0x00000004 /* EMC_QRST */
- 0x0000000a /* EMC_QSAFE */
- 0x0000000b /* EMC_RDV */
- 0x00000607 /* EMC_REFRESH */
- 0x00000000 /* EMC_BURST_REFRESH_NUM */
- 0x00000181 /* EMC_PRE_REFRESH_REQ_CNT */
- 0x00000002 /* EMC_PDEX2WR */
- 0x00000002 /* EMC_PDEX2RD */
- 0x00000001 /* EMC_PCHG2PDEN */
- 0x00000000 /* EMC_ACT2PDEN */
- 0x00000007 /* EMC_AR2PDEN */
- 0x0000000f /* EMC_RW2PDEN */
- 0x00000040 /* EMC_TXSR */
- 0x00000040 /* EMC_TXSRDLL */
- 0x00000004 /* EMC_TCKE */
- 0x00000009 /* EMC_TFAW */
- 0x00000000 /* EMC_TRPAB */
- 0x00000004 /* EMC_TCLKSTABLE */
- 0x00000005 /* EMC_TCLKSTOP */
- 0x00000638 /* EMC_TREFBW */
- 0x00000006 /* EMC_QUSE_EXTRA */
- 0x00000006 /* EMC_FBIO_CFG6 */
- 0x00000000 /* EMC_ODT_WRITE */
- 0x00000000 /* EMC_ODT_READ */
- 0x00004288 /* EMC_FBIO_CFG5 */
- 0x004400a4 /* EMC_CFG_DIG_DLL */
- 0x00008000 /* EMC_CFG_DIG_DLL_PERIOD */
- 0x00080000 /* EMC_DLL_XFORM_DQS0 */
- 0x00080000 /* EMC_DLL_XFORM_DQS1 */
- 0x00080000 /* EMC_DLL_XFORM_DQS2 */
- 0x00080000 /* EMC_DLL_XFORM_DQS3 */
- 0x00080000 /* EMC_DLL_XFORM_DQS4 */
- 0x00080000 /* EMC_DLL_XFORM_DQS5 */
- 0x00080000 /* EMC_DLL_XFORM_DQS6 */
- 0x00080000 /* EMC_DLL_XFORM_DQS7 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE0 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE1 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE2 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE3 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE4 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE5 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE6 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE7 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS0 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS1 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS2 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS3 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS4 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS5 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS6 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS7 */
- 0x00080000 /* EMC_DLL_XFORM_DQ0 */
- 0x00080000 /* EMC_DLL_XFORM_DQ1 */
- 0x00080000 /* EMC_DLL_XFORM_DQ2 */
- 0x00080000 /* EMC_DLL_XFORM_DQ3 */
- 0x000002a0 /* EMC_XM2CMDPADCTRL */
- 0x0800211c /* EMC_XM2DQSPADCTRL2 */
- 0x00000000 /* EMC_XM2DQPADCTRL2 */
- 0x77fff884 /* EMC_XM2CLKPADCTRL */
- 0x01f1f108 /* EMC_XM2COMPPADCTRL */
- 0x05057404 /* EMC_XM2VTTGENPADCTRL */
- 0x54000007 /* EMC_XM2VTTGENPADCTRL2 */
- 0x08000168 /* EMC_XM2QUSEPADCTRL */
- 0x08000000 /* EMC_XM2DQSPADCTRL3 */
- 0x00000802 /* EMC_CTT_TERM_CTRL */
- 0x00020000 /* EMC_ZCAL_INTERVAL */
- 0x00000100 /* EMC_ZCAL_WAIT_CNT */
- 0x000c000c /* EMC_MRS_WAIT_CNT */
- 0xa0f10000 /* EMC_AUTO_CAL_CONFIG */
- 0x00000000 /* EMC_CTT */
- 0x00000000 /* EMC_CTT_DURATION */
- 0x80000d22 /* EMC_DYN_SELF_REF_CONTROL */
- 0xe8000000 /* EMC_FBIO_SPARE */
- 0xff00ff00 /* EMC_CFG_RSV */
- >;
- };
- timing-400000000 {
- clock-frequency = <400000000>;
- nvidia,emc-auto-cal-interval = <0x001fffff>;
- nvidia,emc-mode-1 = <0x80100002>;
- nvidia,emc-mode-2 = <0x80200000>;
- nvidia,emc-mode-reset = <0x80000521>;
- nvidia,emc-zcal-cnt-long = <0x00000040>;
- nvidia,emc-configuration = <
- 0x00000012 /* EMC_RC */
- 0x00000076 /* EMC_RFC */
- 0x0000000c /* EMC_RAS */
- 0x00000004 /* EMC_RP */
- 0x00000003 /* EMC_R2W */
- 0x00000008 /* EMC_W2R */
- 0x00000002 /* EMC_R2P */
- 0x0000000a /* EMC_W2P */
- 0x00000004 /* EMC_RD_RCD */
- 0x00000004 /* EMC_WR_RCD */
- 0x00000002 /* EMC_RRD */
- 0x00000001 /* EMC_REXT */
- 0x00000000 /* EMC_WEXT */
- 0x00000004 /* EMC_WDV */
- 0x00000006 /* EMC_QUSE */
- 0x00000004 /* EMC_QRST */
- 0x0000000a /* EMC_QSAFE */
- 0x0000000c /* EMC_RDV */
- 0x00000bf0 /* EMC_REFRESH */
- 0x00000000 /* EMC_BURST_REFRESH_NUM */
- 0x000002fc /* EMC_PRE_REFRESH_REQ_CNT */
- 0x00000001 /* EMC_PDEX2WR */
- 0x00000008 /* EMC_PDEX2RD */
- 0x00000001 /* EMC_PCHG2PDEN */
- 0x00000000 /* EMC_ACT2PDEN */
- 0x00000008 /* EMC_AR2PDEN */
- 0x0000000f /* EMC_RW2PDEN */
- 0x0000007c /* EMC_TXSR */
- 0x00000200 /* EMC_TXSRDLL */
- 0x00000004 /* EMC_TCKE */
- 0x00000010 /* EMC_TFAW */
- 0x00000000 /* EMC_TRPAB */
- 0x00000004 /* EMC_TCLKSTABLE */
- 0x00000005 /* EMC_TCLKSTOP */
- 0x00000c30 /* EMC_TREFBW */
- 0x00000000 /* EMC_QUSE_EXTRA */
- 0x00000004 /* EMC_FBIO_CFG6 */
- 0x00000000 /* EMC_ODT_WRITE */
- 0x00000000 /* EMC_ODT_READ */
- 0x00007088 /* EMC_FBIO_CFG5 */
- 0x001d0084 /* EMC_CFG_DIG_DLL */
- 0x00008000 /* EMC_CFG_DIG_DLL_PERIOD */
- 0x00044000 /* EMC_DLL_XFORM_DQS0 */
- 0x00044000 /* EMC_DLL_XFORM_DQS1 */
- 0x00044000 /* EMC_DLL_XFORM_DQS2 */
- 0x00044000 /* EMC_DLL_XFORM_DQS3 */
- 0x00044000 /* EMC_DLL_XFORM_DQS4 */
- 0x00044000 /* EMC_DLL_XFORM_DQS5 */
- 0x00044000 /* EMC_DLL_XFORM_DQS6 */
- 0x00044000 /* EMC_DLL_XFORM_DQS7 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE0 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE1 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE2 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE3 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE4 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE5 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE6 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE7 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS0 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS1 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS2 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS3 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS4 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS5 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS6 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS7 */
- 0x00058000 /* EMC_DLL_XFORM_DQ0 */
- 0x00058000 /* EMC_DLL_XFORM_DQ1 */
- 0x00058000 /* EMC_DLL_XFORM_DQ2 */
- 0x00058000 /* EMC_DLL_XFORM_DQ3 */
- 0x000002a0 /* EMC_XM2CMDPADCTRL */
- 0x0800013d /* EMC_XM2DQSPADCTRL2 */
- 0x00000000 /* EMC_XM2DQPADCTRL2 */
- 0x77fff884 /* EMC_XM2CLKPADCTRL */
- 0x01f1f508 /* EMC_XM2COMPPADCTRL */
- 0x05057404 /* EMC_XM2VTTGENPADCTRL */
- 0x54000007 /* EMC_XM2VTTGENPADCTRL2 */
- 0x080001e8 /* EMC_XM2QUSEPADCTRL */
- 0x08000021 /* EMC_XM2DQSPADCTRL3 */
- 0x00000802 /* EMC_CTT_TERM_CTRL */
- 0x00020000 /* EMC_ZCAL_INTERVAL */
- 0x00000100 /* EMC_ZCAL_WAIT_CNT */
- 0x0148000c /* EMC_MRS_WAIT_CNT */
- 0xa0f10000 /* EMC_AUTO_CAL_CONFIG */
- 0x00000000 /* EMC_CTT */
- 0x00000000 /* EMC_CTT_DURATION */
- 0x800018c8 /* EMC_DYN_SELF_REF_CONTROL */
- 0xe8000000 /* EMC_FBIO_SPARE */
- 0xff00ff89 /* EMC_CFG_RSV */
- >;
- };
- timing-800000000 {
- clock-frequency = <800000000>;
- nvidia,emc-auto-cal-interval = <0x001fffff>;
- nvidia,emc-mode-1 = <0x80100002>;
- nvidia,emc-mode-2 = <0x80200018>;
- nvidia,emc-mode-reset = <0x80000d71>;
- nvidia,emc-zcal-cnt-long = <0x00000040>;
- nvidia,emc-cfg-periodic-qrst;
- nvidia,emc-configuration = <
- 0x00000025 /* EMC_RC */
- 0x000000ee /* EMC_RFC */
- 0x0000001a /* EMC_RAS */
- 0x00000009 /* EMC_RP */
- 0x00000005 /* EMC_R2W */
- 0x0000000d /* EMC_W2R */
- 0x00000004 /* EMC_R2P */
- 0x00000013 /* EMC_W2P */
- 0x00000009 /* EMC_RD_RCD */
- 0x00000009 /* EMC_WR_RCD */
- 0x00000003 /* EMC_RRD */
- 0x00000001 /* EMC_REXT */
- 0x00000000 /* EMC_WEXT */
- 0x00000007 /* EMC_WDV */
- 0x0000000a /* EMC_QUSE */
- 0x00000009 /* EMC_QRST */
- 0x0000000b /* EMC_QSAFE */
- 0x00000011 /* EMC_RDV */
- 0x00001820 /* EMC_REFRESH */
- 0x00000000 /* EMC_BURST_REFRESH_NUM */
- 0x00000608 /* EMC_PRE_REFRESH_REQ_CNT */
- 0x00000003 /* EMC_PDEX2WR */
- 0x00000012 /* EMC_PDEX2RD */
- 0x00000001 /* EMC_PCHG2PDEN */
- 0x00000000 /* EMC_ACT2PDEN */
- 0x0000000f /* EMC_AR2PDEN */
- 0x00000018 /* EMC_RW2PDEN */
- 0x000000f8 /* EMC_TXSR */
- 0x00000200 /* EMC_TXSRDLL */
- 0x00000005 /* EMC_TCKE */
- 0x00000020 /* EMC_TFAW */
- 0x00000000 /* EMC_TRPAB */
- 0x00000007 /* EMC_TCLKSTABLE */
- 0x00000008 /* EMC_TCLKSTOP */
- 0x00001860 /* EMC_TREFBW */
- 0x0000000b /* EMC_QUSE_EXTRA */
- 0x00000006 /* EMC_FBIO_CFG6 */
- 0x00000000 /* EMC_ODT_WRITE */
- 0x00000000 /* EMC_ODT_READ */
- 0x00005088 /* EMC_FBIO_CFG5 */
- 0xf0070191 /* EMC_CFG_DIG_DLL */
- 0x00008000 /* EMC_CFG_DIG_DLL_PERIOD */
- 0x0000000c /* EMC_DLL_XFORM_DQS0 */
- 0x007fc00a /* EMC_DLL_XFORM_DQS1 */
- 0x00000008 /* EMC_DLL_XFORM_DQS2 */
- 0x0000000a /* EMC_DLL_XFORM_DQS3 */
- 0x0000000a /* EMC_DLL_XFORM_DQS4 */
- 0x0000000a /* EMC_DLL_XFORM_DQS5 */
- 0x0000000a /* EMC_DLL_XFORM_DQS6 */
- 0x0000000a /* EMC_DLL_XFORM_DQS7 */
- 0x00018000 /* EMC_DLL_XFORM_QUSE0 */
- 0x00018000 /* EMC_DLL_XFORM_QUSE1 */
- 0x00018000 /* EMC_DLL_XFORM_QUSE2 */
- 0x00018000 /* EMC_DLL_XFORM_QUSE3 */
- 0x00018000 /* EMC_DLL_XFORM_QUSE4 */
- 0x00018000 /* EMC_DLL_XFORM_QUSE5 */
- 0x00018000 /* EMC_DLL_XFORM_QUSE6 */
- 0x00018000 /* EMC_DLL_XFORM_QUSE7 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS0 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS1 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS2 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS3 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS4 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS5 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS6 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS7 */
- 0x0000000a /* EMC_DLL_XFORM_DQ0 */
- 0x0000000c /* EMC_DLL_XFORM_DQ1 */
- 0x0000000a /* EMC_DLL_XFORM_DQ2 */
- 0x0000000a /* EMC_DLL_XFORM_DQ3 */
- 0x000002a0 /* EMC_XM2CMDPADCTRL */
- 0x0600013d /* EMC_XM2DQSPADCTRL2 */
- 0x22220000 /* EMC_XM2DQPADCTRL2 */
- 0x77fff884 /* EMC_XM2CLKPADCTRL */
- 0x01f1f501 /* EMC_XM2COMPPADCTRL */
- 0x07077404 /* EMC_XM2VTTGENPADCTRL */
- 0x54000000 /* EMC_XM2VTTGENPADCTRL2 */
- 0x080001e8 /* EMC_XM2QUSEPADCTRL */
- 0x0a000021 /* EMC_XM2DQSPADCTRL3 */
- 0x00000802 /* EMC_CTT_TERM_CTRL */
- 0x00020000 /* EMC_ZCAL_INTERVAL */
- 0x00000100 /* EMC_ZCAL_WAIT_CNT */
- 0x00d0000c /* EMC_MRS_WAIT_CNT */
- 0xa0f10000 /* EMC_AUTO_CAL_CONFIG */
- 0x00000000 /* EMC_CTT */
- 0x00000000 /* EMC_CTT_DURATION */
- 0x8000308c /* EMC_DYN_SELF_REF_CONTROL */
- 0xe8000000 /* EMC_FBIO_SPARE */
- 0xff00ff49 /* EMC_CFG_RSV */
- >;
- };
- };
-};
-&state_default {
- clk_32k_out_pa0 {
- nvidia,pins = "clk_32k_out_pa0";
- nvidia,function = "blink";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- uart3_cts_n_pa1 {
- nvidia,pins = "uart3_cts_n_pa1";
- nvidia,function = "uartc";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- dap2_fs_pa2 {
- nvidia,pins = "dap2_fs_pa2";
- nvidia,function = "i2s1";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- dap2_sclk_pa3 {
- nvidia,pins = "dap2_sclk_pa3";
- nvidia,function = "i2s1";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- dap2_din_pa4 {
- nvidia,pins = "dap2_din_pa4";
- nvidia,function = "i2s1";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- dap2_dout_pa5 {
- nvidia,pins = "dap2_dout_pa5";
- nvidia,function = "i2s1";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- sdmmc3_clk_pa6 {
- nvidia,pins = "sdmmc3_clk_pa6";
- nvidia,function = "sdmmc3";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- sdmmc3_cmd_pa7 {
- nvidia,pins = "sdmmc3_cmd_pa7";
- nvidia,function = "sdmmc3";
- nvidia,pull = <TEGRA_PIN_PULL_UP>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- gmi_a17_pb0 {
- nvidia,pins = "gmi_a17_pb0";
- nvidia,function = "spi4";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- gmi_a18_pb1 {
- nvidia,pins = "gmi_a18_pb1";
- nvidia,function = "spi4";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- lcd_pwr0_pb2 {
- nvidia,pins = "lcd_pwr0_pb2";
- nvidia,function = "displaya";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- lcd_pclk_pb3 {
- nvidia,pins = "lcd_pclk_pb3";
- nvidia,function = "displaya";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- sdmmc3_dat3_pb4 {
- nvidia,pins = "sdmmc3_dat3_pb4";
- nvidia,function = "sdmmc3";
- nvidia,pull = <TEGRA_PIN_PULL_UP>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- sdmmc3_dat2_pb5 {
- nvidia,pins = "sdmmc3_dat2_pb5";
- nvidia,function = "sdmmc3";
- nvidia,pull = <TEGRA_PIN_PULL_UP>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- sdmmc3_dat1_pb6 {
- nvidia,pins = "sdmmc3_dat1_pb6";
- nvidia,function = "sdmmc3";
- nvidia,pull = <TEGRA_PIN_PULL_UP>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- sdmmc3_dat0_pb7 {
- nvidia,pins = "sdmmc3_dat0_pb7";
- nvidia,function = "sdmmc3";
- nvidia,pull = <TEGRA_PIN_PULL_UP>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- uart3_rts_n_pc0 {
- nvidia,pins = "uart3_rts_n_pc0";
- nvidia,function = "uartc";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- lcd_pwr1_pc1 {
- nvidia,pins = "lcd_pwr1_pc1";
- nvidia,function = "displaya";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- uart2_txd_pc2 {
- nvidia,pins = "uart2_txd_pc2";
- nvidia,function = "uartb";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- uart2_rxd_pc3 {
- nvidia,pins = "uart2_rxd_pc3";
- nvidia,function = "uartb";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- gen1_i2c_scl_pc4 {
- nvidia,pins = "gen1_i2c_scl_pc4";
- nvidia,function = "i2c1";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- nvidia,open-drain = <TEGRA_PIN_DISABLE>;
- };
- gen1_i2c_sda_pc5 {
- nvidia,pins = "gen1_i2c_sda_pc5";
- nvidia,function = "i2c1";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- nvidia,open-drain = <TEGRA_PIN_DISABLE>;
- };
- lcd_pwr2_pc6 {
- nvidia,pins = "lcd_pwr2_pc6";
- nvidia,function = "displaya";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- gmi_wp_n_pc7 {
- nvidia,pins = "gmi_wp_n_pc7";
- nvidia,function = "gmi";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- sdmmc3_dat5_pd0 {
- nvidia,pins = "sdmmc3_dat5_pd0";
- nvidia,function = "sdmmc3";
- nvidia,pull = <TEGRA_PIN_PULL_UP>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- sdmmc3_dat4_pd1 {
- nvidia,pins = "sdmmc3_dat4_pd1";
- nvidia,function = "sdmmc3";
- nvidia,pull = <TEGRA_PIN_PULL_UP>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- lcd_dc1_pd2 {
- nvidia,pins = "lcd_dc1_pd2";
- nvidia,function = "displaya";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- sdmmc3_dat6_pd3 {
- nvidia,pins = "sdmmc3_dat6_pd3";
- nvidia,function = "spi4";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- sdmmc3_dat7_pd4 {
- nvidia,pins = "sdmmc3_dat7_pd4";
- nvidia,function = "spi4";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- vi_d1_pd5 {
- nvidia,pins = "vi_d1_pd5";
- nvidia,function = "sdmmc2";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- vi_vsync_pd6 {
- nvidia,pins = "vi_vsync_pd6";
- nvidia,function = "ddr";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- vi_hsync_pd7 {
- nvidia,pins = "vi_hsync_pd7";
- nvidia,function = "ddr";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- lcd_d0_pe0 {
- nvidia,pins = "lcd_d0_pe0";
- nvidia,function = "displaya";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- lcd_d1_pe1 {
- nvidia,pins = "lcd_d1_pe1";
- nvidia,function = "displaya";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- lcd_d2_pe2 {
- nvidia,pins = "lcd_d2_pe2";
- nvidia,function = "displaya";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- lcd_d3_pe3 {
- nvidia,pins = "lcd_d3_pe3";
- nvidia,function = "displaya";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- lcd_d4_pe4 {
- nvidia,pins = "lcd_d4_pe4";
- nvidia,function = "displaya";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- lcd_d5_pe5 {
- nvidia,pins = "lcd_d5_pe5";
- nvidia,function = "displaya";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- lcd_d6_pe6 {
- nvidia,pins = "lcd_d6_pe6";
- nvidia,function = "displaya";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- lcd_d7_pe7 {
- nvidia,pins = "lcd_d7_pe7";
- nvidia,function = "displaya";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- lcd_d8_pf0 {
- nvidia,pins = "lcd_d8_pf0";
- nvidia,function = "displaya";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- lcd_d9_pf1 {
- nvidia,pins = "lcd_d9_pf1";
- nvidia,function = "displaya";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- lcd_d10_pf2 {
- nvidia,pins = "lcd_d10_pf2";
- nvidia,function = "displaya";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- lcd_d11_pf3 {
- nvidia,pins = "lcd_d11_pf3";
- nvidia,function = "displaya";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- lcd_d12_pf4 {
- nvidia,pins = "lcd_d12_pf4";
- nvidia,function = "displaya";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- lcd_d13_pf5 {
- nvidia,pins = "lcd_d13_pf5";
- nvidia,function = "displaya";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- lcd_d14_pf6 {
- nvidia,pins = "lcd_d14_pf6";
- nvidia,function = "displaya";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- lcd_d15_pf7 {
- nvidia,pins = "lcd_d15_pf7";
- nvidia,function = "displaya";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- gmi_ad0_pg0 {
- nvidia,pins = "gmi_ad0_pg0";
- nvidia,function = "nand";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- gmi_ad1_pg1 {
- nvidia,pins = "gmi_ad1_pg1";
- nvidia,function = "nand";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- gmi_ad2_pg2 {
- nvidia,pins = "gmi_ad2_pg2";
- nvidia,function = "nand";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- gmi_ad3_pg3 {
- nvidia,pins = "gmi_ad3_pg3";
- nvidia,function = "nand";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- gmi_ad4_pg4 {
- nvidia,pins = "gmi_ad4_pg4";
- nvidia,function = "nand";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- gmi_ad5_pg5 {
- nvidia,pins = "gmi_ad5_pg5";
- nvidia,function = "nand";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- gmi_ad6_pg6 {
- nvidia,pins = "gmi_ad6_pg6";
- nvidia,function = "nand";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- gmi_ad7_pg7 {
- nvidia,pins = "gmi_ad7_pg7";
- nvidia,function = "nand";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- gmi_ad8_ph0 {
- nvidia,pins = "gmi_ad8_ph0";
- nvidia,function = "pwm0";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- gmi_ad9_ph1 {
- nvidia,pins = "gmi_ad9_ph1";
- nvidia,function = "pwm1";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- gmi_ad10_ph2 {
- nvidia,pins = "gmi_ad10_ph2";
- nvidia,function = "pwm2";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- gmi_ad11_ph3 {
- nvidia,pins = "gmi_ad11_ph3";
- nvidia,function = "nand";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- gmi_ad12_ph4 {
- nvidia,pins = "gmi_ad12_ph4";
- nvidia,function = "nand";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- gmi_ad13_ph5 {
- nvidia,pins = "gmi_ad13_ph5";
- nvidia,function = "nand";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- gmi_ad14_ph6 {
- nvidia,pins = "gmi_ad14_ph6";
- nvidia,function = "nand";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- gmi_wr_n_pi0 {
- nvidia,pins = "gmi_wr_n_pi0";
- nvidia,function = "nand";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- gmi_oe_n_pi1 {
- nvidia,pins = "gmi_oe_n_pi1";
- nvidia,function = "nand";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- gmi_dqs_pi2 {
- nvidia,pins = "gmi_dqs_pi2";
- nvidia,function = "nand";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- gmi_iordy_pi5 {
- nvidia,pins = "gmi_iordy_pi5";
- nvidia,function = "rsvd1";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- gmi_cs7_n_pi6 {
- nvidia,pins = "gmi_cs7_n_pi6";
- nvidia,function = "nand";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- gmi_wait_pi7 {
- nvidia,pins = "gmi_wait_pi7";
- nvidia,function = "nand";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- lcd_de_pj1 {
- nvidia,pins = "lcd_de_pj1";
- nvidia,function = "displaya";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- gmi_cs1_n_pj2 {
- nvidia,pins = "gmi_cs1_n_pj2";
- nvidia,function = "rsvd1";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- lcd_hsync_pj3 {
- nvidia,pins = "lcd_hsync_pj3";
- nvidia,function = "displaya";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- lcd_vsync_pj4 {
- nvidia,pins = "lcd_vsync_pj4";
- nvidia,function = "displaya";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- uart2_cts_n_pj5 {
- nvidia,pins = "uart2_cts_n_pj5";
- nvidia,function = "uartb";
- nvidia,pull = <TEGRA_PIN_PULL_DOWN>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- uart2_rts_n_pj6 {
- nvidia,pins = "uart2_rts_n_pj6";
- nvidia,function = "uartb";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- gmi_a16_pj7 {
- nvidia,pins = "gmi_a16_pj7";
- nvidia,function = "spi4";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- gmi_adv_n_pk0 {
- nvidia,pins = "gmi_adv_n_pk0";
- nvidia,function = "nand";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- gmi_clk_pk1 {
- nvidia,pins = "gmi_clk_pk1";
- nvidia,function = "nand";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- gmi_cs2_n_pk3 {
- nvidia,pins = "gmi_cs2_n_pk3";
- nvidia,function = "rsvd1";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- gmi_cs3_n_pk4 {
- nvidia,pins = "gmi_cs3_n_pk4";
- nvidia,function = "nand";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- spdif_out_pk5 {
- nvidia,pins = "spdif_out_pk5";
- nvidia,function = "spdif";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- spdif_in_pk6 {
- nvidia,pins = "spdif_in_pk6";
- nvidia,function = "spdif";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- gmi_a19_pk7 {
- nvidia,pins = "gmi_a19_pk7";
- nvidia,function = "spi4";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- vi_d2_pl0 {
- nvidia,pins = "vi_d2_pl0";
- nvidia,function = "sdmmc2";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- vi_d3_pl1 {
- nvidia,pins = "vi_d3_pl1";
- nvidia,function = "sdmmc2";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- vi_d4_pl2 {
- nvidia,pins = "vi_d4_pl2";
- nvidia,function = "vi";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- vi_d5_pl3 {
- nvidia,pins = "vi_d5_pl3";
- nvidia,function = "sdmmc2";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- vi_d6_pl4 {
- nvidia,pins = "vi_d6_pl4";
- nvidia,function = "vi";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- vi_d7_pl5 {
- nvidia,pins = "vi_d7_pl5";
- nvidia,function = "sdmmc2";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- vi_d8_pl6 {
- nvidia,pins = "vi_d8_pl6";
- nvidia,function = "sdmmc2";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- vi_d9_pl7 {
- nvidia,pins = "vi_d9_pl7";
- nvidia,function = "sdmmc2";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- lcd_d16_pm0 {
- nvidia,pins = "lcd_d16_pm0";
- nvidia,function = "displaya";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- lcd_d17_pm1 {
- nvidia,pins = "lcd_d17_pm1";
- nvidia,function = "displaya";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- lcd_d18_pm2 {
- nvidia,pins = "lcd_d18_pm2";
- nvidia,function = "displaya";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- lcd_d19_pm3 {
- nvidia,pins = "lcd_d19_pm3";
- nvidia,function = "displaya";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- lcd_d20_pm4 {
- nvidia,pins = "lcd_d20_pm4";
- nvidia,function = "displaya";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- lcd_d21_pm5 {
- nvidia,pins = "lcd_d21_pm5";
- nvidia,function = "displaya";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- lcd_d22_pm6 {
- nvidia,pins = "lcd_d22_pm6";
- nvidia,function = "displaya";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- lcd_d23_pm7 {
- nvidia,pins = "lcd_d23_pm7";
- nvidia,function = "displaya";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- dap1_fs_pn0 {
- nvidia,pins = "dap1_fs_pn0";
- nvidia,function = "i2s0";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- dap1_din_pn1 {
- nvidia,pins = "dap1_din_pn1";
- nvidia,function = "i2s0";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- dap1_dout_pn2 {
- nvidia,pins = "dap1_dout_pn2";
- nvidia,function = "i2s0";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- dap1_sclk_pn3 {
- nvidia,pins = "dap1_sclk_pn3";
- nvidia,function = "i2s0";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- lcd_cs0_n_pn4 {
- nvidia,pins = "lcd_cs0_n_pn4";
- nvidia,function = "displaya";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- lcd_sdout_pn5 {
- nvidia,pins = "lcd_sdout_pn5";
- nvidia,function = "displaya";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- lcd_dc0_pn6 {
- nvidia,pins = "lcd_dc0_pn6";
- nvidia,function = "displaya";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- hdmi_int_pn7 {
- nvidia,pins = "hdmi_int_pn7";
- nvidia,function = "hdmi";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- ulpi_data7_po0 {
- nvidia,pins = "ulpi_data7_po0";
- nvidia,function = "uarta";
- nvidia,pull = <TEGRA_PIN_PULL_DOWN>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- ulpi_data0_po1 {
- nvidia,pins = "ulpi_data0_po1";
- nvidia,function = "uarta";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- ulpi_data1_po2 {
- nvidia,pins = "ulpi_data1_po2";
- nvidia,function = "uarta";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- ulpi_data2_po3 {
- nvidia,pins = "ulpi_data2_po3";
- nvidia,function = "uarta";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- ulpi_data3_po4 {
- nvidia,pins = "ulpi_data3_po4";
- nvidia,function = "uarta";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- ulpi_data4_po5 {
- nvidia,pins = "ulpi_data4_po5";
- nvidia,function = "uarta";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- ulpi_data5_po6 {
- nvidia,pins = "ulpi_data5_po6";
- nvidia,function = "uarta";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- ulpi_data6_po7 {
- nvidia,pins = "ulpi_data6_po7";
- nvidia,function = "uarta";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- dap3_fs_pp0 {
- nvidia,pins = "dap3_fs_pp0";
- nvidia,function = "i2s2";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- dap3_din_pp1 {
- nvidia,pins = "dap3_din_pp1";
- nvidia,function = "i2s2";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- dap3_dout_pp2 {
- nvidia,pins = "dap3_dout_pp2";
- nvidia,function = "i2s2";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- dap3_sclk_pp3 {
- nvidia,pins = "dap3_sclk_pp3";
- nvidia,function = "i2s2";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- dap4_fs_pp4 {
- nvidia,pins = "dap4_fs_pp4";
- nvidia,function = "i2s3";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- dap4_din_pp5 {
- nvidia,pins = "dap4_din_pp5";
- nvidia,function = "i2s3";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- dap4_dout_pp6 {
- nvidia,pins = "dap4_dout_pp6";
- nvidia,function = "i2s3";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- dap4_sclk_pp7 {
- nvidia,pins = "dap4_sclk_pp7";
- nvidia,function = "i2s3";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- kb_col0_pq0 {
- nvidia,pins = "kb_col0_pq0";
- nvidia,function = "kbc";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- kb_col1_pq1 {
- nvidia,pins = "kb_col1_pq1";
- nvidia,function = "kbc";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- kb_col2_pq2 {
- nvidia,pins = "kb_col2_pq2";
- nvidia,function = "kbc";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- kb_col3_pq3 {
- nvidia,pins = "kb_col3_pq3";
- nvidia,function = "kbc";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- kb_col4_pq4 {
- nvidia,pins = "kb_col4_pq4";
- nvidia,function = "kbc";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- kb_col5_pq5 {
- nvidia,pins = "kb_col5_pq5";
- nvidia,function = "kbc";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- kb_col6_pq6 {
- nvidia,pins = "kb_col6_pq6";
- nvidia,function = "kbc";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- kb_col7_pq7 {
- nvidia,pins = "kb_col7_pq7";
- nvidia,function = "kbc";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- kb_row0_pr0 {
- nvidia,pins = "kb_row0_pr0";
- nvidia,function = "kbc";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- kb_row1_pr1 {
- nvidia,pins = "kb_row1_pr1";
- nvidia,function = "kbc";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- kb_row2_pr2 {
- nvidia,pins = "kb_row2_pr2";
- nvidia,function = "kbc";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- kb_row3_pr3 {
- nvidia,pins = "kb_row3_pr3";
- nvidia,function = "kbc";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- kb_row4_pr4 {
- nvidia,pins = "kb_row4_pr4";
- nvidia,function = "kbc";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- kb_row5_pr5 {
- nvidia,pins = "kb_row5_pr5";
- nvidia,function = "kbc";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- kb_row6_pr6 {
- nvidia,pins = "kb_row6_pr6";
- nvidia,function = "kbc";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- kb_row7_pr7 {
- nvidia,pins = "kb_row7_pr7";
- nvidia,function = "kbc";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- kb_row8_ps0 {
- nvidia,pins = "kb_row8_ps0";
- nvidia,function = "kbc";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- kb_row9_ps1 {
- nvidia,pins = "kb_row9_ps1";
- nvidia,function = "kbc";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- kb_row10_ps2 {
- nvidia,pins = "kb_row10_ps2";
- nvidia,function = "kbc";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- kb_row11_ps3 {
- nvidia,pins = "kb_row11_ps3";
- nvidia,function = "kbc";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- kb_row12_ps4 {
- nvidia,pins = "kb_row12_ps4";
- nvidia,function = "kbc";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- kb_row13_ps5 {
- nvidia,pins = "kb_row13_ps5";
- nvidia,function = "kbc";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- kb_row14_ps6 {
- nvidia,pins = "kb_row14_ps6";
- nvidia,function = "kbc";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- kb_row15_ps7 {
- nvidia,pins = "kb_row15_ps7";
- nvidia,function = "kbc";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- vi_pclk_pt0 {
- nvidia,pins = "vi_pclk_pt0";
- nvidia,function = "rsvd1";
- nvidia,pull = <TEGRA_PIN_PULL_UP>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- vi_mclk_pt1 {
- nvidia,pins = "vi_mclk_pt1";
- nvidia,function = "vi";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- vi_d10_pt2 {
- nvidia,pins = "vi_d10_pt2";
- nvidia,function = "ddr";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- vi_d11_pt3 {
- nvidia,pins = "vi_d11_pt3";
- nvidia,function = "ddr";
- nvidia,pull = <TEGRA_PIN_PULL_UP>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- vi_d0_pt4 {
- nvidia,pins = "vi_d0_pt4";
- nvidia,function = "ddr";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- gen2_i2c_scl_pt5 {
- nvidia,pins = "gen2_i2c_scl_pt5";
- nvidia,function = "i2c2";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- nvidia,open-drain = <TEGRA_PIN_DISABLE>;
- };
- gen2_i2c_sda_pt6 {
- nvidia,pins = "gen2_i2c_sda_pt6";
- nvidia,function = "i2c2";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- nvidia,open-drain = <TEGRA_PIN_DISABLE>;
- };
- sdmmc4_cmd_pt7 {
- nvidia,pins = "sdmmc4_cmd_pt7";
- nvidia,function = "sdmmc4";
- nvidia,pull = <TEGRA_PIN_PULL_UP>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- nvidia,io-reset = <TEGRA_PIN_DISABLE>;
- };
- pu0 {
- nvidia,pins = "pu0";
- nvidia,function = "owr";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- pu1 {
- nvidia,pins = "pu1";
- nvidia,function = "rsvd1";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- pu2 {
- nvidia,pins = "pu2";
- nvidia,function = "rsvd1";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- pu3 {
- nvidia,pins = "pu3";
- nvidia,function = "pwm0";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- pu4 {
- nvidia,pins = "pu4";
- nvidia,function = "pwm1";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- pu5 {
- nvidia,pins = "pu5";
- nvidia,function = "rsvd4";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- pu6 {
- nvidia,pins = "pu6";
- nvidia,function = "pwm3";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- jtag_rtck_pu7 {
- nvidia,pins = "jtag_rtck_pu7";
- nvidia,function = "rtck";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- pv0 {
- nvidia,pins = "pv0";
- nvidia,function = "rsvd1";
- nvidia,pull = <TEGRA_PIN_PULL_UP>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- pv1 {
- nvidia,pins = "pv1";
- nvidia,function = "rsvd1";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- pv2 {
- nvidia,pins = "pv2";
- nvidia,function = "owr";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- pv3 {
- nvidia,pins = "pv3";
- nvidia,function = "clk_12m_out";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- ddc_scl_pv4 {
- nvidia,pins = "ddc_scl_pv4";
- nvidia,function = "i2c4";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- ddc_sda_pv5 {
- nvidia,pins = "ddc_sda_pv5";
- nvidia,function = "i2c4";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- crt_hsync_pv6 {
- nvidia,pins = "crt_hsync_pv6";
- nvidia,function = "crt";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- crt_vsync_pv7 {
- nvidia,pins = "crt_vsync_pv7";
- nvidia,function = "crt";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- lcd_cs1_n_pw0 {
- nvidia,pins = "lcd_cs1_n_pw0";
- nvidia,function = "displaya";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- lcd_m1_pw1 {
- nvidia,pins = "lcd_m1_pw1";
- nvidia,function = "displaya";
- nvidia,pull = <TEGRA_PIN_PULL_DOWN>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- spi2_cs1_n_pw2 {
- nvidia,pins = "spi2_cs1_n_pw2";
- nvidia,function = "spi2";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- clk1_out_pw4 {
- nvidia,pins = "clk1_out_pw4";
- nvidia,function = "extperiph1";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- clk2_out_pw5 {
- nvidia,pins = "clk2_out_pw5";
- nvidia,function = "extperiph2";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- uart3_txd_pw6 {
- nvidia,pins = "uart3_txd_pw6";
- nvidia,function = "uartc";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- uart3_rxd_pw7 {
- nvidia,pins = "uart3_rxd_pw7";
- nvidia,function = "uartc";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- spi2_sck_px2 {
- nvidia,pins = "spi2_sck_px2";
- nvidia,function = "gmi";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- spi1_mosi_px4 {
- nvidia,pins = "spi1_mosi_px4";
- nvidia,function = "spi1";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- spi1_sck_px5 {
- nvidia,pins = "spi1_sck_px5";
- nvidia,function = "spi1";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- spi1_cs0_n_px6 {
- nvidia,pins = "spi1_cs0_n_px6";
- nvidia,function = "spi1";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- spi1_miso_px7 {
- nvidia,pins = "spi1_miso_px7";
- nvidia,function = "spi1";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- ulpi_clk_py0 {
- nvidia,pins = "ulpi_clk_py0";
- nvidia,function = "uartd";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- ulpi_dir_py1 {
- nvidia,pins = "ulpi_dir_py1";
- nvidia,function = "uartd";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- ulpi_nxt_py2 {
- nvidia,pins = "ulpi_nxt_py2";
- nvidia,function = "uartd";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- ulpi_stp_py3 {
- nvidia,pins = "ulpi_stp_py3";
- nvidia,function = "uartd";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- sdmmc1_dat3_py4 {
- nvidia,pins = "sdmmc1_dat3_py4";
- nvidia,function = "sdmmc1";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- sdmmc1_dat2_py5 {
- nvidia,pins = "sdmmc1_dat2_py5";
- nvidia,function = "sdmmc1";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- sdmmc1_dat1_py6 {
- nvidia,pins = "sdmmc1_dat1_py6";
- nvidia,function = "sdmmc1";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- sdmmc1_dat0_py7 {
- nvidia,pins = "sdmmc1_dat0_py7";
- nvidia,function = "sdmmc1";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- sdmmc1_clk_pz0 {
- nvidia,pins = "sdmmc1_clk_pz0";
- nvidia,function = "sdmmc1";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- sdmmc1_cmd_pz1 {
- nvidia,pins = "sdmmc1_cmd_pz1";
- nvidia,function = "sdmmc1";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- lcd_sdin_pz2 {
- nvidia,pins = "lcd_sdin_pz2";
- nvidia,function = "displaya";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- lcd_wr_n_pz3 {
- nvidia,pins = "lcd_wr_n_pz3";
- nvidia,function = "displaya";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- lcd_sck_pz4 {
- nvidia,pins = "lcd_sck_pz4";
- nvidia,function = "displaya";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- sys_clk_req_pz5 {
- nvidia,pins = "sys_clk_req_pz5";
- nvidia,function = "sysclk";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- pwr_i2c_scl_pz6 {
- nvidia,pins = "pwr_i2c_scl_pz6";
- nvidia,function = "i2cpwr";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- nvidia,open-drain = <TEGRA_PIN_ENABLE>;
- };
- pwr_i2c_sda_pz7 {
- nvidia,pins = "pwr_i2c_sda_pz7";
- nvidia,function = "i2cpwr";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- nvidia,open-drain = <TEGRA_PIN_ENABLE>;
- };
- sdmmc4_dat0_paa0 {
- nvidia,pins = "sdmmc4_dat0_paa0";
- nvidia,function = "sdmmc4";
- nvidia,pull = <TEGRA_PIN_PULL_UP>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- nvidia,io-reset = <TEGRA_PIN_DISABLE>;
- };
- sdmmc4_dat1_paa1 {
- nvidia,pins = "sdmmc4_dat1_paa1";
- nvidia,function = "sdmmc4";
- nvidia,pull = <TEGRA_PIN_PULL_UP>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- nvidia,io-reset = <TEGRA_PIN_DISABLE>;
- };
- sdmmc4_dat2_paa2 {
- nvidia,pins = "sdmmc4_dat2_paa2";
- nvidia,function = "sdmmc4";
- nvidia,pull = <TEGRA_PIN_PULL_UP>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- nvidia,io-reset = <TEGRA_PIN_DISABLE>;
- };
- sdmmc4_dat3_paa3 {
- nvidia,pins = "sdmmc4_dat3_paa3";
- nvidia,function = "sdmmc4";
- nvidia,pull = <TEGRA_PIN_PULL_UP>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- nvidia,io-reset = <TEGRA_PIN_DISABLE>;
- };
- sdmmc4_dat4_paa4 {
- nvidia,pins = "sdmmc4_dat4_paa4";
- nvidia,function = "sdmmc4";
- nvidia,pull = <TEGRA_PIN_PULL_UP>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- nvidia,io-reset = <TEGRA_PIN_DISABLE>;
- };
- sdmmc4_dat5_paa5 {
- nvidia,pins = "sdmmc4_dat5_paa5";
- nvidia,function = "sdmmc4";
- nvidia,pull = <TEGRA_PIN_PULL_UP>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- nvidia,io-reset = <TEGRA_PIN_DISABLE>;
- };
- sdmmc4_dat6_paa6 {
- nvidia,pins = "sdmmc4_dat6_paa6";
- nvidia,function = "sdmmc4";
- nvidia,pull = <TEGRA_PIN_PULL_UP>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- nvidia,io-reset = <TEGRA_PIN_DISABLE>;
- };
- sdmmc4_dat7_paa7 {
- nvidia,pins = "sdmmc4_dat7_paa7";
- nvidia,function = "sdmmc4";
- nvidia,pull = <TEGRA_PIN_PULL_UP>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- nvidia,io-reset = <TEGRA_PIN_DISABLE>;
- };
- pbb0 {
- nvidia,pins = "pbb0";
- nvidia,function = "i2s4";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- cam_i2c_scl_pbb1 {
- nvidia,pins = "cam_i2c_scl_pbb1";
- nvidia,function = "i2c3";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- nvidia,open-drain = <TEGRA_PIN_DISABLE>;
- };
- cam_i2c_sda_pbb2 {
- nvidia,pins = "cam_i2c_sda_pbb2";
- nvidia,function = "i2c3";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- nvidia,open-drain = <TEGRA_PIN_DISABLE>;
- };
- pbb3 {
- nvidia,pins = "pbb3";
- nvidia,function = "vgp3";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- pbb4 {
- nvidia,pins = "pbb4";
- nvidia,function = "vgp4";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- pbb5 {
- nvidia,pins = "pbb5";
- nvidia,function = "vgp5";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- pbb6 {
- nvidia,pins = "pbb6";
- nvidia,function = "vgp6";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- pbb7 {
- nvidia,pins = "pbb7";
- nvidia,function = "i2s4";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- cam_mclk_pcc0 {
- nvidia,pins = "cam_mclk_pcc0";
- nvidia,function = "vi_alt3";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- pcc1 {
- nvidia,pins = "pcc1";
- nvidia,function = "i2s4";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- pcc2 {
- nvidia,pins = "pcc2";
- nvidia,function = "i2s4";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- sdmmc4_rst_n_pcc3 {
- nvidia,pins = "sdmmc4_rst_n_pcc3";
- nvidia,function = "sdmmc4";
- nvidia,pull = <TEGRA_PIN_PULL_DOWN>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- nvidia,io-reset = <TEGRA_PIN_DISABLE>;
- };
- sdmmc4_clk_pcc4 {
- nvidia,pins = "sdmmc4_clk_pcc4";
- nvidia,function = "sdmmc4";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- nvidia,io-reset = <TEGRA_PIN_DISABLE>;
- };
- clk2_req_pcc5 {
- nvidia,pins = "clk2_req_pcc5";
- nvidia,function = "dap";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- pex_l2_rst_n_pcc6 {
- nvidia,pins = "pex_l2_rst_n_pcc6";
- nvidia,function = "pcie";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- pex_l2_clkreq_n_pcc7 {
- nvidia,pins = "pex_l2_clkreq_n_pcc7";
- nvidia,function = "pcie";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- pex_l0_prsnt_n_pdd0 {
- nvidia,pins = "pex_l0_prsnt_n_pdd0";
- nvidia,function = "pcie";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- pex_l0_rst_n_pdd1 {
- nvidia,pins = "pex_l0_rst_n_pdd1";
- nvidia,function = "pcie";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- pex_l0_clkreq_n_pdd2 {
- nvidia,pins = "pex_l0_clkreq_n_pdd2";
- nvidia,function = "pcie";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- pex_wake_n_pdd3 {
- nvidia,pins = "pex_wake_n_pdd3";
- nvidia,function = "pcie";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- pex_l1_prsnt_n_pdd4 {
- nvidia,pins = "pex_l1_prsnt_n_pdd4";
- nvidia,function = "pcie";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- pex_l1_rst_n_pdd5 {
- nvidia,pins = "pex_l1_rst_n_pdd5";
- nvidia,function = "pcie";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- pex_l1_clkreq_n_pdd6 {
- nvidia,pins = "pex_l1_clkreq_n_pdd6";
- nvidia,function = "pcie";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- pex_l2_prsnt_n_pdd7 {
- nvidia,pins = "pex_l2_prsnt_n_pdd7";
- nvidia,function = "pcie";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- clk3_out_pee0 {
- nvidia,pins = "clk3_out_pee0";
- nvidia,function = "extperiph3";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- clk3_req_pee1 {
- nvidia,pins = "clk3_req_pee1";
- nvidia,function = "dev3";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- clk1_req_pee2 {
- nvidia,pins = "clk1_req_pee2";
- nvidia,function = "dap";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- };
- hdmi_cec_pee3 {
- nvidia,pins = "hdmi_cec_pee3";
- nvidia,function = "cec";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_DISABLE>;
- nvidia,enable-input = <TEGRA_PIN_ENABLE>;
- nvidia,open-drain = <TEGRA_PIN_DISABLE>;
- };
- owr {
- nvidia,pins = "owr";
- nvidia,function = "owr";
- nvidia,pull = <TEGRA_PIN_PULL_NONE>;
- nvidia,tristate = <TEGRA_PIN_ENABLE>;
- nvidia,enable-input = <TEGRA_PIN_DISABLE>;
- };
- drive_groups {
- nvidia,pins = "drive_gma",
- "drive_gmb",
- "drive_gmc",
- "drive_gmd";
- nvidia,pull-down-strength = <9>;
- nvidia,pull-up-strength = <9>;
- nvidia,slew-rate-rising = <TEGRA_PIN_SLEW_RATE_SLOWEST>;
- nvidia,slew-rate-falling = <TEGRA_PIN_SLEW_RATE_SLOWEST>;
- };
-};
&emc_icc_dvfs_opp_table {
- /delete-node/ opp@900000000,1350;
+ /delete-node/ opp-900000000-1350;
};
&emc_bw_dfs_opp_table {
- /delete-node/ opp@900000000;
+ /delete-node/ opp-900000000;
};
diff --git a/arch/arm/boot/dts/tegra30-pegatron-chagall.dts b/arch/arm/boot/dts/tegra30-pegatron-chagall.dts
new file mode 100644
index 0000000..f4b2d42
--- /dev/null
+++ b/arch/arm/boot/dts/tegra30-pegatron-chagall.dts
@@ -0,0 +1,2859 @@
+// SPDX-License-Identifier: GPL-2.0
+/dts-v1/;
+
+#include <dt-bindings/input/gpio-keys.h>
+#include <dt-bindings/input/input.h>
+#include <dt-bindings/thermal/thermal.h>
+
+#include "tegra30.dtsi"
+#include "tegra30-cpu-opp.dtsi"
+#include "tegra30-cpu-opp-microvolt.dtsi"
+#include "tegra30-asus-lvds-display.dtsi"
+
+/ {
+ model = "Pegatron Chagall";
+ compatible = "pegatron,chagall", "nvidia,tegra30";
+ chassis-type = "tablet";
+
+ aliases {
+ mmc0 = &sdmmc4; /* eMMC */
+ mmc1 = &sdmmc1; /* uSD slot */
+ mmc2 = &sdmmc3; /* WiFi */
+
+ rtc0 = &pmic;
+ rtc1 = "/rtc@7000e000";
+
+ display0 = &lcd;
+ display1 = &hdmi;
+
+ serial1 = &uartc; /* Bluetooth */
+ serial2 = &uartb; /* GPS */
+ };
+
+ /*
+ * The decompressor and also some bootloaders rely on a
+ * pre-existing /chosen node to be available to insert the
+ * command line and merge other ATAGS info.
+ */
+ chosen {};
+
+ memory@80000000 {
+ reg = <0x80000000 0x40000000>;
+ };
+
+ reserved-memory {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges;
+
+ linux,cma@80000000 {
+ compatible = "shared-dma-pool";
+ alloc-ranges = <0x80000000 0x30000000>;
+ size = <0x10000000>; /* 256MiB */
+ linux,cma-default;
+ reusable;
+ };
+
+ ramoops@beb00000 {
+ compatible = "ramoops";
+ reg = <0xbeb00000 0x10000>; /* 64kB */
+ console-size = <0x8000>; /* 32kB */
+ record-size = <0x400>; /* 1kB */
+ ecc-size = <16>;
+ };
+
+ trustzone@bfe00000 {
+ reg = <0xbfe00000 0x200000>; /* 2MB */
+ no-map;
+ };
+ };
+
+ host1x@50000000 {
+ hdmi: hdmi@54280000 {
+ status = "okay";
+
+ hdmi-supply = <&hdmi_5v0_sys>;
+ pll-supply = <&vdd_1v8_vio>;
+ vdd-supply = <&vdd_3v3_sys>;
+
+ nvidia,hpd-gpio = <&gpio TEGRA_GPIO(N, 7) GPIO_ACTIVE_HIGH>;
+ nvidia,ddc-i2c-bus = <&hdmi_ddc>;
+ };
+ };
+
+ vde@6001a000 {
+ assigned-clocks = <&tegra_car TEGRA30_CLK_VDE>;
+ assigned-clock-parents = <&tegra_car TEGRA30_CLK_PLL_P>;
+ assigned-clock-rates = <408000000>;
+ };
+
+ pinmux@70000868 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&state_default>;
+
+ state_default: pinmux {
+ /* SDMMC1 pinmux */
+ sdmmc1_clk_pz0 {
+ nvidia,pins = "sdmmc1_clk_pz0";
+ nvidia,function = "sdmmc1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ sdmmc1_dat3_py4 {
+ nvidia,pins = "sdmmc1_dat3_py4",
+ "sdmmc1_dat2_py5",
+ "sdmmc1_dat1_py6",
+ "sdmmc1_dat0_py7",
+ "sdmmc1_cmd_pz1";
+ nvidia,function = "sdmmc1";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ /* SDMMC2 pinmux */
+ vi_d1_pd5 {
+ nvidia,pins = "vi_d1_pd5",
+ "vi_d2_pl0",
+ "vi_d3_pl1",
+ "vi_d5_pl3",
+ "vi_d7_pl5";
+ nvidia,function = "sdmmc2";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ vi_d8_pl6 {
+ nvidia,pins = "vi_d8_pl6",
+ "vi_d9_pl7";
+ nvidia,function = "sdmmc2";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ nvidia,lock = <0>;
+ nvidia,ioreset = <0>;
+ };
+
+ /* SDMMC3 pinmux */
+ sdmmc3_clk_pa6 {
+ nvidia,pins = "sdmmc3_clk_pa6";
+ nvidia,function = "sdmmc3";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ sdmmc3_cmd_pa7 {
+ nvidia,pins = "sdmmc3_cmd_pa7",
+ "sdmmc3_dat3_pb4",
+ "sdmmc3_dat2_pb5",
+ "sdmmc3_dat2_pb5",
+ "sdmmc3_dat1_pb6",
+ "sdmmc3_dat0_pb7",
+ "sdmmc3_dat5_pd0",
+ "sdmmc3_dat4_pd1",
+ "sdmmc3_dat6_pd3",
+ "sdmmc3_dat7_pd4";
+ nvidia,function = "sdmmc3";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ /* SDMMC4 pinmux */
+ sdmmc4_clk_pcc4 {
+ nvidia,pins = "sdmmc4_clk_pcc4";
+ nvidia,function = "sdmmc4";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ sdmmc4_cmd_pt7 {
+ nvidia,pins = "sdmmc4_cmd_pt7",
+ "sdmmc4_dat0_paa0",
+ "sdmmc4_dat1_paa1",
+ "sdmmc4_dat2_paa2",
+ "sdmmc4_dat3_paa3",
+ "sdmmc4_dat4_paa4",
+ "sdmmc4_dat5_paa5",
+ "sdmmc4_dat6_paa6",
+ "sdmmc4_dat7_paa7";
+ nvidia,function = "sdmmc4";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ /* I2C pinmux */
+ gen1_i2c_scl_pc4 {
+ nvidia,pins = "gen1_i2c_scl_pc4",
+ "gen1_i2c_sda_pc5";
+ nvidia,function = "i2c1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ nvidia,open-drain = <TEGRA_PIN_ENABLE>;
+ nvidia,lock = <0>;
+ };
+
+ gen2_i2c_scl_pt5 {
+ nvidia,pins = "gen2_i2c_scl_pt5",
+ "gen2_i2c_sda_pt6";
+ nvidia,function = "i2c2";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ nvidia,open-drain = <TEGRA_PIN_ENABLE>;
+ nvidia,lock = <0>;
+ };
+
+ cam_i2c_scl_pbb1 {
+ nvidia,pins = "cam_i2c_scl_pbb1",
+ "cam_i2c_sda_pbb2";
+ nvidia,function = "i2c3";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ nvidia,open-drain = <TEGRA_PIN_ENABLE>;
+ nvidia,lock = <0>;
+ };
+
+ ddc_scl_pv4 {
+ nvidia,pins = "ddc_scl_pv4",
+ "ddc_sda_pv5";
+ nvidia,function = "i2c4";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ nvidia,lock = <0>;
+ };
+
+ pwr_i2c_scl_pz6 {
+ nvidia,pins = "pwr_i2c_scl_pz6",
+ "pwr_i2c_sda_pz7";
+ nvidia,function = "i2cpwr";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ nvidia,open-drain = <TEGRA_PIN_ENABLE>;
+ nvidia,lock = <0>;
+ };
+
+ /* HDMI-CEC pinmux */
+ hdmi_cec_pee3 {
+ nvidia,pins = "hdmi_cec_pee3";
+ nvidia,function = "cec";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ nvidia,open-drain = <TEGRA_PIN_ENABLE>;
+ nvidia,lock = <0>;
+ };
+
+ /* UART-A */
+ ulpi_data0_po1 {
+ nvidia,pins = "ulpi_data0_po1";
+ nvidia,function = "uarta";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ ulpi_data1_po2 {
+ nvidia,pins = "ulpi_data1_po2",
+ "ulpi_data2_po3",
+ "ulpi_data3_po4",
+ "ulpi_data4_po5",
+ "ulpi_data5_po6",
+ "ulpi_data6_po7";
+ nvidia,function = "uarta";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ ulpi_data7_po0 {
+ nvidia,pins = "ulpi_data7_po0";
+ nvidia,function = "uarta";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ /* UART-B */
+ uart2_txd_pc2 {
+ nvidia,pins = "uart2_txd_pc2",
+ "uart2_rts_n_pj6";
+ nvidia,function = "uartb";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ uart2_rxd_pc3 {
+ nvidia,pins = "uart2_rxd_pc3",
+ "uart2_cts_n_pj5";
+ nvidia,function = "uartb";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ /* UART-C */
+ uart3_cts_n_pa1 {
+ nvidia,pins = "uart3_cts_n_pa1",
+ "uart3_rxd_pw7";
+ nvidia,function = "uartc";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ uart3_rts_n_pc0 {
+ nvidia,pins = "uart3_rts_n_pc0",
+ "uart3_txd_pw6";
+ nvidia,function = "uartc";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ /* UART-D */
+ ulpi_clk_py0 {
+ nvidia,pins = "ulpi_clk_py0",
+ "ulpi_stp_py3";
+ nvidia,function = "uartd";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ ulpi_dir_py1 {
+ nvidia,pins = "ulpi_dir_py1",
+ "ulpi_nxt_py2";
+ nvidia,function = "uartd";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ /* I2S pinmux */
+ dap1_fs_pn0 {
+ nvidia,pins = "dap1_fs_pn0",
+ "dap1_din_pn1",
+ "dap1_dout_pn2",
+ "dap1_sclk_pn3";
+ nvidia,function = "i2s0";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ dap2_fs_pa2 {
+ nvidia,pins = "dap2_fs_pa2",
+ "dap2_sclk_pa3",
+ "dap2_din_pa4",
+ "dap2_dout_pa5";
+ nvidia,function = "i2s1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ dap3_fs_pp0 {
+ nvidia,pins = "dap3_fs_pp0",
+ "dap3_din_pp1",
+ "dap3_dout_pp2",
+ "dap3_sclk_pp3";
+ nvidia,function = "i2s2";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ dap4_fs_pp4 {
+ nvidia,pins = "dap4_fs_pp4",
+ "dap4_din_pp5",
+ "dap4_dout_pp6",
+ "dap4_sclk_pp7";
+ nvidia,function = "i2s3";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ pcc2 {
+ nvidia,pins = "pcc2";
+ nvidia,function = "i2s4";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ /* PCI-e pinmux */
+ pex_l2_rst_n_pcc6 {
+ nvidia,pins = "pex_l2_rst_n_pcc6",
+ "pex_l0_rst_n_pdd1",
+ "pex_l1_rst_n_pdd5";
+ nvidia,function = "pcie";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ pex_l2_clkreq_n_pcc7 {
+ nvidia,pins = "pex_l2_clkreq_n_pcc7",
+ "pex_l0_prsnt_n_pdd0",
+ "pex_l0_clkreq_n_pdd2",
+ "pex_l2_prsnt_n_pdd7";
+ nvidia,function = "pcie";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ pex_wake_n_pdd3 {
+ nvidia,pins = "pex_wake_n_pdd3";
+ nvidia,function = "pcie";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ /* SPI pinmux */
+ spi1_mosi_px4 {
+ nvidia,pins = "spi1_mosi_px4",
+ "spi1_sck_px5",
+ "spi1_cs0_n_px6",
+ "spi1_miso_px7";
+ nvidia,function = "spi1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ spi2_cs1_n_pw2 {
+ nvidia,pins = "spi2_cs1_n_pw2",
+ "spi2_cs2_n_pw3";
+ nvidia,function = "spi2";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ spi2_sck_px2 {
+ nvidia,pins = "spi2_sck_px2";
+ nvidia,function = "gmi";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ gmi_a16_pj7 {
+ nvidia,pins = "gmi_a16_pj7",
+ "gmi_a19_pk7";
+ nvidia,function = "spi4";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ gmi_a17_pb0 {
+ nvidia,pins = "gmi_a17_pb0",
+ "gmi_a18_pb1";
+ nvidia,function = "spi4";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ spi2_mosi_px0 {
+ nvidia,pins = "spi2_mosi_px0";
+ nvidia,function = "spi6";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ spdif_out_pk5 {
+ nvidia,pins = "spdif_out_pk5";
+ nvidia,function = "spdif";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ spdif_in_pk6 {
+ nvidia,pins = "spdif_in_pk6";
+ nvidia,function = "spdif";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ /* Display A pinmux */
+ lcd_pwr0_pb2 {
+ nvidia,pins = "lcd_pwr0_pb2",
+ "lcd_pclk_pb3",
+ "lcd_pwr1_pc1",
+ "lcd_pwr2_pc6",
+ "lcd_d0_pe0",
+ "lcd_d1_pe1",
+ "lcd_d2_pe2",
+ "lcd_d3_pe3",
+ "lcd_d4_pe4",
+ "lcd_d5_pe5",
+ "lcd_d6_pe6",
+ "lcd_d7_pe7",
+ "lcd_d8_pf0",
+ "lcd_d9_pf1",
+ "lcd_d10_pf2",
+ "lcd_d11_pf3",
+ "lcd_d12_pf4",
+ "lcd_d13_pf5",
+ "lcd_d14_pf6",
+ "lcd_d15_pf7",
+ "lcd_de_pj1",
+ "lcd_hsync_pj3",
+ "lcd_vsync_pj4",
+ "lcd_d16_pm0",
+ "lcd_d17_pm1",
+ "lcd_d18_pm2",
+ "lcd_d19_pm3",
+ "lcd_d20_pm4",
+ "lcd_d21_pm5",
+ "lcd_d22_pm6",
+ "lcd_d23_pm7",
+ "lcd_cs0_n_pn4",
+ "lcd_sdout_pn5",
+ "lcd_dc0_pn6",
+ "lcd_sdin_pz2",
+ "lcd_wr_n_pz3",
+ "lcd_sck_pz4",
+ "lcd_cs1_n_pw0",
+ "lcd_m1_pw1";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ lcd_dc1_pd2 {
+ nvidia,pins = "lcd_dc1_pd2";
+ nvidia,function = "displaya";
+ nvidia,pull = <TEGRA_PIN_PULL_DOWN>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ clk_32k_out_pa0 {
+ nvidia,pins = "clk_32k_out_pa0";
+ nvidia,function = "blink";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ /* KBC keys */
+ kb_row0_pr0 {
+ nvidia,pins = "kb_row0_pr0",
+ "kb_row1_pr1",
+ "kb_row2_pr2",
+ "kb_row3_pr3",
+ "kb_row8_ps0",
+ "kb_col0_pq0",
+ "kb_col1_pq1",
+ "kb_col2_pq2",
+ "kb_col3_pq3",
+ "kb_col4_pq4",
+ "kb_col5_pq5",
+ "kb_col7_pq7";
+ nvidia,function = "kbc";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ kb_row4_pr4 {
+ nvidia,pins = "kb_row4_pr4",
+ "kb_row7_pr7",
+ "kb_row10_ps2",
+ "kb_row13_ps5";
+ nvidia,function = "kbc";
+ nvidia,pull = <TEGRA_PIN_PULL_DOWN>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ kb_row11_ps3 {
+ nvidia,pins = "kb_row11_ps3",
+ "kb_row12_ps4",
+ "kb_row15_ps7";
+ nvidia,function = "kbc";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ kb_row14_ps6 {
+ nvidia,pins = "kb_row14_ps6";
+ nvidia,function = "kbc";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ gmi_iordy_pi5 {
+ nvidia,pins = "gmi_iordy_pi5";
+ nvidia,function = "rsvd1";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ vi_pclk_pt0 {
+ nvidia,pins = "vi_pclk_pt0";
+ nvidia,function = "rsvd1";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ nvidia,lock = <0>;
+ nvidia,ioreset = <0>;
+ };
+
+ pu1 {
+ nvidia,pins = "pu1";
+ nvidia,function = "rsvd1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ pu2 {
+ nvidia,pins = "pu2";
+ nvidia,function = "rsvd1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ pv0 {
+ nvidia,pins = "pv0";
+ nvidia,function = "rsvd1";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ pv1 {
+ nvidia,pins = "pv1";
+ nvidia,function = "rsvd1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ pcc1 {
+ nvidia,pins = "pcc1";
+ nvidia,function = "rsvd2";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ sdmmc4_rst_n_pcc3 {
+ nvidia,pins = "sdmmc4_rst_n_pcc3";
+ nvidia,function = "rsvd2";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ pv3 {
+ nvidia,pins = "pv3";
+ nvidia,function = "rsvd2";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ vi_vsync_pd6 {
+ nvidia,pins = "vi_vsync_pd6",
+ "vi_hsync_pd7";
+ nvidia,function = "rsvd2";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ nvidia,lock = <0>;
+ nvidia,ioreset = <0>;
+ };
+
+ vi_d10_pt2 {
+ nvidia,pins = "vi_d10_pt2",
+ "vi_d0_pt4", "pbb0";
+ nvidia,function = "rsvd2";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ vi_d11_pt3 {
+ nvidia,pins = "vi_d11_pt3";
+ nvidia,function = "rsvd2";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ pu0 {
+ nvidia,pins = "pu0";
+ nvidia,function = "rsvd4";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ pu3 {
+ nvidia,pins = "pu3";
+ nvidia,function = "rsvd4";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ pu6 {
+ nvidia,pins = "pu6";
+ nvidia,function = "rsvd4";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ pex_l1_prsnt_n_pdd4 {
+ nvidia,pins = "pex_l1_prsnt_n_pdd4",
+ "pex_l1_clkreq_n_pdd6";
+ nvidia,function = "rsvd4";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ gmi_wait_pi7 {
+ nvidia,pins = "gmi_wait_pi7",
+ "gmi_cs0_n_pj0",
+ "gmi_cs1_n_pj2",
+ "gmi_cs4_n_pk2";
+ nvidia,function = "nand";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ gmi_ad0_pg0 {
+ nvidia,pins = "gmi_ad0_pg0",
+ "gmi_ad1_pg1",
+ "gmi_ad2_pg2",
+ "gmi_ad3_pg3",
+ "gmi_ad4_pg4",
+ "gmi_ad5_pg5",
+ "gmi_ad6_pg6",
+ "gmi_ad7_pg7",
+ "gmi_wr_n_pi0",
+ "gmi_oe_n_pi1",
+ "gmi_dqs_pi2",
+ "gmi_adv_n_pk0",
+ "gmi_clk_pk1";
+ nvidia,function = "nand";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ gmi_cs2_n_pk3 {
+ nvidia,pins = "gmi_cs2_n_pk3";
+ nvidia,function = "rsvd1";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ gmi_cs3_n_pk4 {
+ nvidia,pins = "gmi_cs3_n_pk4";
+ nvidia,function = "nand";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ gmi_ad10_ph2 {
+ nvidia,pins = "gmi_ad10_ph2",
+ "gmi_ad11_ph3",
+ "gmi_ad14_ph6";
+ nvidia,function = "nand";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ gmi_ad13_ph5 {
+ nvidia,pins = "gmi_ad13_ph5",
+ "gmi_ad12_ph4",
+ "gmi_cs7_n_pi6";
+ nvidia,function = "nand";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ gmi_rst_n_pi4 {
+ nvidia,pins = "gmi_rst_n_pi4";
+ nvidia,function = "gmi";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ gmi_ad8_ph0 {
+ nvidia,pins = "gmi_ad8_ph0";
+ nvidia,function = "pwm0";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ gmi_ad9_ph1 {
+ nvidia,pins = "gmi_ad9_ph1";
+ nvidia,function = "pwm1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ gmi_wp_n_pc7 {
+ nvidia,pins = "gmi_wp_n_pc7";
+ nvidia,function = "gmi";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ gmi_cs6_n_pi3 {
+ nvidia,pins = "gmi_cs6_n_pi3";
+ nvidia,function = "sata";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ vi_d4_pl2 {
+ nvidia,pins = "vi_d4_pl2";
+ nvidia,function = "vi";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ vi_d6_pl4 {
+ nvidia,pins = "vi_d6_pl4";
+ nvidia,function = "vi";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ nvidia,lock = <0>;
+ nvidia,ioreset = <0>;
+ };
+
+ vi_mclk_pt1 {
+ nvidia,pins = "vi_mclk_pt1";
+ nvidia,function = "vi";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ /* HDMI hot-plug-detect */
+ hdmi_int_pn7 {
+ nvidia,pins = "hdmi_int_pn7";
+ nvidia,function = "hdmi";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_ENABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ pu4 {
+ nvidia,pins = "pu4";
+ nvidia,function = "pwm1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ pu5 {
+ nvidia,pins = "pu5";
+ nvidia,function = "pwm2";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ jtag_rtck_pu7 {
+ nvidia,pins = "jtag_rtck_pu7";
+ nvidia,function = "rtck";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ crt_hsync_pv6 {
+ nvidia,pins = "crt_hsync_pv6",
+ "crt_vsync_pv7";
+ nvidia,function = "crt";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ clk1_out_pw4 {
+ nvidia,pins = "clk1_out_pw4";
+ nvidia,function = "extperiph1";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ clk2_out_pw5 {
+ nvidia,pins = "clk2_out_pw5";
+ nvidia,function = "extperiph2";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ clk3_out_pee0 {
+ nvidia,pins = "clk3_out_pee0";
+ nvidia,function = "extperiph3";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ sys_clk_req_pz5 {
+ nvidia,pins = "sys_clk_req_pz5";
+ nvidia,function = "sysclk";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ pbb4 {
+ nvidia,pins = "pbb4";
+ nvidia,function = "vgp4";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ pbb5 {
+ nvidia,pins = "pbb5";
+ nvidia,function = "vgp5";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ pbb6 {
+ nvidia,pins = "pbb6";
+ nvidia,function = "vgp6";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ clk1_req_pee2 {
+ nvidia,pins = "clk1_req_pee2";
+ nvidia,function = "dap";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ clk2_req_pcc5 {
+ nvidia,pins = "clk2_req_pcc5";
+ nvidia,function = "dap";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ clk3_req_pee1 {
+ nvidia,pins = "clk3_req_pee1";
+ nvidia,function = "dev3";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ owr {
+ nvidia,pins = "owr";
+ nvidia,function = "owr";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ pv2 {
+ nvidia,pins = "pv2",
+ "kb_row5_pr5";
+ nvidia,function = "owr";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+ };
+
+ pbb3 {
+ nvidia,pins = "pbb3";
+ nvidia,function = "vgp3";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ pbb7 {
+ nvidia,pins = "pbb7";
+ nvidia,function = "i2s4";
+ nvidia,pull = <TEGRA_PIN_PULL_NONE>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ cam_mclk_pcc0 {
+ nvidia,pins = "cam_mclk_pcc0";
+ nvidia,function = "vi_alt3";
+ nvidia,pull = <TEGRA_PIN_PULL_UP>;
+ nvidia,tristate = <TEGRA_PIN_DISABLE>;
+ nvidia,enable-input = <TEGRA_PIN_ENABLE>;
+ };
+
+ /* GPIO power/drive control */
+ drive_dap1 {
+ nvidia,pins = "drive_dap1",
+ "drive_dap2",
+ "drive_dbg",
+ "drive_at5",
+ "drive_gme",
+ "drive_ddc",
+ "drive_ao1",
+ "drive_uart3";
+ nvidia,high-speed-mode = <0>;
+ nvidia,schmitt = <TEGRA_PIN_ENABLE>;
+ nvidia,low-power-mode = <TEGRA_PIN_LP_DRIVE_DIV_1>;
+ nvidia,pull-down-strength = <31>;
+ nvidia,pull-up-strength = <31>;
+ nvidia,slew-rate-rising = <TEGRA_PIN_SLEW_RATE_FASTEST>;
+ nvidia,slew-rate-falling = <TEGRA_PIN_SLEW_RATE_FASTEST>;
+ };
+
+ drive_sdio1 {
+ nvidia,pins = "drive_sdio1";
+ nvidia,high-speed-mode = <0>;
+ nvidia,schmitt = <TEGRA_PIN_DISABLE>;
+ nvidia,pull-down-strength = <5>;
+ nvidia,pull-up-strength = <5>;
+ nvidia,slew-rate-rising = <TEGRA_PIN_SLEW_RATE_FAST>;
+ nvidia,slew-rate-falling = <TEGRA_PIN_SLEW_RATE_FAST>;
+ };
+
+ drive_sdio3 {
+ nvidia,pins = "drive_sdio3";
+ nvidia,high-speed-mode = <0>;
+ nvidia,schmitt = <TEGRA_PIN_DISABLE>;
+ nvidia,pull-down-strength = <46>;
+ nvidia,pull-up-strength = <42>;
+ nvidia,slew-rate-rising = <TEGRA_PIN_SLEW_RATE_FAST>;
+ nvidia,slew-rate-falling = <TEGRA_PIN_SLEW_RATE_FAST>;
+ };
+
+ drive_gma {
+ nvidia,pins = "drive_gma",
+ "drive_gmb",
+ "drive_gmc",
+ "drive_gmd";
+ nvidia,pull-down-strength = <9>;
+ nvidia,pull-up-strength = <9>;
+ nvidia,slew-rate-rising = <TEGRA_PIN_SLEW_RATE_SLOWEST>;
+ nvidia,slew-rate-falling = <TEGRA_PIN_SLEW_RATE_SLOWEST>;
+ };
+
+ drive_lcd2 {
+ nvidia,pins = "drive_lcd2";
+ nvidia,high-speed-mode = <0>;
+ nvidia,schmitt = <TEGRA_PIN_ENABLE>;
+ nvidia,low-power-mode = <TEGRA_PIN_LP_DRIVE_DIV_4>;
+ nvidia,pull-down-strength = <20>;
+ nvidia,pull-up-strength = <20>;
+ nvidia,slew-rate-rising = <TEGRA_PIN_SLEW_RATE_FASTEST>;
+ nvidia,slew-rate-falling = <TEGRA_PIN_SLEW_RATE_FASTEST>;
+ };
+ };
+ };
+
+ uartb: serial@70006040 {
+ compatible = "nvidia,tegra30-hsuart";
+ status = "okay";
+
+ /* Broadcom GPS BCM47511 */
+ };
+
+ uartc: serial@70006200 {
+ compatible = "nvidia,tegra30-hsuart";
+ status = "okay";
+
+ nvidia,adjust-baud-rates = <0 9600 100>,
+ <9600 115200 200>,
+ <1000000 4000000 136>;
+
+ /* Azurewave AW-AH663 BCM4330B1 */
+ bluetooth {
+ compatible = "brcm,bcm4330-bt";
+ max-speed = <4000000>;
+
+ clocks = <&tegra_pmc TEGRA_PMC_CLK_BLINK>;
+ clock-names = "txco";
+
+ interrupt-parent = <&gpio>;
+ interrupts = <TEGRA_GPIO(U, 6) IRQ_TYPE_EDGE_RISING>;
+ interrupt-names = "host-wakeup";
+
+ device-wakeup-gpios = <&gpio TEGRA_GPIO(U, 1) GPIO_ACTIVE_HIGH>;
+ shutdown-gpios = <&gpio TEGRA_GPIO(U, 0) GPIO_ACTIVE_HIGH>;
+
+ vbat-supply = <&vdd_3v3_sys>;
+ vddio-supply = <&vdd_1v8_vio>;
+ };
+ };
+
+ pwm: pwm@7000a000 {
+ status = "okay";
+ };
+
+ lcd_ddc: i2c@7000c000 {
+ status = "okay";
+ clock-frequency = <400000>;
+
+ /* Wolfson Microelectronics WM8903 audio codec */
+ wm8903: audio-codec@1a {
+ compatible = "wlf,wm8903";
+ reg = <0x1a>;
+
+ interrupt-parent = <&gpio>;
+ interrupts = <TEGRA_GPIO(W, 3) IRQ_TYPE_EDGE_BOTH>;
+
+ gpio-controller;
+ #gpio-cells = <2>;
+
+ micdet-cfg = <0>;
+ micdet-delay = <100>;
+
+ gpio-cfg = <0xffffffff 0xffffffff 0 0xffffffff 0xffffffff>;
+
+ AVDD-supply = <&vdd_1v8_vio>;
+ CPVDD-supply = <&vdd_1v8_vio>;
+ DBVDD-supply = <&vdd_1v8_vio>;
+ DCVDD-supply = <&vdd_1v8_vio>;
+ };
+ };
+
+ i2c2: i2c@7000c400 {
+ status = "okay";
+ clock-frequency = <400000>;
+
+ /* Atmel touchscreen */
+ touchscreen@4d {
+ compatible = "atmel,maxtouch";
+ reg = <0x4d>;
+
+ interrupt-parent = <&gpio>;
+ interrupts = <TEGRA_GPIO(H, 4) IRQ_TYPE_EDGE_FALLING>;
+ reset-gpios = <&gpio TEGRA_GPIO(H, 6) GPIO_ACTIVE_LOW>;
+
+ vdda-supply = <&vdd_3v3_sys>;
+ vdd-supply = <&vdd_3v3_sys>;
+ };
+ };
+
+ i2c3: i2c@7000c500 {
+ status = "okay";
+ clock-frequency = <400000>;
+
+ light-sensor@44 {
+ compatible = "isil,isl29023";
+ reg = <0x44>;
+
+ interrupt-parent = <&gpio>;
+ interrupts = <TEGRA_GPIO(Q, 3) IRQ_TYPE_LEVEL_HIGH>;
+
+ vcc-supply = <&vdd_3v3_sen>;
+ };
+
+ /* AsahiKASEI AK8975 magnetometer sensor */
+ magnetometer@c {
+ compatible = "asahi-kasei,ak8975";
+ reg = <0x0c>;
+
+ vdd-supply = <&vdd_3v3_sen>;
+ vid-supply = <&vdd_1v8_vio>;
+
+ mount-matrix = "0", "1", "0",
+ "1", "0", "0",
+ "0", "0", "-1";
+ };
+
+ gyroscope@68 {
+ compatible = "invensense,mpu3050";
+ reg = <0x68>;
+
+ interrupt-parent = <&gpio>;
+ interrupts = <TEGRA_GPIO(X, 1) IRQ_TYPE_EDGE_RISING>;
+
+ vdd-supply = <&vdd_3v3_sen>;
+ vlogic-supply = <&vdd_1v8_vio>;
+
+ mount-matrix = "0", "1", "0",
+ "1", "0", "0",
+ "0", "0", "-1";
+
+ /* External I2C interface */
+ i2c-gate {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ accelerometer@f {
+ compatible = "kionix,kxtf9";
+ reg = <0x0f>;
+
+ interrupt-parent = <&gpio>;
+ interrupts = <TEGRA_GPIO(L, 1) IRQ_TYPE_EDGE_RISING>;
+
+ vdd-supply = <&vdd_1v8_vio>;
+ vddio-supply = <&vdd_1v8_vio>;
+
+ mount-matrix = "-1", "0", "0",
+ "0", "1", "0",
+ "0", "0", "1";
+ };
+ };
+ };
+ };
+
+ hdmi_ddc: i2c@7000c700 {
+ status = "okay";
+ clock-frequency = <93750>;
+ };
+
+ i2c5: i2c@7000d000 {
+ status = "okay";
+ clock-frequency = <400000>;
+
+ nct72: temperature-sensor@4c {
+ compatible = "onnn,nct1008";
+ reg = <0x4c>;
+
+ interrupt-parent = <&gpio>;
+ interrupts = <TEGRA_GPIO(U, 5) IRQ_TYPE_EDGE_FALLING>;
+
+ vcc-supply = <&vdd_3v3_sys>;
+ #thermal-sensor-cells = <1>;
+ };
+
+ /* Texas Instruments TPS659110 PMIC */
+ pmic: pmic@2d {
+ compatible = "ti,tps65911";
+ reg = <0x2d>;
+
+ interrupts = <GIC_SPI 86 IRQ_TYPE_LEVEL_HIGH>;
+ #interrupt-cells = <2>;
+ interrupt-controller;
+ wakeup-source;
+
+ ti,en-gpio-sleep = <0 0 1 0 0 0 0 0 0>;
+ ti,system-power-controller;
+ ti,sleep-keep-ck32k;
+ ti,sleep-enable;
+
+ #gpio-cells = <2>;
+ gpio-controller;
+
+ vcc1-supply = <&vdd_5v0_sys>;
+ vcc2-supply = <&vdd_5v0_sys>;
+ vcc3-supply = <&vdd_1v8_vio>;
+ vcc4-supply = <&vdd_1v8_vio>;
+ vcc5-supply = <&vdd_5v0_sys>;
+ vcc6-supply = <&vddio_1v2_ddr>;
+ vcc7-supply = <&vdd_5v0_sys>;
+ vccio-supply = <&vdd_5v0_sys>;
+
+ pmic-sleep-hog {
+ gpio-hog;
+ gpios = <0 GPIO_ACTIVE_HIGH>,
+ <2 GPIO_ACTIVE_HIGH>,
+ <6 GPIO_ACTIVE_HIGH>,
+ <8 GPIO_ACTIVE_HIGH>;
+ output-high;
+ };
+
+ regulators {
+ /* VDD1 is not used by Chagall */
+
+ vddio_1v2_ddr: vdd2 {
+ regulator-name = "vddio_1v2_ddr";
+ regulator-min-microvolt = <1200000>;
+ regulator-max-microvolt = <1200000>;
+ regulator-always-on;
+ regulator-boot-on;
+ };
+
+ vdd_cpu: vddctrl {
+ regulator-name = "vdd_cpu,vdd_sys";
+ regulator-min-microvolt = <600000>;
+ regulator-max-microvolt = <1400000>;
+ regulator-coupled-with = <&vdd_core>;
+ regulator-coupled-max-spread = <300000>;
+ regulator-max-step-microvolt = <100000>;
+ regulator-always-on;
+ regulator-boot-on;
+ ti,regulator-ext-sleep-control = <1>;
+
+ nvidia,tegra-cpu-regulator;
+ };
+
+ vdd_1v8_vio: vio {
+ regulator-name = "vdd_1v8_gen";
+ /* FIXME: eMMC won't work, if set to 1.8 V */
+ regulator-min-microvolt = <1500000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-always-on;
+ regulator-boot-on;
+ };
+
+ /* eMMC VDD */
+ vcore_emmc: ldo1 {
+ regulator-name = "vdd_emmc_core";
+ regulator-min-microvolt = <1000000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-always-on;
+ };
+
+ /* uSD slot VDD */
+ vdd_usd: ldo2 {
+ regulator-name = "vdd_usd";
+ regulator-min-microvolt = <3200000>;
+ regulator-max-microvolt = <3200000>;
+ };
+
+ /* uSD slot VDDIO */
+ vddio_usd: ldo3 {
+ regulator-name = "vddio_usd";
+ regulator-min-microvolt = <1900000>;
+ regulator-max-microvolt = <3200000>;
+ };
+
+ ldo4 {
+ regulator-name = "vdd_rtc";
+ regulator-min-microvolt = <1200000>;
+ regulator-max-microvolt = <1200000>;
+ regulator-always-on;
+ };
+
+ ldo5 {
+ regulator-name = "vdd_1v3_cam_isp";
+ regulator-min-microvolt = <1300000>;
+ regulator-max-microvolt = <1300000>;
+ };
+
+ ldo6 {
+ regulator-name = "avdd_dsi_csi,pwrdet_mipi";
+ regulator-min-microvolt = <1200000>;
+ regulator-max-microvolt = <1200000>;
+ };
+
+ ldo7 {
+ regulator-name = "vdd_pllm,x,u,a_p_c_s";
+ regulator-min-microvolt = <1200000>;
+ regulator-max-microvolt = <1200000>;
+ regulator-always-on;
+ regulator-boot-on;
+ ti,regulator-ext-sleep-control = <8>;
+ };
+
+ ldo8 {
+ regulator-name = "vdd_ddr_hs";
+ regulator-min-microvolt = <1000000>;
+ regulator-max-microvolt = <1000000>;
+ regulator-always-on;
+ ti,regulator-ext-sleep-control = <8>;
+ };
+ };
+ };
+
+ vdd_core: core-regulator@60 {
+ compatible = "ti,tps62361";
+ reg = <0x60>;
+
+ regulator-name = "tps62361-vout";
+ regulator-min-microvolt = <500000>;
+ regulator-max-microvolt = <1770000>;
+ regulator-coupled-with = <&vdd_cpu>;
+ regulator-coupled-max-spread = <300000>;
+ regulator-max-step-microvolt = <100000>;
+ regulator-boot-on;
+ regulator-always-on;
+ ti,enable-vout-discharge;
+ ti,vsel0-state-high;
+ ti,vsel1-state-high;
+
+ nvidia,tegra-core-regulator;
+ };
+ };
+
+ vdd_5v0_sys: regulator-5v {
+ compatible = "regulator-fixed";
+ regulator-name = "vdd_5v0_sys";
+ regulator-min-microvolt = <5000000>;
+ regulator-max-microvolt = <5000000>;
+ regulator-always-on;
+ regulator-boot-on;
+ };
+
+ vdd_3v3_sys: regulator-3v {
+ compatible = "regulator-fixed";
+ regulator-name = "vdd_3v3_sys";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-always-on;
+ regulator-boot-on;
+ };
+
+ vdd_pnl: regulator-panel {
+ compatible = "regulator-fixed";
+ regulator-name = "vdd_panel";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-enable-ramp-delay = <300000>;
+ gpio = <&gpio TEGRA_GPIO(W, 1) GPIO_ACTIVE_HIGH>;
+ enable-active-high;
+ vin-supply = <&vdd_3v3_sys>;
+ };
+
+ vdd_3v3_sen: regulator-sensors {
+ compatible = "regulator-fixed";
+ regulator-name = "sen_3v3_en";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ gpio = <&gpio TEGRA_GPIO(K, 5) GPIO_ACTIVE_HIGH>;
+ enable-active-high;
+ vin-supply = <&vdd_3v3_sys>;
+ };
+
+ vdd_5v0_bl: regulator-bl {
+ compatible = "regulator-fixed";
+ regulator-name = "vdd_5v0_bl";
+ regulator-min-microvolt = <5000000>;
+ regulator-max-microvolt = <5000000>;
+ regulator-boot-on;
+ gpio = <&gpio TEGRA_GPIO(C, 6) GPIO_ACTIVE_HIGH>;
+ enable-active-high;
+ vin-supply = <&vdd_5v0_sys>;
+ };
+
+ hdmi_5v0_sys: regulator-hdmi {
+ compatible = "regulator-fixed";
+ regulator-name = "hdmi_5v0_sys";
+ regulator-min-microvolt = <5000000>;
+ regulator-max-microvolt = <5000000>;
+ gpio = <&gpio TEGRA_GPIO(P, 2) GPIO_ACTIVE_HIGH>;
+ enable-active-high;
+ vin-supply = <&vdd_5v0_sys>;
+ };
+
+ vdd_vbus_usb1: regulator-usb1 {
+ compatible = "regulator-fixed";
+ regulator-name = "vdd_vbus_micro_usb";
+ regulator-min-microvolt = <5000000>;
+ regulator-max-microvolt = <5000000>;
+ gpio = <&gpio TEGRA_GPIO(DD, 3) GPIO_ACTIVE_HIGH>;
+ enable-active-high;
+ vin-supply = <&vdd_5v0_sys>;
+ };
+
+ vdd_vbus_usb3: regulator-usb3 {
+ compatible = "regulator-fixed";
+ regulator-name = "vdd_vbus_typea_usb";
+ regulator-min-microvolt = <5000000>;
+ regulator-max-microvolt = <5000000>;
+ gpio = <&gpio TEGRA_GPIO(CC, 6) GPIO_ACTIVE_HIGH>;
+ enable-active-high;
+ vin-supply = <&vdd_5v0_sys>;
+ };
+
+ pmc@7000e400 {
+ status = "okay";
+ nvidia,invert-interrupt;
+ nvidia,suspend-mode = <2>;
+ nvidia,cpu-pwr-good-time = <2000>;
+ nvidia,cpu-pwr-off-time = <200>;
+ nvidia,core-pwr-good-time = <3845 3845>;
+ nvidia,core-pwr-off-time = <0>;
+ nvidia,core-power-req-active-high;
+ nvidia,sys-clock-req-active-high;
+ core-supply = <&vdd_core>;
+
+ /* Set DEV_OFF + PWR_OFF_SET bit in DCDC control register of TPS65911 PMIC */
+ i2c-thermtrip {
+ nvidia,i2c-controller-id = <4>;
+ nvidia,bus-addr = <0x2d>;
+ nvidia,reg-addr = <0x3f>;
+ nvidia,reg-data = <0x81>;
+ };
+ };
+
+ memory-controller@7000f000 {
+ emc-timings-0 {
+ /* SAMSUNG K4P8G304EB FGC1 */
+ nvidia,ram-code = <0>;
+
+ timing-25500000 {
+ clock-frequency = <25500000>;
+
+ nvidia,emem-configuration = < 0x00020001 0xc0000010
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000003 0x00000001 0x00000002 0x00000004
+ 0x00000001 0x00000000 0x00000002 0x00000002
+ 0x02020001 0x00060402 0x73e30303 0x001f0000 >;
+ };
+
+ timing-51000000 {
+ clock-frequency = <51000000>;
+
+ nvidia,emem-configuration = < 0x00010001 0xc0000010
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000003 0x00000001 0x00000002 0x00000004
+ 0x00000001 0x00000000 0x00000002 0x00000002
+ 0x02020001 0x00060402 0x72c30303 0x001f0000 >;
+ };
+
+ timing-102000000 {
+ clock-frequency = <102000000>;
+
+ nvidia,emem-configuration = < 0x00000001 0xc0000018
+ 0x00000001 0x00000001 0x00000003 0x00000001
+ 0x00000003 0x00000001 0x00000002 0x00000004
+ 0x00000001 0x00000000 0x00000002 0x00000002
+ 0x02020001 0x00060403 0x72430504 0x001f0000 >;
+ };
+
+ timing-204000000 {
+ clock-frequency = <204000000>;
+
+ nvidia,emem-configuration = < 0x00000003 0xc0000025
+ 0x00000001 0x00000001 0x00000006 0x00000003
+ 0x00000005 0x00000001 0x00000002 0x00000004
+ 0x00000001 0x00000000 0x00000003 0x00000002
+ 0x02030001 0x00070506 0x71e40a07 0x001f0000 >;
+ };
+
+ timing-400000000 {
+ clock-frequency = <400000000>;
+
+ nvidia,emem-configuration = < 0x00000006 0xc0000048
+ 0x00000002 0x00000003 0x0000000c 0x00000007
+ 0x00000009 0x00000001 0x00000002 0x00000006
+ 0x00000001 0x00000000 0x00000004 0x00000004
+ 0x04040001 0x000d090c 0x7026120d 0x001f0000 >;
+ };
+ };
+
+ emc-timings-1 {
+ /* ELPIDA EDB8132B2MA 8D_F */
+ nvidia,ram-code = <1>;
+
+ timing-25500000 {
+ clock-frequency = <25500000>;
+
+ nvidia,emem-configuration = < 0x00020001 0xc0000010
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000003 0x00000001 0x00000002 0x00000004
+ 0x00000001 0x00000000 0x00000002 0x00000002
+ 0x02020001 0x00060402 0x73e30303 0x001f0000 >;
+ };
+
+ timing-51000000 {
+ clock-frequency = <51000000>;
+
+ nvidia,emem-configuration = < 0x00010001 0xc0000010
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000003 0x00000001 0x00000002 0x00000004
+ 0x00000001 0x00000000 0x00000002 0x00000002
+ 0x02020001 0x00060402 0x72c30303 0x001f0000 >;
+ };
+
+ timing-102000000 {
+ clock-frequency = <102000000>;
+
+ nvidia,emem-configuration = < 0x00000001 0xc0000018
+ 0x00000001 0x00000001 0x00000003 0x00000001
+ 0x00000003 0x00000001 0x00000002 0x00000004
+ 0x00000001 0x00000000 0x00000002 0x00000002
+ 0x02020001 0x00060403 0x72430504 0x001f0000 >;
+ };
+
+ timing-204000000 {
+ clock-frequency = <204000000>;
+
+ nvidia,emem-configuration = < 0x00000003 0xc0000025
+ 0x00000001 0x00000001 0x00000006 0x00000003
+ 0x00000005 0x00000001 0x00000002 0x00000004
+ 0x00000001 0x00000000 0x00000003 0x00000002
+ 0x02030001 0x00070506 0x71e40a07 0x001f0000 >;
+ };
+
+ timing-400000000 {
+ clock-frequency = <400000000>;
+
+ nvidia,emem-configuration = < 0x00000006 0xc0000048
+ 0x00000002 0x00000003 0x0000000c 0x00000007
+ 0x00000009 0x00000001 0x00000002 0x00000006
+ 0x00000001 0x00000000 0x00000004 0x00000004
+ 0x04040001 0x000d090c 0x7026120d 0x001f0000 >;
+ };
+ };
+
+ emc-timings-2 {
+ /* SAMSUNG K4P8G304EB FGC2 */
+ nvidia,ram-code = <2>;
+
+ timing-25500000 {
+ clock-frequency = <25500000>;
+
+ nvidia,emem-configuration = < 0x00020001 0xc0000010
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000003 0x00000001 0x00000002 0x00000004
+ 0x00000001 0x00000000 0x00000002 0x00000002
+ 0x02020001 0x00060402 0x73e30303 0x001f0000 >;
+ };
+
+ timing-51000000 {
+ clock-frequency = <51000000>;
+
+ nvidia,emem-configuration = < 0x00010001 0xc0000010
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000003 0x00000001 0x00000002 0x00000004
+ 0x00000001 0x00000000 0x00000002 0x00000002
+ 0x02020001 0x00060402 0x72c30303 0x001f0000 >;
+ };
+
+ timing-102000000 {
+ clock-frequency = <102000000>;
+
+ nvidia,emem-configuration = < 0x00000001 0xc0000018
+ 0x00000001 0x00000001 0x00000003 0x00000001
+ 0x00000003 0x00000001 0x00000002 0x00000004
+ 0x00000001 0x00000000 0x00000002 0x00000002
+ 0x02020001 0x00060403 0x72430504 0x001f0000 >;
+ };
+
+ timing-204000000 {
+ clock-frequency = <204000000>;
+
+ nvidia,emem-configuration = < 0x00000003 0xc0000025
+ 0x00000001 0x00000001 0x00000006 0x00000003
+ 0x00000005 0x00000001 0x00000002 0x00000004
+ 0x00000001 0x00000000 0x00000003 0x00000002
+ 0x02030001 0x00070506 0x71e40a07 0x001f0000 >;
+ };
+
+ timing-533000000 {
+ clock-frequency = <533000000>;
+
+ nvidia,emem-configuration = < 0x00000008 0xc0000060
+ 0x00000003 0x00000004 0x00000010 0x0000000a
+ 0x0000000d 0x00000002 0x00000002 0x00000008
+ 0x00000002 0x00000000 0x00000004 0x00000005
+ 0x05040002 0x00110b10 0x70281811 0x001f0000 >;
+ };
+ };
+
+ emc-timings-3 {
+ /* HYNIX H9TCNNN8JDMMPR NGM */
+ nvidia,ram-code = <3>;
+
+ timing-25500000 {
+ clock-frequency = <25500000>;
+
+ nvidia,emem-configuration = < 0x00020001 0xc0000010
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000003 0x00000001 0x00000002 0x00000004
+ 0x00000001 0x00000000 0x00000002 0x00000002
+ 0x02020001 0x00060402 0x73e30303 0x001f0000 >;
+ };
+
+ timing-51000000 {
+ clock-frequency = <51000000>;
+
+ nvidia,emem-configuration = < 0x00010001 0xc0000010
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000003 0x00000001 0x00000002 0x00000004
+ 0x00000001 0x00000000 0x00000002 0x00000002
+ 0x02020001 0x00060402 0x72c30303 0x001f0000 >;
+ };
+
+ timing-102000000 {
+ clock-frequency = <102000000>;
+
+ nvidia,emem-configuration = < 0x00000001 0xc0000018
+ 0x00000001 0x00000001 0x00000003 0x00000001
+ 0x00000003 0x00000001 0x00000002 0x00000004
+ 0x00000001 0x00000000 0x00000002 0x00000002
+ 0x02020001 0x00060403 0x72430504 0x001f0000 >;
+ };
+
+ timing-204000000 {
+ clock-frequency = <204000000>;
+
+ nvidia,emem-configuration = < 0x00000003 0xc0000025
+ 0x00000001 0x00000001 0x00000006 0x00000003
+ 0x00000005 0x00000001 0x00000002 0x00000004
+ 0x00000001 0x00000000 0x00000003 0x00000002
+ 0x02030001 0x00070506 0x71e40a07 0x001f0000 >;
+ };
+
+ timing-533000000 {
+ clock-frequency = <533000000>;
+
+ nvidia,emem-configuration = < 0x00000008 0xc0000060
+ 0x00000003 0x00000004 0x00000010 0x0000000a
+ 0x0000000d 0x00000002 0x00000002 0x00000008
+ 0x00000002 0x00000000 0x00000004 0x00000005
+ 0x05040002 0x00110b10 0x70281811 0x001f0000 >;
+ };
+ };
+ };
+
+ memory-controller@7000f400 {
+ emc-timings-0 {
+ /* SAMSUNG K4P8G304EB FGC1 */
+ nvidia,ram-code = <0>;
+
+ timing-25500000 {
+ clock-frequency = <25500000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x00010022>;
+ nvidia,emc-mode-2 = <0x00020001>;
+ nvidia,emc-mode-reset = <0x00000000>;
+ nvidia,emc-zcal-cnt-long = <0x00000009>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000001
+ 0x00000003 0x00000002 0x00000002 0x00000004
+ 0x00000004 0x00000001 0x00000005 0x00000002
+ 0x00000002 0x00000001 0x00000001 0x00000000
+ 0x00000001 0x00000003 0x00000001 0x0000000b
+ 0x00000009 0x00000060 0x00000000 0x00000018
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000001 0x00000007 0x00000004 0x00000004
+ 0x00000003 0x00000008 0x00000004 0x00000004
+ 0x00000002 0x0000006b 0x00000004 0x00000004
+ 0x00000000 0x00000000 0x00004282 0x007800a4
+ 0x00008000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00100220 0x0800201c 0x00000000
+ 0x77ffc004 0x01f1f008 0x00000000 0x00000007
+ 0x08000068 0x08000000 0x00000802 0x00064000
+ 0x0000000a 0x00090009 0xa0f10000 0x00000000
+ 0x00000000 0x800001c5 0xe0000000 0xff00ff00 >;
+ };
+
+ timing-51000000 {
+ clock-frequency = <51000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x00010022>;
+ nvidia,emc-mode-2 = <0x00020001>;
+ nvidia,emc-mode-reset = <0x00000000>;
+ nvidia,emc-zcal-cnt-long = <0x00000009>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000003
+ 0x00000006 0x00000002 0x00000002 0x00000004
+ 0x00000004 0x00000001 0x00000005 0x00000002
+ 0x00000002 0x00000001 0x00000001 0x00000000
+ 0x00000001 0x00000003 0x00000001 0x0000000b
+ 0x00000009 0x000000c0 0x00000000 0x00000030
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000001 0x00000007 0x00000008 0x00000008
+ 0x00000003 0x00000008 0x00000004 0x00000004
+ 0x00000002 0x000000d5 0x00000004 0x00000004
+ 0x00000000 0x00000000 0x00004282 0x007800a4
+ 0x00008000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00100220 0x0800201c 0x00000000
+ 0x77ffc004 0x01f1f008 0x00000000 0x00000007
+ 0x08000068 0x08000000 0x00000802 0x00064000
+ 0x00000013 0x00090009 0xa0f10000 0x00000000
+ 0x00000000 0x80000287 0xe0000000 0xff00ff00 >;
+ };
+
+ timing-102000000 {
+ clock-frequency = <102000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x00010022>;
+ nvidia,emc-mode-2 = <0x00020001>;
+ nvidia,emc-mode-reset = <0x00000000>;
+ nvidia,emc-zcal-cnt-long = <0x0000000a>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000006
+ 0x0000000d 0x00000004 0x00000002 0x00000004
+ 0x00000004 0x00000001 0x00000005 0x00000002
+ 0x00000002 0x00000001 0x00000001 0x00000000
+ 0x00000001 0x00000003 0x00000001 0x0000000b
+ 0x00000009 0x00000181 0x00000000 0x00000060
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000001 0x00000007 0x0000000f 0x0000000f
+ 0x00000003 0x00000008 0x00000004 0x00000004
+ 0x00000002 0x000001a9 0x00000004 0x00000004
+ 0x00000000 0x00000000 0x00004282 0x007800a4
+ 0x00008000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00100220 0x0800201c 0x00000000
+ 0x77ffc004 0x01f1f008 0x00000000 0x00000007
+ 0x08000068 0x08000000 0x00000802 0x00064000
+ 0x00000025 0x00090009 0xa0f10000 0x00000000
+ 0x00000000 0x8000040b 0xe0000000 0xff00ff00 >;
+ };
+
+ timing-204000000 {
+ clock-frequency = <204000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x00010042>;
+ nvidia,emc-mode-2 = <0x00020001>;
+ nvidia,emc-mode-reset = <0x00000000>;
+ nvidia,emc-zcal-cnt-long = <0x00000013>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x0000000c
+ 0x0000001a 0x00000008 0x00000003 0x00000005
+ 0x00000004 0x00000001 0x00000006 0x00000003
+ 0x00000003 0x00000002 0x00000002 0x00000000
+ 0x00000001 0x00000003 0x00000001 0x0000000c
+ 0x0000000a 0x00000303 0x00000000 0x000000c0
+ 0x00000001 0x00000001 0x00000003 0x00000000
+ 0x00000001 0x00000007 0x0000001d 0x0000001d
+ 0x00000004 0x0000000b 0x00000005 0x00000004
+ 0x00000002 0x00000351 0x00000004 0x00000006
+ 0x00000000 0x00000000 0x00004282 0x004400a4
+ 0x00008000 0x00080000 0x00080000 0x00080000
+ 0x00080000 0x00080000 0x00080000 0x00080000
+ 0x00080000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00080000 0x00080000 0x00080000
+ 0x00080000 0x000e0220 0x0800201c 0x00000000
+ 0x77ffc004 0x01f1f008 0x00000000 0x00000007
+ 0x08000068 0x08000000 0x00000802 0x00064000
+ 0x0000004a 0x00090009 0xa0f10000 0x00000000
+ 0x00000000 0x80000713 0xe0000000 0xff00ff00 >;
+ };
+
+ timing-400000000 {
+ clock-frequency = <400000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x00010082>;
+ nvidia,emc-mode-2 = <0x00020004>;
+ nvidia,emc-mode-reset = <0x00000000>;
+ nvidia,emc-zcal-cnt-long = <0x00000024>;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000017
+ 0x00000033 0x00000010 0x00000007 0x00000007
+ 0x00000007 0x00000002 0x0000000a 0x00000007
+ 0x00000007 0x00000003 0x00000002 0x00000000
+ 0x00000003 0x00000007 0x00000004 0x0000000d
+ 0x0000000e 0x000005e9 0x00000000 0x0000017a
+ 0x00000002 0x00000002 0x00000007 0x00000000
+ 0x00000001 0x0000000c 0x00000038 0x00000038
+ 0x00000006 0x00000014 0x00000009 0x00000004
+ 0x00000002 0x00000680 0x00000000 0x00000006
+ 0x00000000 0x00000000 0x00006282 0x001d0084
+ 0x00008000 0x00034000 0x00034000 0x00034000
+ 0x00034000 0x00034000 0x00034000 0x00034000
+ 0x00034000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00038000 0x00038000 0x00038000
+ 0x00038000 0x00080220 0x0800003d 0x00000000
+ 0x77ffc004 0x01f1f408 0x00000000 0x00000007
+ 0x08000068 0x08000000 0x00000802 0x00064000
+ 0x00000090 0x000c000c 0xa0f10404 0x00000000
+ 0x00000000 0x80000ce6 0xe0000000 0xff00ff88 >;
+ };
+ };
+
+ emc-timings-1 {
+ /* ELPIDA EDB8132B2MA 8D_F */
+ nvidia,ram-code = <1>;
+
+ timing-25500000 {
+ clock-frequency = <25500000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x00010022>;
+ nvidia,emc-mode-2 = <0x00020001>;
+ nvidia,emc-mode-reset = <0x00000000>;
+ nvidia,emc-zcal-cnt-long = <0x00000009>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000001
+ 0x00000003 0x00000002 0x00000002 0x00000004
+ 0x00000004 0x00000001 0x00000005 0x00000002
+ 0x00000002 0x00000001 0x00000001 0x00000000
+ 0x00000001 0x00000003 0x00000001 0x0000000b
+ 0x0000000a 0x00000060 0x00000000 0x00000018
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000001 0x00000007 0x00000004 0x00000004
+ 0x00000003 0x00000008 0x00000004 0x00000004
+ 0x00000002 0x0000006b 0x00000004 0x00000004
+ 0x00000000 0x00000000 0x00004282 0x007800a4
+ 0x00008000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00100220 0x0800201c 0x00000000
+ 0x77ffc004 0x01f1f008 0x00000000 0x00000007
+ 0x08000068 0x08000000 0x00000802 0x00064000
+ 0x0000000a 0x00090009 0xa0f10000 0x00000000
+ 0x00000000 0x800001c5 0xe0000000 0xff00ff00 >;
+ };
+
+ timing-51000000 {
+ clock-frequency = <51000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x00010022>;
+ nvidia,emc-mode-2 = <0x00020001>;
+ nvidia,emc-mode-reset = <0x00000000>;
+ nvidia,emc-zcal-cnt-long = <0x00000009>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000003
+ 0x00000006 0x00000002 0x00000002 0x00000004
+ 0x00000004 0x00000001 0x00000005 0x00000002
+ 0x00000002 0x00000001 0x00000001 0x00000000
+ 0x00000001 0x00000003 0x00000001 0x0000000b
+ 0x0000000a 0x000000c0 0x00000000 0x00000030
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000001 0x00000007 0x00000008 0x00000008
+ 0x00000003 0x00000008 0x00000004 0x00000004
+ 0x00000002 0x000000d5 0x00000004 0x00000004
+ 0x00000000 0x00000000 0x00004282 0x007800a4
+ 0x00008000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00100220 0x0800201c 0x00000000
+ 0x77ffc004 0x01f1f008 0x00000000 0x00000007
+ 0x08000068 0x08000000 0x00000802 0x00064000
+ 0x00000013 0x00090009 0xa0f10000 0x00000000
+ 0x00000000 0x80000287 0xe0000000 0xff00ff00 >;
+ };
+
+ timing-102000000 {
+ clock-frequency = <102000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x00010022>;
+ nvidia,emc-mode-2 = <0x00020001>;
+ nvidia,emc-mode-reset = <0x00000000>;
+ nvidia,emc-zcal-cnt-long = <0x0000000a>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000006
+ 0x0000000d 0x00000004 0x00000002 0x00000004
+ 0x00000004 0x00000001 0x00000005 0x00000002
+ 0x00000002 0x00000001 0x00000001 0x00000000
+ 0x00000001 0x00000003 0x00000001 0x0000000b
+ 0x0000000a 0x00000181 0x00000000 0x00000060
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000001 0x00000007 0x0000000f 0x0000000f
+ 0x00000003 0x00000008 0x00000004 0x00000004
+ 0x00000002 0x000001a9 0x00000004 0x00000004
+ 0x00000000 0x00000000 0x00004282 0x007800a4
+ 0x00008000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00100220 0x0800201c 0x00000000
+ 0x77ffc004 0x01f1f008 0x00000000 0x00000007
+ 0x08000068 0x08000000 0x00000802 0x00064000
+ 0x00000025 0x00090009 0xa0f10000 0x00000000
+ 0x00000000 0x8000040b 0xe0000000 0xff00ff00 >;
+ };
+
+ timing-204000000 {
+ clock-frequency = <204000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x00010042>;
+ nvidia,emc-mode-2 = <0x00020001>;
+ nvidia,emc-mode-reset = <0x00000000>;
+ nvidia,emc-zcal-cnt-long = <0x00000013>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x0000000c
+ 0x0000001a 0x00000008 0x00000003 0x00000005
+ 0x00000004 0x00000001 0x00000006 0x00000003
+ 0x00000003 0x00000002 0x00000002 0x00000000
+ 0x00000001 0x00000003 0x00000001 0x0000000c
+ 0x0000000a 0x00000303 0x00000000 0x000000c0
+ 0x00000001 0x00000001 0x00000003 0x00000000
+ 0x00000001 0x00000007 0x0000001d 0x0000001d
+ 0x00000004 0x0000000b 0x00000005 0x00000004
+ 0x00000002 0x00000351 0x00000004 0x00000006
+ 0x00000000 0x00000000 0x00004282 0x004400a4
+ 0x00008000 0x00070000 0x00070000 0x00070000
+ 0x00070000 0x00070000 0x00070000 0x00070000
+ 0x00070000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00080000 0x00080000 0x00080000
+ 0x00080000 0x000e0220 0x0800201c 0x00000000
+ 0x77ffc004 0x01f1f008 0x00000000 0x00000007
+ 0x08000068 0x08000000 0x00000802 0x00064000
+ 0x0000004a 0x00090009 0xa0f10000 0x00000000
+ 0x00000000 0x80000713 0xe0000000 0xff00ff00 >;
+ };
+
+ timing-400000000 {
+ clock-frequency = <400000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x00010082>;
+ nvidia,emc-mode-2 = <0x00020004>;
+ nvidia,emc-mode-reset = <0x00000000>;
+ nvidia,emc-zcal-cnt-long = <0x00000024>;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000017
+ 0x00000033 0x00000010 0x00000007 0x00000007
+ 0x00000007 0x00000002 0x0000000a 0x00000007
+ 0x00000007 0x00000003 0x00000002 0x00000000
+ 0x00000003 0x00000007 0x00000004 0x0000000d
+ 0x0000000e 0x000005e9 0x00000000 0x0000017a
+ 0x00000002 0x00000002 0x00000007 0x00000000
+ 0x00000001 0x0000000c 0x00000038 0x00000038
+ 0x00000006 0x00000014 0x00000009 0x00000004
+ 0x00000002 0x00000680 0x00000000 0x00000004
+ 0x00000000 0x00000000 0x00006282 0x001d0084
+ 0x00008000 0x00034000 0x00034000 0x00034000
+ 0x00034000 0x00034000 0x00034000 0x00034000
+ 0x00034000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00048000 0x00048000 0x00048000
+ 0x00048000 0x00060220 0x0800003d 0x00000000
+ 0x77ffc004 0x01f1f408 0x00000000 0x00000007
+ 0x08000068 0x08000000 0x00000802 0x00064000
+ 0x00000090 0x000c000c 0xa0f10000 0x00000000
+ 0x00000000 0x80000ce6 0xe0000000 0xff00ff88 >;
+ };
+ };
+
+ emc-timings-2 {
+ /* SAMSUNG K4P8G304EB FGC2 */
+ nvidia,ram-code = <2>;
+
+ timing-25500000 {
+ clock-frequency = <25500000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x00010022>;
+ nvidia,emc-mode-2 = <0x00020001>;
+ nvidia,emc-mode-reset = <0x00000000>;
+ nvidia,emc-zcal-cnt-long = <0x00000009>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000001
+ 0x00000003 0x00000002 0x00000002 0x00000004
+ 0x00000004 0x00000001 0x00000005 0x00000002
+ 0x00000002 0x00000001 0x00000001 0x00000000
+ 0x00000001 0x00000003 0x00000001 0x0000000b
+ 0x0000000a 0x00000060 0x00000000 0x00000018
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000001 0x00000007 0x00000004 0x00000004
+ 0x00000003 0x00000008 0x00000004 0x00000004
+ 0x00000002 0x0000006b 0x00000004 0x00000004
+ 0x00000000 0x00000000 0x00004282 0x007800a4
+ 0x00008000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00100220 0x0800201c 0x00000000
+ 0x77ffc004 0x01f1f008 0x00000000 0x00000007
+ 0x08000068 0x08000000 0x00000802 0x00064000
+ 0x0000000a 0x00090009 0xa0f10000 0x00000000
+ 0x00000000 0x800001c5 0xe0000000 0xff00ff00 >;
+ };
+
+ timing-51000000 {
+ clock-frequency = <51000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x00010022>;
+ nvidia,emc-mode-2 = <0x00020001>;
+ nvidia,emc-mode-reset = <0x00000000>;
+ nvidia,emc-zcal-cnt-long = <0x00000009>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000003
+ 0x00000006 0x00000002 0x00000002 0x00000004
+ 0x00000004 0x00000001 0x00000005 0x00000002
+ 0x00000002 0x00000001 0x00000001 0x00000000
+ 0x00000001 0x00000003 0x00000001 0x0000000b
+ 0x0000000a 0x000000c0 0x00000000 0x00000030
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000001 0x00000007 0x00000008 0x00000008
+ 0x00000003 0x00000008 0x00000004 0x00000004
+ 0x00000002 0x000000d5 0x00000004 0x00000004
+ 0x00000000 0x00000000 0x00004282 0x007800a4
+ 0x00008000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00100220 0x0800201c 0x00000000
+ 0x77ffc004 0x01f1f008 0x00000000 0x00000007
+ 0x08000068 0x08000000 0x00000802 0x00064000
+ 0x00000013 0x00090009 0xa0f10000 0x00000000
+ 0x00000000 0x80000287 0xe0000000 0xff00ff00 >;
+ };
+
+ timing-102000000 {
+ clock-frequency = <102000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x00010022>;
+ nvidia,emc-mode-2 = <0x00020001>;
+ nvidia,emc-mode-reset = <0x00000000>;
+ nvidia,emc-zcal-cnt-long = <0x0000000a>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000006
+ 0x0000000d 0x00000004 0x00000002 0x00000004
+ 0x00000004 0x00000001 0x00000005 0x00000002
+ 0x00000002 0x00000001 0x00000001 0x00000000
+ 0x00000001 0x00000003 0x00000001 0x0000000b
+ 0x00000009 0x00000181 0x00000000 0x00000060
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000001 0x00000007 0x0000000f 0x0000000f
+ 0x00000003 0x00000008 0x00000004 0x00000004
+ 0x00000002 0x000001a9 0x00000004 0x00000004
+ 0x00000000 0x00000000 0x00004282 0x007800a4
+ 0x00008000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00100220 0x0800201c 0x00000000
+ 0x77ffc004 0x01f1f008 0x00000000 0x00000007
+ 0x08000068 0x08000000 0x00000802 0x00064000
+ 0x00000025 0x00090009 0xa0f10000 0x00000000
+ 0x00000000 0x8000040b 0xe0000000 0xff00ff00 >;
+ };
+
+ timing-204000000 {
+ clock-frequency = <204000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x00010042>;
+ nvidia,emc-mode-2 = <0x00020001>;
+ nvidia,emc-mode-reset = <0x00000000>;
+ nvidia,emc-zcal-cnt-long = <0x00000013>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x0000000c
+ 0x0000001a 0x00000008 0x00000003 0x00000005
+ 0x00000004 0x00000001 0x00000006 0x00000003
+ 0x00000003 0x00000002 0x00000002 0x00000000
+ 0x00000001 0x00000004 0x00000001 0x0000000c
+ 0x0000000a 0x00000303 0x00000000 0x000000c0
+ 0x00000001 0x00000001 0x00000003 0x00000000
+ 0x00000001 0x00000007 0x0000001d 0x0000001d
+ 0x00000004 0x0000000b 0x00000005 0x00000004
+ 0x00000002 0x00000351 0x00000005 0x00000004
+ 0x00000000 0x00000000 0x00004282 0x004400a4
+ 0x00008000 0x00080000 0x00080000 0x00080000
+ 0x00080000 0x00080000 0x00080000 0x00080000
+ 0x00080000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00080000 0x00080000 0x00080000
+ 0x00080000 0x000e0220 0x0800201c 0x00000000
+ 0x77ffc004 0x01f1f008 0x00000000 0x00000007
+ 0x08000068 0x08000000 0x00000802 0x00064000
+ 0x0000004a 0x00090009 0xa0f10000 0x00000000
+ 0x00000000 0x80000713 0xe0000000 0xff00ff00 >;
+ };
+
+ timing-533000000 {
+ clock-frequency = <533000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x000100c2>;
+ nvidia,emc-mode-2 = <0x00020006>;
+ nvidia,emc-mode-reset = <0x00000000>;
+ nvidia,emc-zcal-cnt-long = <0x00000030>;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x0000001f
+ 0x00000045 0x00000016 0x00000009 0x00000008
+ 0x00000009 0x00000003 0x0000000d 0x00000009
+ 0x00000009 0x00000005 0x00000003 0x00000000
+ 0x00000004 0x0000000a 0x00000006 0x0000000d
+ 0x00000010 0x000007df 0x00000000 0x000001f7
+ 0x00000003 0x00000003 0x00000009 0x00000000
+ 0x00000001 0x0000000f 0x0000004b 0x0000004b
+ 0x00000008 0x0000001b 0x0000000c 0x00000004
+ 0x00000002 0x000008aa 0x00000000 0x00000004
+ 0x00000000 0x00000000 0x00006282 0xf0120091
+ 0x00008000 0x007f8008 0x007f8008 0x007f8008
+ 0x007f8008 0x007f8008 0x007f8008 0x007f8008
+ 0x007f8008 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x0000000c 0x0000000c 0x0000000c
+ 0x0000000c 0x00080220 0x0200003d 0x00000000
+ 0x77ffc004 0x01f1f408 0x00000000 0x00000007
+ 0x08000068 0x08000000 0x00000802 0x00064000
+ 0x000000c0 0x000e000e 0xa0f10000 0x00000000
+ 0x00000000 0x800010d9 0xf0000000 0xff00ff88 >;
+ };
+ };
+
+ emc-timings-3 {
+ /* HYNIX H9TCNNN8JDMMPR NGM */
+ nvidia,ram-code = <3>;
+
+ timing-25500000 {
+ clock-frequency = <25500000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x00010022>;
+ nvidia,emc-mode-2 = <0x00020001>;
+ nvidia,emc-mode-reset = <0x00000000>;
+ nvidia,emc-zcal-cnt-long = <0x00000009>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000001
+ 0x00000003 0x00000002 0x00000002 0x00000004
+ 0x00000004 0x00000001 0x00000005 0x00000002
+ 0x00000002 0x00000001 0x00000001 0x00000000
+ 0x00000001 0x00000003 0x00000001 0x0000000b
+ 0x0000000a 0x00000060 0x00000000 0x00000018
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000001 0x00000007 0x00000004 0x00000004
+ 0x00000003 0x00000008 0x00000004 0x00000004
+ 0x00000002 0x0000006b 0x00000004 0x00000004
+ 0x00000000 0x00000000 0x00004282 0x007800a4
+ 0x00008000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00100220 0x0800201c 0x00000000
+ 0x77ffc004 0x01f1f008 0x00000000 0x00000007
+ 0x08000068 0x08000000 0x00000802 0x00064000
+ 0x0000000a 0x00090009 0xa0f10000 0x00000000
+ 0x00000000 0x800001c5 0xe0000000 0xff00ff00 >;
+ };
+
+ timing-51000000 {
+ clock-frequency = <51000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x00010022>;
+ nvidia,emc-mode-2 = <0x00020001>;
+ nvidia,emc-mode-reset = <0x00000000>;
+ nvidia,emc-zcal-cnt-long = <0x00000009>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000003
+ 0x00000006 0x00000002 0x00000002 0x00000004
+ 0x00000004 0x00000001 0x00000005 0x00000002
+ 0x00000002 0x00000001 0x00000001 0x00000000
+ 0x00000001 0x00000003 0x00000001 0x0000000b
+ 0x0000000a 0x000000c0 0x00000000 0x00000030
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000001 0x00000007 0x00000008 0x00000008
+ 0x00000003 0x00000008 0x00000004 0x00000004
+ 0x00000002 0x000000d5 0x00000004 0x00000004
+ 0x00000000 0x00000000 0x00004282 0x007800a4
+ 0x00008000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00100220 0x0800201c 0x00000000
+ 0x77ffc004 0x01f1f008 0x00000000 0x00000007
+ 0x08000068 0x08000000 0x00000802 0x00064000
+ 0x00000013 0x00090009 0xa0f10000 0x00000000
+ 0x00000000 0x80000287 0xe0000000 0xff00ff00 >;
+ };
+
+ timing-102000000 {
+ clock-frequency = <102000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x00010022>;
+ nvidia,emc-mode-2 = <0x00020001>;
+ nvidia,emc-mode-reset = <0x00000000>;
+ nvidia,emc-zcal-cnt-long = <0x0000000a>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x00000006
+ 0x0000000d 0x00000004 0x00000002 0x00000004
+ 0x00000004 0x00000001 0x00000005 0x00000002
+ 0x00000002 0x00000001 0x00000001 0x00000000
+ 0x00000001 0x00000003 0x00000001 0x0000000b
+ 0x0000000a 0x00000181 0x00000000 0x00000060
+ 0x00000001 0x00000001 0x00000002 0x00000000
+ 0x00000001 0x00000007 0x0000000f 0x0000000f
+ 0x00000003 0x00000008 0x00000004 0x00000004
+ 0x00000002 0x000001a9 0x00000004 0x00000004
+ 0x00000000 0x00000000 0x00004282 0x007800a4
+ 0x00008000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x000fc000 0x000fc000 0x000fc000
+ 0x000fc000 0x00100220 0x0800201c 0x00000000
+ 0x77ffc004 0x01f1f008 0x00000000 0x00000007
+ 0x08000068 0x08000000 0x00000802 0x00064000
+ 0x00000025 0x00090009 0xa0f10000 0x00000000
+ 0x00000000 0x8000040b 0xe0000000 0xff00ff00 >;
+ };
+
+ timing-204000000 {
+ clock-frequency = <204000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x00010042>;
+ nvidia,emc-mode-2 = <0x00020001>;
+ nvidia,emc-mode-reset = <0x00000000>;
+ nvidia,emc-zcal-cnt-long = <0x00000013>;
+ nvidia,emc-cfg-dyn-self-ref;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x0000000c
+ 0x0000001a 0x00000008 0x00000003 0x00000005
+ 0x00000004 0x00000001 0x00000006 0x00000003
+ 0x00000003 0x00000002 0x00000002 0x00000000
+ 0x00000001 0x00000003 0x00000001 0x0000000c
+ 0x0000000b 0x00000303 0x00000000 0x000000c0
+ 0x00000001 0x00000001 0x00000003 0x00000000
+ 0x00000001 0x00000007 0x0000001d 0x0000001d
+ 0x00000004 0x0000000b 0x00000005 0x00000004
+ 0x00000002 0x00000351 0x00000004 0x00000006
+ 0x00000000 0x00000000 0x00004282 0x004400a4
+ 0x00008000 0x00072000 0x00072000 0x00072000
+ 0x00072000 0x00072000 0x00072000 0x00072000
+ 0x00072000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00080000 0x00080000 0x00080000
+ 0x00080000 0x000e0220 0x0800201c 0x00000000
+ 0x77ffc004 0x01f1f008 0x00000000 0x00000007
+ 0x08000068 0x08000000 0x00000802 0x00064000
+ 0x0000004a 0x00090009 0xa0f10000 0x00000000
+ 0x00000000 0x80000713 0xd0000000 0xff00ff00 >;
+ };
+
+ timing-533000000 {
+ clock-frequency = <533000000>;
+
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-mode-1 = <0x000100c2>;
+ nvidia,emc-mode-2 = <0x00020006>;
+ nvidia,emc-mode-reset = <0x00000000>;
+ nvidia,emc-zcal-cnt-long = <0x00000030>;
+ nvidia,emc-cfg-periodic-qrst;
+
+ nvidia,emc-configuration = < 0x0000001f
+ 0x00000045 0x00000016 0x00000009 0x00000008
+ 0x00000009 0x00000003 0x0000000d 0x00000009
+ 0x00000009 0x00000005 0x00000003 0x00000000
+ 0x00000004 0x00000009 0x00000006 0x0000000d
+ 0x00000010 0x000007df 0x00000000 0x000001f7
+ 0x00000003 0x00000003 0x00000009 0x00000000
+ 0x00000001 0x0000000f 0x0000004b 0x0000004b
+ 0x00000008 0x0000001b 0x0000000c 0x00000004
+ 0x00000002 0x000008aa 0x00000000 0x00000006
+ 0x00000000 0x00000000 0x00006282 0xf0120091
+ 0x00008000 0x0000000a 0x0000000a 0x0000000a
+ 0x0000000a 0x0000000a 0x0000000a 0x0000000a
+ 0x0000000a 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x00000000 0x00000000 0x00000000
+ 0x00000000 0x0000000c 0x0000000c 0x0000000c
+ 0x0000000c 0x000a0220 0x0800003d 0x00000000
+ 0x77ffc004 0x01f1f408 0x00000000 0x00000007
+ 0x08000068 0x08000000 0x00000802 0x00064000
+ 0x000000c0 0x000e000e 0xa0f10000 0x00000000
+ 0x00000000 0x800010d9 0xe0000000 0xff00ff88 >;
+ };
+ };
+ };
+
+ hda@70030000 {
+ status = "okay";
+ };
+
+ ahub@70080000 {
+ i2s@70080400 { /* i2s1 */
+ status = "okay";
+ };
+
+ /* BT SCO */
+ i2s@70080600 { /* i2s3 */
+ status = "okay";
+ };
+ };
+
+ sdmmc1: mmc@78000000 {
+ status = "okay";
+
+ cd-gpios = <&gpio TEGRA_GPIO(I, 5) GPIO_ACTIVE_LOW>;
+ bus-width = <4>;
+
+ vmmc-supply = <&vdd_usd>; /* ldo2 */
+ vqmmc-supply = <&vddio_usd>; /* ldo3 */
+ };
+
+ brcm_wifi_pwrseq: wifi-pwrseq {
+ compatible = "mmc-pwrseq-simple";
+
+ clocks = <&tegra_pmc TEGRA_PMC_CLK_BLINK>;
+ clock-names = "ext_clock";
+
+ reset-gpios = <&gpio TEGRA_GPIO(D, 3) GPIO_ACTIVE_LOW>;
+ post-power-on-delay-ms = <300>;
+ power-off-delay-us = <300>;
+ };
+
+ sdmmc3: mmc@78000400 {
+ status = "okay";
+
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ assigned-clocks = <&tegra_car TEGRA30_CLK_SDMMC3>;
+ assigned-clock-parents = <&tegra_car TEGRA30_CLK_PLL_C>;
+ assigned-clock-rates = <50000000>;
+
+ max-frequency = <50000000>;
+ keep-power-in-suspend;
+ bus-width = <4>;
+ non-removable;
+
+ mmc-pwrseq = <&brcm_wifi_pwrseq>;
+ vmmc-supply = <&vdd_3v3_sys>;
+ vqmmc-supply = <&vdd_1v8_vio>;
+
+ /* Azurewave AW-AH663 BCM4330B1 */
+ wifi@1 {
+ compatible = "brcm,bcm4329-fmac";
+ reg = <1>;
+
+ interrupt-parent = <&gpio>;
+ interrupts = <TEGRA_GPIO(O, 4) IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "host-wake";
+ };
+ };
+
+ sdmmc4: mmc@78000600 {
+ status = "okay";
+ bus-width = <8>;
+ vmmc-supply = <&vcore_emmc>;
+ vqmmc-supply = <&vdd_1v8_vio>;
+ non-removable;
+ };
+
+ usb@7d000000 {
+ compatible = "nvidia,tegra30-udc";
+ status = "okay";
+ dr_mode = "otg";
+ vbus-supply = <&vdd_vbus_usb1>;
+ };
+
+ usb-phy@7d000000 {
+ status = "okay";
+ dr_mode = "otg";
+ nvidia,hssync-start-delay = <0>;
+ nvidia,xcvr-lsfslew = <2>;
+ nvidia,xcvr-lsrslew = <2>;
+ };
+
+ usb@7d008000 {
+ status = "okay";
+ };
+
+ usb-phy@7d008000 {
+ status = "okay";
+ vbus-supply = <&vdd_vbus_usb3>;
+ };
+
+ mains: ac-adapter-detect {
+ compatible = "gpio-charger";
+ charger-type = "mains";
+ gpios = <&gpio TEGRA_GPIO(V, 1) GPIO_ACTIVE_HIGH>;
+ };
+
+ backlight: backlight {
+ compatible = "pwm-backlight";
+
+ enable-gpios = <&gpio TEGRA_GPIO(H, 2) GPIO_ACTIVE_HIGH>;
+ power-supply = <&vdd_5v0_bl>;
+ pwms = <&pwm 0 5000000>;
+
+ brightness-levels = <1 255>;
+ num-interpolated-steps = <254>;
+ default-brightness-level = <15>;
+ };
+
+ /* PMIC has a built-in 32KHz oscillator which is used by PMC */
+ clk32k_in: clock-32k {
+ compatible = "fixed-clock";
+ #clock-cells = <0>;
+ clock-frequency = <32768>;
+ clock-output-names = "pmic-oscillator";
+ };
+
+ cpus {
+ cpu0: cpu@0 {
+ cpu-supply = <&vdd_cpu>;
+ operating-points-v2 = <&cpu0_opp_table>;
+ #cooling-cells = <2>;
+ };
+ cpu1: cpu@1 {
+ cpu-supply = <&vdd_cpu>;
+ operating-points-v2 = <&cpu0_opp_table>;
+ #cooling-cells = <2>;
+ };
+ cpu2: cpu@2 {
+ cpu-supply = <&vdd_cpu>;
+ operating-points-v2 = <&cpu0_opp_table>;
+ #cooling-cells = <2>;
+ };
+ cpu3: cpu@3 {
+ cpu-supply = <&vdd_cpu>;
+ operating-points-v2 = <&cpu0_opp_table>;
+ #cooling-cells = <2>;
+ };
+ };
+
+ display-panel {
+ compatible = "panel-lvds";
+
+ width-mm = <217>;
+ height-mm = <136>;
+
+ data-mapping = "jeida-24";
+
+ panel-timing {
+ /* 1280x800@60Hz */
+ clock-frequency = <68000000>;
+ hactive = <1280>;
+ vactive = <800>;
+ hfront-porch = <48>;
+ hback-porch = <18>;
+ hsync-len = <30>;
+ vsync-len = <5>;
+ vfront-porch = <3>;
+ vback-porch = <12>;
+ };
+ };
+
+ extcon-keys {
+ compatible = "gpio-keys";
+ interrupt-parent = <&gpio>;
+
+ dock-insert {
+ label = "Chagall Dock";
+ gpios = <&gpio TEGRA_GPIO(S, 4) GPIO_ACTIVE_LOW>;
+ linux,input-type = <EV_SW>;
+ linux,code = <SW_DOCK>;
+ debounce-interval = <10>;
+ wakeup-event-action = <EV_ACT_ASSERTED>;
+ wakeup-source;
+ };
+
+ lineout-detect {
+ label = "Audio dock line-out detect";
+ gpios = <&gpio TEGRA_GPIO(S, 3) GPIO_ACTIVE_LOW>;
+ linux,input-type = <EV_SW>;
+ linux,code = <SW_LINEOUT_INSERT>;
+ debounce-interval = <10>;
+ wakeup-event-action = <EV_ACT_ASSERTED>;
+ wakeup-source;
+ };
+ };
+
+ firmware {
+ trusted-foundations {
+ compatible = "tlm,trusted-foundations";
+ tlm,version-major = <2>;
+ tlm,version-minor = <8>;
+ };
+ };
+
+ gpio-keys {
+ compatible = "gpio-keys";
+ interrupt-parent = <&gpio>;
+
+ power {
+ label = "Power";
+ gpios = <&gpio TEGRA_GPIO(V, 0) GPIO_ACTIVE_LOW>;
+ linux,code = <KEY_POWER>;
+ debounce-interval = <10>;
+ wakeup-event-action = <EV_ACT_ASSERTED>;
+ wakeup-source;
+ };
+
+ volume-up {
+ label = "Volume Up";
+ gpios = <&gpio TEGRA_GPIO(Q, 0) GPIO_ACTIVE_LOW>;
+ linux,code = <KEY_VOLUMEUP>;
+ debounce-interval = <10>;
+ wakeup-event-action = <EV_ACT_ASSERTED>;
+ wakeup-source;
+ };
+
+ volume-down {
+ label = "Volume Down";
+ gpios = <&gpio TEGRA_GPIO(Q, 1) GPIO_ACTIVE_LOW>;
+ linux,code = <KEY_VOLUMEDOWN>;
+ debounce-interval = <10>;
+ wakeup-event-action = <EV_ACT_ASSERTED>;
+ wakeup-source;
+ };
+ };
+
+ haptic-feedback {
+ compatible = "gpio-vibrator";
+ enable-gpios = <&gpio TEGRA_GPIO(U, 4) GPIO_ACTIVE_HIGH>;
+ vcc-supply = <&vdd_3v3_sys>;
+ };
+
+ sound {
+ compatible = "pegatron,tegra-audio-wm8903-chagall",
+ "nvidia,tegra-audio-wm8903";
+ nvidia,model = "Pegatron Chagall WM8903";
+
+ nvidia,audio-routing =
+ "Headphone Jack", "HPOUTR",
+ "Headphone Jack", "HPOUTL",
+ "Int Spk", "ROP",
+ "Int Spk", "RON",
+ "Int Spk", "LOP",
+ "Int Spk", "LON",
+ "IN1R", "Mic Jack",
+ "DMICDAT", "Int Mic";
+
+ nvidia,i2s-controller = <&tegra_i2s1>;
+ nvidia,audio-codec = <&wm8903>;
+
+ nvidia,spkr-en-gpios = <&wm8903 2 GPIO_ACTIVE_HIGH>;
+ nvidia,hp-det-gpios = <&gpio TEGRA_GPIO(W, 2) GPIO_ACTIVE_LOW>;
+ nvidia,headset;
+
+ clocks = <&tegra_car TEGRA30_CLK_PLL_A>,
+ <&tegra_car TEGRA30_CLK_PLL_A_OUT0>,
+ <&tegra_pmc TEGRA_PMC_CLK_OUT_1>;
+ clock-names = "pll_a", "pll_a_out0", "mclk";
+
+ assigned-clocks = <&tegra_car TEGRA30_CLK_EXTERN1>,
+ <&tegra_pmc TEGRA_PMC_CLK_OUT_1>;
+
+ assigned-clock-parents = <&tegra_car TEGRA30_CLK_PLL_A_OUT0>,
+ <&tegra_car TEGRA30_CLK_EXTERN1>;
+ };
+
+ thermal-zones {
+ /*
+ * NCT72 has two sensors:
+ *
+ * 0: internal that monitors ambient/skin temperature
+ * 1: external that is connected to the CPU's diode
+ *
+ * Ideally we should use userspace thermal governor,
+ * but it's a much more complex solution. The "skin"
+ * zone exists as a simpler solution which prevents
+ * Chagall from getting too hot from a user's tactile
+ * perspective. The CPU zone is intended to protect
+ * silicon from damage.
+ */
+
+ skin-thermal {
+ polling-delay-passive = <1000>; /* milliseconds */
+ polling-delay = <5000>; /* milliseconds */
+
+ thermal-sensors = <&nct72 0>;
+
+ trips {
+ trip0: skin-alert {
+ /* throttle at 57C until temperature drops to 56.8C */
+ temperature = <57000>;
+ hysteresis = <200>;
+ type = "passive";
+ };
+
+ trip1: skin-crit {
+ /* shut down at 65C */
+ temperature = <65000>;
+ hysteresis = <2000>;
+ type = "critical";
+ };
+ };
+
+ cooling-maps {
+ map0 {
+ trip = <&trip0>;
+ cooling-device = <&cpu0 THERMAL_NO_LIMIT THERMAL_NO_LIMIT>,
+ <&cpu1 THERMAL_NO_LIMIT THERMAL_NO_LIMIT>,
+ <&cpu2 THERMAL_NO_LIMIT THERMAL_NO_LIMIT>,
+ <&cpu3 THERMAL_NO_LIMIT THERMAL_NO_LIMIT>,
+ <&actmon THERMAL_NO_LIMIT
+ THERMAL_NO_LIMIT>;
+ };
+ };
+ };
+
+ cpu-thermal {
+ polling-delay-passive = <1000>; /* milliseconds */
+ polling-delay = <5000>; /* milliseconds */
+
+ thermal-sensors = <&nct72 1>;
+
+ trips {
+ trip2: cpu-alert {
+ /* throttle at 85C until temperature drops to 84.8C */
+ temperature = <85000>;
+ hysteresis = <200>;
+ type = "passive";
+ };
+
+ trip3: cpu-crit {
+ /* shut down at 90C */
+ temperature = <90000>;
+ hysteresis = <2000>;
+ type = "critical";
+ };
+ };
+
+ cooling-maps {
+ map1 {
+ trip = <&trip2>;
+ cooling-device = <&cpu0 THERMAL_NO_LIMIT THERMAL_NO_LIMIT>,
+ <&cpu1 THERMAL_NO_LIMIT THERMAL_NO_LIMIT>,
+ <&cpu2 THERMAL_NO_LIMIT THERMAL_NO_LIMIT>,
+ <&cpu3 THERMAL_NO_LIMIT THERMAL_NO_LIMIT>,
+ <&actmon THERMAL_NO_LIMIT
+ THERMAL_NO_LIMIT>;
+ };
+ };
+ };
+ };
+};
+
+&emc_icc_dvfs_opp_table {
+ /delete-node/ opp-625000000-1200;
+ /delete-node/ opp-625000000-1250;
+ /delete-node/ opp-667000000-1200;
+ /delete-node/ opp-750000000-1300;
+ /delete-node/ opp-800000000-1300;
+ /delete-node/ opp-900000000-1350;
+};
+
+&emc_bw_dfs_opp_table {
+ /delete-node/ opp-625000000;
+ /delete-node/ opp-667000000;
+ /delete-node/ opp-750000000;
+ /delete-node/ opp-800000000;
+ /delete-node/ opp-900000000;
+};
diff --git a/arch/arm/boot/dts/tegra30-peripherals-opp.dtsi b/arch/arm/boot/dts/tegra30-peripherals-opp.dtsi
index 2c97803..d100a1a 100644
--- a/arch/arm/boot/dts/tegra30-peripherals-opp.dtsi
+++ b/arch/arm/boot/dts/tegra30-peripherals-opp.dtsi
@@ -1,386 +1,1628 @@
// SPDX-License-Identifier: GPL-2.0
/ {
- emc_icc_dvfs_opp_table: emc-dvfs-opp-table {
+ core_opp_table: opp-table-core {
compatible = "operating-points-v2";
+ opp-shared;
- opp@12750000,950 {
+ core_opp_950: opp-950000 {
opp-microvolt = <950000 950000 1350000>;
- opp-hz = /bits/ 64 <12750000>;
- opp-supported-hw = <0x0006>;
+ opp-level = <950000>;
};
- opp@12750000,1000 {
+ core_opp_1000: opp-1000000 {
opp-microvolt = <1000000 1000000 1350000>;
- opp-hz = /bits/ 64 <12750000>;
- opp-supported-hw = <0x0001>;
+ opp-level = <1000000>;
};
- opp@12750000,1250 {
- opp-microvolt = <1250000 1250000 1350000>;
- opp-hz = /bits/ 64 <12750000>;
- opp-supported-hw = <0x0008>;
+ core_opp_1050: opp-1050000 {
+ opp-microvolt = <1050000 1050000 1350000>;
+ opp-level = <1050000>;
};
- opp@25500000,950 {
- opp-microvolt = <950000 950000 1350000>;
- opp-hz = /bits/ 64 <25500000>;
- opp-supported-hw = <0x0006>;
+ core_opp_1100: opp-1100000 {
+ opp-microvolt = <1100000 1100000 1350000>;
+ opp-level = <1100000>;
};
- opp@25500000,1000 {
- opp-microvolt = <1000000 1000000 1350000>;
- opp-hz = /bits/ 64 <25500000>;
- opp-supported-hw = <0x0001>;
+ core_opp_1150: opp-1150000 {
+ opp-microvolt = <1150000 1150000 1350000>;
+ opp-level = <1150000>;
};
- opp@25500000,1250 {
- opp-microvolt = <1250000 1250000 1350000>;
- opp-hz = /bits/ 64 <25500000>;
- opp-supported-hw = <0x0008>;
- };
-
- opp@27000000,950 {
- opp-microvolt = <950000 950000 1350000>;
- opp-hz = /bits/ 64 <27000000>;
- opp-supported-hw = <0x0006>;
- };
-
- opp@27000000,1000 {
- opp-microvolt = <1000000 1000000 1350000>;
- opp-hz = /bits/ 64 <27000000>;
- opp-supported-hw = <0x0001>;
- };
-
- opp@27000000,1250 {
- opp-microvolt = <1250000 1250000 1350000>;
- opp-hz = /bits/ 64 <27000000>;
- opp-supported-hw = <0x0008>;
- };
-
- opp@51000000,950 {
- opp-microvolt = <950000 950000 1350000>;
- opp-hz = /bits/ 64 <51000000>;
- opp-supported-hw = <0x0006>;
- };
-
- opp@51000000,1000 {
- opp-microvolt = <1000000 1000000 1350000>;
- opp-hz = /bits/ 64 <51000000>;
- opp-supported-hw = <0x0001>;
- };
-
- opp@51000000,1250 {
- opp-microvolt = <1250000 1250000 1350000>;
- opp-hz = /bits/ 64 <51000000>;
- opp-supported-hw = <0x0008>;
- };
-
- opp@54000000,950 {
- opp-microvolt = <950000 950000 1350000>;
- opp-hz = /bits/ 64 <54000000>;
- opp-supported-hw = <0x0006>;
- };
-
- opp@54000000,1000 {
- opp-microvolt = <1000000 1000000 1350000>;
- opp-hz = /bits/ 64 <54000000>;
- opp-supported-hw = <0x0001>;
- };
-
- opp@54000000,1250 {
- opp-microvolt = <1250000 1250000 1350000>;
- opp-hz = /bits/ 64 <54000000>;
- opp-supported-hw = <0x0008>;
- };
-
- opp@102000000,950 {
- opp-microvolt = <950000 950000 1350000>;
- opp-hz = /bits/ 64 <102000000>;
- opp-supported-hw = <0x0006>;
- };
-
- opp@102000000,1000 {
- opp-microvolt = <1000000 1000000 1350000>;
- opp-hz = /bits/ 64 <102000000>;
- opp-supported-hw = <0x0001>;
- };
-
- opp@102000000,1250 {
- opp-microvolt = <1250000 1250000 1350000>;
- opp-hz = /bits/ 64 <102000000>;
- opp-supported-hw = <0x0008>;
- };
-
- opp@108000000,1000 {
- opp-microvolt = <1000000 1000000 1350000>;
- opp-hz = /bits/ 64 <108000000>;
- opp-supported-hw = <0x0007>;
- };
-
- opp@108000000,1250 {
- opp-microvolt = <1250000 1250000 1350000>;
- opp-hz = /bits/ 64 <108000000>;
- opp-supported-hw = <0x0008>;
- };
-
- opp@204000000,1000 {
- opp-microvolt = <1000000 1000000 1350000>;
- opp-hz = /bits/ 64 <204000000>;
- opp-supported-hw = <0x0007>;
- opp-suspend;
- };
-
- opp@204000000,1250 {
- opp-microvolt = <1250000 1250000 1350000>;
- opp-hz = /bits/ 64 <204000000>;
- opp-supported-hw = <0x0008>;
- opp-suspend;
- };
-
- opp@333500000,1000 {
- opp-microvolt = <1000000 1000000 1350000>;
- opp-hz = /bits/ 64 <333500000>;
- opp-supported-hw = <0x0006>;
- };
-
- opp@333500000,1200 {
+ core_opp_1200: opp-1200000 {
opp-microvolt = <1200000 1200000 1350000>;
- opp-hz = /bits/ 64 <333500000>;
- opp-supported-hw = <0x0001>;
+ opp-level = <1200000>;
};
- opp@333500000,1250 {
+ core_opp_1250: opp-1250000 {
opp-microvolt = <1250000 1250000 1350000>;
- opp-hz = /bits/ 64 <333500000>;
- opp-supported-hw = <0x0008>;
+ opp-level = <1250000>;
};
- opp@375000000,1000 {
- opp-microvolt = <1000000 1000000 1350000>;
- opp-hz = /bits/ 64 <375000000>;
- opp-supported-hw = <0x0006>;
- };
-
- opp@375000000,1200 {
- opp-microvolt = <1200000 1200000 1350000>;
- opp-hz = /bits/ 64 <375000000>;
- opp-supported-hw = <0x0001>;
- };
-
- opp@375000000,1250 {
- opp-microvolt = <1250000 1250000 1350000>;
- opp-hz = /bits/ 64 <375000000>;
- opp-supported-hw = <0x0008>;
- };
-
- opp@400000000,1000 {
- opp-microvolt = <1000000 1000000 1350000>;
- opp-hz = /bits/ 64 <400000000>;
- opp-supported-hw = <0x0006>;
- };
-
- opp@400000000,1200 {
- opp-microvolt = <1200000 1200000 1350000>;
- opp-hz = /bits/ 64 <400000000>;
- opp-supported-hw = <0x0001>;
- };
-
- opp@400000000,1250 {
- opp-microvolt = <1250000 1250000 1350000>;
- opp-hz = /bits/ 64 <400000000>;
- opp-supported-hw = <0x0008>;
- };
-
- opp@416000000,1200 {
- opp-microvolt = <1200000 1200000 1350000>;
- opp-hz = /bits/ 64 <416000000>;
- opp-supported-hw = <0x0007>;
- };
-
- opp@416000000,1250 {
- opp-microvolt = <1250000 1250000 1350000>;
- opp-hz = /bits/ 64 <416000000>;
- opp-supported-hw = <0x0008>;
- };
-
- opp@450000000,1200 {
- opp-microvolt = <1200000 1200000 1350000>;
- opp-hz = /bits/ 64 <450000000>;
- opp-supported-hw = <0x0007>;
- };
-
- opp@450000000,1250 {
- opp-microvolt = <1250000 1250000 1350000>;
- opp-hz = /bits/ 64 <450000000>;
- opp-supported-hw = <0x0008>;
- };
-
- opp@533000000,1200 {
- opp-microvolt = <1200000 1200000 1350000>;
- opp-hz = /bits/ 64 <533000000>;
- opp-supported-hw = <0x0007>;
- };
-
- opp@533000000,1250 {
- opp-microvolt = <1250000 1250000 1350000>;
- opp-hz = /bits/ 64 <533000000>;
- opp-supported-hw = <0x0008>;
- };
-
- opp@625000000,1200 {
- opp-microvolt = <1200000 1200000 1350000>;
- opp-hz = /bits/ 64 <625000000>;
- opp-supported-hw = <0x0006>;
- };
-
- opp@625000000,1250 {
- opp-microvolt = <1250000 1250000 1350000>;
- opp-hz = /bits/ 64 <625000000>;
- opp-supported-hw = <0x0008>;
- };
-
- opp@667000000,1200 {
- opp-microvolt = <1200000 1200000 1350000>;
- opp-hz = /bits/ 64 <667000000>;
- opp-supported-hw = <0x0006>;
- };
-
- opp@750000000,1300 {
+ core_opp_1300: opp-1300000 {
opp-microvolt = <1300000 1300000 1350000>;
- opp-hz = /bits/ 64 <750000000>;
- opp-supported-hw = <0x0004>;
+ opp-level = <1300000>;
};
- opp@800000000,1300 {
- opp-microvolt = <1300000 1300000 1350000>;
- opp-hz = /bits/ 64 <800000000>;
- opp-supported-hw = <0x0004>;
- };
-
- opp@900000000,1350 {
+ core_opp_1350: opp-1350000 {
opp-microvolt = <1350000 1350000 1350000>;
- opp-hz = /bits/ 64 <900000000>;
- opp-supported-hw = <0x0004>;
+ opp-level = <1350000>;
};
};
- emc_bw_dfs_opp_table: emc-bandwidth-opp-table {
+ emc_icc_dvfs_opp_table: opp-table-emc {
compatible = "operating-points-v2";
- opp@12750000 {
+ opp-12750000-950 {
+ opp-microvolt = <950000 950000 1350000>;
+ opp-hz = /bits/ 64 <12750000>;
+ opp-supported-hw = <0x0006>;
+ required-opps = <&core_opp_950>;
+ };
+
+ opp-12750000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <12750000>;
+ opp-supported-hw = <0x0001>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-12750000-1250 {
+ opp-microvolt = <1250000 1250000 1350000>;
+ opp-hz = /bits/ 64 <12750000>;
+ opp-supported-hw = <0x0008>;
+ required-opps = <&core_opp_1250>;
+ };
+
+ opp-25500000-950 {
+ opp-microvolt = <950000 950000 1350000>;
+ opp-hz = /bits/ 64 <25500000>;
+ opp-supported-hw = <0x0006>;
+ required-opps = <&core_opp_950>;
+ };
+
+ opp-25500000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <25500000>;
+ opp-supported-hw = <0x0001>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-25500000-1250 {
+ opp-microvolt = <1250000 1250000 1350000>;
+ opp-hz = /bits/ 64 <25500000>;
+ opp-supported-hw = <0x0008>;
+ required-opps = <&core_opp_1250>;
+ };
+
+ opp-27000000-950 {
+ opp-microvolt = <950000 950000 1350000>;
+ opp-hz = /bits/ 64 <27000000>;
+ opp-supported-hw = <0x0006>;
+ required-opps = <&core_opp_950>;
+ };
+
+ opp-27000000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <27000000>;
+ opp-supported-hw = <0x0001>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-27000000-1250 {
+ opp-microvolt = <1250000 1250000 1350000>;
+ opp-hz = /bits/ 64 <27000000>;
+ opp-supported-hw = <0x0008>;
+ required-opps = <&core_opp_1250>;
+ };
+
+ opp-51000000-950 {
+ opp-microvolt = <950000 950000 1350000>;
+ opp-hz = /bits/ 64 <51000000>;
+ opp-supported-hw = <0x0006>;
+ required-opps = <&core_opp_950>;
+ };
+
+ opp-51000000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <51000000>;
+ opp-supported-hw = <0x0001>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-51000000-1250 {
+ opp-microvolt = <1250000 1250000 1350000>;
+ opp-hz = /bits/ 64 <51000000>;
+ opp-supported-hw = <0x0008>;
+ required-opps = <&core_opp_1250>;
+ };
+
+ opp-54000000-950 {
+ opp-microvolt = <950000 950000 1350000>;
+ opp-hz = /bits/ 64 <54000000>;
+ opp-supported-hw = <0x0006>;
+ required-opps = <&core_opp_950>;
+ };
+
+ opp-54000000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <54000000>;
+ opp-supported-hw = <0x0001>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-54000000-1250 {
+ opp-microvolt = <1250000 1250000 1350000>;
+ opp-hz = /bits/ 64 <54000000>;
+ opp-supported-hw = <0x0008>;
+ required-opps = <&core_opp_1250>;
+ };
+
+ opp-102000000-950 {
+ opp-microvolt = <950000 950000 1350000>;
+ opp-hz = /bits/ 64 <102000000>;
+ opp-supported-hw = <0x0006>;
+ required-opps = <&core_opp_950>;
+ };
+
+ opp-102000000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <102000000>;
+ opp-supported-hw = <0x0001>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-102000000-1250 {
+ opp-microvolt = <1250000 1250000 1350000>;
+ opp-hz = /bits/ 64 <102000000>;
+ opp-supported-hw = <0x0008>;
+ required-opps = <&core_opp_1250>;
+ };
+
+ opp-108000000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <108000000>;
+ opp-supported-hw = <0x0007>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-108000000-1250 {
+ opp-microvolt = <1250000 1250000 1350000>;
+ opp-hz = /bits/ 64 <108000000>;
+ opp-supported-hw = <0x0008>;
+ required-opps = <&core_opp_1250>;
+ };
+
+ opp-204000000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <204000000>;
+ opp-supported-hw = <0x0007>;
+ required-opps = <&core_opp_1000>;
+ opp-suspend;
+ };
+
+ opp-204000000-1250 {
+ opp-microvolt = <1250000 1250000 1350000>;
+ opp-hz = /bits/ 64 <204000000>;
+ opp-supported-hw = <0x0008>;
+ required-opps = <&core_opp_1250>;
+ opp-suspend;
+ };
+
+ opp-333500000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <333500000>;
+ opp-supported-hw = <0x0006>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-333500000-1200 {
+ opp-microvolt = <1200000 1200000 1350000>;
+ opp-hz = /bits/ 64 <333500000>;
+ opp-supported-hw = <0x0001>;
+ required-opps = <&core_opp_1200>;
+ };
+
+ opp-333500000-1250 {
+ opp-microvolt = <1250000 1250000 1350000>;
+ opp-hz = /bits/ 64 <333500000>;
+ opp-supported-hw = <0x0008>;
+ required-opps = <&core_opp_1250>;
+ };
+
+ opp-375000000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <375000000>;
+ opp-supported-hw = <0x0006>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-375000000-1200 {
+ opp-microvolt = <1200000 1200000 1350000>;
+ opp-hz = /bits/ 64 <375000000>;
+ opp-supported-hw = <0x0001>;
+ required-opps = <&core_opp_1200>;
+ };
+
+ opp-375000000-1250 {
+ opp-microvolt = <1250000 1250000 1350000>;
+ opp-hz = /bits/ 64 <375000000>;
+ opp-supported-hw = <0x0008>;
+ required-opps = <&core_opp_1250>;
+ };
+
+ opp-400000000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <400000000>;
+ opp-supported-hw = <0x0006>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-400000000-1200 {
+ opp-microvolt = <1200000 1200000 1350000>;
+ opp-hz = /bits/ 64 <400000000>;
+ opp-supported-hw = <0x0001>;
+ required-opps = <&core_opp_1200>;
+ };
+
+ opp-400000000-1250 {
+ opp-microvolt = <1250000 1250000 1350000>;
+ opp-hz = /bits/ 64 <400000000>;
+ opp-supported-hw = <0x0008>;
+ required-opps = <&core_opp_1250>;
+ };
+
+ opp-416000000-1200 {
+ opp-microvolt = <1200000 1200000 1350000>;
+ opp-hz = /bits/ 64 <416000000>;
+ opp-supported-hw = <0x0007>;
+ required-opps = <&core_opp_1200>;
+ };
+
+ opp-416000000-1250 {
+ opp-microvolt = <1250000 1250000 1350000>;
+ opp-hz = /bits/ 64 <416000000>;
+ opp-supported-hw = <0x0008>;
+ required-opps = <&core_opp_1250>;
+ };
+
+ opp-450000000-1200 {
+ opp-microvolt = <1200000 1200000 1350000>;
+ opp-hz = /bits/ 64 <450000000>;
+ opp-supported-hw = <0x0007>;
+ required-opps = <&core_opp_1200>;
+ };
+
+ opp-450000000-1250 {
+ opp-microvolt = <1250000 1250000 1350000>;
+ opp-hz = /bits/ 64 <450000000>;
+ opp-supported-hw = <0x0008>;
+ required-opps = <&core_opp_1250>;
+ };
+
+ opp-500000000-1200 {
+ opp-microvolt = <1200000 1200000 1350000>;
+ opp-hz = /bits/ 64 <500000000>;
+ opp-supported-hw = <0x0007>;
+ required-opps = <&core_opp_1200>;
+ };
+
+ opp-500000000-1250 {
+ opp-microvolt = <1250000 1250000 1350000>;
+ opp-hz = /bits/ 64 <500000000>;
+ opp-supported-hw = <0x0008>;
+ required-opps = <&core_opp_1250>;
+ };
+
+ opp-533000000-1200 {
+ opp-microvolt = <1200000 1200000 1350000>;
+ opp-hz = /bits/ 64 <533000000>;
+ opp-supported-hw = <0x0007>;
+ required-opps = <&core_opp_1200>;
+ };
+
+ opp-533000000-1250 {
+ opp-microvolt = <1250000 1250000 1350000>;
+ opp-hz = /bits/ 64 <533000000>;
+ opp-supported-hw = <0x0008>;
+ required-opps = <&core_opp_1250>;
+ };
+
+ opp-625000000-1200 {
+ opp-microvolt = <1200000 1200000 1350000>;
+ opp-hz = /bits/ 64 <625000000>;
+ opp-supported-hw = <0x0006>;
+ required-opps = <&core_opp_1200>;
+ };
+
+ opp-625000000-1250 {
+ opp-microvolt = <1250000 1250000 1350000>;
+ opp-hz = /bits/ 64 <625000000>;
+ opp-supported-hw = <0x0008>;
+ required-opps = <&core_opp_1250>;
+ };
+
+ opp-667000000-1200 {
+ opp-microvolt = <1200000 1200000 1350000>;
+ opp-hz = /bits/ 64 <667000000>;
+ opp-supported-hw = <0x0006>;
+ required-opps = <&core_opp_1200>;
+ };
+
+ opp-750000000-1300 {
+ opp-microvolt = <1300000 1300000 1350000>;
+ opp-hz = /bits/ 64 <750000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1300>;
+ };
+
+ opp-800000000-1300 {
+ opp-microvolt = <1300000 1300000 1350000>;
+ opp-hz = /bits/ 64 <800000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1300>;
+ };
+
+ opp-900000000-1350 {
+ opp-microvolt = <1350000 1350000 1350000>;
+ opp-hz = /bits/ 64 <900000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1350>;
+ };
+ };
+
+ emc_bw_dfs_opp_table: opp-table-actmon {
+ compatible = "operating-points-v2";
+
+ opp-12750000 {
opp-hz = /bits/ 64 <12750000>;
opp-supported-hw = <0x000F>;
opp-peak-kBps = <102000>;
};
- opp@25500000 {
+ opp-25500000 {
opp-hz = /bits/ 64 <25500000>;
opp-supported-hw = <0x000F>;
opp-peak-kBps = <204000>;
};
- opp@27000000 {
+ opp-27000000 {
opp-hz = /bits/ 64 <27000000>;
opp-supported-hw = <0x000F>;
opp-peak-kBps = <216000>;
};
- opp@51000000 {
+ opp-51000000 {
opp-hz = /bits/ 64 <51000000>;
opp-supported-hw = <0x000F>;
opp-peak-kBps = <408000>;
};
- opp@54000000 {
+ opp-54000000 {
opp-hz = /bits/ 64 <54000000>;
opp-supported-hw = <0x000F>;
opp-peak-kBps = <432000>;
};
- opp@102000000 {
+ opp-102000000 {
opp-hz = /bits/ 64 <102000000>;
opp-supported-hw = <0x000F>;
opp-peak-kBps = <816000>;
};
- opp@108000000 {
+ opp-108000000 {
opp-hz = /bits/ 64 <108000000>;
opp-supported-hw = <0x000F>;
opp-peak-kBps = <864000>;
};
- opp@204000000 {
+ opp-204000000 {
opp-hz = /bits/ 64 <204000000>;
opp-supported-hw = <0x000F>;
opp-peak-kBps = <1632000>;
opp-suspend;
};
- opp@333500000 {
+ opp-333500000 {
opp-hz = /bits/ 64 <333500000>;
opp-supported-hw = <0x000F>;
opp-peak-kBps = <2668000>;
};
- opp@375000000 {
+ opp-375000000 {
opp-hz = /bits/ 64 <375000000>;
opp-supported-hw = <0x000F>;
opp-peak-kBps = <3000000>;
};
- opp@400000000 {
+ opp-400000000 {
opp-hz = /bits/ 64 <400000000>;
opp-supported-hw = <0x000F>;
opp-peak-kBps = <3200000>;
};
- opp@416000000 {
+ opp-416000000 {
opp-hz = /bits/ 64 <416000000>;
opp-supported-hw = <0x000F>;
opp-peak-kBps = <3328000>;
};
- opp@450000000 {
+ opp-450000000 {
opp-hz = /bits/ 64 <450000000>;
opp-supported-hw = <0x000F>;
opp-peak-kBps = <3600000>;
};
- opp@533000000 {
+ opp-500000000 {
+ opp-hz = /bits/ 64 <500000000>;
+ opp-supported-hw = <0x000F>;
+ opp-peak-kBps = <4000000>;
+ };
+
+ opp-533000000 {
opp-hz = /bits/ 64 <533000000>;
opp-supported-hw = <0x000F>;
opp-peak-kBps = <4264000>;
};
- opp@625000000 {
+ opp-625000000 {
opp-hz = /bits/ 64 <625000000>;
opp-supported-hw = <0x000E>;
opp-peak-kBps = <5000000>;
};
- opp@667000000 {
+ opp-667000000 {
opp-hz = /bits/ 64 <667000000>;
opp-supported-hw = <0x0006>;
opp-peak-kBps = <5336000>;
};
- opp@750000000 {
+ opp-750000000 {
opp-hz = /bits/ 64 <750000000>;
opp-supported-hw = <0x0004>;
opp-peak-kBps = <6000000>;
};
- opp@800000000 {
+ opp-800000000 {
opp-hz = /bits/ 64 <800000000>;
opp-supported-hw = <0x0004>;
opp-peak-kBps = <6400000>;
};
- opp@900000000 {
+ opp-900000000 {
opp-hz = /bits/ 64 <900000000>;
opp-supported-hw = <0x0004>;
opp-peak-kBps = <7200000>;
};
};
+
+ pcie_dvfs_opp_table: opp-table-pcie {
+ compatible = "operating-points-v2";
+
+ opp-250000000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <250000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1000>;
+ };
+ };
+
+ host1x_dvfs_opp_table: opp-table-host1x {
+ compatible = "operating-points-v2";
+
+ opp-152000000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <152000000>;
+ opp-supported-hw = <0x0007>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-188000000-1050 {
+ opp-microvolt = <1050000 1050000 1350000>;
+ opp-hz = /bits/ 64 <188000000>;
+ opp-supported-hw = <0x0007>;
+ required-opps = <&core_opp_1050>;
+ };
+
+ opp-222000000-1100 {
+ opp-microvolt = <1100000 1100000 1350000>;
+ opp-hz = /bits/ 64 <222000000>;
+ opp-supported-hw = <0x0007>;
+ required-opps = <&core_opp_1100>;
+ };
+
+ opp-242000000-1250 {
+ opp-microvolt = <1250000 1250000 1350000>;
+ opp-hz = /bits/ 64 <242000000>;
+ opp-supported-hw = <0x0008>;
+ required-opps = <&core_opp_1250>;
+ };
+
+ opp-254000000-1150 {
+ opp-microvolt = <1150000 1150000 1350000>;
+ opp-hz = /bits/ 64 <254000000>;
+ opp-supported-hw = <0x0007>;
+ required-opps = <&core_opp_1150>;
+ };
+
+ opp-267000000-1200 {
+ opp-microvolt = <1200000 1200000 1350000>;
+ opp-hz = /bits/ 64 <267000000>;
+ opp-supported-hw = <0x0007>;
+ required-opps = <&core_opp_1200>;
+ };
+
+ opp-300000000-1350 {
+ opp-microvolt = <1350000 1350000 1350000>;
+ opp-hz = /bits/ 64 <300000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1350>;
+ };
+ };
+
+ mpe_dvfs_opp_table: opp-table-mpe {
+ compatible = "operating-points-v2";
+
+ opp-234000000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <234000000>;
+ opp-supported-hw = <0x0003>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-247000000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <247000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-285000000-1050 {
+ opp-microvolt = <1050000 1050000 1350000>;
+ opp-hz = /bits/ 64 <285000000>;
+ opp-supported-hw = <0x0003>;
+ required-opps = <&core_opp_1050>;
+ };
+
+ opp-304000000-1050 {
+ opp-microvolt = <1050000 1050000 1350000>;
+ opp-hz = /bits/ 64 <304000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1050>;
+ };
+
+ opp-332000000-1100 {
+ opp-microvolt = <1100000 1100000 1350000>;
+ opp-hz = /bits/ 64 <332000000>;
+ opp-supported-hw = <0x0003>;
+ required-opps = <&core_opp_1100>;
+ };
+
+ opp-361000000-1100 {
+ opp-microvolt = <1100000 1100000 1350000>;
+ opp-hz = /bits/ 64 <361000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1100>;
+ };
+
+ opp-380000000-1150 {
+ opp-microvolt = <1150000 1150000 1350000>;
+ opp-hz = /bits/ 64 <380000000>;
+ opp-supported-hw = <0x0003>;
+ required-opps = <&core_opp_1150>;
+ };
+
+ opp-408000000-1150 {
+ opp-microvolt = <1150000 1150000 1350000>;
+ opp-hz = /bits/ 64 <408000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1150>;
+ };
+
+ opp-416000000-1200 {
+ opp-microvolt = <1200000 1200000 1350000>;
+ opp-hz = /bits/ 64 <416000000>;
+ opp-supported-hw = <0x0003>;
+ required-opps = <&core_opp_1200>;
+ };
+
+ opp-446000000-1200 {
+ opp-microvolt = <1200000 1200000 1350000>;
+ opp-hz = /bits/ 64 <446000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1200>;
+ };
+
+ opp-484000000-1250 {
+ opp-microvolt = <1250000 1250000 1350000>;
+ opp-hz = /bits/ 64 <484000000>;
+ opp-supported-hw = <0x000C>;
+ required-opps = <&core_opp_1250>;
+ };
+
+ opp-520000000-1300 {
+ opp-microvolt = <1300000 1300000 1350000>;
+ opp-hz = /bits/ 64 <520000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1300>;
+ };
+
+ opp-600000000-1350 {
+ opp-microvolt = <1350000 1350000 1350000>;
+ opp-hz = /bits/ 64 <600000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1350>;
+ };
+ };
+
+ vi_dvfs_opp_table: opp-table-vi {
+ compatible = "operating-points-v2";
+
+ opp-216000000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <216000000>;
+ opp-supported-hw = <0x0003>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-219000000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <219000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-267000000-1050 {
+ opp-microvolt = <1050000 1050000 1350000>;
+ opp-hz = /bits/ 64 <267000000>;
+ opp-supported-hw = <0x0006>;
+ required-opps = <&core_opp_1050>;
+ };
+
+ opp-285000000-1050 {
+ opp-microvolt = <1050000 1050000 1350000>;
+ opp-hz = /bits/ 64 <285000000>;
+ opp-supported-hw = <0x0001>;
+ required-opps = <&core_opp_1050>;
+ };
+
+ opp-300000000-1100 {
+ opp-microvolt = <1100000 1100000 1350000>;
+ opp-hz = /bits/ 64 <300000000>;
+ opp-supported-hw = <0x0007>;
+ required-opps = <&core_opp_1100>;
+ };
+
+ opp-371000000-1150 {
+ opp-microvolt = <1150000 1150000 1350000>;
+ opp-hz = /bits/ 64 <371000000>;
+ opp-supported-hw = <0x0006>;
+ required-opps = <&core_opp_1150>;
+ };
+
+ opp-409000000-1200 {
+ opp-microvolt = <1200000 1200000 1350000>;
+ opp-hz = /bits/ 64 <409000000>;
+ opp-supported-hw = <0x0006>;
+ required-opps = <&core_opp_1200>;
+ };
+
+ opp-425000000-1250 {
+ opp-microvolt = <1250000 1250000 1350000>;
+ opp-hz = /bits/ 64 <425000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1250>;
+ };
+
+ opp-470000000-1250 {
+ opp-microvolt = <1250000 1250000 1350000>;
+ opp-hz = /bits/ 64 <470000000>;
+ opp-supported-hw = <0x0008>;
+ required-opps = <&core_opp_1250>;
+ };
+ };
+
+ epp_dvfs_opp_table: opp-table-epp {
+ compatible = "operating-points-v2";
+
+ opp-267000000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <267000000>;
+ opp-supported-hw = <0x0007>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-285000000-1050 {
+ opp-microvolt = <1050000 1050000 1350000>;
+ opp-hz = /bits/ 64 <285000000>;
+ opp-supported-hw = <0x0003>;
+ required-opps = <&core_opp_1050>;
+ };
+
+ opp-304000000-1050 {
+ opp-microvolt = <1050000 1050000 1350000>;
+ opp-hz = /bits/ 64 <304000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1050>;
+ };
+
+ opp-332000000-1100 {
+ opp-microvolt = <1100000 1100000 1350000>;
+ opp-hz = /bits/ 64 <332000000>;
+ opp-supported-hw = <0x0003>;
+ required-opps = <&core_opp_1100>;
+ };
+
+ opp-361000000-1100 {
+ opp-microvolt = <1100000 1100000 1350000>;
+ opp-hz = /bits/ 64 <361000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1100>;
+ };
+
+ opp-380000000-1150 {
+ opp-microvolt = <1150000 1150000 1350000>;
+ opp-hz = /bits/ 64 <380000000>;
+ opp-supported-hw = <0x0003>;
+ required-opps = <&core_opp_1150>;
+ };
+
+ opp-408000000-1150 {
+ opp-microvolt = <1150000 1150000 1350000>;
+ opp-hz = /bits/ 64 <408000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1150>;
+ };
+
+ opp-416000000-1200 {
+ opp-microvolt = <1200000 1200000 1350000>;
+ opp-hz = /bits/ 64 <416000000>;
+ opp-supported-hw = <0x0003>;
+ required-opps = <&core_opp_1200>;
+ };
+
+ opp-446000000-1200 {
+ opp-microvolt = <1200000 1200000 1350000>;
+ opp-hz = /bits/ 64 <446000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1200>;
+ };
+
+ opp-484000000-1250 {
+ opp-microvolt = <1250000 1250000 1350000>;
+ opp-hz = /bits/ 64 <484000000>;
+ opp-supported-hw = <0x000C>;
+ required-opps = <&core_opp_1250>;
+ };
+
+ opp-520000000-1300 {
+ opp-microvolt = <1300000 1300000 1350000>;
+ opp-hz = /bits/ 64 <520000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1300>;
+ };
+
+ opp-600000000-1350 {
+ opp-microvolt = <1350000 1350000 1350000>;
+ opp-hz = /bits/ 64 <600000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1350>;
+ };
+ };
+
+ gr2d_dvfs_opp_table: opp-table-gr2d {
+ compatible = "operating-points-v2";
+
+ opp-267000000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <267000000>;
+ opp-supported-hw = <0x0007>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-285000000-1050 {
+ opp-microvolt = <1050000 1050000 1350000>;
+ opp-hz = /bits/ 64 <285000000>;
+ opp-supported-hw = <0x0003>;
+ required-opps = <&core_opp_1050>;
+ };
+
+ opp-304000000-1050 {
+ opp-microvolt = <1050000 1050000 1350000>;
+ opp-hz = /bits/ 64 <304000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1050>;
+ };
+
+ opp-332000000-1100 {
+ opp-microvolt = <1100000 1100000 1350000>;
+ opp-hz = /bits/ 64 <332000000>;
+ opp-supported-hw = <0x0003>;
+ required-opps = <&core_opp_1100>;
+ };
+
+ opp-361000000-1100 {
+ opp-microvolt = <1100000 1100000 1350000>;
+ opp-hz = /bits/ 64 <361000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1100>;
+ };
+
+ opp-380000000-1150 {
+ opp-microvolt = <1150000 1150000 1350000>;
+ opp-hz = /bits/ 64 <380000000>;
+ opp-supported-hw = <0x0003>;
+ required-opps = <&core_opp_1150>;
+ };
+
+ opp-408000000-1150 {
+ opp-microvolt = <1150000 1150000 1350000>;
+ opp-hz = /bits/ 64 <408000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1150>;
+ };
+
+ opp-416000000-1200 {
+ opp-microvolt = <1200000 1200000 1350000>;
+ opp-hz = /bits/ 64 <416000000>;
+ opp-supported-hw = <0x0003>;
+ required-opps = <&core_opp_1200>;
+ };
+
+ opp-446000000-1200 {
+ opp-microvolt = <1200000 1200000 1350000>;
+ opp-hz = /bits/ 64 <446000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1200>;
+ };
+
+ opp-484000000-1250 {
+ opp-microvolt = <1250000 1250000 1350000>;
+ opp-hz = /bits/ 64 <484000000>;
+ opp-supported-hw = <0x000C>;
+ required-opps = <&core_opp_1250>;
+ };
+
+ opp-520000000-1300 {
+ opp-microvolt = <1300000 1300000 1350000>;
+ opp-hz = /bits/ 64 <520000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1300>;
+ };
+
+ opp-600000000-1350 {
+ opp-microvolt = <1350000 1350000 1350000>;
+ opp-hz = /bits/ 64 <600000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1350>;
+ };
+ };
+
+ gr3d_dvfs_opp_table: opp-table-gr3d {
+ compatible = "operating-points-v2";
+
+ opp-234000000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <234000000>;
+ opp-supported-hw = <0x0003>;
+ required-opps = <&core_opp_1000>, <&core_opp_1000>;
+ };
+
+ opp-247000000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <247000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1000>, <&core_opp_1000>;
+ };
+
+ opp-285000000-1050 {
+ opp-microvolt = <1050000 1050000 1350000>;
+ opp-hz = /bits/ 64 <285000000>;
+ opp-supported-hw = <0x0003>;
+ required-opps = <&core_opp_1050>, <&core_opp_1050>;
+ };
+
+ opp-304000000-1050 {
+ opp-microvolt = <1050000 1050000 1350000>;
+ opp-hz = /bits/ 64 <304000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1050>, <&core_opp_1050>;
+ };
+
+ opp-332000000-1100 {
+ opp-microvolt = <1100000 1100000 1350000>;
+ opp-hz = /bits/ 64 <332000000>;
+ opp-supported-hw = <0x0003>;
+ required-opps = <&core_opp_1100>, <&core_opp_1100>;
+ };
+
+ opp-361000000-1100 {
+ opp-microvolt = <1100000 1100000 1350000>;
+ opp-hz = /bits/ 64 <361000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1100>, <&core_opp_1100>;
+ };
+
+ opp-380000000-1150 {
+ opp-microvolt = <1150000 1150000 1350000>;
+ opp-hz = /bits/ 64 <380000000>;
+ opp-supported-hw = <0x0003>;
+ required-opps = <&core_opp_1150>, <&core_opp_1150>;
+ };
+
+ opp-408000000-1150 {
+ opp-microvolt = <1150000 1150000 1350000>;
+ opp-hz = /bits/ 64 <408000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1150>, <&core_opp_1150>;
+ };
+
+ opp-416000000-1200 {
+ opp-microvolt = <1200000 1200000 1350000>;
+ opp-hz = /bits/ 64 <416000000>;
+ opp-supported-hw = <0x0003>;
+ required-opps = <&core_opp_1200>, <&core_opp_1200>;
+ };
+
+ opp-446000000-1200 {
+ opp-microvolt = <1200000 1200000 1350000>;
+ opp-hz = /bits/ 64 <446000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1200>, <&core_opp_1200>;
+ };
+
+ opp-484000000-1250 {
+ opp-microvolt = <1250000 1250000 1350000>;
+ opp-hz = /bits/ 64 <484000000>;
+ opp-supported-hw = <0x000C>;
+ required-opps = <&core_opp_1250>, <&core_opp_1250>;
+ };
+
+ opp-520000000-1300 {
+ opp-microvolt = <1300000 1300000 1350000>;
+ opp-hz = /bits/ 64 <520000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1300>, <&core_opp_1300>;
+ };
+
+ opp-600000000-1350 {
+ opp-microvolt = <1350000 1350000 1350000>;
+ opp-hz = /bits/ 64 <600000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1350>, <&core_opp_1350>;
+ };
+ };
+
+ disp1_dvfs_opp_table: opp-table-disp1 {
+ compatible = "operating-points-v2";
+
+ opp-120000000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <120000000>;
+ opp-supported-hw = <0x0009>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-155000000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <155000000>;
+ opp-supported-hw = <0x0006>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-190000000-1200 {
+ opp-microvolt = <1200000 1200000 1350000>;
+ opp-hz = /bits/ 64 <190000000>;
+ opp-supported-hw = <0x0009>;
+ required-opps = <&core_opp_1200>;
+ };
+
+ opp-268000000-1050 {
+ opp-microvolt = <1050000 1050000 1350000>;
+ opp-hz = /bits/ 64 <268000000>;
+ opp-supported-hw = <0x0006>;
+ required-opps = <&core_opp_1050>;
+ };
+ };
+
+ disp2_dvfs_opp_table: opp-table-disp2 {
+ compatible = "operating-points-v2";
+
+ opp-120000000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <120000000>;
+ opp-supported-hw = <0x0009>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-155000000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <155000000>;
+ opp-supported-hw = <0x0006>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-190000000-1200 {
+ opp-microvolt = <1200000 1200000 1350000>;
+ opp-hz = /bits/ 64 <190000000>;
+ opp-supported-hw = <0x0009>;
+ required-opps = <&core_opp_1200>;
+ };
+
+ opp-268000000-1050 {
+ opp-microvolt = <1050000 1050000 1350000>;
+ opp-hz = /bits/ 64 <268000000>;
+ opp-supported-hw = <0x0006>;
+ required-opps = <&core_opp_1050>;
+ };
+ };
+
+ hdmi_dvfs_opp_table: opp-table-hdmi {
+ compatible = "operating-points-v2";
+
+ opp-148500000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <148500000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1000>;
+ };
+ };
+
+ tvo_dvfs_opp_table: opp-table-tvo {
+ compatible = "operating-points-v2";
+
+ opp-297000000-1050 {
+ opp-microvolt = <1050000 1050000 1350000>;
+ opp-hz = /bits/ 64 <297000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1050>;
+ };
+ };
+
+ dsia_dvfs_opp_table: opp-table-dsia {
+ compatible = "operating-points-v2";
+
+ opp-275000000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <275000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1000>;
+ };
+ };
+
+ dsib_dvfs_opp_table: opp-table-dsib {
+ compatible = "operating-points-v2";
+
+ opp-275000000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <275000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1000>;
+ };
+ };
+
+ sclk_dvfs_opp_table: opp-table-sclk {
+ compatible = "operating-points-v2";
+
+ opp-51000000-950 {
+ opp-microvolt = <950000 950000 1350000>;
+ opp-hz = /bits/ 64 <51000000>;
+ opp-supported-hw = <0x0006>;
+ required-opps = <&core_opp_950>;
+ };
+
+ opp-136000000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <136000000>;
+ opp-supported-hw = <0x0001>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-164000000-1050 {
+ opp-microvolt = <1050000 1050000 1350000>;
+ opp-hz = /bits/ 64 <164000000>;
+ opp-supported-hw = <0x0001>;
+ required-opps = <&core_opp_1050>;
+ };
+
+ opp-191000000-1100 {
+ opp-microvolt = <1100000 1100000 1350000>;
+ opp-hz = /bits/ 64 <191000000>;
+ opp-supported-hw = <0x0001>;
+ required-opps = <&core_opp_1100>;
+ };
+
+ opp-205000000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <205000000>;
+ opp-supported-hw = <0x0006>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-216000000-1150 {
+ opp-microvolt = <1150000 1150000 1350000>;
+ opp-hz = /bits/ 64 <216000000>;
+ opp-supported-hw = <0x0001>;
+ required-opps = <&core_opp_1150>;
+ };
+
+ opp-227000000-1100 {
+ opp-microvolt = <1100000 1100000 1350000>;
+ opp-hz = /bits/ 64 <227000000>;
+ opp-supported-hw = <0x0006>;
+ required-opps = <&core_opp_1100>;
+ };
+
+ opp-267000000-1200 {
+ opp-microvolt = <1200000 1200000 1350000>;
+ opp-hz = /bits/ 64 <267000000>;
+ opp-supported-hw = <0x0006>;
+ required-opps = <&core_opp_1200>;
+ };
+
+ opp-334000000-1250 {
+ opp-microvolt = <1250000 1250000 1350000>;
+ opp-hz = /bits/ 64 <334000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1250>;
+ };
+
+ opp-378000000-1250 {
+ opp-microvolt = <1250000 1250000 1350000>;
+ opp-hz = /bits/ 64 <378000000>;
+ opp-supported-hw = <0x0008>;
+ required-opps = <&core_opp_1250>;
+ };
+ };
+
+ pll_c_dvfs_opp_table: opp-table-pllc {
+ compatible = "operating-points-v2";
+
+ opp-533000000-950 {
+ opp-microvolt = <950000 950000 1350000>;
+ opp-hz = /bits/ 64 <533000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_950>;
+ };
+
+ opp-667000000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <667000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-800000000-1100 {
+ opp-microvolt = <1100000 1100000 1350000>;
+ opp-hz = /bits/ 64 <800000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1100>;
+ };
+
+ opp-1066000000-1200 {
+ opp-microvolt = <1200000 1200000 1350000>;
+ opp-hz = /bits/ 64 <1066000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1200>;
+ };
+
+ opp-1200000000-1350 {
+ opp-microvolt = <1350000 1350000 1350000>;
+ opp-hz = /bits/ 64 <1200000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1350>;
+ };
+ };
+
+ pll_e_dvfs_opp_table: opp-table-plle {
+ compatible = "operating-points-v2";
+
+ opp-100000000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <100000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1000>;
+ };
+ };
+
+ pll_m_dvfs_opp_table: opp-table-pllm {
+ compatible = "operating-points-v2";
+
+ opp-533000000-950 {
+ opp-microvolt = <950000 950000 1350000>;
+ opp-hz = /bits/ 64 <533000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_950>;
+ };
+
+ opp-667000000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <667000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-800000000-1100 {
+ opp-microvolt = <1100000 1100000 1350000>;
+ opp-hz = /bits/ 64 <800000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1100>;
+ };
+
+ opp-1066000000-1200 {
+ opp-microvolt = <1200000 1200000 1350000>;
+ opp-hz = /bits/ 64 <1066000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1200>;
+ };
+ };
+
+ vde_dvfs_opp_table: opp-table-vde {
+ compatible = "operating-points-v2";
+
+ opp-228000000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <228000000>;
+ opp-supported-hw = <0x0003>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-247000000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <247000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-275000000-1050 {
+ opp-microvolt = <1050000 1050000 1350000>;
+ opp-hz = /bits/ 64 <275000000>;
+ opp-supported-hw = <0x0003>;
+ required-opps = <&core_opp_1050>;
+ };
+
+ opp-304000000-1050 {
+ opp-microvolt = <1050000 1050000 1350000>;
+ opp-hz = /bits/ 64 <304000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1050>;
+ };
+
+ opp-332000000-1100 {
+ opp-microvolt = <1100000 1100000 1350000>;
+ opp-hz = /bits/ 64 <332000000>;
+ opp-supported-hw = <0x0003>;
+ required-opps = <&core_opp_1100>;
+ };
+
+ opp-352000000-1100 {
+ opp-microvolt = <1100000 1100000 1350000>;
+ opp-hz = /bits/ 64 <352000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1100>;
+ };
+
+ opp-380000000-1150 {
+ opp-microvolt = <1150000 1150000 1350000>;
+ opp-hz = /bits/ 64 <380000000>;
+ opp-supported-hw = <0x0003>;
+ required-opps = <&core_opp_1150>;
+ };
+
+ opp-400000000-1150 {
+ opp-microvolt = <1150000 1150000 1350000>;
+ opp-hz = /bits/ 64 <400000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1150>;
+ };
+
+ opp-416000000-1200 {
+ opp-microvolt = <1200000 1200000 1350000>;
+ opp-hz = /bits/ 64 <416000000>;
+ opp-supported-hw = <0x0003>;
+ required-opps = <&core_opp_1200>;
+ };
+
+ opp-437000000-1200 {
+ opp-microvolt = <1200000 1200000 1350000>;
+ opp-hz = /bits/ 64 <437000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1200>;
+ };
+
+ opp-484000000-1250 {
+ opp-microvolt = <1250000 1250000 1350000>;
+ opp-hz = /bits/ 64 <484000000>;
+ opp-supported-hw = <0x000C>;
+ required-opps = <&core_opp_1250>;
+ };
+
+ opp-520000000-1300 {
+ opp-microvolt = <1300000 1300000 1350000>;
+ opp-hz = /bits/ 64 <520000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1300>;
+ };
+
+ opp-600000000-1350 {
+ opp-microvolt = <1350000 1350000 1350000>;
+ opp-hz = /bits/ 64 <600000000>;
+ opp-supported-hw = <0x0004>;
+ required-opps = <&core_opp_1350>;
+ };
+ };
+
+ fuse_burn_dvfs_opp_table: opp-table-fuseburn {
+ compatible = "operating-points-v2";
+
+ opp-26000000-1150 {
+ opp-microvolt = <1150000 1150000 1350000>;
+ opp-hz = /bits/ 64 <26000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1150>;
+ };
+ };
+
+ nor_dvfs_opp_table: opp-table-nor {
+ compatible = "operating-points-v2";
+
+ opp-108000000-1250 {
+ opp-microvolt = <1250000 1250000 1350000>;
+ opp-hz = /bits/ 64 <108000000>;
+ opp-supported-hw = <0x0008>;
+ required-opps = <&core_opp_1250>;
+ };
+
+ opp-115000000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <115000000>;
+ opp-supported-hw = <0x0007>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-130000000-1050 {
+ opp-microvolt = <1050000 1050000 1350000>;
+ opp-hz = /bits/ 64 <130000000>;
+ opp-supported-hw = <0x0007>;
+ required-opps = <&core_opp_1050>;
+ };
+
+ opp-133000000-1150 {
+ opp-microvolt = <1150000 1150000 1350000>;
+ opp-hz = /bits/ 64 <133000000>;
+ opp-supported-hw = <0x0007>;
+ required-opps = <&core_opp_1150>;
+ };
+ };
+
+ pwm_dvfs_opp_table: opp-table-pwm {
+ compatible = "operating-points-v2";
+
+ opp-408000000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <408000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1000>;
+ };
+ };
+
+ sbc1_dvfs_opp_table: opp-table-sbc1 {
+ compatible = "operating-points-v2";
+
+ opp-52000000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <52000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-60000000-1050 {
+ opp-microvolt = <1050000 1050000 1350000>;
+ opp-hz = /bits/ 64 <60000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1050>;
+ };
+
+ opp-100000000-1200 {
+ opp-microvolt = <1200000 1200000 1350000>;
+ opp-hz = /bits/ 64 <100000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1200>;
+ };
+ };
+
+ sbc2_dvfs_opp_table: opp-table-sbc2 {
+ compatible = "operating-points-v2";
+
+ opp-52000000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <52000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-60000000-1050 {
+ opp-microvolt = <1050000 1050000 1350000>;
+ opp-hz = /bits/ 64 <60000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1050>;
+ };
+
+ opp-100000000-1200 {
+ opp-microvolt = <1200000 1200000 1350000>;
+ opp-hz = /bits/ 64 <100000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1200>;
+ };
+ };
+
+ sbc3_dvfs_opp_table: opp-table-sbc3 {
+ compatible = "operating-points-v2";
+
+ opp-52000000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <52000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-60000000-1050 {
+ opp-microvolt = <1050000 1050000 1350000>;
+ opp-hz = /bits/ 64 <60000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1050>;
+ };
+
+ opp-100000000-1200 {
+ opp-microvolt = <1200000 1200000 1350000>;
+ opp-hz = /bits/ 64 <100000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1200>;
+ };
+ };
+
+ sbc4_dvfs_opp_table: opp-table-sbc4 {
+ compatible = "operating-points-v2";
+
+ opp-52000000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <52000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-60000000-1050 {
+ opp-microvolt = <1050000 1050000 1350000>;
+ opp-hz = /bits/ 64 <60000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1050>;
+ };
+
+ opp-100000000-1200 {
+ opp-microvolt = <1200000 1200000 1350000>;
+ opp-hz = /bits/ 64 <100000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1200>;
+ };
+ };
+
+ sbc5_dvfs_opp_table: opp-table-sbc5 {
+ compatible = "operating-points-v2";
+
+ opp-52000000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <52000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-60000000-1050 {
+ opp-microvolt = <1050000 1050000 1350000>;
+ opp-hz = /bits/ 64 <60000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1050>;
+ };
+
+ opp-100000000-1200 {
+ opp-microvolt = <1200000 1200000 1350000>;
+ opp-hz = /bits/ 64 <100000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1200>;
+ };
+ };
+
+ sbc6_dvfs_opp_table: opp-table-sbc6 {
+ compatible = "operating-points-v2";
+
+ opp-52000000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <52000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1000>;
+ };
+
+ opp-60000000-1050 {
+ opp-microvolt = <1050000 1050000 1350000>;
+ opp-hz = /bits/ 64 <60000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1050>;
+ };
+
+ opp-100000000-1200 {
+ opp-microvolt = <1200000 1200000 1350000>;
+ opp-hz = /bits/ 64 <100000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1200>;
+ };
+ };
+
+ sdmmc1_dvfs_opp_table: opp-table-sdmmc1 {
+ compatible = "operating-points-v2";
+
+ opp-104000000-950 {
+ opp-microvolt = <950000 950000 1350000>;
+ opp-hz = /bits/ 64 <104000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_950>;
+ };
+
+ opp-208000000-1200 {
+ opp-microvolt = <1200000 1200000 1350000>;
+ opp-hz = /bits/ 64 <208000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1200>;
+ };
+ };
+
+ sdmmc3_dvfs_opp_table: opp-table-sdmmc3 {
+ compatible = "operating-points-v2";
+
+ opp-104000000-950 {
+ opp-microvolt = <950000 950000 1350000>;
+ opp-hz = /bits/ 64 <104000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_950>;
+ };
+
+ opp-208000000-1200 {
+ opp-microvolt = <1200000 1200000 1350000>;
+ opp-hz = /bits/ 64 <208000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1200>;
+ };
+ };
+
+ usbd_dvfs_opp_table: opp-table-usbd {
+ compatible = "operating-points-v2";
+
+ opp-480000000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <480000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1000>;
+ };
+ };
+
+ usb2_dvfs_opp_table: opp-table-usb2 {
+ compatible = "operating-points-v2";
+
+ opp-480000000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <480000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1000>;
+ };
+ };
+
+ usb3_dvfs_opp_table: opp-table-usb3 {
+ compatible = "operating-points-v2";
+
+ opp-480000000-1000 {
+ opp-microvolt = <1000000 1000000 1350000>;
+ opp-hz = /bits/ 64 <480000000>;
+ opp-supported-hw = <0x000F>;
+ required-opps = <&core_opp_1000>;
+ };
+ };
};
diff --git a/arch/arm/boot/dts/tegra30.dtsi b/arch/arm/boot/dts/tegra30.dtsi
index ae3df73..9dab8d2 100644
--- a/arch/arm/boot/dts/tegra30.dtsi
+++ b/arch/arm/boot/dts/tegra30.dtsi
@@ -55,6 +55,8 @@ pcie@3000 {
<&tegra_car 72>,
<&tegra_car 74>;
reset-names = "pex", "afi", "pcie_x";
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&pcie_dvfs_opp_table>;
status = "disabled";
pci@1,0 {
@@ -121,9 +123,11 @@ host1x@50000000 {
interrupt-names = "syncpt", "host1x";
clocks = <&tegra_car TEGRA30_CLK_HOST1X>;
clock-names = "host1x";
- resets = <&tegra_car 28>;
- reset-names = "host1x";
+ resets = <&tegra_car 28>, <&mc TEGRA30_MC_RESET_HC>;
+ reset-names = "host1x", "mc";
iommus = <&mc TEGRA_SWGROUP_HC>;
+ power-domains = <&pd_heg>;
+ operating-points-v2 = <&host1x_dvfs_opp_table>;
#address-cells = <1>;
#size-cells = <1>;
@@ -137,8 +141,12 @@ mpe@54040000 {
clocks = <&tegra_car TEGRA30_CLK_MPE>;
resets = <&tegra_car 60>;
reset-names = "mpe";
+ power-domains = <&pd_mpe>;
+ operating-points-v2 = <&mpe_dvfs_opp_table>;
iommus = <&mc TEGRA_SWGROUP_MPE>;
+
+ status = "disabled";
};
vi@54080000 {
@@ -148,8 +156,12 @@ vi@54080000 {
clocks = <&tegra_car TEGRA30_CLK_VI>;
resets = <&tegra_car 20>;
reset-names = "vi";
+ power-domains = <&pd_venc>;
+ operating-points-v2 = <&vi_dvfs_opp_table>;
iommus = <&mc TEGRA_SWGROUP_VI>;
+
+ status = "disabled";
};
epp@540c0000 {
@@ -159,8 +171,12 @@ epp@540c0000 {
clocks = <&tegra_car TEGRA30_CLK_EPP>;
resets = <&tegra_car 19>;
reset-names = "epp";
+ power-domains = <&pd_heg>;
+ operating-points-v2 = <&epp_dvfs_opp_table>;
iommus = <&mc TEGRA_SWGROUP_EPP>;
+
+ status = "disabled";
};
isp@54100000 {
@@ -170,8 +186,11 @@ isp@54100000 {
clocks = <&tegra_car TEGRA30_CLK_ISP>;
resets = <&tegra_car 23>;
reset-names = "isp";
+ power-domains = <&pd_venc>;
iommus = <&mc TEGRA_SWGROUP_ISP>;
+
+ status = "disabled";
};
gr2d@54140000 {
@@ -179,8 +198,10 @@ gr2d@54140000 {
reg = <0x54140000 0x00040000>;
interrupts = <GIC_SPI 72 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&tegra_car TEGRA30_CLK_GR2D>;
- resets = <&tegra_car 21>;
- reset-names = "2d";
+ resets = <&tegra_car 21>, <&mc TEGRA30_MC_RESET_2D>;
+ reset-names = "2d", "mc";
+ power-domains = <&pd_heg>;
+ operating-points-v2 = <&gr2d_dvfs_opp_table>;
iommus = <&mc TEGRA_SWGROUP_G2>;
};
@@ -192,8 +213,13 @@ gr3d@54180000 {
<&tegra_car TEGRA30_CLK_GR3D2>;
clock-names = "3d", "3d2";
resets = <&tegra_car 24>,
- <&tegra_car 98>;
- reset-names = "3d", "3d2";
+ <&tegra_car 98>,
+ <&mc TEGRA30_MC_RESET_3D>,
+ <&mc TEGRA30_MC_RESET_3D2>;
+ reset-names = "3d", "3d2", "mc", "mc2";
+ power-domains = <&pd_3d0>, <&pd_3d1>;
+ power-domain-names = "3d0", "3d1";
+ operating-points-v2 = <&gr3d_dvfs_opp_table>;
iommus = <&mc TEGRA_SWGROUP_NV>,
<&mc TEGRA_SWGROUP_NV2>;
@@ -208,6 +234,8 @@ dc@54200000 {
clock-names = "dc", "parent";
resets = <&tegra_car 27>;
reset-names = "dc";
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&disp1_dvfs_opp_table>;
iommus = <&mc TEGRA_SWGROUP_DC>;
@@ -238,6 +266,8 @@ dc@54240000 {
clock-names = "dc", "parent";
resets = <&tegra_car 26>;
reset-names = "dc";
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&disp2_dvfs_opp_table>;
iommus = <&mc TEGRA_SWGROUP_DCB>;
@@ -268,6 +298,8 @@ hdmi@54280000 {
clock-names = "hdmi", "parent";
resets = <&tegra_car 51>;
reset-names = "hdmi";
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&hdmi_dvfs_opp_table>;
status = "disabled";
};
@@ -276,6 +308,8 @@ tvo@542c0000 {
reg = <0x542c0000 0x00040000>;
interrupts = <GIC_SPI 76 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&tegra_car TEGRA30_CLK_TVO>;
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&tvo_dvfs_opp_table>;
status = "disabled";
};
@@ -287,6 +321,8 @@ dsi@54300000 {
clock-names = "dsi", "parent";
resets = <&tegra_car 48>;
reset-names = "dsi";
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&dsia_dvfs_opp_table>;
status = "disabled";
};
@@ -298,6 +334,8 @@ dsi@54400000 {
clock-names = "dsi", "parent";
resets = <&tegra_car 84>;
reset-names = "dsi";
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&dsib_dvfs_opp_table>;
status = "disabled";
};
};
@@ -358,6 +396,34 @@ tegra_car: clock@60006000 {
reg = <0x60006000 0x1000>;
#clock-cells = <1>;
#reset-cells = <1>;
+
+ sclk {
+ compatible = "nvidia,tegra30-sclk";
+ clocks = <&tegra_car TEGRA30_CLK_SCLK>;
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&sclk_dvfs_opp_table>;
+ };
+
+ pll-c {
+ compatible = "nvidia,tegra30-pllc";
+ clocks = <&tegra_car TEGRA30_CLK_PLL_C>;
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&pll_c_dvfs_opp_table>;
+ };
+
+ pll-e {
+ compatible = "nvidia,tegra30-plle";
+ clocks = <&tegra_car TEGRA30_CLK_PLL_E>;
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&pll_e_dvfs_opp_table>;
+ };
+
+ pll-m {
+ compatible = "nvidia,tegra30-pllm";
+ clocks = <&tegra_car TEGRA30_CLK_PLL_M>;
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&pll_m_dvfs_opp_table>;
+ };
};
flow-controller@60007000 {
@@ -441,9 +507,7 @@ gpio: gpio@6000d000 {
gpio-controller;
#interrupt-cells = <2>;
interrupt-controller;
- /*
gpio-ranges = <&pinmux 0 0 248>;
- */
};
vde@6001a000 {
@@ -468,6 +532,8 @@ vde@6001a000 {
reset-names = "vde", "mc";
resets = <&tegra_car 61>, <&mc TEGRA30_MC_RESET_VDE>;
iommus = <&mc TEGRA_SWGROUP_VDE>;
+ power-domains = <&pd_vde>;
+ operating-points-v2 = <&vde_dvfs_opp_table>;
};
apbmisc@70000800 {
@@ -565,6 +631,8 @@ gmi@70009000 {
clock-names = "gmi";
resets = <&tegra_car 42>;
reset-names = "gmi";
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&nor_dvfs_opp_table>;
status = "disabled";
};
@@ -575,6 +643,8 @@ pwm: pwm@7000a000 {
clocks = <&tegra_car TEGRA30_CLK_PWM>;
resets = <&tegra_car 17>;
reset-names = "pwm";
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&pwm_dvfs_opp_table>;
status = "disabled";
};
@@ -666,7 +736,7 @@ i2c@7000d000 {
};
spi@7000d400 {
- compatible = "nvidia,tegra30-slink", "nvidia,tegra20-slink";
+ compatible = "nvidia,tegra30-slink";
reg = <0x7000d400 0x200>;
interrupts = <GIC_SPI 59 IRQ_TYPE_LEVEL_HIGH>;
#address-cells = <1>;
@@ -676,11 +746,13 @@ spi@7000d400 {
reset-names = "spi";
dmas = <&apbdma 15>, <&apbdma 15>;
dma-names = "rx", "tx";
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&sbc1_dvfs_opp_table>;
status = "disabled";
};
spi@7000d600 {
- compatible = "nvidia,tegra30-slink", "nvidia,tegra20-slink";
+ compatible = "nvidia,tegra30-slink";
reg = <0x7000d600 0x200>;
interrupts = <GIC_SPI 82 IRQ_TYPE_LEVEL_HIGH>;
#address-cells = <1>;
@@ -690,11 +762,13 @@ spi@7000d600 {
reset-names = "spi";
dmas = <&apbdma 16>, <&apbdma 16>;
dma-names = "rx", "tx";
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&sbc2_dvfs_opp_table>;
status = "disabled";
};
spi@7000d800 {
- compatible = "nvidia,tegra30-slink", "nvidia,tegra20-slink";
+ compatible = "nvidia,tegra30-slink";
reg = <0x7000d800 0x200>;
interrupts = <GIC_SPI 83 IRQ_TYPE_LEVEL_HIGH>;
#address-cells = <1>;
@@ -704,11 +778,13 @@ spi@7000d800 {
reset-names = "spi";
dmas = <&apbdma 17>, <&apbdma 17>;
dma-names = "rx", "tx";
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&sbc3_dvfs_opp_table>;
status = "disabled";
};
spi@7000da00 {
- compatible = "nvidia,tegra30-slink", "nvidia,tegra20-slink";
+ compatible = "nvidia,tegra30-slink";
reg = <0x7000da00 0x200>;
interrupts = <GIC_SPI 93 IRQ_TYPE_LEVEL_HIGH>;
#address-cells = <1>;
@@ -718,11 +794,13 @@ spi@7000da00 {
reset-names = "spi";
dmas = <&apbdma 18>, <&apbdma 18>;
dma-names = "rx", "tx";
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&sbc4_dvfs_opp_table>;
status = "disabled";
};
spi@7000dc00 {
- compatible = "nvidia,tegra30-slink", "nvidia,tegra20-slink";
+ compatible = "nvidia,tegra30-slink";
reg = <0x7000dc00 0x200>;
interrupts = <GIC_SPI 94 IRQ_TYPE_LEVEL_HIGH>;
#address-cells = <1>;
@@ -732,11 +810,13 @@ spi@7000dc00 {
reset-names = "spi";
dmas = <&apbdma 27>, <&apbdma 27>;
dma-names = "rx", "tx";
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&sbc5_dvfs_opp_table>;
status = "disabled";
};
spi@7000de00 {
- compatible = "nvidia,tegra30-slink", "nvidia,tegra20-slink";
+ compatible = "nvidia,tegra30-slink";
reg = <0x7000de00 0x200>;
interrupts = <GIC_SPI 79 IRQ_TYPE_LEVEL_HIGH>;
#address-cells = <1>;
@@ -746,6 +826,8 @@ spi@7000de00 {
reset-names = "spi";
dmas = <&apbdma 28>, <&apbdma 28>;
dma-names = "rx", "tx";
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&sbc6_dvfs_opp_table>;
status = "disabled";
};
@@ -765,6 +847,72 @@ tegra_pmc: pmc@7000e400 {
clocks = <&tegra_car TEGRA30_CLK_PCLK>, <&clk32k_in>;
clock-names = "pclk", "clk32k_in";
#clock-cells = <1>;
+
+ pd_core: core-domain {
+ #power-domain-cells = <0>;
+ operating-points-v2 = <&core_opp_table>;
+ };
+
+ powergates {
+ pd_3d0: td {
+ clocks = <&tegra_car TEGRA30_CLK_GR3D>;
+ resets = <&mc TEGRA30_MC_RESET_3D>,
+ <&tegra_car TEGRA30_CLK_GR3D>;
+ power-domains = <&pd_core>;
+ #power-domain-cells = <0>;
+ };
+
+ pd_3d1: td2 {
+ clocks = <&tegra_car TEGRA30_CLK_GR3D2>;
+ resets = <&mc TEGRA30_MC_RESET_3D2>,
+ <&tegra_car TEGRA30_CLK_GR3D2>;
+ power-domains = <&pd_core>;
+ #power-domain-cells = <0>;
+ };
+
+ pd_venc: venc {
+ clocks = <&tegra_car TEGRA30_CLK_ISP>,
+ <&tegra_car TEGRA30_CLK_VI>,
+ <&tegra_car TEGRA30_CLK_CSI>;
+ resets = <&mc TEGRA30_MC_RESET_ISP>,
+ <&mc TEGRA30_MC_RESET_VI>,
+ <&tegra_car TEGRA30_CLK_ISP>,
+ <&tegra_car 20 /* VI */>,
+ <&tegra_car TEGRA30_CLK_CSI>;
+ power-domains = <&pd_core>;
+ #power-domain-cells = <0>;
+ };
+
+ pd_vde: vdec {
+ clocks = <&tegra_car TEGRA30_CLK_VDE>;
+ resets = <&mc TEGRA30_MC_RESET_VDE>,
+ <&tegra_car TEGRA30_CLK_VDE>;
+ power-domains = <&pd_core>;
+ #power-domain-cells = <0>;
+ };
+
+ pd_mpe: mpe {
+ clocks = <&tegra_car TEGRA30_CLK_MPE>;
+ resets = <&mc TEGRA30_MC_RESET_MPE>,
+ <&tegra_car TEGRA30_CLK_MPE>;
+ power-domains = <&pd_core>;
+ #power-domain-cells = <0>;
+ };
+
+ pd_heg: heg {
+ clocks = <&tegra_car TEGRA30_CLK_GR2D>,
+ <&tegra_car TEGRA30_CLK_EPP>,
+ <&tegra_car TEGRA30_CLK_HOST1X>;
+ resets = <&mc TEGRA30_MC_RESET_2D>,
+ <&mc TEGRA30_MC_RESET_EPP>,
+ <&mc TEGRA30_MC_RESET_HC>,
+ <&tegra_car TEGRA30_CLK_GR2D>,
+ <&tegra_car TEGRA30_CLK_EPP>,
+ <&tegra_car TEGRA30_CLK_HOST1X>;
+ power-domains = <&pd_core>;
+ #power-domain-cells = <0>;
+ };
+ };
};
mc: memory-controller@7000f000 {
@@ -785,6 +933,7 @@ emc: memory-controller@7000f400 {
reg = <0x7000f400 0x400>;
interrupts = <GIC_SPI 78 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&tegra_car TEGRA30_CLK_EMC>;
+ power-domains = <&pd_core>;
nvidia,memory-controller = <&mc>;
operating-points-v2 = <&emc_icc_dvfs_opp_table>;
@@ -799,6 +948,8 @@ fuse@7000f800 {
clock-names = "fuse";
resets = <&tegra_car 39>;
reset-names = "fuse";
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&fuse_burn_dvfs_opp_table>;
};
tsensor: tsensor@70014000 {
@@ -921,6 +1072,8 @@ mmc@78000000 {
clock-names = "sdhci";
resets = <&tegra_car 14>;
reset-names = "sdhci";
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&sdmmc1_dvfs_opp_table>;
status = "disabled";
};
@@ -943,6 +1096,8 @@ mmc@78000400 {
clock-names = "sdhci";
resets = <&tegra_car 69>;
reset-names = "sdhci";
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&sdmmc3_dvfs_opp_table>;
status = "disabled";
};
@@ -967,6 +1122,8 @@ usb@7d000000 {
reset-names = "usb";
nvidia,needs-double-reset;
nvidia,phy = <&phy1>;
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&usbd_dvfs_opp_table>;
status = "disabled";
};
@@ -1008,6 +1165,8 @@ usb@7d004000 {
resets = <&tegra_car 58>;
reset-names = "usb";
nvidia,phy = <&phy2>;
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&usb2_dvfs_opp_table>;
status = "disabled";
};
@@ -1048,6 +1207,8 @@ usb@7d008000 {
resets = <&tegra_car 59>;
reset-names = "usb";
nvidia,phy = <&phy3>;
+ power-domains = <&pd_core>;
+ operating-points-v2 = <&usb3_dvfs_opp_table>;
status = "disabled";
};
diff --git a/arch/arm/boot/dts/vf610-zii-dev-rev-b.dts b/arch/arm/boot/dts/vf610-zii-dev-rev-b.dts
index 043ddd7..1f9686c 100644
--- a/arch/arm/boot/dts/vf610-zii-dev-rev-b.dts
+++ b/arch/arm/boot/dts/vf610-zii-dev-rev-b.dts
@@ -149,7 +149,7 @@ switch1port5: port@5 {
reg = <5>;
label = "dsa";
link = <&switch2port9>;
- phy-mode = "rgmii-txid";
+ phy-mode = "1000base-x";
fixed-link {
speed = <1000>;
@@ -211,12 +211,14 @@ port@0 {
reg = <0>;
label = "lan6";
phy-handle = <&switch2phy0>;
+ phy-mode = "sgmii";
};
port@1 {
reg = <1>;
label = "lan7";
phy-handle = <&switch2phy1>;
+ phy-mode = "sgmii";
};
port@2 {
@@ -252,7 +254,7 @@ fixed-link {
switch2port9: port@9 {
reg = <9>;
label = "dsa";
- phy-mode = "rgmii-txid";
+ phy-mode = "1000base-x";
link = <&switch1port5
&switch0port5>;
diff --git a/arch/arm/configs/at91_dt_defconfig b/arch/arm/configs/at91_dt_defconfig
index a6310c8..549d01b 100644
--- a/arch/arm/configs/at91_dt_defconfig
+++ b/arch/arm/configs/at91_dt_defconfig
@@ -216,7 +216,14 @@
CONFIG_NLS_CODEPAGE_850=y
CONFIG_NLS_ISO8859_1=y
CONFIG_NLS_UTF8=y
-CONFIG_CRYPTO_ECB=y
+CONFIG_CRYPTO_CBC=y
+CONFIG_CRYPTO_CFB=y
+CONFIG_CRYPTO_OFB=y
+CONFIG_CRYPTO_XTS=y
+CONFIG_CRYPTO_HMAC=y
+CONFIG_CRYPTO_SHA1=y
+CONFIG_CRYPTO_SHA512=y
+CONFIG_CRYPTO_DES=y
CONFIG_CRYPTO_USER_API_HASH=m
CONFIG_CRYPTO_USER_API_SKCIPHER=m
CONFIG_CRYPTO_DEV_ATMEL_AES=y
diff --git a/arch/arm/configs/cm_x300_defconfig b/arch/arm/configs/cm_x300_defconfig
index 502a9d8..45769d0 100644
--- a/arch/arm/configs/cm_x300_defconfig
+++ b/arch/arm/configs/cm_x300_defconfig
@@ -146,7 +146,6 @@
CONFIG_NFS_V4=y
CONFIG_ROOT_NFS=y
CONFIG_CIFS=m
-CONFIG_CIFS_WEAK_PW_HASH=y
CONFIG_PARTITION_ADVANCED=y
CONFIG_NLS_CODEPAGE_437=m
CONFIG_NLS_ISO8859_1=m
diff --git a/arch/arm/configs/ezx_defconfig b/arch/arm/configs/ezx_defconfig
index a49e699..ec84d80 100644
--- a/arch/arm/configs/ezx_defconfig
+++ b/arch/arm/configs/ezx_defconfig
@@ -314,7 +314,6 @@
CONFIG_SMB_FS=m
CONFIG_CIFS=m
CONFIG_CIFS_STATS=y
-CONFIG_CIFS_WEAK_PW_HASH=y
CONFIG_CIFS_XATTR=y
CONFIG_CIFS_POSIX=y
CONFIG_NLS_CODEPAGE_437=m
diff --git a/arch/arm/configs/gemini_defconfig b/arch/arm/configs/gemini_defconfig
index e6ff844..a7acfee 100644
--- a/arch/arm/configs/gemini_defconfig
+++ b/arch/arm/configs/gemini_defconfig
@@ -63,7 +63,6 @@
CONFIG_SERIAL_8250_NR_UARTS=1
CONFIG_SERIAL_8250_RUNTIME_UARTS=1
CONFIG_SERIAL_OF_PLATFORM=y
-# CONFIG_HW_RANDOM is not set
CONFIG_I2C_GPIO=y
CONFIG_SPI=y
CONFIG_SPI_GPIO=y
@@ -99,5 +98,5 @@
CONFIG_ROMFS_FS=y
CONFIG_NLS_CODEPAGE_437=y
CONFIG_NLS_ISO8859_1=y
-# CONFIG_ENABLE_MUST_CHECK is not set
+CONFIG_CRYPTO_DEV_SL3516=y
CONFIG_DEBUG_FS=y
diff --git a/arch/arm/configs/imote2_defconfig b/arch/arm/configs/imote2_defconfig
index 118c4c9..6db871d 100644
--- a/arch/arm/configs/imote2_defconfig
+++ b/arch/arm/configs/imote2_defconfig
@@ -288,7 +288,6 @@
CONFIG_SMB_FS=m
CONFIG_CIFS=m
CONFIG_CIFS_STATS=y
-CONFIG_CIFS_WEAK_PW_HASH=y
CONFIG_CIFS_XATTR=y
CONFIG_CIFS_POSIX=y
CONFIG_NLS_CODEPAGE_437=m
diff --git a/arch/arm/configs/imx_v6_v7_defconfig b/arch/arm/configs/imx_v6_v7_defconfig
index 1fbb8e4..f7498df 100644
--- a/arch/arm/configs/imx_v6_v7_defconfig
+++ b/arch/arm/configs/imx_v6_v7_defconfig
@@ -378,6 +378,7 @@
CONFIG_RTC_DRV_PCF8563=y
CONFIG_RTC_DRV_M41T80=y
CONFIG_RTC_DRV_RC5T619=y
+CONFIG_RTC_DRV_RV3029C2=y
CONFIG_RTC_DRV_DA9063=y
CONFIG_RTC_DRV_MC13XXX=y
CONFIG_RTC_DRV_MXC=y
diff --git a/arch/arm/configs/multi_v7_defconfig b/arch/arm/configs/multi_v7_defconfig
index c951aee..8863fa9 100644
--- a/arch/arm/configs/multi_v7_defconfig
+++ b/arch/arm/configs/multi_v7_defconfig
@@ -382,6 +382,7 @@
CONFIG_TCG_TIS_I2C_INFINEON=m
CONFIG_I2C_CHARDEV=y
CONFIG_I2C_ARB_GPIO_CHALLENGE=m
+CONFIG_I2C_MUX_GPIO=y
CONFIG_I2C_MUX_PCA954x=y
CONFIG_I2C_MUX_PINCTRL=y
CONFIG_I2C_DEMUX_PINCTRL=y
@@ -680,19 +681,23 @@
CONFIG_DRM_TEGRA=y
CONFIG_DRM_STM=m
CONFIG_DRM_STM_DSI=m
+CONFIG_DRM_PANEL_LVDS=m
CONFIG_DRM_PANEL_SIMPLE=y
CONFIG_DRM_PANEL_EDP=y
CONFIG_DRM_PANEL_SAMSUNG_LD9040=m
+CONFIG_DRM_PANEL_SHARP_LQ101R1SX01=m
CONFIG_DRM_PANEL_ORISETECH_OTM8009A=m
CONFIG_DRM_PANEL_RAYDIUM_RM68200=m
CONFIG_DRM_PANEL_SAMSUNG_S6E63J0X03=m
CONFIG_DRM_PANEL_SAMSUNG_S6E8AA0=m
+CONFIG_DRM_LVDS_CODEC=m
CONFIG_DRM_NXP_PTN3460=m
CONFIG_DRM_PARADE_PS8622=m
CONFIG_DRM_SII902X=m
CONFIG_DRM_SII9234=m
CONFIG_DRM_SIMPLE_BRIDGE=m
CONFIG_DRM_TOSHIBA_TC358764=m
+CONFIG_DRM_TOSHIBA_TC358768=m
CONFIG_DRM_I2C_ADV7511=m
CONFIG_DRM_I2C_ADV7511_AUDIO=y
CONFIG_DRM_STI=m
@@ -757,6 +762,7 @@
CONFIG_SND_SUN4I_CODEC=m
CONFIG_SND_SOC_TEGRA=m
CONFIG_SND_SOC_TEGRA20_I2S=m
+CONFIG_SND_SOC_TEGRA20_SPDIF=m
CONFIG_SND_SOC_TEGRA30_I2S=m
CONFIG_SND_SOC_TEGRA_RT5640=m
CONFIG_SND_SOC_TEGRA_WM8753=m
@@ -952,6 +958,7 @@
CONFIG_STM32_DMA=y
CONFIG_STM32_DMAMUX=y
CONFIG_STM32_MDMA=y
+CONFIG_TEGRA20_APB_DMA=y
CONFIG_UNIPHIER_MDMAC=y
CONFIG_XILINX_DMA=y
CONFIG_QCOM_BAM_DMA=y
diff --git a/arch/arm/configs/nhk8815_defconfig b/arch/arm/configs/nhk8815_defconfig
index 23595fc..907d651 100644
--- a/arch/arm/configs/nhk8815_defconfig
+++ b/arch/arm/configs/nhk8815_defconfig
@@ -127,7 +127,6 @@
CONFIG_NFS_V3_ACL=y
CONFIG_ROOT_NFS=y
CONFIG_CIFS=m
-CONFIG_CIFS_WEAK_PW_HASH=y
CONFIG_NLS_CODEPAGE_437=y
CONFIG_NLS_ASCII=y
CONFIG_NLS_ISO8859_1=y
diff --git a/arch/arm/configs/pxa_defconfig b/arch/arm/configs/pxa_defconfig
index 58f4834..dedaaae 100644
--- a/arch/arm/configs/pxa_defconfig
+++ b/arch/arm/configs/pxa_defconfig
@@ -699,7 +699,6 @@
CONFIG_NFSD_V4=y
CONFIG_CIFS=m
CONFIG_CIFS_STATS=y
-CONFIG_CIFS_WEAK_PW_HASH=y
CONFIG_CIFS_XATTR=y
CONFIG_CIFS_POSIX=y
CONFIG_NLS_DEFAULT="utf8"
diff --git a/arch/arm/configs/sama5_defconfig b/arch/arm/configs/sama5_defconfig
index fe0d7cc..03dd80c 100644
--- a/arch/arm/configs/sama5_defconfig
+++ b/arch/arm/configs/sama5_defconfig
@@ -232,6 +232,14 @@
CONFIG_NLS_CODEPAGE_850=y
CONFIG_NLS_ISO8859_1=y
CONFIG_NLS_UTF8=y
+CONFIG_CRYPTO_CBC=y
+CONFIG_CRYPTO_CFB=y
+CONFIG_CRYPTO_OFB=y
+CONFIG_CRYPTO_XTS=y
+CONFIG_CRYPTO_HMAC=y
+CONFIG_CRYPTO_SHA1=y
+CONFIG_CRYPTO_SHA512=y
+CONFIG_CRYPTO_DES=y
CONFIG_CRYPTO_USER_API_HASH=m
CONFIG_CRYPTO_USER_API_SKCIPHER=m
CONFIG_CRYPTO_DEV_ATMEL_AES=y
diff --git a/arch/arm/configs/sama7_defconfig b/arch/arm/configs/sama7_defconfig
index 938aae4..0368068 100644
--- a/arch/arm/configs/sama7_defconfig
+++ b/arch/arm/configs/sama7_defconfig
@@ -33,7 +33,6 @@
CONFIG_MODULE_FORCE_LOAD=y
CONFIG_MODULE_UNLOAD=y
CONFIG_MODULE_FORCE_UNLOAD=y
-# CONFIG_BLK_DEV_BSG is not set
CONFIG_PARTITION_ADVANCED=y
# CONFIG_EFI_PARTITION is not set
# CONFIG_COREDUMP is not set
@@ -83,6 +82,7 @@
CONFIG_MTD=y
CONFIG_MTD_TESTS=m
CONFIG_MTD_CMDLINE_PARTS=y
+CONFIG_MTD_SPI_NOR=y
CONFIG_BLK_DEV_LOOP=y
CONFIG_BLK_DEV_RAM=y
CONFIG_BLK_DEV_RAM_COUNT=1
@@ -90,6 +90,7 @@
CONFIG_EEPROM_AT24=y
CONFIG_SCSI=y
CONFIG_BLK_DEV_SD=y
+# CONFIG_BLK_DEV_BSG is not set
CONFIG_NETDEVICES=y
CONFIG_MACB=y
CONFIG_MICREL_PHY=y
@@ -104,8 +105,8 @@
CONFIG_I2C_CHARDEV=y
CONFIG_I2C_AT91=y
CONFIG_SPI=y
-CONFIG_SPI_MEM=y
CONFIG_SPI_ATMEL=y
+CONFIG_SPI_ATMEL_QUADSPI=y
CONFIG_SPI_GPIO=y
CONFIG_PINCTRL_AT91=y
CONFIG_PINCTRL_AT91PIO4=y
diff --git a/arch/arm/configs/spear13xx_defconfig b/arch/arm/configs/spear13xx_defconfig
index 3b206a3..0655533 100644
--- a/arch/arm/configs/spear13xx_defconfig
+++ b/arch/arm/configs/spear13xx_defconfig
@@ -61,7 +61,6 @@
CONFIG_SERIAL_AMBA_PL011_CONSOLE=y
# CONFIG_HW_RANDOM is not set
CONFIG_RAW_DRIVER=y
-CONFIG_MAX_RAW_DEVS=8192
CONFIG_I2C=y
CONFIG_I2C_DESIGNWARE_PLATFORM=y
CONFIG_SPI=y
diff --git a/arch/arm/configs/spear3xx_defconfig b/arch/arm/configs/spear3xx_defconfig
index fc5f71c7..afca722 100644
--- a/arch/arm/configs/spear3xx_defconfig
+++ b/arch/arm/configs/spear3xx_defconfig
@@ -41,7 +41,6 @@
CONFIG_SERIAL_AMBA_PL011_CONSOLE=y
# CONFIG_HW_RANDOM is not set
CONFIG_RAW_DRIVER=y
-CONFIG_MAX_RAW_DEVS=8192
CONFIG_I2C=y
CONFIG_I2C_DESIGNWARE_PLATFORM=y
CONFIG_SPI=y
diff --git a/arch/arm/configs/spear6xx_defconfig b/arch/arm/configs/spear6xx_defconfig
index 52a56b8..bc32c02 100644
--- a/arch/arm/configs/spear6xx_defconfig
+++ b/arch/arm/configs/spear6xx_defconfig
@@ -36,7 +36,6 @@
CONFIG_SERIAL_AMBA_PL011=y
CONFIG_SERIAL_AMBA_PL011_CONSOLE=y
CONFIG_RAW_DRIVER=y
-CONFIG_MAX_RAW_DEVS=8192
CONFIG_I2C=y
CONFIG_I2C_DESIGNWARE_PLATFORM=y
CONFIG_SPI=y
diff --git a/arch/arm/configs/tegra_defconfig b/arch/arm/configs/tegra_defconfig
index 8a8f12b..289d022a 100644
--- a/arch/arm/configs/tegra_defconfig
+++ b/arch/arm/configs/tegra_defconfig
@@ -104,6 +104,8 @@
CONFIG_USB_NET_SMSC75XX=y
CONFIG_USB_NET_SMSC95XX=y
CONFIG_BRCMFMAC=m
+CONFIG_MWIFIEX=m
+CONFIG_MWIFIEX_SDIO=m
CONFIG_RT2X00=y
CONFIG_RT2800USB=m
CONFIG_INPUT_JOYDEV=y
@@ -112,6 +114,8 @@
CONFIG_KEYBOARD_TEGRA=y
CONFIG_KEYBOARD_CROS_EC=y
CONFIG_KEYBOARD_CAP11XX=y
+CONFIG_MOUSE_ELAN_I2C=m
+CONFIG_MOUSE_ELAN_I2C_SMBUS=y
CONFIG_MOUSE_PS2_ELANTECH=y
CONFIG_INPUT_TOUCHSCREEN=y
CONFIG_TOUCHSCREEN_ATMEL_MXT=y
@@ -131,6 +135,7 @@
# CONFIG_HW_RANDOM is not set
# CONFIG_I2C_COMPAT is not set
CONFIG_I2C_CHARDEV=y
+CONFIG_I2C_MUX_GPIO=y
CONFIG_I2C_MUX_PCA954x=y
CONFIG_I2C_MUX_PINCTRL=y
CONFIG_I2C_TEGRA=y
@@ -151,7 +156,9 @@
CONFIG_POWER_RESET=y
CONFIG_POWER_RESET_AS3722=y
CONFIG_POWER_RESET_GPIO=y
+CONFIG_POWER_RESET_GPIO_RESTART=y
CONFIG_BATTERY_SBS=y
+CONFIG_CHARGER_BQ24735=y
CONFIG_BATTERY_BQ27XXX=y
CONFIG_CHARGER_GPIO=y
CONFIG_CHARGER_SMB347=y
@@ -200,7 +207,9 @@
CONFIG_DRM_PANEL_LVDS=y
CONFIG_DRM_PANEL_SIMPLE=y
CONFIG_DRM_PANEL_EDP=y
+CONFIG_DRM_PANEL_SHARP_LQ101R1SX01=y
CONFIG_DRM_LVDS_CODEC=y
+CONFIG_DRM_TOSHIBA_TC358768=y
CONFIG_FB=y
CONFIG_BACKLIGHT_CLASS_DEVICE=y
CONFIG_BACKLIGHT_PWM=y
@@ -222,6 +231,7 @@
CONFIG_SND_SOC=y
CONFIG_SND_SOC_TEGRA=y
CONFIG_SND_SOC_TEGRA20_I2S=y
+CONFIG_SND_SOC_TEGRA20_SPDIF=y
CONFIG_SND_SOC_TEGRA30_I2S=y
CONFIG_SND_SOC_TEGRA_RT5640=y
CONFIG_SND_SOC_TEGRA_WM8753=y
@@ -279,6 +289,9 @@
CONFIG_TEGRA_VDE=y
CONFIG_CHROME_PLATFORMS=y
CONFIG_CROS_EC=y
+CONFIG_CROS_EC_I2C=m
+CONFIG_CROS_EC_SPI=m
+CONFIG_I2C_CROS_EC_TUNNEL=m
CONFIG_TEGRA_IOMMU_GART=y
CONFIG_TEGRA_IOMMU_SMMU=y
CONFIG_ARCH_TEGRA_2x_SOC=y
diff --git a/arch/arm/crypto/Makefile b/arch/arm/crypto/Makefile
index eafa898..0274f81 100644
--- a/arch/arm/crypto/Makefile
+++ b/arch/arm/crypto/Makefile
@@ -10,6 +10,7 @@
obj-$(CONFIG_CRYPTO_SHA256_ARM) += sha256-arm.o
obj-$(CONFIG_CRYPTO_SHA512_ARM) += sha512-arm.o
obj-$(CONFIG_CRYPTO_BLAKE2S_ARM) += blake2s-arm.o
+obj-$(if $(CONFIG_CRYPTO_BLAKE2S_ARM),y) += libblake2s-arm.o
obj-$(CONFIG_CRYPTO_BLAKE2B_NEON) += blake2b-neon.o
obj-$(CONFIG_CRYPTO_CHACHA20_NEON) += chacha-neon.o
obj-$(CONFIG_CRYPTO_POLY1305_ARM) += poly1305-arm.o
@@ -31,7 +32,8 @@
sha256-arm-y := sha256-core.o sha256_glue.o $(sha256-arm-neon-y)
sha512-arm-neon-$(CONFIG_KERNEL_MODE_NEON) := sha512-neon-glue.o
sha512-arm-y := sha512-core.o sha512-glue.o $(sha512-arm-neon-y)
-blake2s-arm-y := blake2s-core.o blake2s-glue.o
+blake2s-arm-y := blake2s-shash.o
+libblake2s-arm-y:= blake2s-core.o blake2s-glue.o
blake2b-neon-y := blake2b-neon-core.o blake2b-neon-glue.o
sha1-arm-ce-y := sha1-ce-core.o sha1-ce-glue.o
sha2-arm-ce-y := sha2-ce-core.o sha2-ce-glue.o
diff --git a/arch/arm/crypto/blake2s-core.S b/arch/arm/crypto/blake2s-core.S
index 8634575..df40e46 100644
--- a/arch/arm/crypto/blake2s-core.S
+++ b/arch/arm/crypto/blake2s-core.S
@@ -167,8 +167,8 @@
.endm
//
-// void blake2s_compress_arch(struct blake2s_state *state,
-// const u8 *block, size_t nblocks, u32 inc);
+// void blake2s_compress(struct blake2s_state *state,
+// const u8 *block, size_t nblocks, u32 inc);
//
// Only the first three fields of struct blake2s_state are used:
// u32 h[8]; (inout)
@@ -176,7 +176,7 @@
// u32 f[2]; (in)
//
.align 5
-ENTRY(blake2s_compress_arch)
+ENTRY(blake2s_compress)
push {r0-r2,r4-r11,lr} // keep this an even number
.Lnext_block:
@@ -303,4 +303,4 @@
str r3, [r12], #4
bne 1b
b .Lcopy_block_done
-ENDPROC(blake2s_compress_arch)
+ENDPROC(blake2s_compress)
diff --git a/arch/arm/crypto/blake2s-glue.c b/arch/arm/crypto/blake2s-glue.c
index f2cc1e5..0238a70 100644
--- a/arch/arm/crypto/blake2s-glue.c
+++ b/arch/arm/crypto/blake2s-glue.c
@@ -1,78 +1,7 @@
// SPDX-License-Identifier: GPL-2.0-or-later
-/*
- * BLAKE2s digest algorithm, ARM scalar implementation
- *
- * Copyright 2020 Google LLC
- */
#include <crypto/internal/blake2s.h>
-#include <crypto/internal/hash.h>
-
#include <linux/module.h>
/* defined in blake2s-core.S */
-EXPORT_SYMBOL(blake2s_compress_arch);
-
-static int crypto_blake2s_update_arm(struct shash_desc *desc,
- const u8 *in, unsigned int inlen)
-{
- return crypto_blake2s_update(desc, in, inlen, blake2s_compress_arch);
-}
-
-static int crypto_blake2s_final_arm(struct shash_desc *desc, u8 *out)
-{
- return crypto_blake2s_final(desc, out, blake2s_compress_arch);
-}
-
-#define BLAKE2S_ALG(name, driver_name, digest_size) \
- { \
- .base.cra_name = name, \
- .base.cra_driver_name = driver_name, \
- .base.cra_priority = 200, \
- .base.cra_flags = CRYPTO_ALG_OPTIONAL_KEY, \
- .base.cra_blocksize = BLAKE2S_BLOCK_SIZE, \
- .base.cra_ctxsize = sizeof(struct blake2s_tfm_ctx), \
- .base.cra_module = THIS_MODULE, \
- .digestsize = digest_size, \
- .setkey = crypto_blake2s_setkey, \
- .init = crypto_blake2s_init, \
- .update = crypto_blake2s_update_arm, \
- .final = crypto_blake2s_final_arm, \
- .descsize = sizeof(struct blake2s_state), \
- }
-
-static struct shash_alg blake2s_arm_algs[] = {
- BLAKE2S_ALG("blake2s-128", "blake2s-128-arm", BLAKE2S_128_HASH_SIZE),
- BLAKE2S_ALG("blake2s-160", "blake2s-160-arm", BLAKE2S_160_HASH_SIZE),
- BLAKE2S_ALG("blake2s-224", "blake2s-224-arm", BLAKE2S_224_HASH_SIZE),
- BLAKE2S_ALG("blake2s-256", "blake2s-256-arm", BLAKE2S_256_HASH_SIZE),
-};
-
-static int __init blake2s_arm_mod_init(void)
-{
- return IS_REACHABLE(CONFIG_CRYPTO_HASH) ?
- crypto_register_shashes(blake2s_arm_algs,
- ARRAY_SIZE(blake2s_arm_algs)) : 0;
-}
-
-static void __exit blake2s_arm_mod_exit(void)
-{
- if (IS_REACHABLE(CONFIG_CRYPTO_HASH))
- crypto_unregister_shashes(blake2s_arm_algs,
- ARRAY_SIZE(blake2s_arm_algs));
-}
-
-module_init(blake2s_arm_mod_init);
-module_exit(blake2s_arm_mod_exit);
-
-MODULE_DESCRIPTION("BLAKE2s digest algorithm, ARM scalar implementation");
-MODULE_LICENSE("GPL");
-MODULE_AUTHOR("Eric Biggers <ebiggers@google.com>");
-MODULE_ALIAS_CRYPTO("blake2s-128");
-MODULE_ALIAS_CRYPTO("blake2s-128-arm");
-MODULE_ALIAS_CRYPTO("blake2s-160");
-MODULE_ALIAS_CRYPTO("blake2s-160-arm");
-MODULE_ALIAS_CRYPTO("blake2s-224");
-MODULE_ALIAS_CRYPTO("blake2s-224-arm");
-MODULE_ALIAS_CRYPTO("blake2s-256");
-MODULE_ALIAS_CRYPTO("blake2s-256-arm");
+EXPORT_SYMBOL(blake2s_compress);
diff --git a/arch/arm/crypto/blake2s-shash.c b/arch/arm/crypto/blake2s-shash.c
new file mode 100644
index 0000000..17c1c3bf
--- /dev/null
+++ b/arch/arm/crypto/blake2s-shash.c
@@ -0,0 +1,75 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * BLAKE2s digest algorithm, ARM scalar implementation
+ *
+ * Copyright 2020 Google LLC
+ */
+
+#include <crypto/internal/blake2s.h>
+#include <crypto/internal/hash.h>
+
+#include <linux/module.h>
+
+static int crypto_blake2s_update_arm(struct shash_desc *desc,
+ const u8 *in, unsigned int inlen)
+{
+ return crypto_blake2s_update(desc, in, inlen, blake2s_compress);
+}
+
+static int crypto_blake2s_final_arm(struct shash_desc *desc, u8 *out)
+{
+ return crypto_blake2s_final(desc, out, blake2s_compress);
+}
+
+#define BLAKE2S_ALG(name, driver_name, digest_size) \
+ { \
+ .base.cra_name = name, \
+ .base.cra_driver_name = driver_name, \
+ .base.cra_priority = 200, \
+ .base.cra_flags = CRYPTO_ALG_OPTIONAL_KEY, \
+ .base.cra_blocksize = BLAKE2S_BLOCK_SIZE, \
+ .base.cra_ctxsize = sizeof(struct blake2s_tfm_ctx), \
+ .base.cra_module = THIS_MODULE, \
+ .digestsize = digest_size, \
+ .setkey = crypto_blake2s_setkey, \
+ .init = crypto_blake2s_init, \
+ .update = crypto_blake2s_update_arm, \
+ .final = crypto_blake2s_final_arm, \
+ .descsize = sizeof(struct blake2s_state), \
+ }
+
+static struct shash_alg blake2s_arm_algs[] = {
+ BLAKE2S_ALG("blake2s-128", "blake2s-128-arm", BLAKE2S_128_HASH_SIZE),
+ BLAKE2S_ALG("blake2s-160", "blake2s-160-arm", BLAKE2S_160_HASH_SIZE),
+ BLAKE2S_ALG("blake2s-224", "blake2s-224-arm", BLAKE2S_224_HASH_SIZE),
+ BLAKE2S_ALG("blake2s-256", "blake2s-256-arm", BLAKE2S_256_HASH_SIZE),
+};
+
+static int __init blake2s_arm_mod_init(void)
+{
+ return IS_REACHABLE(CONFIG_CRYPTO_HASH) ?
+ crypto_register_shashes(blake2s_arm_algs,
+ ARRAY_SIZE(blake2s_arm_algs)) : 0;
+}
+
+static void __exit blake2s_arm_mod_exit(void)
+{
+ if (IS_REACHABLE(CONFIG_CRYPTO_HASH))
+ crypto_unregister_shashes(blake2s_arm_algs,
+ ARRAY_SIZE(blake2s_arm_algs));
+}
+
+module_init(blake2s_arm_mod_init);
+module_exit(blake2s_arm_mod_exit);
+
+MODULE_DESCRIPTION("BLAKE2s digest algorithm, ARM scalar implementation");
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Eric Biggers <ebiggers@google.com>");
+MODULE_ALIAS_CRYPTO("blake2s-128");
+MODULE_ALIAS_CRYPTO("blake2s-128-arm");
+MODULE_ALIAS_CRYPTO("blake2s-160");
+MODULE_ALIAS_CRYPTO("blake2s-160-arm");
+MODULE_ALIAS_CRYPTO("blake2s-224");
+MODULE_ALIAS_CRYPTO("blake2s-224-arm");
+MODULE_ALIAS_CRYPTO("blake2s-256");
+MODULE_ALIAS_CRYPTO("blake2s-256-arm");
diff --git a/arch/arm/include/asm/efi.h b/arch/arm/include/asm/efi.h
index a6f3b17..27218ea 100644
--- a/arch/arm/include/asm/efi.h
+++ b/arch/arm/include/asm/efi.h
@@ -17,7 +17,6 @@
#ifdef CONFIG_EFI
void efi_init(void);
-extern void efifb_setup_from_dmi(struct screen_info *si, const char *opt);
int efi_create_mapping(struct mm_struct *mm, efi_memory_desc_t *md);
int efi_set_mapping_permissions(struct mm_struct *mm, efi_memory_desc_t *md);
diff --git a/arch/arm/include/debug/imx-uart.h b/arch/arm/include/debug/imx-uart.h
index c8eb83d..3edbb3c 100644
--- a/arch/arm/include/debug/imx-uart.h
+++ b/arch/arm/include/debug/imx-uart.h
@@ -11,13 +11,6 @@
#define IMX1_UART_BASE_ADDR(n) IMX1_UART##n##_BASE_ADDR
#define IMX1_UART_BASE(n) IMX1_UART_BASE_ADDR(n)
-#define IMX21_UART1_BASE_ADDR 0x1000a000
-#define IMX21_UART2_BASE_ADDR 0x1000b000
-#define IMX21_UART3_BASE_ADDR 0x1000c000
-#define IMX21_UART4_BASE_ADDR 0x1000d000
-#define IMX21_UART_BASE_ADDR(n) IMX21_UART##n##_BASE_ADDR
-#define IMX21_UART_BASE(n) IMX21_UART_BASE_ADDR(n)
-
#define IMX25_UART1_BASE_ADDR 0x43f90000
#define IMX25_UART2_BASE_ADDR 0x43f94000
#define IMX25_UART3_BASE_ADDR 0x5000c000
@@ -26,6 +19,13 @@
#define IMX25_UART_BASE_ADDR(n) IMX25_UART##n##_BASE_ADDR
#define IMX25_UART_BASE(n) IMX25_UART_BASE_ADDR(n)
+#define IMX27_UART1_BASE_ADDR 0x1000a000
+#define IMX27_UART2_BASE_ADDR 0x1000b000
+#define IMX27_UART3_BASE_ADDR 0x1000c000
+#define IMX27_UART4_BASE_ADDR 0x1000d000
+#define IMX27_UART_BASE_ADDR(n) IMX27_UART##n##_BASE_ADDR
+#define IMX27_UART_BASE(n) IMX27_UART_BASE_ADDR(n)
+
#define IMX31_UART1_BASE_ADDR 0x43f90000
#define IMX31_UART2_BASE_ADDR 0x43f94000
#define IMX31_UART3_BASE_ADDR 0x5000c000
@@ -112,10 +112,10 @@
#ifdef CONFIG_DEBUG_IMX1_UART
#define UART_PADDR IMX_DEBUG_UART_BASE(IMX1)
-#elif defined(CONFIG_DEBUG_IMX21_IMX27_UART)
-#define UART_PADDR IMX_DEBUG_UART_BASE(IMX21)
#elif defined(CONFIG_DEBUG_IMX25_UART)
#define UART_PADDR IMX_DEBUG_UART_BASE(IMX25)
+#elif defined(CONFIG_DEBUG_IMX27_UART)
+#define UART_PADDR IMX_DEBUG_UART_BASE(IMX27)
#elif defined(CONFIG_DEBUG_IMX31_UART)
#define UART_PADDR IMX_DEBUG_UART_BASE(IMX31)
#elif defined(CONFIG_DEBUG_IMX35_UART)
diff --git a/arch/arm/kernel/signal.c b/arch/arm/kernel/signal.c
index a41e27a..c532a60 100644
--- a/arch/arm/kernel/signal.c
+++ b/arch/arm/kernel/signal.c
@@ -631,7 +631,7 @@ do_work_pending(struct pt_regs *regs, unsigned int thread_flags, int syscall)
}
}
local_irq_disable();
- thread_flags = current_thread_info()->flags;
+ thread_flags = read_thread_flags();
} while (thread_flags & _TIF_WORK_MASK);
return 0;
}
diff --git a/arch/arm/mach-at91/pm.c b/arch/arm/mach-at91/pm.c
index 8711d68..dd6f4ce 100644
--- a/arch/arm/mach-at91/pm.c
+++ b/arch/arm/mach-at91/pm.c
@@ -645,6 +645,7 @@ static __init int at91_dt_ramc(bool phy_mandatory)
if (!soc_pm.data.ramc[idx]) {
pr_err("unable to map ramc[%d] cpu registers\n", idx);
ret = -ENOMEM;
+ of_node_put(np);
goto unmap_ramc;
}
@@ -670,6 +671,7 @@ static __init int at91_dt_ramc(bool phy_mandatory)
if (!soc_pm.data.ramc_phy) {
pr_err("unable to map ramc phy cpu registers\n");
ret = -ENOMEM;
+ of_node_put(np);
goto unmap_ramc;
}
}
diff --git a/arch/arm/mach-exynos/Kconfig b/arch/arm/mach-exynos/Kconfig
index dd1ae55..f7d9936 100644
--- a/arch/arm/mach-exynos/Kconfig
+++ b/arch/arm/mach-exynos/Kconfig
@@ -20,7 +20,6 @@
select GPIOLIB
select HAVE_ARM_ARCH_TIMER if ARCH_EXYNOS5
select HAVE_ARM_SCU if SMP
- select HAVE_S3C2410_I2C if I2C
select PINCTRL
select PINCTRL_EXYNOS
select PM_GENERIC_DOMAINS if PM
diff --git a/arch/arm/mach-imx/Kconfig b/arch/arm/mach-imx/Kconfig
index b407b02..f296bac 100644
--- a/arch/arm/mach-imx/Kconfig
+++ b/arch/arm/mach-imx/Kconfig
@@ -21,18 +21,6 @@
config MXC_AVIC
bool
-config MXC_DEBUG_BOARD
- bool "Enable MXC debug board(for 3-stack)"
- depends on MACH_MX27_3DS || MACH_MX31_3DS || MACH_MX35_3DS
- help
- The debug board is an integral part of the MXC 3-stack(PDK)
- platforms, it can be attached or removed from the peripheral
- board. On debug board, several debug devices(ethernet, UART,
- buttons, LEDs and JTAG) are implemented. Between the MCU and
- these devices, a CPLD is added as a bridge which performs
- data/address de-multiplexing and decode, signal level shift,
- interrupt control and various board functions.
-
config HAVE_IMX_ANATOP
bool
diff --git a/arch/arm/mach-imx/Makefile b/arch/arm/mach-imx/Makefile
index d1506ef..d5291ed 100644
--- a/arch/arm/mach-imx/Makefile
+++ b/arch/arm/mach-imx/Makefile
@@ -14,8 +14,6 @@
obj-$(CONFIG_MXC_TZIC) += tzic.o
obj-$(CONFIG_MXC_AVIC) += avic.o
-obj-$(CONFIG_MXC_DEBUG_BOARD) += 3ds_debugboard.o
-
ifeq ($(CONFIG_CPU_IDLE),y)
obj-$(CONFIG_SOC_IMX5) += cpuidle-imx5.o
obj-$(CONFIG_SOC_IMX6Q) += cpuidle-imx6q.o
diff --git a/arch/arm/mach-ixp4xx/Kconfig b/arch/arm/mach-ixp4xx/Kconfig
index 365a585..4c787b4 100644
--- a/arch/arm/mach-ixp4xx/Kconfig
+++ b/arch/arm/mach-ixp4xx/Kconfig
@@ -39,19 +39,6 @@
PrPCM1100 Processor Mezanine Module. For more information on
this platform, see <file:Documentation/arm/ixp4xx.rst>.
-#
-# Certain registers and IRQs are only enabled if supporting IXP465 CPUs
-#
-config CPU_IXP46X
- bool
- depends on MACH_IXDP465
- default y
-
-config CPU_IXP43X
- bool
- depends on MACH_KIXRP435
- default y
-
comment "IXP4xx Options"
config IXP4XX_PCI_LEGACY
diff --git a/arch/arm/mach-s3c/Kconfig.s3c64xx b/arch/arm/mach-s3c/Kconfig.s3c64xx
index f3fcb57..af01675 100644
--- a/arch/arm/mach-s3c/Kconfig.s3c64xx
+++ b/arch/arm/mach-s3c/Kconfig.s3c64xx
@@ -12,7 +12,6 @@
select COMMON_CLK_SAMSUNG
select GPIO_SAMSUNG if ATAGS
select GPIOLIB
- select HAVE_S3C2410_I2C if I2C
select HAVE_TCM
select PLAT_SAMSUNG
select PM_GENERIC_DOMAINS if PM
diff --git a/arch/arm/mach-s3c/gpio-samsung.c b/arch/arm/mach-s3c/gpio-samsung.c
index 76ef415..fda2c01 100644
--- a/arch/arm/mach-s3c/gpio-samsung.c
+++ b/arch/arm/mach-s3c/gpio-samsung.c
@@ -35,6 +35,7 @@
#include "gpio-core.h"
#include "gpio-cfg.h"
#include "gpio-cfg-helpers.h"
+#include "hardware-s3c24xx.h"
#include "pm.h"
int samsung_gpio_setpull_updown(struct samsung_gpio_chip *chip,
diff --git a/arch/arm/mach-s3c/mach-jive.c b/arch/arm/mach-s3c/mach-jive.c
index 0785638..285e1f0 100644
--- a/arch/arm/mach-s3c/mach-jive.c
+++ b/arch/arm/mach-s3c/mach-jive.c
@@ -247,6 +247,7 @@ static int __init jive_mtdset(char *options)
case 1:
nand->nr_partitions = ARRAY_SIZE(jive_imageB_nand_part);
nand->partitions = jive_imageB_nand_part;
+ break;
case 0:
/* this is already setup in the nand info */
break;
diff --git a/arch/arm/mach-s5pv210/Kconfig b/arch/arm/mach-s5pv210/Kconfig
index 5a96099..055de57 100644
--- a/arch/arm/mach-s5pv210/Kconfig
+++ b/arch/arm/mach-s5pv210/Kconfig
@@ -12,7 +12,6 @@
select CLKSRC_SAMSUNG_PWM
select COMMON_CLK_SAMSUNG
select GPIOLIB
- select HAVE_S3C2410_I2C if I2C
select PINCTRL
select PINCTRL_EXYNOS
select SOC_SAMSUNG
diff --git a/arch/arm/mach-shmobile/regulator-quirk-rcar-gen2.c b/arch/arm/mach-shmobile/regulator-quirk-rcar-gen2.c
index ee94925..09ef73b 100644
--- a/arch/arm/mach-shmobile/regulator-quirk-rcar-gen2.c
+++ b/arch/arm/mach-shmobile/regulator-quirk-rcar-gen2.c
@@ -154,8 +154,10 @@ static int __init rcar_gen2_regulator_quirk(void)
return -ENODEV;
for_each_matching_node_and_match(np, rcar_gen2_quirk_match, &id) {
- if (!of_device_is_available(np))
+ if (!of_device_is_available(np)) {
+ of_node_put(np);
break;
+ }
ret = of_property_read_u32(np, "reg", &addr);
if (ret) /* Skip invalid entry and continue */
@@ -164,6 +166,7 @@ static int __init rcar_gen2_regulator_quirk(void)
quirk = kzalloc(sizeof(*quirk), GFP_KERNEL);
if (!quirk) {
ret = -ENOMEM;
+ of_node_put(np);
goto err_mem;
}
diff --git a/arch/arm/mm/alignment.c b/arch/arm/mm/alignment.c
index ea81e89..adbb381 100644
--- a/arch/arm/mm/alignment.c
+++ b/arch/arm/mm/alignment.c
@@ -990,7 +990,7 @@ do_alignment(unsigned long addr, unsigned int fsr, struct pt_regs *regs)
* there is no work pending for this thread.
*/
raw_local_irq_disable();
- if (!(current_thread_info()->flags & _TIF_WORK_MASK))
+ if (!(read_thread_flags() & _TIF_WORK_MASK))
set_cr(cr_no_alignment);
}
diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
index eeb6dc0..10ceebb7 100644
--- a/arch/arm/net/bpf_jit_32.c
+++ b/arch/arm/net/bpf_jit_32.c
@@ -163,7 +163,7 @@ static const s8 bpf2a32[][2] = {
[BPF_REG_9] = {STACK_OFFSET(BPF_R9_HI), STACK_OFFSET(BPF_R9_LO)},
/* Read only Frame Pointer to access Stack */
[BPF_REG_FP] = {STACK_OFFSET(BPF_FP_HI), STACK_OFFSET(BPF_FP_LO)},
- /* Temporary Register for internal BPF JIT, can be used
+ /* Temporary Register for BPF JIT, can be used
* for constant blindings and others.
*/
[TMP_REG_1] = {ARM_R7, ARM_R6},
@@ -1199,7 +1199,8 @@ static int emit_bpf_tail_call(struct jit_ctx *ctx)
/* tmp2[0] = array, tmp2[1] = index */
- /* if (tail_call_cnt > MAX_TAIL_CALL_CNT)
+ /*
+ * if (tail_call_cnt >= MAX_TAIL_CALL_CNT)
* goto out;
* tail_call_cnt++;
*/
@@ -1208,7 +1209,7 @@ static int emit_bpf_tail_call(struct jit_ctx *ctx)
tc = arm_bpf_get_reg64(tcc, tmp, ctx);
emit(ARM_CMP_I(tc[0], hi), ctx);
_emit(ARM_COND_EQ, ARM_CMP_I(tc[1], lo), ctx);
- _emit(ARM_COND_HI, ARM_B(jmp_offset), ctx);
+ _emit(ARM_COND_CS, ARM_B(jmp_offset), ctx);
emit(ARM_ADDS_I(tc[1], tc[1], 1), ctx);
emit(ARM_ADC_I(tc[0], tc[0], 0), ctx);
arm_bpf_put_reg64(tcc, tmp, ctx);
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index c4207cf..ef3b5cb 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -150,6 +150,8 @@
select HAVE_ARCH_KASAN_VMALLOC if HAVE_ARCH_KASAN
select HAVE_ARCH_KASAN_SW_TAGS if HAVE_ARCH_KASAN
select HAVE_ARCH_KASAN_HW_TAGS if (HAVE_ARCH_KASAN && ARM64_MTE)
+ # Some instrumentation may be unsound, hence EXPERT
+ select HAVE_ARCH_KCSAN if EXPERT
select HAVE_ARCH_KFENCE
select HAVE_ARCH_KGDB
select HAVE_ARCH_MMAP_RND_BITS
@@ -1545,6 +1547,12 @@
menu "ARMv8.2 architectural features"
+config AS_HAS_ARMV8_2
+ def_bool $(cc-option,-Wa$(comma)-march=armv8.2-a)
+
+config AS_HAS_SHA3
+ def_bool $(as-instr,.arch armv8.2-a+sha3)
+
config ARM64_PMEM
bool "Enable support for persistent memory"
select ARCH_HAS_PMEM_API
diff --git a/arch/arm64/Kconfig.platforms b/arch/arm64/Kconfig.platforms
index 54e3910..7d5d588 100644
--- a/arch/arm64/Kconfig.platforms
+++ b/arch/arm64/Kconfig.platforms
@@ -89,6 +89,7 @@
config ARCH_EXYNOS
bool "ARMv8 based Samsung Exynos SoC family"
select COMMON_CLK_SAMSUNG
+ select CLKSRC_EXYNOS_MCT
select EXYNOS_PM_DOMAINS if PM_GENERIC_DOMAINS
select EXYNOS_PMU
select PINCTRL
diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
index e8cfc58..2f1de88 100644
--- a/arch/arm64/Makefile
+++ b/arch/arm64/Makefile
@@ -58,6 +58,11 @@
include/generated/asm-offsets.h))
endif
+ifeq ($(CONFIG_AS_HAS_ARMV8_2), y)
+# make sure to pass the newest target architecture to -march.
+asm-arch := armv8.2-a
+endif
+
# Ensure that if the compiler supports branch protection we default it
# off, this will be overridden if we are using branch protection.
branch-prot-flags-y += $(call cc-option,-mbranch-protection=none)
diff --git a/arch/arm64/boot/dts/allwinner/Makefile b/arch/arm64/boot/dts/allwinner/Makefile
index a96d9d2..8fa5c06 100644
--- a/arch/arm64/boot/dts/allwinner/Makefile
+++ b/arch/arm64/boot/dts/allwinner/Makefile
@@ -37,3 +37,4 @@
dtb-$(CONFIG_ARCH_SUNXI) += sun50i-h6-pine-h64.dtb
dtb-$(CONFIG_ARCH_SUNXI) += sun50i-h6-pine-h64-model-b.dtb
dtb-$(CONFIG_ARCH_SUNXI) += sun50i-h6-tanix-tx6.dtb
+dtb-$(CONFIG_ARCH_SUNXI) += sun50i-h6-tanix-tx6-mini.dtb
diff --git a/arch/arm64/boot/dts/allwinner/sun50i-a64-pinebook.dts b/arch/arm64/boot/dts/allwinner/sun50i-a64-pinebook.dts
index 34e67f5..63571df 100644
--- a/arch/arm64/boot/dts/allwinner/sun50i-a64-pinebook.dts
+++ b/arch/arm64/boot/dts/allwinner/sun50i-a64-pinebook.dts
@@ -15,6 +15,7 @@
/ {
model = "Pinebook";
compatible = "pine64,pinebook", "allwinner,sun50i-a64";
+ chassis-type = "laptop";
aliases {
serial0 = &uart0;
diff --git a/arch/arm64/boot/dts/allwinner/sun50i-a64-pinephone.dtsi b/arch/arm64/boot/dts/allwinner/sun50i-a64-pinephone.dtsi
index 5b44a97..8784711 100644
--- a/arch/arm64/boot/dts/allwinner/sun50i-a64-pinephone.dtsi
+++ b/arch/arm64/boot/dts/allwinner/sun50i-a64-pinephone.dtsi
@@ -12,6 +12,8 @@
#include <dt-bindings/pwm/pwm.h>
/ {
+ chassis-type = "handset";
+
aliases {
ethernet0 = &rtl8723cs;
serial0 = &uart0;
diff --git a/arch/arm64/boot/dts/allwinner/sun50i-a64-pinetab.dts b/arch/arm64/boot/dts/allwinner/sun50i-a64-pinetab.dts
index adb0b28..0a5607f 100644
--- a/arch/arm64/boot/dts/allwinner/sun50i-a64-pinetab.dts
+++ b/arch/arm64/boot/dts/allwinner/sun50i-a64-pinetab.dts
@@ -16,6 +16,7 @@
/ {
model = "PineTab, Development Sample";
compatible = "pine64,pinetab", "allwinner,sun50i-a64";
+ chassis-type = "tablet";
aliases {
serial0 = &uart0;
diff --git a/arch/arm64/boot/dts/allwinner/sun50i-a64-teres-i.dts b/arch/arm64/boot/dts/allwinner/sun50i-a64-teres-i.dts
index aef571a..aff0660 100644
--- a/arch/arm64/boot/dts/allwinner/sun50i-a64-teres-i.dts
+++ b/arch/arm64/boot/dts/allwinner/sun50i-a64-teres-i.dts
@@ -14,6 +14,7 @@
/ {
model = "Olimex A64 Teres-I";
compatible = "olimex,a64-teres-i", "allwinner,sun50i-a64";
+ chassis-type = "laptop";
aliases {
serial0 = &uart0;
diff --git a/arch/arm64/boot/dts/allwinner/sun50i-a64.dtsi b/arch/arm64/boot/dts/allwinner/sun50i-a64.dtsi
index 5ba3790..de77c87 100644
--- a/arch/arm64/boot/dts/allwinner/sun50i-a64.dtsi
+++ b/arch/arm64/boot/dts/allwinner/sun50i-a64.dtsi
@@ -1146,8 +1146,14 @@ pwm: pwm@1c21400 {
mbus: dram-controller@1c62000 {
compatible = "allwinner,sun50i-a64-mbus";
- reg = <0x01c62000 0x1000>;
- clocks = <&ccu 112>;
+ reg = <0x01c62000 0x1000>,
+ <0x01c63000 0x1000>;
+ reg-names = "mbus", "dram";
+ clocks = <&ccu CLK_MBUS>,
+ <&ccu CLK_DRAM>,
+ <&ccu CLK_BUS_DRAM>;
+ clock-names = "mbus", "dram", "bus";
+ interrupts = <GIC_SPI 69 IRQ_TYPE_LEVEL_HIGH>;
#address-cells = <1>;
#size-cells = <1>;
dma-ranges = <0x00000000 0x40000000 0xc0000000>;
@@ -1220,8 +1226,8 @@ hdmi: hdmi@1ee0000 {
reg-io-width = <1>;
interrupts = <GIC_SPI 88 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&ccu CLK_BUS_HDMI>, <&ccu CLK_HDMI_DDC>,
- <&ccu CLK_HDMI>;
- clock-names = "iahb", "isfr", "tmds";
+ <&ccu CLK_HDMI>, <&rtc 0>;
+ clock-names = "iahb", "isfr", "tmds", "cec";
resets = <&ccu RST_BUS_HDMI1>;
reset-names = "ctrl";
phys = <&hdmi_phy>;
diff --git a/arch/arm64/boot/dts/allwinner/sun50i-h5.dtsi b/arch/arm64/boot/dts/allwinner/sun50i-h5.dtsi
index 9988e87..a56fae7 100644
--- a/arch/arm64/boot/dts/allwinner/sun50i-h5.dtsi
+++ b/arch/arm64/boot/dts/allwinner/sun50i-h5.dtsi
@@ -233,6 +233,10 @@ &display_clocks {
compatible = "allwinner,sun50i-h5-de2-clk";
};
+&mbus {
+ compatible = "allwinner,sun50i-h5-mbus";
+};
+
&mmc0 {
compatible = "allwinner,sun50i-h5-mmc",
"allwinner,sun50i-a64-mmc";
diff --git a/arch/arm64/boot/dts/allwinner/sun50i-h6-tanix-tx6-mini.dts b/arch/arm64/boot/dts/allwinner/sun50i-h6-tanix-tx6-mini.dts
new file mode 100644
index 0000000..08d8416
--- /dev/null
+++ b/arch/arm64/boot/dts/allwinner/sun50i-h6-tanix-tx6-mini.dts
@@ -0,0 +1,15 @@
+// SPDX-License-Identifier: (GPL-2.0+ OR MIT)
+// Copyright (c) 2021 Jernej Skrabec <jernej.skrabec@gmail.com>
+
+/dts-v1/;
+
+#include "sun50i-h6-tanix.dtsi"
+
+/ {
+ model = "Tanix TX6 mini";
+ compatible = "oranth,tanix-tx6-mini", "allwinner,sun50i-h6";
+};
+
+&r_ir {
+ linux,rc-map-name = "rc-tanix-tx3mini";
+};
diff --git a/arch/arm64/boot/dts/allwinner/sun50i-h6-tanix-tx6.dts b/arch/arm64/boot/dts/allwinner/sun50i-h6-tanix-tx6.dts
index 8f2a80f..9a38ff9 100644
--- a/arch/arm64/boot/dts/allwinner/sun50i-h6-tanix-tx6.dts
+++ b/arch/arm64/boot/dts/allwinner/sun50i-h6-tanix-tx6.dts
@@ -3,145 +3,27 @@
/dts-v1/;
-#include "sun50i-h6.dtsi"
-#include "sun50i-h6-cpu-opp.dtsi"
-
-#include <dt-bindings/gpio/gpio.h>
+#include "sun50i-h6-tanix.dtsi"
/ {
model = "Tanix TX6";
compatible = "oranth,tanix-tx6", "allwinner,sun50i-h6";
-
- aliases {
- serial0 = &uart0;
- };
-
- chosen {
- stdout-path = "serial0:115200n8";
- };
-
- connector {
- compatible = "hdmi-connector";
- ddc-en-gpios = <&pio 7 2 GPIO_ACTIVE_HIGH>; /* PH2 */
- type = "a";
-
- port {
- hdmi_con_in: endpoint {
- remote-endpoint = <&hdmi_out_con>;
- };
- };
- };
-
- reg_vcc1v8: regulator-vcc1v8 {
- compatible = "regulator-fixed";
- regulator-name = "vcc1v8";
- regulator-min-microvolt = <1800000>;
- regulator-max-microvolt = <1800000>;
- };
-
- reg_vcc3v3: regulator-vcc3v3 {
- compatible = "regulator-fixed";
- regulator-name = "vcc3v3";
- regulator-min-microvolt = <3300000>;
- regulator-max-microvolt = <3300000>;
- };
-
- reg_vdd_cpu_gpu: regulator-vdd-cpu-gpu {
- compatible = "regulator-fixed";
- regulator-name = "vdd-cpu-gpu";
- regulator-min-microvolt = <1135000>;
- regulator-max-microvolt = <1135000>;
- };
-};
-
-&cpu0 {
- cpu-supply = <®_vdd_cpu_gpu>;
-};
-
-&de {
- status = "okay";
-};
-
-&dwc3 {
- status = "okay";
-};
-
-&ehci0 {
- status = "okay";
-};
-
-&ehci3 {
- status = "okay";
-};
-
-&gpu {
- mali-supply = <®_vdd_cpu_gpu>;
- status = "okay";
-};
-
-&hdmi {
- status = "okay";
-};
-
-&hdmi_out {
- hdmi_out_con: endpoint {
- remote-endpoint = <&hdmi_con_in>;
- };
-};
-
-&mmc0 {
- pinctrl-names = "default";
- pinctrl-0 = <&mmc0_pins>;
- vmmc-supply = <®_vcc3v3>;
- cd-gpios = <&pio 5 6 GPIO_ACTIVE_LOW>;
- bus-width = <4>;
- status = "okay";
-};
-
-&mmc2 {
- vmmc-supply = <®_vcc3v3>;
- vqmmc-supply = <®_vcc1v8>;
- bus-width = <8>;
- non-removable;
- cap-mmc-hw-reset;
- mmc-hs200-1_8v;
- status = "okay";
-};
-
-&ohci0 {
- status = "okay";
-};
-
-&ohci3 {
- status = "okay";
-};
-
-&pio {
- vcc-pc-supply = <®_vcc1v8>;
- vcc-pd-supply = <®_vcc3v3>;
- vcc-pg-supply = <®_vcc1v8>;
};
&r_ir {
linux,rc-map-name = "rc-tanix-tx5max";
- status = "okay";
};
-&uart0 {
+&uart1 {
pinctrl-names = "default";
- pinctrl-0 = <&uart0_ph_pins>;
+ pinctrl-0 = <&uart1_pins>, <&uart1_rts_cts_pins>;
+ uart-has-rtscts;
status = "okay";
-};
-&usb2otg {
- dr_mode = "host";
- status = "okay";
-};
-
-&usb2phy {
- status = "okay";
-};
-
-&usb3phy {
- status = "okay";
+ bluetooth {
+ compatible = "realtek,rtl8822cs-bt";
+ device-wake-gpios = <&r_pio 1 2 GPIO_ACTIVE_HIGH>; /* PM2 */
+ host-wake-gpios = <&r_pio 1 1 GPIO_ACTIVE_HIGH>; /* PM1 */
+ enable-gpios = <&r_pio 1 4 GPIO_ACTIVE_HIGH>; /* PM4 */
+ };
};
diff --git a/arch/arm64/boot/dts/allwinner/sun50i-h6-tanix.dtsi b/arch/arm64/boot/dts/allwinner/sun50i-h6-tanix.dtsi
new file mode 100644
index 0000000..edb71e4
--- /dev/null
+++ b/arch/arm64/boot/dts/allwinner/sun50i-h6-tanix.dtsi
@@ -0,0 +1,189 @@
+// SPDX-License-Identifier: (GPL-2.0+ OR MIT)
+// Copyright (c) 2019 Jernej Skrabec <jernej.skrabec@siol.net>
+
+/dts-v1/;
+
+#include "sun50i-h6.dtsi"
+#include "sun50i-h6-cpu-opp.dtsi"
+
+#include <dt-bindings/gpio/gpio.h>
+
+/ {
+ aliases {
+ serial0 = &uart0;
+ };
+
+ chosen {
+ stdout-path = "serial0:115200n8";
+ };
+
+ connector {
+ compatible = "hdmi-connector";
+ ddc-en-gpios = <&pio 7 2 GPIO_ACTIVE_HIGH>; /* PH2 */
+ type = "a";
+
+ port {
+ hdmi_con_in: endpoint {
+ remote-endpoint = <&hdmi_out_con>;
+ };
+ };
+ };
+
+ /* used for FD650 LED display driver */
+ i2c {
+ compatible = "i2c-gpio";
+ sda-gpios = <&pio 7 6 (GPIO_ACTIVE_HIGH|GPIO_OPEN_DRAIN)>; /* PH6 */
+ scl-gpios = <&pio 7 5 (GPIO_ACTIVE_HIGH|GPIO_OPEN_DRAIN)>; /* PH5 */
+ i2c-gpio,delay-us = <5>;
+ };
+
+ reg_vcc1v8: regulator-vcc1v8 {
+ compatible = "regulator-fixed";
+ regulator-name = "vcc1v8";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ };
+
+ reg_vcc3v3: regulator-vcc3v3 {
+ compatible = "regulator-fixed";
+ regulator-name = "vcc3v3";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ };
+
+ reg_vdd_cpu_gpu: regulator-vdd-cpu-gpu {
+ compatible = "regulator-fixed";
+ regulator-name = "vdd-cpu-gpu";
+ regulator-min-microvolt = <1135000>;
+ regulator-max-microvolt = <1135000>;
+ };
+
+ sound-spdif {
+ compatible = "simple-audio-card";
+ simple-audio-card,name = "sun50i-h6-spdif";
+
+ simple-audio-card,cpu {
+ sound-dai = <&spdif>;
+ };
+
+ simple-audio-card,codec {
+ sound-dai = <&spdif_out>;
+ };
+ };
+
+ spdif_out: spdif-out {
+ #sound-dai-cells = <0>;
+ compatible = "linux,spdif-dit";
+ };
+
+ wifi_pwrseq: wifi-pwrseq {
+ compatible = "mmc-pwrseq-simple";
+ clocks = <&rtc 1>;
+ clock-names = "ext_clock";
+ reset-gpios = <&r_pio 1 3 GPIO_ACTIVE_LOW>; /* PM3 */
+ };
+};
+
+&cpu0 {
+ cpu-supply = <®_vdd_cpu_gpu>;
+};
+
+&de {
+ status = "okay";
+};
+
+&dwc3 {
+ status = "okay";
+};
+
+&ehci0 {
+ status = "okay";
+};
+
+&ehci3 {
+ status = "okay";
+};
+
+&gpu {
+ mali-supply = <®_vdd_cpu_gpu>;
+ status = "okay";
+};
+
+&hdmi {
+ status = "okay";
+};
+
+&hdmi_out {
+ hdmi_out_con: endpoint {
+ remote-endpoint = <&hdmi_con_in>;
+ };
+};
+
+&mmc0 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&mmc0_pins>;
+ vmmc-supply = <®_vcc3v3>;
+ cd-gpios = <&pio 5 6 GPIO_ACTIVE_LOW>;
+ bus-width = <4>;
+ status = "okay";
+};
+
+&mmc1 {
+ vmmc-supply = <®_vcc3v3>;
+ vqmmc-supply = <®_vcc1v8>;
+ mmc-pwrseq = <&wifi_pwrseq>;
+ bus-width = <4>;
+ non-removable;
+ status = "okay";
+};
+
+&mmc2 {
+ vmmc-supply = <®_vcc3v3>;
+ vqmmc-supply = <®_vcc1v8>;
+ bus-width = <8>;
+ non-removable;
+ cap-mmc-hw-reset;
+ mmc-hs200-1_8v;
+ status = "okay";
+};
+
+&ohci0 {
+ status = "okay";
+};
+
+&ohci3 {
+ status = "okay";
+};
+
+&pio {
+ vcc-pc-supply = <®_vcc1v8>;
+ vcc-pd-supply = <®_vcc3v3>;
+ vcc-pg-supply = <®_vcc1v8>;
+};
+
+&r_ir {
+ status = "okay";
+};
+
+&spdif {
+ status = "okay";
+};
+
+&uart0 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&uart0_ph_pins>;
+ status = "okay";
+};
+
+&usb2otg {
+ dr_mode = "host";
+ status = "okay";
+};
+
+&usb2phy {
+ status = "okay";
+};
+
+&usb3phy {
+ status = "okay";
+};
diff --git a/arch/arm64/boot/dts/allwinner/sun50i-h6.dtsi b/arch/arm64/boot/dts/allwinner/sun50i-h6.dtsi
index 46ed529..fbe94ab 100644
--- a/arch/arm64/boot/dts/allwinner/sun50i-h6.dtsi
+++ b/arch/arm64/boot/dts/allwinner/sun50i-h6.dtsi
@@ -153,6 +153,15 @@ mixer0_out_tcon_top_mixer0: endpoint {
};
};
+ video-codec-g2@1c00000 {
+ compatible = "allwinner,sun50i-h6-vpu-g2";
+ reg = <0x01c00000 0x1000>;
+ interrupts = <GIC_SPI 90 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&ccu CLK_BUS_VP9>, <&ccu CLK_VP9>;
+ clock-names = "bus", "mod";
+ resets = <&ccu RST_BUS_VP9>;
+ };
+
video-codec@1c0e000 {
compatible = "allwinner,sun50i-h6-video-engine";
reg = <0x01c0e000 0x2000>;
diff --git a/arch/arm64/boot/dts/altera/socfpga_stratix10.dtsi b/arch/arm64/boot/dts/altera/socfpga_stratix10.dtsi
index d301ac0..3ec301b 100644
--- a/arch/arm64/boot/dts/altera/socfpga_stratix10.dtsi
+++ b/arch/arm64/boot/dts/altera/socfpga_stratix10.dtsi
@@ -594,7 +594,7 @@ emac0-tx-ecc@ff8c0400 {
};
qspi: spi@ff8d2000 {
- compatible = "cdns,qspi-nor";
+ compatible = "intel,socfpga-qspi", "cdns,qspi-nor";
#address-cells = <1>;
#size-cells = <0>;
reg = <0xff8d2000 0x100>,
diff --git a/arch/arm64/boot/dts/altera/socfpga_stratix10_socdk_nand.dts b/arch/arm64/boot/dts/altera/socfpga_stratix10_socdk_nand.dts
index f9b4a39..bbc3db4 100644
--- a/arch/arm64/boot/dts/altera/socfpga_stratix10_socdk_nand.dts
+++ b/arch/arm64/boot/dts/altera/socfpga_stratix10_socdk_nand.dts
@@ -119,28 +119,8 @@ partition@0 {
};
partition@200000 {
- label = "env";
- reg = <0x200000 0x40000>;
- };
-
- partition@240000 {
- label = "dtb";
- reg = <0x240000 0x40000>;
- };
-
- partition@280000 {
- label = "kernel";
- reg = <0x280000 0x2000000>;
- };
-
- partition@2280000 {
- label = "misc";
- reg = <0x2280000 0x2000000>;
- };
-
- partition@4280000 {
- label = "rootfs";
- reg = <0x4280000 0x3bd80000>;
+ label = "root";
+ reg = <0x200000 0x3fe00000>;
};
};
};
diff --git a/arch/arm64/boot/dts/amlogic/meson-g12-common.dtsi b/arch/arm64/boot/dts/amlogic/meson-g12-common.dtsi
index 00c6f53..517519e 100644
--- a/arch/arm64/boot/dts/amlogic/meson-g12-common.dtsi
+++ b/arch/arm64/boot/dts/amlogic/meson-g12-common.dtsi
@@ -58,7 +58,7 @@ efuse: efuse {
secure-monitor = <&sm>;
};
- gpu_opp_table: gpu-opp-table {
+ gpu_opp_table: opp-table-gpu {
compatible = "operating-points-v2";
opp-124999998 {
@@ -159,61 +159,6 @@ &clkc CLKID_PCIE_COMB
status = "disabled";
};
- thermal-zones {
- cpu_thermal: cpu-thermal {
- polling-delay = <1000>;
- polling-delay-passive = <100>;
- thermal-sensors = <&cpu_temp>;
-
- trips {
- cpu_passive: cpu-passive {
- temperature = <85000>; /* millicelsius */
- hysteresis = <2000>; /* millicelsius */
- type = "passive";
- };
-
- cpu_hot: cpu-hot {
- temperature = <95000>; /* millicelsius */
- hysteresis = <2000>; /* millicelsius */
- type = "hot";
- };
-
- cpu_critical: cpu-critical {
- temperature = <110000>; /* millicelsius */
- hysteresis = <2000>; /* millicelsius */
- type = "critical";
- };
- };
- };
-
- ddr_thermal: ddr-thermal {
- polling-delay = <1000>;
- polling-delay-passive = <100>;
- thermal-sensors = <&ddr_temp>;
-
- trips {
- ddr_passive: ddr-passive {
- temperature = <85000>; /* millicelsius */
- hysteresis = <2000>; /* millicelsius */
- type = "passive";
- };
-
- ddr_critical: ddr-critical {
- temperature = <110000>; /* millicelsius */
- hysteresis = <2000>; /* millicelsius */
- type = "critical";
- };
- };
-
- cooling-maps {
- map {
- trip = <&ddr_passive>;
- cooling-device = <&mali THERMAL_NO_LIMIT THERMAL_NO_LIMIT>;
- };
- };
- };
- };
-
ethmac: ethernet@ff3f0000 {
compatible = "amlogic,meson-g12a-dwmac",
"snps,dwmac-3.70a",
@@ -2415,6 +2360,61 @@ mali: gpu@ffe40000 {
};
};
+ thermal-zones {
+ cpu_thermal: cpu-thermal {
+ polling-delay = <1000>;
+ polling-delay-passive = <100>;
+ thermal-sensors = <&cpu_temp>;
+
+ trips {
+ cpu_passive: cpu-passive {
+ temperature = <85000>; /* millicelsius */
+ hysteresis = <2000>; /* millicelsius */
+ type = "passive";
+ };
+
+ cpu_hot: cpu-hot {
+ temperature = <95000>; /* millicelsius */
+ hysteresis = <2000>; /* millicelsius */
+ type = "hot";
+ };
+
+ cpu_critical: cpu-critical {
+ temperature = <110000>; /* millicelsius */
+ hysteresis = <2000>; /* millicelsius */
+ type = "critical";
+ };
+ };
+ };
+
+ ddr_thermal: ddr-thermal {
+ polling-delay = <1000>;
+ polling-delay-passive = <100>;
+ thermal-sensors = <&ddr_temp>;
+
+ trips {
+ ddr_passive: ddr-passive {
+ temperature = <85000>; /* millicelsius */
+ hysteresis = <2000>; /* millicelsius */
+ type = "passive";
+ };
+
+ ddr_critical: ddr-critical {
+ temperature = <110000>; /* millicelsius */
+ hysteresis = <2000>; /* millicelsius */
+ type = "critical";
+ };
+ };
+
+ cooling-maps {
+ map {
+ trip = <&ddr_passive>;
+ cooling-device = <&mali THERMAL_NO_LIMIT THERMAL_NO_LIMIT>;
+ };
+ };
+ };
+ };
+
timer {
compatible = "arm,armv8-timer";
interrupts = <GIC_PPI 13
diff --git a/arch/arm64/boot/dts/amlogic/meson-g12b-odroid-n2.dtsi b/arch/arm64/boot/dts/amlogic/meson-g12b-odroid-n2.dtsi
index e8a00a2..3e968b2 100644
--- a/arch/arm64/boot/dts/amlogic/meson-g12b-odroid-n2.dtsi
+++ b/arch/arm64/boot/dts/amlogic/meson-g12b-odroid-n2.dtsi
@@ -609,7 +609,7 @@ &spifc {
pinctrl-0 = <&nor_pins>;
pinctrl-names = "default";
- mx25u64: spi-flash@0 {
+ mx25u64: flash@0 {
#address-cells = <1>;
#size-cells = <1>;
compatible = "mxicy,mx25u6435f", "jedec,spi-nor";
diff --git a/arch/arm64/boot/dts/amlogic/meson-gxbb-wetek.dtsi b/arch/arm64/boot/dts/amlogic/meson-gxbb-wetek.dtsi
index a350fee..94dafb9 100644
--- a/arch/arm64/boot/dts/amlogic/meson-gxbb-wetek.dtsi
+++ b/arch/arm64/boot/dts/amlogic/meson-gxbb-wetek.dtsi
@@ -6,6 +6,8 @@
*/
#include "meson-gxbb.dtsi"
+#include <dt-bindings/gpio/gpio.h>
+#include <dt-bindings/leds/common.h>
/ {
aliases {
@@ -25,8 +27,10 @@ memory@0 {
leds {
compatible = "gpio-leds";
- led-system {
- label = "wetek-play:system-status";
+ led-power {
+ /* red in suspend or power-off */
+ color = <LED_COLOR_ID_BLUE>;
+ function = LED_FUNCTION_POWER;
gpios = <&gpio_ao GPIOAO_13 GPIO_ACTIVE_HIGH>;
default-state = "on";
panic-indicator;
@@ -64,6 +68,7 @@ vddio_ao18: regulator-vddio_ao18 {
regulator-name = "VDDIO_AO18";
regulator-min-microvolt = <1800000>;
regulator-max-microvolt = <1800000>;
+ regulator-always-on;
};
vcc_3v3: regulator-vcc_3v3 {
@@ -161,6 +166,7 @@ &hdmi_tx {
status = "okay";
pinctrl-0 = <&hdmi_hpd_pins>, <&hdmi_i2c_pins>;
pinctrl-names = "default";
+ hdmi-supply = <&vddio_ao18>;
};
&hdmi_tx_tmds_port {
diff --git a/arch/arm64/boot/dts/amlogic/meson-gxl-s805x-p241.dts b/arch/arm64/boot/dts/amlogic/meson-gxl-s805x-p241.dts
index eb7f5a3..ff906be 100644
--- a/arch/arm64/boot/dts/amlogic/meson-gxl-s805x-p241.dts
+++ b/arch/arm64/boot/dts/amlogic/meson-gxl-s805x-p241.dts
@@ -8,6 +8,7 @@
/dts-v1/;
#include <dt-bindings/input/input.h>
+#include <dt-bindings/sound/meson-aiu.h>
#include "meson-gxl-s805x.dtsi"
@@ -21,6 +22,13 @@ aliases {
ethernet0 = ðmac;
};
+ au2: analog-amplifier {
+ compatible = "simple-audio-amplifier";
+ sound-name-prefix = "AU2";
+ VCC-supply = <&vcc_5v>;
+ enable-gpios = <&gpio GPIOH_5 GPIO_ACTIVE_HIGH>;
+ };
+
chosen {
stdout-path = "serial0:115200n8";
};
@@ -84,6 +92,14 @@ vcc_3v3: regulator-vcc_3v3 {
regulator-max-microvolt = <3300000>;
};
+ vcc_5v: regulator-vcc-5v {
+ compatible = "regulator-fixed";
+ regulator-name = "VCC_5V";
+ regulator-min-microvolt = <5000000>;
+ regulator-max-microvolt = <5000000>;
+ };
+
+
emmc_pwrseq: emmc-pwrseq {
compatible = "mmc-pwrseq-emmc";
reset-gpios = <&gpio BOOT_9 GPIO_ACTIVE_LOW>;
@@ -102,6 +118,68 @@ sdio_pwrseq: sdio-pwrseq {
clocks = <&wifi32k>;
clock-names = "ext_clock";
};
+
+ sound {
+ compatible = "amlogic,gx-sound-card";
+ model = "GXL-P241";
+ audio-aux-devs = <&au2>;
+ audio-widgets = "Line", "Lineout";
+ audio-routing = "AU2 INL", "ACODEC LOLN",
+ "AU2 INR", "ACODEC LORN",
+ "Lineout", "AU2 OUTL",
+ "Lineout", "AU2 OUTR";
+ assigned-clocks = <&clkc CLKID_MPLL0>,
+ <&clkc CLKID_MPLL1>,
+ <&clkc CLKID_MPLL2>;
+ assigned-clock-parents = <0>, <0>, <0>;
+ assigned-clock-rates = <294912000>,
+ <270950400>,
+ <393216000>;
+ status = "okay";
+
+ dai-link-0 {
+ sound-dai = <&aiu AIU_CPU CPU_I2S_FIFO>;
+ };
+
+ dai-link-1 {
+ sound-dai = <&aiu AIU_CPU CPU_I2S_ENCODER>;
+ dai-format = "i2s";
+ mclk-fs = <256>;
+
+ codec-0 {
+ sound-dai = <&aiu AIU_HDMI CTRL_I2S>;
+ };
+
+ codec-1 {
+ sound-dai = <&aiu AIU_ACODEC CTRL_I2S>;
+ };
+ };
+
+ dai-link-2 {
+ sound-dai = <&aiu AIU_HDMI CTRL_OUT>;
+
+ codec-0 {
+ sound-dai = <&hdmi_tx>;
+ };
+ };
+
+ dai-link-3 {
+ sound-dai = <&aiu AIU_ACODEC CTRL_OUT>;
+
+ codec-0 {
+ sound-dai = <&acodec>;
+ };
+ };
+ };
+};
+
+&acodec {
+ AVDD-supply = <&vddio_ao18>;
+ status = "okay";
+};
+
+&aiu {
+ status = "okay";
};
&cec_AO {
@@ -136,6 +214,7 @@ &hdmi_tx {
status = "okay";
pinctrl-0 = <&hdmi_hpd_pins>, <&hdmi_i2c_pins>;
pinctrl-names = "default";
+ hdmi-supply = <&vcc_5v>;
};
&hdmi_tx_tmds_port {
@@ -220,3 +299,7 @@ &usb {
status = "okay";
dr_mode = "host";
};
+
+&usb2_phy0 {
+ phy-supply = <&vcc_5v>;
+};
diff --git a/arch/arm64/boot/dts/amlogic/meson-sm1-odroid.dtsi b/arch/arm64/boot/dts/amlogic/meson-sm1-odroid.dtsi
index 5779e70..0bd1e98 100644
--- a/arch/arm64/boot/dts/amlogic/meson-sm1-odroid.dtsi
+++ b/arch/arm64/boot/dts/amlogic/meson-sm1-odroid.dtsi
@@ -225,6 +225,20 @@ &arb {
status = "okay";
};
+&cec_AO {
+ pinctrl-0 = <&cec_ao_a_h_pins>;
+ pinctrl-names = "default";
+ status = "disabled";
+ hdmi-phandle = <&hdmi_tx>;
+};
+
+&cecb_AO {
+ pinctrl-0 = <&cec_ao_b_h_pins>;
+ pinctrl-names = "default";
+ status = "okay";
+ hdmi-phandle = <&hdmi_tx>;
+};
+
&clkc_audio {
status = "okay";
};
diff --git a/arch/arm64/boot/dts/apple/Makefile b/arch/arm64/boot/dts/apple/Makefile
index cbbd701..c0510c2 100644
--- a/arch/arm64/boot/dts/apple/Makefile
+++ b/arch/arm64/boot/dts/apple/Makefile
@@ -1,2 +1,6 @@
# SPDX-License-Identifier: GPL-2.0
dtb-$(CONFIG_ARCH_APPLE) += t8103-j274.dtb
+dtb-$(CONFIG_ARCH_APPLE) += t8103-j293.dtb
+dtb-$(CONFIG_ARCH_APPLE) += t8103-j313.dtb
+dtb-$(CONFIG_ARCH_APPLE) += t8103-j456.dtb
+dtb-$(CONFIG_ARCH_APPLE) += t8103-j457.dtb
diff --git a/arch/arm64/boot/dts/apple/t8103-j274.dts b/arch/arm64/boot/dts/apple/t8103-j274.dts
index 02c3630..2cd429e 100644
--- a/arch/arm64/boot/dts/apple/t8103-j274.dts
+++ b/arch/arm64/boot/dts/apple/t8103-j274.dts
@@ -10,39 +10,15 @@
/dts-v1/;
#include "t8103.dtsi"
+#include "t8103-jxxx.dtsi"
/ {
compatible = "apple,j274", "apple,t8103", "apple,arm-platform";
model = "Apple Mac mini (M1, 2020)";
aliases {
- serial0 = &serial0;
ethernet0 = ðernet0;
};
-
- chosen {
- #address-cells = <2>;
- #size-cells = <2>;
- ranges;
-
- stdout-path = "serial0";
-
- framebuffer0: framebuffer@0 {
- compatible = "apple,simple-framebuffer", "simple-framebuffer";
- reg = <0 0 0 0>; /* To be filled by loader */
- /* Format properties will be added by loader */
- status = "disabled";
- };
- };
-
- memory@800000000 {
- device_type = "memory";
- reg = <0x8 0 0x2 0>; /* To be filled by loader */
- };
-};
-
-&serial0 {
- status = "okay";
};
/*
@@ -50,9 +26,6 @@ &serial0 {
* on-board devices and properties that are populated by the bootloader
* (such as MAC addresses).
*/
-&port00 {
- bus-range = <1 1>;
-};
&port01 {
bus-range = <2 2>;
@@ -66,3 +39,7 @@ ethernet0: ethernet@0,0 {
local-mac-address = [00 10 18 00 00 00];
};
};
+
+&i2c2 {
+ status = "okay";
+};
diff --git a/arch/arm64/boot/dts/apple/t8103-j293.dts b/arch/arm64/boot/dts/apple/t8103-j293.dts
new file mode 100644
index 0000000..49cdf4b
--- /dev/null
+++ b/arch/arm64/boot/dts/apple/t8103-j293.dts
@@ -0,0 +1,41 @@
+// SPDX-License-Identifier: GPL-2.0+ OR MIT
+/*
+ * Apple MacBook Pro (13-inch, M1, 2020)
+ *
+ * target-type: J293
+ *
+ * Copyright The Asahi Linux Contributors
+ */
+
+/dts-v1/;
+
+#include "t8103.dtsi"
+#include "t8103-jxxx.dtsi"
+
+/ {
+ compatible = "apple,j293", "apple,t8103", "apple,arm-platform";
+ model = "Apple MacBook Pro (13-inch, M1, 2020)";
+};
+
+/*
+ * Remove unused PCIe ports and disable the associated DARTs.
+ */
+
+&pcie0_dart_1 {
+ status = "disabled";
+};
+
+&pcie0_dart_2 {
+ status = "disabled";
+};
+
+/delete-node/ &port01;
+/delete-node/ &port02;
+
+&i2c2 {
+ status = "okay";
+};
+
+&i2c4 {
+ status = "okay";
+};
diff --git a/arch/arm64/boot/dts/apple/t8103-j313.dts b/arch/arm64/boot/dts/apple/t8103-j313.dts
new file mode 100644
index 0000000..b0ebb45
--- /dev/null
+++ b/arch/arm64/boot/dts/apple/t8103-j313.dts
@@ -0,0 +1,33 @@
+// SPDX-License-Identifier: GPL-2.0+ OR MIT
+/*
+ * Apple MacBook Air (M1, 2020)
+ *
+ * target-type: J313
+ *
+ * Copyright The Asahi Linux Contributors
+ */
+
+/dts-v1/;
+
+#include "t8103.dtsi"
+#include "t8103-jxxx.dtsi"
+
+/ {
+ compatible = "apple,j313", "apple,t8103", "apple,arm-platform";
+ model = "Apple MacBook Air (M1, 2020)";
+};
+
+/*
+ * Remove unused PCIe ports and disable the associated DARTs.
+ */
+
+&pcie0_dart_1 {
+ status = "disabled";
+};
+
+&pcie0_dart_2 {
+ status = "disabled";
+};
+
+/delete-node/ &port01;
+/delete-node/ &port02;
diff --git a/arch/arm64/boot/dts/apple/t8103-j456.dts b/arch/arm64/boot/dts/apple/t8103-j456.dts
new file mode 100644
index 0000000..884fddf
--- /dev/null
+++ b/arch/arm64/boot/dts/apple/t8103-j456.dts
@@ -0,0 +1,59 @@
+// SPDX-License-Identifier: GPL-2.0+ OR MIT
+/*
+ * Apple iMac (24-inch, 4x USB-C, M1, 2020)
+ *
+ * target-type: J456
+ *
+ * Copyright The Asahi Linux Contributors
+ */
+
+/dts-v1/;
+
+#include "t8103.dtsi"
+#include "t8103-jxxx.dtsi"
+
+/ {
+ compatible = "apple,j456", "apple,t8103", "apple,arm-platform";
+ model = "Apple iMac (24-inch, 4x USB-C, M1, 2020)";
+
+ aliases {
+ ethernet0 = ðernet0;
+ };
+};
+
+&i2c0 {
+ hpm2: usb-pd@3b {
+ compatible = "apple,cd321x";
+ reg = <0x3b>;
+ interrupt-parent = <&pinctrl_ap>;
+ interrupts = <106 IRQ_TYPE_LEVEL_LOW>;
+ interrupt-names = "irq";
+ };
+
+ hpm3: usb-pd@3c {
+ compatible = "apple,cd321x";
+ reg = <0x3c>;
+ interrupt-parent = <&pinctrl_ap>;
+ interrupts = <106 IRQ_TYPE_LEVEL_LOW>;
+ interrupt-names = "irq";
+ };
+};
+
+/*
+ * Force the bus number assignments so that we can declare some of the
+ * on-board devices and properties that are populated by the bootloader
+ * (such as MAC addresses).
+ */
+
+&port01 {
+ bus-range = <2 2>;
+};
+
+&port02 {
+ bus-range = <3 3>;
+ ethernet0: ethernet@0,0 {
+ reg = <0x30000 0x0 0x0 0x0 0x0>;
+ /* To be filled by the loader */
+ local-mac-address = [00 10 18 00 00 00];
+ };
+};
diff --git a/arch/arm64/boot/dts/apple/t8103-j457.dts b/arch/arm64/boot/dts/apple/t8103-j457.dts
new file mode 100644
index 0000000..d7c6229
--- /dev/null
+++ b/arch/arm64/boot/dts/apple/t8103-j457.dts
@@ -0,0 +1,47 @@
+// SPDX-License-Identifier: GPL-2.0+ OR MIT
+/*
+ * Apple iMac (24-inch, 2x USB-C, M1, 2020)
+ *
+ * target-type: J457
+ *
+ * Copyright The Asahi Linux Contributors
+ */
+
+/dts-v1/;
+
+#include "t8103.dtsi"
+#include "t8103-jxxx.dtsi"
+
+/ {
+ compatible = "apple,j457", "apple,t8103", "apple,arm-platform";
+ model = "Apple iMac (24-inch, 2x USB-C, M1, 2020)";
+
+ aliases {
+ ethernet0 = ðernet0;
+ };
+};
+
+/*
+ * Force the bus number assignments so that we can declare some of the
+ * on-board devices and properties that are populated by the bootloader
+ * (such as MAC addresses).
+ */
+
+&port02 {
+ bus-range = <3 3>;
+ ethernet0: ethernet@0,0 {
+ reg = <0x30000 0x0 0x0 0x0 0x0>;
+ /* To be filled by the loader */
+ local-mac-address = [00 10 18 00 00 00];
+ };
+};
+
+/*
+ * Remove unused PCIe port and disable the associated DART.
+ */
+
+&pcie0_dart_1 {
+ status = "disabled";
+};
+
+/delete-node/ &port01;
diff --git a/arch/arm64/boot/dts/apple/t8103-jxxx.dtsi b/arch/arm64/boot/dts/apple/t8103-jxxx.dtsi
new file mode 100644
index 0000000..fe2ae40
--- /dev/null
+++ b/arch/arm64/boot/dts/apple/t8103-jxxx.dtsi
@@ -0,0 +1,78 @@
+// SPDX-License-Identifier: GPL-2.0+ OR MIT
+/*
+ * Apple M1 Mac mini, MacBook Air/Pro, iMac 24" (M1, 2020/2021)
+ *
+ * This file contains parts common to all Apple M1 devices using the t8103.
+ *
+ * target-type: J274, J293, J313, J456, J457
+ *
+ * Copyright The Asahi Linux Contributors
+ */
+
+/ {
+ aliases {
+ serial0 = &serial0;
+ serial2 = &serial2;
+ wifi0 = &wifi0;
+ };
+
+ chosen {
+ #address-cells = <2>;
+ #size-cells = <2>;
+ ranges;
+
+ stdout-path = "serial0";
+
+ framebuffer0: framebuffer@0 {
+ compatible = "apple,simple-framebuffer", "simple-framebuffer";
+ reg = <0 0 0 0>; /* To be filled by loader */
+ /* Format properties will be added by loader */
+ status = "disabled";
+ };
+ };
+
+ memory@800000000 {
+ device_type = "memory";
+ reg = <0x8 0 0x2 0>; /* To be filled by loader */
+ };
+};
+
+&serial0 {
+ status = "okay";
+};
+
+&serial2 {
+ status = "okay";
+};
+
+&i2c0 {
+ hpm0: usb-pd@38 {
+ compatible = "apple,cd321x";
+ reg = <0x38>;
+ interrupt-parent = <&pinctrl_ap>;
+ interrupts = <106 IRQ_TYPE_LEVEL_LOW>;
+ interrupt-names = "irq";
+ };
+
+ hpm1: usb-pd@3f {
+ compatible = "apple,cd321x";
+ reg = <0x3f>;
+ interrupt-parent = <&pinctrl_ap>;
+ interrupts = <106 IRQ_TYPE_LEVEL_LOW>;
+ interrupt-names = "irq";
+ };
+};
+
+/*
+ * Force the bus number assignments so that we can declare some of the
+ * on-board devices and properties that are populated by the bootloader
+ * (such as MAC addresses).
+ */
+&port00 {
+ bus-range = <1 1>;
+ wifi0: network@0,0 {
+ reg = <0x10000 0x0 0x0 0x0 0x0>;
+ /* To be filled by the loader */
+ local-mac-address = [00 00 00 00 00 00];
+ };
+};
diff --git a/arch/arm64/boot/dts/apple/t8103-pmgr.dtsi b/arch/arm64/boot/dts/apple/t8103-pmgr.dtsi
new file mode 100644
index 0000000..fc51bc8
--- /dev/null
+++ b/arch/arm64/boot/dts/apple/t8103-pmgr.dtsi
@@ -0,0 +1,1138 @@
+// SPDX-License-Identifier: GPL-2.0+ OR MIT
+/*
+ * PMGR Power domains for the Apple T8103 "M1" SoC
+ *
+ * Copyright The Asahi Linux Contributors
+ */
+
+
+&pmgr {
+ ps_sbr: power-controller@100 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x100 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "sbr";
+ apple,always-on; /* Core device */
+ };
+
+ ps_aic: power-controller@108 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x108 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "aic";
+ apple,always-on; /* Core device */
+ };
+
+ ps_dwi: power-controller@110 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x110 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "dwi";
+ apple,always-on; /* Core device */
+ };
+
+ ps_soc_spmi0: power-controller@118 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x118 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "soc_spmi0";
+ };
+
+ ps_soc_spmi1: power-controller@120 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x120 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "soc_spmi1";
+ };
+
+ ps_soc_spmi2: power-controller@128 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x128 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "soc_spmi2";
+ };
+
+ ps_gpio: power-controller@130 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x130 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "gpio";
+ };
+
+ ps_pms_busif: power-controller@138 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x138 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "pms_busif";
+ apple,always-on; /* Core device */
+ };
+
+ ps_pms: power-controller@140 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x140 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "pms";
+ apple,always-on; /* Core device */
+ };
+
+ ps_pms_fpwm0: power-controller@148 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x148 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "pms_fpwm0";
+ power-domains = <&ps_pms>;
+ };
+
+ ps_pms_fpwm1: power-controller@150 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x150 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "pms_fpwm1";
+ power-domains = <&ps_pms>;
+ };
+
+ ps_pms_fpwm2: power-controller@158 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x158 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "pms_fpwm2";
+ power-domains = <&ps_pms>;
+ };
+
+ ps_pms_fpwm3: power-controller@160 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x160 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "pms_fpwm3";
+ power-domains = <&ps_pms>;
+ };
+
+ ps_pms_fpwm4: power-controller@168 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x168 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "pms_fpwm4";
+ power-domains = <&ps_pms>;
+ };
+
+ ps_soc_dpe: power-controller@170 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x170 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "soc_dpe";
+ apple,always-on; /* Core device */
+ };
+
+ ps_pmgr_soc_ocla: power-controller@178 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x178 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "pmgr_soc_ocla";
+ };
+
+ ps_ispsens0: power-controller@180 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x180 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "ispsens0";
+ };
+
+ ps_ispsens1: power-controller@188 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x188 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "ispsens1";
+ };
+
+ ps_ispsens2: power-controller@190 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x190 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "ispsens2";
+ };
+
+ ps_ispsens3: power-controller@198 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x198 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "ispsens3";
+ };
+
+ ps_pcie_ref: power-controller@1a0 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x1a0 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "pcie_ref";
+ };
+
+ ps_aft0: power-controller@1a8 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x1a8 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "aft0";
+ };
+
+ ps_devc0_ivdmc: power-controller@1b0 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x1b0 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "devc0_ivdmc";
+ };
+
+ ps_imx: power-controller@1b8 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x1b8 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "imx";
+ apple,always-on; /* Apple fabric, critical block */
+ };
+
+ ps_sio_busif: power-controller@1c0 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x1c0 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "sio_busif";
+ };
+
+ ps_sio: power-controller@1c8 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x1c8 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "sio";
+ power-domains = <&ps_sio_busif>;
+ };
+
+ ps_sio_cpu: power-controller@1d0 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x1d0 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "sio_cpu";
+ power-domains = <&ps_sio>;
+ };
+
+ ps_fpwm0: power-controller@1d8 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x1d8 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "fpwm0";
+ };
+
+ ps_fpwm1: power-controller@1e0 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x1e0 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "fpwm1";
+ };
+
+ ps_fpwm2: power-controller@1e8 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x1e8 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "fpwm2";
+ };
+
+ ps_i2c0: power-controller@1f0 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x1f0 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "i2c0";
+ power-domains = <&ps_sio>;
+ };
+
+ ps_i2c1: power-controller@1f8 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x1f8 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "i2c1";
+ power-domains = <&ps_sio>;
+ };
+
+ ps_i2c2: power-controller@200 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x200 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "i2c2";
+ power-domains = <&ps_sio>;
+ };
+
+ ps_i2c3: power-controller@208 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x208 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "i2c3";
+ power-domains = <&ps_sio>;
+ };
+
+ ps_i2c4: power-controller@210 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x210 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "i2c4";
+ power-domains = <&ps_sio>;
+ };
+
+ ps_spi_p: power-controller@218 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x218 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "spi_p";
+ power-domains = <&ps_sio>;
+ };
+
+ ps_uart_p: power-controller@220 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x220 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "uart_p";
+ power-domains = <&ps_sio>;
+ };
+
+ ps_audio_p: power-controller@228 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x228 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "audio_p";
+ power-domains = <&ps_sio>;
+ };
+
+ ps_sio_adma: power-controller@230 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x230 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "sio_adma";
+ power-domains = <&ps_sio>, <&ps_pms>;
+ };
+
+ ps_aes: power-controller@238 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x238 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "aes";
+ power-domains = <&ps_sio>;
+ };
+
+ ps_spi0: power-controller@240 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x240 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "spi0";
+ power-domains = <&ps_sio>, <&ps_spi_p>;
+ };
+
+ ps_spi1: power-controller@248 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x248 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "spi1";
+ power-domains = <&ps_sio>, <&ps_spi_p>;
+ };
+
+ ps_spi2: power-controller@250 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x250 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "spi2";
+ power-domains = <&ps_sio>, <&ps_spi_p>;
+ };
+
+ ps_spi3: power-controller@258 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x258 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "spi3";
+ power-domains = <&ps_sio>, <&ps_spi_p>;
+ };
+
+ ps_uart_n: power-controller@268 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x268 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "uart_n";
+ power-domains = <&ps_uart_p>;
+ };
+
+ ps_uart0: power-controller@270 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x270 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "uart0";
+ power-domains = <&ps_uart_p>;
+ };
+
+ ps_uart1: power-controller@278 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x278 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "uart1";
+ power-domains = <&ps_uart_p>;
+ };
+
+ ps_uart2: power-controller@280 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x280 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "uart2";
+ power-domains = <&ps_uart_p>;
+ };
+
+ ps_uart3: power-controller@288 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x288 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "uart3";
+ power-domains = <&ps_uart_p>;
+ };
+
+ ps_uart4: power-controller@290 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x290 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "uart4";
+ power-domains = <&ps_uart_p>;
+ };
+
+ ps_uart5: power-controller@298 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x298 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "uart5";
+ power-domains = <&ps_uart_p>;
+ };
+
+ ps_uart6: power-controller@2a0 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x2a0 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "uart6";
+ power-domains = <&ps_uart_p>;
+ };
+
+ ps_uart7: power-controller@2a8 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x2a8 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "uart7";
+ power-domains = <&ps_uart_p>;
+ };
+
+ ps_uart8: power-controller@2b0 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x2b0 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "uart8";
+ power-domains = <&ps_uart_p>;
+ };
+
+ ps_mca0: power-controller@2b8 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x2b8 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "mca0";
+ power-domains = <&ps_audio_p>, <&ps_sio_adma>;
+ };
+
+ ps_mca1: power-controller@2c0 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x2c0 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "mca1";
+ power-domains = <&ps_audio_p>, <&ps_sio_adma>;
+ };
+
+ ps_mca2: power-controller@2c8 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x2c8 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "mca2";
+ power-domains = <&ps_audio_p>, <&ps_sio_adma>;
+ };
+
+ ps_mca3: power-controller@2d0 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x2d0 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "mca3";
+ power-domains = <&ps_audio_p>, <&ps_sio_adma>;
+ };
+
+ ps_mca4: power-controller@2d8 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x2d8 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "mca4";
+ power-domains = <&ps_audio_p>, <&ps_sio_adma>;
+ };
+
+ ps_mca5: power-controller@2e0 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x2e0 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "mca5";
+ power-domains = <&ps_audio_p>, <&ps_sio_adma>;
+ };
+
+ ps_dpa0: power-controller@2e8 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x2e8 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "dpa0";
+ power-domains = <&ps_audio_p>;
+ };
+
+ ps_dpa1: power-controller@2f0 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x2f0 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "dpa1";
+ power-domains = <&ps_audio_p>;
+ };
+
+ ps_mcc: power-controller@2f8 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x2f8 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "mcc";
+ apple,always-on; /* Memory controller */
+ };
+
+ ps_spi4: power-controller@260 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x260 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "spi4";
+ power-domains = <&ps_sio>, <&ps_spi_p>;
+ };
+
+ ps_dcs0: power-controller@300 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x300 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "dcs0";
+ apple,always-on; /* LPDDR4 interface */
+ };
+
+ ps_dcs1: power-controller@310 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x310 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "dcs1";
+ apple,always-on; /* LPDDR4 interface */
+ };
+
+ ps_dcs2: power-controller@308 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x308 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "dcs2";
+ apple,always-on; /* LPDDR4 interface */
+ };
+
+ ps_dcs3: power-controller@318 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x318 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "dcs3";
+ apple,always-on; /* LPDDR4 interface */
+ };
+
+ ps_smx: power-controller@340 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x340 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "smx";
+ apple,always-on; /* Apple fabric, critical block */
+ };
+
+ ps_apcie: power-controller@348 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x348 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "apcie";
+ power-domains = <&ps_imx>, <&ps_pcie_ref>;
+ };
+
+ ps_rmx: power-controller@350 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x350 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "rmx";
+ /* Apple Fabric, display/image stuff: this can power down */
+ };
+
+ ps_mmx: power-controller@358 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x358 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "mmx";
+ /* Apple Fabric, media stuff: this can power down */
+ };
+
+ ps_disp0_fe: power-controller@360 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x360 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "disp0_fe";
+ power-domains = <&ps_rmx>;
+ apple,always-on; /* TODO: figure out if we can enable PM here */
+ };
+
+ ps_dispext_fe: power-controller@368 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x368 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "dispext_fe";
+ power-domains = <&ps_rmx>;
+ };
+
+ ps_dispext_cpu0: power-controller@378 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x378 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "dispext_cpu0";
+ power-domains = <&ps_dispext_fe>;
+ apple,min-state = <4>;
+ };
+
+ ps_jpg: power-controller@3c0 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x3c0 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "jpg";
+ power-domains = <&ps_mmx>;
+ };
+
+ ps_msr: power-controller@3c8 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x3c8 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "msr";
+ power-domains = <&ps_mmx>;
+ };
+
+ ps_msr_ase_core: power-controller@3d0 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x3d0 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "msr_ase_core";
+ };
+
+ ps_pmp: power-controller@3d8 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x3d8 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "pmp";
+ };
+
+ ps_pms_sram: power-controller@3e0 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x3e0 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "pms_sram";
+ };
+
+ ps_apcie_gp: power-controller@3e8 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x3e8 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "apcie_gp";
+ power-domains = <&ps_apcie>;
+ };
+
+ ps_ans2: power-controller@3f0 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x3f0 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "ans2";
+ /*
+ * The ADT makes ps_apcie_st depend on ps_ans2 instead, but this
+ * doesn't make much sense since ANS2 uses APCIE_ST.
+ */
+ power-domains = <&ps_apcie_st>;
+ };
+
+ ps_gfx: power-controller@3f8 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x3f8 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "gfx";
+ };
+
+ ps_dcs4: power-controller@320 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x320 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "dcs4";
+ apple,always-on; /* LPDDR4 interface */
+ };
+
+ ps_dcs5: power-controller@330 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x330 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "dcs5";
+ apple,always-on; /* LPDDR4 interface */
+ };
+
+ ps_dcs6: power-controller@328 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x328 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "dcs6";
+ apple,always-on; /* LPDDR4 interface */
+ };
+
+ ps_dcs7: power-controller@338 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x338 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "dcs7";
+ apple,always-on; /* LPDDR4 interface */
+ };
+
+ ps_dispdfr_fe: power-controller@3a8 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x3a8 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "dispdfr_fe";
+ power-domains = <&ps_rmx>;
+ };
+
+ ps_dispdfr_be: power-controller@3b0 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x3b0 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "dispdfr_be";
+ power-domains = <&ps_dispdfr_fe>;
+ };
+
+ ps_mipi_dsi: power-controller@3b8 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x3b8 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "mipi_dsi";
+ power-domains = <&ps_dispdfr_be>;
+ };
+
+ ps_isp_sys: power-controller@400 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x400 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "isp_sys";
+ power-domains = <&ps_rmx>;
+ };
+
+ ps_venc_sys: power-controller@408 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x408 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "venc_sys";
+ power-domains = <&ps_mmx>;
+ };
+
+ ps_avd_sys: power-controller@410 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x410 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "avd_sys";
+ power-domains = <&ps_mmx>;
+ };
+
+ ps_apcie_st: power-controller@418 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x418 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "apcie_st";
+ power-domains = <&ps_apcie>;
+ };
+
+ ps_ane_sys: power-controller@470 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x470 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "ane_sys";
+ };
+
+ ps_atc0_common: power-controller@420 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x420 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "atc0_common";
+ };
+
+ ps_atc0_pcie: power-controller@428 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x428 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "atc0_pcie";
+ power-domains = <&ps_atc0_common>;
+ };
+
+ ps_atc0_cio: power-controller@430 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x430 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "atc0_cio";
+ power-domains = <&ps_atc0_common>;
+ };
+
+ ps_atc0_cio_pcie: power-controller@438 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x438 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "atc0_cio_pcie";
+ power-domains = <&ps_atc0_cio>;
+ };
+
+ ps_atc0_cio_usb: power-controller@440 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x440 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "atc0_cio_usb";
+ power-domains = <&ps_atc0_cio>;
+ };
+
+ ps_atc1_common: power-controller@448 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x448 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "atc1_common";
+ };
+
+ ps_atc1_pcie: power-controller@450 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x450 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "atc1_pcie";
+ power-domains = <&ps_atc1_common>;
+ };
+
+ ps_atc1_cio: power-controller@458 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x458 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "atc1_cio";
+ power-domains = <&ps_atc1_common>;
+ };
+
+ ps_atc1_cio_pcie: power-controller@460 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x460 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "atc1_cio_pcie";
+ power-domains = <&ps_atc1_cio>;
+ };
+
+ ps_atc1_cio_usb: power-controller@468 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x468 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "atc1_cio_usb";
+ power-domains = <&ps_atc1_cio>;
+ };
+
+ ps_sep: power-controller@c00 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0xc00 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "sep";
+ apple,always-on; /* Locked on */
+ };
+
+ ps_venc_dma: power-controller@8000 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x8000 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "venc_dma";
+ power-domains = <&ps_venc_sys>;
+ };
+
+ ps_venc_pipe4: power-controller@8008 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x8008 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "venc_pipe4";
+ power-domains = <&ps_venc_dma>;
+ };
+
+ ps_venc_pipe5: power-controller@8010 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x8010 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "venc_pipe5";
+ power-domains = <&ps_venc_dma>;
+ };
+
+ ps_venc_me0: power-controller@8018 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x8018 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "venc_me0";
+ power-domains = <&ps_venc_pipe4>, <&ps_venc_pipe5>;
+ };
+
+ ps_venc_me1: power-controller@8020 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x8020 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "venc_me1";
+ power-domains = <&ps_venc_pipe4>, <&ps_venc_pipe5>;
+ };
+
+ ps_ane_sys_cpu: power-controller@c000 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0xc000 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "ane_sys_cpu";
+ power-domains = <&ps_ane_sys>;
+ };
+
+ ps_disp0_cpu0: power-controller@10018 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x10018 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "disp0_cpu0";
+ power-domains = <&ps_disp0_fe>;
+ apple,always-on; /* TODO: figure out if we can enable PM here */
+ apple,min-state = <4>;
+ };
+};
+
+&pmgr_mini {
+ ps_debug: power-controller@58 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x58 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "debug";
+ apple,always-on; /* Core AON device */
+ };
+
+ ps_nub_spmi0: power-controller@60 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x60 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "nub_spmi0";
+ apple,always-on; /* Core AON device */
+ };
+
+ ps_nub_aon: power-controller@70 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x70 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "nub_aon";
+ apple,always-on; /* Core AON device */
+ };
+
+ ps_nub_gpio: power-controller@80 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x80 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "nub_gpio";
+ apple,always-on; /* Core AON device */
+ };
+
+ ps_nub_fabric: power-controller@a8 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0xa8 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "nub_fabric";
+ apple,always-on; /* Core AON device */
+ };
+
+ ps_nub_sram: power-controller@b0 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0xb0 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "nub_sram";
+ apple,always-on; /* Core AON device */
+ };
+
+ ps_debug_usb: power-controller@b8 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0xb8 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "debug_usb";
+ apple,always-on; /* Core AON device */
+ power-domains = <&ps_debug>;
+ };
+
+ ps_debug_auth: power-controller@c0 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0xc0 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "debug_auth";
+ apple,always-on; /* Core AON device */
+ power-domains = <&ps_debug>;
+ };
+
+ ps_nub_spmi1: power-controller@68 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x68 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "nub_spmi1";
+ apple,always-on; /* Core AON device */
+ };
+
+ ps_msg: power-controller@78 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x78 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "msg";
+ };
+
+ ps_atc0_usb_aon: power-controller@88 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x88 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "atc0_usb_aon";
+ };
+
+ ps_atc1_usb_aon: power-controller@90 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x90 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "atc1_usb_aon";
+ };
+
+ ps_atc0_usb: power-controller@98 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0x98 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "atc0_usb";
+ power-domains = <&ps_atc0_usb_aon>, <&ps_atc0_common>;
+ };
+
+ ps_atc1_usb: power-controller@a0 {
+ compatible = "apple,t8103-pmgr-pwrstate", "apple,pmgr-pwrstate";
+ reg = <0xa0 4>;
+ #power-domain-cells = <0>;
+ #reset-cells = <0>;
+ label = "atc1_usb";
+ power-domains = <&ps_atc1_usb_aon>, <&ps_atc1_common>;
+ };
+};
diff --git a/arch/arm64/boot/dts/apple/t8103.dtsi b/arch/arm64/boot/dts/apple/t8103.dtsi
index 8b61e7f..19afbc9 100644
--- a/arch/arm64/boot/dts/apple/t8103.dtsi
+++ b/arch/arm64/boot/dts/apple/t8103.dtsi
@@ -97,11 +97,11 @@ timer {
<AIC_FIQ AIC_TMR_HV_VIRT IRQ_TYPE_LEVEL_HIGH>;
};
- clk24: clock-24m {
+ clkref: clock-ref {
compatible = "fixed-clock";
#clock-cells = <0>;
clock-frequency = <24000000>;
- clock-output-names = "clk24";
+ clock-output-names = "clkref";
};
soc {
@@ -112,6 +112,73 @@ soc {
ranges;
nonposted-mmio;
+ i2c0: i2c@235010000 {
+ compatible = "apple,t8103-i2c", "apple,i2c";
+ reg = <0x2 0x35010000 0x0 0x4000>;
+ clocks = <&clkref>;
+ interrupt-parent = <&aic>;
+ interrupts = <AIC_IRQ 627 IRQ_TYPE_LEVEL_HIGH>;
+ pinctrl-0 = <&i2c0_pins>;
+ pinctrl-names = "default";
+ #address-cells = <0x1>;
+ #size-cells = <0x0>;
+ power-domains = <&ps_i2c0>;
+ };
+
+ i2c1: i2c@235014000 {
+ compatible = "apple,t8103-i2c", "apple,i2c";
+ reg = <0x2 0x35014000 0x0 0x4000>;
+ clocks = <&clkref>;
+ interrupt-parent = <&aic>;
+ interrupts = <AIC_IRQ 628 IRQ_TYPE_LEVEL_HIGH>;
+ pinctrl-0 = <&i2c1_pins>;
+ pinctrl-names = "default";
+ #address-cells = <0x1>;
+ #size-cells = <0x0>;
+ power-domains = <&ps_i2c1>;
+ };
+
+ i2c2: i2c@235018000 {
+ compatible = "apple,t8103-i2c", "apple,i2c";
+ reg = <0x2 0x35018000 0x0 0x4000>;
+ clocks = <&clkref>;
+ interrupt-parent = <&aic>;
+ interrupts = <AIC_IRQ 629 IRQ_TYPE_LEVEL_HIGH>;
+ pinctrl-0 = <&i2c2_pins>;
+ pinctrl-names = "default";
+ #address-cells = <0x1>;
+ #size-cells = <0x0>;
+ status = "disabled"; /* not used in all devices */
+ power-domains = <&ps_i2c2>;
+ };
+
+ i2c3: i2c@23501c000 {
+ compatible = "apple,t8103-i2c", "apple,i2c";
+ reg = <0x2 0x3501c000 0x0 0x4000>;
+ clocks = <&clkref>;
+ interrupt-parent = <&aic>;
+ interrupts = <AIC_IRQ 630 IRQ_TYPE_LEVEL_HIGH>;
+ pinctrl-0 = <&i2c3_pins>;
+ pinctrl-names = "default";
+ #address-cells = <0x1>;
+ #size-cells = <0x0>;
+ power-domains = <&ps_i2c3>;
+ };
+
+ i2c4: i2c@235020000 {
+ compatible = "apple,t8103-i2c", "apple,i2c";
+ reg = <0x2 0x35020000 0x0 0x4000>;
+ clocks = <&clkref>;
+ interrupt-parent = <&aic>;
+ interrupts = <AIC_IRQ 631 IRQ_TYPE_LEVEL_HIGH>;
+ pinctrl-0 = <&i2c4_pins>;
+ pinctrl-names = "default";
+ #address-cells = <0x1>;
+ #size-cells = <0x0>;
+ power-domains = <&ps_i2c4>;
+ status = "disabled"; /* only used in J293 */
+ };
+
serial0: serial@235200000 {
compatible = "apple,s5l-uart";
reg = <0x2 0x35200000 0x0 0x1000>;
@@ -122,8 +189,21 @@ serial0: serial@235200000 {
* TODO: figure out the clocking properly, there may
* be a third selectable clock.
*/
- clocks = <&clk24>, <&clk24>;
+ clocks = <&clkref>, <&clkref>;
clock-names = "uart", "clk_uart_baud0";
+ power-domains = <&ps_uart0>;
+ status = "disabled";
+ };
+
+ serial2: serial@235208000 {
+ compatible = "apple,s5l-uart";
+ reg = <0x2 0x35208000 0x0 0x1000>;
+ reg-io-width = <4>;
+ interrupt-parent = <&aic>;
+ interrupts = <AIC_IRQ 607 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&clkref>, <&clkref>;
+ clock-names = "uart", "clk_uart_baud0";
+ power-domains = <&ps_uart2>;
status = "disabled";
};
@@ -132,11 +212,20 @@ aic: interrupt-controller@23b100000 {
#interrupt-cells = <3>;
interrupt-controller;
reg = <0x2 0x3b100000 0x0 0x8000>;
+ power-domains = <&ps_aic>;
+ };
+
+ pmgr: power-management@23b700000 {
+ compatible = "apple,t8103-pmgr", "apple,pmgr", "syscon", "simple-mfd";
+ #address-cells = <1>;
+ #size-cells = <1>;
+ reg = <0x2 0x3b700000 0 0x14000>;
};
pinctrl_ap: pinctrl@23c100000 {
compatible = "apple,t8103-pinctrl", "apple,pinctrl";
reg = <0x2 0x3c100000 0x0 0x100000>;
+ power-domains = <&ps_gpio>;
gpio-controller;
#gpio-cells = <2>;
@@ -154,6 +243,31 @@ pinctrl_ap: pinctrl@23c100000 {
<AIC_IRQ 195 IRQ_TYPE_LEVEL_HIGH>,
<AIC_IRQ 196 IRQ_TYPE_LEVEL_HIGH>;
+ i2c0_pins: i2c0-pins {
+ pinmux = <APPLE_PINMUX(192, 1)>,
+ <APPLE_PINMUX(188, 1)>;
+ };
+
+ i2c1_pins: i2c1-pins {
+ pinmux = <APPLE_PINMUX(201, 1)>,
+ <APPLE_PINMUX(199, 1)>;
+ };
+
+ i2c2_pins: i2c2-pins {
+ pinmux = <APPLE_PINMUX(163, 1)>,
+ <APPLE_PINMUX(162, 1)>;
+ };
+
+ i2c3_pins: i2c3-pins {
+ pinmux = <APPLE_PINMUX(73, 1)>,
+ <APPLE_PINMUX(72, 1)>;
+ };
+
+ i2c4_pins: i2c4-pins {
+ pinmux = <APPLE_PINMUX(135, 1)>,
+ <APPLE_PINMUX(134, 1)>;
+ };
+
pcie_pins: pcie-pins {
pinmux = <APPLE_PINMUX(150, 1)>,
<APPLE_PINMUX(151, 1)>,
@@ -161,30 +275,10 @@ pcie_pins: pcie-pins {
};
};
- pinctrl_aop: pinctrl@24a820000 {
- compatible = "apple,t8103-pinctrl", "apple,pinctrl";
- reg = <0x2 0x4a820000 0x0 0x4000>;
-
- gpio-controller;
- #gpio-cells = <2>;
- gpio-ranges = <&pinctrl_aop 0 0 42>;
- apple,npins = <42>;
-
- interrupt-controller;
- #interrupt-cells = <2>;
- interrupt-parent = <&aic>;
- interrupts = <AIC_IRQ 268 IRQ_TYPE_LEVEL_HIGH>,
- <AIC_IRQ 269 IRQ_TYPE_LEVEL_HIGH>,
- <AIC_IRQ 270 IRQ_TYPE_LEVEL_HIGH>,
- <AIC_IRQ 271 IRQ_TYPE_LEVEL_HIGH>,
- <AIC_IRQ 272 IRQ_TYPE_LEVEL_HIGH>,
- <AIC_IRQ 273 IRQ_TYPE_LEVEL_HIGH>,
- <AIC_IRQ 274 IRQ_TYPE_LEVEL_HIGH>;
- };
-
pinctrl_nub: pinctrl@23d1f0000 {
compatible = "apple,t8103-pinctrl", "apple,pinctrl";
reg = <0x2 0x3d1f0000 0x0 0x4000>;
+ power-domains = <&ps_nub_gpio>;
gpio-controller;
#gpio-cells = <2>;
@@ -203,6 +297,21 @@ pinctrl_nub: pinctrl@23d1f0000 {
<AIC_IRQ 336 IRQ_TYPE_LEVEL_HIGH>;
};
+ pmgr_mini: power-management@23d280000 {
+ compatible = "apple,t8103-pmgr", "apple,pmgr", "syscon", "simple-mfd";
+ #address-cells = <1>;
+ #size-cells = <1>;
+ reg = <0x2 0x3d280000 0 0x4000>;
+ };
+
+ wdt: watchdog@23d2b0000 {
+ compatible = "apple,t8103-wdt", "apple,wdt";
+ reg = <0x2 0x3d2b0000 0x0 0x4000>;
+ clocks = <&clkref>;
+ interrupt-parent = <&aic>;
+ interrupts = <AIC_IRQ 338 IRQ_TYPE_LEVEL_HIGH>;
+ };
+
pinctrl_smc: pinctrl@23e820000 {
compatible = "apple,t8103-pinctrl", "apple,pinctrl";
reg = <0x2 0x3e820000 0x0 0x4000>;
@@ -224,12 +333,34 @@ pinctrl_smc: pinctrl@23e820000 {
<AIC_IRQ 397 IRQ_TYPE_LEVEL_HIGH>;
};
+ pinctrl_aop: pinctrl@24a820000 {
+ compatible = "apple,t8103-pinctrl", "apple,pinctrl";
+ reg = <0x2 0x4a820000 0x0 0x4000>;
+
+ gpio-controller;
+ #gpio-cells = <2>;
+ gpio-ranges = <&pinctrl_aop 0 0 42>;
+ apple,npins = <42>;
+
+ interrupt-controller;
+ #interrupt-cells = <2>;
+ interrupt-parent = <&aic>;
+ interrupts = <AIC_IRQ 268 IRQ_TYPE_LEVEL_HIGH>,
+ <AIC_IRQ 269 IRQ_TYPE_LEVEL_HIGH>,
+ <AIC_IRQ 270 IRQ_TYPE_LEVEL_HIGH>,
+ <AIC_IRQ 271 IRQ_TYPE_LEVEL_HIGH>,
+ <AIC_IRQ 272 IRQ_TYPE_LEVEL_HIGH>,
+ <AIC_IRQ 273 IRQ_TYPE_LEVEL_HIGH>,
+ <AIC_IRQ 274 IRQ_TYPE_LEVEL_HIGH>;
+ };
+
pcie0_dart_0: dart@681008000 {
compatible = "apple,t8103-dart";
reg = <0x6 0x81008000 0x0 0x4000>;
#iommu-cells = <1>;
interrupt-parent = <&aic>;
interrupts = <AIC_IRQ 696 IRQ_TYPE_LEVEL_HIGH>;
+ power-domains = <&ps_apcie_gp>;
};
pcie0_dart_1: dart@682008000 {
@@ -238,6 +369,7 @@ pcie0_dart_1: dart@682008000 {
#iommu-cells = <1>;
interrupt-parent = <&aic>;
interrupts = <AIC_IRQ 699 IRQ_TYPE_LEVEL_HIGH>;
+ power-domains = <&ps_apcie_gp>;
};
pcie0_dart_2: dart@683008000 {
@@ -246,6 +378,7 @@ pcie0_dart_2: dart@683008000 {
#iommu-cells = <1>;
interrupt-parent = <&aic>;
interrupts = <AIC_IRQ 702 IRQ_TYPE_LEVEL_HIGH>;
+ power-domains = <&ps_apcie_gp>;
};
pcie0: pcie@690000000 {
@@ -280,6 +413,7 @@ pcie0: pcie@690000000 {
ranges = <0x43000000 0x6 0xa0000000 0x6 0xa0000000 0x0 0x20000000>,
<0x02000000 0x0 0xc0000000 0x6 0xc0000000 0x0 0x40000000>;
+ power-domains = <&ps_apcie_gp>;
pinctrl-0 = <&pcie_pins>;
pinctrl-names = "default";
@@ -287,7 +421,6 @@ port00: pci@0,0 {
device_type = "pci";
reg = <0x0 0x0 0x0 0x0 0x0>;
reset-gpios = <&pinctrl_ap 152 GPIO_ACTIVE_LOW>;
- max-link-speed = <2>;
#address-cells = <3>;
#size-cells = <2>;
@@ -307,7 +440,6 @@ port01: pci@1,0 {
device_type = "pci";
reg = <0x800 0x0 0x0 0x0 0x0>;
reset-gpios = <&pinctrl_ap 153 GPIO_ACTIVE_LOW>;
- max-link-speed = <2>;
#address-cells = <3>;
#size-cells = <2>;
@@ -327,7 +459,6 @@ port02: pci@2,0 {
device_type = "pci";
reg = <0x1000 0x0 0x0 0x0 0x0>;
reset-gpios = <&pinctrl_ap 33 GPIO_ACTIVE_LOW>;
- max-link-speed = <1>;
#address-cells = <3>;
#size-cells = <2>;
@@ -345,3 +476,5 @@ port02: pci@2,0 {
};
};
};
+
+#include "t8103-pmgr.dtsi"
diff --git a/arch/arm64/boot/dts/broadcom/bcm4908/Makefile b/arch/arm64/boot/dts/broadcom/bcm4908/Makefile
index cc75854..6e364e3 100644
--- a/arch/arm64/boot/dts/broadcom/bcm4908/Makefile
+++ b/arch/arm64/boot/dts/broadcom/bcm4908/Makefile
@@ -2,3 +2,4 @@
dtb-$(CONFIG_ARCH_BCM4908) += bcm4906-netgear-r8000p.dtb
dtb-$(CONFIG_ARCH_BCM4908) += bcm4906-tplink-archer-c2300-v1.dtb
dtb-$(CONFIG_ARCH_BCM4908) += bcm4908-asus-gt-ac5300.dtb
+dtb-$(CONFIG_ARCH_BCM4908) += bcm4908-netgear-raxe500.dtb
diff --git a/arch/arm64/boot/dts/broadcom/bcm4908/bcm4908-netgear-raxe500.dts b/arch/arm64/boot/dts/broadcom/bcm4908/bcm4908-netgear-raxe500.dts
new file mode 100644
index 0000000..3c2cf2d
--- /dev/null
+++ b/arch/arm64/boot/dts/broadcom/bcm4908/bcm4908-netgear-raxe500.dts
@@ -0,0 +1,50 @@
+// SPDX-License-Identifier: GPL-2.0-or-later OR MIT
+
+#include "bcm4908.dtsi"
+
+/ {
+ compatible = "netgear,raxe500", "brcm,bcm4908";
+ model = "Netgear RAXE500";
+
+ memory@0 {
+ device_type = "memory";
+ reg = <0x00 0x00 0x00 0x40000000>;
+ };
+};
+
+&ehci {
+ status = "okay";
+};
+
+&ohci {
+ status = "okay";
+};
+
+&xhci {
+ status = "okay";
+};
+
+&ports {
+ port@0 {
+ label = "lan4";
+ };
+
+ port@1 {
+ label = "lan3";
+ };
+
+ port@2 {
+ label = "lan2";
+ };
+
+ port@3 {
+ label = "lan1";
+ };
+
+ port@7 {
+ reg = <7>;
+ phy-mode = "internal";
+ phy-handle = <&phy12>;
+ label = "wan";
+ };
+};
diff --git a/arch/arm64/boot/dts/exynos/exynos5433.dtsi b/arch/arm64/boot/dts/exynos/exynos5433.dtsi
index 4422021..bfe4ed8 100644
--- a/arch/arm64/boot/dts/exynos/exynos5433.dtsi
+++ b/arch/arm64/boot/dts/exynos/exynos5433.dtsi
@@ -1585,7 +1585,7 @@ pwm: pwm@14dd0000 {
status = "disabled";
};
- hsi2c_0: hsi2c@14e40000 {
+ hsi2c_0: i2c@14e40000 {
compatible = "samsung,exynos7-hsi2c";
reg = <0x14e40000 0x1000>;
interrupts = <GIC_SPI 428 IRQ_TYPE_LEVEL_HIGH>;
@@ -1598,7 +1598,7 @@ hsi2c_0: hsi2c@14e40000 {
status = "disabled";
};
- hsi2c_1: hsi2c@14e50000 {
+ hsi2c_1: i2c@14e50000 {
compatible = "samsung,exynos7-hsi2c";
reg = <0x14e50000 0x1000>;
interrupts = <GIC_SPI 429 IRQ_TYPE_LEVEL_HIGH>;
@@ -1611,7 +1611,7 @@ hsi2c_1: hsi2c@14e50000 {
status = "disabled";
};
- hsi2c_2: hsi2c@14e60000 {
+ hsi2c_2: i2c@14e60000 {
compatible = "samsung,exynos7-hsi2c";
reg = <0x14e60000 0x1000>;
interrupts = <GIC_SPI 430 IRQ_TYPE_LEVEL_HIGH>;
@@ -1624,7 +1624,7 @@ hsi2c_2: hsi2c@14e60000 {
status = "disabled";
};
- hsi2c_3: hsi2c@14e70000 {
+ hsi2c_3: i2c@14e70000 {
compatible = "samsung,exynos7-hsi2c";
reg = <0x14e70000 0x1000>;
interrupts = <GIC_SPI 431 IRQ_TYPE_LEVEL_HIGH>;
@@ -1637,7 +1637,7 @@ hsi2c_3: hsi2c@14e70000 {
status = "disabled";
};
- hsi2c_4: hsi2c@14ec0000 {
+ hsi2c_4: i2c@14ec0000 {
compatible = "samsung,exynos7-hsi2c";
reg = <0x14ec0000 0x1000>;
interrupts = <GIC_SPI 424 IRQ_TYPE_LEVEL_HIGH>;
@@ -1650,7 +1650,7 @@ hsi2c_4: hsi2c@14ec0000 {
status = "disabled";
};
- hsi2c_5: hsi2c@14ed0000 {
+ hsi2c_5: i2c@14ed0000 {
compatible = "samsung,exynos7-hsi2c";
reg = <0x14ed0000 0x1000>;
interrupts = <GIC_SPI 425 IRQ_TYPE_LEVEL_HIGH>;
@@ -1663,7 +1663,7 @@ hsi2c_5: hsi2c@14ed0000 {
status = "disabled";
};
- hsi2c_6: hsi2c@14ee0000 {
+ hsi2c_6: i2c@14ee0000 {
compatible = "samsung,exynos7-hsi2c";
reg = <0x14ee0000 0x1000>;
interrupts = <GIC_SPI 426 IRQ_TYPE_LEVEL_HIGH>;
@@ -1676,7 +1676,7 @@ hsi2c_6: hsi2c@14ee0000 {
status = "disabled";
};
- hsi2c_7: hsi2c@14ef0000 {
+ hsi2c_7: i2c@14ef0000 {
compatible = "samsung,exynos7-hsi2c";
reg = <0x14ef0000 0x1000>;
interrupts = <GIC_SPI 427 IRQ_TYPE_LEVEL_HIGH>;
@@ -1689,7 +1689,7 @@ hsi2c_7: hsi2c@14ef0000 {
status = "disabled";
};
- hsi2c_8: hsi2c@14d90000 {
+ hsi2c_8: i2c@14d90000 {
compatible = "samsung,exynos7-hsi2c";
reg = <0x14d90000 0x1000>;
interrupts = <GIC_SPI 443 IRQ_TYPE_LEVEL_HIGH>;
@@ -1702,7 +1702,7 @@ hsi2c_8: hsi2c@14d90000 {
status = "disabled";
};
- hsi2c_9: hsi2c@14da0000 {
+ hsi2c_9: i2c@14da0000 {
compatible = "samsung,exynos7-hsi2c";
reg = <0x14da0000 0x1000>;
interrupts = <GIC_SPI 444 IRQ_TYPE_LEVEL_HIGH>;
@@ -1715,7 +1715,7 @@ hsi2c_9: hsi2c@14da0000 {
status = "disabled";
};
- hsi2c_10: hsi2c@14de0000 {
+ hsi2c_10: i2c@14de0000 {
compatible = "samsung,exynos7-hsi2c";
reg = <0x14de0000 0x1000>;
interrupts = <GIC_SPI 445 IRQ_TYPE_LEVEL_HIGH>;
@@ -1728,7 +1728,7 @@ hsi2c_10: hsi2c@14de0000 {
status = "disabled";
};
- hsi2c_11: hsi2c@14df0000 {
+ hsi2c_11: i2c@14df0000 {
compatible = "samsung,exynos7-hsi2c";
reg = <0x14df0000 0x1000>;
interrupts = <GIC_SPI 446 IRQ_TYPE_LEVEL_HIGH>;
diff --git a/arch/arm64/boot/dts/exynos/exynos7.dtsi b/arch/arm64/boot/dts/exynos/exynos7.dtsi
index c73a597..c3efbc8 100644
--- a/arch/arm64/boot/dts/exynos/exynos7.dtsi
+++ b/arch/arm64/boot/dts/exynos/exynos7.dtsi
@@ -359,7 +359,7 @@ pinctrl_bus1: pinctrl@14870000 {
interrupts = <GIC_SPI 384 IRQ_TYPE_LEVEL_HIGH>;
};
- hsi2c_0: hsi2c@13640000 {
+ hsi2c_0: i2c@13640000 {
compatible = "samsung,exynos7-hsi2c";
reg = <0x13640000 0x1000>;
interrupts = <GIC_SPI 441 IRQ_TYPE_LEVEL_HIGH>;
@@ -372,7 +372,7 @@ hsi2c_0: hsi2c@13640000 {
status = "disabled";
};
- hsi2c_1: hsi2c@13650000 {
+ hsi2c_1: i2c@13650000 {
compatible = "samsung,exynos7-hsi2c";
reg = <0x13650000 0x1000>;
interrupts = <GIC_SPI 442 IRQ_TYPE_LEVEL_HIGH>;
@@ -385,7 +385,7 @@ hsi2c_1: hsi2c@13650000 {
status = "disabled";
};
- hsi2c_2: hsi2c@14e60000 {
+ hsi2c_2: i2c@14e60000 {
compatible = "samsung,exynos7-hsi2c";
reg = <0x14e60000 0x1000>;
interrupts = <GIC_SPI 459 IRQ_TYPE_LEVEL_HIGH>;
@@ -398,7 +398,7 @@ hsi2c_2: hsi2c@14e60000 {
status = "disabled";
};
- hsi2c_3: hsi2c@14e70000 {
+ hsi2c_3: i2c@14e70000 {
compatible = "samsung,exynos7-hsi2c";
reg = <0x14e70000 0x1000>;
interrupts = <GIC_SPI 460 IRQ_TYPE_LEVEL_HIGH>;
@@ -411,7 +411,7 @@ hsi2c_3: hsi2c@14e70000 {
status = "disabled";
};
- hsi2c_4: hsi2c@13660000 {
+ hsi2c_4: i2c@13660000 {
compatible = "samsung,exynos7-hsi2c";
reg = <0x13660000 0x1000>;
interrupts = <GIC_SPI 443 IRQ_TYPE_LEVEL_HIGH>;
@@ -424,7 +424,7 @@ hsi2c_4: hsi2c@13660000 {
status = "disabled";
};
- hsi2c_5: hsi2c@13670000 {
+ hsi2c_5: i2c@13670000 {
compatible = "samsung,exynos7-hsi2c";
reg = <0x13670000 0x1000>;
interrupts = <GIC_SPI 444 IRQ_TYPE_LEVEL_HIGH>;
@@ -437,7 +437,7 @@ hsi2c_5: hsi2c@13670000 {
status = "disabled";
};
- hsi2c_6: hsi2c@14e00000 {
+ hsi2c_6: i2c@14e00000 {
compatible = "samsung,exynos7-hsi2c";
reg = <0x14e00000 0x1000>;
interrupts = <GIC_SPI 461 IRQ_TYPE_LEVEL_HIGH>;
@@ -450,7 +450,7 @@ hsi2c_6: hsi2c@14e00000 {
status = "disabled";
};
- hsi2c_7: hsi2c@13e10000 {
+ hsi2c_7: i2c@13e10000 {
compatible = "samsung,exynos7-hsi2c";
reg = <0x13e10000 0x1000>;
interrupts = <GIC_SPI 462 IRQ_TYPE_LEVEL_HIGH>;
@@ -463,7 +463,7 @@ hsi2c_7: hsi2c@13e10000 {
status = "disabled";
};
- hsi2c_8: hsi2c@14e20000 {
+ hsi2c_8: i2c@14e20000 {
compatible = "samsung,exynos7-hsi2c";
reg = <0x14e20000 0x1000>;
interrupts = <GIC_SPI 463 IRQ_TYPE_LEVEL_HIGH>;
@@ -476,7 +476,7 @@ hsi2c_8: hsi2c@14e20000 {
status = "disabled";
};
- hsi2c_9: hsi2c@13680000 {
+ hsi2c_9: i2c@13680000 {
compatible = "samsung,exynos7-hsi2c";
reg = <0x13680000 0x1000>;
interrupts = <GIC_SPI 445 IRQ_TYPE_LEVEL_HIGH>;
@@ -489,7 +489,7 @@ hsi2c_9: hsi2c@13680000 {
status = "disabled";
};
- hsi2c_10: hsi2c@13690000 {
+ hsi2c_10: i2c@13690000 {
compatible = "samsung,exynos7-hsi2c";
reg = <0x13690000 0x1000>;
interrupts = <GIC_SPI 446 IRQ_TYPE_LEVEL_HIGH>;
@@ -502,7 +502,7 @@ hsi2c_10: hsi2c@13690000 {
status = "disabled";
};
- hsi2c_11: hsi2c@136a0000 {
+ hsi2c_11: i2c@136a0000 {
compatible = "samsung,exynos7-hsi2c";
reg = <0x136a0000 0x1000>;
interrupts = <GIC_SPI 447 IRQ_TYPE_LEVEL_HIGH>;
diff --git a/arch/arm64/boot/dts/exynos/exynosautov9-sadk.dts b/arch/arm64/boot/dts/exynos/exynosautov9-sadk.dts
index ef46d7a..57518cb 100644
--- a/arch/arm64/boot/dts/exynos/exynosautov9-sadk.dts
+++ b/arch/arm64/boot/dts/exynos/exynosautov9-sadk.dts
@@ -54,3 +54,7 @@ &ufs_0 {
vcc-supply = <&ufs_0_fixed_vcc_reg>;
vcc-fixed-regulator;
};
+
+&usi_0 {
+ status = "okay";
+};
diff --git a/arch/arm64/boot/dts/exynos/exynosautov9.dtsi b/arch/arm64/boot/dts/exynos/exynosautov9.dtsi
index a960c0b..de8fcb8 100644
--- a/arch/arm64/boot/dts/exynos/exynosautov9.dtsi
+++ b/arch/arm64/boot/dts/exynos/exynosautov9.dtsi
@@ -7,6 +7,7 @@
*/
#include <dt-bindings/interrupt-controller/arm-gic.h>
+#include <dt-bindings/soc/samsung,exynos-usi.h>
/ {
compatible = "samsung,exynosautov9";
@@ -256,16 +257,35 @@ syscon_fsys2: syscon@17c20000 {
reg = <0x17c20000 0x1000>;
};
- /* USI: UART */
- serial_0: uart@10300000 {
- compatible = "samsung,exynos850-uart";
- reg = <0x10300000 0x100>;
- interrupts = <GIC_SPI 345 IRQ_TYPE_LEVEL_HIGH>;
- pinctrl-names = "default";
- pinctrl-0 = <&uart0_bus_dual>;
+ syscon_peric0: syscon@10220000 {
+ compatible = "samsung,exynosautov9-sysreg", "syscon";
+ reg = <0x10220000 0x2000>;
+ };
+
+ usi_0: usi@103000c0 {
+ compatible = "samsung,exynos850-usi";
+ reg = <0x103000c0 0x20>;
+ samsung,sysreg = <&syscon_peric0 0x1000>;
+ samsung,mode = <USI_V2_UART>;
+ samsung,clkreq-on; /* needed for UART mode */
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges;
clocks = <&uart_clock>, <&uart_clock>;
- clock-names = "uart", "clk_uart_baud0";
+ clock-names = "pclk", "ipclk";
status = "disabled";
+
+ /* USI: UART */
+ serial_0: serial@10300000 {
+ compatible = "samsung,exynos850-uart";
+ reg = <0x10300000 0xc0>;
+ interrupts = <GIC_SPI 345 IRQ_TYPE_LEVEL_HIGH>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&uart0_bus_dual>;
+ clocks = <&uart_clock>, <&uart_clock>;
+ clock-names = "uart", "clk_uart_baud0";
+ status = "disabled";
+ };
};
ufs_0_phy: ufs0-phy@17e04000 {
diff --git a/arch/arm64/boot/dts/freescale/Makefile b/arch/arm64/boot/dts/freescale/Makefile
index a14a617..6d8f0a5 100644
--- a/arch/arm64/boot/dts/freescale/Makefile
+++ b/arch/arm64/boot/dts/freescale/Makefile
@@ -1,4 +1,14 @@
# SPDX-License-Identifier: GPL-2.0
+
+# required for overlay support
+DTC_FLAGS_fsl-ls1028a-qds := -@
+DTC_FLAGS_fsl-ls1028a-qds-13bb := -@
+DTC_FLAGS_fsl-ls1028a-qds-65bb := -@
+DTC_FLAGS_fsl-ls1028a-qds-7777 := -@
+DTC_FLAGS_fsl-ls1028a-qds-85bb := -@
+DTC_FLAGS_fsl-ls1028a-qds-899b := -@
+DTC_FLAGS_fsl-ls1028a-qds-9999 := -@
+
dtb-$(CONFIG_ARCH_LAYERSCAPE) += fsl-ls1012a-frdm.dtb
dtb-$(CONFIG_ARCH_LAYERSCAPE) += fsl-ls1012a-frwy.dtb
dtb-$(CONFIG_ARCH_LAYERSCAPE) += fsl-ls1012a-oxalis.dtb
@@ -11,6 +21,12 @@
dtb-$(CONFIG_ARCH_LAYERSCAPE) += fsl-ls1028a-kontron-sl28-var3-ads2.dtb
dtb-$(CONFIG_ARCH_LAYERSCAPE) += fsl-ls1028a-kontron-sl28-var4.dtb
dtb-$(CONFIG_ARCH_LAYERSCAPE) += fsl-ls1028a-qds.dtb
+dtb-$(CONFIG_ARCH_LAYERSCAPE) += fsl-ls1028a-qds-13bb.dtb
+dtb-$(CONFIG_ARCH_LAYERSCAPE) += fsl-ls1028a-qds-65bb.dtb
+dtb-$(CONFIG_ARCH_LAYERSCAPE) += fsl-ls1028a-qds-7777.dtb
+dtb-$(CONFIG_ARCH_LAYERSCAPE) += fsl-ls1028a-qds-85bb.dtb
+dtb-$(CONFIG_ARCH_LAYERSCAPE) += fsl-ls1028a-qds-899b.dtb
+dtb-$(CONFIG_ARCH_LAYERSCAPE) += fsl-ls1028a-qds-9999.dtb
dtb-$(CONFIG_ARCH_LAYERSCAPE) += fsl-ls1028a-rdb.dtb
dtb-$(CONFIG_ARCH_LAYERSCAPE) += fsl-ls1043a-qds.dtb
dtb-$(CONFIG_ARCH_LAYERSCAPE) += fsl-ls1043a-rdb.dtb
@@ -40,6 +56,7 @@
dtb-$(CONFIG_ARCH_MXC) += imx8mm-icore-mx8mm-edimm2.2.dtb
dtb-$(CONFIG_ARCH_MXC) += imx8mm-kontron-n801x-s.dtb
dtb-$(CONFIG_ARCH_MXC) += imx8mm-nitrogen-r2.dtb
+dtb-$(CONFIG_ARCH_MXC) += imx8mm-tqma8mqml-mba8mx.dtb
dtb-$(CONFIG_ARCH_MXC) += imx8mm-var-som-symphony.dtb
dtb-$(CONFIG_ARCH_MXC) += imx8mm-venice-gw71xx-0x.dtb
dtb-$(CONFIG_ARCH_MXC) += imx8mm-venice-gw72xx-0x.dtb
@@ -47,8 +64,11 @@
dtb-$(CONFIG_ARCH_MXC) += imx8mm-venice-gw7901.dtb
dtb-$(CONFIG_ARCH_MXC) += imx8mm-venice-gw7902.dtb
dtb-$(CONFIG_ARCH_MXC) += imx8mn-beacon-kit.dtb
+dtb-$(CONFIG_ARCH_MXC) += imx8mn-bsh-smm-s2.dtb
+dtb-$(CONFIG_ARCH_MXC) += imx8mn-bsh-smm-s2pro.dtb
dtb-$(CONFIG_ARCH_MXC) += imx8mn-evk.dtb
dtb-$(CONFIG_ARCH_MXC) += imx8mn-ddr4-evk.dtb
+dtb-$(CONFIG_ARCH_MXC) += imx8mn-tqma8mqnl-mba8mx.dtb
dtb-$(CONFIG_ARCH_MXC) += imx8mn-var-som-symphony.dtb
dtb-$(CONFIG_ARCH_MXC) += imx8mn-venice-gw7902.dtb
dtb-$(CONFIG_ARCH_MXC) += imx8mp-evk.dtb
@@ -60,6 +80,7 @@
dtb-$(CONFIG_ARCH_MXC) += imx8mq-librem5-r2.dtb
dtb-$(CONFIG_ARCH_MXC) += imx8mq-librem5-r3.dtb
dtb-$(CONFIG_ARCH_MXC) += imx8mq-librem5-r4.dtb
+dtb-$(CONFIG_ARCH_MXC) += imx8mq-tqma8mq-mba8mx.dtb
dtb-$(CONFIG_ARCH_MXC) += imx8mq-mnt-reform2.dtb
dtb-$(CONFIG_ARCH_MXC) += imx8mq-nitrogen.dtb
dtb-$(CONFIG_ARCH_MXC) += imx8mq-phanbell.dtb
@@ -71,6 +92,7 @@
dtb-$(CONFIG_ARCH_MXC) += imx8qxp-ai_ml.dtb
dtb-$(CONFIG_ARCH_MXC) += imx8qxp-colibri-eval-v3.dtb
dtb-$(CONFIG_ARCH_MXC) += imx8qxp-mek.dtb
+dtb-$(CONFIG_ARCH_MXC) += imx8ulp-evk.dtb
dtb-$(CONFIG_ARCH_S32) += s32g274a-evb.dtb
dtb-$(CONFIG_ARCH_S32) += s32g274a-rdb2.dtb
diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1028a-qds-13bb.dts b/arch/arm64/boot/dts/freescale/fsl-ls1028a-qds-13bb.dts
new file mode 100644
index 0000000..f748a2c
--- /dev/null
+++ b/arch/arm64/boot/dts/freescale/fsl-ls1028a-qds-13bb.dts
@@ -0,0 +1,113 @@
+// SPDX-License-Identifier: (GPL-2.0+ OR MIT)
+/*
+ * Device Tree fragment for LS1028A QDS board, serdes 13bb
+ *
+ * Copyright 2019-2021 NXP
+ *
+ * Requires a LS1028A QDS board with lane B rework.
+ * Requires a SCH-30841 card with lane A of connector rewired to PHY lane C.
+ * Set-up is a SCH-30842 card in slot 1 and SCH-30841 in slot 2.
+ */
+
+/dts-v1/;
+/plugin/;
+
+/ {
+ fragment@0 {
+ target = <&mdio_slot1>;
+
+ __overlay__ {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ slot1_sgmii: ethernet-phy@2 {
+ /* AQR112 */
+ reg = <0x2>;
+ compatible = "ethernet-phy-ieee802.3-c45";
+ };
+ };
+ };
+
+ fragment@1 {
+ target = <&enetc_port0>;
+
+ __overlay__ {
+ phy-handle = <&slot1_sgmii>;
+ phy-mode = "usxgmii";
+ managed = "in-band-status";
+ status = "okay";
+ };
+ };
+
+ fragment@2 {
+ target = <&mdio_slot2>;
+
+ __overlay__ {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ /* 4 ports on AQR412 */
+ slot2_qxgmii0: ethernet-phy@0 {
+ reg = <0x0>;
+ compatible = "ethernet-phy-ieee802.3-c45";
+ };
+
+ slot2_qxgmii1: ethernet-phy@1 {
+ reg = <0x1>;
+ compatible = "ethernet-phy-ieee802.3-c45";
+ };
+
+ slot2_qxgmii2: ethernet-phy@2 {
+ reg = <0x2>;
+ compatible = "ethernet-phy-ieee802.3-c45";
+ };
+
+ slot2_qxgmii3: ethernet-phy@3 {
+ reg = <0x3>;
+ compatible = "ethernet-phy-ieee802.3-c45";
+ };
+ };
+ };
+
+ fragment@3 {
+ target = <&mscc_felix_ports>;
+
+ __overlay__ {
+ port@0 {
+ status = "okay";
+ phy-handle = <&slot2_qxgmii0>;
+ phy-mode = "usxgmii";
+ managed = "in-band-status";
+ };
+
+ port@1 {
+ status = "okay";
+ phy-handle = <&slot2_qxgmii1>;
+ phy-mode = "usxgmii";
+ managed = "in-band-status";
+ };
+
+ port@2 {
+ status = "okay";
+ phy-handle = <&slot2_qxgmii2>;
+ phy-mode = "usxgmii";
+ managed = "in-band-status";
+ };
+
+ port@3 {
+ status = "okay";
+ phy-handle = <&slot2_qxgmii3>;
+ phy-mode = "usxgmii";
+ managed = "in-band-status";
+ };
+ };
+ };
+
+ fragment@4 {
+ target = <&mscc_felix>;
+
+ __overlay__ {
+ status = "okay";
+ };
+ };
+};
diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1028a-qds-65bb.dts b/arch/arm64/boot/dts/freescale/fsl-ls1028a-qds-65bb.dts
new file mode 100644
index 0000000..8ffb707
--- /dev/null
+++ b/arch/arm64/boot/dts/freescale/fsl-ls1028a-qds-65bb.dts
@@ -0,0 +1,108 @@
+// SPDX-License-Identifier: (GPL-2.0+ OR MIT)
+/*
+ * Device Tree fragment for LS1028A QDS board, serdes 69xx
+ *
+ * Copyright 2019-2021 NXP
+ *
+ * Requires a LS1028A QDS board with lane B rework.
+ * Requires a SCH-30842 card in slot 1 and a SCH-28021 card in slot 2.
+ */
+
+/dts-v1/;
+/plugin/;
+
+/ {
+ fragment@0 {
+ target = <&mdio_slot1>;
+
+ __overlay__ {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ slot1_sgmii: ethernet-phy@2 {
+ /* AQR112 */
+ reg = <0x2>;
+ compatible = "ethernet-phy-ieee802.3-c45";
+ };
+ };
+ };
+
+ fragment@1 {
+ target = <&enetc_port0>;
+
+ __overlay__ {
+ phy-handle = <&slot1_sgmii>;
+ phy-mode = "2500base-x";
+ managed = "in-band-status";
+ status = "okay";
+ };
+ };
+
+ fragment@2 {
+ target = <&mdio_slot2>;
+
+ __overlay__ {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ /* 4 ports on VSC8514 */
+ slot2_qsgmii0: ethernet-phy@8 {
+ reg = <0x8>;
+ };
+
+ slot2_qsgmii1: ethernet-phy@9 {
+ reg = <0x9>;
+ };
+
+ slot2_qsgmii2: ethernet-phy@a {
+ reg = <0xa>;
+ };
+
+ slot2_qsgmii3: ethernet-phy@b {
+ reg = <0xb>;
+ };
+ };
+ };
+
+ fragment@3 {
+ target = <&mscc_felix_ports>;
+
+ __overlay__ {
+ port@0 {
+ status = "okay";
+ phy-handle = <&slot2_qsgmii0>;
+ phy-mode = "qsgmii";
+ managed = "in-band-status";
+ };
+
+ port@1 {
+ status = "okay";
+ phy-handle = <&slot2_qsgmii1>;
+ phy-mode = "qsgmii";
+ managed = "in-band-status";
+ };
+
+ port@2 {
+ status = "okay";
+ phy-handle = <&slot2_qsgmii2>;
+ phy-mode = "qsgmii";
+ managed = "in-band-status";
+ };
+
+ port@3 {
+ status = "okay";
+ phy-handle = <&slot2_qsgmii3>;
+ phy-mode = "qsgmii";
+ managed = "in-band-status";
+ };
+ };
+ };
+
+ fragment@4 {
+ target = <&mscc_felix>;
+
+ __overlay__ {
+ status = "okay";
+ };
+ };
+};
diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1028a-qds-7777.dts b/arch/arm64/boot/dts/freescale/fsl-ls1028a-qds-7777.dts
new file mode 100644
index 0000000..eb6a1e6
--- /dev/null
+++ b/arch/arm64/boot/dts/freescale/fsl-ls1028a-qds-7777.dts
@@ -0,0 +1,82 @@
+// SPDX-License-Identifier: (GPL-2.0+ OR MIT)
+/*
+ * Device Tree fragment for LS1028A QDS board, serdes 7777
+ *
+ * Copyright 2019-2021 NXP
+ *
+ * Requires a LS1028A QDS board without lane B rework.
+ * Requires a SCH-30841 card without lane A/C rewire and with a FW with muxing
+ * disabled, plugged in slot 1.
+ */
+
+/dts-v1/;
+/plugin/;
+
+/ {
+ fragment@0 {
+ target = <&mdio_slot1>;
+
+ __overlay__ {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ /* 4 ports on AQR412 */
+ slot1_sxgmii0: ethernet-phy@0 {
+ reg = <0x0>;
+ compatible = "ethernet-phy-ieee802.3-c45";
+ };
+
+ slot1_sxgmii1: ethernet-phy@1 {
+ reg = <0x1>;
+ compatible = "ethernet-phy-ieee802.3-c45";
+ };
+
+ slot1_sxgmii2: ethernet-phy@2 {
+ reg = <0x2>;
+ compatible = "ethernet-phy-ieee802.3-c45";
+ };
+
+ slot1_sxgmii3: ethernet-phy@3 {
+ reg = <0x3>;
+ compatible = "ethernet-phy-ieee802.3-c45";
+ };
+ };
+ };
+
+ fragment@1 {
+ target = <&mscc_felix_ports>;
+
+ __overlay__ {
+ port@0 {
+ status = "okay";
+ phy-handle = <&slot1_sxgmii0>;
+ phy-mode = "2500base-x";
+ };
+
+ port@1 {
+ status = "okay";
+ phy-handle = <&slot1_sxgmii1>;
+ phy-mode = "2500base-x";
+ };
+
+ port@2 {
+ status = "okay";
+ phy-handle = <&slot1_sxgmii2>;
+ phy-mode = "2500base-x";
+ };
+
+ port@3 {
+ status = "okay";
+ phy-handle = <&slot1_sxgmii3>;
+ phy-mode = "2500base-x";
+ };
+ };
+ };
+
+ fragment@2 {
+ target = <&mscc_felix>;
+ __overlay__ {
+ status = "okay";
+ };
+ };
+};
diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1028a-qds-85bb.dts b/arch/arm64/boot/dts/freescale/fsl-ls1028a-qds-85bb.dts
new file mode 100644
index 0000000..8e90c30
--- /dev/null
+++ b/arch/arm64/boot/dts/freescale/fsl-ls1028a-qds-85bb.dts
@@ -0,0 +1,107 @@
+// SPDX-License-Identifier: (GPL-2.0+ OR MIT)
+/*
+ * Device Tree fragment for LS1028A QDS board, serdes 85bb
+ *
+ * Copyright 2019-2021 NXP
+ *
+ * Requires a LS1028A QDS board with lane B rework.
+ * Requires a SCH-24801 card in slot 1 and a SCH-28021 card in slot 2.
+ */
+
+/dts-v1/;
+/plugin/;
+
+/ {
+ fragment@0 {
+ target = <&mdio_slot1>;
+
+ __overlay__ {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ slot1_sgmii: ethernet-phy@1c {
+ /* 1st port on VSC8234 */
+ reg = <0x1c>;
+ };
+ };
+ };
+
+ fragment@1 {
+ target = <&enetc_port0>;
+
+ __overlay__ {
+ phy-handle = <&slot1_sgmii>;
+ phy-mode = "sgmii";
+ managed = "in-band-status";
+ status = "okay";
+ };
+ };
+
+ fragment@2 {
+ target = <&mdio_slot2>;
+
+ __overlay__ {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ /* 4 ports on VSC8514 */
+ slot2_qsgmii0: ethernet-phy@8 {
+ reg = <0x8>;
+ };
+
+ slot2_qsgmii1: ethernet-phy@9 {
+ reg = <0x9>;
+ };
+
+ slot2_qsgmii2: ethernet-phy@a {
+ reg = <0xa>;
+ };
+
+ slot2_qsgmii3: ethernet-phy@b {
+ reg = <0xb>;
+ };
+ };
+ };
+
+ fragment@3 {
+ target = <&mscc_felix_ports>;
+
+ __overlay__ {
+ port@0 {
+ status = "okay";
+ phy-handle = <&slot2_qsgmii0>;
+ phy-mode = "qsgmii";
+ managed = "in-band-status";
+ };
+
+ port@1 {
+ status = "okay";
+ phy-handle = <&slot2_qsgmii1>;
+ phy-mode = "qsgmii";
+ managed = "in-band-status";
+ };
+
+ port@2 {
+ status = "okay";
+ phy-handle = <&slot2_qsgmii2>;
+ phy-mode = "qsgmii";
+ managed = "in-band-status";
+ };
+
+ port@3 {
+ status = "okay";
+ phy-handle = <&slot2_qsgmii3>;
+ phy-mode = "qsgmii";
+ managed = "in-band-status";
+ };
+ };
+ };
+
+ fragment@4 {
+ target = <&mscc_felix>;
+
+ __overlay__ {
+ status = "okay";
+ };
+ };
+};
diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1028a-qds-899b.dts b/arch/arm64/boot/dts/freescale/fsl-ls1028a-qds-899b.dts
new file mode 100644
index 0000000..5d0a094
--- /dev/null
+++ b/arch/arm64/boot/dts/freescale/fsl-ls1028a-qds-899b.dts
@@ -0,0 +1,75 @@
+// SPDX-License-Identifier: (GPL-2.0+ OR MIT)
+/*
+ * Device Tree fragment for LS1028A QDS board, serdes 85xx
+ *
+ * Copyright 2019-2021 NXP
+ *
+ * Requires a LS1028A QDS board without lane B rework.
+ * Requires a SCH-24801 card in slot 1.
+ */
+
+/dts-v1/;
+/plugin/;
+
+/ {
+ fragment@0 {
+ target = <&mdio_slot1>;
+ __overlay__ {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ /* VSC8234 */
+ slot1_sgmii0: ethernet-phy@1c {
+ reg = <0x1c>;
+ };
+
+ slot1_sgmii1: ethernet-phy@1d {
+ reg = <0x1d>;
+ };
+
+ slot1_sgmii2: ethernet-phy@1e {
+ reg = <0x1e>;
+ };
+
+ slot1_sgmii3: ethernet-phy@1f {
+ reg = <0x1f>;
+ };
+ };
+ };
+
+ fragment@1 {
+ target = <&enetc_port0>;
+ __overlay__ {
+ phy-handle = <&slot1_sgmii0>;
+ phy-mode = "sgmii";
+ managed = "in-band-status";
+ status = "okay";
+ };
+ };
+
+ fragment@2 {
+ target = <&mscc_felix_ports>;
+ __overlay__ {
+ port@1 {
+ status = "okay";
+ phy-handle = <&slot1_sgmii1>;
+ phy-mode = "sgmii";
+ managed = "in-band-status";
+ };
+
+ port@2 {
+ status = "okay";
+ phy-handle = <&slot1_sgmii2>;
+ phy-mode = "sgmii";
+ managed = "in-band-status";
+ };
+ };
+ };
+
+ fragment@3 {
+ target = <&mscc_felix>;
+ __overlay__ {
+ status = "okay";
+ };
+ };
+};
diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1028a-qds-9999.dts b/arch/arm64/boot/dts/freescale/fsl-ls1028a-qds-9999.dts
new file mode 100644
index 0000000..1ef743c
--- /dev/null
+++ b/arch/arm64/boot/dts/freescale/fsl-ls1028a-qds-9999.dts
@@ -0,0 +1,79 @@
+// SPDX-License-Identifier: (GPL-2.0+ OR MIT)
+/*
+ * Device Tree fragment for LS1028A QDS board, serdes 85xx
+ *
+ * Copyright 2019-2021 NXP
+ *
+ * Requires a LS1028A QDS board without lane B rework.
+ * Requires a SCH-24801 card in slot 1.
+ */
+
+/dts-v1/;
+/plugin/;
+
+/ {
+ fragment@0 {
+ target = <&mdio_slot1>;
+ __overlay__ {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ /* VSC8234 */
+ slot1_sgmii0: ethernet-phy@1c {
+ reg = <0x1c>;
+ };
+
+ slot1_sgmii1: ethernet-phy@1d {
+ reg = <0x1d>;
+ };
+
+ slot1_sgmii2: ethernet-phy@1e {
+ reg = <0x1e>;
+ };
+
+ slot1_sgmii3: ethernet-phy@1f {
+ reg = <0x1f>;
+ };
+ };
+ };
+
+ fragment@1 {
+ target = <&mscc_felix_ports>;
+ __overlay__ {
+ port@0 {
+ status = "okay";
+ phy-handle = <&slot1_sgmii0>;
+ phy-mode = "sgmii";
+ managed = "in-band-status";
+ };
+
+ port@1 {
+ status = "okay";
+ phy-handle = <&slot1_sgmii1>;
+ phy-mode = "sgmii";
+ managed = "in-band-status";
+ };
+
+ port@2 {
+ status = "okay";
+ phy-handle = <&slot1_sgmii2>;
+ phy-mode = "sgmii";
+ managed = "in-band-status";
+ };
+
+ port@3 {
+ status = "okay";
+ phy-handle = <&slot1_sgmii3>;
+ phy-mode = "sgmii";
+ managed = "in-band-status";
+ };
+ };
+ };
+
+ fragment@2 {
+ target = <&mscc_felix>;
+ __overlay__ {
+ status = "okay";
+ };
+ };
+};
diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1028a-qds.dts b/arch/arm64/boot/dts/freescale/fsl-ls1028a-qds.dts
index 6e2a1da..177bc14 100644
--- a/arch/arm64/boot/dts/freescale/fsl-ls1028a-qds.dts
+++ b/arch/arm64/boot/dts/freescale/fsl-ls1028a-qds.dts
@@ -25,7 +25,7 @@ aliases {
serial1 = &duart1;
mmc0 = &esdhc;
mmc1 = &esdhc1;
- rtc1 = &ftm_alarm0;
+ rtc1 = &ftm_alarm1;
};
chosen {
@@ -211,6 +211,16 @@ &duart1 {
status = "okay";
};
+&enetc_port1 {
+ phy-handle = <&qds_phy1>;
+ phy-mode = "rgmii-id";
+ status = "okay";
+};
+
+&enetc_port2 {
+ status = "okay";
+};
+
&esdhc {
status = "okay";
};
@@ -234,6 +244,10 @@ mt35xu02g0: flash@0 {
};
};
+&ftm_alarm1 {
+ status = "okay";
+};
+
&i2c0 {
status = "okay";
@@ -272,11 +286,6 @@ temperature-sensor@4c {
vcc-supply = <&sb_3v3>;
};
- rtc@51 {
- compatible = "nxp,pcf2129";
- reg = <0x51>;
- };
-
eeprom@56 {
compatible = "atmel,24c512";
reg = <0x56>;
@@ -318,16 +327,28 @@ mux: mux-controller {
};
-&enetc_port1 {
- phy-handle = <&qds_phy1>;
- phy-mode = "rgmii-id";
+&i2c1 {
status = "okay";
+
+ rtc@51 {
+ compatible = "nxp,pcf2129";
+ reg = <0x51>;
+ };
};
&lpuart0 {
status = "okay";
};
+&lpuart1 {
+ status = "okay";
+};
+
+&mscc_felix_port4 {
+ ethernet = <&enetc_port2>;
+ status = "okay";
+};
+
&sai1 {
status = "okay";
};
diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1028a-rdb.dts b/arch/arm64/boot/dts/freescale/fsl-ls1028a-rdb.dts
index 7719f44..68c31cb 100644
--- a/arch/arm64/boot/dts/freescale/fsl-ls1028a-rdb.dts
+++ b/arch/arm64/boot/dts/freescale/fsl-ls1028a-rdb.dts
@@ -2,7 +2,7 @@
/*
* Device Tree file for NXP LS1028A RDB Board.
*
- * Copyright 2018 NXP
+ * Copyright 2018-2021 NXP
*
* Harninder Rai <harninder.rai@nxp.com>
*
@@ -21,7 +21,14 @@ aliases {
serial1 = &duart1;
mmc0 = &esdhc;
mmc1 = &esdhc1;
- rtc1 = &ftm_alarm0;
+ rtc1 = &ftm_alarm1;
+ spi0 = &fspi;
+ ethernet0 = &enetc_port0;
+ ethernet1 = &enetc_port2;
+ ethernet2 = &mscc_felix_port0;
+ ethernet3 = &mscc_felix_port1;
+ ethernet4 = &mscc_felix_port2;
+ ethernet5 = &mscc_felix_port3;
};
chosen {
@@ -102,6 +109,48 @@ can-transceiver {
};
};
+&duart0 {
+ status = "okay";
+};
+
+&duart1 {
+ status = "okay";
+};
+
+&enetc_mdio_pf3 {
+ sgmii_phy0: ethernet-phy@2 {
+ reg = <0x2>;
+ };
+
+ /* VSC8514 QSGMII quad PHY */
+ qsgmii_phy0: ethernet-phy@10 {
+ reg = <0x10>;
+ };
+
+ qsgmii_phy1: ethernet-phy@11 {
+ reg = <0x11>;
+ };
+
+ qsgmii_phy2: ethernet-phy@12 {
+ reg = <0x12>;
+ };
+
+ qsgmii_phy3: ethernet-phy@13 {
+ reg = <0x13>;
+ };
+};
+
+&enetc_port0 {
+ phy-handle = <&sgmii_phy0>;
+ phy-mode = "sgmii";
+ managed = "in-band-status";
+ status = "okay";
+};
+
+&enetc_port2 {
+ status = "okay";
+};
+
&esdhc {
sd-uhs-sdr104;
sd-uhs-sdr50;
@@ -132,6 +181,10 @@ mt35xu02g0: flash@0 {
};
};
+&ftm_alarm1 {
+ status = "okay";
+};
+
&i2c0 {
status = "okay";
@@ -188,48 +241,6 @@ rtc@51 {
};
};
-&duart0 {
- status = "okay";
-};
-
-&duart1 {
- status = "okay";
-};
-
-&enetc_mdio_pf3 {
- sgmii_phy0: ethernet-phy@2 {
- reg = <0x2>;
- };
-
- /* VSC8514 QSGMII quad PHY */
- qsgmii_phy0: ethernet-phy@10 {
- reg = <0x10>;
- };
-
- qsgmii_phy1: ethernet-phy@11 {
- reg = <0x11>;
- };
-
- qsgmii_phy2: ethernet-phy@12 {
- reg = <0x12>;
- };
-
- qsgmii_phy3: ethernet-phy@13 {
- reg = <0x13>;
- };
-};
-
-&enetc_port0 {
- phy-handle = <&sgmii_phy0>;
- phy-mode = "sgmii";
- managed = "in-band-status";
- status = "okay";
-};
-
-&enetc_port2 {
- status = "okay";
-};
-
&mscc_felix {
status = "okay";
};
@@ -275,6 +286,10 @@ &optee {
status = "okay";
};
+&pwm0 {
+ status = "okay";
+};
+
&sai4 {
status = "okay";
};
diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi b/arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi
index fd3f3e8..5bb8c26 100644
--- a/arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi
+++ b/arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi
@@ -66,6 +66,13 @@ CPU_PW20: cpu-pw20 {
};
};
+ rtc_clk: rtc-clk {
+ compatible = "fixed-clock";
+ #clock-cells = <0>;
+ clock-frequency = <32768>;
+ clock-output-names = "rtc_clk";
+ };
+
sysclk: sysclk {
compatible = "fixed-clock";
#clock-cells = <0>;
@@ -637,6 +644,18 @@ pcie1: pcie@3400000 {
status = "disabled";
};
+ pcie_ep1: pcie-ep@3400000 {
+ compatible = "fsl,ls1028a-pcie-ep","fsl,ls-pcie-ep";
+ reg = <0x00 0x03400000 0x0 0x00100000
+ 0x80 0x00000000 0x8 0x00000000>;
+ reg-names = "regs", "addr_space";
+ interrupts = <GIC_SPI 108 IRQ_TYPE_LEVEL_HIGH>; /* PME interrupt */
+ interrupt-names = "pme";
+ num-ib-windows = <6>;
+ num-ob-windows = <8>;
+ status = "disabled";
+ };
+
pcie2: pcie@3500000 {
compatible = "fsl,ls1028a-pcie";
reg = <0x00 0x03500000 0x0 0x00100000>, /* controller registers */
@@ -664,6 +683,18 @@ pcie2: pcie@3500000 {
status = "disabled";
};
+ pcie_ep2: pcie-ep@3500000 {
+ compatible = "fsl,ls1028a-pcie-ep","fsl,ls-pcie-ep";
+ reg = <0x00 0x03500000 0x0 0x00100000
+ 0x88 0x00000000 0x8 0x00000000>;
+ reg-names = "regs", "addr_space";
+ interrupts = <GIC_SPI 113 IRQ_TYPE_LEVEL_HIGH>; /* PME interrupt */
+ interrupt-names = "pme";
+ num-ib-windows = <6>;
+ num-ob-windows = <8>;
+ status = "disabled";
+ };
+
smmu: iommu@5000000 {
compatible = "arm,mmu-500";
reg = <0 0x5000000 0 0x800000>;
@@ -1084,7 +1115,7 @@ mscc_felix: ethernet-switch@0,5 {
interrupts = <GIC_SPI 95 IRQ_TYPE_LEVEL_HIGH>;
status = "disabled";
- ports {
+ mscc_felix_ports: ports {
#address-cells = <1>;
#size-cells = <0>;
@@ -1162,6 +1193,94 @@ ierb@1f0800000 {
reg = <0x01 0xf0800000 0x0 0x10000>;
};
+ pwm0: pwm@2800000 {
+ compatible = "fsl,vf610-ftm-pwm";
+ #pwm-cells = <3>;
+ reg = <0x0 0x2800000 0x0 0x10000>;
+ clock-names = "ftm_sys", "ftm_ext",
+ "ftm_fix", "ftm_cnt_clk_en";
+ clocks = <&clockgen 4 1>, <&clockgen 4 1>,
+ <&rtc_clk>, <&clockgen 4 1>;
+ status = "disabled";
+ };
+
+ pwm1: pwm@2810000 {
+ compatible = "fsl,vf610-ftm-pwm";
+ #pwm-cells = <3>;
+ reg = <0x0 0x2810000 0x0 0x10000>;
+ clock-names = "ftm_sys", "ftm_ext",
+ "ftm_fix", "ftm_cnt_clk_en";
+ clocks = <&clockgen 4 1>, <&clockgen 4 1>,
+ <&rtc_clk>, <&clockgen 4 1>;
+ status = "disabled";
+ };
+
+ pwm2: pwm@2820000 {
+ compatible = "fsl,vf610-ftm-pwm";
+ #pwm-cells = <3>;
+ reg = <0x0 0x2820000 0x0 0x10000>;
+ clock-names = "ftm_sys", "ftm_ext",
+ "ftm_fix", "ftm_cnt_clk_en";
+ clocks = <&clockgen 4 1>, <&clockgen 4 1>,
+ <&rtc_clk>, <&clockgen 4 1>;
+ status = "disabled";
+ };
+
+ pwm3: pwm@2830000 {
+ compatible = "fsl,vf610-ftm-pwm";
+ #pwm-cells = <3>;
+ reg = <0x0 0x2830000 0x0 0x10000>;
+ clock-names = "ftm_sys", "ftm_ext",
+ "ftm_fix", "ftm_cnt_clk_en";
+ clocks = <&clockgen 4 1>, <&clockgen 4 1>,
+ <&rtc_clk>, <&clockgen 4 1>;
+ status = "disabled";
+ };
+
+ pwm4: pwm@2840000 {
+ compatible = "fsl,vf610-ftm-pwm";
+ #pwm-cells = <3>;
+ reg = <0x0 0x2840000 0x0 0x10000>;
+ clock-names = "ftm_sys", "ftm_ext",
+ "ftm_fix", "ftm_cnt_clk_en";
+ clocks = <&clockgen 4 1>, <&clockgen 4 1>,
+ <&rtc_clk>, <&clockgen 4 1>;
+ status = "disabled";
+ };
+
+ pwm5: pwm@2850000 {
+ compatible = "fsl,vf610-ftm-pwm";
+ #pwm-cells = <3>;
+ reg = <0x0 0x2850000 0x0 0x10000>;
+ clock-names = "ftm_sys", "ftm_ext",
+ "ftm_fix", "ftm_cnt_clk_en";
+ clocks = <&clockgen 4 1>, <&clockgen 4 1>,
+ <&rtc_clk>, <&clockgen 4 1>;
+ status = "disabled";
+ };
+
+ pwm6: pwm@2860000 {
+ compatible = "fsl,vf610-ftm-pwm";
+ #pwm-cells = <3>;
+ reg = <0x0 0x2860000 0x0 0x10000>;
+ clock-names = "ftm_sys", "ftm_ext",
+ "ftm_fix", "ftm_cnt_clk_en";
+ clocks = <&clockgen 4 1>, <&clockgen 4 1>,
+ <&rtc_clk>, <&clockgen 4 1>;
+ status = "disabled";
+ };
+
+ pwm7: pwm@2870000 {
+ compatible = "fsl,vf610-ftm-pwm";
+ #pwm-cells = <3>;
+ reg = <0x0 0x2870000 0x0 0x10000>;
+ clock-names = "ftm_sys", "ftm_ext",
+ "ftm_fix", "ftm_cnt_clk_en";
+ clocks = <&clockgen 4 1>, <&clockgen 4 1>,
+ <&rtc_clk>, <&clockgen 4 1>;
+ status = "disabled";
+ };
+
rcpm: power-controller@1e34040 {
compatible = "fsl,ls1028a-rcpm", "fsl,qoriq-rcpm-2.1+";
reg = <0x0 0x1e34040 0x0 0x1c>;
@@ -1174,6 +1293,15 @@ ftm_alarm0: timer@2800000 {
reg = <0x0 0x2800000 0x0 0x10000>;
fsl,rcpm-wakeup = <&rcpm 0x0 0x0 0x0 0x0 0x4000 0x0 0x0>;
interrupts = <GIC_SPI 44 IRQ_TYPE_LEVEL_HIGH>;
+ status = "disabled";
+ };
+
+ ftm_alarm1: timer@2810000 {
+ compatible = "fsl,ls1028a-ftm-alarm";
+ reg = <0x0 0x2810000 0x0 0x10000>;
+ fsl,rcpm-wakeup = <&rcpm 0x0 0x0 0x0 0x0 0x4000 0x0 0x0>;
+ interrupts = <GIC_SPI 45 IRQ_TYPE_LEVEL_HIGH>;
+ status = "disabled";
};
};
diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1043a-rdb.dts b/arch/arm64/boot/dts/freescale/fsl-ls1043a-rdb.dts
index 3516af4..b290605 100644
--- a/arch/arm64/boot/dts/freescale/fsl-ls1043a-rdb.dts
+++ b/arch/arm64/boot/dts/freescale/fsl-ls1043a-rdb.dts
@@ -94,6 +94,8 @@ flash@0 {
compatible = "n25q128a13", "jedec,spi-nor"; /* 16MB */
reg = <0>;
spi-max-frequency = <1000000>; /* input clock */
+ fsl,spi-cs-sck-delay = <100>;
+ fsl,spi-sck-cs-delay = <100>;
};
slic@2 {
diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1088a.dtsi b/arch/arm64/boot/dts/freescale/fsl-ls1088a.dtsi
index f891ef6..3ed1f2c 100644
--- a/arch/arm64/boot/dts/freescale/fsl-ls1088a.dtsi
+++ b/arch/arm64/boot/dts/freescale/fsl-ls1088a.dtsi
@@ -206,6 +206,13 @@ sysclk: sysclk {
clock-output-names = "sysclk";
};
+ reboot {
+ compatible = "syscon-reboot";
+ regmap = <&reset>;
+ offset = <0x0>;
+ mask = <0x02>;
+ };
+
soc {
compatible = "simple-bus";
#address-cells = <2>;
@@ -226,6 +233,11 @@ dcfg: dcfg@1e00000 {
little-endian;
};
+ reset: syscon@1e60000 {
+ compatible = "fsl,ls1088a-reset", "syscon";
+ reg = <0x0 0x1e60000 0x0 0x10000>;
+ };
+
isc: syscon@1f70000 {
compatible = "fsl,ls1088a-isc", "syscon";
reg = <0x0 0x1f70000 0x0 0x10000>;
@@ -485,6 +497,7 @@ usb1: usb@3110000 {
dr_mode = "host";
snps,quirk-frame-length-adjustment = <0x20>;
snps,dis_rxdet_inp3_quirk;
+ snps,incr-burst-type-adjustment = <1>, <4>, <8>, <16>;
status = "disabled";
};
diff --git a/arch/arm64/boot/dts/freescale/fsl-lx2160a-qds.dts b/arch/arm64/boot/dts/freescale/fsl-lx2160a-qds.dts
index d858d9c..2ecfa90 100644
--- a/arch/arm64/boot/dts/freescale/fsl-lx2160a-qds.dts
+++ b/arch/arm64/boot/dts/freescale/fsl-lx2160a-qds.dts
@@ -31,6 +31,130 @@ sb_3v3: regulator-sb3v3 {
regulator-boot-on;
regulator-always-on;
};
+
+ mdio-mux-1 {
+ compatible = "mdio-mux-multiplexer";
+ mux-controls = <&mux 0>;
+ mdio-parent-bus = <&emdio1>;
+ #address-cells=<1>;
+ #size-cells = <0>;
+
+ mdio@0 { /* On-board PHY #1 RGMI1*/
+ reg = <0x00>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ };
+
+ mdio@8 { /* On-board PHY #2 RGMI2*/
+ reg = <0x8>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ };
+
+ mdio@18 { /* Slot #1 */
+ reg = <0x18>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ };
+
+ mdio@19 { /* Slot #2 */
+ reg = <0x19>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ };
+
+ mdio@1a { /* Slot #3 */
+ reg = <0x1a>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ };
+
+ mdio@1b { /* Slot #4 */
+ reg = <0x1b>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ };
+
+ mdio@1c { /* Slot #5 */
+ reg = <0x1c>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ };
+
+ mdio@1d { /* Slot #6 */
+ reg = <0x1d>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ };
+
+ mdio@1e { /* Slot #7 */
+ reg = <0x1e>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ };
+
+ mdio@1f { /* Slot #8 */
+ reg = <0x1f>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ };
+ };
+
+ mdio-mux-2 {
+ compatible = "mdio-mux-multiplexer";
+ mux-controls = <&mux 1>;
+ mdio-parent-bus = <&emdio2>;
+ #address-cells=<1>;
+ #size-cells = <0>;
+
+ mdio@0 { /* Slot #1 (secondary EMI) */
+ reg = <0x00>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ };
+
+ mdio@1 { /* Slot #2 (secondary EMI) */
+ reg = <0x01>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ };
+
+ mdio@2 { /* Slot #3 (secondary EMI) */
+ reg = <0x02>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ };
+
+ mdio@3 { /* Slot #4 (secondary EMI) */
+ reg = <0x03>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ };
+
+ mdio@4 { /* Slot #5 (secondary EMI) */
+ reg = <0x04>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ };
+
+ mdio@5 { /* Slot #6 (secondary EMI) */
+ reg = <0x05>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ };
+
+ mdio@6 { /* Slot #7 (secondary EMI) */
+ reg = <0x06>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ };
+
+ mdio@7 { /* Slot #8 (secondary EMI) */
+ reg = <0x07>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ };
+ };
};
&can0 {
@@ -81,6 +205,14 @@ dflash2: flash@0 {
};
};
+&emdio1 {
+ status = "okay";
+};
+
+&emdio2 {
+ status = "okay";
+};
+
&esdhc0 {
status = "okay";
};
@@ -107,6 +239,19 @@ mt35xu512aba0: flash@0 {
&i2c0 {
status = "okay";
+ fpga@66 {
+ compatible = "fsl,lx2160aqds-fpga", "fsl,fpga-qixis-i2c",
+ "simple-mfd";
+ reg = <0x66>;
+
+ mux: mux-controller {
+ compatible = "reg-mux";
+ #mux-control-cells = <1>;
+ mux-reg-masks = <0x54 0xf8>, /* 0: reg 0x54, bits 7:3 */
+ <0x54 0x07>; /* 1: reg 0x54, bit 2:0 */
+ };
+ };
+
i2c-mux@77 {
compatible = "nxp,pca9547";
reg = <0x77>;
@@ -156,6 +301,10 @@ rtc@51 {
};
};
+&optee {
+ status = "okay";
+};
+
&sata0 {
status = "okay";
};
diff --git a/arch/arm64/boot/dts/freescale/fsl-lx2160a-rdb.dts b/arch/arm64/boot/dts/freescale/fsl-lx2160a-rdb.dts
index 028ff80..0c44b3c 100644
--- a/arch/arm64/boot/dts/freescale/fsl-lx2160a-rdb.dts
+++ b/arch/arm64/boot/dts/freescale/fsl-lx2160a-rdb.dts
@@ -49,6 +49,14 @@ &dpmac4 {
managed = "in-band-status";
};
+&dpmac5 {
+ phy-handle = <&inphi_phy>;
+};
+
+&dpmac6 {
+ phy-handle = <&inphi_phy>;
+};
+
&dpmac17 {
phy-handle = <&rgmii_phy1>;
phy-connection-type = "rgmii-id";
@@ -109,6 +117,15 @@ can-transceiver {
};
};
+&emdio2 {
+ status = "okay";
+
+ inphi_phy: ethernet-phy@0 {
+ compatible = "ethernet-phy-id0210.7440";
+ reg = <0x0>;
+ };
+};
+
&esdhc0 {
sd-uhs-sdr104;
sd-uhs-sdr50;
@@ -202,6 +219,10 @@ rtc@51 {
};
};
+&optee {
+ status = "okay";
+};
+
&pcs_mdio3 {
status = "okay";
};
diff --git a/arch/arm64/boot/dts/freescale/fsl-lx2160a.dtsi b/arch/arm64/boot/dts/freescale/fsl-lx2160a.dtsi
index 2433e6f..7032505 100644
--- a/arch/arm64/boot/dts/freescale/fsl-lx2160a.dtsi
+++ b/arch/arm64/boot/dts/freescale/fsl-lx2160a.dtsi
@@ -1023,6 +1023,7 @@ usb0: usb@3100000 {
interrupts = <GIC_SPI 80 IRQ_TYPE_LEVEL_HIGH>;
dr_mode = "host";
snps,quirk-frame-length-adjustment = <0x20>;
+ usb3-lpm-capable;
snps,dis_rxdet_inp3_quirk;
snps,incr-burst-type-adjustment = <1>, <4>, <8>, <16>;
status = "disabled";
@@ -1034,6 +1035,7 @@ usb1: usb@3110000 {
interrupts = <GIC_SPI 81 IRQ_TYPE_LEVEL_HIGH>;
dr_mode = "host";
snps,quirk-frame-length-adjustment = <0x20>;
+ usb3-lpm-capable;
snps,dis_rxdet_inp3_quirk;
snps,incr-burst-type-adjustment = <1>, <4>, <8>, <16>;
status = "disabled";
@@ -1751,4 +1753,12 @@ dpmac18: ethernet@12 {
};
};
};
+
+ firmware {
+ optee: optee {
+ compatible = "linaro,optee-tz";
+ method = "smc";
+ status = "disabled";
+ };
+ };
};
diff --git a/arch/arm64/boot/dts/freescale/fsl-lx2162a-qds.dts b/arch/arm64/boot/dts/freescale/fsl-lx2162a-qds.dts
index e1defee..a1644ce 100644
--- a/arch/arm64/boot/dts/freescale/fsl-lx2162a-qds.dts
+++ b/arch/arm64/boot/dts/freescale/fsl-lx2162a-qds.dts
@@ -167,6 +167,14 @@ mdio@7 { /* Slot #8 (secondary EMI) */
};
};
+&can0 {
+ status = "okay";
+};
+
+&can1 {
+ status = "okay";
+};
+
&crypto {
status = "okay";
};
@@ -226,10 +234,17 @@ &emdio2 {
};
&esdhc0 {
+ sd-uhs-sdr104;
+ sd-uhs-sdr50;
+ sd-uhs-sdr25;
+ sd-uhs-sdr12;
status = "okay";
};
&esdhc1 {
+ mmc-hs200-1_8v;
+ mmc-hs400-1_8v;
+ bus-width = <8>;
status = "okay";
};
@@ -302,11 +317,17 @@ temperature-sensor@4c {
rtc@51 {
compatible = "nxp,pcf2129";
reg = <0x51>;
+ /* IRQ_RTC_B -> IRQ11_B(CPLD) -> IRQ11(CPU), active low */
+ interrupts-extended = <&extirq 11 IRQ_TYPE_LEVEL_LOW>;
};
};
};
};
+&optee {
+ status = "okay";
+};
+
&sata0 {
status = "okay";
};
diff --git a/arch/arm64/boot/dts/freescale/imx8mm-beacon-baseboard.dtsi b/arch/arm64/boot/dts/freescale/imx8mm-beacon-baseboard.dtsi
index 6f5e636..0da3118 100644
--- a/arch/arm64/boot/dts/freescale/imx8mm-beacon-baseboard.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8mm-beacon-baseboard.dtsi
@@ -43,6 +43,27 @@ reg_audio: regulator-audio {
enable-active-high;
};
+ reg_usbotg1: regulator-usbotg1 {
+ compatible = "regulator-fixed";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_reg_usb_otg1>;
+ regulator-name = "usb_otg_vbus";
+ regulator-min-microvolt = <5000000>;
+ regulator-max-microvolt = <5000000>;
+ gpio = <&gpio4 29 GPIO_ACTIVE_HIGH>;
+ enable-active-high;
+ };
+
+ reg_camera: regulator-camera {
+ compatible = "regulator-fixed";
+ regulator-name = "mipi_pwr";
+ regulator-min-microvolt = <2800000>;
+ regulator-max-microvolt = <2800000>;
+ gpio = <&pca6416_1 0 GPIO_ACTIVE_HIGH>;
+ enable-active-high;
+ startup-delay-us = <100000>;
+ };
+
reg_usdhc2_vmmc: regulator-usdhc2 {
compatible = "regulator-fixed";
regulator-name = "VSD_3V3";
@@ -67,6 +88,10 @@ sound {
};
};
+&csi {
+ status = "okay";
+};
+
&ecspi2 {
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_espi2>;
@@ -90,6 +115,30 @@ &i2c2 {
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_i2c2>;
status = "okay";
+
+ camera@3c {
+ compatible = "ovti,ov5640";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_ov5640>;
+ reg = <0x3c>;
+ clocks = <&clk IMX8MM_CLK_CLKO1>;
+ clock-names = "xclk";
+ assigned-clocks = <&clk IMX8MM_CLK_CLKO1>;
+ assigned-clock-parents = <&clk IMX8MM_CLK_24M>;
+ assigned-clock-rates = <24000000>;
+ AVDD-supply = <®_camera>; /* 2.8v */
+ powerdown-gpios = <&gpio1 7 GPIO_ACTIVE_HIGH>;
+ reset-gpios = <&gpio1 6 GPIO_ACTIVE_LOW>;
+
+ port {
+ /* MIPI CSI-2 bus endpoint */
+ ov5640_to_mipi_csi2: endpoint {
+ remote-endpoint = <&imx8mm_mipi_csi_in>;
+ clock-lanes = <0>;
+ data-lanes = <1 2>;
+ };
+ };
+ };
};
&i2c4 {
@@ -141,6 +190,18 @@ pca6416_1: gpio@21 {
};
};
+&mipi_csi {
+ status = "okay";
+ ports {
+ port@0 {
+ imx8mm_mipi_csi_in: endpoint {
+ remote-endpoint = <&ov5640_to_mipi_csi2>;
+ data-lanes = <1 2>;
+ };
+ };
+ };
+};
+
&sai3 {
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_sai3>;
@@ -169,6 +230,24 @@ &uart3 {
status = "okay";
};
+&usbotg1 {
+ vbus-supply = <®_usbotg1>;
+ disable-over-current;
+ dr_mode="otg";
+ status = "okay";
+};
+
+&usbotg2 {
+ pinctrl-names = "default";
+ disable-over-current;
+ dr_mode="host";
+ status = "okay";
+};
+
+&usbphynop2 {
+ reset-gpios = <&pca6416_1 7 GPIO_ACTIVE_HIGH>;
+};
+
&usdhc2 {
pinctrl-names = "default", "state_100mhz", "state_200mhz";
pinctrl-0 = <&pinctrl_usdhc2>, <&pinctrl_usdhc2_gpio>;
@@ -209,12 +288,26 @@ MX8MM_IOMUXC_SAI3_RXFS_GPIO4_IO28 0x41
>;
};
+ pinctrl_ov5640: ov5640grp {
+ fsl,pins = <
+ MX8MM_IOMUXC_GPIO1_IO07_GPIO1_IO7 0x19
+ MX8MM_IOMUXC_GPIO1_IO06_GPIO1_IO6 0x19
+ MX8MM_IOMUXC_GPIO1_IO14_CCMSRCGPCMIX_CLKO1 0x59
+ >;
+ };
+
pinctrl_pcal6414: pcal6414-gpiogrp {
fsl,pins = <
MX8MM_IOMUXC_SAI2_MCLK_GPIO4_IO27 0x19
>;
};
+ pinctrl_reg_usb_otg1: usbotg1grp {
+ fsl,pins = <
+ MX8MM_IOMUXC_SAI3_RXC_GPIO4_IO29 0x19
+ >;
+ };
+
pinctrl_sai3: sai3grp {
fsl,pins = <
MX8MM_IOMUXC_SAI3_TXFS_SAI3_TX_SYNC 0xd6
diff --git a/arch/arm64/boot/dts/freescale/imx8mm-beacon-som.dtsi b/arch/arm64/boot/dts/freescale/imx8mm-beacon-som.dtsi
index 40f5e7a..cf07987 100644
--- a/arch/arm64/boot/dts/freescale/imx8mm-beacon-som.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8mm-beacon-som.dtsi
@@ -263,7 +263,6 @@ &usdhc1 {
bus-width = <4>;
non-removable;
cap-power-off-card;
- pm-ignore-notify;
keep-power-in-suspend;
mmc-pwrseq = <&usdhc1_pwrseq>;
status = "okay";
diff --git a/arch/arm64/boot/dts/freescale/imx8mm-evk.dtsi b/arch/arm64/boot/dts/freescale/imx8mm-evk.dtsi
index e033d02..3bac87b 100644
--- a/arch/arm64/boot/dts/freescale/imx8mm-evk.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8mm-evk.dtsi
@@ -116,6 +116,13 @@ ethphy0: ethernet-phy@0 {
reg = <0>;
reset-gpios = <&gpio4 22 GPIO_ACTIVE_LOW>;
reset-assert-us = <10000>;
+ qca,disable-smarteee;
+ vddio-supply = <&vddio>;
+
+ vddio: vddio-regulator {
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ };
};
};
};
diff --git a/arch/arm64/boot/dts/freescale/imx8mm-icore-mx8mm-ctouch2.dts b/arch/arm64/boot/dts/freescale/imx8mm-icore-mx8mm-ctouch2.dts
index 5389d6f..5027454 100644
--- a/arch/arm64/boot/dts/freescale/imx8mm-icore-mx8mm-ctouch2.dts
+++ b/arch/arm64/boot/dts/freescale/imx8mm-icore-mx8mm-ctouch2.dts
@@ -91,7 +91,6 @@ &usdhc1 {
max-frequency = <50000000>;
bus-width = <4>;
no-1-8-v;
- pm-ignore-notify;
keep-power-in-suspend;
status = "okay";
};
diff --git a/arch/arm64/boot/dts/freescale/imx8mm-icore-mx8mm-edimm2.2.dts b/arch/arm64/boot/dts/freescale/imx8mm-icore-mx8mm-edimm2.2.dts
index a4a2ada..ddac8bc 100644
--- a/arch/arm64/boot/dts/freescale/imx8mm-icore-mx8mm-edimm2.2.dts
+++ b/arch/arm64/boot/dts/freescale/imx8mm-icore-mx8mm-edimm2.2.dts
@@ -91,7 +91,6 @@ &usdhc1 {
max-frequency = <50000000>;
bus-width = <4>;
no-1-8-v;
- pm-ignore-notify;
keep-power-in-suspend;
status = "okay";
};
diff --git a/arch/arm64/boot/dts/freescale/imx8mm-tqma8mqml-mba8mx.dts b/arch/arm64/boot/dts/freescale/imx8mm-tqma8mqml-mba8mx.dts
new file mode 100644
index 0000000..7844878
--- /dev/null
+++ b/arch/arm64/boot/dts/freescale/imx8mm-tqma8mqml-mba8mx.dts
@@ -0,0 +1,253 @@
+// SPDX-License-Identifier: (GPL-2.0-or-later OR MIT)
+/*
+ * Copyright 2020-2021 TQ-Systems GmbH
+ */
+
+/dts-v1/;
+
+#include "imx8mm-tqma8mqml.dtsi"
+#include "mba8mx.dtsi"
+
+/ {
+ model = "TQ-Systems GmbH i.MX8MM TQMa8MxML on MBa8Mx";
+ compatible = "tq,imx8mm-tqma8mqml-mba8mx", "tq,imx8mm-tqma8mqml", "fsl,imx8mm";
+
+ aliases {
+ eeprom0 = &eeprom3;
+ mmc0 = &usdhc3;
+ mmc1 = &usdhc2;
+ mmc2 = &usdhc1;
+ rtc0 = &pcf85063;
+ rtc1 = &snvs_rtc;
+ };
+
+ reg_usdhc2_vmmc: regulator-vmmc {
+ compatible = "regulator-fixed";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_reg_usdhc2_vmmc>;
+ regulator-name = "VSD_3V3";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ gpio = <&gpio2 19 GPIO_ACTIVE_HIGH>;
+ enable-active-high;
+ startup-delay-us = <100>;
+ off-on-delay-us = <12000>;
+ };
+
+ extcon_usbotg1: extcon-usbotg1 {
+ compatible = "linux,extcon-usb-gpio";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_usb1_extcon>;
+ id-gpio = <&gpio1 10 GPIO_ACTIVE_HIGH>;
+ };
+};
+
+&i2c1 {
+ expander2: gpio@27 {
+ compatible = "nxp,pca9555";
+ reg = <0x27>;
+ gpio-controller;
+ #gpio-cells = <2>;
+ vcc-supply = <®_vcc_3v3>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_expander>;
+ interrupt-parent = <&gpio1>;
+ interrupts = <9 IRQ_TYPE_EDGE_FALLING>;
+ interrupt-controller;
+ #interrupt-cells = <2>;
+ };
+};
+
+&sai3 {
+ assigned-clocks = <&clk IMX8MM_CLK_SAI3>;
+ assigned-clock-parents = <&clk IMX8MM_AUDIO_PLL1_OUT>;
+ clock-names = "bus", "mclk0", "mclk1", "mclk2", "mclk3", "pll8k", "pll11k";
+ clocks = <&clk IMX8MM_CLK_SAI3_IPG>, <&clk IMX8MM_CLK_DUMMY>,
+ <&clk IMX8MM_CLK_SAI3_ROOT>, <&clk IMX8MM_CLK_DUMMY>,
+ <&clk IMX8MM_CLK_DUMMY>, <&clk IMX8MM_AUDIO_PLL1_OUT>,
+ <&clk IMX8MM_AUDIO_PLL2_OUT>;
+};
+
+&tlv320aic3x04 {
+ clock-names = "mclk";
+ clocks = <&clk IMX8MM_CLK_SAI3_ROOT>;
+};
+
+&uart1 {
+ assigned-clocks = <&clk IMX8MM_CLK_UART1>;
+ assigned-clock-parents = <&clk IMX8MM_SYS_PLL1_80M>;
+};
+
+&uart2 {
+ assigned-clocks = <&clk IMX8MM_CLK_UART2>;
+ assigned-clock-parents = <&clk IMX8MM_SYS_PLL1_80M>;
+};
+
+&usbotg1 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_usbotg1>;
+ dr_mode = "otg";
+ extcon = <&extcon_usbotg1>;
+ srp-disable;
+ hnp-disable;
+ adp-disable;
+ power-active-high;
+ over-current-active-low;
+ status = "okay";
+};
+
+&usbotg2 {
+ dr_mode = "host";
+ disable-over-current;
+ vbus-supply = <®_hub_vbus>;
+ status = "okay";
+};
+
+&iomuxc {
+ pinctrl_ecspi1: ecspi1grp {
+ fsl,pins = <MX8MM_IOMUXC_ECSPI1_SCLK_ECSPI1_SCLK 0x00000006>,
+ <MX8MM_IOMUXC_ECSPI1_MOSI_ECSPI1_MOSI 0x00000006>,
+ <MX8MM_IOMUXC_ECSPI1_MISO_ECSPI1_MISO 0x00000006>,
+ <MX8MM_IOMUXC_ECSPI1_SS0_GPIO5_IO9 0x00000006>;
+ };
+
+ pinctrl_ecspi2: ecspi2grp {
+ fsl,pins = <MX8MM_IOMUXC_ECSPI2_SCLK_ECSPI2_SCLK 0x00000006>,
+ <MX8MM_IOMUXC_ECSPI2_MOSI_ECSPI2_MOSI 0x00000006>,
+ <MX8MM_IOMUXC_ECSPI2_MISO_ECSPI2_MISO 0x00000006>,
+ <MX8MM_IOMUXC_ECSPI2_SS0_GPIO5_IO13 0x00000006>;
+ };
+
+ pinctrl_expander: expandergrp {
+ fsl,pins = <MX8MM_IOMUXC_GPIO1_IO09_GPIO1_IO9 0x94>;
+ };
+
+ pinctrl_fec1: fec1grp {
+ fsl,pins = <MX8MM_IOMUXC_ENET_MDC_ENET1_MDC 0x40000002>,
+ <MX8MM_IOMUXC_ENET_MDIO_ENET1_MDIO 0x40000002>,
+ <MX8MM_IOMUXC_ENET_TD3_ENET1_RGMII_TD3 0x14>,
+ <MX8MM_IOMUXC_ENET_TD2_ENET1_RGMII_TD2 0x14>,
+ <MX8MM_IOMUXC_ENET_TD1_ENET1_RGMII_TD1 0x14>,
+ <MX8MM_IOMUXC_ENET_TD0_ENET1_RGMII_TD0 0x14>,
+ <MX8MM_IOMUXC_ENET_RD3_ENET1_RGMII_RD3 0x90>,
+ <MX8MM_IOMUXC_ENET_RD2_ENET1_RGMII_RD2 0x90>,
+ <MX8MM_IOMUXC_ENET_RD1_ENET1_RGMII_RD1 0x90>,
+ <MX8MM_IOMUXC_ENET_RD0_ENET1_RGMII_RD0 0x90>,
+ <MX8MM_IOMUXC_ENET_TXC_ENET1_RGMII_TXC 0x14>,
+ <MX8MM_IOMUXC_ENET_RXC_ENET1_RGMII_RXC 0x90>,
+ <MX8MM_IOMUXC_ENET_RX_CTL_ENET1_RGMII_RX_CTL 0x90>,
+ <MX8MM_IOMUXC_ENET_TX_CTL_ENET1_RGMII_TX_CTL 0x14>;
+ };
+
+ pinctrl_gpiobutton: gpiobuttongrp {
+ fsl,pins = <MX8MM_IOMUXC_GPIO1_IO05_GPIO1_IO5 0x84>,
+ <MX8MM_IOMUXC_GPIO1_IO07_GPIO1_IO7 0x84>,
+ <MX8MM_IOMUXC_SD1_CLK_GPIO2_IO0 0x84>;
+ };
+
+ pinctrl_gpioled: gpioledgrp {
+ fsl,pins = <MX8MM_IOMUXC_GPIO1_IO00_GPIO1_IO0 0x84>,
+ <MX8MM_IOMUXC_NAND_DQS_GPIO3_IO14 0x84>;
+ };
+
+ pinctrl_i2c2: i2c2grp {
+ fsl,pins = <MX8MM_IOMUXC_I2C2_SCL_I2C2_SCL 0x40000004>,
+ <MX8MM_IOMUXC_I2C2_SDA_I2C2_SDA 0x40000004>;
+ };
+
+ pinctrl_i2c2_gpio: i2c2gpiogrp {
+ fsl,pins = <MX8MM_IOMUXC_I2C2_SCL_GPIO5_IO16 0x40000004>,
+ <MX8MM_IOMUXC_I2C2_SDA_GPIO5_IO17 0x40000004>;
+ };
+
+ pinctrl_i2c3: i2c3grp {
+ fsl,pins = <MX8MM_IOMUXC_I2C3_SCL_I2C3_SCL 0x40000004>,
+ <MX8MM_IOMUXC_I2C3_SDA_I2C3_SDA 0x40000004>;
+ };
+
+ pinctrl_i2c3_gpio: i2c3gpiogrp {
+ fsl,pins = <MX8MM_IOMUXC_I2C3_SCL_GPIO5_IO18 0x40000004>,
+ <MX8MM_IOMUXC_I2C3_SDA_GPIO5_IO19 0x40000004>;
+ };
+
+ pinctrl_pwm3: pwm3grp {
+ fsl,pins = <MX8MM_IOMUXC_GPIO1_IO14_PWM3_OUT 0x14>;
+ };
+
+ pinctrl_pwm4: pwm4grp {
+ fsl,pins = <MX8MM_IOMUXC_GPIO1_IO15_PWM4_OUT 0x14>;
+ };
+
+ pinctrl_sai3: sai3grp {
+ fsl,pins = <MX8MM_IOMUXC_SAI3_MCLK_SAI3_MCLK 0x94>,
+ <MX8MM_IOMUXC_SAI3_RXC_SAI3_RX_BCLK 0x94>,
+ <MX8MM_IOMUXC_SAI3_RXFS_SAI3_RX_SYNC 0x94>,
+ <MX8MM_IOMUXC_SAI3_RXD_SAI3_RX_DATA0 0x94>,
+ <MX8MM_IOMUXC_SAI3_TXFS_SAI3_TX_SYNC 0x94>,
+ <MX8MM_IOMUXC_SAI3_TXD_SAI3_TX_DATA0 0x94>,
+ <MX8MM_IOMUXC_SAI3_TXC_SAI3_TX_BCLK 0x94>;
+ };
+
+ pinctrl_uart1: uart1grp {
+ fsl,pins = <MX8MM_IOMUXC_UART1_RXD_UART1_DCE_RX 0x16>,
+ <MX8MM_IOMUXC_UART1_TXD_UART1_DCE_TX 0x16>;
+ };
+
+ pinctrl_uart2: uart2grp {
+ fsl,pins = <MX8MM_IOMUXC_UART2_RXD_UART2_DCE_RX 0x16>,
+ <MX8MM_IOMUXC_UART2_TXD_UART2_DCE_TX 0x16>;
+ };
+
+ pinctrl_uart3: uart3grp {
+ fsl,pins = <MX8MM_IOMUXC_UART3_RXD_UART3_DCE_RX 0x16>,
+ <MX8MM_IOMUXC_UART3_TXD_UART3_DCE_TX 0x16>;
+ };
+
+ pinctrl_uart4: uart4grp {
+ fsl,pins = <MX8MM_IOMUXC_UART4_RXD_UART4_DCE_RX 0x16>,
+ <MX8MM_IOMUXC_UART4_TXD_UART4_DCE_TX 0x16>;
+ };
+
+ pinctrl_usbotg1: usbotg1grp {
+ fsl,pins = <MX8MM_IOMUXC_GPIO1_IO12_USB1_OTG_PWR 0x84>,
+ <MX8MM_IOMUXC_GPIO1_IO13_USB1_OTG_OC 0x84>;
+ };
+
+ pinctrl_usb1_extcon: usb1-extcongrp {
+ fsl,pins = <MX8MM_IOMUXC_GPIO1_IO10_GPIO1_IO10 0x1c0>;
+ };
+
+ pinctrl_usdhc2_gpio: usdhc2grpgpiogrp {
+ fsl,pins = <MX8MM_IOMUXC_SD2_CD_B_GPIO2_IO12 0x84>;
+ };
+
+ pinctrl_usdhc2: usdhc2grp {
+ fsl,pins = <MX8MM_IOMUXC_SD2_CLK_USDHC2_CLK 0x1d4>,
+ <MX8MM_IOMUXC_SD2_CMD_USDHC2_CMD 0x1d4>,
+ <MX8MM_IOMUXC_SD2_DATA0_USDHC2_DATA0 0x1d4>,
+ <MX8MM_IOMUXC_SD2_DATA1_USDHC2_DATA1 0x1d4>,
+ <MX8MM_IOMUXC_SD2_DATA2_USDHC2_DATA2 0x1d4>,
+ <MX8MM_IOMUXC_SD2_DATA3_USDHC2_DATA3 0x1d4>,
+ <MX8MM_IOMUXC_GPIO1_IO04_USDHC2_VSELECT 0x84>;
+ };
+
+ pinctrl_usdhc2_100mhz: usdhc2-100mhzgrp {
+ fsl,pins = <MX8MM_IOMUXC_SD2_CLK_USDHC2_CLK 0x1d4>,
+ <MX8MM_IOMUXC_SD2_CMD_USDHC2_CMD 0x1d4>,
+ <MX8MM_IOMUXC_SD2_DATA0_USDHC2_DATA0 0x1d4>,
+ <MX8MM_IOMUXC_SD2_DATA1_USDHC2_DATA1 0x1d4>,
+ <MX8MM_IOMUXC_SD2_DATA2_USDHC2_DATA2 0x1d4>,
+ <MX8MM_IOMUXC_SD2_DATA3_USDHC2_DATA3 0x1d4>,
+ <MX8MM_IOMUXC_GPIO1_IO04_USDHC2_VSELECT 0x84>;
+ };
+
+ pinctrl_usdhc2_200mhz: usdhc2-200mhzgrp {
+ fsl,pins = <MX8MM_IOMUXC_SD2_CLK_USDHC2_CLK 0x1d4>,
+ <MX8MM_IOMUXC_SD2_CMD_USDHC2_CMD 0x1d4>,
+ <MX8MM_IOMUXC_SD2_DATA0_USDHC2_DATA0 0x1d4>,
+ <MX8MM_IOMUXC_SD2_DATA1_USDHC2_DATA1 0x1d4>,
+ <MX8MM_IOMUXC_SD2_DATA2_USDHC2_DATA2 0x1d4>,
+ <MX8MM_IOMUXC_SD2_DATA3_USDHC2_DATA3 0x1d4>,
+ <MX8MM_IOMUXC_GPIO1_IO04_USDHC2_VSELECT 0x84>;
+ };
+};
diff --git a/arch/arm64/boot/dts/freescale/imx8mm-tqma8mqml.dtsi b/arch/arm64/boot/dts/freescale/imx8mm-tqma8mqml.dtsi
new file mode 100644
index 0000000..284e62ac
--- /dev/null
+++ b/arch/arm64/boot/dts/freescale/imx8mm-tqma8mqml.dtsi
@@ -0,0 +1,335 @@
+// SPDX-License-Identifier: (GPL-2.0-or-later OR MIT)
+/*
+ * Copyright 2020-2021 TQ-Systems GmbH
+ */
+
+#include "imx8mm.dtsi"
+
+/ {
+ model = "TQ-Systems GmbH i.MX8MM TQMa8MxML";
+ compatible = "tq,imx8mm-tqma8mqml", "fsl,imx8mm";
+
+ memory@40000000 {
+ device_type = "memory";
+ /* our minimum RAM config will be 1024 MiB */
+ reg = <0x00000000 0x40000000 0 0x40000000>;
+ };
+
+ /* e-MMC IO, needed for HS modes */
+ reg_vcc1v8: regulator-vcc1v8 {
+ compatible = "regulator-fixed";
+ regulator-name = "TQMA8MXML_VCC1V8";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ };
+
+ /* identical to buck4_reg, but should never change */
+ reg_vcc3v3: regulator-vcc3v3 {
+ compatible = "regulator-fixed";
+ regulator-name = "TQMA8MXML_VCC3V3";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ };
+
+ reserved-memory {
+ #address-cells = <2>;
+ #size-cells = <2>;
+ ranges;
+
+ /* global autoconfigured region for contiguous allocations */
+ linux,cma {
+ compatible = "shared-dma-pool";
+ reusable;
+ /* 640 MiB */
+ size = <0 0x28000000>;
+ /* 1024 - 128 MiB, our minimum RAM config will be 1024 MiB */
+ alloc-ranges = <0 0x40000000 0 0x78000000>;
+ linux,cma-default;
+ };
+ };
+};
+
+&A53_0 {
+ cpu-supply = <&buck2_reg>;
+};
+
+&flexspi {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_flexspi>;
+ status = "okay";
+
+ flash0: flash@0 {
+ compatible = "jedec,spi-nor";
+ reg = <0>;
+ #address-cells = <1>;
+ #size-cells = <1>;
+ spi-max-frequency = <84000000>;
+ spi-tx-bus-width = <1>;
+ spi-rx-bus-width = <4>;
+ };
+};
+
+&gpu_2d {
+ status = "okay";
+};
+
+&gpu_3d {
+ status = "okay";
+};
+
+&i2c1 {
+ clock-frequency = <100000>;
+ pinctrl-names = "default", "gpio";
+ pinctrl-0 = <&pinctrl_i2c1>;
+ pinctrl-1 = <&pinctrl_i2c1_gpio>;
+ scl-gpios = <&gpio5 14 (GPIO_ACTIVE_HIGH | GPIO_OPEN_DRAIN)>;
+ sda-gpios = <&gpio5 15 (GPIO_ACTIVE_HIGH | GPIO_OPEN_DRAIN)>;
+ status = "okay";
+
+ sensor0: temperature-sensor-eeprom@1b {
+ compatible = "nxp,se97", "jedec,jc-42.4-temp";
+ reg = <0x1b>;
+ };
+
+ pca9450: pmic@25 {
+ compatible = "nxp,pca9450a";
+ reg = <0x25>;
+
+ /* PMIC PCA9450 PMIC_nINT GPIO1_IO08 */
+ pinctrl-0 = <&pinctrl_pmic>;
+ pinctrl-names = "default";
+ interrupt-parent = <&gpio1>;
+ interrupts = <8 IRQ_TYPE_LEVEL_LOW>;
+
+ regulators {
+ /* V_0V85_SOC: 0.85 */
+ buck1_reg: BUCK1 {
+ regulator-name = "BUCK1";
+ regulator-min-microvolt = <850000>;
+ regulator-max-microvolt = <850000>;
+ regulator-boot-on;
+ regulator-always-on;
+ regulator-ramp-delay = <3125>;
+ };
+
+ /* VDD_ARM */
+ buck2_reg: BUCK2 {
+ regulator-name = "BUCK2";
+ regulator-min-microvolt = <850000>;
+ regulator-max-microvolt = <1000000>;
+ regulator-boot-on;
+ regulator-always-on;
+ nxp,dvs-run-voltage = <950000>;
+ nxp,dvs-standby-voltage = <850000>;
+ regulator-ramp-delay = <3125>;
+ };
+
+ /* V_0V85_GPU / DRAM / VPU */
+ buck3_reg: BUCK3 {
+ regulator-name = "BUCK3";
+ regulator-min-microvolt = <850000>;
+ regulator-max-microvolt = <950000>;
+ regulator-boot-on;
+ regulator-always-on;
+ regulator-ramp-delay = <3125>;
+ };
+
+ /* VCC3V3 -> VMMC, ... must not be changed */
+ buck4_reg: BUCK4 {
+ regulator-name = "BUCK4";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-boot-on;
+ regulator-always-on;
+ };
+
+ /* V_1V8 -> VQMMC, SPI-NOR, ... must not be changed */
+ buck5_reg: BUCK5 {
+ regulator-name = "BUCK5";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ regulator-boot-on;
+ regulator-always-on;
+ };
+
+ /* V_1V1 -> RAM, ... must not be changed */
+ buck6_reg: BUCK6 {
+ regulator-name = "BUCK6";
+ regulator-min-microvolt = <1100000>;
+ regulator-max-microvolt = <1100000>;
+ regulator-boot-on;
+ regulator-always-on;
+ };
+
+ /* V_1V8_SNVS */
+ ldo1_reg: LDO1 {
+ regulator-name = "LDO1";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ regulator-boot-on;
+ regulator-always-on;
+ };
+
+ /* V_0V8_SNVS */
+ ldo2_reg: LDO2 {
+ regulator-name = "LDO2";
+ regulator-min-microvolt = <800000>;
+ regulator-max-microvolt = <850000>;
+ regulator-boot-on;
+ regulator-always-on;
+ };
+
+ /* V_1V8_ANA */
+ ldo3_reg: LDO3 {
+ regulator-name = "LDO3";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ regulator-boot-on;
+ regulator-always-on;
+ };
+
+ /* V_0V9_MIPI */
+ ldo4_reg: LDO4 {
+ regulator-name = "LDO4";
+ regulator-min-microvolt = <900000>;
+ regulator-max-microvolt = <900000>;
+ regulator-boot-on;
+ regulator-always-on;
+ };
+
+ /* VCC SD IO - switched using SD2 VSELECT */
+ ldo5_reg: LDO5 {
+ regulator-name = "LDO5";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <3300000>;
+ };
+ };
+ };
+
+
+ pcf85063: rtc@51 {
+ compatible = "nxp,pcf85063a";
+ reg = <0x51>;
+ quartz-load-femtofarads = <7000>;
+ };
+
+ eeprom1: eeprom@53 {
+ compatible = "nxp,se97b", "atmel,24c02";
+ read-only;
+ reg = <0x53>;
+ pagesize = <16>;
+ };
+
+ eeprom0: eeprom@57 {
+ compatible = "atmel,24c64";
+ reg = <0x57>;
+ pagesize = <32>;
+ };
+};
+
+&usdhc3 {
+ pinctrl-names = "default", "state_100mhz", "state_200mhz";
+ pinctrl-0 = <&pinctrl_usdhc3>;
+ pinctrl-1 = <&pinctrl_usdhc3_100mhz>;
+ pinctrl-2 = <&pinctrl_usdhc3_200mhz>;
+ bus-width = <8>;
+ non-removable;
+ no-sd;
+ no-sdio;
+ vmmc-supply = <®_vcc3v3>;
+ vqmmc-supply = <®_vcc1v8>;
+ status = "okay";
+};
+
+/*
+ * Attention:
+ * wdog reset is routed to PMIC, PMIC must be preconfigured to force POR
+ * without LDO for SNVS. GPIO1_IO02 must not be used as GPIO.
+ */
+&wdog1 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_wdog>;
+ fsl,ext-reset-output;
+ status = "okay";
+};
+
+&iomuxc {
+ pinctrl_flexspi: flexspigrp {
+ fsl,pins = <MX8MM_IOMUXC_NAND_ALE_QSPI_A_SCLK 0x82>,
+ <MX8MM_IOMUXC_NAND_CE0_B_QSPI_A_SS0_B 0x82>,
+ <MX8MM_IOMUXC_NAND_DATA00_QSPI_A_DATA0 0x82>,
+ <MX8MM_IOMUXC_NAND_DATA01_QSPI_A_DATA1 0x82>,
+ <MX8MM_IOMUXC_NAND_DATA02_QSPI_A_DATA2 0x82>,
+ <MX8MM_IOMUXC_NAND_DATA03_QSPI_A_DATA3 0x82>;
+ };
+
+ pinctrl_i2c1: i2c1grp {
+ fsl,pins = <MX8MM_IOMUXC_I2C1_SCL_I2C1_SCL 0x40000004>,
+ <MX8MM_IOMUXC_I2C1_SDA_I2C1_SDA 0x40000004>;
+ };
+
+ pinctrl_i2c1_gpio: i2c1gpiogrp {
+ fsl,pins = <MX8MM_IOMUXC_I2C1_SCL_GPIO5_IO14 0x40000004>,
+ <MX8MM_IOMUXC_I2C1_SDA_GPIO5_IO15 0x40000004>;
+ };
+
+ pinctrl_pmic: pmicgrp {
+ fsl,pins = <MX8MM_IOMUXC_GPIO1_IO08_GPIO1_IO8 0x94>;
+ };
+
+ pinctrl_reg_usdhc2_vmmc: regusdhc2vmmcgrp {
+ fsl,pins = <MX8MM_IOMUXC_SD2_RESET_B_GPIO2_IO19 0x84>;
+ };
+
+ pinctrl_usdhc3: usdhc3grp {
+ fsl,pins = <MX8MM_IOMUXC_NAND_WE_B_USDHC3_CLK 0x1d4>,
+ <MX8MM_IOMUXC_NAND_WP_B_USDHC3_CMD 0x1d2>,
+ <MX8MM_IOMUXC_NAND_DATA04_USDHC3_DATA0 0x1d4>,
+ <MX8MM_IOMUXC_NAND_DATA05_USDHC3_DATA1 0x1d4>,
+ <MX8MM_IOMUXC_NAND_DATA06_USDHC3_DATA2 0x1d4>,
+ <MX8MM_IOMUXC_NAND_DATA07_USDHC3_DATA3 0x1d4>,
+ <MX8MM_IOMUXC_NAND_RE_B_USDHC3_DATA4 0x1d4>,
+ <MX8MM_IOMUXC_NAND_CE2_B_USDHC3_DATA5 0x1d4>,
+ <MX8MM_IOMUXC_NAND_CE3_B_USDHC3_DATA6 0x1d4>,
+ <MX8MM_IOMUXC_NAND_CLE_USDHC3_DATA7 0x1d4>,
+ <MX8MM_IOMUXC_NAND_CE1_B_USDHC3_STROBE 0x84>,
+ /* option USDHC3_RESET_B not defined, only in RM */
+ <MX8MM_IOMUXC_NAND_READY_B_GPIO3_IO16 0x84>;
+ };
+
+ pinctrl_usdhc3_100mhz: usdhc3-100mhzgrp {
+ fsl,pins = <MX8MM_IOMUXC_NAND_WE_B_USDHC3_CLK 0x1d2>,
+ <MX8MM_IOMUXC_NAND_WP_B_USDHC3_CMD 0x1d2>,
+ <MX8MM_IOMUXC_NAND_DATA04_USDHC3_DATA0 0x1d4>,
+ <MX8MM_IOMUXC_NAND_DATA05_USDHC3_DATA1 0x1d4>,
+ <MX8MM_IOMUXC_NAND_DATA06_USDHC3_DATA2 0x1d4>,
+ <MX8MM_IOMUXC_NAND_DATA07_USDHC3_DATA3 0x1d4>,
+ <MX8MM_IOMUXC_NAND_RE_B_USDHC3_DATA4 0x1d4>,
+ <MX8MM_IOMUXC_NAND_CE2_B_USDHC3_DATA5 0x1d4>,
+ <MX8MM_IOMUXC_NAND_CE3_B_USDHC3_DATA6 0x1d4>,
+ <MX8MM_IOMUXC_NAND_CLE_USDHC3_DATA7 0x1d4>,
+ <MX8MM_IOMUXC_NAND_CE1_B_USDHC3_STROBE 0x84>,
+ /* option USDHC3_RESET_B not defined, only in RM */
+ <MX8MM_IOMUXC_NAND_READY_B_GPIO3_IO16 0x84>;
+ };
+
+ pinctrl_usdhc3_200mhz: usdhc3-200mhzgrp {
+ fsl,pins = <MX8MM_IOMUXC_NAND_WE_B_USDHC3_CLK 0x1d6>,
+ <MX8MM_IOMUXC_NAND_WP_B_USDHC3_CMD 0x1d2>,
+ <MX8MM_IOMUXC_NAND_DATA04_USDHC3_DATA0 0x1d4>,
+ <MX8MM_IOMUXC_NAND_DATA05_USDHC3_DATA1 0x1d4>,
+ <MX8MM_IOMUXC_NAND_DATA06_USDHC3_DATA2 0x1d4>,
+ <MX8MM_IOMUXC_NAND_DATA07_USDHC3_DATA3 0x1d4>,
+ <MX8MM_IOMUXC_NAND_RE_B_USDHC3_DATA4 0x1d4>,
+ <MX8MM_IOMUXC_NAND_CE2_B_USDHC3_DATA5 0x1d4>,
+ <MX8MM_IOMUXC_NAND_CE3_B_USDHC3_DATA6 0x1d4>,
+ <MX8MM_IOMUXC_NAND_CLE_USDHC3_DATA7 0x1d4>,
+ <MX8MM_IOMUXC_NAND_CE1_B_USDHC3_STROBE 0x84>,
+ /* option USDHC3_RESET_B not defined, only in RM */
+ <MX8MM_IOMUXC_NAND_READY_B_GPIO3_IO16 0x84>;
+ };
+
+ pinctrl_wdog: wdoggrp {
+ fsl,pins = <MX8MM_IOMUXC_GPIO1_IO02_WDOG1_WDOG_B 0x84>;
+ };
+};
diff --git a/arch/arm64/boot/dts/freescale/imx8mm.dtsi b/arch/arm64/boot/dts/freescale/imx8mm.dtsi
index c2f3f11..f77f90e 100644
--- a/arch/arm64/boot/dts/freescale/imx8mm.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8mm.dtsi
@@ -65,6 +65,12 @@ A53_0: cpu@0 {
clock-latency = <61036>; /* two CLK32 periods */
clocks = <&clk IMX8MM_CLK_ARM>;
enable-method = "psci";
+ i-cache-size = <0x8000>;
+ i-cache-line-size = <64>;
+ i-cache-sets = <256>;
+ d-cache-size = <0x8000>;
+ d-cache-line-size = <64>;
+ d-cache-sets = <128>;
next-level-cache = <&A53_L2>;
operating-points-v2 = <&a53_opp_table>;
nvmem-cells = <&cpu_speed_grade>;
@@ -80,6 +86,12 @@ A53_1: cpu@1 {
clock-latency = <61036>; /* two CLK32 periods */
clocks = <&clk IMX8MM_CLK_ARM>;
enable-method = "psci";
+ i-cache-size = <0x8000>;
+ i-cache-line-size = <64>;
+ i-cache-sets = <256>;
+ d-cache-size = <0x8000>;
+ d-cache-line-size = <64>;
+ d-cache-sets = <128>;
next-level-cache = <&A53_L2>;
operating-points-v2 = <&a53_opp_table>;
cpu-idle-states = <&cpu_pd_wait>;
@@ -93,6 +105,12 @@ A53_2: cpu@2 {
clock-latency = <61036>; /* two CLK32 periods */
clocks = <&clk IMX8MM_CLK_ARM>;
enable-method = "psci";
+ i-cache-size = <0x8000>;
+ i-cache-line-size = <64>;
+ i-cache-sets = <256>;
+ d-cache-size = <0x8000>;
+ d-cache-line-size = <64>;
+ d-cache-sets = <128>;
next-level-cache = <&A53_L2>;
operating-points-v2 = <&a53_opp_table>;
cpu-idle-states = <&cpu_pd_wait>;
@@ -106,6 +124,12 @@ A53_3: cpu@3 {
clock-latency = <61036>; /* two CLK32 periods */
clocks = <&clk IMX8MM_CLK_ARM>;
enable-method = "psci";
+ i-cache-size = <0x8000>;
+ i-cache-line-size = <64>;
+ i-cache-sets = <256>;
+ d-cache-size = <0x8000>;
+ d-cache-line-size = <64>;
+ d-cache-sets = <128>;
next-level-cache = <&A53_L2>;
operating-points-v2 = <&a53_opp_table>;
cpu-idle-states = <&cpu_pd_wait>;
@@ -114,6 +138,10 @@ A53_3: cpu@3 {
A53_L2: l2-cache0 {
compatible = "cache";
+ cache-level = <2>;
+ cache-size = <0x80000>;
+ cache-line-size = <64>;
+ cache-sets = <512>;
};
};
@@ -591,8 +619,7 @@ clk: clock-controller@30380000 {
<&clk IMX8MM_CLK_IPG_AUDIO_ROOT>,
<&clk IMX8MM_SYS_PLL3>,
<&clk IMX8MM_VIDEO_PLL1>,
- <&clk IMX8MM_AUDIO_PLL1>,
- <&clk IMX8MM_AUDIO_PLL2>;
+ <&clk IMX8MM_AUDIO_PLL1>;
assigned-clock-parents = <&clk IMX8MM_SYS_PLL1_800M>,
<&clk IMX8MM_ARM_PLL_OUT>,
<&clk IMX8MM_SYS_PLL3_OUT>,
@@ -602,8 +629,7 @@ clk: clock-controller@30380000 {
<400000000>,
<750000000>,
<594000000>,
- <393216000>,
- <361267200>;
+ <393216000>;
};
src: reset-controller@30390000 {
@@ -1054,7 +1080,6 @@ fec1: ethernet@30be0000 {
fsl,num-rx-queues = <3>;
nvmem-cells = <&fec_mac_address>;
nvmem-cell-names = "mac-address";
- nvmem_macaddr_swap;
fsl,stop-mode = <&gpr 0x10 3>;
status = "disabled";
};
@@ -1068,6 +1093,22 @@ aips4: bus@32c00000 {
#size-cells = <1>;
ranges = <0x32c00000 0x32c00000 0x400000>;
+ csi: csi@32e20000 {
+ compatible = "fsl,imx8mm-csi", "fsl,imx7-csi";
+ reg = <0x32e20000 0x1000>;
+ interrupts = <GIC_SPI 16 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&clk IMX8MM_CLK_CSI1_ROOT>;
+ clock-names = "mclk";
+ power-domains = <&disp_blk_ctrl IMX8MM_DISPBLK_PD_CSI_BRIDGE>;
+ status = "disabled";
+
+ port {
+ csi_in: endpoint {
+ remote-endpoint = <&imx8mm_mipi_csi_out>;
+ };
+ };
+ };
+
disp_blk_ctrl: blk-ctrl@32e28000 {
compatible = "fsl,imx8mm-disp-blk-ctrl", "syscon";
reg = <0x32e28000 0x100>;
@@ -1095,6 +1136,41 @@ disp_blk_ctrl: blk-ctrl@32e28000 {
#power-domain-cells = <1>;
};
+ mipi_csi: mipi-csi@32e30000 {
+ compatible = "fsl,imx8mm-mipi-csi2";
+ reg = <0x32e30000 0x1000>;
+ interrupts = <GIC_SPI 17 IRQ_TYPE_LEVEL_HIGH>;
+ assigned-clocks = <&clk IMX8MM_CLK_CSI1_CORE>,
+ <&clk IMX8MM_CLK_CSI1_PHY_REF>;
+ assigned-clock-parents = <&clk IMX8MM_SYS_PLL2_1000M>,
+ <&clk IMX8MM_SYS_PLL2_1000M>;
+ clock-frequency = <333000000>;
+ clocks = <&clk IMX8MM_CLK_DISP_APB_ROOT>,
+ <&clk IMX8MM_CLK_CSI1_ROOT>,
+ <&clk IMX8MM_CLK_CSI1_PHY_REF>,
+ <&clk IMX8MM_CLK_DISP_AXI_ROOT>;
+ clock-names = "pclk", "wrap", "phy", "axi";
+ power-domains = <&disp_blk_ctrl IMX8MM_DISPBLK_PD_MIPI_CSI>;
+ status = "disabled";
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ };
+
+ port@1 {
+ reg = <1>;
+
+ imx8mm_mipi_csi_out: endpoint {
+ remote-endpoint = <&csi_in>;
+ };
+ };
+ };
+ };
+
usbotg1: usb@32e40000 {
compatible = "fsl,imx8mm-usb", "fsl,imx7d-usb";
reg = <0x32e40000 0x200>;
diff --git a/arch/arm64/boot/dts/freescale/imx8mn-beacon-baseboard.dtsi b/arch/arm64/boot/dts/freescale/imx8mn-beacon-baseboard.dtsi
index 376ca8f..0f40b43 100644
--- a/arch/arm64/boot/dts/freescale/imx8mn-beacon-baseboard.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8mn-beacon-baseboard.dtsi
@@ -126,7 +126,6 @@ wm8962: audio-codec@1a {
compatible = "wlf,wm8962";
reg = <0x1a>;
clocks = <&clk IMX8MN_CLK_SAI3_ROOT>;
- clock-names = "xclk";
DCVDD-supply = <®_audio>;
DBVDD-supply = <®_audio>;
AVDD-supply = <®_audio>;
diff --git a/arch/arm64/boot/dts/freescale/imx8mn-beacon-som.dtsi b/arch/arm64/boot/dts/freescale/imx8mn-beacon-som.dtsi
index 3b2d627..1133cde 100644
--- a/arch/arm64/boot/dts/freescale/imx8mn-beacon-som.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8mn-beacon-som.dtsi
@@ -274,7 +274,6 @@ &usdhc1 {
bus-width = <4>;
non-removable;
cap-power-off-card;
- pm-ignore-notify;
keep-power-in-suspend;
mmc-pwrseq = <&usdhc1_pwrseq>;
status = "okay";
diff --git a/arch/arm64/boot/dts/freescale/imx8mn-bsh-smm-s2-common.dtsi b/arch/arm64/boot/dts/freescale/imx8mn-bsh-smm-s2-common.dtsi
new file mode 100644
index 0000000..c11895d
--- /dev/null
+++ b/arch/arm64/boot/dts/freescale/imx8mn-bsh-smm-s2-common.dtsi
@@ -0,0 +1,426 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Copyright 2021 Collabora Ltd.
+ * Copyright 2021 BSH Hausgeraete GmbH
+ */
+
+/dts-v1/;
+
+#include "imx8mn.dtsi"
+
+/ {
+ chosen {
+ stdout-path = &uart4;
+ };
+
+ fec_supply: fec-supply-en {
+ compatible = "regulator-fixed";
+ vin-supply = <&buck4_reg>;
+ regulator-name = "tja1101_en";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ gpio = <&gpio2 20 GPIO_ACTIVE_HIGH>;
+ enable-active-high;
+ };
+
+ usdhc2_pwrseq: usdhc2-pwrseq {
+ compatible = "mmc-pwrseq-simple";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_usdhc2_pwrseq>;
+ reset-gpios = <&gpio4 27 GPIO_ACTIVE_LOW>;
+ };
+};
+
+&A53_0 {
+ cpu-supply = <&buck2_reg>;
+};
+
+&A53_1 {
+ cpu-supply = <&buck2_reg>;
+};
+
+&A53_2 {
+ cpu-supply = <&buck2_reg>;
+};
+
+&A53_3 {
+ cpu-supply = <&buck2_reg>;
+};
+
+&ecspi2 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_espi2>;
+ status = "okay";
+};
+
+&fec1 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_fec1>;
+ phy-mode = "rmii";
+ phy-handle = <ðphy0>;
+ phy-supply = <&fec_supply>;
+ fsl,magic-packet;
+ status = "okay";
+
+ mdio {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ ethphy0: ethernet-phy@0 {
+ compatible = "ethernet-phy-ieee802.3-c22";
+ reg = <0>;
+ reset-gpios = <&gpio1 29 GPIO_ACTIVE_LOW>;
+ reset-assert-us = <20>;
+ reset-deassert-us = <2000>;
+ };
+ };
+};
+
+&i2c1 {
+ clock-frequency = <400000>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_i2c1>;
+ status = "okay";
+
+ bd71847: pmic@4b {
+ compatible = "rohm,bd71847";
+ reg = <0x4b>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_pmic>;
+ interrupt-parent = <&gpio1>;
+ interrupts = <3 IRQ_TYPE_LEVEL_LOW>;
+ rohm,reset-snvs-powered;
+
+ #clock-cells = <0>;
+ clocks = <&osc_32k 0>;
+ clock-output-names = "clk-32k-out";
+
+ regulators {
+ buck1_reg: BUCK1 {
+ /* PMIC_BUCK1 - VDD_SOC */
+ regulator-name = "buck1";
+ regulator-min-microvolt = <700000>;
+ regulator-max-microvolt = <1300000>;
+ regulator-boot-on;
+ regulator-always-on;
+ regulator-ramp-delay = <1250>;
+ };
+
+ buck2_reg: BUCK2 {
+ /* PMIC_BUCK2 - VDD_ARM */
+ regulator-name = "buck2";
+ regulator-min-microvolt = <700000>;
+ regulator-max-microvolt = <1300000>;
+ regulator-boot-on;
+ regulator-always-on;
+ regulator-ramp-delay = <1250>;
+ };
+
+ buck3_reg: BUCK3 {
+ /* PMIC_BUCK5 - VDD_DRAM_VPU_GPU */
+ regulator-name = "buck3";
+ regulator-min-microvolt = <700000>;
+ regulator-max-microvolt = <1350000>;
+ regulator-boot-on;
+ regulator-always-on;
+ };
+
+ buck4_reg: BUCK4 {
+ /* PMIC_BUCK6 - VDD_3V3 */
+ regulator-name = "buck4";
+ regulator-min-microvolt = <3000000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-boot-on;
+ regulator-always-on;
+ };
+
+ buck5_reg: BUCK5 {
+ /* PMIC_BUCK7 - VDD_1V8 */
+ regulator-name = "buck5";
+ regulator-min-microvolt = <1605000>;
+ regulator-max-microvolt = <1995000>;
+ regulator-boot-on;
+ regulator-always-on;
+ };
+
+ buck6_reg: BUCK6 {
+ /* PMIC_BUCK8 - NVCC_DRAM */
+ regulator-name = "buck6";
+ regulator-min-microvolt = <800000>;
+ regulator-max-microvolt = <1400000>;
+ regulator-boot-on;
+ regulator-always-on;
+ };
+
+ ldo1_reg: LDO1 {
+ /* PMIC_LDO1 - NVCC_SNVS_1V8 */
+ regulator-name = "ldo1";
+ regulator-min-microvolt = <1600000>;
+ regulator-max-microvolt = <1900000>;
+ regulator-boot-on;
+ regulator-always-on;
+ };
+
+ ldo2_reg: LDO2 {
+ /* PMIC_LDO2 - VDD_SNVS_0V8 */
+ regulator-name = "ldo2";
+ regulator-min-microvolt = <800000>;
+ regulator-max-microvolt = <900000>;
+ regulator-boot-on;
+ regulator-always-on;
+ };
+
+ ldo3_reg: LDO3 {
+ /* PMIC_LDO3 - VDDA_1V8 */
+ regulator-name = "ldo3";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-boot-on;
+ regulator-always-on;
+ };
+
+ ldo4_reg: LDO4 {
+ /* PMIC_LDO4 - VDD_MIPI_0V9 */
+ regulator-name = "ldo4";
+ regulator-min-microvolt = <900000>;
+ regulator-max-microvolt = <1800000>;
+ regulator-boot-on;
+ regulator-always-on;
+ };
+
+ ldo6_reg: LDO6 {
+ /* PMIC_LDO6 - VDD_MIPI_1V2 */
+ regulator-name = "ldo6";
+ regulator-min-microvolt = <900000>;
+ regulator-max-microvolt = <1800000>;
+ regulator-boot-on;
+ regulator-always-on;
+ };
+ };
+ };
+};
+
+&i2c3 {
+ clock-frequency = <400000>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_i2c3>;
+ status = "okay";
+};
+
+&i2c4 {
+ clock-frequency = <400000>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_i2c4>;
+ status = "okay";
+};
+
+&uart2 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_uart2>;
+ status = "okay";
+};
+
+&uart3 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_uart3>;
+ assigned-clocks = <&clk IMX8MN_CLK_UART3>;
+ assigned-clock-parents = <&clk IMX8MN_SYS_PLL1_80M>;
+ uart-has-rtscts;
+ status = "okay";
+
+ bluetooth {
+ compatible = "brcm,bcm43438-bt";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_bluetooth>;
+ shutdown-gpios = <&gpio1 15 GPIO_ACTIVE_HIGH>;
+ device-wakeup-gpios = <&gpio1 18 GPIO_ACTIVE_HIGH>;
+ host-wakeup-gpios = <&gpio1 28 GPIO_ACTIVE_HIGH>;
+ max-speed = <3000000>;
+ };
+};
+
+/* Console */
+&uart4 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_uart4>;
+ status = "okay";
+};
+
+&usbotg1 {
+ dr_mode = "peripheral";
+ disable-over-current;
+ status = "okay";
+};
+
+&usdhc2 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ pinctrl-names = "default", "state_100mhz", "state_200mhz";
+ pinctrl-0 = <&pinctrl_usdhc2>;
+ pinctrl-1 = <&pinctrl_usdhc2_100mhz>;
+ pinctrl-2 = <&pinctrl_usdhc2_200mhz>;
+ mmc-pwrseq = <&usdhc2_pwrseq>;
+ bus-width = <4>;
+ non-removable;
+ status = "okay";
+
+ brcmf: bcrmf@1 {
+ compatible = "brcm,bcm4329-fmac";
+ reg = <1>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_wlan>;
+ interrupt-parent = <&gpio1>;
+ interrupts = <0 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "host-wake";
+ };
+};
+
+&wdog1 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_wdog>;
+ fsl,ext-reset-output;
+ status = "okay";
+};
+
+&iomuxc {
+ pinctrl_bluetooth: bluetoothgrp {
+ fsl,pins = <
+ MX8MN_IOMUXC_GPIO1_IO15_GPIO1_IO15 0x044 /* BT_REG_ON */
+ MX8MN_IOMUXC_ENET_TD3_GPIO1_IO18 0x046 /* BT_DEV_WAKE */
+ MX8MN_IOMUXC_ENET_RD2_GPIO1_IO28 0x090 /* BT_HOST_WAKE */
+ >;
+ };
+
+ pinctrl_espi2: espi2grp {
+ fsl,pins = <
+ MX8MN_IOMUXC_ECSPI2_SCLK_ECSPI2_SCLK 0x082
+ MX8MN_IOMUXC_ECSPI2_MOSI_ECSPI2_MOSI 0x082
+ MX8MN_IOMUXC_ECSPI2_MISO_ECSPI2_MISO 0x082
+ MX8MN_IOMUXC_ECSPI2_SS0_ECSPI2_SS0 0x040
+ >;
+ };
+
+ pinctrl_fec1: fec1grp {
+ fsl,pins = <
+ MX8MN_IOMUXC_ENET_MDC_ENET1_MDC 0x002
+ MX8MN_IOMUXC_ENET_MDIO_ENET1_MDIO 0x002
+ MX8MN_IOMUXC_ENET_RD0_ENET1_RGMII_RD0 0x090
+ MX8MN_IOMUXC_ENET_RD1_ENET1_RGMII_RD1 0x090
+ MX8MN_IOMUXC_ENET_RXC_ENET1_RX_ER 0x090
+ MX8MN_IOMUXC_ENET_TD0_ENET1_RGMII_TD0 0x016
+ MX8MN_IOMUXC_ENET_TD1_ENET1_RGMII_TD1 0x016
+ MX8MN_IOMUXC_ENET_TD2_ENET1_TX_CLK 0x016
+ MX8MN_IOMUXC_ENET_TX_CTL_ENET1_RGMII_TX_CTL 0x016
+ MX8MN_IOMUXC_ENET_RX_CTL_ENET1_RGMII_RX_CTL 0x090
+ MX8MN_IOMUXC_ENET_TXC_ENET1_TX_ER 0x016
+ MX8MN_IOMUXC_SD2_CD_B_GPIO2_IO12 0x150 /* RMII_INT - ENET_INT */
+ MX8MN_IOMUXC_SD2_WP_GPIO2_IO20 0x150 /* RMII_EN - ENET_EN */
+ MX8MN_IOMUXC_SD2_RESET_B_GPIO2_IO19 0x016 /* RMII_WAKE - GPIO_ENET_WAKE */
+ MX8MN_IOMUXC_ENET_RD3_GPIO1_IO29 0x016 /* RMII_RESET - GPIO_ENET_RST */
+ >;
+ };
+
+ pinctrl_i2c1: i2c1grp {
+ fsl,pins = <
+ MX8MN_IOMUXC_I2C1_SCL_I2C1_SCL 0x400000c2
+ MX8MN_IOMUXC_I2C1_SDA_I2C1_SDA 0x400000c2
+ >;
+ };
+
+ pinctrl_i2c3: i2c3grp {
+ fsl,pins = <
+ MX8MN_IOMUXC_I2C3_SCL_I2C3_SCL 0x400000c2
+ MX8MN_IOMUXC_I2C3_SDA_I2C3_SDA 0x400000c2
+ >;
+ };
+
+ pinctrl_i2c4: i2c4grp {
+ fsl,pins = <
+ MX8MN_IOMUXC_I2C4_SCL_I2C4_SCL 0x400000c2
+ MX8MN_IOMUXC_I2C4_SDA_I2C4_SDA 0x400000c2
+ >;
+ };
+
+ pinctrl_pmic: pmicirq {
+ fsl,pins = <
+ MX8MN_IOMUXC_GPIO1_IO03_GPIO1_IO3 0x040
+ >;
+ };
+
+ pinctrl_uart2: uart2grp {
+ fsl,pins = <
+ MX8MN_IOMUXC_UART2_RXD_UART2_DCE_RX 0x040
+ MX8MN_IOMUXC_UART2_TXD_UART2_DCE_TX 0x040
+ >;
+ };
+
+ pinctrl_uart3: uart3grp {
+ fsl,pins = <
+ MX8MN_IOMUXC_UART3_TXD_UART3_DCE_TX 0x040
+ MX8MN_IOMUXC_UART3_RXD_UART3_DCE_RX 0x040
+ MX8MN_IOMUXC_ECSPI1_MISO_UART3_DCE_CTS_B 0x040
+ MX8MN_IOMUXC_ECSPI1_SS0_UART3_DCE_RTS_B 0x040
+ >;
+ };
+
+ pinctrl_uart4: uart4grp {
+ fsl,pins = <
+ MX8MN_IOMUXC_UART4_RXD_UART4_DCE_RX 0x040
+ MX8MN_IOMUXC_UART4_TXD_UART4_DCE_TX 0x040
+ >;
+ };
+
+ pinctrl_usdhc2: usdhc2grp {
+ fsl,pins = <
+ MX8MN_IOMUXC_SD2_CLK_USDHC2_CLK 0x090
+ MX8MN_IOMUXC_SD2_CMD_USDHC2_CMD 0x0d0
+ MX8MN_IOMUXC_SD2_DATA0_USDHC2_DATA0 0x0d0
+ MX8MN_IOMUXC_SD2_DATA1_USDHC2_DATA1 0x0d0
+ MX8MN_IOMUXC_SD2_DATA2_USDHC2_DATA2 0x0d0
+ MX8MN_IOMUXC_SD2_DATA3_USDHC2_DATA3 0x0d0
+ >;
+ };
+
+ pinctrl_usdhc2_100mhz: usdhc2grp100mhz {
+ fsl,pins = <
+ MX8MN_IOMUXC_SD2_CLK_USDHC2_CLK 0x094
+ MX8MN_IOMUXC_SD2_CMD_USDHC2_CMD 0x0d4
+ MX8MN_IOMUXC_SD2_DATA0_USDHC2_DATA0 0x0d4
+ MX8MN_IOMUXC_SD2_DATA1_USDHC2_DATA1 0x0d4
+ MX8MN_IOMUXC_SD2_DATA2_USDHC2_DATA2 0x0d4
+ MX8MN_IOMUXC_SD2_DATA3_USDHC2_DATA3 0x0d4
+ >;
+ };
+
+ pinctrl_usdhc2_200mhz: usdhc2grp200mhz {
+ fsl,pins = <
+ MX8MN_IOMUXC_SD2_CLK_USDHC2_CLK 0x096
+ MX8MN_IOMUXC_SD2_CMD_USDHC2_CMD 0x0d6
+ MX8MN_IOMUXC_SD2_DATA0_USDHC2_DATA0 0x0d6
+ MX8MN_IOMUXC_SD2_DATA1_USDHC2_DATA1 0x0d6
+ MX8MN_IOMUXC_SD2_DATA2_USDHC2_DATA2 0x0d6
+ MX8MN_IOMUXC_SD2_DATA3_USDHC2_DATA3 0x0d6
+ >;
+ };
+
+ pinctrl_usdhc2_pwrseq: usdhc2pwrseqgrp {
+ fsl,pins = <
+ MX8MN_IOMUXC_SAI2_MCLK_GPIO4_IO27 0x040 /* WL_REG_ON */
+ >;
+ };
+
+ pinctrl_wdog: wdoggrp {
+ fsl,pins = <
+ MX8MN_IOMUXC_GPIO1_IO02_WDOG1_WDOG_B 0x046
+ >;
+ };
+
+ pinctrl_wlan: wlangrp {
+ fsl,pins = <
+ MX8MN_IOMUXC_GPIO1_IO00_GPIO1_IO0 0x0d6 /* GPIO_0 - WIFI_GPIO_0 */
+ MX8MN_IOMUXC_GPIO1_IO08_GPIO1_IO8 0x0d6 /* GPIO_1 - WIFI_GPIO_1 */
+ MX8MN_IOMUXC_GPIO1_IO04_GPIO1_IO4 0x0d6 /* BT_GPIO_5 - WIFI_GPIO_5 */
+ MX8MN_IOMUXC_SPDIF_RX_GPIO5_IO4 0x0d6 /* I2S_CLK - WIFI_GPIO_6 */
+ >;
+ };
+};
diff --git a/arch/arm64/boot/dts/freescale/imx8mn-bsh-smm-s2.dts b/arch/arm64/boot/dts/freescale/imx8mn-bsh-smm-s2.dts
new file mode 100644
index 0000000..33f9858
--- /dev/null
+++ b/arch/arm64/boot/dts/freescale/imx8mn-bsh-smm-s2.dts
@@ -0,0 +1,48 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Copyright 2021 Collabora Ltd.
+ * Copyright 2021 BSH Hausgeraete GmbH
+ */
+
+/dts-v1/;
+
+#include "imx8mn-bsh-smm-s2-common.dtsi"
+
+/ {
+ model = "BSH SMM S2";
+ compatible = "bsh,imx8mn-bsh-smm-s2", "fsl,imx8mn";
+
+ memory@40000000 {
+ device_type = "memory";
+ reg = <0x0 0x40000000 0x0 0x10000000>;
+ };
+};
+
+&gpmi {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_gpmi_nand>;
+ nand-on-flash-bbt;
+ status = "okay";
+};
+
+&iomuxc {
+ pinctrl_gpmi_nand: gpmi-nand {
+ fsl,pins = <
+ MX8MN_IOMUXC_NAND_ALE_RAWNAND_ALE 0x00000096
+ MX8MN_IOMUXC_NAND_CE0_B_RAWNAND_CE0_B 0x00000096
+ MX8MN_IOMUXC_NAND_CLE_RAWNAND_CLE 0x00000096
+ MX8MN_IOMUXC_NAND_DATA00_RAWNAND_DATA00 0x00000096
+ MX8MN_IOMUXC_NAND_DATA01_RAWNAND_DATA01 0x00000096
+ MX8MN_IOMUXC_NAND_DATA02_RAWNAND_DATA02 0x00000096
+ MX8MN_IOMUXC_NAND_DATA03_RAWNAND_DATA03 0x00000096
+ MX8MN_IOMUXC_NAND_DATA04_RAWNAND_DATA04 0x00000096
+ MX8MN_IOMUXC_NAND_DATA05_RAWNAND_DATA05 0x00000096
+ MX8MN_IOMUXC_NAND_DATA06_RAWNAND_DATA06 0x00000096
+ MX8MN_IOMUXC_NAND_DATA07_RAWNAND_DATA07 0x00000096
+ MX8MN_IOMUXC_NAND_RE_B_RAWNAND_RE_B 0x00000096
+ MX8MN_IOMUXC_NAND_READY_B_RAWNAND_READY_B 0x00000056
+ MX8MN_IOMUXC_NAND_WE_B_RAWNAND_WE_B 0x00000096
+ MX8MN_IOMUXC_NAND_WP_B_RAWNAND_WP_B 0x00000096
+ >;
+ };
+};
diff --git a/arch/arm64/boot/dts/freescale/imx8mn-bsh-smm-s2pro.dts b/arch/arm64/boot/dts/freescale/imx8mn-bsh-smm-s2pro.dts
new file mode 100644
index 0000000..c6a8ed6
--- /dev/null
+++ b/arch/arm64/boot/dts/freescale/imx8mn-bsh-smm-s2pro.dts
@@ -0,0 +1,80 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Copyright 2021 Collabora Ltd.
+ * Copyright 2021 BSH Hausgeraete GmbH
+ */
+
+/dts-v1/;
+
+#include "imx8mn-bsh-smm-s2-common.dtsi"
+
+/ {
+ model = "BSH SMM S2 PRO";
+ compatible = "bsh,imx8mn-bsh-smm-s2pro", "fsl,imx8mn";
+
+ memory@40000000 {
+ device_type = "memory";
+ reg = <0x0 0x40000000 0x0 0x20000000>;
+ };
+};
+
+/* eMMC */
+&usdhc1 {
+ pinctrl-names = "default", "state_100mhz", "state_200mhz";
+ pinctrl-0 = <&pinctrl_usdhc1>;
+ pinctrl-1 = <&pinctrl_usdhc1_100mhz>;
+ pinctrl-2 = <&pinctrl_usdhc1_200mhz>;
+ bus-width = <8>;
+ non-removable;
+ status = "okay";
+};
+
+&iomuxc {
+ pinctrl_usdhc1: usdhc1grp {
+ fsl,pins = <
+ MX8MN_IOMUXC_SD1_CLK_USDHC1_CLK 0x40000090
+ MX8MN_IOMUXC_SD1_CMD_USDHC1_CMD 0x0d0
+ MX8MN_IOMUXC_SD1_DATA0_USDHC1_DATA0 0x0d0
+ MX8MN_IOMUXC_SD1_DATA1_USDHC1_DATA1 0x0d0
+ MX8MN_IOMUXC_SD1_DATA2_USDHC1_DATA2 0x0d0
+ MX8MN_IOMUXC_SD1_DATA3_USDHC1_DATA3 0x0d0
+ MX8MN_IOMUXC_SD1_DATA4_USDHC1_DATA4 0x0d0
+ MX8MN_IOMUXC_SD1_DATA5_USDHC1_DATA5 0x0d0
+ MX8MN_IOMUXC_SD1_DATA6_USDHC1_DATA6 0x0d0
+ MX8MN_IOMUXC_SD1_DATA7_USDHC1_DATA7 0x0d0
+ MX8MN_IOMUXC_SD1_STROBE_USDHC1_STROBE 0x090
+ >;
+ };
+
+ pinctrl_usdhc1_100mhz: usdhc1grp100mhz {
+ fsl,pins = <
+ MX8MN_IOMUXC_SD1_CLK_USDHC1_CLK 0x40000094
+ MX8MN_IOMUXC_SD1_CMD_USDHC1_CMD 0x0d4
+ MX8MN_IOMUXC_SD1_DATA0_USDHC1_DATA0 0x0d4
+ MX8MN_IOMUXC_SD1_DATA1_USDHC1_DATA1 0x0d4
+ MX8MN_IOMUXC_SD1_DATA2_USDHC1_DATA2 0x0d4
+ MX8MN_IOMUXC_SD1_DATA3_USDHC1_DATA3 0x0d4
+ MX8MN_IOMUXC_SD1_DATA4_USDHC1_DATA4 0x0d4
+ MX8MN_IOMUXC_SD1_DATA5_USDHC1_DATA5 0x0d4
+ MX8MN_IOMUXC_SD1_DATA6_USDHC1_DATA6 0x0d4
+ MX8MN_IOMUXC_SD1_DATA7_USDHC1_DATA7 0x0d4
+ MX8MN_IOMUXC_SD1_STROBE_USDHC1_STROBE 0x094
+ >;
+ };
+
+ pinctrl_usdhc1_200mhz: usdhc1grp200mhz {
+ fsl,pins = <
+ MX8MN_IOMUXC_SD1_CLK_USDHC1_CLK 0x40000096
+ MX8MN_IOMUXC_SD1_CMD_USDHC1_CMD 0x0d6
+ MX8MN_IOMUXC_SD1_DATA0_USDHC1_DATA0 0x0d6
+ MX8MN_IOMUXC_SD1_DATA1_USDHC1_DATA1 0x0d6
+ MX8MN_IOMUXC_SD1_DATA2_USDHC1_DATA2 0x0d6
+ MX8MN_IOMUXC_SD1_DATA3_USDHC1_DATA3 0x0d6
+ MX8MN_IOMUXC_SD1_DATA4_USDHC1_DATA4 0x0d6
+ MX8MN_IOMUXC_SD1_DATA5_USDHC1_DATA5 0x0d6
+ MX8MN_IOMUXC_SD1_DATA6_USDHC1_DATA6 0x0d6
+ MX8MN_IOMUXC_SD1_DATA7_USDHC1_DATA7 0x0d6
+ MX8MN_IOMUXC_SD1_STROBE_USDHC1_STROBE 0x096
+ >;
+ };
+};
diff --git a/arch/arm64/boot/dts/freescale/imx8mn-evk.dtsi b/arch/arm64/boot/dts/freescale/imx8mn-evk.dtsi
index 85e65f8..c3f1519 100644
--- a/arch/arm64/boot/dts/freescale/imx8mn-evk.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8mn-evk.dtsi
@@ -97,6 +97,15 @@ mdio {
ethphy0: ethernet-phy@0 {
compatible = "ethernet-phy-ieee802.3-c22";
reg = <0>;
+ reset-gpios = <&gpio4 22 GPIO_ACTIVE_LOW>;
+ reset-assert-us = <10000>;
+ qca,disable-smarteee;
+ vddio-supply = <&vddio>;
+
+ vddio: vddio-regulator {
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ };
};
};
};
diff --git a/arch/arm64/boot/dts/freescale/imx8mn-tqma8mqnl-mba8mx.dts b/arch/arm64/boot/dts/freescale/imx8mn-tqma8mqnl-mba8mx.dts
new file mode 100644
index 0000000..3f1e49b
--- /dev/null
+++ b/arch/arm64/boot/dts/freescale/imx8mn-tqma8mqnl-mba8mx.dts
@@ -0,0 +1,237 @@
+// SPDX-License-Identifier: (GPL-2.0-or-later OR MIT)
+/*
+ * Copyright 2020-2021 TQ-Systems GmbH
+ */
+
+/dts-v1/;
+
+#include "imx8mn-tqma8mqnl.dtsi"
+#include "mba8mx.dtsi"
+
+/ {
+ model = "TQ-Systems GmbH i.MX8MN TQMa8MxNL on MBa8Mx";
+ compatible = "tq,imx8mn-tqma8mqnl-mba8mx", "tq,imx8mn-tqma8mqnl", "fsl,imx8mn";
+
+ aliases {
+ eeprom0 = &eeprom3;
+ mmc0 = &usdhc3;
+ mmc1 = &usdhc2;
+ mmc2 = &usdhc1;
+ rtc0 = &pcf85063;
+ rtc1 = &snvs_rtc;
+ };
+
+ reg_usdhc2_vmmc: regulator-vmmc {
+ compatible = "regulator-fixed";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_reg_usdhc2_vmmc>;
+ regulator-name = "VSD_3V3";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ gpio = <&gpio2 19 GPIO_ACTIVE_HIGH>;
+ enable-active-high;
+ startup-delay-us = <100>;
+ off-on-delay-us = <12000>;
+ };
+};
+
+/* Located on TQMa8MxML-ADAP */
+&gpio2 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_usb0hub_sel>;
+
+ sel-usb-hub-hog {
+ gpio-hog;
+ gpios = <1 GPIO_ACTIVE_HIGH>;
+ output-high;
+ };
+};
+
+&i2c1 {
+ expander2: gpio@27 {
+ compatible = "nxp,pca9555";
+ reg = <0x27>;
+ gpio-controller;
+ #gpio-cells = <2>;
+ vcc-supply = <®_vcc_3v3>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_expander2>;
+ interrupt-parent = <&gpio1>;
+ interrupts = <9 IRQ_TYPE_EDGE_FALLING>;
+ interrupt-controller;
+ #interrupt-cells = <2>;
+ };
+};
+
+&sai3 {
+ assigned-clocks = <&clk IMX8MN_CLK_SAI3>;
+ assigned-clock-parents = <&clk IMX8MN_AUDIO_PLL1_OUT>;
+ clock-names = "bus", "mclk0", "mclk1", "mclk2", "mclk3", "pll8k", "pll11k";
+ clocks = <&clk IMX8MN_CLK_SAI3_IPG>, <&clk IMX8MN_CLK_DUMMY>,
+ <&clk IMX8MN_CLK_SAI3_ROOT>, <&clk IMX8MN_CLK_DUMMY>,
+ <&clk IMX8MN_CLK_DUMMY>, <&clk IMX8MN_AUDIO_PLL1_OUT>,
+ <&clk IMX8MN_AUDIO_PLL2_OUT>;
+};
+
+&tlv320aic3x04 {
+ clock-names = "mclk";
+ clocks = <&clk IMX8MN_CLK_SAI3_ROOT>;
+};
+
+&usbotg1 {
+ dr_mode = "host";
+ disable-over-current;
+ power-active-high;
+ status = "okay";
+};
+
+&iomuxc {
+ pinctrl_ecspi1: ecspi1grp {
+ fsl,pins = <MX8MN_IOMUXC_ECSPI1_SCLK_ECSPI1_SCLK 0x00000146>,
+ <MX8MN_IOMUXC_ECSPI1_MOSI_ECSPI1_MOSI 0x00000146>,
+ <MX8MN_IOMUXC_ECSPI1_MISO_ECSPI1_MISO 0x00000146>,
+ <MX8MN_IOMUXC_ECSPI1_SS0_GPIO5_IO9 0x00000146>;
+ };
+
+ pinctrl_ecspi2: ecspi2grp {
+ fsl,pins = <MX8MN_IOMUXC_ECSPI2_SCLK_ECSPI2_SCLK 0x00000146>,
+ <MX8MN_IOMUXC_ECSPI2_MOSI_ECSPI2_MOSI 0x00000146>,
+ <MX8MN_IOMUXC_ECSPI2_MISO_ECSPI2_MISO 0x00000146>,
+ <MX8MN_IOMUXC_ECSPI2_SS0_GPIO5_IO13 0x00000146>;
+ };
+
+ pinctrl_expander2: expander2grp {
+ fsl,pins = <MX8MN_IOMUXC_GPIO1_IO09_GPIO1_IO9 0x94>;
+ };
+
+ pinctrl_fec1: fec1grp {
+ fsl,pins = <MX8MN_IOMUXC_ENET_MDC_ENET1_MDC 0x40000002>,
+ <MX8MN_IOMUXC_ENET_MDIO_ENET1_MDIO 0x40000002>,
+ <MX8MN_IOMUXC_ENET_TD3_ENET1_RGMII_TD3 0x14>,
+ <MX8MN_IOMUXC_ENET_TD2_ENET1_RGMII_TD2 0x14>,
+ <MX8MN_IOMUXC_ENET_TD1_ENET1_RGMII_TD1 0x14>,
+ <MX8MN_IOMUXC_ENET_TD0_ENET1_RGMII_TD0 0x14>,
+ <MX8MN_IOMUXC_ENET_RD3_ENET1_RGMII_RD3 0x90>,
+ <MX8MN_IOMUXC_ENET_RD2_ENET1_RGMII_RD2 0x90>,
+ <MX8MN_IOMUXC_ENET_RD1_ENET1_RGMII_RD1 0x90>,
+ <MX8MN_IOMUXC_ENET_RD0_ENET1_RGMII_RD0 0x90>,
+ <MX8MN_IOMUXC_ENET_TXC_ENET1_RGMII_TXC 0x14>,
+ <MX8MN_IOMUXC_ENET_RXC_ENET1_RGMII_RXC 0x90>,
+ <MX8MN_IOMUXC_ENET_RX_CTL_ENET1_RGMII_RX_CTL 0x90>,
+ <MX8MN_IOMUXC_ENET_TX_CTL_ENET1_RGMII_TX_CTL 0x14>;
+ };
+
+ pinctrl_gpiobutton: gpiobuttongrp {
+ fsl,pins = <MX8MN_IOMUXC_GPIO1_IO05_GPIO1_IO5 0x84>,
+ <MX8MN_IOMUXC_GPIO1_IO07_GPIO1_IO7 0x84>,
+ <MX8MN_IOMUXC_SD1_CLK_GPIO2_IO0 0x84>;
+ };
+
+ pinctrl_gpioled: gpioledgrp {
+ fsl,pins = <MX8MN_IOMUXC_GPIO1_IO00_GPIO1_IO0 0x84>,
+ <MX8MN_IOMUXC_NAND_DQS_GPIO3_IO14 0x84>;
+ };
+
+ pinctrl_i2c2: i2c2grp {
+ fsl,pins = <MX8MN_IOMUXC_I2C2_SCL_I2C2_SCL 0x400001C4>,
+ <MX8MN_IOMUXC_I2C2_SDA_I2C2_SDA 0x400001C4>;
+ };
+
+ pinctrl_i2c2_gpio: i2c2gpiogrp {
+ fsl,pins = <MX8MN_IOMUXC_I2C2_SCL_GPIO5_IO16 0x400001C4>,
+ <MX8MN_IOMUXC_I2C2_SDA_GPIO5_IO17 0x400001C4>;
+ };
+
+ pinctrl_i2c3: i2c3grp {
+ fsl,pins = <MX8MN_IOMUXC_I2C3_SCL_I2C3_SCL 0x400001C4>,
+ <MX8MN_IOMUXC_I2C3_SDA_I2C3_SDA 0x400001C4>;
+ };
+
+ pinctrl_i2c3_gpio: i2c3gpiogrp {
+ fsl,pins = <MX8MN_IOMUXC_I2C3_SCL_GPIO5_IO18 0x400001C4>,
+ <MX8MN_IOMUXC_I2C3_SDA_GPIO5_IO19 0x400001C4>;
+ };
+
+ pinctrl_pwm3: pwm3grp {
+ fsl,pins = <MX8MN_IOMUXC_GPIO1_IO14_PWM3_OUT 0x14>;
+ };
+
+ pinctrl_pwm4: pwm4grp {
+ fsl,pins = <MX8MN_IOMUXC_GPIO1_IO15_PWM4_OUT 0x14>;
+ };
+
+ pinctrl_sai3: sai3grp {
+ fsl,pins = <MX8MN_IOMUXC_SAI3_MCLK_SAI3_MCLK 0x94>,
+ <MX8MN_IOMUXC_SAI3_RXC_SAI3_RX_BCLK 0x94>,
+ <MX8MN_IOMUXC_SAI3_RXFS_SAI3_RX_SYNC 0x94>,
+ <MX8MN_IOMUXC_SAI3_RXD_SAI3_RX_DATA0 0x94>,
+ <MX8MN_IOMUXC_SAI3_TXFS_SAI3_TX_SYNC 0x94>,
+ <MX8MN_IOMUXC_SAI3_TXD_SAI3_TX_DATA0 0x94>,
+ <MX8MN_IOMUXC_SAI3_TXC_SAI3_TX_BCLK 0x94>;
+ };
+
+ pinctrl_uart1: uart1grp {
+ fsl,pins = <MX8MN_IOMUXC_UART1_RXD_UART1_DCE_RX 0x16>,
+ <MX8MN_IOMUXC_UART1_TXD_UART1_DCE_TX 0x16>;
+ };
+
+ pinctrl_uart2: uart2grp {
+ fsl,pins = <MX8MN_IOMUXC_UART2_RXD_UART2_DCE_RX 0x16>,
+ <MX8MN_IOMUXC_UART2_TXD_UART2_DCE_TX 0x16>;
+ };
+
+ pinctrl_uart3: uart3grp {
+ fsl,pins = <MX8MN_IOMUXC_UART3_RXD_UART3_DCE_RX 0x16>,
+ <MX8MN_IOMUXC_UART3_TXD_UART3_DCE_TX 0x16>;
+ };
+
+ pinctrl_uart4: uart4grp {
+ fsl,pins = <MX8MN_IOMUXC_UART4_RXD_UART4_DCE_RX 0x16>,
+ <MX8MN_IOMUXC_UART4_TXD_UART4_DCE_TX 0x16>;
+ };
+
+ pinctrl_usb0hub_sel: usb0hub-selgrp {
+ /* SEL_USB_HUB_B */
+ fsl,pins = <MX8MN_IOMUXC_SD1_CMD_GPIO2_IO1 0x84>;
+ };
+
+ pinctrl_usbotg: usbotggrp {
+ fsl,pins = <MX8MN_IOMUXC_GPIO1_IO12_USB1_OTG_PWR 0x84>,
+ <MX8MN_IOMUXC_GPIO1_IO13_USB1_OTG_OC 0x84>,
+ <MX8MN_IOMUXC_GPIO1_IO10_USB1_OTG_ID 0x1C4>;
+ };
+
+ pinctrl_usdhc2: usdhc2grp {
+ fsl,pins = <MX8MN_IOMUXC_SD2_CLK_USDHC2_CLK 0x1d4>,
+ <MX8MN_IOMUXC_SD2_CMD_USDHC2_CMD 0x1d4>,
+ <MX8MN_IOMUXC_SD2_DATA0_USDHC2_DATA0 0x1d4>,
+ <MX8MN_IOMUXC_SD2_DATA1_USDHC2_DATA1 0x1d4>,
+ <MX8MN_IOMUXC_SD2_DATA2_USDHC2_DATA2 0x1d4>,
+ <MX8MN_IOMUXC_SD2_DATA3_USDHC2_DATA3 0x1d4>,
+ <MX8MN_IOMUXC_GPIO1_IO04_USDHC2_VSELECT 0x84>;
+ };
+
+ pinctrl_usdhc2_100mhz: usdhc2-100mhzgrp {
+ fsl,pins = <MX8MN_IOMUXC_SD2_CLK_USDHC2_CLK 0x1d4>,
+ <MX8MN_IOMUXC_SD2_CMD_USDHC2_CMD 0x1d4>,
+ <MX8MN_IOMUXC_SD2_DATA0_USDHC2_DATA0 0x1d4>,
+ <MX8MN_IOMUXC_SD2_DATA1_USDHC2_DATA1 0x1d4>,
+ <MX8MN_IOMUXC_SD2_DATA2_USDHC2_DATA2 0x1d4>,
+ <MX8MN_IOMUXC_SD2_DATA3_USDHC2_DATA3 0x1d4>,
+ <MX8MN_IOMUXC_GPIO1_IO04_USDHC2_VSELECT 0x84>;
+ };
+
+ pinctrl_usdhc2_200mhz: usdhc2-200mhzgrp {
+ fsl,pins = <MX8MN_IOMUXC_SD2_CLK_USDHC2_CLK 0x1d4>,
+ <MX8MN_IOMUXC_SD2_CMD_USDHC2_CMD 0x1d4>,
+ <MX8MN_IOMUXC_SD2_DATA0_USDHC2_DATA0 0x1d4>,
+ <MX8MN_IOMUXC_SD2_DATA1_USDHC2_DATA1 0x1d4>,
+ <MX8MN_IOMUXC_SD2_DATA2_USDHC2_DATA2 0x1d4>,
+ <MX8MN_IOMUXC_SD2_DATA3_USDHC2_DATA3 0x1d4>,
+ <MX8MN_IOMUXC_GPIO1_IO04_USDHC2_VSELECT 0x84>;
+ };
+
+ pinctrl_usdhc2_gpio: usdhc2-gpiogrp {
+ fsl,pins = <MX8MN_IOMUXC_SD2_CD_B_GPIO2_IO12 0x84>;
+ };
+};
diff --git a/arch/arm64/boot/dts/freescale/imx8mn-tqma8mqnl.dtsi b/arch/arm64/boot/dts/freescale/imx8mn-tqma8mqnl.dtsi
new file mode 100644
index 0000000..9ea2894
--- /dev/null
+++ b/arch/arm64/boot/dts/freescale/imx8mn-tqma8mqnl.dtsi
@@ -0,0 +1,322 @@
+// SPDX-License-Identifier: (GPL-2.0-or-later OR MIT)
+/*
+ * Copyright 2020-2021 TQ-Systems GmbH
+ */
+
+#include "imx8mn.dtsi"
+
+/ {
+ model = "TQ-Systems i.MX8MN TQMa8MxNL";
+ compatible = "tq,imx8mn-tqma8mqnl", "fsl,imx8mn";
+
+ memory@40000000 {
+ device_type = "memory";
+ /* our minimum RAM config will be 1024 MiB */
+ reg = <0x00000000 0x40000000 0 0x40000000>;
+ };
+
+ /* e-MMC IO, needed for HS modes */
+ reg_vcc1v8: regulator-vcc1v8 {
+ compatible = "regulator-fixed";
+ regulator-name = "TQMA8MXNL_VCC1V8";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ };
+
+ reg_vcc3v3: regulator-vcc3v3 {
+ compatible = "regulator-fixed";
+ regulator-name = "TQMA8MXNL_VCC3V3";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ };
+
+ reserved-memory {
+ #address-cells = <2>;
+ #size-cells = <2>;
+ ranges;
+
+ /* global autoconfigured region for contiguous allocations */
+ linux,cma {
+ compatible = "shared-dma-pool";
+ reusable;
+ /* 640 MiB */
+ size = <0 0x28000000>;
+ /* 1024 - 128 MiB, our minimum RAM config will be 1024 MiB */
+ alloc-ranges = <0 0x40000000 0 0x78000000>;
+ linux,cma-default;
+ };
+ };
+};
+
+&A53_0 {
+ cpu-supply = <&buck2_reg>;
+};
+
+&flexspi {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_flexspi>;
+ status = "okay";
+
+ flash0: flash@0 {
+ compatible = "jedec,spi-nor";
+ reg = <0>;
+ #address-cells = <1>;
+ #size-cells = <1>;
+ spi-max-frequency = <84000000>;
+ spi-tx-bus-width = <1>;
+ spi-rx-bus-width = <4>;
+ };
+};
+
+&i2c1 {
+ clock-frequency = <100000>;
+ pinctrl-names = "default", "gpio";
+ pinctrl-0 = <&pinctrl_i2c1>;
+ pinctrl-1 = <&pinctrl_i2c1_gpio>;
+ scl-gpios = <&gpio5 14 (GPIO_ACTIVE_HIGH | GPIO_OPEN_DRAIN)>;
+ sda-gpios = <&gpio5 15 (GPIO_ACTIVE_HIGH | GPIO_OPEN_DRAIN)>;
+ status = "okay";
+
+ sensor0: temperature-sensor-eeprom@1b {
+ compatible = "nxp,se97", "jedec,jc-42.4-temp";
+ reg = <0x1b>;
+ };
+
+ pca9450: pmic@25 {
+ compatible = "nxp,pca9450a";
+ reg = <0x25>;
+
+ /* PMIC PCA9450 PMIC_nINT GPIO1_IO08 */
+ pinctrl-0 = <&pinctrl_pmic>;
+ pinctrl-names = "default";
+ interrupt-parent = <&gpio1>;
+ interrupts = <8 IRQ_TYPE_LEVEL_LOW>;
+
+ regulators {
+ /* V_0V85_SOC: 0.85 .. 0.95 */
+ buck1_reg: BUCK1 {
+ regulator-name = "BUCK1";
+ regulator-min-microvolt = <850000>;
+ regulator-max-microvolt = <950000>;
+ regulator-boot-on;
+ regulator-always-on;
+ regulator-ramp-delay = <3125>;
+ };
+
+ /* VDD_ARM */
+ buck2_reg: BUCK2 {
+ regulator-name = "BUCK2";
+ regulator-min-microvolt = <850000>;
+ regulator-max-microvolt = <1000000>;
+ regulator-boot-on;
+ regulator-always-on;
+ nxp,dvs-run-voltage = <950000>;
+ nxp,dvs-standby-voltage = <850000>;
+ regulator-ramp-delay = <3125>;
+ };
+
+ /* V_0V85_GPU / DRAM: shall be equal to BUCK1 for i.MX8MN */
+ buck3_reg: BUCK3 {
+ regulator-name = "BUCK3";
+ regulator-min-microvolt = <850000>;
+ regulator-max-microvolt = <950000>;
+ regulator-boot-on;
+ regulator-always-on;
+ regulator-ramp-delay = <3125>;
+ };
+
+ /* VCC3V3 -> VMMC, ... must not be changed */
+ buck4_reg: BUCK4 {
+ regulator-name = "BUCK4";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-boot-on;
+ regulator-always-on;
+ };
+
+ /* V_1V8 -> VQMMC, SPI-NOR, ... must not be changed */
+ buck5_reg: BUCK5 {
+ regulator-name = "BUCK5";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ regulator-boot-on;
+ regulator-always-on;
+ };
+
+ /* V_1V1 -> RAM, ... must not be changed */
+ buck6_reg: BUCK6 {
+ regulator-name = "BUCK6";
+ regulator-min-microvolt = <1100000>;
+ regulator-max-microvolt = <1100000>;
+ regulator-boot-on;
+ regulator-always-on;
+ };
+
+ /* V_1V8_SNVS */
+ ldo1_reg: LDO1 {
+ regulator-name = "LDO1";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ regulator-boot-on;
+ regulator-always-on;
+ };
+
+ /* V_0V8_SNVS */
+ ldo2_reg: LDO2 {
+ regulator-name = "LDO2";
+ regulator-min-microvolt = <800000>;
+ regulator-max-microvolt = <850000>;
+ regulator-boot-on;
+ regulator-always-on;
+ };
+
+ /* V_1V8_ANA */
+ ldo3_reg: LDO3 {
+ regulator-name = "LDO3";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ regulator-boot-on;
+ regulator-always-on;
+ };
+
+ /* V_0V9_MIPI */
+ ldo4_reg: LDO4 {
+ regulator-name = "LDO4";
+ regulator-min-microvolt = <900000>;
+ regulator-max-microvolt = <900000>;
+ regulator-boot-on;
+ regulator-always-on;
+ };
+
+ /* VCC SD IO - switched using SD2 VSELECT */
+ ldo5_reg: LDO5 {
+ regulator-name = "LDO5";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <3300000>;
+ };
+ };
+ };
+
+ pcf85063: rtc@51 {
+ compatible = "nxp,pcf85063a";
+ reg = <0x51>;
+ quartz-load-femtofarads = <7000>;
+ };
+
+ eeprom1: eeprom@53 {
+ compatible = "nxp,se97b", "atmel,24c02";
+ read-only;
+ reg = <0x53>;
+ pagesize = <16>;
+ };
+
+ eeprom0: eeprom@57 {
+ compatible = "atmel,24c64";
+ reg = <0x57>;
+ pagesize = <32>;
+ };
+};
+
+&usdhc3 {
+ pinctrl-names = "default", "state_100mhz", "state_200mhz";
+ pinctrl-0 = <&pinctrl_usdhc3>;
+ pinctrl-1 = <&pinctrl_usdhc3_100mhz>;
+ pinctrl-2 = <&pinctrl_usdhc3_200mhz>;
+ bus-width = <8>;
+ non-removable;
+ no-sd;
+ no-sdio;
+ vmmc-supply = <®_vcc3v3>;
+ vqmmc-supply = <®_vcc1v8>;
+ status = "okay";
+};
+
+/*
+ * Attention:
+ * wdog reset is routed to PMIC, PMIC must be preconfigured to force POR
+ * without LDO for SNVS. GPIO1_IO02 must not be used as GPIO.
+ */
+&wdog1 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_wdog>;
+ fsl,ext-reset-output;
+ status = "okay";
+};
+
+&iomuxc {
+ pinctrl_flexspi: flexspigrp {
+ fsl,pins = <MX8MN_IOMUXC_NAND_ALE_QSPI_A_SCLK 0x84>,
+ <MX8MN_IOMUXC_NAND_CE0_B_QSPI_A_SS0_B 0x84>,
+ <MX8MN_IOMUXC_NAND_DATA00_QSPI_A_DATA0 0x84>,
+ <MX8MN_IOMUXC_NAND_DATA01_QSPI_A_DATA1 0x84>,
+ <MX8MN_IOMUXC_NAND_DATA02_QSPI_A_DATA2 0x84>,
+ <MX8MN_IOMUXC_NAND_DATA03_QSPI_A_DATA3 0x84>;
+ };
+
+ pinctrl_i2c1: i2c1grp {
+ fsl,pins = <MX8MN_IOMUXC_I2C1_SCL_I2C1_SCL 0x400001c4>,
+ <MX8MN_IOMUXC_I2C1_SDA_I2C1_SDA 0x400001c4>;
+ };
+
+ pinctrl_i2c1_gpio: i2c1gpiogrp {
+ fsl,pins = <MX8MN_IOMUXC_I2C1_SCL_GPIO5_IO14 0x400001c4>,
+ <MX8MN_IOMUXC_I2C1_SDA_GPIO5_IO15 0x400001c4>;
+ };
+
+ pinctrl_pmic: pmicgrp {
+ fsl,pins = <MX8MN_IOMUXC_GPIO1_IO08_GPIO1_IO8 0x84>;
+ };
+
+ pinctrl_reg_usdhc2_vmmc: regusdhc2vmmcgrp {
+ fsl,pins = <MX8MN_IOMUXC_SD2_RESET_B_GPIO2_IO19 0x84>;
+ };
+
+ pinctrl_usdhc3: usdhc3grp {
+ fsl,pins = <MX8MN_IOMUXC_NAND_WE_B_USDHC3_CLK 0x1d4>,
+ <MX8MN_IOMUXC_NAND_WP_B_USDHC3_CMD 0x1d2>,
+ <MX8MN_IOMUXC_NAND_DATA04_USDHC3_DATA0 0x1d4>,
+ <MX8MN_IOMUXC_NAND_DATA05_USDHC3_DATA1 0x1d4>,
+ <MX8MN_IOMUXC_NAND_DATA06_USDHC3_DATA2 0x1d4>,
+ <MX8MN_IOMUXC_NAND_DATA07_USDHC3_DATA3 0x1d4>,
+ <MX8MN_IOMUXC_NAND_RE_B_USDHC3_DATA4 0x1d4>,
+ <MX8MN_IOMUXC_NAND_CE2_B_USDHC3_DATA5 0x1d4>,
+ <MX8MN_IOMUXC_NAND_CE3_B_USDHC3_DATA6 0x1d4>,
+ <MX8MN_IOMUXC_NAND_CLE_USDHC3_DATA7 0x1d4>,
+ <MX8MN_IOMUXC_NAND_CE1_B_USDHC3_STROBE 0x84>,
+ <MX8MN_IOMUXC_NAND_READY_B_USDHC3_RESET_B 0x84>;
+ };
+
+ pinctrl_usdhc3_100mhz: usdhc3-100mhzgrp {
+ fsl,pins = <MX8MN_IOMUXC_NAND_WE_B_USDHC3_CLK 0x1d2>,
+ <MX8MN_IOMUXC_NAND_WP_B_USDHC3_CMD 0x1d2>,
+ <MX8MN_IOMUXC_NAND_DATA04_USDHC3_DATA0 0x1d4>,
+ <MX8MN_IOMUXC_NAND_DATA05_USDHC3_DATA1 0x1d4>,
+ <MX8MN_IOMUXC_NAND_DATA06_USDHC3_DATA2 0x1d4>,
+ <MX8MN_IOMUXC_NAND_DATA07_USDHC3_DATA3 0x1d4>,
+ <MX8MN_IOMUXC_NAND_RE_B_USDHC3_DATA4 0x1d4>,
+ <MX8MN_IOMUXC_NAND_CE2_B_USDHC3_DATA5 0x1d4>,
+ <MX8MN_IOMUXC_NAND_CE3_B_USDHC3_DATA6 0x1d4>,
+ <MX8MN_IOMUXC_NAND_CLE_USDHC3_DATA7 0x1d4>,
+ <MX8MN_IOMUXC_NAND_CE1_B_USDHC3_STROBE 0x84>,
+ <MX8MN_IOMUXC_NAND_READY_B_USDHC3_RESET_B 0x84>;
+ };
+
+ pinctrl_usdhc3_200mhz: usdhc3-200mhzgrp {
+ fsl,pins = <MX8MN_IOMUXC_NAND_WE_B_USDHC3_CLK 0x1d6>,
+ <MX8MN_IOMUXC_NAND_WP_B_USDHC3_CMD 0x1d2>,
+ <MX8MN_IOMUXC_NAND_DATA04_USDHC3_DATA0 0x1d4>,
+ <MX8MN_IOMUXC_NAND_DATA05_USDHC3_DATA1 0x1d4>,
+ <MX8MN_IOMUXC_NAND_DATA06_USDHC3_DATA2 0x1d4>,
+ <MX8MN_IOMUXC_NAND_DATA07_USDHC3_DATA3 0x1d4>,
+ <MX8MN_IOMUXC_NAND_RE_B_USDHC3_DATA4 0x1d4>,
+ <MX8MN_IOMUXC_NAND_CE2_B_USDHC3_DATA5 0x1d4>,
+ <MX8MN_IOMUXC_NAND_CE3_B_USDHC3_DATA6 0x1d4>,
+ <MX8MN_IOMUXC_NAND_CLE_USDHC3_DATA7 0x1d4>,
+ <MX8MN_IOMUXC_NAND_CE1_B_USDHC3_STROBE 0x84>,
+ <MX8MN_IOMUXC_NAND_READY_B_USDHC3_RESET_B 0x84>;
+ };
+
+ pinctrl_wdog: wdoggrp {
+ fsl,pins = <MX8MN_IOMUXC_GPIO1_IO02_WDOG1_WDOG_B 0x84>;
+ };
+};
diff --git a/arch/arm64/boot/dts/freescale/imx8mn.dtsi b/arch/arm64/boot/dts/freescale/imx8mn.dtsi
index da6c942..b8d49d5 100644
--- a/arch/arm64/boot/dts/freescale/imx8mn.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8mn.dtsi
@@ -63,6 +63,12 @@ A53_0: cpu@0 {
clock-latency = <61036>;
clocks = <&clk IMX8MN_CLK_ARM>;
enable-method = "psci";
+ i-cache-size = <0x8000>;
+ i-cache-line-size = <64>;
+ i-cache-sets = <256>;
+ d-cache-size = <0x8000>;
+ d-cache-line-size = <64>;
+ d-cache-sets = <128>;
next-level-cache = <&A53_L2>;
operating-points-v2 = <&a53_opp_table>;
nvmem-cells = <&cpu_speed_grade>;
@@ -78,6 +84,12 @@ A53_1: cpu@1 {
clock-latency = <61036>;
clocks = <&clk IMX8MN_CLK_ARM>;
enable-method = "psci";
+ i-cache-size = <0x8000>;
+ i-cache-line-size = <64>;
+ i-cache-sets = <256>;
+ d-cache-size = <0x8000>;
+ d-cache-line-size = <64>;
+ d-cache-sets = <128>;
next-level-cache = <&A53_L2>;
operating-points-v2 = <&a53_opp_table>;
cpu-idle-states = <&cpu_pd_wait>;
@@ -91,6 +103,12 @@ A53_2: cpu@2 {
clock-latency = <61036>;
clocks = <&clk IMX8MN_CLK_ARM>;
enable-method = "psci";
+ i-cache-size = <0x8000>;
+ i-cache-line-size = <64>;
+ i-cache-sets = <256>;
+ d-cache-size = <0x8000>;
+ d-cache-line-size = <64>;
+ d-cache-sets = <128>;
next-level-cache = <&A53_L2>;
operating-points-v2 = <&a53_opp_table>;
cpu-idle-states = <&cpu_pd_wait>;
@@ -104,6 +122,12 @@ A53_3: cpu@3 {
clock-latency = <61036>;
clocks = <&clk IMX8MN_CLK_ARM>;
enable-method = "psci";
+ i-cache-size = <0x8000>;
+ i-cache-line-size = <64>;
+ i-cache-sets = <256>;
+ d-cache-size = <0x8000>;
+ d-cache-line-size = <64>;
+ d-cache-sets = <128>;
next-level-cache = <&A53_L2>;
operating-points-v2 = <&a53_opp_table>;
cpu-idle-states = <&cpu_pd_wait>;
@@ -112,6 +136,10 @@ A53_3: cpu@3 {
A53_L2: l2-cache0 {
compatible = "cache";
+ cache-level = <2>;
+ cache-size = <0x80000>;
+ cache-line-size = <64>;
+ cache-sets = <512>;
};
};
@@ -948,7 +976,6 @@ fec1: ethernet@30be0000 {
fsl,num-rx-queues = <3>;
nvmem-cells = <&fec_mac_address>;
nvmem-cell-names = "mac-address";
- nvmem_macaddr_swap;
fsl,stop-mode = <&gpr 0x10 3>;
status = "disabled";
};
diff --git a/arch/arm64/boot/dts/freescale/imx8mp-evk.dts b/arch/arm64/boot/dts/freescale/imx8mp-evk.dts
index 7b99fad..2eb9432 100644
--- a/arch/arm64/boot/dts/freescale/imx8mp-evk.dts
+++ b/arch/arm64/boot/dts/freescale/imx8mp-evk.dts
@@ -86,6 +86,9 @@ &eqos {
pinctrl-0 = <&pinctrl_eqos>;
phy-mode = "rgmii-id";
phy-handle = <ðphy0>;
+ snps,force_thresh_dma_mode;
+ snps,mtl-tx-config = <&mtl_tx_setup>;
+ snps,mtl-rx-config = <&mtl_rx_setup>;
status = "okay";
mdio {
@@ -97,6 +100,75 @@ ethphy0: ethernet-phy@1 {
compatible = "ethernet-phy-ieee802.3-c22";
reg = <1>;
eee-broken-1000t;
+ reset-gpios = <&gpio4 22 GPIO_ACTIVE_LOW>;
+ reset-assert-us = <10000>;
+ reset-deassert-us = <80000>;
+ realtek,clkout-disable;
+ };
+ };
+
+ mtl_tx_setup: tx-queues-config {
+ snps,tx-queues-to-use = <5>;
+ snps,tx-sched-sp;
+
+ queue0 {
+ snps,dcb-algorithm;
+ snps,priority = <0x1>;
+ };
+
+ queue1 {
+ snps,dcb-algorithm;
+ snps,priority = <0x2>;
+ };
+
+ queue2 {
+ snps,dcb-algorithm;
+ snps,priority = <0x4>;
+ };
+
+ queue3 {
+ snps,dcb-algorithm;
+ snps,priority = <0x8>;
+ };
+
+ queue4 {
+ snps,dcb-algorithm;
+ snps,priority = <0xf0>;
+ };
+ };
+
+ mtl_rx_setup: rx-queues-config {
+ snps,rx-queues-to-use = <5>;
+ snps,rx-sched-sp;
+
+ queue0 {
+ snps,dcb-algorithm;
+ snps,priority = <0x1>;
+ snps,map-to-dma-channel = <0>;
+ };
+
+ queue1 {
+ snps,dcb-algorithm;
+ snps,priority = <0x2>;
+ snps,map-to-dma-channel = <1>;
+ };
+
+ queue2 {
+ snps,dcb-algorithm;
+ snps,priority = <0x4>;
+ snps,map-to-dma-channel = <2>;
+ };
+
+ queue3 {
+ snps,dcb-algorithm;
+ snps,priority = <0x8>;
+ snps,map-to-dma-channel = <3>;
+ };
+
+ queue4 {
+ snps,dcb-algorithm;
+ snps,priority = <0xf0>;
+ snps,map-to-dma-channel = <4>;
};
};
};
@@ -120,6 +192,7 @@ ethphy1: ethernet-phy@1 {
reset-gpios = <&gpio4 2 GPIO_ACTIVE_LOW>;
reset-assert-us = <10000>;
reset-deassert-us = <80000>;
+ realtek,clkout-disable;
};
};
};
diff --git a/arch/arm64/boot/dts/freescale/imx8mp.dtsi b/arch/arm64/boot/dts/freescale/imx8mp.dtsi
index 04d259d..6b840c0 100644
--- a/arch/arm64/boot/dts/freescale/imx8mp.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8mp.dtsi
@@ -51,6 +51,12 @@ A53_0: cpu@0 {
clock-latency = <61036>;
clocks = <&clk IMX8MP_CLK_ARM>;
enable-method = "psci";
+ i-cache-size = <0x8000>;
+ i-cache-line-size = <64>;
+ i-cache-sets = <256>;
+ d-cache-size = <0x8000>;
+ d-cache-line-size = <64>;
+ d-cache-sets = <128>;
next-level-cache = <&A53_L2>;
#cooling-cells = <2>;
};
@@ -62,6 +68,12 @@ A53_1: cpu@1 {
clock-latency = <61036>;
clocks = <&clk IMX8MP_CLK_ARM>;
enable-method = "psci";
+ i-cache-size = <0x8000>;
+ i-cache-line-size = <64>;
+ i-cache-sets = <256>;
+ d-cache-size = <0x8000>;
+ d-cache-line-size = <64>;
+ d-cache-sets = <128>;
next-level-cache = <&A53_L2>;
#cooling-cells = <2>;
};
@@ -73,6 +85,12 @@ A53_2: cpu@2 {
clock-latency = <61036>;
clocks = <&clk IMX8MP_CLK_ARM>;
enable-method = "psci";
+ i-cache-size = <0x8000>;
+ i-cache-line-size = <64>;
+ i-cache-sets = <256>;
+ d-cache-size = <0x8000>;
+ d-cache-line-size = <64>;
+ d-cache-sets = <128>;
next-level-cache = <&A53_L2>;
#cooling-cells = <2>;
};
@@ -84,12 +102,22 @@ A53_3: cpu@3 {
clock-latency = <61036>;
clocks = <&clk IMX8MP_CLK_ARM>;
enable-method = "psci";
+ i-cache-size = <0x8000>;
+ i-cache-line-size = <64>;
+ i-cache-sets = <256>;
+ d-cache-size = <0x8000>;
+ d-cache-line-size = <64>;
+ d-cache-sets = <128>;
next-level-cache = <&A53_L2>;
#cooling-cells = <2>;
};
A53_L2: l2-cache0 {
compatible = "cache";
+ cache-level = <2>;
+ cache-size = <0x80000>;
+ cache-line-size = <64>;
+ cache-sets = <512>;
};
};
@@ -369,6 +397,10 @@ cpu_speed_grade: speed-grade@10 {
eth_mac1: mac-address@90 {
reg = <0x90 6>;
};
+
+ eth_mac2: mac-address@96 {
+ reg = <0x96 6>;
+ };
};
anatop: anatop@30360000 {
@@ -832,7 +864,6 @@ fec: ethernet@30be0000 {
nvmem-cells = <ð_mac1>;
nvmem-cell-names = "mac-address";
fsl,stop-mode = <&gpr 0x10 3>;
- nvmem_macaddr_swap;
status = "disabled";
};
@@ -854,6 +885,8 @@ eqos: ethernet@30bf0000 {
<&clk IMX8MP_SYS_PLL2_100M>,
<&clk IMX8MP_SYS_PLL2_125M>;
assigned-clock-rates = <0>, <100000000>, <125000000>;
+ nvmem-cells = <ð_mac2>;
+ nvmem-cell-names = "mac-address";
intf_mode = <&gpr 0x4>;
status = "disabled";
};
diff --git a/arch/arm64/boot/dts/freescale/imx8mq-evk.dts b/arch/arm64/boot/dts/freescale/imx8mq-evk.dts
index b83df77..a1b7582 100644
--- a/arch/arm64/boot/dts/freescale/imx8mq-evk.dts
+++ b/arch/arm64/boot/dts/freescale/imx8mq-evk.dts
@@ -169,6 +169,11 @@ ethphy0: ethernet-phy@0 {
reg = <0>;
reset-gpios = <&gpio1 9 GPIO_ACTIVE_LOW>;
reset-assert-us = <10000>;
+ qca,disable-smarteee;
+ vddio-supply = <&vddh>;
+
+ vddh: vddh-regulator {
+ };
};
};
};
@@ -326,6 +331,10 @@ &pgc_gpu {
power-supply = <&sw1a_reg>;
};
+&pgc_vpu {
+ power-supply = <&sw1c_reg>;
+};
+
&qspi0 {
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_qspi>;
diff --git a/arch/arm64/boot/dts/freescale/imx8mq-librem5-r3.dts b/arch/arm64/boot/dts/freescale/imx8mq-librem5-r3.dts
index cd3c3ed..4533a84 100644
--- a/arch/arm64/boot/dts/freescale/imx8mq-librem5-r3.dts
+++ b/arch/arm64/boot/dts/freescale/imx8mq-librem5-r3.dts
@@ -1,14 +1,9 @@
// SPDX-License-Identifier: (GPL-2.0+ OR MIT)
-// Copyright (C) 2020 Purism SPC <kernel@puri.sm>
+// Copyright (C) 2021 Purism SPC <kernel@puri.sm>
/dts-v1/;
-#include "imx8mq-librem5.dtsi"
-
-/ {
- model = "Purism Librem 5r3";
- compatible = "purism,librem5r3", "purism,librem5", "fsl,imx8mq";
-};
+#include "imx8mq-librem5-r3.dtsi"
&a53_opp_table {
opp-1000000000 {
@@ -16,22 +11,6 @@ opp-1000000000 {
};
};
-&accel_gyro {
- mount-matrix = "1", "0", "0",
- "0", "1", "0",
- "0", "0", "-1";
-};
-
-&bq25895 {
- ti,battery-regulation-voltage = <4200000>; /* uV */
- ti,charge-current = <1500000>; /* uA */
- ti,termination-current = <144000>; /* uA */
-};
-
&buck3_reg {
regulator-always-on;
};
-
-&proximity {
- proximity-near-level = <25>;
-};
diff --git a/arch/arm64/boot/dts/freescale/imx8mq-librem5-r3.dtsi b/arch/arm64/boot/dts/freescale/imx8mq-librem5-r3.dtsi
new file mode 100644
index 0000000..e4f8b47
--- /dev/null
+++ b/arch/arm64/boot/dts/freescale/imx8mq-librem5-r3.dtsi
@@ -0,0 +1,45 @@
+// SPDX-License-Identifier: (GPL-2.0+ OR MIT)
+// Copyright (C) 2021 Purism SPC <kernel@puri.sm>
+
+/dts-v1/;
+
+/*
+ * This file describes hardware that is shared among r3 ("Dogwood") and
+ * later revisions of the Librem 5 so it has to be included in dts there.
+ */
+
+#include "imx8mq-librem5.dtsi"
+
+/ {
+ model = "Purism Librem 5r3";
+ compatible = "purism,librem5r3", "purism,librem5", "fsl,imx8mq";
+};
+
+&accel_gyro {
+ mount-matrix = "1", "0", "0",
+ "0", "1", "0",
+ "0", "0", "-1";
+};
+
+&bq25895 {
+ ti,battery-regulation-voltage = <4200000>; /* uV */
+ ti,charge-current = <1500000>; /* uA */
+ ti,termination-current = <144000>; /* uA */
+};
+
+&camera_front {
+ pinctrl-0 = <&pinctrl_csi1>, <&pinctrl_r3_camera_pwr>;
+ shutdown-gpios = <&gpio5 4 GPIO_ACTIVE_LOW>;
+};
+
+&iomuxc {
+ pinctrl_r3_camera_pwr: r3camerapwrgrp {
+ fsl,pins = <
+ MX8MQ_IOMUXC_SPDIF_RX_GPIO5_IO4 0x83
+ >;
+ };
+};
+
+&proximity {
+ proximity-near-level = <25>;
+};
diff --git a/arch/arm64/boot/dts/freescale/imx8mq-librem5-r4.dts b/arch/arm64/boot/dts/freescale/imx8mq-librem5-r4.dts
index cbfb49a..30d65be 100644
--- a/arch/arm64/boot/dts/freescale/imx8mq-librem5-r4.dts
+++ b/arch/arm64/boot/dts/freescale/imx8mq-librem5-r4.dts
@@ -1,31 +1,19 @@
// SPDX-License-Identifier: (GPL-2.0+ OR MIT)
-// Copyright (C) 2020 Purism SPC <kernel@puri.sm>
+// Copyright (C) 2021 Purism SPC <kernel@puri.sm>
/dts-v1/;
-#include "imx8mq-librem5.dtsi"
+#include "imx8mq-librem5-r3.dtsi"
/ {
model = "Purism Librem 5r4";
compatible = "purism,librem5r4", "purism,librem5", "fsl,imx8mq";
};
-&accel_gyro {
- mount-matrix = "1", "0", "0",
- "0", "1", "0",
- "0", "0", "-1";
-};
-
&bat {
maxim,rsns-microohm = <1667>;
};
-&bq25895 {
- ti,battery-regulation-voltage = <4200000>; /* uV */
- ti,charge-current = <1500000>; /* uA */
- ti,termination-current = <144000>; /* uA */
-};
-
&led_backlight {
led-max-microamp = <25000>;
};
diff --git a/arch/arm64/boot/dts/freescale/imx8mq-librem5.dtsi b/arch/arm64/boot/dts/freescale/imx8mq-librem5.dtsi
index 60d47c7..f3e3418 100644
--- a/arch/arm64/boot/dts/freescale/imx8mq-librem5.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8mq-librem5.dtsi
@@ -14,6 +14,7 @@
/ {
model = "Purism Librem 5";
compatible = "purism,librem5", "fsl,imx8mq";
+ chassis-type = "handset";
backlight_dsi: backlight-dsi {
compatible = "led-backlight";
@@ -62,6 +63,40 @@ reg_aud_1v8: regulator-audio-1v8 {
enable-active-high;
};
+ /*
+ * the pinctrl for reg_csi_1v8 and reg_vcam_1v8 is added to the PMIC
+ * since we can't have it twice in the 2 different regulator nodes.
+ */
+ reg_csi_1v8: regulator-csi-1v8 {
+ compatible = "regulator-fixed";
+ regulator-name = "CAMERA_VDDIO_1V8";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ vin-supply = <®_vdd_3v3>;
+ gpio = <&gpio1 0 GPIO_ACTIVE_HIGH>;
+ enable-active-high;
+ };
+
+ /* controlled by the CAMERA_POWER_KEY HKS */
+ reg_vcam_1v2: regulator-vcam-1v2 {
+ compatible = "regulator-fixed";
+ regulator-name = "CAMERA_VDDD_1V2";
+ regulator-min-microvolt = <1200000>;
+ regulator-max-microvolt = <1200000>;
+ vin-supply = <®_vdd_1v8>;
+ enable-active-high;
+ };
+
+ reg_vcam_2v8: regulator-vcam-2v8 {
+ compatible = "regulator-fixed";
+ regulator-name = "CAMERA_VDDA_2V8";
+ regulator-min-microvolt = <2800000>;
+ regulator-max-microvolt = <2800000>;
+ vin-supply = <®_vdd_3v3>;
+ gpio = <&gpio1 0 GPIO_ACTIVE_HIGH>;
+ enable-active-high;
+ };
+
reg_gnss: regulator-gnss {
compatible = "regulator-fixed";
pinctrl-names = "default";
@@ -237,6 +272,10 @@ &A53_3 {
cpu-supply = <&buck2_reg>;
};
+&csi1 {
+ status = "okay";
+};
+
&ddrc {
operating-points-v2 = <&ddrc_opp_table>;
@@ -329,6 +368,20 @@ MX8MQ_IOMUXC_SAI5_MCLK_GPIO3_IO25 0x83
>;
};
+ pinctrl_camera_pwr: camerapwrgrp {
+ fsl,pins = <
+ /* CAMERA_PWR_EN_3V3 */
+ MX8MQ_IOMUXC_GPIO1_IO00_GPIO1_IO0 0x83
+ >;
+ };
+
+ pinctrl_csi1: csi1grp {
+ fsl,pins = <
+ /* CSI1_NRST */
+ MX8MQ_IOMUXC_ENET_RXC_GPIO1_IO25 0x83
+ >;
+ };
+
pinctrl_charger_in: chargeringrp {
fsl,pins = <
/* CHRG_INT */
@@ -725,7 +778,7 @@ pmic: pmic@4b {
compatible = "rohm,bd71837";
reg = <0x4b>;
pinctrl-names = "default";
- pinctrl-0 = <&pinctrl_pmic>;
+ pinctrl-0 = <&pinctrl_pmic>, <&pinctrl_camera_pwr>;
clocks = <&pmic_osc>;
clock-names = "osc";
clock-output-names = "pmic_clk";
@@ -958,6 +1011,31 @@ codec: audio-codec@1a {
>;
};
+ camera_front: camera@20 {
+ compatible = "hynix,hi846";
+ reg = <0x20>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_csi1>;
+ clocks = <&clk IMX8MQ_CLK_CLKO2>;
+ assigned-clocks = <&clk IMX8MQ_CLK_CLKO2>;
+ assigned-clock-rates = <25000000>;
+ reset-gpios = <&gpio1 25 GPIO_ACTIVE_LOW>;
+ vdda-supply = <®_vcam_2v8>;
+ vddd-supply = <®_vcam_1v2>;
+ vddio-supply = <®_csi_1v8>;
+ rotation = <90>;
+ orientation = <0>;
+
+ port {
+ camera1_ep: endpoint {
+ data-lanes = <1 2>;
+ link-frequencies = /bits/ 64
+ <80000000 200000000 300000000>;
+ remote-endpoint = <&mipi1_sensor_ep>;
+ };
+ };
+ };
+
backlight@36 {
compatible = "ti,lm36922";
reg = <0x36>;
@@ -1031,6 +1109,23 @@ &lcdif {
status = "okay";
};
+&mipi_csi1 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "okay";
+
+ ports {
+ port@1 {
+ reg = <1>;
+
+ mipi1_sensor_ep: endpoint {
+ remote-endpoint = <&camera1_ep>;
+ data-lanes = <1 2>;
+ };
+ };
+ };
+};
+
&mipi_dsi {
#address-cells = <1>;
#size-cells = <0>;
diff --git a/arch/arm64/boot/dts/freescale/imx8mq-mnt-reform2.dts b/arch/arm64/boot/dts/freescale/imx8mq-mnt-reform2.dts
index 4f2db61..fa721a1 100644
--- a/arch/arm64/boot/dts/freescale/imx8mq-mnt-reform2.dts
+++ b/arch/arm64/boot/dts/freescale/imx8mq-mnt-reform2.dts
@@ -12,6 +12,7 @@
/ {
model = "MNT Reform 2";
compatible = "mntre,reform2", "boundary,imx8mq-nitrogen8m-som", "fsl,imx8mq";
+ chassis-type = "laptop";
pcie1_refclk: clock-pcie1-refclk {
compatible = "fixed-clock";
diff --git a/arch/arm64/boot/dts/freescale/imx8mq-nitrogen-som.dtsi b/arch/arm64/boot/dts/freescale/imx8mq-nitrogen-som.dtsi
index 36fc428..395f77b 100644
--- a/arch/arm64/boot/dts/freescale/imx8mq-nitrogen-som.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8mq-nitrogen-som.dtsi
@@ -69,6 +69,9 @@ ethphy0: ethernet-phy@4 {
reg = <4>;
interrupt-parent = <&gpio1>;
interrupts = <11 IRQ_TYPE_LEVEL_LOW>;
+ reset-gpios = <&gpio1 9 GPIO_ACTIVE_LOW>;
+ reset-assert-us = <10000>;
+ reset-deassert-us = <300>;
};
};
};
@@ -191,20 +194,20 @@ MX8MQ_IOMUXC_ENET_TD1_ENET1_RGMII_TD1 0x1f
MX8MQ_IOMUXC_ENET_TD2_ENET1_RGMII_TD2 0x1f
MX8MQ_IOMUXC_ENET_TD3_ENET1_RGMII_TD3 0x1f
MX8MQ_IOMUXC_ENET_RX_CTL_ENET1_RGMII_RX_CTL 0x91
- MX8MQ_IOMUXC_ENET_RXC_ENET1_RGMII_RXC 0x91
+ MX8MQ_IOMUXC_ENET_RXC_ENET1_RGMII_RXC 0xd1
MX8MQ_IOMUXC_ENET_RD0_ENET1_RGMII_RD0 0x91
MX8MQ_IOMUXC_ENET_RD1_ENET1_RGMII_RD1 0x91
MX8MQ_IOMUXC_ENET_RD2_ENET1_RGMII_RD2 0x91
- MX8MQ_IOMUXC_ENET_RD3_ENET1_RGMII_RD3 0x91
- MX8MQ_IOMUXC_GPIO1_IO09_GPIO1_IO9 0x19
- MX8MQ_IOMUXC_GPIO1_IO11_GPIO1_IO11 0x59
+ MX8MQ_IOMUXC_ENET_RD3_ENET1_RGMII_RD3 0xd1
+ MX8MQ_IOMUXC_GPIO1_IO09_GPIO1_IO9 0x1
+ MX8MQ_IOMUXC_GPIO1_IO11_GPIO1_IO11 0x41
>;
};
pinctrl_i2c1: i2c1grp {
fsl,pins = <
- MX8MQ_IOMUXC_I2C1_SCL_I2C1_SCL 0x4000007f
- MX8MQ_IOMUXC_I2C1_SDA_I2C1_SDA 0x4000007f
+ MX8MQ_IOMUXC_I2C1_SCL_I2C1_SCL 0x40000022
+ MX8MQ_IOMUXC_I2C1_SDA_I2C1_SDA 0x40000022
>;
};
diff --git a/arch/arm64/boot/dts/freescale/imx8mq-tqma8mq-mba8mx.dts b/arch/arm64/boot/dts/freescale/imx8mq-tqma8mq-mba8mx.dts
new file mode 100644
index 0000000..d7660ea
--- /dev/null
+++ b/arch/arm64/boot/dts/freescale/imx8mq-tqma8mq-mba8mx.dts
@@ -0,0 +1,349 @@
+// SPDX-License-Identifier: (GPL-2.0-or-later OR MIT)
+/*
+ * Copyright 2019-2021 TQ-Systems GmbH
+ */
+
+/dts-v1/;
+
+#include "imx8mq-tqma8mq.dtsi"
+#include "mba8mx.dtsi"
+
+/ {
+ model = "TQ-Systems GmbH i.MX8MQ TQMa8MQ on MBa8Mx";
+ compatible = "tq,imx8mq-tqma8mq-mba8mx", "tq,imx8mq-tqma8mq", "fsl,imx8mq";
+
+ aliases {
+ eeprom0 = &eeprom3;
+ mmc0 = &usdhc1;
+ mmc1 = &usdhc2;
+ rtc0 = &pcf85063;
+ rtc1 = &snvs_rtc;
+ };
+
+ extcon_usbotg: extcon-usbotg0 {
+ compatible = "linux,extcon-usb-gpio";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_usbcon0>;
+ id-gpio = <&gpio1 10 GPIO_ACTIVE_HIGH>;
+ };
+
+ pcie0_refclk: pcie0-refclk {
+ compatible = "fixed-clock";
+ #clock-cells = <0>;
+ clock-frequency = <100000000>;
+ };
+
+ pcie1_refclk: pcie1-refclk {
+ compatible = "fixed-clock";
+ #clock-cells = <0>;
+ clock-frequency = <100000000>;
+ };
+
+ reg_otg_vbus: regulator-otg-vbus {
+ compatible = "regulator-fixed";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_regotgvbus>;
+ regulator-name = "MBA8MQ_OTG_VBUS";
+ regulator-min-microvolt = <5000000>;
+ regulator-max-microvolt = <5000000>;
+ gpio = <&gpio1 12 GPIO_ACTIVE_HIGH>;
+ enable-active-high;
+ };
+
+ reg_usdhc2_vmmc: regulator-vmmc {
+ compatible = "regulator-fixed";
+ regulator-name = "VSD_3V3";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ gpio = <&gpio2 19 GPIO_ACTIVE_HIGH>;
+ enable-active-high;
+ };
+};
+
+&btn2 {
+ gpios = <&gpio3 17 GPIO_ACTIVE_LOW>;
+};
+
+&gpio_leds {
+ led3 {
+ label = "led3";
+ gpios = <&gpio1 8 GPIO_ACTIVE_HIGH>;
+ };
+};
+
+&i2c1 {
+ expander2: gpio@25 {
+ compatible = "nxp,pca9555";
+ reg = <0x25>;
+ gpio-controller;
+ #gpio-cells = <2>;
+ vcc-supply = <®_vcc_3v3>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_expander>;
+ interrupt-parent = <&gpio1>;
+ interrupts = <9 IRQ_TYPE_EDGE_FALLING>;
+ interrupt-controller;
+ #interrupt-cells = <2>;
+
+ mpcie-rst-hog {
+ gpio-hog;
+ gpios = <13 0>;
+ output-high;
+ line-name = "MPCIE_RST#";
+ };
+ };
+};
+
+&irqsteer {
+ status = "okay";
+};
+
+&led2 {
+ gpios = <&gpio3 16 GPIO_ACTIVE_HIGH>;
+};
+
+&pcie0 {
+ reset-gpio = <&expander0 14 GPIO_ACTIVE_LOW>;
+ clocks = <&clk IMX8MQ_CLK_PCIE1_ROOT>,
+ <&clk IMX8MQ_CLK_PCIE1_AUX>,
+ <&clk IMX8MQ_CLK_PCIE1_PHY>,
+ <&pcie0_refclk>;
+ clock-names = "pcie", "pcie_aux", "pcie_phy", "pcie_bus";
+ epdev_on-supply = <®_vcc_3v3>;
+ hard-wired = <1>;
+ status = "okay";
+};
+
+/*
+ * miniPCIe, also usable for cards with USB. Therefore configure the reset as
+ * static gpio hog.
+ */
+&pcie1 {
+ clocks = <&clk IMX8MQ_CLK_PCIE2_ROOT>,
+ <&clk IMX8MQ_CLK_PCIE2_AUX>,
+ <&clk IMX8MQ_CLK_PCIE2_PHY>,
+ <&pcie1_refclk>;
+ clock-names = "pcie", "pcie_aux", "pcie_phy", "pcie_bus";
+ epdev_on-supply = <®_vcc_3v3>;
+ hard-wired = <1>;
+ status = "okay";
+};
+
+&sai3 {
+ assigned-clocks = <&clk IMX8MQ_CLK_SAI3>;
+ assigned-clock-parents = <&clk IMX8MQ_AUDIO_PLL1_OUT>;
+ clock-names = "bus", "mclk0", "mclk1", "mclk2", "mclk3", "pll8k", "pll11k";
+ clocks = <&clk IMX8MQ_CLK_SAI3_IPG>, <&clk IMX8MQ_CLK_DUMMY>,
+ <&clk IMX8MQ_CLK_SAI3_ROOT>, <&clk IMX8MQ_CLK_DUMMY>,
+ <&clk IMX8MQ_CLK_DUMMY>, <&clk IMX8MQ_AUDIO_PLL1_OUT>,
+ <&clk IMX8MQ_AUDIO_PLL2_OUT>;
+};
+
+&tlv320aic3x04 {
+ clock-names = "mclk";
+ clocks = <&clk IMX8MQ_CLK_SAI3_ROOT>;
+};
+
+&uart1 {
+ assigned-clocks = <&clk IMX8MQ_CLK_UART1>;
+ assigned-clock-parents = <&clk IMX8MQ_SYS1_PLL_80M>;
+};
+
+&uart2 {
+ assigned-clocks = <&clk IMX8MQ_CLK_UART2>;
+ assigned-clock-parents = <&clk IMX8MQ_SYS1_PLL_80M>;
+};
+
+/* console */
+&uart3 {
+ assigned-clocks = <&clk IMX8MQ_CLK_UART3>;
+ assigned-clock-parents = <&clk IMX8MQ_CLK_25M>;
+};
+
+&usb3_phy0 {
+ vbus-supply = <®_otg_vbus>;
+ status = "okay";
+};
+
+&usb_dwc3_0 {
+ /* we implement dual role but not full featured OTG */
+ extcon = <&extcon_usbotg>;
+ hnp-disable;
+ srp-disable;
+ adp-disable;
+ /* OC not supported due to non matching active polarity */
+ disable-over-current;
+ dr_mode = "otg";
+ status = "okay";
+};
+
+&usb3_phy1 {
+ status = "okay";
+};
+
+&usb_dwc3_1 {
+ status = "okay";
+ dr_mode = "host";
+};
+
+&wdog1 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_wdog>;
+ fsl,ext-reset-output;
+ status = "okay";
+};
+
+&iomuxc {
+ pinctrl_ecspi1: ecspi1grp {
+ fsl,pins = <MX8MQ_IOMUXC_ECSPI1_SCLK_ECSPI1_SCLK 0x0000004e>,
+ <MX8MQ_IOMUXC_ECSPI1_MOSI_ECSPI1_MOSI 0x0000004e>,
+ <MX8MQ_IOMUXC_ECSPI1_MISO_ECSPI1_MISO 0x0000004e>,
+ <MX8MQ_IOMUXC_ECSPI1_SS0_GPIO5_IO9 0x0000004e>;
+ };
+
+ pinctrl_ecspi2: ecspi2grp {
+ fsl,pins = <MX8MQ_IOMUXC_ECSPI2_SCLK_ECSPI2_SCLK 0x0000004e>,
+ <MX8MQ_IOMUXC_ECSPI2_MOSI_ECSPI2_MOSI 0x0000004e>,
+ <MX8MQ_IOMUXC_ECSPI2_MISO_ECSPI2_MISO 0x0000004e>,
+ <MX8MQ_IOMUXC_ECSPI2_SS0_GPIO5_IO13 0x0000004e>;
+ };
+
+ pinctrl_expander: expandergrp {
+ fsl,pins = <MX8MQ_IOMUXC_GPIO1_IO09_GPIO1_IO9 0xd6>;
+ };
+
+ pinctrl_fec1: fec1grp {
+ fsl,pins = <MX8MQ_IOMUXC_ENET_MDC_ENET1_MDC 0x3>,
+ <MX8MQ_IOMUXC_ENET_MDIO_ENET1_MDIO 0x23>,
+ <MX8MQ_IOMUXC_ENET_TD3_ENET1_RGMII_TD3 0x1f>,
+ <MX8MQ_IOMUXC_ENET_TD2_ENET1_RGMII_TD2 0x1f>,
+ <MX8MQ_IOMUXC_ENET_TD1_ENET1_RGMII_TD1 0x1f>,
+ <MX8MQ_IOMUXC_ENET_TD0_ENET1_RGMII_TD0 0x1f>,
+ <MX8MQ_IOMUXC_ENET_RD3_ENET1_RGMII_RD3 0x91>,
+ <MX8MQ_IOMUXC_ENET_RD2_ENET1_RGMII_RD2 0x91>,
+ <MX8MQ_IOMUXC_ENET_RD1_ENET1_RGMII_RD1 0x91>,
+ <MX8MQ_IOMUXC_ENET_RD0_ENET1_RGMII_RD0 0x91>,
+ <MX8MQ_IOMUXC_ENET_TXC_ENET1_RGMII_TXC 0x1f>,
+ <MX8MQ_IOMUXC_ENET_RXC_ENET1_RGMII_RXC 0x91>,
+ <MX8MQ_IOMUXC_ENET_RX_CTL_ENET1_RGMII_RX_CTL 0x91>,
+ <MX8MQ_IOMUXC_ENET_TX_CTL_ENET1_RGMII_TX_CTL 0x1f>;
+ };
+
+ pinctrl_gpiobutton: gpiobuttongrp {
+ fsl,pins = <MX8MQ_IOMUXC_GPIO1_IO05_GPIO1_IO5 0x41>,
+ <MX8MQ_IOMUXC_GPIO1_IO07_GPIO1_IO7 0x41>,
+ <MX8MQ_IOMUXC_NAND_WE_B_GPIO3_IO17 0x41>;
+ };
+
+ pinctrl_gpioled: gpioledgrp {
+ fsl,pins = <MX8MQ_IOMUXC_GPIO1_IO00_GPIO1_IO0 0x41>,
+ <MX8MQ_IOMUXC_GPIO1_IO08_GPIO1_IO8 0x41>,
+ <MX8MQ_IOMUXC_NAND_READY_B_GPIO3_IO16 0x41>;
+ };
+
+ pinctrl_i2c2: i2c2grp {
+ fsl,pins = <MX8MQ_IOMUXC_I2C2_SCL_I2C2_SCL 0x40000067>,
+ <MX8MQ_IOMUXC_I2C2_SDA_I2C2_SDA 0x40000067>;
+ };
+
+ pinctrl_i2c2_gpio: i2c2gpiogrp {
+ fsl,pins = <MX8MQ_IOMUXC_I2C2_SCL_GPIO5_IO16 0x40000067>,
+ <MX8MQ_IOMUXC_I2C2_SDA_GPIO5_IO17 0x40000067>;
+ };
+
+ pinctrl_i2c3: i2c3grp {
+ fsl,pins = <MX8MQ_IOMUXC_I2C3_SCL_I2C3_SCL 0x40000067>,
+ <MX8MQ_IOMUXC_I2C3_SDA_I2C3_SDA 0x40000067>;
+ };
+
+ pinctrl_i2c3_gpio: i2c3gpiogrp {
+ fsl,pins = <MX8MQ_IOMUXC_I2C3_SCL_GPIO5_IO18 0x40000067>,
+ <MX8MQ_IOMUXC_I2C3_SDA_GPIO5_IO19 0x40000067>;
+ };
+
+ pinctrl_pwm3: pwm3grp {
+ fsl,pins = <MX8MQ_IOMUXC_GPIO1_IO14_PWM3_OUT 0x16>;
+ };
+
+ pinctrl_pwm4: pwm4grp {
+ fsl,pins = <MX8MQ_IOMUXC_GPIO1_IO15_PWM4_OUT 0x16>;
+ };
+
+ pinctrl_regotgvbus: reggotgvbusgrp {
+ /* USB1 OTG PWR as GPIO */
+ fsl,pins = <MX8MQ_IOMUXC_GPIO1_IO12_GPIO1_IO12 0x06>;
+ };
+
+ pinctrl_reg_usdhc2_vmmc: regusdhc2vmmcgrp {
+ fsl,pins = <MX8MQ_IOMUXC_SD2_RESET_B_GPIO2_IO19 0xc1>;
+ };
+
+ pinctrl_sai3: sai3grp {
+ fsl,pins = <MX8MQ_IOMUXC_SAI3_MCLK_SAI3_MCLK 0xd6>,
+ <MX8MQ_IOMUXC_SAI3_RXC_SAI3_RX_BCLK 0xd6>,
+ <MX8MQ_IOMUXC_SAI3_RXFS_SAI3_RX_SYNC 0xd6>,
+ <MX8MQ_IOMUXC_SAI3_RXD_SAI3_RX_DATA0 0xd6>,
+ <MX8MQ_IOMUXC_SAI3_TXFS_SAI3_TX_SYNC 0xd6>,
+ <MX8MQ_IOMUXC_SAI3_TXD_SAI3_TX_DATA0 0xd6>,
+ <MX8MQ_IOMUXC_SAI3_TXC_SAI3_TX_BCLK 0xd6>;
+ };
+
+ pinctrl_uart1: uart1grp {
+ fsl,pins = <MX8MQ_IOMUXC_UART1_RXD_UART1_DCE_RX 0x79>,
+ <MX8MQ_IOMUXC_UART1_TXD_UART1_DCE_TX 0x79>;
+ };
+
+ pinctrl_uart2: uart2grp {
+ fsl,pins = <MX8MQ_IOMUXC_UART2_RXD_UART2_DCE_RX 0x79>,
+ <MX8MQ_IOMUXC_UART2_TXD_UART2_DCE_TX 0x79>;
+ };
+
+ pinctrl_uart3: uart3grp {
+ fsl,pins = <MX8MQ_IOMUXC_UART3_RXD_UART3_DCE_RX 0x79>,
+ <MX8MQ_IOMUXC_UART3_TXD_UART3_DCE_TX 0x79>;
+ };
+
+ pinctrl_uart4: uart4grp {
+ fsl,pins = <MX8MQ_IOMUXC_UART4_RXD_UART4_DCE_RX 0x79>,
+ <MX8MQ_IOMUXC_UART4_TXD_UART4_DCE_TX 0x79>;
+ };
+
+ pinctrl_usbcon0: usb0congrp {
+ /* ID: floating / high: device, low: host -> use PU */
+ fsl,pins = <MX8MQ_IOMUXC_GPIO1_IO10_GPIO1_IO10 0xe6>;
+ };
+
+ pinctrl_usdhc2: usdhc2grp {
+ fsl,pins = <MX8MQ_IOMUXC_SD2_CLK_USDHC2_CLK 0x83>,
+ <MX8MQ_IOMUXC_SD2_CMD_USDHC2_CMD 0xc3>,
+ <MX8MQ_IOMUXC_SD2_DATA0_USDHC2_DATA0 0xc3>,
+ <MX8MQ_IOMUXC_SD2_DATA1_USDHC2_DATA1 0xc3>,
+ <MX8MQ_IOMUXC_SD2_DATA2_USDHC2_DATA2 0xc3>,
+ <MX8MQ_IOMUXC_SD2_DATA3_USDHC2_DATA3 0xc3>,
+ <MX8MQ_IOMUXC_GPIO1_IO04_USDHC2_VSELECT 0xc1>;
+ };
+
+ pinctrl_usdhc2_100mhz: usdhc2-100mhzgrp {
+ fsl,pins = <MX8MQ_IOMUXC_SD2_CLK_USDHC2_CLK 0x85>,
+ <MX8MQ_IOMUXC_SD2_CMD_USDHC2_CMD 0xc5>,
+ <MX8MQ_IOMUXC_SD2_DATA0_USDHC2_DATA0 0xc5>,
+ <MX8MQ_IOMUXC_SD2_DATA1_USDHC2_DATA1 0xc5>,
+ <MX8MQ_IOMUXC_SD2_DATA2_USDHC2_DATA2 0xc5>,
+ <MX8MQ_IOMUXC_SD2_DATA3_USDHC2_DATA3 0xc5>,
+ <MX8MQ_IOMUXC_GPIO1_IO04_USDHC2_VSELECT 0xc1>;
+ };
+
+ pinctrl_usdhc2_200mhz: usdhc2-200mhzgrp {
+ fsl,pins = <MX8MQ_IOMUXC_SD2_CLK_USDHC2_CLK 0x9f>,
+ <MX8MQ_IOMUXC_SD2_CMD_USDHC2_CMD 0xc7>,
+ <MX8MQ_IOMUXC_SD2_DATA0_USDHC2_DATA0 0xc7>,
+ <MX8MQ_IOMUXC_SD2_DATA1_USDHC2_DATA1 0xc7>,
+ <MX8MQ_IOMUXC_SD2_DATA2_USDHC2_DATA2 0xc7>,
+ <MX8MQ_IOMUXC_SD2_DATA3_USDHC2_DATA3 0xc7>,
+ <MX8MQ_IOMUXC_GPIO1_IO04_USDHC2_VSELECT 0xc1>;
+ };
+
+ pinctrl_usdhc2_gpio: usdhc2-gpiogrp {
+ fsl,pins = <MX8MQ_IOMUXC_SD2_CD_B_GPIO2_IO12 0x41>;
+ };
+};
diff --git a/arch/arm64/boot/dts/freescale/imx8mq-tqma8mq.dtsi b/arch/arm64/boot/dts/freescale/imx8mq-tqma8mq.dtsi
new file mode 100644
index 0000000..8aedcdd
--- /dev/null
+++ b/arch/arm64/boot/dts/freescale/imx8mq-tqma8mq.dtsi
@@ -0,0 +1,360 @@
+// SPDX-License-Identifier: (GPL-2.0-or-later OR MIT)
+/*
+ * Copyright 2019-2021 TQ-Systems GmbH
+ */
+
+#include "imx8mq.dtsi"
+
+/ {
+ model = "TQ-Systems GmbH i.MX8MQ TQMa8MQ";
+ compatible = "tq,imx8mq-tqma8mq", "fsl,imx8mq";
+
+ memory@40000000 {
+ device_type = "memory";
+ /* our minimum RAM config will be 1024 MiB */
+ reg = <0x00000000 0x40000000 0 0x40000000>;
+ };
+
+ /* e-MMC IO, needed for HS modes */
+ reg_vcc1v8: regulator-vcc1v8 {
+ compatible = "regulator-fixed";
+ regulator-name = "TQMA8MX_VCC1V8";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ };
+
+ reg_vcc3v3: regulator-vcc3v3 {
+ compatible = "regulator-fixed";
+ regulator-name = "TQMA8MX_VCC3V3";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ };
+
+ reg_vdd_arm: regulator-vdd-arm {
+ compatible = "regulator-gpio";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_dvfs>;
+ regulator-min-microvolt = <900000>;
+ regulator-max-microvolt = <1000000>;
+ regulator-name = "TQMa8Mx_DVFS";
+ regulator-type = "voltage";
+ regulator-settling-time-us = <150000>;
+ gpios = <&gpio1 6 GPIO_ACTIVE_LOW>;
+ states = <900000 0x1 1000000 0x0>;
+ };
+
+ reserved-memory {
+ #address-cells = <2>;
+ #size-cells = <2>;
+ ranges;
+
+ /* global autoconfigured region for contiguous allocations */
+ linux,cma {
+ compatible = "shared-dma-pool";
+ reusable;
+ /* 640 MiB */
+ size = <0 0x28000000>;
+ /* 1024 - 128 MiB, our minimum RAM config will be 1024 MiB */
+ alloc-ranges = <0 0x40000000 0 0x78000000>;
+ linux,cma-default;
+ };
+ };
+};
+
+&A53_0 {
+ cpu-supply = <®_vdd_arm>;
+};
+
+&A53_1 {
+ cpu-supply = <®_vdd_arm>;
+};
+
+&A53_2 {
+ cpu-supply = <®_vdd_arm>;
+};
+
+&A53_3 {
+ cpu-supply = <®_vdd_arm>;
+};
+
+&gpu {
+ status = "okay";
+};
+
+&pgc_gpu {
+ power-supply = <&sw1a_reg>;
+};
+
+&pgc_vpu {
+ power-supply = <&sw1c_reg>;
+};
+
+&i2c1 {
+ clock-frequency = <100000>;
+ pinctrl-names = "default", "gpio";
+ pinctrl-0 = <&pinctrl_i2c1>;
+ pinctrl-1 = <&pinctrl_i2c1_gpio>;
+ scl-gpios = <&gpio5 14 (GPIO_ACTIVE_HIGH | GPIO_OPEN_DRAIN)>;
+ sda-gpios = <&gpio5 15 (GPIO_ACTIVE_HIGH | GPIO_OPEN_DRAIN)>;
+ status = "okay";
+
+ pfuze100: pmic@8 {
+ compatible = "fsl,pfuze100";
+ fsl,pfuze-support-disable-sw;
+ reg = <0x8>;
+
+ regulators {
+ /* VDD_GPU */
+ sw1a_reg: sw1ab {
+ regulator-min-microvolt = <825000>;
+ regulator-max-microvolt = <1100000>;
+ };
+
+ /* VDD_VPU */
+ sw1c_reg: sw1c {
+ regulator-min-microvolt = <825000>;
+ regulator-max-microvolt = <1100000>;
+ };
+
+ /* NVCC_DRAM */
+ sw2_reg: sw2 {
+ regulator-min-microvolt = <1100000>;
+ regulator-max-microvolt = <1100000>;
+ regulator-always-on;
+ };
+
+ /* VDD_DRAM */
+ sw3a_reg: sw3ab {
+ regulator-min-microvolt = <825000>;
+ regulator-max-microvolt = <1100000>;
+ regulator-always-on;
+ };
+
+ /* 1.8 V for QSPI NOR, e-MMC IO, must not be changed */
+ nvcc_1v8_reg: sw4 {
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ regulator-always-on;
+ };
+
+ swbst_reg: swbst {
+ regulator-min-microvolt = <5000000>;
+ regulator-max-microvolt = <5150000>;
+ };
+
+ snvs_reg: vsnvs {
+ regulator-min-microvolt = <1000000>;
+ regulator-max-microvolt = <3000000>;
+ regulator-always-on;
+ };
+
+ vref_reg: vrefddr {
+ regulator-always-on;
+ };
+
+ /* not used */
+ vgen1_reg: vgen1 {
+ regulator-min-microvolt = <800000>;
+ regulator-max-microvolt = <1550000>;
+ };
+
+ /* VDD_PHY_0V9 */
+ vgen2_reg: vgen2 {
+ regulator-min-microvolt = <850000>;
+ regulator-max-microvolt = <975000>;
+ regulator-always-on;
+ };
+
+ /* VDD_PHY_1V8 */
+ vgen3_reg: vgen3 {
+ regulator-min-microvolt = <1675000>;
+ regulator-max-microvolt = <1975000>;
+ regulator-always-on;
+ };
+
+ /* VDDA_1V8 */
+ vgen4_reg: vgen4 {
+ regulator-min-microvolt = <1625000>;
+ regulator-max-microvolt = <1875000>;
+ regulator-always-on;
+ };
+
+ /* VDD_PHY_3V3 */
+ vgen5_reg: vgen5 {
+ regulator-min-microvolt = <3075000>;
+ regulator-max-microvolt = <3625000>;
+ regulator-always-on;
+ };
+
+ /* not used */
+ vgen6_reg: vgen6 {
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <3300000>;
+ };
+ };
+ };
+
+ sensor0: temperature-sensor-eeprom@1b {
+ compatible = "nxp,se97", "jedec,jc-42.4-temp";
+ reg = <0x1b>;
+ };
+
+ pcf85063: rtc@51 {
+ compatible = "nxp,pcf85063a";
+ reg = <0x51>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_rtc>;
+ interrupt-names = "irq";
+ interrupt-parent = <&gpio1>;
+ interrupts = <1 IRQ_TYPE_EDGE_FALLING>;
+ quartz-load-femtofarads = <7000>;
+
+ clock {
+ compatible = "fixed-clock";
+ #clock-cells = <0>;
+ clock-frequency = <32768>;
+ };
+ };
+
+ eeprom1: eeprom@53 {
+ compatible = "nxp,se97b", "atmel,24c02";
+ reg = <0x53>;
+ pagesize = <16>;
+ read-only;
+ };
+
+ eeprom0: eeprom@57 {
+ compatible = "atmel,24c64";
+ reg = <0x57>;
+ pagesize = <32>;
+ };
+};
+
+&pcie0 {
+ /* 3.3V supply, only way to switch on internal 1.8V supply using GPR */
+ vph-supply = <&vgen5_reg>;
+};
+
+&pcie1 {
+ /* 3.3V supply, only way to switch on internal 1.8V supply using GPR */
+ vph-supply = <&vgen5_reg>;
+};
+
+&qspi0 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_qspi>;
+ assigned-clocks = <&clk IMX8MQ_CLK_QSPI>;
+ assigned-clock-parents = <&clk IMX8MQ_SYS1_PLL_266M>;
+ status = "okay";
+
+ flash0: flash@0 {
+ compatible = "jedec,spi-nor";
+ reg = <0>;
+ #address-cells = <1>;
+ #size-cells = <1>;
+ spi-max-frequency = <84000000>;
+ spi-tx-bus-width = <4>;
+ spi-rx-bus-width = <4>;
+ };
+};
+
+&usdhc1 {
+ pinctrl-names = "default", "state_100mhz", "state_200mhz";
+ pinctrl-0 = <&pinctrl_usdhc1>;
+ pinctrl-1 = <&pinctrl_usdhc1_100mhz>;
+ pinctrl-2 = <&pinctrl_usdhc1_200mhz>;
+ bus-width = <8>;
+ non-removable;
+ no-sd;
+ no-sdio;
+ vmmc-supply = <®_vcc3v3>;
+ vqmmc-supply = <®_vcc1v8>;
+ status = "okay";
+};
+
+&vpu {
+ status = "okay";
+};
+
+/* Attention: wdog reset forcing POR needs baseboard support */
+&wdog1 {
+ status = "okay";
+};
+
+&iomuxc {
+ pinctrl_dvfs: dvfsgrp {
+ fsl,pins = <MX8MQ_IOMUXC_GPIO1_IO06_GPIO1_IO6 0x16>;
+ };
+
+ pinctrl_i2c1: i2c1grp {
+ fsl,pins = <MX8MQ_IOMUXC_I2C1_SCL_I2C1_SCL 0x4000007f>,
+ <MX8MQ_IOMUXC_I2C1_SDA_I2C1_SDA 0x4000007f>;
+ };
+
+ pinctrl_i2c1_gpio: i2c1gpiogrp {
+ fsl,pins = <MX8MQ_IOMUXC_I2C1_SCL_GPIO5_IO14 0x40000074>,
+ <MX8MQ_IOMUXC_I2C1_SDA_GPIO5_IO15 0x40000074>;
+ };
+
+ pinctrl_qspi: qspigrp {
+ fsl,pins = <MX8MQ_IOMUXC_NAND_ALE_QSPI_A_SCLK 0x97>,
+ <MX8MQ_IOMUXC_NAND_CE0_B_QSPI_A_SS0_B 0x82>,
+ <MX8MQ_IOMUXC_NAND_DATA00_QSPI_A_DATA0 0x97>,
+ <MX8MQ_IOMUXC_NAND_DATA01_QSPI_A_DATA1 0x97>,
+ <MX8MQ_IOMUXC_NAND_DATA02_QSPI_A_DATA2 0x97>,
+ <MX8MQ_IOMUXC_NAND_DATA03_QSPI_A_DATA3 0x97>;
+ };
+
+ pinctrl_rtc: rtcgrp {
+ fsl,pins = <MX8MQ_IOMUXC_GPIO1_IO01_GPIO1_IO1 0x41>;
+ };
+
+ pinctrl_usdhc1: usdhc1grp {
+ fsl,pins = <MX8MQ_IOMUXC_SD1_CLK_USDHC1_CLK 0x83>,
+ <MX8MQ_IOMUXC_SD1_CMD_USDHC1_CMD 0xc3>,
+ <MX8MQ_IOMUXC_SD1_DATA0_USDHC1_DATA0 0xc3>,
+ <MX8MQ_IOMUXC_SD1_DATA1_USDHC1_DATA1 0xc3>,
+ <MX8MQ_IOMUXC_SD1_DATA2_USDHC1_DATA2 0xc3>,
+ <MX8MQ_IOMUXC_SD1_DATA3_USDHC1_DATA3 0xc3>,
+ <MX8MQ_IOMUXC_SD1_DATA4_USDHC1_DATA4 0xc3>,
+ <MX8MQ_IOMUXC_SD1_DATA5_USDHC1_DATA5 0xc3>,
+ <MX8MQ_IOMUXC_SD1_DATA6_USDHC1_DATA6 0xc3>,
+ <MX8MQ_IOMUXC_SD1_DATA7_USDHC1_DATA7 0xc3>,
+ <MX8MQ_IOMUXC_SD1_STROBE_USDHC1_STROBE 0x83>,
+ <MX8MQ_IOMUXC_SD1_RESET_B_USDHC1_RESET_B 0xc1>;
+ };
+
+ pinctrl_usdhc1_100mhz: usdhc1-100mhzgrp {
+ fsl,pins = <MX8MQ_IOMUXC_SD1_CLK_USDHC1_CLK 0x85>,
+ <MX8MQ_IOMUXC_SD1_CMD_USDHC1_CMD 0xc5>,
+ <MX8MQ_IOMUXC_SD1_DATA0_USDHC1_DATA0 0xc5>,
+ <MX8MQ_IOMUXC_SD1_DATA1_USDHC1_DATA1 0xc5>,
+ <MX8MQ_IOMUXC_SD1_DATA2_USDHC1_DATA2 0xc5>,
+ <MX8MQ_IOMUXC_SD1_DATA3_USDHC1_DATA3 0xc5>,
+ <MX8MQ_IOMUXC_SD1_DATA4_USDHC1_DATA4 0xc5>,
+ <MX8MQ_IOMUXC_SD1_DATA5_USDHC1_DATA5 0xc5>,
+ <MX8MQ_IOMUXC_SD1_DATA6_USDHC1_DATA6 0xc5>,
+ <MX8MQ_IOMUXC_SD1_DATA7_USDHC1_DATA7 0xc5>,
+ <MX8MQ_IOMUXC_SD1_STROBE_USDHC1_STROBE 0x85>,
+ <MX8MQ_IOMUXC_SD1_RESET_B_USDHC1_RESET_B 0xc1>;
+ };
+
+ pinctrl_usdhc1_200mhz: usdhc1-200mhzgrp {
+ fsl,pins = <MX8MQ_IOMUXC_SD1_CLK_USDHC1_CLK 0x87>,
+ <MX8MQ_IOMUXC_SD1_CMD_USDHC1_CMD 0xc7>,
+ <MX8MQ_IOMUXC_SD1_DATA0_USDHC1_DATA0 0xc7>,
+ <MX8MQ_IOMUXC_SD1_DATA1_USDHC1_DATA1 0xc7>,
+ <MX8MQ_IOMUXC_SD1_DATA2_USDHC1_DATA2 0xc7>,
+ <MX8MQ_IOMUXC_SD1_DATA3_USDHC1_DATA3 0xc7>,
+ <MX8MQ_IOMUXC_SD1_DATA4_USDHC1_DATA4 0xc7>,
+ <MX8MQ_IOMUXC_SD1_DATA5_USDHC1_DATA5 0xc7>,
+ <MX8MQ_IOMUXC_SD1_DATA6_USDHC1_DATA6 0xc7>,
+ <MX8MQ_IOMUXC_SD1_DATA7_USDHC1_DATA7 0xc7>,
+ <MX8MQ_IOMUXC_SD1_STROBE_USDHC1_STROBE 0x87>,
+ <MX8MQ_IOMUXC_SD1_RESET_B_USDHC1_RESET_B 0xc1>;
+ };
+
+ pinctrl_wdog: wdoggrp {
+ fsl,pins = <MX8MQ_IOMUXC_GPIO1_IO02_WDOG1_WDOG_B 0xc6>;
+ };
+};
diff --git a/arch/arm64/boot/dts/freescale/imx8mq.dtsi b/arch/arm64/boot/dts/freescale/imx8mq.dtsi
index 71bf497..2df2510 100644
--- a/arch/arm64/boot/dts/freescale/imx8mq.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8mq.dtsi
@@ -102,6 +102,12 @@ A53_0: cpu@0 {
clock-latency = <61036>; /* two CLK32 periods */
clocks = <&clk IMX8MQ_CLK_ARM>;
enable-method = "psci";
+ i-cache-size = <0x8000>;
+ i-cache-line-size = <64>;
+ i-cache-sets = <256>;
+ d-cache-size = <0x8000>;
+ d-cache-line-size = <64>;
+ d-cache-sets = <128>;
next-level-cache = <&A53_L2>;
operating-points-v2 = <&a53_opp_table>;
#cooling-cells = <2>;
@@ -116,6 +122,12 @@ A53_1: cpu@1 {
clock-latency = <61036>; /* two CLK32 periods */
clocks = <&clk IMX8MQ_CLK_ARM>;
enable-method = "psci";
+ i-cache-size = <0x8000>;
+ i-cache-line-size = <64>;
+ i-cache-sets = <256>;
+ d-cache-size = <0x8000>;
+ d-cache-line-size = <64>;
+ d-cache-sets = <128>;
next-level-cache = <&A53_L2>;
operating-points-v2 = <&a53_opp_table>;
#cooling-cells = <2>;
@@ -128,6 +140,12 @@ A53_2: cpu@2 {
clock-latency = <61036>; /* two CLK32 periods */
clocks = <&clk IMX8MQ_CLK_ARM>;
enable-method = "psci";
+ i-cache-size = <0x8000>;
+ i-cache-line-size = <64>;
+ i-cache-sets = <256>;
+ d-cache-size = <0x8000>;
+ d-cache-line-size = <64>;
+ d-cache-sets = <128>;
next-level-cache = <&A53_L2>;
operating-points-v2 = <&a53_opp_table>;
#cooling-cells = <2>;
@@ -140,6 +158,12 @@ A53_3: cpu@3 {
clock-latency = <61036>; /* two CLK32 periods */
clocks = <&clk IMX8MQ_CLK_ARM>;
enable-method = "psci";
+ i-cache-size = <0x8000>;
+ i-cache-line-size = <64>;
+ i-cache-sets = <256>;
+ d-cache-size = <0x8000>;
+ d-cache-line-size = <64>;
+ d-cache-sets = <128>;
next-level-cache = <&A53_L2>;
operating-points-v2 = <&a53_opp_table>;
#cooling-cells = <2>;
@@ -147,6 +171,10 @@ A53_3: cpu@3 {
A53_L2: l2-cache0 {
compatible = "cache";
+ cache-level = <2>;
+ cache-size = <0x100000>;
+ cache-line-size = <64>;
+ cache-sets = <1024>;
};
};
@@ -429,49 +457,49 @@ tmu: tmu@30260000 {
clocks = <&clk IMX8MQ_CLK_TMU_ROOT>;
little-endian;
fsl,tmu-range = <0xb0000 0xa0026 0x80048 0x70061>;
- fsl,tmu-calibration = <0x00000000 0x00000023
- 0x00000001 0x00000029
- 0x00000002 0x0000002f
- 0x00000003 0x00000035
- 0x00000004 0x0000003d
- 0x00000005 0x00000043
- 0x00000006 0x0000004b
- 0x00000007 0x00000051
- 0x00000008 0x00000057
- 0x00000009 0x0000005f
- 0x0000000a 0x00000067
- 0x0000000b 0x0000006f
+ fsl,tmu-calibration = <0x00000000 0x00000023>,
+ <0x00000001 0x00000029>,
+ <0x00000002 0x0000002f>,
+ <0x00000003 0x00000035>,
+ <0x00000004 0x0000003d>,
+ <0x00000005 0x00000043>,
+ <0x00000006 0x0000004b>,
+ <0x00000007 0x00000051>,
+ <0x00000008 0x00000057>,
+ <0x00000009 0x0000005f>,
+ <0x0000000a 0x00000067>,
+ <0x0000000b 0x0000006f>,
- 0x00010000 0x0000001b
- 0x00010001 0x00000023
- 0x00010002 0x0000002b
- 0x00010003 0x00000033
- 0x00010004 0x0000003b
- 0x00010005 0x00000043
- 0x00010006 0x0000004b
- 0x00010007 0x00000055
- 0x00010008 0x0000005d
- 0x00010009 0x00000067
- 0x0001000a 0x00000070
+ <0x00010000 0x0000001b>,
+ <0x00010001 0x00000023>,
+ <0x00010002 0x0000002b>,
+ <0x00010003 0x00000033>,
+ <0x00010004 0x0000003b>,
+ <0x00010005 0x00000043>,
+ <0x00010006 0x0000004b>,
+ <0x00010007 0x00000055>,
+ <0x00010008 0x0000005d>,
+ <0x00010009 0x00000067>,
+ <0x0001000a 0x00000070>,
- 0x00020000 0x00000017
- 0x00020001 0x00000023
- 0x00020002 0x0000002d
- 0x00020003 0x00000037
- 0x00020004 0x00000041
- 0x00020005 0x0000004b
- 0x00020006 0x00000057
- 0x00020007 0x00000063
- 0x00020008 0x0000006f
+ <0x00020000 0x00000017>,
+ <0x00020001 0x00000023>,
+ <0x00020002 0x0000002d>,
+ <0x00020003 0x00000037>,
+ <0x00020004 0x00000041>,
+ <0x00020005 0x0000004b>,
+ <0x00020006 0x00000057>,
+ <0x00020007 0x00000063>,
+ <0x00020008 0x0000006f>,
- 0x00030000 0x00000015
- 0x00030001 0x00000021
- 0x00030002 0x0000002d
- 0x00030003 0x00000039
- 0x00030004 0x00000045
- 0x00030005 0x00000053
- 0x00030006 0x0000005f
- 0x00030007 0x00000071>;
+ <0x00030000 0x00000015>,
+ <0x00030001 0x00000021>,
+ <0x00030002 0x0000002d>,
+ <0x00030003 0x00000039>,
+ <0x00030004 0x00000045>,
+ <0x00030005 0x00000053>,
+ <0x00030006 0x0000005f>,
+ <0x00030007 0x00000071>;
#thermal-sensor-cells = <1>;
};
@@ -1290,7 +1318,6 @@ fec1: ethernet@30be0000 {
fsl,num-rx-queues = <3>;
nvmem-cells = <&fec_mac_address>;
nvmem-cell-names = "mac-address";
- nvmem_macaddr_swap;
fsl,stop-mode = <&iomuxc_gpr 0x10 3>;
status = "disabled";
};
diff --git a/arch/arm64/boot/dts/freescale/imx8qm.dtsi b/arch/arm64/boot/dts/freescale/imx8qm.dtsi
index aebbe2b..4a7c017 100644
--- a/arch/arm64/boot/dts/freescale/imx8qm.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8qm.dtsi
@@ -57,6 +57,12 @@ A53_0: cpu@0 {
compatible = "arm,cortex-a53", "arm,armv8";
reg = <0x0 0x0>;
enable-method = "psci";
+ i-cache-size = <0x8000>;
+ i-cache-line-size = <64>;
+ i-cache-sets = <256>;
+ d-cache-size = <0x8000>;
+ d-cache-line-size = <64>;
+ d-cache-sets = <128>;
next-level-cache = <&A53_L2>;
};
@@ -65,6 +71,12 @@ A53_1: cpu@1 {
compatible = "arm,cortex-a53", "arm,armv8";
reg = <0x0 0x1>;
enable-method = "psci";
+ i-cache-size = <0x8000>;
+ i-cache-line-size = <64>;
+ i-cache-sets = <256>;
+ d-cache-size = <0x8000>;
+ d-cache-line-size = <64>;
+ d-cache-sets = <128>;
next-level-cache = <&A53_L2>;
};
@@ -73,6 +85,12 @@ A53_2: cpu@2 {
compatible = "arm,cortex-a53", "arm,armv8";
reg = <0x0 0x2>;
enable-method = "psci";
+ i-cache-size = <0x8000>;
+ i-cache-line-size = <64>;
+ i-cache-sets = <256>;
+ d-cache-size = <0x8000>;
+ d-cache-line-size = <64>;
+ d-cache-sets = <128>;
next-level-cache = <&A53_L2>;
};
@@ -81,6 +99,12 @@ A53_3: cpu@3 {
compatible = "arm,cortex-a53", "arm,armv8";
reg = <0x0 0x3>;
enable-method = "psci";
+ i-cache-size = <0x8000>;
+ i-cache-line-size = <64>;
+ i-cache-sets = <256>;
+ d-cache-size = <0x8000>;
+ d-cache-line-size = <64>;
+ d-cache-sets = <128>;
next-level-cache = <&A53_L2>;
};
@@ -89,6 +113,12 @@ A72_0: cpu@100 {
compatible = "arm,cortex-a72", "arm,armv8";
reg = <0x0 0x100>;
enable-method = "psci";
+ i-cache-size = <0xC000>;
+ i-cache-line-size = <64>;
+ i-cache-sets = <256>;
+ d-cache-size = <0x8000>;
+ d-cache-line-size = <64>;
+ d-cache-sets = <256>;
next-level-cache = <&A72_L2>;
};
@@ -102,10 +132,18 @@ A72_1: cpu@101 {
A53_L2: l2-cache0 {
compatible = "cache";
+ cache-level = <2>;
+ cache-size = <0x100000>;
+ cache-line-size = <64>;
+ cache-sets = <1024>;
};
A72_L2: l2-cache1 {
compatible = "cache";
+ cache-level = <2>;
+ cache-size = <0x100000>;
+ cache-line-size = <64>;
+ cache-sets = <1024>;
};
};
diff --git a/arch/arm64/boot/dts/freescale/imx8qxp.dtsi b/arch/arm64/boot/dts/freescale/imx8qxp.dtsi
index 617618e..dbec7c1 100644
--- a/arch/arm64/boot/dts/freescale/imx8qxp.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8qxp.dtsi
@@ -58,6 +58,12 @@ A35_0: cpu@0 {
compatible = "arm,cortex-a35";
reg = <0x0 0x0>;
enable-method = "psci";
+ i-cache-size = <0x8000>;
+ i-cache-line-size = <64>;
+ i-cache-sets = <256>;
+ d-cache-size = <0x8000>;
+ d-cache-line-size = <64>;
+ d-cache-sets = <128>;
next-level-cache = <&A35_L2>;
clocks = <&clk IMX_SC_R_A35 IMX_SC_PM_CLK_CPU>;
operating-points-v2 = <&a35_opp_table>;
@@ -69,6 +75,12 @@ A35_1: cpu@1 {
compatible = "arm,cortex-a35";
reg = <0x0 0x1>;
enable-method = "psci";
+ i-cache-size = <0x8000>;
+ i-cache-line-size = <64>;
+ i-cache-sets = <256>;
+ d-cache-size = <0x8000>;
+ d-cache-line-size = <64>;
+ d-cache-sets = <128>;
next-level-cache = <&A35_L2>;
clocks = <&clk IMX_SC_R_A35 IMX_SC_PM_CLK_CPU>;
operating-points-v2 = <&a35_opp_table>;
@@ -80,6 +92,12 @@ A35_2: cpu@2 {
compatible = "arm,cortex-a35";
reg = <0x0 0x2>;
enable-method = "psci";
+ i-cache-size = <0x8000>;
+ i-cache-line-size = <64>;
+ i-cache-sets = <256>;
+ d-cache-size = <0x8000>;
+ d-cache-line-size = <64>;
+ d-cache-sets = <128>;
next-level-cache = <&A35_L2>;
clocks = <&clk IMX_SC_R_A35 IMX_SC_PM_CLK_CPU>;
operating-points-v2 = <&a35_opp_table>;
@@ -91,6 +109,12 @@ A35_3: cpu@3 {
compatible = "arm,cortex-a35";
reg = <0x0 0x3>;
enable-method = "psci";
+ i-cache-size = <0x8000>;
+ i-cache-line-size = <64>;
+ i-cache-sets = <256>;
+ d-cache-size = <0x8000>;
+ d-cache-line-size = <64>;
+ d-cache-sets = <128>;
next-level-cache = <&A35_L2>;
clocks = <&clk IMX_SC_R_A35 IMX_SC_PM_CLK_CPU>;
operating-points-v2 = <&a35_opp_table>;
@@ -99,6 +123,10 @@ A35_3: cpu@3 {
A35_L2: l2-cache0 {
compatible = "cache";
+ cache-level = <2>;
+ cache-size = <0x80000>;
+ cache-line-size = <64>;
+ cache-sets = <1024>;
};
};
diff --git a/arch/arm64/boot/dts/freescale/imx8ulp-evk.dts b/arch/arm64/boot/dts/freescale/imx8ulp-evk.dts
new file mode 100644
index 0000000..33e84c4
--- /dev/null
+++ b/arch/arm64/boot/dts/freescale/imx8ulp-evk.dts
@@ -0,0 +1,64 @@
+// SPDX-License-Identifier: (GPL-2.0+ OR MIT)
+/*
+ * Copyright 2021 NXP
+ */
+
+/dts-v1/;
+
+#include "imx8ulp.dtsi"
+
+/ {
+ model = "NXP i.MX8ULP EVK";
+ compatible = "fsl,imx8ulp-evk", "fsl,imx8ulp";
+
+ chosen {
+ stdout-path = &lpuart5;
+ };
+
+ memory@80000000 {
+ device_type = "memory";
+ reg = <0x0 0x80000000 0 0x80000000>;
+ };
+};
+
+&lpuart5 {
+ /* console */
+ pinctrl-names = "default", "sleep";
+ pinctrl-0 = <&pinctrl_lpuart5>;
+ pinctrl-1 = <&pinctrl_lpuart5>;
+ status = "okay";
+};
+
+&usdhc0 {
+ pinctrl-names = "default", "sleep";
+ pinctrl-0 = <&pinctrl_usdhc0>;
+ pinctrl-1 = <&pinctrl_usdhc0>;
+ non-removable;
+ bus-width = <8>;
+ status = "okay";
+};
+
+&iomuxc1 {
+ pinctrl_lpuart5: lpuart5grp {
+ fsl,pins = <
+ MX8ULP_PAD_PTF14__LPUART5_TX 0x3
+ MX8ULP_PAD_PTF15__LPUART5_RX 0x3
+ >;
+ };
+
+ pinctrl_usdhc0: usdhc0grp {
+ fsl,pins = <
+ MX8ULP_PAD_PTD1__SDHC0_CMD 0x43
+ MX8ULP_PAD_PTD2__SDHC0_CLK 0x10042
+ MX8ULP_PAD_PTD10__SDHC0_D0 0x43
+ MX8ULP_PAD_PTD9__SDHC0_D1 0x43
+ MX8ULP_PAD_PTD8__SDHC0_D2 0x43
+ MX8ULP_PAD_PTD7__SDHC0_D3 0x43
+ MX8ULP_PAD_PTD6__SDHC0_D4 0x43
+ MX8ULP_PAD_PTD5__SDHC0_D5 0x43
+ MX8ULP_PAD_PTD4__SDHC0_D6 0x43
+ MX8ULP_PAD_PTD3__SDHC0_D7 0x43
+ MX8ULP_PAD_PTD11__SDHC0_DQS 0x10042
+ >;
+ };
+};
diff --git a/arch/arm64/boot/dts/freescale/imx8ulp-pinfunc.h b/arch/arm64/boot/dts/freescale/imx8ulp-pinfunc.h
new file mode 100755
index 0000000..b204ac7
--- /dev/null
+++ b/arch/arm64/boot/dts/freescale/imx8ulp-pinfunc.h
@@ -0,0 +1,978 @@
+/* SPDX-License-Identifier: GPL-2.0+ OR MIT */
+/*
+ * Copyright 2021 NXP
+ */
+
+#ifndef __DTS_IMX8ULP_PINFUNC_H
+#define __DTS_IMX8ULP_PINFUNC_H
+
+/*
+ * The pin function ID is a tuple of
+ * <mux_reg input_reg mux_mode input_val>
+ */
+#define MX8ULP_PAD_PTD0__PTD0 0x0000 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTD0__I2S6_RX_BCLK 0x0000 0x0B44 0x7 0x1
+#define MX8ULP_PAD_PTD0__SDHC0_RESET_B 0x0000 0x0000 0x8 0x0
+#define MX8ULP_PAD_PTD0__FLEXSPI2_B_DQS 0x0000 0x0974 0x9 0x1
+#define MX8ULP_PAD_PTD0__CLKOUT2 0x0000 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTD0__EPDC0_SDCLK_B 0x0000 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTD0__LP_APD_DBG_MUX_0 0x0000 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTD0__CLKOUT1 0x0000 0x0000 0xd 0x0
+#define MX8ULP_PAD_PTD0__DEBUG_MUX0_0 0x0000 0x0000 0xe 0x0
+#define MX8ULP_PAD_PTD0__DEBUG_MUX1_0 0x0000 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTD1__PTD1 0x0004 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTD1__I2S6_RX_FS 0x0004 0x0B48 0x7 0x1
+#define MX8ULP_PAD_PTD1__SDHC0_CMD 0x0004 0x0000 0x8 0x0
+#define MX8ULP_PAD_PTD1__FLEXSPI2_B_DATA7 0x0004 0x0970 0x9 0x1
+#define MX8ULP_PAD_PTD1__EPDC0_SDCLK 0x0004 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTD1__DPI0_PCLK 0x0004 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTD1__LP_APD_DBG_MUX_1 0x0004 0x0000 0xd 0x0
+#define MX8ULP_PAD_PTD1__DEBUG_MUX0_1 0x0004 0x0000 0xe 0x0
+#define MX8ULP_PAD_PTD1__DEBUG_MUX1_1 0x0004 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTD2__PTD2 0x0008 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTD2__I2S6_RXD0 0x0008 0x0B34 0x7 0x1
+#define MX8ULP_PAD_PTD2__SDHC0_CLK 0x0008 0x0000 0x8 0x0
+#define MX8ULP_PAD_PTD2__FLEXSPI2_B_DATA6 0x0008 0x096C 0x9 0x1
+#define MX8ULP_PAD_PTD2__EPDC0_SDLE 0x0008 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTD2__DPI0_HSYNC 0x0008 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTD2__LP_APD_DBG_MUX_2 0x0008 0x0000 0xd 0x0
+#define MX8ULP_PAD_PTD2__DEBUG_MUX0_2 0x0008 0x0000 0xe 0x0
+#define MX8ULP_PAD_PTD2__DEBUG_MUX1_2 0x0008 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTD3__PTD3 0x000C 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTD3__I2S6_RXD1 0x000C 0x0B38 0x7 0x1
+#define MX8ULP_PAD_PTD3__SDHC0_D7 0x000C 0x0000 0x8 0x0
+#define MX8ULP_PAD_PTD3__FLEXSPI2_B_DATA5 0x000C 0x0968 0x9 0x1
+#define MX8ULP_PAD_PTD3__EPDC0_GDSP 0x000C 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTD3__DPI0_VSYNC 0x000C 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTD3__LP_APD_DBG_MUX_3 0x000C 0x0000 0xd 0x0
+#define MX8ULP_PAD_PTD3__DEBUG_MUX0_3 0x000C 0x0000 0xe 0x0
+#define MX8ULP_PAD_PTD3__DEBUG_MUX1_3 0x000C 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTD4__PTD4 0x0010 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTD4__EXT_AUD_MCLK3 0x0010 0x0B14 0x4 0x1
+#define MX8ULP_PAD_PTD4__SDHC0_VS 0x0010 0x0000 0x5 0x0
+#define MX8ULP_PAD_PTD4__TPM8_CH5 0x0010 0x0B2C 0x6 0x1
+#define MX8ULP_PAD_PTD4__I2S6_MCLK 0x0010 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTD4__SDHC0_D6 0x0010 0x0000 0x8 0x0
+#define MX8ULP_PAD_PTD4__FLEXSPI2_B_DATA4 0x0010 0x0964 0x9 0x1
+#define MX8ULP_PAD_PTD4__EPDC0_SDCE0 0x0010 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTD4__DPI0_DE 0x0010 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTD4__LP_APD_DBG_MUX_4 0x0010 0x0000 0xd 0x0
+#define MX8ULP_PAD_PTD4__DEBUG_MUX0_4 0x0010 0x0000 0xe 0x0
+#define MX8ULP_PAD_PTD4__DEBUG_MUX1_4 0x0010 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTD5__PTD5 0x0014 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTD5__SDHC0_CD 0x0014 0x0000 0x5 0x0
+#define MX8ULP_PAD_PTD5__TPM8_CH4 0x0014 0x0B28 0x6 0x1
+#define MX8ULP_PAD_PTD5__I2S6_TX_BCLK 0x0014 0x0B4C 0x7 0x1
+#define MX8ULP_PAD_PTD5__SDHC0_D5 0x0014 0x0000 0x8 0x0
+#define MX8ULP_PAD_PTD5__FLEXSPI2_B_SS0_B 0x0014 0x0000 0x9 0x0
+#define MX8ULP_PAD_PTD5__FLEXSPI2_B_SCLK_B 0x0014 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTD5__EPDC0_D0 0x0014 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTD5__DPI0_D0 0x0014 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTD5__LP_APD_DBG_MUX_5 0x0014 0x0000 0xd 0x0
+#define MX8ULP_PAD_PTD5__DEBUG_MUX0_5 0x0014 0x0000 0xe 0x0
+#define MX8ULP_PAD_PTD5__DEBUG_MUX1_5 0x0014 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTD6__PTD6 0x0018 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTD6__SDHC0_WP 0x0018 0x0000 0x5 0x0
+#define MX8ULP_PAD_PTD6__TPM8_CH3 0x0018 0x0B24 0x6 0x1
+#define MX8ULP_PAD_PTD6__I2S6_TX_FS 0x0018 0x0B50 0x7 0x1
+#define MX8ULP_PAD_PTD6__SDHC0_D4 0x0018 0x0000 0x8 0x0
+#define MX8ULP_PAD_PTD6__FLEXSPI2_B_SCLK 0x0018 0x0978 0x9 0x1
+#define MX8ULP_PAD_PTD6__EPDC0_D1 0x0018 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTD6__DPI0_D1 0x0018 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTD6__LP_APD_DBG_MUX_6 0x0018 0x0000 0xd 0x0
+#define MX8ULP_PAD_PTD6__DEBUG_MUX0_6 0x0018 0x0000 0xe 0x0
+#define MX8ULP_PAD_PTD6__DEBUG_MUX1_6 0x0018 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTD7__PTD7 0x001C 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTD7__TPM8_CH2 0x001C 0x0B20 0x6 0x1
+#define MX8ULP_PAD_PTD7__I2S6_TXD0 0x001C 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTD7__SDHC0_D3 0x001C 0x0000 0x8 0x0
+#define MX8ULP_PAD_PTD7__FLEXSPI2_B_DATA3 0x001C 0x0960 0x9 0x1
+#define MX8ULP_PAD_PTD7__EPDC0_D2 0x001C 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTD7__DPI0_D2 0x001C 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTD7__LP_APD_DBG_MUX_7 0x001C 0x0000 0xd 0x0
+#define MX8ULP_PAD_PTD7__DEBUG_MUX0_7 0x001C 0x0000 0xe 0x0
+#define MX8ULP_PAD_PTD7__DEBUG_MUX1_7 0x001C 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTD8__PTD8 0x0020 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTD8__TPM8_CH1 0x0020 0x0B1C 0x6 0x1
+#define MX8ULP_PAD_PTD8__I2S6_TXD1 0x0020 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTD8__SDHC0_D2 0x0020 0x0000 0x8 0x0
+#define MX8ULP_PAD_PTD8__FLEXSPI2_B_DATA2 0x0020 0x095C 0x9 0x1
+#define MX8ULP_PAD_PTD8__EPDC0_D3 0x0020 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTD8__DPI0_D3 0x0020 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTD8__LP_APD_DBG_MUX_8 0x0020 0x0000 0xe 0x0
+#define MX8ULP_PAD_PTD8__DEBUG_MUX1_8 0x0020 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTD9__PTD9 0x0024 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTD9__TPM8_CLKIN 0x0024 0x0B30 0x6 0x1
+#define MX8ULP_PAD_PTD9__I2S6_TXD2 0x0024 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTD9__SDHC0_D1 0x0024 0x0000 0x8 0x0
+#define MX8ULP_PAD_PTD9__FLEXSPI2_B_DATA1 0x0024 0x0958 0x9 0x1
+#define MX8ULP_PAD_PTD9__EPDC0_D4 0x0024 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTD9__DPI0_D4 0x0024 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTD9__LP_APD_DBG_MUX_9 0x0024 0x0000 0xe 0x0
+#define MX8ULP_PAD_PTD9__DEBUG_MUX1_9 0x0024 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTD10__PTD10 0x0028 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTD10__TPM8_CH0 0x0028 0x0B18 0x6 0x1
+#define MX8ULP_PAD_PTD10__I2S6_TXD3 0x0028 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTD10__SDHC0_D0 0x0028 0x0000 0x8 0x0
+#define MX8ULP_PAD_PTD10__FLEXSPI2_B_DATA0 0x0028 0x0954 0x9 0x1
+#define MX8ULP_PAD_PTD10__EPDC0_D5 0x0028 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTD10__DPI0_D5 0x0028 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTD10__LP_APD_DBG_MUX_10 0x0028 0x0000 0xe 0x0
+#define MX8ULP_PAD_PTD10__DEBUG_MUX1_10 0x0028 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTD11__PTD11 0x002C 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTD11__TPM8_CH5 0x002C 0x0B2C 0x6 0x2
+#define MX8ULP_PAD_PTD11__I2S6_RXD2 0x002C 0x0B3C 0x7 0x1
+#define MX8ULP_PAD_PTD11__SDHC0_DQS 0x002C 0x0000 0x8 0x0
+#define MX8ULP_PAD_PTD11__FLEXSPI2_B_SS0_B 0x002C 0x0000 0x9 0x0
+#define MX8ULP_PAD_PTD11__FLEXSPI2_A_SS1_B 0x002C 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTD11__EPDC0_D6 0x002C 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTD11__DPI0_D6 0x002C 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTD11__LP_APD_DBG_MUX_11 0x002C 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTD12__PTD12 0x0030 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTD12__USB0_ID 0x0030 0x0AC8 0x5 0x1
+#define MX8ULP_PAD_PTD12__SDHC2_D3 0x0030 0x0AA4 0x6 0x1
+#define MX8ULP_PAD_PTD12__I2S7_RX_BCLK 0x0030 0x0B64 0x7 0x1
+#define MX8ULP_PAD_PTD12__SDHC1_DQS 0x0030 0x0A84 0x8 0x1
+#define MX8ULP_PAD_PTD12__FLEXSPI2_A_SS0_B 0x0030 0x0000 0x9 0x0
+#define MX8ULP_PAD_PTD12__FLEXSPI2_B_SS1_B 0x0030 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTD12__EPDC0_D7 0x0030 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTD12__DPI0_D7 0x0030 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTD12__LP_APD_DBG_MUX_12 0x0030 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTD13__PTD13 0x0034 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTD13__SPDIF_IN3 0x0034 0x0B80 0x4 0x1
+#define MX8ULP_PAD_PTD13__USB0_PWR 0x0034 0x0000 0x5 0x0
+#define MX8ULP_PAD_PTD13__SDHC2_D2 0x0034 0x0AA0 0x6 0x1
+#define MX8ULP_PAD_PTD13__I2S7_RX_FS 0x0034 0x0B68 0x7 0x1
+#define MX8ULP_PAD_PTD13__SDHC1_RESET_B 0x0034 0x0000 0x8 0x0
+#define MX8ULP_PAD_PTD13__FLEXSPI2_A_SCLK 0x0034 0x0000 0x9 0x0
+#define MX8ULP_PAD_PTD13__CLKOUT2 0x0034 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTD13__EPDC0_D8 0x0034 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTD13__DPI0_D8 0x0034 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTD13__CLKOUT1 0x0034 0x0000 0xd 0x0
+#define MX8ULP_PAD_PTD13__LP_APD_DBG_MUX_13 0x0034 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTD14__PTD14 0x0038 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTD14__SPDIF_OUT3 0x0038 0x0000 0x4 0x0
+#define MX8ULP_PAD_PTD14__USB0_OC 0x0038 0x0AC0 0x5 0x1
+#define MX8ULP_PAD_PTD14__SDHC2_D1 0x0038 0x0A9C 0x6 0x1
+#define MX8ULP_PAD_PTD14__I2S7_RXD0 0x0038 0x0B54 0x7 0x1
+#define MX8ULP_PAD_PTD14__SDHC1_D7 0x0038 0x0A80 0x8 0x1
+#define MX8ULP_PAD_PTD14__FLEXSPI2_A_DATA3 0x0038 0x0000 0x9 0x0
+#define MX8ULP_PAD_PTD14__TRACE0_D7 0x0038 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTD14__EPDC0_D9 0x0038 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTD14__DPI0_D9 0x0038 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTD14__LP_APD_DBG_MUX_14 0x0038 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTD15__PTD15 0x003C 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTD15__SPDIF_IN2 0x003C 0x0B7C 0x4 0x1
+#define MX8ULP_PAD_PTD15__SDHC1_VS 0x003C 0x0000 0x5 0x0
+#define MX8ULP_PAD_PTD15__SDHC2_D0 0x003C 0x0A98 0x6 0x1
+#define MX8ULP_PAD_PTD15__I2S7_TX_BCLK 0x003C 0x0B6C 0x7 0x1
+#define MX8ULP_PAD_PTD15__SDHC1_D6 0x003C 0x0A7C 0x8 0x1
+#define MX8ULP_PAD_PTD15__FLEXSPI2_A_DATA2 0x003C 0x0000 0x9 0x0
+#define MX8ULP_PAD_PTD15__TRACE0_D6 0x003C 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTD15__EPDC0_D10 0x003C 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTD15__DPI0_D10 0x003C 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTD15__LP_APD_DBG_MUX_15 0x003C 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTD16__PTD16 0x0040 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTD16__FXIO1_D31 0x0040 0x08A0 0x2 0x1
+#define MX8ULP_PAD_PTD16__LPSPI4_PCS1 0x0040 0x08F8 0x3 0x1
+#define MX8ULP_PAD_PTD16__SPDIF_OUT2 0x0040 0x0000 0x4 0x0
+#define MX8ULP_PAD_PTD16__SDHC1_CD 0x0040 0x0A58 0x5 0x1
+#define MX8ULP_PAD_PTD16__SDHC2_CLK 0x0040 0x0A90 0x6 0x1
+#define MX8ULP_PAD_PTD16__I2S7_TX_FS 0x0040 0x0B70 0x7 0x1
+#define MX8ULP_PAD_PTD16__SDHC1_D5 0x0040 0x0A78 0x8 0x1
+#define MX8ULP_PAD_PTD16__FLEXSPI2_A_DATA1 0x0040 0x0000 0x9 0x0
+#define MX8ULP_PAD_PTD16__TRACE0_D5 0x0040 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTD16__EPDC0_D11 0x0040 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTD16__DPI0_D11 0x0040 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTD16__LP_APD_DBG_MUX_16 0x0040 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTD17__PTD17 0x0044 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTD17__FXIO1_D30 0x0044 0x089C 0x2 0x1
+#define MX8ULP_PAD_PTD17__LPSPI4_PCS2 0x0044 0x08FC 0x3 0x1
+#define MX8ULP_PAD_PTD17__EXT_AUD_MCLK3 0x0044 0x0B14 0x4 0x2
+#define MX8ULP_PAD_PTD17__SDHC1_WP 0x0044 0x0A88 0x5 0x1
+#define MX8ULP_PAD_PTD17__SDHC2_CMD 0x0044 0x0A94 0x6 0x1
+#define MX8ULP_PAD_PTD17__I2S7_TXD0 0x0044 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTD17__SDHC1_D4 0x0044 0x0A74 0x8 0x1
+#define MX8ULP_PAD_PTD17__FLEXSPI2_A_DATA0 0x0044 0x0000 0x9 0x0
+#define MX8ULP_PAD_PTD17__TRACE0_D4 0x0044 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTD17__EPDC0_D12 0x0044 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTD17__DPI0_D12 0x0044 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTD17__LP_APD_DBG_MUX_17 0x0044 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTD18__PTD18 0x0048 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTD18__FXIO1_D29 0x0048 0x0894 0x2 0x1
+#define MX8ULP_PAD_PTD18__LPSPI4_PCS3 0x0048 0x0900 0x3 0x1
+#define MX8ULP_PAD_PTD18__SPDIF_CLK 0x0048 0x0000 0x4 0x0
+#define MX8ULP_PAD_PTD18__EXT_AUD_MCLK3 0x0048 0x0B14 0x5 0x3
+#define MX8ULP_PAD_PTD18__TPM8_CH0 0x0048 0x0B18 0x6 0x2
+#define MX8ULP_PAD_PTD18__I2S7_MCLK 0x0048 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTD18__SDHC1_D3 0x0048 0x0A70 0x8 0x1
+#define MX8ULP_PAD_PTD18__FLEXSPI2_A_DQS 0x0048 0x0000 0x9 0x0
+#define MX8ULP_PAD_PTD18__TRACE0_D3 0x0048 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTD18__EPDC0_D13 0x0048 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTD18__DPI0_D13 0x0048 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTD18__LP_APD_DBG_MUX_18 0x0048 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTD19__PTD19 0x004C 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTD19__FXIO1_D28 0x004C 0x0890 0x2 0x1
+#define MX8ULP_PAD_PTD19__SPDIF_IN0 0x004C 0x0B74 0x4 0x1
+#define MX8ULP_PAD_PTD19__TPM8_CH1 0x004C 0x0B1C 0x6 0x2
+#define MX8ULP_PAD_PTD19__I2S6_RXD3 0x004C 0x0B40 0x7 0x1
+#define MX8ULP_PAD_PTD19__SDHC1_D2 0x004C 0x0A6C 0x8 0x1
+#define MX8ULP_PAD_PTD19__FLEXSPI2_A_DATA7 0x004C 0x0000 0x9 0x0
+#define MX8ULP_PAD_PTD19__TRACE0_D2 0x004C 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTD19__EPDC0_D14 0x004C 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTD19__DPI0_D14 0x004C 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTD19__LP_APD_DBG_MUX_19 0x004C 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTD20__PTD20 0x0050 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTD20__FXIO1_D27 0x0050 0x088C 0x2 0x1
+#define MX8ULP_PAD_PTD20__LPSPI4_SIN 0x0050 0x0908 0x3 0x1
+#define MX8ULP_PAD_PTD20__SPDIF_OUT0 0x0050 0x0000 0x4 0x0
+#define MX8ULP_PAD_PTD20__TPM8_CLKIN 0x0050 0x0B30 0x6 0x2
+#define MX8ULP_PAD_PTD20__I2S7_RXD1 0x0050 0x0B58 0x7 0x1
+#define MX8ULP_PAD_PTD20__SDHC1_D1 0x0050 0x0A68 0x8 0x1
+#define MX8ULP_PAD_PTD20__FLEXSPI2_A_DATA6 0x0050 0x0000 0x9 0x0
+#define MX8ULP_PAD_PTD20__TRACE0_D1 0x0050 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTD20__EPDC0_D15 0x0050 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTD20__DPI0_D15 0x0050 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTD20__LP_APD_DBG_MUX_20 0x0050 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTD21__PTD21 0x0054 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTD21__FXIO1_D26 0x0054 0x0888 0x2 0x1
+#define MX8ULP_PAD_PTD21__LPSPI4_SOUT 0x0054 0x090C 0x3 0x1
+#define MX8ULP_PAD_PTD21__SPDIF_IN1 0x0054 0x0B78 0x4 0x1
+#define MX8ULP_PAD_PTD21__USB1_PWR 0x0054 0x0000 0x5 0x0
+#define MX8ULP_PAD_PTD21__TPM8_CH2 0x0054 0x0B20 0x6 0x2
+#define MX8ULP_PAD_PTD21__I2S7_TXD1 0x0054 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTD21__SDHC1_D0 0x0054 0x0A64 0x8 0x1
+#define MX8ULP_PAD_PTD21__FLEXSPI2_A_DATA5 0x0054 0x0000 0x9 0x0
+#define MX8ULP_PAD_PTD21__TRACE0_D0 0x0054 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTD21__DPI0_D16 0x0054 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTD21__WDOG5_RST 0x0054 0x0000 0xd 0x0
+#define MX8ULP_PAD_PTD21__LP_APD_DBG_MUX_21 0x0054 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTD22__PTD22 0x0058 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTD22__FXIO1_D25 0x0058 0x0884 0x2 0x1
+#define MX8ULP_PAD_PTD22__LPSPI4_SCK 0x0058 0x0904 0x3 0x1
+#define MX8ULP_PAD_PTD22__SPDIF_OUT1 0x0058 0x0000 0x4 0x0
+#define MX8ULP_PAD_PTD22__USB1_OC 0x0058 0x0AC4 0x5 0x1
+#define MX8ULP_PAD_PTD22__TPM8_CH3 0x0058 0x0B24 0x6 0x2
+#define MX8ULP_PAD_PTD22__I2S7_TXD2 0x0058 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTD22__SDHC1_CLK 0x0058 0x0A5C 0x8 0x1
+#define MX8ULP_PAD_PTD22__FLEXSPI2_A_DATA4 0x0058 0x0000 0x9 0x0
+#define MX8ULP_PAD_PTD22__TRACE0_CLKOUT 0x0058 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTD22__DPI0_D17 0x0058 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTD22__LP_APD_DBG_MUX_22 0x0058 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTD23__PTD23 0x005C 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTD23__FXIO1_D24 0x005C 0x0880 0x2 0x1
+#define MX8ULP_PAD_PTD23__LPSPI4_PCS0 0x005C 0x08F4 0x3 0x1
+#define MX8ULP_PAD_PTD23__USB1_ID 0x005C 0x0ACC 0x5 0x1
+#define MX8ULP_PAD_PTD23__TPM8_CH4 0x005C 0x0B28 0x6 0x2
+#define MX8ULP_PAD_PTD23__I2S7_TXD3 0x005C 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTD23__SDHC1_CMD 0x005C 0x0A60 0x8 0x1
+#define MX8ULP_PAD_PTD23__FLEXSPI2_A_SS0_B 0x005C 0x0000 0x9 0x0
+#define MX8ULP_PAD_PTD23__FLEXSPI2_A_SCLK_B 0x005C 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTD23__DPI0_D18 0x005C 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTD23__LP_APD_DBG_MUX_23 0x005C 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTE0__PTE0 0x0080 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTE0__FXIO1_D23 0x0080 0x087C 0x2 0x1
+#define MX8ULP_PAD_PTE0__SPDIF_IN3 0x0080 0x0B80 0x3 0x2
+#define MX8ULP_PAD_PTE0__LPUART4_CTS_B 0x0080 0x08DC 0x4 0x1
+#define MX8ULP_PAD_PTE0__LPI2C4_SCL 0x0080 0x08C8 0x5 0x1
+#define MX8ULP_PAD_PTE0__TPM8_CLKIN 0x0080 0x0B30 0x6 0x3
+#define MX8ULP_PAD_PTE0__I2S7_RXD2 0x0080 0x0B5C 0x7 0x1
+#define MX8ULP_PAD_PTE0__SDHC2_D1 0x0080 0x0A9C 0x8 0x2
+#define MX8ULP_PAD_PTE0__FLEXSPI2_B_DQS 0x0080 0x0974 0x9 0x2
+#define MX8ULP_PAD_PTE0__ENET0_CRS 0x0080 0x0AE8 0xa 0x1
+#define MX8ULP_PAD_PTE0__DBI0_WRX 0x0080 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTE0__DPI0_D19 0x0080 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTE0__WUU1_P0 0x0080 0x0000 0xd 0x0
+#define MX8ULP_PAD_PTE0__DEBUG_MUX0_8 0x0080 0x0000 0xe 0x0
+#define MX8ULP_PAD_PTE0__DEBUG_MUX1_11 0x0080 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTE1__PTE1 0x0084 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTE1__FXIO1_D22 0x0084 0x0878 0x2 0x1
+#define MX8ULP_PAD_PTE1__SPDIF_OUT3 0x0084 0x0000 0x3 0x0
+#define MX8ULP_PAD_PTE1__LPUART4_RTS_B 0x0084 0x0000 0x4 0x0
+#define MX8ULP_PAD_PTE1__LPI2C4_SDA 0x0084 0x08CC 0x5 0x1
+#define MX8ULP_PAD_PTE1__TPM8_CH0 0x0084 0x0B18 0x6 0x3
+#define MX8ULP_PAD_PTE1__I2S7_RXD3 0x0084 0x0B60 0x7 0x1
+#define MX8ULP_PAD_PTE1__SDHC2_D0 0x0084 0x0A98 0x8 0x2
+#define MX8ULP_PAD_PTE1__FLEXSPI2_B_DATA7 0x0084 0x0970 0x9 0x2
+#define MX8ULP_PAD_PTE1__ENET0_COL 0x0084 0x0AE4 0xa 0x1
+#define MX8ULP_PAD_PTE1__DBI0_CSX 0x0084 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTE1__DPI0_D20 0x0084 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTE1__WUU1_P1 0x0084 0x0000 0xd 0x0
+#define MX8ULP_PAD_PTE1__DEBUG_MUX0_9 0x0084 0x0000 0xe 0x0
+#define MX8ULP_PAD_PTE1__DEBUG_MUX1_12 0x0084 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTE2__PTE2 0x0088 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTE2__FXIO1_D21 0x0088 0x0874 0x2 0x1
+#define MX8ULP_PAD_PTE2__SPDIF_IN2 0x0088 0x0B7C 0x3 0x2
+#define MX8ULP_PAD_PTE2__LPUART4_TX 0x0088 0x08E4 0x4 0x1
+#define MX8ULP_PAD_PTE2__LPI2C4_HREQ 0x0088 0x08C4 0x5 0x1
+#define MX8ULP_PAD_PTE2__TPM8_CH1 0x0088 0x0B1C 0x6 0x3
+#define MX8ULP_PAD_PTE2__EXT_AUD_MCLK3 0x0088 0x0B14 0x7 0x4
+#define MX8ULP_PAD_PTE2__SDHC2_CLK 0x0088 0x0A90 0x8 0x2
+#define MX8ULP_PAD_PTE2__FLEXSPI2_B_DATA6 0x0088 0x096C 0x9 0x2
+#define MX8ULP_PAD_PTE2__ENET0_TXER 0x0088 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTE2__DBI0_DCX 0x0088 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTE2__DPI0_D21 0x0088 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTE2__LP_HV_DBG_MUX_0 0x0088 0x0000 0xd 0x0
+#define MX8ULP_PAD_PTE2__DEBUG_MUX0_10 0x0088 0x0000 0xe 0x0
+#define MX8ULP_PAD_PTE2__DEBUG_MUX1_13 0x0088 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTE3__PTE3 0x008C 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTE3__FXIO1_D20 0x008C 0x0870 0x2 0x1
+#define MX8ULP_PAD_PTE3__SPDIF_OUT2 0x008C 0x0000 0x3 0x0
+#define MX8ULP_PAD_PTE3__LPUART4_RX 0x008C 0x08E0 0x4 0x1
+#define MX8ULP_PAD_PTE3__TPM8_CH2 0x008C 0x0B20 0x6 0x3
+#define MX8ULP_PAD_PTE3__I2S6_MCLK 0x008C 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTE3__SDHC2_CMD 0x008C 0x0A94 0x8 0x2
+#define MX8ULP_PAD_PTE3__FLEXSPI2_B_DATA5 0x008C 0x0968 0x9 0x2
+#define MX8ULP_PAD_PTE3__ENET0_TXCLK 0x008C 0x0B10 0xa 0x1
+#define MX8ULP_PAD_PTE3__DBI0_RWX 0x008C 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTE3__DPI0_D22 0x008C 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTE3__WUU1_P2 0x008C 0x0000 0xd 0x0
+#define MX8ULP_PAD_PTE3__DEBUG_MUX0_11 0x008C 0x0000 0xe 0x0
+#define MX8ULP_PAD_PTE3__DEBUG_MUX1_14 0x008C 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTE4__PTE4 0x0090 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTE4__FXIO1_D19 0x0090 0x0868 0x2 0x1
+#define MX8ULP_PAD_PTE4__SPDIF_CLK 0x0090 0x0000 0x3 0x0
+#define MX8ULP_PAD_PTE4__LPUART5_CTS_B 0x0090 0x08E8 0x4 0x1
+#define MX8ULP_PAD_PTE4__LPI2C5_SCL 0x0090 0x08D4 0x5 0x1
+#define MX8ULP_PAD_PTE4__TPM8_CH3 0x0090 0x0B24 0x6 0x3
+#define MX8ULP_PAD_PTE4__I2S6_RX_BCLK 0x0090 0x0B44 0x7 0x2
+#define MX8ULP_PAD_PTE4__SDHC2_D3 0x0090 0x0AA4 0x8 0x2
+#define MX8ULP_PAD_PTE4__FLEXSPI2_B_DATA4 0x0090 0x0964 0x9 0x2
+#define MX8ULP_PAD_PTE4__ENET0_TXD3 0x0090 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTE4__DBI0_E 0x0090 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTE4__DPI0_D23 0x0090 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTE4__WUU1_P3 0x0090 0x0000 0xd 0x0
+#define MX8ULP_PAD_PTE4__DEBUG_MUX0_12 0x0090 0x0000 0xe 0x0
+#define MX8ULP_PAD_PTE4__DEBUG_MUX1_15 0x0090 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTE5__PTE5 0x0094 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTE5__FXIO1_D18 0x0094 0x0864 0x2 0x1
+#define MX8ULP_PAD_PTE5__SPDIF_IN0 0x0094 0x0B74 0x3 0x2
+#define MX8ULP_PAD_PTE5__LPUART5_RTS_B 0x0094 0x0000 0x4 0x0
+#define MX8ULP_PAD_PTE5__LPI2C5_SDA 0x0094 0x08D8 0x5 0x1
+#define MX8ULP_PAD_PTE5__TPM8_CH4 0x0094 0x0B28 0x6 0x3
+#define MX8ULP_PAD_PTE5__I2S6_RX_FS 0x0094 0x0B48 0x7 0x2
+#define MX8ULP_PAD_PTE5__SDHC2_D2 0x0094 0x0AA0 0x8 0x2
+#define MX8ULP_PAD_PTE5__FLEXSPI2_B_SS0_B 0x0094 0x0000 0x9 0x0
+#define MX8ULP_PAD_PTE5__ENET0_TXD2 0x0094 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTE5__DBI0_D0 0x0094 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTE5__LP_HV_DBG_MUX_1 0x0094 0x0000 0xd 0x0
+#define MX8ULP_PAD_PTE5__DEBUG_MUX0_13 0x0094 0x0000 0xe 0x0
+#define MX8ULP_PAD_PTE5__DEBUG_MUX1_16 0x0094 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTE6__PTE6 0x0098 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTE6__FXIO1_D17 0x0098 0x0860 0x2 0x1
+#define MX8ULP_PAD_PTE6__SPDIF_OUT0 0x0098 0x0000 0x3 0x0
+#define MX8ULP_PAD_PTE6__LPUART5_TX 0x0098 0x08F0 0x4 0x1
+#define MX8ULP_PAD_PTE6__LPI2C5_HREQ 0x0098 0x08D0 0x5 0x1
+#define MX8ULP_PAD_PTE6__TPM8_CH5 0x0098 0x0B2C 0x6 0x3
+#define MX8ULP_PAD_PTE6__I2S6_RXD0 0x0098 0x0B34 0x7 0x2
+#define MX8ULP_PAD_PTE6__SDHC2_D4 0x0098 0x0AA8 0x8 0x1
+#define MX8ULP_PAD_PTE6__FLEXSPI2_B_SCLK 0x0098 0x0978 0x9 0x2
+#define MX8ULP_PAD_PTE6__ENET0_RXCLK 0x0098 0x0B0C 0xa 0x1
+#define MX8ULP_PAD_PTE6__DBI0_D1 0x0098 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTE6__LP_HV_DBG_MUX_2 0x0098 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTE6__WDOG5_RST 0x0098 0x0000 0xd 0x0
+#define MX8ULP_PAD_PTE6__DEBUG_MUX0_14 0x0098 0x0000 0xe 0x0
+#define MX8ULP_PAD_PTE6__DEBUG_MUX1_17 0x0098 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTE7__PTE7 0x009C 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTE7__FXIO1_D16 0x009C 0x085C 0x2 0x1
+#define MX8ULP_PAD_PTE7__SPDIF_IN1 0x009C 0x0B78 0x3 0x2
+#define MX8ULP_PAD_PTE7__LPUART5_RX 0x009C 0x08EC 0x4 0x1
+#define MX8ULP_PAD_PTE7__LPI2C6_HREQ 0x009C 0x09B4 0x5 0x1
+#define MX8ULP_PAD_PTE7__TPM4_CLKIN 0x009C 0x081C 0x6 0x1
+#define MX8ULP_PAD_PTE7__I2S6_RXD1 0x009C 0x0B38 0x7 0x2
+#define MX8ULP_PAD_PTE7__SDHC2_D5 0x009C 0x0AAC 0x8 0x1
+#define MX8ULP_PAD_PTE7__FLEXSPI2_B_DATA3 0x009C 0x0960 0x9 0x2
+#define MX8ULP_PAD_PTE7__ENET0_RXD3 0x009C 0x0B04 0xa 0x1
+#define MX8ULP_PAD_PTE7__DBI0_D2 0x009C 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTE7__EPDC0_BDR1 0x009C 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTE7__WUU1_P4 0x009C 0x0000 0xd 0x0
+#define MX8ULP_PAD_PTE7__DEBUG_MUX0_15 0x009C 0x0000 0xe 0x0
+#define MX8ULP_PAD_PTE7__DEBUG_MUX1_18 0x009C 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTE8__PTE8 0x00A0 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTE8__FXIO1_D15 0x00A0 0x0858 0x2 0x1
+#define MX8ULP_PAD_PTE8__LPSPI4_PCS1 0x00A0 0x08F8 0x3 0x2
+#define MX8ULP_PAD_PTE8__LPUART6_CTS_B 0x00A0 0x09CC 0x4 0x1
+#define MX8ULP_PAD_PTE8__LPI2C6_SCL 0x00A0 0x09B8 0x5 0x1
+#define MX8ULP_PAD_PTE8__TPM4_CH0 0x00A0 0x0804 0x6 0x1
+#define MX8ULP_PAD_PTE8__I2S6_RXD2 0x00A0 0x0B3C 0x7 0x2
+#define MX8ULP_PAD_PTE8__SDHC2_D6 0x00A0 0x0AB0 0x8 0x1
+#define MX8ULP_PAD_PTE8__FLEXSPI2_B_DATA2 0x00A0 0x095C 0x9 0x2
+#define MX8ULP_PAD_PTE8__ENET0_RXD2 0x00A0 0x0B00 0xa 0x1
+#define MX8ULP_PAD_PTE8__DBI0_D3 0x00A0 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTE8__EPDC0_BDR0 0x00A0 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTE8__LP_HV_DBG_MUX_3 0x00A0 0x0000 0xe 0x0
+#define MX8ULP_PAD_PTE8__DEBUG_MUX1_19 0x00A0 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTE9__PTE9 0x00A4 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTE9__FXIO1_D14 0x00A4 0x0854 0x2 0x1
+#define MX8ULP_PAD_PTE9__LPSPI4_PCS2 0x00A4 0x08FC 0x3 0x2
+#define MX8ULP_PAD_PTE9__LPUART6_RTS_B 0x00A4 0x0000 0x4 0x0
+#define MX8ULP_PAD_PTE9__LPI2C6_SDA 0x00A4 0x09BC 0x5 0x1
+#define MX8ULP_PAD_PTE9__TPM4_CH1 0x00A4 0x0808 0x6 0x1
+#define MX8ULP_PAD_PTE9__I2S6_RXD3 0x00A4 0x0B40 0x7 0x2
+#define MX8ULP_PAD_PTE9__SDHC2_D7 0x00A4 0x0AB4 0x8 0x1
+#define MX8ULP_PAD_PTE9__FLEXSPI2_B_DATA1 0x00A4 0x0958 0x9 0x2
+#define MX8ULP_PAD_PTE9__ENET0_1588_TMR3 0x00A4 0x0AE0 0xa 0x1
+#define MX8ULP_PAD_PTE9__DBI0_D4 0x00A4 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTE9__EPDC0_VCOM1 0x00A4 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTE9__LP_HV_DBG_MUX_4 0x00A4 0x0000 0xe 0x0
+#define MX8ULP_PAD_PTE9__DEBUG_MUX1_20 0x00A4 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTE10__PTE10 0x00A8 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTE10__FXIO1_D13 0x00A8 0x0850 0x2 0x1
+#define MX8ULP_PAD_PTE10__LPSPI4_PCS3 0x00A8 0x0900 0x3 0x2
+#define MX8ULP_PAD_PTE10__LPUART6_TX 0x00A8 0x09D4 0x4 0x1
+#define MX8ULP_PAD_PTE10__I3C2_SCL 0x00A8 0x08BC 0x5 0x1
+#define MX8ULP_PAD_PTE10__TPM4_CH2 0x00A8 0x080C 0x6 0x1
+#define MX8ULP_PAD_PTE10__I2S6_TX_BCLK 0x00A8 0x0B4C 0x7 0x2
+#define MX8ULP_PAD_PTE10__SDHC2_DQS 0x00A8 0x0AB8 0x8 0x1
+#define MX8ULP_PAD_PTE10__FLEXSPI2_B_DATA0 0x00A8 0x0954 0x9 0x2
+#define MX8ULP_PAD_PTE10__ENET0_1588_TMR2 0x00A8 0x0ADC 0xa 0x1
+#define MX8ULP_PAD_PTE10__DBI0_D5 0x00A8 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTE10__EPDC0_VCOM0 0x00A8 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTE10__LP_HV_DBG_MUX_5 0x00A8 0x0000 0xe 0x0
+#define MX8ULP_PAD_PTE10__DEBUG_MUX1_21 0x00A8 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTE11__PTE11 0x00AC 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTE11__FXIO1_D12 0x00AC 0x084C 0x2 0x1
+#define MX8ULP_PAD_PTE11__SPDIF_OUT1 0x00AC 0x0000 0x3 0x0
+#define MX8ULP_PAD_PTE11__LPUART6_RX 0x00AC 0x09D0 0x4 0x1
+#define MX8ULP_PAD_PTE11__I3C2_SDA 0x00AC 0x08C0 0x5 0x1
+#define MX8ULP_PAD_PTE11__TPM4_CH3 0x00AC 0x0810 0x6 0x1
+#define MX8ULP_PAD_PTE11__I2S6_TX_FS 0x00AC 0x0B50 0x7 0x2
+#define MX8ULP_PAD_PTE11__FLEXSPI2_B_SCLK_B 0x00AC 0x0000 0x8 0x0
+#define MX8ULP_PAD_PTE11__FLEXSPI2_B_SS0_B 0x00AC 0x0000 0x9 0x0
+#define MX8ULP_PAD_PTE11__ENET0_1588_TMR1 0x00AC 0x0AD8 0xa 0x1
+#define MX8ULP_PAD_PTE11__DBI0_D6 0x00AC 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTE11__EPDC0_PWRCTRL0 0x00AC 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTE11__LP_HV_DBG_MUX_6 0x00AC 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTE12__PTE12 0x00B0 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTE12__FXIO1_D11 0x00B0 0x0848 0x2 0x1
+#define MX8ULP_PAD_PTE12__LPSPI4_SIN 0x00B0 0x0908 0x3 0x2
+#define MX8ULP_PAD_PTE12__LPUART7_CTS_B 0x00B0 0x09D8 0x4 0x1
+#define MX8ULP_PAD_PTE12__LPI2C7_SCL 0x00B0 0x09C4 0x5 0x1
+#define MX8ULP_PAD_PTE12__TPM4_CH4 0x00B0 0x0814 0x6 0x1
+#define MX8ULP_PAD_PTE12__I2S6_TXD0 0x00B0 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTE12__SDHC2_RESET_B 0x00B0 0x0000 0x8 0x0
+#define MX8ULP_PAD_PTE12__FLEXSPI2_B_SS1_B 0x00B0 0x0000 0x9 0x0
+#define MX8ULP_PAD_PTE12__ENET0_1588_TMR0 0x00B0 0x0AD4 0xa 0x1
+#define MX8ULP_PAD_PTE12__DBI0_D7 0x00B0 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTE12__EPDC0_PWRCTRL1 0x00B0 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTE12__WUU1_P5 0x00B0 0x0000 0xd 0x0
+#define MX8ULP_PAD_PTE13__PTE13 0x00B4 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTE13__FXIO1_D10 0x00B4 0x0844 0x2 0x1
+#define MX8ULP_PAD_PTE13__LPSPI4_SOUT 0x00B4 0x090C 0x3 0x2
+#define MX8ULP_PAD_PTE13__LPUART7_RTS_B 0x00B4 0x0000 0x4 0x0
+#define MX8ULP_PAD_PTE13__LPI2C7_SDA 0x00B4 0x09C8 0x5 0x1
+#define MX8ULP_PAD_PTE13__TPM4_CH5 0x00B4 0x0818 0x6 0x1
+#define MX8ULP_PAD_PTE13__I2S6_TXD1 0x00B4 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTE13__SDHC1_WP 0x00B4 0x0A88 0x8 0x2
+#define MX8ULP_PAD_PTE13__ENET0_1588_CLKIN 0x00B4 0x0AD0 0xa 0x1
+#define MX8ULP_PAD_PTE13__DBI0_D8 0x00B4 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTE13__EPDC0_PWRCTRL2 0x00B4 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTE13__LP_HV_DBG_MUX_7 0x00B4 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTE14__PTE14 0x00B8 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTE14__FXIO1_D9 0x00B8 0x08B8 0x2 0x1
+#define MX8ULP_PAD_PTE14__LPSPI4_SCK 0x00B8 0x0904 0x3 0x2
+#define MX8ULP_PAD_PTE14__LPUART7_TX 0x00B8 0x09E0 0x4 0x1
+#define MX8ULP_PAD_PTE14__LPI2C7_HREQ 0x00B8 0x09C0 0x5 0x1
+#define MX8ULP_PAD_PTE14__TPM5_CLKIN 0x00B8 0x0838 0x6 0x1
+#define MX8ULP_PAD_PTE14__I2S6_TXD2 0x00B8 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTE14__SDHC1_CD 0x00B8 0x0A58 0x8 0x2
+#define MX8ULP_PAD_PTE14__ENET0_MDIO 0x00B8 0x0AF0 0xa 0x1
+#define MX8ULP_PAD_PTE14__DBI0_D9 0x00B8 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTE14__EPDC0_PWRCTRL3 0x00B8 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTE14__LP_HV_DBG_MUX_8 0x00B8 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTE15__PTE15 0x00BC 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTE15__FXIO1_D8 0x00BC 0x08B4 0x2 0x1
+#define MX8ULP_PAD_PTE15__LPSPI4_PCS0 0x00BC 0x08F4 0x3 0x2
+#define MX8ULP_PAD_PTE15__LPUART7_RX 0x00BC 0x09DC 0x4 0x1
+#define MX8ULP_PAD_PTE15__I3C2_PUR 0x00BC 0x0000 0x5 0x0
+#define MX8ULP_PAD_PTE15__TPM5_CH0 0x00BC 0x0820 0x6 0x1
+#define MX8ULP_PAD_PTE15__I2S6_TXD3 0x00BC 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTE15__MQS1_LEFT 0x00BC 0x0000 0x8 0x0
+#define MX8ULP_PAD_PTE15__ENET0_MDC 0x00BC 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTE15__DBI0_D10 0x00BC 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTE15__EPDC0_PWRCOM 0x00BC 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTE15__WUU1_P6 0x00BC 0x0000 0xd 0x0
+#define MX8ULP_PAD_PTE16__PTE16 0x00C0 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTE16__FXIO1_D7 0x00C0 0x08B0 0x2 0x1
+#define MX8ULP_PAD_PTE16__LPSPI5_PCS1 0x00C0 0x0914 0x3 0x1
+#define MX8ULP_PAD_PTE16__LPUART4_CTS_B 0x00C0 0x08DC 0x4 0x2
+#define MX8ULP_PAD_PTE16__LPI2C4_SCL 0x00C0 0x08C8 0x5 0x2
+#define MX8ULP_PAD_PTE16__TPM5_CH1 0x00C0 0x0824 0x6 0x1
+#define MX8ULP_PAD_PTE16__MQS1_LEFT 0x00C0 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTE16__MQS1_RIGHT 0x00C0 0x0000 0x8 0x0
+#define MX8ULP_PAD_PTE16__USB0_ID 0x00C0 0x0AC8 0x9 0x2
+#define MX8ULP_PAD_PTE16__ENET0_TXEN 0x00C0 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTE16__DBI0_D11 0x00C0 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTE16__EPDC0_PWRIRQ 0x00C0 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTE16__WDOG3_RST 0x00C0 0x0000 0xd 0x0
+#define MX8ULP_PAD_PTE16__LP_HV_DBG_MUX_9 0x00C0 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTE17__PTE17 0x00C4 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTE17__FXIO1_D6 0x00C4 0x08AC 0x2 0x1
+#define MX8ULP_PAD_PTE17__LPSPI5_PCS2 0x00C4 0x0918 0x3 0x1
+#define MX8ULP_PAD_PTE17__LPUART4_RTS_B 0x00C4 0x0000 0x4 0x0
+#define MX8ULP_PAD_PTE17__LPI2C4_SDA 0x00C4 0x08CC 0x5 0x2
+#define MX8ULP_PAD_PTE17__MQS1_RIGHT 0x00C4 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTE17__SDHC1_VS 0x00C4 0x0000 0x8 0x0
+#define MX8ULP_PAD_PTE17__USB0_PWR 0x00C4 0x0000 0x9 0x0
+#define MX8ULP_PAD_PTE17__ENET0_RXER 0x00C4 0x0B08 0xa 0x1
+#define MX8ULP_PAD_PTE17__DBI0_D12 0x00C4 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTE17__EPDC0_PWRSTAT 0x00C4 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTE17__LP_HV_DBG_MUX_10 0x00C4 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTE18__PTE18 0x00C8 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTE18__FXIO1_D5 0x00C8 0x08A8 0x2 0x1
+#define MX8ULP_PAD_PTE18__LPSPI5_PCS3 0x00C8 0x091C 0x3 0x1
+#define MX8ULP_PAD_PTE18__LPUART4_TX 0x00C8 0x08E4 0x4 0x2
+#define MX8ULP_PAD_PTE18__LPI2C4_HREQ 0x00C8 0x08C4 0x5 0x2
+#define MX8ULP_PAD_PTE18__I2S7_TX_BCLK 0x00C8 0x0B6C 0x7 0x2
+#define MX8ULP_PAD_PTE18__USB0_OC 0x00C8 0x0AC0 0x9 0x2
+#define MX8ULP_PAD_PTE18__ENET0_CRS_DV 0x00C8 0x0AEC 0xa 0x1
+#define MX8ULP_PAD_PTE18__DBI0_D13 0x00C8 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTE18__EPDC0_PWRWAKE 0x00C8 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTE18__LP_HV_DBG_MUX_11 0x00C8 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTE19__PTE19 0x00CC 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTE19__FXIO1_D4 0x00CC 0x08A4 0x2 0x1
+#define MX8ULP_PAD_PTE19__LPUART4_RX 0x00CC 0x08E0 0x4 0x2
+#define MX8ULP_PAD_PTE19__LPI2C5_HREQ 0x00CC 0x08D0 0x5 0x2
+#define MX8ULP_PAD_PTE19__I3C2_PUR 0x00CC 0x0000 0x6 0x0
+#define MX8ULP_PAD_PTE19__I2S7_TX_FS 0x00CC 0x0B70 0x7 0x2
+#define MX8ULP_PAD_PTE19__USB1_PWR 0x00CC 0x0000 0x9 0x0
+#define MX8ULP_PAD_PTE19__ENET0_REFCLK 0x00CC 0x0AF4 0xa 0x1
+#define MX8ULP_PAD_PTE19__DBI0_D14 0x00CC 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTE19__EPDC0_GDCLK 0x00CC 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTE19__WUU1_P7 0x00CC 0x0000 0xd 0x0
+#define MX8ULP_PAD_PTE20__PTE20 0x00D0 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTE20__FXIO1_D3 0x00D0 0x0898 0x2 0x1
+#define MX8ULP_PAD_PTE20__LPSPI5_SIN 0x00D0 0x0924 0x3 0x1
+#define MX8ULP_PAD_PTE20__LPUART5_CTS_B 0x00D0 0x08E8 0x4 0x2
+#define MX8ULP_PAD_PTE20__LPI2C5_SCL 0x00D0 0x08D4 0x5 0x2
+#define MX8ULP_PAD_PTE20__I2S7_TXD0 0x00D0 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTE20__USB1_OC 0x00D0 0x0AC4 0x9 0x2
+#define MX8ULP_PAD_PTE20__ENET0_RXD1 0x00D0 0x0AFC 0xa 0x1
+#define MX8ULP_PAD_PTE20__DBI0_D15 0x00D0 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTE20__EPDC0_GDOE 0x00D0 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTE20__LP_HV_DBG_MUX_12 0x00D0 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTE21__PTE21 0x00D4 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTE21__FXIO1_D2 0x00D4 0x086C 0x2 0x1
+#define MX8ULP_PAD_PTE21__LPSPI5_SOUT 0x00D4 0x0928 0x3 0x1
+#define MX8ULP_PAD_PTE21__LPUART5_RTS_B 0x00D4 0x0000 0x4 0x0
+#define MX8ULP_PAD_PTE21__LPI2C5_SDA 0x00D4 0x08D8 0x5 0x2
+#define MX8ULP_PAD_PTE21__TPM6_CLKIN 0x00D4 0x0994 0x6 0x1
+#define MX8ULP_PAD_PTE21__I2S7_TXD1 0x00D4 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTE21__USB1_ID 0x00D4 0x0ACC 0x9 0x2
+#define MX8ULP_PAD_PTE21__ENET0_RXD0 0x00D4 0x0AF8 0xa 0x1
+#define MX8ULP_PAD_PTE21__EPDC0_GDRL 0x00D4 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTE21__WDOG4_RST 0x00D4 0x0000 0xd 0x0
+#define MX8ULP_PAD_PTE21__LP_HV_DBG_MUX_13 0x00D4 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTE22__PTE22 0x00D8 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTE22__FXIO1_D1 0x00D8 0x0840 0x2 0x1
+#define MX8ULP_PAD_PTE22__LPSPI5_SCK 0x00D8 0x0920 0x3 0x1
+#define MX8ULP_PAD_PTE22__LPUART5_TX 0x00D8 0x08F0 0x4 0x2
+#define MX8ULP_PAD_PTE22__I3C2_SCL 0x00D8 0x08BC 0x5 0x2
+#define MX8ULP_PAD_PTE22__TPM6_CH0 0x00D8 0x097C 0x6 0x1
+#define MX8ULP_PAD_PTE22__I2S7_TXD2 0x00D8 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTE22__EXT_AUD_MCLK3 0x00D8 0x0B14 0x9 0x5
+#define MX8ULP_PAD_PTE22__ENET0_TXD1 0x00D8 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTE22__EPDC0_SDOED 0x00D8 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTE22__CLKOUT2 0x00D8 0x0000 0xd 0x0
+#define MX8ULP_PAD_PTE22__LP_HV_DBG_MUX_14 0x00D8 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTE23__PTE23 0x00DC 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTE23__FXIO1_D0 0x00DC 0x083C 0x2 0x1
+#define MX8ULP_PAD_PTE23__LPSPI5_PCS0 0x00DC 0x0910 0x3 0x1
+#define MX8ULP_PAD_PTE23__LPUART5_RX 0x00DC 0x08EC 0x4 0x2
+#define MX8ULP_PAD_PTE23__I3C2_SDA 0x00DC 0x08C0 0x5 0x2
+#define MX8ULP_PAD_PTE23__TPM6_CH1 0x00DC 0x0980 0x6 0x1
+#define MX8ULP_PAD_PTE23__I2S7_TXD3 0x00DC 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTE23__EXT_AUD_MCLK2 0x00DC 0x0800 0x9 0x1
+#define MX8ULP_PAD_PTE23__ENET0_TXD0 0x00DC 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTE23__EPDC0_SDOEZ 0x00DC 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTE23__CLKOUT1 0x00DC 0x0000 0xd 0x0
+#define MX8ULP_PAD_PTE23__LP_HV_DBG_MUX_15 0x00DC 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTF0__PTF0 0x0100 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTF0__FXIO1_D0 0x0100 0x083C 0x2 0x2
+#define MX8ULP_PAD_PTF0__LPUART6_CTS_B 0x0100 0x09CC 0x4 0x2
+#define MX8ULP_PAD_PTF0__LPI2C6_SCL 0x0100 0x09B8 0x5 0x2
+#define MX8ULP_PAD_PTF0__I2S7_RX_BCLK 0x0100 0x0B64 0x7 0x2
+#define MX8ULP_PAD_PTF0__SDHC1_D1 0x0100 0x0A68 0x8 0x2
+#define MX8ULP_PAD_PTF0__ENET0_RXD1 0x0100 0x0AFC 0x9 0x2
+#define MX8ULP_PAD_PTF0__USB1_ID 0x0100 0x0ACC 0xa 0x3
+#define MX8ULP_PAD_PTF0__EPDC0_SDOE 0x0100 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTF0__DPI0_D23 0x0100 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTF0__WUU1_P8 0x0100 0x0000 0xd 0x0
+#define MX8ULP_PAD_PTF1__PTF1 0x0104 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTF1__FXIO1_D1 0x0104 0x0840 0x2 0x2
+#define MX8ULP_PAD_PTF1__LPUART6_RTS_B 0x0104 0x0000 0x4 0x0
+#define MX8ULP_PAD_PTF1__LPI2C6_SDA 0x0104 0x09BC 0x5 0x2
+#define MX8ULP_PAD_PTF1__I2S7_RX_FS 0x0104 0x0B68 0x7 0x2
+#define MX8ULP_PAD_PTF1__SDHC1_D0 0x0104 0x0A64 0x8 0x2
+#define MX8ULP_PAD_PTF1__ENET0_RXD0 0x0104 0x0AF8 0x9 0x2
+#define MX8ULP_PAD_PTF1__LP_HV_DBG_MUX_16 0x0104 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTF1__EPDC0_SDSHR 0x0104 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTF1__DPI0_D22 0x0104 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTF1__WDOG3_RST 0x0104 0x0000 0xd 0x0
+#define MX8ULP_PAD_PTF1__DEBUG_MUX0_16 0x0104 0x0000 0xe 0x0
+#define MX8ULP_PAD_PTF1__DEBUG_MUX1_22 0x0104 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTF2__PTF2 0x0108 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTF2__FXIO1_D2 0x0108 0x086C 0x2 0x2
+#define MX8ULP_PAD_PTF2__LPUART6_TX 0x0108 0x09D4 0x4 0x2
+#define MX8ULP_PAD_PTF2__LPI2C6_HREQ 0x0108 0x09B4 0x5 0x2
+#define MX8ULP_PAD_PTF2__I2S7_RXD0 0x0108 0x0B54 0x7 0x2
+#define MX8ULP_PAD_PTF2__SDHC1_CLK 0x0108 0x0A5C 0x8 0x2
+#define MX8ULP_PAD_PTF2__ENET0_TXD1 0x0108 0x0000 0x9 0x0
+#define MX8ULP_PAD_PTF2__USB0_ID 0x0108 0x0AC8 0xa 0x3
+#define MX8ULP_PAD_PTF2__EPDC0_SDCE9 0x0108 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTF2__DPI0_D21 0x0108 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTF2__LP_HV_DBG_MUX_17 0x0108 0x0000 0xd 0x0
+#define MX8ULP_PAD_PTF2__DEBUG_MUX0_17 0x0108 0x0000 0xe 0x0
+#define MX8ULP_PAD_PTF2__DEBUG_MUX1_23 0x0108 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTF3__PTF3 0x010C 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTF3__FXIO1_D3 0x010C 0x0898 0x2 0x2
+#define MX8ULP_PAD_PTF3__LPUART6_RX 0x010C 0x09D0 0x4 0x2
+#define MX8ULP_PAD_PTF3__LPI2C7_HREQ 0x010C 0x09C0 0x5 0x2
+#define MX8ULP_PAD_PTF3__I2S7_RXD1 0x010C 0x0B58 0x7 0x2
+#define MX8ULP_PAD_PTF3__SDHC1_CMD 0x010C 0x0A60 0x8 0x2
+#define MX8ULP_PAD_PTF3__ENET0_TXD0 0x010C 0x0000 0x9 0x0
+#define MX8ULP_PAD_PTF3__USB0_PWR 0x010C 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTF3__EPDC0_SDCE8 0x010C 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTF3__DPI0_D20 0x010C 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTF3__WUU1_P9 0x010C 0x0000 0xd 0x0
+#define MX8ULP_PAD_PTF3__DEBUG_MUX1_24 0x010C 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTF4__PTF4 0x0110 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTF4__FXIO1_D4 0x0110 0x08A4 0x2 0x2
+#define MX8ULP_PAD_PTF4__LPSPI4_PCS1 0x0110 0x08F8 0x3 0x3
+#define MX8ULP_PAD_PTF4__LPUART7_CTS_B 0x0110 0x09D8 0x4 0x2
+#define MX8ULP_PAD_PTF4__LPI2C7_SCL 0x0110 0x09C4 0x5 0x2
+#define MX8ULP_PAD_PTF4__TPM7_CLKIN 0x0110 0x09B0 0x6 0x1
+#define MX8ULP_PAD_PTF4__I2S7_RXD2 0x0110 0x0B5C 0x7 0x2
+#define MX8ULP_PAD_PTF4__SDHC1_D3 0x0110 0x0A70 0x8 0x2
+#define MX8ULP_PAD_PTF4__ENET0_TXEN 0x0110 0x0000 0x9 0x0
+#define MX8ULP_PAD_PTF4__USB0_OC 0x0110 0x0AC0 0xa 0x3
+#define MX8ULP_PAD_PTF4__EPDC0_SDCE7 0x0110 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTF4__DPI0_D19 0x0110 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTF4__WUU1_P10 0x0110 0x0000 0xd 0x0
+#define MX8ULP_PAD_PTF4__DEBUG_MUX1_25 0x0110 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTF5__PTF5 0x0114 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTF5__FXIO1_D5 0x0114 0x08A8 0x2 0x2
+#define MX8ULP_PAD_PTF5__LPSPI4_PCS2 0x0114 0x08FC 0x3 0x3
+#define MX8ULP_PAD_PTF5__LPUART7_RTS_B 0x0114 0x0000 0x4 0x0
+#define MX8ULP_PAD_PTF5__LPI2C7_SDA 0x0114 0x09C8 0x5 0x2
+#define MX8ULP_PAD_PTF5__TPM7_CH0 0x0114 0x0998 0x6 0x1
+#define MX8ULP_PAD_PTF5__I2S7_RXD3 0x0114 0x0B60 0x7 0x2
+#define MX8ULP_PAD_PTF5__SDHC1_D2 0x0114 0x0A6C 0x8 0x2
+#define MX8ULP_PAD_PTF5__ENET0_RXER 0x0114 0x0B08 0x9 0x2
+#define MX8ULP_PAD_PTF5__USB1_PWR 0x0114 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTF5__EPDC0_SDCE6 0x0114 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTF5__DPI0_D18 0x0114 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTF5__LP_HV_DBG_MUX_18 0x0114 0x0000 0xd 0x0
+#define MX8ULP_PAD_PTF5__DEBUG_MUX0_18 0x0114 0x0000 0xe 0x0
+#define MX8ULP_PAD_PTF5__DEBUG_MUX1_26 0x0114 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTF6__LP_HV_DBG_MUX_19 0x0118 0x0000 0x0 0x0
+#define MX8ULP_PAD_PTF6__PTF6 0x0118 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTF6__FXIO1_D6 0x0118 0x08AC 0x2 0x2
+#define MX8ULP_PAD_PTF6__LPSPI4_PCS3 0x0118 0x0900 0x3 0x3
+#define MX8ULP_PAD_PTF6__LPUART7_TX 0x0118 0x09E0 0x4 0x2
+#define MX8ULP_PAD_PTF6__I3C2_SCL 0x0118 0x08BC 0x5 0x3
+#define MX8ULP_PAD_PTF6__TPM7_CH1 0x0118 0x099C 0x6 0x1
+#define MX8ULP_PAD_PTF6__I2S7_MCLK 0x0118 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTF6__SDHC1_D4 0x0118 0x0A74 0x8 0x2
+#define MX8ULP_PAD_PTF6__ENET0_CRS_DV 0x0118 0x0AEC 0x9 0x2
+#define MX8ULP_PAD_PTF6__USB1_OC 0x0118 0x0AC4 0xa 0x3
+#define MX8ULP_PAD_PTF6__EPDC0_SDCE5 0x0118 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTF6__DPI0_D17 0x0118 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTF6__WDOG4_RST 0x0118 0x0000 0xd 0x0
+#define MX8ULP_PAD_PTF6__DEBUG_MUX0_19 0x0118 0x0000 0xe 0x0
+#define MX8ULP_PAD_PTF6__DEBUG_MUX1_27 0x0118 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTF7__PTF7 0x011C 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTF7__FXIO1_D7 0x011C 0x08B0 0x2 0x2
+#define MX8ULP_PAD_PTF7__LPUART7_RX 0x011C 0x09DC 0x4 0x2
+#define MX8ULP_PAD_PTF7__I3C2_SDA 0x011C 0x08C0 0x5 0x3
+#define MX8ULP_PAD_PTF7__TPM7_CH2 0x011C 0x09A0 0x6 0x1
+#define MX8ULP_PAD_PTF7__MQS1_LEFT 0x011C 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTF7__SDHC1_D5 0x011C 0x0A78 0x8 0x2
+#define MX8ULP_PAD_PTF7__ENET0_REFCLK 0x011C 0x0AF4 0x9 0x2
+#define MX8ULP_PAD_PTF7__TRACE0_D15 0x011C 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTF7__EPDC0_SDCE4 0x011C 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTF7__DPI0_D16 0x011C 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTF7__WUU1_P11 0x011C 0x0000 0xd 0x0
+#define MX8ULP_PAD_PTF7__DEBUG_MUX1_28 0x011C 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTF8__PTF8 0x0120 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTF8__FXIO1_D8 0x0120 0x08B4 0x2 0x2
+#define MX8ULP_PAD_PTF8__LPSPI4_SIN 0x0120 0x0908 0x3 0x3
+#define MX8ULP_PAD_PTF8__LPUART4_CTS_B 0x0120 0x08DC 0x4 0x3
+#define MX8ULP_PAD_PTF8__LPI2C4_SCL 0x0120 0x08C8 0x5 0x3
+#define MX8ULP_PAD_PTF8__TPM7_CH3 0x0120 0x09A4 0x6 0x1
+#define MX8ULP_PAD_PTF8__MQS1_RIGHT 0x0120 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTF8__SDHC1_D6 0x0120 0x0A7C 0x8 0x2
+#define MX8ULP_PAD_PTF8__ENET0_MDIO 0x0120 0x0AF0 0x9 0x2
+#define MX8ULP_PAD_PTF8__TRACE0_D14 0x0120 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTF8__EPDC0_D15 0x0120 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTF8__DPI0_D15 0x0120 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTF8__LP_HV_DBG_MUX_24 0x0120 0x0000 0xe 0x0
+#define MX8ULP_PAD_PTF8__DEBUG_MUX1_29 0x0120 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTF9__PTF9 0x0124 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTF9__FXIO1_D9 0x0124 0x08B8 0x2 0x2
+#define MX8ULP_PAD_PTF9__LPSPI4_SOUT 0x0124 0x090C 0x3 0x3
+#define MX8ULP_PAD_PTF9__LPUART4_RTS_B 0x0124 0x0000 0x4 0x0
+#define MX8ULP_PAD_PTF9__LPI2C4_SDA 0x0124 0x08CC 0x5 0x3
+#define MX8ULP_PAD_PTF9__TPM7_CH4 0x0124 0x09A8 0x6 0x1
+#define MX8ULP_PAD_PTF9__EXT_AUD_MCLK2 0x0124 0x0800 0x7 0x2
+#define MX8ULP_PAD_PTF9__SDHC1_D7 0x0124 0x0A80 0x8 0x2
+#define MX8ULP_PAD_PTF9__ENET0_MDC 0x0124 0x0000 0x9 0x0
+#define MX8ULP_PAD_PTF9__TRACE0_D13 0x0124 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTF9__EPDC0_D14 0x0124 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTF9__DPI0_D14 0x0124 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTF9__LP_HV_DBG_MUX_25 0x0124 0x0000 0xe 0x0
+#define MX8ULP_PAD_PTF9__DEBUG_MUX1_30 0x0124 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTF10__LP_HV_DBG_MUX_26 0x0128 0x0000 0x0 0x0
+#define MX8ULP_PAD_PTF10__PTF10 0x0128 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTF10__FXIO1_D10 0x0128 0x0844 0x2 0x2
+#define MX8ULP_PAD_PTF10__LPSPI4_SCK 0x0128 0x0904 0x3 0x3
+#define MX8ULP_PAD_PTF10__LPUART4_TX 0x0128 0x08E4 0x4 0x3
+#define MX8ULP_PAD_PTF10__LPI2C4_HREQ 0x0128 0x08C4 0x5 0x3
+#define MX8ULP_PAD_PTF10__TPM7_CH5 0x0128 0x09AC 0x6 0x1
+#define MX8ULP_PAD_PTF10__I2S4_RX_BCLK 0x0128 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTF10__SDHC1_DQS 0x0128 0x0A84 0x8 0x2
+#define MX8ULP_PAD_PTF10__ENET0_1588_CLKIN 0x0128 0x0AD0 0x9 0x2
+#define MX8ULP_PAD_PTF10__TRACE0_D12 0x0128 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTF10__EPDC0_D13 0x0128 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTF10__DPI0_D13 0x0128 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTF10__DEBUG_MUX0_20 0x0128 0x0000 0xe 0x0
+#define MX8ULP_PAD_PTF10__DEBUG_MUX1_31 0x0128 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTF11__PTF11 0x012C 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTF11__FXIO1_D11 0x012C 0x0848 0x2 0x2
+#define MX8ULP_PAD_PTF11__LPSPI4_PCS0 0x012C 0x08F4 0x3 0x3
+#define MX8ULP_PAD_PTF11__LPUART4_RX 0x012C 0x08E0 0x4 0x3
+#define MX8ULP_PAD_PTF11__TPM4_CLKIN 0x012C 0x081C 0x6 0x2
+#define MX8ULP_PAD_PTF11__I2S4_RX_FS 0x012C 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTF11__SDHC1_RESET_B 0x012C 0x0000 0x8 0x0
+#define MX8ULP_PAD_PTF11__ENET0_1588_TMR0 0x012C 0x0AD4 0x9 0x2
+#define MX8ULP_PAD_PTF11__TRACE0_D11 0x012C 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTF11__EPDC0_D12 0x012C 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTF11__DPI0_D12 0x012C 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTF11__LP_HV_DBG_MUX_27 0x012C 0x0000 0xe 0x0
+#define MX8ULP_PAD_PTF11__DEBUG_MUX1_32 0x012C 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTF12__PTF12 0x0130 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTF12__FXIO1_D12 0x0130 0x084C 0x2 0x2
+#define MX8ULP_PAD_PTF12__LPSPI5_PCS1 0x0130 0x0914 0x3 0x2
+#define MX8ULP_PAD_PTF12__LPUART5_CTS_B 0x0130 0x08E8 0x4 0x3
+#define MX8ULP_PAD_PTF12__LPI2C5_SCL 0x0130 0x08D4 0x5 0x3
+#define MX8ULP_PAD_PTF12__TPM4_CH0 0x0130 0x0804 0x6 0x2
+#define MX8ULP_PAD_PTF12__I2S4_RXD0 0x0130 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTF12__SDHC2_WP 0x0130 0x0ABC 0x8 0x1
+#define MX8ULP_PAD_PTF12__ENET0_1588_TMR1 0x0130 0x0AD8 0x9 0x2
+#define MX8ULP_PAD_PTF12__TRACE0_D10 0x0130 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTF12__EPDC0_D11 0x0130 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTF12__DPI0_D11 0x0130 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTF12__LP_HV_DBG_MUX_28 0x0130 0x0000 0xe 0x0
+#define MX8ULP_PAD_PTF12__DEBUG_MUX1_33 0x0130 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTF13__PTF13 0x0134 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTF13__FXIO1_D13 0x0134 0x0850 0x2 0x2
+#define MX8ULP_PAD_PTF13__LPSPI5_PCS2 0x0134 0x0918 0x3 0x2
+#define MX8ULP_PAD_PTF13__LPUART5_RTS_B 0x0134 0x0000 0x4 0x0
+#define MX8ULP_PAD_PTF13__LPI2C5_SDA 0x0134 0x08D8 0x5 0x3
+#define MX8ULP_PAD_PTF13__TPM4_CH1 0x0134 0x0808 0x6 0x2
+#define MX8ULP_PAD_PTF13__I2S4_RXD1 0x0134 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTF13__SDHC2_CD 0x0134 0x0A8C 0x8 0x1
+#define MX8ULP_PAD_PTF13__ENET0_1588_TMR2 0x0134 0x0ADC 0x9 0x2
+#define MX8ULP_PAD_PTF13__TRACE0_D9 0x0134 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTF13__EPDC0_D10 0x0134 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTF13__DPI0_D10 0x0134 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTF13__DEBUG_MUX0_21 0x0134 0x0000 0xe 0x0
+#define MX8ULP_PAD_PTF13__LP_HV_DBG_MUX_29 0x0134 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTF14__PTF14 0x0138 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTF14__FXIO1_D14 0x0138 0x0854 0x2 0x2
+#define MX8ULP_PAD_PTF14__LPSPI5_PCS3 0x0138 0x091C 0x3 0x2
+#define MX8ULP_PAD_PTF14__LPUART5_TX 0x0138 0x08F0 0x4 0x3
+#define MX8ULP_PAD_PTF14__LPI2C5_HREQ 0x0138 0x08D0 0x5 0x3
+#define MX8ULP_PAD_PTF14__TPM4_CH2 0x0138 0x080C 0x6 0x2
+#define MX8ULP_PAD_PTF14__I2S4_MCLK 0x0138 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTF14__SDHC2_VS 0x0138 0x0000 0x8 0x0
+#define MX8ULP_PAD_PTF14__ENET0_1588_TMR3 0x0138 0x0AE0 0x9 0x2
+#define MX8ULP_PAD_PTF14__TRACE0_D8 0x0138 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTF14__EPDC0_D9 0x0138 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTF14__DPI0_D9 0x0138 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTF14__DEBUG_MUX0_22 0x0138 0x0000 0xe 0x0
+#define MX8ULP_PAD_PTF14__LP_HV_DBG_MUX_30 0x0138 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTF15__PTF15 0x013C 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTF15__FXIO1_D15 0x013C 0x0858 0x2 0x2
+#define MX8ULP_PAD_PTF15__LPUART5_RX 0x013C 0x08EC 0x4 0x3
+#define MX8ULP_PAD_PTF15__TPM4_CH3 0x013C 0x0810 0x6 0x2
+#define MX8ULP_PAD_PTF15__I2S4_TX_BCLK 0x013C 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTF15__SDHC2_D1 0x013C 0x0A9C 0x8 0x3
+#define MX8ULP_PAD_PTF15__ENET0_RXD2 0x013C 0x0B00 0x9 0x2
+#define MX8ULP_PAD_PTF15__TRACE0_D7 0x013C 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTF15__EPDC0_D8 0x013C 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTF15__DPI0_D8 0x013C 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTF15__LP_HV_DBG_MUX_31 0x013C 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTF16__PTF16 0x0140 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTF16__FXIO1_D16 0x0140 0x085C 0x2 0x2
+#define MX8ULP_PAD_PTF16__LPSPI5_SIN 0x0140 0x0924 0x3 0x2
+#define MX8ULP_PAD_PTF16__LPUART6_CTS_B 0x0140 0x09CC 0x4 0x3
+#define MX8ULP_PAD_PTF16__LPI2C6_SCL 0x0140 0x09B8 0x5 0x3
+#define MX8ULP_PAD_PTF16__TPM4_CH4 0x0140 0x0814 0x6 0x2
+#define MX8ULP_PAD_PTF16__I2S4_TX_FS 0x0140 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTF16__SDHC2_D0 0x0140 0x0A98 0x8 0x3
+#define MX8ULP_PAD_PTF16__ENET0_RXD3 0x0140 0x0B04 0x9 0x2
+#define MX8ULP_PAD_PTF16__TRACE0_D6 0x0140 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTF16__EPDC0_D7 0x0140 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTF16__DPI0_D7 0x0140 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTF16__LP_HV_DBG_MUX_32 0x0140 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTF17__PTF17 0x0144 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTF17__FXIO1_D17 0x0144 0x0860 0x2 0x2
+#define MX8ULP_PAD_PTF17__LPSPI5_SOUT 0x0144 0x0928 0x3 0x2
+#define MX8ULP_PAD_PTF17__LPUART6_RTS_B 0x0144 0x0000 0x4 0x0
+#define MX8ULP_PAD_PTF17__LPI2C6_SDA 0x0144 0x09BC 0x5 0x3
+#define MX8ULP_PAD_PTF17__TPM4_CH5 0x0144 0x0818 0x6 0x2
+#define MX8ULP_PAD_PTF17__I2S4_TXD0 0x0144 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTF17__SDHC2_CLK 0x0144 0x0A90 0x8 0x3
+#define MX8ULP_PAD_PTF17__ENET0_RXCLK 0x0144 0x0B0C 0x9 0x2
+#define MX8ULP_PAD_PTF17__TRACE0_D5 0x0144 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTF17__EPDC0_D6 0x0144 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTF17__DPI0_D6 0x0144 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTF17__DEBUG_MUX0_23 0x0144 0x0000 0xe 0x0
+#define MX8ULP_PAD_PTF17__LP_HV_DBG_MUX_33 0x0144 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTF18__PTF18 0x0148 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTF18__FXIO1_D18 0x0148 0x0864 0x2 0x2
+#define MX8ULP_PAD_PTF18__LPSPI5_SCK 0x0148 0x0920 0x3 0x2
+#define MX8ULP_PAD_PTF18__LPUART6_TX 0x0148 0x09D4 0x4 0x3
+#define MX8ULP_PAD_PTF18__LPI2C6_HREQ 0x0148 0x09B4 0x5 0x3
+#define MX8ULP_PAD_PTF18__TPM5_CLKIN 0x0148 0x0838 0x6 0x2
+#define MX8ULP_PAD_PTF18__I2S4_TXD1 0x0148 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTF18__SDHC2_CMD 0x0148 0x0A94 0x8 0x3
+#define MX8ULP_PAD_PTF18__ENET0_TXD2 0x0148 0x0000 0x9 0x0
+#define MX8ULP_PAD_PTF18__TRACE0_D4 0x0148 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTF18__EPDC0_D5 0x0148 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTF18__DPI0_D5 0x0148 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTF19__PTF19 0x014C 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTF19__FXIO1_D19 0x014C 0x0868 0x2 0x2
+#define MX8ULP_PAD_PTF19__LPSPI5_PCS0 0x014C 0x0910 0x3 0x2
+#define MX8ULP_PAD_PTF19__LPUART6_RX 0x014C 0x09D0 0x4 0x3
+#define MX8ULP_PAD_PTF19__TPM5_CH0 0x014C 0x0820 0x6 0x2
+#define MX8ULP_PAD_PTF19__I2S5_RX_BCLK 0x014C 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTF19__SDHC2_D3 0x014C 0x0AA4 0x8 0x3
+#define MX8ULP_PAD_PTF19__ENET0_TXD3 0x014C 0x0000 0x9 0x0
+#define MX8ULP_PAD_PTF19__TRACE0_D3 0x014C 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTF19__EPDC0_D4 0x014C 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTF19__DPI0_D4 0x014C 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTF20__PTF20 0x0150 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTF20__FXIO1_D20 0x0150 0x0870 0x2 0x2
+#define MX8ULP_PAD_PTF20__LPUART7_CTS_B 0x0150 0x09D8 0x4 0x3
+#define MX8ULP_PAD_PTF20__LPI2C7_SCL 0x0150 0x09C4 0x5 0x3
+#define MX8ULP_PAD_PTF20__TPM5_CH1 0x0150 0x0824 0x6 0x2
+#define MX8ULP_PAD_PTF20__I2S5_RX_FS 0x0150 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTF20__SDHC2_D2 0x0150 0x0AA0 0x8 0x3
+#define MX8ULP_PAD_PTF20__ENET0_TXCLK 0x0150 0x0B10 0x9 0x2
+#define MX8ULP_PAD_PTF20__TRACE0_D2 0x0150 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTF20__EPDC0_D3 0x0150 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTF20__DPI0_D3 0x0150 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTF21__PTF21 0x0154 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTF21__FXIO1_D21 0x0154 0x0874 0x2 0x2
+#define MX8ULP_PAD_PTF21__SPDIF_CLK 0x0154 0x0000 0x3 0x0
+#define MX8ULP_PAD_PTF21__LPUART7_RTS_B 0x0154 0x0000 0x4 0x0
+#define MX8ULP_PAD_PTF21__LPI2C7_SDA 0x0154 0x09C8 0x5 0x3
+#define MX8ULP_PAD_PTF21__TPM6_CLKIN 0x0154 0x0994 0x6 0x2
+#define MX8ULP_PAD_PTF21__I2S5_RXD0 0x0154 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTF21__SDHC2_D4 0x0154 0x0AA8 0x8 0x2
+#define MX8ULP_PAD_PTF21__ENET0_CRS 0x0154 0x0AE8 0x9 0x2
+#define MX8ULP_PAD_PTF21__TRACE0_D1 0x0154 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTF21__EPDC0_D2 0x0154 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTF21__DPI0_D2 0x0154 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTF22__PTF22 0x0158 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTF22__FXIO1_D22 0x0158 0x0878 0x2 0x2
+#define MX8ULP_PAD_PTF22__SPDIF_IN0 0x0158 0x0B74 0x3 0x3
+#define MX8ULP_PAD_PTF22__LPUART7_TX 0x0158 0x09E0 0x4 0x3
+#define MX8ULP_PAD_PTF22__LPI2C7_HREQ 0x0158 0x09C0 0x5 0x3
+#define MX8ULP_PAD_PTF22__TPM6_CH0 0x0158 0x097C 0x6 0x2
+#define MX8ULP_PAD_PTF22__I2S5_RXD1 0x0158 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTF22__SDHC2_D5 0x0158 0x0AAC 0x8 0x2
+#define MX8ULP_PAD_PTF22__ENET0_COL 0x0158 0x0AE4 0x9 0x2
+#define MX8ULP_PAD_PTF22__TRACE0_D0 0x0158 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTF22__EPDC0_D1 0x0158 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTF22__DPI0_D1 0x0158 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTF23__PTF23 0x015C 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTF23__FXIO1_D23 0x015C 0x087C 0x2 0x2
+#define MX8ULP_PAD_PTF23__SPDIF_OUT0 0x015C 0x0000 0x3 0x0
+#define MX8ULP_PAD_PTF23__LPUART7_RX 0x015C 0x09DC 0x4 0x3
+#define MX8ULP_PAD_PTF23__I3C2_PUR 0x015C 0x0000 0x5 0x0
+#define MX8ULP_PAD_PTF23__TPM6_CH1 0x015C 0x0980 0x6 0x2
+#define MX8ULP_PAD_PTF23__I2S5_RXD2 0x015C 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTF23__SDHC2_D6 0x015C 0x0AB0 0x8 0x2
+#define MX8ULP_PAD_PTF23__ENET0_TXER 0x015C 0x0000 0x9 0x0
+#define MX8ULP_PAD_PTF23__TRACE0_CLKOUT 0x015C 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTF23__EPDC0_D0 0x015C 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTF23__DPI0_D0 0x015C 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTF24__PTF24 0x0160 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTF24__FXIO1_D24 0x0160 0x0880 0x2 0x2
+#define MX8ULP_PAD_PTF24__SPDIF_IN1 0x0160 0x0B78 0x3 0x3
+#define MX8ULP_PAD_PTF24__I3C2_SCL 0x0160 0x08BC 0x5 0x4
+#define MX8ULP_PAD_PTF24__I2S5_RXD3 0x0160 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTF24__SDHC2_D7 0x0160 0x0AB4 0x8 0x2
+#define MX8ULP_PAD_PTF24__DBI0_WRX 0x0160 0x0000 0xa 0x0
+#define MX8ULP_PAD_PTF24__EPDC0_SDCLK 0x0160 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTF24__DPI0_PCLK 0x0160 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTF24__WUU1_P12 0x0160 0x0000 0xd 0x0
+#define MX8ULP_PAD_PTF25__PTF25 0x0164 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTF25__FXIO1_D25 0x0164 0x0884 0x2 0x2
+#define MX8ULP_PAD_PTF25__SPDIF_OUT1 0x0164 0x0000 0x3 0x0
+#define MX8ULP_PAD_PTF25__I3C2_SDA 0x0164 0x08C0 0x5 0x4
+#define MX8ULP_PAD_PTF25__TPM7_CH5 0x0164 0x09AC 0x6 0x2
+#define MX8ULP_PAD_PTF25__I2S5_MCLK 0x0164 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTF25__SDHC2_DQS 0x0164 0x0AB8 0x8 0x2
+#define MX8ULP_PAD_PTF25__EXT_AUD_MCLK2 0x0164 0x0800 0x9 0x3
+#define MX8ULP_PAD_PTF25__EPDC0_GDSP 0x0164 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTF25__DPI0_VSYNC 0x0164 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTF25__WUU1_P13 0x0164 0x0000 0xd 0x0
+#define MX8ULP_PAD_PTF26__PTF26 0x0168 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTF26__FXIO1_D26 0x0168 0x0888 0x2 0x2
+#define MX8ULP_PAD_PTF26__SPDIF_IN2 0x0168 0x0B7C 0x3 0x3
+#define MX8ULP_PAD_PTF26__TPM7_CLKIN 0x0168 0x09B0 0x6 0x2
+#define MX8ULP_PAD_PTF26__I2S5_TX_BCLK 0x0168 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTF26__SDHC2_RESET_B 0x0168 0x0000 0x8 0x0
+#define MX8ULP_PAD_PTF26__EPDC0_SDLE 0x0168 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTF26__DPI0_HSYNC 0x0168 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTF26__WUU1_P14 0x0168 0x0000 0xd 0x0
+#define MX8ULP_PAD_PTF27__PTF27 0x016C 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTF27__FXIO1_D27 0x016C 0x088C 0x2 0x2
+#define MX8ULP_PAD_PTF27__SPDIF_OUT2 0x016C 0x0000 0x3 0x0
+#define MX8ULP_PAD_PTF27__TPM7_CH0 0x016C 0x0998 0x6 0x2
+#define MX8ULP_PAD_PTF27__I2S5_TX_FS 0x016C 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTF27__SDHC2_WP 0x016C 0x0ABC 0x8 0x2
+#define MX8ULP_PAD_PTF27__EPDC0_SDCE0 0x016C 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTF27__DPI0_DE 0x016C 0x0000 0xc 0x0
+#define MX8ULP_PAD_PTF27__WUU1_P15 0x016C 0x0000 0xd 0x0
+#define MX8ULP_PAD_PTF28__PTF28 0x0170 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTF28__FXIO1_D28 0x0170 0x0890 0x2 0x2
+#define MX8ULP_PAD_PTF28__SPDIF_IN3 0x0170 0x0B80 0x3 0x3
+#define MX8ULP_PAD_PTF28__TPM7_CH1 0x0170 0x099C 0x6 0x2
+#define MX8ULP_PAD_PTF28__I2S5_TXD0 0x0170 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTF28__SDHC2_CD 0x0170 0x0A8C 0x8 0x2
+#define MX8ULP_PAD_PTF28__EPDC0_SDCLK_B 0x0170 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTF28__LP_HV_DBG_MUX_20 0x0170 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTF29__PTF29 0x0174 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTF29__FXIO1_D29 0x0174 0x0894 0x2 0x2
+#define MX8ULP_PAD_PTF29__SPDIF_OUT3 0x0174 0x0000 0x3 0x0
+#define MX8ULP_PAD_PTF29__TPM7_CH2 0x0174 0x09A0 0x6 0x2
+#define MX8ULP_PAD_PTF29__I2S5_TXD1 0x0174 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTF29__SDHC2_VS 0x0174 0x0000 0x8 0x0
+#define MX8ULP_PAD_PTF29__EPDC0_SDCE1 0x0174 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTF29__WDOG3_RST 0x0174 0x0000 0xd 0x0
+#define MX8ULP_PAD_PTF29__LP_HV_DBG_MUX_21 0x0174 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTF30__PTF30 0x0178 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTF30__FXIO1_D30 0x0178 0x089C 0x2 0x2
+#define MX8ULP_PAD_PTF30__TPM7_CH3 0x0178 0x09A4 0x6 0x2
+#define MX8ULP_PAD_PTF30__I2S5_TXD2 0x0178 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTF30__MQS1_LEFT 0x0178 0x0000 0x8 0x0
+#define MX8ULP_PAD_PTF30__EPDC0_SDCE2 0x0178 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTF30__WDOG4_RST 0x0178 0x0000 0xd 0x0
+#define MX8ULP_PAD_PTF30__LP_HV_DBG_MUX_22 0x0178 0x0000 0xf 0x0
+#define MX8ULP_PAD_PTF31__PTF31 0x017C 0x0000 0x1 0x0
+#define MX8ULP_PAD_PTF31__FXIO1_D31 0x017C 0x08A0 0x2 0x2
+#define MX8ULP_PAD_PTF31__TPM7_CH4 0x017C 0x09A8 0x6 0x2
+#define MX8ULP_PAD_PTF31__I2S5_TXD3 0x017C 0x0000 0x7 0x0
+#define MX8ULP_PAD_PTF31__MQS1_RIGHT 0x017C 0x0000 0x8 0x0
+#define MX8ULP_PAD_PTF31__EPDC0_SDCE3 0x017C 0x0000 0xb 0x0
+#define MX8ULP_PAD_PTF31__WDOG5_RST 0x017C 0x0000 0xd 0x0
+#define MX8ULP_PAD_PTF31__LP_HV_DBG_MUX_23 0x017C 0x0000 0xf 0x0
+#define MX8ULP_PAD_BOOT_MODE0__BOOT_MODE0 0x0400 0x0000 0x0 0x0
+#define MX8ULP_PAD_BOOT_MODE1__BOOT_MODE1 0x0404 0x0000 0x0 0x0
+
+#endif /* __DTS_IMX8ULP_PINFUNC_H */
diff --git a/arch/arm64/boot/dts/freescale/imx8ulp.dtsi b/arch/arm64/boot/dts/freescale/imx8ulp.dtsi
new file mode 100644
index 0000000..a987ff7
--- /dev/null
+++ b/arch/arm64/boot/dts/freescale/imx8ulp.dtsi
@@ -0,0 +1,434 @@
+// SPDX-License-Identifier: (GPL-2.0+ OR MIT)
+/*
+ * Copyright 2021 NXP
+ */
+
+#include <dt-bindings/clock/imx8ulp-clock.h>
+#include <dt-bindings/gpio/gpio.h>
+#include <dt-bindings/interrupt-controller/arm-gic.h>
+#include <dt-bindings/power/imx8ulp-power.h>
+
+#include "imx8ulp-pinfunc.h"
+
+/ {
+ interrupt-parent = <&gic>;
+ #address-cells = <2>;
+ #size-cells = <2>;
+
+ aliases {
+ gpio0 = &gpiod;
+ gpio1 = &gpioe;
+ gpio2 = &gpiof;
+ mmc0 = &usdhc0;
+ mmc1 = &usdhc1;
+ mmc2 = &usdhc2;
+ serial0 = &lpuart4;
+ serial1 = &lpuart5;
+ serial2 = &lpuart6;
+ serial3 = &lpuart7;
+ };
+
+ cpus {
+ #address-cells = <2>;
+ #size-cells = <0>;
+
+ A35_0: cpu@0 {
+ device_type = "cpu";
+ compatible = "arm,cortex-a35";
+ reg = <0x0 0x0>;
+ enable-method = "psci";
+ next-level-cache = <&A35_L2>;
+ };
+
+ A35_1: cpu@1 {
+ device_type = "cpu";
+ compatible = "arm,cortex-a35";
+ reg = <0x0 0x1>;
+ enable-method = "psci";
+ next-level-cache = <&A35_L2>;
+ };
+
+ A35_L2: l2-cache0 {
+ compatible = "cache";
+ };
+ };
+
+ gic: interrupt-controller@2d400000 {
+ compatible = "arm,gic-v3";
+ reg = <0x0 0x2d400000 0 0x10000>, /* GIC Dist */
+ <0x0 0x2d440000 0 0xc0000>; /* GICR (RD_base + SGI_base) */
+ #interrupt-cells = <3>;
+ interrupt-controller;
+ interrupts = <GIC_PPI 9 IRQ_TYPE_LEVEL_HIGH>;
+ };
+
+ psci {
+ compatible = "arm,psci-1.0";
+ method = "smc";
+ };
+
+ timer {
+ compatible = "arm,armv8-timer";
+ interrupts = <GIC_PPI 13 IRQ_TYPE_LEVEL_LOW>, /* Physical Secure */
+ <GIC_PPI 14 IRQ_TYPE_LEVEL_LOW>, /* Physical Non-Secure */
+ <GIC_PPI 11 IRQ_TYPE_LEVEL_LOW>, /* Virtual */
+ <GIC_PPI 10 IRQ_TYPE_LEVEL_LOW>; /* Hypervisor */
+ };
+
+ frosc: clock-frosc {
+ compatible = "fixed-clock";
+ clock-frequency = <192000000>;
+ clock-output-names = "frosc";
+ #clock-cells = <0>;
+ };
+
+ lposc: clock-lposc {
+ compatible = "fixed-clock";
+ clock-frequency = <1000000>;
+ clock-output-names = "lposc";
+ #clock-cells = <0>;
+ };
+
+ rosc: clock-rosc {
+ compatible = "fixed-clock";
+ clock-frequency = <32768>;
+ clock-output-names = "rosc";
+ #clock-cells = <0>;
+ };
+
+ sosc: clock-sosc {
+ compatible = "fixed-clock";
+ clock-frequency = <24000000>;
+ clock-output-names = "sosc";
+ #clock-cells = <0>;
+ };
+
+ sram@2201f000 {
+ compatible = "mmio-sram";
+ reg = <0x0 0x2201f000 0x0 0x1000>;
+
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges = <0 0x0 0x2201f000 0x1000>;
+
+ scmi_buf: scmi-buf@0 {
+ compatible = "arm,scmi-shmem";
+ reg = <0x0 0x400>;
+ };
+ };
+
+ firmware {
+ scmi {
+ compatible = "arm,scmi-smc";
+ arm,smc-id = <0xc20000fe>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ shmem = <&scmi_buf>;
+
+ scmi_devpd: protocol@11 {
+ reg = <0x11>;
+ #power-domain-cells = <1>;
+ };
+
+ scmi_sensor: protocol@15 {
+ reg = <0x15>;
+ #thermal-sensor-cells = <0>;
+ };
+ };
+ };
+
+ soc@0 {
+ compatible = "simple-bus";
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges = <0x0 0x0 0x0 0x40000000>;
+
+ per_bridge3: bus@29000000 {
+ compatible = "simple-bus";
+ reg = <0x29000000 0x800000>;
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges;
+
+ wdog3: watchdog@292a0000 {
+ compatible = "fsl,imx8ulp-wdt", "fsl,imx7ulp-wdt";
+ reg = <0x292a0000 0x10000>;
+ interrupts = <GIC_SPI 76 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&pcc3 IMX8ULP_CLK_WDOG3>;
+ assigned-clocks = <&pcc3 IMX8ULP_CLK_WDOG3>;
+ assigned-clock-parents = <&cgc1 IMX8ULP_CLK_SOSC_DIV2>;
+ timeout-sec = <40>;
+ };
+
+ cgc1: clock-controller@292c0000 {
+ compatible = "fsl,imx8ulp-cgc1";
+ reg = <0x292c0000 0x10000>;
+ clocks = <&rosc>, <&sosc>, <&frosc>, <&lposc>;
+ clock-names = "rosc", "sosc", "frosc", "lposc";
+ #clock-cells = <1>;
+ };
+
+ pcc3: clock-controller@292d0000 {
+ compatible = "fsl,imx8ulp-pcc3";
+ reg = <0x292d0000 0x10000>;
+ #clock-cells = <1>;
+ };
+
+ tpm5: tpm@29340000 {
+ compatible = "fsl,imx8ulp-tpm", "fsl,imx7ulp-tpm";
+ reg = <0x29340000 0x1000>;
+ interrupts = <GIC_SPI 87 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&pcc3 IMX8ULP_CLK_TPM5>,
+ <&pcc3 IMX8ULP_CLK_TPM5>;
+ clock-names = "ipg", "per";
+ status = "disabled";
+ };
+
+ lpi2c4: i2c@29370000 {
+ compatible = "fsl,imx8ulp-lpi2c", "fsl,imx7ulp-lpi2c";
+ reg = <0x29370000 0x10000>;
+ interrupts = <GIC_SPI 92 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&pcc3 IMX8ULP_CLK_LPI2C4>,
+ <&pcc3 IMX8ULP_CLK_LPI2C4>;
+ clock-names = "per", "ipg";
+ assigned-clocks = <&pcc3 IMX8ULP_CLK_LPI2C4>;
+ assigned-clock-parents = <&cgc1 IMX8ULP_CLK_XBAR_DIVBUS>;
+ assigned-clock-rates = <48000000>;
+ status = "disabled";
+ };
+
+ lpi2c5: i2c@29380000 {
+ compatible = "fsl,imx8ulp-lpi2c", "fsl,imx7ulp-lpi2c";
+ reg = <0x29380000 0x10000>;
+ interrupts = <GIC_SPI 93 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&pcc3 IMX8ULP_CLK_LPI2C5>,
+ <&pcc3 IMX8ULP_CLK_LPI2C5>;
+ clock-names = "per", "ipg";
+ assigned-clocks = <&pcc3 IMX8ULP_CLK_LPI2C5>;
+ assigned-clock-parents = <&cgc1 IMX8ULP_CLK_XBAR_DIVBUS>;
+ assigned-clock-rates = <48000000>;
+ status = "disabled";
+ };
+
+ lpuart4: serial@29390000 {
+ compatible = "fsl,imx8ulp-lpuart", "fsl,imx7ulp-lpuart";
+ reg = <0x29390000 0x1000>;
+ interrupts = <GIC_SPI 99 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&pcc3 IMX8ULP_CLK_LPUART4>;
+ clock-names = "ipg";
+ status = "disabled";
+ };
+
+ lpuart5: serial@293a0000 {
+ compatible = "fsl,imx8ulp-lpuart", "fsl,imx7ulp-lpuart";
+ reg = <0x293a0000 0x1000>;
+ interrupts = <GIC_SPI 100 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&pcc3 IMX8ULP_CLK_LPUART5>;
+ clock-names = "ipg";
+ status = "disabled";
+ };
+
+ lpspi4: spi@293b0000 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ compatible = "fsl,imx8ulp-spi", "fsl,imx7ulp-spi";
+ reg = <0x293b0000 0x10000>;
+ interrupts = <GIC_SPI 97 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&pcc3 IMX8ULP_CLK_LPSPI4>,
+ <&pcc3 IMX8ULP_CLK_LPSPI4>;
+ clock-names = "per", "ipg";
+ assigned-clocks = <&pcc3 IMX8ULP_CLK_LPSPI4>;
+ assigned-clock-parents = <&cgc1 IMX8ULP_CLK_XBAR_DIVBUS>;
+ assigned-clock-rates = <16000000>;
+ status = "disabled";
+ };
+
+ lpspi5: spi@293c0000 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ compatible = "fsl,imx8ulp-spi", "fsl,imx7ulp-spi";
+ reg = <0x293c0000 0x10000>;
+ interrupts = <GIC_SPI 98 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&pcc3 IMX8ULP_CLK_LPSPI5>,
+ <&pcc3 IMX8ULP_CLK_LPSPI5>;
+ clock-names = "per", "ipg";
+ assigned-clocks = <&pcc3 IMX8ULP_CLK_LPSPI5>;
+ assigned-clock-parents = <&cgc1 IMX8ULP_CLK_XBAR_DIVBUS>;
+ assigned-clock-rates = <16000000>;
+ status = "disabled";
+ };
+ };
+
+ per_bridge4: bus@29800000 {
+ compatible = "simple-bus";
+ reg = <0x29800000 0x800000>;
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges;
+
+ pcc4: clock-controller@29800000 {
+ compatible = "fsl,imx8ulp-pcc4";
+ reg = <0x29800000 0x10000>;
+ #clock-cells = <1>;
+ };
+
+ lpi2c6: i2c@29840000 {
+ compatible = "fsl,imx8ulp-lpi2c", "fsl,imx7ulp-lpi2c";
+ reg = <0x29840000 0x10000>;
+ interrupts = <GIC_SPI 94 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&pcc4 IMX8ULP_CLK_LPI2C6>,
+ <&pcc4 IMX8ULP_CLK_LPI2C6>;
+ clock-names = "per", "ipg";
+ assigned-clocks = <&pcc4 IMX8ULP_CLK_LPI2C6>;
+ assigned-clock-parents = <&cgc1 IMX8ULP_CLK_XBAR_DIVBUS>;
+ assigned-clock-rates = <48000000>;
+ status = "disabled";
+ };
+
+ lpi2c7: i2c@29850000 {
+ compatible = "fsl,imx8ulp-lpi2c", "fsl,imx7ulp-lpi2c";
+ reg = <0x29850000 0x10000>;
+ interrupts = <GIC_SPI 95 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&pcc4 IMX8ULP_CLK_LPI2C7>,
+ <&pcc4 IMX8ULP_CLK_LPI2C7>;
+ clock-names = "per", "ipg";
+ assigned-clocks = <&pcc4 IMX8ULP_CLK_LPI2C7>;
+ assigned-clock-parents = <&cgc1 IMX8ULP_CLK_XBAR_DIVBUS>;
+ assigned-clock-rates = <48000000>;
+ status = "disabled";
+ };
+
+ lpuart6: serial@29860000 {
+ compatible = "fsl,imx8ulp-lpuart", "fsl,imx7ulp-lpuart";
+ reg = <0x29860000 0x1000>;
+ interrupts = <GIC_SPI 101 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&pcc4 IMX8ULP_CLK_LPUART6>;
+ clock-names = "ipg";
+ status = "disabled";
+ };
+
+ lpuart7: serial@29870000 {
+ compatible = "fsl,imx8ulp-lpuart", "fsl,imx7ulp-lpuart";
+ reg = <0x29870000 0x1000>;
+ interrupts = <GIC_SPI 102 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&pcc4 IMX8ULP_CLK_LPUART7>;
+ clock-names = "ipg";
+ status = "disabled";
+ };
+
+ iomuxc1: pinctrl@298c0000 {
+ compatible = "fsl,imx8ulp-iomuxc1";
+ reg = <0x298c0000 0x10000>;
+ };
+
+ usdhc0: mmc@298d0000 {
+ compatible = "fsl,imx8ulp-usdhc", "fsl,imx8mm-usdhc";
+ reg = <0x298d0000 0x10000>;
+ interrupts = <GIC_SPI 142 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&cgc1 IMX8ULP_CLK_XBAR_DIVBUS>,
+ <&cgc1 IMX8ULP_CLK_XBAR_AD_DIVPLAT>,
+ <&pcc4 IMX8ULP_CLK_USDHC0>;
+ clock-names = "ipg", "ahb", "per";
+ power-domains = <&scmi_devpd IMX8ULP_PD_USDHC0>;
+ fsl,tuning-start-tap = <20>;
+ fsl,tuning-step= <2>;
+ bus-width = <4>;
+ status = "disabled";
+ };
+
+ usdhc1: mmc@298e0000 {
+ compatible = "fsl,imx8ulp-usdhc", "fsl,imx8mm-usdhc";
+ reg = <0x298e0000 0x10000>;
+ interrupts = <GIC_SPI 143 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&cgc1 IMX8ULP_CLK_XBAR_DIVBUS>,
+ <&cgc1 IMX8ULP_CLK_NIC_PER_DIVPLAT>,
+ <&pcc4 IMX8ULP_CLK_USDHC1>;
+ clock-names = "ipg", "ahb", "per";
+ power-domains = <&scmi_devpd IMX8ULP_PD_USDHC1>;
+ fsl,tuning-start-tap = <20>;
+ fsl,tuning-step= <2>;
+ bus-width = <4>;
+ status = "disabled";
+ };
+
+ usdhc2: mmc@298f0000 {
+ compatible = "fsl,imx8ulp-usdhc", "fsl,imx8mm-usdhc";
+ reg = <0x298f0000 0x10000>;
+ interrupts = <GIC_SPI 144 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&cgc1 IMX8ULP_CLK_XBAR_DIVBUS>,
+ <&cgc1 IMX8ULP_CLK_NIC_PER_DIVPLAT>,
+ <&pcc4 IMX8ULP_CLK_USDHC2>;
+ clock-names = "ipg", "ahb", "per";
+ power-domains = <&scmi_devpd IMX8ULP_PD_USDHC2_USB1>;
+ fsl,tuning-start-tap = <20>;
+ fsl,tuning-step= <2>;
+ bus-width = <4>;
+ status = "disabled";
+ };
+ };
+
+ gpioe: gpio@2d000000 {
+ compatible = "fsl,imx8ulp-gpio", "fsl,imx7ulp-gpio";
+ reg = <0x2d000080 0x1000>, <0x2d000040 0x40>;
+ gpio-controller;
+ #gpio-cells = <2>;
+ interrupts = <GIC_SPI 129 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-controller;
+ #interrupt-cells = <2>;
+ clocks = <&pcc4 IMX8ULP_CLK_RGPIOE>,
+ <&pcc4 IMX8ULP_CLK_PCTLE>;
+ clock-names = "gpio", "port";
+ gpio-ranges = <&iomuxc1 0 32 24>;
+ };
+
+ gpiof: gpio@2d010000 {
+ compatible = "fsl,imx8ulp-gpio", "fsl,imx7ulp-gpio";
+ reg = <0x2d010080 0x1000>, <0x2d010040 0x40>;
+ gpio-controller;
+ #gpio-cells = <2>;
+ interrupts = <GIC_SPI 131 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-controller;
+ #interrupt-cells = <2>;
+ clocks = <&pcc4 IMX8ULP_CLK_RGPIOF>,
+ <&pcc4 IMX8ULP_CLK_PCTLF>;
+ clock-names = "gpio", "port";
+ gpio-ranges = <&iomuxc1 0 64 32>;
+ };
+
+ per_bridge5: bus@2d800000 {
+ compatible = "simple-bus";
+ reg = <0x2d800000 0x800000>;
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges;
+
+ cgc2: clock-controller@2da60000 {
+ compatible = "fsl,imx8ulp-cgc2";
+ reg = <0x2da60000 0x10000>;
+ clocks = <&sosc>, <&frosc>;
+ clock-names = "sosc", "frosc";
+ #clock-cells = <1>;
+ };
+
+ pcc5: clock-controller@2da70000 {
+ compatible = "fsl,imx8ulp-pcc5";
+ reg = <0x2da70000 0x10000>;
+ #clock-cells = <1>;
+ };
+ };
+
+ gpiod: gpio@2e200000 {
+ compatible = "fsl,imx8ulp-gpio", "fsl,imx7ulp-gpio";
+ reg = <0x2e200080 0x1000>, <0x2e200040 0x40>;
+ gpio-controller;
+ #gpio-cells = <2>;
+ interrupts = <GIC_SPI 127 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-controller;
+ #interrupt-cells = <2>;
+ clocks = <&pcc5 IMX8ULP_CLK_RGPIOD>,
+ <&pcc5 IMX8ULP_CLK_RGPIOD>;
+ clock-names = "gpio", "port";
+ gpio-ranges = <&iomuxc1 0 0 24>;
+ };
+ };
+};
diff --git a/arch/arm64/boot/dts/freescale/mba8mx.dtsi b/arch/arm64/boot/dts/freescale/mba8mx.dtsi
new file mode 100644
index 0000000..f27e3c8
--- /dev/null
+++ b/arch/arm64/boot/dts/freescale/mba8mx.dtsi
@@ -0,0 +1,282 @@
+// SPDX-License-Identifier: (GPL-2.0-or-later OR MIT)
+/*
+ * Copyright 2020-2021 TQ-Systems GmbH
+ */
+
+#include <dt-bindings/net/ti-dp83867.h>
+
+/* TQ-Systems GmbH MBa8Mx baseboard */
+
+/ {
+ beeper {
+ compatible = "pwm-beeper";
+ pwms = <&pwm4 0 250000 0>;
+ beeper-hz = <4000>;
+ amp-supply = <®_vcc_3v3>;
+ };
+
+ chosen {
+ // bootargs = "console=ttymxc2,115200 earlycon=ec_imx6q,0x30880000,115200";
+ stdout-path = &uart3;
+ };
+
+ gpio-keys {
+ compatible = "gpio-keys";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_gpiobutton>;
+ autorepeat;
+
+ switch1 {
+ label = "switch1";
+ linux,code = <BTN_0>;
+ gpios = <&gpio1 5 GPIO_ACTIVE_LOW>;
+ wakeup-source;
+ };
+
+ btn2: switch2 {
+ label = "switch2";
+ linux,code = <BTN_1>;
+ gpios = <&gpio2 0 GPIO_ACTIVE_LOW>;
+ wakeup-source;
+ };
+
+ switch3 {
+ label = "switch3";
+ linux,code = <BTN_2>;
+ gpios = <&gpio1 7 GPIO_ACTIVE_LOW>;
+ wakeup-source;
+ };
+ };
+
+ gpio_leds: gpio-leds {
+ compatible = "gpio-leds";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_gpioled>;
+
+ led1 {
+ label = "led1";
+ gpios = <&gpio1 0 GPIO_ACTIVE_HIGH>;
+ linux,default-trigger = "default-on";
+ };
+
+ led2: led2 {
+ label = "led2";
+ gpios = <&gpio3 14 GPIO_ACTIVE_HIGH>;
+ linux,default-trigger = "heartbeat";
+ };
+ };
+
+ reg_hub_vbus: regulator-hub-vbus {
+ compatible = "regulator-fixed";
+ regulator-name = "MBA8MX_HUB_VBUS";
+ regulator-min-microvolt = <5000000>;
+ regulator-max-microvolt = <5000000>;
+ };
+
+ reg_sn65dsi83_1v8: regulator-sn65dsi83-1v8 {
+ compatible = "regulator-fixed";
+ regulator-name = "SN65DSI83_1V8";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ gpio = <&expander0 5 GPIO_ACTIVE_HIGH>;
+ enable-active-high;
+ };
+
+ reg_vcc_3v3: regulator-3v3 {
+ compatible = "regulator-fixed";
+ regulator-name = "MBA8MX_3V3";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ };
+
+ sound {
+ compatible = "fsl,imx-audio-tlv320aic32x4";
+ model = "tqm-tlv320aic32";
+ ssi-controller = <&sai3>;
+ audio-codec = <&tlv320aic3x04>;
+ };
+};
+
+&ecspi1 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_ecspi1>;
+ cs-gpios = <&gpio5 9 GPIO_ACTIVE_LOW>;
+ status = "okay";
+};
+
+&ecspi2 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_ecspi2>;
+ cs-gpios = <&gpio5 13 GPIO_ACTIVE_LOW>;
+ status = "okay";
+};
+
+&fec1 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_fec1>;
+ phy-mode = "rgmii-id";
+ phy-handle = <ðphy0>;
+ phy-supply = <®_vcc_3v3>;
+ fsl,magic-packet;
+ mac-address = [ 00 00 00 00 00 00 ];
+ status = "okay";
+
+ mdio {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ ethphy0: ethernet-phy@e {
+ compatible = "ethernet-phy-ieee802.3-c22";
+ reg = <0xe>;
+ ti,rx-internal-delay = <DP83867_RGMIIDCTL_2_50_NS>;
+ ti,tx-internal-delay = <DP83867_RGMIIDCTL_2_50_NS>;
+ ti,fifo-depth = <DP83867_PHYCR_FIFO_DEPTH_4_B_NIB>;
+ ti,dp83867-rxctrl-strap-quirk;
+ ti,clk-output-sel = <DP83867_CLK_O_SEL_OFF>;
+ enet-phy-lane-no-swap;
+ reset-gpios = <&expander2 7 GPIO_ACTIVE_LOW>;
+ reset-assert-us = <500000>;
+ reset-deassert-us = <500>;
+ };
+ };
+};
+
+&i2c1 {
+ expander0: gpio@23 {
+ compatible = "nxp,pca9555";
+ reg = <0x23>;
+ gpio-controller;
+ #gpio-cells = <2>;
+ vcc-supply = <®_vcc_3v3>;
+ interrupt-parent = <&gpio1>;
+ interrupts = <9 IRQ_TYPE_EDGE_FALLING>;
+ interrupt-controller;
+ #interrupt-cells = <2>;
+
+ sd-mux-oe-hog {
+ gpio-hog;
+ gpios = <8 0>;
+ output-low;
+ line-name = "SD_MUX_EN#";
+ };
+
+ boot-cfg-oe-hog {
+ gpio-hog;
+ gpios = <12 0>;
+ output-high;
+ line-name = "BOOT_CFG_OE#";
+ };
+
+ rst-usb-hub-hog {
+ gpio-hog;
+ gpios = <13 0>;
+ output-high;
+ line-name = "RST_USB_HUB#";
+ };
+ };
+
+ expander1: gpio@24 {
+ compatible = "nxp,pca9555";
+ reg = <0x24>;
+ gpio-controller;
+ #gpio-cells = <2>;
+ vcc-supply = <®_vcc_3v3>;
+ };
+};
+
+&i2c2 {
+ clock-frequency = <100000>;
+ pinctrl-names = "default", "gpio";
+ pinctrl-0 = <&pinctrl_i2c2>;
+ pinctrl-1 = <&pinctrl_i2c2_gpio>;
+ scl-gpios = <&gpio5 16 (GPIO_ACTIVE_HIGH | GPIO_OPEN_DRAIN)>;
+ sda-gpios = <&gpio5 17 (GPIO_ACTIVE_HIGH | GPIO_OPEN_DRAIN)>;
+ status = "okay";
+
+ tlv320aic3x04: audio-codec@18 {
+ compatible = "ti,tlv320aic32x4";
+ reg = <0x18>;
+ reset-gpios = <&expander2 0 GPIO_ACTIVE_LOW>;
+ iov-supply = <®_vcc_3v3>;
+ ldoin-supply = <®_vcc_3v3>;
+ };
+
+ sensor1: sensor@1f {
+ compatible = "nxp,se97", "jedec,jc-42.4-temp";
+ reg = <0x1f>;
+ };
+
+ eeprom3: eeprom@57 {
+ compatible = "nxp,se97b", "atmel,24c02";
+ reg = <0x57>;
+ pagesize = <16>;
+ };
+};
+
+&i2c3 {
+ clock-frequency = <100000>;
+ pinctrl-names = "default", "gpio";
+ pinctrl-0 = <&pinctrl_i2c3>;
+ pinctrl-1 = <&pinctrl_i2c3_gpio>;
+ scl-gpios = <&gpio5 18 (GPIO_ACTIVE_HIGH | GPIO_OPEN_DRAIN)>;
+ sda-gpios = <&gpio5 19 (GPIO_ACTIVE_HIGH | GPIO_OPEN_DRAIN)>;
+ status = "okay";
+};
+
+&pwm3 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_pwm3>;
+ status = "okay";
+};
+
+&pwm4 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_pwm4>;
+ status = "okay";
+};
+
+&sai3 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_sai3>;
+ #sound-dai-cells = <0>;
+ assigned-clock-rates = <49152000>;
+ status = "okay";
+};
+
+&snvs_pwrkey {
+ status = "okay";
+};
+
+&uart1 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_uart1>;
+ status = "okay";
+};
+
+&uart2 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_uart2>;
+ status = "okay";
+};
+
+/* console */
+&uart3 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_uart3>;
+ status = "okay";
+};
+
+/* UART4 is assigned to Cortex-M4 */
+&usdhc2 {
+ pinctrl-names = "default", "state_100mhz", "state_200mhz";
+ pinctrl-0 = <&pinctrl_usdhc2>, <&pinctrl_usdhc2_gpio>;
+ pinctrl-1 = <&pinctrl_usdhc2_100mhz>, <&pinctrl_usdhc2_gpio>;
+ pinctrl-2 = <&pinctrl_usdhc2_200mhz>, <&pinctrl_usdhc2_gpio>;
+ bus-width = <4>;
+ cd-gpios = <&gpio2 12 GPIO_ACTIVE_LOW>;
+ disable-wp;
+ no-mmc;
+ no-sdio;
+ vmmc-supply = <®_usdhc2_vmmc>;
+ status = "okay";
+};
diff --git a/arch/arm64/boot/dts/intel/socfpga_agilex.dtsi b/arch/arm64/boot/dts/intel/socfpga_agilex.dtsi
index 163f33b..0dd2d2e 100644
--- a/arch/arm64/boot/dts/intel/socfpga_agilex.dtsi
+++ b/arch/arm64/boot/dts/intel/socfpga_agilex.dtsi
@@ -628,7 +628,7 @@ sdmmca-ecc@ff8c8c00 {
};
qspi: spi@ff8d2000 {
- compatible = "cdns,qspi-nor";
+ compatible = "intel,socfpga-qspi", "cdns,qspi-nor";
#address-cells = <1>;
#size-cells = <0>;
reg = <0xff8d2000 0x100>,
diff --git a/arch/arm64/boot/dts/intel/socfpga_agilex_socdk_nand.dts b/arch/arm64/boot/dts/intel/socfpga_agilex_socdk_nand.dts
index cc2dcab..57f8348 100644
--- a/arch/arm64/boot/dts/intel/socfpga_agilex_socdk_nand.dts
+++ b/arch/arm64/boot/dts/intel/socfpga_agilex_socdk_nand.dts
@@ -91,24 +91,8 @@ partition@0 {
reg = <0 0x200000>;
};
partition@200000 {
- label = "env";
- reg = <0x200000 0x40000>;
- };
- partition@240000 {
- label = "dtb";
- reg = <0x240000 0x40000>;
- };
- partition@280000 {
- label = "kernel";
- reg = <0x280000 0x2000000>;
- };
- partition@2280000 {
- label = "misc";
- reg = <0x2280000 0x2000000>;
- };
- partition@4280000 {
- label = "rootfs";
- reg = <0x4280000 0x3bd80000>;
+ label = "root";
+ reg = <0x200000 0x3fe00000>;
};
};
};
diff --git a/arch/arm64/boot/dts/intel/socfpga_n5x_socdk.dts b/arch/arm64/boot/dts/intel/socfpga_n5x_socdk.dts
index 01f1307..f3c1310 100644
--- a/arch/arm64/boot/dts/intel/socfpga_n5x_socdk.dts
+++ b/arch/arm64/boot/dts/intel/socfpga_n5x_socdk.dts
@@ -29,6 +29,36 @@ &clkmgr {
compatible = "intel,easic-n5x-clkmgr";
};
+&gmac0 {
+ status = "okay";
+ phy-mode = "rgmii";
+ phy-handle = <&phy0>;
+
+ max-frame-size = <9000>;
+
+ mdio0 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ compatible = "snps,dwmac-mdio";
+ phy0: ethernet-phy@0 {
+ reg = <4>;
+
+ txd0-skew-ps = <0>; /* -420ps */
+ txd1-skew-ps = <0>; /* -420ps */
+ txd2-skew-ps = <0>; /* -420ps */
+ txd3-skew-ps = <0>; /* -420ps */
+ rxd0-skew-ps = <420>; /* 0ps */
+ rxd1-skew-ps = <420>; /* 0ps */
+ rxd2-skew-ps = <420>; /* 0ps */
+ rxd3-skew-ps = <420>; /* 0ps */
+ txen-skew-ps = <0>; /* -420ps */
+ txc-skew-ps = <900>; /* 0ps */
+ rxdv-skew-ps = <420>; /* 0ps */
+ rxc-skew-ps = <1680>; /* 780ps */
+ };
+ };
+};
+
&mmc {
status = "okay";
cap-sd-highspeed;
@@ -40,10 +70,51 @@ &osc1 {
clock-frequency = <25000000>;
};
+&qspi {
+ status = "okay";
+ flash@0 {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ compatible = "micron,mt25qu02g", "jedec,spi-nor";
+ reg = <0>;
+ spi-max-frequency = <100000000>;
+
+ m25p,fast-read;
+ cdns,page-size = <256>;
+ cdns,block-size = <16>;
+ cdns,read-delay = <2>;
+ cdns,tshsl-ns = <50>;
+ cdns,tsd2d-ns = <50>;
+ cdns,tchsh-ns = <4>;
+ cdns,tslch-ns = <4>;
+
+ partitions {
+ compatible = "fixed-partitions";
+ #address-cells = <1>;
+ #size-cells = <1>;
+
+ qspi_boot: partition@0 {
+ label = "Boot and fpga data";
+ reg = <0x0 0x03FE0000>;
+ };
+
+ qspi_rootfs: partition@3FE0000 {
+ label = "Root Filesystem - JFFS2";
+ reg = <0x03FE0000 0x0C020000>;
+ };
+ };
+ };
+};
+
&uart0 {
status = "okay";
};
+&usb0 {
+ status = "okay";
+ disable-over-current;
+};
+
&watchdog0 {
status = "okay";
};
diff --git a/arch/arm64/boot/dts/marvell/armada-37xx.dtsi b/arch/arm64/boot/dts/marvell/armada-37xx.dtsi
index 9acc5d2..673f490 100644
--- a/arch/arm64/boot/dts/marvell/armada-37xx.dtsi
+++ b/arch/arm64/boot/dts/marvell/armada-37xx.dtsi
@@ -265,6 +265,8 @@ comphy: phy@18300 {
"lane2_sata_usb3";
#address-cells = <1>;
#size-cells = <0>;
+ clocks = <&xtalclk>;
+ clock-names = "xtal";
comphy0: phy@0 {
reg = <0>;
diff --git a/arch/arm64/boot/dts/marvell/cn9130-crb.dtsi b/arch/arm64/boot/dts/marvell/cn9130-crb.dtsi
index 505ae69..d9f9f2c 100644
--- a/arch/arm64/boot/dts/marvell/cn9130-crb.dtsi
+++ b/arch/arm64/boot/dts/marvell/cn9130-crb.dtsi
@@ -17,6 +17,8 @@ aliases {
ethernet0 = &cp0_eth0;
ethernet1 = &cp0_eth1;
ethernet2 = &cp0_eth2;
+ gpio1 = &cp0_gpio1;
+ gpio2 = &cp0_gpio2;
};
memory@0 {
@@ -71,6 +73,17 @@ cp0_reg_sd_vcc: cp0_sd_vcc@0 {
enable-active-high;
regulator-always-on;
};
+
+ sfp: sfp {
+ compatible = "sff,sfp";
+ i2c-bus = <&cp0_i2c1>;
+ mod-def0-gpios = <&expander0 3 GPIO_ACTIVE_LOW>;
+ los-gpio = <&expander0 15 GPIO_ACTIVE_HIGH>;
+ tx-disable-gpio = <&expander0 2 GPIO_ACTIVE_HIGH>;
+ tx-fault-gpio = <&cp0_gpio1 24 GPIO_ACTIVE_HIGH>;
+ maximum-power-milliwatt = <3000>;
+ status = "okay";
+ };
};
&uart0 {
@@ -114,6 +127,14 @@ cp0_spi0_pins: cp0-spi-pins-0 {
};
};
+&cp0_gpio1 {
+ status = "okay";
+};
+
+&cp0_gpio2 {
+ status = "okay";
+};
+
&cp0_i2c0 {
pinctrl-names = "default";
pinctrl-0 = <&cp0_i2c0_pins>;
@@ -185,6 +206,125 @@ &cp0_mdio {
phy0: ethernet-phy@0 {
reg = <0>;
};
+
+ switch6: switch0@6 {
+ /* Actual device is MV88E6393X */
+ compatible = "marvell,mv88e6190";
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <6>;
+ interrupt-parent = <&cp0_gpio1>;
+ interrupts = <28 IRQ_TYPE_LEVEL_LOW>;
+ interrupt-controller;
+ #interrupt-cells = <2>;
+
+ dsa,member = <0 0>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@1 {
+ reg = <1>;
+ label = "p1";
+ phy-handle = <&switch0phy1>;
+ };
+
+ port@2 {
+ reg = <2>;
+ label = "p2";
+ phy-handle = <&switch0phy2>;
+ };
+
+ port@3 {
+ reg = <3>;
+ label = "p3";
+ phy-handle = <&switch0phy3>;
+ };
+
+ port@4 {
+ reg = <4>;
+ label = "p4";
+ phy-handle = <&switch0phy4>;
+ };
+
+ port@5 {
+ reg = <5>;
+ label = "p5";
+ phy-handle = <&switch0phy5>;
+ };
+
+ port@6 {
+ reg = <6>;
+ label = "p6";
+ phy-handle = <&switch0phy6>;
+ };
+
+ port@7 {
+ reg = <7>;
+ label = "p7";
+ phy-handle = <&switch0phy7>;
+ };
+
+ port@8 {
+ reg = <8>;
+ label = "p8";
+ phy-handle = <&switch0phy8>;
+ };
+
+ port@9 {
+ reg = <9>;
+ label = "p9";
+ phy-mode = "10gbase-r";
+ sfp = <&sfp>;
+ managed = "in-band-status";
+ };
+
+ port@a {
+ reg = <10>;
+ label = "cpu";
+ ethernet = <&cp0_eth0>;
+ };
+
+ };
+
+ mdio {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ switch0phy1: switch0phy1@1 {
+ reg = <0x1>;
+ };
+
+ switch0phy2: switch0phy2@2 {
+ reg = <0x2>;
+ };
+
+ switch0phy3: switch0phy3@3 {
+ reg = <0x3>;
+ };
+
+ switch0phy4: switch0phy4@4 {
+ reg = <0x4>;
+ };
+
+ switch0phy5: switch0phy5@5 {
+ reg = <0x5>;
+ };
+
+ switch0phy6: switch0phy6@6 {
+ reg = <0x6>;
+ };
+
+ switch0phy7: switch0phy7@7 {
+ reg = <0x7>;
+ };
+
+ switch0phy8: switch0phy8@8 {
+ reg = <0x8>;
+ };
+ };
+ };
};
&cp0_xmdio {
diff --git a/arch/arm64/boot/dts/marvell/cn9130.dtsi b/arch/arm64/boot/dts/marvell/cn9130.dtsi
index a2b7e5e..327b041 100644
--- a/arch/arm64/boot/dts/marvell/cn9130.dtsi
+++ b/arch/arm64/boot/dts/marvell/cn9130.dtsi
@@ -11,6 +11,13 @@ / {
model = "Marvell Armada CN9130 SoC";
compatible = "marvell,cn9130", "marvell,armada-ap807-quad",
"marvell,armada-ap807";
+
+ aliases {
+ gpio1 = &cp0_gpio1;
+ gpio2 = &cp0_gpio2;
+ spi1 = &cp0_spi0;
+ spi2 = &cp0_spi1;
+ };
};
/*
@@ -35,3 +42,11 @@ / {
#undef CP11X_PCIE0_BASE
#undef CP11X_PCIE1_BASE
#undef CP11X_PCIE2_BASE
+
+&cp0_gpio1 {
+ status = "okay";
+};
+
+&cp0_gpio2 {
+ status = "okay";
+};
diff --git a/arch/arm64/boot/dts/mediatek/Makefile b/arch/arm64/boot/dts/mediatek/Makefile
index 4f68ebe..8c1e180 100644
--- a/arch/arm64/boot/dts/mediatek/Makefile
+++ b/arch/arm64/boot/dts/mediatek/Makefile
@@ -7,6 +7,8 @@
dtb-$(CONFIG_ARCH_MEDIATEK) += mt6797-x20-dev.dtb
dtb-$(CONFIG_ARCH_MEDIATEK) += mt7622-rfb1.dtb
dtb-$(CONFIG_ARCH_MEDIATEK) += mt7622-bananapi-bpi-r64.dtb
+dtb-$(CONFIG_ARCH_MEDIATEK) += mt7986a-rfb.dtb
+dtb-$(CONFIG_ARCH_MEDIATEK) += mt7986b-rfb.dtb
dtb-$(CONFIG_ARCH_MEDIATEK) += mt8167-pumpkin.dtb
dtb-$(CONFIG_ARCH_MEDIATEK) += mt8173-elm.dtb
dtb-$(CONFIG_ARCH_MEDIATEK) += mt8173-elm-hana.dtb
@@ -14,16 +16,20 @@
dtb-$(CONFIG_ARCH_MEDIATEK) += mt8173-evb.dtb
dtb-$(CONFIG_ARCH_MEDIATEK) += mt8183-evb.dtb
dtb-$(CONFIG_ARCH_MEDIATEK) += mt8183-kukui-jacuzzi-burnet.dtb
+dtb-$(CONFIG_ARCH_MEDIATEK) += mt8183-kukui-jacuzzi-cozmo.dtb
dtb-$(CONFIG_ARCH_MEDIATEK) += mt8183-kukui-jacuzzi-damu.dtb
dtb-$(CONFIG_ARCH_MEDIATEK) += mt8183-kukui-jacuzzi-fennel-sku1.dtb
dtb-$(CONFIG_ARCH_MEDIATEK) += mt8183-kukui-jacuzzi-fennel-sku6.dtb
+dtb-$(CONFIG_ARCH_MEDIATEK) += mt8183-kukui-jacuzzi-fennel-sku7.dtb
dtb-$(CONFIG_ARCH_MEDIATEK) += mt8183-kukui-jacuzzi-fennel14.dtb
+dtb-$(CONFIG_ARCH_MEDIATEK) += mt8183-kukui-jacuzzi-fennel14-sku2.dtb
dtb-$(CONFIG_ARCH_MEDIATEK) += mt8183-kukui-jacuzzi-juniper-sku16.dtb
dtb-$(CONFIG_ARCH_MEDIATEK) += mt8183-kukui-jacuzzi-kappa.dtb
dtb-$(CONFIG_ARCH_MEDIATEK) += mt8183-kukui-jacuzzi-kenzo.dtb
dtb-$(CONFIG_ARCH_MEDIATEK) += mt8183-kukui-jacuzzi-willow-sku0.dtb
dtb-$(CONFIG_ARCH_MEDIATEK) += mt8183-kukui-jacuzzi-willow-sku1.dtb
dtb-$(CONFIG_ARCH_MEDIATEK) += mt8183-kukui-kakadu.dtb
+dtb-$(CONFIG_ARCH_MEDIATEK) += mt8183-kukui-kakadu-sku22.dtb
dtb-$(CONFIG_ARCH_MEDIATEK) += mt8183-kukui-kodama-sku16.dtb
dtb-$(CONFIG_ARCH_MEDIATEK) += mt8183-kukui-kodama-sku272.dtb
dtb-$(CONFIG_ARCH_MEDIATEK) += mt8183-kukui-kodama-sku288.dtb
diff --git a/arch/arm64/boot/dts/mediatek/mt7986a-rfb.dts b/arch/arm64/boot/dts/mediatek/mt7986a-rfb.dts
new file mode 100644
index 0000000..5cd760a
--- /dev/null
+++ b/arch/arm64/boot/dts/mediatek/mt7986a-rfb.dts
@@ -0,0 +1,57 @@
+// SPDX-License-Identifier: (GPL-2.0 OR MIT)
+/*
+ * Copyright (C) 2021 MediaTek Inc.
+ * Author: Sam.Shih <sam.shih@mediatek.com>
+ */
+
+/dts-v1/;
+#include "mt7986a.dtsi"
+
+/ {
+ model = "MediaTek MT7986a RFB";
+ compatible = "mediatek,mt7986a-rfb";
+
+ aliases {
+ serial0 = &uart0;
+ };
+
+ chosen {
+ stdout-path = "serial0:115200n8";
+ };
+
+ memory {
+ reg = <0 0x40000000 0 0x40000000>;
+ };
+};
+
+&uart0 {
+ status = "okay";
+};
+
+&uart1 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&uart1_pins>;
+ status = "okay";
+};
+
+&uart2 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&uart2_pins>;
+ status = "okay";
+};
+
+&pio {
+ uart1_pins: uart1-pins {
+ mux {
+ function = "uart";
+ groups = "uart1";
+ };
+ };
+
+ uart2_pins: uart2-pins {
+ mux {
+ function = "uart";
+ groups = "uart2";
+ };
+ };
+};
diff --git a/arch/arm64/boot/dts/mediatek/mt7986a.dtsi b/arch/arm64/boot/dts/mediatek/mt7986a.dtsi
new file mode 100644
index 0000000..b8da76b
--- /dev/null
+++ b/arch/arm64/boot/dts/mediatek/mt7986a.dtsi
@@ -0,0 +1,169 @@
+// SPDX-License-Identifier: (GPL-2.0 OR MIT)
+/*
+ * Copyright (C) 2021 MediaTek Inc.
+ * Author: Sam.Shih <sam.shih@mediatek.com>
+ */
+
+#include <dt-bindings/interrupt-controller/irq.h>
+#include <dt-bindings/interrupt-controller/arm-gic.h>
+
+/ {
+ interrupt-parent = <&gic>;
+ #address-cells = <2>;
+ #size-cells = <2>;
+
+ system_clk: dummy40m {
+ compatible = "fixed-clock";
+ clock-frequency = <40000000>;
+ #clock-cells = <0>;
+ };
+
+ cpus {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ cpu0: cpu@0 {
+ device_type = "cpu";
+ compatible = "arm,cortex-a53";
+ enable-method = "psci";
+ reg = <0x0>;
+ #cooling-cells = <2>;
+ };
+
+ cpu1: cpu@1 {
+ device_type = "cpu";
+ compatible = "arm,cortex-a53";
+ enable-method = "psci";
+ reg = <0x1>;
+ #cooling-cells = <2>;
+ };
+
+ cpu2: cpu@2 {
+ device_type = "cpu";
+ compatible = "arm,cortex-a53";
+ enable-method = "psci";
+ reg = <0x2>;
+ #cooling-cells = <2>;
+ };
+
+ cpu3: cpu@3 {
+ device_type = "cpu";
+ enable-method = "psci";
+ compatible = "arm,cortex-a53";
+ reg = <0x3>;
+ #cooling-cells = <2>;
+ };
+ };
+
+ psci {
+ compatible = "arm,psci-0.2";
+ method = "smc";
+ };
+
+ reserved-memory {
+ #address-cells = <2>;
+ #size-cells = <2>;
+ ranges;
+ /* 192 KiB reserved for ARM Trusted Firmware (BL31) */
+ secmon_reserved: secmon@43000000 {
+ reg = <0 0x43000000 0 0x30000>;
+ no-map;
+ };
+ };
+
+ timer {
+ compatible = "arm,armv8-timer";
+ interrupt-parent = <&gic>;
+ interrupts = <GIC_PPI 13 IRQ_TYPE_LEVEL_LOW>,
+ <GIC_PPI 14 IRQ_TYPE_LEVEL_LOW>,
+ <GIC_PPI 11 IRQ_TYPE_LEVEL_LOW>,
+ <GIC_PPI 10 IRQ_TYPE_LEVEL_LOW>;
+ };
+
+ soc {
+ #address-cells = <2>;
+ #size-cells = <2>;
+ compatible = "simple-bus";
+ ranges;
+
+ gic: interrupt-controller@c000000 {
+ compatible = "arm,gic-v3";
+ #interrupt-cells = <3>;
+ interrupt-parent = <&gic>;
+ interrupt-controller;
+ reg = <0 0x0c000000 0 0x10000>, /* GICD */
+ <0 0x0c080000 0 0x80000>, /* GICR */
+ <0 0x0c400000 0 0x2000>, /* GICC */
+ <0 0x0c410000 0 0x1000>, /* GICH */
+ <0 0x0c420000 0 0x2000>; /* GICV */
+ interrupts = <GIC_PPI 9 IRQ_TYPE_LEVEL_HIGH>;
+ };
+
+ watchdog: watchdog@1001c000 {
+ compatible = "mediatek,mt7986-wdt",
+ "mediatek,mt6589-wdt";
+ reg = <0 0x1001c000 0 0x1000>;
+ interrupts = <GIC_SPI 110 IRQ_TYPE_LEVEL_HIGH>;
+ #reset-cells = <1>;
+ status = "disabled";
+ };
+
+ pio: pinctrl@1001f000 {
+ compatible = "mediatek,mt7986a-pinctrl";
+ reg = <0 0x1001f000 0 0x1000>,
+ <0 0x11c30000 0 0x1000>,
+ <0 0x11c40000 0 0x1000>,
+ <0 0x11e20000 0 0x1000>,
+ <0 0x11e30000 0 0x1000>,
+ <0 0x11f00000 0 0x1000>,
+ <0 0x11f10000 0 0x1000>,
+ <0 0x1000b000 0 0x1000>;
+ reg-names = "gpio", "iocfg_rt", "iocfg_rb", "iocfg_lt",
+ "iocfg_lb", "iocfg_tr", "iocfg_tl", "eint";
+ gpio-controller;
+ #gpio-cells = <2>;
+ gpio-ranges = <&pio 0 0 100>;
+ interrupt-controller;
+ interrupts = <GIC_SPI 225 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-parent = <&gic>;
+ #interrupt-cells = <2>;
+ };
+
+ trng: trng@1020f000 {
+ compatible = "mediatek,mt7986-rng",
+ "mediatek,mt7623-rng";
+ reg = <0 0x1020f000 0 0x100>;
+ clocks = <&system_clk>;
+ clock-names = "rng";
+ status = "disabled";
+ };
+
+ uart0: serial@11002000 {
+ compatible = "mediatek,mt7986-uart",
+ "mediatek,mt6577-uart";
+ reg = <0 0x11002000 0 0x400>;
+ interrupts = <GIC_SPI 123 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&system_clk>;
+ status = "disabled";
+ };
+
+ uart1: serial@11003000 {
+ compatible = "mediatek,mt7986-uart",
+ "mediatek,mt6577-uart";
+ reg = <0 0x11003000 0 0x400>;
+ interrupts = <GIC_SPI 124 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&system_clk>;
+ status = "disabled";
+ };
+
+ uart2: serial@11004000 {
+ compatible = "mediatek,mt7986-uart",
+ "mediatek,mt6577-uart";
+ reg = <0 0x11004000 0 0x400>;
+ interrupts = <GIC_SPI 125 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&system_clk>;
+ status = "disabled";
+ };
+
+ };
+
+};
diff --git a/arch/arm64/boot/dts/mediatek/mt7986b-rfb.dts b/arch/arm64/boot/dts/mediatek/mt7986b-rfb.dts
new file mode 100644
index 0000000..5fb752e
--- /dev/null
+++ b/arch/arm64/boot/dts/mediatek/mt7986b-rfb.dts
@@ -0,0 +1,29 @@
+// SPDX-License-Identifier: (GPL-2.0 OR MIT)
+/*
+ * Copyright (C) 2021 MediaTek Inc.
+ * Author: Sam.Shih <sam.shih@mediatek.com>
+ */
+
+/dts-v1/;
+#include "mt7986b.dtsi"
+
+/ {
+ model = "MediaTek MT7986b RFB";
+ compatible = "mediatek,mt7986b-rfb";
+
+ aliases {
+ serial0 = &uart0;
+ };
+
+ chosen {
+ stdout-path = "serial0:115200n8";
+ };
+
+ memory {
+ reg = <0 0x40000000 0 0x40000000>;
+ };
+};
+
+&uart0 {
+ status = "okay";
+};
diff --git a/arch/arm64/boot/dts/mediatek/mt7986b.dtsi b/arch/arm64/boot/dts/mediatek/mt7986b.dtsi
new file mode 100644
index 0000000..23923b9
--- /dev/null
+++ b/arch/arm64/boot/dts/mediatek/mt7986b.dtsi
@@ -0,0 +1,12 @@
+// SPDX-License-Identifier: (GPL-2.0 OR MIT)
+/*
+ * Copyright (C) 2021 MediaTek Inc.
+ * Author: Sam.Shih <sam.shih@mediatek.com>
+ */
+
+#include "mt7986a.dtsi"
+
+&pio {
+ compatible = "mediatek,mt7986b-pinctrl";
+ gpio-ranges = <&pio 0 0 41>, <&pio 66 66 35>;
+};
diff --git a/arch/arm64/boot/dts/mediatek/mt8173-elm.dtsi b/arch/arm64/boot/dts/mediatek/mt8173-elm.dtsi
index e666ebb..9c75fbb 100644
--- a/arch/arm64/boot/dts/mediatek/mt8173-elm.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8173-elm.dtsi
@@ -28,7 +28,7 @@ backlight: backlight {
enable-gpios = <&pio 95 GPIO_ACTIVE_HIGH>;
pinctrl-names = "default";
- pinctrl-0 = <&disp_pwm0_pins>;
+ pinctrl-0 = <&panel_backlight_en_pins>;
status = "okay";
};
@@ -827,6 +827,12 @@ pins_clk {
};
};
+ panel_backlight_en_pins: panel_backlight_en_pins {
+ pins1 {
+ pinmux = <MT8173_PIN_95_PCM_TX__FUNC_GPIO95>;
+ };
+ };
+
panel_fixed_pins: panel_fixed_pins {
pins1 {
pinmux = <MT8173_PIN_41_CMMCLK__FUNC_GPIO41>;
@@ -901,6 +907,8 @@ pins1 {
};
&pwm0 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&disp_pwm0_pins>;
status = "okay";
};
diff --git a/arch/arm64/boot/dts/mediatek/mt8173.dtsi b/arch/arm64/boot/dts/mediatek/mt8173.dtsi
index dee66e5..2b7d331 100644
--- a/arch/arm64/boot/dts/mediatek/mt8173.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8173.dtsi
@@ -1212,6 +1212,7 @@ ufoe@1401a000 {
interrupts = <GIC_SPI 191 IRQ_TYPE_LEVEL_LOW>;
power-domains = <&spm MT8173_POWER_DOMAIN_MM>;
clocks = <&mmsys CLK_MM_DISP_UFOE>;
+ mediatek,gce-client-reg = <&gce SUBSYS_1401XXXX 0xa000 0x1000>;
};
dsi0: dsi@1401b000 {
@@ -1316,6 +1317,7 @@ od@14023000 {
compatible = "mediatek,mt8173-disp-od";
reg = <0 0x14023000 0 0x1000>;
clocks = <&mmsys CLK_MM_DISP_OD>;
+ mediatek,gce-client-reg = <&gce SUBSYS_1402XXXX 0x3000 0x1000>;
};
hdmi0: hdmi@14025000 {
diff --git a/arch/arm64/boot/dts/mediatek/mt8183-evb.dts b/arch/arm64/boot/dts/mediatek/mt8183-evb.dts
index 7bc0a6a..f3fd3cc 100644
--- a/arch/arm64/boot/dts/mediatek/mt8183-evb.dts
+++ b/arch/arm64/boot/dts/mediatek/mt8183-evb.dts
@@ -36,6 +36,14 @@ scp_mem_reserved: scp_mem_region {
no-map;
};
};
+
+ ntc@0 {
+ compatible = "murata,ncp03wf104";
+ pullup-uv = <1800000>;
+ pullup-ohm = <390000>;
+ pulldown-ohm = <0>;
+ io-channels = <&auxadc 0>;
+ };
};
&auxadc {
diff --git a/arch/arm64/boot/dts/mediatek/mt8183-kukui-jacuzzi-cozmo.dts b/arch/arm64/boot/dts/mediatek/mt8183-kukui-jacuzzi-cozmo.dts
new file mode 100644
index 0000000..072133f
--- /dev/null
+++ b/arch/arm64/boot/dts/mediatek/mt8183-kukui-jacuzzi-cozmo.dts
@@ -0,0 +1,36 @@
+// SPDX-License-Identifier: (GPL-2.0 OR MIT)
+/*
+ * Copyright 2021 Google LLC
+ */
+
+/dts-v1/;
+#include "mt8183-kukui-jacuzzi.dtsi"
+#include "mt8183-kukui-audio-ts3a227e-max98357a.dtsi"
+
+/ {
+ model = "Google cozmo board";
+ compatible = "google,cozmo", "mediatek,mt8183";
+};
+
+&i2c_tunnel {
+ google,remote-bus = <0>;
+};
+
+&i2c2 {
+ trackpad@2c {
+ compatible = "hid-over-i2c";
+ reg = <0x2c>;
+ hid-descr-addr = <0x20>;
+
+ pinctrl-names = "default";
+ pinctrl-0 = <&trackpad_pins>;
+
+ interrupts-extended = <&pio 7 IRQ_TYPE_LEVEL_LOW>;
+
+ wakeup-source;
+ };
+};
+
+&qca_wifi {
+ qcom,ath10k-calibration-variant = "GO_COZMO";
+};
diff --git a/arch/arm64/boot/dts/mediatek/mt8183-kukui-jacuzzi-fennel-sku1.dts b/arch/arm64/boot/dts/mediatek/mt8183-kukui-jacuzzi-fennel-sku1.dts
index ef6257c..dec11a4 100644
--- a/arch/arm64/boot/dts/mediatek/mt8183-kukui-jacuzzi-fennel-sku1.dts
+++ b/arch/arm64/boot/dts/mediatek/mt8183-kukui-jacuzzi-fennel-sku1.dts
@@ -5,6 +5,7 @@
/dts-v1/;
#include "mt8183-kukui-jacuzzi-fennel.dtsi"
+#include "mt8183-kukui-audio-da7219-rt1015p.dtsi"
/ {
model = "Google fennel sku1 board";
diff --git a/arch/arm64/boot/dts/mediatek/mt8183-kukui-jacuzzi-fennel-sku6.dts b/arch/arm64/boot/dts/mediatek/mt8183-kukui-jacuzzi-fennel-sku6.dts
index 899c2e4..37e6e58 100644
--- a/arch/arm64/boot/dts/mediatek/mt8183-kukui-jacuzzi-fennel-sku6.dts
+++ b/arch/arm64/boot/dts/mediatek/mt8183-kukui-jacuzzi-fennel-sku6.dts
@@ -5,6 +5,7 @@
/dts-v1/;
#include "mt8183-kukui-jacuzzi-fennel.dtsi"
+#include "mt8183-kukui-audio-da7219-rt1015p.dtsi"
/ {
model = "Google fennel sku6 board";
diff --git a/arch/arm64/boot/dts/mediatek/mt8183-kukui-jacuzzi-fennel-sku7.dts b/arch/arm64/boot/dts/mediatek/mt8183-kukui-jacuzzi-fennel-sku7.dts
new file mode 100644
index 0000000..0e09604
--- /dev/null
+++ b/arch/arm64/boot/dts/mediatek/mt8183-kukui-jacuzzi-fennel-sku7.dts
@@ -0,0 +1,33 @@
+// SPDX-License-Identifier: (GPL-2.0 OR MIT)
+/*
+ * Copyright 2021 Google LLC
+ */
+
+/dts-v1/;
+#include "mt8183-kukui-jacuzzi-fennel.dtsi"
+#include "mt8183-kukui-audio-ts3a227e-rt1015p.dtsi"
+
+/ {
+ model = "Google fennel sku7 board";
+ compatible = "google,fennel-sku7", "google,fennel", "mediatek,mt8183";
+};
+
+&touchscreen {
+ status = "okay";
+
+ compatible = "hid-over-i2c";
+ reg = <0x10>;
+ interrupt-parent = <&pio>;
+ interrupts = <155 IRQ_TYPE_LEVEL_LOW>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&touchscreen_pins>;
+
+ post-power-on-delay-ms = <10>;
+ hid-descr-addr = <0x0001>;
+};
+
+
+&qca_wifi {
+ qcom,ath10k-calibration-variant = "GO_FENNEL";
+};
+
diff --git a/arch/arm64/boot/dts/mediatek/mt8183-kukui-jacuzzi-fennel.dtsi b/arch/arm64/boot/dts/mediatek/mt8183-kukui-jacuzzi-fennel.dtsi
index 577519a..bbe6c33 100644
--- a/arch/arm64/boot/dts/mediatek/mt8183-kukui-jacuzzi-fennel.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8183-kukui-jacuzzi-fennel.dtsi
@@ -5,7 +5,6 @@
/dts-v1/;
#include "mt8183-kukui-jacuzzi.dtsi"
-#include "mt8183-kukui-audio-da7219-rt1015p.dtsi"
&mt6358codec {
mediatek,dmic-mode = <1>; /* one-wire */
diff --git a/arch/arm64/boot/dts/mediatek/mt8183-kukui-jacuzzi-fennel14-sku2.dts b/arch/arm64/boot/dts/mediatek/mt8183-kukui-jacuzzi-fennel14-sku2.dts
new file mode 100644
index 0000000..3fc5a61
--- /dev/null
+++ b/arch/arm64/boot/dts/mediatek/mt8183-kukui-jacuzzi-fennel14-sku2.dts
@@ -0,0 +1,17 @@
+// SPDX-License-Identifier: (GPL-2.0 OR MIT)
+/*
+ * Copyright 2021 Google LLC
+ */
+
+/dts-v1/;
+#include "mt8183-kukui-jacuzzi-fennel.dtsi"
+#include "mt8183-kukui-audio-ts3a227e-rt1015p.dtsi"
+
+/ {
+ model = "Google fennel14 sku2 board";
+ compatible = "google,fennel-sku2", "google,fennel", "mediatek,mt8183";
+};
+
+&qca_wifi {
+ qcom,ath10k-calibration-variant = "GO_FENNEL14";
+};
diff --git a/arch/arm64/boot/dts/mediatek/mt8183-kukui-jacuzzi-fennel14.dts b/arch/arm64/boot/dts/mediatek/mt8183-kukui-jacuzzi-fennel14.dts
index e8c41f6..23ad0b9 100644
--- a/arch/arm64/boot/dts/mediatek/mt8183-kukui-jacuzzi-fennel14.dts
+++ b/arch/arm64/boot/dts/mediatek/mt8183-kukui-jacuzzi-fennel14.dts
@@ -5,6 +5,7 @@
/dts-v1/;
#include "mt8183-kukui-jacuzzi-fennel.dtsi"
+#include "mt8183-kukui-audio-da7219-rt1015p.dtsi"
/ {
model = "Google fennel14 sku0 board";
diff --git a/arch/arm64/boot/dts/mediatek/mt8183-kukui-jacuzzi.dtsi b/arch/arm64/boot/dts/mediatek/mt8183-kukui-jacuzzi.dtsi
index d8826c8..8f7bf33f 100644
--- a/arch/arm64/boot/dts/mediatek/mt8183-kukui-jacuzzi.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8183-kukui-jacuzzi.dtsi
@@ -9,7 +9,6 @@ / {
panel: panel {
compatible = "auo,b116xw03";
power-supply = <&pp3300_panel>;
- ddc-i2c-bus = <&i2c4>;
backlight = <&backlight_lcd0>;
port {
diff --git a/arch/arm64/boot/dts/mediatek/mt8183-kukui-kakadu-sku22.dts b/arch/arm64/boot/dts/mediatek/mt8183-kukui-kakadu-sku22.dts
new file mode 100644
index 0000000..3a724e6
--- /dev/null
+++ b/arch/arm64/boot/dts/mediatek/mt8183-kukui-kakadu-sku22.dts
@@ -0,0 +1,19 @@
+// SPDX-License-Identifier: (GPL-2.0 OR MIT)
+/*
+ * Copyright 2021 Google LLC
+ */
+
+/dts-v1/;
+#include "mt8183-kukui-kakadu.dtsi"
+#include "mt8183-kukui-audio-rt1015p.dtsi"
+
+/ {
+ model = "MediaTek kakadu board sku22";
+ compatible = "google,kakadu-rev3-sku22", "google,kakadu-rev2-sku22",
+ "google,kakadu", "mediatek,mt8183";
+};
+
+&sound {
+ compatible = "mediatek,mt8183_mt6358_ts3a227_rt1015p";
+};
+
diff --git a/arch/arm64/boot/dts/mediatek/mt8183-kukui.dtsi b/arch/arm64/boot/dts/mediatek/mt8183-kukui.dtsi
index b42d81d..0f9480f 100644
--- a/arch/arm64/boot/dts/mediatek/mt8183-kukui.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8183-kukui.dtsi
@@ -157,33 +157,33 @@ tboard_thermistor1: thermal-sensor1 {
#thermal-sensor-cells = <0>;
io-channels = <&auxadc 0>;
io-channel-names = "sensor-channel";
- temperature-lookup-table = < (-5000) 4241
- 0 4063
- 5000 3856
- 10000 3621
- 15000 3364
- 20000 3091
- 25000 2810
- 30000 2526
- 35000 2247
- 40000 1982
- 45000 1734
- 50000 1507
- 55000 1305
- 60000 1122
- 65000 964
- 70000 827
- 75000 710
- 80000 606
- 85000 519
- 90000 445
- 95000 382
- 100000 330
- 105000 284
- 110000 245
- 115000 213
- 120000 183
- 125000 161>;
+ temperature-lookup-table = < (-5000) 1553
+ 0 1488
+ 5000 1412
+ 10000 1326
+ 15000 1232
+ 20000 1132
+ 25000 1029
+ 30000 925
+ 35000 823
+ 40000 726
+ 45000 635
+ 50000 552
+ 55000 478
+ 60000 411
+ 65000 353
+ 70000 303
+ 75000 260
+ 80000 222
+ 85000 190
+ 90000 163
+ 95000 140
+ 100000 121
+ 105000 104
+ 110000 90
+ 115000 78
+ 120000 67
+ 125000 59>;
};
tboard_thermistor2: thermal-sensor2 {
@@ -191,33 +191,33 @@ tboard_thermistor2: thermal-sensor2 {
#thermal-sensor-cells = <0>;
io-channels = <&auxadc 1>;
io-channel-names = "sensor-channel";
- temperature-lookup-table = < (-5000) 4241
- 0 4063
- 5000 3856
- 10000 3621
- 15000 3364
- 20000 3091
- 25000 2810
- 30000 2526
- 35000 2247
- 40000 1982
- 45000 1734
- 50000 1507
- 55000 1305
- 60000 1122
- 65000 964
- 70000 827
- 75000 710
- 80000 606
- 85000 519
- 90000 445
- 95000 382
- 100000 330
- 105000 284
- 110000 245
- 115000 213
- 120000 183
- 125000 161>;
+ temperature-lookup-table = < (-5000) 1553
+ 0 1488
+ 5000 1412
+ 10000 1326
+ 15000 1232
+ 20000 1132
+ 25000 1029
+ 30000 925
+ 35000 823
+ 40000 726
+ 45000 635
+ 50000 552
+ 55000 478
+ 60000 411
+ 65000 353
+ 70000 303
+ 75000 260
+ 80000 222
+ 85000 190
+ 90000 163
+ 95000 140
+ 100000 121
+ 105000 104
+ 110000 90
+ 115000 78
+ 120000 67
+ 125000 59>;
};
};
@@ -813,7 +813,7 @@ &scp {
cros_ec {
compatible = "google,cros-ec-rpmsg";
- mtk,rpmsg-name = "cros-ec-rpmsg";
+ mediatek,rpmsg-name = "cros-ec-rpmsg";
};
};
@@ -886,6 +886,20 @@ usbc_extcon: extcon0 {
cbas {
compatible = "google,cros-cbas";
};
+
+ typec {
+ compatible = "google,cros-ec-typec";
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ usb_c0: connector@0 {
+ compatible = "usb-c-connector";
+ reg = <0>;
+ power-role = "dual";
+ data-role = "host";
+ try-power-role = "sink";
+ };
+ };
};
};
diff --git a/arch/arm64/boot/dts/mediatek/mt8183.dtsi b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
index ba4584f..00f2ddd 100644
--- a/arch/arm64/boot/dts/mediatek/mt8183.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8183.dtsi
@@ -367,6 +367,70 @@ sysirq: interrupt-controller@c530a80 {
reg = <0 0x0c530a80 0 0x50>;
};
+ cpu_debug0: cpu-debug@d410000 {
+ compatible = "arm,coresight-cpu-debug", "arm,primecell";
+ reg = <0x0 0xd410000 0x0 0x1000>;
+ clocks = <&infracfg CLK_INFRA_DEBUGSYS>;
+ clock-names = "apb_pclk";
+ cpu = <&cpu0>;
+ };
+
+ cpu_debug1: cpu-debug@d510000 {
+ compatible = "arm,coresight-cpu-debug", "arm,primecell";
+ reg = <0x0 0xd510000 0x0 0x1000>;
+ clocks = <&infracfg CLK_INFRA_DEBUGSYS>;
+ clock-names = "apb_pclk";
+ cpu = <&cpu1>;
+ };
+
+ cpu_debug2: cpu-debug@d610000 {
+ compatible = "arm,coresight-cpu-debug", "arm,primecell";
+ reg = <0x0 0xd610000 0x0 0x1000>;
+ clocks = <&infracfg CLK_INFRA_DEBUGSYS>;
+ clock-names = "apb_pclk";
+ cpu = <&cpu2>;
+ };
+
+ cpu_debug3: cpu-debug@d710000 {
+ compatible = "arm,coresight-cpu-debug", "arm,primecell";
+ reg = <0x0 0xd710000 0x0 0x1000>;
+ clocks = <&infracfg CLK_INFRA_DEBUGSYS>;
+ clock-names = "apb_pclk";
+ cpu = <&cpu3>;
+ };
+
+ cpu_debug4: cpu-debug@d810000 {
+ compatible = "arm,coresight-cpu-debug", "arm,primecell";
+ reg = <0x0 0xd810000 0x0 0x1000>;
+ clocks = <&infracfg CLK_INFRA_DEBUGSYS>;
+ clock-names = "apb_pclk";
+ cpu = <&cpu4>;
+ };
+
+ cpu_debug5: cpu-debug@d910000 {
+ compatible = "arm,coresight-cpu-debug", "arm,primecell";
+ reg = <0x0 0xd910000 0x0 0x1000>;
+ clocks = <&infracfg CLK_INFRA_DEBUGSYS>;
+ clock-names = "apb_pclk";
+ cpu = <&cpu5>;
+ };
+
+ cpu_debug6: cpu-debug@da10000 {
+ compatible = "arm,coresight-cpu-debug", "arm,primecell";
+ reg = <0x0 0xda10000 0x0 0x1000>;
+ clocks = <&infracfg CLK_INFRA_DEBUGSYS>;
+ clock-names = "apb_pclk";
+ cpu = <&cpu6>;
+ };
+
+ cpu_debug7: cpu-debug@db10000 {
+ compatible = "arm,coresight-cpu-debug", "arm,primecell";
+ reg = <0x0 0xdb10000 0x0 0x1000>;
+ clocks = <&infracfg CLK_INFRA_DEBUGSYS>;
+ clock-names = "apb_pclk";
+ cpu = <&cpu7>;
+ };
+
topckgen: syscon@10000000 {
compatible = "mediatek,mt8183-topckgen", "syscon";
reg = <0 0x10000000 0 0x1000>;
diff --git a/arch/arm64/boot/dts/mediatek/mt8192.dtsi b/arch/arm64/boot/dts/mediatek/mt8192.dtsi
index c7c7d4e..53d790c 100644
--- a/arch/arm64/boot/dts/mediatek/mt8192.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8192.dtsi
@@ -479,7 +479,7 @@ audsys: clock-controller@11210000 {
#clock-cells = <1>;
};
- i2c3: i2c3@11cb0000 {
+ i2c3: i2c@11cb0000 {
compatible = "mediatek,mt8192-i2c";
reg = <0 0x11cb0000 0 0x1000>,
<0 0x10217300 0 0x80>;
@@ -498,7 +498,7 @@ imp_iic_wrap_e: clock-controller@11cb1000 {
#clock-cells = <1>;
};
- i2c7: i2c7@11d00000 {
+ i2c7: i2c@11d00000 {
compatible = "mediatek,mt8192-i2c";
reg = <0 0x11d00000 0 0x1000>,
<0 0x10217600 0 0x180>;
@@ -511,7 +511,7 @@ i2c7: i2c7@11d00000 {
status = "disabled";
};
- i2c8: i2c8@11d01000 {
+ i2c8: i2c@11d01000 {
compatible = "mediatek,mt8192-i2c";
reg = <0 0x11d01000 0 0x1000>,
<0 0x10217780 0 0x180>;
@@ -524,7 +524,7 @@ i2c8: i2c8@11d01000 {
status = "disabled";
};
- i2c9: i2c9@11d02000 {
+ i2c9: i2c@11d02000 {
compatible = "mediatek,mt8192-i2c";
reg = <0 0x11d02000 0 0x1000>,
<0 0x10217900 0 0x180>;
@@ -543,7 +543,7 @@ imp_iic_wrap_s: clock-controller@11d03000 {
#clock-cells = <1>;
};
- i2c1: i2c1@11d20000 {
+ i2c1: i2c@11d20000 {
compatible = "mediatek,mt8192-i2c";
reg = <0 0x11d20000 0 0x1000>,
<0 0x10217100 0 0x80>;
@@ -556,7 +556,7 @@ i2c1: i2c1@11d20000 {
status = "disabled";
};
- i2c2: i2c2@11d21000 {
+ i2c2: i2c@11d21000 {
compatible = "mediatek,mt8192-i2c";
reg = <0 0x11d21000 0 0x1000>,
<0 0x10217180 0 0x180>;
@@ -569,7 +569,7 @@ i2c2: i2c2@11d21000 {
status = "disabled";
};
- i2c4: i2c4@11d22000 {
+ i2c4: i2c@11d22000 {
compatible = "mediatek,mt8192-i2c";
reg = <0 0x11d22000 0 0x1000>,
<0 0x10217380 0 0x180>;
@@ -588,7 +588,7 @@ imp_iic_wrap_ws: clock-controller@11d23000 {
#clock-cells = <1>;
};
- i2c5: i2c5@11e00000 {
+ i2c5: i2c@11e00000 {
compatible = "mediatek,mt8192-i2c";
reg = <0 0x11e00000 0 0x1000>,
<0 0x10217500 0 0x80>;
@@ -607,7 +607,7 @@ imp_iic_wrap_w: clock-controller@11e01000 {
#clock-cells = <1>;
};
- i2c0: i2c0@11f00000 {
+ i2c0: i2c@11f00000 {
compatible = "mediatek,mt8192-i2c";
reg = <0 0x11f00000 0 0x1000>,
<0 0x10217080 0 0x80>;
@@ -620,7 +620,7 @@ i2c0: i2c0@11f00000 {
status = "disabled";
};
- i2c6: i2c6@11f01000 {
+ i2c6: i2c@11f01000 {
compatible = "mediatek,mt8192-i2c";
reg = <0 0x11f01000 0 0x1000>,
<0 0x10217580 0 0x80>;
diff --git a/arch/arm64/boot/dts/mediatek/mt8516.dtsi b/arch/arm64/boot/dts/mediatek/mt8516.dtsi
index bbe5a14..d1b67c8 100644
--- a/arch/arm64/boot/dts/mediatek/mt8516.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8516.dtsi
@@ -345,14 +345,9 @@ i2c0: i2c@11009000 {
reg = <0 0x11009000 0 0x90>,
<0 0x11000180 0 0x80>;
interrupts = <GIC_SPI 80 IRQ_TYPE_LEVEL_LOW>;
- clocks = <&topckgen CLK_TOP_AHB_INFRA_D2>,
- <&infracfg CLK_IFR_I2C0_SEL>,
- <&topckgen CLK_TOP_I2C0>,
+ clocks = <&topckgen CLK_TOP_I2C0>,
<&topckgen CLK_TOP_APDMA>;
- clock-names = "main-source",
- "main-sel",
- "main",
- "dma";
+ clock-names = "main", "dma";
#address-cells = <1>;
#size-cells = <0>;
status = "disabled";
@@ -364,14 +359,9 @@ i2c1: i2c@1100a000 {
reg = <0 0x1100a000 0 0x90>,
<0 0x11000200 0 0x80>;
interrupts = <GIC_SPI 81 IRQ_TYPE_LEVEL_LOW>;
- clocks = <&topckgen CLK_TOP_AHB_INFRA_D2>,
- <&infracfg CLK_IFR_I2C1_SEL>,
- <&topckgen CLK_TOP_I2C1>,
+ clocks = <&topckgen CLK_TOP_I2C1>,
<&topckgen CLK_TOP_APDMA>;
- clock-names = "main-source",
- "main-sel",
- "main",
- "dma";
+ clock-names = "main", "dma";
#address-cells = <1>;
#size-cells = <0>;
status = "disabled";
@@ -383,14 +373,9 @@ i2c2: i2c@1100b000 {
reg = <0 0x1100b000 0 0x90>,
<0 0x11000280 0 0x80>;
interrupts = <GIC_SPI 82 IRQ_TYPE_LEVEL_LOW>;
- clocks = <&topckgen CLK_TOP_AHB_INFRA_D2>,
- <&infracfg CLK_IFR_I2C2_SEL>,
- <&topckgen CLK_TOP_I2C2>,
+ clocks = <&topckgen CLK_TOP_I2C2>,
<&topckgen CLK_TOP_APDMA>;
- clock-names = "main-source",
- "main-sel",
- "main",
- "dma";
+ clock-names = "main", "dma";
#address-cells = <1>;
#size-cells = <0>;
status = "disabled";
diff --git a/arch/arm64/boot/dts/nvidia/Makefile b/arch/arm64/boot/dts/nvidia/Makefile
index c80f7dc..ea3f338 100644
--- a/arch/arm64/boot/dts/nvidia/Makefile
+++ b/arch/arm64/boot/dts/nvidia/Makefile
@@ -12,3 +12,4 @@
dtb-$(CONFIG_ARCH_TEGRA_194_SOC) += tegra194-p3509-0000+p3668-0000.dtb
dtb-$(CONFIG_ARCH_TEGRA_194_SOC) += tegra194-p3509-0000+p3668-0001.dtb
dtb-$(CONFIG_ARCH_TEGRA_234_SOC) += tegra234-sim-vdk.dtb
+dtb-$(CONFIG_ARCH_TEGRA_234_SOC) += tegra234-p3737-0000+p3701-0000.dtb
diff --git a/arch/arm64/boot/dts/nvidia/tegra132-norrin.dts b/arch/arm64/boot/dts/nvidia/tegra132-norrin.dts
index 8a51751..f16acb4 100644
--- a/arch/arm64/boot/dts/nvidia/tegra132-norrin.dts
+++ b/arch/arm64/boot/dts/nvidia/tegra132-norrin.dts
@@ -564,7 +564,7 @@ as3722: pmic@40 {
pinctrl-names = "default";
pinctrl-0 = <&as3722_default>;
- as3722_default: pinmux@0 {
+ as3722_default: pinmux {
gpio0 {
pins = "gpio0";
function = "gpio";
@@ -770,7 +770,7 @@ i2c_20: i2c-tunnel {
google,remote-bus = <0>;
- charger: bq24735 {
+ charger: bq24735@9 {
compatible = "ti,bq24735";
reg = <0x9>;
interrupt-parent = <&gpio>;
@@ -781,7 +781,7 @@ TEGRA_GPIO(J, 0)
GPIO_ACTIVE_HIGH>;
};
- battery: smart-battery {
+ battery: smart-battery@b {
compatible = "sbs,sbs-battery";
reg = <0xb>;
sbs,i2c-retry-count = <2>;
@@ -885,14 +885,12 @@ MATRIX_KEY(0x07, 0x0b, KEY_UP)
pmc@7000e400 {
nvidia,invert-interrupt;
nvidia,suspend-mode = <0>;
- #wake-cells = <3>;
nvidia,cpu-pwr-good-time = <500>;
nvidia,cpu-pwr-off-time = <300>;
nvidia,core-pwr-good-time = <641 3845>;
nvidia,core-pwr-off-time = <61036>;
nvidia,core-power-req-active-high;
nvidia,sys-clock-req-active-high;
- nvidia,reset-gpio = <&gpio TEGRA_GPIO(I, 5) GPIO_ACTIVE_LOW>;
};
usb@70090000 {
@@ -1023,7 +1021,7 @@ backlight: backlight {
default-brightness-level = <6>;
};
- clk32k_in: clock@0 {
+ clk32k_in: clock-32k {
compatible = "fixed-clock";
clock-frequency = <32768>;
#clock-cells = <0>;
@@ -1057,7 +1055,7 @@ panel: panel {
ddc-i2c-bus = <&dpaux>;
};
- vdd_mux: regulator@0 {
+ vdd_mux: regulator-vdd-mux {
compatible = "regulator-fixed";
regulator-name = "+VDD_MUX";
regulator-min-microvolt = <19000000>;
@@ -1066,7 +1064,7 @@ vdd_mux: regulator@0 {
regulator-boot-on;
};
- vdd_5v0_sys: regulator@1 {
+ vdd_5v0_sys: regulator-vdd-5v0-sys {
compatible = "regulator-fixed";
regulator-name = "+5V_SYS";
regulator-min-microvolt = <5000000>;
@@ -1076,7 +1074,7 @@ vdd_5v0_sys: regulator@1 {
vin-supply = <&vdd_mux>;
};
- vdd_3v3_sys: regulator@2 {
+ vdd_3v3_sys: regulator-vdd-3v3-sys {
compatible = "regulator-fixed";
regulator-name = "+3.3V_SYS";
regulator-min-microvolt = <3300000>;
@@ -1086,7 +1084,7 @@ vdd_3v3_sys: regulator@2 {
vin-supply = <&vdd_mux>;
};
- vdd_3v3_run: regulator@3 {
+ vdd_3v3_run: regulator-vdd-3v3-run {
compatible = "regulator-fixed";
regulator-name = "+3.3V_RUN";
regulator-min-microvolt = <3300000>;
@@ -1098,7 +1096,7 @@ vdd_3v3_run: regulator@3 {
vin-supply = <&vdd_3v3_sys>;
};
- vdd_3v3_hdmi: regulator@4 {
+ vdd_3v3_hdmi: regulator-vdd-3v3-hdmi {
compatible = "regulator-fixed";
regulator-name = "+3.3V_AVDD_HDMI_AP_GATED";
regulator-min-microvolt = <3300000>;
@@ -1106,7 +1104,7 @@ vdd_3v3_hdmi: regulator@4 {
vin-supply = <&vdd_3v3_run>;
};
- vdd_led: regulator@5 {
+ vdd_led: regulator-vdd-led {
compatible = "regulator-fixed";
regulator-name = "+VDD_LED";
regulator-min-microvolt = <3300000>;
@@ -1116,7 +1114,7 @@ vdd_led: regulator@5 {
vin-supply = <&vdd_mux>;
};
- vdd_usb1_vbus: regulator@6 {
+ vdd_usb1_vbus: regulator-vdd-usb1-vbus {
compatible = "regulator-fixed";
regulator-name = "+5V_USB_HS";
regulator-min-microvolt = <5000000>;
@@ -1127,7 +1125,7 @@ vdd_usb1_vbus: regulator@6 {
vin-supply = <&vdd_5v0_sys>;
};
- vdd_usb3_vbus: regulator@7 {
+ vdd_usb3_vbus: regulator-vdd-usb3-vbus {
compatible = "regulator-fixed";
regulator-name = "+5V_USB_SS";
regulator-min-microvolt = <5000000>;
@@ -1138,7 +1136,7 @@ vdd_usb3_vbus: regulator@7 {
vin-supply = <&vdd_5v0_sys>;
};
- vdd_3v3_panel: regulator@8 {
+ vdd_3v3_panel: regulator-vdd-3v3-panel {
compatible = "regulator-fixed";
regulator-name = "+3.3V_PANEL";
regulator-min-microvolt = <3300000>;
@@ -1148,7 +1146,7 @@ vdd_3v3_panel: regulator@8 {
vin-supply = <&vdd_3v3_sys>;
};
- vdd_hdmi_pll: regulator@9 {
+ vdd_hdmi_pll: regulator-vdd-hdmi-pll {
compatible = "regulator-fixed";
regulator-name = "+1.05V_RUN_AVDD_HDMI_PLL_AP_GATE";
regulator-min-microvolt = <1050000>;
@@ -1157,7 +1155,7 @@ vdd_hdmi_pll: regulator@9 {
vin-supply = <&vdd_1v05_run>;
};
- vdd_5v0_hdmi: regulator@10 {
+ vdd_5v0_hdmi: regulator-vdd-5v0-hdmi {
compatible = "regulator-fixed";
regulator-name = "+5V_HDMI_CON";
regulator-min-microvolt = <5000000>;
@@ -1167,7 +1165,7 @@ vdd_5v0_hdmi: regulator@10 {
vin-supply = <&vdd_5v0_sys>;
};
- vdd_5v0_ts: regulator@11 {
+ vdd_5v0_ts: regulator-vdd-5v0-ts {
compatible = "regulator-fixed";
regulator-name = "+5V_VDD_TS";
regulator-min-microvolt = <5000000>;
@@ -1178,7 +1176,7 @@ vdd_5v0_ts: regulator@11 {
enable-active-high;
};
- vdd_3v3_lp0: regulator@12 {
+ vdd_3v3_lp0: regulator-vdd-3v3-lp0 {
compatible = "regulator-fixed";
regulator-name = "+3.3V_LP0";
regulator-min-microvolt = <3300000>;
diff --git a/arch/arm64/boot/dts/nvidia/tegra132-peripherals-opp.dtsi b/arch/arm64/boot/dts/nvidia/tegra132-peripherals-opp.dtsi
new file mode 100644
index 0000000..66ffb7f
--- /dev/null
+++ b/arch/arm64/boot/dts/nvidia/tegra132-peripherals-opp.dtsi
@@ -0,0 +1,426 @@
+// SPDX-License-Identifier: GPL-2.0
+
+/ {
+ /* EMC DVFS OPP table */
+ emc_icc_dvfs_opp_table: opp-table-dvfs0 {
+ compatible = "operating-points-v2";
+
+ opp-12750000-800 {
+ opp-microvolt = <800000 800000 1150000>;
+ opp-hz = /bits/ 64 <12750000>;
+ opp-supported-hw = <0x0003>;
+ };
+
+ opp-12750000-950 {
+ opp-microvolt = <950000 950000 1150000>;
+ opp-hz = /bits/ 64 <12750000>;
+ opp-supported-hw = <0x0008>;
+ };
+
+ opp-12750000-1050 {
+ opp-microvolt = <1050000 1050000 1150000>;
+ opp-hz = /bits/ 64 <12750000>;
+ opp-supported-hw = <0x0010>;
+ };
+
+ opp-12750000-1110 {
+ opp-microvolt = <1110000 1110000 1150000>;
+ opp-hz = /bits/ 64 <12750000>;
+ opp-supported-hw = <0x0004>;
+ };
+
+ opp-20400000-800 {
+ opp-microvolt = <800000 800000 1150000>;
+ opp-hz = /bits/ 64 <20400000>;
+ opp-supported-hw = <0x0003>;
+ };
+
+ opp-20400000-950 {
+ opp-microvolt = <950000 950000 1150000>;
+ opp-hz = /bits/ 64 <20400000>;
+ opp-supported-hw = <0x0008>;
+ };
+
+ opp-20400000-1050 {
+ opp-microvolt = <1050000 1050000 1150000>;
+ opp-hz = /bits/ 64 <20400000>;
+ opp-supported-hw = <0x0010>;
+ };
+
+ opp-20400000-1110 {
+ opp-microvolt = <1110000 1110000 1150000>;
+ opp-hz = /bits/ 64 <20400000>;
+ opp-supported-hw = <0x0004>;
+ };
+
+ opp-40800000-800 {
+ opp-microvolt = <800000 800000 1150000>;
+ opp-hz = /bits/ 64 <40800000>;
+ opp-supported-hw = <0x0003>;
+ };
+
+ opp-40800000-950 {
+ opp-microvolt = <950000 950000 1150000>;
+ opp-hz = /bits/ 64 <40800000>;
+ opp-supported-hw = <0x0008>;
+ };
+
+ opp-40800000-1050 {
+ opp-microvolt = <1050000 1050000 1150000>;
+ opp-hz = /bits/ 64 <40800000>;
+ opp-supported-hw = <0x0010>;
+ };
+
+ opp-40800000-1110 {
+ opp-microvolt = <1110000 1110000 1150000>;
+ opp-hz = /bits/ 64 <40800000>;
+ opp-supported-hw = <0x0004>;
+ };
+
+ opp-68000000-800 {
+ opp-microvolt = <800000 800000 1150000>;
+ opp-hz = /bits/ 64 <68000000>;
+ opp-supported-hw = <0x0003>;
+ };
+
+ opp-68000000-950 {
+ opp-microvolt = <950000 950000 1150000>;
+ opp-hz = /bits/ 64 <68000000>;
+ opp-supported-hw = <0x0008>;
+ };
+
+ opp-68000000-1050 {
+ opp-microvolt = <1050000 1050000 1150000>;
+ opp-hz = /bits/ 64 <68000000>;
+ opp-supported-hw = <0x0010>;
+ };
+
+ opp-68000000-1110 {
+ opp-microvolt = <1110000 1110000 1150000>;
+ opp-hz = /bits/ 64 <68000000>;
+ opp-supported-hw = <0x0004>;
+ };
+
+ opp-102000000-800 {
+ opp-microvolt = <800000 800000 1150000>;
+ opp-hz = /bits/ 64 <102000000>;
+ opp-supported-hw = <0x0003>;
+ };
+
+ opp-102000000-950 {
+ opp-microvolt = <950000 950000 1150000>;
+ opp-hz = /bits/ 64 <102000000>;
+ opp-supported-hw = <0x0008>;
+ };
+
+ opp-102000000-1050 {
+ opp-microvolt = <1050000 1050000 1150000>;
+ opp-hz = /bits/ 64 <102000000>;
+ opp-supported-hw = <0x0010>;
+ };
+
+ opp-102000000-1110 {
+ opp-microvolt = <1110000 1110000 1150000>;
+ opp-hz = /bits/ 64 <102000000>;
+ opp-supported-hw = <0x0004>;
+ };
+
+ opp-204000000-800 {
+ opp-microvolt = <800000 800000 1150000>;
+ opp-hz = /bits/ 64 <204000000>;
+ opp-supported-hw = <0x0003>;
+ opp-suspend;
+ };
+
+ opp-204000000-950 {
+ opp-microvolt = <950000 950000 1150000>;
+ opp-hz = /bits/ 64 <204000000>;
+ opp-supported-hw = <0x0008>;
+ opp-suspend;
+ };
+
+ opp-204000000-1050 {
+ opp-microvolt = <1050000 1050000 1150000>;
+ opp-hz = /bits/ 64 <204000000>;
+ opp-supported-hw = <0x0010>;
+ opp-suspend;
+ };
+
+ opp-204000000-1110 {
+ opp-microvolt = <1110000 1110000 1150000>;
+ opp-hz = /bits/ 64 <204000000>;
+ opp-supported-hw = <0x0004>;
+ opp-suspend;
+ };
+
+ opp-264000000-800 {
+ opp-microvolt = <800000 800000 1150000>;
+ opp-hz = /bits/ 64 <264000000>;
+ opp-supported-hw = <0x0003>;
+ };
+
+ opp-264000000-950 {
+ opp-microvolt = <950000 950000 1150000>;
+ opp-hz = /bits/ 64 <264000000>;
+ opp-supported-hw = <0x0008>;
+ };
+
+ opp-264000000-1050 {
+ opp-microvolt = <1050000 1050000 1150000>;
+ opp-hz = /bits/ 64 <264000000>;
+ opp-supported-hw = <0x0010>;
+ };
+
+ opp-264000000-1110 {
+ opp-microvolt = <1110000 1110000 1150000>;
+ opp-hz = /bits/ 64 <264000000>;
+ opp-supported-hw = <0x0004>;
+ };
+
+ opp-300000000-850 {
+ opp-microvolt = <850000 850000 1150000>;
+ opp-hz = /bits/ 64 <300000000>;
+ opp-supported-hw = <0x0003>;
+ };
+
+ opp-300000000-950 {
+ opp-microvolt = <950000 950000 1150000>;
+ opp-hz = /bits/ 64 <300000000>;
+ opp-supported-hw = <0x0008>;
+ };
+
+ opp-300000000-1050 {
+ opp-microvolt = <1050000 1050000 1150000>;
+ opp-hz = /bits/ 64 <300000000>;
+ opp-supported-hw = <0x0010>;
+ };
+
+ opp-300000000-1110 {
+ opp-microvolt = <1110000 1110000 1150000>;
+ opp-hz = /bits/ 64 <300000000>;
+ opp-supported-hw = <0x0004>;
+ };
+
+ opp-348000000-850 {
+ opp-microvolt = <850000 850000 1150000>;
+ opp-hz = /bits/ 64 <348000000>;
+ opp-supported-hw = <0x0003>;
+ };
+
+ opp-348000000-950 {
+ opp-microvolt = <950000 950000 1150000>;
+ opp-hz = /bits/ 64 <348000000>;
+ opp-supported-hw = <0x0008>;
+ };
+
+ opp-348000000-1050 {
+ opp-microvolt = <1050000 1050000 1150000>;
+ opp-hz = /bits/ 64 <348000000>;
+ opp-supported-hw = <0x0010>;
+ };
+
+ opp-348000000-1110 {
+ opp-microvolt = <1110000 1110000 1150000>;
+ opp-hz = /bits/ 64 <348000000>;
+ opp-supported-hw = <0x0004>;
+ };
+
+ opp-396000000-950 {
+ opp-microvolt = <950000 950000 1150000>;
+ opp-hz = /bits/ 64 <396000000>;
+ opp-supported-hw = <0x0008>;
+ };
+
+ opp-396000000-1000 {
+ opp-microvolt = <1000000 1000000 1150000>;
+ opp-hz = /bits/ 64 <396000000>;
+ opp-supported-hw = <0x0003>;
+ };
+
+ opp-396000000-1050 {
+ opp-microvolt = <1050000 1050000 1150000>;
+ opp-hz = /bits/ 64 <396000000>;
+ opp-supported-hw = <0x0010>;
+ };
+
+ opp-396000000-1110 {
+ opp-microvolt = <1110000 1110000 1150000>;
+ opp-hz = /bits/ 64 <396000000>;
+ opp-supported-hw = <0x0004>;
+ };
+
+ opp-528000000-950 {
+ opp-microvolt = <950000 950000 1150000>;
+ opp-hz = /bits/ 64 <528000000>;
+ opp-supported-hw = <0x0008>;
+ };
+
+ opp-528000000-1000 {
+ opp-microvolt = <1000000 1000000 1150000>;
+ opp-hz = /bits/ 64 <528000000>;
+ opp-supported-hw = <0x0003>;
+ };
+
+ opp-528000000-1050 {
+ opp-microvolt = <1050000 1050000 1150000>;
+ opp-hz = /bits/ 64 <528000000>;
+ opp-supported-hw = <0x0010>;
+ };
+
+ opp-528000000-1110 {
+ opp-microvolt = <1110000 1110000 1150000>;
+ opp-hz = /bits/ 64 <528000000>;
+ opp-supported-hw = <0x0004>;
+ };
+
+ opp-600000000-950 {
+ opp-microvolt = <950000 950000 1150000>;
+ opp-hz = /bits/ 64 <600000000>;
+ opp-supported-hw = <0x0008>;
+ };
+
+ opp-600000000-1000 {
+ opp-microvolt = <1000000 1000000 1150000>;
+ opp-hz = /bits/ 64 <600000000>;
+ opp-supported-hw = <0x0003>;
+ };
+
+ opp-600000000-1050 {
+ opp-microvolt = <1050000 1050000 1150000>;
+ opp-hz = /bits/ 64 <600000000>;
+ opp-supported-hw = <0x0010>;
+ };
+
+ opp-600000000-1110 {
+ opp-microvolt = <1110000 1110000 1150000>;
+ opp-hz = /bits/ 64 <600000000>;
+ opp-supported-hw = <0x0004>;
+ };
+
+ opp-792000000-1000 {
+ opp-microvolt = <1000000 1000000 1150000>;
+ opp-hz = /bits/ 64 <792000000>;
+ opp-supported-hw = <0x000B>;
+ };
+
+ opp-792000000-1050 {
+ opp-microvolt = <1050000 1050000 1150000>;
+ opp-hz = /bits/ 64 <792000000>;
+ opp-supported-hw = <0x0010>;
+ };
+
+ opp-792000000-1110 {
+ opp-microvolt = <1110000 1110000 1150000>;
+ opp-hz = /bits/ 64 <792000000>;
+ opp-supported-hw = <0x0004>;
+ };
+
+ opp-924000000-1100 {
+ opp-microvolt = <1100000 1100000 1150000>;
+ opp-hz = /bits/ 64 <924000000>;
+ opp-supported-hw = <0x0013>;
+ };
+
+ opp-1200000000-1100 {
+ opp-microvolt = <1100000 1100000 1150000>;
+ opp-hz = /bits/ 64 <1200000000>;
+ opp-supported-hw = <0x0003>;
+ };
+ };
+
+ /* EMC bandwidth OPP table */
+ emc_bw_dfs_opp_table: opp-table-dvfs1 {
+ compatible = "operating-points-v2";
+
+ opp-12750000 {
+ opp-hz = /bits/ 64 <12750000>;
+ opp-supported-hw = <0x001F>;
+ opp-peak-kBps = <204000>;
+ };
+
+ opp-20400000 {
+ opp-hz = /bits/ 64 <20400000>;
+ opp-supported-hw = <0x001F>;
+ opp-peak-kBps = <326400>;
+ };
+
+ opp-40800000 {
+ opp-hz = /bits/ 64 <40800000>;
+ opp-supported-hw = <0x001F>;
+ opp-peak-kBps = <652800>;
+ };
+
+ opp-68000000 {
+ opp-hz = /bits/ 64 <68000000>;
+ opp-supported-hw = <0x001F>;
+ opp-peak-kBps = <1088000>;
+ };
+
+ opp-102000000 {
+ opp-hz = /bits/ 64 <102000000>;
+ opp-supported-hw = <0x001F>;
+ opp-peak-kBps = <1632000>;
+ };
+
+ opp-204000000 {
+ opp-hz = /bits/ 64 <204000000>;
+ opp-supported-hw = <0x001F>;
+ opp-peak-kBps = <3264000>;
+ opp-suspend;
+ };
+
+ opp-264000000 {
+ opp-hz = /bits/ 64 <264000000>;
+ opp-supported-hw = <0x001F>;
+ opp-peak-kBps = <4224000>;
+ };
+
+ opp-300000000 {
+ opp-hz = /bits/ 64 <300000000>;
+ opp-supported-hw = <0x001F>;
+ opp-peak-kBps = <4800000>;
+ };
+
+ opp-348000000 {
+ opp-hz = /bits/ 64 <348000000>;
+ opp-supported-hw = <0x001F>;
+ opp-peak-kBps = <5568000>;
+ };
+
+ opp-396000000 {
+ opp-hz = /bits/ 64 <396000000>;
+ opp-supported-hw = <0x001F>;
+ opp-peak-kBps = <6336000>;
+ };
+
+ opp-528000000 {
+ opp-hz = /bits/ 64 <528000000>;
+ opp-supported-hw = <0x001F>;
+ opp-peak-kBps = <8448000>;
+ };
+
+ opp-600000000 {
+ opp-hz = /bits/ 64 <600000000>;
+ opp-supported-hw = <0x001F>;
+ opp-peak-kBps = <9600000>;
+ };
+
+ opp-792000000 {
+ opp-hz = /bits/ 64 <792000000>;
+ opp-supported-hw = <0x001F>;
+ opp-peak-kBps = <12672000>;
+ };
+
+ opp-924000000 {
+ opp-hz = /bits/ 64 <924000000>;
+ opp-supported-hw = <0x0013>;
+ opp-peak-kBps = <14784000>;
+ };
+
+ opp-1200000000 {
+ opp-hz = /bits/ 64 <1200000000>;
+ opp-supported-hw = <0x0003>;
+ opp-peak-kBps = <19200000>;
+ };
+ };
+};
diff --git a/arch/arm64/boot/dts/nvidia/tegra132.dtsi b/arch/arm64/boot/dts/nvidia/tegra132.dtsi
index 63aa312..3673f79 100644
--- a/arch/arm64/boot/dts/nvidia/tegra132.dtsi
+++ b/arch/arm64/boot/dts/nvidia/tegra132.dtsi
@@ -8,6 +8,8 @@
#include <dt-bindings/thermal/tegra124-soctherm.h>
#include <dt-bindings/soc/tegra-pmc.h>
+#include "tegra132-peripherals-opp.dtsi"
+
/ {
compatible = "nvidia,tegra132", "nvidia,tegra124";
interrupt-parent = <&lic>;
@@ -210,7 +212,7 @@ lic: interrupt-controller@60004000 {
};
timer@60005000 {
- compatible = "nvidia,tegra124-timer", "nvidia,tegra20-timer";
+ compatible = "nvidia,tegra124-timer", "nvidia,tegra30-timer";
reg = <0x0 0x60005000 0x0 0x400>;
interrupts = <GIC_SPI 0 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 1 IRQ_TYPE_LEVEL_HIGH>,
@@ -244,6 +246,10 @@ actmon@6000c800 {
clock-names = "actmon", "emc";
resets = <&tegra_car 119>;
reset-names = "actmon";
+ operating-points-v2 = <&emc_bw_dfs_opp_table>;
+ interconnects = <&mc TEGRA124_MC_MPCORER &emc>;
+ interconnect-names = "cpu-read";
+ #cooling-cells = <2>;
};
gpio: gpio@6000d000 {
@@ -394,7 +400,7 @@ pwm: pwm@7000a000 {
};
i2c@7000c000 {
- compatible = "nvidia,tegra124-i2c", "nvidia,tegra114-i2c";
+ compatible = "nvidia,tegra124-i2c";
reg = <0x0 0x7000c000 0x0 0x100>;
interrupts = <GIC_SPI 38 IRQ_TYPE_LEVEL_HIGH>;
#address-cells = <1>;
@@ -409,7 +415,7 @@ i2c@7000c000 {
};
i2c@7000c400 {
- compatible = "nvidia,tegra124-i2c", "nvidia,tegra114-i2c";
+ compatible = "nvidia,tegra124-i2c";
reg = <0x0 0x7000c400 0x0 0x100>;
interrupts = <GIC_SPI 84 IRQ_TYPE_LEVEL_HIGH>;
#address-cells = <1>;
@@ -424,7 +430,7 @@ i2c@7000c400 {
};
i2c@7000c500 {
- compatible = "nvidia,tegra124-i2c", "nvidia,tegra114-i2c";
+ compatible = "nvidia,tegra124-i2c";
reg = <0x0 0x7000c500 0x0 0x100>;
interrupts = <GIC_SPI 92 IRQ_TYPE_LEVEL_HIGH>;
#address-cells = <1>;
@@ -439,7 +445,7 @@ i2c@7000c500 {
};
i2c@7000c700 {
- compatible = "nvidia,tegra124-i2c", "nvidia,tegra114-i2c";
+ compatible = "nvidia,tegra124-i2c";
reg = <0x0 0x7000c700 0x0 0x100>;
interrupts = <GIC_SPI 120 IRQ_TYPE_LEVEL_HIGH>;
#address-cells = <1>;
@@ -454,7 +460,7 @@ i2c@7000c700 {
};
i2c@7000d000 {
- compatible = "nvidia,tegra124-i2c", "nvidia,tegra114-i2c";
+ compatible = "nvidia,tegra124-i2c";
reg = <0x0 0x7000d000 0x0 0x100>;
interrupts = <GIC_SPI 53 IRQ_TYPE_LEVEL_HIGH>;
#address-cells = <1>;
@@ -469,7 +475,7 @@ i2c@7000d000 {
};
i2c@7000d100 {
- compatible = "nvidia,tegra124-i2c", "nvidia,tegra114-i2c";
+ compatible = "nvidia,tegra124-i2c";
reg = <0x0 0x7000d100 0x0 0x100>;
interrupts = <GIC_SPI 63 IRQ_TYPE_LEVEL_HIGH>;
#address-cells = <1>;
@@ -607,15 +613,20 @@ mc: memory-controller@70019000 {
interrupts = <GIC_SPI 77 IRQ_TYPE_LEVEL_HIGH>;
#iommu-cells = <1>;
+ #reset-cells = <1>;
+ #interconnect-cells = <1>;
};
emc: external-memory-controller@7001b000 {
- compatible = "nvidia,tegra132-emc";
+ compatible = "nvidia,tegra132-emc", "nvidia,tegra124-emc";
reg = <0x0 0x7001b000 0x0 0x1000>;
clocks = <&tegra_car TEGRA124_CLK_EMC>;
clock-names = "emc";
nvidia,memory-controller = <&mc>;
+ operating-points-v2 = <&emc_icc_dvfs_opp_table>;
+
+ #interconnect-cells = <0>;
};
sata@70020000 {
@@ -624,10 +635,8 @@ sata@70020000 {
<0x0 0x70020000 0x0 0x7000>; /* SATA */
interrupts = <GIC_SPI 23 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&tegra_car TEGRA124_CLK_SATA>,
- <&tegra_car TEGRA124_CLK_SATA_OOB>,
- <&tegra_car TEGRA124_CLK_CML1>,
- <&tegra_car TEGRA124_CLK_PLL_E>;
- clock-names = "sata", "sata-oob", "cml1", "pll_e";
+ <&tegra_car TEGRA124_CLK_SATA_OOB>;
+ clock-names = "sata", "sata-oob";
resets = <&tegra_car 124>,
<&tegra_car 129>,
<&tegra_car 123>;
@@ -665,8 +674,8 @@ usb@70090000 {
<&tegra_car TEGRA124_CLK_XUSB_HOST_SRC>,
<&tegra_car TEGRA124_CLK_XUSB_FALCON_SRC>,
<&tegra_car TEGRA124_CLK_XUSB_SS>,
- <&tegra_car TEGRA124_CLK_XUSB_SS_SRC>,
<&tegra_car TEGRA124_CLK_XUSB_SS_DIV2>,
+ <&tegra_car TEGRA124_CLK_XUSB_SS_SRC>,
<&tegra_car TEGRA124_CLK_XUSB_HS_SRC>,
<&tegra_car TEGRA124_CLK_XUSB_FS_SRC>,
<&tegra_car TEGRA124_CLK_PLL_U_480M>,
@@ -674,7 +683,7 @@ usb@70090000 {
<&tegra_car TEGRA124_CLK_PLL_E>;
clock-names = "xusb_host", "xusb_host_src",
"xusb_falcon_src", "xusb_ss",
- "xusb_ss_src", "xusb_ss_div2",
+ "xusb_ss_div2", "xusb_ss_src",
"xusb_hs_src", "xusb_fs_src",
"pll_u_480m", "clk_m", "pll_e";
resets = <&tegra_car 89>, <&tegra_car 156>,
@@ -886,7 +895,7 @@ throttle_heavy: heavy {
};
thermal-zones {
- cpu {
+ cpu-thermal {
polling-delay-passive = <1000>;
polling-delay = <0>;
@@ -914,7 +923,8 @@ map0 {
};
};
};
- mem {
+
+ mem-thermal {
polling-delay-passive = <0>;
polling-delay = <0>;
@@ -941,7 +951,8 @@ cooling-maps {
*/
};
};
- gpu {
+
+ gpu-thermal {
polling-delay-passive = <1000>;
polling-delay = <0>;
@@ -969,7 +980,8 @@ map0 {
};
};
};
- pllx {
+
+ pllx-thermal {
polling-delay-passive = <0>;
polling-delay = <0>;
diff --git a/arch/arm64/boot/dts/nvidia/tegra186-p2771-0000.dts b/arch/arm64/boot/dts/nvidia/tegra186-p2771-0000.dts
index 52fa258..c4dee05 100644
--- a/arch/arm64/boot/dts/nvidia/tegra186-p2771-0000.dts
+++ b/arch/arm64/boot/dts/nvidia/tegra186-p2771-0000.dts
@@ -1945,19 +1945,19 @@ power-monitor@42 {
#address-cells = <1>;
#size-cells = <0>;
- channel@0 {
+ input@0 {
reg = <0x0>;
label = "VDD_MUX";
shunt-resistor-micro-ohms = <20000>;
};
- channel@1 {
+ input@1 {
reg = <0x1>;
label = "VDD_5V0_IO_SYS";
shunt-resistor-micro-ohms = <5000>;
};
- channel@2 {
+ input@2 {
reg = <0x2>;
label = "VDD_3V3_SYS";
shunt-resistor-micro-ohms = <10000>;
@@ -1970,19 +1970,19 @@ power-monitor@43 {
#address-cells = <1>;
#size-cells = <0>;
- channel@0 {
+ input@0 {
reg = <0x0>;
label = "VDD_3V3_IO_SLP";
shunt-resistor-micro-ohms = <10000>;
};
- channel@1 {
+ input@1 {
reg = <0x1>;
label = "VDD_1V8_IO";
shunt-resistor-micro-ohms = <10000>;
};
- channel@2 {
+ input@2 {
reg = <0x2>;
label = "VDD_M2_IN";
shunt-resistor-micro-ohms = <10000>;
@@ -2250,7 +2250,7 @@ volume-down {
};
};
- vdd_sd: regulator@100 {
+ vdd_sd: regulator-vdd-sd {
compatible = "regulator-fixed";
regulator-name = "SD_CARD_SW_PWR";
regulator-min-microvolt = <3300000>;
@@ -2262,7 +2262,7 @@ vdd_sd: regulator@100 {
vin-supply = <&vdd_3v3_sys>;
};
- vdd_hdmi: regulator@101 {
+ vdd_hdmi: regulator-vdd-hdmi {
compatible = "regulator-fixed";
regulator-name = "VDD_HDMI_5V0";
regulator-min-microvolt = <5000000>;
@@ -2274,7 +2274,7 @@ vdd_hdmi: regulator@101 {
vin-supply = <&vdd_5v0_sys>;
};
- vdd_usb0: regulator@102 {
+ vdd_usb0: regulator-vdd-usb0 {
compatible = "regulator-fixed";
regulator-name = "VDD_USB0";
regulator-min-microvolt = <5000000>;
@@ -2286,7 +2286,7 @@ vdd_usb0: regulator@102 {
vin-supply = <&vdd_5v0_sys>;
};
- vdd_usb1: regulator@103 {
+ vdd_usb1: regulator-vdd-usb1 {
compatible = "regulator-fixed";
regulator-name = "VDD_USB1";
regulator-min-microvolt = <5000000>;
diff --git a/arch/arm64/boot/dts/nvidia/tegra186-p3310.dtsi b/arch/arm64/boot/dts/nvidia/tegra186-p3310.dtsi
index fcd71bf..aff857d 100644
--- a/arch/arm64/boot/dts/nvidia/tegra186-p3310.dtsi
+++ b/arch/arm64/boot/dts/nvidia/tegra186-p3310.dtsi
@@ -44,7 +44,7 @@ mdio {
#address-cells = <1>;
#size-cells = <0>;
- phy: phy@0 {
+ phy: ethernet-phy@0 {
compatible = "ethernet-phy-ieee802.3-c22";
reg = <0x0>;
interrupt-parent = <&gpio>;
@@ -73,19 +73,19 @@ power-monitor@40 {
#address-cells = <1>;
#size-cells = <0>;
- channel@0 {
+ input@0 {
reg = <0x0>;
label = "VDD_SYS_GPU";
shunt-resistor-micro-ohms = <10000>;
};
- channel@1 {
+ input@1 {
reg = <0x1>;
label = "VDD_SYS_SOC";
shunt-resistor-micro-ohms = <10000>;
};
- channel@2 {
+ input@2 {
reg = <0x2>;
label = "VDD_3V8_WIFI";
shunt-resistor-micro-ohms = <10000>;
@@ -98,19 +98,19 @@ power-monitor@41 {
#address-cells = <1>;
#size-cells = <0>;
- channel@0 {
+ input@0 {
reg = <0x0>;
label = "VDD_IN";
shunt-resistor-micro-ohms = <5000>;
};
- channel@1 {
+ input@1 {
reg = <0x1>;
label = "VDD_SYS_CPU";
shunt-resistor-micro-ohms = <10000>;
};
- channel@2 {
+ input@2 {
reg = <0x2>;
label = "VDD_5V0_DDR";
shunt-resistor-micro-ohms = <10000>;
@@ -393,7 +393,7 @@ psci {
method = "smc";
};
- gnd: regulator@0 {
+ gnd: regulator-gnd {
compatible = "regulator-fixed";
regulator-name = "GND";
regulator-min-microvolt = <0>;
@@ -402,7 +402,7 @@ gnd: regulator@0 {
regulator-boot-on;
};
- vdd_5v0_sys: regulator@1 {
+ vdd_5v0_sys: regulator-vdd-5v0-sys {
compatible = "regulator-fixed";
regulator-name = "VDD_5V0_SYS";
regulator-min-microvolt = <5000000>;
@@ -411,7 +411,7 @@ vdd_5v0_sys: regulator@1 {
regulator-boot-on;
};
- vdd_1v8_ap: regulator@2 {
+ vdd_1v8_ap: regulator-vdd-1v8-ap {
compatible = "regulator-fixed";
regulator-name = "VDD_1V8_AP";
regulator-min-microvolt = <1800000>;
diff --git a/arch/arm64/boot/dts/nvidia/tegra186-p3509-0000+p3636-0001.dts b/arch/arm64/boot/dts/nvidia/tegra186-p3509-0000+p3636-0001.dts
index af33fe9..4631504 100644
--- a/arch/arm64/boot/dts/nvidia/tegra186-p3509-0000+p3636-0001.dts
+++ b/arch/arm64/boot/dts/nvidia/tegra186-p3509-0000+p3636-0001.dts
@@ -46,7 +46,7 @@ mdio {
#address-cells = <1>;
#size-cells = <0>;
- phy: phy@0 {
+ phy: ethernet-phy@0 {
compatible = "ethernet-phy-ieee802.3-c22";
reg = <0x0>;
interrupt-parent = <&gpio_aon>;
@@ -81,22 +81,22 @@ power-monitor@40 {
#address-cells = <1>;
#size-cells = <0>;
- channel@0 {
+ input@0 {
reg = <0>;
label = "VDD_IN";
shunt-resistor-micro-ohms = <5>;
};
- channel@1 {
+ input@1 {
reg = <1>;
label = "VDD_CPU_GPU";
shunt-resistor-micro-ohms = <5>;
};
- channel@2 {
+ input@2 {
reg = <2>;
label = "VDD_SOC";
- shunt-resistor-micro-ohms = <>;
+ shunt-resistor-micro-ohms = <5>;
};
};
};
@@ -593,7 +593,7 @@ psci {
method = "smc";
};
- gnd: regulator@0 {
+ gnd: regulator-gnd {
compatible = "regulator-fixed";
regulator-name = "GND";
regulator-min-microvolt = <0>;
@@ -602,7 +602,7 @@ gnd: regulator@0 {
regulator-boot-on;
};
- vdd_5v0_sys: regulator@1 {
+ vdd_5v0_sys: regulator-vdd-5v0-sys {
compatible = "regulator-fixed";
regulator-name = "VDD_5V0_SYS";
regulator-min-microvolt = <5000000>;
@@ -611,7 +611,7 @@ vdd_5v0_sys: regulator@1 {
regulator-boot-on;
};
- vdd_1v8_ap: regulator@2 {
+ vdd_1v8_ap: regulator-vdd-1v8-ap {
compatible = "regulator-fixed";
regulator-name = "VDD_1V8_AP";
regulator-min-microvolt = <1800000>;
@@ -623,7 +623,7 @@ vdd_1v8_ap: regulator@2 {
vin-supply = <&vdd_1v8>;
};
- vdd_hdmi: regulator@3 {
+ vdd_hdmi: regulator-vdd-hdmi {
compatible = "regulator-fixed";
regulator-name = "VDD_5V0_HDMI_CON";
regulator-min-microvolt = <5000000>;
@@ -633,7 +633,7 @@ vdd_hdmi: regulator@3 {
};
thermal-zones {
- cpu {
+ cpu-thermal {
polling-delay = <0>;
polling-delay-passive = <500>;
status = "okay";
@@ -687,21 +687,7 @@ cpu-passive {
};
};
- gpu {
- polling-delay = <0>;
- polling-delay-passive = <500>;
- status = "okay";
-
- trips {
- gpu_alert0: critical {
- temperature = <99000>;
- hysteresis = <0>;
- type = "critical";
- };
- };
- };
-
- aux {
+ aux-thermal {
polling-delay = <0>;
polling-delay-passive = <500>;
status = "okay";
@@ -714,6 +700,20 @@ aux_alert0: critical {
};
};
};
+
+ gpu-thermal {
+ polling-delay = <0>;
+ polling-delay-passive = <500>;
+ status = "okay";
+
+ trips {
+ gpu_alert0: critical {
+ temperature = <99000>;
+ hysteresis = <0>;
+ type = "critical";
+ };
+ };
+ };
};
aconnect@2900000 {
diff --git a/arch/arm64/boot/dts/nvidia/tegra186.dtsi b/arch/arm64/boot/dts/nvidia/tegra186.dtsi
index 9ac4f014..c91afff 100644
--- a/arch/arm64/boot/dts/nvidia/tegra186.dtsi
+++ b/arch/arm64/boot/dts/nvidia/tegra186.dtsi
@@ -508,6 +508,22 @@ emc: external-memory-controller@2c60000 {
};
};
+ timer@3010000 {
+ compatible = "nvidia,tegra186-timer";
+ reg = <0x0 0x03010000 0x0 0x000e0000>;
+ interrupts = <GIC_SPI 0 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 1 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 2 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 3 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 4 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 5 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 6 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 7 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 8 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 9 IRQ_TYPE_LEVEL_HIGH>;
+ status = "disabled";
+ };
+
uarta: serial@3100000 {
compatible = "nvidia,tegra186-uart", "nvidia,tegra20-uart";
reg = <0x0 0x03100000 0x0 0x40>;
@@ -569,7 +585,7 @@ uartf: serial@3150000 {
};
gen1_i2c: i2c@3160000 {
- compatible = "nvidia,tegra186-i2c", "nvidia,tegra210-i2c";
+ compatible = "nvidia,tegra186-i2c";
reg = <0x0 0x03160000 0x0 0x10000>;
interrupts = <GIC_SPI 25 IRQ_TYPE_LEVEL_HIGH>;
#address-cells = <1>;
@@ -582,7 +598,7 @@ gen1_i2c: i2c@3160000 {
};
cam_i2c: i2c@3180000 {
- compatible = "nvidia,tegra186-i2c", "nvidia,tegra210-i2c";
+ compatible = "nvidia,tegra186-i2c";
reg = <0x0 0x03180000 0x0 0x10000>;
interrupts = <GIC_SPI 27 IRQ_TYPE_LEVEL_HIGH>;
#address-cells = <1>;
@@ -596,7 +612,7 @@ cam_i2c: i2c@3180000 {
/* shares pads with dpaux1 */
dp_aux_ch1_i2c: i2c@3190000 {
- compatible = "nvidia,tegra186-i2c", "nvidia,tegra210-i2c";
+ compatible = "nvidia,tegra186-i2c";
reg = <0x0 0x03190000 0x0 0x10000>;
interrupts = <GIC_SPI 28 IRQ_TYPE_LEVEL_HIGH>;
#address-cells = <1>;
@@ -613,7 +629,7 @@ dp_aux_ch1_i2c: i2c@3190000 {
/* controlled by BPMP, should not be enabled */
pwr_i2c: i2c@31a0000 {
- compatible = "nvidia,tegra186-i2c", "nvidia,tegra210-i2c";
+ compatible = "nvidia,tegra186-i2c";
reg = <0x0 0x031a0000 0x0 0x10000>;
interrupts = <GIC_SPI 29 IRQ_TYPE_LEVEL_HIGH>;
#address-cells = <1>;
@@ -627,7 +643,7 @@ pwr_i2c: i2c@31a0000 {
/* shares pads with dpaux0 */
dp_aux_ch0_i2c: i2c@31b0000 {
- compatible = "nvidia,tegra186-i2c", "nvidia,tegra210-i2c";
+ compatible = "nvidia,tegra186-i2c";
reg = <0x0 0x031b0000 0x0 0x10000>;
interrupts = <GIC_SPI 30 IRQ_TYPE_LEVEL_HIGH>;
#address-cells = <1>;
@@ -643,7 +659,7 @@ dp_aux_ch0_i2c: i2c@31b0000 {
};
gen7_i2c: i2c@31c0000 {
- compatible = "nvidia,tegra186-i2c", "nvidia,tegra210-i2c";
+ compatible = "nvidia,tegra186-i2c";
reg = <0x0 0x031c0000 0x0 0x10000>;
interrupts = <GIC_SPI 31 IRQ_TYPE_LEVEL_HIGH>;
#address-cells = <1>;
@@ -656,7 +672,7 @@ gen7_i2c: i2c@31c0000 {
};
gen9_i2c: i2c@31e0000 {
- compatible = "nvidia,tegra186-i2c", "nvidia,tegra210-i2c";
+ compatible = "nvidia,tegra186-i2c";
reg = <0x0 0x031e0000 0x0 0x10000>;
interrupts = <GIC_SPI 33 IRQ_TYPE_LEVEL_HIGH>;
#address-cells = <1>;
@@ -1073,7 +1089,7 @@ hsp_top0: hsp@3c00000 {
};
gen2_i2c: i2c@c240000 {
- compatible = "nvidia,tegra186-i2c", "nvidia,tegra210-i2c";
+ compatible = "nvidia,tegra186-i2c";
reg = <0x0 0x0c240000 0x0 0x10000>;
interrupts = <GIC_SPI 26 IRQ_TYPE_LEVEL_HIGH>;
#address-cells = <1>;
@@ -1086,7 +1102,7 @@ gen2_i2c: i2c@c240000 {
};
gen8_i2c: i2c@c250000 {
- compatible = "nvidia,tegra186-i2c", "nvidia,tegra210-i2c";
+ compatible = "nvidia,tegra186-i2c";
reg = <0x0 0x0c250000 0x0 0x10000>;
interrupts = <GIC_SPI 32 IRQ_TYPE_LEVEL_HIGH>;
#address-cells = <1>;
@@ -1199,7 +1215,7 @@ sdmmc3_1v8: sdmmc3-1v8 {
ccplex@e000000 {
compatible = "nvidia,tegra186-ccplex-cluster";
- reg = <0x0 0x0e000000 0x0 0x3fffff>;
+ reg = <0x0 0x0e000000 0x0 0x400000>;
nvidia,bpmp = <&bpmp>;
};
@@ -1538,6 +1554,21 @@ vic@15340000 {
iommus = <&smmu TEGRA186_SID_VIC>;
};
+ nvjpg@15380000 {
+ compatible = "nvidia,tegra186-nvjpg";
+ reg = <0x15380000 0x40000>;
+ clocks = <&bpmp TEGRA186_CLK_NVJPG>;
+ clock-names = "nvjpg";
+ resets = <&bpmp TEGRA186_RESET_NVJPG>;
+ reset-names = "nvjpg";
+
+ power-domains = <&bpmp TEGRA186_POWER_DOMAIN_NVJPG>;
+ interconnects = <&mc TEGRA186_MEMORY_CLIENT_NVJPGSRD &emc>,
+ <&mc TEGRA186_MEMORY_CLIENT_NVJPGSWR &emc>;
+ interconnect-names = "dma-mem", "write";
+ iommus = <&smmu TEGRA186_SID_NVJPG>;
+ };
+
dsib: dsi@15400000 {
compatible = "nvidia,tegra186-dsi";
reg = <0x15400000 0x10000>;
@@ -1569,6 +1600,21 @@ nvdec@15480000 {
iommus = <&smmu TEGRA186_SID_NVDEC>;
};
+ nvenc@154c0000 {
+ compatible = "nvidia,tegra186-nvenc";
+ reg = <0x154c0000 0x40000>;
+ clocks = <&bpmp TEGRA186_CLK_NVENC>;
+ clock-names = "nvenc";
+ resets = <&bpmp TEGRA186_RESET_NVENC>;
+ reset-names = "nvenc";
+
+ power-domains = <&bpmp TEGRA186_POWER_DOMAIN_MPE>;
+ interconnects = <&mc TEGRA186_MEMORY_CLIENT_NVENCSRD &emc>,
+ <&mc TEGRA186_MEMORY_CLIENT_NVENCSWR &emc>;
+ interconnect-names = "dma-mem", "write";
+ iommus = <&smmu TEGRA186_SID_NVENC>;
+ };
+
sor0: sor@15540000 {
compatible = "nvidia,tegra186-sor";
reg = <0x15540000 0x10000>;
@@ -1771,7 +1817,7 @@ bpmp: bpmp {
iommus = <&smmu TEGRA186_SID_BPMP>;
mboxes = <&hsp_top0 TEGRA_HSP_MBOX_TYPE_DB
TEGRA_HSP_DB_MASTER_BPMP>;
- shmem = <&cpu_bpmp_tx &cpu_bpmp_rx>;
+ shmem = <&cpu_bpmp_tx>, <&cpu_bpmp_rx>;
#clock-cells = <1>;
#reset-cells = <1>;
#power-domain-cells = <1>;
@@ -1930,12 +1976,12 @@ sound {
};
thermal-zones {
- a57 {
+ /* Cortex-A57 cluster */
+ cpu-thermal {
polling-delay = <0>;
polling-delay-passive = <1000>;
- thermal-sensors =
- <&bpmp_thermal TEGRA186_BPMP_THERMAL_ZONE_CPU>;
+ thermal-sensors = <&bpmp_thermal TEGRA186_BPMP_THERMAL_ZONE_CPU>;
trips {
critical {
@@ -1949,12 +1995,12 @@ cooling-maps {
};
};
- denver {
+ /* Denver cluster */
+ aux-thermal {
polling-delay = <0>;
polling-delay-passive = <1000>;
- thermal-sensors =
- <&bpmp_thermal TEGRA186_BPMP_THERMAL_ZONE_AUX>;
+ thermal-sensors = <&bpmp_thermal TEGRA186_BPMP_THERMAL_ZONE_AUX>;
trips {
critical {
@@ -1968,12 +2014,11 @@ cooling-maps {
};
};
- gpu {
+ gpu-thermal {
polling-delay = <0>;
polling-delay-passive = <1000>;
- thermal-sensors =
- <&bpmp_thermal TEGRA186_BPMP_THERMAL_ZONE_GPU>;
+ thermal-sensors = <&bpmp_thermal TEGRA186_BPMP_THERMAL_ZONE_GPU>;
trips {
critical {
@@ -1987,12 +2032,11 @@ cooling-maps {
};
};
- pll {
+ pll-thermal {
polling-delay = <0>;
polling-delay-passive = <1000>;
- thermal-sensors =
- <&bpmp_thermal TEGRA186_BPMP_THERMAL_ZONE_PLLX>;
+ thermal-sensors = <&bpmp_thermal TEGRA186_BPMP_THERMAL_ZONE_PLLX>;
trips {
critical {
@@ -2006,12 +2050,11 @@ cooling-maps {
};
};
- always_on {
+ ao-thermal {
polling-delay = <0>;
polling-delay-passive = <1000>;
- thermal-sensors =
- <&bpmp_thermal TEGRA186_BPMP_THERMAL_ZONE_AO>;
+ thermal-sensors = <&bpmp_thermal TEGRA186_BPMP_THERMAL_ZONE_AO>;
trips {
critical {
diff --git a/arch/arm64/boot/dts/nvidia/tegra194-p2888.dtsi b/arch/arm64/boot/dts/nvidia/tegra194-p2888.dtsi
index c4058ee..a7d7cfd 100644
--- a/arch/arm64/boot/dts/nvidia/tegra194-p2888.dtsi
+++ b/arch/arm64/boot/dts/nvidia/tegra194-p2888.dtsi
@@ -39,7 +39,7 @@ mdio {
#address-cells = <1>;
#size-cells = <0>;
- phy: phy@0 {
+ phy: ethernet-phy@0 {
compatible = "ethernet-phy-ieee802.3-c22";
reg = <0x0>;
interrupt-parent = <&gpio>;
@@ -317,7 +317,7 @@ temperature-sensor@4c {
};
};
- vdd_5v0_sys: regulator@0 {
+ vdd_5v0_sys: regulator-vdd-5v0-sys {
compatible = "regulator-fixed";
regulator-name = "VIN_SYS_5V0";
regulator-min-microvolt = <5000000>;
@@ -326,7 +326,7 @@ vdd_5v0_sys: regulator@0 {
regulator-boot-on;
};
- vdd_hdmi: regulator@1 {
+ vdd_hdmi: regulator-vdd-hdmi {
compatible = "regulator-fixed";
regulator-name = "VDD_5V0_HDMI_CON";
regulator-min-microvolt = <5000000>;
@@ -335,7 +335,7 @@ vdd_hdmi: regulator@1 {
enable-active-high;
};
- vdd_3v3_pcie: regulator@2 {
+ vdd_3v3_pcie: regulator-vdd-3v3-pcie {
compatible = "regulator-fixed";
regulator-name = "PEX_3V3";
regulator-min-microvolt = <3300000>;
@@ -345,7 +345,7 @@ vdd_3v3_pcie: regulator@2 {
enable-active-high;
};
- vdd_12v_pcie: regulator@3 {
+ vdd_12v_pcie: regulator-vdd-12v-pcie {
compatible = "regulator-fixed";
regulator-name = "VDD_12V";
regulator-min-microvolt = <1200000>;
@@ -354,7 +354,7 @@ vdd_12v_pcie: regulator@3 {
regulator-boot-on;
};
- vdd_5v_sata: regulator@4 {
+ vdd_5v_sata: regulator-vdd-5v0-sata {
compatible = "regulator-fixed";
regulator-name = "VDD_5V_SATA";
regulator-min-microvolt = <5000000>;
diff --git a/arch/arm64/boot/dts/nvidia/tegra194-p2972-0000.dts b/arch/arm64/boot/dts/nvidia/tegra194-p2972-0000.dts
index 9f34871..2478ece9e 100644
--- a/arch/arm64/boot/dts/nvidia/tegra194-p2972-0000.dts
+++ b/arch/arm64/boot/dts/nvidia/tegra194-p2972-0000.dts
@@ -1031,7 +1031,7 @@ i2s6_cif_ep: endpoint {
i2s6_port: port@1 {
reg = <1>;
- i2s6_dap_ep: endpoint@0 {
+ i2s6_dap_ep: endpoint {
dai-format = "i2s";
/* Place holder for external Codec */
};
@@ -2097,7 +2097,7 @@ sound {
};
thermal-zones {
- cpu {
+ cpu-thermal {
polling-delay = <0>;
polling-delay-passive = <500>;
status = "okay";
@@ -2151,7 +2151,7 @@ cpu-passive {
};
};
- gpu {
+ gpu-thermal {
polling-delay = <0>;
polling-delay-passive = <500>;
status = "okay";
@@ -2165,7 +2165,7 @@ gpu_alert0: critical {
};
};
- aux {
+ aux-thermal {
polling-delay = <0>;
polling-delay-passive = <500>;
status = "okay";
diff --git a/arch/arm64/boot/dts/nvidia/tegra194-p3509-0000.dtsi b/arch/arm64/boot/dts/nvidia/tegra194-p3509-0000.dtsi
index a055f17..1323fa9 100644
--- a/arch/arm64/boot/dts/nvidia/tegra194-p3509-0000.dtsi
+++ b/arch/arm64/boot/dts/nvidia/tegra194-p3509-0000.dtsi
@@ -989,7 +989,7 @@ i2s5_cif_ep: endpoint {
i2s5_port: port@1 {
reg = <1>;
- i2s5_dap_ep: endpoint@0 {
+ i2s5_dap_ep: endpoint {
dai-format = "i2s";
/* Place holder for external Codec */
};
@@ -1878,7 +1878,7 @@ spi@3270000 {
status = "okay";
flash@0 {
- compatible = "spi-nor";
+ compatible = "jedec,spi-nor";
reg = <0>;
spi-max-frequency = <102000000>;
spi-tx-bus-width = <4>;
@@ -2000,7 +2000,7 @@ power {
};
};
- vdd_5v0_sys: regulator@100 {
+ vdd_5v0_sys: regulator-vdd-5v0-sys {
compatible = "regulator-fixed";
regulator-name = "VDD_5V_SYS";
regulator-min-microvolt = <5000000>;
@@ -2009,7 +2009,7 @@ vdd_5v0_sys: regulator@100 {
regulator-boot-on;
};
- vdd_3v3_sys: regulator@101 {
+ vdd_3v3_sys: regulator-vdd-3v3-sys {
compatible = "regulator-fixed";
regulator-name = "VDD_3V3_SYS";
regulator-min-microvolt = <3300000>;
@@ -2018,7 +2018,7 @@ vdd_3v3_sys: regulator@101 {
regulator-boot-on;
};
- vdd_3v3_ao: regulator@102 {
+ vdd_3v3_ao: regulator-vdd-3v3-ao {
compatible = "regulator-fixed";
regulator-name = "VDD_3V3_AO";
regulator-min-microvolt = <3300000>;
@@ -2027,7 +2027,7 @@ vdd_3v3_ao: regulator@102 {
regulator-boot-on;
};
- vdd_1v8: regulator@103 {
+ vdd_1v8: regulator-vdd-1v8 {
compatible = "regulator-fixed";
regulator-name = "VDD_1V8";
regulator-min-microvolt = <1800000>;
@@ -2036,7 +2036,7 @@ vdd_1v8: regulator@103 {
regulator-boot-on;
};
- vdd_hdmi: regulator@104 {
+ vdd_hdmi: regulator-vdd-hdmi {
compatible = "regulator-fixed";
regulator-name = "VDD_5V0_HDMI_CON";
regulator-min-microvolt = <5000000>;
@@ -2103,7 +2103,7 @@ sound {
};
thermal-zones {
- cpu {
+ cpu-thermal {
polling-delay = <0>;
polling-delay-passive = <500>;
status = "okay";
@@ -2157,7 +2157,7 @@ cpu-passive {
};
};
- gpu {
+ gpu-thermal {
polling-delay = <0>;
polling-delay-passive = <500>;
status = "okay";
@@ -2171,7 +2171,7 @@ gpu_alert0: critical {
};
};
- aux {
+ aux-thermal {
polling-delay = <0>;
polling-delay-passive = <500>;
status = "okay";
diff --git a/arch/arm64/boot/dts/nvidia/tegra194-p3668-0000.dtsi b/arch/arm64/boot/dts/nvidia/tegra194-p3668-0000.dtsi
index 14da420..8c2c709 100644
--- a/arch/arm64/boot/dts/nvidia/tegra194-p3668-0000.dtsi
+++ b/arch/arm64/boot/dts/nvidia/tegra194-p3668-0000.dtsi
@@ -20,7 +20,7 @@ mmc@3400000 {
};
};
- vdd_3v3_sd: regulator@0 {
+ vdd_3v3_sd: regulator-vdd-3v3-sd {
compatible = "regulator-fixed";
regulator-name = "VDD_3V3_SD";
regulator-min-microvolt = <3300000>;
diff --git a/arch/arm64/boot/dts/nvidia/tegra194-p3668.dtsi b/arch/arm64/boot/dts/nvidia/tegra194-p3668.dtsi
index f16b0aa..0bd66f9 100644
--- a/arch/arm64/boot/dts/nvidia/tegra194-p3668.dtsi
+++ b/arch/arm64/boot/dts/nvidia/tegra194-p3668.dtsi
@@ -36,7 +36,7 @@ mdio {
#address-cells = <1>;
#size-cells = <0>;
- phy: phy@0 {
+ phy: ethernet-phy@0 {
compatible = "ethernet-phy-ieee802.3-c22";
reg = <0x0>;
interrupt-parent = <&gpio>;
diff --git a/arch/arm64/boot/dts/nvidia/tegra194.dtsi b/arch/arm64/boot/dts/nvidia/tegra194.dtsi
index 851e049..3c4acfc 100644
--- a/arch/arm64/boot/dts/nvidia/tegra194.dtsi
+++ b/arch/arm64/boot/dts/nvidia/tegra194.dtsi
@@ -3,6 +3,7 @@
#include <dt-bindings/gpio/tegra194-gpio.h>
#include <dt-bindings/interrupt-controller/arm-gic.h>
#include <dt-bindings/mailbox/tegra186-hsp.h>
+#include <dt-bindings/pinctrl/pinctrl-tegra-io-pad.h>
#include <dt-bindings/pinctrl/pinctrl-tegra.h>
#include <dt-bindings/power/tegra194-powergate.h>
#include <dt-bindings/reset/tegra194-reset.h>
@@ -535,7 +536,6 @@ pex_rst_c5_out_state: pex_rst_c5_out {
pex_rst {
nvidia,pins = "pex_l5_rst_n_pgg1";
nvidia,schmitt = <TEGRA_PIN_DISABLE>;
- nvidia,lpdr = <TEGRA_PIN_ENABLE>;
nvidia,enable-input = <TEGRA_PIN_DISABLE>;
nvidia,io-hv = <TEGRA_PIN_ENABLE>;
nvidia,tristate = <TEGRA_PIN_DISABLE>;
@@ -547,7 +547,6 @@ clkreq_c5_bi_dir_state: clkreq_c5_bi_dir {
clkreq {
nvidia,pins = "pex_l5_clkreq_n_pgg0";
nvidia,schmitt = <TEGRA_PIN_DISABLE>;
- nvidia,lpdr = <TEGRA_PIN_ENABLE>;
nvidia,enable-input = <TEGRA_PIN_ENABLE>;
nvidia,io-hv = <TEGRA_PIN_ENABLE>;
nvidia,tristate = <TEGRA_PIN_DISABLE>;
@@ -593,6 +592,7 @@ emc: external-memory-controller@2c60000 {
compatible = "nvidia,tegra194-emc";
reg = <0x0 0x02c60000 0x0 0x90000>,
<0x0 0x01780000 0x0 0x80000>;
+ interrupts = <GIC_SPI 224 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&bpmp TEGRA194_CLK_EMC>;
clock-names = "emc";
@@ -893,6 +893,9 @@ sdmmc1: mmc@3400000 {
<&mc TEGRA194_MEMORY_CLIENT_SDMMCWA &emc>;
interconnect-names = "dma-mem", "write";
iommus = <&smmu TEGRA194_SID_SDMMC1>;
+ pinctrl-names = "sdmmc-3v3", "sdmmc-1v8";
+ pinctrl-0 = <&sdmmc1_3v3>;
+ pinctrl-1 = <&sdmmc1_1v8>;
nvidia,pad-autocal-pull-up-offset-3v3-timeout =
<0x07>;
nvidia,pad-autocal-pull-down-offset-3v3-timeout =
@@ -904,6 +907,10 @@ sdmmc1: mmc@3400000 {
nvidia,pad-autocal-pull-down-offset-sdr104 = <0x00>;
nvidia,default-tap = <0x9>;
nvidia,default-trim = <0x5>;
+ sd-uhs-sdr25;
+ sd-uhs-sdr50;
+ sd-uhs-ddr50;
+ sd-uhs-sdr104;
status = "disabled";
};
@@ -920,6 +927,9 @@ sdmmc3: mmc@3440000 {
<&mc TEGRA194_MEMORY_CLIENT_SDMMCW &emc>;
interconnect-names = "dma-mem", "write";
iommus = <&smmu TEGRA194_SID_SDMMC3>;
+ pinctrl-names = "sdmmc-3v3", "sdmmc-1v8";
+ pinctrl-0 = <&sdmmc3_3v3>;
+ pinctrl-1 = <&sdmmc3_1v8>;
nvidia,pad-autocal-pull-up-offset-1v8 = <0x00>;
nvidia,pad-autocal-pull-down-offset-1v8 = <0x7a>;
nvidia,pad-autocal-pull-up-offset-3v3-timeout = <0x07>;
@@ -932,6 +942,10 @@ sdmmc3: mmc@3440000 {
nvidia,pad-autocal-pull-down-offset-sdr104 = <0x00>;
nvidia,default-tap = <0x9>;
nvidia,default-trim = <0x5>;
+ sd-uhs-sdr25;
+ sd-uhs-sdr50;
+ sd-uhs-ddr50;
+ sd-uhs-sdr104;
status = "disabled";
};
@@ -963,6 +977,11 @@ sdmmc4: mmc@3460000 {
nvidia,default-tap = <0x8>;
nvidia,default-trim = <0x14>;
nvidia,dqs-trim = <40>;
+ cap-mmc-highspeed;
+ mmc-ddr-1_8v;
+ mmc-hs200-1_8v;
+ mmc-hs400-1_8v;
+ mmc-hs400-enhanced-strobe;
supports-cqe;
status = "disabled";
};
@@ -1182,7 +1201,7 @@ cec@3960000 {
};
hsp_top0: hsp@3c00000 {
- compatible = "nvidia,tegra194-hsp", "nvidia,tegra186-hsp";
+ compatible = "nvidia,tegra194-hsp";
reg = <0x03c00000 0xa0000>;
interrupts = <GIC_SPI 176 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 120 IRQ_TYPE_LEVEL_HIGH>,
@@ -1360,7 +1379,7 @@ p2u_hsio_11: phy@3f40000 {
};
hsp_aon: hsp@c150000 {
- compatible = "nvidia,tegra194-hsp", "nvidia,tegra186-hsp";
+ compatible = "nvidia,tegra194-hsp";
reg = <0x0c150000 0x90000>;
interrupts = <GIC_SPI 133 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 134 IRQ_TYPE_LEVEL_HIGH>,
@@ -1472,6 +1491,101 @@ pmc: pmc@c360000 {
#interrupt-cells = <2>;
interrupt-controller;
+ sdmmc1_3v3: sdmmc1-3v3 {
+ pins = "sdmmc1-hv";
+ power-source = <TEGRA_IO_PAD_VOLTAGE_3V3>;
+ };
+
+ sdmmc1_1v8: sdmmc1-1v8 {
+ pins = "sdmmc1-hv";
+ power-source = <TEGRA_IO_PAD_VOLTAGE_1V8>;
+ };
+ sdmmc3_3v3: sdmmc3-3v3 {
+ pins = "sdmmc3-hv";
+ power-source = <TEGRA_IO_PAD_VOLTAGE_3V3>;
+ };
+
+ sdmmc3_1v8: sdmmc3-1v8 {
+ pins = "sdmmc3-hv";
+ power-source = <TEGRA_IO_PAD_VOLTAGE_1V8>;
+ };
+
+ };
+
+ iommu@10000000 {
+ compatible = "nvidia,tegra194-smmu", "nvidia,smmu-500";
+ reg = <0x10000000 0x800000>;
+ interrupts = <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>;
+ stream-match-mask = <0x7f80>;
+ #global-interrupts = <1>;
+ #iommu-cells = <1>;
+
+ nvidia,memory-controller = <&mc>;
+ status = "okay";
};
smmu: iommu@12000000 {
@@ -1703,6 +1817,23 @@ vic@15340000 {
<&mc TEGRA194_MEMORY_CLIENT_VICSWR &emc>;
interconnect-names = "dma-mem", "write";
iommus = <&smmu TEGRA194_SID_VIC>;
+ dma-coherent;
+ };
+
+ nvjpg@15380000 {
+ compatible = "nvidia,tegra194-nvjpg";
+ reg = <0x15380000 0x40000>;
+ clocks = <&bpmp TEGRA194_CLK_NVJPG>;
+ clock-names = "nvjpg";
+ resets = <&bpmp TEGRA194_RESET_NVJPG>;
+ reset-names = "nvjpg";
+
+ power-domains = <&bpmp TEGRA194_POWER_DOMAIN_NVJPG>;
+ interconnects = <&mc TEGRA194_MEMORY_CLIENT_NVJPGSRD &emc>,
+ <&mc TEGRA194_MEMORY_CLIENT_NVJPGSWR &emc>;
+ interconnect-names = "dma-mem", "write";
+ iommus = <&smmu TEGRA194_SID_NVJPG>;
+ dma-coherent;
};
nvdec@15480000 {
@@ -1724,6 +1855,25 @@ nvdec@15480000 {
nvidia,host1x-class = <0xf0>;
};
+ nvenc@154c0000 {
+ compatible = "nvidia,tegra194-nvenc";
+ reg = <0x154c0000 0x40000>;
+ clocks = <&bpmp TEGRA194_CLK_NVENC>;
+ clock-names = "nvenc";
+ resets = <&bpmp TEGRA194_RESET_NVENC>;
+ reset-names = "nvenc";
+
+ power-domains = <&bpmp TEGRA194_POWER_DOMAIN_NVENCA>;
+ interconnects = <&mc TEGRA194_MEMORY_CLIENT_NVENCSRD &emc>,
+ <&mc TEGRA194_MEMORY_CLIENT_NVENCSRD1 &emc>,
+ <&mc TEGRA194_MEMORY_CLIENT_NVENCSWR &emc>;
+ interconnect-names = "dma-mem", "read-1", "write";
+ iommus = <&smmu TEGRA194_SID_NVENC>;
+ dma-coherent;
+
+ nvidia,host1x-class = <0x21>;
+ };
+
dpaux0: dpaux@155c0000 {
compatible = "nvidia,tegra194-dpaux";
reg = <0x155c0000 0x10000>;
@@ -1860,6 +2010,25 @@ i2c-bus {
};
};
+ nvenc@15a80000 {
+ compatible = "nvidia,tegra194-nvenc";
+ reg = <0x15a80000 0x00040000>;
+ clocks = <&bpmp TEGRA194_CLK_NVENC1>;
+ clock-names = "nvenc";
+ resets = <&bpmp TEGRA194_RESET_NVENC1>;
+ reset-names = "nvenc";
+
+ power-domains = <&bpmp TEGRA194_POWER_DOMAIN_NVENCB>;
+ interconnects = <&mc TEGRA194_MEMORY_CLIENT_NVENC1SRD &emc>,
+ <&mc TEGRA194_MEMORY_CLIENT_NVENC1SRD1 &emc>,
+ <&mc TEGRA194_MEMORY_CLIENT_NVENC1SWR &emc>;
+ interconnect-names = "dma-mem", "read-1", "write";
+ iommus = <&smmu TEGRA194_SID_NVENC1>;
+ dma-coherent;
+
+ nvidia,host1x-class = <0x22>;
+ };
+
sor0: sor@15b00000 {
compatible = "nvidia,tegra194-sor";
reg = <0x15b00000 0x40000>;
@@ -2007,7 +2176,6 @@ pcie@14100000 {
#size-cells = <2>;
device_type = "pci";
num-lanes = <1>;
- num-viewport = <8>;
linux,pci-domain = <1>;
clocks = <&bpmp TEGRA194_CLK_PEX0_CORE_1>;
@@ -2040,7 +2208,6 @@ pcie@14100000 {
interconnects = <&mc TEGRA194_MEMORY_CLIENT_PCIE1R &emc>,
<&mc TEGRA194_MEMORY_CLIENT_PCIE1W &emc>;
interconnect-names = "dma-mem", "write";
- iommus = <&smmu TEGRA194_SID_PCIE1>;
iommu-map = <0x0 &smmu TEGRA194_SID_PCIE1 0x1000>;
iommu-map-mask = <0x0>;
dma-coherent;
@@ -2061,7 +2228,6 @@ pcie@14120000 {
#size-cells = <2>;
device_type = "pci";
num-lanes = <1>;
- num-viewport = <8>;
linux,pci-domain = <2>;
clocks = <&bpmp TEGRA194_CLK_PEX0_CORE_2>;
@@ -2094,7 +2260,6 @@ pcie@14120000 {
interconnects = <&mc TEGRA194_MEMORY_CLIENT_PCIE2AR &emc>,
<&mc TEGRA194_MEMORY_CLIENT_PCIE2AW &emc>;
interconnect-names = "dma-mem", "write";
- iommus = <&smmu TEGRA194_SID_PCIE2>;
iommu-map = <0x0 &smmu TEGRA194_SID_PCIE2 0x1000>;
iommu-map-mask = <0x0>;
dma-coherent;
@@ -2115,7 +2280,6 @@ pcie@14140000 {
#size-cells = <2>;
device_type = "pci";
num-lanes = <1>;
- num-viewport = <8>;
linux,pci-domain = <3>;
clocks = <&bpmp TEGRA194_CLK_PEX0_CORE_3>;
@@ -2148,7 +2312,6 @@ pcie@14140000 {
interconnects = <&mc TEGRA194_MEMORY_CLIENT_PCIE3R &emc>,
<&mc TEGRA194_MEMORY_CLIENT_PCIE3W &emc>;
interconnect-names = "dma-mem", "write";
- iommus = <&smmu TEGRA194_SID_PCIE3>;
iommu-map = <0x0 &smmu TEGRA194_SID_PCIE3 0x1000>;
iommu-map-mask = <0x0>;
dma-coherent;
@@ -2169,7 +2332,6 @@ pcie@14160000 {
#size-cells = <2>;
device_type = "pci";
num-lanes = <4>;
- num-viewport = <8>;
linux,pci-domain = <4>;
clocks = <&bpmp TEGRA194_CLK_PEX0_CORE_4>;
@@ -2202,7 +2364,6 @@ pcie@14160000 {
interconnects = <&mc TEGRA194_MEMORY_CLIENT_PCIE4R &emc>,
<&mc TEGRA194_MEMORY_CLIENT_PCIE4W &emc>;
interconnect-names = "dma-mem", "write";
- iommus = <&smmu TEGRA194_SID_PCIE4>;
iommu-map = <0x0 &smmu TEGRA194_SID_PCIE4 0x1000>;
iommu-map-mask = <0x0>;
dma-coherent;
@@ -2223,7 +2384,6 @@ pcie@14180000 {
#size-cells = <2>;
device_type = "pci";
num-lanes = <8>;
- num-viewport = <8>;
linux,pci-domain = <0>;
clocks = <&bpmp TEGRA194_CLK_PEX0_CORE_0>;
@@ -2256,7 +2416,6 @@ pcie@14180000 {
interconnects = <&mc TEGRA194_MEMORY_CLIENT_PCIE0R &emc>,
<&mc TEGRA194_MEMORY_CLIENT_PCIE0W &emc>;
interconnect-names = "dma-mem", "write";
- iommus = <&smmu TEGRA194_SID_PCIE0>;
iommu-map = <0x0 &smmu TEGRA194_SID_PCIE0 0x1000>;
iommu-map-mask = <0x0>;
dma-coherent;
@@ -2277,15 +2436,13 @@ pcie@141a0000 {
#size-cells = <2>;
device_type = "pci";
num-lanes = <8>;
- num-viewport = <8>;
linux,pci-domain = <5>;
pinctrl-names = "default";
pinctrl-0 = <&pex_rst_c5_out_state>, <&clkreq_c5_bi_dir_state>;
- clocks = <&bpmp TEGRA194_CLK_PEX1_CORE_5>,
- <&bpmp TEGRA194_CLK_PEX1_CORE_5M>;
- clock-names = "core", "core_m";
+ clocks = <&bpmp TEGRA194_CLK_PEX1_CORE_5>;
+ clock-names = "core";
resets = <&bpmp TEGRA194_RESET_PEX1_CORE_5_APB>,
<&bpmp TEGRA194_RESET_PEX1_CORE_5>;
@@ -2314,7 +2471,6 @@ pcie@141a0000 {
interconnects = <&mc TEGRA194_MEMORY_CLIENT_PCIE5R &emc>,
<&mc TEGRA194_MEMORY_CLIENT_PCIE5W &emc>;
interconnect-names = "dma-mem", "write";
- iommus = <&smmu TEGRA194_SID_PCIE5>;
iommu-map = <0x0 &smmu TEGRA194_SID_PCIE5 0x1000>;
iommu-map-mask = <0x0>;
dma-coherent;
@@ -2354,7 +2510,6 @@ pcie-ep@14160000 {
interconnects = <&mc TEGRA194_MEMORY_CLIENT_PCIE4R &emc>,
<&mc TEGRA194_MEMORY_CLIENT_PCIE4W &emc>;
interconnect-names = "dma-mem", "write";
- iommus = <&smmu TEGRA194_SID_PCIE4>;
iommu-map = <0x0 &smmu TEGRA194_SID_PCIE4 0x1000>;
iommu-map-mask = <0x0>;
dma-coherent;
@@ -2394,7 +2549,6 @@ pcie-ep@14180000 {
interconnects = <&mc TEGRA194_MEMORY_CLIENT_PCIE0R &emc>,
<&mc TEGRA194_MEMORY_CLIENT_PCIE0W &emc>;
interconnect-names = "dma-mem", "write";
- iommus = <&smmu TEGRA194_SID_PCIE0>;
iommu-map = <0x0 &smmu TEGRA194_SID_PCIE0 0x1000>;
iommu-map-mask = <0x0>;
dma-coherent;
@@ -2437,7 +2591,6 @@ pcie-ep@141a0000 {
interconnects = <&mc TEGRA194_MEMORY_CLIENT_PCIE5R &emc>,
<&mc TEGRA194_MEMORY_CLIENT_PCIE5W &emc>;
interconnect-names = "dma-mem", "write";
- iommus = <&smmu TEGRA194_SID_PCIE5>;
iommu-map = <0x0 &smmu TEGRA194_SID_PCIE5 0x1000>;
iommu-map-mask = <0x0>;
dma-coherent;
@@ -2467,7 +2620,7 @@ bpmp: bpmp {
compatible = "nvidia,tegra186-bpmp";
mboxes = <&hsp_top0 TEGRA_HSP_MBOX_TYPE_DB
TEGRA_HSP_DB_MASTER_BPMP>;
- shmem = <&cpu_bpmp_tx &cpu_bpmp_rx>;
+ shmem = <&cpu_bpmp_tx>, <&cpu_bpmp_rx>;
#clock-cells = <1>;
#reset-cells = <1>;
#power-domain-cells = <1>;
@@ -2731,7 +2884,7 @@ sound {
iommus = <&smmu TEGRA194_SID_APE>;
};
- tcu: tcu {
+ tcu: serial {
compatible = "nvidia,tegra194-tcu";
mboxes = <&hsp_top0 TEGRA_HSP_MBOX_TYPE_SM TEGRA_HSP_SM_RX(0)>,
<&hsp_aon TEGRA_HSP_MBOX_TYPE_SM TEGRA_HSP_SM_TX(1)>;
@@ -2739,39 +2892,33 @@ tcu: tcu {
};
thermal-zones {
- cpu {
- thermal-sensors = <&{/bpmp/thermal}
- TEGRA194_BPMP_THERMAL_ZONE_CPU>;
+ cpu-thermal {
+ thermal-sensors = <&{/bpmp/thermal} TEGRA194_BPMP_THERMAL_ZONE_CPU>;
status = "disabled";
};
- gpu {
- thermal-sensors = <&{/bpmp/thermal}
- TEGRA194_BPMP_THERMAL_ZONE_GPU>;
+ gpu-thermal {
+ thermal-sensors = <&{/bpmp/thermal} TEGRA194_BPMP_THERMAL_ZONE_GPU>;
status = "disabled";
};
- aux {
- thermal-sensors = <&{/bpmp/thermal}
- TEGRA194_BPMP_THERMAL_ZONE_AUX>;
+ aux-thermal {
+ thermal-sensors = <&{/bpmp/thermal} TEGRA194_BPMP_THERMAL_ZONE_AUX>;
status = "disabled";
};
- pllx {
- thermal-sensors = <&{/bpmp/thermal}
- TEGRA194_BPMP_THERMAL_ZONE_PLLX>;
+ pllx-thermal {
+ thermal-sensors = <&{/bpmp/thermal} TEGRA194_BPMP_THERMAL_ZONE_PLLX>;
status = "disabled";
};
- ao {
- thermal-sensors = <&{/bpmp/thermal}
- TEGRA194_BPMP_THERMAL_ZONE_AO>;
+ ao-thermal {
+ thermal-sensors = <&{/bpmp/thermal} TEGRA194_BPMP_THERMAL_ZONE_AO>;
status = "disabled";
};
- tj {
- thermal-sensors = <&{/bpmp/thermal}
- TEGRA194_BPMP_THERMAL_ZONE_TJ_MAX>;
+ tj-thermal {
+ thermal-sensors = <&{/bpmp/thermal} TEGRA194_BPMP_THERMAL_ZONE_TJ_MAX>;
status = "disabled";
};
};
diff --git a/arch/arm64/boot/dts/nvidia/tegra210-p2180.dtsi b/arch/arm64/boot/dts/nvidia/tegra210-p2180.dtsi
index 6077d57..75eb743 100644
--- a/arch/arm64/boot/dts/nvidia/tegra210-p2180.dtsi
+++ b/arch/arm64/boot/dts/nvidia/tegra210-p2180.dtsi
@@ -301,7 +301,7 @@ mmc@700b0600 {
vqmmc-supply = <&vdd_1v8>;
};
- clk32k_in: clock@0 {
+ clk32k_in: clock-32k {
compatible = "fixed-clock";
clock-frequency = <32768>;
#clock-cells = <0>;
@@ -336,7 +336,7 @@ psci {
method = "smc";
};
- vdd_gpu: regulator@100 {
+ vdd_gpu: regulator-vdd-gpu {
compatible = "pwm-regulator";
pwms = <&pwm 1 8000>;
regulator-name = "VDD_GPU";
diff --git a/arch/arm64/boot/dts/nvidia/tegra210-p2371-2180.dts b/arch/arm64/boot/dts/nvidia/tegra210-p2371-2180.dts
index 2e17df6..328fbfe 100644
--- a/arch/arm64/boot/dts/nvidia/tegra210-p2371-2180.dts
+++ b/arch/arm64/boot/dts/nvidia/tegra210-p2371-2180.dts
@@ -11,11 +11,8 @@ / {
pcie@1003000 {
status = "okay";
- avdd-pll-uerefe-supply = <&avdd_1v05_pll>;
hvddio-pex-supply = <&vdd_1v8>;
dvddio-pex-supply = <&vdd_pex_1v05>;
- dvdd-pex-pll-supply = <&vdd_pex_1v05>;
- hvdd-pex-pll-e-supply = <&vdd_1v8>;
vddio-pex-ctl-supply = <&vdd_1v8>;
pci@1,0 {
diff --git a/arch/arm64/boot/dts/nvidia/tegra210-p2530.dtsi b/arch/arm64/boot/dts/nvidia/tegra210-p2530.dtsi
index 58aa051..0a70dae 100644
--- a/arch/arm64/boot/dts/nvidia/tegra210-p2530.dtsi
+++ b/arch/arm64/boot/dts/nvidia/tegra210-p2530.dtsi
@@ -40,7 +40,7 @@ mmc@700b0600 {
non-removable;
};
- clk32k_in: clock@0 {
+ clk32k_in: clock-32k {
compatible = "fixed-clock";
clock-frequency = <32768>;
#clock-cells = <0>;
diff --git a/arch/arm64/boot/dts/nvidia/tegra210-p2597.dtsi b/arch/arm64/boot/dts/nvidia/tegra210-p2597.dtsi
index d8409c1..4b43b89 100644
--- a/arch/arm64/boot/dts/nvidia/tegra210-p2597.dtsi
+++ b/arch/arm64/boot/dts/nvidia/tegra210-p2597.dtsi
@@ -1361,11 +1361,6 @@ usb@70090000 {
dvddio-pex-supply = <&vdd_pex_1v05>;
hvddio-pex-supply = <&vdd_1v8>;
avdd-usb-supply = <&vdd_3v3_sys>;
- /* XXX what are these? */
- avdd-pll-utmip-supply = <&vdd_1v8>;
- avdd-pll-uerefe-supply = <&vdd_pex_1v05>;
- dvdd-usb-ss-pll-supply = <&vdd_pex_1v05>;
- hvdd-usb-ss-pll-e-supply = <&vdd_1v8>;
status = "okay";
@@ -1555,7 +1550,7 @@ volume_up {
};
};
- vdd_sys_mux: regulator@0 {
+ vdd_sys_mux: regulator-vdd-sys-mux {
compatible = "regulator-fixed";
regulator-name = "VDD_SYS_MUX";
regulator-min-microvolt = <5000000>;
@@ -1564,7 +1559,7 @@ vdd_sys_mux: regulator@0 {
regulator-boot-on;
};
- vdd_5v0_sys: regulator@1 {
+ vdd_5v0_sys: regulator-vdd-5v0-sys {
compatible = "regulator-fixed";
regulator-name = "VDD_5V0_SYS";
regulator-min-microvolt = <5000000>;
@@ -1576,7 +1571,7 @@ vdd_5v0_sys: regulator@1 {
vin-supply = <&vdd_sys_mux>;
};
- vdd_3v3_sys: regulator@2 {
+ vdd_3v3_sys: regulator-vdd-3v3-sys {
compatible = "regulator-fixed";
regulator-name = "VDD_3V3_SYS";
regulator-min-microvolt = <3300000>;
@@ -1588,10 +1583,9 @@ vdd_3v3_sys: regulator@2 {
vin-supply = <&vdd_sys_mux>;
regulator-enable-ramp-delay = <160>;
- regulator-disable-ramp-delay = <10000>;
};
- vdd_5v0_io: regulator@3 {
+ vdd_5v0_io: regulator-vdd-5v0-io {
compatible = "regulator-fixed";
regulator-name = "VDD_5V0_IO_SYS";
regulator-min-microvolt = <5000000>;
@@ -1600,7 +1594,7 @@ vdd_5v0_io: regulator@3 {
regulator-boot-on;
};
- vdd_3v3_sd: regulator@4 {
+ vdd_3v3_sd: regulator-vdd-3v3-sd {
compatible = "regulator-fixed";
regulator-name = "VDD_3V3_SD";
regulator-min-microvolt = <3300000>;
@@ -1610,10 +1604,9 @@ vdd_3v3_sd: regulator@4 {
vin-supply = <&vdd_3v3_sys>;
regulator-enable-ramp-delay = <472>;
- regulator-disable-ramp-delay = <4880>;
};
- vdd_dsi_csi: regulator@5 {
+ vdd_dsi_csi: regulator-vdd-dsi-csi {
compatible = "regulator-fixed";
regulator-name = "AVDD_DSI_CSI_1V2";
regulator-min-microvolt = <1200000>;
@@ -1621,7 +1614,7 @@ vdd_dsi_csi: regulator@5 {
vin-supply = <&vdd_sys_1v2>;
};
- vdd_3v3_dis: regulator@6 {
+ vdd_3v3_dis: regulator-vdd-3v3-dis {
compatible = "regulator-fixed";
regulator-name = "VDD_DIS_3V3_LCD";
regulator-min-microvolt = <3300000>;
@@ -1632,7 +1625,7 @@ vdd_3v3_dis: regulator@6 {
vin-supply = <&vdd_3v3_sys>;
};
- vdd_1v8_dis: regulator@7 {
+ vdd_1v8_dis: regulator-vdd-1v8-dis {
compatible = "regulator-fixed";
regulator-name = "VDD_LCD_1V8_DIS";
regulator-min-microvolt = <1800000>;
@@ -1643,7 +1636,7 @@ vdd_1v8_dis: regulator@7 {
vin-supply = <&vdd_1v8>;
};
- vdd_5v0_rtl: regulator@8 {
+ vdd_5v0_rtl: regulator-vdd-5v0-rtl {
compatible = "regulator-fixed";
regulator-name = "RTL_5V";
regulator-min-microvolt = <5000000>;
@@ -1653,7 +1646,7 @@ vdd_5v0_rtl: regulator@8 {
vin-supply = <&vdd_5v0_sys>;
};
- vdd_usb_vbus: regulator@9 {
+ vdd_usb_vbus: regulator-vdd-usb-vbus {
compatible = "regulator-fixed";
regulator-name = "USB_VBUS_EN1";
regulator-min-microvolt = <5000000>;
@@ -1663,7 +1656,7 @@ vdd_usb_vbus: regulator@9 {
vin-supply = <&vdd_5v0_sys>;
};
- vdd_hdmi: regulator@10 {
+ vdd_hdmi: regulator-vdd-hdmi {
compatible = "regulator-fixed";
regulator-name = "VDD_HDMI_5V0";
regulator-min-microvolt = <5000000>;
@@ -1673,7 +1666,7 @@ vdd_hdmi: regulator@10 {
vin-supply = <&vdd_5v0_sys>;
};
- vdd_cam_1v2: regulator@11 {
+ vdd_cam_1v2: regulator-vdd-cam-1v8 {
compatible = "regulator-fixed";
regulator-name = "vdd-cam-1v2";
regulator-min-microvolt = <1200000>;
@@ -1683,7 +1676,7 @@ vdd_cam_1v2: regulator@11 {
vin-supply = <&vdd_3v3_sys>;
};
- vdd_cam_2v8: regulator@12 {
+ vdd_cam_2v8: regulator-vdd-cam-2v8 {
compatible = "regulator-fixed";
regulator-name = "vdd-cam-2v8";
regulator-min-microvolt = <2800000>;
@@ -1693,7 +1686,7 @@ vdd_cam_2v8: regulator@12 {
vin-supply = <&vdd_3v3_sys>;
};
- vdd_cam_1v8: regulator@13 {
+ vdd_cam_1v8: regulator-vdd-cam-1v8 {
compatible = "regulator-fixed";
regulator-name = "vdd-cam-1v8";
regulator-min-microvolt = <1800000>;
@@ -1703,7 +1696,7 @@ vdd_cam_1v8: regulator@13 {
vin-supply = <&vdd_3v3_sys>;
};
- vdd_usb_vbus_otg: regulator@14 {
+ vdd_usb_vbus_otg: regulator-vdd-usb-vbus-otg {
compatible = "regulator-fixed";
regulator-name = "USB_VBUS_EN0";
regulator-min-microvolt = <5000000>;
diff --git a/arch/arm64/boot/dts/nvidia/tegra210-p2894.dtsi b/arch/arm64/boot/dts/nvidia/tegra210-p2894.dtsi
index 41beab6..10347b6 100644
--- a/arch/arm64/boot/dts/nvidia/tegra210-p2894.dtsi
+++ b/arch/arm64/boot/dts/nvidia/tegra210-p2894.dtsi
@@ -1383,7 +1383,7 @@ gpio5_6_7 {
};
};
- gpio@0 {
+ hog-0 {
gpio-hog;
output-high;
gpios = <2 GPIO_ACTIVE_HIGH>,
@@ -1586,7 +1586,7 @@ mmc@700b0600 {
status = "okay";
};
- clk32k_in: clock@0 {
+ clk32k_in: clock-32k {
compatible = "fixed-clock";
clock-frequency = <32768>;
#clock-cells = <0>;
@@ -1635,7 +1635,7 @@ psci {
method = "smc";
};
- battery_reg: regulator@0 {
+ battery_reg: regulator-vdd-ac-bat {
compatible = "regulator-fixed";
regulator-name = "vdd-ac-bat";
regulator-min-microvolt = <5000000>;
@@ -1643,7 +1643,7 @@ battery_reg: regulator@0 {
regulator-always-on;
};
- vdd_3v3: regulator@1 {
+ vdd_3v3: regulator-vdd-3v3 {
compatible = "regulator-fixed";
regulator-name = "vdd-3v3";
regulator-enable-ramp-delay = <160>;
@@ -1655,7 +1655,7 @@ vdd_3v3: regulator@1 {
enable-active-high;
};
- max77620_gpio7: regulator@2 {
+ max77620_gpio7: regulator-max77620-gpio7 {
compatible = "regulator-fixed";
regulator-name = "max77620-gpio7";
regulator-enable-ramp-delay = <240>;
@@ -1669,7 +1669,7 @@ max77620_gpio7: regulator@2 {
enable-active-high;
};
- lcd_bl_en: regulator@3 {
+ lcd_bl_en: regulator-lcd-bl-en {
compatible = "regulator-fixed";
regulator-name = "lcd-bl-en";
regulator-min-microvolt = <1800000>;
@@ -1680,7 +1680,7 @@ lcd_bl_en: regulator@3 {
enable-active-high;
};
- en_vdd_sd: regulator@4 {
+ en_vdd_sd: regulator-vdd-sd {
compatible = "regulator-fixed";
regulator-name = "en-vdd-sd";
regulator-enable-ramp-delay = <472>;
@@ -1692,7 +1692,7 @@ en_vdd_sd: regulator@4 {
enable-active-high;
};
- en_vdd_cam: regulator@5 {
+ en_vdd_cam: regulator-vdd-cam {
compatible = "regulator-fixed";
regulator-name = "en-vdd-cam";
regulator-min-microvolt = <1800000>;
@@ -1702,7 +1702,7 @@ en_vdd_cam: regulator@5 {
enable-active-high;
};
- vdd_sys_boost: regulator@6 {
+ vdd_sys_boost: regulator-vdd-sys-boost {
compatible = "regulator-fixed";
regulator-name = "vdd-sys-boost";
regulator-enable-ramp-delay = <3090>;
@@ -1714,7 +1714,7 @@ vdd_sys_boost: regulator@6 {
enable-active-high;
};
- vdd_hdmi: regulator@7 {
+ vdd_hdmi: regulator-vdd-hdmi {
compatible = "regulator-fixed";
regulator-name = "vdd-hdmi";
regulator-enable-ramp-delay = <468>;
@@ -1727,21 +1727,21 @@ vdd_hdmi: regulator@7 {
enable-active-high;
};
- en_vdd_cpu_fixed: regulator@8 {
+ en_vdd_cpu_fixed: regulator-vdd-cpu-fixed {
compatible = "regulator-fixed";
regulator-name = "vdd-cpu-fixed";
regulator-min-microvolt = <1000000>;
regulator-max-microvolt = <1000000>;
};
- vdd_aux_3v3: regulator@9 {
+ vdd_aux_3v3: regulator-vdd-aux-3v3 {
compatible = "regulator-fixed";
regulator-name = "aux-3v3";
regulator-min-microvolt = <3300000>;
regulator-max-microvolt = <3300000>;
};
- vdd_snsr_pm: regulator@10 {
+ vdd_snsr_pm: regulator-vdd-snsr-pm {
compatible = "regulator-fixed";
regulator-name = "snsr_pm";
regulator-min-microvolt = <3300000>;
@@ -1750,7 +1750,7 @@ vdd_snsr_pm: regulator@10 {
enable-active-high;
};
- vdd_usb_5v0: regulator@11 {
+ vdd_usb_5v0: regulator-vdd-usb-5v0 {
compatible = "regulator-fixed";
status = "disabled";
regulator-name = "vdd-usb-5v0";
@@ -1761,7 +1761,7 @@ vdd_usb_5v0: regulator@11 {
enable-active-high;
};
- vdd_cdc_1v2_aud: regulator@101 {
+ vdd_cdc_1v2_aud: regulator-vdd-cdc-1v2-aud {
compatible = "regulator-fixed";
status = "disabled";
regulator-name = "vdd_cdc_1v2_aud";
@@ -1772,7 +1772,7 @@ vdd_cdc_1v2_aud: regulator@101 {
enable-active-high;
};
- vdd_disp_3v0: regulator@12 {
+ vdd_disp_3v0: regulator-vdd-disp-3v0 {
compatible = "regulator-fixed";
regulator-name = "vdd-disp-3v0";
regulator-enable-ramp-delay = <232>;
@@ -1784,7 +1784,7 @@ vdd_disp_3v0: regulator@12 {
enable-active-high;
};
- vdd_fan: regulator@13 {
+ vdd_fan: regulator-vdd-fan {
compatible = "regulator-fixed";
regulator-name = "vdd-fan";
regulator-enable-ramp-delay = <284>;
@@ -1795,7 +1795,7 @@ vdd_fan: regulator@13 {
enable-active-high;
};
- usb_vbus1: regulator@14 {
+ usb_vbus1: regulator-usb-vbus1 {
compatible = "regulator-fixed";
regulator-name = "usb-vbus1";
regulator-min-microvolt = <5000000>;
@@ -1806,7 +1806,7 @@ usb_vbus1: regulator@14 {
gpio-open-drain;
};
- usb_vbus2: regulator@15 {
+ usb_vbus2: regulator-usb-vbus2 {
compatible = "regulator-fixed";
regulator-name = "usb-vbus2";
regulator-min-microvolt = <5000000>;
@@ -1817,7 +1817,7 @@ usb_vbus2: regulator@15 {
gpio-open-drain;
};
- vdd_3v3_eth: regulator@16 {
+ vdd_3v3_eth: regulator-vdd-3v3-eth {
compatible = "regulator-fixed";
regulator-name = "vdd-3v3-eth-a02";
regulator-min-microvolt = <3300000>;
diff --git a/arch/arm64/boot/dts/nvidia/tegra210-p3450-0000.dts b/arch/arm64/boot/dts/nvidia/tegra210-p3450-0000.dts
index 030f264..72c2dc3 100644
--- a/arch/arm64/boot/dts/nvidia/tegra210-p3450-0000.dts
+++ b/arch/arm64/boot/dts/nvidia/tegra210-p3450-0000.dts
@@ -30,11 +30,8 @@ memory@80000000 {
pcie@1003000 {
status = "okay";
- avdd-pll-uerefe-supply = <&vdd_pex_1v05>;
hvddio-pex-supply = <&vdd_1v8>;
dvddio-pex-supply = <&vdd_pex_1v05>;
- dvdd-pex-pll-supply = <&vdd_pex_1v05>;
- hvdd-pex-pll-e-supply = <&vdd_1v8>;
vddio-pex-ctl-supply = <&vdd_1v8>;
pci@1,0 {
@@ -266,7 +263,6 @@ vdd_soc: sd0 {
regulator-min-microvolt = <1000000>;
regulator-max-microvolt = <1170000>;
regulator-enable-ramp-delay = <146>;
- regulator-disable-ramp-delay = <4080>;
regulator-ramp-delay = <27500>;
regulator-ramp-delay-scale = <300>;
regulator-always-on;
@@ -282,7 +278,6 @@ vdd_ddr: sd1 {
regulator-min-microvolt = <1150000>;
regulator-max-microvolt = <1150000>;
regulator-enable-ramp-delay = <176>;
- regulator-disable-ramp-delay = <145800>;
regulator-ramp-delay = <27500>;
regulator-ramp-delay-scale = <300>;
regulator-always-on;
@@ -298,7 +293,6 @@ vdd_pre: sd2 {
regulator-min-microvolt = <1350000>;
regulator-max-microvolt = <1350000>;
regulator-enable-ramp-delay = <176>;
- regulator-disable-ramp-delay = <32000>;
regulator-ramp-delay = <27500>;
regulator-ramp-delay-scale = <350>;
regulator-always-on;
@@ -314,7 +308,6 @@ vdd_1v8: sd3 {
regulator-min-microvolt = <1800000>;
regulator-max-microvolt = <1800000>;
regulator-enable-ramp-delay = <242>;
- regulator-disable-ramp-delay = <118000>;
regulator-ramp-delay = <27500>;
regulator-ramp-delay-scale = <360>;
regulator-always-on;
@@ -330,7 +323,6 @@ vdd_sys_1v2: ldo0 {
regulator-min-microvolt = <1200000>;
regulator-max-microvolt = <1200000>;
regulator-enable-ramp-delay = <26>;
- regulator-disable-ramp-delay = <626>;
regulator-ramp-delay = <100000>;
regulator-ramp-delay-scale = <200>;
regulator-always-on;
@@ -346,7 +338,6 @@ vdd_pex_1v05: ldo1 {
regulator-min-microvolt = <1050000>;
regulator-max-microvolt = <1050000>;
regulator-enable-ramp-delay = <22>;
- regulator-disable-ramp-delay = <650>;
regulator-ramp-delay = <100000>;
regulator-ramp-delay-scale = <200>;
@@ -360,7 +351,6 @@ vddio_sdmmc: ldo2 {
regulator-min-microvolt = <1800000>;
regulator-max-microvolt = <3300000>;
regulator-enable-ramp-delay = <62>;
- regulator-disable-ramp-delay = <650>;
regulator-ramp-delay = <100000>;
regulator-ramp-delay-scale = <200>;
@@ -378,7 +368,6 @@ vdd_rtc: ldo4 {
regulator-min-microvolt = <850000>;
regulator-max-microvolt = <1100000>;
regulator-enable-ramp-delay = <22>;
- regulator-disable-ramp-delay = <610>;
regulator-ramp-delay = <100000>;
regulator-ramp-delay-scale = <200>;
regulator-disable-active-discharge;
@@ -403,7 +392,6 @@ avdd_1v05_pll: ldo7 {
regulator-min-microvolt = <1050000>;
regulator-max-microvolt = <1050000>;
regulator-enable-ramp-delay = <24>;
- regulator-disable-ramp-delay = <2768>;
regulator-ramp-delay = <100000>;
regulator-ramp-delay-scale = <200>;
@@ -417,7 +405,6 @@ avdd_1v05: ldo8 {
regulator-min-microvolt = <1050000>;
regulator-max-microvolt = <1050000>;
regulator-enable-ramp-delay = <22>;
- regulator-disable-ramp-delay = <1160>;
regulator-ramp-delay = <100000>;
regulator-ramp-delay-scale = <200>;
@@ -456,11 +443,6 @@ usb@70090000 {
avdd-usb-supply = <&vdd_3v3_sys>;
dvddio-pex-supply = <&vdd_pex_1v05>;
hvddio-pex-supply = <&vdd_1v8>;
- /* these really belong to the XUSB pad controller */
- avdd-pll-utmip-supply = <&vdd_1v8>;
- avdd-pll-uerefe-supply = <&vdd_pex_1v05>;
- dvdd-usb-ss-pll-supply = <&vdd_pex_1v05>;
- hvdd-usb-ss-pll-e-supply = <&vdd_1v8>;
status = "okay";
};
@@ -688,7 +670,7 @@ i2s4_cif_ep: endpoint {
i2s4_port: port@1 {
reg = <1>;
- i2s4_dap_ep: endpoint@0 {
+ i2s4_dap_ep: endpoint {
dai-format = "i2s";
/* Placeholder for external Codec */
};
@@ -706,7 +688,7 @@ ports {
port@0 {
reg = <0>;
- dmic1_cif_ep: endpoint@0 {
+ dmic1_cif_ep: endpoint {
remote-endpoint = <&xbar_dmic1_ep>;
};
};
@@ -714,7 +696,7 @@ dmic1_cif_ep: endpoint@0 {
dmic1_port: port@1 {
reg = <1>;
- dmic1_dap_ep: endpoint@0 {
+ dmic1_dap_ep: endpoint {
/* Placeholder for external Codec */
};
};
@@ -731,7 +713,7 @@ ports {
port@0 {
reg = <0>;
- dmic2_cif_ep: endpoint@0 {
+ dmic2_cif_ep: endpoint {
remote-endpoint = <&xbar_dmic2_ep>;
};
};
@@ -739,7 +721,7 @@ dmic2_cif_ep: endpoint@0 {
dmic2_port: port@1 {
reg = <1>;
- dmic2_dap_ep: endpoint@0 {
+ dmic2_dap_ep: endpoint {
/* Placeholder for external Codec */
};
};
@@ -1637,7 +1619,7 @@ spi@70410000 {
status = "okay";
flash@0 {
- compatible = "spi-nor";
+ compatible = "jedec,spi-nor";
reg = <0>;
spi-max-frequency = <104000000>;
spi-tx-bus-width = <2>;
@@ -1645,7 +1627,7 @@ flash@0 {
};
};
- clk32k_in: clock@0 {
+ clk32k_in: clock-32k {
compatible = "fixed-clock";
clock-frequency = <32768>;
#clock-cells = <0>;
@@ -1684,7 +1666,7 @@ fan: fan {
};
thermal-zones {
- cpu {
+ cpu-thermal {
trips {
cpu_trip_critical: critical {
temperature = <96500>;
@@ -1762,7 +1744,7 @@ psci {
method = "smc";
};
- vdd_5v0_sys: regulator@0 {
+ vdd_5v0_sys: regulator-vdd-5v0-sys {
compatible = "regulator-fixed";
regulator-name = "VDD_5V0_SYS";
@@ -1772,14 +1754,13 @@ vdd_5v0_sys: regulator@0 {
regulator-boot-on;
};
- vdd_3v3_sys: regulator@1 {
+ vdd_3v3_sys: regulator-vdd-3v3-sys {
compatible = "regulator-fixed";
regulator-name = "VDD_3V3_SYS";
regulator-min-microvolt = <3300000>;
regulator-max-microvolt = <3300000>;
regulator-enable-ramp-delay = <240>;
- regulator-disable-ramp-delay = <11340>;
regulator-always-on;
regulator-boot-on;
@@ -1789,7 +1770,7 @@ vdd_3v3_sys: regulator@1 {
vin-supply = <&vdd_5v0_sys>;
};
- vdd_3v3_sd: regulator@2 {
+ vdd_3v3_sd: regulator-vdd-3v3-sd {
compatible = "regulator-fixed";
regulator-name = "VDD_3V3_SD";
@@ -1802,7 +1783,7 @@ vdd_3v3_sd: regulator@2 {
vin-supply = <&vdd_3v3_sys>;
};
- vdd_hdmi: regulator@3 {
+ vdd_hdmi: regulator-vdd-hdmi-5v0 {
compatible = "regulator-fixed";
regulator-name = "VDD_HDMI_5V0";
@@ -1812,7 +1793,7 @@ vdd_hdmi: regulator@3 {
vin-supply = <&vdd_5v0_sys>;
};
- vdd_hub_3v3: regulator@4 {
+ vdd_hub_3v3: regulator-vdd-hub-3v3 {
compatible = "regulator-fixed";
regulator-name = "VDD_HUB_3V3";
@@ -1825,7 +1806,7 @@ vdd_hub_3v3: regulator@4 {
vin-supply = <&vdd_5v0_sys>;
};
- vdd_cpu: regulator@5 {
+ vdd_cpu: regulator-vdd-cpu {
compatible = "regulator-fixed";
regulator-name = "VDD_CPU";
@@ -1840,7 +1821,7 @@ vdd_cpu: regulator@5 {
vin-supply = <&vdd_5v0_sys>;
};
- vdd_gpu: regulator@6 {
+ vdd_gpu: regulator-vdd-gpu {
compatible = "pwm-regulator";
pwms = <&pwm 1 8000>;
@@ -1855,7 +1836,7 @@ vdd_gpu: regulator@6 {
vin-supply = <&vdd_5v0_sys>;
};
- avdd_io_edp_1v05: regulator@7 {
+ avdd_io_edp_1v05: regulator-avdd-io-epd-1v05 {
compatible = "regulator-fixed";
regulator-name = "AVDD_IO_EDP_1V05";
@@ -1868,7 +1849,7 @@ avdd_io_edp_1v05: regulator@7 {
vin-supply = <&avdd_1v05_pll>;
};
- vdd_5v0_usb: regulator@8 {
+ vdd_5v0_usb: regulator-vdd-5v-usb {
compatible = "regulator-fixed";
regulator-name = "VDD_5V_USB";
diff --git a/arch/arm64/boot/dts/nvidia/tegra210-smaug.dts b/arch/arm64/boot/dts/nvidia/tegra210-smaug.dts
index 131c064..a263d51 100644
--- a/arch/arm64/boot/dts/nvidia/tegra210-smaug.dts
+++ b/arch/arm64/boot/dts/nvidia/tegra210-smaug.dts
@@ -1642,10 +1642,6 @@ usb@70090000 {
dvddio-pex-supply = <&avddio_1v05>;
hvddio-pex-supply = <&pp1800>;
avdd-usb-supply = <&pp3300>;
- avdd-pll-utmip-supply = <&pp1800>;
- avdd-pll-uerefe-supply = <&pp1050_avdd>;
- dvdd-pex-pll-supply = <&avddio_1v05>;
- hvdd-pex-pll-e-supply = <&pp1800>;
status = "okay";
};
@@ -1726,7 +1722,7 @@ interrupt-controller@702f9000 {
};
};
- clk32k_in: clock@0 {
+ clk32k_in: clock-32k {
compatible = "fixed-clock";
clock-frequency = <32768>;
#clock-cells = <0>;
@@ -1807,7 +1803,7 @@ psci {
method = "smc";
};
- ppvar_sys: regulator@0 {
+ ppvar_sys: regulator-ppvar-sys {
compatible = "regulator-fixed";
regulator-name = "PPVAR_SYS";
regulator-min-microvolt = <4400000>;
@@ -1815,7 +1811,7 @@ ppvar_sys: regulator@0 {
regulator-always-on;
};
- pplcd_vdd: regulator@1 {
+ pplcd_vdd: regulator-pplcd-vdd {
compatible = "regulator-fixed";
regulator-name = "PPLCD_VDD";
regulator-min-microvolt = <4400000>;
@@ -1825,7 +1821,7 @@ pplcd_vdd: regulator@1 {
regulator-boot-on;
};
- pp3000_always: regulator@2 {
+ pp3000_always: regulator-pp3000-always {
compatible = "regulator-fixed";
regulator-name = "PP3000_ALWAYS";
regulator-min-microvolt = <3000000>;
@@ -1833,7 +1829,7 @@ pp3000_always: regulator@2 {
regulator-always-on;
};
- pp3300: regulator@3 {
+ pp3300: regulator-pp3000 {
compatible = "regulator-fixed";
regulator-name = "PP3300";
regulator-min-microvolt = <3300000>;
@@ -1843,7 +1839,7 @@ pp3300: regulator@3 {
enable-active-high;
};
- pp5000: regulator@4 {
+ pp5000: regulator-pp5000 {
compatible = "regulator-fixed";
regulator-name = "PP5000";
regulator-min-microvolt = <5000000>;
@@ -1851,7 +1847,7 @@ pp5000: regulator@4 {
regulator-always-on;
};
- pp1800_lcdio: regulator@5 {
+ pp1800_lcdio: regulator-pp1800-lcdio {
compatible = "regulator-fixed";
regulator-name = "PP1800_LCDIO";
regulator-min-microvolt = <1800000>;
@@ -1861,7 +1857,7 @@ pp1800_lcdio: regulator@5 {
regulator-boot-on;
};
- pp1800_cam: regulator@6 {
+ pp1800_cam: regulator-pp1800-cam {
compatible = "regulator-fixed";
regulator-name = "PP1800_CAM";
regulator-min-microvolt = <1800000>;
@@ -1870,7 +1866,7 @@ pp1800_cam: regulator@6 {
enable-active-high;
};
- usbc_vbus: regulator@7 {
+ usbc_vbus: regulator-usbc-vbus {
compatible = "regulator-fixed";
regulator-name = "USBC_VBUS";
regulator-min-microvolt = <5000000>;
diff --git a/arch/arm64/boot/dts/nvidia/tegra210.dtsi b/arch/arm64/boot/dts/nvidia/tegra210.dtsi
index ccdc0de..218a2b3 100644
--- a/arch/arm64/boot/dts/nvidia/tegra210.dtsi
+++ b/arch/arm64/boot/dts/nvidia/tegra210.dtsi
@@ -93,8 +93,8 @@ host1x@50000000 {
interrupt-names = "syncpt", "host1x";
clocks = <&tegra_car TEGRA210_CLK_HOST1X>;
clock-names = "host1x";
- resets = <&tegra_car 28>;
- reset-names = "host1x";
+ resets = <&tegra_car 28>, <&mc TEGRA210_MC_RESET_HC>;
+ reset-names = "host1x", "mc";
#address-cells = <2>;
#size-cells = <2>;
@@ -181,6 +181,12 @@ csi@838 {
tsec@54100000 {
compatible = "nvidia,tegra210-tsec";
reg = <0x0 0x54100000 0x0 0x00040000>;
+ interrupts = <GIC_SPI 50 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&tegra_car TEGRA210_CLK_TSEC>;
+ clock-names = "tsec";
+ resets = <&tegra_car 83>;
+ reset-names = "tsec";
+ status = "disabled";
};
dc@54200000 {
@@ -283,6 +289,11 @@ nvenc@544c0000 {
tsec@54500000 {
compatible = "nvidia,tegra210-tsec";
reg = <0x0 0x54500000 0x0 0x00040000>;
+ interrupts = <GIC_SPI 80 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&tegra_car TEGRA210_CLK_TSECB>;
+ clock-names = "tsec";
+ resets = <&tegra_car 206>;
+ reset-names = "tsec";
status = "disabled";
};
@@ -1015,8 +1026,8 @@ usb@70090000 {
<&tegra_car TEGRA210_CLK_XUSB_HOST_SRC>,
<&tegra_car TEGRA210_CLK_XUSB_FALCON_SRC>,
<&tegra_car TEGRA210_CLK_XUSB_SS>,
- <&tegra_car TEGRA210_CLK_XUSB_SS_SRC>,
<&tegra_car TEGRA210_CLK_XUSB_SS_DIV2>,
+ <&tegra_car TEGRA210_CLK_XUSB_SS_SRC>,
<&tegra_car TEGRA210_CLK_XUSB_HS_SRC>,
<&tegra_car TEGRA210_CLK_XUSB_FS_SRC>,
<&tegra_car TEGRA210_CLK_PLL_U_480M>,
@@ -1024,7 +1035,7 @@ usb@70090000 {
<&tegra_car TEGRA210_CLK_PLL_E>;
clock-names = "xusb_host", "xusb_host_src",
"xusb_falcon_src", "xusb_ss",
- "xusb_ss_src", "xusb_ss_div2",
+ "xusb_ss_div2", "xusb_ss_src",
"xusb_hs_src", "xusb_fs_src",
"pll_u_480m", "clk_m", "pll_e";
resets = <&tegra_car 89>, <&tegra_car 156>,
@@ -1981,7 +1992,7 @@ sound {
};
thermal-zones {
- cpu {
+ cpu-thermal {
polling-delay-passive = <1000>;
polling-delay = <0>;
@@ -2010,7 +2021,7 @@ map0 {
};
};
- mem {
+ mem-thermal {
polling-delay-passive = <0>;
polling-delay = <0>;
@@ -2056,7 +2067,7 @@ dram-active {
};
};
- gpu {
+ gpu-thermal {
polling-delay-passive = <1000>;
polling-delay = <0>;
@@ -2085,7 +2096,7 @@ map0 {
};
};
- pllx {
+ pllx-thermal {
polling-delay-passive = <0>;
polling-delay = <0>;
diff --git a/arch/arm64/boot/dts/nvidia/tegra234-p3701-0000.dtsi b/arch/arm64/boot/dts/nvidia/tegra234-p3701-0000.dtsi
new file mode 100644
index 0000000..d95a542
--- /dev/null
+++ b/arch/arm64/boot/dts/nvidia/tegra234-p3701-0000.dtsi
@@ -0,0 +1,24 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include "tegra234.dtsi"
+
+/ {
+ model = "NVIDIA Jetson AGX Orin";
+ compatible = "nvidia,p3701-0000", "nvidia,tegra234";
+
+ bus@0 {
+ mmc@3460000 {
+ status = "okay";
+ bus-width = <8>;
+ non-removable;
+ };
+
+ rtc@c2a0000 {
+ status = "okay";
+ };
+
+ pmc@c360000 {
+ nvidia,invert-interrupt;
+ };
+ };
+};
diff --git a/arch/arm64/boot/dts/nvidia/tegra234-p3737-0000+p3701-0000.dts b/arch/arm64/boot/dts/nvidia/tegra234-p3737-0000+p3701-0000.dts
new file mode 100644
index 0000000..efbbb87
--- /dev/null
+++ b/arch/arm64/boot/dts/nvidia/tegra234-p3737-0000+p3701-0000.dts
@@ -0,0 +1,24 @@
+// SPDX-License-Identifier: GPL-2.0
+/dts-v1/;
+
+#include "tegra234-p3701-0000.dtsi"
+#include "tegra234-p3737-0000.dtsi"
+
+/ {
+ model = "NVIDIA Jetson AGX Orin Developer Kit";
+ compatible = "nvidia,p3737-0000+p3701-0000", "nvidia,p3701-0000", "nvidia,tegra234";
+
+ aliases {
+ mmc3 = "/bus@0/mmc@3460000";
+ serial0 = &tcu;
+ };
+
+ chosen {
+ bootargs = "console=ttyS0,115200n8";
+ stdout-path = "serial0:115200n8";
+ };
+
+ serial {
+ status = "okay";
+ };
+};
diff --git a/arch/arm64/boot/dts/nvidia/tegra234-p3737-0000.dtsi b/arch/arm64/boot/dts/nvidia/tegra234-p3737-0000.dtsi
new file mode 100644
index 0000000..a85993c
--- /dev/null
+++ b/arch/arm64/boot/dts/nvidia/tegra234-p3737-0000.dtsi
@@ -0,0 +1,5 @@
+// SPDX-License-Identifier: GPL-2.0
+
+/ {
+ compatible = "nvidia,p3737-0000";
+};
diff --git a/arch/arm64/boot/dts/nvidia/tegra234-sim-vdk.dts b/arch/arm64/boot/dts/nvidia/tegra234-sim-vdk.dts
index b5d9a552..5804acf 100644
--- a/arch/arm64/boot/dts/nvidia/tegra234-sim-vdk.dts
+++ b/arch/arm64/boot/dts/nvidia/tegra234-sim-vdk.dts
@@ -26,7 +26,6 @@ mmc@3460000 {
status = "okay";
bus-width = <8>;
non-removable;
- only-1-8-v;
};
rtc@c2a0000 {
diff --git a/arch/arm64/boot/dts/nvidia/tegra234.dtsi b/arch/arm64/boot/dts/nvidia/tegra234.dtsi
index f0efb3a..6b6f158 100644
--- a/arch/arm64/boot/dts/nvidia/tegra234.dtsi
+++ b/arch/arm64/boot/dts/nvidia/tegra234.dtsi
@@ -3,6 +3,7 @@
#include <dt-bindings/clock/tegra234-clock.h>
#include <dt-bindings/interrupt-controller/arm-gic.h>
#include <dt-bindings/mailbox/tegra186-hsp.h>
+#include <dt-bindings/memory/tegra234-mc.h>
#include <dt-bindings/reset/tegra234-reset.h>
/ {
@@ -25,6 +26,113 @@ misc@100000 {
status = "okay";
};
+ gpio: gpio@2200000 {
+ compatible = "nvidia,tegra234-gpio";
+ reg-names = "security", "gpio";
+ reg = <0x02200000 0x10000>,
+ <0x02210000 0x10000>;
+ interrupts = <GIC_SPI 288 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 289 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 290 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 291 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 292 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 293 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 294 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 295 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 296 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 297 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 298 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 299 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 300 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 301 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 302 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 303 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 304 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 305 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 306 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 307 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 308 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 309 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 310 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 311 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 312 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 313 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 314 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 315 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 316 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 317 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 318 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 319 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 320 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 321 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 322 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 323 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 324 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 325 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 326 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 327 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 328 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 329 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 330 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 331 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 332 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 333 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 334 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 335 IRQ_TYPE_LEVEL_HIGH>;
+ #interrupt-cells = <2>;
+ interrupt-controller;
+ #gpio-cells = <2>;
+ gpio-controller;
+ };
+
+ mc: memory-controller@2c00000 {
+ compatible = "nvidia,tegra234-mc";
+ reg = <0x02c00000 0x100000>,
+ <0x02b80000 0x040000>,
+ <0x01700000 0x100000>;
+ interrupts = <GIC_SPI 223 IRQ_TYPE_LEVEL_HIGH>;
+ #interconnect-cells = <1>;
+ status = "okay";
+
+ #address-cells = <2>;
+ #size-cells = <2>;
+
+ ranges = <0x01700000 0x0 0x01700000 0x0 0x100000>,
+ <0x02b80000 0x0 0x02b80000 0x0 0x040000>,
+ <0x02c00000 0x0 0x02c00000 0x0 0x100000>;
+
+ /*
+ * Bit 39 of addresses passing through the memory
+ * controller selects the XBAR format used when memory
+ * is accessed. This is used to transparently access
+ * memory in the XBAR format used by the discrete GPU
+ * (bit 39 set) or Tegra (bit 39 clear).
+ *
+ * As a consequence, the operating system must ensure
+ * that bit 39 is never used implicitly, for example
+ * via an I/O virtual address mapping of an IOMMU. If
+ * devices require access to the XBAR switch, their
+ * drivers must set this bit explicitly.
+ *
+ * Limit the DMA range for memory clients to [38:0].
+ */
+ dma-ranges = <0x0 0x0 0x0 0x80 0x0>;
+
+ emc: external-memory-controller@2c60000 {
+ compatible = "nvidia,tegra234-emc";
+ reg = <0x0 0x02c60000 0x0 0x90000>,
+ <0x0 0x01780000 0x0 0x80000>;
+ interrupts = <GIC_SPI 224 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&bpmp TEGRA234_CLK_EMC>;
+ clock-names = "emc";
+ status = "okay";
+
+ #interconnect-cells = <0>;
+
+ nvidia,bpmp = <&bpmp>;
+ };
+ };
+
uarta: serial@3100000 {
compatible = "nvidia,tegra234-uart", "nvidia,tegra20-uart";
reg = <0x03100000 0x10000>;
@@ -40,11 +148,27 @@ mmc@3460000 {
compatible = "nvidia,tegra234-sdhci", "nvidia,tegra186-sdhci";
reg = <0x03460000 0x20000>;
interrupts = <GIC_SPI 65 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&bpmp TEGRA234_CLK_SDMMC4>;
- clock-names = "sdhci";
+ clocks = <&bpmp TEGRA234_CLK_SDMMC4>,
+ <&bpmp TEGRA234_CLK_SDMMC_LEGACY_TM>;
+ clock-names = "sdhci", "tmclk";
+ assigned-clocks = <&bpmp TEGRA234_CLK_SDMMC4>,
+ <&bpmp TEGRA234_CLK_PLLC4>;
+ assigned-clock-parents = <&bpmp TEGRA234_CLK_PLLC4>;
resets = <&bpmp TEGRA234_RESET_SDMMC4>;
reset-names = "sdhci";
- dma-coherent;
+ interconnects = <&mc TEGRA234_MEMORY_CLIENT_SDMMCRAB &emc>,
+ <&mc TEGRA234_MEMORY_CLIENT_SDMMCWAB &emc>;
+ interconnect-names = "dma-mem", "write";
+ nvidia,pad-autocal-pull-up-offset-hs400 = <0x00>;
+ nvidia,pad-autocal-pull-down-offset-hs400 = <0x00>;
+ nvidia,pad-autocal-pull-up-offset-1v8-timeout = <0x0a>;
+ nvidia,pad-autocal-pull-down-offset-1v8-timeout = <0x0a>;
+ nvidia,pad-autocal-pull-up-offset-3v3-timeout = <0x0a>;
+ nvidia,pad-autocal-pull-down-offset-3v3-timeout = <0x0a>;
+ nvidia,default-tap = <0x8>;
+ nvidia,default-trim = <0x14>;
+ nvidia,dqs-trim = <40>;
+ supports-cqe;
status = "disabled";
};
@@ -93,9 +217,26 @@ rtc@c2a0000 {
reg = <0x0c2a0000 0x10000>;
interrupt-parent = <&pmc>;
interrupts = <73 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&bpmp TEGRA234_CLK_CLK_32K>;
+ clock-names = "rtc";
status = "disabled";
};
+ gpio_aon: gpio@c2f0000 {
+ compatible = "nvidia,tegra234-gpio-aon";
+ reg-names = "security", "gpio";
+ reg = <0x0c2f0000 0x1000>,
+ <0x0c2f1000 0x1000>;
+ interrupts = <GIC_SPI 56 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 57 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 58 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 59 IRQ_TYPE_LEVEL_HIGH>;
+ #interrupt-cells = <2>;
+ interrupt-controller;
+ #gpio-cells = <2>;
+ gpio-controller;
+ };
+
pmc: pmc@c360000 {
compatible = "nvidia,tegra234-pmc";
reg = <0x0c360000 0x10000>,
@@ -122,21 +263,21 @@ gic: interrupt-controller@f400000 {
};
};
- sysram@40000000 {
+ sram@40000000 {
compatible = "nvidia,tegra234-sysram", "mmio-sram";
- reg = <0x0 0x40000000 0x0 0x50000>;
+ reg = <0x0 0x40000000 0x0 0x80000>;
#address-cells = <1>;
#size-cells = <1>;
- ranges = <0x0 0x0 0x40000000 0x50000>;
+ ranges = <0x0 0x0 0x40000000 0x80000>;
- cpu_bpmp_tx: shmem@4e000 {
- reg = <0x4e000 0x1000>;
+ cpu_bpmp_tx: sram@70000 {
+ reg = <0x70000 0x1000>;
label = "cpu-bpmp-tx";
pool;
};
- cpu_bpmp_rx: shmem@4f000 {
- reg = <0x4f000 0x1000>;
+ cpu_bpmp_rx: sram@71000 {
+ reg = <0x71000 0x1000>;
label = "cpu-bpmp-rx";
pool;
};
@@ -146,10 +287,15 @@ bpmp: bpmp {
compatible = "nvidia,tegra234-bpmp", "nvidia,tegra186-bpmp";
mboxes = <&hsp_top0 TEGRA_HSP_MBOX_TYPE_DB
TEGRA_HSP_DB_MASTER_BPMP>;
- shmem = <&cpu_bpmp_tx &cpu_bpmp_rx>;
+ shmem = <&cpu_bpmp_tx>, <&cpu_bpmp_rx>;
#clock-cells = <1>;
#reset-cells = <1>;
#power-domain-cells = <1>;
+ interconnects = <&mc TEGRA234_MEMORY_CLIENT_BPMPR &emc>,
+ <&mc TEGRA234_MEMORY_CLIENT_BPMPW &emc>,
+ <&mc TEGRA234_MEMORY_CLIENT_BPMPDMAR &emc>,
+ <&mc TEGRA234_MEMORY_CLIENT_BPMPDMAW &emc>;
+ interconnect-names = "read", "write", "dma-mem", "dma-write";
bpmp_i2c: i2c {
compatible = "nvidia,tegra186-bpmp-i2c";
@@ -163,12 +309,373 @@ cpus {
#address-cells = <1>;
#size-cells = <0>;
- cpu@0 {
+ cpu0_0: cpu@0 {
+ compatible = "arm,cortex-a78";
device_type = "cpu";
- reg = <0x000>;
+ reg = <0x00000>;
enable-method = "psci";
+
+ i-cache-size = <65536>;
+ i-cache-line-size = <64>;
+ i-cache-sets = <256>;
+ d-cache-size = <65536>;
+ d-cache-line-size = <64>;
+ d-cache-sets = <256>;
+ next-level-cache = <&l2c0_0>;
};
+
+ cpu0_1: cpu@100 {
+ compatible = "arm,cortex-a78";
+ device_type = "cpu";
+ reg = <0x00100>;
+
+ enable-method = "psci";
+
+ i-cache-size = <65536>;
+ i-cache-line-size = <64>;
+ i-cache-sets = <256>;
+ d-cache-size = <65536>;
+ d-cache-line-size = <64>;
+ d-cache-sets = <256>;
+ next-level-cache = <&l2c0_1>;
+ };
+
+ cpu0_2: cpu@200 {
+ compatible = "arm,cortex-a78";
+ device_type = "cpu";
+ reg = <0x00200>;
+
+ enable-method = "psci";
+
+ i-cache-size = <65536>;
+ i-cache-line-size = <64>;
+ i-cache-sets = <256>;
+ d-cache-size = <65536>;
+ d-cache-line-size = <64>;
+ d-cache-sets = <256>;
+ next-level-cache = <&l2c0_2>;
+ };
+
+ cpu0_3: cpu@300 {
+ compatible = "arm,cortex-a78";
+ device_type = "cpu";
+ reg = <0x00300>;
+
+ enable-method = "psci";
+
+ i-cache-size = <65536>;
+ i-cache-line-size = <64>;
+ i-cache-sets = <256>;
+ d-cache-size = <65536>;
+ d-cache-line-size = <64>;
+ d-cache-sets = <256>;
+ next-level-cache = <&l2c0_3>;
+ };
+
+ cpu1_0: cpu@10000 {
+ compatible = "arm,cortex-a78";
+ device_type = "cpu";
+ reg = <0x10000>;
+
+ enable-method = "psci";
+
+ i-cache-size = <65536>;
+ i-cache-line-size = <64>;
+ i-cache-sets = <256>;
+ d-cache-size = <65536>;
+ d-cache-line-size = <64>;
+ d-cache-sets = <256>;
+ next-level-cache = <&l2c1_0>;
+ };
+
+ cpu1_1: cpu@10100 {
+ compatible = "arm,cortex-a78";
+ device_type = "cpu";
+ reg = <0x10100>;
+
+ enable-method = "psci";
+
+ i-cache-size = <65536>;
+ i-cache-line-size = <64>;
+ i-cache-sets = <256>;
+ d-cache-size = <65536>;
+ d-cache-line-size = <64>;
+ d-cache-sets = <256>;
+ next-level-cache = <&l2c1_1>;
+ };
+
+ cpu1_2: cpu@10200 {
+ compatible = "arm,cortex-a78";
+ device_type = "cpu";
+ reg = <0x10200>;
+
+ enable-method = "psci";
+
+ i-cache-size = <65536>;
+ i-cache-line-size = <64>;
+ i-cache-sets = <256>;
+ d-cache-size = <65536>;
+ d-cache-line-size = <64>;
+ d-cache-sets = <256>;
+ next-level-cache = <&l2c1_2>;
+ };
+
+ cpu1_3: cpu@10300 {
+ compatible = "arm,cortex-a78";
+ device_type = "cpu";
+ reg = <0x10300>;
+
+ enable-method = "psci";
+
+ i-cache-size = <65536>;
+ i-cache-line-size = <64>;
+ i-cache-sets = <256>;
+ d-cache-size = <65536>;
+ d-cache-line-size = <64>;
+ d-cache-sets = <256>;
+ next-level-cache = <&l2c1_3>;
+ };
+
+ cpu2_0: cpu@20000 {
+ compatible = "arm,cortex-a78";
+ device_type = "cpu";
+ reg = <0x20000>;
+
+ enable-method = "psci";
+
+ i-cache-size = <65536>;
+ i-cache-line-size = <64>;
+ i-cache-sets = <256>;
+ d-cache-size = <65536>;
+ d-cache-line-size = <64>;
+ d-cache-sets = <256>;
+ next-level-cache = <&l2c2_0>;
+ };
+
+ cpu2_1: cpu@20100 {
+ compatible = "arm,cortex-a78";
+ device_type = "cpu";
+ reg = <0x20100>;
+
+ enable-method = "psci";
+
+ i-cache-size = <65536>;
+ i-cache-line-size = <64>;
+ i-cache-sets = <256>;
+ d-cache-size = <65536>;
+ d-cache-line-size = <64>;
+ d-cache-sets = <256>;
+ next-level-cache = <&l2c2_1>;
+ };
+
+ cpu2_2: cpu@20200 {
+ compatible = "arm,cortex-a78";
+ device_type = "cpu";
+ reg = <0x20200>;
+
+ enable-method = "psci";
+
+ i-cache-size = <65536>;
+ i-cache-line-size = <64>;
+ i-cache-sets = <256>;
+ d-cache-size = <65536>;
+ d-cache-line-size = <64>;
+ d-cache-sets = <256>;
+ next-level-cache = <&l2c2_2>;
+ };
+
+ cpu2_3: cpu@20300 {
+ compatible = "arm,cortex-a78";
+ device_type = "cpu";
+ reg = <0x20300>;
+
+ enable-method = "psci";
+
+ i-cache-size = <65536>;
+ i-cache-line-size = <64>;
+ i-cache-sets = <256>;
+ d-cache-size = <65536>;
+ d-cache-line-size = <64>;
+ d-cache-sets = <256>;
+ next-level-cache = <&l2c2_3>;
+ };
+
+ cpu-map {
+ cluster0 {
+ core0 {
+ cpu = <&cpu0_0>;
+ };
+
+ core1 {
+ cpu = <&cpu0_1>;
+ };
+
+ core2 {
+ cpu = <&cpu0_2>;
+ };
+
+ core3 {
+ cpu = <&cpu0_3>;
+ };
+ };
+
+ cluster1 {
+ core0 {
+ cpu = <&cpu1_0>;
+ };
+
+ core1 {
+ cpu = <&cpu1_1>;
+ };
+
+ core2 {
+ cpu = <&cpu1_2>;
+ };
+
+ core3 {
+ cpu = <&cpu1_3>;
+ };
+ };
+
+ cluster2 {
+ core0 {
+ cpu = <&cpu2_0>;
+ };
+
+ core1 {
+ cpu = <&cpu2_1>;
+ };
+
+ core2 {
+ cpu = <&cpu2_2>;
+ };
+
+ core3 {
+ cpu = <&cpu2_3>;
+ };
+ };
+ };
+
+ l2c0_0: l2-cache00 {
+ cache-size = <262144>;
+ cache-line-size = <64>;
+ cache-sets = <512>;
+ cache-unified;
+ next-level-cache = <&l3c0>;
+ };
+
+ l2c0_1: l2-cache01 {
+ cache-size = <262144>;
+ cache-line-size = <64>;
+ cache-sets = <512>;
+ cache-unified;
+ next-level-cache = <&l3c0>;
+ };
+
+ l2c0_2: l2-cache02 {
+ cache-size = <262144>;
+ cache-line-size = <64>;
+ cache-sets = <512>;
+ cache-unified;
+ next-level-cache = <&l3c0>;
+ };
+
+ l2c0_3: l2-cache03 {
+ cache-size = <262144>;
+ cache-line-size = <64>;
+ cache-sets = <512>;
+ cache-unified;
+ next-level-cache = <&l3c0>;
+ };
+
+ l2c1_0: l2-cache10 {
+ cache-size = <262144>;
+ cache-line-size = <64>;
+ cache-sets = <512>;
+ cache-unified;
+ next-level-cache = <&l3c1>;
+ };
+
+ l2c1_1: l2-cache11 {
+ cache-size = <262144>;
+ cache-line-size = <64>;
+ cache-sets = <512>;
+ cache-unified;
+ next-level-cache = <&l3c1>;
+ };
+
+ l2c1_2: l2-cache12 {
+ cache-size = <262144>;
+ cache-line-size = <64>;
+ cache-sets = <512>;
+ cache-unified;
+ next-level-cache = <&l3c1>;
+ };
+
+ l2c1_3: l2-cache13 {
+ cache-size = <262144>;
+ cache-line-size = <64>;
+ cache-sets = <512>;
+ cache-unified;
+ next-level-cache = <&l3c1>;
+ };
+
+ l2c2_0: l2-cache20 {
+ cache-size = <262144>;
+ cache-line-size = <64>;
+ cache-sets = <512>;
+ cache-unified;
+ next-level-cache = <&l3c2>;
+ };
+
+ l2c2_1: l2-cache21 {
+ cache-size = <262144>;
+ cache-line-size = <64>;
+ cache-sets = <512>;
+ cache-unified;
+ next-level-cache = <&l3c2>;
+ };
+
+ l2c2_2: l2-cache22 {
+ cache-size = <262144>;
+ cache-line-size = <64>;
+ cache-sets = <512>;
+ cache-unified;
+ next-level-cache = <&l3c2>;
+ };
+
+ l2c2_3: l2-cache23 {
+ cache-size = <262144>;
+ cache-line-size = <64>;
+ cache-sets = <512>;
+ cache-unified;
+ next-level-cache = <&l3c2>;
+ };
+
+ l3c0: l3-cache0 {
+ cache-size = <2097152>;
+ cache-line-size = <64>;
+ cache-sets = <2048>;
+ };
+
+ l3c1: l3-cache1 {
+ cache-size = <2097152>;
+ cache-line-size = <64>;
+ cache-sets = <2048>;
+ };
+
+ l3c2: l3-cache2 {
+ cache-size = <2097152>;
+ cache-line-size = <64>;
+ cache-sets = <2048>;
+ };
+ };
+
+ pmu {
+ compatible = "arm,cortex-a78-pmu";
+ interrupts = <GIC_PPI 7 IRQ_TYPE_LEVEL_HIGH>;
+ status = "okay";
};
psci {
@@ -177,6 +684,14 @@ psci {
method = "smc";
};
+ tcu: serial {
+ compatible = "nvidia,tegra234-tcu", "nvidia,tegra194-tcu";
+ mboxes = <&hsp_top0 TEGRA_HSP_MBOX_TYPE_SM TEGRA_HSP_SM_RX(0)>,
+ <&hsp_aon TEGRA_HSP_MBOX_TYPE_SM TEGRA_HSP_SM_TX(1)>;
+ mbox-names = "rx", "tx";
+ status = "disabled";
+ };
+
timer {
compatible = "arm,armv8-timer";
interrupts = <GIC_PPI 13 (GIC_CPU_MASK_SIMPLE(4) | IRQ_TYPE_LEVEL_LOW)>,
diff --git a/arch/arm64/boot/dts/qcom/Makefile b/arch/arm64/boot/dts/qcom/Makefile
index 6b816eb..f7232052 100644
--- a/arch/arm64/boot/dts/qcom/Makefile
+++ b/arch/arm64/boot/dts/qcom/Makefile
@@ -15,6 +15,7 @@
dtb-$(CONFIG_ARCH_QCOM) += msm8916-mtp.dtb
dtb-$(CONFIG_ARCH_QCOM) += msm8916-samsung-a3u-eur.dtb
dtb-$(CONFIG_ARCH_QCOM) += msm8916-samsung-a5u-eur.dtb
+dtb-$(CONFIG_ARCH_QCOM) += msm8916-samsung-j5.dtb
dtb-$(CONFIG_ARCH_QCOM) += msm8916-samsung-serranove.dtb
dtb-$(CONFIG_ARCH_QCOM) += msm8916-wingtech-wt88047.dtb
dtb-$(CONFIG_ARCH_QCOM) += msm8992-bullhead-rev-101.dtb
@@ -57,6 +58,7 @@
dtb-$(CONFIG_ARCH_QCOM) += sc7180-trogdor-coachz-r3-lte.dtb
dtb-$(CONFIG_ARCH_QCOM) += sc7180-trogdor-homestar-r2.dtb
dtb-$(CONFIG_ARCH_QCOM) += sc7180-trogdor-homestar-r3.dtb
+dtb-$(CONFIG_ARCH_QCOM) += sc7180-trogdor-homestar-r4.dtb
dtb-$(CONFIG_ARCH_QCOM) += sc7180-trogdor-lazor-r0.dtb
dtb-$(CONFIG_ARCH_QCOM) += sc7180-trogdor-lazor-r1.dtb
dtb-$(CONFIG_ARCH_QCOM) += sc7180-trogdor-lazor-r1-kb.dtb
@@ -64,9 +66,14 @@
dtb-$(CONFIG_ARCH_QCOM) += sc7180-trogdor-lazor-r3.dtb
dtb-$(CONFIG_ARCH_QCOM) += sc7180-trogdor-lazor-r3-kb.dtb
dtb-$(CONFIG_ARCH_QCOM) += sc7180-trogdor-lazor-r3-lte.dtb
-dtb-$(CONFIG_ARCH_QCOM) += sc7180-trogdor-lazor-limozeen.dtb
-dtb-$(CONFIG_ARCH_QCOM) += sc7180-trogdor-lazor-limozeen-nots.dtb
+dtb-$(CONFIG_ARCH_QCOM) += sc7180-trogdor-lazor-r9.dtb
+dtb-$(CONFIG_ARCH_QCOM) += sc7180-trogdor-lazor-r9-kb.dtb
+dtb-$(CONFIG_ARCH_QCOM) += sc7180-trogdor-lazor-r9-lte.dtb
+dtb-$(CONFIG_ARCH_QCOM) += sc7180-trogdor-lazor-limozeen-r4.dtb
+dtb-$(CONFIG_ARCH_QCOM) += sc7180-trogdor-lazor-limozeen-r9.dtb
dtb-$(CONFIG_ARCH_QCOM) += sc7180-trogdor-lazor-limozeen-nots-r4.dtb
+dtb-$(CONFIG_ARCH_QCOM) += sc7180-trogdor-lazor-limozeen-nots-r5.dtb
+dtb-$(CONFIG_ARCH_QCOM) += sc7180-trogdor-lazor-limozeen-nots-r9.dtb
dtb-$(CONFIG_ARCH_QCOM) += sc7180-trogdor-pompom-r1.dtb
dtb-$(CONFIG_ARCH_QCOM) += sc7180-trogdor-pompom-r1-lte.dtb
dtb-$(CONFIG_ARCH_QCOM) += sc7180-trogdor-pompom-r2.dtb
@@ -78,6 +85,7 @@
dtb-$(CONFIG_ARCH_QCOM) += sc7280-herobrine.dtb
dtb-$(CONFIG_ARCH_QCOM) += sc7280-idp.dtb
dtb-$(CONFIG_ARCH_QCOM) += sc7280-idp2.dtb
+dtb-$(CONFIG_ARCH_QCOM) += sc7280-crd.dtb
dtb-$(CONFIG_ARCH_QCOM) += sdm630-sony-xperia-ganges-kirin.dtb
dtb-$(CONFIG_ARCH_QCOM) += sdm630-sony-xperia-nile-discovery.dtb
dtb-$(CONFIG_ARCH_QCOM) += sdm630-sony-xperia-nile-pioneer.dtb
@@ -91,6 +99,9 @@
dtb-$(CONFIG_ARCH_QCOM) += sdm845-mtp.dtb
dtb-$(CONFIG_ARCH_QCOM) += sdm845-oneplus-enchilada.dtb
dtb-$(CONFIG_ARCH_QCOM) += sdm845-oneplus-fajita.dtb
+dtb-$(CONFIG_ARCH_QCOM) += sdm845-sony-xperia-tama-akari.dtb
+dtb-$(CONFIG_ARCH_QCOM) += sdm845-sony-xperia-tama-akatsuki.dtb
+dtb-$(CONFIG_ARCH_QCOM) += sdm845-sony-xperia-tama-apollo.dtb
dtb-$(CONFIG_ARCH_QCOM) += sdm845-xiaomi-beryllium.dtb
dtb-$(CONFIG_ARCH_QCOM) += sdm850-lenovo-yoga-c630.dtb
dtb-$(CONFIG_ARCH_QCOM) += sm6125-sony-xperia-seine-pdx201.dtb
@@ -106,4 +117,8 @@
dtb-$(CONFIG_ARCH_QCOM) += sm8250-sony-xperia-edo-pdx203.dtb
dtb-$(CONFIG_ARCH_QCOM) += sm8250-sony-xperia-edo-pdx206.dtb
dtb-$(CONFIG_ARCH_QCOM) += sm8350-hdk.dtb
+dtb-$(CONFIG_ARCH_QCOM) += sm8350-microsoft-surface-duo2.dtb
dtb-$(CONFIG_ARCH_QCOM) += sm8350-mtp.dtb
+dtb-$(CONFIG_ARCH_QCOM) += sm8350-sony-xperia-sagami-pdx214.dtb
+dtb-$(CONFIG_ARCH_QCOM) += sm8350-sony-xperia-sagami-pdx215.dtb
+dtb-$(CONFIG_ARCH_QCOM) += sm8450-qrd.dtb
diff --git a/arch/arm64/boot/dts/qcom/apq8096-db820c.dts b/arch/arm64/boot/dts/qcom/apq8096-db820c.dts
index d01a512..f623db8 100644
--- a/arch/arm64/boot/dts/qcom/apq8096-db820c.dts
+++ b/arch/arm64/boot/dts/qcom/apq8096-db820c.dts
@@ -166,6 +166,7 @@ bluetooth {
&adsp_pil {
status = "okay";
+ firmware-name = "qcom/apq8096/adsp.mbn";
};
&blsp2_i2c1 {
@@ -232,6 +233,7 @@ &hdmi_phy {
&hsusb_phy1 {
status = "okay";
+ vdd-supply = <&vreg_l28a_0p925>;
vdda-pll-supply = <&vreg_l12a_1p8>;
vdda-phy-dpdm-supply = <&vreg_l24a_3p075>;
};
@@ -239,6 +241,7 @@ &hsusb_phy1 {
&hsusb_phy2 {
status = "okay";
+ vdd-supply = <&vreg_l28a_0p925>;
vdda-pll-supply = <&vreg_l12a_1p8>;
vdda-phy-dpdm-supply = <&vreg_l24a_3p075>;
};
@@ -632,6 +635,8 @@ &pm8994_mpps {
&pm8994_spmi_regulators {
qcom,saw-reg = <&saw3>;
+ vdd_s11-supply = <&vph_pwr>;
+
s9 {
qcom,saw-slave;
};
@@ -640,6 +645,7 @@ s10 {
};
s11 {
qcom,saw-leader;
+ regulator-name = "VDD_APCC";
regulator-always-on;
regulator-min-microvolt = <980000>;
regulator-max-microvolt = <980000>;
@@ -672,6 +678,8 @@ pinconf {
};
&pmi8994_spmi_regulators {
+ vdd_s2-supply = <&vph_pwr>;
+
vdd_gfx: s2@1700 {
reg = <0x1700 0x100>;
regulator-name = "VDD_GFX";
@@ -709,7 +717,7 @@ pm8994-regulators {
vdd_l17_l29-supply = <&vph_pwr_bbyp>;
vdd_l20_l21-supply = <&vph_pwr_bbyp>;
vdd_l25-supply = <&vreg_s3a_1p3>;
- vdd_lvs1_lvs2-supply = <&vreg_s4a_1p8>;
+ vdd_lvs1_2-supply = <&vreg_s4a_1p8>;
vreg_s3a_1p3: s3 {
regulator-name = "vreg_s3a_1p3";
diff --git a/arch/arm64/boot/dts/qcom/ipq6018.dtsi b/arch/arm64/boot/dts/qcom/ipq6018.dtsi
index 933b561..66ec561 100644
--- a/arch/arm64/boot/dts/qcom/ipq6018.dtsi
+++ b/arch/arm64/boot/dts/qcom/ipq6018.dtsi
@@ -220,7 +220,7 @@ tlmm: pinctrl@1000000 {
interrupts = <GIC_SPI 208 IRQ_TYPE_LEVEL_HIGH>;
gpio-controller;
#gpio-cells = <2>;
- gpio-ranges = <&tlmm 0 80>;
+ gpio-ranges = <&tlmm 0 0 80>;
interrupt-controller;
#interrupt-cells = <2>;
diff --git a/arch/arm64/boot/dts/qcom/ipq8074.dtsi b/arch/arm64/boot/dts/qcom/ipq8074.dtsi
index 6c6a0f8..e6cc261 100644
--- a/arch/arm64/boot/dts/qcom/ipq8074.dtsi
+++ b/arch/arm64/boot/dts/qcom/ipq8074.dtsi
@@ -230,6 +230,18 @@ pcie_phy1: phy@8e200 {
};
};
+ mdio: mdio@90000 {
+ compatible = "qcom,ipq4019-mdio";
+ reg = <0x00090000 0x64>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ clocks = <&gcc GCC_MDIO_AHB_CLK>;
+ clock-names = "gcc_mdio_ahb_clk";
+
+ status = "disabled";
+ };
+
prng: rng@e3000 {
compatible = "qcom,prng-ee";
reg = <0x000e3000 0x1000>;
diff --git a/arch/arm64/boot/dts/qcom/msm8916-samsung-j5.dts b/arch/arm64/boot/dts/qcom/msm8916-samsung-j5.dts
new file mode 100644
index 0000000..687bea4
--- /dev/null
+++ b/arch/arm64/boot/dts/qcom/msm8916-samsung-j5.dts
@@ -0,0 +1,209 @@
+// SPDX-License-Identifier: GPL-2.0-only
+
+/dts-v1/;
+
+#include "msm8916-pm8916.dtsi"
+#include <dt-bindings/gpio/gpio.h>
+
+/ {
+ model = "Samsung Galaxy J5 (2015)";
+ compatible = "samsung,j5", "qcom,msm8916";
+ chassis-type = "handset";
+
+ aliases {
+ serial0 = &blsp1_uart2;
+ };
+
+ chosen {
+ stdout-path = "serial0";
+ };
+
+ reserved-memory {
+ /* Additional memory used by Samsung firmware modifications */
+ tz-apps@85500000 {
+ reg = <0x0 0x85500000 0x0 0xb00000>;
+ no-map;
+ };
+ };
+
+ gpio-keys {
+ compatible = "gpio-keys";
+
+ pinctrl-names = "default";
+ pinctrl-0 = <&gpio_keys_default>;
+
+ label = "GPIO Buttons";
+
+ volume-up {
+ label = "Volume Up";
+ gpios = <&msmgpio 107 GPIO_ACTIVE_LOW>;
+ linux,code = <KEY_VOLUMEUP>;
+ };
+
+ home-key {
+ lable = "Home Key";
+ gpios = <&msmgpio 109 GPIO_ACTIVE_LOW>;
+ linux,code = <KEY_HOMEPAGE>;
+ };
+ };
+};
+
+&blsp1_uart2 {
+ status = "okay";
+};
+
+&pm8916_resin {
+ status = "okay";
+ linux,code = <KEY_VOLUMEDOWN>;
+};
+
+/* FIXME: Replace with SM5703 MUIC when driver is available */
+&pm8916_usbin {
+ status = "okay";
+};
+
+&pronto {
+ status = "okay";
+};
+
+&sdhc_1 {
+ status = "okay";
+
+ pinctrl-names = "default", "sleep";
+ pinctrl-0 = <&sdc1_clk_on &sdc1_cmd_on &sdc1_data_on>;
+ pinctrl-1 = <&sdc1_clk_off &sdc1_cmd_off &sdc1_data_off>;
+};
+
+&sdhc_2 {
+ status = "okay";
+
+ pinctrl-names = "default", "sleep";
+ pinctrl-0 = <&sdc2_clk_on &sdc2_cmd_on &sdc2_data_on &sdc2_cd_on>;
+ pinctrl-1 = <&sdc2_clk_off &sdc2_cmd_off &sdc2_data_off &sdc2_cd_off>;
+
+ cd-gpios = <&msmgpio 38 GPIO_ACTIVE_LOW>;
+};
+
+&usb {
+ status = "okay";
+ dr_mode = "peripheral";
+ extcon = <&pm8916_usbin>;
+};
+
+&usb_hs_phy {
+ extcon = <&pm8916_usbin>;
+ qcom,init-seq = /bits/ 8 <0x1 0x19 0x2 0x0b>;
+};
+
+&smd_rpm_regulators {
+ vdd_l1_l2_l3-supply = <&pm8916_s3>;
+ vdd_l4_l5_l6-supply = <&pm8916_s4>;
+ vdd_l7-supply = <&pm8916_s4>;
+
+ s3 {
+ regulator-min-microvolt = <1200000>;
+ regulator-max-microvolt = <1300000>;
+ };
+
+ s4 {
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <2100000>;
+ };
+
+ l1 {
+ regulator-min-microvolt = <1225000>;
+ regulator-max-microvolt = <1225000>;
+ };
+
+ l2 {
+ regulator-min-microvolt = <1200000>;
+ regulator-max-microvolt = <1200000>;
+ };
+
+ l4 {
+ regulator-min-microvolt = <2050000>;
+ regulator-max-microvolt = <2050000>;
+ };
+
+ l5 {
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ };
+
+ l6 {
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ };
+
+ l7 {
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ };
+
+ l8 {
+ regulator-min-microvolt = <2850000>;
+ regulator-max-microvolt = <2900000>;
+ };
+
+ l9 {
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ };
+
+ l10 {
+ regulator-min-microvolt = <2700000>;
+ regulator-max-microvolt = <2800000>;
+ };
+
+ l11 {
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <2950000>;
+ regulator-allow-set-load;
+ regulator-system-load = <200000>;
+ };
+
+ l12 {
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <2950000>;
+ };
+
+ l13 {
+ regulator-min-microvolt = <3075000>;
+ regulator-max-microvolt = <3075000>;
+ };
+
+ l14 {
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <3300000>;
+ };
+
+ l15 {
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <3300000>;
+ };
+
+ l16 {
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <3300000>;
+ };
+
+ l17 {
+ regulator-min-microvolt = <3000000>;
+ regulator-max-microvolt = <3000000>;
+ };
+
+ l18 {
+ regulator-min-microvolt = <2700000>;
+ regulator-max-microvolt = <2700000>;
+ };
+};
+
+&msmgpio {
+ gpio_keys_default: gpio-keys-default {
+ pins = "gpio107", "gpio109";
+ function = "gpio";
+
+ drive-strength = <2>;
+ bias-pull-up;
+ };
+};
diff --git a/arch/arm64/boot/dts/qcom/msm8916-samsung-serranove.dts b/arch/arm64/boot/dts/qcom/msm8916-samsung-serranove.dts
index a78f87a..58dfbff 100644
--- a/arch/arm64/boot/dts/qcom/msm8916-samsung-serranove.dts
+++ b/arch/arm64/boot/dts/qcom/msm8916-samsung-serranove.dts
@@ -23,6 +23,7 @@
/ {
model = "Samsung Galaxy S4 Mini Value Edition";
compatible = "samsung,serranove", "qcom,msm8916";
+ chassis-type = "handset";
aliases {
serial0 = &blsp1_uart2;
diff --git a/arch/arm64/boot/dts/qcom/msm8916.dtsi b/arch/arm64/boot/dts/qcom/msm8916.dtsi
index c1c42f2..41897eb 100644
--- a/arch/arm64/boot/dts/qcom/msm8916.dtsi
+++ b/arch/arm64/boot/dts/qcom/msm8916.dtsi
@@ -19,8 +19,8 @@ / {
#size-cells = <2>;
aliases {
- sdhc1 = &sdhc_1; /* SDC1 eMMC slot */
- sdhc2 = &sdhc_2; /* SDC2 SD card slot */
+ mmc0 = &sdhc_1; /* SDC1 eMMC slot */
+ mmc1 = &sdhc_2; /* SDC2 SD card slot */
};
chosen { };
@@ -453,6 +453,11 @@ rpm_msg_ram: sram@60000 {
reg = <0x00060000 0x8000>;
};
+ sram@290000 {
+ compatible = "qcom,msm8916-rpm-stats";
+ reg = <0x00290000 0x10000>;
+ };
+
bimc: interconnect@400000 {
compatible = "qcom,msm8916-bimc";
reg = <0x00400000 0x62000>;
diff --git a/arch/arm64/boot/dts/qcom/msm8992-xiaomi-libra.dts b/arch/arm64/boot/dts/qcom/msm8992-xiaomi-libra.dts
index 69fcb6b..84558ab 100644
--- a/arch/arm64/boot/dts/qcom/msm8992-xiaomi-libra.dts
+++ b/arch/arm64/boot/dts/qcom/msm8992-xiaomi-libra.dts
@@ -42,7 +42,6 @@ framebuffer0: framebuffer@3404000 {
gpio_keys {
compatible = "gpio-keys";
- input-name = "gpio-keys";
#address-cells = <1>;
#size-cells = <0>;
autorepeat;
diff --git a/arch/arm64/boot/dts/qcom/msm8994-msft-lumia-octagon.dtsi b/arch/arm64/boot/dts/qcom/msm8994-msft-lumia-octagon.dtsi
index 3a3790a..cc038f9 100644
--- a/arch/arm64/boot/dts/qcom/msm8994-msft-lumia-octagon.dtsi
+++ b/arch/arm64/boot/dts/qcom/msm8994-msft-lumia-octagon.dtsi
@@ -62,7 +62,6 @@ divclk4: divclk4 {
gpio-keys {
compatible = "gpio-keys";
- input-name = "gpio-keys";
autorepeat;
volupkey {
diff --git a/arch/arm64/boot/dts/qcom/msm8994-sony-xperia-kitakami.dtsi b/arch/arm64/boot/dts/qcom/msm8994-sony-xperia-kitakami.dtsi
index 48de66b..dde7ed1 100644
--- a/arch/arm64/boot/dts/qcom/msm8994-sony-xperia-kitakami.dtsi
+++ b/arch/arm64/boot/dts/qcom/msm8994-sony-xperia-kitakami.dtsi
@@ -29,7 +29,6 @@ / {
gpio_keys {
compatible = "gpio-keys";
- input-name = "gpio-keys";
#address-cells = <1>;
#size-cells = <0>;
autorepeat;
@@ -223,7 +222,7 @@ pm8994_regulators: pm8994-regulators {
vdd_l17_l29-supply = <&pmi8994_bby>;
vdd_l20_l21-supply = <&pmi8994_bby>;
vdd_l25-supply = <&pm8994_s3>;
- vdd_lvs1_lvs2-supply = <&pm8994_s4>;
+ vdd_lvs1_2-supply = <&pm8994_s4>;
/* S1, S2, S6 and S12 are managed by RPMPD */
diff --git a/arch/arm64/boot/dts/qcom/msm8996-sony-xperia-tone.dtsi b/arch/arm64/boot/dts/qcom/msm8996-sony-xperia-tone.dtsi
index ff7f39d..3bb50ce 100644
--- a/arch/arm64/boot/dts/qcom/msm8996-sony-xperia-tone.dtsi
+++ b/arch/arm64/boot/dts/qcom/msm8996-sony-xperia-tone.dtsi
@@ -184,6 +184,7 @@ &camera2_rst {
&hsusb_phy1 {
status = "okay";
+ vdd-supply = <&pm8994_l28>;
vdda-pll-supply = <&pm8994_l12>;
vdda-phy-dpdm-supply = <&pm8994_l24>;
};
@@ -589,9 +590,27 @@ pmi-gpio10-nc {
};
};
-&pmi8994_spmi_regulators {
+&pm8994_spmi_regulators {
qcom,saw-reg = <&saw3>;
+ pm8994_s9: s9 {
+ qcom,saw-slave;
+ };
+
+ pm8994_s10: s10 {
+ qcom,saw-slave;
+ };
+
+ pm8994_s11: s11 {
+ qcom,saw-leader;
+ regulator-name = "vdd_apcc";
+ regulator-always-on;
+ regulator-min-microvolt = <470000>;
+ regulator-max-microvolt = <1140000>;
+ };
+};
+
+&pmi8994_spmi_regulators {
vdd_gfx:
pmi8994_s2: s2 {
/* Pinned to a high value for now to avoid random crashes. */
@@ -600,21 +619,6 @@ pmi8994_s2: s2 {
regulator-name = "vdd_gfx";
regulator-always-on;
};
-
- pmi8994_s9: s9 {
- qcom,saw-slave;
- };
-
- pmi8994_s10: s10 {
- qcom,saw-slave;
- };
-
- pmi8994_s11: s11 {
- qcom,saw-leader;
- regulator-always-on;
- regulator-min-microvolt = <470000>;
- regulator-max-microvolt = <1140000>;
- };
};
&pmi8994_wled {
diff --git a/arch/arm64/boot/dts/qcom/msm8996-xiaomi-common.dtsi b/arch/arm64/boot/dts/qcom/msm8996-xiaomi-common.dtsi
index 01e573f..7a9fcbe 100644
--- a/arch/arm64/boot/dts/qcom/msm8996-xiaomi-common.dtsi
+++ b/arch/arm64/boot/dts/qcom/msm8996-xiaomi-common.dtsi
@@ -207,8 +207,8 @@ &blsp2_i2c3 {
status = "okay";
label = "TYPEC_I2C";
- typec: tusb320@47 {
- compatible = "ti,tusb320";
+ typec: tusb320l@47 {
+ compatible = "ti,tusb320l";
reg = <0x47>;
interrupt-parent = <&tlmm>;
interrupts = <63 IRQ_TYPE_EDGE_RISING>;
diff --git a/arch/arm64/boot/dts/qcom/msm8996-xiaomi-scorpio.dts b/arch/arm64/boot/dts/qcom/msm8996-xiaomi-scorpio.dts
index e5b8402..27a45dd 100644
--- a/arch/arm64/boot/dts/qcom/msm8996-xiaomi-scorpio.dts
+++ b/arch/arm64/boot/dts/qcom/msm8996-xiaomi-scorpio.dts
@@ -60,6 +60,20 @@ &adsp_pil {
};
&blsp2_i2c6 {
+ touchkey: touchkey@28 {
+ compatible = "cypress,sf3155";
+ reg = <0x28>;
+ interrupt-parent = <&tlmm>;
+ interrupts = <77 IRQ_TYPE_EDGE_FALLING>;
+ avdd-supply = <&vreg_l6a_1p8>;
+ vdd-supply = <&vdd_3v2_tp>;
+ linux,keycodes = <KEY_BACK KEY_MENU>;
+
+ pinctrl-names = "default", "sleep";
+ pinctrl-0 = <&touchkey_default>;
+ pinctrl-1 = <&touchkey_sleep>;
+ };
+
touchscreen: atmel-mxt-ts@4a {
compatible = "atmel,maxtouch";
reg = <0x4a>;
@@ -416,6 +430,20 @@ &tlmm {
"RFFE1_DATA", /* GPIO_148 */
"RFFE1_CLK"; /* GPIO_149 */
+ touchkey_default: touchkey_default {
+ pins = "gpio77";
+ function = "gpio";
+ drive-strength = <16>;
+ bias-pull-up;
+ };
+
+ touchkey_sleep: touchkey_sleep {
+ pins = "gpio77";
+ function = "gpio";
+ drive-strength = <2>;
+ bias-disable;
+ };
+
touchscreen_default: touchscreen_default {
pins = "gpio75", "gpio125";
function = "gpio";
diff --git a/arch/arm64/boot/dts/qcom/msm8996.dtsi b/arch/arm64/boot/dts/qcom/msm8996.dtsi
index bccc2d0..91bc974 100644
--- a/arch/arm64/boot/dts/qcom/msm8996.dtsi
+++ b/arch/arm64/boot/dts/qcom/msm8996.dtsi
@@ -962,7 +962,6 @@ hdmi_phy: hdmi-phy@9a0600 {
gpu: gpu@b00000 {
compatible = "qcom,adreno-530.2", "qcom,adreno";
- #stream-id-cells = <16>;
reg = <0x00b00000 0x3f000>;
reg-names = "kgsl_3d0_reg_memory";
@@ -987,9 +986,6 @@ gpu: gpu@b00000 {
nvmem-cells = <&speedbin_efuse>;
nvmem-cell-names = "speed_bin";
- qcom,gpu-quirk-two-pass-use-wfi;
- qcom,gpu-quirk-fault-detect-mask;
-
operating-points-v2 = <&gpu_opp_table>;
status = "disabled";
diff --git a/arch/arm64/boot/dts/qcom/msm8998-clamshell.dtsi b/arch/arm64/boot/dts/qcom/msm8998-clamshell.dtsi
index 3f60575..b3b3525 100644
--- a/arch/arm64/boot/dts/qcom/msm8998-clamshell.dtsi
+++ b/arch/arm64/boot/dts/qcom/msm8998-clamshell.dtsi
@@ -102,6 +102,7 @@ pm8005_s1: s1 { /* VDD_GFX supply */
&qusb2phy {
status = "okay";
+ vdd-supply = <&vreg_l1a_0p875>;
vdda-pll-supply = <&vreg_l12a_1p8>;
vdda-phy-dpdm-supply = <&vreg_l24a_3p075>;
};
diff --git a/arch/arm64/boot/dts/qcom/msm8998-fxtec-pro1.dts b/arch/arm64/boot/dts/qcom/msm8998-fxtec-pro1.dts
index 3d495ce..dc5b9b2 100644
--- a/arch/arm64/boot/dts/qcom/msm8998-fxtec-pro1.dts
+++ b/arch/arm64/boot/dts/qcom/msm8998-fxtec-pro1.dts
@@ -29,7 +29,6 @@ extcon_usb: extcon-usb {
gpio-hall-sensors {
compatible = "gpio-keys";
- input-name = "hall-sensors";
label = "Hall sensors";
pinctrl-names = "default";
pinctrl-0 = <&hall_sensor1_default>;
@@ -46,7 +45,6 @@ hall-sensor1 {
gpio-kb-extra-keys {
compatible = "gpio-keys";
- input-name = "extra-kb-keys";
label = "Keyboard extra keys";
pinctrl-names = "default";
pinctrl-0 = <&gpio_kb_pins_extra>;
@@ -102,7 +100,6 @@ alt {
gpio-keys {
compatible = "gpio-keys";
- input-name = "side-buttons";
label = "Side buttons";
#address-cells = <1>;
#size-cells = <0>;
diff --git a/arch/arm64/boot/dts/qcom/msm8998-oneplus-common.dtsi b/arch/arm64/boot/dts/qcom/msm8998-oneplus-common.dtsi
index 6541880..9823d48 100644
--- a/arch/arm64/boot/dts/qcom/msm8998-oneplus-common.dtsi
+++ b/arch/arm64/boot/dts/qcom/msm8998-oneplus-common.dtsi
@@ -260,6 +260,7 @@ pinconf {
&qusb2phy {
status = "okay";
+ vdd-supply = <&vreg_l1a_0p875>;
vdda-pll-supply = <&vreg_l12a_1p8>;
vdda-phy-dpdm-supply = <&vreg_l24a_3p075>;
};
diff --git a/arch/arm64/boot/dts/qcom/msm8998-sony-xperia-yoshino.dtsi b/arch/arm64/boot/dts/qcom/msm8998-sony-xperia-yoshino.dtsi
index 91e3912..47488a1 100644
--- a/arch/arm64/boot/dts/qcom/msm8998-sony-xperia-yoshino.dtsi
+++ b/arch/arm64/boot/dts/qcom/msm8998-sony-xperia-yoshino.dtsi
@@ -93,7 +93,6 @@ vph_pwr: vph-pwr-regulator {
gpio-keys {
compatible = "gpio-keys";
- input-name = "gpio-keys";
label = "Side buttons";
pinctrl-names = "default";
pinctrl-0 = <&vol_down_pin_a>, <&cam_focus_pin_a>,
@@ -126,7 +125,6 @@ camera-focus {
gpio-hall-sensor {
compatible = "gpio-keys";
- input-name = "hall-sensors";
label = "Hall sensors";
pinctrl-names = "default";
pinctrl-0 = <&hall_sensor0_default>;
diff --git a/arch/arm64/boot/dts/qcom/msm8998.dtsi b/arch/arm64/boot/dts/qcom/msm8998.dtsi
index 408f265..f273bc1 100644
--- a/arch/arm64/boot/dts/qcom/msm8998.dtsi
+++ b/arch/arm64/boot/dts/qcom/msm8998.dtsi
@@ -1446,7 +1446,6 @@ adreno_gpu: gpu@5000000 {
iommus = <&adreno_smmu 0>;
operating-points-v2 = <&gpu_opp_table>;
power-domains = <&rpmpd MSM8998_VDDMX>;
- #stream-id-cells = <16>;
status = "disabled";
gpu_opp_table: opp-table {
diff --git a/arch/arm64/boot/dts/qcom/pm660.dtsi b/arch/arm64/boot/dts/qcom/pm660.dtsi
index d0ef8a1..c482663 100644
--- a/arch/arm64/boot/dts/qcom/pm660.dtsi
+++ b/arch/arm64/boot/dts/qcom/pm660.dtsi
@@ -54,14 +54,24 @@ pon: pon@800 {
mode-bootloader = <0x2>;
mode-recovery = <0x1>;
- pwrkey {
+ pon_pwrkey: pwrkey {
compatible = "qcom,pm8941-pwrkey";
interrupts = <0x0 0x8 0 IRQ_TYPE_EDGE_BOTH>;
debounce = <15625>;
bias-pull-up;
linux,code = <KEY_POWER>;
+
+ status = "disabled";
};
+ pon_resin: resin {
+ compatible = "qcom,pm8941-resin";
+ interrupts = <0x0 0x8 1 IRQ_TYPE_EDGE_BOTH>;
+ debounce = <15625>;
+ bias-pull-up;
+
+ status = "disabled";
+ };
};
pm660_temp: temp-alarm@2400 {
diff --git a/arch/arm64/boot/dts/qcom/pm8998.dtsi b/arch/arm64/boot/dts/qcom/pm8998.dtsi
index 6f5bb6b..d09f295 100644
--- a/arch/arm64/boot/dts/qcom/pm8998.dtsi
+++ b/arch/arm64/boot/dts/qcom/pm8998.dtsi
@@ -84,6 +84,16 @@ adc-chan@6 {
};
};
+ pm8998_adc_tm: adc-tm@3400 {
+ compatible = "qcom,spmi-adc-tm-hc";
+ reg = <0x3400>;
+ interrupts = <0x0 0x34 0x0 IRQ_TYPE_EDGE_RISING>;
+ #thermal-sensor-cells = <1>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
rtc@6000 {
compatible = "qcom,pm8941-rtc";
reg = <0x6000>, <0x6100>;
diff --git a/arch/arm64/boot/dts/qcom/pmk8350.dtsi b/arch/arm64/boot/dts/qcom/pmk8350.dtsi
index 769f972..0f94c46 100644
--- a/arch/arm64/boot/dts/qcom/pmk8350.dtsi
+++ b/arch/arm64/boot/dts/qcom/pmk8350.dtsi
@@ -19,16 +19,17 @@ pmk8350_pon: pon@1300 {
compatible = "qcom,pm8998-pon";
reg = <0x1300>;
- pwrkey {
+ pon_pwrkey: pwrkey {
compatible = "qcom,pmk8350-pwrkey";
interrupts = <0x0 0x13 0x7 IRQ_TYPE_EDGE_BOTH>;
linux,code = <KEY_POWER>;
+ status = "disabled";
};
- resin {
+ pon_resin: resin {
compatible = "qcom,pmk8350-resin";
interrupts = <0x0 0x13 0x6 IRQ_TYPE_EDGE_BOTH>;
- linux,code = <KEY_VOLUMEDOWN>;
+ status = "disabled";
};
};
diff --git a/arch/arm64/boot/dts/qcom/sc7180-trogdor-homestar-r2.dts b/arch/arm64/boot/dts/qcom/sc7180-trogdor-homestar-r2.dts
index db6c2da..7003298 100644
--- a/arch/arm64/boot/dts/qcom/sc7180-trogdor-homestar-r2.dts
+++ b/arch/arm64/boot/dts/qcom/sc7180-trogdor-homestar-r2.dts
@@ -7,6 +7,8 @@
/dts-v1/;
+#include "sc7180.dtsi"
+#include "sc7180-trogdor-ti-sn65dsi86.dtsi"
#include "sc7180-trogdor-homestar.dtsi"
/ {
diff --git a/arch/arm64/boot/dts/qcom/sc7180-trogdor-homestar-r3.dts b/arch/arm64/boot/dts/qcom/sc7180-trogdor-homestar-r3.dts
index 3fd8aa5..e92e2e9 100644
--- a/arch/arm64/boot/dts/qcom/sc7180-trogdor-homestar-r3.dts
+++ b/arch/arm64/boot/dts/qcom/sc7180-trogdor-homestar-r3.dts
@@ -7,9 +7,11 @@
/dts-v1/;
+#include "sc7180.dtsi"
+#include "sc7180-trogdor-ti-sn65dsi86.dtsi"
#include "sc7180-trogdor-homestar.dtsi"
/ {
- model = "Google Homestar (rev3+)";
- compatible = "google,homestar", "qcom,sc7180";
+ model = "Google Homestar (rev3)";
+ compatible = "google,homestar-rev3", "qcom,sc7180";
};
diff --git a/arch/arm64/boot/dts/qcom/sc7180-trogdor-homestar-r4.dts b/arch/arm64/boot/dts/qcom/sc7180-trogdor-homestar-r4.dts
new file mode 100644
index 0000000..0de0c97
--- /dev/null
+++ b/arch/arm64/boot/dts/qcom/sc7180-trogdor-homestar-r4.dts
@@ -0,0 +1,21 @@
+// SPDX-License-Identifier: (GPL-2.0+ OR MIT)
+/*
+ * Google Homestar board device tree source
+ *
+ * Copyright 2021 Google LLC.
+ */
+
+/dts-v1/;
+
+#include "sc7180.dtsi"
+#include "sc7180-trogdor-parade-ps8640.dtsi"
+#include "sc7180-trogdor-homestar.dtsi"
+
+/ {
+ model = "Google Homestar (rev4+)";
+ compatible = "google,homestar", "qcom,sc7180";
+};
+
+&pp3300_brij_ps8640 {
+ regulator-enable-ramp-delay = <4000>;
+};
diff --git a/arch/arm64/boot/dts/qcom/sc7180-trogdor-homestar.dtsi b/arch/arm64/boot/dts/qcom/sc7180-trogdor-homestar.dtsi
index 4ab890b..f32369a 100644
--- a/arch/arm64/boot/dts/qcom/sc7180-trogdor-homestar.dtsi
+++ b/arch/arm64/boot/dts/qcom/sc7180-trogdor-homestar.dtsi
@@ -5,13 +5,10 @@
* Copyright 2021 Google LLC.
*/
-#include "sc7180.dtsi"
-
ap_ec_spi: &spi6 {};
ap_h1_spi: &spi0 {};
#include "sc7180-trogdor.dtsi"
-#include "sc7180-trogdor-ti-sn65dsi86.dtsi"
/ {
/* BOARD-SPECIFIC TOP LEVEL NODES */
diff --git a/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-limozeen-nots-r4.dts b/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-limozeen-nots-r4.dts
index 6ebde08..850776c 100644
--- a/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-limozeen-nots-r4.dts
+++ b/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-limozeen-nots-r4.dts
@@ -5,7 +5,7 @@
* Copyright 2020 Google LLC.
*/
-#include "sc7180-trogdor-lazor-limozeen-nots.dts"
+#include "sc7180-trogdor-lazor-limozeen-nots-r5.dts"
/ {
model = "Google Lazor Limozeen without Touchscreen (rev4)";
diff --git a/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-limozeen-nots-r5.dts b/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-limozeen-nots-r5.dts
new file mode 100644
index 0000000..f360ff2
--- /dev/null
+++ b/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-limozeen-nots-r5.dts
@@ -0,0 +1,31 @@
+// SPDX-License-Identifier: (GPL-2.0+ OR MIT)
+/*
+ * Google Lazor Limozeen board device tree source
+ *
+ * Copyright 2021 Google LLC.
+ */
+
+/dts-v1/;
+
+#include "sc7180.dtsi"
+#include "sc7180-trogdor-ti-sn65dsi86.dtsi"
+#include "sc7180-trogdor-lazor.dtsi"
+#include "sc7180-trogdor-lte-sku.dtsi"
+
+/ {
+ model = "Google Lazor Limozeen without Touchscreen (rev5 - rev8)";
+ /* No sku5 post-rev5 */
+ compatible = "google,lazor-rev5-sku5", "google,lazor-rev5-sku6",
+ "google,lazor-rev6-sku6", "google,lazor-rev7-sku6",
+ "google,lazor-rev8-sku6", "qcom,sc7180";
+};
+
+/delete-node/&ap_ts;
+
+&panel {
+ compatible = "innolux,n116bca-ea1", "innolux,n116bge";
+};
+
+&sdhc_2 {
+ status = "okay";
+};
diff --git a/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-limozeen-nots-r9.dts b/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-limozeen-nots-r9.dts
new file mode 100644
index 0000000..4e35aec
--- /dev/null
+++ b/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-limozeen-nots-r9.dts
@@ -0,0 +1,28 @@
+// SPDX-License-Identifier: (GPL-2.0+ OR MIT)
+/*
+ * Google Lazor Limozeen board device tree source
+ *
+ * Copyright 2021 Google LLC.
+ */
+
+/dts-v1/;
+
+#include "sc7180.dtsi"
+#include "sc7180-trogdor-parade-ps8640.dtsi"
+#include "sc7180-trogdor-lazor.dtsi"
+#include "sc7180-trogdor-lte-sku.dtsi"
+
+/ {
+ model = "Google Lazor Limozeen without Touchscreen (rev9+)";
+ compatible = "google,lazor-sku6", "qcom,sc7180";
+};
+
+/delete-node/&ap_ts;
+
+&panel {
+ compatible = "innolux,n116bca-ea1", "innolux,n116bge";
+};
+
+&sdhc_2 {
+ status = "okay";
+};
diff --git a/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-limozeen-nots.dts b/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-limozeen-nots.dts
deleted file mode 100644
index 0456c7e..0000000
--- a/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-limozeen-nots.dts
+++ /dev/null
@@ -1,26 +0,0 @@
-// SPDX-License-Identifier: (GPL-2.0+ OR MIT)
-/*
- * Google Lazor Limozeen board device tree source
- *
- * Copyright 2020 Google LLC.
- */
-
-/dts-v1/;
-
-#include "sc7180-trogdor-lazor.dtsi"
-#include "sc7180-trogdor-lte-sku.dtsi"
-
-/ {
- model = "Google Lazor Limozeen without Touchscreen";
- compatible = "google,lazor-sku6", "google,lazor-sku5", "qcom,sc7180";
-};
-
-/delete-node/&ap_ts;
-
-&panel {
- compatible = "innolux,n116bca-ea1", "innolux,n116bge";
-};
-
-&sdhc_2 {
- status = "okay";
-};
diff --git a/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-limozeen-r4.dts b/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-limozeen-r4.dts
new file mode 100644
index 0000000..42b4bbc
--- /dev/null
+++ b/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-limozeen-r4.dts
@@ -0,0 +1,46 @@
+// SPDX-License-Identifier: (GPL-2.0+ OR MIT)
+/*
+ * Google Lazor Limozeen board device tree source
+ *
+ * Copyright 2021 Google LLC.
+ */
+
+/dts-v1/;
+
+#include "sc7180.dtsi"
+#include "sc7180-trogdor-ti-sn65dsi86.dtsi"
+#include "sc7180-trogdor-lazor.dtsi"
+#include "sc7180-trogdor-lte-sku.dtsi"
+
+/ {
+ model = "Google Lazor Limozeen (rev4 - rev8)";
+ compatible = "google,lazor-rev4-sku4", "google,lazor-rev5-sku4",
+ "google,lazor-rev6-sku4", "google,lazor-rev7-sku4",
+ "google,lazor-rev8-sku4", "qcom,sc7180";
+};
+
+/delete-node/&ap_ts;
+
+&ap_ts_pen_1v8 {
+ ap_ts: touchscreen@10 {
+ compatible = "elan,ekth3500";
+ reg = <0x10>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&ts_int_l>, <&ts_reset_l>;
+
+ interrupt-parent = <&tlmm>;
+ interrupts = <9 IRQ_TYPE_LEVEL_LOW>;
+
+ vcc33-supply = <&pp3300_ts>;
+
+ reset-gpios = <&tlmm 8 GPIO_ACTIVE_LOW>;
+ };
+};
+
+&panel {
+ compatible = "auo,b116xa01";
+};
+
+&sdhc_2 {
+ status = "okay";
+};
diff --git a/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-limozeen-r9.dts b/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-limozeen-r9.dts
new file mode 100644
index 0000000..dc47842
--- /dev/null
+++ b/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-limozeen-r9.dts
@@ -0,0 +1,44 @@
+// SPDX-License-Identifier: (GPL-2.0+ OR MIT)
+/*
+ * Google Lazor Limozeen board device tree source
+ *
+ * Copyright 2021 Google LLC.
+ */
+
+/dts-v1/;
+
+#include "sc7180.dtsi"
+#include "sc7180-trogdor-parade-ps8640.dtsi"
+#include "sc7180-trogdor-lazor.dtsi"
+#include "sc7180-trogdor-lte-sku.dtsi"
+
+/ {
+ model = "Google Lazor Limozeen (rev9+)";
+ compatible = "google,lazor-sku4", "qcom,sc7180";
+};
+
+/delete-node/&ap_ts;
+
+&ap_ts_pen_1v8 {
+ ap_ts: touchscreen@10 {
+ compatible = "elan,ekth3500";
+ reg = <0x10>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&ts_int_l>, <&ts_reset_l>;
+
+ interrupt-parent = <&tlmm>;
+ interrupts = <9 IRQ_TYPE_LEVEL_LOW>;
+
+ vcc33-supply = <&pp3300_ts>;
+
+ reset-gpios = <&tlmm 8 GPIO_ACTIVE_LOW>;
+ };
+};
+
+&panel {
+ compatible = "auo,b116xa01";
+};
+
+&sdhc_2 {
+ status = "okay";
+};
diff --git a/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-limozeen.dts b/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-limozeen.dts
deleted file mode 100644
index e6ad6da..0000000
--- a/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-limozeen.dts
+++ /dev/null
@@ -1,42 +0,0 @@
-// SPDX-License-Identifier: (GPL-2.0+ OR MIT)
-/*
- * Google Lazor Limozeen board device tree source
- *
- * Copyright 2020 Google LLC.
- */
-
-/dts-v1/;
-
-#include "sc7180-trogdor-lazor.dtsi"
-#include "sc7180-trogdor-lte-sku.dtsi"
-
-/ {
- model = "Google Lazor Limozeen";
- compatible = "google,lazor-sku4", "qcom,sc7180";
-};
-
-/delete-node/&ap_ts;
-
-&ap_ts_pen_1v8 {
- ap_ts: touchscreen@10 {
- compatible = "elan,ekth3500";
- reg = <0x10>;
- pinctrl-names = "default";
- pinctrl-0 = <&ts_int_l>, <&ts_reset_l>;
-
- interrupt-parent = <&tlmm>;
- interrupts = <9 IRQ_TYPE_LEVEL_LOW>;
-
- vcc33-supply = <&pp3300_ts>;
-
- reset-gpios = <&tlmm 8 GPIO_ACTIVE_LOW>;
- };
-};
-
-&panel {
- compatible = "auo,b116xa01";
-};
-
-&sdhc_2 {
- status = "okay";
-};
diff --git a/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r0.dts b/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r0.dts
index 30e3e769..b142006 100644
--- a/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r0.dts
+++ b/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r0.dts
@@ -7,6 +7,8 @@
/dts-v1/;
+#include "sc7180.dtsi"
+#include "sc7180-trogdor-ti-sn65dsi86.dtsi"
#include "sc7180-trogdor-lazor.dtsi"
/ {
diff --git a/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r1.dts b/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r1.dts
index c2ef063..5974079 100644
--- a/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r1.dts
+++ b/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r1.dts
@@ -7,6 +7,8 @@
/dts-v1/;
+#include "sc7180.dtsi"
+#include "sc7180-trogdor-ti-sn65dsi86.dtsi"
#include "sc7180-trogdor-lazor.dtsi"
/ {
diff --git a/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r3-kb.dts b/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r3-kb.dts
index dcb41af..18ef9da 100644
--- a/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r3-kb.dts
+++ b/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r3-kb.dts
@@ -7,12 +7,17 @@
/dts-v1/;
+#include "sc7180.dtsi"
+#include "sc7180-trogdor-ti-sn65dsi86.dtsi"
#include "sc7180-trogdor-lazor.dtsi"
#include "sc7180-lite.dtsi"
/ {
- model = "Google Lazor (rev3+) with KB Backlight";
- compatible = "google,lazor-sku2", "qcom,sc7180";
+ model = "Google Lazor (rev3 - 8) with KB Backlight";
+ compatible = "google,lazor-rev3-sku2", "google,lazor-rev4-sku2",
+ "google,lazor-rev5-sku2", "google,lazor-rev6-sku2",
+ "google,lazor-rev7-sku2", "google,lazor-rev8-sku2",
+ "qcom,sc7180";
};
&keyboard_backlight {
diff --git a/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r3-lte.dts b/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r3-lte.dts
index be44900..c5c9fef 100644
--- a/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r3-lte.dts
+++ b/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r3-lte.dts
@@ -7,12 +7,17 @@
/dts-v1/;
+#include "sc7180.dtsi"
+#include "sc7180-trogdor-ti-sn65dsi86.dtsi"
#include "sc7180-trogdor-lazor.dtsi"
#include "sc7180-trogdor-lte-sku.dtsi"
/ {
- model = "Google Lazor (rev3+) with LTE";
- compatible = "google,lazor-sku0", "qcom,sc7180";
+ model = "Google Lazor (rev3 - 8) with LTE";
+ compatible = "google,lazor-rev3-sku0", "google,lazor-rev4-sku0",
+ "google,lazor-rev5-sku0", "google,lazor-rev6-sku0",
+ "google,lazor-rev7-sku0", "google,lazor-rev8-sku0",
+ "qcom,sc7180";
};
&ap_sar_sensor {
diff --git a/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r3.dts b/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r3.dts
index b474df4..7adcedb 100644
--- a/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r3.dts
+++ b/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r3.dts
@@ -7,10 +7,14 @@
/dts-v1/;
+#include "sc7180.dtsi"
+#include "sc7180-trogdor-ti-sn65dsi86.dtsi"
#include "sc7180-trogdor-lazor.dtsi"
#include "sc7180-lite.dtsi"
/ {
- model = "Google Lazor (rev3+)";
- compatible = "google,lazor", "qcom,sc7180";
+ model = "Google Lazor (rev3 - 8)";
+ compatible = "google,lazor-rev3", "google,lazor-rev4",
+ "google,lazor-rev5", "google,lazor-rev6", "google,lazor-rev7",
+ "google,lazor-rev8", "qcom,sc7180";
};
diff --git a/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r9-kb.dts b/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r9-kb.dts
new file mode 100644
index 0000000..7f5c015
--- /dev/null
+++ b/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r9-kb.dts
@@ -0,0 +1,22 @@
+// SPDX-License-Identifier: (GPL-2.0+ OR MIT)
+/*
+ * Google Lazor board device tree source
+ *
+ * Copyright 2020 Google LLC.
+ */
+
+/dts-v1/;
+
+#include "sc7180.dtsi"
+#include "sc7180-trogdor-parade-ps8640.dtsi"
+#include "sc7180-trogdor-lazor.dtsi"
+#include "sc7180-lite.dtsi"
+
+/ {
+ model = "Google Lazor (rev9+) with KB Backlight";
+ compatible = "google,lazor-sku2", "qcom,sc7180";
+};
+
+&keyboard_backlight {
+ status = "okay";
+};
diff --git a/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r9-lte.dts b/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r9-lte.dts
new file mode 100644
index 0000000..344b57c
--- /dev/null
+++ b/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r9-lte.dts
@@ -0,0 +1,30 @@
+// SPDX-License-Identifier: (GPL-2.0+ OR MIT)
+/*
+ * Google Lazor board device tree source
+ *
+ * Copyright 2021 Google LLC.
+ */
+
+/dts-v1/;
+
+#include "sc7180.dtsi"
+#include "sc7180-trogdor-parade-ps8640.dtsi"
+#include "sc7180-trogdor-lazor.dtsi"
+#include "sc7180-trogdor-lte-sku.dtsi"
+
+/ {
+ model = "Google Lazor (rev9+) with LTE";
+ compatible = "google,lazor-sku0", "qcom,sc7180";
+};
+
+&ap_sar_sensor {
+ status = "okay";
+};
+
+&ap_sar_sensor_i2c {
+ status = "okay";
+};
+
+&keyboard_backlight {
+ status = "okay";
+};
diff --git a/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r9.dts b/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r9.dts
new file mode 100644
index 0000000..83f6a4e
--- /dev/null
+++ b/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r9.dts
@@ -0,0 +1,18 @@
+// SPDX-License-Identifier: (GPL-2.0+ OR MIT)
+/*
+ * Google Lazor board device tree source
+ *
+ * Copyright 2021 Google LLC.
+ */
+
+/dts-v1/;
+
+#include "sc7180.dtsi"
+#include "sc7180-trogdor-parade-ps8640.dtsi"
+#include "sc7180-trogdor-lazor.dtsi"
+#include "sc7180-lite.dtsi"
+
+/ {
+ model = "Google Lazor (rev9+)";
+ compatible = "google,lazor", "qcom,sc7180";
+};
diff --git a/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor.dtsi b/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor.dtsi
index 8b79fbb..69666f9 100644
--- a/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor.dtsi
+++ b/arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor.dtsi
@@ -5,13 +5,10 @@
* Copyright 2020 Google LLC.
*/
-#include "sc7180.dtsi"
-
ap_ec_spi: &spi6 {};
ap_h1_spi: &spi0 {};
#include "sc7180-trogdor.dtsi"
-#include "sc7180-trogdor-ti-sn65dsi86.dtsi"
&ap_sar_sensor {
semtech,cs0-ground;
diff --git a/arch/arm64/boot/dts/qcom/sc7180-trogdor-parade-ps8640.dtsi b/arch/arm64/boot/dts/qcom/sc7180-trogdor-parade-ps8640.dtsi
index a3d6954..6a84fba 100644
--- a/arch/arm64/boot/dts/qcom/sc7180-trogdor-parade-ps8640.dtsi
+++ b/arch/arm64/boot/dts/qcom/sc7180-trogdor-parade-ps8640.dtsi
@@ -5,6 +5,8 @@
* Copyright 2021 Google LLC.
*/
+#include <dt-bindings/gpio/gpio.h>
+
/ {
pp3300_brij_ps8640: pp3300-brij-ps8640 {
compatible = "regulator-fixed";
diff --git a/arch/arm64/boot/dts/qcom/sc7180-trogdor-ti-sn65dsi86.dtsi b/arch/arm64/boot/dts/qcom/sc7180-trogdor-ti-sn65dsi86.dtsi
index 97d5e45..f869e6a 100644
--- a/arch/arm64/boot/dts/qcom/sc7180-trogdor-ti-sn65dsi86.dtsi
+++ b/arch/arm64/boot/dts/qcom/sc7180-trogdor-ti-sn65dsi86.dtsi
@@ -5,9 +5,10 @@
* Copyright 2021 Google LLC.
*/
+#include <dt-bindings/gpio/gpio.h>
+
&dsi0_out {
remote-endpoint = <&sn65dsi86_in>;
- data-lanes = <0 1 2 3>;
};
edp_brij_i2c: &i2c2 {
diff --git a/arch/arm64/boot/dts/qcom/sc7180-trogdor.dtsi b/arch/arm64/boot/dts/qcom/sc7180-trogdor.dtsi
index d4f4441..bd5909ff 100644
--- a/arch/arm64/boot/dts/qcom/sc7180-trogdor.dtsi
+++ b/arch/arm64/boot/dts/qcom/sc7180-trogdor.dtsi
@@ -603,6 +603,10 @@ &dsi0 {
vdda-supply = <&vdda_mipi_dsi0_1p2>;
};
+&dsi0_out {
+ data-lanes = <0 1 2 3>;
+};
+
&dsi_phy {
status = "okay";
vdds-supply = <&vdda_mipi_dsi0_pll>;
diff --git a/arch/arm64/boot/dts/qcom/sc7180.dtsi b/arch/arm64/boot/dts/qcom/sc7180.dtsi
index faf8b807..2151cd8 100644
--- a/arch/arm64/boot/dts/qcom/sc7180.dtsi
+++ b/arch/arm64/boot/dts/qcom/sc7180.dtsi
@@ -1952,7 +1952,6 @@ glink-edge {
gpu: gpu@5000000 {
compatible = "qcom,adreno-618.0", "qcom,adreno";
- #stream-id-cells = <16>;
reg = <0 0x05000000 0 0x40000>, <0 0x0509e000 0 0x1000>,
<0 0x05061000 0 0x800>;
reg-names = "kgsl_3d0_reg_memory", "cx_mem", "cx_dbgc";
diff --git a/arch/arm64/boot/dts/qcom/sc7280-crd.dts b/arch/arm64/boot/dts/qcom/sc7280-crd.dts
new file mode 100644
index 0000000..cd2755c
--- /dev/null
+++ b/arch/arm64/boot/dts/qcom/sc7280-crd.dts
@@ -0,0 +1,93 @@
+// SPDX-License-Identifier: BSD-3-Clause
+/*
+ * sc7280 CRD board device tree source
+ *
+ * Copyright (c) 2021 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+/dts-v1/;
+
+#include "sc7280-idp.dtsi"
+#include "sc7280-idp-ec-h1.dtsi"
+
+/ {
+ model = "Qualcomm Technologies, Inc. sc7280 CRD platform";
+ compatible = "qcom,sc7280-crd", "google,hoglin", "qcom,sc7280";
+
+ aliases {
+ serial0 = &uart5;
+ };
+
+ chosen {
+ stdout-path = "serial0:115200n8";
+ };
+};
+
+ap_tp_i2c: &i2c0 {
+ status = "okay";
+ clock-frequency = <400000>;
+
+ trackpad: trackpad@15 {
+ compatible = "hid-over-i2c";
+ reg = <0x15>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&tp_int_odl>;
+
+ interrupt-parent = <&tlmm>;
+ interrupts = <7 IRQ_TYPE_EDGE_FALLING>;
+
+ post-power-on-delay-ms = <20>;
+ hid-descr-addr = <0x0001>;
+ vdd-supply = <&vreg_l18b_1p8>;
+
+ wakeup-source;
+ };
+};
+
+ap_ts_pen_1v8: &i2c13 {
+ status = "okay";
+ clock-frequency = <400000>;
+
+ ap_ts: touchscreen@5c {
+ compatible = "hid-over-i2c";
+ reg = <0x5c>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&ts_int_l>, <&ts_reset_l>;
+
+ interrupt-parent = <&tlmm>;
+ interrupts = <55 IRQ_TYPE_LEVEL_LOW>;
+
+ post-power-on-delay-ms = <500>;
+ hid-descr-addr = <0x0000>;
+
+ vdd-supply = <&vreg_l19b_1p8>;
+ };
+};
+
+&nvme_3v3_regulator {
+ gpio = <&tlmm 51 GPIO_ACTIVE_HIGH>;
+};
+
+&nvme_pwren {
+ pins = "gpio51";
+};
+
+&tlmm {
+ tp_int_odl: tp-int-odl {
+ pins = "gpio7";
+ function = "gpio";
+ bias-disable;
+ };
+
+ ts_int_l: ts-int-l {
+ pins = "gpio55";
+ function = "gpio";
+ bias-pull-up;
+ };
+
+ ts_reset_l: ts-reset-l {
+ pins = "gpio54";
+ function = "gpio";
+ bias-disable;
+ };
+};
diff --git a/arch/arm64/boot/dts/qcom/sc7280-idp-ec-h1.dtsi b/arch/arm64/boot/dts/qcom/sc7280-idp-ec-h1.dtsi
new file mode 100644
index 0000000..0896a61
--- /dev/null
+++ b/arch/arm64/boot/dts/qcom/sc7280-idp-ec-h1.dtsi
@@ -0,0 +1,105 @@
+// SPDX-License-Identifier: BSD-3-Clause
+/*
+ * sc7280 EC/H1 over SPI (common between IDP2 and CRD)
+ *
+ * Copyright (c) 2021 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+ap_ec_spi: &spi10 {
+ status = "okay";
+
+ pinctrl-0 = <&qup_spi10_data_clk>, <&qup_spi10_cs_gpio_init_high>, <&qup_spi10_cs_gpio>;
+ cs-gpios = <&tlmm 43 GPIO_ACTIVE_LOW>;
+
+ cros_ec: ec@0 {
+ compatible = "google,cros-ec-spi";
+ reg = <0>;
+ interrupt-parent = <&tlmm>;
+ interrupts = <18 IRQ_TYPE_LEVEL_LOW>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&ap_ec_int_l>;
+ spi-max-frequency = <3000000>;
+
+ cros_ec_pwm: ec-pwm {
+ compatible = "google,cros-ec-pwm";
+ #pwm-cells = <1>;
+ };
+
+ i2c_tunnel: i2c-tunnel {
+ compatible = "google,cros-ec-i2c-tunnel";
+ google,remote-bus = <0>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ };
+
+ typec {
+ compatible = "google,cros-ec-typec";
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ usb_c0: connector@0 {
+ compatible = "usb-c-connector";
+ reg = <0>;
+ label = "left";
+ power-role = "dual";
+ data-role = "host";
+ try-power-role = "source";
+ };
+
+ usb_c1: connector@1 {
+ compatible = "usb-c-connector";
+ reg = <1>;
+ label = "right";
+ power-role = "dual";
+ data-role = "host";
+ try-power-role = "source";
+ };
+ };
+ };
+};
+
+#include <arm/cros-ec-keyboard.dtsi>
+#include <arm/cros-ec-sbs.dtsi>
+
+ap_h1_spi: &spi14 {
+ status = "okay";
+
+ pinctrl-0 = <&qup_spi14_data_clk>, <&qup_spi14_cs_gpio_init_high>, <&qup_spi14_cs_gpio>;
+ cs-gpios = <&tlmm 59 GPIO_ACTIVE_LOW>;
+
+ cr50: tpm@0 {
+ compatible = "google,cr50";
+ reg = <0>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&h1_ap_int_odl>;
+ spi-max-frequency = <800000>;
+ interrupt-parent = <&tlmm>;
+ interrupts = <104 IRQ_TYPE_EDGE_RISING>;
+ };
+};
+
+&tlmm {
+ ap_ec_int_l: ap-ec-int-l {
+ pins = "gpio18";
+ function = "gpio";
+ input-enable;
+ bias-pull-up;
+ };
+
+ h1_ap_int_odl: h1-ap-int-odl {
+ pins = "gpio104";
+ function = "gpio";
+ input-enable;
+ bias-pull-up;
+ };
+
+ qup_spi10_cs_gpio_init_high: qup-spi10-cs-gpio-init-high {
+ pins = "gpio43";
+ output-high;
+ };
+
+ qup_spi14_cs_gpio_init_high: qup-spi14-cs-gpio-init-high {
+ pins = "gpio59";
+ output-high;
+ };
+};
diff --git a/arch/arm64/boot/dts/qcom/sc7280-idp2.dts b/arch/arm64/boot/dts/qcom/sc7280-idp2.dts
index 3ae9969..0382c77 100644
--- a/arch/arm64/boot/dts/qcom/sc7280-idp2.dts
+++ b/arch/arm64/boot/dts/qcom/sc7280-idp2.dts
@@ -8,6 +8,7 @@
/dts-v1/;
#include "sc7280-idp.dtsi"
+#include "sc7280-idp-ec-h1.dtsi"
/ {
model = "Qualcomm Technologies, Inc. sc7280 IDP SKU2 platform";
diff --git a/arch/arm64/boot/dts/qcom/sc7280.dtsi b/arch/arm64/boot/dts/qcom/sc7280.dtsi
index 365a2e0..937c2e0 100644
--- a/arch/arm64/boot/dts/qcom/sc7280.dtsi
+++ b/arch/arm64/boot/dts/qcom/sc7280.dtsi
@@ -129,6 +129,11 @@ wlan_fw_mem: memory@80c00000 {
no-map;
};
+ video_mem: memory@8b200000 {
+ reg = <0x0 0x8b200000 0x0 0x500000>;
+ no-map;
+ };
+
ipa_fw_mem: memory@8b700000 {
reg = <0 0x8b700000 0 0x10000>;
no-map;
@@ -574,9 +579,10 @@ gcc: clock-controller@100000 {
reg = <0 0x00100000 0 0x1f0000>;
clocks = <&rpmhcc RPMH_CXO_CLK>,
<&rpmhcc RPMH_CXO_CLK_A>, <&sleep_clk>,
- <0>, <0>, <0>, <0>, <0>, <0>;
+ <0>, <&pcie1_lane 0>,
+ <0>, <0>, <0>, <0>;
clock-names = "bi_tcxo", "bi_tcxo_ao", "sleep_clk",
- "pcie_0_pipe_clk", "pcie_1_pipe-clk",
+ "pcie_0_pipe_clk", "pcie_1_pipe_clk",
"ufs_phy_rx_symbol_0_clk", "ufs_phy_rx_symbol_1_clk",
"ufs_phy_tx_symbol_0_clk",
"usb3_phy_wrapper_gcc_usb30_pipe_clk";
@@ -1592,10 +1598,10 @@ pcie1: pci@1c08000 {
interrupt-names = "msi";
#interrupt-cells = <1>;
interrupt-map-mask = <0 0 0 0x7>;
- interrupt-map = <0 0 0 1 &intc 0 434 IRQ_TYPE_LEVEL_HIGH>,
- <0 0 0 2 &intc 0 435 IRQ_TYPE_LEVEL_HIGH>,
- <0 0 0 3 &intc 0 438 IRQ_TYPE_LEVEL_HIGH>,
- <0 0 0 4 &intc 0 439 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-map = <0 0 0 1 &intc 0 0 0 434 IRQ_TYPE_LEVEL_HIGH>,
+ <0 0 0 2 &intc 0 0 0 435 IRQ_TYPE_LEVEL_HIGH>,
+ <0 0 0 3 &intc 0 0 0 438 IRQ_TYPE_LEVEL_HIGH>,
+ <0 0 0 4 &intc 0 0 0 439 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&gcc GCC_PCIE_1_PIPE_CLK>,
<&gcc GCC_PCIE_1_PIPE_CLK_SRC>,
@@ -1747,7 +1753,6 @@ lpass_ag_noc: interconnect@3c40000 {
gpu: gpu@3d00000 {
compatible = "qcom,adreno-635.0", "qcom,adreno";
- #stream-id-cells = <16>;
reg = <0 0x03d00000 0 0x40000>,
<0 0x03d9e000 0 0x1000>,
<0 0x03d61000 0 0x800>;
@@ -2675,6 +2680,76 @@ usb_1_dwc3: usb@a600000 {
};
};
+ venus: video-codec@aa00000 {
+ compatible = "qcom,sc7280-venus";
+ reg = <0 0x0aa00000 0 0xd0600>;
+ interrupts = <GIC_SPI 174 IRQ_TYPE_LEVEL_HIGH>;
+
+ clocks = <&videocc VIDEO_CC_MVSC_CORE_CLK>,
+ <&videocc VIDEO_CC_MVSC_CTL_AXI_CLK>,
+ <&videocc VIDEO_CC_VENUS_AHB_CLK>,
+ <&videocc VIDEO_CC_MVS0_CORE_CLK>,
+ <&videocc VIDEO_CC_MVS0_AXI_CLK>;
+ clock-names = "core", "bus", "iface",
+ "vcodec_core", "vcodec_bus";
+
+ power-domains = <&videocc MVSC_GDSC>,
+ <&videocc MVS0_GDSC>,
+ <&rpmhpd SC7280_CX>;
+ power-domain-names = "venus", "vcodec0", "cx";
+ operating-points-v2 = <&venus_opp_table>;
+
+ interconnects = <&gem_noc MASTER_APPSS_PROC 0 &cnoc2 SLAVE_VENUS_CFG 0>,
+ <&mmss_noc MASTER_VIDEO_P0 0 &mc_virt SLAVE_EBI1 0>;
+ interconnect-names = "cpu-cfg", "video-mem";
+
+ iommus = <&apps_smmu 0x2180 0x20>,
+ <&apps_smmu 0x2184 0x20>;
+ memory-region = <&video_mem>;
+
+ video-decoder {
+ compatible = "venus-decoder";
+ };
+
+ video-encoder {
+ compatible = "venus-encoder";
+ };
+
+ video-firmware {
+ iommus = <&apps_smmu 0x21a2 0x0>;
+ };
+
+ venus_opp_table: venus-opp-table {
+ compatible = "operating-points-v2";
+
+ opp-133330000 {
+ opp-hz = /bits/ 64 <133330000>;
+ required-opps = <&rpmhpd_opp_low_svs>;
+ };
+
+ opp-240000000 {
+ opp-hz = /bits/ 64 <240000000>;
+ required-opps = <&rpmhpd_opp_svs>;
+ };
+
+ opp-335000000 {
+ opp-hz = /bits/ 64 <335000000>;
+ required-opps = <&rpmhpd_opp_svs_l1>;
+ };
+
+ opp-424000000 {
+ opp-hz = /bits/ 64 <424000000>;
+ required-opps = <&rpmhpd_opp_nom>;
+ };
+
+ opp-460000048 {
+ opp-hz = /bits/ 64 <460000048>;
+ required-opps = <&rpmhpd_opp_turbo>;
+ };
+ };
+
+ };
+
videocc: clock-controller@aaf0000 {
compatible = "qcom,sc7280-videocc";
reg = <0 0xaaf0000 0 0x10000>;
diff --git a/arch/arm64/boot/dts/qcom/sdm630-sony-xperia-nile.dtsi b/arch/arm64/boot/dts/qcom/sdm630-sony-xperia-nile.dtsi
index 11d0a8c..42af1fa 100644
--- a/arch/arm64/boot/dts/qcom/sdm630-sony-xperia-nile.dtsi
+++ b/arch/arm64/boot/dts/qcom/sdm630-sony-xperia-nile.dtsi
@@ -90,7 +90,6 @@ cam_vana_rear_vreg: cam_vana_rear_vreg {
gpio_keys {
status = "okay";
compatible = "gpio-keys";
- input-name = "gpio-keys";
#address-cells = <1>;
#size-cells = <0>;
@@ -215,14 +214,14 @@ &blsp2_uart1 {
/* HCI Bluetooth */
};
-&pon {
- volup {
- compatible = "qcom,pm8941-resin";
- interrupts = <0x0 0x8 1 IRQ_TYPE_EDGE_BOTH>;
- debounce = <15625>;
- bias-pull-up;
- linux,code = <KEY_VOLUMEUP>;
- };
+&pon_pwrkey {
+ status = "okay";
+};
+
+&pon_resin {
+ status = "okay";
+
+ linux,code = <KEY_VOLUMEUP>;
};
&qusb2phy {
diff --git a/arch/arm64/boot/dts/qcom/sdm630.dtsi b/arch/arm64/boot/dts/qcom/sdm630.dtsi
index 3e0165b..9217c3a 100644
--- a/arch/arm64/boot/dts/qcom/sdm630.dtsi
+++ b/arch/arm64/boot/dts/qcom/sdm630.dtsi
@@ -19,6 +19,11 @@ / {
#address-cells = <2>;
#size-cells = <2>;
+ aliases {
+ mmc1 = &sdhc_1;
+ mmc2 = &sdhc_2;
+ };
+
chosen { };
clocks {
@@ -1014,7 +1019,6 @@ sd-cd {
adreno_gpu: gpu@5000000 {
compatible = "qcom,adreno-508.0", "qcom,adreno";
- #stream-id-cells = <16>;
reg = <0x05000000 0x40000>;
reg-names = "kgsl_3d0_reg_memory";
diff --git a/arch/arm64/boot/dts/qcom/sdm636-sony-xperia-ganges-mermaid.dts b/arch/arm64/boot/dts/qcom/sdm636-sony-xperia-ganges-mermaid.dts
index bba1c2b..b96da53 100644
--- a/arch/arm64/boot/dts/qcom/sdm636-sony-xperia-ganges-mermaid.dts
+++ b/arch/arm64/boot/dts/qcom/sdm636-sony-xperia-ganges-mermaid.dts
@@ -11,6 +11,7 @@
/ {
model = "Sony Xperia 10 Plus";
compatible = "sony,mermaid-row", "qcom,sdm636";
+ chassis-type = "handset";
/* SDM636 v1 */
qcom,msm-id = <345 0>;
diff --git a/arch/arm64/boot/dts/qcom/sdm660-xiaomi-lavender.dts b/arch/arm64/boot/dts/qcom/sdm660-xiaomi-lavender.dts
index 1edc53f..dcbaacf 100644
--- a/arch/arm64/boot/dts/qcom/sdm660-xiaomi-lavender.dts
+++ b/arch/arm64/boot/dts/qcom/sdm660-xiaomi-lavender.dts
@@ -1,11 +1,16 @@
// SPDX-License-Identifier: GPL-2.0-only
/*
* Copyright (c) 2020, Alexey Minnekhanov <alexey.min@gmail.com>
+ * Copyright (c) 2021, Dang Huynh <danct12@riseup.net>
*/
/dts-v1/;
#include "sdm660.dtsi"
+#include "pm660.dtsi"
+#include "pm660l.dtsi"
+#include <dt-bindings/input/input.h>
+#include <dt-bindings/input/gpio-keys.h>
/ {
model = "Xiaomi Redmi Note 7";
@@ -17,7 +22,41 @@ aliases {
};
chosen {
+ #address-cells = <2>;
+ #size-cells = <2>;
+ ranges;
+
stdout-path = "serial0:115200n8";
+
+ framebuffer0: framebuffer@9d400000 {
+ compatible = "simple-framebuffer";
+ reg = <0 0x9d400000 0 (1080 * 2340 * 4)>;
+ width = <1080>;
+ height = <2340>;
+ stride = <(1080 * 4)>;
+ format = "a8r8g8b8";
+ };
+ };
+
+ vph_pwr: vph-pwr-regulator {
+ compatible = "regulator-fixed";
+ regulator-name = "vph_pwr";
+ regulator-min-microvolt = <3700000>;
+ regulator-max-microvolt = <3700000>;
+
+ regulator-always-on;
+ regulator-boot-on;
+ };
+
+ gpio-keys {
+ compatible = "gpio-keys";
+
+ volup {
+ label = "Volume Up";
+ gpios = <&pm660l_gpios 7 GPIO_ACTIVE_LOW>;
+ linux,code = <KEY_VOLUMEUP>;
+ debounce-interval = <15>;
+ };
};
reserved-memory {
@@ -33,6 +72,20 @@ ramoops@a0000000 {
ftrace-size = <0x0>;
pmsg-size = <0x20000>;
};
+
+ framebuffer_mem: memory@9d400000 {
+ reg = <0x0 0x9d400000 0x0 0x23ff000>;
+ no-map;
+ };
+ };
+
+ /*
+ * Until we hook up type-c detection, we
+ * have to stick with this. But it works.
+ */
+ extcon_usb: extcon-usb {
+ compatible = "linux,extcon-usb-gpio";
+ id-gpio = <&tlmm 58 GPIO_ACTIVE_HIGH>;
};
};
@@ -40,6 +93,304 @@ &blsp1_uart2 {
status = "okay";
};
+&pon_pwrkey {
+ status = "okay";
+};
+
+&pon_resin {
+ status = "okay";
+
+ linux,code = <KEY_VOLUMEDOWN>;
+};
+
+&qusb2phy {
+ status = "okay";
+
+ vdd-supply = <&vreg_l1b_0p925>;
+ vdda-phy-dpdm-supply = <&vreg_l7b_3p125>;
+};
+
+&rpm_requests {
+ pm660l-regulators {
+ compatible = "qcom,rpm-pm660l-regulators";
+
+ vdd_s1-supply = <&vph_pwr>;
+ vdd_s2-supply = <&vph_pwr>;
+ vdd_s3_s4-supply = <&vph_pwr>;
+ vdd_s5-supply = <&vph_pwr>;
+ vdd_s6-supply = <&vph_pwr>;
+
+ vdd_l1_l9_l10-supply = <&vreg_s2b_1p05>;
+ vdd_l2-supply = <&vreg_bob>;
+ vdd_l3_l5_l7_l8-supply = <&vreg_bob>;
+ vdd_l4_l6-supply = <&vreg_bob>;
+ vdd_bob-supply = <&vph_pwr>;
+
+ vreg_s1b_1p125: s1 {
+ regulator-min-microvolt = <1125000>;
+ regulator-max-microvolt = <1125000>;
+ regulator-enable-ramp-delay = <200>;
+ };
+
+ vreg_s2b_1p05: s2 {
+ regulator-min-microvolt = <1050000>;
+ regulator-max-microvolt = <1050000>;
+ regulator-enable-ramp-delay = <200>;
+ };
+
+ /* LDOs */
+ vreg_l1b_0p925: l1 {
+ regulator-min-microvolt = <800000>;
+ regulator-max-microvolt = <925000>;
+ regulator-enable-ramp-delay = <250>;
+ regulator-allow-set-load;
+ };
+
+ /* SDHCI 3.3V signal doesn't seem to be supported. */
+ vreg_l2b_2p95: l2 {
+ regulator-min-microvolt = <1648000>;
+ regulator-max-microvolt = <2696000>;
+ regulator-enable-ramp-delay = <250>;
+ regulator-allow-set-load;
+ };
+
+ vreg_l3b_3p3: l3 {
+ regulator-min-microvolt = <1700000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-enable-ramp-delay = <250>;
+ regulator-allow-set-load;
+ };
+
+ vreg_l4b_2p95: l4 {
+ regulator-min-microvolt = <2944000>;
+ regulator-max-microvolt = <2952000>;
+ regulator-enable-ramp-delay = <250>;
+
+ regulator-min-microamp = <200>;
+ regulator-max-microamp = <600000>;
+ regulator-system-load = <570000>;
+ regulator-allow-set-load;
+ };
+
+ /*
+ * Downstream specifies a range of 1721-3600mV,
+ * but the only assigned consumers are SDHCI2 VMMC
+ * and Coresight QPDI that both request pinned 2.95V.
+ * Tighten the range to 1.8-3.328 (closest to 3.3) to
+ * make the mmc driver happy.
+ */
+ vreg_l5b_2p95: l5 {
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <3328000>;
+ regulator-enable-ramp-delay = <250>;
+ regulator-allow-set-load;
+ regulator-system-load = <800000>;
+ };
+
+ vreg_l7b_3p125: l7 {
+ regulator-min-microvolt = <2700000>;
+ regulator-max-microvolt = <3125000>;
+ regulator-enable-ramp-delay = <250>;
+ };
+
+ vreg_l8b_3p3: l8 {
+ regulator-min-microvolt = <3200000>;
+ regulator-max-microvolt = <3400000>;
+ regulator-enable-ramp-delay = <250>;
+ };
+
+ vreg_bob: bob {
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3600000>;
+ regulator-enable-ramp-delay = <500>;
+ };
+ };
+
+ pm660-regulators {
+ compatible = "qcom,rpm-pm660-regulators";
+
+ vdd_s1-supply = <&vph_pwr>;
+ vdd_s2-supply = <&vph_pwr>;
+ vdd_s3-supply = <&vph_pwr>;
+ vdd_s4-supply = <&vph_pwr>;
+ vdd_s5-supply = <&vph_pwr>;
+ vdd_s6-supply = <&vph_pwr>;
+
+ vdd_l1_l6_l7-supply = <&vreg_s5a_1p35>;
+ vdd_l2_l3-supply = <&vreg_s2b_1p05>;
+ vdd_l5-supply = <&vreg_s2b_1p05>;
+ vdd_l8_l9_l10_l11_l12_l13_l14-supply = <&vreg_s4a_2p04>;
+ vdd_l15_l16_l17_l18_l19-supply = <&vreg_bob>;
+
+ /*
+ * S1A (FTAPC0), S2A (FTAPC1), S3A (HFAPC1) are managed
+ * by the Core Power Reduction hardened (CPRh) and the
+ * Operating State Manager (OSM) HW automatically.
+ */
+
+ vreg_s4a_2p04: s4 {
+ regulator-min-microvolt = <1805000>;
+ regulator-max-microvolt = <2040000>;
+ regulator-enable-ramp-delay = <200>;
+ regulator-always-on;
+ };
+
+ vreg_s5a_1p35: s5 {
+ regulator-min-microvolt = <1224000>;
+ regulator-max-microvolt = <1350000>;
+ regulator-enable-ramp-delay = <200>;
+ };
+
+ vreg_s6a_0p87: s6 {
+ regulator-min-microvolt = <504000>;
+ regulator-max-microvolt = <992000>;
+ regulator-enable-ramp-delay = <150>;
+ };
+
+ /* LDOs */
+ vreg_l1a_1p225: l1 {
+ regulator-min-microvolt = <1150000>;
+ regulator-max-microvolt = <1250000>;
+ regulator-enable-ramp-delay = <250>;
+ regulator-allow-set-load;
+ };
+
+ vreg_l2a_1p0: l2 {
+ regulator-min-microvolt = <950000>;
+ regulator-max-microvolt = <1010000>;
+ regulator-enable-ramp-delay = <250>;
+ };
+
+ vreg_l3a_1p0: l3 {
+ regulator-min-microvolt = <950000>;
+ regulator-max-microvolt = <1010000>;
+ regulator-enable-ramp-delay = <250>;
+ };
+
+ vreg_l5a_0p848: l5 {
+ regulator-min-microvolt = <525000>;
+ regulator-max-microvolt = <950000>;
+ regulator-enable-ramp-delay = <250>;
+ };
+
+ vreg_l6a_1p3: l6 {
+ regulator-min-microvolt = <1200000>;
+ regulator-max-microvolt = <1370000>;
+ regulator-allow-set-load;
+ regulator-enable-ramp-delay = <250>;
+ };
+
+ vreg_l7a_1p2: l7 {
+ regulator-min-microvolt = <1200000>;
+ regulator-max-microvolt = <1200000>;
+ regulator-enable-ramp-delay = <250>;
+ };
+
+ vreg_l8a_1p8: l8 {
+ regulator-min-microvolt = <1750000>;
+ regulator-max-microvolt = <1800000>;
+ regulator-enable-ramp-delay = <250>;
+ regulator-system-load = <325000>;
+ regulator-allow-set-load;
+ };
+
+ vreg_l9a_1p8: l9 {
+ regulator-min-microvolt = <1750000>;
+ regulator-max-microvolt = <1900000>;
+ regulator-enable-ramp-delay = <250>;
+ regulator-allow-set-load;
+ };
+
+ vreg_l10a_1p8: l10 {
+ regulator-min-microvolt = <1780000>;
+ regulator-max-microvolt = <1950000>;
+ regulator-enable-ramp-delay = <250>;
+ regulator-allow-set-load;
+ };
+
+ vreg_l11a_1p8: l11 {
+ regulator-min-microvolt = <1780000>;
+ regulator-max-microvolt = <1950000>;
+ regulator-enable-ramp-delay = <250>;
+ };
+
+ vreg_l12a_1p8: l12 {
+ regulator-min-microvolt = <1780000>;
+ regulator-max-microvolt = <1950000>;
+ regulator-enable-ramp-delay = <250>;
+ };
+
+ /* This gives power to the LPDDR4: never turn it off! */
+ vreg_l13a_1p8: l13 {
+ regulator-min-microvolt = <1780000>;
+ regulator-max-microvolt = <1950000>;
+ regulator-enable-ramp-delay = <250>;
+ regulator-boot-on;
+ regulator-always-on;
+ };
+
+ vreg_l14a_1p8: l14 {
+ regulator-min-microvolt = <1710000>;
+ regulator-max-microvolt = <1900000>;
+ regulator-enable-ramp-delay = <250>;
+ };
+
+ vreg_l15a_1p8: l15 {
+ regulator-min-microvolt = <1650000>;
+ regulator-max-microvolt = <2950000>;
+ regulator-enable-ramp-delay = <250>;
+ };
+
+ vreg_l16a_2p7: l16 {
+ regulator-min-microvolt = <2800000>;
+ regulator-max-microvolt = <2800000>;
+ regulator-enable-ramp-delay = <250>;
+ regulator-always-on;
+ };
+
+ vreg_l17a_1p8: l17 {
+ regulator-min-microvolt = <1648000>;
+ regulator-max-microvolt = <2952000>;
+ regulator-enable-ramp-delay = <250>;
+ };
+
+ vreg_l19a_3p3: l19 {
+ regulator-min-microvolt = <3312000>;
+ regulator-max-microvolt = <3400000>;
+ regulator-enable-ramp-delay = <250>;
+ regulator-allow-set-load;
+ };
+ };
+};
+
+&sdhc_1 {
+ status = "okay";
+ supports-cqe;
+
+ mmc-hs200-1_8v;
+ mmc-hs400-1_8v;
+ mmc-hs400-enhanced-strobe;
+
+ vmmc-supply = <&vreg_l4b_2p95>;
+ vqmmc-supply = <&vreg_l8a_1p8>;
+};
+
+&sdhc_2 {
+ status = "okay";
+
+ vmmc-supply = <&vreg_l5b_2p95>;
+ vqmmc-supply = <&vreg_l2b_2p95>;
+};
+
&tlmm {
gpio-reserved-ranges = <8 4>;
};
+
+&usb3 {
+ status = "okay";
+};
+
+&usb3_dwc3 {
+ dr_mode = "peripheral";
+ extcon = <&extcon_usb>;
+};
diff --git a/arch/arm64/boot/dts/qcom/sdm845-mtp.dts b/arch/arm64/boot/dts/qcom/sdm845-mtp.dts
index 5e6e8f4..7713e80 100644
--- a/arch/arm64/boot/dts/qcom/sdm845-mtp.dts
+++ b/arch/arm64/boot/dts/qcom/sdm845-mtp.dts
@@ -9,6 +9,8 @@
#include <dt-bindings/regulator/qcom,rpmh-regulator.h>
#include "sdm845.dtsi"
+#include "pm8998.dtsi"
+#include "pmi8998.dtsi"
/ {
model = "Qualcomm Technologies, Inc. SDM845 MTP";
@@ -45,6 +47,68 @@ vreg_s4a_1p8: pm8998-smps4 {
vin-supply = <&vph_pwr>;
};
+
+ thermal-zones {
+ xo_thermal: xo-thermal {
+ polling-delay-passive = <0>;
+ polling-delay = <0>;
+
+ thermal-sensors = <&pm8998_adc_tm 1>;
+
+ trips {
+ trip-point {
+ temperature = <125000>;
+ hysteresis = <10000>;
+ type = "passive";
+ };
+ };
+ };
+
+ msm_thermal: msm-thermal {
+ polling-delay-passive = <0>;
+ polling-delay = <0>;
+
+ thermal-sensors = <&pm8998_adc_tm 2>;
+
+ trips {
+ trip-point {
+ temperature = <125000>;
+ hysteresis = <10000>;
+ type = "passive";
+ };
+ };
+ };
+
+ pa_thermal: pa-thermal {
+ polling-delay-passive = <0>;
+ polling-delay = <0>;
+
+ thermal-sensors = <&pm8998_adc_tm 3>;
+
+ trips {
+ trip-point {
+ temperature = <125000>;
+ hysteresis = <10000>;
+ type = "passive";
+ };
+ };
+ };
+
+ quiet_thermal: quiet-thermal {
+ polling-delay-passive = <0>;
+ polling-delay = <0>;
+
+ thermal-sensors = <&pm8998_adc_tm 4>;
+
+ trips {
+ trip-point {
+ temperature = <125000>;
+ hysteresis = <10000>;
+ type = "passive";
+ };
+ };
+ };
+ };
};
&adsp_pas {
@@ -467,6 +531,82 @@ &mss_pil {
firmware-name = "qcom/sdm845/mba.mbn", "qcom/sdm845/modem.mbn";
};
+&pm8998_adc {
+ adc-chan@4c {
+ reg = <ADC5_XO_THERM_100K_PU>;
+ label = "xo_therm";
+ qcom,ratiometric;
+ qcom,hw-settle-time-us = <200>;
+ };
+
+ adc-chan@4d {
+ reg = <ADC5_AMUX_THM1_100K_PU>;
+ label = "msm_therm";
+ qcom,ratiometric;
+ qcom,hw-settle-time-us = <200>;
+ };
+
+ adc-chan@4f {
+ reg = <ADC5_AMUX_THM3_100K_PU>;
+ label = "pa_therm1";
+ qcom,ratiometric;
+ qcom,hw-settle-time-us = <200>;
+ };
+
+ adc-chan@51 {
+ reg = <ADC5_AMUX_THM5_100K_PU>;
+ label = "quiet_therm";
+ qcom,ratiometric;
+ qcom,hw-settle-time-us = <200>;
+ };
+
+ adc-chan@83 {
+ reg = <ADC5_VPH_PWR>;
+ label = "vph_pwr";
+ qcom,ratiometric;
+ qcom,hw-settle-time-us = <200>;
+ };
+
+ adc-chan@85 {
+ reg = <ADC5_VCOIN>;
+ label = "vcoin";
+ qcom,ratiometric;
+ qcom,hw-settle-time-us = <200>;
+ };
+};
+
+&pm8998_adc_tm {
+ status = "okay";
+
+ xo-thermistor@1 {
+ reg = <1>;
+ io-channels = <&pm8998_adc ADC5_XO_THERM_100K_PU>;
+ qcom,ratiometric;
+ qcom,hw-settle-time-us = <200>;
+ };
+
+ msm-thermistor@2 {
+ reg = <2>;
+ io-channels = <&pm8998_adc ADC5_AMUX_THM1_100K_PU>;
+ qcom,ratiometric;
+ qcom,hw-settle-time-us = <200>;
+ };
+
+ pa-thermistor@3 {
+ reg = <3>;
+ io-channels = <&pm8998_adc ADC5_AMUX_THM3_100K_PU>;
+ qcom,ratiometric;
+ qcom,hw-settle-time-us = <200>;
+ };
+
+ quiet-thermistor@4 {
+ reg = <4>;
+ io-channels = <&pm8998_adc ADC5_AMUX_THM5_100K_PU>;
+ qcom,ratiometric;
+ qcom,hw-settle-time-us = <200>;
+ };
+};
+
&qupv3_id_1 {
status = "okay";
};
diff --git a/arch/arm64/boot/dts/qcom/sdm845-oneplus-common.dtsi b/arch/arm64/boot/dts/qcom/sdm845-oneplus-common.dtsi
index 3e04aeb..7f42e531 100644
--- a/arch/arm64/boot/dts/qcom/sdm845-oneplus-common.dtsi
+++ b/arch/arm64/boot/dts/qcom/sdm845-oneplus-common.dtsi
@@ -646,6 +646,7 @@ mux {
&venus {
status = "okay";
+ firmware-name = "qcom/sdm845/oneplus6/venus.mbn";
};
&wifi {
diff --git a/arch/arm64/boot/dts/qcom/sdm845-oneplus-enchilada.dts b/arch/arm64/boot/dts/qcom/sdm845-oneplus-enchilada.dts
index 7349307..5936b47 100644
--- a/arch/arm64/boot/dts/qcom/sdm845-oneplus-enchilada.dts
+++ b/arch/arm64/boot/dts/qcom/sdm845-oneplus-enchilada.dts
@@ -11,6 +11,8 @@ / {
model = "OnePlus 6";
compatible = "oneplus,enchilada", "qcom,sdm845";
chassis-type = "handset";
+ qcom,msm-id = <0x141 0x20001>;
+ qcom,board-id = <8 0 17819 22>;
};
&display_panel {
diff --git a/arch/arm64/boot/dts/qcom/sdm845-oneplus-fajita.dts b/arch/arm64/boot/dts/qcom/sdm845-oneplus-fajita.dts
index b63ebc4..78a0b99 100644
--- a/arch/arm64/boot/dts/qcom/sdm845-oneplus-fajita.dts
+++ b/arch/arm64/boot/dts/qcom/sdm845-oneplus-fajita.dts
@@ -11,6 +11,8 @@ / {
model = "OnePlus 6T";
compatible = "oneplus,fajita", "qcom,sdm845";
chassis-type = "handset";
+ qcom,msm-id = <0x141 0x20001>;
+ qcom,board-id = <8 0 18801 41>;
};
&display_panel {
diff --git a/arch/arm64/boot/dts/qcom/sdm845-sony-xperia-tama-akari.dts b/arch/arm64/boot/dts/qcom/sdm845-sony-xperia-tama-akari.dts
new file mode 100644
index 0000000..34f84f1
--- /dev/null
+++ b/arch/arm64/boot/dts/qcom/sdm845-sony-xperia-tama-akari.dts
@@ -0,0 +1,13 @@
+// SPDX-License-Identifier: BSD-3-Clause
+/*
+ * Copyright (c) 2021, Konrad Dybcio <konrad.dybcio@somainline.org>
+ */
+
+/dts-v1/;
+
+#include "sdm845-sony-xperia-tama.dtsi"
+
+/ {
+ model = "Sony Xperia XZ2";
+ compatible = "sony,akari-row", "qcom,sdm845";
+};
diff --git a/arch/arm64/boot/dts/qcom/sdm845-sony-xperia-tama-akatsuki.dts b/arch/arm64/boot/dts/qcom/sdm845-sony-xperia-tama-akatsuki.dts
new file mode 100644
index 0000000..8a0d94e
--- /dev/null
+++ b/arch/arm64/boot/dts/qcom/sdm845-sony-xperia-tama-akatsuki.dts
@@ -0,0 +1,29 @@
+// SPDX-License-Identifier: BSD-3-Clause
+/*
+ * Copyright (c) 2021, Konrad Dybcio <konrad.dybcio@somainline.org>
+ */
+
+/dts-v1/;
+
+#include "sdm845-sony-xperia-tama.dtsi"
+
+/ {
+ model = "Sony Xperia XZ3";
+ compatible = "sony,akatsuki-row", "qcom,sdm845";
+};
+
+/* For the future: WLED + LAB/IBB/OLEDB are not used on Akatsuki */
+&vreg_l14a_1p8 {
+ regulator-min-microvolt = <1840000>;
+ regulator-max-microvolt = <1840000>;
+};
+
+&vreg_l22a_2p8 {
+ regulator-min-microvolt = <2700000>;
+ regulator-max-microvolt = <2700000>;
+};
+
+&vreg_l28a_2p8 {
+ regulator-min-microvolt = <3000000>;
+ regulator-max-microvolt = <3000000>;
+};
diff --git a/arch/arm64/boot/dts/qcom/sdm845-sony-xperia-tama-apollo.dts b/arch/arm64/boot/dts/qcom/sdm845-sony-xperia-tama-apollo.dts
new file mode 100644
index 0000000..c9e62c7
--- /dev/null
+++ b/arch/arm64/boot/dts/qcom/sdm845-sony-xperia-tama-apollo.dts
@@ -0,0 +1,13 @@
+// SPDX-License-Identifier: BSD-3-Clause
+/*
+ * Copyright (c) 2021, Konrad Dybcio <konrad.dybcio@somainline.org>
+ */
+
+/dts-v1/;
+
+#include "sdm845-sony-xperia-tama.dtsi"
+
+/ {
+ model = "Sony Xperia XZ2 Compact";
+ compatible = "sony,apollo-row", "qcom,sdm845";
+};
diff --git a/arch/arm64/boot/dts/qcom/sdm845-sony-xperia-tama.dtsi b/arch/arm64/boot/dts/qcom/sdm845-sony-xperia-tama.dtsi
new file mode 100644
index 0000000..281fe6d
--- /dev/null
+++ b/arch/arm64/boot/dts/qcom/sdm845-sony-xperia-tama.dtsi
@@ -0,0 +1,438 @@
+// SPDX-License-Identifier: BSD-3-Clause
+/*
+ * Copyright (c) 2021, Konrad Dybcio <konrad.dybcio@somainline.org>
+ */
+
+#include <dt-bindings/input/input.h>
+#include <dt-bindings/regulator/qcom,rpmh-regulator.h>
+#include "sdm845.dtsi"
+#include "pm8005.dtsi"
+#include "pm8998.dtsi"
+#include "pmi8998.dtsi"
+
+/ {
+ qcom,msm-id = <321 0x20001>; /* SDM845 v2.1 */
+ qcom,board-id = <8 0>;
+
+ gpio-keys {
+ compatible = "gpio-keys";
+
+ /* Neither Camera Focus, nor Camera Shutter seem to work... */
+
+ vol-down {
+ label = "volume_down";
+ gpios = <&pm8998_gpio 5 GPIO_ACTIVE_LOW>;
+ linux,code = <KEY_VOLUMEDOWN>;
+ debounce-interval = <15>;
+ gpio-key,wakeup;
+ };
+ };
+
+ vph_pwr: vph-pwr-regulator {
+ compatible = "regulator-fixed";
+ regulator-name = "vph_pwr";
+ regulator-min-microvolt = <3700000>;
+ regulator-max-microvolt = <3700000>;
+ };
+
+ vreg_s4a_1p8: pm8998-smps4 {
+ compatible = "regulator-fixed";
+ regulator-name = "vreg_s4a_1p8";
+
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+
+ regulator-always-on;
+ regulator-boot-on;
+
+ vin-supply = <&vph_pwr>;
+ };
+
+ reserved-memory {
+ /* SONY was cool and didn't diverge from MTP this time, yay! */
+ cont_splash_mem: memory@9d400000 {
+ reg = <0x0 0x9d400000 0x0 0x2400000>;
+ no-map;
+ };
+
+ ramoops@ffc00000 {
+ compatible = "ramoops";
+ reg = <0x0 0xffc00000 0x0 0x100000>;
+ record-size = <0x10000>;
+ console-size = <0x60000>;
+ ftrace-size = <0x10000>;
+ pmsg-size = <0x20000>;
+ ecc-size = <16>;
+ no-map;
+ };
+ };
+};
+
+&apps_rsc {
+ pm8998-rpmh-regulators {
+ compatible = "qcom,pm8998-rpmh-regulators";
+ qcom,pmic-id = "a";
+
+ vdd-s1-supply = <&vph_pwr>;
+ vdd-s2-supply = <&vph_pwr>;
+ vdd-s3-supply = <&vph_pwr>;
+ vdd-s4-supply = <&vph_pwr>;
+ vdd-s5-supply = <&vph_pwr>;
+ vdd-s6-supply = <&vph_pwr>;
+ vdd-s7-supply = <&vph_pwr>;
+ vdd-s8-supply = <&vph_pwr>;
+ vdd-s9-supply = <&vph_pwr>;
+ vdd-s10-supply = <&vph_pwr>;
+ vdd-s11-supply = <&vph_pwr>;
+ vdd-s12-supply = <&vph_pwr>;
+ vdd-s13-supply = <&vph_pwr>;
+ vdd-l1-l27-supply = <&vreg_s7a_0p9>;
+ vdd-l2-l8-l17-supply = <&vreg_s3a_1p3>;
+ vdd-l3-l11-supply = <&vreg_s7a_0p9>;
+ vdd-l4-l5-supply = <&vreg_s7a_0p9>;
+ vdd-l6-supply = <&vph_pwr>;
+ vdd-l7-l12-l14-l15-supply = <&vreg_s5a_1p9>;
+ vdd-l9-supply = <&vreg_s5a_1p9>;
+ vdd-l10-l23-l25-supply = <&src_vreg_bob>;
+ vdd-l13-l19-l21-supply = <&src_vreg_bob>;
+ vdd-l16-l28-supply = <&src_vreg_bob>;
+ vdd-l18-l22-supply = <&src_vreg_bob>;
+ vdd-l20-l24-supply = <&src_vreg_bob>;
+ vdd-l26-supply = <&vreg_s3a_1p3>;
+ vin-lvs-1-2-supply = <&vreg_s4a_1p8>;
+
+ vreg_s2a_1p1: smps2 {
+ regulator-min-microvolt = <1100000>;
+ regulator-max-microvolt = <1100000>;
+ };
+
+ vreg_s3a_1p3: smps3 {
+ regulator-min-microvolt = <1352000>;
+ regulator-max-microvolt = <1352000>;
+ };
+
+ vreg_s5a_1p9: smps5 {
+ regulator-min-microvolt = <1904000>;
+ regulator-max-microvolt = <2040000>;
+ };
+
+ vreg_s7a_0p9: smps7 {
+ regulator-min-microvolt = <900000>;
+ regulator-max-microvolt = <1028000>;
+ };
+
+ vreg_l1a_0p9: ldo1 {
+ regulator-min-microvolt = <880000>;
+ regulator-max-microvolt = <880000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l2a_1p2: ldo2 {
+ regulator-min-microvolt = <1200000>;
+ regulator-max-microvolt = <1200000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ regulator-always-on;
+ };
+
+ vreg_l3a_1p0: ldo3 {
+ regulator-min-microvolt = <1000000>;
+ regulator-max-microvolt = <1000000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l5a_0p8: ldo5 {
+ regulator-min-microvolt = <800000>;
+ regulator-max-microvolt = <800000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l6a_1p8: ldo6 {
+ regulator-min-microvolt = <1856000>;
+ regulator-max-microvolt = <1856000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l7a_1p8: ldo7 {
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l8a_1p2: ldo8 {
+ regulator-min-microvolt = <1200000>;
+ regulator-max-microvolt = <1248000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l9a_1p7: ldo9 {
+ regulator-min-microvolt = <1704000>;
+ regulator-max-microvolt = <2928000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l10a_1p7: ldo10 {
+ regulator-min-microvolt = <1704000>;
+ regulator-max-microvolt = <2928000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l11a_1p0: ldo11 {
+ regulator-min-microvolt = <1000000>;
+ regulator-max-microvolt = <1048000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l12a_1p8: ldo12 {
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l13a_1p8: ldo13 {
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <2960000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l14a_1p8: ldo14 {
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l15a_1p8: ldo15 {
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l16a_2p7: ldo16 {
+ regulator-min-microvolt = <2704000>;
+ regulator-max-microvolt = <2704000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l17a_1p3: ldo17 {
+ regulator-min-microvolt = <1304000>;
+ regulator-max-microvolt = <1304000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l18a_2p7: ldo18 {
+ regulator-min-microvolt = <2704000>;
+ regulator-max-microvolt = <2960000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l19a_2p7: ldo19 {
+ regulator-min-microvolt = <2700000>;
+ regulator-max-microvolt = <2700000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+
+ /*
+ * The driver *really* doesn't want this regualtor to exist,
+ * saying that it could not get the current voltage (-ENOTRECOVERABLE)
+ * even though it surely is used on these devices (as a voltage
+ * source for camera autofocus)
+ */
+ status = "disabled";
+ };
+
+ vreg_l20a_2p7: ldo20 {
+ regulator-min-microvolt = <2704000>;
+ regulator-max-microvolt = <2960000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l21a_2p7: ldo21 {
+ regulator-min-microvolt = <2704000>;
+ regulator-max-microvolt = <2960000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l22a_2p8: ldo22 {
+ regulator-min-microvolt = <2800000>;
+ regulator-max-microvolt = <2800000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l23a_3p0: ldo23 {
+ regulator-min-microvolt = <3000000>;
+ regulator-max-microvolt = <3312000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l24a_3p1: ldo24 {
+ regulator-min-microvolt = <3088000>;
+ regulator-max-microvolt = <3088000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l25a_3p0: ldo25 {
+ regulator-min-microvolt = <3000000>;
+ regulator-max-microvolt = <3312000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l26a_1p2: ldo26 {
+ regulator-min-microvolt = <1200000>;
+ regulator-max-microvolt = <1200000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l28a_2p8: ldo28 {
+ regulator-min-microvolt = <2856000>;
+ regulator-max-microvolt = <3008000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_lvs1a_1p8: lvs1 {
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_lvs2a_1p8: lvs2 {
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+ };
+
+ pmi8998-rpmh-regulators {
+ compatible = "qcom,pmi8998-rpmh-regulators";
+ qcom,pmic-id = "b";
+
+ src_vreg_bob: bob {
+ regulator-min-microvolt = <3312000>;
+ regulator-max-microvolt = <3600000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+ };
+
+ pm8005-rpmh-regulators {
+ compatible = "qcom,pm8005-rpmh-regulators";
+ qcom,pmic-id = "c";
+
+ vdd-s1-supply = <&vph_pwr>;
+ vdd-s2-supply = <&vph_pwr>;
+ vdd-s3-supply = <&vph_pwr>;
+ vdd-s4-supply = <&vph_pwr>;
+
+ vreg_s3c_0p6: smps3 {
+ regulator-min-microvolt = <600000>;
+ regulator-max-microvolt = <600000>;
+ };
+ };
+};
+
+&gcc {
+ protected-clocks = <GCC_QSPI_CORE_CLK>,
+ <GCC_QSPI_CORE_CLK_SRC>,
+ <GCC_QSPI_CNOC_PERIPH_AHB_CLK>,
+ <GCC_LPASS_Q6_AXI_CLK>,
+ <GCC_LPASS_SWAY_CLK>;
+};
+
+&i2c5 {
+ status = "okay";
+ clock-frequency = <400000>;
+
+ /* Synaptics touchscreen @ 2c, 3c */
+};
+
+&i2c10 {
+ status = "okay";
+ clock-frequency = <400000>;
+
+ /* Qcom SMB1355 @ 8, c */
+ /* NXP PN547 NFC @ 28 */
+ /* Renesas IDTP9221 Qi charger @ 61 */
+};
+
+&i2c14 {
+ status = "okay";
+ clock-frequency = <400000>;
+
+ /* SONY ToF sensor @ 52 */
+ /* AMS TCS3490 RGB+IR color sensor @ 72 */
+};
+
+&qupv3_id_0 {
+ status = "okay";
+};
+
+&qupv3_id_1 {
+ status = "okay";
+};
+
+&sdhc_2 {
+ status = "okay";
+
+ vmmc-supply = <&vreg_l21a_2p7>;
+ vqmmc-supply = <&vreg_l13a_1p8>;
+ cd-gpios = <&tlmm 126 GPIO_ACTIVE_HIGH>;
+ pinctrl-0 = <&sdc2_default_state>;
+ pinctrl-names = "default";
+ bus-width = <4>;
+ no-sdio;
+ no-emmc;
+};
+
+&tlmm {
+ gpio-reserved-ranges = <0 4>, <81 4>;
+
+ sdc2_default_state: sdc2-default-state {
+ clk {
+ pins = "sdc2_clk";
+ drive-strength = <16>;
+ bias-disable;
+ };
+
+ cmd {
+ pins = "sdc2_cmd";
+ drive-strength = <10>;
+ bias-pull-up;
+ };
+
+ data {
+ pins = "sdc2_data";
+ drive-strength = <10>;
+ bias-pull-up;
+ };
+ };
+};
+
+&uart6 {
+ status = "okay";
+};
+
+&uart9 {
+ status = "okay";
+};
+
+/* BIG WARNING! DO NOT TOUCH UFS, YOUR DEVICE WILL DIE! */
+&ufs_mem_hc { status = "disabled"; };
+&ufs_mem_phy { status = "disabled"; };
+
+&usb_1 {
+ status = "okay";
+
+ qcom,select-utmi-as-pipe-clk;
+};
+
+&usb_1_dwc3 {
+ dr_mode = "peripheral";
+
+ maximum-speed = "high-speed";
+ phys = <&usb_1_hsphy>;
+ phy-names = "usb2-phy";
+};
+
+&usb_1_hsphy {
+ status = "okay";
+
+ vdd-supply = <&vreg_l1a_0p9>;
+ vdda-pll-supply = <&vreg_l12a_1p8>;
+ vdda-phy-dpdm-supply = <&vreg_l24a_3p1>;
+};
diff --git a/arch/arm64/boot/dts/qcom/sdm845-xiaomi-beryllium.dts b/arch/arm64/boot/dts/qcom/sdm845-xiaomi-beryllium.dts
index 580d4cc..3673895 100644
--- a/arch/arm64/boot/dts/qcom/sdm845-xiaomi-beryllium.dts
+++ b/arch/arm64/boot/dts/qcom/sdm845-xiaomi-beryllium.dts
@@ -514,6 +514,7 @@ &usb_1_qmpphy {
&venus {
status = "okay";
+ firmware-name = "qcom/sdm845/beryllium/venus.mbn";
};
&wcd9340{
diff --git a/arch/arm64/boot/dts/qcom/sdm845.dtsi b/arch/arm64/boot/dts/qcom/sdm845.dtsi
index 5260875..cfdeaa8 100644
--- a/arch/arm64/boot/dts/qcom/sdm845.dtsi
+++ b/arch/arm64/boot/dts/qcom/sdm845.dtsi
@@ -1078,7 +1078,7 @@ gcc: clock-controller@100000 {
};
qfprom@784000 {
- compatible = "qcom,qfprom";
+ compatible = "qcom,sdm845-qfprom", "qcom,qfprom";
reg = <0 0x00784000 0 0x8ff>;
#address-cells = <1>;
#size-cells = <1>;
@@ -4415,7 +4415,6 @@ dsi1_phy: dsi-phy@ae96400 {
gpu: gpu@5000000 {
compatible = "qcom,adreno-630.2", "qcom,adreno";
- #stream-id-cells = <16>;
reg = <0 0x5000000 0 0x40000>, <0 0x509e000 0 0x10>;
reg-names = "kgsl_3d0_reg_memory", "cx_mem";
diff --git a/arch/arm64/boot/dts/qcom/sdm850-lenovo-yoga-c630.dts b/arch/arm64/boot/dts/qcom/sdm850-lenovo-yoga-c630.dts
index d6b2ba4..58845a1 100644
--- a/arch/arm64/boot/dts/qcom/sdm850-lenovo-yoga-c630.dts
+++ b/arch/arm64/boot/dts/qcom/sdm850-lenovo-yoga-c630.dts
@@ -523,6 +523,10 @@ dai@0 {
dai@1 {
reg = <1>;
};
+
+ dai@2 {
+ reg = <2>;
+ };
};
&sound {
@@ -535,6 +539,7 @@ &sound {
"SpkrLeft IN", "SPK1 OUT",
"SpkrRight IN", "SPK2 OUT",
"MM_DL1", "MultiMedia1 Playback",
+ "MM_DL3", "MultiMedia3 Playback",
"MultiMedia2 Capture", "MM_UL2";
mm1-dai-link {
@@ -551,6 +556,13 @@ cpu {
};
};
+ mm3-dai-link {
+ link-name = "MultiMedia3";
+ cpu {
+ sound-dai = <&q6asmdai MSM_FRONTEND_DAI_MULTIMEDIA3>;
+ };
+ };
+
slim-dai-link {
link-name = "SLIM Playback";
cpu {
@@ -580,6 +592,21 @@ codec {
sound-dai = <&wcd9340 1>;
};
};
+
+ slim-wcd-dai-link {
+ link-name = "SLIM WCD Playback";
+ cpu {
+ sound-dai = <&q6afedai SLIMBUS_1_RX>;
+ };
+
+ platform {
+ sound-dai = <&q6routing>;
+ };
+
+ codec {
+ sound-dai = <&wcd9340 2>;
+ };
+ };
};
&tlmm {
@@ -744,6 +771,9 @@ &wcd9340{
vdd-tx-supply = <&vreg_s4a_1p8>;
vdd-rx-supply = <&vreg_s4a_1p8>;
vdd-io-supply = <&vreg_s4a_1p8>;
+ qcom,mbhc-buttons-vthreshold-microvolt = <75000 150000 237000 500000 500000 500000 500000 500000>;
+ qcom,mbhc-headset-vthreshold-microvolt = <1700000>;
+ qcom,mbhc-headphone-vthreshold-microvolt = <50000>;
swm: swm@c85 {
left_spkr: wsa8810-left{
diff --git a/arch/arm64/boot/dts/qcom/sm6125-sony-xperia-seine-pdx201.dts b/arch/arm64/boot/dts/qcom/sm6125-sony-xperia-seine-pdx201.dts
index 45eab02..871ccbb 100644
--- a/arch/arm64/boot/dts/qcom/sm6125-sony-xperia-seine-pdx201.dts
+++ b/arch/arm64/boot/dts/qcom/sm6125-sony-xperia-seine-pdx201.dts
@@ -42,7 +42,6 @@ extcon_usb: extcon-usb {
gpio-keys {
status = "okay";
compatible = "gpio-keys";
- input-name = "gpio-keys";
#address-cells = <1>;
#size-cells = <0>;
autorepeat;
diff --git a/arch/arm64/boot/dts/qcom/sm6125.dtsi b/arch/arm64/boot/dts/qcom/sm6125.dtsi
index 51286dd..49e6bca 100644
--- a/arch/arm64/boot/dts/qcom/sm6125.dtsi
+++ b/arch/arm64/boot/dts/qcom/sm6125.dtsi
@@ -302,6 +302,56 @@ rpmcc: clock-controller {
compatible = "qcom,rpmcc-sm6125", "qcom,rpmcc";
#clock-cells = <1>;
};
+
+ rpmpd: power-controller {
+ compatible = "qcom,sm6125-rpmpd";
+ #power-domain-cells = <1>;
+ operating-points-v2 = <&rpmpd_opp_table>;
+
+ rpmpd_opp_table: opp-table {
+ compatible = "operating-points-v2";
+
+ rpmpd_opp_ret: opp1 {
+ opp-level = <RPM_SMD_LEVEL_RETENTION>;
+ };
+
+ rpmpd_opp_ret_plus: opp2 {
+ opp-level = <RPM_SMD_LEVEL_RETENTION_PLUS>;
+ };
+
+ rpmpd_opp_min_svs: opp3 {
+ opp-level = <RPM_SMD_LEVEL_MIN_SVS>;
+ };
+
+ rpmpd_opp_low_svs: opp4 {
+ opp-level = <RPM_SMD_LEVEL_LOW_SVS>;
+ };
+
+ rpmpd_opp_svs: opp5 {
+ opp-level = <RPM_SMD_LEVEL_SVS>;
+ };
+
+ rpmpd_opp_svs_plus: opp6 {
+ opp-level = <RPM_SMD_LEVEL_SVS_PLUS>;
+ };
+
+ rpmpd_opp_nom: opp7 {
+ opp-level = <RPM_SMD_LEVEL_NOM>;
+ };
+
+ rpmpd_opp_nom_plus: opp8 {
+ opp-level = <RPM_SMD_LEVEL_NOM_PLUS>;
+ };
+
+ rpmpd_opp_turbo: opp9 {
+ opp-level = <RPM_SMD_LEVEL_TURBO>;
+ };
+
+ rpmpd_opp_turbo_no_cpr: opp10 {
+ opp-level = <RPM_SMD_LEVEL_TURBO_NO_CPR>;
+ };
+ };
+ };
};
};
@@ -398,6 +448,9 @@ sdhc_1: sdhci@4744000 {
<&gcc GCC_SDCC1_APPS_CLK>,
<&xo_board>;
clock-names = "iface", "core", "xo";
+
+ power-domains = <&rpmpd 0>;
+
bus-width = <8>;
non-removable;
status = "disabled";
@@ -421,6 +474,8 @@ sdhc_2: sdhci@4784000 {
pinctrl-1 = <&sdc2_state_off>;
pinctrl-names = "default", "sleep";
+ power-domains = <&rpmpd 0>;
+
bus-width = <4>;
status = "disabled";
};
diff --git a/arch/arm64/boot/dts/qcom/sm6350.dtsi b/arch/arm64/boot/dts/qcom/sm6350.dtsi
index 973e18f..d7c9edf 100644
--- a/arch/arm64/boot/dts/qcom/sm6350.dtsi
+++ b/arch/arm64/boot/dts/qcom/sm6350.dtsi
@@ -361,6 +361,80 @@ smem {
hwlocks = <&tcsr_mutex 3>;
};
+ smp2p-adsp {
+ compatible = "qcom,smp2p";
+ qcom,smem = <443>, <429>;
+ interrupts-extended = <&ipcc IPCC_CLIENT_LPASS
+ IPCC_MPROC_SIGNAL_SMP2P
+ IRQ_TYPE_EDGE_RISING>;
+ mboxes = <&ipcc IPCC_CLIENT_LPASS
+ IPCC_MPROC_SIGNAL_SMP2P>;
+
+ qcom,local-pid = <0>;
+ qcom,remote-pid = <2>;
+
+ smp2p_adsp_out: master-kernel {
+ qcom,entry-name = "master-kernel";
+ #qcom,smem-state-cells = <1>;
+ };
+
+ smp2p_adsp_in: slave-kernel {
+ qcom,entry-name = "slave-kernel";
+ interrupt-controller;
+ #interrupt-cells = <2>;
+ };
+ };
+
+ smp2p-cdsp {
+ compatible = "qcom,smp2p";
+ qcom,smem = <94>, <432>;
+ interrupts-extended = <&ipcc IPCC_CLIENT_CDSP
+ IPCC_MPROC_SIGNAL_SMP2P
+ IRQ_TYPE_EDGE_RISING>;
+ mboxes = <&ipcc IPCC_CLIENT_CDSP
+ IPCC_MPROC_SIGNAL_SMP2P>;
+
+ qcom,local-pid = <0>;
+ qcom,remote-pid = <5>;
+
+ smp2p_cdsp_out: master-kernel {
+ qcom,entry-name = "master-kernel";
+ #qcom,smem-state-cells = <1>;
+ };
+
+ smp2p_cdsp_in: slave-kernel {
+ qcom,entry-name = "slave-kernel";
+ interrupt-controller;
+ #interrupt-cells = <2>;
+ };
+ };
+
+ smp2p-mpss {
+ compatible = "qcom,smp2p";
+ qcom,smem = <435>, <428>;
+
+ interrupts-extended = <&ipcc IPCC_CLIENT_MPSS
+ IPCC_MPROC_SIGNAL_SMP2P
+ IRQ_TYPE_EDGE_RISING>;
+ mboxes = <&ipcc IPCC_CLIENT_MPSS
+ IPCC_MPROC_SIGNAL_SMP2P>;
+
+ qcom,local-pid = <0>;
+ qcom,remote-pid = <1>;
+
+ modem_smp2p_out: master-kernel {
+ qcom,entry-name = "master-kernel";
+ #qcom,smem-state-cells = <1>;
+ };
+
+ modem_smp2p_in: slave-kernel {
+ qcom,entry-name = "slave-kernel";
+
+ interrupt-controller;
+ #interrupt-cells = <2>;
+ };
+ };
+
soc: soc@0 {
#address-cells = <2>;
#size-cells = <2>;
@@ -473,6 +547,211 @@ tcsr_mutex: hwlock@1f40000 {
#hwlock-cells = <1>;
};
+ adsp: remoteproc@3000000 {
+ compatible = "qcom,sm6350-adsp-pas";
+ reg = <0 0x03000000 0 0x100>;
+
+ interrupts-extended = <&pdc 6 IRQ_TYPE_LEVEL_HIGH>,
+ <&smp2p_adsp_in 0 IRQ_TYPE_EDGE_RISING>,
+ <&smp2p_adsp_in 1 IRQ_TYPE_EDGE_RISING>,
+ <&smp2p_adsp_in 2 IRQ_TYPE_EDGE_RISING>,
+ <&smp2p_adsp_in 3 IRQ_TYPE_EDGE_RISING>;
+ interrupt-names = "wdog", "fatal", "ready",
+ "handover", "stop-ack";
+
+ clocks = <&rpmhcc RPMH_CXO_CLK>;
+ clock-names = "xo";
+
+ power-domains = <&rpmhpd SM6350_LCX>,
+ <&rpmhpd SM6350_LMX>;
+ power-domain-names = "lcx", "lmx";
+
+ memory-region = <&pil_adsp_mem>;
+
+ qcom,qmp = <&aoss_qmp>;
+
+ qcom,smem-states = <&smp2p_adsp_out 0>;
+ qcom,smem-state-names = "stop";
+
+ status = "disabled";
+
+ glink-edge {
+ interrupts-extended = <&ipcc IPCC_CLIENT_LPASS
+ IPCC_MPROC_SIGNAL_GLINK_QMP
+ IRQ_TYPE_EDGE_RISING>;
+ mboxes = <&ipcc IPCC_CLIENT_LPASS
+ IPCC_MPROC_SIGNAL_GLINK_QMP>;
+
+ label = "lpass";
+ qcom,remote-pid = <2>;
+
+ fastrpc {
+ compatible = "qcom,fastrpc";
+ qcom,glink-channels = "fastrpcglink-apps-dsp";
+ label = "adsp";
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ compute-cb@3 {
+ compatible = "qcom,fastrpc-compute-cb";
+ reg = <3>;
+ iommus = <&apps_smmu 0x1003 0x0>;
+ };
+
+ compute-cb@4 {
+ compatible = "qcom,fastrpc-compute-cb";
+ reg = <4>;
+ iommus = <&apps_smmu 0x1004 0x0>;
+ };
+
+ compute-cb@5 {
+ compatible = "qcom,fastrpc-compute-cb";
+ reg = <5>;
+ iommus = <&apps_smmu 0x1005 0x0>;
+ qcom,nsessions = <5>;
+ };
+ };
+ };
+ };
+
+ mpss: remoteproc@4080000 {
+ compatible = "qcom,sm6350-mpss-pas";
+ reg = <0x0 0x04080000 0x0 0x4040>;
+
+ interrupts-extended = <&intc GIC_SPI 136 IRQ_TYPE_EDGE_RISING>,
+ <&modem_smp2p_in 0 IRQ_TYPE_EDGE_RISING>,
+ <&modem_smp2p_in 1 IRQ_TYPE_EDGE_RISING>,
+ <&modem_smp2p_in 2 IRQ_TYPE_EDGE_RISING>,
+ <&modem_smp2p_in 3 IRQ_TYPE_EDGE_RISING>,
+ <&modem_smp2p_in 7 IRQ_TYPE_EDGE_RISING>;
+ interrupt-names = "wdog", "fatal", "ready", "handover",
+ "stop-ack", "shutdown-ack";
+
+ clocks = <&rpmhcc RPMH_CXO_CLK>;
+ clock-names = "xo";
+
+ power-domains = <&rpmhpd SM6350_CX>,
+ <&rpmhpd SM6350_MSS>;
+ power-domain-names = "cx", "mss";
+
+ memory-region = <&pil_modem_mem>;
+
+ qcom,qmp = <&aoss_qmp>;
+
+ qcom,smem-states = <&modem_smp2p_out 0>;
+ qcom,smem-state-names = "stop";
+
+ status = "disabled";
+
+ glink-edge {
+ interrupts-extended = <&ipcc IPCC_CLIENT_MPSS
+ IPCC_MPROC_SIGNAL_GLINK_QMP
+ IRQ_TYPE_EDGE_RISING>;
+ mboxes = <&ipcc IPCC_CLIENT_MPSS
+ IPCC_MPROC_SIGNAL_GLINK_QMP>;
+ label = "modem";
+ qcom,remote-pid = <1>;
+ };
+ };
+
+ cdsp: remoteproc@8300000 {
+ compatible = "qcom,sm6350-cdsp-pas";
+ reg = <0 0x08300000 0 0x10000>;
+
+ interrupts-extended = <&intc GIC_SPI 578 IRQ_TYPE_LEVEL_HIGH>,
+ <&smp2p_cdsp_in 0 IRQ_TYPE_EDGE_RISING>,
+ <&smp2p_cdsp_in 1 IRQ_TYPE_EDGE_RISING>,
+ <&smp2p_cdsp_in 2 IRQ_TYPE_EDGE_RISING>,
+ <&smp2p_cdsp_in 3 IRQ_TYPE_EDGE_RISING>;
+ interrupt-names = "wdog", "fatal", "ready",
+ "handover", "stop-ack";
+
+ clocks = <&rpmhcc RPMH_CXO_CLK>;
+ clock-names = "xo";
+
+ power-domains = <&rpmhpd SM6350_CX>,
+ <&rpmhpd SM6350_MX>;
+ power-domain-names = "cx", "mx";
+
+ memory-region = <&pil_cdsp_mem>;
+
+ qcom,qmp = <&aoss_qmp>;
+
+ qcom,smem-states = <&smp2p_cdsp_out 0>;
+ qcom,smem-state-names = "stop";
+
+ status = "disabled";
+
+ glink-edge {
+ interrupts-extended = <&ipcc IPCC_CLIENT_CDSP
+ IPCC_MPROC_SIGNAL_GLINK_QMP
+ IRQ_TYPE_EDGE_RISING>;
+ mboxes = <&ipcc IPCC_CLIENT_CDSP
+ IPCC_MPROC_SIGNAL_GLINK_QMP>;
+
+ label = "cdsp";
+ qcom,remote-pid = <5>;
+
+ fastrpc {
+ compatible = "qcom,fastrpc";
+ qcom,glink-channels = "fastrpcglink-apps-dsp";
+ label = "cdsp";
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ compute-cb@1 {
+ compatible = "qcom,fastrpc-compute-cb";
+ reg = <1>;
+ iommus = <&apps_smmu 0x1401 0x20>;
+ };
+
+ compute-cb@2 {
+ compatible = "qcom,fastrpc-compute-cb";
+ reg = <2>;
+ iommus = <&apps_smmu 0x1402 0x20>;
+ };
+
+ compute-cb@3 {
+ compatible = "qcom,fastrpc-compute-cb";
+ reg = <3>;
+ iommus = <&apps_smmu 0x1403 0x20>;
+ };
+
+ compute-cb@4 {
+ compatible = "qcom,fastrpc-compute-cb";
+ reg = <4>;
+ iommus = <&apps_smmu 0x1404 0x20>;
+ };
+
+ compute-cb@5 {
+ compatible = "qcom,fastrpc-compute-cb";
+ reg = <5>;
+ iommus = <&apps_smmu 0x1405 0x20>;
+ };
+
+ compute-cb@6 {
+ compatible = "qcom,fastrpc-compute-cb";
+ reg = <6>;
+ iommus = <&apps_smmu 0x1406 0x20>;
+ };
+
+ compute-cb@7 {
+ compatible = "qcom,fastrpc-compute-cb";
+ reg = <7>;
+ iommus = <&apps_smmu 0x1407 0x20>;
+ };
+
+ compute-cb@8 {
+ compatible = "qcom,fastrpc-compute-cb";
+ reg = <8>;
+ iommus = <&apps_smmu 0x1408 0x20>;
+ };
+
+ /* note: secure cb9 in downstream */
+ };
+ };
+ };
+
sdhc_2: sdhci@8804000 {
compatible = "qcom,sm6350-sdhci", "qcom,sdhci-msm-v5";
reg = <0 0x08804000 0 0x1000>;
@@ -531,10 +810,10 @@ usb_1_qmpphy: phy@88e9000 {
ranges;
clocks = <&gcc GCC_USB3_PRIM_PHY_AUX_CLK>,
+ <&xo_board>,
<&rpmhcc RPMH_QLINK_CLK>,
- <&gcc GCC_USB3_PRIM_PHY_COM_AUX_CLK>,
- <&xo_board>;
- clock-names = "aux", "ref", "com_aux", "cfg_ahb";
+ <&gcc GCC_USB3_PRIM_PHY_COM_AUX_CLK>;
+ clock-names = "aux", "cfg_ahb", "ref", "com_aux";
resets = <&gcc GCC_USB3_DP_PHY_PRIM_BCR>,
<&gcc GCC_USB3_PHY_PRIM_BCR>;
@@ -592,11 +871,12 @@ usb_1: usb@a6f8800 {
"sleep";
interrupts-extended = <&intc GIC_SPI 130 IRQ_TYPE_LEVEL_HIGH>,
- <&pdc 14 IRQ_TYPE_EDGE_BOTH>,
+ <&pdc 17 IRQ_TYPE_LEVEL_HIGH>,
<&pdc 15 IRQ_TYPE_EDGE_BOTH>,
- <&pdc 17 IRQ_TYPE_LEVEL_HIGH>;
- interrupt-names = "hs_phy_irq", "dp_hs_phy_irq",
- "dm_hs_phy_irq", "ss_phy_irq";
+ <&pdc 14 IRQ_TYPE_EDGE_BOTH>;
+
+ interrupt-names = "hs_phy_irq", "ss_phy_irq",
+ "dm_hs_phy_irq", "dp_hs_phy_irq";
power-domains = <&gcc USB30_PRIM_GDSC>;
@@ -631,7 +911,7 @@ tsens0: thermal-sensor@c263000 {
reg = <0 0x0c263000 0 0x1ff>, /* TM */
<0 0x0c222000 0 0x8>; /* SROT */
#qcom,sensors = <16>;
- interrupts = <&pdc 26 IRQ_TYPE_LEVEL_HIGH>,
+ interrupts-extended = <&pdc 26 IRQ_TYPE_LEVEL_HIGH>,
<&pdc 28 IRQ_TYPE_LEVEL_HIGH>;
interrupt-names = "uplow", "critical";
#thermal-sensor-cells = <1>;
@@ -642,7 +922,7 @@ tsens1: thermal-sensor@c265000 {
reg = <0 0x0c265000 0 0x1ff>, /* TM */
<0 0x0c223000 0 0x8>; /* SROT */
#qcom,sensors = <16>;
- interrupts = <&pdc 27 IRQ_TYPE_LEVEL_HIGH>,
+ interrupts-extended = <&pdc 27 IRQ_TYPE_LEVEL_HIGH>,
<&pdc 29 IRQ_TYPE_LEVEL_HIGH>;
interrupt-names = "uplow", "critical";
#thermal-sensor-cells = <1>;
@@ -656,7 +936,6 @@ aoss_qmp: power-controller@c300000 {
mboxes = <&ipcc IPCC_CLIENT_AOP IPCC_MPROC_SIGNAL_GLINK_QMP>;
#clock-cells = <0>;
- #power-domain-cells = <1>;
};
spmi_bus: spmi@c440000 {
diff --git a/arch/arm64/boot/dts/qcom/sm7225-fairphone-fp4.dts b/arch/arm64/boot/dts/qcom/sm7225-fairphone-fp4.dts
index 8d6fd22..d4af9e0 100644
--- a/arch/arm64/boot/dts/qcom/sm7225-fairphone-fp4.dts
+++ b/arch/arm64/boot/dts/qcom/sm7225-fairphone-fp4.dts
@@ -15,6 +15,7 @@
/ {
model = "Fairphone 4";
compatible = "fairphone,fp4", "qcom,sm7225";
+ chassis-type = "handset";
/* required for bootloader to select correct board */
qcom,msm-id = <434 0x10000>, <459 0x10000>;
@@ -52,6 +53,27 @@ volume-up {
gpios = <&pm6350_gpios 2 GPIO_ACTIVE_LOW>;
};
};
+
+ reserved-memory {
+ /*
+ * The rmtfs memory region in downstream is 'dynamically allocated'
+ * but given the same address every time. Hard code it as this address is
+ * where the modem firmware expects it to be.
+ */
+ memory@efe01000 {
+ compatible = "qcom,rmtfs-mem";
+ reg = <0 0xefe01000 0 0x600000>;
+ no-map;
+
+ qcom,client-id = <1>;
+ qcom,vmid = <15>;
+ };
+ };
+};
+
+&adsp {
+ status = "okay";
+ firmware-name = "qcom/sm7225/fairphone4/adsp.mdt";
};
&apps_rsc {
@@ -268,6 +290,16 @@ vreg_bob: bob {
};
};
+&cdsp {
+ status = "okay";
+ firmware-name = "qcom/sm7225/fairphone4/cdsp.mdt";
+};
+
+&mpss {
+ status = "okay";
+ firmware-name = "qcom/sm7225/fairphone4/modem.mdt";
+};
+
&pm6350_gpios {
gpio_keys_pin: gpio-keys-pin {
pins = "gpio2";
diff --git a/arch/arm64/boot/dts/qcom/sm8150.dtsi b/arch/arm64/boot/dts/qcom/sm8150.dtsi
index 81b4ff2..6012322 100644
--- a/arch/arm64/boot/dts/qcom/sm8150.dtsi
+++ b/arch/arm64/boot/dts/qcom/sm8150.dtsi
@@ -1785,7 +1785,6 @@ gpu: gpu@2c00000 {
compatible = "qcom,adreno-640.1",
"qcom,adreno",
"amd,imageon";
- #stream-id-cells = <16>;
reg = <0 0x02c00000 0 0x40000>;
reg-names = "kgsl_3d0_reg_memory";
diff --git a/arch/arm64/boot/dts/qcom/sm8250-mtp.dts b/arch/arm64/boot/dts/qcom/sm8250-mtp.dts
index 5ffbcdd..fb99cc2 100644
--- a/arch/arm64/boot/dts/qcom/sm8250-mtp.dts
+++ b/arch/arm64/boot/dts/qcom/sm8250-mtp.dts
@@ -6,6 +6,9 @@
/dts-v1/;
#include <dt-bindings/regulator/qcom,rpmh-regulator.h>
+#include <dt-bindings/sound/qcom,q6afe.h>
+#include <dt-bindings/sound/qcom,q6asm.h>
+#include <dt-bindings/gpio/gpio.h>
#include "sm8250.dtsi"
#include "pm8150.dtsi"
#include "pm8150b.dtsi"
@@ -624,8 +627,196 @@ &slpi {
firmware-name = "qcom/sm8250/slpi.mbn";
};
+&soc {
+ wcd938x: codec {
+ compatible = "qcom,wcd9380-codec";
+ #sound-dai-cells = <1>;
+ reset-gpios = <&tlmm 32 0>;
+ vdd-buck-supply = <&vreg_s4a_1p8>;
+ vdd-rxtx-supply = <&vreg_s4a_1p8>;
+ vdd-io-supply = <&vreg_s4a_1p8>;
+ vdd-mic-bias-supply = <&vreg_bob>;
+ qcom,micbias1-microvolt = <1800000>;
+ qcom,micbias2-microvolt = <1800000>;
+ qcom,micbias3-microvolt = <1800000>;
+ qcom,micbias4-microvolt = <1800000>;
+ qcom,mbhc-buttons-vthreshold-microvolt = <75000 150000 237000 500000 500000 500000 500000 500000>;
+ qcom,mbhc-headset-vthreshold-microvolt = <1700000>;
+ qcom,mbhc-headphone-vthreshold-microvolt = <50000>;
+ qcom,rx-device = <&wcd_rx>;
+ qcom,tx-device = <&wcd_tx>;
+ };
+};
+
+&sound {
+ compatible = "qcom,sm8250-sndcard";
+ model = "SM8250-MTP-WCD9380-WSA8810-VA-DMIC";
+ audio-routing =
+ "SpkrLeft IN", "WSA_SPK1 OUT",
+ "SpkrRight IN", "WSA_SPK2 OUT",
+ "IN1_HPHL", "HPHL_OUT",
+ "IN2_HPHR", "HPHR_OUT",
+ "AMIC1", "MIC BIAS1",
+ "AMIC2", "MIC BIAS2",
+ "AMIC3", "MIC BIAS3",
+ "AMIC4", "MIC BIAS3",
+ "AMIC5", "MIC BIAS4",
+ "TX SWR_ADC0", "ADC1_OUTPUT",
+ "TX SWR_ADC1", "ADC2_OUTPUT",
+ "TX SWR_ADC2", "ADC3_OUTPUT",
+ "TX SWR_ADC3", "ADC4_OUTPUT",
+ "TX SWR_DMIC0", "DMIC1_OUTPUT",
+ "TX SWR_DMIC1", "DMIC2_OUTPUT",
+ "TX SWR_DMIC2", "DMIC3_OUTPUT",
+ "TX SWR_DMIC3", "DMIC4_OUTPUT",
+ "TX SWR_DMIC4", "DMIC5_OUTPUT",
+ "TX SWR_DMIC5", "DMIC6_OUTPUT",
+ "TX SWR_DMIC6", "DMIC7_OUTPUT",
+ "TX SWR_DMIC7", "DMIC8_OUTPUT";
+
+ mm1-dai-link {
+ link-name = "MultiMedia1";
+ cpu {
+ sound-dai = <&q6asmdai MSM_FRONTEND_DAI_MULTIMEDIA1>;
+ };
+ };
+
+ mm2-dai-link {
+ link-name = "MultiMedia2";
+ cpu {
+ sound-dai = <&q6asmdai MSM_FRONTEND_DAI_MULTIMEDIA2>;
+ };
+ };
+
+ mm3-dai-link {
+ link-name = "MultiMedia3";
+ cpu {
+ sound-dai = <&q6asmdai MSM_FRONTEND_DAI_MULTIMEDIA3>;
+ };
+ };
+
+ wcd-playback-dai-link {
+ link-name = "WCD Playback";
+ cpu {
+ sound-dai = <&q6afedai RX_CODEC_DMA_RX_0>;
+ };
+ codec {
+ sound-dai = <&wcd938x 0>, <&swr1 0>, <&rxmacro 0>;
+ };
+ platform {
+ sound-dai = <&q6routing>;
+ };
+ };
+
+ wcd-capture-dai-link {
+ link-name = "WCD Capture";
+ cpu {
+ sound-dai = <&q6afedai TX_CODEC_DMA_TX_3>;
+ };
+
+ codec {
+ sound-dai = <&wcd938x 1>, <&swr2 0>, <&txmacro 0>;
+ };
+ platform {
+ sound-dai = <&q6routing>;
+ };
+ };
+
+ wsa-dai-link {
+ link-name = "WSA Playback";
+ cpu {
+ sound-dai = <&q6afedai WSA_CODEC_DMA_RX_0>;
+ };
+
+ codec {
+ sound-dai = <&left_spkr>, <&right_spkr>, <&swr0 0>, <&wsamacro 0>;
+ };
+ platform {
+ sound-dai = <&q6routing>;
+ };
+ };
+
+ va-dai-link {
+ link-name = "VA Capture";
+ cpu {
+ sound-dai = <&q6afedai VA_CODEC_DMA_TX_0>;
+ };
+
+ platform {
+ sound-dai = <&q6routing>;
+ };
+
+ codec {
+ sound-dai = <&vamacro 0>;
+ };
+ };
+};
+
+&swr0 {
+ left_spkr: wsa8810-right@0,3{
+ compatible = "sdw10217211000";
+ reg = <0 3>;
+ powerdown-gpios = <&tlmm 26 GPIO_ACTIVE_HIGH>;
+ #thermal-sensor-cells = <0>;
+ sound-name-prefix = "SpkrLeft";
+ #sound-dai-cells = <0>;
+ };
+
+ right_spkr: wsa8810-left@0,4{
+ compatible = "sdw10217211000";
+ reg = <0 4>;
+ powerdown-gpios = <&tlmm 127 GPIO_ACTIVE_HIGH>;
+ #thermal-sensor-cells = <0>;
+ sound-name-prefix = "SpkrRight";
+ #sound-dai-cells = <0>;
+ };
+};
+
+&swr1 {
+ wcd_rx: wcd9380-rx@0,4 {
+ compatible = "sdw20217010d00";
+ reg = <0 4>;
+ qcom,rx-port-mapping = <1 2 3 4 5>;
+ };
+};
+
+&swr2 {
+ wcd_tx: wcd9380-tx@0,3 {
+ compatible = "sdw20217010d00";
+ reg = <0 3>;
+ qcom,tx-port-mapping = <2 3 4 5>;
+ };
+};
+
&tlmm {
gpio-reserved-ranges = <28 4>, <40 4>;
+
+ wcd938x_reset_default: wcd938x_reset_default {
+ mux {
+ pins = "gpio32";
+ function = "gpio";
+ };
+
+ config {
+ pins = "gpio32";
+ drive-strength = <16>;
+ output-high;
+ };
+ };
+
+ wcd938x_reset_sleep: wcd938x_reset_sleep {
+ mux {
+ pins = "gpio32";
+ function = "gpio";
+ };
+
+ config {
+ pins = "gpio32";
+ drive-strength = <16>;
+ bias-disable;
+ output-low;
+ };
+ };
};
&uart12 {
diff --git a/arch/arm64/boot/dts/qcom/sm8250.dtsi b/arch/arm64/boot/dts/qcom/sm8250.dtsi
index 6f6129b..5617a46 100644
--- a/arch/arm64/boot/dts/qcom/sm8250.dtsi
+++ b/arch/arm64/boot/dts/qcom/sm8250.dtsi
@@ -99,6 +99,9 @@ CPU0: cpu@0 {
dynamic-power-coefficient = <205>;
next-level-cache = <&L2_0>;
qcom,freq-domain = <&cpufreq_hw 0>;
+ operating-points-v2 = <&cpu0_opp_table>;
+ interconnects = <&gem_noc MASTER_AMPSS_M0 &mc_virt SLAVE_EBI_CH0>,
+ <&epss_l3 MASTER_OSM_L3_APPS &epss_l3 SLAVE_OSM_L3>;
#cooling-cells = <2>;
L2_0: l2-cache {
compatible = "cache";
@@ -118,6 +121,9 @@ CPU1: cpu@100 {
dynamic-power-coefficient = <205>;
next-level-cache = <&L2_100>;
qcom,freq-domain = <&cpufreq_hw 0>;
+ operating-points-v2 = <&cpu0_opp_table>;
+ interconnects = <&gem_noc MASTER_AMPSS_M0 &mc_virt SLAVE_EBI_CH0>,
+ <&epss_l3 MASTER_OSM_L3_APPS &epss_l3 SLAVE_OSM_L3>;
#cooling-cells = <2>;
L2_100: l2-cache {
compatible = "cache";
@@ -134,6 +140,9 @@ CPU2: cpu@200 {
dynamic-power-coefficient = <205>;
next-level-cache = <&L2_200>;
qcom,freq-domain = <&cpufreq_hw 0>;
+ operating-points-v2 = <&cpu0_opp_table>;
+ interconnects = <&gem_noc MASTER_AMPSS_M0 &mc_virt SLAVE_EBI_CH0>,
+ <&epss_l3 MASTER_OSM_L3_APPS &epss_l3 SLAVE_OSM_L3>;
#cooling-cells = <2>;
L2_200: l2-cache {
compatible = "cache";
@@ -150,6 +159,9 @@ CPU3: cpu@300 {
dynamic-power-coefficient = <205>;
next-level-cache = <&L2_300>;
qcom,freq-domain = <&cpufreq_hw 0>;
+ operating-points-v2 = <&cpu0_opp_table>;
+ interconnects = <&gem_noc MASTER_AMPSS_M0 &mc_virt SLAVE_EBI_CH0>,
+ <&epss_l3 MASTER_OSM_L3_APPS &epss_l3 SLAVE_OSM_L3>;
#cooling-cells = <2>;
L2_300: l2-cache {
compatible = "cache";
@@ -166,6 +178,9 @@ CPU4: cpu@400 {
dynamic-power-coefficient = <379>;
next-level-cache = <&L2_400>;
qcom,freq-domain = <&cpufreq_hw 1>;
+ operating-points-v2 = <&cpu4_opp_table>;
+ interconnects = <&gem_noc MASTER_AMPSS_M0 &mc_virt SLAVE_EBI_CH0>,
+ <&epss_l3 MASTER_OSM_L3_APPS &epss_l3 SLAVE_OSM_L3>;
#cooling-cells = <2>;
L2_400: l2-cache {
compatible = "cache";
@@ -182,6 +197,9 @@ CPU5: cpu@500 {
dynamic-power-coefficient = <379>;
next-level-cache = <&L2_500>;
qcom,freq-domain = <&cpufreq_hw 1>;
+ operating-points-v2 = <&cpu4_opp_table>;
+ interconnects = <&gem_noc MASTER_AMPSS_M0 &mc_virt SLAVE_EBI_CH0>,
+ <&epss_l3 MASTER_OSM_L3_APPS &epss_l3 SLAVE_OSM_L3>;
#cooling-cells = <2>;
L2_500: l2-cache {
compatible = "cache";
@@ -199,6 +217,9 @@ CPU6: cpu@600 {
dynamic-power-coefficient = <379>;
next-level-cache = <&L2_600>;
qcom,freq-domain = <&cpufreq_hw 1>;
+ operating-points-v2 = <&cpu4_opp_table>;
+ interconnects = <&gem_noc MASTER_AMPSS_M0 &mc_virt SLAVE_EBI_CH0>,
+ <&epss_l3 MASTER_OSM_L3_APPS &epss_l3 SLAVE_OSM_L3>;
#cooling-cells = <2>;
L2_600: l2-cache {
compatible = "cache";
@@ -215,6 +236,9 @@ CPU7: cpu@700 {
dynamic-power-coefficient = <444>;
next-level-cache = <&L2_700>;
qcom,freq-domain = <&cpufreq_hw 2>;
+ operating-points-v2 = <&cpu7_opp_table>;
+ interconnects = <&gem_noc MASTER_AMPSS_M0 &mc_virt SLAVE_EBI_CH0>,
+ <&epss_l3 MASTER_OSM_L3_APPS &epss_l3 SLAVE_OSM_L3>;
#cooling-cells = <2>;
L2_700: l2-cache {
compatible = "cache";
@@ -259,6 +283,296 @@ core7 {
};
};
+ cpu0_opp_table: cpu0_opp_table {
+ compatible = "operating-points-v2";
+ opp-shared;
+
+ cpu0_opp1: opp-300000000 {
+ opp-hz = /bits/ 64 <300000000>;
+ opp-peak-kBps = <800000 9600000>;
+ };
+
+ cpu0_opp2: opp-403200000 {
+ opp-hz = /bits/ 64 <403200000>;
+ opp-peak-kBps = <800000 9600000>;
+ };
+
+ cpu0_opp3: opp-518400000 {
+ opp-hz = /bits/ 64 <518400000>;
+ opp-peak-kBps = <800000 16588800>;
+ };
+
+ cpu0_opp4: opp-614400000 {
+ opp-hz = /bits/ 64 <614400000>;
+ opp-peak-kBps = <800000 16588800>;
+ };
+
+ cpu0_opp5: opp-691200000 {
+ opp-hz = /bits/ 64 <691200000>;
+ opp-peak-kBps = <800000 19660800>;
+ };
+
+ cpu0_opp6: opp-787200000 {
+ opp-hz = /bits/ 64 <787200000>;
+ opp-peak-kBps = <1804000 19660800>;
+ };
+
+ cpu0_opp7: opp-883200000 {
+ opp-hz = /bits/ 64 <883200000>;
+ opp-peak-kBps = <1804000 23347200>;
+ };
+
+ cpu0_opp8: opp-979200000 {
+ opp-hz = /bits/ 64 <979200000>;
+ opp-peak-kBps = <1804000 26419200>;
+ };
+
+ cpu0_opp9: opp-1075200000 {
+ opp-hz = /bits/ 64 <1075200000>;
+ opp-peak-kBps = <1804000 29491200>;
+ };
+
+ cpu0_opp10: opp-1171200000 {
+ opp-hz = /bits/ 64 <1171200000>;
+ opp-peak-kBps = <1804000 32563200>;
+ };
+
+ cpu0_opp11: opp-1248000000 {
+ opp-hz = /bits/ 64 <1248000000>;
+ opp-peak-kBps = <1804000 36249600>;
+ };
+
+ cpu0_opp12: opp-1344000000 {
+ opp-hz = /bits/ 64 <1344000000>;
+ opp-peak-kBps = <2188000 36249600>;
+ };
+
+ cpu0_opp13: opp-1420800000 {
+ opp-hz = /bits/ 64 <1420800000>;
+ opp-peak-kBps = <2188000 39321600>;
+ };
+
+ cpu0_opp14: opp-1516800000 {
+ opp-hz = /bits/ 64 <1516800000>;
+ opp-peak-kBps = <3072000 42393600>;
+ };
+
+ cpu0_opp15: opp-1612800000 {
+ opp-hz = /bits/ 64 <1612800000>;
+ opp-peak-kBps = <3072000 42393600>;
+ };
+
+ cpu0_opp16: opp-1708800000 {
+ opp-hz = /bits/ 64 <1708800000>;
+ opp-peak-kBps = <4068000 42393600>;
+ };
+
+ cpu0_opp17: opp-1804800000 {
+ opp-hz = /bits/ 64 <1804800000>;
+ opp-peak-kBps = <4068000 42393600>;
+ };
+ };
+
+ cpu4_opp_table: cpu4_opp_table {
+ compatible = "operating-points-v2";
+ opp-shared;
+
+ cpu4_opp1: opp-710400000 {
+ opp-hz = /bits/ 64 <710400000>;
+ opp-peak-kBps = <1804000 19660800>;
+ };
+
+ cpu4_opp2: opp-825600000 {
+ opp-hz = /bits/ 64 <825600000>;
+ opp-peak-kBps = <2188000 23347200>;
+ };
+
+ cpu4_opp3: opp-940800000 {
+ opp-hz = /bits/ 64 <940800000>;
+ opp-peak-kBps = <2188000 26419200>;
+ };
+
+ cpu4_opp4: opp-1056000000 {
+ opp-hz = /bits/ 64 <1056000000>;
+ opp-peak-kBps = <3072000 26419200>;
+ };
+
+ cpu4_opp5: opp-1171200000 {
+ opp-hz = /bits/ 64 <1171200000>;
+ opp-peak-kBps = <3072000 29491200>;
+ };
+
+ cpu4_opp6: opp-1286400000 {
+ opp-hz = /bits/ 64 <1286400000>;
+ opp-peak-kBps = <4068000 29491200>;
+ };
+
+ cpu4_opp7: opp-1382400000 {
+ opp-hz = /bits/ 64 <1382400000>;
+ opp-peak-kBps = <4068000 32563200>;
+ };
+
+ cpu4_opp8: opp-1478400000 {
+ opp-hz = /bits/ 64 <1478400000>;
+ opp-peak-kBps = <4068000 32563200>;
+ };
+
+ cpu4_opp9: opp-1574400000 {
+ opp-hz = /bits/ 64 <1574400000>;
+ opp-peak-kBps = <5412000 39321600>;
+ };
+
+ cpu4_opp10: opp-1670400000 {
+ opp-hz = /bits/ 64 <1670400000>;
+ opp-peak-kBps = <5412000 42393600>;
+ };
+
+ cpu4_opp11: opp-1766400000 {
+ opp-hz = /bits/ 64 <1766400000>;
+ opp-peak-kBps = <5412000 45465600>;
+ };
+
+ cpu4_opp12: opp-1862400000 {
+ opp-hz = /bits/ 64 <1862400000>;
+ opp-peak-kBps = <6220000 45465600>;
+ };
+
+ cpu4_opp13: opp-1958400000 {
+ opp-hz = /bits/ 64 <1958400000>;
+ opp-peak-kBps = <6220000 48537600>;
+ };
+
+ cpu4_opp14: opp-2054400000 {
+ opp-hz = /bits/ 64 <2054400000>;
+ opp-peak-kBps = <7216000 48537600>;
+ };
+
+ cpu4_opp15: opp-2150400000 {
+ opp-hz = /bits/ 64 <2150400000>;
+ opp-peak-kBps = <7216000 51609600>;
+ };
+
+ cpu4_opp16: opp-2246400000 {
+ opp-hz = /bits/ 64 <2246400000>;
+ opp-peak-kBps = <7216000 51609600>;
+ };
+
+ cpu4_opp17: opp-2342400000 {
+ opp-hz = /bits/ 64 <2342400000>;
+ opp-peak-kBps = <8368000 51609600>;
+ };
+
+ cpu4_opp18: opp-2419200000 {
+ opp-hz = /bits/ 64 <2419200000>;
+ opp-peak-kBps = <8368000 51609600>;
+ };
+ };
+
+ cpu7_opp_table: cpu7_opp_table {
+ compatible = "operating-points-v2";
+ opp-shared;
+
+ cpu7_opp1: opp-844800000 {
+ opp-hz = /bits/ 64 <844800000>;
+ opp-peak-kBps = <2188000 19660800>;
+ };
+
+ cpu7_opp2: opp-960000000 {
+ opp-hz = /bits/ 64 <960000000>;
+ opp-peak-kBps = <2188000 26419200>;
+ };
+
+ cpu7_opp3: opp-1075200000 {
+ opp-hz = /bits/ 64 <1075200000>;
+ opp-peak-kBps = <3072000 26419200>;
+ };
+
+ cpu7_opp4: opp-1190400000 {
+ opp-hz = /bits/ 64 <1190400000>;
+ opp-peak-kBps = <3072000 29491200>;
+ };
+
+ cpu7_opp5: opp-1305600000 {
+ opp-hz = /bits/ 64 <1305600000>;
+ opp-peak-kBps = <4068000 32563200>;
+ };
+
+ cpu7_opp6: opp-1401600000 {
+ opp-hz = /bits/ 64 <1401600000>;
+ opp-peak-kBps = <4068000 32563200>;
+ };
+
+ cpu7_opp7: opp-1516800000 {
+ opp-hz = /bits/ 64 <1516800000>;
+ opp-peak-kBps = <4068000 36249600>;
+ };
+
+ cpu7_opp8: opp-1632000000 {
+ opp-hz = /bits/ 64 <1632000000>;
+ opp-peak-kBps = <5412000 39321600>;
+ };
+
+ cpu7_opp9: opp-1747200000 {
+ opp-hz = /bits/ 64 <1708800000>;
+ opp-peak-kBps = <5412000 42393600>;
+ };
+
+ cpu7_opp10: opp-1862400000 {
+ opp-hz = /bits/ 64 <1862400000>;
+ opp-peak-kBps = <6220000 45465600>;
+ };
+
+ cpu7_opp11: opp-1977600000 {
+ opp-hz = /bits/ 64 <1977600000>;
+ opp-peak-kBps = <6220000 48537600>;
+ };
+
+ cpu7_opp12: opp-2073600000 {
+ opp-hz = /bits/ 64 <2073600000>;
+ opp-peak-kBps = <7216000 48537600>;
+ };
+
+ cpu7_opp13: opp-2169600000 {
+ opp-hz = /bits/ 64 <2169600000>;
+ opp-peak-kBps = <7216000 51609600>;
+ };
+
+ cpu7_opp14: opp-2265600000 {
+ opp-hz = /bits/ 64 <2265600000>;
+ opp-peak-kBps = <7216000 51609600>;
+ };
+
+ cpu7_opp15: opp-2361600000 {
+ opp-hz = /bits/ 64 <2361600000>;
+ opp-peak-kBps = <8368000 51609600>;
+ };
+
+ cpu7_opp16: opp-2457600000 {
+ opp-hz = /bits/ 64 <2457600000>;
+ opp-peak-kBps = <8368000 51609600>;
+ };
+
+ cpu7_opp17: opp-2553600000 {
+ opp-hz = /bits/ 64 <2553600000>;
+ opp-peak-kBps = <8368000 51609600>;
+ };
+
+ cpu7_opp18: opp-2649600000 {
+ opp-hz = /bits/ 64 <2649600000>;
+ opp-peak-kBps = <8368000 51609600>;
+ };
+
+ cpu7_opp19: opp-2745600000 {
+ opp-hz = /bits/ 64 <2745600000>;
+ opp-peak-kBps = <8368000 51609600>;
+ };
+
+ cpu7_opp20: opp-2841600000 {
+ opp-hz = /bits/ 64 <2841600000>;
+ opp-peak-kBps = <8368000 51609600>;
+ };
+ };
+
firmware {
scm: scm {
compatible = "qcom,scm";
@@ -1831,6 +2145,101 @@ vamacro: codec@3370000 {
#sound-dai-cells = <1>;
};
+ rxmacro: rxmacro@3200000 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&rx_swr_active>;
+ compatible = "qcom,sm8250-lpass-rx-macro";
+ reg = <0 0x3200000 0 0x1000>;
+
+ clocks = <&q6afecc LPASS_CLK_ID_TX_CORE_MCLK LPASS_CLK_ATTRIBUTE_COUPLE_NO>,
+ <&q6afecc LPASS_CLK_ID_TX_CORE_NPL_MCLK LPASS_CLK_ATTRIBUTE_COUPLE_NO>,
+ <&q6afecc LPASS_HW_MACRO_VOTE LPASS_CLK_ATTRIBUTE_COUPLE_NO>,
+ <&q6afecc LPASS_HW_DCODEC_VOTE LPASS_CLK_ATTRIBUTE_COUPLE_NO>,
+ <&vamacro>;
+
+ clock-names = "mclk", "npl", "macro", "dcodec", "fsgen";
+
+ #clock-cells = <0>;
+ clock-frequency = <9600000>;
+ clock-output-names = "mclk";
+ #sound-dai-cells = <1>;
+ };
+
+ swr1: soundwire-controller@3210000 {
+ reg = <0 0x3210000 0 0x2000>;
+ compatible = "qcom,soundwire-v1.5.1";
+ interrupts = <GIC_SPI 298 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&rxmacro>;
+ clock-names = "iface";
+ label = "RX";
+ qcom,din-ports = <0>;
+ qcom,dout-ports = <5>;
+
+ qcom,ports-sinterval-low = /bits/ 8 <0x03 0x1F 0x1F 0x07 0x00>;
+ qcom,ports-offset1 = /bits/ 8 <0x00 0x00 0x0B 0x01 0x00>;
+ qcom,ports-offset2 = /bits/ 8 <0x00 0x00 0x0B 0x00 0x00>;
+ qcom,ports-hstart = /bits/ 8 <0xFF 0x03 0xFF 0xFF 0xFF>;
+ qcom,ports-hstop = /bits/ 8 <0xFF 0x06 0xFF 0xFF 0xFF>;
+ qcom,ports-word-length = /bits/ 8 <0x01 0x07 0x04 0xFF 0xFF>;
+ qcom,ports-block-pack-mode = /bits/ 8 <0xFF 0x00 0x01 0xFF 0xFF>;
+ qcom,ports-lane-control = /bits/ 8 <0x01 0x00 0x00 0x00 0x00>;
+ qcom,ports-block-group-count = /bits/ 8 <0xFF 0xFF 0xFF 0xFF 0x00>;
+
+ #sound-dai-cells = <1>;
+ #address-cells = <2>;
+ #size-cells = <0>;
+ };
+
+ txmacro: txmacro@3220000 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&tx_swr_active>;
+ compatible = "qcom,sm8250-lpass-tx-macro";
+ reg = <0 0x3220000 0 0x1000>;
+
+ clocks = <&q6afecc LPASS_CLK_ID_TX_CORE_MCLK LPASS_CLK_ATTRIBUTE_COUPLE_NO>,
+ <&q6afecc LPASS_CLK_ID_TX_CORE_NPL_MCLK LPASS_CLK_ATTRIBUTE_COUPLE_NO>,
+ <&q6afecc LPASS_HW_MACRO_VOTE LPASS_CLK_ATTRIBUTE_COUPLE_NO>,
+ <&q6afecc LPASS_HW_DCODEC_VOTE LPASS_CLK_ATTRIBUTE_COUPLE_NO>,
+ <&vamacro>;
+
+ clock-names = "mclk", "npl", "macro", "dcodec", "fsgen";
+
+ #clock-cells = <0>;
+ clock-frequency = <9600000>;
+ clock-output-names = "mclk";
+ #address-cells = <2>;
+ #size-cells = <2>;
+ #sound-dai-cells = <1>;
+ };
+
+ /* tx macro */
+ swr2: soundwire-controller@3230000 {
+ reg = <0 0x3230000 0 0x2000>;
+ compatible = "qcom,soundwire-v1.5.1";
+ interrupts-extended = <&intc GIC_SPI 297 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "core";
+
+ clocks = <&txmacro>;
+ clock-names = "iface";
+ label = "TX";
+
+ qcom,din-ports = <5>;
+ qcom,dout-ports = <0>;
+ qcom,ports-sinterval-low = /bits/ 8 <0xFF 0x01 0x01 0x03 0x03>;
+ qcom,ports-offset1 = /bits/ 8 <0xFF 0x01 0x00 0x02 0x00>;
+ qcom,ports-offset2 = /bits/ 8 <0xFF 0x00 0x00 0x00 0x00>;
+ qcom,ports-block-pack-mode = /bits/ 8 <0xFF 0xFF 0xFF 0xFF 0xFF>;
+ qcom,ports-hstart = /bits/ 8 <0xFF 0xFF 0xFF 0xFF 0xFF>;
+ qcom,ports-hstop = /bits/ 8 <0xFF 0xFF 0xFF 0xFF 0xFF>;
+ qcom,ports-word-length = /bits/ 8 <0xFF 0xFF 0xFF 0xFF 0xFF>;
+ qcom,ports-block-group-count = /bits/ 8 <0xFF 0xFF 0xFF 0xFF 0xFF>;
+ qcom,ports-lane-control = /bits/ 8 <0xFF 0x00 0x01 0x00 0x01>;
+ qcom,port-offset = <1>;
+ #sound-dai-cells = <1>;
+ #address-cells = <2>;
+ #size-cells = <0>;
+ };
+
aoncc: clock-controller@3380000 {
compatible = "qcom,sm8250-lpass-aoncc";
reg = <0 0x03380000 0 0x40000>;
@@ -1923,12 +2332,73 @@ data {
input-enable;
};
};
+
+ rx_swr_active: rx_swr-active-pins {
+ clk {
+ pins = "gpio3";
+ function = "swr_rx_clk";
+ drive-strength = <2>;
+ slew-rate = <1>;
+ bias-disable;
+ };
+
+ data {
+ pins = "gpio4", "gpio5";
+ function = "swr_rx_data";
+ drive-strength = <2>;
+ slew-rate = <1>;
+ bias-bus-hold;
+ };
+ };
+
+ tx_swr_active: tx_swr-active-pins {
+ clk {
+ pins = "gpio0";
+ function = "swr_tx_clk";
+ drive-strength = <2>;
+ slew-rate = <1>;
+ bias-disable;
+ };
+
+ data {
+ pins = "gpio1", "gpio2";
+ function = "swr_tx_data";
+ drive-strength = <2>;
+ slew-rate = <1>;
+ bias-bus-hold;
+ };
+ };
+
+ tx_swr_sleep: tx_swr-sleep-pins {
+ clk {
+ pins = "gpio0";
+ function = "swr_tx_clk";
+ drive-strength = <2>;
+ input-enable;
+ bias-pull-down;
+ };
+
+ data1 {
+ pins = "gpio1";
+ function = "swr_tx_data";
+ drive-strength = <2>;
+ input-enable;
+ bias-bus-hold;
+ };
+
+ data2 {
+ pins = "gpio2";
+ function = "swr_tx_data";
+ drive-strength = <2>;
+ input-enable;
+ bias-pull-down;
+ };
+ };
};
gpu: gpu@3d00000 {
compatible = "qcom,adreno-650.2",
"qcom,adreno";
- #stream-id-cells = <16>;
reg = <0 0x03d00000 0 0x40000>;
reg-names = "kgsl_3d0_reg_memory";
diff --git a/arch/arm64/boot/dts/qcom/sm8350-hdk.dts b/arch/arm64/boot/dts/qcom/sm8350-hdk.dts
index be06237..1e5e940 100644
--- a/arch/arm64/boot/dts/qcom/sm8350-hdk.dts
+++ b/arch/arm64/boot/dts/qcom/sm8350-hdk.dts
@@ -5,7 +5,6 @@
/dts-v1/;
-#include <dt-bindings/gpio/gpio.h>
#include <dt-bindings/regulator/qcom,rpmh-regulator.h>
#include "sm8350.dtsi"
diff --git a/arch/arm64/boot/dts/qcom/sm8350-microsoft-surface-duo2.dts b/arch/arm64/boot/dts/qcom/sm8350-microsoft-surface-duo2.dts
new file mode 100644
index 0000000..9cb1d84
--- /dev/null
+++ b/arch/arm64/boot/dts/qcom/sm8350-microsoft-surface-duo2.dts
@@ -0,0 +1,369 @@
+// SPDX-License-Identifier: BSD-3-Clause
+/*
+ * Copyright (C) 2021, Microsoft Corporation
+ */
+
+/dts-v1/;
+
+#include <dt-bindings/regulator/qcom,rpmh-regulator.h>
+#include "sm8350.dtsi"
+#include "pm8350.dtsi"
+#include "pm8350b.dtsi"
+#include "pm8350c.dtsi"
+#include "pmk8350.dtsi"
+#include "pmr735a.dtsi"
+#include "pmr735b.dtsi"
+
+/ {
+ model = "Microsoft Surface Duo 2";
+ compatible = "microsoft,surface-duo2", "qcom,sm8350";
+ chassis-type = "handset";
+
+ aliases {
+ serial0 = &uart2;
+ };
+
+ chosen {
+ stdout-path = "serial0:115200n8";
+ };
+
+ vph_pwr: vph-pwr-regulator {
+ compatible = "regulator-fixed";
+ regulator-name = "vph_pwr";
+ regulator-min-microvolt = <3700000>;
+ regulator-max-microvolt = <3700000>;
+
+ regulator-always-on;
+ regulator-boot-on;
+ };
+};
+
+&adsp {
+ status = "okay";
+ firmware-name = "qcom/sm8350/microsoft/adsp.mbn";
+};
+
+&apps_rsc {
+ pm8350-rpmh-regulators {
+ compatible = "qcom,pm8350-rpmh-regulators";
+ qcom,pmic-id = "b";
+
+ vdd-s1-supply = <&vph_pwr>;
+ vdd-s2-supply = <&vph_pwr>;
+ vdd-s3-supply = <&vph_pwr>;
+ vdd-s4-supply = <&vph_pwr>;
+ vdd-s5-supply = <&vph_pwr>;
+ vdd-s6-supply = <&vph_pwr>;
+ vdd-s7-supply = <&vph_pwr>;
+ vdd-s8-supply = <&vph_pwr>;
+ vdd-s9-supply = <&vph_pwr>;
+ vdd-s10-supply = <&vph_pwr>;
+ vdd-s11-supply = <&vph_pwr>;
+ vdd-s12-supply = <&vph_pwr>;
+
+ vdd-l1-l4-supply = <&vreg_s11b_0p95>;
+ vdd-l2-l7-supply = <&vreg_bob>;
+ vdd-l3-l5-supply = <&vreg_bob>;
+ vdd-l6-l9-l10-supply = <&vreg_s11b_0p95>;
+ vdd-l8-supply = <&vreg_s2c_0p8>;
+
+ vreg_s10b_1p8: smps10 {
+ regulator-name = "vreg_s10b_1p8";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ };
+
+ vreg_s11b_0p95: smps11 {
+ regulator-name = "vreg_s11b_0p95";
+ regulator-min-microvolt = <752000>;
+ regulator-max-microvolt = <1000000>;
+ };
+
+ vreg_s12b_1p25: smps12 {
+ regulator-name = "vreg_s12b_1p25";
+ regulator-min-microvolt = <1224000>;
+ regulator-max-microvolt = <1360000>;
+ };
+
+ vreg_l1b_0p88: ldo1 {
+ regulator-name = "vreg_l1b_0p88";
+ regulator-min-microvolt = <912000>;
+ regulator-max-microvolt = <920000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l2b_3p07: ldo2 {
+ regulator-name = "vreg_l2b_3p07";
+ regulator-min-microvolt = <3072000>;
+ regulator-max-microvolt = <3072000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l3b_0p9: ldo3 {
+ regulator-name = "vreg_l3b_0p9";
+ regulator-min-microvolt = <904000>;
+ regulator-max-microvolt = <904000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l5b_0p88: ldo5 {
+ regulator-name = "vreg_l3b_0p9";
+ regulator-min-microvolt = <880000>;
+ regulator-max-microvolt = <888000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l6b_1p2: ldo6 {
+ regulator-name = "vreg_l6b_1p2";
+ regulator-min-microvolt = <1200000>;
+ regulator-max-microvolt = <1208000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l7b_2p96: ldo7 {
+ regulator-name = "vreg_l7b_2p96";
+ regulator-min-microvolt = <2400000>;
+ regulator-max-microvolt = <3008000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l9b_1p2: ldo9 {
+ regulator-name = "vreg_l9b_1p2";
+ regulator-min-microvolt = <1200000>;
+ regulator-max-microvolt = <1200000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+ };
+
+ pm8350c-rpmh-regulators {
+ compatible = "qcom,pm8350c-rpmh-regulators";
+ qcom,pmic-id = "c";
+
+ vdd-s1-supply = <&vph_pwr>;
+ vdd-s2-supply = <&vph_pwr>;
+ vdd-s3-supply = <&vph_pwr>;
+ vdd-s4-supply = <&vph_pwr>;
+ vdd-s5-supply = <&vph_pwr>;
+ vdd-s6-supply = <&vph_pwr>;
+ vdd-s7-supply = <&vph_pwr>;
+ vdd-s8-supply = <&vph_pwr>;
+ vdd-s9-supply = <&vph_pwr>;
+ vdd-s10-supply = <&vph_pwr>;
+
+ vdd-l1-l12-supply = <&vreg_s1c_1p86>;
+ vdd-l2-l8-supply = <&vreg_s1c_1p86>;
+ vdd-l3-l4-l5-l7-l13-supply = <&vreg_bob>;
+ vdd-l6-l9-l11-supply = <&vreg_bob>;
+ vdd-l10-supply = <&vreg_s12b_1p25>;
+
+ vdd-bob-supply = <&vph_pwr>;
+
+ vreg_s1c_1p86: smps1 {
+ regulator-name = "vreg_s1c_1p86";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1952000>;
+ };
+
+ vreg_s2c_0p8: smps2 {
+ regulator-name = "vreg_s2c_0p8";
+ regulator-min-microvolt = <640000>;
+ regulator-max-microvolt = <1000000>;
+ };
+
+ vreg_s10c_1p05: smps10 {
+ regulator-name = "vreg_s10c_1p05";
+ regulator-min-microvolt = <1048000>;
+ regulator-max-microvolt = <1128000>;
+ };
+
+ vreg_bob: bob {
+ regulator-name = "vreg_bob";
+ regulator-min-microvolt = <3008000>;
+ regulator-max-microvolt = <3960000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_AUTO>;
+ };
+
+ vreg_l1c_1p8: ldo1 {
+ regulator-name = "vreg_l1c_1p8";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l2c_1p8: ldo2 {
+ regulator-name = "vreg_l2c_1p8";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l3c_3p0: ldo3 {
+ regulator-name = "vreg_l3c_3p0";
+ regulator-min-microvolt = <3008000>;
+ regulator-max-microvolt = <3008000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l4c_uim1: ldo4 {
+ regulator-name = "vreg_l4c_uim1";
+ regulator-min-microvolt = <1704000>;
+ regulator-max-microvolt = <3000000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l5c_uim2: ldo5 {
+ regulator-name = "vreg_l5c_uim2";
+ regulator-min-microvolt = <1704000>;
+ regulator-max-microvolt = <3000000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l6c_1p8: ldo6 {
+ regulator-name = "vreg_l6c_1p8";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <2960000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l7c_3p0: ldo7 {
+ regulator-name = "vreg_l7c_3p0";
+ regulator-min-microvolt = <3008000>;
+ regulator-max-microvolt = <3008000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l8c_1p8: ldo8 {
+ regulator-name = "vreg_l8c_1p8";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l9c_2p96: ldo9 {
+ regulator-name = "vreg_l9c_2p96";
+ regulator-min-microvolt = <2960000>;
+ regulator-max-microvolt = <3008000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l10c_1p2: ldo10 {
+ regulator-name = "vreg_l10c_1p2";
+ regulator-min-microvolt = <1200000>;
+ regulator-max-microvolt = <1200000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l11c_2p96: ldo11 {
+ regulator-name = "vreg_l11c_2p96";
+ regulator-min-microvolt = <2400000>;
+ regulator-max-microvolt = <3008000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l12c_1p8: ldo12 {
+ regulator-name = "vreg_l12c_1p8";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <2000000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l13c_3p0: ldo13 {
+ regulator-name = "vreg_l13c_3p0";
+ regulator-min-microvolt = <3000000>;
+ regulator-max-microvolt = <3000000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+ };
+};
+
+&cdsp {
+ status = "okay";
+ firmware-name = "qcom/sm8350/microsoft/cdsp.mbn";
+};
+
+&ipa {
+ status = "okay";
+
+ memory-region = <&pil_ipa_fw_mem>;
+};
+
+&mpss {
+ status = "okay";
+ firmware-name = "qcom/sm8350/microsoft/modem.mbn";
+};
+
+&qupv3_id_0 {
+ status = "okay";
+};
+
+&slpi {
+ status = "okay";
+ firmware-name = "qcom/sm8350/microsoft/slpi.mbn";
+};
+
+&tlmm {
+ gpio-reserved-ranges = <4 4>, <12 4>, <56 4>, <76 4>;
+};
+
+&uart2 {
+ status = "okay";
+};
+
+&ufs_mem_hc {
+ status = "okay";
+
+ reset-gpios = <&tlmm 203 GPIO_ACTIVE_LOW>;
+
+ vcc-supply = <&vreg_l7b_2p96>;
+ vcc-max-microamp = <800000>;
+ vccq-supply = <&vreg_l9b_1p2>;
+ vccq-max-microamp = <900000>;
+};
+
+&ufs_mem_phy {
+ status = "okay";
+
+ vdda-phy-supply = <&vreg_l5b_0p88>;
+ vdda-max-microamp = <91600>;
+ vdda-pll-supply = <&vreg_l6b_1p2>;
+ vdda-pll-max-microamp = <19000>;
+};
+
+&usb_1 {
+ status = "okay";
+ dr_mode = "peripheral";
+};
+
+&usb_1_hsphy {
+ status = "okay";
+
+ vdda-pll-supply = <&vreg_l5b_0p88>;
+ vdda18-supply = <&vreg_l1c_1p8>;
+ vdda33-supply = <&vreg_l2b_3p07>;
+};
+
+&usb_1_qmpphy {
+ status = "okay";
+
+ vdda-phy-supply = <&vreg_l6b_1p2>;
+ vdda-pll-supply = <&vreg_l1b_0p88>;
+};
+
+&usb_2 {
+ status = "okay";
+};
+
+&usb_2_hsphy {
+ status = "okay";
+
+ vdda-pll-supply = <&vreg_l5b_0p88>;
+ vdda18-supply = <&vreg_l1c_1p8>;
+ vdda33-supply = <&vreg_l2b_3p07>;
+};
+
+&usb_2_qmpphy {
+ status = "okay";
+
+ vdda-phy-supply = <&vreg_l6b_1p2>;
+ vdda-pll-supply = <&vreg_l5b_0p88>;
+};
diff --git a/arch/arm64/boot/dts/qcom/sm8350-mtp.dts b/arch/arm64/boot/dts/qcom/sm8350-mtp.dts
index 06eedbe..52cf304 100644
--- a/arch/arm64/boot/dts/qcom/sm8350-mtp.dts
+++ b/arch/arm64/boot/dts/qcom/sm8350-mtp.dts
@@ -5,7 +5,6 @@
/dts-v1/;
-#include <dt-bindings/gpio/gpio.h>
#include <dt-bindings/regulator/qcom,rpmh-regulator.h>
#include "sm8350.dtsi"
#include "pm8350.dtsi"
@@ -290,6 +289,15 @@ &pmk8350_rtc {
status = "okay";
};
+&pon_pwrkey {
+ status = "okay";
+};
+
+&pon_resin {
+ status = "okay";
+ linux,code = <KEY_VOLUMEDOWN>;
+};
+
&qupv3_id_0 {
status = "okay";
};
diff --git a/arch/arm64/boot/dts/qcom/sm8350-sony-xperia-sagami-pdx214.dts b/arch/arm64/boot/dts/qcom/sm8350-sony-xperia-sagami-pdx214.dts
new file mode 100644
index 0000000..cc65050
--- /dev/null
+++ b/arch/arm64/boot/dts/qcom/sm8350-sony-xperia-sagami-pdx214.dts
@@ -0,0 +1,19 @@
+// SPDX-License-Identifier: BSD-3-Clause
+/*
+ * Copyright (c) 2021, Konrad Dybcio <konrad.dybcio@somainline.org>
+ */
+
+/dts-v1/;
+
+#include "sm8350-sony-xperia-sagami.dtsi"
+
+/ {
+ model = "Sony Xperia 5 III";
+ compatible = "sony,pdx214-generic", "qcom,sm8350";
+};
+
+&framebuffer {
+ width = <1080>;
+ height = <2520>;
+ stride = <(1080 * 4)>;
+};
diff --git a/arch/arm64/boot/dts/qcom/sm8350-sony-xperia-sagami-pdx215.dts b/arch/arm64/boot/dts/qcom/sm8350-sony-xperia-sagami-pdx215.dts
new file mode 100644
index 0000000..d21bbeb
--- /dev/null
+++ b/arch/arm64/boot/dts/qcom/sm8350-sony-xperia-sagami-pdx215.dts
@@ -0,0 +1,13 @@
+// SPDX-License-Identifier: BSD-3-Clause
+/*
+ * Copyright (c) 2021, Konrad Dybcio <konrad.dybcio@somainline.org>
+ */
+
+/dts-v1/;
+
+#include "sm8350-sony-xperia-sagami.dtsi"
+
+/ {
+ model = "Sony Xperia 1 III";
+ compatible = "sony,pdx215-generic", "qcom,sm8350";
+};
diff --git a/arch/arm64/boot/dts/qcom/sm8350-sony-xperia-sagami.dtsi b/arch/arm64/boot/dts/qcom/sm8350-sony-xperia-sagami.dtsi
new file mode 100644
index 0000000..90b13cb
--- /dev/null
+++ b/arch/arm64/boot/dts/qcom/sm8350-sony-xperia-sagami.dtsi
@@ -0,0 +1,259 @@
+// SPDX-License-Identifier: BSD-3-Clause
+/*
+ * Copyright (c) 2021, Konrad Dybcio <konrad.dybcio@somainline.org>
+ */
+
+#include "sm8350.dtsi"
+#include "pm8350.dtsi"
+#include "pm8350b.dtsi"
+#include "pm8350c.dtsi"
+#include "pmk8350.dtsi"
+#include "pmr735a.dtsi"
+#include "pmr735b.dtsi"
+
+/ {
+ /*
+ * Yes, you are correct, there is NO MORE {msm,board,pmic}-id on SM8350!
+ * Adding it will cause the bootloader to go crazy and randomly crash
+ * shortly after closing UEFI boot services.. Perhaps that has something
+ * to do with the OS running inside a VM now..?
+ */
+
+ chassis-type = "handset";
+
+ chosen {
+ #address-cells = <2>;
+ #size-cells = <2>;
+ ranges;
+
+ framebuffer: framebuffer@e1000000 {
+ compatible = "simple-framebuffer";
+ reg = <0 0xe1000000 0 0x2300000>;
+
+ /* The display, even though it's 4K, initializes at 1080-ish p */
+ width = <1096>;
+ height = <2560>;
+ stride = <(1096 * 4)>;
+ format = "a8r8g8b8";
+ /*
+ * That's (going to be) a lot of clocks, but it's necessary due
+ * to unused clk cleanup & no panel driver yet
+ */
+ clocks = <&gcc GCC_DISP_HF_AXI_CLK>,
+ <&gcc GCC_DISP_SF_AXI_CLK>;
+ };
+ };
+
+ gpio-keys {
+ compatible = "gpio-keys";
+
+ /* For reasons still unknown, GAssist key and Camera Focus/Shutter don't work.. */
+
+ vol-down {
+ label = "Volume Down";
+ linux,code = <KEY_VOLUMEDOWN>;
+ gpios = <&pmk8350_gpios 3 GPIO_ACTIVE_LOW>;
+ debounce-interval = <15>;
+ linux,can-disable;
+ gpio-key,wakeup;
+ };
+ };
+
+ reserved-memory {
+ cont_splash_mem: memory@e1000000 {
+ reg = <0 0xe1000000 0 0x2300000>;
+ no-map;
+ };
+
+ ramoops@ffc00000 {
+ compatible = "ramoops";
+ reg = <0 0xffc00000 0 0x100000>;
+ console-size = <0x40000>;
+ record-size = <0x1000>;
+ no-map;
+ };
+ };
+};
+
+&adsp {
+ status = "okay";
+ firmware-name = "qcom/adsp.mbn";
+};
+
+&cdsp {
+ status = "okay";
+ firmware-name = "qcom/cdsp.mbn";
+};
+
+&i2c1 {
+ status = "okay";
+ clock-frequency = <1000000>;
+
+ /* Some subset of SONY IMX663 camera sensor @ 38 */
+};
+
+&i2c2 {
+ status = "okay";
+ clock-frequency = <400000>;
+
+ /* Richwave RTC6226 FM Radio Receiver @ 64 */
+};
+
+&i2c4 {
+ status = "okay";
+ clock-frequency = <400000>;
+
+ /* Samsung Touchscreen (needs I2C GPI DMA) @ 48 */
+};
+
+&i2c11 {
+ status = "okay";
+ clock-frequency = <1000000>;
+
+ cs35l41_l: cs35l41@40 {
+ compatible = "cirrus,cs35l41";
+ reg = <0x40>;
+ interrupt-parent = <&tlmm>;
+ interrupts = <36 IRQ_TYPE_LEVEL_LOW>;
+ reset-gpios = <&tlmm 37 GPIO_ACTIVE_HIGH>;
+ cirrus,boost-peak-milliamp = <4000>;
+ cirrus,boost-ind-nanohenry = <1000>;
+ cirrus,boost-cap-microfarad = <15>;
+ cirrus,asp-sdout-hiz = <3>;
+ cirrus,gpio2-src-select = <2>;
+ cirrus,gpio2-output-enable;
+ #sound-dai-cells = <1>;
+ };
+
+ cs35l41_r: cs35l41@41 {
+ compatible = "cirrus,cs35l41";
+ reg = <0x41>;
+ interrupt-parent = <&tlmm>;
+ interrupts = <36 IRQ_TYPE_LEVEL_LOW>;
+ reset-gpios = <&tlmm 37 GPIO_ACTIVE_HIGH>;
+ cirrus,boost-peak-milliamp = <4000>;
+ cirrus,boost-ind-nanohenry = <1000>;
+ cirrus,boost-cap-microfarad = <15>;
+ cirrus,asp-sdout-hiz = <3>;
+ cirrus,gpio2-src-select = <2>;
+ cirrus,gpio2-output-enable;
+ #sound-dai-cells = <1>;
+ };
+};
+
+&i2c12 {
+ status = "okay";
+ /* Clock frequency was not specified downstream, let's park it to 100 KHz */
+ clock-frequency = <100000>;
+
+ /* AMS TCS3490 RGB+IR color sensor @ 72 */
+};
+
+&i2c13 {
+ status = "okay";
+ /* Clock frequency was not specified downstream, let's park it to 100 KHz */
+ clock-frequency = <100000>;
+
+ /* Qualcomm PM8008i/PM8008j (?) @ 8, 9, c, d */
+ /* Dialog SLG51000 CMIC @ 75 */
+};
+
+&i2c15 {
+ status = "okay";
+ clock-frequency = <400000>;
+
+ /* NXP SN1X0 NFC @ 28 */
+};
+
+&i2c17 {
+ status = "okay";
+ clock-frequency = <1000000>;
+
+ /* Cirrus Logic CS40L25A boosted haptics driver @ 40 */
+};
+
+&ipa {
+ status = "okay";
+ memory-region = <&pil_ipa_fw_mem>;
+ firmware-name = "qcom/ipa_fws.mbn";
+};
+
+&mpss {
+ status = "okay";
+ firmware-name = "qcom/modem.mbn";
+};
+
+&pmk8350_rtc {
+ status = "okay";
+};
+
+&pon_pwrkey {
+ status = "okay";
+};
+
+&pon_resin {
+ status = "okay";
+ linux,code = <KEY_VOLUMEUP>;
+};
+
+&qupv3_id_0 {
+ status = "okay";
+};
+
+&qupv3_id_1 {
+ status = "okay";
+};
+
+&qupv3_id_2 {
+ status = "okay";
+};
+
+&slpi {
+ status = "okay";
+ firmware-name = "qcom/slpi.mbn";
+};
+
+&spi14 {
+ status = "okay";
+
+ /* NXP SN1X0 NFC Secure Element @ 0 */
+};
+
+&tlmm {
+ gpio-reserved-ranges = <44 4>;
+
+ ts_int_default: ts-int-default {
+ pin = "gpio23";
+ function = "gpio";
+ drive-strength = <2>;
+ bias-disable;
+ input-enable;
+ };
+};
+
+/* BIG WARNING! DO NOT TOUCH UFS, YOUR DEVICE WILL DIE! */
+&ufs_mem_hc { status = "disabled"; };
+&ufs_mem_phy { status = "disabled"; };
+
+/* TODO: Make USB3 work (perhaps needs regulators for higher-current operation?) */
+&usb_1 {
+ status = "okay";
+
+ qcom,select-utmi-as-pipe-clk;
+};
+
+&usb_1_dwc3 {
+ dr_mode = "peripheral";
+
+ maximum-speed = "high-speed";
+ phys = <&usb_1_hsphy>;
+ phy-names = "usb2-phy";
+};
+
+&usb_1_hsphy {
+ status = "okay";
+};
+
+&usb_1_qmpphy {
+ status = "okay";
+};
diff --git a/arch/arm64/boot/dts/qcom/sm8350.dtsi b/arch/arm64/boot/dts/qcom/sm8350.dtsi
index d134280..53b39e7 100644
--- a/arch/arm64/boot/dts/qcom/sm8350.dtsi
+++ b/arch/arm64/boot/dts/qcom/sm8350.dtsi
@@ -6,6 +6,7 @@
#include <dt-bindings/interrupt-controller/arm-gic.h>
#include <dt-bindings/clock/qcom,gcc-sm8350.h>
#include <dt-bindings/clock/qcom,rpmh.h>
+#include <dt-bindings/gpio/gpio.h>
#include <dt-bindings/interconnect/qcom,sm8350.h>
#include <dt-bindings/mailbox/qcom-ipcc.h>
#include <dt-bindings/power/qcom-rpmpd.h>
@@ -583,8 +584,30 @@ gcc: clock-controller@100000 {
#clock-cells = <1>;
#reset-cells = <1>;
#power-domain-cells = <1>;
- clock-names = "bi_tcxo", "sleep_clk";
- clocks = <&rpmhcc RPMH_CXO_CLK>, <&sleep_clk>;
+ clock-names = "bi_tcxo",
+ "sleep_clk",
+ "pcie_0_pipe_clk",
+ "pcie_1_pipe_clk",
+ "ufs_card_rx_symbol_0_clk",
+ "ufs_card_rx_symbol_1_clk",
+ "ufs_card_tx_symbol_0_clk",
+ "ufs_phy_rx_symbol_0_clk",
+ "ufs_phy_rx_symbol_1_clk",
+ "ufs_phy_tx_symbol_0_clk",
+ "usb3_phy_wrapper_gcc_usb30_pipe_clk",
+ "usb3_uni_phy_sec_gcc_usb30_pipe_clk";
+ clocks = <&rpmhcc RPMH_CXO_CLK>,
+ <&sleep_clk>,
+ <0>,
+ <0>,
+ <0>,
+ <0>,
+ <0>,
+ <0>,
+ <0>,
+ <0>,
+ <0>,
+ <0>;
};
ipcc: mailbox@408000 {
@@ -596,17 +619,305 @@ ipcc: mailbox@408000 {
#mbox-cells = <2>;
};
+ qup_opp_table_100mhz: qup-100mhz-opp-table {
+ compatible = "operating-points-v2";
+
+ opp-50000000 {
+ opp-hz = /bits/ 64 <50000000>;
+ required-opps = <&rpmhpd_opp_min_svs>;
+ };
+
+ opp-75000000 {
+ opp-hz = /bits/ 64 <75000000>;
+ required-opps = <&rpmhpd_opp_low_svs>;
+ };
+
+ opp-100000000 {
+ opp-hz = /bits/ 64 <100000000>;
+ required-opps = <&rpmhpd_opp_svs>;
+ };
+ };
+
+ qup_opp_table_120mhz: qup-120mhz-opp-table {
+ compatible = "operating-points-v2";
+
+ opp-50000000 {
+ opp-hz = /bits/ 64 <50000000>;
+ required-opps = <&rpmhpd_opp_min_svs>;
+ };
+
+ opp-75000000 {
+ opp-hz = /bits/ 64 <75000000>;
+ required-opps = <&rpmhpd_opp_low_svs>;
+ };
+
+ opp-120000000 {
+ opp-hz = /bits/ 64 <120000000>;
+ required-opps = <&rpmhpd_opp_svs>;
+ };
+ };
+
+ qupv3_id_2: geniqup@8c0000 {
+ compatible = "qcom,geni-se-qup";
+ reg = <0x0 0x008c0000 0x0 0x6000>;
+ clock-names = "m-ahb", "s-ahb";
+ clocks = <&gcc GCC_QUPV3_WRAP_2_M_AHB_CLK>,
+ <&gcc GCC_QUPV3_WRAP_2_S_AHB_CLK>;
+ iommus = <&apps_smmu 0x5e3 0x0>;
+ #address-cells = <2>;
+ #size-cells = <2>;
+ ranges;
+ status = "disabled";
+
+ i2c14: i2c@880000 {
+ compatible = "qcom,geni-i2c";
+ reg = <0 0x00880000 0 0x4000>;
+ clock-names = "se";
+ clocks = <&gcc GCC_QUPV3_WRAP2_S0_CLK>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&qup_i2c14_default>;
+ interrupts = <GIC_SPI 373 IRQ_TYPE_LEVEL_HIGH>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ spi14: spi@880000 {
+ compatible = "qcom,geni-spi";
+ reg = <0 0x00880000 0 0x4000>;
+ clock-names = "se";
+ clocks = <&gcc GCC_QUPV3_WRAP2_S0_CLK>;
+ interrupts = <GIC_SPI 373 IRQ_TYPE_LEVEL_HIGH>;
+ power-domains = <&rpmhpd SM8350_CX>;
+ operating-points-v2 = <&qup_opp_table_120mhz>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ i2c15: i2c@884000 {
+ compatible = "qcom,geni-i2c";
+ reg = <0 0x00884000 0 0x4000>;
+ clock-names = "se";
+ clocks = <&gcc GCC_QUPV3_WRAP2_S1_CLK>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&qup_i2c15_default>;
+ interrupts = <GIC_SPI 583 IRQ_TYPE_LEVEL_HIGH>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ spi15: spi@884000 {
+ compatible = "qcom,geni-spi";
+ reg = <0 0x00884000 0 0x4000>;
+ clock-names = "se";
+ clocks = <&gcc GCC_QUPV3_WRAP2_S1_CLK>;
+ interrupts = <GIC_SPI 583 IRQ_TYPE_LEVEL_HIGH>;
+ power-domains = <&rpmhpd SM8350_CX>;
+ operating-points-v2 = <&qup_opp_table_120mhz>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ i2c16: i2c@888000 {
+ compatible = "qcom,geni-i2c";
+ reg = <0 0x00888000 0 0x4000>;
+ clock-names = "se";
+ clocks = <&gcc GCC_QUPV3_WRAP2_S2_CLK>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&qup_i2c16_default>;
+ interrupts = <GIC_SPI 584 IRQ_TYPE_LEVEL_HIGH>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ spi16: spi@888000 {
+ compatible = "qcom,geni-spi";
+ reg = <0 0x00888000 0 0x4000>;
+ clock-names = "se";
+ clocks = <&gcc GCC_QUPV3_WRAP2_S2_CLK>;
+ interrupts = <GIC_SPI 584 IRQ_TYPE_LEVEL_HIGH>;
+ power-domains = <&rpmhpd SM8350_CX>;
+ operating-points-v2 = <&qup_opp_table_100mhz>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ i2c17: i2c@88c000 {
+ compatible = "qcom,geni-i2c";
+ reg = <0 0x0088c000 0 0x4000>;
+ clock-names = "se";
+ clocks = <&gcc GCC_QUPV3_WRAP2_S3_CLK>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&qup_i2c17_default>;
+ interrupts = <GIC_SPI 585 IRQ_TYPE_LEVEL_HIGH>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ spi17: spi@88c000 {
+ compatible = "qcom,geni-spi";
+ reg = <0 0x0088c000 0 0x4000>;
+ clock-names = "se";
+ clocks = <&gcc GCC_QUPV3_WRAP2_S3_CLK>;
+ interrupts = <GIC_SPI 585 IRQ_TYPE_LEVEL_HIGH>;
+ power-domains = <&rpmhpd SM8350_CX>;
+ operating-points-v2 = <&qup_opp_table_100mhz>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ /* QUP no. 18 seems to be strictly SPI/UART-only */
+
+ spi18: spi@890000 {
+ compatible = "qcom,geni-spi";
+ reg = <0 0x00890000 0 0x4000>;
+ clock-names = "se";
+ clocks = <&gcc GCC_QUPV3_WRAP2_S4_CLK>;
+ interrupts = <GIC_SPI 586 IRQ_TYPE_LEVEL_HIGH>;
+ power-domains = <&rpmhpd SM8350_CX>;
+ operating-points-v2 = <&qup_opp_table_100mhz>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ uart18: serial@890000 {
+ compatible = "qcom,geni-uart";
+ reg = <0 0x00890000 0 0x4000>;
+ clock-names = "se";
+ clocks = <&gcc GCC_QUPV3_WRAP2_S4_CLK>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&qup_uart18_default>;
+ interrupts = <GIC_SPI 586 IRQ_TYPE_LEVEL_HIGH>;
+ power-domains = <&rpmhpd SM8350_CX>;
+ operating-points-v2 = <&qup_opp_table_100mhz>;
+ status = "disabled";
+ };
+
+ i2c19: i2c@894000 {
+ compatible = "qcom,geni-i2c";
+ reg = <0 0x00894000 0 0x4000>;
+ clock-names = "se";
+ clocks = <&gcc GCC_QUPV3_WRAP2_S5_CLK>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&qup_i2c19_default>;
+ interrupts = <GIC_SPI 587 IRQ_TYPE_LEVEL_HIGH>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ spi19: spi@894000 {
+ compatible = "qcom,geni-spi";
+ reg = <0 0x00894000 0 0x4000>;
+ clock-names = "se";
+ clocks = <&gcc GCC_QUPV3_WRAP2_S5_CLK>;
+ interrupts = <GIC_SPI 587 IRQ_TYPE_LEVEL_HIGH>;
+ power-domains = <&rpmhpd SM8350_CX>;
+ operating-points-v2 = <&qup_opp_table_100mhz>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+ };
+
qupv3_id_0: geniqup@9c0000 {
compatible = "qcom,geni-se-qup";
reg = <0x0 0x009c0000 0x0 0x6000>;
clock-names = "m-ahb", "s-ahb";
clocks = <&gcc GCC_QUPV3_WRAP_0_M_AHB_CLK>,
<&gcc GCC_QUPV3_WRAP_0_S_AHB_CLK>;
+ iommus = <&apps_smmu 0x5a3 0>;
#address-cells = <2>;
#size-cells = <2>;
ranges;
status = "disabled";
+ i2c0: i2c@980000 {
+ compatible = "qcom,geni-i2c";
+ reg = <0 0x00980000 0 0x4000>;
+ clock-names = "se";
+ clocks = <&gcc GCC_QUPV3_WRAP0_S0_CLK>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&qup_i2c0_default>;
+ interrupts = <GIC_SPI 601 IRQ_TYPE_LEVEL_HIGH>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ spi0: spi@980000 {
+ compatible = "qcom,geni-spi";
+ reg = <0 0x00980000 0 0x4000>;
+ clock-names = "se";
+ clocks = <&gcc GCC_QUPV3_WRAP0_S0_CLK>;
+ interrupts = <GIC_SPI 601 IRQ_TYPE_LEVEL_HIGH>;
+ power-domains = <&rpmhpd SM8350_CX>;
+ operating-points-v2 = <&qup_opp_table_100mhz>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ i2c1: i2c@984000 {
+ compatible = "qcom,geni-i2c";
+ reg = <0 0x00984000 0 0x4000>;
+ clock-names = "se";
+ clocks = <&gcc GCC_QUPV3_WRAP0_S1_CLK>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&qup_i2c1_default>;
+ interrupts = <GIC_SPI 602 IRQ_TYPE_LEVEL_HIGH>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ spi1: spi@984000 {
+ compatible = "qcom,geni-spi";
+ reg = <0 0x00984000 0 0x4000>;
+ clock-names = "se";
+ clocks = <&gcc GCC_QUPV3_WRAP0_S1_CLK>;
+ interrupts = <GIC_SPI 602 IRQ_TYPE_LEVEL_HIGH>;
+ power-domains = <&rpmhpd SM8350_CX>;
+ operating-points-v2 = <&qup_opp_table_100mhz>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ i2c2: i2c@988000 {
+ compatible = "qcom,geni-i2c";
+ reg = <0 0x00988000 0 0x4000>;
+ clock-names = "se";
+ clocks = <&gcc GCC_QUPV3_WRAP0_S2_CLK>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&qup_i2c2_default>;
+ interrupts = <GIC_SPI 603 IRQ_TYPE_LEVEL_HIGH>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ spi2: spi@988000 {
+ compatible = "qcom,geni-spi";
+ reg = <0 0x00988000 0 0x4000>;
+ clock-names = "se";
+ clocks = <&gcc GCC_QUPV3_WRAP0_S2_CLK>;
+ interrupts = <GIC_SPI 603 IRQ_TYPE_LEVEL_HIGH>;
+ power-domains = <&rpmhpd SM8350_CX>;
+ operating-points-v2 = <&qup_opp_table_100mhz>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
uart2: serial@98c000 {
compatible = "qcom,geni-debug-uart";
reg = <0 0x0098c000 0 0x4000>;
@@ -615,6 +926,140 @@ uart2: serial@98c000 {
pinctrl-names = "default";
pinctrl-0 = <&qup_uart3_default_state>;
interrupts = <GIC_SPI 604 IRQ_TYPE_LEVEL_HIGH>;
+ power-domains = <&rpmhpd SM8350_CX>;
+ operating-points-v2 = <&qup_opp_table_100mhz>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ /* QUP no. 3 seems to be strictly SPI-only */
+
+ spi3: spi@98c000 {
+ compatible = "qcom,geni-spi";
+ reg = <0 0x0098c000 0 0x4000>;
+ clock-names = "se";
+ clocks = <&gcc GCC_QUPV3_WRAP0_S3_CLK>;
+ interrupts = <GIC_SPI 604 IRQ_TYPE_LEVEL_HIGH>;
+ power-domains = <&rpmhpd SM8350_CX>;
+ operating-points-v2 = <&qup_opp_table_100mhz>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ i2c4: i2c@990000 {
+ compatible = "qcom,geni-i2c";
+ reg = <0 0x00990000 0 0x4000>;
+ clock-names = "se";
+ clocks = <&gcc GCC_QUPV3_WRAP0_S4_CLK>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&qup_i2c4_default>;
+ interrupts = <GIC_SPI 605 IRQ_TYPE_LEVEL_HIGH>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ spi4: spi@990000 {
+ compatible = "qcom,geni-spi";
+ reg = <0 0x00990000 0 0x4000>;
+ clock-names = "se";
+ clocks = <&gcc GCC_QUPV3_WRAP0_S4_CLK>;
+ interrupts = <GIC_SPI 605 IRQ_TYPE_LEVEL_HIGH>;
+ power-domains = <&rpmhpd SM8350_CX>;
+ operating-points-v2 = <&qup_opp_table_100mhz>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ i2c5: i2c@994000 {
+ compatible = "qcom,geni-i2c";
+ reg = <0 0x00994000 0 0x4000>;
+ clock-names = "se";
+ clocks = <&gcc GCC_QUPV3_WRAP0_S5_CLK>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&qup_i2c5_default>;
+ interrupts = <GIC_SPI 606 IRQ_TYPE_LEVEL_HIGH>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ spi5: spi@994000 {
+ compatible = "qcom,geni-spi";
+ reg = <0 0x00994000 0 0x4000>;
+ clock-names = "se";
+ clocks = <&gcc GCC_QUPV3_WRAP0_S5_CLK>;
+ interrupts = <GIC_SPI 606 IRQ_TYPE_LEVEL_HIGH>;
+ power-domains = <&rpmhpd SM8350_CX>;
+ operating-points-v2 = <&qup_opp_table_100mhz>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ i2c6: i2c@998000 {
+ compatible = "qcom,geni-i2c";
+ reg = <0 0x00998000 0 0x4000>;
+ clock-names = "se";
+ clocks = <&gcc GCC_QUPV3_WRAP0_S6_CLK>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&qup_i2c6_default>;
+ interrupts = <GIC_SPI 607 IRQ_TYPE_LEVEL_HIGH>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ spi6: spi@998000 {
+ compatible = "qcom,geni-spi";
+ reg = <0 0x00998000 0 0x4000>;
+ clock-names = "se";
+ clocks = <&gcc GCC_QUPV3_WRAP0_S6_CLK>;
+ interrupts = <GIC_SPI 607 IRQ_TYPE_LEVEL_HIGH>;
+ power-domains = <&rpmhpd SM8350_CX>;
+ operating-points-v2 = <&qup_opp_table_100mhz>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ uart6: serial@998000 {
+ compatible = "qcom,geni-uart";
+ reg = <0 0x00998000 0 0x4000>;
+ clock-names = "se";
+ clocks = <&gcc GCC_QUPV3_WRAP0_S6_CLK>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&qup_uart6_default>;
+ interrupts = <GIC_SPI 607 IRQ_TYPE_LEVEL_HIGH>;
+ power-domains = <&rpmhpd SM8350_CX>;
+ operating-points-v2 = <&qup_opp_table_100mhz>;
+ status = "disabled";
+ };
+
+ i2c7: i2c@99c000 {
+ compatible = "qcom,geni-i2c";
+ reg = <0 0x0099c000 0 0x4000>;
+ clock-names = "se";
+ clocks = <&gcc GCC_QUPV3_WRAP0_S7_CLK>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&qup_i2c7_default>;
+ interrupts = <GIC_SPI 608 IRQ_TYPE_LEVEL_HIGH>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ spi7: spi@99c000 {
+ compatible = "qcom,geni-spi";
+ reg = <0 0x0099c000 0 0x4000>;
+ clock-names = "se";
+ clocks = <&gcc GCC_QUPV3_WRAP0_S7_CLK>;
+ interrupts = <GIC_SPI 608 IRQ_TYPE_LEVEL_HIGH>;
+ power-domains = <&rpmhpd SM8350_CX>;
+ operating-points-v2 = <&qup_opp_table_100mhz>;
#address-cells = <1>;
#size-cells = <0>;
status = "disabled";
@@ -627,23 +1072,167 @@ qupv3_id_1: geniqup@ac0000 {
clock-names = "m-ahb", "s-ahb";
clocks = <&gcc GCC_QUPV3_WRAP_1_M_AHB_CLK>,
<&gcc GCC_QUPV3_WRAP_1_S_AHB_CLK>;
+ iommus = <&apps_smmu 0x43 0>;
#address-cells = <2>;
#size-cells = <2>;
ranges;
status = "disabled";
+ i2c8: i2c@a80000 {
+ compatible = "qcom,geni-i2c";
+ reg = <0 0x00a80000 0 0x4000>;
+ clock-names = "se";
+ clocks = <&gcc GCC_QUPV3_WRAP1_S0_CLK>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&qup_i2c8_default>;
+ interrupts = <GIC_SPI 353 IRQ_TYPE_LEVEL_HIGH>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ spi8: spi@a80000 {
+ compatible = "qcom,geni-spi";
+ reg = <0 0x00a80000 0 0x4000>;
+ clock-names = "se";
+ clocks = <&gcc GCC_QUPV3_WRAP1_S0_CLK>;
+ interrupts = <GIC_SPI 353 IRQ_TYPE_LEVEL_HIGH>;
+ power-domains = <&rpmhpd SM8350_CX>;
+ operating-points-v2 = <&qup_opp_table_120mhz>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ i2c9: i2c@a84000 {
+ compatible = "qcom,geni-i2c";
+ reg = <0 0x00a84000 0 0x4000>;
+ clock-names = "se";
+ clocks = <&gcc GCC_QUPV3_WRAP1_S1_CLK>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&qup_i2c9_default>;
+ interrupts = <GIC_SPI 354 IRQ_TYPE_LEVEL_HIGH>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ spi9: spi@a84000 {
+ compatible = "qcom,geni-spi";
+ reg = <0 0x00a84000 0 0x4000>;
+ clock-names = "se";
+ clocks = <&gcc GCC_QUPV3_WRAP1_S1_CLK>;
+ interrupts = <GIC_SPI 354 IRQ_TYPE_LEVEL_HIGH>;
+ power-domains = <&rpmhpd SM8350_CX>;
+ operating-points-v2 = <&qup_opp_table_100mhz>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ i2c10: i2c@a88000 {
+ compatible = "qcom,geni-i2c";
+ reg = <0 0x00a88000 0 0x4000>;
+ clock-names = "se";
+ clocks = <&gcc GCC_QUPV3_WRAP1_S2_CLK>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&qup_i2c10_default>;
+ interrupts = <GIC_SPI 355 IRQ_TYPE_LEVEL_HIGH>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ spi10: spi@a88000 {
+ compatible = "qcom,geni-spi";
+ reg = <0 0x00a88000 0 0x4000>;
+ clock-names = "se";
+ clocks = <&gcc GCC_QUPV3_WRAP1_S2_CLK>;
+ interrupts = <GIC_SPI 355 IRQ_TYPE_LEVEL_HIGH>;
+ power-domains = <&rpmhpd SM8350_CX>;
+ operating-points-v2 = <&qup_opp_table_100mhz>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ i2c11: i2c@a8c000 {
+ compatible = "qcom,geni-i2c";
+ reg = <0 0x00a8c000 0 0x4000>;
+ clock-names = "se";
+ clocks = <&gcc GCC_QUPV3_WRAP1_S3_CLK>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&qup_i2c11_default>;
+ interrupts = <GIC_SPI 356 IRQ_TYPE_LEVEL_HIGH>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ spi11: spi@a8c000 {
+ compatible = "qcom,geni-spi";
+ reg = <0 0x00a8c000 0 0x4000>;
+ clock-names = "se";
+ clocks = <&gcc GCC_QUPV3_WRAP1_S3_CLK>;
+ interrupts = <GIC_SPI 356 IRQ_TYPE_LEVEL_HIGH>;
+ power-domains = <&rpmhpd SM8350_CX>;
+ operating-points-v2 = <&qup_opp_table_100mhz>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ i2c12: i2c@a90000 {
+ compatible = "qcom,geni-i2c";
+ reg = <0 0x00a90000 0 0x4000>;
+ clock-names = "se";
+ clocks = <&gcc GCC_QUPV3_WRAP1_S4_CLK>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&qup_i2c12_default>;
+ interrupts = <GIC_SPI 357 IRQ_TYPE_LEVEL_HIGH>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ spi12: spi@a90000 {
+ compatible = "qcom,geni-spi";
+ reg = <0 0x00a90000 0 0x4000>;
+ clock-names = "se";
+ clocks = <&gcc GCC_QUPV3_WRAP1_S4_CLK>;
+ interrupts = <GIC_SPI 357 IRQ_TYPE_LEVEL_HIGH>;
+ power-domains = <&rpmhpd SM8350_CX>;
+ operating-points-v2 = <&qup_opp_table_100mhz>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
i2c13: i2c@a94000 {
compatible = "qcom,geni-i2c";
reg = <0 0x00a94000 0 0x4000>;
clock-names = "se";
clocks = <&gcc GCC_QUPV3_WRAP1_S5_CLK>;
pinctrl-names = "default";
- pinctrl-0 = <&qup_i2c13_default_state>;
+ pinctrl-0 = <&qup_i2c13_default>;
interrupts = <GIC_SPI 358 IRQ_TYPE_LEVEL_HIGH>;
#address-cells = <1>;
#size-cells = <0>;
status = "disabled";
};
+
+ spi13: spi@a94000 {
+ compatible = "qcom,geni-spi";
+ reg = <0 0x00a94000 0 0x4000>;
+ clock-names = "se";
+ clocks = <&gcc GCC_QUPV3_WRAP1_S5_CLK>;
+ interrupts = <GIC_SPI 358 IRQ_TYPE_LEVEL_HIGH>;
+ power-domains = <&rpmhpd SM8350_CX>;
+ operating-points-v2 = <&qup_opp_table_100mhz>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
};
apps_smmu: iommu@15000000 {
@@ -910,7 +1499,7 @@ tsens0: thermal-sensor@c263000 {
reg = <0 0x0c263000 0 0x1ff>, /* TM */
<0 0x0c222000 0 0x8>; /* SROT */
#qcom,sensors = <15>;
- interrupts = <&pdc 26 IRQ_TYPE_LEVEL_HIGH>,
+ interrupts-extended = <&pdc 26 IRQ_TYPE_LEVEL_HIGH>,
<&pdc 28 IRQ_TYPE_LEVEL_HIGH>;
interrupt-names = "uplow", "critical";
#thermal-sensor-cells = <1>;
@@ -921,7 +1510,7 @@ tsens1: thermal-sensor@c265000 {
reg = <0 0x0c265000 0 0x1ff>, /* TM */
<0 0x0c223000 0 0x8>; /* SROT */
#qcom,sensors = <14>;
- interrupts = <&pdc 27 IRQ_TYPE_LEVEL_HIGH>,
+ interrupts-extended = <&pdc 27 IRQ_TYPE_LEVEL_HIGH>,
<&pdc 29 IRQ_TYPE_LEVEL_HIGH>;
interrupt-names = "uplow", "critical";
#thermal-sensor-cells = <1>;
@@ -982,17 +1571,144 @@ tx {
};
};
- qup_i2c13_default_state: qup-i2c13-default-state {
- mux {
- pins = "gpio0", "gpio1";
- function = "qup13";
- };
+ qup_uart6_default: qup-uart6-default {
+ pins = "gpio30", "gpio31";
+ function = "qup6";
+ drive-strength = <2>;
+ bias-disable;
+ };
- config {
- pins = "gpio0", "gpio1";
- drive-strength = <2>;
- bias-pull-up;
- };
+ qup_uart18_default: qup-uart18-default {
+ pins = "gpio58", "gpio59";
+ function = "qup18";
+ drive-strength = <2>;
+ bias-disable;
+ };
+
+ qup_i2c0_default: qup-i2c0-default {
+ pins = "gpio4", "gpio5";
+ function = "qup0";
+ drive-strength = <2>;
+ bias-pull-up;
+ };
+
+ qup_i2c1_default: qup-i2c1-default {
+ pins = "gpio8", "gpio9";
+ function = "qup1";
+ drive-strength = <2>;
+ bias-pull-up;
+ };
+
+ qup_i2c2_default: qup-i2c2-default {
+ pins = "gpio12", "gpio13";
+ function = "qup2";
+ drive-strength = <2>;
+ bias-pull-up;
+ };
+
+ qup_i2c4_default: qup-i2c4-default {
+ pins = "gpio20", "gpio21";
+ function = "qup4";
+ drive-strength = <2>;
+ bias-pull-up;
+ };
+
+ qup_i2c5_default: qup-i2c5-default {
+ pins = "gpio24", "gpio25";
+ function = "qup5";
+ drive-strength = <2>;
+ bias-pull-up;
+ };
+
+ qup_i2c6_default: qup-i2c6-default {
+ pins = "gpio28", "gpio29";
+ function = "qup6";
+ drive-strength = <2>;
+ bias-pull-up;
+ };
+
+ qup_i2c7_default: qup-i2c7-default {
+ pins = "gpio32", "gpio33";
+ function = "qup7";
+ drive-strength = <2>;
+ bias-disable;
+ };
+
+ qup_i2c8_default: qup-i2c8-default {
+ pins = "gpio36", "gpio37";
+ function = "qup8";
+ drive-strength = <2>;
+ bias-pull-up;
+ };
+
+ qup_i2c9_default: qup-i2c9-default {
+ pins = "gpio40", "gpio41";
+ function = "qup9";
+ drive-strength = <2>;
+ bias-pull-up;
+ };
+
+ qup_i2c10_default: qup-i2c10-default {
+ pins = "gpio44", "gpio45";
+ function = "qup10";
+ drive-strength = <2>;
+ bias-pull-up;
+ };
+
+ qup_i2c11_default: qup-i2c11-default {
+ pins = "gpio48", "gpio49";
+ function = "qup11";
+ drive-strength = <2>;
+ bias-pull-up;
+ };
+
+ qup_i2c12_default: qup-i2c12-default {
+ pins = "gpio52", "gpio53";
+ function = "qup12";
+ drive-strength = <2>;
+ bias-pull-up;
+ };
+
+ qup_i2c13_default: qup-i2c13-default {
+ pins = "gpio0", "gpio1";
+ function = "qup13";
+ drive-strength = <2>;
+ bias-pull-up;
+ };
+
+ qup_i2c14_default: qup-i2c14-default {
+ pins = "gpio56", "gpio57";
+ function = "qup14";
+ drive-strength = <2>;
+ bias-disable;
+ };
+
+ qup_i2c15_default: qup-i2c15-default {
+ pins = "gpio60", "gpio61";
+ function = "qup15";
+ drive-strength = <2>;
+ bias-disable;
+ };
+
+ qup_i2c16_default: qup-i2c16-default {
+ pins = "gpio64", "gpio65";
+ function = "qup16";
+ drive-strength = <2>;
+ bias-disable;
+ };
+
+ qup_i2c17_default: qup-i2c17-default {
+ pins = "gpio72", "gpio73";
+ function = "qup17";
+ drive-strength = <2>;
+ bias-disable;
+ };
+
+ qup_i2c19_default: qup-i2c19-default {
+ pins = "gpio76", "gpio77";
+ function = "qup19";
+ drive-strength = <2>;
+ bias-disable;
};
};
@@ -1007,6 +1723,8 @@ intc: interrupt-controller@17a00000 {
compatible = "arm,gic-v3";
#interrupt-cells = <3>;
interrupt-controller;
+ #redistributor-regions = <1>;
+ redistributor-stride = <0 0x20000>;
reg = <0x0 0x17a00000 0x0 0x10000>, /* GICD */
<0x0 0x17a60000 0x0 0x100000>; /* GICR * 8 */
interrupts = <GIC_PPI 9 IRQ_TYPE_LEVEL_HIGH>;
@@ -1514,6 +2232,12 @@ gem_noc: interconnect@9100000 {
qcom,bcm-voters = <&apps_bcm_voter>;
};
+ system-cache-controller@9200000 {
+ compatible = "qcom,sm8350-llcc";
+ reg = <0 0x09200000 0 0x1d0000>, <0 0x09600000 0 0x50000>;
+ reg-names = "llcc_base", "llcc_broadcast_base";
+ };
+
usb_1: usb@a6f8800 {
compatible = "qcom,sm8350-dwc3", "qcom,dwc3";
reg = <0 0x0a6f8800 0 0x400>;
@@ -2447,7 +3171,7 @@ camera1_alert0: trip-point0 {
};
};
- camera-thermal-bottom {
+ cam-thermal-bottom {
polling-delay-passive = <250>;
polling-delay = <1000>;
diff --git a/arch/arm64/boot/dts/qcom/sm8450-qrd.dts b/arch/arm64/boot/dts/qcom/sm8450-qrd.dts
new file mode 100644
index 0000000..b68ab24
--- /dev/null
+++ b/arch/arm64/boot/dts/qcom/sm8450-qrd.dts
@@ -0,0 +1,398 @@
+// SPDX-License-Identifier: BSD-3-Clause
+/*
+ * Copyright (c) 2021, Linaro Limited
+ */
+
+/dts-v1/;
+
+#include <dt-bindings/regulator/qcom,rpmh-regulator.h>
+#include "sm8450.dtsi"
+
+/ {
+ model = "Qualcomm Technologies, Inc. SM8450 QRD";
+ compatible = "qcom,sm8450-qrd", "qcom,sm8450";
+
+ aliases {
+ serial0 = &uart7;
+ };
+
+ chosen {
+ stdout-path = "serial0:115200n8";
+ };
+
+ vph_pwr: vph-pwr-regulator {
+ compatible = "regulator-fixed";
+ regulator-name = "vph_pwr";
+ regulator-min-microvolt = <3700000>;
+ regulator-max-microvolt = <3700000>;
+
+ regulator-always-on;
+ regulator-boot-on;
+ };
+};
+
+&apps_rsc {
+ pm8350-rpmh-regulators {
+ compatible = "qcom,pm8350-rpmh-regulators";
+ qcom,pmic-id = "b";
+
+ vdd-s1-supply = <&vph_pwr>;
+ vdd-s2-supply = <&vph_pwr>;
+ vdd-s3-supply = <&vph_pwr>;
+ vdd-s4-supply = <&vph_pwr>;
+ vdd-s5-supply = <&vph_pwr>;
+ vdd-s6-supply = <&vph_pwr>;
+ vdd-s7-supply = <&vph_pwr>;
+ vdd-s8-supply = <&vph_pwr>;
+ vdd-s9-supply = <&vph_pwr>;
+ vdd-s10-supply = <&vph_pwr>;
+ vdd-s11-supply = <&vph_pwr>;
+ vdd-s12-supply = <&vph_pwr>;
+
+ vdd-l1-l4-supply = <&vreg_s11b_0p95>;
+ vdd-l2-l7-supply = <&vreg_bob>;
+ vdd-l3-l5-supply = <&vreg_bob>;
+ vdd-l6-l9-l10-supply = <&vreg_s12b_1p25>;
+ vdd-l8-supply = <&vreg_s2h_0p95>;
+
+ vreg_s10b_1p8: smps10 {
+ regulator-name = "vreg_s10b_1p8";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ };
+
+ vreg_s11b_0p95: smps11 {
+ regulator-name = "vreg_s11b_0p95";
+ regulator-min-microvolt = <848000>;
+ regulator-max-microvolt = <1104000>;
+ };
+
+ vreg_s12b_1p25: smps12 {
+ regulator-name = "vreg_s12b_1p25";
+ regulator-min-microvolt = <1224000>;
+ regulator-max-microvolt = <1400000>;
+ };
+
+ vreg_l1b_0p91: ldo1 {
+ regulator-name = "vreg_l1b_0p91";
+ regulator-min-microvolt = <912000>;
+ regulator-max-microvolt = <920000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l2b_3p07: ldo2 {
+ regulator-name = "vreg_l2b_3p07";
+ regulator-min-microvolt = <3072000>;
+ regulator-max-microvolt = <3072000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l3b_0p9: ldo3 {
+ regulator-name = "vreg_l3b_0p9";
+ regulator-min-microvolt = <904000>;
+ regulator-max-microvolt = <904000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l5b_0p88: ldo5 {
+ regulator-name = "vreg_l5b_0p88";
+ regulator-min-microvolt = <880000>;
+ regulator-max-microvolt = <888000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l6b_1p2: ldo6 {
+ regulator-name = "vreg_l6b_1p2";
+ regulator-min-microvolt = <1200000>;
+ regulator-max-microvolt = <1200000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l7b_2p5: ldo7 {
+ regulator-name = "vreg_l7b_2p5";
+ regulator-min-microvolt = <2504000>;
+ regulator-max-microvolt = <2504000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l9b_1p2: ldo9 {
+ regulator-name = "vreg_l9b_1p2";
+ regulator-min-microvolt = <1200000>;
+ regulator-max-microvolt = <1200000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+ };
+
+ pm8350c-rpmh-regulators {
+ compatible = "qcom,pm8350c-rpmh-regulators";
+ qcom,pmic-id = "c";
+
+ vdd-s1-supply = <&vph_pwr>;
+ vdd-s2-supply = <&vph_pwr>;
+ vdd-s3-supply = <&vph_pwr>;
+ vdd-s4-supply = <&vph_pwr>;
+ vdd-s5-supply = <&vph_pwr>;
+ vdd-s6-supply = <&vph_pwr>;
+ vdd-s7-supply = <&vph_pwr>;
+ vdd-s8-supply = <&vph_pwr>;
+ vdd-s9-supply = <&vph_pwr>;
+ vdd-s10-supply = <&vph_pwr>;
+
+ vdd-l1-l12-supply = <&vreg_bob>;
+ vdd-l2-l8-supply = <&vreg_bob>;
+ vdd-l3-l4-l5-l7-l13-supply = <&vreg_bob>;
+ vdd-l6-l9-l11-supply = <&vreg_bob>;
+
+ vdd-bob-supply = <&vph_pwr>;
+
+ vreg_s1c_1p86: smps1 {
+ regulator-name = "vreg_s1c_1p86";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <2024000>;
+ };
+
+ vreg_s10c_1p05: smps10 {
+ regulator-name = "vreg_s10c_1p05";
+ regulator-min-microvolt = <1000000>;
+ regulator-max-microvolt = <1100000>;
+ };
+
+ vreg_bob: bob {
+ regulator-name = "vreg_bob";
+ regulator-min-microvolt = <3008000>;
+ regulator-max-microvolt = <3960000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_AUTO>;
+ };
+
+ vreg_l1c_1p8: ldo1 {
+ regulator-name = "vreg_l1c_1p8";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l3c_3p0: ldo3 {
+ regulator-name = "vreg_l3c_3p0";
+ regulator-min-microvolt = <3296000>;
+ regulator-max-microvolt = <3304000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l4c_1p8: ldo4 {
+ regulator-name = "vreg_l4c_1p8";
+ regulator-min-microvolt = <1704000>;
+ regulator-max-microvolt = <3000000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l5c_1p8: ldo5 {
+ regulator-name = "vreg_l5c_1p8";
+ regulator-min-microvolt = <1704000>;
+ regulator-max-microvolt = <3000000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l6c_1p8: ldo6 {
+ regulator-name = "vreg_l6c_1p8";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <3008000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l7c_3p0: ldo7 {
+ regulator-name = "vreg_l7c_3p0";
+ regulator-min-microvolt = <3008000>;
+ regulator-max-microvolt = <3008000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l8c_1p8: ldo8 {
+ regulator-name = "vreg_l8c_1p8";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l9c_2p96: ldo9 {
+ regulator-name = "vreg_l9c_2p96";
+ regulator-min-microvolt = <2960000>;
+ regulator-max-microvolt = <3008000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l12c_1p8: ldo12 {
+ regulator-name = "vreg_l12c_1p8";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1968000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l13c_3p0: ldo13 {
+ regulator-name = "vreg_l13c_3p0";
+ regulator-min-microvolt = <3000000>;
+ regulator-max-microvolt = <3000000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+ };
+
+ pm8450-rpmh-regulators {
+ compatible = "qcom,pm8450-rpmh-regulators";
+ qcom,pmic-id = "h";
+
+ vdd-s1-supply = <&vph_pwr>;
+ vdd-s2-supply = <&vph_pwr>;
+ vdd-s3-supply = <&vph_pwr>;
+ vdd-s4-supply = <&vph_pwr>;
+ vdd-s5-supply = <&vph_pwr>;
+ vdd-s6-supply = <&vph_pwr>;
+
+ vdd-l2-supply = <&vreg_bob>;
+ vdd-l3-supply = <&vreg_bob>;
+ vdd-l4-supply = <&vreg_bob>;
+
+ vreg_s2h_0p95: smps2 {
+ regulator-name = "vreg_s2h_0p95";
+ regulator-min-microvolt = <848000>;
+ regulator-max-microvolt = <1104000>;
+ };
+
+ vreg_s3h_0p5: smps3 {
+ regulator-name = "vreg_s3h_0p5";
+ regulator-min-microvolt = <500000>;
+ regulator-max-microvolt = <500000>;
+ };
+
+ vreg_l2h_0p91: ldo2 {
+ regulator-name = "vreg_l2h_0p91";
+ regulator-min-microvolt = <880000>;
+ regulator-max-microvolt = <912000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ vreg_l3h_0p91: ldo3 {
+ regulator-name = "vreg_l3h_0p91";
+ regulator-min-microvolt = <912000>;
+ regulator-max-microvolt = <912000>;
+ regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ };
+
+ };
+
+ pmr735a-rpmh-regulators {
+ compatible = "qcom,pmr735a-rpmh-regulators";
+ qcom,pmic-id = "e";
+
+ vdd-s1-supply = <&vph_pwr>;
+ vdd-s2-supply = <&vph_pwr>;
+ vdd-s3-supply = <&vph_pwr>;
+
+ vdd-l1-l2-supply = <&vreg_s2e_0p85>;
+ vdd-l3-supply = <&vreg_s1e_1p25>;
+ vdd-l4-supply = <&vreg_s1c_1p86>;
+ vdd-l5-l6-supply = <&vreg_s1c_1p86>;
+ vdd-l7-bob-supply = <&vreg_bob>;
+
+ vreg_s1e_1p25: smps1 {
+ regulator-name = "vreg_s1e_1p25";
+ regulator-min-microvolt = <1200000>;
+ regulator-max-microvolt = <1296000>;
+ };
+
+ vreg_s2e_0p85: smps2 {
+ regulator-name = "vreg_s2e_0p85";
+ regulator-min-microvolt = <500000>;
+ regulator-max-microvolt = <1040000>;
+ };
+
+ vreg_l1e_0p8: ldo1 {
+ regulator-name = "vreg_l1e_0p8";
+ regulator-min-microvolt = <800000>;
+ regulator-max-microvolt = <800000>;
+ };
+
+ vreg_l2e_0p8: ldo2 {
+ regulator-name = "vreg_l2e_0p8";
+ regulator-min-microvolt = <800000>;
+ regulator-max-microvolt = <800000>;
+ };
+
+ vreg_l3e_1p2: ldo3 {
+ regulator-name = "vreg_l3e_1p2";
+ regulator-min-microvolt = <1200000>;
+ regulator-max-microvolt = <1200000>;
+ };
+
+ vreg_l4e_1p7: ldo4 {
+ regulator-name = "vreg_l4e_1p7";
+ regulator-min-microvolt = <1776000>;
+ regulator-max-microvolt = <1776000>;
+ };
+
+ vreg_l5e_0p88: ldo5 {
+ regulator-name = "vreg_l5e_0p88";
+ regulator-min-microvolt = <880000>;
+ regulator-max-microvolt = <880000>;
+ };
+
+ vreg_l6e_1p2: ldo6 {
+ regulator-name = "vreg_l6e_1p2";
+ regulator-min-microvolt = <1200000>;
+ regulator-max-microvolt = <1200000>;
+ };
+ };
+};
+
+&qupv3_id_0 {
+ status = "okay";
+};
+
+&tlmm {
+ gpio-reserved-ranges = <28 4>, <36 4>;
+};
+
+&uart7 {
+ status = "okay";
+};
+
+&ufs_mem_hc {
+ status = "okay";
+
+ reset-gpios = <&tlmm 210 GPIO_ACTIVE_LOW>;
+
+ vcc-supply = <&vreg_l7b_2p5>;
+ vcc-max-microamp = <1100000>;
+ vccq-supply = <&vreg_l9b_1p2>;
+ vccq-max-microamp = <1200000>;
+};
+
+&ufs_mem_phy {
+ status = "okay";
+
+ vdda-phy-supply = <&vreg_l5b_0p88>;
+ vdda-pll-supply = <&vreg_l6b_1p2>;
+ vdda-max-microamp = <173000>;
+ vdda-pll-max-microamp = <24900>;
+};
+
+&usb_1 {
+ status = "okay";
+};
+
+&usb_1_dwc3 {
+ dr_mode = "peripheral";
+};
+
+&usb_1_hsphy {
+ status = "okay";
+
+ vdda-pll-supply = <&vreg_l5b_0p88>;
+ vdda18-supply = <&vreg_l1c_1p8>;
+ vdda33-supply = <&vreg_l2b_3p07>;
+};
+
+&usb_1_qmpphy {
+ status = "okay";
+
+ vdda-phy-supply = <&vreg_l6b_1p2>;
+ vdda-pll-supply = <&vreg_l1b_0p91>;
+};
diff --git a/arch/arm64/boot/dts/qcom/sm8450.dtsi b/arch/arm64/boot/dts/qcom/sm8450.dtsi
new file mode 100644
index 0000000..10c25ad
--- /dev/null
+++ b/arch/arm64/boot/dts/qcom/sm8450.dtsi
@@ -0,0 +1,1115 @@
+// SPDX-License-Identifier: BSD-3-Clause
+/*
+ * Copyright (c) 2021, Linaro Limited
+ */
+
+#include <dt-bindings/interrupt-controller/arm-gic.h>
+#include <dt-bindings/clock/qcom,gcc-sm8450.h>
+#include <dt-bindings/clock/qcom,rpmh.h>
+#include <dt-bindings/gpio/gpio.h>
+#include <dt-bindings/power/qcom-rpmpd.h>
+#include <dt-bindings/soc/qcom,rpmh-rsc.h>
+
+/ {
+ interrupt-parent = <&intc>;
+
+ #address-cells = <2>;
+ #size-cells = <2>;
+
+ chosen { };
+
+ clocks {
+ xo_board: xo-board {
+ compatible = "fixed-clock";
+ #clock-cells = <0>;
+ clock-frequency = <76800000>;
+ };
+
+ sleep_clk: sleep-clk {
+ compatible = "fixed-clock";
+ #clock-cells = <0>;
+ clock-frequency = <32000>;
+ };
+ };
+
+ cpus {
+ #address-cells = <2>;
+ #size-cells = <0>;
+
+ CPU0: cpu@0 {
+ device_type = "cpu";
+ compatible = "qcom,kryo780";
+ reg = <0x0 0x0>;
+ enable-method = "psci";
+ next-level-cache = <&L2_0>;
+ power-domains = <&CPU_PD0>;
+ power-domain-names = "psci";
+ qcom,freq-domain = <&cpufreq_hw 0>;
+ L2_0: l2-cache {
+ compatible = "cache";
+ next-level-cache = <&L3_0>;
+ L3_0: l3-cache {
+ compatible = "cache";
+ };
+ };
+ };
+
+ CPU1: cpu@100 {
+ device_type = "cpu";
+ compatible = "qcom,kryo780";
+ reg = <0x0 0x100>;
+ enable-method = "psci";
+ next-level-cache = <&L2_100>;
+ power-domains = <&CPU_PD1>;
+ power-domain-names = "psci";
+ qcom,freq-domain = <&cpufreq_hw 0>;
+ L2_100: l2-cache {
+ compatible = "cache";
+ next-level-cache = <&L3_0>;
+ };
+ };
+
+ CPU2: cpu@200 {
+ device_type = "cpu";
+ compatible = "qcom,kryo780";
+ reg = <0x0 0x200>;
+ enable-method = "psci";
+ next-level-cache = <&L2_200>;
+ power-domains = <&CPU_PD2>;
+ power-domain-names = "psci";
+ qcom,freq-domain = <&cpufreq_hw 0>;
+ L2_200: l2-cache {
+ compatible = "cache";
+ next-level-cache = <&L3_0>;
+ };
+ };
+
+ CPU3: cpu@300 {
+ device_type = "cpu";
+ compatible = "qcom,kryo780";
+ reg = <0x0 0x300>;
+ enable-method = "psci";
+ next-level-cache = <&L2_300>;
+ power-domains = <&CPU_PD3>;
+ power-domain-names = "psci";
+ qcom,freq-domain = <&cpufreq_hw 0>;
+ L2_300: l2-cache {
+ compatible = "cache";
+ next-level-cache = <&L3_0>;
+ };
+ };
+
+ CPU4: cpu@400 {
+ device_type = "cpu";
+ compatible = "qcom,kryo780";
+ reg = <0x0 0x400>;
+ enable-method = "psci";
+ next-level-cache = <&L2_400>;
+ power-domains = <&CPU_PD4>;
+ power-domain-names = "psci";
+ qcom,freq-domain = <&cpufreq_hw 1>;
+ L2_400: l2-cache {
+ compatible = "cache";
+ next-level-cache = <&L3_0>;
+ };
+ };
+
+ CPU5: cpu@500 {
+ device_type = "cpu";
+ compatible = "qcom,kryo780";
+ reg = <0x0 0x500>;
+ enable-method = "psci";
+ next-level-cache = <&L2_500>;
+ power-domains = <&CPU_PD5>;
+ power-domain-names = "psci";
+ qcom,freq-domain = <&cpufreq_hw 1>;
+ L2_500: l2-cache {
+ compatible = "cache";
+ next-level-cache = <&L3_0>;
+ };
+
+ };
+
+ CPU6: cpu@600 {
+ device_type = "cpu";
+ compatible = "qcom,kryo780";
+ reg = <0x0 0x600>;
+ enable-method = "psci";
+ next-level-cache = <&L2_600>;
+ power-domains = <&CPU_PD6>;
+ power-domain-names = "psci";
+ qcom,freq-domain = <&cpufreq_hw 1>;
+ L2_600: l2-cache {
+ compatible = "cache";
+ next-level-cache = <&L3_0>;
+ };
+ };
+
+ CPU7: cpu@700 {
+ device_type = "cpu";
+ compatible = "qcom,kryo780";
+ reg = <0x0 0x700>;
+ enable-method = "psci";
+ next-level-cache = <&L2_700>;
+ power-domains = <&CPU_PD7>;
+ power-domain-names = "psci";
+ qcom,freq-domain = <&cpufreq_hw 2>;
+ L2_700: l2-cache {
+ compatible = "cache";
+ next-level-cache = <&L3_0>;
+ };
+ };
+
+ cpu-map {
+ cluster0 {
+ core0 {
+ cpu = <&CPU0>;
+ };
+
+ core1 {
+ cpu = <&CPU1>;
+ };
+
+ core2 {
+ cpu = <&CPU2>;
+ };
+
+ core3 {
+ cpu = <&CPU3>;
+ };
+
+ core4 {
+ cpu = <&CPU4>;
+ };
+
+ core5 {
+ cpu = <&CPU5>;
+ };
+
+ core6 {
+ cpu = <&CPU6>;
+ };
+
+ core7 {
+ cpu = <&CPU7>;
+ };
+ };
+ };
+
+ idle-states {
+ entry-method = "psci";
+
+ LITTLE_CPU_SLEEP_0: cpu-sleep-0-0 {
+ compatible = "arm,idle-state";
+ idle-state-name = "silver-rail-power-collapse";
+ arm,psci-suspend-param = <0x40000004>;
+ entry-latency-us = <274>;
+ exit-latency-us = <480>;
+ min-residency-us = <3934>;
+ local-timer-stop;
+ };
+
+ BIG_CPU_SLEEP_0: cpu-sleep-1-0 {
+ compatible = "arm,idle-state";
+ idle-state-name = "gold-rail-power-collapse";
+ arm,psci-suspend-param = <0x40000004>;
+ entry-latency-us = <327>;
+ exit-latency-us = <1502>;
+ min-residency-us = <4488>;
+ local-timer-stop;
+ };
+ };
+
+ domain-idle-states {
+ CLUSTER_SLEEP_0: cluster-sleep-0 {
+ compatible = "domain-idle-state";
+ idle-state-name = "cluster-l3-off";
+ arm,psci-suspend-param = <0x4100c344>;
+ entry-latency-us = <584>;
+ exit-latency-us = <2332>;
+ min-residency-us = <6118>;
+ local-timer-stop;
+ };
+
+ CLUSTER_SLEEP_1: cluster-sleep-1 {
+ compatible = "domain-idle-state";
+ idle-state-name = "cluster-power-collapse";
+ arm,psci-suspend-param = <0x4100c344>;
+ entry-latency-us = <2893>;
+ exit-latency-us = <4023>;
+ min-residency-us = <9987>;
+ local-timer-stop;
+ };
+ };
+ };
+
+ firmware {
+ scm: scm {
+ compatible = "qcom,scm-sm8450", "qcom,scm";
+ #reset-cells = <1>;
+ };
+ };
+
+ memory@a0000000 {
+ device_type = "memory";
+ /* We expect the bootloader to fill in the size */
+ reg = <0x0 0xa0000000 0x0 0x0>;
+ };
+
+ pmu {
+ compatible = "arm,armv8-pmuv3";
+ interrupts = <GIC_PPI 7 IRQ_TYPE_LEVEL_HIGH>;
+ };
+
+ psci {
+ compatible = "arm,psci-1.0";
+ method = "smc";
+
+ CPU_PD0: cpu0 {
+ #power-domain-cells = <0>;
+ power-domains = <&CLUSTER_PD>;
+ domain-idle-states = <&LITTLE_CPU_SLEEP_0>;
+ };
+
+ CPU_PD1: cpu1 {
+ #power-domain-cells = <0>;
+ power-domains = <&CLUSTER_PD>;
+ domain-idle-states = <&LITTLE_CPU_SLEEP_0>;
+ };
+
+ CPU_PD2: cpu2 {
+ #power-domain-cells = <0>;
+ power-domains = <&CLUSTER_PD>;
+ domain-idle-states = <&LITTLE_CPU_SLEEP_0>;
+ };
+
+ CPU_PD3: cpu3 {
+ #power-domain-cells = <0>;
+ power-domains = <&CLUSTER_PD>;
+ domain-idle-states = <&LITTLE_CPU_SLEEP_0>;
+ };
+
+ CPU_PD4: cpu4 {
+ #power-domain-cells = <0>;
+ power-domains = <&CLUSTER_PD>;
+ domain-idle-states = <&BIG_CPU_SLEEP_0>;
+ };
+
+ CPU_PD5: cpu5 {
+ #power-domain-cells = <0>;
+ power-domains = <&CLUSTER_PD>;
+ domain-idle-states = <&BIG_CPU_SLEEP_0>;
+ };
+
+ CPU_PD6: cpu6 {
+ #power-domain-cells = <0>;
+ power-domains = <&CLUSTER_PD>;
+ domain-idle-states = <&BIG_CPU_SLEEP_0>;
+ };
+
+ CPU_PD7: cpu7 {
+ #power-domain-cells = <0>;
+ power-domains = <&CLUSTER_PD>;
+ domain-idle-states = <&BIG_CPU_SLEEP_0>;
+ };
+
+ CLUSTER_PD: cpu-cluster0 {
+ #power-domain-cells = <0>;
+ domain-idle-states = <&CLUSTER_SLEEP_0>;
+ };
+ };
+
+ reserved_memory: reserved-memory {
+ #address-cells = <2>;
+ #size-cells = <2>;
+ ranges;
+
+ hyp_mem: memory@80000000 {
+ reg = <0x0 0x80000000 0x0 0x600000>;
+ no-map;
+ };
+
+ xbl_dt_log_mem: memory@80600000 {
+ reg = <0x0 0x80600000 0x0 0x40000>;
+ no-map;
+ };
+
+ xbl_ramdump_mem: memory@80640000 {
+ reg = <0x0 0x80640000 0x0 0x180000>;
+ no-map;
+ };
+
+ xbl_sc_mem: memory@807c0000 {
+ reg = <0x0 0x807c0000 0x0 0x40000>;
+ no-map;
+ };
+
+ aop_image_mem: memory@80800000 {
+ reg = <0x0 0x80800000 0x0 0x60000>;
+ no-map;
+ };
+
+ aop_cmd_db_mem: memory@80860000 {
+ compatible = "qcom,cmd-db";
+ reg = <0x0 0x80860000 0x0 0x20000>;
+ no-map;
+ };
+
+ aop_config_mem: memory@80880000 {
+ reg = <0x0 0x80880000 0x0 0x20000>;
+ no-map;
+ };
+
+ tme_crash_dump_mem: memory@808a0000 {
+ reg = <0x0 0x808a0000 0x0 0x40000>;
+ no-map;
+ };
+
+ tme_log_mem: memory@808e0000 {
+ reg = <0x0 0x808e0000 0x0 0x4000>;
+ no-map;
+ };
+
+ uefi_log_mem: memory@808e4000 {
+ reg = <0x0 0x808e4000 0x0 0x10000>;
+ no-map;
+ };
+
+ /* secdata region can be reused by apps */
+ smem: memory@80900000 {
+ compatible = "qcom,smem";
+ reg = <0x0 0x80900000 0x0 0x200000>;
+ hwlocks = <&tcsr_mutex 3>;
+ no-map;
+ };
+
+ cpucp_fw_mem: memory@80b00000 {
+ reg = <0x0 0x80b00000 0x0 0x100000>;
+ no-map;
+ };
+
+ cdsp_secure_heap: memory@80c00000 {
+ reg = <0x0 0x80c00000 0x0 0x4600000>;
+ no-map;
+ };
+
+ camera_mem: memory@85200000 {
+ reg = <0x0 0x85200000 0x0 0x500000>;
+ no-map;
+ };
+
+ video_mem: memory@85700000 {
+ reg = <0x0 0x85700000 0x0 0x700000>;
+ no-map;
+ };
+
+ adsp_mem: memory@85e00000 {
+ reg = <0x0 0x85e00000 0x0 0x2100000>;
+ no-map;
+ };
+
+ slpi_mem: memory@88000000 {
+ reg = <0x0 0x88000000 0x0 0x1900000>;
+ no-map;
+ };
+
+ cdsp_mem: memory@89900000 {
+ reg = <0x0 0x89900000 0x0 0x2000000>;
+ no-map;
+ };
+
+ ipa_fw_mem: memory@8b900000 {
+ reg = <0x0 0x8b900000 0x0 0x10000>;
+ no-map;
+ };
+
+ ipa_gsi_mem: memory@8b910000 {
+ reg = <0x0 0x8b910000 0x0 0xa000>;
+ no-map;
+ };
+
+ gpu_micro_code_mem: memory@8b91a000 {
+ reg = <0x0 0x8b91a000 0x0 0x2000>;
+ no-map;
+ };
+
+ spss_region_mem: memory@8ba00000 {
+ reg = <0x0 0x8ba00000 0x0 0x180000>;
+ no-map;
+ };
+
+ /* First part of the "SPU secure shared memory" region */
+ spu_tz_shared_mem: memory@8bb80000 {
+ reg = <0x0 0x8bb80000 0x0 0x60000>;
+ no-map;
+ };
+
+ /* Second part of the "SPU secure shared memory" region */
+ spu_modem_shared_mem: memory@8bbe0000 {
+ reg = <0x0 0x8bbe0000 0x0 0x20000>;
+ no-map;
+ };
+
+ mpss_mem: memory@8bc00000 {
+ reg = <0x0 0x8bc00000 0x0 0x13200000>;
+ no-map;
+ };
+
+ cvp_mem: memory@9ee00000 {
+ reg = <0x0 0x9ee00000 0x0 0x700000>;
+ no-map;
+ };
+
+ global_sync_mem: memory@a6f00000 {
+ reg = <0x0 0xa6f00000 0x0 0x100000>;
+ no-map;
+ };
+
+ /* uefi region can be reused by APPS */
+
+ /* Linux kernel image is loaded at 0xa0000000 */
+
+ oem_vm_mem: memory@bb000000 {
+ reg = <0x0 0xbb000000 0x0 0x5000000>;
+ no-map;
+ };
+
+ mte_mem: memory@c0000000 {
+ reg = <0x0 0xc0000000 0x0 0x20000000>;
+ no-map;
+ };
+
+ qheebsp_reserved_mem: memory@e0000000 {
+ reg = <0x0 0xe0000000 0x0 0x600000>;
+ no-map;
+ };
+
+ cpusys_vm_mem: memory@e0600000 {
+ reg = <0x0 0xe0600000 0x0 0x400000>;
+ no-map;
+ };
+
+ hyp_reserved_mem: memory@e0a00000 {
+ reg = <0x0 0xe0a00000 0x0 0x100000>;
+ no-map;
+ };
+
+ trust_ui_vm_mem: memory@e0b00000 {
+ reg = <0x0 0xe0b00000 0x0 0x4af3000>;
+ no-map;
+ };
+
+ trust_ui_vm_qrtr: memory@e55f3000 {
+ reg = <0x0 0xe55f3000 0x0 0x9000>;
+ no-map;
+ };
+
+ trust_ui_vm_vblk0_ring: memory@e55fc000 {
+ reg = <0x0 0xe55fc000 0x0 0x4000>;
+ no-map;
+ };
+
+ trust_ui_vm_swiotlb: memory@e5600000 {
+ reg = <0x0 0xe5600000 0x0 0x100000>;
+ no-map;
+ };
+
+ tz_stat_mem: memory@e8800000 {
+ reg = <0x0 0xe8800000 0x0 0x100000>;
+ no-map;
+ };
+
+ tags_mem: memory@e8900000 {
+ reg = <0x0 0xe8900000 0x0 0x1200000>;
+ no-map;
+ };
+
+ qtee_mem: memory@e9b00000 {
+ reg = <0x0 0xe9b00000 0x0 0x500000>;
+ no-map;
+ };
+
+ trusted_apps_mem: memory@ea000000 {
+ reg = <0x0 0xea000000 0x0 0x3900000>;
+ no-map;
+ };
+
+ trusted_apps_ext_mem: memory@ed900000 {
+ reg = <0x0 0xed900000 0x0 0x3b00000>;
+ no-map;
+ };
+ };
+
+ soc: soc@0 {
+ #address-cells = <2>;
+ #size-cells = <2>;
+ ranges = <0 0 0 0 0x10 0>;
+ dma-ranges = <0 0 0 0 0x10 0>;
+ compatible = "simple-bus";
+
+ gcc: clock-controller@100000 {
+ compatible = "qcom,gcc-sm8450";
+ reg = <0x0 0x00100000 0x0 0x1f4200>;
+ #clock-cells = <1>;
+ #reset-cells = <1>;
+ #power-domain-cells = <1>;
+ clock-names = "bi_tcxo", "sleep_clk";
+ clocks = <&rpmhcc RPMH_CXO_CLK>, <&sleep_clk>;
+ };
+
+ qupv3_id_0: geniqup@9c0000 {
+ compatible = "qcom,geni-se-qup";
+ reg = <0x0 0x009c0000 0x0 0x2000>;
+ clock-names = "m-ahb", "s-ahb";
+ clocks = <&gcc GCC_QUPV3_WRAP_0_M_AHB_CLK>,
+ <&gcc GCC_QUPV3_WRAP_0_S_AHB_CLK>;
+ #address-cells = <2>;
+ #size-cells = <2>;
+ ranges;
+ status = "disabled";
+
+ uart7: serial@99c000 {
+ compatible = "qcom,geni-debug-uart";
+ reg = <0 0x0099c000 0 0x4000>;
+ clock-names = "se";
+ clocks = <&gcc GCC_QUPV3_WRAP0_S7_CLK>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&qup_uart7_tx>, <&qup_uart7_rx>;
+ interrupts = <GIC_SPI 608 IRQ_TYPE_LEVEL_HIGH>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+ };
+
+ qupv3_id_1: geniqup@ac0000 {
+ compatible = "qcom,geni-se-qup";
+ reg = <0x0 0x00ac0000 0x0 0x6000>;
+ clock-names = "m-ahb", "s-ahb";
+ clocks = <&gcc GCC_QUPV3_WRAP_1_M_AHB_CLK>,
+ <&gcc GCC_QUPV3_WRAP_1_S_AHB_CLK>;
+ #address-cells = <2>;
+ #size-cells = <2>;
+ ranges;
+ status = "disabled";
+
+ i2c13: i2c@a94000 {
+ compatible = "qcom,geni-i2c";
+ reg = <0 0x00a94000 0 0x4000>;
+ clock-names = "se";
+ clocks = <&gcc GCC_QUPV3_WRAP1_S5_CLK>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&qup_i2c13_data_clk>;
+ interrupts = <GIC_SPI 358 IRQ_TYPE_LEVEL_HIGH>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ i2c14: i2c@a98000 {
+ compatible = "qcom,geni-i2c";
+ reg = <0 0x00a98000 0 0x4000>;
+ clock-names = "se";
+ clocks = <&gcc GCC_QUPV3_WRAP1_S6_CLK>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&qup_i2c14_data_clk>;
+ interrupts = <GIC_SPI 363 IRQ_TYPE_LEVEL_HIGH>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+ };
+
+ tcsr_mutex: hwlock@1f40000 {
+ compatible = "qcom,tcsr-mutex";
+ reg = <0x0 0x01f40000 0x0 0x40000>;
+ #hwlock-cells = <1>;
+ };
+
+ usb_1_hsphy: phy@88e3000 {
+ compatible = "qcom,sm8450-usb-hs-phy",
+ "qcom,usb-snps-hs-7nm-phy";
+ reg = <0 0x088e3000 0 0x400>;
+ status = "disabled";
+ #phy-cells = <0>;
+
+ clocks = <&rpmhcc RPMH_CXO_CLK>;
+ clock-names = "ref";
+
+ resets = <&gcc GCC_QUSB2PHY_PRIM_BCR>;
+ };
+
+ usb_1_qmpphy: phy-wrapper@88e9000 {
+ compatible = "qcom,sm8450-qmp-usb3-phy";
+ reg = <0 0x088e9000 0 0x200>,
+ <0 0x088e8000 0 0x20>;
+ status = "disabled";
+ #address-cells = <2>;
+ #size-cells = <2>;
+ ranges;
+
+ clocks = <&gcc GCC_USB3_PRIM_PHY_AUX_CLK>,
+ <&rpmhcc RPMH_CXO_CLK>,
+ <&gcc GCC_USB3_PRIM_PHY_COM_AUX_CLK>;
+ clock-names = "aux", "ref_clk_src", "com_aux";
+
+ resets = <&gcc GCC_USB3_DP_PHY_PRIM_BCR>,
+ <&gcc GCC_USB3_PHY_PRIM_BCR>;
+ reset-names = "phy", "common";
+
+ usb_1_ssphy: phy@88e9200 {
+ reg = <0 0x088e9200 0 0x200>,
+ <0 0x088e9400 0 0x200>,
+ <0 0x088e9c00 0 0x400>,
+ <0 0x088e9600 0 0x200>,
+ <0 0x088e9800 0 0x200>,
+ <0 0x088e9a00 0 0x100>;
+ #phy-cells = <0>;
+ #clock-cells = <1>;
+ clocks = <&gcc GCC_USB3_PRIM_PHY_PIPE_CLK>;
+ clock-names = "pipe0";
+ clock-output-names = "usb3_phy_pipe_clk_src";
+ };
+ };
+
+ pdc: interrupt-controller@b220000 {
+ compatible = "qcom,sm8450-pdc", "qcom,pdc";
+ reg = <0 0x0b220000 0 0x30000>, <0 0x174000f0 0 0x64>;
+ qcom,pdc-ranges = <0 480 12>, <14 494 24>, <40 520 54>,
+ <94 609 31>, <125 63 1>, <126 716 12>;
+ #interrupt-cells = <2>;
+ interrupt-parent = <&intc>;
+ interrupt-controller;
+ };
+
+ tlmm: pinctrl@f100000 {
+ compatible = "qcom,sm8450-tlmm";
+ reg = <0 0x0f100000 0 0x300000>;
+ interrupts = <GIC_SPI 208 IRQ_TYPE_LEVEL_HIGH>;
+ gpio-controller;
+ #gpio-cells = <2>;
+ interrupt-controller;
+ #interrupt-cells = <2>;
+ gpio-ranges = <&tlmm 0 0 211>;
+ wakeup-parent = <&pdc>;
+
+ qup_i2c13_data_clk: qup-i2c13-data-clk {
+ pins = "gpio48", "gpio49";
+ function = "qup13";
+ drive-strength = <2>;
+ bias-pull-up;
+ };
+
+ qup_i2c14_data_clk: qup-i2c14-data-clk {
+ pins = "gpio52", "gpio53";
+ function = "qup14";
+ drive-strength = <2>;
+ bias-pull-up;
+ };
+
+ qup_uart7_rx: qup-uart7-rx {
+ pins = "gpio26";
+ function = "qup7";
+ drive-strength = <2>;
+ bias-disable;
+ };
+
+ qup_uart7_tx: qup-uart7-tx {
+ pins = "gpio27";
+ function = "qup7";
+ drive-strength = <2>;
+ bias-disable;
+ };
+ };
+
+ apps_smmu: iommu@15000000 {
+ compatible = "qcom,sm8450-smmu-500", "arm,mmu-500";
+ reg = <0 0x15000000 0 0x100000>;
+ #iommu-cells = <2>;
+ #global-interrupts = <2>;
+ interrupts = <GIC_SPI 65 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 97 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 98 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 99 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 100 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 101 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 102 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 103 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 104 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 105 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 106 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 107 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 108 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 109 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 110 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 111 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 112 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 113 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 114 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 115 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 116 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 117 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 118 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 181 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 182 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 183 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 184 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 185 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 186 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 187 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 188 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 189 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 190 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 191 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 192 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 315 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 316 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 317 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 318 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 319 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 320 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 321 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 322 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 323 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 324 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 325 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 326 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 327 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 328 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 329 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 330 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 331 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 332 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 333 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 334 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 335 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 336 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 337 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 338 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 339 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 340 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 341 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 342 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 343 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 344 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 345 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 395 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 396 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 397 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 398 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 399 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 400 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 401 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 402 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 403 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 404 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 405 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 406 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 407 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 408 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 409 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 418 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 419 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 412 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 421 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 707 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 424 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 425 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 690 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 691 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 692 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 693 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 694 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 695 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 696 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 697 IRQ_TYPE_LEVEL_HIGH>;
+ };
+
+ intc: interrupt-controller@17100000 {
+ compatible = "arm,gic-v3";
+ #interrupt-cells = <3>;
+ interrupt-controller;
+ #redistributor-regions = <1>;
+ redistributor-stride = <0x0 0x40000>;
+ reg = <0x0 0x17100000 0x0 0x10000>, /* GICD */
+ <0x0 0x17180000 0x0 0x200000>; /* GICR * 8 */
+ interrupts = <GIC_PPI 9 IRQ_TYPE_LEVEL_HIGH>;
+ };
+
+ timer@17420000 {
+ compatible = "arm,armv7-timer-mem";
+ #address-cells = <2>;
+ #size-cells = <2>;
+ ranges;
+ reg = <0x0 0x17420000 0x0 0x1000>;
+ clock-frequency = <19200000>;
+
+ frame@17421000 {
+ frame-number = <0>;
+ interrupts = <GIC_SPI 8 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 6 IRQ_TYPE_LEVEL_HIGH>;
+ reg = <0x0 0x17421000 0x0 0x1000>,
+ <0x0 0x17422000 0x0 0x1000>;
+ };
+
+ frame@17423000 {
+ frame-number = <1>;
+ interrupts = <GIC_SPI 9 IRQ_TYPE_LEVEL_HIGH>;
+ reg = <0x0 0x17423000 0x0 0x1000>;
+ status = "disabled";
+ };
+
+ frame@17425000 {
+ frame-number = <2>;
+ interrupts = <GIC_SPI 10 IRQ_TYPE_LEVEL_HIGH>;
+ reg = <0x0 0x17425000 0x0 0x1000>;
+ status = "disabled";
+ };
+
+ frame@17427000 {
+ frame-number = <3>;
+ interrupts = <GIC_SPI 11 IRQ_TYPE_LEVEL_HIGH>;
+ reg = <0x0 0x17427000 0x0 0x1000>;
+ status = "disabled";
+ };
+
+ frame@17429000 {
+ frame-number = <4>;
+ interrupts = <GIC_SPI 12 IRQ_TYPE_LEVEL_HIGH>;
+ reg = <0x0 0x17429000 0x0 0x1000>;
+ status = "disabled";
+ };
+
+ frame@1742b000 {
+ frame-number = <5>;
+ interrupts = <GIC_SPI 13 IRQ_TYPE_LEVEL_HIGH>;
+ reg = <0x0 0x1742b000 0x0 0x1000>;
+ status = "disabled";
+ };
+
+ frame@1742d000 {
+ frame-number = <6>;
+ interrupts = <GIC_SPI 14 IRQ_TYPE_LEVEL_HIGH>;
+ reg = <0x0 0x1742d000 0x0 0x1000>;
+ status = "disabled";
+ };
+ };
+
+ apps_rsc: rsc@17a00000 {
+ label = "apps_rsc";
+ compatible = "qcom,rpmh-rsc";
+ reg = <0x0 0x17a00000 0x0 0x10000>,
+ <0x0 0x17a10000 0x0 0x10000>,
+ <0x0 0x17a20000 0x0 0x10000>,
+ <0x0 0x17a30000 0x0 0x10000>;
+ reg-names = "drv-0", "drv-1", "drv-2", "drv-3";
+ interrupts = <GIC_SPI 3 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 4 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 5 IRQ_TYPE_LEVEL_HIGH>;
+ qcom,tcs-offset = <0xd00>;
+ qcom,drv-id = <2>;
+ qcom,tcs-config = <ACTIVE_TCS 3>, <SLEEP_TCS 2>,
+ <WAKE_TCS 2>, <CONTROL_TCS 0>;
+
+ apps_bcm_voter: bcm-voter {
+ compatible = "qcom,bcm-voter";
+ };
+
+ rpmhcc: clock-controller {
+ compatible = "qcom,sm8450-rpmh-clk";
+ #clock-cells = <1>;
+ clock-names = "xo";
+ clocks = <&xo_board>;
+ };
+
+ rpmhpd: power-controller {
+ compatible = "qcom,sm8450-rpmhpd";
+ #power-domain-cells = <1>;
+ operating-points-v2 = <&rpmhpd_opp_table>;
+
+ rpmhpd_opp_table: opp-table {
+ compatible = "operating-points-v2";
+
+ rpmhpd_opp_ret: opp1 {
+ opp-level = <RPMH_REGULATOR_LEVEL_RETENTION>;
+ };
+
+ rpmhpd_opp_min_svs: opp2 {
+ opp-level = <RPMH_REGULATOR_LEVEL_MIN_SVS>;
+ };
+
+ rpmhpd_opp_low_svs: opp3 {
+ opp-level = <RPMH_REGULATOR_LEVEL_LOW_SVS>;
+ };
+
+ rpmhpd_opp_svs: opp4 {
+ opp-level = <RPMH_REGULATOR_LEVEL_SVS>;
+ };
+
+ rpmhpd_opp_svs_l1: opp5 {
+ opp-level = <RPMH_REGULATOR_LEVEL_SVS_L1>;
+ };
+
+ rpmhpd_opp_nom: opp6 {
+ opp-level = <RPMH_REGULATOR_LEVEL_NOM>;
+ };
+
+ rpmhpd_opp_nom_l1: opp7 {
+ opp-level = <RPMH_REGULATOR_LEVEL_NOM_L1>;
+ };
+
+ rpmhpd_opp_nom_l2: opp8 {
+ opp-level = <RPMH_REGULATOR_LEVEL_NOM_L2>;
+ };
+
+ rpmhpd_opp_turbo: opp9 {
+ opp-level = <RPMH_REGULATOR_LEVEL_TURBO>;
+ };
+
+ rpmhpd_opp_turbo_l1: opp10 {
+ opp-level = <RPMH_REGULATOR_LEVEL_TURBO_L1>;
+ };
+ };
+ };
+ };
+
+ cpufreq_hw: cpufreq@17d91000 {
+ compatible = "qcom,sm8450-cpufreq-epss", "qcom,cpufreq-epss";
+ reg = <0 0x17d91000 0 0x1000>,
+ <0 0x17d92000 0 0x1000>,
+ <0 0x17d93000 0 0x1000>;
+ reg-names = "freq-domain0", "freq-domain1", "freq-domain2";
+ clocks = <&rpmhcc RPMH_CXO_CLK>, <&gcc GCC_GPLL0>;
+ clock-names = "xo", "alternate";
+ interrupts = <GIC_SPI 30 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 31 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 19 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "dcvsh-irq-0", "dcvsh-irq-1", "dcvsh-irq-2";
+ #freq-domain-cells = <1>;
+ };
+
+ ufs_mem_hc: ufshc@1d84000 {
+ compatible = "qcom,sm8450-ufshc", "qcom,ufshc",
+ "jedec,ufs-2.0";
+ reg = <0 0x01d84000 0 0x3000>;
+ interrupts = <GIC_SPI 265 IRQ_TYPE_LEVEL_HIGH>;
+ phys = <&ufs_mem_phy_lanes>;
+ phy-names = "ufsphy";
+ lanes-per-direction = <2>;
+ #reset-cells = <1>;
+ resets = <&gcc GCC_UFS_PHY_BCR>;
+ reset-names = "rst";
+
+ power-domains = <&gcc UFS_PHY_GDSC>;
+
+ iommus = <&apps_smmu 0xe0 0x0>;
+
+ clock-names =
+ "core_clk",
+ "bus_aggr_clk",
+ "iface_clk",
+ "core_clk_unipro",
+ "ref_clk",
+ "tx_lane0_sync_clk",
+ "rx_lane0_sync_clk",
+ "rx_lane1_sync_clk";
+ clocks =
+ <&gcc GCC_UFS_PHY_AXI_CLK>,
+ <&gcc GCC_AGGRE_UFS_PHY_AXI_CLK>,
+ <&gcc GCC_UFS_PHY_AHB_CLK>,
+ <&gcc GCC_UFS_PHY_UNIPRO_CORE_CLK>,
+ <&rpmhcc RPMH_CXO_CLK>,
+ <&gcc GCC_UFS_PHY_TX_SYMBOL_0_CLK>,
+ <&gcc GCC_UFS_PHY_RX_SYMBOL_0_CLK>,
+ <&gcc GCC_UFS_PHY_RX_SYMBOL_1_CLK>;
+ freq-table-hz =
+ <75000000 300000000>,
+ <0 0>,
+ <0 0>,
+ <75000000 300000000>,
+ <75000000 300000000>,
+ <0 0>,
+ <0 0>,
+ <0 0>;
+ status = "disabled";
+ };
+
+ ufs_mem_phy: phy@1d87000 {
+ compatible = "qcom,sm8450-qmp-ufs-phy";
+ reg = <0 0x01d87000 0 0xe10>;
+ #address-cells = <2>;
+ #size-cells = <2>;
+ ranges;
+ clock-names = "ref", "ref_aux", "qref";
+ clocks = <&rpmhcc RPMH_CXO_CLK>,
+ <&gcc GCC_UFS_PHY_PHY_AUX_CLK>,
+ <&gcc GCC_UFS_0_CLKREF_EN>;
+
+ resets = <&ufs_mem_hc 0>;
+ reset-names = "ufsphy";
+ status = "disabled";
+
+ ufs_mem_phy_lanes: lanes@1d87400 {
+ reg = <0 0x01d87400 0 0x108>,
+ <0 0x01d87600 0 0x1e0>,
+ <0 0x01d87c00 0 0x1dc>,
+ <0 0x01d87800 0 0x108>,
+ <0 0x01d87a00 0 0x1e0>;
+ #phy-cells = <0>;
+ #clock-cells = <0>;
+ };
+ };
+
+ usb_1: usb@a6f8800 {
+ compatible = "qcom,sm8450-dwc3", "qcom,dwc3";
+ reg = <0 0x0a6f8800 0 0x400>;
+ status = "disabled";
+ #address-cells = <2>;
+ #size-cells = <2>;
+ ranges;
+
+ clocks = <&gcc GCC_CFG_NOC_USB3_PRIM_AXI_CLK>,
+ <&gcc GCC_USB30_PRIM_MASTER_CLK>,
+ <&gcc GCC_AGGRE_USB3_PRIM_AXI_CLK>,
+ <&gcc GCC_USB30_PRIM_MOCK_UTMI_CLK>,
+ <&gcc GCC_USB30_PRIM_SLEEP_CLK>;
+ clock-names = "cfg_noc", "core", "iface", "mock_utmi",
+ "sleep";
+
+ assigned-clocks = <&gcc GCC_USB30_PRIM_MOCK_UTMI_CLK>,
+ <&gcc GCC_USB30_PRIM_MASTER_CLK>;
+ assigned-clock-rates = <19200000>, <200000000>;
+
+ interrupts-extended = <&intc GIC_SPI 130 IRQ_TYPE_LEVEL_HIGH>,
+ <&pdc 14 IRQ_TYPE_EDGE_BOTH>,
+ <&pdc 15 IRQ_TYPE_EDGE_BOTH>,
+ <&pdc 17 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "hs_phy_irq", "dp_hs_phy_irq",
+ "dm_hs_phy_irq", "ss_phy_irq";
+
+ power-domains = <&gcc USB30_PRIM_GDSC>;
+
+ resets = <&gcc GCC_USB30_PRIM_BCR>;
+
+ usb_1_dwc3: usb@a600000 {
+ compatible = "snps,dwc3";
+ reg = <0 0x0a600000 0 0xcd00>;
+ interrupts = <GIC_SPI 133 IRQ_TYPE_LEVEL_HIGH>;
+ iommus = <&apps_smmu 0x0 0x0>;
+ snps,dis_u2_susphy_quirk;
+ snps,dis_enblslpm_quirk;
+ phys = <&usb_1_hsphy>, <&usb_1_ssphy>;
+ phy-names = "usb2-phy", "usb3-phy";
+ };
+ };
+ };
+
+ timer {
+ compatible = "arm,armv8-timer";
+ interrupts = <GIC_PPI 13 (GIC_CPU_MASK_SIMPLE(8) | IRQ_TYPE_LEVEL_LOW)>,
+ <GIC_PPI 14 (GIC_CPU_MASK_SIMPLE(8) | IRQ_TYPE_LEVEL_LOW)>,
+ <GIC_PPI 11 (GIC_CPU_MASK_SIMPLE(8) | IRQ_TYPE_LEVEL_LOW)>,
+ <GIC_PPI 10 (GIC_CPU_MASK_SIMPLE(8) | IRQ_TYPE_LEVEL_LOW)>;
+ clock-frequency = <19200000>;
+ };
+};
diff --git a/arch/arm64/boot/dts/renesas/Makefile b/arch/arm64/boot/dts/renesas/Makefile
index d1c5c21..5bc8065 100644
--- a/arch/arm64/boot/dts/renesas/Makefile
+++ b/arch/arm64/boot/dts/renesas/Makefile
@@ -63,6 +63,8 @@
dtb-$(CONFIG_ARCH_R8A779A0) += r8a779a0-falcon.dtb
+dtb-$(CONFIG_ARCH_R8A779F0) += r8a779f0-spider.dtb
+
dtb-$(CONFIG_ARCH_R8A77951) += r8a779m1-salvator-xs.dtb
dtb-$(CONFIG_ARCH_R8A77951) += r8a779m1-ulcb.dtb
dtb-$(CONFIG_ARCH_R8A77951) += r8a779m1-ulcb-kf.dtb
diff --git a/arch/arm64/boot/dts/renesas/cat875.dtsi b/arch/arm64/boot/dts/renesas/cat875.dtsi
index a69d24e..8c9da8b 100644
--- a/arch/arm64/boot/dts/renesas/cat875.dtsi
+++ b/arch/arm64/boot/dts/renesas/cat875.dtsi
@@ -18,6 +18,7 @@ &avb {
pinctrl-names = "default";
renesas,no-ether-link;
phy-handle = <&phy0>;
+ phy-mode = "rgmii-id";
status = "okay";
phy0: ethernet-phy@0 {
diff --git a/arch/arm64/boot/dts/renesas/r8a774a1.dtsi b/arch/arm64/boot/dts/renesas/r8a774a1.dtsi
index 6f4fffa..eda6a84 100644
--- a/arch/arm64/boot/dts/renesas/r8a774a1.dtsi
+++ b/arch/arm64/boot/dts/renesas/r8a774a1.dtsi
@@ -58,7 +58,7 @@ can_clk: can {
clock-frequency = <0>;
};
- cluster0_opp: opp_table0 {
+ cluster0_opp: opp-table-0 {
compatible = "operating-points-v2";
opp-shared;
@@ -80,7 +80,7 @@ opp-1500000000 {
};
};
- cluster1_opp: opp_table1 {
+ cluster1_opp: opp-table-1 {
compatible = "operating-points-v2";
opp-shared;
@@ -2276,7 +2276,8 @@ sdhi0: mmc@ee100000 {
"renesas,rcar-gen3-sdhi";
reg = <0 0xee100000 0 0x2000>;
interrupts = <GIC_SPI 165 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&cpg CPG_MOD 314>;
+ clocks = <&cpg CPG_MOD 314>, <&cpg CPG_CORE R8A774A1_CLK_SD0H>;
+ clock-names = "core", "clkh";
max-frequency = <200000000>;
power-domains = <&sysc R8A774A1_PD_ALWAYS_ON>;
resets = <&cpg 314>;
@@ -2288,7 +2289,8 @@ sdhi1: mmc@ee120000 {
"renesas,rcar-gen3-sdhi";
reg = <0 0xee120000 0 0x2000>;
interrupts = <GIC_SPI 166 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&cpg CPG_MOD 313>;
+ clocks = <&cpg CPG_MOD 313>, <&cpg CPG_CORE R8A774A1_CLK_SD1H>;
+ clock-names = "core", "clkh";
max-frequency = <200000000>;
power-domains = <&sysc R8A774A1_PD_ALWAYS_ON>;
resets = <&cpg 313>;
@@ -2300,7 +2302,8 @@ sdhi2: mmc@ee140000 {
"renesas,rcar-gen3-sdhi";
reg = <0 0xee140000 0 0x2000>;
interrupts = <GIC_SPI 167 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&cpg CPG_MOD 312>;
+ clocks = <&cpg CPG_MOD 312>, <&cpg CPG_CORE R8A774A1_CLK_SD2H>;
+ clock-names = "core", "clkh";
max-frequency = <200000000>;
power-domains = <&sysc R8A774A1_PD_ALWAYS_ON>;
resets = <&cpg 312>;
@@ -2312,7 +2315,8 @@ sdhi3: mmc@ee160000 {
"renesas,rcar-gen3-sdhi";
reg = <0 0xee160000 0 0x2000>;
interrupts = <GIC_SPI 168 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&cpg CPG_MOD 311>;
+ clocks = <&cpg CPG_MOD 311>, <&cpg CPG_CORE R8A774A1_CLK_SD3H>;
+ clock-names = "core", "clkh";
max-frequency = <200000000>;
power-domains = <&sysc R8A774A1_PD_ALWAYS_ON>;
resets = <&cpg 311>;
@@ -2784,7 +2788,7 @@ prr: chipid@fff00044 {
};
thermal-zones {
- sensor_thermal1: sensor-thermal1 {
+ sensor1_thermal: sensor1-thermal {
polling-delay-passive = <250>;
polling-delay = <1000>;
thermal-sensors = <&tsc 0>;
@@ -2799,7 +2803,7 @@ sensor1_crit: sensor1-crit {
};
};
- sensor_thermal2: sensor-thermal2 {
+ sensor2_thermal: sensor2-thermal {
polling-delay-passive = <250>;
polling-delay = <1000>;
thermal-sensors = <&tsc 1>;
@@ -2814,7 +2818,7 @@ sensor2_crit: sensor2-crit {
};
};
- sensor_thermal3: sensor-thermal3 {
+ sensor3_thermal: sensor3-thermal {
polling-delay-passive = <250>;
polling-delay = <1000>;
thermal-sensors = <&tsc 2>;
diff --git a/arch/arm64/boot/dts/renesas/r8a774b1.dtsi b/arch/arm64/boot/dts/renesas/r8a774b1.dtsi
index 0f7bdfc..44f79fb 100644
--- a/arch/arm64/boot/dts/renesas/r8a774b1.dtsi
+++ b/arch/arm64/boot/dts/renesas/r8a774b1.dtsi
@@ -47,7 +47,7 @@ can_clk: can {
clock-frequency = <0>;
};
- cluster0_opp: opp_table0 {
+ cluster0_opp: opp-table-0 {
compatible = "operating-points-v2";
opp-shared;
@@ -2133,7 +2133,8 @@ sdhi0: mmc@ee100000 {
"renesas,rcar-gen3-sdhi";
reg = <0 0xee100000 0 0x2000>;
interrupts = <GIC_SPI 165 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&cpg CPG_MOD 314>;
+ clocks = <&cpg CPG_MOD 314>, <&cpg CPG_CORE R8A774B1_CLK_SD0H>;
+ clock-names = "core", "clkh";
max-frequency = <200000000>;
power-domains = <&sysc R8A774B1_PD_ALWAYS_ON>;
resets = <&cpg 314>;
@@ -2145,7 +2146,8 @@ sdhi1: mmc@ee120000 {
"renesas,rcar-gen3-sdhi";
reg = <0 0xee120000 0 0x2000>;
interrupts = <GIC_SPI 166 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&cpg CPG_MOD 313>;
+ clocks = <&cpg CPG_MOD 313>, <&cpg CPG_CORE R8A774B1_CLK_SD1H>;
+ clock-names = "core", "clkh";
max-frequency = <200000000>;
power-domains = <&sysc R8A774B1_PD_ALWAYS_ON>;
resets = <&cpg 313>;
@@ -2157,7 +2159,8 @@ sdhi2: mmc@ee140000 {
"renesas,rcar-gen3-sdhi";
reg = <0 0xee140000 0 0x2000>;
interrupts = <GIC_SPI 167 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&cpg CPG_MOD 312>;
+ clocks = <&cpg CPG_MOD 312>, <&cpg CPG_CORE R8A774B1_CLK_SD2H>;
+ clock-names = "core", "clkh";
max-frequency = <200000000>;
power-domains = <&sysc R8A774B1_PD_ALWAYS_ON>;
resets = <&cpg 312>;
@@ -2169,7 +2172,8 @@ sdhi3: mmc@ee160000 {
"renesas,rcar-gen3-sdhi";
reg = <0 0xee160000 0 0x2000>;
interrupts = <GIC_SPI 168 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&cpg CPG_MOD 311>;
+ clocks = <&cpg CPG_MOD 311>, <&cpg CPG_CORE R8A774B1_CLK_SD3H>;
+ clock-names = "core", "clkh";
max-frequency = <200000000>;
power-domains = <&sysc R8A774B1_PD_ALWAYS_ON>;
resets = <&cpg 311>;
@@ -2629,7 +2633,7 @@ prr: chipid@fff00044 {
};
thermal-zones {
- sensor_thermal1: sensor-thermal1 {
+ sensor1_thermal: sensor1-thermal {
polling-delay-passive = <250>;
polling-delay = <1000>;
thermal-sensors = <&tsc 0>;
@@ -2644,7 +2648,7 @@ sensor1_crit: sensor1-crit {
};
};
- sensor_thermal2: sensor-thermal2 {
+ sensor2_thermal: sensor2-thermal {
polling-delay-passive = <250>;
polling-delay = <1000>;
thermal-sensors = <&tsc 1>;
@@ -2659,7 +2663,7 @@ sensor2_crit: sensor2-crit {
};
};
- sensor_thermal3: sensor-thermal3 {
+ sensor3_thermal: sensor3-thermal {
polling-delay-passive = <250>;
polling-delay = <1000>;
thermal-sensors = <&tsc 2>;
diff --git a/arch/arm64/boot/dts/renesas/r8a774c0.dtsi b/arch/arm64/boot/dts/renesas/r8a774c0.dtsi
index d597772..b8dcbbb 100644
--- a/arch/arm64/boot/dts/renesas/r8a774c0.dtsi
+++ b/arch/arm64/boot/dts/renesas/r8a774c0.dtsi
@@ -44,7 +44,7 @@ can_clk: can {
clock-frequency = <0>;
};
- cluster1_opp: opp_table10 {
+ cluster1_opp: opp-table-1 {
compatible = "operating-points-v2";
opp-shared;
opp-800000000 {
@@ -1626,7 +1626,8 @@ sdhi0: mmc@ee100000 {
"renesas,rcar-gen3-sdhi";
reg = <0 0xee100000 0 0x2000>;
interrupts = <GIC_SPI 165 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&cpg CPG_MOD 314>;
+ clocks = <&cpg CPG_MOD 314>, <&cpg CPG_CORE R8A774C0_CLK_SD0H>;
+ clock-names = "core", "clkh";
max-frequency = <200000000>;
power-domains = <&sysc R8A774C0_PD_ALWAYS_ON>;
resets = <&cpg 314>;
@@ -1638,7 +1639,8 @@ sdhi1: mmc@ee120000 {
"renesas,rcar-gen3-sdhi";
reg = <0 0xee120000 0 0x2000>;
interrupts = <GIC_SPI 166 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&cpg CPG_MOD 313>;
+ clocks = <&cpg CPG_MOD 313>, <&cpg CPG_CORE R8A774C0_CLK_SD1H>;
+ clock-names = "core", "clkh";
max-frequency = <200000000>;
power-domains = <&sysc R8A774C0_PD_ALWAYS_ON>;
resets = <&cpg 313>;
@@ -1650,7 +1652,8 @@ sdhi3: mmc@ee160000 {
"renesas,rcar-gen3-sdhi";
reg = <0 0xee160000 0 0x2000>;
interrupts = <GIC_SPI 168 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&cpg CPG_MOD 311>;
+ clocks = <&cpg CPG_MOD 311>, <&cpg CPG_CORE R8A774C0_CLK_SD3H>;
+ clock-names = "core", "clkh";
max-frequency = <200000000>;
power-domains = <&sysc R8A774C0_PD_ALWAYS_ON>;
resets = <&cpg 311>;
diff --git a/arch/arm64/boot/dts/renesas/r8a774e1.dtsi b/arch/arm64/boot/dts/renesas/r8a774e1.dtsi
index 379a130..e6d8610 100644
--- a/arch/arm64/boot/dts/renesas/r8a774e1.dtsi
+++ b/arch/arm64/boot/dts/renesas/r8a774e1.dtsi
@@ -47,7 +47,7 @@ can_clk: can {
clock-frequency = <0>;
};
- cluster0_opp: opp_table0 {
+ cluster0_opp: opp-table-0 {
compatible = "operating-points-v2";
opp-shared;
@@ -69,7 +69,7 @@ opp-1500000000 {
};
};
- cluster1_opp: opp_table1 {
+ cluster1_opp: opp-table-1 {
compatible = "operating-points-v2";
opp-shared;
@@ -2362,7 +2362,8 @@ sdhi0: mmc@ee100000 {
"renesas,rcar-gen3-sdhi";
reg = <0 0xee100000 0 0x2000>;
interrupts = <GIC_SPI 165 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&cpg CPG_MOD 314>;
+ clocks = <&cpg CPG_MOD 314>, <&cpg CPG_CORE R8A774E1_CLK_SD0H>;
+ clock-names = "core", "clkh";
max-frequency = <200000000>;
power-domains = <&sysc R8A774E1_PD_ALWAYS_ON>;
resets = <&cpg 314>;
@@ -2375,7 +2376,8 @@ sdhi1: mmc@ee120000 {
"renesas,rcar-gen3-sdhi";
reg = <0 0xee120000 0 0x2000>;
interrupts = <GIC_SPI 166 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&cpg CPG_MOD 313>;
+ clocks = <&cpg CPG_MOD 313>, <&cpg CPG_CORE R8A774E1_CLK_SD1H>;
+ clock-names = "core", "clkh";
max-frequency = <200000000>;
power-domains = <&sysc R8A774E1_PD_ALWAYS_ON>;
resets = <&cpg 313>;
@@ -2388,7 +2390,8 @@ sdhi2: mmc@ee140000 {
"renesas,rcar-gen3-sdhi";
reg = <0 0xee140000 0 0x2000>;
interrupts = <GIC_SPI 167 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&cpg CPG_MOD 312>;
+ clocks = <&cpg CPG_MOD 312>, <&cpg CPG_CORE R8A774E1_CLK_SD2H>;
+ clock-names = "core", "clkh";
max-frequency = <200000000>;
power-domains = <&sysc R8A774E1_PD_ALWAYS_ON>;
resets = <&cpg 312>;
@@ -2401,7 +2404,8 @@ sdhi3: mmc@ee160000 {
"renesas,rcar-gen3-sdhi";
reg = <0 0xee160000 0 0x2000>;
interrupts = <GIC_SPI 168 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&cpg CPG_MOD 311>;
+ clocks = <&cpg CPG_MOD 311>, <&cpg CPG_CORE R8A774E1_CLK_SD3H>;
+ clock-names = "core", "clkh";
max-frequency = <200000000>;
power-domains = <&sysc R8A774E1_PD_ALWAYS_ON>;
resets = <&cpg 311>;
@@ -2904,7 +2908,7 @@ prr: chipid@fff00044 {
};
thermal-zones {
- sensor_thermal1: sensor-thermal1 {
+ sensor1_thermal: sensor1-thermal {
polling-delay-passive = <250>;
polling-delay = <1000>;
thermal-sensors = <&tsc 0>;
@@ -2919,7 +2923,7 @@ sensor1_crit: sensor1-crit {
};
};
- sensor_thermal2: sensor-thermal2 {
+ sensor2_thermal: sensor2-thermal {
polling-delay-passive = <250>;
polling-delay = <1000>;
thermal-sensors = <&tsc 1>;
@@ -2934,7 +2938,7 @@ sensor2_crit: sensor2-crit {
};
};
- sensor_thermal3: sensor-thermal3 {
+ sensor3_thermal: sensor3-thermal {
polling-delay-passive = <250>;
polling-delay = <1000>;
thermal-sensors = <&tsc 2>;
diff --git a/arch/arm64/boot/dts/renesas/r8a77951.dtsi b/arch/arm64/boot/dts/renesas/r8a77951.dtsi
index 1768a3e..9265a57 100644
--- a/arch/arm64/boot/dts/renesas/r8a77951.dtsi
+++ b/arch/arm64/boot/dts/renesas/r8a77951.dtsi
@@ -62,7 +62,7 @@ can_clk: can {
clock-frequency = <0>;
};
- cluster0_opp: opp_table0 {
+ cluster0_opp: opp-table-0 {
compatible = "operating-points-v2";
opp-shared;
@@ -96,7 +96,7 @@ opp-1700000000 {
};
};
- cluster1_opp: opp_table1 {
+ cluster1_opp: opp-table-1 {
compatible = "operating-points-v2";
opp-shared;
@@ -2668,7 +2668,8 @@ sdhi0: mmc@ee100000 {
"renesas,rcar-gen3-sdhi";
reg = <0 0xee100000 0 0x2000>;
interrupts = <GIC_SPI 165 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&cpg CPG_MOD 314>;
+ clocks = <&cpg CPG_MOD 314>, <&cpg CPG_CORE R8A7795_CLK_SD0H>;
+ clock-names = "core", "clkh";
max-frequency = <200000000>;
power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
resets = <&cpg 314>;
@@ -2681,7 +2682,8 @@ sdhi1: mmc@ee120000 {
"renesas,rcar-gen3-sdhi";
reg = <0 0xee120000 0 0x2000>;
interrupts = <GIC_SPI 166 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&cpg CPG_MOD 313>;
+ clocks = <&cpg CPG_MOD 313>, <&cpg CPG_CORE R8A7795_CLK_SD1H>;
+ clock-names = "core", "clkh";
max-frequency = <200000000>;
power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
resets = <&cpg 313>;
@@ -2694,7 +2696,8 @@ sdhi2: mmc@ee140000 {
"renesas,rcar-gen3-sdhi";
reg = <0 0xee140000 0 0x2000>;
interrupts = <GIC_SPI 167 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&cpg CPG_MOD 312>;
+ clocks = <&cpg CPG_MOD 312>, <&cpg CPG_CORE R8A7795_CLK_SD2H>;
+ clock-names = "core", "clkh";
max-frequency = <200000000>;
power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
resets = <&cpg 312>;
@@ -2707,7 +2710,8 @@ sdhi3: mmc@ee160000 {
"renesas,rcar-gen3-sdhi";
reg = <0 0xee160000 0 0x2000>;
interrupts = <GIC_SPI 168 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&cpg CPG_MOD 311>;
+ clocks = <&cpg CPG_MOD 311>, <&cpg CPG_CORE R8A7795_CLK_SD3H>;
+ clock-names = "core", "clkh";
max-frequency = <200000000>;
power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
resets = <&cpg 311>;
@@ -3375,7 +3379,7 @@ prr: chipid@fff00044 {
};
thermal-zones {
- sensor_thermal1: sensor-thermal1 {
+ sensor1_thermal: sensor1-thermal {
polling-delay-passive = <250>;
polling-delay = <1000>;
thermal-sensors = <&tsc 0>;
@@ -3390,7 +3394,7 @@ sensor1_crit: sensor1-crit {
};
};
- sensor_thermal2: sensor-thermal2 {
+ sensor2_thermal: sensor2-thermal {
polling-delay-passive = <250>;
polling-delay = <1000>;
thermal-sensors = <&tsc 1>;
@@ -3405,7 +3409,7 @@ sensor2_crit: sensor2-crit {
};
};
- sensor_thermal3: sensor-thermal3 {
+ sensor3_thermal: sensor3-thermal {
polling-delay-passive = <250>;
polling-delay = <1000>;
thermal-sensors = <&tsc 2>;
diff --git a/arch/arm64/boot/dts/renesas/r8a77960.dtsi b/arch/arm64/boot/dts/renesas/r8a77960.dtsi
index 2bd8169..26f7103 100644
--- a/arch/arm64/boot/dts/renesas/r8a77960.dtsi
+++ b/arch/arm64/boot/dts/renesas/r8a77960.dtsi
@@ -57,7 +57,7 @@ can_clk: can {
clock-frequency = <0>;
};
- cluster0_opp: opp_table0 {
+ cluster0_opp: opp-table-0 {
compatible = "operating-points-v2";
opp-shared;
@@ -97,7 +97,7 @@ opp-1800000000 {
};
};
- cluster1_opp: opp_table1 {
+ cluster1_opp: opp-table-1 {
compatible = "operating-points-v2";
opp-shared;
@@ -2468,7 +2468,8 @@ sdhi0: mmc@ee100000 {
"renesas,rcar-gen3-sdhi";
reg = <0 0xee100000 0 0x2000>;
interrupts = <GIC_SPI 165 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&cpg CPG_MOD 314>;
+ clocks = <&cpg CPG_MOD 314>, <&cpg CPG_CORE R8A7796_CLK_SD0H>;
+ clock-names = "core", "clkh";
max-frequency = <200000000>;
power-domains = <&sysc R8A7796_PD_ALWAYS_ON>;
resets = <&cpg 314>;
@@ -2481,7 +2482,8 @@ sdhi1: mmc@ee120000 {
"renesas,rcar-gen3-sdhi";
reg = <0 0xee120000 0 0x2000>;
interrupts = <GIC_SPI 166 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&cpg CPG_MOD 313>;
+ clocks = <&cpg CPG_MOD 313>, <&cpg CPG_CORE R8A7796_CLK_SD1H>;
+ clock-names = "core", "clkh";
max-frequency = <200000000>;
power-domains = <&sysc R8A7796_PD_ALWAYS_ON>;
resets = <&cpg 313>;
@@ -2494,7 +2496,8 @@ sdhi2: mmc@ee140000 {
"renesas,rcar-gen3-sdhi";
reg = <0 0xee140000 0 0x2000>;
interrupts = <GIC_SPI 167 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&cpg CPG_MOD 312>;
+ clocks = <&cpg CPG_MOD 312>, <&cpg CPG_CORE R8A7796_CLK_SD2H>;
+ clock-names = "core", "clkh";
max-frequency = <200000000>;
power-domains = <&sysc R8A7796_PD_ALWAYS_ON>;
resets = <&cpg 312>;
@@ -2507,7 +2510,8 @@ sdhi3: mmc@ee160000 {
"renesas,rcar-gen3-sdhi";
reg = <0 0xee160000 0 0x2000>;
interrupts = <GIC_SPI 168 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&cpg CPG_MOD 311>;
+ clocks = <&cpg CPG_MOD 311>, <&cpg CPG_CORE R8A7796_CLK_SD3H>;
+ clock-names = "core", "clkh";
max-frequency = <200000000>;
power-domains = <&sysc R8A7796_PD_ALWAYS_ON>;
resets = <&cpg 311>;
@@ -2972,7 +2976,7 @@ prr: chipid@fff00044 {
};
thermal-zones {
- sensor_thermal1: sensor-thermal1 {
+ sensor1_thermal: sensor1-thermal {
polling-delay-passive = <250>;
polling-delay = <1000>;
thermal-sensors = <&tsc 0>;
@@ -2987,7 +2991,7 @@ sensor1_crit: sensor1-crit {
};
};
- sensor_thermal2: sensor-thermal2 {
+ sensor2_thermal: sensor2-thermal {
polling-delay-passive = <250>;
polling-delay = <1000>;
thermal-sensors = <&tsc 1>;
@@ -3002,7 +3006,7 @@ sensor2_crit: sensor2-crit {
};
};
- sensor_thermal3: sensor-thermal3 {
+ sensor3_thermal: sensor3-thermal {
polling-delay-passive = <250>;
polling-delay = <1000>;
thermal-sensors = <&tsc 2>;
diff --git a/arch/arm64/boot/dts/renesas/r8a77961.dtsi b/arch/arm64/boot/dts/renesas/r8a77961.dtsi
index 86d59e7..ac9b587 100644
--- a/arch/arm64/boot/dts/renesas/r8a77961.dtsi
+++ b/arch/arm64/boot/dts/renesas/r8a77961.dtsi
@@ -46,7 +46,7 @@ can_clk: can {
clock-frequency = <0>;
};
- cluster0_opp: opp_table0 {
+ cluster0_opp: opp-table-0 {
compatible = "operating-points-v2";
opp-shared;
@@ -86,7 +86,7 @@ opp-1800000000 {
};
};
- cluster1_opp: opp_table1 {
+ cluster1_opp: opp-table-1 {
compatible = "operating-points-v2";
opp-shared;
@@ -2312,7 +2312,8 @@ sdhi0: mmc@ee100000 {
"renesas,rcar-gen3-sdhi";
reg = <0 0xee100000 0 0x2000>;
interrupts = <GIC_SPI 165 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&cpg CPG_MOD 314>;
+ clocks = <&cpg CPG_MOD 314>, <&cpg CPG_CORE R8A77961_CLK_SD0H>;
+ clock-names = "core", "clkh";
max-frequency = <200000000>;
power-domains = <&sysc R8A77961_PD_ALWAYS_ON>;
resets = <&cpg 314>;
@@ -2325,7 +2326,8 @@ sdhi1: mmc@ee120000 {
"renesas,rcar-gen3-sdhi";
reg = <0 0xee120000 0 0x2000>;
interrupts = <GIC_SPI 166 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&cpg CPG_MOD 313>;
+ clocks = <&cpg CPG_MOD 313>, <&cpg CPG_CORE R8A77961_CLK_SD1H>;
+ clock-names = "core", "clkh";
max-frequency = <200000000>;
power-domains = <&sysc R8A77961_PD_ALWAYS_ON>;
resets = <&cpg 313>;
@@ -2338,7 +2340,8 @@ sdhi2: mmc@ee140000 {
"renesas,rcar-gen3-sdhi";
reg = <0 0xee140000 0 0x2000>;
interrupts = <GIC_SPI 167 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&cpg CPG_MOD 312>;
+ clocks = <&cpg CPG_MOD 312>, <&cpg CPG_CORE R8A77961_CLK_SD2H>;
+ clock-names = "core", "clkh";
max-frequency = <200000000>;
power-domains = <&sysc R8A77961_PD_ALWAYS_ON>;
resets = <&cpg 312>;
@@ -2351,7 +2354,8 @@ sdhi3: mmc@ee160000 {
"renesas,rcar-gen3-sdhi";
reg = <0 0xee160000 0 0x2000>;
interrupts = <GIC_SPI 168 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&cpg CPG_MOD 311>;
+ clocks = <&cpg CPG_MOD 311>, <&cpg CPG_CORE R8A77961_CLK_SD3H>;
+ clock-names = "core", "clkh";
max-frequency = <200000000>;
power-domains = <&sysc R8A77961_PD_ALWAYS_ON>;
resets = <&cpg 311>;
@@ -2730,7 +2734,7 @@ prr: chipid@fff00044 {
};
thermal-zones {
- sensor_thermal1: sensor-thermal1 {
+ sensor1_thermal: sensor1-thermal {
polling-delay-passive = <250>;
polling-delay = <1000>;
thermal-sensors = <&tsc 0>;
@@ -2745,7 +2749,7 @@ sensor1_crit: sensor1-crit {
};
};
- sensor_thermal2: sensor-thermal2 {
+ sensor2_thermal: sensor2-thermal {
polling-delay-passive = <250>;
polling-delay = <1000>;
thermal-sensors = <&tsc 1>;
@@ -2760,7 +2764,7 @@ sensor2_crit: sensor2-crit {
};
};
- sensor_thermal3: sensor-thermal3 {
+ sensor3_thermal: sensor3-thermal {
polling-delay-passive = <250>;
polling-delay = <1000>;
thermal-sensors = <&tsc 2>;
diff --git a/arch/arm64/boot/dts/renesas/r8a77965.dtsi b/arch/arm64/boot/dts/renesas/r8a77965.dtsi
index 08df756..f898aad 100644
--- a/arch/arm64/boot/dts/renesas/r8a77965.dtsi
+++ b/arch/arm64/boot/dts/renesas/r8a77965.dtsi
@@ -62,7 +62,7 @@ can_clk: can {
clock-frequency = <0>;
};
- cluster0_opp: opp_table0 {
+ cluster0_opp: opp-table-0 {
compatible = "operating-points-v2";
opp-shared;
@@ -2315,7 +2315,8 @@ sdhi0: mmc@ee100000 {
"renesas,rcar-gen3-sdhi";
reg = <0 0xee100000 0 0x2000>;
interrupts = <GIC_SPI 165 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&cpg CPG_MOD 314>;
+ clocks = <&cpg CPG_MOD 314>, <&cpg CPG_CORE R8A77965_CLK_SD0H>;
+ clock-names = "core", "clkh";
max-frequency = <200000000>;
power-domains = <&sysc R8A77965_PD_ALWAYS_ON>;
resets = <&cpg 314>;
@@ -2328,7 +2329,8 @@ sdhi1: mmc@ee120000 {
"renesas,rcar-gen3-sdhi";
reg = <0 0xee120000 0 0x2000>;
interrupts = <GIC_SPI 166 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&cpg CPG_MOD 313>;
+ clocks = <&cpg CPG_MOD 313>, <&cpg CPG_CORE R8A77965_CLK_SD1H>;
+ clock-names = "core", "clkh";
max-frequency = <200000000>;
power-domains = <&sysc R8A77965_PD_ALWAYS_ON>;
resets = <&cpg 313>;
@@ -2341,7 +2343,8 @@ sdhi2: mmc@ee140000 {
"renesas,rcar-gen3-sdhi";
reg = <0 0xee140000 0 0x2000>;
interrupts = <GIC_SPI 167 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&cpg CPG_MOD 312>;
+ clocks = <&cpg CPG_MOD 312>, <&cpg CPG_CORE R8A77965_CLK_SD2H>;
+ clock-names = "core", "clkh";
max-frequency = <200000000>;
power-domains = <&sysc R8A77965_PD_ALWAYS_ON>;
resets = <&cpg 312>;
@@ -2354,7 +2357,8 @@ sdhi3: mmc@ee160000 {
"renesas,rcar-gen3-sdhi";
reg = <0 0xee160000 0 0x2000>;
interrupts = <GIC_SPI 168 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&cpg CPG_MOD 311>;
+ clocks = <&cpg CPG_MOD 311>, <&cpg CPG_CORE R8A77965_CLK_SD3H>;
+ clock-names = "core", "clkh";
max-frequency = <200000000>;
power-domains = <&sysc R8A77965_PD_ALWAYS_ON>;
resets = <&cpg 311>;
@@ -2784,7 +2788,7 @@ prr: chipid@fff00044 {
};
thermal-zones {
- sensor_thermal1: sensor-thermal1 {
+ sensor1_thermal: sensor1-thermal {
polling-delay-passive = <250>;
polling-delay = <1000>;
thermal-sensors = <&tsc 0>;
@@ -2799,7 +2803,7 @@ sensor1_crit: sensor1-crit {
};
};
- sensor_thermal2: sensor-thermal2 {
+ sensor2_thermal: sensor2-thermal {
polling-delay-passive = <250>;
polling-delay = <1000>;
thermal-sensors = <&tsc 1>;
@@ -2814,7 +2818,7 @@ sensor2_crit: sensor2-crit {
};
};
- sensor_thermal3: sensor-thermal3 {
+ sensor3_thermal: sensor3-thermal {
polling-delay-passive = <250>;
polling-delay = <1000>;
thermal-sensors = <&tsc 2>;
diff --git a/arch/arm64/boot/dts/renesas/r8a77980.dtsi b/arch/arm64/boot/dts/renesas/r8a77980.dtsi
index 6347d15..347c068 100644
--- a/arch/arm64/boot/dts/renesas/r8a77980.dtsi
+++ b/arch/arm64/boot/dts/renesas/r8a77980.dtsi
@@ -1339,7 +1339,8 @@ mmc0: mmc@ee140000 {
"renesas,rcar-gen3-sdhi";
reg = <0 0xee140000 0 0x2000>;
interrupts = <GIC_SPI 165 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&cpg CPG_MOD 314>;
+ clocks = <&cpg CPG_MOD 314>, <&cpg CPG_CORE R8A77980_CLK_SD0H>;
+ clock-names = "core", "clkh";
power-domains = <&sysc R8A77980_PD_ALWAYS_ON>;
resets = <&cpg 314>;
max-frequency = <200000000>;
@@ -1580,7 +1581,7 @@ prr: chipid@fff00044 {
};
thermal-zones {
- thermal-sensor-1 {
+ sensor1_thermal: sensor1-thermal {
polling-delay-passive = <250>;
polling-delay = <1000>;
thermal-sensors = <&tsc 0>;
@@ -1599,7 +1600,7 @@ sensor1-critical {
};
};
- thermal-sensor-2 {
+ sensor2_thermal: sensor2-thermal {
polling-delay-passive = <250>;
polling-delay = <1000>;
thermal-sensors = <&tsc 1>;
diff --git a/arch/arm64/boot/dts/renesas/r8a77990.dtsi b/arch/arm64/boot/dts/renesas/r8a77990.dtsi
index 0ea300a..14caedd 100644
--- a/arch/arm64/boot/dts/renesas/r8a77990.dtsi
+++ b/arch/arm64/boot/dts/renesas/r8a77990.dtsi
@@ -55,7 +55,7 @@ can_clk: can {
clock-frequency = <0>;
};
- cluster1_opp: opp_table10 {
+ cluster1_opp: opp-table-1 {
compatible = "operating-points-v2";
opp-shared;
opp-800000000 {
@@ -1788,7 +1788,8 @@ sdhi0: mmc@ee100000 {
"renesas,rcar-gen3-sdhi";
reg = <0 0xee100000 0 0x2000>;
interrupts = <GIC_SPI 165 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&cpg CPG_MOD 314>;
+ clocks = <&cpg CPG_MOD 314>, <&cpg CPG_CORE R8A77990_CLK_SD0H>;
+ clock-names = "core", "clkh";
max-frequency = <200000000>;
power-domains = <&sysc R8A77990_PD_ALWAYS_ON>;
resets = <&cpg 314>;
@@ -1801,7 +1802,8 @@ sdhi1: mmc@ee120000 {
"renesas,rcar-gen3-sdhi";
reg = <0 0xee120000 0 0x2000>;
interrupts = <GIC_SPI 166 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&cpg CPG_MOD 313>;
+ clocks = <&cpg CPG_MOD 313>, <&cpg CPG_CORE R8A77990_CLK_SD1H>;
+ clock-names = "core", "clkh";
max-frequency = <200000000>;
power-domains = <&sysc R8A77990_PD_ALWAYS_ON>;
resets = <&cpg 313>;
@@ -1814,7 +1816,8 @@ sdhi3: mmc@ee160000 {
"renesas,rcar-gen3-sdhi";
reg = <0 0xee160000 0 0x2000>;
interrupts = <GIC_SPI 168 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&cpg CPG_MOD 311>;
+ clocks = <&cpg CPG_MOD 311>, <&cpg CPG_CORE R8A77990_CLK_SD3H>;
+ clock-names = "core", "clkh";
max-frequency = <200000000>;
power-domains = <&sysc R8A77990_PD_ALWAYS_ON>;
resets = <&cpg 311>;
diff --git a/arch/arm64/boot/dts/renesas/r8a77995.dtsi b/arch/arm64/boot/dts/renesas/r8a77995.dtsi
index 16ad5fc..f29f398 100644
--- a/arch/arm64/boot/dts/renesas/r8a77995.dtsi
+++ b/arch/arm64/boot/dts/renesas/r8a77995.dtsi
@@ -1216,7 +1216,8 @@ sdhi2: mmc@ee140000 {
"renesas,rcar-gen3-sdhi";
reg = <0 0xee140000 0 0x2000>;
interrupts = <GIC_SPI 167 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&cpg CPG_MOD 312>;
+ clocks = <&cpg CPG_MOD 312>, <&cpg CPG_CORE R8A77995_CLK_SD0H>;
+ clock-names = "core", "clkh";
max-frequency = <200000000>;
power-domains = <&sysc R8A77995_PD_ALWAYS_ON>;
resets = <&cpg 312>;
diff --git a/arch/arm64/boot/dts/renesas/r8a779a0-falcon-cpu.dtsi b/arch/arm64/boot/dts/renesas/r8a779a0-falcon-cpu.dtsi
index cd2f0d6..6af3f4f 100644
--- a/arch/arm64/boot/dts/renesas/r8a779a0-falcon-cpu.dtsi
+++ b/arch/arm64/boot/dts/renesas/r8a779a0-falcon-cpu.dtsi
@@ -98,6 +98,27 @@ memory@700000000 {
reg = <0x7 0x00000000 0x0 0x80000000>;
};
+ mini-dp-con {
+ compatible = "dp-connector";
+ label = "CN5";
+ type = "mini";
+
+ port {
+ mini_dp_con_in: endpoint {
+ remote-endpoint = <&sn65dsi86_out>;
+ };
+ };
+ };
+
+ reg_1p2v: regulator-1p2v {
+ compatible = "regulator-fixed";
+ regulator-name = "fixed-1.2V";
+ regulator-min-microvolt = <1200000>;
+ regulator-max-microvolt = <1200000>;
+ regulator-boot-on;
+ regulator-always-on;
+ };
+
reg_1p8v: regulator-1p8v {
compatible = "regulator-fixed";
regulator-name = "fixed-1.8V";
@@ -115,6 +136,29 @@ reg_3p3v: regulator-3p3v {
regulator-boot-on;
regulator-always-on;
};
+
+ sn65dsi86_refclk: clk-x6 {
+ compatible = "fixed-clock";
+ #clock-cells = <0>;
+ clock-frequency = <38400000>;
+ };
+};
+
+&dsi0 {
+ status = "okay";
+
+ ports {
+ port@1 {
+ dsi0_out: endpoint {
+ remote-endpoint = <&sn65dsi86_in>;
+ data-lanes = <1 2 3 4>;
+ };
+ };
+ };
+};
+
+&du {
+ status = "okay";
};
&extal_clk {
@@ -146,6 +190,41 @@ &i2c1 {
status = "okay";
clock-frequency = <400000>;
+
+ bridge@2c {
+ compatible = "ti,sn65dsi86";
+ reg = <0x2c>;
+
+ clocks = <&sn65dsi86_refclk>;
+ clock-names = "refclk";
+
+ interrupt-parent = <&gpio1>;
+ interrupts = <24 IRQ_TYPE_LEVEL_HIGH>;
+
+ vccio-supply = <®_1p8v>;
+ vpll-supply = <®_1p8v>;
+ vcca-supply = <®_1p2v>;
+ vcc-supply = <®_1p2v>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ sn65dsi86_in: endpoint {
+ remote-endpoint = <&dsi0_out>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+ sn65dsi86_out: endpoint {
+ remote-endpoint = <&mini_dp_con_in>;
+ };
+ };
+ };
+ };
};
&i2c6 {
diff --git a/arch/arm64/boot/dts/renesas/r8a779a0.dtsi b/arch/arm64/boot/dts/renesas/r8a779a0.dtsi
index 43bf2cb..1e7ed12 100644
--- a/arch/arm64/boot/dts/renesas/r8a779a0.dtsi
+++ b/arch/arm64/boot/dts/renesas/r8a779a0.dtsi
@@ -87,7 +87,7 @@ rwdt: watchdog@e6020000 {
status = "disabled";
};
- pfc: pin-controller@e6050000 {
+ pfc: pinctrl@e6050000 {
compatible = "renesas,pfc-r8a779a0";
reg = <0 0xe6050000 0 0x16c>, <0 0xe6050800 0 0x16c>,
<0 0xe6058000 0 0x16c>, <0 0xe6058800 0 0x16c>,
@@ -1993,7 +1993,8 @@ mmc0: mmc@ee140000 {
"renesas,rcar-gen3-sdhi";
reg = <0 0xee140000 0 0x2000>;
interrupts = <GIC_SPI 236 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&cpg CPG_MOD 706>;
+ clocks = <&cpg CPG_MOD 706>, <&cpg CPG_CORE R8A779A0_CLK_SD0H>;
+ clock-names = "core", "clkh";
power-domains = <&sysc R8A779A0_PD_ALWAYS_ON>;
resets = <&cpg 706>;
max-frequency = <200000000>;
@@ -2268,6 +2269,40 @@ csi43isp3: endpoint {
};
};
+ du: display@feb00000 {
+ compatible = "renesas,du-r8a779a0";
+ reg = <0 0xfeb00000 0 0x40000>;
+ interrupts = <GIC_SPI 143 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 144 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&cpg CPG_MOD 411>;
+ clock-names = "du.0";
+ power-domains = <&sysc R8A779A0_PD_ALWAYS_ON>;
+ resets = <&cpg 411>;
+ reset-names = "du.0";
+ renesas,vsps = <&vspd0 0>, <&vspd1 0>;
+
+ status = "disabled";
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ du_out_dsi0: endpoint {
+ remote-endpoint = <&dsi0_in>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+ du_out_dsi1: endpoint {
+ remote-endpoint = <&dsi1_in>;
+ };
+ };
+ };
+ };
+
isp0: isp@fed00000 {
compatible = "renesas,r8a779a0-isp";
reg = <0 0xfed00000 0 0x10000>;
@@ -2600,6 +2635,62 @@ isp3vin31: endpoint {
};
};
+ dsi0: dsi-encoder@fed80000 {
+ compatible = "renesas,r8a779a0-dsi-csi2-tx";
+ reg = <0 0xfed80000 0 0x10000>;
+ power-domains = <&sysc R8A779A0_PD_ALWAYS_ON>;
+ clocks = <&cpg CPG_MOD 415>,
+ <&cpg CPG_CORE R8A779A0_CLK_DSI>,
+ <&cpg CPG_CORE R8A779A0_CLK_CL16MCK>;
+ clock-names = "fck", "dsi", "pll";
+ resets = <&cpg 415>;
+ status = "disabled";
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ dsi0_in: endpoint {
+ remote-endpoint = <&du_out_dsi0>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+ };
+ };
+ };
+
+ dsi1: dsi-encoder@fed90000 {
+ compatible = "renesas,r8a779a0-dsi-csi2-tx";
+ reg = <0 0xfed90000 0 0x10000>;
+ power-domains = <&sysc R8A779A0_PD_ALWAYS_ON>;
+ clocks = <&cpg CPG_MOD 416>,
+ <&cpg CPG_CORE R8A779A0_CLK_DSI>,
+ <&cpg CPG_CORE R8A779A0_CLK_CL16MCK>;
+ clock-names = "fck", "dsi", "pll";
+ resets = <&cpg 416>;
+ status = "disabled";
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ dsi1_in: endpoint {
+ remote-endpoint = <&du_out_dsi1>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+ };
+ };
+ };
+
prr: chipid@fff00044 {
compatible = "renesas,prr";
reg = <0 0xfff00044 0 4>;
@@ -2607,7 +2698,7 @@ prr: chipid@fff00044 {
};
thermal-zones {
- sensor_thermal1: sensor-thermal1 {
+ sensor1_thermal: sensor1-thermal {
polling-delay-passive = <250>;
polling-delay = <1000>;
thermal-sensors = <&tsc 0>;
@@ -2621,7 +2712,7 @@ sensor1_crit: sensor1-crit {
};
};
- sensor_thermal2: sensor-thermal2 {
+ sensor2_thermal: sensor2-thermal {
polling-delay-passive = <250>;
polling-delay = <1000>;
thermal-sensors = <&tsc 1>;
@@ -2635,7 +2726,7 @@ sensor2_crit: sensor2-crit {
};
};
- sensor_thermal3: sensor-thermal3 {
+ sensor3_thermal: sensor3-thermal {
polling-delay-passive = <250>;
polling-delay = <1000>;
thermal-sensors = <&tsc 2>;
@@ -2649,7 +2740,7 @@ sensor3_crit: sensor3-crit {
};
};
- sensor_thermal4: sensor-thermal4 {
+ sensor4_thermal: sensor4-thermal {
polling-delay-passive = <250>;
polling-delay = <1000>;
thermal-sensors = <&tsc 3>;
@@ -2663,7 +2754,7 @@ sensor4_crit: sensor4-crit {
};
};
- sensor_thermal5: sensor-thermal5 {
+ sensor5_thermal: sensor5-thermal {
polling-delay-passive = <250>;
polling-delay = <1000>;
thermal-sensors = <&tsc 4>;
diff --git a/arch/arm64/boot/dts/renesas/r8a779f0-spider-cpu.dtsi b/arch/arm64/boot/dts/renesas/r8a779f0-spider-cpu.dtsi
new file mode 100644
index 0000000..1565865
--- /dev/null
+++ b/arch/arm64/boot/dts/renesas/r8a779f0-spider-cpu.dtsi
@@ -0,0 +1,36 @@
+// SPDX-License-Identifier: (GPL-2.0 or MIT)
+/*
+ * Device Tree Source for the Spider CPU board
+ *
+ * Copyright (C) 2021 Renesas Electronics Corp.
+ */
+
+#include "r8a779f0.dtsi"
+
+/ {
+ model = "Renesas Spider CPU board";
+ compatible = "renesas,spider-cpu", "renesas,r8a779f0";
+
+ memory@48000000 {
+ device_type = "memory";
+ /* first 128MB is reserved for secure area. */
+ reg = <0x0 0x48000000 0x0 0x78000000>;
+ };
+
+ memory@480000000 {
+ device_type = "memory";
+ reg = <0x4 0x80000000 0x0 0x80000000>;
+ };
+};
+
+&extal_clk {
+ clock-frequency = <20000000>;
+};
+
+&extalr_clk {
+ clock-frequency = <32768>;
+};
+
+&scif3 {
+ status = "okay";
+};
diff --git a/arch/arm64/boot/dts/renesas/r8a779f0-spider.dts b/arch/arm64/boot/dts/renesas/r8a779f0-spider.dts
new file mode 100644
index 0000000..f286254
--- /dev/null
+++ b/arch/arm64/boot/dts/renesas/r8a779f0-spider.dts
@@ -0,0 +1,22 @@
+// SPDX-License-Identifier: (GPL-2.0 or MIT)
+/*
+ * Device Tree Source for the Spider CPU and BreakOut boards
+ *
+ * Copyright (C) 2021 Renesas Electronics Corp.
+ */
+
+/dts-v1/;
+#include "r8a779f0-spider-cpu.dtsi"
+
+/ {
+ model = "Renesas Spider CPU and Breakout boards based on r8a779f0";
+ compatible = "renesas,spider-breakout", "renesas,spider-cpu", "renesas,r8a779f0";
+
+ aliases {
+ serial0 = &scif3;
+ };
+
+ chosen {
+ stdout-path = "serial0:115200n8";
+ };
+};
diff --git a/arch/arm64/boot/dts/renesas/r8a779f0.dtsi b/arch/arm64/boot/dts/renesas/r8a779f0.dtsi
new file mode 100644
index 0000000..eda5977
--- /dev/null
+++ b/arch/arm64/boot/dts/renesas/r8a779f0.dtsi
@@ -0,0 +1,121 @@
+// SPDX-License-Identifier: (GPL-2.0 or MIT)
+/*
+ * Device Tree Source for the R-Car S4-8 (R8A779F0) SoC
+ *
+ * Copyright (C) 2021 Renesas Electronics Corp.
+ */
+
+#include <dt-bindings/clock/r8a779f0-cpg-mssr.h>
+#include <dt-bindings/interrupt-controller/arm-gic.h>
+#include <dt-bindings/power/r8a779f0-sysc.h>
+
+/ {
+ compatible = "renesas,r8a779f0";
+ #address-cells = <2>;
+ #size-cells = <2>;
+
+ cpus {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ a55_0: cpu@0 {
+ compatible = "arm,cortex-a55";
+ reg = <0>;
+ device_type = "cpu";
+ power-domains = <&sysc R8A779F0_PD_A1E0D0C0>;
+ };
+ };
+
+ extal_clk: extal {
+ compatible = "fixed-clock";
+ #clock-cells = <0>;
+ /* This value must be overridden by the board */
+ clock-frequency = <0>;
+ };
+
+ extalr_clk: extalr {
+ compatible = "fixed-clock";
+ #clock-cells = <0>;
+ /* This value must be overridden by the board */
+ clock-frequency = <0>;
+ };
+
+ pmu_a55 {
+ compatible = "arm,cortex-a55-pmu";
+ interrupts-extended = <&gic GIC_PPI 7 IRQ_TYPE_LEVEL_LOW>;
+ };
+
+ /* External SCIF clock - to be overridden by boards that provide it */
+ scif_clk: scif {
+ compatible = "fixed-clock";
+ #clock-cells = <0>;
+ clock-frequency = <0>;
+ };
+
+ soc: soc {
+ compatible = "simple-bus";
+ interrupt-parent = <&gic>;
+ #address-cells = <2>;
+ #size-cells = <2>;
+ ranges;
+
+ cpg: clock-controller@e6150000 {
+ compatible = "renesas,r8a779f0-cpg-mssr";
+ reg = <0 0xe6150000 0 0x4000>;
+ clocks = <&extal_clk>, <&extalr_clk>;
+ clock-names = "extal", "extalr";
+ #clock-cells = <2>;
+ #power-domain-cells = <0>;
+ #reset-cells = <1>;
+ };
+
+ rst: reset-controller@e6160000 {
+ compatible = "renesas,r8a779f0-rst";
+ reg = <0 0xe6160000 0 0x4000>;
+ };
+
+ sysc: system-controller@e6180000 {
+ compatible = "renesas,r8a779f0-sysc";
+ reg = <0 0xe6180000 0 0x4000>;
+ #power-domain-cells = <1>;
+ };
+
+ scif3: serial@e6c50000 {
+ compatible = "renesas,scif-r8a779f0",
+ "renesas,rcar-gen4-scif", "renesas,scif";
+ reg = <0 0xe6c50000 0 64>;
+ interrupts = <GIC_SPI 252 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&cpg CPG_MOD 704>,
+ <&cpg CPG_CORE R8A779F0_CLK_S0D3_PER>,
+ <&scif_clk>;
+ clock-names = "fck", "brg_int", "scif_clk";
+ power-domains = <&sysc R8A779F0_PD_ALWAYS_ON>;
+ resets = <&cpg 704>;
+ status = "disabled";
+ };
+
+ gic: interrupt-controller@f1000000 {
+ compatible = "arm,gic-v3";
+ #interrupt-cells = <3>;
+ #address-cells = <0>;
+ interrupt-controller;
+ reg = <0x0 0xf1000000 0 0x20000>,
+ <0x0 0xf1060000 0 0x110000>;
+ interrupts = <GIC_PPI 9
+ (GIC_CPU_MASK_SIMPLE(1) | IRQ_TYPE_LEVEL_HIGH)>;
+ };
+
+ prr: chipid@fff00044 {
+ compatible = "renesas,prr";
+ reg = <0 0xfff00044 0 4>;
+ };
+ };
+
+ timer {
+ compatible = "arm,armv8-timer";
+ interrupts-extended = <&gic GIC_PPI 13 (GIC_CPU_MASK_SIMPLE(1) | IRQ_TYPE_LEVEL_LOW)>,
+ <&gic GIC_PPI 14 (GIC_CPU_MASK_SIMPLE(1) | IRQ_TYPE_LEVEL_LOW)>,
+ <&gic GIC_PPI 11 (GIC_CPU_MASK_SIMPLE(1) | IRQ_TYPE_LEVEL_LOW)>,
+ <&gic GIC_PPI 10 (GIC_CPU_MASK_SIMPLE(1) | IRQ_TYPE_LEVEL_LOW)>;
+ };
+};
diff --git a/arch/arm64/boot/dts/renesas/r9a07g044.dtsi b/arch/arm64/boot/dts/renesas/r9a07g044.dtsi
index 485ef5f..19287cc 100644
--- a/arch/arm64/boot/dts/renesas/r9a07g044.dtsi
+++ b/arch/arm64/boot/dts/renesas/r9a07g044.dtsi
@@ -42,9 +42,31 @@ extal_clk: extal {
clock-frequency = <0>;
};
- psci {
- compatible = "arm,psci-1.0", "arm,psci-0.2";
- method = "smc";
+ cluster0_opp: opp-table-0 {
+ compatible = "operating-points-v2";
+ opp-shared;
+
+ opp-150000000 {
+ opp-hz = /bits/ 64 <150000000>;
+ opp-microvolt = <1100000>;
+ clock-latency-ns = <300000>;
+ };
+ opp-300000000 {
+ opp-hz = /bits/ 64 <300000000>;
+ opp-microvolt = <1100000>;
+ clock-latency-ns = <300000>;
+ };
+ opp-600000000 {
+ opp-hz = /bits/ 64 <600000000>;
+ opp-microvolt = <1100000>;
+ clock-latency-ns = <300000>;
+ };
+ opp-1200000000 {
+ opp-hz = /bits/ 64 <1200000000>;
+ opp-microvolt = <1100000>;
+ clock-latency-ns = <300000>;
+ opp-suspend;
+ };
};
cpus {
@@ -66,8 +88,11 @@ cpu0: cpu@0 {
compatible = "arm,cortex-a55";
reg = <0>;
device_type = "cpu";
+ #cooling-cells = <2>;
next-level-cache = <&L3_CA55>;
enable-method = "psci";
+ clocks = <&cpg CPG_CORE R9A07G044_CLK_I>;
+ operating-points-v2 = <&cluster0_opp>;
};
cpu1: cpu@100 {
@@ -76,6 +101,8 @@ cpu1: cpu@100 {
device_type = "cpu";
next-level-cache = <&L3_CA55>;
enable-method = "psci";
+ clocks = <&cpg CPG_CORE R9A07G044_CLK_I>;
+ operating-points-v2 = <&cluster0_opp>;
};
L3_CA55: cache-controller-0 {
@@ -85,6 +112,55 @@ L3_CA55: cache-controller-0 {
};
};
+ gpu_opp_table: opp-table-1 {
+ compatible = "operating-points-v2";
+
+ opp-500000000 {
+ opp-hz = /bits/ 64 <500000000>;
+ opp-microvolt = <1100000>;
+ };
+
+ opp-400000000 {
+ opp-hz = /bits/ 64 <400000000>;
+ opp-microvolt = <1100000>;
+ };
+
+ opp-250000000 {
+ opp-hz = /bits/ 64 <250000000>;
+ opp-microvolt = <1100000>;
+ };
+
+ opp-200000000 {
+ opp-hz = /bits/ 64 <200000000>;
+ opp-microvolt = <1100000>;
+ };
+
+ opp-125000000 {
+ opp-hz = /bits/ 64 <125000000>;
+ opp-microvolt = <1100000>;
+ };
+
+ opp-100000000 {
+ opp-hz = /bits/ 64 <100000000>;
+ opp-microvolt = <1100000>;
+ };
+
+ opp-62500000 {
+ opp-hz = /bits/ 64 <62500000>;
+ opp-microvolt = <1100000>;
+ };
+
+ opp-50000000 {
+ opp-hz = /bits/ 64 <50000000>;
+ opp-microvolt = <1100000>;
+ };
+ };
+
+ psci {
+ compatible = "arm,psci-1.0", "arm,psci-0.2";
+ method = "smc";
+ };
+
soc: soc {
compatible = "simple-bus";
interrupt-parent = <&gic>;
@@ -176,6 +252,54 @@ ssi3: ssi@1004a800 {
status = "disabled";
};
+ spi0: spi@1004ac00 {
+ compatible = "renesas,r9a07g044-rspi", "renesas,rspi-rz";
+ reg = <0 0x1004ac00 0 0x400>;
+ interrupts = <GIC_SPI 415 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 413 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 414 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "error", "rx", "tx";
+ clocks = <&cpg CPG_MOD R9A07G044_RSPI0_CLKB>;
+ resets = <&cpg R9A07G044_RSPI0_RST>;
+ power-domains = <&cpg>;
+ num-cs = <1>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ spi1: spi@1004b000 {
+ compatible = "renesas,r9a07g044-rspi", "renesas,rspi-rz";
+ reg = <0 0x1004b000 0 0x400>;
+ interrupts = <GIC_SPI 418 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 416 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 417 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "error", "rx", "tx";
+ clocks = <&cpg CPG_MOD R9A07G044_RSPI1_CLKB>;
+ resets = <&cpg R9A07G044_RSPI1_RST>;
+ power-domains = <&cpg>;
+ num-cs = <1>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ spi2: spi@1004b400 {
+ compatible = "renesas,r9a07g044-rspi", "renesas,rspi-rz";
+ reg = <0 0x1004b400 0 0x400>;
+ interrupts = <GIC_SPI 421 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 419 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 420 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "error", "rx", "tx";
+ clocks = <&cpg CPG_MOD R9A07G044_RSPI2_CLKB>;
+ resets = <&cpg R9A07G044_RSPI2_RST>;
+ power-domains = <&cpg>;
+ num-cs = <1>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
scif0: serial@1004b800 {
compatible = "renesas,scif-r9a07g044";
reg = <0 0x1004b800 0 0x400>;
@@ -194,6 +318,108 @@ scif0: serial@1004b800 {
status = "disabled";
};
+ scif1: serial@1004bc00 {
+ compatible = "renesas,scif-r9a07g044";
+ reg = <0 0x1004bc00 0 0x400>;
+ interrupts = <GIC_SPI 385 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 387 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 388 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 386 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 389 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 389 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "eri", "rxi", "txi",
+ "bri", "dri", "tei";
+ clocks = <&cpg CPG_MOD R9A07G044_SCIF1_CLK_PCK>;
+ clock-names = "fck";
+ power-domains = <&cpg>;
+ resets = <&cpg R9A07G044_SCIF1_RST_SYSTEM_N>;
+ status = "disabled";
+ };
+
+ scif2: serial@1004c000 {
+ compatible = "renesas,scif-r9a07g044";
+ reg = <0 0x1004c000 0 0x400>;
+ interrupts = <GIC_SPI 390 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 392 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 393 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 391 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 394 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 394 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "eri", "rxi", "txi",
+ "bri", "dri", "tei";
+ clocks = <&cpg CPG_MOD R9A07G044_SCIF2_CLK_PCK>;
+ clock-names = "fck";
+ power-domains = <&cpg>;
+ resets = <&cpg R9A07G044_SCIF2_RST_SYSTEM_N>;
+ status = "disabled";
+ };
+
+ scif3: serial@1004c400 {
+ compatible = "renesas,scif-r9a07g044";
+ reg = <0 0x1004c400 0 0x400>;
+ interrupts = <GIC_SPI 395 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 397 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 398 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 396 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 399 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 399 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "eri", "rxi", "txi",
+ "bri", "dri", "tei";
+ clocks = <&cpg CPG_MOD R9A07G044_SCIF3_CLK_PCK>;
+ clock-names = "fck";
+ power-domains = <&cpg>;
+ resets = <&cpg R9A07G044_SCIF3_RST_SYSTEM_N>;
+ status = "disabled";
+ };
+
+ scif4: serial@1004c800 {
+ compatible = "renesas,scif-r9a07g044";
+ reg = <0 0x1004c800 0 0x400>;
+ interrupts = <GIC_SPI 400 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 402 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 403 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 401 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 404 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 404 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "eri", "rxi", "txi",
+ "bri", "dri", "tei";
+ clocks = <&cpg CPG_MOD R9A07G044_SCIF4_CLK_PCK>;
+ clock-names = "fck";
+ power-domains = <&cpg>;
+ resets = <&cpg R9A07G044_SCIF4_RST_SYSTEM_N>;
+ status = "disabled";
+ };
+
+ sci0: serial@1004d000 {
+ compatible = "renesas,r9a07g044-sci", "renesas,sci";
+ reg = <0 0x1004d000 0 0x400>;
+ interrupts = <GIC_SPI 405 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 406 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 407 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 408 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "eri", "rxi", "txi", "tei";
+ clocks = <&cpg CPG_MOD R9A07G044_SCI0_CLKP>;
+ clock-names = "fck";
+ power-domains = <&cpg>;
+ resets = <&cpg R9A07G044_SCI0_RST>;
+ status = "disabled";
+ };
+
+ sci1: serial@1004d400 {
+ compatible = "renesas,r9a07g044-sci", "renesas,sci";
+ reg = <0 0x1004d400 0 0x400>;
+ interrupts = <GIC_SPI 409 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 410 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 411 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 412 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "eri", "rxi", "txi", "tei";
+ clocks = <&cpg CPG_MOD R9A07G044_SCI1_CLKP>;
+ clock-names = "fck";
+ power-domains = <&cpg>;
+ resets = <&cpg R9A07G044_SCI1_RST>;
+ status = "disabled";
+ };
+
canfd: can@10050000 {
compatible = "renesas,r9a07g044-canfd", "renesas,rzg2l-canfd";
reg = <0 0x10050000 0 0x8000>;
@@ -358,6 +584,16 @@ channel@7 {
};
};
+ tsu: thermal@10059400 {
+ compatible = "renesas,r9a07g044-tsu",
+ "renesas,rzg2l-tsu";
+ reg = <0 0x10059400 0 0x400>;
+ clocks = <&cpg CPG_MOD R9A07G044_TSU_PCLK>;
+ resets = <&cpg R9A07G044_TSU_PRESETN>;
+ power-domains = <&cpg>;
+ #thermal-sensor-cells = <1>;
+ };
+
sbc: spi@10060000 {
compatible = "renesas,r9a07g044-rpc-if",
"renesas,rzg2l-rpc-if";
@@ -397,7 +633,7 @@ sysc: system-controller@11020000 {
status = "disabled";
};
- pinctrl: pin-controller@11030000 {
+ pinctrl: pinctrl@11030000 {
compatible = "renesas,r9a07g044-pinctrl";
reg = <0 0x11030000 0 0x10000>;
gpio-controller;
@@ -446,6 +682,27 @@ dmac: dma-controller@11820000 {
dma-channels = <16>;
};
+ gpu: gpu@11840000 {
+ compatible = "renesas,r9a07g044-mali",
+ "arm,mali-bifrost";
+ reg = <0x0 0x11840000 0x0 0x10000>;
+ interrupts = <GIC_SPI 154 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 155 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 153 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 156 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "job", "mmu", "gpu", "event";
+ clocks = <&cpg CPG_MOD R9A07G044_GPU_CLK>,
+ <&cpg CPG_MOD R9A07G044_GPU_AXI_CLK>,
+ <&cpg CPG_MOD R9A07G044_GPU_ACE_CLK>;
+ clock-names = "gpu", "bus", "bus_ace";
+ power-domains = <&cpg>;
+ resets = <&cpg R9A07G044_GPU_RESETN>,
+ <&cpg R9A07G044_GPU_AXI_RESETN>,
+ <&cpg R9A07G044_GPU_ACE_RESETN>;
+ reset-names = "rst", "axi_rst", "ace_rst";
+ operating-points-v2 = <&gpu_opp_table>;
+ };
+
gic: interrupt-controller@11900000 {
compatible = "arm,gic-v3";
#interrupt-cells = <3>;
@@ -463,10 +720,10 @@ sdhi0: mmc@11c00000 {
interrupts = <GIC_SPI 104 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 105 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&cpg CPG_MOD R9A07G044_SDHI0_IMCLK>,
- <&cpg CPG_MOD R9A07G044_SDHI0_IMCLK2>,
<&cpg CPG_MOD R9A07G044_SDHI0_CLK_HS>,
+ <&cpg CPG_MOD R9A07G044_SDHI0_IMCLK2>,
<&cpg CPG_MOD R9A07G044_SDHI0_ACLK>;
- clock-names = "imclk", "imclk2", "clk_hs", "aclk";
+ clock-names = "core", "clkh", "cd", "aclk";
resets = <&cpg R9A07G044_SDHI0_IXRST>;
power-domains = <&cpg>;
status = "disabled";
@@ -479,10 +736,10 @@ sdhi1: mmc@11c10000 {
interrupts = <GIC_SPI 106 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 107 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&cpg CPG_MOD R9A07G044_SDHI1_IMCLK>,
- <&cpg CPG_MOD R9A07G044_SDHI1_IMCLK2>,
<&cpg CPG_MOD R9A07G044_SDHI1_CLK_HS>,
+ <&cpg CPG_MOD R9A07G044_SDHI1_IMCLK2>,
<&cpg CPG_MOD R9A07G044_SDHI1_ACLK>;
- clock-names = "imclk", "imclk2", "clk_hs", "aclk";
+ clock-names = "core", "clkh", "cd", "aclk";
resets = <&cpg R9A07G044_SDHI1_IXRST>;
power-domains = <&cpg>;
status = "disabled";
@@ -641,6 +898,115 @@ hsusb: usb@11c60000 {
power-domains = <&cpg>;
status = "disabled";
};
+
+ wdt0: watchdog@12800800 {
+ compatible = "renesas,r9a07g044-wdt",
+ "renesas,rzg2l-wdt";
+ reg = <0 0x12800800 0 0x400>;
+ clocks = <&cpg CPG_MOD R9A07G044_WDT0_PCLK>,
+ <&cpg CPG_MOD R9A07G044_WDT0_CLK>;
+ clock-names = "pclk", "oscclk";
+ interrupts = <GIC_SPI 49 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 50 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "wdt", "perrout";
+ resets = <&cpg R9A07G044_WDT0_PRESETN>;
+ power-domains = <&cpg>;
+ status = "disabled";
+ };
+
+ wdt1: watchdog@12800c00 {
+ compatible = "renesas,r9a07g044-wdt",
+ "renesas,rzg2l-wdt";
+ reg = <0 0x12800C00 0 0x400>;
+ clocks = <&cpg CPG_MOD R9A07G044_WDT1_PCLK>,
+ <&cpg CPG_MOD R9A07G044_WDT1_CLK>;
+ clock-names = "pclk", "oscclk";
+ interrupts = <GIC_SPI 51 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 52 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "wdt", "perrout";
+ resets = <&cpg R9A07G044_WDT1_PRESETN>;
+ power-domains = <&cpg>;
+ status = "disabled";
+ };
+
+ wdt2: watchdog@12800400 {
+ compatible = "renesas,r9a07g044-wdt",
+ "renesas,rzg2l-wdt";
+ reg = <0 0x12800400 0 0x400>;
+ clocks = <&cpg CPG_MOD R9A07G044_WDT2_PCLK>,
+ <&cpg CPG_MOD R9A07G044_WDT2_CLK>;
+ clock-names = "pclk", "oscclk";
+ interrupts = <GIC_SPI 53 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 54 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "wdt", "perrout";
+ resets = <&cpg R9A07G044_WDT2_PRESETN>;
+ power-domains = <&cpg>;
+ status = "disabled";
+ };
+
+ ostm0: timer@12801000 {
+ compatible = "renesas,r9a07g044-ostm",
+ "renesas,ostm";
+ reg = <0x0 0x12801000 0x0 0x400>;
+ interrupts = <GIC_SPI 46 IRQ_TYPE_EDGE_RISING>;
+ clocks = <&cpg CPG_MOD R9A07G044_OSTM0_PCLK>;
+ resets = <&cpg R9A07G044_OSTM0_PRESETZ>;
+ power-domains = <&cpg>;
+ status = "disabled";
+ };
+
+ ostm1: timer@12801400 {
+ compatible = "renesas,r9a07g044-ostm",
+ "renesas,ostm";
+ reg = <0x0 0x12801400 0x0 0x400>;
+ interrupts = <GIC_SPI 47 IRQ_TYPE_EDGE_RISING>;
+ clocks = <&cpg CPG_MOD R9A07G044_OSTM1_PCLK>;
+ resets = <&cpg R9A07G044_OSTM1_PRESETZ>;
+ power-domains = <&cpg>;
+ status = "disabled";
+ };
+
+ ostm2: timer@12801800 {
+ compatible = "renesas,r9a07g044-ostm",
+ "renesas,ostm";
+ reg = <0x0 0x12801800 0x0 0x400>;
+ interrupts = <GIC_SPI 48 IRQ_TYPE_EDGE_RISING>;
+ clocks = <&cpg CPG_MOD R9A07G044_OSTM2_PCLK>;
+ resets = <&cpg R9A07G044_OSTM2_PRESETZ>;
+ power-domains = <&cpg>;
+ status = "disabled";
+ };
+ };
+
+ thermal-zones {
+ cpu-thermal {
+ polling-delay-passive = <250>;
+ polling-delay = <1000>;
+ thermal-sensors = <&tsu 0>;
+ sustainable-power = <717>;
+
+ cooling-maps {
+ map0 {
+ trip = <&target>;
+ cooling-device = <&cpu0 0 2>;
+ contribution = <1024>;
+ };
+ };
+
+ trips {
+ sensor_crit: sensor-crit {
+ temperature = <125000>;
+ hysteresis = <1000>;
+ type = "critical";
+ };
+
+ target: trip-point {
+ temperature = <100000>;
+ hysteresis = <1000>;
+ type = "passive";
+ };
+ };
+ };
};
timer {
diff --git a/arch/arm64/boot/dts/renesas/rzg2l-smarc-som.dtsi b/arch/arm64/boot/dts/renesas/rzg2l-smarc-som.dtsi
index 7e84a29..9112e79 100644
--- a/arch/arm64/boot/dts/renesas/rzg2l-smarc-som.dtsi
+++ b/arch/arm64/boot/dts/renesas/rzg2l-smarc-som.dtsi
@@ -52,6 +52,15 @@ reg_3p3v: regulator1 {
regulator-always-on;
};
+ reg_1p1v: regulator-vdd-core {
+ compatible = "regulator-fixed";
+ regulator-name = "fixed-1.1V";
+ regulator-min-microvolt = <1100000>;
+ regulator-max-microvolt = <1100000>;
+ regulator-boot-on;
+ regulator-always-on;
+ };
+
vccq_sdhi0: regulator-vccq-sdhi0 {
compatible = "regulator-gpio";
@@ -130,6 +139,18 @@ &extal_clk {
clock-frequency = <24000000>;
};
+&gpu {
+ mali-supply = <®_1p1v>;
+};
+
+&ostm1 {
+ status = "okay";
+};
+
+&ostm2 {
+ status = "okay";
+};
+
&pinctrl {
adc_pins: adc {
pinmux = <RZG2L_PORT_PINMUX(9, 0, 2)>; /* ADC_TRG */
@@ -178,6 +199,18 @@ gpio-sd0-pwr-en-hog {
line-name = "gpio_sd0_pwr_en";
};
+ qspi0_pins: qspi0 {
+ qspi0-data {
+ pins = "QSPI0_IO0", "QSPI0_IO1", "QSPI0_IO2", "QSPI0_IO3";
+ power-source = <1800>;
+ };
+
+ qspi0-ctrl {
+ pins = "QSPI0_SPCLK", "QSPI0_SSL", "QSPI_RESET#";
+ power-source = <1800>;
+ };
+ };
+
/*
* SD0 device selection is XOR between GPIO_SD0_DEV_SEL and SW1[2]
* The below switch logic can be used to select the device between
@@ -243,6 +276,34 @@ sd0_mux_uhs {
};
};
+&sbc {
+ pinctrl-0 = <&qspi0_pins>;
+ pinctrl-names = "default";
+ status = "okay";
+
+ flash@0 {
+ compatible = "micron,mt25qu512a", "jedec,spi-nor";
+ reg = <0>;
+ m25p,fast-read;
+ spi-max-frequency = <50000000>;
+ spi-rx-bus-width = <4>;
+
+ partitions {
+ compatible = "fixed-partitions";
+ #address-cells = <1>;
+ #size-cells = <1>;
+
+ boot@0 {
+ reg = <0x00000000 0x2000000>;
+ read-only;
+ };
+ user@2000000 {
+ reg = <0x2000000 0x2000000>;
+ };
+ };
+ };
+};
+
#if SDHI
&sdhi0 {
pinctrl-0 = <&sdhi0_pins>;
@@ -273,3 +334,18 @@ &sdhi0 {
status = "okay";
};
#endif
+
+&wdt0 {
+ status = "okay";
+ timeout-sec = <60>;
+};
+
+&wdt1 {
+ status = "okay";
+ timeout-sec = <60>;
+};
+
+&wdt2 {
+ status = "okay";
+ timeout-sec = <60>;
+};
diff --git a/arch/arm64/boot/dts/renesas/rzg2l-smarc.dtsi b/arch/arm64/boot/dts/renesas/rzg2l-smarc.dtsi
index 2863e48..6f2a8bd 100644
--- a/arch/arm64/boot/dts/renesas/rzg2l-smarc.dtsi
+++ b/arch/arm64/boot/dts/renesas/rzg2l-smarc.dtsi
@@ -21,9 +21,13 @@
*
*/
+/* comment the #define statement to disable SCIF2 (SER0) on PMOD1 (CN7) */
+#define PMOD1_SER0 1
+
/ {
aliases {
serial0 = &scif0;
+ serial1 = &scif2;
i2c0 = &i2c0;
i2c1 = &i2c1;
i2c3 = &i2c3;
@@ -208,6 +212,13 @@ scif0_pins: scif0 {
<RZG2L_PORT_PINMUX(38, 1, 1)>; /* RxD */
};
+ scif2_pins: scif2 {
+ pinmux = <RZG2L_PORT_PINMUX(48, 0, 1)>, /* TxD */
+ <RZG2L_PORT_PINMUX(48, 1, 1)>, /* RxD */
+ <RZG2L_PORT_PINMUX(48, 3, 1)>, /* CTS# */
+ <RZG2L_PORT_PINMUX(48, 4, 1)>; /* RTS# */
+ };
+
sd1-pwr-en-hog {
gpio-hog;
gpios = <RZG2L_GPIO(39, 2) GPIO_ACTIVE_HIGH>;
@@ -252,6 +263,13 @@ sound_clk_pins: sound_clk {
input-enable;
};
+ spi1_pins: spi1 {
+ pinmux = <RZG2L_PORT_PINMUX(44, 0, 1)>, /* CK */
+ <RZG2L_PORT_PINMUX(44, 1, 1)>, /* MOSI */
+ <RZG2L_PORT_PINMUX(44, 2, 1)>, /* MISO */
+ <RZG2L_PORT_PINMUX(44, 3, 1)>; /* SSL */
+ };
+
ssi0_pins: ssi0 {
pinmux = <RZG2L_PORT_PINMUX(45, 0, 1)>, /* BCK */
<RZG2L_PORT_PINMUX(45, 1, 1)>, /* RCK */
@@ -277,6 +295,23 @@ &scif0 {
status = "okay";
};
+/*
+ * To enable SCIF2 (SER0) on PMOD1 (CN7)
+ * SW1 should be at position 2->3 so that SER0_CTS# line is activated
+ * SW2 should be at position 2->3 so that SER0_TX line is activated
+ * SW3 should be at position 2->3 so that SER0_RX line is activated
+ * SW4 should be at position 2->3 so that SER0_RTS# line is activated
+ */
+#if PMOD1_SER0
+&scif2 {
+ pinctrl-0 = <&scif2_pins>;
+ pinctrl-names = "default";
+
+ uart-has-rtscts;
+ status = "okay";
+};
+#endif
+
&sdhi1 {
pinctrl-0 = <&sdhi1_pins>;
pinctrl-1 = <&sdhi1_pins_uhs>;
@@ -290,6 +325,13 @@ &sdhi1 {
status = "okay";
};
+&spi1 {
+ pinctrl-0 = <&spi1_pins>;
+ pinctrl-names = "default";
+
+ status = "okay";
+};
+
&ssi0 {
pinctrl-0 = <&ssi0_pins>;
pinctrl-names = "default";
diff --git a/arch/arm64/boot/dts/renesas/salvator-common.dtsi b/arch/arm64/boot/dts/renesas/salvator-common.dtsi
index bf37777..6092dc4 100644
--- a/arch/arm64/boot/dts/renesas/salvator-common.dtsi
+++ b/arch/arm64/boot/dts/renesas/salvator-common.dtsi
@@ -97,6 +97,7 @@ hdmi0-out {
port {
hdmi0_con: endpoint {
+ remote-endpoint = <&rcar_dw_hdmi0_out>;
};
};
};
@@ -424,10 +425,6 @@ dw_hdmi0_snd_in: endpoint {
};
};
-&hdmi0_con {
- remote-endpoint = <&rcar_dw_hdmi0_out>;
-};
-
#ifdef SOC_HAS_HDMI1
&hdmi1 {
status = "okay";
diff --git a/arch/arm64/boot/dts/renesas/ulcb.dtsi b/arch/arm64/boot/dts/renesas/ulcb.dtsi
index 7edffe7..a7e93df 100644
--- a/arch/arm64/boot/dts/renesas/ulcb.dtsi
+++ b/arch/arm64/boot/dts/renesas/ulcb.dtsi
@@ -48,6 +48,7 @@ hdmi0-out {
port {
hdmi0_con: endpoint {
+ remote-endpoint = <&rcar_dw_hdmi0_out>;
};
};
};
@@ -199,10 +200,6 @@ dw_hdmi0_snd_in: endpoint {
};
};
-&hdmi0_con {
- remote-endpoint = <&rcar_dw_hdmi0_out>;
-};
-
&i2c2 {
pinctrl-0 = <&i2c2_pins>;
pinctrl-names = "default";
diff --git a/arch/arm64/boot/dts/rockchip/px30.dtsi b/arch/arm64/boot/dts/rockchip/px30.dtsi
index 00f50b0..f972704 100644
--- a/arch/arm64/boot/dts/rockchip/px30.dtsi
+++ b/arch/arm64/boot/dts/rockchip/px30.dtsi
@@ -1072,7 +1072,7 @@ vpu_mmu: iommu@ff442800 {
};
dsi: dsi@ff450000 {
- compatible = "rockchip,px30-mipi-dsi";
+ compatible = "rockchip,px30-mipi-dsi", "snps,dw-mipi-dsi";
reg = <0x0 0xff450000 0x0 0x10000>;
interrupts = <GIC_SPI 75 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&cru PCLK_MIPI_DSI>;
diff --git a/arch/arm64/boot/dts/rockchip/rk3399-kobol-helios64.dts b/arch/arm64/boot/dts/rockchip/rk3399-kobol-helios64.dts
index b6ac00f..1eb287a 100644
--- a/arch/arm64/boot/dts/rockchip/rk3399-kobol-helios64.dts
+++ b/arch/arm64/boot/dts/rockchip/rk3399-kobol-helios64.dts
@@ -89,6 +89,30 @@ led-1 {
};
};
+ hdd_a_power: hdd-a-power {
+ compatible = "regulator-fixed";
+ enable-active-high;
+ gpio = <&gpio1 RK_PA0 GPIO_ACTIVE_HIGH>;
+ pinctrl-0 = <&hdd_a_power_en>;
+ pinctrl-names = "default";
+ regulator-always-on;
+ regulator-boot-on;
+ regulator-name = "hdd_a_power";
+ startup-delay-us = <2000000>;
+ };
+
+ hdd_b_power: hdd-b-power {
+ compatible = "regulator-fixed";
+ enable-active-high;
+ gpio = <&gpio1 RK_PA1 GPIO_ACTIVE_HIGH>;
+ pinctrl-0 = <&hdd_b_power_en>;
+ pinctrl-names = "default";
+ regulator-always-on;
+ regulator-boot-on;
+ regulator-name = "hdd_b_power";
+ startup-delay-us = <2000000>;
+ };
+
pcie_power: pcie-power {
compatible = "regulator-fixed";
enable-active-high;
@@ -101,6 +125,18 @@ pcie_power: pcie-power {
vin-supply = <&vcc5v0_perdev>;
};
+ usblan_power: usblan-power {
+ compatible = "regulator-fixed";
+ enable-active-high;
+ gpio = <&gpio1 RK_PC7 GPIO_ACTIVE_HIGH>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&usb_lan_en>;
+ regulator-name = "usblan_power";
+ regulator-always-on;
+ regulator-boot-on;
+ vin-supply = <&vcc5v0_usb>;
+ };
+
vcc1v8_sys_s0: vcc1v8-sys-s0 {
compatible = "regulator-fixed";
regulator-name = "vcc1v8_sys_s0";
@@ -390,7 +426,10 @@ &pcie_phy {
};
&pcie0 {
+ ep-gpios = <&gpio2 RK_PD4 GPIO_ACTIVE_HIGH>;
+ max-link-speed = <2>;
num-lanes = <2>;
+ pinctrl-names = "default";
status = "okay";
vpcie12v-supply = <&vcc12v_dcin>;
@@ -430,9 +469,21 @@ pmic_int_l: pmic-int-l {
};
power {
+ hdd_a_power_en: hdd-a-power-en {
+ rockchip,pins = <1 RK_PA0 RK_FUNC_GPIO &pcfg_pull_none>;
+ };
+
+ hdd_b_power_en: hdd-b-power-en {
+ rockchip,pins = <1 RK_PA1 RK_FUNC_GPIO &pcfg_pull_none>;
+ };
+
vcc5v0_usb_en: vcc5v0-usb-en {
rockchip,pins = <1 RK_PC6 RK_FUNC_GPIO &pcfg_pull_none>;
};
+
+ usb_lan_en: usb-lan-en {
+ rockchip,pins = <1 RK_PC7 RK_FUNC_GPIO &pcfg_pull_none>;
+ };
};
vcc3v0-sd {
@@ -531,5 +582,41 @@ &usbdrd3_1 {
usb@fe900000 {
dr_mode = "host";
status = "okay";
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ hub@1 {
+ compatible = "usb2109,0815";
+ reg = <1>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@1 {
+ reg = <1>;
+ #trigger-source-cells = <0>;
+ };
+
+ port@2 {
+ reg = <2>;
+ #trigger-source-cells = <0>;
+ };
+
+ port@3 {
+ reg = <3>;
+ #trigger-source-cells = <0>;
+ };
+
+ device@4 {
+ compatible = "usbbda,8156";
+ reg = <4>;
+ #address-cells = <2>;
+ #size-cells = <0>;
+
+ interface@0 { /* interface 0 of configuration 1 */
+ compatible = "usbbda,8156.config1.0";
+ reg = <0 1>;
+ };
+ };
+ };
};
};
diff --git a/arch/arm64/boot/dts/rockchip/rk3399-rock-pi-4.dtsi b/arch/arm64/boot/dts/rockchip/rk3399-rock-pi-4.dtsi
index 6a434be..92acf6e 100644
--- a/arch/arm64/boot/dts/rockchip/rk3399-rock-pi-4.dtsi
+++ b/arch/arm64/boot/dts/rockchip/rk3399-rock-pi-4.dtsi
@@ -36,7 +36,7 @@ sdio_pwrseq: sdio-pwrseq {
reset-gpios = <&gpio0 RK_PB2 GPIO_ACTIVE_LOW>;
};
- sound {
+ sound: sound {
compatible = "audio-graph-card";
label = "Analog";
dais = <&i2s0_p0>;
@@ -543,6 +543,16 @@ bt_wake_l: bt-wake-l {
};
};
+ es8316 {
+ hp_detect: hp-detect {
+ rockchip,pins = <1 RK_PA0 RK_FUNC_GPIO &pcfg_pull_none>;
+ };
+
+ hp_int: hp-int {
+ rockchip,pins = <1 RK_PA1 RK_FUNC_GPIO &pcfg_pull_up>;
+ };
+ };
+
pcie {
pcie_pwr_en: pcie-pwr-en {
rockchip,pins = <2 RK_PD2 RK_FUNC_GPIO &pcfg_pull_none>;
diff --git a/arch/arm64/boot/dts/rockchip/rk3399-rock-pi-4a-plus.dts b/arch/arm64/boot/dts/rockchip/rk3399-rock-pi-4a-plus.dts
index 281a04b..f5a68d8 100644
--- a/arch/arm64/boot/dts/rockchip/rk3399-rock-pi-4a-plus.dts
+++ b/arch/arm64/boot/dts/rockchip/rk3399-rock-pi-4a-plus.dts
@@ -12,3 +12,14 @@ / {
model = "Radxa ROCK Pi 4A+";
compatible = "radxa,rockpi4a-plus", "radxa,rockpi4", "rockchip,rk3399";
};
+
+&es8316 {
+ pinctrl-0 = <&hp_detect &hp_int>;
+ pinctrl-names = "default";
+ interrupt-parent = <&gpio1>;
+ interrupts = <RK_PA1 IRQ_TYPE_LEVEL_HIGH>;
+};
+
+&sound {
+ hp-det-gpio = <&gpio1 RK_PA0 GPIO_ACTIVE_HIGH>;
+};
diff --git a/arch/arm64/boot/dts/rockchip/rk3399-rock-pi-4b-plus.dts b/arch/arm64/boot/dts/rockchip/rk3399-rock-pi-4b-plus.dts
index dfad13d..cec3b7b 100644
--- a/arch/arm64/boot/dts/rockchip/rk3399-rock-pi-4b-plus.dts
+++ b/arch/arm64/boot/dts/rockchip/rk3399-rock-pi-4b-plus.dts
@@ -17,6 +17,13 @@ aliases {
};
};
+&es8316 {
+ pinctrl-0 = <&hp_detect &hp_int>;
+ pinctrl-names = "default";
+ interrupt-parent = <&gpio1>;
+ interrupts = <RK_PA1 IRQ_TYPE_LEVEL_HIGH>;
+};
+
&sdio0 {
status = "okay";
@@ -31,17 +38,24 @@ brcmf: wifi@1 {
};
};
+&sound {
+ hp-det-gpio = <&gpio1 RK_PA0 GPIO_ACTIVE_HIGH>;
+};
+
&uart0 {
status = "okay";
bluetooth {
- compatible = "brcm,bcm43438-bt";
+ compatible = "brcm,bcm4345c5";
clocks = <&rk808 1>;
- clock-names = "ext_clock";
+ clock-names = "lpo";
device-wakeup-gpios = <&gpio2 RK_PD3 GPIO_ACTIVE_HIGH>;
host-wakeup-gpios = <&gpio0 RK_PA4 GPIO_ACTIVE_HIGH>;
shutdown-gpios = <&gpio0 RK_PB1 GPIO_ACTIVE_HIGH>;
+ max-speed = <1500000>;
pinctrl-names = "default";
pinctrl-0 = <&bt_host_wake_l &bt_wake_l &bt_enable_h>;
+ vbat-supply = <&vcc3v3_sys>;
+ vddio-supply = <&vcc_1v8>;
};
};
diff --git a/arch/arm64/boot/dts/rockchip/rk3399-rock-pi-4b.dts b/arch/arm64/boot/dts/rockchip/rk3399-rock-pi-4b.dts
index 6c63e61..cf48746 100644
--- a/arch/arm64/boot/dts/rockchip/rk3399-rock-pi-4b.dts
+++ b/arch/arm64/boot/dts/rockchip/rk3399-rock-pi-4b.dts
@@ -34,13 +34,16 @@ &uart0 {
status = "okay";
bluetooth {
- compatible = "brcm,bcm43438-bt";
+ compatible = "brcm,bcm4345c5";
clocks = <&rk808 1>;
- clock-names = "ext_clock";
+ clock-names = "lpo";
device-wakeup-gpios = <&gpio2 RK_PD3 GPIO_ACTIVE_HIGH>;
host-wakeup-gpios = <&gpio0 RK_PA4 GPIO_ACTIVE_HIGH>;
shutdown-gpios = <&gpio0 RK_PB1 GPIO_ACTIVE_HIGH>;
+ max-speed = <1500000>;
pinctrl-names = "default";
pinctrl-0 = <&bt_host_wake_l &bt_wake_l &bt_enable_h>;
+ vbat-supply = <&vcc3v3_sys>;
+ vddio-supply = <&vcc_1v8>;
};
};
diff --git a/arch/arm64/boot/dts/rockchip/rk3399-rock-pi-4c.dts b/arch/arm64/boot/dts/rockchip/rk3399-rock-pi-4c.dts
index 99169bc..793d848 100644
--- a/arch/arm64/boot/dts/rockchip/rk3399-rock-pi-4c.dts
+++ b/arch/arm64/boot/dts/rockchip/rk3399-rock-pi-4c.dts
@@ -17,6 +17,13 @@ aliases {
};
};
+&es8316 {
+ pinctrl-0 = <&hp_detect &hp_int>;
+ pinctrl-names = "default";
+ interrupt-parent = <&gpio1>;
+ interrupts = <RK_PA1 IRQ_TYPE_LEVEL_HIGH>;
+};
+
&sdio0 {
status = "okay";
@@ -31,18 +38,25 @@ brcmf: wifi@1 {
};
};
+&sound {
+ hp-det-gpio = <&gpio1 RK_PA0 GPIO_ACTIVE_HIGH>;
+};
+
&uart0 {
status = "okay";
bluetooth {
- compatible = "brcm,bcm43438-bt";
+ compatible = "brcm,bcm4345c5";
clocks = <&rk808 1>;
- clock-names = "ext_clock";
+ clock-names = "lpo";
device-wakeup-gpios = <&gpio2 RK_PD3 GPIO_ACTIVE_HIGH>;
host-wakeup-gpios = <&gpio0 RK_PA4 GPIO_ACTIVE_HIGH>;
shutdown-gpios = <&gpio0 RK_PB1 GPIO_ACTIVE_HIGH>;
+ max-speed = <1500000>;
pinctrl-names = "default";
pinctrl-0 = <&bt_host_wake_l &bt_wake_l &bt_enable_h>;
+ vbat-supply = <&vcc3v3_sys>;
+ vddio-supply = <&vcc_1v8>;
};
};
diff --git a/arch/arm64/boot/dts/rockchip/rk3566-quartz64-a.dts b/arch/arm64/boot/dts/rockchip/rk3566-quartz64-a.dts
index 4d4b2a3..166399b 100644
--- a/arch/arm64/boot/dts/rockchip/rk3566-quartz64-a.dts
+++ b/arch/arm64/boot/dts/rockchip/rk3566-quartz64-a.dts
@@ -509,6 +509,11 @@ &spdif {
status = "okay";
};
+&spi1 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&spi1m1_cs0 &spi1m1_pins>;
+};
+
&tsadc {
/* tshut mode 0:CRU 1:GPIO */
rockchip,hw-tshut-mode = <1>;
diff --git a/arch/arm64/boot/dts/rockchip/rk356x.dtsi b/arch/arm64/boot/dts/rockchip/rk356x.dtsi
index 46d9552..a68033a 100644
--- a/arch/arm64/boot/dts/rockchip/rk356x.dtsi
+++ b/arch/arm64/boot/dts/rockchip/rk356x.dtsi
@@ -39,6 +39,10 @@ aliases {
serial7 = &uart7;
serial8 = &uart8;
serial9 = &uart9;
+ spi0 = &spi0;
+ spi1 = &spi1;
+ spi2 = &spi2;
+ spi3 = &spi3;
};
cpus {
@@ -269,7 +273,7 @@ pwm0: pwm@fdd70000 {
clocks = <&pmucru CLK_PWM0>, <&pmucru PCLK_PWM0>;
clock-names = "pwm", "pclk";
pinctrl-0 = <&pwm0m0_pins>;
- pinctrl-names = "active";
+ pinctrl-names = "default";
#pwm-cells = <3>;
status = "disabled";
};
@@ -280,7 +284,7 @@ pwm1: pwm@fdd70010 {
clocks = <&pmucru CLK_PWM0>, <&pmucru PCLK_PWM0>;
clock-names = "pwm", "pclk";
pinctrl-0 = <&pwm1m0_pins>;
- pinctrl-names = "active";
+ pinctrl-names = "default";
#pwm-cells = <3>;
status = "disabled";
};
@@ -291,7 +295,7 @@ pwm2: pwm@fdd70020 {
clocks = <&pmucru CLK_PWM0>, <&pmucru PCLK_PWM0>;
clock-names = "pwm", "pclk";
pinctrl-0 = <&pwm2m0_pins>;
- pinctrl-names = "active";
+ pinctrl-names = "default";
#pwm-cells = <3>;
status = "disabled";
};
@@ -302,7 +306,7 @@ pwm3: pwm@fdd70030 {
clocks = <&pmucru CLK_PWM0>, <&pmucru PCLK_PWM0>;
clock-names = "pwm", "pclk";
pinctrl-0 = <&pwm3_pins>;
- pinctrl-names = "active";
+ pinctrl-names = "default";
#pwm-cells = <3>;
status = "disabled";
};
@@ -742,6 +746,66 @@ wdt: watchdog@fe600000 {
clock-names = "tclk", "pclk";
};
+ spi0: spi@fe610000 {
+ compatible = "rockchip,rk3568-spi", "rockchip,rk3066-spi";
+ reg = <0x0 0xfe610000 0x0 0x1000>;
+ interrupts = <GIC_SPI 103 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&cru CLK_SPI0>, <&cru PCLK_SPI0>;
+ clock-names = "spiclk", "apb_pclk";
+ dmas = <&dmac0 20>, <&dmac0 21>;
+ dma-names = "tx", "rx";
+ pinctrl-names = "default";
+ pinctrl-0 = <&spi0m0_cs0 &spi0m0_cs1 &spi0m0_pins>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ spi1: spi@fe620000 {
+ compatible = "rockchip,rk3568-spi", "rockchip,rk3066-spi";
+ reg = <0x0 0xfe620000 0x0 0x1000>;
+ interrupts = <GIC_SPI 104 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&cru CLK_SPI1>, <&cru PCLK_SPI1>;
+ clock-names = "spiclk", "apb_pclk";
+ dmas = <&dmac0 22>, <&dmac0 23>;
+ dma-names = "tx", "rx";
+ pinctrl-names = "default";
+ pinctrl-0 = <&spi1m0_cs0 &spi1m0_cs1 &spi1m0_pins>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ spi2: spi@fe630000 {
+ compatible = "rockchip,rk3568-spi", "rockchip,rk3066-spi";
+ reg = <0x0 0xfe630000 0x0 0x1000>;
+ interrupts = <GIC_SPI 105 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&cru CLK_SPI2>, <&cru PCLK_SPI2>;
+ clock-names = "spiclk", "apb_pclk";
+ dmas = <&dmac0 24>, <&dmac0 25>;
+ dma-names = "tx", "rx";
+ pinctrl-names = "default";
+ pinctrl-0 = <&spi2m0_cs0 &spi2m0_cs1 &spi2m0_pins>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ spi3: spi@fe640000 {
+ compatible = "rockchip,rk3568-spi", "rockchip,rk3066-spi";
+ reg = <0x0 0xfe640000 0x0 0x1000>;
+ interrupts = <GIC_SPI 106 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&cru CLK_SPI3>, <&cru PCLK_SPI3>;
+ clock-names = "spiclk", "apb_pclk";
+ dmas = <&dmac0 26>, <&dmac0 27>;
+ dma-names = "tx", "rx";
+ pinctrl-names = "default";
+ pinctrl-0 = <&spi3m0_cs0 &spi3m0_cs1 &spi3m0_pins>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
uart1: serial@fe650000 {
compatible = "rockchip,rk3568-uart", "snps,dw-apb-uart";
reg = <0x0 0xfe650000 0x0 0x100>;
@@ -951,7 +1015,7 @@ pwm4: pwm@fe6e0000 {
clocks = <&cru CLK_PWM1>, <&cru PCLK_PWM1>;
clock-names = "pwm", "pclk";
pinctrl-0 = <&pwm4_pins>;
- pinctrl-names = "active";
+ pinctrl-names = "default";
#pwm-cells = <3>;
status = "disabled";
};
@@ -962,7 +1026,7 @@ pwm5: pwm@fe6e0010 {
clocks = <&cru CLK_PWM1>, <&cru PCLK_PWM1>;
clock-names = "pwm", "pclk";
pinctrl-0 = <&pwm5_pins>;
- pinctrl-names = "active";
+ pinctrl-names = "default";
#pwm-cells = <3>;
status = "disabled";
};
@@ -973,7 +1037,7 @@ pwm6: pwm@fe6e0020 {
clocks = <&cru CLK_PWM1>, <&cru PCLK_PWM1>;
clock-names = "pwm", "pclk";
pinctrl-0 = <&pwm6_pins>;
- pinctrl-names = "active";
+ pinctrl-names = "default";
#pwm-cells = <3>;
status = "disabled";
};
@@ -984,7 +1048,7 @@ pwm7: pwm@fe6e0030 {
clocks = <&cru CLK_PWM1>, <&cru PCLK_PWM1>;
clock-names = "pwm", "pclk";
pinctrl-0 = <&pwm7_pins>;
- pinctrl-names = "active";
+ pinctrl-names = "default";
#pwm-cells = <3>;
status = "disabled";
};
@@ -995,7 +1059,7 @@ pwm8: pwm@fe6f0000 {
clocks = <&cru CLK_PWM2>, <&cru PCLK_PWM2>;
clock-names = "pwm", "pclk";
pinctrl-0 = <&pwm8m0_pins>;
- pinctrl-names = "active";
+ pinctrl-names = "default";
#pwm-cells = <3>;
status = "disabled";
};
@@ -1006,7 +1070,7 @@ pwm9: pwm@fe6f0010 {
clocks = <&cru CLK_PWM2>, <&cru PCLK_PWM2>;
clock-names = "pwm", "pclk";
pinctrl-0 = <&pwm9m0_pins>;
- pinctrl-names = "active";
+ pinctrl-names = "default";
#pwm-cells = <3>;
status = "disabled";
};
@@ -1017,7 +1081,7 @@ pwm10: pwm@fe6f0020 {
clocks = <&cru CLK_PWM2>, <&cru PCLK_PWM2>;
clock-names = "pwm", "pclk";
pinctrl-0 = <&pwm10m0_pins>;
- pinctrl-names = "active";
+ pinctrl-names = "default";
#pwm-cells = <3>;
status = "disabled";
};
@@ -1028,7 +1092,7 @@ pwm11: pwm@fe6f0030 {
clocks = <&cru CLK_PWM2>, <&cru PCLK_PWM2>;
clock-names = "pwm", "pclk";
pinctrl-0 = <&pwm11m0_pins>;
- pinctrl-names = "active";
+ pinctrl-names = "default";
#pwm-cells = <3>;
status = "disabled";
};
@@ -1039,7 +1103,7 @@ pwm12: pwm@fe700000 {
clocks = <&cru CLK_PWM3>, <&cru PCLK_PWM3>;
clock-names = "pwm", "pclk";
pinctrl-0 = <&pwm12m0_pins>;
- pinctrl-names = "active";
+ pinctrl-names = "default";
#pwm-cells = <3>;
status = "disabled";
};
@@ -1050,7 +1114,7 @@ pwm13: pwm@fe700010 {
clocks = <&cru CLK_PWM3>, <&cru PCLK_PWM3>;
clock-names = "pwm", "pclk";
pinctrl-0 = <&pwm13m0_pins>;
- pinctrl-names = "active";
+ pinctrl-names = "default";
#pwm-cells = <3>;
status = "disabled";
};
@@ -1061,7 +1125,7 @@ pwm14: pwm@fe700020 {
clocks = <&cru CLK_PWM3>, <&cru PCLK_PWM3>;
clock-names = "pwm", "pclk";
pinctrl-0 = <&pwm14m0_pins>;
- pinctrl-names = "active";
+ pinctrl-names = "default";
#pwm-cells = <3>;
status = "disabled";
};
@@ -1072,7 +1136,7 @@ pwm15: pwm@fe700030 {
clocks = <&cru CLK_PWM3>, <&cru PCLK_PWM3>;
clock-names = "pwm", "pclk";
pinctrl-0 = <&pwm15m0_pins>;
- pinctrl-names = "active";
+ pinctrl-names = "default";
#pwm-cells = <3>;
status = "disabled";
};
diff --git a/arch/arm64/boot/dts/ti/Makefile b/arch/arm64/boot/dts/ti/Makefile
index 71f6097..90be511 100644
--- a/arch/arm64/boot/dts/ti/Makefile
+++ b/arch/arm64/boot/dts/ti/Makefile
@@ -17,5 +17,7 @@
dtb-$(CONFIG_ARCH_K3) += k3-j7200-common-proc-board.dtb
+dtb-$(CONFIG_ARCH_K3) += k3-j721s2-common-proc-board.dtb
+
dtb-$(CONFIG_ARCH_K3) += k3-am642-evm.dtb
dtb-$(CONFIG_ARCH_K3) += k3-am642-sk.dtb
diff --git a/arch/arm64/boot/dts/ti/k3-am64-main.dtsi b/arch/arm64/boot/dts/ti/k3-am64-main.dtsi
index 5ad638b..012011d 100644
--- a/arch/arm64/boot/dts/ti/k3-am64-main.dtsi
+++ b/arch/arm64/boot/dts/ti/k3-am64-main.dtsi
@@ -564,6 +564,14 @@ cpts@39000000 {
ti,cpts-ext-ts-inputs = <8>;
};
+ timesync_router: pinctrl@a40000 {
+ compatible = "pinctrl-single";
+ reg = <0x0 0xa40000 0x0 0x800>;
+ #pinctrl-cells = <1>;
+ pinctrl-single,register-width = <32>;
+ pinctrl-single,function-mask = <0x000107ff>;
+ };
+
usbss0: cdns-usb@f900000{
compatible = "ti,am64-usb";
reg = <0x00 0xf900000 0x00 0x100>;
@@ -1253,4 +1261,32 @@ icssg1_mdio: mdio@32400 {
bus_freq = <1000000>;
};
};
+
+ main_mcan0: can@20701000 {
+ compatible = "bosch,m_can";
+ reg = <0x00 0x20701000 0x00 0x200>,
+ <0x00 0x20708000 0x00 0x8000>;
+ reg-names = "m_can", "message_ram";
+ power-domains = <&k3_pds 98 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 98 5>, <&k3_clks 98 0>;
+ clock-names = "hclk", "cclk";
+ interrupts = <GIC_SPI 155 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 156 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "int0", "int1";
+ bosch,mram-cfg = <0x0 128 64 64 64 64 32 32>;
+ };
+
+ main_mcan1: can@20711000 {
+ compatible = "bosch,m_can";
+ reg = <0x00 0x20711000 0x00 0x200>,
+ <0x00 0x20718000 0x00 0x8000>;
+ reg-names = "m_can", "message_ram";
+ power-domains = <&k3_pds 99 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 99 5>, <&k3_clks 99 0>;
+ clock-names = "hclk", "cclk";
+ interrupts = <GIC_SPI 158 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 159 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "int0", "int1";
+ bosch,mram-cfg = <0x0 128 64 64 64 64 32 32>;
+ };
};
diff --git a/arch/arm64/boot/dts/ti/k3-am642-evm.dts b/arch/arm64/boot/dts/ti/k3-am642-evm.dts
index 6726c4c..e94ae17 100644
--- a/arch/arm64/boot/dts/ti/k3-am642-evm.dts
+++ b/arch/arm64/boot/dts/ti/k3-am642-evm.dts
@@ -184,6 +184,20 @@ cpsw3g_phy3: ethernet-phy@3 {
};
};
};
+
+ transceiver1: can-phy0 {
+ compatible = "ti,tcan1042";
+ #phy-cells = <0>;
+ max-bitrate = <5000000>;
+ standby-gpios = <&exp1 8 GPIO_ACTIVE_HIGH>;
+ };
+
+ transceiver2: can-phy1 {
+ compatible = "ti,tcan1042";
+ #phy-cells = <0>;
+ max-bitrate = <5000000>;
+ standby-gpios = <&exp1 9 GPIO_ACTIVE_HIGH>;
+ };
};
&main_pmx0 {
@@ -294,6 +308,20 @@ main_ecap0_pins_default: main-ecap0-pins-default {
AM64X_IOPAD(0x0270, PIN_INPUT, 0) /* (D18) ECAP0_IN_APWM_OUT */
>;
};
+
+ main_mcan0_pins_default: main-mcan0-pins-default {
+ pinctrl-single,pins = <
+ AM64X_IOPAD(0x0254, PIN_INPUT, 0) /* (B17) MCAN0_RX */
+ AM64X_IOPAD(0x0250, PIN_OUTPUT, 0) /* (A17) MCAN0_TX */
+ >;
+ };
+
+ main_mcan1_pins_default: main-mcan1-pins-default {
+ pinctrl-single,pins = <
+ AM64X_IOPAD(0x025c, PIN_INPUT, 0) /* (D17) MCAN1_RX */
+ AM64X_IOPAD(0x0258, PIN_OUTPUT, 0) /* (C17) MCAN1_TX */
+ >;
+ };
};
&main_uart0 {
@@ -638,3 +666,15 @@ &icssg0_mdio {
&icssg1_mdio {
status = "disabled";
};
+
+&main_mcan0 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&main_mcan0_pins_default>;
+ phys = <&transceiver1>;
+};
+
+&main_mcan1 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&main_mcan1_pins_default>;
+ phys = <&transceiver2>;
+};
diff --git a/arch/arm64/boot/dts/ti/k3-am642-sk.dts b/arch/arm64/boot/dts/ti/k3-am642-sk.dts
index 6b04745..a9785be 100644
--- a/arch/arm64/boot/dts/ti/k3-am642-sk.dts
+++ b/arch/arm64/boot/dts/ti/k3-am642-sk.dts
@@ -525,3 +525,11 @@ &icssg0_mdio {
&icssg1_mdio {
status = "disabled";
};
+
+&main_mcan0 {
+ status = "disabled";
+};
+
+&main_mcan1 {
+ status = "disabled";
+};
diff --git a/arch/arm64/boot/dts/ti/k3-am642.dtsi b/arch/arm64/boot/dts/ti/k3-am642.dtsi
index e2b397c..8a76f48 100644
--- a/arch/arm64/boot/dts/ti/k3-am642.dtsi
+++ b/arch/arm64/boot/dts/ti/k3-am642.dtsi
@@ -60,6 +60,6 @@ L2_0: l2-cache0 {
cache-level = <2>;
cache-size = <0x40000>;
cache-line-size = <64>;
- cache-sets = <512>;
+ cache-sets = <256>;
};
};
diff --git a/arch/arm64/boot/dts/ti/k3-am65-iot2050-common.dtsi b/arch/arm64/boot/dts/ti/k3-am65-iot2050-common.dtsi
index 65da226..3079eae 100644
--- a/arch/arm64/boot/dts/ti/k3-am65-iot2050-common.dtsi
+++ b/arch/arm64/boot/dts/ti/k3-am65-iot2050-common.dtsi
@@ -646,6 +646,14 @@ &pcie1_rc {
reset-gpios = <&wkup_gpio0 27 GPIO_ACTIVE_HIGH>;
};
+&m_can0 {
+ status = "disabled";
+};
+
+&m_can1 {
+ status = "disabled";
+};
+
&pcie1_ep {
status = "disabled";
};
@@ -731,3 +739,15 @@ &icssg1_mdio {
&icssg2_mdio {
status = "disabled";
};
+
+&mcasp0 {
+ status = "disabled";
+};
+
+&mcasp1 {
+ status = "disabled";
+};
+
+&mcasp2 {
+ status = "disabled";
+};
diff --git a/arch/arm64/boot/dts/ti/k3-am65-mcu.dtsi b/arch/arm64/boot/dts/ti/k3-am65-mcu.dtsi
index c93ff15..8d592bf 100644
--- a/arch/arm64/boot/dts/ti/k3-am65-mcu.dtsi
+++ b/arch/arm64/boot/dts/ti/k3-am65-mcu.dtsi
@@ -159,6 +159,36 @@ mcu_udmap: dma-controller@285c0000 {
};
};
+ m_can0: mcan@40528000 {
+ compatible = "bosch,m_can";
+ reg = <0x0 0x40528000 0x0 0x400>,
+ <0x0 0x40500000 0x0 0x4400>;
+ reg-names = "m_can", "message_ram";
+ power-domains = <&k3_pds 102 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 102 5>, <&k3_clks 102 0>;
+ clock-names = "hclk", "cclk";
+ interrupt-parent = <&gic500>;
+ interrupts = <GIC_SPI 544 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 545 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "int0", "int1";
+ bosch,mram-cfg = <0x0 128 64 64 64 64 32 32>;
+ };
+
+ m_can1: mcan@40568000 {
+ compatible = "bosch,m_can";
+ reg = <0x0 0x40568000 0x0 0x400>,
+ <0x0 0x40540000 0x0 0x4400>;
+ reg-names = "m_can", "message_ram";
+ power-domains = <&k3_pds 103 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 103 5>, <&k3_clks 103 0>;
+ clock-names = "hclk", "cclk";
+ interrupt-parent = <&gic500>;
+ interrupts = <GIC_SPI 547 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 548 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "int0", "int1";
+ bosch,mram-cfg = <0x0 128 64 64 64 64 32 32>;
+ };
+
fss: fss@47000000 {
compatible = "simple-bus";
#address-cells = <2>;
diff --git a/arch/arm64/boot/dts/ti/k3-am654-base-board.dts b/arch/arm64/boot/dts/ti/k3-am654-base-board.dts
index cfbcebf..9043f91 100644
--- a/arch/arm64/boot/dts/ti/k3-am654-base-board.dts
+++ b/arch/arm64/boot/dts/ti/k3-am654-base-board.dts
@@ -416,6 +416,14 @@ &pcie1_ep {
status = "disabled";
};
+&m_can0 {
+ status = "disabled";
+};
+
+&m_can1 {
+ status = "disabled";
+};
+
&mailbox0_cluster0 {
interrupts = <436>;
diff --git a/arch/arm64/boot/dts/ti/k3-j7200-main.dtsi b/arch/arm64/boot/dts/ti/k3-j7200-main.dtsi
index d60ef4f..05a627a 100644
--- a/arch/arm64/boot/dts/ti/k3-j7200-main.dtsi
+++ b/arch/arm64/boot/dts/ti/k3-j7200-main.dtsi
@@ -32,7 +32,7 @@ scm_conf: scm-conf@100000 {
#size-cells = <1>;
ranges = <0x00 0x00 0x00100000 0x1c000>;
- serdes_ln_ctrl: serdes-ln-ctrl@4080 {
+ serdes_ln_ctrl: mux-controller@4080 {
compatible = "mmio-mux";
#mux-control-cells = <1>;
mux-reg-masks = <0x4080 0x3>, <0x4084 0x3>, /* SERDES0 lane0/1 select */
diff --git a/arch/arm64/boot/dts/ti/k3-j7200.dtsi b/arch/arm64/boot/dts/ti/k3-j7200.dtsi
index 47567cb..64fef4e 100644
--- a/arch/arm64/boot/dts/ti/k3-j7200.dtsi
+++ b/arch/arm64/boot/dts/ti/k3-j7200.dtsi
@@ -62,7 +62,7 @@ cpu0: cpu@0 {
i-cache-sets = <256>;
d-cache-size = <0x8000>;
d-cache-line-size = <64>;
- d-cache-sets = <128>;
+ d-cache-sets = <256>;
next-level-cache = <&L2_0>;
};
@@ -76,7 +76,7 @@ cpu1: cpu@1 {
i-cache-sets = <256>;
d-cache-size = <0x8000>;
d-cache-line-size = <64>;
- d-cache-sets = <128>;
+ d-cache-sets = <256>;
next-level-cache = <&L2_0>;
};
};
@@ -86,7 +86,7 @@ L2_0: l2-cache0 {
cache-level = <2>;
cache-size = <0x100000>;
cache-line-size = <64>;
- cache-sets = <2048>;
+ cache-sets = <1024>;
next-level-cache = <&msmc_l3>;
};
diff --git a/arch/arm64/boot/dts/ti/k3-j721e-common-proc-board.dts b/arch/arm64/boot/dts/ti/k3-j721e-common-proc-board.dts
index dc2bc67..2d75969 100644
--- a/arch/arm64/boot/dts/ti/k3-j721e-common-proc-board.dts
+++ b/arch/arm64/boot/dts/ti/k3-j721e-common-proc-board.dts
@@ -112,6 +112,42 @@ sound0: sound@0 {
"cpb-codec-scki",
"cpb-codec-scki-48000", "cpb-codec-scki-44100";
};
+
+ transceiver1: can-phy0 {
+ compatible = "ti,tcan1043";
+ #phy-cells = <0>;
+ max-bitrate = <5000000>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&mcu_mcan0_gpio_pins_default>;
+ standby-gpios = <&wkup_gpio0 54 GPIO_ACTIVE_LOW>;
+ enable-gpios = <&wkup_gpio0 0 GPIO_ACTIVE_HIGH>;
+ };
+
+ transceiver2: can-phy1 {
+ compatible = "ti,tcan1042";
+ #phy-cells = <0>;
+ max-bitrate = <5000000>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&mcu_mcan1_gpio_pins_default>;
+ standby-gpios = <&wkup_gpio0 2 GPIO_ACTIVE_HIGH>;
+ };
+
+ transceiver3: can-phy2 {
+ compatible = "ti,tcan1043";
+ #phy-cells = <0>;
+ max-bitrate = <5000000>;
+ standby-gpios = <&exp2 7 GPIO_ACTIVE_LOW>;
+ enable-gpios = <&exp2 6 GPIO_ACTIVE_HIGH>;
+ };
+
+ transceiver4: can-phy3 {
+ compatible = "ti,tcan1042";
+ #phy-cells = <0>;
+ max-bitrate = <5000000>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&main_mcan2_gpio_pins_default>;
+ standby-gpios = <&main_gpio0 127 GPIO_ACTIVE_HIGH>;
+ };
};
&main_pmx0 {
@@ -207,6 +243,26 @@ audi_ext_refclk2_pins_default: audi-ext-refclk2-pins-default {
J721E_IOPAD(0x1a4, PIN_OUTPUT, 3) /* (W26) RGMII6_RXC.AUDIO_EXT_REFCLK2 */
>;
};
+
+ main_mcan0_pins_default: main-mcan0-pins-default {
+ pinctrl-single,pins = <
+ J721E_IOPAD(0x208, PIN_INPUT, 0) /* (W5) MCAN0_RX */
+ J721E_IOPAD(0x20c, PIN_OUTPUT, 0) /* (W6) MCAN0_TX */
+ >;
+ };
+
+ main_mcan2_pins_default: main-mcan2-pins-default {
+ pinctrl-single,pins = <
+ J721E_IOPAD(0x01f0, PIN_INPUT, 3) /* (AC2) MCAN2_RX.GPIO0_123 */
+ J721E_IOPAD(0x01f4, PIN_OUTPUT, 3) /* (AB1) MCAN2_TX.GPIO0_124 */
+ >;
+ };
+
+ main_mcan2_gpio_pins_default: main-mcan2-gpio-pins-default {
+ pinctrl-single,pins = <
+ J721E_IOPAD(0x200, PIN_INPUT, 7) /* (AC4) UART1_CTSn.GPIO0_127 */
+ >;
+ };
};
&wkup_pmx0 {
@@ -252,6 +308,33 @@ J721E_WKUP_IOPAD(0x008c, PIN_OUTPUT, 0) /* MCU_MDIO0_MDC */
J721E_WKUP_IOPAD(0x0088, PIN_INPUT, 0) /* MCU_MDIO0_MDIO */
>;
};
+
+ mcu_mcan0_pins_default: mcu-mcan0-pins-default {
+ pinctrl-single,pins = <
+ J721E_WKUP_IOPAD(0xac, PIN_INPUT, 0) /* (C29) MCU_MCAN0_RX */
+ J721E_WKUP_IOPAD(0xa8, PIN_OUTPUT, 0) /* (D29) MCU_MCAN0_TX */
+ >;
+ };
+
+ mcu_mcan0_gpio_pins_default: mcu-mcan0-gpio-pins-default {
+ pinctrl-single,pins = <
+ J721E_WKUP_IOPAD(0xb0, PIN_INPUT, 7) /* (F26) WKUP_GPIO0_0 */
+ J721E_WKUP_IOPAD(0x98, PIN_INPUT, 7) /* (E28) MCU_SPI0_D1.WKUP_GPIO0_54 */
+ >;
+ };
+
+ mcu_mcan1_pins_default: mcu-mcan1-pins-default {
+ pinctrl-single,pins = <
+ J721E_WKUP_IOPAD(0xc4, PIN_INPUT, 0) /* (G24) WKUP_GPIO0_5.MCU_MCAN1_RX */
+ J721E_WKUP_IOPAD(0xc0, PIN_OUTPUT, 0) /* (G25) WKUP_GPIO0_4.MCU_MCAN1_TX */
+ >;
+ };
+
+ mcu_mcan1_gpio_pins_default: mcu-mcan1-gpio-pins-default {
+ pinctrl-single,pins = <
+ J721E_WKUP_IOPAD(0xb8, PIN_INPUT, 7) /* (F28) WKUP_GPIO0_2 */
+ >;
+ };
};
&wkup_uart0 {
@@ -773,3 +856,75 @@ &icssg0_mdio {
&icssg1_mdio {
status = "disabled";
};
+
+&mcu_mcan0 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&mcu_mcan0_pins_default>;
+ phys = <&transceiver1>;
+};
+
+&mcu_mcan1 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&mcu_mcan1_pins_default>;
+ phys = <&transceiver2>;
+};
+
+&main_mcan0 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&main_mcan0_pins_default>;
+ phys = <&transceiver3>;
+};
+
+&main_mcan1 {
+ status = "disabled";
+};
+
+&main_mcan2 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&main_mcan2_pins_default>;
+ phys = <&transceiver4>;
+};
+
+&main_mcan3 {
+ status = "disabled";
+};
+
+&main_mcan4 {
+ status = "disabled";
+};
+
+&main_mcan5 {
+ status = "disabled";
+};
+
+&main_mcan6 {
+ status = "disabled";
+};
+
+&main_mcan7 {
+ status = "disabled";
+};
+
+&main_mcan8 {
+ status = "disabled";
+};
+
+&main_mcan9 {
+ status = "disabled";
+};
+
+&main_mcan10 {
+ status = "disabled";
+};
+
+&main_mcan11 {
+ status = "disabled";
+};
+
+&main_mcan12 {
+ status = "disabled";
+};
+
+&main_mcan13 {
+ status = "disabled";
+};
diff --git a/arch/arm64/boot/dts/ti/k3-j721e-main.dtsi b/arch/arm64/boot/dts/ti/k3-j721e-main.dtsi
index 08c8d1b..5998612 100644
--- a/arch/arm64/boot/dts/ti/k3-j721e-main.dtsi
+++ b/arch/arm64/boot/dts/ti/k3-j721e-main.dtsi
@@ -42,7 +42,7 @@ scm_conf: scm-conf@100000 {
#size-cells = <1>;
ranges = <0x0 0x0 0x00100000 0x1c000>;
- serdes_ln_ctrl: mux@4080 {
+ serdes_ln_ctrl: mux-controller@4080 {
compatible = "mmio-mux";
reg = <0x00004080 0x50>;
#mux-control-cells = <1>;
@@ -1940,4 +1940,200 @@ icssg1_mdio: mdio@32400 {
bus_freq = <1000000>;
};
};
+
+ main_mcan0: can@2701000 {
+ compatible = "bosch,m_can";
+ reg = <0x00 0x02701000 0x00 0x200>,
+ <0x00 0x02708000 0x00 0x8000>;
+ reg-names = "m_can", "message_ram";
+ power-domains = <&k3_pds 156 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 156 0>, <&k3_clks 156 1>;
+ clock-names = "hclk", "cclk";
+ interrupts = <GIC_SPI 124 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 125 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "int0", "int1";
+ bosch,mram-cfg = <0x0 128 64 64 64 64 32 32>;
+ };
+
+ main_mcan1: can@2711000 {
+ compatible = "bosch,m_can";
+ reg = <0x00 0x02711000 0x00 0x200>,
+ <0x00 0x02718000 0x00 0x8000>;
+ reg-names = "m_can", "message_ram";
+ power-domains = <&k3_pds 158 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 158 0>, <&k3_clks 158 1>;
+ clock-names = "hclk", "cclk";
+ interrupts = <GIC_SPI 127 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 128 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "int0", "int1";
+ bosch,mram-cfg = <0x0 128 64 64 64 64 32 32>;
+ };
+
+ main_mcan2: can@2721000 {
+ compatible = "bosch,m_can";
+ reg = <0x00 0x02721000 0x00 0x200>,
+ <0x00 0x02728000 0x00 0x8000>;
+ reg-names = "m_can", "message_ram";
+ power-domains = <&k3_pds 160 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 160 0>, <&k3_clks 160 1>;
+ clock-names = "hclk", "cclk";
+ interrupts = <GIC_SPI 130 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 131 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "int0", "int1";
+ bosch,mram-cfg = <0x0 128 64 64 64 64 32 32>;
+ };
+
+ main_mcan3: can@2731000 {
+ compatible = "bosch,m_can";
+ reg = <0x00 0x02731000 0x00 0x200>,
+ <0x00 0x02738000 0x00 0x8000>;
+ reg-names = "m_can", "message_ram";
+ power-domains = <&k3_pds 161 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 161 0>, <&k3_clks 161 1>;
+ clock-names = "hclk", "cclk";
+ interrupts = <GIC_SPI 133 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 134 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "int0", "int1";
+ bosch,mram-cfg = <0x0 128 64 64 64 64 32 32>;
+ };
+
+ main_mcan4: can@2741000 {
+ compatible = "bosch,m_can";
+ reg = <0x00 0x02741000 0x00 0x200>,
+ <0x00 0x02748000 0x00 0x8000>;
+ reg-names = "m_can", "message_ram";
+ power-domains = <&k3_pds 162 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 162 0>, <&k3_clks 162 1>;
+ clock-names = "hclk", "cclk";
+ interrupts = <GIC_SPI 136 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 137 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "int0", "int1";
+ bosch,mram-cfg = <0x0 128 64 64 64 64 32 32>;
+ };
+
+ main_mcan5: can@2751000 {
+ compatible = "bosch,m_can";
+ reg = <0x00 0x02751000 0x00 0x200>,
+ <0x00 0x02758000 0x00 0x8000>;
+ reg-names = "m_can", "message_ram";
+ power-domains = <&k3_pds 163 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 163 0>, <&k3_clks 163 1>;
+ clock-names = "hclk", "cclk";
+ interrupts = <GIC_SPI 139 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 140 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "int0", "int1";
+ bosch,mram-cfg = <0x0 128 64 64 64 64 32 32>;
+ };
+
+ main_mcan6: can@2761000 {
+ compatible = "bosch,m_can";
+ reg = <0x00 0x02761000 0x00 0x200>,
+ <0x00 0x02768000 0x00 0x8000>;
+ reg-names = "m_can", "message_ram";
+ power-domains = <&k3_pds 164 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 164 0>, <&k3_clks 164 1>;
+ clock-names = "hclk", "cclk";
+ interrupts = <GIC_SPI 142 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 143 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "int0", "int1";
+ bosch,mram-cfg = <0x0 128 64 64 64 64 32 32>;
+ };
+
+ main_mcan7: can@2771000 {
+ compatible = "bosch,m_can";
+ reg = <0x00 0x02771000 0x00 0x200>,
+ <0x00 0x02778000 0x00 0x8000>;
+ reg-names = "m_can", "message_ram";
+ power-domains = <&k3_pds 165 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 165 0>, <&k3_clks 165 1>;
+ clock-names = "hclk", "cclk";
+ interrupts = <GIC_SPI 145 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 146 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "int0", "int1";
+ bosch,mram-cfg = <0x0 128 64 64 64 64 32 32>;
+ };
+
+ main_mcan8: can@2781000 {
+ compatible = "bosch,m_can";
+ reg = <0x00 0x02781000 0x00 0x200>,
+ <0x00 0x02788000 0x00 0x8000>;
+ reg-names = "m_can", "message_ram";
+ power-domains = <&k3_pds 166 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 166 0>, <&k3_clks 166 1>;
+ clock-names = "hclk", "cclk";
+ interrupts = <GIC_SPI 576 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 577 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "int0", "int1";
+ bosch,mram-cfg = <0x0 128 64 64 64 64 32 32>;
+ };
+
+ main_mcan9: can@2791000 {
+ compatible = "bosch,m_can";
+ reg = <0x00 0x02791000 0x00 0x200>,
+ <0x00 0x02798000 0x00 0x8000>;
+ reg-names = "m_can", "message_ram";
+ power-domains = <&k3_pds 167 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 167 0>, <&k3_clks 167 1>;
+ clock-names = "hclk", "cclk";
+ interrupts = <GIC_SPI 579 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 580 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "int0", "int1";
+ bosch,mram-cfg = <0x0 128 64 64 64 64 32 32>;
+ };
+
+ main_mcan10: can@27a1000 {
+ compatible = "bosch,m_can";
+ reg = <0x00 0x027a1000 0x00 0x200>,
+ <0x00 0x027a8000 0x00 0x8000>;
+ reg-names = "m_can", "message_ram";
+ power-domains = <&k3_pds 168 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 168 0>, <&k3_clks 168 1>;
+ clock-names = "hclk", "cclk";
+ interrupts = <GIC_SPI 582 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 583 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "int0", "int1";
+ bosch,mram-cfg = <0x0 128 64 64 64 64 32 32>;
+ };
+
+ main_mcan11: can@27b1000 {
+ compatible = "bosch,m_can";
+ reg = <0x00 0x027b1000 0x00 0x200>,
+ <0x00 0x027b8000 0x00 0x8000>;
+ reg-names = "m_can", "message_ram";
+ power-domains = <&k3_pds 169 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 169 0>, <&k3_clks 169 1>;
+ clock-names = "hclk", "cclk";
+ interrupts = <GIC_SPI 585 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 586 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "int0", "int1";
+ bosch,mram-cfg = <0x0 128 64 64 64 64 32 32>;
+ };
+
+ main_mcan12: can@27c1000 {
+ compatible = "bosch,m_can";
+ reg = <0x00 0x027c1000 0x00 0x200>,
+ <0x00 0x027c8000 0x00 0x8000>;
+ reg-names = "m_can", "message_ram";
+ power-domains = <&k3_pds 170 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 170 0>, <&k3_clks 170 1>;
+ clock-names = "hclk", "cclk";
+ interrupts = <GIC_SPI 588 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 589 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "int0", "int1";
+ bosch,mram-cfg = <0x0 128 64 64 64 64 32 32>;
+ };
+
+ main_mcan13: can@27d1000 {
+ compatible = "bosch,m_can";
+ reg = <0x00 0x027d1000 0x00 0x200>,
+ <0x00 0x027d8000 0x00 0x8000>;
+ reg-names = "m_can", "message_ram";
+ power-domains = <&k3_pds 171 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 171 0>, <&k3_clks 171 1>;
+ clock-names = "hclk", "cclk";
+ interrupts = <GIC_SPI 591 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 592 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "int0", "int1";
+ bosch,mram-cfg = <0x0 128 64 64 64 64 32 32>;
+ };
};
diff --git a/arch/arm64/boot/dts/ti/k3-j721e-mcu-wakeup.dtsi b/arch/arm64/boot/dts/ti/k3-j721e-mcu-wakeup.dtsi
index d2dceda..b4972dfb 100644
--- a/arch/arm64/boot/dts/ti/k3-j721e-mcu-wakeup.dtsi
+++ b/arch/arm64/boot/dts/ti/k3-j721e-mcu-wakeup.dtsi
@@ -390,4 +390,32 @@ mcu_r5fss0_core1: r5f@41400000 {
ti,loczrama = <1>;
};
};
+
+ mcu_mcan0: can@40528000 {
+ compatible = "bosch,m_can";
+ reg = <0x00 0x40528000 0x00 0x200>,
+ <0x00 0x40500000 0x00 0x8000>;
+ reg-names = "m_can", "message_ram";
+ power-domains = <&k3_pds 172 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 172 0>, <&k3_clks 172 1>;
+ clock-names = "hclk", "cclk";
+ interrupts = <GIC_SPI 832 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 833 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "int0", "int1";
+ bosch,mram-cfg = <0x0 128 64 64 64 64 32 32>;
+ };
+
+ mcu_mcan1: can@40568000 {
+ compatible = "bosch,m_can";
+ reg = <0x00 0x40568000 0x00 0x200>,
+ <0x00 0x40540000 0x00 0x8000>;
+ reg-names = "m_can", "message_ram";
+ power-domains = <&k3_pds 173 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 173 0>, <&k3_clks 173 1>;
+ clock-names = "hclk", "cclk";
+ interrupts = <GIC_SPI 835 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 836 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "int0", "int1";
+ bosch,mram-cfg = <0x0 128 64 64 64 64 32 32>;
+ };
};
diff --git a/arch/arm64/boot/dts/ti/k3-j721e.dtsi b/arch/arm64/boot/dts/ti/k3-j721e.dtsi
index 214359e..4a3872f 100644
--- a/arch/arm64/boot/dts/ti/k3-j721e.dtsi
+++ b/arch/arm64/boot/dts/ti/k3-j721e.dtsi
@@ -64,7 +64,7 @@ cpu0: cpu@0 {
i-cache-sets = <256>;
d-cache-size = <0x8000>;
d-cache-line-size = <64>;
- d-cache-sets = <128>;
+ d-cache-sets = <256>;
next-level-cache = <&L2_0>;
};
@@ -78,7 +78,7 @@ cpu1: cpu@1 {
i-cache-sets = <256>;
d-cache-size = <0x8000>;
d-cache-line-size = <64>;
- d-cache-sets = <128>;
+ d-cache-sets = <256>;
next-level-cache = <&L2_0>;
};
};
@@ -88,7 +88,7 @@ L2_0: l2-cache0 {
cache-level = <2>;
cache-size = <0x100000>;
cache-line-size = <64>;
- cache-sets = <2048>;
+ cache-sets = <1024>;
next-level-cache = <&msmc_l3>;
};
diff --git a/arch/arm64/boot/dts/ti/k3-j721s2-common-proc-board.dts b/arch/arm64/boot/dts/ti/k3-j721s2-common-proc-board.dts
new file mode 100644
index 0000000..a5a24f9
--- /dev/null
+++ b/arch/arm64/boot/dts/ti/k3-j721s2-common-proc-board.dts
@@ -0,0 +1,421 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2021 Texas Instruments Incorporated - https://www.ti.com/
+ *
+ * Common Processor Board: https://www.ti.com/tool/J721EXCPXEVM
+ */
+
+/dts-v1/;
+
+#include "k3-j721s2-som-p0.dtsi"
+#include <dt-bindings/net/ti-dp83867.h>
+
+/ {
+ compatible = "ti,j721s2-evm", "ti,j721s2";
+ model = "Texas Instruments J721S2 EVM";
+
+ chosen {
+ stdout-path = "serial10:115200n8";
+ bootargs = "console=ttyS10,115200n8 earlycon=ns16550a,mmio32,2880000";
+ };
+
+ evm_12v0: fixedregulator-evm12v0 {
+ /* main supply */
+ compatible = "regulator-fixed";
+ regulator-name = "evm_12v0";
+ regulator-min-microvolt = <12000000>;
+ regulator-max-microvolt = <12000000>;
+ regulator-always-on;
+ regulator-boot-on;
+ };
+
+ vsys_3v3: fixedregulator-vsys3v3 {
+ /* Output of LM5140 */
+ compatible = "regulator-fixed";
+ regulator-name = "vsys_3v3";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ vin-supply = <&evm_12v0>;
+ regulator-always-on;
+ regulator-boot-on;
+ };
+
+ vsys_5v0: fixedregulator-vsys5v0 {
+ /* Output of LM5140 */
+ compatible = "regulator-fixed";
+ regulator-name = "vsys_5v0";
+ regulator-min-microvolt = <5000000>;
+ regulator-max-microvolt = <5000000>;
+ vin-supply = <&evm_12v0>;
+ regulator-always-on;
+ regulator-boot-on;
+ };
+
+ vdd_mmc1: fixedregulator-sd {
+ /* Output of TPS22918 */
+ compatible = "regulator-fixed";
+ regulator-name = "vdd_mmc1";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-boot-on;
+ enable-active-high;
+ vin-supply = <&vsys_3v3>;
+ gpio = <&exp2 2 GPIO_ACTIVE_HIGH>;
+ };
+
+ vdd_sd_dv: gpio-regulator-TLV71033 {
+ /* Output of TLV71033 */
+ compatible = "regulator-gpio";
+ regulator-name = "tlv71033";
+ pinctrl-names = "default";
+ pinctrl-0 = <&vdd_sd_dv_pins_default>;
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-boot-on;
+ vin-supply = <&vsys_5v0>;
+ gpios = <&main_gpio0 8 GPIO_ACTIVE_HIGH>;
+ states = <1800000 0x0>,
+ <3300000 0x1>;
+ };
+
+ transceiver1: can-phy1 {
+ compatible = "ti,tcan1043";
+ #phy-cells = <0>;
+ max-bitrate = <5000000>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&mcu_mcan0_gpio_pins_default>;
+ standby-gpios = <&wkup_gpio0 69 GPIO_ACTIVE_LOW>;
+ enable-gpios = <&wkup_gpio0 0 GPIO_ACTIVE_HIGH>;
+ };
+
+ transceiver2: can-phy2 {
+ compatible = "ti,tcan1042";
+ #phy-cells = <0>;
+ max-bitrate = <5000000>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&mcu_mcan1_gpio_pins_default>;
+ standby-gpios = <&wkup_gpio0 2 GPIO_ACTIVE_HIGH>;
+ };
+
+};
+
+&main_pmx0 {
+ main_uart8_pins_default: main-uart8-pins-default {
+ pinctrl-single,pins = <
+ J721S2_IOPAD(0x040, PIN_INPUT, 14) /* (AC28) MCASP0_AXR0.UART8_CTSn */
+ J721S2_IOPAD(0x044, PIN_OUTPUT, 14) /* (Y26) MCASP0_AXR1.UART8_RTSn */
+ J721S2_IOPAD(0x0d0, PIN_INPUT, 11) /* (AF26) SPI0_CS1.UART8_RXD */
+ J721S2_IOPAD(0x0d4, PIN_OUTPUT, 11) /* (AH27) SPI0_CLK.UART8_TXD */
+ >;
+ };
+
+ main_i2c3_pins_default: main-i2c3-pins-default {
+ pinctrl-single,pins = <
+ J721S2_IOPAD(0x064, PIN_INPUT_PULLUP, 13) /* (W28) MCAN0_TX.I2C3_SCL */
+ J721S2_IOPAD(0x060, PIN_INPUT_PULLUP, 13) /* (AC27) MCASP2_AXR1.I2C3_SDA */
+ >;
+ };
+
+ main_mmc1_pins_default: main-mmc1-pins-default {
+ pinctrl-single,pins = <
+ J721S2_IOPAD(0x104, PIN_INPUT, 0) /* (P23) MMC1_CLK */
+ J721S2_IOPAD(0x108, PIN_INPUT, 0) /* (N24) MMC1_CMD */
+ J721S2_IOPAD(0x100, PIN_INPUT, 0) /* (###) MMC1_CLKLB */
+ J721S2_IOPAD(0x0fc, PIN_INPUT, 0) /* (M23) MMC1_DAT0 */
+ J721S2_IOPAD(0x0f8, PIN_INPUT, 0) /* (P24) MMC1_DAT1 */
+ J721S2_IOPAD(0x0f4, PIN_INPUT, 0) /* (R24) MMC1_DAT2 */
+ J721S2_IOPAD(0x0f0, PIN_INPUT, 0) /* (R22) MMC1_DAT3 */
+ J721S2_IOPAD(0x0e8, PIN_INPUT, 8) /* (AE25) TIMER_IO0.MMC1_SDCD */
+ >;
+ };
+
+ vdd_sd_dv_pins_default: vdd-sd-dv-pins-default {
+ pinctrl-single,pins = <
+ J721S2_IOPAD(0x020, PIN_INPUT, 7) /* (AA23) MCAN15_RX.GPIO0_8 */
+ >;
+ };
+};
+
+&wkup_pmx0 {
+ mcu_cpsw_pins_default: mcu-cpsw-pins-default {
+ pinctrl-single,pins = <
+ J721S2_WKUP_IOPAD(0x094, PIN_INPUT, 0) /* (B22) MCU_RGMII1_RD0 */
+ J721S2_WKUP_IOPAD(0x090, PIN_INPUT, 0) /* (B21) MCU_RGMII1_RD1 */
+ J721S2_WKUP_IOPAD(0x08c, PIN_INPUT, 0) /* (C22) MCU_RGMII1_RD2 */
+ J721S2_WKUP_IOPAD(0x088, PIN_INPUT, 0) /* (D23) MCU_RGMII1_RD3 */
+ J721S2_WKUP_IOPAD(0x084, PIN_INPUT, 0) /* (D22) MCU_RGMII1_RXC */
+ J721S2_WKUP_IOPAD(0x06c, PIN_INPUT, 0) /* (E23) MCU_RGMII1_RX_CTL */
+ J721S2_WKUP_IOPAD(0x07c, PIN_OUTPUT, 0) /* (F23) MCU_RGMII1_TD0 */
+ J721S2_WKUP_IOPAD(0x078, PIN_OUTPUT, 0) /* (G22) MCU_RGMII1_TD1 */
+ J721S2_WKUP_IOPAD(0x074, PIN_OUTPUT, 0) /* (E21) MCU_RGMII1_TD2 */
+ J721S2_WKUP_IOPAD(0x070, PIN_OUTPUT, 0) /* (E22) MCU_RGMII1_TD3 */
+ J721S2_WKUP_IOPAD(0x080, PIN_OUTPUT, 0) /* (F21) MCU_RGMII1_TXC */
+ J721S2_WKUP_IOPAD(0x068, PIN_OUTPUT, 0) /* (F22) MCU_RGMII1_TX_CTL */
+ >;
+ };
+
+ mcu_mdio_pins_default: mcu-mdio-pins-default {
+ pinctrl-single,pins = <
+ J721S2_WKUP_IOPAD(0x09c, PIN_OUTPUT, 0) /* (A21) MCU_MDIO0_MDC */
+ J721S2_WKUP_IOPAD(0x098, PIN_INPUT, 0) /* (A22) MCU_MDIO0_MDIO */
+ >;
+ };
+
+ mcu_mcan0_pins_default: mcu-mcan0-pins-default {
+ pinctrl-single,pins = <
+ J721S2_WKUP_IOPAD(0x0bc, PIN_INPUT, 0) /* (E28) MCU_MCAN0_RX */
+ J721S2_WKUP_IOPAD(0x0b8, PIN_OUTPUT, 0) /* (E27) MCU_MCAN0_TX */
+ >;
+ };
+
+ mcu_mcan1_pins_default: mcu-mcan1-pins-default {
+ pinctrl-single,pins = <
+ J721S2_WKUP_IOPAD(0x0d4, PIN_INPUT, 0) /* (F26) WKUP_GPIO0_5.MCU_MCAN1_RX */
+ J721S2_WKUP_IOPAD(0x0d0, PIN_OUTPUT, 0) /* (C23) WKUP_GPIO0_4.MCU_MCAN1_TX */
+ >;
+ };
+
+ mcu_mcan0_gpio_pins_default: mcu-mcan0-gpio-pins-default {
+ pinctrl-single,pins = <
+ J721S2_WKUP_IOPAD(0x0c0, PIN_INPUT, 7) /* (D26) WKUP_GPIO0_0 */
+ J721S2_WKUP_IOPAD(0x0a8, PIN_INPUT, 7) /* (B25) MCU_SPI0_D1.WKUP_GPIO0_69 */
+ >;
+ };
+
+ mcu_mcan1_gpio_pins_default: mcu-mcan1-gpio-pins-default {
+ pinctrl-single,pins = <
+ J721S2_WKUP_IOPAD(0x0c8, PIN_INPUT, 7) /* (C28) WKUP_GPIO0_2 */
+ >;
+ };
+};
+
+&main_gpio2 {
+ status = "disabled";
+};
+
+&main_gpio4 {
+ status = "disabled";
+};
+
+&main_gpio6 {
+ status = "disabled";
+};
+
+&wkup_gpio1 {
+ status = "disabled";
+};
+
+&wkup_uart0 {
+ status = "reserved";
+};
+
+&main_uart0 {
+ status = "disabled";
+};
+
+&main_uart1 {
+ status = "disabled";
+};
+
+&main_uart2 {
+ status = "disabled";
+};
+
+&main_uart3 {
+ status = "disabled";
+};
+
+&main_uart4 {
+ status = "disabled";
+};
+
+&main_uart5 {
+ status = "disabled";
+};
+
+&main_uart6 {
+ status = "disabled";
+};
+
+&main_uart7 {
+ status = "disabled";
+};
+
+&main_uart8 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&main_uart8_pins_default>;
+ /* Shared with TFA on this platform */
+ power-domains = <&k3_pds 357 TI_SCI_PD_SHARED>;
+};
+
+&main_uart9 {
+ status = "disabled";
+};
+
+&main_i2c0 {
+ clock-frequency = <400000>;
+
+ exp1: gpio@20 {
+ compatible = "ti,tca6416";
+ reg = <0x20>;
+ gpio-controller;
+ #gpio-cells = <2>;
+ gpio-line-names = "PCIE_2L_MODE_SEL", "PCIE_2L_PERSTZ", "PCIE_2L_RC_RSTZ",
+ "PCIE_2L_EP_RST_EN", "PCIE_1L_MODE_SEL", "PCIE_1L_PERSTZ",
+ "PCIE_1L_RC_RSTZ", "PCIE_1L_EP_RST_EN", "PCIE_2L_PRSNT#",
+ "PCIE_1L_PRSNT#", "CDCI1_OE1/OE4", "CDCI1_OE2/OE3", "EXP_MUX1",
+ "EXP_MUX2", "EXP_MUX3", "GESI_EXP_PHY_RSTz";
+ };
+
+ exp2: gpio@22 {
+ compatible = "ti,tca6424";
+ reg = <0x22>;
+ gpio-controller;
+ #gpio-cells = <2>;
+ gpio-line-names = "APPLE_AUTH_RSTZ", "MLB_RSTZ", "GPIO_USD_PWR_EN", "USBC_PWR_EN",
+ "USBC_MODE_SEL1", "USBC_MODE_SEL0", "MCAN0_EN", "MCAN0_STB#",
+ "MUX_SPAREMUX_SPARE", "MCASP/TRACE_MUX_S0", "MCASP/TRACE_MUX_S1",
+ "MLB_MUX_SEL", "MCAN_MUX_SEL", "MCASP2/SPI3_MUX_SEL", "PCIe_CLKREQn_MUX_SEL",
+ "CDCI2_RSTZ", "ENET_EXP_PWRDN", "ENET_EXP_RESETZ", "ENET_I2CMUX_SEL",
+ "ENET_EXP_SPARE2", "M2PCIE_RTSZ", "USER_INPUT1", "USER_LED1", "USER_LED2";
+ };
+};
+
+&main_i2c1 {
+ status = "disabled";
+};
+
+&main_i2c2 {
+ status = "disabled";
+};
+
+&main_i2c3 {
+ status = "disabled";
+};
+
+&main_i2c4 {
+ status = "disabled";
+};
+
+&main_i2c5 {
+ status = "disabled";
+};
+
+&main_i2c6 {
+ status = "disabled";
+};
+
+&main_sdhci0 {
+ /* eMMC */
+ non-removable;
+ ti,driver-strength-ohm = <50>;
+ disable-wp;
+};
+
+&main_sdhci1 {
+ /* SD card */
+ pinctrl-0 = <&main_mmc1_pins_default>;
+ pinctrl-names = "default";
+ disable-wp;
+ vmmc-supply = <&vdd_mmc1>;
+ vqmmc-supply = <&vdd_sd_dv>;
+};
+
+&mcu_cpsw {
+ pinctrl-names = "default";
+ pinctrl-0 = <&mcu_cpsw_pins_default &mcu_mdio_pins_default>;
+};
+
+&davinci_mdio {
+ phy0: ethernet-phy@0 {
+ reg = <0>;
+ ti,rx-internal-delay = <DP83867_RGMIIDCTL_2_00_NS>;
+ ti,fifo-depth = <DP83867_PHYCR_FIFO_DEPTH_4_B_NIB>;
+ ti,min-output-impedance;
+ };
+};
+
+&cpsw_port1 {
+ phy-mode = "rgmii-rxid";
+ phy-handle = <&phy0>;
+};
+
+&mcu_mcan0 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&mcu_mcan0_pins_default>;
+ phys = <&transceiver1>;
+};
+
+&mcu_mcan1 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&mcu_mcan1_pins_default>;
+ phys = <&transceiver2>;
+};
+
+&main_mcan0 {
+ status = "disabled";
+};
+
+&main_mcan1 {
+ status = "disabled";
+};
+
+&main_mcan2 {
+ status = "disabled";
+};
+
+&main_mcan3 {
+ status = "disabled";
+};
+
+&main_mcan4 {
+ status = "disabled";
+};
+
+&main_mcan5 {
+ status = "disabled";
+};
+
+&main_mcan6 {
+ status = "disabled";
+};
+
+&main_mcan7 {
+ status = "disabled";
+};
+
+&main_mcan8 {
+ status = "disabled";
+};
+
+&main_mcan9 {
+ status = "disabled";
+};
+
+&main_mcan10 {
+ status = "disabled";
+};
+
+&main_mcan11 {
+ status = "disabled";
+};
+
+&main_mcan12 {
+ status = "disabled";
+};
+
+&main_mcan13 {
+ status = "disabled";
+};
+
+&main_mcan14 {
+ status = "disabled";
+};
+
+&main_mcan15 {
+ status = "disabled";
+};
+
+&main_mcan17 {
+ status = "disabled";
+};
diff --git a/arch/arm64/boot/dts/ti/k3-j721s2-main.dtsi b/arch/arm64/boot/dts/ti/k3-j721s2-main.dtsi
new file mode 100644
index 0000000..b04db1d
--- /dev/null
+++ b/arch/arm64/boot/dts/ti/k3-j721s2-main.dtsi
@@ -0,0 +1,937 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Device Tree Source for J721S2 SoC Family Main Domain peripherals
+ *
+ * Copyright (C) 2021 Texas Instruments Incorporated - https://www.ti.com/
+ */
+
+&cbass_main {
+ msmc_ram: sram@70000000 {
+ compatible = "mmio-sram";
+ reg = <0x0 0x70000000 0x0 0x400000>;
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges = <0x0 0x0 0x70000000 0x400000>;
+
+ atf-sram@0 {
+ reg = <0x0 0x20000>;
+ };
+
+ tifs-sram@1f0000 {
+ reg = <0x1f0000 0x10000>;
+ };
+
+ l3cache-sram@200000 {
+ reg = <0x200000 0x200000>;
+ };
+ };
+
+ gic500: interrupt-controller@1800000 {
+ compatible = "arm,gic-v3";
+ #address-cells = <2>;
+ #size-cells = <2>;
+ ranges;
+ #interrupt-cells = <3>;
+ interrupt-controller;
+ reg = <0x00 0x01800000 0x00 0x200000>, /* GICD */
+ <0x00 0x01900000 0x00 0x100000>; /* GICR */
+
+ /* vcpumntirq: virtual CPU interface maintenance interrupt */
+ interrupts = <GIC_PPI 9 IRQ_TYPE_LEVEL_HIGH>;
+
+ gic_its: msi-controller@1820000 {
+ compatible = "arm,gic-v3-its";
+ reg = <0x00 0x01820000 0x00 0x10000>;
+ socionext,synquacer-pre-its = <0x1000000 0x400000>;
+ msi-controller;
+ #msi-cells = <1>;
+ };
+ };
+
+ main_gpio_intr: interrupt-controller@a00000 {
+ compatible = "ti,sci-intr";
+ reg = <0x00 0x00a00000 0x00 0x800>;
+ ti,intr-trigger-type = <1>;
+ interrupt-controller;
+ interrupt-parent = <&gic500>;
+ #interrupt-cells = <1>;
+ ti,sci = <&sms>;
+ ti,sci-dev-id = <148>;
+ ti,interrupt-ranges = <8 360 56>;
+ };
+
+ main_pmx0: pinctrl@11c000 {
+ compatible = "pinctrl-single";
+ /* Proxy 0 addressing */
+ reg = <0x0 0x11c000 0x0 0x120>;
+ #pinctrl-cells = <1>;
+ pinctrl-single,register-width = <32>;
+ pinctrl-single,function-mask = <0xffffffff>;
+ };
+
+ main_uart0: serial@2800000 {
+ compatible = "ti,j721e-uart", "ti,am654-uart";
+ reg = <0x00 0x02800000 0x00 0x200>;
+ interrupts = <GIC_SPI 192 IRQ_TYPE_LEVEL_HIGH>;
+ current-speed = <115200>;
+ clocks = <&k3_clks 146 3>;
+ clock-names = "fclk";
+ power-domains = <&k3_pds 146 TI_SCI_PD_EXCLUSIVE>;
+ };
+
+ main_uart1: serial@2810000 {
+ compatible = "ti,j721e-uart", "ti,am654-uart";
+ reg = <0x00 0x02810000 0x00 0x200>;
+ interrupts = <GIC_SPI 193 IRQ_TYPE_LEVEL_HIGH>;
+ current-speed = <115200>;
+ clocks = <&k3_clks 350 3>;
+ clock-names = "fclk";
+ power-domains = <&k3_pds 350 TI_SCI_PD_EXCLUSIVE>;
+ };
+
+ main_uart2: serial@2820000 {
+ compatible = "ti,j721e-uart", "ti,am654-uart";
+ reg = <0x00 0x02820000 0x00 0x200>;
+ interrupts = <GIC_SPI 194 IRQ_TYPE_LEVEL_HIGH>;
+ current-speed = <115200>;
+ clocks = <&k3_clks 351 3>;
+ clock-names = "fclk";
+ power-domains = <&k3_pds 351 TI_SCI_PD_EXCLUSIVE>;
+ };
+
+ main_uart3: serial@2830000 {
+ compatible = "ti,j721e-uart", "ti,am654-uart";
+ reg = <0x00 0x02830000 0x00 0x200>;
+ interrupts = <GIC_SPI 195 IRQ_TYPE_LEVEL_HIGH>;
+ current-speed = <115200>;
+ clocks = <&k3_clks 352 3>;
+ clock-names = "fclk";
+ power-domains = <&k3_pds 352 TI_SCI_PD_EXCLUSIVE>;
+ };
+
+ main_uart4: serial@2840000 {
+ compatible = "ti,j721e-uart", "ti,am654-uart";
+ reg = <0x00 0x02840000 0x00 0x200>;
+ interrupts = <GIC_SPI 196 IRQ_TYPE_LEVEL_HIGH>;
+ current-speed = <115200>;
+ clocks = <&k3_clks 353 3>;
+ clock-names = "fclk";
+ power-domains = <&k3_pds 353 TI_SCI_PD_EXCLUSIVE>;
+ };
+
+ main_uart5: serial@2850000 {
+ compatible = "ti,j721e-uart", "ti,am654-uart";
+ reg = <0x00 0x02850000 0x00 0x200>;
+ interrupts = <GIC_SPI 197 IRQ_TYPE_LEVEL_HIGH>;
+ current-speed = <115200>;
+ clocks = <&k3_clks 354 3>;
+ clock-names = "fclk";
+ power-domains = <&k3_pds 354 TI_SCI_PD_EXCLUSIVE>;
+ };
+
+ main_uart6: serial@2860000 {
+ compatible = "ti,j721e-uart", "ti,am654-uart";
+ reg = <0x00 0x02860000 0x00 0x200>;
+ interrupts = <GIC_SPI 198 IRQ_TYPE_LEVEL_HIGH>;
+ current-speed = <115200>;
+ clocks = <&k3_clks 355 3>;
+ clock-names = "fclk";
+ power-domains = <&k3_pds 355 TI_SCI_PD_EXCLUSIVE>;
+ };
+
+ main_uart7: serial@2870000 {
+ compatible = "ti,j721e-uart", "ti,am654-uart";
+ reg = <0x00 0x02870000 0x00 0x200>;
+ interrupts = <GIC_SPI 199 IRQ_TYPE_LEVEL_HIGH>;
+ current-speed = <115200>;
+ clocks = <&k3_clks 356 3>;
+ clock-names = "fclk";
+ power-domains = <&k3_pds 356 TI_SCI_PD_EXCLUSIVE>;
+ };
+
+ main_uart8: serial@2880000 {
+ compatible = "ti,j721e-uart", "ti,am654-uart";
+ reg = <0x00 0x02880000 0x00 0x200>;
+ interrupts = <GIC_SPI 248 IRQ_TYPE_LEVEL_HIGH>;
+ current-speed = <115200>;
+ clocks = <&k3_clks 357 3>;
+ clock-names = "fclk";
+ power-domains = <&k3_pds 357 TI_SCI_PD_EXCLUSIVE>;
+ };
+
+ main_uart9: serial@2890000 {
+ compatible = "ti,j721e-uart", "ti,am654-uart";
+ reg = <0x00 0x02890000 0x00 0x200>;
+ interrupts = <GIC_SPI 249 IRQ_TYPE_LEVEL_HIGH>;
+ current-speed = <115200>;
+ clocks = <&k3_clks 358 3>;
+ clock-names = "fclk";
+ power-domains = <&k3_pds 358 TI_SCI_PD_EXCLUSIVE>;
+ };
+
+ main_gpio0: gpio@600000 {
+ compatible = "ti,j721e-gpio", "ti,keystone-gpio";
+ reg = <0x00 0x00600000 0x00 0x100>;
+ gpio-controller;
+ #gpio-cells = <2>;
+ interrupt-parent = <&main_gpio_intr>;
+ interrupts = <145>, <146>, <147>, <148>, <149>;
+ interrupt-controller;
+ #interrupt-cells = <2>;
+ ti,ngpio = <66>;
+ ti,davinci-gpio-unbanked = <0>;
+ power-domains = <&k3_pds 111 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 111 0>;
+ clock-names = "gpio";
+ };
+
+ main_gpio2: gpio@610000 {
+ compatible = "ti,j721e-gpio", "ti,keystone-gpio";
+ reg = <0x00 0x00610000 0x00 0x100>;
+ gpio-controller;
+ #gpio-cells = <2>;
+ interrupt-parent = <&main_gpio_intr>;
+ interrupts = <154>, <155>, <156>, <157>, <158>;
+ interrupt-controller;
+ #interrupt-cells = <2>;
+ ti,ngpio = <66>;
+ ti,davinci-gpio-unbanked = <0>;
+ power-domains = <&k3_pds 112 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 112 0>;
+ clock-names = "gpio";
+ };
+
+ main_gpio4: gpio@620000 {
+ compatible = "ti,j721e-gpio", "ti,keystone-gpio";
+ reg = <0x00 0x00620000 0x00 0x100>;
+ gpio-controller;
+ #gpio-cells = <2>;
+ interrupt-parent = <&main_gpio_intr>;
+ interrupts = <163>, <164>, <165>, <166>, <167>;
+ interrupt-controller;
+ #interrupt-cells = <2>;
+ ti,ngpio = <66>;
+ ti,davinci-gpio-unbanked = <0>;
+ power-domains = <&k3_pds 113 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 113 0>;
+ clock-names = "gpio";
+ };
+
+ main_gpio6: gpio@630000 {
+ compatible = "ti,j721e-gpio", "ti,keystone-gpio";
+ reg = <0x00 0x00630000 0x00 0x100>;
+ gpio-controller;
+ #gpio-cells = <2>;
+ interrupt-parent = <&main_gpio_intr>;
+ interrupts = <172>, <173>, <174>, <175>, <176>;
+ interrupt-controller;
+ #interrupt-cells = <2>;
+ ti,ngpio = <66>;
+ ti,davinci-gpio-unbanked = <0>;
+ power-domains = <&k3_pds 114 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 114 0>;
+ clock-names = "gpio";
+ };
+
+ main_i2c0: i2c@2000000 {
+ compatible = "ti,j721e-i2c", "ti,omap4-i2c";
+ reg = <0x00 0x02000000 0x00 0x100>;
+ interrupts = <GIC_SPI 200 IRQ_TYPE_LEVEL_HIGH>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ clocks = <&k3_clks 214 1>;
+ clock-names = "fck";
+ power-domains = <&k3_pds 214 TI_SCI_PD_EXCLUSIVE>;
+ };
+
+ main_i2c1: i2c@2010000 {
+ compatible = "ti,j721e-i2c", "ti,omap4-i2c";
+ reg = <0x00 0x02010000 0x00 0x100>;
+ interrupts = <GIC_SPI 201 IRQ_TYPE_LEVEL_HIGH>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ clocks = <&k3_clks 215 1>;
+ clock-names = "fck";
+ power-domains = <&k3_pds 215 TI_SCI_PD_EXCLUSIVE>;
+ };
+
+ main_i2c2: i2c@2020000 {
+ compatible = "ti,j721e-i2c", "ti,omap4-i2c";
+ reg = <0x00 0x02020000 0x00 0x100>;
+ interrupts = <GIC_SPI 202 IRQ_TYPE_LEVEL_HIGH>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ clocks = <&k3_clks 216 1>;
+ clock-names = "fck";
+ power-domains = <&k3_pds 216 TI_SCI_PD_EXCLUSIVE>;
+ };
+
+ main_i2c3: i2c@2030000 {
+ compatible = "ti,j721e-i2c", "ti,omap4-i2c";
+ reg = <0x00 0x02030000 0x00 0x100>;
+ interrupts = <GIC_SPI 203 IRQ_TYPE_LEVEL_HIGH>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ clocks = <&k3_clks 217 1>;
+ clock-names = "fck";
+ power-domains = <&k3_pds 217 TI_SCI_PD_EXCLUSIVE>;
+ };
+
+ main_i2c4: i2c@2040000 {
+ compatible = "ti,j721e-i2c", "ti,omap4-i2c";
+ reg = <0x00 0x02040000 0x00 0x100>;
+ interrupts = <GIC_SPI 204 IRQ_TYPE_LEVEL_HIGH>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ clocks = <&k3_clks 218 1>;
+ clock-names = "fck";
+ power-domains = <&k3_pds 218 TI_SCI_PD_EXCLUSIVE>;
+ };
+
+ main_i2c5: i2c@2050000 {
+ compatible = "ti,j721e-i2c", "ti,omap4-i2c";
+ reg = <0x00 0x02050000 0x00 0x100>;
+ interrupts = <GIC_SPI 205 IRQ_TYPE_LEVEL_HIGH>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ clocks = <&k3_clks 219 1>;
+ clock-names = "fck";
+ power-domains = <&k3_pds 219 TI_SCI_PD_EXCLUSIVE>;
+ };
+
+ main_i2c6: i2c@2060000 {
+ compatible = "ti,j721e-i2c", "ti,omap4-i2c";
+ reg = <0x00 0x02060000 0x00 0x100>;
+ interrupts = <GIC_SPI 206 IRQ_TYPE_LEVEL_HIGH>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ clocks = <&k3_clks 220 1>;
+ clock-names = "fck";
+ power-domains = <&k3_pds 220 TI_SCI_PD_EXCLUSIVE>;
+ };
+
+ main_sdhci0: mmc@4f80000 {
+ compatible = "ti,j721e-sdhci-8bit";
+ reg = <0x00 0x04f80000 0x00 0x1000>,
+ <0x00 0x04f88000 0x00 0x400>;
+ interrupts = <GIC_SPI 3 IRQ_TYPE_LEVEL_HIGH>;
+ power-domains = <&k3_pds 98 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 98 7>, <&k3_clks 98 1>;
+ clock-names = "clk_ahb", "clk_xin";
+ assigned-clocks = <&k3_clks 98 1>;
+ assigned-clock-parents = <&k3_clks 98 2>;
+ bus-width = <8>;
+ ti,otap-del-sel-legacy = <0x0>;
+ ti,otap-del-sel-mmc-hs = <0x0>;
+ ti,otap-del-sel-ddr52 = <0x6>;
+ ti,otap-del-sel-hs200 = <0x8>;
+ ti,otap-del-sel-hs400 = <0x5>;
+ ti,itap-del-sel-legacy = <0x10>;
+ ti,itap-del-sel-mmc-hs = <0xa>;
+ ti,strobe-sel = <0x77>;
+ ti,clkbuf-sel = <0x7>;
+ ti,trm-icp = <0x8>;
+ mmc-ddr-1_8v;
+ mmc-hs200-1_8v;
+ mmc-hs400-1_8v;
+ dma-coherent;
+ };
+
+ main_sdhci1: mmc@4fb0000 {
+ compatible = "ti,j721e-sdhci-4bit";
+ reg = <0x00 0x04fb0000 0x00 0x1000>,
+ <0x00 0x04fb8000 0x00 0x400>;
+ interrupts = <GIC_SPI 4 IRQ_TYPE_LEVEL_HIGH>;
+ power-domains = <&k3_pds 99 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 99 8>, <&k3_clks 99 1>;
+ clock-names = "clk_ahb", "clk_xin";
+ assigned-clocks = <&k3_clks 99 1>;
+ assigned-clock-parents = <&k3_clks 99 2>;
+ bus-width = <4>;
+ ti,otap-del-sel-legacy = <0x0>;
+ ti,otap-del-sel-sd-hs = <0x0>;
+ ti,otap-del-sel-sdr12 = <0xf>;
+ ti,otap-del-sel-sdr25 = <0xf>;
+ ti,otap-del-sel-sdr50 = <0xc>;
+ ti,otap-del-sel-sdr104 = <0x5>;
+ ti,otap-del-sel-ddr50 = <0xc>;
+ ti,itap-del-sel-legacy = <0x0>;
+ ti,itap-del-sel-sd-hs = <0x0>;
+ ti,itap-del-sel-sdr12 = <0x0>;
+ ti,itap-del-sel-sdr25 = <0x0>;
+ ti,clkbuf-sel = <0x7>;
+ ti,trm-icp = <0x8>;
+ dma-coherent;
+ /* Masking support for SDR104 capability */
+ sdhci-caps-mask = <0x00000003 0x00000000>;
+ };
+
+ main_navss: bus@30000000 {
+ compatible = "simple-mfd";
+ #address-cells = <2>;
+ #size-cells = <2>;
+ ranges = <0x00 0x30000000 0x00 0x30000000 0x00 0x0c400000>;
+ ti,sci-dev-id = <224>;
+ dma-coherent;
+ dma-ranges;
+
+ main_navss_intr: interrupt-controller@310e0000 {
+ compatible = "ti,sci-intr";
+ reg = <0x00 0x310e0000 0x00 0x4000>;
+ ti,intr-trigger-type = <4>;
+ interrupt-controller;
+ interrupt-parent = <&gic500>;
+ #interrupt-cells = <1>;
+ ti,sci = <&sms>;
+ ti,sci-dev-id = <227>;
+ ti,interrupt-ranges = <0 64 64>,
+ <64 448 64>,
+ <128 672 64>;
+ };
+
+ main_udmass_inta: msi-controller@33d00000 {
+ compatible = "ti,sci-inta";
+ reg = <0x00 0x33d00000 0x00 0x100000>;
+ interrupt-controller;
+ #interrupt-cells = <0>;
+ interrupt-parent = <&main_navss_intr>;
+ msi-controller;
+ ti,sci = <&sms>;
+ ti,sci-dev-id = <265>;
+ ti,interrupt-ranges = <0 0 256>;
+ };
+
+ secure_proxy_main: mailbox@32c00000 {
+ compatible = "ti,am654-secure-proxy";
+ #mbox-cells = <1>;
+ reg-names = "target_data", "rt", "scfg";
+ reg = <0x00 0x32c00000 0x00 0x100000>,
+ <0x00 0x32400000 0x00 0x100000>,
+ <0x00 0x32800000 0x00 0x100000>;
+ interrupt-names = "rx_011";
+ interrupts = <GIC_SPI 37 IRQ_TYPE_LEVEL_HIGH>;
+ };
+
+ hwspinlock: spinlock@30e00000 {
+ compatible = "ti,am654-hwspinlock";
+ reg = <0x00 0x30e00000 0x00 0x1000>;
+ #hwlock-cells = <1>;
+ };
+
+ mailbox0_cluster0: mailbox@31f80000 {
+ compatible = "ti,am654-mailbox";
+ reg = <0x00 0x31f80000 0x00 0x200>;
+ #mbox-cells = <1>;
+ ti,mbox-num-users = <4>;
+ ti,mbox-num-fifos = <16>;
+ interrupt-parent = <&main_navss_intr>;
+ };
+
+ mailbox0_cluster1: mailbox@31f81000 {
+ compatible = "ti,am654-mailbox";
+ reg = <0x00 0x31f81000 0x00 0x200>;
+ #mbox-cells = <1>;
+ ti,mbox-num-users = <4>;
+ ti,mbox-num-fifos = <16>;
+ interrupt-parent = <&main_navss_intr>;
+ };
+
+ mailbox0_cluster2: mailbox@31f82000 {
+ compatible = "ti,am654-mailbox";
+ reg = <0x00 0x31f82000 0x00 0x200>;
+ #mbox-cells = <1>;
+ ti,mbox-num-users = <4>;
+ ti,mbox-num-fifos = <16>;
+ interrupt-parent = <&main_navss_intr>;
+ };
+
+ mailbox0_cluster3: mailbox@31f83000 {
+ compatible = "ti,am654-mailbox";
+ reg = <0x00 0x31f83000 0x00 0x200>;
+ #mbox-cells = <1>;
+ ti,mbox-num-users = <4>;
+ ti,mbox-num-fifos = <16>;
+ interrupt-parent = <&main_navss_intr>;
+ };
+
+ mailbox0_cluster4: mailbox@31f84000 {
+ compatible = "ti,am654-mailbox";
+ reg = <0x00 0x31f84000 0x00 0x200>;
+ #mbox-cells = <1>;
+ ti,mbox-num-users = <4>;
+ ti,mbox-num-fifos = <16>;
+ interrupt-parent = <&main_navss_intr>;
+ };
+
+ mailbox0_cluster5: mailbox@31f85000 {
+ compatible = "ti,am654-mailbox";
+ reg = <0x00 0x31f85000 0x00 0x200>;
+ #mbox-cells = <1>;
+ ti,mbox-num-users = <4>;
+ ti,mbox-num-fifos = <16>;
+ interrupt-parent = <&main_navss_intr>;
+ };
+
+ mailbox0_cluster6: mailbox@31f86000 {
+ compatible = "ti,am654-mailbox";
+ reg = <0x00 0x31f86000 0x00 0x200>;
+ #mbox-cells = <1>;
+ ti,mbox-num-users = <4>;
+ ti,mbox-num-fifos = <16>;
+ interrupt-parent = <&main_navss_intr>;
+ };
+
+ mailbox0_cluster7: mailbox@31f87000 {
+ compatible = "ti,am654-mailbox";
+ reg = <0x00 0x31f87000 0x00 0x200>;
+ #mbox-cells = <1>;
+ ti,mbox-num-users = <4>;
+ ti,mbox-num-fifos = <16>;
+ interrupt-parent = <&main_navss_intr>;
+ };
+
+ mailbox0_cluster8: mailbox@31f88000 {
+ compatible = "ti,am654-mailbox";
+ reg = <0x00 0x31f88000 0x00 0x200>;
+ #mbox-cells = <1>;
+ ti,mbox-num-users = <4>;
+ ti,mbox-num-fifos = <16>;
+ interrupt-parent = <&main_navss_intr>;
+ };
+
+ mailbox0_cluster9: mailbox@31f89000 {
+ compatible = "ti,am654-mailbox";
+ reg = <0x00 0x31f89000 0x00 0x200>;
+ #mbox-cells = <1>;
+ ti,mbox-num-users = <4>;
+ ti,mbox-num-fifos = <16>;
+ interrupt-parent = <&main_navss_intr>;
+ };
+
+ mailbox0_cluster10: mailbox@31f8a000 {
+ compatible = "ti,am654-mailbox";
+ reg = <0x00 0x31f8a000 0x00 0x200>;
+ #mbox-cells = <1>;
+ ti,mbox-num-users = <4>;
+ ti,mbox-num-fifos = <16>;
+ interrupt-parent = <&main_navss_intr>;
+ };
+
+ mailbox0_cluster11: mailbox@31f8b000 {
+ compatible = "ti,am654-mailbox";
+ reg = <0x00 0x31f8b000 0x00 0x200>;
+ #mbox-cells = <1>;
+ ti,mbox-num-users = <4>;
+ ti,mbox-num-fifos = <16>;
+ interrupt-parent = <&main_navss_intr>;
+ };
+
+ mailbox1_cluster0: mailbox@31f90000 {
+ compatible = "ti,am654-mailbox";
+ reg = <0x00 0x31f90000 0x00 0x200>;
+ #mbox-cells = <1>;
+ ti,mbox-num-users = <4>;
+ ti,mbox-num-fifos = <16>;
+ interrupt-parent = <&main_navss_intr>;
+ };
+
+ mailbox1_cluster1: mailbox@31f91000 {
+ compatible = "ti,am654-mailbox";
+ reg = <0x00 0x31f91000 0x00 0x200>;
+ #mbox-cells = <1>;
+ ti,mbox-num-users = <4>;
+ ti,mbox-num-fifos = <16>;
+ interrupt-parent = <&main_navss_intr>;
+ };
+
+ mailbox1_cluster2: mailbox@31f92000 {
+ compatible = "ti,am654-mailbox";
+ reg = <0x00 0x31f92000 0x00 0x200>;
+ #mbox-cells = <1>;
+ ti,mbox-num-users = <4>;
+ ti,mbox-num-fifos = <16>;
+ interrupt-parent = <&main_navss_intr>;
+ };
+
+ mailbox1_cluster3: mailbox@31f93000 {
+ compatible = "ti,am654-mailbox";
+ reg = <0x00 0x31f93000 0x00 0x200>;
+ #mbox-cells = <1>;
+ ti,mbox-num-users = <4>;
+ ti,mbox-num-fifos = <16>;
+ interrupt-parent = <&main_navss_intr>;
+ };
+
+ mailbox1_cluster4: mailbox@31f94000 {
+ compatible = "ti,am654-mailbox";
+ reg = <0x00 0x31f94000 0x00 0x200>;
+ #mbox-cells = <1>;
+ ti,mbox-num-users = <4>;
+ ti,mbox-num-fifos = <16>;
+ interrupt-parent = <&main_navss_intr>;
+ };
+
+ mailbox1_cluster5: mailbox@31f95000 {
+ compatible = "ti,am654-mailbox";
+ reg = <0x00 0x31f95000 0x00 0x200>;
+ #mbox-cells = <1>;
+ ti,mbox-num-users = <4>;
+ ti,mbox-num-fifos = <16>;
+ interrupt-parent = <&main_navss_intr>;
+ };
+
+ mailbox1_cluster6: mailbox@31f96000 {
+ compatible = "ti,am654-mailbox";
+ reg = <0x00 0x31f96000 0x00 0x200>;
+ #mbox-cells = <1>;
+ ti,mbox-num-users = <4>;
+ ti,mbox-num-fifos = <16>;
+ interrupt-parent = <&main_navss_intr>;
+ };
+
+ mailbox1_cluster7: mailbox@31f97000 {
+ compatible = "ti,am654-mailbox";
+ reg = <0x00 0x31f97000 0x00 0x200>;
+ #mbox-cells = <1>;
+ ti,mbox-num-users = <4>;
+ ti,mbox-num-fifos = <16>;
+ interrupt-parent = <&main_navss_intr>;
+ };
+
+ mailbox1_cluster8: mailbox@31f98000 {
+ compatible = "ti,am654-mailbox";
+ reg = <0x00 0x31f98000 0x00 0x200>;
+ #mbox-cells = <1>;
+ ti,mbox-num-users = <4>;
+ ti,mbox-num-fifos = <16>;
+ interrupt-parent = <&main_navss_intr>;
+ };
+
+ mailbox1_cluster9: mailbox@31f99000 {
+ compatible = "ti,am654-mailbox";
+ reg = <0x00 0x31f99000 0x00 0x200>;
+ #mbox-cells = <1>;
+ ti,mbox-num-users = <4>;
+ ti,mbox-num-fifos = <16>;
+ interrupt-parent = <&main_navss_intr>;
+ };
+
+ mailbox1_cluster10: mailbox@31f9a000 {
+ compatible = "ti,am654-mailbox";
+ reg = <0x00 0x31f9a000 0x00 0x200>;
+ #mbox-cells = <1>;
+ ti,mbox-num-users = <4>;
+ ti,mbox-num-fifos = <16>;
+ interrupt-parent = <&main_navss_intr>;
+ };
+
+ mailbox1_cluster11: mailbox@31f9b000 {
+ compatible = "ti,am654-mailbox";
+ reg = <0x00 0x31f9b000 0x00 0x200>;
+ #mbox-cells = <1>;
+ ti,mbox-num-users = <4>;
+ ti,mbox-num-fifos = <16>;
+ interrupt-parent = <&main_navss_intr>;
+ };
+
+ main_ringacc: ringacc@3c000000 {
+ compatible = "ti,am654-navss-ringacc";
+ reg = <0x0 0x3c000000 0x0 0x400000>,
+ <0x0 0x38000000 0x0 0x400000>,
+ <0x0 0x31120000 0x0 0x100>,
+ <0x0 0x33000000 0x0 0x40000>;
+ reg-names = "rt", "fifos", "proxy_gcfg", "proxy_target";
+ ti,num-rings = <1024>;
+ ti,sci-rm-range-gp-rings = <0x1>;
+ ti,sci = <&sms>;
+ ti,sci-dev-id = <259>;
+ msi-parent = <&main_udmass_inta>;
+ };
+
+ main_udmap: dma-controller@31150000 {
+ compatible = "ti,j721e-navss-main-udmap";
+ reg = <0x0 0x31150000 0x0 0x100>,
+ <0x0 0x34000000 0x0 0x80000>,
+ <0x0 0x35000000 0x0 0x200000>;
+ reg-names = "gcfg", "rchanrt", "tchanrt";
+ msi-parent = <&main_udmass_inta>;
+ #dma-cells = <1>;
+
+ ti,sci = <&sms>;
+ ti,sci-dev-id = <263>;
+ ti,ringacc = <&main_ringacc>;
+
+ ti,sci-rm-range-tchan = <0x0d>, /* TX_CHAN */
+ <0x0f>, /* TX_HCHAN */
+ <0x10>; /* TX_UHCHAN */
+ ti,sci-rm-range-rchan = <0x0a>, /* RX_CHAN */
+ <0x0b>, /* RX_HCHAN */
+ <0x0c>; /* RX_UHCHAN */
+ ti,sci-rm-range-rflow = <0x00>; /* GP RFLOW */
+ };
+
+ cpts@310d0000 {
+ compatible = "ti,j721e-cpts";
+ reg = <0x0 0x310d0000 0x0 0x400>;
+ reg-names = "cpts";
+ clocks = <&k3_clks 226 5>;
+ clock-names = "cpts";
+ interrupts-extended = <&main_navss_intr 391>;
+ interrupt-names = "cpts";
+ ti,cpts-periodic-outputs = <6>;
+ ti,cpts-ext-ts-inputs = <8>;
+ };
+ };
+
+ main_mcan0: can@2701000 {
+ compatible = "bosch,m_can";
+ reg = <0x00 0x02701000 0x00 0x200>,
+ <0x00 0x02708000 0x00 0x8000>;
+ reg-names = "m_can", "message_ram";
+ power-domains = <&k3_pds 182 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 182 0>, <&k3_clks 182 1>;
+ clock-names = "hclk", "cclk";
+ interrupts = <GIC_SPI 124 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 125 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "int0", "int1";
+ bosch,mram-cfg = <0x0 128 64 64 64 64 32 32>;
+ };
+
+ main_mcan1: can@2711000 {
+ compatible = "bosch,m_can";
+ reg = <0x00 0x02711000 0x00 0x200>,
+ <0x00 0x02718000 0x00 0x8000>;
+ reg-names = "m_can", "message_ram";
+ power-domains = <&k3_pds 183 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 183 0>, <&k3_clks 183 1>;
+ clock-names = "hclk", "cclk";
+ interrupts = <GIC_SPI 127 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 128 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "int0", "int1";
+ bosch,mram-cfg = <0x0 128 64 64 64 64 32 32>;
+ };
+
+ main_mcan2: can@2721000 {
+ compatible = "bosch,m_can";
+ reg = <0x00 0x02721000 0x00 0x200>,
+ <0x00 0x02728000 0x00 0x8000>;
+ reg-names = "m_can", "message_ram";
+ power-domains = <&k3_pds 184 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 184 0>, <&k3_clks 184 1>;
+ clock-names = "hclk", "cclk";
+ interrupts = <GIC_SPI 130 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 131 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "int0", "int1";
+ bosch,mram-cfg = <0x0 128 64 64 64 64 32 32>;
+ };
+
+ main_mcan3: can@2731000 {
+ compatible = "bosch,m_can";
+ reg = <0x00 0x02731000 0x00 0x200>,
+ <0x00 0x02738000 0x00 0x8000>;
+ reg-names = "m_can", "message_ram";
+ power-domains = <&k3_pds 185 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 185 0>, <&k3_clks 185 1>;
+ clock-names = "hclk", "cclk";
+ interrupts = <GIC_SPI 133 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 134 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "int0", "int1";
+ bosch,mram-cfg = <0x0 128 64 64 64 64 32 32>;
+ };
+
+ main_mcan4: can@2741000 {
+ compatible = "bosch,m_can";
+ reg = <0x00 0x02741000 0x00 0x200>,
+ <0x00 0x02748000 0x00 0x8000>;
+ reg-names = "m_can", "message_ram";
+ power-domains = <&k3_pds 186 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 186 0>, <&k3_clks 186 1>;
+ clock-names = "hclk", "cclk";
+ interrupts = <GIC_SPI 136 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 137 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "int0", "int1";
+ bosch,mram-cfg = <0x0 128 64 64 64 64 32 32>;
+ };
+
+ main_mcan5: can@2751000 {
+ compatible = "bosch,m_can";
+ reg = <0x00 0x02751000 0x00 0x200>,
+ <0x00 0x02758000 0x00 0x8000>;
+ reg-names = "m_can", "message_ram";
+ power-domains = <&k3_pds 187 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 187 0>, <&k3_clks 187 1>;
+ clock-names = "hclk", "cclk";
+ interrupts = <GIC_SPI 139 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 140 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "int0", "int1";
+ bosch,mram-cfg = <0x0 128 64 64 64 64 32 32>;
+ };
+
+ main_mcan6: can@2761000 {
+ compatible = "bosch,m_can";
+ reg = <0x00 0x02761000 0x00 0x200>,
+ <0x00 0x02768000 0x00 0x8000>;
+ reg-names = "m_can", "message_ram";
+ power-domains = <&k3_pds 188 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 188 0>, <&k3_clks 188 1>;
+ clock-names = "hclk", "cclk";
+ interrupts = <GIC_SPI 142 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 143 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "int0", "int1";
+ bosch,mram-cfg = <0x0 128 64 64 64 64 32 32>;
+ };
+
+ main_mcan7: can@2771000 {
+ compatible = "bosch,m_can";
+ reg = <0x00 0x02771000 0x00 0x200>,
+ <0x00 0x02778000 0x00 0x8000>;
+ reg-names = "m_can", "message_ram";
+ power-domains = <&k3_pds 189 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 189 0>, <&k3_clks 189 1>;
+ clock-names = "hclk", "cclk";
+ interrupts = <GIC_SPI 145 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 146 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "int0", "int1";
+ bosch,mram-cfg = <0x0 128 64 64 64 64 32 32>;
+ };
+
+ main_mcan8: can@2781000 {
+ compatible = "bosch,m_can";
+ reg = <0x00 0x02781000 0x00 0x200>,
+ <0x00 0x02788000 0x00 0x8000>;
+ reg-names = "m_can", "message_ram";
+ power-domains = <&k3_pds 190 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 190 0>, <&k3_clks 190 1>;
+ clock-names = "hclk", "cclk";
+ interrupts = <GIC_SPI 576 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 577 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "int0", "int1";
+ bosch,mram-cfg = <0x0 128 64 64 64 64 32 32>;
+ };
+
+ main_mcan9: can@2791000 {
+ compatible = "bosch,m_can";
+ reg = <0x00 0x02791000 0x00 0x200>,
+ <0x00 0x02798000 0x00 0x8000>;
+ reg-names = "m_can", "message_ram";
+ power-domains = <&k3_pds 191 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 191 0>, <&k3_clks 191 1>;
+ clock-names = "hclk", "cclk";
+ interrupts = <GIC_SPI 579 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 580 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "int0", "int1";
+ bosch,mram-cfg = <0x0 128 64 64 64 64 32 32>;
+ };
+
+ main_mcan10: can@27a1000 {
+ compatible = "bosch,m_can";
+ reg = <0x00 0x027a1000 0x00 0x200>,
+ <0x00 0x027a8000 0x00 0x8000>;
+ reg-names = "m_can", "message_ram";
+ power-domains = <&k3_pds 192 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 192 0>, <&k3_clks 192 1>;
+ clock-names = "hclk", "cclk";
+ interrupts = <GIC_SPI 582 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 583 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "int0", "int1";
+ bosch,mram-cfg = <0x0 128 64 64 64 64 32 32>;
+ };
+
+ main_mcan11: can@27b1000 {
+ compatible = "bosch,m_can";
+ reg = <0x00 0x027b1000 0x00 0x200>,
+ <0x00 0x027b8000 0x00 0x8000>;
+ reg-names = "m_can", "message_ram";
+ power-domains = <&k3_pds 193 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 193 0>, <&k3_clks 193 1>;
+ clock-names = "hclk", "cclk";
+ interrupts = <GIC_SPI 585 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 586 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "int0", "int1";
+ bosch,mram-cfg = <0x0 128 64 64 64 64 32 32>;
+ };
+
+ main_mcan12: can@27c1000 {
+ compatible = "bosch,m_can";
+ reg = <0x00 0x027c1000 0x00 0x200>,
+ <0x00 0x027c8000 0x00 0x8000>;
+ reg-names = "m_can", "message_ram";
+ power-domains = <&k3_pds 194 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 194 0>, <&k3_clks 194 1>;
+ clock-names = "hclk", "cclk";
+ interrupts = <GIC_SPI 588 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 589 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "int0", "int1";
+ bosch,mram-cfg = <0x0 128 64 64 64 64 32 32>;
+ };
+
+ main_mcan13: can@27d1000 {
+ compatible = "bosch,m_can";
+ reg = <0x00 0x027d1000 0x00 0x200>,
+ <0x00 0x027d8000 0x00 0x8000>;
+ reg-names = "m_can", "message_ram";
+ power-domains = <&k3_pds 195 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 195 0>, <&k3_clks 195 1>;
+ clock-names = "hclk", "cclk";
+ interrupts = <GIC_SPI 591 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 592 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "int0", "int1";
+ bosch,mram-cfg = <0x0 128 64 64 64 64 32 32>;
+ };
+
+ main_mcan14: can@2681000 {
+ compatible = "bosch,m_can";
+ reg = <0x00 0x02681000 0x00 0x200>,
+ <0x00 0x02688000 0x00 0x8000>;
+ reg-names = "m_can", "message_ram";
+ power-domains = <&k3_pds 197 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 197 0>, <&k3_clks 197 1>;
+ clock-names = "hclk", "cclk";
+ interrupts = <GIC_SPI 594 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 595 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "int0", "int1";
+ bosch,mram-cfg = <0x0 128 64 64 64 64 32 32>;
+ };
+
+ main_mcan15: can@2691000 {
+ compatible = "bosch,m_can";
+ reg = <0x00 0x02691000 0x00 0x200>,
+ <0x00 0x02698000 0x00 0x8000>;
+ reg-names = "m_can", "message_ram";
+ power-domains = <&k3_pds 199 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 199 0>, <&k3_clks 199 1>;
+ clock-names = "hclk", "cclk";
+ interrupts = <GIC_SPI 597 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 598 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "int0", "int1";
+ bosch,mram-cfg = <0x0 128 64 64 64 64 32 32>;
+ };
+
+ main_mcan16: can@26a1000 {
+ compatible = "bosch,m_can";
+ reg = <0x00 0x026a1000 0x00 0x200>,
+ <0x00 0x026a8000 0x00 0x8000>;
+ reg-names = "m_can", "message_ram";
+ power-domains = <&k3_pds 201 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 201 0>, <&k3_clks 201 1>;
+ clock-names = "hclk", "cclk";
+ interrupts = <GIC_SPI 784 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 785 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "int0", "int1";
+ bosch,mram-cfg = <0x0 128 64 64 64 64 32 32>;
+ };
+
+ main_mcan17: can@26b1000 {
+ compatible = "bosch,m_can";
+ reg = <0x00 0x026b1000 0x00 0x200>,
+ <0x00 0x026b8000 0x00 0x8000>;
+ reg-names = "m_can", "message_ram";
+ power-domains = <&k3_pds 206 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 206 0>, <&k3_clks 206 1>;
+ clock-names = "hclk", "cclk";
+ interrupts = <GIC_SPI 787 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 788 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "int0", "int1";
+ bosch,mram-cfg = <0x0 128 64 64 64 64 32 32>;
+ };
+};
diff --git a/arch/arm64/boot/dts/ti/k3-j721s2-mcu-wakeup.dtsi b/arch/arm64/boot/dts/ti/k3-j721s2-mcu-wakeup.dtsi
new file mode 100644
index 0000000..7521963
--- /dev/null
+++ b/arch/arm64/boot/dts/ti/k3-j721s2-mcu-wakeup.dtsi
@@ -0,0 +1,302 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Device Tree Source for J721S2 SoC Family MCU/WAKEUP Domain peripherals
+ *
+ * Copyright (C) 2021 Texas Instruments Incorporated - https://www.ti.com/
+ */
+
+&cbass_mcu_wakeup {
+ sms: system-controller@44083000 {
+ compatible = "ti,k2g-sci";
+ ti,host-id = <12>;
+
+ mbox-names = "rx", "tx";
+
+ mboxes= <&secure_proxy_main 11>,
+ <&secure_proxy_main 13>;
+
+ reg-names = "debug_messages";
+ reg = <0x00 0x44083000 0x00 0x1000>;
+
+ k3_pds: power-controller {
+ compatible = "ti,sci-pm-domain";
+ #power-domain-cells = <2>;
+ };
+
+ k3_clks: clock-controller {
+ compatible = "ti,k2g-sci-clk";
+ #clock-cells = <2>;
+ };
+
+ k3_reset: reset-controller {
+ compatible = "ti,sci-reset";
+ #reset-cells = <2>;
+ };
+ };
+
+ chipid@43000014 {
+ compatible = "ti,am654-chipid";
+ reg = <0x00 0x43000014 0x00 0x4>;
+ };
+
+ mcu_ram: sram@41c00000 {
+ compatible = "mmio-sram";
+ reg = <0x00 0x41c00000 0x00 0x100000>;
+ ranges = <0x00 0x00 0x41c00000 0x100000>;
+ #address-cells = <1>;
+ #size-cells = <1>;
+ };
+
+ wkup_pmx0: pinctrl@4301c000 {
+ compatible = "pinctrl-single";
+ /* Proxy 0 addressing */
+ reg = <0x00 0x4301c000 0x00 0x178>;
+ #pinctrl-cells = <1>;
+ pinctrl-single,register-width = <32>;
+ pinctrl-single,function-mask = <0xffffffff>;
+ };
+
+ wkup_gpio_intr: interrupt-controller@42200000 {
+ compatible = "ti,sci-intr";
+ reg = <0x00 0x42200000 0x00 0x400>;
+ ti,intr-trigger-type = <1>;
+ interrupt-controller;
+ interrupt-parent = <&gic500>;
+ #interrupt-cells = <1>;
+ ti,sci = <&sms>;
+ ti,sci-dev-id = <125>;
+ ti,interrupt-ranges = <16 928 16>;
+ };
+
+ mcu_conf: syscon@40f00000 {
+ compatible = "syscon", "simple-mfd";
+ reg = <0x0 0x40f00000 0x0 0x20000>;
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges = <0x0 0x0 0x40f00000 0x20000>;
+
+ phy_gmii_sel: phy@4040 {
+ compatible = "ti,am654-phy-gmii-sel";
+ reg = <0x4040 0x4>;
+ #phy-cells = <1>;
+ };
+
+ };
+
+ wkup_uart0: serial@42300000 {
+ compatible = "ti,j721e-uart", "ti,am654-uart";
+ reg = <0x00 0x42300000 0x00 0x200>;
+ interrupts = <GIC_SPI 897 IRQ_TYPE_LEVEL_HIGH>;
+ current-speed = <115200>;
+ clocks = <&k3_clks 359 3>;
+ clock-names = "fclk";
+ power-domains = <&k3_pds 359 TI_SCI_PD_EXCLUSIVE>;
+ };
+
+ mcu_uart0: serial@40a00000 {
+ compatible = "ti,j721e-uart", "ti,am654-uart";
+ reg = <0x00 0x40a00000 0x00 0x200>;
+ interrupts = <GIC_SPI 846 IRQ_TYPE_LEVEL_HIGH>;
+ current-speed = <115200>;
+ clocks = <&k3_clks 149 3>;
+ clock-names = "fclk";
+ power-domains = <&k3_pds 149 TI_SCI_PD_EXCLUSIVE>;
+ };
+
+ wkup_gpio0: gpio@42110000 {
+ compatible = "ti,j721e-gpio", "ti,keystone-gpio";
+ reg = <0x00 0x42110000 0x00 0x100>;
+ gpio-controller;
+ #gpio-cells = <2>;
+ interrupt-parent = <&main_gpio_intr>;
+ interrupts = <103>, <104>, <105>, <106>, <107>, <108>;
+ interrupt-controller;
+ #interrupt-cells = <2>;
+ ti,ngpio = <89>;
+ ti,davinci-gpio-unbanked = <0>;
+ power-domains = <&k3_pds 115 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 115 0>;
+ clock-names = "gpio";
+ };
+
+ wkup_gpio1: gpio@42100000 {
+ compatible = "ti,j721e-gpio", "ti,keystone-gpio";
+ reg = <0x00 0x42100000 0x00 0x100>;
+ gpio-controller;
+ #gpio-cells = <2>;
+ interrupt-parent = <&main_gpio_intr>;
+ interrupts = <112>, <113>, <114>, <115>, <116>, <117>;
+ interrupt-controller;
+ #interrupt-cells = <2>;
+ ti,ngpio = <89>;
+ ti,davinci-gpio-unbanked = <0>;
+ power-domains = <&k3_pds 116 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 116 0>;
+ clock-names = "gpio";
+ };
+
+ wkup_i2c0: i2c@42120000 {
+ compatible = "ti,j721e-i2c", "ti,omap4-i2c";
+ reg = <0x00 0x42120000 0x00 0x100>;
+ interrupts = <GIC_SPI 896 IRQ_TYPE_LEVEL_HIGH>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ clocks = <&k3_clks 223 1>;
+ clock-names = "fck";
+ power-domains = <&k3_pds 223 TI_SCI_PD_EXCLUSIVE>;
+ };
+
+ mcu_i2c0: i2c@40b00000 {
+ compatible = "ti,j721e-i2c", "ti,omap4-i2c";
+ reg = <0x00 0x40b00000 0x00 0x100>;
+ interrupts = <GIC_SPI 852 IRQ_TYPE_LEVEL_HIGH>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ clocks = <&k3_clks 221 1>;
+ clock-names = "fck";
+ power-domains = <&k3_pds 221 TI_SCI_PD_EXCLUSIVE>;
+ };
+
+ mcu_i2c1: i2c@40b10000 {
+ compatible = "ti,j721e-i2c", "ti,omap4-i2c";
+ reg = <0x00 0x40b10000 0x00 0x100>;
+ interrupts = <GIC_SPI 853 IRQ_TYPE_LEVEL_HIGH>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ clocks = <&k3_clks 222 1>;
+ clock-names = "fck";
+ power-domains = <&k3_pds 222 TI_SCI_PD_EXCLUSIVE>;
+ };
+
+ mcu_mcan0: can@40528000 {
+ compatible = "bosch,m_can";
+ reg = <0x00 0x40528000 0x00 0x200>,
+ <0x00 0x40500000 0x00 0x8000>;
+ reg-names = "m_can", "message_ram";
+ power-domains = <&k3_pds 207 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 207 0>, <&k3_clks 207 1>;
+ clock-names = "hclk", "cclk";
+ interrupts = <GIC_SPI 832 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 833 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "int0", "int1";
+ bosch,mram-cfg = <0x0 128 64 64 64 64 32 32>;
+ };
+
+ mcu_mcan1: can@40568000 {
+ compatible = "bosch,m_can";
+ reg = <0x00 0x40568000 0x00 0x200>,
+ <0x00 0x40540000 0x00 0x8000>;
+ reg-names = "m_can", "message_ram";
+ power-domains = <&k3_pds 208 TI_SCI_PD_EXCLUSIVE>;
+ clocks = <&k3_clks 208 0>, <&k3_clks 208 1>;
+ clock-names = "hclk", "cclk";
+ interrupts = <GIC_SPI 835 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 836 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "int0", "int1";
+ bosch,mram-cfg = <0x0 128 64 64 64 64 32 32>;
+ };
+
+ mcu_navss: bus@28380000{
+ compatible = "simple-mfd";
+ #address-cells = <2>;
+ #size-cells = <2>;
+ ranges = <0x00 0x28380000 0x00 0x28380000 0x00 0x03880000>;
+ dma-coherent;
+ dma-ranges;
+
+ ti,sci-dev-id = <267>;
+
+ mcu_ringacc: ringacc@2b800000 {
+ compatible = "ti,am654-navss-ringacc";
+ reg = <0x0 0x2b800000 0x0 0x400000>,
+ <0x0 0x2b000000 0x0 0x400000>,
+ <0x0 0x28590000 0x0 0x100>,
+ <0x0 0x2a500000 0x0 0x40000>;
+ reg-names = "rt", "fifos", "proxy_gcfg", "proxy_target";
+ ti,num-rings = <286>;
+ ti,sci-rm-range-gp-rings = <0x1>;
+ ti,sci = <&sms>;
+ ti,sci-dev-id = <272>;
+ msi-parent = <&main_udmass_inta>;
+ };
+
+ mcu_udmap: dma-controller@285c0000 {
+ compatible = "ti,j721e-navss-mcu-udmap";
+ reg = <0x0 0x285c0000 0x0 0x100>,
+ <0x0 0x2a800000 0x0 0x40000>,
+ <0x0 0x2aa00000 0x0 0x40000>;
+ reg-names = "gcfg", "rchanrt", "tchanrt";
+ msi-parent = <&main_udmass_inta>;
+ #dma-cells = <1>;
+
+ ti,sci = <&sms>;
+ ti,sci-dev-id = <273>;
+ ti,ringacc = <&mcu_ringacc>;
+ ti,sci-rm-range-tchan = <0x0d>, /* TX_CHAN */
+ <0x0f>; /* TX_HCHAN */
+ ti,sci-rm-range-rchan = <0x0a>, /* RX_CHAN */
+ <0x0b>; /* RX_HCHAN */
+ ti,sci-rm-range-rflow = <0x00>; /* GP RFLOW */
+ };
+ };
+
+ mcu_cpsw: ethernet@46000000 {
+ compatible = "ti,j721e-cpsw-nuss";
+ #address-cells = <2>;
+ #size-cells = <2>;
+ reg = <0x0 0x46000000 0x0 0x200000>;
+ reg-names = "cpsw_nuss";
+ ranges = <0x0 0x0 0x0 0x46000000 0x0 0x200000>;
+ dma-coherent;
+ clocks = <&k3_clks 29 28>;
+ clock-names = "fck";
+ power-domains = <&k3_pds 29 TI_SCI_PD_EXCLUSIVE>;
+
+ dmas = <&mcu_udmap 0xf000>,
+ <&mcu_udmap 0xf001>,
+ <&mcu_udmap 0xf002>,
+ <&mcu_udmap 0xf003>,
+ <&mcu_udmap 0xf004>,
+ <&mcu_udmap 0xf005>,
+ <&mcu_udmap 0xf006>,
+ <&mcu_udmap 0xf007>,
+ <&mcu_udmap 0x7000>;
+ dma-names = "tx0", "tx1", "tx2", "tx3",
+ "tx4", "tx5", "tx6", "tx7",
+ "rx";
+
+ ethernet-ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ cpsw_port1: port@1 {
+ reg = <1>;
+ ti,mac-only;
+ label = "port1";
+ ti,syscon-efuse = <&mcu_conf 0x200>;
+ phys = <&phy_gmii_sel 1>;
+ };
+ };
+
+ davinci_mdio: mdio@f00 {
+ compatible = "ti,cpsw-mdio","ti,davinci_mdio";
+ reg = <0x0 0xf00 0x0 0x100>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ clocks = <&k3_clks 29 28>;
+ clock-names = "fck";
+ bus_freq = <1000000>;
+ };
+
+ cpts@3d000 {
+ compatible = "ti,am65-cpts";
+ reg = <0x0 0x3d000 0x0 0x400>;
+ clocks = <&k3_clks 29 3>;
+ clock-names = "cpts";
+ interrupts-extended = <&gic500 GIC_SPI 858 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "cpts";
+ ti,cpts-ext-ts-inputs = <4>;
+ ti,cpts-periodic-outputs = <2>;
+ };
+ };
+};
diff --git a/arch/arm64/boot/dts/ti/k3-j721s2-som-p0.dtsi b/arch/arm64/boot/dts/ti/k3-j721s2-som-p0.dtsi
new file mode 100644
index 0000000..76f0cea
--- /dev/null
+++ b/arch/arm64/boot/dts/ti/k3-j721s2-som-p0.dtsi
@@ -0,0 +1,175 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * SoM: https://www.ti.com/lit/zip/sprr439
+ *
+ * Copyright (C) 2021 Texas Instruments Incorporated - https://www.ti.com/
+ */
+
+/dts-v1/;
+
+#include "k3-j721s2.dtsi"
+#include <dt-bindings/gpio/gpio.h>
+
+/ {
+ memory@80000000 {
+ device_type = "memory";
+ /* 16 GB RAM */
+ reg = <0x00 0x80000000 0x00 0x80000000>,
+ <0x08 0x80000000 0x03 0x80000000>;
+ };
+
+ /* Reserving memory regions still pending */
+ reserved_memory: reserved-memory {
+ #address-cells = <2>;
+ #size-cells = <2>;
+ ranges;
+
+ secure_ddr: optee@9e800000 {
+ reg = <0x00 0x9e800000 0x00 0x01800000>;
+ alignment = <0x1000>;
+ no-map;
+ };
+ };
+
+ transceiver0: can-phy0 {
+ /* standby pin has been grounded by default */
+ compatible = "ti,tcan1042";
+ #phy-cells = <0>;
+ max-bitrate = <5000000>;
+ };
+};
+
+&main_pmx0 {
+ main_i2c0_pins_default: main-i2c0-pins-default {
+ pinctrl-single,pins = <
+ J721S2_IOPAD(0x0e0, PIN_INPUT_PULLUP, 0) /* (AH25) I2C0_SCL */
+ J721S2_IOPAD(0x0e4, PIN_INPUT_PULLUP, 0) /* (AE24) I2C0_SDA */
+ >;
+ };
+
+ main_mcan16_pins_default: main-mcan16-pins-default {
+ pinctrl-single,pins = <
+ J721S2_IOPAD(0x028, PIN_INPUT, 0) /* (AB24) MCAN16_RX */
+ J721S2_IOPAD(0x024, PIN_OUTPUT, 0) /* (Y28) MCAN16_TX */
+ >;
+ };
+};
+
+&main_i2c0 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&main_i2c0_pins_default>;
+ clock-frequency = <400000>;
+
+ exp_som: gpio@21 {
+ compatible = "ti,tca6408";
+ reg = <0x21>;
+ gpio-controller;
+ #gpio-cells = <2>;
+ gpio-line-names = "USB2.0_MUX_SEL", "CANUART_MUX1_SEL0",
+ "CANUART_MUX2_SEL0", "CANUART_MUX_SEL1",
+ "GPIO_RGMII1_RST", "GPIO_eDP_ENABLE",
+ "GPIO_LIN_EN", "CAN_STB";
+ };
+};
+
+&main_mcan16 {
+ pinctrl-0 = <&main_mcan16_pins_default>;
+ pinctrl-names = "default";
+ phys = <&transceiver0>;
+};
+
+&mailbox0_cluster0 {
+ status = "disabled";
+};
+
+&mailbox0_cluster1 {
+ status = "disabled";
+};
+
+&mailbox0_cluster2 {
+ status = "disabled";
+};
+
+&mailbox0_cluster3 {
+ status = "disabled";
+};
+
+&mailbox0_cluster4 {
+ status = "disabled";
+};
+
+&mailbox0_cluster5 {
+ status = "disabled";
+};
+
+&mailbox0_cluster6 {
+ status = "disabled";
+};
+
+&mailbox0_cluster7 {
+ status = "disabled";
+};
+
+&mailbox0_cluster8 {
+ status = "disabled";
+};
+
+&mailbox0_cluster9 {
+ status = "disabled";
+};
+
+&mailbox0_cluster10 {
+ status = "disabled";
+};
+
+&mailbox0_cluster11 {
+ status = "disabled";
+};
+
+&mailbox1_cluster0 {
+ status = "disabled";
+};
+
+&mailbox1_cluster1 {
+ status = "disabled";
+};
+
+&mailbox1_cluster2 {
+ status = "disabled";
+};
+
+&mailbox1_cluster3 {
+ status = "disabled";
+};
+
+&mailbox1_cluster4 {
+ status = "disabled";
+};
+
+&mailbox1_cluster5 {
+ status = "disabled";
+};
+
+&mailbox1_cluster6 {
+ status = "disabled";
+};
+
+&mailbox1_cluster7 {
+ status = "disabled";
+};
+
+&mailbox1_cluster8 {
+ status = "disabled";
+};
+
+&mailbox1_cluster9 {
+ status = "disabled";
+};
+
+&mailbox1_cluster10 {
+ status = "disabled";
+};
+
+&mailbox1_cluster11 {
+ status = "disabled";
+};
diff --git a/arch/arm64/boot/dts/ti/k3-j721s2.dtsi b/arch/arm64/boot/dts/ti/k3-j721s2.dtsi
new file mode 100644
index 0000000..80d3cae
--- /dev/null
+++ b/arch/arm64/boot/dts/ti/k3-j721s2.dtsi
@@ -0,0 +1,189 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Device Tree Source for J721S2 SoC Family
+ *
+ * TRM (SPRUJ28 – NOVEMBER 2021) : http://www.ti.com/lit/pdf/spruj28
+ *
+ * Copyright (C) 2021 Texas Instruments Incorporated - https://www.ti.com/
+ *
+ */
+
+#include <dt-bindings/interrupt-controller/irq.h>
+#include <dt-bindings/interrupt-controller/arm-gic.h>
+#include <dt-bindings/pinctrl/k3.h>
+#include <dt-bindings/soc/ti,sci_pm_domain.h>
+
+/ {
+
+ model = "Texas Instruments K3 J721S2 SoC";
+ compatible = "ti,j721s2";
+ interrupt-parent = <&gic500>;
+ #address-cells = <2>;
+ #size-cells = <2>;
+
+ aliases {
+ serial0 = &wkup_uart0;
+ serial1 = &mcu_uart0;
+ serial2 = &main_uart0;
+ serial3 = &main_uart1;
+ serial4 = &main_uart2;
+ serial5 = &main_uart3;
+ serial6 = &main_uart4;
+ serial7 = &main_uart5;
+ serial8 = &main_uart6;
+ serial9 = &main_uart7;
+ serial10 = &main_uart8;
+ serial11 = &main_uart9;
+ mmc0 = &main_sdhci0;
+ mmc1 = &main_sdhci1;
+ can0 = &main_mcan16;
+ can1 = &mcu_mcan0;
+ can2 = &mcu_mcan1;
+ can3 = &main_mcan3;
+ can4 = &main_mcan5;
+ };
+
+ chosen { };
+
+ cpus {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ cpu-map {
+ cluster0: cluster0 {
+ core0 {
+ cpu = <&cpu0>;
+ };
+
+ core1 {
+ cpu = <&cpu1>;
+ };
+ };
+ };
+
+ cpu0: cpu@0 {
+ compatible = "arm,cortex-a72";
+ reg = <0x000>;
+ device_type = "cpu";
+ enable-method = "psci";
+ i-cache-size = <0xc000>;
+ i-cache-line-size = <64>;
+ i-cache-sets = <256>;
+ d-cache-size = <0x8000>;
+ d-cache-line-size = <64>;
+ d-cache-sets = <256>;
+ next-level-cache = <&L2_0>;
+ };
+
+ cpu1: cpu@1 {
+ compatible = "arm,cortex-a72";
+ reg = <0x001>;
+ device_type = "cpu";
+ enable-method = "psci";
+ i-cache-size = <0xc000>;
+ i-cache-line-size = <64>;
+ i-cache-sets = <256>;
+ d-cache-size = <0x8000>;
+ d-cache-line-size = <64>;
+ d-cache-sets = <256>;
+ next-level-cache = <&L2_0>;
+ };
+ };
+
+ L2_0: l2-cache0 {
+ compatible = "cache";
+ cache-level = <2>;
+ cache-size = <0x100000>;
+ cache-line-size = <64>;
+ cache-sets = <1024>;
+ next-level-cache = <&msmc_l3>;
+ };
+
+ msmc_l3: l3-cache0 {
+ compatible = "cache";
+ cache-level = <3>;
+ };
+
+ firmware {
+ optee {
+ compatible = "linaro,optee-tz";
+ method = "smc";
+ };
+
+ psci: psci {
+ compatible = "arm,psci-1.0";
+ method = "smc";
+ };
+ };
+
+ a72_timer0: timer-cl0-cpu0 {
+ compatible = "arm,armv8-timer";
+ interrupts = <GIC_PPI 13 IRQ_TYPE_LEVEL_LOW>, /* cntpsirq */
+ <GIC_PPI 14 IRQ_TYPE_LEVEL_LOW>, /* cntpnsirq */
+ <GIC_PPI 11 IRQ_TYPE_LEVEL_LOW>, /* cntvirq */
+ <GIC_PPI 10 IRQ_TYPE_LEVEL_LOW>; /* cnthpirq */
+
+ };
+
+ pmu: pmu {
+ compatible = "arm,cortex-a72-pmu";
+ /* Recommendation from GIC500 TRM Table A.3 */
+ interrupts = <GIC_PPI 7 IRQ_TYPE_LEVEL_HIGH>;
+ };
+
+ cbass_main: bus@100000 {
+ compatible = "simple-bus";
+ #address-cells = <2>;
+ #size-cells = <2>;
+ ranges = <0x00 0x00100000 0x00 0x00100000 0x00 0x00020000>, /* ctrl mmr */
+ <0x00 0x00600000 0x00 0x00600000 0x00 0x00031100>, /* GPIO */
+ <0x00 0x01000000 0x00 0x01000000 0x00 0x0d000000>, /* Most peripherals */
+ <0x00 0x0d800000 0x00 0x0d800000 0x00 0x00800000>, /* PCIe Core*/
+ <0x00 0x18000000 0x00 0x18000000 0x00 0x08000000>, /* PCIe1 DAT0 */
+ <0x00 0x64800000 0x00 0x64800000 0x00 0x0070c000>, /* C71_1 */
+ <0x00 0x65800000 0x00 0x65800000 0x00 0x0070c000>, /* C71_2 */
+ <0x00 0x70000000 0x00 0x70000000 0x00 0x00400000>, /* MSMC RAM */
+ <0x00 0x30000000 0x00 0x30000000 0x00 0x0c400000>, /* MAIN NAVSS */
+ <0x41 0x00000000 0x41 0x00000000 0x01 0x00000000>, /* PCIe1 DAT1 */
+ <0x4e 0x20000000 0x4e 0x20000000 0x00 0x00080000>, /* GPU */
+
+ /* MCUSS_WKUP Range */
+ <0x00 0x28380000 0x00 0x28380000 0x00 0x03880000>,
+ <0x00 0x40200000 0x00 0x40200000 0x00 0x00998400>,
+ <0x00 0x40f00000 0x00 0x40f00000 0x00 0x00020000>,
+ <0x00 0x41000000 0x00 0x41000000 0x00 0x00020000>,
+ <0x00 0x41400000 0x00 0x41400000 0x00 0x00020000>,
+ <0x00 0x41c00000 0x00 0x41c00000 0x00 0x00100000>,
+ <0x00 0x42040000 0x00 0x42040000 0x00 0x03ac2400>,
+ <0x00 0x45100000 0x00 0x45100000 0x00 0x00c24000>,
+ <0x00 0x46000000 0x00 0x46000000 0x00 0x00200000>,
+ <0x00 0x47000000 0x00 0x47000000 0x00 0x00068400>,
+ <0x00 0x50000000 0x00 0x50000000 0x00 0x10000000>,
+ <0x05 0x00000000 0x05 0x00000000 0x01 0x00000000>,
+ <0x07 0x00000000 0x07 0x00000000 0x01 0x00000000>;
+
+ cbass_mcu_wakeup: bus@28380000 {
+ compatible = "simple-bus";
+ #address-cells = <2>;
+ #size-cells = <2>;
+ ranges = <0x00 0x28380000 0x00 0x28380000 0x00 0x03880000>, /* MCU NAVSS*/
+ <0x00 0x40200000 0x00 0x40200000 0x00 0x00998400>, /* First peripheral window */
+ <0x00 0x40f00000 0x00 0x40f00000 0x00 0x00020000>, /* CTRL_MMR0 */
+ <0x00 0x41000000 0x00 0x41000000 0x00 0x00020000>, /* MCU R5F Core0 */
+ <0x00 0x41400000 0x00 0x41400000 0x00 0x00020000>, /* MCU R5F Core1 */
+ <0x00 0x41c00000 0x00 0x41c00000 0x00 0x00100000>, /* MCU SRAM */
+ <0x00 0x42040000 0x00 0x42040000 0x00 0x03ac2400>, /* WKUP peripheral window */
+ <0x00 0x45100000 0x00 0x45100000 0x00 0x00c24000>, /* MMRs, remaining NAVSS */
+ <0x00 0x46000000 0x00 0x46000000 0x00 0x00200000>, /* CPSW */
+ <0x00 0x47000000 0x00 0x47000000 0x00 0x00068400>, /* OSPI register space */
+ <0x00 0x50000000 0x00 0x50000000 0x00 0x10000000>, /* FSS OSPI0/1 data region 0 */
+ <0x05 0x00000000 0x05 0x00000000 0x01 0x00000000>, /* FSS OSPI0 data region 3 */
+ <0x07 0x00000000 0x07 0x00000000 0x01 0x00000000>; /* FSS OSPI1 data region 3*/
+
+ };
+
+ };
+};
+
+/* Now include peripherals from each bus segment */
+#include "k3-j721s2-main.dtsi"
+#include "k3-j721s2-mcu-wakeup.dtsi"
diff --git a/arch/arm64/configs/defconfig b/arch/arm64/configs/defconfig
index f2e2b9bd..30516dc 100644
--- a/arch/arm64/configs/defconfig
+++ b/arch/arm64/configs/defconfig
@@ -363,6 +363,7 @@
CONFIG_AT803X_PHY=y
CONFIG_REALTEK_PHY=y
CONFIG_ROCKCHIP_PHY=y
+CONFIG_DP83867_PHY=y
CONFIG_VITESSE_PHY=y
CONFIG_USB_PEGASUS=m
CONFIG_USB_RTL8150=m
@@ -397,6 +398,7 @@
CONFIG_INPUT_MISC=y
CONFIG_INPUT_PM8941_PWRKEY=y
CONFIG_INPUT_PM8XXX_VIBRATOR=m
+CONFIG_INPUT_PWM_BEEPER=m
CONFIG_INPUT_PWM_VIBRA=m
CONFIG_INPUT_HISI_POWERKEY=y
# CONFIG_SERIO_SERPORT is not set
@@ -461,6 +463,7 @@
CONFIG_I2C_QUP=y
CONFIG_I2C_RIIC=y
CONFIG_I2C_RK3X=y
+CONFIG_I2C_S3C2410=y
CONFIG_I2C_SH_MOBILE=y
CONFIG_I2C_TEGRA=y
CONFIG_I2C_UNIPHIER_F=y
@@ -504,6 +507,7 @@
CONFIG_PINCTRL_IMX8QM=y
CONFIG_PINCTRL_IMX8QXP=y
CONFIG_PINCTRL_IMX8DXL=y
+CONFIG_PINCTRL_IMX8ULP=y
CONFIG_PINCTRL_MSM=y
CONFIG_PINCTRL_IPQ8074=y
CONFIG_PINCTRL_IPQ6018=y
@@ -555,6 +559,7 @@
CONFIG_CHARGER_BQ25890=m
CONFIG_CHARGER_BQ25980=m
CONFIG_SENSORS_ARM_SCPI=y
+CONFIG_SENSORS_JC42=m
CONFIG_SENSORS_LM90=m
CONFIG_SENSORS_PWM_FAN=m
CONFIG_SENSORS_RASPBERRYPI_HWMON=m
@@ -668,6 +673,7 @@
CONFIG_SDR_PLATFORM_DRIVERS=y
CONFIG_VIDEO_RCAR_DRIF=m
CONFIG_VIDEO_IMX219=m
+CONFIG_VIDEO_OV5640=m
CONFIG_VIDEO_OV5645=m
CONFIG_VIDEO_QCOM_CAMSS=m
CONFIG_DRM=m
@@ -792,6 +798,7 @@
CONFIG_SND_SOC_SIMPLE_AMPLIFIER=m
CONFIG_SND_SOC_SIMPLE_MUX=m
CONFIG_SND_SOC_TAS571X=m
+CONFIG_SND_SOC_TLV320AIC32X4_I2C=m
CONFIG_SND_SOC_WCD934X=m
CONFIG_SND_SOC_WM8904=m
CONFIG_SND_SOC_WM8960=m
@@ -907,6 +914,7 @@
CONFIG_RTC_DRV_HYM8563=m
CONFIG_RTC_DRV_MAX77686=y
CONFIG_RTC_DRV_RK808=m
+CONFIG_RTC_DRV_PCF85063=m
CONFIG_RTC_DRV_PCF85363=m
CONFIG_RTC_DRV_M41T80=m
CONFIG_RTC_DRV_RX8581=m
@@ -958,6 +966,7 @@
CONFIG_STAGING=y
CONFIG_STAGING_MEDIA=y
CONFIG_VIDEO_HANTRO=m
+CONFIG_VIDEO_IMX_MEDIA=m
CONFIG_CHROME_PLATFORMS=y
CONFIG_CROS_EC=y
CONFIG_CROS_EC_I2C=y
@@ -979,6 +988,7 @@
CONFIG_CLK_IMX8MP=y
CONFIG_CLK_IMX8MQ=y
CONFIG_CLK_IMX8QXP=y
+CONFIG_CLK_IMX8ULP=y
CONFIG_TI_SCI_CLK=y
CONFIG_COMMON_CLK_QCOM=y
CONFIG_QCOM_A53PLL=y
@@ -1069,6 +1079,7 @@
CONFIG_ARCH_R8A77990=y
CONFIG_ARCH_R8A77995=y
CONFIG_ARCH_R8A779A0=y
+CONFIG_ARCH_R8A779F0=y
CONFIG_ARCH_R9A07G044=y
CONFIG_ROCKCHIP_PM_DOMAINS=y
CONFIG_ARCH_TEGRA_132_SOC=y
@@ -1174,6 +1185,8 @@
CONFIG_MUX_MMIO=y
CONFIG_INTERCONNECT=y
CONFIG_INTERCONNECT_IMX=m
+CONFIG_INTERCONNECT_IMX8MM=m
+CONFIG_INTERCONNECT_IMX8MN=m
CONFIG_INTERCONNECT_IMX8MQ=m
CONFIG_INTERCONNECT_QCOM=y
CONFIG_INTERCONNECT_QCOM_MSM8916=m
diff --git a/arch/arm64/crypto/aes-modes.S b/arch/arm64/crypto/aes-modes.S
index b495de2..ff01f01 100644
--- a/arch/arm64/crypto/aes-modes.S
+++ b/arch/arm64/crypto/aes-modes.S
@@ -363,15 +363,15 @@
adr x16, 1f
sub x16, x16, x12, lsl #3
br x16
- hint 34 // bti c
+ bti c
mov v0.d[0], vctr.d[0]
- hint 34 // bti c
+ bti c
mov v1.d[0], vctr.d[0]
- hint 34 // bti c
+ bti c
mov v2.d[0], vctr.d[0]
- hint 34 // bti c
+ bti c
mov v3.d[0], vctr.d[0]
-ST5( hint 34 )
+ST5( bti c )
ST5( mov v4.d[0], vctr.d[0] )
1: b 2f
.previous
diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h
index 136d13f..e8bd0af 100644
--- a/arch/arm64/include/asm/assembler.h
+++ b/arch/arm64/include/asm/assembler.h
@@ -791,6 +791,16 @@ alternative_endif
.endm
/*
+ * Branch Target Identifier (BTI)
+ */
+ .macro bti, targets
+ .equ .L__bti_targets_c, 34
+ .equ .L__bti_targets_j, 36
+ .equ .L__bti_targets_jc,38
+ hint #.L__bti_targets_\targets
+ .endm
+
+/*
* This macro emits a program property note section identifying
* architecture features which require special handling, mainly for
* use in assembly files included in the VDSO.
diff --git a/arch/arm64/include/asm/atomic_ll_sc.h b/arch/arm64/include/asm/atomic_ll_sc.h
index 13869b7..fe0db8d 100644
--- a/arch/arm64/include/asm/atomic_ll_sc.h
+++ b/arch/arm64/include/asm/atomic_ll_sc.h
@@ -44,11 +44,11 @@ __ll_sc_atomic_##op(int i, atomic_t *v) \
\
asm volatile("// atomic_" #op "\n" \
__LL_SC_FALLBACK( \
-" prfm pstl1strm, %2\n" \
-"1: ldxr %w0, %2\n" \
-" " #asm_op " %w0, %w0, %w3\n" \
-" stxr %w1, %w0, %2\n" \
-" cbnz %w1, 1b\n") \
+ " prfm pstl1strm, %2\n" \
+ "1: ldxr %w0, %2\n" \
+ " " #asm_op " %w0, %w0, %w3\n" \
+ " stxr %w1, %w0, %2\n" \
+ " cbnz %w1, 1b\n") \
: "=&r" (result), "=&r" (tmp), "+Q" (v->counter) \
: __stringify(constraint) "r" (i)); \
}
@@ -62,12 +62,12 @@ __ll_sc_atomic_##op##_return##name(int i, atomic_t *v) \
\
asm volatile("// atomic_" #op "_return" #name "\n" \
__LL_SC_FALLBACK( \
-" prfm pstl1strm, %2\n" \
-"1: ld" #acq "xr %w0, %2\n" \
-" " #asm_op " %w0, %w0, %w3\n" \
-" st" #rel "xr %w1, %w0, %2\n" \
-" cbnz %w1, 1b\n" \
-" " #mb ) \
+ " prfm pstl1strm, %2\n" \
+ "1: ld" #acq "xr %w0, %2\n" \
+ " " #asm_op " %w0, %w0, %w3\n" \
+ " st" #rel "xr %w1, %w0, %2\n" \
+ " cbnz %w1, 1b\n" \
+ " " #mb ) \
: "=&r" (result), "=&r" (tmp), "+Q" (v->counter) \
: __stringify(constraint) "r" (i) \
: cl); \
@@ -84,12 +84,12 @@ __ll_sc_atomic_fetch_##op##name(int i, atomic_t *v) \
\
asm volatile("// atomic_fetch_" #op #name "\n" \
__LL_SC_FALLBACK( \
-" prfm pstl1strm, %3\n" \
-"1: ld" #acq "xr %w0, %3\n" \
-" " #asm_op " %w1, %w0, %w4\n" \
-" st" #rel "xr %w2, %w1, %3\n" \
-" cbnz %w2, 1b\n" \
-" " #mb ) \
+ " prfm pstl1strm, %3\n" \
+ "1: ld" #acq "xr %w0, %3\n" \
+ " " #asm_op " %w1, %w0, %w4\n" \
+ " st" #rel "xr %w2, %w1, %3\n" \
+ " cbnz %w2, 1b\n" \
+ " " #mb ) \
: "=&r" (result), "=&r" (val), "=&r" (tmp), "+Q" (v->counter) \
: __stringify(constraint) "r" (i) \
: cl); \
@@ -143,11 +143,11 @@ __ll_sc_atomic64_##op(s64 i, atomic64_t *v) \
\
asm volatile("// atomic64_" #op "\n" \
__LL_SC_FALLBACK( \
-" prfm pstl1strm, %2\n" \
-"1: ldxr %0, %2\n" \
-" " #asm_op " %0, %0, %3\n" \
-" stxr %w1, %0, %2\n" \
-" cbnz %w1, 1b") \
+ " prfm pstl1strm, %2\n" \
+ "1: ldxr %0, %2\n" \
+ " " #asm_op " %0, %0, %3\n" \
+ " stxr %w1, %0, %2\n" \
+ " cbnz %w1, 1b") \
: "=&r" (result), "=&r" (tmp), "+Q" (v->counter) \
: __stringify(constraint) "r" (i)); \
}
@@ -161,12 +161,12 @@ __ll_sc_atomic64_##op##_return##name(s64 i, atomic64_t *v) \
\
asm volatile("// atomic64_" #op "_return" #name "\n" \
__LL_SC_FALLBACK( \
-" prfm pstl1strm, %2\n" \
-"1: ld" #acq "xr %0, %2\n" \
-" " #asm_op " %0, %0, %3\n" \
-" st" #rel "xr %w1, %0, %2\n" \
-" cbnz %w1, 1b\n" \
-" " #mb ) \
+ " prfm pstl1strm, %2\n" \
+ "1: ld" #acq "xr %0, %2\n" \
+ " " #asm_op " %0, %0, %3\n" \
+ " st" #rel "xr %w1, %0, %2\n" \
+ " cbnz %w1, 1b\n" \
+ " " #mb ) \
: "=&r" (result), "=&r" (tmp), "+Q" (v->counter) \
: __stringify(constraint) "r" (i) \
: cl); \
@@ -176,19 +176,19 @@ __ll_sc_atomic64_##op##_return##name(s64 i, atomic64_t *v) \
#define ATOMIC64_FETCH_OP(name, mb, acq, rel, cl, op, asm_op, constraint)\
static inline long \
-__ll_sc_atomic64_fetch_##op##name(s64 i, atomic64_t *v) \
+__ll_sc_atomic64_fetch_##op##name(s64 i, atomic64_t *v) \
{ \
s64 result, val; \
unsigned long tmp; \
\
asm volatile("// atomic64_fetch_" #op #name "\n" \
__LL_SC_FALLBACK( \
-" prfm pstl1strm, %3\n" \
-"1: ld" #acq "xr %0, %3\n" \
-" " #asm_op " %1, %0, %4\n" \
-" st" #rel "xr %w2, %1, %3\n" \
-" cbnz %w2, 1b\n" \
-" " #mb ) \
+ " prfm pstl1strm, %3\n" \
+ "1: ld" #acq "xr %0, %3\n" \
+ " " #asm_op " %1, %0, %4\n" \
+ " st" #rel "xr %w2, %1, %3\n" \
+ " cbnz %w2, 1b\n" \
+ " " #mb ) \
: "=&r" (result), "=&r" (val), "=&r" (tmp), "+Q" (v->counter) \
: __stringify(constraint) "r" (i) \
: cl); \
@@ -241,14 +241,14 @@ __ll_sc_atomic64_dec_if_positive(atomic64_t *v)
asm volatile("// atomic64_dec_if_positive\n"
__LL_SC_FALLBACK(
-" prfm pstl1strm, %2\n"
-"1: ldxr %0, %2\n"
-" subs %0, %0, #1\n"
-" b.lt 2f\n"
-" stlxr %w1, %0, %2\n"
-" cbnz %w1, 1b\n"
-" dmb ish\n"
-"2:")
+ " prfm pstl1strm, %2\n"
+ "1: ldxr %0, %2\n"
+ " subs %0, %0, #1\n"
+ " b.lt 2f\n"
+ " stlxr %w1, %0, %2\n"
+ " cbnz %w1, 1b\n"
+ " dmb ish\n"
+ "2:")
: "=&r" (result), "=&r" (tmp), "+Q" (v->counter)
:
: "cc", "memory");
diff --git a/arch/arm64/include/asm/atomic_lse.h b/arch/arm64/include/asm/atomic_lse.h
index da3280f..d955ade 100644
--- a/arch/arm64/include/asm/atomic_lse.h
+++ b/arch/arm64/include/asm/atomic_lse.h
@@ -11,13 +11,13 @@
#define __ASM_ATOMIC_LSE_H
#define ATOMIC_OP(op, asm_op) \
-static inline void __lse_atomic_##op(int i, atomic_t *v) \
+static inline void __lse_atomic_##op(int i, atomic_t *v) \
{ \
asm volatile( \
__LSE_PREAMBLE \
-" " #asm_op " %w[i], %[v]\n" \
- : [i] "+r" (i), [v] "+Q" (v->counter) \
- : "r" (v)); \
+ " " #asm_op " %w[i], %[v]\n" \
+ : [v] "+Q" (v->counter) \
+ : [i] "r" (i)); \
}
ATOMIC_OP(andnot, stclr)
@@ -25,19 +25,27 @@ ATOMIC_OP(or, stset)
ATOMIC_OP(xor, steor)
ATOMIC_OP(add, stadd)
+static inline void __lse_atomic_sub(int i, atomic_t *v)
+{
+ __lse_atomic_add(-i, v);
+}
+
#undef ATOMIC_OP
#define ATOMIC_FETCH_OP(name, mb, op, asm_op, cl...) \
static inline int __lse_atomic_fetch_##op##name(int i, atomic_t *v) \
{ \
+ int old; \
+ \
asm volatile( \
__LSE_PREAMBLE \
-" " #asm_op #mb " %w[i], %w[i], %[v]" \
- : [i] "+r" (i), [v] "+Q" (v->counter) \
- : "r" (v) \
+ " " #asm_op #mb " %w[i], %w[old], %[v]" \
+ : [v] "+Q" (v->counter), \
+ [old] "=r" (old) \
+ : [i] "r" (i) \
: cl); \
\
- return i; \
+ return old; \
}
#define ATOMIC_FETCH_OPS(op, asm_op) \
@@ -54,51 +62,46 @@ ATOMIC_FETCH_OPS(add, ldadd)
#undef ATOMIC_FETCH_OP
#undef ATOMIC_FETCH_OPS
-#define ATOMIC_OP_ADD_RETURN(name, mb, cl...) \
-static inline int __lse_atomic_add_return##name(int i, atomic_t *v) \
+#define ATOMIC_FETCH_OP_SUB(name) \
+static inline int __lse_atomic_fetch_sub##name(int i, atomic_t *v) \
{ \
- u32 tmp; \
- \
- asm volatile( \
- __LSE_PREAMBLE \
- " ldadd" #mb " %w[i], %w[tmp], %[v]\n" \
- " add %w[i], %w[i], %w[tmp]" \
- : [i] "+r" (i), [v] "+Q" (v->counter), [tmp] "=&r" (tmp) \
- : "r" (v) \
- : cl); \
- \
- return i; \
+ return __lse_atomic_fetch_add##name(-i, v); \
}
-ATOMIC_OP_ADD_RETURN(_relaxed, )
-ATOMIC_OP_ADD_RETURN(_acquire, a, "memory")
-ATOMIC_OP_ADD_RETURN(_release, l, "memory")
-ATOMIC_OP_ADD_RETURN( , al, "memory")
+ATOMIC_FETCH_OP_SUB(_relaxed)
+ATOMIC_FETCH_OP_SUB(_acquire)
+ATOMIC_FETCH_OP_SUB(_release)
+ATOMIC_FETCH_OP_SUB( )
-#undef ATOMIC_OP_ADD_RETURN
+#undef ATOMIC_FETCH_OP_SUB
+
+#define ATOMIC_OP_ADD_SUB_RETURN(name) \
+static inline int __lse_atomic_add_return##name(int i, atomic_t *v) \
+{ \
+ return __lse_atomic_fetch_add##name(i, v) + i; \
+} \
+ \
+static inline int __lse_atomic_sub_return##name(int i, atomic_t *v) \
+{ \
+ return __lse_atomic_fetch_sub(i, v) - i; \
+}
+
+ATOMIC_OP_ADD_SUB_RETURN(_relaxed)
+ATOMIC_OP_ADD_SUB_RETURN(_acquire)
+ATOMIC_OP_ADD_SUB_RETURN(_release)
+ATOMIC_OP_ADD_SUB_RETURN( )
+
+#undef ATOMIC_OP_ADD_SUB_RETURN
static inline void __lse_atomic_and(int i, atomic_t *v)
{
- asm volatile(
- __LSE_PREAMBLE
- " mvn %w[i], %w[i]\n"
- " stclr %w[i], %[v]"
- : [i] "+&r" (i), [v] "+Q" (v->counter)
- : "r" (v));
+ return __lse_atomic_andnot(~i, v);
}
#define ATOMIC_FETCH_OP_AND(name, mb, cl...) \
static inline int __lse_atomic_fetch_and##name(int i, atomic_t *v) \
{ \
- asm volatile( \
- __LSE_PREAMBLE \
- " mvn %w[i], %w[i]\n" \
- " ldclr" #mb " %w[i], %w[i], %[v]" \
- : [i] "+&r" (i), [v] "+Q" (v->counter) \
- : "r" (v) \
- : cl); \
- \
- return i; \
+ return __lse_atomic_fetch_andnot##name(~i, v); \
}
ATOMIC_FETCH_OP_AND(_relaxed, )
@@ -108,69 +111,14 @@ ATOMIC_FETCH_OP_AND( , al, "memory")
#undef ATOMIC_FETCH_OP_AND
-static inline void __lse_atomic_sub(int i, atomic_t *v)
-{
- asm volatile(
- __LSE_PREAMBLE
- " neg %w[i], %w[i]\n"
- " stadd %w[i], %[v]"
- : [i] "+&r" (i), [v] "+Q" (v->counter)
- : "r" (v));
-}
-
-#define ATOMIC_OP_SUB_RETURN(name, mb, cl...) \
-static inline int __lse_atomic_sub_return##name(int i, atomic_t *v) \
-{ \
- u32 tmp; \
- \
- asm volatile( \
- __LSE_PREAMBLE \
- " neg %w[i], %w[i]\n" \
- " ldadd" #mb " %w[i], %w[tmp], %[v]\n" \
- " add %w[i], %w[i], %w[tmp]" \
- : [i] "+&r" (i), [v] "+Q" (v->counter), [tmp] "=&r" (tmp) \
- : "r" (v) \
- : cl); \
- \
- return i; \
-}
-
-ATOMIC_OP_SUB_RETURN(_relaxed, )
-ATOMIC_OP_SUB_RETURN(_acquire, a, "memory")
-ATOMIC_OP_SUB_RETURN(_release, l, "memory")
-ATOMIC_OP_SUB_RETURN( , al, "memory")
-
-#undef ATOMIC_OP_SUB_RETURN
-
-#define ATOMIC_FETCH_OP_SUB(name, mb, cl...) \
-static inline int __lse_atomic_fetch_sub##name(int i, atomic_t *v) \
-{ \
- asm volatile( \
- __LSE_PREAMBLE \
- " neg %w[i], %w[i]\n" \
- " ldadd" #mb " %w[i], %w[i], %[v]" \
- : [i] "+&r" (i), [v] "+Q" (v->counter) \
- : "r" (v) \
- : cl); \
- \
- return i; \
-}
-
-ATOMIC_FETCH_OP_SUB(_relaxed, )
-ATOMIC_FETCH_OP_SUB(_acquire, a, "memory")
-ATOMIC_FETCH_OP_SUB(_release, l, "memory")
-ATOMIC_FETCH_OP_SUB( , al, "memory")
-
-#undef ATOMIC_FETCH_OP_SUB
-
#define ATOMIC64_OP(op, asm_op) \
static inline void __lse_atomic64_##op(s64 i, atomic64_t *v) \
{ \
asm volatile( \
__LSE_PREAMBLE \
-" " #asm_op " %[i], %[v]\n" \
- : [i] "+r" (i), [v] "+Q" (v->counter) \
- : "r" (v)); \
+ " " #asm_op " %[i], %[v]\n" \
+ : [v] "+Q" (v->counter) \
+ : [i] "r" (i)); \
}
ATOMIC64_OP(andnot, stclr)
@@ -178,19 +126,27 @@ ATOMIC64_OP(or, stset)
ATOMIC64_OP(xor, steor)
ATOMIC64_OP(add, stadd)
+static inline void __lse_atomic64_sub(s64 i, atomic64_t *v)
+{
+ __lse_atomic64_add(-i, v);
+}
+
#undef ATOMIC64_OP
#define ATOMIC64_FETCH_OP(name, mb, op, asm_op, cl...) \
static inline long __lse_atomic64_fetch_##op##name(s64 i, atomic64_t *v)\
{ \
+ s64 old; \
+ \
asm volatile( \
__LSE_PREAMBLE \
-" " #asm_op #mb " %[i], %[i], %[v]" \
- : [i] "+r" (i), [v] "+Q" (v->counter) \
- : "r" (v) \
+ " " #asm_op #mb " %[i], %[old], %[v]" \
+ : [v] "+Q" (v->counter), \
+ [old] "=r" (old) \
+ : [i] "r" (i) \
: cl); \
\
- return i; \
+ return old; \
}
#define ATOMIC64_FETCH_OPS(op, asm_op) \
@@ -207,51 +163,46 @@ ATOMIC64_FETCH_OPS(add, ldadd)
#undef ATOMIC64_FETCH_OP
#undef ATOMIC64_FETCH_OPS
-#define ATOMIC64_OP_ADD_RETURN(name, mb, cl...) \
-static inline long __lse_atomic64_add_return##name(s64 i, atomic64_t *v)\
+#define ATOMIC64_FETCH_OP_SUB(name) \
+static inline long __lse_atomic64_fetch_sub##name(s64 i, atomic64_t *v) \
{ \
- unsigned long tmp; \
- \
- asm volatile( \
- __LSE_PREAMBLE \
- " ldadd" #mb " %[i], %x[tmp], %[v]\n" \
- " add %[i], %[i], %x[tmp]" \
- : [i] "+r" (i), [v] "+Q" (v->counter), [tmp] "=&r" (tmp) \
- : "r" (v) \
- : cl); \
- \
- return i; \
+ return __lse_atomic64_fetch_add##name(-i, v); \
}
-ATOMIC64_OP_ADD_RETURN(_relaxed, )
-ATOMIC64_OP_ADD_RETURN(_acquire, a, "memory")
-ATOMIC64_OP_ADD_RETURN(_release, l, "memory")
-ATOMIC64_OP_ADD_RETURN( , al, "memory")
+ATOMIC64_FETCH_OP_SUB(_relaxed)
+ATOMIC64_FETCH_OP_SUB(_acquire)
+ATOMIC64_FETCH_OP_SUB(_release)
+ATOMIC64_FETCH_OP_SUB( )
-#undef ATOMIC64_OP_ADD_RETURN
+#undef ATOMIC64_FETCH_OP_SUB
+
+#define ATOMIC64_OP_ADD_SUB_RETURN(name) \
+static inline long __lse_atomic64_add_return##name(s64 i, atomic64_t *v)\
+{ \
+ return __lse_atomic64_fetch_add##name(i, v) + i; \
+} \
+ \
+static inline long __lse_atomic64_sub_return##name(s64 i, atomic64_t *v)\
+{ \
+ return __lse_atomic64_fetch_sub##name(i, v) - i; \
+}
+
+ATOMIC64_OP_ADD_SUB_RETURN(_relaxed)
+ATOMIC64_OP_ADD_SUB_RETURN(_acquire)
+ATOMIC64_OP_ADD_SUB_RETURN(_release)
+ATOMIC64_OP_ADD_SUB_RETURN( )
+
+#undef ATOMIC64_OP_ADD_SUB_RETURN
static inline void __lse_atomic64_and(s64 i, atomic64_t *v)
{
- asm volatile(
- __LSE_PREAMBLE
- " mvn %[i], %[i]\n"
- " stclr %[i], %[v]"
- : [i] "+&r" (i), [v] "+Q" (v->counter)
- : "r" (v));
+ return __lse_atomic64_andnot(~i, v);
}
#define ATOMIC64_FETCH_OP_AND(name, mb, cl...) \
static inline long __lse_atomic64_fetch_and##name(s64 i, atomic64_t *v) \
{ \
- asm volatile( \
- __LSE_PREAMBLE \
- " mvn %[i], %[i]\n" \
- " ldclr" #mb " %[i], %[i], %[v]" \
- : [i] "+&r" (i), [v] "+Q" (v->counter) \
- : "r" (v) \
- : cl); \
- \
- return i; \
+ return __lse_atomic64_fetch_andnot##name(~i, v); \
}
ATOMIC64_FETCH_OP_AND(_relaxed, )
@@ -261,61 +212,6 @@ ATOMIC64_FETCH_OP_AND( , al, "memory")
#undef ATOMIC64_FETCH_OP_AND
-static inline void __lse_atomic64_sub(s64 i, atomic64_t *v)
-{
- asm volatile(
- __LSE_PREAMBLE
- " neg %[i], %[i]\n"
- " stadd %[i], %[v]"
- : [i] "+&r" (i), [v] "+Q" (v->counter)
- : "r" (v));
-}
-
-#define ATOMIC64_OP_SUB_RETURN(name, mb, cl...) \
-static inline long __lse_atomic64_sub_return##name(s64 i, atomic64_t *v) \
-{ \
- unsigned long tmp; \
- \
- asm volatile( \
- __LSE_PREAMBLE \
- " neg %[i], %[i]\n" \
- " ldadd" #mb " %[i], %x[tmp], %[v]\n" \
- " add %[i], %[i], %x[tmp]" \
- : [i] "+&r" (i), [v] "+Q" (v->counter), [tmp] "=&r" (tmp) \
- : "r" (v) \
- : cl); \
- \
- return i; \
-}
-
-ATOMIC64_OP_SUB_RETURN(_relaxed, )
-ATOMIC64_OP_SUB_RETURN(_acquire, a, "memory")
-ATOMIC64_OP_SUB_RETURN(_release, l, "memory")
-ATOMIC64_OP_SUB_RETURN( , al, "memory")
-
-#undef ATOMIC64_OP_SUB_RETURN
-
-#define ATOMIC64_FETCH_OP_SUB(name, mb, cl...) \
-static inline long __lse_atomic64_fetch_sub##name(s64 i, atomic64_t *v) \
-{ \
- asm volatile( \
- __LSE_PREAMBLE \
- " neg %[i], %[i]\n" \
- " ldadd" #mb " %[i], %[i], %[v]" \
- : [i] "+&r" (i), [v] "+Q" (v->counter) \
- : "r" (v) \
- : cl); \
- \
- return i; \
-}
-
-ATOMIC64_FETCH_OP_SUB(_relaxed, )
-ATOMIC64_FETCH_OP_SUB(_acquire, a, "memory")
-ATOMIC64_FETCH_OP_SUB(_release, l, "memory")
-ATOMIC64_FETCH_OP_SUB( , al, "memory")
-
-#undef ATOMIC64_FETCH_OP_SUB
-
static inline s64 __lse_atomic64_dec_if_positive(atomic64_t *v)
{
unsigned long tmp;
diff --git a/arch/arm64/include/asm/barrier.h b/arch/arm64/include/asm/barrier.h
index 1c5a005..62217be3 100644
--- a/arch/arm64/include/asm/barrier.h
+++ b/arch/arm64/include/asm/barrier.h
@@ -26,6 +26,14 @@
#define __tsb_csync() asm volatile("hint #18" : : : "memory")
#define csdb() asm volatile("hint #20" : : : "memory")
+/*
+ * Data Gathering Hint:
+ * This instruction prevents merging memory accesses with Normal-NC or
+ * Device-GRE attributes before the hint instruction with any memory accesses
+ * appearing after the hint instruction.
+ */
+#define dgh() asm volatile("hint #6" : : : "memory")
+
#ifdef CONFIG_ARM64_PSEUDO_NMI
#define pmr_sync() \
do { \
@@ -46,6 +54,7 @@
#define dma_rmb() dmb(oshld)
#define dma_wmb() dmb(oshst)
+#define io_stop_wc() dgh()
#define tsb_csync() \
do { \
diff --git a/arch/arm64/include/asm/cpu.h b/arch/arm64/include/asm/cpu.h
index 0f6d16f..a58e366 100644
--- a/arch/arm64/include/asm/cpu.h
+++ b/arch/arm64/include/asm/cpu.h
@@ -51,6 +51,7 @@ struct cpuinfo_arm64 {
u64 reg_id_aa64dfr1;
u64 reg_id_aa64isar0;
u64 reg_id_aa64isar1;
+ u64 reg_id_aa64isar2;
u64 reg_id_aa64mmfr0;
u64 reg_id_aa64mmfr1;
u64 reg_id_aa64mmfr2;
diff --git a/arch/arm64/include/asm/efi.h b/arch/arm64/include/asm/efi.h
index d3e1825..ad55079 100644
--- a/arch/arm64/include/asm/efi.h
+++ b/arch/arm64/include/asm/efi.h
@@ -14,7 +14,6 @@
#ifdef CONFIG_EFI
extern void efi_init(void);
-extern void efifb_setup_from_dmi(struct screen_info *si, const char *opt);
#else
#define efi_init()
#endif
diff --git a/arch/arm64/include/asm/extable.h b/arch/arm64/include/asm/extable.h
index 8b300dd..72b0e71 100644
--- a/arch/arm64/include/asm/extable.h
+++ b/arch/arm64/include/asm/extable.h
@@ -33,15 +33,6 @@ do { \
(b)->data = (tmp).data; \
} while (0)
-static inline bool in_bpf_jit(struct pt_regs *regs)
-{
- if (!IS_ENABLED(CONFIG_BPF_JIT))
- return false;
-
- return regs->pc >= BPF_JIT_REGION_START &&
- regs->pc < BPF_JIT_REGION_END;
-}
-
#ifdef CONFIG_BPF_JIT
bool ex_handler_bpf(const struct exception_table_entry *ex,
struct pt_regs *regs);
diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
index dbb4b30..cb24385 100644
--- a/arch/arm64/include/asm/fpsimd.h
+++ b/arch/arm64/include/asm/fpsimd.h
@@ -51,8 +51,8 @@ extern void fpsimd_bind_state_to_cpu(struct user_fpsimd_state *state,
extern void fpsimd_flush_task_state(struct task_struct *target);
extern void fpsimd_save_and_flush_cpu_state(void);
-/* Maximum VL that SVE VL-agnostic software can transparently support */
-#define SVE_VL_ARCH_MAX 0x100
+/* Maximum VL that SVE/SME VL-agnostic software can transparently support */
+#define VL_ARCH_MAX 0x100
/* Offset of FFR in the SVE register dump */
static inline size_t sve_ffr_offset(int vl)
@@ -122,7 +122,7 @@ extern void fpsimd_sync_to_sve(struct task_struct *task);
extern void sve_sync_to_fpsimd(struct task_struct *task);
extern void sve_sync_from_fpsimd_zeropad(struct task_struct *task);
-extern int sve_set_vector_length(struct task_struct *task,
+extern int vec_set_vector_length(struct task_struct *task, enum vec_type type,
unsigned long vl, unsigned long flags);
extern int sve_set_current_vl(unsigned long arg);
diff --git a/arch/arm64/include/asm/hwcap.h b/arch/arm64/include/asm/hwcap.h
index b100e00..f68fbb2 100644
--- a/arch/arm64/include/asm/hwcap.h
+++ b/arch/arm64/include/asm/hwcap.h
@@ -106,6 +106,8 @@
#define KERNEL_HWCAP_BTI __khwcap2_feature(BTI)
#define KERNEL_HWCAP_MTE __khwcap2_feature(MTE)
#define KERNEL_HWCAP_ECV __khwcap2_feature(ECV)
+#define KERNEL_HWCAP_AFP __khwcap2_feature(AFP)
+#define KERNEL_HWCAP_RPRES __khwcap2_feature(RPRES)
/*
* This yields a mask that user programs can use to figure out what
diff --git a/arch/arm64/include/asm/linkage.h b/arch/arm64/include/asm/linkage.h
index 9906541..b77e9b3 100644
--- a/arch/arm64/include/asm/linkage.h
+++ b/arch/arm64/include/asm/linkage.h
@@ -1,48 +1,43 @@
#ifndef __ASM_LINKAGE_H
#define __ASM_LINKAGE_H
+#ifdef __ASSEMBLY__
+#include <asm/assembler.h>
+#endif
+
#define __ALIGN .align 2
#define __ALIGN_STR ".align 2"
-#if defined(CONFIG_ARM64_BTI_KERNEL) && defined(__aarch64__)
-
/*
- * Since current versions of gas reject the BTI instruction unless we
- * set the architecture version to v8.5 we use the hint instruction
- * instead.
- */
-#define BTI_C hint 34 ;
-
-/*
- * When using in-kernel BTI we need to ensure that PCS-conformant assembly
- * functions have suitable annotations. Override SYM_FUNC_START to insert
- * a BTI landing pad at the start of everything.
+ * When using in-kernel BTI we need to ensure that PCS-conformant
+ * assembly functions have suitable annotations. Override
+ * SYM_FUNC_START to insert a BTI landing pad at the start of
+ * everything, the override is done unconditionally so we're more
+ * likely to notice any drift from the overridden definitions.
*/
#define SYM_FUNC_START(name) \
SYM_START(name, SYM_L_GLOBAL, SYM_A_ALIGN) \
- BTI_C
+ bti c ;
#define SYM_FUNC_START_NOALIGN(name) \
SYM_START(name, SYM_L_GLOBAL, SYM_A_NONE) \
- BTI_C
+ bti c ;
#define SYM_FUNC_START_LOCAL(name) \
SYM_START(name, SYM_L_LOCAL, SYM_A_ALIGN) \
- BTI_C
+ bti c ;
#define SYM_FUNC_START_LOCAL_NOALIGN(name) \
SYM_START(name, SYM_L_LOCAL, SYM_A_NONE) \
- BTI_C
+ bti c ;
#define SYM_FUNC_START_WEAK(name) \
SYM_START(name, SYM_L_WEAK, SYM_A_ALIGN) \
- BTI_C
+ bti c ;
#define SYM_FUNC_START_WEAK_NOALIGN(name) \
SYM_START(name, SYM_L_WEAK, SYM_A_NONE) \
- BTI_C
-
-#endif
+ bti c ;
/*
* Annotate a function as position independent, i.e., safe to be called before
diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h
index 1b9a1e2..0af70d9 100644
--- a/arch/arm64/include/asm/memory.h
+++ b/arch/arm64/include/asm/memory.h
@@ -44,11 +44,8 @@
#define _PAGE_OFFSET(va) (-(UL(1) << (va)))
#define PAGE_OFFSET (_PAGE_OFFSET(VA_BITS))
#define KIMAGE_VADDR (MODULES_END)
-#define BPF_JIT_REGION_START (_PAGE_END(VA_BITS_MIN))
-#define BPF_JIT_REGION_SIZE (SZ_128M)
-#define BPF_JIT_REGION_END (BPF_JIT_REGION_START + BPF_JIT_REGION_SIZE)
#define MODULES_END (MODULES_VADDR + MODULES_VSIZE)
-#define MODULES_VADDR (BPF_JIT_REGION_END)
+#define MODULES_VADDR (_PAGE_END(VA_BITS_MIN))
#define MODULES_VSIZE (SZ_128M)
#define VMEMMAP_START (-(UL(1) << (VA_BITS - VMEMMAP_SHIFT)))
#define VMEMMAP_END (VMEMMAP_START + VMEMMAP_SIZE)
diff --git a/arch/arm64/include/asm/mte-kasan.h b/arch/arm64/include/asm/mte-kasan.h
index 478b9bc..e4704a4 100644
--- a/arch/arm64/include/asm/mte-kasan.h
+++ b/arch/arm64/include/asm/mte-kasan.h
@@ -84,10 +84,12 @@ static inline void __dc_gzva(u64 p)
static inline void mte_set_mem_tag_range(void *addr, size_t size, u8 tag,
bool init)
{
- u64 curr, mask, dczid_bs, end1, end2, end3;
+ u64 curr, mask, dczid, dczid_bs, dczid_dzp, end1, end2, end3;
/* Read DC G(Z)VA block size from the system register. */
- dczid_bs = 4ul << (read_cpuid(DCZID_EL0) & 0xf);
+ dczid = read_cpuid(DCZID_EL0);
+ dczid_bs = 4ul << (dczid & 0xf);
+ dczid_dzp = (dczid >> 4) & 1;
curr = (u64)__tag_set(addr, tag);
mask = dczid_bs - 1;
@@ -106,7 +108,7 @@ static inline void mte_set_mem_tag_range(void *addr, size_t size, u8 tag,
*/
#define SET_MEMTAG_RANGE(stg_post, dc_gva) \
do { \
- if (size >= 2 * dczid_bs) { \
+ if (!dczid_dzp && size >= 2 * dczid_bs) {\
do { \
curr = stg_post(curr); \
} while (curr < end1); \
diff --git a/arch/arm64/include/asm/stacktrace.h b/arch/arm64/include/asm/stacktrace.h
index 6564a01..e77cdef 100644
--- a/arch/arm64/include/asm/stacktrace.h
+++ b/arch/arm64/include/asm/stacktrace.h
@@ -47,6 +47,10 @@ struct stack_info {
* @prev_type: The type of stack this frame record was on, or a synthetic
* value of STACK_TYPE_UNKNOWN. This is used to detect a
* transition from one stack to another.
+ *
+ * @kr_cur: When KRETPROBES is selected, holds the kretprobe instance
+ * associated with the most recently encountered replacement lr
+ * value.
*/
struct stackframe {
unsigned long fp;
@@ -59,9 +63,6 @@ struct stackframe {
#endif
};
-extern int unwind_frame(struct task_struct *tsk, struct stackframe *frame);
-extern void walk_stackframe(struct task_struct *tsk, struct stackframe *frame,
- bool (*fn)(void *, unsigned long), void *data);
extern void dump_backtrace(struct pt_regs *regs, struct task_struct *tsk,
const char *loglvl);
@@ -146,7 +147,4 @@ static inline bool on_accessible_stack(const struct task_struct *tsk,
return false;
}
-void start_backtrace(struct stackframe *frame, unsigned long fp,
- unsigned long pc);
-
#endif /* __ASM_STACKTRACE_H */
diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index 16b3f1a..4704f58 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -182,6 +182,7 @@
#define SYS_ID_AA64ISAR0_EL1 sys_reg(3, 0, 0, 6, 0)
#define SYS_ID_AA64ISAR1_EL1 sys_reg(3, 0, 0, 6, 1)
+#define SYS_ID_AA64ISAR2_EL1 sys_reg(3, 0, 0, 6, 2)
#define SYS_ID_AA64MMFR0_EL1 sys_reg(3, 0, 0, 7, 0)
#define SYS_ID_AA64MMFR1_EL1 sys_reg(3, 0, 0, 7, 1)
@@ -771,6 +772,20 @@
#define ID_AA64ISAR1_GPI_NI 0x0
#define ID_AA64ISAR1_GPI_IMP_DEF 0x1
+/* id_aa64isar2 */
+#define ID_AA64ISAR2_RPRES_SHIFT 4
+#define ID_AA64ISAR2_WFXT_SHIFT 0
+
+#define ID_AA64ISAR2_RPRES_8BIT 0x0
+#define ID_AA64ISAR2_RPRES_12BIT 0x1
+/*
+ * Value 0x1 has been removed from the architecture, and is
+ * reserved, but has not yet been removed from the ARM ARM
+ * as of ARM DDI 0487G.b.
+ */
+#define ID_AA64ISAR2_WFXT_NI 0x0
+#define ID_AA64ISAR2_WFXT_SUPPORTED 0x2
+
/* id_aa64pfr0 */
#define ID_AA64PFR0_CSV3_SHIFT 60
#define ID_AA64PFR0_CSV2_SHIFT 56
@@ -889,6 +904,7 @@
#endif
/* id_aa64mmfr1 */
+#define ID_AA64MMFR1_AFP_SHIFT 44
#define ID_AA64MMFR1_ETS_SHIFT 36
#define ID_AA64MMFR1_TWED_SHIFT 32
#define ID_AA64MMFR1_XNX_SHIFT 28
diff --git a/arch/arm64/include/uapi/asm/hwcap.h b/arch/arm64/include/uapi/asm/hwcap.h
index 7b23b16..f037318 100644
--- a/arch/arm64/include/uapi/asm/hwcap.h
+++ b/arch/arm64/include/uapi/asm/hwcap.h
@@ -76,5 +76,7 @@
#define HWCAP2_BTI (1 << 17)
#define HWCAP2_MTE (1 << 18)
#define HWCAP2_ECV (1 << 19)
+#define HWCAP2_AFP (1 << 20)
+#define HWCAP2_RPRES (1 << 21)
#endif /* _UAPI__ASM_HWCAP_H */
diff --git a/arch/arm64/kernel/acpi.c b/arch/arm64/kernel/acpi.c
index f385172..e4dea8d 100644
--- a/arch/arm64/kernel/acpi.c
+++ b/arch/arm64/kernel/acpi.c
@@ -22,6 +22,7 @@
#include <linux/irq_work.h>
#include <linux/memblock.h>
#include <linux/of_fdt.h>
+#include <linux/libfdt.h>
#include <linux/smp.h>
#include <linux/serial_core.h>
#include <linux/pgtable.h>
@@ -62,29 +63,22 @@ static int __init parse_acpi(char *arg)
}
early_param("acpi", parse_acpi);
-static int __init dt_scan_depth1_nodes(unsigned long node,
- const char *uname, int depth,
- void *data)
+static bool __init dt_is_stub(void)
{
- /*
- * Ignore anything not directly under the root node; we'll
- * catch its parent instead.
- */
- if (depth != 1)
- return 0;
+ int node;
- if (strcmp(uname, "chosen") == 0)
- return 0;
+ fdt_for_each_subnode(node, initial_boot_params, 0) {
+ const char *name = fdt_get_name(initial_boot_params, node, NULL);
+ if (strcmp(name, "chosen") == 0)
+ continue;
+ if (strcmp(name, "hypervisor") == 0 &&
+ of_flat_dt_is_compatible(node, "xen,xen"))
+ continue;
- if (strcmp(uname, "hypervisor") == 0 &&
- of_flat_dt_is_compatible(node, "xen,xen"))
- return 0;
+ return false;
+ }
- /*
- * This node at depth 1 is neither a chosen node nor a xen node,
- * which we do not expect.
- */
- return 1;
+ return true;
}
/*
@@ -205,8 +199,7 @@ void __init acpi_boot_table_init(void)
* and ACPI has not been [force] enabled (acpi=on|force)
*/
if (param_acpi_off ||
- (!param_acpi_on && !param_acpi_force &&
- of_scan_flat_dt(dt_scan_depth1_nodes, NULL)))
+ (!param_acpi_on && !param_acpi_force && !dt_is_stub()))
goto done;
/*
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 6f3e677..a46ab3b 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -225,6 +225,11 @@ static const struct arm64_ftr_bits ftr_id_aa64isar1[] = {
ARM64_FTR_END,
};
+static const struct arm64_ftr_bits ftr_id_aa64isar2[] = {
+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64ISAR2_RPRES_SHIFT, 4, 0),
+ ARM64_FTR_END,
+};
+
static const struct arm64_ftr_bits ftr_id_aa64pfr0[] = {
ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_CSV3_SHIFT, 4, 0),
ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_CSV2_SHIFT, 4, 0),
@@ -325,6 +330,7 @@ static const struct arm64_ftr_bits ftr_id_aa64mmfr0[] = {
};
static const struct arm64_ftr_bits ftr_id_aa64mmfr1[] = {
+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_AFP_SHIFT, 4, 0),
ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_ETS_SHIFT, 4, 0),
ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_TWED_SHIFT, 4, 0),
ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_XNX_SHIFT, 4, 0),
@@ -637,6 +643,7 @@ static const struct __ftr_reg_entry {
ARM64_FTR_REG(SYS_ID_AA64ISAR0_EL1, ftr_id_aa64isar0),
ARM64_FTR_REG_OVERRIDE(SYS_ID_AA64ISAR1_EL1, ftr_id_aa64isar1,
&id_aa64isar1_override),
+ ARM64_FTR_REG(SYS_ID_AA64ISAR2_EL1, ftr_id_aa64isar2),
/* Op1 = 0, CRn = 0, CRm = 7 */
ARM64_FTR_REG(SYS_ID_AA64MMFR0_EL1, ftr_id_aa64mmfr0),
@@ -933,6 +940,7 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)
init_cpu_ftr_reg(SYS_ID_AA64DFR1_EL1, info->reg_id_aa64dfr1);
init_cpu_ftr_reg(SYS_ID_AA64ISAR0_EL1, info->reg_id_aa64isar0);
init_cpu_ftr_reg(SYS_ID_AA64ISAR1_EL1, info->reg_id_aa64isar1);
+ init_cpu_ftr_reg(SYS_ID_AA64ISAR2_EL1, info->reg_id_aa64isar2);
init_cpu_ftr_reg(SYS_ID_AA64MMFR0_EL1, info->reg_id_aa64mmfr0);
init_cpu_ftr_reg(SYS_ID_AA64MMFR1_EL1, info->reg_id_aa64mmfr1);
init_cpu_ftr_reg(SYS_ID_AA64MMFR2_EL1, info->reg_id_aa64mmfr2);
@@ -1151,6 +1159,8 @@ void update_cpu_features(int cpu,
info->reg_id_aa64isar0, boot->reg_id_aa64isar0);
taint |= check_update_ftr_reg(SYS_ID_AA64ISAR1_EL1, cpu,
info->reg_id_aa64isar1, boot->reg_id_aa64isar1);
+ taint |= check_update_ftr_reg(SYS_ID_AA64ISAR2_EL1, cpu,
+ info->reg_id_aa64isar2, boot->reg_id_aa64isar2);
/*
* Differing PARange support is fine as long as all peripherals and
@@ -1272,6 +1282,7 @@ u64 __read_sysreg_by_encoding(u32 sys_id)
read_sysreg_case(SYS_ID_AA64MMFR2_EL1);
read_sysreg_case(SYS_ID_AA64ISAR0_EL1);
read_sysreg_case(SYS_ID_AA64ISAR1_EL1);
+ read_sysreg_case(SYS_ID_AA64ISAR2_EL1);
read_sysreg_case(SYS_CNTFRQ_EL0);
read_sysreg_case(SYS_CTR_EL0);
@@ -2476,6 +2487,8 @@ static const struct arm64_cpu_capabilities arm64_elf_hwcaps[] = {
HWCAP_CAP(SYS_ID_AA64PFR1_EL1, ID_AA64PFR1_MTE_SHIFT, FTR_UNSIGNED, ID_AA64PFR1_MTE, CAP_HWCAP, KERNEL_HWCAP_MTE),
#endif /* CONFIG_ARM64_MTE */
HWCAP_CAP(SYS_ID_AA64MMFR0_EL1, ID_AA64MMFR0_ECV_SHIFT, FTR_UNSIGNED, 1, CAP_HWCAP, KERNEL_HWCAP_ECV),
+ HWCAP_CAP(SYS_ID_AA64MMFR1_EL1, ID_AA64MMFR1_AFP_SHIFT, FTR_UNSIGNED, 1, CAP_HWCAP, KERNEL_HWCAP_AFP),
+ HWCAP_CAP(SYS_ID_AA64ISAR2_EL1, ID_AA64ISAR2_RPRES_SHIFT, FTR_UNSIGNED, 1, CAP_HWCAP, KERNEL_HWCAP_RPRES),
{},
};
diff --git a/arch/arm64/kernel/cpuinfo.c b/arch/arm64/kernel/cpuinfo.c
index 6e27b75..591c18a 100644
--- a/arch/arm64/kernel/cpuinfo.c
+++ b/arch/arm64/kernel/cpuinfo.c
@@ -95,6 +95,8 @@ static const char *const hwcap_str[] = {
[KERNEL_HWCAP_BTI] = "bti",
[KERNEL_HWCAP_MTE] = "mte",
[KERNEL_HWCAP_ECV] = "ecv",
+ [KERNEL_HWCAP_AFP] = "afp",
+ [KERNEL_HWCAP_RPRES] = "rpres",
};
#ifdef CONFIG_COMPAT
@@ -391,6 +393,7 @@ static void __cpuinfo_store_cpu(struct cpuinfo_arm64 *info)
info->reg_id_aa64dfr1 = read_cpuid(ID_AA64DFR1_EL1);
info->reg_id_aa64isar0 = read_cpuid(ID_AA64ISAR0_EL1);
info->reg_id_aa64isar1 = read_cpuid(ID_AA64ISAR1_EL1);
+ info->reg_id_aa64isar2 = read_cpuid(ID_AA64ISAR2_EL1);
info->reg_id_aa64mmfr0 = read_cpuid(ID_AA64MMFR0_EL1);
info->reg_id_aa64mmfr1 = read_cpuid(ID_AA64MMFR1_EL1);
info->reg_id_aa64mmfr2 = read_cpuid(ID_AA64MMFR2_EL1);
diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c
index f7408edf..ef7fcef 100644
--- a/arch/arm64/kernel/entry-common.c
+++ b/arch/arm64/kernel/entry-common.c
@@ -129,7 +129,7 @@ static __always_inline void prepare_exit_to_user_mode(struct pt_regs *regs)
local_daif_mask();
- flags = READ_ONCE(current_thread_info()->flags);
+ flags = read_thread_flags();
if (unlikely(flags & _TIF_WORK_MASK))
do_notify_resume(regs, flags);
}
diff --git a/arch/arm64/kernel/entry-ftrace.S b/arch/arm64/kernel/entry-ftrace.S
index 8cf970d..e535480 100644
--- a/arch/arm64/kernel/entry-ftrace.S
+++ b/arch/arm64/kernel/entry-ftrace.S
@@ -77,17 +77,13 @@
.endm
SYM_CODE_START(ftrace_regs_caller)
-#ifdef BTI_C
- BTI_C
-#endif
+ bti c
ftrace_regs_entry 1
b ftrace_common
SYM_CODE_END(ftrace_regs_caller)
SYM_CODE_START(ftrace_caller)
-#ifdef BTI_C
- BTI_C
-#endif
+ bti c
ftrace_regs_entry 0
b ftrace_common
SYM_CODE_END(ftrace_caller)
diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
index 2f69ae4..772ec2ec 100644
--- a/arch/arm64/kernel/entry.S
+++ b/arch/arm64/kernel/entry.S
@@ -966,8 +966,10 @@
mov sp, x1
mov x1, x0 // address to complete_and_resume
- /* x0 = (x0 <= 1) ? EVENT_COMPLETE:EVENT_COMPLETE_AND_RESUME */
- cmp x0, #1
+ /* x0 = (x0 <= SDEI_EV_FAILED) ?
+ * EVENT_COMPLETE:EVENT_COMPLETE_AND_RESUME
+ */
+ cmp x0, #SDEI_EV_FAILED
mov_q x2, SDEI_1_0_FN_SDEI_EVENT_COMPLETE
mov_q x3, SDEI_1_0_FN_SDEI_EVENT_COMPLETE_AND_RESUME
csel x0, x2, x3, ls
diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
index fa244c4..f2307d6 100644
--- a/arch/arm64/kernel/fpsimd.c
+++ b/arch/arm64/kernel/fpsimd.c
@@ -15,6 +15,7 @@
#include <linux/compiler.h>
#include <linux/cpu.h>
#include <linux/cpu_pm.h>
+#include <linux/ctype.h>
#include <linux/kernel.h>
#include <linux/linkage.h>
#include <linux/irqflags.h>
@@ -406,12 +407,13 @@ static unsigned int find_supported_vector_length(enum vec_type type,
#if defined(CONFIG_ARM64_SVE) && defined(CONFIG_SYSCTL)
-static int sve_proc_do_default_vl(struct ctl_table *table, int write,
+static int vec_proc_do_default_vl(struct ctl_table *table, int write,
void *buffer, size_t *lenp, loff_t *ppos)
{
- struct vl_info *info = &vl_info[ARM64_VEC_SVE];
+ struct vl_info *info = table->extra1;
+ enum vec_type type = info->type;
int ret;
- int vl = get_sve_default_vl();
+ int vl = get_default_vl(type);
struct ctl_table tmp_table = {
.data = &vl,
.maxlen = sizeof(vl),
@@ -428,7 +430,7 @@ static int sve_proc_do_default_vl(struct ctl_table *table, int write,
if (!sve_vl_valid(vl))
return -EINVAL;
- set_sve_default_vl(find_supported_vector_length(ARM64_VEC_SVE, vl));
+ set_default_vl(type, find_supported_vector_length(type, vl));
return 0;
}
@@ -436,7 +438,8 @@ static struct ctl_table sve_default_vl_table[] = {
{
.procname = "sve_default_vector_length",
.mode = 0644,
- .proc_handler = sve_proc_do_default_vl,
+ .proc_handler = vec_proc_do_default_vl,
+ .extra1 = &vl_info[ARM64_VEC_SVE],
},
{ }
};
@@ -629,7 +632,7 @@ void sve_sync_from_fpsimd_zeropad(struct task_struct *task)
__fpsimd_to_sve(sst, fst, vq);
}
-int sve_set_vector_length(struct task_struct *task,
+int vec_set_vector_length(struct task_struct *task, enum vec_type type,
unsigned long vl, unsigned long flags)
{
if (flags & ~(unsigned long)(PR_SVE_VL_INHERIT |
@@ -640,33 +643,35 @@ int sve_set_vector_length(struct task_struct *task,
return -EINVAL;
/*
- * Clamp to the maximum vector length that VL-agnostic SVE code can
- * work with. A flag may be assigned in the future to allow setting
- * of larger vector lengths without confusing older software.
+ * Clamp to the maximum vector length that VL-agnostic code
+ * can work with. A flag may be assigned in the future to
+ * allow setting of larger vector lengths without confusing
+ * older software.
*/
- if (vl > SVE_VL_ARCH_MAX)
- vl = SVE_VL_ARCH_MAX;
+ if (vl > VL_ARCH_MAX)
+ vl = VL_ARCH_MAX;
- vl = find_supported_vector_length(ARM64_VEC_SVE, vl);
+ vl = find_supported_vector_length(type, vl);
if (flags & (PR_SVE_VL_INHERIT |
PR_SVE_SET_VL_ONEXEC))
- task_set_sve_vl_onexec(task, vl);
+ task_set_vl_onexec(task, type, vl);
else
/* Reset VL to system default on next exec: */
- task_set_sve_vl_onexec(task, 0);
+ task_set_vl_onexec(task, type, 0);
/* Only actually set the VL if not deferred: */
if (flags & PR_SVE_SET_VL_ONEXEC)
goto out;
- if (vl == task_get_sve_vl(task))
+ if (vl == task_get_vl(task, type))
goto out;
/*
* To ensure the FPSIMD bits of the SVE vector registers are preserved,
* write any live register state back to task_struct, and convert to a
- * non-SVE thread.
+ * regular FPSIMD thread. Since the vector length can only be changed
+ * with a syscall we can't be in streaming mode while reconfiguring.
*/
if (task == current) {
get_cpu_fpsimd_context();
@@ -687,10 +692,10 @@ int sve_set_vector_length(struct task_struct *task,
*/
sve_free(task);
- task_set_sve_vl(task, vl);
+ task_set_vl(task, type, vl);
out:
- update_tsk_thread_flag(task, TIF_SVE_VL_INHERIT,
+ update_tsk_thread_flag(task, vec_vl_inherit_flag(type),
flags & PR_SVE_VL_INHERIT);
return 0;
@@ -698,20 +703,21 @@ int sve_set_vector_length(struct task_struct *task,
/*
* Encode the current vector length and flags for return.
- * This is only required for prctl(): ptrace has separate fields
+ * This is only required for prctl(): ptrace has separate fields.
+ * SVE and SME use the same bits for _ONEXEC and _INHERIT.
*
- * flags are as for sve_set_vector_length().
+ * flags are as for vec_set_vector_length().
*/
-static int sve_prctl_status(unsigned long flags)
+static int vec_prctl_status(enum vec_type type, unsigned long flags)
{
int ret;
if (flags & PR_SVE_SET_VL_ONEXEC)
- ret = task_get_sve_vl_onexec(current);
+ ret = task_get_vl_onexec(current, type);
else
- ret = task_get_sve_vl(current);
+ ret = task_get_vl(current, type);
- if (test_thread_flag(TIF_SVE_VL_INHERIT))
+ if (test_thread_flag(vec_vl_inherit_flag(type)))
ret |= PR_SVE_VL_INHERIT;
return ret;
@@ -729,11 +735,11 @@ int sve_set_current_vl(unsigned long arg)
if (!system_supports_sve() || is_compat_task())
return -EINVAL;
- ret = sve_set_vector_length(current, vl, flags);
+ ret = vec_set_vector_length(current, ARM64_VEC_SVE, vl, flags);
if (ret)
return ret;
- return sve_prctl_status(flags);
+ return vec_prctl_status(ARM64_VEC_SVE, flags);
}
/* PR_SVE_GET_VL */
@@ -742,7 +748,7 @@ int sve_get_current_vl(void)
if (!system_supports_sve() || is_compat_task())
return -EINVAL;
- return sve_prctl_status(0);
+ return vec_prctl_status(ARM64_VEC_SVE, 0);
}
static void vec_probe_vqs(struct vl_info *info,
@@ -1107,7 +1113,7 @@ static void fpsimd_flush_thread_vl(enum vec_type type)
vl = get_default_vl(type);
if (WARN_ON(!sve_vl_valid(vl)))
- vl = SVE_VL_MIN;
+ vl = vl_info[type].min_vl;
supported_vl = find_supported_vector_length(type, vl);
if (WARN_ON(supported_vl != vl))
@@ -1213,7 +1219,8 @@ void fpsimd_bind_state_to_cpu(struct user_fpsimd_state *st, void *sve_state,
/*
* Load the userland FPSIMD state of 'current' from memory, but only if the
* FPSIMD state already held in the registers is /not/ the most recent FPSIMD
- * state of 'current'
+ * state of 'current'. This is called when we are preparing to return to
+ * userspace to ensure that userspace sees a good register state.
*/
void fpsimd_restore_current_state(void)
{
@@ -1244,7 +1251,9 @@ void fpsimd_restore_current_state(void)
/*
* Load an updated userland FPSIMD state for 'current' from memory and set the
* flag that indicates that the FPSIMD register contents are the most recent
- * FPSIMD state of 'current'
+ * FPSIMD state of 'current'. This is used by the signal code to restore the
+ * register state when returning from a signal handler in FPSIMD only cases,
+ * any SVE context will be discarded.
*/
void fpsimd_update_current_state(struct user_fpsimd_state const *state)
{
diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c
index 2758f75..6328308 100644
--- a/arch/arm64/kernel/hibernate.c
+++ b/arch/arm64/kernel/hibernate.c
@@ -7,10 +7,6 @@
* Ubuntu project, hibernation support for mach-dove
* Copyright (C) 2010 Nokia Corporation (Hiroshi Doyu)
* Copyright (C) 2010 Texas Instruments, Inc. (Teerth Reddy et al.)
- * https://lkml.org/lkml/2010/6/18/4
- * https://lists.linux-foundation.org/pipermail/linux-pm/2010-June/027422.html
- * https://patchwork.kernel.org/patch/96442/
- *
* Copyright (C) 2006 Rafael J. Wysocki <rjw@sisk.pl>
*/
#define pr_fmt(x) "hibernate: " x
diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c
index 6fb31c1..e16b248 100644
--- a/arch/arm64/kernel/machine_kexec.c
+++ b/arch/arm64/kernel/machine_kexec.c
@@ -104,13 +104,15 @@ static void *kexec_page_alloc(void *arg)
{
struct kimage *kimage = (struct kimage *)arg;
struct page *page = kimage_alloc_control_pages(kimage, 0);
+ void *vaddr = NULL;
if (!page)
return NULL;
- memset(page_address(page), 0, PAGE_SIZE);
+ vaddr = page_address(page);
+ memset(vaddr, 0, PAGE_SIZE);
- return page_address(page);
+ return vaddr;
}
int machine_kexec_post_load(struct kimage *kimage)
diff --git a/arch/arm64/kernel/perf_callchain.c b/arch/arm64/kernel/perf_callchain.c
index 4a72c27..e9b7d99 100644
--- a/arch/arm64/kernel/perf_callchain.c
+++ b/arch/arm64/kernel/perf_callchain.c
@@ -5,10 +5,10 @@
* Copyright (C) 2015 ARM Limited
*/
#include <linux/perf_event.h>
+#include <linux/stacktrace.h>
#include <linux/uaccess.h>
#include <asm/pointer_auth.h>
-#include <asm/stacktrace.h>
struct frame_tail {
struct frame_tail __user *fp;
@@ -132,30 +132,21 @@ void perf_callchain_user(struct perf_callchain_entry_ctx *entry,
}
}
-/*
- * Gets called by walk_stackframe() for every stackframe. This will be called
- * whist unwinding the stackframe and is like a subroutine return so we use
- * the PC.
- */
static bool callchain_trace(void *data, unsigned long pc)
{
struct perf_callchain_entry_ctx *entry = data;
- perf_callchain_store(entry, pc);
- return true;
+ return perf_callchain_store(entry, pc) == 0;
}
void perf_callchain_kernel(struct perf_callchain_entry_ctx *entry,
struct pt_regs *regs)
{
- struct stackframe frame;
-
if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) {
/* We don't support guest os callchain now */
return;
}
- start_backtrace(&frame, regs->regs[29], regs->pc);
- walk_stackframe(current, &frame, callchain_trace, entry);
+ arch_stack_walk(callchain_trace, entry, current, regs);
}
unsigned long perf_instruction_pointer(struct pt_regs *regs)
diff --git a/arch/arm64/kernel/perf_event.c b/arch/arm64/kernel/perf_event.c
index b404446..cab678e 100644
--- a/arch/arm64/kernel/perf_event.c
+++ b/arch/arm64/kernel/perf_event.c
@@ -285,15 +285,24 @@ static const struct attribute_group armv8_pmuv3_events_attr_group = {
PMU_FORMAT_ATTR(event, "config:0-15");
PMU_FORMAT_ATTR(long, "config1:0");
+PMU_FORMAT_ATTR(rdpmc, "config1:1");
+
+static int sysctl_perf_user_access __read_mostly;
static inline bool armv8pmu_event_is_64bit(struct perf_event *event)
{
return event->attr.config1 & 0x1;
}
+static inline bool armv8pmu_event_want_user_access(struct perf_event *event)
+{
+ return event->attr.config1 & 0x2;
+}
+
static struct attribute *armv8_pmuv3_format_attrs[] = {
&format_attr_event.attr,
&format_attr_long.attr,
+ &format_attr_rdpmc.attr,
NULL,
};
@@ -362,7 +371,7 @@ static const struct attribute_group armv8_pmuv3_caps_attr_group = {
*/
#define ARMV8_IDX_CYCLE_COUNTER 0
#define ARMV8_IDX_COUNTER0 1
-
+#define ARMV8_IDX_CYCLE_COUNTER_USER 32
/*
* We unconditionally enable ARMv8.5-PMU long event counter support
@@ -374,18 +383,22 @@ static bool armv8pmu_has_long_event(struct arm_pmu *cpu_pmu)
return (cpu_pmu->pmuver >= ID_AA64DFR0_PMUVER_8_5);
}
+static inline bool armv8pmu_event_has_user_read(struct perf_event *event)
+{
+ return event->hw.flags & PERF_EVENT_FLAG_USER_READ_CNT;
+}
+
/*
* We must chain two programmable counters for 64 bit events,
* except when we have allocated the 64bit cycle counter (for CPU
- * cycles event). This must be called only when the event has
- * a counter allocated.
+ * cycles event) or when user space counter access is enabled.
*/
static inline bool armv8pmu_event_is_chained(struct perf_event *event)
{
int idx = event->hw.idx;
struct arm_pmu *cpu_pmu = to_arm_pmu(event->pmu);
- return !WARN_ON(idx < 0) &&
+ return !armv8pmu_event_has_user_read(event) &&
armv8pmu_event_is_64bit(event) &&
!armv8pmu_has_long_event(cpu_pmu) &&
(idx != ARMV8_IDX_CYCLE_COUNTER);
@@ -718,6 +731,28 @@ static inline u32 armv8pmu_getreset_flags(void)
return value;
}
+static void armv8pmu_disable_user_access(void)
+{
+ write_sysreg(0, pmuserenr_el0);
+}
+
+static void armv8pmu_enable_user_access(struct arm_pmu *cpu_pmu)
+{
+ int i;
+ struct pmu_hw_events *cpuc = this_cpu_ptr(cpu_pmu->hw_events);
+
+ /* Clear any unused counters to avoid leaking their contents */
+ for_each_clear_bit(i, cpuc->used_mask, cpu_pmu->num_events) {
+ if (i == ARMV8_IDX_CYCLE_COUNTER)
+ write_sysreg(0, pmccntr_el0);
+ else
+ armv8pmu_write_evcntr(i, 0);
+ }
+
+ write_sysreg(0, pmuserenr_el0);
+ write_sysreg(ARMV8_PMU_USERENR_ER | ARMV8_PMU_USERENR_CR, pmuserenr_el0);
+}
+
static void armv8pmu_enable_event(struct perf_event *event)
{
/*
@@ -761,6 +796,14 @@ static void armv8pmu_disable_event(struct perf_event *event)
static void armv8pmu_start(struct arm_pmu *cpu_pmu)
{
+ struct perf_event_context *task_ctx =
+ this_cpu_ptr(cpu_pmu->pmu.pmu_cpu_context)->task_ctx;
+
+ if (sysctl_perf_user_access && task_ctx && task_ctx->nr_user)
+ armv8pmu_enable_user_access(cpu_pmu);
+ else
+ armv8pmu_disable_user_access();
+
/* Enable all counters */
armv8pmu_pmcr_write(armv8pmu_pmcr_read() | ARMV8_PMU_PMCR_E);
}
@@ -878,13 +921,16 @@ static int armv8pmu_get_event_idx(struct pmu_hw_events *cpuc,
if (evtype == ARMV8_PMUV3_PERFCTR_CPU_CYCLES) {
if (!test_and_set_bit(ARMV8_IDX_CYCLE_COUNTER, cpuc->used_mask))
return ARMV8_IDX_CYCLE_COUNTER;
+ else if (armv8pmu_event_is_64bit(event) &&
+ armv8pmu_event_want_user_access(event) &&
+ !armv8pmu_has_long_event(cpu_pmu))
+ return -EAGAIN;
}
/*
* Otherwise use events counters
*/
- if (armv8pmu_event_is_64bit(event) &&
- !armv8pmu_has_long_event(cpu_pmu))
+ if (armv8pmu_event_is_chained(event))
return armv8pmu_get_chain_idx(cpuc, cpu_pmu);
else
return armv8pmu_get_single_idx(cpuc, cpu_pmu);
@@ -900,6 +946,22 @@ static void armv8pmu_clear_event_idx(struct pmu_hw_events *cpuc,
clear_bit(idx - 1, cpuc->used_mask);
}
+static int armv8pmu_user_event_idx(struct perf_event *event)
+{
+ if (!sysctl_perf_user_access || !armv8pmu_event_has_user_read(event))
+ return 0;
+
+ /*
+ * We remap the cycle counter index to 32 to
+ * match the offset applied to the rest of
+ * the counter indices.
+ */
+ if (event->hw.idx == ARMV8_IDX_CYCLE_COUNTER)
+ return ARMV8_IDX_CYCLE_COUNTER_USER;
+
+ return event->hw.idx;
+}
+
/*
* Add an event filter to a given event.
*/
@@ -996,6 +1058,25 @@ static int __armv8_pmuv3_map_event(struct perf_event *event,
if (armv8pmu_event_is_64bit(event))
event->hw.flags |= ARMPMU_EVT_64BIT;
+ /*
+ * User events must be allocated into a single counter, and so
+ * must not be chained.
+ *
+ * Most 64-bit events require long counter support, but 64-bit
+ * CPU_CYCLES events can be placed into the dedicated cycle
+ * counter when this is free.
+ */
+ if (armv8pmu_event_want_user_access(event)) {
+ if (!(event->attach_state & PERF_ATTACH_TASK))
+ return -EINVAL;
+ if (armv8pmu_event_is_64bit(event) &&
+ (hw_event_id != ARMV8_PMUV3_PERFCTR_CPU_CYCLES) &&
+ !armv8pmu_has_long_event(armpmu))
+ return -EOPNOTSUPP;
+
+ event->hw.flags |= PERF_EVENT_FLAG_USER_READ_CNT;
+ }
+
/* Only expose micro/arch events supported by this PMU */
if ((hw_event_id > 0) && (hw_event_id < ARMV8_PMUV3_MAX_COMMON_EVENTS)
&& test_bit(hw_event_id, armpmu->pmceid_bitmap)) {
@@ -1104,6 +1185,43 @@ static int armv8pmu_probe_pmu(struct arm_pmu *cpu_pmu)
return probe.present ? 0 : -ENODEV;
}
+static void armv8pmu_disable_user_access_ipi(void *unused)
+{
+ armv8pmu_disable_user_access();
+}
+
+static int armv8pmu_proc_user_access_handler(struct ctl_table *table, int write,
+ void *buffer, size_t *lenp, loff_t *ppos)
+{
+ int ret = proc_dointvec_minmax(table, write, buffer, lenp, ppos);
+ if (ret || !write || sysctl_perf_user_access)
+ return ret;
+
+ on_each_cpu(armv8pmu_disable_user_access_ipi, NULL, 1);
+ return 0;
+}
+
+static struct ctl_table armv8_pmu_sysctl_table[] = {
+ {
+ .procname = "perf_user_access",
+ .data = &sysctl_perf_user_access,
+ .maxlen = sizeof(unsigned int),
+ .mode = 0644,
+ .proc_handler = armv8pmu_proc_user_access_handler,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
+ },
+ { }
+};
+
+static void armv8_pmu_register_sysctl_table(void)
+{
+ static u32 tbl_registered = 0;
+
+ if (!cmpxchg_relaxed(&tbl_registered, 0, 1))
+ register_sysctl("kernel", armv8_pmu_sysctl_table);
+}
+
static int armv8_pmu_init(struct arm_pmu *cpu_pmu, char *name,
int (*map_event)(struct perf_event *event),
const struct attribute_group *events,
@@ -1127,6 +1245,8 @@ static int armv8_pmu_init(struct arm_pmu *cpu_pmu, char *name,
cpu_pmu->set_event_filter = armv8pmu_set_event_filter;
cpu_pmu->filter_match = armv8pmu_filter_match;
+ cpu_pmu->pmu.event_idx = armv8pmu_user_event_idx;
+
cpu_pmu->name = name;
cpu_pmu->map_event = map_event;
cpu_pmu->attr_groups[ARMPMU_ATTR_GROUP_EVENTS] = events ?
@@ -1136,6 +1256,7 @@ static int armv8_pmu_init(struct arm_pmu *cpu_pmu, char *name,
cpu_pmu->attr_groups[ARMPMU_ATTR_GROUP_CAPS] = caps ?
caps : &armv8_pmuv3_caps_attr_group;
+ armv8_pmu_register_sysctl_table();
return 0;
}
@@ -1145,17 +1266,32 @@ static int armv8_pmu_init_nogroups(struct arm_pmu *cpu_pmu, char *name,
return armv8_pmu_init(cpu_pmu, name, map_event, NULL, NULL, NULL);
}
-static int armv8_pmuv3_init(struct arm_pmu *cpu_pmu)
-{
- return armv8_pmu_init_nogroups(cpu_pmu, "armv8_pmuv3",
- armv8_pmuv3_map_event);
+#define PMUV3_INIT_SIMPLE(name) \
+static int name##_pmu_init(struct arm_pmu *cpu_pmu) \
+{ \
+ return armv8_pmu_init_nogroups(cpu_pmu, #name, armv8_pmuv3_map_event);\
}
-static int armv8_a34_pmu_init(struct arm_pmu *cpu_pmu)
-{
- return armv8_pmu_init_nogroups(cpu_pmu, "armv8_cortex_a34",
- armv8_pmuv3_map_event);
-}
+PMUV3_INIT_SIMPLE(armv8_pmuv3)
+
+PMUV3_INIT_SIMPLE(armv8_cortex_a34)
+PMUV3_INIT_SIMPLE(armv8_cortex_a55)
+PMUV3_INIT_SIMPLE(armv8_cortex_a65)
+PMUV3_INIT_SIMPLE(armv8_cortex_a75)
+PMUV3_INIT_SIMPLE(armv8_cortex_a76)
+PMUV3_INIT_SIMPLE(armv8_cortex_a77)
+PMUV3_INIT_SIMPLE(armv8_cortex_a78)
+PMUV3_INIT_SIMPLE(armv9_cortex_a510)
+PMUV3_INIT_SIMPLE(armv9_cortex_a710)
+PMUV3_INIT_SIMPLE(armv8_cortex_x1)
+PMUV3_INIT_SIMPLE(armv9_cortex_x2)
+PMUV3_INIT_SIMPLE(armv8_neoverse_e1)
+PMUV3_INIT_SIMPLE(armv8_neoverse_n1)
+PMUV3_INIT_SIMPLE(armv9_neoverse_n2)
+PMUV3_INIT_SIMPLE(armv8_neoverse_v1)
+
+PMUV3_INIT_SIMPLE(armv8_nvidia_carmel)
+PMUV3_INIT_SIMPLE(armv8_nvidia_denver)
static int armv8_a35_pmu_init(struct arm_pmu *cpu_pmu)
{
@@ -1169,24 +1305,12 @@ static int armv8_a53_pmu_init(struct arm_pmu *cpu_pmu)
armv8_a53_map_event);
}
-static int armv8_a55_pmu_init(struct arm_pmu *cpu_pmu)
-{
- return armv8_pmu_init_nogroups(cpu_pmu, "armv8_cortex_a55",
- armv8_pmuv3_map_event);
-}
-
static int armv8_a57_pmu_init(struct arm_pmu *cpu_pmu)
{
return armv8_pmu_init_nogroups(cpu_pmu, "armv8_cortex_a57",
armv8_a57_map_event);
}
-static int armv8_a65_pmu_init(struct arm_pmu *cpu_pmu)
-{
- return armv8_pmu_init_nogroups(cpu_pmu, "armv8_cortex_a65",
- armv8_pmuv3_map_event);
-}
-
static int armv8_a72_pmu_init(struct arm_pmu *cpu_pmu)
{
return armv8_pmu_init_nogroups(cpu_pmu, "armv8_cortex_a72",
@@ -1199,42 +1323,6 @@ static int armv8_a73_pmu_init(struct arm_pmu *cpu_pmu)
armv8_a73_map_event);
}
-static int armv8_a75_pmu_init(struct arm_pmu *cpu_pmu)
-{
- return armv8_pmu_init_nogroups(cpu_pmu, "armv8_cortex_a75",
- armv8_pmuv3_map_event);
-}
-
-static int armv8_a76_pmu_init(struct arm_pmu *cpu_pmu)
-{
- return armv8_pmu_init_nogroups(cpu_pmu, "armv8_cortex_a76",
- armv8_pmuv3_map_event);
-}
-
-static int armv8_a77_pmu_init(struct arm_pmu *cpu_pmu)
-{
- return armv8_pmu_init_nogroups(cpu_pmu, "armv8_cortex_a77",
- armv8_pmuv3_map_event);
-}
-
-static int armv8_a78_pmu_init(struct arm_pmu *cpu_pmu)
-{
- return armv8_pmu_init_nogroups(cpu_pmu, "armv8_cortex_a78",
- armv8_pmuv3_map_event);
-}
-
-static int armv8_e1_pmu_init(struct arm_pmu *cpu_pmu)
-{
- return armv8_pmu_init_nogroups(cpu_pmu, "armv8_neoverse_e1",
- armv8_pmuv3_map_event);
-}
-
-static int armv8_n1_pmu_init(struct arm_pmu *cpu_pmu)
-{
- return armv8_pmu_init_nogroups(cpu_pmu, "armv8_neoverse_n1",
- armv8_pmuv3_map_event);
-}
-
static int armv8_thunder_pmu_init(struct arm_pmu *cpu_pmu)
{
return armv8_pmu_init_nogroups(cpu_pmu, "armv8_cavium_thunder",
@@ -1248,23 +1336,31 @@ static int armv8_vulcan_pmu_init(struct arm_pmu *cpu_pmu)
}
static const struct of_device_id armv8_pmu_of_device_ids[] = {
- {.compatible = "arm,armv8-pmuv3", .data = armv8_pmuv3_init},
- {.compatible = "arm,cortex-a34-pmu", .data = armv8_a34_pmu_init},
+ {.compatible = "arm,armv8-pmuv3", .data = armv8_pmuv3_pmu_init},
+ {.compatible = "arm,cortex-a34-pmu", .data = armv8_cortex_a34_pmu_init},
{.compatible = "arm,cortex-a35-pmu", .data = armv8_a35_pmu_init},
{.compatible = "arm,cortex-a53-pmu", .data = armv8_a53_pmu_init},
- {.compatible = "arm,cortex-a55-pmu", .data = armv8_a55_pmu_init},
+ {.compatible = "arm,cortex-a55-pmu", .data = armv8_cortex_a55_pmu_init},
{.compatible = "arm,cortex-a57-pmu", .data = armv8_a57_pmu_init},
- {.compatible = "arm,cortex-a65-pmu", .data = armv8_a65_pmu_init},
+ {.compatible = "arm,cortex-a65-pmu", .data = armv8_cortex_a65_pmu_init},
{.compatible = "arm,cortex-a72-pmu", .data = armv8_a72_pmu_init},
{.compatible = "arm,cortex-a73-pmu", .data = armv8_a73_pmu_init},
- {.compatible = "arm,cortex-a75-pmu", .data = armv8_a75_pmu_init},
- {.compatible = "arm,cortex-a76-pmu", .data = armv8_a76_pmu_init},
- {.compatible = "arm,cortex-a77-pmu", .data = armv8_a77_pmu_init},
- {.compatible = "arm,cortex-a78-pmu", .data = armv8_a78_pmu_init},
- {.compatible = "arm,neoverse-e1-pmu", .data = armv8_e1_pmu_init},
- {.compatible = "arm,neoverse-n1-pmu", .data = armv8_n1_pmu_init},
+ {.compatible = "arm,cortex-a75-pmu", .data = armv8_cortex_a75_pmu_init},
+ {.compatible = "arm,cortex-a76-pmu", .data = armv8_cortex_a76_pmu_init},
+ {.compatible = "arm,cortex-a77-pmu", .data = armv8_cortex_a77_pmu_init},
+ {.compatible = "arm,cortex-a78-pmu", .data = armv8_cortex_a78_pmu_init},
+ {.compatible = "arm,cortex-a510-pmu", .data = armv9_cortex_a510_pmu_init},
+ {.compatible = "arm,cortex-a710-pmu", .data = armv9_cortex_a710_pmu_init},
+ {.compatible = "arm,cortex-x1-pmu", .data = armv8_cortex_x1_pmu_init},
+ {.compatible = "arm,cortex-x2-pmu", .data = armv9_cortex_x2_pmu_init},
+ {.compatible = "arm,neoverse-e1-pmu", .data = armv8_neoverse_e1_pmu_init},
+ {.compatible = "arm,neoverse-n1-pmu", .data = armv8_neoverse_n1_pmu_init},
+ {.compatible = "arm,neoverse-n2-pmu", .data = armv9_neoverse_n2_pmu_init},
+ {.compatible = "arm,neoverse-v1-pmu", .data = armv8_neoverse_v1_pmu_init},
{.compatible = "cavium,thunder-pmu", .data = armv8_thunder_pmu_init},
{.compatible = "brcm,vulcan-pmu", .data = armv8_vulcan_pmu_init},
+ {.compatible = "nvidia,carmel-pmu", .data = armv8_nvidia_carmel_pmu_init},
+ {.compatible = "nvidia,denver-pmu", .data = armv8_nvidia_denver_pmu_init},
{},
};
@@ -1287,7 +1383,7 @@ static int __init armv8_pmu_driver_init(void)
if (acpi_disabled)
return platform_driver_register(&armv8_pmu_driver);
else
- return arm_pmu_acpi_probe(armv8_pmuv3_init);
+ return arm_pmu_acpi_probe(armv8_pmuv3_pmu_init);
}
device_initcall(armv8_pmu_driver_init)
@@ -1301,6 +1397,14 @@ void arch_perf_update_userpage(struct perf_event *event,
userpg->cap_user_time = 0;
userpg->cap_user_time_zero = 0;
userpg->cap_user_time_short = 0;
+ userpg->cap_user_rdpmc = armv8pmu_event_has_user_read(event);
+
+ if (userpg->cap_user_rdpmc) {
+ if (event->hw.flags & ARMPMU_EVT_64BIT)
+ userpg->pmc_width = 64;
+ else
+ userpg->pmc_width = 32;
+ }
do {
rd = sched_clock_read_begin(&seq);
diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c
index aacf2f5..5369e64 100644
--- a/arch/arm64/kernel/process.c
+++ b/arch/arm64/kernel/process.c
@@ -40,6 +40,7 @@
#include <linux/percpu.h>
#include <linux/thread_info.h>
#include <linux/prctl.h>
+#include <linux/stacktrace.h>
#include <asm/alternative.h>
#include <asm/compat.h>
@@ -439,34 +440,26 @@ static void entry_task_switch(struct task_struct *next)
/*
* ARM erratum 1418040 handling, affecting the 32bit view of CNTVCT.
- * Assuming the virtual counter is enabled at the beginning of times:
- *
- * - disable access when switching from a 64bit task to a 32bit task
- * - enable access when switching from a 32bit task to a 64bit task
+ * Ensure access is disabled when switching to a 32bit task, ensure
+ * access is enabled when switching to a 64bit task.
*/
-static void erratum_1418040_thread_switch(struct task_struct *prev,
- struct task_struct *next)
+static void erratum_1418040_thread_switch(struct task_struct *next)
{
- bool prev32, next32;
- u64 val;
-
- if (!IS_ENABLED(CONFIG_ARM64_ERRATUM_1418040))
+ if (!IS_ENABLED(CONFIG_ARM64_ERRATUM_1418040) ||
+ !this_cpu_has_cap(ARM64_WORKAROUND_1418040))
return;
- prev32 = is_compat_thread(task_thread_info(prev));
- next32 = is_compat_thread(task_thread_info(next));
-
- if (prev32 == next32 || !this_cpu_has_cap(ARM64_WORKAROUND_1418040))
- return;
-
- val = read_sysreg(cntkctl_el1);
-
- if (!next32)
- val |= ARCH_TIMER_USR_VCT_ACCESS_EN;
+ if (is_compat_thread(task_thread_info(next)))
+ sysreg_clear_set(cntkctl_el1, ARCH_TIMER_USR_VCT_ACCESS_EN, 0);
else
- val &= ~ARCH_TIMER_USR_VCT_ACCESS_EN;
+ sysreg_clear_set(cntkctl_el1, 0, ARCH_TIMER_USR_VCT_ACCESS_EN);
+}
- write_sysreg(val, cntkctl_el1);
+static void erratum_1418040_new_exec(void)
+{
+ preempt_disable();
+ erratum_1418040_thread_switch(current);
+ preempt_enable();
}
/*
@@ -490,7 +483,8 @@ void update_sctlr_el1(u64 sctlr)
/*
* Thread switching.
*/
-__notrace_funcgraph struct task_struct *__switch_to(struct task_struct *prev,
+__notrace_funcgraph __sched
+struct task_struct *__switch_to(struct task_struct *prev,
struct task_struct *next)
{
struct task_struct *last;
@@ -501,7 +495,7 @@ __notrace_funcgraph struct task_struct *__switch_to(struct task_struct *prev,
contextidr_thread_switch(next);
entry_task_switch(next);
ssbs_thread_switch(next);
- erratum_1418040_thread_switch(prev, next);
+ erratum_1418040_thread_switch(next);
ptrauth_thread_switch_user(next);
/*
@@ -528,30 +522,37 @@ __notrace_funcgraph struct task_struct *__switch_to(struct task_struct *prev,
return last;
}
+struct wchan_info {
+ unsigned long pc;
+ int count;
+};
+
+static bool get_wchan_cb(void *arg, unsigned long pc)
+{
+ struct wchan_info *wchan_info = arg;
+
+ if (!in_sched_functions(pc)) {
+ wchan_info->pc = pc;
+ return false;
+ }
+ return wchan_info->count++ < 16;
+}
+
unsigned long __get_wchan(struct task_struct *p)
{
- struct stackframe frame;
- unsigned long stack_page, ret = 0;
- int count = 0;
+ struct wchan_info wchan_info = {
+ .pc = 0,
+ .count = 0,
+ };
- stack_page = (unsigned long)try_get_task_stack(p);
- if (!stack_page)
+ if (!try_get_task_stack(p))
return 0;
- start_backtrace(&frame, thread_saved_fp(p), thread_saved_pc(p));
+ arch_stack_walk(get_wchan_cb, &wchan_info, p, NULL);
- do {
- if (unwind_frame(p, &frame))
- goto out;
- if (!in_sched_functions(frame.pc)) {
- ret = frame.pc;
- goto out;
- }
- } while (count++ < 16);
-
-out:
put_task_stack(p);
- return ret;
+
+ return wchan_info.pc;
}
unsigned long arch_align_stack(unsigned long sp)
@@ -611,6 +612,7 @@ void arch_setup_new_exec(void)
current->mm->context.flags = mmflags;
ptrauth_thread_init_user();
mte_thread_init_user();
+ erratum_1418040_new_exec();
if (task_spec_ssb_noexec(current)) {
arch_prctl_spec_ctrl_set(current, PR_SPEC_STORE_BYPASS,
diff --git a/arch/arm64/kernel/ptrace.c b/arch/arm64/kernel/ptrace.c
index 88a9034..39dbdfd 100644
--- a/arch/arm64/kernel/ptrace.c
+++ b/arch/arm64/kernel/ptrace.c
@@ -812,9 +812,9 @@ static int sve_set(struct task_struct *target,
/*
* Apart from SVE_PT_REGS_MASK, all SVE_PT_* flags are consumed by
- * sve_set_vector_length(), which will also validate them for us:
+ * vec_set_vector_length(), which will also validate them for us:
*/
- ret = sve_set_vector_length(target, header.vl,
+ ret = vec_set_vector_length(target, ARM64_VEC_SVE, header.vl,
((unsigned long)header.flags & ~SVE_PT_REGS_MASK) << 16);
if (ret)
goto out;
@@ -1839,7 +1839,7 @@ static void tracehook_report_syscall(struct pt_regs *regs,
int syscall_trace_enter(struct pt_regs *regs)
{
- unsigned long flags = READ_ONCE(current_thread_info()->flags);
+ unsigned long flags = read_thread_flags();
if (flags & (_TIF_SYSCALL_EMU | _TIF_SYSCALL_TRACE)) {
tracehook_report_syscall(regs, PTRACE_SYSCALL_ENTER);
@@ -1862,7 +1862,7 @@ int syscall_trace_enter(struct pt_regs *regs)
void syscall_trace_exit(struct pt_regs *regs)
{
- unsigned long flags = READ_ONCE(current_thread_info()->flags);
+ unsigned long flags = read_thread_flags();
audit_syscall_exit(regs);
diff --git a/arch/arm64/kernel/return_address.c b/arch/arm64/kernel/return_address.c
index a6d18755..6833001 100644
--- a/arch/arm64/kernel/return_address.c
+++ b/arch/arm64/kernel/return_address.c
@@ -9,9 +9,9 @@
#include <linux/export.h>
#include <linux/ftrace.h>
#include <linux/kprobes.h>
+#include <linux/stacktrace.h>
#include <asm/stack_pointer.h>
-#include <asm/stacktrace.h>
struct return_address_data {
unsigned int level;
@@ -35,15 +35,11 @@ NOKPROBE_SYMBOL(save_return_addr);
void *return_address(unsigned int level)
{
struct return_address_data data;
- struct stackframe frame;
data.level = level + 2;
data.addr = NULL;
- start_backtrace(&frame,
- (unsigned long)__builtin_frame_address(0),
- (unsigned long)return_address);
- walk_stackframe(current, &frame, save_return_addr, &data);
+ arch_stack_walk(save_return_addr, &data, current, NULL);
if (!data.level)
return data.addr;
diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
index be5f85b..f705739 100644
--- a/arch/arm64/kernel/setup.c
+++ b/arch/arm64/kernel/setup.c
@@ -189,11 +189,16 @@ static void __init setup_machine_fdt(phys_addr_t dt_phys)
if (!dt_virt || !early_init_dt_scan(dt_virt)) {
pr_crit("\n"
- "Error: invalid device tree blob at physical address %pa (virtual address 0x%p)\n"
+ "Error: invalid device tree blob at physical address %pa (virtual address 0x%px)\n"
"The dtb must be 8-byte aligned and must not exceed 2 MB in size\n"
"\nPlease check your bootloader.",
&dt_phys, dt_virt);
+ /*
+ * Note that in this _really_ early stage we cannot even BUG()
+ * or oops, so the least terrible thing to do is cpu_relax(),
+ * or else we could end-up printing non-initialized data, etc.
+ */
while (true)
cpu_relax();
}
@@ -232,12 +237,14 @@ static void __init request_standard_resources(void)
if (memblock_is_nomap(region)) {
res->name = "reserved";
res->flags = IORESOURCE_MEM;
+ res->start = __pfn_to_phys(memblock_region_reserved_base_pfn(region));
+ res->end = __pfn_to_phys(memblock_region_reserved_end_pfn(region)) - 1;
} else {
res->name = "System RAM";
res->flags = IORESOURCE_SYSTEM_RAM | IORESOURCE_BUSY;
+ res->start = __pfn_to_phys(memblock_region_memory_base_pfn(region));
+ res->end = __pfn_to_phys(memblock_region_memory_end_pfn(region)) - 1;
}
- res->start = __pfn_to_phys(memblock_region_memory_base_pfn(region));
- res->end = __pfn_to_phys(memblock_region_memory_end_pfn(region)) - 1;
request_resource(&iomem_resource, res);
diff --git a/arch/arm64/kernel/signal.c b/arch/arm64/kernel/signal.c
index 8f6372b..d8aaf4b 100644
--- a/arch/arm64/kernel/signal.c
+++ b/arch/arm64/kernel/signal.c
@@ -948,7 +948,7 @@ void do_notify_resume(struct pt_regs *regs, unsigned long thread_flags)
}
local_daif_mask();
- thread_flags = READ_ONCE(current_thread_info()->flags);
+ thread_flags = read_thread_flags();
} while (thread_flags & _TIF_WORK_MASK);
}
diff --git a/arch/arm64/kernel/stacktrace.c b/arch/arm64/kernel/stacktrace.c
index 94f83cd4..0fb58fe 100644
--- a/arch/arm64/kernel/stacktrace.c
+++ b/arch/arm64/kernel/stacktrace.c
@@ -33,8 +33,8 @@
*/
-void start_backtrace(struct stackframe *frame, unsigned long fp,
- unsigned long pc)
+static void start_backtrace(struct stackframe *frame, unsigned long fp,
+ unsigned long pc)
{
frame->fp = fp;
frame->pc = pc;
@@ -63,7 +63,8 @@ void start_backtrace(struct stackframe *frame, unsigned long fp,
* records (e.g. a cycle), determined based on the location and fp value of A
* and the location (but not the fp value) of B.
*/
-int notrace unwind_frame(struct task_struct *tsk, struct stackframe *frame)
+static int notrace unwind_frame(struct task_struct *tsk,
+ struct stackframe *frame)
{
unsigned long fp = frame->fp;
struct stack_info info;
@@ -141,8 +142,9 @@ int notrace unwind_frame(struct task_struct *tsk, struct stackframe *frame)
}
NOKPROBE_SYMBOL(unwind_frame);
-void notrace walk_stackframe(struct task_struct *tsk, struct stackframe *frame,
- bool (*fn)(void *, unsigned long), void *data)
+static void notrace walk_stackframe(struct task_struct *tsk,
+ struct stackframe *frame,
+ bool (*fn)(void *, unsigned long), void *data)
{
while (1) {
int ret;
@@ -156,24 +158,20 @@ void notrace walk_stackframe(struct task_struct *tsk, struct stackframe *frame,
}
NOKPROBE_SYMBOL(walk_stackframe);
-static void dump_backtrace_entry(unsigned long where, const char *loglvl)
+static bool dump_backtrace_entry(void *arg, unsigned long where)
{
+ char *loglvl = arg;
printk("%s %pSb\n", loglvl, (void *)where);
+ return true;
}
void dump_backtrace(struct pt_regs *regs, struct task_struct *tsk,
const char *loglvl)
{
- struct stackframe frame;
- int skip = 0;
-
pr_debug("%s(regs = %p tsk = %p)\n", __func__, regs, tsk);
- if (regs) {
- if (user_mode(regs))
- return;
- skip = 1;
- }
+ if (regs && user_mode(regs))
+ return;
if (!tsk)
tsk = current;
@@ -181,36 +179,8 @@ void dump_backtrace(struct pt_regs *regs, struct task_struct *tsk,
if (!try_get_task_stack(tsk))
return;
- if (tsk == current) {
- start_backtrace(&frame,
- (unsigned long)__builtin_frame_address(0),
- (unsigned long)dump_backtrace);
- } else {
- /*
- * task blocked in __switch_to
- */
- start_backtrace(&frame,
- thread_saved_fp(tsk),
- thread_saved_pc(tsk));
- }
-
printk("%sCall trace:\n", loglvl);
- do {
- /* skip until specified stack frame */
- if (!skip) {
- dump_backtrace_entry(frame.pc, loglvl);
- } else if (frame.fp == regs->regs[29]) {
- skip = 0;
- /*
- * Mostly, this is the case where this function is
- * called in panic/abort. As exception handler's
- * stack frame does not contain the corresponding pc
- * at which an exception has taken place, use regs->pc
- * instead.
- */
- dump_backtrace_entry(regs->pc, loglvl);
- }
- } while (!unwind_frame(tsk, &frame));
+ arch_stack_walk(dump_backtrace_entry, (void *)loglvl, tsk, regs);
put_task_stack(tsk);
}
@@ -221,8 +191,6 @@ void show_stack(struct task_struct *tsk, unsigned long *sp, const char *loglvl)
barrier();
}
-#ifdef CONFIG_STACKTRACE
-
noinline notrace void arch_stack_walk(stack_trace_consume_fn consume_entry,
void *cookie, struct task_struct *task,
struct pt_regs *regs)
@@ -241,5 +209,3 @@ noinline notrace void arch_stack_walk(stack_trace_consume_fn consume_entry,
walk_stackframe(task, &frame, consume_entry, cookie);
}
-
-#endif
diff --git a/arch/arm64/kernel/syscall.c b/arch/arm64/kernel/syscall.c
index 50a0f1a..c938603 100644
--- a/arch/arm64/kernel/syscall.c
+++ b/arch/arm64/kernel/syscall.c
@@ -81,7 +81,7 @@ void syscall_trace_exit(struct pt_regs *regs);
static void el0_svc_common(struct pt_regs *regs, int scno, int sc_nr,
const syscall_fn_t syscall_table[])
{
- unsigned long flags = current_thread_info()->flags;
+ unsigned long flags = read_thread_flags();
regs->orig_x0 = regs->regs[0];
regs->syscallno = scno;
@@ -148,7 +148,7 @@ static void el0_svc_common(struct pt_regs *regs, int scno, int sc_nr,
*/
if (!has_syscall_work(flags) && !IS_ENABLED(CONFIG_DEBUG_RSEQ)) {
local_daif_mask();
- flags = current_thread_info()->flags;
+ flags = read_thread_flags();
if (!has_syscall_work(flags) && !(flags & _TIF_SINGLESTEP))
return;
local_daif_restore(DAIF_PROCCTX);
diff --git a/arch/arm64/kernel/time.c b/arch/arm64/kernel/time.c
index eebbc8d..b5855eb 100644
--- a/arch/arm64/kernel/time.c
+++ b/arch/arm64/kernel/time.c
@@ -18,6 +18,7 @@
#include <linux/timex.h>
#include <linux/errno.h>
#include <linux/profile.h>
+#include <linux/stacktrace.h>
#include <linux/syscore_ops.h>
#include <linux/timer.h>
#include <linux/irq.h>
@@ -29,25 +30,25 @@
#include <clocksource/arm_arch_timer.h>
#include <asm/thread_info.h>
-#include <asm/stacktrace.h>
#include <asm/paravirt.h>
+static bool profile_pc_cb(void *arg, unsigned long pc)
+{
+ unsigned long *prof_pc = arg;
+
+ if (in_lock_functions(pc))
+ return true;
+ *prof_pc = pc;
+ return false;
+}
+
unsigned long profile_pc(struct pt_regs *regs)
{
- struct stackframe frame;
+ unsigned long prof_pc = 0;
- if (!in_lock_functions(regs->pc))
- return regs->pc;
+ arch_stack_walk(profile_pc_cb, &prof_pc, current, regs);
- start_backtrace(&frame, regs->regs[29], regs->pc);
-
- do {
- int ret = unwind_frame(NULL, &frame);
- if (ret < 0)
- return 0;
- } while (in_lock_functions(frame.pc));
-
- return frame.pc;
+ return prof_pc;
}
EXPORT_SYMBOL(profile_pc);
diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c
index 7b21213..e8986e6 100644
--- a/arch/arm64/kernel/traps.c
+++ b/arch/arm64/kernel/traps.c
@@ -994,7 +994,7 @@ static struct break_hook bug_break_hook = {
static int reserved_fault_handler(struct pt_regs *regs, unsigned int esr)
{
pr_err("%s generated an invalid instruction at %pS!\n",
- in_bpf_jit(regs) ? "BPF JIT" : "Kernel text patching",
+ "Kernel text patching",
(void *)instruction_pointer(regs));
/* We cannot handle this */
diff --git a/arch/arm64/kernel/vdso/Makefile b/arch/arm64/kernel/vdso/Makefile
index 700767d..6081349 100644
--- a/arch/arm64/kernel/vdso/Makefile
+++ b/arch/arm64/kernel/vdso/Makefile
@@ -32,6 +32,7 @@
CFLAGS_REMOVE_vgettimeofday.o = $(CC_FLAGS_FTRACE) -Os $(CC_FLAGS_SCS) $(GCC_PLUGINS_CFLAGS) \
$(CC_FLAGS_LTO)
KASAN_SANITIZE := n
+KCSAN_SANITIZE := n
UBSAN_SANITIZE := n
OBJECT_FILES_NON_STANDARD := y
KCOV_INSTRUMENT := n
diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c
index 275a273..5abe061 100644
--- a/arch/arm64/kvm/handle_exit.c
+++ b/arch/arm64/kvm/handle_exit.c
@@ -140,9 +140,12 @@ static int kvm_handle_unknown_ec(struct kvm_vcpu *vcpu)
return 1;
}
+/*
+ * Guest access to SVE registers should be routed to this handler only
+ * when the system doesn't support SVE.
+ */
static int handle_sve(struct kvm_vcpu *vcpu)
{
- /* Until SVE is supported for guests: */
kvm_inject_undefined(vcpu);
return 1;
}
diff --git a/arch/arm64/kvm/hyp/nvhe/Makefile b/arch/arm64/kvm/hyp/nvhe/Makefile
index c3c1197..24b2c24 100644
--- a/arch/arm64/kvm/hyp/nvhe/Makefile
+++ b/arch/arm64/kvm/hyp/nvhe/Makefile
@@ -89,6 +89,7 @@
# cause crashes. Just disable it.
GCOV_PROFILE := n
KASAN_SANITIZE := n
+KCSAN_SANITIZE := n
UBSAN_SANITIZE := n
KCOV_INSTRUMENT := n
diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c
index 426bd7f..27386f0 100644
--- a/arch/arm64/kvm/reset.c
+++ b/arch/arm64/kvm/reset.c
@@ -52,10 +52,10 @@ int kvm_arm_init_sve(void)
* The get_sve_reg()/set_sve_reg() ioctl interface will need
* to be extended with multiple register slice support in
* order to support vector lengths greater than
- * SVE_VL_ARCH_MAX:
+ * VL_ARCH_MAX:
*/
- if (WARN_ON(kvm_sve_max_vl > SVE_VL_ARCH_MAX))
- kvm_sve_max_vl = SVE_VL_ARCH_MAX;
+ if (WARN_ON(kvm_sve_max_vl > VL_ARCH_MAX))
+ kvm_sve_max_vl = VL_ARCH_MAX;
/*
* Don't even try to make use of vector lengths that
@@ -103,7 +103,7 @@ static int kvm_vcpu_finalize_sve(struct kvm_vcpu *vcpu)
* set_sve_vls(). Double-check here just to be sure:
*/
if (WARN_ON(!sve_vl_valid(vl) || vl > sve_max_virtualisable_vl() ||
- vl > SVE_VL_ARCH_MAX))
+ vl > VL_ARCH_MAX))
return -EIO;
buf = kzalloc(SVE_SIG_REGS_SIZE(sve_vq_from_vl(vl)), GFP_KERNEL_ACCOUNT);
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index e3ec1a4..4dc2fba 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1525,7 +1525,7 @@ static const struct sys_reg_desc sys_reg_descs[] = {
/* CRm=6 */
ID_SANITISED(ID_AA64ISAR0_EL1),
ID_SANITISED(ID_AA64ISAR1_EL1),
- ID_UNALLOCATED(6,2),
+ ID_SANITISED(ID_AA64ISAR2_EL1),
ID_UNALLOCATED(6,3),
ID_UNALLOCATED(6,4),
ID_UNALLOCATED(6,5),
diff --git a/arch/arm64/lib/clear_page.S b/arch/arm64/lib/clear_page.S
index b84b179..1fd5d79 100644
--- a/arch/arm64/lib/clear_page.S
+++ b/arch/arm64/lib/clear_page.S
@@ -16,6 +16,7 @@
*/
SYM_FUNC_START_PI(clear_page)
mrs x1, dczid_el0
+ tbnz x1, #4, 2f /* Branch if DC ZVA is prohibited */
and w1, w1, #0xf
mov x2, #4
lsl x1, x2, x1
@@ -25,5 +26,14 @@
tst x0, #(PAGE_SIZE - 1)
b.ne 1b
ret
+
+2: stnp xzr, xzr, [x0]
+ stnp xzr, xzr, [x0, #16]
+ stnp xzr, xzr, [x0, #32]
+ stnp xzr, xzr, [x0, #48]
+ add x0, x0, #64
+ tst x0, #(PAGE_SIZE - 1)
+ b.ne 2b
+ ret
SYM_FUNC_END_PI(clear_page)
EXPORT_SYMBOL(clear_page)
diff --git a/arch/arm64/lib/kasan_sw_tags.S b/arch/arm64/lib/kasan_sw_tags.S
index 5b04464..20784ce 100644
--- a/arch/arm64/lib/kasan_sw_tags.S
+++ b/arch/arm64/lib/kasan_sw_tags.S
@@ -38,9 +38,7 @@
* incremented by 256 prior to return).
*/
SYM_CODE_START(__hwasan_tag_mismatch)
-#ifdef BTI_C
- BTI_C
-#endif
+ bti c
add x29, sp, #232
stp x2, x3, [sp, #8 * 2]
stp x4, x5, [sp, #8 * 4]
diff --git a/arch/arm64/lib/mte.S b/arch/arm64/lib/mte.S
index e83643b..f531dcb9 100644
--- a/arch/arm64/lib/mte.S
+++ b/arch/arm64/lib/mte.S
@@ -43,17 +43,23 @@
* x0 - address to the beginning of the page
*/
SYM_FUNC_START(mte_zero_clear_page_tags)
+ and x0, x0, #(1 << MTE_TAG_SHIFT) - 1 // clear the tag
mrs x1, dczid_el0
+ tbnz x1, #4, 2f // Branch if DC GZVA is prohibited
and w1, w1, #0xf
mov x2, #4
lsl x1, x2, x1
- and x0, x0, #(1 << MTE_TAG_SHIFT) - 1 // clear the tag
1: dc gzva, x0
add x0, x0, x1
tst x0, #(PAGE_SIZE - 1)
b.ne 1b
ret
+
+2: stz2g x0, [x0], #(MTE_GRANULE_SIZE * 2)
+ tst x0, #(PAGE_SIZE - 1)
+ b.ne 2b
+ ret
SYM_FUNC_END(mte_zero_clear_page_tags)
/*
diff --git a/arch/arm64/lib/xor-neon.c b/arch/arm64/lib/xor-neon.c
index 11bf4f8a..d189cf4 100644
--- a/arch/arm64/lib/xor-neon.c
+++ b/arch/arm64/lib/xor-neon.c
@@ -167,7 +167,7 @@ void xor_arm64_neon_5(unsigned long bytes, unsigned long *p1,
} while (--lines > 0);
}
-struct xor_block_template const xor_block_inner_neon = {
+struct xor_block_template xor_block_inner_neon __ro_after_init = {
.name = "__inner_neon__",
.do_2 = xor_arm64_neon_2,
.do_3 = xor_arm64_neon_3,
@@ -176,6 +176,151 @@ struct xor_block_template const xor_block_inner_neon = {
};
EXPORT_SYMBOL(xor_block_inner_neon);
+static inline uint64x2_t eor3(uint64x2_t p, uint64x2_t q, uint64x2_t r)
+{
+ uint64x2_t res;
+
+ asm(ARM64_ASM_PREAMBLE ".arch_extension sha3\n"
+ "eor3 %0.16b, %1.16b, %2.16b, %3.16b"
+ : "=w"(res) : "w"(p), "w"(q), "w"(r));
+ return res;
+}
+
+static void xor_arm64_eor3_3(unsigned long bytes, unsigned long *p1,
+ unsigned long *p2, unsigned long *p3)
+{
+ uint64_t *dp1 = (uint64_t *)p1;
+ uint64_t *dp2 = (uint64_t *)p2;
+ uint64_t *dp3 = (uint64_t *)p3;
+
+ register uint64x2_t v0, v1, v2, v3;
+ long lines = bytes / (sizeof(uint64x2_t) * 4);
+
+ do {
+ /* p1 ^= p2 ^ p3 */
+ v0 = eor3(vld1q_u64(dp1 + 0), vld1q_u64(dp2 + 0),
+ vld1q_u64(dp3 + 0));
+ v1 = eor3(vld1q_u64(dp1 + 2), vld1q_u64(dp2 + 2),
+ vld1q_u64(dp3 + 2));
+ v2 = eor3(vld1q_u64(dp1 + 4), vld1q_u64(dp2 + 4),
+ vld1q_u64(dp3 + 4));
+ v3 = eor3(vld1q_u64(dp1 + 6), vld1q_u64(dp2 + 6),
+ vld1q_u64(dp3 + 6));
+
+ /* store */
+ vst1q_u64(dp1 + 0, v0);
+ vst1q_u64(dp1 + 2, v1);
+ vst1q_u64(dp1 + 4, v2);
+ vst1q_u64(dp1 + 6, v3);
+
+ dp1 += 8;
+ dp2 += 8;
+ dp3 += 8;
+ } while (--lines > 0);
+}
+
+static void xor_arm64_eor3_4(unsigned long bytes, unsigned long *p1,
+ unsigned long *p2, unsigned long *p3,
+ unsigned long *p4)
+{
+ uint64_t *dp1 = (uint64_t *)p1;
+ uint64_t *dp2 = (uint64_t *)p2;
+ uint64_t *dp3 = (uint64_t *)p3;
+ uint64_t *dp4 = (uint64_t *)p4;
+
+ register uint64x2_t v0, v1, v2, v3;
+ long lines = bytes / (sizeof(uint64x2_t) * 4);
+
+ do {
+ /* p1 ^= p2 ^ p3 */
+ v0 = eor3(vld1q_u64(dp1 + 0), vld1q_u64(dp2 + 0),
+ vld1q_u64(dp3 + 0));
+ v1 = eor3(vld1q_u64(dp1 + 2), vld1q_u64(dp2 + 2),
+ vld1q_u64(dp3 + 2));
+ v2 = eor3(vld1q_u64(dp1 + 4), vld1q_u64(dp2 + 4),
+ vld1q_u64(dp3 + 4));
+ v3 = eor3(vld1q_u64(dp1 + 6), vld1q_u64(dp2 + 6),
+ vld1q_u64(dp3 + 6));
+
+ /* p1 ^= p4 */
+ v0 = veorq_u64(v0, vld1q_u64(dp4 + 0));
+ v1 = veorq_u64(v1, vld1q_u64(dp4 + 2));
+ v2 = veorq_u64(v2, vld1q_u64(dp4 + 4));
+ v3 = veorq_u64(v3, vld1q_u64(dp4 + 6));
+
+ /* store */
+ vst1q_u64(dp1 + 0, v0);
+ vst1q_u64(dp1 + 2, v1);
+ vst1q_u64(dp1 + 4, v2);
+ vst1q_u64(dp1 + 6, v3);
+
+ dp1 += 8;
+ dp2 += 8;
+ dp3 += 8;
+ dp4 += 8;
+ } while (--lines > 0);
+}
+
+static void xor_arm64_eor3_5(unsigned long bytes, unsigned long *p1,
+ unsigned long *p2, unsigned long *p3,
+ unsigned long *p4, unsigned long *p5)
+{
+ uint64_t *dp1 = (uint64_t *)p1;
+ uint64_t *dp2 = (uint64_t *)p2;
+ uint64_t *dp3 = (uint64_t *)p3;
+ uint64_t *dp4 = (uint64_t *)p4;
+ uint64_t *dp5 = (uint64_t *)p5;
+
+ register uint64x2_t v0, v1, v2, v3;
+ long lines = bytes / (sizeof(uint64x2_t) * 4);
+
+ do {
+ /* p1 ^= p2 ^ p3 */
+ v0 = eor3(vld1q_u64(dp1 + 0), vld1q_u64(dp2 + 0),
+ vld1q_u64(dp3 + 0));
+ v1 = eor3(vld1q_u64(dp1 + 2), vld1q_u64(dp2 + 2),
+ vld1q_u64(dp3 + 2));
+ v2 = eor3(vld1q_u64(dp1 + 4), vld1q_u64(dp2 + 4),
+ vld1q_u64(dp3 + 4));
+ v3 = eor3(vld1q_u64(dp1 + 6), vld1q_u64(dp2 + 6),
+ vld1q_u64(dp3 + 6));
+
+ /* p1 ^= p4 ^ p5 */
+ v0 = eor3(v0, vld1q_u64(dp4 + 0), vld1q_u64(dp5 + 0));
+ v1 = eor3(v1, vld1q_u64(dp4 + 2), vld1q_u64(dp5 + 2));
+ v2 = eor3(v2, vld1q_u64(dp4 + 4), vld1q_u64(dp5 + 4));
+ v3 = eor3(v3, vld1q_u64(dp4 + 6), vld1q_u64(dp5 + 6));
+
+ /* store */
+ vst1q_u64(dp1 + 0, v0);
+ vst1q_u64(dp1 + 2, v1);
+ vst1q_u64(dp1 + 4, v2);
+ vst1q_u64(dp1 + 6, v3);
+
+ dp1 += 8;
+ dp2 += 8;
+ dp3 += 8;
+ dp4 += 8;
+ dp5 += 8;
+ } while (--lines > 0);
+}
+
+static int __init xor_neon_init(void)
+{
+ if (IS_ENABLED(CONFIG_AS_HAS_SHA3) && cpu_have_named_feature(SHA3)) {
+ xor_block_inner_neon.do_3 = xor_arm64_eor3_3;
+ xor_block_inner_neon.do_4 = xor_arm64_eor3_4;
+ xor_block_inner_neon.do_5 = xor_arm64_eor3_5;
+ }
+ return 0;
+}
+module_init(xor_neon_init);
+
+static void __exit xor_neon_exit(void)
+{
+}
+module_exit(xor_neon_exit);
+
MODULE_AUTHOR("Jackie Liu <liuyun01@kylinos.cn>");
MODULE_DESCRIPTION("ARMv8 XOR Extensions");
MODULE_LICENSE("GPL");
diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S
index 5051b3c..7d0563d 100644
--- a/arch/arm64/mm/cache.S
+++ b/arch/arm64/mm/cache.S
@@ -140,15 +140,7 @@
* - start - kernel start address of region
* - end - kernel end address of region
*/
-SYM_FUNC_START_LOCAL(__dma_inv_area)
SYM_FUNC_START_PI(dcache_inval_poc)
- /* FALLTHROUGH */
-
-/*
- * __dma_inv_area(start, end)
- * - start - virtual start address of region
- * - end - virtual end address of region
- */
dcache_line_size x2, x3
sub x3, x2, #1
tst x1, x3 // end cache line aligned?
@@ -167,7 +159,6 @@
dsb sy
ret
SYM_FUNC_END_PI(dcache_inval_poc)
-SYM_FUNC_END(__dma_inv_area)
/*
* dcache_clean_poc(start, end)
@@ -178,19 +169,10 @@
* - start - virtual start address of region
* - end - virtual end address of region
*/
-SYM_FUNC_START_LOCAL(__dma_clean_area)
SYM_FUNC_START_PI(dcache_clean_poc)
- /* FALLTHROUGH */
-
-/*
- * __dma_clean_area(start, end)
- * - start - virtual start address of region
- * - end - virtual end address of region
- */
dcache_by_line_op cvac, sy, x0, x1, x2, x3
ret
SYM_FUNC_END_PI(dcache_clean_poc)
-SYM_FUNC_END(__dma_clean_area)
/*
* dcache_clean_pop(start, end)
@@ -232,8 +214,8 @@
SYM_FUNC_START_PI(__dma_map_area)
add x1, x0, x1
cmp w2, #DMA_FROM_DEVICE
- b.eq __dma_inv_area
- b __dma_clean_area
+ b.eq __pi_dcache_inval_poc
+ b __pi_dcache_clean_poc
SYM_FUNC_END_PI(__dma_map_area)
/*
@@ -245,6 +227,6 @@
SYM_FUNC_START_PI(__dma_unmap_area)
add x1, x0, x1
cmp w2, #DMA_TO_DEVICE
- b.ne __dma_inv_area
+ b.ne __pi_dcache_inval_poc
ret
SYM_FUNC_END_PI(__dma_unmap_area)
diff --git a/arch/arm64/mm/context.c b/arch/arm64/mm/context.c
index cd72576..b8b4cf0 100644
--- a/arch/arm64/mm/context.c
+++ b/arch/arm64/mm/context.c
@@ -35,8 +35,8 @@ static unsigned long *pinned_asid_map;
#define ASID_FIRST_VERSION (1UL << asid_bits)
#define NUM_USER_ASIDS ASID_FIRST_VERSION
-#define asid2idx(asid) ((asid) & ~ASID_MASK)
-#define idx2asid(idx) asid2idx(idx)
+#define ctxid2asid(asid) ((asid) & ~ASID_MASK)
+#define asid2ctxid(asid, genid) ((asid) | (genid))
/* Get the ASIDBits supported by the current CPU */
static u32 get_cpu_asid_bits(void)
@@ -50,10 +50,10 @@ static u32 get_cpu_asid_bits(void)
pr_warn("CPU%d: Unknown ASID size (%d); assuming 8-bit\n",
smp_processor_id(), fld);
fallthrough;
- case 0:
+ case ID_AA64MMFR0_ASID_8:
asid = 8;
break;
- case 2:
+ case ID_AA64MMFR0_ASID_16:
asid = 16;
}
@@ -120,7 +120,7 @@ static void flush_context(void)
*/
if (asid == 0)
asid = per_cpu(reserved_asids, i);
- __set_bit(asid2idx(asid), asid_map);
+ __set_bit(ctxid2asid(asid), asid_map);
per_cpu(reserved_asids, i) = asid;
}
@@ -162,7 +162,7 @@ static u64 new_context(struct mm_struct *mm)
u64 generation = atomic64_read(&asid_generation);
if (asid != 0) {
- u64 newasid = generation | (asid & ~ASID_MASK);
+ u64 newasid = asid2ctxid(ctxid2asid(asid), generation);
/*
* If our current ASID was active during a rollover, we
@@ -183,7 +183,7 @@ static u64 new_context(struct mm_struct *mm)
* We had a valid ASID in a previous life, so try to re-use
* it if possible.
*/
- if (!__test_and_set_bit(asid2idx(asid), asid_map))
+ if (!__test_and_set_bit(ctxid2asid(asid), asid_map))
return newasid;
}
@@ -209,7 +209,7 @@ static u64 new_context(struct mm_struct *mm)
set_asid:
__set_bit(asid, asid_map);
cur_idx = asid;
- return idx2asid(asid) | generation;
+ return asid2ctxid(asid, generation);
}
void check_and_switch_context(struct mm_struct *mm)
@@ -300,13 +300,13 @@ unsigned long arm64_mm_context_get(struct mm_struct *mm)
}
nr_pinned_asids++;
- __set_bit(asid2idx(asid), pinned_asid_map);
+ __set_bit(ctxid2asid(asid), pinned_asid_map);
refcount_set(&mm->context.pinned, 1);
out_unlock:
raw_spin_unlock_irqrestore(&cpu_asid_lock, flags);
- asid &= ~ASID_MASK;
+ asid = ctxid2asid(asid);
/* Set the equivalent of USER_ASID_BIT */
if (asid && arm64_kernel_unmapped_at_el0())
@@ -327,7 +327,7 @@ void arm64_mm_context_put(struct mm_struct *mm)
raw_spin_lock_irqsave(&cpu_asid_lock, flags);
if (refcount_dec_and_test(&mm->context.pinned)) {
- __clear_bit(asid2idx(asid), pinned_asid_map);
+ __clear_bit(ctxid2asid(asid), pinned_asid_map);
nr_pinned_asids--;
}
diff --git a/arch/arm64/mm/extable.c b/arch/arm64/mm/extable.c
index c3d5381..c0181e6 100644
--- a/arch/arm64/mm/extable.c
+++ b/arch/arm64/mm/extable.c
@@ -10,9 +10,6 @@
#include <asm/asm-extable.h>
#include <asm/ptrace.h>
-typedef bool (*ex_handler_t)(const struct exception_table_entry *,
- struct pt_regs *);
-
static inline unsigned long
get_ex_fixup(const struct exception_table_entry *ex)
{
diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c
index 9ae24e3..9a9e767 100644
--- a/arch/arm64/mm/fault.c
+++ b/arch/arm64/mm/fault.c
@@ -297,6 +297,8 @@ static void die_kernel_fault(const char *msg, unsigned long addr,
pr_alert("Unable to handle kernel %s at virtual address %016lx\n", msg,
addr);
+ kasan_non_canonical_hook(addr);
+
mem_abort_decode(esr);
show_pte(addr);
@@ -813,11 +815,8 @@ void do_mem_abort(unsigned long far, unsigned int esr, struct pt_regs *regs)
if (!inf->fn(far, esr, regs))
return;
- if (!user_mode(regs)) {
- pr_alert("Unhandled fault at 0x%016lx\n", addr);
- mem_abort_decode(esr);
- show_pte(addr);
- }
+ if (!user_mode(regs))
+ die_kernel_fault(inf->name, addr, esr, regs);
/*
* At this point we have an unrecognized fault type whose tag bits may
diff --git a/arch/arm64/mm/ptdump.c b/arch/arm64/mm/ptdump.c
index 1c40353..9bc4066 100644
--- a/arch/arm64/mm/ptdump.c
+++ b/arch/arm64/mm/ptdump.c
@@ -41,8 +41,6 @@ static struct addr_marker address_markers[] = {
{ 0 /* KASAN_SHADOW_START */, "Kasan shadow start" },
{ KASAN_SHADOW_END, "Kasan shadow end" },
#endif
- { BPF_JIT_REGION_START, "BPF start" },
- { BPF_JIT_REGION_END, "BPF end" },
{ MODULES_VADDR, "Modules start" },
{ MODULES_END, "Modules end" },
{ VMALLOC_START, "vmalloc() area" },
diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
index 3a8a714..e96d4d8 100644
--- a/arch/arm64/net/bpf_jit_comp.c
+++ b/arch/arm64/net/bpf_jit_comp.c
@@ -44,7 +44,7 @@ static const int bpf2a64[] = {
[BPF_REG_9] = A64_R(22),
/* read-only frame pointer to access stack */
[BPF_REG_FP] = A64_R(25),
- /* temporary registers for internal BPF JIT */
+ /* temporary registers for BPF JIT */
[TMP_REG_1] = A64_R(10),
[TMP_REG_2] = A64_R(11),
[TMP_REG_3] = A64_R(12),
@@ -287,13 +287,14 @@ static int emit_bpf_tail_call(struct jit_ctx *ctx)
emit(A64_CMP(0, r3, tmp), ctx);
emit(A64_B_(A64_COND_CS, jmp_offset), ctx);
- /* if (tail_call_cnt > MAX_TAIL_CALL_CNT)
+ /*
+ * if (tail_call_cnt >= MAX_TAIL_CALL_CNT)
* goto out;
* tail_call_cnt++;
*/
emit_a64_mov_i64(tmp, MAX_TAIL_CALL_CNT, ctx);
emit(A64_CMP(1, tcc, tmp), ctx);
- emit(A64_B_(A64_COND_HI, jmp_offset), ctx);
+ emit(A64_B_(A64_COND_CS, jmp_offset), ctx);
emit(A64_ADD_I(1, tcc, tcc, 1), ctx);
/* prog = array->ptrs[index];
@@ -791,7 +792,10 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
u64 imm64;
imm64 = (u64)insn1.imm << 32 | (u32)imm;
- emit_a64_mov_i64(dst, imm64, ctx);
+ if (bpf_pseudo_func(insn))
+ emit_addr_mov_i64(dst, imm64, ctx);
+ else
+ emit_a64_mov_i64(dst, imm64, ctx);
return 1;
}
@@ -1141,15 +1145,12 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
u64 bpf_jit_alloc_exec_limit(void)
{
- return BPF_JIT_REGION_SIZE;
+ return VMALLOC_END - VMALLOC_START;
}
void *bpf_jit_alloc_exec(unsigned long size)
{
- return __vmalloc_node_range(size, PAGE_SIZE, BPF_JIT_REGION_START,
- BPF_JIT_REGION_END, GFP_KERNEL,
- PAGE_KERNEL, 0, NUMA_NO_NODE,
- __builtin_return_address(0));
+ return vmalloc(size);
}
void bpf_jit_free_exec(void *addr)
diff --git a/arch/m68k/Kconfig b/arch/m68k/Kconfig
index 0b50da0..4cae3fb 100644
--- a/arch/m68k/Kconfig
+++ b/arch/m68k/Kconfig
@@ -9,6 +9,7 @@
select ARCH_HAVE_NMI_SAFE_CMPXCHG if RMW_INSNS
select ARCH_MIGHT_HAVE_PC_PARPORT if ISA
select ARCH_NO_PREEMPT if !COLDFIRE
+ select ARCH_USE_MEMTEST if MMU_MOTOROLA
select ARCH_WANT_IPC_PARSE_VERSION
select BINFMT_FLAT_ARGVP_ENVP_ON_STACK
select DMA_DIRECT_REMAP if HAS_DMA && MMU && !COLDFIRE
diff --git a/arch/m68k/configs/amiga_defconfig b/arch/m68k/configs/amiga_defconfig
index e2b14ea..a4b6c71 100644
--- a/arch/m68k/configs/amiga_defconfig
+++ b/arch/m68k/configs/amiga_defconfig
@@ -302,7 +302,6 @@
CONFIG_MPLS_IPTUNNEL=m
CONFIG_NET_NSH=m
CONFIG_AF_KCM=m
-CONFIG_MCTP=m
# CONFIG_WIRELESS is not set
CONFIG_PSAMPLE=m
CONFIG_NET_IFE=m
@@ -317,7 +316,6 @@
CONFIG_AMIGA_FLOPPY=y
CONFIG_AMIGA_Z2RAM=y
CONFIG_BLK_DEV_LOOP=y
-CONFIG_BLK_DEV_CRYPTOLOOP=m
CONFIG_BLK_DEV_DRBD=m
CONFIG_BLK_DEV_NBD=m
CONFIG_BLK_DEV_RAM=y
diff --git a/arch/m68k/configs/apollo_defconfig b/arch/m68k/configs/apollo_defconfig
index 41f3f3b..2db7219 100644
--- a/arch/m68k/configs/apollo_defconfig
+++ b/arch/m68k/configs/apollo_defconfig
@@ -298,7 +298,6 @@
CONFIG_MPLS_IPTUNNEL=m
CONFIG_NET_NSH=m
CONFIG_AF_KCM=m
-CONFIG_MCTP=m
# CONFIG_WIRELESS is not set
CONFIG_PSAMPLE=m
CONFIG_NET_IFE=m
@@ -307,7 +306,6 @@
CONFIG_TEST_ASYNC_DRIVER_PROBE=m
CONFIG_CONNECTOR=m
CONFIG_BLK_DEV_LOOP=y
-CONFIG_BLK_DEV_CRYPTOLOOP=m
CONFIG_BLK_DEV_DRBD=m
CONFIG_BLK_DEV_NBD=m
CONFIG_BLK_DEV_RAM=y
diff --git a/arch/m68k/configs/atari_defconfig b/arch/m68k/configs/atari_defconfig
index 3fd0ea1..c266a70 100644
--- a/arch/m68k/configs/atari_defconfig
+++ b/arch/m68k/configs/atari_defconfig
@@ -305,7 +305,6 @@
CONFIG_MPLS_IPTUNNEL=m
CONFIG_NET_NSH=m
CONFIG_AF_KCM=m
-CONFIG_MCTP=m
# CONFIG_WIRELESS is not set
CONFIG_PSAMPLE=m
CONFIG_NET_IFE=m
@@ -318,7 +317,6 @@
CONFIG_PARPORT_1284=y
CONFIG_ATARI_FLOPPY=y
CONFIG_BLK_DEV_LOOP=y
-CONFIG_BLK_DEV_CRYPTOLOOP=m
CONFIG_BLK_DEV_DRBD=m
CONFIG_BLK_DEV_NBD=m
CONFIG_BLK_DEV_RAM=y
diff --git a/arch/m68k/configs/bvme6000_defconfig b/arch/m68k/configs/bvme6000_defconfig
index 11a19f8..f644f08 100644
--- a/arch/m68k/configs/bvme6000_defconfig
+++ b/arch/m68k/configs/bvme6000_defconfig
@@ -295,7 +295,6 @@
CONFIG_MPLS_IPTUNNEL=m
CONFIG_NET_NSH=m
CONFIG_AF_KCM=m
-CONFIG_MCTP=m
# CONFIG_WIRELESS is not set
CONFIG_PSAMPLE=m
CONFIG_NET_IFE=m
@@ -304,7 +303,6 @@
CONFIG_TEST_ASYNC_DRIVER_PROBE=m
CONFIG_CONNECTOR=m
CONFIG_BLK_DEV_LOOP=y
-CONFIG_BLK_DEV_CRYPTOLOOP=m
CONFIG_BLK_DEV_DRBD=m
CONFIG_BLK_DEV_NBD=m
CONFIG_BLK_DEV_RAM=y
diff --git a/arch/m68k/configs/hp300_defconfig b/arch/m68k/configs/hp300_defconfig
index ee98127..e4924650 100644
--- a/arch/m68k/configs/hp300_defconfig
+++ b/arch/m68k/configs/hp300_defconfig
@@ -297,7 +297,6 @@
CONFIG_MPLS_IPTUNNEL=m
CONFIG_NET_NSH=m
CONFIG_AF_KCM=m
-CONFIG_MCTP=m
# CONFIG_WIRELESS is not set
CONFIG_PSAMPLE=m
CONFIG_NET_IFE=m
@@ -306,7 +305,6 @@
CONFIG_TEST_ASYNC_DRIVER_PROBE=m
CONFIG_CONNECTOR=m
CONFIG_BLK_DEV_LOOP=y
-CONFIG_BLK_DEV_CRYPTOLOOP=m
CONFIG_BLK_DEV_DRBD=m
CONFIG_BLK_DEV_NBD=m
CONFIG_BLK_DEV_RAM=y
diff --git a/arch/m68k/configs/mac_defconfig b/arch/m68k/configs/mac_defconfig
index a48656d..2411387 100644
--- a/arch/m68k/configs/mac_defconfig
+++ b/arch/m68k/configs/mac_defconfig
@@ -299,7 +299,6 @@
CONFIG_MPLS_IPTUNNEL=m
CONFIG_NET_NSH=m
CONFIG_AF_KCM=m
-CONFIG_MCTP=m
# CONFIG_WIRELESS is not set
CONFIG_PSAMPLE=m
CONFIG_NET_IFE=m
@@ -309,7 +308,6 @@
CONFIG_CONNECTOR=m
CONFIG_BLK_DEV_SWIM=m
CONFIG_BLK_DEV_LOOP=y
-CONFIG_BLK_DEV_CRYPTOLOOP=m
CONFIG_BLK_DEV_DRBD=m
CONFIG_BLK_DEV_NBD=m
CONFIG_BLK_DEV_RAM=y
diff --git a/arch/m68k/configs/multi_defconfig b/arch/m68k/configs/multi_defconfig
index b9b7a5c..6a7e4be 100644
--- a/arch/m68k/configs/multi_defconfig
+++ b/arch/m68k/configs/multi_defconfig
@@ -319,7 +319,6 @@
CONFIG_MPLS_IPTUNNEL=m
CONFIG_NET_NSH=m
CONFIG_AF_KCM=m
-CONFIG_MCTP=m
# CONFIG_WIRELESS is not set
CONFIG_PSAMPLE=m
CONFIG_NET_IFE=m
@@ -338,7 +337,6 @@
CONFIG_BLK_DEV_SWIM=m
CONFIG_AMIGA_Z2RAM=y
CONFIG_BLK_DEV_LOOP=y
-CONFIG_BLK_DEV_CRYPTOLOOP=m
CONFIG_BLK_DEV_DRBD=m
CONFIG_BLK_DEV_NBD=m
CONFIG_BLK_DEV_RAM=y
diff --git a/arch/m68k/configs/mvme147_defconfig b/arch/m68k/configs/mvme147_defconfig
index 3887b1c..1d22324 100644
--- a/arch/m68k/configs/mvme147_defconfig
+++ b/arch/m68k/configs/mvme147_defconfig
@@ -294,7 +294,6 @@
CONFIG_MPLS_IPTUNNEL=m
CONFIG_NET_NSH=m
CONFIG_AF_KCM=m
-CONFIG_MCTP=m
# CONFIG_WIRELESS is not set
CONFIG_PSAMPLE=m
CONFIG_NET_IFE=m
@@ -303,7 +302,6 @@
CONFIG_TEST_ASYNC_DRIVER_PROBE=m
CONFIG_CONNECTOR=m
CONFIG_BLK_DEV_LOOP=y
-CONFIG_BLK_DEV_CRYPTOLOOP=m
CONFIG_BLK_DEV_DRBD=m
CONFIG_BLK_DEV_NBD=m
CONFIG_BLK_DEV_RAM=y
diff --git a/arch/m68k/configs/mvme16x_defconfig b/arch/m68k/configs/mvme16x_defconfig
index 19e5992..961f789 100644
--- a/arch/m68k/configs/mvme16x_defconfig
+++ b/arch/m68k/configs/mvme16x_defconfig
@@ -295,7 +295,6 @@
CONFIG_MPLS_IPTUNNEL=m
CONFIG_NET_NSH=m
CONFIG_AF_KCM=m
-CONFIG_MCTP=m
# CONFIG_WIRELESS is not set
CONFIG_PSAMPLE=m
CONFIG_NET_IFE=m
@@ -304,7 +303,6 @@
CONFIG_TEST_ASYNC_DRIVER_PROBE=m
CONFIG_CONNECTOR=m
CONFIG_BLK_DEV_LOOP=y
-CONFIG_BLK_DEV_CRYPTOLOOP=m
CONFIG_BLK_DEV_DRBD=m
CONFIG_BLK_DEV_NBD=m
CONFIG_BLK_DEV_RAM=y
diff --git a/arch/m68k/configs/q40_defconfig b/arch/m68k/configs/q40_defconfig
index af25efa..ff4b5e4 100644
--- a/arch/m68k/configs/q40_defconfig
+++ b/arch/m68k/configs/q40_defconfig
@@ -296,7 +296,6 @@
CONFIG_MPLS_IPTUNNEL=m
CONFIG_NET_NSH=m
CONFIG_AF_KCM=m
-CONFIG_MCTP=m
# CONFIG_WIRELESS is not set
CONFIG_PSAMPLE=m
CONFIG_NET_IFE=m
@@ -308,7 +307,6 @@
CONFIG_PARPORT_PC=m
CONFIG_PARPORT_1284=y
CONFIG_BLK_DEV_LOOP=y
-CONFIG_BLK_DEV_CRYPTOLOOP=m
CONFIG_BLK_DEV_DRBD=m
CONFIG_BLK_DEV_NBD=m
CONFIG_BLK_DEV_RAM=y
diff --git a/arch/m68k/configs/sun3_defconfig b/arch/m68k/configs/sun3_defconfig
index 14dd96d..5f228621 100644
--- a/arch/m68k/configs/sun3_defconfig
+++ b/arch/m68k/configs/sun3_defconfig
@@ -292,7 +292,6 @@
CONFIG_MPLS_IPTUNNEL=m
CONFIG_NET_NSH=m
CONFIG_AF_KCM=m
-CONFIG_MCTP=m
# CONFIG_WIRELESS is not set
CONFIG_PSAMPLE=m
CONFIG_NET_IFE=m
@@ -301,7 +300,6 @@
CONFIG_TEST_ASYNC_DRIVER_PROBE=m
CONFIG_CONNECTOR=m
CONFIG_BLK_DEV_LOOP=y
-CONFIG_BLK_DEV_CRYPTOLOOP=m
CONFIG_BLK_DEV_DRBD=m
CONFIG_BLK_DEV_NBD=m
CONFIG_BLK_DEV_RAM=y
diff --git a/arch/m68k/configs/sun3x_defconfig b/arch/m68k/configs/sun3x_defconfig
index 8ac2402..a600cb9 100644
--- a/arch/m68k/configs/sun3x_defconfig
+++ b/arch/m68k/configs/sun3x_defconfig
@@ -292,7 +292,6 @@
CONFIG_MPLS_IPTUNNEL=m
CONFIG_NET_NSH=m
CONFIG_AF_KCM=m
-CONFIG_MCTP=m
# CONFIG_WIRELESS is not set
CONFIG_PSAMPLE=m
CONFIG_NET_IFE=m
@@ -301,7 +300,6 @@
CONFIG_TEST_ASYNC_DRIVER_PROBE=m
CONFIG_CONNECTOR=m
CONFIG_BLK_DEV_LOOP=y
-CONFIG_BLK_DEV_CRYPTOLOOP=m
CONFIG_BLK_DEV_DRBD=m
CONFIG_BLK_DEV_NBD=m
CONFIG_BLK_DEV_RAM=y
diff --git a/arch/m68k/kernel/setup_mm.c b/arch/m68k/kernel/setup_mm.c
index 4b51bfd..49e573b 100644
--- a/arch/m68k/kernel/setup_mm.c
+++ b/arch/m68k/kernel/setup_mm.c
@@ -338,13 +338,6 @@ void __init setup_arch(char **cmdline_p)
panic("No configuration setup");
}
- paging_init();
-
-#ifdef CONFIG_NATFEAT
- nf_init();
-#endif
-
-#ifndef CONFIG_SUN3
#ifdef CONFIG_BLK_DEV_INITRD
if (m68k_ramdisk.size) {
memblock_reserve(m68k_ramdisk.addr, m68k_ramdisk.size);
@@ -354,6 +347,12 @@ void __init setup_arch(char **cmdline_p)
}
#endif
+ paging_init();
+
+#ifdef CONFIG_NATFEAT
+ nf_init();
+#endif
+
#ifdef CONFIG_ATARI
if (MACH_IS_ATARI)
atari_stram_reserve_pages((void *)availmem);
@@ -364,8 +363,6 @@ void __init setup_arch(char **cmdline_p)
}
#endif
-#endif /* !CONFIG_SUN3 */
-
/* set ISA defs early as possible */
#if defined(CONFIG_ISA) && defined(MULTI_ISA)
if (MACH_IS_Q40) {
diff --git a/arch/m68k/mm/motorola.c b/arch/m68k/mm/motorola.c
index 2b05bb2..ecbe948 100644
--- a/arch/m68k/mm/motorola.c
+++ b/arch/m68k/mm/motorola.c
@@ -457,6 +457,8 @@ void __init paging_init(void)
flush_tlb_all();
+ early_memtest(min_addr, max_addr);
+
/*
* initialize the bad page table and bad page to point
* to a couple of allocated pages
diff --git a/arch/microblaze/kernel/signal.c b/arch/microblaze/kernel/signal.c
index fc61eb0..23e8a93 100644
--- a/arch/microblaze/kernel/signal.c
+++ b/arch/microblaze/kernel/signal.c
@@ -283,7 +283,7 @@ static void do_signal(struct pt_regs *regs, int in_syscall)
#ifdef DEBUG_SIG
pr_info("do signal: %p %d\n", regs, in_syscall);
pr_info("do signal2: %lx %lx %ld [%lx]\n", regs->pc, regs->r1,
- regs->r12, current_thread_info()->flags);
+ regs->r12, read_thread_flags());
#endif
if (get_signal(&ksig)) {
diff --git a/arch/mips/configs/decstation_64_defconfig b/arch/mips/configs/decstation_64_defconfig
index 85f1955..e2ed105 100644
--- a/arch/mips/configs/decstation_64_defconfig
+++ b/arch/mips/configs/decstation_64_defconfig
@@ -144,7 +144,6 @@
CONFIG_EXT3_FS=y
CONFIG_EXT3_FS_POSIX_ACL=y
CONFIG_EXT3_FS_SECURITY=y
-# CONFIG_MANDATORY_FILE_LOCKING is not set
CONFIG_ISO9660_FS=y
CONFIG_JOLIET=y
CONFIG_PROC_KCORE=y
diff --git a/arch/mips/configs/decstation_defconfig b/arch/mips/configs/decstation_defconfig
index 30a6eaf..7e987d6 100644
--- a/arch/mips/configs/decstation_defconfig
+++ b/arch/mips/configs/decstation_defconfig
@@ -140,7 +140,6 @@
CONFIG_EXT3_FS=y
CONFIG_EXT3_FS_POSIX_ACL=y
CONFIG_EXT3_FS_SECURITY=y
-# CONFIG_MANDATORY_FILE_LOCKING is not set
CONFIG_ISO9660_FS=y
CONFIG_JOLIET=y
CONFIG_PROC_KCORE=y
diff --git a/arch/mips/configs/decstation_r4k_defconfig b/arch/mips/configs/decstation_r4k_defconfig
index e2b58db..6df5f6f 100644
--- a/arch/mips/configs/decstation_r4k_defconfig
+++ b/arch/mips/configs/decstation_r4k_defconfig
@@ -140,7 +140,6 @@
CONFIG_EXT3_FS=y
CONFIG_EXT3_FS_POSIX_ACL=y
CONFIG_EXT3_FS_SECURITY=y
-# CONFIG_MANDATORY_FILE_LOCKING is not set
CONFIG_ISO9660_FS=y
CONFIG_JOLIET=y
CONFIG_PROC_KCORE=y
diff --git a/arch/mips/configs/fuloong2e_defconfig b/arch/mips/configs/fuloong2e_defconfig
index 5c24ac7..ba47c5e 100644
--- a/arch/mips/configs/fuloong2e_defconfig
+++ b/arch/mips/configs/fuloong2e_defconfig
@@ -206,7 +206,6 @@
CONFIG_NFSD_V4=y
CONFIG_CIFS=m
CONFIG_CIFS_STATS2=y
-CONFIG_CIFS_WEAK_PW_HASH=y
CONFIG_CIFS_XATTR=y
CONFIG_CIFS_POSIX=y
CONFIG_CIFS_DEBUG2=y
diff --git a/arch/mips/configs/ip27_defconfig b/arch/mips/configs/ip27_defconfig
index 638d7cf..821630a 100644
--- a/arch/mips/configs/ip27_defconfig
+++ b/arch/mips/configs/ip27_defconfig
@@ -223,7 +223,6 @@
CONFIG_NORTEL_HERMES=m
CONFIG_P54_COMMON=m
CONFIG_P54_PCI=m
-CONFIG_PRISM54=m
CONFIG_LIBERTAS=m
CONFIG_LIBERTAS_THINFIRM=m
CONFIG_MWL8K=m
diff --git a/arch/mips/configs/malta_defconfig b/arch/mips/configs/malta_defconfig
index 9cb2cf2..3321bb5 100644
--- a/arch/mips/configs/malta_defconfig
+++ b/arch/mips/configs/malta_defconfig
@@ -302,7 +302,6 @@
CONFIG_HOSTAP_FIRMWARE_NVRAM=y
CONFIG_HOSTAP_PLX=m
CONFIG_HOSTAP_PCI=m
-CONFIG_PRISM54=m
CONFIG_LIBERTAS=m
CONFIG_INPUT_MOUSEDEV=y
CONFIG_MOUSE_PS2_ELANTECH=y
diff --git a/arch/mips/configs/malta_kvm_defconfig b/arch/mips/configs/malta_kvm_defconfig
index 5924e48..009b303 100644
--- a/arch/mips/configs/malta_kvm_defconfig
+++ b/arch/mips/configs/malta_kvm_defconfig
@@ -310,7 +310,6 @@
CONFIG_HOSTAP_FIRMWARE_NVRAM=y
CONFIG_HOSTAP_PLX=m
CONFIG_HOSTAP_PCI=m
-CONFIG_PRISM54=m
CONFIG_LIBERTAS=m
CONFIG_INPUT_MOUSEDEV=y
CONFIG_SERIAL_8250=y
diff --git a/arch/mips/configs/malta_qemu_32r6_defconfig b/arch/mips/configs/malta_qemu_32r6_defconfig
index 614af02..6fb9bc2 100644
--- a/arch/mips/configs/malta_qemu_32r6_defconfig
+++ b/arch/mips/configs/malta_qemu_32r6_defconfig
@@ -165,7 +165,6 @@
CONFIG_NFS_FS=y
CONFIG_ROOT_NFS=y
CONFIG_CIFS=m
-CONFIG_CIFS_WEAK_PW_HASH=y
CONFIG_CIFS_XATTR=y
CONFIG_CIFS_POSIX=y
CONFIG_NLS_CODEPAGE_437=m
diff --git a/arch/mips/configs/maltaaprp_defconfig b/arch/mips/configs/maltaaprp_defconfig
index 9c051f8..eb72df5 100644
--- a/arch/mips/configs/maltaaprp_defconfig
+++ b/arch/mips/configs/maltaaprp_defconfig
@@ -166,7 +166,6 @@
CONFIG_NFS_FS=y
CONFIG_ROOT_NFS=y
CONFIG_CIFS=m
-CONFIG_CIFS_WEAK_PW_HASH=y
CONFIG_CIFS_XATTR=y
CONFIG_CIFS_POSIX=y
CONFIG_NLS_CODEPAGE_437=m
diff --git a/arch/mips/configs/maltasmvp_defconfig b/arch/mips/configs/maltasmvp_defconfig
index 2e90d97..1fb40d3 100644
--- a/arch/mips/configs/maltasmvp_defconfig
+++ b/arch/mips/configs/maltasmvp_defconfig
@@ -167,7 +167,6 @@
CONFIG_NFS_FS=y
CONFIG_ROOT_NFS=y
CONFIG_CIFS=m
-CONFIG_CIFS_WEAK_PW_HASH=y
CONFIG_CIFS_XATTR=y
CONFIG_CIFS_POSIX=y
CONFIG_NLS_CODEPAGE_437=m
diff --git a/arch/mips/configs/maltasmvp_eva_defconfig b/arch/mips/configs/maltasmvp_eva_defconfig
index d1f7fdb..75cb778 100644
--- a/arch/mips/configs/maltasmvp_eva_defconfig
+++ b/arch/mips/configs/maltasmvp_eva_defconfig
@@ -169,7 +169,6 @@
CONFIG_NFS_FS=y
CONFIG_ROOT_NFS=y
CONFIG_CIFS=m
-CONFIG_CIFS_WEAK_PW_HASH=y
CONFIG_CIFS_XATTR=y
CONFIG_CIFS_POSIX=y
CONFIG_NLS_CODEPAGE_437=m
diff --git a/arch/mips/configs/maltaup_defconfig b/arch/mips/configs/maltaup_defconfig
index 48e5bd4..7b4f247 100644
--- a/arch/mips/configs/maltaup_defconfig
+++ b/arch/mips/configs/maltaup_defconfig
@@ -165,7 +165,6 @@
CONFIG_NFS_FS=y
CONFIG_ROOT_NFS=y
CONFIG_CIFS=m
-CONFIG_CIFS_WEAK_PW_HASH=y
CONFIG_CIFS_XATTR=y
CONFIG_CIFS_POSIX=y
CONFIG_NLS_CODEPAGE_437=m
diff --git a/arch/mips/configs/maltaup_xpa_defconfig b/arch/mips/configs/maltaup_xpa_defconfig
index c0d3156ef..e214e13 100644
--- a/arch/mips/configs/maltaup_xpa_defconfig
+++ b/arch/mips/configs/maltaup_xpa_defconfig
@@ -309,7 +309,6 @@
CONFIG_HOSTAP_FIRMWARE_NVRAM=y
CONFIG_HOSTAP_PLX=m
CONFIG_HOSTAP_PCI=m
-CONFIG_PRISM54=m
CONFIG_LIBERTAS=m
CONFIG_INPUT_MOUSEDEV=y
CONFIG_MOUSE_PS2_ELANTECH=y
diff --git a/arch/mips/include/asm/mach-lantiq/xway/xway_dma.h b/arch/mips/include/asm/mach-lantiq/xway/xway_dma.h
index 31ca915..3dee15c 100644
--- a/arch/mips/include/asm/mach-lantiq/xway/xway_dma.h
+++ b/arch/mips/include/asm/mach-lantiq/xway/xway_dma.h
@@ -8,7 +8,7 @@
#define LTQ_DMA_H__
#define LTQ_DESC_SIZE 0x08 /* each descriptor is 64bit */
-#define LTQ_DESC_NUM 0x40 /* 64 descriptors / channel */
+#define LTQ_DESC_NUM 0xC0 /* 192 descriptors / channel */
#define LTQ_DMA_OWN BIT(31) /* owner bit */
#define LTQ_DMA_C BIT(30) /* complete bit */
diff --git a/arch/mips/net/bpf_jit_comp32.c b/arch/mips/net/bpf_jit_comp32.c
index bd996ed..044b11b 100644
--- a/arch/mips/net/bpf_jit_comp32.c
+++ b/arch/mips/net/bpf_jit_comp32.c
@@ -1381,8 +1381,7 @@ void build_prologue(struct jit_context *ctx)
* 16-byte area in the parent's stack frame. On a tail call, the
* calling function jumps into the prologue after these instructions.
*/
- emit(ctx, ori, MIPS_R_T9, MIPS_R_ZERO,
- min(MAX_TAIL_CALL_CNT + 1, 0xffff));
+ emit(ctx, ori, MIPS_R_T9, MIPS_R_ZERO, min(MAX_TAIL_CALL_CNT, 0xffff));
emit(ctx, sw, MIPS_R_T9, 0, MIPS_R_SP);
/*
diff --git a/arch/mips/net/bpf_jit_comp64.c b/arch/mips/net/bpf_jit_comp64.c
index 815ade7..6475828 100644
--- a/arch/mips/net/bpf_jit_comp64.c
+++ b/arch/mips/net/bpf_jit_comp64.c
@@ -552,7 +552,7 @@ void build_prologue(struct jit_context *ctx)
* On a tail call, the calling function jumps into the prologue
* after this instruction.
*/
- emit(ctx, addiu, tc, MIPS_R_ZERO, min(MAX_TAIL_CALL_CNT + 1, 0xffff));
+ emit(ctx, ori, tc, MIPS_R_ZERO, min(MAX_TAIL_CALL_CNT, 0xffff));
/* === Entry-point for tail calls === */
diff --git a/arch/openrisc/kernel/signal.c b/arch/openrisc/kernel/signal.c
index 99516c9..92c5b70 100644
--- a/arch/openrisc/kernel/signal.c
+++ b/arch/openrisc/kernel/signal.c
@@ -313,7 +313,7 @@ do_work_pending(struct pt_regs *regs, unsigned int thread_flags, int syscall)
}
}
local_irq_disable();
- thread_flags = current_thread_info()->flags;
+ thread_flags = read_thread_flags();
} while (thread_flags & _TIF_WORK_MASK);
return 0;
}
diff --git a/arch/powerpc/configs/pmac32_defconfig b/arch/powerpc/configs/pmac32_defconfig
index 7aefac5..13885ec 100644
--- a/arch/powerpc/configs/pmac32_defconfig
+++ b/arch/powerpc/configs/pmac32_defconfig
@@ -169,7 +169,6 @@
CONFIG_B43=m
CONFIG_B43LEGACY=m
CONFIG_P54_COMMON=m
-CONFIG_PRISM54=m
CONFIG_INPUT_EVDEV=y
# CONFIG_KEYBOARD_ATKBD is not set
# CONFIG_MOUSE_PS2 is not set
diff --git a/arch/powerpc/configs/ppc6xx_defconfig b/arch/powerpc/configs/ppc6xx_defconfig
index 6697c5e..bb549cb 100644
--- a/arch/powerpc/configs/ppc6xx_defconfig
+++ b/arch/powerpc/configs/ppc6xx_defconfig
@@ -1022,7 +1022,6 @@
CONFIG_NFSD_V3_ACL=y
CONFIG_NFSD_V4=y
CONFIG_CIFS=m
-CONFIG_CIFS_WEAK_PW_HASH=y
CONFIG_CIFS_UPCALL=y
CONFIG_CIFS_XATTR=y
CONFIG_CIFS_POSIX=y
diff --git a/arch/powerpc/configs/pseries_defconfig b/arch/powerpc/configs/pseries_defconfig
index de7641a..e64f224 100644
--- a/arch/powerpc/configs/pseries_defconfig
+++ b/arch/powerpc/configs/pseries_defconfig
@@ -189,7 +189,6 @@
CONFIG_VIRTIO_CONSOLE=m
CONFIG_IBM_BSR=m
CONFIG_RAW_DRIVER=y
-CONFIG_MAX_RAW_DEVS=1024
CONFIG_I2C_CHARDEV=y
CONFIG_FB=y
CONFIG_FIRMWARE_EDID=y
diff --git a/arch/powerpc/kernel/Makefile b/arch/powerpc/kernel/Makefile
index 5fa68c2..b039877 100644
--- a/arch/powerpc/kernel/Makefile
+++ b/arch/powerpc/kernel/Makefile
@@ -47,7 +47,7 @@
udbg.o misc.o io.o misc_$(BITS).o \
of_platform.o prom_parse.o firmware.o \
hw_breakpoint_constraints.o interrupt.o \
- kdebugfs.o
+ kdebugfs.o stacktrace.o
obj-y += ptrace/
obj-$(CONFIG_PPC64) += setup_64.o \
paca.o nvram_64.o note.o
@@ -116,7 +116,6 @@
obj-$(CONFIG_KPROBES_ON_FTRACE) += kprobes-ftrace.o
obj-$(CONFIG_UPROBES) += uprobes.o
obj-$(CONFIG_PPC_UDBG_16550) += legacy_serial.o udbg_16550.o
-obj-$(CONFIG_STACKTRACE) += stacktrace.o
obj-$(CONFIG_SWIOTLB) += dma-swiotlb.o
obj-$(CONFIG_ARCH_HAS_DMA_SET_MASK) += dma-mask.o
diff --git a/arch/powerpc/kernel/interrupt.c b/arch/powerpc/kernel/interrupt.c
index 835b626..563ebfc 100644
--- a/arch/powerpc/kernel/interrupt.c
+++ b/arch/powerpc/kernel/interrupt.c
@@ -148,7 +148,7 @@ notrace long system_call_exception(long r3, long r4, long r5,
*/
if (IS_ENABLED(CONFIG_PPC_TRANSACTIONAL_MEM) &&
unlikely(MSR_TM_TRANSACTIONAL(regs->msr)))
- current_thread_info()->flags |= _TIF_RESTOREALL;
+ set_bits(_TIF_RESTOREALL, ¤t_thread_info()->flags);
/*
* If the system call was made with a transaction active, doom it and
@@ -181,7 +181,7 @@ notrace long system_call_exception(long r3, long r4, long r5,
local_irq_enable();
- if (unlikely(current_thread_info()->flags & _TIF_SYSCALL_DOTRACE)) {
+ if (unlikely(read_thread_flags() & _TIF_SYSCALL_DOTRACE)) {
if (unlikely(trap_is_unsupported_scv(regs))) {
/* Unsupported scv vector */
_exception(SIGILL, regs, ILL_ILLOPC, regs->nip);
@@ -343,7 +343,7 @@ interrupt_exit_user_prepare_main(unsigned long ret, struct pt_regs *regs)
unsigned long ti_flags;
again:
- ti_flags = READ_ONCE(current_thread_info()->flags);
+ ti_flags = read_thread_flags();
while (unlikely(ti_flags & (_TIF_USER_WORK_MASK & ~_TIF_RESTORE_TM))) {
local_irq_enable();
if (ti_flags & _TIF_NEED_RESCHED) {
@@ -359,7 +359,7 @@ interrupt_exit_user_prepare_main(unsigned long ret, struct pt_regs *regs)
do_notify_resume(regs, ti_flags);
}
local_irq_disable();
- ti_flags = READ_ONCE(current_thread_info()->flags);
+ ti_flags = read_thread_flags();
}
if (IS_ENABLED(CONFIG_PPC_BOOK3S_64) && IS_ENABLED(CONFIG_PPC_FPU)) {
@@ -437,7 +437,7 @@ notrace unsigned long syscall_exit_prepare(unsigned long r3,
/* Check whether the syscall is issued inside a restartable sequence */
rseq_syscall(regs);
- ti_flags = current_thread_info()->flags;
+ ti_flags = read_thread_flags();
if (unlikely(r3 >= (unsigned long)-MAX_ERRNO) && is_not_scv) {
if (likely(!(ti_flags & (_TIF_NOERROR | _TIF_RESTOREALL)))) {
@@ -532,8 +532,7 @@ notrace unsigned long interrupt_exit_kernel_prepare(struct pt_regs *regs)
unsigned long flags;
unsigned long ret = 0;
unsigned long kuap;
- bool stack_store = current_thread_info()->flags &
- _TIF_EMULATE_STACK_STORE;
+ bool stack_store = read_thread_flags() & _TIF_EMULATE_STACK_STORE;
if (regs_is_unrecoverable(regs))
unrecoverable_exception(regs);
@@ -554,7 +553,7 @@ notrace unsigned long interrupt_exit_kernel_prepare(struct pt_regs *regs)
again:
if (IS_ENABLED(CONFIG_PREEMPT)) {
/* Return to preemptible kernel context */
- if (unlikely(current_thread_info()->flags & _TIF_NEED_RESCHED)) {
+ if (unlikely(read_thread_flags() & _TIF_NEED_RESCHED)) {
if (preempt_count() == 0)
preempt_schedule_irq();
}
diff --git a/arch/powerpc/kernel/ptrace/ptrace.c b/arch/powerpc/kernel/ptrace/ptrace.c
index 7c7093c..c43f77e 100644
--- a/arch/powerpc/kernel/ptrace/ptrace.c
+++ b/arch/powerpc/kernel/ptrace/ptrace.c
@@ -260,8 +260,7 @@ long do_syscall_trace_enter(struct pt_regs *regs)
{
u32 flags;
- flags = READ_ONCE(current_thread_info()->flags) &
- (_TIF_SYSCALL_EMU | _TIF_SYSCALL_TRACE);
+ flags = read_thread_flags() & (_TIF_SYSCALL_EMU | _TIF_SYSCALL_TRACE);
if (flags) {
int rc = tracehook_report_syscall_entry(regs);
diff --git a/arch/powerpc/mm/ptdump/ptdump.c b/arch/powerpc/mm/ptdump/ptdump.c
index bf25119..32bfb21 100644
--- a/arch/powerpc/mm/ptdump/ptdump.c
+++ b/arch/powerpc/mm/ptdump/ptdump.c
@@ -183,7 +183,7 @@ static void note_prot_wx(struct pg_state *st, unsigned long addr)
{
pte_t pte = __pte(st->current_flags);
- if (!IS_ENABLED(CONFIG_PPC_DEBUG_WX) || !st->check_wx)
+ if (!IS_ENABLED(CONFIG_DEBUG_WX) || !st->check_wx)
return;
if (!pte_write(pte) || !pte_exec(pte))
diff --git a/arch/powerpc/net/bpf_jit_comp32.c b/arch/powerpc/net/bpf_jit_comp32.c
index 0da31d4..8a4faa0 100644
--- a/arch/powerpc/net/bpf_jit_comp32.c
+++ b/arch/powerpc/net/bpf_jit_comp32.c
@@ -221,13 +221,13 @@ static int bpf_jit_emit_tail_call(u32 *image, struct codegen_context *ctx, u32 o
PPC_BCC(COND_GE, out);
/*
- * if (tail_call_cnt > MAX_TAIL_CALL_CNT)
+ * if (tail_call_cnt >= MAX_TAIL_CALL_CNT)
* goto out;
*/
EMIT(PPC_RAW_CMPLWI(_R0, MAX_TAIL_CALL_CNT));
/* tail_call_cnt++; */
EMIT(PPC_RAW_ADDIC(_R0, _R0, 1));
- PPC_BCC(COND_GT, out);
+ PPC_BCC(COND_GE, out);
/* prog = array->ptrs[index]; */
EMIT(PPC_RAW_RLWINM(_R3, b2p_index, 2, 0, 29));
diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_comp64.c
index 8b5157c..8571aaf 100644
--- a/arch/powerpc/net/bpf_jit_comp64.c
+++ b/arch/powerpc/net/bpf_jit_comp64.c
@@ -228,12 +228,12 @@ static int bpf_jit_emit_tail_call(u32 *image, struct codegen_context *ctx, u32 o
PPC_BCC(COND_GE, out);
/*
- * if (tail_call_cnt > MAX_TAIL_CALL_CNT)
+ * if (tail_call_cnt >= MAX_TAIL_CALL_CNT)
* goto out;
*/
PPC_BPF_LL(b2p[TMP_REG_1], 1, bpf_jit_stack_tailcallcnt(ctx));
EMIT(PPC_RAW_CMPLWI(b2p[TMP_REG_1], MAX_TAIL_CALL_CNT));
- PPC_BCC(COND_GT, out);
+ PPC_BCC(COND_GE, out);
/*
* tail_call_cnt++;
diff --git a/arch/riscv/Kconfig.socs b/arch/riscv/Kconfig.socs
index 30676eb..6ec44a2 100644
--- a/arch/riscv/Kconfig.socs
+++ b/arch/riscv/Kconfig.socs
@@ -19,6 +19,14 @@
help
This enables support for SiFive SoC platform hardware.
+config SOC_STARFIVE
+ bool "StarFive SoCs"
+ select PINCTRL
+ select RESET_CONTROLLER
+ select SIFIVE_PLIC
+ help
+ This enables support for StarFive SoC platform hardware.
+
config SOC_VIRT
bool "QEMU Virt Machine"
select CLINT_TIMER if RISCV_M_MODE
diff --git a/arch/riscv/boot/dts/Makefile b/arch/riscv/boot/dts/Makefile
index fe996b8..ff17499 100644
--- a/arch/riscv/boot/dts/Makefile
+++ b/arch/riscv/boot/dts/Makefile
@@ -1,5 +1,6 @@
# SPDX-License-Identifier: GPL-2.0
subdir-y += sifive
+subdir-y += starfive
subdir-$(CONFIG_SOC_CANAAN_K210_DTB_BUILTIN) += canaan
subdir-y += microchip
diff --git a/arch/riscv/boot/dts/starfive/Makefile b/arch/riscv/boot/dts/starfive/Makefile
new file mode 100644
index 0000000..0ea1bc1
--- /dev/null
+++ b/arch/riscv/boot/dts/starfive/Makefile
@@ -0,0 +1,2 @@
+# SPDX-License-Identifier: GPL-2.0
+dtb-$(CONFIG_SOC_STARFIVE) += jh7100-beaglev-starlight.dtb
diff --git a/arch/riscv/boot/dts/starfive/jh7100-beaglev-starlight.dts b/arch/riscv/boot/dts/starfive/jh7100-beaglev-starlight.dts
new file mode 100644
index 0000000..c9af67f
--- /dev/null
+++ b/arch/riscv/boot/dts/starfive/jh7100-beaglev-starlight.dts
@@ -0,0 +1,164 @@
+// SPDX-License-Identifier: GPL-2.0 OR MIT
+/*
+ * Copyright (C) 2021 StarFive Technology Co., Ltd.
+ * Copyright (C) 2021 Emil Renner Berthing <kernel@esmil.dk>
+ */
+
+/dts-v1/;
+#include "jh7100.dtsi"
+#include <dt-bindings/gpio/gpio.h>
+#include <dt-bindings/leds/common.h>
+#include <dt-bindings/pinctrl/pinctrl-starfive.h>
+
+/ {
+ model = "BeagleV Starlight Beta";
+ compatible = "beagle,beaglev-starlight-jh7100-r0", "starfive,jh7100";
+
+ aliases {
+ serial0 = &uart3;
+ };
+
+ chosen {
+ stdout-path = "serial0:115200n8";
+ };
+
+ cpus {
+ timebase-frequency = <6250000>;
+ };
+
+ memory@80000000 {
+ device_type = "memory";
+ reg = <0x0 0x80000000 0x2 0x0>;
+ };
+
+ leds {
+ compatible = "gpio-leds";
+
+ led-ack {
+ gpios = <&gpio 43 GPIO_ACTIVE_HIGH>;
+ color = <LED_COLOR_ID_GREEN>;
+ function = LED_FUNCTION_HEARTBEAT;
+ linux,default-trigger = "heartbeat";
+ label = "ack";
+ };
+ };
+};
+
+&gpio {
+ i2c0_pins: i2c0-0 {
+ i2c-pins {
+ pinmux = <GPIOMUX(62, GPO_LOW,
+ GPO_I2C0_PAD_SCK_OEN,
+ GPI_I2C0_PAD_SCK_IN)>,
+ <GPIOMUX(61, GPO_LOW,
+ GPO_I2C0_PAD_SDA_OEN,
+ GPI_I2C0_PAD_SDA_IN)>;
+ bias-disable; /* external pull-up */
+ input-enable;
+ input-schmitt-enable;
+ };
+ };
+
+ i2c1_pins: i2c1-0 {
+ i2c-pins {
+ pinmux = <GPIOMUX(47, GPO_LOW,
+ GPO_I2C1_PAD_SCK_OEN,
+ GPI_I2C1_PAD_SCK_IN)>,
+ <GPIOMUX(48, GPO_LOW,
+ GPO_I2C1_PAD_SDA_OEN,
+ GPI_I2C1_PAD_SDA_IN)>;
+ bias-pull-up;
+ input-enable;
+ input-schmitt-enable;
+ };
+ };
+
+ i2c2_pins: i2c2-0 {
+ i2c-pins {
+ pinmux = <GPIOMUX(60, GPO_LOW,
+ GPO_I2C2_PAD_SCK_OEN,
+ GPI_I2C2_PAD_SCK_IN)>,
+ <GPIOMUX(59, GPO_LOW,
+ GPO_I2C2_PAD_SDA_OEN,
+ GPI_I2C2_PAD_SDA_IN)>;
+ bias-disable; /* external pull-up */
+ input-enable;
+ input-schmitt-enable;
+ };
+ };
+
+ uart3_pins: uart3-0 {
+ rx-pins {
+ pinmux = <GPIOMUX(13, GPO_LOW, GPO_DISABLE,
+ GPI_UART3_PAD_SIN)>;
+ bias-pull-up;
+ drive-strength = <14>;
+ input-enable;
+ input-schmitt-enable;
+ slew-rate = <0>;
+ };
+ tx-pins {
+ pinmux = <GPIOMUX(14, GPO_UART3_PAD_SOUT,
+ GPO_ENABLE, GPI_NONE)>;
+ bias-disable;
+ drive-strength = <35>;
+ input-disable;
+ input-schmitt-disable;
+ slew-rate = <0>;
+ };
+ };
+};
+
+&i2c0 {
+ clock-frequency = <100000>;
+ i2c-sda-hold-time-ns = <300>;
+ i2c-sda-falling-time-ns = <500>;
+ i2c-scl-falling-time-ns = <500>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&i2c0_pins>;
+ status = "okay";
+
+ pmic@5e {
+ compatible = "ti,tps65086";
+ reg = <0x5e>;
+ gpio-controller;
+ #gpio-cells = <2>;
+
+ regulators {
+ };
+ };
+};
+
+&i2c1 {
+ clock-frequency = <400000>;
+ i2c-sda-hold-time-ns = <300>;
+ i2c-sda-falling-time-ns = <100>;
+ i2c-scl-falling-time-ns = <100>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&i2c1_pins>;
+ status = "okay";
+};
+
+&i2c2 {
+ clock-frequency = <100000>;
+ i2c-sda-hold-time-ns = <300>;
+ i2c-sda-falling-time-ns = <500>;
+ i2c-scl-falling-time-ns = <500>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&i2c2_pins>;
+ status = "okay";
+};
+
+&osc_sys {
+ clock-frequency = <25000000>;
+};
+
+&osc_aud {
+ clock-frequency = <27000000>;
+};
+
+&uart3 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&uart3_pins>;
+ status = "okay";
+};
diff --git a/arch/riscv/boot/dts/starfive/jh7100.dtsi b/arch/riscv/boot/dts/starfive/jh7100.dtsi
new file mode 100644
index 0000000..69f22f9
--- /dev/null
+++ b/arch/riscv/boot/dts/starfive/jh7100.dtsi
@@ -0,0 +1,230 @@
+// SPDX-License-Identifier: GPL-2.0 OR MIT
+/*
+ * Copyright (C) 2021 StarFive Technology Co., Ltd.
+ * Copyright (C) 2021 Emil Renner Berthing <kernel@esmil.dk>
+ */
+
+/dts-v1/;
+#include <dt-bindings/clock/starfive-jh7100.h>
+#include <dt-bindings/reset/starfive-jh7100.h>
+
+/ {
+ compatible = "starfive,jh7100";
+ #address-cells = <2>;
+ #size-cells = <2>;
+
+ cpus {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ cpu@0 {
+ compatible = "sifive,u74-mc", "riscv";
+ reg = <0>;
+ d-cache-block-size = <64>;
+ d-cache-sets = <64>;
+ d-cache-size = <32768>;
+ d-tlb-sets = <1>;
+ d-tlb-size = <32>;
+ device_type = "cpu";
+ i-cache-block-size = <64>;
+ i-cache-sets = <64>;
+ i-cache-size = <32768>;
+ i-tlb-sets = <1>;
+ i-tlb-size = <32>;
+ mmu-type = "riscv,sv39";
+ riscv,isa = "rv64imafdc";
+ tlb-split;
+
+ cpu0_intc: interrupt-controller {
+ compatible = "riscv,cpu-intc";
+ interrupt-controller;
+ #interrupt-cells = <1>;
+ };
+ };
+
+ cpu@1 {
+ compatible = "sifive,u74-mc", "riscv";
+ reg = <1>;
+ d-cache-block-size = <64>;
+ d-cache-sets = <64>;
+ d-cache-size = <32768>;
+ d-tlb-sets = <1>;
+ d-tlb-size = <32>;
+ device_type = "cpu";
+ i-cache-block-size = <64>;
+ i-cache-sets = <64>;
+ i-cache-size = <32768>;
+ i-tlb-sets = <1>;
+ i-tlb-size = <32>;
+ mmu-type = "riscv,sv39";
+ riscv,isa = "rv64imafdc";
+ tlb-split;
+
+ cpu1_intc: interrupt-controller {
+ compatible = "riscv,cpu-intc";
+ interrupt-controller;
+ #interrupt-cells = <1>;
+ };
+ };
+ };
+
+ osc_sys: osc_sys {
+ compatible = "fixed-clock";
+ #clock-cells = <0>;
+ /* This value must be overridden by the board */
+ clock-frequency = <0>;
+ };
+
+ osc_aud: osc_aud {
+ compatible = "fixed-clock";
+ #clock-cells = <0>;
+ /* This value must be overridden by the board */
+ clock-frequency = <0>;
+ };
+
+ gmac_rmii_ref: gmac_rmii_ref {
+ compatible = "fixed-clock";
+ #clock-cells = <0>;
+ /* Should be overridden by the board when needed */
+ clock-frequency = <0>;
+ };
+
+ gmac_gr_mii_rxclk: gmac_gr_mii_rxclk {
+ compatible = "fixed-clock";
+ #clock-cells = <0>;
+ /* Should be overridden by the board when needed */
+ clock-frequency = <0>;
+ };
+
+ soc {
+ compatible = "simple-bus";
+ interrupt-parent = <&plic>;
+ #address-cells = <2>;
+ #size-cells = <2>;
+ ranges;
+
+ clint: clint@2000000 {
+ compatible = "starfive,jh7100-clint", "sifive,clint0";
+ reg = <0x0 0x2000000 0x0 0x10000>;
+ interrupts-extended = <&cpu0_intc 3 &cpu0_intc 7
+ &cpu1_intc 3 &cpu1_intc 7>;
+ };
+
+ plic: interrupt-controller@c000000 {
+ compatible = "starfive,jh7100-plic", "sifive,plic-1.0.0";
+ reg = <0x0 0xc000000 0x0 0x4000000>;
+ interrupts-extended = <&cpu0_intc 11 &cpu0_intc 9
+ &cpu1_intc 11 &cpu1_intc 9>;
+ interrupt-controller;
+ #address-cells = <0>;
+ #interrupt-cells = <1>;
+ riscv,ndev = <127>;
+ };
+
+ clkgen: clock-controller@11800000 {
+ compatible = "starfive,jh7100-clkgen";
+ reg = <0x0 0x11800000 0x0 0x10000>;
+ clocks = <&osc_sys>, <&osc_aud>, <&gmac_rmii_ref>, <&gmac_gr_mii_rxclk>;
+ clock-names = "osc_sys", "osc_aud", "gmac_rmii_ref", "gmac_gr_mii_rxclk";
+ #clock-cells = <1>;
+ };
+
+ rstgen: reset-controller@11840000 {
+ compatible = "starfive,jh7100-reset";
+ reg = <0x0 0x11840000 0x0 0x10000>;
+ #reset-cells = <1>;
+ };
+
+ i2c0: i2c@118b0000 {
+ compatible = "snps,designware-i2c";
+ reg = <0x0 0x118b0000 0x0 0x10000>;
+ clocks = <&clkgen JH7100_CLK_I2C0_CORE>,
+ <&clkgen JH7100_CLK_I2C0_APB>;
+ clock-names = "ref", "pclk";
+ resets = <&rstgen JH7100_RSTN_I2C0_APB>;
+ interrupts = <96>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ i2c1: i2c@118c0000 {
+ compatible = "snps,designware-i2c";
+ reg = <0x0 0x118c0000 0x0 0x10000>;
+ clocks = <&clkgen JH7100_CLK_I2C1_CORE>,
+ <&clkgen JH7100_CLK_I2C1_APB>;
+ clock-names = "ref", "pclk";
+ resets = <&rstgen JH7100_RSTN_I2C1_APB>;
+ interrupts = <97>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ gpio: pinctrl@11910000 {
+ compatible = "starfive,jh7100-pinctrl";
+ reg = <0x0 0x11910000 0x0 0x10000>,
+ <0x0 0x11858000 0x0 0x1000>;
+ reg-names = "gpio", "padctl";
+ clocks = <&clkgen JH7100_CLK_GPIO_APB>;
+ resets = <&rstgen JH7100_RSTN_GPIO_APB>;
+ interrupts = <32>;
+ gpio-controller;
+ #gpio-cells = <2>;
+ interrupt-controller;
+ #interrupt-cells = <2>;
+ };
+
+ uart2: serial@12430000 {
+ compatible = "starfive,jh7100-uart", "snps,dw-apb-uart";
+ reg = <0x0 0x12430000 0x0 0x10000>;
+ clocks = <&clkgen JH7100_CLK_UART2_CORE>,
+ <&clkgen JH7100_CLK_UART2_APB>;
+ clock-names = "baudclk", "apb_pclk";
+ resets = <&rstgen JH7100_RSTN_UART2_APB>;
+ interrupts = <72>;
+ reg-io-width = <4>;
+ reg-shift = <2>;
+ status = "disabled";
+ };
+
+ uart3: serial@12440000 {
+ compatible = "starfive,jh7100-uart", "snps,dw-apb-uart";
+ reg = <0x0 0x12440000 0x0 0x10000>;
+ clocks = <&clkgen JH7100_CLK_UART3_CORE>,
+ <&clkgen JH7100_CLK_UART3_APB>;
+ clock-names = "baudclk", "apb_pclk";
+ resets = <&rstgen JH7100_RSTN_UART3_APB>;
+ interrupts = <73>;
+ reg-io-width = <4>;
+ reg-shift = <2>;
+ status = "disabled";
+ };
+
+ i2c2: i2c@12450000 {
+ compatible = "snps,designware-i2c";
+ reg = <0x0 0x12450000 0x0 0x10000>;
+ clocks = <&clkgen JH7100_CLK_I2C2_CORE>,
+ <&clkgen JH7100_CLK_I2C2_APB>;
+ clock-names = "ref", "pclk";
+ resets = <&rstgen JH7100_RSTN_I2C2_APB>;
+ interrupts = <74>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+
+ i2c3: i2c@12460000 {
+ compatible = "snps,designware-i2c";
+ reg = <0x0 0x12460000 0x0 0x10000>;
+ clocks = <&clkgen JH7100_CLK_I2C3_CORE>,
+ <&clkgen JH7100_CLK_I2C3_APB>;
+ clock-names = "ref", "pclk";
+ resets = <&rstgen JH7100_RSTN_I2C3_APB>;
+ interrupts = <75>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+ };
+ };
+};
diff --git a/arch/riscv/include/asm/efi.h b/arch/riscv/include/asm/efi.h
index 49b398f..cc4f678 100644
--- a/arch/riscv/include/asm/efi.h
+++ b/arch/riscv/include/asm/efi.h
@@ -13,7 +13,6 @@
#ifdef CONFIG_EFI
extern void efi_init(void);
-extern void efifb_setup_from_dmi(struct screen_info *si, const char *opt);
#else
#define efi_init()
#endif
diff --git a/arch/riscv/kernel/stacktrace.c b/arch/riscv/kernel/stacktrace.c
index 0fcdc02..201ee20 100644
--- a/arch/riscv/kernel/stacktrace.c
+++ b/arch/riscv/kernel/stacktrace.c
@@ -139,12 +139,8 @@ unsigned long __get_wchan(struct task_struct *task)
return pc;
}
-#ifdef CONFIG_STACKTRACE
-
noinline void arch_stack_walk(stack_trace_consume_fn consume_entry, void *cookie,
struct task_struct *task, struct pt_regs *regs)
{
walk_stackframe(task, regs, consume_entry, cookie);
}
-
-#endif /* CONFIG_STACKTRACE */
diff --git a/arch/riscv/net/bpf_jit_comp32.c b/arch/riscv/net/bpf_jit_comp32.c
index e649742..529a83b 100644
--- a/arch/riscv/net/bpf_jit_comp32.c
+++ b/arch/riscv/net/bpf_jit_comp32.c
@@ -799,11 +799,10 @@ static int emit_bpf_tail_call(int insn, struct rv_jit_context *ctx)
emit_bcc(BPF_JGE, lo(idx_reg), RV_REG_T1, off, ctx);
/*
- * temp_tcc = tcc - 1;
- * if (tcc < 0)
+ * if (--tcc < 0)
* goto out;
*/
- emit(rv_addi(RV_REG_T1, RV_REG_TCC, -1), ctx);
+ emit(rv_addi(RV_REG_TCC, RV_REG_TCC, -1), ctx);
off = ninsns_rvoff(tc_ninsn - (ctx->ninsns - start_insn));
emit_bcc(BPF_JSLT, RV_REG_TCC, RV_REG_ZERO, off, ctx);
@@ -829,7 +828,6 @@ static int emit_bpf_tail_call(int insn, struct rv_jit_context *ctx)
if (is_12b_check(off, insn))
return -1;
emit(rv_lw(RV_REG_T0, off, RV_REG_T0), ctx);
- emit(rv_addi(RV_REG_TCC, RV_REG_T1, 0), ctx);
/* Epilogue jumps to *(t0 + 4). */
__build_epilogue(true, ctx);
return 0;
diff --git a/arch/riscv/net/bpf_jit_comp64.c b/arch/riscv/net/bpf_jit_comp64.c
index f2a779c..603630b 100644
--- a/arch/riscv/net/bpf_jit_comp64.c
+++ b/arch/riscv/net/bpf_jit_comp64.c
@@ -327,12 +327,12 @@ static int emit_bpf_tail_call(int insn, struct rv_jit_context *ctx)
off = ninsns_rvoff(tc_ninsn - (ctx->ninsns - start_insn));
emit_branch(BPF_JGE, RV_REG_A2, RV_REG_T1, off, ctx);
- /* if (TCC-- < 0)
+ /* if (--TCC < 0)
* goto out;
*/
- emit_addi(RV_REG_T1, tcc, -1, ctx);
+ emit_addi(RV_REG_TCC, tcc, -1, ctx);
off = ninsns_rvoff(tc_ninsn - (ctx->ninsns - start_insn));
- emit_branch(BPF_JSLT, tcc, RV_REG_ZERO, off, ctx);
+ emit_branch(BPF_JSLT, RV_REG_TCC, RV_REG_ZERO, off, ctx);
/* prog = array->ptrs[index];
* if (!prog)
@@ -352,7 +352,6 @@ static int emit_bpf_tail_call(int insn, struct rv_jit_context *ctx)
if (is_12b_check(off, insn))
return -1;
emit_ld(RV_REG_T3, off, RV_REG_T2, ctx);
- emit_mv(RV_REG_TCC, RV_REG_T1, ctx);
__build_epilogue(true, ctx);
return 0;
}
diff --git a/arch/s390/configs/debug_defconfig b/arch/s390/configs/debug_defconfig
index e45cc27..354e51d 100644
--- a/arch/s390/configs/debug_defconfig
+++ b/arch/s390/configs/debug_defconfig
@@ -770,6 +770,7 @@
CONFIG_CRYPTO_SHA3_512_S390=m
CONFIG_CRYPTO_DES_S390=m
CONFIG_CRYPTO_AES_S390=m
+CONFIG_CRYPTO_CHACHA_S390=m
CONFIG_CRYPTO_GHASH_S390=m
CONFIG_CRYPTO_CRC32_S390=y
CONFIG_CRYPTO_DEV_VIRTIO=m
diff --git a/arch/s390/configs/defconfig b/arch/s390/configs/defconfig
index 1c750bfc..8dee6c3 100644
--- a/arch/s390/configs/defconfig
+++ b/arch/s390/configs/defconfig
@@ -757,6 +757,7 @@
CONFIG_CRYPTO_SHA3_512_S390=m
CONFIG_CRYPTO_DES_S390=m
CONFIG_CRYPTO_AES_S390=m
+CONFIG_CRYPTO_CHACHA_S390=m
CONFIG_CRYPTO_GHASH_S390=m
CONFIG_CRYPTO_CRC32_S390=y
CONFIG_CRYPTO_DEV_VIRTIO=m
diff --git a/arch/s390/crypto/Makefile b/arch/s390/crypto/Makefile
index 12889d4..c63abfe 100644
--- a/arch/s390/crypto/Makefile
+++ b/arch/s390/crypto/Makefile
@@ -11,9 +11,11 @@
obj-$(CONFIG_CRYPTO_DES_S390) += des_s390.o
obj-$(CONFIG_CRYPTO_AES_S390) += aes_s390.o
obj-$(CONFIG_CRYPTO_PAES_S390) += paes_s390.o
+obj-$(CONFIG_CRYPTO_CHACHA_S390) += chacha_s390.o
obj-$(CONFIG_S390_PRNG) += prng.o
obj-$(CONFIG_CRYPTO_GHASH_S390) += ghash_s390.o
obj-$(CONFIG_CRYPTO_CRC32_S390) += crc32-vx_s390.o
obj-$(CONFIG_ARCH_RANDOM) += arch_random.o
crc32-vx_s390-y := crc32-vx.o crc32le-vx.o crc32be-vx.o
+chacha_s390-y := chacha-glue.o chacha-s390.o
diff --git a/arch/s390/crypto/chacha-glue.c b/arch/s390/crypto/chacha-glue.c
new file mode 100644
index 0000000..ccfff73
--- /dev/null
+++ b/arch/s390/crypto/chacha-glue.c
@@ -0,0 +1,100 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * s390 ChaCha stream cipher.
+ *
+ * Copyright IBM Corp. 2021
+ */
+
+#define KMSG_COMPONENT "chacha_s390"
+#define pr_fmt(fmt) KMSG_COMPONENT ": " fmt
+
+#include <crypto/internal/chacha.h>
+#include <crypto/internal/skcipher.h>
+#include <crypto/algapi.h>
+#include <linux/cpufeature.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/sizes.h>
+#include <asm/fpu/api.h>
+#include "chacha-s390.h"
+
+static void chacha20_crypt_s390(u32 *state, u8 *dst, const u8 *src,
+ unsigned int nbytes, const u32 *key,
+ u32 *counter)
+{
+ struct kernel_fpu vxstate;
+
+ kernel_fpu_begin(&vxstate, KERNEL_VXR);
+ chacha20_vx(dst, src, nbytes, key, counter);
+ kernel_fpu_end(&vxstate, KERNEL_VXR);
+
+ *counter += round_up(nbytes, CHACHA_BLOCK_SIZE) / CHACHA_BLOCK_SIZE;
+}
+
+static int chacha20_s390(struct skcipher_request *req)
+{
+ struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(req);
+ struct chacha_ctx *ctx = crypto_skcipher_ctx(tfm);
+ u32 state[CHACHA_STATE_WORDS] __aligned(16);
+ struct skcipher_walk walk;
+ unsigned int nbytes;
+ int rc;
+
+ rc = skcipher_walk_virt(&walk, req, false);
+ chacha_init_generic(state, ctx->key, req->iv);
+
+ while (walk.nbytes > 0) {
+ nbytes = walk.nbytes;
+ if (nbytes < walk.total)
+ nbytes = round_down(nbytes, walk.stride);
+
+ if (nbytes <= CHACHA_BLOCK_SIZE) {
+ chacha_crypt_generic(state, walk.dst.virt.addr,
+ walk.src.virt.addr, nbytes,
+ ctx->nrounds);
+ } else {
+ chacha20_crypt_s390(state, walk.dst.virt.addr,
+ walk.src.virt.addr, nbytes,
+ &state[4], &state[12]);
+ }
+ rc = skcipher_walk_done(&walk, walk.nbytes - nbytes);
+ }
+ return rc;
+}
+
+static struct skcipher_alg chacha_algs[] = {
+ {
+ .base.cra_name = "chacha20",
+ .base.cra_driver_name = "chacha20-s390",
+ .base.cra_priority = 900,
+ .base.cra_blocksize = 1,
+ .base.cra_ctxsize = sizeof(struct chacha_ctx),
+ .base.cra_module = THIS_MODULE,
+
+ .min_keysize = CHACHA_KEY_SIZE,
+ .max_keysize = CHACHA_KEY_SIZE,
+ .ivsize = CHACHA_IV_SIZE,
+ .chunksize = CHACHA_BLOCK_SIZE,
+ .setkey = chacha20_setkey,
+ .encrypt = chacha20_s390,
+ .decrypt = chacha20_s390,
+ }
+};
+
+static int __init chacha_mod_init(void)
+{
+ return crypto_register_skciphers(chacha_algs, ARRAY_SIZE(chacha_algs));
+}
+
+static void __exit chacha_mod_fini(void)
+{
+ crypto_unregister_skciphers(chacha_algs, ARRAY_SIZE(chacha_algs));
+}
+
+module_cpu_feature_match(VXRS, chacha_mod_init);
+module_exit(chacha_mod_fini);
+
+MODULE_DESCRIPTION("ChaCha20 stream cipher");
+MODULE_LICENSE("GPL v2");
+
+MODULE_ALIAS_CRYPTO("chacha20");
diff --git a/arch/s390/crypto/chacha-s390.S b/arch/s390/crypto/chacha-s390.S
new file mode 100644
index 0000000..badf5c4
--- /dev/null
+++ b/arch/s390/crypto/chacha-s390.S
@@ -0,0 +1,907 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Original implementation written by Andy Polyakov, @dot-asm.
+ * This is an adaptation of the original code for kernel use.
+ *
+ * Copyright (C) 2006-2019 CRYPTOGAMS by <appro@openssl.org>. All Rights Reserved.
+ */
+
+#include <linux/linkage.h>
+#include <asm/nospec-insn.h>
+#include <asm/vx-insn.h>
+
+#define SP %r15
+#define FRAME (16 * 8 + 4 * 8)
+
+.data
+.align 32
+
+.Lsigma:
+.long 0x61707865,0x3320646e,0x79622d32,0x6b206574 # endian-neutral
+.long 1,0,0,0
+.long 2,0,0,0
+.long 3,0,0,0
+.long 0x03020100,0x07060504,0x0b0a0908,0x0f0e0d0c # byte swap
+
+.long 0,1,2,3
+.long 0x61707865,0x61707865,0x61707865,0x61707865 # smashed sigma
+.long 0x3320646e,0x3320646e,0x3320646e,0x3320646e
+.long 0x79622d32,0x79622d32,0x79622d32,0x79622d32
+.long 0x6b206574,0x6b206574,0x6b206574,0x6b206574
+
+.previous
+
+ GEN_BR_THUNK %r14
+
+.text
+
+#############################################################################
+# void chacha20_vx_4x(u8 *out, counst u8 *inp, size_t len,
+# counst u32 *key, const u32 *counter)
+
+#define OUT %r2
+#define INP %r3
+#define LEN %r4
+#define KEY %r5
+#define COUNTER %r6
+
+#define BEPERM %v31
+#define CTR %v26
+
+#define K0 %v16
+#define K1 %v17
+#define K2 %v18
+#define K3 %v19
+
+#define XA0 %v0
+#define XA1 %v1
+#define XA2 %v2
+#define XA3 %v3
+
+#define XB0 %v4
+#define XB1 %v5
+#define XB2 %v6
+#define XB3 %v7
+
+#define XC0 %v8
+#define XC1 %v9
+#define XC2 %v10
+#define XC3 %v11
+
+#define XD0 %v12
+#define XD1 %v13
+#define XD2 %v14
+#define XD3 %v15
+
+#define XT0 %v27
+#define XT1 %v28
+#define XT2 %v29
+#define XT3 %v30
+
+ENTRY(chacha20_vx_4x)
+ stmg %r6,%r7,6*8(SP)
+
+ larl %r7,.Lsigma
+ lhi %r0,10
+ lhi %r1,0
+
+ VL K0,0,,%r7 # load sigma
+ VL K1,0,,KEY # load key
+ VL K2,16,,KEY
+ VL K3,0,,COUNTER # load counter
+
+ VL BEPERM,0x40,,%r7
+ VL CTR,0x50,,%r7
+
+ VLM XA0,XA3,0x60,%r7,4 # load [smashed] sigma
+
+ VREPF XB0,K1,0 # smash the key
+ VREPF XB1,K1,1
+ VREPF XB2,K1,2
+ VREPF XB3,K1,3
+
+ VREPF XD0,K3,0
+ VREPF XD1,K3,1
+ VREPF XD2,K3,2
+ VREPF XD3,K3,3
+ VAF XD0,XD0,CTR
+
+ VREPF XC0,K2,0
+ VREPF XC1,K2,1
+ VREPF XC2,K2,2
+ VREPF XC3,K2,3
+
+.Loop_4x:
+ VAF XA0,XA0,XB0
+ VX XD0,XD0,XA0
+ VERLLF XD0,XD0,16
+
+ VAF XA1,XA1,XB1
+ VX XD1,XD1,XA1
+ VERLLF XD1,XD1,16
+
+ VAF XA2,XA2,XB2
+ VX XD2,XD2,XA2
+ VERLLF XD2,XD2,16
+
+ VAF XA3,XA3,XB3
+ VX XD3,XD3,XA3
+ VERLLF XD3,XD3,16
+
+ VAF XC0,XC0,XD0
+ VX XB0,XB0,XC0
+ VERLLF XB0,XB0,12
+
+ VAF XC1,XC1,XD1
+ VX XB1,XB1,XC1
+ VERLLF XB1,XB1,12
+
+ VAF XC2,XC2,XD2
+ VX XB2,XB2,XC2
+ VERLLF XB2,XB2,12
+
+ VAF XC3,XC3,XD3
+ VX XB3,XB3,XC3
+ VERLLF XB3,XB3,12
+
+ VAF XA0,XA0,XB0
+ VX XD0,XD0,XA0
+ VERLLF XD0,XD0,8
+
+ VAF XA1,XA1,XB1
+ VX XD1,XD1,XA1
+ VERLLF XD1,XD1,8
+
+ VAF XA2,XA2,XB2
+ VX XD2,XD2,XA2
+ VERLLF XD2,XD2,8
+
+ VAF XA3,XA3,XB3
+ VX XD3,XD3,XA3
+ VERLLF XD3,XD3,8
+
+ VAF XC0,XC0,XD0
+ VX XB0,XB0,XC0
+ VERLLF XB0,XB0,7
+
+ VAF XC1,XC1,XD1
+ VX XB1,XB1,XC1
+ VERLLF XB1,XB1,7
+
+ VAF XC2,XC2,XD2
+ VX XB2,XB2,XC2
+ VERLLF XB2,XB2,7
+
+ VAF XC3,XC3,XD3
+ VX XB3,XB3,XC3
+ VERLLF XB3,XB3,7
+
+ VAF XA0,XA0,XB1
+ VX XD3,XD3,XA0
+ VERLLF XD3,XD3,16
+
+ VAF XA1,XA1,XB2
+ VX XD0,XD0,XA1
+ VERLLF XD0,XD0,16
+
+ VAF XA2,XA2,XB3
+ VX XD1,XD1,XA2
+ VERLLF XD1,XD1,16
+
+ VAF XA3,XA3,XB0
+ VX XD2,XD2,XA3
+ VERLLF XD2,XD2,16
+
+ VAF XC2,XC2,XD3
+ VX XB1,XB1,XC2
+ VERLLF XB1,XB1,12
+
+ VAF XC3,XC3,XD0
+ VX XB2,XB2,XC3
+ VERLLF XB2,XB2,12
+
+ VAF XC0,XC0,XD1
+ VX XB3,XB3,XC0
+ VERLLF XB3,XB3,12
+
+ VAF XC1,XC1,XD2
+ VX XB0,XB0,XC1
+ VERLLF XB0,XB0,12
+
+ VAF XA0,XA0,XB1
+ VX XD3,XD3,XA0
+ VERLLF XD3,XD3,8
+
+ VAF XA1,XA1,XB2
+ VX XD0,XD0,XA1
+ VERLLF XD0,XD0,8
+
+ VAF XA2,XA2,XB3
+ VX XD1,XD1,XA2
+ VERLLF XD1,XD1,8
+
+ VAF XA3,XA3,XB0
+ VX XD2,XD2,XA3
+ VERLLF XD2,XD2,8
+
+ VAF XC2,XC2,XD3
+ VX XB1,XB1,XC2
+ VERLLF XB1,XB1,7
+
+ VAF XC3,XC3,XD0
+ VX XB2,XB2,XC3
+ VERLLF XB2,XB2,7
+
+ VAF XC0,XC0,XD1
+ VX XB3,XB3,XC0
+ VERLLF XB3,XB3,7
+
+ VAF XC1,XC1,XD2
+ VX XB0,XB0,XC1
+ VERLLF XB0,XB0,7
+ brct %r0,.Loop_4x
+
+ VAF XD0,XD0,CTR
+
+ VMRHF XT0,XA0,XA1 # transpose data
+ VMRHF XT1,XA2,XA3
+ VMRLF XT2,XA0,XA1
+ VMRLF XT3,XA2,XA3
+ VPDI XA0,XT0,XT1,0b0000
+ VPDI XA1,XT0,XT1,0b0101
+ VPDI XA2,XT2,XT3,0b0000
+ VPDI XA3,XT2,XT3,0b0101
+
+ VMRHF XT0,XB0,XB1
+ VMRHF XT1,XB2,XB3
+ VMRLF XT2,XB0,XB1
+ VMRLF XT3,XB2,XB3
+ VPDI XB0,XT0,XT1,0b0000
+ VPDI XB1,XT0,XT1,0b0101
+ VPDI XB2,XT2,XT3,0b0000
+ VPDI XB3,XT2,XT3,0b0101
+
+ VMRHF XT0,XC0,XC1
+ VMRHF XT1,XC2,XC3
+ VMRLF XT2,XC0,XC1
+ VMRLF XT3,XC2,XC3
+ VPDI XC0,XT0,XT1,0b0000
+ VPDI XC1,XT0,XT1,0b0101
+ VPDI XC2,XT2,XT3,0b0000
+ VPDI XC3,XT2,XT3,0b0101
+
+ VMRHF XT0,XD0,XD1
+ VMRHF XT1,XD2,XD3
+ VMRLF XT2,XD0,XD1
+ VMRLF XT3,XD2,XD3
+ VPDI XD0,XT0,XT1,0b0000
+ VPDI XD1,XT0,XT1,0b0101
+ VPDI XD2,XT2,XT3,0b0000
+ VPDI XD3,XT2,XT3,0b0101
+
+ VAF XA0,XA0,K0
+ VAF XB0,XB0,K1
+ VAF XC0,XC0,K2
+ VAF XD0,XD0,K3
+
+ VPERM XA0,XA0,XA0,BEPERM
+ VPERM XB0,XB0,XB0,BEPERM
+ VPERM XC0,XC0,XC0,BEPERM
+ VPERM XD0,XD0,XD0,BEPERM
+
+ VLM XT0,XT3,0,INP,0
+
+ VX XT0,XT0,XA0
+ VX XT1,XT1,XB0
+ VX XT2,XT2,XC0
+ VX XT3,XT3,XD0
+
+ VSTM XT0,XT3,0,OUT,0
+
+ la INP,0x40(INP)
+ la OUT,0x40(OUT)
+ aghi LEN,-0x40
+
+ VAF XA0,XA1,K0
+ VAF XB0,XB1,K1
+ VAF XC0,XC1,K2
+ VAF XD0,XD1,K3
+
+ VPERM XA0,XA0,XA0,BEPERM
+ VPERM XB0,XB0,XB0,BEPERM
+ VPERM XC0,XC0,XC0,BEPERM
+ VPERM XD0,XD0,XD0,BEPERM
+
+ .insn rilu,0xc20e00000000,LEN,0x40 # clgfi LEN,0x40
+ jl .Ltail_4x
+
+ VLM XT0,XT3,0,INP,0
+
+ VX XT0,XT0,XA0
+ VX XT1,XT1,XB0
+ VX XT2,XT2,XC0
+ VX XT3,XT3,XD0
+
+ VSTM XT0,XT3,0,OUT,0
+
+ la INP,0x40(INP)
+ la OUT,0x40(OUT)
+ aghi LEN,-0x40
+ je .Ldone_4x
+
+ VAF XA0,XA2,K0
+ VAF XB0,XB2,K1
+ VAF XC0,XC2,K2
+ VAF XD0,XD2,K3
+
+ VPERM XA0,XA0,XA0,BEPERM
+ VPERM XB0,XB0,XB0,BEPERM
+ VPERM XC0,XC0,XC0,BEPERM
+ VPERM XD0,XD0,XD0,BEPERM
+
+ .insn rilu,0xc20e00000000,LEN,0x40 # clgfi LEN,0x40
+ jl .Ltail_4x
+
+ VLM XT0,XT3,0,INP,0
+
+ VX XT0,XT0,XA0
+ VX XT1,XT1,XB0
+ VX XT2,XT2,XC0
+ VX XT3,XT3,XD0
+
+ VSTM XT0,XT3,0,OUT,0
+
+ la INP,0x40(INP)
+ la OUT,0x40(OUT)
+ aghi LEN,-0x40
+ je .Ldone_4x
+
+ VAF XA0,XA3,K0
+ VAF XB0,XB3,K1
+ VAF XC0,XC3,K2
+ VAF XD0,XD3,K3
+
+ VPERM XA0,XA0,XA0,BEPERM
+ VPERM XB0,XB0,XB0,BEPERM
+ VPERM XC0,XC0,XC0,BEPERM
+ VPERM XD0,XD0,XD0,BEPERM
+
+ .insn rilu,0xc20e00000000,LEN,0x40 # clgfi LEN,0x40
+ jl .Ltail_4x
+
+ VLM XT0,XT3,0,INP,0
+
+ VX XT0,XT0,XA0
+ VX XT1,XT1,XB0
+ VX XT2,XT2,XC0
+ VX XT3,XT3,XD0
+
+ VSTM XT0,XT3,0,OUT,0
+
+.Ldone_4x:
+ lmg %r6,%r7,6*8(SP)
+ BR_EX %r14
+
+.Ltail_4x:
+ VLR XT0,XC0
+ VLR XT1,XD0
+
+ VST XA0,8*8+0x00,,SP
+ VST XB0,8*8+0x10,,SP
+ VST XT0,8*8+0x20,,SP
+ VST XT1,8*8+0x30,,SP
+
+ lghi %r1,0
+
+.Loop_tail_4x:
+ llgc %r5,0(%r1,INP)
+ llgc %r6,8*8(%r1,SP)
+ xr %r6,%r5
+ stc %r6,0(%r1,OUT)
+ la %r1,1(%r1)
+ brct LEN,.Loop_tail_4x
+
+ lmg %r6,%r7,6*8(SP)
+ BR_EX %r14
+ENDPROC(chacha20_vx_4x)
+
+#undef OUT
+#undef INP
+#undef LEN
+#undef KEY
+#undef COUNTER
+
+#undef BEPERM
+
+#undef K0
+#undef K1
+#undef K2
+#undef K3
+
+
+#############################################################################
+# void chacha20_vx(u8 *out, counst u8 *inp, size_t len,
+# counst u32 *key, const u32 *counter)
+
+#define OUT %r2
+#define INP %r3
+#define LEN %r4
+#define KEY %r5
+#define COUNTER %r6
+
+#define BEPERM %v31
+
+#define K0 %v27
+#define K1 %v24
+#define K2 %v25
+#define K3 %v26
+
+#define A0 %v0
+#define B0 %v1
+#define C0 %v2
+#define D0 %v3
+
+#define A1 %v4
+#define B1 %v5
+#define C1 %v6
+#define D1 %v7
+
+#define A2 %v8
+#define B2 %v9
+#define C2 %v10
+#define D2 %v11
+
+#define A3 %v12
+#define B3 %v13
+#define C3 %v14
+#define D3 %v15
+
+#define A4 %v16
+#define B4 %v17
+#define C4 %v18
+#define D4 %v19
+
+#define A5 %v20
+#define B5 %v21
+#define C5 %v22
+#define D5 %v23
+
+#define T0 %v27
+#define T1 %v28
+#define T2 %v29
+#define T3 %v30
+
+ENTRY(chacha20_vx)
+ .insn rilu,0xc20e00000000,LEN,256 # clgfi LEN,256
+ jle chacha20_vx_4x
+ stmg %r6,%r7,6*8(SP)
+
+ lghi %r1,-FRAME
+ lgr %r0,SP
+ la SP,0(%r1,SP)
+ stg %r0,0(SP) # back-chain
+
+ larl %r7,.Lsigma
+ lhi %r0,10
+
+ VLM K1,K2,0,KEY,0 # load key
+ VL K3,0,,COUNTER # load counter
+
+ VLM K0,BEPERM,0,%r7,4 # load sigma, increments, ...
+
+.Loop_outer_vx:
+ VLR A0,K0
+ VLR B0,K1
+ VLR A1,K0
+ VLR B1,K1
+ VLR A2,K0
+ VLR B2,K1
+ VLR A3,K0
+ VLR B3,K1
+ VLR A4,K0
+ VLR B4,K1
+ VLR A5,K0
+ VLR B5,K1
+
+ VLR D0,K3
+ VAF D1,K3,T1 # K[3]+1
+ VAF D2,K3,T2 # K[3]+2
+ VAF D3,K3,T3 # K[3]+3
+ VAF D4,D2,T2 # K[3]+4
+ VAF D5,D2,T3 # K[3]+5
+
+ VLR C0,K2
+ VLR C1,K2
+ VLR C2,K2
+ VLR C3,K2
+ VLR C4,K2
+ VLR C5,K2
+
+ VLR T1,D1
+ VLR T2,D2
+ VLR T3,D3
+
+.Loop_vx:
+ VAF A0,A0,B0
+ VAF A1,A1,B1
+ VAF A2,A2,B2
+ VAF A3,A3,B3
+ VAF A4,A4,B4
+ VAF A5,A5,B5
+ VX D0,D0,A0
+ VX D1,D1,A1
+ VX D2,D2,A2
+ VX D3,D3,A3
+ VX D4,D4,A4
+ VX D5,D5,A5
+ VERLLF D0,D0,16
+ VERLLF D1,D1,16
+ VERLLF D2,D2,16
+ VERLLF D3,D3,16
+ VERLLF D4,D4,16
+ VERLLF D5,D5,16
+
+ VAF C0,C0,D0
+ VAF C1,C1,D1
+ VAF C2,C2,D2
+ VAF C3,C3,D3
+ VAF C4,C4,D4
+ VAF C5,C5,D5
+ VX B0,B0,C0
+ VX B1,B1,C1
+ VX B2,B2,C2
+ VX B3,B3,C3
+ VX B4,B4,C4
+ VX B5,B5,C5
+ VERLLF B0,B0,12
+ VERLLF B1,B1,12
+ VERLLF B2,B2,12
+ VERLLF B3,B3,12
+ VERLLF B4,B4,12
+ VERLLF B5,B5,12
+
+ VAF A0,A0,B0
+ VAF A1,A1,B1
+ VAF A2,A2,B2
+ VAF A3,A3,B3
+ VAF A4,A4,B4
+ VAF A5,A5,B5
+ VX D0,D0,A0
+ VX D1,D1,A1
+ VX D2,D2,A2
+ VX D3,D3,A3
+ VX D4,D4,A4
+ VX D5,D5,A5
+ VERLLF D0,D0,8
+ VERLLF D1,D1,8
+ VERLLF D2,D2,8
+ VERLLF D3,D3,8
+ VERLLF D4,D4,8
+ VERLLF D5,D5,8
+
+ VAF C0,C0,D0
+ VAF C1,C1,D1
+ VAF C2,C2,D2
+ VAF C3,C3,D3
+ VAF C4,C4,D4
+ VAF C5,C5,D5
+ VX B0,B0,C0
+ VX B1,B1,C1
+ VX B2,B2,C2
+ VX B3,B3,C3
+ VX B4,B4,C4
+ VX B5,B5,C5
+ VERLLF B0,B0,7
+ VERLLF B1,B1,7
+ VERLLF B2,B2,7
+ VERLLF B3,B3,7
+ VERLLF B4,B4,7
+ VERLLF B5,B5,7
+
+ VSLDB C0,C0,C0,8
+ VSLDB C1,C1,C1,8
+ VSLDB C2,C2,C2,8
+ VSLDB C3,C3,C3,8
+ VSLDB C4,C4,C4,8
+ VSLDB C5,C5,C5,8
+ VSLDB B0,B0,B0,4
+ VSLDB B1,B1,B1,4
+ VSLDB B2,B2,B2,4
+ VSLDB B3,B3,B3,4
+ VSLDB B4,B4,B4,4
+ VSLDB B5,B5,B5,4
+ VSLDB D0,D0,D0,12
+ VSLDB D1,D1,D1,12
+ VSLDB D2,D2,D2,12
+ VSLDB D3,D3,D3,12
+ VSLDB D4,D4,D4,12
+ VSLDB D5,D5,D5,12
+
+ VAF A0,A0,B0
+ VAF A1,A1,B1
+ VAF A2,A2,B2
+ VAF A3,A3,B3
+ VAF A4,A4,B4
+ VAF A5,A5,B5
+ VX D0,D0,A0
+ VX D1,D1,A1
+ VX D2,D2,A2
+ VX D3,D3,A3
+ VX D4,D4,A4
+ VX D5,D5,A5
+ VERLLF D0,D0,16
+ VERLLF D1,D1,16
+ VERLLF D2,D2,16
+ VERLLF D3,D3,16
+ VERLLF D4,D4,16
+ VERLLF D5,D5,16
+
+ VAF C0,C0,D0
+ VAF C1,C1,D1
+ VAF C2,C2,D2
+ VAF C3,C3,D3
+ VAF C4,C4,D4
+ VAF C5,C5,D5
+ VX B0,B0,C0
+ VX B1,B1,C1
+ VX B2,B2,C2
+ VX B3,B3,C3
+ VX B4,B4,C4
+ VX B5,B5,C5
+ VERLLF B0,B0,12
+ VERLLF B1,B1,12
+ VERLLF B2,B2,12
+ VERLLF B3,B3,12
+ VERLLF B4,B4,12
+ VERLLF B5,B5,12
+
+ VAF A0,A0,B0
+ VAF A1,A1,B1
+ VAF A2,A2,B2
+ VAF A3,A3,B3
+ VAF A4,A4,B4
+ VAF A5,A5,B5
+ VX D0,D0,A0
+ VX D1,D1,A1
+ VX D2,D2,A2
+ VX D3,D3,A3
+ VX D4,D4,A4
+ VX D5,D5,A5
+ VERLLF D0,D0,8
+ VERLLF D1,D1,8
+ VERLLF D2,D2,8
+ VERLLF D3,D3,8
+ VERLLF D4,D4,8
+ VERLLF D5,D5,8
+
+ VAF C0,C0,D0
+ VAF C1,C1,D1
+ VAF C2,C2,D2
+ VAF C3,C3,D3
+ VAF C4,C4,D4
+ VAF C5,C5,D5
+ VX B0,B0,C0
+ VX B1,B1,C1
+ VX B2,B2,C2
+ VX B3,B3,C3
+ VX B4,B4,C4
+ VX B5,B5,C5
+ VERLLF B0,B0,7
+ VERLLF B1,B1,7
+ VERLLF B2,B2,7
+ VERLLF B3,B3,7
+ VERLLF B4,B4,7
+ VERLLF B5,B5,7
+
+ VSLDB C0,C0,C0,8
+ VSLDB C1,C1,C1,8
+ VSLDB C2,C2,C2,8
+ VSLDB C3,C3,C3,8
+ VSLDB C4,C4,C4,8
+ VSLDB C5,C5,C5,8
+ VSLDB B0,B0,B0,12
+ VSLDB B1,B1,B1,12
+ VSLDB B2,B2,B2,12
+ VSLDB B3,B3,B3,12
+ VSLDB B4,B4,B4,12
+ VSLDB B5,B5,B5,12
+ VSLDB D0,D0,D0,4
+ VSLDB D1,D1,D1,4
+ VSLDB D2,D2,D2,4
+ VSLDB D3,D3,D3,4
+ VSLDB D4,D4,D4,4
+ VSLDB D5,D5,D5,4
+ brct %r0,.Loop_vx
+
+ VAF A0,A0,K0
+ VAF B0,B0,K1
+ VAF C0,C0,K2
+ VAF D0,D0,K3
+ VAF A1,A1,K0
+ VAF D1,D1,T1 # +K[3]+1
+
+ VPERM A0,A0,A0,BEPERM
+ VPERM B0,B0,B0,BEPERM
+ VPERM C0,C0,C0,BEPERM
+ VPERM D0,D0,D0,BEPERM
+
+ .insn rilu,0xc20e00000000,LEN,0x40 # clgfi LEN,0x40
+ jl .Ltail_vx
+
+ VAF D2,D2,T2 # +K[3]+2
+ VAF D3,D3,T3 # +K[3]+3
+ VLM T0,T3,0,INP,0
+
+ VX A0,A0,T0
+ VX B0,B0,T1
+ VX C0,C0,T2
+ VX D0,D0,T3
+
+ VLM K0,T3,0,%r7,4 # re-load sigma and increments
+
+ VSTM A0,D0,0,OUT,0
+
+ la INP,0x40(INP)
+ la OUT,0x40(OUT)
+ aghi LEN,-0x40
+ je .Ldone_vx
+
+ VAF B1,B1,K1
+ VAF C1,C1,K2
+
+ VPERM A0,A1,A1,BEPERM
+ VPERM B0,B1,B1,BEPERM
+ VPERM C0,C1,C1,BEPERM
+ VPERM D0,D1,D1,BEPERM
+
+ .insn rilu,0xc20e00000000,LEN,0x40 # clgfi LEN,0x40
+ jl .Ltail_vx
+
+ VLM A1,D1,0,INP,0
+
+ VX A0,A0,A1
+ VX B0,B0,B1
+ VX C0,C0,C1
+ VX D0,D0,D1
+
+ VSTM A0,D0,0,OUT,0
+
+ la INP,0x40(INP)
+ la OUT,0x40(OUT)
+ aghi LEN,-0x40
+ je .Ldone_vx
+
+ VAF A2,A2,K0
+ VAF B2,B2,K1
+ VAF C2,C2,K2
+
+ VPERM A0,A2,A2,BEPERM
+ VPERM B0,B2,B2,BEPERM
+ VPERM C0,C2,C2,BEPERM
+ VPERM D0,D2,D2,BEPERM
+
+ .insn rilu,0xc20e00000000,LEN,0x40 # clgfi LEN,0x40
+ jl .Ltail_vx
+
+ VLM A1,D1,0,INP,0
+
+ VX A0,A0,A1
+ VX B0,B0,B1
+ VX C0,C0,C1
+ VX D0,D0,D1
+
+ VSTM A0,D0,0,OUT,0
+
+ la INP,0x40(INP)
+ la OUT,0x40(OUT)
+ aghi LEN,-0x40
+ je .Ldone_vx
+
+ VAF A3,A3,K0
+ VAF B3,B3,K1
+ VAF C3,C3,K2
+ VAF D2,K3,T3 # K[3]+3
+
+ VPERM A0,A3,A3,BEPERM
+ VPERM B0,B3,B3,BEPERM
+ VPERM C0,C3,C3,BEPERM
+ VPERM D0,D3,D3,BEPERM
+
+ .insn rilu,0xc20e00000000,LEN,0x40 # clgfi LEN,0x40
+ jl .Ltail_vx
+
+ VAF D3,D2,T1 # K[3]+4
+ VLM A1,D1,0,INP,0
+
+ VX A0,A0,A1
+ VX B0,B0,B1
+ VX C0,C0,C1
+ VX D0,D0,D1
+
+ VSTM A0,D0,0,OUT,0
+
+ la INP,0x40(INP)
+ la OUT,0x40(OUT)
+ aghi LEN,-0x40
+ je .Ldone_vx
+
+ VAF A4,A4,K0
+ VAF B4,B4,K1
+ VAF C4,C4,K2
+ VAF D4,D4,D3 # +K[3]+4
+ VAF D3,D3,T1 # K[3]+5
+ VAF K3,D2,T3 # K[3]+=6
+
+ VPERM A0,A4,A4,BEPERM
+ VPERM B0,B4,B4,BEPERM
+ VPERM C0,C4,C4,BEPERM
+ VPERM D0,D4,D4,BEPERM
+
+ .insn rilu,0xc20e00000000,LEN,0x40 # clgfi LEN,0x40
+ jl .Ltail_vx
+
+ VLM A1,D1,0,INP,0
+
+ VX A0,A0,A1
+ VX B0,B0,B1
+ VX C0,C0,C1
+ VX D0,D0,D1
+
+ VSTM A0,D0,0,OUT,0
+
+ la INP,0x40(INP)
+ la OUT,0x40(OUT)
+ aghi LEN,-0x40
+ je .Ldone_vx
+
+ VAF A5,A5,K0
+ VAF B5,B5,K1
+ VAF C5,C5,K2
+ VAF D5,D5,D3 # +K[3]+5
+
+ VPERM A0,A5,A5,BEPERM
+ VPERM B0,B5,B5,BEPERM
+ VPERM C0,C5,C5,BEPERM
+ VPERM D0,D5,D5,BEPERM
+
+ .insn rilu,0xc20e00000000,LEN,0x40 # clgfi LEN,0x40
+ jl .Ltail_vx
+
+ VLM A1,D1,0,INP,0
+
+ VX A0,A0,A1
+ VX B0,B0,B1
+ VX C0,C0,C1
+ VX D0,D0,D1
+
+ VSTM A0,D0,0,OUT,0
+
+ la INP,0x40(INP)
+ la OUT,0x40(OUT)
+ lhi %r0,10
+ aghi LEN,-0x40
+ jne .Loop_outer_vx
+
+.Ldone_vx:
+ lmg %r6,%r7,FRAME+6*8(SP)
+ la SP,FRAME(SP)
+ BR_EX %r14
+
+.Ltail_vx:
+ VSTM A0,D0,8*8,SP,3
+ lghi %r1,0
+
+.Loop_tail_vx:
+ llgc %r5,0(%r1,INP)
+ llgc %r6,8*8(%r1,SP)
+ xr %r6,%r5
+ stc %r6,0(%r1,OUT)
+ la %r1,1(%r1)
+ brct LEN,.Loop_tail_vx
+
+ lmg %r6,%r7,FRAME+6*8(SP)
+ la SP,FRAME(SP)
+ BR_EX %r14
+ENDPROC(chacha20_vx)
+
+.previous
diff --git a/arch/s390/crypto/chacha-s390.h b/arch/s390/crypto/chacha-s390.h
new file mode 100644
index 0000000..733744c
--- /dev/null
+++ b/arch/s390/crypto/chacha-s390.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * s390 ChaCha stream cipher.
+ *
+ * Copyright IBM Corp. 2021
+ */
+
+#ifndef _CHACHA_S390_H
+#define _CHACHA_S390_H
+
+void chacha20_vx(u8 *out, const u8 *inp, size_t len, const u32 *key,
+ const u32 *counter);
+
+#endif /* _CHACHA_S390_H */
diff --git a/arch/s390/include/asm/ap.h b/arch/s390/include/asm/ap.h
index 3afbee2..c0c8a1f 100644
--- a/arch/s390/include/asm/ap.h
+++ b/arch/s390/include/asm/ap.h
@@ -12,6 +12,8 @@
#ifndef _ASM_S390_AP_H_
#define _ASM_S390_AP_H_
+#include <linux/io.h>
+
/**
* The ap_qid_t identifier of an ap queue.
* If the AP facilities test (APFT) facility is available,
@@ -238,7 +240,7 @@ static inline struct ap_queue_status ap_aqic(ap_qid_t qid,
struct ap_qirq_ctrl qirqctrl;
struct ap_queue_status status;
} reg1;
- void *reg2 = ind;
+ unsigned long reg2 = virt_to_phys(ind);
reg1.qirqctrl = qirqctrl;
diff --git a/arch/s390/include/asm/diag.h b/arch/s390/include/asm/diag.h
index b3a8cb4d..bdcd64f 100644
--- a/arch/s390/include/asm/diag.h
+++ b/arch/s390/include/asm/diag.h
@@ -47,8 +47,8 @@ static inline void diag10_range(unsigned long start_pfn, unsigned long num_pfn)
{
unsigned long start_addr, end_addr;
- start_addr = start_pfn << PAGE_SHIFT;
- end_addr = (start_pfn + num_pfn - 1) << PAGE_SHIFT;
+ start_addr = pfn_to_phys(start_pfn);
+ end_addr = pfn_to_phys(start_pfn + num_pfn - 1);
diag_stat_inc(DIAG_STAT_X010);
asm volatile(
diff --git a/arch/s390/include/asm/nmi.h b/arch/s390/include/asm/nmi.h
index 2db45d7..55c9051 100644
--- a/arch/s390/include/asm/nmi.h
+++ b/arch/s390/include/asm/nmi.h
@@ -98,9 +98,9 @@ struct mcesa {
struct pt_regs;
-void nmi_alloc_boot_cpu(struct lowcore *lc);
-int nmi_alloc_per_cpu(struct lowcore *lc);
-void nmi_free_per_cpu(struct lowcore *lc);
+void nmi_alloc_mcesa_early(u64 *mcesad);
+int nmi_alloc_mcesa(u64 *mcesad);
+void nmi_free_mcesa(u64 *mcesad);
void s390_handle_mcck(void);
void __s390_handle_mcck(void);
diff --git a/arch/s390/include/asm/pci_dma.h b/arch/s390/include/asm/pci_dma.h
index 3b8e89d..91e6342 100644
--- a/arch/s390/include/asm/pci_dma.h
+++ b/arch/s390/include/asm/pci_dma.h
@@ -97,23 +97,23 @@ static inline unsigned int calc_px(dma_addr_t ptr)
return ((unsigned long) ptr >> PAGE_SHIFT) & ZPCI_PT_MASK;
}
-static inline void set_pt_pfaa(unsigned long *entry, void *pfaa)
+static inline void set_pt_pfaa(unsigned long *entry, phys_addr_t pfaa)
{
*entry &= ZPCI_PTE_FLAG_MASK;
- *entry |= ((unsigned long) pfaa & ZPCI_PTE_ADDR_MASK);
+ *entry |= (pfaa & ZPCI_PTE_ADDR_MASK);
}
-static inline void set_rt_sto(unsigned long *entry, void *sto)
+static inline void set_rt_sto(unsigned long *entry, phys_addr_t sto)
{
*entry &= ZPCI_RTE_FLAG_MASK;
- *entry |= ((unsigned long) sto & ZPCI_RTE_ADDR_MASK);
+ *entry |= (sto & ZPCI_RTE_ADDR_MASK);
*entry |= ZPCI_TABLE_TYPE_RTX;
}
-static inline void set_st_pto(unsigned long *entry, void *pto)
+static inline void set_st_pto(unsigned long *entry, phys_addr_t pto)
{
*entry &= ZPCI_STE_FLAG_MASK;
- *entry |= ((unsigned long) pto & ZPCI_STE_ADDR_MASK);
+ *entry |= (pto & ZPCI_STE_ADDR_MASK);
*entry |= ZPCI_TABLE_TYPE_SX;
}
@@ -169,16 +169,19 @@ static inline int pt_entry_isvalid(unsigned long entry)
static inline unsigned long *get_rt_sto(unsigned long entry)
{
- return ((entry & ZPCI_TABLE_TYPE_MASK) == ZPCI_TABLE_TYPE_RTX)
- ? (unsigned long *) (entry & ZPCI_RTE_ADDR_MASK)
- : NULL;
+ if ((entry & ZPCI_TABLE_TYPE_MASK) == ZPCI_TABLE_TYPE_RTX)
+ return phys_to_virt(entry & ZPCI_RTE_ADDR_MASK);
+ else
+ return NULL;
+
}
static inline unsigned long *get_st_pto(unsigned long entry)
{
- return ((entry & ZPCI_TABLE_TYPE_MASK) == ZPCI_TABLE_TYPE_SX)
- ? (unsigned long *) (entry & ZPCI_STE_ADDR_MASK)
- : NULL;
+ if ((entry & ZPCI_TABLE_TYPE_MASK) == ZPCI_TABLE_TYPE_SX)
+ return phys_to_virt(entry & ZPCI_STE_ADDR_MASK);
+ else
+ return NULL;
}
/* Prototypes */
@@ -186,7 +189,7 @@ void dma_free_seg_table(unsigned long);
unsigned long *dma_alloc_cpu_table(void);
void dma_cleanup_tables(unsigned long *);
unsigned long *dma_walk_cpu_trans(unsigned long *rto, dma_addr_t dma_addr);
-void dma_update_cpu_trans(unsigned long *entry, void *page_addr, int flags);
+void dma_update_cpu_trans(unsigned long *entry, phys_addr_t page_addr, int flags);
extern const struct dma_map_ops s390_pci_dma_ops;
diff --git a/arch/s390/include/asm/processor.h b/arch/s390/include/asm/processor.h
index f54c152..5581b64 100644
--- a/arch/s390/include/asm/processor.h
+++ b/arch/s390/include/asm/processor.h
@@ -88,11 +88,10 @@ extern void __bpon(void);
* User space process size: 2GB for 31 bit, 4TB or 8PT for 64 bit.
*/
-#define TASK_SIZE_OF(tsk) (test_tsk_thread_flag(tsk, TIF_31BIT) ? \
+#define TASK_SIZE (test_thread_flag(TIF_31BIT) ? \
_REGION3_SIZE : TASK_SIZE_MAX)
#define TASK_UNMAPPED_BASE (test_thread_flag(TIF_31BIT) ? \
(_REGION3_SIZE >> 1) : (_REGION2_SIZE >> 1))
-#define TASK_SIZE TASK_SIZE_OF(current)
#define TASK_SIZE_MAX (-PAGE_SIZE)
#define STACK_TOP (test_thread_flag(TIF_31BIT) ? \
diff --git a/arch/s390/include/asm/qdio.h b/arch/s390/include/asm/qdio.h
index 4b9b14b..54ae2dc 100644
--- a/arch/s390/include/asm/qdio.h
+++ b/arch/s390/include/asm/qdio.h
@@ -18,7 +18,6 @@
#define QDIO_MAX_BUFFERS_MASK (QDIO_MAX_BUFFERS_PER_Q - 1)
#define QDIO_BUFNR(num) ((num) & QDIO_MAX_BUFFERS_MASK)
#define QDIO_MAX_ELEMENTS_PER_BUFFER 16
-#define QDIO_SBAL_SIZE 256
#define QDIO_QETH_QFMT 0
#define QDIO_ZFCP_QFMT 1
@@ -92,8 +91,8 @@ struct qdr {
* @pfmt: implementation dependent parameter format
* @rflags: QEBSM
* @ac: adapter characteristics
- * @isliba: absolute address of first input SLIB
- * @osliba: absolute address of first output SLIB
+ * @isliba: logical address of first input SLIB
+ * @osliba: logical address of first output SLIB
* @ebcnam: adapter identifier in EBCDIC
* @parm: implementation dependent parameters
*/
@@ -313,7 +312,7 @@ typedef void qdio_handler_t(struct ccw_device *, unsigned int, int,
* @qib_rflags: rflags to set
* @no_input_qs: number of input queues
* @no_output_qs: number of output queues
- * @input_handler: handler to be called for input queues
+ * @input_handler: handler to be called for input queues, and device-wide errors
* @output_handler: handler to be called for output queues
* @irq_poll: Data IRQ polling handler
* @scan_threshold: # of in-use buffers that triggers scan on output queue
@@ -337,9 +336,6 @@ struct qdio_initialize {
struct qdio_buffer ***output_sbal_addr_array;
};
-#define QDIO_FLAG_SYNC_INPUT 0x01
-#define QDIO_FLAG_SYNC_OUTPUT 0x02
-
int qdio_alloc_buffers(struct qdio_buffer **buf, unsigned int count);
void qdio_free_buffers(struct qdio_buffer **buf, unsigned int count);
void qdio_reset_buffers(struct qdio_buffer **buf, unsigned int count);
@@ -349,13 +345,18 @@ extern int qdio_allocate(struct ccw_device *cdev, unsigned int no_input_qs,
extern int qdio_establish(struct ccw_device *cdev,
struct qdio_initialize *init_data);
extern int qdio_activate(struct ccw_device *);
-extern int do_QDIO(struct ccw_device *cdev, unsigned int callflags, int q_nr,
- unsigned int bufnr, unsigned int count, struct qaob *aob);
extern int qdio_start_irq(struct ccw_device *cdev);
extern int qdio_stop_irq(struct ccw_device *cdev);
-extern int qdio_inspect_queue(struct ccw_device *cdev, unsigned int nr,
- bool is_input, unsigned int *bufnr,
- unsigned int *error);
+extern int qdio_inspect_input_queue(struct ccw_device *cdev, unsigned int nr,
+ unsigned int *bufnr, unsigned int *error);
+extern int qdio_inspect_output_queue(struct ccw_device *cdev, unsigned int nr,
+ unsigned int *bufnr, unsigned int *error);
+extern int qdio_add_bufs_to_input_queue(struct ccw_device *cdev,
+ unsigned int q_nr, unsigned int bufnr,
+ unsigned int count);
+extern int qdio_add_bufs_to_output_queue(struct ccw_device *cdev,
+ unsigned int q_nr, unsigned int bufnr,
+ unsigned int count, struct qaob *aob);
extern int qdio_shutdown(struct ccw_device *, int);
extern int qdio_free(struct ccw_device *);
extern int qdio_get_ssqd_desc(struct ccw_device *, struct qdio_ssqd_desc *);
diff --git a/arch/s390/include/asm/vx-insn.h b/arch/s390/include/asm/vx-insn.h
index 0c05a67..87e6cc2 100644
--- a/arch/s390/include/asm/vx-insn.h
+++ b/arch/s390/include/asm/vx-insn.h
@@ -372,6 +372,16 @@
MRXBOPC \hint, 0x36, v1, v3
.endm
+/* VECTOR STORE */
+.macro VST vr1, disp, index="%r0", base
+ VX_NUM v1, \vr1
+ GR_NUM x2, \index
+ GR_NUM b2, \base /* Base register */
+ .word 0xE700 | ((v1&15) << 4) | (x2&15)
+ .word (b2 << 12) | (\disp)
+ MRXBOPC 0, 0x0E, v1
+.endm
+
/* VECTOR STORE MULTIPLE */
.macro VSTM vfrom, vto, disp, base, hint=3
VX_NUM v1, \vfrom
@@ -411,6 +421,81 @@
VUPLL \vr1, \vr2, 2
.endm
+/* VECTOR PERMUTE DOUBLEWORD IMMEDIATE */
+.macro VPDI vr1, vr2, vr3, m4
+ VX_NUM v1, \vr1
+ VX_NUM v2, \vr2
+ VX_NUM v3, \vr3
+ .word 0xE700 | ((v1&15) << 4) | (v2&15)
+ .word ((v3&15) << 12)
+ MRXBOPC \m4, 0x84, v1, v2, v3
+.endm
+
+/* VECTOR REPLICATE */
+.macro VREP vr1, vr3, imm2, m4
+ VX_NUM v1, \vr1
+ VX_NUM v3, \vr3
+ .word 0xE700 | ((v1&15) << 4) | (v3&15)
+ .word \imm2
+ MRXBOPC \m4, 0x4D, v1, v3
+.endm
+.macro VREPB vr1, vr3, imm2
+ VREP \vr1, \vr3, \imm2, 0
+.endm
+.macro VREPH vr1, vr3, imm2
+ VREP \vr1, \vr3, \imm2, 1
+.endm
+.macro VREPF vr1, vr3, imm2
+ VREP \vr1, \vr3, \imm2, 2
+.endm
+.macro VREPG vr1, vr3, imm2
+ VREP \vr1, \vr3, \imm2, 3
+.endm
+
+/* VECTOR MERGE HIGH */
+.macro VMRH vr1, vr2, vr3, m4
+ VX_NUM v1, \vr1
+ VX_NUM v2, \vr2
+ VX_NUM v3, \vr3
+ .word 0xE700 | ((v1&15) << 4) | (v2&15)
+ .word ((v3&15) << 12)
+ MRXBOPC \m4, 0x61, v1, v2, v3
+.endm
+.macro VMRHB vr1, vr2, vr3
+ VMRH \vr1, \vr2, \vr3, 0
+.endm
+.macro VMRHH vr1, vr2, vr3
+ VMRH \vr1, \vr2, \vr3, 1
+.endm
+.macro VMRHF vr1, vr2, vr3
+ VMRH \vr1, \vr2, \vr3, 2
+.endm
+.macro VMRHG vr1, vr2, vr3
+ VMRH \vr1, \vr2, \vr3, 3
+.endm
+
+/* VECTOR MERGE LOW */
+.macro VMRL vr1, vr2, vr3, m4
+ VX_NUM v1, \vr1
+ VX_NUM v2, \vr2
+ VX_NUM v3, \vr3
+ .word 0xE700 | ((v1&15) << 4) | (v2&15)
+ .word ((v3&15) << 12)
+ MRXBOPC \m4, 0x60, v1, v2, v3
+.endm
+.macro VMRLB vr1, vr2, vr3
+ VMRL \vr1, \vr2, \vr3, 0
+.endm
+.macro VMRLH vr1, vr2, vr3
+ VMRL \vr1, \vr2, \vr3, 1
+.endm
+.macro VMRLF vr1, vr2, vr3
+ VMRL \vr1, \vr2, \vr3, 2
+.endm
+.macro VMRLG vr1, vr2, vr3
+ VMRL \vr1, \vr2, \vr3, 3
+.endm
+
/* Vector integer instructions */
@@ -557,5 +642,37 @@
VESRAV \vr1, \vr2, \vr3, 3
.endm
+/* VECTOR ELEMENT ROTATE LEFT LOGICAL */
+.macro VERLL vr1, vr3, disp, base="%r0", m4
+ VX_NUM v1, \vr1
+ VX_NUM v3, \vr3
+ GR_NUM b2, \base
+ .word 0xE700 | ((v1&15) << 4) | (v3&15)
+ .word (b2 << 12) | (\disp)
+ MRXBOPC \m4, 0x33, v1, v3
+.endm
+.macro VERLLB vr1, vr3, disp, base="%r0"
+ VERLL \vr1, \vr3, \disp, \base, 0
+.endm
+.macro VERLLH vr1, vr3, disp, base="%r0"
+ VERLL \vr1, \vr3, \disp, \base, 1
+.endm
+.macro VERLLF vr1, vr3, disp, base="%r0"
+ VERLL \vr1, \vr3, \disp, \base, 2
+.endm
+.macro VERLLG vr1, vr3, disp, base="%r0"
+ VERLL \vr1, \vr3, \disp, \base, 3
+.endm
+
+/* VECTOR SHIFT LEFT DOUBLE BY BYTE */
+.macro VSLDB vr1, vr2, vr3, imm4
+ VX_NUM v1, \vr1
+ VX_NUM v2, \vr2
+ VX_NUM v3, \vr3
+ .word 0xE700 | ((v1&15) << 4) | (v2&15)
+ .word ((v3&15) << 12) | (\imm4)
+ MRXBOPC 0, 0x77, v1, v2, v3
+.endm
+
#endif /* __ASSEMBLY__ */
#endif /* __ASM_S390_VX_INSN_H */
diff --git a/arch/s390/kernel/Makefile b/arch/s390/kernel/Makefile
index 80f500f..be8007f 100644
--- a/arch/s390/kernel/Makefile
+++ b/arch/s390/kernel/Makefile
@@ -40,7 +40,7 @@
obj-y += runtime_instr.o cache.o fpu.o dumpstack.o guarded_storage.o sthyi.o
obj-y += entry.o reipl.o relocate_kernel.o kdebugfs.o alternative.o
obj-y += nospec-branch.o ipl_vmparm.o machine_kexec_reloc.o unwind_bc.o
-obj-y += smp.o text_amode31.o
+obj-y += smp.o text_amode31.o stacktrace.o
extra-y += head64.o vmlinux.lds
@@ -55,7 +55,6 @@
obj-$(CONFIG_COMPAT) += compat_linux.o compat_signal.o
obj-$(CONFIG_COMPAT) += $(compat-obj-y)
obj-$(CONFIG_EARLY_PRINTK) += early_printk.o
-obj-$(CONFIG_STACKTRACE) += stacktrace.o
obj-$(CONFIG_KPROBES) += kprobes.o
obj-$(CONFIG_KPROBES) += kprobes_insn_page.o
obj-$(CONFIG_FUNCTION_TRACER) += mcount.o ftrace.o
diff --git a/arch/s390/kernel/crash_dump.c b/arch/s390/kernel/crash_dump.c
index 785d54c..af82021 100644
--- a/arch/s390/kernel/crash_dump.c
+++ b/arch/s390/kernel/crash_dump.c
@@ -60,7 +60,7 @@ struct save_area * __init save_area_alloc(bool is_boot_cpu)
{
struct save_area *sa;
- sa = (void *) memblock_phys_alloc(sizeof(*sa), 8);
+ sa = memblock_alloc(sizeof(*sa), 8);
if (!sa)
panic("Failed to allocate save area\n");
diff --git a/arch/s390/kernel/dis.c b/arch/s390/kernel/dis.c
index ec55154..90bbb4e 100644
--- a/arch/s390/kernel/dis.c
+++ b/arch/s390/kernel/dis.c
@@ -278,6 +278,7 @@ static const unsigned char formats[][6] = {
[INSTR_SIL_RDI] = { D_20, B_16, I16_32, 0, 0, 0 },
[INSTR_SIL_RDU] = { D_20, B_16, U16_32, 0, 0, 0 },
[INSTR_SIY_IRD] = { D20_20, B_16, I8_8, 0, 0, 0 },
+ [INSTR_SIY_RD] = { D20_20, B_16, 0, 0, 0, 0 },
[INSTR_SIY_URD] = { D20_20, B_16, U8_8, 0, 0, 0 },
[INSTR_SI_RD] = { D_20, B_16, 0, 0, 0, 0 },
[INSTR_SI_URD] = { D_20, B_16, U8_8, 0, 0, 0 },
diff --git a/arch/s390/kernel/machine_kexec.c b/arch/s390/kernel/machine_kexec.c
index 0505e55..a16467b 100644
--- a/arch/s390/kernel/machine_kexec.c
+++ b/arch/s390/kernel/machine_kexec.c
@@ -86,7 +86,7 @@ static noinline void __machine_kdump(void *image)
continue;
}
/* Store status of the boot CPU */
- mcesa = (struct mcesa *)(S390_lowcore.mcesad & MCESA_ORIGIN_MASK);
+ mcesa = __va(S390_lowcore.mcesad & MCESA_ORIGIN_MASK);
if (MACHINE_HAS_VX)
save_vx_regs((__vector128 *) mcesa->vector_save_area);
if (MACHINE_HAS_GS) {
diff --git a/arch/s390/kernel/nmi.c b/arch/s390/kernel/nmi.c
index 20f8e18..1cf1e37 100644
--- a/arch/s390/kernel/nmi.c
+++ b/arch/s390/kernel/nmi.c
@@ -58,27 +58,27 @@ static inline unsigned long nmi_get_mcesa_size(void)
/*
* The initial machine check extended save area for the boot CPU.
- * It will be replaced by nmi_init() with an allocated structure.
- * The structure is required for machine check happening early in
- * the boot process.
+ * It will be replaced on the boot CPU reinit with an allocated
+ * structure. The structure is required for machine check happening
+ * early in the boot process.
*/
static struct mcesa boot_mcesa __initdata __aligned(MCESA_MAX_SIZE);
-void __init nmi_alloc_boot_cpu(struct lowcore *lc)
+void __init nmi_alloc_mcesa_early(u64 *mcesad)
{
if (!nmi_needs_mcesa())
return;
- lc->mcesad = (unsigned long) &boot_mcesa;
+ *mcesad = __pa(&boot_mcesa);
if (MACHINE_HAS_GS)
- lc->mcesad |= ilog2(MCESA_MAX_SIZE);
+ *mcesad |= ilog2(MCESA_MAX_SIZE);
}
-static int __init nmi_init(void)
+static void __init nmi_alloc_cache(void)
{
- unsigned long origin, cr0, size;
+ unsigned long size;
if (!nmi_needs_mcesa())
- return 0;
+ return;
size = nmi_get_mcesa_size();
if (size > MCESA_MIN_SIZE)
mcesa_origin_lc = ilog2(size);
@@ -86,40 +86,31 @@ static int __init nmi_init(void)
mcesa_cache = kmem_cache_create("nmi_save_areas", size, size, 0, NULL);
if (!mcesa_cache)
panic("Couldn't create nmi save area cache");
- origin = (unsigned long) kmem_cache_alloc(mcesa_cache, GFP_KERNEL);
- if (!origin)
- panic("Couldn't allocate nmi save area");
- /* The pointer is stored with mcesa_bits ORed in */
- kmemleak_not_leak((void *) origin);
- __ctl_store(cr0, 0, 0);
- __ctl_clear_bit(0, 28); /* disable lowcore protection */
- /* Replace boot_mcesa on the boot CPU */
- S390_lowcore.mcesad = origin | mcesa_origin_lc;
- __ctl_load(cr0, 0, 0);
- return 0;
}
-early_initcall(nmi_init);
-int nmi_alloc_per_cpu(struct lowcore *lc)
+int __ref nmi_alloc_mcesa(u64 *mcesad)
{
unsigned long origin;
+ *mcesad = 0;
if (!nmi_needs_mcesa())
return 0;
+ if (!mcesa_cache)
+ nmi_alloc_cache();
origin = (unsigned long) kmem_cache_alloc(mcesa_cache, GFP_KERNEL);
if (!origin)
return -ENOMEM;
/* The pointer is stored with mcesa_bits ORed in */
kmemleak_not_leak((void *) origin);
- lc->mcesad = origin | mcesa_origin_lc;
+ *mcesad = __pa(origin) | mcesa_origin_lc;
return 0;
}
-void nmi_free_per_cpu(struct lowcore *lc)
+void nmi_free_mcesa(u64 *mcesad)
{
if (!nmi_needs_mcesa())
return;
- kmem_cache_free(mcesa_cache, (void *)(lc->mcesad & MCESA_ORIGIN_MASK));
+ kmem_cache_free(mcesa_cache, __va(*mcesad & MCESA_ORIGIN_MASK));
}
static notrace void s390_handle_damage(void)
@@ -246,7 +237,7 @@ static int notrace s390_validate_registers(union mci mci, int umode)
: "Q" (S390_lowcore.fpt_creg_save_area));
}
- mcesa = (struct mcesa *)(S390_lowcore.mcesad & MCESA_ORIGIN_MASK);
+ mcesa = __va(S390_lowcore.mcesad & MCESA_ORIGIN_MASK);
if (!MACHINE_HAS_VX) {
/* Validate floating point registers */
asm volatile(
diff --git a/arch/s390/kernel/process.c b/arch/s390/kernel/process.c
index e8858b2..71d86f7 100644
--- a/arch/s390/kernel/process.c
+++ b/arch/s390/kernel/process.c
@@ -139,7 +139,6 @@ int copy_thread(unsigned long clone_flags, unsigned long new_stackp,
(unsigned long)__ret_from_fork;
frame->childregs.gprs[9] = new_stackp; /* function */
frame->childregs.gprs[10] = arg;
- frame->childregs.gprs[11] = (unsigned long)do_exit;
frame->childregs.orig_gpr2 = -1;
frame->childregs.last_break = 1;
return 0;
diff --git a/arch/s390/kernel/setup.c b/arch/s390/kernel/setup.c
index 225ab2d..f2c25d1 100644
--- a/arch/s390/kernel/setup.c
+++ b/arch/s390/kernel/setup.c
@@ -445,7 +445,7 @@ static void __init setup_lowcore_dat_off(void)
lc->lpp = LPP_MAGIC;
lc->machine_flags = S390_lowcore.machine_flags;
lc->preempt_count = S390_lowcore.preempt_count;
- nmi_alloc_boot_cpu(lc);
+ nmi_alloc_mcesa_early(&lc->mcesad);
lc->sys_enter_timer = S390_lowcore.sys_enter_timer;
lc->exit_timer = S390_lowcore.exit_timer;
lc->user_timer = S390_lowcore.user_timer;
diff --git a/arch/s390/kernel/smp.c b/arch/s390/kernel/smp.c
index 78a8ea6..2bad902 100644
--- a/arch/s390/kernel/smp.c
+++ b/arch/s390/kernel/smp.c
@@ -212,7 +212,7 @@ static int pcpu_alloc_lowcore(struct pcpu *pcpu, int cpu)
lc->return_lpswe = gen_lpswe(__LC_RETURN_PSW);
lc->return_mcck_lpswe = gen_lpswe(__LC_RETURN_MCCK_PSW);
lc->preempt_count = PREEMPT_DISABLED;
- if (nmi_alloc_per_cpu(lc))
+ if (nmi_alloc_mcesa(&lc->mcesad))
goto out;
lowcore_ptr[cpu] = lc;
pcpu_sigp_retry(pcpu, SIGP_SET_PREFIX, (u32)(unsigned long) lc);
@@ -239,7 +239,7 @@ static void pcpu_free_lowcore(struct pcpu *pcpu)
mcck_stack = lc->mcck_stack - STACK_INIT_OFFSET;
pcpu_sigp_retry(pcpu, SIGP_SET_PREFIX, 0);
lowcore_ptr[cpu] = NULL;
- nmi_free_per_cpu(lc);
+ nmi_free_mcesa(&lc->mcesad);
stack_free(async_stack);
stack_free(mcck_stack);
free_pages(nodat_stack, THREAD_SIZE_ORDER);
@@ -622,7 +622,7 @@ int smp_store_status(int cpu)
return -EIO;
if (!MACHINE_HAS_VX && !MACHINE_HAS_GS)
return 0;
- pa = __pa(lc->mcesad & MCESA_ORIGIN_MASK);
+ pa = lc->mcesad & MCESA_ORIGIN_MASK;
if (MACHINE_HAS_GS)
pa |= lc->mcesad & MCESA_LC_MASK;
if (__pcpu_sigp_relax(pcpu->address, SIGP_STORE_ADDITIONAL_STATUS,
@@ -658,26 +658,22 @@ int smp_store_status(int cpu)
* deactivates the elfcorehdr= kernel parameter
*/
static __init void smp_save_cpu_vxrs(struct save_area *sa, u16 addr,
- bool is_boot_cpu, unsigned long page)
+ bool is_boot_cpu, __vector128 *vxrs)
{
- __vector128 *vxrs = (__vector128 *) page;
-
if (is_boot_cpu)
vxrs = boot_cpu_vector_save_area;
else
- __pcpu_sigp_relax(addr, SIGP_STORE_ADDITIONAL_STATUS, page);
+ __pcpu_sigp_relax(addr, SIGP_STORE_ADDITIONAL_STATUS, __pa(vxrs));
save_area_add_vxrs(sa, vxrs);
}
static __init void smp_save_cpu_regs(struct save_area *sa, u16 addr,
- bool is_boot_cpu, unsigned long page)
+ bool is_boot_cpu, void *regs)
{
- void *regs = (void *) page;
-
if (is_boot_cpu)
copy_oldmem_kernel(regs, (void *) __LC_FPREGS_SAVE_AREA, 512);
else
- __pcpu_sigp_relax(addr, SIGP_STORE_STATUS_AT_ADDRESS, page);
+ __pcpu_sigp_relax(addr, SIGP_STORE_STATUS_AT_ADDRESS, __pa(regs));
save_area_add_regs(sa, regs);
}
@@ -685,14 +681,14 @@ void __init smp_save_dump_cpus(void)
{
int addr, boot_cpu_addr, max_cpu_addr;
struct save_area *sa;
- unsigned long page;
bool is_boot_cpu;
+ void *page;
if (!(oldmem_data.start || is_ipl_type_dump()))
/* No previous system present, normal boot. */
return;
/* Allocate a page as dumping area for the store status sigps */
- page = memblock_phys_alloc_range(PAGE_SIZE, PAGE_SIZE, 0, 1UL << 31);
+ page = memblock_alloc_low(PAGE_SIZE, PAGE_SIZE);
if (!page)
panic("ERROR: Failed to allocate %lx bytes below %lx\n",
PAGE_SIZE, 1UL << 31);
@@ -723,7 +719,7 @@ void __init smp_save_dump_cpus(void)
/* Get the CPU registers */
smp_save_cpu_regs(sa, addr, is_boot_cpu, page);
}
- memblock_phys_free(page, PAGE_SIZE);
+ memblock_free(page, PAGE_SIZE);
diag_amode31_ops.diag308_reset();
pcpu_set_smt(0);
}
@@ -880,7 +876,7 @@ void __init smp_detect_cpus(void)
/* Add CPUs present at boot */
__smp_rescan_cpus(info, true);
- memblock_phys_free((unsigned long)info, sizeof(*info));
+ memblock_free(info, sizeof(*info));
}
/*
@@ -1271,14 +1267,15 @@ static int __init smp_reinit_ipl_cpu(void)
{
unsigned long async_stack, nodat_stack, mcck_stack;
struct lowcore *lc, *lc_ipl;
- unsigned long flags;
+ unsigned long flags, cr0;
+ u64 mcesad;
lc_ipl = lowcore_ptr[0];
lc = (struct lowcore *) __get_free_pages(GFP_KERNEL | GFP_DMA, LC_ORDER);
nodat_stack = __get_free_pages(GFP_KERNEL, THREAD_SIZE_ORDER);
async_stack = stack_alloc();
mcck_stack = stack_alloc();
- if (!lc || !nodat_stack || !async_stack || !mcck_stack)
+ if (!lc || !nodat_stack || !async_stack || !mcck_stack || nmi_alloc_mcesa(&mcesad))
panic("Couldn't allocate memory");
local_irq_save(flags);
@@ -1287,6 +1284,10 @@ static int __init smp_reinit_ipl_cpu(void)
S390_lowcore.nodat_stack = nodat_stack + STACK_INIT_OFFSET;
S390_lowcore.async_stack = async_stack + STACK_INIT_OFFSET;
S390_lowcore.mcck_stack = mcck_stack + STACK_INIT_OFFSET;
+ __ctl_store(cr0, 0, 0);
+ __ctl_clear_bit(0, 28); /* disable lowcore protection */
+ S390_lowcore.mcesad = mcesad;
+ __ctl_load(cr0, 0, 0);
lowcore_ptr[0] = lc;
local_mcck_enable();
local_irq_restore(flags);
diff --git a/arch/s390/kernel/uv.c b/arch/s390/kernel/uv.c
index 386d4e4..a542507 100644
--- a/arch/s390/kernel/uv.c
+++ b/arch/s390/kernel/uv.c
@@ -30,7 +30,7 @@ int __bootdata_preserved(prot_virt_host);
EXPORT_SYMBOL(prot_virt_host);
EXPORT_SYMBOL(uv_info);
-static int __init uv_init(unsigned long stor_base, unsigned long stor_len)
+static int __init uv_init(phys_addr_t stor_base, unsigned long stor_len)
{
struct uv_cb_init uvcb = {
.header.cmd = UVC_CMD_INIT_UV,
@@ -49,12 +49,12 @@ static int __init uv_init(unsigned long stor_base, unsigned long stor_len)
void __init setup_uv(void)
{
- unsigned long uv_stor_base;
+ void *uv_stor_base;
if (!is_prot_virt_host())
return;
- uv_stor_base = (unsigned long)memblock_alloc_try_nid(
+ uv_stor_base = memblock_alloc_try_nid(
uv_info.uv_base_stor_len, SZ_1M, SZ_2G,
MEMBLOCK_ALLOC_ACCESSIBLE, NUMA_NO_NODE);
if (!uv_stor_base) {
@@ -63,8 +63,8 @@ void __init setup_uv(void)
goto fail;
}
- if (uv_init(uv_stor_base, uv_info.uv_base_stor_len)) {
- memblock_phys_free(uv_stor_base, uv_info.uv_base_stor_len);
+ if (uv_init(__pa(uv_stor_base), uv_info.uv_base_stor_len)) {
+ memblock_free(uv_stor_base, uv_info.uv_base_stor_len);
goto fail;
}
diff --git a/arch/s390/mm/cmm.c b/arch/s390/mm/cmm.c
index 2203164..9141ed4 100644
--- a/arch/s390/mm/cmm.c
+++ b/arch/s390/mm/cmm.c
@@ -90,7 +90,7 @@ static long cmm_alloc_pages(long nr, long *counter,
} else
free_page((unsigned long) npa);
}
- diag10_range(addr >> PAGE_SHIFT, 1);
+ diag10_range(virt_to_pfn(addr), 1);
pa->pages[pa->index++] = addr;
(*counter)++;
spin_unlock(&cmm_lock);
diff --git a/arch/s390/mm/fault.c b/arch/s390/mm/fault.c
index d30f598..6ed2886 100644
--- a/arch/s390/mm/fault.c
+++ b/arch/s390/mm/fault.c
@@ -115,7 +115,7 @@ static void dump_pagetable(unsigned long asce, unsigned long address)
pr_cont("R1:%016lx ", *table);
if (*table & _REGION_ENTRY_INVALID)
goto out;
- table = (unsigned long *)(*table & _REGION_ENTRY_ORIGIN);
+ table = __va(*table & _REGION_ENTRY_ORIGIN);
fallthrough;
case _ASCE_TYPE_REGION2:
table += (address & _REGION2_INDEX) >> _REGION2_SHIFT;
@@ -124,7 +124,7 @@ static void dump_pagetable(unsigned long asce, unsigned long address)
pr_cont("R2:%016lx ", *table);
if (*table & _REGION_ENTRY_INVALID)
goto out;
- table = (unsigned long *)(*table & _REGION_ENTRY_ORIGIN);
+ table = __va(*table & _REGION_ENTRY_ORIGIN);
fallthrough;
case _ASCE_TYPE_REGION3:
table += (address & _REGION3_INDEX) >> _REGION3_SHIFT;
@@ -133,7 +133,7 @@ static void dump_pagetable(unsigned long asce, unsigned long address)
pr_cont("R3:%016lx ", *table);
if (*table & (_REGION_ENTRY_INVALID | _REGION3_ENTRY_LARGE))
goto out;
- table = (unsigned long *)(*table & _REGION_ENTRY_ORIGIN);
+ table = __va(*table & _REGION_ENTRY_ORIGIN);
fallthrough;
case _ASCE_TYPE_SEGMENT:
table += (address & _SEGMENT_INDEX) >> _SEGMENT_SHIFT;
@@ -142,7 +142,7 @@ static void dump_pagetable(unsigned long asce, unsigned long address)
pr_cont("S:%016lx ", *table);
if (*table & (_SEGMENT_ENTRY_INVALID | _SEGMENT_ENTRY_LARGE))
goto out;
- table = (unsigned long *)(*table & _SEGMENT_ENTRY_ORIGIN);
+ table = __va(*table & _SEGMENT_ENTRY_ORIGIN);
}
table += (address & _PAGE_INDEX) >> _PAGE_SHIFT;
if (bad_address(table))
diff --git a/arch/s390/mm/hugetlbpage.c b/arch/s390/mm/hugetlbpage.c
index da36d13..082793d 100644
--- a/arch/s390/mm/hugetlbpage.c
+++ b/arch/s390/mm/hugetlbpage.c
@@ -9,6 +9,7 @@
#define KMSG_COMPONENT "hugetlb"
#define pr_fmt(fmt) KMSG_COMPONENT ": " fmt
+#include <asm/pgalloc.h>
#include <linux/mm.h>
#include <linux/hugetlb.h>
#include <linux/mman.h>
diff --git a/arch/s390/mm/init.c b/arch/s390/mm/init.c
index 8c6f258..86ffd0d 100644
--- a/arch/s390/mm/init.c
+++ b/arch/s390/mm/init.c
@@ -215,6 +215,9 @@ void free_initmem(void)
__set_memory((unsigned long)_sinittext,
(unsigned long)(_einittext - _sinittext) >> PAGE_SHIFT,
SET_MEMORY_RW | SET_MEMORY_NX);
+ free_reserved_area(sclp_early_sccb,
+ sclp_early_sccb + EXT_SCCB_READ_SCP,
+ POISON_FREE_INITMEM, "unused early sccb");
free_initmem_default(POISON_FREE_INITMEM);
}
diff --git a/arch/s390/mm/pgalloc.c b/arch/s390/mm/pgalloc.c
index 781965f..fd35c1a 100644
--- a/arch/s390/mm/pgalloc.c
+++ b/arch/s390/mm/pgalloc.c
@@ -176,7 +176,75 @@ void page_table_free_pgste(struct page *page)
#endif /* CONFIG_PGSTE */
/*
- * page table entry allocation/free routines.
+ * A 2KB-pgtable is either upper or lower half of a normal page.
+ * The second half of the page may be unused or used as another
+ * 2KB-pgtable.
+ *
+ * Whenever possible the parent page for a new 2KB-pgtable is picked
+ * from the list of partially allocated pages mm_context_t::pgtable_list.
+ * In case the list is empty a new parent page is allocated and added to
+ * the list.
+ *
+ * When a parent page gets fully allocated it contains 2KB-pgtables in both
+ * upper and lower halves and is removed from mm_context_t::pgtable_list.
+ *
+ * When 2KB-pgtable is freed from to fully allocated parent page that
+ * page turns partially allocated and added to mm_context_t::pgtable_list.
+ *
+ * If 2KB-pgtable is freed from the partially allocated parent page that
+ * page turns unused and gets removed from mm_context_t::pgtable_list.
+ * Furthermore, the unused parent page is released.
+ *
+ * As follows from the above, no unallocated or fully allocated parent
+ * pages are contained in mm_context_t::pgtable_list.
+ *
+ * The upper byte (bits 24-31) of the parent page _refcount is used
+ * for tracking contained 2KB-pgtables and has the following format:
+ *
+ * PP AA
+ * 01234567 upper byte (bits 24-31) of struct page::_refcount
+ * || ||
+ * || |+--- upper 2KB-pgtable is allocated
+ * || +---- lower 2KB-pgtable is allocated
+ * |+------- upper 2KB-pgtable is pending for removal
+ * +-------- lower 2KB-pgtable is pending for removal
+ *
+ * (See commit 620b4e903179 ("s390: use _refcount for pgtables") on why
+ * using _refcount is possible).
+ *
+ * When 2KB-pgtable is allocated the corresponding AA bit is set to 1.
+ * The parent page is either:
+ * - added to mm_context_t::pgtable_list in case the second half of the
+ * parent page is still unallocated;
+ * - removed from mm_context_t::pgtable_list in case both hales of the
+ * parent page are allocated;
+ * These operations are protected with mm_context_t::lock.
+ *
+ * When 2KB-pgtable is deallocated the corresponding AA bit is set to 0
+ * and the corresponding PP bit is set to 1 in a single atomic operation.
+ * Thus, PP and AA bits corresponding to the same 2KB-pgtable are mutually
+ * exclusive and may never be both set to 1!
+ * The parent page is either:
+ * - added to mm_context_t::pgtable_list in case the second half of the
+ * parent page is still allocated;
+ * - removed from mm_context_t::pgtable_list in case the second half of
+ * the parent page is unallocated;
+ * These operations are protected with mm_context_t::lock.
+ *
+ * It is important to understand that mm_context_t::lock only protects
+ * mm_context_t::pgtable_list and AA bits, but not the parent page itself
+ * and PP bits.
+ *
+ * Releasing the parent page happens whenever the PP bit turns from 1 to 0,
+ * while both AA bits and the second PP bit are already unset. Then the
+ * parent page does not contain any 2KB-pgtable fragment anymore, and it has
+ * also been removed from mm_context_t::pgtable_list. It is safe to release
+ * the page therefore.
+ *
+ * PGSTE memory spaces use full 4KB-pgtables and do not need most of the
+ * logic described above. Both AA bits are set to 1 to denote a 4KB-pgtable
+ * while the PP bits are never used, nor such a page is added to or removed
+ * from mm_context_t::pgtable_list.
*/
unsigned long *page_table_alloc(struct mm_struct *mm)
{
@@ -192,14 +260,23 @@ unsigned long *page_table_alloc(struct mm_struct *mm)
page = list_first_entry(&mm->context.pgtable_list,
struct page, lru);
mask = atomic_read(&page->_refcount) >> 24;
- mask = (mask | (mask >> 4)) & 3;
- if (mask != 3) {
+ /*
+ * The pending removal bits must also be checked.
+ * Failure to do so might lead to an impossible
+ * value of (i.e 0x13 or 0x23) written to _refcount.
+ * Such values violate the assumption that pending and
+ * allocation bits are mutually exclusive, and the rest
+ * of the code unrails as result. That could lead to
+ * a whole bunch of races and corruptions.
+ */
+ mask = (mask | (mask >> 4)) & 0x03U;
+ if (mask != 0x03U) {
table = (unsigned long *) page_to_virt(page);
bit = mask & 1; /* =1 -> second 2K */
if (bit)
table += PTRS_PER_PTE;
atomic_xor_bits(&page->_refcount,
- 1U << (bit + 24));
+ 0x01U << (bit + 24));
list_del(&page->lru);
}
}
@@ -220,12 +297,12 @@ unsigned long *page_table_alloc(struct mm_struct *mm)
table = (unsigned long *) page_to_virt(page);
if (mm_alloc_pgste(mm)) {
/* Return 4K page table with PGSTEs */
- atomic_xor_bits(&page->_refcount, 3 << 24);
+ atomic_xor_bits(&page->_refcount, 0x03U << 24);
memset64((u64 *)table, _PAGE_INVALID, PTRS_PER_PTE);
memset64((u64 *)table + PTRS_PER_PTE, 0, PTRS_PER_PTE);
} else {
/* Return the first 2K fragment of the page */
- atomic_xor_bits(&page->_refcount, 1 << 24);
+ atomic_xor_bits(&page->_refcount, 0x01U << 24);
memset64((u64 *)table, _PAGE_INVALID, 2 * PTRS_PER_PTE);
spin_lock_bh(&mm->context.lock);
list_add(&page->lru, &mm->context.pgtable_list);
@@ -234,29 +311,53 @@ unsigned long *page_table_alloc(struct mm_struct *mm)
return table;
}
+static void page_table_release_check(struct page *page, void *table,
+ unsigned int half, unsigned int mask)
+{
+ char msg[128];
+
+ if (!IS_ENABLED(CONFIG_DEBUG_VM) || !mask)
+ return;
+ snprintf(msg, sizeof(msg),
+ "Invalid pgtable %p release half 0x%02x mask 0x%02x",
+ table, half, mask);
+ dump_page(page, msg);
+}
+
void page_table_free(struct mm_struct *mm, unsigned long *table)
{
+ unsigned int mask, bit, half;
struct page *page;
- unsigned int bit, mask;
page = virt_to_page(table);
if (!mm_alloc_pgste(mm)) {
/* Free 2K page table fragment of a 4K page */
bit = ((unsigned long) table & ~PAGE_MASK)/(PTRS_PER_PTE*sizeof(pte_t));
spin_lock_bh(&mm->context.lock);
- mask = atomic_xor_bits(&page->_refcount, 1U << (bit + 24));
+ /*
+ * Mark the page for delayed release. The actual release
+ * will happen outside of the critical section from this
+ * function or from __tlb_remove_table()
+ */
+ mask = atomic_xor_bits(&page->_refcount, 0x11U << (bit + 24));
mask >>= 24;
- if (mask & 3)
+ if (mask & 0x03U)
list_add(&page->lru, &mm->context.pgtable_list);
else
list_del(&page->lru);
spin_unlock_bh(&mm->context.lock);
- if (mask != 0)
+ mask = atomic_xor_bits(&page->_refcount, 0x10U << (bit + 24));
+ mask >>= 24;
+ if (mask != 0x00U)
return;
+ half = 0x01U << bit;
} else {
- atomic_xor_bits(&page->_refcount, 3U << 24);
+ half = 0x03U;
+ mask = atomic_xor_bits(&page->_refcount, 0x03U << 24);
+ mask >>= 24;
}
+ page_table_release_check(page, table, half, mask);
pgtable_pte_page_dtor(page);
__free_page(page);
}
@@ -272,47 +373,54 @@ void page_table_free_rcu(struct mmu_gather *tlb, unsigned long *table,
page = virt_to_page(table);
if (mm_alloc_pgste(mm)) {
gmap_unlink(mm, table, vmaddr);
- table = (unsigned long *) ((unsigned long)table | 3);
+ table = (unsigned long *) ((unsigned long)table | 0x03U);
tlb_remove_table(tlb, table);
return;
}
bit = ((unsigned long) table & ~PAGE_MASK) / (PTRS_PER_PTE*sizeof(pte_t));
spin_lock_bh(&mm->context.lock);
+ /*
+ * Mark the page for delayed release. The actual release will happen
+ * outside of the critical section from __tlb_remove_table() or from
+ * page_table_free()
+ */
mask = atomic_xor_bits(&page->_refcount, 0x11U << (bit + 24));
mask >>= 24;
- if (mask & 3)
+ if (mask & 0x03U)
list_add_tail(&page->lru, &mm->context.pgtable_list);
else
list_del(&page->lru);
spin_unlock_bh(&mm->context.lock);
- table = (unsigned long *) ((unsigned long) table | (1U << bit));
+ table = (unsigned long *) ((unsigned long) table | (0x01U << bit));
tlb_remove_table(tlb, table);
}
void __tlb_remove_table(void *_table)
{
- unsigned int mask = (unsigned long) _table & 3;
+ unsigned int mask = (unsigned long) _table & 0x03U, half = mask;
void *table = (void *)((unsigned long) _table ^ mask);
struct page *page = virt_to_page(table);
- switch (mask) {
- case 0: /* pmd, pud, or p4d */
+ switch (half) {
+ case 0x00U: /* pmd, pud, or p4d */
free_pages((unsigned long) table, 2);
- break;
- case 1: /* lower 2K of a 4K page table */
- case 2: /* higher 2K of a 4K page table */
+ return;
+ case 0x01U: /* lower 2K of a 4K page table */
+ case 0x02U: /* higher 2K of a 4K page table */
mask = atomic_xor_bits(&page->_refcount, mask << (4 + 24));
mask >>= 24;
- if (mask != 0)
- break;
- fallthrough;
- case 3: /* 4K page table with pgstes */
- if (mask & 3)
- atomic_xor_bits(&page->_refcount, 3 << 24);
- pgtable_pte_page_dtor(page);
- __free_page(page);
+ if (mask != 0x00U)
+ return;
+ break;
+ case 0x03U: /* 4K page table with pgstes */
+ mask = atomic_xor_bits(&page->_refcount, 0x03U << 24);
+ mask >>= 24;
break;
}
+
+ page_table_release_check(page, table, half, mask);
+ pgtable_pte_page_dtor(page);
+ __free_page(page);
}
/*
@@ -322,34 +430,34 @@ void __tlb_remove_table(void *_table)
static struct kmem_cache *base_pgt_cache;
-static unsigned long base_pgt_alloc(void)
+static unsigned long *base_pgt_alloc(void)
{
- u64 *table;
+ unsigned long *table;
table = kmem_cache_alloc(base_pgt_cache, GFP_KERNEL);
if (table)
- memset64(table, _PAGE_INVALID, PTRS_PER_PTE);
- return (unsigned long) table;
-}
-
-static void base_pgt_free(unsigned long table)
-{
- kmem_cache_free(base_pgt_cache, (void *) table);
-}
-
-static unsigned long base_crst_alloc(unsigned long val)
-{
- unsigned long table;
-
- table = __get_free_pages(GFP_KERNEL, CRST_ALLOC_ORDER);
- if (table)
- crst_table_init((unsigned long *)table, val);
+ memset64((u64 *)table, _PAGE_INVALID, PTRS_PER_PTE);
return table;
}
-static void base_crst_free(unsigned long table)
+static void base_pgt_free(unsigned long *table)
{
- free_pages(table, CRST_ALLOC_ORDER);
+ kmem_cache_free(base_pgt_cache, table);
+}
+
+static unsigned long *base_crst_alloc(unsigned long val)
+{
+ unsigned long *table;
+
+ table = (unsigned long *)__get_free_pages(GFP_KERNEL, CRST_ALLOC_ORDER);
+ if (table)
+ crst_table_init(table, val);
+ return table;
+}
+
+static void base_crst_free(unsigned long *table)
+{
+ free_pages((unsigned long)table, CRST_ALLOC_ORDER);
}
#define BASE_ADDR_END_FUNC(NAME, SIZE) \
@@ -377,14 +485,14 @@ static inline unsigned long base_lra(unsigned long address)
return real;
}
-static int base_page_walk(unsigned long origin, unsigned long addr,
+static int base_page_walk(unsigned long *origin, unsigned long addr,
unsigned long end, int alloc)
{
unsigned long *pte, next;
if (!alloc)
return 0;
- pte = (unsigned long *) origin;
+ pte = origin;
pte += (addr & _PAGE_INDEX) >> _PAGE_SHIFT;
do {
next = base_page_addr_end(addr, end);
@@ -393,13 +501,13 @@ static int base_page_walk(unsigned long origin, unsigned long addr,
return 0;
}
-static int base_segment_walk(unsigned long origin, unsigned long addr,
+static int base_segment_walk(unsigned long *origin, unsigned long addr,
unsigned long end, int alloc)
{
- unsigned long *ste, next, table;
+ unsigned long *ste, next, *table;
int rc;
- ste = (unsigned long *) origin;
+ ste = origin;
ste += (addr & _SEGMENT_INDEX) >> _SEGMENT_SHIFT;
do {
next = base_segment_addr_end(addr, end);
@@ -409,9 +517,9 @@ static int base_segment_walk(unsigned long origin, unsigned long addr,
table = base_pgt_alloc();
if (!table)
return -ENOMEM;
- *ste = table | _SEGMENT_ENTRY;
+ *ste = __pa(table) | _SEGMENT_ENTRY;
}
- table = *ste & _SEGMENT_ENTRY_ORIGIN;
+ table = __va(*ste & _SEGMENT_ENTRY_ORIGIN);
rc = base_page_walk(table, addr, next, alloc);
if (rc)
return rc;
@@ -422,13 +530,13 @@ static int base_segment_walk(unsigned long origin, unsigned long addr,
return 0;
}
-static int base_region3_walk(unsigned long origin, unsigned long addr,
+static int base_region3_walk(unsigned long *origin, unsigned long addr,
unsigned long end, int alloc)
{
- unsigned long *rtte, next, table;
+ unsigned long *rtte, next, *table;
int rc;
- rtte = (unsigned long *) origin;
+ rtte = origin;
rtte += (addr & _REGION3_INDEX) >> _REGION3_SHIFT;
do {
next = base_region3_addr_end(addr, end);
@@ -438,9 +546,9 @@ static int base_region3_walk(unsigned long origin, unsigned long addr,
table = base_crst_alloc(_SEGMENT_ENTRY_EMPTY);
if (!table)
return -ENOMEM;
- *rtte = table | _REGION3_ENTRY;
+ *rtte = __pa(table) | _REGION3_ENTRY;
}
- table = *rtte & _REGION_ENTRY_ORIGIN;
+ table = __va(*rtte & _REGION_ENTRY_ORIGIN);
rc = base_segment_walk(table, addr, next, alloc);
if (rc)
return rc;
@@ -450,13 +558,13 @@ static int base_region3_walk(unsigned long origin, unsigned long addr,
return 0;
}
-static int base_region2_walk(unsigned long origin, unsigned long addr,
+static int base_region2_walk(unsigned long *origin, unsigned long addr,
unsigned long end, int alloc)
{
- unsigned long *rste, next, table;
+ unsigned long *rste, next, *table;
int rc;
- rste = (unsigned long *) origin;
+ rste = origin;
rste += (addr & _REGION2_INDEX) >> _REGION2_SHIFT;
do {
next = base_region2_addr_end(addr, end);
@@ -466,9 +574,9 @@ static int base_region2_walk(unsigned long origin, unsigned long addr,
table = base_crst_alloc(_REGION3_ENTRY_EMPTY);
if (!table)
return -ENOMEM;
- *rste = table | _REGION2_ENTRY;
+ *rste = __pa(table) | _REGION2_ENTRY;
}
- table = *rste & _REGION_ENTRY_ORIGIN;
+ table = __va(*rste & _REGION_ENTRY_ORIGIN);
rc = base_region3_walk(table, addr, next, alloc);
if (rc)
return rc;
@@ -478,13 +586,13 @@ static int base_region2_walk(unsigned long origin, unsigned long addr,
return 0;
}
-static int base_region1_walk(unsigned long origin, unsigned long addr,
+static int base_region1_walk(unsigned long *origin, unsigned long addr,
unsigned long end, int alloc)
{
- unsigned long *rfte, next, table;
+ unsigned long *rfte, next, *table;
int rc;
- rfte = (unsigned long *) origin;
+ rfte = origin;
rfte += (addr & _REGION1_INDEX) >> _REGION1_SHIFT;
do {
next = base_region1_addr_end(addr, end);
@@ -494,9 +602,9 @@ static int base_region1_walk(unsigned long origin, unsigned long addr,
table = base_crst_alloc(_REGION2_ENTRY_EMPTY);
if (!table)
return -ENOMEM;
- *rfte = table | _REGION1_ENTRY;
+ *rfte = __pa(table) | _REGION1_ENTRY;
}
- table = *rfte & _REGION_ENTRY_ORIGIN;
+ table = __va(*rfte & _REGION_ENTRY_ORIGIN);
rc = base_region2_walk(table, addr, next, alloc);
if (rc)
return rc;
@@ -515,7 +623,7 @@ static int base_region1_walk(unsigned long origin, unsigned long addr,
*/
void base_asce_free(unsigned long asce)
{
- unsigned long table = asce & _ASCE_ORIGIN;
+ unsigned long *table = __va(asce & _ASCE_ORIGIN);
if (!asce)
return;
@@ -567,7 +675,7 @@ static int base_pgt_cache_init(void)
*/
unsigned long base_asce_alloc(unsigned long addr, unsigned long num_pages)
{
- unsigned long asce, table, end;
+ unsigned long asce, *table, end;
int rc;
if (base_pgt_cache_init())
@@ -578,25 +686,25 @@ unsigned long base_asce_alloc(unsigned long addr, unsigned long num_pages)
if (!table)
return 0;
rc = base_segment_walk(table, addr, end, 1);
- asce = table | _ASCE_TYPE_SEGMENT | _ASCE_TABLE_LENGTH;
+ asce = __pa(table) | _ASCE_TYPE_SEGMENT | _ASCE_TABLE_LENGTH;
} else if (end <= _REGION2_SIZE) {
table = base_crst_alloc(_REGION3_ENTRY_EMPTY);
if (!table)
return 0;
rc = base_region3_walk(table, addr, end, 1);
- asce = table | _ASCE_TYPE_REGION3 | _ASCE_TABLE_LENGTH;
+ asce = __pa(table) | _ASCE_TYPE_REGION3 | _ASCE_TABLE_LENGTH;
} else if (end <= _REGION1_SIZE) {
table = base_crst_alloc(_REGION2_ENTRY_EMPTY);
if (!table)
return 0;
rc = base_region2_walk(table, addr, end, 1);
- asce = table | _ASCE_TYPE_REGION2 | _ASCE_TABLE_LENGTH;
+ asce = __pa(table) | _ASCE_TYPE_REGION2 | _ASCE_TABLE_LENGTH;
} else {
table = base_crst_alloc(_REGION1_ENTRY_EMPTY);
if (!table)
return 0;
rc = base_region1_walk(table, addr, end, 1);
- asce = table | _ASCE_TYPE_REGION1 | _ASCE_TABLE_LENGTH;
+ asce = __pa(table) | _ASCE_TYPE_REGION1 | _ASCE_TABLE_LENGTH;
}
if (rc) {
base_asce_free(asce);
diff --git a/arch/s390/net/bpf_jit_comp.c b/arch/s390/net/bpf_jit_comp.c
index 233cc9b..9ff2bd8 100644
--- a/arch/s390/net/bpf_jit_comp.c
+++ b/arch/s390/net/bpf_jit_comp.c
@@ -1369,7 +1369,7 @@ static noinline int bpf_jit_insn(struct bpf_jit *jit, struct bpf_prog *fp,
jit->prg);
/*
- * if (tail_call_cnt++ > MAX_TAIL_CALL_CNT)
+ * if (tail_call_cnt++ >= MAX_TAIL_CALL_CNT)
* goto out;
*/
@@ -1381,9 +1381,9 @@ static noinline int bpf_jit_insn(struct bpf_jit *jit, struct bpf_prog *fp,
EMIT4_IMM(0xa7080000, REG_W0, 1);
/* laal %w1,%w0,off(%r15) */
EMIT6_DISP_LH(0xeb000000, 0x00fa, REG_W1, REG_W0, REG_15, off);
- /* clij %w1,MAX_TAIL_CALL_CNT,0x2,out */
+ /* clij %w1,MAX_TAIL_CALL_CNT-1,0x2,out */
patch_2_clij = jit->prg;
- EMIT6_PCREL_RIEC(0xec000000, 0x007f, REG_W1, MAX_TAIL_CALL_CNT,
+ EMIT6_PCREL_RIEC(0xec000000, 0x007f, REG_W1, MAX_TAIL_CALL_CNT - 1,
2, jit->prg);
/*
diff --git a/arch/s390/pci/pci.c b/arch/s390/pci/pci.c
index 2f9b78f..792f8e0 100644
--- a/arch/s390/pci/pci.c
+++ b/arch/s390/pci/pci.c
@@ -771,7 +771,7 @@ int zpci_hot_reset_device(struct zpci_dev *zdev)
if (zdev->dma_table)
rc = zpci_register_ioat(zdev, 0, zdev->start_dma, zdev->end_dma,
- (u64)zdev->dma_table);
+ virt_to_phys(zdev->dma_table));
else
rc = zpci_dma_init_device(zdev);
if (rc) {
diff --git a/arch/s390/pci/pci_dma.c b/arch/s390/pci/pci_dma.c
index 1f4540d6..f46833a 100644
--- a/arch/s390/pci/pci_dma.c
+++ b/arch/s390/pci/pci_dma.c
@@ -74,7 +74,7 @@ static unsigned long *dma_get_seg_table_origin(unsigned long *entry)
if (!sto)
return NULL;
- set_rt_sto(entry, sto);
+ set_rt_sto(entry, virt_to_phys(sto));
validate_rt_entry(entry);
entry_clr_protected(entry);
}
@@ -91,7 +91,7 @@ static unsigned long *dma_get_page_table_origin(unsigned long *entry)
pto = dma_alloc_page_table();
if (!pto)
return NULL;
- set_st_pto(entry, pto);
+ set_st_pto(entry, virt_to_phys(pto));
validate_st_entry(entry);
entry_clr_protected(entry);
}
@@ -117,7 +117,7 @@ unsigned long *dma_walk_cpu_trans(unsigned long *rto, dma_addr_t dma_addr)
return &pto[px];
}
-void dma_update_cpu_trans(unsigned long *entry, void *page_addr, int flags)
+void dma_update_cpu_trans(unsigned long *entry, phys_addr_t page_addr, int flags)
{
if (flags & ZPCI_PTE_INVALID) {
invalidate_pt_entry(entry);
@@ -132,11 +132,11 @@ void dma_update_cpu_trans(unsigned long *entry, void *page_addr, int flags)
entry_clr_protected(entry);
}
-static int __dma_update_trans(struct zpci_dev *zdev, unsigned long pa,
+static int __dma_update_trans(struct zpci_dev *zdev, phys_addr_t pa,
dma_addr_t dma_addr, size_t size, int flags)
{
unsigned int nr_pages = PAGE_ALIGN(size) >> PAGE_SHIFT;
- u8 *page_addr = (u8 *) (pa & PAGE_MASK);
+ phys_addr_t page_addr = (pa & PAGE_MASK);
unsigned long irq_flags;
unsigned long *entry;
int i, rc = 0;
@@ -217,7 +217,7 @@ static int __dma_purge_tlb(struct zpci_dev *zdev, dma_addr_t dma_addr,
return ret;
}
-static int dma_update_trans(struct zpci_dev *zdev, unsigned long pa,
+static int dma_update_trans(struct zpci_dev *zdev, phys_addr_t pa,
dma_addr_t dma_addr, size_t size, int flags)
{
int rc;
@@ -400,7 +400,7 @@ static void *s390_dma_alloc(struct device *dev, size_t size,
{
struct zpci_dev *zdev = to_zpci(to_pci_dev(dev));
struct page *page;
- unsigned long pa;
+ phys_addr_t pa;
dma_addr_t map;
size = PAGE_ALIGN(size);
@@ -411,18 +411,18 @@ static void *s390_dma_alloc(struct device *dev, size_t size,
pa = page_to_phys(page);
map = s390_dma_map_pages(dev, page, 0, size, DMA_BIDIRECTIONAL, 0);
if (dma_mapping_error(dev, map)) {
- free_pages(pa, get_order(size));
+ __free_pages(page, get_order(size));
return NULL;
}
atomic64_add(size / PAGE_SIZE, &zdev->allocated_pages);
if (dma_handle)
*dma_handle = map;
- return (void *) pa;
+ return phys_to_virt(pa);
}
static void s390_dma_free(struct device *dev, size_t size,
- void *pa, dma_addr_t dma_handle,
+ void *vaddr, dma_addr_t dma_handle,
unsigned long attrs)
{
struct zpci_dev *zdev = to_zpci(to_pci_dev(dev));
@@ -430,7 +430,7 @@ static void s390_dma_free(struct device *dev, size_t size,
size = PAGE_ALIGN(size);
atomic64_sub(size / PAGE_SIZE, &zdev->allocated_pages);
s390_dma_unmap_pages(dev, dma_handle, size, DMA_BIDIRECTIONAL, 0);
- free_pages((unsigned long) pa, get_order(size));
+ free_pages((unsigned long)vaddr, get_order(size));
}
/* Map a segment into a contiguous dma address area */
@@ -443,7 +443,7 @@ static int __s390_dma_map_sg(struct device *dev, struct scatterlist *sg,
dma_addr_t dma_addr_base, dma_addr;
int flags = ZPCI_PTE_VALID;
struct scatterlist *s;
- unsigned long pa = 0;
+ phys_addr_t pa = 0;
int ret;
dma_addr_base = dma_alloc_address(dev, nr_pages);
@@ -598,7 +598,7 @@ int zpci_dma_init_device(struct zpci_dev *zdev)
}
if (zpci_register_ioat(zdev, 0, zdev->start_dma, zdev->end_dma,
- (u64)zdev->dma_table)) {
+ virt_to_phys(zdev->dma_table))) {
rc = -EIO;
goto free_bitmap;
}
diff --git a/arch/s390/pci/pci_insn.c b/arch/s390/pci/pci_insn.c
index 28d863a..4dd58b1 100644
--- a/arch/s390/pci/pci_insn.c
+++ b/arch/s390/pci/pci_insn.c
@@ -365,10 +365,7 @@ EXPORT_SYMBOL_GPL(zpci_write_block);
static inline void __pciwb_mio(void)
{
- unsigned long unused = 0;
-
- asm volatile (".insn rre,0xb9d50000,%[op],%[op]\n"
- : [op] "+d" (unused));
+ asm volatile (".insn rre,0xb9d50000,0,0\n");
}
void zpci_barrier(void)
diff --git a/arch/s390/pci/pci_irq.c b/arch/s390/pci/pci_irq.c
index 954bb7a..aefd306 100644
--- a/arch/s390/pci/pci_irq.c
+++ b/arch/s390/pci/pci_irq.c
@@ -45,9 +45,9 @@ static int zpci_set_airq(struct zpci_dev *zdev)
fib.fmt0.isc = PCI_ISC;
fib.fmt0.sum = 1; /* enable summary notifications */
fib.fmt0.noi = airq_iv_end(zdev->aibv);
- fib.fmt0.aibv = (unsigned long) zdev->aibv->vector;
+ fib.fmt0.aibv = virt_to_phys(zdev->aibv->vector);
fib.fmt0.aibvo = 0; /* each zdev has its own interrupt vector */
- fib.fmt0.aisb = (unsigned long) zpci_sbv->vector + (zdev->aisb/64)*8;
+ fib.fmt0.aisb = virt_to_phys(zpci_sbv->vector) + (zdev->aisb / 64) * 8;
fib.fmt0.aisbo = zdev->aisb & 63;
return zpci_mod_fc(req, &fib, &status) ? -EIO : 0;
@@ -422,7 +422,7 @@ static int __init zpci_directed_irq_init(void)
iib.diib.isc = PCI_ISC;
iib.diib.nr_cpus = num_possible_cpus();
- iib.diib.disb_addr = (u64) zpci_sbv->vector;
+ iib.diib.disb_addr = virt_to_phys(zpci_sbv->vector);
__zpci_set_irq_ctrl(SIC_IRQ_MODE_DIRECT, 0, &iib);
zpci_ibv = kcalloc(num_possible_cpus(), sizeof(*zpci_ibv),
diff --git a/arch/s390/tools/opcodes.txt b/arch/s390/tools/opcodes.txt
index 6db9820..5f008e79 100644
--- a/arch/s390/tools/opcodes.txt
+++ b/arch/s390/tools/opcodes.txt
@@ -276,6 +276,7 @@
b286 qsi S_RD
b287 lsctl S_RD
b28e qctri S_RD
+b28f qpaci S_RD
b299 srnm S_RD
b29c stfpc S_RD
b29d lfpc S_RD
@@ -1098,7 +1099,7 @@
eb62 mric RSY_RDRU
eb6a asi SIY_IRD
eb6e alsi SIY_IRD
-eb71 lpswey SIY_URD
+eb71 lpswey SIY_RD
eb7a agsi SIY_IRD
eb7e algsi SIY_IRD
eb80 icmh RSY_RURD
diff --git a/arch/sh/configs/titan_defconfig b/arch/sh/configs/titan_defconfig
index ba887f1..cd5c589 100644
--- a/arch/sh/configs/titan_defconfig
+++ b/arch/sh/configs/titan_defconfig
@@ -242,7 +242,6 @@
CONFIG_NFSD_V3=y
CONFIG_SMB_FS=m
CONFIG_CIFS=m
-CONFIG_CIFS_WEAK_PW_HASH=y
CONFIG_PARTITION_ADVANCED=y
CONFIG_NLS_CODEPAGE_437=m
CONFIG_NLS_ASCII=m
diff --git a/arch/sparc/net/bpf_jit_comp_64.c b/arch/sparc/net/bpf_jit_comp_64.c
index 9a2f20c..b1e3878 100644
--- a/arch/sparc/net/bpf_jit_comp_64.c
+++ b/arch/sparc/net/bpf_jit_comp_64.c
@@ -227,7 +227,7 @@ static const int bpf2sparc[] = {
[BPF_REG_AX] = G7,
- /* temporary register for internal BPF JIT */
+ /* temporary register for BPF JIT */
[TMP_REG_1] = G1,
[TMP_REG_2] = G2,
[TMP_REG_3] = G3,
@@ -867,7 +867,7 @@ static void emit_tail_call(struct jit_ctx *ctx)
emit(LD32 | IMMED | RS1(SP) | S13(off) | RD(tmp), ctx);
emit_cmpi(tmp, MAX_TAIL_CALL_CNT, ctx);
#define OFFSET2 13
- emit_branch(BGU, ctx->idx, ctx->idx + OFFSET2, ctx);
+ emit_branch(BGEU, ctx->idx, ctx->idx + OFFSET2, ctx);
emit_nop(ctx);
emit_alu_K(ADD, tmp, 1, ctx);
diff --git a/arch/um/drivers/vector_kern.c b/arch/um/drivers/vector_kern.c
index cde6db1..4fc1a5d 100644
--- a/arch/um/drivers/vector_kern.c
+++ b/arch/um/drivers/vector_kern.c
@@ -1441,7 +1441,9 @@ static int vector_net_load_bpf_flash(struct net_device *dev,
}
static void vector_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct vector_private *vp = netdev_priv(netdev);
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 5c2ccb8..8910b09 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -269,6 +269,7 @@
select HAVE_ARCH_KCSAN if X86_64
select X86_FEATURE_NAMES if PROC_FS
select PROC_PID_ARCH_STATUS if PROC_FS
+ select HAVE_ARCH_NODE_DEV_GROUP if X86_SGX
imply IMA_SECURE_AND_OR_TRUSTED_BOOT if EFI
config INSTRUCTION_DECODER
@@ -1523,16 +1524,20 @@
helps to determine the effectiveness of preserving large and huge
page mappings when mapping protections are changed.
+config X86_MEM_ENCRYPT
+ select ARCH_HAS_FORCE_DMA_UNENCRYPTED
+ select DYNAMIC_PHYSICAL_MASK
+ select ARCH_HAS_RESTRICTED_VIRTIO_MEMORY_ACCESS
+ def_bool n
+
config AMD_MEM_ENCRYPT
bool "AMD Secure Memory Encryption (SME) support"
depends on X86_64 && CPU_SUP_AMD
select DMA_COHERENT_POOL
- select DYNAMIC_PHYSICAL_MASK
select ARCH_USE_MEMREMAP_PROT
- select ARCH_HAS_FORCE_DMA_UNENCRYPTED
select INSTRUCTION_DECODER
- select ARCH_HAS_RESTRICTED_VIRTIO_MEMORY_ACCESS
select ARCH_HAS_CC_PLATFORM
+ select X86_MEM_ENCRYPT
help
Say yes to enable support for the encryption of system memory.
This requires an AMD processor that supports Secure Memory
@@ -1917,6 +1922,7 @@
select SRCU
select MMU_NOTIFIER
select NUMA_KEEP_MEMINFO if NUMA
+ select XARRAY_MULTI
help
Intel(R) Software Guard eXtensions (SGX) is a set of CPU instructions
that can be used by applications to set aside private regions of code
diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/Makefile
index 431bf7f..e118136 100644
--- a/arch/x86/boot/compressed/Makefile
+++ b/arch/x86/boot/compressed/Makefile
@@ -28,7 +28,11 @@
targets := vmlinux vmlinux.bin vmlinux.bin.gz vmlinux.bin.bz2 vmlinux.bin.lzma \
vmlinux.bin.xz vmlinux.bin.lzo vmlinux.bin.lz4 vmlinux.bin.zst
-KBUILD_CFLAGS := -m$(BITS) -O2
+# CLANG_FLAGS must come before any cc-disable-warning or cc-option calls in
+# case of cross compiling, as it has the '--target=' flag, which is needed to
+# avoid errors with '-march=i386', and future flags may depend on the target to
+# be valid.
+KBUILD_CFLAGS := -m$(BITS) -O2 $(CLANG_FLAGS)
KBUILD_CFLAGS += -fno-strict-aliasing -fPIE
KBUILD_CFLAGS += -Wundef
KBUILD_CFLAGS += -DDISABLE_BRANCH_PROFILING
@@ -47,7 +51,6 @@
# Disable relocation relaxation in case the link is not PIE.
KBUILD_CFLAGS += $(call as-option,-Wa$(comma)-mrelax-relocations=no)
KBUILD_CFLAGS += -include $(srctree)/include/linux/hidden.h
-KBUILD_CFLAGS += $(CLANG_FLAGS)
# sev.c indirectly inludes inat-table.h which is generated during
# compilation and stored in $(objtree). Add the directory to the includes so
diff --git a/arch/x86/boot/compressed/sev.c b/arch/x86/boot/compressed/sev.c
index 670e998..28bcf04 100644
--- a/arch/x86/boot/compressed/sev.c
+++ b/arch/x86/boot/compressed/sev.c
@@ -122,7 +122,7 @@ static enum es_result vc_read_mem(struct es_em_ctxt *ctxt,
static bool early_setup_sev_es(void)
{
if (!sev_es_negotiate_protocol())
- sev_es_terminate(GHCB_SEV_ES_REASON_PROTOCOL_UNSUPPORTED);
+ sev_es_terminate(GHCB_SEV_ES_PROT_UNSUPPORTED);
if (set_page_decrypted((unsigned long)&boot_ghcb_page))
return false;
@@ -175,7 +175,7 @@ void do_boot_stage2_vc(struct pt_regs *regs, unsigned long exit_code)
enum es_result result;
if (!boot_ghcb && !early_setup_sev_es())
- sev_es_terminate(GHCB_SEV_ES_REASON_GENERAL_REQUEST);
+ sev_es_terminate(GHCB_SEV_ES_GEN_REQ);
vc_ghcb_invalidate(boot_ghcb);
result = vc_init_em_ctxt(&ctxt, regs, exit_code);
@@ -202,5 +202,5 @@ void do_boot_stage2_vc(struct pt_regs *regs, unsigned long exit_code)
if (result == ES_OK)
vc_finish_insn(&ctxt);
else if (result != ES_RETRY)
- sev_es_terminate(GHCB_SEV_ES_REASON_GENERAL_REQUEST);
+ sev_es_terminate(GHCB_SEV_ES_GEN_REQ);
}
diff --git a/arch/x86/boot/string.h b/arch/x86/boot/string.h
index a232da4..e5d2c6b 100644
--- a/arch/x86/boot/string.h
+++ b/arch/x86/boot/string.h
@@ -8,8 +8,10 @@
#undef memcmp
void *memcpy(void *dst, const void *src, size_t len);
+void *memmove(void *dst, const void *src, size_t len);
void *memset(void *dst, int c, size_t len);
int memcmp(const void *s1, const void *s2, size_t len);
+int bcmp(const void *s1, const void *s2, size_t len);
/* Access builtin version by default. */
#define memcpy(d,s,l) __builtin_memcpy(d,s,l)
@@ -25,6 +27,7 @@ extern size_t strnlen(const char *s, size_t maxlen);
extern unsigned int atou(const char *s);
extern unsigned long long simple_strtoull(const char *cp, char **endp,
unsigned int base);
+long simple_strtol(const char *cp, char **endp, unsigned int base);
int kstrtoull(const char *s, unsigned int base, unsigned long long *res);
int boot_kstrtoul(const char *s, unsigned int base, unsigned long *res);
diff --git a/arch/x86/crypto/Makefile b/arch/x86/crypto/Makefile
index f307c93..c3af959 100644
--- a/arch/x86/crypto/Makefile
+++ b/arch/x86/crypto/Makefile
@@ -62,7 +62,9 @@
sha512-ssse3-y := sha512-ssse3-asm.o sha512-avx-asm.o sha512-avx2-asm.o sha512_ssse3_glue.o
obj-$(CONFIG_CRYPTO_BLAKE2S_X86) += blake2s-x86_64.o
-blake2s-x86_64-y := blake2s-core.o blake2s-glue.o
+blake2s-x86_64-y := blake2s-shash.o
+obj-$(if $(CONFIG_CRYPTO_BLAKE2S_X86),y) += libblake2s-x86_64.o
+libblake2s-x86_64-y := blake2s-core.o blake2s-glue.o
obj-$(CONFIG_CRYPTO_GHASH_CLMUL_NI_INTEL) += ghash-clmulni-intel.o
ghash-clmulni-intel-y := ghash-clmulni-intel_asm.o ghash-clmulni-intel_glue.o
diff --git a/arch/x86/crypto/blake2s-glue.c b/arch/x86/crypto/blake2s-glue.c
index a40365a..69853c1 100644
--- a/arch/x86/crypto/blake2s-glue.c
+++ b/arch/x86/crypto/blake2s-glue.c
@@ -5,7 +5,6 @@
#include <crypto/internal/blake2s.h>
#include <crypto/internal/simd.h>
-#include <crypto/internal/hash.h>
#include <linux/types.h>
#include <linux/jump_label.h>
@@ -28,9 +27,8 @@ asmlinkage void blake2s_compress_avx512(struct blake2s_state *state,
static __ro_after_init DEFINE_STATIC_KEY_FALSE(blake2s_use_ssse3);
static __ro_after_init DEFINE_STATIC_KEY_FALSE(blake2s_use_avx512);
-void blake2s_compress_arch(struct blake2s_state *state,
- const u8 *block, size_t nblocks,
- const u32 inc)
+void blake2s_compress(struct blake2s_state *state, const u8 *block,
+ size_t nblocks, const u32 inc)
{
/* SIMD disables preemption, so relax after processing each page. */
BUILD_BUG_ON(SZ_4K / BLAKE2S_BLOCK_SIZE < 8);
@@ -56,49 +54,12 @@ void blake2s_compress_arch(struct blake2s_state *state,
block += blocks * BLAKE2S_BLOCK_SIZE;
} while (nblocks);
}
-EXPORT_SYMBOL(blake2s_compress_arch);
-
-static int crypto_blake2s_update_x86(struct shash_desc *desc,
- const u8 *in, unsigned int inlen)
-{
- return crypto_blake2s_update(desc, in, inlen, blake2s_compress_arch);
-}
-
-static int crypto_blake2s_final_x86(struct shash_desc *desc, u8 *out)
-{
- return crypto_blake2s_final(desc, out, blake2s_compress_arch);
-}
-
-#define BLAKE2S_ALG(name, driver_name, digest_size) \
- { \
- .base.cra_name = name, \
- .base.cra_driver_name = driver_name, \
- .base.cra_priority = 200, \
- .base.cra_flags = CRYPTO_ALG_OPTIONAL_KEY, \
- .base.cra_blocksize = BLAKE2S_BLOCK_SIZE, \
- .base.cra_ctxsize = sizeof(struct blake2s_tfm_ctx), \
- .base.cra_module = THIS_MODULE, \
- .digestsize = digest_size, \
- .setkey = crypto_blake2s_setkey, \
- .init = crypto_blake2s_init, \
- .update = crypto_blake2s_update_x86, \
- .final = crypto_blake2s_final_x86, \
- .descsize = sizeof(struct blake2s_state), \
- }
-
-static struct shash_alg blake2s_algs[] = {
- BLAKE2S_ALG("blake2s-128", "blake2s-128-x86", BLAKE2S_128_HASH_SIZE),
- BLAKE2S_ALG("blake2s-160", "blake2s-160-x86", BLAKE2S_160_HASH_SIZE),
- BLAKE2S_ALG("blake2s-224", "blake2s-224-x86", BLAKE2S_224_HASH_SIZE),
- BLAKE2S_ALG("blake2s-256", "blake2s-256-x86", BLAKE2S_256_HASH_SIZE),
-};
+EXPORT_SYMBOL(blake2s_compress);
static int __init blake2s_mod_init(void)
{
- if (!boot_cpu_has(X86_FEATURE_SSSE3))
- return 0;
-
- static_branch_enable(&blake2s_use_ssse3);
+ if (boot_cpu_has(X86_FEATURE_SSSE3))
+ static_branch_enable(&blake2s_use_ssse3);
if (IS_ENABLED(CONFIG_AS_AVX512) &&
boot_cpu_has(X86_FEATURE_AVX) &&
@@ -109,26 +70,9 @@ static int __init blake2s_mod_init(void)
XFEATURE_MASK_AVX512, NULL))
static_branch_enable(&blake2s_use_avx512);
- return IS_REACHABLE(CONFIG_CRYPTO_HASH) ?
- crypto_register_shashes(blake2s_algs,
- ARRAY_SIZE(blake2s_algs)) : 0;
-}
-
-static void __exit blake2s_mod_exit(void)
-{
- if (IS_REACHABLE(CONFIG_CRYPTO_HASH) && boot_cpu_has(X86_FEATURE_SSSE3))
- crypto_unregister_shashes(blake2s_algs, ARRAY_SIZE(blake2s_algs));
+ return 0;
}
module_init(blake2s_mod_init);
-module_exit(blake2s_mod_exit);
-MODULE_ALIAS_CRYPTO("blake2s-128");
-MODULE_ALIAS_CRYPTO("blake2s-128-x86");
-MODULE_ALIAS_CRYPTO("blake2s-160");
-MODULE_ALIAS_CRYPTO("blake2s-160-x86");
-MODULE_ALIAS_CRYPTO("blake2s-224");
-MODULE_ALIAS_CRYPTO("blake2s-224-x86");
-MODULE_ALIAS_CRYPTO("blake2s-256");
-MODULE_ALIAS_CRYPTO("blake2s-256-x86");
MODULE_LICENSE("GPL v2");
diff --git a/arch/x86/crypto/blake2s-shash.c b/arch/x86/crypto/blake2s-shash.c
new file mode 100644
index 0000000..f9e2fec
--- /dev/null
+++ b/arch/x86/crypto/blake2s-shash.c
@@ -0,0 +1,77 @@
+// SPDX-License-Identifier: GPL-2.0 OR MIT
+/*
+ * Copyright (C) 2015-2019 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Reserved.
+ */
+
+#include <crypto/internal/blake2s.h>
+#include <crypto/internal/simd.h>
+#include <crypto/internal/hash.h>
+
+#include <linux/types.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/sizes.h>
+
+#include <asm/cpufeature.h>
+#include <asm/processor.h>
+
+static int crypto_blake2s_update_x86(struct shash_desc *desc,
+ const u8 *in, unsigned int inlen)
+{
+ return crypto_blake2s_update(desc, in, inlen, blake2s_compress);
+}
+
+static int crypto_blake2s_final_x86(struct shash_desc *desc, u8 *out)
+{
+ return crypto_blake2s_final(desc, out, blake2s_compress);
+}
+
+#define BLAKE2S_ALG(name, driver_name, digest_size) \
+ { \
+ .base.cra_name = name, \
+ .base.cra_driver_name = driver_name, \
+ .base.cra_priority = 200, \
+ .base.cra_flags = CRYPTO_ALG_OPTIONAL_KEY, \
+ .base.cra_blocksize = BLAKE2S_BLOCK_SIZE, \
+ .base.cra_ctxsize = sizeof(struct blake2s_tfm_ctx), \
+ .base.cra_module = THIS_MODULE, \
+ .digestsize = digest_size, \
+ .setkey = crypto_blake2s_setkey, \
+ .init = crypto_blake2s_init, \
+ .update = crypto_blake2s_update_x86, \
+ .final = crypto_blake2s_final_x86, \
+ .descsize = sizeof(struct blake2s_state), \
+ }
+
+static struct shash_alg blake2s_algs[] = {
+ BLAKE2S_ALG("blake2s-128", "blake2s-128-x86", BLAKE2S_128_HASH_SIZE),
+ BLAKE2S_ALG("blake2s-160", "blake2s-160-x86", BLAKE2S_160_HASH_SIZE),
+ BLAKE2S_ALG("blake2s-224", "blake2s-224-x86", BLAKE2S_224_HASH_SIZE),
+ BLAKE2S_ALG("blake2s-256", "blake2s-256-x86", BLAKE2S_256_HASH_SIZE),
+};
+
+static int __init blake2s_mod_init(void)
+{
+ if (IS_REACHABLE(CONFIG_CRYPTO_HASH) && boot_cpu_has(X86_FEATURE_SSSE3))
+ return crypto_register_shashes(blake2s_algs, ARRAY_SIZE(blake2s_algs));
+ return 0;
+}
+
+static void __exit blake2s_mod_exit(void)
+{
+ if (IS_REACHABLE(CONFIG_CRYPTO_HASH) && boot_cpu_has(X86_FEATURE_SSSE3))
+ crypto_unregister_shashes(blake2s_algs, ARRAY_SIZE(blake2s_algs));
+}
+
+module_init(blake2s_mod_init);
+module_exit(blake2s_mod_exit);
+
+MODULE_ALIAS_CRYPTO("blake2s-128");
+MODULE_ALIAS_CRYPTO("blake2s-128-x86");
+MODULE_ALIAS_CRYPTO("blake2s-160");
+MODULE_ALIAS_CRYPTO("blake2s-160-x86");
+MODULE_ALIAS_CRYPTO("blake2s-224");
+MODULE_ALIAS_CRYPTO("blake2s-224-x86");
+MODULE_ALIAS_CRYPTO("blake2s-256");
+MODULE_ALIAS_CRYPTO("blake2s-256-x86");
+MODULE_LICENSE("GPL v2");
diff --git a/arch/x86/entry/vdso/Makefile b/arch/x86/entry/vdso/Makefile
index a2dddcc..693f8b9 100644
--- a/arch/x86/entry/vdso/Makefile
+++ b/arch/x86/entry/vdso/Makefile
@@ -172,7 +172,7 @@
# The DSO images are built using a special linker script.
#
quiet_cmd_vdso = VDSO $@
- cmd_vdso = $(LD) -nostdlib -o $@ \
+ cmd_vdso = $(LD) -o $@ \
$(VDSO_LDFLAGS) $(VDSO_LDFLAGS_$(filter %.lds,$(^F))) \
-T $(filter %.lds,$^) $(filter %.o,$^) && \
sh $(srctree)/$(src)/checkundef.sh '$(NM)' '$@'
diff --git a/arch/x86/events/amd/iommu.c b/arch/x86/events/amd/iommu.c
index 913745f..b15f7b9 100644
--- a/arch/x86/events/amd/iommu.c
+++ b/arch/x86/events/amd/iommu.c
@@ -161,7 +161,7 @@ static int get_next_avail_iommu_bnk_cntr(struct perf_event *event)
raw_spin_lock_irqsave(&piommu->lock, flags);
- for (bank = 0, shift = 0; bank < max_banks; bank++) {
+ for (bank = 0; bank < max_banks; bank++) {
for (cntr = 0; cntr < max_cntrs; cntr++) {
shift = bank + (bank*3) + cntr;
if (piommu->cntr_assign_mask & BIT_ULL(shift)) {
diff --git a/arch/x86/events/core.c b/arch/x86/events/core.c
index 38b2c779..68dea7c 100644
--- a/arch/x86/events/core.c
+++ b/arch/x86/events/core.c
@@ -2476,7 +2476,7 @@ static int x86_pmu_event_init(struct perf_event *event)
if (READ_ONCE(x86_pmu.attr_rdpmc) &&
!(event->hw.flags & PERF_X86_EVENT_LARGE_PEBS))
- event->hw.flags |= PERF_X86_EVENT_RDPMC_ALLOWED;
+ event->hw.flags |= PERF_EVENT_FLAG_USER_READ_CNT;
return err;
}
@@ -2510,7 +2510,7 @@ void perf_clear_dirty_counters(void)
static void x86_pmu_event_mapped(struct perf_event *event, struct mm_struct *mm)
{
- if (!(event->hw.flags & PERF_X86_EVENT_RDPMC_ALLOWED))
+ if (!(event->hw.flags & PERF_EVENT_FLAG_USER_READ_CNT))
return;
/*
@@ -2531,7 +2531,7 @@ static void x86_pmu_event_mapped(struct perf_event *event, struct mm_struct *mm)
static void x86_pmu_event_unmapped(struct perf_event *event, struct mm_struct *mm)
{
- if (!(event->hw.flags & PERF_X86_EVENT_RDPMC_ALLOWED))
+ if (!(event->hw.flags & PERF_EVENT_FLAG_USER_READ_CNT))
return;
if (atomic_dec_and_test(&mm->context.perf_rdpmc_allowed))
@@ -2542,7 +2542,7 @@ static int x86_pmu_event_idx(struct perf_event *event)
{
struct hw_perf_event *hwc = &event->hw;
- if (!(hwc->flags & PERF_X86_EVENT_RDPMC_ALLOWED))
+ if (!(hwc->flags & PERF_EVENT_FLAG_USER_READ_CNT))
return 0;
if (is_metric_idx(hwc->idx))
@@ -2725,7 +2725,7 @@ void arch_perf_update_userpage(struct perf_event *event,
userpg->cap_user_time = 0;
userpg->cap_user_time_zero = 0;
userpg->cap_user_rdpmc =
- !!(event->hw.flags & PERF_X86_EVENT_RDPMC_ALLOWED);
+ !!(event->hw.flags & PERF_EVENT_FLAG_USER_READ_CNT);
userpg->pmc_width = x86_pmu.cntval_bits;
if (!using_native_sched_clock() || !sched_clock_stable())
diff --git a/arch/x86/events/perf_event.h b/arch/x86/events/perf_event.h
index 5480db2..9d376e5 100644
--- a/arch/x86/events/perf_event.h
+++ b/arch/x86/events/perf_event.h
@@ -74,7 +74,7 @@ static inline bool constraint_match(struct event_constraint *c, u64 ecode)
#define PERF_X86_EVENT_PEBS_NA_HSW 0x0010 /* haswell style datala, unknown */
#define PERF_X86_EVENT_EXCL 0x0020 /* HT exclusivity on counter */
#define PERF_X86_EVENT_DYNAMIC 0x0040 /* dynamic alloc'd constraint */
-#define PERF_X86_EVENT_RDPMC_ALLOWED 0x0080 /* grant rdpmc permission */
+
#define PERF_X86_EVENT_EXCL_ACCT 0x0100 /* accounted EXCL event */
#define PERF_X86_EVENT_AUTO_RELOAD 0x0200 /* use PEBS auto-reload */
#define PERF_X86_EVENT_LARGE_PEBS 0x0400 /* use large PEBS */
diff --git a/arch/x86/include/asm/amd_nb.h b/arch/x86/include/asm/amd_nb.h
index 455066a..00d1a40 100644
--- a/arch/x86/include/asm/amd_nb.h
+++ b/arch/x86/include/asm/amd_nb.h
@@ -24,7 +24,6 @@ extern int amd_set_subcaches(int, unsigned long);
extern int amd_smn_read(u16 node, u32 address, u32 *value);
extern int amd_smn_write(u16 node, u32 address, u32 value);
-extern int amd_df_indirect_read(u16 node, u8 func, u16 reg, u8 instance_id, u32 *lo);
struct amd_l3_cache {
unsigned indices;
diff --git a/arch/x86/include/asm/efi.h b/arch/x86/include/asm/efi.h
index 4d0b126..63158fd 100644
--- a/arch/x86/include/asm/efi.h
+++ b/arch/x86/include/asm/efi.h
@@ -197,8 +197,6 @@ static inline bool efi_runtime_supported(void)
extern void parse_efi_setup(u64 phys_addr, u32 data_len);
-extern void efifb_setup_from_dmi(struct screen_info *si, const char *opt);
-
extern void efi_thunk_runtime_setup(void);
efi_status_t efi_set_virtual_address_map(unsigned long memory_map_size,
unsigned long descriptor_size,
diff --git a/arch/x86/include/asm/fpu/signal.h b/arch/x86/include/asm/fpu/signal.h
index 22b0273..e1c9df9 100644
--- a/arch/x86/include/asm/fpu/signal.h
+++ b/arch/x86/include/asm/fpu/signal.h
@@ -41,7 +41,4 @@ extern void fpu__clear_user_states(struct fpu *fpu);
extern bool fpu__restore_sig(void __user *buf, int ia32_frame);
extern void restore_fpregs_from_fpstate(struct fpstate *fpstate, u64 mask);
-
-extern bool copy_fpstate_to_sigframe(void __user *buf, void __user *fp, int size);
-
#endif /* _ASM_X86_FPU_SIGNAL_H */
diff --git a/arch/x86/include/asm/insn-eval.h b/arch/x86/include/asm/insn-eval.h
index 4ec3613..43785ee3 100644
--- a/arch/x86/include/asm/insn-eval.h
+++ b/arch/x86/include/asm/insn-eval.h
@@ -19,6 +19,7 @@ bool insn_has_rep_prefix(struct insn *insn);
void __user *insn_get_addr_ref(struct insn *insn, struct pt_regs *regs);
int insn_get_modrm_rm_off(struct insn *insn, struct pt_regs *regs);
int insn_get_modrm_reg_off(struct insn *insn, struct pt_regs *regs);
+unsigned long *insn_get_modrm_reg_ptr(struct insn *insn, struct pt_regs *regs);
unsigned long insn_get_seg_base(struct pt_regs *regs, int seg_reg_idx);
int insn_get_code_seg_params(struct pt_regs *regs);
int insn_get_effective_ip(struct pt_regs *regs, unsigned long *ip);
@@ -29,4 +30,16 @@ int insn_fetch_from_user_inatomic(struct pt_regs *regs,
bool insn_decode_from_regs(struct insn *insn, struct pt_regs *regs,
unsigned char buf[MAX_INSN_SIZE], int buf_size);
+enum mmio_type {
+ MMIO_DECODE_FAILED,
+ MMIO_WRITE,
+ MMIO_WRITE_IMM,
+ MMIO_READ,
+ MMIO_READ_ZERO_EXTEND,
+ MMIO_READ_SIGN_EXTEND,
+ MMIO_MOVS,
+};
+
+enum mmio_type insn_decode_mmio(struct insn *insn, int *bytes);
+
#endif /* _ASM_X86_INSN_EVAL_H */
diff --git a/arch/x86/include/asm/io.h b/arch/x86/include/asm/io.h
index 5c6a4af0..f6d91ec 100644
--- a/arch/x86/include/asm/io.h
+++ b/arch/x86/include/asm/io.h
@@ -40,6 +40,7 @@
#include <linux/string.h>
#include <linux/compiler.h>
+#include <linux/cc_platform.h>
#include <asm/page.h>
#include <asm/early_ioremap.h>
#include <asm/pgtable_types.h>
@@ -256,21 +257,6 @@ static inline void slow_down_io(void)
#endif
-#ifdef CONFIG_AMD_MEM_ENCRYPT
-#include <linux/jump_label.h>
-
-extern struct static_key_false sev_enable_key;
-static inline bool sev_key_active(void)
-{
- return static_branch_unlikely(&sev_enable_key);
-}
-
-#else /* !CONFIG_AMD_MEM_ENCRYPT */
-
-static inline bool sev_key_active(void) { return false; }
-
-#endif /* CONFIG_AMD_MEM_ENCRYPT */
-
#define BUILDIO(bwl, bw, type) \
static inline void out##bwl(unsigned type value, int port) \
{ \
@@ -301,7 +287,7 @@ static inline unsigned type in##bwl##_p(int port) \
\
static inline void outs##bwl(int port, const void *addr, unsigned long count) \
{ \
- if (sev_key_active()) { \
+ if (cc_platform_has(CC_ATTR_GUEST_UNROLL_STRING_IO)) { \
unsigned type *value = (unsigned type *)addr; \
while (count) { \
out##bwl(*value, port); \
@@ -317,7 +303,7 @@ static inline void outs##bwl(int port, const void *addr, unsigned long count) \
\
static inline void ins##bwl(int port, void *addr, unsigned long count) \
{ \
- if (sev_key_active()) { \
+ if (cc_platform_has(CC_ATTR_GUEST_UNROLL_STRING_IO)) { \
unsigned type *value = (unsigned type *)addr; \
while (count) { \
*value = in##bwl(port); \
diff --git a/arch/x86/include/asm/irqflags.h b/arch/x86/include/asm/irqflags.h
index c5ce984..8776139 100644
--- a/arch/x86/include/asm/irqflags.h
+++ b/arch/x86/include/asm/irqflags.h
@@ -114,8 +114,6 @@ static __always_inline unsigned long arch_local_irq_save(void)
#define SAVE_FLAGS pushfq; popq %rax
#endif
-#define INTERRUPT_RETURN jmp native_iret
-
#endif
#endif /* __ASSEMBLY__ */
@@ -143,8 +141,13 @@ static __always_inline void arch_local_irq_restore(unsigned long flags)
#ifdef CONFIG_X86_64
#ifdef CONFIG_XEN_PV
#define SWAPGS ALTERNATIVE "swapgs", "", X86_FEATURE_XENPV
+#define INTERRUPT_RETURN \
+ ANNOTATE_RETPOLINE_SAFE; \
+ ALTERNATIVE_TERNARY("jmp *paravirt_iret(%rip);", \
+ X86_FEATURE_XENPV, "jmp xen_iret;", "jmp native_iret;")
#else
#define SWAPGS swapgs
+#define INTERRUPT_RETURN jmp native_iret
#endif
#endif
#endif /* !__ASSEMBLY__ */
diff --git a/arch/x86/include/asm/mce.h b/arch/x86/include/asm/mce.h
index 8f6395d..cc73061 100644
--- a/arch/x86/include/asm/mce.h
+++ b/arch/x86/include/asm/mce.h
@@ -313,31 +313,22 @@ enum smca_bank_types {
SMCA_SMU, /* System Management Unit */
SMCA_SMU_V2,
SMCA_MP5, /* Microprocessor 5 Unit */
+ SMCA_MPDMA, /* MPDMA Unit */
SMCA_NBIO, /* Northbridge IO Unit */
SMCA_PCIE, /* PCI Express Unit */
SMCA_PCIE_V2,
SMCA_XGMI_PCS, /* xGMI PCS Unit */
+ SMCA_NBIF, /* NBIF Unit */
+ SMCA_SHUB, /* System HUB Unit */
+ SMCA_SATA, /* SATA Unit */
+ SMCA_USB, /* USB Unit */
+ SMCA_GMI_PCS, /* GMI PCS Unit */
SMCA_XGMI_PHY, /* xGMI PHY Unit */
SMCA_WAFL_PHY, /* WAFL PHY Unit */
+ SMCA_GMI_PHY, /* GMI PHY Unit */
N_SMCA_BANK_TYPES
};
-#define HWID_MCATYPE(hwid, mcatype) (((hwid) << 16) | (mcatype))
-
-struct smca_hwid {
- unsigned int bank_type; /* Use with smca_bank_types for easy indexing. */
- u32 hwid_mcatype; /* (hwid,mcatype) tuple */
- u8 count; /* Number of instances. */
-};
-
-struct smca_bank {
- struct smca_hwid *hwid;
- u32 id; /* Value of MCA_IPID[InstanceId]. */
- u8 sysfs_id; /* Value used for sysfs name. */
-};
-
-extern struct smca_bank smca_banks[MAX_NR_BANKS];
-
extern const char *smca_get_long_name(enum smca_bank_types t);
extern bool amd_mce_is_memory_error(struct mce *m);
@@ -345,16 +336,13 @@ extern int mce_threshold_create_device(unsigned int cpu);
extern int mce_threshold_remove_device(unsigned int cpu);
void mce_amd_feature_init(struct cpuinfo_x86 *c);
-int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr);
-enum smca_bank_types smca_get_bank_type(unsigned int bank);
+enum smca_bank_types smca_get_bank_type(unsigned int cpu, unsigned int bank);
#else
static inline int mce_threshold_create_device(unsigned int cpu) { return 0; };
static inline int mce_threshold_remove_device(unsigned int cpu) { return 0; };
static inline bool amd_mce_is_memory_error(struct mce *m) { return false; };
static inline void mce_amd_feature_init(struct cpuinfo_x86 *c) { }
-static inline int
-umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr) { return -EINVAL; };
#endif
static inline void mce_hygon_feature_init(struct cpuinfo_x86 *c) { return mce_amd_feature_init(c); }
diff --git a/arch/x86/include/asm/mtrr.h b/arch/x86/include/asm/mtrr.h
index 829df26..76d7260 100644
--- a/arch/x86/include/asm/mtrr.h
+++ b/arch/x86/include/asm/mtrr.h
@@ -24,8 +24,8 @@
#define _ASM_X86_MTRR_H
#include <uapi/asm/mtrr.h>
-#include <asm/memtype.h>
+void mtrr_bp_init(void);
/*
* The following functions are for use by other drivers that cannot use
@@ -43,7 +43,6 @@ extern int mtrr_del(int reg, unsigned long base, unsigned long size);
extern int mtrr_del_page(int reg, unsigned long base, unsigned long size);
extern void mtrr_centaur_report_mcr(int mcr, u32 lo, u32 hi);
extern void mtrr_ap_init(void);
-extern void mtrr_bp_init(void);
extern void set_mtrr_aps_delayed_init(void);
extern void mtrr_aps_init(void);
extern void mtrr_bp_restore(void);
@@ -84,11 +83,6 @@ static inline int mtrr_trim_uncached_memory(unsigned long end_pfn)
static inline void mtrr_centaur_report_mcr(int mcr, u32 lo, u32 hi)
{
}
-static inline void mtrr_bp_init(void)
-{
- pat_disable("PAT support disabled because CONFIG_MTRR is disabled in the kernel.");
-}
-
#define mtrr_ap_init() do {} while (0)
#define set_mtrr_aps_delayed_init() do {} while (0)
#define mtrr_aps_init() do {} while (0)
diff --git a/arch/x86/include/asm/page_64.h b/arch/x86/include/asm/page_64.h
index 4bde0dc..e9c8629 100644
--- a/arch/x86/include/asm/page_64.h
+++ b/arch/x86/include/asm/page_64.h
@@ -5,6 +5,7 @@
#include <asm/page_64_types.h>
#ifndef __ASSEMBLY__
+#include <asm/cpufeatures.h>
#include <asm/alternative.h>
/* duplicated to the one in bootmem.h */
diff --git a/arch/x86/include/asm/paravirt.h b/arch/x86/include/asm/paravirt.h
index 21c4a69..27d2762 100644
--- a/arch/x86/include/asm/paravirt.h
+++ b/arch/x86/include/asm/paravirt.h
@@ -752,11 +752,6 @@ extern void default_banner(void);
#define PARA_SITE(ptype, ops) _PVSITE(ptype, ops, .quad, 8)
#define PARA_INDIRECT(addr) *addr(%rip)
-#define INTERRUPT_RETURN \
- ANNOTATE_RETPOLINE_SAFE; \
- ALTERNATIVE_TERNARY("jmp *paravirt_iret(%rip);", \
- X86_FEATURE_XENPV, "jmp xen_iret;", "jmp native_iret;")
-
#ifdef CONFIG_DEBUG_ENTRY
.macro PARA_IRQ_save_fl
PARA_SITE(PARA_PATCH(PV_IRQ_save_fl),
diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h
index 448cd01..a34430b 100644
--- a/arch/x86/include/asm/pgtable.h
+++ b/arch/x86/include/asm/pgtable.h
@@ -22,6 +22,7 @@
#define pgprot_decrypted(prot) __pgprot(__sme_clr(pgprot_val(prot)))
#ifndef __ASSEMBLY__
+#include <linux/spinlock.h>
#include <asm/x86_init.h>
#include <asm/pkru.h>
#include <asm/fpu/api.h>
diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/processor.h
index 355d38c..2c5f12a 100644
--- a/arch/x86/include/asm/processor.h
+++ b/arch/x86/include/asm/processor.h
@@ -855,4 +855,12 @@ enum mds_mitigations {
MDS_MITIGATION_VMWERV,
};
+#ifdef CONFIG_X86_SGX
+int arch_memory_failure(unsigned long pfn, int flags);
+#define arch_memory_failure arch_memory_failure
+
+bool arch_is_platform_page(u64 paddr);
+#define arch_is_platform_page arch_is_platform_page
+#endif
+
#endif /* _ASM_X86_PROCESSOR_H */
diff --git a/arch/x86/include/asm/realmode.h b/arch/x86/include/asm/realmode.h
index 5db5d08..331474b 100644
--- a/arch/x86/include/asm/realmode.h
+++ b/arch/x86/include/asm/realmode.h
@@ -89,6 +89,7 @@ static inline void set_real_mode_mem(phys_addr_t mem)
}
void reserve_real_mode(void);
+void load_trampoline_pgtable(void);
#endif /* __ASSEMBLY__ */
diff --git a/arch/x86/include/asm/set_memory.h b/arch/x86/include/asm/set_memory.h
index 8726175..ff0f2d9 100644
--- a/arch/x86/include/asm/set_memory.h
+++ b/arch/x86/include/asm/set_memory.h
@@ -2,6 +2,7 @@
#ifndef _ASM_X86_SET_MEMORY_H
#define _ASM_X86_SET_MEMORY_H
+#include <linux/mm.h>
#include <asm/page.h>
#include <asm-generic/set_memory.h>
@@ -99,6 +100,9 @@ static inline int set_mce_nospec(unsigned long pfn, bool unmap)
unsigned long decoy_addr;
int rc;
+ /* SGX pages are not in the 1:1 map */
+ if (arch_is_platform_page(pfn << PAGE_SHIFT))
+ return 0;
/*
* We would like to just call:
* set_memory_XX((unsigned long)pfn_to_kaddr(pfn), 1);
diff --git a/arch/x86/include/asm/sev-common.h b/arch/x86/include/asm/sev-common.h
index 6acaf5a..1b2fd32 100644
--- a/arch/x86/include/asm/sev-common.h
+++ b/arch/x86/include/asm/sev-common.h
@@ -18,20 +18,19 @@
/* SEV Information Request/Response */
#define GHCB_MSR_SEV_INFO_RESP 0x001
#define GHCB_MSR_SEV_INFO_REQ 0x002
-#define GHCB_MSR_VER_MAX_POS 48
-#define GHCB_MSR_VER_MAX_MASK 0xffff
-#define GHCB_MSR_VER_MIN_POS 32
-#define GHCB_MSR_VER_MIN_MASK 0xffff
-#define GHCB_MSR_CBIT_POS 24
-#define GHCB_MSR_CBIT_MASK 0xff
-#define GHCB_MSR_SEV_INFO(_max, _min, _cbit) \
- ((((_max) & GHCB_MSR_VER_MAX_MASK) << GHCB_MSR_VER_MAX_POS) | \
- (((_min) & GHCB_MSR_VER_MIN_MASK) << GHCB_MSR_VER_MIN_POS) | \
- (((_cbit) & GHCB_MSR_CBIT_MASK) << GHCB_MSR_CBIT_POS) | \
+
+#define GHCB_MSR_SEV_INFO(_max, _min, _cbit) \
+ /* GHCBData[63:48] */ \
+ ((((_max) & 0xffff) << 48) | \
+ /* GHCBData[47:32] */ \
+ (((_min) & 0xffff) << 32) | \
+ /* GHCBData[31:24] */ \
+ (((_cbit) & 0xff) << 24) | \
GHCB_MSR_SEV_INFO_RESP)
+
#define GHCB_MSR_INFO(v) ((v) & 0xfffUL)
-#define GHCB_MSR_PROTO_MAX(v) (((v) >> GHCB_MSR_VER_MAX_POS) & GHCB_MSR_VER_MAX_MASK)
-#define GHCB_MSR_PROTO_MIN(v) (((v) >> GHCB_MSR_VER_MIN_POS) & GHCB_MSR_VER_MIN_MASK)
+#define GHCB_MSR_PROTO_MAX(v) (((v) >> 48) & 0xffff)
+#define GHCB_MSR_PROTO_MIN(v) (((v) >> 32) & 0xffff)
/* CPUID Request/Response */
#define GHCB_MSR_CPUID_REQ 0x004
@@ -46,30 +45,36 @@
#define GHCB_CPUID_REQ_EBX 1
#define GHCB_CPUID_REQ_ECX 2
#define GHCB_CPUID_REQ_EDX 3
-#define GHCB_CPUID_REQ(fn, reg) \
- (GHCB_MSR_CPUID_REQ | \
- (((unsigned long)reg & GHCB_MSR_CPUID_REG_MASK) << GHCB_MSR_CPUID_REG_POS) | \
- (((unsigned long)fn) << GHCB_MSR_CPUID_FUNC_POS))
+#define GHCB_CPUID_REQ(fn, reg) \
+ /* GHCBData[11:0] */ \
+ (GHCB_MSR_CPUID_REQ | \
+ /* GHCBData[31:12] */ \
+ (((unsigned long)(reg) & 0x3) << 30) | \
+ /* GHCBData[63:32] */ \
+ (((unsigned long)fn) << 32))
/* AP Reset Hold */
-#define GHCB_MSR_AP_RESET_HOLD_REQ 0x006
-#define GHCB_MSR_AP_RESET_HOLD_RESP 0x007
+#define GHCB_MSR_AP_RESET_HOLD_REQ 0x006
+#define GHCB_MSR_AP_RESET_HOLD_RESP 0x007
/* GHCB Hypervisor Feature Request/Response */
-#define GHCB_MSR_HV_FT_REQ 0x080
-#define GHCB_MSR_HV_FT_RESP 0x081
+#define GHCB_MSR_HV_FT_REQ 0x080
+#define GHCB_MSR_HV_FT_RESP 0x081
#define GHCB_MSR_TERM_REQ 0x100
#define GHCB_MSR_TERM_REASON_SET_POS 12
#define GHCB_MSR_TERM_REASON_SET_MASK 0xf
#define GHCB_MSR_TERM_REASON_POS 16
#define GHCB_MSR_TERM_REASON_MASK 0xff
-#define GHCB_SEV_TERM_REASON(reason_set, reason_val) \
- (((((u64)reason_set) & GHCB_MSR_TERM_REASON_SET_MASK) << GHCB_MSR_TERM_REASON_SET_POS) | \
- ((((u64)reason_val) & GHCB_MSR_TERM_REASON_MASK) << GHCB_MSR_TERM_REASON_POS))
-#define GHCB_SEV_ES_REASON_GENERAL_REQUEST 0
-#define GHCB_SEV_ES_REASON_PROTOCOL_UNSUPPORTED 1
+#define GHCB_SEV_TERM_REASON(reason_set, reason_val) \
+ /* GHCBData[15:12] */ \
+ (((((u64)reason_set) & 0xf) << 12) | \
+ /* GHCBData[23:16] */ \
+ ((((u64)reason_val) & 0xff) << 16))
+
+#define GHCB_SEV_ES_GEN_REQ 0
+#define GHCB_SEV_ES_PROT_UNSUPPORTED 1
#define GHCB_RESP_CODE(v) ((v) & GHCB_MSR_INFO_MASK)
diff --git a/arch/x86/include/asm/tlbflush.h b/arch/x86/include/asm/tlbflush.h
index b587a9e..98fa0a1 100644
--- a/arch/x86/include/asm/tlbflush.h
+++ b/arch/x86/include/asm/tlbflush.h
@@ -261,4 +261,9 @@ extern void arch_tlbbatch_flush(struct arch_tlbflush_unmap_batch *batch);
#endif /* !MODULE */
+static inline void __native_tlb_flush_global(unsigned long cr4)
+{
+ native_write_cr4(cr4 ^ X86_CR4_PGE);
+ native_write_cr4(cr4);
+}
#endif /* _ASM_X86_TLBFLUSH_H */
diff --git a/arch/x86/include/asm/uaccess.h b/arch/x86/include/asm/uaccess.h
index 33a6840..8ab9e79 100644
--- a/arch/x86/include/asm/uaccess.h
+++ b/arch/x86/include/asm/uaccess.h
@@ -314,11 +314,12 @@ do { \
do { \
__chk_user_ptr(ptr); \
switch (size) { \
- unsigned char x_u8__; \
- case 1: \
+ case 1: { \
+ unsigned char x_u8__; \
__get_user_asm(x_u8__, ptr, "b", "=q", label); \
(x) = x_u8__; \
break; \
+ } \
case 2: \
__get_user_asm(x, ptr, "w", "=r", label); \
break; \
diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
index 2ff3e60..6aef9ee 100644
--- a/arch/x86/kernel/Makefile
+++ b/arch/x86/kernel/Makefile
@@ -84,7 +84,7 @@
obj-y += step.o
obj-$(CONFIG_INTEL_TXT) += tboot.o
obj-$(CONFIG_ISA_DMA_API) += i8237.o
-obj-$(CONFIG_STACKTRACE) += stacktrace.o
+obj-y += stacktrace.o
obj-y += cpu/
obj-y += acpi/
obj-y += reboot.o
diff --git a/arch/x86/kernel/amd_nb.c b/arch/x86/kernel/amd_nb.c
index c92c9c7..f814d5f 100644
--- a/arch/x86/kernel/amd_nb.c
+++ b/arch/x86/kernel/amd_nb.c
@@ -29,7 +29,7 @@
#define PCI_DEVICE_ID_AMD_19H_M40H_DF_F4 0x167d
#define PCI_DEVICE_ID_AMD_19H_M50H_DF_F4 0x166e
-/* Protect the PCI config register pairs used for SMN and DF indirect access. */
+/* Protect the PCI config register pairs used for SMN. */
static DEFINE_MUTEX(smn_mutex);
static u32 *flush_words;
@@ -182,53 +182,6 @@ int amd_smn_write(u16 node, u32 address, u32 value)
}
EXPORT_SYMBOL_GPL(amd_smn_write);
-/*
- * Data Fabric Indirect Access uses FICAA/FICAD.
- *
- * Fabric Indirect Configuration Access Address (FICAA): Constructed based
- * on the device's Instance Id and the PCI function and register offset of
- * the desired register.
- *
- * Fabric Indirect Configuration Access Data (FICAD): There are FICAD LO
- * and FICAD HI registers but so far we only need the LO register.
- */
-int amd_df_indirect_read(u16 node, u8 func, u16 reg, u8 instance_id, u32 *lo)
-{
- struct pci_dev *F4;
- u32 ficaa;
- int err = -ENODEV;
-
- if (node >= amd_northbridges.num)
- goto out;
-
- F4 = node_to_amd_nb(node)->link;
- if (!F4)
- goto out;
-
- ficaa = 1;
- ficaa |= reg & 0x3FC;
- ficaa |= (func & 0x7) << 11;
- ficaa |= instance_id << 16;
-
- mutex_lock(&smn_mutex);
-
- err = pci_write_config_dword(F4, 0x5C, ficaa);
- if (err) {
- pr_warn("Error writing DF Indirect FICAA, FICAA=0x%x\n", ficaa);
- goto out_unlock;
- }
-
- err = pci_read_config_dword(F4, 0x98, lo);
- if (err)
- pr_warn("Error reading DF Indirect FICAD LO, FICAA=0x%x.\n", ficaa);
-
-out_unlock:
- mutex_unlock(&smn_mutex);
-
-out:
- return err;
-}
-EXPORT_SYMBOL_GPL(amd_df_indirect_read);
int amd_cache_northbridges(void)
{
diff --git a/arch/x86/kernel/cc_platform.c b/arch/x86/kernel/cc_platform.c
index 03bb2f3..8a25b1c 100644
--- a/arch/x86/kernel/cc_platform.c
+++ b/arch/x86/kernel/cc_platform.c
@@ -50,6 +50,14 @@ static bool amd_cc_platform_has(enum cc_attr attr)
case CC_ATTR_GUEST_STATE_ENCRYPT:
return sev_status & MSR_AMD64_SEV_ES_ENABLED;
+ /*
+ * With SEV, the rep string I/O instructions need to be unrolled
+ * but SEV-ES supports them through the #VC handler.
+ */
+ case CC_ATTR_GUEST_UNROLL_STRING_IO:
+ return (sev_status & MSR_AMD64_SEV_ENABLED) &&
+ !(sev_status & MSR_AMD64_SEV_ES_ENABLED);
+
default:
return false;
}
diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c
index 0083464..7b8382c 100644
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -384,7 +384,7 @@ void native_write_cr0(unsigned long val)
}
EXPORT_SYMBOL(native_write_cr0);
-void native_write_cr4(unsigned long val)
+void __no_profile native_write_cr4(unsigned long val)
{
unsigned long bits_changed = 0;
@@ -1787,6 +1787,17 @@ EXPORT_PER_CPU_SYMBOL(__preempt_count);
DEFINE_PER_CPU(unsigned long, cpu_current_top_of_stack) = TOP_OF_INIT_STACK;
+static void wrmsrl_cstar(unsigned long val)
+{
+ /*
+ * Intel CPUs do not support 32-bit SYSCALL. Writing to MSR_CSTAR
+ * is so far ignored by the CPU, but raises a #VE trap in a TDX
+ * guest. Avoid the pointless write on all Intel CPUs.
+ */
+ if (boot_cpu_data.x86_vendor != X86_VENDOR_INTEL)
+ wrmsrl(MSR_CSTAR, val);
+}
+
/* May not be marked __init: used by software suspend */
void syscall_init(void)
{
@@ -1794,7 +1805,7 @@ void syscall_init(void)
wrmsrl(MSR_LSTAR, (unsigned long)entry_SYSCALL_64);
#ifdef CONFIG_IA32_EMULATION
- wrmsrl(MSR_CSTAR, (unsigned long)entry_SYSCALL_compat);
+ wrmsrl_cstar((unsigned long)entry_SYSCALL_compat);
/*
* This only works on Intel CPUs.
* On AMD CPUs these MSRs are 32-bit, CPU truncates MSR_IA32_SYSENTER_EIP.
@@ -1806,7 +1817,7 @@ void syscall_init(void)
(unsigned long)(cpu_entry_stack(smp_processor_id()) + 1));
wrmsrl_safe(MSR_IA32_SYSENTER_EIP, (u64)entry_SYSENTER_compat);
#else
- wrmsrl(MSR_CSTAR, (unsigned long)ignore_sysret);
+ wrmsrl_cstar((unsigned long)ignore_sysret);
wrmsrl_safe(MSR_IA32_SYSENTER_CS, (u64)GDT_ENTRY_INVALID_SEG);
wrmsrl_safe(MSR_IA32_SYSENTER_ESP, 0ULL);
wrmsrl_safe(MSR_IA32_SYSENTER_EIP, 0ULL);
diff --git a/arch/x86/kernel/cpu/mce/amd.c b/arch/x86/kernel/cpu/mce/amd.c
index fc85eb1..a1e2f41 100644
--- a/arch/x86/kernel/cpu/mce/amd.c
+++ b/arch/x86/kernel/cpu/mce/amd.c
@@ -71,6 +71,22 @@ static const char * const smca_umc_block_names[] = {
"misc_umc"
};
+#define HWID_MCATYPE(hwid, mcatype) (((hwid) << 16) | (mcatype))
+
+struct smca_hwid {
+ unsigned int bank_type; /* Use with smca_bank_types for easy indexing. */
+ u32 hwid_mcatype; /* (hwid,mcatype) tuple */
+};
+
+struct smca_bank {
+ const struct smca_hwid *hwid;
+ u32 id; /* Value of MCA_IPID[InstanceId]. */
+ u8 sysfs_id; /* Value used for sysfs name. */
+};
+
+static DEFINE_PER_CPU_READ_MOSTLY(struct smca_bank[MAX_NR_BANKS], smca_banks);
+static DEFINE_PER_CPU_READ_MOSTLY(u8[N_SMCA_BANK_TYPES], smca_bank_counts);
+
struct smca_bank_name {
const char *name; /* Short name for sysfs */
const char *long_name; /* Long name for pretty-printing */
@@ -95,11 +111,18 @@ static struct smca_bank_name smca_names[] = {
[SMCA_PSP ... SMCA_PSP_V2] = { "psp", "Platform Security Processor" },
[SMCA_SMU ... SMCA_SMU_V2] = { "smu", "System Management Unit" },
[SMCA_MP5] = { "mp5", "Microprocessor 5 Unit" },
+ [SMCA_MPDMA] = { "mpdma", "MPDMA Unit" },
[SMCA_NBIO] = { "nbio", "Northbridge IO Unit" },
[SMCA_PCIE ... SMCA_PCIE_V2] = { "pcie", "PCI Express Unit" },
[SMCA_XGMI_PCS] = { "xgmi_pcs", "Ext Global Memory Interconnect PCS Unit" },
+ [SMCA_NBIF] = { "nbif", "NBIF Unit" },
+ [SMCA_SHUB] = { "shub", "System Hub Unit" },
+ [SMCA_SATA] = { "sata", "SATA Unit" },
+ [SMCA_USB] = { "usb", "USB Unit" },
+ [SMCA_GMI_PCS] = { "gmi_pcs", "Global Memory Interconnect PCS Unit" },
[SMCA_XGMI_PHY] = { "xgmi_phy", "Ext Global Memory Interconnect PHY Unit" },
[SMCA_WAFL_PHY] = { "wafl_phy", "WAFL PHY Unit" },
+ [SMCA_GMI_PHY] = { "gmi_phy", "Global Memory Interconnect PHY Unit" },
};
static const char *smca_get_name(enum smca_bank_types t)
@@ -119,14 +142,14 @@ const char *smca_get_long_name(enum smca_bank_types t)
}
EXPORT_SYMBOL_GPL(smca_get_long_name);
-enum smca_bank_types smca_get_bank_type(unsigned int bank)
+enum smca_bank_types smca_get_bank_type(unsigned int cpu, unsigned int bank)
{
struct smca_bank *b;
if (bank >= MAX_NR_BANKS)
return N_SMCA_BANK_TYPES;
- b = &smca_banks[bank];
+ b = &per_cpu(smca_banks, cpu)[bank];
if (!b->hwid)
return N_SMCA_BANK_TYPES;
@@ -134,7 +157,7 @@ enum smca_bank_types smca_get_bank_type(unsigned int bank)
}
EXPORT_SYMBOL_GPL(smca_get_bank_type);
-static struct smca_hwid smca_hwid_mcatypes[] = {
+static const struct smca_hwid smca_hwid_mcatypes[] = {
/* { bank_type, hwid_mcatype } */
/* Reserved type */
@@ -174,6 +197,9 @@ static struct smca_hwid smca_hwid_mcatypes[] = {
/* Microprocessor 5 Unit MCA type */
{ SMCA_MP5, HWID_MCATYPE(0x01, 0x2) },
+ /* MPDMA MCA type */
+ { SMCA_MPDMA, HWID_MCATYPE(0x01, 0x3) },
+
/* Northbridge IO Unit MCA type */
{ SMCA_NBIO, HWID_MCATYPE(0x18, 0x0) },
@@ -181,19 +207,17 @@ static struct smca_hwid smca_hwid_mcatypes[] = {
{ SMCA_PCIE, HWID_MCATYPE(0x46, 0x0) },
{ SMCA_PCIE_V2, HWID_MCATYPE(0x46, 0x1) },
- /* xGMI PCS MCA type */
{ SMCA_XGMI_PCS, HWID_MCATYPE(0x50, 0x0) },
-
- /* xGMI PHY MCA type */
+ { SMCA_NBIF, HWID_MCATYPE(0x6C, 0x0) },
+ { SMCA_SHUB, HWID_MCATYPE(0x80, 0x0) },
+ { SMCA_SATA, HWID_MCATYPE(0xA8, 0x0) },
+ { SMCA_USB, HWID_MCATYPE(0xAA, 0x0) },
+ { SMCA_GMI_PCS, HWID_MCATYPE(0x241, 0x0) },
{ SMCA_XGMI_PHY, HWID_MCATYPE(0x259, 0x0) },
-
- /* WAFL PHY MCA type */
{ SMCA_WAFL_PHY, HWID_MCATYPE(0x267, 0x0) },
+ { SMCA_GMI_PHY, HWID_MCATYPE(0x269, 0x0) },
};
-struct smca_bank smca_banks[MAX_NR_BANKS];
-EXPORT_SYMBOL_GPL(smca_banks);
-
/*
* In SMCA enabled processors, we can have multiple banks for a given IP type.
* So to define a unique name for each bank, we use a temp c-string to append
@@ -249,8 +273,9 @@ static void smca_set_misc_banks_map(unsigned int bank, unsigned int cpu)
static void smca_configure(unsigned int bank, unsigned int cpu)
{
+ u8 *bank_counts = this_cpu_ptr(smca_bank_counts);
+ const struct smca_hwid *s_hwid;
unsigned int i, hwid_mcatype;
- struct smca_hwid *s_hwid;
u32 high, low;
u32 smca_config = MSR_AMD64_SMCA_MCx_CONFIG(bank);
@@ -286,10 +311,6 @@ static void smca_configure(unsigned int bank, unsigned int cpu)
smca_set_misc_banks_map(bank, cpu);
- /* Return early if this bank was already initialized. */
- if (smca_banks[bank].hwid && smca_banks[bank].hwid->hwid_mcatype != 0)
- return;
-
if (rdmsr_safe(MSR_AMD64_SMCA_MCx_IPID(bank), &low, &high)) {
pr_warn("Failed to read MCA_IPID for bank %d\n", bank);
return;
@@ -300,10 +321,11 @@ static void smca_configure(unsigned int bank, unsigned int cpu)
for (i = 0; i < ARRAY_SIZE(smca_hwid_mcatypes); i++) {
s_hwid = &smca_hwid_mcatypes[i];
+
if (hwid_mcatype == s_hwid->hwid_mcatype) {
- smca_banks[bank].hwid = s_hwid;
- smca_banks[bank].id = low;
- smca_banks[bank].sysfs_id = s_hwid->count++;
+ this_cpu_ptr(smca_banks)[bank].hwid = s_hwid;
+ this_cpu_ptr(smca_banks)[bank].id = low;
+ this_cpu_ptr(smca_banks)[bank].sysfs_id = bank_counts[s_hwid->bank_type]++;
break;
}
}
@@ -589,7 +611,7 @@ prepare_threshold_block(unsigned int bank, unsigned int block, u32 addr,
bool amd_filter_mce(struct mce *m)
{
- enum smca_bank_types bank_type = smca_get_bank_type(m->bank);
+ enum smca_bank_types bank_type = smca_get_bank_type(m->extcpu, m->bank);
struct cpuinfo_x86 *c = &boot_cpu_data;
/* See Family 17h Models 10h-2Fh Erratum #1114. */
@@ -627,7 +649,7 @@ static void disable_err_thresholding(struct cpuinfo_x86 *c, unsigned int bank)
} else if (c->x86 == 0x17 &&
(c->x86_model >= 0x10 && c->x86_model <= 0x2F)) {
- if (smca_get_bank_type(bank) != SMCA_IF)
+ if (smca_get_bank_type(smp_processor_id(), bank) != SMCA_IF)
return;
msrs[0] = MSR_AMD64_SMCA_MCx_MISC(bank);
@@ -689,213 +711,13 @@ void mce_amd_feature_init(struct cpuinfo_x86 *c)
deferred_error_interrupt_enable(c);
}
-int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
-{
- u64 dram_base_addr, dram_limit_addr, dram_hole_base;
- /* We start from the normalized address */
- u64 ret_addr = norm_addr;
-
- u32 tmp;
-
- u8 die_id_shift, die_id_mask, socket_id_shift, socket_id_mask;
- u8 intlv_num_dies, intlv_num_chan, intlv_num_sockets;
- u8 intlv_addr_sel, intlv_addr_bit;
- u8 num_intlv_bits, hashed_bit;
- u8 lgcy_mmio_hole_en, base = 0;
- u8 cs_mask, cs_id = 0;
- bool hash_enabled = false;
-
- /* Read D18F0x1B4 (DramOffset), check if base 1 is used. */
- if (amd_df_indirect_read(nid, 0, 0x1B4, umc, &tmp))
- goto out_err;
-
- /* Remove HiAddrOffset from normalized address, if enabled: */
- if (tmp & BIT(0)) {
- u64 hi_addr_offset = (tmp & GENMASK_ULL(31, 20)) << 8;
-
- if (norm_addr >= hi_addr_offset) {
- ret_addr -= hi_addr_offset;
- base = 1;
- }
- }
-
- /* Read D18F0x110 (DramBaseAddress). */
- if (amd_df_indirect_read(nid, 0, 0x110 + (8 * base), umc, &tmp))
- goto out_err;
-
- /* Check if address range is valid. */
- if (!(tmp & BIT(0))) {
- pr_err("%s: Invalid DramBaseAddress range: 0x%x.\n",
- __func__, tmp);
- goto out_err;
- }
-
- lgcy_mmio_hole_en = tmp & BIT(1);
- intlv_num_chan = (tmp >> 4) & 0xF;
- intlv_addr_sel = (tmp >> 8) & 0x7;
- dram_base_addr = (tmp & GENMASK_ULL(31, 12)) << 16;
-
- /* {0, 1, 2, 3} map to address bits {8, 9, 10, 11} respectively */
- if (intlv_addr_sel > 3) {
- pr_err("%s: Invalid interleave address select %d.\n",
- __func__, intlv_addr_sel);
- goto out_err;
- }
-
- /* Read D18F0x114 (DramLimitAddress). */
- if (amd_df_indirect_read(nid, 0, 0x114 + (8 * base), umc, &tmp))
- goto out_err;
-
- intlv_num_sockets = (tmp >> 8) & 0x1;
- intlv_num_dies = (tmp >> 10) & 0x3;
- dram_limit_addr = ((tmp & GENMASK_ULL(31, 12)) << 16) | GENMASK_ULL(27, 0);
-
- intlv_addr_bit = intlv_addr_sel + 8;
-
- /* Re-use intlv_num_chan by setting it equal to log2(#channels) */
- switch (intlv_num_chan) {
- case 0: intlv_num_chan = 0; break;
- case 1: intlv_num_chan = 1; break;
- case 3: intlv_num_chan = 2; break;
- case 5: intlv_num_chan = 3; break;
- case 7: intlv_num_chan = 4; break;
-
- case 8: intlv_num_chan = 1;
- hash_enabled = true;
- break;
- default:
- pr_err("%s: Invalid number of interleaved channels %d.\n",
- __func__, intlv_num_chan);
- goto out_err;
- }
-
- num_intlv_bits = intlv_num_chan;
-
- if (intlv_num_dies > 2) {
- pr_err("%s: Invalid number of interleaved nodes/dies %d.\n",
- __func__, intlv_num_dies);
- goto out_err;
- }
-
- num_intlv_bits += intlv_num_dies;
-
- /* Add a bit if sockets are interleaved. */
- num_intlv_bits += intlv_num_sockets;
-
- /* Assert num_intlv_bits <= 4 */
- if (num_intlv_bits > 4) {
- pr_err("%s: Invalid interleave bits %d.\n",
- __func__, num_intlv_bits);
- goto out_err;
- }
-
- if (num_intlv_bits > 0) {
- u64 temp_addr_x, temp_addr_i, temp_addr_y;
- u8 die_id_bit, sock_id_bit, cs_fabric_id;
-
- /*
- * Read FabricBlockInstanceInformation3_CS[BlockFabricID].
- * This is the fabric id for this coherent slave. Use
- * umc/channel# as instance id of the coherent slave
- * for FICAA.
- */
- if (amd_df_indirect_read(nid, 0, 0x50, umc, &tmp))
- goto out_err;
-
- cs_fabric_id = (tmp >> 8) & 0xFF;
- die_id_bit = 0;
-
- /* If interleaved over more than 1 channel: */
- if (intlv_num_chan) {
- die_id_bit = intlv_num_chan;
- cs_mask = (1 << die_id_bit) - 1;
- cs_id = cs_fabric_id & cs_mask;
- }
-
- sock_id_bit = die_id_bit;
-
- /* Read D18F1x208 (SystemFabricIdMask). */
- if (intlv_num_dies || intlv_num_sockets)
- if (amd_df_indirect_read(nid, 1, 0x208, umc, &tmp))
- goto out_err;
-
- /* If interleaved over more than 1 die. */
- if (intlv_num_dies) {
- sock_id_bit = die_id_bit + intlv_num_dies;
- die_id_shift = (tmp >> 24) & 0xF;
- die_id_mask = (tmp >> 8) & 0xFF;
-
- cs_id |= ((cs_fabric_id & die_id_mask) >> die_id_shift) << die_id_bit;
- }
-
- /* If interleaved over more than 1 socket. */
- if (intlv_num_sockets) {
- socket_id_shift = (tmp >> 28) & 0xF;
- socket_id_mask = (tmp >> 16) & 0xFF;
-
- cs_id |= ((cs_fabric_id & socket_id_mask) >> socket_id_shift) << sock_id_bit;
- }
-
- /*
- * The pre-interleaved address consists of XXXXXXIIIYYYYY
- * where III is the ID for this CS, and XXXXXXYYYYY are the
- * address bits from the post-interleaved address.
- * "num_intlv_bits" has been calculated to tell us how many "I"
- * bits there are. "intlv_addr_bit" tells us how many "Y" bits
- * there are (where "I" starts).
- */
- temp_addr_y = ret_addr & GENMASK_ULL(intlv_addr_bit-1, 0);
- temp_addr_i = (cs_id << intlv_addr_bit);
- temp_addr_x = (ret_addr & GENMASK_ULL(63, intlv_addr_bit)) << num_intlv_bits;
- ret_addr = temp_addr_x | temp_addr_i | temp_addr_y;
- }
-
- /* Add dram base address */
- ret_addr += dram_base_addr;
-
- /* If legacy MMIO hole enabled */
- if (lgcy_mmio_hole_en) {
- if (amd_df_indirect_read(nid, 0, 0x104, umc, &tmp))
- goto out_err;
-
- dram_hole_base = tmp & GENMASK(31, 24);
- if (ret_addr >= dram_hole_base)
- ret_addr += (BIT_ULL(32) - dram_hole_base);
- }
-
- if (hash_enabled) {
- /* Save some parentheses and grab ls-bit at the end. */
- hashed_bit = (ret_addr >> 12) ^
- (ret_addr >> 18) ^
- (ret_addr >> 21) ^
- (ret_addr >> 30) ^
- cs_id;
-
- hashed_bit &= BIT(0);
-
- if (hashed_bit != ((ret_addr >> intlv_addr_bit) & BIT(0)))
- ret_addr ^= BIT(intlv_addr_bit);
- }
-
- /* Is calculated system address is above DRAM limit address? */
- if (ret_addr > dram_limit_addr)
- goto out_err;
-
- *sys_addr = ret_addr;
- return 0;
-
-out_err:
- return -EINVAL;
-}
-EXPORT_SYMBOL_GPL(umc_normaddr_to_sysaddr);
-
bool amd_mce_is_memory_error(struct mce *m)
{
/* ErrCodeExt[20:16] */
u8 xec = (m->status >> 16) & 0x1f;
if (mce_flags.smca)
- return smca_get_bank_type(m->bank) == SMCA_UMC && xec == 0x0;
+ return smca_get_bank_type(m->extcpu, m->bank) == SMCA_UMC && xec == 0x0;
return m->bank == 4 && xec == 0x8;
}
@@ -1211,7 +1033,7 @@ static struct kobj_type threshold_ktype = {
.release = threshold_block_release,
};
-static const char *get_name(unsigned int bank, struct threshold_block *b)
+static const char *get_name(unsigned int cpu, unsigned int bank, struct threshold_block *b)
{
enum smca_bank_types bank_type;
@@ -1222,7 +1044,7 @@ static const char *get_name(unsigned int bank, struct threshold_block *b)
return th_names[bank];
}
- bank_type = smca_get_bank_type(bank);
+ bank_type = smca_get_bank_type(cpu, bank);
if (bank_type >= N_SMCA_BANK_TYPES)
return NULL;
@@ -1232,12 +1054,12 @@ static const char *get_name(unsigned int bank, struct threshold_block *b)
return NULL;
}
- if (smca_banks[bank].hwid->count == 1)
+ if (per_cpu(smca_bank_counts, cpu)[bank_type] == 1)
return smca_get_name(bank_type);
snprintf(buf_mcatype, MAX_MCATYPE_NAME_LEN,
- "%s_%x", smca_get_name(bank_type),
- smca_banks[bank].sysfs_id);
+ "%s_%u", smca_get_name(bank_type),
+ per_cpu(smca_banks, cpu)[bank].sysfs_id);
return buf_mcatype;
}
@@ -1293,7 +1115,7 @@ static int allocate_threshold_blocks(unsigned int cpu, struct threshold_bank *tb
else
tb->blocks = b;
- err = kobject_init_and_add(&b->kobj, &threshold_ktype, tb->kobj, get_name(bank, b));
+ err = kobject_init_and_add(&b->kobj, &threshold_ktype, tb->kobj, get_name(cpu, bank, b));
if (err)
goto out_free;
recurse:
@@ -1348,7 +1170,7 @@ static int threshold_create_bank(struct threshold_bank **bp, unsigned int cpu,
struct device *dev = this_cpu_read(mce_device);
struct amd_northbridge *nb = NULL;
struct threshold_bank *b = NULL;
- const char *name = get_name(bank, NULL);
+ const char *name = get_name(cpu, bank, NULL);
int err = 0;
if (!dev)
diff --git a/arch/x86/kernel/cpu/mce/core.c b/arch/x86/kernel/cpu/mce/core.c
index 6ed3653..5818b83 100644
--- a/arch/x86/kernel/cpu/mce/core.c
+++ b/arch/x86/kernel/cpu/mce/core.c
@@ -99,7 +99,6 @@ struct mca_config mca_cfg __read_mostly = {
static DEFINE_PER_CPU(struct mce, mces_seen);
static unsigned long mce_need_notify;
-static int cpu_missing;
/*
* MCA banks polled by the period polling timer for corrected events.
@@ -128,7 +127,7 @@ static struct irq_work mce_irq_work;
BLOCKING_NOTIFIER_HEAD(x86_mce_decoder_chain);
/* Do initial initialization of a struct mce */
-noinstr void mce_setup(struct mce *m)
+void mce_setup(struct mce *m)
{
memset(m, 0, sizeof(struct mce));
m->cpu = m->extcpu = smp_processor_id();
@@ -267,11 +266,17 @@ static void wait_for_panic(void)
panic("Panicing machine check CPU died");
}
-static void mce_panic(const char *msg, struct mce *final, char *exp)
+static noinstr void mce_panic(const char *msg, struct mce *final, char *exp)
{
- int apei_err = 0;
struct llist_node *pending;
struct mce_evt_llist *l;
+ int apei_err = 0;
+
+ /*
+ * Allow instrumentation around external facilities usage. Not that it
+ * matters a whole lot since the machine is going to panic anyway.
+ */
+ instrumentation_begin();
if (!fake_panic) {
/*
@@ -286,7 +291,7 @@ static void mce_panic(const char *msg, struct mce *final, char *exp)
} else {
/* Don't log too much for fake panic */
if (atomic_inc_return(&mce_fake_panicked) > 1)
- return;
+ goto out;
}
pending = mce_gen_pool_prepare_records();
/* First print corrected ones that are still unlogged */
@@ -314,8 +319,6 @@ static void mce_panic(const char *msg, struct mce *final, char *exp)
if (!apei_err)
apei_err = apei_write_mce(final);
}
- if (cpu_missing)
- pr_emerg(HW_ERR "Some CPUs didn't answer in synchronization\n");
if (exp)
pr_emerg(HW_ERR "Machine check: %s\n", exp);
if (!fake_panic) {
@@ -324,6 +327,9 @@ static void mce_panic(const char *msg, struct mce *final, char *exp)
panic(msg);
} else
pr_emerg(HW_ERR "Fake kernel panic: %s\n", msg);
+
+out:
+ instrumentation_end();
}
/* Support code for software error injection */
@@ -365,7 +371,7 @@ void ex_handler_msr_mce(struct pt_regs *regs, bool wrmsr)
}
/* MSR access wrappers used for error injection */
-static noinstr u64 mce_rdmsrl(u32 msr)
+noinstr u64 mce_rdmsrl(u32 msr)
{
DECLARE_ARGS(val, low, high);
@@ -433,9 +439,15 @@ static noinstr void mce_wrmsrl(u32 msr, u64 v)
* check into our "mce" struct so that we can use it later to assess
* the severity of the problem as we read per-bank specific details.
*/
-static inline void mce_gather_info(struct mce *m, struct pt_regs *regs)
+static noinstr void mce_gather_info(struct mce *m, struct pt_regs *regs)
{
+ /*
+ * Enable instrumentation around mce_setup() which calls external
+ * facilities.
+ */
+ instrumentation_begin();
mce_setup(m);
+ instrumentation_end();
m->mcgstatus = mce_rdmsrl(MSR_IA32_MCG_STATUS);
if (regs) {
@@ -636,7 +648,7 @@ static struct notifier_block mce_default_nb = {
/*
* Read ADDR and MISC registers.
*/
-static void mce_read_aux(struct mce *m, int i)
+static noinstr void mce_read_aux(struct mce *m, int i)
{
if (m->status & MCI_STATUS_MISCV)
m->misc = mce_rdmsrl(mca_msr_reg(i, MCA_MISC));
@@ -871,8 +883,13 @@ static cpumask_t mce_missing_cpus = CPU_MASK_ALL;
/*
* Check if a timeout waiting for other CPUs happened.
*/
-static int mce_timed_out(u64 *t, const char *msg)
+static noinstr int mce_timed_out(u64 *t, const char *msg)
{
+ int ret = 0;
+
+ /* Enable instrumentation around calls to external facilities */
+ instrumentation_begin();
+
/*
* The others already did panic for some reason.
* Bail out like in a timeout.
@@ -891,13 +908,17 @@ static int mce_timed_out(u64 *t, const char *msg)
cpumask_pr_args(&mce_missing_cpus));
mce_panic(msg, NULL, NULL);
}
- cpu_missing = 1;
- return 1;
+ ret = 1;
+ goto out;
}
*t -= SPINUNIT;
+
out:
touch_nmi_watchdog();
- return 0;
+
+ instrumentation_end();
+
+ return ret;
}
/*
@@ -986,14 +1007,13 @@ static atomic_t global_nwo;
* in the entry order.
* TBD double check parallel CPU hotunplug
*/
-static int mce_start(int *no_way_out)
+static noinstr int mce_start(int *no_way_out)
{
- int order;
- int cpus = num_online_cpus();
u64 timeout = (u64)mca_cfg.monarch_timeout * NSEC_PER_USEC;
+ int order, ret = -1;
if (!timeout)
- return -1;
+ return ret;
atomic_add(*no_way_out, &global_nwo);
/*
@@ -1003,14 +1023,17 @@ static int mce_start(int *no_way_out)
order = atomic_inc_return(&mce_callin);
cpumask_clear_cpu(smp_processor_id(), &mce_missing_cpus);
+ /* Enable instrumentation around calls to external facilities */
+ instrumentation_begin();
+
/*
* Wait for everyone.
*/
- while (atomic_read(&mce_callin) != cpus) {
+ while (atomic_read(&mce_callin) != num_online_cpus()) {
if (mce_timed_out(&timeout,
"Timeout: Not all CPUs entered broadcast exception handler")) {
atomic_set(&global_nwo, 0);
- return -1;
+ goto out;
}
ndelay(SPINUNIT);
}
@@ -1036,7 +1059,7 @@ static int mce_start(int *no_way_out)
if (mce_timed_out(&timeout,
"Timeout: Subject CPUs unable to finish machine check processing")) {
atomic_set(&global_nwo, 0);
- return -1;
+ goto out;
}
ndelay(SPINUNIT);
}
@@ -1047,17 +1070,25 @@ static int mce_start(int *no_way_out)
*/
*no_way_out = atomic_read(&global_nwo);
- return order;
+ ret = order;
+
+out:
+ instrumentation_end();
+
+ return ret;
}
/*
* Synchronize between CPUs after main scanning loop.
* This invokes the bulk of the Monarch processing.
*/
-static int mce_end(int order)
+static noinstr int mce_end(int order)
{
- int ret = -1;
u64 timeout = (u64)mca_cfg.monarch_timeout * NSEC_PER_USEC;
+ int ret = -1;
+
+ /* Allow instrumentation around external facilities. */
+ instrumentation_begin();
if (!timeout)
goto reset;
@@ -1070,14 +1101,11 @@ static int mce_end(int order)
atomic_inc(&mce_executing);
if (order == 1) {
- /* CHECKME: Can this race with a parallel hotplug? */
- int cpus = num_online_cpus();
-
/*
* Monarch: Wait for everyone to go through their scanning
* loops.
*/
- while (atomic_read(&mce_executing) <= cpus) {
+ while (atomic_read(&mce_executing) <= num_online_cpus()) {
if (mce_timed_out(&timeout,
"Timeout: Monarch CPU unable to finish machine check processing"))
goto reset;
@@ -1101,7 +1129,8 @@ static int mce_end(int order)
/*
* Don't reset anything. That's done by the Monarch.
*/
- return 0;
+ ret = 0;
+ goto out;
}
/*
@@ -1117,6 +1146,10 @@ static int mce_end(int order)
* Let others run again.
*/
atomic_set(&mce_executing, 0);
+
+out:
+ instrumentation_end();
+
return ret;
}
@@ -1165,13 +1198,14 @@ static noinstr bool mce_check_crashing_cpu(void)
return false;
}
-static void __mc_scan_banks(struct mce *m, struct pt_regs *regs, struct mce *final,
- unsigned long *toclear, unsigned long *valid_banks,
- int no_way_out, int *worst)
+static __always_inline int
+__mc_scan_banks(struct mce *m, struct pt_regs *regs, struct mce *final,
+ unsigned long *toclear, unsigned long *valid_banks, int no_way_out,
+ int *worst)
{
struct mce_bank *mce_banks = this_cpu_ptr(mce_banks_array);
struct mca_config *cfg = &mca_cfg;
- int severity, i;
+ int severity, i, taint = 0;
for (i = 0; i < this_cpu_read(mce_num_banks); i++) {
__clear_bit(i, toclear);
@@ -1198,7 +1232,7 @@ static void __mc_scan_banks(struct mce *m, struct pt_regs *regs, struct mce *fin
continue;
/* Set taint even when machine check was not enabled. */
- add_taint(TAINT_MACHINE_CHECK, LOCKDEP_NOW_UNRELIABLE);
+ taint++;
severity = mce_severity(m, regs, cfg->tolerant, NULL, true);
@@ -1221,7 +1255,13 @@ static void __mc_scan_banks(struct mce *m, struct pt_regs *regs, struct mce *fin
/* assuming valid severity level != 0 */
m->severity = severity;
+ /*
+ * Enable instrumentation around the mce_log() call which is
+ * done in #MC context, where instrumentation is disabled.
+ */
+ instrumentation_begin();
mce_log(m);
+ instrumentation_end();
if (severity > *worst) {
*final = *m;
@@ -1231,6 +1271,8 @@ static void __mc_scan_banks(struct mce *m, struct pt_regs *regs, struct mce *fin
/* mce_clear_state will clear *final, save locally for use later */
*m = *final;
+
+ return taint;
}
static void kill_me_now(struct callback_head *ch)
@@ -1320,11 +1362,11 @@ static noinstr void unexpected_machine_check(struct pt_regs *regs)
}
/*
- * The actual machine check handler. This only handles real
- * exceptions when something got corrupted coming in through int 18.
+ * The actual machine check handler. This only handles real exceptions when
+ * something got corrupted coming in through int 18.
*
- * This is executed in NMI context not subject to normal locking rules. This
- * implies that most kernel services cannot be safely used. Don't even
+ * This is executed in #MC context not subject to normal locking rules.
+ * This implies that most kernel services cannot be safely used. Don't even
* think about putting a printk in there!
*
* On Intel systems this is entered on all CPUs in parallel through
@@ -1336,12 +1378,20 @@ static noinstr void unexpected_machine_check(struct pt_regs *regs)
* issues: if the machine check was due to a failure of the memory
* backing the user stack, tracing that reads the user stack will cause
* potentially infinite recursion.
+ *
+ * Currently, the #MC handler calls out to a number of external facilities
+ * and, therefore, allows instrumentation around them. The optimal thing to
+ * have would be to do the absolutely minimal work required in #MC context
+ * and have instrumentation disabled only around that. Further processing can
+ * then happen in process context where instrumentation is allowed. Achieving
+ * that requires careful auditing and modifications. Until then, the code
+ * allows instrumentation temporarily, where required. *
*/
noinstr void do_machine_check(struct pt_regs *regs)
{
- int worst = 0, order, no_way_out, kill_current_task, lmce;
- DECLARE_BITMAP(valid_banks, MAX_NR_BANKS);
- DECLARE_BITMAP(toclear, MAX_NR_BANKS);
+ int worst = 0, order, no_way_out, kill_current_task, lmce, taint = 0;
+ DECLARE_BITMAP(valid_banks, MAX_NR_BANKS) = { 0 };
+ DECLARE_BITMAP(toclear, MAX_NR_BANKS) = { 0 };
struct mca_config *cfg = &mca_cfg;
struct mce m, *final;
char *msg = NULL;
@@ -1385,7 +1435,6 @@ noinstr void do_machine_check(struct pt_regs *regs)
final = this_cpu_ptr(&mces_seen);
*final = m;
- memset(valid_banks, 0, sizeof(valid_banks));
no_way_out = mce_no_way_out(&m, &msg, valid_banks, regs);
barrier();
@@ -1419,7 +1468,7 @@ noinstr void do_machine_check(struct pt_regs *regs)
order = mce_start(&no_way_out);
}
- __mc_scan_banks(&m, regs, final, toclear, valid_banks, no_way_out, &worst);
+ taint = __mc_scan_banks(&m, regs, final, toclear, valid_banks, no_way_out, &worst);
if (!no_way_out)
mce_clear_state(toclear);
@@ -1451,6 +1500,16 @@ noinstr void do_machine_check(struct pt_regs *regs)
}
}
+ /*
+ * Enable instrumentation around the external facilities like task_work_add()
+ * (via queue_task_work()), fixup_exception() etc. For now, that is. Fixing this
+ * properly would need a lot more involved reorganization.
+ */
+ instrumentation_begin();
+
+ if (taint)
+ add_taint(TAINT_MACHINE_CHECK, LOCKDEP_NOW_UNRELIABLE);
+
if (worst != MCE_AR_SEVERITY && !kill_current_task)
goto out;
@@ -1482,7 +1541,10 @@ noinstr void do_machine_check(struct pt_regs *regs)
if (m.kflags & MCE_IN_KERNEL_COPYIN)
queue_task_work(&m, msg, kill_me_never);
}
+
out:
+ instrumentation_end();
+
mce_wrmsrl(MSR_IA32_MCG_STATUS, 0);
}
EXPORT_SYMBOL_GPL(do_machine_check);
@@ -2702,7 +2764,6 @@ struct dentry *mce_get_debugfs_dir(void)
static void mce_reset(void)
{
- cpu_missing = 0;
atomic_set(&mce_fake_panicked, 0);
atomic_set(&mce_executing, 0);
atomic_set(&mce_callin, 0);
diff --git a/arch/x86/kernel/cpu/mce/inject.c b/arch/x86/kernel/cpu/mce/inject.c
index 0bfc140..5fbd7ff 100644
--- a/arch/x86/kernel/cpu/mce/inject.c
+++ b/arch/x86/kernel/cpu/mce/inject.c
@@ -74,7 +74,6 @@ MCE_INJECT_SET(status);
MCE_INJECT_SET(misc);
MCE_INJECT_SET(addr);
MCE_INJECT_SET(synd);
-MCE_INJECT_SET(ipid);
#define MCE_INJECT_GET(reg) \
static int inj_##reg##_get(void *data, u64 *val) \
@@ -95,6 +94,20 @@ DEFINE_SIMPLE_ATTRIBUTE(status_fops, inj_status_get, inj_status_set, "%llx\n");
DEFINE_SIMPLE_ATTRIBUTE(misc_fops, inj_misc_get, inj_misc_set, "%llx\n");
DEFINE_SIMPLE_ATTRIBUTE(addr_fops, inj_addr_get, inj_addr_set, "%llx\n");
DEFINE_SIMPLE_ATTRIBUTE(synd_fops, inj_synd_get, inj_synd_set, "%llx\n");
+
+/* Use the user provided IPID value on a sw injection. */
+static int inj_ipid_set(void *data, u64 val)
+{
+ struct mce *m = (struct mce *)data;
+
+ if (cpu_feature_enabled(X86_FEATURE_SMCA)) {
+ if (inj_type == SW_INJ)
+ m->ipid = val;
+ }
+
+ return 0;
+}
+
DEFINE_SIMPLE_ATTRIBUTE(ipid_fops, inj_ipid_get, inj_ipid_set, "%llx\n");
static void setup_inj_struct(struct mce *m)
@@ -350,7 +363,7 @@ static ssize_t flags_write(struct file *filp, const char __user *ubuf,
char buf[MAX_FLAG_OPT_SIZE], *__buf;
int err;
- if (cnt > MAX_FLAG_OPT_SIZE)
+ if (!cnt || cnt > MAX_FLAG_OPT_SIZE)
return -EINVAL;
if (copy_from_user(&buf, ubuf, cnt))
@@ -490,6 +503,8 @@ static void do_inject(void)
i_mce.tsc = rdtsc_ordered();
+ i_mce.status |= MCI_STATUS_VAL;
+
if (i_mce.misc)
i_mce.status |= MCI_STATUS_MISCV;
@@ -577,6 +592,33 @@ static int inj_bank_set(void *data, u64 val)
}
m->bank = val;
+
+ /*
+ * sw-only injection allows to write arbitrary values into the MCA
+ * registers because it tests only the decoding paths.
+ */
+ if (inj_type == SW_INJ)
+ goto inject;
+
+ /*
+ * Read IPID value to determine if a bank is populated on the target
+ * CPU.
+ */
+ if (cpu_feature_enabled(X86_FEATURE_SMCA)) {
+ u64 ipid;
+
+ if (rdmsrl_on_cpu(m->extcpu, MSR_AMD64_SMCA_MCx_IPID(val), &ipid)) {
+ pr_err("Error reading IPID on CPU%d\n", m->extcpu);
+ return -EINVAL;
+ }
+
+ if (!ipid) {
+ pr_err("Cannot inject into unpopulated bank %llu\n", val);
+ return -ENODEV;
+ }
+ }
+
+inject:
do_inject();
/* Reset injection struct */
diff --git a/arch/x86/kernel/cpu/mce/internal.h b/arch/x86/kernel/cpu/mce/internal.h
index acd61c4..52c6339 100644
--- a/arch/x86/kernel/cpu/mce/internal.h
+++ b/arch/x86/kernel/cpu/mce/internal.h
@@ -207,4 +207,6 @@ static inline void pentium_machine_check(struct pt_regs *regs) {}
static inline void winchip_machine_check(struct pt_regs *regs) {}
#endif
+noinstr u64 mce_rdmsrl(u32 msr);
+
#endif /* __X86_MCE_INTERNAL_H__ */
diff --git a/arch/x86/kernel/cpu/mce/severity.c b/arch/x86/kernel/cpu/mce/severity.c
index bb019a59..7aa2bda 100644
--- a/arch/x86/kernel/cpu/mce/severity.c
+++ b/arch/x86/kernel/cpu/mce/severity.c
@@ -222,6 +222,9 @@ static bool is_copy_from_user(struct pt_regs *regs)
struct insn insn;
int ret;
+ if (!regs)
+ return false;
+
if (copy_from_kernel_nofault(insn_buf, (void *)regs->ip, MAX_INSN_SIZE))
return false;
@@ -263,24 +266,36 @@ static bool is_copy_from_user(struct pt_regs *regs)
* distinguish an exception taken in user from from one
* taken in the kernel.
*/
-static int error_context(struct mce *m, struct pt_regs *regs)
+static noinstr int error_context(struct mce *m, struct pt_regs *regs)
{
+ int fixup_type;
+ bool copy_user;
+
if ((m->cs & 3) == 3)
return IN_USER;
+
if (!mc_recoverable(m->mcgstatus))
return IN_KERNEL;
- switch (ex_get_fixup_type(m->ip)) {
+ /* Allow instrumentation around external facilities usage. */
+ instrumentation_begin();
+ fixup_type = ex_get_fixup_type(m->ip);
+ copy_user = is_copy_from_user(regs);
+ instrumentation_end();
+
+ switch (fixup_type) {
case EX_TYPE_UACCESS:
case EX_TYPE_COPY:
- if (!regs || !is_copy_from_user(regs))
+ if (!copy_user)
return IN_KERNEL;
m->kflags |= MCE_IN_KERNEL_COPYIN;
fallthrough;
+
case EX_TYPE_FAULT_MCE_SAFE:
case EX_TYPE_DEFAULT_MCE_SAFE:
m->kflags |= MCE_IN_KERNEL_RECOV;
return IN_KERNEL_RECOV;
+
default:
return IN_KERNEL;
}
@@ -288,8 +303,7 @@ static int error_context(struct mce *m, struct pt_regs *regs)
static int mce_severity_amd_smca(struct mce *m, enum context err_ctx)
{
- u32 addr = MSR_AMD64_SMCA_MCx_CONFIG(m->bank);
- u32 low, high;
+ u64 mcx_cfg;
/*
* We need to look at the following bits:
@@ -300,11 +314,10 @@ static int mce_severity_amd_smca(struct mce *m, enum context err_ctx)
if (!mce_flags.succor)
return MCE_PANIC_SEVERITY;
- if (rdmsr_safe(addr, &low, &high))
- return MCE_PANIC_SEVERITY;
+ mcx_cfg = mce_rdmsrl(MSR_AMD64_SMCA_MCx_CONFIG(m->bank));
/* TCC (Task context corrupt). If set and if IN_KERNEL, panic. */
- if ((low & MCI_CONFIG_MCAX) &&
+ if ((mcx_cfg & MCI_CONFIG_MCAX) &&
(m->status & MCI_STATUS_TCC) &&
(err_ctx == IN_KERNEL))
return MCE_PANIC_SEVERITY;
@@ -317,8 +330,8 @@ static int mce_severity_amd_smca(struct mce *m, enum context err_ctx)
* See AMD Error Scope Hierarchy table in a newer BKDG. For example
* 49125_15h_Models_30h-3Fh_BKDG.pdf, section "RAS Features"
*/
-static int mce_severity_amd(struct mce *m, struct pt_regs *regs, int tolerant,
- char **msg, bool is_excp)
+static noinstr int mce_severity_amd(struct mce *m, struct pt_regs *regs, int tolerant,
+ char **msg, bool is_excp)
{
enum context ctx = error_context(m, regs);
@@ -370,8 +383,8 @@ static int mce_severity_amd(struct mce *m, struct pt_regs *regs, int tolerant,
return MCE_KEEP_SEVERITY;
}
-static int mce_severity_intel(struct mce *m, struct pt_regs *regs,
- int tolerant, char **msg, bool is_excp)
+static noinstr int mce_severity_intel(struct mce *m, struct pt_regs *regs,
+ int tolerant, char **msg, bool is_excp)
{
enum exception excp = (is_excp ? EXCP_CONTEXT : NO_EXCP);
enum context ctx = error_context(m, regs);
@@ -407,8 +420,8 @@ static int mce_severity_intel(struct mce *m, struct pt_regs *regs,
}
}
-int mce_severity(struct mce *m, struct pt_regs *regs, int tolerant, char **msg,
- bool is_excp)
+int noinstr mce_severity(struct mce *m, struct pt_regs *regs, int tolerant, char **msg,
+ bool is_excp)
{
if (boot_cpu_data.x86_vendor == X86_VENDOR_AMD ||
boot_cpu_data.x86_vendor == X86_VENDOR_HYGON)
diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
index ff55df6..2a0f836 100644
--- a/arch/x86/kernel/cpu/mshyperv.c
+++ b/arch/x86/kernel/cpu/mshyperv.c
@@ -79,7 +79,7 @@ DEFINE_IDTENTRY_SYSVEC(sysvec_hyperv_stimer0)
inc_irq_stat(hyperv_stimer0_count);
if (hv_stimer0_handler)
hv_stimer0_handler();
- add_interrupt_randomness(HYPERV_STIMER0_VECTOR, 0);
+ add_interrupt_randomness(HYPERV_STIMER0_VECTOR);
ack_APIC_irq();
set_irq_regs(old_regs);
diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
index c9f0f3d..eaf25a2 100644
--- a/arch/x86/kernel/cpu/resctrl/monitor.c
+++ b/arch/x86/kernel/cpu/resctrl/monitor.c
@@ -282,7 +282,7 @@ static u64 mbm_overflow_count(u64 prev_msr, u64 cur_msr, unsigned int width)
u64 shift = 64 - width, chunks;
chunks = (cur_msr << shift) - (prev_msr << shift);
- return chunks >>= shift;
+ return chunks >> shift;
}
static u64 __mon_event_count(u32 rmid, struct rmid_read *rr)
diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
index 8471a8b..4b41efc 100644
--- a/arch/x86/kernel/cpu/sgx/main.c
+++ b/arch/x86/kernel/cpu/sgx/main.c
@@ -6,11 +6,13 @@
#include <linux/highmem.h>
#include <linux/kthread.h>
#include <linux/miscdevice.h>
+#include <linux/node.h>
#include <linux/pagemap.h>
#include <linux/ratelimit.h>
#include <linux/sched/mm.h>
#include <linux/sched/signal.h>
#include <linux/slab.h>
+#include <linux/sysfs.h>
#include <asm/sgx.h>
#include "driver.h"
#include "encl.h"
@@ -20,6 +22,7 @@ struct sgx_epc_section sgx_epc_sections[SGX_MAX_EPC_SECTIONS];
static int sgx_nr_epc_sections;
static struct task_struct *ksgxd_tsk;
static DECLARE_WAIT_QUEUE_HEAD(ksgxd_waitq);
+static DEFINE_XARRAY(sgx_epc_address_space);
/*
* These variables are part of the state of the reclaimer, and must be accessed
@@ -60,6 +63,24 @@ static void __sgx_sanitize_pages(struct list_head *dirty_page_list)
page = list_first_entry(dirty_page_list, struct sgx_epc_page, list);
+ /*
+ * Checking page->poison without holding the node->lock
+ * is racy, but losing the race (i.e. poison is set just
+ * after the check) just means __eremove() will be uselessly
+ * called for a page that sgx_free_epc_page() will put onto
+ * the node->sgx_poison_page_list later.
+ */
+ if (page->poison) {
+ struct sgx_epc_section *section = &sgx_epc_sections[page->section];
+ struct sgx_numa_node *node = section->node;
+
+ spin_lock(&node->lock);
+ list_move(&page->list, &node->sgx_poison_page_list);
+ spin_unlock(&node->lock);
+
+ continue;
+ }
+
ret = __eremove(sgx_get_epc_virt_addr(page));
if (!ret) {
/*
@@ -471,6 +492,7 @@ static struct sgx_epc_page *__sgx_alloc_epc_page_from_node(int nid)
page = list_first_entry(&node->free_page_list, struct sgx_epc_page, list);
list_del_init(&page->list);
+ page->flags = 0;
spin_unlock(&node->lock);
atomic_long_dec(&sgx_nr_free_pages);
@@ -624,7 +646,12 @@ void sgx_free_epc_page(struct sgx_epc_page *page)
spin_lock(&node->lock);
- list_add_tail(&page->list, &node->free_page_list);
+ page->owner = NULL;
+ if (page->poison)
+ list_add(&page->list, &node->sgx_poison_page_list);
+ else
+ list_add_tail(&page->list, &node->free_page_list);
+ page->flags = SGX_EPC_PAGE_IS_FREE;
spin_unlock(&node->lock);
atomic_long_inc(&sgx_nr_free_pages);
@@ -648,17 +675,102 @@ static bool __init sgx_setup_epc_section(u64 phys_addr, u64 size,
}
section->phys_addr = phys_addr;
+ xa_store_range(&sgx_epc_address_space, section->phys_addr,
+ phys_addr + size - 1, section, GFP_KERNEL);
for (i = 0; i < nr_pages; i++) {
section->pages[i].section = index;
section->pages[i].flags = 0;
section->pages[i].owner = NULL;
+ section->pages[i].poison = 0;
list_add_tail(§ion->pages[i].list, &sgx_dirty_page_list);
}
return true;
}
+bool arch_is_platform_page(u64 paddr)
+{
+ return !!xa_load(&sgx_epc_address_space, paddr);
+}
+EXPORT_SYMBOL_GPL(arch_is_platform_page);
+
+static struct sgx_epc_page *sgx_paddr_to_page(u64 paddr)
+{
+ struct sgx_epc_section *section;
+
+ section = xa_load(&sgx_epc_address_space, paddr);
+ if (!section)
+ return NULL;
+
+ return §ion->pages[PFN_DOWN(paddr - section->phys_addr)];
+}
+
+/*
+ * Called in process context to handle a hardware reported
+ * error in an SGX EPC page.
+ * If the MF_ACTION_REQUIRED bit is set in flags, then the
+ * context is the task that consumed the poison data. Otherwise
+ * this is called from a kernel thread unrelated to the page.
+ */
+int arch_memory_failure(unsigned long pfn, int flags)
+{
+ struct sgx_epc_page *page = sgx_paddr_to_page(pfn << PAGE_SHIFT);
+ struct sgx_epc_section *section;
+ struct sgx_numa_node *node;
+
+ /*
+ * mm/memory-failure.c calls this routine for all errors
+ * where there isn't a "struct page" for the address. But that
+ * includes other address ranges besides SGX.
+ */
+ if (!page)
+ return -ENXIO;
+
+ /*
+ * If poison was consumed synchronously. Send a SIGBUS to
+ * the task. Hardware has already exited the SGX enclave and
+ * will not allow re-entry to an enclave that has a memory
+ * error. The signal may help the task understand why the
+ * enclave is broken.
+ */
+ if (flags & MF_ACTION_REQUIRED)
+ force_sig(SIGBUS);
+
+ section = &sgx_epc_sections[page->section];
+ node = section->node;
+
+ spin_lock(&node->lock);
+
+ /* Already poisoned? Nothing more to do */
+ if (page->poison)
+ goto out;
+
+ page->poison = 1;
+
+ /*
+ * If the page is on a free list, move it to the per-node
+ * poison page list.
+ */
+ if (page->flags & SGX_EPC_PAGE_IS_FREE) {
+ list_move(&page->list, &node->sgx_poison_page_list);
+ goto out;
+ }
+
+ /*
+ * TBD: Add additional plumbing to enable pre-emptive
+ * action for asynchronous poison notification. Until
+ * then just hope that the poison:
+ * a) is not accessed - sgx_free_epc_page() will deal with it
+ * when the user gives it back
+ * b) results in a recoverable machine check rather than
+ * a fatal one
+ */
+out:
+ spin_unlock(&node->lock);
+ return 0;
+}
+
/**
* A section metric is concatenated in a way that @low bits 12-31 define the
* bits 12-31 of the metric and @high bits 0-19 define the bits 32-51 of the
@@ -670,6 +782,48 @@ static inline u64 __init sgx_calc_section_metric(u64 low, u64 high)
((high & GENMASK_ULL(19, 0)) << 32);
}
+#ifdef CONFIG_NUMA
+static ssize_t sgx_total_bytes_show(struct device *dev, struct device_attribute *attr, char *buf)
+{
+ return sysfs_emit(buf, "%lu\n", sgx_numa_nodes[dev->id].size);
+}
+static DEVICE_ATTR_RO(sgx_total_bytes);
+
+static umode_t arch_node_attr_is_visible(struct kobject *kobj,
+ struct attribute *attr, int idx)
+{
+ /* Make all x86/ attributes invisible when SGX is not initialized: */
+ if (nodes_empty(sgx_numa_mask))
+ return 0;
+
+ return attr->mode;
+}
+
+static struct attribute *arch_node_dev_attrs[] = {
+ &dev_attr_sgx_total_bytes.attr,
+ NULL,
+};
+
+const struct attribute_group arch_node_dev_group = {
+ .name = "x86",
+ .attrs = arch_node_dev_attrs,
+ .is_visible = arch_node_attr_is_visible,
+};
+
+static void __init arch_update_sysfs_visibility(int nid)
+{
+ struct node *node = node_devices[nid];
+ int ret;
+
+ ret = sysfs_update_group(&node->dev.kobj, &arch_node_dev_group);
+
+ if (ret)
+ pr_err("sysfs update failed (%d), files may be invisible", ret);
+}
+#else /* !CONFIG_NUMA */
+static void __init arch_update_sysfs_visibility(int nid) {}
+#endif
+
static bool __init sgx_page_cache_init(void)
{
u32 eax, ebx, ecx, edx, type;
@@ -713,10 +867,16 @@ static bool __init sgx_page_cache_init(void)
if (!node_isset(nid, sgx_numa_mask)) {
spin_lock_init(&sgx_numa_nodes[nid].lock);
INIT_LIST_HEAD(&sgx_numa_nodes[nid].free_page_list);
+ INIT_LIST_HEAD(&sgx_numa_nodes[nid].sgx_poison_page_list);
node_set(nid, sgx_numa_mask);
+ sgx_numa_nodes[nid].size = 0;
+
+ /* Make SGX-specific node sysfs files visible: */
+ arch_update_sysfs_visibility(nid);
}
sgx_epc_sections[i].node = &sgx_numa_nodes[nid];
+ sgx_numa_nodes[nid].size += size;
sgx_nr_epc_sections++;
}
diff --git a/arch/x86/kernel/cpu/sgx/sgx.h b/arch/x86/kernel/cpu/sgx/sgx.h
index 4628ace..0f17def 100644
--- a/arch/x86/kernel/cpu/sgx/sgx.h
+++ b/arch/x86/kernel/cpu/sgx/sgx.h
@@ -26,9 +26,13 @@
/* Pages, which are being tracked by the page reclaimer. */
#define SGX_EPC_PAGE_RECLAIMER_TRACKED BIT(0)
+/* Pages on free list */
+#define SGX_EPC_PAGE_IS_FREE BIT(1)
+
struct sgx_epc_page {
unsigned int section;
- unsigned int flags;
+ u16 flags;
+ u16 poison;
struct sgx_encl_page *owner;
struct list_head list;
};
@@ -39,6 +43,8 @@ struct sgx_epc_page {
*/
struct sgx_numa_node {
struct list_head free_page_list;
+ struct list_head sgx_poison_page_list;
+ unsigned long size;
spinlock_t lock;
};
diff --git a/arch/x86/kernel/early-quirks.c b/arch/x86/kernel/early-quirks.c
index 391a4e2b..fd2d3ab 100644
--- a/arch/x86/kernel/early-quirks.c
+++ b/arch/x86/kernel/early-quirks.c
@@ -554,6 +554,7 @@ static const struct pci_device_id intel_early_ids[] __initconst = {
INTEL_RKL_IDS(&gen11_early_ops),
INTEL_ADLS_IDS(&gen11_early_ops),
INTEL_ADLP_IDS(&gen11_early_ops),
+ INTEL_RPLS_IDS(&gen11_early_ops),
};
struct resource intel_graphics_stolen_res __ro_after_init = DEFINE_RES_MEM(0, 0);
diff --git a/arch/x86/kernel/fpu/core.c b/arch/x86/kernel/fpu/core.c
index 8ea306b..dd3777a 100644
--- a/arch/x86/kernel/fpu/core.c
+++ b/arch/x86/kernel/fpu/core.c
@@ -99,6 +99,19 @@ bool irq_fpu_usable(void)
EXPORT_SYMBOL(irq_fpu_usable);
/*
+ * Track AVX512 state use because it is known to slow the max clock
+ * speed of the core.
+ */
+static void update_avx_timestamp(struct fpu *fpu)
+{
+
+#define AVX512_TRACKING_MASK (XFEATURE_MASK_ZMM_Hi256 | XFEATURE_MASK_Hi16_ZMM)
+
+ if (fpu->fpstate->regs.xsave.header.xfeatures & AVX512_TRACKING_MASK)
+ fpu->avx512_timestamp = jiffies;
+}
+
+/*
* Save the FPU register state in fpu->fpstate->regs. The register state is
* preserved.
*
@@ -116,13 +129,7 @@ void save_fpregs_to_fpstate(struct fpu *fpu)
{
if (likely(use_xsave())) {
os_xsave(fpu->fpstate);
-
- /*
- * AVX512 state is tracked here because its use is
- * known to slow the max clock speed of the core.
- */
- if (fpu->fpstate->regs.xsave.header.xfeatures & XFEATURE_MASK_AVX512)
- fpu->avx512_timestamp = jiffies;
+ update_avx_timestamp(fpu);
return;
}
diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c
index fc5371a..de563db 100644
--- a/arch/x86/kernel/head64.c
+++ b/arch/x86/kernel/head64.c
@@ -126,6 +126,36 @@ static bool __head check_la57_support(unsigned long physaddr)
}
#endif
+static unsigned long sme_postprocess_startup(struct boot_params *bp, pmdval_t *pmd)
+{
+ unsigned long vaddr, vaddr_end;
+ int i;
+
+ /* Encrypt the kernel and related (if SME is active) */
+ sme_encrypt_kernel(bp);
+
+ /*
+ * Clear the memory encryption mask from the .bss..decrypted section.
+ * The bss section will be memset to zero later in the initialization so
+ * there is no need to zero it after changing the memory encryption
+ * attribute.
+ */
+ if (sme_get_me_mask()) {
+ vaddr = (unsigned long)__start_bss_decrypted;
+ vaddr_end = (unsigned long)__end_bss_decrypted;
+ for (; vaddr < vaddr_end; vaddr += PMD_SIZE) {
+ i = pmd_index(vaddr);
+ pmd[i] -= sme_get_me_mask();
+ }
+ }
+
+ /*
+ * Return the SME encryption mask (if SME is active) to be used as a
+ * modifier for the initial pgdir entry programmed into CR3.
+ */
+ return sme_get_me_mask();
+}
+
/* Code in __startup_64() can be relocated during execution, but the compiler
* doesn't have to generate PC-relative relocations when accessing globals from
* that function. Clang actually does not generate them, which leads to
@@ -135,7 +165,6 @@ static bool __head check_la57_support(unsigned long physaddr)
unsigned long __head __startup_64(unsigned long physaddr,
struct boot_params *bp)
{
- unsigned long vaddr, vaddr_end;
unsigned long load_delta, *p;
unsigned long pgtable_flags;
pgdval_t *pgd;
@@ -276,34 +305,7 @@ unsigned long __head __startup_64(unsigned long physaddr,
*/
*fixup_long(&phys_base, physaddr) += load_delta - sme_get_me_mask();
- /* Encrypt the kernel and related (if SME is active) */
- sme_encrypt_kernel(bp);
-
- /*
- * Clear the memory encryption mask from the .bss..decrypted section.
- * The bss section will be memset to zero later in the initialization so
- * there is no need to zero it after changing the memory encryption
- * attribute.
- *
- * This is early code, use an open coded check for SME instead of
- * using cc_platform_has(). This eliminates worries about removing
- * instrumentation or checking boot_cpu_data in the cc_platform_has()
- * function.
- */
- if (sme_get_me_mask()) {
- vaddr = (unsigned long)__start_bss_decrypted;
- vaddr_end = (unsigned long)__end_bss_decrypted;
- for (; vaddr < vaddr_end; vaddr += PMD_SIZE) {
- i = pmd_index(vaddr);
- pmd[i] -= sme_get_me_mask();
- }
- }
-
- /*
- * Return the SME encryption mask (if SME is active) to be used as a
- * modifier for the initial pgdir entry programmed into CR3.
- */
- return sme_get_me_mask();
+ return sme_postprocess_startup(bp, pmd);
}
unsigned long __startup_secondary_64(void)
@@ -485,6 +487,10 @@ asmlinkage __visible void __init x86_64_start_kernel(char * real_mode_data)
clear_bss();
+ /*
+ * This needs to happen *before* kasan_early_init() because latter maps stuff
+ * into that page.
+ */
clear_page(init_top_pgt);
/*
@@ -496,6 +502,16 @@ asmlinkage __visible void __init x86_64_start_kernel(char * real_mode_data)
kasan_early_init();
+ /*
+ * Flush global TLB entries which could be left over from the trampoline page
+ * table.
+ *
+ * This needs to happen *after* kasan_early_init() as KASAN-enabled .configs
+ * instrument native_write_cr4() so KASAN must be initialized for that
+ * instrumentation to work.
+ */
+ __native_tlb_flush_global(this_cpu_read(cpu_tlbstate.cr4));
+
idt_setup_early_handler();
copy_bootdata(__va(real_mode_data));
diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S
index d8b3ebd..9c63fc5 100644
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -166,9 +166,26 @@
call sev_verify_cbit
popq %rsi
- /* Switch to new page-table */
+ /*
+ * Switch to new page-table
+ *
+ * For the boot CPU this switches to early_top_pgt which still has the
+ * indentity mappings present. The secondary CPUs will switch to the
+ * init_top_pgt here, away from the trampoline_pgd and unmap the
+ * indentity mapped ranges.
+ */
movq %rax, %cr3
+ /*
+ * Do a global TLB flush after the CR3 switch to make sure the TLB
+ * entries from the identity mapping are flushed.
+ */
+ movq %cr4, %rcx
+ movq %rcx, %rax
+ xorq $X86_CR4_PGE, %rcx
+ movq %rcx, %cr4
+ movq %rax, %cr4
+
/* Ensure I am executing from virtual addresses */
movq $1f, %rax
ANNOTATE_RETPOLINE_SAFE
diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c
index 04143a6..5d48103 100644
--- a/arch/x86/kernel/process.c
+++ b/arch/x86/kernel/process.c
@@ -365,7 +365,7 @@ void arch_setup_new_exec(void)
clear_thread_flag(TIF_SSBD);
task_clear_spec_ssb_disable(current);
task_clear_spec_ssb_noexec(current);
- speculation_ctrl_update(task_thread_info(current)->flags);
+ speculation_ctrl_update(read_thread_flags());
}
}
@@ -617,7 +617,7 @@ static unsigned long speculation_ctrl_update_tif(struct task_struct *tsk)
clear_tsk_thread_flag(tsk, TIF_SPEC_IB);
}
/* Return the updated threadinfo flags*/
- return task_thread_info(tsk)->flags;
+ return read_task_thread_flags(tsk);
}
void speculation_ctrl_update(unsigned long tif)
@@ -653,8 +653,8 @@ void __switch_to_xtra(struct task_struct *prev_p, struct task_struct *next_p)
{
unsigned long tifp, tifn;
- tifn = READ_ONCE(task_thread_info(next_p)->flags);
- tifp = READ_ONCE(task_thread_info(prev_p)->flags);
+ tifn = read_task_thread_flags(next_p);
+ tifp = read_task_thread_flags(prev_p);
switch_to_bitmap(tifp);
diff --git a/arch/x86/kernel/process.h b/arch/x86/kernel/process.h
index 1d0797b..76b547b 100644
--- a/arch/x86/kernel/process.h
+++ b/arch/x86/kernel/process.h
@@ -13,8 +13,8 @@ void __switch_to_xtra(struct task_struct *prev_p, struct task_struct *next_p);
static inline void switch_to_extra(struct task_struct *prev,
struct task_struct *next)
{
- unsigned long next_tif = task_thread_info(next)->flags;
- unsigned long prev_tif = task_thread_info(prev)->flags;
+ unsigned long next_tif = read_task_thread_flags(next);
+ unsigned long prev_tif = read_task_thread_flags(prev);
if (IS_ENABLED(CONFIG_SMP)) {
/*
diff --git a/arch/x86/kernel/reboot.c b/arch/x86/kernel/reboot.c
index 0a40df6..fa700b4 100644
--- a/arch/x86/kernel/reboot.c
+++ b/arch/x86/kernel/reboot.c
@@ -113,17 +113,9 @@ void __noreturn machine_real_restart(unsigned int type)
spin_unlock(&rtc_lock);
/*
- * Switch back to the initial page table.
+ * Switch to the trampoline page table.
*/
-#ifdef CONFIG_X86_32
- load_cr3(initial_page_table);
-#else
- write_cr3(real_mode_header->trampoline_pgd);
-
- /* Exiting long mode will fail if CR4.PCIDE is set. */
- if (boot_cpu_has(X86_FEATURE_PCID))
- cr4_clear_bits(X86_CR4_PCIDE);
-#endif
+ load_trampoline_pgtable();
/* Jump to the identity-mapped low memory code */
#ifdef CONFIG_X86_32
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index e04f5e6..f7a132e 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -40,6 +40,7 @@
#include <asm/kasan.h>
#include <asm/kaslr.h>
#include <asm/mce.h>
+#include <asm/memtype.h>
#include <asm/mtrr.h>
#include <asm/realmode.h>
#include <asm/olpc_ofw.h>
@@ -967,7 +968,11 @@ void __init setup_arch(char **cmdline_p)
max_pfn = e820__end_of_ram_pfn();
/* update e820 for memory not covered by WB MTRRs */
- mtrr_bp_init();
+ if (IS_ENABLED(CONFIG_MTRR))
+ mtrr_bp_init();
+ else
+ pat_disable("PAT support disabled because CONFIG_MTRR is disabled in the kernel.");
+
if (mtrr_trim_uncached_memory(max_pfn))
max_pfn = e820__end_of_ram_pfn();
diff --git a/arch/x86/kernel/sev-shared.c b/arch/x86/kernel/sev-shared.c
index 787dc5f..ce98768 100644
--- a/arch/x86/kernel/sev-shared.c
+++ b/arch/x86/kernel/sev-shared.c
@@ -221,7 +221,7 @@ void __init do_vc_no_ghcb(struct pt_regs *regs, unsigned long exit_code)
fail:
/* Terminate the guest */
- sev_es_terminate(GHCB_SEV_ES_REASON_GENERAL_REQUEST);
+ sev_es_terminate(GHCB_SEV_ES_GEN_REQ);
}
static enum es_result vc_insn_string_read(struct es_em_ctxt *ctxt,
diff --git a/arch/x86/kernel/sev.c b/arch/x86/kernel/sev.c
index a9fc2ac..e6d316a 100644
--- a/arch/x86/kernel/sev.c
+++ b/arch/x86/kernel/sev.c
@@ -26,6 +26,7 @@
#include <asm/fpu/xcr.h>
#include <asm/processor.h>
#include <asm/realmode.h>
+#include <asm/setup.h>
#include <asm/traps.h>
#include <asm/svm.h>
#include <asm/smp.h>
@@ -86,9 +87,6 @@ struct ghcb_state {
static DEFINE_PER_CPU(struct sev_es_runtime_data*, runtime_data);
DEFINE_STATIC_KEY_FALSE(sev_es_enable_key);
-/* Needed in vc_early_forward_exception */
-void do_early_exception(struct pt_regs *regs, int trapnr);
-
static __always_inline bool on_vc_stack(struct pt_regs *regs)
{
unsigned long sp = regs->sp;
@@ -209,9 +207,6 @@ static noinstr struct ghcb *__sev_get_ghcb(struct ghcb_state *state)
return ghcb;
}
-/* Needed in vc_early_forward_exception */
-void do_early_exception(struct pt_regs *regs, int trapnr);
-
static inline u64 sev_es_rd_ghcb_msr(void)
{
return __rdmsr(MSR_AMD64_SEV_ES_GHCB);
@@ -797,22 +792,6 @@ static void __init vc_early_forward_exception(struct es_em_ctxt *ctxt)
do_early_exception(ctxt->regs, trapnr);
}
-static long *vc_insn_get_reg(struct es_em_ctxt *ctxt)
-{
- long *reg_array;
- int offset;
-
- reg_array = (long *)ctxt->regs;
- offset = insn_get_modrm_reg_off(&ctxt->insn, ctxt->regs);
-
- if (offset < 0)
- return NULL;
-
- offset /= sizeof(long);
-
- return reg_array + offset;
-}
-
static long *vc_insn_get_rm(struct es_em_ctxt *ctxt)
{
long *reg_array;
@@ -860,76 +839,6 @@ static enum es_result vc_do_mmio(struct ghcb *ghcb, struct es_em_ctxt *ctxt,
return sev_es_ghcb_hv_call(ghcb, true, ctxt, exit_code, exit_info_1, exit_info_2);
}
-static enum es_result vc_handle_mmio_twobyte_ops(struct ghcb *ghcb,
- struct es_em_ctxt *ctxt)
-{
- struct insn *insn = &ctxt->insn;
- unsigned int bytes = 0;
- enum es_result ret;
- int sign_byte;
- long *reg_data;
-
- switch (insn->opcode.bytes[1]) {
- /* MMIO Read w/ zero-extension */
- case 0xb6:
- bytes = 1;
- fallthrough;
- case 0xb7:
- if (!bytes)
- bytes = 2;
-
- ret = vc_do_mmio(ghcb, ctxt, bytes, true);
- if (ret)
- break;
-
- /* Zero extend based on operand size */
- reg_data = vc_insn_get_reg(ctxt);
- if (!reg_data)
- return ES_DECODE_FAILED;
-
- memset(reg_data, 0, insn->opnd_bytes);
-
- memcpy(reg_data, ghcb->shared_buffer, bytes);
- break;
-
- /* MMIO Read w/ sign-extension */
- case 0xbe:
- bytes = 1;
- fallthrough;
- case 0xbf:
- if (!bytes)
- bytes = 2;
-
- ret = vc_do_mmio(ghcb, ctxt, bytes, true);
- if (ret)
- break;
-
- /* Sign extend based on operand size */
- reg_data = vc_insn_get_reg(ctxt);
- if (!reg_data)
- return ES_DECODE_FAILED;
-
- if (bytes == 1) {
- u8 *val = (u8 *)ghcb->shared_buffer;
-
- sign_byte = (*val & 0x80) ? 0xff : 0x00;
- } else {
- u16 *val = (u16 *)ghcb->shared_buffer;
-
- sign_byte = (*val & 0x8000) ? 0xff : 0x00;
- }
- memset(reg_data, sign_byte, insn->opnd_bytes);
-
- memcpy(reg_data, ghcb->shared_buffer, bytes);
- break;
-
- default:
- ret = ES_UNSUPPORTED;
- }
-
- return ret;
-}
-
/*
* The MOVS instruction has two memory operands, which raises the
* problem that it is not known whether the access to the source or the
@@ -997,83 +906,79 @@ static enum es_result vc_handle_mmio_movs(struct es_em_ctxt *ctxt,
return ES_RETRY;
}
-static enum es_result vc_handle_mmio(struct ghcb *ghcb,
- struct es_em_ctxt *ctxt)
+static enum es_result vc_handle_mmio(struct ghcb *ghcb, struct es_em_ctxt *ctxt)
{
struct insn *insn = &ctxt->insn;
unsigned int bytes = 0;
+ enum mmio_type mmio;
enum es_result ret;
+ u8 sign_byte;
long *reg_data;
- switch (insn->opcode.bytes[0]) {
- /* MMIO Write */
- case 0x88:
- bytes = 1;
- fallthrough;
- case 0x89:
- if (!bytes)
- bytes = insn->opnd_bytes;
+ mmio = insn_decode_mmio(insn, &bytes);
+ if (mmio == MMIO_DECODE_FAILED)
+ return ES_DECODE_FAILED;
- reg_data = vc_insn_get_reg(ctxt);
+ if (mmio != MMIO_WRITE_IMM && mmio != MMIO_MOVS) {
+ reg_data = insn_get_modrm_reg_ptr(insn, ctxt->regs);
if (!reg_data)
return ES_DECODE_FAILED;
+ }
+ switch (mmio) {
+ case MMIO_WRITE:
memcpy(ghcb->shared_buffer, reg_data, bytes);
-
ret = vc_do_mmio(ghcb, ctxt, bytes, false);
break;
-
- case 0xc6:
- bytes = 1;
- fallthrough;
- case 0xc7:
- if (!bytes)
- bytes = insn->opnd_bytes;
-
+ case MMIO_WRITE_IMM:
memcpy(ghcb->shared_buffer, insn->immediate1.bytes, bytes);
-
ret = vc_do_mmio(ghcb, ctxt, bytes, false);
break;
-
- /* MMIO Read */
- case 0x8a:
- bytes = 1;
- fallthrough;
- case 0x8b:
- if (!bytes)
- bytes = insn->opnd_bytes;
-
+ case MMIO_READ:
ret = vc_do_mmio(ghcb, ctxt, bytes, true);
if (ret)
break;
- reg_data = vc_insn_get_reg(ctxt);
- if (!reg_data)
- return ES_DECODE_FAILED;
-
/* Zero-extend for 32-bit operation */
if (bytes == 4)
*reg_data = 0;
memcpy(reg_data, ghcb->shared_buffer, bytes);
break;
+ case MMIO_READ_ZERO_EXTEND:
+ ret = vc_do_mmio(ghcb, ctxt, bytes, true);
+ if (ret)
+ break;
- /* MOVS instruction */
- case 0xa4:
- bytes = 1;
- fallthrough;
- case 0xa5:
- if (!bytes)
- bytes = insn->opnd_bytes;
-
- ret = vc_handle_mmio_movs(ctxt, bytes);
+ /* Zero extend based on operand size */
+ memset(reg_data, 0, insn->opnd_bytes);
+ memcpy(reg_data, ghcb->shared_buffer, bytes);
break;
- /* Two-Byte Opcodes */
- case 0x0f:
- ret = vc_handle_mmio_twobyte_ops(ghcb, ctxt);
+ case MMIO_READ_SIGN_EXTEND:
+ ret = vc_do_mmio(ghcb, ctxt, bytes, true);
+ if (ret)
+ break;
+
+ if (bytes == 1) {
+ u8 *val = (u8 *)ghcb->shared_buffer;
+
+ sign_byte = (*val & 0x80) ? 0xff : 0x00;
+ } else {
+ u16 *val = (u16 *)ghcb->shared_buffer;
+
+ sign_byte = (*val & 0x8000) ? 0xff : 0x00;
+ }
+
+ /* Sign extend based on operand size */
+ memset(reg_data, sign_byte, insn->opnd_bytes);
+ memcpy(reg_data, ghcb->shared_buffer, bytes);
+ break;
+ case MMIO_MOVS:
+ ret = vc_handle_mmio_movs(ctxt, bytes);
break;
default:
ret = ES_UNSUPPORTED;
+ break;
}
return ret;
@@ -1432,7 +1337,7 @@ DEFINE_IDTENTRY_VC_KERNEL(exc_vmm_communication)
show_regs(regs);
/* Ask hypervisor to sev_es_terminate */
- sev_es_terminate(GHCB_SEV_ES_REASON_GENERAL_REQUEST);
+ sev_es_terminate(GHCB_SEV_ES_GEN_REQ);
/* If that fails and we get here - just panic */
panic("Returned from Terminate-Request to Hypervisor\n");
@@ -1480,7 +1385,7 @@ bool __init handle_vc_boot_ghcb(struct pt_regs *regs)
/* Do initial setup or terminate the guest */
if (unlikely(boot_ghcb == NULL && !sev_es_setup_ghcb()))
- sev_es_terminate(GHCB_SEV_ES_REASON_GENERAL_REQUEST);
+ sev_es_terminate(GHCB_SEV_ES_GEN_REQ);
vc_ghcb_invalidate(boot_ghcb);
diff --git a/arch/x86/kvm/debugfs.c b/arch/x86/kvm/debugfs.c
index 54a83a7..f33c804 100644
--- a/arch/x86/kvm/debugfs.c
+++ b/arch/x86/kvm/debugfs.c
@@ -95,6 +95,9 @@ static int kvm_mmu_rmaps_stat_show(struct seq_file *m, void *v)
unsigned int *log[KVM_NR_PAGE_SIZES], *cur;
int i, j, k, l, ret;
+ if (!kvm_memslots_have_rmaps(kvm))
+ return 0;
+
ret = -ENOMEM;
memset(log, 0, sizeof(log));
for (i = 0; i < KVM_NR_PAGE_SIZES; i++) {
diff --git a/arch/x86/kvm/mmu/spte.c b/arch/x86/kvm/mmu/spte.c
index 0c76c45..fad546d 100644
--- a/arch/x86/kvm/mmu/spte.c
+++ b/arch/x86/kvm/mmu/spte.c
@@ -16,6 +16,7 @@
#include "spte.h"
#include <asm/e820/api.h>
+#include <asm/memtype.h>
#include <asm/vmx.h>
static bool __read_mostly enable_mmio_caching = true;
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 7656a2c..be28831 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -1565,7 +1565,7 @@ static int sev_lock_two_vms(struct kvm *dst_kvm, struct kvm *src_kvm)
r = -EINTR;
if (mutex_lock_killable(&dst_kvm->lock))
goto release_src;
- if (mutex_lock_killable(&src_kvm->lock))
+ if (mutex_lock_killable_nested(&src_kvm->lock, SINGLE_DEPTH_NESTING))
goto unlock_dst;
return 0;
diff --git a/arch/x86/lib/copy_user_64.S b/arch/x86/lib/copy_user_64.S
index 2797e63..a2cbeae 100644
--- a/arch/x86/lib/copy_user_64.S
+++ b/arch/x86/lib/copy_user_64.S
@@ -200,8 +200,8 @@
*/
SYM_FUNC_START(copy_user_enhanced_fast_string)
ASM_STAC
- cmpl $64,%edx
- jb .L_copy_short_string /* less then 64 bytes, avoid the costly 'rep' */
+ /* CPUs without FSRM should avoid rep movsb for short copies */
+ ALTERNATIVE "cmpl $64, %edx; jb .L_copy_short_string", "", X86_FEATURE_FSRM
movl %edx,%ecx
1: rep
movsb
@@ -225,6 +225,7 @@
* Don't try to copy the tail if machine check happened
*
* Input:
+ * eax trap number written by ex_handler_copy()
* rdi destination
* rsi source
* rdx count
@@ -233,12 +234,20 @@
* eax uncopied bytes or 0 if successful.
*/
SYM_CODE_START_LOCAL(.Lcopy_user_handle_tail)
+ cmp $X86_TRAP_MC,%eax
+ je 3f
+
movl %edx,%ecx
1: rep movsb
2: mov %ecx,%eax
ASM_CLAC
ret
+3:
+ movl %edx,%eax
+ ASM_CLAC
+ RET
+
_ASM_EXTABLE_CPY(1b, 2b)
SYM_CODE_END(.Lcopy_user_handle_tail)
diff --git a/arch/x86/lib/insn-eval.c b/arch/x86/lib/insn-eval.c
index eb3ccff..53e57ef 100644
--- a/arch/x86/lib/insn-eval.c
+++ b/arch/x86/lib/insn-eval.c
@@ -37,8 +37,6 @@ enum reg_type {
*/
static bool is_string_insn(struct insn *insn)
{
- insn_get_opcode(insn);
-
/* All string instructions have a 1-byte opcode. */
if (insn->opcode.nbytes != 1)
return false;
@@ -851,6 +849,26 @@ int insn_get_modrm_reg_off(struct insn *insn, struct pt_regs *regs)
}
/**
+ * insn_get_modrm_reg_ptr() - Obtain register pointer based on ModRM byte
+ * @insn: Instruction containing the ModRM byte
+ * @regs: Register values as seen when entering kernel mode
+ *
+ * Returns:
+ *
+ * The register indicated by the reg part of the ModRM byte.
+ * The register is obtained as a pointer within pt_regs.
+ */
+unsigned long *insn_get_modrm_reg_ptr(struct insn *insn, struct pt_regs *regs)
+{
+ int offset;
+
+ offset = insn_get_modrm_reg_off(insn, regs);
+ if (offset < 0)
+ return NULL;
+ return (void *)regs + offset;
+}
+
+/**
* get_seg_base_limit() - obtain base address and limit of a segment
* @insn: Instruction. Must be valid.
* @regs: Register values as seen when entering kernel mode
@@ -1405,6 +1423,9 @@ void __user *insn_get_addr_ref(struct insn *insn, struct pt_regs *regs)
if (!insn || !regs)
return (void __user *)-1L;
+ if (insn_get_opcode(insn))
+ return (void __user *)-1L;
+
switch (insn->addr_bytes) {
case 2:
return get_addr_ref_16(insn, regs);
@@ -1539,3 +1560,87 @@ bool insn_decode_from_regs(struct insn *insn, struct pt_regs *regs,
return true;
}
+
+/**
+ * insn_decode_mmio() - Decode a MMIO instruction
+ * @insn: Structure to store decoded instruction
+ * @bytes: Returns size of memory operand
+ *
+ * Decodes instruction that used for Memory-mapped I/O.
+ *
+ * Returns:
+ *
+ * Type of the instruction. Size of the memory operand is stored in
+ * @bytes. If decode failed, MMIO_DECODE_FAILED returned.
+ */
+enum mmio_type insn_decode_mmio(struct insn *insn, int *bytes)
+{
+ enum mmio_type type = MMIO_DECODE_FAILED;
+
+ *bytes = 0;
+
+ if (insn_get_opcode(insn))
+ return MMIO_DECODE_FAILED;
+
+ switch (insn->opcode.bytes[0]) {
+ case 0x88: /* MOV m8,r8 */
+ *bytes = 1;
+ fallthrough;
+ case 0x89: /* MOV m16/m32/m64, r16/m32/m64 */
+ if (!*bytes)
+ *bytes = insn->opnd_bytes;
+ type = MMIO_WRITE;
+ break;
+
+ case 0xc6: /* MOV m8, imm8 */
+ *bytes = 1;
+ fallthrough;
+ case 0xc7: /* MOV m16/m32/m64, imm16/imm32/imm64 */
+ if (!*bytes)
+ *bytes = insn->opnd_bytes;
+ type = MMIO_WRITE_IMM;
+ break;
+
+ case 0x8a: /* MOV r8, m8 */
+ *bytes = 1;
+ fallthrough;
+ case 0x8b: /* MOV r16/r32/r64, m16/m32/m64 */
+ if (!*bytes)
+ *bytes = insn->opnd_bytes;
+ type = MMIO_READ;
+ break;
+
+ case 0xa4: /* MOVS m8, m8 */
+ *bytes = 1;
+ fallthrough;
+ case 0xa5: /* MOVS m16/m32/m64, m16/m32/m64 */
+ if (!*bytes)
+ *bytes = insn->opnd_bytes;
+ type = MMIO_MOVS;
+ break;
+
+ case 0x0f: /* Two-byte instruction */
+ switch (insn->opcode.bytes[1]) {
+ case 0xb6: /* MOVZX r16/r32/r64, m8 */
+ *bytes = 1;
+ fallthrough;
+ case 0xb7: /* MOVZX r32/r64, m16 */
+ if (!*bytes)
+ *bytes = 2;
+ type = MMIO_READ_ZERO_EXTEND;
+ break;
+
+ case 0xbe: /* MOVSX r16/r32/r64, m8 */
+ *bytes = 1;
+ fallthrough;
+ case 0xbf: /* MOVSX r32/r64, m16 */
+ if (!*bytes)
+ *bytes = 2;
+ type = MMIO_READ_SIGN_EXTEND;
+ break;
+ }
+ break;
+ }
+
+ return type;
+}
diff --git a/arch/x86/mm/Makefile b/arch/x86/mm/Makefile
index 5864219..fe3d306 100644
--- a/arch/x86/mm/Makefile
+++ b/arch/x86/mm/Makefile
@@ -2,9 +2,11 @@
# Kernel does not boot with instrumentation of tlb.c and mem_encrypt*.c
KCOV_INSTRUMENT_tlb.o := n
KCOV_INSTRUMENT_mem_encrypt.o := n
+KCOV_INSTRUMENT_mem_encrypt_amd.o := n
KCOV_INSTRUMENT_mem_encrypt_identity.o := n
KASAN_SANITIZE_mem_encrypt.o := n
+KASAN_SANITIZE_mem_encrypt_amd.o := n
KASAN_SANITIZE_mem_encrypt_identity.o := n
# Disable KCSAN entirely, because otherwise we get warnings that some functions
@@ -13,6 +15,7 @@
ifdef CONFIG_FUNCTION_TRACER
CFLAGS_REMOVE_mem_encrypt.o = -pg
+CFLAGS_REMOVE_mem_encrypt_amd.o = -pg
CFLAGS_REMOVE_mem_encrypt_identity.o = -pg
endif
@@ -52,6 +55,8 @@
obj-$(CONFIG_RANDOMIZE_MEMORY) += kaslr.o
obj-$(CONFIG_PAGE_TABLE_ISOLATION) += pti.o
-obj-$(CONFIG_AMD_MEM_ENCRYPT) += mem_encrypt.o
+obj-$(CONFIG_X86_MEM_ENCRYPT) += mem_encrypt.o
+obj-$(CONFIG_AMD_MEM_ENCRYPT) += mem_encrypt_amd.o
+
obj-$(CONFIG_AMD_MEM_ENCRYPT) += mem_encrypt_identity.o
obj-$(CONFIG_AMD_MEM_ENCRYPT) += mem_encrypt_boot.o
diff --git a/arch/x86/mm/init.c b/arch/x86/mm/init.c
index 1895986..4ba024d 100644
--- a/arch/x86/mm/init.c
+++ b/arch/x86/mm/init.c
@@ -714,6 +714,11 @@ static void __init memory_map_bottom_up(unsigned long map_start,
static void __init init_trampoline(void)
{
#ifdef CONFIG_X86_64
+ /*
+ * The code below will alias kernel page-tables in the user-range of the
+ * address space, including the Global bit. So global TLB entries will
+ * be created when using the trampoline page-table.
+ */
if (!kaslr_memory_enabled())
trampoline_pgd_entry = init_top_pgt[pgd_index(__PAGE_OFFSET)];
else
diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c
index 3609822..96d34eb 100644
--- a/arch/x86/mm/init_64.c
+++ b/arch/x86/mm/init_64.c
@@ -981,7 +981,7 @@ static void __meminit free_pagetable(struct page *page, int order)
if (PageReserved(page)) {
__ClearPageReserved(page);
- magic = (unsigned long)page->freelist;
+ magic = page->index;
if (magic == SECTION_INFO || magic == MIX_SECTION_INFO) {
while (nr_pages--)
put_page_bootmem(page++);
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index 3548730..50d2099 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -1,419 +1,18 @@
// SPDX-License-Identifier: GPL-2.0-only
/*
- * AMD Memory Encryption Support
+ * Memory Encryption Support Common Code
*
* Copyright (C) 2016 Advanced Micro Devices, Inc.
*
* Author: Tom Lendacky <thomas.lendacky@amd.com>
*/
-#define DISABLE_BRANCH_PROFILING
-
-#include <linux/linkage.h>
-#include <linux/init.h>
-#include <linux/mm.h>
#include <linux/dma-direct.h>
-#include <linux/swiotlb.h>
-#include <linux/mem_encrypt.h>
-#include <linux/device.h>
-#include <linux/kernel.h>
-#include <linux/bitops.h>
#include <linux/dma-mapping.h>
-#include <linux/virtio_config.h>
+#include <linux/swiotlb.h>
#include <linux/cc_platform.h>
-
-#include <asm/tlbflush.h>
-#include <asm/fixmap.h>
-#include <asm/setup.h>
-#include <asm/bootparam.h>
-#include <asm/set_memory.h>
-#include <asm/cacheflush.h>
-#include <asm/processor-flags.h>
-#include <asm/msr.h>
-#include <asm/cmdline.h>
-
-#include "mm_internal.h"
-
-/*
- * Since SME related variables are set early in the boot process they must
- * reside in the .data section so as not to be zeroed out when the .bss
- * section is later cleared.
- */
-u64 sme_me_mask __section(".data") = 0;
-u64 sev_status __section(".data") = 0;
-u64 sev_check_data __section(".data") = 0;
-EXPORT_SYMBOL(sme_me_mask);
-DEFINE_STATIC_KEY_FALSE(sev_enable_key);
-EXPORT_SYMBOL_GPL(sev_enable_key);
-
-/* Buffer used for early in-place encryption by BSP, no locking needed */
-static char sme_early_buffer[PAGE_SIZE] __initdata __aligned(PAGE_SIZE);
-
-/*
- * This routine does not change the underlying encryption setting of the
- * page(s) that map this memory. It assumes that eventually the memory is
- * meant to be accessed as either encrypted or decrypted but the contents
- * are currently not in the desired state.
- *
- * This routine follows the steps outlined in the AMD64 Architecture
- * Programmer's Manual Volume 2, Section 7.10.8 Encrypt-in-Place.
- */
-static void __init __sme_early_enc_dec(resource_size_t paddr,
- unsigned long size, bool enc)
-{
- void *src, *dst;
- size_t len;
-
- if (!sme_me_mask)
- return;
-
- wbinvd();
-
- /*
- * There are limited number of early mapping slots, so map (at most)
- * one page at time.
- */
- while (size) {
- len = min_t(size_t, sizeof(sme_early_buffer), size);
-
- /*
- * Create mappings for the current and desired format of
- * the memory. Use a write-protected mapping for the source.
- */
- src = enc ? early_memremap_decrypted_wp(paddr, len) :
- early_memremap_encrypted_wp(paddr, len);
-
- dst = enc ? early_memremap_encrypted(paddr, len) :
- early_memremap_decrypted(paddr, len);
-
- /*
- * If a mapping can't be obtained to perform the operation,
- * then eventual access of that area in the desired mode
- * will cause a crash.
- */
- BUG_ON(!src || !dst);
-
- /*
- * Use a temporary buffer, of cache-line multiple size, to
- * avoid data corruption as documented in the APM.
- */
- memcpy(sme_early_buffer, src, len);
- memcpy(dst, sme_early_buffer, len);
-
- early_memunmap(dst, len);
- early_memunmap(src, len);
-
- paddr += len;
- size -= len;
- }
-}
-
-void __init sme_early_encrypt(resource_size_t paddr, unsigned long size)
-{
- __sme_early_enc_dec(paddr, size, true);
-}
-
-void __init sme_early_decrypt(resource_size_t paddr, unsigned long size)
-{
- __sme_early_enc_dec(paddr, size, false);
-}
-
-static void __init __sme_early_map_unmap_mem(void *vaddr, unsigned long size,
- bool map)
-{
- unsigned long paddr = (unsigned long)vaddr - __PAGE_OFFSET;
- pmdval_t pmd_flags, pmd;
-
- /* Use early_pmd_flags but remove the encryption mask */
- pmd_flags = __sme_clr(early_pmd_flags);
-
- do {
- pmd = map ? (paddr & PMD_MASK) + pmd_flags : 0;
- __early_make_pgtable((unsigned long)vaddr, pmd);
-
- vaddr += PMD_SIZE;
- paddr += PMD_SIZE;
- size = (size <= PMD_SIZE) ? 0 : size - PMD_SIZE;
- } while (size);
-
- flush_tlb_local();
-}
-
-void __init sme_unmap_bootdata(char *real_mode_data)
-{
- struct boot_params *boot_data;
- unsigned long cmdline_paddr;
-
- if (!cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT))
- return;
-
- /* Get the command line address before unmapping the real_mode_data */
- boot_data = (struct boot_params *)real_mode_data;
- cmdline_paddr = boot_data->hdr.cmd_line_ptr | ((u64)boot_data->ext_cmd_line_ptr << 32);
-
- __sme_early_map_unmap_mem(real_mode_data, sizeof(boot_params), false);
-
- if (!cmdline_paddr)
- return;
-
- __sme_early_map_unmap_mem(__va(cmdline_paddr), COMMAND_LINE_SIZE, false);
-}
-
-void __init sme_map_bootdata(char *real_mode_data)
-{
- struct boot_params *boot_data;
- unsigned long cmdline_paddr;
-
- if (!cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT))
- return;
-
- __sme_early_map_unmap_mem(real_mode_data, sizeof(boot_params), true);
-
- /* Get the command line address after mapping the real_mode_data */
- boot_data = (struct boot_params *)real_mode_data;
- cmdline_paddr = boot_data->hdr.cmd_line_ptr | ((u64)boot_data->ext_cmd_line_ptr << 32);
-
- if (!cmdline_paddr)
- return;
-
- __sme_early_map_unmap_mem(__va(cmdline_paddr), COMMAND_LINE_SIZE, true);
-}
-
-void __init sme_early_init(void)
-{
- unsigned int i;
-
- if (!sme_me_mask)
- return;
-
- early_pmd_flags = __sme_set(early_pmd_flags);
-
- __supported_pte_mask = __sme_set(__supported_pte_mask);
-
- /* Update the protection map with memory encryption mask */
- for (i = 0; i < ARRAY_SIZE(protection_map); i++)
- protection_map[i] = pgprot_encrypted(protection_map[i]);
-
- if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT))
- swiotlb_force = SWIOTLB_FORCE;
-}
-
-void __init sev_setup_arch(void)
-{
- phys_addr_t total_mem = memblock_phys_mem_size();
- unsigned long size;
-
- if (!cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT))
- return;
-
- /*
- * For SEV, all DMA has to occur via shared/unencrypted pages.
- * SEV uses SWIOTLB to make this happen without changing device
- * drivers. However, depending on the workload being run, the
- * default 64MB of SWIOTLB may not be enough and SWIOTLB may
- * run out of buffers for DMA, resulting in I/O errors and/or
- * performance degradation especially with high I/O workloads.
- *
- * Adjust the default size of SWIOTLB for SEV guests using
- * a percentage of guest memory for SWIOTLB buffers.
- * Also, as the SWIOTLB bounce buffer memory is allocated
- * from low memory, ensure that the adjusted size is within
- * the limits of low available memory.
- *
- * The percentage of guest memory used here for SWIOTLB buffers
- * is more of an approximation of the static adjustment which
- * 64MB for <1G, and ~128M to 256M for 1G-to-4G, i.e., the 6%
- */
- size = total_mem * 6 / 100;
- size = clamp_val(size, IO_TLB_DEFAULT_SIZE, SZ_1G);
- swiotlb_adjust_size(size);
-}
-
-static unsigned long pg_level_to_pfn(int level, pte_t *kpte, pgprot_t *ret_prot)
-{
- unsigned long pfn = 0;
- pgprot_t prot;
-
- switch (level) {
- case PG_LEVEL_4K:
- pfn = pte_pfn(*kpte);
- prot = pte_pgprot(*kpte);
- break;
- case PG_LEVEL_2M:
- pfn = pmd_pfn(*(pmd_t *)kpte);
- prot = pmd_pgprot(*(pmd_t *)kpte);
- break;
- case PG_LEVEL_1G:
- pfn = pud_pfn(*(pud_t *)kpte);
- prot = pud_pgprot(*(pud_t *)kpte);
- break;
- default:
- WARN_ONCE(1, "Invalid level for kpte\n");
- return 0;
- }
-
- if (ret_prot)
- *ret_prot = prot;
-
- return pfn;
-}
-
-void notify_range_enc_status_changed(unsigned long vaddr, int npages, bool enc)
-{
-#ifdef CONFIG_PARAVIRT
- unsigned long sz = npages << PAGE_SHIFT;
- unsigned long vaddr_end = vaddr + sz;
-
- while (vaddr < vaddr_end) {
- int psize, pmask, level;
- unsigned long pfn;
- pte_t *kpte;
-
- kpte = lookup_address(vaddr, &level);
- if (!kpte || pte_none(*kpte)) {
- WARN_ONCE(1, "kpte lookup for vaddr\n");
- return;
- }
-
- pfn = pg_level_to_pfn(level, kpte, NULL);
- if (!pfn)
- continue;
-
- psize = page_level_size(level);
- pmask = page_level_mask(level);
-
- notify_page_enc_status_changed(pfn, psize >> PAGE_SHIFT, enc);
-
- vaddr = (vaddr & pmask) + psize;
- }
-#endif
-}
-
-static void __init __set_clr_pte_enc(pte_t *kpte, int level, bool enc)
-{
- pgprot_t old_prot, new_prot;
- unsigned long pfn, pa, size;
- pte_t new_pte;
-
- pfn = pg_level_to_pfn(level, kpte, &old_prot);
- if (!pfn)
- return;
-
- new_prot = old_prot;
- if (enc)
- pgprot_val(new_prot) |= _PAGE_ENC;
- else
- pgprot_val(new_prot) &= ~_PAGE_ENC;
-
- /* If prot is same then do nothing. */
- if (pgprot_val(old_prot) == pgprot_val(new_prot))
- return;
-
- pa = pfn << PAGE_SHIFT;
- size = page_level_size(level);
-
- /*
- * We are going to perform in-place en-/decryption and change the
- * physical page attribute from C=1 to C=0 or vice versa. Flush the
- * caches to ensure that data gets accessed with the correct C-bit.
- */
- clflush_cache_range(__va(pa), size);
-
- /* Encrypt/decrypt the contents in-place */
- if (enc)
- sme_early_encrypt(pa, size);
- else
- sme_early_decrypt(pa, size);
-
- /* Change the page encryption mask. */
- new_pte = pfn_pte(pfn, new_prot);
- set_pte_atomic(kpte, new_pte);
-}
-
-static int __init early_set_memory_enc_dec(unsigned long vaddr,
- unsigned long size, bool enc)
-{
- unsigned long vaddr_end, vaddr_next, start;
- unsigned long psize, pmask;
- int split_page_size_mask;
- int level, ret;
- pte_t *kpte;
-
- start = vaddr;
- vaddr_next = vaddr;
- vaddr_end = vaddr + size;
-
- for (; vaddr < vaddr_end; vaddr = vaddr_next) {
- kpte = lookup_address(vaddr, &level);
- if (!kpte || pte_none(*kpte)) {
- ret = 1;
- goto out;
- }
-
- if (level == PG_LEVEL_4K) {
- __set_clr_pte_enc(kpte, level, enc);
- vaddr_next = (vaddr & PAGE_MASK) + PAGE_SIZE;
- continue;
- }
-
- psize = page_level_size(level);
- pmask = page_level_mask(level);
-
- /*
- * Check whether we can change the large page in one go.
- * We request a split when the address is not aligned and
- * the number of pages to set/clear encryption bit is smaller
- * than the number of pages in the large page.
- */
- if (vaddr == (vaddr & pmask) &&
- ((vaddr_end - vaddr) >= psize)) {
- __set_clr_pte_enc(kpte, level, enc);
- vaddr_next = (vaddr & pmask) + psize;
- continue;
- }
-
- /*
- * The virtual address is part of a larger page, create the next
- * level page table mapping (4K or 2M). If it is part of a 2M
- * page then we request a split of the large page into 4K
- * chunks. A 1GB large page is split into 2M pages, resp.
- */
- if (level == PG_LEVEL_2M)
- split_page_size_mask = 0;
- else
- split_page_size_mask = 1 << PG_LEVEL_2M;
-
- /*
- * kernel_physical_mapping_change() does not flush the TLBs, so
- * a TLB flush is required after we exit from the for loop.
- */
- kernel_physical_mapping_change(__pa(vaddr & pmask),
- __pa((vaddr_end & pmask) + psize),
- split_page_size_mask);
- }
-
- ret = 0;
-
- notify_range_enc_status_changed(start, PAGE_ALIGN(size) >> PAGE_SHIFT, enc);
-out:
- __flush_tlb_all();
- return ret;
-}
-
-int __init early_set_memory_decrypted(unsigned long vaddr, unsigned long size)
-{
- return early_set_memory_enc_dec(vaddr, size, false);
-}
-
-int __init early_set_memory_encrypted(unsigned long vaddr, unsigned long size)
-{
- return early_set_memory_enc_dec(vaddr, size, true);
-}
-
-void __init early_set_mem_enc_dec_hypercall(unsigned long vaddr, int npages, bool enc)
-{
- notify_range_enc_status_changed(vaddr, npages, enc);
-}
+#include <linux/mem_encrypt.h>
+#include <linux/virtio_config.h>
/* Override for DMA direct allocation check - ARCH_HAS_FORCE_DMA_UNENCRYPTED */
bool force_dma_unencrypted(struct device *dev)
@@ -441,30 +40,6 @@ bool force_dma_unencrypted(struct device *dev)
return false;
}
-void __init mem_encrypt_free_decrypted_mem(void)
-{
- unsigned long vaddr, vaddr_end, npages;
- int r;
-
- vaddr = (unsigned long)__start_bss_decrypted_unused;
- vaddr_end = (unsigned long)__end_bss_decrypted;
- npages = (vaddr_end - vaddr) >> PAGE_SHIFT;
-
- /*
- * The unused memory range was mapped decrypted, change the encryption
- * attribute from decrypted to encrypted before freeing it.
- */
- if (cc_platform_has(CC_ATTR_MEM_ENCRYPT)) {
- r = set_memory_encrypted(vaddr, npages);
- if (r) {
- pr_warn("failed to free unused decrypted pages\n");
- return;
- }
- }
-
- free_init_pages("unused decrypted", vaddr, vaddr_end);
-}
-
static void print_mem_encrypt_feature_info(void)
{
pr_info("AMD Memory Encryption Features active:");
@@ -493,20 +68,12 @@ static void print_mem_encrypt_feature_info(void)
/* Architecture __weak replacement functions */
void __init mem_encrypt_init(void)
{
- if (!sme_me_mask)
+ if (!cc_platform_has(CC_ATTR_MEM_ENCRYPT))
return;
/* Call into SWIOTLB to update the SWIOTLB DMA buffers */
swiotlb_update_mem_attributes();
- /*
- * With SEV, we need to unroll the rep string I/O instructions,
- * but SEV-ES supports them through the #VC handler.
- */
- if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT) &&
- !cc_platform_has(CC_ATTR_GUEST_STATE_ENCRYPT))
- static_branch_enable(&sev_enable_key);
-
print_mem_encrypt_feature_info();
}
diff --git a/arch/x86/mm/mem_encrypt_amd.c b/arch/x86/mm/mem_encrypt_amd.c
new file mode 100644
index 0000000..2b2d018
--- /dev/null
+++ b/arch/x86/mm/mem_encrypt_amd.c
@@ -0,0 +1,438 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * AMD Memory Encryption Support
+ *
+ * Copyright (C) 2016 Advanced Micro Devices, Inc.
+ *
+ * Author: Tom Lendacky <thomas.lendacky@amd.com>
+ */
+
+#define DISABLE_BRANCH_PROFILING
+
+#include <linux/linkage.h>
+#include <linux/init.h>
+#include <linux/mm.h>
+#include <linux/dma-direct.h>
+#include <linux/swiotlb.h>
+#include <linux/mem_encrypt.h>
+#include <linux/device.h>
+#include <linux/kernel.h>
+#include <linux/bitops.h>
+#include <linux/dma-mapping.h>
+#include <linux/virtio_config.h>
+#include <linux/cc_platform.h>
+
+#include <asm/tlbflush.h>
+#include <asm/fixmap.h>
+#include <asm/setup.h>
+#include <asm/bootparam.h>
+#include <asm/set_memory.h>
+#include <asm/cacheflush.h>
+#include <asm/processor-flags.h>
+#include <asm/msr.h>
+#include <asm/cmdline.h>
+
+#include "mm_internal.h"
+
+/*
+ * Since SME related variables are set early in the boot process they must
+ * reside in the .data section so as not to be zeroed out when the .bss
+ * section is later cleared.
+ */
+u64 sme_me_mask __section(".data") = 0;
+u64 sev_status __section(".data") = 0;
+u64 sev_check_data __section(".data") = 0;
+EXPORT_SYMBOL(sme_me_mask);
+
+/* Buffer used for early in-place encryption by BSP, no locking needed */
+static char sme_early_buffer[PAGE_SIZE] __initdata __aligned(PAGE_SIZE);
+
+/*
+ * This routine does not change the underlying encryption setting of the
+ * page(s) that map this memory. It assumes that eventually the memory is
+ * meant to be accessed as either encrypted or decrypted but the contents
+ * are currently not in the desired state.
+ *
+ * This routine follows the steps outlined in the AMD64 Architecture
+ * Programmer's Manual Volume 2, Section 7.10.8 Encrypt-in-Place.
+ */
+static void __init __sme_early_enc_dec(resource_size_t paddr,
+ unsigned long size, bool enc)
+{
+ void *src, *dst;
+ size_t len;
+
+ if (!sme_me_mask)
+ return;
+
+ wbinvd();
+
+ /*
+ * There are limited number of early mapping slots, so map (at most)
+ * one page at time.
+ */
+ while (size) {
+ len = min_t(size_t, sizeof(sme_early_buffer), size);
+
+ /*
+ * Create mappings for the current and desired format of
+ * the memory. Use a write-protected mapping for the source.
+ */
+ src = enc ? early_memremap_decrypted_wp(paddr, len) :
+ early_memremap_encrypted_wp(paddr, len);
+
+ dst = enc ? early_memremap_encrypted(paddr, len) :
+ early_memremap_decrypted(paddr, len);
+
+ /*
+ * If a mapping can't be obtained to perform the operation,
+ * then eventual access of that area in the desired mode
+ * will cause a crash.
+ */
+ BUG_ON(!src || !dst);
+
+ /*
+ * Use a temporary buffer, of cache-line multiple size, to
+ * avoid data corruption as documented in the APM.
+ */
+ memcpy(sme_early_buffer, src, len);
+ memcpy(dst, sme_early_buffer, len);
+
+ early_memunmap(dst, len);
+ early_memunmap(src, len);
+
+ paddr += len;
+ size -= len;
+ }
+}
+
+void __init sme_early_encrypt(resource_size_t paddr, unsigned long size)
+{
+ __sme_early_enc_dec(paddr, size, true);
+}
+
+void __init sme_early_decrypt(resource_size_t paddr, unsigned long size)
+{
+ __sme_early_enc_dec(paddr, size, false);
+}
+
+static void __init __sme_early_map_unmap_mem(void *vaddr, unsigned long size,
+ bool map)
+{
+ unsigned long paddr = (unsigned long)vaddr - __PAGE_OFFSET;
+ pmdval_t pmd_flags, pmd;
+
+ /* Use early_pmd_flags but remove the encryption mask */
+ pmd_flags = __sme_clr(early_pmd_flags);
+
+ do {
+ pmd = map ? (paddr & PMD_MASK) + pmd_flags : 0;
+ __early_make_pgtable((unsigned long)vaddr, pmd);
+
+ vaddr += PMD_SIZE;
+ paddr += PMD_SIZE;
+ size = (size <= PMD_SIZE) ? 0 : size - PMD_SIZE;
+ } while (size);
+
+ flush_tlb_local();
+}
+
+void __init sme_unmap_bootdata(char *real_mode_data)
+{
+ struct boot_params *boot_data;
+ unsigned long cmdline_paddr;
+
+ if (!cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT))
+ return;
+
+ /* Get the command line address before unmapping the real_mode_data */
+ boot_data = (struct boot_params *)real_mode_data;
+ cmdline_paddr = boot_data->hdr.cmd_line_ptr | ((u64)boot_data->ext_cmd_line_ptr << 32);
+
+ __sme_early_map_unmap_mem(real_mode_data, sizeof(boot_params), false);
+
+ if (!cmdline_paddr)
+ return;
+
+ __sme_early_map_unmap_mem(__va(cmdline_paddr), COMMAND_LINE_SIZE, false);
+}
+
+void __init sme_map_bootdata(char *real_mode_data)
+{
+ struct boot_params *boot_data;
+ unsigned long cmdline_paddr;
+
+ if (!cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT))
+ return;
+
+ __sme_early_map_unmap_mem(real_mode_data, sizeof(boot_params), true);
+
+ /* Get the command line address after mapping the real_mode_data */
+ boot_data = (struct boot_params *)real_mode_data;
+ cmdline_paddr = boot_data->hdr.cmd_line_ptr | ((u64)boot_data->ext_cmd_line_ptr << 32);
+
+ if (!cmdline_paddr)
+ return;
+
+ __sme_early_map_unmap_mem(__va(cmdline_paddr), COMMAND_LINE_SIZE, true);
+}
+
+void __init sme_early_init(void)
+{
+ unsigned int i;
+
+ if (!sme_me_mask)
+ return;
+
+ early_pmd_flags = __sme_set(early_pmd_flags);
+
+ __supported_pte_mask = __sme_set(__supported_pte_mask);
+
+ /* Update the protection map with memory encryption mask */
+ for (i = 0; i < ARRAY_SIZE(protection_map); i++)
+ protection_map[i] = pgprot_encrypted(protection_map[i]);
+
+ if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT))
+ swiotlb_force = SWIOTLB_FORCE;
+}
+
+void __init sev_setup_arch(void)
+{
+ phys_addr_t total_mem = memblock_phys_mem_size();
+ unsigned long size;
+
+ if (!cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT))
+ return;
+
+ /*
+ * For SEV, all DMA has to occur via shared/unencrypted pages.
+ * SEV uses SWIOTLB to make this happen without changing device
+ * drivers. However, depending on the workload being run, the
+ * default 64MB of SWIOTLB may not be enough and SWIOTLB may
+ * run out of buffers for DMA, resulting in I/O errors and/or
+ * performance degradation especially with high I/O workloads.
+ *
+ * Adjust the default size of SWIOTLB for SEV guests using
+ * a percentage of guest memory for SWIOTLB buffers.
+ * Also, as the SWIOTLB bounce buffer memory is allocated
+ * from low memory, ensure that the adjusted size is within
+ * the limits of low available memory.
+ *
+ * The percentage of guest memory used here for SWIOTLB buffers
+ * is more of an approximation of the static adjustment which
+ * 64MB for <1G, and ~128M to 256M for 1G-to-4G, i.e., the 6%
+ */
+ size = total_mem * 6 / 100;
+ size = clamp_val(size, IO_TLB_DEFAULT_SIZE, SZ_1G);
+ swiotlb_adjust_size(size);
+}
+
+static unsigned long pg_level_to_pfn(int level, pte_t *kpte, pgprot_t *ret_prot)
+{
+ unsigned long pfn = 0;
+ pgprot_t prot;
+
+ switch (level) {
+ case PG_LEVEL_4K:
+ pfn = pte_pfn(*kpte);
+ prot = pte_pgprot(*kpte);
+ break;
+ case PG_LEVEL_2M:
+ pfn = pmd_pfn(*(pmd_t *)kpte);
+ prot = pmd_pgprot(*(pmd_t *)kpte);
+ break;
+ case PG_LEVEL_1G:
+ pfn = pud_pfn(*(pud_t *)kpte);
+ prot = pud_pgprot(*(pud_t *)kpte);
+ break;
+ default:
+ WARN_ONCE(1, "Invalid level for kpte\n");
+ return 0;
+ }
+
+ if (ret_prot)
+ *ret_prot = prot;
+
+ return pfn;
+}
+
+void notify_range_enc_status_changed(unsigned long vaddr, int npages, bool enc)
+{
+#ifdef CONFIG_PARAVIRT
+ unsigned long sz = npages << PAGE_SHIFT;
+ unsigned long vaddr_end = vaddr + sz;
+
+ while (vaddr < vaddr_end) {
+ int psize, pmask, level;
+ unsigned long pfn;
+ pte_t *kpte;
+
+ kpte = lookup_address(vaddr, &level);
+ if (!kpte || pte_none(*kpte)) {
+ WARN_ONCE(1, "kpte lookup for vaddr\n");
+ return;
+ }
+
+ pfn = pg_level_to_pfn(level, kpte, NULL);
+ if (!pfn)
+ continue;
+
+ psize = page_level_size(level);
+ pmask = page_level_mask(level);
+
+ notify_page_enc_status_changed(pfn, psize >> PAGE_SHIFT, enc);
+
+ vaddr = (vaddr & pmask) + psize;
+ }
+#endif
+}
+
+static void __init __set_clr_pte_enc(pte_t *kpte, int level, bool enc)
+{
+ pgprot_t old_prot, new_prot;
+ unsigned long pfn, pa, size;
+ pte_t new_pte;
+
+ pfn = pg_level_to_pfn(level, kpte, &old_prot);
+ if (!pfn)
+ return;
+
+ new_prot = old_prot;
+ if (enc)
+ pgprot_val(new_prot) |= _PAGE_ENC;
+ else
+ pgprot_val(new_prot) &= ~_PAGE_ENC;
+
+ /* If prot is same then do nothing. */
+ if (pgprot_val(old_prot) == pgprot_val(new_prot))
+ return;
+
+ pa = pfn << PAGE_SHIFT;
+ size = page_level_size(level);
+
+ /*
+ * We are going to perform in-place en-/decryption and change the
+ * physical page attribute from C=1 to C=0 or vice versa. Flush the
+ * caches to ensure that data gets accessed with the correct C-bit.
+ */
+ clflush_cache_range(__va(pa), size);
+
+ /* Encrypt/decrypt the contents in-place */
+ if (enc)
+ sme_early_encrypt(pa, size);
+ else
+ sme_early_decrypt(pa, size);
+
+ /* Change the page encryption mask. */
+ new_pte = pfn_pte(pfn, new_prot);
+ set_pte_atomic(kpte, new_pte);
+}
+
+static int __init early_set_memory_enc_dec(unsigned long vaddr,
+ unsigned long size, bool enc)
+{
+ unsigned long vaddr_end, vaddr_next, start;
+ unsigned long psize, pmask;
+ int split_page_size_mask;
+ int level, ret;
+ pte_t *kpte;
+
+ start = vaddr;
+ vaddr_next = vaddr;
+ vaddr_end = vaddr + size;
+
+ for (; vaddr < vaddr_end; vaddr = vaddr_next) {
+ kpte = lookup_address(vaddr, &level);
+ if (!kpte || pte_none(*kpte)) {
+ ret = 1;
+ goto out;
+ }
+
+ if (level == PG_LEVEL_4K) {
+ __set_clr_pte_enc(kpte, level, enc);
+ vaddr_next = (vaddr & PAGE_MASK) + PAGE_SIZE;
+ continue;
+ }
+
+ psize = page_level_size(level);
+ pmask = page_level_mask(level);
+
+ /*
+ * Check whether we can change the large page in one go.
+ * We request a split when the address is not aligned and
+ * the number of pages to set/clear encryption bit is smaller
+ * than the number of pages in the large page.
+ */
+ if (vaddr == (vaddr & pmask) &&
+ ((vaddr_end - vaddr) >= psize)) {
+ __set_clr_pte_enc(kpte, level, enc);
+ vaddr_next = (vaddr & pmask) + psize;
+ continue;
+ }
+
+ /*
+ * The virtual address is part of a larger page, create the next
+ * level page table mapping (4K or 2M). If it is part of a 2M
+ * page then we request a split of the large page into 4K
+ * chunks. A 1GB large page is split into 2M pages, resp.
+ */
+ if (level == PG_LEVEL_2M)
+ split_page_size_mask = 0;
+ else
+ split_page_size_mask = 1 << PG_LEVEL_2M;
+
+ /*
+ * kernel_physical_mapping_change() does not flush the TLBs, so
+ * a TLB flush is required after we exit from the for loop.
+ */
+ kernel_physical_mapping_change(__pa(vaddr & pmask),
+ __pa((vaddr_end & pmask) + psize),
+ split_page_size_mask);
+ }
+
+ ret = 0;
+
+ notify_range_enc_status_changed(start, PAGE_ALIGN(size) >> PAGE_SHIFT, enc);
+out:
+ __flush_tlb_all();
+ return ret;
+}
+
+int __init early_set_memory_decrypted(unsigned long vaddr, unsigned long size)
+{
+ return early_set_memory_enc_dec(vaddr, size, false);
+}
+
+int __init early_set_memory_encrypted(unsigned long vaddr, unsigned long size)
+{
+ return early_set_memory_enc_dec(vaddr, size, true);
+}
+
+void __init early_set_mem_enc_dec_hypercall(unsigned long vaddr, int npages, bool enc)
+{
+ notify_range_enc_status_changed(vaddr, npages, enc);
+}
+
+void __init mem_encrypt_free_decrypted_mem(void)
+{
+ unsigned long vaddr, vaddr_end, npages;
+ int r;
+
+ vaddr = (unsigned long)__start_bss_decrypted_unused;
+ vaddr_end = (unsigned long)__end_bss_decrypted;
+ npages = (vaddr_end - vaddr) >> PAGE_SHIFT;
+
+ /*
+ * The unused memory range was mapped decrypted, change the encryption
+ * attribute from decrypted to encrypted before freeing it.
+ */
+ if (cc_platform_has(CC_ATTR_MEM_ENCRYPT)) {
+ r = set_memory_encrypted(vaddr, npages);
+ if (r) {
+ pr_warn("failed to free unused decrypted pages\n");
+ return;
+ }
+ }
+
+ free_init_pages("unused decrypted", vaddr, vaddr_end);
+}
diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c
index 59ba296..a6cf56a 100644
--- a/arch/x86/mm/tlb.c
+++ b/arch/x86/mm/tlb.c
@@ -361,7 +361,7 @@ static void l1d_flush_evaluate(unsigned long prev_mm, unsigned long next_mm,
static unsigned long mm_mangle_tif_spec_bits(struct task_struct *next)
{
- unsigned long next_tif = task_thread_info(next)->flags;
+ unsigned long next_tif = read_task_thread_flags(next);
unsigned long spec_bits = (next_tif >> TIF_SPEC_IB) & LAST_USER_MM_SPEC_MASK;
/*
@@ -1148,7 +1148,7 @@ void flush_tlb_one_user(unsigned long addr)
*/
STATIC_NOPV void native_flush_tlb_global(void)
{
- unsigned long cr4, flags;
+ unsigned long flags;
if (static_cpu_has(X86_FEATURE_INVPCID)) {
/*
@@ -1168,11 +1168,7 @@ STATIC_NOPV void native_flush_tlb_global(void)
*/
raw_local_irq_save(flags);
- cr4 = this_cpu_read(cpu_tlbstate.cr4);
- /* toggle PGE */
- native_write_cr4(cr4 ^ X86_CR4_PGE);
- /* write old PGE again and flush TLBs */
- native_write_cr4(cr4);
+ __native_tlb_flush_global(this_cpu_read(cpu_tlbstate.cr4));
raw_local_irq_restore(flags);
}
diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index bafe36e..ce1f86f 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -1,9 +1,9 @@
// SPDX-License-Identifier: GPL-2.0-only
/*
- * bpf_jit_comp.c: BPF JIT compiler
+ * BPF JIT compiler
*
* Copyright (C) 2011-2013 Eric Dumazet (eric.dumazet@gmail.com)
- * Internal BPF Copyright (c) 2011-2014 PLUMgrid, http://plumgrid.com
+ * Copyright (c) 2011-2014 PLUMgrid, http://plumgrid.com
*/
#include <linux/netdevice.h>
#include <linux/filter.h>
@@ -412,7 +412,7 @@ static void emit_indirect_jump(u8 **pprog, int reg, u8 *ip)
* ... bpf_tail_call(void *ctx, struct bpf_array *array, u64 index) ...
* if (index >= array->map.max_entries)
* goto out;
- * if (++tail_call_cnt > MAX_TAIL_CALL_CNT)
+ * if (tail_call_cnt++ >= MAX_TAIL_CALL_CNT)
* goto out;
* prog = array->ptrs[index];
* if (prog == NULL)
@@ -446,14 +446,14 @@ static void emit_bpf_tail_call_indirect(u8 **pprog, bool *callee_regs_used,
EMIT2(X86_JBE, offset); /* jbe out */
/*
- * if (tail_call_cnt > MAX_TAIL_CALL_CNT)
+ * if (tail_call_cnt++ >= MAX_TAIL_CALL_CNT)
* goto out;
*/
EMIT2_off32(0x8B, 0x85, tcc_off); /* mov eax, dword ptr [rbp - tcc_off] */
EMIT3(0x83, 0xF8, MAX_TAIL_CALL_CNT); /* cmp eax, MAX_TAIL_CALL_CNT */
offset = ctx->tail_call_indirect_label - (prog + 2 - start);
- EMIT2(X86_JA, offset); /* ja out */
+ EMIT2(X86_JAE, offset); /* jae out */
EMIT3(0x83, 0xC0, 0x01); /* add eax, 1 */
EMIT2_off32(0x89, 0x85, tcc_off); /* mov dword ptr [rbp - tcc_off], eax */
@@ -504,14 +504,14 @@ static void emit_bpf_tail_call_direct(struct bpf_jit_poke_descriptor *poke,
int offset;
/*
- * if (tail_call_cnt > MAX_TAIL_CALL_CNT)
+ * if (tail_call_cnt++ >= MAX_TAIL_CALL_CNT)
* goto out;
*/
EMIT2_off32(0x8B, 0x85, tcc_off); /* mov eax, dword ptr [rbp - tcc_off] */
EMIT3(0x83, 0xF8, MAX_TAIL_CALL_CNT); /* cmp eax, MAX_TAIL_CALL_CNT */
offset = ctx->tail_call_direct_label - (prog + 2 - start);
- EMIT2(X86_JA, offset); /* ja out */
+ EMIT2(X86_JAE, offset); /* jae out */
EMIT3(0x83, 0xC0, 0x01); /* add eax, 1 */
EMIT2_off32(0x89, 0x85, tcc_off); /* mov dword ptr [rbp - tcc_off], eax */
@@ -1976,7 +1976,7 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i
void *orig_call)
{
int ret, i, nr_args = m->nr_args;
- int stack_size = nr_args * 8;
+ int regs_off, ip_off, args_off, stack_size = nr_args * 8;
struct bpf_tramp_progs *fentry = &tprogs[BPF_TRAMP_FENTRY];
struct bpf_tramp_progs *fexit = &tprogs[BPF_TRAMP_FEXIT];
struct bpf_tramp_progs *fmod_ret = &tprogs[BPF_TRAMP_MODIFY_RETURN];
@@ -1991,14 +1991,39 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i
if (!is_valid_bpf_tramp_flags(flags))
return -EINVAL;
+ /* Generated trampoline stack layout:
+ *
+ * RBP + 8 [ return address ]
+ * RBP + 0 [ RBP ]
+ *
+ * RBP - 8 [ return value ] BPF_TRAMP_F_CALL_ORIG or
+ * BPF_TRAMP_F_RET_FENTRY_RET flags
+ *
+ * [ reg_argN ] always
+ * [ ... ]
+ * RBP - regs_off [ reg_arg1 ] program's ctx pointer
+ *
+ * RBP - args_off [ args count ] always
+ *
+ * RBP - ip_off [ traced function ] BPF_TRAMP_F_IP_ARG flag
+ */
+
/* room for return value of orig_call or fentry prog */
save_ret = flags & (BPF_TRAMP_F_CALL_ORIG | BPF_TRAMP_F_RET_FENTRY_RET);
if (save_ret)
stack_size += 8;
+ regs_off = stack_size;
+
+ /* args count */
+ stack_size += 8;
+ args_off = stack_size;
+
if (flags & BPF_TRAMP_F_IP_ARG)
stack_size += 8; /* room for IP address argument */
+ ip_off = stack_size;
+
if (flags & BPF_TRAMP_F_SKIP_FRAME)
/* skip patched call instruction and point orig_call to actual
* body of the kernel function.
@@ -2012,23 +2037,25 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i
EMIT4(0x48, 0x83, 0xEC, stack_size); /* sub rsp, stack_size */
EMIT1(0x53); /* push rbx */
+ /* Store number of arguments of the traced function:
+ * mov rax, nr_args
+ * mov QWORD PTR [rbp - args_off], rax
+ */
+ emit_mov_imm64(&prog, BPF_REG_0, 0, (u32) nr_args);
+ emit_stx(&prog, BPF_DW, BPF_REG_FP, BPF_REG_0, -args_off);
+
if (flags & BPF_TRAMP_F_IP_ARG) {
/* Store IP address of the traced function:
* mov rax, QWORD PTR [rbp + 8]
* sub rax, X86_PATCH_SIZE
- * mov QWORD PTR [rbp - stack_size], rax
+ * mov QWORD PTR [rbp - ip_off], rax
*/
emit_ldx(&prog, BPF_DW, BPF_REG_0, BPF_REG_FP, 8);
EMIT4(0x48, 0x83, 0xe8, X86_PATCH_SIZE);
- emit_stx(&prog, BPF_DW, BPF_REG_FP, BPF_REG_0, -stack_size);
-
- /* Continue with stack_size for regs storage, stack will
- * be correctly restored with 'leave' instruction.
- */
- stack_size -= 8;
+ emit_stx(&prog, BPF_DW, BPF_REG_FP, BPF_REG_0, -ip_off);
}
- save_regs(m, &prog, nr_args, stack_size);
+ save_regs(m, &prog, nr_args, regs_off);
if (flags & BPF_TRAMP_F_CALL_ORIG) {
/* arg1: mov rdi, im */
@@ -2040,7 +2067,7 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i
}
if (fentry->nr_progs)
- if (invoke_bpf(m, &prog, fentry, stack_size,
+ if (invoke_bpf(m, &prog, fentry, regs_off,
flags & BPF_TRAMP_F_RET_FENTRY_RET))
return -EINVAL;
@@ -2050,7 +2077,7 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i
if (!branches)
return -ENOMEM;
- if (invoke_bpf_mod_ret(m, &prog, fmod_ret, stack_size,
+ if (invoke_bpf_mod_ret(m, &prog, fmod_ret, regs_off,
branches)) {
ret = -EINVAL;
goto cleanup;
@@ -2058,7 +2085,7 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i
}
if (flags & BPF_TRAMP_F_CALL_ORIG) {
- restore_regs(m, &prog, nr_args, stack_size);
+ restore_regs(m, &prog, nr_args, regs_off);
/* call original function */
if (emit_call(&prog, orig_call, prog)) {
@@ -2088,13 +2115,13 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i
}
if (fexit->nr_progs)
- if (invoke_bpf(m, &prog, fexit, stack_size, false)) {
+ if (invoke_bpf(m, &prog, fexit, regs_off, false)) {
ret = -EINVAL;
goto cleanup;
}
if (flags & BPF_TRAMP_F_RESTORE_REGS)
- restore_regs(m, &prog, nr_args, stack_size);
+ restore_regs(m, &prog, nr_args, regs_off);
/* This needs to be done regardless. If there were fmod_ret programs,
* the return value is only updated on the stack and still needs to be
diff --git a/arch/x86/net/bpf_jit_comp32.c b/arch/x86/net/bpf_jit_comp32.c
index da9b7cf..429a89c 100644
--- a/arch/x86/net/bpf_jit_comp32.c
+++ b/arch/x86/net/bpf_jit_comp32.c
@@ -1323,7 +1323,7 @@ static void emit_bpf_tail_call(u8 **pprog, u8 *ip)
EMIT2(IA32_JBE, jmp_label(jmp_label1, 2));
/*
- * if (tail_call_cnt > MAX_TAIL_CALL_CNT)
+ * if (tail_call_cnt++ >= MAX_TAIL_CALL_CNT)
* goto out;
*/
lo = (u32)MAX_TAIL_CALL_CNT;
@@ -1337,7 +1337,7 @@ static void emit_bpf_tail_call(u8 **pprog, u8 *ip)
/* cmp ecx,lo */
EMIT3(0x83, add_1reg(0xF8, IA32_ECX), lo);
- /* ja out */
+ /* jae out */
EMIT2(IA32_JAE, jmp_label(jmp_label1, 2));
/* add eax,0x1 */
diff --git a/arch/x86/platform/ce4100/falconfalls.dts b/arch/x86/platform/ce4100/falconfalls.dts
index 0ac3d43..65fa3d8 100644
--- a/arch/x86/platform/ce4100/falconfalls.dts
+++ b/arch/x86/platform/ce4100/falconfalls.dts
@@ -249,7 +249,7 @@ i2c@1 {
gpio@26 {
#gpio-cells = <2>;
- compatible = "ti,pcf8575";
+ compatible = "nxp,pcf8575";
reg = <0x26>;
gpio-controller;
};
@@ -263,7 +263,7 @@ i2c@2 {
gpio@26 {
#gpio-cells = <2>;
- compatible = "ti,pcf8575";
+ compatible = "nxp,pcf8575";
reg = <0x26>;
gpio-controller;
};
diff --git a/arch/x86/purgatory/Makefile b/arch/x86/purgatory/Makefile
index 95ea17a..ae53d54 100644
--- a/arch/x86/purgatory/Makefile
+++ b/arch/x86/purgatory/Makefile
@@ -16,7 +16,7 @@
# When linking purgatory.ro with -r unresolved symbols are not checked,
# also link a purgatory.chk binary without -r to check for unresolved symbols.
-PURGATORY_LDFLAGS := -e purgatory_start -nostdlib -z nodefaultlib
+PURGATORY_LDFLAGS := -e purgatory_start -z nodefaultlib
LDFLAGS_purgatory.ro := -r $(PURGATORY_LDFLAGS)
LDFLAGS_purgatory.chk := $(PURGATORY_LDFLAGS)
targets += purgatory.ro purgatory.chk
diff --git a/arch/x86/realmode/init.c b/arch/x86/realmode/init.c
index 38d24d2..c5e29db 100644
--- a/arch/x86/realmode/init.c
+++ b/arch/x86/realmode/init.c
@@ -17,6 +17,32 @@ u32 *trampoline_cr4_features;
/* Hold the pgd entry used on booting additional CPUs */
pgd_t trampoline_pgd_entry;
+void load_trampoline_pgtable(void)
+{
+#ifdef CONFIG_X86_32
+ load_cr3(initial_page_table);
+#else
+ /*
+ * This function is called before exiting to real-mode and that will
+ * fail with CR4.PCIDE still set.
+ */
+ if (boot_cpu_has(X86_FEATURE_PCID))
+ cr4_clear_bits(X86_CR4_PCIDE);
+
+ write_cr3(real_mode_header->trampoline_pgd);
+#endif
+
+ /*
+ * The CR3 write above will not flush global TLB entries.
+ * Stale, global entries from previous page tables may still be
+ * present. Flush those stale entries.
+ *
+ * This ensures that memory accessed while running with
+ * trampoline_pgd is *actually* mapped into trampoline_pgd.
+ */
+ __flush_tlb_all();
+}
+
void __init reserve_real_mode(void)
{
phys_addr_t mem;
diff --git a/arch/x86/tools/relocs.c b/arch/x86/tools/relocs.c
index c736cf2..e2c5b29 100644
--- a/arch/x86/tools/relocs.c
+++ b/arch/x86/tools/relocs.c
@@ -68,7 +68,7 @@ static const char * const sym_regex_kernel[S_NSYMTYPES] = {
"(__parainstructions|__alt_instructions)(_end)?|"
"(__iommu_table|__apicdrivers|__smp_locks)(_end)?|"
"__(start|end)_pci_.*|"
-#if CONFIG_FW_LOADER_BUILTIN
+#if CONFIG_FW_LOADER
"__(start|end)_builtin_fw|"
#endif
"__(start|stop)___ksymtab(_gpl)?|"
diff --git a/crypto/Kconfig b/crypto/Kconfig
index 285f826..55718de 100644
--- a/crypto/Kconfig
+++ b/crypto/Kconfig
@@ -1919,9 +1919,10 @@
config CRYPTO_HASH_INFO
bool
-source "lib/crypto/Kconfig"
source "drivers/crypto/Kconfig"
source "crypto/asymmetric_keys/Kconfig"
source "certs/Kconfig"
endif # if CRYPTO
+
+source "lib/crypto/Kconfig"
diff --git a/drivers/acpi/apei/einj.c b/drivers/acpi/apei/einj.c
index edb2622..95cc2a9 100644
--- a/drivers/acpi/apei/einj.c
+++ b/drivers/acpi/apei/einj.c
@@ -545,7 +545,8 @@ static int einj_error_inject(u32 type, u32 flags, u64 param1, u64 param2,
((region_intersects(base_addr, size, IORESOURCE_SYSTEM_RAM, IORES_DESC_NONE)
!= REGION_INTERSECTS) &&
(region_intersects(base_addr, size, IORESOURCE_MEM, IORES_DESC_PERSISTENT_MEMORY)
- != REGION_INTERSECTS)))
+ != REGION_INTERSECTS) &&
+ !arch_is_platform_page(base_addr)))
return -EINVAL;
inject:
diff --git a/drivers/acpi/apei/ghes.c b/drivers/acpi/apei/ghes.c
index 0c8330e..0c5c9ac 100644
--- a/drivers/acpi/apei/ghes.c
+++ b/drivers/acpi/apei/ghes.c
@@ -449,7 +449,7 @@ static bool ghes_do_memory_failure(u64 physical_addr, int flags)
return false;
pfn = PHYS_PFN(physical_addr);
- if (!pfn_valid(pfn)) {
+ if (!pfn_valid(pfn) && !arch_is_platform_page(physical_addr)) {
pr_warn_ratelimited(FW_WARN GHES_PFX
"Invalid address in generic error data: %#llx\n",
physical_addr);
diff --git a/drivers/acpi/scan.c b/drivers/acpi/scan.c
index 4dd3a9e..1185ecea5 100644
--- a/drivers/acpi/scan.c
+++ b/drivers/acpi/scan.c
@@ -810,6 +810,12 @@ static const char * const acpi_ignore_dep_ids[] = {
NULL
};
+/* List of HIDs for which we honor deps of matching ACPI devs, when checking _DEP lists. */
+static const char * const acpi_honor_dep_ids[] = {
+ "INT3472", /* Camera sensor PMIC / clk and regulator info */
+ NULL
+};
+
static struct acpi_device *acpi_bus_get_parent(acpi_handle handle)
{
struct acpi_device *device;
@@ -1784,8 +1790,12 @@ static void acpi_scan_dep_init(struct acpi_device *adev)
struct acpi_dep_data *dep;
list_for_each_entry(dep, &acpi_dep_list, node) {
- if (dep->consumer == adev->handle)
+ if (dep->consumer == adev->handle) {
+ if (dep->honor_dep)
+ adev->flags.honor_deps = 1;
+
adev->dep_unmet++;
+ }
}
}
@@ -1989,7 +1999,7 @@ static u32 acpi_scan_check_dep(acpi_handle handle, bool check_dep)
for (count = 0, i = 0; i < dep_devices.count; i++) {
struct acpi_device_info *info;
struct acpi_dep_data *dep;
- bool skip;
+ bool skip, honor_dep;
status = acpi_get_object_info(dep_devices.handles[i], &info);
if (ACPI_FAILURE(status)) {
@@ -1998,6 +2008,7 @@ static u32 acpi_scan_check_dep(acpi_handle handle, bool check_dep)
}
skip = acpi_info_matches_ids(info, acpi_ignore_dep_ids);
+ honor_dep = acpi_info_matches_ids(info, acpi_honor_dep_ids);
kfree(info);
if (skip)
@@ -2011,6 +2022,7 @@ static u32 acpi_scan_check_dep(acpi_handle handle, bool check_dep)
dep->supplier = dep_devices.handles[i];
dep->consumer = handle;
+ dep->honor_dep = honor_dep;
mutex_lock(&acpi_dep_list_lock);
list_add_tail(&dep->node , &acpi_dep_list);
@@ -2176,8 +2188,8 @@ static void acpi_bus_attach(struct acpi_device *device, bool first_pass)
register_dock_dependent_device(device, ejd);
acpi_bus_get_status(device);
- /* Skip devices that are not present. */
- if (!acpi_device_is_present(device)) {
+ /* Skip devices that are not ready for enumeration (e.g. not present) */
+ if (!acpi_dev_ready_for_enumeration(device)) {
device->flags.initialized = false;
acpi_device_clear_enumerated(device);
device->flags.power_manageable = 0;
@@ -2340,6 +2352,23 @@ void acpi_dev_clear_dependencies(struct acpi_device *supplier)
EXPORT_SYMBOL_GPL(acpi_dev_clear_dependencies);
/**
+ * acpi_dev_ready_for_enumeration - Check if the ACPI device is ready for enumeration
+ * @device: Pointer to the &struct acpi_device to check
+ *
+ * Check if the device is present and has no unmet dependencies.
+ *
+ * Return true if the device is ready for enumeratino. Otherwise, return false.
+ */
+bool acpi_dev_ready_for_enumeration(const struct acpi_device *device)
+{
+ if (device->flags.honor_deps && device->dep_unmet)
+ return false;
+
+ return acpi_device_is_present(device);
+}
+EXPORT_SYMBOL_GPL(acpi_dev_ready_for_enumeration);
+
+/**
* acpi_dev_get_first_consumer_dev - Return ACPI device dependent on @supplier
* @supplier: Pointer to the dependee device
*
diff --git a/drivers/android/binder_alloc.c b/drivers/android/binder_alloc.c
index 340515f..47bc74a 100644
--- a/drivers/android/binder_alloc.c
+++ b/drivers/android/binder_alloc.c
@@ -671,7 +671,7 @@ static void binder_free_buf_locked(struct binder_alloc *alloc,
BUG_ON(buffer->user_data > alloc->buffer + alloc->buffer_size);
if (buffer->async_transaction) {
- alloc->free_async_space += size + sizeof(struct binder_buffer);
+ alloc->free_async_space += buffer_size + sizeof(struct binder_buffer);
binder_alloc_debug(BINDER_DEBUG_BUFFER_ALLOC_ASYNC,
"%d: binder_free_buf size %zd async free %zd\n",
diff --git a/drivers/auxdisplay/charlcd.c b/drivers/auxdisplay/charlcd.c
index 304accd..6d309e4 100644
--- a/drivers/auxdisplay/charlcd.c
+++ b/drivers/auxdisplay/charlcd.c
@@ -37,7 +37,7 @@ struct charlcd_priv {
bool must_clear;
/* contains the LCD config state */
- unsigned long int flags;
+ unsigned long flags;
/* Current escape sequence and it's length or -1 if outside */
struct {
@@ -578,6 +578,9 @@ static int charlcd_init(struct charlcd *lcd)
* Since charlcd_init_display() needs to write data, we have to
* enable mark the LCD initialized just before.
*/
+ if (WARN_ON(!lcd->ops->init_display))
+ return -EINVAL;
+
ret = lcd->ops->init_display(lcd);
if (ret)
return ret;
diff --git a/drivers/base/node.c b/drivers/base/node.c
index b5a4ba18..87acc47 100644
--- a/drivers/base/node.c
+++ b/drivers/base/node.c
@@ -581,6 +581,9 @@ static const struct attribute_group node_dev_group = {
static const struct attribute_group *node_dev_groups[] = {
&node_dev_group,
+#ifdef CONFIG_HAVE_ARCH_NODE_DEV_GROUP
+ &arch_node_dev_group,
+#endif
NULL
};
diff --git a/drivers/base/regmap/regmap.c b/drivers/base/regmap/regmap.c
index 21a0c25..2d74f9f 100644
--- a/drivers/base/regmap/regmap.c
+++ b/drivers/base/regmap/regmap.c
@@ -876,6 +876,7 @@ struct regmap *__regmap_init(struct device *dev,
if (!bus) {
map->reg_read = config->reg_read;
map->reg_write = config->reg_write;
+ map->reg_update_bits = config->reg_update_bits;
map->defer_caching = false;
goto skip_format_initialization;
diff --git a/drivers/bluetooth/Kconfig b/drivers/bluetooth/Kconfig
index 8518423..36380e6 100644
--- a/drivers/bluetooth/Kconfig
+++ b/drivers/bluetooth/Kconfig
@@ -19,6 +19,10 @@
tristate
select FW_LOADER
+config BT_MTK
+ tristate
+ select FW_LOADER
+
config BT_HCIBTUSB
tristate "HCI USB driver"
depends on USB
@@ -55,6 +59,7 @@
config BT_HCIBTUSB_MTK
bool "MediaTek protocol support"
depends on BT_HCIBTUSB
+ select BT_MTK
default n
help
The MediaTek protocol support enables firmware download
@@ -383,6 +388,7 @@
config BT_MTKSDIO
tristate "MediaTek HCI SDIO driver"
depends on MMC
+ select BT_MTK
help
MediaTek Bluetooth HCI SDIO driver.
This driver is required if you want to use MediaTek Bluetooth
diff --git a/drivers/bluetooth/Makefile b/drivers/bluetooth/Makefile
index 16286ea..3321a8a 100644
--- a/drivers/bluetooth/Makefile
+++ b/drivers/bluetooth/Makefile
@@ -25,6 +25,7 @@
obj-$(CONFIG_BT_BCM) += btbcm.o
obj-$(CONFIG_BT_RTL) += btrtl.o
obj-$(CONFIG_BT_QCA) += btqca.o
+obj-$(CONFIG_BT_MTK) += btmtk.o
obj-$(CONFIG_BT_VIRTIO) += virtio_bt.o
diff --git a/drivers/bluetooth/bfusb.c b/drivers/bluetooth/bfusb.c
index 5a321b4..cab9393 100644
--- a/drivers/bluetooth/bfusb.c
+++ b/drivers/bluetooth/bfusb.c
@@ -628,6 +628,9 @@ static int bfusb_probe(struct usb_interface *intf, const struct usb_device_id *i
data->bulk_out_ep = bulk_out_ep->desc.bEndpointAddress;
data->bulk_pkt_size = le16_to_cpu(bulk_out_ep->desc.wMaxPacketSize);
+ if (!data->bulk_pkt_size)
+ goto done;
+
rwlock_init(&data->lock);
data->reassembly = NULL;
diff --git a/drivers/bluetooth/btbcm.c b/drivers/bluetooth/btbcm.c
index e4182ac..d9ceca7 100644
--- a/drivers/bluetooth/btbcm.c
+++ b/drivers/bluetooth/btbcm.c
@@ -8,6 +8,7 @@
#include <linux/module.h>
#include <linux/firmware.h>
+#include <linux/dmi.h>
#include <asm/unaligned.h>
#include <net/bluetooth/bluetooth.h>
@@ -343,6 +344,52 @@ static struct sk_buff *btbcm_read_usb_product(struct hci_dev *hdev)
return skb;
}
+static const struct dmi_system_id disable_broken_read_transmit_power[] = {
+ {
+ .matches = {
+ DMI_MATCH(DMI_BOARD_VENDOR, "Apple Inc."),
+ DMI_MATCH(DMI_PRODUCT_NAME, "MacBookPro16,1"),
+ },
+ },
+ {
+ .matches = {
+ DMI_MATCH(DMI_BOARD_VENDOR, "Apple Inc."),
+ DMI_MATCH(DMI_PRODUCT_NAME, "MacBookPro16,2"),
+ },
+ },
+ {
+ .matches = {
+ DMI_MATCH(DMI_BOARD_VENDOR, "Apple Inc."),
+ DMI_MATCH(DMI_PRODUCT_NAME, "MacBookPro16,4"),
+ },
+ },
+ {
+ .matches = {
+ DMI_MATCH(DMI_BOARD_VENDOR, "Apple Inc."),
+ DMI_MATCH(DMI_PRODUCT_NAME, "MacBookAir8,1"),
+ },
+ },
+ {
+ .matches = {
+ DMI_MATCH(DMI_BOARD_VENDOR, "Apple Inc."),
+ DMI_MATCH(DMI_PRODUCT_NAME, "MacBookAir8,2"),
+ },
+ },
+ {
+ .matches = {
+ DMI_MATCH(DMI_BOARD_VENDOR, "Apple Inc."),
+ DMI_MATCH(DMI_PRODUCT_NAME, "iMac20,1"),
+ },
+ },
+ {
+ .matches = {
+ DMI_MATCH(DMI_BOARD_VENDOR, "Apple Inc."),
+ DMI_MATCH(DMI_PRODUCT_NAME, "iMac20,2"),
+ },
+ },
+ { }
+};
+
static int btbcm_read_info(struct hci_dev *hdev)
{
struct sk_buff *skb;
@@ -363,6 +410,10 @@ static int btbcm_read_info(struct hci_dev *hdev)
bt_dev_info(hdev, "BCM: features 0x%2.2x", skb->data[1]);
kfree_skb(skb);
+ /* Read DMI and disable broken Read LE Min/Max Tx Power */
+ if (dmi_first_match(disable_broken_read_transmit_power))
+ set_bit(HCI_QUIRK_BROKEN_READ_TRANSMIT_POWER, &hdev->quirks);
+
return 0;
}
diff --git a/drivers/bluetooth/btintel.c b/drivers/bluetooth/btintel.c
index 9359bff..1a4f8b2 100644
--- a/drivers/bluetooth/btintel.c
+++ b/drivers/bluetooth/btintel.c
@@ -2081,14 +2081,16 @@ static int btintel_prepare_fw_download_tlv(struct hci_dev *hdev,
if (ver->img_type == 0x03) {
btintel_clear_flag(hdev, INTEL_BOOTLOADER);
btintel_check_bdaddr(hdev);
- }
-
- /* If the OTP has no valid Bluetooth device address, then there will
- * also be no valid address for the operational firmware.
- */
- if (!bacmp(&ver->otp_bd_addr, BDADDR_ANY)) {
- bt_dev_info(hdev, "No device address configured");
- set_bit(HCI_QUIRK_INVALID_BDADDR, &hdev->quirks);
+ } else {
+ /*
+ * Check for valid bd address in boot loader mode. Device
+ * will be marked as unconfigured if empty bd address is
+ * found.
+ */
+ if (!bacmp(&ver->otp_bd_addr, BDADDR_ANY)) {
+ bt_dev_info(hdev, "No device address configured");
+ set_bit(HCI_QUIRK_INVALID_BDADDR, &hdev->quirks);
+ }
}
btintel_get_fw_name_tlv(ver, fwname, sizeof(fwname), "sfi");
@@ -2353,8 +2355,15 @@ static int btintel_setup_combined(struct hci_dev *hdev)
* As a workaround, send HCI Reset command first which will reset the
* number of completed commands and allow normal command processing
* from now on.
+ *
+ * Regarding the INTEL_BROKEN_SHUTDOWN_LED flag, these devices maybe
+ * in the SW_RFKILL ON state as a workaround of fixing LED issue during
+ * the shutdown() procedure, and once the device is in SW_RFKILL ON
+ * state, the only way to exit out of it is sending the HCI_Reset
+ * command.
*/
- if (btintel_test_flag(hdev, INTEL_BROKEN_INITIAL_NCMD)) {
+ if (btintel_test_flag(hdev, INTEL_BROKEN_INITIAL_NCMD) ||
+ btintel_test_flag(hdev, INTEL_BROKEN_SHUTDOWN_LED)) {
skb = __hci_cmd_sync(hdev, HCI_OP_RESET, 0, NULL,
HCI_INIT_TIMEOUT);
if (IS_ERR(skb)) {
@@ -2426,12 +2435,6 @@ static int btintel_setup_combined(struct hci_dev *hdev)
set_bit(HCI_QUIRK_WIDEBAND_SPEECH_SUPPORTED,
&hdev->quirks);
- /* These devices have an issue with LED which doesn't
- * go off immediately during shutdown. Set the flag
- * here to send the LED OFF command during shutdown.
- */
- btintel_set_flag(hdev, INTEL_BROKEN_LED);
-
err = btintel_legacy_rom_setup(hdev, &ver);
break;
case 0x0b: /* SfP */
@@ -2466,6 +2469,10 @@ static int btintel_setup_combined(struct hci_dev *hdev)
goto exit_error;
}
+ /* memset ver_tlv to start with clean state as few fields are exclusive
+ * to bootloader mode and are not populated in operational mode
+ */
+ memset(&ver_tlv, 0, sizeof(ver_tlv));
/* For TLV type device, parse the tlv data */
err = btintel_parse_version_tlv(hdev, &ver_tlv, skb);
if (err) {
@@ -2492,10 +2499,14 @@ static int btintel_setup_combined(struct hci_dev *hdev)
case 0x12: /* ThP */
case 0x13: /* HrP */
case 0x14: /* CcP */
- /* Some legacy bootloader devices from JfP supports both old
- * and TLV based HCI_Intel_Read_Version command. But we don't
- * want to use the TLV based setup routines for those legacy
- * bootloader device.
+ /* Some legacy bootloader devices starting from JfP,
+ * the operational firmware supports both old and TLV based
+ * HCI_Intel_Read_Version command based on the command
+ * parameter.
+ *
+ * For upgrading firmware case, the TLV based version cannot
+ * be used because the firmware filename for legacy bootloader
+ * is based on the old format.
*
* Also, it is not easy to convert TLV based version from the
* legacy version format.
@@ -2507,6 +2518,20 @@ static int btintel_setup_combined(struct hci_dev *hdev)
err = btintel_read_version(hdev, &ver);
if (err)
return err;
+
+ /* Apply the device specific HCI quirks
+ *
+ * All Legacy bootloader devices support WBS
+ */
+ set_bit(HCI_QUIRK_WIDEBAND_SPEECH_SUPPORTED, &hdev->quirks);
+
+ /* Valid LE States quirk for JfP/ThP familiy */
+ if (ver.hw_variant == 0x11 || ver.hw_variant == 0x12)
+ set_bit(HCI_QUIRK_VALID_LE_STATES, &hdev->quirks);
+
+ /* Setup MSFT Extension support */
+ btintel_set_msft_opcode(hdev, ver.hw_variant);
+
err = btintel_bootloader_setup(hdev, &ver);
break;
case 0x17:
@@ -2562,9 +2587,10 @@ static int btintel_shutdown_combined(struct hci_dev *hdev)
/* Some platforms have an issue with BT LED when the interface is
* down or BT radio is turned off, which takes 5 seconds to BT LED
- * goes off. This command turns off the BT LED immediately.
+ * goes off. As a workaround, sends HCI_Intel_SW_RFKILL to put the
+ * device in the RFKILL ON state which turns off the BT LED immediately.
*/
- if (btintel_test_flag(hdev, INTEL_BROKEN_LED)) {
+ if (btintel_test_flag(hdev, INTEL_BROKEN_SHUTDOWN_LED)) {
skb = __hci_cmd_sync(hdev, 0xfc3f, 0, NULL, HCI_INIT_TIMEOUT);
if (IS_ERR(skb)) {
ret = PTR_ERR(skb);
diff --git a/drivers/bluetooth/btintel.h b/drivers/bluetooth/btintel.h
index e500c0d..c9b24e9 100644
--- a/drivers/bluetooth/btintel.h
+++ b/drivers/bluetooth/btintel.h
@@ -150,7 +150,7 @@ enum {
INTEL_FIRMWARE_FAILED,
INTEL_BOOTING,
INTEL_BROKEN_INITIAL_NCMD,
- INTEL_BROKEN_LED,
+ INTEL_BROKEN_SHUTDOWN_LED,
INTEL_ROM_LEGACY,
__INTEL_NUM_FLAGS,
diff --git a/drivers/bluetooth/btmrvl_main.c b/drivers/bluetooth/btmrvl_main.c
index 5ccbe4d..181338f 100644
--- a/drivers/bluetooth/btmrvl_main.c
+++ b/drivers/bluetooth/btmrvl_main.c
@@ -1,4 +1,4 @@
-/**
+/*
* Marvell Bluetooth driver
*
* Copyright (C) 2009, Marvell International Ltd.
diff --git a/drivers/bluetooth/btmtk.c b/drivers/bluetooth/btmtk.c
new file mode 100644
index 0000000..526dfdf
--- /dev/null
+++ b/drivers/bluetooth/btmtk.c
@@ -0,0 +1,290 @@
+// SPDX-License-Identifier: ISC
+/* Copyright (C) 2021 MediaTek Inc.
+ *
+ */
+#include <linux/module.h>
+#include <linux/firmware.h>
+
+#include <net/bluetooth/bluetooth.h>
+#include <net/bluetooth/hci_core.h>
+
+#include "btmtk.h"
+
+#define VERSION "0.1"
+
+/* It is for mt79xx download rom patch*/
+#define MTK_FW_ROM_PATCH_HEADER_SIZE 32
+#define MTK_FW_ROM_PATCH_GD_SIZE 64
+#define MTK_FW_ROM_PATCH_SEC_MAP_SIZE 64
+#define MTK_SEC_MAP_COMMON_SIZE 12
+#define MTK_SEC_MAP_NEED_SEND_SIZE 52
+
+struct btmtk_patch_header {
+ u8 datetime[16];
+ u8 platform[4];
+ __le16 hwver;
+ __le16 swver;
+ __le32 magicnum;
+} __packed;
+
+struct btmtk_global_desc {
+ __le32 patch_ver;
+ __le32 sub_sys;
+ __le32 feature_opt;
+ __le32 section_num;
+} __packed;
+
+struct btmtk_section_map {
+ __le32 sectype;
+ __le32 secoffset;
+ __le32 secsize;
+ union {
+ __le32 u4SecSpec[13];
+ struct {
+ __le32 dlAddr;
+ __le32 dlsize;
+ __le32 seckeyidx;
+ __le32 alignlen;
+ __le32 sectype;
+ __le32 dlmodecrctype;
+ __le32 crc;
+ __le32 reserved[6];
+ } bin_info_spec;
+ };
+} __packed;
+
+int btmtk_setup_firmware_79xx(struct hci_dev *hdev, const char *fwname,
+ wmt_cmd_sync_func_t wmt_cmd_sync)
+{
+ struct btmtk_hci_wmt_params wmt_params;
+ struct btmtk_global_desc *globaldesc = NULL;
+ struct btmtk_section_map *sectionmap;
+ const struct firmware *fw;
+ const u8 *fw_ptr;
+ const u8 *fw_bin_ptr;
+ int err, dlen, i, status;
+ u8 flag, first_block, retry;
+ u32 section_num, dl_size, section_offset;
+ u8 cmd[64];
+
+ err = request_firmware(&fw, fwname, &hdev->dev);
+ if (err < 0) {
+ bt_dev_err(hdev, "Failed to load firmware file (%d)", err);
+ return err;
+ }
+
+ fw_ptr = fw->data;
+ fw_bin_ptr = fw_ptr;
+ globaldesc = (struct btmtk_global_desc *)(fw_ptr + MTK_FW_ROM_PATCH_HEADER_SIZE);
+ section_num = le32_to_cpu(globaldesc->section_num);
+
+ for (i = 0; i < section_num; i++) {
+ first_block = 1;
+ fw_ptr = fw_bin_ptr;
+ sectionmap = (struct btmtk_section_map *)(fw_ptr + MTK_FW_ROM_PATCH_HEADER_SIZE +
+ MTK_FW_ROM_PATCH_GD_SIZE + MTK_FW_ROM_PATCH_SEC_MAP_SIZE * i);
+
+ section_offset = le32_to_cpu(sectionmap->secoffset);
+ dl_size = le32_to_cpu(sectionmap->bin_info_spec.dlsize);
+
+ if (dl_size > 0) {
+ retry = 20;
+ while (retry > 0) {
+ cmd[0] = 0; /* 0 means legacy dl mode. */
+ memcpy(cmd + 1,
+ fw_ptr + MTK_FW_ROM_PATCH_HEADER_SIZE +
+ MTK_FW_ROM_PATCH_GD_SIZE +
+ MTK_FW_ROM_PATCH_SEC_MAP_SIZE * i +
+ MTK_SEC_MAP_COMMON_SIZE,
+ MTK_SEC_MAP_NEED_SEND_SIZE + 1);
+
+ wmt_params.op = BTMTK_WMT_PATCH_DWNLD;
+ wmt_params.status = &status;
+ wmt_params.flag = 0;
+ wmt_params.dlen = MTK_SEC_MAP_NEED_SEND_SIZE + 1;
+ wmt_params.data = &cmd;
+
+ err = wmt_cmd_sync(hdev, &wmt_params);
+ if (err < 0) {
+ bt_dev_err(hdev, "Failed to send wmt patch dwnld (%d)",
+ err);
+ goto err_release_fw;
+ }
+
+ if (status == BTMTK_WMT_PATCH_UNDONE) {
+ break;
+ } else if (status == BTMTK_WMT_PATCH_PROGRESS) {
+ msleep(100);
+ retry--;
+ } else if (status == BTMTK_WMT_PATCH_DONE) {
+ goto next_section;
+ } else {
+ bt_dev_err(hdev, "Failed wmt patch dwnld status (%d)",
+ status);
+ err = -EIO;
+ goto err_release_fw;
+ }
+ }
+
+ fw_ptr += section_offset;
+ wmt_params.op = BTMTK_WMT_PATCH_DWNLD;
+ wmt_params.status = NULL;
+
+ while (dl_size > 0) {
+ dlen = min_t(int, 250, dl_size);
+ if (first_block == 1) {
+ flag = 1;
+ first_block = 0;
+ } else if (dl_size - dlen <= 0) {
+ flag = 3;
+ } else {
+ flag = 2;
+ }
+
+ wmt_params.flag = flag;
+ wmt_params.dlen = dlen;
+ wmt_params.data = fw_ptr;
+
+ err = wmt_cmd_sync(hdev, &wmt_params);
+ if (err < 0) {
+ bt_dev_err(hdev, "Failed to send wmt patch dwnld (%d)",
+ err);
+ goto err_release_fw;
+ }
+
+ dl_size -= dlen;
+ fw_ptr += dlen;
+ }
+ }
+next_section:
+ continue;
+ }
+ /* Wait a few moments for firmware activation done */
+ usleep_range(100000, 120000);
+
+err_release_fw:
+ release_firmware(fw);
+
+ return err;
+}
+EXPORT_SYMBOL_GPL(btmtk_setup_firmware_79xx);
+
+int btmtk_setup_firmware(struct hci_dev *hdev, const char *fwname,
+ wmt_cmd_sync_func_t wmt_cmd_sync)
+{
+ struct btmtk_hci_wmt_params wmt_params;
+ const struct firmware *fw;
+ const u8 *fw_ptr;
+ size_t fw_size;
+ int err, dlen;
+ u8 flag, param;
+
+ err = request_firmware(&fw, fwname, &hdev->dev);
+ if (err < 0) {
+ bt_dev_err(hdev, "Failed to load firmware file (%d)", err);
+ return err;
+ }
+
+ /* Power on data RAM the firmware relies on. */
+ param = 1;
+ wmt_params.op = BTMTK_WMT_FUNC_CTRL;
+ wmt_params.flag = 3;
+ wmt_params.dlen = sizeof(param);
+ wmt_params.data = ¶m;
+ wmt_params.status = NULL;
+
+ err = wmt_cmd_sync(hdev, &wmt_params);
+ if (err < 0) {
+ bt_dev_err(hdev, "Failed to power on data RAM (%d)", err);
+ goto err_release_fw;
+ }
+
+ fw_ptr = fw->data;
+ fw_size = fw->size;
+
+ /* The size of patch header is 30 bytes, should be skip */
+ if (fw_size < 30) {
+ err = -EINVAL;
+ goto err_release_fw;
+ }
+
+ fw_size -= 30;
+ fw_ptr += 30;
+ flag = 1;
+
+ wmt_params.op = BTMTK_WMT_PATCH_DWNLD;
+ wmt_params.status = NULL;
+
+ while (fw_size > 0) {
+ dlen = min_t(int, 250, fw_size);
+
+ /* Tell device the position in sequence */
+ if (fw_size - dlen <= 0)
+ flag = 3;
+ else if (fw_size < fw->size - 30)
+ flag = 2;
+
+ wmt_params.flag = flag;
+ wmt_params.dlen = dlen;
+ wmt_params.data = fw_ptr;
+
+ err = wmt_cmd_sync(hdev, &wmt_params);
+ if (err < 0) {
+ bt_dev_err(hdev, "Failed to send wmt patch dwnld (%d)",
+ err);
+ goto err_release_fw;
+ }
+
+ fw_size -= dlen;
+ fw_ptr += dlen;
+ }
+
+ wmt_params.op = BTMTK_WMT_RST;
+ wmt_params.flag = 4;
+ wmt_params.dlen = 0;
+ wmt_params.data = NULL;
+ wmt_params.status = NULL;
+
+ /* Activate funciton the firmware providing to */
+ err = wmt_cmd_sync(hdev, &wmt_params);
+ if (err < 0) {
+ bt_dev_err(hdev, "Failed to send wmt rst (%d)", err);
+ goto err_release_fw;
+ }
+
+ /* Wait a few moments for firmware activation done */
+ usleep_range(10000, 12000);
+
+err_release_fw:
+ release_firmware(fw);
+
+ return err;
+}
+EXPORT_SYMBOL_GPL(btmtk_setup_firmware);
+
+int btmtk_set_bdaddr(struct hci_dev *hdev, const bdaddr_t *bdaddr)
+{
+ struct sk_buff *skb;
+ long ret;
+
+ skb = __hci_cmd_sync(hdev, 0xfc1a, 6, bdaddr, HCI_INIT_TIMEOUT);
+ if (IS_ERR(skb)) {
+ ret = PTR_ERR(skb);
+ bt_dev_err(hdev, "changing Mediatek device address failed (%ld)",
+ ret);
+ return ret;
+ }
+ kfree_skb(skb);
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(btmtk_set_bdaddr);
+
+MODULE_AUTHOR("Sean Wang <sean.wang@mediatek.com>");
+MODULE_AUTHOR("Mark Chen <mark-yw.chen@mediatek.com>");
+MODULE_DESCRIPTION("Bluetooth support for MediaTek devices ver " VERSION);
+MODULE_VERSION(VERSION);
+MODULE_LICENSE("GPL");
+MODULE_FIRMWARE(FIRMWARE_MT7663);
+MODULE_FIRMWARE(FIRMWARE_MT7668);
+MODULE_FIRMWARE(FIRMWARE_MT7961);
diff --git a/drivers/bluetooth/btmtk.h b/drivers/bluetooth/btmtk.h
new file mode 100644
index 0000000..6e7b0c7
--- /dev/null
+++ b/drivers/bluetooth/btmtk.h
@@ -0,0 +1,111 @@
+/* SPDX-License-Identifier: ISC */
+/* Copyright (C) 2021 MediaTek Inc. */
+
+#define FIRMWARE_MT7663 "mediatek/mt7663pr2h.bin"
+#define FIRMWARE_MT7668 "mediatek/mt7668pr2h.bin"
+#define FIRMWARE_MT7961 "mediatek/BT_RAM_CODE_MT7961_1_2_hdr.bin"
+
+#define HCI_WMT_MAX_EVENT_SIZE 64
+
+#define BTMTK_WMT_REG_READ 0x2
+
+enum {
+ BTMTK_WMT_PATCH_DWNLD = 0x1,
+ BTMTK_WMT_TEST = 0x2,
+ BTMTK_WMT_WAKEUP = 0x3,
+ BTMTK_WMT_HIF = 0x4,
+ BTMTK_WMT_FUNC_CTRL = 0x6,
+ BTMTK_WMT_RST = 0x7,
+ BTMTK_WMT_REGISTER = 0x8,
+ BTMTK_WMT_SEMAPHORE = 0x17,
+};
+
+enum {
+ BTMTK_WMT_INVALID,
+ BTMTK_WMT_PATCH_UNDONE,
+ BTMTK_WMT_PATCH_PROGRESS,
+ BTMTK_WMT_PATCH_DONE,
+ BTMTK_WMT_ON_UNDONE,
+ BTMTK_WMT_ON_DONE,
+ BTMTK_WMT_ON_PROGRESS,
+};
+
+struct btmtk_wmt_hdr {
+ u8 dir;
+ u8 op;
+ __le16 dlen;
+ u8 flag;
+} __packed;
+
+struct btmtk_hci_wmt_cmd {
+ struct btmtk_wmt_hdr hdr;
+ u8 data[];
+} __packed;
+
+struct btmtk_hci_wmt_evt {
+ struct hci_event_hdr hhdr;
+ struct btmtk_wmt_hdr whdr;
+} __packed;
+
+struct btmtk_hci_wmt_evt_funcc {
+ struct btmtk_hci_wmt_evt hwhdr;
+ __be16 status;
+} __packed;
+
+struct btmtk_hci_wmt_evt_reg {
+ struct btmtk_hci_wmt_evt hwhdr;
+ u8 rsv[2];
+ u8 num;
+ __le32 addr;
+ __le32 val;
+} __packed;
+
+struct btmtk_tci_sleep {
+ u8 mode;
+ __le16 duration;
+ __le16 host_duration;
+ u8 host_wakeup_pin;
+ u8 time_compensation;
+} __packed;
+
+struct btmtk_hci_wmt_params {
+ u8 op;
+ u8 flag;
+ u16 dlen;
+ const void *data;
+ u32 *status;
+};
+
+typedef int (*wmt_cmd_sync_func_t)(struct hci_dev *,
+ struct btmtk_hci_wmt_params *);
+
+#if IS_ENABLED(CONFIG_BT_MTK)
+
+int btmtk_set_bdaddr(struct hci_dev *hdev, const bdaddr_t *bdaddr);
+
+int btmtk_setup_firmware_79xx(struct hci_dev *hdev, const char *fwname,
+ wmt_cmd_sync_func_t wmt_cmd_sync);
+
+int btmtk_setup_firmware(struct hci_dev *hdev, const char *fwname,
+ wmt_cmd_sync_func_t wmt_cmd_sync);
+#else
+
+static inline int btmtk_set_bdaddr(struct hci_dev *hdev,
+ const bdaddr_t *bdaddr)
+{
+ return -EOPNOTSUPP;
+}
+
+static int btmtk_setup_firmware_79xx(struct hci_dev *hdev, const char *fwname,
+ wmt_cmd_sync_func_t wmt_cmd_sync)
+{
+ return -EOPNOTSUPP;
+}
+
+static int btmtk_setup_firmware(struct hci_dev *hdev, const char *fwname,
+ wmt_cmd_sync_func_t wmt_cmd_sync)
+{
+ return -EOPNOTSUPP;
+}
+
+#endif
diff --git a/drivers/bluetooth/btmtksdio.c b/drivers/bluetooth/btmtksdio.c
index 9872ef1..b5ea8d3 100644
--- a/drivers/bluetooth/btmtksdio.c
+++ b/drivers/bluetooth/btmtksdio.c
@@ -12,7 +12,6 @@
#include <asm/unaligned.h>
#include <linux/atomic.h>
-#include <linux/firmware.h>
#include <linux/init.h>
#include <linux/iopoll.h>
#include <linux/kernel.h>
@@ -28,26 +27,32 @@
#include <net/bluetooth/hci_core.h>
#include "h4_recv.h"
+#include "btmtk.h"
#define VERSION "0.1"
-#define FIRMWARE_MT7663 "mediatek/mt7663pr2h.bin"
-#define FIRMWARE_MT7668 "mediatek/mt7668pr2h.bin"
-
#define MTKBTSDIO_AUTOSUSPEND_DELAY 8000
static bool enable_autosuspend;
struct btmtksdio_data {
const char *fwname;
+ u16 chipid;
};
static const struct btmtksdio_data mt7663_data = {
.fwname = FIRMWARE_MT7663,
+ .chipid = 0x7663,
};
static const struct btmtksdio_data mt7668_data = {
.fwname = FIRMWARE_MT7668,
+ .chipid = 0x7668,
+};
+
+static const struct btmtksdio_data mt7921_data = {
+ .fwname = FIRMWARE_MT7961,
+ .chipid = 0x7921,
};
static const struct sdio_device_id btmtksdio_table[] = {
@@ -55,6 +60,8 @@ static const struct sdio_device_id btmtksdio_table[] = {
.driver_data = (kernel_ulong_t)&mt7663_data },
{SDIO_DEVICE(SDIO_VENDOR_ID_MEDIATEK, SDIO_DEVICE_ID_MEDIATEK_MT7668),
.driver_data = (kernel_ulong_t)&mt7668_data },
+ {SDIO_DEVICE(SDIO_VENDOR_ID_MEDIATEK, SDIO_DEVICE_ID_MEDIATEK_MT7961),
+ .driver_data = (kernel_ulong_t)&mt7921_data },
{ } /* Terminating entry */
};
MODULE_DEVICE_TABLE(sdio, btmtksdio_table);
@@ -86,28 +93,13 @@ MODULE_DEVICE_TABLE(sdio, btmtksdio_table);
#define MTK_REG_CRDR 0x1c
+#define MTK_REG_CRPLR 0x24
+
#define MTK_SDIO_BLOCK_SIZE 256
#define BTMTKSDIO_TX_WAIT_VND_EVT 1
-
-enum {
- MTK_WMT_PATCH_DWNLD = 0x1,
- MTK_WMT_TEST = 0x2,
- MTK_WMT_WAKEUP = 0x3,
- MTK_WMT_HIF = 0x4,
- MTK_WMT_FUNC_CTRL = 0x6,
- MTK_WMT_RST = 0x7,
- MTK_WMT_SEMAPHORE = 0x17,
-};
-
-enum {
- BTMTK_WMT_INVALID,
- BTMTK_WMT_PATCH_UNDONE,
- BTMTK_WMT_PATCH_DONE,
- BTMTK_WMT_ON_UNDONE,
- BTMTK_WMT_ON_DONE,
- BTMTK_WMT_ON_PROGRESS,
-};
+#define BTMTKSDIO_HW_TX_READY 2
+#define BTMTKSDIO_FUNC_ENABLED 3
struct mtkbtsdio_hdr {
__le16 len;
@@ -115,50 +107,12 @@ struct mtkbtsdio_hdr {
u8 bt_type;
} __packed;
-struct mtk_wmt_hdr {
- u8 dir;
- u8 op;
- __le16 dlen;
- u8 flag;
-} __packed;
-
-struct mtk_hci_wmt_cmd {
- struct mtk_wmt_hdr hdr;
- u8 data[256];
-} __packed;
-
-struct btmtk_hci_wmt_evt {
- struct hci_event_hdr hhdr;
- struct mtk_wmt_hdr whdr;
-} __packed;
-
-struct btmtk_hci_wmt_evt_funcc {
- struct btmtk_hci_wmt_evt hwhdr;
- __be16 status;
-} __packed;
-
-struct btmtk_tci_sleep {
- u8 mode;
- __le16 duration;
- __le16 host_duration;
- u8 host_wakeup_pin;
- u8 time_compensation;
-} __packed;
-
-struct btmtk_hci_wmt_params {
- u8 op;
- u8 flag;
- u16 dlen;
- const void *data;
- u32 *status;
-};
-
struct btmtksdio_dev {
struct hci_dev *hdev;
struct sdio_func *func;
struct device *dev;
- struct work_struct tx_work;
+ struct work_struct txrx_work;
unsigned long tx_state;
struct sk_buff_head txq;
@@ -172,29 +126,35 @@ static int mtk_hci_wmt_sync(struct hci_dev *hdev,
{
struct btmtksdio_dev *bdev = hci_get_drvdata(hdev);
struct btmtk_hci_wmt_evt_funcc *wmt_evt_funcc;
+ struct btmtk_hci_wmt_evt_reg *wmt_evt_reg;
u32 hlen, status = BTMTK_WMT_INVALID;
struct btmtk_hci_wmt_evt *wmt_evt;
- struct mtk_hci_wmt_cmd wc;
- struct mtk_wmt_hdr *hdr;
+ struct btmtk_hci_wmt_cmd *wc;
+ struct btmtk_wmt_hdr *hdr;
int err;
+ /* Send the WMT command and wait until the WMT event returns */
hlen = sizeof(*hdr) + wmt_params->dlen;
if (hlen > 255)
return -EINVAL;
- hdr = (struct mtk_wmt_hdr *)&wc;
+ wc = kzalloc(hlen, GFP_KERNEL);
+ if (!wc)
+ return -ENOMEM;
+
+ hdr = &wc->hdr;
hdr->dir = 1;
hdr->op = wmt_params->op;
hdr->dlen = cpu_to_le16(wmt_params->dlen + 1);
hdr->flag = wmt_params->flag;
- memcpy(wc.data, wmt_params->data, wmt_params->dlen);
+ memcpy(wc->data, wmt_params->data, wmt_params->dlen);
set_bit(BTMTKSDIO_TX_WAIT_VND_EVT, &bdev->tx_state);
- err = __hci_cmd_send(hdev, 0xfc6f, hlen, &wc);
+ err = __hci_cmd_send(hdev, 0xfc6f, hlen, wc);
if (err < 0) {
clear_bit(BTMTKSDIO_TX_WAIT_VND_EVT, &bdev->tx_state);
- return err;
+ goto err_free_wc;
}
/* The vendor specific WMT commands are all answered by a vendor
@@ -211,13 +171,14 @@ static int mtk_hci_wmt_sync(struct hci_dev *hdev,
if (err == -EINTR) {
bt_dev_err(hdev, "Execution of wmt command interrupted");
clear_bit(BTMTKSDIO_TX_WAIT_VND_EVT, &bdev->tx_state);
- return err;
+ goto err_free_wc;
}
if (err) {
bt_dev_err(hdev, "Execution of wmt command timed out");
clear_bit(BTMTKSDIO_TX_WAIT_VND_EVT, &bdev->tx_state);
- return -ETIMEDOUT;
+ err = -ETIMEDOUT;
+ goto err_free_wc;
}
/* Parse and handle the return WMT event */
@@ -230,13 +191,13 @@ static int mtk_hci_wmt_sync(struct hci_dev *hdev,
}
switch (wmt_evt->whdr.op) {
- case MTK_WMT_SEMAPHORE:
+ case BTMTK_WMT_SEMAPHORE:
if (wmt_evt->whdr.flag == 2)
status = BTMTK_WMT_PATCH_UNDONE;
else
status = BTMTK_WMT_PATCH_DONE;
break;
- case MTK_WMT_FUNC_CTRL:
+ case BTMTK_WMT_FUNC_CTRL:
wmt_evt_funcc = (struct btmtk_hci_wmt_evt_funcc *)wmt_evt;
if (be16_to_cpu(wmt_evt_funcc->status) == 0x404)
status = BTMTK_WMT_ON_DONE;
@@ -245,6 +206,19 @@ static int mtk_hci_wmt_sync(struct hci_dev *hdev,
else
status = BTMTK_WMT_ON_UNDONE;
break;
+ case BTMTK_WMT_PATCH_DWNLD:
+ if (wmt_evt->whdr.flag == 2)
+ status = BTMTK_WMT_PATCH_DONE;
+ else if (wmt_evt->whdr.flag == 1)
+ status = BTMTK_WMT_PATCH_PROGRESS;
+ else
+ status = BTMTK_WMT_PATCH_UNDONE;
+ break;
+ case BTMTK_WMT_REGISTER:
+ wmt_evt_reg = (struct btmtk_hci_wmt_evt_reg *)wmt_evt;
+ if (le16_to_cpu(wmt_evt->whdr.dlen) == 12)
+ status = le32_to_cpu(wmt_evt_reg->val);
+ break;
}
if (wmt_params->status)
@@ -253,6 +227,8 @@ static int mtk_hci_wmt_sync(struct hci_dev *hdev,
err_free_skb:
kfree_skb(bdev->evt_skb);
bdev->evt_skb = NULL;
+err_free_wc:
+ kfree(wc);
return err;
}
@@ -279,6 +255,7 @@ static int btmtksdio_tx_packet(struct btmtksdio_dev *bdev,
sdio_hdr->reserved = cpu_to_le16(0);
sdio_hdr->bt_type = hci_skb_pkt_type(skb);
+ clear_bit(BTMTKSDIO_HW_TX_READY, &bdev->tx_state);
err = sdio_writesb(bdev->func, MTK_REG_CTDR, skb->data,
round_up(skb->len, MTK_SDIO_BLOCK_SIZE));
if (err < 0)
@@ -301,32 +278,6 @@ static u32 btmtksdio_drv_own_query(struct btmtksdio_dev *bdev)
return sdio_readl(bdev->func, MTK_REG_CHLPCR, NULL);
}
-static void btmtksdio_tx_work(struct work_struct *work)
-{
- struct btmtksdio_dev *bdev = container_of(work, struct btmtksdio_dev,
- tx_work);
- struct sk_buff *skb;
- int err;
-
- pm_runtime_get_sync(bdev->dev);
-
- sdio_claim_host(bdev->func);
-
- while ((skb = skb_dequeue(&bdev->txq))) {
- err = btmtksdio_tx_packet(bdev, skb);
- if (err < 0) {
- bdev->hdev->stat.err_tx++;
- skb_queue_head(&bdev->txq, skb);
- break;
- }
- }
-
- sdio_release_host(bdev->func);
-
- pm_runtime_mark_last_busy(bdev->dev);
- pm_runtime_put_autosuspend(bdev->dev);
-}
-
static int btmtksdio_recv_event(struct hci_dev *hdev, struct sk_buff *skb)
{
struct btmtksdio_dev *bdev = hci_get_drvdata(hdev);
@@ -374,8 +325,29 @@ static int btmtksdio_recv_event(struct hci_dev *hdev, struct sk_buff *skb)
return err;
}
+static int btmtksdio_recv_acl(struct hci_dev *hdev, struct sk_buff *skb)
+{
+ struct btmtksdio_dev *bdev = hci_get_drvdata(hdev);
+ u16 handle = le16_to_cpu(hci_acl_hdr(skb)->handle);
+
+ switch (handle) {
+ case 0xfc6f:
+ /* Firmware dump from device: when the firmware hangs, the
+ * device can no longer suspend and thus disable auto-suspend.
+ */
+ pm_runtime_forbid(bdev->dev);
+ fallthrough;
+ case 0x05ff:
+ case 0x05fe:
+ /* Firmware debug logging */
+ return hci_recv_diag(hdev, skb);
+ }
+
+ return hci_recv_frame(hdev, skb);
+}
+
static const struct h4_recv_pkt mtk_recv_pkts[] = {
- { H4_RECV_ACL, .recv = hci_recv_frame },
+ { H4_RECV_ACL, .recv = btmtksdio_recv_acl },
{ H4_RECV_SCO, .recv = hci_recv_frame },
{ H4_RECV_EVENT, .recv = btmtksdio_recv_event },
};
@@ -477,70 +449,90 @@ static int btmtksdio_rx_packet(struct btmtksdio_dev *bdev, u16 rx_size)
return err;
}
-static void btmtksdio_interrupt(struct sdio_func *func)
+static void btmtksdio_txrx_work(struct work_struct *work)
{
- struct btmtksdio_dev *bdev = sdio_get_drvdata(func);
- u32 int_status;
- u16 rx_size;
-
- /* It is required that the host gets ownership from the device before
- * accessing any register, however, if SDIO host is not being released,
- * a potential deadlock probably happens in a circular wait between SDIO
- * IRQ work and PM runtime work. So, we have to explicitly release SDIO
- * host here and claim again after the PM runtime work is all done.
- */
- sdio_release_host(bdev->func);
+ struct btmtksdio_dev *bdev = container_of(work, struct btmtksdio_dev,
+ txrx_work);
+ unsigned long txrx_timeout;
+ u32 int_status, rx_size;
+ struct sk_buff *skb;
+ int err;
pm_runtime_get_sync(bdev->dev);
sdio_claim_host(bdev->func);
/* Disable interrupt */
- sdio_writel(func, C_INT_EN_CLR, MTK_REG_CHLPCR, NULL);
+ sdio_writel(bdev->func, C_INT_EN_CLR, MTK_REG_CHLPCR, 0);
- int_status = sdio_readl(func, MTK_REG_CHISR, NULL);
+ txrx_timeout = jiffies + 5 * HZ;
- /* Ack an interrupt as soon as possible before any operation on
- * hardware.
- *
- * Note that we don't ack any status during operations to avoid race
- * condition between the host and the device such as it's possible to
- * mistakenly ack RX_DONE for the next packet and then cause interrupts
- * not be raised again but there is still pending data in the hardware
- * FIFO.
- */
- sdio_writel(func, int_status, MTK_REG_CHISR, NULL);
+ do {
+ int_status = sdio_readl(bdev->func, MTK_REG_CHISR, NULL);
- if (unlikely(!int_status))
- bt_dev_err(bdev->hdev, "CHISR is 0");
+ /* Ack an interrupt as soon as possible before any operation on
+ * hardware.
+ *
+ * Note that we don't ack any status during operations to avoid race
+ * condition between the host and the device such as it's possible to
+ * mistakenly ack RX_DONE for the next packet and then cause interrupts
+ * not be raised again but there is still pending data in the hardware
+ * FIFO.
+ */
+ sdio_writel(bdev->func, int_status, MTK_REG_CHISR, NULL);
- if (int_status & FW_OWN_BACK_INT)
- bt_dev_dbg(bdev->hdev, "Get fw own back");
+ if (int_status & FW_OWN_BACK_INT)
+ bt_dev_dbg(bdev->hdev, "Get fw own back");
- if (int_status & TX_EMPTY)
- schedule_work(&bdev->tx_work);
- else if (unlikely(int_status & TX_FIFO_OVERFLOW))
- bt_dev_warn(bdev->hdev, "Tx fifo overflow");
+ if (int_status & TX_EMPTY)
+ set_bit(BTMTKSDIO_HW_TX_READY, &bdev->tx_state);
- if (int_status & RX_DONE_INT) {
- rx_size = (int_status & RX_PKT_LEN) >> 16;
+ else if (unlikely(int_status & TX_FIFO_OVERFLOW))
+ bt_dev_warn(bdev->hdev, "Tx fifo overflow");
- if (btmtksdio_rx_packet(bdev, rx_size) < 0)
- bdev->hdev->stat.err_rx++;
- }
+ if (test_bit(BTMTKSDIO_HW_TX_READY, &bdev->tx_state)) {
+ skb = skb_dequeue(&bdev->txq);
+ if (skb) {
+ err = btmtksdio_tx_packet(bdev, skb);
+ if (err < 0) {
+ bdev->hdev->stat.err_tx++;
+ skb_queue_head(&bdev->txq, skb);
+ }
+ }
+ }
+
+ if (int_status & RX_DONE_INT) {
+ rx_size = sdio_readl(bdev->func, MTK_REG_CRPLR, NULL);
+ rx_size = (rx_size & RX_PKT_LEN) >> 16;
+ if (btmtksdio_rx_packet(bdev, rx_size) < 0)
+ bdev->hdev->stat.err_rx++;
+ }
+ } while (int_status || time_is_before_jiffies(txrx_timeout));
/* Enable interrupt */
- sdio_writel(func, C_INT_EN_SET, MTK_REG_CHLPCR, NULL);
+ sdio_writel(bdev->func, C_INT_EN_SET, MTK_REG_CHLPCR, 0);
+
+ sdio_release_host(bdev->func);
pm_runtime_mark_last_busy(bdev->dev);
pm_runtime_put_autosuspend(bdev->dev);
}
+static void btmtksdio_interrupt(struct sdio_func *func)
+{
+ struct btmtksdio_dev *bdev = sdio_get_drvdata(func);
+
+ /* Disable interrupt */
+ sdio_writel(bdev->func, C_INT_EN_CLR, MTK_REG_CHLPCR, 0);
+
+ schedule_work(&bdev->txrx_work);
+}
+
static int btmtksdio_open(struct hci_dev *hdev)
{
struct btmtksdio_dev *bdev = hci_get_drvdata(hdev);
+ u32 status, val;
int err;
- u32 status;
sdio_claim_host(bdev->func);
@@ -548,6 +540,8 @@ static int btmtksdio_open(struct hci_dev *hdev)
if (err < 0)
goto err_release_host;
+ set_bit(BTMTKSDIO_FUNC_ENABLED, &bdev->tx_state);
+
/* Get ownership from the device */
sdio_writel(bdev->func, C_FW_OWN_REQ_CLR, MTK_REG_CHLPCR, &err);
if (err < 0)
@@ -580,13 +574,22 @@ static int btmtksdio_open(struct hci_dev *hdev)
/* SDIO CMD 5 allows the SDIO device back to idle state an
* synchronous interrupt is supported in SDIO 4-bit mode
*/
- sdio_writel(bdev->func, SDIO_INT_CTL | SDIO_RE_INIT_EN,
- MTK_REG_CSDIOCSR, &err);
+ val = sdio_readl(bdev->func, MTK_REG_CSDIOCSR, &err);
if (err < 0)
goto err_release_irq;
- /* Setup write-1-clear for CHISR register */
- sdio_writel(bdev->func, C_INT_CLR_CTRL, MTK_REG_CHCR, &err);
+ val |= SDIO_INT_CTL;
+ sdio_writel(bdev->func, val, MTK_REG_CSDIOCSR, &err);
+ if (err < 0)
+ goto err_release_irq;
+
+ /* Explitly set write-1-clear method */
+ val = sdio_readl(bdev->func, MTK_REG_CHCR, &err);
+ if (err < 0)
+ goto err_release_irq;
+
+ val |= C_INT_CLR_CTRL;
+ sdio_writel(bdev->func, val, MTK_REG_CHCR, &err);
if (err < 0)
goto err_release_irq;
@@ -630,6 +633,8 @@ static int btmtksdio_close(struct hci_dev *hdev)
sdio_release_irq(bdev->func);
+ cancel_work_sync(&bdev->txrx_work);
+
/* Return ownership to the device */
sdio_writel(bdev->func, C_FW_OWN_REQ_SET, MTK_REG_CHLPCR, NULL);
@@ -638,6 +643,7 @@ static int btmtksdio_close(struct hci_dev *hdev)
if (err < 0)
bt_dev_err(bdev->hdev, "Cannot return ownership to device");
+ clear_bit(BTMTKSDIO_FUNC_ENABLED, &bdev->tx_state);
sdio_disable_func(bdev->func);
sdio_release_host(bdev->func);
@@ -651,7 +657,7 @@ static int btmtksdio_flush(struct hci_dev *hdev)
skb_queue_purge(&bdev->txq);
- cancel_work_sync(&bdev->tx_work);
+ cancel_work_sync(&bdev->txrx_work);
return 0;
}
@@ -663,7 +669,7 @@ static int btmtksdio_func_query(struct hci_dev *hdev)
u8 param = 0;
/* Query whether the function is enabled */
- wmt_params.op = MTK_WMT_FUNC_CTRL;
+ wmt_params.op = BTMTK_WMT_FUNC_CTRL;
wmt_params.flag = 4;
wmt_params.dlen = sizeof(param);
wmt_params.data = ¶m;
@@ -678,111 +684,16 @@ static int btmtksdio_func_query(struct hci_dev *hdev)
return status;
}
-static int mtk_setup_firmware(struct hci_dev *hdev, const char *fwname)
+static int mt76xx_setup(struct hci_dev *hdev, const char *fwname)
{
struct btmtk_hci_wmt_params wmt_params;
- const struct firmware *fw;
- const u8 *fw_ptr;
- size_t fw_size;
- int err, dlen;
- u8 flag, param;
-
- err = request_firmware(&fw, fwname, &hdev->dev);
- if (err < 0) {
- bt_dev_err(hdev, "Failed to load firmware file (%d)", err);
- return err;
- }
-
- /* Power on data RAM the firmware relies on. */
- param = 1;
- wmt_params.op = MTK_WMT_FUNC_CTRL;
- wmt_params.flag = 3;
- wmt_params.dlen = sizeof(param);
- wmt_params.data = ¶m;
- wmt_params.status = NULL;
-
- err = mtk_hci_wmt_sync(hdev, &wmt_params);
- if (err < 0) {
- bt_dev_err(hdev, "Failed to power on data RAM (%d)", err);
- goto free_fw;
- }
-
- fw_ptr = fw->data;
- fw_size = fw->size;
-
- /* The size of patch header is 30 bytes, should be skip */
- if (fw_size < 30) {
- err = -EINVAL;
- goto free_fw;
- }
-
- fw_size -= 30;
- fw_ptr += 30;
- flag = 1;
-
- wmt_params.op = MTK_WMT_PATCH_DWNLD;
- wmt_params.status = NULL;
-
- while (fw_size > 0) {
- dlen = min_t(int, 250, fw_size);
-
- /* Tell device the position in sequence */
- if (fw_size - dlen <= 0)
- flag = 3;
- else if (fw_size < fw->size - 30)
- flag = 2;
-
- wmt_params.flag = flag;
- wmt_params.dlen = dlen;
- wmt_params.data = fw_ptr;
-
- err = mtk_hci_wmt_sync(hdev, &wmt_params);
- if (err < 0) {
- bt_dev_err(hdev, "Failed to send wmt patch dwnld (%d)",
- err);
- goto free_fw;
- }
-
- fw_size -= dlen;
- fw_ptr += dlen;
- }
-
- wmt_params.op = MTK_WMT_RST;
- wmt_params.flag = 4;
- wmt_params.dlen = 0;
- wmt_params.data = NULL;
- wmt_params.status = NULL;
-
- /* Activate funciton the firmware providing to */
- err = mtk_hci_wmt_sync(hdev, &wmt_params);
- if (err < 0) {
- bt_dev_err(hdev, "Failed to send wmt rst (%d)", err);
- goto free_fw;
- }
-
- /* Wait a few moments for firmware activation done */
- usleep_range(10000, 12000);
-
-free_fw:
- release_firmware(fw);
- return err;
-}
-
-static int btmtksdio_setup(struct hci_dev *hdev)
-{
- struct btmtksdio_dev *bdev = hci_get_drvdata(hdev);
- struct btmtk_hci_wmt_params wmt_params;
- ktime_t calltime, delta, rettime;
struct btmtk_tci_sleep tci_sleep;
- unsigned long long duration;
struct sk_buff *skb;
int err, status;
u8 param = 0x1;
- calltime = ktime_get();
-
/* Query whether the firmware is already download */
- wmt_params.op = MTK_WMT_SEMAPHORE;
+ wmt_params.op = BTMTK_WMT_SEMAPHORE;
wmt_params.flag = 1;
wmt_params.dlen = 0;
wmt_params.data = NULL;
@@ -800,7 +711,7 @@ static int btmtksdio_setup(struct hci_dev *hdev)
}
/* Setup a firmware which the device definitely requires */
- err = mtk_setup_firmware(hdev, bdev->data->fwname);
+ err = btmtk_setup_firmware(hdev, fwname, mtk_hci_wmt_sync);
if (err < 0)
return err;
@@ -823,7 +734,7 @@ static int btmtksdio_setup(struct hci_dev *hdev)
}
/* Enable Bluetooth protocol */
- wmt_params.op = MTK_WMT_FUNC_CTRL;
+ wmt_params.op = BTMTK_WMT_FUNC_CTRL;
wmt_params.flag = 0;
wmt_params.dlen = sizeof(param);
wmt_params.data = ¶m;
@@ -852,6 +763,116 @@ static int btmtksdio_setup(struct hci_dev *hdev)
}
kfree_skb(skb);
+ return 0;
+}
+
+static int mt79xx_setup(struct hci_dev *hdev, const char *fwname)
+{
+ struct btmtk_hci_wmt_params wmt_params;
+ u8 param = 0x1;
+ int err;
+
+ err = btmtk_setup_firmware_79xx(hdev, fwname, mtk_hci_wmt_sync);
+ if (err < 0) {
+ bt_dev_err(hdev, "Failed to setup 79xx firmware (%d)", err);
+ return err;
+ }
+
+ /* Enable Bluetooth protocol */
+ wmt_params.op = BTMTK_WMT_FUNC_CTRL;
+ wmt_params.flag = 0;
+ wmt_params.dlen = sizeof(param);
+ wmt_params.data = ¶m;
+ wmt_params.status = NULL;
+
+ err = mtk_hci_wmt_sync(hdev, &wmt_params);
+ if (err < 0) {
+ bt_dev_err(hdev, "Failed to send wmt func ctrl (%d)", err);
+ return err;
+ }
+
+ hci_set_msft_opcode(hdev, 0xFD30);
+ hci_set_aosp_capable(hdev);
+
+ return err;
+}
+
+static int btsdio_mtk_reg_read(struct hci_dev *hdev, u32 reg, u32 *val)
+{
+ struct btmtk_hci_wmt_params wmt_params;
+ struct reg_read_cmd {
+ u8 type;
+ u8 rsv;
+ u8 num;
+ __le32 addr;
+ } __packed reg_read = {
+ .type = 1,
+ .num = 1,
+ };
+ u32 status;
+ int err;
+
+ reg_read.addr = cpu_to_le32(reg);
+ wmt_params.op = BTMTK_WMT_REGISTER;
+ wmt_params.flag = BTMTK_WMT_REG_READ;
+ wmt_params.dlen = sizeof(reg_read);
+ wmt_params.data = ®_read;
+ wmt_params.status = &status;
+
+ err = mtk_hci_wmt_sync(hdev, &wmt_params);
+ if (err < 0) {
+ bt_dev_err(hdev, "Failed to read reg(%d)", err);
+ return err;
+ }
+
+ *val = status;
+
+ return err;
+}
+
+static int btmtksdio_setup(struct hci_dev *hdev)
+{
+ struct btmtksdio_dev *bdev = hci_get_drvdata(hdev);
+ ktime_t calltime, delta, rettime;
+ unsigned long long duration;
+ char fwname[64];
+ int err, dev_id;
+ u32 fw_version = 0;
+
+ calltime = ktime_get();
+ set_bit(BTMTKSDIO_HW_TX_READY, &bdev->tx_state);
+
+ switch (bdev->data->chipid) {
+ case 0x7921:
+ err = btsdio_mtk_reg_read(hdev, 0x70010200, &dev_id);
+ if (err < 0) {
+ bt_dev_err(hdev, "Failed to get device id (%d)", err);
+ return err;
+ }
+
+ err = btsdio_mtk_reg_read(hdev, 0x80021004, &fw_version);
+ if (err < 0) {
+ bt_dev_err(hdev, "Failed to get fw version (%d)", err);
+ return err;
+ }
+
+ snprintf(fwname, sizeof(fwname),
+ "mediatek/BT_RAM_CODE_MT%04x_1_%x_hdr.bin",
+ dev_id & 0xffff, (fw_version & 0xff) + 1);
+ err = mt79xx_setup(hdev, fwname);
+ if (err < 0)
+ return err;
+ break;
+ case 0x7663:
+ case 0x7668:
+ err = mt76xx_setup(hdev, bdev->data->fwname);
+ if (err < 0)
+ return err;
+ break;
+ default:
+ return -ENODEV;
+ }
+
rettime = ktime_get();
delta = ktime_sub(rettime, calltime);
duration = (unsigned long long)ktime_to_ns(delta) >> 10;
@@ -891,7 +912,7 @@ static int btmtksdio_shutdown(struct hci_dev *hdev)
pm_runtime_get_sync(bdev->dev);
/* Disable the device */
- wmt_params.op = MTK_WMT_FUNC_CTRL;
+ wmt_params.op = BTMTK_WMT_FUNC_CTRL;
wmt_params.flag = 0;
wmt_params.dlen = sizeof(param);
wmt_params.data = ¶m;
@@ -932,7 +953,7 @@ static int btmtksdio_send_frame(struct hci_dev *hdev, struct sk_buff *skb)
skb_queue_tail(&bdev->txq, skb);
- schedule_work(&bdev->tx_work);
+ schedule_work(&bdev->txrx_work);
return 0;
}
@@ -955,7 +976,7 @@ static int btmtksdio_probe(struct sdio_func *func,
bdev->dev = &func->dev;
bdev->func = func;
- INIT_WORK(&bdev->tx_work, btmtksdio_tx_work);
+ INIT_WORK(&bdev->txrx_work, btmtksdio_txrx_work);
skb_queue_head_init(&bdev->txq);
/* Initialize and register HCI device */
@@ -976,6 +997,8 @@ static int btmtksdio_probe(struct sdio_func *func,
hdev->setup = btmtksdio_setup;
hdev->shutdown = btmtksdio_shutdown;
hdev->send = btmtksdio_send_frame;
+ hdev->set_bdaddr = btmtk_set_bdaddr;
+
SET_HCIDEV_DEV(hdev, &func->dev);
hdev->manufacturer = 70;
@@ -1042,6 +1065,11 @@ static int btmtksdio_runtime_suspend(struct device *dev)
if (!bdev)
return 0;
+ if (!test_bit(BTMTKSDIO_FUNC_ENABLED, &bdev->tx_state))
+ return 0;
+
+ sdio_set_host_pm_flags(func, MMC_PM_KEEP_POWER);
+
sdio_claim_host(bdev->func);
sdio_writel(bdev->func, C_FW_OWN_REQ_SET, MTK_REG_CHLPCR, &err);
@@ -1069,6 +1097,9 @@ static int btmtksdio_runtime_resume(struct device *dev)
if (!bdev)
return 0;
+ if (!test_bit(BTMTKSDIO_FUNC_ENABLED, &bdev->tx_state))
+ return 0;
+
sdio_claim_host(bdev->func);
sdio_writel(bdev->func, C_FW_OWN_REQ_CLR, MTK_REG_CHLPCR, &err);
@@ -1112,5 +1143,3 @@ MODULE_AUTHOR("Sean Wang <sean.wang@mediatek.com>");
MODULE_DESCRIPTION("MediaTek Bluetooth SDIO driver ver " VERSION);
MODULE_VERSION(VERSION);
MODULE_LICENSE("GPL");
-MODULE_FIRMWARE(FIRMWARE_MT7663);
-MODULE_FIRMWARE(FIRMWARE_MT7668);
diff --git a/drivers/bluetooth/btqca.c b/drivers/bluetooth/btqca.c
index be04d74..c9064d3 100644
--- a/drivers/bluetooth/btqca.c
+++ b/drivers/bluetooth/btqca.c
@@ -6,6 +6,7 @@
*/
#include <linux/module.h>
#include <linux/firmware.h>
+#include <linux/vmalloc.h>
#include <net/bluetooth/bluetooth.h>
#include <net/bluetooth/hci_core.h>
@@ -141,6 +142,50 @@ static int qca_read_fw_build_info(struct hci_dev *hdev)
return err;
}
+static int qca_send_patch_config_cmd(struct hci_dev *hdev)
+{
+ const u8 cmd[] = { EDL_PATCH_CONFIG_CMD, 0x01, 0, 0, 0 };
+ struct sk_buff *skb;
+ struct edl_event_hdr *edl;
+ int err;
+
+ bt_dev_dbg(hdev, "QCA Patch config");
+
+ skb = __hci_cmd_sync_ev(hdev, EDL_PATCH_CMD_OPCODE, sizeof(cmd),
+ cmd, HCI_EV_VENDOR, HCI_INIT_TIMEOUT);
+ if (IS_ERR(skb)) {
+ err = PTR_ERR(skb);
+ bt_dev_err(hdev, "Sending QCA Patch config failed (%d)", err);
+ return err;
+ }
+
+ if (skb->len != 2) {
+ bt_dev_err(hdev, "QCA Patch config cmd size mismatch len %d", skb->len);
+ err = -EILSEQ;
+ goto out;
+ }
+
+ edl = (struct edl_event_hdr *)(skb->data);
+ if (!edl) {
+ bt_dev_err(hdev, "QCA Patch config with no header");
+ err = -EILSEQ;
+ goto out;
+ }
+
+ if (edl->cresp != EDL_PATCH_CONFIG_RES_EVT || edl->rtype != EDL_PATCH_CONFIG_CMD) {
+ bt_dev_err(hdev, "QCA Wrong packet received %d %d", edl->cresp,
+ edl->rtype);
+ err = -EIO;
+ goto out;
+ }
+
+ err = 0;
+
+out:
+ kfree_skb(skb);
+ return err;
+}
+
static int qca_send_reset(struct hci_dev *hdev)
{
struct sk_buff *skb;
@@ -551,6 +596,9 @@ int qca_uart_setup(struct hci_dev *hdev, uint8_t baudrate,
*/
rom_ver = ((soc_ver & 0x00000f00) >> 0x04) | (soc_ver & 0x0000000f);
+ if (soc_type == QCA_WCN6750)
+ qca_send_patch_config_cmd(hdev);
+
/* Download rampatch file */
config.type = TLV_TYPE_PATCH;
if (qca_is_wcn399x(soc_type)) {
diff --git a/drivers/bluetooth/btqca.h b/drivers/bluetooth/btqca.h
index 30afa77..61e9a50 100644
--- a/drivers/bluetooth/btqca.h
+++ b/drivers/bluetooth/btqca.h
@@ -13,6 +13,7 @@
#define EDL_PATCH_TLV_REQ_CMD (0x1E)
#define EDL_GET_BUILD_INFO_CMD (0x20)
#define EDL_NVM_ACCESS_SET_REQ_CMD (0x01)
+#define EDL_PATCH_CONFIG_CMD (0x28)
#define MAX_SIZE_PER_TLV_SEGMENT (243)
#define QCA_PRE_SHUTDOWN_CMD (0xFC08)
#define QCA_DISABLE_LOGGING (0xFC17)
@@ -24,6 +25,7 @@
#define EDL_CMD_EXE_STATUS_EVT (0x00)
#define EDL_SET_BAUDRATE_RSP_EVT (0x92)
#define EDL_NVM_ACCESS_CODE_EVT (0x0B)
+#define EDL_PATCH_CONFIG_RES_EVT (0x00)
#define QCA_DISABLE_LOGGING_SUB_OP (0x14)
#define EDL_TAG_ID_HCI (17)
diff --git a/drivers/bluetooth/btsdio.c b/drivers/bluetooth/btsdio.c
index 199e8f7..795be33 100644
--- a/drivers/bluetooth/btsdio.c
+++ b/drivers/bluetooth/btsdio.c
@@ -295,6 +295,8 @@ static int btsdio_probe(struct sdio_func *func,
switch (func->device) {
case SDIO_DEVICE_ID_BROADCOM_43341:
case SDIO_DEVICE_ID_BROADCOM_43430:
+ case SDIO_DEVICE_ID_BROADCOM_4345:
+ case SDIO_DEVICE_ID_BROADCOM_43455:
case SDIO_DEVICE_ID_BROADCOM_4356:
return -ENODEV;
}
diff --git a/drivers/bluetooth/btusb.c b/drivers/bluetooth/btusb.c
index 75c8376..c30d131 100644
--- a/drivers/bluetooth/btusb.c
+++ b/drivers/bluetooth/btusb.c
@@ -16,6 +16,7 @@
#include <linux/of_irq.h>
#include <linux/suspend.h>
#include <linux/gpio/consumer.h>
+#include <linux/debugfs.h>
#include <asm/unaligned.h>
#include <net/bluetooth/bluetooth.h>
@@ -24,13 +25,13 @@
#include "btintel.h"
#include "btbcm.h"
#include "btrtl.h"
+#include "btmtk.h"
#define VERSION "0.8"
static bool disable_scofix;
static bool force_scofix;
static bool enable_autosuspend = IS_ENABLED(CONFIG_BT_HCIBTUSB_AUTOSUSPEND);
-
static bool reset = true;
static struct usb_driver btusb_driver;
@@ -59,6 +60,7 @@ static struct usb_driver btusb_driver;
#define BTUSB_WIDEBAND_SPEECH 0x400000
#define BTUSB_VALID_LE_STATES 0x800000
#define BTUSB_QCA_WCN6855 0x1000000
+#define BTUSB_INTEL_BROKEN_SHUTDOWN_LED 0x2000000
#define BTUSB_INTEL_BROKEN_INITIAL_NCMD 0x4000000
static const struct usb_device_id btusb_table[] = {
@@ -295,6 +297,24 @@ static const struct usb_device_id blacklist_table[] = {
{ USB_DEVICE(0x0cf3, 0xe600), .driver_info = BTUSB_QCA_WCN6855 |
BTUSB_WIDEBAND_SPEECH |
BTUSB_VALID_LE_STATES },
+ { USB_DEVICE(0x0489, 0xe0cc), .driver_info = BTUSB_QCA_WCN6855 |
+ BTUSB_WIDEBAND_SPEECH |
+ BTUSB_VALID_LE_STATES },
+ { USB_DEVICE(0x0489, 0xe0d6), .driver_info = BTUSB_QCA_WCN6855 |
+ BTUSB_WIDEBAND_SPEECH |
+ BTUSB_VALID_LE_STATES },
+ { USB_DEVICE(0x0489, 0xe0e3), .driver_info = BTUSB_QCA_WCN6855 |
+ BTUSB_WIDEBAND_SPEECH |
+ BTUSB_VALID_LE_STATES },
+ { USB_DEVICE(0x10ab, 0x9309), .driver_info = BTUSB_QCA_WCN6855 |
+ BTUSB_WIDEBAND_SPEECH |
+ BTUSB_VALID_LE_STATES },
+ { USB_DEVICE(0x10ab, 0x9409), .driver_info = BTUSB_QCA_WCN6855 |
+ BTUSB_WIDEBAND_SPEECH |
+ BTUSB_VALID_LE_STATES },
+ { USB_DEVICE(0x0489, 0xe0d0), .driver_info = BTUSB_QCA_WCN6855 |
+ BTUSB_WIDEBAND_SPEECH |
+ BTUSB_VALID_LE_STATES },
/* Broadcom BCM2035 */
{ USB_DEVICE(0x0a5c, 0x2009), .driver_info = BTUSB_BCM92035 },
@@ -365,10 +385,13 @@ static const struct usb_device_id blacklist_table[] = {
{ USB_DEVICE(0x8087, 0x0033), .driver_info = BTUSB_INTEL_COMBINED },
{ USB_DEVICE(0x8087, 0x07da), .driver_info = BTUSB_CSR },
{ USB_DEVICE(0x8087, 0x07dc), .driver_info = BTUSB_INTEL_COMBINED |
- BTUSB_INTEL_BROKEN_INITIAL_NCMD },
- { USB_DEVICE(0x8087, 0x0a2a), .driver_info = BTUSB_INTEL_COMBINED },
+ BTUSB_INTEL_BROKEN_INITIAL_NCMD |
+ BTUSB_INTEL_BROKEN_SHUTDOWN_LED },
+ { USB_DEVICE(0x8087, 0x0a2a), .driver_info = BTUSB_INTEL_COMBINED |
+ BTUSB_INTEL_BROKEN_SHUTDOWN_LED },
{ USB_DEVICE(0x8087, 0x0a2b), .driver_info = BTUSB_INTEL_COMBINED },
- { USB_DEVICE(0x8087, 0x0aa7), .driver_info = BTUSB_INTEL_COMBINED },
+ { USB_DEVICE(0x8087, 0x0aa7), .driver_info = BTUSB_INTEL_COMBINED |
+ BTUSB_INTEL_BROKEN_SHUTDOWN_LED },
{ USB_DEVICE(0x8087, 0x0aaa), .driver_info = BTUSB_INTEL_COMBINED },
/* Other Intel Bluetooth devices */
@@ -384,6 +407,8 @@ static const struct usb_device_id blacklist_table[] = {
/* Realtek 8852AE Bluetooth devices */
{ USB_DEVICE(0x0bda, 0xc852), .driver_info = BTUSB_REALTEK |
BTUSB_WIDEBAND_SPEECH },
+ { USB_DEVICE(0x0bda, 0x385a), .driver_info = BTUSB_REALTEK |
+ BTUSB_WIDEBAND_SPEECH },
{ USB_DEVICE(0x0bda, 0x4852), .driver_info = BTUSB_REALTEK |
BTUSB_WIDEBAND_SPEECH },
{ USB_DEVICE(0x04c5, 0x165c), .driver_info = BTUSB_REALTEK |
@@ -423,6 +448,14 @@ static const struct usb_device_id blacklist_table[] = {
BTUSB_WIDEBAND_SPEECH |
BTUSB_VALID_LE_STATES },
+ /* MediaTek MT7922A Bluetooth devices */
+ { USB_DEVICE(0x0489, 0xe0d8), .driver_info = BTUSB_MEDIATEK |
+ BTUSB_WIDEBAND_SPEECH |
+ BTUSB_VALID_LE_STATES },
+ { USB_DEVICE(0x0489, 0xe0d9), .driver_info = BTUSB_MEDIATEK |
+ BTUSB_WIDEBAND_SPEECH |
+ BTUSB_VALID_LE_STATES },
+
/* Additional Realtek 8723AE Bluetooth devices */
{ USB_DEVICE(0x0930, 0x021d), .driver_info = BTUSB_REALTEK },
{ USB_DEVICE(0x13d3, 0x3394), .driver_info = BTUSB_REALTEK },
@@ -549,8 +582,13 @@ struct btusb_data {
unsigned long flags;
- struct work_struct work;
- struct work_struct waker;
+ bool poll_sync;
+ int intr_interval;
+ struct work_struct work;
+ struct work_struct waker;
+ struct delayed_work rx_work;
+
+ struct sk_buff_head acl_q;
struct usb_anchor deferred;
struct usb_anchor tx_anchor;
@@ -715,6 +753,16 @@ static inline void btusb_free_frags(struct btusb_data *data)
spin_unlock_irqrestore(&data->rxlock, flags);
}
+static int btusb_recv_event(struct btusb_data *data, struct sk_buff *skb)
+{
+ if (data->intr_interval) {
+ /* Trigger dequeue immediatelly if an event is received */
+ schedule_delayed_work(&data->rx_work, 0);
+ }
+
+ return data->recv_event(data->hdev, skb);
+}
+
static int btusb_recv_intr(struct btusb_data *data, void *buffer, int count)
{
struct sk_buff *skb;
@@ -760,7 +808,7 @@ static int btusb_recv_intr(struct btusb_data *data, void *buffer, int count)
if (!hci_skb_expect(skb)) {
/* Complete frame */
- data->recv_event(data->hdev, skb);
+ btusb_recv_event(data, skb);
skb = NULL;
}
}
@@ -771,6 +819,20 @@ static int btusb_recv_intr(struct btusb_data *data, void *buffer, int count)
return err;
}
+static int btusb_recv_acl(struct btusb_data *data, struct sk_buff *skb)
+{
+ /* Only queue ACL packet if intr_interval is set as it means
+ * force_poll_sync has been enabled.
+ */
+ if (!data->intr_interval)
+ return data->recv_acl(data->hdev, skb);
+
+ skb_queue_tail(&data->acl_q, skb);
+ schedule_delayed_work(&data->rx_work, data->intr_interval);
+
+ return 0;
+}
+
static int btusb_recv_bulk(struct btusb_data *data, void *buffer, int count)
{
struct sk_buff *skb;
@@ -818,7 +880,7 @@ static int btusb_recv_bulk(struct btusb_data *data, void *buffer, int count)
if (!hci_skb_expect(skb)) {
/* Complete frame */
- data->recv_acl(data->hdev, skb);
+ btusb_recv_acl(data, skb);
skb = NULL;
}
}
@@ -924,6 +986,8 @@ static void btusb_intr_complete(struct urb *urb)
if (err != -EPERM && err != -ENODEV)
bt_dev_err(hdev, "urb %p failed to resubmit (%d)",
urb, -err);
+ if (err != -EPERM)
+ hci_cmd_sync_cancel(hdev, -err);
usb_unanchor_urb(urb);
}
}
@@ -967,9 +1031,33 @@ static int btusb_submit_intr_urb(struct hci_dev *hdev, gfp_t mem_flags)
if (err != -EPERM && err != -ENODEV)
bt_dev_err(hdev, "urb %p submission failed (%d)",
urb, -err);
+ if (err != -EPERM)
+ hci_cmd_sync_cancel(hdev, -err);
usb_unanchor_urb(urb);
}
+ /* Only initialize intr_interval if URB poll sync is enabled */
+ if (!data->poll_sync)
+ goto done;
+
+ /* The units are frames (milliseconds) for full and low speed devices,
+ * and microframes (1/8 millisecond) for highspeed and SuperSpeed
+ * devices.
+ *
+ * This is done once on open/resume so it shouldn't change even if
+ * force_poll_sync changes.
+ */
+ switch (urb->dev->speed) {
+ case USB_SPEED_SUPER_PLUS:
+ case USB_SPEED_SUPER: /* units are 125us */
+ data->intr_interval = usecs_to_jiffies(urb->interval * 125);
+ break;
+ default:
+ data->intr_interval = msecs_to_jiffies(urb->interval);
+ break;
+ }
+
+done:
usb_free_urb(urb);
return err;
@@ -1322,10 +1410,13 @@ static void btusb_tx_complete(struct urb *urb)
if (!test_bit(HCI_RUNNING, &hdev->flags))
goto done;
- if (!urb->status)
+ if (!urb->status) {
hdev->stat.byte_tx += urb->transfer_buffer_length;
- else
+ } else {
+ if (hci_skb_pkt_type(skb) == HCI_COMMAND_PKT)
+ hci_cmd_sync_cancel(hdev, -urb->status);
hdev->stat.err_tx++;
+ }
done:
spin_lock_irqsave(&data->txlock, flags);
@@ -1429,9 +1520,12 @@ static int btusb_close(struct hci_dev *hdev)
BT_DBG("%s", hdev->name);
+ cancel_delayed_work(&data->rx_work);
cancel_work_sync(&data->work);
cancel_work_sync(&data->waker);
+ skb_queue_purge(&data->acl_q);
+
clear_bit(BTUSB_ISOC_RUNNING, &data->flags);
clear_bit(BTUSB_BULK_RUNNING, &data->flags);
clear_bit(BTUSB_INTR_RUNNING, &data->flags);
@@ -1463,6 +1557,10 @@ static int btusb_flush(struct hci_dev *hdev)
BT_DBG("%s", hdev->name);
+ cancel_delayed_work(&data->rx_work);
+
+ skb_queue_purge(&data->acl_q);
+
usb_kill_anchored_urbs(&data->tx_anchor);
btusb_free_frags(data);
@@ -1826,6 +1924,17 @@ static void btusb_waker(struct work_struct *work)
usb_autopm_put_interface(data->intf);
}
+static void btusb_rx_work(struct work_struct *work)
+{
+ struct btusb_data *data = container_of(work, struct btusb_data,
+ rx_work.work);
+ struct sk_buff *skb;
+
+ /* Dequeue ACL data received during the interval */
+ while ((skb = skb_dequeue(&data->acl_q)))
+ data->recv_acl(data->hdev, skb);
+}
+
static int btusb_setup_bcm92035(struct hci_dev *hdev)
{
struct sk_buff *skb;
@@ -2131,122 +2240,6 @@ static int btusb_send_frame_intel(struct hci_dev *hdev, struct sk_buff *skb)
#define MTK_BT_RST_DONE 0x00000100
#define MTK_BT_RESET_WAIT_MS 100
#define MTK_BT_RESET_NUM_TRIES 10
-#define FIRMWARE_MT7663 "mediatek/mt7663pr2h.bin"
-#define FIRMWARE_MT7668 "mediatek/mt7668pr2h.bin"
-
-#define HCI_WMT_MAX_EVENT_SIZE 64
-/* It is for mt79xx download rom patch*/
-#define MTK_FW_ROM_PATCH_HEADER_SIZE 32
-#define MTK_FW_ROM_PATCH_GD_SIZE 64
-#define MTK_FW_ROM_PATCH_SEC_MAP_SIZE 64
-#define MTK_SEC_MAP_COMMON_SIZE 12
-#define MTK_SEC_MAP_NEED_SEND_SIZE 52
-
-enum {
- BTMTK_WMT_PATCH_DWNLD = 0x1,
- BTMTK_WMT_FUNC_CTRL = 0x6,
- BTMTK_WMT_RST = 0x7,
- BTMTK_WMT_SEMAPHORE = 0x17,
-};
-
-enum {
- BTMTK_WMT_INVALID,
- BTMTK_WMT_PATCH_UNDONE,
- BTMTK_WMT_PATCH_PROGRESS,
- BTMTK_WMT_PATCH_DONE,
- BTMTK_WMT_ON_UNDONE,
- BTMTK_WMT_ON_DONE,
- BTMTK_WMT_ON_PROGRESS,
-};
-
-struct btmtk_wmt_hdr {
- u8 dir;
- u8 op;
- __le16 dlen;
- u8 flag;
-} __packed;
-
-struct btmtk_hci_wmt_cmd {
- struct btmtk_wmt_hdr hdr;
- u8 data[];
-} __packed;
-
-struct btmtk_hci_wmt_evt {
- struct hci_event_hdr hhdr;
- struct btmtk_wmt_hdr whdr;
-} __packed;
-
-struct btmtk_hci_wmt_evt_funcc {
- struct btmtk_hci_wmt_evt hwhdr;
- __be16 status;
-} __packed;
-
-struct btmtk_tci_sleep {
- u8 mode;
- __le16 duration;
- __le16 host_duration;
- u8 host_wakeup_pin;
- u8 time_compensation;
-} __packed;
-
-struct btmtk_hci_wmt_params {
- u8 op;
- u8 flag;
- u16 dlen;
- const void *data;
- u32 *status;
-};
-
-struct btmtk_patch_header {
- u8 datetime[16];
- u8 platform[4];
- __le16 hwver;
- __le16 swver;
- __le32 magicnum;
-} __packed;
-
-struct btmtk_global_desc {
- __le32 patch_ver;
- __le32 sub_sys;
- __le32 feature_opt;
- __le32 section_num;
-} __packed;
-
-struct btmtk_section_map {
- __le32 sectype;
- __le32 secoffset;
- __le32 secsize;
- union {
- __le32 u4SecSpec[13];
- struct {
- __le32 dlAddr;
- __le32 dlsize;
- __le32 seckeyidx;
- __le32 alignlen;
- __le32 sectype;
- __le32 dlmodecrctype;
- __le32 crc;
- __le32 reserved[6];
- } bin_info_spec;
- };
-} __packed;
-
-static int btusb_set_bdaddr_mtk(struct hci_dev *hdev, const bdaddr_t *bdaddr)
-{
- struct sk_buff *skb;
- long ret;
-
- skb = __hci_cmd_sync(hdev, 0xfc1a, sizeof(bdaddr), bdaddr, HCI_INIT_TIMEOUT);
- if (IS_ERR(skb)) {
- ret = PTR_ERR(skb);
- bt_dev_err(hdev, "changing Mediatek device address failed (%ld)",
- ret);
- return ret;
- }
- kfree_skb(skb);
-
- return 0;
-}
static void btusb_mtk_wmt_recv(struct urb *urb)
{
@@ -2265,6 +2258,7 @@ static void btusb_mtk_wmt_recv(struct urb *urb)
skb = bt_skb_alloc(HCI_WMT_MAX_EVENT_SIZE, GFP_ATOMIC);
if (!skb) {
hdev->stat.err_rx++;
+ kfree(urb->setup_packet);
return;
}
@@ -2285,6 +2279,7 @@ static void btusb_mtk_wmt_recv(struct urb *urb)
data->evt_skb = skb_clone(skb, GFP_ATOMIC);
if (!data->evt_skb) {
kfree_skb(skb);
+ kfree(urb->setup_packet);
return;
}
}
@@ -2293,6 +2288,7 @@ static void btusb_mtk_wmt_recv(struct urb *urb)
if (err < 0) {
kfree_skb(data->evt_skb);
data->evt_skb = NULL;
+ kfree(urb->setup_packet);
return;
}
@@ -2303,6 +2299,7 @@ static void btusb_mtk_wmt_recv(struct urb *urb)
wake_up_bit(&data->flags,
BTUSB_TX_WAIT_VND_EVT);
}
+ kfree(urb->setup_packet);
return;
} else if (urb->status == -ENOENT) {
/* Avoid suspend failed when usb_kill_urb */
@@ -2323,6 +2320,7 @@ static void btusb_mtk_wmt_recv(struct urb *urb)
usb_anchor_urb(urb, &data->ctrl_anchor);
err = usb_submit_urb(urb, GFP_ATOMIC);
if (err < 0) {
+ kfree(urb->setup_packet);
/* -EPERM: urb is being killed;
* -ENODEV: device got disconnected
*/
@@ -2497,209 +2495,6 @@ static int btusb_mtk_hci_wmt_sync(struct hci_dev *hdev,
return err;
}
-static int btusb_mtk_setup_firmware_79xx(struct hci_dev *hdev, const char *fwname)
-{
- struct btmtk_hci_wmt_params wmt_params;
- struct btmtk_global_desc *globaldesc = NULL;
- struct btmtk_section_map *sectionmap;
- const struct firmware *fw;
- const u8 *fw_ptr;
- const u8 *fw_bin_ptr;
- int err, dlen, i, status;
- u8 flag, first_block, retry;
- u32 section_num, dl_size, section_offset;
- u8 cmd[64];
-
- err = request_firmware(&fw, fwname, &hdev->dev);
- if (err < 0) {
- bt_dev_err(hdev, "Failed to load firmware file (%d)", err);
- return err;
- }
-
- fw_ptr = fw->data;
- fw_bin_ptr = fw_ptr;
- globaldesc = (struct btmtk_global_desc *)(fw_ptr + MTK_FW_ROM_PATCH_HEADER_SIZE);
- section_num = le32_to_cpu(globaldesc->section_num);
-
- for (i = 0; i < section_num; i++) {
- first_block = 1;
- fw_ptr = fw_bin_ptr;
- sectionmap = (struct btmtk_section_map *)(fw_ptr + MTK_FW_ROM_PATCH_HEADER_SIZE +
- MTK_FW_ROM_PATCH_GD_SIZE + MTK_FW_ROM_PATCH_SEC_MAP_SIZE * i);
-
- section_offset = le32_to_cpu(sectionmap->secoffset);
- dl_size = le32_to_cpu(sectionmap->bin_info_spec.dlsize);
-
- if (dl_size > 0) {
- retry = 20;
- while (retry > 0) {
- cmd[0] = 0; /* 0 means legacy dl mode. */
- memcpy(cmd + 1,
- fw_ptr + MTK_FW_ROM_PATCH_HEADER_SIZE +
- MTK_FW_ROM_PATCH_GD_SIZE + MTK_FW_ROM_PATCH_SEC_MAP_SIZE * i +
- MTK_SEC_MAP_COMMON_SIZE,
- MTK_SEC_MAP_NEED_SEND_SIZE + 1);
-
- wmt_params.op = BTMTK_WMT_PATCH_DWNLD;
- wmt_params.status = &status;
- wmt_params.flag = 0;
- wmt_params.dlen = MTK_SEC_MAP_NEED_SEND_SIZE + 1;
- wmt_params.data = &cmd;
-
- err = btusb_mtk_hci_wmt_sync(hdev, &wmt_params);
- if (err < 0) {
- bt_dev_err(hdev, "Failed to send wmt patch dwnld (%d)",
- err);
- goto err_release_fw;
- }
-
- if (status == BTMTK_WMT_PATCH_UNDONE) {
- break;
- } else if (status == BTMTK_WMT_PATCH_PROGRESS) {
- msleep(100);
- retry--;
- } else if (status == BTMTK_WMT_PATCH_DONE) {
- goto next_section;
- } else {
- bt_dev_err(hdev, "Failed wmt patch dwnld status (%d)",
- status);
- goto err_release_fw;
- }
- }
-
- fw_ptr += section_offset;
- wmt_params.op = BTMTK_WMT_PATCH_DWNLD;
- wmt_params.status = NULL;
-
- while (dl_size > 0) {
- dlen = min_t(int, 250, dl_size);
- if (first_block == 1) {
- flag = 1;
- first_block = 0;
- } else if (dl_size - dlen <= 0) {
- flag = 3;
- } else {
- flag = 2;
- }
-
- wmt_params.flag = flag;
- wmt_params.dlen = dlen;
- wmt_params.data = fw_ptr;
-
- err = btusb_mtk_hci_wmt_sync(hdev, &wmt_params);
- if (err < 0) {
- bt_dev_err(hdev, "Failed to send wmt patch dwnld (%d)",
- err);
- goto err_release_fw;
- }
-
- dl_size -= dlen;
- fw_ptr += dlen;
- }
- }
-next_section:
- continue;
- }
- /* Wait a few moments for firmware activation done */
- usleep_range(100000, 120000);
-
-err_release_fw:
- release_firmware(fw);
-
- return err;
-}
-
-static int btusb_mtk_setup_firmware(struct hci_dev *hdev, const char *fwname)
-{
- struct btmtk_hci_wmt_params wmt_params;
- const struct firmware *fw;
- const u8 *fw_ptr;
- size_t fw_size;
- int err, dlen;
- u8 flag, param;
-
- err = request_firmware(&fw, fwname, &hdev->dev);
- if (err < 0) {
- bt_dev_err(hdev, "Failed to load firmware file (%d)", err);
- return err;
- }
-
- /* Power on data RAM the firmware relies on. */
- param = 1;
- wmt_params.op = BTMTK_WMT_FUNC_CTRL;
- wmt_params.flag = 3;
- wmt_params.dlen = sizeof(param);
- wmt_params.data = ¶m;
- wmt_params.status = NULL;
-
- err = btusb_mtk_hci_wmt_sync(hdev, &wmt_params);
- if (err < 0) {
- bt_dev_err(hdev, "Failed to power on data RAM (%d)", err);
- goto err_release_fw;
- }
-
- fw_ptr = fw->data;
- fw_size = fw->size;
-
- /* The size of patch header is 30 bytes, should be skip */
- if (fw_size < 30) {
- err = -EINVAL;
- goto err_release_fw;
- }
-
- fw_size -= 30;
- fw_ptr += 30;
- flag = 1;
-
- wmt_params.op = BTMTK_WMT_PATCH_DWNLD;
- wmt_params.status = NULL;
-
- while (fw_size > 0) {
- dlen = min_t(int, 250, fw_size);
-
- /* Tell device the position in sequence */
- if (fw_size - dlen <= 0)
- flag = 3;
- else if (fw_size < fw->size - 30)
- flag = 2;
-
- wmt_params.flag = flag;
- wmt_params.dlen = dlen;
- wmt_params.data = fw_ptr;
-
- err = btusb_mtk_hci_wmt_sync(hdev, &wmt_params);
- if (err < 0) {
- bt_dev_err(hdev, "Failed to send wmt patch dwnld (%d)",
- err);
- goto err_release_fw;
- }
-
- fw_size -= dlen;
- fw_ptr += dlen;
- }
-
- wmt_params.op = BTMTK_WMT_RST;
- wmt_params.flag = 4;
- wmt_params.dlen = 0;
- wmt_params.data = NULL;
- wmt_params.status = NULL;
-
- /* Activate funciton the firmware providing to */
- err = btusb_mtk_hci_wmt_sync(hdev, &wmt_params);
- if (err < 0) {
- bt_dev_err(hdev, "Failed to send wmt rst (%d)", err);
- goto err_release_fw;
- }
-
- /* Wait a few moments for firmware activation done */
- usleep_range(10000, 12000);
-
-err_release_fw:
- release_firmware(fw);
-
- return err;
-}
-
static int btusb_mtk_func_query(struct hci_dev *hdev)
{
struct btmtk_hci_wmt_params wmt_params;
@@ -2857,7 +2652,12 @@ static int btusb_mtk_setup(struct hci_dev *hdev)
snprintf(fw_bin_name, sizeof(fw_bin_name),
"mediatek/BT_RAM_CODE_MT%04x_1_%x_hdr.bin",
dev_id & 0xffff, (fw_version & 0xff) + 1);
- err = btusb_mtk_setup_firmware_79xx(hdev, fw_bin_name);
+ err = btmtk_setup_firmware_79xx(hdev, fw_bin_name,
+ btusb_mtk_hci_wmt_sync);
+ if (err < 0) {
+ bt_dev_err(hdev, "Failed to set up firmware (%d)", err);
+ return err;
+ }
/* It's Device EndPoint Reset Option Register */
btusb_mtk_uhw_reg_write(data, MTK_EP_RST_OPT, MTK_EP_RST_IN_OUT_OPT);
@@ -2877,6 +2677,7 @@ static int btusb_mtk_setup(struct hci_dev *hdev)
}
hci_set_msft_opcode(hdev, 0xFD30);
+ hci_set_aosp_capable(hdev);
goto done;
default:
bt_dev_err(hdev, "Unsupported hardware variant (%08x)",
@@ -2903,7 +2704,8 @@ static int btusb_mtk_setup(struct hci_dev *hdev)
}
/* Setup a firmware which the device definitely requires */
- err = btusb_mtk_setup_firmware(hdev, fwname);
+ err = btmtk_setup_firmware(hdev, fwname,
+ btusb_mtk_hci_wmt_sync);
if (err < 0)
return err;
@@ -3064,9 +2866,6 @@ static int btusb_recv_acl_mtk(struct hci_dev *hdev, struct sk_buff *skb)
return hci_recv_frame(hdev, skb);
}
-MODULE_FIRMWARE(FIRMWARE_MT7663);
-MODULE_FIRMWARE(FIRMWARE_MT7668);
-
#ifdef CONFIG_PM
/* Configure an out-of-band gpio as wake-up pin, if specified in device tree */
static int marvell_config_oob_wake(struct hci_dev *hdev)
@@ -3190,11 +2989,15 @@ static int btusb_set_bdaddr_wcn6855(struct hci_dev *hdev,
#define QCA_DFU_TIMEOUT 3000
#define QCA_FLAG_MULTI_NVM 0x80
+#define WCN6855_2_0_RAM_VERSION_GF 0x400c1200
+#define WCN6855_2_1_RAM_VERSION_GF 0x400c1211
+
struct qca_version {
__le32 rom_version;
__le32 patch_version;
__le32 ram_version;
- __le16 board_id;
+ __u8 chip_id;
+ __u8 platform_id;
__le16 flag;
__u8 reserved[4];
} __packed;
@@ -3221,6 +3024,7 @@ static const struct qca_device_info qca_devices_table[] = {
{ 0x00000302, 28, 4, 16 }, /* Rome 3.2 */
{ 0x00130100, 40, 4, 16 }, /* WCN6855 1.0 */
{ 0x00130200, 40, 4, 16 }, /* WCN6855 2.0 */
+ { 0x00130201, 40, 4, 16 }, /* WCN6855 2.1 */
};
static int btusb_qca_send_vendor_req(struct usb_device *udev, u8 request,
@@ -3375,6 +3179,50 @@ static int btusb_setup_qca_load_rampatch(struct hci_dev *hdev,
return err;
}
+static void btusb_generate_qca_nvm_name(char *fwname, size_t max_size,
+ const struct qca_version *ver)
+{
+ u32 rom_version = le32_to_cpu(ver->rom_version);
+ u16 flag = le16_to_cpu(ver->flag);
+
+ if (((flag >> 8) & 0xff) == QCA_FLAG_MULTI_NVM) {
+ /* The board_id should be split into two bytes
+ * The 1st byte is chip ID, and the 2nd byte is platform ID
+ * For example, board ID 0x010A, 0x01 is platform ID. 0x0A is chip ID
+ * we have several platforms, and platform IDs are continuously added
+ * Platform ID:
+ * 0x00 is for Mobile
+ * 0x01 is for X86
+ * 0x02 is for Automotive
+ * 0x03 is for Consumer electronic
+ */
+ u16 board_id = (ver->chip_id << 8) + ver->platform_id;
+ const char *variant;
+
+ switch (le32_to_cpu(ver->ram_version)) {
+ case WCN6855_2_0_RAM_VERSION_GF:
+ case WCN6855_2_1_RAM_VERSION_GF:
+ variant = "_gf";
+ break;
+ default:
+ variant = "";
+ break;
+ }
+
+ if (board_id == 0) {
+ snprintf(fwname, max_size, "qca/nvm_usb_%08x%s.bin",
+ rom_version, variant);
+ } else {
+ snprintf(fwname, max_size, "qca/nvm_usb_%08x%s_%04x.bin",
+ rom_version, variant, board_id);
+ }
+ } else {
+ snprintf(fwname, max_size, "qca/nvm_usb_%08x.bin",
+ rom_version);
+ }
+
+}
+
static int btusb_setup_qca_load_nvm(struct hci_dev *hdev,
struct qca_version *ver,
const struct qca_device_info *info)
@@ -3383,20 +3231,7 @@ static int btusb_setup_qca_load_nvm(struct hci_dev *hdev,
char fwname[64];
int err;
- if (((ver->flag >> 8) & 0xff) == QCA_FLAG_MULTI_NVM) {
- /* if boardid equal 0, use default nvm without surfix */
- if (le16_to_cpu(ver->board_id) == 0x0) {
- snprintf(fwname, sizeof(fwname), "qca/nvm_usb_%08x.bin",
- le32_to_cpu(ver->rom_version));
- } else {
- snprintf(fwname, sizeof(fwname), "qca/nvm_usb_%08x_%04x.bin",
- le32_to_cpu(ver->rom_version),
- le16_to_cpu(ver->board_id));
- }
- } else {
- snprintf(fwname, sizeof(fwname), "qca/nvm_usb_%08x.bin",
- le32_to_cpu(ver->rom_version));
- }
+ btusb_generate_qca_nvm_name(fwname, sizeof(fwname), ver);
err = request_firmware(&fw, fwname, &hdev->dev);
if (err) {
@@ -3661,6 +3496,49 @@ static int btusb_shutdown_qca(struct hci_dev *hdev)
return 0;
}
+static ssize_t force_poll_sync_read(struct file *file, char __user *user_buf,
+ size_t count, loff_t *ppos)
+{
+ struct btusb_data *data = file->private_data;
+ char buf[3];
+
+ buf[0] = data->poll_sync ? 'Y' : 'N';
+ buf[1] = '\n';
+ buf[2] = '\0';
+ return simple_read_from_buffer(user_buf, count, ppos, buf, 2);
+}
+
+static ssize_t force_poll_sync_write(struct file *file,
+ const char __user *user_buf,
+ size_t count, loff_t *ppos)
+{
+ struct btusb_data *data = file->private_data;
+ bool enable;
+ int err;
+
+ err = kstrtobool_from_user(user_buf, count, &enable);
+ if (err)
+ return err;
+
+ /* Only allow changes while the adapter is down */
+ if (test_bit(HCI_UP, &data->hdev->flags))
+ return -EPERM;
+
+ if (data->poll_sync == enable)
+ return -EALREADY;
+
+ data->poll_sync = enable;
+
+ return count;
+}
+
+static const struct file_operations force_poll_sync_fops = {
+ .open = simple_open,
+ .read = force_poll_sync_read,
+ .write = force_poll_sync_write,
+ .llseek = default_llseek,
+};
+
static int btusb_probe(struct usb_interface *intf,
const struct usb_device_id *id)
{
@@ -3744,6 +3622,10 @@ static int btusb_probe(struct usb_interface *intf,
INIT_WORK(&data->work, btusb_work);
INIT_WORK(&data->waker, btusb_waker);
+ INIT_DELAYED_WORK(&data->rx_work, btusb_rx_work);
+
+ skb_queue_head_init(&data->acl_q);
+
init_usb_anchor(&data->deferred);
init_usb_anchor(&data->tx_anchor);
spin_lock_init(&data->txlock);
@@ -3857,6 +3739,9 @@ static int btusb_probe(struct usb_interface *intf,
if (id->driver_info & BTUSB_INTEL_BROKEN_INITIAL_NCMD)
btintel_set_flag(hdev, INTEL_BROKEN_INITIAL_NCMD);
+
+ if (id->driver_info & BTUSB_INTEL_BROKEN_SHUTDOWN_LED)
+ btintel_set_flag(hdev, INTEL_BROKEN_SHUTDOWN_LED);
}
if (id->driver_info & BTUSB_MARVELL)
@@ -3868,7 +3753,7 @@ static int btusb_probe(struct usb_interface *intf,
hdev->shutdown = btusb_mtk_shutdown;
hdev->manufacturer = 70;
hdev->cmd_timeout = btusb_mtk_cmd_timeout;
- hdev->set_bdaddr = btusb_set_bdaddr_mtk;
+ hdev->set_bdaddr = btmtk_set_bdaddr;
set_bit(HCI_QUIRK_NON_PERSISTENT_SETUP, &hdev->quirks);
data->recv_acl = btusb_recv_acl_mtk;
}
@@ -4009,6 +3894,9 @@ static int btusb_probe(struct usb_interface *intf,
usb_set_intfdata(intf, data);
+ debugfs_create_file("force_poll_sync", 0644, hdev->debugfs, data,
+ &force_poll_sync_fops);
+
return 0;
out_free_dev:
diff --git a/drivers/bluetooth/hci_bcm.c b/drivers/bluetooth/hci_bcm.c
index ef54afa..d634a27 100644
--- a/drivers/bluetooth/hci_bcm.c
+++ b/drivers/bluetooth/hci_bcm.c
@@ -1188,7 +1188,12 @@ static int bcm_probe(struct platform_device *pdev)
return -ENOMEM;
dev->dev = &pdev->dev;
- dev->irq = platform_get_irq(pdev, 0);
+
+ ret = platform_get_irq(pdev, 0);
+ if (ret < 0)
+ return ret;
+
+ dev->irq = ret;
/* Initialize routing field to an unused value */
dev->pcm_int_params[0] = 0xff;
@@ -1508,7 +1513,6 @@ static const struct of_device_id bcm_bluetooth_of_match[] = {
{ .compatible = "brcm,bcm4330-bt" },
{ .compatible = "brcm,bcm4334-bt" },
{ .compatible = "brcm,bcm4345c5" },
- { .compatible = "brcm,bcm4330-bt" },
{ .compatible = "brcm,bcm43438-bt", .data = &bcm43438_device_data },
{ .compatible = "brcm,bcm43540-bt", .data = &bcm4354_device_data },
{ .compatible = "brcm,bcm4335a0" },
diff --git a/drivers/bluetooth/hci_h4.c b/drivers/bluetooth/hci_h4.c
index 4b3b14a..1d0cdf0 100644
--- a/drivers/bluetooth/hci_h4.c
+++ b/drivers/bluetooth/hci_h4.c
@@ -252,7 +252,7 @@ struct sk_buff *h4_recv_buf(struct hci_dev *hdev, struct sk_buff *skb,
}
if (!dlen) {
- hu->padding = (skb->len - 1) % alignment;
+ hu->padding = (skb->len + 1) % alignment;
hu->padding = (alignment - hu->padding) % alignment;
/* No more data, complete frame */
@@ -260,7 +260,7 @@ struct sk_buff *h4_recv_buf(struct hci_dev *hdev, struct sk_buff *skb,
skb = NULL;
}
} else {
- hu->padding = (skb->len - 1) % alignment;
+ hu->padding = (skb->len + 1) % alignment;
hu->padding = (alignment - hu->padding) % alignment;
/* Complete frame */
diff --git a/drivers/bluetooth/hci_qca.c b/drivers/bluetooth/hci_qca.c
index dd768a8..f6e91fb 100644
--- a/drivers/bluetooth/hci_qca.c
+++ b/drivers/bluetooth/hci_qca.c
@@ -1928,6 +1928,9 @@ static int qca_power_off(struct hci_dev *hdev)
hu->hdev->hw_error = NULL;
hu->hdev->cmd_timeout = NULL;
+ del_timer_sync(&qca->wake_retrans_timer);
+ del_timer_sync(&qca->tx_idle_timer);
+
/* Stop sending shutdown command if soc crashes. */
if (soc_type != QCA_ROME
&& qca->memdump_state == QCA_MEMDUMP_IDLE) {
@@ -2056,14 +2059,14 @@ static int qca_serdev_probe(struct serdev_device *serdev)
qcadev->bt_en = devm_gpiod_get_optional(&serdev->dev, "enable",
GPIOD_OUT_LOW);
- if (!qcadev->bt_en && data->soc_type == QCA_WCN6750) {
+ if (IS_ERR_OR_NULL(qcadev->bt_en) && data->soc_type == QCA_WCN6750) {
dev_err(&serdev->dev, "failed to acquire BT_EN gpio\n");
power_ctrl_enabled = false;
}
qcadev->sw_ctrl = devm_gpiod_get_optional(&serdev->dev, "swctrl",
GPIOD_IN);
- if (!qcadev->sw_ctrl && data->soc_type == QCA_WCN6750)
+ if (IS_ERR_OR_NULL(qcadev->sw_ctrl) && data->soc_type == QCA_WCN6750)
dev_warn(&serdev->dev, "failed to acquire SW_CTRL gpio\n");
qcadev->susclk = devm_clk_get_optional(&serdev->dev, NULL);
@@ -2085,7 +2088,7 @@ static int qca_serdev_probe(struct serdev_device *serdev)
qcadev->bt_en = devm_gpiod_get_optional(&serdev->dev, "enable",
GPIOD_OUT_LOW);
- if (!qcadev->bt_en) {
+ if (IS_ERR_OR_NULL(qcadev->bt_en)) {
dev_warn(&serdev->dev, "failed to acquire enable gpio\n");
power_ctrl_enabled = false;
}
diff --git a/drivers/bluetooth/hci_vhci.c b/drivers/bluetooth/hci_vhci.c
index b45db0d..c443c3b0 100644
--- a/drivers/bluetooth/hci_vhci.c
+++ b/drivers/bluetooth/hci_vhci.c
@@ -38,9 +38,12 @@ struct vhci_data {
struct mutex open_mutex;
struct delayed_work open_timeout;
+ struct work_struct suspend_work;
bool suspended;
bool wakeup;
+ __u16 msft_opcode;
+ bool aosp_capable;
};
static int vhci_open_dev(struct hci_dev *hdev)
@@ -114,6 +117,17 @@ static ssize_t force_suspend_read(struct file *file, char __user *user_buf,
return simple_read_from_buffer(user_buf, count, ppos, buf, 2);
}
+static void vhci_suspend_work(struct work_struct *work)
+{
+ struct vhci_data *data = container_of(work, struct vhci_data,
+ suspend_work);
+
+ if (data->suspended)
+ hci_suspend_dev(data->hdev);
+ else
+ hci_resume_dev(data->hdev);
+}
+
static ssize_t force_suspend_write(struct file *file,
const char __user *user_buf,
size_t count, loff_t *ppos)
@@ -129,16 +143,10 @@ static ssize_t force_suspend_write(struct file *file,
if (data->suspended == enable)
return -EALREADY;
- if (enable)
- err = hci_suspend_dev(data->hdev);
- else
- err = hci_resume_dev(data->hdev);
-
- if (err)
- return err;
-
data->suspended = enable;
+ schedule_work(&data->suspend_work);
+
return count;
}
@@ -176,6 +184,8 @@ static ssize_t force_wakeup_write(struct file *file,
if (data->wakeup == enable)
return -EALREADY;
+ data->wakeup = enable;
+
return count;
}
@@ -186,6 +196,88 @@ static const struct file_operations force_wakeup_fops = {
.llseek = default_llseek,
};
+static int msft_opcode_set(void *data, u64 val)
+{
+ struct vhci_data *vhci = data;
+
+ if (val > 0xffff || hci_opcode_ogf(val) != 0x3f)
+ return -EINVAL;
+
+ if (vhci->msft_opcode)
+ return -EALREADY;
+
+ vhci->msft_opcode = val;
+
+ return 0;
+}
+
+static int msft_opcode_get(void *data, u64 *val)
+{
+ struct vhci_data *vhci = data;
+
+ *val = vhci->msft_opcode;
+
+ return 0;
+}
+
+DEFINE_DEBUGFS_ATTRIBUTE(msft_opcode_fops, msft_opcode_get, msft_opcode_set,
+ "%llu\n");
+
+static ssize_t aosp_capable_read(struct file *file, char __user *user_buf,
+ size_t count, loff_t *ppos)
+{
+ struct vhci_data *vhci = file->private_data;
+ char buf[3];
+
+ buf[0] = vhci->aosp_capable ? 'Y' : 'N';
+ buf[1] = '\n';
+ buf[2] = '\0';
+ return simple_read_from_buffer(user_buf, count, ppos, buf, 2);
+}
+
+static ssize_t aosp_capable_write(struct file *file,
+ const char __user *user_buf, size_t count,
+ loff_t *ppos)
+{
+ struct vhci_data *vhci = file->private_data;
+ bool enable;
+ int err;
+
+ err = kstrtobool_from_user(user_buf, count, &enable);
+ if (err)
+ return err;
+
+ if (!enable)
+ return -EINVAL;
+
+ if (vhci->aosp_capable)
+ return -EALREADY;
+
+ vhci->aosp_capable = enable;
+
+ return count;
+}
+
+static const struct file_operations aosp_capable_fops = {
+ .open = simple_open,
+ .read = aosp_capable_read,
+ .write = aosp_capable_write,
+ .llseek = default_llseek,
+};
+
+static int vhci_setup(struct hci_dev *hdev)
+{
+ struct vhci_data *vhci = hci_get_drvdata(hdev);
+
+ if (vhci->msft_opcode)
+ hci_set_msft_opcode(hdev, vhci->msft_opcode);
+
+ if (vhci->aosp_capable)
+ hci_set_aosp_capable(hdev);
+
+ return 0;
+}
+
static int __vhci_create_device(struct vhci_data *data, __u8 opcode)
{
struct hci_dev *hdev;
@@ -228,6 +320,8 @@ static int __vhci_create_device(struct vhci_data *data, __u8 opcode)
hdev->get_data_path_id = vhci_get_data_path_id;
hdev->get_codec_config_data = vhci_get_codec_config_data;
hdev->wakeup = vhci_wakeup;
+ hdev->setup = vhci_setup;
+ set_bit(HCI_QUIRK_NON_PERSISTENT_SETUP, &hdev->quirks);
/* bit 6 is for external configuration */
if (opcode & 0x40)
@@ -237,6 +331,8 @@ static int __vhci_create_device(struct vhci_data *data, __u8 opcode)
if (opcode & 0x80)
set_bit(HCI_QUIRK_RAW_DEVICE, &hdev->quirks);
+ set_bit(HCI_QUIRK_VALID_LE_STATES, &hdev->quirks);
+
if (hci_register_dev(hdev) < 0) {
BT_ERR("Can't register HCI device");
hci_free_dev(hdev);
@@ -251,6 +347,14 @@ static int __vhci_create_device(struct vhci_data *data, __u8 opcode)
debugfs_create_file("force_wakeup", 0644, hdev->debugfs, data,
&force_wakeup_fops);
+ if (IS_ENABLED(CONFIG_BT_MSFTEXT))
+ debugfs_create_file("msft_opcode", 0644, hdev->debugfs, data,
+ &msft_opcode_fops);
+
+ if (IS_ENABLED(CONFIG_BT_AOSPEXT))
+ debugfs_create_file("aosp_capable", 0644, hdev->debugfs, data,
+ &aosp_capable_fops);
+
hci_skb_pkt_type(skb) = HCI_VENDOR_PKT;
skb_put_u8(skb, 0xff);
@@ -440,6 +544,7 @@ static int vhci_open(struct inode *inode, struct file *file)
mutex_init(&data->open_mutex);
INIT_DELAYED_WORK(&data->open_timeout, vhci_open_timeout);
+ INIT_WORK(&data->suspend_work, vhci_suspend_work);
file->private_data = data;
nonseekable_open(inode, file);
@@ -455,6 +560,7 @@ static int vhci_release(struct inode *inode, struct file *file)
struct hci_dev *hdev;
cancel_delayed_work_sync(&data->open_timeout);
+ flush_work(&data->suspend_work);
hdev = data->hdev;
diff --git a/drivers/bluetooth/virtio_bt.c b/drivers/bluetooth/virtio_bt.c
index 57908ce..076e494 100644
--- a/drivers/bluetooth/virtio_bt.c
+++ b/drivers/bluetooth/virtio_bt.c
@@ -202,6 +202,9 @@ static void virtbt_rx_handle(struct virtio_bluetooth *vbt, struct sk_buff *skb)
hci_skb_pkt_type(skb) = pkt_type;
hci_recv_frame(vbt->hdev, skb);
break;
+ default:
+ kfree_skb(skb);
+ break;
}
}
diff --git a/drivers/bus/imx-weim.c b/drivers/bus/imx-weim.c
index 28bb65a..bccb275 100644
--- a/drivers/bus/imx-weim.c
+++ b/drivers/bus/imx-weim.c
@@ -21,6 +21,7 @@ struct imx_weim_devtype {
unsigned int cs_stride;
unsigned int wcr_offset;
unsigned int wcr_bcm;
+ unsigned int wcr_cont_bclk;
};
static const struct imx_weim_devtype imx1_weim_devtype = {
@@ -41,6 +42,7 @@ static const struct imx_weim_devtype imx50_weim_devtype = {
.cs_stride = 0x18,
.wcr_offset = 0x90,
.wcr_bcm = BIT(0),
+ .wcr_cont_bclk = BIT(3),
};
static const struct imx_weim_devtype imx51_weim_devtype = {
@@ -206,8 +208,20 @@ static int weim_parse_dt(struct platform_device *pdev, void __iomem *base)
if (of_property_read_bool(pdev->dev.of_node, "fsl,burst-clk-enable")) {
if (devtype->wcr_bcm) {
reg = readl(base + devtype->wcr_offset);
- writel(reg | devtype->wcr_bcm,
- base + devtype->wcr_offset);
+ reg |= devtype->wcr_bcm;
+
+ if (of_property_read_bool(pdev->dev.of_node,
+ "fsl,continuous-burst-clk")) {
+ if (devtype->wcr_cont_bclk) {
+ reg |= devtype->wcr_cont_bclk;
+ } else {
+ dev_err(&pdev->dev,
+ "continuous burst clk not supported.\n");
+ return -EINVAL;
+ }
+ }
+
+ writel(reg, base + devtype->wcr_offset);
} else {
dev_err(&pdev->dev, "burst clk mode not supported.\n");
return -EINVAL;
diff --git a/drivers/bus/tegra-gmi.c b/drivers/bus/tegra-gmi.c
index a657078..35b59f9 100644
--- a/drivers/bus/tegra-gmi.c
+++ b/drivers/bus/tegra-gmi.c
@@ -13,8 +13,11 @@
#include <linux/io.h>
#include <linux/module.h>
#include <linux/of_device.h>
+#include <linux/pm_runtime.h>
#include <linux/reset.h>
+#include <soc/tegra/common.h>
+
#define TEGRA_GMI_CONFIG 0x00
#define TEGRA_GMI_CONFIG_GO BIT(31)
#define TEGRA_GMI_BUS_WIDTH_32BIT BIT(30)
@@ -54,9 +57,10 @@ static int tegra_gmi_enable(struct tegra_gmi *gmi)
{
int err;
- err = clk_prepare_enable(gmi->clk);
- if (err < 0) {
- dev_err(gmi->dev, "failed to enable clock: %d\n", err);
+ pm_runtime_enable(gmi->dev);
+ err = pm_runtime_resume_and_get(gmi->dev);
+ if (err) {
+ pm_runtime_disable(gmi->dev);
return err;
}
@@ -83,7 +87,9 @@ static void tegra_gmi_disable(struct tegra_gmi *gmi)
writel(config, gmi->base + TEGRA_GMI_CONFIG);
reset_control_assert(gmi->rst);
- clk_disable_unprepare(gmi->clk);
+
+ pm_runtime_put_sync_suspend(gmi->dev);
+ pm_runtime_force_suspend(gmi->dev);
}
static int tegra_gmi_parse_dt(struct tegra_gmi *gmi)
@@ -213,6 +219,7 @@ static int tegra_gmi_probe(struct platform_device *pdev)
if (!gmi)
return -ENOMEM;
+ platform_set_drvdata(pdev, gmi);
gmi->dev = dev;
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
@@ -232,6 +239,10 @@ static int tegra_gmi_probe(struct platform_device *pdev)
return PTR_ERR(gmi->rst);
}
+ err = devm_tegra_core_dev_init_opp_table_common(&pdev->dev);
+ if (err)
+ return err;
+
err = tegra_gmi_parse_dt(gmi);
if (err)
return err;
@@ -247,8 +258,6 @@ static int tegra_gmi_probe(struct platform_device *pdev)
return err;
}
- platform_set_drvdata(pdev, gmi);
-
return 0;
}
@@ -262,6 +271,34 @@ static int tegra_gmi_remove(struct platform_device *pdev)
return 0;
}
+static int __maybe_unused tegra_gmi_runtime_resume(struct device *dev)
+{
+ struct tegra_gmi *gmi = dev_get_drvdata(dev);
+ int err;
+
+ err = clk_prepare_enable(gmi->clk);
+ if (err < 0) {
+ dev_err(gmi->dev, "failed to enable clock: %d\n", err);
+ return err;
+ }
+
+ return 0;
+}
+
+static int __maybe_unused tegra_gmi_runtime_suspend(struct device *dev)
+{
+ struct tegra_gmi *gmi = dev_get_drvdata(dev);
+
+ clk_disable_unprepare(gmi->clk);
+
+ return 0;
+}
+
+static const struct dev_pm_ops tegra_gmi_pm = {
+ SET_RUNTIME_PM_OPS(tegra_gmi_runtime_suspend, tegra_gmi_runtime_resume,
+ NULL)
+};
+
static const struct of_device_id tegra_gmi_id_table[] = {
{ .compatible = "nvidia,tegra20-gmi", },
{ .compatible = "nvidia,tegra30-gmi", },
@@ -275,6 +312,7 @@ static struct platform_driver tegra_gmi_driver = {
.driver = {
.name = "tegra-gmi",
.of_match_table = tegra_gmi_id_table,
+ .pm = &tegra_gmi_pm,
},
};
module_platform_driver(tegra_gmi_driver);
diff --git a/drivers/char/agp/intel-gtt.c b/drivers/char/agp/intel-gtt.c
index 5bfdf22..c53cc98 100644
--- a/drivers/char/agp/intel-gtt.c
+++ b/drivers/char/agp/intel-gtt.c
@@ -20,6 +20,7 @@
#include <linux/kernel.h>
#include <linux/pagemap.h>
#include <linux/agp_backend.h>
+#include <linux/intel-iommu.h>
#include <linux/delay.h>
#include <asm/smp.h>
#include "agp.h"
diff --git a/drivers/char/random.c b/drivers/char/random.c
index 605969e..227fb78 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1,8 +1,7 @@
/*
* random.c -- A strong random number generator
*
- * Copyright (C) 2017 Jason A. Donenfeld <Jason@zx2c4.com>. All
- * Rights Reserved.
+ * Copyright (C) 2017-2022 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Reserved.
*
* Copyright Matt Mackall <mpm@selenic.com>, 2003, 2004, 2005
*
@@ -78,12 +77,12 @@
* an *estimate* of how many bits of randomness have been stored into
* the random number generator's internal state.
*
- * When random bytes are desired, they are obtained by taking the SHA
- * hash of the contents of the "entropy pool". The SHA hash avoids
+ * When random bytes are desired, they are obtained by taking the BLAKE2s
+ * hash of the contents of the "entropy pool". The BLAKE2s hash avoids
* exposing the internal state of the entropy pool. It is believed to
* be computationally infeasible to derive any useful information
- * about the input of SHA from its output. Even if it is possible to
- * analyze SHA in some clever way, as long as the amount of data
+ * about the input of BLAKE2s from its output. Even if it is possible to
+ * analyze BLAKE2s in some clever way, as long as the amount of data
* returned from the generator is less than the inherent entropy in
* the pool, the output data is totally unpredictable. For this
* reason, the routine decreases its internal estimate of how many
@@ -93,7 +92,7 @@
* If this estimate goes to zero, the routine can still generate
* random numbers; however, an attacker may (at least in theory) be
* able to infer the future output of the generator from prior
- * outputs. This requires successful cryptanalysis of SHA, which is
+ * outputs. This requires successful cryptanalysis of BLAKE2s, which is
* not believed to be feasible, but there is a remote possibility.
* Nonetheless, these numbers should be useful for the vast majority
* of purposes.
@@ -200,8 +199,11 @@
* void add_device_randomness(const void *buf, unsigned int size);
* void add_input_randomness(unsigned int type, unsigned int code,
* unsigned int value);
- * void add_interrupt_randomness(int irq, int irq_flags);
+ * void add_interrupt_randomness(int irq);
* void add_disk_randomness(struct gendisk *disk);
+ * void add_hwgenerator_randomness(const char *buffer, size_t count,
+ * size_t entropy);
+ * void add_bootloader_randomness(const void *buf, unsigned int size);
*
* add_device_randomness() is for adding data to the random pool that
* is likely to differ between two devices (or possibly even per boot).
@@ -228,6 +230,14 @@
* particular randomness source. They do this by keeping track of the
* first and second order deltas of the event timings.
*
+ * add_hwgenerator_randomness() is for true hardware RNGs, and will credit
+ * entropy as specified by the caller. If the entropy pool is full it will
+ * block until more entropy is needed.
+ *
+ * add_bootloader_randomness() is the same as add_hwgenerator_randomness() or
+ * add_device_randomness(), depending on whether or not the configuration
+ * option CONFIG_RANDOM_TRUST_BOOTLOADER is set.
+ *
* Ensuring unpredictability at system startup
* ============================================
*
@@ -336,7 +346,7 @@
#include <linux/completion.h>
#include <linux/uuid.h>
#include <crypto/chacha.h>
-#include <crypto/sha1.h>
+#include <crypto/blake2s.h>
#include <asm/processor.h>
#include <linux/uaccess.h>
@@ -356,10 +366,7 @@
#define INPUT_POOL_WORDS (1 << (INPUT_POOL_SHIFT-5))
#define OUTPUT_POOL_SHIFT 10
#define OUTPUT_POOL_WORDS (1 << (OUTPUT_POOL_SHIFT-5))
-#define EXTRACT_SIZE 10
-
-
-#define LONGS(x) (((x) + sizeof(unsigned long) - 1)/sizeof(unsigned long))
+#define EXTRACT_SIZE (BLAKE2S_HASH_SIZE / 2)
/*
* To allow fractional bits to be tracked, the entropy_count field is
@@ -395,7 +402,7 @@ static int random_write_wakeup_bits = 28 * OUTPUT_POOL_WORDS;
* Thanks to Colin Plumb for suggesting this.
*
* The mixing operation is much less sensitive than the output hash,
- * where we use SHA-1. All that we want of mixing operation is that
+ * where we use BLAKE2s. All that we want of mixing operation is that
* it be a good non-cryptographic hash; i.e. it not produce collisions
* when fed "random" data of the sort we expect to see. As long as
* the pool state differs for different inputs, we have preserved the
@@ -450,6 +457,10 @@ struct crng_state {
static struct crng_state primary_crng = {
.lock = __SPIN_LOCK_UNLOCKED(primary_crng.lock),
+ .state[0] = CHACHA_CONSTANT_EXPA,
+ .state[1] = CHACHA_CONSTANT_ND_3,
+ .state[2] = CHACHA_CONSTANT_2_BY,
+ .state[3] = CHACHA_CONSTANT_TE_K,
};
/*
@@ -461,6 +472,7 @@ static struct crng_state primary_crng = {
* its value (from 0->1->2).
*/
static int crng_init = 0;
+static bool crng_need_final_init = false;
#define crng_ready() (likely(crng_init > 1))
static int crng_init_cnt = 0;
static unsigned long crng_global_init_time = 0;
@@ -539,7 +551,7 @@ static void _mix_pool_bytes(struct entropy_store *r, const void *in,
unsigned long i, tap1, tap2, tap3, tap4, tap5;
int input_rotate;
int wordmask = r->poolinfo->poolwords - 1;
- const char *bytes = in;
+ const unsigned char *bytes = in;
__u32 w;
tap1 = r->poolinfo->tap1;
@@ -751,7 +763,6 @@ static int credit_entropy_bits_safe(struct entropy_store *r, int nbits)
static DECLARE_WAIT_QUEUE_HEAD(crng_init_wait);
-#ifdef CONFIG_NUMA
/*
* Hack to deal with crazy userspace progams when they are all trying
* to access /dev/urandom in parallel. The programs are almost
@@ -759,7 +770,6 @@ static DECLARE_WAIT_QUEUE_HEAD(crng_init_wait);
* their brain damage.
*/
static struct crng_state **crng_node_pool __read_mostly;
-#endif
static void invalidate_batched_entropy(void);
static void numa_crng_init(void);
@@ -807,7 +817,7 @@ static bool __init crng_init_try_arch_early(struct crng_state *crng)
return arch_init;
}
-static void __maybe_unused crng_initialize_secondary(struct crng_state *crng)
+static void crng_initialize_secondary(struct crng_state *crng)
{
chacha_init_consts(crng->state);
_get_random_bytes(&crng->state[4], sizeof(__u32) * 12);
@@ -817,18 +827,46 @@ static void __maybe_unused crng_initialize_secondary(struct crng_state *crng)
static void __init crng_initialize_primary(struct crng_state *crng)
{
- chacha_init_consts(crng->state);
_extract_entropy(&input_pool, &crng->state[4], sizeof(__u32) * 12, 0);
- if (crng_init_try_arch_early(crng) && trust_cpu) {
+ if (crng_init_try_arch_early(crng) && trust_cpu && crng_init < 2) {
invalidate_batched_entropy();
numa_crng_init();
crng_init = 2;
- pr_notice("crng done (trusting CPU's manufacturer)\n");
+ pr_notice("crng init done (trusting CPU's manufacturer)\n");
}
crng->init_time = jiffies - CRNG_RESEED_INTERVAL - 1;
}
-#ifdef CONFIG_NUMA
+static void crng_finalize_init(struct crng_state *crng)
+{
+ if (crng != &primary_crng || crng_init >= 2)
+ return;
+ if (!system_wq) {
+ /* We can't call numa_crng_init until we have workqueues,
+ * so mark this for processing later. */
+ crng_need_final_init = true;
+ return;
+ }
+
+ invalidate_batched_entropy();
+ numa_crng_init();
+ crng_init = 2;
+ process_random_ready_list();
+ wake_up_interruptible(&crng_init_wait);
+ kill_fasync(&fasync, SIGIO, POLL_IN);
+ pr_notice("crng init done\n");
+ if (unseeded_warning.missed) {
+ pr_notice("%d get_random_xx warning(s) missed due to ratelimiting\n",
+ unseeded_warning.missed);
+ unseeded_warning.missed = 0;
+ }
+ if (urandom_warning.missed) {
+ pr_notice("%d urandom warning(s) missed due to ratelimiting\n",
+ urandom_warning.missed);
+ urandom_warning.missed = 0;
+ }
+}
+
static void do_numa_crng_init(struct work_struct *work)
{
int i;
@@ -843,8 +881,8 @@ static void do_numa_crng_init(struct work_struct *work)
crng_initialize_secondary(crng);
pool[i] = crng;
}
- mb();
- if (cmpxchg(&crng_node_pool, NULL, pool)) {
+ /* pairs with READ_ONCE() in select_crng() */
+ if (cmpxchg_release(&crng_node_pool, NULL, pool) != NULL) {
for_each_node(i)
kfree(pool[i]);
kfree(pool);
@@ -855,20 +893,35 @@ static DECLARE_WORK(numa_crng_init_work, do_numa_crng_init);
static void numa_crng_init(void)
{
- schedule_work(&numa_crng_init_work);
+ if (IS_ENABLED(CONFIG_NUMA))
+ schedule_work(&numa_crng_init_work);
}
-#else
-static void numa_crng_init(void) {}
-#endif
+
+static struct crng_state *select_crng(void)
+{
+ if (IS_ENABLED(CONFIG_NUMA)) {
+ struct crng_state **pool;
+ int nid = numa_node_id();
+
+ /* pairs with cmpxchg_release() in do_numa_crng_init() */
+ pool = READ_ONCE(crng_node_pool);
+ if (pool && pool[nid])
+ return pool[nid];
+ }
+
+ return &primary_crng;
+}
/*
* crng_fast_load() can be called by code in the interrupt service
- * path. So we can't afford to dilly-dally.
+ * path. So we can't afford to dilly-dally. Returns the number of
+ * bytes processed from cp.
*/
-static int crng_fast_load(const char *cp, size_t len)
+static size_t crng_fast_load(const char *cp, size_t len)
{
unsigned long flags;
char *p;
+ size_t ret = 0;
if (!spin_trylock_irqsave(&primary_crng.lock, flags))
return 0;
@@ -879,7 +932,7 @@ static int crng_fast_load(const char *cp, size_t len)
p = (unsigned char *) &primary_crng.state[4];
while (len > 0 && crng_init_cnt < CRNG_INIT_CNT_THRESH) {
p[crng_init_cnt % CHACHA_KEY_SIZE] ^= *cp;
- cp++; crng_init_cnt++; len--;
+ cp++; crng_init_cnt++; len--; ret++;
}
spin_unlock_irqrestore(&primary_crng.lock, flags);
if (crng_init_cnt >= CRNG_INIT_CNT_THRESH) {
@@ -887,7 +940,7 @@ static int crng_fast_load(const char *cp, size_t len)
crng_init = 1;
pr_notice("fast init done\n");
}
- return 1;
+ return ret;
}
/*
@@ -962,41 +1015,24 @@ static void crng_reseed(struct crng_state *crng, struct entropy_store *r)
crng->state[i+4] ^= buf.key[i] ^ rv;
}
memzero_explicit(&buf, sizeof(buf));
- crng->init_time = jiffies;
+ WRITE_ONCE(crng->init_time, jiffies);
spin_unlock_irqrestore(&crng->lock, flags);
- if (crng == &primary_crng && crng_init < 2) {
- invalidate_batched_entropy();
- numa_crng_init();
- crng_init = 2;
- process_random_ready_list();
- wake_up_interruptible(&crng_init_wait);
- kill_fasync(&fasync, SIGIO, POLL_IN);
- pr_notice("crng init done\n");
- if (unseeded_warning.missed) {
- pr_notice("%d get_random_xx warning(s) missed due to ratelimiting\n",
- unseeded_warning.missed);
- unseeded_warning.missed = 0;
- }
- if (urandom_warning.missed) {
- pr_notice("%d urandom warning(s) missed due to ratelimiting\n",
- urandom_warning.missed);
- urandom_warning.missed = 0;
- }
- }
+ crng_finalize_init(crng);
}
static void _extract_crng(struct crng_state *crng,
__u8 out[CHACHA_BLOCK_SIZE])
{
- unsigned long v, flags;
+ unsigned long flags, init_time;
- if (crng_ready() &&
- (time_after(crng_global_init_time, crng->init_time) ||
- time_after(jiffies, crng->init_time + CRNG_RESEED_INTERVAL)))
- crng_reseed(crng, crng == &primary_crng ? &input_pool : NULL);
+ if (crng_ready()) {
+ init_time = READ_ONCE(crng->init_time);
+ if (time_after(READ_ONCE(crng_global_init_time), init_time) ||
+ time_after(jiffies, init_time + CRNG_RESEED_INTERVAL))
+ crng_reseed(crng, crng == &primary_crng ?
+ &input_pool : NULL);
+ }
spin_lock_irqsave(&crng->lock, flags);
- if (arch_get_random_long(&v))
- crng->state[14] ^= v;
chacha20_block(&crng->state[0], out);
if (crng->state[12] == 0)
crng->state[13]++;
@@ -1005,15 +1041,7 @@ static void _extract_crng(struct crng_state *crng,
static void extract_crng(__u8 out[CHACHA_BLOCK_SIZE])
{
- struct crng_state *crng = NULL;
-
-#ifdef CONFIG_NUMA
- if (crng_node_pool)
- crng = crng_node_pool[numa_node_id()];
- if (crng == NULL)
-#endif
- crng = &primary_crng;
- _extract_crng(crng, out);
+ _extract_crng(select_crng(), out);
}
/*
@@ -1042,15 +1070,7 @@ static void _crng_backtrack_protect(struct crng_state *crng,
static void crng_backtrack_protect(__u8 tmp[CHACHA_BLOCK_SIZE], int used)
{
- struct crng_state *crng = NULL;
-
-#ifdef CONFIG_NUMA
- if (crng_node_pool)
- crng = crng_node_pool[numa_node_id()];
- if (crng == NULL)
-#endif
- crng = &primary_crng;
- _crng_backtrack_protect(crng, tmp, used);
+ _crng_backtrack_protect(select_crng(), tmp, used);
}
static ssize_t extract_crng_user(void __user *buf, size_t nbytes)
@@ -1242,7 +1262,7 @@ static __u32 get_reg(struct fast_pool *f, struct pt_regs *regs)
return *ptr;
}
-void add_interrupt_randomness(int irq, int irq_flags)
+void add_interrupt_randomness(int irq)
{
struct entropy_store *r;
struct fast_pool *fast_pool = this_cpu_ptr(&irq_randomness);
@@ -1269,7 +1289,7 @@ void add_interrupt_randomness(int irq, int irq_flags)
if (unlikely(crng_init == 0)) {
if ((fast_pool->count >= 64) &&
crng_fast_load((char *) fast_pool->pool,
- sizeof(fast_pool->pool))) {
+ sizeof(fast_pool->pool)) > 0) {
fast_pool->count = 0;
fast_pool->last = now;
}
@@ -1368,56 +1388,49 @@ static size_t account(struct entropy_store *r, size_t nbytes, int min,
*/
static void extract_buf(struct entropy_store *r, __u8 *out)
{
- int i;
- union {
- __u32 w[5];
- unsigned long l[LONGS(20)];
- } hash;
- __u32 workspace[SHA1_WORKSPACE_WORDS];
+ struct blake2s_state state __aligned(__alignof__(unsigned long));
+ u8 hash[BLAKE2S_HASH_SIZE];
+ unsigned long *salt;
unsigned long flags;
+ blake2s_init(&state, sizeof(hash));
+
/*
* If we have an architectural hardware random number
- * generator, use it for SHA's initial vector
+ * generator, use it for BLAKE2's salt & personal fields.
*/
- sha1_init(hash.w);
- for (i = 0; i < LONGS(20); i++) {
+ for (salt = (unsigned long *)&state.h[4];
+ salt < (unsigned long *)&state.h[8]; ++salt) {
unsigned long v;
if (!arch_get_random_long(&v))
break;
- hash.l[i] = v;
+ *salt ^= v;
}
- /* Generate a hash across the pool, 16 words (512 bits) at a time */
+ /* Generate a hash across the pool */
spin_lock_irqsave(&r->lock, flags);
- for (i = 0; i < r->poolinfo->poolwords; i += 16)
- sha1_transform(hash.w, (__u8 *)(r->pool + i), workspace);
+ blake2s_update(&state, (const u8 *)r->pool,
+ r->poolinfo->poolwords * sizeof(*r->pool));
+ blake2s_final(&state, hash); /* final zeros out state */
/*
* We mix the hash back into the pool to prevent backtracking
* attacks (where the attacker knows the state of the pool
* plus the current outputs, and attempts to find previous
- * ouputs), unless the hash function can be inverted. By
- * mixing at least a SHA1 worth of hash data back, we make
+ * outputs), unless the hash function can be inverted. By
+ * mixing at least a hash worth of hash data back, we make
* brute-forcing the feedback as hard as brute-forcing the
* hash.
*/
- __mix_pool_bytes(r, hash.w, sizeof(hash.w));
+ __mix_pool_bytes(r, hash, sizeof(hash));
spin_unlock_irqrestore(&r->lock, flags);
- memzero_explicit(workspace, sizeof(workspace));
-
- /*
- * In case the hash function has some recognizable output
- * pattern, we fold it in half. Thus, we always feed back
- * twice as much data as we output.
+ /* Note that EXTRACT_SIZE is half of hash size here, because above
+ * we've dumped the full length back into mixer. By reducing the
+ * amount that we emit, we retain a level of forward secrecy.
*/
- hash.w[0] ^= hash.w[3];
- hash.w[1] ^= hash.w[4];
- hash.w[2] ^= rol32(hash.w[2], 16);
-
- memcpy(out, &hash, EXTRACT_SIZE);
- memzero_explicit(&hash, sizeof(hash));
+ memcpy(out, hash, EXTRACT_SIZE);
+ memzero_explicit(hash, sizeof(hash));
}
static ssize_t _extract_entropy(struct entropy_store *r, void *buf,
@@ -1775,6 +1788,8 @@ static void __init init_std_data(struct entropy_store *r)
int __init rand_initialize(void)
{
init_std_data(&input_pool);
+ if (crng_need_final_init)
+ crng_finalize_init(&primary_crng);
crng_initialize_primary(&primary_crng);
crng_global_init_time = jiffies;
if (ratelimit_disable) {
@@ -1816,7 +1831,6 @@ urandom_read_nowarn(struct file *file, char __user *buf, size_t nbytes,
static ssize_t
urandom_read(struct file *file, char __user *buf, size_t nbytes, loff_t *ppos)
{
- unsigned long flags;
static int maxwarn = 10;
if (!crng_ready() && maxwarn > 0) {
@@ -1824,9 +1838,6 @@ urandom_read(struct file *file, char __user *buf, size_t nbytes, loff_t *ppos)
if (__ratelimit(&urandom_warning))
pr_notice("%s: uninitialized urandom read (%zd bytes read)\n",
current->comm, nbytes);
- spin_lock_irqsave(&primary_crng.lock, flags);
- crng_init_cnt = 0;
- spin_unlock_irqrestore(&primary_crng.lock, flags);
}
return urandom_read_nowarn(file, buf, nbytes, ppos);
@@ -1949,7 +1960,7 @@ static long random_ioctl(struct file *f, unsigned int cmd, unsigned long arg)
if (crng_init < 2)
return -ENODATA;
crng_reseed(&primary_crng, &input_pool);
- crng_global_init_time = jiffies - 1;
+ WRITE_ONCE(crng_global_init_time, jiffies - 1);
return 0;
default:
return -EINVAL;
@@ -2275,15 +2286,20 @@ void add_hwgenerator_randomness(const char *buffer, size_t count,
struct entropy_store *poolp = &input_pool;
if (unlikely(crng_init == 0)) {
- crng_fast_load(buffer, count);
- return;
+ size_t ret = crng_fast_load(buffer, count);
+ mix_pool_bytes(poolp, buffer, ret);
+ count -= ret;
+ buffer += ret;
+ if (!count || crng_init == 0)
+ return;
}
/* Suspend writing if we're above the trickle threshold.
* We'll be woken up again once below random_write_wakeup_thresh,
* or when the calling thread is about to terminate.
*/
- wait_event_interruptible(random_write_wait, kthread_should_stop() ||
+ wait_event_interruptible(random_write_wait,
+ !system_wq || kthread_should_stop() ||
ENTROPY_BITS(&input_pool) <= random_write_wakeup_bits);
mix_pool_bytes(poolp, buffer, count);
credit_entropy_bits(poolp, entropy);
diff --git a/drivers/clk/Kconfig b/drivers/clk/Kconfig
index c5b3dc9..c91931c 100644
--- a/drivers/clk/Kconfig
+++ b/drivers/clk/Kconfig
@@ -412,6 +412,7 @@
source "drivers/clk/sifive/Kconfig"
source "drivers/clk/socfpga/Kconfig"
source "drivers/clk/sprd/Kconfig"
+source "drivers/clk/starfive/Kconfig"
source "drivers/clk/sunxi/Kconfig"
source "drivers/clk/sunxi-ng/Kconfig"
source "drivers/clk/tegra/Kconfig"
diff --git a/drivers/clk/Makefile b/drivers/clk/Makefile
index e423121..a9bb247 100644
--- a/drivers/clk/Makefile
+++ b/drivers/clk/Makefile
@@ -109,6 +109,7 @@
obj-$(CONFIG_PLAT_SPEAR) += spear/
obj-y += sprd/
obj-$(CONFIG_ARCH_STI) += st/
+obj-$(CONFIG_SOC_STARFIVE) += starfive/
obj-$(CONFIG_ARCH_SUNXI) += sunxi/
obj-$(CONFIG_SUNXI_CCU) += sunxi-ng/
obj-$(CONFIG_ARCH_TEGRA) += tegra/
diff --git a/drivers/clk/bcm/clk-bcm2835.c b/drivers/clk/bcm/clk-bcm2835.c
index a254512..3667b4d 100644
--- a/drivers/clk/bcm/clk-bcm2835.c
+++ b/drivers/clk/bcm/clk-bcm2835.c
@@ -932,8 +932,7 @@ static int bcm2835_clock_is_on(struct clk_hw *hw)
static u32 bcm2835_clock_choose_div(struct clk_hw *hw,
unsigned long rate,
- unsigned long parent_rate,
- bool round_up)
+ unsigned long parent_rate)
{
struct bcm2835_clock *clock = bcm2835_clock_from_hw(hw);
const struct bcm2835_clock_data *data = clock->data;
@@ -945,10 +944,6 @@ static u32 bcm2835_clock_choose_div(struct clk_hw *hw,
rem = do_div(temp, rate);
div = temp;
-
- /* Round up and mask off the unused bits */
- if (round_up && ((div & unused_frac_mask) != 0 || rem != 0))
- div += unused_frac_mask + 1;
div &= ~unused_frac_mask;
/* different clamping limits apply for a mash clock */
@@ -1079,7 +1074,7 @@ static int bcm2835_clock_set_rate(struct clk_hw *hw,
struct bcm2835_clock *clock = bcm2835_clock_from_hw(hw);
struct bcm2835_cprman *cprman = clock->cprman;
const struct bcm2835_clock_data *data = clock->data;
- u32 div = bcm2835_clock_choose_div(hw, rate, parent_rate, false);
+ u32 div = bcm2835_clock_choose_div(hw, rate, parent_rate);
u32 ctl;
spin_lock(&cprman->regs_lock);
@@ -1130,7 +1125,7 @@ static unsigned long bcm2835_clock_choose_div_and_prate(struct clk_hw *hw,
if (!(BIT(parent_idx) & data->set_rate_parent)) {
*prate = clk_hw_get_rate(parent);
- *div = bcm2835_clock_choose_div(hw, rate, *prate, true);
+ *div = bcm2835_clock_choose_div(hw, rate, *prate);
*avgrate = bcm2835_clock_rate_from_divisor(clock, *prate, *div);
@@ -1216,7 +1211,7 @@ static int bcm2835_clock_determine_rate(struct clk_hw *hw,
rate = bcm2835_clock_choose_div_and_prate(hw, i, req->rate,
&div, &prate,
&avgrate);
- if (rate > best_rate && rate <= req->rate) {
+ if (abs(req->rate - rate) < abs(req->rate - best_rate)) {
best_parent = parent;
best_prate = prate;
best_rate = rate;
diff --git a/drivers/clk/starfive/Kconfig b/drivers/clk/starfive/Kconfig
new file mode 100644
index 0000000..c0fa9d5
--- /dev/null
+++ b/drivers/clk/starfive/Kconfig
@@ -0,0 +1,9 @@
+# SPDX-License-Identifier: GPL-2.0
+
+config CLK_STARFIVE_JH7100
+ bool "StarFive JH7100 clock support"
+ depends on SOC_STARFIVE || COMPILE_TEST
+ default SOC_STARFIVE
+ help
+ Say yes here to support the clock controller on the StarFive JH7100
+ SoC.
diff --git a/drivers/clk/starfive/Makefile b/drivers/clk/starfive/Makefile
new file mode 100644
index 0000000..09759cc
--- /dev/null
+++ b/drivers/clk/starfive/Makefile
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0
+# StarFive Clock
+obj-$(CONFIG_CLK_STARFIVE_JH7100) += clk-starfive-jh7100.o
diff --git a/drivers/clk/starfive/clk-starfive-jh7100.c b/drivers/clk/starfive/clk-starfive-jh7100.c
new file mode 100644
index 0000000..25d31af
--- /dev/null
+++ b/drivers/clk/starfive/clk-starfive-jh7100.c
@@ -0,0 +1,689 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * StarFive JH7100 Clock Generator Driver
+ *
+ * Copyright 2021 Ahmad Fatoum, Pengutronix
+ * Copyright (C) 2021 Glider bv
+ * Copyright (C) 2021 Emil Renner Berthing <kernel@esmil.dk>
+ */
+
+#include <linux/bits.h>
+#include <linux/clk-provider.h>
+#include <linux/debugfs.h>
+#include <linux/device.h>
+#include <linux/init.h>
+#include <linux/io.h>
+#include <linux/kernel.h>
+#include <linux/mod_devicetable.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+
+#include <dt-bindings/clock/starfive-jh7100.h>
+
+/* external clocks */
+#define JH7100_CLK_OSC_SYS (JH7100_CLK_END + 0)
+#define JH7100_CLK_OSC_AUD (JH7100_CLK_END + 1)
+#define JH7100_CLK_GMAC_RMII_REF (JH7100_CLK_END + 2)
+#define JH7100_CLK_GMAC_GR_MII_RX (JH7100_CLK_END + 3)
+
+/* register fields */
+#define JH7100_CLK_ENABLE BIT(31)
+#define JH7100_CLK_INVERT BIT(30)
+#define JH7100_CLK_MUX_MASK GENMASK(27, 24)
+#define JH7100_CLK_MUX_SHIFT 24
+#define JH7100_CLK_DIV_MASK GENMASK(23, 0)
+
+/* clock data */
+#define JH7100_GATE(_idx, _name, _flags, _parent) [_idx] = { \
+ .name = _name, \
+ .flags = CLK_SET_RATE_PARENT | (_flags), \
+ .max = JH7100_CLK_ENABLE, \
+ .parents = { [0] = _parent }, \
+}
+
+#define JH7100__DIV(_idx, _name, _max, _parent) [_idx] = { \
+ .name = _name, \
+ .flags = 0, \
+ .max = _max, \
+ .parents = { [0] = _parent }, \
+}
+
+#define JH7100_GDIV(_idx, _name, _flags, _max, _parent) [_idx] = { \
+ .name = _name, \
+ .flags = _flags, \
+ .max = JH7100_CLK_ENABLE | (_max), \
+ .parents = { [0] = _parent }, \
+}
+
+#define JH7100__MUX(_idx, _name, _nparents, ...) [_idx] = { \
+ .name = _name, \
+ .flags = 0, \
+ .max = ((_nparents) - 1) << JH7100_CLK_MUX_SHIFT, \
+ .parents = { __VA_ARGS__ }, \
+}
+
+#define JH7100_GMUX(_idx, _name, _flags, _nparents, ...) [_idx] = { \
+ .name = _name, \
+ .flags = _flags, \
+ .max = JH7100_CLK_ENABLE | \
+ (((_nparents) - 1) << JH7100_CLK_MUX_SHIFT), \
+ .parents = { __VA_ARGS__ }, \
+}
+
+#define JH7100__INV(_idx, _name, _parent) [_idx] = { \
+ .name = _name, \
+ .flags = CLK_SET_RATE_PARENT, \
+ .max = JH7100_CLK_INVERT, \
+ .parents = { [0] = _parent }, \
+}
+
+static const struct {
+ const char *name;
+ unsigned long flags;
+ u32 max;
+ u8 parents[4];
+} jh7100_clk_data[] __initconst = {
+ JH7100__MUX(JH7100_CLK_CPUNDBUS_ROOT, "cpundbus_root", 4,
+ JH7100_CLK_OSC_SYS,
+ JH7100_CLK_PLL0_OUT,
+ JH7100_CLK_PLL1_OUT,
+ JH7100_CLK_PLL2_OUT),
+ JH7100__MUX(JH7100_CLK_DLA_ROOT, "dla_root", 3,
+ JH7100_CLK_OSC_SYS,
+ JH7100_CLK_PLL1_OUT,
+ JH7100_CLK_PLL2_OUT),
+ JH7100__MUX(JH7100_CLK_DSP_ROOT, "dsp_root", 4,
+ JH7100_CLK_OSC_SYS,
+ JH7100_CLK_PLL0_OUT,
+ JH7100_CLK_PLL1_OUT,
+ JH7100_CLK_PLL2_OUT),
+ JH7100__MUX(JH7100_CLK_GMACUSB_ROOT, "gmacusb_root", 3,
+ JH7100_CLK_OSC_SYS,
+ JH7100_CLK_PLL0_OUT,
+ JH7100_CLK_PLL2_OUT),
+ JH7100__MUX(JH7100_CLK_PERH0_ROOT, "perh0_root", 2,
+ JH7100_CLK_OSC_SYS,
+ JH7100_CLK_PLL0_OUT),
+ JH7100__MUX(JH7100_CLK_PERH1_ROOT, "perh1_root", 2,
+ JH7100_CLK_OSC_SYS,
+ JH7100_CLK_PLL2_OUT),
+ JH7100__MUX(JH7100_CLK_VIN_ROOT, "vin_root", 3,
+ JH7100_CLK_OSC_SYS,
+ JH7100_CLK_PLL1_OUT,
+ JH7100_CLK_PLL2_OUT),
+ JH7100__MUX(JH7100_CLK_VOUT_ROOT, "vout_root", 3,
+ JH7100_CLK_OSC_AUD,
+ JH7100_CLK_PLL0_OUT,
+ JH7100_CLK_PLL2_OUT),
+ JH7100_GDIV(JH7100_CLK_AUDIO_ROOT, "audio_root", 0, 8, JH7100_CLK_PLL0_OUT),
+ JH7100__MUX(JH7100_CLK_CDECHIFI4_ROOT, "cdechifi4_root", 3,
+ JH7100_CLK_OSC_SYS,
+ JH7100_CLK_PLL1_OUT,
+ JH7100_CLK_PLL2_OUT),
+ JH7100__MUX(JH7100_CLK_CDEC_ROOT, "cdec_root", 3,
+ JH7100_CLK_OSC_SYS,
+ JH7100_CLK_PLL0_OUT,
+ JH7100_CLK_PLL1_OUT),
+ JH7100__MUX(JH7100_CLK_VOUTBUS_ROOT, "voutbus_root", 3,
+ JH7100_CLK_OSC_AUD,
+ JH7100_CLK_PLL0_OUT,
+ JH7100_CLK_PLL2_OUT),
+ JH7100__DIV(JH7100_CLK_CPUNBUS_ROOT_DIV, "cpunbus_root_div", 2, JH7100_CLK_CPUNDBUS_ROOT),
+ JH7100__DIV(JH7100_CLK_DSP_ROOT_DIV, "dsp_root_div", 4, JH7100_CLK_DSP_ROOT),
+ JH7100__DIV(JH7100_CLK_PERH0_SRC, "perh0_src", 4, JH7100_CLK_PERH0_ROOT),
+ JH7100__DIV(JH7100_CLK_PERH1_SRC, "perh1_src", 4, JH7100_CLK_PERH1_ROOT),
+ JH7100_GDIV(JH7100_CLK_PLL0_TESTOUT, "pll0_testout", 0, 31, JH7100_CLK_PERH0_SRC),
+ JH7100_GDIV(JH7100_CLK_PLL1_TESTOUT, "pll1_testout", 0, 31, JH7100_CLK_DLA_ROOT),
+ JH7100_GDIV(JH7100_CLK_PLL2_TESTOUT, "pll2_testout", 0, 31, JH7100_CLK_PERH1_SRC),
+ JH7100__MUX(JH7100_CLK_PLL2_REF, "pll2_refclk", 2,
+ JH7100_CLK_OSC_SYS,
+ JH7100_CLK_OSC_AUD),
+ JH7100__DIV(JH7100_CLK_CPU_CORE, "cpu_core", 8, JH7100_CLK_CPUNBUS_ROOT_DIV),
+ JH7100__DIV(JH7100_CLK_CPU_AXI, "cpu_axi", 8, JH7100_CLK_CPU_CORE),
+ JH7100__DIV(JH7100_CLK_AHB_BUS, "ahb_bus", 8, JH7100_CLK_CPUNBUS_ROOT_DIV),
+ JH7100__DIV(JH7100_CLK_APB1_BUS, "apb1_bus", 8, JH7100_CLK_AHB_BUS),
+ JH7100__DIV(JH7100_CLK_APB2_BUS, "apb2_bus", 8, JH7100_CLK_AHB_BUS),
+ JH7100_GATE(JH7100_CLK_DOM3AHB_BUS, "dom3ahb_bus", CLK_IS_CRITICAL, JH7100_CLK_AHB_BUS),
+ JH7100_GATE(JH7100_CLK_DOM7AHB_BUS, "dom7ahb_bus", CLK_IS_CRITICAL, JH7100_CLK_AHB_BUS),
+ JH7100_GATE(JH7100_CLK_U74_CORE0, "u74_core0", CLK_IS_CRITICAL, JH7100_CLK_CPU_CORE),
+ JH7100_GDIV(JH7100_CLK_U74_CORE1, "u74_core1", CLK_IS_CRITICAL, 8, JH7100_CLK_CPU_CORE),
+ JH7100_GATE(JH7100_CLK_U74_AXI, "u74_axi", CLK_IS_CRITICAL, JH7100_CLK_CPU_AXI),
+ JH7100_GATE(JH7100_CLK_U74RTC_TOGGLE, "u74rtc_toggle", CLK_IS_CRITICAL, JH7100_CLK_OSC_SYS),
+ JH7100_GATE(JH7100_CLK_SGDMA2P_AXI, "sgdma2p_axi", 0, JH7100_CLK_CPU_AXI),
+ JH7100_GATE(JH7100_CLK_DMA2PNOC_AXI, "dma2pnoc_axi", 0, JH7100_CLK_CPU_AXI),
+ JH7100_GATE(JH7100_CLK_SGDMA2P_AHB, "sgdma2p_ahb", 0, JH7100_CLK_AHB_BUS),
+ JH7100__DIV(JH7100_CLK_DLA_BUS, "dla_bus", 4, JH7100_CLK_DLA_ROOT),
+ JH7100_GATE(JH7100_CLK_DLA_AXI, "dla_axi", 0, JH7100_CLK_DLA_BUS),
+ JH7100_GATE(JH7100_CLK_DLANOC_AXI, "dlanoc_axi", 0, JH7100_CLK_DLA_BUS),
+ JH7100_GATE(JH7100_CLK_DLA_APB, "dla_apb", 0, JH7100_CLK_APB1_BUS),
+ JH7100_GDIV(JH7100_CLK_VP6_CORE, "vp6_core", 0, 4, JH7100_CLK_DSP_ROOT_DIV),
+ JH7100__DIV(JH7100_CLK_VP6BUS_SRC, "vp6bus_src", 4, JH7100_CLK_DSP_ROOT),
+ JH7100_GDIV(JH7100_CLK_VP6_AXI, "vp6_axi", 0, 4, JH7100_CLK_VP6BUS_SRC),
+ JH7100__DIV(JH7100_CLK_VCDECBUS_SRC, "vcdecbus_src", 4, JH7100_CLK_CDECHIFI4_ROOT),
+ JH7100__DIV(JH7100_CLK_VDEC_BUS, "vdec_bus", 8, JH7100_CLK_VCDECBUS_SRC),
+ JH7100_GATE(JH7100_CLK_VDEC_AXI, "vdec_axi", 0, JH7100_CLK_VDEC_BUS),
+ JH7100_GATE(JH7100_CLK_VDECBRG_MAIN, "vdecbrg_mainclk", 0, JH7100_CLK_VDEC_BUS),
+ JH7100_GDIV(JH7100_CLK_VDEC_BCLK, "vdec_bclk", 0, 8, JH7100_CLK_VCDECBUS_SRC),
+ JH7100_GDIV(JH7100_CLK_VDEC_CCLK, "vdec_cclk", 0, 8, JH7100_CLK_CDEC_ROOT),
+ JH7100_GATE(JH7100_CLK_VDEC_APB, "vdec_apb", 0, JH7100_CLK_APB1_BUS),
+ JH7100_GDIV(JH7100_CLK_JPEG_AXI, "jpeg_axi", 0, 8, JH7100_CLK_CPUNBUS_ROOT_DIV),
+ JH7100_GDIV(JH7100_CLK_JPEG_CCLK, "jpeg_cclk", 0, 8, JH7100_CLK_CPUNBUS_ROOT_DIV),
+ JH7100_GATE(JH7100_CLK_JPEG_APB, "jpeg_apb", 0, JH7100_CLK_APB1_BUS),
+ JH7100_GDIV(JH7100_CLK_GC300_2X, "gc300_2x", 0, 8, JH7100_CLK_CDECHIFI4_ROOT),
+ JH7100_GATE(JH7100_CLK_GC300_AHB, "gc300_ahb", 0, JH7100_CLK_AHB_BUS),
+ JH7100__DIV(JH7100_CLK_JPCGC300_AXIBUS, "jpcgc300_axibus", 8, JH7100_CLK_VCDECBUS_SRC),
+ JH7100_GATE(JH7100_CLK_GC300_AXI, "gc300_axi", 0, JH7100_CLK_JPCGC300_AXIBUS),
+ JH7100_GATE(JH7100_CLK_JPCGC300_MAIN, "jpcgc300_mainclk", 0, JH7100_CLK_JPCGC300_AXIBUS),
+ JH7100__DIV(JH7100_CLK_VENC_BUS, "venc_bus", 8, JH7100_CLK_VCDECBUS_SRC),
+ JH7100_GATE(JH7100_CLK_VENC_AXI, "venc_axi", 0, JH7100_CLK_VENC_BUS),
+ JH7100_GATE(JH7100_CLK_VENCBRG_MAIN, "vencbrg_mainclk", 0, JH7100_CLK_VENC_BUS),
+ JH7100_GDIV(JH7100_CLK_VENC_BCLK, "venc_bclk", 0, 8, JH7100_CLK_VCDECBUS_SRC),
+ JH7100_GDIV(JH7100_CLK_VENC_CCLK, "venc_cclk", 0, 8, JH7100_CLK_CDEC_ROOT),
+ JH7100_GATE(JH7100_CLK_VENC_APB, "venc_apb", 0, JH7100_CLK_APB1_BUS),
+ JH7100_GDIV(JH7100_CLK_DDRPLL_DIV2, "ddrpll_div2", CLK_IS_CRITICAL, 2, JH7100_CLK_PLL1_OUT),
+ JH7100_GDIV(JH7100_CLK_DDRPLL_DIV4, "ddrpll_div4", CLK_IS_CRITICAL, 2, JH7100_CLK_DDRPLL_DIV2),
+ JH7100_GDIV(JH7100_CLK_DDRPLL_DIV8, "ddrpll_div8", CLK_IS_CRITICAL, 2, JH7100_CLK_DDRPLL_DIV4),
+ JH7100_GDIV(JH7100_CLK_DDROSC_DIV2, "ddrosc_div2", CLK_IS_CRITICAL, 2, JH7100_CLK_OSC_SYS),
+ JH7100_GMUX(JH7100_CLK_DDRC0, "ddrc0", CLK_IS_CRITICAL, 4,
+ JH7100_CLK_DDROSC_DIV2,
+ JH7100_CLK_DDRPLL_DIV2,
+ JH7100_CLK_DDRPLL_DIV4,
+ JH7100_CLK_DDRPLL_DIV8),
+ JH7100_GMUX(JH7100_CLK_DDRC1, "ddrc1", CLK_IS_CRITICAL, 4,
+ JH7100_CLK_DDROSC_DIV2,
+ JH7100_CLK_DDRPLL_DIV2,
+ JH7100_CLK_DDRPLL_DIV4,
+ JH7100_CLK_DDRPLL_DIV8),
+ JH7100_GATE(JH7100_CLK_DDRPHY_APB, "ddrphy_apb", 0, JH7100_CLK_APB1_BUS),
+ JH7100__DIV(JH7100_CLK_NOC_ROB, "noc_rob", 8, JH7100_CLK_CPUNBUS_ROOT_DIV),
+ JH7100__DIV(JH7100_CLK_NOC_COG, "noc_cog", 8, JH7100_CLK_DLA_ROOT),
+ JH7100_GATE(JH7100_CLK_NNE_AHB, "nne_ahb", 0, JH7100_CLK_AHB_BUS),
+ JH7100__DIV(JH7100_CLK_NNEBUS_SRC1, "nnebus_src1", 4, JH7100_CLK_DSP_ROOT),
+ JH7100__MUX(JH7100_CLK_NNE_BUS, "nne_bus", 2,
+ JH7100_CLK_CPU_AXI,
+ JH7100_CLK_NNEBUS_SRC1),
+ JH7100_GATE(JH7100_CLK_NNE_AXI, "nne_axi", 0, JH7100_CLK_NNE_BUS),
+ JH7100_GATE(JH7100_CLK_NNENOC_AXI, "nnenoc_axi", 0, JH7100_CLK_NNE_BUS),
+ JH7100_GATE(JH7100_CLK_DLASLV_AXI, "dlaslv_axi", 0, JH7100_CLK_NNE_BUS),
+ JH7100_GATE(JH7100_CLK_DSPX2C_AXI, "dspx2c_axi", CLK_IS_CRITICAL, JH7100_CLK_NNE_BUS),
+ JH7100__DIV(JH7100_CLK_HIFI4_SRC, "hifi4_src", 4, JH7100_CLK_CDECHIFI4_ROOT),
+ JH7100__DIV(JH7100_CLK_HIFI4_COREFREE, "hifi4_corefree", 8, JH7100_CLK_HIFI4_SRC),
+ JH7100_GATE(JH7100_CLK_HIFI4_CORE, "hifi4_core", 0, JH7100_CLK_HIFI4_COREFREE),
+ JH7100__DIV(JH7100_CLK_HIFI4_BUS, "hifi4_bus", 8, JH7100_CLK_HIFI4_COREFREE),
+ JH7100_GATE(JH7100_CLK_HIFI4_AXI, "hifi4_axi", 0, JH7100_CLK_HIFI4_BUS),
+ JH7100_GATE(JH7100_CLK_HIFI4NOC_AXI, "hifi4noc_axi", 0, JH7100_CLK_HIFI4_BUS),
+ JH7100__DIV(JH7100_CLK_SGDMA1P_BUS, "sgdma1p_bus", 8, JH7100_CLK_CPUNBUS_ROOT_DIV),
+ JH7100_GATE(JH7100_CLK_SGDMA1P_AXI, "sgdma1p_axi", 0, JH7100_CLK_SGDMA1P_BUS),
+ JH7100_GATE(JH7100_CLK_DMA1P_AXI, "dma1p_axi", 0, JH7100_CLK_SGDMA1P_BUS),
+ JH7100_GDIV(JH7100_CLK_X2C_AXI, "x2c_axi", CLK_IS_CRITICAL, 8, JH7100_CLK_CPUNBUS_ROOT_DIV),
+ JH7100__DIV(JH7100_CLK_USB_BUS, "usb_bus", 8, JH7100_CLK_CPUNBUS_ROOT_DIV),
+ JH7100_GATE(JH7100_CLK_USB_AXI, "usb_axi", 0, JH7100_CLK_USB_BUS),
+ JH7100_GATE(JH7100_CLK_USBNOC_AXI, "usbnoc_axi", 0, JH7100_CLK_USB_BUS),
+ JH7100__DIV(JH7100_CLK_USBPHY_ROOTDIV, "usbphy_rootdiv", 4, JH7100_CLK_GMACUSB_ROOT),
+ JH7100_GDIV(JH7100_CLK_USBPHY_125M, "usbphy_125m", 0, 8, JH7100_CLK_USBPHY_ROOTDIV),
+ JH7100_GDIV(JH7100_CLK_USBPHY_PLLDIV25M, "usbphy_plldiv25m", 0, 32, JH7100_CLK_USBPHY_ROOTDIV),
+ JH7100__MUX(JH7100_CLK_USBPHY_25M, "usbphy_25m", 2,
+ JH7100_CLK_OSC_SYS,
+ JH7100_CLK_USBPHY_PLLDIV25M),
+ JH7100__DIV(JH7100_CLK_AUDIO_DIV, "audio_div", 131072, JH7100_CLK_AUDIO_ROOT),
+ JH7100_GATE(JH7100_CLK_AUDIO_SRC, "audio_src", 0, JH7100_CLK_AUDIO_DIV),
+ JH7100_GATE(JH7100_CLK_AUDIO_12288, "audio_12288", 0, JH7100_CLK_OSC_AUD),
+ JH7100_GDIV(JH7100_CLK_VIN_SRC, "vin_src", 0, 4, JH7100_CLK_VIN_ROOT),
+ JH7100__DIV(JH7100_CLK_ISP0_BUS, "isp0_bus", 8, JH7100_CLK_VIN_SRC),
+ JH7100_GATE(JH7100_CLK_ISP0_AXI, "isp0_axi", 0, JH7100_CLK_ISP0_BUS),
+ JH7100_GATE(JH7100_CLK_ISP0NOC_AXI, "isp0noc_axi", 0, JH7100_CLK_ISP0_BUS),
+ JH7100_GATE(JH7100_CLK_ISPSLV_AXI, "ispslv_axi", 0, JH7100_CLK_ISP0_BUS),
+ JH7100__DIV(JH7100_CLK_ISP1_BUS, "isp1_bus", 8, JH7100_CLK_VIN_SRC),
+ JH7100_GATE(JH7100_CLK_ISP1_AXI, "isp1_axi", 0, JH7100_CLK_ISP1_BUS),
+ JH7100_GATE(JH7100_CLK_ISP1NOC_AXI, "isp1noc_axi", 0, JH7100_CLK_ISP1_BUS),
+ JH7100__DIV(JH7100_CLK_VIN_BUS, "vin_bus", 8, JH7100_CLK_VIN_SRC),
+ JH7100_GATE(JH7100_CLK_VIN_AXI, "vin_axi", 0, JH7100_CLK_VIN_BUS),
+ JH7100_GATE(JH7100_CLK_VINNOC_AXI, "vinnoc_axi", 0, JH7100_CLK_VIN_BUS),
+ JH7100_GDIV(JH7100_CLK_VOUT_SRC, "vout_src", 0, 4, JH7100_CLK_VOUT_ROOT),
+ JH7100__DIV(JH7100_CLK_DISPBUS_SRC, "dispbus_src", 4, JH7100_CLK_VOUTBUS_ROOT),
+ JH7100__DIV(JH7100_CLK_DISP_BUS, "disp_bus", 4, JH7100_CLK_DISPBUS_SRC),
+ JH7100_GATE(JH7100_CLK_DISP_AXI, "disp_axi", 0, JH7100_CLK_DISP_BUS),
+ JH7100_GATE(JH7100_CLK_DISPNOC_AXI, "dispnoc_axi", 0, JH7100_CLK_DISP_BUS),
+ JH7100_GATE(JH7100_CLK_SDIO0_AHB, "sdio0_ahb", 0, JH7100_CLK_AHB_BUS),
+ JH7100_GDIV(JH7100_CLK_SDIO0_CCLKINT, "sdio0_cclkint", 0, 24, JH7100_CLK_PERH0_SRC),
+ JH7100__INV(JH7100_CLK_SDIO0_CCLKINT_INV, "sdio0_cclkint_inv", JH7100_CLK_SDIO0_CCLKINT),
+ JH7100_GATE(JH7100_CLK_SDIO1_AHB, "sdio1_ahb", 0, JH7100_CLK_AHB_BUS),
+ JH7100_GDIV(JH7100_CLK_SDIO1_CCLKINT, "sdio1_cclkint", 0, 24, JH7100_CLK_PERH1_SRC),
+ JH7100__INV(JH7100_CLK_SDIO1_CCLKINT_INV, "sdio1_cclkint_inv", JH7100_CLK_SDIO1_CCLKINT),
+ JH7100_GATE(JH7100_CLK_GMAC_AHB, "gmac_ahb", 0, JH7100_CLK_AHB_BUS),
+ JH7100__DIV(JH7100_CLK_GMAC_ROOT_DIV, "gmac_root_div", 8, JH7100_CLK_GMACUSB_ROOT),
+ JH7100_GDIV(JH7100_CLK_GMAC_PTP_REF, "gmac_ptp_refclk", 0, 31, JH7100_CLK_GMAC_ROOT_DIV),
+ JH7100_GDIV(JH7100_CLK_GMAC_GTX, "gmac_gtxclk", 0, 255, JH7100_CLK_GMAC_ROOT_DIV),
+ JH7100_GDIV(JH7100_CLK_GMAC_RMII_TX, "gmac_rmii_txclk", 0, 8, JH7100_CLK_GMAC_RMII_REF),
+ JH7100_GDIV(JH7100_CLK_GMAC_RMII_RX, "gmac_rmii_rxclk", 0, 8, JH7100_CLK_GMAC_RMII_REF),
+ JH7100__MUX(JH7100_CLK_GMAC_TX, "gmac_tx", 3,
+ JH7100_CLK_GMAC_GTX,
+ JH7100_CLK_GMAC_TX_INV,
+ JH7100_CLK_GMAC_RMII_TX),
+ JH7100__INV(JH7100_CLK_GMAC_TX_INV, "gmac_tx_inv", JH7100_CLK_GMAC_TX),
+ JH7100__MUX(JH7100_CLK_GMAC_RX_PRE, "gmac_rx_pre", 2,
+ JH7100_CLK_GMAC_GR_MII_RX,
+ JH7100_CLK_GMAC_RMII_RX),
+ JH7100__INV(JH7100_CLK_GMAC_RX_INV, "gmac_rx_inv", JH7100_CLK_GMAC_RX_PRE),
+ JH7100_GATE(JH7100_CLK_GMAC_RMII, "gmac_rmii", 0, JH7100_CLK_GMAC_RMII_REF),
+ JH7100_GDIV(JH7100_CLK_GMAC_TOPHYREF, "gmac_tophyref", 0, 127, JH7100_CLK_GMAC_ROOT_DIV),
+ JH7100_GATE(JH7100_CLK_SPI2AHB_AHB, "spi2ahb_ahb", 0, JH7100_CLK_AHB_BUS),
+ JH7100_GDIV(JH7100_CLK_SPI2AHB_CORE, "spi2ahb_core", 0, 31, JH7100_CLK_PERH0_SRC),
+ JH7100_GATE(JH7100_CLK_EZMASTER_AHB, "ezmaster_ahb", 0, JH7100_CLK_AHB_BUS),
+ JH7100_GATE(JH7100_CLK_E24_AHB, "e24_ahb", 0, JH7100_CLK_AHB_BUS),
+ JH7100_GATE(JH7100_CLK_E24RTC_TOGGLE, "e24rtc_toggle", 0, JH7100_CLK_OSC_SYS),
+ JH7100_GATE(JH7100_CLK_QSPI_AHB, "qspi_ahb", 0, JH7100_CLK_AHB_BUS),
+ JH7100_GATE(JH7100_CLK_QSPI_APB, "qspi_apb", 0, JH7100_CLK_APB1_BUS),
+ JH7100_GDIV(JH7100_CLK_QSPI_REF, "qspi_refclk", 0, 31, JH7100_CLK_PERH0_SRC),
+ JH7100_GATE(JH7100_CLK_SEC_AHB, "sec_ahb", 0, JH7100_CLK_AHB_BUS),
+ JH7100_GATE(JH7100_CLK_AES, "aes_clk", 0, JH7100_CLK_SEC_AHB),
+ JH7100_GATE(JH7100_CLK_SHA, "sha_clk", 0, JH7100_CLK_SEC_AHB),
+ JH7100_GATE(JH7100_CLK_PKA, "pka_clk", 0, JH7100_CLK_SEC_AHB),
+ JH7100_GATE(JH7100_CLK_TRNG_APB, "trng_apb", 0, JH7100_CLK_APB1_BUS),
+ JH7100_GATE(JH7100_CLK_OTP_APB, "otp_apb", 0, JH7100_CLK_APB1_BUS),
+ JH7100_GATE(JH7100_CLK_UART0_APB, "uart0_apb", 0, JH7100_CLK_APB1_BUS),
+ JH7100_GDIV(JH7100_CLK_UART0_CORE, "uart0_core", 0, 63, JH7100_CLK_PERH1_SRC),
+ JH7100_GATE(JH7100_CLK_UART1_APB, "uart1_apb", 0, JH7100_CLK_APB1_BUS),
+ JH7100_GDIV(JH7100_CLK_UART1_CORE, "uart1_core", 0, 63, JH7100_CLK_PERH1_SRC),
+ JH7100_GATE(JH7100_CLK_SPI0_APB, "spi0_apb", 0, JH7100_CLK_APB1_BUS),
+ JH7100_GDIV(JH7100_CLK_SPI0_CORE, "spi0_core", 0, 63, JH7100_CLK_PERH1_SRC),
+ JH7100_GATE(JH7100_CLK_SPI1_APB, "spi1_apb", 0, JH7100_CLK_APB1_BUS),
+ JH7100_GDIV(JH7100_CLK_SPI1_CORE, "spi1_core", 0, 63, JH7100_CLK_PERH1_SRC),
+ JH7100_GATE(JH7100_CLK_I2C0_APB, "i2c0_apb", 0, JH7100_CLK_APB1_BUS),
+ JH7100_GDIV(JH7100_CLK_I2C0_CORE, "i2c0_core", 0, 63, JH7100_CLK_PERH1_SRC),
+ JH7100_GATE(JH7100_CLK_I2C1_APB, "i2c1_apb", 0, JH7100_CLK_APB1_BUS),
+ JH7100_GDIV(JH7100_CLK_I2C1_CORE, "i2c1_core", 0, 63, JH7100_CLK_PERH1_SRC),
+ JH7100_GATE(JH7100_CLK_GPIO_APB, "gpio_apb", 0, JH7100_CLK_APB1_BUS),
+ JH7100_GATE(JH7100_CLK_UART2_APB, "uart2_apb", 0, JH7100_CLK_APB2_BUS),
+ JH7100_GDIV(JH7100_CLK_UART2_CORE, "uart2_core", 0, 63, JH7100_CLK_PERH0_SRC),
+ JH7100_GATE(JH7100_CLK_UART3_APB, "uart3_apb", 0, JH7100_CLK_APB2_BUS),
+ JH7100_GDIV(JH7100_CLK_UART3_CORE, "uart3_core", 0, 63, JH7100_CLK_PERH0_SRC),
+ JH7100_GATE(JH7100_CLK_SPI2_APB, "spi2_apb", 0, JH7100_CLK_APB2_BUS),
+ JH7100_GDIV(JH7100_CLK_SPI2_CORE, "spi2_core", 0, 63, JH7100_CLK_PERH0_SRC),
+ JH7100_GATE(JH7100_CLK_SPI3_APB, "spi3_apb", 0, JH7100_CLK_APB2_BUS),
+ JH7100_GDIV(JH7100_CLK_SPI3_CORE, "spi3_core", 0, 63, JH7100_CLK_PERH0_SRC),
+ JH7100_GATE(JH7100_CLK_I2C2_APB, "i2c2_apb", 0, JH7100_CLK_APB2_BUS),
+ JH7100_GDIV(JH7100_CLK_I2C2_CORE, "i2c2_core", 0, 63, JH7100_CLK_PERH0_SRC),
+ JH7100_GATE(JH7100_CLK_I2C3_APB, "i2c3_apb", 0, JH7100_CLK_APB2_BUS),
+ JH7100_GDIV(JH7100_CLK_I2C3_CORE, "i2c3_core", 0, 63, JH7100_CLK_PERH0_SRC),
+ JH7100_GATE(JH7100_CLK_WDTIMER_APB, "wdtimer_apb", 0, JH7100_CLK_APB2_BUS),
+ JH7100_GDIV(JH7100_CLK_WDT_CORE, "wdt_coreclk", 0, 63, JH7100_CLK_PERH0_SRC),
+ JH7100_GDIV(JH7100_CLK_TIMER0_CORE, "timer0_coreclk", 0, 63, JH7100_CLK_PERH0_SRC),
+ JH7100_GDIV(JH7100_CLK_TIMER1_CORE, "timer1_coreclk", 0, 63, JH7100_CLK_PERH0_SRC),
+ JH7100_GDIV(JH7100_CLK_TIMER2_CORE, "timer2_coreclk", 0, 63, JH7100_CLK_PERH0_SRC),
+ JH7100_GDIV(JH7100_CLK_TIMER3_CORE, "timer3_coreclk", 0, 63, JH7100_CLK_PERH0_SRC),
+ JH7100_GDIV(JH7100_CLK_TIMER4_CORE, "timer4_coreclk", 0, 63, JH7100_CLK_PERH0_SRC),
+ JH7100_GDIV(JH7100_CLK_TIMER5_CORE, "timer5_coreclk", 0, 63, JH7100_CLK_PERH0_SRC),
+ JH7100_GDIV(JH7100_CLK_TIMER6_CORE, "timer6_coreclk", 0, 63, JH7100_CLK_PERH0_SRC),
+ JH7100_GATE(JH7100_CLK_VP6INTC_APB, "vp6intc_apb", 0, JH7100_CLK_APB2_BUS),
+ JH7100_GATE(JH7100_CLK_PWM_APB, "pwm_apb", 0, JH7100_CLK_APB2_BUS),
+ JH7100_GATE(JH7100_CLK_MSI_APB, "msi_apb", 0, JH7100_CLK_APB2_BUS),
+ JH7100_GATE(JH7100_CLK_TEMP_APB, "temp_apb", 0, JH7100_CLK_APB2_BUS),
+ JH7100_GDIV(JH7100_CLK_TEMP_SENSE, "temp_sense", 0, 31, JH7100_CLK_OSC_SYS),
+ JH7100_GATE(JH7100_CLK_SYSERR_APB, "syserr_apb", 0, JH7100_CLK_APB2_BUS),
+};
+
+struct jh7100_clk {
+ struct clk_hw hw;
+ unsigned int idx;
+ unsigned int max_div;
+};
+
+struct jh7100_clk_priv {
+ /* protect clk enable and set rate/parent from happening at the same time */
+ spinlock_t rmw_lock;
+ struct device *dev;
+ void __iomem *base;
+ struct clk_hw *pll[3];
+ struct jh7100_clk reg[JH7100_CLK_PLL0_OUT];
+};
+
+static struct jh7100_clk *jh7100_clk_from(struct clk_hw *hw)
+{
+ return container_of(hw, struct jh7100_clk, hw);
+}
+
+static struct jh7100_clk_priv *jh7100_priv_from(struct jh7100_clk *clk)
+{
+ return container_of(clk, struct jh7100_clk_priv, reg[clk->idx]);
+}
+
+static u32 jh7100_clk_reg_get(struct jh7100_clk *clk)
+{
+ struct jh7100_clk_priv *priv = jh7100_priv_from(clk);
+ void __iomem *reg = priv->base + 4 * clk->idx;
+
+ return readl_relaxed(reg);
+}
+
+static void jh7100_clk_reg_rmw(struct jh7100_clk *clk, u32 mask, u32 value)
+{
+ struct jh7100_clk_priv *priv = jh7100_priv_from(clk);
+ void __iomem *reg = priv->base + 4 * clk->idx;
+ unsigned long flags;
+
+ spin_lock_irqsave(&priv->rmw_lock, flags);
+ value |= readl_relaxed(reg) & ~mask;
+ writel_relaxed(value, reg);
+ spin_unlock_irqrestore(&priv->rmw_lock, flags);
+}
+
+static int jh7100_clk_enable(struct clk_hw *hw)
+{
+ struct jh7100_clk *clk = jh7100_clk_from(hw);
+
+ jh7100_clk_reg_rmw(clk, JH7100_CLK_ENABLE, JH7100_CLK_ENABLE);
+ return 0;
+}
+
+static void jh7100_clk_disable(struct clk_hw *hw)
+{
+ struct jh7100_clk *clk = jh7100_clk_from(hw);
+
+ jh7100_clk_reg_rmw(clk, JH7100_CLK_ENABLE, 0);
+}
+
+static int jh7100_clk_is_enabled(struct clk_hw *hw)
+{
+ struct jh7100_clk *clk = jh7100_clk_from(hw);
+
+ return !!(jh7100_clk_reg_get(clk) & JH7100_CLK_ENABLE);
+}
+
+static unsigned long jh7100_clk_recalc_rate(struct clk_hw *hw,
+ unsigned long parent_rate)
+{
+ struct jh7100_clk *clk = jh7100_clk_from(hw);
+ u32 div = jh7100_clk_reg_get(clk) & JH7100_CLK_DIV_MASK;
+
+ return div ? parent_rate / div : 0;
+}
+
+static unsigned long jh7100_clk_bestdiv(struct jh7100_clk *clk,
+ unsigned long rate, unsigned long parent)
+{
+ unsigned long max = clk->max_div;
+ unsigned long div = DIV_ROUND_UP(parent, rate);
+
+ return min(div, max);
+}
+
+static int jh7100_clk_determine_rate(struct clk_hw *hw,
+ struct clk_rate_request *req)
+{
+ struct jh7100_clk *clk = jh7100_clk_from(hw);
+ unsigned long parent = req->best_parent_rate;
+ unsigned long rate = clamp(req->rate, req->min_rate, req->max_rate);
+ unsigned long div = jh7100_clk_bestdiv(clk, rate, parent);
+ unsigned long result = parent / div;
+
+ /*
+ * we want the result clamped by min_rate and max_rate if possible:
+ * case 1: div hits the max divider value, which means it's less than
+ * parent / rate, so the result is greater than rate and min_rate in
+ * particular. we can't do anything about result > max_rate because the
+ * divider doesn't go any further.
+ * case 2: div = DIV_ROUND_UP(parent, rate) which means the result is
+ * always lower or equal to rate and max_rate. however the result may
+ * turn out lower than min_rate, but then the next higher rate is fine:
+ * div - 1 = ceil(parent / rate) - 1 < parent / rate
+ * and thus
+ * min_rate <= rate < parent / (div - 1)
+ */
+ if (result < req->min_rate && div > 1)
+ result = parent / (div - 1);
+
+ req->rate = result;
+ return 0;
+}
+
+static int jh7100_clk_set_rate(struct clk_hw *hw,
+ unsigned long rate,
+ unsigned long parent_rate)
+{
+ struct jh7100_clk *clk = jh7100_clk_from(hw);
+ unsigned long div = jh7100_clk_bestdiv(clk, rate, parent_rate);
+
+ jh7100_clk_reg_rmw(clk, JH7100_CLK_DIV_MASK, div);
+ return 0;
+}
+
+static u8 jh7100_clk_get_parent(struct clk_hw *hw)
+{
+ struct jh7100_clk *clk = jh7100_clk_from(hw);
+ u32 value = jh7100_clk_reg_get(clk);
+
+ return (value & JH7100_CLK_MUX_MASK) >> JH7100_CLK_MUX_SHIFT;
+}
+
+static int jh7100_clk_set_parent(struct clk_hw *hw, u8 index)
+{
+ struct jh7100_clk *clk = jh7100_clk_from(hw);
+ u32 value = (u32)index << JH7100_CLK_MUX_SHIFT;
+
+ jh7100_clk_reg_rmw(clk, JH7100_CLK_MUX_MASK, value);
+ return 0;
+}
+
+static int jh7100_clk_mux_determine_rate(struct clk_hw *hw,
+ struct clk_rate_request *req)
+{
+ return clk_mux_determine_rate_flags(hw, req, 0);
+}
+
+static int jh7100_clk_get_phase(struct clk_hw *hw)
+{
+ struct jh7100_clk *clk = jh7100_clk_from(hw);
+ u32 value = jh7100_clk_reg_get(clk);
+
+ return (value & JH7100_CLK_INVERT) ? 180 : 0;
+}
+
+static int jh7100_clk_set_phase(struct clk_hw *hw, int degrees)
+{
+ struct jh7100_clk *clk = jh7100_clk_from(hw);
+ u32 value;
+
+ if (degrees == 0)
+ value = 0;
+ else if (degrees == 180)
+ value = JH7100_CLK_INVERT;
+ else
+ return -EINVAL;
+
+ jh7100_clk_reg_rmw(clk, JH7100_CLK_INVERT, value);
+ return 0;
+}
+
+#ifdef CONFIG_DEBUG_FS
+static void jh7100_clk_debug_init(struct clk_hw *hw, struct dentry *dentry)
+{
+ static const struct debugfs_reg32 jh7100_clk_reg = {
+ .name = "CTRL",
+ .offset = 0,
+ };
+ struct jh7100_clk *clk = jh7100_clk_from(hw);
+ struct jh7100_clk_priv *priv = jh7100_priv_from(clk);
+ struct debugfs_regset32 *regset;
+
+ regset = devm_kzalloc(priv->dev, sizeof(*regset), GFP_KERNEL);
+ if (!regset)
+ return;
+
+ regset->regs = &jh7100_clk_reg;
+ regset->nregs = 1;
+ regset->base = priv->base + 4 * clk->idx;
+
+ debugfs_create_regset32("registers", 0400, dentry, regset);
+}
+#else
+#define jh7100_clk_debug_init NULL
+#endif
+
+static const struct clk_ops jh7100_clk_gate_ops = {
+ .enable = jh7100_clk_enable,
+ .disable = jh7100_clk_disable,
+ .is_enabled = jh7100_clk_is_enabled,
+ .debug_init = jh7100_clk_debug_init,
+};
+
+static const struct clk_ops jh7100_clk_div_ops = {
+ .recalc_rate = jh7100_clk_recalc_rate,
+ .determine_rate = jh7100_clk_determine_rate,
+ .set_rate = jh7100_clk_set_rate,
+ .debug_init = jh7100_clk_debug_init,
+};
+
+static const struct clk_ops jh7100_clk_gdiv_ops = {
+ .enable = jh7100_clk_enable,
+ .disable = jh7100_clk_disable,
+ .is_enabled = jh7100_clk_is_enabled,
+ .recalc_rate = jh7100_clk_recalc_rate,
+ .determine_rate = jh7100_clk_determine_rate,
+ .set_rate = jh7100_clk_set_rate,
+ .debug_init = jh7100_clk_debug_init,
+};
+
+static const struct clk_ops jh7100_clk_mux_ops = {
+ .determine_rate = jh7100_clk_mux_determine_rate,
+ .set_parent = jh7100_clk_set_parent,
+ .get_parent = jh7100_clk_get_parent,
+ .debug_init = jh7100_clk_debug_init,
+};
+
+static const struct clk_ops jh7100_clk_gmux_ops = {
+ .enable = jh7100_clk_enable,
+ .disable = jh7100_clk_disable,
+ .is_enabled = jh7100_clk_is_enabled,
+ .determine_rate = jh7100_clk_mux_determine_rate,
+ .set_parent = jh7100_clk_set_parent,
+ .get_parent = jh7100_clk_get_parent,
+ .debug_init = jh7100_clk_debug_init,
+};
+
+static const struct clk_ops jh7100_clk_inv_ops = {
+ .get_phase = jh7100_clk_get_phase,
+ .set_phase = jh7100_clk_set_phase,
+ .debug_init = jh7100_clk_debug_init,
+};
+
+static const struct clk_ops *__init jh7100_clk_ops(u32 max)
+{
+ if (max & JH7100_CLK_DIV_MASK) {
+ if (max & JH7100_CLK_ENABLE)
+ return &jh7100_clk_gdiv_ops;
+ return &jh7100_clk_div_ops;
+ }
+
+ if (max & JH7100_CLK_MUX_MASK) {
+ if (max & JH7100_CLK_ENABLE)
+ return &jh7100_clk_gmux_ops;
+ return &jh7100_clk_mux_ops;
+ }
+
+ if (max & JH7100_CLK_ENABLE)
+ return &jh7100_clk_gate_ops;
+
+ return &jh7100_clk_inv_ops;
+}
+
+static struct clk_hw *jh7100_clk_get(struct of_phandle_args *clkspec, void *data)
+{
+ struct jh7100_clk_priv *priv = data;
+ unsigned int idx = clkspec->args[0];
+
+ if (idx < JH7100_CLK_PLL0_OUT)
+ return &priv->reg[idx].hw;
+
+ if (idx < JH7100_CLK_END)
+ return priv->pll[idx - JH7100_CLK_PLL0_OUT];
+
+ return ERR_PTR(-EINVAL);
+}
+
+static int __init clk_starfive_jh7100_probe(struct platform_device *pdev)
+{
+ struct jh7100_clk_priv *priv;
+ unsigned int idx;
+ int ret;
+
+ priv = devm_kzalloc(&pdev->dev, sizeof(*priv), GFP_KERNEL);
+ if (!priv)
+ return -ENOMEM;
+
+ spin_lock_init(&priv->rmw_lock);
+ priv->dev = &pdev->dev;
+ priv->base = devm_platform_ioremap_resource(pdev, 0);
+ if (IS_ERR(priv->base))
+ return PTR_ERR(priv->base);
+
+ priv->pll[0] = devm_clk_hw_register_fixed_factor(priv->dev, "pll0_out",
+ "osc_sys", 0, 40, 1);
+ if (IS_ERR(priv->pll[0]))
+ return PTR_ERR(priv->pll[0]);
+
+ priv->pll[1] = devm_clk_hw_register_fixed_factor(priv->dev, "pll1_out",
+ "osc_sys", 0, 64, 1);
+ if (IS_ERR(priv->pll[1]))
+ return PTR_ERR(priv->pll[1]);
+
+ priv->pll[2] = devm_clk_hw_register_fixed_factor(priv->dev, "pll2_out",
+ "pll2_refclk", 0, 55, 1);
+ if (IS_ERR(priv->pll[2]))
+ return PTR_ERR(priv->pll[2]);
+
+ for (idx = 0; idx < JH7100_CLK_PLL0_OUT; idx++) {
+ u32 max = jh7100_clk_data[idx].max;
+ struct clk_parent_data parents[4] = {};
+ struct clk_init_data init = {
+ .name = jh7100_clk_data[idx].name,
+ .ops = jh7100_clk_ops(max),
+ .parent_data = parents,
+ .num_parents = ((max & JH7100_CLK_MUX_MASK) >> JH7100_CLK_MUX_SHIFT) + 1,
+ .flags = jh7100_clk_data[idx].flags,
+ };
+ struct jh7100_clk *clk = &priv->reg[idx];
+ unsigned int i;
+
+ for (i = 0; i < init.num_parents; i++) {
+ unsigned int pidx = jh7100_clk_data[idx].parents[i];
+
+ if (pidx < JH7100_CLK_PLL0_OUT)
+ parents[i].hw = &priv->reg[pidx].hw;
+ else if (pidx < JH7100_CLK_END)
+ parents[i].hw = priv->pll[pidx - JH7100_CLK_PLL0_OUT];
+ else if (pidx == JH7100_CLK_OSC_SYS)
+ parents[i].fw_name = "osc_sys";
+ else if (pidx == JH7100_CLK_OSC_AUD)
+ parents[i].fw_name = "osc_aud";
+ else if (pidx == JH7100_CLK_GMAC_RMII_REF)
+ parents[i].fw_name = "gmac_rmii_ref";
+ else if (pidx == JH7100_CLK_GMAC_GR_MII_RX)
+ parents[i].fw_name = "gmac_gr_mii_rxclk";
+ }
+
+ clk->hw.init = &init;
+ clk->idx = idx;
+ clk->max_div = max & JH7100_CLK_DIV_MASK;
+
+ ret = devm_clk_hw_register(priv->dev, &clk->hw);
+ if (ret)
+ return ret;
+ }
+
+ return devm_of_clk_add_hw_provider(priv->dev, jh7100_clk_get, priv);
+}
+
+static const struct of_device_id clk_starfive_jh7100_match[] = {
+ { .compatible = "starfive,jh7100-clkgen" },
+ { /* sentinel */ }
+};
+
+static struct platform_driver clk_starfive_jh7100_driver = {
+ .driver = {
+ .name = "clk-starfive-jh7100",
+ .of_match_table = clk_starfive_jh7100_match,
+ .suppress_bind_attrs = true,
+ },
+};
+builtin_platform_driver_probe(clk_starfive_jh7100_driver, clk_starfive_jh7100_probe);
diff --git a/drivers/clk/sunxi-ng/ccu-sun50i-a64.h b/drivers/clk/sunxi-ng/ccu-sun50i-a64.h
index 54d1f96..a8c11c0 100644
--- a/drivers/clk/sunxi-ng/ccu-sun50i-a64.h
+++ b/drivers/clk/sunxi-ng/ccu-sun50i-a64.h
@@ -51,8 +51,6 @@
#define CLK_USB_OHCI1_12M 92
-#define CLK_DRAM 94
-
/* All the DRAM gates are exported */
/* And the DSI and GPU module clock is exported */
diff --git a/drivers/clk/sunxi-ng/ccu-sun8i-h3.h b/drivers/clk/sunxi-ng/ccu-sun8i-h3.h
index d8c3844..e13f3c4 100644
--- a/drivers/clk/sunxi-ng/ccu-sun8i-h3.h
+++ b/drivers/clk/sunxi-ng/ccu-sun8i-h3.h
@@ -42,8 +42,6 @@
/* The first bunch of module clocks are exported */
-#define CLK_DRAM 96
-
/* All the DRAM gates are exported */
/* Some more module clocks are exported */
diff --git a/drivers/crypto/Kconfig b/drivers/crypto/Kconfig
index 51690e7..4f70567 100644
--- a/drivers/crypto/Kconfig
+++ b/drivers/crypto/Kconfig
@@ -213,6 +213,18 @@
key sizes and XTS mode is hardware accelerated for 256 and
512 bit keys.
+config CRYPTO_CHACHA_S390
+ tristate "ChaCha20 stream cipher"
+ depends on S390
+ select CRYPTO_ALGAPI
+ select CRYPTO_SKCIPHER
+ select CRYPTO_CHACHA20
+ help
+ This is the s390 SIMD implementation of the ChaCha20 stream
+ cipher (RFC 7539).
+
+ It is available as of z13.
+
config S390_PRNG
tristate "Pseudo random number generator device driver"
depends on S390
diff --git a/drivers/dma-buf/Makefile b/drivers/dma-buf/Makefile
index 1ef0212..511805d 100644
--- a/drivers/dma-buf/Makefile
+++ b/drivers/dma-buf/Makefile
@@ -11,6 +11,7 @@
dmabuf_selftests-y := \
selftest.o \
st-dma-fence.o \
- st-dma-fence-chain.o
+ st-dma-fence-chain.o \
+ st-dma-resv.o
obj-$(CONFIG_DMABUF_SELFTESTS) += dmabuf_selftests.o
diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c
index 6437b2e..602b12d 100644
--- a/drivers/dma-buf/dma-buf.c
+++ b/drivers/dma-buf/dma-buf.c
@@ -299,10 +299,8 @@ static __poll_t dma_buf_poll(struct file *file, poll_table *poll)
/**
* dma_buf_set_name - Set a name to a specific dma_buf to track the usage.
- * The name of the dma-buf buffer can only be set when the dma-buf is not
- * attached to any devices. It could theoritically support changing the
- * name of the dma-buf if the same piece of memory is used for multiple
- * purpose between different devices.
+ * It could support changing the name of the dma-buf if the same
+ * piece of memory is used for multiple purpose between different devices.
*
* @dmabuf: [in] dmabuf buffer that will be renamed.
* @buf: [in] A piece of userspace memory that contains the name of
@@ -315,25 +313,16 @@ static __poll_t dma_buf_poll(struct file *file, poll_table *poll)
static long dma_buf_set_name(struct dma_buf *dmabuf, const char __user *buf)
{
char *name = strndup_user(buf, DMA_BUF_NAME_LEN);
- long ret = 0;
if (IS_ERR(name))
return PTR_ERR(name);
- dma_resv_lock(dmabuf->resv, NULL);
- if (!list_empty(&dmabuf->attachments)) {
- ret = -EBUSY;
- kfree(name);
- goto out_unlock;
- }
spin_lock(&dmabuf->name_lock);
kfree(dmabuf->name);
dmabuf->name = name;
spin_unlock(&dmabuf->name_lock);
-out_unlock:
- dma_resv_unlock(dmabuf->resv);
- return ret;
+ return 0;
}
static long dma_buf_ioctl(struct file *file,
@@ -1058,8 +1047,8 @@ EXPORT_SYMBOL_NS_GPL(dma_buf_move_notify, DMA_BUF);
*
* Interfaces::
*
- * void \*dma_buf_vmap(struct dma_buf \*dmabuf)
- * void dma_buf_vunmap(struct dma_buf \*dmabuf, void \*vaddr)
+ * void \*dma_buf_vmap(struct dma_buf \*dmabuf, struct dma_buf_map \*map)
+ * void dma_buf_vunmap(struct dma_buf \*dmabuf, struct dma_buf_map \*map)
*
* The vmap call can fail if there is no vmap support in the exporter, or if
* it runs out of vmalloc space. Note that the dma-buf layer keeps a reference
@@ -1338,8 +1327,6 @@ static int dma_buf_debug_show(struct seq_file *s, void *unused)
{
struct dma_buf *buf_obj;
struct dma_buf_attachment *attach_obj;
- struct dma_resv_iter cursor;
- struct dma_fence *fence;
int count = 0, attach_count;
size_t size = 0;
int ret;
@@ -1370,14 +1357,7 @@ static int dma_buf_debug_show(struct seq_file *s, void *unused)
buf_obj->name ?: "");
spin_unlock(&buf_obj->name_lock);
- dma_resv_for_each_fence(&cursor, buf_obj->resv, true, fence) {
- seq_printf(s, "\t%s fence: %s %s %ssignalled\n",
- dma_resv_iter_is_exclusive(&cursor) ?
- "Exclusive" : "Shared",
- fence->ops->get_driver_name(fence),
- fence->ops->get_timeline_name(fence),
- dma_fence_is_signaled(fence) ? "" : "un");
- }
+ dma_resv_describe(buf_obj->resv, s);
seq_puts(s, "\tAttached Devices:\n");
attach_count = 0;
diff --git a/drivers/dma-buf/dma-fence-array.c b/drivers/dma-buf/dma-fence-array.c
index d3fbd95..3e07f96 100644
--- a/drivers/dma-buf/dma-fence-array.c
+++ b/drivers/dma-buf/dma-fence-array.c
@@ -104,7 +104,11 @@ static bool dma_fence_array_signaled(struct dma_fence *fence)
{
struct dma_fence_array *array = to_dma_fence_array(fence);
- return atomic_read(&array->num_pending) <= 0;
+ if (atomic_read(&array->num_pending) > 0)
+ return false;
+
+ dma_fence_array_clear_pending_error(array);
+ return true;
}
static void dma_fence_array_release(struct dma_fence *fence)
diff --git a/drivers/dma-buf/dma-fence.c b/drivers/dma-buf/dma-fence.c
index 1e82ecd..066400e 100644
--- a/drivers/dma-buf/dma-fence.c
+++ b/drivers/dma-buf/dma-fence.c
@@ -15,6 +15,7 @@
#include <linux/atomic.h>
#include <linux/dma-fence.h>
#include <linux/sched/signal.h>
+#include <linux/seq_file.h>
#define CREATE_TRACE_POINTS
#include <trace/events/dma_fence.h>
@@ -908,6 +909,22 @@ dma_fence_wait_any_timeout(struct dma_fence **fences, uint32_t count,
EXPORT_SYMBOL(dma_fence_wait_any_timeout);
/**
+ * dma_fence_describe - Dump fence describtion into seq_file
+ * @fence: the 6fence to describe
+ * @seq: the seq_file to put the textual description into
+ *
+ * Dump a textual description of the fence and it's state into the seq_file.
+ */
+void dma_fence_describe(struct dma_fence *fence, struct seq_file *seq)
+{
+ seq_printf(seq, "%s %s seq %llu %ssignalled\n",
+ fence->ops->get_driver_name(fence),
+ fence->ops->get_timeline_name(fence), fence->seqno,
+ dma_fence_is_signaled(fence) ? "" : "un");
+}
+EXPORT_SYMBOL(dma_fence_describe);
+
+/**
* dma_fence_init - Initialize a custom fence.
* @fence: the fence to initialize
* @ops: the dma_fence_ops for operations on this fence
diff --git a/drivers/dma-buf/dma-resv.c b/drivers/dma-buf/dma-resv.c
index 9eb2baa..4deea75 100644
--- a/drivers/dma-buf/dma-resv.c
+++ b/drivers/dma-buf/dma-resv.c
@@ -38,6 +38,7 @@
#include <linux/mm.h>
#include <linux/sched/mm.h>
#include <linux/mmu_notifier.h>
+#include <linux/seq_file.h>
/**
* DOC: Reservation Object Overview
@@ -304,8 +305,7 @@ void dma_resv_add_excl_fence(struct dma_resv *obj, struct dma_fence *fence)
if (old)
i = old->shared_count;
- if (fence)
- dma_fence_get(fence);
+ dma_fence_get(fence);
write_seqcount_begin(&obj->seq);
/* write_seqcount_begin provides the necessary memory barrier */
@@ -666,6 +666,28 @@ bool dma_resv_test_signaled(struct dma_resv *obj, bool test_all)
}
EXPORT_SYMBOL_GPL(dma_resv_test_signaled);
+/**
+ * dma_resv_describe - Dump description of the resv object into seq_file
+ * @obj: the reservation object
+ * @seq: the seq_file to dump the description into
+ *
+ * Dump a textual description of the fences inside an dma_resv object into the
+ * seq_file.
+ */
+void dma_resv_describe(struct dma_resv *obj, struct seq_file *seq)
+{
+ struct dma_resv_iter cursor;
+ struct dma_fence *fence;
+
+ dma_resv_for_each_fence(&cursor, obj, true, fence) {
+ seq_printf(seq, "\t%s fence:",
+ dma_resv_iter_is_exclusive(&cursor) ?
+ "Exclusive" : "Shared");
+ dma_fence_describe(fence, seq);
+ }
+}
+EXPORT_SYMBOL_GPL(dma_resv_describe);
+
#if IS_ENABLED(CONFIG_LOCKDEP)
static int __init dma_resv_lockdep(void)
{
diff --git a/drivers/dma-buf/selftests.h b/drivers/dma-buf/selftests.h
index bc8cea6..97d73aa 100644
--- a/drivers/dma-buf/selftests.h
+++ b/drivers/dma-buf/selftests.h
@@ -12,3 +12,4 @@
selftest(sanitycheck, __sanitycheck__) /* keep first (igt selfcheck) */
selftest(dma_fence, dma_fence)
selftest(dma_fence_chain, dma_fence_chain)
+selftest(dma_resv, dma_resv)
diff --git a/drivers/dma-buf/st-dma-resv.c b/drivers/dma-buf/st-dma-resv.c
new file mode 100644
index 0000000..bc32b3e
--- /dev/null
+++ b/drivers/dma-buf/st-dma-resv.c
@@ -0,0 +1,371 @@
+/* SPDX-License-Identifier: MIT */
+
+/*
+* Copyright © 2019 Intel Corporation
+* Copyright © 2021 Advanced Micro Devices, Inc.
+*/
+
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+#include <linux/dma-resv.h>
+
+#include "selftest.h"
+
+static struct spinlock fence_lock;
+
+static const char *fence_name(struct dma_fence *f)
+{
+ return "selftest";
+}
+
+static const struct dma_fence_ops fence_ops = {
+ .get_driver_name = fence_name,
+ .get_timeline_name = fence_name,
+};
+
+static struct dma_fence *alloc_fence(void)
+{
+ struct dma_fence *f;
+
+ f = kmalloc(sizeof(*f), GFP_KERNEL);
+ if (!f)
+ return NULL;
+
+ dma_fence_init(f, &fence_ops, &fence_lock, 0, 0);
+ return f;
+}
+
+static int sanitycheck(void *arg)
+{
+ struct dma_resv resv;
+ struct dma_fence *f;
+ int r;
+
+ f = alloc_fence();
+ if (!f)
+ return -ENOMEM;
+
+ dma_fence_signal(f);
+ dma_fence_put(f);
+
+ dma_resv_init(&resv);
+ r = dma_resv_lock(&resv, NULL);
+ if (r)
+ pr_err("Resv locking failed\n");
+ else
+ dma_resv_unlock(&resv);
+ dma_resv_fini(&resv);
+ return r;
+}
+
+static int test_signaling(void *arg, bool shared)
+{
+ struct dma_resv resv;
+ struct dma_fence *f;
+ int r;
+
+ f = alloc_fence();
+ if (!f)
+ return -ENOMEM;
+
+ dma_resv_init(&resv);
+ r = dma_resv_lock(&resv, NULL);
+ if (r) {
+ pr_err("Resv locking failed\n");
+ goto err_free;
+ }
+
+ if (shared) {
+ r = dma_resv_reserve_shared(&resv, 1);
+ if (r) {
+ pr_err("Resv shared slot allocation failed\n");
+ goto err_unlock;
+ }
+
+ dma_resv_add_shared_fence(&resv, f);
+ } else {
+ dma_resv_add_excl_fence(&resv, f);
+ }
+
+ if (dma_resv_test_signaled(&resv, shared)) {
+ pr_err("Resv unexpectedly signaled\n");
+ r = -EINVAL;
+ goto err_unlock;
+ }
+ dma_fence_signal(f);
+ if (!dma_resv_test_signaled(&resv, shared)) {
+ pr_err("Resv not reporting signaled\n");
+ r = -EINVAL;
+ goto err_unlock;
+ }
+err_unlock:
+ dma_resv_unlock(&resv);
+err_free:
+ dma_resv_fini(&resv);
+ dma_fence_put(f);
+ return r;
+}
+
+static int test_excl_signaling(void *arg)
+{
+ return test_signaling(arg, false);
+}
+
+static int test_shared_signaling(void *arg)
+{
+ return test_signaling(arg, true);
+}
+
+static int test_for_each(void *arg, bool shared)
+{
+ struct dma_resv_iter cursor;
+ struct dma_fence *f, *fence;
+ struct dma_resv resv;
+ int r;
+
+ f = alloc_fence();
+ if (!f)
+ return -ENOMEM;
+
+ dma_resv_init(&resv);
+ r = dma_resv_lock(&resv, NULL);
+ if (r) {
+ pr_err("Resv locking failed\n");
+ goto err_free;
+ }
+
+ if (shared) {
+ r = dma_resv_reserve_shared(&resv, 1);
+ if (r) {
+ pr_err("Resv shared slot allocation failed\n");
+ goto err_unlock;
+ }
+
+ dma_resv_add_shared_fence(&resv, f);
+ } else {
+ dma_resv_add_excl_fence(&resv, f);
+ }
+
+ r = -ENOENT;
+ dma_resv_for_each_fence(&cursor, &resv, shared, fence) {
+ if (!r) {
+ pr_err("More than one fence found\n");
+ r = -EINVAL;
+ goto err_unlock;
+ }
+ if (f != fence) {
+ pr_err("Unexpected fence\n");
+ r = -EINVAL;
+ goto err_unlock;
+ }
+ if (dma_resv_iter_is_exclusive(&cursor) != !shared) {
+ pr_err("Unexpected fence usage\n");
+ r = -EINVAL;
+ goto err_unlock;
+ }
+ r = 0;
+ }
+ if (r) {
+ pr_err("No fence found\n");
+ goto err_unlock;
+ }
+ dma_fence_signal(f);
+err_unlock:
+ dma_resv_unlock(&resv);
+err_free:
+ dma_resv_fini(&resv);
+ dma_fence_put(f);
+ return r;
+}
+
+static int test_excl_for_each(void *arg)
+{
+ return test_for_each(arg, false);
+}
+
+static int test_shared_for_each(void *arg)
+{
+ return test_for_each(arg, true);
+}
+
+static int test_for_each_unlocked(void *arg, bool shared)
+{
+ struct dma_resv_iter cursor;
+ struct dma_fence *f, *fence;
+ struct dma_resv resv;
+ int r;
+
+ f = alloc_fence();
+ if (!f)
+ return -ENOMEM;
+
+ dma_resv_init(&resv);
+ r = dma_resv_lock(&resv, NULL);
+ if (r) {
+ pr_err("Resv locking failed\n");
+ goto err_free;
+ }
+
+ if (shared) {
+ r = dma_resv_reserve_shared(&resv, 1);
+ if (r) {
+ pr_err("Resv shared slot allocation failed\n");
+ dma_resv_unlock(&resv);
+ goto err_free;
+ }
+
+ dma_resv_add_shared_fence(&resv, f);
+ } else {
+ dma_resv_add_excl_fence(&resv, f);
+ }
+ dma_resv_unlock(&resv);
+
+ r = -ENOENT;
+ dma_resv_iter_begin(&cursor, &resv, shared);
+ dma_resv_for_each_fence_unlocked(&cursor, fence) {
+ if (!r) {
+ pr_err("More than one fence found\n");
+ r = -EINVAL;
+ goto err_iter_end;
+ }
+ if (!dma_resv_iter_is_restarted(&cursor)) {
+ pr_err("No restart flag\n");
+ goto err_iter_end;
+ }
+ if (f != fence) {
+ pr_err("Unexpected fence\n");
+ r = -EINVAL;
+ goto err_iter_end;
+ }
+ if (dma_resv_iter_is_exclusive(&cursor) != !shared) {
+ pr_err("Unexpected fence usage\n");
+ r = -EINVAL;
+ goto err_iter_end;
+ }
+
+ /* We use r as state here */
+ if (r == -ENOENT) {
+ r = -EINVAL;
+ /* That should trigger an restart */
+ cursor.seq--;
+ } else if (r == -EINVAL) {
+ r = 0;
+ }
+ }
+ if (r)
+ pr_err("No fence found\n");
+err_iter_end:
+ dma_resv_iter_end(&cursor);
+ dma_fence_signal(f);
+err_free:
+ dma_resv_fini(&resv);
+ dma_fence_put(f);
+ return r;
+}
+
+static int test_excl_for_each_unlocked(void *arg)
+{
+ return test_for_each_unlocked(arg, false);
+}
+
+static int test_shared_for_each_unlocked(void *arg)
+{
+ return test_for_each_unlocked(arg, true);
+}
+
+static int test_get_fences(void *arg, bool shared)
+{
+ struct dma_fence *f, *excl = NULL, **fences = NULL;
+ struct dma_resv resv;
+ int r, i;
+
+ f = alloc_fence();
+ if (!f)
+ return -ENOMEM;
+
+ dma_resv_init(&resv);
+ r = dma_resv_lock(&resv, NULL);
+ if (r) {
+ pr_err("Resv locking failed\n");
+ goto err_resv;
+ }
+
+ if (shared) {
+ r = dma_resv_reserve_shared(&resv, 1);
+ if (r) {
+ pr_err("Resv shared slot allocation failed\n");
+ dma_resv_unlock(&resv);
+ goto err_resv;
+ }
+
+ dma_resv_add_shared_fence(&resv, f);
+ } else {
+ dma_resv_add_excl_fence(&resv, f);
+ }
+ dma_resv_unlock(&resv);
+
+ r = dma_resv_get_fences(&resv, &excl, &i, &fences);
+ if (r) {
+ pr_err("get_fences failed\n");
+ goto err_free;
+ }
+
+ if (shared) {
+ if (excl != NULL) {
+ pr_err("get_fences returned unexpected excl fence\n");
+ goto err_free;
+ }
+ if (i != 1 || fences[0] != f) {
+ pr_err("get_fences returned unexpected shared fence\n");
+ goto err_free;
+ }
+ } else {
+ if (excl != f) {
+ pr_err("get_fences returned unexpected excl fence\n");
+ goto err_free;
+ }
+ if (i != 0) {
+ pr_err("get_fences returned unexpected shared fence\n");
+ goto err_free;
+ }
+ }
+
+ dma_fence_signal(f);
+err_free:
+ dma_fence_put(excl);
+ while (i--)
+ dma_fence_put(fences[i]);
+ kfree(fences);
+err_resv:
+ dma_resv_fini(&resv);
+ dma_fence_put(f);
+ return r;
+}
+
+static int test_excl_get_fences(void *arg)
+{
+ return test_get_fences(arg, false);
+}
+
+static int test_shared_get_fences(void *arg)
+{
+ return test_get_fences(arg, true);
+}
+
+int dma_resv(void)
+{
+ static const struct subtest tests[] = {
+ SUBTEST(sanitycheck),
+ SUBTEST(test_excl_signaling),
+ SUBTEST(test_shared_signaling),
+ SUBTEST(test_excl_for_each),
+ SUBTEST(test_shared_for_each),
+ SUBTEST(test_excl_for_each_unlocked),
+ SUBTEST(test_shared_for_each_unlocked),
+ SUBTEST(test_excl_get_fences),
+ SUBTEST(test_shared_get_fences),
+ };
+
+ spin_lock_init(&fence_lock);
+ return subtests(tests, NULL);
+}
diff --git a/drivers/edac/Kconfig b/drivers/edac/Kconfig
index 2fc4c3f..58ab636 100644
--- a/drivers/edac/Kconfig
+++ b/drivers/edac/Kconfig
@@ -484,7 +484,7 @@
config EDAC_SYNOPSYS
tristate "Synopsys DDR Memory Controller"
- depends on ARCH_ZYNQ || ARCH_ZYNQMP
+ depends on ARCH_ZYNQ || ARCH_ZYNQMP || ARCH_INTEL_SOCFPGA
help
Support for error detection and correction on the Synopsys DDR
memory controller.
diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
index 4fce750..fba609a 100644
--- a/drivers/edac/amd64_edac.c
+++ b/drivers/edac/amd64_edac.c
@@ -988,6 +988,281 @@ static int sys_addr_to_csrow(struct mem_ctl_info *mci, u64 sys_addr)
return csrow;
}
+/* Protect the PCI config register pairs used for DF indirect access. */
+static DEFINE_MUTEX(df_indirect_mutex);
+
+/*
+ * Data Fabric Indirect Access uses FICAA/FICAD.
+ *
+ * Fabric Indirect Configuration Access Address (FICAA): Constructed based
+ * on the device's Instance Id and the PCI function and register offset of
+ * the desired register.
+ *
+ * Fabric Indirect Configuration Access Data (FICAD): There are FICAD LO
+ * and FICAD HI registers but so far we only need the LO register.
+ *
+ * Use Instance Id 0xFF to indicate a broadcast read.
+ */
+#define DF_BROADCAST 0xFF
+static int __df_indirect_read(u16 node, u8 func, u16 reg, u8 instance_id, u32 *lo)
+{
+ struct pci_dev *F4;
+ u32 ficaa;
+ int err = -ENODEV;
+
+ if (node >= amd_nb_num())
+ goto out;
+
+ F4 = node_to_amd_nb(node)->link;
+ if (!F4)
+ goto out;
+
+ ficaa = (instance_id == DF_BROADCAST) ? 0 : 1;
+ ficaa |= reg & 0x3FC;
+ ficaa |= (func & 0x7) << 11;
+ ficaa |= instance_id << 16;
+
+ mutex_lock(&df_indirect_mutex);
+
+ err = pci_write_config_dword(F4, 0x5C, ficaa);
+ if (err) {
+ pr_warn("Error writing DF Indirect FICAA, FICAA=0x%x\n", ficaa);
+ goto out_unlock;
+ }
+
+ err = pci_read_config_dword(F4, 0x98, lo);
+ if (err)
+ pr_warn("Error reading DF Indirect FICAD LO, FICAA=0x%x.\n", ficaa);
+
+out_unlock:
+ mutex_unlock(&df_indirect_mutex);
+
+out:
+ return err;
+}
+
+static int df_indirect_read_instance(u16 node, u8 func, u16 reg, u8 instance_id, u32 *lo)
+{
+ return __df_indirect_read(node, func, reg, instance_id, lo);
+}
+
+static int df_indirect_read_broadcast(u16 node, u8 func, u16 reg, u32 *lo)
+{
+ return __df_indirect_read(node, func, reg, DF_BROADCAST, lo);
+}
+
+struct addr_ctx {
+ u64 ret_addr;
+ u32 tmp;
+ u16 nid;
+ u8 inst_id;
+};
+
+static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
+{
+ u64 dram_base_addr, dram_limit_addr, dram_hole_base;
+
+ u8 die_id_shift, die_id_mask, socket_id_shift, socket_id_mask;
+ u8 intlv_num_dies, intlv_num_chan, intlv_num_sockets;
+ u8 intlv_addr_sel, intlv_addr_bit;
+ u8 num_intlv_bits, hashed_bit;
+ u8 lgcy_mmio_hole_en, base = 0;
+ u8 cs_mask, cs_id = 0;
+ bool hash_enabled = false;
+
+ struct addr_ctx ctx;
+
+ memset(&ctx, 0, sizeof(ctx));
+
+ /* Start from the normalized address */
+ ctx.ret_addr = norm_addr;
+
+ ctx.nid = nid;
+ ctx.inst_id = umc;
+
+ /* Read D18F0x1B4 (DramOffset), check if base 1 is used. */
+ if (df_indirect_read_instance(nid, 0, 0x1B4, umc, &ctx.tmp))
+ goto out_err;
+
+ /* Remove HiAddrOffset from normalized address, if enabled: */
+ if (ctx.tmp & BIT(0)) {
+ u64 hi_addr_offset = (ctx.tmp & GENMASK_ULL(31, 20)) << 8;
+
+ if (norm_addr >= hi_addr_offset) {
+ ctx.ret_addr -= hi_addr_offset;
+ base = 1;
+ }
+ }
+
+ /* Read D18F0x110 (DramBaseAddress). */
+ if (df_indirect_read_instance(nid, 0, 0x110 + (8 * base), umc, &ctx.tmp))
+ goto out_err;
+
+ /* Check if address range is valid. */
+ if (!(ctx.tmp & BIT(0))) {
+ pr_err("%s: Invalid DramBaseAddress range: 0x%x.\n",
+ __func__, ctx.tmp);
+ goto out_err;
+ }
+
+ lgcy_mmio_hole_en = ctx.tmp & BIT(1);
+ intlv_num_chan = (ctx.tmp >> 4) & 0xF;
+ intlv_addr_sel = (ctx.tmp >> 8) & 0x7;
+ dram_base_addr = (ctx.tmp & GENMASK_ULL(31, 12)) << 16;
+
+ /* {0, 1, 2, 3} map to address bits {8, 9, 10, 11} respectively */
+ if (intlv_addr_sel > 3) {
+ pr_err("%s: Invalid interleave address select %d.\n",
+ __func__, intlv_addr_sel);
+ goto out_err;
+ }
+
+ /* Read D18F0x114 (DramLimitAddress). */
+ if (df_indirect_read_instance(nid, 0, 0x114 + (8 * base), umc, &ctx.tmp))
+ goto out_err;
+
+ intlv_num_sockets = (ctx.tmp >> 8) & 0x1;
+ intlv_num_dies = (ctx.tmp >> 10) & 0x3;
+ dram_limit_addr = ((ctx.tmp & GENMASK_ULL(31, 12)) << 16) | GENMASK_ULL(27, 0);
+
+ intlv_addr_bit = intlv_addr_sel + 8;
+
+ /* Re-use intlv_num_chan by setting it equal to log2(#channels) */
+ switch (intlv_num_chan) {
+ case 0: intlv_num_chan = 0; break;
+ case 1: intlv_num_chan = 1; break;
+ case 3: intlv_num_chan = 2; break;
+ case 5: intlv_num_chan = 3; break;
+ case 7: intlv_num_chan = 4; break;
+
+ case 8: intlv_num_chan = 1;
+ hash_enabled = true;
+ break;
+ default:
+ pr_err("%s: Invalid number of interleaved channels %d.\n",
+ __func__, intlv_num_chan);
+ goto out_err;
+ }
+
+ num_intlv_bits = intlv_num_chan;
+
+ if (intlv_num_dies > 2) {
+ pr_err("%s: Invalid number of interleaved nodes/dies %d.\n",
+ __func__, intlv_num_dies);
+ goto out_err;
+ }
+
+ num_intlv_bits += intlv_num_dies;
+
+ /* Add a bit if sockets are interleaved. */
+ num_intlv_bits += intlv_num_sockets;
+
+ /* Assert num_intlv_bits <= 4 */
+ if (num_intlv_bits > 4) {
+ pr_err("%s: Invalid interleave bits %d.\n",
+ __func__, num_intlv_bits);
+ goto out_err;
+ }
+
+ if (num_intlv_bits > 0) {
+ u64 temp_addr_x, temp_addr_i, temp_addr_y;
+ u8 die_id_bit, sock_id_bit, cs_fabric_id;
+
+ /*
+ * Read FabricBlockInstanceInformation3_CS[BlockFabricID].
+ * This is the fabric id for this coherent slave. Use
+ * umc/channel# as instance id of the coherent slave
+ * for FICAA.
+ */
+ if (df_indirect_read_instance(nid, 0, 0x50, umc, &ctx.tmp))
+ goto out_err;
+
+ cs_fabric_id = (ctx.tmp >> 8) & 0xFF;
+ die_id_bit = 0;
+
+ /* If interleaved over more than 1 channel: */
+ if (intlv_num_chan) {
+ die_id_bit = intlv_num_chan;
+ cs_mask = (1 << die_id_bit) - 1;
+ cs_id = cs_fabric_id & cs_mask;
+ }
+
+ sock_id_bit = die_id_bit;
+
+ /* Read D18F1x208 (SystemFabricIdMask). */
+ if (intlv_num_dies || intlv_num_sockets)
+ if (df_indirect_read_broadcast(nid, 1, 0x208, &ctx.tmp))
+ goto out_err;
+
+ /* If interleaved over more than 1 die. */
+ if (intlv_num_dies) {
+ sock_id_bit = die_id_bit + intlv_num_dies;
+ die_id_shift = (ctx.tmp >> 24) & 0xF;
+ die_id_mask = (ctx.tmp >> 8) & 0xFF;
+
+ cs_id |= ((cs_fabric_id & die_id_mask) >> die_id_shift) << die_id_bit;
+ }
+
+ /* If interleaved over more than 1 socket. */
+ if (intlv_num_sockets) {
+ socket_id_shift = (ctx.tmp >> 28) & 0xF;
+ socket_id_mask = (ctx.tmp >> 16) & 0xFF;
+
+ cs_id |= ((cs_fabric_id & socket_id_mask) >> socket_id_shift) << sock_id_bit;
+ }
+
+ /*
+ * The pre-interleaved address consists of XXXXXXIIIYYYYY
+ * where III is the ID for this CS, and XXXXXXYYYYY are the
+ * address bits from the post-interleaved address.
+ * "num_intlv_bits" has been calculated to tell us how many "I"
+ * bits there are. "intlv_addr_bit" tells us how many "Y" bits
+ * there are (where "I" starts).
+ */
+ temp_addr_y = ctx.ret_addr & GENMASK_ULL(intlv_addr_bit - 1, 0);
+ temp_addr_i = (cs_id << intlv_addr_bit);
+ temp_addr_x = (ctx.ret_addr & GENMASK_ULL(63, intlv_addr_bit)) << num_intlv_bits;
+ ctx.ret_addr = temp_addr_x | temp_addr_i | temp_addr_y;
+ }
+
+ /* Add dram base address */
+ ctx.ret_addr += dram_base_addr;
+
+ /* If legacy MMIO hole enabled */
+ if (lgcy_mmio_hole_en) {
+ if (df_indirect_read_broadcast(nid, 0, 0x104, &ctx.tmp))
+ goto out_err;
+
+ dram_hole_base = ctx.tmp & GENMASK(31, 24);
+ if (ctx.ret_addr >= dram_hole_base)
+ ctx.ret_addr += (BIT_ULL(32) - dram_hole_base);
+ }
+
+ if (hash_enabled) {
+ /* Save some parentheses and grab ls-bit at the end. */
+ hashed_bit = (ctx.ret_addr >> 12) ^
+ (ctx.ret_addr >> 18) ^
+ (ctx.ret_addr >> 21) ^
+ (ctx.ret_addr >> 30) ^
+ cs_id;
+
+ hashed_bit &= BIT(0);
+
+ if (hashed_bit != ((ctx.ret_addr >> intlv_addr_bit) & BIT(0)))
+ ctx.ret_addr ^= BIT(intlv_addr_bit);
+ }
+
+ /* Is calculated system address is above DRAM limit address? */
+ if (ctx.ret_addr > dram_limit_addr)
+ goto out_err;
+
+ *sys_addr = ctx.ret_addr;
+ return 0;
+
+out_err:
+ return -EINVAL;
+}
+
static int get_channel_from_ecc_syndrome(struct mem_ctl_info *, u16);
/*
@@ -2650,6 +2925,26 @@ static struct amd64_family_type family_types[] = {
.dbam_to_cs = f17_addr_mask_to_cs_size,
}
},
+ [F19_M10H_CPUS] = {
+ .ctl_name = "F19h_M10h",
+ .f0_id = PCI_DEVICE_ID_AMD_19H_M10H_DF_F0,
+ .f6_id = PCI_DEVICE_ID_AMD_19H_M10H_DF_F6,
+ .max_mcs = 12,
+ .ops = {
+ .early_channel_count = f17_early_channel_count,
+ .dbam_to_cs = f17_addr_mask_to_cs_size,
+ }
+ },
+ [F19_M50H_CPUS] = {
+ .ctl_name = "F19h_M50h",
+ .f0_id = PCI_DEVICE_ID_AMD_19H_M50H_DF_F0,
+ .f6_id = PCI_DEVICE_ID_AMD_19H_M50H_DF_F6,
+ .max_mcs = 2,
+ .ops = {
+ .early_channel_count = f17_early_channel_count,
+ .dbam_to_cs = f17_addr_mask_to_cs_size,
+ }
+ },
};
/*
@@ -3687,11 +3982,25 @@ static struct amd64_family_type *per_family_init(struct amd64_pvt *pvt)
break;
case 0x19:
- if (pvt->model >= 0x20 && pvt->model <= 0x2f) {
+ if (pvt->model >= 0x10 && pvt->model <= 0x1f) {
+ fam_type = &family_types[F19_M10H_CPUS];
+ pvt->ops = &family_types[F19_M10H_CPUS].ops;
+ break;
+ } else if (pvt->model >= 0x20 && pvt->model <= 0x2f) {
fam_type = &family_types[F17_M70H_CPUS];
pvt->ops = &family_types[F17_M70H_CPUS].ops;
fam_type->ctl_name = "F19h_M20h";
break;
+ } else if (pvt->model >= 0x50 && pvt->model <= 0x5f) {
+ fam_type = &family_types[F19_M50H_CPUS];
+ pvt->ops = &family_types[F19_M50H_CPUS].ops;
+ fam_type->ctl_name = "F19h_M50h";
+ break;
+ } else if (pvt->model >= 0xa0 && pvt->model <= 0xaf) {
+ fam_type = &family_types[F19_M10H_CPUS];
+ pvt->ops = &family_types[F19_M10H_CPUS].ops;
+ fam_type->ctl_name = "F19h_MA0h";
+ break;
}
fam_type = &family_types[F19_CPUS];
pvt->ops = &family_types[F19_CPUS].ops;
diff --git a/drivers/edac/amd64_edac.h b/drivers/edac/amd64_edac.h
index 85aa820..352bda9 100644
--- a/drivers/edac/amd64_edac.h
+++ b/drivers/edac/amd64_edac.h
@@ -96,7 +96,7 @@
/* Hardware limit on ChipSelect rows per MC and processors per system */
#define NUM_CHIPSELECTS 8
#define DRAM_RANGES 8
-#define NUM_CONTROLLERS 8
+#define NUM_CONTROLLERS 12
#define ON true
#define OFF false
@@ -126,6 +126,10 @@
#define PCI_DEVICE_ID_AMD_17H_M70H_DF_F6 0x1446
#define PCI_DEVICE_ID_AMD_19H_DF_F0 0x1650
#define PCI_DEVICE_ID_AMD_19H_DF_F6 0x1656
+#define PCI_DEVICE_ID_AMD_19H_M10H_DF_F0 0x14ad
+#define PCI_DEVICE_ID_AMD_19H_M10H_DF_F6 0x14b3
+#define PCI_DEVICE_ID_AMD_19H_M50H_DF_F0 0x166a
+#define PCI_DEVICE_ID_AMD_19H_M50H_DF_F6 0x1670
/*
* Function 1 - Address Map
@@ -298,6 +302,8 @@ enum amd_families {
F17_M60H_CPUS,
F17_M70H_CPUS,
F19_CPUS,
+ F19_M10H_CPUS,
+ F19_M50H_CPUS,
NUM_FAMILIES,
};
diff --git a/drivers/edac/edac_mc.c b/drivers/edac/edac_mc.c
index 9f82ca2..9d9aabd 100644
--- a/drivers/edac/edac_mc.c
+++ b/drivers/edac/edac_mc.c
@@ -162,6 +162,8 @@ const char * const edac_mem_types[] = {
[MEM_LPDDR4] = "Low-Power-DDR4-RAM",
[MEM_LRDDR4] = "Load-Reduced-DDR4-RAM",
[MEM_DDR5] = "Unbuffered-DDR5",
+ [MEM_RDDR5] = "Registered-DDR5",
+ [MEM_LRDDR5] = "Load-Reduced-DDR5-RAM",
[MEM_NVDIMM] = "Non-volatile-RAM",
[MEM_WIO2] = "Wide-IO-2",
[MEM_HBM2] = "High-bandwidth-memory-Gen2",
diff --git a/drivers/edac/i10nm_base.c b/drivers/edac/i10nm_base.c
index 83345bf..6cf50ee 100644
--- a/drivers/edac/i10nm_base.c
+++ b/drivers/edac/i10nm_base.c
@@ -358,6 +358,9 @@ static int i10nm_get_hbm_munits(void)
mbase = ioremap(base + off, I10NM_HBM_IMC_MMIO_SIZE);
if (!mbase) {
+ pci_dev_put(d->imc[lmc].mdev);
+ d->imc[lmc].mdev = NULL;
+
i10nm_printk(KERN_ERR, "Failed to ioremap for hbm mc 0x%llx\n",
base + off);
return -ENOMEM;
@@ -368,6 +371,12 @@ static int i10nm_get_hbm_munits(void)
mcmtr = I10NM_GET_MCMTR(&d->imc[lmc], 0);
if (!I10NM_IS_HBM_IMC(mcmtr)) {
+ iounmap(d->imc[lmc].mbase);
+ d->imc[lmc].mbase = NULL;
+ d->imc[lmc].hbm_mc = false;
+ pci_dev_put(d->imc[lmc].mdev);
+ d->imc[lmc].mdev = NULL;
+
i10nm_printk(KERN_ERR, "This isn't an hbm mc!\n");
return -ENODEV;
}
diff --git a/drivers/edac/mce_amd.c b/drivers/edac/mce_amd.c
index 67dbf4c..cc5c63f 100644
--- a/drivers/edac/mce_amd.c
+++ b/drivers/edac/mce_amd.c
@@ -399,6 +399,63 @@ static const char * const smca_mp5_mce_desc[] = {
"Instruction Tag Cache Bank B ECC or parity error",
};
+static const char * const smca_mpdma_mce_desc[] = {
+ "Main SRAM [31:0] bank ECC or parity error",
+ "Main SRAM [63:32] bank ECC or parity error",
+ "Main SRAM [95:64] bank ECC or parity error",
+ "Main SRAM [127:96] bank ECC or parity error",
+ "Data Cache Bank A ECC or parity error",
+ "Data Cache Bank B ECC or parity error",
+ "Data Tag Cache Bank A ECC or parity error",
+ "Data Tag Cache Bank B ECC or parity error",
+ "Instruction Cache Bank A ECC or parity error",
+ "Instruction Cache Bank B ECC or parity error",
+ "Instruction Tag Cache Bank A ECC or parity error",
+ "Instruction Tag Cache Bank B ECC or parity error",
+ "Data Cache Bank A ECC or parity error",
+ "Data Cache Bank B ECC or parity error",
+ "Data Tag Cache Bank A ECC or parity error",
+ "Data Tag Cache Bank B ECC or parity error",
+ "Instruction Cache Bank A ECC or parity error",
+ "Instruction Cache Bank B ECC or parity error",
+ "Instruction Tag Cache Bank A ECC or parity error",
+ "Instruction Tag Cache Bank B ECC or parity error",
+ "Data Cache Bank A ECC or parity error",
+ "Data Cache Bank B ECC or parity error",
+ "Data Tag Cache Bank A ECC or parity error",
+ "Data Tag Cache Bank B ECC or parity error",
+ "Instruction Cache Bank A ECC or parity error",
+ "Instruction Cache Bank B ECC or parity error",
+ "Instruction Tag Cache Bank A ECC or parity error",
+ "Instruction Tag Cache Bank B ECC or parity error",
+ "System Hub Read Buffer ECC or parity error",
+ "MPDMA TVF DVSEC Memory ECC or parity error",
+ "MPDMA TVF MMIO Mailbox0 ECC or parity error",
+ "MPDMA TVF MMIO Mailbox1 ECC or parity error",
+ "MPDMA TVF Doorbell Memory ECC or parity error",
+ "MPDMA TVF SDP Slave Memory 0 ECC or parity error",
+ "MPDMA TVF SDP Slave Memory 1 ECC or parity error",
+ "MPDMA TVF SDP Slave Memory 2 ECC or parity error",
+ "MPDMA TVF SDP Master Memory 0 ECC or parity error",
+ "MPDMA TVF SDP Master Memory 1 ECC or parity error",
+ "MPDMA TVF SDP Master Memory 2 ECC or parity error",
+ "MPDMA TVF SDP Master Memory 3 ECC or parity error",
+ "MPDMA TVF SDP Master Memory 4 ECC or parity error",
+ "MPDMA TVF SDP Master Memory 5 ECC or parity error",
+ "MPDMA TVF SDP Master Memory 6 ECC or parity error",
+ "MPDMA PTE Command FIFO ECC or parity error",
+ "MPDMA PTE Hub Data FIFO ECC or parity error",
+ "MPDMA PTE Internal Data FIFO ECC or parity error",
+ "MPDMA PTE Command Memory DMA ECC or parity error",
+ "MPDMA PTE Command Memory Internal ECC or parity error",
+ "MPDMA PTE DMA Completion FIFO ECC or parity error",
+ "MPDMA PTE Tablewalk Completion FIFO ECC or parity error",
+ "MPDMA PTE Descriptor Completion FIFO ECC or parity error",
+ "MPDMA PTE ReadOnly Completion FIFO ECC or parity error",
+ "MPDMA PTE DirectWrite Completion FIFO ECC or parity error",
+ "SDP Watchdog Timer expired",
+};
+
static const char * const smca_nbio_mce_desc[] = {
"ECC or Parity error",
"PCIE error",
@@ -448,7 +505,7 @@ static const char * const smca_xgmipcs_mce_desc[] = {
"Rx Replay Timeout Error",
"LinkSub Tx Timeout Error",
"LinkSub Rx Timeout Error",
- "Rx CMD Pocket Error",
+ "Rx CMD Packet Error",
};
static const char * const smca_xgmiphy_mce_desc[] = {
@@ -458,11 +515,66 @@ static const char * const smca_xgmiphy_mce_desc[] = {
"PHY APB error",
};
-static const char * const smca_waflphy_mce_desc[] = {
- "RAM ECC Error",
- "ARC instruction buffer parity error",
- "ARC data buffer parity error",
- "PHY APB error",
+static const char * const smca_nbif_mce_desc[] = {
+ "Timeout error from GMI",
+ "SRAM ECC error",
+ "NTB Error Event",
+ "SDP Parity error",
+};
+
+static const char * const smca_sata_mce_desc[] = {
+ "Parity error for port 0",
+ "Parity error for port 1",
+ "Parity error for port 2",
+ "Parity error for port 3",
+ "Parity error for port 4",
+ "Parity error for port 5",
+ "Parity error for port 6",
+ "Parity error for port 7",
+};
+
+static const char * const smca_usb_mce_desc[] = {
+ "Parity error or ECC error for S0 RAM0",
+ "Parity error or ECC error for S0 RAM1",
+ "Parity error or ECC error for S0 RAM2",
+ "Parity error for PHY RAM0",
+ "Parity error for PHY RAM1",
+ "AXI Slave Response error",
+};
+
+static const char * const smca_gmipcs_mce_desc[] = {
+ "Data Loss Error",
+ "Training Error",
+ "Replay Parity Error",
+ "Rx Fifo Underflow Error",
+ "Rx Fifo Overflow Error",
+ "CRC Error",
+ "BER Exceeded Error",
+ "Tx Fifo Underflow Error",
+ "Replay Buffer Parity Error",
+ "Tx Overflow Error",
+ "Replay Fifo Overflow Error",
+ "Replay Fifo Underflow Error",
+ "Elastic Fifo Overflow Error",
+ "Deskew Error",
+ "Offline Error",
+ "Data Startup Limit Error",
+ "FC Init Timeout Error",
+ "Recovery Timeout Error",
+ "Ready Serial Timeout Error",
+ "Ready Serial Attempt Error",
+ "Recovery Attempt Error",
+ "Recovery Relock Attempt Error",
+ "Deskew Abort Error",
+ "Rx Buffer Error",
+ "Rx LFDS Fifo Overflow Error",
+ "Rx LFDS Fifo Underflow Error",
+ "LinkSub Tx Timeout Error",
+ "LinkSub Rx Timeout Error",
+ "Rx CMD Packet Error",
+ "LFDS Training Timeout Error",
+ "LFDS FC Init Timeout Error",
+ "Data Loss Error",
};
struct smca_mce_desc {
@@ -490,12 +602,21 @@ static struct smca_mce_desc smca_mce_descs[] = {
[SMCA_SMU] = { smca_smu_mce_desc, ARRAY_SIZE(smca_smu_mce_desc) },
[SMCA_SMU_V2] = { smca_smu2_mce_desc, ARRAY_SIZE(smca_smu2_mce_desc) },
[SMCA_MP5] = { smca_mp5_mce_desc, ARRAY_SIZE(smca_mp5_mce_desc) },
+ [SMCA_MPDMA] = { smca_mpdma_mce_desc, ARRAY_SIZE(smca_mpdma_mce_desc) },
[SMCA_NBIO] = { smca_nbio_mce_desc, ARRAY_SIZE(smca_nbio_mce_desc) },
[SMCA_PCIE] = { smca_pcie_mce_desc, ARRAY_SIZE(smca_pcie_mce_desc) },
[SMCA_PCIE_V2] = { smca_pcie2_mce_desc, ARRAY_SIZE(smca_pcie2_mce_desc) },
[SMCA_XGMI_PCS] = { smca_xgmipcs_mce_desc, ARRAY_SIZE(smca_xgmipcs_mce_desc) },
+ /* NBIF and SHUB have the same error descriptions, for now. */
+ [SMCA_NBIF] = { smca_nbif_mce_desc, ARRAY_SIZE(smca_nbif_mce_desc) },
+ [SMCA_SHUB] = { smca_nbif_mce_desc, ARRAY_SIZE(smca_nbif_mce_desc) },
+ [SMCA_SATA] = { smca_sata_mce_desc, ARRAY_SIZE(smca_sata_mce_desc) },
+ [SMCA_USB] = { smca_usb_mce_desc, ARRAY_SIZE(smca_usb_mce_desc) },
+ [SMCA_GMI_PCS] = { smca_gmipcs_mce_desc, ARRAY_SIZE(smca_gmipcs_mce_desc) },
+ /* All the PHY bank types have the same error descriptions, for now. */
[SMCA_XGMI_PHY] = { smca_xgmiphy_mce_desc, ARRAY_SIZE(smca_xgmiphy_mce_desc) },
- [SMCA_WAFL_PHY] = { smca_waflphy_mce_desc, ARRAY_SIZE(smca_waflphy_mce_desc) },
+ [SMCA_WAFL_PHY] = { smca_xgmiphy_mce_desc, ARRAY_SIZE(smca_xgmiphy_mce_desc) },
+ [SMCA_GMI_PHY] = { smca_xgmiphy_mce_desc, ARRAY_SIZE(smca_xgmiphy_mce_desc) },
};
static bool f12h_mc0_mce(u16 ec, u8 xec)
@@ -1045,20 +1166,13 @@ static void decode_mc6_mce(struct mce *m)
/* Decode errors according to Scalable MCA specification */
static void decode_smca_error(struct mce *m)
{
- struct smca_hwid *hwid;
- enum smca_bank_types bank_type;
+ enum smca_bank_types bank_type = smca_get_bank_type(m->extcpu, m->bank);
const char *ip_name;
u8 xec = XEC(m->status, xec_mask);
- if (m->bank >= ARRAY_SIZE(smca_banks))
+ if (bank_type >= N_SMCA_BANK_TYPES)
return;
- hwid = smca_banks[m->bank].hwid;
- if (!hwid)
- return;
-
- bank_type = hwid->bank_type;
-
if (bank_type == SMCA_RESERVED) {
pr_emerg(HW_ERR "Bank %d is reserved.\n", m->bank);
return;
diff --git a/drivers/edac/sb_edac.c b/drivers/edac/sb_edac.c
index 1522d4a..9678ab9 100644
--- a/drivers/edac/sb_edac.c
+++ b/drivers/edac/sb_edac.c
@@ -3439,7 +3439,7 @@ MODULE_DEVICE_TABLE(x86cpu, sbridge_cpuids);
static int sbridge_probe(const struct x86_cpu_id *id)
{
- int rc = -ENODEV;
+ int rc;
u8 mc, num_mc = 0;
struct sbridge_dev *sbridge_dev;
struct pci_id_table *ptable = (struct pci_id_table *)id->driver_data;
diff --git a/drivers/edac/sifive_edac.c b/drivers/edac/sifive_edac.c
index 3a3dcb1..ee800ae 100644
--- a/drivers/edac/sifive_edac.c
+++ b/drivers/edac/sifive_edac.c
@@ -19,7 +19,7 @@ struct sifive_edac_priv {
struct edac_device_ctl_info *dci;
};
-/**
+/*
* EDAC error callback
*
* @event: non-zero if unrecoverable.
diff --git a/drivers/edac/synopsys_edac.c b/drivers/edac/synopsys_edac.c
index 7d08627..f05ff02 100644
--- a/drivers/edac/synopsys_edac.c
+++ b/drivers/edac/synopsys_edac.c
@@ -101,6 +101,7 @@
/* DDR ECC Quirks */
#define DDR_ECC_INTR_SUPPORT BIT(0)
#define DDR_ECC_DATA_POISON_SUPPORT BIT(1)
+#define DDR_ECC_INTR_SELF_CLEAR BIT(2)
/* ZynqMP Enhanced DDR memory controller registers that are relevant to ECC */
/* ECC Configuration Registers */
@@ -171,6 +172,10 @@
#define DDR_QOS_IRQ_EN_OFST 0x20208
#define DDR_QOS_IRQ_DB_OFST 0x2020C
+/* DDR QOS Interrupt register definitions */
+#define DDR_UE_MASK BIT(9)
+#define DDR_CE_MASK BIT(8)
+
/* ECC Corrected Error Register Mask and Shifts*/
#define ECC_CEADDR0_RW_MASK 0x3FFFF
#define ECC_CEADDR0_RNK_MASK BIT(24)
@@ -533,10 +538,16 @@ static irqreturn_t intr_handler(int irq, void *dev_id)
priv = mci->pvt_info;
p_data = priv->p_data;
- regval = readl(priv->baseaddr + DDR_QOS_IRQ_STAT_OFST);
- regval &= (DDR_QOSCE_MASK | DDR_QOSUE_MASK);
- if (!(regval & ECC_CE_UE_INTR_MASK))
- return IRQ_NONE;
+ /*
+ * v3.0 of the controller has the ce/ue bits cleared automatically,
+ * so this condition does not apply.
+ */
+ if (!(priv->p_data->quirks & DDR_ECC_INTR_SELF_CLEAR)) {
+ regval = readl(priv->baseaddr + DDR_QOS_IRQ_STAT_OFST);
+ regval &= (DDR_QOSCE_MASK | DDR_QOSUE_MASK);
+ if (!(regval & ECC_CE_UE_INTR_MASK))
+ return IRQ_NONE;
+ }
status = p_data->get_error_info(priv);
if (status)
@@ -548,7 +559,9 @@ static irqreturn_t intr_handler(int irq, void *dev_id)
edac_dbg(3, "Total error count CE %d UE %d\n",
priv->ce_cnt, priv->ue_cnt);
- writel(regval, priv->baseaddr + DDR_QOS_IRQ_STAT_OFST);
+ /* v3.0 of the controller does not have this register */
+ if (!(priv->p_data->quirks & DDR_ECC_INTR_SELF_CLEAR))
+ writel(regval, priv->baseaddr + DDR_QOS_IRQ_STAT_OFST);
return IRQ_HANDLED;
}
@@ -834,8 +847,13 @@ static void mc_init(struct mem_ctl_info *mci, struct platform_device *pdev)
static void enable_intr(struct synps_edac_priv *priv)
{
/* Enable UE/CE Interrupts */
- writel(DDR_QOSUE_MASK | DDR_QOSCE_MASK,
- priv->baseaddr + DDR_QOS_IRQ_EN_OFST);
+ if (priv->p_data->quirks & DDR_ECC_INTR_SELF_CLEAR)
+ writel(DDR_UE_MASK | DDR_CE_MASK,
+ priv->baseaddr + ECC_CLR_OFST);
+ else
+ writel(DDR_QOSUE_MASK | DDR_QOSCE_MASK,
+ priv->baseaddr + DDR_QOS_IRQ_EN_OFST);
+
}
static void disable_intr(struct synps_edac_priv *priv)
@@ -890,6 +908,19 @@ static const struct synps_platform_data zynqmp_edac_def = {
),
};
+static const struct synps_platform_data synopsys_edac_def = {
+ .get_error_info = zynqmp_get_error_info,
+ .get_mtype = zynqmp_get_mtype,
+ .get_dtype = zynqmp_get_dtype,
+ .get_ecc_state = zynqmp_get_ecc_state,
+ .quirks = (DDR_ECC_INTR_SUPPORT | DDR_ECC_INTR_SELF_CLEAR
+#ifdef CONFIG_EDAC_DEBUG
+ | DDR_ECC_DATA_POISON_SUPPORT
+#endif
+ ),
+};
+
+
static const struct of_device_id synps_edac_match[] = {
{
.compatible = "xlnx,zynq-ddrc-a05",
@@ -900,6 +931,10 @@ static const struct of_device_id synps_edac_match[] = {
.data = (void *)&zynqmp_edac_def
},
{
+ .compatible = "snps,ddrc-3.80a",
+ .data = (void *)&synopsys_edac_def
+ },
+ {
/* end of table */
}
};
@@ -1352,8 +1387,7 @@ static int mc_probe(struct platform_device *pdev)
}
}
- if (of_device_is_compatible(pdev->dev.of_node,
- "xlnx,zynqmp-ddrc-2.40a"))
+ if (priv->p_data->quirks & DDR_ECC_INTR_SUPPORT)
setup_address_map(priv);
#endif
diff --git a/drivers/firmware/ti_sci.c b/drivers/firmware/ti_sci.c
index 235c7e7..5ae2040 100644
--- a/drivers/firmware/ti_sci.c
+++ b/drivers/firmware/ti_sci.c
@@ -1759,7 +1759,7 @@ static int ti_sci_get_resource_range(const struct ti_sci_handle *handle,
desc->num = resp->range_num;
desc->start_sec = resp->range_start_sec;
desc->num_sec = resp->range_num_sec;
- };
+ }
fail:
ti_sci_put_one_xfer(&info->minfo, xfer);
diff --git a/drivers/firmware/xilinx/zynqmp.c b/drivers/firmware/xilinx/zynqmp.c
index 3dd45a7..0dd1178 100644
--- a/drivers/firmware/xilinx/zynqmp.c
+++ b/drivers/firmware/xilinx/zynqmp.c
@@ -1434,7 +1434,10 @@ static int zynqmp_firmware_probe(struct platform_device *pdev)
return ret;
/* Check PM API version number */
- zynqmp_pm_get_api_version(&pm_api_version);
+ ret = zynqmp_pm_get_api_version(&pm_api_version);
+ if (ret)
+ return ret;
+
if (pm_api_version < ZYNQMP_PM_VERSION) {
panic("%s Platform Management API version error. Expected: v%d.%d - Found: v%d.%d\n",
__func__,
diff --git a/drivers/gpio/gpio-aspeed-sgpio.c b/drivers/gpio/gpio-aspeed-sgpio.c
index 3d6ef37..b3a9b84 100644
--- a/drivers/gpio/gpio-aspeed-sgpio.c
+++ b/drivers/gpio/gpio-aspeed-sgpio.c
@@ -395,7 +395,7 @@ static void aspeed_sgpio_irq_handler(struct irq_desc *desc)
reg = ioread32(bank_reg(data, bank, reg_irq_status));
for_each_set_bit(p, ®, 32)
- generic_handle_domain_irq(gc->irq.domain, i * 32 + p * 2);
+ generic_handle_domain_irq(gc->irq.domain, (i * 32 + p) * 2);
}
chained_irq_exit(ic, desc);
diff --git a/drivers/gpu/drm/Kconfig b/drivers/gpu/drm/Kconfig
index 0039df2..b1f22e4 100644
--- a/drivers/gpu/drm/Kconfig
+++ b/drivers/gpu/drm/Kconfig
@@ -8,6 +8,7 @@
menuconfig DRM
tristate "Direct Rendering Manager (XFree86 4.1.0 and higher DRI support)"
depends on (AGP || AGP=n) && !EMULATED_CMPXCHG && HAS_DMA
+ select DRM_NOMODESET
select DRM_PANEL_ORIENTATION_QUIRKS
select HDMI
select FB_CMDLINE
@@ -211,20 +212,13 @@
Helpers for ttm-based gem objects
config DRM_GEM_CMA_HELPER
- bool
+ tristate
depends on DRM
help
Choose this if you need the GEM CMA helper functions
-config DRM_KMS_CMA_HELPER
- bool
- depends on DRM
- select DRM_GEM_CMA_HELPER
- help
- Choose this if you need the KMS CMA helper functions
-
config DRM_GEM_SHMEM_HELPER
- bool
+ tristate
depends on DRM && MMU
help
Choose this if you need the GEM shmem helper functions
@@ -394,6 +388,8 @@
source "drivers/gpu/drm/gud/Kconfig"
+source "drivers/gpu/drm/sprd/Kconfig"
+
config DRM_HYPERV
tristate "DRM Support for Hyper-V synthetic video device"
depends on DRM && PCI && MMU && HYPERV
@@ -492,6 +488,15 @@
config DRM_PANEL_ORIENTATION_QUIRKS
tristate
+# Separate option because nomodeset parameter is global and expected built-in
+config DRM_NOMODESET
+ bool
+ default n
+
config DRM_LIB_RANDOM
bool
default n
+
+config DRM_PRIVACY_SCREEN
+ bool
+ default n
diff --git a/drivers/gpu/drm/Makefile b/drivers/gpu/drm/Makefile
index 0dff40b..301a44d 100644
--- a/drivers/gpu/drm/Makefile
+++ b/drivers/gpu/drm/Makefile
@@ -4,37 +4,44 @@
# Direct Rendering Infrastructure (DRI) in XFree86 4.1.0 and higher.
drm-y := drm_aperture.o drm_auth.o drm_cache.o \
- drm_file.o drm_gem.o drm_ioctl.o drm_irq.o \
+ drm_file.o drm_gem.o drm_ioctl.o \
drm_drv.o \
- drm_sysfs.o drm_hashtab.o drm_mm.o \
+ drm_sysfs.o drm_mm.o \
drm_crtc.o drm_fourcc.o drm_modes.o drm_edid.o drm_displayid.o \
- drm_encoder_slave.o \
drm_trace_points.o drm_prime.o \
- drm_rect.o drm_vma_manager.o drm_flip_work.o \
+ drm_vma_manager.o \
drm_modeset_lock.o drm_atomic.o drm_bridge.o \
drm_framebuffer.o drm_connector.o drm_blend.o \
drm_encoder.o drm_mode_object.o drm_property.o \
drm_plane.o drm_color_mgmt.o drm_print.o \
drm_dumb_buffers.o drm_mode_config.o drm_vblank.o \
drm_syncobj.o drm_lease.o drm_writeback.o drm_client.o \
- drm_client_modeset.o drm_atomic_uapi.o drm_hdcp.o \
+ drm_client_modeset.o drm_atomic_uapi.o \
drm_managed.o drm_vblank_work.o
drm-$(CONFIG_DRM_LEGACY) += drm_agpsupport.o drm_bufs.o drm_context.o drm_dma.o \
- drm_legacy_misc.o drm_lock.o drm_memory.o drm_scatter.o \
- drm_vm.o
+ drm_hashtab.o drm_irq.o drm_legacy_misc.o drm_lock.o \
+ drm_memory.o drm_scatter.o drm_vm.o
drm-$(CONFIG_DRM_LIB_RANDOM) += lib/drm_random.o
drm-$(CONFIG_COMPAT) += drm_ioc32.o
-drm-$(CONFIG_DRM_GEM_CMA_HELPER) += drm_gem_cma_helper.o
-drm-$(CONFIG_DRM_GEM_SHMEM_HELPER) += drm_gem_shmem_helper.o
drm-$(CONFIG_DRM_PANEL) += drm_panel.o
drm-$(CONFIG_OF) += drm_of.o
drm-$(CONFIG_PCI) += drm_pci.o
drm-$(CONFIG_DEBUG_FS) += drm_debugfs.o drm_debugfs_crc.o
drm-$(CONFIG_DRM_LOAD_EDID_FIRMWARE) += drm_edid_load.o
+drm-$(CONFIG_DRM_PRIVACY_SCREEN) += drm_privacy_screen.o drm_privacy_screen_x86.o
obj-$(CONFIG_DRM_DP_AUX_BUS) += drm_dp_aux_bus.o
+obj-$(CONFIG_DRM_NOMODESET) += drm_nomodeset.o
+
+drm_cma_helper-y := drm_gem_cma_helper.o
+drm_cma_helper-$(CONFIG_DRM_KMS_HELPER) += drm_fb_cma_helper.o
+obj-$(CONFIG_DRM_GEM_CMA_HELPER) += drm_cma_helper.o
+
+drm_shmem_helper-y := drm_gem_shmem_helper.o
+obj-$(CONFIG_DRM_GEM_SHMEM_HELPER) += drm_shmem_helper.o
+
drm_vram_helper-y := drm_gem_vram_helper.o
obj-$(CONFIG_DRM_VRAM_HELPER) += drm_vram_helper.o
@@ -42,18 +49,18 @@
obj-$(CONFIG_DRM_TTM_HELPER) += drm_ttm_helper.o
drm_kms_helper-y := drm_bridge_connector.o drm_crtc_helper.o drm_dp_helper.o \
- drm_dsc.o drm_probe_helper.o \
+ drm_dsc.o drm_encoder_slave.o drm_flip_work.o drm_hdcp.o \
+ drm_probe_helper.o \
drm_plane_helper.o drm_dp_mst_topology.o drm_atomic_helper.o \
drm_kms_helper_common.o drm_dp_dual_mode_helper.o \
drm_simple_kms_helper.o drm_modeset_helper.o \
drm_scdc_helper.o drm_gem_atomic_helper.o \
drm_gem_framebuffer_helper.o \
drm_atomic_state_helper.o drm_damage_helper.o \
- drm_format_helper.o drm_self_refresh_helper.o
+ drm_format_helper.o drm_self_refresh_helper.o drm_rect.o
drm_kms_helper-$(CONFIG_DRM_PANEL_BRIDGE) += bridge/panel.o
drm_kms_helper-$(CONFIG_DRM_FBDEV_EMULATION) += drm_fb_helper.o
-drm_kms_helper-$(CONFIG_DRM_KMS_CMA_HELPER) += drm_fb_cma_helper.o
drm_kms_helper-$(CONFIG_DRM_DP_AUX_CHARDEV) += drm_dp_aux_dev.o
drm_kms_helper-$(CONFIG_DRM_DP_CEC) += drm_dp_cec.o
@@ -127,3 +134,4 @@
obj-y += xlnx/
obj-y += gud/
obj-$(CONFIG_DRM_HYPERV) += hyperv/
+obj-$(CONFIG_DRM_SPRD) += sprd/
diff --git a/drivers/gpu/drm/amd/amdgpu/Makefile b/drivers/gpu/drm/amd/amdgpu/Makefile
index 6537265..7fedbb72 100644
--- a/drivers/gpu/drm/amd/amdgpu/Makefile
+++ b/drivers/gpu/drm/amd/amdgpu/Makefile
@@ -45,7 +45,7 @@
amdgpu_atombios.o atombios_crtc.o amdgpu_connectors.o \
atom.o amdgpu_fence.o amdgpu_ttm.o amdgpu_object.o amdgpu_gart.o \
amdgpu_encoders.o amdgpu_display.o amdgpu_i2c.o \
- amdgpu_fb.o amdgpu_gem.o amdgpu_ring.o \
+ amdgpu_gem.o amdgpu_ring.o \
amdgpu_cs.o amdgpu_bios.o amdgpu_benchmark.o amdgpu_test.o \
atombios_dp.o amdgpu_afmt.o amdgpu_trace_points.o \
atombios_encoders.o amdgpu_sa.o atombios_i2c.o \
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
index b85b67a..d8b854f 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
@@ -458,7 +458,6 @@ struct amdgpu_flip_work {
uint64_t base;
struct drm_pending_vblank_event *event;
struct amdgpu_bo *old_abo;
- struct dma_fence *excl;
unsigned shared_count;
struct dma_fence **shared;
struct dma_fence_cb cb;
@@ -813,6 +812,7 @@ struct amd_powerplay {
#define AMDGPU_RESET_MAGIC_NUM 64
#define AMDGPU_MAX_DF_PERFMONS 4
+#define AMDGPU_PRODUCT_NAME_LEN 64
struct amdgpu_device {
struct device *dev;
struct pci_dev *pdev;
@@ -1077,13 +1077,14 @@ struct amdgpu_device {
bool runpm;
bool in_runpm;
bool has_pr3;
+ bool is_fw_fb;
bool pm_sysfs_en;
bool ucode_sysfs_en;
/* Chip product information */
char product_number[16];
- char product_name[32];
+ char product_name[AMDGPU_PRODUCT_NAME_LEN];
char serial[20];
atomic_t throttling_logging_enabled;
@@ -1096,7 +1097,9 @@ struct amdgpu_device {
pci_channel_state_t pci_channel_state;
struct amdgpu_reset_control *reset_cntl;
- uint32_t ip_versions[HW_ID_MAX][HWIP_MAX_INSTANCE];
+ uint32_t ip_versions[MAX_HWIP][HWIP_MAX_INSTANCE];
+
+ bool ram_is_direct_mapped;
};
static inline struct amdgpu_device *drm_to_adev(struct drm_device *ddev)
@@ -1317,6 +1320,8 @@ void amdgpu_device_flush_hdp(struct amdgpu_device *adev,
void amdgpu_device_invalidate_hdp(struct amdgpu_device *adev,
struct amdgpu_ring *ring);
+void amdgpu_device_halt(struct amdgpu_device *adev);
+
/* atpx handler */
#if defined(CONFIG_VGA_SWITCHEROO)
void amdgpu_register_atpx_handler(void);
@@ -1360,8 +1365,6 @@ int amdgpu_device_resume(struct drm_device *dev, bool fbcon);
u32 amdgpu_get_vblank_counter_kms(struct drm_crtc *crtc);
int amdgpu_enable_vblank_kms(struct drm_crtc *crtc);
void amdgpu_disable_vblank_kms(struct drm_crtc *crtc);
-long amdgpu_kms_compat_ioctl(struct file *filp, unsigned int cmd,
- unsigned long arg);
int amdgpu_info_ioctl(struct drm_device *dev, void *data,
struct drm_file *filp);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c
index 7077f21..776a947 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c
@@ -72,7 +72,7 @@ void amdgpu_amdkfd_device_probe(struct amdgpu_device *adev)
if (!kfd_initialized)
return;
- adev->kfd.dev = kgd2kfd_probe((struct kgd_dev *)adev, vf);
+ adev->kfd.dev = kgd2kfd_probe(adev, vf);
if (adev->kfd.dev)
amdgpu_amdkfd_total_mem_size += adev->gmc.real_vram_size;
@@ -233,19 +233,16 @@ int amdgpu_amdkfd_post_reset(struct amdgpu_device *adev)
return r;
}
-void amdgpu_amdkfd_gpu_reset(struct kgd_dev *kgd)
+void amdgpu_amdkfd_gpu_reset(struct amdgpu_device *adev)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)kgd;
-
if (amdgpu_device_should_recover_gpu(adev))
amdgpu_device_gpu_recover(adev, NULL);
}
-int amdgpu_amdkfd_alloc_gtt_mem(struct kgd_dev *kgd, size_t size,
+int amdgpu_amdkfd_alloc_gtt_mem(struct amdgpu_device *adev, size_t size,
void **mem_obj, uint64_t *gpu_addr,
void **cpu_ptr, bool cp_mqd_gfx9)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)kgd;
struct amdgpu_bo *bo = NULL;
struct amdgpu_bo_param bp;
int r;
@@ -314,7 +311,7 @@ int amdgpu_amdkfd_alloc_gtt_mem(struct kgd_dev *kgd, size_t size,
return r;
}
-void amdgpu_amdkfd_free_gtt_mem(struct kgd_dev *kgd, void *mem_obj)
+void amdgpu_amdkfd_free_gtt_mem(struct amdgpu_device *adev, void *mem_obj)
{
struct amdgpu_bo *bo = (struct amdgpu_bo *) mem_obj;
@@ -325,10 +322,9 @@ void amdgpu_amdkfd_free_gtt_mem(struct kgd_dev *kgd, void *mem_obj)
amdgpu_bo_unref(&(bo));
}
-int amdgpu_amdkfd_alloc_gws(struct kgd_dev *kgd, size_t size,
+int amdgpu_amdkfd_alloc_gws(struct amdgpu_device *adev, size_t size,
void **mem_obj)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)kgd;
struct amdgpu_bo *bo = NULL;
struct amdgpu_bo_user *ubo;
struct amdgpu_bo_param bp;
@@ -355,18 +351,16 @@ int amdgpu_amdkfd_alloc_gws(struct kgd_dev *kgd, size_t size,
return 0;
}
-void amdgpu_amdkfd_free_gws(struct kgd_dev *kgd, void *mem_obj)
+void amdgpu_amdkfd_free_gws(struct amdgpu_device *adev, void *mem_obj)
{
struct amdgpu_bo *bo = (struct amdgpu_bo *)mem_obj;
amdgpu_bo_unref(&bo);
}
-uint32_t amdgpu_amdkfd_get_fw_version(struct kgd_dev *kgd,
+uint32_t amdgpu_amdkfd_get_fw_version(struct amdgpu_device *adev,
enum kgd_engine_type type)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)kgd;
-
switch (type) {
case KGD_ENGINE_PFP:
return adev->gfx.pfp_fw_version;
@@ -399,11 +393,9 @@ uint32_t amdgpu_amdkfd_get_fw_version(struct kgd_dev *kgd,
return 0;
}
-void amdgpu_amdkfd_get_local_mem_info(struct kgd_dev *kgd,
+void amdgpu_amdkfd_get_local_mem_info(struct amdgpu_device *adev,
struct kfd_local_mem_info *mem_info)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)kgd;
-
memset(mem_info, 0, sizeof(*mem_info));
mem_info->local_mem_size_public = adev->gmc.visible_vram_size;
@@ -428,19 +420,15 @@ void amdgpu_amdkfd_get_local_mem_info(struct kgd_dev *kgd,
mem_info->mem_clk_max = 100;
}
-uint64_t amdgpu_amdkfd_get_gpu_clock_counter(struct kgd_dev *kgd)
+uint64_t amdgpu_amdkfd_get_gpu_clock_counter(struct amdgpu_device *adev)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)kgd;
-
if (adev->gfx.funcs->get_gpu_clock_counter)
return adev->gfx.funcs->get_gpu_clock_counter(adev);
return 0;
}
-uint32_t amdgpu_amdkfd_get_max_engine_clock_in_mhz(struct kgd_dev *kgd)
+uint32_t amdgpu_amdkfd_get_max_engine_clock_in_mhz(struct amdgpu_device *adev)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)kgd;
-
/* the sclk is in quantas of 10kHz */
if (amdgpu_sriov_vf(adev))
return adev->clock.default_sclk / 100;
@@ -450,9 +438,8 @@ uint32_t amdgpu_amdkfd_get_max_engine_clock_in_mhz(struct kgd_dev *kgd)
return 100;
}
-void amdgpu_amdkfd_get_cu_info(struct kgd_dev *kgd, struct kfd_cu_info *cu_info)
+void amdgpu_amdkfd_get_cu_info(struct amdgpu_device *adev, struct kfd_cu_info *cu_info)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)kgd;
struct amdgpu_cu_info acu_info = adev->gfx.cu_info;
memset(cu_info, 0, sizeof(*cu_info));
@@ -473,13 +460,12 @@ void amdgpu_amdkfd_get_cu_info(struct kgd_dev *kgd, struct kfd_cu_info *cu_info)
cu_info->lds_size = acu_info.lds_size;
}
-int amdgpu_amdkfd_get_dmabuf_info(struct kgd_dev *kgd, int dma_buf_fd,
- struct kgd_dev **dma_buf_kgd,
+int amdgpu_amdkfd_get_dmabuf_info(struct amdgpu_device *adev, int dma_buf_fd,
+ struct amdgpu_device **dmabuf_adev,
uint64_t *bo_size, void *metadata_buffer,
size_t buffer_size, uint32_t *metadata_size,
uint32_t *flags)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)kgd;
struct dma_buf *dma_buf;
struct drm_gem_object *obj;
struct amdgpu_bo *bo;
@@ -507,8 +493,8 @@ int amdgpu_amdkfd_get_dmabuf_info(struct kgd_dev *kgd, int dma_buf_fd,
goto out_put;
r = 0;
- if (dma_buf_kgd)
- *dma_buf_kgd = (struct kgd_dev *)adev;
+ if (dmabuf_adev)
+ *dmabuf_adev = adev;
if (bo_size)
*bo_size = amdgpu_bo_size(bo);
if (metadata_buffer)
@@ -528,32 +514,18 @@ int amdgpu_amdkfd_get_dmabuf_info(struct kgd_dev *kgd, int dma_buf_fd,
return r;
}
-uint64_t amdgpu_amdkfd_get_vram_usage(struct kgd_dev *kgd)
+uint64_t amdgpu_amdkfd_get_vram_usage(struct amdgpu_device *adev)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)kgd;
struct ttm_resource_manager *vram_man = ttm_manager_type(&adev->mman.bdev, TTM_PL_VRAM);
return amdgpu_vram_mgr_usage(vram_man);
}
-uint64_t amdgpu_amdkfd_get_hive_id(struct kgd_dev *kgd)
+uint8_t amdgpu_amdkfd_get_xgmi_hops_count(struct amdgpu_device *dst,
+ struct amdgpu_device *src)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)kgd;
-
- return adev->gmc.xgmi.hive_id;
-}
-
-uint64_t amdgpu_amdkfd_get_unique_id(struct kgd_dev *kgd)
-{
- struct amdgpu_device *adev = (struct amdgpu_device *)kgd;
-
- return adev->unique_id;
-}
-
-uint8_t amdgpu_amdkfd_get_xgmi_hops_count(struct kgd_dev *dst, struct kgd_dev *src)
-{
- struct amdgpu_device *peer_adev = (struct amdgpu_device *)src;
- struct amdgpu_device *adev = (struct amdgpu_device *)dst;
+ struct amdgpu_device *peer_adev = src;
+ struct amdgpu_device *adev = dst;
int ret = amdgpu_xgmi_get_hops_count(adev, peer_adev);
if (ret < 0) {
@@ -565,16 +537,18 @@ uint8_t amdgpu_amdkfd_get_xgmi_hops_count(struct kgd_dev *dst, struct kgd_dev *s
return (uint8_t)ret;
}
-int amdgpu_amdkfd_get_xgmi_bandwidth_mbytes(struct kgd_dev *dst, struct kgd_dev *src, bool is_min)
+int amdgpu_amdkfd_get_xgmi_bandwidth_mbytes(struct amdgpu_device *dst,
+ struct amdgpu_device *src,
+ bool is_min)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)dst, *peer_adev;
+ struct amdgpu_device *adev = dst, *peer_adev;
int num_links;
if (adev->asic_type != CHIP_ALDEBARAN)
return 0;
if (src)
- peer_adev = (struct amdgpu_device *)src;
+ peer_adev = src;
/* num links returns 0 for indirect peers since indirect route is unknown. */
num_links = is_min ? 1 : amdgpu_xgmi_get_num_links(adev, peer_adev);
@@ -589,9 +563,8 @@ int amdgpu_amdkfd_get_xgmi_bandwidth_mbytes(struct kgd_dev *dst, struct kgd_dev
return (num_links * 16 * 25000)/BITS_PER_BYTE;
}
-int amdgpu_amdkfd_get_pcie_bandwidth_mbytes(struct kgd_dev *dev, bool is_min)
+int amdgpu_amdkfd_get_pcie_bandwidth_mbytes(struct amdgpu_device *adev, bool is_min)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)dev;
int num_lanes_shift = (is_min ? ffs(adev->pm.pcie_mlw_mask) :
fls(adev->pm.pcie_mlw_mask)) - 1;
int gen_speed_shift = (is_min ? ffs(adev->pm.pcie_gen_mask &
@@ -647,39 +620,11 @@ int amdgpu_amdkfd_get_pcie_bandwidth_mbytes(struct kgd_dev *dev, bool is_min)
return (num_lanes_factor * gen_speed_mbits_factor)/BITS_PER_BYTE;
}
-uint64_t amdgpu_amdkfd_get_mmio_remap_phys_addr(struct kgd_dev *kgd)
-{
- struct amdgpu_device *adev = (struct amdgpu_device *)kgd;
-
- return adev->rmmio_remap.bus_addr;
-}
-
-uint32_t amdgpu_amdkfd_get_num_gws(struct kgd_dev *kgd)
-{
- struct amdgpu_device *adev = (struct amdgpu_device *)kgd;
-
- return adev->gds.gws_size;
-}
-
-uint32_t amdgpu_amdkfd_get_asic_rev_id(struct kgd_dev *kgd)
-{
- struct amdgpu_device *adev = (struct amdgpu_device *)kgd;
-
- return adev->rev_id;
-}
-
-int amdgpu_amdkfd_get_noretry(struct kgd_dev *kgd)
-{
- struct amdgpu_device *adev = (struct amdgpu_device *)kgd;
-
- return adev->gmc.noretry;
-}
-
-int amdgpu_amdkfd_submit_ib(struct kgd_dev *kgd, enum kgd_engine_type engine,
+int amdgpu_amdkfd_submit_ib(struct amdgpu_device *adev,
+ enum kgd_engine_type engine,
uint32_t vmid, uint64_t gpu_addr,
uint32_t *ib_cmd, uint32_t ib_len)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)kgd;
struct amdgpu_job *job;
struct amdgpu_ib *ib;
struct amdgpu_ring *ring;
@@ -730,10 +675,8 @@ int amdgpu_amdkfd_submit_ib(struct kgd_dev *kgd, enum kgd_engine_type engine,
return ret;
}
-void amdgpu_amdkfd_set_compute_idle(struct kgd_dev *kgd, bool idle)
+void amdgpu_amdkfd_set_compute_idle(struct amdgpu_device *adev, bool idle)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)kgd;
-
amdgpu_dpm_switch_power_profile(adev,
PP_SMC_POWER_PROFILE_COMPUTE,
!idle);
@@ -747,10 +690,9 @@ bool amdgpu_amdkfd_is_kfd_vmid(struct amdgpu_device *adev, u32 vmid)
return false;
}
-int amdgpu_amdkfd_flush_gpu_tlb_vmid(struct kgd_dev *kgd, uint16_t vmid)
+int amdgpu_amdkfd_flush_gpu_tlb_vmid(struct amdgpu_device *adev,
+ uint16_t vmid)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)kgd;
-
if (adev->family == AMDGPU_FAMILY_AI) {
int i;
@@ -763,10 +705,9 @@ int amdgpu_amdkfd_flush_gpu_tlb_vmid(struct kgd_dev *kgd, uint16_t vmid)
return 0;
}
-int amdgpu_amdkfd_flush_gpu_tlb_pasid(struct kgd_dev *kgd, uint16_t pasid,
- enum TLB_FLUSH_TYPE flush_type)
+int amdgpu_amdkfd_flush_gpu_tlb_pasid(struct amdgpu_device *adev,
+ uint16_t pasid, enum TLB_FLUSH_TYPE flush_type)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)kgd;
bool all_hub = false;
if (adev->family == AMDGPU_FAMILY_AI)
@@ -775,21 +716,18 @@ int amdgpu_amdkfd_flush_gpu_tlb_pasid(struct kgd_dev *kgd, uint16_t pasid,
return amdgpu_gmc_flush_gpu_tlb_pasid(adev, pasid, flush_type, all_hub);
}
-bool amdgpu_amdkfd_have_atomics_support(struct kgd_dev *kgd)
+bool amdgpu_amdkfd_have_atomics_support(struct amdgpu_device *adev)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)kgd;
-
return adev->have_atomics_support;
}
-void amdgpu_amdkfd_ras_poison_consumption_handler(struct kgd_dev *kgd)
+void amdgpu_amdkfd_ras_poison_consumption_handler(struct amdgpu_device *adev, bool reset)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)kgd;
struct ras_err_data err_data = {0, 0, 0, NULL};
/* CPU MCA will handle page retirement if connected_to_cpu is 1 */
if (!adev->gmc.xgmi.connected_to_cpu)
- amdgpu_umc_process_ras_data_cb(adev, &err_data, NULL);
- else
- amdgpu_amdkfd_gpu_reset(kgd);
+ amdgpu_umc_poison_handler(adev, &err_data, reset);
+ else if (reset)
+ amdgpu_amdkfd_gpu_reset(adev);
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
index a15a478..61f899e 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
@@ -144,14 +144,16 @@ void amdgpu_amdkfd_interrupt(struct amdgpu_device *adev,
void amdgpu_amdkfd_device_probe(struct amdgpu_device *adev);
void amdgpu_amdkfd_device_init(struct amdgpu_device *adev);
void amdgpu_amdkfd_device_fini_sw(struct amdgpu_device *adev);
-int amdgpu_amdkfd_submit_ib(struct kgd_dev *kgd, enum kgd_engine_type engine,
+int amdgpu_amdkfd_submit_ib(struct amdgpu_device *adev,
+ enum kgd_engine_type engine,
uint32_t vmid, uint64_t gpu_addr,
uint32_t *ib_cmd, uint32_t ib_len);
-void amdgpu_amdkfd_set_compute_idle(struct kgd_dev *kgd, bool idle);
-bool amdgpu_amdkfd_have_atomics_support(struct kgd_dev *kgd);
-int amdgpu_amdkfd_flush_gpu_tlb_vmid(struct kgd_dev *kgd, uint16_t vmid);
-int amdgpu_amdkfd_flush_gpu_tlb_pasid(struct kgd_dev *kgd, uint16_t pasid,
- enum TLB_FLUSH_TYPE flush_type);
+void amdgpu_amdkfd_set_compute_idle(struct amdgpu_device *adev, bool idle);
+bool amdgpu_amdkfd_have_atomics_support(struct amdgpu_device *adev);
+int amdgpu_amdkfd_flush_gpu_tlb_vmid(struct amdgpu_device *adev,
+ uint16_t vmid);
+int amdgpu_amdkfd_flush_gpu_tlb_pasid(struct amdgpu_device *adev,
+ uint16_t pasid, enum TLB_FLUSH_TYPE flush_type);
bool amdgpu_amdkfd_is_kfd_vmid(struct amdgpu_device *adev, u32 vmid);
@@ -159,7 +161,7 @@ int amdgpu_amdkfd_pre_reset(struct amdgpu_device *adev);
int amdgpu_amdkfd_post_reset(struct amdgpu_device *adev);
-void amdgpu_amdkfd_gpu_reset(struct kgd_dev *kgd);
+void amdgpu_amdkfd_gpu_reset(struct amdgpu_device *adev);
int amdgpu_queue_mask_bit_to_set_resource_bit(struct amdgpu_device *adev,
int queue_bit);
@@ -198,37 +200,36 @@ int amdgpu_amdkfd_evict_userptr(struct kgd_mem *mem, struct mm_struct *mm)
}
#endif
/* Shared API */
-int amdgpu_amdkfd_alloc_gtt_mem(struct kgd_dev *kgd, size_t size,
+int amdgpu_amdkfd_alloc_gtt_mem(struct amdgpu_device *adev, size_t size,
void **mem_obj, uint64_t *gpu_addr,
void **cpu_ptr, bool mqd_gfx9);
-void amdgpu_amdkfd_free_gtt_mem(struct kgd_dev *kgd, void *mem_obj);
-int amdgpu_amdkfd_alloc_gws(struct kgd_dev *kgd, size_t size, void **mem_obj);
-void amdgpu_amdkfd_free_gws(struct kgd_dev *kgd, void *mem_obj);
+void amdgpu_amdkfd_free_gtt_mem(struct amdgpu_device *adev, void *mem_obj);
+int amdgpu_amdkfd_alloc_gws(struct amdgpu_device *adev, size_t size,
+ void **mem_obj);
+void amdgpu_amdkfd_free_gws(struct amdgpu_device *adev, void *mem_obj);
int amdgpu_amdkfd_add_gws_to_process(void *info, void *gws, struct kgd_mem **mem);
int amdgpu_amdkfd_remove_gws_from_process(void *info, void *mem);
-uint32_t amdgpu_amdkfd_get_fw_version(struct kgd_dev *kgd,
+uint32_t amdgpu_amdkfd_get_fw_version(struct amdgpu_device *adev,
enum kgd_engine_type type);
-void amdgpu_amdkfd_get_local_mem_info(struct kgd_dev *kgd,
+void amdgpu_amdkfd_get_local_mem_info(struct amdgpu_device *adev,
struct kfd_local_mem_info *mem_info);
-uint64_t amdgpu_amdkfd_get_gpu_clock_counter(struct kgd_dev *kgd);
+uint64_t amdgpu_amdkfd_get_gpu_clock_counter(struct amdgpu_device *adev);
-uint32_t amdgpu_amdkfd_get_max_engine_clock_in_mhz(struct kgd_dev *kgd);
-void amdgpu_amdkfd_get_cu_info(struct kgd_dev *kgd, struct kfd_cu_info *cu_info);
-int amdgpu_amdkfd_get_dmabuf_info(struct kgd_dev *kgd, int dma_buf_fd,
- struct kgd_dev **dmabuf_kgd,
+uint32_t amdgpu_amdkfd_get_max_engine_clock_in_mhz(struct amdgpu_device *adev);
+void amdgpu_amdkfd_get_cu_info(struct amdgpu_device *adev,
+ struct kfd_cu_info *cu_info);
+int amdgpu_amdkfd_get_dmabuf_info(struct amdgpu_device *adev, int dma_buf_fd,
+ struct amdgpu_device **dmabuf_adev,
uint64_t *bo_size, void *metadata_buffer,
size_t buffer_size, uint32_t *metadata_size,
uint32_t *flags);
-uint64_t amdgpu_amdkfd_get_vram_usage(struct kgd_dev *kgd);
-uint64_t amdgpu_amdkfd_get_hive_id(struct kgd_dev *kgd);
-uint64_t amdgpu_amdkfd_get_unique_id(struct kgd_dev *kgd);
-uint64_t amdgpu_amdkfd_get_mmio_remap_phys_addr(struct kgd_dev *kgd);
-uint32_t amdgpu_amdkfd_get_num_gws(struct kgd_dev *kgd);
-uint32_t amdgpu_amdkfd_get_asic_rev_id(struct kgd_dev *kgd);
-int amdgpu_amdkfd_get_noretry(struct kgd_dev *kgd);
-uint8_t amdgpu_amdkfd_get_xgmi_hops_count(struct kgd_dev *dst, struct kgd_dev *src);
-int amdgpu_amdkfd_get_xgmi_bandwidth_mbytes(struct kgd_dev *dst, struct kgd_dev *src, bool is_min);
-int amdgpu_amdkfd_get_pcie_bandwidth_mbytes(struct kgd_dev *dev, bool is_min);
+uint64_t amdgpu_amdkfd_get_vram_usage(struct amdgpu_device *adev);
+uint8_t amdgpu_amdkfd_get_xgmi_hops_count(struct amdgpu_device *dst,
+ struct amdgpu_device *src);
+int amdgpu_amdkfd_get_xgmi_bandwidth_mbytes(struct amdgpu_device *dst,
+ struct amdgpu_device *src,
+ bool is_min);
+int amdgpu_amdkfd_get_pcie_bandwidth_mbytes(struct amdgpu_device *adev, bool is_min);
/* Read user wptr from a specified user address space with page fault
* disabled. The memory must be pinned and mapped to the hardware when
@@ -258,45 +259,55 @@ int amdgpu_amdkfd_get_pcie_bandwidth_mbytes(struct kgd_dev *dev, bool is_min);
(&((struct amdgpu_fpriv *) \
((struct drm_file *)(drm_priv))->driver_priv)->vm)
-int amdgpu_amdkfd_gpuvm_acquire_process_vm(struct kgd_dev *kgd,
+int amdgpu_amdkfd_gpuvm_acquire_process_vm(struct amdgpu_device *adev,
struct file *filp, u32 pasid,
void **process_info,
struct dma_fence **ef);
-void amdgpu_amdkfd_gpuvm_release_process_vm(struct kgd_dev *kgd, void *drm_priv);
+void amdgpu_amdkfd_gpuvm_release_process_vm(struct amdgpu_device *adev,
+ void *drm_priv);
uint64_t amdgpu_amdkfd_gpuvm_get_process_page_dir(void *drm_priv);
int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu(
- struct kgd_dev *kgd, uint64_t va, uint64_t size,
+ struct amdgpu_device *adev, uint64_t va, uint64_t size,
void *drm_priv, struct kgd_mem **mem,
uint64_t *offset, uint32_t flags);
int amdgpu_amdkfd_gpuvm_free_memory_of_gpu(
- struct kgd_dev *kgd, struct kgd_mem *mem, void *drm_priv,
+ struct amdgpu_device *adev, struct kgd_mem *mem, void *drm_priv,
uint64_t *size);
int amdgpu_amdkfd_gpuvm_map_memory_to_gpu(
- struct kgd_dev *kgd, struct kgd_mem *mem, void *drm_priv, bool *table_freed);
+ struct amdgpu_device *adev, struct kgd_mem *mem, void *drm_priv,
+ bool *table_freed);
int amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu(
- struct kgd_dev *kgd, struct kgd_mem *mem, void *drm_priv);
+ struct amdgpu_device *adev, struct kgd_mem *mem, void *drm_priv);
int amdgpu_amdkfd_gpuvm_sync_memory(
- struct kgd_dev *kgd, struct kgd_mem *mem, bool intr);
-int amdgpu_amdkfd_gpuvm_map_gtt_bo_to_kernel(struct kgd_dev *kgd,
+ struct amdgpu_device *adev, struct kgd_mem *mem, bool intr);
+int amdgpu_amdkfd_gpuvm_map_gtt_bo_to_kernel(struct amdgpu_device *adev,
struct kgd_mem *mem, void **kptr, uint64_t *size);
-void amdgpu_amdkfd_gpuvm_unmap_gtt_bo_from_kernel(struct kgd_dev *kgd, struct kgd_mem *mem);
+void amdgpu_amdkfd_gpuvm_unmap_gtt_bo_from_kernel(struct amdgpu_device *adev,
+ struct kgd_mem *mem);
int amdgpu_amdkfd_gpuvm_restore_process_bos(void *process_info,
struct dma_fence **ef);
-int amdgpu_amdkfd_gpuvm_get_vm_fault_info(struct kgd_dev *kgd,
+int amdgpu_amdkfd_gpuvm_get_vm_fault_info(struct amdgpu_device *adev,
struct kfd_vm_fault_info *info);
-int amdgpu_amdkfd_gpuvm_import_dmabuf(struct kgd_dev *kgd,
+int amdgpu_amdkfd_gpuvm_import_dmabuf(struct amdgpu_device *adev,
struct dma_buf *dmabuf,
uint64_t va, void *drm_priv,
struct kgd_mem **mem, uint64_t *size,
uint64_t *mmap_offset);
-int amdgpu_amdkfd_get_tile_config(struct kgd_dev *kgd,
+int amdgpu_amdkfd_get_tile_config(struct amdgpu_device *adev,
struct tile_config *config);
-void amdgpu_amdkfd_ras_poison_consumption_handler(struct kgd_dev *kgd);
+void amdgpu_amdkfd_ras_poison_consumption_handler(struct amdgpu_device *adev,
+ bool reset);
#if IS_ENABLED(CONFIG_HSA_AMD)
void amdgpu_amdkfd_gpuvm_init_mem_limits(void);
void amdgpu_amdkfd_gpuvm_destroy_cb(struct amdgpu_device *adev,
struct amdgpu_vm *vm);
+
+/**
+ * @amdgpu_amdkfd_release_notify() - Notify KFD when GEM object is released
+ *
+ * Allows KFD to release its resources associated with the GEM object.
+ */
void amdgpu_amdkfd_release_notify(struct amdgpu_bo *bo);
void amdgpu_amdkfd_reserve_system_mem(uint64_t size);
#else
@@ -324,7 +335,7 @@ int kgd2kfd_schedule_evict_and_restore_process(struct mm_struct *mm,
#if IS_ENABLED(CONFIG_HSA_AMD)
int kgd2kfd_init(void);
void kgd2kfd_exit(void);
-struct kfd_dev *kgd2kfd_probe(struct kgd_dev *kgd, bool vf);
+struct kfd_dev *kgd2kfd_probe(struct amdgpu_device *adev, bool vf);
bool kgd2kfd_device_init(struct kfd_dev *kfd,
struct drm_device *ddev,
const struct kgd2kfd_shared_resources *gpu_resources);
@@ -348,7 +359,7 @@ static inline void kgd2kfd_exit(void)
}
static inline
-struct kfd_dev *kgd2kfd_probe(struct kgd_dev *kgd, bool vf)
+struct kfd_dev *kgd2kfd_probe(struct amdgpu_device *adev, bool vf)
{
return NULL;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_arcturus.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_arcturus.c
index 5a7f680..abe93b3 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_arcturus.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_arcturus.c
@@ -57,11 +57,6 @@
(*dump)[i++][1] = RREG32(addr); \
} while (0)
-static inline struct amdgpu_device *get_amdgpu_device(struct kgd_dev *kgd)
-{
- return (struct amdgpu_device *)kgd;
-}
-
static inline struct v9_sdma_mqd *get_sdma_mqd(void *mqd)
{
return (struct v9_sdma_mqd *)mqd;
@@ -123,10 +118,9 @@ static uint32_t get_sdma_rlc_reg_offset(struct amdgpu_device *adev,
return sdma_rlc_reg_offset;
}
-int kgd_arcturus_hqd_sdma_load(struct kgd_dev *kgd, void *mqd,
+int kgd_arcturus_hqd_sdma_load(struct amdgpu_device *adev, void *mqd,
uint32_t __user *wptr, struct mm_struct *mm)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
struct v9_sdma_mqd *m;
uint32_t sdma_rlc_reg_offset;
unsigned long end_jiffies;
@@ -193,11 +187,10 @@ int kgd_arcturus_hqd_sdma_load(struct kgd_dev *kgd, void *mqd,
return 0;
}
-int kgd_arcturus_hqd_sdma_dump(struct kgd_dev *kgd,
+int kgd_arcturus_hqd_sdma_dump(struct amdgpu_device *adev,
uint32_t engine_id, uint32_t queue_id,
uint32_t (**dump)[2], uint32_t *n_regs)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
uint32_t sdma_rlc_reg_offset = get_sdma_rlc_reg_offset(adev,
engine_id, queue_id);
uint32_t i = 0, reg;
@@ -225,9 +218,9 @@ int kgd_arcturus_hqd_sdma_dump(struct kgd_dev *kgd,
return 0;
}
-bool kgd_arcturus_hqd_sdma_is_occupied(struct kgd_dev *kgd, void *mqd)
+bool kgd_arcturus_hqd_sdma_is_occupied(struct amdgpu_device *adev,
+ void *mqd)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
struct v9_sdma_mqd *m;
uint32_t sdma_rlc_reg_offset;
uint32_t sdma_rlc_rb_cntl;
@@ -244,10 +237,9 @@ bool kgd_arcturus_hqd_sdma_is_occupied(struct kgd_dev *kgd, void *mqd)
return false;
}
-int kgd_arcturus_hqd_sdma_destroy(struct kgd_dev *kgd, void *mqd,
+int kgd_arcturus_hqd_sdma_destroy(struct amdgpu_device *adev, void *mqd,
unsigned int utimeout)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
struct v9_sdma_mqd *m;
uint32_t sdma_rlc_reg_offset;
uint32_t temp;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_arcturus.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_arcturus.h
index ce08131b..756c1a5 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_arcturus.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_arcturus.h
@@ -20,11 +20,12 @@
* OTHER DEALINGS IN THE SOFTWARE.
*/
-int kgd_arcturus_hqd_sdma_load(struct kgd_dev *kgd, void *mqd,
+int kgd_arcturus_hqd_sdma_load(struct amdgpu_device *adev, void *mqd,
uint32_t __user *wptr, struct mm_struct *mm);
-int kgd_arcturus_hqd_sdma_dump(struct kgd_dev *kgd,
+int kgd_arcturus_hqd_sdma_dump(struct amdgpu_device *adev,
uint32_t engine_id, uint32_t queue_id,
uint32_t (**dump)[2], uint32_t *n_regs);
-bool kgd_arcturus_hqd_sdma_is_occupied(struct kgd_dev *kgd, void *mqd);
-int kgd_arcturus_hqd_sdma_destroy(struct kgd_dev *kgd, void *mqd,
+bool kgd_arcturus_hqd_sdma_is_occupied(struct amdgpu_device *adev,
+ void *mqd);
+int kgd_arcturus_hqd_sdma_destroy(struct amdgpu_device *adev, void *mqd,
unsigned int utimeout);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v10.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v10.c
index 960acf6..7b7f4b2 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v10.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v10.c
@@ -39,37 +39,26 @@ enum hqd_dequeue_request_type {
SAVE_WAVES
};
-static inline struct amdgpu_device *get_amdgpu_device(struct kgd_dev *kgd)
-{
- return (struct amdgpu_device *)kgd;
-}
-
-static void lock_srbm(struct kgd_dev *kgd, uint32_t mec, uint32_t pipe,
+static void lock_srbm(struct amdgpu_device *adev, uint32_t mec, uint32_t pipe,
uint32_t queue, uint32_t vmid)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
-
mutex_lock(&adev->srbm_mutex);
nv_grbm_select(adev, mec, pipe, queue, vmid);
}
-static void unlock_srbm(struct kgd_dev *kgd)
+static void unlock_srbm(struct amdgpu_device *adev)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
-
nv_grbm_select(adev, 0, 0, 0, 0);
mutex_unlock(&adev->srbm_mutex);
}
-static void acquire_queue(struct kgd_dev *kgd, uint32_t pipe_id,
+static void acquire_queue(struct amdgpu_device *adev, uint32_t pipe_id,
uint32_t queue_id)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
-
uint32_t mec = (pipe_id / adev->gfx.mec.num_pipe_per_mec) + 1;
uint32_t pipe = (pipe_id % adev->gfx.mec.num_pipe_per_mec);
- lock_srbm(kgd, mec, pipe, queue_id, 0);
+ lock_srbm(adev, mec, pipe, queue_id, 0);
}
static uint64_t get_queue_mask(struct amdgpu_device *adev,
@@ -81,33 +70,29 @@ static uint64_t get_queue_mask(struct amdgpu_device *adev,
return 1ull << bit;
}
-static void release_queue(struct kgd_dev *kgd)
+static void release_queue(struct amdgpu_device *adev)
{
- unlock_srbm(kgd);
+ unlock_srbm(adev);
}
-static void kgd_program_sh_mem_settings(struct kgd_dev *kgd, uint32_t vmid,
+static void kgd_program_sh_mem_settings(struct amdgpu_device *adev, uint32_t vmid,
uint32_t sh_mem_config,
uint32_t sh_mem_ape1_base,
uint32_t sh_mem_ape1_limit,
uint32_t sh_mem_bases)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
-
- lock_srbm(kgd, 0, 0, 0, vmid);
+ lock_srbm(adev, 0, 0, 0, vmid);
WREG32_SOC15(GC, 0, mmSH_MEM_CONFIG, sh_mem_config);
WREG32_SOC15(GC, 0, mmSH_MEM_BASES, sh_mem_bases);
/* APE1 no longer exists on GFX9 */
- unlock_srbm(kgd);
+ unlock_srbm(adev);
}
-static int kgd_set_pasid_vmid_mapping(struct kgd_dev *kgd, u32 pasid,
+static int kgd_set_pasid_vmid_mapping(struct amdgpu_device *adev, u32 pasid,
unsigned int vmid)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
-
/*
* We have to assume that there is no outstanding mapping.
* The ATC_VMID_PASID_MAPPING_UPDATE_STATUS bit could be 0 because
@@ -150,22 +135,21 @@ static int kgd_set_pasid_vmid_mapping(struct kgd_dev *kgd, u32 pasid,
* but still works
*/
-static int kgd_init_interrupts(struct kgd_dev *kgd, uint32_t pipe_id)
+static int kgd_init_interrupts(struct amdgpu_device *adev, uint32_t pipe_id)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
uint32_t mec;
uint32_t pipe;
mec = (pipe_id / adev->gfx.mec.num_pipe_per_mec) + 1;
pipe = (pipe_id % adev->gfx.mec.num_pipe_per_mec);
- lock_srbm(kgd, mec, pipe, 0, 0);
+ lock_srbm(adev, mec, pipe, 0, 0);
WREG32_SOC15(GC, 0, mmCPC_INT_CNTL,
CP_INT_CNTL_RING0__TIME_STAMP_INT_ENABLE_MASK |
CP_INT_CNTL_RING0__OPCODE_ERROR_INT_ENABLE_MASK);
- unlock_srbm(kgd);
+ unlock_srbm(adev);
return 0;
}
@@ -218,12 +202,11 @@ static inline struct v10_sdma_mqd *get_sdma_mqd(void *mqd)
return (struct v10_sdma_mqd *)mqd;
}
-static int kgd_hqd_load(struct kgd_dev *kgd, void *mqd, uint32_t pipe_id,
- uint32_t queue_id, uint32_t __user *wptr,
- uint32_t wptr_shift, uint32_t wptr_mask,
- struct mm_struct *mm)
+static int kgd_hqd_load(struct amdgpu_device *adev, void *mqd,
+ uint32_t pipe_id, uint32_t queue_id,
+ uint32_t __user *wptr, uint32_t wptr_shift,
+ uint32_t wptr_mask, struct mm_struct *mm)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
struct v10_compute_mqd *m;
uint32_t *mqd_hqd;
uint32_t reg, hqd_base, data;
@@ -231,7 +214,7 @@ static int kgd_hqd_load(struct kgd_dev *kgd, void *mqd, uint32_t pipe_id,
m = get_mqd(mqd);
pr_debug("Load hqd of pipe %d queue %d\n", pipe_id, queue_id);
- acquire_queue(kgd, pipe_id, queue_id);
+ acquire_queue(adev, pipe_id, queue_id);
/* HQD registers extend from CP_MQD_BASE_ADDR to CP_HQD_EOP_WPTR_MEM. */
mqd_hqd = &m->cp_mqd_base_addr_lo;
@@ -296,16 +279,15 @@ static int kgd_hqd_load(struct kgd_dev *kgd, void *mqd, uint32_t pipe_id,
data = REG_SET_FIELD(m->cp_hqd_active, CP_HQD_ACTIVE, ACTIVE, 1);
WREG32_SOC15(GC, 0, mmCP_HQD_ACTIVE, data);
- release_queue(kgd);
+ release_queue(adev);
return 0;
}
-static int kgd_hiq_mqd_load(struct kgd_dev *kgd, void *mqd,
+static int kgd_hiq_mqd_load(struct amdgpu_device *adev, void *mqd,
uint32_t pipe_id, uint32_t queue_id,
uint32_t doorbell_off)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
struct amdgpu_ring *kiq_ring = &adev->gfx.kiq.ring;
struct v10_compute_mqd *m;
uint32_t mec, pipe;
@@ -313,7 +295,7 @@ static int kgd_hiq_mqd_load(struct kgd_dev *kgd, void *mqd,
m = get_mqd(mqd);
- acquire_queue(kgd, pipe_id, queue_id);
+ acquire_queue(adev, pipe_id, queue_id);
mec = (pipe_id / adev->gfx.mec.num_pipe_per_mec) + 1;
pipe = (pipe_id % adev->gfx.mec.num_pipe_per_mec);
@@ -349,16 +331,15 @@ static int kgd_hiq_mqd_load(struct kgd_dev *kgd, void *mqd,
out_unlock:
spin_unlock(&adev->gfx.kiq.ring_lock);
- release_queue(kgd);
+ release_queue(adev);
return r;
}
-static int kgd_hqd_dump(struct kgd_dev *kgd,
+static int kgd_hqd_dump(struct amdgpu_device *adev,
uint32_t pipe_id, uint32_t queue_id,
uint32_t (**dump)[2], uint32_t *n_regs)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
uint32_t i = 0, reg;
#define HQD_N_REGS 56
#define DUMP_REG(addr) do { \
@@ -372,13 +353,13 @@ static int kgd_hqd_dump(struct kgd_dev *kgd,
if (*dump == NULL)
return -ENOMEM;
- acquire_queue(kgd, pipe_id, queue_id);
+ acquire_queue(adev, pipe_id, queue_id);
for (reg = SOC15_REG_OFFSET(GC, 0, mmCP_MQD_BASE_ADDR);
reg <= SOC15_REG_OFFSET(GC, 0, mmCP_HQD_PQ_WPTR_HI); reg++)
DUMP_REG(reg);
- release_queue(kgd);
+ release_queue(adev);
WARN_ON_ONCE(i != HQD_N_REGS);
*n_regs = i;
@@ -386,10 +367,9 @@ static int kgd_hqd_dump(struct kgd_dev *kgd,
return 0;
}
-static int kgd_hqd_sdma_load(struct kgd_dev *kgd, void *mqd,
+static int kgd_hqd_sdma_load(struct amdgpu_device *adev, void *mqd,
uint32_t __user *wptr, struct mm_struct *mm)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
struct v10_sdma_mqd *m;
uint32_t sdma_rlc_reg_offset;
unsigned long end_jiffies;
@@ -456,11 +436,10 @@ static int kgd_hqd_sdma_load(struct kgd_dev *kgd, void *mqd,
return 0;
}
-static int kgd_hqd_sdma_dump(struct kgd_dev *kgd,
+static int kgd_hqd_sdma_dump(struct amdgpu_device *adev,
uint32_t engine_id, uint32_t queue_id,
uint32_t (**dump)[2], uint32_t *n_regs)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
uint32_t sdma_rlc_reg_offset = get_sdma_rlc_reg_offset(adev,
engine_id, queue_id);
uint32_t i = 0, reg;
@@ -488,15 +467,15 @@ static int kgd_hqd_sdma_dump(struct kgd_dev *kgd,
return 0;
}
-static bool kgd_hqd_is_occupied(struct kgd_dev *kgd, uint64_t queue_address,
- uint32_t pipe_id, uint32_t queue_id)
+static bool kgd_hqd_is_occupied(struct amdgpu_device *adev,
+ uint64_t queue_address, uint32_t pipe_id,
+ uint32_t queue_id)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
uint32_t act;
bool retval = false;
uint32_t low, high;
- acquire_queue(kgd, pipe_id, queue_id);
+ acquire_queue(adev, pipe_id, queue_id);
act = RREG32_SOC15(GC, 0, mmCP_HQD_ACTIVE);
if (act) {
low = lower_32_bits(queue_address >> 8);
@@ -506,13 +485,12 @@ static bool kgd_hqd_is_occupied(struct kgd_dev *kgd, uint64_t queue_address,
high == RREG32_SOC15(GC, 0, mmCP_HQD_PQ_BASE_HI))
retval = true;
}
- release_queue(kgd);
+ release_queue(adev);
return retval;
}
-static bool kgd_hqd_sdma_is_occupied(struct kgd_dev *kgd, void *mqd)
+static bool kgd_hqd_sdma_is_occupied(struct amdgpu_device *adev, void *mqd)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
struct v10_sdma_mqd *m;
uint32_t sdma_rlc_reg_offset;
uint32_t sdma_rlc_rb_cntl;
@@ -529,12 +507,11 @@ static bool kgd_hqd_sdma_is_occupied(struct kgd_dev *kgd, void *mqd)
return false;
}
-static int kgd_hqd_destroy(struct kgd_dev *kgd, void *mqd,
+static int kgd_hqd_destroy(struct amdgpu_device *adev, void *mqd,
enum kfd_preempt_type reset_type,
unsigned int utimeout, uint32_t pipe_id,
uint32_t queue_id)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
enum hqd_dequeue_request_type type;
unsigned long end_jiffies;
uint32_t temp;
@@ -548,7 +525,7 @@ static int kgd_hqd_destroy(struct kgd_dev *kgd, void *mqd,
int retry;
#endif
- acquire_queue(kgd, pipe_id, queue_id);
+ acquire_queue(adev, pipe_id, queue_id);
if (m->cp_hqd_vmid == 0)
WREG32_FIELD15(GC, 0, RLC_CP_SCHEDULERS, scheduler1, 0);
@@ -633,20 +610,19 @@ static int kgd_hqd_destroy(struct kgd_dev *kgd, void *mqd,
break;
if (time_after(jiffies, end_jiffies)) {
pr_err("cp queue preemption time out.\n");
- release_queue(kgd);
+ release_queue(adev);
return -ETIME;
}
usleep_range(500, 1000);
}
- release_queue(kgd);
+ release_queue(adev);
return 0;
}
-static int kgd_hqd_sdma_destroy(struct kgd_dev *kgd, void *mqd,
+static int kgd_hqd_sdma_destroy(struct amdgpu_device *adev, void *mqd,
unsigned int utimeout)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
struct v10_sdma_mqd *m;
uint32_t sdma_rlc_reg_offset;
uint32_t temp;
@@ -683,11 +659,10 @@ static int kgd_hqd_sdma_destroy(struct kgd_dev *kgd, void *mqd,
return 0;
}
-static bool get_atc_vmid_pasid_mapping_info(struct kgd_dev *kgd,
+static bool get_atc_vmid_pasid_mapping_info(struct amdgpu_device *adev,
uint8_t vmid, uint16_t *p_pasid)
{
uint32_t value;
- struct amdgpu_device *adev = (struct amdgpu_device *) kgd;
value = RREG32(SOC15_REG_OFFSET(ATHUB, 0, mmATC_VMID0_PASID_MAPPING)
+ vmid);
@@ -696,12 +671,12 @@ static bool get_atc_vmid_pasid_mapping_info(struct kgd_dev *kgd,
return !!(value & ATC_VMID0_PASID_MAPPING__VALID_MASK);
}
-static int kgd_address_watch_disable(struct kgd_dev *kgd)
+static int kgd_address_watch_disable(struct amdgpu_device *adev)
{
return 0;
}
-static int kgd_address_watch_execute(struct kgd_dev *kgd,
+static int kgd_address_watch_execute(struct amdgpu_device *adev,
unsigned int watch_point_id,
uint32_t cntl_val,
uint32_t addr_hi,
@@ -710,11 +685,10 @@ static int kgd_address_watch_execute(struct kgd_dev *kgd,
return 0;
}
-static int kgd_wave_control_execute(struct kgd_dev *kgd,
+static int kgd_wave_control_execute(struct amdgpu_device *adev,
uint32_t gfx_index_val,
uint32_t sq_cmd)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
uint32_t data = 0;
mutex_lock(&adev->grbm_idx_mutex);
@@ -735,18 +709,16 @@ static int kgd_wave_control_execute(struct kgd_dev *kgd,
return 0;
}
-static uint32_t kgd_address_watch_get_offset(struct kgd_dev *kgd,
+static uint32_t kgd_address_watch_get_offset(struct amdgpu_device *adev,
unsigned int watch_point_id,
unsigned int reg_offset)
{
return 0;
}
-static void set_vm_context_page_table_base(struct kgd_dev *kgd, uint32_t vmid,
- uint64_t page_table_base)
+static void set_vm_context_page_table_base(struct amdgpu_device *adev,
+ uint32_t vmid, uint64_t page_table_base)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
-
if (!amdgpu_amdkfd_is_kfd_vmid(adev, vmid)) {
pr_err("trying to set page table base for wrong VMID %u\n",
vmid);
@@ -757,12 +729,10 @@ static void set_vm_context_page_table_base(struct kgd_dev *kgd, uint32_t vmid,
adev->gfxhub.funcs->setup_vm_pt_regs(adev, vmid, page_table_base);
}
-static void program_trap_handler_settings(struct kgd_dev *kgd,
+static void program_trap_handler_settings(struct amdgpu_device *adev,
uint32_t vmid, uint64_t tba_addr, uint64_t tma_addr)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
-
- lock_srbm(kgd, 0, 0, 0, vmid);
+ lock_srbm(adev, 0, 0, 0, vmid);
/*
* Program TBA registers
@@ -781,7 +751,7 @@ static void program_trap_handler_settings(struct kgd_dev *kgd,
WREG32(SOC15_REG_OFFSET(GC, 0, mmSQ_SHADER_TMA_HI),
upper_32_bits(tma_addr >> 8));
- unlock_srbm(kgd);
+ unlock_srbm(adev);
}
const struct kfd2kgd_calls gfx_v10_kfd2kgd = {
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v10_3.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v10_3.c
index dac0d75..1f37d35 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v10_3.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v10_3.c
@@ -38,37 +38,26 @@ enum hqd_dequeue_request_type {
SAVE_WAVES
};
-static inline struct amdgpu_device *get_amdgpu_device(struct kgd_dev *kgd)
-{
- return (struct amdgpu_device *)kgd;
-}
-
-static void lock_srbm(struct kgd_dev *kgd, uint32_t mec, uint32_t pipe,
+static void lock_srbm(struct amdgpu_device *adev, uint32_t mec, uint32_t pipe,
uint32_t queue, uint32_t vmid)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
-
mutex_lock(&adev->srbm_mutex);
nv_grbm_select(adev, mec, pipe, queue, vmid);
}
-static void unlock_srbm(struct kgd_dev *kgd)
+static void unlock_srbm(struct amdgpu_device *adev)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
-
nv_grbm_select(adev, 0, 0, 0, 0);
mutex_unlock(&adev->srbm_mutex);
}
-static void acquire_queue(struct kgd_dev *kgd, uint32_t pipe_id,
+static void acquire_queue(struct amdgpu_device *adev, uint32_t pipe_id,
uint32_t queue_id)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
-
uint32_t mec = (pipe_id / adev->gfx.mec.num_pipe_per_mec) + 1;
uint32_t pipe = (pipe_id % adev->gfx.mec.num_pipe_per_mec);
- lock_srbm(kgd, mec, pipe, queue_id, 0);
+ lock_srbm(adev, mec, pipe, queue_id, 0);
}
static uint64_t get_queue_mask(struct amdgpu_device *adev,
@@ -80,34 +69,30 @@ static uint64_t get_queue_mask(struct amdgpu_device *adev,
return 1ull << bit;
}
-static void release_queue(struct kgd_dev *kgd)
+static void release_queue(struct amdgpu_device *adev)
{
- unlock_srbm(kgd);
+ unlock_srbm(adev);
}
-static void program_sh_mem_settings_v10_3(struct kgd_dev *kgd, uint32_t vmid,
+static void program_sh_mem_settings_v10_3(struct amdgpu_device *adev, uint32_t vmid,
uint32_t sh_mem_config,
uint32_t sh_mem_ape1_base,
uint32_t sh_mem_ape1_limit,
uint32_t sh_mem_bases)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
-
- lock_srbm(kgd, 0, 0, 0, vmid);
+ lock_srbm(adev, 0, 0, 0, vmid);
WREG32_SOC15(GC, 0, mmSH_MEM_CONFIG, sh_mem_config);
WREG32_SOC15(GC, 0, mmSH_MEM_BASES, sh_mem_bases);
/* APE1 no longer exists on GFX9 */
- unlock_srbm(kgd);
+ unlock_srbm(adev);
}
/* ATC is defeatured on Sienna_Cichlid */
-static int set_pasid_vmid_mapping_v10_3(struct kgd_dev *kgd, unsigned int pasid,
+static int set_pasid_vmid_mapping_v10_3(struct amdgpu_device *adev, unsigned int pasid,
unsigned int vmid)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
-
uint32_t value = pasid << IH_VMID_0_LUT__PASID__SHIFT;
/* Mapping vmid to pasid also for IH block */
@@ -118,22 +103,21 @@ static int set_pasid_vmid_mapping_v10_3(struct kgd_dev *kgd, unsigned int pasid,
return 0;
}
-static int init_interrupts_v10_3(struct kgd_dev *kgd, uint32_t pipe_id)
+static int init_interrupts_v10_3(struct amdgpu_device *adev, uint32_t pipe_id)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
uint32_t mec;
uint32_t pipe;
mec = (pipe_id / adev->gfx.mec.num_pipe_per_mec) + 1;
pipe = (pipe_id % adev->gfx.mec.num_pipe_per_mec);
- lock_srbm(kgd, mec, pipe, 0, 0);
+ lock_srbm(adev, mec, pipe, 0, 0);
WREG32_SOC15(GC, 0, mmCPC_INT_CNTL,
CP_INT_CNTL_RING0__TIME_STAMP_INT_ENABLE_MASK |
CP_INT_CNTL_RING0__OPCODE_ERROR_INT_ENABLE_MASK);
- unlock_srbm(kgd);
+ unlock_srbm(adev);
return 0;
}
@@ -188,12 +172,11 @@ static inline struct v10_sdma_mqd *get_sdma_mqd(void *mqd)
return (struct v10_sdma_mqd *)mqd;
}
-static int hqd_load_v10_3(struct kgd_dev *kgd, void *mqd, uint32_t pipe_id,
- uint32_t queue_id, uint32_t __user *wptr,
- uint32_t wptr_shift, uint32_t wptr_mask,
- struct mm_struct *mm)
+static int hqd_load_v10_3(struct amdgpu_device *adev, void *mqd,
+ uint32_t pipe_id, uint32_t queue_id,
+ uint32_t __user *wptr, uint32_t wptr_shift,
+ uint32_t wptr_mask, struct mm_struct *mm)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
struct v10_compute_mqd *m;
uint32_t *mqd_hqd;
uint32_t reg, hqd_base, data;
@@ -201,7 +184,7 @@ static int hqd_load_v10_3(struct kgd_dev *kgd, void *mqd, uint32_t pipe_id,
m = get_mqd(mqd);
pr_debug("Load hqd of pipe %d queue %d\n", pipe_id, queue_id);
- acquire_queue(kgd, pipe_id, queue_id);
+ acquire_queue(adev, pipe_id, queue_id);
/* HIQ is set during driver init period with vmid set to 0*/
if (m->cp_hqd_vmid == 0) {
@@ -281,16 +264,15 @@ static int hqd_load_v10_3(struct kgd_dev *kgd, void *mqd, uint32_t pipe_id,
data = REG_SET_FIELD(m->cp_hqd_active, CP_HQD_ACTIVE, ACTIVE, 1);
WREG32_SOC15(GC, 0, mmCP_HQD_ACTIVE, data);
- release_queue(kgd);
+ release_queue(adev);
return 0;
}
-static int hiq_mqd_load_v10_3(struct kgd_dev *kgd, void *mqd,
+static int hiq_mqd_load_v10_3(struct amdgpu_device *adev, void *mqd,
uint32_t pipe_id, uint32_t queue_id,
uint32_t doorbell_off)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
struct amdgpu_ring *kiq_ring = &adev->gfx.kiq.ring;
struct v10_compute_mqd *m;
uint32_t mec, pipe;
@@ -298,7 +280,7 @@ static int hiq_mqd_load_v10_3(struct kgd_dev *kgd, void *mqd,
m = get_mqd(mqd);
- acquire_queue(kgd, pipe_id, queue_id);
+ acquire_queue(adev, pipe_id, queue_id);
mec = (pipe_id / adev->gfx.mec.num_pipe_per_mec) + 1;
pipe = (pipe_id % adev->gfx.mec.num_pipe_per_mec);
@@ -334,16 +316,15 @@ static int hiq_mqd_load_v10_3(struct kgd_dev *kgd, void *mqd,
out_unlock:
spin_unlock(&adev->gfx.kiq.ring_lock);
- release_queue(kgd);
+ release_queue(adev);
return r;
}
-static int hqd_dump_v10_3(struct kgd_dev *kgd,
+static int hqd_dump_v10_3(struct amdgpu_device *adev,
uint32_t pipe_id, uint32_t queue_id,
uint32_t (**dump)[2], uint32_t *n_regs)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
uint32_t i = 0, reg;
#define HQD_N_REGS 56
#define DUMP_REG(addr) do { \
@@ -357,13 +338,13 @@ static int hqd_dump_v10_3(struct kgd_dev *kgd,
if (*dump == NULL)
return -ENOMEM;
- acquire_queue(kgd, pipe_id, queue_id);
+ acquire_queue(adev, pipe_id, queue_id);
for (reg = SOC15_REG_OFFSET(GC, 0, mmCP_MQD_BASE_ADDR);
reg <= SOC15_REG_OFFSET(GC, 0, mmCP_HQD_PQ_WPTR_HI); reg++)
DUMP_REG(reg);
- release_queue(kgd);
+ release_queue(adev);
WARN_ON_ONCE(i != HQD_N_REGS);
*n_regs = i;
@@ -371,10 +352,9 @@ static int hqd_dump_v10_3(struct kgd_dev *kgd,
return 0;
}
-static int hqd_sdma_load_v10_3(struct kgd_dev *kgd, void *mqd,
+static int hqd_sdma_load_v10_3(struct amdgpu_device *adev, void *mqd,
uint32_t __user *wptr, struct mm_struct *mm)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
struct v10_sdma_mqd *m;
uint32_t sdma_rlc_reg_offset;
unsigned long end_jiffies;
@@ -441,11 +421,10 @@ static int hqd_sdma_load_v10_3(struct kgd_dev *kgd, void *mqd,
return 0;
}
-static int hqd_sdma_dump_v10_3(struct kgd_dev *kgd,
+static int hqd_sdma_dump_v10_3(struct amdgpu_device *adev,
uint32_t engine_id, uint32_t queue_id,
uint32_t (**dump)[2], uint32_t *n_regs)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
uint32_t sdma_rlc_reg_offset = get_sdma_rlc_reg_offset(adev,
engine_id, queue_id);
uint32_t i = 0, reg;
@@ -473,15 +452,15 @@ static int hqd_sdma_dump_v10_3(struct kgd_dev *kgd,
return 0;
}
-static bool hqd_is_occupied_v10_3(struct kgd_dev *kgd, uint64_t queue_address,
- uint32_t pipe_id, uint32_t queue_id)
+static bool hqd_is_occupied_v10_3(struct amdgpu_device *adev,
+ uint64_t queue_address, uint32_t pipe_id,
+ uint32_t queue_id)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
uint32_t act;
bool retval = false;
uint32_t low, high;
- acquire_queue(kgd, pipe_id, queue_id);
+ acquire_queue(adev, pipe_id, queue_id);
act = RREG32_SOC15(GC, 0, mmCP_HQD_ACTIVE);
if (act) {
low = lower_32_bits(queue_address >> 8);
@@ -491,13 +470,13 @@ static bool hqd_is_occupied_v10_3(struct kgd_dev *kgd, uint64_t queue_address,
high == RREG32_SOC15(GC, 0, mmCP_HQD_PQ_BASE_HI))
retval = true;
}
- release_queue(kgd);
+ release_queue(adev);
return retval;
}
-static bool hqd_sdma_is_occupied_v10_3(struct kgd_dev *kgd, void *mqd)
+static bool hqd_sdma_is_occupied_v10_3(struct amdgpu_device *adev,
+ void *mqd)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
struct v10_sdma_mqd *m;
uint32_t sdma_rlc_reg_offset;
uint32_t sdma_rlc_rb_cntl;
@@ -514,18 +493,17 @@ static bool hqd_sdma_is_occupied_v10_3(struct kgd_dev *kgd, void *mqd)
return false;
}
-static int hqd_destroy_v10_3(struct kgd_dev *kgd, void *mqd,
+static int hqd_destroy_v10_3(struct amdgpu_device *adev, void *mqd,
enum kfd_preempt_type reset_type,
unsigned int utimeout, uint32_t pipe_id,
uint32_t queue_id)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
enum hqd_dequeue_request_type type;
unsigned long end_jiffies;
uint32_t temp;
struct v10_compute_mqd *m = get_mqd(mqd);
- acquire_queue(kgd, pipe_id, queue_id);
+ acquire_queue(adev, pipe_id, queue_id);
if (m->cp_hqd_vmid == 0)
WREG32_FIELD15(GC, 0, RLC_CP_SCHEDULERS, scheduler1, 0);
@@ -555,20 +533,19 @@ static int hqd_destroy_v10_3(struct kgd_dev *kgd, void *mqd,
if (time_after(jiffies, end_jiffies)) {
pr_err("cp queue pipe %d queue %d preemption failed\n",
pipe_id, queue_id);
- release_queue(kgd);
+ release_queue(adev);
return -ETIME;
}
usleep_range(500, 1000);
}
- release_queue(kgd);
+ release_queue(adev);
return 0;
}
-static int hqd_sdma_destroy_v10_3(struct kgd_dev *kgd, void *mqd,
+static int hqd_sdma_destroy_v10_3(struct amdgpu_device *adev, void *mqd,
unsigned int utimeout)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
struct v10_sdma_mqd *m;
uint32_t sdma_rlc_reg_offset;
uint32_t temp;
@@ -606,12 +583,12 @@ static int hqd_sdma_destroy_v10_3(struct kgd_dev *kgd, void *mqd,
}
-static int address_watch_disable_v10_3(struct kgd_dev *kgd)
+static int address_watch_disable_v10_3(struct amdgpu_device *adev)
{
return 0;
}
-static int address_watch_execute_v10_3(struct kgd_dev *kgd,
+static int address_watch_execute_v10_3(struct amdgpu_device *adev,
unsigned int watch_point_id,
uint32_t cntl_val,
uint32_t addr_hi,
@@ -620,11 +597,10 @@ static int address_watch_execute_v10_3(struct kgd_dev *kgd,
return 0;
}
-static int wave_control_execute_v10_3(struct kgd_dev *kgd,
+static int wave_control_execute_v10_3(struct amdgpu_device *adev,
uint32_t gfx_index_val,
uint32_t sq_cmd)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
uint32_t data = 0;
mutex_lock(&adev->grbm_idx_mutex);
@@ -645,28 +621,24 @@ static int wave_control_execute_v10_3(struct kgd_dev *kgd,
return 0;
}
-static uint32_t address_watch_get_offset_v10_3(struct kgd_dev *kgd,
+static uint32_t address_watch_get_offset_v10_3(struct amdgpu_device *adev,
unsigned int watch_point_id,
unsigned int reg_offset)
{
return 0;
}
-static void set_vm_context_page_table_base_v10_3(struct kgd_dev *kgd, uint32_t vmid,
- uint64_t page_table_base)
+static void set_vm_context_page_table_base_v10_3(struct amdgpu_device *adev,
+ uint32_t vmid, uint64_t page_table_base)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
-
/* SDMA is on gfxhub as well for Navi1* series */
adev->gfxhub.funcs->setup_vm_pt_regs(adev, vmid, page_table_base);
}
-static void program_trap_handler_settings_v10_3(struct kgd_dev *kgd,
+static void program_trap_handler_settings_v10_3(struct amdgpu_device *adev,
uint32_t vmid, uint64_t tba_addr, uint64_t tma_addr)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
-
- lock_srbm(kgd, 0, 0, 0, vmid);
+ lock_srbm(adev, 0, 0, 0, vmid);
/*
* Program TBA registers
@@ -685,15 +657,14 @@ static void program_trap_handler_settings_v10_3(struct kgd_dev *kgd,
WREG32(SOC15_REG_OFFSET(GC, 0, mmSQ_SHADER_TMA_HI),
upper_32_bits(tma_addr >> 8));
- unlock_srbm(kgd);
+ unlock_srbm(adev);
}
#if 0
-uint32_t enable_debug_trap_v10_3(struct kgd_dev *kgd,
+uint32_t enable_debug_trap_v10_3(struct amdgpu_device *adev,
uint32_t trap_debug_wave_launch_mode,
uint32_t vmid)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
uint32_t data = 0;
uint32_t orig_wave_cntl_value;
uint32_t orig_stall_vmid;
@@ -720,10 +691,8 @@ uint32_t enable_debug_trap_v10_3(struct kgd_dev *kgd,
return 0;
}
-uint32_t disable_debug_trap_v10_3(struct kgd_dev *kgd)
+uint32_t disable_debug_trap_v10_3(struct amdgpu_device *adev)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
-
mutex_lock(&adev->grbm_idx_mutex);
WREG32(SOC15_REG_OFFSET(GC, 0, mmSPI_GDBG_TRAP_MASK), 0);
@@ -733,11 +702,10 @@ uint32_t disable_debug_trap_v10_3(struct kgd_dev *kgd)
return 0;
}
-uint32_t set_wave_launch_trap_override_v10_3(struct kgd_dev *kgd,
+uint32_t set_wave_launch_trap_override_v10_3(struct amdgpu_device *adev,
uint32_t trap_override,
uint32_t trap_mask)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
uint32_t data = 0;
mutex_lock(&adev->grbm_idx_mutex);
@@ -762,11 +730,10 @@ uint32_t set_wave_launch_trap_override_v10_3(struct kgd_dev *kgd,
return 0;
}
-uint32_t set_wave_launch_mode_v10_3(struct kgd_dev *kgd,
+uint32_t set_wave_launch_mode_v10_3(struct amdgpu_device *adev,
uint8_t wave_launch_mode,
uint32_t vmid)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
uint32_t data = 0;
bool is_stall_mode;
bool is_mode_set;
@@ -805,16 +772,14 @@ uint32_t set_wave_launch_mode_v10_3(struct kgd_dev *kgd,
* sem_rearm_wait_time -- Wait Count for Semaphore re-arm.
* deq_retry_wait_time -- Wait Count for Global Wave Syncs.
*/
-void get_iq_wait_times_v10_3(struct kgd_dev *kgd,
+void get_iq_wait_times_v10_3(struct amdgpu_device *adev,
uint32_t *wait_times)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
-
*wait_times = RREG32(SOC15_REG_OFFSET(GC, 0, mmCP_IQ_WAIT_TIME2));
}
-void build_grace_period_packet_info_v10_3(struct kgd_dev *kgd,
+void build_grace_period_packet_info_v10_3(struct amdgpu_device *adev,
uint32_t wait_times,
uint32_t grace_period,
uint32_t *reg_offset,
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v7.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v7.c
index b91d27e..36528da 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v7.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v7.c
@@ -82,68 +82,54 @@ union TCP_WATCH_CNTL_BITS {
float f32All;
};
-static inline struct amdgpu_device *get_amdgpu_device(struct kgd_dev *kgd)
-{
- return (struct amdgpu_device *)kgd;
-}
-
-static void lock_srbm(struct kgd_dev *kgd, uint32_t mec, uint32_t pipe,
+static void lock_srbm(struct amdgpu_device *adev, uint32_t mec, uint32_t pipe,
uint32_t queue, uint32_t vmid)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
uint32_t value = PIPEID(pipe) | MEID(mec) | VMID(vmid) | QUEUEID(queue);
mutex_lock(&adev->srbm_mutex);
WREG32(mmSRBM_GFX_CNTL, value);
}
-static void unlock_srbm(struct kgd_dev *kgd)
+static void unlock_srbm(struct amdgpu_device *adev)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
-
WREG32(mmSRBM_GFX_CNTL, 0);
mutex_unlock(&adev->srbm_mutex);
}
-static void acquire_queue(struct kgd_dev *kgd, uint32_t pipe_id,
+static void acquire_queue(struct amdgpu_device *adev, uint32_t pipe_id,
uint32_t queue_id)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
-
uint32_t mec = (pipe_id / adev->gfx.mec.num_pipe_per_mec) + 1;
uint32_t pipe = (pipe_id % adev->gfx.mec.num_pipe_per_mec);
- lock_srbm(kgd, mec, pipe, queue_id, 0);
+ lock_srbm(adev, mec, pipe, queue_id, 0);
}
-static void release_queue(struct kgd_dev *kgd)
+static void release_queue(struct amdgpu_device *adev)
{
- unlock_srbm(kgd);
+ unlock_srbm(adev);
}
-static void kgd_program_sh_mem_settings(struct kgd_dev *kgd, uint32_t vmid,
+static void kgd_program_sh_mem_settings(struct amdgpu_device *adev, uint32_t vmid,
uint32_t sh_mem_config,
uint32_t sh_mem_ape1_base,
uint32_t sh_mem_ape1_limit,
uint32_t sh_mem_bases)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
-
- lock_srbm(kgd, 0, 0, 0, vmid);
+ lock_srbm(adev, 0, 0, 0, vmid);
WREG32(mmSH_MEM_CONFIG, sh_mem_config);
WREG32(mmSH_MEM_APE1_BASE, sh_mem_ape1_base);
WREG32(mmSH_MEM_APE1_LIMIT, sh_mem_ape1_limit);
WREG32(mmSH_MEM_BASES, sh_mem_bases);
- unlock_srbm(kgd);
+ unlock_srbm(adev);
}
-static int kgd_set_pasid_vmid_mapping(struct kgd_dev *kgd, u32 pasid,
+static int kgd_set_pasid_vmid_mapping(struct amdgpu_device *adev, u32 pasid,
unsigned int vmid)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
-
/*
* We have to assume that there is no outstanding mapping.
* The ATC_VMID_PASID_MAPPING_UPDATE_STATUS bit could be 0 because
@@ -165,21 +151,20 @@ static int kgd_set_pasid_vmid_mapping(struct kgd_dev *kgd, u32 pasid,
return 0;
}
-static int kgd_init_interrupts(struct kgd_dev *kgd, uint32_t pipe_id)
+static int kgd_init_interrupts(struct amdgpu_device *adev, uint32_t pipe_id)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
uint32_t mec;
uint32_t pipe;
mec = (pipe_id / adev->gfx.mec.num_pipe_per_mec) + 1;
pipe = (pipe_id % adev->gfx.mec.num_pipe_per_mec);
- lock_srbm(kgd, mec, pipe, 0, 0);
+ lock_srbm(adev, mec, pipe, 0, 0);
WREG32(mmCPC_INT_CNTL, CP_INT_CNTL_RING0__TIME_STAMP_INT_ENABLE_MASK |
CP_INT_CNTL_RING0__OPCODE_ERROR_INT_ENABLE_MASK);
- unlock_srbm(kgd);
+ unlock_srbm(adev);
return 0;
}
@@ -207,12 +192,11 @@ static inline struct cik_sdma_rlc_registers *get_sdma_mqd(void *mqd)
return (struct cik_sdma_rlc_registers *)mqd;
}
-static int kgd_hqd_load(struct kgd_dev *kgd, void *mqd, uint32_t pipe_id,
- uint32_t queue_id, uint32_t __user *wptr,
- uint32_t wptr_shift, uint32_t wptr_mask,
- struct mm_struct *mm)
+static int kgd_hqd_load(struct amdgpu_device *adev, void *mqd,
+ uint32_t pipe_id, uint32_t queue_id,
+ uint32_t __user *wptr, uint32_t wptr_shift,
+ uint32_t wptr_mask, struct mm_struct *mm)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
struct cik_mqd *m;
uint32_t *mqd_hqd;
uint32_t reg, wptr_val, data;
@@ -220,7 +204,7 @@ static int kgd_hqd_load(struct kgd_dev *kgd, void *mqd, uint32_t pipe_id,
m = get_mqd(mqd);
- acquire_queue(kgd, pipe_id, queue_id);
+ acquire_queue(adev, pipe_id, queue_id);
/* HQD registers extend from CP_MQD_BASE_ADDR to CP_MQD_CONTROL. */
mqd_hqd = &m->cp_mqd_base_addr_lo;
@@ -239,25 +223,24 @@ static int kgd_hqd_load(struct kgd_dev *kgd, void *mqd, uint32_t pipe_id,
* release srbm_mutex to avoid circular dependency between
* srbm_mutex->mm_sem->reservation_ww_class_mutex->srbm_mutex.
*/
- release_queue(kgd);
+ release_queue(adev);
valid_wptr = read_user_wptr(mm, wptr, wptr_val);
- acquire_queue(kgd, pipe_id, queue_id);
+ acquire_queue(adev, pipe_id, queue_id);
if (valid_wptr)
WREG32(mmCP_HQD_PQ_WPTR, (wptr_val << wptr_shift) & wptr_mask);
data = REG_SET_FIELD(m->cp_hqd_active, CP_HQD_ACTIVE, ACTIVE, 1);
WREG32(mmCP_HQD_ACTIVE, data);
- release_queue(kgd);
+ release_queue(adev);
return 0;
}
-static int kgd_hqd_dump(struct kgd_dev *kgd,
+static int kgd_hqd_dump(struct amdgpu_device *adev,
uint32_t pipe_id, uint32_t queue_id,
uint32_t (**dump)[2], uint32_t *n_regs)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
uint32_t i = 0, reg;
#define HQD_N_REGS (35+4)
#define DUMP_REG(addr) do { \
@@ -271,7 +254,7 @@ static int kgd_hqd_dump(struct kgd_dev *kgd,
if (*dump == NULL)
return -ENOMEM;
- acquire_queue(kgd, pipe_id, queue_id);
+ acquire_queue(adev, pipe_id, queue_id);
DUMP_REG(mmCOMPUTE_STATIC_THREAD_MGMT_SE0);
DUMP_REG(mmCOMPUTE_STATIC_THREAD_MGMT_SE1);
@@ -281,7 +264,7 @@ static int kgd_hqd_dump(struct kgd_dev *kgd,
for (reg = mmCP_MQD_BASE_ADDR; reg <= mmCP_MQD_CONTROL; reg++)
DUMP_REG(reg);
- release_queue(kgd);
+ release_queue(adev);
WARN_ON_ONCE(i != HQD_N_REGS);
*n_regs = i;
@@ -289,10 +272,9 @@ static int kgd_hqd_dump(struct kgd_dev *kgd,
return 0;
}
-static int kgd_hqd_sdma_load(struct kgd_dev *kgd, void *mqd,
+static int kgd_hqd_sdma_load(struct amdgpu_device *adev, void *mqd,
uint32_t __user *wptr, struct mm_struct *mm)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
struct cik_sdma_rlc_registers *m;
unsigned long end_jiffies;
uint32_t sdma_rlc_reg_offset;
@@ -345,11 +327,10 @@ static int kgd_hqd_sdma_load(struct kgd_dev *kgd, void *mqd,
return 0;
}
-static int kgd_hqd_sdma_dump(struct kgd_dev *kgd,
+static int kgd_hqd_sdma_dump(struct amdgpu_device *adev,
uint32_t engine_id, uint32_t queue_id,
uint32_t (**dump)[2], uint32_t *n_regs)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
uint32_t sdma_offset = engine_id * SDMA1_REGISTER_OFFSET +
queue_id * KFD_CIK_SDMA_QUEUE_OFFSET;
uint32_t i = 0, reg;
@@ -372,15 +353,15 @@ static int kgd_hqd_sdma_dump(struct kgd_dev *kgd,
return 0;
}
-static bool kgd_hqd_is_occupied(struct kgd_dev *kgd, uint64_t queue_address,
- uint32_t pipe_id, uint32_t queue_id)
+static bool kgd_hqd_is_occupied(struct amdgpu_device *adev,
+ uint64_t queue_address, uint32_t pipe_id,
+ uint32_t queue_id)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
uint32_t act;
bool retval = false;
uint32_t low, high;
- acquire_queue(kgd, pipe_id, queue_id);
+ acquire_queue(adev, pipe_id, queue_id);
act = RREG32(mmCP_HQD_ACTIVE);
if (act) {
low = lower_32_bits(queue_address >> 8);
@@ -390,13 +371,12 @@ static bool kgd_hqd_is_occupied(struct kgd_dev *kgd, uint64_t queue_address,
high == RREG32(mmCP_HQD_PQ_BASE_HI))
retval = true;
}
- release_queue(kgd);
+ release_queue(adev);
return retval;
}
-static bool kgd_hqd_sdma_is_occupied(struct kgd_dev *kgd, void *mqd)
+static bool kgd_hqd_sdma_is_occupied(struct amdgpu_device *adev, void *mqd)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
struct cik_sdma_rlc_registers *m;
uint32_t sdma_rlc_reg_offset;
uint32_t sdma_rlc_rb_cntl;
@@ -412,12 +392,11 @@ static bool kgd_hqd_sdma_is_occupied(struct kgd_dev *kgd, void *mqd)
return false;
}
-static int kgd_hqd_destroy(struct kgd_dev *kgd, void *mqd,
+static int kgd_hqd_destroy(struct amdgpu_device *adev, void *mqd,
enum kfd_preempt_type reset_type,
unsigned int utimeout, uint32_t pipe_id,
uint32_t queue_id)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
uint32_t temp;
enum hqd_dequeue_request_type type;
unsigned long flags, end_jiffies;
@@ -426,7 +405,7 @@ static int kgd_hqd_destroy(struct kgd_dev *kgd, void *mqd,
if (amdgpu_in_reset(adev))
return -EIO;
- acquire_queue(kgd, pipe_id, queue_id);
+ acquire_queue(adev, pipe_id, queue_id);
WREG32(mmCP_HQD_PQ_DOORBELL_CONTROL, 0);
switch (reset_type) {
@@ -504,20 +483,19 @@ static int kgd_hqd_destroy(struct kgd_dev *kgd, void *mqd,
break;
if (time_after(jiffies, end_jiffies)) {
pr_err("cp queue preemption time out\n");
- release_queue(kgd);
+ release_queue(adev);
return -ETIME;
}
usleep_range(500, 1000);
}
- release_queue(kgd);
+ release_queue(adev);
return 0;
}
-static int kgd_hqd_sdma_destroy(struct kgd_dev *kgd, void *mqd,
+static int kgd_hqd_sdma_destroy(struct amdgpu_device *adev, void *mqd,
unsigned int utimeout)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
struct cik_sdma_rlc_registers *m;
uint32_t sdma_rlc_reg_offset;
uint32_t temp;
@@ -551,9 +529,8 @@ static int kgd_hqd_sdma_destroy(struct kgd_dev *kgd, void *mqd,
return 0;
}
-static int kgd_address_watch_disable(struct kgd_dev *kgd)
+static int kgd_address_watch_disable(struct amdgpu_device *adev)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
union TCP_WATCH_CNTL_BITS cntl;
unsigned int i;
@@ -571,13 +548,12 @@ static int kgd_address_watch_disable(struct kgd_dev *kgd)
return 0;
}
-static int kgd_address_watch_execute(struct kgd_dev *kgd,
+static int kgd_address_watch_execute(struct amdgpu_device *adev,
unsigned int watch_point_id,
uint32_t cntl_val,
uint32_t addr_hi,
uint32_t addr_lo)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
union TCP_WATCH_CNTL_BITS cntl;
cntl.u32All = cntl_val;
@@ -602,11 +578,10 @@ static int kgd_address_watch_execute(struct kgd_dev *kgd,
return 0;
}
-static int kgd_wave_control_execute(struct kgd_dev *kgd,
+static int kgd_wave_control_execute(struct amdgpu_device *adev,
uint32_t gfx_index_val,
uint32_t sq_cmd)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
uint32_t data;
mutex_lock(&adev->grbm_idx_mutex);
@@ -627,18 +602,17 @@ static int kgd_wave_control_execute(struct kgd_dev *kgd,
return 0;
}
-static uint32_t kgd_address_watch_get_offset(struct kgd_dev *kgd,
+static uint32_t kgd_address_watch_get_offset(struct amdgpu_device *adev,
unsigned int watch_point_id,
unsigned int reg_offset)
{
return watchRegs[watch_point_id * ADDRESS_WATCH_REG_MAX + reg_offset];
}
-static bool get_atc_vmid_pasid_mapping_info(struct kgd_dev *kgd,
+static bool get_atc_vmid_pasid_mapping_info(struct amdgpu_device *adev,
uint8_t vmid, uint16_t *p_pasid)
{
uint32_t value;
- struct amdgpu_device *adev = (struct amdgpu_device *) kgd;
value = RREG32(mmATC_VMID0_PASID_MAPPING + vmid);
*p_pasid = value & ATC_VMID0_PASID_MAPPING__PASID_MASK;
@@ -646,21 +620,17 @@ static bool get_atc_vmid_pasid_mapping_info(struct kgd_dev *kgd,
return !!(value & ATC_VMID0_PASID_MAPPING__VALID_MASK);
}
-static void set_scratch_backing_va(struct kgd_dev *kgd,
+static void set_scratch_backing_va(struct amdgpu_device *adev,
uint64_t va, uint32_t vmid)
{
- struct amdgpu_device *adev = (struct amdgpu_device *) kgd;
-
- lock_srbm(kgd, 0, 0, 0, vmid);
+ lock_srbm(adev, 0, 0, 0, vmid);
WREG32(mmSH_HIDDEN_PRIVATE_BASE_VMID, va);
- unlock_srbm(kgd);
+ unlock_srbm(adev);
}
-static void set_vm_context_page_table_base(struct kgd_dev *kgd, uint32_t vmid,
- uint64_t page_table_base)
+static void set_vm_context_page_table_base(struct amdgpu_device *adev,
+ uint32_t vmid, uint64_t page_table_base)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
-
if (!amdgpu_amdkfd_is_kfd_vmid(adev, vmid)) {
pr_err("trying to set page table base for wrong VMID\n");
return;
@@ -676,10 +646,8 @@ static void set_vm_context_page_table_base(struct kgd_dev *kgd, uint32_t vmid,
* @vmid: vmid pointer
* read vmid from register (CIK).
*/
-static uint32_t read_vmid_from_vmfault_reg(struct kgd_dev *kgd)
+static uint32_t read_vmid_from_vmfault_reg(struct amdgpu_device *adev)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
-
uint32_t status = RREG32(mmVM_CONTEXT1_PROTECTION_FAULT_STATUS);
return REG_GET_FIELD(status, VM_CONTEXT1_PROTECTION_FAULT_STATUS, VMID);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v8.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v8.c
index 5ce0ce7..52832cd 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v8.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v8.c
@@ -39,68 +39,54 @@ enum hqd_dequeue_request_type {
RESET_WAVES
};
-static inline struct amdgpu_device *get_amdgpu_device(struct kgd_dev *kgd)
-{
- return (struct amdgpu_device *)kgd;
-}
-
-static void lock_srbm(struct kgd_dev *kgd, uint32_t mec, uint32_t pipe,
+static void lock_srbm(struct amdgpu_device *adev, uint32_t mec, uint32_t pipe,
uint32_t queue, uint32_t vmid)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
uint32_t value = PIPEID(pipe) | MEID(mec) | VMID(vmid) | QUEUEID(queue);
mutex_lock(&adev->srbm_mutex);
WREG32(mmSRBM_GFX_CNTL, value);
}
-static void unlock_srbm(struct kgd_dev *kgd)
+static void unlock_srbm(struct amdgpu_device *adev)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
-
WREG32(mmSRBM_GFX_CNTL, 0);
mutex_unlock(&adev->srbm_mutex);
}
-static void acquire_queue(struct kgd_dev *kgd, uint32_t pipe_id,
+static void acquire_queue(struct amdgpu_device *adev, uint32_t pipe_id,
uint32_t queue_id)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
-
uint32_t mec = (pipe_id / adev->gfx.mec.num_pipe_per_mec) + 1;
uint32_t pipe = (pipe_id % adev->gfx.mec.num_pipe_per_mec);
- lock_srbm(kgd, mec, pipe, queue_id, 0);
+ lock_srbm(adev, mec, pipe, queue_id, 0);
}
-static void release_queue(struct kgd_dev *kgd)
+static void release_queue(struct amdgpu_device *adev)
{
- unlock_srbm(kgd);
+ unlock_srbm(adev);
}
-static void kgd_program_sh_mem_settings(struct kgd_dev *kgd, uint32_t vmid,
+static void kgd_program_sh_mem_settings(struct amdgpu_device *adev, uint32_t vmid,
uint32_t sh_mem_config,
uint32_t sh_mem_ape1_base,
uint32_t sh_mem_ape1_limit,
uint32_t sh_mem_bases)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
-
- lock_srbm(kgd, 0, 0, 0, vmid);
+ lock_srbm(adev, 0, 0, 0, vmid);
WREG32(mmSH_MEM_CONFIG, sh_mem_config);
WREG32(mmSH_MEM_APE1_BASE, sh_mem_ape1_base);
WREG32(mmSH_MEM_APE1_LIMIT, sh_mem_ape1_limit);
WREG32(mmSH_MEM_BASES, sh_mem_bases);
- unlock_srbm(kgd);
+ unlock_srbm(adev);
}
-static int kgd_set_pasid_vmid_mapping(struct kgd_dev *kgd, u32 pasid,
+static int kgd_set_pasid_vmid_mapping(struct amdgpu_device *adev, u32 pasid,
unsigned int vmid)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
-
/*
* We have to assume that there is no outstanding mapping.
* The ATC_VMID_PASID_MAPPING_UPDATE_STATUS bit could be 0 because
@@ -123,21 +109,20 @@ static int kgd_set_pasid_vmid_mapping(struct kgd_dev *kgd, u32 pasid,
return 0;
}
-static int kgd_init_interrupts(struct kgd_dev *kgd, uint32_t pipe_id)
+static int kgd_init_interrupts(struct amdgpu_device *adev, uint32_t pipe_id)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
uint32_t mec;
uint32_t pipe;
mec = (pipe_id / adev->gfx.mec.num_pipe_per_mec) + 1;
pipe = (pipe_id % adev->gfx.mec.num_pipe_per_mec);
- lock_srbm(kgd, mec, pipe, 0, 0);
+ lock_srbm(adev, mec, pipe, 0, 0);
WREG32(mmCPC_INT_CNTL, CP_INT_CNTL_RING0__TIME_STAMP_INT_ENABLE_MASK |
CP_INT_CNTL_RING0__OPCODE_ERROR_INT_ENABLE_MASK);
- unlock_srbm(kgd);
+ unlock_srbm(adev);
return 0;
}
@@ -165,12 +150,11 @@ static inline struct vi_sdma_mqd *get_sdma_mqd(void *mqd)
return (struct vi_sdma_mqd *)mqd;
}
-static int kgd_hqd_load(struct kgd_dev *kgd, void *mqd, uint32_t pipe_id,
- uint32_t queue_id, uint32_t __user *wptr,
- uint32_t wptr_shift, uint32_t wptr_mask,
- struct mm_struct *mm)
+static int kgd_hqd_load(struct amdgpu_device *adev, void *mqd,
+ uint32_t pipe_id, uint32_t queue_id,
+ uint32_t __user *wptr, uint32_t wptr_shift,
+ uint32_t wptr_mask, struct mm_struct *mm)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
struct vi_mqd *m;
uint32_t *mqd_hqd;
uint32_t reg, wptr_val, data;
@@ -178,7 +162,7 @@ static int kgd_hqd_load(struct kgd_dev *kgd, void *mqd, uint32_t pipe_id,
m = get_mqd(mqd);
- acquire_queue(kgd, pipe_id, queue_id);
+ acquire_queue(adev, pipe_id, queue_id);
/* HIQ is set during driver init period with vmid set to 0*/
if (m->cp_hqd_vmid == 0) {
@@ -206,7 +190,7 @@ static int kgd_hqd_load(struct kgd_dev *kgd, void *mqd, uint32_t pipe_id,
* on ASICs that do not support context-save.
* EOP writes/reads can start anywhere in the ring.
*/
- if (get_amdgpu_device(kgd)->asic_type != CHIP_TONGA) {
+ if (adev->asic_type != CHIP_TONGA) {
WREG32(mmCP_HQD_EOP_RPTR, m->cp_hqd_eop_rptr);
WREG32(mmCP_HQD_EOP_WPTR, m->cp_hqd_eop_wptr);
WREG32(mmCP_HQD_EOP_WPTR_MEM, m->cp_hqd_eop_wptr_mem);
@@ -226,25 +210,24 @@ static int kgd_hqd_load(struct kgd_dev *kgd, void *mqd, uint32_t pipe_id,
* release srbm_mutex to avoid circular dependency between
* srbm_mutex->mm_sem->reservation_ww_class_mutex->srbm_mutex.
*/
- release_queue(kgd);
+ release_queue(adev);
valid_wptr = read_user_wptr(mm, wptr, wptr_val);
- acquire_queue(kgd, pipe_id, queue_id);
+ acquire_queue(adev, pipe_id, queue_id);
if (valid_wptr)
WREG32(mmCP_HQD_PQ_WPTR, (wptr_val << wptr_shift) & wptr_mask);
data = REG_SET_FIELD(m->cp_hqd_active, CP_HQD_ACTIVE, ACTIVE, 1);
WREG32(mmCP_HQD_ACTIVE, data);
- release_queue(kgd);
+ release_queue(adev);
return 0;
}
-static int kgd_hqd_dump(struct kgd_dev *kgd,
+static int kgd_hqd_dump(struct amdgpu_device *adev,
uint32_t pipe_id, uint32_t queue_id,
uint32_t (**dump)[2], uint32_t *n_regs)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
uint32_t i = 0, reg;
#define HQD_N_REGS (54+4)
#define DUMP_REG(addr) do { \
@@ -258,7 +241,7 @@ static int kgd_hqd_dump(struct kgd_dev *kgd,
if (*dump == NULL)
return -ENOMEM;
- acquire_queue(kgd, pipe_id, queue_id);
+ acquire_queue(adev, pipe_id, queue_id);
DUMP_REG(mmCOMPUTE_STATIC_THREAD_MGMT_SE0);
DUMP_REG(mmCOMPUTE_STATIC_THREAD_MGMT_SE1);
@@ -268,7 +251,7 @@ static int kgd_hqd_dump(struct kgd_dev *kgd,
for (reg = mmCP_MQD_BASE_ADDR; reg <= mmCP_HQD_EOP_DONES; reg++)
DUMP_REG(reg);
- release_queue(kgd);
+ release_queue(adev);
WARN_ON_ONCE(i != HQD_N_REGS);
*n_regs = i;
@@ -276,10 +259,9 @@ static int kgd_hqd_dump(struct kgd_dev *kgd,
return 0;
}
-static int kgd_hqd_sdma_load(struct kgd_dev *kgd, void *mqd,
+static int kgd_hqd_sdma_load(struct amdgpu_device *adev, void *mqd,
uint32_t __user *wptr, struct mm_struct *mm)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
struct vi_sdma_mqd *m;
unsigned long end_jiffies;
uint32_t sdma_rlc_reg_offset;
@@ -331,11 +313,10 @@ static int kgd_hqd_sdma_load(struct kgd_dev *kgd, void *mqd,
return 0;
}
-static int kgd_hqd_sdma_dump(struct kgd_dev *kgd,
+static int kgd_hqd_sdma_dump(struct amdgpu_device *adev,
uint32_t engine_id, uint32_t queue_id,
uint32_t (**dump)[2], uint32_t *n_regs)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
uint32_t sdma_offset = engine_id * SDMA1_REGISTER_OFFSET +
queue_id * KFD_VI_SDMA_QUEUE_OFFSET;
uint32_t i = 0, reg;
@@ -367,15 +348,15 @@ static int kgd_hqd_sdma_dump(struct kgd_dev *kgd,
return 0;
}
-static bool kgd_hqd_is_occupied(struct kgd_dev *kgd, uint64_t queue_address,
- uint32_t pipe_id, uint32_t queue_id)
+static bool kgd_hqd_is_occupied(struct amdgpu_device *adev,
+ uint64_t queue_address, uint32_t pipe_id,
+ uint32_t queue_id)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
uint32_t act;
bool retval = false;
uint32_t low, high;
- acquire_queue(kgd, pipe_id, queue_id);
+ acquire_queue(adev, pipe_id, queue_id);
act = RREG32(mmCP_HQD_ACTIVE);
if (act) {
low = lower_32_bits(queue_address >> 8);
@@ -385,13 +366,12 @@ static bool kgd_hqd_is_occupied(struct kgd_dev *kgd, uint64_t queue_address,
high == RREG32(mmCP_HQD_PQ_BASE_HI))
retval = true;
}
- release_queue(kgd);
+ release_queue(adev);
return retval;
}
-static bool kgd_hqd_sdma_is_occupied(struct kgd_dev *kgd, void *mqd)
+static bool kgd_hqd_sdma_is_occupied(struct amdgpu_device *adev, void *mqd)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
struct vi_sdma_mqd *m;
uint32_t sdma_rlc_reg_offset;
uint32_t sdma_rlc_rb_cntl;
@@ -407,12 +387,11 @@ static bool kgd_hqd_sdma_is_occupied(struct kgd_dev *kgd, void *mqd)
return false;
}
-static int kgd_hqd_destroy(struct kgd_dev *kgd, void *mqd,
+static int kgd_hqd_destroy(struct amdgpu_device *adev, void *mqd,
enum kfd_preempt_type reset_type,
unsigned int utimeout, uint32_t pipe_id,
uint32_t queue_id)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
uint32_t temp;
enum hqd_dequeue_request_type type;
unsigned long flags, end_jiffies;
@@ -422,7 +401,7 @@ static int kgd_hqd_destroy(struct kgd_dev *kgd, void *mqd,
if (amdgpu_in_reset(adev))
return -EIO;
- acquire_queue(kgd, pipe_id, queue_id);
+ acquire_queue(adev, pipe_id, queue_id);
if (m->cp_hqd_vmid == 0)
WREG32_FIELD(RLC_CP_SCHEDULERS, scheduler1, 0);
@@ -502,20 +481,19 @@ static int kgd_hqd_destroy(struct kgd_dev *kgd, void *mqd,
break;
if (time_after(jiffies, end_jiffies)) {
pr_err("cp queue preemption time out.\n");
- release_queue(kgd);
+ release_queue(adev);
return -ETIME;
}
usleep_range(500, 1000);
}
- release_queue(kgd);
+ release_queue(adev);
return 0;
}
-static int kgd_hqd_sdma_destroy(struct kgd_dev *kgd, void *mqd,
+static int kgd_hqd_sdma_destroy(struct amdgpu_device *adev, void *mqd,
unsigned int utimeout)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
struct vi_sdma_mqd *m;
uint32_t sdma_rlc_reg_offset;
uint32_t temp;
@@ -549,11 +527,10 @@ static int kgd_hqd_sdma_destroy(struct kgd_dev *kgd, void *mqd,
return 0;
}
-static bool get_atc_vmid_pasid_mapping_info(struct kgd_dev *kgd,
+static bool get_atc_vmid_pasid_mapping_info(struct amdgpu_device *adev,
uint8_t vmid, uint16_t *p_pasid)
{
uint32_t value;
- struct amdgpu_device *adev = (struct amdgpu_device *) kgd;
value = RREG32(mmATC_VMID0_PASID_MAPPING + vmid);
*p_pasid = value & ATC_VMID0_PASID_MAPPING__PASID_MASK;
@@ -561,12 +538,12 @@ static bool get_atc_vmid_pasid_mapping_info(struct kgd_dev *kgd,
return !!(value & ATC_VMID0_PASID_MAPPING__VALID_MASK);
}
-static int kgd_address_watch_disable(struct kgd_dev *kgd)
+static int kgd_address_watch_disable(struct amdgpu_device *adev)
{
return 0;
}
-static int kgd_address_watch_execute(struct kgd_dev *kgd,
+static int kgd_address_watch_execute(struct amdgpu_device *adev,
unsigned int watch_point_id,
uint32_t cntl_val,
uint32_t addr_hi,
@@ -575,11 +552,10 @@ static int kgd_address_watch_execute(struct kgd_dev *kgd,
return 0;
}
-static int kgd_wave_control_execute(struct kgd_dev *kgd,
+static int kgd_wave_control_execute(struct amdgpu_device *adev,
uint32_t gfx_index_val,
uint32_t sq_cmd)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
uint32_t data = 0;
mutex_lock(&adev->grbm_idx_mutex);
@@ -600,28 +576,24 @@ static int kgd_wave_control_execute(struct kgd_dev *kgd,
return 0;
}
-static uint32_t kgd_address_watch_get_offset(struct kgd_dev *kgd,
+static uint32_t kgd_address_watch_get_offset(struct amdgpu_device *adev,
unsigned int watch_point_id,
unsigned int reg_offset)
{
return 0;
}
-static void set_scratch_backing_va(struct kgd_dev *kgd,
+static void set_scratch_backing_va(struct amdgpu_device *adev,
uint64_t va, uint32_t vmid)
{
- struct amdgpu_device *adev = (struct amdgpu_device *) kgd;
-
- lock_srbm(kgd, 0, 0, 0, vmid);
+ lock_srbm(adev, 0, 0, 0, vmid);
WREG32(mmSH_HIDDEN_PRIVATE_BASE_VMID, va);
- unlock_srbm(kgd);
+ unlock_srbm(adev);
}
-static void set_vm_context_page_table_base(struct kgd_dev *kgd, uint32_t vmid,
- uint64_t page_table_base)
+static void set_vm_context_page_table_base(struct amdgpu_device *adev,
+ uint32_t vmid, uint64_t page_table_base)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
-
if (!amdgpu_amdkfd_is_kfd_vmid(adev, vmid)) {
pr_err("trying to set page table base for wrong VMID\n");
return;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v9.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v9.c
index bcc1cbe..1abf662 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v9.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v9.c
@@ -46,37 +46,26 @@ enum hqd_dequeue_request_type {
SAVE_WAVES
};
-static inline struct amdgpu_device *get_amdgpu_device(struct kgd_dev *kgd)
-{
- return (struct amdgpu_device *)kgd;
-}
-
-static void lock_srbm(struct kgd_dev *kgd, uint32_t mec, uint32_t pipe,
+static void lock_srbm(struct amdgpu_device *adev, uint32_t mec, uint32_t pipe,
uint32_t queue, uint32_t vmid)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
-
mutex_lock(&adev->srbm_mutex);
soc15_grbm_select(adev, mec, pipe, queue, vmid);
}
-static void unlock_srbm(struct kgd_dev *kgd)
+static void unlock_srbm(struct amdgpu_device *adev)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
-
soc15_grbm_select(adev, 0, 0, 0, 0);
mutex_unlock(&adev->srbm_mutex);
}
-static void acquire_queue(struct kgd_dev *kgd, uint32_t pipe_id,
+static void acquire_queue(struct amdgpu_device *adev, uint32_t pipe_id,
uint32_t queue_id)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
-
uint32_t mec = (pipe_id / adev->gfx.mec.num_pipe_per_mec) + 1;
uint32_t pipe = (pipe_id % adev->gfx.mec.num_pipe_per_mec);
- lock_srbm(kgd, mec, pipe, queue_id, 0);
+ lock_srbm(adev, mec, pipe, queue_id, 0);
}
static uint64_t get_queue_mask(struct amdgpu_device *adev,
@@ -88,33 +77,29 @@ static uint64_t get_queue_mask(struct amdgpu_device *adev,
return 1ull << bit;
}
-static void release_queue(struct kgd_dev *kgd)
+static void release_queue(struct amdgpu_device *adev)
{
- unlock_srbm(kgd);
+ unlock_srbm(adev);
}
-void kgd_gfx_v9_program_sh_mem_settings(struct kgd_dev *kgd, uint32_t vmid,
+void kgd_gfx_v9_program_sh_mem_settings(struct amdgpu_device *adev, uint32_t vmid,
uint32_t sh_mem_config,
uint32_t sh_mem_ape1_base,
uint32_t sh_mem_ape1_limit,
uint32_t sh_mem_bases)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
-
- lock_srbm(kgd, 0, 0, 0, vmid);
+ lock_srbm(adev, 0, 0, 0, vmid);
WREG32_RLC(SOC15_REG_OFFSET(GC, 0, mmSH_MEM_CONFIG), sh_mem_config);
WREG32_RLC(SOC15_REG_OFFSET(GC, 0, mmSH_MEM_BASES), sh_mem_bases);
/* APE1 no longer exists on GFX9 */
- unlock_srbm(kgd);
+ unlock_srbm(adev);
}
-int kgd_gfx_v9_set_pasid_vmid_mapping(struct kgd_dev *kgd, u32 pasid,
+int kgd_gfx_v9_set_pasid_vmid_mapping(struct amdgpu_device *adev, u32 pasid,
unsigned int vmid)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
-
/*
* We have to assume that there is no outstanding mapping.
* The ATC_VMID_PASID_MAPPING_UPDATE_STATUS bit could be 0 because
@@ -171,22 +156,21 @@ int kgd_gfx_v9_set_pasid_vmid_mapping(struct kgd_dev *kgd, u32 pasid,
* but still works
*/
-int kgd_gfx_v9_init_interrupts(struct kgd_dev *kgd, uint32_t pipe_id)
+int kgd_gfx_v9_init_interrupts(struct amdgpu_device *adev, uint32_t pipe_id)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
uint32_t mec;
uint32_t pipe;
mec = (pipe_id / adev->gfx.mec.num_pipe_per_mec) + 1;
pipe = (pipe_id % adev->gfx.mec.num_pipe_per_mec);
- lock_srbm(kgd, mec, pipe, 0, 0);
+ lock_srbm(adev, mec, pipe, 0, 0);
- WREG32(SOC15_REG_OFFSET(GC, 0, mmCPC_INT_CNTL),
+ WREG32_SOC15(GC, 0, mmCPC_INT_CNTL,
CP_INT_CNTL_RING0__TIME_STAMP_INT_ENABLE_MASK |
CP_INT_CNTL_RING0__OPCODE_ERROR_INT_ENABLE_MASK);
- unlock_srbm(kgd);
+ unlock_srbm(adev);
return 0;
}
@@ -233,19 +217,18 @@ static inline struct v9_sdma_mqd *get_sdma_mqd(void *mqd)
return (struct v9_sdma_mqd *)mqd;
}
-int kgd_gfx_v9_hqd_load(struct kgd_dev *kgd, void *mqd, uint32_t pipe_id,
- uint32_t queue_id, uint32_t __user *wptr,
- uint32_t wptr_shift, uint32_t wptr_mask,
- struct mm_struct *mm)
+int kgd_gfx_v9_hqd_load(struct amdgpu_device *adev, void *mqd,
+ uint32_t pipe_id, uint32_t queue_id,
+ uint32_t __user *wptr, uint32_t wptr_shift,
+ uint32_t wptr_mask, struct mm_struct *mm)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
struct v9_mqd *m;
uint32_t *mqd_hqd;
uint32_t reg, hqd_base, data;
m = get_mqd(mqd);
- acquire_queue(kgd, pipe_id, queue_id);
+ acquire_queue(adev, pipe_id, queue_id);
/* HQD registers extend from CP_MQD_BASE_ADDR to CP_HQD_EOP_WPTR_MEM. */
mqd_hqd = &m->cp_mqd_base_addr_lo;
@@ -296,7 +279,7 @@ int kgd_gfx_v9_hqd_load(struct kgd_dev *kgd, void *mqd, uint32_t pipe_id,
lower_32_bits((uintptr_t)wptr));
WREG32_RLC(SOC15_REG_OFFSET(GC, 0, mmCP_HQD_PQ_WPTR_POLL_ADDR_HI),
upper_32_bits((uintptr_t)wptr));
- WREG32(SOC15_REG_OFFSET(GC, 0, mmCP_PQ_WPTR_POLL_CNTL1),
+ WREG32_SOC15(GC, 0, mmCP_PQ_WPTR_POLL_CNTL1,
(uint32_t)get_queue_mask(adev, pipe_id, queue_id));
}
@@ -308,16 +291,15 @@ int kgd_gfx_v9_hqd_load(struct kgd_dev *kgd, void *mqd, uint32_t pipe_id,
data = REG_SET_FIELD(m->cp_hqd_active, CP_HQD_ACTIVE, ACTIVE, 1);
WREG32_RLC(SOC15_REG_OFFSET(GC, 0, mmCP_HQD_ACTIVE), data);
- release_queue(kgd);
+ release_queue(adev);
return 0;
}
-int kgd_gfx_v9_hiq_mqd_load(struct kgd_dev *kgd, void *mqd,
+int kgd_gfx_v9_hiq_mqd_load(struct amdgpu_device *adev, void *mqd,
uint32_t pipe_id, uint32_t queue_id,
uint32_t doorbell_off)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
struct amdgpu_ring *kiq_ring = &adev->gfx.kiq.ring;
struct v9_mqd *m;
uint32_t mec, pipe;
@@ -325,7 +307,7 @@ int kgd_gfx_v9_hiq_mqd_load(struct kgd_dev *kgd, void *mqd,
m = get_mqd(mqd);
- acquire_queue(kgd, pipe_id, queue_id);
+ acquire_queue(adev, pipe_id, queue_id);
mec = (pipe_id / adev->gfx.mec.num_pipe_per_mec) + 1;
pipe = (pipe_id % adev->gfx.mec.num_pipe_per_mec);
@@ -361,16 +343,15 @@ int kgd_gfx_v9_hiq_mqd_load(struct kgd_dev *kgd, void *mqd,
out_unlock:
spin_unlock(&adev->gfx.kiq.ring_lock);
- release_queue(kgd);
+ release_queue(adev);
return r;
}
-int kgd_gfx_v9_hqd_dump(struct kgd_dev *kgd,
+int kgd_gfx_v9_hqd_dump(struct amdgpu_device *adev,
uint32_t pipe_id, uint32_t queue_id,
uint32_t (**dump)[2], uint32_t *n_regs)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
uint32_t i = 0, reg;
#define HQD_N_REGS 56
#define DUMP_REG(addr) do { \
@@ -384,13 +365,13 @@ int kgd_gfx_v9_hqd_dump(struct kgd_dev *kgd,
if (*dump == NULL)
return -ENOMEM;
- acquire_queue(kgd, pipe_id, queue_id);
+ acquire_queue(adev, pipe_id, queue_id);
for (reg = SOC15_REG_OFFSET(GC, 0, mmCP_MQD_BASE_ADDR);
reg <= SOC15_REG_OFFSET(GC, 0, mmCP_HQD_PQ_WPTR_HI); reg++)
DUMP_REG(reg);
- release_queue(kgd);
+ release_queue(adev);
WARN_ON_ONCE(i != HQD_N_REGS);
*n_regs = i;
@@ -398,10 +379,9 @@ int kgd_gfx_v9_hqd_dump(struct kgd_dev *kgd,
return 0;
}
-static int kgd_hqd_sdma_load(struct kgd_dev *kgd, void *mqd,
+static int kgd_hqd_sdma_load(struct amdgpu_device *adev, void *mqd,
uint32_t __user *wptr, struct mm_struct *mm)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
struct v9_sdma_mqd *m;
uint32_t sdma_rlc_reg_offset;
unsigned long end_jiffies;
@@ -468,11 +448,10 @@ static int kgd_hqd_sdma_load(struct kgd_dev *kgd, void *mqd,
return 0;
}
-static int kgd_hqd_sdma_dump(struct kgd_dev *kgd,
+static int kgd_hqd_sdma_dump(struct amdgpu_device *adev,
uint32_t engine_id, uint32_t queue_id,
uint32_t (**dump)[2], uint32_t *n_regs)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
uint32_t sdma_rlc_reg_offset = get_sdma_rlc_reg_offset(adev,
engine_id, queue_id);
uint32_t i = 0, reg;
@@ -500,31 +479,30 @@ static int kgd_hqd_sdma_dump(struct kgd_dev *kgd,
return 0;
}
-bool kgd_gfx_v9_hqd_is_occupied(struct kgd_dev *kgd, uint64_t queue_address,
- uint32_t pipe_id, uint32_t queue_id)
+bool kgd_gfx_v9_hqd_is_occupied(struct amdgpu_device *adev,
+ uint64_t queue_address, uint32_t pipe_id,
+ uint32_t queue_id)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
uint32_t act;
bool retval = false;
uint32_t low, high;
- acquire_queue(kgd, pipe_id, queue_id);
- act = RREG32(SOC15_REG_OFFSET(GC, 0, mmCP_HQD_ACTIVE));
+ acquire_queue(adev, pipe_id, queue_id);
+ act = RREG32_SOC15(GC, 0, mmCP_HQD_ACTIVE);
if (act) {
low = lower_32_bits(queue_address >> 8);
high = upper_32_bits(queue_address >> 8);
- if (low == RREG32(SOC15_REG_OFFSET(GC, 0, mmCP_HQD_PQ_BASE)) &&
- high == RREG32(SOC15_REG_OFFSET(GC, 0, mmCP_HQD_PQ_BASE_HI)))
+ if (low == RREG32_SOC15(GC, 0, mmCP_HQD_PQ_BASE) &&
+ high == RREG32_SOC15(GC, 0, mmCP_HQD_PQ_BASE_HI))
retval = true;
}
- release_queue(kgd);
+ release_queue(adev);
return retval;
}
-static bool kgd_hqd_sdma_is_occupied(struct kgd_dev *kgd, void *mqd)
+static bool kgd_hqd_sdma_is_occupied(struct amdgpu_device *adev, void *mqd)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
struct v9_sdma_mqd *m;
uint32_t sdma_rlc_reg_offset;
uint32_t sdma_rlc_rb_cntl;
@@ -541,12 +519,11 @@ static bool kgd_hqd_sdma_is_occupied(struct kgd_dev *kgd, void *mqd)
return false;
}
-int kgd_gfx_v9_hqd_destroy(struct kgd_dev *kgd, void *mqd,
+int kgd_gfx_v9_hqd_destroy(struct amdgpu_device *adev, void *mqd,
enum kfd_preempt_type reset_type,
unsigned int utimeout, uint32_t pipe_id,
uint32_t queue_id)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
enum hqd_dequeue_request_type type;
unsigned long end_jiffies;
uint32_t temp;
@@ -555,7 +532,7 @@ int kgd_gfx_v9_hqd_destroy(struct kgd_dev *kgd, void *mqd,
if (amdgpu_in_reset(adev))
return -EIO;
- acquire_queue(kgd, pipe_id, queue_id);
+ acquire_queue(adev, pipe_id, queue_id);
if (m->cp_hqd_vmid == 0)
WREG32_FIELD15_RLC(GC, 0, RLC_CP_SCHEDULERS, scheduler1, 0);
@@ -579,25 +556,24 @@ int kgd_gfx_v9_hqd_destroy(struct kgd_dev *kgd, void *mqd,
end_jiffies = (utimeout * HZ / 1000) + jiffies;
while (true) {
- temp = RREG32(SOC15_REG_OFFSET(GC, 0, mmCP_HQD_ACTIVE));
+ temp = RREG32_SOC15(GC, 0, mmCP_HQD_ACTIVE);
if (!(temp & CP_HQD_ACTIVE__ACTIVE_MASK))
break;
if (time_after(jiffies, end_jiffies)) {
pr_err("cp queue preemption time out.\n");
- release_queue(kgd);
+ release_queue(adev);
return -ETIME;
}
usleep_range(500, 1000);
}
- release_queue(kgd);
+ release_queue(adev);
return 0;
}
-static int kgd_hqd_sdma_destroy(struct kgd_dev *kgd, void *mqd,
+static int kgd_hqd_sdma_destroy(struct amdgpu_device *adev, void *mqd,
unsigned int utimeout)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
struct v9_sdma_mqd *m;
uint32_t sdma_rlc_reg_offset;
uint32_t temp;
@@ -634,11 +610,10 @@ static int kgd_hqd_sdma_destroy(struct kgd_dev *kgd, void *mqd,
return 0;
}
-bool kgd_gfx_v9_get_atc_vmid_pasid_mapping_info(struct kgd_dev *kgd,
+bool kgd_gfx_v9_get_atc_vmid_pasid_mapping_info(struct amdgpu_device *adev,
uint8_t vmid, uint16_t *p_pasid)
{
uint32_t value;
- struct amdgpu_device *adev = (struct amdgpu_device *) kgd;
value = RREG32(SOC15_REG_OFFSET(ATHUB, 0, mmATC_VMID0_PASID_MAPPING)
+ vmid);
@@ -647,12 +622,12 @@ bool kgd_gfx_v9_get_atc_vmid_pasid_mapping_info(struct kgd_dev *kgd,
return !!(value & ATC_VMID0_PASID_MAPPING__VALID_MASK);
}
-int kgd_gfx_v9_address_watch_disable(struct kgd_dev *kgd)
+int kgd_gfx_v9_address_watch_disable(struct amdgpu_device *adev)
{
return 0;
}
-int kgd_gfx_v9_address_watch_execute(struct kgd_dev *kgd,
+int kgd_gfx_v9_address_watch_execute(struct amdgpu_device *adev,
unsigned int watch_point_id,
uint32_t cntl_val,
uint32_t addr_hi,
@@ -661,17 +636,16 @@ int kgd_gfx_v9_address_watch_execute(struct kgd_dev *kgd,
return 0;
}
-int kgd_gfx_v9_wave_control_execute(struct kgd_dev *kgd,
+int kgd_gfx_v9_wave_control_execute(struct amdgpu_device *adev,
uint32_t gfx_index_val,
uint32_t sq_cmd)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
uint32_t data = 0;
mutex_lock(&adev->grbm_idx_mutex);
WREG32_SOC15_RLC_SHADOW(GC, 0, mmGRBM_GFX_INDEX, gfx_index_val);
- WREG32(SOC15_REG_OFFSET(GC, 0, mmSQ_CMD), sq_cmd);
+ WREG32_SOC15(GC, 0, mmSQ_CMD, sq_cmd);
data = REG_SET_FIELD(data, GRBM_GFX_INDEX,
INSTANCE_BROADCAST_WRITES, 1);
@@ -686,18 +660,16 @@ int kgd_gfx_v9_wave_control_execute(struct kgd_dev *kgd,
return 0;
}
-uint32_t kgd_gfx_v9_address_watch_get_offset(struct kgd_dev *kgd,
+uint32_t kgd_gfx_v9_address_watch_get_offset(struct amdgpu_device *adev,
unsigned int watch_point_id,
unsigned int reg_offset)
{
return 0;
}
-void kgd_gfx_v9_set_vm_context_page_table_base(struct kgd_dev *kgd,
+void kgd_gfx_v9_set_vm_context_page_table_base(struct amdgpu_device *adev,
uint32_t vmid, uint64_t page_table_base)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
-
if (!amdgpu_amdkfd_is_kfd_vmid(adev, vmid)) {
pr_err("trying to set page table base for wrong VMID %u\n",
vmid);
@@ -750,7 +722,7 @@ static void get_wave_count(struct amdgpu_device *adev, int queue_idx,
pipe_idx = queue_idx / adev->gfx.mec.num_queue_per_pipe;
queue_slot = queue_idx % adev->gfx.mec.num_queue_per_pipe;
soc15_grbm_select(adev, 1, pipe_idx, queue_slot, 0);
- reg_val = RREG32(SOC15_REG_OFFSET(GC, 0, mmSPI_CSQ_WF_ACTIVE_COUNT_0) +
+ reg_val = RREG32_SOC15_IP(GC, SOC15_REG_OFFSET(GC, 0, mmSPI_CSQ_WF_ACTIVE_COUNT_0) +
queue_slot);
*wave_cnt = reg_val & SPI_CSQ_WF_ACTIVE_COUNT_0__COUNT_MASK;
if (*wave_cnt != 0)
@@ -804,7 +776,7 @@ static void get_wave_count(struct amdgpu_device *adev, int queue_idx,
*
* Reading registers referenced above involves programming GRBM appropriately
*/
-void kgd_gfx_v9_get_cu_occupancy(struct kgd_dev *kgd, int pasid,
+void kgd_gfx_v9_get_cu_occupancy(struct amdgpu_device *adev, int pasid,
int *pasid_wave_cnt, int *max_waves_per_cu)
{
int qidx;
@@ -818,10 +790,8 @@ void kgd_gfx_v9_get_cu_occupancy(struct kgd_dev *kgd, int pasid,
int pasid_tmp;
int max_queue_cnt;
int vmid_wave_cnt = 0;
- struct amdgpu_device *adev;
DECLARE_BITMAP(cp_queue_bitmap, KGD_MAX_QUEUES);
- adev = get_amdgpu_device(kgd);
lock_spi_csq_mutexes(adev);
soc15_grbm_select(adev, 1, 0, 0, 0);
@@ -839,8 +809,7 @@ void kgd_gfx_v9_get_cu_occupancy(struct kgd_dev *kgd, int pasid,
for (sh_idx = 0; sh_idx < sh_cnt; sh_idx++) {
gfx_v9_0_select_se_sh(adev, se_idx, sh_idx, 0xffffffff);
- queue_map = RREG32(SOC15_REG_OFFSET(GC, 0,
- mmSPI_CSQ_WF_ACTIVE_STATUS));
+ queue_map = RREG32_SOC15(GC, 0, mmSPI_CSQ_WF_ACTIVE_STATUS);
/*
* Assumption: queue map encodes following schema: four
@@ -882,30 +851,28 @@ void kgd_gfx_v9_get_cu_occupancy(struct kgd_dev *kgd, int pasid,
adev->gfx.cu_info.max_waves_per_simd;
}
-void kgd_gfx_v9_program_trap_handler_settings(struct kgd_dev *kgd,
+void kgd_gfx_v9_program_trap_handler_settings(struct amdgpu_device *adev,
uint32_t vmid, uint64_t tba_addr, uint64_t tma_addr)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
-
- lock_srbm(kgd, 0, 0, 0, vmid);
+ lock_srbm(adev, 0, 0, 0, vmid);
/*
* Program TBA registers
*/
- WREG32(SOC15_REG_OFFSET(GC, 0, mmSQ_SHADER_TBA_LO),
+ WREG32_SOC15(GC, 0, mmSQ_SHADER_TBA_LO,
lower_32_bits(tba_addr >> 8));
- WREG32(SOC15_REG_OFFSET(GC, 0, mmSQ_SHADER_TBA_HI),
+ WREG32_SOC15(GC, 0, mmSQ_SHADER_TBA_HI,
upper_32_bits(tba_addr >> 8));
/*
* Program TMA registers
*/
- WREG32(SOC15_REG_OFFSET(GC, 0, mmSQ_SHADER_TMA_LO),
+ WREG32_SOC15(GC, 0, mmSQ_SHADER_TMA_LO,
lower_32_bits(tma_addr >> 8));
- WREG32(SOC15_REG_OFFSET(GC, 0, mmSQ_SHADER_TMA_HI),
+ WREG32_SOC15(GC, 0, mmSQ_SHADER_TMA_HI,
upper_32_bits(tma_addr >> 8));
- unlock_srbm(kgd);
+ unlock_srbm(adev);
}
const struct kfd2kgd_calls gfx_v9_kfd2kgd = {
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v9.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v9.h
index c635911..24be49d 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v9.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v9.h
@@ -22,48 +22,49 @@
-void kgd_gfx_v9_program_sh_mem_settings(struct kgd_dev *kgd, uint32_t vmid,
+void kgd_gfx_v9_program_sh_mem_settings(struct amdgpu_device *adev, uint32_t vmid,
uint32_t sh_mem_config,
uint32_t sh_mem_ape1_base, uint32_t sh_mem_ape1_limit,
uint32_t sh_mem_bases);
-int kgd_gfx_v9_set_pasid_vmid_mapping(struct kgd_dev *kgd, u32 pasid,
+int kgd_gfx_v9_set_pasid_vmid_mapping(struct amdgpu_device *adev, u32 pasid,
unsigned int vmid);
-int kgd_gfx_v9_init_interrupts(struct kgd_dev *kgd, uint32_t pipe_id);
-int kgd_gfx_v9_hqd_load(struct kgd_dev *kgd, void *mqd, uint32_t pipe_id,
+int kgd_gfx_v9_init_interrupts(struct amdgpu_device *adev, uint32_t pipe_id);
+int kgd_gfx_v9_hqd_load(struct amdgpu_device *adev, void *mqd, uint32_t pipe_id,
uint32_t queue_id, uint32_t __user *wptr,
uint32_t wptr_shift, uint32_t wptr_mask,
struct mm_struct *mm);
-int kgd_gfx_v9_hiq_mqd_load(struct kgd_dev *kgd, void *mqd,
+int kgd_gfx_v9_hiq_mqd_load(struct amdgpu_device *adev, void *mqd,
uint32_t pipe_id, uint32_t queue_id,
uint32_t doorbell_off);
-int kgd_gfx_v9_hqd_dump(struct kgd_dev *kgd,
+int kgd_gfx_v9_hqd_dump(struct amdgpu_device *adev,
uint32_t pipe_id, uint32_t queue_id,
uint32_t (**dump)[2], uint32_t *n_regs);
-bool kgd_gfx_v9_hqd_is_occupied(struct kgd_dev *kgd, uint64_t queue_address,
- uint32_t pipe_id, uint32_t queue_id);
-int kgd_gfx_v9_hqd_destroy(struct kgd_dev *kgd, void *mqd,
+bool kgd_gfx_v9_hqd_is_occupied(struct amdgpu_device *adev,
+ uint64_t queue_address, uint32_t pipe_id,
+ uint32_t queue_id);
+int kgd_gfx_v9_hqd_destroy(struct amdgpu_device *adev, void *mqd,
enum kfd_preempt_type reset_type,
unsigned int utimeout, uint32_t pipe_id,
uint32_t queue_id);
-int kgd_gfx_v9_address_watch_disable(struct kgd_dev *kgd);
-int kgd_gfx_v9_address_watch_execute(struct kgd_dev *kgd,
+int kgd_gfx_v9_address_watch_disable(struct amdgpu_device *adev);
+int kgd_gfx_v9_address_watch_execute(struct amdgpu_device *adev,
unsigned int watch_point_id,
uint32_t cntl_val,
uint32_t addr_hi,
uint32_t addr_lo);
-int kgd_gfx_v9_wave_control_execute(struct kgd_dev *kgd,
+int kgd_gfx_v9_wave_control_execute(struct amdgpu_device *adev,
uint32_t gfx_index_val,
uint32_t sq_cmd);
-uint32_t kgd_gfx_v9_address_watch_get_offset(struct kgd_dev *kgd,
+uint32_t kgd_gfx_v9_address_watch_get_offset(struct amdgpu_device *adev,
unsigned int watch_point_id,
unsigned int reg_offset);
-bool kgd_gfx_v9_get_atc_vmid_pasid_mapping_info(struct kgd_dev *kgd,
+bool kgd_gfx_v9_get_atc_vmid_pasid_mapping_info(struct amdgpu_device *adev,
uint8_t vmid, uint16_t *p_pasid);
-void kgd_gfx_v9_set_vm_context_page_table_base(struct kgd_dev *kgd,
+void kgd_gfx_v9_set_vm_context_page_table_base(struct amdgpu_device *adev,
uint32_t vmid, uint64_t page_table_base);
-void kgd_gfx_v9_get_cu_occupancy(struct kgd_dev *kgd, int pasid,
+void kgd_gfx_v9_get_cu_occupancy(struct amdgpu_device *adev, int pasid,
int *pasid_wave_cnt, int *max_waves_per_cu);
-void kgd_gfx_v9_program_trap_handler_settings(struct kgd_dev *kgd,
+void kgd_gfx_v9_program_trap_handler_settings(struct amdgpu_device *adev,
uint32_t vmid, uint64_t tba_addr, uint64_t tma_addr);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
index 6348559..f9bab96 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
@@ -60,12 +60,6 @@ static const char * const domain_bit_to_string[] = {
static void amdgpu_amdkfd_restore_userptr_worker(struct work_struct *work);
-
-static inline struct amdgpu_device *get_amdgpu_device(struct kgd_dev *kgd)
-{
- return (struct amdgpu_device *)kgd;
-}
-
static bool kfd_mem_is_attached(struct amdgpu_vm *avm,
struct kgd_mem *mem)
{
@@ -126,8 +120,19 @@ static size_t amdgpu_amdkfd_acc_size(uint64_t size)
PAGE_ALIGN(size);
}
+/**
+ * @amdgpu_amdkfd_reserve_mem_limit() - Decrease available memory by size
+ * of buffer including any reserved for control structures
+ *
+ * @adev: Device to which allocated BO belongs to
+ * @size: Size of buffer, in bytes, encapsulated by B0. This should be
+ * equivalent to amdgpu_bo_size(BO)
+ * @alloc_flag: Flag used in allocating a BO as noted above
+ *
+ * Return: returns -ENOMEM in case of error, ZERO otherwise
+ */
static int amdgpu_amdkfd_reserve_mem_limit(struct amdgpu_device *adev,
- uint64_t size, u32 domain, bool sg)
+ uint64_t size, u32 alloc_flag)
{
uint64_t reserved_for_pt =
ESTIMATE_PT_SIZE(amdgpu_amdkfd_total_mem_size);
@@ -137,20 +142,24 @@ static int amdgpu_amdkfd_reserve_mem_limit(struct amdgpu_device *adev,
acc_size = amdgpu_amdkfd_acc_size(size);
vram_needed = 0;
- if (domain == AMDGPU_GEM_DOMAIN_GTT) {
- /* TTM GTT memory */
+ if (alloc_flag & KFD_IOC_ALLOC_MEM_FLAGS_GTT) {
system_mem_needed = acc_size + size;
ttm_mem_needed = acc_size + size;
- } else if (domain == AMDGPU_GEM_DOMAIN_CPU && !sg) {
- /* Userptr */
- system_mem_needed = acc_size + size;
- ttm_mem_needed = acc_size;
- } else {
- /* VRAM and SG */
+ } else if (alloc_flag & KFD_IOC_ALLOC_MEM_FLAGS_VRAM) {
system_mem_needed = acc_size;
ttm_mem_needed = acc_size;
- if (domain == AMDGPU_GEM_DOMAIN_VRAM)
- vram_needed = size;
+ vram_needed = size;
+ } else if (alloc_flag & KFD_IOC_ALLOC_MEM_FLAGS_USERPTR) {
+ system_mem_needed = acc_size + size;
+ ttm_mem_needed = acc_size;
+ } else if (alloc_flag &
+ (KFD_IOC_ALLOC_MEM_FLAGS_DOORBELL |
+ KFD_IOC_ALLOC_MEM_FLAGS_MMIO_REMAP)) {
+ system_mem_needed = acc_size;
+ ttm_mem_needed = acc_size;
+ } else {
+ pr_err("%s: Invalid BO type %#x\n", __func__, alloc_flag);
+ return -ENOMEM;
}
spin_lock(&kfd_mem_limit.mem_limit_lock);
@@ -166,64 +175,72 @@ static int amdgpu_amdkfd_reserve_mem_limit(struct amdgpu_device *adev,
(adev->kfd.vram_used + vram_needed >
adev->gmc.real_vram_size - reserved_for_pt)) {
ret = -ENOMEM;
- } else {
- kfd_mem_limit.system_mem_used += system_mem_needed;
- kfd_mem_limit.ttm_mem_used += ttm_mem_needed;
- adev->kfd.vram_used += vram_needed;
+ goto release;
}
+ /* Update memory accounting by decreasing available system
+ * memory, TTM memory and GPU memory as computed above
+ */
+ adev->kfd.vram_used += vram_needed;
+ kfd_mem_limit.system_mem_used += system_mem_needed;
+ kfd_mem_limit.ttm_mem_used += ttm_mem_needed;
+
+release:
spin_unlock(&kfd_mem_limit.mem_limit_lock);
return ret;
}
static void unreserve_mem_limit(struct amdgpu_device *adev,
- uint64_t size, u32 domain, bool sg)
+ uint64_t size, u32 alloc_flag)
{
size_t acc_size;
acc_size = amdgpu_amdkfd_acc_size(size);
spin_lock(&kfd_mem_limit.mem_limit_lock);
- if (domain == AMDGPU_GEM_DOMAIN_GTT) {
+
+ if (alloc_flag & KFD_IOC_ALLOC_MEM_FLAGS_GTT) {
kfd_mem_limit.system_mem_used -= (acc_size + size);
kfd_mem_limit.ttm_mem_used -= (acc_size + size);
- } else if (domain == AMDGPU_GEM_DOMAIN_CPU && !sg) {
- kfd_mem_limit.system_mem_used -= (acc_size + size);
- kfd_mem_limit.ttm_mem_used -= acc_size;
- } else {
+ } else if (alloc_flag & KFD_IOC_ALLOC_MEM_FLAGS_VRAM) {
kfd_mem_limit.system_mem_used -= acc_size;
kfd_mem_limit.ttm_mem_used -= acc_size;
- if (domain == AMDGPU_GEM_DOMAIN_VRAM) {
- adev->kfd.vram_used -= size;
- WARN_ONCE(adev->kfd.vram_used < 0,
- "kfd VRAM memory accounting unbalanced");
- }
+ adev->kfd.vram_used -= size;
+ } else if (alloc_flag & KFD_IOC_ALLOC_MEM_FLAGS_USERPTR) {
+ kfd_mem_limit.system_mem_used -= (acc_size + size);
+ kfd_mem_limit.ttm_mem_used -= acc_size;
+ } else if (alloc_flag &
+ (KFD_IOC_ALLOC_MEM_FLAGS_DOORBELL |
+ KFD_IOC_ALLOC_MEM_FLAGS_MMIO_REMAP)) {
+ kfd_mem_limit.system_mem_used -= acc_size;
+ kfd_mem_limit.ttm_mem_used -= acc_size;
+ } else {
+ pr_err("%s: Invalid BO type %#x\n", __func__, alloc_flag);
+ goto release;
}
- WARN_ONCE(kfd_mem_limit.system_mem_used < 0,
- "kfd system memory accounting unbalanced");
- WARN_ONCE(kfd_mem_limit.ttm_mem_used < 0,
- "kfd TTM memory accounting unbalanced");
+ WARN_ONCE(adev->kfd.vram_used < 0,
+ "KFD VRAM memory accounting unbalanced");
+ WARN_ONCE(kfd_mem_limit.ttm_mem_used < 0,
+ "KFD TTM memory accounting unbalanced");
+ WARN_ONCE(kfd_mem_limit.system_mem_used < 0,
+ "KFD system memory accounting unbalanced");
+
+release:
spin_unlock(&kfd_mem_limit.mem_limit_lock);
}
void amdgpu_amdkfd_release_notify(struct amdgpu_bo *bo)
{
struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev);
- u32 domain = bo->preferred_domains;
- bool sg = (bo->preferred_domains == AMDGPU_GEM_DOMAIN_CPU);
+ u32 alloc_flags = bo->kfd_bo->alloc_flags;
+ u64 size = amdgpu_bo_size(bo);
- if (bo->flags & AMDGPU_AMDKFD_CREATE_USERPTR_BO) {
- domain = AMDGPU_GEM_DOMAIN_CPU;
- sg = false;
- }
-
- unreserve_mem_limit(adev, amdgpu_bo_size(bo), domain, sg);
+ unreserve_mem_limit(adev, size, alloc_flags);
kfree(bo->kfd_bo);
}
-
/* amdgpu_amdkfd_remove_eviction_fence - Removes eviction fence from BO's
* reservation object.
*
@@ -691,10 +708,12 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem,
pr_debug("\t add VA 0x%llx - 0x%llx to vm %p\n", va,
va + bo_size, vm);
- if (adev == bo_adev || (mem->domain == AMDGPU_GEM_DOMAIN_VRAM &&
- amdgpu_xgmi_same_hive(adev, bo_adev))) {
- /* Mappings on the local GPU and VRAM mappings in the
- * local hive share the original BO
+ if (adev == bo_adev ||
+ (amdgpu_ttm_tt_get_usermm(mem->bo->tbo.ttm) && adev->ram_is_direct_mapped) ||
+ (mem->domain == AMDGPU_GEM_DOMAIN_VRAM && amdgpu_xgmi_same_hive(adev, bo_adev))) {
+ /* Mappings on the local GPU, or VRAM mappings in the
+ * local hive, or userptr mapping IOMMU direct map mode
+ * share the original BO
*/
attachment[i]->type = KFD_MEM_ATT_SHARED;
bo[i] = mem->bo;
@@ -1272,12 +1291,60 @@ static int init_kfd_vm(struct amdgpu_vm *vm, void **process_info,
return ret;
}
-int amdgpu_amdkfd_gpuvm_acquire_process_vm(struct kgd_dev *kgd,
+/**
+ * amdgpu_amdkfd_gpuvm_pin_bo() - Pins a BO using following criteria
+ * @bo: Handle of buffer object being pinned
+ * @domain: Domain into which BO should be pinned
+ *
+ * - USERPTR BOs are UNPINNABLE and will return error
+ * - All other BO types (GTT, VRAM, MMIO and DOORBELL) will have their
+ * PIN count incremented. It is valid to PIN a BO multiple times
+ *
+ * Return: ZERO if successful in pinning, Non-Zero in case of error.
+ */
+static int amdgpu_amdkfd_gpuvm_pin_bo(struct amdgpu_bo *bo, u32 domain)
+{
+ int ret = 0;
+
+ ret = amdgpu_bo_reserve(bo, false);
+ if (unlikely(ret))
+ return ret;
+
+ ret = amdgpu_bo_pin_restricted(bo, domain, 0, 0);
+ if (ret)
+ pr_err("Error in Pinning BO to domain: %d\n", domain);
+
+ amdgpu_bo_sync_wait(bo, AMDGPU_FENCE_OWNER_KFD, false);
+ amdgpu_bo_unreserve(bo);
+
+ return ret;
+}
+
+/**
+ * amdgpu_amdkfd_gpuvm_unpin_bo() - Unpins BO using following criteria
+ * @bo: Handle of buffer object being unpinned
+ *
+ * - Is a illegal request for USERPTR BOs and is ignored
+ * - All other BO types (GTT, VRAM, MMIO and DOORBELL) will have their
+ * PIN count decremented. Calls to UNPIN must balance calls to PIN
+ */
+static void amdgpu_amdkfd_gpuvm_unpin_bo(struct amdgpu_bo *bo)
+{
+ int ret = 0;
+
+ ret = amdgpu_bo_reserve(bo, false);
+ if (unlikely(ret))
+ return;
+
+ amdgpu_bo_unpin(bo);
+ amdgpu_bo_unreserve(bo);
+}
+
+int amdgpu_amdkfd_gpuvm_acquire_process_vm(struct amdgpu_device *adev,
struct file *filp, u32 pasid,
void **process_info,
struct dma_fence **ef)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
struct amdgpu_fpriv *drv_priv;
struct amdgpu_vm *avm;
int ret;
@@ -1353,12 +1420,12 @@ void amdgpu_amdkfd_gpuvm_destroy_cb(struct amdgpu_device *adev,
}
}
-void amdgpu_amdkfd_gpuvm_release_process_vm(struct kgd_dev *kgd, void *drm_priv)
+void amdgpu_amdkfd_gpuvm_release_process_vm(struct amdgpu_device *adev,
+ void *drm_priv)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
struct amdgpu_vm *avm;
- if (WARN_ON(!kgd || !drm_priv))
+ if (WARN_ON(!adev || !drm_priv))
return;
avm = drm_priv_to_vm(drm_priv);
@@ -1386,11 +1453,10 @@ uint64_t amdgpu_amdkfd_gpuvm_get_process_page_dir(void *drm_priv)
}
int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu(
- struct kgd_dev *kgd, uint64_t va, uint64_t size,
+ struct amdgpu_device *adev, uint64_t va, uint64_t size,
void *drm_priv, struct kgd_mem **mem,
uint64_t *offset, uint32_t flags)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
struct amdgpu_vm *avm = drm_priv_to_vm(drm_priv);
enum ttm_bo_type bo_type = ttm_bo_type_device;
struct sg_table *sg = NULL;
@@ -1454,7 +1520,7 @@ int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu(
amdgpu_sync_create(&(*mem)->sync);
- ret = amdgpu_amdkfd_reserve_mem_limit(adev, size, alloc_domain, !!sg);
+ ret = amdgpu_amdkfd_reserve_mem_limit(adev, size, flags);
if (ret) {
pr_debug("Insufficient memory\n");
goto err_reserve_limit;
@@ -1495,6 +1561,15 @@ int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu(
ret = init_user_pages(*mem, user_addr);
if (ret)
goto allocate_init_user_pages_failed;
+ } else if (flags & (KFD_IOC_ALLOC_MEM_FLAGS_DOORBELL |
+ KFD_IOC_ALLOC_MEM_FLAGS_MMIO_REMAP)) {
+ ret = amdgpu_amdkfd_gpuvm_pin_bo(bo, AMDGPU_GEM_DOMAIN_GTT);
+ if (ret) {
+ pr_err("Pinning MMIO/DOORBELL BO during ALLOC FAILED\n");
+ goto err_pin_bo;
+ }
+ bo->allowed_domains = AMDGPU_GEM_DOMAIN_GTT;
+ bo->preferred_domains = AMDGPU_GEM_DOMAIN_GTT;
}
if (offset)
@@ -1503,13 +1578,14 @@ int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu(
return 0;
allocate_init_user_pages_failed:
+err_pin_bo:
remove_kgd_mem_from_kfd_bo_list(*mem, avm->process_info);
drm_vma_node_revoke(&gobj->vma_node, drm_priv);
err_node_allow:
/* Don't unreserve system mem limit twice */
goto err_reserve_limit;
err_bo_create:
- unreserve_mem_limit(adev, size, alloc_domain, !!sg);
+ unreserve_mem_limit(adev, size, flags);
err_reserve_limit:
mutex_destroy(&(*mem)->lock);
if (gobj)
@@ -1525,7 +1601,7 @@ int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu(
}
int amdgpu_amdkfd_gpuvm_free_memory_of_gpu(
- struct kgd_dev *kgd, struct kgd_mem *mem, void *drm_priv,
+ struct amdgpu_device *adev, struct kgd_mem *mem, void *drm_priv,
uint64_t *size)
{
struct amdkfd_process_info *process_info = mem->process_info;
@@ -1538,6 +1614,14 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu(
bool is_imported = false;
mutex_lock(&mem->lock);
+
+ /* Unpin MMIO/DOORBELL BO's that were pinnned during allocation */
+ if (mem->alloc_flags &
+ (KFD_IOC_ALLOC_MEM_FLAGS_DOORBELL |
+ KFD_IOC_ALLOC_MEM_FLAGS_MMIO_REMAP)) {
+ amdgpu_amdkfd_gpuvm_unpin_bo(mem->bo);
+ }
+
mapped_to_gpu_memory = mem->mapped_to_gpu_memory;
is_imported = mem->is_imported;
mutex_unlock(&mem->lock);
@@ -1617,10 +1701,9 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu(
}
int amdgpu_amdkfd_gpuvm_map_memory_to_gpu(
- struct kgd_dev *kgd, struct kgd_mem *mem,
+ struct amdgpu_device *adev, struct kgd_mem *mem,
void *drm_priv, bool *table_freed)
{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
struct amdgpu_vm *avm = drm_priv_to_vm(drm_priv);
int ret;
struct amdgpu_bo *bo;
@@ -1747,7 +1830,7 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu(
}
int amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu(
- struct kgd_dev *kgd, struct kgd_mem *mem, void *drm_priv)
+ struct amdgpu_device *adev, struct kgd_mem *mem, void *drm_priv)
{
struct amdgpu_vm *avm = drm_priv_to_vm(drm_priv);
struct amdkfd_process_info *process_info = avm->process_info;
@@ -1808,7 +1891,7 @@ int amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu(
}
int amdgpu_amdkfd_gpuvm_sync_memory(
- struct kgd_dev *kgd, struct kgd_mem *mem, bool intr)
+ struct amdgpu_device *adev, struct kgd_mem *mem, bool intr)
{
struct amdgpu_sync sync;
int ret;
@@ -1824,7 +1907,7 @@ int amdgpu_amdkfd_gpuvm_sync_memory(
return ret;
}
-int amdgpu_amdkfd_gpuvm_map_gtt_bo_to_kernel(struct kgd_dev *kgd,
+int amdgpu_amdkfd_gpuvm_map_gtt_bo_to_kernel(struct amdgpu_device *adev,
struct kgd_mem *mem, void **kptr, uint64_t *size)
{
int ret;
@@ -1880,7 +1963,8 @@ int amdgpu_amdkfd_gpuvm_map_gtt_bo_to_kernel(struct kgd_dev *kgd,
return ret;
}
-void amdgpu_amdkfd_gpuvm_unmap_gtt_bo_from_kernel(struct kgd_dev *kgd, struct kgd_mem *mem)
+void amdgpu_amdkfd_gpuvm_unmap_gtt_bo_from_kernel(struct amdgpu_device *adev,
+ struct kgd_mem *mem)
{
struct amdgpu_bo *bo = mem->bo;
@@ -1890,12 +1974,9 @@ void amdgpu_amdkfd_gpuvm_unmap_gtt_bo_from_kernel(struct kgd_dev *kgd, struct kg
amdgpu_bo_unreserve(bo);
}
-int amdgpu_amdkfd_gpuvm_get_vm_fault_info(struct kgd_dev *kgd,
- struct kfd_vm_fault_info *mem)
+int amdgpu_amdkfd_gpuvm_get_vm_fault_info(struct amdgpu_device *adev,
+ struct kfd_vm_fault_info *mem)
{
- struct amdgpu_device *adev;
-
- adev = (struct amdgpu_device *)kgd;
if (atomic_read(&adev->gmc.vm_fault_info_updated) == 1) {
*mem = *adev->gmc.vm_fault_info;
mb();
@@ -1904,13 +1985,12 @@ int amdgpu_amdkfd_gpuvm_get_vm_fault_info(struct kgd_dev *kgd,
return 0;
}
-int amdgpu_amdkfd_gpuvm_import_dmabuf(struct kgd_dev *kgd,
+int amdgpu_amdkfd_gpuvm_import_dmabuf(struct amdgpu_device *adev,
struct dma_buf *dma_buf,
uint64_t va, void *drm_priv,
struct kgd_mem **mem, uint64_t *size,
uint64_t *mmap_offset)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)kgd;
struct amdgpu_vm *avm = drm_priv_to_vm(drm_priv);
struct drm_gem_object *obj;
struct amdgpu_bo *bo;
@@ -2537,11 +2617,9 @@ int amdgpu_amdkfd_remove_gws_from_process(void *info, void *mem)
}
/* Returns GPU-specific tiling mode information */
-int amdgpu_amdkfd_get_tile_config(struct kgd_dev *kgd,
+int amdgpu_amdkfd_get_tile_config(struct amdgpu_device *adev,
struct tile_config *config)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)kgd;
-
config->gb_addr_config = adev->gfx.config.gb_addr_config;
config->tile_config_ptr = adev->gfx.config.tile_mode_array;
config->num_tile_configs =
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_atomfirmware.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_atomfirmware.c
index 97178b307..4d4ddf0 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_atomfirmware.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_atomfirmware.c
@@ -470,8 +470,8 @@ bool amdgpu_atomfirmware_dynamic_boot_config_supported(struct amdgpu_device *ade
/**
* amdgpu_atomfirmware_ras_rom_addr -- Get the RAS EEPROM addr from VBIOS
- * adev: amdgpu_device pointer
- * i2c_address: pointer to u8; if not NULL, will contain
+ * @adev: amdgpu_device pointer
+ * @i2c_address: pointer to u8; if not NULL, will contain
* the RAS EEPROM address if the function returns true
*
* Return true if VBIOS supports RAS EEPROM address reporting,
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_atpx_handler.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_atpx_handler.c
index 7abe950..d6d986b 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_atpx_handler.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_atpx_handler.c
@@ -11,6 +11,7 @@
#include <linux/pci.h>
#include <linux/delay.h>
+#include "amdgpu.h"
#include "amd_acpi.h"
#define AMDGPU_PX_QUIRK_FORCE_ATPX (1 << 0)
@@ -165,7 +166,7 @@ static void amdgpu_atpx_parse_functions(struct amdgpu_atpx_functions *f, u32 mas
}
/**
- * amdgpu_atpx_validate_functions - validate ATPX functions
+ * amdgpu_atpx_validate - validate ATPX functions
*
* @atpx: amdgpu atpx struct
*
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_connectors.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_connectors.c
index 0de66f5..c16a270 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_connectors.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_connectors.c
@@ -108,7 +108,7 @@ int amdgpu_connector_get_monitor_bpc(struct drm_connector *connector)
case DRM_MODE_CONNECTOR_DVII:
case DRM_MODE_CONNECTOR_HDMIB:
if (amdgpu_connector->use_digital) {
- if (drm_detect_hdmi_monitor(amdgpu_connector_edid(connector))) {
+ if (connector->display_info.is_hdmi) {
if (connector->display_info.bpc)
bpc = connector->display_info.bpc;
}
@@ -116,7 +116,7 @@ int amdgpu_connector_get_monitor_bpc(struct drm_connector *connector)
break;
case DRM_MODE_CONNECTOR_DVID:
case DRM_MODE_CONNECTOR_HDMIA:
- if (drm_detect_hdmi_monitor(amdgpu_connector_edid(connector))) {
+ if (connector->display_info.is_hdmi) {
if (connector->display_info.bpc)
bpc = connector->display_info.bpc;
}
@@ -125,7 +125,7 @@ int amdgpu_connector_get_monitor_bpc(struct drm_connector *connector)
dig_connector = amdgpu_connector->con_priv;
if ((dig_connector->dp_sink_type == CONNECTOR_OBJECT_ID_DISPLAYPORT) ||
(dig_connector->dp_sink_type == CONNECTOR_OBJECT_ID_eDP) ||
- drm_detect_hdmi_monitor(amdgpu_connector_edid(connector))) {
+ connector->display_info.is_hdmi) {
if (connector->display_info.bpc)
bpc = connector->display_info.bpc;
}
@@ -149,7 +149,7 @@ int amdgpu_connector_get_monitor_bpc(struct drm_connector *connector)
break;
}
- if (drm_detect_hdmi_monitor(amdgpu_connector_edid(connector))) {
+ if (connector->display_info.is_hdmi) {
/*
* Pre DCE-8 hw can't handle > 12 bpc, and more than 12 bpc doesn't make
* much sense without support for > 12 bpc framebuffers. RGB 4:4:4 at
@@ -315,8 +315,10 @@ static void amdgpu_connector_get_edid(struct drm_connector *connector)
if (!amdgpu_connector->edid) {
/* some laptops provide a hardcoded edid in rom for LCDs */
if (((connector->connector_type == DRM_MODE_CONNECTOR_LVDS) ||
- (connector->connector_type == DRM_MODE_CONNECTOR_eDP)))
+ (connector->connector_type == DRM_MODE_CONNECTOR_eDP))) {
amdgpu_connector->edid = amdgpu_connector_get_hardcoded_edid(adev);
+ drm_connector_update_edid_property(connector, amdgpu_connector->edid);
+ }
}
}
@@ -326,6 +328,7 @@ static void amdgpu_connector_free_edid(struct drm_connector *connector)
kfree(amdgpu_connector->edid);
amdgpu_connector->edid = NULL;
+ drm_connector_update_edid_property(connector, NULL);
}
static int amdgpu_connector_ddc_get_modes(struct drm_connector *connector)
@@ -387,6 +390,9 @@ amdgpu_connector_lcd_native_mode(struct drm_encoder *encoder)
native_mode->vdisplay != 0 &&
native_mode->clock != 0) {
mode = drm_mode_duplicate(dev, native_mode);
+ if (!mode)
+ return NULL;
+
mode->type = DRM_MODE_TYPE_PREFERRED | DRM_MODE_TYPE_DRIVER;
drm_mode_set_name(mode);
@@ -401,6 +407,9 @@ amdgpu_connector_lcd_native_mode(struct drm_encoder *encoder)
* simpler.
*/
mode = drm_cvt_mode(dev, native_mode->hdisplay, native_mode->vdisplay, 60, true, false, false);
+ if (!mode)
+ return NULL;
+
mode->type = DRM_MODE_TYPE_PREFERRED | DRM_MODE_TYPE_DRIVER;
DRM_DEBUG_KMS("Adding cvt approximation of native panel mode %s\n", mode->name);
}
@@ -1171,7 +1180,7 @@ static enum drm_mode_status amdgpu_connector_dvi_mode_valid(struct drm_connector
(amdgpu_connector->connector_object_id == CONNECTOR_OBJECT_ID_DUAL_LINK_DVI_D) ||
(amdgpu_connector->connector_object_id == CONNECTOR_OBJECT_ID_HDMI_TYPE_B)) {
return MODE_OK;
- } else if (drm_detect_hdmi_monitor(amdgpu_connector_edid(connector))) {
+ } else if (connector->display_info.is_hdmi) {
/* HDMI 1.3+ supports max clock of 340 Mhz */
if (mode->clock > 340000)
return MODE_CLOCK_HIGH;
@@ -1463,7 +1472,7 @@ static enum drm_mode_status amdgpu_connector_dp_mode_valid(struct drm_connector
(amdgpu_dig_connector->dp_sink_type == CONNECTOR_OBJECT_ID_eDP)) {
return amdgpu_atombios_dp_mode_valid_helper(connector, mode);
} else {
- if (drm_detect_hdmi_monitor(amdgpu_connector_edid(connector))) {
+ if (connector->display_info.is_hdmi) {
/* HDMI 1.3+ supports max clock of 340 Mhz */
if (mode->clock > 340000)
return MODE_CLOCK_HIGH;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c
index 164d6a9..25e2e5b 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c
@@ -1618,6 +1618,9 @@ int amdgpu_debugfs_init(struct amdgpu_device *adev)
if (!debugfs_initialized())
return 0;
+ debugfs_create_x32("amdgpu_smu_debug", 0600, root,
+ &adev->pm.smu_debug_mask);
+
ent = debugfs_create_file("amdgpu_preempt_ib", 0600, root, adev,
&fops_ib_preempt);
if (IS_ERR(ent)) {
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index 1e651b9..a8b08a7 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -30,6 +30,7 @@
#include <linux/module.h>
#include <linux/console.h>
#include <linux/slab.h>
+#include <linux/iommu.h>
#include <drm/drm_atomic_helper.h>
#include <drm/drm_probe_helper.h>
@@ -331,7 +332,7 @@ void amdgpu_device_mm_access(struct amdgpu_device *adev, loff_t pos,
}
/**
- * amdgpu_device_vram_access - access vram by vram aperature
+ * amdgpu_device_aper_access - access vram by vram aperature
*
* @adev: amdgpu_device pointer
* @pos: offset of the buffer in vram
@@ -550,11 +551,11 @@ void amdgpu_device_wreg(struct amdgpu_device *adev,
trace_amdgpu_device_wreg(adev->pdev->device, reg, v);
}
-/*
+/**
* amdgpu_mm_wreg_mmio_rlc - write register either with mmio or with RLC path if in range
*
* this function is invoked only the debugfs register access
- * */
+ */
void amdgpu_mm_wreg_mmio_rlc(struct amdgpu_device *adev,
uint32_t reg, uint32_t v)
{
@@ -1100,7 +1101,7 @@ static void amdgpu_device_wb_fini(struct amdgpu_device *adev)
}
/**
- * amdgpu_device_wb_init- Init Writeback driver info and allocate memory
+ * amdgpu_device_wb_init - Init Writeback driver info and allocate memory
*
* @adev: amdgpu_device pointer
*
@@ -2316,6 +2317,10 @@ static int amdgpu_device_ip_init(struct amdgpu_device *adev)
/* need to do gmc hw init early so we can allocate gpu mem */
if (adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_GMC) {
+ /* Try to reserve bad pages early */
+ if (amdgpu_sriov_vf(adev))
+ amdgpu_virt_exchange_data(adev);
+
r = amdgpu_device_vram_scratch_init(adev);
if (r) {
DRM_ERROR("amdgpu_vram_scratch_init failed %d\n", r);
@@ -2347,7 +2352,7 @@ static int amdgpu_device_ip_init(struct amdgpu_device *adev)
}
if (amdgpu_sriov_vf(adev))
- amdgpu_virt_init_data_exchange(adev);
+ amdgpu_virt_exchange_data(adev);
r = amdgpu_ib_pool_init(adev);
if (r) {
@@ -2614,11 +2619,10 @@ static int amdgpu_device_ip_late_init(struct amdgpu_device *adev)
if (r)
DRM_ERROR("enable mgpu fan boost failed (%d).\n", r);
- /* For XGMI + passthrough configuration on arcturus, enable light SBR */
- if (adev->asic_type == CHIP_ARCTURUS &&
- amdgpu_passthrough(adev) &&
- adev->gmc.xgmi.num_physical_nodes > 1)
- smu_set_light_sbr(&adev->smu, true);
+ /* For passthrough configuration on arcturus and aldebaran, enable special handling SBR */
+ if (amdgpu_passthrough(adev) && ((adev->asic_type == CHIP_ARCTURUS && adev->gmc.xgmi.num_physical_nodes > 1)||
+ adev->asic_type == CHIP_ALDEBARAN ))
+ smu_handle_passthrough_sbr(&adev->smu, true);
if (adev->gmc.xgmi.num_physical_nodes > 1) {
mutex_lock(&mgpu_info.mutex);
@@ -2657,6 +2661,36 @@ static int amdgpu_device_ip_late_init(struct amdgpu_device *adev)
return 0;
}
+/**
+ * amdgpu_device_smu_fini_early - smu hw_fini wrapper
+ *
+ * @adev: amdgpu_device pointer
+ *
+ * For ASICs need to disable SMC first
+ */
+static void amdgpu_device_smu_fini_early(struct amdgpu_device *adev)
+{
+ int i, r;
+
+ if (adev->ip_versions[GC_HWIP][0] > IP_VERSION(9, 0, 0))
+ return;
+
+ for (i = 0; i < adev->num_ip_blocks; i++) {
+ if (!adev->ip_blocks[i].status.hw)
+ continue;
+ if (adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_SMC) {
+ r = adev->ip_blocks[i].version->funcs->hw_fini((void *)adev);
+ /* XXX handle errors */
+ if (r) {
+ DRM_DEBUG("hw_fini of IP block <%s> failed %d\n",
+ adev->ip_blocks[i].version->funcs->name, r);
+ }
+ adev->ip_blocks[i].status.hw = false;
+ break;
+ }
+ }
+}
+
static int amdgpu_device_ip_fini_early(struct amdgpu_device *adev)
{
int i, r;
@@ -2677,21 +2711,8 @@ static int amdgpu_device_ip_fini_early(struct amdgpu_device *adev)
amdgpu_device_set_pg_state(adev, AMD_PG_STATE_UNGATE);
amdgpu_device_set_cg_state(adev, AMD_CG_STATE_UNGATE);
- /* need to disable SMC first */
- for (i = 0; i < adev->num_ip_blocks; i++) {
- if (!adev->ip_blocks[i].status.hw)
- continue;
- if (adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_SMC) {
- r = adev->ip_blocks[i].version->funcs->hw_fini((void *)adev);
- /* XXX handle errors */
- if (r) {
- DRM_DEBUG("hw_fini of IP block <%s> failed %d\n",
- adev->ip_blocks[i].version->funcs->name, r);
- }
- adev->ip_blocks[i].status.hw = false;
- break;
- }
- }
+ /* Workaroud for ASICs need to disable SMC first */
+ amdgpu_device_smu_fini_early(adev);
for (i = adev->num_ip_blocks - 1; i >= 0; i--) {
if (!adev->ip_blocks[i].status.hw)
@@ -2733,8 +2754,6 @@ static int amdgpu_device_ip_fini(struct amdgpu_device *adev)
if (amdgpu_sriov_vf(adev) && adev->virt.ras_init_done)
amdgpu_virt_release_ras_err_handler_data(adev);
- amdgpu_ras_pre_fini(adev);
-
if (adev->gmc.xgmi.num_physical_nodes > 1)
amdgpu_xgmi_remove_device(adev);
@@ -3166,6 +3185,12 @@ static void amdgpu_device_detect_sriov_bios(struct amdgpu_device *adev)
bool amdgpu_device_asic_has_dc_support(enum amd_asic_type asic_type)
{
switch (asic_type) {
+#ifdef CONFIG_DRM_AMDGPU_SI
+ case CHIP_HAINAN:
+#endif
+ case CHIP_TOPAZ:
+ /* chips with no display hardware */
+ return false;
#if defined(CONFIG_DRM_AMD_DC)
case CHIP_TAHITI:
case CHIP_PITCAIRN:
@@ -3367,6 +3392,22 @@ static int amdgpu_device_get_job_timeout_settings(struct amdgpu_device *adev)
return ret;
}
+/**
+ * amdgpu_device_check_iommu_direct_map - check if RAM direct mapped to GPU
+ *
+ * @adev: amdgpu_device pointer
+ *
+ * RAM direct mapped to GPU if IOMMU is not enabled or is pass through mode
+ */
+static void amdgpu_device_check_iommu_direct_map(struct amdgpu_device *adev)
+{
+ struct iommu_domain *domain;
+
+ domain = iommu_get_domain_for_dev(adev->dev);
+ if (!domain || domain->type == IOMMU_DOMAIN_IDENTITY)
+ adev->ram_is_direct_mapped = true;
+}
+
static const struct attribute *amdgpu_dev_attributes[] = {
&dev_attr_product_name.attr,
&dev_attr_product_number.attr,
@@ -3541,6 +3582,13 @@ int amdgpu_device_init(struct amdgpu_device *adev,
if (r)
return r;
+ /* Need to get xgmi info early to decide the reset behavior*/
+ if (adev->gmc.xgmi.supported) {
+ r = adev->gfxhub.funcs->get_xgmi_info(adev);
+ if (r)
+ return r;
+ }
+
/* enable PCIE atomic ops */
if (amdgpu_sriov_vf(adev))
adev->have_atomics_support = ((struct amd_sriov_msg_pf2vf_info *)
@@ -3687,8 +3735,6 @@ int amdgpu_device_init(struct amdgpu_device *adev,
/* Get a log2 for easy divisions. */
adev->mm_stats.log2_max_MBps = ilog2(max(1u, max_MBps));
- amdgpu_fbdev_init(adev);
-
r = amdgpu_pm_sysfs_init(adev);
if (r) {
adev->pm_sysfs_en = false;
@@ -3772,6 +3818,8 @@ int amdgpu_device_init(struct amdgpu_device *adev,
queue_delayed_work(system_wq, &mgpu_info.delayed_reset_work,
msecs_to_jiffies(AMDGPU_RESUME_MS));
+ amdgpu_device_check_iommu_direct_map(adev);
+
return 0;
release_ras_con:
@@ -3805,7 +3853,7 @@ static void amdgpu_device_unmap_mmio(struct amdgpu_device *adev)
}
/**
- * amdgpu_device_fini - tear down the driver
+ * amdgpu_device_fini_hw - tear down the driver
*
* @adev: amdgpu_device pointer
*
@@ -3846,17 +3894,21 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev)
amdgpu_ucode_sysfs_fini(adev);
sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes);
- amdgpu_fbdev_fini(adev);
+ /* disable ras feature must before hw fini */
+ amdgpu_ras_pre_fini(adev);
amdgpu_device_ip_fini_early(adev);
amdgpu_irq_fini_hw(adev);
- ttm_device_clear_dma_mappings(&adev->mman.bdev);
+ if (adev->mman.initialized)
+ ttm_device_clear_dma_mappings(&adev->mman.bdev);
amdgpu_gart_dummy_page_fini(adev);
- amdgpu_device_unmap_mmio(adev);
+ if (drm_dev_is_unplugged(adev_to_drm(adev)))
+ amdgpu_device_unmap_mmio(adev);
+
}
void amdgpu_device_fini_sw(struct amdgpu_device *adev)
@@ -3942,7 +3994,7 @@ int amdgpu_device_suspend(struct drm_device *dev, bool fbcon)
drm_kms_helper_poll_disable(dev);
if (fbcon)
- amdgpu_fbdev_set_suspend(adev, 1);
+ drm_fb_helper_set_suspend_unlocked(adev_to_drm(adev)->fb_helper, true);
cancel_delayed_work_sync(&adev->delayed_init_work);
@@ -4019,7 +4071,7 @@ int amdgpu_device_resume(struct drm_device *dev, bool fbcon)
flush_delayed_work(&adev->delayed_init_work);
if (fbcon)
- amdgpu_fbdev_set_suspend(adev, 0);
+ drm_fb_helper_set_suspend_unlocked(adev_to_drm(adev)->fb_helper, false);
drm_kms_helper_poll_enable(dev);
@@ -4288,6 +4340,9 @@ static int amdgpu_device_reset_sriov(struct amdgpu_device *adev,
bool from_hypervisor)
{
int r;
+ struct amdgpu_hive_info *hive = NULL;
+
+ amdgpu_amdkfd_pre_reset(adev);
amdgpu_amdkfd_pre_reset(adev);
@@ -4316,9 +4371,19 @@ static int amdgpu_device_reset_sriov(struct amdgpu_device *adev,
if (r)
goto error;
- amdgpu_irq_gpu_reset_resume_helper(adev);
- r = amdgpu_ib_ring_tests(adev);
- amdgpu_amdkfd_post_reset(adev);
+ hive = amdgpu_get_xgmi_hive(adev);
+ /* Update PSP FW topology after reset */
+ if (hive && adev->gmc.xgmi.num_physical_nodes > 1)
+ r = amdgpu_xgmi_update_topology(hive, adev);
+
+ if (hive)
+ amdgpu_put_xgmi_hive(hive);
+
+ if (!r) {
+ amdgpu_irq_gpu_reset_resume_helper(adev);
+ r = amdgpu_ib_ring_tests(adev);
+ amdgpu_amdkfd_post_reset(adev);
+ }
error:
if (!r && adev->virt.gim_feature & AMDGIM_FEATURE_GIM_FLR_VRAMLOST) {
@@ -4461,7 +4526,7 @@ int amdgpu_device_mode1_reset(struct amdgpu_device *adev)
int amdgpu_device_pre_asic_reset(struct amdgpu_device *adev,
struct amdgpu_reset_context *reset_context)
{
- int i, j, r = 0;
+ int i, r = 0;
struct amdgpu_job *job = NULL;
bool need_full_reset =
test_bit(AMDGPU_NEED_FULL_RESET, &reset_context->flags);
@@ -4483,15 +4548,8 @@ int amdgpu_device_pre_asic_reset(struct amdgpu_device *adev,
/*clear job fence from fence drv to avoid force_completion
*leave NULL and vm flush fence in fence drv */
- for (j = 0; j <= ring->fence_drv.num_fences_mask; j++) {
- struct dma_fence *old, **ptr;
+ amdgpu_fence_driver_clear_job_fences(ring);
- ptr = &ring->fence_drv.fences[j];
- old = rcu_dereference_protected(*ptr, 1);
- if (old && test_bit(AMDGPU_FENCE_FLAG_EMBED_IN_JOB_BIT, &old->flags)) {
- RCU_INIT_POINTER(*ptr, NULL);
- }
- }
/* after all hw jobs are reset, hw fence is meaningless, so force_completion */
amdgpu_fence_driver_force_completion(ring);
}
@@ -4651,7 +4709,7 @@ int amdgpu_do_asic_reset(struct list_head *device_list_handle,
if (r)
goto out;
- amdgpu_fbdev_set_suspend(tmp_adev, 0);
+ drm_fb_helper_set_suspend_unlocked(adev_to_drm(tmp_adev)->fb_helper, false);
/*
* The GPU enters bad state once faulty pages
@@ -4750,7 +4808,7 @@ static int amdgpu_device_lock_hive_adev(struct amdgpu_device *adev, struct amdgp
{
struct amdgpu_device *tmp_adev = NULL;
- if (adev->gmc.xgmi.num_physical_nodes > 1) {
+ if (!amdgpu_sriov_vf(adev) && (adev->gmc.xgmi.num_physical_nodes > 1)) {
if (!hive) {
dev_err(adev->dev, "Hive is NULL while device has multiple xgmi nodes");
return -ENODEV;
@@ -4962,7 +5020,8 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev,
* We always reset all schedulers for device and all devices for XGMI
* hive so that should take care of them too.
*/
- hive = amdgpu_get_xgmi_hive(adev);
+ if (!amdgpu_sriov_vf(adev))
+ hive = amdgpu_get_xgmi_hive(adev);
if (hive) {
if (atomic_cmpxchg(&hive->in_reset, 0, 1) != 0) {
DRM_INFO("Bailing on TDR for s_job:%llx, hive: %llx as another already in progress",
@@ -5003,7 +5062,7 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev,
* to put adev in the 1st position.
*/
INIT_LIST_HEAD(&device_list);
- if (adev->gmc.xgmi.num_physical_nodes > 1) {
+ if (!amdgpu_sriov_vf(adev) && (adev->gmc.xgmi.num_physical_nodes > 1)) {
list_for_each_entry(tmp_adev, &hive->device_list, gmc.xgmi.head)
list_add_tail(&tmp_adev->reset_list, &device_list);
if (!list_is_first(&adev->reset_list, &device_list))
@@ -5042,7 +5101,7 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev,
*/
amdgpu_unregister_gpu_instance(tmp_adev);
- amdgpu_fbdev_set_suspend(tmp_adev, 1);
+ drm_fb_helper_set_suspend_unlocked(adev_to_drm(adev)->fb_helper, true);
/* disable ras on ALL IPs */
if (!need_emergency_restart &&
@@ -5637,3 +5696,42 @@ void amdgpu_device_invalidate_hdp(struct amdgpu_device *adev,
amdgpu_asic_invalidate_hdp(adev, ring);
}
+
+/**
+ * amdgpu_device_halt() - bring hardware to some kind of halt state
+ *
+ * @adev: amdgpu_device pointer
+ *
+ * Bring hardware to some kind of halt state so that no one can touch it
+ * any more. It will help to maintain error context when error occurred.
+ * Compare to a simple hang, the system will keep stable at least for SSH
+ * access. Then it should be trivial to inspect the hardware state and
+ * see what's going on. Implemented as following:
+ *
+ * 1. drm_dev_unplug() makes device inaccessible to user space(IOCTLs, etc),
+ * clears all CPU mappings to device, disallows remappings through page faults
+ * 2. amdgpu_irq_disable_all() disables all interrupts
+ * 3. amdgpu_fence_driver_hw_fini() signals all HW fences
+ * 4. set adev->no_hw_access to avoid potential crashes after setp 5
+ * 5. amdgpu_device_unmap_mmio() clears all MMIO mappings
+ * 6. pci_disable_device() and pci_wait_for_pending_transaction()
+ * flush any in flight DMA operations
+ */
+void amdgpu_device_halt(struct amdgpu_device *adev)
+{
+ struct pci_dev *pdev = adev->pdev;
+ struct drm_device *ddev = adev_to_drm(adev);
+
+ drm_dev_unplug(ddev);
+
+ amdgpu_irq_disable_all(adev);
+
+ amdgpu_fence_driver_hw_fini(adev);
+
+ adev->no_hw_access = true;
+
+ amdgpu_device_unmap_mmio(adev);
+
+ pci_disable_device(pdev);
+ pci_wait_for_pending_transaction(pdev);
+}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.c
index ea00090..028190d 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.c
@@ -67,7 +67,8 @@
#include "smuio_v11_0_6.h"
#include "smuio_v13_0.h"
-MODULE_FIRMWARE("amdgpu/ip_discovery.bin");
+#define FIRMWARE_IP_DISCOVERY "amdgpu/ip_discovery.bin"
+MODULE_FIRMWARE(FIRMWARE_IP_DISCOVERY);
#define mmRCC_CONFIG_MEMSIZE 0xde3
#define mmMM_INDEX 0x0
@@ -179,7 +180,7 @@ static int hw_id_map[MAX_HWIP] = {
[DCI_HWIP] = DCI_HWID,
};
-static int amdgpu_discovery_read_binary(struct amdgpu_device *adev, uint8_t *binary)
+static int amdgpu_discovery_read_binary_from_vram(struct amdgpu_device *adev, uint8_t *binary)
{
uint64_t vram_size = (uint64_t)RREG32(mmRCC_CONFIG_MEMSIZE) << 20;
uint64_t pos = vram_size - DISCOVERY_TMR_OFFSET;
@@ -189,6 +190,34 @@ static int amdgpu_discovery_read_binary(struct amdgpu_device *adev, uint8_t *bin
return 0;
}
+static int amdgpu_discovery_read_binary_from_file(struct amdgpu_device *adev, uint8_t *binary)
+{
+ const struct firmware *fw;
+ const char *fw_name;
+ int r;
+
+ switch (amdgpu_discovery) {
+ case 2:
+ fw_name = FIRMWARE_IP_DISCOVERY;
+ break;
+ default:
+ dev_warn(adev->dev, "amdgpu_discovery is not set properly\n");
+ return -EINVAL;
+ }
+
+ r = request_firmware(&fw, fw_name, adev->dev);
+ if (r) {
+ dev_err(adev->dev, "can't load firmware \"%s\"\n",
+ fw_name);
+ return r;
+ }
+
+ memcpy((u8 *)binary, (u8 *)fw->data, adev->mman.discovery_tmr_size);
+ release_firmware(fw);
+
+ return 0;
+}
+
static uint16_t amdgpu_discovery_calculate_checksum(uint8_t *data, uint32_t size)
{
uint16_t checksum = 0;
@@ -206,13 +235,20 @@ static inline bool amdgpu_discovery_verify_checksum(uint8_t *data, uint32_t size
return !!(amdgpu_discovery_calculate_checksum(data, size) == expected);
}
+static inline bool amdgpu_discovery_verify_binary_signature(uint8_t *binary)
+{
+ struct binary_header *bhdr;
+ bhdr = (struct binary_header *)binary;
+
+ return (le32_to_cpu(bhdr->binary_signature) == BINARY_SIGNATURE);
+}
+
static int amdgpu_discovery_init(struct amdgpu_device *adev)
{
struct table_info *info;
struct binary_header *bhdr;
struct ip_discovery_header *ihdr;
struct gpu_info_header *ghdr;
- const struct firmware *fw;
uint16_t offset;
uint16_t size;
uint16_t checksum;
@@ -223,31 +259,32 @@ static int amdgpu_discovery_init(struct amdgpu_device *adev)
if (!adev->mman.discovery_bin)
return -ENOMEM;
- if (amdgpu_discovery == 2) {
- r = request_firmware(&fw, "amdgpu/ip_discovery.bin", adev->dev);
- if (r)
- goto get_from_vram;
- dev_info(adev->dev, "Using IP discovery from file\n");
- memcpy((u8 *)adev->mman.discovery_bin, (u8 *)fw->data,
- adev->mman.discovery_tmr_size);
- release_firmware(fw);
- } else {
-get_from_vram:
- r = amdgpu_discovery_read_binary(adev, adev->mman.discovery_bin);
+ r = amdgpu_discovery_read_binary_from_vram(adev, adev->mman.discovery_bin);
+ if (r) {
+ dev_err(adev->dev, "failed to read ip discovery binary from vram\n");
+ r = -EINVAL;
+ goto out;
+ }
+
+ if(!amdgpu_discovery_verify_binary_signature(adev->mman.discovery_bin)) {
+ dev_warn(adev->dev, "get invalid ip discovery binary signature from vram\n");
+ /* retry read ip discovery binary from file */
+ r = amdgpu_discovery_read_binary_from_file(adev, adev->mman.discovery_bin);
if (r) {
- DRM_ERROR("failed to read ip discovery binary\n");
+ dev_err(adev->dev, "failed to read ip discovery binary from file\n");
+ r = -EINVAL;
+ goto out;
+ }
+ /* check the ip discovery binary signature */
+ if(!amdgpu_discovery_verify_binary_signature(adev->mman.discovery_bin)) {
+ dev_warn(adev->dev, "get invalid ip discovery binary signature from file\n");
+ r = -EINVAL;
goto out;
}
}
bhdr = (struct binary_header *)adev->mman.discovery_bin;
- if (le32_to_cpu(bhdr->binary_signature) != BINARY_SIGNATURE) {
- DRM_ERROR("invalid ip discovery binary signature\n");
- r = -EINVAL;
- goto out;
- }
-
offset = offsetof(struct binary_header, binary_checksum) +
sizeof(bhdr->binary_checksum);
size = le16_to_cpu(bhdr->binary_size) - offset;
@@ -255,7 +292,7 @@ static int amdgpu_discovery_init(struct amdgpu_device *adev)
if (!amdgpu_discovery_verify_checksum(adev->mman.discovery_bin + offset,
size, checksum)) {
- DRM_ERROR("invalid ip discovery binary checksum\n");
+ dev_err(adev->dev, "invalid ip discovery binary checksum\n");
r = -EINVAL;
goto out;
}
@@ -266,14 +303,14 @@ static int amdgpu_discovery_init(struct amdgpu_device *adev)
ihdr = (struct ip_discovery_header *)(adev->mman.discovery_bin + offset);
if (le32_to_cpu(ihdr->signature) != DISCOVERY_TABLE_SIGNATURE) {
- DRM_ERROR("invalid ip discovery data table signature\n");
+ dev_err(adev->dev, "invalid ip discovery data table signature\n");
r = -EINVAL;
goto out;
}
if (!amdgpu_discovery_verify_checksum(adev->mman.discovery_bin + offset,
le16_to_cpu(ihdr->size), checksum)) {
- DRM_ERROR("invalid ip discovery data table checksum\n");
+ dev_err(adev->dev, "invalid ip discovery data table checksum\n");
r = -EINVAL;
goto out;
}
@@ -285,7 +322,7 @@ static int amdgpu_discovery_init(struct amdgpu_device *adev)
if (!amdgpu_discovery_verify_checksum(adev->mman.discovery_bin + offset,
le32_to_cpu(ghdr->size), checksum)) {
- DRM_ERROR("invalid gc data table checksum\n");
+ dev_err(adev->dev, "invalid gc data table checksum\n");
r = -EINVAL;
goto out;
}
@@ -379,8 +416,18 @@ int amdgpu_discovery_reg_base_init(struct amdgpu_device *adev)
ip->major, ip->minor,
ip->revision);
- if (le16_to_cpu(ip->hw_id) == VCN_HWID)
+ if (le16_to_cpu(ip->hw_id) == VCN_HWID) {
+ /* Bit [5:0]: original revision value
+ * Bit [7:6]: en/decode capability:
+ * 0b00 : VCN function normally
+ * 0b10 : encode is disabled
+ * 0b01 : decode is disabled
+ */
+ adev->vcn.vcn_config[adev->vcn.num_vcn_inst] =
+ ip->revision & 0xc0;
+ ip->revision &= ~0xc0;
adev->vcn.num_vcn_inst++;
+ }
if (le16_to_cpu(ip->hw_id) == SDMA0_HWID ||
le16_to_cpu(ip->hw_id) == SDMA1_HWID ||
le16_to_cpu(ip->hw_id) == SDMA2_HWID ||
@@ -472,14 +519,6 @@ int amdgpu_discovery_get_ip_version(struct amdgpu_device *adev, int hw_id, int n
return -EINVAL;
}
-
-int amdgpu_discovery_get_vcn_version(struct amdgpu_device *adev, int vcn_instance,
- int *major, int *minor, int *revision)
-{
- return amdgpu_discovery_get_ip_version(adev, VCN_HWID,
- vcn_instance, major, minor, revision);
-}
-
void amdgpu_discovery_harvest_ip(struct amdgpu_device *adev)
{
struct binary_header *bhdr;
@@ -526,10 +565,15 @@ void amdgpu_discovery_harvest_ip(struct amdgpu_device *adev)
}
}
+union gc_info {
+ struct gc_info_v1_0 v1;
+ struct gc_info_v2_0 v2;
+};
+
int amdgpu_discovery_get_gfx_info(struct amdgpu_device *adev)
{
struct binary_header *bhdr;
- struct gc_info_v1_0 *gc_info;
+ union gc_info *gc_info;
if (!adev->mman.discovery_bin) {
DRM_ERROR("ip discovery uninitialized\n");
@@ -537,28 +581,55 @@ int amdgpu_discovery_get_gfx_info(struct amdgpu_device *adev)
}
bhdr = (struct binary_header *)adev->mman.discovery_bin;
- gc_info = (struct gc_info_v1_0 *)(adev->mman.discovery_bin +
+ gc_info = (union gc_info *)(adev->mman.discovery_bin +
le16_to_cpu(bhdr->table_list[GC].offset));
-
- adev->gfx.config.max_shader_engines = le32_to_cpu(gc_info->gc_num_se);
- adev->gfx.config.max_cu_per_sh = 2 * (le32_to_cpu(gc_info->gc_num_wgp0_per_sa) +
- le32_to_cpu(gc_info->gc_num_wgp1_per_sa));
- adev->gfx.config.max_sh_per_se = le32_to_cpu(gc_info->gc_num_sa_per_se);
- adev->gfx.config.max_backends_per_se = le32_to_cpu(gc_info->gc_num_rb_per_se);
- adev->gfx.config.max_texture_channel_caches = le32_to_cpu(gc_info->gc_num_gl2c);
- adev->gfx.config.max_gprs = le32_to_cpu(gc_info->gc_num_gprs);
- adev->gfx.config.max_gs_threads = le32_to_cpu(gc_info->gc_num_max_gs_thds);
- adev->gfx.config.gs_vgt_table_depth = le32_to_cpu(gc_info->gc_gs_table_depth);
- adev->gfx.config.gs_prim_buffer_depth = le32_to_cpu(gc_info->gc_gsprim_buff_depth);
- adev->gfx.config.double_offchip_lds_buf = le32_to_cpu(gc_info->gc_double_offchip_lds_buffer);
- adev->gfx.cu_info.wave_front_size = le32_to_cpu(gc_info->gc_wave_size);
- adev->gfx.cu_info.max_waves_per_simd = le32_to_cpu(gc_info->gc_max_waves_per_simd);
- adev->gfx.cu_info.max_scratch_slots_per_cu = le32_to_cpu(gc_info->gc_max_scratch_slots_per_cu);
- adev->gfx.cu_info.lds_size = le32_to_cpu(gc_info->gc_lds_size);
- adev->gfx.config.num_sc_per_sh = le32_to_cpu(gc_info->gc_num_sc_per_se) /
- le32_to_cpu(gc_info->gc_num_sa_per_se);
- adev->gfx.config.num_packer_per_sc = le32_to_cpu(gc_info->gc_num_packer_per_sc);
-
+ switch (gc_info->v1.header.version_major) {
+ case 1:
+ adev->gfx.config.max_shader_engines = le32_to_cpu(gc_info->v1.gc_num_se);
+ adev->gfx.config.max_cu_per_sh = 2 * (le32_to_cpu(gc_info->v1.gc_num_wgp0_per_sa) +
+ le32_to_cpu(gc_info->v1.gc_num_wgp1_per_sa));
+ adev->gfx.config.max_sh_per_se = le32_to_cpu(gc_info->v1.gc_num_sa_per_se);
+ adev->gfx.config.max_backends_per_se = le32_to_cpu(gc_info->v1.gc_num_rb_per_se);
+ adev->gfx.config.max_texture_channel_caches = le32_to_cpu(gc_info->v1.gc_num_gl2c);
+ adev->gfx.config.max_gprs = le32_to_cpu(gc_info->v1.gc_num_gprs);
+ adev->gfx.config.max_gs_threads = le32_to_cpu(gc_info->v1.gc_num_max_gs_thds);
+ adev->gfx.config.gs_vgt_table_depth = le32_to_cpu(gc_info->v1.gc_gs_table_depth);
+ adev->gfx.config.gs_prim_buffer_depth = le32_to_cpu(gc_info->v1.gc_gsprim_buff_depth);
+ adev->gfx.config.double_offchip_lds_buf = le32_to_cpu(gc_info->v1.gc_double_offchip_lds_buffer);
+ adev->gfx.cu_info.wave_front_size = le32_to_cpu(gc_info->v1.gc_wave_size);
+ adev->gfx.cu_info.max_waves_per_simd = le32_to_cpu(gc_info->v1.gc_max_waves_per_simd);
+ adev->gfx.cu_info.max_scratch_slots_per_cu = le32_to_cpu(gc_info->v1.gc_max_scratch_slots_per_cu);
+ adev->gfx.cu_info.lds_size = le32_to_cpu(gc_info->v1.gc_lds_size);
+ adev->gfx.config.num_sc_per_sh = le32_to_cpu(gc_info->v1.gc_num_sc_per_se) /
+ le32_to_cpu(gc_info->v1.gc_num_sa_per_se);
+ adev->gfx.config.num_packer_per_sc = le32_to_cpu(gc_info->v1.gc_num_packer_per_sc);
+ break;
+ case 2:
+ adev->gfx.config.max_shader_engines = le32_to_cpu(gc_info->v2.gc_num_se);
+ adev->gfx.config.max_cu_per_sh = le32_to_cpu(gc_info->v2.gc_num_cu_per_sh);
+ adev->gfx.config.max_sh_per_se = le32_to_cpu(gc_info->v2.gc_num_sh_per_se);
+ adev->gfx.config.max_backends_per_se = le32_to_cpu(gc_info->v2.gc_num_rb_per_se);
+ adev->gfx.config.max_texture_channel_caches = le32_to_cpu(gc_info->v2.gc_num_tccs);
+ adev->gfx.config.max_gprs = le32_to_cpu(gc_info->v2.gc_num_gprs);
+ adev->gfx.config.max_gs_threads = le32_to_cpu(gc_info->v2.gc_num_max_gs_thds);
+ adev->gfx.config.gs_vgt_table_depth = le32_to_cpu(gc_info->v2.gc_gs_table_depth);
+ adev->gfx.config.gs_prim_buffer_depth = le32_to_cpu(gc_info->v2.gc_gsprim_buff_depth);
+ adev->gfx.config.double_offchip_lds_buf = le32_to_cpu(gc_info->v2.gc_double_offchip_lds_buffer);
+ adev->gfx.cu_info.wave_front_size = le32_to_cpu(gc_info->v2.gc_wave_size);
+ adev->gfx.cu_info.max_waves_per_simd = le32_to_cpu(gc_info->v2.gc_max_waves_per_simd);
+ adev->gfx.cu_info.max_scratch_slots_per_cu = le32_to_cpu(gc_info->v2.gc_max_scratch_slots_per_cu);
+ adev->gfx.cu_info.lds_size = le32_to_cpu(gc_info->v2.gc_lds_size);
+ adev->gfx.config.num_sc_per_sh = le32_to_cpu(gc_info->v2.gc_num_sc_per_se) /
+ le32_to_cpu(gc_info->v2.gc_num_sh_per_se);
+ adev->gfx.config.num_packer_per_sc = le32_to_cpu(gc_info->v2.gc_num_packer_per_sc);
+ break;
+ default:
+ dev_err(adev->dev,
+ "Unhandled GC info table %d.%d\n",
+ gc_info->v1.header.version_major,
+ gc_info->v1.header.version_minor);
+ return -EINVAL;
+ }
return 0;
}
@@ -917,7 +988,6 @@ static int amdgpu_discovery_set_mm_ip_blocks(struct amdgpu_device *adev)
break;
case IP_VERSION(3, 0, 0):
case IP_VERSION(3, 0, 16):
- case IP_VERSION(3, 0, 64):
case IP_VERSION(3, 1, 1):
case IP_VERSION(3, 0, 2):
case IP_VERSION(3, 0, 192):
@@ -954,7 +1024,7 @@ static int amdgpu_discovery_set_mes_ip_blocks(struct amdgpu_device *adev)
amdgpu_device_ip_block_add(adev, &mes_v10_1_ip_block);
break;
default:
- break;;
+ break;
}
return 0;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.h
index 0ea029e3..14537ce 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.h
@@ -33,8 +33,6 @@ void amdgpu_discovery_harvest_ip(struct amdgpu_device *adev);
int amdgpu_discovery_get_ip_version(struct amdgpu_device *adev, int hw_id, int number_instance,
int *major, int *minor, int *revision);
-int amdgpu_discovery_get_vcn_version(struct amdgpu_device *adev, int vcn_instance,
- int *major, int *minor, int *revision);
int amdgpu_discovery_get_gfx_info(struct amdgpu_device *adev);
int amdgpu_discovery_set_ip_blocks(struct amdgpu_device *adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_display.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_display.c
index dc50c05..82011e7 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_display.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_display.c
@@ -83,9 +83,6 @@ static void amdgpu_display_flip_work_func(struct work_struct *__work)
unsigned i;
int vpos, hpos;
- if (amdgpu_display_flip_handle_fence(work, &work->excl))
- return;
-
for (i = 0; i < work->shared_count; ++i)
if (amdgpu_display_flip_handle_fence(work, &work->shared[i]))
return;
@@ -203,7 +200,7 @@ int amdgpu_display_crtc_page_flip_target(struct drm_crtc *crtc,
goto unpin;
}
- r = dma_resv_get_fences(new_abo->tbo.base.resv, &work->excl,
+ r = dma_resv_get_fences(new_abo->tbo.base.resv, NULL,
&work->shared_count, &work->shared);
if (unlikely(r != 0)) {
DRM_ERROR("failed to get fences for buffer\n");
@@ -253,7 +250,6 @@ int amdgpu_display_crtc_page_flip_target(struct drm_crtc *crtc,
cleanup:
amdgpu_bo_unref(&work->old_abo);
- dma_fence_put(work->excl);
for (i = 0; i < work->shared_count; ++i)
dma_fence_put(work->shared[i]);
kfree(work->shared);
@@ -1364,7 +1360,7 @@ bool amdgpu_display_crtc_scaling_mode_fixup(struct drm_crtc *crtc,
if ((!(mode->flags & DRM_MODE_FLAG_INTERLACE)) &&
((amdgpu_encoder->underscan_type == UNDERSCAN_ON) ||
((amdgpu_encoder->underscan_type == UNDERSCAN_AUTO) &&
- drm_detect_hdmi_monitor(amdgpu_connector_edid(connector)) &&
+ connector->display_info.is_hdmi &&
amdgpu_display_is_hdtv_mode(mode)))) {
if (amdgpu_encoder->underscan_hborder != 0)
amdgpu_crtc->h_border = amdgpu_encoder->underscan_hborder;
@@ -1603,13 +1599,10 @@ int amdgpu_display_suspend_helper(struct amdgpu_device *adev)
continue;
}
robj = gem_to_amdgpu_bo(fb->obj[0]);
- /* don't unpin kernel fb objects */
- if (!amdgpu_fbdev_robj_is_fb(adev, robj)) {
- r = amdgpu_bo_reserve(robj, true);
- if (r == 0) {
- amdgpu_bo_unpin(robj);
- amdgpu_bo_unreserve(robj);
- }
+ r = amdgpu_bo_reserve(robj, true);
+ if (r == 0) {
+ amdgpu_bo_unpin(robj);
+ amdgpu_bo_unreserve(robj);
}
}
return 0;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
index ae6ab93..579adfa 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
@@ -61,9 +61,6 @@ static int amdgpu_dma_buf_attach(struct dma_buf *dmabuf,
if (pci_p2pdma_distance_many(adev->pdev, &attach->dev, 1, true) < 0)
attach->peer2peer = false;
- if (attach->dev->driver == adev->dev->driver)
- return 0;
-
r = pm_runtime_get_sync(adev_to_drm(adev)->dev);
if (r < 0)
goto out;
@@ -384,7 +381,7 @@ amdgpu_dma_buf_move_notify(struct dma_buf_attachment *attach)
struct amdgpu_vm_bo_base *bo_base;
int r;
- if (bo->tbo.resource->mem_type == TTM_PL_SYSTEM)
+ if (!bo->tbo.resource || bo->tbo.resource->mem_type == TTM_PL_SYSTEM)
return;
r = ttm_bo_validate(&bo->tbo, &placement, &ctx);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
index ad95de6..b63ed1d 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
@@ -31,7 +31,6 @@
#include "amdgpu_drv.h"
#include <drm/drm_pciids.h>
-#include <linux/console.h>
#include <linux/module.h>
#include <linux/pm_runtime.h>
#include <linux/vga_switcheroo.h>
@@ -39,6 +38,7 @@
#include <linux/mmu_notifier.h>
#include <linux/suspend.h>
#include <linux/cc_platform.h>
+#include <linux/fb.h>
#include "amdgpu.h"
#include "amdgpu_irq.h"
@@ -314,9 +314,12 @@ module_param_named(dpm, amdgpu_dpm, int, 0444);
/**
* DOC: fw_load_type (int)
- * Set different firmware loading type for debugging (0 = direct, 1 = SMU, 2 = PSP). The default is -1 (auto).
+ * Set different firmware loading type for debugging, if supported.
+ * Set to 0 to force direct loading if supported by the ASIC. Set
+ * to -1 to select the default loading mode for the ASIC, as defined
+ * by the driver. The default is -1 (auto).
*/
-MODULE_PARM_DESC(fw_load_type, "firmware loading type (0 = direct, 1 = SMU, 2 = PSP, -1 = auto)");
+MODULE_PARM_DESC(fw_load_type, "firmware loading type (0 = force direct if supported, -1 = auto)");
module_param_named(fw_load_type, amdgpu_fw_load_type, int, 0444);
/**
@@ -328,10 +331,11 @@ module_param_named(aspm, amdgpu_aspm, int, 0444);
/**
* DOC: runpm (int)
- * Override for runtime power management control for dGPUs in PX/HG laptops. The amdgpu driver can dynamically power down
- * the dGPU on PX/HG laptops when it is idle. The default is -1 (auto enable). Setting the value to 0 disables this functionality.
+ * Override for runtime power management control for dGPUs. The amdgpu driver can dynamically power down
+ * the dGPUs when they are idle if supported. The default is -1 (auto enable).
+ * Setting the value to 0 disables this functionality.
*/
-MODULE_PARM_DESC(runpm, "PX runtime pm (2 = force enable with BAMACO, 1 = force enable with BACO, 0 = disable, -1 = PX only default)");
+MODULE_PARM_DESC(runpm, "PX runtime pm (2 = force enable with BAMACO, 1 = force enable with BACO, 0 = disable, -1 = auto)");
module_param_named(runpm, amdgpu_runtime_pm, int, 0444);
/**
@@ -1889,6 +1893,26 @@ MODULE_DEVICE_TABLE(pci, pciidlist);
static const struct drm_driver amdgpu_kms_driver;
+static bool amdgpu_is_fw_framebuffer(resource_size_t base,
+ resource_size_t size)
+{
+ bool found = false;
+#if IS_REACHABLE(CONFIG_FB)
+ struct apertures_struct *a;
+
+ a = alloc_apertures(1);
+ if (!a)
+ return false;
+
+ a->ranges[0].base = base;
+ a->ranges[0].size = size;
+
+ found = is_firmware_framebuffer(a);
+ kfree(a);
+#endif
+ return found;
+}
+
static int amdgpu_pci_probe(struct pci_dev *pdev,
const struct pci_device_id *ent)
{
@@ -1897,6 +1921,8 @@ static int amdgpu_pci_probe(struct pci_dev *pdev,
unsigned long flags = ent->driver_data;
int ret, retry = 0, i;
bool supports_atomic = false;
+ bool is_fw_fb;
+ resource_size_t base, size;
/* skip devices which are owned by radeon */
for (i = 0; i < ARRAY_SIZE(amdgpu_unsupported_pciidlist); i++) {
@@ -1965,6 +1991,10 @@ static int amdgpu_pci_probe(struct pci_dev *pdev,
}
#endif
+ base = pci_resource_start(pdev, 0);
+ size = pci_resource_len(pdev, 0);
+ is_fw_fb = amdgpu_is_fw_framebuffer(base, size);
+
/* Get rid of things like offb */
ret = drm_aperture_remove_conflicting_pci_framebuffers(pdev, &amdgpu_kms_driver);
if (ret)
@@ -1977,6 +2007,7 @@ static int amdgpu_pci_probe(struct pci_dev *pdev,
adev->dev = &pdev->dev;
adev->pdev = pdev;
ddev = adev_to_drm(adev);
+ adev->is_fw_fb = is_fw_fb;
if (!supports_atomic)
ddev->driver_features &= ~DRIVER_ATOMIC;
@@ -2002,6 +2033,19 @@ static int amdgpu_pci_probe(struct pci_dev *pdev,
goto err_pci;
}
+ /*
+ * 1. don't init fbdev on hw without DCE
+ * 2. don't init fbdev if there are no connectors
+ */
+ if (adev->mode_info.mode_config_initialized &&
+ !list_empty(&adev_to_drm(adev)->mode_config.connector_list)) {
+ /* select 8 bpp console on low vram cards */
+ if (adev->gmc.real_vram_size <= (32*1024*1024))
+ drm_fbdev_generic_setup(adev_to_drm(adev), 8);
+ else
+ drm_fbdev_generic_setup(adev_to_drm(adev), 32);
+ }
+
ret = amdgpu_debugfs_init(adev);
if (ret)
DRM_ERROR("Creating debugfs files failed (%d).\n", ret);
@@ -2153,7 +2197,10 @@ static int amdgpu_pmops_suspend(struct device *dev)
adev->in_s3 = true;
r = amdgpu_device_suspend(drm_dev, true);
adev->in_s3 = false;
-
+ if (r)
+ return r;
+ if (!adev->in_s0ix)
+ r = amdgpu_asic_reset(adev);
return r;
}
@@ -2234,12 +2281,27 @@ static int amdgpu_pmops_runtime_suspend(struct device *dev)
if (amdgpu_device_supports_px(drm_dev))
drm_dev->switch_power_state = DRM_SWITCH_POWER_CHANGING;
+ /*
+ * By setting mp1_state as PP_MP1_STATE_UNLOAD, MP1 will do some
+ * proper cleanups and put itself into a state ready for PNP. That
+ * can address some random resuming failure observed on BOCO capable
+ * platforms.
+ * TODO: this may be also needed for PX capable platform.
+ */
+ if (amdgpu_device_supports_boco(drm_dev))
+ adev->mp1_state = PP_MP1_STATE_UNLOAD;
+
ret = amdgpu_device_suspend(drm_dev, false);
if (ret) {
adev->in_runpm = false;
+ if (amdgpu_device_supports_boco(drm_dev))
+ adev->mp1_state = PP_MP1_STATE_NONE;
return ret;
}
+ if (amdgpu_device_supports_boco(drm_dev))
+ adev->mp1_state = PP_MP1_STATE_NONE;
+
if (amdgpu_device_supports_px(drm_dev)) {
/* Only need to handle PCI state in the driver for ATPX
* PCI core handles it for _PR3.
@@ -2516,10 +2578,8 @@ static int __init amdgpu_init(void)
{
int r;
- if (vgacon_text_force()) {
- DRM_ERROR("VGACON disables amdgpu kernel modesetting.\n");
+ if (drm_firmware_drivers_only())
return -EINVAL;
- }
r = amdgpu_sync_init();
if (r)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.h
index e3a4f70..8178323 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.h
@@ -45,4 +45,7 @@
long amdgpu_drm_ioctl(struct file *filp,
unsigned int cmd, unsigned long arg);
+long amdgpu_kms_compat_ioctl(struct file *filp,
+ unsigned int cmd, unsigned long arg);
+
#endif
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_encoders.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_encoders.c
index af4ef84..c96e458 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_encoders.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_encoders.c
@@ -222,7 +222,7 @@ bool amdgpu_dig_monitor_is_duallink(struct drm_encoder *encoder,
case DRM_MODE_CONNECTOR_HDMIB:
if (amdgpu_connector->use_digital) {
/* HDMI 1.3 supports up to 340 Mhz over single link */
- if (drm_detect_hdmi_monitor(amdgpu_connector_edid(connector))) {
+ if (connector->display_info.is_hdmi) {
if (pixel_clock > 340000)
return true;
else
@@ -244,7 +244,7 @@ bool amdgpu_dig_monitor_is_duallink(struct drm_encoder *encoder,
return false;
else {
/* HDMI 1.3 supports up to 340 Mhz over single link */
- if (drm_detect_hdmi_monitor(amdgpu_connector_edid(connector))) {
+ if (connector->display_info.is_hdmi) {
if (pixel_clock > 340000)
return true;
else
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fb.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fb.c
deleted file mode 100644
index cd0acbe..0000000
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fb.c
+++ /dev/null
@@ -1,388 +0,0 @@
-/*
- * Copyright © 2007 David Airlie
- *
- * Permission is hereby granted, free of charge, to any person obtaining a
- * copy of this software and associated documentation files (the "Software"),
- * to deal in the Software without restriction, including without limitation
- * the rights to use, copy, modify, merge, publish, distribute, sublicense,
- * and/or sell copies of the Software, and to permit persons to whom the
- * Software is furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice (including the next
- * paragraph) shall be included in all copies or substantial portions of the
- * Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
- * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
- * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
- * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
- * DEALINGS IN THE SOFTWARE.
- *
- * Authors:
- * David Airlie
- */
-
-#include <linux/module.h>
-#include <linux/pm_runtime.h>
-#include <linux/slab.h>
-#include <linux/vga_switcheroo.h>
-
-#include <drm/amdgpu_drm.h>
-#include <drm/drm_crtc.h>
-#include <drm/drm_crtc_helper.h>
-#include <drm/drm_fb_helper.h>
-#include <drm/drm_fourcc.h>
-
-#include "amdgpu.h"
-#include "cikd.h"
-#include "amdgpu_gem.h"
-
-#include "amdgpu_display.h"
-
-/* object hierarchy -
- this contains a helper + a amdgpu fb
- the helper contains a pointer to amdgpu framebuffer baseclass.
-*/
-
-static int
-amdgpufb_open(struct fb_info *info, int user)
-{
- struct drm_fb_helper *fb_helper = info->par;
- int ret = pm_runtime_get_sync(fb_helper->dev->dev);
- if (ret < 0 && ret != -EACCES) {
- pm_runtime_mark_last_busy(fb_helper->dev->dev);
- pm_runtime_put_autosuspend(fb_helper->dev->dev);
- return ret;
- }
- return 0;
-}
-
-static int
-amdgpufb_release(struct fb_info *info, int user)
-{
- struct drm_fb_helper *fb_helper = info->par;
-
- pm_runtime_mark_last_busy(fb_helper->dev->dev);
- pm_runtime_put_autosuspend(fb_helper->dev->dev);
- return 0;
-}
-
-static const struct fb_ops amdgpufb_ops = {
- .owner = THIS_MODULE,
- DRM_FB_HELPER_DEFAULT_OPS,
- .fb_open = amdgpufb_open,
- .fb_release = amdgpufb_release,
- .fb_fillrect = drm_fb_helper_cfb_fillrect,
- .fb_copyarea = drm_fb_helper_cfb_copyarea,
- .fb_imageblit = drm_fb_helper_cfb_imageblit,
-};
-
-
-int amdgpu_align_pitch(struct amdgpu_device *adev, int width, int cpp, bool tiled)
-{
- int aligned = width;
- int pitch_mask = 0;
-
- switch (cpp) {
- case 1:
- pitch_mask = 255;
- break;
- case 2:
- pitch_mask = 127;
- break;
- case 3:
- case 4:
- pitch_mask = 63;
- break;
- }
-
- aligned += pitch_mask;
- aligned &= ~pitch_mask;
- return aligned * cpp;
-}
-
-static void amdgpufb_destroy_pinned_object(struct drm_gem_object *gobj)
-{
- struct amdgpu_bo *abo = gem_to_amdgpu_bo(gobj);
- int ret;
-
- ret = amdgpu_bo_reserve(abo, true);
- if (likely(ret == 0)) {
- amdgpu_bo_kunmap(abo);
- amdgpu_bo_unpin(abo);
- amdgpu_bo_unreserve(abo);
- }
- drm_gem_object_put(gobj);
-}
-
-static int amdgpufb_create_pinned_object(struct amdgpu_fbdev *rfbdev,
- struct drm_mode_fb_cmd2 *mode_cmd,
- struct drm_gem_object **gobj_p)
-{
- const struct drm_format_info *info;
- struct amdgpu_device *adev = rfbdev->adev;
- struct drm_gem_object *gobj = NULL;
- struct amdgpu_bo *abo = NULL;
- bool fb_tiled = false; /* useful for testing */
- u32 tiling_flags = 0, domain;
- int ret;
- int aligned_size, size;
- int height = mode_cmd->height;
- u32 cpp;
- u64 flags = AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED |
- AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS |
- AMDGPU_GEM_CREATE_VRAM_CLEARED;
-
- info = drm_get_format_info(adev_to_drm(adev), mode_cmd);
- cpp = info->cpp[0];
-
- /* need to align pitch with crtc limits */
- mode_cmd->pitches[0] = amdgpu_align_pitch(adev, mode_cmd->width, cpp,
- fb_tiled);
- domain = amdgpu_display_supported_domains(adev, flags);
- height = ALIGN(mode_cmd->height, 8);
- size = mode_cmd->pitches[0] * height;
- aligned_size = ALIGN(size, PAGE_SIZE);
- ret = amdgpu_gem_object_create(adev, aligned_size, 0, domain, flags,
- ttm_bo_type_device, NULL, &gobj);
- if (ret) {
- pr_err("failed to allocate framebuffer (%d)\n", aligned_size);
- return -ENOMEM;
- }
- abo = gem_to_amdgpu_bo(gobj);
-
- if (fb_tiled)
- tiling_flags = AMDGPU_TILING_SET(ARRAY_MODE, GRPH_ARRAY_2D_TILED_THIN1);
-
- ret = amdgpu_bo_reserve(abo, false);
- if (unlikely(ret != 0))
- goto out_unref;
-
- if (tiling_flags) {
- ret = amdgpu_bo_set_tiling_flags(abo,
- tiling_flags);
- if (ret)
- dev_err(adev->dev, "FB failed to set tiling flags\n");
- }
-
- ret = amdgpu_bo_pin(abo, domain);
- if (ret) {
- amdgpu_bo_unreserve(abo);
- goto out_unref;
- }
-
- ret = amdgpu_ttm_alloc_gart(&abo->tbo);
- if (ret) {
- amdgpu_bo_unreserve(abo);
- dev_err(adev->dev, "%p bind failed\n", abo);
- goto out_unref;
- }
-
- ret = amdgpu_bo_kmap(abo, NULL);
- amdgpu_bo_unreserve(abo);
- if (ret) {
- goto out_unref;
- }
-
- *gobj_p = gobj;
- return 0;
-out_unref:
- amdgpufb_destroy_pinned_object(gobj);
- *gobj_p = NULL;
- return ret;
-}
-
-static int amdgpufb_create(struct drm_fb_helper *helper,
- struct drm_fb_helper_surface_size *sizes)
-{
- struct amdgpu_fbdev *rfbdev = (struct amdgpu_fbdev *)helper;
- struct amdgpu_device *adev = rfbdev->adev;
- struct fb_info *info;
- struct drm_framebuffer *fb = NULL;
- struct drm_mode_fb_cmd2 mode_cmd;
- struct drm_gem_object *gobj = NULL;
- struct amdgpu_bo *abo = NULL;
- int ret;
-
- memset(&mode_cmd, 0, sizeof(mode_cmd));
- mode_cmd.width = sizes->surface_width;
- mode_cmd.height = sizes->surface_height;
-
- if (sizes->surface_bpp == 24)
- sizes->surface_bpp = 32;
-
- mode_cmd.pixel_format = drm_mode_legacy_fb_format(sizes->surface_bpp,
- sizes->surface_depth);
-
- ret = amdgpufb_create_pinned_object(rfbdev, &mode_cmd, &gobj);
- if (ret) {
- DRM_ERROR("failed to create fbcon object %d\n", ret);
- return ret;
- }
-
- abo = gem_to_amdgpu_bo(gobj);
-
- /* okay we have an object now allocate the framebuffer */
- info = drm_fb_helper_alloc_fbi(helper);
- if (IS_ERR(info)) {
- ret = PTR_ERR(info);
- goto out;
- }
-
- ret = amdgpu_display_gem_fb_init(adev_to_drm(adev), &rfbdev->rfb,
- &mode_cmd, gobj);
- if (ret) {
- DRM_ERROR("failed to initialize framebuffer %d\n", ret);
- goto out;
- }
-
- fb = &rfbdev->rfb.base;
-
- /* setup helper */
- rfbdev->helper.fb = fb;
-
- info->fbops = &amdgpufb_ops;
-
- info->fix.smem_start = amdgpu_gmc_vram_cpu_pa(adev, abo);
- info->fix.smem_len = amdgpu_bo_size(abo);
- info->screen_base = amdgpu_bo_kptr(abo);
- info->screen_size = amdgpu_bo_size(abo);
-
- drm_fb_helper_fill_info(info, &rfbdev->helper, sizes);
-
- /* setup aperture base/size for vesafb takeover */
- info->apertures->ranges[0].base = adev_to_drm(adev)->mode_config.fb_base;
- info->apertures->ranges[0].size = adev->gmc.aper_size;
-
- /* Use default scratch pixmap (info->pixmap.flags = FB_PIXMAP_SYSTEM) */
-
- if (info->screen_base == NULL) {
- ret = -ENOSPC;
- goto out;
- }
-
- DRM_INFO("fb mappable at 0x%lX\n", info->fix.smem_start);
- DRM_INFO("vram apper at 0x%lX\n", (unsigned long)adev->gmc.aper_base);
- DRM_INFO("size %lu\n", (unsigned long)amdgpu_bo_size(abo));
- DRM_INFO("fb depth is %d\n", fb->format->depth);
- DRM_INFO(" pitch is %d\n", fb->pitches[0]);
-
- vga_switcheroo_client_fb_set(adev->pdev, info);
- return 0;
-
-out:
- if (fb && ret) {
- drm_gem_object_put(gobj);
- drm_framebuffer_unregister_private(fb);
- drm_framebuffer_cleanup(fb);
- kfree(fb);
- }
- return ret;
-}
-
-static int amdgpu_fbdev_destroy(struct drm_device *dev, struct amdgpu_fbdev *rfbdev)
-{
- struct amdgpu_framebuffer *rfb = &rfbdev->rfb;
- int i;
-
- drm_fb_helper_unregister_fbi(&rfbdev->helper);
-
- if (rfb->base.obj[0]) {
- for (i = 0; i < rfb->base.format->num_planes; i++)
- drm_gem_object_put(rfb->base.obj[0]);
- amdgpufb_destroy_pinned_object(rfb->base.obj[0]);
- rfb->base.obj[0] = NULL;
- drm_framebuffer_unregister_private(&rfb->base);
- drm_framebuffer_cleanup(&rfb->base);
- }
- drm_fb_helper_fini(&rfbdev->helper);
-
- return 0;
-}
-
-static const struct drm_fb_helper_funcs amdgpu_fb_helper_funcs = {
- .fb_probe = amdgpufb_create,
-};
-
-int amdgpu_fbdev_init(struct amdgpu_device *adev)
-{
- struct amdgpu_fbdev *rfbdev;
- int bpp_sel = 32;
- int ret;
-
- /* don't init fbdev on hw without DCE */
- if (!adev->mode_info.mode_config_initialized)
- return 0;
-
- /* don't init fbdev if there are no connectors */
- if (list_empty(&adev_to_drm(adev)->mode_config.connector_list))
- return 0;
-
- /* select 8 bpp console on low vram cards */
- if (adev->gmc.real_vram_size <= (32*1024*1024))
- bpp_sel = 8;
-
- rfbdev = kzalloc(sizeof(struct amdgpu_fbdev), GFP_KERNEL);
- if (!rfbdev)
- return -ENOMEM;
-
- rfbdev->adev = adev;
- adev->mode_info.rfbdev = rfbdev;
-
- drm_fb_helper_prepare(adev_to_drm(adev), &rfbdev->helper,
- &amdgpu_fb_helper_funcs);
-
- ret = drm_fb_helper_init(adev_to_drm(adev), &rfbdev->helper);
- if (ret) {
- kfree(rfbdev);
- return ret;
- }
-
- /* disable all the possible outputs/crtcs before entering KMS mode */
- if (!amdgpu_device_has_dc_support(adev) && !amdgpu_virtual_display)
- drm_helper_disable_unused_functions(adev_to_drm(adev));
-
- drm_fb_helper_initial_config(&rfbdev->helper, bpp_sel);
- return 0;
-}
-
-void amdgpu_fbdev_fini(struct amdgpu_device *adev)
-{
- if (!adev->mode_info.rfbdev)
- return;
-
- amdgpu_fbdev_destroy(adev_to_drm(adev), adev->mode_info.rfbdev);
- kfree(adev->mode_info.rfbdev);
- adev->mode_info.rfbdev = NULL;
-}
-
-void amdgpu_fbdev_set_suspend(struct amdgpu_device *adev, int state)
-{
- if (adev->mode_info.rfbdev)
- drm_fb_helper_set_suspend_unlocked(&adev->mode_info.rfbdev->helper,
- state);
-}
-
-int amdgpu_fbdev_total_size(struct amdgpu_device *adev)
-{
- struct amdgpu_bo *robj;
- int size = 0;
-
- if (!adev->mode_info.rfbdev)
- return 0;
-
- robj = gem_to_amdgpu_bo(adev->mode_info.rfbdev->rfb.base.obj[0]);
- size += amdgpu_bo_size(robj);
- return size;
-}
-
-bool amdgpu_fbdev_robj_is_fb(struct amdgpu_device *adev, struct amdgpu_bo *robj)
-{
- if (!adev->mode_info.rfbdev)
- return false;
- if (robj == gem_to_amdgpu_bo(adev->mode_info.rfbdev->rfb.base.obj[0]))
- return true;
- return false;
-}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
index 3b7e86e..45977a7 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
@@ -77,11 +77,13 @@ void amdgpu_fence_slab_fini(void)
* Cast helper
*/
static const struct dma_fence_ops amdgpu_fence_ops;
+static const struct dma_fence_ops amdgpu_job_fence_ops;
static inline struct amdgpu_fence *to_amdgpu_fence(struct dma_fence *f)
{
struct amdgpu_fence *__f = container_of(f, struct amdgpu_fence, base);
- if (__f->base.ops == &amdgpu_fence_ops)
+ if (__f->base.ops == &amdgpu_fence_ops ||
+ __f->base.ops == &amdgpu_job_fence_ops)
return __f;
return NULL;
@@ -158,19 +160,18 @@ int amdgpu_fence_emit(struct amdgpu_ring *ring, struct dma_fence **f, struct amd
}
seq = ++ring->fence_drv.sync_seq;
- if (job != NULL && job->job_run_counter) {
+ if (job && job->job_run_counter) {
/* reinit seq for resubmitted jobs */
fence->seqno = seq;
} else {
- dma_fence_init(fence, &amdgpu_fence_ops,
- &ring->fence_drv.lock,
- adev->fence_context + ring->idx,
- seq);
- }
-
- if (job != NULL) {
- /* mark this fence has a parent job */
- set_bit(AMDGPU_FENCE_FLAG_EMBED_IN_JOB_BIT, &fence->flags);
+ if (job)
+ dma_fence_init(fence, &amdgpu_job_fence_ops,
+ &ring->fence_drv.lock,
+ adev->fence_context + ring->idx, seq);
+ else
+ dma_fence_init(fence, &amdgpu_fence_ops,
+ &ring->fence_drv.lock,
+ adev->fence_context + ring->idx, seq);
}
amdgpu_ring_emit_fence(ring, ring->fence_drv.gpu_addr,
@@ -546,9 +547,6 @@ void amdgpu_fence_driver_hw_fini(struct amdgpu_device *adev)
if (!ring || !ring->fence_drv.initialized)
continue;
- if (!ring->no_scheduler)
- drm_sched_stop(&ring->sched, NULL);
-
/* You can't wait for HW to signal if it's gone */
if (!drm_dev_is_unplugged(adev_to_drm(adev)))
r = amdgpu_fence_wait_empty(ring);
@@ -608,11 +606,6 @@ void amdgpu_fence_driver_hw_init(struct amdgpu_device *adev)
if (!ring || !ring->fence_drv.initialized)
continue;
- if (!ring->no_scheduler) {
- drm_sched_resubmit_jobs(&ring->sched);
- drm_sched_start(&ring->sched, true);
- }
-
/* enable the interrupt */
if (ring->fence_drv.irq_src)
amdgpu_irq_get(adev, ring->fence_drv.irq_src,
@@ -621,6 +614,25 @@ void amdgpu_fence_driver_hw_init(struct amdgpu_device *adev)
}
/**
+ * amdgpu_fence_driver_clear_job_fences - clear job embedded fences of ring
+ *
+ * @ring: fence of the ring to be cleared
+ *
+ */
+void amdgpu_fence_driver_clear_job_fences(struct amdgpu_ring *ring)
+{
+ int i;
+ struct dma_fence *old, **ptr;
+
+ for (i = 0; i <= ring->fence_drv.num_fences_mask; i++) {
+ ptr = &ring->fence_drv.fences[i];
+ old = rcu_dereference_protected(*ptr, 1);
+ if (old && old->ops == &amdgpu_job_fence_ops)
+ RCU_INIT_POINTER(*ptr, NULL);
+ }
+}
+
+/**
* amdgpu_fence_driver_force_completion - force signal latest fence of ring
*
* @ring: fence of the ring to signal
@@ -643,16 +655,14 @@ static const char *amdgpu_fence_get_driver_name(struct dma_fence *fence)
static const char *amdgpu_fence_get_timeline_name(struct dma_fence *f)
{
- struct amdgpu_ring *ring;
+ return (const char *)to_amdgpu_fence(f)->ring->name;
+}
- if (test_bit(AMDGPU_FENCE_FLAG_EMBED_IN_JOB_BIT, &f->flags)) {
- struct amdgpu_job *job = container_of(f, struct amdgpu_job, hw_fence);
+static const char *amdgpu_job_fence_get_timeline_name(struct dma_fence *f)
+{
+ struct amdgpu_job *job = container_of(f, struct amdgpu_job, hw_fence);
- ring = to_amdgpu_ring(job->base.sched);
- } else {
- ring = to_amdgpu_fence(f)->ring;
- }
- return (const char *)ring->name;
+ return (const char *)to_amdgpu_ring(job->base.sched)->name;
}
/**
@@ -665,18 +675,25 @@ static const char *amdgpu_fence_get_timeline_name(struct dma_fence *f)
*/
static bool amdgpu_fence_enable_signaling(struct dma_fence *f)
{
- struct amdgpu_ring *ring;
+ if (!timer_pending(&to_amdgpu_fence(f)->ring->fence_drv.fallback_timer))
+ amdgpu_fence_schedule_fallback(to_amdgpu_fence(f)->ring);
- if (test_bit(AMDGPU_FENCE_FLAG_EMBED_IN_JOB_BIT, &f->flags)) {
- struct amdgpu_job *job = container_of(f, struct amdgpu_job, hw_fence);
+ return true;
+}
- ring = to_amdgpu_ring(job->base.sched);
- } else {
- ring = to_amdgpu_fence(f)->ring;
- }
+/**
+ * amdgpu_job_fence_enable_signaling - enable signalling on job fence
+ * @f: fence
+ *
+ * This is the simliar function with amdgpu_fence_enable_signaling above, it
+ * only handles the job embedded fence.
+ */
+static bool amdgpu_job_fence_enable_signaling(struct dma_fence *f)
+{
+ struct amdgpu_job *job = container_of(f, struct amdgpu_job, hw_fence);
- if (!timer_pending(&ring->fence_drv.fallback_timer))
- amdgpu_fence_schedule_fallback(ring);
+ if (!timer_pending(&to_amdgpu_ring(job->base.sched)->fence_drv.fallback_timer))
+ amdgpu_fence_schedule_fallback(to_amdgpu_ring(job->base.sched));
return true;
}
@@ -692,19 +709,23 @@ static void amdgpu_fence_free(struct rcu_head *rcu)
{
struct dma_fence *f = container_of(rcu, struct dma_fence, rcu);
- if (test_bit(AMDGPU_FENCE_FLAG_EMBED_IN_JOB_BIT, &f->flags)) {
- /* free job if fence has a parent job */
- struct amdgpu_job *job;
-
- job = container_of(f, struct amdgpu_job, hw_fence);
- kfree(job);
- } else {
/* free fence_slab if it's separated fence*/
- struct amdgpu_fence *fence;
+ kmem_cache_free(amdgpu_fence_slab, to_amdgpu_fence(f));
+}
- fence = to_amdgpu_fence(f);
- kmem_cache_free(amdgpu_fence_slab, fence);
- }
+/**
+ * amdgpu_job_fence_free - free up the job with embedded fence
+ *
+ * @rcu: RCU callback head
+ *
+ * Free up the job with embedded fence after the RCU grace period.
+ */
+static void amdgpu_job_fence_free(struct rcu_head *rcu)
+{
+ struct dma_fence *f = container_of(rcu, struct dma_fence, rcu);
+
+ /* free job if fence has a parent job */
+ kfree(container_of(f, struct amdgpu_job, hw_fence));
}
/**
@@ -720,6 +741,19 @@ static void amdgpu_fence_release(struct dma_fence *f)
call_rcu(&f->rcu, amdgpu_fence_free);
}
+/**
+ * amdgpu_job_fence_release - callback that job embedded fence can be freed
+ *
+ * @f: fence
+ *
+ * This is the simliar function with amdgpu_fence_release above, it
+ * only handles the job embedded fence.
+ */
+static void amdgpu_job_fence_release(struct dma_fence *f)
+{
+ call_rcu(&f->rcu, amdgpu_job_fence_free);
+}
+
static const struct dma_fence_ops amdgpu_fence_ops = {
.get_driver_name = amdgpu_fence_get_driver_name,
.get_timeline_name = amdgpu_fence_get_timeline_name,
@@ -727,6 +761,12 @@ static const struct dma_fence_ops amdgpu_fence_ops = {
.release = amdgpu_fence_release,
};
+static const struct dma_fence_ops amdgpu_job_fence_ops = {
+ .get_driver_name = amdgpu_fence_get_driver_name,
+ .get_timeline_name = amdgpu_job_fence_get_timeline_name,
+ .enable_signaling = amdgpu_job_fence_enable_signaling,
+ .release = amdgpu_job_fence_release,
+};
/*
* Fence debugfs
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fru_eeprom.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fru_eeprom.c
index 7709cae..2a786e7 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fru_eeprom.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fru_eeprom.c
@@ -56,6 +56,9 @@ static bool is_fru_eeprom_supported(struct amdgpu_device *adev)
return true;
else
return false;
+ case CHIP_ALDEBARAN:
+ /* All Aldebaran SKUs have the FRU */
+ return true;
default:
return false;
}
@@ -88,13 +91,17 @@ static int amdgpu_fru_read_eeprom(struct amdgpu_device *adev, uint32_t addrptr,
int amdgpu_fru_get_product_info(struct amdgpu_device *adev)
{
- unsigned char buff[34];
+ unsigned char buff[AMDGPU_PRODUCT_NAME_LEN+2];
u32 addrptr;
int size, len;
+ int offset = 2;
if (!is_fru_eeprom_supported(adev))
return 0;
+ if (adev->asic_type == CHIP_ALDEBARAN)
+ offset = 0;
+
/* If algo exists, it means that the i2c_adapter's initialized */
if (!adev->pm.smu_i2c.algo) {
DRM_WARN("Cannot access FRU, EEPROM accessor not initialized");
@@ -131,15 +138,13 @@ int amdgpu_fru_get_product_info(struct amdgpu_device *adev)
}
len = size;
- /* Product name should only be 32 characters. Any more,
- * and something could be wrong. Cap it at 32 to be safe
- */
- if (len >= sizeof(adev->product_name)) {
- DRM_WARN("FRU Product Number is larger than 32 characters. This is likely a mistake");
- len = sizeof(adev->product_name) - 1;
+ if (len >= AMDGPU_PRODUCT_NAME_LEN) {
+ DRM_WARN("FRU Product Name is larger than %d characters. This is likely a mistake",
+ AMDGPU_PRODUCT_NAME_LEN);
+ len = AMDGPU_PRODUCT_NAME_LEN - 1;
}
/* Start at 2 due to buff using fields 0 and 1 for the address */
- memcpy(adev->product_name, &buff[2], len);
+ memcpy(adev->product_name, &buff[offset], len);
adev->product_name[len] = '\0';
addrptr += size + 1;
@@ -157,7 +162,7 @@ int amdgpu_fru_get_product_info(struct amdgpu_device *adev)
DRM_WARN("FRU Product Number is larger than 16 characters. This is likely a mistake");
len = sizeof(adev->product_number) - 1;
}
- memcpy(adev->product_number, &buff[2], len);
+ memcpy(adev->product_number, &buff[offset], len);
adev->product_number[len] = '\0';
addrptr += size + 1;
@@ -184,7 +189,7 @@ int amdgpu_fru_get_product_info(struct amdgpu_device *adev)
DRM_WARN("FRU Serial Number is larger than 16 characters. This is likely a mistake");
len = sizeof(adev->serial) - 1;
}
- memcpy(adev->serial, &buff[2], len);
+ memcpy(adev->serial, &buff[offset], len);
adev->serial[len] = '\0';
return 0;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
index a1e63ba..9a6507a 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
@@ -264,6 +264,9 @@ static int amdgpu_gem_object_mmap(struct drm_gem_object *obj, struct vm_area_str
!(vma->vm_flags & (VM_READ | VM_WRITE | VM_EXEC)))
vma->vm_flags &= ~VM_MAYWRITE;
+ if (bo->kfd_bo)
+ vma->vm_flags |= VM_DONTCOPY;
+
return drm_gem_ttm_mmap(obj, vma);
}
@@ -877,6 +880,32 @@ int amdgpu_gem_op_ioctl(struct drm_device *dev, void *data,
return r;
}
+static int amdgpu_gem_align_pitch(struct amdgpu_device *adev,
+ int width,
+ int cpp,
+ bool tiled)
+{
+ int aligned = width;
+ int pitch_mask = 0;
+
+ switch (cpp) {
+ case 1:
+ pitch_mask = 255;
+ break;
+ case 2:
+ pitch_mask = 127;
+ break;
+ case 3:
+ case 4:
+ pitch_mask = 63;
+ break;
+ }
+
+ aligned += pitch_mask;
+ aligned &= ~pitch_mask;
+ return aligned * cpp;
+}
+
int amdgpu_mode_dumb_create(struct drm_file *file_priv,
struct drm_device *dev,
struct drm_mode_create_dumb *args)
@@ -885,7 +914,8 @@ int amdgpu_mode_dumb_create(struct drm_file *file_priv,
struct drm_gem_object *gobj;
uint32_t handle;
u64 flags = AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED |
- AMDGPU_GEM_CREATE_CPU_GTT_USWC;
+ AMDGPU_GEM_CREATE_CPU_GTT_USWC |
+ AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS;
u32 domain;
int r;
@@ -897,8 +927,8 @@ int amdgpu_mode_dumb_create(struct drm_file *file_priv,
if (adev->mman.buffer_funcs_enabled)
flags |= AMDGPU_GEM_CREATE_VRAM_CLEARED;
- args->pitch = amdgpu_align_pitch(adev, args->width,
- DIV_ROUND_UP(args->bpp, 8), 0);
+ args->pitch = amdgpu_gem_align_pitch(adev, args->width,
+ DIV_ROUND_UP(args->bpp, 8), 0);
args->size = (u64)args->pitch * args->height;
args->size = ALIGN(args->size, PAGE_SIZE);
domain = amdgpu_bo_get_preferred_domain(adev,
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c
index 08478fc..2430d62 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c
@@ -350,6 +350,7 @@ static inline uint64_t amdgpu_gmc_fault_key(uint64_t addr, uint16_t pasid)
* amdgpu_gmc_filter_faults - filter VM faults
*
* @adev: amdgpu device structure
+ * @ih: interrupt ring that the fault received from
* @addr: address of the VM fault
* @pasid: PASID of the process causing the fault
* @timestamp: timestamp of the fault
@@ -358,7 +359,8 @@ static inline uint64_t amdgpu_gmc_fault_key(uint64_t addr, uint16_t pasid)
* True if the fault was filtered and should not be processed further.
* False if the fault is a new one and needs to be handled.
*/
-bool amdgpu_gmc_filter_faults(struct amdgpu_device *adev, uint64_t addr,
+bool amdgpu_gmc_filter_faults(struct amdgpu_device *adev,
+ struct amdgpu_ih_ring *ih, uint64_t addr,
uint16_t pasid, uint64_t timestamp)
{
struct amdgpu_gmc *gmc = &adev->gmc;
@@ -366,6 +368,10 @@ bool amdgpu_gmc_filter_faults(struct amdgpu_device *adev, uint64_t addr,
struct amdgpu_gmc_fault *fault;
uint32_t hash;
+ /* Stale retry fault if timestamp goes backward */
+ if (amdgpu_ih_ts_after(timestamp, ih->processed_timestamp))
+ return true;
+
/* If we don't have space left in the ring buffer return immediately */
stamp = max(timestamp, AMDGPU_GMC_FAULT_TIMEOUT + 1) -
AMDGPU_GMC_FAULT_TIMEOUT;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h
index e552011..8458ceb 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h
@@ -316,7 +316,8 @@ void amdgpu_gmc_gart_location(struct amdgpu_device *adev,
struct amdgpu_gmc *mc);
void amdgpu_gmc_agp_location(struct amdgpu_device *adev,
struct amdgpu_gmc *mc);
-bool amdgpu_gmc_filter_faults(struct amdgpu_device *adev, uint64_t addr,
+bool amdgpu_gmc_filter_faults(struct amdgpu_device *adev,
+ struct amdgpu_ih_ring *ih, uint64_t addr,
uint16_t pasid, uint64_t timestamp);
void amdgpu_gmc_filter_faults_remove(struct amdgpu_device *adev, uint64_t addr,
uint16_t pasid);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.c
index 0c7963dfa..3df1465 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.c
@@ -164,52 +164,32 @@ void amdgpu_ih_ring_write(struct amdgpu_ih_ring *ih, const uint32_t *iv,
}
}
-/* Waiter helper that checks current rptr matches or passes checkpoint wptr */
-static bool amdgpu_ih_has_checkpoint_processed(struct amdgpu_device *adev,
- struct amdgpu_ih_ring *ih,
- uint32_t checkpoint_wptr,
- uint32_t *prev_rptr)
-{
- uint32_t cur_rptr = ih->rptr | (*prev_rptr & ~ih->ptr_mask);
-
- /* rptr has wrapped. */
- if (cur_rptr < *prev_rptr)
- cur_rptr += ih->ptr_mask + 1;
- *prev_rptr = cur_rptr;
-
- /* check ring is empty to workaround missing wptr overflow flag */
- return cur_rptr >= checkpoint_wptr ||
- (cur_rptr & ih->ptr_mask) == amdgpu_ih_get_wptr(adev, ih);
-}
-
/**
- * amdgpu_ih_wait_on_checkpoint_process - wait to process IVs up to checkpoint
+ * amdgpu_ih_wait_on_checkpoint_process_ts - wait to process IVs up to checkpoint
*
* @adev: amdgpu_device pointer
* @ih: ih ring to process
*
* Used to ensure ring has processed IVs up to the checkpoint write pointer.
*/
-int amdgpu_ih_wait_on_checkpoint_process(struct amdgpu_device *adev,
+int amdgpu_ih_wait_on_checkpoint_process_ts(struct amdgpu_device *adev,
struct amdgpu_ih_ring *ih)
{
- uint32_t checkpoint_wptr, rptr;
+ uint32_t checkpoint_wptr;
+ uint64_t checkpoint_ts;
+ long timeout = HZ;
if (!ih->enabled || adev->shutdown)
return -ENODEV;
checkpoint_wptr = amdgpu_ih_get_wptr(adev, ih);
- /* Order wptr with rptr. */
+ /* Order wptr with ring data. */
rmb();
- rptr = READ_ONCE(ih->rptr);
+ checkpoint_ts = amdgpu_ih_decode_iv_ts(adev, ih, checkpoint_wptr, -1);
- /* wptr has wrapped. */
- if (rptr > checkpoint_wptr)
- checkpoint_wptr += ih->ptr_mask + 1;
-
- return wait_event_interruptible(ih->wait_process,
- amdgpu_ih_has_checkpoint_processed(adev, ih,
- checkpoint_wptr, &rptr));
+ return wait_event_interruptible_timeout(ih->wait_process,
+ amdgpu_ih_ts_after(checkpoint_ts, ih->processed_timestamp) ||
+ ih->rptr == amdgpu_ih_get_wptr(adev, ih), timeout);
}
/**
@@ -299,3 +279,18 @@ void amdgpu_ih_decode_iv_helper(struct amdgpu_device *adev,
/* wptr/rptr are in bytes! */
ih->rptr += 32;
}
+
+uint64_t amdgpu_ih_decode_iv_ts_helper(struct amdgpu_ih_ring *ih, u32 rptr,
+ signed int offset)
+{
+ uint32_t iv_size = 32;
+ uint32_t ring_index;
+ uint32_t dw1, dw2;
+
+ rptr += iv_size * offset;
+ ring_index = (rptr & ih->ptr_mask) >> 2;
+
+ dw1 = le32_to_cpu(ih->ring[ring_index + 1]);
+ dw2 = le32_to_cpu(ih->ring[ring_index + 2]);
+ return dw1 | ((u64)(dw2 & 0xffff) << 32);
+}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.h
index 0649b59..dd1c2ed 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.h
@@ -68,20 +68,30 @@ struct amdgpu_ih_ring {
/* For waiting on IH processing at checkpoint. */
wait_queue_head_t wait_process;
+ uint64_t processed_timestamp;
};
+/* return true if time stamp t2 is after t1 with 48bit wrap around */
+#define amdgpu_ih_ts_after(t1, t2) \
+ (((int64_t)((t2) << 16) - (int64_t)((t1) << 16)) > 0LL)
+
/* provided by the ih block */
struct amdgpu_ih_funcs {
/* ring read/write ptr handling, called from interrupt context */
u32 (*get_wptr)(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih);
void (*decode_iv)(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih,
struct amdgpu_iv_entry *entry);
+ uint64_t (*decode_iv_ts)(struct amdgpu_ih_ring *ih, u32 rptr,
+ signed int offset);
void (*set_rptr)(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih);
};
#define amdgpu_ih_get_wptr(adev, ih) (adev)->irq.ih_funcs->get_wptr((adev), (ih))
#define amdgpu_ih_decode_iv(adev, iv) \
(adev)->irq.ih_funcs->decode_iv((adev), (ih), (iv))
+#define amdgpu_ih_decode_iv_ts(adev, ih, rptr, offset) \
+ (WARN_ON_ONCE(!(adev)->irq.ih_funcs->decode_iv_ts) ? 0 : \
+ (adev)->irq.ih_funcs->decode_iv_ts((ih), (rptr), (offset)))
#define amdgpu_ih_set_rptr(adev, ih) (adev)->irq.ih_funcs->set_rptr((adev), (ih))
int amdgpu_ih_ring_init(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih,
@@ -89,10 +99,12 @@ int amdgpu_ih_ring_init(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih,
void amdgpu_ih_ring_fini(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih);
void amdgpu_ih_ring_write(struct amdgpu_ih_ring *ih, const uint32_t *iv,
unsigned int num_dw);
-int amdgpu_ih_wait_on_checkpoint_process(struct amdgpu_device *adev,
- struct amdgpu_ih_ring *ih);
+int amdgpu_ih_wait_on_checkpoint_process_ts(struct amdgpu_device *adev,
+ struct amdgpu_ih_ring *ih);
int amdgpu_ih_process(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih);
void amdgpu_ih_decode_iv_helper(struct amdgpu_device *adev,
struct amdgpu_ih_ring *ih,
struct amdgpu_iv_entry *entry);
+uint64_t amdgpu_ih_decode_iv_ts_helper(struct amdgpu_ih_ring *ih, u32 rptr,
+ signed int offset);
#endif
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ioc32.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ioc32.c
index 5cf142e..a1cbd7c 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ioc32.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ioc32.c
@@ -1,4 +1,4 @@
-/**
+/*
* \file amdgpu_ioc32.c
*
* 32-bit ioctl compatibility routines for the AMDGPU DRM.
@@ -37,12 +37,9 @@
long amdgpu_kms_compat_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
{
unsigned int nr = DRM_IOCTL_NR(cmd);
- int ret;
if (nr < DRM_COMMAND_BASE)
return drm_compat_ioctl(filp, cmd, arg);
- ret = amdgpu_drm_ioctl(filp, cmd, arg);
-
- return ret;
+ return amdgpu_drm_ioctl(filp, cmd, arg);
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
index cc2e0c9..f5cbc27 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
@@ -333,7 +333,6 @@ int amdgpu_irq_init(struct amdgpu_device *adev)
if (!amdgpu_device_has_dc_support(adev)) {
if (!adev->enable_virtual_display)
/* Disable vblank IRQs aggressively for power-saving */
- /* XXX: can this be enabled for DC? */
adev_to_drm(adev)->vblank_disable_immediate = true;
r = drm_vblank_init(adev_to_drm(adev), adev->mode_info.num_crtc);
@@ -391,7 +390,7 @@ void amdgpu_irq_fini_hw(struct amdgpu_device *adev)
}
/**
- * amdgpu_irq_fini - shut down interrupt handling
+ * amdgpu_irq_fini_sw - shut down interrupt handling
*
* @adev: amdgpu device pointer
*
@@ -529,6 +528,9 @@ void amdgpu_irq_dispatch(struct amdgpu_device *adev,
/* Send it to amdkfd as well if it isn't already handled */
if (!handled)
amdgpu_amdkfd_interrupt(adev, entry.iv_entry);
+
+ if (amdgpu_ih_ts_after(ih->processed_timestamp, entry.timestamp))
+ ih->processed_timestamp = entry.timestamp;
}
/**
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c
index 651c7ab..09ad179 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c
@@ -206,6 +206,12 @@ int amdgpu_driver_load_kms(struct amdgpu_device *adev, unsigned long flags)
adev->runpm = true;
break;
}
+ /* XXX: disable runtime pm if we are the primary adapter
+ * to avoid displays being re-enabled after DPMS.
+ * This needs to be sorted out and fixed properly.
+ */
+ if (adev->is_fw_fb)
+ adev->runpm = false;
if (adev->runpm)
dev_info(adev->dev, "Using BACO for runtime pm\n");
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_mode.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_mode.h
index 89fb372..6043bf6 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_mode.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_mode.h
@@ -232,8 +232,6 @@ struct amdgpu_i2c_chan {
struct mutex mutex;
};
-struct amdgpu_fbdev;
-
struct amdgpu_afmt {
bool enabled;
int offset;
@@ -309,13 +307,6 @@ struct amdgpu_framebuffer {
uint64_t address;
};
-struct amdgpu_fbdev {
- struct drm_fb_helper helper;
- struct amdgpu_framebuffer rfb;
- struct list_head fbdev_list;
- struct amdgpu_device *adev;
-};
-
struct amdgpu_mode_info {
struct atom_context *atom_context;
struct card_info *atom_card_info;
@@ -341,8 +332,6 @@ struct amdgpu_mode_info {
struct edid *bios_hardcoded_edid;
int bios_hardcoded_edid_size;
- /* pointer to fbdev info structure */
- struct amdgpu_fbdev *rfbdev;
/* firmware flags */
u32 firmware_flags;
/* pointer to backlight encoder */
@@ -631,15 +620,6 @@ bool amdgpu_crtc_get_scanout_position(struct drm_crtc *crtc,
int *hpos, ktime_t *stime, ktime_t *etime,
const struct drm_display_mode *mode);
-/* fbdev layer */
-int amdgpu_fbdev_init(struct amdgpu_device *adev);
-void amdgpu_fbdev_fini(struct amdgpu_device *adev);
-void amdgpu_fbdev_set_suspend(struct amdgpu_device *adev, int state);
-int amdgpu_fbdev_total_size(struct amdgpu_device *adev);
-bool amdgpu_fbdev_robj_is_fb(struct amdgpu_device *adev, struct amdgpu_bo *robj);
-
-int amdgpu_align_pitch(struct amdgpu_device *adev, int width, int bpp, bool tiled);
-
/* amdgpu_display.c */
void amdgpu_display_print_display_setup(struct drm_device *dev);
int amdgpu_display_modeset_create_props(struct amdgpu_device *adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
index 4fcfc23..3a7b56e 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
@@ -1032,9 +1032,14 @@ int amdgpu_bo_init(struct amdgpu_device *adev)
/* On A+A platform, VRAM can be mapped as WB */
if (!adev->gmc.xgmi.connected_to_cpu) {
/* reserve PAT memory space to WC for VRAM */
- arch_io_reserve_memtype_wc(adev->gmc.aper_base,
+ int r = arch_io_reserve_memtype_wc(adev->gmc.aper_base,
adev->gmc.aper_size);
+ if (r) {
+ DRM_ERROR("Unable to set WC memtype for the aperture base\n");
+ return r;
+ }
+
/* Add an MTRR for the VRAM */
adev->gmc.vram_mtrr = arch_phys_wc_add(adev->gmc.aper_base,
adev->gmc.aper_size);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_pll.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_pll.c
index 4eaec44..0bb2466 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_pll.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_pll.c
@@ -69,6 +69,7 @@ static void amdgpu_pll_reduce_ratio(unsigned *nom, unsigned *den,
/**
* amdgpu_pll_get_fb_ref_div - feedback and ref divider calculation
*
+ * @adev: amdgpu_device pointer
* @nom: nominator
* @den: denominator
* @post_div: post divider
@@ -106,6 +107,7 @@ static void amdgpu_pll_get_fb_ref_div(struct amdgpu_device *adev, unsigned int n
/**
* amdgpu_pll_compute - compute PLL paramaters
*
+ * @adev: amdgpu_device pointer
* @pll: information about the PLL
* @freq: requested frequency
* @dot_clock_p: resulting pixel clock
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_pmu.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_pmu.c
index 82e9ecf..71ee361 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_pmu.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_pmu.c
@@ -233,6 +233,10 @@ static void amdgpu_perf_start(struct perf_event *event, int flags)
if (WARN_ON_ONCE(!(hwc->state & PERF_HES_STOPPED)))
return;
+ if ((!pe->adev->df.funcs) ||
+ (!pe->adev->df.funcs->pmc_start))
+ return;
+
WARN_ON_ONCE(!(hwc->state & PERF_HES_UPTODATE));
hwc->state = 0;
@@ -268,6 +272,10 @@ static void amdgpu_perf_read(struct perf_event *event)
pmu);
u64 count, prev;
+ if ((!pe->adev->df.funcs) ||
+ (!pe->adev->df.funcs->pmc_get_count))
+ return;
+
do {
prev = local64_read(&hwc->prev_count);
@@ -297,6 +305,10 @@ static void amdgpu_perf_stop(struct perf_event *event, int flags)
if (hwc->state & PERF_HES_UPTODATE)
return;
+ if ((!pe->adev->df.funcs) ||
+ (!pe->adev->df.funcs->pmc_stop))
+ return;
+
switch (hwc->config_base) {
case AMDGPU_PMU_EVENT_CONFIG_TYPE_DF:
case AMDGPU_PMU_EVENT_CONFIG_TYPE_XGMI:
@@ -326,6 +338,10 @@ static int amdgpu_perf_add(struct perf_event *event, int flags)
struct amdgpu_pmu_entry,
pmu);
+ if ((!pe->adev->df.funcs) ||
+ (!pe->adev->df.funcs->pmc_start))
+ return -EINVAL;
+
switch (pe->pmu_perf_type) {
case AMDGPU_PMU_PERF_TYPE_DF:
hwc->config_base = AMDGPU_PMU_EVENT_CONFIG_TYPE_DF;
@@ -371,6 +387,9 @@ static void amdgpu_perf_del(struct perf_event *event, int flags)
struct amdgpu_pmu_entry *pe = container_of(event->pmu,
struct amdgpu_pmu_entry,
pmu);
+ if ((!pe->adev->df.funcs) ||
+ (!pe->adev->df.funcs->pmc_stop))
+ return;
amdgpu_perf_stop(event, PERF_EF_UPDATE);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_preempt_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_preempt_mgr.c
index d02c863..786afe4 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_preempt_mgr.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_preempt_mgr.c
@@ -59,7 +59,7 @@ static DEVICE_ATTR_RO(mem_info_preempt_used);
* @man: TTM memory type manager
* @tbo: TTM BO we need this range for
* @place: placement flags and restrictions
- * @mem: the resulting mem object
+ * @res: TTM memory object
*
* Dummy, just count the space used without allocating resources or any limit.
*/
@@ -85,7 +85,7 @@ static int amdgpu_preempt_mgr_new(struct ttm_resource_manager *man,
* amdgpu_preempt_mgr_del - free ranges
*
* @man: TTM memory type manager
- * @mem: TTM memory object
+ * @res: TTM memory object
*
* Free the allocated GTT again.
*/
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
index c641f84..dee17a0 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
@@ -518,7 +518,7 @@ static struct psp_gfx_cmd_resp *acquire_psp_cmd_buf(struct psp_context *psp)
return cmd;
}
-void release_psp_cmd_buf(struct psp_context *psp)
+static void release_psp_cmd_buf(struct psp_context *psp)
{
mutex_unlock(&psp->mutex);
}
@@ -2017,12 +2017,16 @@ static int psp_hw_start(struct psp_context *psp)
return ret;
}
+ if (amdgpu_sriov_vf(adev) && amdgpu_in_reset(adev))
+ goto skip_pin_bo;
+
ret = psp_tmr_init(psp);
if (ret) {
DRM_ERROR("PSP tmr init failed!\n");
return ret;
}
+skip_pin_bo:
/*
* For ASICs with DF Cstate management centralized
* to PMFW, TMR setup should be performed after PMFW
@@ -2452,6 +2456,18 @@ static int psp_load_fw(struct amdgpu_device *adev)
return ret;
}
+ if (amdgpu_sriov_vf(adev) && amdgpu_in_reset(adev)) {
+ if (adev->gmc.xgmi.num_physical_nodes > 1) {
+ ret = psp_xgmi_initialize(psp, false, true);
+ /* Warning the XGMI seesion initialize failure
+ * Instead of stop driver initialization
+ */
+ if (ret)
+ dev_err(psp->adev->dev,
+ "XGMI: Failed to initialize XGMI session\n");
+ }
+ }
+
if (psp->ta_fw) {
ret = psp_ras_initialize(psp);
if (ret)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
index 08133de..91e6e87 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
@@ -867,9 +867,9 @@ static int amdgpu_ras_enable_all_features(struct amdgpu_device *adev,
/* feature ctl end */
-void amdgpu_ras_mca_query_error_status(struct amdgpu_device *adev,
- struct ras_common_if *ras_block,
- struct ras_err_data *err_data)
+static void amdgpu_ras_mca_query_error_status(struct amdgpu_device *adev,
+ struct ras_common_if *ras_block,
+ struct ras_err_data *err_data)
{
switch (ras_block->sub_block_index) {
case AMDGPU_RAS_MCA_BLOCK__MP0:
@@ -892,6 +892,38 @@ void amdgpu_ras_mca_query_error_status(struct amdgpu_device *adev,
}
}
+static void amdgpu_ras_get_ecc_info(struct amdgpu_device *adev, struct ras_err_data *err_data)
+{
+ struct amdgpu_ras *ras = amdgpu_ras_get_context(adev);
+ int ret = 0;
+
+ /*
+ * choosing right query method according to
+ * whether smu support query error information
+ */
+ ret = smu_get_ecc_info(&adev->smu, (void *)&(ras->umc_ecc));
+ if (ret == -EOPNOTSUPP) {
+ if (adev->umc.ras_funcs &&
+ adev->umc.ras_funcs->query_ras_error_count)
+ adev->umc.ras_funcs->query_ras_error_count(adev, err_data);
+
+ /* umc query_ras_error_address is also responsible for clearing
+ * error status
+ */
+ if (adev->umc.ras_funcs &&
+ adev->umc.ras_funcs->query_ras_error_address)
+ adev->umc.ras_funcs->query_ras_error_address(adev, err_data);
+ } else if (!ret) {
+ if (adev->umc.ras_funcs &&
+ adev->umc.ras_funcs->ecc_info_query_ras_error_count)
+ adev->umc.ras_funcs->ecc_info_query_ras_error_count(adev, err_data);
+
+ if (adev->umc.ras_funcs &&
+ adev->umc.ras_funcs->ecc_info_query_ras_error_address)
+ adev->umc.ras_funcs->ecc_info_query_ras_error_address(adev, err_data);
+ }
+}
+
/* query/inject/cure begin */
int amdgpu_ras_query_error_status(struct amdgpu_device *adev,
struct ras_query_if *info)
@@ -905,15 +937,7 @@ int amdgpu_ras_query_error_status(struct amdgpu_device *adev,
switch (info->head.block) {
case AMDGPU_RAS_BLOCK__UMC:
- if (adev->umc.ras_funcs &&
- adev->umc.ras_funcs->query_ras_error_count)
- adev->umc.ras_funcs->query_ras_error_count(adev, &err_data);
- /* umc query_ras_error_address is also responsible for clearing
- * error status
- */
- if (adev->umc.ras_funcs &&
- adev->umc.ras_funcs->query_ras_error_address)
- adev->umc.ras_funcs->query_ras_error_address(adev, &err_data);
+ amdgpu_ras_get_ecc_info(adev, &err_data);
break;
case AMDGPU_RAS_BLOCK__SDMA:
if (adev->sdma.funcs->query_ras_error_count) {
@@ -1137,9 +1161,9 @@ int amdgpu_ras_error_inject(struct amdgpu_device *adev,
/**
* amdgpu_ras_query_error_count -- Get error counts of all IPs
- * adev: pointer to AMD GPU device
- * ce_count: pointer to an integer to be set to the count of correctible errors.
- * ue_count: pointer to an integer to be set to the count of uncorrectible
+ * @adev: pointer to AMD GPU device
+ * @ce_count: pointer to an integer to be set to the count of correctible errors.
+ * @ue_count: pointer to an integer to be set to the count of uncorrectible
* errors.
*
* If set, @ce_count or @ue_count, count and return the corresponding
@@ -1723,6 +1747,16 @@ static void amdgpu_ras_log_on_err_counter(struct amdgpu_device *adev)
if (info.head.block == AMDGPU_RAS_BLOCK__PCIE_BIF)
continue;
+ /*
+ * this is a workaround for aldebaran, skip send msg to
+ * smu to get ecc_info table due to smu handle get ecc
+ * info table failed temporarily.
+ * should be removed until smu fix handle ecc_info table.
+ */
+ if ((info.head.block == AMDGPU_RAS_BLOCK__UMC) &&
+ (adev->ip_versions[MP1_HWIP][0] == IP_VERSION(13, 0, 2)))
+ continue;
+
amdgpu_ras_query_error_status(adev, &info);
}
}
@@ -1935,9 +1969,11 @@ int amdgpu_ras_save_bad_pages(struct amdgpu_device *adev)
if (!con || !con->eh_data)
return 0;
+ mutex_lock(&con->recovery_lock);
control = &con->eeprom_control;
data = con->eh_data;
save_count = data->count - control->ras_num_recs;
+ mutex_unlock(&con->recovery_lock);
/* only new entries are saved */
if (save_count > 0) {
if (amdgpu_ras_eeprom_append(control,
@@ -2336,7 +2372,11 @@ int amdgpu_ras_init(struct amdgpu_device *adev)
}
/* Init poison supported flag, the default value is false */
- if (adev->df.funcs &&
+ if (adev->gmc.xgmi.connected_to_cpu) {
+ /* enabled by default when GPU is connected to CPU */
+ con->poison_supported = true;
+ }
+ else if (adev->df.funcs &&
adev->df.funcs->query_ras_poison_mode &&
adev->umc.ras_funcs &&
adev->umc.ras_funcs->query_ras_poison_mode) {
@@ -2477,7 +2517,6 @@ void amdgpu_ras_late_fini(struct amdgpu_device *adev,
amdgpu_ras_sysfs_remove(adev, ras_block);
if (ih_info->cb)
amdgpu_ras_interrupt_remove_handler(adev, ih_info);
- amdgpu_ras_feature_enable(adev, ras_block, 0);
}
/* do some init work after IP late init as dependence.
@@ -2647,7 +2686,7 @@ static int amdgpu_bad_page_notifier(struct notifier_block *nb,
* and error occurred in DramECC (Extended error code = 0) then only
* process the error, else bail out.
*/
- if (!m || !((smca_get_bank_type(m->bank) == SMCA_UMC_V2) &&
+ if (!m || !((smca_get_bank_type(m->extcpu, m->bank) == SMCA_UMC_V2) &&
(XEC(m->status, 0x3f) == 0x0)))
return NOTIFY_DONE;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.h
index e36f4de..1c70812 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.h
@@ -319,6 +319,19 @@ struct ras_common_if {
char name[32];
};
+#define MAX_UMC_CHANNEL_NUM 32
+
+struct ecc_info_per_ch {
+ uint16_t ce_count_lo_chip;
+ uint16_t ce_count_hi_chip;
+ uint64_t mca_umc_status;
+ uint64_t mca_umc_addr;
+};
+
+struct umc_ecc_info {
+ struct ecc_info_per_ch ecc[MAX_UMC_CHANNEL_NUM];
+};
+
struct amdgpu_ras {
/* ras infrastructure */
/* for ras itself. */
@@ -358,6 +371,9 @@ struct amdgpu_ras {
struct delayed_work ras_counte_delay_work;
atomic_t ras_ue_count;
atomic_t ras_ce_count;
+
+ /* record umc error info queried from smu */
+ struct umc_ecc_info umc_ecc;
};
struct ras_fs_data {
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h
index 4d380e7..fae7d18 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h
@@ -53,9 +53,6 @@ enum amdgpu_ring_priority_level {
#define AMDGPU_FENCE_FLAG_INT (1 << 1)
#define AMDGPU_FENCE_FLAG_TC_WB_ONLY (1 << 2)
-/* fence flag bit to indicate the face is embedded in job*/
-#define AMDGPU_FENCE_FLAG_EMBED_IN_JOB_BIT (DMA_FENCE_FLAG_USER_BITS + 1)
-
#define to_amdgpu_ring(s) container_of((s), struct amdgpu_ring, sched)
#define AMDGPU_IB_POOL_SIZE (1024 * 1024)
@@ -114,6 +111,7 @@ struct amdgpu_fence_driver {
struct dma_fence **fences;
};
+void amdgpu_fence_driver_clear_job_fences(struct amdgpu_ring *ring);
void amdgpu_fence_driver_force_completion(struct amdgpu_ring *ring);
int amdgpu_fence_driver_init_ring(struct amdgpu_ring *ring,
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c
index 862eb3c..f7d8487 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c
@@ -252,41 +252,25 @@ int amdgpu_sync_resv(struct amdgpu_device *adev, struct amdgpu_sync *sync,
struct dma_resv *resv, enum amdgpu_sync_mode mode,
void *owner)
{
- struct dma_resv_list *flist;
+ struct dma_resv_iter cursor;
struct dma_fence *f;
- unsigned i;
- int r = 0;
+ int r;
if (resv == NULL)
return -EINVAL;
- /* always sync to the exclusive fence */
- f = dma_resv_excl_fence(resv);
- dma_fence_chain_for_each(f, f) {
- struct dma_fence_chain *chain = to_dma_fence_chain(f);
+ dma_resv_for_each_fence(&cursor, resv, true, f) {
+ dma_fence_chain_for_each(f, f) {
+ struct dma_fence_chain *chain = to_dma_fence_chain(f);
- if (amdgpu_sync_test_fence(adev, mode, owner, chain ?
- chain->fence : f)) {
- r = amdgpu_sync_fence(sync, f);
- dma_fence_put(f);
- if (r)
- return r;
- break;
- }
- }
-
- flist = dma_resv_shared_list(resv);
- if (!flist)
- return 0;
-
- for (i = 0; i < flist->shared_count; ++i) {
- f = rcu_dereference_protected(flist->shared[i],
- dma_resv_held(resv));
-
- if (amdgpu_sync_test_fence(adev, mode, owner, f)) {
- r = amdgpu_sync_fence(sync, f);
- if (r)
- return r;
+ if (amdgpu_sync_test_fence(adev, mode, owner, chain ?
+ chain->fence : f)) {
+ r = amdgpu_sync_fence(sync, f);
+ dma_fence_put(f);
+ if (r)
+ return r;
+ break;
+ }
}
}
return 0;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
index c875f1c..fb0d8bf 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
@@ -116,17 +116,8 @@ static void amdgpu_evict_flags(struct ttm_buffer_object *bo,
abo = ttm_to_amdgpu_bo(bo);
if (abo->flags & AMDGPU_AMDKFD_CREATE_SVM_BO) {
- struct dma_fence *fence;
- struct dma_resv *resv = &bo->base._resv;
-
- rcu_read_lock();
- fence = rcu_dereference(resv->fence_excl);
- if (fence && !fence->ops->signaled)
- dma_fence_enable_sw_signaling(fence);
-
placement->num_placement = 0;
placement->num_busy_placement = 0;
- rcu_read_unlock();
return;
}
@@ -922,11 +913,6 @@ static int amdgpu_ttm_backend_bind(struct ttm_device *bdev,
ttm->num_pages, bo_mem, ttm);
}
- if (bo_mem->mem_type == AMDGPU_PL_GDS ||
- bo_mem->mem_type == AMDGPU_PL_GWS ||
- bo_mem->mem_type == AMDGPU_PL_OA)
- return -EINVAL;
-
if (bo_mem->mem_type != TTM_PL_TT ||
!amdgpu_gtt_mgr_has_gart_addr(bo_mem)) {
gtt->offset = AMDGPU_BO_INVALID_OFFSET;
@@ -1353,10 +1339,9 @@ static bool amdgpu_ttm_bo_eviction_valuable(struct ttm_buffer_object *bo,
const struct ttm_place *place)
{
unsigned long num_pages = bo->resource->num_pages;
+ struct dma_resv_iter resv_cursor;
struct amdgpu_res_cursor cursor;
- struct dma_resv_list *flist;
struct dma_fence *f;
- int i;
/* Swapout? */
if (bo->resource->mem_type == TTM_PL_SYSTEM)
@@ -1370,14 +1355,9 @@ static bool amdgpu_ttm_bo_eviction_valuable(struct ttm_buffer_object *bo,
* If true, then return false as any KFD process needs all its BOs to
* be resident to run successfully
*/
- flist = dma_resv_shared_list(bo->base.resv);
- if (flist) {
- for (i = 0; i < flist->shared_count; ++i) {
- f = rcu_dereference_protected(flist->shared[i],
- dma_resv_held(bo->base.resv));
- if (amdkfd_fence_check_mm(f, current->mm))
- return false;
- }
+ dma_resv_for_each_fence(&resv_cursor, bo->base.resv, true, f) {
+ if (amdkfd_fence_check_mm(f, current->mm))
+ return false;
}
switch (bo->resource->mem_type) {
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_umc.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_umc.c
index a90029ee..46264a4 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_umc.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_umc.c
@@ -23,6 +23,120 @@
#include "amdgpu_ras.h"
+static int amdgpu_umc_do_page_retirement(struct amdgpu_device *adev,
+ void *ras_error_status,
+ struct amdgpu_iv_entry *entry,
+ bool reset)
+{
+ struct ras_err_data *err_data = (struct ras_err_data *)ras_error_status;
+ struct amdgpu_ras *con = amdgpu_ras_get_context(adev);
+ int ret = 0;
+
+ kgd2kfd_set_sram_ecc_flag(adev->kfd.dev);
+ ret = smu_get_ecc_info(&adev->smu, (void *)&(con->umc_ecc));
+ if (ret == -EOPNOTSUPP) {
+ if (adev->umc.ras_funcs &&
+ adev->umc.ras_funcs->query_ras_error_count)
+ adev->umc.ras_funcs->query_ras_error_count(adev, ras_error_status);
+
+ if (adev->umc.ras_funcs &&
+ adev->umc.ras_funcs->query_ras_error_address &&
+ adev->umc.max_ras_err_cnt_per_query) {
+ err_data->err_addr =
+ kcalloc(adev->umc.max_ras_err_cnt_per_query,
+ sizeof(struct eeprom_table_record), GFP_KERNEL);
+
+ /* still call query_ras_error_address to clear error status
+ * even NOMEM error is encountered
+ */
+ if(!err_data->err_addr)
+ dev_warn(adev->dev, "Failed to alloc memory for "
+ "umc error address record!\n");
+
+ /* umc query_ras_error_address is also responsible for clearing
+ * error status
+ */
+ adev->umc.ras_funcs->query_ras_error_address(adev, ras_error_status);
+ }
+ } else if (!ret) {
+ if (adev->umc.ras_funcs &&
+ adev->umc.ras_funcs->ecc_info_query_ras_error_count)
+ adev->umc.ras_funcs->ecc_info_query_ras_error_count(adev, ras_error_status);
+
+ if (adev->umc.ras_funcs &&
+ adev->umc.ras_funcs->ecc_info_query_ras_error_address &&
+ adev->umc.max_ras_err_cnt_per_query) {
+ err_data->err_addr =
+ kcalloc(adev->umc.max_ras_err_cnt_per_query,
+ sizeof(struct eeprom_table_record), GFP_KERNEL);
+
+ /* still call query_ras_error_address to clear error status
+ * even NOMEM error is encountered
+ */
+ if(!err_data->err_addr)
+ dev_warn(adev->dev, "Failed to alloc memory for "
+ "umc error address record!\n");
+
+ /* umc query_ras_error_address is also responsible for clearing
+ * error status
+ */
+ adev->umc.ras_funcs->ecc_info_query_ras_error_address(adev, ras_error_status);
+ }
+ }
+
+ /* only uncorrectable error needs gpu reset */
+ if (err_data->ue_count) {
+ dev_info(adev->dev, "%ld uncorrectable hardware errors "
+ "detected in UMC block\n",
+ err_data->ue_count);
+
+ if ((amdgpu_bad_page_threshold != 0) &&
+ err_data->err_addr_cnt) {
+ amdgpu_ras_add_bad_pages(adev, err_data->err_addr,
+ err_data->err_addr_cnt);
+ amdgpu_ras_save_bad_pages(adev);
+
+ if (adev->smu.ppt_funcs && adev->smu.ppt_funcs->send_hbm_bad_pages_num)
+ adev->smu.ppt_funcs->send_hbm_bad_pages_num(&adev->smu, con->eeprom_control.ras_num_recs);
+ }
+
+ if (reset)
+ amdgpu_ras_reset_gpu(adev);
+ }
+
+ kfree(err_data->err_addr);
+ return AMDGPU_RAS_SUCCESS;
+}
+
+int amdgpu_umc_poison_handler(struct amdgpu_device *adev,
+ void *ras_error_status,
+ bool reset)
+{
+ int ret;
+ struct ras_err_data *err_data = (struct ras_err_data *)ras_error_status;
+ struct ras_common_if head = {
+ .block = AMDGPU_RAS_BLOCK__UMC,
+ };
+ struct ras_manager *obj = amdgpu_ras_find_obj(adev, &head);
+
+ ret =
+ amdgpu_umc_do_page_retirement(adev, ras_error_status, NULL, reset);
+
+ if (ret == AMDGPU_RAS_SUCCESS && obj) {
+ obj->err_data.ue_count += err_data->ue_count;
+ obj->err_data.ce_count += err_data->ce_count;
+ }
+
+ return ret;
+}
+
+static int amdgpu_umc_process_ras_data_cb(struct amdgpu_device *adev,
+ void *ras_error_status,
+ struct amdgpu_iv_entry *entry)
+{
+ return amdgpu_umc_do_page_retirement(adev, ras_error_status, entry, true);
+}
+
int amdgpu_umc_ras_late_init(struct amdgpu_device *adev)
{
int r;
@@ -88,61 +202,6 @@ void amdgpu_umc_ras_fini(struct amdgpu_device *adev)
}
}
-int amdgpu_umc_process_ras_data_cb(struct amdgpu_device *adev,
- void *ras_error_status,
- struct amdgpu_iv_entry *entry)
-{
- struct ras_err_data *err_data = (struct ras_err_data *)ras_error_status;
- struct amdgpu_ras *con = amdgpu_ras_get_context(adev);
-
- kgd2kfd_set_sram_ecc_flag(adev->kfd.dev);
- if (adev->umc.ras_funcs &&
- adev->umc.ras_funcs->query_ras_error_count)
- adev->umc.ras_funcs->query_ras_error_count(adev, ras_error_status);
-
- if (adev->umc.ras_funcs &&
- adev->umc.ras_funcs->query_ras_error_address &&
- adev->umc.max_ras_err_cnt_per_query) {
- err_data->err_addr =
- kcalloc(adev->umc.max_ras_err_cnt_per_query,
- sizeof(struct eeprom_table_record), GFP_KERNEL);
-
- /* still call query_ras_error_address to clear error status
- * even NOMEM error is encountered
- */
- if(!err_data->err_addr)
- dev_warn(adev->dev, "Failed to alloc memory for "
- "umc error address record!\n");
-
- /* umc query_ras_error_address is also responsible for clearing
- * error status
- */
- adev->umc.ras_funcs->query_ras_error_address(adev, ras_error_status);
- }
-
- /* only uncorrectable error needs gpu reset */
- if (err_data->ue_count) {
- dev_info(adev->dev, "%ld uncorrectable hardware errors "
- "detected in UMC block\n",
- err_data->ue_count);
-
- if ((amdgpu_bad_page_threshold != 0) &&
- err_data->err_addr_cnt) {
- amdgpu_ras_add_bad_pages(adev, err_data->err_addr,
- err_data->err_addr_cnt);
- amdgpu_ras_save_bad_pages(adev);
-
- if (adev->smu.ppt_funcs && adev->smu.ppt_funcs->send_hbm_bad_pages_num)
- adev->smu.ppt_funcs->send_hbm_bad_pages_num(&adev->smu, con->eeprom_control.ras_num_recs);
- }
-
- amdgpu_ras_reset_gpu(adev);
- }
-
- kfree(err_data->err_addr);
- return AMDGPU_RAS_SUCCESS;
-}
-
int amdgpu_umc_process_ecc_irq(struct amdgpu_device *adev,
struct amdgpu_irq_src *source,
struct amdgpu_iv_entry *entry)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_umc.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_umc.h
index 1f5fe23..b72194e 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_umc.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_umc.h
@@ -49,6 +49,10 @@ struct amdgpu_umc_ras_funcs {
void (*query_ras_error_address)(struct amdgpu_device *adev,
void *ras_error_status);
bool (*query_ras_poison_mode)(struct amdgpu_device *adev);
+ void (*ecc_info_query_ras_error_count)(struct amdgpu_device *adev,
+ void *ras_error_status);
+ void (*ecc_info_query_ras_error_address)(struct amdgpu_device *adev,
+ void *ras_error_status);
};
struct amdgpu_umc_funcs {
@@ -74,9 +78,9 @@ struct amdgpu_umc {
int amdgpu_umc_ras_late_init(struct amdgpu_device *adev);
void amdgpu_umc_ras_fini(struct amdgpu_device *adev);
-int amdgpu_umc_process_ras_data_cb(struct amdgpu_device *adev,
+int amdgpu_umc_poison_handler(struct amdgpu_device *adev,
void *ras_error_status,
- struct amdgpu_iv_entry *entry);
+ bool reset);
int amdgpu_umc_process_ecc_irq(struct amdgpu_device *adev,
struct amdgpu_irq_src *source,
struct amdgpu_iv_entry *entry);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c
index 688bef1..344f711 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c
@@ -434,7 +434,6 @@ void amdgpu_vce_free_handles(struct amdgpu_device *adev, struct drm_file *filp)
*
* @ring: ring we should submit the msg to
* @handle: VCE session handle to use
- * @bo: amdgpu object for which we query the offset
* @fence: optional fence to return
*
* Open up a stream for HW test
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c
index 585961c2..9a19a6a 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c
@@ -286,20 +286,13 @@ int amdgpu_vcn_sw_fini(struct amdgpu_device *adev)
bool amdgpu_vcn_is_disabled_vcn(struct amdgpu_device *adev, enum vcn_ring_type type, uint32_t vcn_instance)
{
bool ret = false;
+ int vcn_config = adev->vcn.vcn_config[vcn_instance];
- int major;
- int minor;
- int revision;
-
- /* if cannot find IP data, then this VCN does not exist */
- if (amdgpu_discovery_get_vcn_version(adev, vcn_instance, &major, &minor, &revision) != 0)
- return true;
-
- if ((type == VCN_ENCODE_RING) && (revision & VCN_BLOCK_ENCODE_DISABLE_MASK)) {
+ if ((type == VCN_ENCODE_RING) && (vcn_config & VCN_BLOCK_ENCODE_DISABLE_MASK)) {
ret = true;
- } else if ((type == VCN_DECODE_RING) && (revision & VCN_BLOCK_DECODE_DISABLE_MASK)) {
+ } else if ((type == VCN_DECODE_RING) && (vcn_config & VCN_BLOCK_DECODE_DISABLE_MASK)) {
ret = true;
- } else if ((type == VCN_UNIFIED_RING) && (revision & VCN_BLOCK_QUEUE_DISABLE_MASK)) {
+ } else if ((type == VCN_UNIFIED_RING) && (vcn_config & VCN_BLOCK_QUEUE_DISABLE_MASK)) {
ret = true;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.h
index bfa27ea..5d3728b 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.h
@@ -235,6 +235,7 @@ struct amdgpu_vcn {
uint8_t num_vcn_inst;
struct amdgpu_vcn_inst inst[AMDGPU_MAX_VCN_INSTANCES];
+ uint8_t vcn_config[AMDGPU_MAX_VCN_INSTANCES];
struct amdgpu_vcn_reg internal;
struct mutex vcn_pg_lock;
struct mutex vcn1_jpeg1_workaround;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
index 04cf9b2..f8e574c 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
@@ -283,17 +283,15 @@ static int amdgpu_virt_init_ras_err_handler_data(struct amdgpu_device *adev)
*data = kmalloc(sizeof(struct amdgpu_virt_ras_err_handler_data), GFP_KERNEL);
if (!*data)
- return -ENOMEM;
+ goto data_failure;
bps = kmalloc_array(align_space, sizeof((*data)->bps), GFP_KERNEL);
- bps_bo = kmalloc_array(align_space, sizeof((*data)->bps_bo), GFP_KERNEL);
+ if (!bps)
+ goto bps_failure;
- if (!bps || !bps_bo) {
- kfree(bps);
- kfree(bps_bo);
- kfree(*data);
- return -ENOMEM;
- }
+ bps_bo = kmalloc_array(align_space, sizeof((*data)->bps_bo), GFP_KERNEL);
+ if (!bps_bo)
+ goto bps_bo_failure;
(*data)->bps = bps;
(*data)->bps_bo = bps_bo;
@@ -303,6 +301,13 @@ static int amdgpu_virt_init_ras_err_handler_data(struct amdgpu_device *adev)
virt->ras_init_done = true;
return 0;
+
+bps_bo_failure:
+ kfree(bps);
+bps_failure:
+ kfree(*data);
+data_failure:
+ return -ENOMEM;
}
static void amdgpu_virt_ras_release_bp(struct amdgpu_device *adev)
@@ -617,19 +622,37 @@ void amdgpu_virt_fini_data_exchange(struct amdgpu_device *adev)
void amdgpu_virt_init_data_exchange(struct amdgpu_device *adev)
{
- uint64_t bp_block_offset = 0;
- uint32_t bp_block_size = 0;
- struct amd_sriov_msg_pf2vf_info *pf2vf_v2 = NULL;
-
adev->virt.fw_reserve.p_pf2vf = NULL;
adev->virt.fw_reserve.p_vf2pf = NULL;
adev->virt.vf2pf_update_interval_ms = 0;
- if (adev->mman.fw_vram_usage_va != NULL) {
+ if (adev->bios != NULL) {
adev->virt.vf2pf_update_interval_ms = 2000;
adev->virt.fw_reserve.p_pf2vf =
(struct amd_sriov_msg_pf2vf_info_header *)
+ (adev->bios + (AMD_SRIOV_MSG_PF2VF_OFFSET_KB << 10));
+
+ amdgpu_virt_read_pf2vf_data(adev);
+ }
+
+ if (adev->virt.vf2pf_update_interval_ms != 0) {
+ INIT_DELAYED_WORK(&adev->virt.vf2pf_work, amdgpu_virt_update_vf2pf_work_item);
+ schedule_delayed_work(&(adev->virt.vf2pf_work), msecs_to_jiffies(adev->virt.vf2pf_update_interval_ms));
+ }
+}
+
+
+void amdgpu_virt_exchange_data(struct amdgpu_device *adev)
+{
+ uint64_t bp_block_offset = 0;
+ uint32_t bp_block_size = 0;
+ struct amd_sriov_msg_pf2vf_info *pf2vf_v2 = NULL;
+
+ if (adev->mman.fw_vram_usage_va != NULL) {
+
+ adev->virt.fw_reserve.p_pf2vf =
+ (struct amd_sriov_msg_pf2vf_info_header *)
(adev->mman.fw_vram_usage_va + (AMD_SRIOV_MSG_PF2VF_OFFSET_KB << 10));
adev->virt.fw_reserve.p_vf2pf =
(struct amd_sriov_msg_vf2pf_info_header *)
@@ -658,16 +681,10 @@ void amdgpu_virt_init_data_exchange(struct amdgpu_device *adev)
(adev->bios + (AMD_SRIOV_MSG_PF2VF_OFFSET_KB << 10));
amdgpu_virt_read_pf2vf_data(adev);
-
- return;
- }
-
- if (adev->virt.vf2pf_update_interval_ms != 0) {
- INIT_DELAYED_WORK(&adev->virt.vf2pf_work, amdgpu_virt_update_vf2pf_work_item);
- schedule_delayed_work(&(adev->virt.vf2pf_work), adev->virt.vf2pf_update_interval_ms);
}
}
+
void amdgpu_detect_virtualization(struct amdgpu_device *adev)
{
uint32_t reg;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h
index 8d4c20b..9adfb8d 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h
@@ -308,6 +308,7 @@ int amdgpu_virt_alloc_mm_table(struct amdgpu_device *adev);
void amdgpu_virt_free_mm_table(struct amdgpu_device *adev);
void amdgpu_virt_release_ras_err_handler_data(struct amdgpu_device *adev);
void amdgpu_virt_init_data_exchange(struct amdgpu_device *adev);
+void amdgpu_virt_exchange_data(struct amdgpu_device *adev);
void amdgpu_virt_fini_data_exchange(struct amdgpu_device *adev);
void amdgpu_detect_virtualization(struct amdgpu_device *adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vkms.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vkms.c
index ac9a8cd..2dcc68e 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vkms.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vkms.c
@@ -16,6 +16,8 @@
#include "ivsrcid/ivsrcid_vislands30.h"
#include "amdgpu_vkms.h"
#include "amdgpu_display.h"
+#include "atom.h"
+#include "amdgpu_irq.h"
/**
* DOC: amdgpu_vkms
@@ -41,16 +43,16 @@ static const u32 amdgpu_vkms_formats[] = {
static enum hrtimer_restart amdgpu_vkms_vblank_simulate(struct hrtimer *timer)
{
- struct amdgpu_vkms_output *output = container_of(timer,
- struct amdgpu_vkms_output,
- vblank_hrtimer);
- struct drm_crtc *crtc = &output->crtc;
+ struct amdgpu_crtc *amdgpu_crtc = container_of(timer, struct amdgpu_crtc, vblank_timer);
+ struct drm_crtc *crtc = &amdgpu_crtc->base;
+ struct amdgpu_vkms_output *output = drm_crtc_to_amdgpu_vkms_output(crtc);
u64 ret_overrun;
bool ret;
- ret_overrun = hrtimer_forward_now(&output->vblank_hrtimer,
+ ret_overrun = hrtimer_forward_now(&amdgpu_crtc->vblank_timer,
output->period_ns);
- WARN_ON(ret_overrun != 1);
+ if (ret_overrun != 1)
+ DRM_WARN("%s: vblank timer overrun\n", __func__);
ret = drm_crtc_handle_vblank(crtc);
if (!ret)
@@ -65,22 +67,21 @@ static int amdgpu_vkms_enable_vblank(struct drm_crtc *crtc)
unsigned int pipe = drm_crtc_index(crtc);
struct drm_vblank_crtc *vblank = &dev->vblank[pipe];
struct amdgpu_vkms_output *out = drm_crtc_to_amdgpu_vkms_output(crtc);
+ struct amdgpu_crtc *amdgpu_crtc = to_amdgpu_crtc(crtc);
drm_calc_timestamping_constants(crtc, &crtc->mode);
- hrtimer_init(&out->vblank_hrtimer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
- out->vblank_hrtimer.function = &amdgpu_vkms_vblank_simulate;
out->period_ns = ktime_set(0, vblank->framedur_ns);
- hrtimer_start(&out->vblank_hrtimer, out->period_ns, HRTIMER_MODE_REL);
+ hrtimer_start(&amdgpu_crtc->vblank_timer, out->period_ns, HRTIMER_MODE_REL);
return 0;
}
static void amdgpu_vkms_disable_vblank(struct drm_crtc *crtc)
{
- struct amdgpu_vkms_output *out = drm_crtc_to_amdgpu_vkms_output(crtc);
+ struct amdgpu_crtc *amdgpu_crtc = to_amdgpu_crtc(crtc);
- hrtimer_cancel(&out->vblank_hrtimer);
+ hrtimer_cancel(&amdgpu_crtc->vblank_timer);
}
static bool amdgpu_vkms_get_vblank_timestamp(struct drm_crtc *crtc,
@@ -92,13 +93,14 @@ static bool amdgpu_vkms_get_vblank_timestamp(struct drm_crtc *crtc,
unsigned int pipe = crtc->index;
struct amdgpu_vkms_output *output = drm_crtc_to_amdgpu_vkms_output(crtc);
struct drm_vblank_crtc *vblank = &dev->vblank[pipe];
+ struct amdgpu_crtc *amdgpu_crtc = to_amdgpu_crtc(crtc);
if (!READ_ONCE(vblank->enabled)) {
*vblank_time = ktime_get();
return true;
}
- *vblank_time = READ_ONCE(output->vblank_hrtimer.node.expires);
+ *vblank_time = READ_ONCE(amdgpu_crtc->vblank_timer.node.expires);
if (WARN_ON(*vblank_time == vblank->time))
return true;
@@ -165,6 +167,8 @@ static const struct drm_crtc_helper_funcs amdgpu_vkms_crtc_helper_funcs = {
static int amdgpu_vkms_crtc_init(struct drm_device *dev, struct drm_crtc *crtc,
struct drm_plane *primary, struct drm_plane *cursor)
{
+ struct amdgpu_device *adev = drm_to_adev(dev);
+ struct amdgpu_crtc *amdgpu_crtc = to_amdgpu_crtc(crtc);
int ret;
ret = drm_crtc_init_with_planes(dev, crtc, primary, cursor,
@@ -176,6 +180,17 @@ static int amdgpu_vkms_crtc_init(struct drm_device *dev, struct drm_crtc *crtc,
drm_crtc_helper_add(crtc, &amdgpu_vkms_crtc_helper_funcs);
+ amdgpu_crtc->crtc_id = drm_crtc_index(crtc);
+ adev->mode_info.crtcs[drm_crtc_index(crtc)] = amdgpu_crtc;
+
+ amdgpu_crtc->pll_id = ATOM_PPLL_INVALID;
+ amdgpu_crtc->encoder = NULL;
+ amdgpu_crtc->connector = NULL;
+ amdgpu_crtc->vsync_timer_enabled = AMDGPU_IRQ_STATE_DISABLE;
+
+ hrtimer_init(&amdgpu_crtc->vblank_timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+ amdgpu_crtc->vblank_timer.function = &amdgpu_vkms_vblank_simulate;
+
return ret;
}
@@ -375,6 +390,7 @@ static struct drm_plane *amdgpu_vkms_plane_init(struct drm_device *dev,
int index)
{
struct drm_plane *plane;
+ uint64_t modifiers[] = {DRM_FORMAT_MOD_LINEAR, DRM_FORMAT_MOD_INVALID};
int ret;
plane = kzalloc(sizeof(*plane), GFP_KERNEL);
@@ -385,7 +401,7 @@ static struct drm_plane *amdgpu_vkms_plane_init(struct drm_device *dev,
&amdgpu_vkms_plane_funcs,
amdgpu_vkms_formats,
ARRAY_SIZE(amdgpu_vkms_formats),
- NULL, type, NULL);
+ modifiers, type, NULL);
if (ret) {
kfree(plane);
return ERR_PTR(ret);
@@ -396,12 +412,12 @@ static struct drm_plane *amdgpu_vkms_plane_init(struct drm_device *dev,
return plane;
}
-int amdgpu_vkms_output_init(struct drm_device *dev,
- struct amdgpu_vkms_output *output, int index)
+static int amdgpu_vkms_output_init(struct drm_device *dev, struct
+ amdgpu_vkms_output *output, int index)
{
struct drm_connector *connector = &output->connector;
struct drm_encoder *encoder = &output->encoder;
- struct drm_crtc *crtc = &output->crtc;
+ struct drm_crtc *crtc = &output->crtc.base;
struct drm_plane *primary, *cursor = NULL;
int ret;
@@ -465,6 +481,11 @@ static int amdgpu_vkms_sw_init(void *handle)
int r, i;
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ adev->amdgpu_vkms_output = kcalloc(adev->mode_info.num_crtc,
+ sizeof(struct amdgpu_vkms_output), GFP_KERNEL);
+ if (!adev->amdgpu_vkms_output)
+ return -ENOMEM;
+
adev_to_drm(adev)->max_vblank_count = 0;
adev_to_drm(adev)->mode_config.funcs = &amdgpu_vkms_mode_funcs;
@@ -481,10 +502,6 @@ static int amdgpu_vkms_sw_init(void *handle)
if (r)
return r;
- adev->amdgpu_vkms_output = kcalloc(adev->mode_info.num_crtc, sizeof(struct amdgpu_vkms_output), GFP_KERNEL);
- if (!adev->amdgpu_vkms_output)
- return -ENOMEM;
-
/* allocate crtcs, encoders, connectors */
for (i = 0; i < adev->mode_info.num_crtc; i++) {
r = amdgpu_vkms_output_init(adev_to_drm(adev), &adev->amdgpu_vkms_output[i], i);
@@ -504,15 +521,16 @@ static int amdgpu_vkms_sw_fini(void *handle)
int i = 0;
for (i = 0; i < adev->mode_info.num_crtc; i++)
- if (adev->amdgpu_vkms_output[i].vblank_hrtimer.function)
- hrtimer_cancel(&adev->amdgpu_vkms_output[i].vblank_hrtimer);
+ if (adev->mode_info.crtcs[i])
+ hrtimer_cancel(&adev->mode_info.crtcs[i]->vblank_timer);
+
+ drm_kms_helper_poll_fini(adev_to_drm(adev));
+ drm_mode_config_cleanup(adev_to_drm(adev));
+
+ adev->mode_info.mode_config_initialized = false;
kfree(adev->mode_info.bios_hardcoded_edid);
kfree(adev->amdgpu_vkms_output);
-
- drm_kms_helper_poll_fini(adev_to_drm(adev));
-
- adev->mode_info.mode_config_initialized = false;
return 0;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vkms.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vkms.h
index 97f1b79..4f8722ff 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vkms.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vkms.h
@@ -10,15 +10,14 @@
#define YRES_MAX 16384
#define drm_crtc_to_amdgpu_vkms_output(target) \
- container_of(target, struct amdgpu_vkms_output, crtc)
+ container_of(target, struct amdgpu_vkms_output, crtc.base)
extern const struct amdgpu_ip_block_version amdgpu_vkms_ip_block;
struct amdgpu_vkms_output {
- struct drm_crtc crtc;
+ struct amdgpu_crtc crtc;
struct drm_encoder encoder;
struct drm_connector connector;
- struct hrtimer vblank_hrtimer;
ktime_t period_ns;
struct drm_pending_vblank_event *event;
};
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index 0e7dc23..b37fc7d 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -53,7 +53,7 @@
* can be mapped as snooped (cached system pages) or unsnooped
* (uncached system pages).
* Each VM has an ID associated with it and there is a page table
- * associated with each VMID. When execting a command buffer,
+ * associated with each VMID. When executing a command buffer,
* the kernel tells the the ring what VMID to use for that command
* buffer. VMIDs are allocated dynamically as commands are submitted.
* The userspace drivers maintain their own address space and the kernel
@@ -2102,30 +2102,14 @@ static void amdgpu_vm_free_mapping(struct amdgpu_device *adev,
static void amdgpu_vm_prt_fini(struct amdgpu_device *adev, struct amdgpu_vm *vm)
{
struct dma_resv *resv = vm->root.bo->tbo.base.resv;
- struct dma_fence *excl, **shared;
- unsigned i, shared_count;
- int r;
+ struct dma_resv_iter cursor;
+ struct dma_fence *fence;
- r = dma_resv_get_fences(resv, &excl, &shared_count, &shared);
- if (r) {
- /* Not enough memory to grab the fence list, as last resort
- * block for all the fences to complete.
- */
- dma_resv_wait_timeout(resv, true, false,
- MAX_SCHEDULE_TIMEOUT);
- return;
- }
-
- /* Add a callback for each fence in the reservation object */
- amdgpu_vm_prt_get(adev);
- amdgpu_vm_add_prt_cb(adev, excl);
-
- for (i = 0; i < shared_count; ++i) {
+ dma_resv_for_each_fence(&cursor, resv, true, fence) {
+ /* Add a callback for each fence in the reservation object */
amdgpu_vm_prt_get(adev);
- amdgpu_vm_add_prt_cb(adev, shared[i]);
+ amdgpu_vm_add_prt_cb(adev, fence);
}
-
- kfree(shared);
}
/**
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_xgmi.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_xgmi.c
index 567df2d..a38c6a7 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_xgmi.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_xgmi.c
@@ -265,6 +265,11 @@ static ssize_t amdgpu_xgmi_show_error(struct device *dev,
ficaa_pie_ctl_in = AMDGPU_XGMI_SET_FICAA(0x200);
ficaa_pie_status_in = AMDGPU_XGMI_SET_FICAA(0x208);
+ if ((!adev->df.funcs) ||
+ (!adev->df.funcs->get_fica) ||
+ (!adev->df.funcs->set_fica))
+ return -EINVAL;
+
fica_out = adev->df.funcs->get_fica(adev, ficaa_pie_ctl_in);
if (fica_out != 0x1f)
pr_err("xGMI error counters not enabled!\n");
diff --git a/drivers/gpu/drm/amd/amdgpu/atombios_encoders.c b/drivers/gpu/drm/amd/amdgpu/atombios_encoders.c
index 6134ed9..a92d86e 100644
--- a/drivers/gpu/drm/amd/amdgpu/atombios_encoders.c
+++ b/drivers/gpu/drm/amd/amdgpu/atombios_encoders.c
@@ -469,7 +469,7 @@ int amdgpu_atombios_encoder_get_encoder_mode(struct drm_encoder *encoder)
if (amdgpu_connector->use_digital &&
(amdgpu_connector->audio == AMDGPU_AUDIO_ENABLE))
return ATOM_ENCODER_MODE_HDMI;
- else if (drm_detect_hdmi_monitor(amdgpu_connector_edid(connector)) &&
+ else if (connector->display_info.is_hdmi &&
(amdgpu_connector->audio == AMDGPU_AUDIO_AUTO))
return ATOM_ENCODER_MODE_HDMI;
else if (amdgpu_connector->use_digital)
@@ -488,7 +488,7 @@ int amdgpu_atombios_encoder_get_encoder_mode(struct drm_encoder *encoder)
if (amdgpu_audio != 0) {
if (amdgpu_connector->audio == AMDGPU_AUDIO_ENABLE)
return ATOM_ENCODER_MODE_HDMI;
- else if (drm_detect_hdmi_monitor(amdgpu_connector_edid(connector)) &&
+ else if (connector->display_info.is_hdmi &&
(amdgpu_connector->audio == AMDGPU_AUDIO_AUTO))
return ATOM_ENCODER_MODE_HDMI;
else
@@ -506,7 +506,7 @@ int amdgpu_atombios_encoder_get_encoder_mode(struct drm_encoder *encoder)
} else if (amdgpu_audio != 0) {
if (amdgpu_connector->audio == AMDGPU_AUDIO_ENABLE)
return ATOM_ENCODER_MODE_HDMI;
- else if (drm_detect_hdmi_monitor(amdgpu_connector_edid(connector)) &&
+ else if (connector->display_info.is_hdmi &&
(amdgpu_connector->audio == AMDGPU_AUDIO_AUTO))
return ATOM_ENCODER_MODE_HDMI;
else
diff --git a/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c b/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c
index b200b9e..8318ee8 100644
--- a/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c
@@ -2092,22 +2092,18 @@ static int dce_v8_0_pick_dig_encoder(struct drm_encoder *encoder)
return 1;
else
return 0;
- break;
case ENCODER_OBJECT_ID_INTERNAL_UNIPHY1:
if (dig->linkb)
return 3;
else
return 2;
- break;
case ENCODER_OBJECT_ID_INTERNAL_UNIPHY2:
if (dig->linkb)
return 5;
else
return 4;
- break;
case ENCODER_OBJECT_ID_INTERNAL_UNIPHY3:
return 6;
- break;
default:
DRM_ERROR("invalid encoder_id: 0x%x\n", amdgpu_encoder->encoder_id);
return 0;
diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c
index edb3e3b..9189fb8 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c
@@ -63,6 +63,13 @@
#define mmGCEA_PROBE_MAP 0x070c
#define mmGCEA_PROBE_MAP_BASE_IDX 0
+#define GFX9_RLCG_GC_WRITE_OLD (0x8 << 28)
+#define GFX9_RLCG_GC_WRITE (0x0 << 28)
+#define GFX9_RLCG_GC_READ (0x1 << 28)
+#define GFX9_RLCG_VFGATE_DISABLED 0x4000000
+#define GFX9_RLCG_WRONG_OPERATION_TYPE 0x2000000
+#define GFX9_RLCG_NOT_IN_RANGE 0x1000000
+
MODULE_FIRMWARE("amdgpu/vega10_ce.bin");
MODULE_FIRMWARE("amdgpu/vega10_pfp.bin");
MODULE_FIRMWARE("amdgpu/vega10_me.bin");
@@ -739,7 +746,7 @@ static const u32 GFX_RLC_SRM_INDEX_CNTL_DATA_OFFSETS[] =
mmRLC_SRM_INDEX_CNTL_DATA_7 - mmRLC_SRM_INDEX_CNTL_DATA_0,
};
-static void gfx_v9_0_rlcg_w(struct amdgpu_device *adev, u32 offset, u32 v, u32 flag)
+static u32 gfx_v9_0_rlcg_rw(struct amdgpu_device *adev, u32 offset, u32 v, uint32_t flag)
{
static void *scratch_reg0;
static void *scratch_reg1;
@@ -748,21 +755,20 @@ static void gfx_v9_0_rlcg_w(struct amdgpu_device *adev, u32 offset, u32 v, u32 f
static void *spare_int;
static uint32_t grbm_cntl;
static uint32_t grbm_idx;
+ uint32_t i = 0;
+ uint32_t retries = 50000;
+ u32 ret = 0;
+ u32 tmp;
scratch_reg0 = adev->rmmio + (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG0_BASE_IDX] + mmSCRATCH_REG0)*4;
scratch_reg1 = adev->rmmio + (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG1_BASE_IDX] + mmSCRATCH_REG1)*4;
- scratch_reg2 = adev->rmmio + (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG1_BASE_IDX] + mmSCRATCH_REG2)*4;
- scratch_reg3 = adev->rmmio + (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG1_BASE_IDX] + mmSCRATCH_REG3)*4;
+ scratch_reg2 = adev->rmmio + (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG2_BASE_IDX] + mmSCRATCH_REG2)*4;
+ scratch_reg3 = adev->rmmio + (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG3_BASE_IDX] + mmSCRATCH_REG3)*4;
spare_int = adev->rmmio + (adev->reg_offset[GC_HWIP][0][mmRLC_SPARE_INT_BASE_IDX] + mmRLC_SPARE_INT)*4;
grbm_cntl = adev->reg_offset[GC_HWIP][0][mmGRBM_GFX_CNTL_BASE_IDX] + mmGRBM_GFX_CNTL;
grbm_idx = adev->reg_offset[GC_HWIP][0][mmGRBM_GFX_INDEX_BASE_IDX] + mmGRBM_GFX_INDEX;
- if (amdgpu_sriov_runtime(adev)) {
- pr_err("shouldn't call rlcg write register during runtime\n");
- return;
- }
-
if (offset == grbm_cntl || offset == grbm_idx) {
if (offset == grbm_cntl)
writel(v, scratch_reg2);
@@ -771,41 +777,95 @@ static void gfx_v9_0_rlcg_w(struct amdgpu_device *adev, u32 offset, u32 v, u32 f
writel(v, ((void __iomem *)adev->rmmio) + (offset * 4));
} else {
- uint32_t i = 0;
- uint32_t retries = 50000;
-
+ /*
+ * SCRATCH_REG0 = read/write value
+ * SCRATCH_REG1[30:28] = command
+ * SCRATCH_REG1[19:0] = address in dword
+ * SCRATCH_REG1[26:24] = Error reporting
+ */
writel(v, scratch_reg0);
- writel(offset | 0x80000000, scratch_reg1);
+ writel(offset | flag, scratch_reg1);
writel(1, spare_int);
- for (i = 0; i < retries; i++) {
- u32 tmp;
+ for (i = 0; i < retries; i++) {
tmp = readl(scratch_reg1);
- if (!(tmp & 0x80000000))
+ if (!(tmp & flag))
break;
udelay(10);
}
- if (i >= retries)
- pr_err("timeout: rlcg program reg:0x%05x failed !\n", offset);
+
+ if (i >= retries) {
+ if (amdgpu_sriov_reg_indirect_gc(adev)) {
+ if (tmp & GFX9_RLCG_VFGATE_DISABLED)
+ pr_err("The vfgate is disabled, program reg:0x%05x failed!\n", offset);
+ else if (tmp & GFX9_RLCG_WRONG_OPERATION_TYPE)
+ pr_err("Wrong operation type, program reg:0x%05x failed!\n", offset);
+ else if (tmp & GFX9_RLCG_NOT_IN_RANGE)
+ pr_err("The register is not in range, program reg:0x%05x failed!\n", offset);
+ else
+ pr_err("Unknown error type, program reg:0x%05x failed!\n", offset);
+ } else
+ pr_err("timeout: rlcg program reg:0x%05x failed!\n", offset);
+ }
}
+ ret = readl(scratch_reg0);
+
+ return ret;
+}
+
+static bool gfx_v9_0_get_rlcg_flag(struct amdgpu_device *adev, u32 acc_flags, u32 hwip,
+ int write, u32 *rlcg_flag)
+{
+
+ switch (hwip) {
+ case GC_HWIP:
+ if (amdgpu_sriov_reg_indirect_gc(adev)) {
+ *rlcg_flag = write ? GFX9_RLCG_GC_WRITE : GFX9_RLCG_GC_READ;
+
+ return true;
+ /* only in new version, AMDGPU_REGS_NO_KIQ and AMDGPU_REGS_RLC enabled simultaneously */
+ } else if ((acc_flags & AMDGPU_REGS_RLC) && !(acc_flags & AMDGPU_REGS_NO_KIQ) && write) {
+ *rlcg_flag = GFX9_RLCG_GC_WRITE_OLD;
+ return true;
+ }
+
+ break;
+ default:
+ return false;
+ }
+
+ return false;
+}
+
+static u32 gfx_v9_0_sriov_rreg(struct amdgpu_device *adev, u32 offset, u32 acc_flags, u32 hwip)
+{
+ u32 rlcg_flag;
+
+ if (!amdgpu_sriov_runtime(adev) && gfx_v9_0_get_rlcg_flag(adev, acc_flags, hwip, 0, &rlcg_flag))
+ return gfx_v9_0_rlcg_rw(adev, offset, 0, rlcg_flag);
+
+ if (acc_flags & AMDGPU_REGS_NO_KIQ)
+ return RREG32_NO_KIQ(offset);
+ else
+ return RREG32(offset);
}
static void gfx_v9_0_sriov_wreg(struct amdgpu_device *adev, u32 offset,
- u32 v, u32 acc_flags, u32 hwip)
+ u32 value, u32 acc_flags, u32 hwip)
{
- if ((acc_flags & AMDGPU_REGS_RLC) &&
- amdgpu_sriov_fullaccess(adev)) {
- gfx_v9_0_rlcg_w(adev, offset, v, acc_flags);
+ u32 rlcg_flag;
+ if (!amdgpu_sriov_runtime(adev) && gfx_v9_0_get_rlcg_flag(adev, acc_flags, hwip, 1, &rlcg_flag)) {
+ gfx_v9_0_rlcg_rw(adev, offset, value, rlcg_flag);
return;
}
if (acc_flags & AMDGPU_REGS_NO_KIQ)
- WREG32_NO_KIQ(offset, v);
+ WREG32_NO_KIQ(offset, value);
else
- WREG32(offset, v);
+ WREG32(offset, value);
}
#define VEGA10_GB_ADDR_CONFIG_GOLDEN 0x2a114042
@@ -5135,7 +5195,7 @@ static void gfx_v9_0_update_spm_vmid(struct amdgpu_device *adev, unsigned vmid)
if (amdgpu_sriov_is_pp_one_vf(adev))
data = RREG32_NO_KIQ(reg);
else
- data = RREG32(reg);
+ data = RREG32_SOC15(GC, 0, mmRLC_SPM_MC_CNTL);
data &= ~RLC_SPM_MC_CNTL__RLC_SPM_VMID_MASK;
data |= (vmid & RLC_SPM_MC_CNTL__RLC_SPM_VMID_MASK) << RLC_SPM_MC_CNTL__RLC_SPM_VMID__SHIFT;
@@ -5191,6 +5251,7 @@ static const struct amdgpu_rlc_funcs gfx_v9_0_rlc_funcs = {
.start = gfx_v9_0_rlc_start,
.update_spm_vmid = gfx_v9_0_update_spm_vmid,
.sriov_wreg = gfx_v9_0_sriov_wreg,
+ .sriov_rreg = gfx_v9_0_sriov_rreg,
.is_rlcg_access_range = gfx_v9_0_is_rlcg_access_range,
};
@@ -5796,16 +5857,16 @@ static void gfx_v9_0_set_compute_eop_interrupt_state(struct amdgpu_device *adev,
switch (state) {
case AMDGPU_IRQ_STATE_DISABLE:
- mec_int_cntl = RREG32(mec_int_cntl_reg);
+ mec_int_cntl = RREG32_SOC15_IP(GC,mec_int_cntl_reg);
mec_int_cntl = REG_SET_FIELD(mec_int_cntl, CP_ME1_PIPE0_INT_CNTL,
TIME_STAMP_INT_ENABLE, 0);
- WREG32(mec_int_cntl_reg, mec_int_cntl);
+ WREG32_SOC15_IP(GC, mec_int_cntl_reg, mec_int_cntl);
break;
case AMDGPU_IRQ_STATE_ENABLE:
- mec_int_cntl = RREG32(mec_int_cntl_reg);
+ mec_int_cntl = RREG32_SOC15_IP(GC, mec_int_cntl_reg);
mec_int_cntl = REG_SET_FIELD(mec_int_cntl, CP_ME1_PIPE0_INT_CNTL,
TIME_STAMP_INT_ENABLE, 1);
- WREG32(mec_int_cntl_reg, mec_int_cntl);
+ WREG32_SOC15_IP(GC, mec_int_cntl_reg, mec_int_cntl);
break;
default:
break;
diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c
index 3ec5ff5..3d5d47a 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c
@@ -107,7 +107,7 @@ static int gmc_v10_0_process_interrupt(struct amdgpu_device *adev,
/* Process it onyl if it's the first fault for this address */
if (entry->ih != &adev->irq.ih_soft &&
- amdgpu_gmc_filter_faults(adev, addr, entry->pasid,
+ amdgpu_gmc_filter_faults(adev, entry->ih, addr, entry->pasid,
entry->timestamp))
return 1;
@@ -914,12 +914,6 @@ static int gmc_v10_0_sw_init(void *handle)
return r;
}
- if (adev->gmc.xgmi.supported) {
- r = adev->gfxhub.funcs->get_xgmi_info(adev);
- if (r)
- return r;
- }
-
r = gmc_v10_0_mc_init(adev);
if (r)
return r;
@@ -992,10 +986,14 @@ static int gmc_v10_0_gart_enable(struct amdgpu_device *adev)
return -EINVAL;
}
+ if (amdgpu_sriov_vf(adev) && amdgpu_in_reset(adev))
+ goto skip_pin_bo;
+
r = amdgpu_gart_table_vram_pin(adev);
if (r)
return r;
+skip_pin_bo:
r = adev->gfxhub.funcs->gart_enable(adev);
if (r)
return r;
diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
index 492ebed..63b890f 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
@@ -515,10 +515,10 @@ static void gmc_v8_0_mc_program(struct amdgpu_device *adev)
static int gmc_v8_0_mc_init(struct amdgpu_device *adev)
{
int r;
+ u32 tmp;
adev->gmc.vram_width = amdgpu_atombios_get_vram_width(adev);
if (!adev->gmc.vram_width) {
- u32 tmp;
int chansize, numchan;
/* Get VRAM informations */
@@ -562,8 +562,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device *adev)
adev->gmc.vram_width = numchan * chansize;
}
/* size in MB on si */
- adev->gmc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL;
- adev->gmc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL;
+ tmp = RREG32(mmCONFIG_MEMSIZE);
+ /* some boards may have garbage in the upper 16 bits */
+ if (tmp & 0xffff0000) {
+ DRM_INFO("Probable bad vram size: 0x%08x\n", tmp);
+ if (tmp & 0xffff)
+ tmp &= 0xffff;
+ }
+ adev->gmc.mc_vram_size = tmp * 1024ULL * 1024ULL;
+ adev->gmc.real_vram_size = adev->gmc.mc_vram_size;
if (!(adev->flags & AMD_IS_APU)) {
r = amdgpu_device_resize_fb_bar(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
index d84523c..57f2729 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
@@ -478,9 +478,18 @@ static int gmc_v9_0_vm_fault_interrupt_state(struct amdgpu_device *adev,
hub = &adev->vmhub[j];
for (i = 0; i < 16; i++) {
reg = hub->vm_context0_cntl + i;
- tmp = RREG32(reg);
+
+ if (j == AMDGPU_GFXHUB_0)
+ tmp = RREG32_SOC15_IP(GC, reg);
+ else
+ tmp = RREG32_SOC15_IP(MMHUB, reg);
+
tmp &= ~bits;
- WREG32(reg, tmp);
+
+ if (j == AMDGPU_GFXHUB_0)
+ WREG32_SOC15_IP(GC, reg, tmp);
+ else
+ WREG32_SOC15_IP(MMHUB, reg, tmp);
}
}
break;
@@ -489,9 +498,18 @@ static int gmc_v9_0_vm_fault_interrupt_state(struct amdgpu_device *adev,
hub = &adev->vmhub[j];
for (i = 0; i < 16; i++) {
reg = hub->vm_context0_cntl + i;
- tmp = RREG32(reg);
+
+ if (j == AMDGPU_GFXHUB_0)
+ tmp = RREG32_SOC15_IP(GC, reg);
+ else
+ tmp = RREG32_SOC15_IP(MMHUB, reg);
+
tmp |= bits;
- WREG32(reg, tmp);
+
+ if (j == AMDGPU_GFXHUB_0)
+ WREG32_SOC15_IP(GC, reg, tmp);
+ else
+ WREG32_SOC15_IP(MMHUB, reg, tmp);
}
}
break;
@@ -523,7 +541,7 @@ static int gmc_v9_0_process_interrupt(struct amdgpu_device *adev,
/* Process it onyl if it's the first fault for this address */
if (entry->ih != &adev->irq.ih_soft &&
- amdgpu_gmc_filter_faults(adev, addr, entry->pasid,
+ amdgpu_gmc_filter_faults(adev, entry->ih, addr, entry->pasid,
entry->timestamp))
return 1;
@@ -788,9 +806,12 @@ static void gmc_v9_0_flush_gpu_tlb(struct amdgpu_device *adev, uint32_t vmid,
/* TODO: It needs to continue working on debugging with semaphore for GFXHUB as well. */
if (use_semaphore) {
for (j = 0; j < adev->usec_timeout; j++) {
- /* a read return value of 1 means semaphore acuqire */
- tmp = RREG32_NO_KIQ(hub->vm_inv_eng0_sem +
- hub->eng_distance * eng);
+ /* a read return value of 1 means semaphore acquire */
+ if (vmhub == AMDGPU_GFXHUB_0)
+ tmp = RREG32_SOC15_IP_NO_KIQ(GC, hub->vm_inv_eng0_sem + hub->eng_distance * eng);
+ else
+ tmp = RREG32_SOC15_IP_NO_KIQ(MMHUB, hub->vm_inv_eng0_sem + hub->eng_distance * eng);
+
if (tmp & 0x1)
break;
udelay(1);
@@ -801,8 +822,10 @@ static void gmc_v9_0_flush_gpu_tlb(struct amdgpu_device *adev, uint32_t vmid,
}
do {
- WREG32_NO_KIQ(hub->vm_inv_eng0_req +
- hub->eng_distance * eng, inv_req);
+ if (vmhub == AMDGPU_GFXHUB_0)
+ WREG32_SOC15_IP_NO_KIQ(GC, hub->vm_inv_eng0_req + hub->eng_distance * eng, inv_req);
+ else
+ WREG32_SOC15_IP_NO_KIQ(MMHUB, hub->vm_inv_eng0_req + hub->eng_distance * eng, inv_req);
/*
* Issue a dummy read to wait for the ACK register to
@@ -815,8 +838,11 @@ static void gmc_v9_0_flush_gpu_tlb(struct amdgpu_device *adev, uint32_t vmid,
hub->eng_distance * eng);
for (j = 0; j < adev->usec_timeout; j++) {
- tmp = RREG32_NO_KIQ(hub->vm_inv_eng0_ack +
- hub->eng_distance * eng);
+ if (vmhub == AMDGPU_GFXHUB_0)
+ tmp = RREG32_SOC15_IP_NO_KIQ(GC, hub->vm_inv_eng0_ack + hub->eng_distance * eng);
+ else
+ tmp = RREG32_SOC15_IP_NO_KIQ(MMHUB, hub->vm_inv_eng0_ack + hub->eng_distance * eng);
+
if (tmp & (1 << vmid))
break;
udelay(1);
@@ -827,13 +853,16 @@ static void gmc_v9_0_flush_gpu_tlb(struct amdgpu_device *adev, uint32_t vmid,
} while (inv_req);
/* TODO: It needs to continue working on debugging with semaphore for GFXHUB as well. */
- if (use_semaphore)
+ if (use_semaphore) {
/*
* add semaphore release after invalidation,
* write with 0 means semaphore release
*/
- WREG32_NO_KIQ(hub->vm_inv_eng0_sem +
- hub->eng_distance * eng, 0);
+ if (vmhub == AMDGPU_GFXHUB_0)
+ WREG32_SOC15_IP_NO_KIQ(GC, hub->vm_inv_eng0_sem + hub->eng_distance * eng, 0);
+ else
+ WREG32_SOC15_IP_NO_KIQ(MMHUB, hub->vm_inv_eng0_sem + hub->eng_distance * eng, 0);
+ }
spin_unlock(&adev->gmc.invalidate_lock);
@@ -1294,7 +1323,8 @@ static int gmc_v9_0_late_init(void *handle)
if (!amdgpu_sriov_vf(adev) &&
(adev->ip_versions[UMC_HWIP][0] == IP_VERSION(6, 0, 0))) {
if (!(adev->ras_enabled & (1 << AMDGPU_RAS_BLOCK__UMC))) {
- if (adev->df.funcs->enable_ecc_force_par_wr_rmw)
+ if (adev->df.funcs &&
+ adev->df.funcs->enable_ecc_force_par_wr_rmw)
adev->df.funcs->enable_ecc_force_par_wr_rmw(adev, false);
}
}
@@ -1505,9 +1535,11 @@ static int gmc_v9_0_sw_init(void *handle)
chansize = 64;
else
chansize = 128;
-
- numchan = adev->df.funcs->get_hbm_channel_number(adev);
- adev->gmc.vram_width = numchan * chansize;
+ if (adev->df.funcs &&
+ adev->df.funcs->get_hbm_channel_number) {
+ numchan = adev->df.funcs->get_hbm_channel_number(adev);
+ adev->gmc.vram_width = numchan * chansize;
+ }
}
adev->gmc.vram_type = vram_type;
@@ -1596,12 +1628,6 @@ static int gmc_v9_0_sw_init(void *handle)
}
adev->need_swiotlb = drm_need_swiotlb(44);
- if (adev->gmc.xgmi.supported) {
- r = adev->gfxhub.funcs->get_xgmi_info(adev);
- if (r)
- return r;
- }
-
r = gmc_v9_0_mc_init(adev);
if (r)
return r;
@@ -1714,10 +1740,14 @@ static int gmc_v9_0_gart_enable(struct amdgpu_device *adev)
return -EINVAL;
}
+ if (amdgpu_sriov_vf(adev) && amdgpu_in_reset(adev))
+ goto skip_pin_bo;
+
r = amdgpu_gart_table_vram_pin(adev);
if (r)
return r;
+skip_pin_bo:
r = adev->gfxhub.funcs->gart_enable(adev);
if (r)
return r;
@@ -1742,7 +1772,7 @@ static int gmc_v9_0_hw_init(void *handle)
{
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
bool value;
- int r, i;
+ int i;
/* The sequence of these two function calls matters.*/
gmc_v9_0_init_golden_registers(adev);
@@ -1777,9 +1807,7 @@ static int gmc_v9_0_hw_init(void *handle)
if (adev->umc.funcs && adev->umc.funcs->init_registers)
adev->umc.funcs->init_registers(adev);
- r = gmc_v9_0_gart_enable(adev);
-
- return r;
+ return gmc_v9_0_gart_enable(adev);
}
/**
diff --git a/drivers/gpu/drm/amd/amdgpu/mxgpu_ai.c b/drivers/gpu/drm/amd/amdgpu/mxgpu_ai.c
index 23b066b..0077e73 100644
--- a/drivers/gpu/drm/amd/amdgpu/mxgpu_ai.c
+++ b/drivers/gpu/drm/amd/amdgpu/mxgpu_ai.c
@@ -252,11 +252,12 @@ static void xgpu_ai_mailbox_flr_work(struct work_struct *work)
* otherwise the mailbox msg will be ruined/reseted by
* the VF FLR.
*/
- if (!down_write_trylock(&adev->reset_sem))
+ if (atomic_cmpxchg(&adev->in_gpu_reset, 0, 1) != 0)
return;
+ down_write(&adev->reset_sem);
+
amdgpu_virt_fini_data_exchange(adev);
- atomic_set(&adev->in_gpu_reset, 1);
xgpu_ai_mailbox_trans_msg(adev, IDH_READY_TO_RESET, 0, 0, 0);
diff --git a/drivers/gpu/drm/amd/amdgpu/mxgpu_ai.h b/drivers/gpu/drm/amd/amdgpu/mxgpu_ai.h
index bd3b231..f9aa4d0 100644
--- a/drivers/gpu/drm/amd/amdgpu/mxgpu_ai.h
+++ b/drivers/gpu/drm/amd/amdgpu/mxgpu_ai.h
@@ -26,7 +26,7 @@
#define AI_MAILBOX_POLL_ACK_TIMEDOUT 500
#define AI_MAILBOX_POLL_MSG_TIMEDOUT 6000
-#define AI_MAILBOX_POLL_FLR_TIMEDOUT 5000
+#define AI_MAILBOX_POLL_FLR_TIMEDOUT 10000
#define AI_MAILBOX_POLL_MSG_REP_MAX 11
enum idh_request {
diff --git a/drivers/gpu/drm/amd/amdgpu/mxgpu_nv.c b/drivers/gpu/drm/amd/amdgpu/mxgpu_nv.c
index a35e6d8..477d0dd 100644
--- a/drivers/gpu/drm/amd/amdgpu/mxgpu_nv.c
+++ b/drivers/gpu/drm/amd/amdgpu/mxgpu_nv.c
@@ -281,11 +281,12 @@ static void xgpu_nv_mailbox_flr_work(struct work_struct *work)
* otherwise the mailbox msg will be ruined/reseted by
* the VF FLR.
*/
- if (!down_write_trylock(&adev->reset_sem))
+ if (atomic_cmpxchg(&adev->in_gpu_reset, 0, 1) != 0)
return;
+ down_write(&adev->reset_sem);
+
amdgpu_virt_fini_data_exchange(adev);
- atomic_set(&adev->in_gpu_reset, 1);
xgpu_nv_mailbox_trans_msg(adev, IDH_READY_TO_RESET, 0, 0, 0);
diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
index 38241cf..8ce5b8ca 100644
--- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
@@ -716,6 +716,7 @@ static const struct amd_ip_funcs navi10_ih_ip_funcs = {
static const struct amdgpu_ih_funcs navi10_ih_funcs = {
.get_wptr = navi10_ih_get_wptr,
.decode_iv = amdgpu_ih_decode_iv_helper,
+ .decode_iv_ts = amdgpu_ih_decode_iv_ts_helper,
.set_rptr = navi10_ih_set_rptr
};
diff --git a/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c b/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c
index 2176ef8..d0e76b3 100644
--- a/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c
@@ -277,69 +277,9 @@ static bool psp_v11_0_is_sos_alive(struct psp_context *psp)
return sol_reg != 0x0;
}
-static int psp_v11_0_bootloader_load_kdb(struct psp_context *psp)
-{
- int ret;
- uint32_t psp_gfxdrv_command_reg = 0;
- struct amdgpu_device *adev = psp->adev;
-
- /* Check tOS sign of life register to confirm sys driver and sOS
- * are already been loaded.
- */
- if (psp_v11_0_is_sos_alive(psp))
- return 0;
-
- ret = psp_v11_0_wait_for_bootloader(psp);
- if (ret)
- return ret;
-
- /* Copy PSP KDB binary to memory */
- psp_copy_fw(psp, psp->kdb.start_addr, psp->kdb.size_bytes);
-
- /* Provide the PSP KDB to bootloader */
- WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36,
- (uint32_t)(psp->fw_pri_mc_addr >> 20));
- psp_gfxdrv_command_reg = PSP_BL__LOAD_KEY_DATABASE;
- WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_35,
- psp_gfxdrv_command_reg);
-
- ret = psp_v11_0_wait_for_bootloader(psp);
-
- return ret;
-}
-
-static int psp_v11_0_bootloader_load_spl(struct psp_context *psp)
-{
- int ret;
- uint32_t psp_gfxdrv_command_reg = 0;
- struct amdgpu_device *adev = psp->adev;
-
- /* Check tOS sign of life register to confirm sys driver and sOS
- * are already been loaded.
- */
- if (psp_v11_0_is_sos_alive(psp))
- return 0;
-
- ret = psp_v11_0_wait_for_bootloader(psp);
- if (ret)
- return ret;
-
- /* Copy PSP SPL binary to memory */
- psp_copy_fw(psp, psp->spl.start_addr, psp->spl.size_bytes);
-
- /* Provide the PSP SPL to bootloader */
- WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36,
- (uint32_t)(psp->fw_pri_mc_addr >> 20));
- psp_gfxdrv_command_reg = PSP_BL__LOAD_TOS_SPL_TABLE;
- WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_35,
- psp_gfxdrv_command_reg);
-
- ret = psp_v11_0_wait_for_bootloader(psp);
-
- return ret;
-}
-
-static int psp_v11_0_bootloader_load_sysdrv(struct psp_context *psp)
+static int psp_v11_0_bootloader_load_component(struct psp_context *psp,
+ struct psp_bin_desc *bin_desc,
+ enum psp_bootloader_cmd bl_cmd)
{
int ret;
uint32_t psp_gfxdrv_command_reg = 0;
@@ -356,23 +296,35 @@ static int psp_v11_0_bootloader_load_sysdrv(struct psp_context *psp)
return ret;
/* Copy PSP System Driver binary to memory */
- psp_copy_fw(psp, psp->sys.start_addr, psp->sys.size_bytes);
+ psp_copy_fw(psp, bin_desc->start_addr, bin_desc->size_bytes);
/* Provide the sys driver to bootloader */
WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36,
(uint32_t)(psp->fw_pri_mc_addr >> 20));
- psp_gfxdrv_command_reg = PSP_BL__LOAD_SYSDRV;
+ psp_gfxdrv_command_reg = bl_cmd;
WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_35,
psp_gfxdrv_command_reg);
- /* there might be handshake issue with hardware which needs delay */
- mdelay(20);
-
ret = psp_v11_0_wait_for_bootloader(psp);
return ret;
}
+static int psp_v11_0_bootloader_load_kdb(struct psp_context *psp)
+{
+ return psp_v11_0_bootloader_load_component(psp, &psp->kdb, PSP_BL__LOAD_KEY_DATABASE);
+}
+
+static int psp_v11_0_bootloader_load_spl(struct psp_context *psp)
+{
+ return psp_v11_0_bootloader_load_component(psp, &psp->spl, PSP_BL__LOAD_TOS_SPL_TABLE);
+}
+
+static int psp_v11_0_bootloader_load_sysdrv(struct psp_context *psp)
+{
+ return psp_v11_0_bootloader_load_component(psp, &psp->sys, PSP_BL__LOAD_SYSDRV);
+}
+
static int psp_v11_0_bootloader_load_sos(struct psp_context *psp)
{
int ret;
diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v5_0.c b/drivers/gpu/drm/amd/amdgpu/sdma_v5_0.c
index 853d151..81e0335 100644
--- a/drivers/gpu/drm/amd/amdgpu/sdma_v5_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/sdma_v5_0.c
@@ -481,8 +481,6 @@ static void sdma_v5_0_ring_emit_ib(struct amdgpu_ring *ring,
* sdma_v5_0_ring_emit_mem_sync - flush the IB by graphics cache rinse
*
* @ring: amdgpu ring pointer
- * @job: job to retrieve vmid from
- * @ib: IB object to schedule
*
* flush the IB by graphics cache rinse.
*/
diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v5_2.c b/drivers/gpu/drm/amd/amdgpu/sdma_v5_2.c
index 4d4d1aa..d3d6d5b 100644
--- a/drivers/gpu/drm/amd/amdgpu/sdma_v5_2.c
+++ b/drivers/gpu/drm/amd/amdgpu/sdma_v5_2.c
@@ -368,8 +368,6 @@ static void sdma_v5_2_ring_emit_ib(struct amdgpu_ring *ring,
* sdma_v5_2_ring_emit_mem_sync - flush the IB by graphics cache rinse
*
* @ring: amdgpu ring pointer
- * @job: job to retrieve vmid from
- * @ib: IB object to schedule
*
* flush the IB by graphics cache rinse.
*/
@@ -544,9 +542,6 @@ static void sdma_v5_2_ctx_switch_enable(struct amdgpu_device *adev, bool enable)
}
for (i = 0; i < adev->sdma.num_instances; i++) {
- f32_cntl = RREG32(sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_CNTL));
- f32_cntl = REG_SET_FIELD(f32_cntl, SDMA0_CNTL,
- AUTO_CTXSW_ENABLE, enable ? 1 : 0);
if (enable && amdgpu_sdma_phase_quantum) {
WREG32_SOC15_IP(GC, sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_PHASE0_QUANTUM),
phase_quantum);
@@ -555,7 +550,13 @@ static void sdma_v5_2_ctx_switch_enable(struct amdgpu_device *adev, bool enable)
WREG32_SOC15_IP(GC, sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_PHASE2_QUANTUM),
phase_quantum);
}
- WREG32(sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_CNTL), f32_cntl);
+
+ if (!amdgpu_sriov_vf(adev)) {
+ f32_cntl = RREG32(sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_CNTL));
+ f32_cntl = REG_SET_FIELD(f32_cntl, SDMA0_CNTL,
+ AUTO_CTXSW_ENABLE, enable ? 1 : 0);
+ WREG32(sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_CNTL), f32_cntl);
+ }
}
}
@@ -578,10 +579,12 @@ static void sdma_v5_2_enable(struct amdgpu_device *adev, bool enable)
sdma_v5_2_rlc_stop(adev);
}
- for (i = 0; i < adev->sdma.num_instances; i++) {
- f32_cntl = RREG32(sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_F32_CNTL));
- f32_cntl = REG_SET_FIELD(f32_cntl, SDMA0_F32_CNTL, HALT, enable ? 0 : 1);
- WREG32(sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_F32_CNTL), f32_cntl);
+ if (!amdgpu_sriov_vf(adev)) {
+ for (i = 0; i < adev->sdma.num_instances; i++) {
+ f32_cntl = RREG32(sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_F32_CNTL));
+ f32_cntl = REG_SET_FIELD(f32_cntl, SDMA0_F32_CNTL, HALT, enable ? 0 : 1);
+ WREG32(sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_F32_CNTL), f32_cntl);
+ }
}
}
@@ -610,7 +613,8 @@ static int sdma_v5_2_gfx_resume(struct amdgpu_device *adev)
ring = &adev->sdma.instance[i].ring;
wb_offset = (ring->rptr_offs * 4);
- WREG32_SOC15_IP(GC, sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_SEM_WAIT_FAIL_TIMER_CNTL), 0);
+ if (!amdgpu_sriov_vf(adev))
+ WREG32_SOC15_IP(GC, sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_SEM_WAIT_FAIL_TIMER_CNTL), 0);
/* Set ring buffer size in dwords */
rb_bufsz = order_base_2(ring->ring_size / 4);
@@ -685,32 +689,34 @@ static int sdma_v5_2_gfx_resume(struct amdgpu_device *adev)
sdma_v5_2_ring_set_wptr(ring);
/* set minor_ptr_update to 0 after wptr programed */
+
WREG32_SOC15_IP(GC, sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_GFX_MINOR_PTR_UPDATE), 0);
- /* set utc l1 enable flag always to 1 */
- temp = RREG32(sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_CNTL));
- temp = REG_SET_FIELD(temp, SDMA0_CNTL, UTC_L1_ENABLE, 1);
-
- /* enable MCBP */
- temp = REG_SET_FIELD(temp, SDMA0_CNTL, MIDCMD_PREEMPT_ENABLE, 1);
- WREG32(sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_CNTL), temp);
-
- /* Set up RESP_MODE to non-copy addresses */
- temp = RREG32_SOC15_IP(GC, sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_UTCL1_CNTL));
- temp = REG_SET_FIELD(temp, SDMA0_UTCL1_CNTL, RESP_MODE, 3);
- temp = REG_SET_FIELD(temp, SDMA0_UTCL1_CNTL, REDO_DELAY, 9);
- WREG32_SOC15_IP(GC, sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_UTCL1_CNTL), temp);
-
- /* program default cache read and write policy */
- temp = RREG32_SOC15_IP(GC, sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_UTCL1_PAGE));
- /* clean read policy and write policy bits */
- temp &= 0xFF0FFF;
- temp |= ((CACHE_READ_POLICY_L2__DEFAULT << 12) |
- (CACHE_WRITE_POLICY_L2__DEFAULT << 14) |
- SDMA0_UTCL1_PAGE__LLC_NOALLOC_MASK);
- WREG32_SOC15_IP(GC, sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_UTCL1_PAGE), temp);
-
+ /* SRIOV VF has no control of any of registers below */
if (!amdgpu_sriov_vf(adev)) {
+ /* set utc l1 enable flag always to 1 */
+ temp = RREG32(sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_CNTL));
+ temp = REG_SET_FIELD(temp, SDMA0_CNTL, UTC_L1_ENABLE, 1);
+
+ /* enable MCBP */
+ temp = REG_SET_FIELD(temp, SDMA0_CNTL, MIDCMD_PREEMPT_ENABLE, 1);
+ WREG32(sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_CNTL), temp);
+
+ /* Set up RESP_MODE to non-copy addresses */
+ temp = RREG32_SOC15_IP(GC, sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_UTCL1_CNTL));
+ temp = REG_SET_FIELD(temp, SDMA0_UTCL1_CNTL, RESP_MODE, 3);
+ temp = REG_SET_FIELD(temp, SDMA0_UTCL1_CNTL, REDO_DELAY, 9);
+ WREG32_SOC15_IP(GC, sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_UTCL1_CNTL), temp);
+
+ /* program default cache read and write policy */
+ temp = RREG32_SOC15_IP(GC, sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_UTCL1_PAGE));
+ /* clean read policy and write policy bits */
+ temp &= 0xFF0FFF;
+ temp |= ((CACHE_READ_POLICY_L2__DEFAULT << 12) |
+ (CACHE_WRITE_POLICY_L2__DEFAULT << 14) |
+ SDMA0_UTCL1_PAGE__LLC_NOALLOC_MASK);
+ WREG32_SOC15_IP(GC, sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_UTCL1_PAGE), temp);
+
/* unhalt engine */
temp = RREG32(sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_F32_CNTL));
temp = REG_SET_FIELD(temp, SDMA0_F32_CNTL, HALT, 0);
@@ -1438,13 +1444,14 @@ static int sdma_v5_2_set_trap_irq_state(struct amdgpu_device *adev,
enum amdgpu_interrupt_state state)
{
u32 sdma_cntl;
-
u32 reg_offset = sdma_v5_2_get_reg_offset(adev, type, mmSDMA0_CNTL);
- sdma_cntl = RREG32(reg_offset);
- sdma_cntl = REG_SET_FIELD(sdma_cntl, SDMA0_CNTL, TRAP_ENABLE,
- state == AMDGPU_IRQ_STATE_ENABLE ? 1 : 0);
- WREG32(reg_offset, sdma_cntl);
+ if (!amdgpu_sriov_vf(adev)) {
+ sdma_cntl = RREG32(reg_offset);
+ sdma_cntl = REG_SET_FIELD(sdma_cntl, SDMA0_CNTL, TRAP_ENABLE,
+ state == AMDGPU_IRQ_STATE_ENABLE ? 1 : 0);
+ WREG32(reg_offset, sdma_cntl);
+ }
return 0;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/soc15.c b/drivers/gpu/drm/amd/amdgpu/soc15.c
index de9b553..0fc1747 100644
--- a/drivers/gpu/drm/amd/amdgpu/soc15.c
+++ b/drivers/gpu/drm/amd/amdgpu/soc15.c
@@ -744,7 +744,7 @@ static void soc15_reg_base_init(struct amdgpu_device *adev)
vega10_reg_base_init(adev);
break;
case CHIP_RENOIR:
- /* It's safe to do ip discovery here for Renior,
+ /* It's safe to do ip discovery here for Renoir,
* it doesn't support SRIOV. */
if (amdgpu_discovery) {
r = amdgpu_discovery_reg_base_init(adev);
@@ -1238,7 +1238,9 @@ static int soc15_common_sw_init(void *handle)
if (amdgpu_sriov_vf(adev))
xgpu_ai_mailbox_add_irq_id(adev);
- adev->df.funcs->sw_init(adev);
+ if (adev->df.funcs &&
+ adev->df.funcs->sw_init)
+ adev->df.funcs->sw_init(adev);
return 0;
}
@@ -1250,7 +1252,10 @@ static int soc15_common_sw_fini(void *handle)
if (adev->nbio.ras_funcs &&
adev->nbio.ras_funcs->ras_fini)
adev->nbio.ras_funcs->ras_fini(adev);
- adev->df.funcs->sw_fini(adev);
+
+ if (adev->df.funcs &&
+ adev->df.funcs->sw_fini)
+ adev->df.funcs->sw_fini(adev);
return 0;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/soc15_common.h b/drivers/gpu/drm/amd/amdgpu/soc15_common.h
index 8a9ca87..473767e 100644
--- a/drivers/gpu/drm/amd/amdgpu/soc15_common.h
+++ b/drivers/gpu/drm/amd/amdgpu/soc15_common.h
@@ -51,6 +51,8 @@
#define RREG32_SOC15_IP(ip, reg) __RREG32_SOC15_RLC__(reg, 0, ip##_HWIP)
+#define RREG32_SOC15_IP_NO_KIQ(ip, reg) __RREG32_SOC15_RLC__(reg, AMDGPU_REGS_NO_KIQ, ip##_HWIP)
+
#define RREG32_SOC15_NO_KIQ(ip, inst, reg) \
__RREG32_SOC15_RLC__(adev->reg_offset[ip##_HWIP][inst][reg##_BASE_IDX] + reg, \
AMDGPU_REGS_NO_KIQ, ip##_HWIP)
@@ -65,6 +67,9 @@
#define WREG32_SOC15_IP(ip, reg, value) \
__WREG32_SOC15_RLC__(reg, value, 0, ip##_HWIP)
+#define WREG32_SOC15_IP_NO_KIQ(ip, reg, value) \
+ __WREG32_SOC15_RLC__(reg, value, AMDGPU_REGS_NO_KIQ, ip##_HWIP)
+
#define WREG32_SOC15_NO_KIQ(ip, inst, reg, value) \
__WREG32_SOC15_RLC__(adev->reg_offset[ip##_HWIP][inst][reg##_BASE_IDX] + reg, \
value, AMDGPU_REGS_NO_KIQ, ip##_HWIP)
diff --git a/drivers/gpu/drm/amd/amdgpu/umc_v6_7.c b/drivers/gpu/drm/amd/amdgpu/umc_v6_7.c
index f7ec3fe..6dd1e19 100644
--- a/drivers/gpu/drm/amd/amdgpu/umc_v6_7.c
+++ b/drivers/gpu/drm/amd/amdgpu/umc_v6_7.c
@@ -50,6 +50,165 @@ static inline uint32_t get_umc_v6_7_reg_offset(struct amdgpu_device *adev,
return adev->umc.channel_offs * ch_inst + UMC_V6_7_INST_DIST * umc_inst;
}
+static inline uint32_t get_umc_v6_7_channel_index(struct amdgpu_device *adev,
+ uint32_t umc_inst,
+ uint32_t ch_inst)
+{
+ return adev->umc.channel_idx_tbl[umc_inst * adev->umc.channel_inst_num + ch_inst];
+}
+
+static void umc_v6_7_ecc_info_query_correctable_error_count(struct amdgpu_device *adev,
+ uint32_t channel_index,
+ unsigned long *error_count)
+{
+ uint32_t ecc_err_cnt;
+ uint64_t mc_umc_status;
+ struct amdgpu_ras *ras = amdgpu_ras_get_context(adev);
+
+ /*
+ * select the lower chip and check the error count
+ * skip add error count, calc error counter only from mca_umc_status
+ */
+ ecc_err_cnt = ras->umc_ecc.ecc[channel_index].ce_count_lo_chip;
+
+ /*
+ * select the higher chip and check the err counter
+ * skip add error count, calc error counter only from mca_umc_status
+ */
+ ecc_err_cnt = ras->umc_ecc.ecc[channel_index].ce_count_hi_chip;
+
+ /* check for SRAM correctable error
+ MCUMC_STATUS is a 64 bit register */
+ mc_umc_status = ras->umc_ecc.ecc[channel_index].mca_umc_status;
+ if (REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, Val) == 1 &&
+ REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, CECC) == 1)
+ *error_count += 1;
+}
+
+static void umc_v6_7_ecc_info_querry_uncorrectable_error_count(struct amdgpu_device *adev,
+ uint32_t channel_index,
+ unsigned long *error_count)
+{
+ uint64_t mc_umc_status;
+ struct amdgpu_ras *ras = amdgpu_ras_get_context(adev);
+
+ /* check the MCUMC_STATUS */
+ mc_umc_status = ras->umc_ecc.ecc[channel_index].mca_umc_status;
+ if ((REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, Val) == 1) &&
+ (REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, Deferred) == 1 ||
+ REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, UECC) == 1 ||
+ REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, PCC) == 1 ||
+ REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, UC) == 1 ||
+ REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, TCC) == 1))
+ *error_count += 1;
+}
+
+static void umc_v6_7_ecc_info_query_ras_error_count(struct amdgpu_device *adev,
+ void *ras_error_status)
+{
+ struct ras_err_data *err_data = (struct ras_err_data *)ras_error_status;
+
+ uint32_t umc_inst = 0;
+ uint32_t ch_inst = 0;
+ uint32_t umc_reg_offset = 0;
+ uint32_t channel_index = 0;
+
+ /*TODO: driver needs to toggle DF Cstate to ensure
+ * safe access of UMC registers. Will add the protection */
+ LOOP_UMC_INST_AND_CH(umc_inst, ch_inst) {
+ umc_reg_offset = get_umc_v6_7_reg_offset(adev,
+ umc_inst,
+ ch_inst);
+ channel_index = get_umc_v6_7_channel_index(adev,
+ umc_inst,
+ ch_inst);
+ umc_v6_7_ecc_info_query_correctable_error_count(adev,
+ channel_index,
+ &(err_data->ce_count));
+ umc_v6_7_ecc_info_querry_uncorrectable_error_count(adev,
+ channel_index,
+ &(err_data->ue_count));
+ }
+}
+
+static void umc_v6_7_ecc_info_query_error_address(struct amdgpu_device *adev,
+ struct ras_err_data *err_data,
+ uint32_t umc_reg_offset,
+ uint32_t ch_inst,
+ uint32_t umc_inst)
+{
+ uint64_t mc_umc_status, err_addr, retired_page;
+ struct eeprom_table_record *err_rec;
+ uint32_t channel_index;
+ struct amdgpu_ras *ras = amdgpu_ras_get_context(adev);
+
+ channel_index =
+ adev->umc.channel_idx_tbl[umc_inst * adev->umc.channel_inst_num + ch_inst];
+
+ mc_umc_status = ras->umc_ecc.ecc[channel_index].mca_umc_status;
+
+ if (mc_umc_status == 0)
+ return;
+
+ if (!err_data->err_addr)
+ return;
+
+ err_rec = &err_data->err_addr[err_data->err_addr_cnt];
+
+ /* calculate error address if ue/ce error is detected */
+ if (REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, Val) == 1 &&
+ (REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, UECC) == 1 ||
+ REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, CECC) == 1)) {
+
+ err_addr = ras->umc_ecc.ecc[channel_index].mca_umc_addr;
+ err_addr = REG_GET_FIELD(err_addr, MCA_UMC_UMC0_MCUMC_ADDRT0, ErrorAddr);
+
+ /* translate umc channel address to soc pa, 3 parts are included */
+ retired_page = ADDR_OF_8KB_BLOCK(err_addr) |
+ ADDR_OF_256B_BLOCK(channel_index) |
+ OFFSET_IN_256B_BLOCK(err_addr);
+
+ /* we only save ue error information currently, ce is skipped */
+ if (REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, UECC)
+ == 1) {
+ err_rec->address = err_addr;
+ /* page frame address is saved */
+ err_rec->retired_page = retired_page >> AMDGPU_GPU_PAGE_SHIFT;
+ err_rec->ts = (uint64_t)ktime_get_real_seconds();
+ err_rec->err_type = AMDGPU_RAS_EEPROM_ERR_NON_RECOVERABLE;
+ err_rec->cu = 0;
+ err_rec->mem_channel = channel_index;
+ err_rec->mcumc_id = umc_inst;
+
+ err_data->err_addr_cnt++;
+ }
+ }
+}
+
+static void umc_v6_7_ecc_info_query_ras_error_address(struct amdgpu_device *adev,
+ void *ras_error_status)
+{
+ struct ras_err_data *err_data = (struct ras_err_data *)ras_error_status;
+
+ uint32_t umc_inst = 0;
+ uint32_t ch_inst = 0;
+ uint32_t umc_reg_offset = 0;
+
+ /*TODO: driver needs to toggle DF Cstate to ensure
+ * safe access of UMC resgisters. Will add the protection
+ * when firmware interface is ready */
+ LOOP_UMC_INST_AND_CH(umc_inst, ch_inst) {
+ umc_reg_offset = get_umc_v6_7_reg_offset(adev,
+ umc_inst,
+ ch_inst);
+ umc_v6_7_ecc_info_query_error_address(adev,
+ err_data,
+ umc_reg_offset,
+ ch_inst,
+ umc_inst);
+ }
+}
+
static void umc_v6_7_query_correctable_error_count(struct amdgpu_device *adev,
uint32_t umc_reg_offset,
unsigned long *error_count)
@@ -327,4 +486,6 @@ const struct amdgpu_umc_ras_funcs umc_v6_7_ras_funcs = {
.query_ras_error_count = umc_v6_7_query_ras_error_count,
.query_ras_error_address = umc_v6_7_query_ras_error_address,
.query_ras_poison_mode = umc_v6_7_query_ras_poison_mode,
+ .ecc_info_query_ras_error_count = umc_v6_7_ecc_info_query_ras_error_count,
+ .ecc_info_query_ras_error_address = umc_v6_7_ecc_info_query_ras_error_address,
};
diff --git a/drivers/gpu/drm/amd/amdgpu/vcn_v1_0.c b/drivers/gpu/drm/amd/amdgpu/vcn_v1_0.c
index d54d720..3799226d 100644
--- a/drivers/gpu/drm/amd/amdgpu/vcn_v1_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/vcn_v1_0.c
@@ -246,6 +246,13 @@ static int vcn_v1_0_suspend(void *handle)
{
int r;
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ bool idle_work_unexecuted;
+
+ idle_work_unexecuted = cancel_delayed_work_sync(&adev->vcn.idle_work);
+ if (idle_work_unexecuted) {
+ if (adev->pm.dpm_enabled)
+ amdgpu_dpm_enable_uvd(adev, false);
+ }
r = vcn_v1_0_hw_fini(adev);
if (r)
diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
index a9ca698..3070466 100644
--- a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
@@ -640,6 +640,7 @@ const struct amd_ip_funcs vega10_ih_ip_funcs = {
static const struct amdgpu_ih_funcs vega10_ih_funcs = {
.get_wptr = vega10_ih_get_wptr,
.decode_iv = amdgpu_ih_decode_iv_helper,
+ .decode_iv_ts = amdgpu_ih_decode_iv_ts_helper,
.set_rptr = vega10_ih_set_rptr
};
diff --git a/drivers/gpu/drm/amd/amdgpu/vega20_ih.c b/drivers/gpu/drm/amd/amdgpu/vega20_ih.c
index f51dfc3..3b4eb82 100644
--- a/drivers/gpu/drm/amd/amdgpu/vega20_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/vega20_ih.c
@@ -688,6 +688,7 @@ const struct amd_ip_funcs vega20_ih_ip_funcs = {
static const struct amdgpu_ih_funcs vega20_ih_funcs = {
.get_wptr = vega20_ih_get_wptr,
.decode_iv = amdgpu_ih_decode_iv_helper,
+ .decode_iv_ts = amdgpu_ih_decode_iv_ts_helper,
.set_rptr = vega20_ih_set_rptr
};
diff --git a/drivers/gpu/drm/amd/amdkfd/cik_event_interrupt.c b/drivers/gpu/drm/amd/amdkfd/cik_event_interrupt.c
index f623301..d60576c 100644
--- a/drivers/gpu/drm/amd/amdkfd/cik_event_interrupt.c
+++ b/drivers/gpu/drm/amd/amdkfd/cik_event_interrupt.c
@@ -43,15 +43,15 @@ static bool cik_event_interrupt_isr(struct kfd_dev *dev,
*/
if ((ihre->source_id == CIK_INTSRC_GFX_PAGE_INV_FAULT ||
ihre->source_id == CIK_INTSRC_GFX_MEM_PROT_FAULT) &&
- dev->device_info->asic_family == CHIP_HAWAII) {
+ dev->adev->asic_type == CHIP_HAWAII) {
struct cik_ih_ring_entry *tmp_ihre =
(struct cik_ih_ring_entry *)patched_ihre;
*patched_flag = true;
*tmp_ihre = *ihre;
- vmid = f2g->read_vmid_from_vmfault_reg(dev->kgd);
- ret = f2g->get_atc_vmid_pasid_mapping_info(dev->kgd, vmid, &pasid);
+ vmid = f2g->read_vmid_from_vmfault_reg(dev->adev);
+ ret = f2g->get_atc_vmid_pasid_mapping_info(dev->adev, vmid, &pasid);
tmp_ihre->ring_id &= 0x000000ff;
tmp_ihre->ring_id |= vmid << 8;
@@ -113,7 +113,7 @@ static void cik_event_interrupt_wq(struct kfd_dev *dev,
kfd_process_vm_fault(dev->dqm, pasid);
memset(&info, 0, sizeof(info));
- amdgpu_amdkfd_gpuvm_get_vm_fault_info(dev->kgd, &info);
+ amdgpu_amdkfd_gpuvm_get_vm_fault_info(dev->adev, &info);
if (!info.page_addr && !info.status)
return;
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
index 24ebd61..4bfc0c8 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
@@ -321,7 +321,7 @@ static int kfd_ioctl_create_queue(struct file *filep, struct kfd_process *p,
/* Return gpu_id as doorbell offset for mmap usage */
args->doorbell_offset = KFD_MMAP_TYPE_DOORBELL;
args->doorbell_offset |= KFD_MMAP_GPU_ID(args->gpu_id);
- if (KFD_IS_SOC15(dev->device_info->asic_family))
+ if (KFD_IS_SOC15(dev))
/* On SOC15 ASICs, include the doorbell offset within the
* process doorbell frame, which is 2 pages.
*/
@@ -580,7 +580,7 @@ static int kfd_ioctl_dbg_register(struct file *filep,
if (!dev)
return -EINVAL;
- if (dev->device_info->asic_family == CHIP_CARRIZO) {
+ if (dev->adev->asic_type == CHIP_CARRIZO) {
pr_debug("kfd_ioctl_dbg_register not supported on CZ\n");
return -EINVAL;
}
@@ -631,7 +631,7 @@ static int kfd_ioctl_dbg_unregister(struct file *filep,
if (!dev || !dev->dbgmgr)
return -EINVAL;
- if (dev->device_info->asic_family == CHIP_CARRIZO) {
+ if (dev->adev->asic_type == CHIP_CARRIZO) {
pr_debug("kfd_ioctl_dbg_unregister not supported on CZ\n");
return -EINVAL;
}
@@ -676,7 +676,7 @@ static int kfd_ioctl_dbg_address_watch(struct file *filep,
if (!dev)
return -EINVAL;
- if (dev->device_info->asic_family == CHIP_CARRIZO) {
+ if (dev->adev->asic_type == CHIP_CARRIZO) {
pr_debug("kfd_ioctl_dbg_wave_control not supported on CZ\n");
return -EINVAL;
}
@@ -784,7 +784,7 @@ static int kfd_ioctl_dbg_wave_control(struct file *filep,
if (!dev)
return -EINVAL;
- if (dev->device_info->asic_family == CHIP_CARRIZO) {
+ if (dev->adev->asic_type == CHIP_CARRIZO) {
pr_debug("kfd_ioctl_dbg_wave_control not supported on CZ\n");
return -EINVAL;
}
@@ -851,7 +851,7 @@ static int kfd_ioctl_get_clock_counters(struct file *filep,
dev = kfd_device_by_id(args->gpu_id);
if (dev)
/* Reading GPU clock counter from KGD */
- args->gpu_clock_counter = amdgpu_amdkfd_get_gpu_clock_counter(dev->kgd);
+ args->gpu_clock_counter = amdgpu_amdkfd_get_gpu_clock_counter(dev->adev);
else
/* Node without GPU resource */
args->gpu_clock_counter = 0;
@@ -1041,7 +1041,7 @@ static int kfd_ioctl_create_event(struct file *filp, struct kfd_process *p,
goto out_unlock;
}
- err = amdgpu_amdkfd_gpuvm_map_gtt_bo_to_kernel(kfd->kgd,
+ err = amdgpu_amdkfd_gpuvm_map_gtt_bo_to_kernel(kfd->adev,
mem, &kern_addr, &size);
if (err) {
pr_err("Failed to map event page to kernel\n");
@@ -1051,7 +1051,7 @@ static int kfd_ioctl_create_event(struct file *filp, struct kfd_process *p,
err = kfd_event_page_set(p, kern_addr, size);
if (err) {
pr_err("Failed to set event page\n");
- amdgpu_amdkfd_gpuvm_unmap_gtt_bo_from_kernel(kfd->kgd, mem);
+ amdgpu_amdkfd_gpuvm_unmap_gtt_bo_from_kernel(kfd->adev, mem);
goto out_unlock;
}
@@ -1137,7 +1137,7 @@ static int kfd_ioctl_set_scratch_backing_va(struct file *filep,
if (dev->dqm->sched_policy == KFD_SCHED_POLICY_NO_HWS &&
pdd->qpd.vmid != 0 && dev->kfd2kgd->set_scratch_backing_va)
dev->kfd2kgd->set_scratch_backing_va(
- dev->kgd, args->va_addr, pdd->qpd.vmid);
+ dev->adev, args->va_addr, pdd->qpd.vmid);
return 0;
@@ -1158,7 +1158,7 @@ static int kfd_ioctl_get_tile_config(struct file *filep,
if (!dev)
return -EINVAL;
- amdgpu_amdkfd_get_tile_config(dev->kgd, &config);
+ amdgpu_amdkfd_get_tile_config(dev->adev, &config);
args->gb_addr_config = config.gb_addr_config;
args->num_banks = config.num_banks;
@@ -1244,7 +1244,7 @@ bool kfd_dev_is_large_bar(struct kfd_dev *dev)
if (dev->use_iommu_v2)
return false;
- amdgpu_amdkfd_get_local_mem_info(dev->kgd, &mem_info);
+ amdgpu_amdkfd_get_local_mem_info(dev->adev, &mem_info);
if (mem_info.local_mem_size_private == 0 &&
mem_info.local_mem_size_public > 0)
return true;
@@ -1313,7 +1313,7 @@ static int kfd_ioctl_alloc_memory_of_gpu(struct file *filep,
err = -EINVAL;
goto err_unlock;
}
- offset = amdgpu_amdkfd_get_mmio_remap_phys_addr(dev->kgd);
+ offset = dev->adev->rmmio_remap.bus_addr;
if (!offset) {
err = -ENOMEM;
goto err_unlock;
@@ -1321,7 +1321,7 @@ static int kfd_ioctl_alloc_memory_of_gpu(struct file *filep,
}
err = amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu(
- dev->kgd, args->va_addr, args->size,
+ dev->adev, args->va_addr, args->size,
pdd->drm_priv, (struct kgd_mem **) &mem, &offset,
flags);
@@ -1353,7 +1353,7 @@ static int kfd_ioctl_alloc_memory_of_gpu(struct file *filep,
return 0;
err_free:
- amdgpu_amdkfd_gpuvm_free_memory_of_gpu(dev->kgd, (struct kgd_mem *)mem,
+ amdgpu_amdkfd_gpuvm_free_memory_of_gpu(dev->adev, (struct kgd_mem *)mem,
pdd->drm_priv, NULL);
err_unlock:
mutex_unlock(&p->mutex);
@@ -1399,7 +1399,7 @@ static int kfd_ioctl_free_memory_of_gpu(struct file *filep,
goto err_unlock;
}
- ret = amdgpu_amdkfd_gpuvm_free_memory_of_gpu(dev->kgd,
+ ret = amdgpu_amdkfd_gpuvm_free_memory_of_gpu(dev->adev,
(struct kgd_mem *)mem, pdd->drm_priv, &size);
/* If freeing the buffer failed, leave the handle in place for
@@ -1484,7 +1484,7 @@ static int kfd_ioctl_map_memory_to_gpu(struct file *filep,
goto get_mem_obj_from_handle_failed;
}
err = amdgpu_amdkfd_gpuvm_map_memory_to_gpu(
- peer->kgd, (struct kgd_mem *)mem,
+ peer->adev, (struct kgd_mem *)mem,
peer_pdd->drm_priv, &table_freed);
if (err) {
pr_err("Failed to map to gpu %d/%d\n",
@@ -1496,7 +1496,7 @@ static int kfd_ioctl_map_memory_to_gpu(struct file *filep,
mutex_unlock(&p->mutex);
- err = amdgpu_amdkfd_gpuvm_sync_memory(dev->kgd, (struct kgd_mem *) mem, true);
+ err = amdgpu_amdkfd_gpuvm_sync_memory(dev->adev, (struct kgd_mem *) mem, true);
if (err) {
pr_debug("Sync memory failed, wait interrupted by user signal\n");
goto sync_memory_failed;
@@ -1593,7 +1593,7 @@ static int kfd_ioctl_unmap_memory_from_gpu(struct file *filep,
goto get_mem_obj_from_handle_failed;
}
err = amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu(
- peer->kgd, (struct kgd_mem *)mem, peer_pdd->drm_priv);
+ peer->adev, (struct kgd_mem *)mem, peer_pdd->drm_priv);
if (err) {
pr_err("Failed to unmap from gpu %d/%d\n",
i, args->n_devices);
@@ -1603,8 +1603,8 @@ static int kfd_ioctl_unmap_memory_from_gpu(struct file *filep,
}
mutex_unlock(&p->mutex);
- if (dev->device_info->asic_family == CHIP_ALDEBARAN) {
- err = amdgpu_amdkfd_gpuvm_sync_memory(dev->kgd,
+ if (KFD_GC_VERSION(dev) == IP_VERSION(9, 4, 2)) {
+ err = amdgpu_amdkfd_gpuvm_sync_memory(dev->adev,
(struct kgd_mem *) mem, true);
if (err) {
pr_debug("Sync memory failed, wait interrupted by user signal\n");
@@ -1680,7 +1680,7 @@ static int kfd_ioctl_get_dmabuf_info(struct file *filep,
{
struct kfd_ioctl_get_dmabuf_info_args *args = data;
struct kfd_dev *dev = NULL;
- struct kgd_dev *dma_buf_kgd;
+ struct amdgpu_device *dmabuf_adev;
void *metadata_buffer = NULL;
uint32_t flags;
unsigned int i;
@@ -1700,15 +1700,15 @@ static int kfd_ioctl_get_dmabuf_info(struct file *filep,
}
/* Get dmabuf info from KGD */
- r = amdgpu_amdkfd_get_dmabuf_info(dev->kgd, args->dmabuf_fd,
- &dma_buf_kgd, &args->size,
+ r = amdgpu_amdkfd_get_dmabuf_info(dev->adev, args->dmabuf_fd,
+ &dmabuf_adev, &args->size,
metadata_buffer, args->metadata_size,
&args->metadata_size, &flags);
if (r)
goto exit;
/* Reverse-lookup gpu_id from kgd pointer */
- dev = kfd_device_by_kgd(dma_buf_kgd);
+ dev = kfd_device_by_adev(dmabuf_adev);
if (!dev) {
r = -EINVAL;
goto exit;
@@ -1758,7 +1758,7 @@ static int kfd_ioctl_import_dmabuf(struct file *filep,
goto err_unlock;
}
- r = amdgpu_amdkfd_gpuvm_import_dmabuf(dev->kgd, dmabuf,
+ r = amdgpu_amdkfd_gpuvm_import_dmabuf(dev->adev, dmabuf,
args->va_addr, pdd->drm_priv,
(struct kgd_mem **)&mem, &size,
NULL);
@@ -1779,7 +1779,7 @@ static int kfd_ioctl_import_dmabuf(struct file *filep,
return 0;
err_free:
- amdgpu_amdkfd_gpuvm_free_memory_of_gpu(dev->kgd, (struct kgd_mem *)mem,
+ amdgpu_amdkfd_gpuvm_free_memory_of_gpu(dev->adev, (struct kgd_mem *)mem,
pdd->drm_priv, NULL);
err_unlock:
mutex_unlock(&p->mutex);
@@ -2066,7 +2066,7 @@ static int kfd_mmio_mmap(struct kfd_dev *dev, struct kfd_process *process,
if (vma->vm_end - vma->vm_start != PAGE_SIZE)
return -EINVAL;
- address = amdgpu_amdkfd_get_mmio_remap_phys_addr(dev->kgd);
+ address = dev->adev->rmmio_remap.bus_addr;
vma->vm_flags |= VM_IO | VM_DONTCOPY | VM_DONTEXPAND | VM_NORESERVE |
VM_DONTDUMP | VM_PFNMAP;
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_crat.c b/drivers/gpu/drm/amd/amdkfd/kfd_crat.c
index cfedfb1..f187596 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_crat.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_crat.c
@@ -1340,7 +1340,7 @@ static int kfd_fill_gpu_cache_info(struct kfd_dev *kdev,
int ret;
unsigned int num_cu_shared;
- switch (kdev->device_info->asic_family) {
+ switch (kdev->adev->asic_type) {
case CHIP_KAVERI:
pcache_info = kaveri_cache_info;
num_of_cache_types = ARRAY_SIZE(kaveri_cache_info);
@@ -1377,67 +1377,71 @@ static int kfd_fill_gpu_cache_info(struct kfd_dev *kdev,
pcache_info = vegam_cache_info;
num_of_cache_types = ARRAY_SIZE(vegam_cache_info);
break;
- case CHIP_VEGA10:
- pcache_info = vega10_cache_info;
- num_of_cache_types = ARRAY_SIZE(vega10_cache_info);
- break;
- case CHIP_VEGA12:
- pcache_info = vega12_cache_info;
- num_of_cache_types = ARRAY_SIZE(vega12_cache_info);
- break;
- case CHIP_VEGA20:
- case CHIP_ARCTURUS:
- pcache_info = vega20_cache_info;
- num_of_cache_types = ARRAY_SIZE(vega20_cache_info);
- break;
- case CHIP_ALDEBARAN:
- pcache_info = aldebaran_cache_info;
- num_of_cache_types = ARRAY_SIZE(aldebaran_cache_info);
- break;
- case CHIP_RAVEN:
- pcache_info = raven_cache_info;
- num_of_cache_types = ARRAY_SIZE(raven_cache_info);
- break;
- case CHIP_RENOIR:
- pcache_info = renoir_cache_info;
- num_of_cache_types = ARRAY_SIZE(renoir_cache_info);
- break;
- case CHIP_NAVI10:
- case CHIP_NAVI12:
- case CHIP_CYAN_SKILLFISH:
- pcache_info = navi10_cache_info;
- num_of_cache_types = ARRAY_SIZE(navi10_cache_info);
- break;
- case CHIP_NAVI14:
- pcache_info = navi14_cache_info;
- num_of_cache_types = ARRAY_SIZE(navi14_cache_info);
- break;
- case CHIP_SIENNA_CICHLID:
- pcache_info = sienna_cichlid_cache_info;
- num_of_cache_types = ARRAY_SIZE(sienna_cichlid_cache_info);
- break;
- case CHIP_NAVY_FLOUNDER:
- pcache_info = navy_flounder_cache_info;
- num_of_cache_types = ARRAY_SIZE(navy_flounder_cache_info);
- break;
- case CHIP_DIMGREY_CAVEFISH:
- pcache_info = dimgrey_cavefish_cache_info;
- num_of_cache_types = ARRAY_SIZE(dimgrey_cavefish_cache_info);
- break;
- case CHIP_VANGOGH:
- pcache_info = vangogh_cache_info;
- num_of_cache_types = ARRAY_SIZE(vangogh_cache_info);
- break;
- case CHIP_BEIGE_GOBY:
- pcache_info = beige_goby_cache_info;
- num_of_cache_types = ARRAY_SIZE(beige_goby_cache_info);
- break;
- case CHIP_YELLOW_CARP:
- pcache_info = yellow_carp_cache_info;
- num_of_cache_types = ARRAY_SIZE(yellow_carp_cache_info);
- break;
default:
- return -EINVAL;
+ switch(KFD_GC_VERSION(kdev)) {
+ case IP_VERSION(9, 0, 1):
+ pcache_info = vega10_cache_info;
+ num_of_cache_types = ARRAY_SIZE(vega10_cache_info);
+ break;
+ case IP_VERSION(9, 2, 1):
+ pcache_info = vega12_cache_info;
+ num_of_cache_types = ARRAY_SIZE(vega12_cache_info);
+ break;
+ case IP_VERSION(9, 4, 0):
+ case IP_VERSION(9, 4, 1):
+ pcache_info = vega20_cache_info;
+ num_of_cache_types = ARRAY_SIZE(vega20_cache_info);
+ break;
+ case IP_VERSION(9, 4, 2):
+ pcache_info = aldebaran_cache_info;
+ num_of_cache_types = ARRAY_SIZE(aldebaran_cache_info);
+ break;
+ case IP_VERSION(9, 1, 0):
+ case IP_VERSION(9, 2, 2):
+ pcache_info = raven_cache_info;
+ num_of_cache_types = ARRAY_SIZE(raven_cache_info);
+ break;
+ case IP_VERSION(9, 3, 0):
+ pcache_info = renoir_cache_info;
+ num_of_cache_types = ARRAY_SIZE(renoir_cache_info);
+ break;
+ case IP_VERSION(10, 1, 10):
+ case IP_VERSION(10, 1, 2):
+ case IP_VERSION(10, 1, 3):
+ pcache_info = navi10_cache_info;
+ num_of_cache_types = ARRAY_SIZE(navi10_cache_info);
+ break;
+ case IP_VERSION(10, 1, 1):
+ pcache_info = navi14_cache_info;
+ num_of_cache_types = ARRAY_SIZE(navi14_cache_info);
+ break;
+ case IP_VERSION(10, 3, 0):
+ pcache_info = sienna_cichlid_cache_info;
+ num_of_cache_types = ARRAY_SIZE(sienna_cichlid_cache_info);
+ break;
+ case IP_VERSION(10, 3, 2):
+ pcache_info = navy_flounder_cache_info;
+ num_of_cache_types = ARRAY_SIZE(navy_flounder_cache_info);
+ break;
+ case IP_VERSION(10, 3, 4):
+ pcache_info = dimgrey_cavefish_cache_info;
+ num_of_cache_types = ARRAY_SIZE(dimgrey_cavefish_cache_info);
+ break;
+ case IP_VERSION(10, 3, 1):
+ pcache_info = vangogh_cache_info;
+ num_of_cache_types = ARRAY_SIZE(vangogh_cache_info);
+ break;
+ case IP_VERSION(10, 3, 5):
+ pcache_info = beige_goby_cache_info;
+ num_of_cache_types = ARRAY_SIZE(beige_goby_cache_info);
+ break;
+ case IP_VERSION(10, 3, 3):
+ pcache_info = yellow_carp_cache_info;
+ num_of_cache_types = ARRAY_SIZE(yellow_carp_cache_info);
+ break;
+ default:
+ return -EINVAL;
+ }
}
*size_filled = 0;
@@ -1963,8 +1967,6 @@ static int kfd_fill_gpu_direct_io_link_to_cpu(int *avail_size,
struct crat_subtype_iolink *sub_type_hdr,
uint32_t proximity_domain)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)kdev->kgd;
-
*avail_size -= sizeof(struct crat_subtype_iolink);
if (*avail_size < 0)
return -ENOMEM;
@@ -1981,7 +1983,7 @@ static int kfd_fill_gpu_direct_io_link_to_cpu(int *avail_size,
/* Fill in IOLINK subtype.
* TODO: Fill-in other fields of iolink subtype
*/
- if (adev->gmc.xgmi.connected_to_cpu) {
+ if (kdev->adev->gmc.xgmi.connected_to_cpu) {
/*
* with host gpu xgmi link, host can access gpu memory whether
* or not pcie bar type is large, so always create bidirectional
@@ -1990,19 +1992,19 @@ static int kfd_fill_gpu_direct_io_link_to_cpu(int *avail_size,
sub_type_hdr->flags |= CRAT_IOLINK_FLAGS_BI_DIRECTIONAL;
sub_type_hdr->io_interface_type = CRAT_IOLINK_TYPE_XGMI;
sub_type_hdr->num_hops_xgmi = 1;
- if (adev->asic_type == CHIP_ALDEBARAN) {
+ if (KFD_GC_VERSION(kdev) == IP_VERSION(9, 4, 2)) {
sub_type_hdr->minimum_bandwidth_mbs =
amdgpu_amdkfd_get_xgmi_bandwidth_mbytes(
- kdev->kgd, NULL, true);
+ kdev->adev, NULL, true);
sub_type_hdr->maximum_bandwidth_mbs =
sub_type_hdr->minimum_bandwidth_mbs;
}
} else {
sub_type_hdr->io_interface_type = CRAT_IOLINK_TYPE_PCIEXPRESS;
sub_type_hdr->minimum_bandwidth_mbs =
- amdgpu_amdkfd_get_pcie_bandwidth_mbytes(kdev->kgd, true);
+ amdgpu_amdkfd_get_pcie_bandwidth_mbytes(kdev->adev, true);
sub_type_hdr->maximum_bandwidth_mbs =
- amdgpu_amdkfd_get_pcie_bandwidth_mbytes(kdev->kgd, false);
+ amdgpu_amdkfd_get_pcie_bandwidth_mbytes(kdev->adev, false);
}
sub_type_hdr->proximity_domain_from = proximity_domain;
@@ -2044,11 +2046,11 @@ static int kfd_fill_gpu_xgmi_link_to_gpu(int *avail_size,
sub_type_hdr->proximity_domain_from = proximity_domain_from;
sub_type_hdr->proximity_domain_to = proximity_domain_to;
sub_type_hdr->num_hops_xgmi =
- amdgpu_amdkfd_get_xgmi_hops_count(kdev->kgd, peer_kdev->kgd);
+ amdgpu_amdkfd_get_xgmi_hops_count(kdev->adev, peer_kdev->adev);
sub_type_hdr->maximum_bandwidth_mbs =
- amdgpu_amdkfd_get_xgmi_bandwidth_mbytes(kdev->kgd, peer_kdev->kgd, false);
+ amdgpu_amdkfd_get_xgmi_bandwidth_mbytes(kdev->adev, peer_kdev->adev, false);
sub_type_hdr->minimum_bandwidth_mbs = sub_type_hdr->maximum_bandwidth_mbs ?
- amdgpu_amdkfd_get_xgmi_bandwidth_mbytes(kdev->kgd, NULL, true) : 0;
+ amdgpu_amdkfd_get_xgmi_bandwidth_mbytes(kdev->adev, NULL, true) : 0;
return 0;
}
@@ -2114,7 +2116,7 @@ static int kfd_create_vcrat_image_gpu(void *pcrat_image,
cu->flags |= CRAT_CU_FLAGS_GPU_PRESENT;
cu->proximity_domain = proximity_domain;
- amdgpu_amdkfd_get_cu_info(kdev->kgd, &cu_info);
+ amdgpu_amdkfd_get_cu_info(kdev->adev, &cu_info);
cu->num_simd_per_cu = cu_info.simd_per_cu;
cu->num_simd_cores = cu_info.simd_per_cu * cu_info.cu_active_number;
cu->max_waves_simd = cu_info.max_waves_per_simd;
@@ -2145,7 +2147,7 @@ static int kfd_create_vcrat_image_gpu(void *pcrat_image,
* report the total FB size (public+private) as a single
* private heap.
*/
- amdgpu_amdkfd_get_local_mem_info(kdev->kgd, &local_mem_info);
+ amdgpu_amdkfd_get_local_mem_info(kdev->adev, &local_mem_info);
sub_type_hdr = (typeof(sub_type_hdr))((char *)sub_type_hdr +
sub_type_hdr->length);
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_dbgdev.c b/drivers/gpu/drm/amd/amdkfd/kfd_dbgdev.c
index 159add0f..1e30717 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_dbgdev.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_dbgdev.c
@@ -41,7 +41,7 @@
static void dbgdev_address_watch_disable_nodiq(struct kfd_dev *dev)
{
- dev->kfd2kgd->address_watch_disable(dev->kgd);
+ dev->kfd2kgd->address_watch_disable(dev->adev);
}
static int dbgdev_diq_submit_ib(struct kfd_dbgdev *dbgdev,
@@ -322,7 +322,7 @@ static int dbgdev_address_watch_nodiq(struct kfd_dbgdev *dbgdev,
pr_debug("\t\t%30s\n", "* * * * * * * * * * * * * * * * * *");
pdd->dev->kfd2kgd->address_watch_execute(
- dbgdev->dev->kgd,
+ dbgdev->dev->adev,
i,
cntl.u32All,
addrHi.u32All,
@@ -420,7 +420,7 @@ static int dbgdev_address_watch_diq(struct kfd_dbgdev *dbgdev,
aw_reg_add_dword =
dbgdev->dev->kfd2kgd->address_watch_get_offset(
- dbgdev->dev->kgd,
+ dbgdev->dev->adev,
i,
ADDRESS_WATCH_REG_CNTL);
@@ -431,7 +431,7 @@ static int dbgdev_address_watch_diq(struct kfd_dbgdev *dbgdev,
aw_reg_add_dword =
dbgdev->dev->kfd2kgd->address_watch_get_offset(
- dbgdev->dev->kgd,
+ dbgdev->dev->adev,
i,
ADDRESS_WATCH_REG_ADDR_HI);
@@ -441,7 +441,7 @@ static int dbgdev_address_watch_diq(struct kfd_dbgdev *dbgdev,
aw_reg_add_dword =
dbgdev->dev->kfd2kgd->address_watch_get_offset(
- dbgdev->dev->kgd,
+ dbgdev->dev->adev,
i,
ADDRESS_WATCH_REG_ADDR_LO);
@@ -457,7 +457,7 @@ static int dbgdev_address_watch_diq(struct kfd_dbgdev *dbgdev,
aw_reg_add_dword =
dbgdev->dev->kfd2kgd->address_watch_get_offset(
- dbgdev->dev->kgd,
+ dbgdev->dev->adev,
i,
ADDRESS_WATCH_REG_CNTL);
@@ -752,7 +752,7 @@ static int dbgdev_wave_control_nodiq(struct kfd_dbgdev *dbgdev,
pr_debug("\t\t %30s\n", "* * * * * * * * * * * * * * * * * *");
- return dbgdev->dev->kfd2kgd->wave_control_execute(dbgdev->dev->kgd,
+ return dbgdev->dev->kfd2kgd->wave_control_execute(dbgdev->dev->adev,
reg_gfx_index.u32All,
reg_sq_cmd.u32All);
}
@@ -784,7 +784,7 @@ int dbgdev_wave_reset_wavefronts(struct kfd_dev *dev, struct kfd_process *p)
for (vmid = first_vmid_to_scan; vmid <= last_vmid_to_scan; vmid++) {
status = dev->kfd2kgd->get_atc_vmid_pasid_mapping_info
- (dev->kgd, vmid, &queried_pasid);
+ (dev->adev, vmid, &queried_pasid);
if (status && queried_pasid == p->pasid) {
pr_debug("Killing wave fronts of vmid %d and pasid 0x%x\n",
@@ -811,7 +811,7 @@ int dbgdev_wave_reset_wavefronts(struct kfd_dev *dev, struct kfd_process *p)
/* for non DIQ we need to patch the VMID: */
reg_sq_cmd.bits.vm_id = vmid;
- dev->kfd2kgd->wave_control_execute(dev->kgd,
+ dev->kfd2kgd->wave_control_execute(dev->adev,
reg_gfx_index.u32All,
reg_sq_cmd.u32All);
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device.c b/drivers/gpu/drm/amd/amdkfd/kfd_device.c
index 3b119db1..127d41d 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_device.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_device.c
@@ -53,770 +53,310 @@ extern const struct kfd2kgd_calls aldebaran_kfd2kgd;
extern const struct kfd2kgd_calls gfx_v10_kfd2kgd;
extern const struct kfd2kgd_calls gfx_v10_3_kfd2kgd;
-#ifdef KFD_SUPPORT_IOMMU_V2
-static const struct kfd_device_info kaveri_device_info = {
- .asic_family = CHIP_KAVERI,
- .asic_name = "kaveri",
- .gfx_target_version = 70000,
- .max_pasid_bits = 16,
- /* max num of queues for KV.TODO should be a dynamic value */
- .max_no_of_hqd = 24,
- .doorbell_size = 4,
- .ih_ring_entry_size = 4 * sizeof(uint32_t),
- .event_interrupt_class = &event_interrupt_class_cik,
- .num_of_watch_points = 4,
- .mqd_size_aligned = MQD_SIZE_ALIGNED,
- .supports_cwsr = false,
- .needs_iommu_device = true,
- .needs_pci_atomics = false,
- .num_sdma_engines = 2,
- .num_xgmi_sdma_engines = 0,
- .num_sdma_queues_per_engine = 2,
-};
-
-static const struct kfd_device_info carrizo_device_info = {
- .asic_family = CHIP_CARRIZO,
- .asic_name = "carrizo",
- .gfx_target_version = 80001,
- .max_pasid_bits = 16,
- /* max num of queues for CZ.TODO should be a dynamic value */
- .max_no_of_hqd = 24,
- .doorbell_size = 4,
- .ih_ring_entry_size = 4 * sizeof(uint32_t),
- .event_interrupt_class = &event_interrupt_class_cik,
- .num_of_watch_points = 4,
- .mqd_size_aligned = MQD_SIZE_ALIGNED,
- .supports_cwsr = true,
- .needs_iommu_device = true,
- .needs_pci_atomics = false,
- .num_sdma_engines = 2,
- .num_xgmi_sdma_engines = 0,
- .num_sdma_queues_per_engine = 2,
-};
-
-static const struct kfd_device_info raven_device_info = {
- .asic_family = CHIP_RAVEN,
- .asic_name = "raven",
- .gfx_target_version = 90002,
- .max_pasid_bits = 16,
- .max_no_of_hqd = 24,
- .doorbell_size = 8,
- .ih_ring_entry_size = 8 * sizeof(uint32_t),
- .event_interrupt_class = &event_interrupt_class_v9,
- .num_of_watch_points = 4,
- .mqd_size_aligned = MQD_SIZE_ALIGNED,
- .supports_cwsr = true,
- .needs_iommu_device = true,
- .needs_pci_atomics = true,
- .num_sdma_engines = 1,
- .num_xgmi_sdma_engines = 0,
- .num_sdma_queues_per_engine = 2,
-};
-#endif
-
-#ifdef CONFIG_DRM_AMDGPU_CIK
-static const struct kfd_device_info hawaii_device_info = {
- .asic_family = CHIP_HAWAII,
- .asic_name = "hawaii",
- .gfx_target_version = 70001,
- .max_pasid_bits = 16,
- /* max num of queues for KV.TODO should be a dynamic value */
- .max_no_of_hqd = 24,
- .doorbell_size = 4,
- .ih_ring_entry_size = 4 * sizeof(uint32_t),
- .event_interrupt_class = &event_interrupt_class_cik,
- .num_of_watch_points = 4,
- .mqd_size_aligned = MQD_SIZE_ALIGNED,
- .supports_cwsr = false,
- .needs_iommu_device = false,
- .needs_pci_atomics = false,
- .num_sdma_engines = 2,
- .num_xgmi_sdma_engines = 0,
- .num_sdma_queues_per_engine = 2,
-};
-#endif
-
-static const struct kfd_device_info tonga_device_info = {
- .asic_family = CHIP_TONGA,
- .asic_name = "tonga",
- .gfx_target_version = 80002,
- .max_pasid_bits = 16,
- .max_no_of_hqd = 24,
- .doorbell_size = 4,
- .ih_ring_entry_size = 4 * sizeof(uint32_t),
- .event_interrupt_class = &event_interrupt_class_cik,
- .num_of_watch_points = 4,
- .mqd_size_aligned = MQD_SIZE_ALIGNED,
- .supports_cwsr = false,
- .needs_iommu_device = false,
- .needs_pci_atomics = true,
- .num_sdma_engines = 2,
- .num_xgmi_sdma_engines = 0,
- .num_sdma_queues_per_engine = 2,
-};
-
-static const struct kfd_device_info fiji_device_info = {
- .asic_family = CHIP_FIJI,
- .asic_name = "fiji",
- .gfx_target_version = 80003,
- .max_pasid_bits = 16,
- .max_no_of_hqd = 24,
- .doorbell_size = 4,
- .ih_ring_entry_size = 4 * sizeof(uint32_t),
- .event_interrupt_class = &event_interrupt_class_cik,
- .num_of_watch_points = 4,
- .mqd_size_aligned = MQD_SIZE_ALIGNED,
- .supports_cwsr = true,
- .needs_iommu_device = false,
- .needs_pci_atomics = true,
- .num_sdma_engines = 2,
- .num_xgmi_sdma_engines = 0,
- .num_sdma_queues_per_engine = 2,
-};
-
-static const struct kfd_device_info fiji_vf_device_info = {
- .asic_family = CHIP_FIJI,
- .asic_name = "fiji",
- .gfx_target_version = 80003,
- .max_pasid_bits = 16,
- .max_no_of_hqd = 24,
- .doorbell_size = 4,
- .ih_ring_entry_size = 4 * sizeof(uint32_t),
- .event_interrupt_class = &event_interrupt_class_cik,
- .num_of_watch_points = 4,
- .mqd_size_aligned = MQD_SIZE_ALIGNED,
- .supports_cwsr = true,
- .needs_iommu_device = false,
- .needs_pci_atomics = false,
- .num_sdma_engines = 2,
- .num_xgmi_sdma_engines = 0,
- .num_sdma_queues_per_engine = 2,
-};
-
-
-static const struct kfd_device_info polaris10_device_info = {
- .asic_family = CHIP_POLARIS10,
- .asic_name = "polaris10",
- .gfx_target_version = 80003,
- .max_pasid_bits = 16,
- .max_no_of_hqd = 24,
- .doorbell_size = 4,
- .ih_ring_entry_size = 4 * sizeof(uint32_t),
- .event_interrupt_class = &event_interrupt_class_cik,
- .num_of_watch_points = 4,
- .mqd_size_aligned = MQD_SIZE_ALIGNED,
- .supports_cwsr = true,
- .needs_iommu_device = false,
- .needs_pci_atomics = true,
- .num_sdma_engines = 2,
- .num_xgmi_sdma_engines = 0,
- .num_sdma_queues_per_engine = 2,
-};
-
-static const struct kfd_device_info polaris10_vf_device_info = {
- .asic_family = CHIP_POLARIS10,
- .asic_name = "polaris10",
- .gfx_target_version = 80003,
- .max_pasid_bits = 16,
- .max_no_of_hqd = 24,
- .doorbell_size = 4,
- .ih_ring_entry_size = 4 * sizeof(uint32_t),
- .event_interrupt_class = &event_interrupt_class_cik,
- .num_of_watch_points = 4,
- .mqd_size_aligned = MQD_SIZE_ALIGNED,
- .supports_cwsr = true,
- .needs_iommu_device = false,
- .needs_pci_atomics = false,
- .num_sdma_engines = 2,
- .num_xgmi_sdma_engines = 0,
- .num_sdma_queues_per_engine = 2,
-};
-
-static const struct kfd_device_info polaris11_device_info = {
- .asic_family = CHIP_POLARIS11,
- .asic_name = "polaris11",
- .gfx_target_version = 80003,
- .max_pasid_bits = 16,
- .max_no_of_hqd = 24,
- .doorbell_size = 4,
- .ih_ring_entry_size = 4 * sizeof(uint32_t),
- .event_interrupt_class = &event_interrupt_class_cik,
- .num_of_watch_points = 4,
- .mqd_size_aligned = MQD_SIZE_ALIGNED,
- .supports_cwsr = true,
- .needs_iommu_device = false,
- .needs_pci_atomics = true,
- .num_sdma_engines = 2,
- .num_xgmi_sdma_engines = 0,
- .num_sdma_queues_per_engine = 2,
-};
-
-static const struct kfd_device_info polaris12_device_info = {
- .asic_family = CHIP_POLARIS12,
- .asic_name = "polaris12",
- .gfx_target_version = 80003,
- .max_pasid_bits = 16,
- .max_no_of_hqd = 24,
- .doorbell_size = 4,
- .ih_ring_entry_size = 4 * sizeof(uint32_t),
- .event_interrupt_class = &event_interrupt_class_cik,
- .num_of_watch_points = 4,
- .mqd_size_aligned = MQD_SIZE_ALIGNED,
- .supports_cwsr = true,
- .needs_iommu_device = false,
- .needs_pci_atomics = true,
- .num_sdma_engines = 2,
- .num_xgmi_sdma_engines = 0,
- .num_sdma_queues_per_engine = 2,
-};
-
-static const struct kfd_device_info vegam_device_info = {
- .asic_family = CHIP_VEGAM,
- .asic_name = "vegam",
- .gfx_target_version = 80003,
- .max_pasid_bits = 16,
- .max_no_of_hqd = 24,
- .doorbell_size = 4,
- .ih_ring_entry_size = 4 * sizeof(uint32_t),
- .event_interrupt_class = &event_interrupt_class_cik,
- .num_of_watch_points = 4,
- .mqd_size_aligned = MQD_SIZE_ALIGNED,
- .supports_cwsr = true,
- .needs_iommu_device = false,
- .needs_pci_atomics = true,
- .num_sdma_engines = 2,
- .num_xgmi_sdma_engines = 0,
- .num_sdma_queues_per_engine = 2,
-};
-
-static const struct kfd_device_info vega10_device_info = {
- .asic_family = CHIP_VEGA10,
- .asic_name = "vega10",
- .gfx_target_version = 90000,
- .max_pasid_bits = 16,
- .max_no_of_hqd = 24,
- .doorbell_size = 8,
- .ih_ring_entry_size = 8 * sizeof(uint32_t),
- .event_interrupt_class = &event_interrupt_class_v9,
- .num_of_watch_points = 4,
- .mqd_size_aligned = MQD_SIZE_ALIGNED,
- .supports_cwsr = true,
- .needs_iommu_device = false,
- .needs_pci_atomics = false,
- .num_sdma_engines = 2,
- .num_xgmi_sdma_engines = 0,
- .num_sdma_queues_per_engine = 2,
-};
-
-static const struct kfd_device_info vega10_vf_device_info = {
- .asic_family = CHIP_VEGA10,
- .asic_name = "vega10",
- .gfx_target_version = 90000,
- .max_pasid_bits = 16,
- .max_no_of_hqd = 24,
- .doorbell_size = 8,
- .ih_ring_entry_size = 8 * sizeof(uint32_t),
- .event_interrupt_class = &event_interrupt_class_v9,
- .num_of_watch_points = 4,
- .mqd_size_aligned = MQD_SIZE_ALIGNED,
- .supports_cwsr = true,
- .needs_iommu_device = false,
- .needs_pci_atomics = false,
- .num_sdma_engines = 2,
- .num_xgmi_sdma_engines = 0,
- .num_sdma_queues_per_engine = 2,
-};
-
-static const struct kfd_device_info vega12_device_info = {
- .asic_family = CHIP_VEGA12,
- .asic_name = "vega12",
- .gfx_target_version = 90004,
- .max_pasid_bits = 16,
- .max_no_of_hqd = 24,
- .doorbell_size = 8,
- .ih_ring_entry_size = 8 * sizeof(uint32_t),
- .event_interrupt_class = &event_interrupt_class_v9,
- .num_of_watch_points = 4,
- .mqd_size_aligned = MQD_SIZE_ALIGNED,
- .supports_cwsr = true,
- .needs_iommu_device = false,
- .needs_pci_atomics = false,
- .num_sdma_engines = 2,
- .num_xgmi_sdma_engines = 0,
- .num_sdma_queues_per_engine = 2,
-};
-
-static const struct kfd_device_info vega20_device_info = {
- .asic_family = CHIP_VEGA20,
- .asic_name = "vega20",
- .gfx_target_version = 90006,
- .max_pasid_bits = 16,
- .max_no_of_hqd = 24,
- .doorbell_size = 8,
- .ih_ring_entry_size = 8 * sizeof(uint32_t),
- .event_interrupt_class = &event_interrupt_class_v9,
- .num_of_watch_points = 4,
- .mqd_size_aligned = MQD_SIZE_ALIGNED,
- .supports_cwsr = true,
- .needs_iommu_device = false,
- .needs_pci_atomics = false,
- .num_sdma_engines = 2,
- .num_xgmi_sdma_engines = 0,
- .num_sdma_queues_per_engine = 8,
-};
-
-static const struct kfd_device_info arcturus_device_info = {
- .asic_family = CHIP_ARCTURUS,
- .asic_name = "arcturus",
- .gfx_target_version = 90008,
- .max_pasid_bits = 16,
- .max_no_of_hqd = 24,
- .doorbell_size = 8,
- .ih_ring_entry_size = 8 * sizeof(uint32_t),
- .event_interrupt_class = &event_interrupt_class_v9,
- .num_of_watch_points = 4,
- .mqd_size_aligned = MQD_SIZE_ALIGNED,
- .supports_cwsr = true,
- .needs_iommu_device = false,
- .needs_pci_atomics = false,
- .num_sdma_engines = 2,
- .num_xgmi_sdma_engines = 6,
- .num_sdma_queues_per_engine = 8,
-};
-
-static const struct kfd_device_info aldebaran_device_info = {
- .asic_family = CHIP_ALDEBARAN,
- .asic_name = "aldebaran",
- .gfx_target_version = 90010,
- .max_pasid_bits = 16,
- .max_no_of_hqd = 24,
- .doorbell_size = 8,
- .ih_ring_entry_size = 8 * sizeof(uint32_t),
- .event_interrupt_class = &event_interrupt_class_v9,
- .num_of_watch_points = 4,
- .mqd_size_aligned = MQD_SIZE_ALIGNED,
- .supports_cwsr = true,
- .needs_iommu_device = false,
- .needs_pci_atomics = false,
- .num_sdma_engines = 2,
- .num_xgmi_sdma_engines = 3,
- .num_sdma_queues_per_engine = 8,
-};
-
-static const struct kfd_device_info renoir_device_info = {
- .asic_family = CHIP_RENOIR,
- .asic_name = "renoir",
- .gfx_target_version = 90012,
- .max_pasid_bits = 16,
- .max_no_of_hqd = 24,
- .doorbell_size = 8,
- .ih_ring_entry_size = 8 * sizeof(uint32_t),
- .event_interrupt_class = &event_interrupt_class_v9,
- .num_of_watch_points = 4,
- .mqd_size_aligned = MQD_SIZE_ALIGNED,
- .supports_cwsr = true,
- .needs_iommu_device = false,
- .needs_pci_atomics = false,
- .num_sdma_engines = 1,
- .num_xgmi_sdma_engines = 0,
- .num_sdma_queues_per_engine = 2,
-};
-
-static const struct kfd_device_info navi10_device_info = {
- .asic_family = CHIP_NAVI10,
- .asic_name = "navi10",
- .gfx_target_version = 100100,
- .max_pasid_bits = 16,
- .max_no_of_hqd = 24,
- .doorbell_size = 8,
- .ih_ring_entry_size = 8 * sizeof(uint32_t),
- .event_interrupt_class = &event_interrupt_class_v9,
- .num_of_watch_points = 4,
- .mqd_size_aligned = MQD_SIZE_ALIGNED,
- .needs_iommu_device = false,
- .supports_cwsr = true,
- .needs_pci_atomics = true,
- .no_atomic_fw_version = 145,
- .num_sdma_engines = 2,
- .num_xgmi_sdma_engines = 0,
- .num_sdma_queues_per_engine = 8,
-};
-
-static const struct kfd_device_info navi12_device_info = {
- .asic_family = CHIP_NAVI12,
- .asic_name = "navi12",
- .gfx_target_version = 100101,
- .max_pasid_bits = 16,
- .max_no_of_hqd = 24,
- .doorbell_size = 8,
- .ih_ring_entry_size = 8 * sizeof(uint32_t),
- .event_interrupt_class = &event_interrupt_class_v9,
- .num_of_watch_points = 4,
- .mqd_size_aligned = MQD_SIZE_ALIGNED,
- .needs_iommu_device = false,
- .supports_cwsr = true,
- .needs_pci_atomics = true,
- .no_atomic_fw_version = 145,
- .num_sdma_engines = 2,
- .num_xgmi_sdma_engines = 0,
- .num_sdma_queues_per_engine = 8,
-};
-
-static const struct kfd_device_info navi14_device_info = {
- .asic_family = CHIP_NAVI14,
- .asic_name = "navi14",
- .gfx_target_version = 100102,
- .max_pasid_bits = 16,
- .max_no_of_hqd = 24,
- .doorbell_size = 8,
- .ih_ring_entry_size = 8 * sizeof(uint32_t),
- .event_interrupt_class = &event_interrupt_class_v9,
- .num_of_watch_points = 4,
- .mqd_size_aligned = MQD_SIZE_ALIGNED,
- .needs_iommu_device = false,
- .supports_cwsr = true,
- .needs_pci_atomics = true,
- .no_atomic_fw_version = 145,
- .num_sdma_engines = 2,
- .num_xgmi_sdma_engines = 0,
- .num_sdma_queues_per_engine = 8,
-};
-
-static const struct kfd_device_info sienna_cichlid_device_info = {
- .asic_family = CHIP_SIENNA_CICHLID,
- .asic_name = "sienna_cichlid",
- .gfx_target_version = 100300,
- .max_pasid_bits = 16,
- .max_no_of_hqd = 24,
- .doorbell_size = 8,
- .ih_ring_entry_size = 8 * sizeof(uint32_t),
- .event_interrupt_class = &event_interrupt_class_v9,
- .num_of_watch_points = 4,
- .mqd_size_aligned = MQD_SIZE_ALIGNED,
- .needs_iommu_device = false,
- .supports_cwsr = true,
- .needs_pci_atomics = true,
- .no_atomic_fw_version = 92,
- .num_sdma_engines = 4,
- .num_xgmi_sdma_engines = 0,
- .num_sdma_queues_per_engine = 8,
-};
-
-static const struct kfd_device_info navy_flounder_device_info = {
- .asic_family = CHIP_NAVY_FLOUNDER,
- .asic_name = "navy_flounder",
- .gfx_target_version = 100301,
- .max_pasid_bits = 16,
- .max_no_of_hqd = 24,
- .doorbell_size = 8,
- .ih_ring_entry_size = 8 * sizeof(uint32_t),
- .event_interrupt_class = &event_interrupt_class_v9,
- .num_of_watch_points = 4,
- .mqd_size_aligned = MQD_SIZE_ALIGNED,
- .needs_iommu_device = false,
- .supports_cwsr = true,
- .needs_pci_atomics = true,
- .no_atomic_fw_version = 92,
- .num_sdma_engines = 2,
- .num_xgmi_sdma_engines = 0,
- .num_sdma_queues_per_engine = 8,
-};
-
-static const struct kfd_device_info vangogh_device_info = {
- .asic_family = CHIP_VANGOGH,
- .asic_name = "vangogh",
- .gfx_target_version = 100303,
- .max_pasid_bits = 16,
- .max_no_of_hqd = 24,
- .doorbell_size = 8,
- .ih_ring_entry_size = 8 * sizeof(uint32_t),
- .event_interrupt_class = &event_interrupt_class_v9,
- .num_of_watch_points = 4,
- .mqd_size_aligned = MQD_SIZE_ALIGNED,
- .needs_iommu_device = false,
- .supports_cwsr = true,
- .needs_pci_atomics = true,
- .no_atomic_fw_version = 92,
- .num_sdma_engines = 1,
- .num_xgmi_sdma_engines = 0,
- .num_sdma_queues_per_engine = 2,
-};
-
-static const struct kfd_device_info dimgrey_cavefish_device_info = {
- .asic_family = CHIP_DIMGREY_CAVEFISH,
- .asic_name = "dimgrey_cavefish",
- .gfx_target_version = 100302,
- .max_pasid_bits = 16,
- .max_no_of_hqd = 24,
- .doorbell_size = 8,
- .ih_ring_entry_size = 8 * sizeof(uint32_t),
- .event_interrupt_class = &event_interrupt_class_v9,
- .num_of_watch_points = 4,
- .mqd_size_aligned = MQD_SIZE_ALIGNED,
- .needs_iommu_device = false,
- .supports_cwsr = true,
- .needs_pci_atomics = true,
- .no_atomic_fw_version = 92,
- .num_sdma_engines = 2,
- .num_xgmi_sdma_engines = 0,
- .num_sdma_queues_per_engine = 8,
-};
-
-static const struct kfd_device_info beige_goby_device_info = {
- .asic_family = CHIP_BEIGE_GOBY,
- .asic_name = "beige_goby",
- .gfx_target_version = 100304,
- .max_pasid_bits = 16,
- .max_no_of_hqd = 24,
- .doorbell_size = 8,
- .ih_ring_entry_size = 8 * sizeof(uint32_t),
- .event_interrupt_class = &event_interrupt_class_v9,
- .num_of_watch_points = 4,
- .mqd_size_aligned = MQD_SIZE_ALIGNED,
- .needs_iommu_device = false,
- .supports_cwsr = true,
- .needs_pci_atomics = true,
- .no_atomic_fw_version = 92,
- .num_sdma_engines = 1,
- .num_xgmi_sdma_engines = 0,
- .num_sdma_queues_per_engine = 8,
-};
-
-static const struct kfd_device_info yellow_carp_device_info = {
- .asic_family = CHIP_YELLOW_CARP,
- .asic_name = "yellow_carp",
- .gfx_target_version = 100305,
- .max_pasid_bits = 16,
- .max_no_of_hqd = 24,
- .doorbell_size = 8,
- .ih_ring_entry_size = 8 * sizeof(uint32_t),
- .event_interrupt_class = &event_interrupt_class_v9,
- .num_of_watch_points = 4,
- .mqd_size_aligned = MQD_SIZE_ALIGNED,
- .needs_iommu_device = false,
- .supports_cwsr = true,
- .needs_pci_atomics = true,
- .no_atomic_fw_version = 92,
- .num_sdma_engines = 1,
- .num_xgmi_sdma_engines = 0,
- .num_sdma_queues_per_engine = 2,
-};
-
-static const struct kfd_device_info cyan_skillfish_device_info = {
- .asic_family = CHIP_CYAN_SKILLFISH,
- .asic_name = "cyan_skillfish",
- .gfx_target_version = 100103,
- .max_pasid_bits = 16,
- .max_no_of_hqd = 24,
- .doorbell_size = 8,
- .ih_ring_entry_size = 8 * sizeof(uint32_t),
- .event_interrupt_class = &event_interrupt_class_v9,
- .num_of_watch_points = 4,
- .mqd_size_aligned = MQD_SIZE_ALIGNED,
- .needs_iommu_device = false,
- .supports_cwsr = true,
- .needs_pci_atomics = true,
- .num_sdma_engines = 2,
- .num_xgmi_sdma_engines = 0,
- .num_sdma_queues_per_engine = 8,
-};
-
static int kfd_gtt_sa_init(struct kfd_dev *kfd, unsigned int buf_size,
unsigned int chunk_size);
static void kfd_gtt_sa_fini(struct kfd_dev *kfd);
static int kfd_resume(struct kfd_dev *kfd);
-struct kfd_dev *kgd2kfd_probe(struct kgd_dev *kgd, bool vf)
+static void kfd_device_info_set_sdma_queue_num(struct kfd_dev *kfd)
{
- struct kfd_dev *kfd;
- const struct kfd_device_info *device_info;
- const struct kfd2kgd_calls *f2g;
- struct amdgpu_device *adev = (struct amdgpu_device *)kgd;
+ uint32_t sdma_version = kfd->adev->ip_versions[SDMA0_HWIP][0];
+
+ switch (sdma_version) {
+ case IP_VERSION(4, 0, 0):/* VEGA10 */
+ case IP_VERSION(4, 0, 1):/* VEGA12 */
+ case IP_VERSION(4, 1, 0):/* RAVEN */
+ case IP_VERSION(4, 1, 1):/* RAVEN */
+ case IP_VERSION(4, 1, 2):/* RENIOR */
+ case IP_VERSION(5, 2, 1):/* VANGOGH */
+ case IP_VERSION(5, 2, 3):/* YELLOW_CARP */
+ kfd->device_info.num_sdma_queues_per_engine = 2;
+ break;
+ case IP_VERSION(4, 2, 0):/* VEGA20 */
+ case IP_VERSION(4, 2, 2):/* ARCTUTUS */
+ case IP_VERSION(4, 4, 0):/* ALDEBARAN */
+ case IP_VERSION(5, 0, 0):/* NAVI10 */
+ case IP_VERSION(5, 0, 1):/* CYAN_SKILLFISH */
+ case IP_VERSION(5, 0, 2):/* NAVI14 */
+ case IP_VERSION(5, 0, 5):/* NAVI12 */
+ case IP_VERSION(5, 2, 0):/* SIENNA_CICHLID */
+ case IP_VERSION(5, 2, 2):/* NAVY_FLOUDER */
+ case IP_VERSION(5, 2, 4):/* DIMGREY_CAVEFISH */
+ case IP_VERSION(5, 2, 5):/* BEIGE_GOBY */
+ kfd->device_info.num_sdma_queues_per_engine = 8;
+ break;
+ default:
+ dev_warn(kfd_device,
+ "Default sdma queue per engine(8) is set due to "
+ "mismatch of sdma ip block(SDMA_HWIP:0x%x).\n",
+ sdma_version);
+ kfd->device_info.num_sdma_queues_per_engine = 8;
+ }
+}
+
+static void kfd_device_info_set_event_interrupt_class(struct kfd_dev *kfd)
+{
+ uint32_t gc_version = KFD_GC_VERSION(kfd);
+
+ switch (gc_version) {
+ case IP_VERSION(9, 0, 1): /* VEGA10 */
+ case IP_VERSION(9, 1, 0): /* RAVEN */
+ case IP_VERSION(9, 2, 1): /* VEGA12 */
+ case IP_VERSION(9, 2, 2): /* RAVEN */
+ case IP_VERSION(9, 3, 0): /* RENOIR */
+ case IP_VERSION(9, 4, 0): /* VEGA20 */
+ case IP_VERSION(9, 4, 1): /* ARCTURUS */
+ case IP_VERSION(9, 4, 2): /* ALDEBARAN */
+ case IP_VERSION(10, 3, 1): /* VANGOGH */
+ case IP_VERSION(10, 3, 3): /* YELLOW_CARP */
+ case IP_VERSION(10, 1, 3): /* CYAN_SKILLFISH */
+ case IP_VERSION(10, 1, 10): /* NAVI10 */
+ case IP_VERSION(10, 1, 2): /* NAVI12 */
+ case IP_VERSION(10, 1, 1): /* NAVI14 */
+ case IP_VERSION(10, 3, 0): /* SIENNA_CICHLID */
+ case IP_VERSION(10, 3, 2): /* NAVY_FLOUNDER */
+ case IP_VERSION(10, 3, 4): /* DIMGREY_CAVEFISH */
+ case IP_VERSION(10, 3, 5): /* BEIGE_GOBY */
+ kfd->device_info.event_interrupt_class = &event_interrupt_class_v9;
+ break;
+ default:
+ dev_warn(kfd_device, "v9 event interrupt handler is set due to "
+ "mismatch of gc ip block(GC_HWIP:0x%x).\n", gc_version);
+ kfd->device_info.event_interrupt_class = &event_interrupt_class_v9;
+ }
+}
+
+static void kfd_device_info_init(struct kfd_dev *kfd,
+ bool vf, uint32_t gfx_target_version)
+{
+ uint32_t gc_version = KFD_GC_VERSION(kfd);
+ uint32_t asic_type = kfd->adev->asic_type;
+
+ kfd->device_info.max_pasid_bits = 16;
+ kfd->device_info.max_no_of_hqd = 24;
+ kfd->device_info.num_of_watch_points = 4;
+ kfd->device_info.mqd_size_aligned = MQD_SIZE_ALIGNED;
+ kfd->device_info.gfx_target_version = gfx_target_version;
+
+ if (KFD_IS_SOC15(kfd)) {
+ kfd->device_info.doorbell_size = 8;
+ kfd->device_info.ih_ring_entry_size = 8 * sizeof(uint32_t);
+ kfd->device_info.supports_cwsr = true;
+
+ kfd_device_info_set_sdma_queue_num(kfd);
+
+ kfd_device_info_set_event_interrupt_class(kfd);
+
+ /* Raven */
+ if (gc_version == IP_VERSION(9, 1, 0) ||
+ gc_version == IP_VERSION(9, 2, 2))
+ kfd->device_info.needs_iommu_device = true;
+
+ if (gc_version < IP_VERSION(11, 0, 0)) {
+ /* Navi2x+, Navi1x+ */
+ if (gc_version >= IP_VERSION(10, 3, 0))
+ kfd->device_info.no_atomic_fw_version = 92;
+ else if (gc_version >= IP_VERSION(10, 1, 1))
+ kfd->device_info.no_atomic_fw_version = 145;
+
+ /* Navi1x+ */
+ if (gc_version >= IP_VERSION(10, 1, 1))
+ kfd->device_info.needs_pci_atomics = true;
+ }
+ } else {
+ kfd->device_info.doorbell_size = 4;
+ kfd->device_info.ih_ring_entry_size = 4 * sizeof(uint32_t);
+ kfd->device_info.event_interrupt_class = &event_interrupt_class_cik;
+ kfd->device_info.num_sdma_queues_per_engine = 2;
+
+ if (asic_type != CHIP_KAVERI &&
+ asic_type != CHIP_HAWAII &&
+ asic_type != CHIP_TONGA)
+ kfd->device_info.supports_cwsr = true;
+
+ if (asic_type == CHIP_KAVERI ||
+ asic_type == CHIP_CARRIZO)
+ kfd->device_info.needs_iommu_device = true;
+
+ if (asic_type != CHIP_HAWAII && !vf)
+ kfd->device_info.needs_pci_atomics = true;
+ }
+}
+
+struct kfd_dev *kgd2kfd_probe(struct amdgpu_device *adev, bool vf)
+{
+ struct kfd_dev *kfd = NULL;
+ const struct kfd2kgd_calls *f2g = NULL;
struct pci_dev *pdev = adev->pdev;
+ uint32_t gfx_target_version = 0;
switch (adev->asic_type) {
#ifdef KFD_SUPPORT_IOMMU_V2
#ifdef CONFIG_DRM_AMDGPU_CIK
case CHIP_KAVERI:
- if (vf)
- device_info = NULL;
- else
- device_info = &kaveri_device_info;
- f2g = &gfx_v7_kfd2kgd;
+ gfx_target_version = 70000;
+ if (!vf)
+ f2g = &gfx_v7_kfd2kgd;
break;
#endif
case CHIP_CARRIZO:
- if (vf)
- device_info = NULL;
- else
- device_info = &carrizo_device_info;
- f2g = &gfx_v8_kfd2kgd;
+ gfx_target_version = 80001;
+ if (!vf)
+ f2g = &gfx_v8_kfd2kgd;
break;
#endif
#ifdef CONFIG_DRM_AMDGPU_CIK
case CHIP_HAWAII:
- if (vf)
- device_info = NULL;
- else
- device_info = &hawaii_device_info;
- f2g = &gfx_v7_kfd2kgd;
+ gfx_target_version = 70001;
+ if (!amdgpu_exp_hw_support)
+ pr_info(
+ "KFD support on Hawaii is experimental. See modparam exp_hw_support\n"
+ );
+ else if (!vf)
+ f2g = &gfx_v7_kfd2kgd;
break;
#endif
case CHIP_TONGA:
- if (vf)
- device_info = NULL;
- else
- device_info = &tonga_device_info;
- f2g = &gfx_v8_kfd2kgd;
+ gfx_target_version = 80002;
+ if (!vf)
+ f2g = &gfx_v8_kfd2kgd;
break;
case CHIP_FIJI:
- if (vf)
- device_info = &fiji_vf_device_info;
- else
- device_info = &fiji_device_info;
+ gfx_target_version = 80003;
f2g = &gfx_v8_kfd2kgd;
break;
case CHIP_POLARIS10:
- if (vf)
- device_info = &polaris10_vf_device_info;
- else
- device_info = &polaris10_device_info;
+ gfx_target_version = 80003;
f2g = &gfx_v8_kfd2kgd;
break;
case CHIP_POLARIS11:
- if (vf)
- device_info = NULL;
- else
- device_info = &polaris11_device_info;
- f2g = &gfx_v8_kfd2kgd;
+ gfx_target_version = 80003;
+ if (!vf)
+ f2g = &gfx_v8_kfd2kgd;
break;
case CHIP_POLARIS12:
- if (vf)
- device_info = NULL;
- else
- device_info = &polaris12_device_info;
- f2g = &gfx_v8_kfd2kgd;
+ gfx_target_version = 80003;
+ if (!vf)
+ f2g = &gfx_v8_kfd2kgd;
break;
case CHIP_VEGAM:
- if (vf)
- device_info = NULL;
- else
- device_info = &vegam_device_info;
- f2g = &gfx_v8_kfd2kgd;
+ gfx_target_version = 80003;
+ if (!vf)
+ f2g = &gfx_v8_kfd2kgd;
break;
default:
switch (adev->ip_versions[GC_HWIP][0]) {
+ /* Vega 10 */
case IP_VERSION(9, 0, 1):
- if (vf)
- device_info = &vega10_vf_device_info;
- else
- device_info = &vega10_device_info;
+ gfx_target_version = 90000;
f2g = &gfx_v9_kfd2kgd;
break;
#ifdef KFD_SUPPORT_IOMMU_V2
+ /* Raven */
case IP_VERSION(9, 1, 0):
case IP_VERSION(9, 2, 2):
- if (vf)
- device_info = NULL;
- else
- device_info = &raven_device_info;
- f2g = &gfx_v9_kfd2kgd;
+ gfx_target_version = 90002;
+ if (!vf)
+ f2g = &gfx_v9_kfd2kgd;
break;
#endif
+ /* Vega12 */
case IP_VERSION(9, 2, 1):
- if (vf)
- device_info = NULL;
- else
- device_info = &vega12_device_info;
- f2g = &gfx_v9_kfd2kgd;
+ gfx_target_version = 90004;
+ if (!vf)
+ f2g = &gfx_v9_kfd2kgd;
break;
+ /* Renoir */
case IP_VERSION(9, 3, 0):
- if (vf)
- device_info = NULL;
- else
- device_info = &renoir_device_info;
- f2g = &gfx_v9_kfd2kgd;
+ gfx_target_version = 90012;
+ if (!vf)
+ f2g = &gfx_v9_kfd2kgd;
break;
+ /* Vega20 */
case IP_VERSION(9, 4, 0):
- if (vf)
- device_info = NULL;
- else
- device_info = &vega20_device_info;
- f2g = &gfx_v9_kfd2kgd;
+ gfx_target_version = 90006;
+ if (!vf)
+ f2g = &gfx_v9_kfd2kgd;
break;
+ /* Arcturus */
case IP_VERSION(9, 4, 1):
- device_info = &arcturus_device_info;
+ gfx_target_version = 90008;
f2g = &arcturus_kfd2kgd;
break;
+ /* Aldebaran */
case IP_VERSION(9, 4, 2):
- device_info = &aldebaran_device_info;
+ gfx_target_version = 90010;
f2g = &aldebaran_kfd2kgd;
break;
+ /* Navi10 */
case IP_VERSION(10, 1, 10):
- if (vf)
- device_info = NULL;
- else
- device_info = &navi10_device_info;
- f2g = &gfx_v10_kfd2kgd;
+ gfx_target_version = 100100;
+ if (!vf)
+ f2g = &gfx_v10_kfd2kgd;
break;
+ /* Navi12 */
case IP_VERSION(10, 1, 2):
- device_info = &navi12_device_info;
+ gfx_target_version = 100101;
f2g = &gfx_v10_kfd2kgd;
break;
+ /* Navi14 */
case IP_VERSION(10, 1, 1):
- if (vf)
- device_info = NULL;
- else
- device_info = &navi14_device_info;
- f2g = &gfx_v10_kfd2kgd;
+ gfx_target_version = 100102;
+ if (!vf)
+ f2g = &gfx_v10_kfd2kgd;
break;
+ /* Cyan Skillfish */
case IP_VERSION(10, 1, 3):
- if (vf)
- device_info = NULL;
- else
- device_info = &cyan_skillfish_device_info;
- f2g = &gfx_v10_kfd2kgd;
+ gfx_target_version = 100103;
+ if (!vf)
+ f2g = &gfx_v10_kfd2kgd;
break;
+ /* Sienna Cichlid */
case IP_VERSION(10, 3, 0):
- device_info = &sienna_cichlid_device_info;
+ gfx_target_version = 100300;
f2g = &gfx_v10_3_kfd2kgd;
break;
+ /* Navy Flounder */
case IP_VERSION(10, 3, 2):
- device_info = &navy_flounder_device_info;
+ gfx_target_version = 100301;
f2g = &gfx_v10_3_kfd2kgd;
break;
+ /* Van Gogh */
case IP_VERSION(10, 3, 1):
- if (vf)
- device_info = NULL;
- else
- device_info = &vangogh_device_info;
- f2g = &gfx_v10_3_kfd2kgd;
+ gfx_target_version = 100303;
+ if (!vf)
+ f2g = &gfx_v10_3_kfd2kgd;
break;
+ /* Dimgrey Cavefish */
case IP_VERSION(10, 3, 4):
- device_info = &dimgrey_cavefish_device_info;
+ gfx_target_version = 100302;
f2g = &gfx_v10_3_kfd2kgd;
break;
+ /* Beige Goby */
case IP_VERSION(10, 3, 5):
- device_info = &beige_goby_device_info;
+ gfx_target_version = 100304;
f2g = &gfx_v10_3_kfd2kgd;
break;
+ /* Yellow Carp */
case IP_VERSION(10, 3, 3):
- if (vf)
- device_info = NULL;
- else
- device_info = &yellow_carp_device_info;
- f2g = &gfx_v10_3_kfd2kgd;
+ gfx_target_version = 100305;
+ if (!vf)
+ f2g = &gfx_v10_3_kfd2kgd;
break;
default:
- return NULL;
+ break;
}
break;
}
- if (!device_info || !f2g) {
- dev_err(kfd_device, "%s %s not supported in kfd\n",
- amdgpu_asic_name[adev->asic_type], vf ? "VF" : "");
+ if (!f2g) {
+ if (adev->ip_versions[GC_HWIP][0])
+ dev_err(kfd_device, "GC IP %06x %s not supported in kfd\n",
+ adev->ip_versions[GC_HWIP][0], vf ? "VF" : "");
+ else
+ dev_err(kfd_device, "%s %s not supported in kfd\n",
+ amdgpu_asic_name[adev->asic_type], vf ? "VF" : "");
return NULL;
}
@@ -824,8 +364,8 @@ struct kfd_dev *kgd2kfd_probe(struct kgd_dev *kgd, bool vf)
if (!kfd)
return NULL;
- kfd->kgd = kgd;
- kfd->device_info = device_info;
+ kfd->adev = adev;
+ kfd_device_info_init(kfd, vf, gfx_target_version);
kfd->pdev = pdev;
kfd->init_complete = false;
kfd->kfd2kgd = f2g;
@@ -844,24 +384,24 @@ struct kfd_dev *kgd2kfd_probe(struct kgd_dev *kgd, bool vf)
static void kfd_cwsr_init(struct kfd_dev *kfd)
{
- if (cwsr_enable && kfd->device_info->supports_cwsr) {
- if (kfd->device_info->asic_family < CHIP_VEGA10) {
+ if (cwsr_enable && kfd->device_info.supports_cwsr) {
+ if (KFD_GC_VERSION(kfd) < IP_VERSION(9, 0, 1)) {
BUILD_BUG_ON(sizeof(cwsr_trap_gfx8_hex) > PAGE_SIZE);
kfd->cwsr_isa = cwsr_trap_gfx8_hex;
kfd->cwsr_isa_size = sizeof(cwsr_trap_gfx8_hex);
- } else if (kfd->device_info->asic_family == CHIP_ARCTURUS) {
+ } else if (KFD_GC_VERSION(kfd) == IP_VERSION(9, 4, 1)) {
BUILD_BUG_ON(sizeof(cwsr_trap_arcturus_hex) > PAGE_SIZE);
kfd->cwsr_isa = cwsr_trap_arcturus_hex;
kfd->cwsr_isa_size = sizeof(cwsr_trap_arcturus_hex);
- } else if (kfd->device_info->asic_family == CHIP_ALDEBARAN) {
+ } else if (KFD_GC_VERSION(kfd) == IP_VERSION(9, 4, 2)) {
BUILD_BUG_ON(sizeof(cwsr_trap_aldebaran_hex) > PAGE_SIZE);
kfd->cwsr_isa = cwsr_trap_aldebaran_hex;
kfd->cwsr_isa_size = sizeof(cwsr_trap_aldebaran_hex);
- } else if (kfd->device_info->asic_family < CHIP_NAVI10) {
+ } else if (KFD_GC_VERSION(kfd) < IP_VERSION(10, 1, 1)) {
BUILD_BUG_ON(sizeof(cwsr_trap_gfx9_hex) > PAGE_SIZE);
kfd->cwsr_isa = cwsr_trap_gfx9_hex;
kfd->cwsr_isa_size = sizeof(cwsr_trap_gfx9_hex);
- } else if (kfd->device_info->asic_family < CHIP_SIENNA_CICHLID) {
+ } else if (KFD_GC_VERSION(kfd) < IP_VERSION(10, 3, 0)) {
BUILD_BUG_ON(sizeof(cwsr_trap_nv1x_hex) > PAGE_SIZE);
kfd->cwsr_isa = cwsr_trap_nv1x_hex;
kfd->cwsr_isa_size = sizeof(cwsr_trap_nv1x_hex);
@@ -882,18 +422,17 @@ static int kfd_gws_init(struct kfd_dev *kfd)
if (kfd->dqm->sched_policy == KFD_SCHED_POLICY_NO_HWS)
return 0;
- if (hws_gws_support
- || (kfd->device_info->asic_family == CHIP_VEGA10
- && kfd->mec2_fw_version >= 0x81b3)
- || (kfd->device_info->asic_family >= CHIP_VEGA12
- && kfd->device_info->asic_family <= CHIP_RAVEN
- && kfd->mec2_fw_version >= 0x1b3)
- || (kfd->device_info->asic_family == CHIP_ARCTURUS
- && kfd->mec2_fw_version >= 0x30)
- || (kfd->device_info->asic_family == CHIP_ALDEBARAN
- && kfd->mec2_fw_version >= 0x28))
- ret = amdgpu_amdkfd_alloc_gws(kfd->kgd,
- amdgpu_amdkfd_get_num_gws(kfd->kgd), &kfd->gws);
+ if (hws_gws_support || (KFD_IS_SOC15(kfd) &&
+ ((KFD_GC_VERSION(kfd) == IP_VERSION(9, 0, 1)
+ && kfd->mec2_fw_version >= 0x81b3) ||
+ (KFD_GC_VERSION(kfd) <= IP_VERSION(9, 4, 0)
+ && kfd->mec2_fw_version >= 0x1b3) ||
+ (KFD_GC_VERSION(kfd) == IP_VERSION(9, 4, 1)
+ && kfd->mec2_fw_version >= 0x30) ||
+ (KFD_GC_VERSION(kfd) == IP_VERSION(9, 4, 2)
+ && kfd->mec2_fw_version >= 0x28))))
+ ret = amdgpu_amdkfd_alloc_gws(kfd->adev,
+ kfd->adev->gds.gws_size, &kfd->gws);
return ret;
}
@@ -910,11 +449,11 @@ bool kgd2kfd_device_init(struct kfd_dev *kfd,
unsigned int size, map_process_packet_size;
kfd->ddev = ddev;
- kfd->mec_fw_version = amdgpu_amdkfd_get_fw_version(kfd->kgd,
+ kfd->mec_fw_version = amdgpu_amdkfd_get_fw_version(kfd->adev,
KGD_ENGINE_MEC1);
- kfd->mec2_fw_version = amdgpu_amdkfd_get_fw_version(kfd->kgd,
+ kfd->mec2_fw_version = amdgpu_amdkfd_get_fw_version(kfd->adev,
KGD_ENGINE_MEC2);
- kfd->sdma_fw_version = amdgpu_amdkfd_get_fw_version(kfd->kgd,
+ kfd->sdma_fw_version = amdgpu_amdkfd_get_fw_version(kfd->adev,
KGD_ENGINE_SDMA1);
kfd->shared_resources = *gpu_resources;
@@ -927,16 +466,16 @@ bool kgd2kfd_device_init(struct kfd_dev *kfd,
* 32 and 64-bit requests are possible and must be
* supported.
*/
- kfd->pci_atomic_requested = amdgpu_amdkfd_have_atomics_support(kfd->kgd);
+ kfd->pci_atomic_requested = amdgpu_amdkfd_have_atomics_support(kfd->adev);
if (!kfd->pci_atomic_requested &&
- kfd->device_info->needs_pci_atomics &&
- (!kfd->device_info->no_atomic_fw_version ||
- kfd->mec_fw_version < kfd->device_info->no_atomic_fw_version)) {
+ kfd->device_info.needs_pci_atomics &&
+ (!kfd->device_info.no_atomic_fw_version ||
+ kfd->mec_fw_version < kfd->device_info.no_atomic_fw_version)) {
dev_info(kfd_device,
"skipped device %x:%x, PCI rejects atomics %d<%d\n",
kfd->pdev->vendor, kfd->pdev->device,
kfd->mec_fw_version,
- kfd->device_info->no_atomic_fw_version);
+ kfd->device_info.no_atomic_fw_version);
return false;
}
@@ -953,16 +492,15 @@ bool kgd2kfd_device_init(struct kfd_dev *kfd,
/* calculate max size of mqds needed for queues */
size = max_num_of_queues_per_device *
- kfd->device_info->mqd_size_aligned;
+ kfd->device_info.mqd_size_aligned;
/*
* calculate max size of runlist packet.
* There can be only 2 packets at once
*/
- map_process_packet_size =
- kfd->device_info->asic_family == CHIP_ALDEBARAN ?
+ map_process_packet_size = KFD_GC_VERSION(kfd) == IP_VERSION(9, 4, 2) ?
sizeof(struct pm4_mes_map_process_aldebaran) :
- sizeof(struct pm4_mes_map_process);
+ sizeof(struct pm4_mes_map_process);
size += (KFD_MAX_NUM_OF_PROCESSES * map_process_packet_size +
max_num_of_queues_per_device * sizeof(struct pm4_mes_map_queues)
+ sizeof(struct pm4_mes_runlist)) * 2;
@@ -974,7 +512,7 @@ bool kgd2kfd_device_init(struct kfd_dev *kfd,
size += 512 * 1024;
if (amdgpu_amdkfd_alloc_gtt_mem(
- kfd->kgd, size, &kfd->gtt_mem,
+ kfd->adev, size, &kfd->gtt_mem,
&kfd->gtt_start_gpu_addr, &kfd->gtt_start_cpu_ptr,
false)) {
dev_err(kfd_device, "Could not allocate %d bytes\n", size);
@@ -995,9 +533,9 @@ bool kgd2kfd_device_init(struct kfd_dev *kfd,
goto kfd_doorbell_error;
}
- kfd->hive_id = amdgpu_amdkfd_get_hive_id(kfd->kgd);
+ kfd->hive_id = kfd->adev->gmc.xgmi.hive_id;
- kfd->noretry = amdgpu_amdkfd_get_noretry(kfd->kgd);
+ kfd->noretry = kfd->adev->gmc.noretry;
if (kfd_interrupt_init(kfd)) {
dev_err(kfd_device, "Error initializing interrupts\n");
@@ -1015,7 +553,7 @@ bool kgd2kfd_device_init(struct kfd_dev *kfd,
*/
if (kfd_gws_init(kfd)) {
dev_err(kfd_device, "Could not allocate %d gws\n",
- amdgpu_amdkfd_get_num_gws(kfd->kgd));
+ kfd->adev->gds.gws_size);
goto gws_error;
}
@@ -1030,7 +568,7 @@ bool kgd2kfd_device_init(struct kfd_dev *kfd,
kfd_cwsr_init(kfd);
- svm_migrate_init((struct amdgpu_device *)kfd->kgd);
+ svm_migrate_init(kfd->adev);
if(kgd2kfd_resume_iommu(kfd))
goto device_iommu_error;
@@ -1068,10 +606,10 @@ bool kgd2kfd_device_init(struct kfd_dev *kfd,
kfd_doorbell_error:
kfd_gtt_sa_fini(kfd);
kfd_gtt_sa_init_error:
- amdgpu_amdkfd_free_gtt_mem(kfd->kgd, kfd->gtt_mem);
+ amdgpu_amdkfd_free_gtt_mem(kfd->adev, kfd->gtt_mem);
alloc_gtt_mem_failure:
if (kfd->gws)
- amdgpu_amdkfd_free_gws(kfd->kgd, kfd->gws);
+ amdgpu_amdkfd_free_gws(kfd->adev, kfd->gws);
dev_err(kfd_device,
"device %x:%x NOT added due to errors\n",
kfd->pdev->vendor, kfd->pdev->device);
@@ -1088,9 +626,9 @@ void kgd2kfd_device_exit(struct kfd_dev *kfd)
kfd_doorbell_fini(kfd);
ida_destroy(&kfd->doorbell_ida);
kfd_gtt_sa_fini(kfd);
- amdgpu_amdkfd_free_gtt_mem(kfd->kgd, kfd->gtt_mem);
+ amdgpu_amdkfd_free_gtt_mem(kfd->adev, kfd->gtt_mem);
if (kfd->gws)
- amdgpu_amdkfd_free_gws(kfd->kgd, kfd->gws);
+ amdgpu_amdkfd_free_gws(kfd->adev, kfd->gws);
}
kfree(kfd);
@@ -1229,7 +767,7 @@ void kgd2kfd_interrupt(struct kfd_dev *kfd, const void *ih_ring_entry)
if (!kfd->init_complete)
return;
- if (kfd->device_info->ih_ring_entry_size > sizeof(patched_ihre)) {
+ if (kfd->device_info.ih_ring_entry_size > sizeof(patched_ihre)) {
dev_err_once(kfd_device, "Ring entry too small\n");
return;
}
@@ -1526,7 +1064,7 @@ void kgd2kfd_set_sram_ecc_flag(struct kfd_dev *kfd)
void kfd_inc_compute_active(struct kfd_dev *kfd)
{
if (atomic_inc_return(&kfd->compute_profile) == 1)
- amdgpu_amdkfd_set_compute_idle(kfd->kgd, false);
+ amdgpu_amdkfd_set_compute_idle(kfd->adev, false);
}
void kfd_dec_compute_active(struct kfd_dev *kfd)
@@ -1534,7 +1072,7 @@ void kfd_dec_compute_active(struct kfd_dev *kfd)
int count = atomic_dec_return(&kfd->compute_profile);
if (count == 0)
- amdgpu_amdkfd_set_compute_idle(kfd->kgd, true);
+ amdgpu_amdkfd_set_compute_idle(kfd->adev, true);
WARN_ONCE(count < 0, "Compute profile ref. count error");
}
@@ -1544,6 +1082,26 @@ void kgd2kfd_smi_event_throttle(struct kfd_dev *kfd, uint64_t throttle_bitmask)
kfd_smi_event_update_thermal_throttling(kfd, throttle_bitmask);
}
+/* kfd_get_num_sdma_engines returns the number of PCIe optimized SDMA and
+ * kfd_get_num_xgmi_sdma_engines returns the number of XGMI SDMA.
+ * When the device has more than two engines, we reserve two for PCIe to enable
+ * full-duplex and the rest are used as XGMI.
+ */
+unsigned int kfd_get_num_sdma_engines(struct kfd_dev *kdev)
+{
+ /* If XGMI is not supported, all SDMA engines are PCIe */
+ if (!kdev->adev->gmc.xgmi.supported)
+ return kdev->adev->sdma.num_instances;
+
+ return min(kdev->adev->sdma.num_instances, 2);
+}
+
+unsigned int kfd_get_num_xgmi_sdma_engines(struct kfd_dev *kdev)
+{
+ /* After reserved for PCIe, the rest of engines are XGMI */
+ return kdev->adev->sdma.num_instances - kfd_get_num_sdma_engines(kdev);
+}
+
#if defined(CONFIG_DEBUG_FS)
/* This function will send a package to HIQ to hang the HWS
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
index 93e33dd..19890e3 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
@@ -47,7 +47,7 @@ static int execute_queues_cpsch(struct device_queue_manager *dqm,
uint32_t filter_param);
static int unmap_queues_cpsch(struct device_queue_manager *dqm,
enum kfd_unmap_queues_filter filter,
- uint32_t filter_param);
+ uint32_t filter_param, bool reset);
static int map_queues_cpsch(struct device_queue_manager *dqm);
@@ -99,38 +99,29 @@ unsigned int get_pipes_per_mec(struct device_queue_manager *dqm)
return dqm->dev->shared_resources.num_pipe_per_mec;
}
-static unsigned int get_num_sdma_engines(struct device_queue_manager *dqm)
-{
- return dqm->dev->device_info->num_sdma_engines;
-}
-
-static unsigned int get_num_xgmi_sdma_engines(struct device_queue_manager *dqm)
-{
- return dqm->dev->device_info->num_xgmi_sdma_engines;
-}
-
static unsigned int get_num_all_sdma_engines(struct device_queue_manager *dqm)
{
- return get_num_sdma_engines(dqm) + get_num_xgmi_sdma_engines(dqm);
+ return kfd_get_num_sdma_engines(dqm->dev) +
+ kfd_get_num_xgmi_sdma_engines(dqm->dev);
}
unsigned int get_num_sdma_queues(struct device_queue_manager *dqm)
{
- return dqm->dev->device_info->num_sdma_engines
- * dqm->dev->device_info->num_sdma_queues_per_engine;
+ return kfd_get_num_sdma_engines(dqm->dev) *
+ dqm->dev->device_info.num_sdma_queues_per_engine;
}
unsigned int get_num_xgmi_sdma_queues(struct device_queue_manager *dqm)
{
- return dqm->dev->device_info->num_xgmi_sdma_engines
- * dqm->dev->device_info->num_sdma_queues_per_engine;
+ return kfd_get_num_xgmi_sdma_engines(dqm->dev) *
+ dqm->dev->device_info.num_sdma_queues_per_engine;
}
void program_sh_mem_settings(struct device_queue_manager *dqm,
struct qcm_process_device *qpd)
{
return dqm->dev->kfd2kgd->program_sh_mem_settings(
- dqm->dev->kgd, qpd->vmid,
+ dqm->dev->adev, qpd->vmid,
qpd->sh_mem_config,
qpd->sh_mem_ape1_base,
qpd->sh_mem_ape1_limit,
@@ -157,7 +148,7 @@ static int allocate_doorbell(struct qcm_process_device *qpd, struct queue *q)
{
struct kfd_dev *dev = qpd->dqm->dev;
- if (!KFD_IS_SOC15(dev->device_info->asic_family)) {
+ if (!KFD_IS_SOC15(dev)) {
/* On pre-SOC15 chips we need to use the queue ID to
* preserve the user mode ABI.
*/
@@ -202,7 +193,7 @@ static void deallocate_doorbell(struct qcm_process_device *qpd,
unsigned int old;
struct kfd_dev *dev = qpd->dqm->dev;
- if (!KFD_IS_SOC15(dev->device_info->asic_family) ||
+ if (!KFD_IS_SOC15(dev) ||
q->properties.type == KFD_QUEUE_TYPE_SDMA ||
q->properties.type == KFD_QUEUE_TYPE_SDMA_XGMI)
return;
@@ -216,7 +207,7 @@ static void program_trap_handler_settings(struct device_queue_manager *dqm,
{
if (dqm->dev->kfd2kgd->program_trap_handler_settings)
dqm->dev->kfd2kgd->program_trap_handler_settings(
- dqm->dev->kgd, qpd->vmid,
+ dqm->dev->adev, qpd->vmid,
qpd->tba_addr, qpd->tma_addr);
}
@@ -250,21 +241,20 @@ static int allocate_vmid(struct device_queue_manager *dqm,
program_sh_mem_settings(dqm, qpd);
- if (dqm->dev->device_info->asic_family >= CHIP_VEGA10 &&
- dqm->dev->cwsr_enabled)
+ if (KFD_IS_SOC15(dqm->dev) && dqm->dev->cwsr_enabled)
program_trap_handler_settings(dqm, qpd);
/* qpd->page_table_base is set earlier when register_process()
* is called, i.e. when the first queue is created.
*/
- dqm->dev->kfd2kgd->set_vm_context_page_table_base(dqm->dev->kgd,
+ dqm->dev->kfd2kgd->set_vm_context_page_table_base(dqm->dev->adev,
qpd->vmid,
qpd->page_table_base);
/* invalidate the VM context after pasid and vmid mapping is set up */
kfd_flush_tlb(qpd_to_pdd(qpd), TLB_FLUSH_LEGACY);
if (dqm->dev->kfd2kgd->set_scratch_backing_va)
- dqm->dev->kfd2kgd->set_scratch_backing_va(dqm->dev->kgd,
+ dqm->dev->kfd2kgd->set_scratch_backing_va(dqm->dev->adev,
qpd->sh_hidden_private_base, qpd->vmid);
return 0;
@@ -283,7 +273,7 @@ static int flush_texture_cache_nocpsch(struct kfd_dev *kdev,
if (ret)
return ret;
- return amdgpu_amdkfd_submit_ib(kdev->kgd, KGD_ENGINE_MEC1, qpd->vmid,
+ return amdgpu_amdkfd_submit_ib(kdev->adev, KGD_ENGINE_MEC1, qpd->vmid,
qpd->ib_base, (uint32_t *)qpd->ib_kaddr,
pmf->release_mem_size / sizeof(uint32_t));
}
@@ -293,7 +283,7 @@ static void deallocate_vmid(struct device_queue_manager *dqm,
struct queue *q)
{
/* On GFX v7, CP doesn't flush TC at dequeue */
- if (q->device->device_info->asic_family == CHIP_HAWAII)
+ if (q->device->adev->asic_type == CHIP_HAWAII)
if (flush_texture_cache_nocpsch(q->device, qpd))
pr_err("Failed to flush TC\n");
@@ -580,7 +570,7 @@ static int update_queue(struct device_queue_manager *dqm, struct queue *q,
/* Make sure the queue is unmapped before updating the MQD */
if (dqm->sched_policy != KFD_SCHED_POLICY_NO_HWS) {
retval = unmap_queues_cpsch(dqm,
- KFD_UNMAP_QUEUES_FILTER_DYNAMIC_QUEUES, 0);
+ KFD_UNMAP_QUEUES_FILTER_DYNAMIC_QUEUES, 0, false);
if (retval) {
pr_err("unmap queue failed\n");
goto out_unlock;
@@ -776,7 +766,7 @@ static int restore_process_queues_nocpsch(struct device_queue_manager *dqm,
if (!list_empty(&qpd->queues_list)) {
dqm->dev->kfd2kgd->set_vm_context_page_table_base(
- dqm->dev->kgd,
+ dqm->dev->adev,
qpd->vmid,
qpd->page_table_base);
kfd_flush_tlb(pdd, TLB_FLUSH_LEGACY);
@@ -954,7 +944,7 @@ set_pasid_vmid_mapping(struct device_queue_manager *dqm, u32 pasid,
unsigned int vmid)
{
return dqm->dev->kfd2kgd->set_pasid_vmid_mapping(
- dqm->dev->kgd, pasid, vmid);
+ dqm->dev->adev, pasid, vmid);
}
static void init_interrupts(struct device_queue_manager *dqm)
@@ -963,7 +953,7 @@ static void init_interrupts(struct device_queue_manager *dqm)
for (i = 0 ; i < get_pipes_per_mec(dqm) ; i++)
if (is_pipe_enabled(dqm, 0, i))
- dqm->dev->kfd2kgd->init_interrupts(dqm->dev->kgd, i);
+ dqm->dev->kfd2kgd->init_interrupts(dqm->dev->adev, i);
}
static int initialize_nocpsch(struct device_queue_manager *dqm)
@@ -1017,7 +1007,7 @@ static int start_nocpsch(struct device_queue_manager *dqm)
pr_info("SW scheduler is used");
init_interrupts(dqm);
- if (dqm->dev->device_info->asic_family == CHIP_HAWAII)
+ if (dqm->dev->adev->asic_type == CHIP_HAWAII)
return pm_init(&dqm->packet_mgr, dqm);
dqm->sched_running = true;
@@ -1026,7 +1016,7 @@ static int start_nocpsch(struct device_queue_manager *dqm)
static int stop_nocpsch(struct device_queue_manager *dqm)
{
- if (dqm->dev->device_info->asic_family == CHIP_HAWAII)
+ if (dqm->dev->adev->asic_type == CHIP_HAWAII)
pm_uninit(&dqm->packet_mgr, false);
dqm->sched_running = false;
@@ -1055,9 +1045,9 @@ static int allocate_sdma_queue(struct device_queue_manager *dqm,
dqm->sdma_bitmap &= ~(1ULL << bit);
q->sdma_id = bit;
q->properties.sdma_engine_id = q->sdma_id %
- get_num_sdma_engines(dqm);
+ kfd_get_num_sdma_engines(dqm->dev);
q->properties.sdma_queue_id = q->sdma_id /
- get_num_sdma_engines(dqm);
+ kfd_get_num_sdma_engines(dqm->dev);
} else if (q->properties.type == KFD_QUEUE_TYPE_SDMA_XGMI) {
if (dqm->xgmi_sdma_bitmap == 0) {
pr_err("No more XGMI SDMA queue to allocate\n");
@@ -1072,10 +1062,11 @@ static int allocate_sdma_queue(struct device_queue_manager *dqm,
* assumes the first N engines are always
* PCIe-optimized ones
*/
- q->properties.sdma_engine_id = get_num_sdma_engines(dqm) +
- q->sdma_id % get_num_xgmi_sdma_engines(dqm);
+ q->properties.sdma_engine_id =
+ kfd_get_num_sdma_engines(dqm->dev) +
+ q->sdma_id % kfd_get_num_xgmi_sdma_engines(dqm->dev);
q->properties.sdma_queue_id = q->sdma_id /
- get_num_xgmi_sdma_engines(dqm);
+ kfd_get_num_xgmi_sdma_engines(dqm->dev);
}
pr_debug("SDMA engine id: %d\n", q->properties.sdma_engine_id);
@@ -1132,7 +1123,7 @@ static int set_sched_resources(struct device_queue_manager *dqm)
res.queue_mask |= 1ull
<< amdgpu_queue_mask_bit_to_set_resource_bit(
- (struct amdgpu_device *)dqm->dev->kgd, i);
+ dqm->dev->adev, i);
}
res.gws_mask = ~0ull;
res.oac_mask = res.gds_heap_base = res.gds_heap_size = 0;
@@ -1232,7 +1223,7 @@ static int stop_cpsch(struct device_queue_manager *dqm)
}
if (!dqm->is_hws_hang)
- unmap_queues_cpsch(dqm, KFD_UNMAP_QUEUES_FILTER_ALL_QUEUES, 0);
+ unmap_queues_cpsch(dqm, KFD_UNMAP_QUEUES_FILTER_ALL_QUEUES, 0, false);
hanging = dqm->is_hws_hang || dqm->is_resetting;
dqm->sched_running = false;
@@ -1428,7 +1419,7 @@ static int map_queues_cpsch(struct device_queue_manager *dqm)
/* dqm->lock mutex has to be locked before calling this function */
static int unmap_queues_cpsch(struct device_queue_manager *dqm,
enum kfd_unmap_queues_filter filter,
- uint32_t filter_param)
+ uint32_t filter_param, bool reset)
{
int retval = 0;
struct mqd_manager *mqd_mgr;
@@ -1441,7 +1432,7 @@ static int unmap_queues_cpsch(struct device_queue_manager *dqm,
return retval;
retval = pm_send_unmap_queue(&dqm->packet_mgr, KFD_QUEUE_TYPE_COMPUTE,
- filter, filter_param, false, 0);
+ filter, filter_param, reset, 0);
if (retval)
return retval;
@@ -1485,6 +1476,21 @@ static int unmap_queues_cpsch(struct device_queue_manager *dqm,
return retval;
}
+/* only for compute queue */
+static int reset_queues_cpsch(struct device_queue_manager *dqm,
+ uint16_t pasid)
+{
+ int retval;
+
+ dqm_lock(dqm);
+
+ retval = unmap_queues_cpsch(dqm, KFD_UNMAP_QUEUES_FILTER_BY_PASID,
+ pasid, true);
+
+ dqm_unlock(dqm);
+ return retval;
+}
+
/* dqm->lock mutex has to be locked before calling this function */
static int execute_queues_cpsch(struct device_queue_manager *dqm,
enum kfd_unmap_queues_filter filter,
@@ -1494,7 +1500,7 @@ static int execute_queues_cpsch(struct device_queue_manager *dqm,
if (dqm->is_hws_hang)
return -EIO;
- retval = unmap_queues_cpsch(dqm, filter, filter_param);
+ retval = unmap_queues_cpsch(dqm, filter, filter_param, false);
if (retval)
return retval;
@@ -1847,10 +1853,10 @@ static int allocate_hiq_sdma_mqd(struct device_queue_manager *dqm)
struct kfd_mem_obj *mem_obj = &dqm->hiq_sdma_mqd;
uint32_t size = dqm->mqd_mgrs[KFD_MQD_TYPE_SDMA]->mqd_size *
get_num_all_sdma_engines(dqm) *
- dev->device_info->num_sdma_queues_per_engine +
+ dev->device_info.num_sdma_queues_per_engine +
dqm->mqd_mgrs[KFD_MQD_TYPE_HIQ]->mqd_size;
- retval = amdgpu_amdkfd_alloc_gtt_mem(dev->kgd, size,
+ retval = amdgpu_amdkfd_alloc_gtt_mem(dev->adev, size,
&(mem_obj->gtt_mem), &(mem_obj->gpu_addr),
(void *)&(mem_obj->cpu_ptr), false);
@@ -1867,7 +1873,7 @@ struct device_queue_manager *device_queue_manager_init(struct kfd_dev *dev)
if (!dqm)
return NULL;
- switch (dev->device_info->asic_family) {
+ switch (dev->adev->asic_type) {
/* HWS is not available on Hawaii. */
case CHIP_HAWAII:
/* HWS depends on CWSR for timely dequeue. CWSR is not
@@ -1905,6 +1911,7 @@ struct device_queue_manager *device_queue_manager_init(struct kfd_dev *dev)
dqm->ops.evict_process_queues = evict_process_queues_cpsch;
dqm->ops.restore_process_queues = restore_process_queues_cpsch;
dqm->ops.get_wave_state = get_wave_state;
+ dqm->ops.reset_queues = reset_queues_cpsch;
break;
case KFD_SCHED_POLICY_NO_HWS:
/* initialize dqm for no cp scheduling */
@@ -1930,7 +1937,7 @@ struct device_queue_manager *device_queue_manager_init(struct kfd_dev *dev)
goto out_free;
}
- switch (dev->device_info->asic_family) {
+ switch (dev->adev->asic_type) {
case CHIP_CARRIZO:
device_queue_manager_init_vi(&dqm->asic_ops);
break;
@@ -1952,31 +1959,16 @@ struct device_queue_manager *device_queue_manager_init(struct kfd_dev *dev)
device_queue_manager_init_vi_tonga(&dqm->asic_ops);
break;
- case CHIP_VEGA10:
- case CHIP_VEGA12:
- case CHIP_VEGA20:
- case CHIP_RAVEN:
- case CHIP_RENOIR:
- case CHIP_ARCTURUS:
- case CHIP_ALDEBARAN:
- device_queue_manager_init_v9(&dqm->asic_ops);
- break;
- case CHIP_NAVI10:
- case CHIP_NAVI12:
- case CHIP_NAVI14:
- case CHIP_SIENNA_CICHLID:
- case CHIP_NAVY_FLOUNDER:
- case CHIP_VANGOGH:
- case CHIP_DIMGREY_CAVEFISH:
- case CHIP_BEIGE_GOBY:
- case CHIP_YELLOW_CARP:
- case CHIP_CYAN_SKILLFISH:
- device_queue_manager_init_v10_navi10(&dqm->asic_ops);
- break;
default:
- WARN(1, "Unexpected ASIC family %u",
- dev->device_info->asic_family);
- goto out_free;
+ if (KFD_GC_VERSION(dev) >= IP_VERSION(10, 1, 1))
+ device_queue_manager_init_v10_navi10(&dqm->asic_ops);
+ else if (KFD_GC_VERSION(dev) >= IP_VERSION(9, 0, 1))
+ device_queue_manager_init_v9(&dqm->asic_ops);
+ else {
+ WARN(1, "Unexpected ASIC family %u",
+ dev->adev->asic_type);
+ goto out_free;
+ }
}
if (init_mqd_managers(dqm))
@@ -2000,7 +1992,7 @@ static void deallocate_hiq_sdma_mqd(struct kfd_dev *dev,
{
WARN(!mqd, "No hiq sdma mqd trunk to free");
- amdgpu_amdkfd_free_gtt_mem(dev->kgd, mqd->gtt_mem);
+ amdgpu_amdkfd_free_gtt_mem(dev->adev, mqd->gtt_mem);
}
void device_queue_manager_uninit(struct device_queue_manager *dqm)
@@ -2031,7 +2023,7 @@ static void kfd_process_hw_exception(struct work_struct *work)
{
struct device_queue_manager *dqm = container_of(work,
struct device_queue_manager, hw_exception_work);
- amdgpu_amdkfd_gpu_reset(dqm->dev->kgd);
+ amdgpu_amdkfd_gpu_reset(dqm->dev->adev);
}
#if defined(CONFIG_DEBUG_FS)
@@ -2070,7 +2062,7 @@ int dqm_debugfs_hqds(struct seq_file *m, void *data)
return 0;
}
- r = dqm->dev->kfd2kgd->hqd_dump(dqm->dev->kgd,
+ r = dqm->dev->kfd2kgd->hqd_dump(dqm->dev->adev,
KFD_CIK_HIQ_PIPE, KFD_CIK_HIQ_QUEUE,
&dump, &n_regs);
if (!r) {
@@ -2092,7 +2084,7 @@ int dqm_debugfs_hqds(struct seq_file *m, void *data)
continue;
r = dqm->dev->kfd2kgd->hqd_dump(
- dqm->dev->kgd, pipe, queue, &dump, &n_regs);
+ dqm->dev->adev, pipe, queue, &dump, &n_regs);
if (r)
break;
@@ -2106,10 +2098,10 @@ int dqm_debugfs_hqds(struct seq_file *m, void *data)
for (pipe = 0; pipe < get_num_all_sdma_engines(dqm); pipe++) {
for (queue = 0;
- queue < dqm->dev->device_info->num_sdma_queues_per_engine;
+ queue < dqm->dev->device_info.num_sdma_queues_per_engine;
queue++) {
r = dqm->dev->kfd2kgd->hqd_sdma_dump(
- dqm->dev->kgd, pipe, queue, &dump, &n_regs);
+ dqm->dev->adev, pipe, queue, &dump, &n_regs);
if (r)
break;
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.h b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.h
index 499fc0e..e145e4de 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.h
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.h
@@ -81,6 +81,8 @@ struct device_process_node {
*
* @get_wave_state: Retrieves context save state and optionally copies the
* control stack, if kept in the MQD, to the given userspace address.
+ *
+ * @reset_queues: reset queues which consume RAS poison
*/
struct device_queue_manager_ops {
@@ -134,6 +136,9 @@ struct device_queue_manager_ops {
void __user *ctl_stack,
u32 *ctl_stack_used_size,
u32 *save_area_used_size);
+
+ int (*reset_queues)(struct device_queue_manager *dqm,
+ uint16_t pasid);
};
struct device_queue_manager_asic_ops {
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager_v9.c b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager_v9.c
index b5c3d13..f20434d 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager_v9.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager_v9.c
@@ -62,7 +62,7 @@ static int update_qpd_v9(struct device_queue_manager *dqm,
SH_MEM_ALIGNMENT_MODE_UNALIGNED <<
SH_MEM_CONFIG__ALIGNMENT_MODE__SHIFT;
- if (dqm->dev->device_info->asic_family == CHIP_ALDEBARAN) {
+ if (KFD_GC_VERSION(dqm->dev) == IP_VERSION(9, 4, 2)) {
/* Aldebaran can safely support different XNACK modes
* per process
*/
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c b/drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c
index 768d153..0dbcf54 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c
@@ -48,7 +48,7 @@
/* # of doorbell bytes allocated for each process. */
size_t kfd_doorbell_process_slice(struct kfd_dev *kfd)
{
- return roundup(kfd->device_info->doorbell_size *
+ return roundup(kfd->device_info.doorbell_size *
KFD_MAX_NUM_OF_QUEUES_PER_PROCESS,
PAGE_SIZE);
}
@@ -180,7 +180,7 @@ void __iomem *kfd_get_kernel_doorbell(struct kfd_dev *kfd,
if (inx >= KFD_MAX_NUM_OF_QUEUES_PER_PROCESS)
return NULL;
- inx *= kfd->device_info->doorbell_size / sizeof(u32);
+ inx *= kfd->device_info.doorbell_size / sizeof(u32);
/*
* Calculating the kernel doorbell offset using the first
@@ -201,7 +201,7 @@ void kfd_release_kernel_doorbell(struct kfd_dev *kfd, u32 __iomem *db_addr)
unsigned int inx;
inx = (unsigned int)(db_addr - kfd->doorbell_kernel_ptr)
- * sizeof(u32) / kfd->device_info->doorbell_size;
+ * sizeof(u32) / kfd->device_info.doorbell_size;
mutex_lock(&kfd->doorbell_mutex);
__clear_bit(inx, kfd->doorbell_available_index);
@@ -239,7 +239,7 @@ unsigned int kfd_get_doorbell_dw_offset_in_bar(struct kfd_dev *kfd,
return kfd->doorbell_base_dw_offset +
pdd->doorbell_index
* kfd_doorbell_process_slice(kfd) / sizeof(u32) +
- doorbell_id * kfd->device_info->doorbell_size / sizeof(u32);
+ doorbell_id * kfd->device_info.doorbell_size / sizeof(u32);
}
uint64_t kfd_get_number_elems(struct kfd_dev *kfd)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_events.c b/drivers/gpu/drm/amd/amdkfd/kfd_events.c
index 3eea4ed..afe72dd 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_events.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_events.c
@@ -935,8 +935,10 @@ void kfd_signal_iommu_event(struct kfd_dev *dev, u32 pasid,
/* Workaround on Raven to not kill the process when memory is freed
* before IOMMU is able to finish processing all the excessive PPRs
*/
- if (dev->device_info->asic_family != CHIP_RAVEN &&
- dev->device_info->asic_family != CHIP_RENOIR) {
+
+ if (KFD_GC_VERSION(dev) != IP_VERSION(9, 1, 0) &&
+ KFD_GC_VERSION(dev) != IP_VERSION(9, 2, 2) &&
+ KFD_GC_VERSION(dev) != IP_VERSION(9, 3, 0)) {
mutex_lock(&p->event_mutex);
/* Lookup events by type and signal them */
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_flat_memory.c b/drivers/gpu/drm/amd/amdkfd/kfd_flat_memory.c
index d138889..2e2b7ce 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_flat_memory.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_flat_memory.c
@@ -394,7 +394,7 @@ int kfd_init_apertures(struct kfd_process *process)
pdd->gpuvm_base = pdd->gpuvm_limit = 0;
pdd->scratch_base = pdd->scratch_limit = 0;
} else {
- switch (dev->device_info->asic_family) {
+ switch (dev->adev->asic_type) {
case CHIP_KAVERI:
case CHIP_HAWAII:
case CHIP_CARRIZO:
@@ -406,29 +406,14 @@ int kfd_init_apertures(struct kfd_process *process)
case CHIP_VEGAM:
kfd_init_apertures_vi(pdd, id);
break;
- case CHIP_VEGA10:
- case CHIP_VEGA12:
- case CHIP_VEGA20:
- case CHIP_RAVEN:
- case CHIP_RENOIR:
- case CHIP_ARCTURUS:
- case CHIP_ALDEBARAN:
- case CHIP_NAVI10:
- case CHIP_NAVI12:
- case CHIP_NAVI14:
- case CHIP_SIENNA_CICHLID:
- case CHIP_NAVY_FLOUNDER:
- case CHIP_VANGOGH:
- case CHIP_DIMGREY_CAVEFISH:
- case CHIP_BEIGE_GOBY:
- case CHIP_YELLOW_CARP:
- case CHIP_CYAN_SKILLFISH:
- kfd_init_apertures_v9(pdd, id);
- break;
default:
- WARN(1, "Unexpected ASIC family %u",
- dev->device_info->asic_family);
- return -EINVAL;
+ if (KFD_GC_VERSION(dev) >= IP_VERSION(9, 0, 1))
+ kfd_init_apertures_v9(pdd, id);
+ else {
+ WARN(1, "Unexpected ASIC family %u",
+ dev->adev->asic_type);
+ return -EINVAL;
+ }
}
if (!dev->use_iommu_v2) {
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_int_process_v9.c b/drivers/gpu/drm/amd/amdkfd/kfd_int_process_v9.c
index 543e7ea..b8ac28f 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_int_process_v9.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_int_process_v9.c
@@ -89,6 +89,44 @@ enum SQ_INTERRUPT_ERROR_TYPE {
#define KFD_SQ_INT_DATA__ERR_TYPE_MASK 0xF00000
#define KFD_SQ_INT_DATA__ERR_TYPE__SHIFT 20
+static void event_interrupt_poison_consumption(struct kfd_dev *dev,
+ uint16_t pasid, uint16_t source_id)
+{
+ int ret = -EINVAL;
+ struct kfd_process *p = kfd_lookup_process_by_pasid(pasid);
+
+ if (!p)
+ return;
+
+ /* all queues of a process will be unmapped in one time */
+ if (atomic_read(&p->poison)) {
+ kfd_unref_process(p);
+ return;
+ }
+
+ atomic_set(&p->poison, 1);
+ kfd_unref_process(p);
+
+ switch (source_id) {
+ case SOC15_INTSRC_SQ_INTERRUPT_MSG:
+ if (dev->dqm->ops.reset_queues)
+ ret = dev->dqm->ops.reset_queues(dev->dqm, pasid);
+ break;
+ case SOC15_INTSRC_SDMA_ECC:
+ default:
+ break;
+ }
+
+ kfd_signal_poison_consumed_event(dev, pasid);
+
+ /* resetting queue passes, do page retirement without gpu reset
+ resetting queue fails, fallback to gpu reset solution */
+ if (!ret)
+ amdgpu_amdkfd_ras_poison_consumption_handler(dev->adev, false);
+ else
+ amdgpu_amdkfd_ras_poison_consumption_handler(dev->adev, true);
+}
+
static bool event_interrupt_isr_v9(struct kfd_dev *dev,
const uint32_t *ih_ring_entry,
uint32_t *patched_ihre,
@@ -135,7 +173,7 @@ static bool event_interrupt_isr_v9(struct kfd_dev *dev,
*patched_flag = true;
memcpy(patched_ihre, ih_ring_entry,
- dev->device_info->ih_ring_entry_size);
+ dev->device_info.ih_ring_entry_size);
pasid = dev->dqm->vmid_pasid[vmid];
@@ -230,8 +268,7 @@ static void event_interrupt_wq_v9(struct kfd_dev *dev,
sq_intr_err);
if (sq_intr_err != SQ_INTERRUPT_ERROR_TYPE_ILLEGAL_INST &&
sq_intr_err != SQ_INTERRUPT_ERROR_TYPE_MEMVIOL) {
- kfd_signal_poison_consumed_event(dev, pasid);
- amdgpu_amdkfd_ras_poison_consumption_handler(dev->kgd);
+ event_interrupt_poison_consumption(dev, pasid, source_id);
return;
}
break;
@@ -252,8 +289,7 @@ static void event_interrupt_wq_v9(struct kfd_dev *dev,
if (source_id == SOC15_INTSRC_SDMA_TRAP) {
kfd_signal_event_interrupt(pasid, context_id0 & 0xfffffff, 28);
} else if (source_id == SOC15_INTSRC_SDMA_ECC) {
- kfd_signal_poison_consumed_event(dev, pasid);
- amdgpu_amdkfd_ras_poison_consumption_handler(dev->kgd);
+ event_interrupt_poison_consumption(dev, pasid, source_id);
return;
}
} else if (client_id == SOC15_IH_CLIENTID_VMC ||
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_interrupt.c b/drivers/gpu/drm/amd/amdkfd/kfd_interrupt.c
index bc47f6a..81887c2 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_interrupt.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_interrupt.c
@@ -54,7 +54,7 @@ int kfd_interrupt_init(struct kfd_dev *kfd)
int r;
r = kfifo_alloc(&kfd->ih_fifo,
- KFD_IH_NUM_ENTRIES * kfd->device_info->ih_ring_entry_size,
+ KFD_IH_NUM_ENTRIES * kfd->device_info.ih_ring_entry_size,
GFP_KERNEL);
if (r) {
dev_err(kfd_chardev(), "Failed to allocate IH fifo\n");
@@ -114,8 +114,8 @@ bool enqueue_ih_ring_entry(struct kfd_dev *kfd, const void *ih_ring_entry)
int count;
count = kfifo_in(&kfd->ih_fifo, ih_ring_entry,
- kfd->device_info->ih_ring_entry_size);
- if (count != kfd->device_info->ih_ring_entry_size) {
+ kfd->device_info.ih_ring_entry_size);
+ if (count != kfd->device_info.ih_ring_entry_size) {
dev_err_ratelimited(kfd_chardev(),
"Interrupt ring overflow, dropping interrupt %d\n",
count);
@@ -133,11 +133,11 @@ static bool dequeue_ih_ring_entry(struct kfd_dev *kfd, void *ih_ring_entry)
int count;
count = kfifo_out(&kfd->ih_fifo, ih_ring_entry,
- kfd->device_info->ih_ring_entry_size);
+ kfd->device_info.ih_ring_entry_size);
- WARN_ON(count && count != kfd->device_info->ih_ring_entry_size);
+ WARN_ON(count && count != kfd->device_info.ih_ring_entry_size);
- return count == kfd->device_info->ih_ring_entry_size;
+ return count == kfd->device_info.ih_ring_entry_size;
}
static void interrupt_wq(struct work_struct *work)
@@ -146,13 +146,13 @@ static void interrupt_wq(struct work_struct *work)
interrupt_work);
uint32_t ih_ring_entry[KFD_MAX_RING_ENTRY_SIZE];
- if (dev->device_info->ih_ring_entry_size > sizeof(ih_ring_entry)) {
+ if (dev->device_info.ih_ring_entry_size > sizeof(ih_ring_entry)) {
dev_err_once(kfd_chardev(), "Ring entry too small\n");
return;
}
while (dequeue_ih_ring_entry(dev, ih_ring_entry))
- dev->device_info->event_interrupt_class->interrupt_wq(dev,
+ dev->device_info.event_interrupt_class->interrupt_wq(dev,
ih_ring_entry);
}
@@ -163,7 +163,7 @@ bool interrupt_is_wanted(struct kfd_dev *dev,
/* integer and bitwise OR so there is no boolean short-circuiting */
unsigned int wanted = 0;
- wanted |= dev->device_info->event_interrupt_class->interrupt_isr(dev,
+ wanted |= dev->device_info.event_interrupt_class->interrupt_isr(dev,
ih_ring_entry, patched_ihre, flag);
return wanted != 0;
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_iommu.c b/drivers/gpu/drm/amd/amdkfd/kfd_iommu.c
index 73f2257..66ad8d0 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_iommu.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_iommu.c
@@ -89,7 +89,7 @@ int kfd_iommu_device_init(struct kfd_dev *kfd)
}
pasid_limit = min_t(unsigned int,
- (unsigned int)(1 << kfd->device_info->max_pasid_bits),
+ (unsigned int)(1 << kfd->device_info.max_pasid_bits),
iommu_info.max_pasids);
if (!kfd_set_pasid_limit(pasid_limit)) {
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c b/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
index 64b4ac3..16f8bc4 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
@@ -91,7 +91,7 @@ static bool kq_initialize(struct kernel_queue *kq, struct kfd_dev *dev,
kq->pq_gpu_addr = kq->pq->gpu_addr;
/* For CIK family asics, kq->eop_mem is not needed */
- if (dev->device_info->asic_family > CHIP_MULLINS) {
+ if (dev->adev->asic_type > CHIP_MULLINS) {
retval = kfd_gtt_sa_allocate(dev, PAGE_SIZE, &kq->eop_mem);
if (retval != 0)
goto err_eop_allocate_vidmem;
@@ -111,7 +111,7 @@ static bool kq_initialize(struct kernel_queue *kq, struct kfd_dev *dev,
kq->rptr_kernel = kq->rptr_mem->cpu_ptr;
kq->rptr_gpu_addr = kq->rptr_mem->gpu_addr;
- retval = kfd_gtt_sa_allocate(dev, dev->device_info->doorbell_size,
+ retval = kfd_gtt_sa_allocate(dev, dev->device_info.doorbell_size,
&kq->wptr_mem);
if (retval != 0)
@@ -297,7 +297,7 @@ void kq_submit_packet(struct kernel_queue *kq)
}
pr_debug("\n");
#endif
- if (kq->dev->device_info->doorbell_size == 8) {
+ if (kq->dev->device_info.doorbell_size == 8) {
*kq->wptr64_kernel = kq->pending_wptr64;
write_kernel_doorbell64(kq->queue->properties.doorbell_ptr,
kq->pending_wptr64);
@@ -310,7 +310,7 @@ void kq_submit_packet(struct kernel_queue *kq)
void kq_rollback_packet(struct kernel_queue *kq)
{
- if (kq->dev->device_info->doorbell_size == 8) {
+ if (kq->dev->device_info.doorbell_size == 8) {
kq->pending_wptr64 = *kq->wptr64_kernel;
kq->pending_wptr = *kq->wptr_kernel %
(kq->queue->properties.queue_size / 4);
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c b/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c
index 9b9c2b9..ed53851 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c
@@ -108,8 +108,8 @@ svm_migrate_gart_map(struct amdgpu_ring *ring, uint64_t npages,
* svm_migrate_copy_memory_gart - sdma copy data between ram and vram
*
* @adev: amdgpu device the sdma ring running
- * @src: source page address array
- * @dst: destination page address array
+ * @sys: system DMA pointer to be copied
+ * @vram: vram destination DMA pointer
* @npages: number of pages to copy
* @direction: enum MIGRATION_COPY_DIR
* @mfence: output, sdma fence to signal after sdma is done
@@ -549,7 +549,7 @@ static void svm_migrate_page_free(struct page *page)
if (svm_bo) {
pr_debug_ratelimited("ref: %d\n", kref_read(&svm_bo->kref));
- svm_range_bo_unref(svm_bo);
+ svm_range_bo_unref_async(svm_bo);
}
}
@@ -938,7 +938,7 @@ int svm_migrate_init(struct amdgpu_device *adev)
void *r;
/* Page migration works on Vega10 or newer */
- if (kfddev->device_info->asic_family < CHIP_VEGA10)
+ if (!KFD_IS_SOC15(kfddev))
return -EINVAL;
pgmap = &kfddev->pgmap;
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.c
index c021519..e2825ad 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.c
@@ -71,7 +71,7 @@ struct kfd_mem_obj *allocate_sdma_mqd(struct kfd_dev *dev,
return NULL;
offset = (q->sdma_engine_id *
- dev->device_info->num_sdma_queues_per_engine +
+ dev->device_info.num_sdma_queues_per_engine +
q->sdma_queue_id) *
dev->dqm->mqd_mgrs[KFD_MQD_TYPE_SDMA]->mqd_size;
@@ -100,7 +100,7 @@ void mqd_symmetrically_map_cu_mask(struct mqd_manager *mm,
struct kfd_cu_info cu_info;
uint32_t cu_per_sh[KFD_MAX_NUM_SE][KFD_MAX_NUM_SH_PER_SE] = {0};
int i, se, sh, cu;
- amdgpu_amdkfd_get_cu_info(mm->dev->kgd, &cu_info);
+ amdgpu_amdkfd_get_cu_info(mm->dev->adev, &cu_info);
if (cu_mask_count > cu_info.cu_active_number)
cu_mask_count = cu_info.cu_active_number;
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_cik.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_cik.c
index 8128f4d..e9a8e21 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_cik.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_cik.c
@@ -171,7 +171,7 @@ static int load_mqd(struct mqd_manager *mm, void *mqd, uint32_t pipe_id,
uint32_t wptr_shift = (p->format == KFD_QUEUE_FORMAT_AQL ? 4 : 0);
uint32_t wptr_mask = (uint32_t)((p->queue_size / 4) - 1);
- return mm->dev->kfd2kgd->hqd_load(mm->dev->kgd, mqd, pipe_id, queue_id,
+ return mm->dev->kfd2kgd->hqd_load(mm->dev->adev, mqd, pipe_id, queue_id,
(uint32_t __user *)p->write_ptr,
wptr_shift, wptr_mask, mms);
}
@@ -180,7 +180,7 @@ static int load_mqd_sdma(struct mqd_manager *mm, void *mqd,
uint32_t pipe_id, uint32_t queue_id,
struct queue_properties *p, struct mm_struct *mms)
{
- return mm->dev->kfd2kgd->hqd_sdma_load(mm->dev->kgd, mqd,
+ return mm->dev->kfd2kgd->hqd_sdma_load(mm->dev->adev, mqd,
(uint32_t __user *)p->write_ptr,
mms);
}
@@ -276,7 +276,7 @@ static int destroy_mqd(struct mqd_manager *mm, void *mqd,
unsigned int timeout, uint32_t pipe_id,
uint32_t queue_id)
{
- return mm->dev->kfd2kgd->hqd_destroy(mm->dev->kgd, mqd, type, timeout,
+ return mm->dev->kfd2kgd->hqd_destroy(mm->dev->adev, mqd, type, timeout,
pipe_id, queue_id);
}
@@ -289,7 +289,7 @@ static int destroy_mqd_sdma(struct mqd_manager *mm, void *mqd,
unsigned int timeout, uint32_t pipe_id,
uint32_t queue_id)
{
- return mm->dev->kfd2kgd->hqd_sdma_destroy(mm->dev->kgd, mqd, timeout);
+ return mm->dev->kfd2kgd->hqd_sdma_destroy(mm->dev->adev, mqd, timeout);
}
static bool is_occupied(struct mqd_manager *mm, void *mqd,
@@ -297,7 +297,7 @@ static bool is_occupied(struct mqd_manager *mm, void *mqd,
uint32_t queue_id)
{
- return mm->dev->kfd2kgd->hqd_is_occupied(mm->dev->kgd, queue_address,
+ return mm->dev->kfd2kgd->hqd_is_occupied(mm->dev->adev, queue_address,
pipe_id, queue_id);
}
@@ -306,7 +306,7 @@ static bool is_occupied_sdma(struct mqd_manager *mm, void *mqd,
uint64_t queue_address, uint32_t pipe_id,
uint32_t queue_id)
{
- return mm->dev->kfd2kgd->hqd_sdma_is_occupied(mm->dev->kgd, mqd);
+ return mm->dev->kfd2kgd->hqd_sdma_is_occupied(mm->dev->adev, mqd);
}
/*
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v10.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v10.c
index 270160f..d74d8a6 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v10.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v10.c
@@ -148,7 +148,7 @@ static int load_mqd(struct mqd_manager *mm, void *mqd,
/* AQL write pointer counts in 64B packets, PM4/CP counts in dwords. */
uint32_t wptr_shift = (p->format == KFD_QUEUE_FORMAT_AQL ? 4 : 0);
- r = mm->dev->kfd2kgd->hqd_load(mm->dev->kgd, mqd, pipe_id, queue_id,
+ r = mm->dev->kfd2kgd->hqd_load(mm->dev->adev, mqd, pipe_id, queue_id,
(uint32_t __user *)p->write_ptr,
wptr_shift, 0, mms);
return r;
@@ -158,7 +158,7 @@ static int hiq_load_mqd_kiq(struct mqd_manager *mm, void *mqd,
uint32_t pipe_id, uint32_t queue_id,
struct queue_properties *p, struct mm_struct *mms)
{
- return mm->dev->kfd2kgd->hiq_mqd_load(mm->dev->kgd, mqd, pipe_id,
+ return mm->dev->kfd2kgd->hiq_mqd_load(mm->dev->adev, mqd, pipe_id,
queue_id, p->doorbell_off);
}
@@ -239,7 +239,7 @@ static int destroy_mqd(struct mqd_manager *mm, void *mqd,
uint32_t queue_id)
{
return mm->dev->kfd2kgd->hqd_destroy
- (mm->dev->kgd, mqd, type, timeout,
+ (mm->dev->adev, mqd, type, timeout,
pipe_id, queue_id);
}
@@ -254,7 +254,7 @@ static bool is_occupied(struct mqd_manager *mm, void *mqd,
uint32_t queue_id)
{
return mm->dev->kfd2kgd->hqd_is_occupied(
- mm->dev->kgd, queue_address,
+ mm->dev->adev, queue_address,
pipe_id, queue_id);
}
@@ -320,7 +320,7 @@ static int load_mqd_sdma(struct mqd_manager *mm, void *mqd,
uint32_t pipe_id, uint32_t queue_id,
struct queue_properties *p, struct mm_struct *mms)
{
- return mm->dev->kfd2kgd->hqd_sdma_load(mm->dev->kgd, mqd,
+ return mm->dev->kfd2kgd->hqd_sdma_load(mm->dev->adev, mqd,
(uint32_t __user *)p->write_ptr,
mms);
}
@@ -363,14 +363,14 @@ static int destroy_mqd_sdma(struct mqd_manager *mm, void *mqd,
unsigned int timeout, uint32_t pipe_id,
uint32_t queue_id)
{
- return mm->dev->kfd2kgd->hqd_sdma_destroy(mm->dev->kgd, mqd, timeout);
+ return mm->dev->kfd2kgd->hqd_sdma_destroy(mm->dev->adev, mqd, timeout);
}
static bool is_occupied_sdma(struct mqd_manager *mm, void *mqd,
uint64_t queue_address, uint32_t pipe_id,
uint32_t queue_id)
{
- return mm->dev->kfd2kgd->hqd_sdma_is_occupied(mm->dev->kgd, mqd);
+ return mm->dev->kfd2kgd->hqd_sdma_is_occupied(mm->dev->adev, mqd);
}
#if defined(CONFIG_DEBUG_FS)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c
index 4e5932f..326eb22 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c
@@ -108,7 +108,7 @@ static struct kfd_mem_obj *allocate_mqd(struct kfd_dev *kfd,
mqd_mem_obj = kzalloc(sizeof(struct kfd_mem_obj), GFP_KERNEL);
if (!mqd_mem_obj)
return NULL;
- retval = amdgpu_amdkfd_alloc_gtt_mem(kfd->kgd,
+ retval = amdgpu_amdkfd_alloc_gtt_mem(kfd->adev,
ALIGN(q->ctl_stack_size, PAGE_SIZE) +
ALIGN(sizeof(struct v9_mqd), PAGE_SIZE),
&(mqd_mem_obj->gtt_mem),
@@ -199,7 +199,7 @@ static int load_mqd(struct mqd_manager *mm, void *mqd,
/* AQL write pointer counts in 64B packets, PM4/CP counts in dwords. */
uint32_t wptr_shift = (p->format == KFD_QUEUE_FORMAT_AQL ? 4 : 0);
- return mm->dev->kfd2kgd->hqd_load(mm->dev->kgd, mqd, pipe_id, queue_id,
+ return mm->dev->kfd2kgd->hqd_load(mm->dev->adev, mqd, pipe_id, queue_id,
(uint32_t __user *)p->write_ptr,
wptr_shift, 0, mms);
}
@@ -208,7 +208,7 @@ static int hiq_load_mqd_kiq(struct mqd_manager *mm, void *mqd,
uint32_t pipe_id, uint32_t queue_id,
struct queue_properties *p, struct mm_struct *mms)
{
- return mm->dev->kfd2kgd->hiq_mqd_load(mm->dev->kgd, mqd, pipe_id,
+ return mm->dev->kfd2kgd->hiq_mqd_load(mm->dev->adev, mqd, pipe_id,
queue_id, p->doorbell_off);
}
@@ -291,7 +291,7 @@ static int destroy_mqd(struct mqd_manager *mm, void *mqd,
uint32_t queue_id)
{
return mm->dev->kfd2kgd->hqd_destroy
- (mm->dev->kgd, mqd, type, timeout,
+ (mm->dev->adev, mqd, type, timeout,
pipe_id, queue_id);
}
@@ -301,7 +301,7 @@ static void free_mqd(struct mqd_manager *mm, void *mqd,
struct kfd_dev *kfd = mm->dev;
if (mqd_mem_obj->gtt_mem) {
- amdgpu_amdkfd_free_gtt_mem(kfd->kgd, mqd_mem_obj->gtt_mem);
+ amdgpu_amdkfd_free_gtt_mem(kfd->adev, mqd_mem_obj->gtt_mem);
kfree(mqd_mem_obj);
} else {
kfd_gtt_sa_free(mm->dev, mqd_mem_obj);
@@ -313,7 +313,7 @@ static bool is_occupied(struct mqd_manager *mm, void *mqd,
uint32_t queue_id)
{
return mm->dev->kfd2kgd->hqd_is_occupied(
- mm->dev->kgd, queue_address,
+ mm->dev->adev, queue_address,
pipe_id, queue_id);
}
@@ -375,7 +375,7 @@ static int load_mqd_sdma(struct mqd_manager *mm, void *mqd,
uint32_t pipe_id, uint32_t queue_id,
struct queue_properties *p, struct mm_struct *mms)
{
- return mm->dev->kfd2kgd->hqd_sdma_load(mm->dev->kgd, mqd,
+ return mm->dev->kfd2kgd->hqd_sdma_load(mm->dev->adev, mqd,
(uint32_t __user *)p->write_ptr,
mms);
}
@@ -418,14 +418,14 @@ static int destroy_mqd_sdma(struct mqd_manager *mm, void *mqd,
unsigned int timeout, uint32_t pipe_id,
uint32_t queue_id)
{
- return mm->dev->kfd2kgd->hqd_sdma_destroy(mm->dev->kgd, mqd, timeout);
+ return mm->dev->kfd2kgd->hqd_sdma_destroy(mm->dev->adev, mqd, timeout);
}
static bool is_occupied_sdma(struct mqd_manager *mm, void *mqd,
uint64_t queue_address, uint32_t pipe_id,
uint32_t queue_id)
{
- return mm->dev->kfd2kgd->hqd_sdma_is_occupied(mm->dev->kgd, mqd);
+ return mm->dev->kfd2kgd->hqd_sdma_is_occupied(mm->dev->adev, mqd);
}
#if defined(CONFIG_DEBUG_FS)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c
index cd9220e..d456e95 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c
@@ -162,7 +162,7 @@ static int load_mqd(struct mqd_manager *mm, void *mqd,
uint32_t wptr_shift = (p->format == KFD_QUEUE_FORMAT_AQL ? 4 : 0);
uint32_t wptr_mask = (uint32_t)((p->queue_size / 4) - 1);
- return mm->dev->kfd2kgd->hqd_load(mm->dev->kgd, mqd, pipe_id, queue_id,
+ return mm->dev->kfd2kgd->hqd_load(mm->dev->adev, mqd, pipe_id, queue_id,
(uint32_t __user *)p->write_ptr,
wptr_shift, wptr_mask, mms);
}
@@ -265,7 +265,7 @@ static int destroy_mqd(struct mqd_manager *mm, void *mqd,
uint32_t queue_id)
{
return mm->dev->kfd2kgd->hqd_destroy
- (mm->dev->kgd, mqd, type, timeout,
+ (mm->dev->adev, mqd, type, timeout,
pipe_id, queue_id);
}
@@ -280,7 +280,7 @@ static bool is_occupied(struct mqd_manager *mm, void *mqd,
uint32_t queue_id)
{
return mm->dev->kfd2kgd->hqd_is_occupied(
- mm->dev->kgd, queue_address,
+ mm->dev->adev, queue_address,
pipe_id, queue_id);
}
@@ -347,7 +347,7 @@ static int load_mqd_sdma(struct mqd_manager *mm, void *mqd,
uint32_t pipe_id, uint32_t queue_id,
struct queue_properties *p, struct mm_struct *mms)
{
- return mm->dev->kfd2kgd->hqd_sdma_load(mm->dev->kgd, mqd,
+ return mm->dev->kfd2kgd->hqd_sdma_load(mm->dev->adev, mqd,
(uint32_t __user *)p->write_ptr,
mms);
}
@@ -389,14 +389,14 @@ static int destroy_mqd_sdma(struct mqd_manager *mm, void *mqd,
unsigned int timeout, uint32_t pipe_id,
uint32_t queue_id)
{
- return mm->dev->kfd2kgd->hqd_sdma_destroy(mm->dev->kgd, mqd, timeout);
+ return mm->dev->kfd2kgd->hqd_sdma_destroy(mm->dev->adev, mqd, timeout);
}
static bool is_occupied_sdma(struct mqd_manager *mm, void *mqd,
uint64_t queue_address, uint32_t pipe_id,
uint32_t queue_id)
{
- return mm->dev->kfd2kgd->hqd_sdma_is_occupied(mm->dev->kgd, mqd);
+ return mm->dev->kfd2kgd->hqd_sdma_is_occupied(mm->dev->adev, mqd);
}
#if defined(CONFIG_DEBUG_FS)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager.c
index e547f1f..1439420 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager.c
@@ -223,7 +223,7 @@ static int pm_create_runlist_ib(struct packet_manager *pm,
int pm_init(struct packet_manager *pm, struct device_queue_manager *dqm)
{
- switch (dqm->dev->device_info->asic_family) {
+ switch (dqm->dev->adev->asic_type) {
case CHIP_KAVERI:
case CHIP_HAWAII:
/* PM4 packet structures on CIK are the same as on VI */
@@ -236,31 +236,16 @@ int pm_init(struct packet_manager *pm, struct device_queue_manager *dqm)
case CHIP_VEGAM:
pm->pmf = &kfd_vi_pm_funcs;
break;
- case CHIP_VEGA10:
- case CHIP_VEGA12:
- case CHIP_VEGA20:
- case CHIP_RAVEN:
- case CHIP_RENOIR:
- case CHIP_ARCTURUS:
- case CHIP_NAVI10:
- case CHIP_NAVI12:
- case CHIP_NAVI14:
- case CHIP_SIENNA_CICHLID:
- case CHIP_NAVY_FLOUNDER:
- case CHIP_VANGOGH:
- case CHIP_DIMGREY_CAVEFISH:
- case CHIP_BEIGE_GOBY:
- case CHIP_YELLOW_CARP:
- case CHIP_CYAN_SKILLFISH:
- pm->pmf = &kfd_v9_pm_funcs;
- break;
- case CHIP_ALDEBARAN:
- pm->pmf = &kfd_aldebaran_pm_funcs;
- break;
default:
- WARN(1, "Unexpected ASIC family %u",
- dqm->dev->device_info->asic_family);
- return -EINVAL;
+ if (KFD_GC_VERSION(dqm->dev) == IP_VERSION(9, 4, 2))
+ pm->pmf = &kfd_aldebaran_pm_funcs;
+ else if (KFD_GC_VERSION(dqm->dev) >= IP_VERSION(9, 0, 1))
+ pm->pmf = &kfd_v9_pm_funcs;
+ else {
+ WARN(1, "Unexpected ASIC family %u",
+ dqm->dev->adev->asic_type);
+ return -EINVAL;
+ }
}
pm->dqm = dqm;
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_vi.c b/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_vi.c
index 08442e7..3c0658e 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_vi.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_vi.c
@@ -110,8 +110,8 @@ static int pm_runlist_vi(struct packet_manager *pm, uint32_t *buffer,
return 0;
}
-int pm_set_resources_vi(struct packet_manager *pm, uint32_t *buffer,
- struct scheduling_resources *res)
+static int pm_set_resources_vi(struct packet_manager *pm, uint32_t *buffer,
+ struct scheduling_resources *res)
{
struct pm4_mes_set_resources *packet;
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
index 8fd48d0..ea68f3b 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
@@ -183,7 +183,8 @@ enum cache_policy {
cache_policy_noncoherent
};
-#define KFD_IS_SOC15(chip) ((chip) >= CHIP_VEGA10)
+#define KFD_GC_VERSION(dev) ((dev)->adev->ip_versions[GC_HWIP][0])
+#define KFD_IS_SOC15(dev) ((KFD_GC_VERSION(dev)) >= (IP_VERSION(9, 0, 1)))
struct kfd_event_interrupt_class {
bool (*interrupt_isr)(struct kfd_dev *dev,
@@ -194,8 +195,6 @@ struct kfd_event_interrupt_class {
};
struct kfd_device_info {
- enum amd_asic_type asic_family;
- const char *asic_name;
uint32_t gfx_target_version;
const struct kfd_event_interrupt_class *event_interrupt_class;
unsigned int max_pasid_bits;
@@ -208,11 +207,12 @@ struct kfd_device_info {
bool needs_iommu_device;
bool needs_pci_atomics;
uint32_t no_atomic_fw_version;
- unsigned int num_sdma_engines;
- unsigned int num_xgmi_sdma_engines;
unsigned int num_sdma_queues_per_engine;
};
+unsigned int kfd_get_num_sdma_engines(struct kfd_dev *kdev);
+unsigned int kfd_get_num_xgmi_sdma_engines(struct kfd_dev *kdev);
+
struct kfd_mem_obj {
uint32_t range_start;
uint32_t range_end;
@@ -228,9 +228,9 @@ struct kfd_vmid_info {
};
struct kfd_dev {
- struct kgd_dev *kgd;
+ struct amdgpu_device *adev;
- const struct kfd_device_info *device_info;
+ struct kfd_device_info device_info;
struct pci_dev *pdev;
struct drm_device *ddev;
@@ -856,6 +856,8 @@ struct kfd_process {
struct svm_range_list svms;
bool xnack_enabled;
+
+ atomic_t poison;
};
#define KFD_PROCESS_TABLE_SIZE 5 /* bits: 32 entries */
@@ -891,7 +893,7 @@ struct kfd_process *kfd_lookup_process_by_pasid(u32 pasid);
struct kfd_process *kfd_lookup_process_by_mm(const struct mm_struct *mm);
int kfd_process_gpuidx_from_gpuid(struct kfd_process *p, uint32_t gpu_id);
-int kfd_process_gpuid_from_kgd(struct kfd_process *p,
+int kfd_process_gpuid_from_adev(struct kfd_process *p,
struct amdgpu_device *adev, uint32_t *gpuid,
uint32_t *gpuidx);
static inline int kfd_process_gpuid_from_gpuidx(struct kfd_process *p,
@@ -984,7 +986,7 @@ struct kfd_topology_device *kfd_topology_device_by_proximity_domain(
struct kfd_topology_device *kfd_topology_device_by_id(uint32_t gpu_id);
struct kfd_dev *kfd_device_by_id(uint32_t gpu_id);
struct kfd_dev *kfd_device_by_pci_dev(const struct pci_dev *pdev);
-struct kfd_dev *kfd_device_by_kgd(const struct kgd_dev *kgd);
+struct kfd_dev *kfd_device_by_adev(const struct amdgpu_device *adev);
int kfd_topology_enum_kfd_devices(uint8_t idx, struct kfd_dev **kdev);
int kfd_numa_node_to_apic_id(int numa_node_id);
void kfd_double_confirm_iommu_support(struct kfd_dev *gpu);
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process.c b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
index b993011..f1930ff 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_process.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
@@ -251,14 +251,13 @@ static void kfd_sdma_activity_worker(struct work_struct *work)
}
/**
- * @kfd_get_cu_occupancy - Collect number of waves in-flight on this device
+ * kfd_get_cu_occupancy - Collect number of waves in-flight on this device
* by current process. Translates acquired wave count into number of compute units
* that are occupied.
*
- * @atr: Handle of attribute that allows reporting of wave count. The attribute
+ * @attr: Handle of attribute that allows reporting of wave count. The attribute
* handle encapsulates GPU device it is associated with, thereby allowing collection
* of waves in flight, etc
- *
* @buffer: Handle of user provided buffer updated with wave count
*
* Return: Number of bytes written to user buffer or an error value
@@ -288,7 +287,7 @@ static int kfd_get_cu_occupancy(struct attribute *attr, char *buffer)
/* Collect wave count from device if it supports */
wave_cnt = 0;
max_waves_per_cu = 0;
- dev->kfd2kgd->get_cu_occupancy(dev->kgd, proc->pasid, &wave_cnt,
+ dev->kfd2kgd->get_cu_occupancy(dev->adev, proc->pasid, &wave_cnt,
&max_waves_per_cu);
/* Translate wave count to number of compute units */
@@ -692,12 +691,12 @@ static void kfd_process_free_gpuvm(struct kgd_mem *mem,
struct kfd_dev *dev = pdd->dev;
if (kptr) {
- amdgpu_amdkfd_gpuvm_unmap_gtt_bo_from_kernel(dev->kgd, mem);
+ amdgpu_amdkfd_gpuvm_unmap_gtt_bo_from_kernel(dev->adev, mem);
kptr = NULL;
}
- amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu(dev->kgd, mem, pdd->drm_priv);
- amdgpu_amdkfd_gpuvm_free_memory_of_gpu(dev->kgd, mem, pdd->drm_priv,
+ amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu(dev->adev, mem, pdd->drm_priv);
+ amdgpu_amdkfd_gpuvm_free_memory_of_gpu(dev->adev, mem, pdd->drm_priv,
NULL);
}
@@ -714,24 +713,24 @@ static int kfd_process_alloc_gpuvm(struct kfd_process_device *pdd,
struct kfd_dev *kdev = pdd->dev;
int err;
- err = amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu(kdev->kgd, gpu_va, size,
+ err = amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu(kdev->adev, gpu_va, size,
pdd->drm_priv, mem, NULL, flags);
if (err)
goto err_alloc_mem;
- err = amdgpu_amdkfd_gpuvm_map_memory_to_gpu(kdev->kgd, *mem,
+ err = amdgpu_amdkfd_gpuvm_map_memory_to_gpu(kdev->adev, *mem,
pdd->drm_priv, NULL);
if (err)
goto err_map_mem;
- err = amdgpu_amdkfd_gpuvm_sync_memory(kdev->kgd, *mem, true);
+ err = amdgpu_amdkfd_gpuvm_sync_memory(kdev->adev, *mem, true);
if (err) {
pr_debug("Sync memory failed, wait interrupted by user signal\n");
goto sync_memory_failed;
}
if (kptr) {
- err = amdgpu_amdkfd_gpuvm_map_gtt_bo_to_kernel(kdev->kgd,
+ err = amdgpu_amdkfd_gpuvm_map_gtt_bo_to_kernel(kdev->adev,
(struct kgd_mem *)*mem, kptr, NULL);
if (err) {
pr_debug("Map GTT BO to kernel failed\n");
@@ -742,10 +741,10 @@ static int kfd_process_alloc_gpuvm(struct kfd_process_device *pdd,
return err;
sync_memory_failed:
- amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu(kdev->kgd, *mem, pdd->drm_priv);
+ amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu(kdev->adev, *mem, pdd->drm_priv);
err_map_mem:
- amdgpu_amdkfd_gpuvm_free_memory_of_gpu(kdev->kgd, *mem, pdd->drm_priv,
+ amdgpu_amdkfd_gpuvm_free_memory_of_gpu(kdev->adev, *mem, pdd->drm_priv,
NULL);
err_alloc_mem:
*mem = NULL;
@@ -940,10 +939,10 @@ static void kfd_process_device_free_bos(struct kfd_process_device *pdd)
if (!peer_pdd->drm_priv)
continue;
amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu(
- peer_pdd->dev->kgd, mem, peer_pdd->drm_priv);
+ peer_pdd->dev->adev, mem, peer_pdd->drm_priv);
}
- amdgpu_amdkfd_gpuvm_free_memory_of_gpu(pdd->dev->kgd, mem,
+ amdgpu_amdkfd_gpuvm_free_memory_of_gpu(pdd->dev->adev, mem,
pdd->drm_priv, NULL);
kfd_process_device_remove_obj_handle(pdd, id);
}
@@ -974,7 +973,7 @@ static void kfd_process_kunmap_signal_bo(struct kfd_process *p)
if (!mem)
goto out;
- amdgpu_amdkfd_gpuvm_unmap_gtt_bo_from_kernel(kdev->kgd, mem);
+ amdgpu_amdkfd_gpuvm_unmap_gtt_bo_from_kernel(kdev->adev, mem);
out:
mutex_unlock(&p->mutex);
@@ -1003,7 +1002,7 @@ static void kfd_process_destroy_pdds(struct kfd_process *p)
if (pdd->drm_file) {
amdgpu_amdkfd_gpuvm_release_process_vm(
- pdd->dev->kgd, pdd->drm_priv);
+ pdd->dev->adev, pdd->drm_priv);
fput(pdd->drm_file);
}
@@ -1011,7 +1010,7 @@ static void kfd_process_destroy_pdds(struct kfd_process *p)
free_pages((unsigned long)pdd->qpd.cwsr_kaddr,
get_order(KFD_CWSR_TBA_TMA_SIZE));
- kfree(pdd->qpd.doorbell_bitmap);
+ bitmap_free(pdd->qpd.doorbell_bitmap);
idr_destroy(&pdd->alloc_idr);
kfd_free_process_doorbells(pdd->dev, pdd->doorbell_index);
@@ -1317,14 +1316,13 @@ bool kfd_process_xnack_mode(struct kfd_process *p, bool supported)
* support the SVM APIs and don't need to be considered
* for the XNACK mode selection.
*/
- if (dev->device_info->asic_family < CHIP_VEGA10)
+ if (!KFD_IS_SOC15(dev))
continue;
/* Aldebaran can always support XNACK because it can support
* per-process XNACK mode selection. But let the dev->noretry
* setting still influence the default XNACK mode.
*/
- if (supported &&
- dev->device_info->asic_family == CHIP_ALDEBARAN)
+ if (supported && KFD_GC_VERSION(dev) == IP_VERSION(9, 4, 2))
continue;
/* GFXv10 and later GPUs do not support shader preemption
@@ -1332,7 +1330,7 @@ bool kfd_process_xnack_mode(struct kfd_process *p, bool supported)
* management and memory-manager-related preemptions or
* even deadlocks.
*/
- if (dev->device_info->asic_family >= CHIP_NAVI10)
+ if (KFD_GC_VERSION(dev) >= IP_VERSION(10, 1, 1))
return false;
if (dev->noretry)
@@ -1431,12 +1429,11 @@ static int init_doorbell_bitmap(struct qcm_process_device *qpd,
int range_start = dev->shared_resources.non_cp_doorbells_start;
int range_end = dev->shared_resources.non_cp_doorbells_end;
- if (!KFD_IS_SOC15(dev->device_info->asic_family))
+ if (!KFD_IS_SOC15(dev))
return 0;
- qpd->doorbell_bitmap =
- kzalloc(DIV_ROUND_UP(KFD_MAX_NUM_OF_QUEUES_PER_PROCESS,
- BITS_PER_BYTE), GFP_KERNEL);
+ qpd->doorbell_bitmap = bitmap_zalloc(KFD_MAX_NUM_OF_QUEUES_PER_PROCESS,
+ GFP_KERNEL);
if (!qpd->doorbell_bitmap)
return -ENOMEM;
@@ -1448,9 +1445,9 @@ static int init_doorbell_bitmap(struct qcm_process_device *qpd,
for (i = 0; i < KFD_MAX_NUM_OF_QUEUES_PER_PROCESS / 2; i++) {
if (i >= range_start && i <= range_end) {
- set_bit(i, qpd->doorbell_bitmap);
- set_bit(i + KFD_QUEUE_DOORBELL_MIRROR_OFFSET,
- qpd->doorbell_bitmap);
+ __set_bit(i, qpd->doorbell_bitmap);
+ __set_bit(i + KFD_QUEUE_DOORBELL_MIRROR_OFFSET,
+ qpd->doorbell_bitmap);
}
}
@@ -1547,7 +1544,7 @@ int kfd_process_device_init_vm(struct kfd_process_device *pdd,
dev = pdd->dev;
ret = amdgpu_amdkfd_gpuvm_acquire_process_vm(
- dev->kgd, drm_file, p->pasid,
+ dev->adev, drm_file, p->pasid,
&p->kgd_process_info, &p->ef);
if (ret) {
pr_err("Failed to create process VM object\n");
@@ -1779,14 +1776,13 @@ int kfd_process_gpuidx_from_gpuid(struct kfd_process *p, uint32_t gpu_id)
}
int
-kfd_process_gpuid_from_kgd(struct kfd_process *p, struct amdgpu_device *adev,
+kfd_process_gpuid_from_adev(struct kfd_process *p, struct amdgpu_device *adev,
uint32_t *gpuid, uint32_t *gpuidx)
{
- struct kgd_dev *kgd = (struct kgd_dev *)adev;
int i;
for (i = 0; i < p->n_pdds; i++)
- if (p->pdds[i] && p->pdds[i]->dev->kgd == kgd) {
+ if (p->pdds[i] && p->pdds[i]->dev->adev == adev) {
*gpuid = p->pdds[i]->dev->id;
*gpuidx = i;
return 0;
@@ -1951,10 +1947,10 @@ void kfd_flush_tlb(struct kfd_process_device *pdd, enum TLB_FLUSH_TYPE type)
* only happens when the first queue is created.
*/
if (pdd->qpd.vmid)
- amdgpu_amdkfd_flush_gpu_tlb_vmid(dev->kgd,
+ amdgpu_amdkfd_flush_gpu_tlb_vmid(dev->adev,
pdd->qpd.vmid);
} else {
- amdgpu_amdkfd_flush_gpu_tlb_pasid(dev->kgd,
+ amdgpu_amdkfd_flush_gpu_tlb_pasid(dev->adev,
pdd->process->pasid, type);
}
}
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c
index 3627e7a..5e5c84a 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c
@@ -118,7 +118,7 @@ int pqm_set_gws(struct process_queue_manager *pqm, unsigned int qid,
return ret;
pqn->q->gws = mem;
- pdd->qpd.num_gws = gws ? amdgpu_amdkfd_get_num_gws(dev->kgd) : 0;
+ pdd->qpd.num_gws = gws ? dev->adev->gds.gws_size : 0;
return pqn->q->device->dqm->ops.update_queue(pqn->q->device->dqm,
pqn->q, NULL);
@@ -135,9 +135,8 @@ void kfd_process_dequeue_from_all_devices(struct kfd_process *p)
int pqm_init(struct process_queue_manager *pqm, struct kfd_process *p)
{
INIT_LIST_HEAD(&pqm->queues);
- pqm->queue_slot_bitmap =
- kzalloc(DIV_ROUND_UP(KFD_MAX_NUM_OF_QUEUES_PER_PROCESS,
- BITS_PER_BYTE), GFP_KERNEL);
+ pqm->queue_slot_bitmap = bitmap_zalloc(KFD_MAX_NUM_OF_QUEUES_PER_PROCESS,
+ GFP_KERNEL);
if (!pqm->queue_slot_bitmap)
return -ENOMEM;
pqm->process = p;
@@ -159,7 +158,7 @@ void pqm_uninit(struct process_queue_manager *pqm)
kfree(pqn);
}
- kfree(pqm->queue_slot_bitmap);
+ bitmap_free(pqm->queue_slot_bitmap);
pqm->queue_slot_bitmap = NULL;
}
@@ -220,7 +219,7 @@ int pqm_create_queue(struct process_queue_manager *pqm,
* Hence we also check the type as well
*/
if ((pdd->qpd.is_debug) || (type == KFD_QUEUE_TYPE_DIQ))
- max_queues = dev->device_info->max_no_of_hqd/2;
+ max_queues = dev->device_info.max_no_of_hqd/2;
if (pdd->qpd.queue_count >= max_queues)
return -ENOSPC;
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_smi_events.c b/drivers/gpu/drm/amd/amdkfd/kfd_smi_events.c
index ed4bc5f..deae12d 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_smi_events.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_smi_events.c
@@ -207,7 +207,6 @@ void kfd_smi_event_update_gpu_reset(struct kfd_dev *dev, bool post_reset)
void kfd_smi_event_update_thermal_throttling(struct kfd_dev *dev,
uint64_t throttle_bitmask)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)dev->kgd;
/*
* ThermalThrottle msg = throttle_bitmask(8):
* thermal_interrupt_count(16):
@@ -223,14 +222,13 @@ void kfd_smi_event_update_thermal_throttling(struct kfd_dev *dev,
len = snprintf(fifo_in, sizeof(fifo_in), "%x %llx:%llx\n",
KFD_SMI_EVENT_THERMAL_THROTTLE, throttle_bitmask,
- atomic64_read(&adev->smu.throttle_int_counter));
+ atomic64_read(&dev->adev->smu.throttle_int_counter));
add_event_to_kfifo(dev, KFD_SMI_EVENT_THERMAL_THROTTLE, fifo_in, len);
}
void kfd_smi_event_update_vmfault(struct kfd_dev *dev, uint16_t pasid)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)dev->kgd;
struct amdgpu_task_info task_info;
/* VmFault msg = (hex)uint32_pid(8) + :(1) + task name(16) = 25 */
/* 1 byte event + 1 byte space + 25 bytes msg + 1 byte \n +
@@ -243,7 +241,7 @@ void kfd_smi_event_update_vmfault(struct kfd_dev *dev, uint16_t pasid)
return;
memset(&task_info, 0, sizeof(struct amdgpu_task_info));
- amdgpu_vm_get_task_info(adev, pasid, &task_info);
+ amdgpu_vm_get_task_info(dev->adev, pasid, &task_info);
/* Report VM faults from user applications, not retry from kernel */
if (!task_info.pid)
return;
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
index 3cb4681..aa5ee91 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
@@ -193,7 +193,6 @@ svm_range_dma_map(struct svm_range *prange, unsigned long *bitmap,
for_each_set_bit(gpuidx, bitmap, MAX_GPU_INSTANCE) {
struct kfd_process_device *pdd;
- struct amdgpu_device *adev;
pr_debug("mapping to gpu idx 0x%x\n", gpuidx);
pdd = kfd_process_device_from_gpuidx(p, gpuidx);
@@ -201,9 +200,8 @@ svm_range_dma_map(struct svm_range *prange, unsigned long *bitmap,
pr_debug("failed to find device idx %d\n", gpuidx);
return -EINVAL;
}
- adev = (struct amdgpu_device *)pdd->dev->kgd;
- r = svm_range_dma_map_dev(adev, prange, offset, npages,
+ r = svm_range_dma_map_dev(pdd->dev->adev, prange, offset, npages,
hmm_pfns, gpuidx);
if (r)
break;
@@ -334,6 +332,8 @@ static void svm_range_bo_release(struct kref *kref)
struct svm_range_bo *svm_bo;
svm_bo = container_of(kref, struct svm_range_bo, kref);
+ pr_debug("svm_bo 0x%p\n", svm_bo);
+
spin_lock(&svm_bo->list_lock);
while (!list_empty(&svm_bo->range_list)) {
struct svm_range *prange =
@@ -367,12 +367,33 @@ static void svm_range_bo_release(struct kref *kref)
kfree(svm_bo);
}
-void svm_range_bo_unref(struct svm_range_bo *svm_bo)
+static void svm_range_bo_wq_release(struct work_struct *work)
{
- if (!svm_bo)
- return;
+ struct svm_range_bo *svm_bo;
- kref_put(&svm_bo->kref, svm_range_bo_release);
+ svm_bo = container_of(work, struct svm_range_bo, release_work);
+ svm_range_bo_release(&svm_bo->kref);
+}
+
+static void svm_range_bo_release_async(struct kref *kref)
+{
+ struct svm_range_bo *svm_bo;
+
+ svm_bo = container_of(kref, struct svm_range_bo, kref);
+ pr_debug("svm_bo 0x%p\n", svm_bo);
+ INIT_WORK(&svm_bo->release_work, svm_range_bo_wq_release);
+ schedule_work(&svm_bo->release_work);
+}
+
+void svm_range_bo_unref_async(struct svm_range_bo *svm_bo)
+{
+ kref_put(&svm_bo->kref, svm_range_bo_release_async);
+}
+
+static void svm_range_bo_unref(struct svm_range_bo *svm_bo)
+{
+ if (svm_bo)
+ kref_put(&svm_bo->kref, svm_range_bo_release);
}
static bool
@@ -581,7 +602,7 @@ svm_range_get_adev_by_id(struct svm_range *prange, uint32_t gpu_id)
return NULL;
}
- return (struct amdgpu_device *)pdd->dev->kgd;
+ return pdd->dev->adev;
}
struct kfd_process_device *
@@ -593,7 +614,7 @@ svm_range_get_pdd_by_adev(struct svm_range *prange, struct amdgpu_device *adev)
p = container_of(prange->svms, struct kfd_process, svms);
- r = kfd_process_gpuid_from_kgd(p, adev, &gpuid, &gpu_idx);
+ r = kfd_process_gpuid_from_adev(p, adev, &gpuid, &gpu_idx);
if (r) {
pr_debug("failed to get device id by adev %p\n", adev);
return NULL;
@@ -706,6 +727,61 @@ svm_range_apply_attrs(struct kfd_process *p, struct svm_range *prange,
}
}
+static bool
+svm_range_is_same_attrs(struct kfd_process *p, struct svm_range *prange,
+ uint32_t nattr, struct kfd_ioctl_svm_attribute *attrs)
+{
+ uint32_t i;
+ int gpuidx;
+
+ for (i = 0; i < nattr; i++) {
+ switch (attrs[i].type) {
+ case KFD_IOCTL_SVM_ATTR_PREFERRED_LOC:
+ if (prange->preferred_loc != attrs[i].value)
+ return false;
+ break;
+ case KFD_IOCTL_SVM_ATTR_PREFETCH_LOC:
+ /* Prefetch should always trigger a migration even
+ * if the value of the attribute didn't change.
+ */
+ return false;
+ case KFD_IOCTL_SVM_ATTR_ACCESS:
+ case KFD_IOCTL_SVM_ATTR_ACCESS_IN_PLACE:
+ case KFD_IOCTL_SVM_ATTR_NO_ACCESS:
+ gpuidx = kfd_process_gpuidx_from_gpuid(p,
+ attrs[i].value);
+ if (attrs[i].type == KFD_IOCTL_SVM_ATTR_NO_ACCESS) {
+ if (test_bit(gpuidx, prange->bitmap_access) ||
+ test_bit(gpuidx, prange->bitmap_aip))
+ return false;
+ } else if (attrs[i].type == KFD_IOCTL_SVM_ATTR_ACCESS) {
+ if (!test_bit(gpuidx, prange->bitmap_access))
+ return false;
+ } else {
+ if (!test_bit(gpuidx, prange->bitmap_aip))
+ return false;
+ }
+ break;
+ case KFD_IOCTL_SVM_ATTR_SET_FLAGS:
+ if ((prange->flags & attrs[i].value) != attrs[i].value)
+ return false;
+ break;
+ case KFD_IOCTL_SVM_ATTR_CLR_FLAGS:
+ if ((prange->flags & attrs[i].value) != 0)
+ return false;
+ break;
+ case KFD_IOCTL_SVM_ATTR_GRANULARITY:
+ if (prange->granularity != attrs[i].value)
+ return false;
+ break;
+ default:
+ WARN_ONCE(1, "svm_range_check_attrs wasn't called?");
+ }
+ }
+
+ return true;
+}
+
/**
* svm_range_debug_dump - print all range information from svms
* @svms: svm range list header
@@ -743,14 +819,6 @@ static void svm_range_debug_dump(struct svm_range_list *svms)
}
}
-static bool
-svm_range_is_same_attrs(struct svm_range *old, struct svm_range *new)
-{
- return (old->prefetch_loc == new->prefetch_loc &&
- old->flags == new->flags &&
- old->granularity == new->granularity);
-}
-
static int
svm_range_split_array(void *ppnew, void *ppold, size_t size,
uint64_t old_start, uint64_t old_n,
@@ -943,7 +1011,7 @@ svm_range_split(struct svm_range *prange, uint64_t start, uint64_t last,
}
static int
-svm_range_split_tail(struct svm_range *prange, struct svm_range *new,
+svm_range_split_tail(struct svm_range *prange,
uint64_t new_last, struct list_head *insert_list)
{
struct svm_range *tail;
@@ -955,7 +1023,7 @@ svm_range_split_tail(struct svm_range *prange, struct svm_range *new,
}
static int
-svm_range_split_head(struct svm_range *prange, struct svm_range *new,
+svm_range_split_head(struct svm_range *prange,
uint64_t new_start, struct list_head *insert_list)
{
struct svm_range *head;
@@ -1053,8 +1121,8 @@ svm_range_get_pte_flags(struct amdgpu_device *adev, struct svm_range *prange,
if (domain == SVM_RANGE_VRAM_DOMAIN)
bo_adev = amdgpu_ttm_adev(prange->svm_bo->bo->tbo.bdev);
- switch (adev->asic_type) {
- case CHIP_ARCTURUS:
+ switch (KFD_GC_VERSION(adev->kfd.dev)) {
+ case IP_VERSION(9, 4, 1):
if (domain == SVM_RANGE_VRAM_DOMAIN) {
if (bo_adev == adev) {
mapping_flags |= coherent ?
@@ -1070,7 +1138,7 @@ svm_range_get_pte_flags(struct amdgpu_device *adev, struct svm_range *prange,
AMDGPU_VM_MTYPE_UC : AMDGPU_VM_MTYPE_NC;
}
break;
- case CHIP_ALDEBARAN:
+ case IP_VERSION(9, 4, 2):
if (domain == SVM_RANGE_VRAM_DOMAIN) {
if (bo_adev == adev) {
mapping_flags |= coherent ?
@@ -1129,7 +1197,6 @@ svm_range_unmap_from_gpus(struct svm_range *prange, unsigned long start,
DECLARE_BITMAP(bitmap, MAX_GPU_INSTANCE);
struct kfd_process_device *pdd;
struct dma_fence *fence = NULL;
- struct amdgpu_device *adev;
struct kfd_process *p;
uint32_t gpuidx;
int r = 0;
@@ -1145,9 +1212,9 @@ svm_range_unmap_from_gpus(struct svm_range *prange, unsigned long start,
pr_debug("failed to find device idx %d\n", gpuidx);
return -EINVAL;
}
- adev = (struct amdgpu_device *)pdd->dev->kgd;
- r = svm_range_unmap_from_gpu(adev, drm_priv_to_vm(pdd->drm_priv),
+ r = svm_range_unmap_from_gpu(pdd->dev->adev,
+ drm_priv_to_vm(pdd->drm_priv),
start, last, &fence);
if (r)
break;
@@ -1159,7 +1226,7 @@ svm_range_unmap_from_gpus(struct svm_range *prange, unsigned long start,
if (r)
break;
}
- amdgpu_amdkfd_flush_gpu_tlb_pasid((struct kgd_dev *)adev,
+ amdgpu_amdkfd_flush_gpu_tlb_pasid(pdd->dev->adev,
p->pasid, TLB_FLUSH_HEAVYWEIGHT);
}
@@ -1172,7 +1239,6 @@ svm_range_map_to_gpu(struct amdgpu_device *adev, struct amdgpu_vm *vm,
unsigned long npages, bool readonly, dma_addr_t *dma_addr,
struct amdgpu_device *bo_adev, struct dma_fence **fence)
{
- struct amdgpu_bo_va bo_va;
bool table_freed = false;
uint64_t pte_flags;
unsigned long last_start;
@@ -1185,9 +1251,6 @@ svm_range_map_to_gpu(struct amdgpu_device *adev, struct amdgpu_vm *vm,
pr_debug("svms 0x%p [0x%lx 0x%lx] readonly %d\n", prange->svms,
last_start, last_start + npages - 1, readonly);
- if (prange->svm_bo && prange->ttm_res)
- bo_va.is_xgmi = amdgpu_xgmi_same_hive(adev, bo_adev);
-
for (i = offset; i < offset + npages; i++) {
last_domain = dma_addr[i] & SVM_RANGE_VRAM_DOMAIN;
dma_addr[i] &= ~SVM_RANGE_VRAM_DOMAIN;
@@ -1243,8 +1306,7 @@ svm_range_map_to_gpu(struct amdgpu_device *adev, struct amdgpu_vm *vm,
struct kfd_process *p;
p = container_of(prange->svms, struct kfd_process, svms);
- amdgpu_amdkfd_flush_gpu_tlb_pasid((struct kgd_dev *)adev,
- p->pasid, TLB_FLUSH_LEGACY);
+ amdgpu_amdkfd_flush_gpu_tlb_pasid(adev, p->pasid, TLB_FLUSH_LEGACY);
}
out:
return r;
@@ -1257,7 +1319,6 @@ svm_range_map_to_gpus(struct svm_range *prange, unsigned long offset,
{
struct kfd_process_device *pdd;
struct amdgpu_device *bo_adev;
- struct amdgpu_device *adev;
struct kfd_process *p;
struct dma_fence *fence = NULL;
uint32_t gpuidx;
@@ -1276,19 +1337,18 @@ svm_range_map_to_gpus(struct svm_range *prange, unsigned long offset,
pr_debug("failed to find device idx %d\n", gpuidx);
return -EINVAL;
}
- adev = (struct amdgpu_device *)pdd->dev->kgd;
pdd = kfd_bind_process_to_device(pdd->dev, p);
if (IS_ERR(pdd))
return -EINVAL;
- if (bo_adev && adev != bo_adev &&
- !amdgpu_xgmi_same_hive(adev, bo_adev)) {
+ if (bo_adev && pdd->dev->adev != bo_adev &&
+ !amdgpu_xgmi_same_hive(pdd->dev->adev, bo_adev)) {
pr_debug("cannot map to device idx %d\n", gpuidx);
continue;
}
- r = svm_range_map_to_gpu(adev, drm_priv_to_vm(pdd->drm_priv),
+ r = svm_range_map_to_gpu(pdd->dev->adev, drm_priv_to_vm(pdd->drm_priv),
prange, offset, npages, readonly,
prange->dma_addr[gpuidx],
bo_adev, wait ? &fence : NULL);
@@ -1322,7 +1382,6 @@ struct svm_validate_context {
static int svm_range_reserve_bos(struct svm_validate_context *ctx)
{
struct kfd_process_device *pdd;
- struct amdgpu_device *adev;
struct amdgpu_vm *vm;
uint32_t gpuidx;
int r;
@@ -1334,7 +1393,6 @@ static int svm_range_reserve_bos(struct svm_validate_context *ctx)
pr_debug("failed to find device idx %d\n", gpuidx);
return -EINVAL;
}
- adev = (struct amdgpu_device *)pdd->dev->kgd;
vm = drm_priv_to_vm(pdd->drm_priv);
ctx->tv[gpuidx].bo = &vm->root.bo->tbo;
@@ -1356,9 +1414,9 @@ static int svm_range_reserve_bos(struct svm_validate_context *ctx)
r = -EINVAL;
goto unreserve_out;
}
- adev = (struct amdgpu_device *)pdd->dev->kgd;
- r = amdgpu_vm_validate_pt_bos(adev, drm_priv_to_vm(pdd->drm_priv),
+ r = amdgpu_vm_validate_pt_bos(pdd->dev->adev,
+ drm_priv_to_vm(pdd->drm_priv),
svm_range_bo_validate, NULL);
if (r) {
pr_debug("failed %d validate pt bos\n", r);
@@ -1381,12 +1439,10 @@ static void svm_range_unreserve_bos(struct svm_validate_context *ctx)
static void *kfd_svm_page_owner(struct kfd_process *p, int32_t gpuidx)
{
struct kfd_process_device *pdd;
- struct amdgpu_device *adev;
pdd = kfd_process_device_from_gpuidx(p, gpuidx);
- adev = (struct amdgpu_device *)pdd->dev->kgd;
- return SVM_ADEV_PGMAP_OWNER(adev);
+ return SVM_ADEV_PGMAP_OWNER(pdd->dev->adev);
}
/*
@@ -1660,6 +1716,10 @@ static void svm_range_restore_work(struct work_struct *work)
/**
* svm_range_evict - evict svm range
+ * @prange: svm range structure
+ * @mm: current process mm_struct
+ * @start: starting process queue number
+ * @last: last process queue number
*
* Stop all queues of the process to ensure GPU doesn't access the memory, then
* return to let CPU evict the buffer and proceed CPU pagetable update.
@@ -1764,46 +1824,49 @@ static struct svm_range *svm_range_clone(struct svm_range *old)
}
/**
- * svm_range_handle_overlap - split overlap ranges
- * @svms: svm range list header
- * @new: range added with this attributes
- * @start: range added start address, in pages
- * @last: range last address, in pages
- * @update_list: output, the ranges attributes are updated. For set_attr, this
- * will do validation and map to GPUs. For unmap, this will be
- * removed and unmap from GPUs
- * @insert_list: output, the ranges will be inserted into svms, attributes are
- * not changes. For set_attr, this will add into svms.
- * @remove_list:output, the ranges will be removed from svms
- * @left: the remaining range after overlap, For set_attr, this will be added
- * as new range.
+ * svm_range_add - add svm range and handle overlap
+ * @p: the range add to this process svms
+ * @start: page size aligned
+ * @size: page size aligned
+ * @nattr: number of attributes
+ * @attrs: array of attributes
+ * @update_list: output, the ranges need validate and update GPU mapping
+ * @insert_list: output, the ranges need insert to svms
+ * @remove_list: output, the ranges are replaced and need remove from svms
*
- * Total have 5 overlap cases.
+ * Check if the virtual address range has overlap with any existing ranges,
+ * split partly overlapping ranges and add new ranges in the gaps. All changes
+ * should be applied to the range_list and interval tree transactionally. If
+ * any range split or allocation fails, the entire update fails. Therefore any
+ * existing overlapping svm_ranges are cloned and the original svm_ranges left
+ * unchanged.
*
- * This function handles overlap of an address interval with existing
- * struct svm_ranges for applying new attributes. This may require
- * splitting existing struct svm_ranges. All changes should be applied to
- * the range_list and interval tree transactionally. If any split operation
- * fails, the entire update fails. Therefore the existing overlapping
- * svm_ranges are cloned and the original svm_ranges left unchanged. If the
- * transaction succeeds, the modified clones are added and the originals
- * freed. Otherwise the clones are removed and the old svm_ranges remain.
+ * If the transaction succeeds, the caller can update and insert clones and
+ * new ranges, then free the originals.
*
- * Context: The caller must hold svms->lock
+ * Otherwise the caller can free the clones and new ranges, while the old
+ * svm_ranges remain unchanged.
+ *
+ * Context: Process context, caller must hold svms->lock
+ *
+ * Return:
+ * 0 - OK, otherwise error code
*/
static int
-svm_range_handle_overlap(struct svm_range_list *svms, struct svm_range *new,
- unsigned long start, unsigned long last,
- struct list_head *update_list,
- struct list_head *insert_list,
- struct list_head *remove_list,
- unsigned long *left)
+svm_range_add(struct kfd_process *p, uint64_t start, uint64_t size,
+ uint32_t nattr, struct kfd_ioctl_svm_attribute *attrs,
+ struct list_head *update_list, struct list_head *insert_list,
+ struct list_head *remove_list)
{
+ unsigned long last = start + size - 1UL;
+ struct svm_range_list *svms = &p->svms;
struct interval_tree_node *node;
struct svm_range *prange;
struct svm_range *tmp;
int r = 0;
+ pr_debug("svms 0x%p [0x%llx 0x%lx]\n", &p->svms, start, last);
+
INIT_LIST_HEAD(update_list);
INIT_LIST_HEAD(insert_list);
INIT_LIST_HEAD(remove_list);
@@ -1811,18 +1874,24 @@ svm_range_handle_overlap(struct svm_range_list *svms, struct svm_range *new,
node = interval_tree_iter_first(&svms->objects, start, last);
while (node) {
struct interval_tree_node *next;
- struct svm_range *old;
unsigned long next_start;
pr_debug("found overlap node [0x%lx 0x%lx]\n", node->start,
node->last);
- old = container_of(node, struct svm_range, it_node);
+ prange = container_of(node, struct svm_range, it_node);
next = interval_tree_iter_next(node, start, last);
next_start = min(node->last, last) + 1;
- if (node->start < start || node->last > last) {
- /* node intersects the updated range, clone+split it */
+ if (svm_range_is_same_attrs(p, prange, nattr, attrs)) {
+ /* nothing to do */
+ } else if (node->start < start || node->last > last) {
+ /* node intersects the update range and its attributes
+ * will change. Clone and split it, apply updates only
+ * to the overlapping part
+ */
+ struct svm_range *old = prange;
+
prange = svm_range_clone(old);
if (!prange) {
r = -ENOMEM;
@@ -1831,17 +1900,18 @@ svm_range_handle_overlap(struct svm_range_list *svms, struct svm_range *new,
list_add(&old->remove_list, remove_list);
list_add(&prange->insert_list, insert_list);
+ list_add(&prange->update_list, update_list);
if (node->start < start) {
pr_debug("change old range start\n");
- r = svm_range_split_head(prange, new, start,
+ r = svm_range_split_head(prange, start,
insert_list);
if (r)
goto out;
}
if (node->last > last) {
pr_debug("change old range last\n");
- r = svm_range_split_tail(prange, new, last,
+ r = svm_range_split_tail(prange, last,
insert_list);
if (r)
goto out;
@@ -1850,16 +1920,12 @@ svm_range_handle_overlap(struct svm_range_list *svms, struct svm_range *new,
/* The node is contained within start..last,
* just update it
*/
- prange = old;
- }
-
- if (!svm_range_is_same_attrs(prange, new))
list_add(&prange->update_list, update_list);
+ }
/* insert a new node if needed */
if (node->start > start) {
- prange = svm_range_new(prange->svms, start,
- node->start - 1);
+ prange = svm_range_new(svms, start, node->start - 1);
if (!prange) {
r = -ENOMEM;
goto out;
@@ -1873,8 +1939,16 @@ svm_range_handle_overlap(struct svm_range_list *svms, struct svm_range *new,
start = next_start;
}
- if (left && start <= last)
- *left = last - start + 1;
+ /* add a final range at the end if needed */
+ if (start <= last) {
+ prange = svm_range_new(svms, start, last);
+ if (!prange) {
+ r = -ENOMEM;
+ goto out;
+ }
+ list_add(&prange->insert_list, insert_list);
+ list_add(&prange->update_list, update_list);
+ }
out:
if (r)
@@ -1962,7 +2036,6 @@ svm_range_handle_list_op(struct svm_range_list *svms, struct svm_range *prange)
static void svm_range_drain_retry_fault(struct svm_range_list *svms)
{
struct kfd_process_device *pdd;
- struct amdgpu_device *adev;
struct kfd_process *p;
int drain;
uint32_t i;
@@ -1980,9 +2053,9 @@ static void svm_range_drain_retry_fault(struct svm_range_list *svms)
continue;
pr_debug("drain retry fault gpu %d svms %p\n", i, svms);
- adev = (struct amdgpu_device *)pdd->dev->kgd;
- amdgpu_ih_wait_on_checkpoint_process(adev, &adev->irq.ih1);
+ amdgpu_ih_wait_on_checkpoint_process_ts(pdd->dev->adev,
+ &pdd->dev->adev->irq.ih1);
pr_debug("drain retry fault gpu %d svms 0x%p done\n", i, svms);
}
if (atomic_cmpxchg(&svms->drain_pagefaults, drain, 0) != drain)
@@ -2172,6 +2245,9 @@ svm_range_unmap_from_cpu(struct mm_struct *mm, struct svm_range *prange,
/**
* svm_range_cpu_invalidate_pagetables - interval notifier callback
+ * @mni: mmu_interval_notifier struct
+ * @range: mmu_notifier_range struct
+ * @cur_seq: value to pass to mmu_interval_set_seq()
*
* If event is MMU_NOTIFY_UNMAP, this is from CPU unmap range, otherwise, it
* is from migration, or CPU page invalidation callback.
@@ -2201,8 +2277,8 @@ svm_range_cpu_invalidate_pagetables(struct mmu_interval_notifier *mni,
start = mni->interval_tree.start;
last = mni->interval_tree.last;
- start = (start > range->start ? start : range->start) >> PAGE_SHIFT;
- last = (last < (range->end - 1) ? last : range->end - 1) >> PAGE_SHIFT;
+ start = max(start, range->start) >> PAGE_SHIFT;
+ last = min(last, range->end - 1) >> PAGE_SHIFT;
pr_debug("[0x%lx 0x%lx] range[0x%lx 0x%lx] notifier[0x%lx 0x%lx] %d\n",
start, last, range->start >> PAGE_SHIFT,
(range->end - 1) >> PAGE_SHIFT,
@@ -2304,7 +2380,7 @@ svm_range_best_restore_location(struct svm_range *prange,
p = container_of(prange->svms, struct kfd_process, svms);
- r = kfd_process_gpuid_from_kgd(p, adev, &gpuid, gpuidx);
+ r = kfd_process_gpuid_from_adev(p, adev, &gpuid, gpuidx);
if (r < 0) {
pr_debug("failed to get gpuid from kgd\n");
return -1;
@@ -2481,7 +2557,7 @@ svm_range *svm_range_create_unregistered_range(struct amdgpu_device *adev,
pr_debug("Failed to create prange in address [0x%llx]\n", addr);
return NULL;
}
- if (kfd_process_gpuid_from_kgd(p, adev, &gpuid, &gpuidx)) {
+ if (kfd_process_gpuid_from_adev(p, adev, &gpuid, &gpuidx)) {
pr_debug("failed to get gpuid from kgd\n");
svm_range_free(prange);
return NULL;
@@ -2548,7 +2624,7 @@ svm_range_count_fault(struct amdgpu_device *adev, struct kfd_process *p,
uint32_t gpuid;
int r;
- r = kfd_process_gpuid_from_kgd(p, adev, &gpuid, &gpuidx);
+ r = kfd_process_gpuid_from_adev(p, adev, &gpuid, &gpuidx);
if (r < 0)
return;
}
@@ -2895,59 +2971,6 @@ svm_range_is_valid(struct kfd_process *p, uint64_t start, uint64_t size)
}
/**
- * svm_range_add - add svm range and handle overlap
- * @p: the range add to this process svms
- * @start: page size aligned
- * @size: page size aligned
- * @nattr: number of attributes
- * @attrs: array of attributes
- * @update_list: output, the ranges need validate and update GPU mapping
- * @insert_list: output, the ranges need insert to svms
- * @remove_list: output, the ranges are replaced and need remove from svms
- *
- * Check if the virtual address range has overlap with the registered ranges,
- * split the overlapped range, copy and adjust pages address and vram nodes in
- * old and new ranges.
- *
- * Context: Process context, caller must hold svms->lock
- *
- * Return:
- * 0 - OK, otherwise error code
- */
-static int
-svm_range_add(struct kfd_process *p, uint64_t start, uint64_t size,
- uint32_t nattr, struct kfd_ioctl_svm_attribute *attrs,
- struct list_head *update_list, struct list_head *insert_list,
- struct list_head *remove_list)
-{
- uint64_t last = start + size - 1UL;
- struct svm_range_list *svms;
- struct svm_range new = {0};
- struct svm_range *prange;
- unsigned long left = 0;
- int r = 0;
-
- pr_debug("svms 0x%p [0x%llx 0x%llx]\n", &p->svms, start, last);
-
- svm_range_apply_attrs(p, &new, nattr, attrs);
-
- svms = &p->svms;
-
- r = svm_range_handle_overlap(svms, &new, start, last, update_list,
- insert_list, remove_list, &left);
- if (r)
- return r;
-
- if (left) {
- prange = svm_range_new(svms, last - left + 1, last);
- list_add(&prange->insert_list, insert_list);
- list_add(&prange->update_list, update_list);
- }
-
- return 0;
-}
-
-/**
* svm_range_best_prefetch_location - decide the best prefetch location
* @prange: svm range structure
*
@@ -2980,7 +3003,6 @@ svm_range_best_prefetch_location(struct svm_range *prange)
uint32_t best_loc = prange->prefetch_loc;
struct kfd_process_device *pdd;
struct amdgpu_device *bo_adev;
- struct amdgpu_device *adev;
struct kfd_process *p;
uint32_t gpuidx;
@@ -3008,12 +3030,11 @@ svm_range_best_prefetch_location(struct svm_range *prange)
pr_debug("failed to get device by idx 0x%x\n", gpuidx);
continue;
}
- adev = (struct amdgpu_device *)pdd->dev->kgd;
- if (adev == bo_adev)
+ if (pdd->dev->adev == bo_adev)
continue;
- if (!amdgpu_xgmi_same_hive(adev, bo_adev)) {
+ if (!amdgpu_xgmi_same_hive(pdd->dev->adev, bo_adev)) {
best_loc = 0;
break;
}
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_svm.h b/drivers/gpu/drm/amd/amdkfd/kfd_svm.h
index 6dc91c3..2f8a95e 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_svm.h
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_svm.h
@@ -48,6 +48,7 @@ struct svm_range_bo {
struct work_struct eviction_work;
struct svm_range_list *svms;
uint32_t evicting;
+ struct work_struct release_work;
};
enum svm_work_list_ops {
@@ -195,7 +196,7 @@ void svm_range_list_lock_and_flush_work(struct svm_range_list *svms, struct mm_s
*/
#define KFD_IS_SVM_API_SUPPORTED(dev) ((dev)->pgmap.type != 0)
-void svm_range_bo_unref(struct svm_range_bo *svm_bo);
+void svm_range_bo_unref_async(struct svm_range_bo *svm_bo);
#else
struct kfd_process;
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_topology.c b/drivers/gpu/drm/amd/amdkfd/kfd_topology.c
index dd593ad..948fbb3 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_topology.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_topology.c
@@ -113,7 +113,7 @@ struct kfd_dev *kfd_device_by_pci_dev(const struct pci_dev *pdev)
return device;
}
-struct kfd_dev *kfd_device_by_kgd(const struct kgd_dev *kgd)
+struct kfd_dev *kfd_device_by_adev(const struct amdgpu_device *adev)
{
struct kfd_topology_device *top_dev;
struct kfd_dev *device = NULL;
@@ -121,7 +121,7 @@ struct kfd_dev *kfd_device_by_kgd(const struct kgd_dev *kgd)
down_read(&topology_lock);
list_for_each_entry(top_dev, &topology_device_list, list)
- if (top_dev->gpu && top_dev->gpu->kgd == kgd) {
+ if (top_dev->gpu && top_dev->gpu->adev == adev) {
device = top_dev->gpu;
break;
}
@@ -503,7 +503,7 @@ static ssize_t node_show(struct kobject *kobj, struct attribute *attr,
if (dev->gpu) {
log_max_watch_addr =
- __ilog2_u32(dev->gpu->device_info->num_of_watch_points);
+ __ilog2_u32(dev->gpu->device_info.num_of_watch_points);
if (log_max_watch_addr) {
dev->node_props.capability |=
@@ -515,7 +515,7 @@ static ssize_t node_show(struct kobject *kobj, struct attribute *attr,
HSA_CAP_WATCH_POINTS_TOTALBITS_MASK);
}
- if (dev->gpu->device_info->asic_family == CHIP_TONGA)
+ if (dev->gpu->adev->asic_type == CHIP_TONGA)
dev->node_props.capability |=
HSA_CAP_AQL_QUEUE_DOUBLE_MAP;
@@ -531,7 +531,7 @@ static ssize_t node_show(struct kobject *kobj, struct attribute *attr,
sysfs_show_32bit_prop(buffer, offs, "sdma_fw_version",
dev->gpu->sdma_fw_version);
sysfs_show_64bit_prop(buffer, offs, "unique_id",
- amdgpu_amdkfd_get_unique_id(dev->gpu->kgd));
+ dev->gpu->adev->unique_id);
}
@@ -1106,7 +1106,7 @@ static uint32_t kfd_generate_gpu_id(struct kfd_dev *gpu)
if (!gpu)
return 0;
- amdgpu_amdkfd_get_local_mem_info(gpu->kgd, &local_mem_info);
+ amdgpu_amdkfd_get_local_mem_info(gpu->adev, &local_mem_info);
local_mem_size = local_mem_info.local_mem_size_private +
local_mem_info.local_mem_size_public;
@@ -1189,7 +1189,7 @@ static void kfd_fill_mem_clk_max_info(struct kfd_topology_device *dev)
* for APUs - If CRAT from ACPI reports more than one bank, then
* all the banks will report the same mem_clk_max information
*/
- amdgpu_amdkfd_get_local_mem_info(dev->gpu->kgd, &local_mem_info);
+ amdgpu_amdkfd_get_local_mem_info(dev->gpu->adev, &local_mem_info);
list_for_each_entry(mem, &dev->mem_props, list)
mem->mem_clk_max = local_mem_info.mem_clk_max;
@@ -1217,8 +1217,7 @@ static void kfd_set_iolink_no_atomics(struct kfd_topology_device *dev,
/* set gpu (dev) flags. */
} else {
if (!dev->gpu->pci_atomic_requested ||
- dev->gpu->device_info->asic_family ==
- CHIP_HAWAII)
+ dev->gpu->adev->asic_type == CHIP_HAWAII)
link->flags |= CRAT_IOLINK_FLAGS_NO_ATOMICS_32_BIT |
CRAT_IOLINK_FLAGS_NO_ATOMICS_64_BIT;
}
@@ -1239,7 +1238,7 @@ static void kfd_set_iolink_non_coherent(struct kfd_topology_device *to_dev,
*/
if (inbound_link->iolink_type == CRAT_IOLINK_TYPE_PCIEXPRESS ||
(inbound_link->iolink_type == CRAT_IOLINK_TYPE_XGMI &&
- to_dev->gpu->device_info->asic_family == CHIP_VEGA20)) {
+ KFD_GC_VERSION(to_dev->gpu) == IP_VERSION(9, 4, 0))) {
outbound_link->flags |= CRAT_IOLINK_FLAGS_NON_COHERENT;
inbound_link->flags |= CRAT_IOLINK_FLAGS_NON_COHERENT;
}
@@ -1286,7 +1285,8 @@ int kfd_topology_add_device(struct kfd_dev *gpu)
void *crat_image = NULL;
size_t image_size = 0;
int proximity_domain;
- struct amdgpu_device *adev;
+ int i;
+ const char *asic_name = amdgpu_asic_name[gpu->adev->asic_type];
INIT_LIST_HEAD(&temp_topology_device_list);
@@ -1296,10 +1296,8 @@ int kfd_topology_add_device(struct kfd_dev *gpu)
proximity_domain = atomic_inc_return(&topology_crat_proximity_domain);
- adev = (struct amdgpu_device *)(gpu->kgd);
-
/* Include the CPU in xGMI hive if xGMI connected by assigning it the hive ID. */
- if (gpu->hive_id && adev->gmc.xgmi.connected_to_cpu) {
+ if (gpu->hive_id && gpu->adev->gmc.xgmi.connected_to_cpu) {
struct kfd_topology_device *top_dev;
down_read(&topology_lock);
@@ -1372,45 +1370,48 @@ int kfd_topology_add_device(struct kfd_dev *gpu)
* needed for the topology
*/
- amdgpu_amdkfd_get_cu_info(dev->gpu->kgd, &cu_info);
+ amdgpu_amdkfd_get_cu_info(dev->gpu->adev, &cu_info);
- strncpy(dev->node_props.name, gpu->device_info->asic_name,
- KFD_TOPOLOGY_PUBLIC_NAME_SIZE);
+ for (i = 0; i < KFD_TOPOLOGY_PUBLIC_NAME_SIZE-1; i++) {
+ dev->node_props.name[i] = __tolower(asic_name[i]);
+ if (asic_name[i] == '\0')
+ break;
+ }
+ dev->node_props.name[i] = '\0';
dev->node_props.simd_arrays_per_engine =
cu_info.num_shader_arrays_per_engine;
- dev->node_props.gfx_target_version = gpu->device_info->gfx_target_version;
+ dev->node_props.gfx_target_version = gpu->device_info.gfx_target_version;
dev->node_props.vendor_id = gpu->pdev->vendor;
dev->node_props.device_id = gpu->pdev->device;
dev->node_props.capability |=
- ((amdgpu_amdkfd_get_asic_rev_id(dev->gpu->kgd) <<
- HSA_CAP_ASIC_REVISION_SHIFT) &
+ ((dev->gpu->adev->rev_id << HSA_CAP_ASIC_REVISION_SHIFT) &
HSA_CAP_ASIC_REVISION_MASK);
dev->node_props.location_id = pci_dev_id(gpu->pdev);
dev->node_props.domain = pci_domain_nr(gpu->pdev->bus);
dev->node_props.max_engine_clk_fcompute =
- amdgpu_amdkfd_get_max_engine_clock_in_mhz(dev->gpu->kgd);
+ amdgpu_amdkfd_get_max_engine_clock_in_mhz(dev->gpu->adev);
dev->node_props.max_engine_clk_ccompute =
cpufreq_quick_get_max(0) / 1000;
dev->node_props.drm_render_minor =
gpu->shared_resources.drm_render_minor;
dev->node_props.hive_id = gpu->hive_id;
- dev->node_props.num_sdma_engines = gpu->device_info->num_sdma_engines;
+ dev->node_props.num_sdma_engines = kfd_get_num_sdma_engines(gpu);
dev->node_props.num_sdma_xgmi_engines =
- gpu->device_info->num_xgmi_sdma_engines;
+ kfd_get_num_xgmi_sdma_engines(gpu);
dev->node_props.num_sdma_queues_per_engine =
- gpu->device_info->num_sdma_queues_per_engine;
+ gpu->device_info.num_sdma_queues_per_engine;
dev->node_props.num_gws = (dev->gpu->gws &&
dev->gpu->dqm->sched_policy != KFD_SCHED_POLICY_NO_HWS) ?
- amdgpu_amdkfd_get_num_gws(dev->gpu->kgd) : 0;
+ dev->gpu->adev->gds.gws_size : 0;
dev->node_props.num_cp_queues = get_cp_queues_num(dev->gpu->dqm);
kfd_fill_mem_clk_max_info(dev);
kfd_fill_iolink_non_crat_info(dev);
- switch (dev->gpu->device_info->asic_family) {
+ switch (dev->gpu->adev->asic_type) {
case CHIP_KAVERI:
case CHIP_HAWAII:
case CHIP_TONGA:
@@ -1429,30 +1430,14 @@ int kfd_topology_add_device(struct kfd_dev *gpu)
HSA_CAP_DOORBELL_TYPE_TOTALBITS_SHIFT) &
HSA_CAP_DOORBELL_TYPE_TOTALBITS_MASK);
break;
- case CHIP_VEGA10:
- case CHIP_VEGA12:
- case CHIP_VEGA20:
- case CHIP_RAVEN:
- case CHIP_RENOIR:
- case CHIP_ARCTURUS:
- case CHIP_ALDEBARAN:
- case CHIP_NAVI10:
- case CHIP_NAVI12:
- case CHIP_NAVI14:
- case CHIP_SIENNA_CICHLID:
- case CHIP_NAVY_FLOUNDER:
- case CHIP_VANGOGH:
- case CHIP_DIMGREY_CAVEFISH:
- case CHIP_BEIGE_GOBY:
- case CHIP_YELLOW_CARP:
- case CHIP_CYAN_SKILLFISH:
- dev->node_props.capability |= ((HSA_CAP_DOORBELL_TYPE_2_0 <<
- HSA_CAP_DOORBELL_TYPE_TOTALBITS_SHIFT) &
- HSA_CAP_DOORBELL_TYPE_TOTALBITS_MASK);
- break;
default:
- WARN(1, "Unexpected ASIC family %u",
- dev->gpu->device_info->asic_family);
+ if (KFD_GC_VERSION(dev->gpu) >= IP_VERSION(9, 0, 1))
+ dev->node_props.capability |= ((HSA_CAP_DOORBELL_TYPE_2_0 <<
+ HSA_CAP_DOORBELL_TYPE_TOTALBITS_SHIFT) &
+ HSA_CAP_DOORBELL_TYPE_TOTALBITS_MASK);
+ else
+ WARN(1, "Unexpected ASIC family %u",
+ dev->gpu->adev->asic_type);
}
/*
@@ -1469,7 +1454,7 @@ int kfd_topology_add_device(struct kfd_dev *gpu)
* because it doesn't consider masked out CUs
* max_waves_per_simd: Carrizo reports wrong max_waves_per_simd
*/
- if (dev->gpu->device_info->asic_family == CHIP_CARRIZO) {
+ if (dev->gpu->adev->asic_type == CHIP_CARRIZO) {
dev->node_props.simd_count =
cu_info.simd_per_cu * cu_info.cu_active_number;
dev->node_props.max_waves_per_simd = 10;
@@ -1477,16 +1462,17 @@ int kfd_topology_add_device(struct kfd_dev *gpu)
/* kfd only concerns sram ecc on GFX and HBM ecc on UMC */
dev->node_props.capability |=
- ((adev->ras_enabled & BIT(AMDGPU_RAS_BLOCK__GFX)) != 0) ?
+ ((dev->gpu->adev->ras_enabled & BIT(AMDGPU_RAS_BLOCK__GFX)) != 0) ?
HSA_CAP_SRAM_EDCSUPPORTED : 0;
- dev->node_props.capability |= ((adev->ras_enabled & BIT(AMDGPU_RAS_BLOCK__UMC)) != 0) ?
+ dev->node_props.capability |=
+ ((dev->gpu->adev->ras_enabled & BIT(AMDGPU_RAS_BLOCK__UMC)) != 0) ?
HSA_CAP_MEM_EDCSUPPORTED : 0;
- if (adev->asic_type != CHIP_VEGA10)
- dev->node_props.capability |= (adev->ras_enabled != 0) ?
+ if (KFD_GC_VERSION(dev->gpu) != IP_VERSION(9, 0, 1))
+ dev->node_props.capability |= (dev->gpu->adev->ras_enabled != 0) ?
HSA_CAP_RASEVENTNOTIFY : 0;
- if (KFD_IS_SVM_API_SUPPORTED(adev->kfd.dev))
+ if (KFD_IS_SVM_API_SUPPORTED(dev->gpu->adev->kfd.dev))
dev->node_props.capability |= HSA_CAP_SVMAPI_SUPPORTED;
kfd_debug_print_topology();
@@ -1592,7 +1578,7 @@ void kfd_double_confirm_iommu_support(struct kfd_dev *gpu)
gpu->use_iommu_v2 = false;
- if (!gpu->device_info->needs_iommu_device)
+ if (!gpu->device_info.needs_iommu_device)
return;
down_read(&topology_lock);
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_topology.h b/drivers/gpu/drm/amd/amdkfd/kfd_topology.h
index a8db017..f0cc59d 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_topology.h
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_topology.h
@@ -25,38 +25,11 @@
#include <linux/types.h>
#include <linux/list.h>
+#include <linux/kfd_sysfs.h>
#include "kfd_crat.h"
#define KFD_TOPOLOGY_PUBLIC_NAME_SIZE 32
-#define HSA_CAP_HOT_PLUGGABLE 0x00000001
-#define HSA_CAP_ATS_PRESENT 0x00000002
-#define HSA_CAP_SHARED_WITH_GRAPHICS 0x00000004
-#define HSA_CAP_QUEUE_SIZE_POW2 0x00000008
-#define HSA_CAP_QUEUE_SIZE_32BIT 0x00000010
-#define HSA_CAP_QUEUE_IDLE_EVENT 0x00000020
-#define HSA_CAP_VA_LIMIT 0x00000040
-#define HSA_CAP_WATCH_POINTS_SUPPORTED 0x00000080
-#define HSA_CAP_WATCH_POINTS_TOTALBITS_MASK 0x00000f00
-#define HSA_CAP_WATCH_POINTS_TOTALBITS_SHIFT 8
-#define HSA_CAP_DOORBELL_TYPE_TOTALBITS_MASK 0x00003000
-#define HSA_CAP_DOORBELL_TYPE_TOTALBITS_SHIFT 12
-
-#define HSA_CAP_DOORBELL_TYPE_PRE_1_0 0x0
-#define HSA_CAP_DOORBELL_TYPE_1_0 0x1
-#define HSA_CAP_DOORBELL_TYPE_2_0 0x2
-#define HSA_CAP_AQL_QUEUE_DOUBLE_MAP 0x00004000
-
-#define HSA_CAP_RESERVED_WAS_SRAM_EDCSUPPORTED 0x00080000 /* Old buggy user mode depends on this being 0 */
-#define HSA_CAP_MEM_EDCSUPPORTED 0x00100000
-#define HSA_CAP_RASEVENTNOTIFY 0x00200000
-#define HSA_CAP_ASIC_REVISION_MASK 0x03c00000
-#define HSA_CAP_ASIC_REVISION_SHIFT 22
-#define HSA_CAP_SRAM_EDCSUPPORTED 0x04000000
-#define HSA_CAP_SVMAPI_SUPPORTED 0x08000000
-#define HSA_CAP_FLAGS_COHERENTHOSTACCESS 0x10000000
-#define HSA_CAP_RESERVED 0xe00f8000
-
struct kfd_node_properties {
uint64_t hive_id;
uint32_t cpu_cores_count;
@@ -93,17 +66,6 @@ struct kfd_node_properties {
char name[KFD_TOPOLOGY_PUBLIC_NAME_SIZE];
};
-#define HSA_MEM_HEAP_TYPE_SYSTEM 0
-#define HSA_MEM_HEAP_TYPE_FB_PUBLIC 1
-#define HSA_MEM_HEAP_TYPE_FB_PRIVATE 2
-#define HSA_MEM_HEAP_TYPE_GPU_GDS 3
-#define HSA_MEM_HEAP_TYPE_GPU_LDS 4
-#define HSA_MEM_HEAP_TYPE_GPU_SCRATCH 5
-
-#define HSA_MEM_FLAGS_HOT_PLUGGABLE 0x00000001
-#define HSA_MEM_FLAGS_NON_VOLATILE 0x00000002
-#define HSA_MEM_FLAGS_RESERVED 0xfffffffc
-
struct kfd_mem_properties {
struct list_head list;
uint32_t heap_type;
@@ -116,12 +78,6 @@ struct kfd_mem_properties {
struct attribute attr;
};
-#define HSA_CACHE_TYPE_DATA 0x00000001
-#define HSA_CACHE_TYPE_INSTRUCTION 0x00000002
-#define HSA_CACHE_TYPE_CPU 0x00000004
-#define HSA_CACHE_TYPE_HSACU 0x00000008
-#define HSA_CACHE_TYPE_RESERVED 0xfffffff0
-
struct kfd_cache_properties {
struct list_head list;
uint32_t processor_id_low;
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
index e727f1d..2f0b14f 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
@@ -624,7 +624,7 @@ static void dm_dcn_vertical_interrupt0_high_irq(void *interrupt_params)
#endif /* CONFIG_DRM_AMD_SECURE_DISPLAY */
/**
- * dmub_aux_setconfig_reply_callback - Callback for AUX or SET_CONFIG command.
+ * dmub_aux_setconfig_callback - Callback for AUX or SET_CONFIG command.
* @adev: amdgpu_device pointer
* @notify: dmub notification structure
*
@@ -632,7 +632,8 @@ static void dm_dcn_vertical_interrupt0_high_irq(void *interrupt_params)
* Copies dmub notification to DM which is to be read by AUX command.
* issuing thread and also signals the event to wake up the thread.
*/
-void dmub_aux_setconfig_callback(struct amdgpu_device *adev, struct dmub_notification *notify)
+static void dmub_aux_setconfig_callback(struct amdgpu_device *adev,
+ struct dmub_notification *notify)
{
if (adev->dm.dmub_notify)
memcpy(adev->dm.dmub_notify, notify, sizeof(struct dmub_notification));
@@ -648,7 +649,8 @@ void dmub_aux_setconfig_callback(struct amdgpu_device *adev, struct dmub_notific
* Dmub Hpd interrupt processing callback. Gets displayindex through the
* ink index and calls helper to do the processing.
*/
-void dmub_hpd_callback(struct amdgpu_device *adev, struct dmub_notification *notify)
+static void dmub_hpd_callback(struct amdgpu_device *adev,
+ struct dmub_notification *notify)
{
struct amdgpu_dm_connector *aconnector;
struct amdgpu_dm_connector *hpd_aconnector = NULL;
@@ -705,8 +707,10 @@ void dmub_hpd_callback(struct amdgpu_device *adev, struct dmub_notification *not
* to dmub interrupt handling thread
* Return: true if successfully registered, false if there is existing registration
*/
-bool register_dmub_notify_callback(struct amdgpu_device *adev, enum dmub_notification_type type,
-dmub_notify_interrupt_callback_t callback, bool dmub_int_thread_offload)
+static bool register_dmub_notify_callback(struct amdgpu_device *adev,
+ enum dmub_notification_type type,
+ dmub_notify_interrupt_callback_t callback,
+ bool dmub_int_thread_offload)
{
if (callback != NULL && type < ARRAY_SIZE(adev->dm.dmub_thread_offload)) {
adev->dm.dmub_callback[type] = callback;
@@ -790,8 +794,7 @@ static void dm_dmub_outbox1_low_irq(void *interrupt_params)
plink = adev->dm.dc->links[notify.link_index];
if (plink) {
plink->hpd_status =
- notify.hpd_status ==
- DP_HPD_PLUG ? true : false;
+ notify.hpd_status == DP_HPD_PLUG;
}
}
queue_work(adev->dm.delayed_hpd_wq, &dmub_hpd_wrk->handle_hpd_work);
@@ -1459,8 +1462,21 @@ static int amdgpu_dm_init(struct amdgpu_device *adev)
if (amdgpu_dc_feature_mask & DC_EDP_NO_POWER_SEQUENCING)
init_data.flags.edp_no_power_sequencing = true;
+#ifdef CONFIG_DRM_AMD_DC_DCN
+ if (amdgpu_dc_feature_mask & DC_DISABLE_LTTPR_DP1_4A)
+ init_data.flags.allow_lttpr_non_transparent_mode.bits.DP1_4A = true;
+ if (amdgpu_dc_feature_mask & DC_DISABLE_LTTPR_DP2_0)
+ init_data.flags.allow_lttpr_non_transparent_mode.bits.DP2_0 = true;
+#endif
+
init_data.flags.power_down_display_on_boot = true;
+ if (check_seamless_boot_capability(adev)) {
+ init_data.flags.power_down_display_on_boot = false;
+ init_data.flags.allow_seamless_boot_optimization = true;
+ DRM_INFO("Seamless boot condition check passed\n");
+ }
+
INIT_LIST_HEAD(&adev->dm.da_list);
/* Display Core create. */
adev->dm.dc = dc_create(&init_data);
@@ -1485,8 +1501,10 @@ static int amdgpu_dm_init(struct amdgpu_device *adev)
if (amdgpu_dc_debug_mask & DC_DISABLE_STUTTER)
adev->dm.dc->debug.disable_stutter = true;
- if (amdgpu_dc_debug_mask & DC_DISABLE_DSC)
+ if (amdgpu_dc_debug_mask & DC_DISABLE_DSC) {
adev->dm.dc->debug.disable_dsc = true;
+ adev->dm.dc->debug.disable_dsc_edp = true;
+ }
if (amdgpu_dc_debug_mask & DC_DISABLE_CLOCK_GATING)
adev->dm.dc->debug.disable_clock_gate = true;
@@ -2309,14 +2327,6 @@ static enum dc_status amdgpu_dm_commit_zero_streams(struct dc *dc)
goto fail;
}
-
- res = dc_validate_global_state(dc, context, false);
-
- if (res != DC_OK) {
- DRM_ERROR("%s:resource validation failed, dc_status:%d\n", __func__, res);
- goto fail;
- }
-
res = dc_commit_state(dc, context);
fail:
@@ -2600,15 +2610,6 @@ static int dm_resume(void *handle)
= 0xffffffff;
}
}
-#if defined(CONFIG_DRM_AMD_DC_DCN)
- /*
- * Resource allocation happens for link encoders for newer ASIC in
- * dc_validate_global_state, so we need to revalidate it.
- *
- * This shouldn't fail (it passed once before), so warn if it does.
- */
- WARN_ON(dc_validate_global_state(dm->dc, dc_state, false) != DC_OK);
-#endif
WARN_ON(!dc_commit_state(dm->dc, dc_state));
@@ -2965,13 +2966,12 @@ void amdgpu_dm_update_connector_after_detect(
aconnector->edid =
(struct edid *)sink->dc_edid.raw_edid;
- drm_connector_update_edid_property(connector,
- aconnector->edid);
if (aconnector->dc_link->aux_mode)
drm_dp_cec_set_edid(&aconnector->dm_dp_aux.aux,
aconnector->edid);
}
+ drm_connector_update_edid_property(connector, aconnector->edid);
amdgpu_dm_update_freesync_caps(connector, aconnector->edid);
update_connector_ext_caps(aconnector);
} else {
@@ -3039,7 +3039,7 @@ static void handle_hpd_irq_helper(struct amdgpu_dm_connector *aconnector)
drm_modeset_unlock_all(dev);
if (aconnector->base.force == DRM_FORCE_UNSPECIFIED)
- drm_kms_helper_hotplug_event(dev);
+ drm_kms_helper_connector_hotplug_event(connector);
} else if (dc_link_detect(aconnector->dc_link, DETECT_REASON_HPD)) {
if (new_connection_type == dc_connection_none &&
@@ -3054,7 +3054,7 @@ static void handle_hpd_irq_helper(struct amdgpu_dm_connector *aconnector)
drm_modeset_unlock_all(dev);
if (aconnector->base.force == DRM_FORCE_UNSPECIFIED)
- drm_kms_helper_hotplug_event(dev);
+ drm_kms_helper_connector_hotplug_event(connector);
}
mutex_unlock(&aconnector->hpd_lock);
@@ -3248,7 +3248,7 @@ static void handle_hpd_rx_irq(void *param)
dm_restore_drm_connector_state(dev, connector);
drm_modeset_unlock_all(dev);
- drm_kms_helper_hotplug_event(dev);
+ drm_kms_helper_connector_hotplug_event(connector);
} else if (dc_link_detect(dc_link, DETECT_REASON_HPDRX)) {
if (aconnector->fake_enable)
@@ -3261,7 +3261,7 @@ static void handle_hpd_rx_irq(void *param)
dm_restore_drm_connector_state(dev, connector);
drm_modeset_unlock_all(dev);
- drm_kms_helper_hotplug_event(dev);
+ drm_kms_helper_connector_hotplug_event(connector);
}
}
#ifdef CONFIG_DRM_AMD_DC_HDCP
@@ -4281,6 +4281,14 @@ static int amdgpu_dm_initialize_drm_device(struct amdgpu_device *adev)
}
+ /*
+ * Disable vblank IRQs aggressively for power-saving.
+ *
+ * TODO: Fix vblank control helpers to delay PSR entry to allow this when PSR
+ * is also supported.
+ */
+ adev_to_drm(adev)->vblank_disable_immediate = !psr_feature_enabled;
+
/* Software is initialized. Now we can register interrupt handlers. */
switch (adev->asic_type) {
#if defined(CONFIG_DRM_AMD_DC_SI)
@@ -6066,11 +6074,72 @@ static void update_dsc_caps(struct amdgpu_dm_connector *aconnector,
{
stream->timing.flags.DSC = 0;
- if (aconnector->dc_link && sink->sink_signal == SIGNAL_TYPE_DISPLAY_PORT) {
- dc_dsc_parse_dsc_dpcd(aconnector->dc_link->ctx->dc,
- aconnector->dc_link->dpcd_caps.dsc_caps.dsc_basic_caps.raw,
- aconnector->dc_link->dpcd_caps.dsc_caps.dsc_branch_decoder_caps.raw,
- dsc_caps);
+ if (aconnector->dc_link && (sink->sink_signal == SIGNAL_TYPE_DISPLAY_PORT ||
+ sink->sink_signal == SIGNAL_TYPE_EDP)) {
+ if (sink->link->dpcd_caps.dongle_type == DISPLAY_DONGLE_NONE ||
+ sink->link->dpcd_caps.dongle_type == DISPLAY_DONGLE_DP_HDMI_CONVERTER)
+ dc_dsc_parse_dsc_dpcd(aconnector->dc_link->ctx->dc,
+ aconnector->dc_link->dpcd_caps.dsc_caps.dsc_basic_caps.raw,
+ aconnector->dc_link->dpcd_caps.dsc_caps.dsc_branch_decoder_caps.raw,
+ dsc_caps);
+ }
+}
+
+static void apply_dsc_policy_for_edp(struct amdgpu_dm_connector *aconnector,
+ struct dc_sink *sink, struct dc_stream_state *stream,
+ struct dsc_dec_dpcd_caps *dsc_caps,
+ uint32_t max_dsc_target_bpp_limit_override)
+{
+ const struct dc_link_settings *verified_link_cap = NULL;
+ uint32_t link_bw_in_kbps;
+ uint32_t edp_min_bpp_x16, edp_max_bpp_x16;
+ struct dc *dc = sink->ctx->dc;
+ struct dc_dsc_bw_range bw_range = {0};
+ struct dc_dsc_config dsc_cfg = {0};
+
+ verified_link_cap = dc_link_get_link_cap(stream->link);
+ link_bw_in_kbps = dc_link_bandwidth_kbps(stream->link, verified_link_cap);
+ edp_min_bpp_x16 = 8 * 16;
+ edp_max_bpp_x16 = 8 * 16;
+
+ if (edp_max_bpp_x16 > dsc_caps->edp_max_bits_per_pixel)
+ edp_max_bpp_x16 = dsc_caps->edp_max_bits_per_pixel;
+
+ if (edp_max_bpp_x16 < edp_min_bpp_x16)
+ edp_min_bpp_x16 = edp_max_bpp_x16;
+
+ if (dc_dsc_compute_bandwidth_range(dc->res_pool->dscs[0],
+ dc->debug.dsc_min_slice_height_override,
+ edp_min_bpp_x16, edp_max_bpp_x16,
+ dsc_caps,
+ &stream->timing,
+ &bw_range)) {
+
+ if (bw_range.max_kbps < link_bw_in_kbps) {
+ if (dc_dsc_compute_config(dc->res_pool->dscs[0],
+ dsc_caps,
+ dc->debug.dsc_min_slice_height_override,
+ max_dsc_target_bpp_limit_override,
+ 0,
+ &stream->timing,
+ &dsc_cfg)) {
+ stream->timing.dsc_cfg = dsc_cfg;
+ stream->timing.flags.DSC = 1;
+ stream->timing.dsc_cfg.bits_per_pixel = edp_max_bpp_x16;
+ }
+ return;
+ }
+ }
+
+ if (dc_dsc_compute_config(dc->res_pool->dscs[0],
+ dsc_caps,
+ dc->debug.dsc_min_slice_height_override,
+ max_dsc_target_bpp_limit_override,
+ link_bw_in_kbps,
+ &stream->timing,
+ &dsc_cfg)) {
+ stream->timing.dsc_cfg = dsc_cfg;
+ stream->timing.flags.DSC = 1;
}
}
@@ -6081,6 +6150,9 @@ static void apply_dsc_policy_for_stream(struct amdgpu_dm_connector *aconnector,
struct drm_connector *drm_connector = &aconnector->base;
uint32_t link_bandwidth_kbps;
uint32_t max_dsc_target_bpp_limit_override = 0;
+ struct dc *dc = sink->ctx->dc;
+ uint32_t max_supported_bw_in_kbps, timing_bw_in_kbps;
+ uint32_t dsc_max_supported_bw_in_kbps;
link_bandwidth_kbps = dc_link_bandwidth_kbps(aconnector->dc_link,
dc_link_get_link_cap(aconnector->dc_link));
@@ -6093,17 +6165,43 @@ static void apply_dsc_policy_for_stream(struct amdgpu_dm_connector *aconnector,
dc_dsc_policy_set_enable_dsc_when_not_needed(
aconnector->dsc_settings.dsc_force_enable == DSC_CLK_FORCE_ENABLE);
- if (aconnector->dc_link && sink->sink_signal == SIGNAL_TYPE_DISPLAY_PORT) {
+ if (aconnector->dc_link && sink->sink_signal == SIGNAL_TYPE_EDP && !dc->debug.disable_dsc_edp &&
+ dc->caps.edp_dsc_support && aconnector->dsc_settings.dsc_force_enable != DSC_CLK_FORCE_DISABLE) {
- if (dc_dsc_compute_config(aconnector->dc_link->ctx->dc->res_pool->dscs[0],
+ apply_dsc_policy_for_edp(aconnector, sink, stream, dsc_caps, max_dsc_target_bpp_limit_override);
+
+ } else if (aconnector->dc_link && sink->sink_signal == SIGNAL_TYPE_DISPLAY_PORT) {
+ if (sink->link->dpcd_caps.dongle_type == DISPLAY_DONGLE_NONE) {
+ if (dc_dsc_compute_config(aconnector->dc_link->ctx->dc->res_pool->dscs[0],
dsc_caps,
aconnector->dc_link->ctx->dc->debug.dsc_min_slice_height_override,
max_dsc_target_bpp_limit_override,
link_bandwidth_kbps,
&stream->timing,
&stream->timing.dsc_cfg)) {
- stream->timing.flags.DSC = 1;
- DRM_DEBUG_DRIVER("%s: [%s] DSC is selected from SST RX\n", __func__, drm_connector->name);
+ stream->timing.flags.DSC = 1;
+ DRM_DEBUG_DRIVER("%s: [%s] DSC is selected from SST RX\n",
+ __func__, drm_connector->name);
+ }
+ } else if (sink->link->dpcd_caps.dongle_type == DISPLAY_DONGLE_DP_HDMI_CONVERTER) {
+ timing_bw_in_kbps = dc_bandwidth_in_kbps_from_timing(&stream->timing);
+ max_supported_bw_in_kbps = link_bandwidth_kbps;
+ dsc_max_supported_bw_in_kbps = link_bandwidth_kbps;
+
+ if (timing_bw_in_kbps > max_supported_bw_in_kbps &&
+ max_supported_bw_in_kbps > 0 &&
+ dsc_max_supported_bw_in_kbps > 0)
+ if (dc_dsc_compute_config(aconnector->dc_link->ctx->dc->res_pool->dscs[0],
+ dsc_caps,
+ aconnector->dc_link->ctx->dc->debug.dsc_min_slice_height_override,
+ max_dsc_target_bpp_limit_override,
+ dsc_max_supported_bw_in_kbps,
+ &stream->timing,
+ &stream->timing.dsc_cfg)) {
+ stream->timing.flags.DSC = 1;
+ DRM_DEBUG_DRIVER("%s: [%s] DSC is selected from DP-HDMI PCON\n",
+ __func__, drm_connector->name);
+ }
}
}
@@ -8247,15 +8345,8 @@ void amdgpu_dm_connector_init_helper(struct amdgpu_display_manager *dm,
break;
case DRM_MODE_CONNECTOR_DisplayPort:
aconnector->base.polled = DRM_CONNECTOR_POLL_HPD;
- if (link->is_dig_mapping_flexible &&
- link->dc->res_pool->funcs->link_encs_assign) {
- link->link_enc =
- link_enc_cfg_get_link_enc_used_by_link(link->ctx->dc, link);
- if (!link->link_enc)
- link->link_enc =
- link_enc_cfg_get_next_avail_link_enc(link->ctx->dc);
- }
-
+ link->link_enc = dp_get_link_enc(link);
+ ASSERT(link->link_enc);
if (link->link_enc)
aconnector->base.ycbcr_420_allowed =
link->link_enc->features.dp_ycbcr420_supported ? true : false;
@@ -10658,6 +10749,24 @@ static int dm_update_plane_state(struct dc *dc,
return ret;
}
+static void dm_get_oriented_plane_size(struct drm_plane_state *plane_state,
+ int *src_w, int *src_h)
+{
+ switch (plane_state->rotation & DRM_MODE_ROTATE_MASK) {
+ case DRM_MODE_ROTATE_90:
+ case DRM_MODE_ROTATE_270:
+ *src_w = plane_state->src_h >> 16;
+ *src_h = plane_state->src_w >> 16;
+ break;
+ case DRM_MODE_ROTATE_0:
+ case DRM_MODE_ROTATE_180:
+ default:
+ *src_w = plane_state->src_w >> 16;
+ *src_h = plane_state->src_h >> 16;
+ break;
+ }
+}
+
static int dm_check_crtc_cursor(struct drm_atomic_state *state,
struct drm_crtc *crtc,
struct drm_crtc_state *new_crtc_state)
@@ -10666,6 +10775,8 @@ static int dm_check_crtc_cursor(struct drm_atomic_state *state,
struct drm_plane_state *new_cursor_state, *new_underlying_state;
int i;
int cursor_scale_w, cursor_scale_h, underlying_scale_w, underlying_scale_h;
+ int cursor_src_w, cursor_src_h;
+ int underlying_src_w, underlying_src_h;
/* On DCE and DCN there is no dedicated hardware cursor plane. We get a
* cursor per pipe but it's going to inherit the scaling and
@@ -10677,10 +10788,9 @@ static int dm_check_crtc_cursor(struct drm_atomic_state *state,
return 0;
}
- cursor_scale_w = new_cursor_state->crtc_w * 1000 /
- (new_cursor_state->src_w >> 16);
- cursor_scale_h = new_cursor_state->crtc_h * 1000 /
- (new_cursor_state->src_h >> 16);
+ dm_get_oriented_plane_size(new_cursor_state, &cursor_src_w, &cursor_src_h);
+ cursor_scale_w = new_cursor_state->crtc_w * 1000 / cursor_src_w;
+ cursor_scale_h = new_cursor_state->crtc_h * 1000 / cursor_src_h;
for_each_new_plane_in_state_reverse(state, underlying, new_underlying_state, i) {
/* Narrow down to non-cursor planes on the same CRTC as the cursor */
@@ -10691,10 +10801,10 @@ static int dm_check_crtc_cursor(struct drm_atomic_state *state,
if (!new_underlying_state->fb)
continue;
- underlying_scale_w = new_underlying_state->crtc_w * 1000 /
- (new_underlying_state->src_w >> 16);
- underlying_scale_h = new_underlying_state->crtc_h * 1000 /
- (new_underlying_state->src_h >> 16);
+ dm_get_oriented_plane_size(new_underlying_state,
+ &underlying_src_w, &underlying_src_h);
+ underlying_scale_w = new_underlying_state->crtc_w * 1000 / underlying_src_w;
+ underlying_scale_h = new_underlying_state->crtc_h * 1000 / underlying_src_h;
if (cursor_scale_w != underlying_scale_w ||
cursor_scale_h != underlying_scale_h) {
@@ -10789,8 +10899,10 @@ static int amdgpu_dm_atomic_check(struct drm_device *dev,
trace_amdgpu_dm_atomic_check_begin(state);
ret = drm_atomic_helper_check_modeset(dev, state);
- if (ret)
+ if (ret) {
+ DRM_DEBUG_DRIVER("drm_atomic_helper_check_modeset() failed\n");
goto fail;
+ }
/* Check connector changes */
for_each_oldnew_connector_in_state(state, connector, old_con_state, new_con_state, i) {
@@ -10806,6 +10918,7 @@ static int amdgpu_dm_atomic_check(struct drm_device *dev,
new_crtc_state = drm_atomic_get_crtc_state(state, new_con_state->crtc);
if (IS_ERR(new_crtc_state)) {
+ DRM_DEBUG_DRIVER("drm_atomic_get_crtc_state() failed\n");
ret = PTR_ERR(new_crtc_state);
goto fail;
}
@@ -10820,8 +10933,10 @@ static int amdgpu_dm_atomic_check(struct drm_device *dev,
for_each_oldnew_crtc_in_state(state, crtc, old_crtc_state, new_crtc_state, i) {
if (drm_atomic_crtc_needs_modeset(new_crtc_state)) {
ret = add_affected_mst_dsc_crtcs(state, crtc);
- if (ret)
+ if (ret) {
+ DRM_DEBUG_DRIVER("add_affected_mst_dsc_crtcs() failed\n");
goto fail;
+ }
}
}
}
@@ -10836,19 +10951,25 @@ static int amdgpu_dm_atomic_check(struct drm_device *dev,
continue;
ret = amdgpu_dm_verify_lut_sizes(new_crtc_state);
- if (ret)
+ if (ret) {
+ DRM_DEBUG_DRIVER("amdgpu_dm_verify_lut_sizes() failed\n");
goto fail;
+ }
if (!new_crtc_state->enable)
continue;
ret = drm_atomic_add_affected_connectors(state, crtc);
- if (ret)
+ if (ret) {
+ DRM_DEBUG_DRIVER("drm_atomic_add_affected_connectors() failed\n");
goto fail;
+ }
ret = drm_atomic_add_affected_planes(state, crtc);
- if (ret)
+ if (ret) {
+ DRM_DEBUG_DRIVER("drm_atomic_add_affected_planes() failed\n");
goto fail;
+ }
if (dm_old_crtc_state->dsc_force_changed)
new_crtc_state->mode_changed = true;
@@ -10885,6 +11006,7 @@ static int amdgpu_dm_atomic_check(struct drm_device *dev,
if (IS_ERR(new_plane_state)) {
ret = PTR_ERR(new_plane_state);
+ DRM_DEBUG_DRIVER("new_plane_state is BAD\n");
goto fail;
}
}
@@ -10897,8 +11019,10 @@ static int amdgpu_dm_atomic_check(struct drm_device *dev,
new_plane_state,
false,
&lock_and_validation_needed);
- if (ret)
+ if (ret) {
+ DRM_DEBUG_DRIVER("dm_update_plane_state() failed\n");
goto fail;
+ }
}
/* Disable all crtcs which require disable */
@@ -10908,8 +11032,10 @@ static int amdgpu_dm_atomic_check(struct drm_device *dev,
new_crtc_state,
false,
&lock_and_validation_needed);
- if (ret)
+ if (ret) {
+ DRM_DEBUG_DRIVER("DISABLE: dm_update_crtc_state() failed\n");
goto fail;
+ }
}
/* Enable all crtcs which require enable */
@@ -10919,8 +11045,10 @@ static int amdgpu_dm_atomic_check(struct drm_device *dev,
new_crtc_state,
true,
&lock_and_validation_needed);
- if (ret)
+ if (ret) {
+ DRM_DEBUG_DRIVER("ENABLE: dm_update_crtc_state() failed\n");
goto fail;
+ }
}
/* Add new/modified planes */
@@ -10930,20 +11058,26 @@ static int amdgpu_dm_atomic_check(struct drm_device *dev,
new_plane_state,
true,
&lock_and_validation_needed);
- if (ret)
+ if (ret) {
+ DRM_DEBUG_DRIVER("dm_update_plane_state() failed\n");
goto fail;
+ }
}
/* Run this here since we want to validate the streams we created */
ret = drm_atomic_helper_check_planes(dev, state);
- if (ret)
+ if (ret) {
+ DRM_DEBUG_DRIVER("drm_atomic_helper_check_planes() failed\n");
goto fail;
+ }
/* Check cursor planes scaling */
for_each_new_crtc_in_state(state, crtc, new_crtc_state, i) {
ret = dm_check_crtc_cursor(state, crtc, new_crtc_state);
- if (ret)
+ if (ret) {
+ DRM_DEBUG_DRIVER("dm_check_crtc_cursor() failed\n");
goto fail;
+ }
}
if (state->legacy_cursor_update) {
@@ -11030,20 +11164,28 @@ static int amdgpu_dm_atomic_check(struct drm_device *dev,
*/
if (lock_and_validation_needed) {
ret = dm_atomic_get_state(state, &dm_state);
- if (ret)
+ if (ret) {
+ DRM_DEBUG_DRIVER("dm_atomic_get_state() failed\n");
goto fail;
+ }
ret = do_aquire_global_lock(dev, state);
- if (ret)
+ if (ret) {
+ DRM_DEBUG_DRIVER("do_aquire_global_lock() failed\n");
goto fail;
+ }
#if defined(CONFIG_DRM_AMD_DC_DCN)
- if (!compute_mst_dsc_configs_for_state(state, dm_state->context, vars))
+ if (!compute_mst_dsc_configs_for_state(state, dm_state->context, vars)) {
+ DRM_DEBUG_DRIVER("compute_mst_dsc_configs_for_state() failed\n");
goto fail;
+ }
ret = dm_update_mst_vcpi_slots_for_dsc(state, dm_state->context, vars);
- if (ret)
+ if (ret) {
+ DRM_DEBUG_DRIVER("dm_update_mst_vcpi_slots_for_dsc() failed\n");
goto fail;
+ }
#endif
/*
@@ -11053,12 +11195,13 @@ static int amdgpu_dm_atomic_check(struct drm_device *dev,
* to get stuck in an infinite loop and hang eventually.
*/
ret = drm_dp_mst_atomic_check(state);
- if (ret)
+ if (ret) {
+ DRM_DEBUG_DRIVER("drm_dp_mst_atomic_check() failed\n");
goto fail;
- status = dc_validate_global_state(dc, dm_state->context, false);
+ }
+ status = dc_validate_global_state(dc, dm_state->context, true);
if (status != DC_OK) {
- drm_dbg_atomic(dev,
- "DC global validation failure: %s (%d)",
+ DRM_DEBUG_DRIVER("DC global validation failure: %s (%d)",
dc_status_to_str(status), status);
ret = -EINVAL;
goto fail;
@@ -11180,7 +11323,7 @@ static bool dm_edid_parser_send_cea(struct amdgpu_display_manager *dm,
sizeof(cmd.edid_cea) - sizeof(cmd.edid_cea.header);
input->offset = offset;
input->length = length;
- input->total_length = total_length;
+ input->cea_total_length = total_length;
memcpy(input->payload, data, length);
res = dc_dmub_srv_cmd_with_reply_data(dm->dc->ctx->dmub_srv, &cmd);
@@ -11487,8 +11630,10 @@ uint32_t dm_read_reg_func(const struct dc_context *ctx, uint32_t address,
return value;
}
-int amdgpu_dm_set_dmub_async_sync_status(bool is_cmd_aux, struct dc_context *ctx,
- uint8_t status_type, uint32_t *operation_result)
+static int amdgpu_dm_set_dmub_async_sync_status(bool is_cmd_aux,
+ struct dc_context *ctx,
+ uint8_t status_type,
+ uint32_t *operation_result)
{
struct amdgpu_device *adev = ctx->driver_context;
int return_status = -1;
@@ -11559,3 +11704,24 @@ int amdgpu_dm_process_dmub_aux_transfer_sync(bool is_cmd_aux, struct dc_context
ctx, DMUB_ASYNC_TO_SYNC_ACCESS_SUCCESS,
(uint32_t *)operation_result);
}
+
+/*
+ * Check whether seamless boot is supported.
+ *
+ * So far we only support seamless boot on CHIP_VANGOGH.
+ * If everything goes well, we may consider expanding
+ * seamless boot to other ASICs.
+ */
+bool check_seamless_boot_capability(struct amdgpu_device *adev)
+{
+ switch (adev->asic_type) {
+ case CHIP_VANGOGH:
+ if (!adev->mman.keep_stolen_vga_memory)
+ return true;
+ break;
+ default:
+ break;
+ }
+
+ return false;
+}
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h
index 37e61a88..c98e402 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h
@@ -50,9 +50,9 @@
#define AMDGPU_DMUB_NOTIFICATION_MAX 5
-/**
+/*
* DMUB Async to Sync Mechanism Status
- **/
+ */
#define DMUB_ASYNC_TO_SYNC_ACCESS_FAIL 1
#define DMUB_ASYNC_TO_SYNC_ACCESS_TIMEOUT 2
#define DMUB_ASYNC_TO_SYNC_ACCESS_SUCCESS 3
@@ -731,4 +731,7 @@ extern const struct drm_encoder_helper_funcs amdgpu_dm_encoder_helper_funcs;
int amdgpu_dm_process_dmub_aux_transfer_sync(bool is_cmd_aux,
struct dc_context *ctx, unsigned int link_index,
void *payload, void *operation_result);
+
+bool check_seamless_boot_capability(struct amdgpu_device *adev);
+
#endif /* __AMDGPU_DM_H__ */
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_color.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_color.c
index a022e5b..a711773 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_color.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_color.c
@@ -285,8 +285,12 @@ static int __set_input_tf(struct dc_transfer_func *func,
}
/**
+ * amdgpu_dm_verify_lut_sizes
+ * @crtc_state: the DRM CRTC state
+ *
* Verifies that the Degamma and Gamma LUTs attached to the |crtc_state| are of
* the expected size.
+ *
* Returns 0 on success.
*/
int amdgpu_dm_verify_lut_sizes(const struct drm_crtc_state *crtc_state)
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_debugfs.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_debugfs.c
index 9d43ecb..26719ef 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_debugfs.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_debugfs.c
@@ -824,6 +824,48 @@ static int dmub_fw_state_show(struct seq_file *m, void *data)
return seq_write(m, state_base, state_size);
}
+/* psr_capability_show() - show eDP panel PSR capability
+ *
+ * The read function: sink_psr_capability_show
+ * Shows if sink has PSR capability or not.
+ * If yes - the PSR version is appended
+ *
+ * cat /sys/kernel/debug/dri/0/eDP-X/psr_capability
+ *
+ * Expected output:
+ * "Sink support: no\n" - if panel doesn't support PSR
+ * "Sink support: yes [0x01]\n" - if panel supports PSR1
+ * "Driver support: no\n" - if driver doesn't support PSR
+ * "Driver support: yes [0x01]\n" - if driver supports PSR1
+ */
+static int psr_capability_show(struct seq_file *m, void *data)
+{
+ struct drm_connector *connector = m->private;
+ struct amdgpu_dm_connector *aconnector = to_amdgpu_dm_connector(connector);
+ struct dc_link *link = aconnector->dc_link;
+
+ if (!link)
+ return -ENODEV;
+
+ if (link->type == dc_connection_none)
+ return -ENODEV;
+
+ if (!(link->connector_signal & SIGNAL_TYPE_EDP))
+ return -ENODEV;
+
+ seq_printf(m, "Sink support: %s", yesno(link->dpcd_caps.psr_caps.psr_version != 0));
+ if (link->dpcd_caps.psr_caps.psr_version)
+ seq_printf(m, " [0x%02x]", link->dpcd_caps.psr_caps.psr_version);
+ seq_puts(m, "\n");
+
+ seq_printf(m, "Driver support: %s", yesno(link->psr_settings.psr_feature_enabled));
+ if (link->psr_settings.psr_version)
+ seq_printf(m, " [0x%02x]", link->psr_settings.psr_version);
+ seq_puts(m, "\n");
+
+ return 0;
+}
+
/*
* Returns the current and maximum output bpc for the connector.
* Example usage: cat /sys/kernel/debug/dri/0/DP-1/output_bpc
@@ -1243,7 +1285,7 @@ static ssize_t trigger_hotplug(struct file *f, const char __user *buf,
dm_restore_drm_connector_state(dev, connector);
drm_modeset_unlock_all(dev);
- drm_kms_helper_hotplug_event(dev);
+ drm_kms_helper_connector_hotplug_event(connector);
} else if (param[0] == 0) {
if (!aconnector->dc_link)
goto unlock;
@@ -1265,7 +1307,7 @@ static ssize_t trigger_hotplug(struct file *f, const char __user *buf,
dm_restore_drm_connector_state(dev, connector);
drm_modeset_unlock_all(dev);
- drm_kms_helper_hotplug_event(dev);
+ drm_kms_helper_connector_hotplug_event(connector);
}
unlock:
@@ -2467,6 +2509,7 @@ DEFINE_SHOW_ATTRIBUTE(dp_lttpr_status);
DEFINE_SHOW_ATTRIBUTE(hdcp_sink_capability);
#endif
DEFINE_SHOW_ATTRIBUTE(internal_display);
+DEFINE_SHOW_ATTRIBUTE(psr_capability);
static const struct file_operations dp_dsc_clock_en_debugfs_fops = {
.owner = THIS_MODULE,
@@ -2712,6 +2755,138 @@ static const struct {
{"internal_display", &internal_display_fops}
};
+/*
+ * Returns supported customized link rates by this eDP panel.
+ * Example usage: cat /sys/kernel/debug/dri/0/eDP-x/ilr_setting
+ */
+static int edp_ilr_show(struct seq_file *m, void *unused)
+{
+ struct amdgpu_dm_connector *aconnector = to_amdgpu_dm_connector(m->private);
+ struct dc_link *link = aconnector->dc_link;
+ uint8_t supported_link_rates[16];
+ uint32_t link_rate_in_khz;
+ uint32_t entry = 0;
+ uint8_t dpcd_rev;
+
+ memset(supported_link_rates, 0, sizeof(supported_link_rates));
+ dm_helpers_dp_read_dpcd(link->ctx, link, DP_SUPPORTED_LINK_RATES,
+ supported_link_rates, sizeof(supported_link_rates));
+
+ dpcd_rev = link->dpcd_caps.dpcd_rev.raw;
+
+ if (dpcd_rev >= DP_DPCD_REV_13 &&
+ (supported_link_rates[entry+1] != 0 || supported_link_rates[entry] != 0)) {
+
+ for (entry = 0; entry < 16; entry += 2) {
+ link_rate_in_khz = (supported_link_rates[entry+1] * 0x100 +
+ supported_link_rates[entry]) * 200;
+ seq_printf(m, "[%d] %d kHz\n", entry/2, link_rate_in_khz);
+ }
+ } else {
+ seq_printf(m, "ILR is not supported by this eDP panel.\n");
+ }
+
+ return 0;
+}
+
+/*
+ * Set supported customized link rate to eDP panel.
+ *
+ * echo <lane_count> <link_rate option> > ilr_setting
+ *
+ * for example, supported ILR : [0] 1620000 kHz [1] 2160000 kHz [2] 2430000 kHz ...
+ * echo 4 1 > /sys/kernel/debug/dri/0/eDP-x/ilr_setting
+ * to set 4 lanes and 2.16 GHz
+ */
+static ssize_t edp_ilr_write(struct file *f, const char __user *buf,
+ size_t size, loff_t *pos)
+{
+ struct amdgpu_dm_connector *connector = file_inode(f)->i_private;
+ struct dc_link *link = connector->dc_link;
+ struct amdgpu_device *adev = drm_to_adev(connector->base.dev);
+ struct dc *dc = (struct dc *)link->dc;
+ struct dc_link_settings prefer_link_settings;
+ char *wr_buf = NULL;
+ const uint32_t wr_buf_size = 40;
+ /* 0: lane_count; 1: link_rate */
+ int max_param_num = 2;
+ uint8_t param_nums = 0;
+ long param[2];
+ bool valid_input = true;
+
+ if (size == 0)
+ return -EINVAL;
+
+ wr_buf = kcalloc(wr_buf_size, sizeof(char), GFP_KERNEL);
+ if (!wr_buf)
+ return -ENOMEM;
+
+ if (parse_write_buffer_into_params(wr_buf, wr_buf_size,
+ (long *)param, buf,
+ max_param_num,
+ ¶m_nums)) {
+ kfree(wr_buf);
+ return -EINVAL;
+ }
+
+ if (param_nums <= 0) {
+ kfree(wr_buf);
+ return -EINVAL;
+ }
+
+ switch (param[0]) {
+ case LANE_COUNT_ONE:
+ case LANE_COUNT_TWO:
+ case LANE_COUNT_FOUR:
+ break;
+ default:
+ valid_input = false;
+ break;
+ }
+
+ if (param[1] >= link->dpcd_caps.edp_supported_link_rates_count)
+ valid_input = false;
+
+ if (!valid_input) {
+ kfree(wr_buf);
+ DRM_DEBUG_DRIVER("Invalid Input value. No HW will be programmed\n");
+ prefer_link_settings.use_link_rate_set = false;
+ dc_link_set_preferred_training_settings(dc, NULL, NULL, link, true);
+ return size;
+ }
+
+ /* save user force lane_count, link_rate to preferred settings
+ * spread spectrum will not be changed
+ */
+ prefer_link_settings.link_spread = link->cur_link_settings.link_spread;
+ prefer_link_settings.lane_count = param[0];
+ prefer_link_settings.use_link_rate_set = true;
+ prefer_link_settings.link_rate_set = param[1];
+ prefer_link_settings.link_rate = link->dpcd_caps.edp_supported_link_rates[param[1]];
+
+ mutex_lock(&adev->dm.dc_lock);
+ dc_link_set_preferred_training_settings(dc, &prefer_link_settings,
+ NULL, link, false);
+ mutex_unlock(&adev->dm.dc_lock);
+
+ kfree(wr_buf);
+ return size;
+}
+
+static int edp_ilr_open(struct inode *inode, struct file *file)
+{
+ return single_open(file, edp_ilr_show, inode->i_private);
+}
+
+static const struct file_operations edp_ilr_debugfs_fops = {
+ .owner = THIS_MODULE,
+ .open = edp_ilr_open,
+ .read = seq_read,
+ .llseek = seq_lseek,
+ .release = single_release,
+ .write = edp_ilr_write
+};
+
void connector_debugfs_init(struct amdgpu_dm_connector *connector)
{
int i;
@@ -2726,11 +2901,14 @@ void connector_debugfs_init(struct amdgpu_dm_connector *connector)
}
}
if (connector->base.connector_type == DRM_MODE_CONNECTOR_eDP) {
+ debugfs_create_file_unsafe("psr_capability", 0444, dir, connector, &psr_capability_fops);
debugfs_create_file_unsafe("psr_state", 0444, dir, connector, &psr_fops);
debugfs_create_file("amdgpu_current_backlight_pwm", 0444, dir, connector,
¤t_backlight_fops);
debugfs_create_file("amdgpu_target_backlight_pwm", 0444, dir, connector,
&target_backlight_fops);
+ debugfs_create_file("ilr_setting", 0644, dir, connector,
+ &edp_ilr_debugfs_fops);
}
for (i = 0; i < ARRAY_SIZE(connector_debugfs_entries); i++) {
@@ -2909,10 +3087,13 @@ static int crc_win_update_set(void *data, u64 val)
struct amdgpu_device *adev = drm_to_adev(new_crtc->dev);
struct crc_rd_work *crc_rd_wrk = adev->dm.crc_rd_wrk;
+ if (!crc_rd_wrk)
+ return 0;
+
if (val) {
spin_lock_irq(&adev_to_drm(adev)->event_lock);
spin_lock_irq(&crc_rd_wrk->crc_rd_work_lock);
- if (crc_rd_wrk && crc_rd_wrk->crtc) {
+ if (crc_rd_wrk->crtc) {
old_crtc = crc_rd_wrk->crtc;
old_acrtc = to_amdgpu_crtc(old_crtc);
}
@@ -3190,6 +3371,32 @@ static int disable_hpd_get(void *data, u64 *val)
DEFINE_DEBUGFS_ATTRIBUTE(disable_hpd_ops, disable_hpd_get,
disable_hpd_set, "%llu\n");
+#if defined(CONFIG_DRM_AMD_DC_DCN)
+/*
+ * Temporary w/a to force sst sequence in M42D DP2 mst receiver
+ * Example usage: echo 1 > /sys/kernel/debug/dri/0/amdgpu_dm_dp_set_mst_en_for_sst
+ */
+static int dp_force_sst_set(void *data, u64 val)
+{
+ struct amdgpu_device *adev = data;
+
+ adev->dm.dc->debug.set_mst_en_for_sst = val;
+
+ return 0;
+}
+
+static int dp_force_sst_get(void *data, u64 *val)
+{
+ struct amdgpu_device *adev = data;
+
+ *val = adev->dm.dc->debug.set_mst_en_for_sst;
+
+ return 0;
+}
+DEFINE_DEBUGFS_ATTRIBUTE(dp_set_mst_en_for_sst_ops, dp_force_sst_get,
+ dp_force_sst_set, "%llu\n");
+#endif
+
/*
* Sets the DC visual confirm debug option from the given string.
* Example usage: echo 1 > /sys/kernel/debug/dri/0/amdgpu_visual_confirm
@@ -3299,6 +3506,10 @@ void dtn_debugfs_init(struct amdgpu_device *adev)
adev, &mst_topo_fops);
debugfs_create_file("amdgpu_dm_dtn_log", 0644, root, adev,
&dtn_log_fops);
+#if defined(CONFIG_DRM_AMD_DC_DCN)
+ debugfs_create_file("amdgpu_dm_dp_set_mst_en_for_sst", 0644, root, adev,
+ &dp_set_mst_en_for_sst_ops);
+#endif
debugfs_create_file_unsafe("amdgpu_dm_visual_confirm", 0644, root, adev,
&visual_confirm_fops);
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_helpers.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_helpers.c
index 8cbeeb7..29f07c2 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_helpers.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_helpers.c
@@ -83,16 +83,17 @@ static int amdgpu_dm_patch_edid_caps(struct dc_edid_caps *edid_caps)
* void
* */
enum dc_edid_status dm_helpers_parse_edid_caps(
- struct dc_context *ctx,
+ struct dc_link *link,
const struct dc_edid *edid,
struct dc_edid_caps *edid_caps)
{
+ struct amdgpu_dm_connector *aconnector = link->priv;
+ struct drm_connector *connector = &aconnector->base;
struct edid *edid_buf = (struct edid *) edid->raw_edid;
struct cea_sad *sads;
int sad_count = -1;
int sadb_count = -1;
int i = 0;
- int j = 0;
uint8_t *sadb = NULL;
enum dc_edid_status result = EDID_OK;
@@ -111,23 +112,11 @@ enum dc_edid_status dm_helpers_parse_edid_caps(
edid_caps->manufacture_week = edid_buf->mfg_week;
edid_caps->manufacture_year = edid_buf->mfg_year;
- /* One of the four detailed_timings stores the monitor name. It's
- * stored in an array of length 13. */
- for (i = 0; i < 4; i++) {
- if (edid_buf->detailed_timings[i].data.other_data.type == 0xfc) {
- while (j < 13 && edid_buf->detailed_timings[i].data.other_data.data.str.str[j]) {
- if (edid_buf->detailed_timings[i].data.other_data.data.str.str[j] == '\n')
- break;
+ drm_edid_get_monitor_name(edid_buf,
+ edid_caps->display_name,
+ AUDIO_INFO_DISPLAY_NAME_SIZE_IN_CHARS);
- edid_caps->display_name[j] =
- edid_buf->detailed_timings[i].data.other_data.data.str.str[j];
- j++;
- }
- }
- }
-
- edid_caps->edid_hdmi = drm_detect_hdmi_monitor(
- (struct edid *) edid->raw_edid);
+ edid_caps->edid_hdmi = connector->display_info.is_hdmi;
sad_count = drm_edid_to_sad((struct edid *) edid->raw_edid, &sads);
if (sad_count <= 0)
@@ -584,9 +573,18 @@ bool dm_helpers_dp_write_dsc_enable(
ret = drm_dp_dpcd_write(aconnector->dsc_aux, DP_DSC_ENABLE, &enable_dsc, 1);
}
- if (stream->signal == SIGNAL_TYPE_DISPLAY_PORT) {
- ret = dm_helpers_dp_write_dpcd(ctx, stream->link, DP_DSC_ENABLE, &enable_dsc, 1);
- DC_LOG_DC("Send DSC %s to sst display\n", enable_dsc ? "enable" : "disable");
+ if (stream->signal == SIGNAL_TYPE_DISPLAY_PORT || stream->signal == SIGNAL_TYPE_EDP) {
+#if defined(CONFIG_DRM_AMD_DC_DCN)
+ if (stream->sink->link->dpcd_caps.dongle_type == DISPLAY_DONGLE_NONE) {
+#endif
+ ret = dm_helpers_dp_write_dpcd(ctx, stream->link, DP_DSC_ENABLE, &enable_dsc, 1);
+ DC_LOG_DC("Send DSC %s to SST RX\n", enable_dsc ? "enable" : "disable");
+#if defined(CONFIG_DRM_AMD_DC_DCN)
+ } else if (stream->sink->link->dpcd_caps.dongle_type == DISPLAY_DONGLE_DP_HDMI_CONVERTER) {
+ ret = dm_helpers_dp_write_dpcd(ctx, stream->link, DP_DSC_ENABLE, &enable_dsc, 1);
+ DC_LOG_DC("Send DSC %s to DP-HDMI PCON\n", enable_dsc ? "enable" : "disable");
+ }
+#endif
}
return (ret > 0);
@@ -650,14 +648,8 @@ enum dc_edid_status dm_helpers_read_local_edid(
/* We don't need the original edid anymore */
kfree(edid);
- /* connector->display_info will be parsed from EDID and saved
- * into drm_connector->display_info from edid by call stack
- * below:
- * drm_parse_ycbcr420_deep_color_info
- * drm_parse_hdmi_forum_vsdb
- * drm_parse_cea_ext
- * drm_add_display_info
- * drm_connector_update_edid_property
+ /* connector->display_info is parsed from EDID and saved
+ * into drm_connector->display_info
*
* drm_connector->display_info will be used by amdgpu_dm funcs,
* like fill_stream_properties_from_drm_display_mode
@@ -665,7 +657,7 @@ enum dc_edid_status dm_helpers_read_local_edid(
amdgpu_dm_update_connector_after_detect(aconnector);
edid_status = dm_helpers_parse_edid_caps(
- ctx,
+ link,
&sink->dc_edid,
&sink->edid_caps);
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_psr.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_psr.c
index c022e56..c510638 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_psr.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_psr.c
@@ -26,6 +26,73 @@
#include "amdgpu_dm_psr.h"
#include "dc.h"
#include "dm_helpers.h"
+#include "amdgpu_dm.h"
+
+static bool link_get_psr_caps(struct dc_link *link)
+{
+ uint8_t psr_dpcd_data[EDP_PSR_RECEIVER_CAP_SIZE];
+ uint8_t edp_rev_dpcd_data;
+
+
+
+ if (!dm_helpers_dp_read_dpcd(NULL, link, DP_PSR_SUPPORT,
+ psr_dpcd_data, sizeof(psr_dpcd_data)))
+ return false;
+
+ if (!dm_helpers_dp_read_dpcd(NULL, link, DP_EDP_DPCD_REV,
+ &edp_rev_dpcd_data, sizeof(edp_rev_dpcd_data)))
+ return false;
+
+ link->dpcd_caps.psr_caps.psr_version = psr_dpcd_data[0];
+ link->dpcd_caps.psr_caps.edp_revision = edp_rev_dpcd_data;
+
+#ifdef CONFIG_DRM_AMD_DC_DCN
+ if (link->dpcd_caps.psr_caps.psr_version > 0x1) {
+ uint8_t alpm_dpcd_data;
+ uint8_t su_granularity_dpcd_data;
+
+ if (!dm_helpers_dp_read_dpcd(NULL, link, DP_RECEIVER_ALPM_CAP,
+ &alpm_dpcd_data, sizeof(alpm_dpcd_data)))
+ return false;
+
+ if (!dm_helpers_dp_read_dpcd(NULL, link, DP_PSR2_SU_Y_GRANULARITY,
+ &su_granularity_dpcd_data, sizeof(su_granularity_dpcd_data)))
+ return false;
+
+ link->dpcd_caps.psr_caps.y_coordinate_required = psr_dpcd_data[1] & DP_PSR2_SU_Y_COORDINATE_REQUIRED;
+ link->dpcd_caps.psr_caps.su_granularity_required = psr_dpcd_data[1] & DP_PSR2_SU_GRANULARITY_REQUIRED;
+
+ link->dpcd_caps.psr_caps.alpm_cap = alpm_dpcd_data & DP_ALPM_CAP;
+ link->dpcd_caps.psr_caps.standby_support = alpm_dpcd_data & (1 << 1);
+
+ link->dpcd_caps.psr_caps.su_y_granularity = su_granularity_dpcd_data;
+ }
+#endif
+ return true;
+}
+
+#ifdef CONFIG_DRM_AMD_DC_DCN
+static bool link_supports_psrsu(struct dc_link *link)
+{
+ struct dc *dc = link->ctx->dc;
+
+ if (!dc->caps.dmcub_support)
+ return false;
+
+ if (dc->ctx->dce_version < DCN_VERSION_3_1)
+ return false;
+
+ if (!link->dpcd_caps.psr_caps.alpm_cap ||
+ !link->dpcd_caps.psr_caps.y_coordinate_required)
+ return false;
+
+ if (link->dpcd_caps.psr_caps.su_granularity_required &&
+ !link->dpcd_caps.psr_caps.su_y_granularity)
+ return false;
+
+ return true;
+}
+#endif
/*
* amdgpu_dm_set_psr_caps() - set link psr capabilities
@@ -34,26 +101,34 @@
*/
void amdgpu_dm_set_psr_caps(struct dc_link *link)
{
- uint8_t dpcd_data[EDP_PSR_RECEIVER_CAP_SIZE];
-
if (!(link->connector_signal & SIGNAL_TYPE_EDP))
return;
+
if (link->type == dc_connection_none)
return;
- if (dm_helpers_dp_read_dpcd(NULL, link, DP_PSR_SUPPORT,
- dpcd_data, sizeof(dpcd_data))) {
- link->dpcd_caps.psr_caps.psr_version = dpcd_data[0];
- if (dpcd_data[0] == 0) {
- link->psr_settings.psr_version = DC_PSR_VERSION_UNSUPPORTED;
- link->psr_settings.psr_feature_enabled = false;
- } else {
- link->psr_settings.psr_version = DC_PSR_VERSION_1;
- link->psr_settings.psr_feature_enabled = true;
- }
-
- DRM_INFO("PSR support:%d\n", link->psr_settings.psr_feature_enabled);
+ if (!link_get_psr_caps(link)) {
+ DRM_ERROR("amdgpu: Failed to read PSR Caps!\n");
+ return;
}
+
+ if (link->dpcd_caps.psr_caps.psr_version == 0) {
+ link->psr_settings.psr_version = DC_PSR_VERSION_UNSUPPORTED;
+ link->psr_settings.psr_feature_enabled = false;
+
+ } else {
+#ifdef CONFIG_DRM_AMD_DC_DCN
+ if (link_supports_psrsu(link))
+ link->psr_settings.psr_version = DC_PSR_VERSION_SU_1;
+ else
+#endif
+ link->psr_settings.psr_version = DC_PSR_VERSION_1;
+
+ link->psr_settings.psr_feature_enabled = true;
+ }
+
+ DRM_INFO("PSR support:%d\n", link->psr_settings.psr_feature_enabled);
+
}
/*
diff --git a/drivers/gpu/drm/amd/display/dc/bios/bios_parser2.c b/drivers/gpu/drm/amd/display/dc/bios/bios_parser2.c
index a4bef43..1e385d5 100644
--- a/drivers/gpu/drm/amd/display/dc/bios/bios_parser2.c
+++ b/drivers/gpu/drm/amd/display/dc/bios/bios_parser2.c
@@ -2995,7 +2995,7 @@ static bool bios_parser2_construct(
&bp->object_info_tbl.revision);
if (bp->object_info_tbl.revision.major == 1
- && bp->object_info_tbl.revision.minor >= 4) {
+ && bp->object_info_tbl.revision.minor == 4) {
struct display_object_info_table_v1_4 *tbl_v1_4;
tbl_v1_4 = GET_IMAGE(struct display_object_info_table_v1_4,
@@ -3004,8 +3004,10 @@ static bool bios_parser2_construct(
return false;
bp->object_info_tbl.v1_4 = tbl_v1_4;
- } else
+ } else {
+ ASSERT(0);
return false;
+ }
dal_firmware_parser_init_cmd_tbl(bp);
dal_bios_parser_init_cmd_tbl_helper2(&bp->cmd_helper, dce_version);
diff --git a/drivers/gpu/drm/amd/display/dc/calcs/dcn_calcs.c b/drivers/gpu/drm/amd/display/dc/calcs/dcn_calcs.c
index 6b248cd..ec19678 100644
--- a/drivers/gpu/drm/amd/display/dc/calcs/dcn_calcs.c
+++ b/drivers/gpu/drm/amd/display/dc/calcs/dcn_calcs.c
@@ -739,7 +739,9 @@ static void hack_bounding_box(struct dcn_bw_internal_vars *v,
hack_force_pipe_split(v, context->streams[0]->timing.pix_clk_100hz);
}
-unsigned int get_highest_allowed_voltage_level(uint32_t chip_family, uint32_t hw_internal_rev, uint32_t pci_revision_id)
+static unsigned int get_highest_allowed_voltage_level(uint32_t chip_family,
+ uint32_t hw_internal_rev,
+ uint32_t pci_revision_id)
{
/* for low power RV2 variants, the highest voltage level we want is 0 */
if ((chip_family == FAMILY_RV) &&
@@ -763,7 +765,7 @@ unsigned int get_highest_allowed_voltage_level(uint32_t chip_family, uint32_t hw
return 4;
}
-bool dcn_validate_bandwidth(
+bool dcn10_validate_bandwidth(
struct dc *dc,
struct dc_state *context,
bool fast_validate)
diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/clk_mgr.c b/drivers/gpu/drm/amd/display/dc/clk_mgr/clk_mgr.c
index 26f96ee..9200c8c 100644
--- a/drivers/gpu/drm/amd/display/dc/clk_mgr/clk_mgr.c
+++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/clk_mgr.c
@@ -308,8 +308,7 @@ void dc_destroy_clk_mgr(struct clk_mgr *clk_mgr_base)
case FAMILY_NV:
if (ASICREV_IS_SIENNA_CICHLID_P(clk_mgr_base->ctx->asic_id.hw_internal_rev)) {
dcn3_clk_mgr_destroy(clk_mgr);
- }
- if (ASICREV_IS_DIMGREY_CAVEFISH_P(clk_mgr_base->ctx->asic_id.hw_internal_rev)) {
+ } else if (ASICREV_IS_DIMGREY_CAVEFISH_P(clk_mgr_base->ctx->asic_id.hw_internal_rev)) {
dcn3_clk_mgr_destroy(clk_mgr);
}
if (ASICREV_IS_BEIGE_GOBY_P(clk_mgr_base->ctx->asic_id.hw_internal_rev)) {
diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn10/rv1_clk_mgr.c b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn10/rv1_clk_mgr.c
index 76ec8ec..60761ff 100644
--- a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn10/rv1_clk_mgr.c
+++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn10/rv1_clk_mgr.c
@@ -34,7 +34,7 @@
#include "rv1_clk_mgr_vbios_smu.h"
#include "rv1_clk_mgr_clk.h"
-void rv1_init_clocks(struct clk_mgr *clk_mgr)
+static void rv1_init_clocks(struct clk_mgr *clk_mgr)
{
memset(&(clk_mgr->clks), 0, sizeof(struct dc_clocks));
}
diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn10/rv1_clk_mgr_vbios_smu.c b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn10/rv1_clk_mgr_vbios_smu.c
index fe18bb9..06bab24 100644
--- a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn10/rv1_clk_mgr_vbios_smu.c
+++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn10/rv1_clk_mgr_vbios_smu.c
@@ -28,6 +28,8 @@
#include "reg_helper.h"
#include <linux/delay.h>
+#include "rv1_clk_mgr_vbios_smu.h"
+
#define MAX_INSTANCE 5
#define MAX_SEGMENT 5
diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn20/dcn20_clk_mgr.c b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn20/dcn20_clk_mgr.c
index 2108bff..9f35f2e 100644
--- a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn20/dcn20_clk_mgr.c
+++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn20/dcn20_clk_mgr.c
@@ -409,7 +409,7 @@ void dcn2_init_clocks(struct clk_mgr *clk_mgr)
clk_mgr->clks.prev_p_state_change_support = true;
}
-void dcn2_enable_pme_wa(struct clk_mgr *clk_mgr_base)
+static void dcn2_enable_pme_wa(struct clk_mgr *clk_mgr_base)
{
struct clk_mgr_internal *clk_mgr = TO_CLK_MGR_INTERNAL(clk_mgr_base);
struct pp_smu_funcs_nv *pp_smu = NULL;
diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn201/dcn201_clk_mgr.c b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn201/dcn201_clk_mgr.c
index db99502..fbdd0a9 100644
--- a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn201/dcn201_clk_mgr.c
+++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn201/dcn201_clk_mgr.c
@@ -74,42 +74,6 @@ static const struct clk_mgr_mask clk_mgr_mask = {
CLK_COMMON_MASK_SH_LIST_DCN201_BASE(_MASK)
};
-void dcn201_update_clocks_vbios(struct clk_mgr *clk_mgr,
- struct dc_state *context,
- bool safe_to_lower)
-{
- struct dc_clocks *new_clocks = &context->bw_ctx.bw.dcn.clk;
-
- bool update_dppclk = false;
- bool update_dispclk = false;
-
- if (should_set_clock(safe_to_lower, new_clocks->dppclk_khz, clk_mgr->clks.dppclk_khz)) {
- clk_mgr->clks.dppclk_khz = new_clocks->dppclk_khz;
- update_dppclk = true;
- }
-
- if (should_set_clock(safe_to_lower, new_clocks->dispclk_khz, clk_mgr->clks.dispclk_khz)) {
- clk_mgr->clks.dispclk_khz = new_clocks->dispclk_khz;
- update_dispclk = true;
- }
-
- if (update_dppclk || update_dispclk) {
- struct bp_set_dce_clock_parameters dce_clk_params;
- struct dc_bios *bp = clk_mgr->ctx->dc_bios;
-
- if (update_dispclk) {
- memset(&dce_clk_params, 0, sizeof(dce_clk_params));
- dce_clk_params.target_clock_frequency = new_clocks->dispclk_khz;
- dce_clk_params.pll_id = CLOCK_SOURCE_ID_DFS;
- dce_clk_params.clock_type = DCECLOCK_TYPE_DISPLAY_CLOCK;
- bp->funcs->set_dce_clock(bp, &dce_clk_params);
- }
- /* currently there is no DCECLOCK_TYPE_DPPCLK type defined in VBIOS interface.
- * vbios program DPPCLK to the same DispCLK limitation
- */
- }
-}
-
static void dcn201_init_clocks(struct clk_mgr *clk_mgr)
{
memset(&(clk_mgr->clks), 0, sizeof(struct dc_clocks));
@@ -126,10 +90,8 @@ static void dcn201_update_clocks(struct clk_mgr *clk_mgr_base,
struct clk_mgr_internal *clk_mgr = TO_CLK_MGR_INTERNAL(clk_mgr_base);
struct dc_clocks *new_clocks = &context->bw_ctx.bw.dcn.clk;
struct dc *dc = clk_mgr_base->ctx->dc;
- int display_count;
bool update_dppclk = false;
bool update_dispclk = false;
- bool enter_display_off = false;
bool dpp_clock_lowered = false;
bool force_reset = false;
bool p_state_change_support;
@@ -145,10 +107,7 @@ static void dcn201_update_clocks(struct clk_mgr *clk_mgr_base,
dcn2_read_clocks_from_hw_dentist(clk_mgr_base);
}
- display_count = clk_mgr_helper_get_active_display_cnt(dc, context);
-
- if (display_count == 0)
- enter_display_off = true;
+ clk_mgr_helper_get_active_display_cnt(dc, context);
if (should_set_clock(safe_to_lower, new_clocks->phyclk_khz, clk_mgr_base->clks.phyclk_khz))
clk_mgr_base->clks.phyclk_khz = new_clocks->phyclk_khz;
diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn21/rn_clk_mgr.c b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn21/rn_clk_mgr.c
index ac2d4c4..fbda423 100644
--- a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn21/rn_clk_mgr.c
+++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn21/rn_clk_mgr.c
@@ -56,9 +56,7 @@
/* TODO: evaluate how to lower or disable all dcn clocks in screen off case */
-int rn_get_active_display_cnt_wa(
- struct dc *dc,
- struct dc_state *context)
+static int rn_get_active_display_cnt_wa(struct dc *dc, struct dc_state *context)
{
int i, display_count;
bool tmds_present = false;
@@ -77,7 +75,8 @@ int rn_get_active_display_cnt_wa(
const struct dc_link *link = dc->links[i];
/* abusing the fact that the dig and phy are coupled to see if the phy is enabled */
- if (link->link_enc->funcs->is_dig_enabled(link->link_enc))
+ if (link->link_enc->funcs->is_dig_enabled &&
+ link->link_enc->funcs->is_dig_enabled(link->link_enc))
display_count++;
}
@@ -88,7 +87,7 @@ int rn_get_active_display_cnt_wa(
return display_count;
}
-void rn_set_low_power_state(struct clk_mgr *clk_mgr_base)
+static void rn_set_low_power_state(struct clk_mgr *clk_mgr_base)
{
struct clk_mgr_internal *clk_mgr = TO_CLK_MGR_INTERNAL(clk_mgr_base);
@@ -122,7 +121,7 @@ static void rn_update_clocks_update_dpp_dto(struct clk_mgr_internal *clk_mgr,
}
-void rn_update_clocks(struct clk_mgr *clk_mgr_base,
+static void rn_update_clocks(struct clk_mgr *clk_mgr_base,
struct dc_state *context,
bool safe_to_lower)
{
@@ -437,25 +436,14 @@ static void rn_dump_clk_registers(struct clk_state_registers_and_bypass *regs_an
}
}
-/* This function produce translated logical clk state values*/
-void rn_get_clk_states(struct clk_mgr *clk_mgr_base, struct clk_states *s)
-{
- struct clk_state_registers_and_bypass sb = { 0 };
- struct clk_log_info log_info = { 0 };
-
- rn_dump_clk_registers(&sb, clk_mgr_base, &log_info);
-
- s->dprefclk_khz = sb.dprefclk * 1000;
-}
-
-void rn_enable_pme_wa(struct clk_mgr *clk_mgr_base)
+static void rn_enable_pme_wa(struct clk_mgr *clk_mgr_base)
{
struct clk_mgr_internal *clk_mgr = TO_CLK_MGR_INTERNAL(clk_mgr_base);
rn_vbios_smu_enable_pme_wa(clk_mgr);
}
-void rn_init_clocks(struct clk_mgr *clk_mgr)
+static void rn_init_clocks(struct clk_mgr *clk_mgr)
{
memset(&(clk_mgr->clks), 0, sizeof(struct dc_clocks));
// Assumption is that boot state always supports pstate
diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn21/rn_clk_mgr_vbios_smu.c b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn21/rn_clk_mgr_vbios_smu.c
index 9f7eed6..8161a6a 100644
--- a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn21/rn_clk_mgr_vbios_smu.c
+++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn21/rn_clk_mgr_vbios_smu.c
@@ -33,6 +33,8 @@
#include "mp/mp_12_0_0_offset.h"
#include "mp/mp_12_0_0_sh_mask.h"
+#include "rn_clk_mgr_vbios_smu.h"
+
#define REG(reg_name) \
(MP0_BASE.instance[0].segment[mm ## reg_name ## _BASE_IDX] + mm ## reg_name)
@@ -86,7 +88,9 @@ static uint32_t rn_smu_wait_for_response(struct clk_mgr_internal *clk_mgr, unsig
}
-int rn_vbios_smu_send_msg_with_param(struct clk_mgr_internal *clk_mgr, unsigned int msg_id, unsigned int param)
+static int rn_vbios_smu_send_msg_with_param(struct clk_mgr_internal *clk_mgr,
+ unsigned int msg_id,
+ unsigned int param)
{
uint32_t result;
diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn30/dcn30_clk_mgr.c b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn30/dcn30_clk_mgr.c
index 1861a14..f977f29 100644
--- a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn30/dcn30_clk_mgr.c
+++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn30/dcn30_clk_mgr.c
@@ -252,6 +252,7 @@ static void dcn3_update_clocks(struct clk_mgr *clk_mgr_base,
bool update_dispclk = false;
bool enter_display_off = false;
bool dpp_clock_lowered = false;
+ bool update_pstate_unsupported_clk = false;
struct dmcu *dmcu = clk_mgr_base->ctx->dc->res_pool->dmcu;
bool force_reset = false;
bool update_uclk = false;
@@ -299,13 +300,28 @@ static void dcn3_update_clocks(struct clk_mgr *clk_mgr_base,
clk_mgr_base->clks.prev_p_state_change_support = clk_mgr_base->clks.p_state_change_support;
total_plane_count = clk_mgr_helper_get_active_plane_cnt(dc, context);
p_state_change_support = new_clocks->p_state_change_support || (total_plane_count == 0);
- if (should_update_pstate_support(safe_to_lower, p_state_change_support, clk_mgr_base->clks.p_state_change_support)) {
+
+ // invalidate the current P-State forced min in certain dc_mode_softmax situations
+ if (dc->clk_mgr->dc_mode_softmax_enabled && safe_to_lower && !p_state_change_support) {
+ if ((new_clocks->dramclk_khz <= dc->clk_mgr->bw_params->dc_mode_softmax_memclk * 1000) !=
+ (clk_mgr_base->clks.dramclk_khz <= dc->clk_mgr->bw_params->dc_mode_softmax_memclk * 1000))
+ update_pstate_unsupported_clk = true;
+ }
+
+ if (should_update_pstate_support(safe_to_lower, p_state_change_support, clk_mgr_base->clks.p_state_change_support) ||
+ update_pstate_unsupported_clk) {
clk_mgr_base->clks.p_state_change_support = p_state_change_support;
/* to disable P-State switching, set UCLK min = max */
- if (!clk_mgr_base->clks.p_state_change_support)
- dcn30_smu_set_hard_min_by_freq(clk_mgr, PPCLK_UCLK,
+ if (!clk_mgr_base->clks.p_state_change_support) {
+ if (dc->clk_mgr->dc_mode_softmax_enabled &&
+ new_clocks->dramclk_khz <= dc->clk_mgr->bw_params->dc_mode_softmax_memclk * 1000)
+ dcn30_smu_set_hard_min_by_freq(clk_mgr, PPCLK_UCLK,
+ dc->clk_mgr->bw_params->dc_mode_softmax_memclk);
+ else
+ dcn30_smu_set_hard_min_by_freq(clk_mgr, PPCLK_UCLK,
clk_mgr_base->bw_params->clk_table.entries[clk_mgr_base->bw_params->clk_table.num_entries - 1].memclk_mhz);
+ }
}
/* Always update saved value, even if new value not set due to P-State switching unsupported */
@@ -421,6 +437,24 @@ static void dcn3_set_hard_max_memclk(struct clk_mgr *clk_mgr_base)
clk_mgr_base->bw_params->clk_table.entries[clk_mgr_base->bw_params->clk_table.num_entries - 1].memclk_mhz);
}
+static void dcn3_set_max_memclk(struct clk_mgr *clk_mgr_base, unsigned int memclk_mhz)
+{
+ struct clk_mgr_internal *clk_mgr = TO_CLK_MGR_INTERNAL(clk_mgr_base);
+
+ if (!clk_mgr->smu_present)
+ return;
+
+ dcn30_smu_set_hard_max_by_freq(clk_mgr, PPCLK_UCLK, memclk_mhz);
+}
+static void dcn3_set_min_memclk(struct clk_mgr *clk_mgr_base, unsigned int memclk_mhz)
+{
+ struct clk_mgr_internal *clk_mgr = TO_CLK_MGR_INTERNAL(clk_mgr_base);
+
+ if (!clk_mgr->smu_present)
+ return;
+ dcn30_smu_set_hard_min_by_freq(clk_mgr, PPCLK_UCLK, memclk_mhz);
+}
+
/* Get current memclk states, update bounding box */
static void dcn3_get_memclk_states_from_smu(struct clk_mgr *clk_mgr_base)
{
@@ -436,6 +470,8 @@ static void dcn3_get_memclk_states_from_smu(struct clk_mgr *clk_mgr_base)
&num_levels);
clk_mgr_base->bw_params->clk_table.num_entries = num_levels ? num_levels : 1;
+ clk_mgr_base->bw_params->dc_mode_softmax_memclk = dcn30_smu_get_dc_mode_max_dpm_freq(clk_mgr, PPCLK_UCLK);
+
/* Refresh bounding box */
clk_mgr_base->ctx->dc->res_pool->funcs->update_bw_bounding_box(
clk_mgr->base.ctx->dc, clk_mgr_base->bw_params);
@@ -505,6 +541,8 @@ static struct clk_mgr_funcs dcn3_funcs = {
.notify_wm_ranges = dcn3_notify_wm_ranges,
.set_hard_min_memclk = dcn3_set_hard_min_memclk,
.set_hard_max_memclk = dcn3_set_hard_max_memclk,
+ .set_max_memclk = dcn3_set_max_memclk,
+ .set_min_memclk = dcn3_set_min_memclk,
.get_memclk_states_from_smu = dcn3_get_memclk_states_from_smu,
.are_clock_states_equal = dcn3_are_clock_states_equal,
.enable_pme_wa = dcn3_enable_pme_wa,
diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn301/dcn301_smu.c b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn301/dcn301_smu.c
index 6ea6426..d9920d9 100644
--- a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn301/dcn301_smu.c
+++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn301/dcn301_smu.c
@@ -88,9 +88,9 @@ static uint32_t dcn301_smu_wait_for_response(struct clk_mgr_internal *clk_mgr, u
return res_val;
}
-int dcn301_smu_send_msg_with_param(
- struct clk_mgr_internal *clk_mgr,
- unsigned int msg_id, unsigned int param)
+static int dcn301_smu_send_msg_with_param(struct clk_mgr_internal *clk_mgr,
+ unsigned int msg_id,
+ unsigned int param)
{
uint32_t result;
diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn301/vg_clk_mgr.c b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn301/vg_clk_mgr.c
index 3eee32f..48005de 100644
--- a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn301/vg_clk_mgr.c
+++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn301/vg_clk_mgr.c
@@ -89,9 +89,9 @@ static int vg_get_active_display_cnt_wa(
return display_count;
}
-void vg_update_clocks(struct clk_mgr *clk_mgr_base,
- struct dc_state *context,
- bool safe_to_lower)
+static void vg_update_clocks(struct clk_mgr *clk_mgr_base,
+ struct dc_state *context,
+ bool safe_to_lower)
{
struct clk_mgr_internal *clk_mgr = TO_CLK_MGR_INTERNAL(clk_mgr_base);
struct dc_clocks *new_clocks = &context->bw_ctx.bw.dcn.clk;
@@ -367,18 +367,6 @@ static void vg_dump_clk_registers(struct clk_state_registers_and_bypass *regs_an
}
}
-/* This function produce translated logical clk state values*/
-void vg_get_clk_states(struct clk_mgr *clk_mgr_base, struct clk_states *s)
-{
-
- struct clk_state_registers_and_bypass sb = { 0 };
- struct clk_log_info log_info = { 0 };
-
- vg_dump_clk_registers(&sb, clk_mgr_base, &log_info);
-
- s->dprefclk_khz = sb.dprefclk * 1000;
-}
-
static void vg_enable_pme_wa(struct clk_mgr *clk_mgr_base)
{
struct clk_mgr_internal *clk_mgr = TO_CLK_MGR_INTERNAL(clk_mgr_base);
@@ -386,7 +374,7 @@ static void vg_enable_pme_wa(struct clk_mgr *clk_mgr_base)
dcn301_smu_enable_pme_wa(clk_mgr);
}
-void vg_init_clocks(struct clk_mgr *clk_mgr)
+static void vg_init_clocks(struct clk_mgr *clk_mgr)
{
memset(&(clk_mgr->clks), 0, sizeof(struct dc_clocks));
// Assumption is that boot state always supports pstate
@@ -753,7 +741,7 @@ void vg_clk_mgr_construct(
sizeof(struct watermarks),
&clk_mgr->smu_wm_set.mc_address.quad_part);
- if (clk_mgr->smu_wm_set.wm_set == 0) {
+ if (!clk_mgr->smu_wm_set.wm_set) {
clk_mgr->smu_wm_set.wm_set = &dummy_wms;
clk_mgr->smu_wm_set.mc_address.quad_part = 0;
}
diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn31/dcn31_clk_mgr.c b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn31/dcn31_clk_mgr.c
index f4c9a45..4162ce4 100644
--- a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn31/dcn31_clk_mgr.c
+++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn31/dcn31_clk_mgr.c
@@ -66,7 +66,7 @@
#define TO_CLK_MGR_DCN31(clk_mgr)\
container_of(clk_mgr, struct clk_mgr_dcn31, base)
-int dcn31_get_active_display_cnt_wa(
+static int dcn31_get_active_display_cnt_wa(
struct dc *dc,
struct dc_state *context)
{
@@ -118,7 +118,7 @@ static void dcn31_disable_otg_wa(struct clk_mgr *clk_mgr_base, bool disable)
}
}
-static void dcn31_update_clocks(struct clk_mgr *clk_mgr_base,
+void dcn31_update_clocks(struct clk_mgr *clk_mgr_base,
struct dc_state *context,
bool safe_to_lower)
{
@@ -158,6 +158,7 @@ static void dcn31_update_clocks(struct clk_mgr *clk_mgr_base,
union display_idle_optimization_u idle_info = { 0 };
idle_info.idle_info.df_request_disabled = 1;
idle_info.idle_info.phy_ref_clk_off = 1;
+ idle_info.idle_info.s0i2_rdy = 1;
dcn31_smu_set_display_idle_optimization(clk_mgr, idle_info.data);
/* update power state */
clk_mgr_base->clks.pwr_state = DCN_PWR_STATE_LOW_POWER;
@@ -284,7 +285,7 @@ static void dcn31_enable_pme_wa(struct clk_mgr *clk_mgr_base)
dcn31_smu_enable_pme_wa(clk_mgr);
}
-static void dcn31_init_clocks(struct clk_mgr *clk_mgr)
+void dcn31_init_clocks(struct clk_mgr *clk_mgr)
{
memset(&(clk_mgr->clks), 0, sizeof(struct dc_clocks));
// Assumption is that boot state always supports pstate
@@ -294,7 +295,7 @@ static void dcn31_init_clocks(struct clk_mgr *clk_mgr)
clk_mgr->clks.zstate_support = DCN_ZSTATE_SUPPORT_UNKNOWN;
}
-static bool dcn31_are_clock_states_equal(struct dc_clocks *a,
+bool dcn31_are_clock_states_equal(struct dc_clocks *a,
struct dc_clocks *b)
{
if (a->dispclk_khz != b->dispclk_khz)
@@ -540,10 +541,9 @@ static unsigned int find_clk_for_voltage(
return clock;
}
-void dcn31_clk_mgr_helper_populate_bw_params(
- struct clk_mgr_internal *clk_mgr,
- struct integrated_info *bios_info,
- const DpmClocks_t *clock_table)
+static void dcn31_clk_mgr_helper_populate_bw_params(struct clk_mgr_internal *clk_mgr,
+ struct integrated_info *bios_info,
+ const DpmClocks_t *clock_table)
{
int i, j;
struct clk_bw_params *bw_params = clk_mgr->base.bw_params;
diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn31/dcn31_clk_mgr.h b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn31/dcn31_clk_mgr.h
index f8f1005..961b10a 100644
--- a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn31/dcn31_clk_mgr.h
+++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn31/dcn31_clk_mgr.h
@@ -39,6 +39,13 @@ struct clk_mgr_dcn31 {
struct dcn31_smu_watermark_set smu_wm_set;
};
+bool dcn31_are_clock_states_equal(struct dc_clocks *a,
+ struct dc_clocks *b);
+void dcn31_init_clocks(struct clk_mgr *clk_mgr);
+void dcn31_update_clocks(struct clk_mgr *clk_mgr_base,
+ struct dc_state *context,
+ bool safe_to_lower);
+
void dcn31_clk_mgr_construct(struct dc_context *ctx,
struct clk_mgr_dcn31 *clk_mgr,
struct pp_smu_funcs *pp_smu,
diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn31/dcn31_smu.c b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn31/dcn31_smu.c
index 8c2b77e..b7ace23 100644
--- a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn31/dcn31_smu.c
+++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn31/dcn31_smu.c
@@ -95,9 +95,9 @@ static uint32_t dcn31_smu_wait_for_response(struct clk_mgr_internal *clk_mgr, un
return res_val;
}
-int dcn31_smu_send_msg_with_param(
- struct clk_mgr_internal *clk_mgr,
- unsigned int msg_id, unsigned int param)
+static int dcn31_smu_send_msg_with_param(struct clk_mgr_internal *clk_mgr,
+ unsigned int msg_id,
+ unsigned int param)
{
uint32_t result;
diff --git a/drivers/gpu/drm/amd/display/dc/core/dc.c b/drivers/gpu/drm/amd/display/dc/core/dc.c
index 0ded4de..01c8849 100644
--- a/drivers/gpu/drm/amd/display/dc/core/dc.c
+++ b/drivers/gpu/drm/amd/display/dc/core/dc.c
@@ -221,9 +221,9 @@ static bool create_links(
link = link_create(&link_init_params);
if (link) {
- dc->links[dc->link_count] = link;
- link->dc = dc;
- ++dc->link_count;
+ dc->links[dc->link_count] = link;
+ link->dc = dc;
+ ++dc->link_count;
}
}
@@ -274,24 +274,6 @@ static bool create_links(
goto failed_alloc;
}
-#if defined(CONFIG_DRM_AMD_DC_DCN)
- if (IS_FPGA_MAXIMUS_DC(dc->ctx->dce_environment) &&
- dc->caps.dp_hpo &&
- link->dc->res_pool->res_cap->num_hpo_dp_link_encoder > 0) {
- /* FPGA case - Allocate HPO DP link encoder */
- if (i < link->dc->res_pool->res_cap->num_hpo_dp_link_encoder) {
- link->hpo_dp_link_enc = link->dc->res_pool->hpo_dp_link_enc[i];
-
- if (link->hpo_dp_link_enc == NULL) {
- BREAK_TO_DEBUGGER();
- goto failed_alloc;
- }
- link->hpo_dp_link_enc->hpd_source = link->link_enc->hpd_source;
- link->hpo_dp_link_enc->transmitter = link->link_enc->transmitter;
- }
- }
-#endif
-
link->link_status.dpcd_caps = &link->dpcd_caps;
enc_init.ctx = dc->ctx;
@@ -808,6 +790,10 @@ void dc_stream_set_static_screen_params(struct dc *dc,
static void dc_destruct(struct dc *dc)
{
+ // reset link encoder assignment table on destruct
+ if (dc->res_pool && dc->res_pool->funcs->link_encs_assign)
+ link_enc_cfg_init(dc, dc->current_state);
+
if (dc->current_state) {
dc_release_state(dc->current_state);
dc->current_state = NULL;
@@ -1016,8 +1002,6 @@ static bool dc_construct(struct dc *dc,
goto fail;
}
- dc_resource_state_construct(dc, dc->current_state);
-
if (!create_links(dc, init_params->num_virtual_links))
goto fail;
@@ -1027,8 +1011,7 @@ static bool dc_construct(struct dc *dc,
if (!create_link_encoders(dc))
goto fail;
- /* Initialise DIG link encoder resource tracking variables. */
- link_enc_cfg_init(dc, dc->current_state);
+ dc_resource_state_construct(dc, dc->current_state);
return true;
@@ -1830,6 +1813,19 @@ bool dc_commit_state(struct dc *dc, struct dc_state *context)
dc_stream_log(dc, stream);
}
+ /*
+ * Previous validation was perfomred with fast_validation = true and
+ * the full DML state required for hardware programming was skipped.
+ *
+ * Re-validate here to calculate these parameters / watermarks.
+ */
+ result = dc_validate_global_state(dc, context, false);
+ if (result != DC_OK) {
+ DC_LOG_ERROR("DC commit global validation failure: %s (%d)",
+ dc_status_to_str(result), result);
+ return result;
+ }
+
result = dc_commit_state_no_check(dc, context);
return (result == DC_OK);
@@ -2870,7 +2866,8 @@ static void commit_planes_for_stream(struct dc *dc,
#endif
if ((update_type != UPDATE_TYPE_FAST) && stream->update_flags.bits.dsc_changed)
- if (top_pipe_to_program->stream_res.tg->funcs->lock_doublebuffer_enable) {
+ if (top_pipe_to_program &&
+ top_pipe_to_program->stream_res.tg->funcs->lock_doublebuffer_enable) {
if (should_use_dmub_lock(stream->link)) {
union dmub_hw_lock_flags hw_locks = { 0 };
struct dmub_hw_lock_inst_flags inst_flags = { 0 };
@@ -2979,12 +2976,12 @@ static void commit_planes_for_stream(struct dc *dc,
#ifdef CONFIG_DRM_AMD_DC_DCN
if (dc->debug.validate_dml_output) {
for (i = 0; i < dc->res_pool->pipe_count; i++) {
- struct pipe_ctx cur_pipe = context->res_ctx.pipe_ctx[i];
- if (cur_pipe.stream == NULL)
+ struct pipe_ctx *cur_pipe = &context->res_ctx.pipe_ctx[i];
+ if (cur_pipe->stream == NULL)
continue;
- cur_pipe.plane_res.hubp->funcs->validate_dml_output(
- cur_pipe.plane_res.hubp, dc->ctx,
+ cur_pipe->plane_res.hubp->funcs->validate_dml_output(
+ cur_pipe->plane_res.hubp, dc->ctx,
&context->res_ctx.pipe_ctx[i].rq_regs,
&context->res_ctx.pipe_ctx[i].dlg_regs,
&context->res_ctx.pipe_ctx[i].ttu_regs);
@@ -3426,7 +3423,7 @@ struct dc_sink *dc_link_add_remote_sink(
goto fail_add_sink;
edid_status = dm_helpers_parse_edid_caps(
- link->ctx,
+ link,
&dc_sink->dc_edid,
&dc_sink->edid_caps);
@@ -3583,6 +3580,98 @@ void dc_lock_memory_clock_frequency(struct dc *dc)
core_link_enable_stream(dc->current_state, &dc->current_state->res_ctx.pipe_ctx[i]);
}
+static void blank_and_force_memclk(struct dc *dc, bool apply, unsigned int memclk_mhz)
+{
+ struct dc_state *context = dc->current_state;
+ struct hubp *hubp;
+ struct pipe_ctx *pipe;
+ int i;
+
+ for (i = 0; i < dc->res_pool->pipe_count; i++) {
+ pipe = &context->res_ctx.pipe_ctx[i];
+
+ if (pipe->stream != NULL) {
+ dc->hwss.disable_pixel_data(dc, pipe, true);
+
+ // wait for double buffer
+ pipe->stream_res.tg->funcs->wait_for_state(pipe->stream_res.tg, CRTC_STATE_VACTIVE);
+ pipe->stream_res.tg->funcs->wait_for_state(pipe->stream_res.tg, CRTC_STATE_VBLANK);
+ pipe->stream_res.tg->funcs->wait_for_state(pipe->stream_res.tg, CRTC_STATE_VACTIVE);
+
+ hubp = pipe->plane_res.hubp;
+ hubp->funcs->set_blank_regs(hubp, true);
+ }
+ }
+
+ dc->clk_mgr->funcs->set_max_memclk(dc->clk_mgr, memclk_mhz);
+ dc->clk_mgr->funcs->set_min_memclk(dc->clk_mgr, memclk_mhz);
+
+ for (i = 0; i < dc->res_pool->pipe_count; i++) {
+ pipe = &context->res_ctx.pipe_ctx[i];
+
+ if (pipe->stream != NULL) {
+ dc->hwss.disable_pixel_data(dc, pipe, false);
+
+ hubp = pipe->plane_res.hubp;
+ hubp->funcs->set_blank_regs(hubp, false);
+ }
+ }
+}
+
+
+/**
+ * dc_enable_dcmode_clk_limit() - lower clocks in dc (battery) mode
+ * @dc: pointer to dc of the dm calling this
+ * @enable: True = transition to DC mode, false = transition back to AC mode
+ *
+ * Some SoCs define additional clock limits when in DC mode, DM should
+ * invoke this function when the platform undergoes a power source transition
+ * so DC can apply/unapply the limit. This interface may be disruptive to
+ * the onscreen content.
+ *
+ * Context: Triggered by OS through DM interface, or manually by escape calls.
+ * Need to hold a dclock when doing so.
+ *
+ * Return: none (void function)
+ *
+ */
+void dc_enable_dcmode_clk_limit(struct dc *dc, bool enable)
+{
+ uint32_t hw_internal_rev = dc->ctx->asic_id.hw_internal_rev;
+ unsigned int softMax, maxDPM, funcMin;
+ bool p_state_change_support;
+
+ if (!ASICREV_IS_BEIGE_GOBY_P(hw_internal_rev))
+ return;
+
+ softMax = dc->clk_mgr->bw_params->dc_mode_softmax_memclk;
+ maxDPM = dc->clk_mgr->bw_params->clk_table.entries[dc->clk_mgr->bw_params->clk_table.num_entries - 1].memclk_mhz;
+ funcMin = (dc->clk_mgr->clks.dramclk_khz + 999) / 1000;
+ p_state_change_support = dc->clk_mgr->clks.p_state_change_support;
+
+ if (enable && !dc->clk_mgr->dc_mode_softmax_enabled) {
+ if (p_state_change_support) {
+ if (funcMin <= softMax)
+ dc->clk_mgr->funcs->set_max_memclk(dc->clk_mgr, softMax);
+ // else: No-Op
+ } else {
+ if (funcMin <= softMax)
+ blank_and_force_memclk(dc, true, softMax);
+ // else: No-Op
+ }
+ } else if (!enable && dc->clk_mgr->dc_mode_softmax_enabled) {
+ if (p_state_change_support) {
+ if (funcMin <= softMax)
+ dc->clk_mgr->funcs->set_max_memclk(dc->clk_mgr, maxDPM);
+ // else: No-Op
+ } else {
+ if (funcMin <= softMax)
+ blank_and_force_memclk(dc, true, maxDPM);
+ // else: No-Op
+ }
+ }
+ dc->clk_mgr->dc_mode_softmax_enabled = enable;
+}
bool dc_is_plane_eligible_for_idle_optimizations(struct dc *dc, struct dc_plane_state *plane,
struct dc_cursor_attributes *cursor_attr)
{
diff --git a/drivers/gpu/drm/amd/display/dc/core/dc_debug.c b/drivers/gpu/drm/amd/display/dc/core/dc_debug.c
index 21be2a6..6437625 100644
--- a/drivers/gpu/drm/amd/display/dc/core/dc_debug.c
+++ b/drivers/gpu/drm/amd/display/dc/core/dc_debug.c
@@ -422,6 +422,8 @@ char *dc_status_to_str(enum dc_status status)
return "The operation is not supported.";
case DC_UNSUPPORTED_VALUE:
return "The value specified is not supported.";
+ case DC_NO_LINK_ENC_RESOURCE:
+ return "No link encoder resource";
case DC_ERROR_UNEXPECTED:
return "Unexpected error";
}
diff --git a/drivers/gpu/drm/amd/display/dc/core/dc_link.c b/drivers/gpu/drm/amd/display/dc/core/dc_link.c
index c8457ba..dc1380b 100644
--- a/drivers/gpu/drm/amd/display/dc/core/dc_link.c
+++ b/drivers/gpu/drm/amd/display/dc/core/dc_link.c
@@ -66,31 +66,6 @@
/*******************************************************************************
* Private functions
******************************************************************************/
-#if defined(CONFIG_DRM_AMD_DC_DCN)
-static bool add_dp_hpo_link_encoder_to_link(struct dc_link *link)
-{
- struct hpo_dp_link_encoder *enc = resource_get_unused_hpo_dp_link_encoder(
- link->dc->res_pool);
-
- if (!link->hpo_dp_link_enc && enc) {
- link->hpo_dp_link_enc = enc;
- link->hpo_dp_link_enc->transmitter = link->link_enc->transmitter;
- link->hpo_dp_link_enc->hpd_source = link->link_enc->hpd_source;
- }
-
- return (link->hpo_dp_link_enc != NULL);
-}
-
-static void remove_dp_hpo_link_encoder_from_link(struct dc_link *link)
-{
- if (link->hpo_dp_link_enc) {
- link->hpo_dp_link_enc->hpd_source = HPD_SOURCEID_UNKNOWN;
- link->hpo_dp_link_enc->transmitter = TRANSMITTER_UNKNOWN;
- link->hpo_dp_link_enc = NULL;
- }
-}
-#endif
-
static void dc_link_destruct(struct dc_link *link)
{
int i;
@@ -118,12 +93,6 @@ static void dc_link_destruct(struct dc_link *link)
link->link_enc->funcs->destroy(&link->link_enc);
}
-#if defined(CONFIG_DRM_AMD_DC_DCN)
- if (link->hpo_dp_link_enc) {
- remove_dp_hpo_link_encoder_from_link(link);
- }
-#endif
-
if (link->local_sink)
dc_sink_release(link->local_sink);
@@ -270,10 +239,10 @@ bool dc_link_detect_sink(struct dc_link *link, enum dc_connection_type *type)
/* Link may not have physical HPD pin. */
if (link->ep_type != DISPLAY_ENDPOINT_PHY) {
- if (link->hpd_status)
- *type = dc_connection_single;
- else
+ if (link->is_hpd_pending || !link->hpd_status)
*type = dc_connection_none;
+ else
+ *type = dc_connection_single;
return true;
}
@@ -881,6 +850,7 @@ static bool dc_link_detect_helper(struct dc_link *link,
enum dc_connection_type pre_connection_type = dc_connection_none;
bool perform_dp_seamless_boot = false;
const uint32_t post_oui_delay = 30; // 30ms
+ struct link_resource link_res = { 0 };
DC_LOGGER_INIT(link->ctx->logger);
@@ -975,7 +945,10 @@ static bool dc_link_detect_helper(struct dc_link *link,
#if defined(CONFIG_DRM_AMD_DC_DCN)
if (dp_get_link_encoding_format(&link->reported_link_cap) == DP_128b_132b_ENCODING)
- add_dp_hpo_link_encoder_to_link(link);
+ link_res.hpo_dp_link_enc = resource_get_hpo_dp_link_enc_for_det_lt(
+ &link->dc->current_state->res_ctx,
+ link->dc->res_pool,
+ link);
#endif
if (link->type == dc_connection_mst_branch) {
@@ -986,7 +959,7 @@ static bool dc_link_detect_helper(struct dc_link *link,
* empty which leads to allocate_mst_payload() has "0"
* pbn_per_slot value leading to exception on dc_fixpt_div()
*/
- dp_verify_mst_link_cap(link);
+ dp_verify_mst_link_cap(link, &link_res);
/*
* This call will initiate MST topology discovery. Which
@@ -1150,6 +1123,7 @@ static bool dc_link_detect_helper(struct dc_link *link,
// verify link cap for SST non-seamless boot
if (!perform_dp_seamless_boot)
dp_verify_link_cap_with_retries(link,
+ &link_res,
&link->reported_link_cap,
LINK_TRAINING_MAX_VERIFY_RETRY);
} else {
@@ -1844,6 +1818,8 @@ static void enable_stream_features(struct pipe_ctx *pipe_ctx)
union down_spread_ctrl old_downspread;
union down_spread_ctrl new_downspread;
+ memset(&old_downspread, 0, sizeof(old_downspread));
+
core_link_read_dpcd(link, DP_DOWNSPREAD_CTRL,
&old_downspread.raw, sizeof(old_downspread));
@@ -2015,6 +1991,57 @@ static enum dc_status enable_link_dp_mst(
return enable_link_dp(state, pipe_ctx);
}
+void dc_link_blank_all_dp_displays(struct dc *dc)
+{
+ unsigned int i;
+ uint8_t dpcd_power_state = '\0';
+ enum dc_status status = DC_ERROR_UNEXPECTED;
+
+ for (i = 0; i < dc->link_count; i++) {
+ if ((dc->links[i]->connector_signal != SIGNAL_TYPE_DISPLAY_PORT) ||
+ (dc->links[i]->priv == NULL) || (dc->links[i]->local_sink == NULL))
+ continue;
+
+ /* DP 2.0 spec requires that we read LTTPR caps first */
+ dp_retrieve_lttpr_cap(dc->links[i]);
+ /* if any of the displays are lit up turn them off */
+ status = core_link_read_dpcd(dc->links[i], DP_SET_POWER,
+ &dpcd_power_state, sizeof(dpcd_power_state));
+
+ if (status == DC_OK && dpcd_power_state == DP_POWER_STATE_D0)
+ dc_link_blank_dp_stream(dc->links[i], true);
+ }
+
+}
+
+void dc_link_blank_dp_stream(struct dc_link *link, bool hw_init)
+{
+ unsigned int j;
+ struct dc *dc = link->ctx->dc;
+ enum signal_type signal = link->connector_signal;
+
+ if ((signal == SIGNAL_TYPE_EDP) ||
+ (signal == SIGNAL_TYPE_DISPLAY_PORT)) {
+ if (link->ep_type == DISPLAY_ENDPOINT_PHY &&
+ link->link_enc->funcs->get_dig_frontend &&
+ link->link_enc->funcs->is_dig_enabled(link->link_enc)) {
+ unsigned int fe = link->link_enc->funcs->get_dig_frontend(link->link_enc);
+
+ if (fe != ENGINE_ID_UNKNOWN)
+ for (j = 0; j < dc->res_pool->stream_enc_count; j++) {
+ if (fe == dc->res_pool->stream_enc[j]->id) {
+ dc->res_pool->stream_enc[j]->funcs->dp_blank(link,
+ dc->res_pool->stream_enc[j]);
+ break;
+ }
+ }
+ }
+
+ if ((!link->wa_flags.dp_keep_receiver_powered) || hw_init)
+ dp_receiver_power_ctrl(link, false);
+ }
+}
+
static bool get_ext_hdmi_settings(struct pipe_ctx *pipe_ctx,
enum engine_id eng_id,
struct ext_hdmi_settings *settings)
@@ -2452,7 +2479,8 @@ static void write_i2c_redriver_setting(
DC_LOG_DEBUG("Set redriver failed");
}
-static void disable_link(struct dc_link *link, enum signal_type signal)
+static void disable_link(struct dc_link *link, const struct link_resource *link_res,
+ enum signal_type signal)
{
/*
* TODO: implement call for dp_set_hw_test_pattern
@@ -2471,20 +2499,20 @@ static void disable_link(struct dc_link *link, enum signal_type signal)
struct dc_link_settings link_settings = link->cur_link_settings;
#endif
if (dc_is_dp_sst_signal(signal))
- dp_disable_link_phy(link, signal);
+ dp_disable_link_phy(link, link_res, signal);
else
- dp_disable_link_phy_mst(link, signal);
+ dp_disable_link_phy_mst(link, link_res, signal);
if (dc_is_dp_sst_signal(signal) ||
link->mst_stream_alloc_table.stream_count == 0) {
#if defined(CONFIG_DRM_AMD_DC_DCN)
if (dp_get_link_encoding_format(&link_settings) == DP_8b_10b_ENCODING) {
dp_set_fec_enable(link, false);
- dp_set_fec_ready(link, false);
+ dp_set_fec_ready(link, link_res, false);
}
#else
dp_set_fec_enable(link, false);
- dp_set_fec_ready(link, false);
+ dp_set_fec_ready(link, link_res, false);
#endif
}
} else {
@@ -2595,7 +2623,7 @@ static enum dc_status enable_link(
* new link settings.
*/
if (link->link_status.link_active) {
- disable_link(link, pipe_ctx->stream->signal);
+ disable_link(link, &pipe_ctx->link_res, pipe_ctx->stream->signal);
}
switch (pipe_ctx->stream->signal) {
@@ -2715,8 +2743,23 @@ static bool dp_active_dongle_validate_timing(
return false;
}
+#if defined(CONFIG_DRM_AMD_DC_DCN)
+ if (dongle_caps->dp_hdmi_frl_max_link_bw_in_kbps > 0) { // DP to HDMI FRL converter
+ struct dc_crtc_timing outputTiming = *timing;
+
+ if (timing->flags.DSC && !timing->dsc_cfg.is_frl)
+ /* DP input has DSC, HDMI FRL output doesn't have DSC, remove DSC from output timing */
+ outputTiming.flags.DSC = 0;
+ if (dc_bandwidth_in_kbps_from_timing(&outputTiming) > dongle_caps->dp_hdmi_frl_max_link_bw_in_kbps)
+ return false;
+ } else { // DP to HDMI TMDS converter
+ if (get_timing_pixel_clock_100hz(timing) > (dongle_caps->dp_hdmi_max_pixel_clk_in_khz * 10))
+ return false;
+ }
+#else
if (get_timing_pixel_clock_100hz(timing) > (dongle_caps->dp_hdmi_max_pixel_clk_in_khz * 10))
return false;
+#endif
#if defined(CONFIG_DRM_AMD_DC_DCN)
}
@@ -2962,7 +3005,7 @@ bool dc_link_set_psr_allow_active(struct dc_link *link, const bool *allow_active
link->psr_settings.psr_power_opt = *power_opts;
if (psr != NULL && link->psr_settings.psr_feature_enabled && psr->funcs->psr_set_power_opt)
- psr->funcs->psr_set_power_opt(psr, link->psr_settings.psr_power_opt);
+ psr->funcs->psr_set_power_opt(psr, link->psr_settings.psr_power_opt, panel_inst);
}
/* Enable or Disable PSR */
@@ -3350,11 +3393,12 @@ static void dc_log_vcp_x_y(const struct dc_link *link, struct fixed31_32 avg_tim
/*
* Payload allocation/deallocation for SST introduced in DP2.0
*/
-enum dc_status dc_link_update_sst_payload(struct pipe_ctx *pipe_ctx, bool allocate)
+static enum dc_status dc_link_update_sst_payload(struct pipe_ctx *pipe_ctx,
+ bool allocate)
{
struct dc_stream_state *stream = pipe_ctx->stream;
struct dc_link *link = stream->link;
- struct hpo_dp_link_encoder *hpo_dp_link_encoder = link->hpo_dp_link_enc;
+ struct hpo_dp_link_encoder *hpo_dp_link_encoder = pipe_ctx->link_res.hpo_dp_link_enc;
struct hpo_dp_stream_encoder *hpo_dp_stream_encoder = pipe_ctx->stream_res.hpo_dp_stream_enc;
struct link_mst_stream_allocation_table proposed_table = {0};
struct fixed31_32 avg_time_slots_per_mtp;
@@ -3436,7 +3480,7 @@ enum dc_status dc_link_allocate_mst_payload(struct pipe_ctx *pipe_ctx)
struct link_encoder *link_encoder = NULL;
struct stream_encoder *stream_encoder = pipe_ctx->stream_res.stream_enc;
#if defined(CONFIG_DRM_AMD_DC_DCN)
- struct hpo_dp_link_encoder *hpo_dp_link_encoder = link->hpo_dp_link_enc;
+ struct hpo_dp_link_encoder *hpo_dp_link_encoder = pipe_ctx->link_res.hpo_dp_link_enc;
struct hpo_dp_stream_encoder *hpo_dp_stream_encoder = pipe_ctx->stream_res.hpo_dp_stream_enc;
#endif
struct dp_mst_stream_allocation_table proposed_table = {0};
@@ -3766,7 +3810,7 @@ static enum dc_status deallocate_mst_payload(struct pipe_ctx *pipe_ctx)
struct link_encoder *link_encoder = NULL;
struct stream_encoder *stream_encoder = pipe_ctx->stream_res.stream_enc;
#if defined(CONFIG_DRM_AMD_DC_DCN)
- struct hpo_dp_link_encoder *hpo_dp_link_encoder = link->hpo_dp_link_enc;
+ struct hpo_dp_link_encoder *hpo_dp_link_encoder = pipe_ctx->link_res.hpo_dp_link_enc;
struct hpo_dp_stream_encoder *hpo_dp_stream_encoder = pipe_ctx->stream_res.hpo_dp_stream_enc;
#endif
struct dp_mst_stream_allocation_table proposed_table = {0};
@@ -3929,9 +3973,6 @@ static void update_psp_stream_config(struct pipe_ctx *pipe_ctx, bool dpms_off)
struct cp_psp *cp_psp = &pipe_ctx->stream->ctx->cp_psp;
#if defined(CONFIG_DRM_AMD_DC_DCN)
struct link_encoder *link_enc = NULL;
- struct dc_state *state = pipe_ctx->stream->ctx->dc->current_state;
- struct link_enc_assignment link_enc_assign;
- int i;
#endif
if (cp_psp && cp_psp->funcs.update_stream_config) {
@@ -3945,12 +3986,9 @@ static void update_psp_stream_config(struct pipe_ctx *pipe_ctx, bool dpms_off)
config.dig_be = pipe_ctx->stream->link->link_enc_hw_inst;
#if defined(CONFIG_DRM_AMD_DC_DCN)
config.stream_enc_idx = pipe_ctx->stream_res.stream_enc->id - ENGINE_ID_DIGA;
-
+
if (pipe_ctx->stream->link->ep_type == DISPLAY_ENDPOINT_PHY ||
pipe_ctx->stream->link->ep_type == DISPLAY_ENDPOINT_USB4_DPIA) {
- link_enc = pipe_ctx->stream->link->link_enc;
- config.dio_output_type = pipe_ctx->stream->link->ep_type;
- config.dio_output_idx = link_enc->transmitter - TRANSMITTER_UNIPHY_A;
if (pipe_ctx->stream->link->ep_type == DISPLAY_ENDPOINT_PHY)
link_enc = pipe_ctx->stream->link->link_enc;
else if (pipe_ctx->stream->link->ep_type == DISPLAY_ENDPOINT_USB4_DPIA)
@@ -3959,18 +3997,15 @@ static void update_psp_stream_config(struct pipe_ctx *pipe_ctx, bool dpms_off)
pipe_ctx->stream->ctx->dc,
pipe_ctx->stream);
}
+ ASSERT(link_enc);
+
// Initialize PHY ID with ABCDE - 01234 mapping except when it is B0
config.phy_idx = link_enc->transmitter - TRANSMITTER_UNIPHY_A;
- //look up the link_enc_assignment for the current pipe_ctx
- for (i = 0; i < state->stream_count; i++) {
- if (pipe_ctx->stream == state->streams[i]) {
- link_enc_assign = state->res_ctx.link_enc_cfg_ctx.link_enc_assignments[i];
- }
- }
// Add flag to guard new A0 DIG mapping
- if (pipe_ctx->stream->ctx->dc->enable_c20_dtm_b0 == true) {
- config.dig_be = link_enc_assign.eng_id;
+ if (pipe_ctx->stream->ctx->dc->enable_c20_dtm_b0 == true &&
+ pipe_ctx->stream->link->dc->ctx->dce_version == DCN_VERSION_3_1) {
+ config.dig_be = link_enc->preferred_engine;
config.dio_output_type = pipe_ctx->stream->link->ep_type;
config.dio_output_idx = link_enc->transmitter - TRANSMITTER_UNIPHY_A;
} else {
@@ -3982,10 +4017,8 @@ static void update_psp_stream_config(struct pipe_ctx *pipe_ctx, bool dpms_off)
if (pipe_ctx->stream->ctx->dc->enable_c20_dtm_b0 == true &&
link_enc->ctx->asic_id.hw_internal_rev == YELLOW_CARP_B0) {
if (pipe_ctx->stream->link->ep_type == DISPLAY_ENDPOINT_USB4_DPIA) {
- link_enc = link_enc_assign.stream->link_enc;
-
// enum ID 1-4 maps to DPIA PHY ID 0-3
- config.phy_idx = link_enc_assign.ep_id.link_id.enum_id - ENUM_ID_1;
+ config.phy_idx = pipe_ctx->stream->link->link_id.enum_id - ENUM_ID_1;
} else { // for non DPIA mode over B0, ABCDE maps to 01564
switch (link_enc->transmitter) {
@@ -4022,7 +4055,8 @@ static void update_psp_stream_config(struct pipe_ctx *pipe_ctx, bool dpms_off)
config.link_enc_idx = link_enc->transmitter - TRANSMITTER_UNIPHY_A;
if (is_dp_128b_132b_signal(pipe_ctx)) {
config.stream_enc_idx = pipe_ctx->stream_res.hpo_dp_stream_enc->id - ENGINE_ID_HPO_DP_0;
- config.link_enc_idx = pipe_ctx->stream->link->hpo_dp_link_enc->inst;
+
+ config.link_enc_idx = pipe_ctx->link_res.hpo_dp_link_enc->inst;
config.dp2_enabled = 1;
}
#endif
@@ -4053,7 +4087,7 @@ static void fpga_dp_hpo_enable_link_and_stream(struct dc_state *state, struct pi
stream->link->cur_link_settings = link_settings;
/* Enable clock, Configure lane count, and Enable Link Encoder*/
- enable_dp_hpo_output(stream->link, &stream->link->cur_link_settings);
+ enable_dp_hpo_output(stream->link, &pipe_ctx->link_res, &stream->link->cur_link_settings);
#ifdef DIAGS_BUILD
/* Workaround for FPGA HPO capture DP link data:
@@ -4103,12 +4137,12 @@ static void fpga_dp_hpo_enable_link_and_stream(struct dc_state *state, struct pi
proposed_table.stream_allocations[0].hpo_dp_stream_enc = pipe_ctx->stream_res.hpo_dp_stream_enc;
}
- stream->link->hpo_dp_link_enc->funcs->update_stream_allocation_table(
- stream->link->hpo_dp_link_enc,
+ pipe_ctx->link_res.hpo_dp_link_enc->funcs->update_stream_allocation_table(
+ pipe_ctx->link_res.hpo_dp_link_enc,
&proposed_table);
- stream->link->hpo_dp_link_enc->funcs->set_throttled_vcp_size(
- stream->link->hpo_dp_link_enc,
+ pipe_ctx->link_res.hpo_dp_link_enc->funcs->set_throttled_vcp_size(
+ pipe_ctx->link_res.hpo_dp_link_enc,
pipe_ctx->stream_res.hpo_dp_stream_enc->inst,
avg_time_slots_per_mtp);
@@ -4258,7 +4292,8 @@ void core_link_enable_stream(
/* eDP lit up by bios already, no need to enable again. */
if (pipe_ctx->stream->signal == SIGNAL_TYPE_EDP &&
apply_edp_fast_boot_optimization &&
- !pipe_ctx->stream->timing.flags.DSC) {
+ !pipe_ctx->stream->timing.flags.DSC &&
+ !pipe_ctx->next_odm_pipe) {
pipe_ctx->stream->dpms_off = false;
#if defined(CONFIG_DRM_AMD_DC_HDCP)
update_psp_stream_config(pipe_ctx, false);
@@ -4296,7 +4331,8 @@ void core_link_enable_stream(
if (status != DC_FAIL_DP_LINK_TRAINING ||
pipe_ctx->stream->signal == SIGNAL_TYPE_DISPLAY_PORT_MST) {
if (false == stream->link->link_status.link_active)
- disable_link(stream->link, pipe_ctx->stream->signal);
+ disable_link(stream->link, &pipe_ctx->link_res,
+ pipe_ctx->stream->signal);
BREAK_TO_DEBUGGER();
return;
}
@@ -4445,14 +4481,14 @@ void core_link_disable_stream(struct pipe_ctx *pipe_ctx)
* state machine.
* In DP2 or MST mode, our encoder will stay video active
*/
- disable_link(pipe_ctx->stream->link, pipe_ctx->stream->signal);
+ disable_link(pipe_ctx->stream->link, &pipe_ctx->link_res, pipe_ctx->stream->signal);
dc->hwss.disable_stream(pipe_ctx);
} else {
dc->hwss.disable_stream(pipe_ctx);
- disable_link(pipe_ctx->stream->link, pipe_ctx->stream->signal);
+ disable_link(pipe_ctx->stream->link, &pipe_ctx->link_res, pipe_ctx->stream->signal);
}
#else
- disable_link(pipe_ctx->stream->link, pipe_ctx->stream->signal);
+ disable_link(pipe_ctx->stream->link, &pipe_ctx->link_res, pipe_ctx->stream->signal);
dc->hwss.disable_stream(pipe_ctx);
#endif
@@ -4535,16 +4571,22 @@ void dc_link_set_drive_settings(struct dc *dc,
{
int i;
+ struct pipe_ctx *pipe = NULL;
+ const struct link_resource *link_res;
- for (i = 0; i < dc->link_count; i++) {
- if (dc->links[i] == link)
- break;
+ link_res = dc_link_get_cur_link_res(link);
+
+ for (i = 0; i < MAX_PIPES; i++) {
+ pipe = &dc->current_state->res_ctx.pipe_ctx[i];
+ if (pipe->stream && pipe->stream->link) {
+ if (pipe->stream->link == link)
+ break;
+ }
}
-
- if (i >= dc->link_count)
+ if (pipe && link_res)
+ dc_link_dp_set_drive_settings(pipe->stream->link, link_res, lt_settings);
+ else
ASSERT_CRITICAL(false);
-
- dc_link_dp_set_drive_settings(dc->links[i], lt_settings);
}
void dc_link_set_preferred_link_settings(struct dc *dc,
@@ -4605,11 +4647,9 @@ void dc_link_set_preferred_training_settings(struct dc *dc,
if (link_setting != NULL) {
link->preferred_link_setting = *link_setting;
#if defined(CONFIG_DRM_AMD_DC_DCN)
- if (dp_get_link_encoding_format(link_setting) ==
- DP_128b_132b_ENCODING && !link->hpo_dp_link_enc) {
- if (!add_dp_hpo_link_encoder_to_link(link))
- memset(&link->preferred_link_setting, 0, sizeof(link->preferred_link_setting));
- }
+ if (dp_get_link_encoding_format(link_setting) == DP_128b_132b_ENCODING)
+ /* TODO: add dc update for acquiring link res */
+ skip_immediate_retrain = true;
#endif
} else {
link->preferred_link_setting.lane_count = LANE_COUNT_UNKNOWN;
@@ -4736,6 +4776,9 @@ void dc_link_overwrite_extended_receiver_cap(
bool dc_link_is_fec_supported(const struct dc_link *link)
{
+ /* TODO - use asic cap instead of link_enc->features
+ * we no longer know which link enc to use for this link before commit
+ */
struct link_encoder *link_enc = NULL;
/* Links supporting dynamically assigned link encoder will be assigned next
@@ -4765,6 +4808,8 @@ bool dc_link_should_enable_fec(const struct dc_link *link)
link->local_sink &&
link->local_sink->edid_caps.panel_patch.disable_fec) ||
(link->connector_signal == SIGNAL_TYPE_EDP
+ // enable FEC for EDP if DSC is supported
+ && link->dpcd_caps.dsc_caps.dsc_basic_caps.fields.dsc_support.DSC_SUPPORT == false
))
is_fec_disable = true;
@@ -4828,3 +4873,125 @@ uint32_t dc_bandwidth_in_kbps_from_timing(
return kbps;
}
+
+const struct link_resource *dc_link_get_cur_link_res(const struct dc_link *link)
+{
+ int i;
+ struct pipe_ctx *pipe = NULL;
+ const struct link_resource *link_res = NULL;
+
+ for (i = 0; i < MAX_PIPES; i++) {
+ pipe = &link->dc->current_state->res_ctx.pipe_ctx[i];
+ if (pipe->stream && pipe->stream->link && pipe->top_pipe == NULL) {
+ if (pipe->stream->link == link) {
+ link_res = &pipe->link_res;
+ break;
+ }
+ }
+ }
+
+ return link_res;
+}
+
+/**
+ * dc_get_cur_link_res_map() - take a snapshot of current link resource allocation state
+ * @dc: pointer to dc of the dm calling this
+ * @map: a dc link resource snapshot defined internally to dc.
+ *
+ * DM needs to capture a snapshot of current link resource allocation mapping
+ * and store it in its persistent storage.
+ *
+ * Some of the link resource is using first come first serve policy.
+ * The allocation mapping depends on original hotplug order. This information
+ * is lost after driver is loaded next time. The snapshot is used in order to
+ * restore link resource to its previous state so user will get consistent
+ * link capability allocation across reboot.
+ *
+ * Return: none (void function)
+ *
+ */
+void dc_get_cur_link_res_map(const struct dc *dc, uint32_t *map)
+{
+#if defined(CONFIG_DRM_AMD_DC_DCN)
+ struct dc_link *link;
+ uint8_t i;
+ uint32_t hpo_dp_recycle_map = 0;
+
+ *map = 0;
+
+ if (dc->caps.dp_hpo) {
+ for (i = 0; i < dc->caps.max_links; i++) {
+ link = dc->links[i];
+ if (link->link_status.link_active &&
+ dp_get_link_encoding_format(&link->reported_link_cap) == DP_128b_132b_ENCODING &&
+ dp_get_link_encoding_format(&link->cur_link_settings) != DP_128b_132b_ENCODING)
+ /* hpo dp link encoder is considered as recycled, when RX reports 128b/132b encoding capability
+ * but current link doesn't use it.
+ */
+ hpo_dp_recycle_map |= (1 << i);
+ }
+ *map |= (hpo_dp_recycle_map << LINK_RES_HPO_DP_REC_MAP__SHIFT);
+ }
+#endif
+}
+
+/**
+ * dc_restore_link_res_map() - restore link resource allocation state from a snapshot
+ * @dc: pointer to dc of the dm calling this
+ * @map: a dc link resource snapshot defined internally to dc.
+ *
+ * DM needs to call this function after initial link detection on boot and
+ * before first commit streams to restore link resource allocation state
+ * from previous boot session.
+ *
+ * Some of the link resource is using first come first serve policy.
+ * The allocation mapping depends on original hotplug order. This information
+ * is lost after driver is loaded next time. The snapshot is used in order to
+ * restore link resource to its previous state so user will get consistent
+ * link capability allocation across reboot.
+ *
+ * Return: none (void function)
+ *
+ */
+void dc_restore_link_res_map(const struct dc *dc, uint32_t *map)
+{
+#if defined(CONFIG_DRM_AMD_DC_DCN)
+ struct dc_link *link;
+ uint8_t i;
+ unsigned int available_hpo_dp_count;
+ uint32_t hpo_dp_recycle_map = (*map & LINK_RES_HPO_DP_REC_MAP__MASK)
+ >> LINK_RES_HPO_DP_REC_MAP__SHIFT;
+
+ if (dc->caps.dp_hpo) {
+ available_hpo_dp_count = dc->res_pool->hpo_dp_link_enc_count;
+ /* remove excess 128b/132b encoding support for not recycled links */
+ for (i = 0; i < dc->caps.max_links; i++) {
+ if ((hpo_dp_recycle_map & (1 << i)) == 0) {
+ link = dc->links[i];
+ if (link->type != dc_connection_none &&
+ dp_get_link_encoding_format(&link->verified_link_cap) == DP_128b_132b_ENCODING) {
+ if (available_hpo_dp_count > 0)
+ available_hpo_dp_count--;
+ else
+ /* remove 128b/132b encoding capability by limiting verified link rate to HBR3 */
+ link->verified_link_cap.link_rate = LINK_RATE_HIGH3;
+ }
+ }
+ }
+ /* remove excess 128b/132b encoding support for recycled links */
+ for (i = 0; i < dc->caps.max_links; i++) {
+ if ((hpo_dp_recycle_map & (1 << i)) != 0) {
+ link = dc->links[i];
+ if (link->type != dc_connection_none &&
+ dp_get_link_encoding_format(&link->verified_link_cap) == DP_128b_132b_ENCODING) {
+ if (available_hpo_dp_count > 0)
+ available_hpo_dp_count--;
+ else
+ /* remove 128b/132b encoding capability by limiting verified link rate to HBR3 */
+ link->verified_link_cap.link_rate = LINK_RATE_HIGH3;
+ }
+ }
+ }
+ }
+#endif
+}
diff --git a/drivers/gpu/drm/amd/display/dc/core/dc_link_ddc.c b/drivers/gpu/drm/amd/display/dc/core/dc_link_ddc.c
index 60539b1..24dc662 100644
--- a/drivers/gpu/drm/amd/display/dc/core/dc_link_ddc.c
+++ b/drivers/gpu/drm/amd/display/dc/core/dc_link_ddc.c
@@ -626,7 +626,7 @@ bool dal_ddc_submit_aux_command(struct ddc_service *ddc,
do {
struct aux_payload current_payload;
bool is_end_of_payload = (retrieved + DEFAULT_AUX_MAX_DATA_SIZE) >=
- payload->length ? true : false;
+ payload->length;
uint32_t payload_length = is_end_of_payload ?
payload->length - retrieved : DEFAULT_AUX_MAX_DATA_SIZE;
diff --git a/drivers/gpu/drm/amd/display/dc/core/dc_link_dp.c b/drivers/gpu/drm/amd/display/dc/core/dc_link_dp.c
index 13bc69d..05e2165 100644
--- a/drivers/gpu/drm/amd/display/dc/core/dc_link_dp.c
+++ b/drivers/gpu/drm/amd/display/dc/core/dc_link_dp.c
@@ -100,6 +100,7 @@ static const struct dp_lt_fallback_entry dp_lt_fallbacks[] = {
#endif
static bool decide_fallback_link_setting(
+ struct dc_link *link,
struct dc_link_settings initial_link_settings,
struct dc_link_settings *current_link_setting,
enum link_training_result training_result);
@@ -398,6 +399,223 @@ static uint8_t get_dpcd_link_rate(const struct dc_link_settings *link_settings)
}
#endif
+static void vendor_specific_lttpr_wa_one_start(struct dc_link *link)
+{
+ const uint8_t vendor_lttpr_write_data[4] = {0x1, 0x50, 0x63, 0xff};
+ const uint8_t offset = dp_convert_to_count(
+ link->dpcd_caps.lttpr_caps.phy_repeater_cnt);
+ uint32_t vendor_lttpr_write_address = 0xF004F;
+
+ if (offset != 0xFF)
+ vendor_lttpr_write_address +=
+ ((DP_REPEATER_CONFIGURATION_AND_STATUS_SIZE) * (offset - 1));
+
+ /* W/A for certain LTTPR to reset their lane settings, part one of two */
+ core_link_write_dpcd(
+ link,
+ vendor_lttpr_write_address,
+ &vendor_lttpr_write_data[0],
+ sizeof(vendor_lttpr_write_data));
+}
+
+static void vendor_specific_lttpr_wa_one_end(
+ struct dc_link *link,
+ uint8_t retry_count)
+{
+ const uint8_t vendor_lttpr_write_data[4] = {0x1, 0x50, 0x63, 0x0};
+ const uint8_t offset = dp_convert_to_count(
+ link->dpcd_caps.lttpr_caps.phy_repeater_cnt);
+ uint32_t vendor_lttpr_write_address = 0xF004F;
+
+ if (!retry_count) {
+ if (offset != 0xFF)
+ vendor_lttpr_write_address +=
+ ((DP_REPEATER_CONFIGURATION_AND_STATUS_SIZE) * (offset - 1));
+
+ /* W/A for certain LTTPR to reset their lane settings, part two of two */
+ core_link_write_dpcd(
+ link,
+ vendor_lttpr_write_address,
+ &vendor_lttpr_write_data[0],
+ sizeof(vendor_lttpr_write_data));
+ }
+}
+
+static void vendor_specific_lttpr_wa_one_two(
+ struct dc_link *link,
+ const uint8_t rate)
+{
+ if (link->apply_vendor_specific_lttpr_link_rate_wa) {
+ uint8_t toggle_rate = 0x0;
+
+ if (rate == 0x6)
+ toggle_rate = 0xA;
+ else
+ toggle_rate = 0x6;
+
+ if (link->vendor_specific_lttpr_link_rate_wa == rate) {
+ /* W/A for certain LTTPR to reset internal state for link training */
+ core_link_write_dpcd(
+ link,
+ DP_LINK_BW_SET,
+ &toggle_rate,
+ 1);
+ }
+
+ /* Store the last attempted link rate for this link */
+ link->vendor_specific_lttpr_link_rate_wa = rate;
+ }
+}
+
+static void vendor_specific_lttpr_wa_three(
+ struct dc_link *link,
+ union lane_adjust dpcd_lane_adjust[LANE_COUNT_DP_MAX])
+{
+ const uint8_t vendor_lttpr_write_data_vs[3] = {0x0, 0x53, 0x63};
+ const uint8_t vendor_lttpr_write_data_pe[3] = {0x0, 0x54, 0x63};
+ const uint8_t offset = dp_convert_to_count(
+ link->dpcd_caps.lttpr_caps.phy_repeater_cnt);
+ uint32_t vendor_lttpr_write_address = 0xF004F;
+ uint32_t vendor_lttpr_read_address = 0xF0053;
+ uint8_t dprx_vs = 0;
+ uint8_t dprx_pe = 0;
+ uint8_t lane;
+
+ if (offset != 0xFF) {
+ vendor_lttpr_write_address +=
+ ((DP_REPEATER_CONFIGURATION_AND_STATUS_SIZE) * (offset - 1));
+ vendor_lttpr_read_address +=
+ ((DP_REPEATER_CONFIGURATION_AND_STATUS_SIZE) * (offset - 1));
+ }
+
+ /* W/A to read lane settings requested by DPRX */
+ core_link_write_dpcd(
+ link,
+ vendor_lttpr_write_address,
+ &vendor_lttpr_write_data_vs[0],
+ sizeof(vendor_lttpr_write_data_vs));
+ core_link_read_dpcd(
+ link,
+ vendor_lttpr_read_address,
+ &dprx_vs,
+ 1);
+ core_link_write_dpcd(
+ link,
+ vendor_lttpr_write_address,
+ &vendor_lttpr_write_data_pe[0],
+ sizeof(vendor_lttpr_write_data_pe));
+ core_link_read_dpcd(
+ link,
+ vendor_lttpr_read_address,
+ &dprx_pe,
+ 1);
+
+ for (lane = 0; lane < LANE_COUNT_DP_MAX; lane++) {
+ dpcd_lane_adjust[lane].bits.VOLTAGE_SWING_LANE = (dprx_vs >> (2 * lane)) & 0x3;
+ dpcd_lane_adjust[lane].bits.PRE_EMPHASIS_LANE = (dprx_pe >> (2 * lane)) & 0x3;
+ }
+}
+
+static void vendor_specific_lttpr_wa_three_dpcd(
+ struct dc_link *link,
+ union dpcd_training_lane dpcd_lane_adjust[LANE_COUNT_DP_MAX])
+{
+ union lane_adjust lane_adjust[LANE_COUNT_DP_MAX];
+ uint8_t lane = 0;
+
+ vendor_specific_lttpr_wa_three(link, lane_adjust);
+
+ for (lane = 0; lane < LANE_COUNT_DP_MAX; lane++) {
+ dpcd_lane_adjust[lane].bits.VOLTAGE_SWING_SET = lane_adjust[lane].bits.VOLTAGE_SWING_LANE;
+ dpcd_lane_adjust[lane].bits.PRE_EMPHASIS_SET = lane_adjust[lane].bits.PRE_EMPHASIS_LANE;
+ }
+}
+
+static void vendor_specific_lttpr_wa_four(
+ struct dc_link *link,
+ bool apply_wa)
+{
+ const uint8_t vendor_lttpr_write_data_one[4] = {0x1, 0x55, 0x63, 0x8};
+ const uint8_t vendor_lttpr_write_data_two[4] = {0x1, 0x55, 0x63, 0x0};
+ const uint8_t offset = dp_convert_to_count(
+ link->dpcd_caps.lttpr_caps.phy_repeater_cnt);
+ uint32_t vendor_lttpr_write_address = 0xF004F;
+#if defined(CONFIG_DRM_AMD_DC_DP2_0)
+ uint8_t sink_status = 0;
+ uint8_t i;
+#endif
+
+ if (offset != 0xFF)
+ vendor_lttpr_write_address +=
+ ((DP_REPEATER_CONFIGURATION_AND_STATUS_SIZE) * (offset - 1));
+
+ /* W/A to pass through DPCD write of TPS=0 to DPRX */
+ if (apply_wa) {
+ core_link_write_dpcd(
+ link,
+ vendor_lttpr_write_address,
+ &vendor_lttpr_write_data_one[0],
+ sizeof(vendor_lttpr_write_data_one));
+ }
+
+ /* clear training pattern set */
+ dpcd_set_training_pattern(link, DP_TRAINING_PATTERN_VIDEOIDLE);
+
+ if (apply_wa) {
+ core_link_write_dpcd(
+ link,
+ vendor_lttpr_write_address,
+ &vendor_lttpr_write_data_two[0],
+ sizeof(vendor_lttpr_write_data_two));
+ }
+
+#if defined(CONFIG_DRM_AMD_DC_DP2_0)
+ /* poll for intra-hop disable */
+ for (i = 0; i < 10; i++) {
+ if ((core_link_read_dpcd(link, DP_SINK_STATUS, &sink_status, 1) == DC_OK) &&
+ (sink_status & DP_INTRA_HOP_AUX_REPLY_INDICATION) == 0)
+ break;
+ udelay(1000);
+ }
+#endif
+}
+
+static void vendor_specific_lttpr_wa_five(
+ struct dc_link *link,
+ const union dpcd_training_lane dpcd_lane_adjust[LANE_COUNT_DP_MAX],
+ uint8_t lane_count)
+{
+ const uint32_t vendor_lttpr_write_address = 0xF004F;
+ const uint8_t vendor_lttpr_write_data_reset[4] = {0x1, 0x50, 0x63, 0xFF};
+ uint8_t vendor_lttpr_write_data_vs[4] = {0x1, 0x51, 0x63, 0x0};
+ uint8_t vendor_lttpr_write_data_pe[4] = {0x1, 0x52, 0x63, 0x0};
+ uint8_t lane = 0;
+
+ for (lane = 0; lane < lane_count; lane++) {
+ vendor_lttpr_write_data_vs[3] |=
+ dpcd_lane_adjust[lane].bits.VOLTAGE_SWING_SET << (2 * lane);
+ vendor_lttpr_write_data_pe[3] |=
+ dpcd_lane_adjust[lane].bits.PRE_EMPHASIS_SET << (2 * lane);
+ }
+
+ /* Force LTTPR to output desired VS and PE */
+ core_link_write_dpcd(
+ link,
+ vendor_lttpr_write_address,
+ &vendor_lttpr_write_data_reset[0],
+ sizeof(vendor_lttpr_write_data_reset));
+ core_link_write_dpcd(
+ link,
+ vendor_lttpr_write_address,
+ &vendor_lttpr_write_data_vs[0],
+ sizeof(vendor_lttpr_write_data_vs));
+ core_link_write_dpcd(
+ link,
+ vendor_lttpr_write_address,
+ &vendor_lttpr_write_data_pe[0],
+ sizeof(vendor_lttpr_write_data_pe));
+}
+
enum dc_status dpcd_set_link_settings(
struct dc_link *link,
const struct link_training_settings *lt_settings)
@@ -430,7 +648,7 @@ enum dc_status dpcd_set_link_settings(
status = core_link_write_dpcd(link, DP_LANE_COUNT_SET,
&lane_count_set.raw, 1);
- if (link->dpcd_caps.dpcd_rev.raw >= DPCD_REV_14 &&
+ if (link->dpcd_caps.dpcd_rev.raw >= DPCD_REV_13 &&
lt_settings->link_settings.use_link_rate_set == true) {
rate = 0;
/* WA for some MUX chips that will power down with eDP and lose supported
@@ -452,6 +670,15 @@ enum dc_status dpcd_set_link_settings(
#else
rate = (uint8_t) (lt_settings->link_settings.link_rate);
#endif
+ if (link->dc->debug.apply_vendor_specific_lttpr_wa &&
+ (link->chip_caps & EXT_DISPLAY_PATH_CAPS__DP_FIXED_VS_EN) &&
+ link->lttpr_mode == LTTPR_MODE_TRANSPARENT)
+ vendor_specific_lttpr_wa_one_start(link);
+
+ if (link->dc->debug.apply_vendor_specific_lttpr_wa &&
+ (link->chip_caps & EXT_DISPLAY_PATH_CAPS__DP_FIXED_VS_EN))
+ vendor_specific_lttpr_wa_one_two(link, rate);
+
status = core_link_write_dpcd(link, DP_LINK_BW_SET, &rate, 1);
}
@@ -1024,6 +1251,7 @@ bool dp_is_max_vs_reached(
static bool perform_post_lt_adj_req_sequence(
struct dc_link *link,
+ const struct link_resource *link_res,
struct link_training_settings *lt_settings)
{
enum dc_lane_count lane_count =
@@ -1087,6 +1315,7 @@ static bool perform_post_lt_adj_req_sequence(
lt_settings->hw_lane_settings, lt_settings->dpcd_lane_settings);
dc_link_dp_set_drive_settings(link,
+ link_res,
lt_settings);
break;
}
@@ -1161,6 +1390,7 @@ enum link_training_result dp_get_cr_failure(enum dc_lane_count ln_count,
static enum link_training_result perform_channel_equalization_sequence(
struct dc_link *link,
+ const struct link_resource *link_res,
struct link_training_settings *lt_settings,
uint32_t offset)
{
@@ -1183,12 +1413,12 @@ static enum link_training_result perform_channel_equalization_sequence(
tr_pattern = DP_TRAINING_PATTERN_SEQUENCE_4;
#endif
- dp_set_hw_training_pattern(link, tr_pattern, offset);
+ dp_set_hw_training_pattern(link, link_res, tr_pattern, offset);
for (retries_ch_eq = 0; retries_ch_eq <= LINK_TRAINING_MAX_RETRY_COUNT;
retries_ch_eq++) {
- dp_set_hw_lane_settings(link, lt_settings, offset);
+ dp_set_hw_lane_settings(link, link_res, lt_settings, offset);
/* 2. update DPCD*/
if (!retries_ch_eq)
@@ -1211,6 +1441,12 @@ static enum link_training_result perform_channel_equalization_sequence(
dp_translate_training_aux_read_interval(
link->dpcd_caps.lttpr_caps.aux_rd_interval[offset - 1]);
+ if (link->dc->debug.apply_vendor_specific_lttpr_wa &&
+ (link->chip_caps & EXT_DISPLAY_PATH_CAPS__DP_FIXED_VS_EN) &&
+ link->lttpr_mode == LTTPR_MODE_TRANSPARENT) {
+ wait_time_microsec = 16000;
+ }
+
dp_wait_for_training_aux_rd_interval(
link,
wait_time_microsec);
@@ -1246,18 +1482,20 @@ static enum link_training_result perform_channel_equalization_sequence(
}
static void start_clock_recovery_pattern_early(struct dc_link *link,
+ const struct link_resource *link_res,
struct link_training_settings *lt_settings,
uint32_t offset)
{
DC_LOG_HW_LINK_TRAINING("%s\n GPU sends TPS1. Wait 400us.\n",
__func__);
- dp_set_hw_training_pattern(link, lt_settings->pattern_for_cr, offset);
- dp_set_hw_lane_settings(link, lt_settings, offset);
+ dp_set_hw_training_pattern(link, link_res, lt_settings->pattern_for_cr, offset);
+ dp_set_hw_lane_settings(link, link_res, lt_settings, offset);
udelay(400);
}
static enum link_training_result perform_clock_recovery_sequence(
struct dc_link *link,
+ const struct link_resource *link_res,
struct link_training_settings *lt_settings,
uint32_t offset)
{
@@ -1273,7 +1511,7 @@ static enum link_training_result perform_clock_recovery_sequence(
retry_count = 0;
if (!link->ctx->dc->work_arounds.lt_early_cr_pattern)
- dp_set_hw_training_pattern(link, lt_settings->pattern_for_cr, offset);
+ dp_set_hw_training_pattern(link, link_res, lt_settings->pattern_for_cr, offset);
/* najeeb - The synaptics MST hub can put the LT in
* infinite loop by switching the VS
@@ -1290,6 +1528,7 @@ static enum link_training_result perform_clock_recovery_sequence(
/* 1. call HWSS to set lane settings*/
dp_set_hw_lane_settings(
link,
+ link_res,
lt_settings,
offset);
@@ -1311,8 +1550,10 @@ static enum link_training_result perform_clock_recovery_sequence(
/* 3. wait receiver to lock-on*/
wait_time_microsec = lt_settings->cr_pattern_time;
- if (link->lttpr_mode == LTTPR_MODE_NON_TRANSPARENT)
- wait_time_microsec = TRAINING_AUX_RD_INTERVAL;
+ if (link->dc->debug.apply_vendor_specific_lttpr_wa &&
+ (link->chip_caps & EXT_DISPLAY_PATH_CAPS__DP_FIXED_VS_EN)) {
+ wait_time_microsec = 16000;
+ }
dp_wait_for_training_aux_rd_interval(
link,
@@ -1329,6 +1570,13 @@ static enum link_training_result perform_clock_recovery_sequence(
dpcd_lane_adjust,
offset);
+ if (link->dc->debug.apply_vendor_specific_lttpr_wa &&
+ (link->chip_caps & EXT_DISPLAY_PATH_CAPS__DP_FIXED_VS_EN) &&
+ link->lttpr_mode == LTTPR_MODE_TRANSPARENT) {
+ vendor_specific_lttpr_wa_one_end(link, retry_count);
+ vendor_specific_lttpr_wa_three(link, dpcd_lane_adjust);
+ }
+
/* 5. check CR done*/
if (dp_is_cr_done(lane_count, dpcd_lane_status))
return LINK_TRAINING_SUCCESS;
@@ -1379,13 +1627,14 @@ static enum link_training_result perform_clock_recovery_sequence(
static inline enum link_training_result dp_transition_to_video_idle(
struct dc_link *link,
+ const struct link_resource *link_res,
struct link_training_settings *lt_settings,
enum link_training_result status)
{
union lane_count_set lane_count_set = {0};
/* 4. mainlink output idle pattern*/
- dp_set_hw_test_pattern(link, DP_TEST_PATTERN_VIDEO_MODE, NULL, 0);
+ dp_set_hw_test_pattern(link, link_res, DP_TEST_PATTERN_VIDEO_MODE, NULL, 0);
/*
* 5. post training adjust if required
@@ -1409,7 +1658,7 @@ static inline enum link_training_result dp_transition_to_video_idle(
}
if (status == LINK_TRAINING_SUCCESS &&
- perform_post_lt_adj_req_sequence(link, lt_settings) == false)
+ perform_post_lt_adj_req_sequence(link, link_res, lt_settings) == false)
status = LINK_TRAINING_LQA_FAIL;
lane_count_set.bits.LANE_COUNT_SET = lt_settings->link_settings.lane_count;
@@ -1852,10 +2101,11 @@ static void print_status_message(
void dc_link_dp_set_drive_settings(
struct dc_link *link,
+ const struct link_resource *link_res,
struct link_training_settings *lt_settings)
{
/* program ASIC PHY settings*/
- dp_set_hw_lane_settings(link, lt_settings, DPRX);
+ dp_set_hw_lane_settings(link, link_res, lt_settings, DPRX);
dp_hw_to_dpcd_lane_settings(lt_settings,
lt_settings->hw_lane_settings, lt_settings->dpcd_lane_settings);
@@ -1866,6 +2116,7 @@ void dc_link_dp_set_drive_settings(
bool dc_link_dp_perform_link_training_skip_aux(
struct dc_link *link,
+ const struct link_resource *link_res,
const struct dc_link_settings *link_setting)
{
struct link_training_settings lt_settings = {0};
@@ -1882,10 +2133,10 @@ bool dc_link_dp_perform_link_training_skip_aux(
/* 1. Perform_clock_recovery_sequence. */
/* transmit training pattern for clock recovery */
- dp_set_hw_training_pattern(link, lt_settings.pattern_for_cr, DPRX);
+ dp_set_hw_training_pattern(link, link_res, lt_settings.pattern_for_cr, DPRX);
/* call HWSS to set lane settings*/
- dp_set_hw_lane_settings(link, <_settings, DPRX);
+ dp_set_hw_lane_settings(link, link_res, <_settings, DPRX);
/* wait receiver to lock-on*/
dp_wait_for_training_aux_rd_interval(link, lt_settings.cr_pattern_time);
@@ -1893,10 +2144,10 @@ bool dc_link_dp_perform_link_training_skip_aux(
/* 2. Perform_channel_equalization_sequence. */
/* transmit training pattern for channel equalization. */
- dp_set_hw_training_pattern(link, lt_settings.pattern_for_eq, DPRX);
+ dp_set_hw_training_pattern(link, link_res, lt_settings.pattern_for_eq, DPRX);
/* call HWSS to set lane settings*/
- dp_set_hw_lane_settings(link, <_settings, DPRX);
+ dp_set_hw_lane_settings(link, link_res, <_settings, DPRX);
/* wait receiver to lock-on. */
dp_wait_for_training_aux_rd_interval(link, lt_settings.eq_pattern_time);
@@ -1904,7 +2155,7 @@ bool dc_link_dp_perform_link_training_skip_aux(
/* 3. Perform_link_training_int. */
/* Mainlink output idle pattern. */
- dp_set_hw_test_pattern(link, DP_TEST_PATTERN_VIDEO_MODE, NULL, 0);
+ dp_set_hw_test_pattern(link, link_res, DP_TEST_PATTERN_VIDEO_MODE, NULL, 0);
print_status_message(link, <_settings, LINK_TRAINING_SUCCESS);
@@ -1985,6 +2236,7 @@ static void dpcd_128b_132b_get_aux_rd_interval(struct dc_link *link,
static enum link_training_result dp_perform_128b_132b_channel_eq_done_sequence(
struct dc_link *link,
+ const struct link_resource *link_res,
struct link_training_settings *lt_settings)
{
uint8_t loop_count;
@@ -1996,7 +2248,7 @@ static enum link_training_result dp_perform_128b_132b_channel_eq_done_sequence(
union lane_adjust dpcd_lane_adjust[LANE_COUNT_DP_MAX] = {0};
/* Transmit 128b/132b_TPS1 over Main-Link */
- dp_set_hw_training_pattern(link, lt_settings->pattern_for_cr, DPRX);
+ dp_set_hw_training_pattern(link, link_res, lt_settings->pattern_for_cr, DPRX);
/* Set TRAINING_PATTERN_SET to 01h */
dpcd_set_training_pattern(link, lt_settings->pattern_for_cr);
@@ -2006,8 +2258,8 @@ static enum link_training_result dp_perform_128b_132b_channel_eq_done_sequence(
&dpcd_lane_status_updated, dpcd_lane_adjust, DPRX);
dp_decide_lane_settings(lt_settings, dpcd_lane_adjust,
lt_settings->hw_lane_settings, lt_settings->dpcd_lane_settings);
- dp_set_hw_lane_settings(link, lt_settings, DPRX);
- dp_set_hw_training_pattern(link, lt_settings->pattern_for_eq, DPRX);
+ dp_set_hw_lane_settings(link, link_res, lt_settings, DPRX);
+ dp_set_hw_training_pattern(link, link_res, lt_settings->pattern_for_eq, DPRX);
/* Set loop counter to start from 1 */
loop_count = 1;
@@ -2034,7 +2286,7 @@ static enum link_training_result dp_perform_128b_132b_channel_eq_done_sequence(
} else if (dpcd_lane_status_updated.bits.LT_FAILED_128b_132b) {
status = DP_128b_132b_LT_FAILED;
} else {
- dp_set_hw_lane_settings(link, lt_settings, DPRX);
+ dp_set_hw_lane_settings(link, link_res, lt_settings, DPRX);
dpcd_set_lane_settings(link, lt_settings, DPRX);
}
loop_count++;
@@ -2063,6 +2315,7 @@ static enum link_training_result dp_perform_128b_132b_channel_eq_done_sequence(
static enum link_training_result dp_perform_128b_132b_cds_done_sequence(
struct dc_link *link,
+ const struct link_resource *link_res,
struct link_training_settings *lt_settings)
{
/* Assumption: assume hardware has transmitted eq pattern */
@@ -2099,6 +2352,7 @@ static enum link_training_result dp_perform_128b_132b_cds_done_sequence(
static enum link_training_result dp_perform_8b_10b_link_training(
struct dc_link *link,
+ const struct link_resource *link_res,
struct link_training_settings *lt_settings)
{
enum link_training_result status = LINK_TRAINING_SUCCESS;
@@ -2108,7 +2362,7 @@ static enum link_training_result dp_perform_8b_10b_link_training(
uint8_t lane = 0;
if (link->ctx->dc->work_arounds.lt_early_cr_pattern)
- start_clock_recovery_pattern_early(link, lt_settings, DPRX);
+ start_clock_recovery_pattern_early(link, link_res, lt_settings, DPRX);
/* 1. set link rate, lane count and spread. */
dpcd_set_link_settings(link, lt_settings);
@@ -2122,12 +2376,13 @@ static enum link_training_result dp_perform_8b_10b_link_training(
for (repeater_id = repeater_cnt; (repeater_id > 0 && status == LINK_TRAINING_SUCCESS);
repeater_id--) {
- status = perform_clock_recovery_sequence(link, lt_settings, repeater_id);
+ status = perform_clock_recovery_sequence(link, link_res, lt_settings, repeater_id);
if (status != LINK_TRAINING_SUCCESS)
break;
status = perform_channel_equalization_sequence(link,
+ link_res,
lt_settings,
repeater_id);
@@ -2142,9 +2397,10 @@ static enum link_training_result dp_perform_8b_10b_link_training(
}
if (status == LINK_TRAINING_SUCCESS) {
- status = perform_clock_recovery_sequence(link, lt_settings, DPRX);
+ status = perform_clock_recovery_sequence(link, link_res, lt_settings, DPRX);
if (status == LINK_TRAINING_SUCCESS) {
status = perform_channel_equalization_sequence(link,
+ link_res,
lt_settings,
DPRX);
}
@@ -2156,6 +2412,7 @@ static enum link_training_result dp_perform_8b_10b_link_training(
#if defined(CONFIG_DRM_AMD_DC_DCN)
static enum link_training_result dp_perform_128b_132b_link_training(
struct dc_link *link,
+ const struct link_resource *link_res,
struct link_training_settings *lt_settings)
{
enum link_training_result result = LINK_TRAINING_SUCCESS;
@@ -2167,23 +2424,358 @@ static enum link_training_result dp_perform_128b_132b_link_training(
decide_8b_10b_training_settings(link,
<_settings->link_settings,
&legacy_settings);
- return dp_perform_8b_10b_link_training(link, &legacy_settings);
+ return dp_perform_8b_10b_link_training(link, link_res, &legacy_settings);
}
dpcd_set_link_settings(link, lt_settings);
if (result == LINK_TRAINING_SUCCESS)
- result = dp_perform_128b_132b_channel_eq_done_sequence(link, lt_settings);
+ result = dp_perform_128b_132b_channel_eq_done_sequence(link, link_res, lt_settings);
if (result == LINK_TRAINING_SUCCESS)
- result = dp_perform_128b_132b_cds_done_sequence(link, lt_settings);
+ result = dp_perform_128b_132b_cds_done_sequence(link, link_res, lt_settings);
return result;
}
#endif
+static enum link_training_result dc_link_dp_perform_fixed_vs_pe_training_sequence(
+ struct dc_link *link,
+ const struct link_resource *link_res,
+ struct link_training_settings *lt_settings)
+{
+ const uint8_t vendor_lttpr_write_data_reset[4] = {0x1, 0x50, 0x63, 0xFF};
+ const uint8_t offset = dp_convert_to_count(
+ link->dpcd_caps.lttpr_caps.phy_repeater_cnt);
+ const uint8_t vendor_lttpr_write_data_intercept_en[4] = {0x1, 0x55, 0x63, 0x0};
+ const uint8_t vendor_lttpr_write_data_intercept_dis[4] = {0x1, 0x55, 0x63, 0x68};
+ uint8_t vendor_lttpr_write_data_vs[4] = {0x1, 0x51, 0x63, 0x0};
+ uint8_t vendor_lttpr_write_data_pe[4] = {0x1, 0x52, 0x63, 0x0};
+ uint32_t vendor_lttpr_write_address = 0xF004F;
+ enum link_training_result status = LINK_TRAINING_SUCCESS;
+ uint8_t lane = 0;
+ union down_spread_ctrl downspread = {0};
+ union lane_count_set lane_count_set = {0};
+ uint8_t toggle_rate;
+ uint8_t rate;
+
+ /* Only 8b/10b is supported */
+ ASSERT(dp_get_link_encoding_format(<_settings->link_settings) ==
+ DP_8b_10b_ENCODING);
+
+ if (offset != 0xFF) {
+ vendor_lttpr_write_address +=
+ ((DP_REPEATER_CONFIGURATION_AND_STATUS_SIZE) * (offset - 1));
+ }
+
+ /* Vendor specific: Reset lane settings */
+ core_link_write_dpcd(
+ link,
+ vendor_lttpr_write_address,
+ &vendor_lttpr_write_data_reset[0],
+ sizeof(vendor_lttpr_write_data_reset));
+ core_link_write_dpcd(
+ link,
+ vendor_lttpr_write_address,
+ &vendor_lttpr_write_data_vs[0],
+ sizeof(vendor_lttpr_write_data_vs));
+ core_link_write_dpcd(
+ link,
+ vendor_lttpr_write_address,
+ &vendor_lttpr_write_data_pe[0],
+ sizeof(vendor_lttpr_write_data_pe));
+
+ /* Vendor specific: Enable intercept */
+ core_link_write_dpcd(
+ link,
+ vendor_lttpr_write_address,
+ &vendor_lttpr_write_data_intercept_en[0],
+ sizeof(vendor_lttpr_write_data_intercept_en));
+
+ /* 1. set link rate, lane count and spread. */
+
+ downspread.raw = (uint8_t)(lt_settings->link_settings.link_spread);
+
+ lane_count_set.bits.LANE_COUNT_SET =
+ lt_settings->link_settings.lane_count;
+
+ lane_count_set.bits.ENHANCED_FRAMING = lt_settings->enhanced_framing;
+ lane_count_set.bits.POST_LT_ADJ_REQ_GRANTED = 0;
+
+
+ if (lt_settings->pattern_for_eq < DP_TRAINING_PATTERN_SEQUENCE_4) {
+ lane_count_set.bits.POST_LT_ADJ_REQ_GRANTED =
+ link->dpcd_caps.max_ln_count.bits.POST_LT_ADJ_REQ_SUPPORTED;
+ }
+
+ core_link_write_dpcd(link, DP_DOWNSPREAD_CTRL,
+ &downspread.raw, sizeof(downspread));
+
+ core_link_write_dpcd(link, DP_LANE_COUNT_SET,
+ &lane_count_set.raw, 1);
+
+#if defined(CONFIG_DRM_AMD_DC_DCN)
+ rate = get_dpcd_link_rate(<_settings->link_settings);
+#else
+ rate = (uint8_t) (lt_settings->link_settings.link_rate);
+#endif
+
+ /* Vendor specific: Toggle link rate */
+ toggle_rate = (rate == 0x6) ? 0xA : 0x6;
+
+ if (link->vendor_specific_lttpr_link_rate_wa == rate) {
+ core_link_write_dpcd(
+ link,
+ DP_LINK_BW_SET,
+ &toggle_rate,
+ 1);
+ }
+
+ link->vendor_specific_lttpr_link_rate_wa = rate;
+
+ core_link_write_dpcd(link, DP_LINK_BW_SET, &rate, 1);
+
+ DC_LOG_HW_LINK_TRAINING("%s\n %x rate = %x\n %x lane = %x framing = %x\n %x spread = %x\n",
+ __func__,
+ DP_LINK_BW_SET,
+ lt_settings->link_settings.link_rate,
+ DP_LANE_COUNT_SET,
+ lt_settings->link_settings.lane_count,
+ lt_settings->enhanced_framing,
+ DP_DOWNSPREAD_CTRL,
+ lt_settings->link_settings.link_spread);
+
+ /* 2. Perform link training */
+
+ /* Perform Clock Recovery Sequence */
+ if (status == LINK_TRAINING_SUCCESS) {
+ uint32_t retries_cr;
+ uint32_t retry_count;
+ uint32_t wait_time_microsec;
+ enum dc_lane_count lane_count = lt_settings->link_settings.lane_count;
+ union lane_status dpcd_lane_status[LANE_COUNT_DP_MAX];
+ union lane_align_status_updated dpcd_lane_status_updated;
+ union lane_adjust dpcd_lane_adjust[LANE_COUNT_DP_MAX] = {0};
+
+ retries_cr = 0;
+ retry_count = 0;
+
+ while ((retries_cr < LINK_TRAINING_MAX_RETRY_COUNT) &&
+ (retry_count < LINK_TRAINING_MAX_CR_RETRY)) {
+
+ memset(&dpcd_lane_status, '\0', sizeof(dpcd_lane_status));
+ memset(&dpcd_lane_status_updated, '\0',
+ sizeof(dpcd_lane_status_updated));
+
+ /* 1. call HWSS to set lane settings */
+ dp_set_hw_lane_settings(
+ link,
+ link_res,
+ lt_settings,
+ 0);
+
+ /* 2. update DPCD of the receiver */
+ if (!retry_count) {
+ /* EPR #361076 - write as a 5-byte burst,
+ * but only for the 1-st iteration.
+ */
+ dpcd_set_lt_pattern_and_lane_settings(
+ link,
+ lt_settings,
+ lt_settings->pattern_for_cr,
+ 0);
+ /* Vendor specific: Disable intercept */
+ core_link_write_dpcd(
+ link,
+ vendor_lttpr_write_address,
+ &vendor_lttpr_write_data_intercept_dis[0],
+ sizeof(vendor_lttpr_write_data_intercept_dis));
+ } else {
+ vendor_lttpr_write_data_vs[3] = 0;
+ vendor_lttpr_write_data_pe[3] = 0;
+
+ for (lane = 0; lane < lane_count; lane++) {
+ vendor_lttpr_write_data_vs[3] |=
+ lt_settings->dpcd_lane_settings[lane].bits.VOLTAGE_SWING_SET << (2 * lane);
+ vendor_lttpr_write_data_pe[3] |=
+ lt_settings->dpcd_lane_settings[lane].bits.PRE_EMPHASIS_SET << (2 * lane);
+ }
+
+ /* Vendor specific: Update VS and PE to DPRX requested value */
+ core_link_write_dpcd(
+ link,
+ vendor_lttpr_write_address,
+ &vendor_lttpr_write_data_vs[0],
+ sizeof(vendor_lttpr_write_data_vs));
+ core_link_write_dpcd(
+ link,
+ vendor_lttpr_write_address,
+ &vendor_lttpr_write_data_pe[0],
+ sizeof(vendor_lttpr_write_data_pe));
+
+ dpcd_set_lane_settings(
+ link,
+ lt_settings,
+ 0);
+ }
+
+ /* 3. wait receiver to lock-on*/
+ wait_time_microsec = lt_settings->cr_pattern_time;
+
+ dp_wait_for_training_aux_rd_interval(
+ link,
+ wait_time_microsec);
+
+ /* 4. Read lane status and requested drive
+ * settings as set by the sink
+ */
+ dp_get_lane_status_and_lane_adjust(
+ link,
+ lt_settings,
+ dpcd_lane_status,
+ &dpcd_lane_status_updated,
+ dpcd_lane_adjust,
+ 0);
+
+ /* 5. check CR done*/
+ if (dp_is_cr_done(lane_count, dpcd_lane_status)) {
+ status = LINK_TRAINING_SUCCESS;
+ break;
+ }
+
+ /* 6. max VS reached*/
+ if (dp_is_max_vs_reached(lt_settings))
+ break;
+
+ /* 7. same lane settings */
+ /* Note: settings are the same for all lanes,
+ * so comparing first lane is sufficient
+ */
+ if (lt_settings->dpcd_lane_settings[0].bits.VOLTAGE_SWING_SET ==
+ dpcd_lane_adjust[0].bits.VOLTAGE_SWING_LANE)
+ retries_cr++;
+ else
+ retries_cr = 0;
+
+ /* 8. update VS/PE/PC2 in lt_settings*/
+ dp_decide_lane_settings(lt_settings, dpcd_lane_adjust,
+ lt_settings->hw_lane_settings, lt_settings->dpcd_lane_settings);
+ retry_count++;
+ }
+
+ if (retry_count >= LINK_TRAINING_MAX_CR_RETRY) {
+ ASSERT(0);
+ DC_LOG_ERROR("%s: Link Training Error, could not get CR after %d tries. Possibly voltage swing issue",
+ __func__,
+ LINK_TRAINING_MAX_CR_RETRY);
+
+ }
+
+ status = dp_get_cr_failure(lane_count, dpcd_lane_status);
+ }
+
+ /* Perform Channel EQ Sequence */
+ if (status == LINK_TRAINING_SUCCESS) {
+ enum dc_dp_training_pattern tr_pattern;
+ uint32_t retries_ch_eq;
+ uint32_t wait_time_microsec;
+ enum dc_lane_count lane_count = lt_settings->link_settings.lane_count;
+ union lane_align_status_updated dpcd_lane_status_updated = {0};
+ union lane_status dpcd_lane_status[LANE_COUNT_DP_MAX] = {0};
+ union lane_adjust dpcd_lane_adjust[LANE_COUNT_DP_MAX] = {0};
+
+ /* Note: also check that TPS4 is a supported feature*/
+ tr_pattern = lt_settings->pattern_for_eq;
+
+ dp_set_hw_training_pattern(link, link_res, tr_pattern, 0);
+
+ status = LINK_TRAINING_EQ_FAIL_EQ;
+
+ for (retries_ch_eq = 0; retries_ch_eq <= LINK_TRAINING_MAX_RETRY_COUNT;
+ retries_ch_eq++) {
+
+ dp_set_hw_lane_settings(link, link_res, lt_settings, 0);
+
+ vendor_lttpr_write_data_vs[3] = 0;
+ vendor_lttpr_write_data_pe[3] = 0;
+
+ for (lane = 0; lane < lane_count; lane++) {
+ vendor_lttpr_write_data_vs[3] |=
+ lt_settings->dpcd_lane_settings[lane].bits.VOLTAGE_SWING_SET << (2 * lane);
+ vendor_lttpr_write_data_pe[3] |=
+ lt_settings->dpcd_lane_settings[lane].bits.PRE_EMPHASIS_SET << (2 * lane);
+ }
+
+ /* Vendor specific: Update VS and PE to DPRX requested value */
+ core_link_write_dpcd(
+ link,
+ vendor_lttpr_write_address,
+ &vendor_lttpr_write_data_vs[0],
+ sizeof(vendor_lttpr_write_data_vs));
+ core_link_write_dpcd(
+ link,
+ vendor_lttpr_write_address,
+ &vendor_lttpr_write_data_pe[0],
+ sizeof(vendor_lttpr_write_data_pe));
+
+ /* 2. update DPCD*/
+ if (!retries_ch_eq)
+ /* EPR #361076 - write as a 5-byte burst,
+ * but only for the 1-st iteration
+ */
+
+ dpcd_set_lt_pattern_and_lane_settings(
+ link,
+ lt_settings,
+ tr_pattern, 0);
+ else
+ dpcd_set_lane_settings(link, lt_settings, 0);
+
+ /* 3. wait for receiver to lock-on*/
+ wait_time_microsec = lt_settings->eq_pattern_time;
+
+ dp_wait_for_training_aux_rd_interval(
+ link,
+ wait_time_microsec);
+
+ /* 4. Read lane status and requested
+ * drive settings as set by the sink
+ */
+ dp_get_lane_status_and_lane_adjust(
+ link,
+ lt_settings,
+ dpcd_lane_status,
+ &dpcd_lane_status_updated,
+ dpcd_lane_adjust,
+ 0);
+
+ /* 5. check CR done*/
+ if (!dp_is_cr_done(lane_count, dpcd_lane_status)) {
+ status = LINK_TRAINING_EQ_FAIL_CR;
+ break;
+ }
+
+ /* 6. check CHEQ done*/
+ if (dp_is_ch_eq_done(lane_count, dpcd_lane_status) &&
+ dp_is_symbol_locked(lane_count, dpcd_lane_status) &&
+ dp_is_interlane_aligned(dpcd_lane_status_updated)) {
+ status = LINK_TRAINING_SUCCESS;
+ break;
+ }
+
+ /* 7. update VS/PE/PC2 in lt_settings*/
+ dp_decide_lane_settings(lt_settings, dpcd_lane_adjust,
+ lt_settings->hw_lane_settings, lt_settings->dpcd_lane_settings);
+ }
+ }
+
+ return status;
+}
+
+
enum link_training_result dc_link_dp_perform_link_training(
struct dc_link *link,
+ const struct link_resource *link_res,
const struct dc_link_settings *link_settings,
bool skip_video_pattern)
{
@@ -2203,30 +2795,51 @@ enum link_training_result dc_link_dp_perform_link_training(
<_settings);
/* reset previous training states */
- dpcd_exit_training_mode(link);
+ if (link->dc->debug.apply_vendor_specific_lttpr_wa &&
+ (link->chip_caps & EXT_DISPLAY_PATH_CAPS__DP_FIXED_VS_EN) &&
+ link->lttpr_mode == LTTPR_MODE_TRANSPARENT) {
+ link->apply_vendor_specific_lttpr_link_rate_wa = true;
+ vendor_specific_lttpr_wa_four(link, true);
+ } else {
+ dpcd_exit_training_mode(link);
+ }
/* configure link prior to entering training mode */
dpcd_configure_lttpr_mode(link, <_settings);
- dp_set_fec_ready(link, lt_settings.should_set_fec_ready);
+ dp_set_fec_ready(link, link_res, lt_settings.should_set_fec_ready);
dpcd_configure_channel_coding(link, <_settings);
/* enter training mode:
* Per DP specs starting from here, DPTX device shall not issue
* Non-LT AUX transactions inside training mode.
*/
- if (encoding == DP_8b_10b_ENCODING)
- status = dp_perform_8b_10b_link_training(link, <_settings);
+ if (!link->dc->debug.apply_vendor_specific_lttpr_wa &&
+ (link->chip_caps & EXT_DISPLAY_PATH_CAPS__DP_FIXED_VS_EN) &&
+ link->lttpr_mode == LTTPR_MODE_TRANSPARENT)
+ status = dc_link_dp_perform_fixed_vs_pe_training_sequence(link, link_res, <_settings);
+ else if (encoding == DP_8b_10b_ENCODING)
+ status = dp_perform_8b_10b_link_training(link, link_res, <_settings);
#if defined(CONFIG_DRM_AMD_DC_DCN)
else if (encoding == DP_128b_132b_ENCODING)
- status = dp_perform_128b_132b_link_training(link, <_settings);
+ status = dp_perform_128b_132b_link_training(link, link_res, <_settings);
#endif
else
ASSERT(0);
- /* exit training mode and switch to video idle */
- dpcd_exit_training_mode(link);
+ /* exit training mode */
+ if (link->dc->debug.apply_vendor_specific_lttpr_wa &&
+ (link->chip_caps & EXT_DISPLAY_PATH_CAPS__DP_FIXED_VS_EN) &&
+ link->lttpr_mode == LTTPR_MODE_TRANSPARENT) {
+ link->apply_vendor_specific_lttpr_link_rate_wa = false;
+ vendor_specific_lttpr_wa_four(link, (status != LINK_TRAINING_SUCCESS));
+ } else {
+ dpcd_exit_training_mode(link);
+ }
+
+ /* switch to video idle */
if ((status == LINK_TRAINING_SUCCESS) || !skip_video_pattern)
status = dp_transition_to_video_idle(link,
+ link_res,
<_settings,
status);
@@ -2278,6 +2891,7 @@ bool perform_link_training_with_retries(
dp_enable_link_phy(
link,
+ &pipe_ctx->link_res,
signal,
pipe_ctx->clock_source->id,
¤t_setting);
@@ -2305,23 +2919,24 @@ bool perform_link_training_with_retries(
dp_set_panel_mode(link, panel_mode);
if (link->aux_access_disabled) {
- dc_link_dp_perform_link_training_skip_aux(link, ¤t_setting);
+ dc_link_dp_perform_link_training_skip_aux(link, &pipe_ctx->link_res, ¤t_setting);
return true;
} else {
/** @todo Consolidate USB4 DP and DPx.x training. */
if (link->ep_type == DISPLAY_ENDPOINT_USB4_DPIA) {
status = dc_link_dpia_perform_link_training(link,
- ¤t_setting,
- skip_video_pattern);
+ &pipe_ctx->link_res,
+ ¤t_setting,
+ skip_video_pattern);
/* Transmit idle pattern once training successful. */
if (status == LINK_TRAINING_SUCCESS)
- dp_set_hw_test_pattern(link, DP_TEST_PATTERN_VIDEO_MODE,
- NULL, 0);
+ dp_set_hw_test_pattern(link, &pipe_ctx->link_res, DP_TEST_PATTERN_VIDEO_MODE, NULL, 0);
} else {
status = dc_link_dp_perform_link_training(link,
- ¤t_setting,
- skip_video_pattern);
+ &pipe_ctx->link_res,
+ ¤t_setting,
+ skip_video_pattern);
}
if (status == LINK_TRAINING_SUCCESS)
@@ -2336,7 +2951,7 @@ bool perform_link_training_with_retries(
DC_LOG_WARNING("%s: Link training attempt %u of %d failed\n",
__func__, (unsigned int)j + 1, attempts);
- dp_disable_link_phy(link, signal);
+ dp_disable_link_phy(link, &pipe_ctx->link_res, signal);
/* Abort link training if failure due to sink being unplugged. */
if (status == LINK_TRAINING_ABORT) {
@@ -2349,7 +2964,7 @@ bool perform_link_training_with_retries(
uint32_t req_bw;
uint32_t link_bw;
- decide_fallback_link_setting(*link_setting, ¤t_setting, status);
+ decide_fallback_link_setting(link, *link_setting, ¤t_setting, status);
/* Fail link training if reduced link bandwidth no longer meets
* stream requirements.
*/
@@ -2385,12 +3000,13 @@ static enum clock_source_id get_clock_source_id(struct dc_link *link)
return dp_cs_id;
}
-static void set_dp_mst_mode(struct dc_link *link, bool mst_enable)
+static void set_dp_mst_mode(struct dc_link *link, const struct link_resource *link_res,
+ bool mst_enable)
{
if (mst_enable == false &&
link->type == dc_connection_mst_branch) {
/* Disable MST on link. Use only local sink. */
- dp_disable_link_phy_mst(link, link->connector_signal);
+ dp_disable_link_phy_mst(link, link_res, link->connector_signal);
link->type = dc_connection_single;
link->local_sink = link->remote_sinks[0];
@@ -2401,7 +3017,7 @@ static void set_dp_mst_mode(struct dc_link *link, bool mst_enable)
link->type == dc_connection_single &&
link->remote_sinks[0] != NULL) {
/* Re-enable MST on link. */
- dp_disable_link_phy(link, link->connector_signal);
+ dp_disable_link_phy(link, link_res, link->connector_signal);
dp_enable_mst_on_sink(link, true);
link->type = dc_connection_mst_branch;
@@ -2427,6 +3043,7 @@ bool dc_link_dp_sync_lt_begin(struct dc_link *link)
enum link_training_result dc_link_dp_sync_lt_attempt(
struct dc_link *link,
+ const struct link_resource *link_res,
struct dc_link_settings *link_settings,
struct dc_link_training_overrides *lt_overrides)
{
@@ -2446,14 +3063,14 @@ enum link_training_result dc_link_dp_sync_lt_attempt(
<_settings);
/* Setup MST Mode */
if (lt_overrides->mst_enable)
- set_dp_mst_mode(link, *lt_overrides->mst_enable);
+ set_dp_mst_mode(link, link_res, *lt_overrides->mst_enable);
/* Disable link */
- dp_disable_link_phy(link, link->connector_signal);
+ dp_disable_link_phy(link, link_res, link->connector_signal);
/* Enable link */
dp_cs_id = get_clock_source_id(link);
- dp_enable_link_phy(link, link->connector_signal,
+ dp_enable_link_phy(link, link_res, link->connector_signal,
dp_cs_id, link_settings);
/* Set FEC enable */
@@ -2461,7 +3078,7 @@ enum link_training_result dc_link_dp_sync_lt_attempt(
if (dp_get_link_encoding_format(link_settings) == DP_8b_10b_ENCODING) {
#endif
fec_enable = lt_overrides->fec_enable && *lt_overrides->fec_enable;
- dp_set_fec_ready(link, fec_enable);
+ dp_set_fec_ready(link, NULL, fec_enable);
#if defined(CONFIG_DRM_AMD_DC_DCN)
}
#endif
@@ -2478,7 +3095,7 @@ enum link_training_result dc_link_dp_sync_lt_attempt(
/* Attempt to train with given link training settings */
if (link->ctx->dc->work_arounds.lt_early_cr_pattern)
- start_clock_recovery_pattern_early(link, <_settings, DPRX);
+ start_clock_recovery_pattern_early(link, link_res, <_settings, DPRX);
/* Set link rate, lane count and spread. */
dpcd_set_link_settings(link, <_settings);
@@ -2486,9 +3103,10 @@ enum link_training_result dc_link_dp_sync_lt_attempt(
/* 2. perform link training (set link training done
* to false is done as well)
*/
- lt_status = perform_clock_recovery_sequence(link, <_settings, DPRX);
+ lt_status = perform_clock_recovery_sequence(link, link_res, <_settings, DPRX);
if (lt_status == LINK_TRAINING_SUCCESS) {
lt_status = perform_channel_equalization_sequence(link,
+ link_res,
<_settings,
DPRX);
}
@@ -2509,11 +3127,11 @@ bool dc_link_dp_sync_lt_end(struct dc_link *link, bool link_down)
#if defined(CONFIG_DRM_AMD_DC_DCN)
struct dc_link_settings link_settings = link->cur_link_settings;
#endif
- dp_disable_link_phy(link, link->connector_signal);
+ dp_disable_link_phy(link, NULL, link->connector_signal);
#if defined(CONFIG_DRM_AMD_DC_DCN)
if (dp_get_link_encoding_format(&link_settings) == DP_8b_10b_ENCODING)
#endif
- dp_set_fec_ready(link, false);
+ dp_set_fec_ready(link, NULL, false);
}
link->sync_lt_in_progress = false;
@@ -2568,7 +3186,8 @@ bool dc_link_dp_get_max_link_enc_cap(const struct dc_link *link, struct dc_link_
return false;
}
-static struct dc_link_settings get_max_link_cap(struct dc_link *link)
+static struct dc_link_settings get_max_link_cap(struct dc_link *link,
+ const struct link_resource *link_res)
{
struct dc_link_settings max_link_cap = {0};
#if defined(CONFIG_DRM_AMD_DC_DCN)
@@ -2592,9 +3211,11 @@ static struct dc_link_settings get_max_link_cap(struct dc_link *link)
if (link_enc)
link_enc->funcs->get_max_link_cap(link_enc, &max_link_cap);
#if defined(CONFIG_DRM_AMD_DC_DCN)
- if (max_link_cap.link_rate >= LINK_RATE_UHBR10 &&
- !link->hpo_dp_link_enc)
- max_link_cap.link_rate = LINK_RATE_HIGH3;
+ if (max_link_cap.link_rate >= LINK_RATE_UHBR10) {
+ if (!link_res->hpo_dp_link_enc ||
+ link->dc->debug.disable_uhbr)
+ max_link_cap.link_rate = LINK_RATE_HIGH3;
+ }
#endif
/* Lower link settings based on sink's link cap */
@@ -2612,7 +3233,7 @@ static struct dc_link_settings get_max_link_cap(struct dc_link *link)
* account for lttpr repeaters cap
* notes: repeaters do not snoop in the DPRX Capabilities addresses (3.6.3).
*/
- if (link->lttpr_mode == LTTPR_MODE_NON_TRANSPARENT) {
+ if (link->lttpr_mode != LTTPR_MODE_NON_LTTPR) {
if (link->dpcd_caps.lttpr_caps.max_lane_count < max_link_cap.lane_count)
max_link_cap.lane_count = link->dpcd_caps.lttpr_caps.max_lane_count;
@@ -2751,6 +3372,7 @@ bool hpd_rx_irq_check_link_loss_status(
bool dp_verify_link_cap(
struct dc_link *link,
+ const struct link_resource *link_res,
struct dc_link_settings *known_limit_link_setting,
int *fail_count)
{
@@ -2768,7 +3390,7 @@ bool dp_verify_link_cap(
/* link training starts with the maximum common settings
* supported by both sink and ASIC.
*/
- max_link_cap = get_max_link_cap(link);
+ max_link_cap = get_max_link_cap(link, link_res);
initial_link_settings = get_common_supported_link_settings(
*known_limit_link_setting,
max_link_cap);
@@ -2808,7 +3430,7 @@ bool dp_verify_link_cap(
* find the physical link capability
*/
/* disable PHY done possible by BIOS, will be done by driver itself */
- dp_disable_link_phy(link, link->connector_signal);
+ dp_disable_link_phy(link, link_res, link->connector_signal);
dp_cs_id = get_clock_source_id(link);
@@ -2820,8 +3442,8 @@ bool dp_verify_link_cap(
*/
if (link->link_enc && link->link_enc->features.flags.bits.DP_IS_USB_C &&
link->dc->debug.usbc_combo_phy_reset_wa) {
- dp_enable_link_phy(link, link->connector_signal, dp_cs_id, cur);
- dp_disable_link_phy(link, link->connector_signal);
+ dp_enable_link_phy(link, link_res, link->connector_signal, dp_cs_id, cur);
+ dp_disable_link_phy(link, link_res, link->connector_signal);
}
do {
@@ -2832,6 +3454,7 @@ bool dp_verify_link_cap(
dp_enable_link_phy(
link,
+ link_res,
link->connector_signal,
dp_cs_id,
cur);
@@ -2842,6 +3465,7 @@ bool dp_verify_link_cap(
else {
status = dc_link_dp_perform_link_training(
link,
+ link_res,
cur,
skip_video_pattern);
if (status == LINK_TRAINING_SUCCESS)
@@ -2863,8 +3487,8 @@ bool dp_verify_link_cap(
* setting or before returning we'll enable it later
* based on the actual mode we're driving
*/
- dp_disable_link_phy(link, link->connector_signal);
- } while (!success && decide_fallback_link_setting(
+ dp_disable_link_phy(link, link_res, link->connector_signal);
+ } while (!success && decide_fallback_link_setting(link,
initial_link_settings, cur, status));
/* Link Training failed for all Link Settings
@@ -2887,6 +3511,7 @@ bool dp_verify_link_cap(
bool dp_verify_link_cap_with_retries(
struct dc_link *link,
+ const struct link_resource *link_res,
struct dc_link_settings *known_limit_link_setting,
int attempts)
{
@@ -2904,7 +3529,7 @@ bool dp_verify_link_cap_with_retries(
link->verified_link_cap.link_rate = LINK_RATE_LOW;
link->verified_link_cap.link_spread = LINK_SPREAD_DISABLED;
break;
- } else if (dp_verify_link_cap(link,
+ } else if (dp_verify_link_cap(link, link_res,
known_limit_link_setting,
&fail_count) && fail_count == 0) {
success = true;
@@ -2916,13 +3541,13 @@ bool dp_verify_link_cap_with_retries(
}
bool dp_verify_mst_link_cap(
- struct dc_link *link)
+ struct dc_link *link, const struct link_resource *link_res)
{
struct dc_link_settings max_link_cap = {0};
if (dp_get_link_encoding_format(&link->reported_link_cap) ==
DP_8b_10b_ENCODING) {
- max_link_cap = get_max_link_cap(link);
+ max_link_cap = get_max_link_cap(link, link_res);
link->verified_link_cap = get_common_supported_link_settings(
link->reported_link_cap,
max_link_cap);
@@ -2931,6 +3556,7 @@ bool dp_verify_mst_link_cap(
else if (dp_get_link_encoding_format(&link->reported_link_cap) ==
DP_128b_132b_ENCODING) {
dp_verify_link_cap_with_retries(link,
+ link_res,
&link->reported_link_cap,
LINK_TRAINING_MAX_VERIFY_RETRY);
}
@@ -3116,6 +3742,7 @@ static bool decide_fallback_link_setting_max_bw_policy(
* and no further fallback could be done
*/
static bool decide_fallback_link_setting(
+ struct dc_link *link,
struct dc_link_settings initial_link_settings,
struct dc_link_settings *current_link_setting,
enum link_training_result training_result)
@@ -3123,7 +3750,8 @@ static bool decide_fallback_link_setting(
if (!current_link_setting)
return false;
#if defined(CONFIG_DRM_AMD_DC_DCN)
- if (dp_get_link_encoding_format(&initial_link_settings) == DP_128b_132b_ENCODING)
+ if (dp_get_link_encoding_format(&initial_link_settings) == DP_128b_132b_ENCODING ||
+ link->dc->debug.force_dp2_lt_fallback_method)
return decide_fallback_link_setting_max_bw_policy(&initial_link_settings,
current_link_setting);
#endif
@@ -3346,6 +3974,148 @@ bool decide_edp_link_settings(struct dc_link *link, struct dc_link_settings *lin
return false;
}
+static bool decide_edp_link_settings_with_dsc(struct dc_link *link,
+ struct dc_link_settings *link_setting,
+ uint32_t req_bw,
+ enum dc_link_rate max_link_rate)
+{
+ struct dc_link_settings initial_link_setting;
+ struct dc_link_settings current_link_setting;
+ uint32_t link_bw;
+
+ unsigned int policy = 0;
+
+ policy = link->ctx->dc->debug.force_dsc_edp_policy;
+ if (max_link_rate == LINK_RATE_UNKNOWN)
+ max_link_rate = link->verified_link_cap.link_rate;
+ /*
+ * edp_supported_link_rates_count is only valid for eDP v1.4 or higher.
+ * Per VESA eDP spec, "The DPCD revision for eDP v1.4 is 13h"
+ */
+ if ((link->dpcd_caps.dpcd_rev.raw < DPCD_REV_13 ||
+ link->dpcd_caps.edp_supported_link_rates_count == 0)) {
+ /* for DSC enabled case, we search for minimum lane count */
+ memset(&initial_link_setting, 0, sizeof(initial_link_setting));
+ initial_link_setting.lane_count = LANE_COUNT_ONE;
+ initial_link_setting.link_rate = LINK_RATE_LOW;
+ initial_link_setting.link_spread = LINK_SPREAD_DISABLED;
+ initial_link_setting.use_link_rate_set = false;
+ initial_link_setting.link_rate_set = 0;
+ current_link_setting = initial_link_setting;
+ if (req_bw > dc_link_bandwidth_kbps(link, &link->verified_link_cap))
+ return false;
+
+ /* search for the minimum link setting that:
+ * 1. is supported according to the link training result
+ * 2. could support the b/w requested by the timing
+ */
+ while (current_link_setting.link_rate <=
+ max_link_rate) {
+ link_bw = dc_link_bandwidth_kbps(
+ link,
+ ¤t_link_setting);
+ if (req_bw <= link_bw) {
+ *link_setting = current_link_setting;
+ return true;
+ }
+ if (policy) {
+ /* minimize lane */
+ if (current_link_setting.link_rate < max_link_rate) {
+ current_link_setting.link_rate =
+ increase_link_rate(
+ current_link_setting.link_rate);
+ } else {
+ if (current_link_setting.lane_count <
+ link->verified_link_cap.lane_count) {
+ current_link_setting.lane_count =
+ increase_lane_count(
+ current_link_setting.lane_count);
+ current_link_setting.link_rate = initial_link_setting.link_rate;
+ } else
+ break;
+ }
+ } else {
+ /* minimize link rate */
+ if (current_link_setting.lane_count <
+ link->verified_link_cap.lane_count) {
+ current_link_setting.lane_count =
+ increase_lane_count(
+ current_link_setting.lane_count);
+ } else {
+ current_link_setting.link_rate =
+ increase_link_rate(
+ current_link_setting.link_rate);
+ current_link_setting.lane_count =
+ initial_link_setting.lane_count;
+ }
+ }
+ }
+ return false;
+ }
+
+ /* if optimize edp link is supported */
+ memset(&initial_link_setting, 0, sizeof(initial_link_setting));
+ initial_link_setting.lane_count = LANE_COUNT_ONE;
+ initial_link_setting.link_rate = link->dpcd_caps.edp_supported_link_rates[0];
+ initial_link_setting.link_spread = LINK_SPREAD_DISABLED;
+ initial_link_setting.use_link_rate_set = true;
+ initial_link_setting.link_rate_set = 0;
+ current_link_setting = initial_link_setting;
+
+ /* search for the minimum link setting that:
+ * 1. is supported according to the link training result
+ * 2. could support the b/w requested by the timing
+ */
+ while (current_link_setting.link_rate <=
+ max_link_rate) {
+ link_bw = dc_link_bandwidth_kbps(
+ link,
+ ¤t_link_setting);
+ if (req_bw <= link_bw) {
+ *link_setting = current_link_setting;
+ return true;
+ }
+ if (policy) {
+ /* minimize lane */
+ if (current_link_setting.link_rate_set <
+ link->dpcd_caps.edp_supported_link_rates_count
+ && current_link_setting.link_rate < max_link_rate) {
+ current_link_setting.link_rate_set++;
+ current_link_setting.link_rate =
+ link->dpcd_caps.edp_supported_link_rates[current_link_setting.link_rate_set];
+ } else {
+ if (current_link_setting.lane_count < link->verified_link_cap.lane_count) {
+ current_link_setting.lane_count =
+ increase_lane_count(
+ current_link_setting.lane_count);
+ current_link_setting.link_rate_set = initial_link_setting.link_rate_set;
+ current_link_setting.link_rate =
+ link->dpcd_caps.edp_supported_link_rates[current_link_setting.link_rate_set];
+ } else
+ break;
+ }
+ } else {
+ /* minimize link rate */
+ if (current_link_setting.lane_count <
+ link->verified_link_cap.lane_count) {
+ current_link_setting.lane_count =
+ increase_lane_count(
+ current_link_setting.lane_count);
+ } else {
+ if (current_link_setting.link_rate_set < link->dpcd_caps.edp_supported_link_rates_count) {
+ current_link_setting.link_rate_set++;
+ current_link_setting.link_rate =
+ link->dpcd_caps.edp_supported_link_rates[current_link_setting.link_rate_set];
+ current_link_setting.lane_count =
+ initial_link_setting.lane_count;
+ } else
+ break;
+ }
+ }
+ }
+ return false;
+}
+
static bool decide_mst_link_settings(const struct dc_link *link, struct dc_link_settings *link_setting)
{
*link_setting = link->verified_link_cap;
@@ -3380,7 +4150,25 @@ void decide_link_settings(struct dc_stream_state *stream,
if (decide_mst_link_settings(link, link_setting))
return;
} else if (link->connector_signal == SIGNAL_TYPE_EDP) {
- if (decide_edp_link_settings(link, link_setting, req_bw))
+ /* enable edp link optimization for DSC eDP case */
+ if (stream->timing.flags.DSC) {
+ enum dc_link_rate max_link_rate = LINK_RATE_UNKNOWN;
+
+ if (link->ctx->dc->debug.force_dsc_edp_policy) {
+ /* calculate link max link rate cap*/
+ struct dc_link_settings tmp_link_setting;
+ struct dc_crtc_timing tmp_timing = stream->timing;
+ uint32_t orig_req_bw;
+
+ tmp_link_setting.link_rate = LINK_RATE_UNKNOWN;
+ tmp_timing.flags.DSC = 0;
+ orig_req_bw = dc_bandwidth_in_kbps_from_timing(&tmp_timing);
+ decide_edp_link_settings(link, &tmp_link_setting, orig_req_bw);
+ max_link_rate = tmp_link_setting.link_rate;
+ }
+ if (decide_edp_link_settings_with_dsc(link, link_setting, req_bw, max_link_rate))
+ return;
+ } else if (decide_edp_link_settings(link, link_setting, req_bw))
return;
} else if (decide_dp_link_settings(link, link_setting, req_bw))
return;
@@ -3421,7 +4209,6 @@ static bool handle_hpd_irq_psr_sink(struct dc_link *link)
&psr_configuration.raw,
sizeof(psr_configuration.raw));
-
if (psr_configuration.bits.ENABLE) {
unsigned char dpcdbuf[3] = {0};
union psr_error_status psr_error_status;
@@ -3453,10 +4240,12 @@ static bool handle_hpd_irq_psr_sink(struct dc_link *link)
sizeof(psr_error_status.raw));
/* PSR error, disable and re-enable PSR */
- allow_active = false;
- dc_link_set_psr_allow_active(link, &allow_active, true, false, NULL);
- allow_active = true;
- dc_link_set_psr_allow_active(link, &allow_active, true, false, NULL);
+ if (link->psr_settings.psr_allow_active) {
+ allow_active = false;
+ dc_link_set_psr_allow_active(link, &allow_active, true, false, NULL);
+ allow_active = true;
+ dc_link_set_psr_allow_active(link, &allow_active, true, false, NULL);
+ }
return true;
} else if (psr_sink_psr_status.bits.SINK_SELF_REFRESH_STATUS ==
@@ -3534,6 +4323,13 @@ static void dp_test_send_phy_test_pattern(struct dc_link *link)
&dpcd_lane_adjustment[0].raw,
sizeof(dpcd_lane_adjustment));
+ if (link->dc->debug.apply_vendor_specific_lttpr_wa &&
+ (link->chip_caps & EXT_DISPLAY_PATH_CAPS__DP_FIXED_VS_EN) &&
+ link->lttpr_mode == LTTPR_MODE_TRANSPARENT)
+ vendor_specific_lttpr_wa_three_dpcd(
+ link,
+ link_training_settings.dpcd_lane_settings);
+
/*get post cursor 2 parameters
* For DP 1.1a or eariler, this DPCD register's value is 0
* For DP 1.2 or later:
@@ -4153,6 +4949,56 @@ static int translate_dpcd_max_bpc(enum dpcd_downstream_port_max_bpc bpc)
return -1;
}
+#if defined(CONFIG_DRM_AMD_DC_DCN)
+uint32_t dc_link_bw_kbps_from_raw_frl_link_rate_data(uint8_t bw)
+{
+ switch (bw) {
+ case 0b001:
+ return 9000000;
+ case 0b010:
+ return 18000000;
+ case 0b011:
+ return 24000000;
+ case 0b100:
+ return 32000000;
+ case 0b101:
+ return 40000000;
+ case 0b110:
+ return 48000000;
+ }
+
+ return 0;
+}
+
+/**
+ * Return PCON's post FRL link training supported BW if its non-zero, otherwise return max_supported_frl_bw.
+ */
+static uint32_t intersect_frl_link_bw_support(
+ const uint32_t max_supported_frl_bw_in_kbps,
+ const union hdmi_encoded_link_bw hdmi_encoded_link_bw)
+{
+ uint32_t supported_bw_in_kbps = max_supported_frl_bw_in_kbps;
+
+ // HDMI_ENCODED_LINK_BW bits are only valid if HDMI Link Configuration bit is 1 (FRL mode)
+ if (hdmi_encoded_link_bw.bits.FRL_MODE) {
+ if (hdmi_encoded_link_bw.bits.BW_48Gbps)
+ supported_bw_in_kbps = 48000000;
+ else if (hdmi_encoded_link_bw.bits.BW_40Gbps)
+ supported_bw_in_kbps = 40000000;
+ else if (hdmi_encoded_link_bw.bits.BW_32Gbps)
+ supported_bw_in_kbps = 32000000;
+ else if (hdmi_encoded_link_bw.bits.BW_24Gbps)
+ supported_bw_in_kbps = 24000000;
+ else if (hdmi_encoded_link_bw.bits.BW_18Gbps)
+ supported_bw_in_kbps = 18000000;
+ else if (hdmi_encoded_link_bw.bits.BW_9Gbps)
+ supported_bw_in_kbps = 9000000;
+ }
+
+ return supported_bw_in_kbps;
+}
+#endif
+
static void read_dp_device_vendor_id(struct dc_link *link)
{
struct dp_device_vendor_id dp_id;
@@ -4264,6 +5110,27 @@ static void get_active_converter_info(
translate_dpcd_max_bpc(
hdmi_color_caps.bits.MAX_BITS_PER_COLOR_COMPONENT);
+#if defined(CONFIG_DRM_AMD_DC_DCN)
+ if (link->dc->caps.hdmi_frl_pcon_support) {
+ union hdmi_encoded_link_bw hdmi_encoded_link_bw;
+
+ link->dpcd_caps.dongle_caps.dp_hdmi_frl_max_link_bw_in_kbps =
+ dc_link_bw_kbps_from_raw_frl_link_rate_data(
+ hdmi_color_caps.bits.MAX_ENCODED_LINK_BW_SUPPORT);
+
+ // Intersect reported max link bw support with the supported link rate post FRL link training
+ if (core_link_read_dpcd(link, DP_PCON_HDMI_POST_FRL_STATUS,
+ &hdmi_encoded_link_bw.raw, sizeof(hdmi_encoded_link_bw)) == DC_OK) {
+ link->dpcd_caps.dongle_caps.dp_hdmi_frl_max_link_bw_in_kbps = intersect_frl_link_bw_support(
+ link->dpcd_caps.dongle_caps.dp_hdmi_frl_max_link_bw_in_kbps,
+ hdmi_encoded_link_bw);
+ }
+
+ if (link->dpcd_caps.dongle_caps.dp_hdmi_frl_max_link_bw_in_kbps > 0)
+ link->dpcd_caps.dongle_caps.extendedCapValid = true;
+ }
+#endif
+
if (link->dpcd_caps.dongle_caps.dp_hdmi_max_pixel_clk_in_khz != 0)
link->dpcd_caps.dongle_caps.extendedCapValid = true;
}
@@ -4454,7 +5321,7 @@ bool dp_retrieve_lttpr_cap(struct dc_link *link)
lttpr_dpcd_data,
sizeof(lttpr_dpcd_data));
if (status != DC_OK) {
- dm_error("%s: Read LTTPR caps data failed.\n", __func__);
+ DC_LOG_DP2("%s: Read LTTPR caps data failed.\n", __func__);
return false;
}
@@ -5218,7 +6085,7 @@ bool dc_link_dp_set_test_pattern(
DP_TEST_PATTERN_VIDEO_MODE) {
/* Set CRTC Test Pattern */
set_crtc_test_pattern(link, pipe_ctx, test_pattern, test_pattern_color_space);
- dp_set_hw_test_pattern(link, test_pattern,
+ dp_set_hw_test_pattern(link, &pipe_ctx->link_res, test_pattern,
(uint8_t *)p_custom_pattern,
(uint32_t)cust_pattern_size);
@@ -5240,8 +6107,18 @@ bool dc_link_dp_set_test_pattern(
if (is_dp_phy_pattern(test_pattern)) {
/* Set DPCD Lane Settings before running test pattern */
if (p_link_settings != NULL) {
- dp_set_hw_lane_settings(link, p_link_settings, DPRX);
- dpcd_set_lane_settings(link, p_link_settings, DPRX);
+ if (link->dc->debug.apply_vendor_specific_lttpr_wa &&
+ (link->chip_caps & EXT_DISPLAY_PATH_CAPS__DP_FIXED_VS_EN) &&
+ link->lttpr_mode == LTTPR_MODE_TRANSPARENT) {
+ dpcd_set_lane_settings(link, p_link_settings, DPRX);
+ vendor_specific_lttpr_wa_five(
+ link,
+ p_link_settings->dpcd_lane_settings,
+ p_link_settings->link_settings.lane_count);
+ } else {
+ dp_set_hw_lane_settings(link, &pipe_ctx->link_res, p_link_settings, DPRX);
+ dpcd_set_lane_settings(link, p_link_settings, DPRX);
+ }
}
/* Blank stream if running test pattern */
@@ -5254,7 +6131,7 @@ bool dc_link_dp_set_test_pattern(
pipes->stream_res.stream_enc->funcs->dp_blank(link, pipe_ctx->stream_res.stream_enc);
}
- dp_set_hw_test_pattern(link, test_pattern,
+ dp_set_hw_test_pattern(link, &pipe_ctx->link_res, test_pattern,
(uint8_t *)p_custom_pattern,
(uint32_t)cust_pattern_size);
@@ -5574,7 +6451,7 @@ enum dp_panel_mode dp_get_panel_mode(struct dc_link *link)
return DP_PANEL_MODE_DEFAULT;
}
-enum dc_status dp_set_fec_ready(struct dc_link *link, bool ready)
+enum dc_status dp_set_fec_ready(struct dc_link *link, const struct link_resource *link_res, bool ready)
{
/* FEC has to be "set ready" before the link training.
* The policy is to always train with FEC
@@ -5665,6 +6542,23 @@ void dp_set_fec_enable(struct dc_link *link, bool enable)
}
}
+struct link_encoder *dp_get_link_enc(struct dc_link *link)
+{
+ struct link_encoder *link_enc;
+
+ link_enc = link->link_enc;
+ if (link->is_dig_mapping_flexible &&
+ link->dc->res_pool->funcs->link_encs_assign) {
+ link_enc = link_enc_cfg_get_link_enc_used_by_link(link->ctx->dc,
+ link);
+ if (!link->link_enc)
+ link_enc = link_enc_cfg_get_next_avail_link_enc(
+ link->ctx->dc);
+ }
+
+ return link_enc;
+}
+
void dpcd_set_source_specific_data(struct dc_link *link)
{
if (!link->dc->vendor_signature.is_valid) {
@@ -5885,7 +6779,10 @@ bool is_edp_ilr_optimization_required(struct dc_link *link, struct dc_crtc_timin
req_bw = dc_bandwidth_in_kbps_from_timing(crtc_timing);
- decide_edp_link_settings(link, &link_setting, req_bw);
+ if (!crtc_timing->flags.DSC)
+ decide_edp_link_settings(link, &link_setting, req_bw);
+ else
+ decide_edp_link_settings_with_dsc(link, &link_setting, req_bw, LINK_RATE_UNKNOWN);
if (link->dpcd_caps.edp_supported_link_rates[link_rate_set] != link_setting.link_rate ||
lane_count_set.bits.LANE_COUNT_SET != link_setting.lane_count) {
@@ -6121,8 +7018,21 @@ struct fixed31_32 calculate_sst_avg_time_slots_per_mtp(
bool is_dp_128b_132b_signal(struct pipe_ctx *pipe_ctx)
{
+ /* If this assert is hit then we have a link encoder dynamic management issue */
+ ASSERT(pipe_ctx->stream_res.hpo_dp_stream_enc ? pipe_ctx->link_res.hpo_dp_link_enc != NULL : true);
return (pipe_ctx->stream_res.hpo_dp_stream_enc &&
- pipe_ctx->stream->link->hpo_dp_link_enc &&
+ pipe_ctx->link_res.hpo_dp_link_enc &&
dc_is_dp_signal(pipe_ctx->stream->signal));
}
#endif
+
+void edp_panel_backlight_power_on(struct dc_link *link)
+{
+ if (link->connector_signal != SIGNAL_TYPE_EDP)
+ return;
+
+ link->dc->hwss.edp_power_control(link, true);
+ link->dc->hwss.edp_wait_for_hpd_ready(link, true);
+ if (link->dc->hwss.edp_backlight_control)
+ link->dc->hwss.edp_backlight_control(link, true);
+}
diff --git a/drivers/gpu/drm/amd/display/dc/core/dc_link_dpia.c b/drivers/gpu/drm/amd/display/dc/core/dc_link_dpia.c
index b1c9f77..0e95bc5 100644
--- a/drivers/gpu/drm/amd/display/dc/core/dc_link_dpia.c
+++ b/drivers/gpu/drm/amd/display/dc/core/dc_link_dpia.c
@@ -77,7 +77,9 @@ enum dc_status dpcd_get_tunneling_device_data(struct dc_link *link)
* @param[in] link_setting Lane count, link rate and downspread control.
* @param[out] lt_settings Link settings and drive settings (voltage swing and pre-emphasis).
*/
-static enum link_training_result dpia_configure_link(struct dc_link *link,
+static enum link_training_result dpia_configure_link(
+ struct dc_link *link,
+ const struct link_resource *link_res,
const struct dc_link_settings *link_setting,
struct link_training_settings *lt_settings)
{
@@ -94,25 +96,25 @@ static enum link_training_result dpia_configure_link(struct dc_link *link,
lt_settings);
status = dpcd_configure_channel_coding(link, lt_settings);
- if (status != DC_OK && !link->hpd_status)
+ if (status != DC_OK && link->is_hpd_pending)
return LINK_TRAINING_ABORT;
/* Configure lttpr mode */
status = dpcd_configure_lttpr_mode(link, lt_settings);
- if (status != DC_OK && !link->hpd_status)
+ if (status != DC_OK && link->is_hpd_pending)
return LINK_TRAINING_ABORT;
/* Set link rate, lane count and spread. */
status = dpcd_set_link_settings(link, lt_settings);
- if (status != DC_OK && !link->hpd_status)
+ if (status != DC_OK && link->is_hpd_pending)
return LINK_TRAINING_ABORT;
if (link->preferred_training_settings.fec_enable)
fec_enable = *link->preferred_training_settings.fec_enable;
else
fec_enable = true;
- status = dp_set_fec_ready(link, fec_enable);
- if (status != DC_OK && !link->hpd_status)
+ status = dp_set_fec_ready(link, link_res, fec_enable);
+ if (status != DC_OK && link->is_hpd_pending)
return LINK_TRAINING_ABORT;
return LINK_TRAINING_SUCCESS;
@@ -252,7 +254,9 @@ static enum dc_status dpcd_set_lt_pattern(struct dc_link *link,
* @param lt_settings link_setting and drive settings (voltage swing and pre-emphasis).
* @param hop The Hop in display path. DPRX = 0.
*/
-static enum link_training_result dpia_training_cr_non_transparent(struct dc_link *link,
+static enum link_training_result dpia_training_cr_non_transparent(
+ struct dc_link *link,
+ const struct link_resource *link_res,
struct link_training_settings *lt_settings,
uint32_t hop)
{
@@ -388,7 +392,7 @@ static enum link_training_result dpia_training_cr_non_transparent(struct dc_link
}
/* Abort link training if clock recovery failed due to HPD unplug. */
- if (!link->hpd_status)
+ if (link->is_hpd_pending)
result = LINK_TRAINING_ABORT;
DC_LOG_HW_LINK_TRAINING("%s\n DPIA(%d) clock recovery\n"
@@ -411,7 +415,9 @@ static enum link_training_result dpia_training_cr_non_transparent(struct dc_link
* @param link DPIA link being trained.
* @param lt_settings link_setting and drive settings (voltage swing and pre-emphasis).
*/
-static enum link_training_result dpia_training_cr_transparent(struct dc_link *link,
+static enum link_training_result dpia_training_cr_transparent(
+ struct dc_link *link,
+ const struct link_resource *link_res,
struct link_training_settings *lt_settings)
{
enum link_training_result result = LINK_TRAINING_CR_FAIL_LANE0;
@@ -490,7 +496,7 @@ static enum link_training_result dpia_training_cr_transparent(struct dc_link *li
}
/* Abort link training if clock recovery failed due to HPD unplug. */
- if (!link->hpd_status)
+ if (link->is_hpd_pending)
result = LINK_TRAINING_ABORT;
DC_LOG_HW_LINK_TRAINING("%s\n DPIA(%d) clock recovery\n"
@@ -511,16 +517,18 @@ static enum link_training_result dpia_training_cr_transparent(struct dc_link *li
* @param lt_settings link_setting and drive settings (voltage swing and pre-emphasis).
* @param hop The Hop in display path. DPRX = 0.
*/
-static enum link_training_result dpia_training_cr_phase(struct dc_link *link,
+static enum link_training_result dpia_training_cr_phase(
+ struct dc_link *link,
+ const struct link_resource *link_res,
struct link_training_settings *lt_settings,
uint32_t hop)
{
enum link_training_result result = LINK_TRAINING_CR_FAIL_LANE0;
if (link->lttpr_mode == LTTPR_MODE_NON_TRANSPARENT)
- result = dpia_training_cr_non_transparent(link, lt_settings, hop);
+ result = dpia_training_cr_non_transparent(link, link_res, lt_settings, hop);
else
- result = dpia_training_cr_transparent(link, lt_settings);
+ result = dpia_training_cr_transparent(link, link_res, lt_settings);
return result;
}
@@ -561,7 +569,9 @@ static uint32_t dpia_get_eq_aux_rd_interval(const struct dc_link *link,
* @param lt_settings link_setting and drive settings (voltage swing and pre-emphasis).
* @param hop The Hop in display path. DPRX = 0.
*/
-static enum link_training_result dpia_training_eq_non_transparent(struct dc_link *link,
+static enum link_training_result dpia_training_eq_non_transparent(
+ struct dc_link *link,
+ const struct link_resource *link_res,
struct link_training_settings *lt_settings,
uint32_t hop)
{
@@ -675,7 +685,7 @@ static enum link_training_result dpia_training_eq_non_transparent(struct dc_link
}
/* Abort link training if equalization failed due to HPD unplug. */
- if (!link->hpd_status)
+ if (link->is_hpd_pending)
result = LINK_TRAINING_ABORT;
DC_LOG_HW_LINK_TRAINING("%s\n DPIA(%d) equalization\n"
@@ -700,7 +710,9 @@ static enum link_training_result dpia_training_eq_non_transparent(struct dc_link
* @param lt_settings link_setting and drive settings (voltage swing and pre-emphasis).
* @param hop The Hop in display path. DPRX = 0.
*/
-static enum link_training_result dpia_training_eq_transparent(struct dc_link *link,
+static enum link_training_result dpia_training_eq_transparent(
+ struct dc_link *link,
+ const struct link_resource *link_res,
struct link_training_settings *lt_settings)
{
enum link_training_result result = LINK_TRAINING_EQ_FAIL_EQ;
@@ -758,7 +770,7 @@ static enum link_training_result dpia_training_eq_transparent(struct dc_link *li
}
/* Abort link training if equalization failed due to HPD unplug. */
- if (!link->hpd_status)
+ if (link->is_hpd_pending)
result = LINK_TRAINING_ABORT;
DC_LOG_HW_LINK_TRAINING("%s\n DPIA(%d) equalization\n"
@@ -779,16 +791,18 @@ static enum link_training_result dpia_training_eq_transparent(struct dc_link *li
* @param lt_settings link_setting and drive settings (voltage swing and pre-emphasis).
* @param hop The Hop in display path. DPRX = 0.
*/
-static enum link_training_result dpia_training_eq_phase(struct dc_link *link,
+static enum link_training_result dpia_training_eq_phase(
+ struct dc_link *link,
+ const struct link_resource *link_res,
struct link_training_settings *lt_settings,
uint32_t hop)
{
enum link_training_result result;
if (link->lttpr_mode == LTTPR_MODE_NON_TRANSPARENT)
- result = dpia_training_eq_non_transparent(link, lt_settings, hop);
+ result = dpia_training_eq_non_transparent(link, link_res, lt_settings, hop);
else
- result = dpia_training_eq_transparent(link, lt_settings);
+ result = dpia_training_eq_transparent(link, link_res, lt_settings);
return result;
}
@@ -892,10 +906,10 @@ static void dpia_training_abort(struct dc_link *link, uint32_t hop)
__func__,
link->link_id.enum_id - ENUM_ID_1,
link->lttpr_mode,
- link->hpd_status);
+ link->is_hpd_pending);
/* Abandon clean-up if sink unplugged. */
- if (!link->hpd_status)
+ if (link->is_hpd_pending)
return;
if (hop != DPRX)
@@ -908,7 +922,9 @@ static void dpia_training_abort(struct dc_link *link, uint32_t hop)
core_link_send_set_config(link, DPIA_SET_CFG_SET_LINK, data);
}
-enum link_training_result dc_link_dpia_perform_link_training(struct dc_link *link,
+enum link_training_result dc_link_dpia_perform_link_training(
+ struct dc_link *link,
+ const struct link_resource *link_res,
const struct dc_link_settings *link_setting,
bool skip_video_pattern)
{
@@ -918,7 +934,7 @@ enum link_training_result dc_link_dpia_perform_link_training(struct dc_link *lin
int8_t repeater_id; /* Current hop. */
/* Configure link as prescribed in link_setting and set LTTPR mode. */
- result = dpia_configure_link(link, link_setting, <_settings);
+ result = dpia_configure_link(link, link_res, link_setting, <_settings);
if (result != LINK_TRAINING_SUCCESS)
return result;
@@ -930,12 +946,12 @@ enum link_training_result dc_link_dpia_perform_link_training(struct dc_link *lin
*/
for (repeater_id = repeater_cnt; repeater_id >= 0; repeater_id--) {
/* Clock recovery. */
- result = dpia_training_cr_phase(link, <_settings, repeater_id);
+ result = dpia_training_cr_phase(link, link_res, <_settings, repeater_id);
if (result != LINK_TRAINING_SUCCESS)
break;
/* Equalization. */
- result = dpia_training_eq_phase(link, <_settings, repeater_id);
+ result = dpia_training_eq_phase(link, link_res, <_settings, repeater_id);
if (result != LINK_TRAINING_SUCCESS)
break;
diff --git a/drivers/gpu/drm/amd/display/dc/core/dc_link_enc_cfg.c b/drivers/gpu/drm/amd/display/dc/core/dc_link_enc_cfg.c
index 25e48a8..a55944d 100644
--- a/drivers/gpu/drm/amd/display/dc/core/dc_link_enc_cfg.c
+++ b/drivers/gpu/drm/amd/display/dc/core/dc_link_enc_cfg.c
@@ -118,7 +118,10 @@ static void remove_link_enc_assignment(
*/
if (get_stream_using_link_enc(state, eng_id) == NULL)
state->res_ctx.link_enc_cfg_ctx.link_enc_avail[eng_idx] = eng_id;
+
stream->link_enc = NULL;
+ state->res_ctx.link_enc_cfg_ctx.link_enc_assignments[i].eng_id = ENGINE_ID_UNKNOWN;
+ state->res_ctx.link_enc_cfg_ctx.link_enc_assignments[i].stream = NULL;
break;
}
}
@@ -148,6 +151,7 @@ static void add_link_enc_assignment(
.ep_type = stream->link->ep_type},
.eng_id = eng_id,
.stream = stream};
+ dc_stream_retain(stream);
state->res_ctx.link_enc_cfg_ctx.link_enc_avail[eng_idx] = ENGINE_ID_UNKNOWN;
stream->link_enc = stream->ctx->dc->res_pool->link_encoders[eng_idx];
break;
@@ -227,7 +231,7 @@ static struct link_encoder *get_link_enc_used_by_link(
.link_id = link->link_id,
.ep_type = link->ep_type};
- for (i = 0; i < state->stream_count; i++) {
+ for (i = 0; i < MAX_PIPES; i++) {
struct link_enc_assignment assignment = state->res_ctx.link_enc_cfg_ctx.link_enc_assignments[i];
if (assignment.valid == true && are_ep_ids_equal(&assignment.ep_id, &ep_id))
@@ -237,28 +241,18 @@ static struct link_encoder *get_link_enc_used_by_link(
return link_enc;
}
/* Clear all link encoder assignments. */
-static void clear_enc_assignments(struct dc_state *state)
+static void clear_enc_assignments(const struct dc *dc, struct dc_state *state)
{
int i;
- enum engine_id eng_id;
- struct dc_stream_state *stream;
for (i = 0; i < MAX_PIPES; i++) {
state->res_ctx.link_enc_cfg_ctx.link_enc_assignments[i].valid = false;
- eng_id = state->res_ctx.link_enc_cfg_ctx.link_enc_assignments[i].eng_id;
- stream = state->res_ctx.link_enc_cfg_ctx.link_enc_assignments[i].stream;
- if (eng_id != ENGINE_ID_UNKNOWN)
- state->res_ctx.link_enc_cfg_ctx.link_enc_avail[eng_id - ENGINE_ID_DIGA] = eng_id;
- if (stream)
- stream->link_enc = NULL;
+ state->res_ctx.link_enc_cfg_ctx.link_enc_assignments[i].eng_id = ENGINE_ID_UNKNOWN;
+ if (state->res_ctx.link_enc_cfg_ctx.link_enc_assignments[i].stream != NULL) {
+ dc_stream_release(state->res_ctx.link_enc_cfg_ctx.link_enc_assignments[i].stream);
+ state->res_ctx.link_enc_cfg_ctx.link_enc_assignments[i].stream = NULL;
+ }
}
-}
-
-void link_enc_cfg_init(
- struct dc *dc,
- struct dc_state *state)
-{
- int i;
for (i = 0; i < dc->res_pool->res_cap->num_dig_link_enc; i++) {
if (dc->res_pool->link_encoders[i])
@@ -266,8 +260,13 @@ void link_enc_cfg_init(
else
state->res_ctx.link_enc_cfg_ctx.link_enc_avail[i] = ENGINE_ID_UNKNOWN;
}
+}
- clear_enc_assignments(state);
+void link_enc_cfg_init(
+ const struct dc *dc,
+ struct dc_state *state)
+{
+ clear_enc_assignments(dc, state);
state->res_ctx.link_enc_cfg_ctx.mode = LINK_ENC_CFG_STEADY;
}
@@ -284,12 +283,9 @@ void link_enc_cfg_link_encs_assign(
ASSERT(state->stream_count == stream_count);
- if (stream_count == 0)
- clear_enc_assignments(state);
-
/* Release DIG link encoder resources before running assignment algorithm. */
- for (i = 0; i < stream_count; i++)
- dc->res_pool->funcs->link_enc_unassign(state, streams[i]);
+ for (i = 0; i < dc->current_state->stream_count; i++)
+ dc->res_pool->funcs->link_enc_unassign(state, dc->current_state->streams[i]);
for (i = 0; i < MAX_PIPES; i++)
ASSERT(state->res_ctx.link_enc_cfg_ctx.link_enc_assignments[i].valid == false);
@@ -544,6 +540,7 @@ bool link_enc_cfg_validate(struct dc *dc, struct dc_state *state)
uint8_t dig_stream_count = 0;
int matching_stream_ptrs = 0;
int eng_ids_per_ep_id[MAX_PIPES] = {0};
+ int valid_bitmap = 0;
/* (1) No. valid entries same as stream count. */
for (i = 0; i < MAX_PIPES; i++) {
@@ -625,5 +622,15 @@ bool link_enc_cfg_validate(struct dc *dc, struct dc_state *state)
is_valid = valid_entries && valid_stream_ptrs && valid_uniqueness && valid_avail && valid_streams;
ASSERT(is_valid);
+ if (is_valid == false) {
+ valid_bitmap =
+ (valid_entries & 0x1) |
+ ((valid_stream_ptrs & 0x1) << 1) |
+ ((valid_uniqueness & 0x1) << 2) |
+ ((valid_avail & 0x1) << 3) |
+ ((valid_streams & 0x1) << 4);
+ dm_error("Invalid link encoder assignments: 0x%x\n", valid_bitmap);
+ }
+
return is_valid;
}
diff --git a/drivers/gpu/drm/amd/display/dc/core/dc_link_hwss.c b/drivers/gpu/drm/amd/display/dc/core/dc_link_hwss.c
index 368e834..45d03d3 100644
--- a/drivers/gpu/drm/amd/display/dc/core/dc_link_hwss.c
+++ b/drivers/gpu/drm/amd/display/dc/core/dc_link_hwss.c
@@ -71,6 +71,7 @@ void dp_source_sequence_trace(struct dc_link *link, uint8_t dp_test_mode)
void dp_enable_link_phy(
struct dc_link *link,
+ const struct link_resource *link_res,
enum signal_type signal,
enum clock_source_id clock_source,
const struct dc_link_settings *link_settings)
@@ -135,7 +136,7 @@ void dp_enable_link_phy(
#if defined(CONFIG_DRM_AMD_DC_DCN)
if (dp_get_link_encoding_format(link_settings) == DP_128b_132b_ENCODING) {
- enable_dp_hpo_output(link, link_settings);
+ enable_dp_hpo_output(link, link_res, link_settings);
} else if (dp_get_link_encoding_format(link_settings) == DP_8b_10b_ENCODING) {
if (dc_is_dp_sst_signal(signal)) {
link_enc->funcs->enable_dp_output(
@@ -236,12 +237,13 @@ bool edp_receiver_ready_T7(struct dc_link *link)
return result;
}
-void dp_disable_link_phy(struct dc_link *link, enum signal_type signal)
+void dp_disable_link_phy(struct dc_link *link, const struct link_resource *link_res,
+ enum signal_type signal)
{
struct dc *dc = link->ctx->dc;
struct dmcu *dmcu = dc->res_pool->dmcu;
#if defined(CONFIG_DRM_AMD_DC_DCN)
- struct hpo_dp_link_encoder *hpo_link_enc = link->hpo_dp_link_enc;
+ struct hpo_dp_link_encoder *hpo_link_enc = link_res->hpo_dp_link_enc;
#endif
struct link_encoder *link_enc;
@@ -260,7 +262,7 @@ void dp_disable_link_phy(struct dc_link *link, enum signal_type signal)
link->dc->hwss.edp_backlight_control(link, false);
#if defined(CONFIG_DRM_AMD_DC_DCN)
if (dp_get_link_encoding_format(&link->cur_link_settings) == DP_128b_132b_ENCODING)
- disable_dp_hpo_output(link, signal);
+ disable_dp_hpo_output(link, link_res, signal);
else
link_enc->funcs->disable_output(link_enc, signal);
#else
@@ -274,7 +276,7 @@ void dp_disable_link_phy(struct dc_link *link, enum signal_type signal)
#if defined(CONFIG_DRM_AMD_DC_DCN)
if (dp_get_link_encoding_format(&link->cur_link_settings) == DP_128b_132b_ENCODING &&
hpo_link_enc)
- disable_dp_hpo_output(link, signal);
+ disable_dp_hpo_output(link, link_res, signal);
else
link_enc->funcs->disable_output(link_enc, signal);
#else
@@ -294,13 +296,14 @@ void dp_disable_link_phy(struct dc_link *link, enum signal_type signal)
dc->clk_mgr->funcs->notify_link_rate_change(dc->clk_mgr, link);
}
-void dp_disable_link_phy_mst(struct dc_link *link, enum signal_type signal)
+void dp_disable_link_phy_mst(struct dc_link *link, const struct link_resource *link_res,
+ enum signal_type signal)
{
/* MST disable link only when no stream use the link */
if (link->mst_stream_alloc_table.stream_count > 0)
return;
- dp_disable_link_phy(link, signal);
+ dp_disable_link_phy(link, link_res, signal);
/* set the sink to SST mode after disabling the link */
dp_enable_mst_on_sink(link, false);
@@ -308,6 +311,7 @@ void dp_disable_link_phy_mst(struct dc_link *link, enum signal_type signal)
bool dp_set_hw_training_pattern(
struct dc_link *link,
+ const struct link_resource *link_res,
enum dc_dp_training_pattern pattern,
uint32_t offset)
{
@@ -338,7 +342,7 @@ bool dp_set_hw_training_pattern(
break;
}
- dp_set_hw_test_pattern(link, test_pattern, NULL, 0);
+ dp_set_hw_test_pattern(link, link_res, test_pattern, NULL, 0);
return true;
}
@@ -349,6 +353,7 @@ bool dp_set_hw_training_pattern(
#endif
void dp_set_hw_lane_settings(
struct dc_link *link,
+ const struct link_resource *link_res,
const struct link_training_settings *link_settings,
uint32_t offset)
{
@@ -361,8 +366,8 @@ void dp_set_hw_lane_settings(
#if defined(CONFIG_DRM_AMD_DC_DCN)
if (dp_get_link_encoding_format(&link_settings->link_settings) ==
DP_128b_132b_ENCODING) {
- link->hpo_dp_link_enc->funcs->set_ffe(
- link->hpo_dp_link_enc,
+ link_res->hpo_dp_link_enc->funcs->set_ffe(
+ link_res->hpo_dp_link_enc,
&link_settings->link_settings,
link_settings->lane_settings[0].FFE_PRESET.raw);
} else if (dp_get_link_encoding_format(&link_settings->link_settings)
@@ -379,6 +384,7 @@ void dp_set_hw_lane_settings(
void dp_set_hw_test_pattern(
struct dc_link *link,
+ const struct link_resource *link_res,
enum dp_test_pattern test_pattern,
uint8_t *custom_pattern,
uint32_t custom_pattern_size)
@@ -406,8 +412,8 @@ void dp_set_hw_test_pattern(
#if defined(CONFIG_DRM_AMD_DC_DCN)
switch (link_encoding_format) {
case DP_128b_132b_ENCODING:
- link->hpo_dp_link_enc->funcs->set_link_test_pattern(
- link->hpo_dp_link_enc, &pattern_param);
+ link_res->hpo_dp_link_enc->funcs->set_link_test_pattern(
+ link_res->hpo_dp_link_enc, &pattern_param);
break;
case DP_8b_10b_ENCODING:
ASSERT(encoder);
@@ -446,7 +452,7 @@ void dp_retrain_link_dp_test(struct dc_link *link,
pipes[i].stream_res.stream_enc);
/* disable any test pattern that might be active */
- dp_set_hw_test_pattern(link,
+ dp_set_hw_test_pattern(link, &pipes[i].link_res,
DP_TEST_PATTERN_VIDEO_MODE, NULL, 0);
dp_receiver_power_ctrl(link, false);
@@ -763,7 +769,9 @@ static enum phyd32clk_clock_source get_phyd32clk_src(struct dc_link *link)
}
}
-void enable_dp_hpo_output(struct dc_link *link, const struct dc_link_settings *link_settings)
+void enable_dp_hpo_output(struct dc_link *link,
+ const struct link_resource *link_res,
+ const struct dc_link_settings *link_settings)
{
const struct dc *dc = link->dc;
enum phyd32clk_clock_source phyd32clk;
@@ -789,10 +797,11 @@ void enable_dp_hpo_output(struct dc_link *link, const struct dc_link_settings *l
}
} else {
/* DP2.0 HW: call transmitter control to enable PHY */
- link->hpo_dp_link_enc->funcs->enable_link_phy(
- link->hpo_dp_link_enc,
+ link_res->hpo_dp_link_enc->funcs->enable_link_phy(
+ link_res->hpo_dp_link_enc,
link_settings,
- link->link_enc->transmitter);
+ link->link_enc->transmitter,
+ link->link_enc->hpd_source);
}
/* DCCG muxing and DTBCLK DTO */
@@ -806,24 +815,26 @@ void enable_dp_hpo_output(struct dc_link *link, const struct dc_link_settings *l
phyd32clk = get_phyd32clk_src(link);
dc->res_pool->dccg->funcs->enable_symclk32_le(
dc->res_pool->dccg,
- link->hpo_dp_link_enc->inst,
+ link_res->hpo_dp_link_enc->inst,
phyd32clk);
- link->hpo_dp_link_enc->funcs->link_enable(
- link->hpo_dp_link_enc,
- link_settings->lane_count);
+ link_res->hpo_dp_link_enc->funcs->link_enable(
+ link_res->hpo_dp_link_enc,
+ link_settings->lane_count);
}
}
-void disable_dp_hpo_output(struct dc_link *link, enum signal_type signal)
+void disable_dp_hpo_output(struct dc_link *link,
+ const struct link_resource *link_res,
+ enum signal_type signal)
{
const struct dc *dc = link->dc;
- link->hpo_dp_link_enc->funcs->link_disable(link->hpo_dp_link_enc);
+ link_res->hpo_dp_link_enc->funcs->link_disable(link_res->hpo_dp_link_enc);
if (IS_FPGA_MAXIMUS_DC(dc->ctx->dce_environment)) {
dc->res_pool->dccg->funcs->disable_symclk32_le(
dc->res_pool->dccg,
- link->hpo_dp_link_enc->inst);
+ link_res->hpo_dp_link_enc->inst);
dc->res_pool->dccg->funcs->set_physymclk(
dc->res_pool->dccg,
@@ -834,8 +845,8 @@ void disable_dp_hpo_output(struct dc_link *link, enum signal_type signal)
dm_set_phyd32clk(dc->ctx, 0);
} else {
/* DP2.0 HW: call transmitter control to disable PHY */
- link->hpo_dp_link_enc->funcs->disable_link_phy(
- link->hpo_dp_link_enc,
+ link_res->hpo_dp_link_enc->funcs->disable_link_phy(
+ link_res->hpo_dp_link_enc,
signal);
}
}
diff --git a/drivers/gpu/drm/amd/display/dc/core/dc_resource.c b/drivers/gpu/drm/amd/display/dc/core/dc_resource.c
index e2d9a46..de5c7d1 100644
--- a/drivers/gpu/drm/amd/display/dc/core/dc_resource.c
+++ b/drivers/gpu/drm/amd/display/dc/core/dc_resource.c
@@ -1724,6 +1724,94 @@ static void update_hpo_dp_stream_engine_usage(
res_ctx->is_hpo_dp_stream_enc_acquired[i] = acquired;
}
}
+
+static inline int find_acquired_hpo_dp_link_enc_for_link(
+ const struct resource_context *res_ctx,
+ const struct dc_link *link)
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(res_ctx->hpo_dp_link_enc_to_link_idx); i++)
+ if (res_ctx->hpo_dp_link_enc_ref_cnts[i] > 0 &&
+ res_ctx->hpo_dp_link_enc_to_link_idx[i] == link->link_index)
+ return i;
+
+ return -1;
+}
+
+static inline int find_free_hpo_dp_link_enc(const struct resource_context *res_ctx,
+ const struct resource_pool *pool)
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(res_ctx->hpo_dp_link_enc_ref_cnts); i++)
+ if (res_ctx->hpo_dp_link_enc_ref_cnts[i] == 0)
+ break;
+
+ return (i < ARRAY_SIZE(res_ctx->hpo_dp_link_enc_ref_cnts) &&
+ i < pool->hpo_dp_link_enc_count) ? i : -1;
+}
+
+static inline void acquire_hpo_dp_link_enc(
+ struct resource_context *res_ctx,
+ unsigned int link_index,
+ int enc_index)
+{
+ res_ctx->hpo_dp_link_enc_to_link_idx[enc_index] = link_index;
+ res_ctx->hpo_dp_link_enc_ref_cnts[enc_index] = 1;
+}
+
+static inline void retain_hpo_dp_link_enc(
+ struct resource_context *res_ctx,
+ int enc_index)
+{
+ res_ctx->hpo_dp_link_enc_ref_cnts[enc_index]++;
+}
+
+static inline void release_hpo_dp_link_enc(
+ struct resource_context *res_ctx,
+ int enc_index)
+{
+ ASSERT(res_ctx->hpo_dp_link_enc_ref_cnts[enc_index] > 0);
+ res_ctx->hpo_dp_link_enc_ref_cnts[enc_index]--;
+}
+
+static bool add_hpo_dp_link_enc_to_ctx(struct resource_context *res_ctx,
+ const struct resource_pool *pool,
+ struct pipe_ctx *pipe_ctx,
+ struct dc_stream_state *stream)
+{
+ int enc_index;
+
+ enc_index = find_acquired_hpo_dp_link_enc_for_link(res_ctx, stream->link);
+
+ if (enc_index >= 0) {
+ retain_hpo_dp_link_enc(res_ctx, enc_index);
+ } else {
+ enc_index = find_free_hpo_dp_link_enc(res_ctx, pool);
+ if (enc_index >= 0)
+ acquire_hpo_dp_link_enc(res_ctx, stream->link->link_index, enc_index);
+ }
+
+ if (enc_index >= 0)
+ pipe_ctx->link_res.hpo_dp_link_enc = pool->hpo_dp_link_enc[enc_index];
+
+ return pipe_ctx->link_res.hpo_dp_link_enc != NULL;
+}
+
+static void remove_hpo_dp_link_enc_from_ctx(struct resource_context *res_ctx,
+ struct pipe_ctx *pipe_ctx,
+ struct dc_stream_state *stream)
+{
+ int enc_index;
+
+ enc_index = find_acquired_hpo_dp_link_enc_for_link(res_ctx, stream->link);
+
+ if (enc_index >= 0) {
+ release_hpo_dp_link_enc(res_ctx, enc_index);
+ pipe_ctx->link_res.hpo_dp_link_enc = NULL;
+ }
+}
#endif
/* TODO: release audio object */
@@ -1886,6 +1974,7 @@ enum dc_status dc_remove_stream_from_ctx(
&new_ctx->res_ctx, dc->res_pool,
del_pipe->stream_res.hpo_dp_stream_enc,
false);
+ remove_hpo_dp_link_enc_from_ctx(&new_ctx->res_ctx, del_pipe, del_pipe->stream);
}
#endif
@@ -2082,7 +2171,6 @@ static void mark_seamless_boot_stream(
{
struct dc_bios *dcb = dc->ctx->dc_bios;
- /* TODO: Check Linux */
if (dc->config.allow_seamless_boot_optimization &&
!dcb->funcs->is_accelerated_mode(dcb)) {
if (dc_validate_seamless_boot_timing(dc, stream->sink, &stream->timing))
@@ -2162,6 +2250,8 @@ enum dc_status resource_map_pool_resources(
&context->res_ctx, pool,
pipe_ctx->stream_res.hpo_dp_stream_enc,
true);
+ if (!add_hpo_dp_link_enc_to_ctx(&context->res_ctx, pool, pipe_ctx, stream))
+ return DC_NO_LINK_ENC_RESOURCE;
}
}
#endif
@@ -2228,6 +2318,9 @@ void dc_resource_state_construct(
struct dc_state *dst_ctx)
{
dst_ctx->clk_mgr = dc->clk_mgr;
+
+ /* Initialise DIG link encoder resource tracking variables. */
+ link_enc_cfg_init(dc, dst_ctx);
}
@@ -2510,17 +2603,7 @@ static void set_avi_info_frame(
/* TODO : We should handle YCC quantization */
/* but we do not have matrix calculation */
- if (stream->qy_bit == 1) {
- if (color_space == COLOR_SPACE_SRGB ||
- color_space == COLOR_SPACE_2020_RGB_FULLRANGE)
- hdmi_info.bits.YQ0_YQ1 = YYC_QUANTIZATION_LIMITED_RANGE;
- else if (color_space == COLOR_SPACE_SRGB_LIMITED ||
- color_space == COLOR_SPACE_2020_RGB_LIMITEDRANGE)
- hdmi_info.bits.YQ0_YQ1 = YYC_QUANTIZATION_LIMITED_RANGE;
- else
- hdmi_info.bits.YQ0_YQ1 = YYC_QUANTIZATION_LIMITED_RANGE;
- } else
- hdmi_info.bits.YQ0_YQ1 = YYC_QUANTIZATION_LIMITED_RANGE;
+ hdmi_info.bits.YQ0_YQ1 = YYC_QUANTIZATION_LIMITED_RANGE;
///VIC
format = stream->timing.timing_3d_format;
@@ -2844,6 +2927,8 @@ bool pipe_need_reprogram(
#if defined(CONFIG_DRM_AMD_DC_DCN)
if (pipe_ctx_old->stream_res.hpo_dp_stream_enc != pipe_ctx->stream_res.hpo_dp_stream_enc)
return true;
+ if (pipe_ctx_old->link_res.hpo_dp_link_enc != pipe_ctx->link_res.hpo_dp_link_enc)
+ return true;
#endif
/* DIG link encoder resource assignment for stream changed. */
@@ -3112,21 +3197,22 @@ void get_audio_check(struct audio_info *aud_modes,
}
#if defined(CONFIG_DRM_AMD_DC_DCN)
-struct hpo_dp_link_encoder *resource_get_unused_hpo_dp_link_encoder(
- const struct resource_pool *pool)
+struct hpo_dp_link_encoder *resource_get_hpo_dp_link_enc_for_det_lt(
+ const struct resource_context *res_ctx,
+ const struct resource_pool *pool,
+ const struct dc_link *link)
{
- uint8_t i;
- struct hpo_dp_link_encoder *enc = NULL;
+ struct hpo_dp_link_encoder *hpo_dp_link_enc = NULL;
+ int enc_index;
- ASSERT(pool->hpo_dp_link_enc_count <= MAX_HPO_DP2_LINK_ENCODERS);
+ enc_index = find_acquired_hpo_dp_link_enc_for_link(res_ctx, link);
- for (i = 0; i < pool->hpo_dp_link_enc_count; i++) {
- if (pool->hpo_dp_link_enc[i]->transmitter == TRANSMITTER_UNKNOWN) {
- enc = pool->hpo_dp_link_enc[i];
- break;
- }
- }
+ if (enc_index < 0)
+ enc_index = find_free_hpo_dp_link_enc(res_ctx, pool);
- return enc;
+ if (enc_index >= 0)
+ hpo_dp_link_enc = pool->hpo_dp_link_enc[enc_index];
+
+ return hpo_dp_link_enc;
}
#endif
diff --git a/drivers/gpu/drm/amd/display/dc/core/dc_sink.c b/drivers/gpu/drm/amd/display/dc/core/dc_sink.c
index a249a0e..4b5e4d8 100644
--- a/drivers/gpu/drm/amd/display/dc/core/dc_sink.c
+++ b/drivers/gpu/drm/amd/display/dc/core/dc_sink.c
@@ -33,14 +33,6 @@
* Private functions
******************************************************************************/
-static void dc_sink_destruct(struct dc_sink *sink)
-{
- if (sink->dc_container_id) {
- kfree(sink->dc_container_id);
- sink->dc_container_id = NULL;
- }
-}
-
static bool dc_sink_construct(struct dc_sink *sink, const struct dc_sink_init_data *init_params)
{
@@ -75,7 +67,7 @@ void dc_sink_retain(struct dc_sink *sink)
static void dc_sink_free(struct kref *kref)
{
struct dc_sink *sink = container_of(kref, struct dc_sink, refcount);
- dc_sink_destruct(sink);
+ kfree(sink->dc_container_id);
kfree(sink);
}
diff --git a/drivers/gpu/drm/amd/display/dc/dc.h b/drivers/gpu/drm/amd/display/dc/dc.h
index 618e798..da2c78c 100644
--- a/drivers/gpu/drm/amd/display/dc/dc.h
+++ b/drivers/gpu/drm/amd/display/dc/dc.h
@@ -47,7 +47,7 @@ struct aux_payload;
struct set_config_cmd_payload;
struct dmub_notification;
-#define DC_VER "3.2.160"
+#define DC_VER "3.2.167"
#define MAX_SURFACES 3
#define MAX_PLANES 6
@@ -75,6 +75,16 @@ enum dc_plane_type {
DC_PLANE_TYPE_DCN_UNIVERSAL,
};
+// Sizes defined as multiples of 64KB
+enum det_size {
+ DET_SIZE_DEFAULT = 0,
+ DET_SIZE_192KB = 3,
+ DET_SIZE_256KB = 4,
+ DET_SIZE_320KB = 5,
+ DET_SIZE_384KB = 6
+};
+
+
struct dc_plane_cap {
enum dc_plane_type type;
uint32_t blends_with_above : 1;
@@ -187,7 +197,9 @@ struct dc_caps {
struct dc_color_caps color;
#if defined(CONFIG_DRM_AMD_DC_DCN)
bool dp_hpo;
+ bool hdmi_frl_pcon_support;
#endif
+ bool edp_dsc_support;
bool vbios_lttpr_aware;
bool vbios_lttpr_enable;
};
@@ -509,7 +521,8 @@ union dpia_debug_options {
uint32_t force_non_lttpr:1;
uint32_t extend_aux_rd_interval:1;
uint32_t disable_mst_dsc_work_around:1;
- uint32_t reserved:28;
+ uint32_t hpd_delay_in_ms:12;
+ uint32_t reserved:16;
} bits;
uint32_t raw;
};
@@ -574,6 +587,8 @@ struct dc_debug_options {
bool native422_support;
bool disable_dsc;
enum visual_confirm visual_confirm;
+ int visual_confirm_rect_height;
+
bool sanity_checks;
bool max_disp_clk;
bool surface_trace;
@@ -668,11 +683,15 @@ struct dc_debug_options {
bool validate_dml_output;
bool enable_dmcub_surface_flip;
bool usbc_combo_phy_reset_wa;
+ bool disable_dsc_edp;
+ unsigned int force_dsc_edp_policy;
bool enable_dram_clock_change_one_display_vactive;
#if defined(CONFIG_DRM_AMD_DC_DCN)
/* TODO - remove once tested */
bool legacy_dp2_lt;
bool set_mst_en_for_sst;
+ bool disable_uhbr;
+ bool force_dp2_lt_fallback_method;
#endif
union mem_low_power_enable_options enable_mem_low_power;
union root_clock_optimization_options root_clock_optimization;
@@ -685,11 +704,14 @@ struct dc_debug_options {
/* FEC/PSR1 sequence enable delay in 100us */
uint8_t fec_enable_delay_in100us;
bool enable_driver_sequence_debug;
+ enum det_size crb_alloc_policy;
+ int crb_alloc_policy_min_disp_count;
#if defined(CONFIG_DRM_AMD_DC_DCN)
bool disable_z10;
bool enable_sw_cntl_psr;
union dpia_debug_options dpia_debug;
#endif
+ bool apply_vendor_specific_lttpr_wa;
};
struct gpu_info_soc_bounding_box_v1_0;
@@ -1290,6 +1312,11 @@ struct dc_sink_dsc_caps {
// 'true' if these are virtual DPCD's DSC caps (immediately upstream of sink in MST topology),
// 'false' if they are sink's DSC caps
bool is_virtual_dpcd_dsc;
+#if defined(CONFIG_DRM_AMD_DC_DCN)
+ // 'true' if MST topology supports DSC passthrough for sink
+ // 'false' if MST topology does not support DSC passthrough
+ bool is_dsc_passthrough_supported;
+#endif
struct dsc_dec_dpcd_caps dsc_dec_caps;
};
@@ -1405,6 +1432,9 @@ void dc_unlock_memory_clock_frequency(struct dc *dc);
*/
void dc_lock_memory_clock_frequency(struct dc *dc);
+/* set soft max for memclk, to be used for AC/DC switching clock limitations */
+void dc_enable_dcmode_clk_limit(struct dc *dc, bool enable);
+
/* cleanup on driver unload */
void dc_hardware_release(struct dc *dc);
diff --git a/drivers/gpu/drm/amd/display/dc/dc_dmub_srv.c b/drivers/gpu/drm/amd/display/dc/dc_dmub_srv.c
index 360f319..541376f 100644
--- a/drivers/gpu/drm/amd/display/dc/dc_dmub_srv.c
+++ b/drivers/gpu/drm/amd/display/dc/dc_dmub_srv.c
@@ -115,13 +115,44 @@ void dc_dmub_srv_wait_idle(struct dc_dmub_srv *dc_dmub_srv)
}
}
+void dc_dmub_srv_clear_inbox0_ack(struct dc_dmub_srv *dmub_srv)
+{
+ struct dmub_srv *dmub = dmub_srv->dmub;
+ struct dc_context *dc_ctx = dmub_srv->ctx;
+ enum dmub_status status = DMUB_STATUS_OK;
+
+ status = dmub_srv_clear_inbox0_ack(dmub);
+ if (status != DMUB_STATUS_OK) {
+ DC_ERROR("Error clearing INBOX0 ack: status=%d\n", status);
+ dc_dmub_srv_log_diagnostic_data(dmub_srv);
+ }
+}
+
+void dc_dmub_srv_wait_for_inbox0_ack(struct dc_dmub_srv *dmub_srv)
+{
+ struct dmub_srv *dmub = dmub_srv->dmub;
+ struct dc_context *dc_ctx = dmub_srv->ctx;
+ enum dmub_status status = DMUB_STATUS_OK;
+
+ status = dmub_srv_wait_for_inbox0_ack(dmub, 100000);
+ if (status != DMUB_STATUS_OK) {
+ DC_ERROR("Error waiting for INBOX0 HW Lock Ack\n");
+ dc_dmub_srv_log_diagnostic_data(dmub_srv);
+ }
+}
+
void dc_dmub_srv_send_inbox0_cmd(struct dc_dmub_srv *dmub_srv,
union dmub_inbox0_data_register data)
{
struct dmub_srv *dmub = dmub_srv->dmub;
- if (dmub->hw_funcs.send_inbox0_cmd)
- dmub->hw_funcs.send_inbox0_cmd(dmub, data);
- // TODO: Add wait command -- poll register for ACK
+ struct dc_context *dc_ctx = dmub_srv->ctx;
+ enum dmub_status status = DMUB_STATUS_OK;
+
+ status = dmub_srv_send_inbox0_cmd(dmub, data);
+ if (status != DMUB_STATUS_OK) {
+ DC_ERROR("Error sending INBOX0 cmd\n");
+ dc_dmub_srv_log_diagnostic_data(dmub_srv);
+ }
}
bool dc_dmub_srv_cmd_with_reply_data(struct dc_dmub_srv *dc_dmub_srv, union dmub_rb_cmd *cmd)
diff --git a/drivers/gpu/drm/amd/display/dc/dc_dmub_srv.h b/drivers/gpu/drm/amd/display/dc/dc_dmub_srv.h
index 3e35eee..7e4e2ec 100644
--- a/drivers/gpu/drm/amd/display/dc/dc_dmub_srv.h
+++ b/drivers/gpu/drm/amd/display/dc/dc_dmub_srv.h
@@ -68,6 +68,8 @@ bool dc_dmub_srv_get_dmub_outbox0_msg(const struct dc *dc, struct dmcub_trace_bu
void dc_dmub_trace_event_control(struct dc *dc, bool enable);
+void dc_dmub_srv_clear_inbox0_ack(struct dc_dmub_srv *dmub_srv);
+void dc_dmub_srv_wait_for_inbox0_ack(struct dc_dmub_srv *dmub_srv);
void dc_dmub_srv_send_inbox0_cmd(struct dc_dmub_srv *dmub_srv, union dmub_inbox0_data_register data);
bool dc_dmub_srv_get_diagnostic_data(struct dc_dmub_srv *dc_dmub_srv, struct dmub_diagnostic_data *dmub_oca);
diff --git a/drivers/gpu/drm/amd/display/dc/dc_dp_types.h b/drivers/gpu/drm/amd/display/dc/dc_dp_types.h
index e68e9a8..353dac4 100644
--- a/drivers/gpu/drm/amd/display/dc/dc_dp_types.h
+++ b/drivers/gpu/drm/amd/display/dc/dc_dp_types.h
@@ -378,7 +378,14 @@ enum dpcd_downstream_port_detailed_type {
union dwnstream_port_caps_byte2 {
struct {
uint8_t MAX_BITS_PER_COLOR_COMPONENT:2;
+#if defined(CONFIG_DRM_AMD_DC_DCN)
+ uint8_t MAX_ENCODED_LINK_BW_SUPPORT:3;
+ uint8_t SOURCE_CONTROL_MODE_SUPPORT:1;
+ uint8_t CONCURRENT_LINK_BRING_UP_SEQ_SUPPORT:1;
+ uint8_t RESERVED:1;
+#else
uint8_t RESERVED:6;
+#endif
} bits;
uint8_t raw;
};
@@ -416,6 +423,30 @@ union dwnstream_port_caps_byte3_hdmi {
uint8_t raw;
};
+#if defined(CONFIG_DRM_AMD_DC_DCN)
+union hdmi_sink_encoded_link_bw_support {
+ struct {
+ uint8_t HDMI_SINK_ENCODED_LINK_BW_SUPPORT:3;
+ uint8_t RESERVED:5;
+ } bits;
+ uint8_t raw;
+};
+
+union hdmi_encoded_link_bw {
+ struct {
+ uint8_t FRL_MODE:1; // Bit 0
+ uint8_t BW_9Gbps:1;
+ uint8_t BW_18Gbps:1;
+ uint8_t BW_24Gbps:1;
+ uint8_t BW_32Gbps:1;
+ uint8_t BW_40Gbps:1;
+ uint8_t BW_48Gbps:1;
+ uint8_t RESERVED:1; // Bit 7
+ } bits;
+ uint8_t raw;
+};
+#endif
+
/*4-byte structure for detailed capabilities of a down-stream port
(DP-to-TMDS converter).*/
union dwnstream_portxcaps {
@@ -852,6 +883,15 @@ struct psr_caps {
unsigned char psr_version;
unsigned int psr_rfb_setup_time;
bool psr_exit_link_training_required;
+ unsigned char edp_revision;
+ unsigned char support_ver;
+ bool su_granularity_required;
+ bool y_coordinate_required;
+ uint8_t su_y_granularity;
+ bool alpm_cap;
+ bool standby_support;
+ uint8_t rate_control_caps;
+ unsigned int psr_power_opt_flag;
};
/* Length of router topology ID read from DPCD in bytes. */
diff --git a/drivers/gpu/drm/amd/display/dc/dc_hw_types.h b/drivers/gpu/drm/amd/display/dc/dc_hw_types.h
index 52355fe..eac34f5 100644
--- a/drivers/gpu/drm/amd/display/dc/dc_hw_types.h
+++ b/drivers/gpu/drm/amd/display/dc/dc_hw_types.h
@@ -741,6 +741,9 @@ struct dc_dsc_config {
uint32_t version_minor; /* DSC minor version. Full version is formed as 1.version_minor. */
bool ycbcr422_simple; /* Tell DSC engine to convert YCbCr 4:2:2 to 'YCbCr 4:2:2 simple'. */
int32_t rc_buffer_size; /* DSC RC buffer block size in bytes */
+#if defined(CONFIG_DRM_AMD_DC_DCN)
+ bool is_frl; /* indicate if DSC is applied based on HDMI FRL sink's capability */
+#endif
bool is_dp; /* indicate if DSC is applied based on DP's capability */
};
struct dc_crtc_timing {
diff --git a/drivers/gpu/drm/amd/display/dc/dc_link.h b/drivers/gpu/drm/amd/display/dc/dc_link.h
index fad3d88..c0e37ad 100644
--- a/drivers/gpu/drm/amd/display/dc/dc_link.h
+++ b/drivers/gpu/drm/amd/display/dc/dc_link.h
@@ -30,6 +30,8 @@
#include "dc_types.h"
#include "grph_object_defs.h"
+struct link_resource;
+
enum dc_link_fec_state {
dc_link_fec_not_ready,
dc_link_fec_ready,
@@ -113,6 +115,7 @@ struct dc_link {
* DIG encoder. */
bool is_dig_mapping_flexible;
bool hpd_status; /* HPD status of link without physical HPD pin. */
+ bool is_hpd_pending; /* Indicates a new received hpd */
bool edp_sink_present;
@@ -159,9 +162,6 @@ struct dc_link {
struct panel_cntl *panel_cntl;
struct link_encoder *link_enc;
-#if defined(CONFIG_DRM_AMD_DC_DCN)
- struct hpo_dp_link_encoder *hpo_dp_link_enc;
-#endif
struct graphics_object_id link_id;
/* Endpoint type distinguishes display endpoints which do not have entries
* in the BIOS connector table from those that do. Helps when tracking link
@@ -185,6 +185,10 @@ struct dc_link {
/* Drive settings read from integrated info table */
struct dc_lane_settings bios_forced_drive_settings;
+ /* Vendor specific LTTPR workaround variables */
+ uint8_t vendor_specific_lttpr_link_rate_wa;
+ bool apply_vendor_specific_lttpr_link_rate_wa;
+
/* MST record stream using this link */
struct link_flags {
bool dp_keep_receiver_powered;
@@ -291,6 +295,10 @@ bool dc_link_setup_psr(struct dc_link *dc_link,
void dc_link_get_psr_residency(const struct dc_link *link, uint32_t *residency);
+void dc_link_blank_all_dp_displays(struct dc *dc);
+
+void dc_link_blank_dp_stream(struct dc_link *link, bool hw_init);
+
/* Request DC to detect if there is a Panel connected.
* boot - If this call is during initial boot.
* Return false for any type of detection failure or MST detection
@@ -302,7 +310,7 @@ enum dc_detect_reason {
DETECT_REASON_HPD,
DETECT_REASON_HPDRX,
DETECT_REASON_FALLBACK,
- DETECT_REASON_RETRAIN
+ DETECT_REASON_RETRAIN,
};
bool dc_link_detect(struct dc_link *dc_link, enum dc_detect_reason reason);
@@ -350,14 +358,17 @@ void dc_link_remove_remote_sink(
void dc_link_dp_set_drive_settings(
struct dc_link *link,
+ const struct link_resource *link_res,
struct link_training_settings *lt_settings);
bool dc_link_dp_perform_link_training_skip_aux(
struct dc_link *link,
+ const struct link_resource *link_res,
const struct dc_link_settings *link_setting);
enum link_training_result dc_link_dp_perform_link_training(
struct dc_link *link,
+ const struct link_resource *link_res,
const struct dc_link_settings *link_settings,
bool skip_video_pattern);
@@ -365,6 +376,7 @@ bool dc_link_dp_sync_lt_begin(struct dc_link *link);
enum link_training_result dc_link_dp_sync_lt_attempt(
struct dc_link *link,
+ const struct link_resource *link_res,
struct dc_link_settings *link_setting,
struct dc_link_training_overrides *lt_settings);
@@ -442,6 +454,13 @@ bool dc_link_is_fec_supported(const struct dc_link *link);
bool dc_link_should_enable_fec(const struct dc_link *link);
#if defined(CONFIG_DRM_AMD_DC_DCN)
+uint32_t dc_link_bw_kbps_from_raw_frl_link_rate_data(uint8_t bw);
enum dp_link_encoding dc_link_dp_mst_decide_link_encoding_format(const struct dc_link *link);
#endif
+
+const struct link_resource *dc_link_get_cur_link_res(const struct dc_link *link);
+/* take a snapshot of current link resource allocation state */
+void dc_get_cur_link_res_map(const struct dc *dc, uint32_t *map);
+/* restore link resource allocation state from a snapshot */
+void dc_restore_link_res_map(const struct dc *dc, uint32_t *map);
#endif /* DC_LINK_H_ */
diff --git a/drivers/gpu/drm/amd/display/dc/dc_types.h b/drivers/gpu/drm/amd/display/dc/dc_types.h
index 388457f..0285a4b 100644
--- a/drivers/gpu/drm/amd/display/dc/dc_types.h
+++ b/drivers/gpu/drm/amd/display/dc/dc_types.h
@@ -430,6 +430,7 @@ struct dc_dongle_caps {
uint32_t dp_hdmi_max_bpc;
uint32_t dp_hdmi_max_pixel_clk_in_khz;
#if defined(CONFIG_DRM_AMD_DC_DCN)
+ uint32_t dp_hdmi_frl_max_link_bw_in_kbps;
struct dc_dongle_dfp_cap_ext dfp_cap_ext;
#endif
};
@@ -950,6 +951,7 @@ enum dc_gpu_mem_alloc_type {
enum dc_psr_version {
DC_PSR_VERSION_1 = 0,
+ DC_PSR_VERSION_SU_1 = 1,
DC_PSR_VERSION_UNSUPPORTED = 0xFFFFFFFF,
};
diff --git a/drivers/gpu/drm/amd/display/dc/dce/dce_audio.c b/drivers/gpu/drm/amd/display/dc/dce/dce_audio.c
index 27218ed..70eaac01 100644
--- a/drivers/gpu/drm/amd/display/dc/dce/dce_audio.c
+++ b/drivers/gpu/drm/amd/display/dc/dce/dce_audio.c
@@ -67,9 +67,6 @@ static void write_indirect_azalia_reg(struct audio *audio,
/* AZALIA_F0_CODEC_ENDPOINT_DATA endpoint data */
REG_SET(AZALIA_F0_CODEC_ENDPOINT_DATA, 0,
AZALIA_ENDPOINT_REG_DATA, reg_data);
-
- DC_LOG_HW_AUDIO("AUDIO:write_indirect_azalia_reg: index: %u data: %u\n",
- reg_index, reg_data);
}
static uint32_t read_indirect_azalia_reg(struct audio *audio, uint32_t reg_index)
@@ -85,9 +82,6 @@ static uint32_t read_indirect_azalia_reg(struct audio *audio, uint32_t reg_index
/* AZALIA_F0_CODEC_ENDPOINT_DATA endpoint data */
value = REG_READ(AZALIA_F0_CODEC_ENDPOINT_DATA);
- DC_LOG_HW_AUDIO("AUDIO:read_indirect_azalia_reg: index: %u data: %u\n",
- reg_index, value);
-
return value;
}
diff --git a/drivers/gpu/drm/amd/display/dc/dce/dce_audio.h b/drivers/gpu/drm/amd/display/dc/dce/dce_audio.h
index 5622d5e..dbd2cfe 100644
--- a/drivers/gpu/drm/amd/display/dc/dce/dce_audio.h
+++ b/drivers/gpu/drm/amd/display/dc/dce/dce_audio.h
@@ -113,6 +113,7 @@ struct dce_audio_shift {
uint8_t DCCG_AUDIO_DTO2_USE_512FBR_DTO;
uint32_t DCCG_AUDIO_DTO0_USE_512FBR_DTO;
uint32_t DCCG_AUDIO_DTO1_USE_512FBR_DTO;
+ uint32_t CLOCK_GATING_DISABLE;
};
struct dce_audio_mask {
@@ -132,6 +133,7 @@ struct dce_audio_mask {
uint32_t DCCG_AUDIO_DTO2_USE_512FBR_DTO;
uint32_t DCCG_AUDIO_DTO0_USE_512FBR_DTO;
uint32_t DCCG_AUDIO_DTO1_USE_512FBR_DTO;
+ uint32_t CLOCK_GATING_DISABLE;
};
diff --git a/drivers/gpu/drm/amd/display/dc/dce/dce_link_encoder.c b/drivers/gpu/drm/amd/display/dc/dce/dce_link_encoder.c
index 1e77ffe..f1c61d5 100644
--- a/drivers/gpu/drm/amd/display/dc/dce/dce_link_encoder.c
+++ b/drivers/gpu/drm/amd/display/dc/dce/dce_link_encoder.c
@@ -788,8 +788,9 @@ static bool dce110_link_encoder_validate_hdmi_output(
crtc_timing->pixel_encoding == PIXEL_ENCODING_YCBCR420)
return false;
- if (!enc110->base.features.flags.bits.HDMI_6GB_EN &&
- adjusted_pix_clk_khz >= 300000)
+ if ((!enc110->base.features.flags.bits.HDMI_6GB_EN ||
+ enc110->base.ctx->dc->debug.hdmi20_disable) &&
+ adjusted_pix_clk_khz >= 300000)
return false;
if (enc110->base.ctx->dc->debug.hdmi20_disable &&
crtc_timing->pixel_encoding == PIXEL_ENCODING_YCBCR420)
diff --git a/drivers/gpu/drm/amd/display/dc/dce/dmub_hw_lock_mgr.c b/drivers/gpu/drm/amd/display/dc/dce/dmub_hw_lock_mgr.c
index 9baf8ca..b1b2e3c 100644
--- a/drivers/gpu/drm/amd/display/dc/dce/dmub_hw_lock_mgr.c
+++ b/drivers/gpu/drm/amd/display/dc/dce/dmub_hw_lock_mgr.c
@@ -56,8 +56,11 @@ void dmub_hw_lock_mgr_inbox0_cmd(struct dc_dmub_srv *dmub_srv,
union dmub_inbox0_cmd_lock_hw hw_lock_cmd)
{
union dmub_inbox0_data_register data = { 0 };
+
data.inbox0_cmd_lock_hw = hw_lock_cmd;
+ dc_dmub_srv_clear_inbox0_ack(dmub_srv);
dc_dmub_srv_send_inbox0_cmd(dmub_srv, data);
+ dc_dmub_srv_wait_for_inbox0_ack(dmub_srv);
}
bool should_use_dmub_lock(struct dc_link *link)
diff --git a/drivers/gpu/drm/amd/display/dc/dce/dmub_psr.c b/drivers/gpu/drm/amd/display/dc/dce/dmub_psr.c
index 90eb8ee..87ed48d 100644
--- a/drivers/gpu/drm/amd/display/dc/dce/dmub_psr.c
+++ b/drivers/gpu/drm/amd/display/dc/dce/dmub_psr.c
@@ -230,7 +230,7 @@ static void dmub_psr_set_level(struct dmub_psr *dmub, uint16_t psr_level, uint8_
/**
* Set PSR power optimization flags.
*/
-static void dmub_psr_set_power_opt(struct dmub_psr *dmub, unsigned int power_opt)
+static void dmub_psr_set_power_opt(struct dmub_psr *dmub, unsigned int power_opt, uint8_t panel_inst)
{
union dmub_rb_cmd cmd;
struct dc_context *dc = dmub->ctx;
@@ -239,7 +239,9 @@ static void dmub_psr_set_power_opt(struct dmub_psr *dmub, unsigned int power_opt
cmd.psr_set_power_opt.header.type = DMUB_CMD__PSR;
cmd.psr_set_power_opt.header.sub_type = DMUB_CMD__SET_PSR_POWER_OPT;
cmd.psr_set_power_opt.header.payload_bytes = sizeof(struct dmub_cmd_psr_set_power_opt_data);
+ cmd.psr_set_power_opt.psr_set_power_opt_data.cmd_version = DMUB_CMD_PSR_CONTROL_VERSION_1;
cmd.psr_set_power_opt.psr_set_power_opt_data.power_opt = power_opt;
+ cmd.psr_set_power_opt.psr_set_power_opt_data.panel_inst = panel_inst;
dc_dmub_srv_cmd_queue(dc->dmub_srv, &cmd);
dc_dmub_srv_cmd_execute(dc->dmub_srv);
@@ -327,6 +329,16 @@ static bool dmub_psr_copy_settings(struct dmub_psr *dmub,
copy_settings_data->fec_enable_delay_in100us = link->dc->debug.fec_enable_delay_in100us;
copy_settings_data->cmd_version = DMUB_CMD_PSR_CONTROL_VERSION_1;
copy_settings_data->panel_inst = panel_inst;
+ copy_settings_data->dsc_enable_status = (pipe_ctx->stream->timing.flags.DSC == 1);
+
+ if (link->fec_state == dc_link_fec_enabled &&
+ (!memcmp(link->dpcd_caps.sink_dev_id_str, DP_SINK_DEVICE_STR_ID_1,
+ sizeof(link->dpcd_caps.sink_dev_id_str)) ||
+ !memcmp(link->dpcd_caps.sink_dev_id_str, DP_SINK_DEVICE_STR_ID_2,
+ sizeof(link->dpcd_caps.sink_dev_id_str))))
+ copy_settings_data->debug.bitfields.force_wakeup_by_tps3 = 1;
+ else
+ copy_settings_data->debug.bitfields.force_wakeup_by_tps3 = 0;
dc_dmub_srv_cmd_queue(dc->dmub_srv, &cmd);
dc_dmub_srv_cmd_execute(dc->dmub_srv);
diff --git a/drivers/gpu/drm/amd/display/dc/dce/dmub_psr.h b/drivers/gpu/drm/amd/display/dc/dce/dmub_psr.h
index 5dbd479..01acc01 100644
--- a/drivers/gpu/drm/amd/display/dc/dce/dmub_psr.h
+++ b/drivers/gpu/drm/amd/display/dc/dce/dmub_psr.h
@@ -46,7 +46,7 @@ struct dmub_psr_funcs {
void (*psr_force_static)(struct dmub_psr *dmub, uint8_t panel_inst);
void (*psr_get_residency)(struct dmub_psr *dmub, uint32_t *residency,
uint8_t panel_inst);
- void (*psr_set_power_opt)(struct dmub_psr *dmub, unsigned int power_opt);
+ void (*psr_set_power_opt)(struct dmub_psr *dmub, unsigned int power_opt, uint8_t panel_inst);
};
struct dmub_psr *dmub_psr_create(struct dc_context *ctx);
diff --git a/drivers/gpu/drm/amd/display/dc/dce110/dce110_hw_sequencer.c b/drivers/gpu/drm/amd/display/dc/dce110/dce110_hw_sequencer.c
index 24e47df..78192ec 100644
--- a/drivers/gpu/drm/amd/display/dc/dce110/dce110_hw_sequencer.c
+++ b/drivers/gpu/drm/amd/display/dc/dce110/dce110_hw_sequencer.c
@@ -69,6 +69,8 @@
#include "dcn10/dcn10_hw_sequencer.h"
+#include "dce110_hw_sequencer.h"
+
#define GAMMA_HW_POINTS_NUM 256
/*
@@ -1602,6 +1604,11 @@ static enum dc_status apply_single_controller_ctx_to_hw(
pipe_ctx->stream_res.stream_enc,
pipe_ctx->stream_res.tg->inst);
+ if (dc_is_dp_signal(pipe_ctx->stream->signal) &&
+ pipe_ctx->stream_res.stream_enc->funcs->reset_fifo)
+ pipe_ctx->stream_res.stream_enc->funcs->reset_fifo(
+ pipe_ctx->stream_res.stream_enc);
+
if (dc_is_dp_signal(pipe_ctx->stream->signal))
dp_source_sequence_trace(link, DPCD_SOURCE_SEQ_AFTER_CONNECT_DIG_FE_OTG);
@@ -1655,30 +1662,12 @@ static enum dc_status apply_single_controller_ctx_to_hw(
static void power_down_encoders(struct dc *dc)
{
- int i, j;
+ int i;
for (i = 0; i < dc->link_count; i++) {
enum signal_type signal = dc->links[i]->connector_signal;
- if ((signal == SIGNAL_TYPE_EDP) ||
- (signal == SIGNAL_TYPE_DISPLAY_PORT)) {
- if (dc->links[i]->link_enc->funcs->get_dig_frontend &&
- dc->links[i]->link_enc->funcs->is_dig_enabled(dc->links[i]->link_enc)) {
- unsigned int fe = dc->links[i]->link_enc->funcs->get_dig_frontend(
- dc->links[i]->link_enc);
-
- for (j = 0; j < dc->res_pool->stream_enc_count; j++) {
- if (fe == dc->res_pool->stream_enc[j]->id) {
- dc->res_pool->stream_enc[j]->funcs->dp_blank(dc->links[i],
- dc->res_pool->stream_enc[j]);
- break;
- }
- }
- }
-
- if (!dc->links[i]->wa_flags.dp_keep_receiver_powered)
- dp_receiver_power_ctrl(dc->links[i], false);
- }
+ dc_link_blank_dp_stream(dc->links[i], false);
if (signal != SIGNAL_TYPE_EDP)
signal = SIGNAL_TYPE_NONE;
@@ -1805,7 +1794,6 @@ void dce110_enable_accelerated_mode(struct dc *dc, struct dc_state *context)
struct dc_stream_state *edp_streams[MAX_NUM_EDP];
struct dc_link *edp_link_with_sink = NULL;
struct dc_link *edp_link = NULL;
- struct dc_stream_state *edp_stream = NULL;
struct dce_hwseq *hws = dc->hwseq;
int edp_with_sink_num;
int edp_num;
@@ -1826,27 +1814,29 @@ void dce110_enable_accelerated_mode(struct dc *dc, struct dc_state *context)
get_edp_streams(context, edp_streams, &edp_stream_num);
// Check fastboot support, disable on DCE8 because of blank screens
- if (edp_num && dc->ctx->dce_version != DCE_VERSION_8_0 &&
+ if (edp_num && edp_stream_num && dc->ctx->dce_version != DCE_VERSION_8_0 &&
dc->ctx->dce_version != DCE_VERSION_8_1 &&
dc->ctx->dce_version != DCE_VERSION_8_3) {
for (i = 0; i < edp_num; i++) {
edp_link = edp_links[i];
+ if (edp_link != edp_streams[0]->link)
+ continue;
// enable fastboot if backend is enabled on eDP
- if (edp_link->link_enc->funcs->is_dig_enabled(edp_link->link_enc)) {
- /* Set optimization flag on eDP stream*/
- if (edp_stream_num && edp_link->link_status.link_active) {
- edp_stream = edp_streams[0];
- can_apply_edp_fast_boot = !is_edp_ilr_optimization_required(edp_stream->link, &edp_stream->timing);
- edp_stream->apply_edp_fast_boot_optimization = can_apply_edp_fast_boot;
- if (can_apply_edp_fast_boot)
- DC_LOG_EVENT_LINK_TRAINING("eDP fast boot disabled to optimize link rate\n");
+ if (edp_link->link_enc->funcs->is_dig_enabled &&
+ edp_link->link_enc->funcs->is_dig_enabled(edp_link->link_enc) &&
+ edp_link->link_status.link_active) {
+ struct dc_stream_state *edp_stream = edp_streams[0];
- break;
- }
+ can_apply_edp_fast_boot = !is_edp_ilr_optimization_required(edp_stream->link, &edp_stream->timing);
+ edp_stream->apply_edp_fast_boot_optimization = can_apply_edp_fast_boot;
+ if (can_apply_edp_fast_boot)
+ DC_LOG_EVENT_LINK_TRAINING("eDP fast boot disabled to optimize link rate\n");
+
+ break;
}
}
// We are trying to enable eDP, don't power down VDD
- if (edp_stream_num)
+ if (can_apply_edp_fast_boot)
keep_edp_vdd_on = true;
}
diff --git a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_dpp.c b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_dpp.c
index 91fdfcd..db7ca4b 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_dpp.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_dpp.c
@@ -119,14 +119,6 @@ void dpp_read_state(struct dpp *dpp_base,
}
}
-/* Program gamut remap in bypass mode */
-void dpp_set_gamut_remap_bypass(struct dcn10_dpp *dpp)
-{
- REG_SET(CM_GAMUT_REMAP_CONTROL, 0,
- CM_GAMUT_REMAP_MODE, 0);
- /* Gamut remap in bypass */
-}
-
#define IDENTITY_RATIO(ratio) (dc_fixpt_u2d19(ratio) == (1 << 19))
bool dpp1_get_optimal_number_of_taps(
diff --git a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_dpp_dscl.c b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_dpp_dscl.c
index 44293d6..f607a0e 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_dpp_dscl.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_dpp_dscl.c
@@ -39,6 +39,10 @@
#define BLACK_OFFSET_RGB_Y 0x0
#define BLACK_OFFSET_CBCR 0x8000
+#define VISUAL_CONFIRM_RECT_HEIGHT_DEFAULT 3
+#define VISUAL_CONFIRM_RECT_HEIGHT_MIN 1
+#define VISUAL_CONFIRM_RECT_HEIGHT_MAX 10
+
#define REG(reg)\
dpp->tf_regs->reg
@@ -85,51 +89,6 @@ enum dscl_mode_sel {
DSCL_MODE_DSCL_BYPASS = 6
};
-static void dpp1_dscl_set_overscan(
- struct dcn10_dpp *dpp,
- const struct scaler_data *data)
-{
- uint32_t left = data->recout.x;
- uint32_t top = data->recout.y;
-
- int right = data->h_active - data->recout.x - data->recout.width;
- int bottom = data->v_active - data->recout.y - data->recout.height;
-
- if (right < 0) {
- BREAK_TO_DEBUGGER();
- right = 0;
- }
- if (bottom < 0) {
- BREAK_TO_DEBUGGER();
- bottom = 0;
- }
-
- REG_SET_2(DSCL_EXT_OVERSCAN_LEFT_RIGHT, 0,
- EXT_OVERSCAN_LEFT, left,
- EXT_OVERSCAN_RIGHT, right);
-
- REG_SET_2(DSCL_EXT_OVERSCAN_TOP_BOTTOM, 0,
- EXT_OVERSCAN_BOTTOM, bottom,
- EXT_OVERSCAN_TOP, top);
-}
-
-static void dpp1_dscl_set_otg_blank(
- struct dcn10_dpp *dpp, const struct scaler_data *data)
-{
- uint32_t h_blank_start = data->h_active;
- uint32_t h_blank_end = 0;
- uint32_t v_blank_start = data->v_active;
- uint32_t v_blank_end = 0;
-
- REG_SET_2(OTG_H_BLANK, 0,
- OTG_H_BLANK_START, h_blank_start,
- OTG_H_BLANK_END, h_blank_end);
-
- REG_SET_2(OTG_V_BLANK, 0,
- OTG_V_BLANK_START, v_blank_start,
- OTG_V_BLANK_END, v_blank_end);
-}
-
static int dpp1_dscl_get_pixel_depth_val(enum lb_pixel_depth depth)
{
if (depth == LB_PIXEL_DEPTH_30BPP)
@@ -551,58 +510,6 @@ static enum lb_memory_config dpp1_dscl_find_lb_memory_config(struct dcn10_dpp *d
return LB_MEMORY_CONFIG_0;
}
-void dpp1_dscl_set_scaler_auto_scale(
- struct dpp *dpp_base,
- const struct scaler_data *scl_data)
-{
- enum lb_memory_config lb_config;
- struct dcn10_dpp *dpp = TO_DCN10_DPP(dpp_base);
- enum dscl_mode_sel dscl_mode = dpp1_dscl_get_dscl_mode(
- dpp_base, scl_data, dpp_base->ctx->dc->debug.always_scale);
- bool ycbcr = scl_data->format >= PIXEL_FORMAT_VIDEO_BEGIN
- && scl_data->format <= PIXEL_FORMAT_VIDEO_END;
-
- dpp1_dscl_set_overscan(dpp, scl_data);
-
- dpp1_dscl_set_otg_blank(dpp, scl_data);
-
- REG_UPDATE(SCL_MODE, DSCL_MODE, dscl_mode);
-
- if (dscl_mode == DSCL_MODE_DSCL_BYPASS)
- return;
-
- lb_config = dpp1_dscl_find_lb_memory_config(dpp, scl_data);
- dpp1_dscl_set_lb(dpp, &scl_data->lb_params, lb_config);
-
- if (dscl_mode == DSCL_MODE_SCALING_444_BYPASS)
- return;
-
- /* TODO: v_min */
- REG_SET_3(DSCL_AUTOCAL, 0,
- AUTOCAL_MODE, AUTOCAL_MODE_AUTOSCALE,
- AUTOCAL_NUM_PIPE, 0,
- AUTOCAL_PIPE_ID, 0);
-
- /* Black offsets */
- if (ycbcr)
- REG_SET_2(SCL_BLACK_OFFSET, 0,
- SCL_BLACK_OFFSET_RGB_Y, BLACK_OFFSET_RGB_Y,
- SCL_BLACK_OFFSET_CBCR, BLACK_OFFSET_CBCR);
- else
-
- REG_SET_2(SCL_BLACK_OFFSET, 0,
- SCL_BLACK_OFFSET_RGB_Y, BLACK_OFFSET_RGB_Y,
- SCL_BLACK_OFFSET_CBCR, BLACK_OFFSET_RGB_Y);
-
- REG_SET_4(SCL_TAP_CONTROL, 0,
- SCL_V_NUM_TAPS, scl_data->taps.v_taps - 1,
- SCL_H_NUM_TAPS, scl_data->taps.h_taps - 1,
- SCL_V_NUM_TAPS_C, scl_data->taps.v_taps_c - 1,
- SCL_H_NUM_TAPS_C, scl_data->taps.h_taps_c - 1);
-
- dpp1_dscl_set_scl_filter(dpp, scl_data, ycbcr);
-}
-
static void dpp1_dscl_set_manual_ratio_init(
struct dcn10_dpp *dpp, const struct scaler_data *data)
@@ -685,9 +592,17 @@ static void dpp1_dscl_set_recout(struct dcn10_dpp *dpp,
const struct rect *recout)
{
int visual_confirm_on = 0;
+ unsigned short visual_confirm_rect_height = VISUAL_CONFIRM_RECT_HEIGHT_DEFAULT;
+
if (dpp->base.ctx->dc->debug.visual_confirm != VISUAL_CONFIRM_DISABLE)
visual_confirm_on = 1;
+ /* Check bounds to ensure the VC bar height was set to a sane value */
+ if ((dpp->base.ctx->dc->debug.visual_confirm_rect_height >= VISUAL_CONFIRM_RECT_HEIGHT_MIN) &&
+ (dpp->base.ctx->dc->debug.visual_confirm_rect_height <= VISUAL_CONFIRM_RECT_HEIGHT_MAX)) {
+ visual_confirm_rect_height = dpp->base.ctx->dc->debug.visual_confirm_rect_height;
+ }
+
REG_SET_2(RECOUT_START, 0,
/* First pixel of RECOUT in the active OTG area */
RECOUT_START_X, recout->x,
@@ -699,7 +614,7 @@ static void dpp1_dscl_set_recout(struct dcn10_dpp *dpp,
RECOUT_WIDTH, recout->width,
/* Number of RECOUT vertical lines */
RECOUT_HEIGHT, recout->height
- - visual_confirm_on * 2 * (dpp->base.inst + 1));
+ - visual_confirm_on * 2 * (dpp->base.inst + visual_confirm_rect_height));
}
/**
diff --git a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hw_sequencer.c b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hw_sequencer.c
index 04d7bdd..f190154 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hw_sequencer.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hw_sequencer.c
@@ -77,9 +77,9 @@
#define PGFSM_POWER_ON 0
#define PGFSM_POWER_OFF 2
-void print_microsec(struct dc_context *dc_ctx,
- struct dc_log_buffer_ctx *log_ctx,
- uint32_t ref_cycle)
+static void print_microsec(struct dc_context *dc_ctx,
+ struct dc_log_buffer_ctx *log_ctx,
+ uint32_t ref_cycle)
{
const uint32_t ref_clk_mhz = dc_ctx->dc->res_pool->ref_clocks.dchub_ref_clock_inKhz / 1000;
static const unsigned int frac = 1000;
@@ -132,7 +132,8 @@ static void log_mpc_crc(struct dc *dc,
REG_READ(DPP_TOP0_DPP_CRC_VAL_B_A), REG_READ(DPP_TOP0_DPP_CRC_VAL_R_G));
}
-void dcn10_log_hubbub_state(struct dc *dc, struct dc_log_buffer_ctx *log_ctx)
+static void dcn10_log_hubbub_state(struct dc *dc,
+ struct dc_log_buffer_ctx *log_ctx)
{
struct dc_context *dc_ctx = dc->ctx;
struct dcn_hubbub_wm wm;
@@ -467,8 +468,6 @@ void dcn10_log_hw_state(struct dc *dc,
log_mpc_crc(dc, log_ctx);
{
- int hpo_dp_link_enc_count = 0;
-
if (pool->hpo_dp_stream_enc_count > 0) {
DTN_INFO("DP HPO S_ENC: Enabled OTG Format Depth Vid SDP Compressed Link\n");
for (i = 0; i < pool->hpo_dp_stream_enc_count; i++) {
@@ -499,18 +498,14 @@ void dcn10_log_hw_state(struct dc *dc,
}
/* log DP HPO L_ENC section if any hpo_dp_link_enc exists */
- for (i = 0; i < dc->link_count; i++)
- if (dc->links[i]->hpo_dp_link_enc)
- hpo_dp_link_enc_count++;
-
- if (hpo_dp_link_enc_count) {
+ if (pool->hpo_dp_link_enc_count) {
DTN_INFO("DP HPO L_ENC: Enabled Mode Lanes Stream Slots VC Rate X VC Rate Y\n");
- for (i = 0; i < dc->link_count; i++) {
- struct hpo_dp_link_encoder *hpo_dp_link_enc = dc->links[i]->hpo_dp_link_enc;
+ for (i = 0; i < pool->hpo_dp_link_enc_count; i++) {
+ struct hpo_dp_link_encoder *hpo_dp_link_enc = pool->hpo_dp_link_enc[i];
struct hpo_dp_link_enc_state hpo_dp_le_state = {0};
- if (hpo_dp_link_enc && hpo_dp_link_enc->funcs->read_state) {
+ if (hpo_dp_link_enc->funcs->read_state) {
hpo_dp_link_enc->funcs->read_state(hpo_dp_link_enc, &hpo_dp_le_state);
DTN_INFO("[%d]: %d %6s %d %d %d %d %d\n",
hpo_dp_link_enc->inst,
@@ -1362,11 +1357,48 @@ void dcn10_init_pipes(struct dc *dc, struct dc_state *context)
tg->funcs->tg_init(tg);
}
+
+ /* Power gate DSCs */
+ if (hws->funcs.dsc_pg_control != NULL) {
+ uint32_t num_opps = 0;
+ uint32_t opp_id_src0 = OPP_ID_INVALID;
+ uint32_t opp_id_src1 = OPP_ID_INVALID;
+
+ // Step 1: To find out which OPTC is running & OPTC DSC is ON
+ for (i = 0; i < dc->res_pool->res_cap->num_timing_generator; i++) {
+ uint32_t optc_dsc_state = 0;
+ struct timing_generator *tg = dc->res_pool->timing_generators[i];
+
+ if (tg->funcs->is_tg_enabled(tg)) {
+ if (tg->funcs->get_dsc_status)
+ tg->funcs->get_dsc_status(tg, &optc_dsc_state);
+ // Only one OPTC with DSC is ON, so if we got one result, we would exit this block.
+ // non-zero value is DSC enabled
+ if (optc_dsc_state != 0) {
+ tg->funcs->get_optc_source(tg, &num_opps, &opp_id_src0, &opp_id_src1);
+ break;
+ }
+ }
+ }
+
+ // Step 2: To power down DSC but skip DSC of running OPTC
+ for (i = 0; i < dc->res_pool->res_cap->num_dsc; i++) {
+ struct dcn_dsc_state s = {0};
+
+ dc->res_pool->dscs[i]->funcs->dsc_read_state(dc->res_pool->dscs[i], &s);
+
+ if ((s.dsc_opp_source == opp_id_src0 || s.dsc_opp_source == opp_id_src1) &&
+ s.dsc_clock_en && s.dsc_fw_en)
+ continue;
+
+ hws->funcs.dsc_pg_control(hws, dc->res_pool->dscs[i]->inst, false);
+ }
+ }
}
void dcn10_init_hw(struct dc *dc)
{
- int i, j;
+ int i;
struct abm *abm = dc->res_pool->abm;
struct dmcu *dmcu = dc->res_pool->dmcu;
struct dce_hwseq *hws = dc->hwseq;
@@ -1468,43 +1500,8 @@ void dcn10_init_hw(struct dc *dc)
dmub_enable_outbox_notification(dc);
/* we want to turn off all dp displays before doing detection */
- if (dc->config.power_down_display_on_boot) {
- uint8_t dpcd_power_state = '\0';
- enum dc_status status = DC_ERROR_UNEXPECTED;
-
- for (i = 0; i < dc->link_count; i++) {
- if (dc->links[i]->connector_signal != SIGNAL_TYPE_DISPLAY_PORT)
- continue;
-
- /* DP 2.0 requires that LTTPR Caps be read first */
- dp_retrieve_lttpr_cap(dc->links[i]);
-
- /*
- * If any of the displays are lit up turn them off.
- * The reason is that some MST hubs cannot be turned off
- * completely until we tell them to do so.
- * If not turned off, then displays connected to MST hub
- * won't light up.
- */
- status = core_link_read_dpcd(dc->links[i], DP_SET_POWER,
- &dpcd_power_state, sizeof(dpcd_power_state));
- if (status == DC_OK && dpcd_power_state == DP_POWER_STATE_D0) {
- /* blank dp stream before power off receiver*/
- if (dc->links[i]->link_enc->funcs->get_dig_frontend) {
- unsigned int fe = dc->links[i]->link_enc->funcs->get_dig_frontend(dc->links[i]->link_enc);
-
- for (j = 0; j < dc->res_pool->stream_enc_count; j++) {
- if (fe == dc->res_pool->stream_enc[j]->id) {
- dc->res_pool->stream_enc[j]->funcs->dp_blank(dc->links[i],
- dc->res_pool->stream_enc[j]);
- break;
- }
- }
- }
- dp_receiver_power_ctrl(dc->links[i], false);
- }
- }
- }
+ if (dc->config.power_down_display_on_boot)
+ dc_link_blank_all_dp_displays(dc);
/* If taking control over from VBIOS, we may want to optimize our first
* mode set, so we need to skip powering down pipes until we know which
@@ -1970,10 +1967,9 @@ static bool wait_for_reset_trigger_to_occur(
return rc;
}
-uint64_t reduceSizeAndFraction(
- uint64_t *numerator,
- uint64_t *denominator,
- bool checkUint32Bounary)
+static uint64_t reduceSizeAndFraction(uint64_t *numerator,
+ uint64_t *denominator,
+ bool checkUint32Bounary)
{
int i;
bool ret = checkUint32Bounary == false;
@@ -2021,7 +2017,7 @@ uint64_t reduceSizeAndFraction(
return ret;
}
-bool is_low_refresh_rate(struct pipe_ctx *pipe)
+static bool is_low_refresh_rate(struct pipe_ctx *pipe)
{
uint32_t master_pipe_refresh_rate =
pipe->stream->timing.pix_clk_100hz * 100 /
@@ -2030,7 +2026,8 @@ bool is_low_refresh_rate(struct pipe_ctx *pipe)
return master_pipe_refresh_rate <= 30;
}
-uint8_t get_clock_divider(struct pipe_ctx *pipe, bool account_low_refresh_rate)
+static uint8_t get_clock_divider(struct pipe_ctx *pipe,
+ bool account_low_refresh_rate)
{
uint32_t clock_divider = 1;
uint32_t numpipes = 1;
@@ -2050,10 +2047,8 @@ uint8_t get_clock_divider(struct pipe_ctx *pipe, bool account_low_refresh_rate)
return clock_divider;
}
-int dcn10_align_pixel_clocks(
- struct dc *dc,
- int group_size,
- struct pipe_ctx *grouped_pipes[])
+static int dcn10_align_pixel_clocks(struct dc *dc, int group_size,
+ struct pipe_ctx *grouped_pipes[])
{
struct dc_context *dc_ctx = dc->ctx;
int i, master = -1, embedded = -1;
@@ -2342,7 +2337,7 @@ static void mmhub_read_vm_context0_settings(struct dcn10_hubp *hubp1,
}
-void dcn10_program_pte_vm(struct dce_hwseq *hws, struct hubp *hubp)
+static void dcn10_program_pte_vm(struct dce_hwseq *hws, struct hubp *hubp)
{
struct dcn10_hubp *hubp1 = TO_DCN10_HUBP(hubp);
struct vm_system_aperture_param apt = {0};
@@ -2624,7 +2619,7 @@ static void dcn10_update_dchubp_dpp(
/* new calculated dispclk, dppclk are stored in
* context->bw_ctx.bw.dcn.clk.dispclk_khz / dppclk_khz. current
* dispclk, dppclk are from dc->clk_mgr->clks.dispclk_khz.
- * dcn_validate_bandwidth compute new dispclk, dppclk.
+ * dcn10_validate_bandwidth compute new dispclk, dppclk.
* dispclk will put in use after optimize_bandwidth when
* ramp_up_dispclk_with_dpp is called.
* there are two places for dppclk be put in use. One location
@@ -2638,7 +2633,7 @@ static void dcn10_update_dchubp_dpp(
* for example, eDP + external dp, change resolution of DP from
* 1920x1080x144hz to 1280x960x60hz.
* before change: dispclk = 337889 dppclk = 337889
- * change mode, dcn_validate_bandwidth calculate
+ * change mode, dcn10_validate_bandwidth calculate
* dispclk = 143122 dppclk = 143122
* update_dchubp_dpp be executed before dispclk be updated,
* dispclk = 337889, but dppclk use new value dispclk /2 =
diff --git a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_init.c b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_init.c
index 34001a3..10e613e 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_init.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_init.c
@@ -78,6 +78,7 @@ static const struct hw_sequencer_funcs dcn10_funcs = {
.get_clock = dcn10_get_clock,
.get_vupdate_offset_from_vsync = dcn10_get_vupdate_offset_from_vsync,
.calc_vupdate_position = dcn10_calc_vupdate_position,
+ .power_down = dce110_power_down,
.set_backlight_level = dce110_set_backlight_level,
.set_abm_immediate_disable = dce110_set_abm_immediate_disable,
.set_pipe = dce110_set_pipe,
diff --git a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_link_encoder.c b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_link_encoder.c
index 2dc4b4e..f4b34c1 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_link_encoder.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_link_encoder.c
@@ -646,8 +646,9 @@ static bool dcn10_link_encoder_validate_hdmi_output(
crtc_timing->pixel_encoding == PIXEL_ENCODING_YCBCR420)
return false;
- if (!enc10->base.features.flags.bits.HDMI_6GB_EN &&
- adjusted_pix_clk_100hz >= 3000000)
+ if ((!enc10->base.features.flags.bits.HDMI_6GB_EN ||
+ enc10->base.ctx->dc->debug.hdmi20_disable) &&
+ adjusted_pix_clk_100hz >= 3000000)
return false;
if (enc10->base.ctx->dc->debug.hdmi20_disable &&
crtc_timing->pixel_encoding == PIXEL_ENCODING_YCBCR420)
diff --git a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_opp.c b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_opp.c
index d54d731..2c40935 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_opp.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_opp.c
@@ -348,36 +348,6 @@ void opp1_program_stereo(
*/
}
-void opp1_program_oppbuf(
- struct output_pixel_processor *opp,
- struct oppbuf_params *oppbuf)
-{
- struct dcn10_opp *oppn10 = TO_DCN10_OPP(opp);
-
- /* Program the oppbuf active width to be the frame width from mpc */
- REG_UPDATE(OPPBUF_CONTROL, OPPBUF_ACTIVE_WIDTH, oppbuf->active_width);
-
- /* Specifies the number of segments in multi-segment mode (DP-MSO operation)
- * description "In 1/2/4 segment mode, specifies the horizontal active width in pixels of the display panel.
- * In 4 segment split left/right mode, specifies the horizontal 1/2 active width in pixels of the display panel.
- * Used to determine segment boundaries in multi-segment mode. Used to determine the width of the vertical active space in 3D frame packed modes.
- * OPPBUF_ACTIVE_WIDTH must be integer divisible by the total number of segments."
- */
- REG_UPDATE(OPPBUF_CONTROL, OPPBUF_DISPLAY_SEGMENTATION, oppbuf->mso_segmentation);
-
- /* description "Specifies the number of overlap pixels (1-8 overlapping pixels supported), used in multi-segment mode (DP-MSO operation)" */
- REG_UPDATE(OPPBUF_CONTROL, OPPBUF_OVERLAP_PIXEL_NUM, oppbuf->mso_overlap_pixel_num);
-
- /* description "Specifies the number of times a pixel is replicated (0-15 pixel replications supported).
- * A value of 0 disables replication. The total number of times a pixel is output is OPPBUF_PIXEL_REPETITION + 1."
- */
- REG_UPDATE(OPPBUF_CONTROL, OPPBUF_PIXEL_REPETITION, oppbuf->pixel_repetition);
-
- /* Controls the number of padded pixels at the end of a segment */
- if (REG(OPPBUF_CONTROL1))
- REG_UPDATE(OPPBUF_CONTROL1, OPPBUF_NUM_SEGMENT_PADDED_PIXELS, oppbuf->num_segment_padded_pixels);
-}
-
void opp1_pipe_clock_control(struct output_pixel_processor *opp, bool enable)
{
struct dcn10_opp *oppn10 = TO_DCN10_OPP(opp);
diff --git a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_optc.c b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_optc.c
index 3d2a284..b1671b0 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_optc.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_optc.c
@@ -132,22 +132,6 @@ void optc1_setup_vertical_interrupt2(
}
/**
- * Vupdate keepout can be set to a window to block the update lock for that pipe from changing.
- * Start offset begins with vstartup and goes for x number of clocks,
- * end offset starts from end of vupdate to x number of clocks.
- */
-void optc1_set_vupdate_keepout(struct timing_generator *optc,
- struct vupdate_keepout_params *params)
-{
- struct optc *optc1 = DCN10TG_FROM_TG(optc);
-
- REG_SET_3(OTG_VUPDATE_KEEPOUT, 0,
- MASTER_UPDATE_LOCK_VUPDATE_KEEPOUT_START_OFFSET, params->start_offset,
- MASTER_UPDATE_LOCK_VUPDATE_KEEPOUT_END_OFFSET, params->end_offset,
- OTG_MASTER_UPDATE_LOCK_VUPDATE_KEEPOUT_EN, params->enable);
-}
-
-/**
* program_timing_generator used by mode timing set
* Program CRTC Timing Registers - OTG_H_*, OTG_V_*, Pixel repetition.
* Including SYNC. Call BIOS command table to program Timings.
@@ -876,7 +860,7 @@ void optc1_set_static_screen_control(
OTG_STATIC_SCREEN_FRAME_COUNT, num_frames);
}
-void optc1_setup_manual_trigger(struct timing_generator *optc)
+static void optc1_setup_manual_trigger(struct timing_generator *optc)
{
struct optc *optc1 = DCN10TG_FROM_TG(optc);
@@ -894,7 +878,7 @@ void optc1_setup_manual_trigger(struct timing_generator *optc)
OTG_TRIGA_CLEAR, 1);
}
-void optc1_program_manual_trigger(struct timing_generator *optc)
+static void optc1_program_manual_trigger(struct timing_generator *optc)
{
struct optc *optc1 = DCN10TG_FROM_TG(optc);
diff --git a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_resource.c b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_resource.c
index f37551e..858b721 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_resource.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_resource.c
@@ -686,9 +686,8 @@ static struct output_pixel_processor *dcn10_opp_create(
return &opp->base;
}
-struct dce_aux *dcn10_aux_engine_create(
- struct dc_context *ctx,
- uint32_t inst)
+static struct dce_aux *dcn10_aux_engine_create(struct dc_context *ctx,
+ uint32_t inst)
{
struct aux_engine_dce110 *aux_engine =
kzalloc(sizeof(struct aux_engine_dce110), GFP_KERNEL);
@@ -724,9 +723,8 @@ static const struct dce_i2c_mask i2c_masks = {
I2C_COMMON_MASK_SH_LIST_DCE110(_MASK)
};
-struct dce_i2c_hw *dcn10_i2c_hw_create(
- struct dc_context *ctx,
- uint32_t inst)
+static struct dce_i2c_hw *dcn10_i2c_hw_create(struct dc_context *ctx,
+ uint32_t inst)
{
struct dce_i2c_hw *dce_i2c_hw =
kzalloc(sizeof(struct dce_i2c_hw), GFP_KERNEL);
@@ -805,7 +803,7 @@ static const struct encoder_feature_support link_enc_feature = {
.flags.bits.IS_TPS4_CAPABLE = true
};
-struct link_encoder *dcn10_link_encoder_create(
+static struct link_encoder *dcn10_link_encoder_create(
const struct encoder_init_data *enc_init_data)
{
struct dcn10_link_encoder *enc10 =
@@ -847,7 +845,7 @@ static struct panel_cntl *dcn10_panel_cntl_create(const struct panel_cntl_init_d
return &panel_cntl->base;
}
-struct clock_source *dcn10_clock_source_create(
+static struct clock_source *dcn10_clock_source_create(
struct dc_context *ctx,
struct dc_bios *bios,
enum clock_source_id id,
@@ -945,7 +943,7 @@ static const struct resource_create_funcs res_create_maximus_funcs = {
.create_hwseq = dcn10_hwseq_create,
};
-void dcn10_clock_source_destroy(struct clock_source **clk_src)
+static void dcn10_clock_source_destroy(struct clock_source **clk_src)
{
kfree(TO_DCE110_CLK_SRC(*clk_src));
*clk_src = NULL;
@@ -978,10 +976,8 @@ static void dcn10_resource_destruct(struct dcn10_resource_pool *pool)
pool->base.mpc = NULL;
}
- if (pool->base.hubbub != NULL) {
- kfree(pool->base.hubbub);
- pool->base.hubbub = NULL;
- }
+ kfree(pool->base.hubbub);
+ pool->base.hubbub = NULL;
for (i = 0; i < pool->base.pipe_count; i++) {
if (pool->base.opps[i] != NULL)
@@ -1011,14 +1007,10 @@ static void dcn10_resource_destruct(struct dcn10_resource_pool *pool)
for (i = 0; i < pool->base.res_cap->num_ddc; i++) {
if (pool->base.engines[i] != NULL)
dce110_engine_destroy(&pool->base.engines[i]);
- if (pool->base.hw_i2cs[i] != NULL) {
- kfree(pool->base.hw_i2cs[i]);
- pool->base.hw_i2cs[i] = NULL;
- }
- if (pool->base.sw_i2cs[i] != NULL) {
- kfree(pool->base.sw_i2cs[i]);
- pool->base.sw_i2cs[i] = NULL;
- }
+ kfree(pool->base.hw_i2cs[i]);
+ pool->base.hw_i2cs[i] = NULL;
+ kfree(pool->base.sw_i2cs[i]);
+ pool->base.sw_i2cs[i] = NULL;
}
for (i = 0; i < pool->base.audio_count; i++) {
@@ -1128,7 +1120,7 @@ static enum dc_status build_mapped_resource(
return DC_OK;
}
-enum dc_status dcn10_add_stream_to_ctx(
+static enum dc_status dcn10_add_stream_to_ctx(
struct dc *dc,
struct dc_state *new_ctx,
struct dc_stream_state *dc_stream)
@@ -1320,7 +1312,7 @@ static const struct resource_funcs dcn10_res_pool_funcs = {
.destroy = dcn10_destroy_resource_pool,
.link_enc_create = dcn10_link_encoder_create,
.panel_cntl_create = dcn10_panel_cntl_create,
- .validate_bandwidth = dcn_validate_bandwidth,
+ .validate_bandwidth = dcn10_validate_bandwidth,
.acquire_idle_pipe_for_layer = dcn10_acquire_idle_pipe_for_layer,
.validate_plane = dcn10_validate_plane,
.validate_global = dcn10_validate_global,
diff --git a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_stream_encoder.c b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_stream_encoder.c
index b0c08ee..bf4436d 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_stream_encoder.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_stream_encoder.c
@@ -902,6 +902,19 @@ void enc1_stream_encoder_stop_dp_info_packets(
}
+void enc1_stream_encoder_reset_fifo(
+ struct stream_encoder *enc)
+{
+ struct dcn10_stream_encoder *enc1 = DCN10STRENC_FROM_STRENC(enc);
+
+ /* set DIG_START to 0x1 to reset FIFO */
+ REG_UPDATE(DIG_FE_CNTL, DIG_START, 1);
+ udelay(100);
+
+ /* write 0 to take the FIFO out of reset */
+ REG_UPDATE(DIG_FE_CNTL, DIG_START, 0);
+}
+
void enc1_stream_encoder_dp_blank(
struct dc_link *link,
struct stream_encoder *enc)
@@ -1587,6 +1600,8 @@ static const struct stream_encoder_funcs dcn10_str_enc_funcs = {
enc1_stream_encoder_send_immediate_sdp_message,
.stop_dp_info_packets =
enc1_stream_encoder_stop_dp_info_packets,
+ .reset_fifo =
+ enc1_stream_encoder_reset_fifo,
.dp_blank =
enc1_stream_encoder_dp_blank,
.dp_unblank =
diff --git a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_stream_encoder.h b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_stream_encoder.h
index 687d7e4..a146a41 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_stream_encoder.h
+++ b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_stream_encoder.h
@@ -626,6 +626,9 @@ void enc1_stream_encoder_send_immediate_sdp_message(
void enc1_stream_encoder_stop_dp_info_packets(
struct stream_encoder *enc);
+void enc1_stream_encoder_reset_fifo(
+ struct stream_encoder *enc);
+
void enc1_stream_encoder_dp_blank(
struct dc_link *link,
struct stream_encoder *enc);
diff --git a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_dpp.c b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_dpp.c
index a9e420c..970b65e 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_dpp.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_dpp.c
@@ -251,20 +251,6 @@ static void dpp2_cnv_setup (
}
-void dpp2_cnv_set_bias_scale(
- struct dpp *dpp_base,
- struct dc_bias_and_scale *bias_and_scale)
-{
- struct dcn20_dpp *dpp = TO_DCN20_DPP(dpp_base);
-
- REG_UPDATE(FCNV_FP_BIAS_R, FCNV_FP_BIAS_R, bias_and_scale->bias_red);
- REG_UPDATE(FCNV_FP_BIAS_G, FCNV_FP_BIAS_G, bias_and_scale->bias_green);
- REG_UPDATE(FCNV_FP_BIAS_B, FCNV_FP_BIAS_B, bias_and_scale->bias_blue);
- REG_UPDATE(FCNV_FP_SCALE_R, FCNV_FP_SCALE_R, bias_and_scale->scale_red);
- REG_UPDATE(FCNV_FP_SCALE_G, FCNV_FP_SCALE_G, bias_and_scale->scale_green);
- REG_UPDATE(FCNV_FP_SCALE_B, FCNV_FP_SCALE_B, bias_and_scale->scale_blue);
-}
-
/*compute the maximum number of lines that we can fit in the line buffer*/
void dscl2_calc_lb_num_partitions(
const struct scaler_data *scl_data,
diff --git a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_dsc.c b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_dsc.c
index 79b640e..ef5c4c0 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_dsc.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_dsc.c
@@ -162,6 +162,8 @@ static void dsc2_read_state(struct display_stream_compressor *dsc, struct dcn_ds
REG_GET(DSCC_PPS_CONFIG2, PIC_WIDTH, &s->dsc_pic_width);
REG_GET(DSCC_PPS_CONFIG2, PIC_HEIGHT, &s->dsc_pic_height);
REG_GET(DSCC_PPS_CONFIG7, SLICE_BPG_OFFSET, &s->dsc_slice_bpg_offset);
+ REG_GET_2(DSCRM_DSC_FORWARD_CONFIG, DSCRM_DSC_FORWARD_EN, &s->dsc_fw_en,
+ DSCRM_DSC_OPP_PIPE_SOURCE, &s->dsc_opp_source);
}
diff --git a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_dwb_scl.c b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_dwb_scl.c
index 880954a..994fb73 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_dwb_scl.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_dwb_scl.c
@@ -527,7 +527,7 @@ static const uint16_t filter_12tap_16p_183[108] = {
0, 84, 16328, 16032, 416, 1944, 1944, 416, 16032, 16328, 84, 0,
};
-const uint16_t *wbscl_get_filter_3tap_16p(struct fixed31_32 ratio)
+static const uint16_t *wbscl_get_filter_3tap_16p(struct fixed31_32 ratio)
{
if (ratio.value < dc_fixpt_one.value)
return filter_3tap_16p_upscale;
@@ -539,7 +539,7 @@ const uint16_t *wbscl_get_filter_3tap_16p(struct fixed31_32 ratio)
return filter_3tap_16p_183;
}
-const uint16_t *wbscl_get_filter_4tap_16p(struct fixed31_32 ratio)
+static const uint16_t *wbscl_get_filter_4tap_16p(struct fixed31_32 ratio)
{
if (ratio.value < dc_fixpt_one.value)
return filter_4tap_16p_upscale;
diff --git a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_hubp.c b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_hubp.c
index 5adf42a..dc1752e 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_hubp.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_hubp.c
@@ -192,9 +192,8 @@ void hubp2_vready_at_or_After_vsync(struct hubp *hubp,
REG_UPDATE(DCHUBP_CNTL, HUBP_VREADY_AT_OR_AFTER_VSYNC, value);
}
-void hubp2_program_requestor(
- struct hubp *hubp,
- struct _vcs_dpi_display_rq_regs_st *rq_regs)
+static void hubp2_program_requestor(struct hubp *hubp,
+ struct _vcs_dpi_display_rq_regs_st *rq_regs)
{
struct dcn20_hubp *hubp2 = TO_DCN20_HUBP(hubp);
@@ -930,6 +929,16 @@ bool hubp2_is_flip_pending(struct hubp *hubp)
void hubp2_set_blank(struct hubp *hubp, bool blank)
{
+ hubp2_set_blank_regs(hubp, blank);
+
+ if (blank) {
+ hubp->mpcc_id = 0xf;
+ hubp->opp_id = OPP_ID_INVALID;
+ }
+}
+
+void hubp2_set_blank_regs(struct hubp *hubp, bool blank)
+{
struct dcn20_hubp *hubp2 = TO_DCN20_HUBP(hubp);
uint32_t blank_en = blank ? 1 : 0;
@@ -951,9 +960,6 @@ void hubp2_set_blank(struct hubp *hubp, bool blank)
HUBP_NO_OUTSTANDING_REQ, 1,
1, 200);
}
-
- hubp->mpcc_id = 0xf;
- hubp->opp_id = OPP_ID_INVALID;
}
}
@@ -1285,7 +1291,7 @@ void hubp2_read_state(struct hubp *hubp)
}
-void hubp2_validate_dml_output(struct hubp *hubp,
+static void hubp2_validate_dml_output(struct hubp *hubp,
struct dc_context *ctx,
struct _vcs_dpi_display_rq_regs_st *dml_rq_regs,
struct _vcs_dpi_display_dlg_regs_st *dml_dlg_attr,
@@ -1603,6 +1609,7 @@ static struct hubp_funcs dcn20_hubp_funcs = {
.hubp_setup_interdependent = hubp2_setup_interdependent,
.hubp_set_vm_system_aperture_settings = hubp2_set_vm_system_aperture_settings,
.set_blank = hubp2_set_blank,
+ .set_blank_regs = hubp2_set_blank_regs,
.dcc_control = hubp2_dcc_control,
.mem_program_viewport = min_set_viewport,
.set_cursor_attributes = hubp2_cursor_set_attributes,
diff --git a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_hubp.h b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_hubp.h
index eea2254..9204c3e 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_hubp.h
+++ b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_hubp.h
@@ -330,6 +330,7 @@ void hubp2_program_surface_config(
bool hubp2_is_flip_pending(struct hubp *hubp);
void hubp2_set_blank(struct hubp *hubp, bool blank);
+void hubp2_set_blank_regs(struct hubp *hubp, bool blank);
void hubp2_cursor_set_position(
struct hubp *hubp,
diff --git a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_hwseq.c b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_hwseq.c
index e6af99a..4991e93 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_hwseq.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_hwseq.c
@@ -615,6 +615,11 @@ void dcn20_disable_plane(struct dc *dc, struct pipe_ctx *pipe_ctx)
pipe_ctx->pipe_idx);
}
+void dcn20_disable_pixel_data(struct dc *dc, struct pipe_ctx *pipe_ctx, bool blank)
+{
+ dcn20_blank_pixel_data(dc, pipe_ctx, blank);
+}
+
static int calc_mpc_flow_ctrl_cnt(const struct dc_stream_state *stream,
int opp_cnt)
{
@@ -1080,10 +1085,8 @@ static void dcn20_power_on_plane(
}
}
-void dcn20_enable_plane(
- struct dc *dc,
- struct pipe_ctx *pipe_ctx,
- struct dc_state *context)
+static void dcn20_enable_plane(struct dc *dc, struct pipe_ctx *pipe_ctx,
+ struct dc_state *context)
{
//if (dc->debug.sanity_checks) {
// dcn10_verify_allow_pstate_change_high(dc);
@@ -1842,6 +1845,11 @@ void dcn20_optimize_bandwidth(
dc->res_pool->ref_clocks.dchub_ref_clock_inKhz / 1000,
true);
+ if (dc->clk_mgr->dc_mode_softmax_enabled)
+ if (dc->clk_mgr->clks.dramclk_khz > dc->clk_mgr->bw_params->dc_mode_softmax_memclk * 1000 &&
+ context->bw_ctx.bw.dcn.clk.dramclk_khz <= dc->clk_mgr->bw_params->dc_mode_softmax_memclk * 1000)
+ dc->clk_mgr->funcs->set_max_memclk(dc->clk_mgr, dc->clk_mgr->bw_params->dc_mode_softmax_memclk);
+
dc->clk_mgr->funcs->update_clocks(
dc->clk_mgr,
context,
@@ -2406,7 +2414,7 @@ void dcn20_enable_stream(struct pipe_ctx *pipe_ctx)
pipe_ctx->stream_res.hpo_dp_stream_enc->funcs->map_stream_to_link(
pipe_ctx->stream_res.hpo_dp_stream_enc,
pipe_ctx->stream_res.hpo_dp_stream_enc->inst,
- link->hpo_dp_link_enc->inst);
+ pipe_ctx->link_res.hpo_dp_link_enc->inst);
}
if (!is_dp_128b_132b_signal(pipe_ctx) && link_enc)
diff --git a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_hwseq.h b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_hwseq.h
index 6bba191..33a36c0 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_hwseq.h
+++ b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_hwseq.h
@@ -53,6 +53,10 @@ void dcn20_enable_stream(struct pipe_ctx *pipe_ctx);
void dcn20_unblank_stream(struct pipe_ctx *pipe_ctx,
struct dc_link_settings *link_settings);
void dcn20_disable_plane(struct dc *dc, struct pipe_ctx *pipe_ctx);
+void dcn20_disable_pixel_data(
+ struct dc *dc,
+ struct pipe_ctx *pipe_ctx,
+ bool blank);
void dcn20_blank_pixel_data(
struct dc *dc,
struct pipe_ctx *pipe_ctx,
diff --git a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_init.c b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_init.c
index 5cfd4b0..91e4885 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_init.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_init.c
@@ -27,6 +27,8 @@
#include "dcn10/dcn10_hw_sequencer.h"
#include "dcn20_hwseq.h"
+#include "dcn20_init.h"
+
static const struct hw_sequencer_funcs dcn20_funcs = {
.program_gamut_remap = dcn10_program_gamut_remap,
.init_hw = dcn10_init_hw,
diff --git a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_mpc.c b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_mpc.c
index 947eb0d..15734db 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_mpc.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_mpc.c
@@ -400,10 +400,9 @@ static void mpc20_program_ogam_pwl(
}
-void apply_DEDCN20_305_wa(
- struct mpc *mpc,
- int mpcc_id, enum dc_lut_mode current_mode,
- enum dc_lut_mode next_mode)
+static void apply_DEDCN20_305_wa(struct mpc *mpc, int mpcc_id,
+ enum dc_lut_mode current_mode,
+ enum dc_lut_mode next_mode)
{
struct dcn20_mpc *mpc20 = TO_DCN20_MPC(mpc);
@@ -525,7 +524,7 @@ static void mpc2_init_mpcc(struct mpcc *mpcc, int mpcc_inst)
mpcc->sm_cfg.enable = false;
}
-struct mpcc *mpc2_get_mpcc_for_dpp(struct mpc_tree *tree, int dpp_id)
+static struct mpcc *mpc2_get_mpcc_for_dpp(struct mpc_tree *tree, int dpp_id)
{
struct mpcc *tmp_mpcc = tree->opp_list;
diff --git a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_optc.c b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_optc.c
index c90b851..0340fdd 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_optc.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_optc.c
@@ -73,21 +73,6 @@ bool optc2_enable_crtc(struct timing_generator *optc)
}
/**
- * DRR double buffering control to select buffer point
- * for V_TOTAL, H_TOTAL, VTOTAL_MIN, VTOTAL_MAX, VTOTAL_MIN_SEL and VTOTAL_MAX_SEL registers
- * Options: anytime, start of frame, dp start of frame (range timing)
- */
-void optc2_set_timing_db_mode(struct timing_generator *optc, bool enable)
-{
- struct optc *optc1 = DCN10TG_FROM_TG(optc);
-
- uint32_t blank_data_double_buffer_enable = enable ? 1 : 0;
-
- REG_UPDATE(OTG_DOUBLE_BUFFER_CONTROL,
- OTG_RANGE_TIMING_DBUF_UPDATE_MODE, blank_data_double_buffer_enable);
-}
-
-/**
*For the below, I'm not sure how your GSL parameters are stored in your env,
* so I will assume a gsl_params struct for now
*/
@@ -110,30 +95,6 @@ void optc2_set_gsl(struct timing_generator *optc,
}
-/* Use the gsl allow flip as the master update lock */
-void optc2_use_gsl_as_master_update_lock(struct timing_generator *optc,
- const struct gsl_params *params)
-{
- struct optc *optc1 = DCN10TG_FROM_TG(optc);
-
- REG_UPDATE(OTG_GSL_CONTROL,
- OTG_MASTER_UPDATE_LOCK_GSL_EN, params->master_update_lock_gsl_en);
-}
-
-/* You can control the GSL timing by limiting GSL to a window (X,Y) */
-void optc2_set_gsl_window(struct timing_generator *optc,
- const struct gsl_params *params)
-{
- struct optc *optc1 = DCN10TG_FROM_TG(optc);
-
- REG_SET_2(OTG_GSL_WINDOW_X, 0,
- OTG_GSL_WINDOW_START_X, params->gsl_window_start_x,
- OTG_GSL_WINDOW_END_X, params->gsl_window_end_x);
- REG_SET_2(OTG_GSL_WINDOW_Y, 0,
- OTG_GSL_WINDOW_START_Y, params->gsl_window_start_y,
- OTG_GSL_WINDOW_END_Y, params->gsl_window_end_y);
-}
-
void optc2_set_gsl_source_select(
struct timing_generator *optc,
int group_idx,
@@ -156,18 +117,6 @@ void optc2_set_gsl_source_select(
}
}
-/* DSC encoder frame start controls: x = h position, line_num = # of lines from vstartup */
-void optc2_set_dsc_encoder_frame_start(struct timing_generator *optc,
- int x_position,
- int line_num)
-{
- struct optc *optc1 = DCN10TG_FROM_TG(optc);
-
- REG_SET_2(OTG_DSC_START_POSITION, 0,
- OTG_DSC_START_POSITION_X, x_position,
- OTG_DSC_START_POSITION_LINE_NUM, line_num);
-}
-
/* Set DSC-related configuration.
* dsc_mode: 0 disables DSC, other values enable DSC in specified format
* sc_bytes_per_pixel: Bytes per pixel in u3.28 format
@@ -190,6 +139,19 @@ void optc2_set_dsc_config(struct timing_generator *optc,
OPTC_DSC_SLICE_WIDTH, dsc_slice_width);
}
+/* Get DSC-related configuration.
+ * dsc_mode: 0 disables DSC, other values enable DSC in specified format
+ */
+void optc2_get_dsc_status(struct timing_generator *optc,
+ uint32_t *dsc_mode)
+{
+ struct optc *optc1 = DCN10TG_FROM_TG(optc);
+
+ REG_GET(OPTC_DATA_FORMAT_CONTROL,
+ OPTC_DSC_MODE, dsc_mode);
+}
+
+
/*TEMP: Need to figure out inheritance model here.*/
bool optc2_is_two_pixels_per_containter(const struct dc_crtc_timing *timing)
{
@@ -280,8 +242,8 @@ void optc2_get_optc_source(struct timing_generator *optc,
*num_of_src_opp = 1;
}
-void optc2_set_dwb_source(struct timing_generator *optc,
- uint32_t dwb_pipe_inst)
+static void optc2_set_dwb_source(struct timing_generator *optc,
+ uint32_t dwb_pipe_inst)
{
struct optc *optc1 = DCN10TG_FROM_TG(optc);
@@ -293,7 +255,7 @@ void optc2_set_dwb_source(struct timing_generator *optc,
OPTC_DWB1_SOURCE_SELECT, optc->inst);
}
-void optc2_align_vblanks(
+static void optc2_align_vblanks(
struct timing_generator *optc_master,
struct timing_generator *optc_slave,
uint32_t master_pixel_clock_100Hz,
@@ -579,6 +541,7 @@ static struct timing_generator_funcs dcn20_tg_funcs = {
.get_crc = optc1_get_crc,
.configure_crc = optc2_configure_crc,
.set_dsc_config = optc2_set_dsc_config,
+ .get_dsc_status = optc2_get_dsc_status,
.set_dwb_source = optc2_set_dwb_source,
.set_odm_bypass = optc2_set_odm_bypass,
.set_odm_combine = optc2_set_odm_combine,
diff --git a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_optc.h b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_optc.h
index be19a68..f7968b9 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_optc.h
+++ b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_optc.h
@@ -98,6 +98,9 @@ void optc2_set_dsc_config(struct timing_generator *optc,
uint32_t dsc_bytes_per_pixel,
uint32_t dsc_slice_width);
+void optc2_get_dsc_status(struct timing_generator *optc,
+ uint32_t *dsc_mode);
+
void optc2_set_odm_bypass(struct timing_generator *optc,
const struct dc_crtc_timing *dc_crtc_timing);
diff --git a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_resource.c b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_resource.c
index 3883f91..2bc93df 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_resource.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_resource.c
@@ -1069,7 +1069,7 @@ static const struct dc_debug_options debug_defaults_drv = {
.timing_trace = false,
.clock_trace = true,
.disable_pplib_clock_request = true,
- .pipe_split_policy = MPC_SPLIT_AVOID_MULT_DISP,
+ .pipe_split_policy = MPC_SPLIT_DYNAMIC,
.force_single_disp_pipe_split = false,
.disable_dcc = DCC_ENABLE,
.vsr_support = true,
@@ -3093,8 +3093,7 @@ static enum dcn_zstate_support_state decide_zstate_support(struct dc *dc, struc
else if (context->stream_count == 1 && context->streams[0]->signal == SIGNAL_TYPE_EDP) {
struct dc_link *link = context->streams[0]->sink->link;
- if ((link->link_index == 0 && link->psr_settings.psr_feature_enabled)
- || context->bw_ctx.dml.vba.StutterPeriod > 5000.0)
+ if (link->link_index == 0 && context->bw_ctx.dml.vba.StutterPeriod > 5000.0)
return DCN_ZSTATE_SUPPORT_ALLOW;
else
return DCN_ZSTATE_SUPPORT_DISALLOW;
@@ -3796,6 +3795,8 @@ static bool dcn20_resource_construct(
dc->caps.color.mpc.ogam_rom_caps.hlg = 0;
dc->caps.color.mpc.ocsc = 1;
+ dc->caps.hdmi_frl_pcon_support = true;
+
if (dc->ctx->dce_environment == DCE_ENV_PRODUCTION_DRV) {
dc->debug = debug_defaults_drv;
} else if (dc->ctx->dce_environment == DCE_ENV_FPGA_MAXIMUS) {
diff --git a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_stream_encoder.c b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_stream_encoder.c
index aab25ca..8a70f92 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_stream_encoder.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_stream_encoder.c
@@ -593,6 +593,8 @@ static const struct stream_encoder_funcs dcn20_str_enc_funcs = {
enc1_stream_encoder_send_immediate_sdp_message,
.stop_dp_info_packets =
enc1_stream_encoder_stop_dp_info_packets,
+ .reset_fifo =
+ enc1_stream_encoder_reset_fifo,
.dp_blank =
enc1_stream_encoder_dp_blank,
.dp_unblank =
diff --git a/drivers/gpu/drm/amd/display/dc/dcn201/dcn201_dccg.c b/drivers/gpu/drm/amd/display/dc/dcn201/dcn201_dccg.c
index f5bf04f..9a34021 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn201/dcn201_dccg.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn201/dcn201_dccg.c
@@ -44,7 +44,8 @@
#define DC_LOGGER \
dccg->ctx->logger
-void dccg201_update_dpp_dto(struct dccg *dccg, int dpp_inst, int req_dppclk)
+static void dccg201_update_dpp_dto(struct dccg *dccg, int dpp_inst,
+ int req_dppclk)
{
/* vbios handles it */
}
diff --git a/drivers/gpu/drm/amd/display/dc/dcn201/dcn201_hubp.c b/drivers/gpu/drm/amd/display/dc/dcn201/dcn201_hubp.c
index 6b6f74d..35dd4ba 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn201/dcn201_hubp.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn201/dcn201_hubp.c
@@ -55,7 +55,7 @@ static void hubp201_program_surface_config(
hubp1_program_pixel_format(hubp, format);
}
-void hubp201_program_deadline(
+static void hubp201_program_deadline(
struct hubp *hubp,
struct _vcs_dpi_display_dlg_regs_st *dlg_attr,
struct _vcs_dpi_display_ttu_regs_st *ttu_attr)
@@ -63,9 +63,8 @@ void hubp201_program_deadline(
hubp1_program_deadline(hubp, dlg_attr, ttu_attr);
}
-void hubp201_program_requestor(
- struct hubp *hubp,
- struct _vcs_dpi_display_rq_regs_st *rq_regs)
+static void hubp201_program_requestor(struct hubp *hubp,
+ struct _vcs_dpi_display_rq_regs_st *rq_regs)
{
struct dcn201_hubp *hubp201 = TO_DCN201_HUBP(hubp);
diff --git a/drivers/gpu/drm/amd/display/dc/dcn201/dcn201_hwseq.c b/drivers/gpu/drm/amd/display/dc/dcn201/dcn201_hwseq.c
index cfd09b3f..fe22530 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn201/dcn201_hwseq.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn201/dcn201_hwseq.c
@@ -134,11 +134,12 @@ void dcn201_update_plane_addr(const struct dc *dc, struct pipe_ctx *pipe_ctx)
PHYSICAL_ADDRESS_LOC addr;
struct dc_plane_state *plane_state = pipe_ctx->plane_state;
struct dce_hwseq *hws = dc->hwseq;
- struct dc_plane_address uma = plane_state->address;
+ struct dc_plane_address uma;
if (plane_state == NULL)
return;
+ uma = plane_state->address;
addr_patched = patch_address_for_sbs_tb_stereo(pipe_ctx, &addr);
plane_address_in_gpu_space_to_uma(hws, &uma);
diff --git a/drivers/gpu/drm/amd/display/dc/dcn201/dcn201_link_encoder.c b/drivers/gpu/drm/amd/display/dc/dcn201/dcn201_link_encoder.c
index a65e8f7..7f9ec59 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn201/dcn201_link_encoder.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn201/dcn201_link_encoder.c
@@ -50,8 +50,8 @@
#define IND_REG(index) \
(enc10->link_regs->index)
-void dcn201_link_encoder_get_max_link_cap(struct link_encoder *enc,
- struct dc_link_settings *link_settings)
+static void dcn201_link_encoder_get_max_link_cap(struct link_encoder *enc,
+ struct dc_link_settings *link_settings)
{
uint32_t value1, value2;
struct dcn10_link_encoder *enc10 = TO_DCN10_LINK_ENC(enc);
@@ -66,7 +66,7 @@ void dcn201_link_encoder_get_max_link_cap(struct link_encoder *enc,
}
}
-bool dcn201_link_encoder_is_in_alt_mode(struct link_encoder *enc)
+static bool dcn201_link_encoder_is_in_alt_mode(struct link_encoder *enc)
{
uint32_t value;
struct dcn10_link_encoder *enc10 = TO_DCN10_LINK_ENC(enc);
diff --git a/drivers/gpu/drm/amd/display/dc/dcn201/dcn201_resource.c b/drivers/gpu/drm/amd/display/dc/dcn201/dcn201_resource.c
index 0fa3810..0bb7d3d 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn201/dcn201_resource.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn201/dcn201_resource.c
@@ -603,7 +603,7 @@ static const struct dc_debug_options debug_defaults_drv = {
.timing_trace = false,
.clock_trace = true,
.disable_pplib_clock_request = true,
- .pipe_split_policy = MPC_SPLIT_AVOID,
+ .pipe_split_policy = MPC_SPLIT_DYNAMIC,
.force_single_disp_pipe_split = false,
.disable_dcc = DCC_ENABLE,
.vsr_support = true,
@@ -672,9 +672,8 @@ static struct output_pixel_processor *dcn201_opp_create(
return &opp->base;
}
-struct dce_aux *dcn201_aux_engine_create(
- struct dc_context *ctx,
- uint32_t inst)
+static struct dce_aux *dcn201_aux_engine_create(struct dc_context *ctx,
+ uint32_t inst)
{
struct aux_engine_dce110 *aux_engine =
kzalloc(sizeof(struct aux_engine_dce110), GFP_ATOMIC);
@@ -706,9 +705,8 @@ static const struct dce_i2c_mask i2c_masks = {
I2C_COMMON_MASK_SH_LIST_DCN2(_MASK)
};
-struct dce_i2c_hw *dcn201_i2c_hw_create(
- struct dc_context *ctx,
- uint32_t inst)
+static struct dce_i2c_hw *dcn201_i2c_hw_create(struct dc_context *ctx,
+ uint32_t inst)
{
struct dce_i2c_hw *dce_i2c_hw =
kzalloc(sizeof(struct dce_i2c_hw), GFP_ATOMIC);
@@ -789,7 +787,7 @@ static const struct encoder_feature_support link_enc_feature = {
.flags.bits.IS_TPS4_CAPABLE = true
};
-struct link_encoder *dcn201_link_encoder_create(
+static struct link_encoder *dcn201_link_encoder_create(
const struct encoder_init_data *enc_init_data)
{
struct dcn20_link_encoder *enc20 =
@@ -811,7 +809,7 @@ struct link_encoder *dcn201_link_encoder_create(
return &enc10->base;
}
-struct clock_source *dcn201_clock_source_create(
+static struct clock_source *dcn201_clock_source_create(
struct dc_context *ctx,
struct dc_bios *bios,
enum clock_source_id id,
@@ -906,7 +904,7 @@ static const struct resource_create_funcs res_create_maximus_funcs = {
.create_hwseq = dcn201_hwseq_create,
};
-void dcn201_clock_source_destroy(struct clock_source **clk_src)
+static void dcn201_clock_source_destroy(struct clock_source **clk_src)
{
kfree(TO_DCE110_CLK_SRC(*clk_src));
*clk_src = NULL;
diff --git a/drivers/gpu/drm/amd/display/dc/dcn21/dcn21_hubbub.c b/drivers/gpu/drm/amd/display/dc/dcn21/dcn21_hubbub.c
index 36044cb..c5e200d 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn21/dcn21_hubbub.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn21/dcn21_hubbub.c
@@ -680,7 +680,7 @@ void hubbub21_wm_read_state(struct hubbub *hubbub,
DCHUBBUB_ARB_ALLOW_DRAM_CLK_CHANGE_WATERMARK_D, &s->dram_clk_chanage);
}
-void hubbub21_apply_DEDCN21_147_wa(struct hubbub *hubbub)
+static void hubbub21_apply_DEDCN21_147_wa(struct hubbub *hubbub)
{
struct dcn20_hubbub *hubbub1 = TO_DCN20_HUBBUB(hubbub);
uint32_t prog_wm_value;
diff --git a/drivers/gpu/drm/amd/display/dc/dcn21/dcn21_hubp.c b/drivers/gpu/drm/amd/display/dc/dcn21/dcn21_hubp.c
index 3de1bcf..58e459c 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn21/dcn21_hubp.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn21/dcn21_hubp.c
@@ -183,7 +183,7 @@ static void hubp21_setup(
}
-void hubp21_set_viewport(
+static void hubp21_set_viewport(
struct hubp *hubp,
const struct rect *viewport,
const struct rect *viewport_c)
@@ -225,8 +225,8 @@ void hubp21_set_viewport(
SEC_VIEWPORT_Y_START_C, viewport_c->y);
}
-void hubp21_set_vm_system_aperture_settings(struct hubp *hubp,
- struct vm_system_aperture_param *apt)
+static void hubp21_set_vm_system_aperture_settings(struct hubp *hubp,
+ struct vm_system_aperture_param *apt)
{
struct dcn21_hubp *hubp21 = TO_DCN21_HUBP(hubp);
@@ -248,7 +248,7 @@ void hubp21_set_vm_system_aperture_settings(struct hubp *hubp,
SYSTEM_ACCESS_MODE, 0x3);
}
-void hubp21_validate_dml_output(struct hubp *hubp,
+static void hubp21_validate_dml_output(struct hubp *hubp,
struct dc_context *ctx,
struct _vcs_dpi_display_rq_regs_st *dml_rq_regs,
struct _vcs_dpi_display_dlg_regs_st *dml_dlg_attr,
@@ -664,7 +664,8 @@ static void program_surface_flip_and_addr(struct hubp *hubp, struct surface_flip
flip_regs->DCSURF_PRIMARY_SURFACE_ADDRESS);
}
-void dmcub_PLAT_54186_wa(struct hubp *hubp, struct surface_flip_registers *flip_regs)
+static void dmcub_PLAT_54186_wa(struct hubp *hubp,
+ struct surface_flip_registers *flip_regs)
{
struct dc_dmub_srv *dmcub = hubp->ctx->dmub_srv;
struct dcn21_hubp *hubp21 = TO_DCN21_HUBP(hubp);
@@ -697,7 +698,7 @@ void dmcub_PLAT_54186_wa(struct hubp *hubp, struct surface_flip_registers *flip_
PERF_TRACE(); // TODO: remove after performance is stable.
}
-bool hubp21_program_surface_flip_and_addr(
+static bool hubp21_program_surface_flip_and_addr(
struct hubp *hubp,
const struct dc_plane_address *address,
bool flip_immediate)
@@ -805,7 +806,7 @@ bool hubp21_program_surface_flip_and_addr(
return true;
}
-void hubp21_init(struct hubp *hubp)
+static void hubp21_init(struct hubp *hubp)
{
// DEDCN21-133: Inconsistent row starting line for flip between DPTE and Meta
// This is a chicken bit to enable the ECO fix.
diff --git a/drivers/gpu/drm/amd/display/dc/dcn21/dcn21_init.c b/drivers/gpu/drm/amd/display/dc/dcn21/dcn21_init.c
index 54c11ba..b270f0b 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn21/dcn21_init.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn21/dcn21_init.c
@@ -28,6 +28,8 @@
#include "dcn20/dcn20_hwseq.h"
#include "dcn21_hwseq.h"
+#include "dcn21_init.h"
+
static const struct hw_sequencer_funcs dcn21_funcs = {
.program_gamut_remap = dcn10_program_gamut_remap,
.init_hw = dcn10_init_hw,
diff --git a/drivers/gpu/drm/amd/display/dc/dcn21/dcn21_link_encoder.c b/drivers/gpu/drm/amd/display/dc/dcn21/dcn21_link_encoder.c
index aa46c35..0a1ba6e 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn21/dcn21_link_encoder.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn21/dcn21_link_encoder.c
@@ -203,7 +203,7 @@ static bool update_cfg_data(
return true;
}
-bool dcn21_link_encoder_acquire_phy(struct link_encoder *enc)
+static bool dcn21_link_encoder_acquire_phy(struct link_encoder *enc)
{
struct dcn10_link_encoder *enc10 = TO_DCN10_LINK_ENC(enc);
int value;
@@ -277,7 +277,7 @@ void dcn21_link_encoder_enable_dp_output(
}
-void dcn21_link_encoder_enable_dp_mst_output(
+static void dcn21_link_encoder_enable_dp_mst_output(
struct link_encoder *enc,
const struct dc_link_settings *link_settings,
enum clock_source_id clock_source)
@@ -288,9 +288,8 @@ void dcn21_link_encoder_enable_dp_mst_output(
dcn10_link_encoder_enable_dp_mst_output(enc, link_settings, clock_source);
}
-void dcn21_link_encoder_disable_output(
- struct link_encoder *enc,
- enum signal_type signal)
+static void dcn21_link_encoder_disable_output(struct link_encoder *enc,
+ enum signal_type signal)
{
dcn10_link_encoder_disable_output(enc, signal);
diff --git a/drivers/gpu/drm/amd/display/dc/dcn21/dcn21_resource.c b/drivers/gpu/drm/amd/display/dc/dcn21/dcn21_resource.c
index d452a0d..e5cc6bf 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn21/dcn21_resource.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn21/dcn21_resource.c
@@ -784,9 +784,8 @@ static const struct dce_i2c_mask i2c_masks = {
I2C_COMMON_MASK_SH_LIST_DCN2(_MASK)
};
-struct dce_i2c_hw *dcn21_i2c_hw_create(
- struct dc_context *ctx,
- uint32_t inst)
+static struct dce_i2c_hw *dcn21_i2c_hw_create(struct dc_context *ctx,
+ uint32_t inst)
{
struct dce_i2c_hw *dce_i2c_hw =
kzalloc(sizeof(struct dce_i2c_hw), GFP_KERNEL);
@@ -874,7 +873,7 @@ static const struct dc_debug_options debug_defaults_drv = {
.clock_trace = true,
.disable_pplib_clock_request = true,
.min_disp_clk_khz = 100000,
- .pipe_split_policy = MPC_SPLIT_AVOID_MULT_DISP,
+ .pipe_split_policy = MPC_SPLIT_DYNAMIC,
.force_single_disp_pipe_split = false,
.disable_dcc = DCC_ENABLE,
.vsr_support = true,
@@ -1093,7 +1092,7 @@ static void patch_bounding_box(struct dc *dc, struct _vcs_dpi_soc_bounding_box_s
}
}
-void dcn21_calculate_wm(
+static void dcn21_calculate_wm(
struct dc *dc, struct dc_state *context,
display_e2e_pipe_params_st *pipes,
int *out_pipe_cnt,
@@ -1390,7 +1389,7 @@ static noinline bool dcn21_validate_bandwidth_fp(struct dc *dc,
* with DC_FP_START()/DC_FP_END(). Use the same approach as for
* dcn20_validate_bandwidth in dcn20_resource.c.
*/
-bool dcn21_validate_bandwidth(struct dc *dc, struct dc_state *context,
+static bool dcn21_validate_bandwidth(struct dc *dc, struct dc_state *context,
bool fast_validate)
{
bool voltage_supported;
@@ -1480,8 +1479,8 @@ static struct hubbub *dcn21_hubbub_create(struct dc_context *ctx)
return &hubbub->base;
}
-struct output_pixel_processor *dcn21_opp_create(
- struct dc_context *ctx, uint32_t inst)
+static struct output_pixel_processor *dcn21_opp_create(struct dc_context *ctx,
+ uint32_t inst)
{
struct dcn20_opp *opp =
kzalloc(sizeof(struct dcn20_opp), GFP_KERNEL);
@@ -1496,9 +1495,8 @@ struct output_pixel_processor *dcn21_opp_create(
return &opp->base;
}
-struct timing_generator *dcn21_timing_generator_create(
- struct dc_context *ctx,
- uint32_t instance)
+static struct timing_generator *dcn21_timing_generator_create(struct dc_context *ctx,
+ uint32_t instance)
{
struct optc *tgn10 =
kzalloc(sizeof(struct optc), GFP_KERNEL);
@@ -1518,7 +1516,7 @@ struct timing_generator *dcn21_timing_generator_create(
return &tgn10->base;
}
-struct mpc *dcn21_mpc_create(struct dc_context *ctx)
+static struct mpc *dcn21_mpc_create(struct dc_context *ctx)
{
struct dcn20_mpc *mpc20 = kzalloc(sizeof(struct dcn20_mpc),
GFP_KERNEL);
@@ -1545,8 +1543,8 @@ static void read_dce_straps(
}
-struct display_stream_compressor *dcn21_dsc_create(
- struct dc_context *ctx, uint32_t inst)
+static struct display_stream_compressor *dcn21_dsc_create(struct dc_context *ctx,
+ uint32_t inst)
{
struct dcn20_dsc *dsc =
kzalloc(sizeof(struct dcn20_dsc), GFP_KERNEL);
@@ -1683,9 +1681,8 @@ static struct dc_cap_funcs cap_funcs = {
.get_dcc_compression_cap = dcn20_get_dcc_compression_cap
};
-struct stream_encoder *dcn21_stream_encoder_create(
- enum engine_id eng_id,
- struct dc_context *ctx)
+static struct stream_encoder *dcn21_stream_encoder_create(enum engine_id eng_id,
+ struct dc_context *ctx)
{
struct dcn10_stream_encoder *enc1 =
kzalloc(sizeof(struct dcn10_stream_encoder), GFP_KERNEL);
@@ -1917,7 +1914,7 @@ static int dcn21_populate_dml_pipes_from_context(
return pipe_cnt;
}
-enum dc_status dcn21_patch_unknown_plane_state(struct dc_plane_state *plane_state)
+static enum dc_status dcn21_patch_unknown_plane_state(struct dc_plane_state *plane_state)
{
enum dc_status result = DC_OK;
@@ -2028,6 +2025,8 @@ static bool dcn21_resource_construct(
dc->caps.color.mpc.ogam_rom_caps.hlg = 0;
dc->caps.color.mpc.ocsc = 1;
+ dc->caps.hdmi_frl_pcon_support = true;
+
if (dc->ctx->dce_environment == DCE_ENV_PRODUCTION_DRV)
dc->debug = debug_defaults_drv;
else if (dc->ctx->dce_environment == DCE_ENV_FPGA_MAXIMUS) {
diff --git a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_dio_stream_encoder.c b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_dio_stream_encoder.c
index ebd9c35..8daa1273 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_dio_stream_encoder.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_dio_stream_encoder.c
@@ -50,22 +50,6 @@
enc1->base.ctx
-void convert_dc_info_packet_to_128(
- const struct dc_info_packet *info_packet,
- struct dc_info_packet_128 *info_packet_128)
-{
- unsigned int i;
-
- info_packet_128->hb0 = info_packet->hb0;
- info_packet_128->hb1 = info_packet->hb1;
- info_packet_128->hb2 = info_packet->hb2;
- info_packet_128->hb3 = info_packet->hb3;
-
- for (i = 0; i < 32; i++) {
- info_packet_128->sb[i] = info_packet->sb[i];
- }
-
-}
static void enc3_update_hdmi_info_packet(
struct dcn10_stream_encoder *enc1,
uint32_t packet_index,
@@ -489,7 +473,7 @@ static void enc3_dp_set_odm_combine(
}
/* setup stream encoder in dvi mode */
-void enc3_stream_encoder_dvi_set_stream_attribute(
+static void enc3_stream_encoder_dvi_set_stream_attribute(
struct stream_encoder *enc,
struct dc_crtc_timing *crtc_timing,
bool is_dual_link)
@@ -805,6 +789,8 @@ static const struct stream_encoder_funcs dcn30_str_enc_funcs = {
enc3_stream_encoder_update_dp_info_packets,
.stop_dp_info_packets =
enc1_stream_encoder_stop_dp_info_packets,
+ .reset_fifo =
+ enc1_stream_encoder_reset_fifo,
.dp_blank =
enc1_stream_encoder_dp_blank,
.dp_unblank =
diff --git a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_dpp.c b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_dpp.c
index c1d967e..ab3918c 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_dpp.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_dpp.c
@@ -41,8 +41,7 @@
dpp->tf_shift->field_name, dpp->tf_mask->field_name
-void dpp30_read_state(struct dpp *dpp_base,
- struct dcn_dpp_state *s)
+static void dpp30_read_state(struct dpp *dpp_base, struct dcn_dpp_state *s)
{
struct dcn20_dpp *dpp = TO_DCN20_DPP(dpp_base);
@@ -373,7 +372,7 @@ void dpp3_set_cursor_attributes(
}
-bool dpp3_get_optimal_number_of_taps(
+static bool dpp3_get_optimal_number_of_taps(
struct dpp *dpp,
struct scaler_data *scl_data,
const struct scaling_taps *in_taps)
@@ -474,22 +473,7 @@ bool dpp3_get_optimal_number_of_taps(
return true;
}
-void dpp3_cnv_set_bias_scale(
- struct dpp *dpp_base,
- struct dc_bias_and_scale *bias_and_scale)
-{
- struct dcn3_dpp *dpp = TO_DCN30_DPP(dpp_base);
-
- REG_UPDATE(FCNV_FP_BIAS_R, FCNV_FP_BIAS_R, bias_and_scale->bias_red);
- REG_UPDATE(FCNV_FP_BIAS_G, FCNV_FP_BIAS_G, bias_and_scale->bias_green);
- REG_UPDATE(FCNV_FP_BIAS_B, FCNV_FP_BIAS_B, bias_and_scale->bias_blue);
- REG_UPDATE(FCNV_FP_SCALE_R, FCNV_FP_SCALE_R, bias_and_scale->scale_red);
- REG_UPDATE(FCNV_FP_SCALE_G, FCNV_FP_SCALE_G, bias_and_scale->scale_green);
- REG_UPDATE(FCNV_FP_SCALE_B, FCNV_FP_SCALE_B, bias_and_scale->scale_blue);
-}
-
-void dpp3_deferred_update(
- struct dpp *dpp_base)
+static void dpp3_deferred_update(struct dpp *dpp_base)
{
int bypass_state;
struct dcn3_dpp *dpp = TO_DCN30_DPP(dpp_base);
@@ -751,8 +735,8 @@ static enum dc_lut_mode dpp3_get_blndgam_current(struct dpp *dpp_base)
return mode;
}
-bool dpp3_program_blnd_lut(
- struct dpp *dpp_base, const struct pwl_params *params)
+static bool dpp3_program_blnd_lut(struct dpp *dpp_base,
+ const struct pwl_params *params)
{
enum dc_lut_mode current_mode;
enum dc_lut_mode next_mode;
@@ -1164,9 +1148,8 @@ static void dpp3_program_shaper_lutb_settings(
}
-bool dpp3_program_shaper(
- struct dpp *dpp_base,
- const struct pwl_params *params)
+static bool dpp3_program_shaper(struct dpp *dpp_base,
+ const struct pwl_params *params)
{
enum dc_lut_mode current_mode;
enum dc_lut_mode next_mode;
@@ -1355,9 +1338,8 @@ static void dpp3_select_3dlut_ram_mask(
REG_SET(CM_3DLUT_INDEX, 0, CM_3DLUT_INDEX, 0);
}
-bool dpp3_program_3dlut(
- struct dpp *dpp_base,
- struct tetrahedral_params *params)
+static bool dpp3_program_3dlut(struct dpp *dpp_base,
+ struct tetrahedral_params *params)
{
enum dc_lut_mode mode;
bool is_17x17x17;
diff --git a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_hubp.c b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_hubp.c
index eac0892..6a4dcafb 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_hubp.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_hubp.c
@@ -490,6 +490,7 @@ static struct hubp_funcs dcn30_hubp_funcs = {
.hubp_setup_interdependent = hubp2_setup_interdependent,
.hubp_set_vm_system_aperture_settings = hubp3_set_vm_system_aperture_settings,
.set_blank = hubp2_set_blank,
+ .set_blank_regs = hubp2_set_blank_regs,
.dcc_control = hubp3_dcc_control,
.mem_program_viewport = min_set_viewport,
.set_cursor_attributes = hubp2_cursor_set_attributes,
diff --git a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_hwseq.c b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_hwseq.c
index df27171..1db1ca1 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_hwseq.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_hwseq.c
@@ -344,6 +344,17 @@ void dcn30_enable_writeback(
dwb->funcs->enable(dwb, &wb_info->dwb_params);
}
+void dcn30_prepare_bandwidth(struct dc *dc,
+ struct dc_state *context)
+{
+ if (dc->clk_mgr->dc_mode_softmax_enabled)
+ if (dc->clk_mgr->clks.dramclk_khz <= dc->clk_mgr->bw_params->dc_mode_softmax_memclk * 1000 &&
+ context->bw_ctx.bw.dcn.clk.dramclk_khz > dc->clk_mgr->bw_params->dc_mode_softmax_memclk * 1000)
+ dc->clk_mgr->funcs->set_max_memclk(dc->clk_mgr, dc->clk_mgr->bw_params->clk_table.entries[dc->clk_mgr->bw_params->clk_table.num_entries - 1].memclk_mhz);
+
+ dcn20_prepare_bandwidth(dc, context);
+}
+
void dcn30_disable_writeback(
struct dc *dc,
unsigned int dwb_pipe_inst)
@@ -437,7 +448,7 @@ void dcn30_init_hw(struct dc *dc)
struct dce_hwseq *hws = dc->hwseq;
struct dc_bios *dcb = dc->ctx->dc_bios;
struct resource_pool *res_pool = dc->res_pool;
- int i, j;
+ int i;
int edp_num;
uint32_t backlight = MAX_BACKLIGHT_LEVEL;
@@ -534,41 +545,8 @@ void dcn30_init_hw(struct dc *dc)
hws->funcs.dsc_pg_control(hws, res_pool->dscs[i]->inst, false);
/* we want to turn off all dp displays before doing detection */
- if (dc->config.power_down_display_on_boot) {
- uint8_t dpcd_power_state = '\0';
- enum dc_status status = DC_ERROR_UNEXPECTED;
-
- for (i = 0; i < dc->link_count; i++) {
- if (dc->links[i]->connector_signal != SIGNAL_TYPE_DISPLAY_PORT)
- continue;
- /* DP 2.0 states that LTTPR regs must be read first */
- dp_retrieve_lttpr_cap(dc->links[i]);
-
- /* if any of the displays are lit up turn them off */
- status = core_link_read_dpcd(dc->links[i], DP_SET_POWER,
- &dpcd_power_state, sizeof(dpcd_power_state));
- if (status == DC_OK && dpcd_power_state == DP_POWER_STATE_D0) {
- /* blank dp stream before power off receiver*/
- if (dc->links[i]->link_enc->funcs->get_dig_frontend) {
- unsigned int fe;
-
- fe = dc->links[i]->link_enc->funcs->get_dig_frontend(
- dc->links[i]->link_enc);
- if (fe == ENGINE_ID_UNKNOWN)
- continue;
-
- for (j = 0; j < dc->res_pool->stream_enc_count; j++) {
- if (fe == dc->res_pool->stream_enc[j]->id) {
- dc->res_pool->stream_enc[j]->funcs->dp_blank(dc->links[i],
- dc->res_pool->stream_enc[j]);
- break;
- }
- }
- }
- dp_receiver_power_ctrl(dc->links[i], false);
- }
- }
- }
+ if (dc->config.power_down_display_on_boot)
+ dc_link_blank_all_dp_displays(dc);
/* If taking control over from VBIOS, we may want to optimize our first
* mode set, so we need to skip powering down pipes until we know which
diff --git a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_hwseq.h b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_hwseq.h
index e9a0005..73e7b69 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_hwseq.h
+++ b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_hwseq.h
@@ -27,7 +27,7 @@
#define __DC_HWSS_DCN30_H__
#include "hw_sequencer_private.h"
-
+#include "dcn20/dcn20_hwseq.h"
struct dc;
void dcn30_init_hw(struct dc *dc);
@@ -47,6 +47,9 @@ void dcn30_disable_writeback(
struct dc *dc,
unsigned int dwb_pipe_inst);
+void dcn30_prepare_bandwidth(struct dc *dc,
+ struct dc_state *context);
+
bool dcn30_mmhubbub_warmup(
struct dc *dc,
unsigned int num_dwb,
diff --git a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_init.c b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_init.c
index 93f32a3..bb34731 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_init.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_init.c
@@ -29,6 +29,8 @@
#include "dcn21/dcn21_hwseq.h"
#include "dcn30_hwseq.h"
+#include "dcn30_init.h"
+
static const struct hw_sequencer_funcs dcn30_funcs = {
.program_gamut_remap = dcn10_program_gamut_remap,
.init_hw = dcn30_init_hw,
@@ -53,6 +55,7 @@ static const struct hw_sequencer_funcs dcn30_funcs = {
.enable_audio_stream = dce110_enable_audio_stream,
.disable_audio_stream = dce110_disable_audio_stream,
.disable_plane = dcn20_disable_plane,
+ .disable_pixel_data = dcn20_disable_pixel_data,
.pipe_control_lock = dcn20_pipe_control_lock,
.interdependent_update_lock = dcn10_lock_all_pipes,
.cursor_lock = dcn10_cursor_lock,
diff --git a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_mmhubbub.c b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_mmhubbub.c
index 1c4b171..7a93eff 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_mmhubbub.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_mmhubbub.c
@@ -100,7 +100,7 @@ static void mmhubbub3_warmup_mcif(struct mcif_wb *mcif_wb,
REG_UPDATE(MMHUBBUB_WARMUP_CONTROL_STATUS, MMHUBBUB_WARMUP_EN, false);
}
-void mmhubbub3_config_mcif_buf(struct mcif_wb *mcif_wb,
+static void mmhubbub3_config_mcif_buf(struct mcif_wb *mcif_wb,
struct mcif_buf_params *params,
unsigned int dest_height)
{
diff --git a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_mpc.c b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_mpc.c
index 9514973..0ce0d61 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_mpc.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_mpc.c
@@ -1362,7 +1362,7 @@ uint32_t mpcc3_acquire_rmu(struct mpc *mpc, int mpcc_id, int rmu_idx)
return -1;
}
-int mpcc3_release_rmu(struct mpc *mpc, int mpcc_id)
+static int mpcc3_release_rmu(struct mpc *mpc, int mpcc_id)
{
struct dcn30_mpc *mpc30 = TO_DCN30_MPC(mpc);
int rmu_idx;
diff --git a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_optc.c b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_optc.c
index 5d9e641..f5e8916 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_optc.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_optc.c
@@ -332,6 +332,7 @@ static struct timing_generator_funcs dcn30_tg_funcs = {
.get_crc = optc1_get_crc,
.configure_crc = optc2_configure_crc,
.set_dsc_config = optc3_set_dsc_config,
+ .get_dsc_status = optc2_get_dsc_status,
.set_dwb_source = NULL,
.set_odm_bypass = optc3_set_odm_bypass,
.set_odm_combine = optc3_set_odm_combine,
diff --git a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_resource.c b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_resource.c
index 79a66e0..602ec9a 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_resource.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_resource.c
@@ -816,7 +816,7 @@ static const struct dc_plane_cap plane_cap = {
.argb8888 = true,
.nv12 = true,
.fp16 = true,
- .p010 = false,
+ .p010 = true,
.ayuv = false,
},
@@ -840,7 +840,7 @@ static const struct dc_debug_options debug_defaults_drv = {
.timing_trace = false,
.clock_trace = true,
.disable_pplib_clock_request = true,
- .pipe_split_policy = MPC_SPLIT_AVOID_MULT_DISP,
+ .pipe_split_policy = MPC_SPLIT_DYNAMIC,
.force_single_disp_pipe_split = false,
.disable_dcc = DCC_ENABLE,
.vsr_support = true,
@@ -875,7 +875,7 @@ static const struct dc_debug_options debug_defaults_diags = {
.use_max_lb = true
};
-void dcn30_dpp_destroy(struct dpp **dpp)
+static void dcn30_dpp_destroy(struct dpp **dpp)
{
kfree(TO_DCN20_DPP(*dpp));
*dpp = NULL;
@@ -992,7 +992,7 @@ static struct mpc *dcn30_mpc_create(
return &mpc30->base;
}
-struct hubbub *dcn30_hubbub_create(struct dc_context *ctx)
+static struct hubbub *dcn30_hubbub_create(struct dc_context *ctx)
{
int i;
@@ -1143,9 +1143,8 @@ static struct afmt *dcn30_afmt_create(
return &afmt3->base;
}
-struct stream_encoder *dcn30_stream_encoder_create(
- enum engine_id eng_id,
- struct dc_context *ctx)
+static struct stream_encoder *dcn30_stream_encoder_create(enum engine_id eng_id,
+ struct dc_context *ctx)
{
struct dcn10_stream_encoder *enc1;
struct vpg *vpg;
@@ -1179,8 +1178,7 @@ struct stream_encoder *dcn30_stream_encoder_create(
return &enc1->base;
}
-struct dce_hwseq *dcn30_hwseq_create(
- struct dc_context *ctx)
+static struct dce_hwseq *dcn30_hwseq_create(struct dc_context *ctx)
{
struct dce_hwseq *hws = kzalloc(sizeof(struct dce_hwseq), GFP_KERNEL);
@@ -2639,6 +2637,8 @@ static bool dcn30_resource_construct(
dc->caps.color.mpc.ogam_rom_caps.hlg = 0;
dc->caps.color.mpc.ocsc = 1;
+ dc->caps.hdmi_frl_pcon_support = true;
+
/* read VBIOS LTTPR caps */
{
if (ctx->dc_bios->funcs->get_lttpr_caps) {
diff --git a/drivers/gpu/drm/amd/display/dc/dcn301/dcn301_init.c b/drivers/gpu/drm/amd/display/dc/dcn301/dcn301_init.c
index e85b695..3d42a1a 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn301/dcn301_init.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn301/dcn301_init.c
@@ -30,6 +30,8 @@
#include "dcn30/dcn30_hwseq.h"
#include "dcn301_hwseq.h"
+#include "dcn301_init.h"
+
static const struct hw_sequencer_funcs dcn301_funcs = {
.program_gamut_remap = dcn10_program_gamut_remap,
.init_hw = dcn10_init_hw,
diff --git a/drivers/gpu/drm/amd/display/dc/dcn301/dcn301_panel_cntl.c b/drivers/gpu/drm/amd/display/dc/dcn301/dcn301_panel_cntl.c
index 736bda3..ad0df1a 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn301/dcn301_panel_cntl.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn301/dcn301_panel_cntl.c
@@ -93,7 +93,7 @@ static unsigned int dcn301_get_16_bit_backlight_from_pwm(struct panel_cntl *pane
return (uint32_t)(current_backlight);
}
-uint32_t dcn301_panel_cntl_hw_init(struct panel_cntl *panel_cntl)
+static uint32_t dcn301_panel_cntl_hw_init(struct panel_cntl *panel_cntl)
{
struct dcn301_panel_cntl *dcn301_panel_cntl = TO_DCN301_PANEL_CNTL(panel_cntl);
uint32_t value;
@@ -147,7 +147,7 @@ uint32_t dcn301_panel_cntl_hw_init(struct panel_cntl *panel_cntl)
return current_backlight;
}
-void dcn301_panel_cntl_destroy(struct panel_cntl **panel_cntl)
+static void dcn301_panel_cntl_destroy(struct panel_cntl **panel_cntl)
{
struct dcn301_panel_cntl *dcn301_panel_cntl = TO_DCN301_PANEL_CNTL(*panel_cntl);
@@ -155,7 +155,7 @@ void dcn301_panel_cntl_destroy(struct panel_cntl **panel_cntl)
*panel_cntl = NULL;
}
-bool dcn301_is_panel_backlight_on(struct panel_cntl *panel_cntl)
+static bool dcn301_is_panel_backlight_on(struct panel_cntl *panel_cntl)
{
struct dcn301_panel_cntl *dcn301_panel_cntl = TO_DCN301_PANEL_CNTL(panel_cntl);
uint32_t value;
@@ -165,7 +165,7 @@ bool dcn301_is_panel_backlight_on(struct panel_cntl *panel_cntl)
return value;
}
-bool dcn301_is_panel_powered_on(struct panel_cntl *panel_cntl)
+static bool dcn301_is_panel_powered_on(struct panel_cntl *panel_cntl)
{
struct dcn301_panel_cntl *dcn301_panel_cntl = TO_DCN301_PANEL_CNTL(panel_cntl);
uint32_t pwr_seq_state, dig_on, dig_on_ovrd;
@@ -177,7 +177,7 @@ bool dcn301_is_panel_powered_on(struct panel_cntl *panel_cntl)
return (pwr_seq_state == 1) || (dig_on == 1 && dig_on_ovrd == 1);
}
-void dcn301_store_backlight_level(struct panel_cntl *panel_cntl)
+static void dcn301_store_backlight_level(struct panel_cntl *panel_cntl)
{
struct dcn301_panel_cntl *dcn301_panel_cntl = TO_DCN301_PANEL_CNTL(panel_cntl);
diff --git a/drivers/gpu/drm/amd/display/dc/dcn301/dcn301_resource.c b/drivers/gpu/drm/amd/display/dc/dcn301/dcn301_resource.c
index fbaa03f..c1c6e60 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn301/dcn301_resource.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn301/dcn301_resource.c
@@ -656,7 +656,7 @@ static const struct dc_plane_cap plane_cap = {
.argb8888 = true,
.nv12 = true,
.fp16 = true,
- .p010 = false,
+ .p010 = true,
.ayuv = false,
},
@@ -686,7 +686,7 @@ static const struct dc_debug_options debug_defaults_drv = {
.disable_clock_gate = true,
.disable_pplib_clock_request = true,
.disable_pplib_wm_range = true,
- .pipe_split_policy = MPC_SPLIT_AVOID_MULT_DISP,
+ .pipe_split_policy = MPC_SPLIT_DYNAMIC,
.force_single_disp_pipe_split = false,
.disable_dcc = DCC_ENABLE,
.vsr_support = true,
@@ -717,15 +717,13 @@ static const struct dc_debug_options debug_defaults_diags = {
.use_max_lb = false,
};
-void dcn301_dpp_destroy(struct dpp **dpp)
+static void dcn301_dpp_destroy(struct dpp **dpp)
{
kfree(TO_DCN20_DPP(*dpp));
*dpp = NULL;
}
-struct dpp *dcn301_dpp_create(
- struct dc_context *ctx,
- uint32_t inst)
+static struct dpp *dcn301_dpp_create(struct dc_context *ctx, uint32_t inst)
{
struct dcn3_dpp *dpp =
kzalloc(sizeof(struct dcn3_dpp), GFP_KERNEL);
@@ -741,8 +739,8 @@ struct dpp *dcn301_dpp_create(
kfree(dpp);
return NULL;
}
-struct output_pixel_processor *dcn301_opp_create(
- struct dc_context *ctx, uint32_t inst)
+static struct output_pixel_processor *dcn301_opp_create(struct dc_context *ctx,
+ uint32_t inst)
{
struct dcn20_opp *opp =
kzalloc(sizeof(struct dcn20_opp), GFP_KERNEL);
@@ -757,9 +755,7 @@ struct output_pixel_processor *dcn301_opp_create(
return &opp->base;
}
-struct dce_aux *dcn301_aux_engine_create(
- struct dc_context *ctx,
- uint32_t inst)
+static struct dce_aux *dcn301_aux_engine_create(struct dc_context *ctx, uint32_t inst)
{
struct aux_engine_dce110 *aux_engine =
kzalloc(sizeof(struct aux_engine_dce110), GFP_KERNEL);
@@ -793,9 +789,7 @@ static const struct dce_i2c_mask i2c_masks = {
I2C_COMMON_MASK_SH_LIST_DCN2(_MASK)
};
-struct dce_i2c_hw *dcn301_i2c_hw_create(
- struct dc_context *ctx,
- uint32_t inst)
+static struct dce_i2c_hw *dcn301_i2c_hw_create(struct dc_context *ctx, uint32_t inst)
{
struct dce_i2c_hw *dce_i2c_hw =
kzalloc(sizeof(struct dce_i2c_hw), GFP_KERNEL);
@@ -829,7 +823,7 @@ static struct mpc *dcn301_mpc_create(
return &mpc30->base;
}
-struct hubbub *dcn301_hubbub_create(struct dc_context *ctx)
+static struct hubbub *dcn301_hubbub_create(struct dc_context *ctx)
{
int i;
@@ -860,9 +854,8 @@ struct hubbub *dcn301_hubbub_create(struct dc_context *ctx)
return &hubbub3->base;
}
-struct timing_generator *dcn301_timing_generator_create(
- struct dc_context *ctx,
- uint32_t instance)
+static struct timing_generator *dcn301_timing_generator_create(
+ struct dc_context *ctx, uint32_t instance)
{
struct optc *tgn10 =
kzalloc(sizeof(struct optc), GFP_KERNEL);
@@ -894,7 +887,7 @@ static const struct encoder_feature_support link_enc_feature = {
.flags.bits.IS_TPS4_CAPABLE = true
};
-struct link_encoder *dcn301_link_encoder_create(
+static struct link_encoder *dcn301_link_encoder_create(
const struct encoder_init_data *enc_init_data)
{
struct dcn20_link_encoder *enc20 =
@@ -915,7 +908,7 @@ struct link_encoder *dcn301_link_encoder_create(
return &enc20->enc10.base;
}
-struct panel_cntl *dcn301_panel_cntl_create(const struct panel_cntl_init_data *init_data)
+static struct panel_cntl *dcn301_panel_cntl_create(const struct panel_cntl_init_data *init_data)
{
struct dcn301_panel_cntl *panel_cntl =
kzalloc(sizeof(struct dcn301_panel_cntl), GFP_KERNEL);
@@ -997,9 +990,8 @@ static struct afmt *dcn301_afmt_create(
return &afmt3->base;
}
-struct stream_encoder *dcn301_stream_encoder_create(
- enum engine_id eng_id,
- struct dc_context *ctx)
+static struct stream_encoder *dcn301_stream_encoder_create(enum engine_id eng_id,
+ struct dc_context *ctx)
{
struct dcn10_stream_encoder *enc1;
struct vpg *vpg;
@@ -1033,8 +1025,7 @@ struct stream_encoder *dcn301_stream_encoder_create(
return &enc1->base;
}
-struct dce_hwseq *dcn301_hwseq_create(
- struct dc_context *ctx)
+static struct dce_hwseq *dcn301_hwseq_create(struct dc_context *ctx)
{
struct dce_hwseq *hws = kzalloc(sizeof(struct dce_hwseq), GFP_KERNEL);
@@ -1182,9 +1173,7 @@ static void dcn301_destruct(struct dcn301_resource_pool *pool)
dcn_dccg_destroy(&pool->base.dccg);
}
-struct hubp *dcn301_hubp_create(
- struct dc_context *ctx,
- uint32_t inst)
+static struct hubp *dcn301_hubp_create(struct dc_context *ctx, uint32_t inst)
{
struct dcn20_hubp *hubp2 =
kzalloc(sizeof(struct dcn20_hubp), GFP_KERNEL);
@@ -1201,7 +1190,7 @@ struct hubp *dcn301_hubp_create(
return NULL;
}
-bool dcn301_dwbc_create(struct dc_context *ctx, struct resource_pool *pool)
+static bool dcn301_dwbc_create(struct dc_context *ctx, struct resource_pool *pool)
{
int i;
uint32_t pipe_count = pool->res_cap->num_dwb;
@@ -1226,7 +1215,7 @@ bool dcn301_dwbc_create(struct dc_context *ctx, struct resource_pool *pool)
return true;
}
-bool dcn301_mmhubbub_create(struct dc_context *ctx, struct resource_pool *pool)
+static bool dcn301_mmhubbub_create(struct dc_context *ctx, struct resource_pool *pool)
{
int i;
uint32_t pipe_count = pool->res_cap->num_dwb;
@@ -1449,9 +1438,7 @@ static bool dcn301_resource_construct(
dc->caps.post_blend_color_processing = true;
dc->caps.force_dp_tps4_for_cp2520 = true;
dc->caps.extended_aux_timeout_support = true;
-#ifdef CONFIG_DRM_AMD_DC_DMUB
dc->caps.dmcub_support = true;
-#endif
/* Color pipeline capabilities */
dc->caps.color.dpp.dcn_arch = 1;
@@ -1487,6 +1474,23 @@ static bool dcn301_resource_construct(
dc->caps.color.mpc.ogam_rom_caps.hlg = 0;
dc->caps.color.mpc.ocsc = 1;
+ /* read VBIOS LTTPR caps */
+ if (ctx->dc_bios->funcs->get_lttpr_caps) {
+ enum bp_result bp_query_result;
+ uint8_t is_vbios_lttpr_enable = 0;
+
+ bp_query_result = ctx->dc_bios->funcs->get_lttpr_caps(ctx->dc_bios, &is_vbios_lttpr_enable);
+ dc->caps.vbios_lttpr_enable = (bp_query_result == BP_RESULT_OK) && !!is_vbios_lttpr_enable;
+ }
+
+ if (ctx->dc_bios->funcs->get_lttpr_interop) {
+ enum bp_result bp_query_result;
+ uint8_t is_vbios_interop_enabled = 0;
+
+ bp_query_result = ctx->dc_bios->funcs->get_lttpr_interop(ctx->dc_bios, &is_vbios_interop_enabled);
+ dc->caps.vbios_lttpr_aware = (bp_query_result == BP_RESULT_OK) && !!is_vbios_interop_enabled;
+ }
+
if (dc->ctx->dce_environment == DCE_ENV_PRODUCTION_DRV)
dc->debug = debug_defaults_drv;
else if (dc->ctx->dce_environment == DCE_ENV_FPGA_MAXIMUS) {
diff --git a/drivers/gpu/drm/amd/display/dc/dcn302/dcn302_init.c b/drivers/gpu/drm/amd/display/dc/dcn302/dcn302_init.c
index d88b901..eb375f3 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn302/dcn302_init.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn302/dcn302_init.c
@@ -29,6 +29,8 @@
#include "dc.h"
+#include "dcn302_init.h"
+
void dcn302_hw_sequencer_construct(struct dc *dc)
{
dcn30_hw_sequencer_construct(dc);
diff --git a/drivers/gpu/drm/amd/display/dc/dcn302/dcn302_resource.c b/drivers/gpu/drm/amd/display/dc/dcn302/dcn302_resource.c
index fcf96cf..2e9cbfa 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn302/dcn302_resource.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn302/dcn302_resource.c
@@ -211,7 +211,7 @@ static const struct dc_debug_options debug_defaults_drv = {
.timing_trace = false,
.clock_trace = true,
.disable_pplib_clock_request = true,
- .pipe_split_policy = MPC_SPLIT_AVOID_MULT_DISP,
+ .pipe_split_policy = MPC_SPLIT_DYNAMIC,
.force_single_disp_pipe_split = false,
.disable_dcc = DCC_ENABLE,
.vsr_support = true,
@@ -276,7 +276,7 @@ static const struct dc_plane_cap plane_cap = {
.argb8888 = true,
.nv12 = true,
.fp16 = true,
- .p010 = false,
+ .p010 = true,
.ayuv = false,
},
.max_upscale_factor = {
@@ -1557,6 +1557,24 @@ static bool dcn302_resource_construct(
dc->caps.color.mpc.ogam_rom_caps.hlg = 0;
dc->caps.color.mpc.ocsc = 1;
+ /* read VBIOS LTTPR caps */
+ if (ctx->dc_bios->funcs->get_lttpr_caps) {
+ enum bp_result bp_query_result;
+ uint8_t is_vbios_lttpr_enable = 0;
+
+ bp_query_result = ctx->dc_bios->funcs->get_lttpr_caps(ctx->dc_bios, &is_vbios_lttpr_enable);
+ dc->caps.vbios_lttpr_enable = (bp_query_result == BP_RESULT_OK) && !!is_vbios_lttpr_enable;
+ }
+
+ if (ctx->dc_bios->funcs->get_lttpr_interop) {
+ enum bp_result bp_query_result;
+ uint8_t is_vbios_interop_enabled = 0;
+
+ bp_query_result = ctx->dc_bios->funcs->get_lttpr_interop(ctx->dc_bios,
+ &is_vbios_interop_enabled);
+ dc->caps.vbios_lttpr_aware = (bp_query_result == BP_RESULT_OK) && !!is_vbios_interop_enabled;
+ }
+
if (dc->ctx->dce_environment == DCE_ENV_PRODUCTION_DRV)
dc->debug = debug_defaults_drv;
else
diff --git a/drivers/gpu/drm/amd/display/dc/dcn303/dcn303_dccg.h b/drivers/gpu/drm/amd/display/dc/dcn303/dcn303_dccg.h
index a79c54b..294bd75 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn303/dcn303_dccg.h
+++ b/drivers/gpu/drm/amd/display/dc/dcn303/dcn303_dccg.h
@@ -15,7 +15,11 @@
SR(DPPCLK_DTO_CTRL),\
DCCG_SRII(DTO_PARAM, DPPCLK, 0),\
DCCG_SRII(DTO_PARAM, DPPCLK, 1),\
- SR(REFCLK_CNTL)
+ SR(REFCLK_CNTL),\
+ SR(DISPCLK_FREQ_CHANGE_CNTL),\
+ DCCG_SRII(PIXEL_RATE_CNTL, OTG, 0),\
+ DCCG_SRII(PIXEL_RATE_CNTL, OTG, 1)
+
#define DCCG_MASK_SH_LIST_DCN3_03(mask_sh) \
DCCG_SFI(DPPCLK_DTO_CTRL, DTO_ENABLE, DPPCLK, 0, mask_sh),\
@@ -25,6 +29,18 @@
DCCG_SF(DPPCLK0_DTO_PARAM, DPPCLK0_DTO_PHASE, mask_sh),\
DCCG_SF(DPPCLK0_DTO_PARAM, DPPCLK0_DTO_MODULO, mask_sh),\
DCCG_SF(REFCLK_CNTL, REFCLK_CLOCK_EN, mask_sh),\
- DCCG_SF(REFCLK_CNTL, REFCLK_SRC_SEL, mask_sh)
+ DCCG_SF(REFCLK_CNTL, REFCLK_SRC_SEL, mask_sh),\
+ DCCG_SF(DISPCLK_FREQ_CHANGE_CNTL, DISPCLK_STEP_DELAY, mask_sh),\
+ DCCG_SF(DISPCLK_FREQ_CHANGE_CNTL, DISPCLK_STEP_SIZE, mask_sh),\
+ DCCG_SF(DISPCLK_FREQ_CHANGE_CNTL, DISPCLK_FREQ_RAMP_DONE, mask_sh),\
+ DCCG_SF(DISPCLK_FREQ_CHANGE_CNTL, DISPCLK_MAX_ERRDET_CYCLES, mask_sh),\
+ DCCG_SF(DISPCLK_FREQ_CHANGE_CNTL, DCCG_FIFO_ERRDET_RESET, mask_sh),\
+ DCCG_SF(DISPCLK_FREQ_CHANGE_CNTL, DCCG_FIFO_ERRDET_STATE, mask_sh),\
+ DCCG_SF(DISPCLK_FREQ_CHANGE_CNTL, DCCG_FIFO_ERRDET_OVR_EN, mask_sh),\
+ DCCG_SF(DISPCLK_FREQ_CHANGE_CNTL, DISPCLK_CHG_FWD_CORR_DISABLE, mask_sh),\
+ DCCG_SFII(OTG, PIXEL_RATE_CNTL, OTG, ADD_PIXEL, 0, mask_sh),\
+ DCCG_SFII(OTG, PIXEL_RATE_CNTL, OTG, ADD_PIXEL, 1, mask_sh),\
+ DCCG_SFII(OTG, PIXEL_RATE_CNTL, OTG, DROP_PIXEL, 0, mask_sh),\
+ DCCG_SFII(OTG, PIXEL_RATE_CNTL, OTG, DROP_PIXEL, 1, mask_sh)
#endif //__DCN303_DCCG_H__
diff --git a/drivers/gpu/drm/amd/display/dc/dcn303/dcn303_init.c b/drivers/gpu/drm/amd/display/dc/dcn303/dcn303_init.c
index aa5dbba..f499f8a 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn303/dcn303_init.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn303/dcn303_init.c
@@ -9,6 +9,8 @@
#include "dcn30/dcn30_init.h"
#include "dc.h"
+#include "dcn303_init.h"
+
void dcn303_hw_sequencer_construct(struct dc *dc)
{
dcn30_hw_sequencer_construct(dc);
diff --git a/drivers/gpu/drm/amd/display/dc/dcn303/dcn303_resource.c b/drivers/gpu/drm/amd/display/dc/dcn303/dcn303_resource.c
index 4a9b640..2de687f6 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn303/dcn303_resource.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn303/dcn303_resource.c
@@ -193,7 +193,7 @@ static const struct dc_debug_options debug_defaults_drv = {
.timing_trace = false,
.clock_trace = true,
.disable_pplib_clock_request = true,
- .pipe_split_policy = MPC_SPLIT_AVOID_MULT_DISP,
+ .pipe_split_policy = MPC_SPLIT_DYNAMIC,
.force_single_disp_pipe_split = false,
.disable_dcc = DCC_ENABLE,
.vsr_support = true,
@@ -254,7 +254,7 @@ static const struct dc_plane_cap plane_cap = {
.argb8888 = true,
.nv12 = true,
.fp16 = true,
- .p010 = false,
+ .p010 = true,
.ayuv = false,
},
.max_upscale_factor = {
@@ -1500,6 +1500,23 @@ static bool dcn303_resource_construct(
dc->caps.color.mpc.ogam_rom_caps.hlg = 0;
dc->caps.color.mpc.ocsc = 1;
+ /* read VBIOS LTTPR caps */
+ if (ctx->dc_bios->funcs->get_lttpr_caps) {
+ enum bp_result bp_query_result;
+ uint8_t is_vbios_lttpr_enable = 0;
+
+ bp_query_result = ctx->dc_bios->funcs->get_lttpr_caps(ctx->dc_bios, &is_vbios_lttpr_enable);
+ dc->caps.vbios_lttpr_enable = (bp_query_result == BP_RESULT_OK) && !!is_vbios_lttpr_enable;
+ }
+
+ if (ctx->dc_bios->funcs->get_lttpr_interop) {
+ enum bp_result bp_query_result;
+ uint8_t is_vbios_interop_enabled = 0;
+
+ bp_query_result = ctx->dc_bios->funcs->get_lttpr_interop(ctx->dc_bios, &is_vbios_interop_enabled);
+ dc->caps.vbios_lttpr_aware = (bp_query_result == BP_RESULT_OK) && !!is_vbios_interop_enabled;
+ }
+
if (dc->ctx->dce_environment == DCE_ENV_PRODUCTION_DRV)
dc->debug = debug_defaults_drv;
else
diff --git a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_dccg.c b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_dccg.c
index 815481a..ea4f8e0 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_dccg.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_dccg.c
@@ -462,7 +462,7 @@ void dccg31_set_physymclk(
}
/* Controls the generation of pixel valid for OTG in (OTG -> HPO case) */
-void dccg31_set_dtbclk_dto(
+static void dccg31_set_dtbclk_dto(
struct dccg *dccg,
int dtbclk_inst,
int req_dtbclk_khz,
diff --git a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_dio_link_encoder.c b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_dio_link_encoder.c
index ee6f13b..71c359f 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_dio_link_encoder.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_dio_link_encoder.c
@@ -67,6 +67,39 @@
#define MIN(X, Y) ((X) < (Y) ? (X) : (Y))
#endif
+static uint8_t phy_id_from_transmitter(enum transmitter t)
+{
+ uint8_t phy_id;
+
+ switch (t) {
+ case TRANSMITTER_UNIPHY_A:
+ phy_id = 0;
+ break;
+ case TRANSMITTER_UNIPHY_B:
+ phy_id = 1;
+ break;
+ case TRANSMITTER_UNIPHY_C:
+ phy_id = 2;
+ break;
+ case TRANSMITTER_UNIPHY_D:
+ phy_id = 3;
+ break;
+ case TRANSMITTER_UNIPHY_E:
+ phy_id = 4;
+ break;
+ case TRANSMITTER_UNIPHY_F:
+ phy_id = 5;
+ break;
+ case TRANSMITTER_UNIPHY_G:
+ phy_id = 6;
+ break;
+ default:
+ phy_id = 0;
+ break;
+ }
+ return phy_id;
+}
+
void dcn31_link_encoder_set_dio_phy_mux(
struct link_encoder *enc,
enum encoder_type_select sel,
@@ -141,7 +174,7 @@ void dcn31_link_encoder_set_dio_phy_mux(
}
}
-void enc31_hw_init(struct link_encoder *enc)
+static void enc31_hw_init(struct link_encoder *enc)
{
struct dcn10_link_encoder *enc10 = TO_DCN10_LINK_ENC(enc);
@@ -536,57 +569,45 @@ void dcn31_link_encoder_disable_output(
bool dcn31_link_encoder_is_in_alt_mode(struct link_encoder *enc)
{
struct dcn10_link_encoder *enc10 = TO_DCN10_LINK_ENC(enc);
- uint32_t dp_alt_mode_disable;
+ struct dc_dmub_srv *dc_dmub_srv = enc->ctx->dmub_srv;
+ union dmub_rb_cmd cmd;
bool is_usb_c_alt_mode = false;
- if (enc->features.flags.bits.DP_IS_USB_C) {
- if (enc->ctx->asic_id.hw_internal_rev != YELLOW_CARP_B0) {
- // [Note] no need to check hw_internal_rev once phy mux selection is ready
- REG_GET(RDPCSTX_PHY_CNTL6, RDPCS_PHY_DPALT_DISABLE, &dp_alt_mode_disable);
- } else {
- /*
- * B0 phys use a new set of registers to check whether alt mode is disabled.
- * if value == 1 alt mode is disabled, otherwise it is enabled.
- */
- if ((enc10->base.transmitter == TRANSMITTER_UNIPHY_A)
- || (enc10->base.transmitter == TRANSMITTER_UNIPHY_B)
- || (enc10->base.transmitter == TRANSMITTER_UNIPHY_E)) {
- REG_GET(RDPCSTX_PHY_CNTL6, RDPCS_PHY_DPALT_DISABLE, &dp_alt_mode_disable);
- } else {
- // [Note] need to change TRANSMITTER_UNIPHY_C/D to F/G once phy mux selection is ready
- REG_GET(RDPCSPIPE_PHY_CNTL6, RDPCS_PHY_DPALT_DISABLE, &dp_alt_mode_disable);
- }
- }
+ if (enc->features.flags.bits.DP_IS_USB_C && dc_dmub_srv) {
+ memset(&cmd, 0, sizeof(cmd));
+ cmd.query_dp_alt.header.type = DMUB_CMD__VBIOS;
+ cmd.query_dp_alt.header.sub_type = DMUB_CMD__VBIOS_TRANSMITTER_QUERY_DP_ALT;
+ cmd.query_dp_alt.header.payload_bytes = sizeof(cmd.panel_cntl.data);
+ cmd.query_dp_alt.data.phy_id = phy_id_from_transmitter(enc10->base.transmitter);
- is_usb_c_alt_mode = (dp_alt_mode_disable == 0);
+ if (!dc_dmub_srv_cmd_with_reply_data(dc_dmub_srv, &cmd))
+ return false;
+
+ is_usb_c_alt_mode = (cmd.query_dp_alt.data.is_dp_alt_disable == 0);
}
return is_usb_c_alt_mode;
}
-void dcn31_link_encoder_get_max_link_cap(struct link_encoder *enc,
- struct dc_link_settings *link_settings)
+void dcn31_link_encoder_get_max_link_cap(struct link_encoder *enc, struct dc_link_settings *link_settings)
{
struct dcn10_link_encoder *enc10 = TO_DCN10_LINK_ENC(enc);
- uint32_t is_in_usb_c_dp4_mode = 0;
+ struct dc_dmub_srv *dc_dmub_srv = enc->ctx->dmub_srv;
+ union dmub_rb_cmd cmd;
dcn10_link_encoder_get_max_link_cap(enc, link_settings);
- /* in usb c dp2 mode, max lane count is 2 */
- if (enc->funcs->is_in_alt_mode && enc->funcs->is_in_alt_mode(enc)) {
- if (enc->ctx->asic_id.hw_internal_rev != YELLOW_CARP_B0) {
- // [Note] no need to check hw_internal_rev once phy mux selection is ready
- REG_GET(RDPCSTX_PHY_CNTL6, RDPCS_PHY_DPALT_DP4, &is_in_usb_c_dp4_mode);
- } else {
- if ((enc10->base.transmitter == TRANSMITTER_UNIPHY_A)
- || (enc10->base.transmitter == TRANSMITTER_UNIPHY_B)
- || (enc10->base.transmitter == TRANSMITTER_UNIPHY_E)) {
- REG_GET(RDPCSTX_PHY_CNTL6, RDPCS_PHY_DPALT_DP4, &is_in_usb_c_dp4_mode);
- } else {
- REG_GET(RDPCSPIPE_PHY_CNTL6, RDPCS_PHY_DPALT_DP4, &is_in_usb_c_dp4_mode);
- }
- }
- if (!is_in_usb_c_dp4_mode)
+ if (enc->features.flags.bits.DP_IS_USB_C && dc_dmub_srv) {
+ memset(&cmd, 0, sizeof(cmd));
+ cmd.query_dp_alt.header.type = DMUB_CMD__VBIOS;
+ cmd.query_dp_alt.header.sub_type = DMUB_CMD__VBIOS_TRANSMITTER_QUERY_DP_ALT;
+ cmd.query_dp_alt.header.payload_bytes = sizeof(cmd.panel_cntl.data);
+ cmd.query_dp_alt.data.phy_id = phy_id_from_transmitter(enc10->base.transmitter);
+
+ if (!dc_dmub_srv_cmd_with_reply_data(dc_dmub_srv, &cmd))
+ return;
+
+ if (cmd.query_dp_alt.data.is_usb && cmd.query_dp_alt.data.is_dp4 == 0)
link_settings->lane_count = MIN(LANE_COUNT_TWO, link_settings->lane_count);
}
}
diff --git a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_hpo_dp_link_encoder.c b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_hpo_dp_link_encoder.c
index 6c08e21..80dfaa4 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_hpo_dp_link_encoder.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_hpo_dp_link_encoder.c
@@ -499,7 +499,8 @@ static enum bp_result link_transmitter_control(
void dcn31_hpo_dp_link_enc_enable_dp_output(
struct hpo_dp_link_encoder *enc,
const struct dc_link_settings *link_settings,
- enum transmitter transmitter)
+ enum transmitter transmitter,
+ enum hpd_source_id hpd_source)
{
struct dcn31_hpo_dp_link_encoder *enc3 = DCN3_1_HPO_DP_LINK_ENC_FROM_HPO_LINK_ENC(enc);
struct bp_transmitter_control cntl = { 0 };
@@ -508,6 +509,9 @@ void dcn31_hpo_dp_link_enc_enable_dp_output(
/* Set the transmitter */
enc3->base.transmitter = transmitter;
+ /* Set the hpd source */
+ enc3->base.hpd_source = hpd_source;
+
/* Enable the PHY */
cntl.action = TRANSMITTER_CONTROL_ENABLE;
cntl.engine_id = ENGINE_ID_UNKNOWN;
diff --git a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_hpo_dp_link_encoder.h b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_hpo_dp_link_encoder.h
index 0706cca..e324e9b 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_hpo_dp_link_encoder.h
+++ b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_hpo_dp_link_encoder.h
@@ -184,7 +184,8 @@ void hpo_dp_link_encoder31_construct(struct dcn31_hpo_dp_link_encoder *enc31,
void dcn31_hpo_dp_link_enc_enable_dp_output(
struct hpo_dp_link_encoder *enc,
const struct dc_link_settings *link_settings,
- enum transmitter transmitter);
+ enum transmitter transmitter,
+ enum hpd_source_id hpd_source);
void dcn31_hpo_dp_link_enc_disable_output(
struct hpo_dp_link_encoder *enc,
diff --git a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_hpo_dp_stream_encoder.c b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_hpo_dp_stream_encoder.c
index 565f12d..5065904 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_hpo_dp_stream_encoder.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_hpo_dp_stream_encoder.c
@@ -358,8 +358,8 @@ static void dcn31_hpo_dp_stream_enc_set_stream_attribute(
h_width = hw_crtc_timing.h_border_left + hw_crtc_timing.h_addressable + hw_crtc_timing.h_border_right;
v_height = hw_crtc_timing.v_border_top + hw_crtc_timing.v_addressable + hw_crtc_timing.v_border_bottom;
- hsp = hw_crtc_timing.flags.HSYNC_POSITIVE_POLARITY ? 0x80 : 0;
- vsp = hw_crtc_timing.flags.VSYNC_POSITIVE_POLARITY ? 0x80 : 0;
+ hsp = hw_crtc_timing.flags.HSYNC_POSITIVE_POLARITY ? 0 : 0x80;
+ vsp = hw_crtc_timing.flags.VSYNC_POSITIVE_POLARITY ? 0 : 0x80;
v_freq = hw_crtc_timing.pix_clk_100hz * 100;
/* MSA Packet Mapping to 32-bit Link Symbols - DP2 spec, section 2.7.4.1
diff --git a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_hwseq.c b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_hwseq.c
index 4d4ac4ce..4206ce5 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_hwseq.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_hwseq.c
@@ -112,7 +112,7 @@ void dcn31_init_hw(struct dc *dc)
struct dc_bios *dcb = dc->ctx->dc_bios;
struct resource_pool *res_pool = dc->res_pool;
uint32_t backlight = MAX_BACKLIGHT_LEVEL;
- int i, j;
+ int i;
if (dc->clk_mgr && dc->clk_mgr->funcs->init_clocks)
dc->clk_mgr->funcs->init_clocks(dc->clk_mgr);
@@ -192,50 +192,13 @@ void dcn31_init_hw(struct dc *dc)
link->link_status.link_active = true;
}
- /* Power gate DSCs */
- for (i = 0; i < res_pool->res_cap->num_dsc; i++)
- if (hws->funcs.dsc_pg_control != NULL)
- hws->funcs.dsc_pg_control(hws, res_pool->dscs[i]->inst, false);
-
/* Enables outbox notifications for usb4 dpia */
if (dc->res_pool->usb4_dpia_count)
dmub_enable_outbox_notification(dc);
/* we want to turn off all dp displays before doing detection */
- if (dc->config.power_down_display_on_boot) {
- uint8_t dpcd_power_state = '\0';
- enum dc_status status = DC_ERROR_UNEXPECTED;
-
- for (i = 0; i < dc->link_count; i++) {
- if (dc->links[i]->connector_signal != SIGNAL_TYPE_DISPLAY_PORT)
- continue;
-
- /* if any of the displays are lit up turn them off */
- status = core_link_read_dpcd(dc->links[i], DP_SET_POWER,
- &dpcd_power_state, sizeof(dpcd_power_state));
- if (status == DC_OK && dpcd_power_state == DP_POWER_STATE_D0) {
- /* blank dp stream before power off receiver*/
- if (dc->links[i]->ep_type == DISPLAY_ENDPOINT_PHY &&
- dc->links[i]->link_enc->funcs->get_dig_frontend) {
- unsigned int fe;
-
- fe = dc->links[i]->link_enc->funcs->get_dig_frontend(
- dc->links[i]->link_enc);
- if (fe == ENGINE_ID_UNKNOWN)
- continue;
-
- for (j = 0; j < dc->res_pool->stream_enc_count; j++) {
- if (fe == dc->res_pool->stream_enc[j]->id) {
- dc->res_pool->stream_enc[j]->funcs->dp_blank(dc->links[i],
- dc->res_pool->stream_enc[j]);
- break;
- }
- }
- }
- dp_receiver_power_ctrl(dc->links[i], false);
- }
- }
- }
+ if (dc->config.power_down_display_on_boot)
+ dc_link_blank_all_dp_displays(dc);
/* If taking control over from VBIOS, we may want to optimize our first
* mode set, so we need to skip powering down pipes until we know which
diff --git a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_init.c b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_init.c
index 4f6e639..d7559e5 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_init.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_init.c
@@ -31,6 +31,8 @@
#include "dcn301/dcn301_hwseq.h"
#include "dcn31/dcn31_hwseq.h"
+#include "dcn31_init.h"
+
static const struct hw_sequencer_funcs dcn31_funcs = {
.program_gamut_remap = dcn10_program_gamut_remap,
.init_hw = dcn31_init_hw,
@@ -101,6 +103,7 @@ static const struct hw_sequencer_funcs dcn31_funcs = {
.z10_restore = dcn31_z10_restore,
.z10_save_init = dcn31_z10_save_init,
.set_disp_pattern_generator = dcn30_set_disp_pattern_generator,
+ .optimize_pwr_state = dcn21_optimize_pwr_state,
.exit_optimized_pwr_state = dcn21_exit_optimized_pwr_state,
.update_visual_confirm_color = dcn20_update_visual_confirm_color,
};
@@ -150,4 +153,9 @@ void dcn31_hw_sequencer_construct(struct dc *dc)
dc->hwss.init_hw = dcn20_fpga_init_hw;
dc->hwseq->funcs.init_pipes = NULL;
}
+ if (dc->debug.disable_z10) {
+ /*hw not support z10 or sw disable it*/
+ dc->hwss.z10_restore = NULL;
+ dc->hwss.z10_save_init = NULL;
+ }
}
diff --git a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_optc.c b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_optc.c
index a4b1d98..e8562fa 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_optc.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_optc.c
@@ -256,6 +256,7 @@ static struct timing_generator_funcs dcn31_tg_funcs = {
.get_crc = optc1_get_crc,
.configure_crc = optc2_configure_crc,
.set_dsc_config = optc3_set_dsc_config,
+ .get_dsc_status = optc2_get_dsc_status,
.set_dwb_source = NULL,
.set_odm_bypass = optc3_set_odm_bypass,
.set_odm_combine = optc31_set_odm_combine,
diff --git a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_panel_cntl.c b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_panel_cntl.c
index 3b37213..83ece02 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_panel_cntl.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_panel_cntl.c
@@ -65,7 +65,7 @@ static uint32_t dcn31_get_16_bit_backlight_from_pwm(struct panel_cntl *panel_cnt
return cmd.panel_cntl.data.current_backlight;
}
-uint32_t dcn31_panel_cntl_hw_init(struct panel_cntl *panel_cntl)
+static uint32_t dcn31_panel_cntl_hw_init(struct panel_cntl *panel_cntl)
{
struct dcn31_panel_cntl *dcn31_panel_cntl = TO_DCN31_PANEL_CNTL(panel_cntl);
struct dc_dmub_srv *dc_dmub_srv = panel_cntl->ctx->dmub_srv;
@@ -96,7 +96,7 @@ uint32_t dcn31_panel_cntl_hw_init(struct panel_cntl *panel_cntl)
return cmd.panel_cntl.data.current_backlight;
}
-void dcn31_panel_cntl_destroy(struct panel_cntl **panel_cntl)
+static void dcn31_panel_cntl_destroy(struct panel_cntl **panel_cntl)
{
struct dcn31_panel_cntl *dcn31_panel_cntl = TO_DCN31_PANEL_CNTL(*panel_cntl);
@@ -104,7 +104,7 @@ void dcn31_panel_cntl_destroy(struct panel_cntl **panel_cntl)
*panel_cntl = NULL;
}
-bool dcn31_is_panel_backlight_on(struct panel_cntl *panel_cntl)
+static bool dcn31_is_panel_backlight_on(struct panel_cntl *panel_cntl)
{
union dmub_rb_cmd cmd;
@@ -114,7 +114,7 @@ bool dcn31_is_panel_backlight_on(struct panel_cntl *panel_cntl)
return cmd.panel_cntl.data.is_backlight_on;
}
-bool dcn31_is_panel_powered_on(struct panel_cntl *panel_cntl)
+static bool dcn31_is_panel_powered_on(struct panel_cntl *panel_cntl)
{
union dmub_rb_cmd cmd;
@@ -124,7 +124,7 @@ bool dcn31_is_panel_powered_on(struct panel_cntl *panel_cntl)
return cmd.panel_cntl.data.is_powered_on;
}
-void dcn31_store_backlight_level(struct panel_cntl *panel_cntl)
+static void dcn31_store_backlight_level(struct panel_cntl *panel_cntl)
{
union dmub_rb_cmd cmd;
diff --git a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_resource.c b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_resource.c
index 1889629..4d9c64d 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_resource.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_resource.c
@@ -355,6 +355,14 @@ static const struct dce110_clk_src_regs clk_src_regs[] = {
clk_src_regs(3, D),
clk_src_regs(4, E)
};
+/*pll_id being rempped in dmub, in driver it is logical instance*/
+static const struct dce110_clk_src_regs clk_src_regs_b0[] = {
+ clk_src_regs(0, A),
+ clk_src_regs(1, B),
+ clk_src_regs(2, F),
+ clk_src_regs(3, G),
+ clk_src_regs(4, E)
+};
static const struct dce110_clk_src_shift cs_shift = {
CS_COMMON_MASK_SH_LIST_DCN2_0(__SHIFT)
@@ -485,7 +493,8 @@ static const struct dcn31_apg_mask apg_mask = {
SE_DCN3_REG_LIST(id)\
}
-static const struct dcn10_stream_enc_registers stream_enc_regs[] = {
+/* Some encoders won't be initialized here - but they're logical, not physical. */
+static const struct dcn10_stream_enc_registers stream_enc_regs[ENGINE_ID_COUNT] = {
stream_enc_regs(0),
stream_enc_regs(1),
stream_enc_regs(2),
@@ -968,7 +977,7 @@ static const struct dc_plane_cap plane_cap = {
.argb8888 = true,
.nv12 = true,
.fp16 = true,
- .p010 = false,
+ .p010 = true,
.ayuv = false,
},
@@ -994,7 +1003,7 @@ static const struct dc_debug_options debug_defaults_drv = {
.timing_trace = false,
.clock_trace = true,
.disable_pplib_clock_request = false,
- .pipe_split_policy = MPC_SPLIT_AVOID,
+ .pipe_split_policy = MPC_SPLIT_DYNAMIC,
.force_single_disp_pipe_split = false,
.disable_dcc = DCC_ENABLE,
.vsr_support = true,
@@ -1023,6 +1032,7 @@ static const struct dc_debug_options debug_defaults_drv = {
},
.optimize_edp_link_rate = true,
.enable_sw_cntl_psr = true,
+ .apply_vendor_specific_lttpr_wa = true,
};
static const struct dc_debug_options debug_defaults_diags = {
@@ -1270,7 +1280,7 @@ static struct link_encoder *dcn31_link_enc_create_minimal(
return &enc20->enc10.base;
}
-struct panel_cntl *dcn31_panel_cntl_create(const struct panel_cntl_init_data *init_data)
+static struct panel_cntl *dcn31_panel_cntl_create(const struct panel_cntl_init_data *init_data)
{
struct dcn31_panel_cntl *panel_cntl =
kzalloc(sizeof(struct dcn31_panel_cntl), GFP_KERNEL);
@@ -1774,6 +1784,7 @@ static int dcn31_populate_dml_pipes_from_context(
int i, pipe_cnt;
struct resource_context *res_ctx = &context->res_ctx;
struct pipe_ctx *pipe;
+ bool upscaled = false;
dcn20_populate_dml_pipes_from_context(dc, context, pipes, fast_validate);
@@ -1785,6 +1796,11 @@ static int dcn31_populate_dml_pipes_from_context(
pipe = &res_ctx->pipe_ctx[i];
timing = &pipe->stream->timing;
+ if (pipe->plane_state &&
+ (pipe->plane_state->src_rect.height < pipe->plane_state->dst_rect.height ||
+ pipe->plane_state->src_rect.width < pipe->plane_state->dst_rect.width))
+ upscaled = true;
+
/*
* Immediate flip can be set dynamically after enabling the plane.
* We need to require support for immediate flip or underflow can be
@@ -1829,6 +1845,11 @@ static int dcn31_populate_dml_pipes_from_context(
context->bw_ctx.dml.ip.det_buffer_size_kbytes = 192;
pipes[0].pipe.src.unbounded_req_mode = true;
}
+ } else if (context->stream_count >= dc->debug.crb_alloc_policy_min_disp_count
+ && dc->debug.crb_alloc_policy > DET_SIZE_DEFAULT) {
+ context->bw_ctx.dml.ip.det_buffer_size_kbytes = dc->debug.crb_alloc_policy * 64;
+ } else if (context->stream_count >= 3 && upscaled) {
+ context->bw_ctx.dml.ip.det_buffer_size_kbytes = 192;
}
return pipe_cnt;
@@ -2199,6 +2220,8 @@ static bool dcn31_resource_construct(
dc->caps.post_blend_color_processing = true;
dc->caps.force_dp_tps4_for_cp2520 = true;
dc->caps.dp_hpo = true;
+ dc->caps.hdmi_frl_pcon_support = true;
+ dc->caps.edp_dsc_support = true;
dc->caps.extended_aux_timeout_support = true;
dc->caps.dmcub_support = true;
dc->caps.is_apu = true;
@@ -2276,14 +2299,27 @@ static bool dcn31_resource_construct(
dcn30_clock_source_create(ctx, ctx->dc_bios,
CLOCK_SOURCE_COMBO_PHY_PLL1,
&clk_src_regs[1], false);
- pool->base.clock_sources[DCN31_CLK_SRC_PLL2] =
+ /*move phypllx_pixclk_resync to dmub next*/
+ if (dc->ctx->asic_id.hw_internal_rev == YELLOW_CARP_B0) {
+ pool->base.clock_sources[DCN31_CLK_SRC_PLL2] =
+ dcn30_clock_source_create(ctx, ctx->dc_bios,
+ CLOCK_SOURCE_COMBO_PHY_PLL2,
+ &clk_src_regs_b0[2], false);
+ pool->base.clock_sources[DCN31_CLK_SRC_PLL3] =
+ dcn30_clock_source_create(ctx, ctx->dc_bios,
+ CLOCK_SOURCE_COMBO_PHY_PLL3,
+ &clk_src_regs_b0[3], false);
+ } else {
+ pool->base.clock_sources[DCN31_CLK_SRC_PLL2] =
dcn30_clock_source_create(ctx, ctx->dc_bios,
CLOCK_SOURCE_COMBO_PHY_PLL2,
&clk_src_regs[2], false);
- pool->base.clock_sources[DCN31_CLK_SRC_PLL3] =
+ pool->base.clock_sources[DCN31_CLK_SRC_PLL3] =
dcn30_clock_source_create(ctx, ctx->dc_bios,
CLOCK_SOURCE_COMBO_PHY_PLL3,
&clk_src_regs[3], false);
+ }
+
pool->base.clock_sources[DCN31_CLK_SRC_PLL4] =
dcn30_clock_source_create(ctx, ctx->dc_bios,
CLOCK_SOURCE_COMBO_PHY_PLL4,
diff --git a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_resource.h b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_resource.h
index 416fe7a..a513363 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_resource.h
+++ b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_resource.h
@@ -49,4 +49,35 @@ struct resource_pool *dcn31_create_resource_pool(
const struct dc_init_data *init_data,
struct dc *dc);
+/*temp: B0 specific before switch to dcn313 headers*/
+#ifndef regPHYPLLF_PIXCLK_RESYNC_CNTL
+#define regPHYPLLF_PIXCLK_RESYNC_CNTL 0x007e
+#define regPHYPLLF_PIXCLK_RESYNC_CNTL_BASE_IDX 1
+#define regPHYPLLG_PIXCLK_RESYNC_CNTL 0x005f
+#define regPHYPLLG_PIXCLK_RESYNC_CNTL_BASE_IDX 1
+
+//PHYPLLF_PIXCLK_RESYNC_CNTL
+#define PHYPLLF_PIXCLK_RESYNC_CNTL__PHYPLLF_PIXCLK_RESYNC_ENABLE__SHIFT 0x0
+#define PHYPLLF_PIXCLK_RESYNC_CNTL__PHYPLLF_DEEP_COLOR_DTO_ENABLE_STATUS__SHIFT 0x1
+#define PHYPLLF_PIXCLK_RESYNC_CNTL__PHYPLLF_DCCG_DEEP_COLOR_CNTL__SHIFT 0x4
+#define PHYPLLF_PIXCLK_RESYNC_CNTL__PHYPLLF_PIXCLK_ENABLE__SHIFT 0x8
+#define PHYPLLF_PIXCLK_RESYNC_CNTL__PHYPLLF_PIXCLK_DOUBLE_RATE_ENABLE__SHIFT 0x9
+#define PHYPLLF_PIXCLK_RESYNC_CNTL__PHYPLLF_PIXCLK_RESYNC_ENABLE_MASK 0x00000001L
+#define PHYPLLF_PIXCLK_RESYNC_CNTL__PHYPLLF_DEEP_COLOR_DTO_ENABLE_STATUS_MASK 0x00000002L
+#define PHYPLLF_PIXCLK_RESYNC_CNTL__PHYPLLF_DCCG_DEEP_COLOR_CNTL_MASK 0x00000030L
+#define PHYPLLF_PIXCLK_RESYNC_CNTL__PHYPLLF_PIXCLK_ENABLE_MASK 0x00000100L
+#define PHYPLLF_PIXCLK_RESYNC_CNTL__PHYPLLF_PIXCLK_DOUBLE_RATE_ENABLE_MASK 0x00000200L
+
+//PHYPLLG_PIXCLK_RESYNC_CNTL
+#define PHYPLLG_PIXCLK_RESYNC_CNTL__PHYPLLG_PIXCLK_RESYNC_ENABLE__SHIFT 0x0
+#define PHYPLLG_PIXCLK_RESYNC_CNTL__PHYPLLG_DEEP_COLOR_DTO_ENABLE_STATUS__SHIFT 0x1
+#define PHYPLLG_PIXCLK_RESYNC_CNTL__PHYPLLG_DCCG_DEEP_COLOR_CNTL__SHIFT 0x4
+#define PHYPLLG_PIXCLK_RESYNC_CNTL__PHYPLLG_PIXCLK_ENABLE__SHIFT 0x8
+#define PHYPLLG_PIXCLK_RESYNC_CNTL__PHYPLLG_PIXCLK_DOUBLE_RATE_ENABLE__SHIFT 0x9
+#define PHYPLLG_PIXCLK_RESYNC_CNTL__PHYPLLG_PIXCLK_RESYNC_ENABLE_MASK 0x00000001L
+#define PHYPLLG_PIXCLK_RESYNC_CNTL__PHYPLLG_DEEP_COLOR_DTO_ENABLE_STATUS_MASK 0x00000002L
+#define PHYPLLG_PIXCLK_RESYNC_CNTL__PHYPLLG_DCCG_DEEP_COLOR_CNTL_MASK 0x00000030L
+#define PHYPLLG_PIXCLK_RESYNC_CNTL__PHYPLLG_PIXCLK_ENABLE_MASK 0x00000100L
+#define PHYPLLG_PIXCLK_RESYNC_CNTL__PHYPLLG_PIXCLK_DOUBLE_RATE_ENABLE_MASK 0x00000200L
+#endif
#endif /* _DCN31_RESOURCE_H_ */
diff --git a/drivers/gpu/drm/amd/display/dc/dm_helpers.h b/drivers/gpu/drm/amd/display/dc/dm_helpers.h
index 0fe66b0..7f94e3f 100644
--- a/drivers/gpu/drm/amd/display/dc/dm_helpers.h
+++ b/drivers/gpu/drm/amd/display/dc/dm_helpers.h
@@ -59,7 +59,7 @@ void dm_helpers_free_gpu_mem(
void *pvMem);
enum dc_edid_status dm_helpers_parse_edid_caps(
- struct dc_context *ctx,
+ struct dc_link *link,
const struct dc_edid *edid,
struct dc_edid_caps *edid_caps);
diff --git a/drivers/gpu/drm/amd/display/dc/dml/dcn21/display_rq_dlg_calc_21.c b/drivers/gpu/drm/amd/display/dc/dml/dcn21/display_rq_dlg_calc_21.c
index 46c433c..8bc27de 100644
--- a/drivers/gpu/drm/amd/display/dc/dml/dcn21/display_rq_dlg_calc_21.c
+++ b/drivers/gpu/drm/amd/display/dc/dml/dcn21/display_rq_dlg_calc_21.c
@@ -1711,14 +1711,6 @@ void dml21_rq_dlg_get_dlg_reg(
dml_print("DML_DLG: Calculation for pipe[%d] end\n", pipe_idx);
}
-void dml_rq_dlg_get_arb_params(struct display_mode_lib *mode_lib, display_arb_params_st *arb_param)
-{
- memset(arb_param, 0, sizeof(*arb_param));
- arb_param->max_req_outstanding = 256;
- arb_param->min_req_outstanding = 68;
- arb_param->sat_level_us = 60;
-}
-
static void calculate_ttu_cursor(
struct display_mode_lib *mode_lib,
double *refcyc_per_req_delivery_pre_cur,
diff --git a/drivers/gpu/drm/amd/display/dc/dml/dcn31/display_mode_vba_31.c b/drivers/gpu/drm/amd/display/dc/dml/dcn31/display_mode_vba_31.c
index 7e937bd..6feb234 100644
--- a/drivers/gpu/drm/amd/display/dc/dml/dcn31/display_mode_vba_31.c
+++ b/drivers/gpu/drm/amd/display/dc/dml/dcn31/display_mode_vba_31.c
@@ -422,62 +422,8 @@ static void CalculateUrgentBurstFactor(
static void UseMinimumDCFCLK(
struct display_mode_lib *mode_lib,
- int MaxInterDCNTileRepeaters,
int MaxPrefetchMode,
- double FinalDRAMClockChangeLatency,
- double SREnterPlusExitTime,
- int ReturnBusWidth,
- int RoundTripPingLatencyCycles,
- int ReorderingBytes,
- int PixelChunkSizeInKByte,
- int MetaChunkSize,
- bool GPUVMEnable,
- int GPUVMMaxPageTableLevels,
- bool HostVMEnable,
- int NumberOfActivePlanes,
- double HostVMMinPageSize,
- int HostVMMaxNonCachedPageTableLevels,
- bool DynamicMetadataVMEnabled,
- enum immediate_flip_requirement ImmediateFlipRequirement,
- bool ProgressiveToInterlaceUnitInOPP,
- double MaxAveragePercentOfIdealFabricAndSDPPortBWDisplayCanUseInNormalSystemOperation,
- double PercentOfIdealFabricAndSDPPortBWReceivedAfterUrgLatency,
- int VTotal[],
- int VActive[],
- int DynamicMetadataTransmittedBytes[],
- int DynamicMetadataLinesBeforeActiveRequired[],
- bool Interlace[],
- double RequiredDPPCLK[][2][DC__NUM_DPP__MAX],
- double RequiredDISPCLK[][2],
- double UrgLatency[],
- unsigned int NoOfDPP[][2][DC__NUM_DPP__MAX],
- double ProjectedDCFCLKDeepSleep[][2],
- double MaximumVStartup[][2][DC__NUM_DPP__MAX],
- double TotalVActivePixelBandwidth[][2],
- double TotalVActiveCursorBandwidth[][2],
- double TotalMetaRowBandwidth[][2],
- double TotalDPTERowBandwidth[][2],
- unsigned int TotalNumberOfActiveDPP[][2],
- unsigned int TotalNumberOfDCCActiveDPP[][2],
- int dpte_group_bytes[],
- double PrefetchLinesY[][2][DC__NUM_DPP__MAX],
- double PrefetchLinesC[][2][DC__NUM_DPP__MAX],
- int swath_width_luma_ub_all_states[][2][DC__NUM_DPP__MAX],
- int swath_width_chroma_ub_all_states[][2][DC__NUM_DPP__MAX],
- int BytePerPixelY[],
- int BytePerPixelC[],
- int HTotal[],
- double PixelClock[],
- double PDEAndMetaPTEBytesPerFrame[][2][DC__NUM_DPP__MAX],
- double DPTEBytesPerRow[][2][DC__NUM_DPP__MAX],
- double MetaRowBytes[][2][DC__NUM_DPP__MAX],
- bool DynamicMetadataEnable[],
- double VActivePixelBandwidth[][2][DC__NUM_DPP__MAX],
- double VActiveCursorBandwidth[][2][DC__NUM_DPP__MAX],
- double ReadBandwidthLuma[],
- double ReadBandwidthChroma[],
- double DCFCLKPerState[],
- double DCFCLKState[][2]);
+ int ReorderingBytes);
static void CalculatePixelDeliveryTimes(
unsigned int NumberOfActivePlanes,
@@ -3949,6 +3895,102 @@ static double TruncToValidBPP(
return BPP_INVALID;
}
+static noinline void CalculatePrefetchSchedulePerPlane(
+ struct display_mode_lib *mode_lib,
+ double HostVMInefficiencyFactor,
+ int i,
+ unsigned j,
+ unsigned k)
+{
+ struct vba_vars_st *v = &mode_lib->vba;
+ Pipe myPipe;
+
+ myPipe.DPPCLK = v->RequiredDPPCLK[i][j][k];
+ myPipe.DISPCLK = v->RequiredDISPCLK[i][j];
+ myPipe.PixelClock = v->PixelClock[k];
+ myPipe.DCFCLKDeepSleep = v->ProjectedDCFCLKDeepSleep[i][j];
+ myPipe.DPPPerPlane = v->NoOfDPP[i][j][k];
+ myPipe.ScalerEnabled = v->ScalerEnabled[k];
+ myPipe.SourceScan = v->SourceScan[k];
+ myPipe.BlockWidth256BytesY = v->Read256BlockWidthY[k];
+ myPipe.BlockHeight256BytesY = v->Read256BlockHeightY[k];
+ myPipe.BlockWidth256BytesC = v->Read256BlockWidthC[k];
+ myPipe.BlockHeight256BytesC = v->Read256BlockHeightC[k];
+ myPipe.InterlaceEnable = v->Interlace[k];
+ myPipe.NumberOfCursors = v->NumberOfCursors[k];
+ myPipe.VBlank = v->VTotal[k] - v->VActive[k];
+ myPipe.HTotal = v->HTotal[k];
+ myPipe.DCCEnable = v->DCCEnable[k];
+ myPipe.ODMCombineIsEnabled = v->ODMCombineEnablePerState[i][k] == dm_odm_combine_mode_4to1
+ || v->ODMCombineEnablePerState[i][k] == dm_odm_combine_mode_2to1;
+ myPipe.SourcePixelFormat = v->SourcePixelFormat[k];
+ myPipe.BytePerPixelY = v->BytePerPixelY[k];
+ myPipe.BytePerPixelC = v->BytePerPixelC[k];
+ myPipe.ProgressiveToInterlaceUnitInOPP = v->ProgressiveToInterlaceUnitInOPP;
+ v->NoTimeForPrefetch[i][j][k] = CalculatePrefetchSchedule(
+ mode_lib,
+ HostVMInefficiencyFactor,
+ &myPipe,
+ v->DSCDelayPerState[i][k],
+ v->DPPCLKDelaySubtotal + v->DPPCLKDelayCNVCFormater,
+ v->DPPCLKDelaySCL,
+ v->DPPCLKDelaySCLLBOnly,
+ v->DPPCLKDelayCNVCCursor,
+ v->DISPCLKDelaySubtotal,
+ v->SwathWidthYThisState[k] / v->HRatio[k],
+ v->OutputFormat[k],
+ v->MaxInterDCNTileRepeaters,
+ dml_min(v->MaxVStartup, v->MaximumVStartup[i][j][k]),
+ v->MaximumVStartup[i][j][k],
+ v->GPUVMMaxPageTableLevels,
+ v->GPUVMEnable,
+ v->HostVMEnable,
+ v->HostVMMaxNonCachedPageTableLevels,
+ v->HostVMMinPageSize,
+ v->DynamicMetadataEnable[k],
+ v->DynamicMetadataVMEnabled,
+ v->DynamicMetadataLinesBeforeActiveRequired[k],
+ v->DynamicMetadataTransmittedBytes[k],
+ v->UrgLatency[i],
+ v->ExtraLatency,
+ v->TimeCalc,
+ v->PDEAndMetaPTEBytesPerFrame[i][j][k],
+ v->MetaRowBytes[i][j][k],
+ v->DPTEBytesPerRow[i][j][k],
+ v->PrefetchLinesY[i][j][k],
+ v->SwathWidthYThisState[k],
+ v->PrefillY[k],
+ v->MaxNumSwY[k],
+ v->PrefetchLinesC[i][j][k],
+ v->SwathWidthCThisState[k],
+ v->PrefillC[k],
+ v->MaxNumSwC[k],
+ v->swath_width_luma_ub_this_state[k],
+ v->swath_width_chroma_ub_this_state[k],
+ v->SwathHeightYThisState[k],
+ v->SwathHeightCThisState[k],
+ v->TWait,
+ &v->DSTXAfterScaler[k],
+ &v->DSTYAfterScaler[k],
+ &v->LineTimesForPrefetch[k],
+ &v->PrefetchBW[k],
+ &v->LinesForMetaPTE[k],
+ &v->LinesForMetaAndDPTERow[k],
+ &v->VRatioPreY[i][j][k],
+ &v->VRatioPreC[i][j][k],
+ &v->RequiredPrefetchPixelDataBWLuma[i][j][k],
+ &v->RequiredPrefetchPixelDataBWChroma[i][j][k],
+ &v->NoTimeForDynamicMetadata[i][j][k],
+ &v->Tno_bw[k],
+ &v->prefetch_vmrow_bw[k],
+ &v->dummy7[k],
+ &v->dummy8[k],
+ &v->dummy13[k],
+ &v->VUpdateOffsetPix[k],
+ &v->VUpdateWidthPix[k],
+ &v->VReadyOffsetPix[k]);
+}
+
void dml31_ModeSupportAndSystemConfigurationFull(struct display_mode_lib *mode_lib)
{
struct vba_vars_st *v = &mode_lib->vba;
@@ -5079,66 +5121,8 @@ void dml31_ModeSupportAndSystemConfigurationFull(struct display_mode_lib *mode_l
}
}
- if (v->UseMinimumRequiredDCFCLK == true) {
- UseMinimumDCFCLK(
- mode_lib,
- v->MaxInterDCNTileRepeaters,
- MaxPrefetchMode,
- v->DRAMClockChangeLatency,
- v->SREnterPlusExitTime,
- v->ReturnBusWidth,
- v->RoundTripPingLatencyCycles,
- ReorderingBytes,
- v->PixelChunkSizeInKByte,
- v->MetaChunkSize,
- v->GPUVMEnable,
- v->GPUVMMaxPageTableLevels,
- v->HostVMEnable,
- v->NumberOfActivePlanes,
- v->HostVMMinPageSize,
- v->HostVMMaxNonCachedPageTableLevels,
- v->DynamicMetadataVMEnabled,
- v->ImmediateFlipRequirement[0],
- v->ProgressiveToInterlaceUnitInOPP,
- v->MaxAveragePercentOfIdealFabricAndSDPPortBWDisplayCanUseInNormalSystemOperation,
- v->PercentOfIdealFabricAndSDPPortBWReceivedAfterUrgLatency,
- v->VTotal,
- v->VActive,
- v->DynamicMetadataTransmittedBytes,
- v->DynamicMetadataLinesBeforeActiveRequired,
- v->Interlace,
- v->RequiredDPPCLK,
- v->RequiredDISPCLK,
- v->UrgLatency,
- v->NoOfDPP,
- v->ProjectedDCFCLKDeepSleep,
- v->MaximumVStartup,
- v->TotalVActivePixelBandwidth,
- v->TotalVActiveCursorBandwidth,
- v->TotalMetaRowBandwidth,
- v->TotalDPTERowBandwidth,
- v->TotalNumberOfActiveDPP,
- v->TotalNumberOfDCCActiveDPP,
- v->dpte_group_bytes,
- v->PrefetchLinesY,
- v->PrefetchLinesC,
- v->swath_width_luma_ub_all_states,
- v->swath_width_chroma_ub_all_states,
- v->BytePerPixelY,
- v->BytePerPixelC,
- v->HTotal,
- v->PixelClock,
- v->PDEAndMetaPTEBytesPerFrame,
- v->DPTEBytesPerRow,
- v->MetaRowBytes,
- v->DynamicMetadataEnable,
- v->VActivePixelBandwidth,
- v->VActiveCursorBandwidth,
- v->ReadBandwidthLuma,
- v->ReadBandwidthChroma,
- v->DCFCLKPerState,
- v->DCFCLKState);
- }
+ if (v->UseMinimumRequiredDCFCLK == true)
+ UseMinimumDCFCLK(mode_lib, MaxPrefetchMode, ReorderingBytes);
for (i = 0; i < v->soc.num_states; ++i) {
for (j = 0; j <= 1; ++j) {
@@ -5276,92 +5260,9 @@ void dml31_ModeSupportAndSystemConfigurationFull(struct display_mode_lib *mode_l
v->SREnterPlusExitTime);
for (k = 0; k < v->NumberOfActivePlanes; k++) {
- Pipe myPipe;
-
- myPipe.DPPCLK = v->RequiredDPPCLK[i][j][k];
- myPipe.DISPCLK = v->RequiredDISPCLK[i][j];
- myPipe.PixelClock = v->PixelClock[k];
- myPipe.DCFCLKDeepSleep = v->ProjectedDCFCLKDeepSleep[i][j];
- myPipe.DPPPerPlane = v->NoOfDPP[i][j][k];
- myPipe.ScalerEnabled = v->ScalerEnabled[k];
- myPipe.SourceScan = v->SourceScan[k];
- myPipe.BlockWidth256BytesY = v->Read256BlockWidthY[k];
- myPipe.BlockHeight256BytesY = v->Read256BlockHeightY[k];
- myPipe.BlockWidth256BytesC = v->Read256BlockWidthC[k];
- myPipe.BlockHeight256BytesC = v->Read256BlockHeightC[k];
- myPipe.InterlaceEnable = v->Interlace[k];
- myPipe.NumberOfCursors = v->NumberOfCursors[k];
- myPipe.VBlank = v->VTotal[k] - v->VActive[k];
- myPipe.HTotal = v->HTotal[k];
- myPipe.DCCEnable = v->DCCEnable[k];
- myPipe.ODMCombineIsEnabled = v->ODMCombineEnablePerState[i][k] == dm_odm_combine_mode_4to1
- || v->ODMCombineEnablePerState[i][k] == dm_odm_combine_mode_2to1;
- myPipe.SourcePixelFormat = v->SourcePixelFormat[k];
- myPipe.BytePerPixelY = v->BytePerPixelY[k];
- myPipe.BytePerPixelC = v->BytePerPixelC[k];
- myPipe.ProgressiveToInterlaceUnitInOPP = v->ProgressiveToInterlaceUnitInOPP;
- v->NoTimeForPrefetch[i][j][k] = CalculatePrefetchSchedule(
- mode_lib,
- HostVMInefficiencyFactor,
- &myPipe,
- v->DSCDelayPerState[i][k],
- v->DPPCLKDelaySubtotal + v->DPPCLKDelayCNVCFormater,
- v->DPPCLKDelaySCL,
- v->DPPCLKDelaySCLLBOnly,
- v->DPPCLKDelayCNVCCursor,
- v->DISPCLKDelaySubtotal,
- v->SwathWidthYThisState[k] / v->HRatio[k],
- v->OutputFormat[k],
- v->MaxInterDCNTileRepeaters,
- dml_min(v->MaxVStartup, v->MaximumVStartup[i][j][k]),
- v->MaximumVStartup[i][j][k],
- v->GPUVMMaxPageTableLevels,
- v->GPUVMEnable,
- v->HostVMEnable,
- v->HostVMMaxNonCachedPageTableLevels,
- v->HostVMMinPageSize,
- v->DynamicMetadataEnable[k],
- v->DynamicMetadataVMEnabled,
- v->DynamicMetadataLinesBeforeActiveRequired[k],
- v->DynamicMetadataTransmittedBytes[k],
- v->UrgLatency[i],
- v->ExtraLatency,
- v->TimeCalc,
- v->PDEAndMetaPTEBytesPerFrame[i][j][k],
- v->MetaRowBytes[i][j][k],
- v->DPTEBytesPerRow[i][j][k],
- v->PrefetchLinesY[i][j][k],
- v->SwathWidthYThisState[k],
- v->PrefillY[k],
- v->MaxNumSwY[k],
- v->PrefetchLinesC[i][j][k],
- v->SwathWidthCThisState[k],
- v->PrefillC[k],
- v->MaxNumSwC[k],
- v->swath_width_luma_ub_this_state[k],
- v->swath_width_chroma_ub_this_state[k],
- v->SwathHeightYThisState[k],
- v->SwathHeightCThisState[k],
- v->TWait,
- &v->DSTXAfterScaler[k],
- &v->DSTYAfterScaler[k],
- &v->LineTimesForPrefetch[k],
- &v->PrefetchBW[k],
- &v->LinesForMetaPTE[k],
- &v->LinesForMetaAndDPTERow[k],
- &v->VRatioPreY[i][j][k],
- &v->VRatioPreC[i][j][k],
- &v->RequiredPrefetchPixelDataBWLuma[i][j][k],
- &v->RequiredPrefetchPixelDataBWChroma[i][j][k],
- &v->NoTimeForDynamicMetadata[i][j][k],
- &v->Tno_bw[k],
- &v->prefetch_vmrow_bw[k],
- &v->dummy7[k],
- &v->dummy8[k],
- &v->dummy13[k],
- &v->VUpdateOffsetPix[k],
- &v->VUpdateWidthPix[k],
- &v->VReadyOffsetPix[k]);
+ CalculatePrefetchSchedulePerPlane(mode_lib,
+ HostVMInefficiencyFactor,
+ i, j, k);
}
for (k = 0; k < v->NumberOfActivePlanes; k++) {
@@ -7249,69 +7150,15 @@ static double CalculateUrgentLatency(
static void UseMinimumDCFCLK(
struct display_mode_lib *mode_lib,
- int MaxInterDCNTileRepeaters,
int MaxPrefetchMode,
- double FinalDRAMClockChangeLatency,
- double SREnterPlusExitTime,
- int ReturnBusWidth,
- int RoundTripPingLatencyCycles,
- int ReorderingBytes,
- int PixelChunkSizeInKByte,
- int MetaChunkSize,
- bool GPUVMEnable,
- int GPUVMMaxPageTableLevels,
- bool HostVMEnable,
- int NumberOfActivePlanes,
- double HostVMMinPageSize,
- int HostVMMaxNonCachedPageTableLevels,
- bool DynamicMetadataVMEnabled,
- enum immediate_flip_requirement ImmediateFlipRequirement,
- bool ProgressiveToInterlaceUnitInOPP,
- double MaxAveragePercentOfIdealFabricAndSDPPortBWDisplayCanUseInNormalSystemOperation,
- double PercentOfIdealFabricAndSDPPortBWReceivedAfterUrgLatency,
- int VTotal[],
- int VActive[],
- int DynamicMetadataTransmittedBytes[],
- int DynamicMetadataLinesBeforeActiveRequired[],
- bool Interlace[],
- double RequiredDPPCLK[][2][DC__NUM_DPP__MAX],
- double RequiredDISPCLK[][2],
- double UrgLatency[],
- unsigned int NoOfDPP[][2][DC__NUM_DPP__MAX],
- double ProjectedDCFCLKDeepSleep[][2],
- double MaximumVStartup[][2][DC__NUM_DPP__MAX],
- double TotalVActivePixelBandwidth[][2],
- double TotalVActiveCursorBandwidth[][2],
- double TotalMetaRowBandwidth[][2],
- double TotalDPTERowBandwidth[][2],
- unsigned int TotalNumberOfActiveDPP[][2],
- unsigned int TotalNumberOfDCCActiveDPP[][2],
- int dpte_group_bytes[],
- double PrefetchLinesY[][2][DC__NUM_DPP__MAX],
- double PrefetchLinesC[][2][DC__NUM_DPP__MAX],
- int swath_width_luma_ub_all_states[][2][DC__NUM_DPP__MAX],
- int swath_width_chroma_ub_all_states[][2][DC__NUM_DPP__MAX],
- int BytePerPixelY[],
- int BytePerPixelC[],
- int HTotal[],
- double PixelClock[],
- double PDEAndMetaPTEBytesPerFrame[][2][DC__NUM_DPP__MAX],
- double DPTEBytesPerRow[][2][DC__NUM_DPP__MAX],
- double MetaRowBytes[][2][DC__NUM_DPP__MAX],
- bool DynamicMetadataEnable[],
- double VActivePixelBandwidth[][2][DC__NUM_DPP__MAX],
- double VActiveCursorBandwidth[][2][DC__NUM_DPP__MAX],
- double ReadBandwidthLuma[],
- double ReadBandwidthChroma[],
- double DCFCLKPerState[],
- double DCFCLKState[][2])
+ int ReorderingBytes)
{
struct vba_vars_st *v = &mode_lib->vba;
int dummy1, i, j, k;
double NormalEfficiency, dummy2, dummy3;
double TotalMaxPrefetchFlipDPTERowBandwidth[DC__VOLTAGE_STATES][2];
- NormalEfficiency = PercentOfIdealFabricAndSDPPortBWReceivedAfterUrgLatency / 100.0;
+ NormalEfficiency = v->PercentOfIdealFabricAndSDPPortBWReceivedAfterUrgLatency / 100.0;
for (i = 0; i < v->soc.num_states; ++i) {
for (j = 0; j <= 1; ++j) {
double PixelDCFCLKCyclesRequiredInPrefetch[DC__NUM_DPP__MAX];
@@ -7329,61 +7176,61 @@ static void UseMinimumDCFCLK(
double MinimumTvmPlus2Tr0;
TotalMaxPrefetchFlipDPTERowBandwidth[i][j] = 0;
- for (k = 0; k < NumberOfActivePlanes; ++k) {
+ for (k = 0; k < v->NumberOfActivePlanes; ++k) {
TotalMaxPrefetchFlipDPTERowBandwidth[i][j] = TotalMaxPrefetchFlipDPTERowBandwidth[i][j]
- + NoOfDPP[i][j][k] * DPTEBytesPerRow[i][j][k] / (15.75 * HTotal[k] / PixelClock[k]);
+ + v->NoOfDPP[i][j][k] * v->DPTEBytesPerRow[i][j][k] / (15.75 * v->HTotal[k] / v->PixelClock[k]);
}
- for (k = 0; k <= NumberOfActivePlanes - 1; ++k) {
- NoOfDPPState[k] = NoOfDPP[i][j][k];
+ for (k = 0; k <= v->NumberOfActivePlanes - 1; ++k) {
+ NoOfDPPState[k] = v->NoOfDPP[i][j][k];
}
- MinimumTWait = CalculateTWait(MaxPrefetchMode, FinalDRAMClockChangeLatency, UrgLatency[i], SREnterPlusExitTime);
- NonDPTEBandwidth = TotalVActivePixelBandwidth[i][j] + TotalVActiveCursorBandwidth[i][j] + TotalMetaRowBandwidth[i][j];
- DPTEBandwidth = (HostVMEnable == true || ImmediateFlipRequirement == dm_immediate_flip_required) ?
- TotalMaxPrefetchFlipDPTERowBandwidth[i][j] : TotalDPTERowBandwidth[i][j];
+ MinimumTWait = CalculateTWait(MaxPrefetchMode, v->FinalDRAMClockChangeLatency, v->UrgLatency[i], v->SREnterPlusExitTime);
+ NonDPTEBandwidth = v->TotalVActivePixelBandwidth[i][j] + v->TotalVActiveCursorBandwidth[i][j] + v->TotalMetaRowBandwidth[i][j];
+ DPTEBandwidth = (v->HostVMEnable == true || v->ImmediateFlipRequirement[0] == dm_immediate_flip_required) ?
+ TotalMaxPrefetchFlipDPTERowBandwidth[i][j] : v->TotalDPTERowBandwidth[i][j];
DCFCLKRequiredForAverageBandwidth = dml_max3(
- ProjectedDCFCLKDeepSleep[i][j],
- (NonDPTEBandwidth + TotalDPTERowBandwidth[i][j]) / ReturnBusWidth
- / (MaxAveragePercentOfIdealFabricAndSDPPortBWDisplayCanUseInNormalSystemOperation / 100),
- (NonDPTEBandwidth + DPTEBandwidth / NormalEfficiency) / NormalEfficiency / ReturnBusWidth);
+ v->ProjectedDCFCLKDeepSleep[i][j],
+ (NonDPTEBandwidth + v->TotalDPTERowBandwidth[i][j]) / v->ReturnBusWidth
+ / (v->MaxAveragePercentOfIdealFabricAndSDPPortBWDisplayCanUseInNormalSystemOperation / 100),
+ (NonDPTEBandwidth + DPTEBandwidth / NormalEfficiency) / NormalEfficiency / v->ReturnBusWidth);
ExtraLatencyBytes = CalculateExtraLatencyBytes(
ReorderingBytes,
- TotalNumberOfActiveDPP[i][j],
- PixelChunkSizeInKByte,
- TotalNumberOfDCCActiveDPP[i][j],
- MetaChunkSize,
- GPUVMEnable,
- HostVMEnable,
- NumberOfActivePlanes,
+ v->TotalNumberOfActiveDPP[i][j],
+ v->PixelChunkSizeInKByte,
+ v->TotalNumberOfDCCActiveDPP[i][j],
+ v->MetaChunkSize,
+ v->GPUVMEnable,
+ v->HostVMEnable,
+ v->NumberOfActivePlanes,
NoOfDPPState,
- dpte_group_bytes,
+ v->dpte_group_bytes,
1,
- HostVMMinPageSize,
- HostVMMaxNonCachedPageTableLevels);
- ExtraLatencyCycles = RoundTripPingLatencyCycles + __DML_ARB_TO_RET_DELAY__ + ExtraLatencyBytes / NormalEfficiency / ReturnBusWidth;
- for (k = 0; k < NumberOfActivePlanes; ++k) {
+ v->HostVMMinPageSize,
+ v->HostVMMaxNonCachedPageTableLevels);
+ ExtraLatencyCycles = v->RoundTripPingLatencyCycles + __DML_ARB_TO_RET_DELAY__ + ExtraLatencyBytes / NormalEfficiency / v->ReturnBusWidth;
+ for (k = 0; k < v->NumberOfActivePlanes; ++k) {
double DCFCLKCyclesRequiredInPrefetch;
double ExpectedPrefetchBWAcceleration;
double PrefetchTime;
- PixelDCFCLKCyclesRequiredInPrefetch[k] = (PrefetchLinesY[i][j][k] * swath_width_luma_ub_all_states[i][j][k] * BytePerPixelY[k]
- + PrefetchLinesC[i][j][k] * swath_width_chroma_ub_all_states[i][j][k] * BytePerPixelC[k]) / NormalEfficiency / ReturnBusWidth;
+ PixelDCFCLKCyclesRequiredInPrefetch[k] = (v->PrefetchLinesY[i][j][k] * v->swath_width_luma_ub_all_states[i][j][k] * v->BytePerPixelY[k]
+ + v->PrefetchLinesC[i][j][k] * v->swath_width_chroma_ub_all_states[i][j][k] * v->BytePerPixelC[k]) / NormalEfficiency / v->ReturnBusWidth;
DCFCLKCyclesRequiredInPrefetch = 2 * ExtraLatencyCycles / NoOfDPPState[k]
- + PDEAndMetaPTEBytesPerFrame[i][j][k] / NormalEfficiency / NormalEfficiency / ReturnBusWidth * (GPUVMMaxPageTableLevels > 2 ? 1 : 0)
- + 2 * DPTEBytesPerRow[i][j][k] / NormalEfficiency / NormalEfficiency / ReturnBusWidth
- + 2 * MetaRowBytes[i][j][k] / NormalEfficiency / ReturnBusWidth + PixelDCFCLKCyclesRequiredInPrefetch[k];
- PrefetchPixelLinesTime[k] = dml_max(PrefetchLinesY[i][j][k], PrefetchLinesC[i][j][k]) * HTotal[k] / PixelClock[k];
- ExpectedPrefetchBWAcceleration = (VActivePixelBandwidth[i][j][k] + VActiveCursorBandwidth[i][j][k])
- / (ReadBandwidthLuma[k] + ReadBandwidthChroma[k]);
+ + v->PDEAndMetaPTEBytesPerFrame[i][j][k] / NormalEfficiency / NormalEfficiency / v->ReturnBusWidth * (v->GPUVMMaxPageTableLevels > 2 ? 1 : 0)
+ + 2 * v->DPTEBytesPerRow[i][j][k] / NormalEfficiency / NormalEfficiency / v->ReturnBusWidth
+ + 2 * v->MetaRowBytes[i][j][k] / NormalEfficiency / v->ReturnBusWidth + PixelDCFCLKCyclesRequiredInPrefetch[k];
+ PrefetchPixelLinesTime[k] = dml_max(v->PrefetchLinesY[i][j][k], v->PrefetchLinesC[i][j][k]) * v->HTotal[k] / v->PixelClock[k];
+ ExpectedPrefetchBWAcceleration = (v->VActivePixelBandwidth[i][j][k] + v->VActiveCursorBandwidth[i][j][k])
+ / (v->ReadBandwidthLuma[k] + v->ReadBandwidthChroma[k]);
DynamicMetadataVMExtraLatency[k] =
- (GPUVMEnable == true && DynamicMetadataEnable[k] == true && DynamicMetadataVMEnabled == true) ?
- UrgLatency[i] * GPUVMMaxPageTableLevels * (HostVMEnable == true ? HostVMMaxNonCachedPageTableLevels + 1 : 1) : 0;
- PrefetchTime = (MaximumVStartup[i][j][k] - 1) * HTotal[k] / PixelClock[k] - MinimumTWait
- - UrgLatency[i]
- * ((GPUVMMaxPageTableLevels <= 2 ? GPUVMMaxPageTableLevels : GPUVMMaxPageTableLevels - 2)
- * (HostVMEnable == true ? HostVMMaxNonCachedPageTableLevels + 1 : 1) - 1)
+ (v->GPUVMEnable == true && v->DynamicMetadataEnable[k] == true && v->DynamicMetadataVMEnabled == true) ?
+ v->UrgLatency[i] * v->GPUVMMaxPageTableLevels * (v->HostVMEnable == true ? v->HostVMMaxNonCachedPageTableLevels + 1 : 1) : 0;
+ PrefetchTime = (v->MaximumVStartup[i][j][k] - 1) * v->HTotal[k] / v->PixelClock[k] - MinimumTWait
+ - v->UrgLatency[i]
+ * ((v->GPUVMMaxPageTableLevels <= 2 ? v->GPUVMMaxPageTableLevels : v->GPUVMMaxPageTableLevels - 2)
+ * (v->HostVMEnable == true ? v->HostVMMaxNonCachedPageTableLevels + 1 : 1) - 1)
- DynamicMetadataVMExtraLatency[k];
if (PrefetchTime > 0) {
@@ -7392,14 +7239,14 @@ static void UseMinimumDCFCLK(
/ (PrefetchTime * PixelDCFCLKCyclesRequiredInPrefetch[k] / DCFCLKCyclesRequiredInPrefetch);
DCFCLKRequiredForPeakBandwidthPerPlane[k] = NoOfDPPState[k] * PixelDCFCLKCyclesRequiredInPrefetch[k] / PrefetchPixelLinesTime[k]
* dml_max(1.0, ExpectedVRatioPrefetch) * dml_max(1.0, ExpectedVRatioPrefetch / 4) * ExpectedPrefetchBWAcceleration;
- if (HostVMEnable == true || ImmediateFlipRequirement == dm_immediate_flip_required) {
+ if (v->HostVMEnable == true || v->ImmediateFlipRequirement[0] == dm_immediate_flip_required) {
DCFCLKRequiredForPeakBandwidthPerPlane[k] = DCFCLKRequiredForPeakBandwidthPerPlane[k]
- + NoOfDPPState[k] * DPTEBandwidth / NormalEfficiency / NormalEfficiency / ReturnBusWidth;
+ + NoOfDPPState[k] * DPTEBandwidth / NormalEfficiency / NormalEfficiency / v->ReturnBusWidth;
}
} else {
- DCFCLKRequiredForPeakBandwidthPerPlane[k] = DCFCLKPerState[i];
+ DCFCLKRequiredForPeakBandwidthPerPlane[k] = v->DCFCLKPerState[i];
}
- if (DynamicMetadataEnable[k] == true) {
+ if (v->DynamicMetadataEnable[k] == true) {
double TSetupPipe;
double TdmbfPipe;
double TdmsksPipe;
@@ -7407,17 +7254,17 @@ static void UseMinimumDCFCLK(
double AllowedTimeForUrgentExtraLatency;
CalculateVupdateAndDynamicMetadataParameters(
- MaxInterDCNTileRepeaters,
- RequiredDPPCLK[i][j][k],
- RequiredDISPCLK[i][j],
- ProjectedDCFCLKDeepSleep[i][j],
- PixelClock[k],
- HTotal[k],
- VTotal[k] - VActive[k],
- DynamicMetadataTransmittedBytes[k],
- DynamicMetadataLinesBeforeActiveRequired[k],
- Interlace[k],
- ProgressiveToInterlaceUnitInOPP,
+ v->MaxInterDCNTileRepeaters,
+ v->RequiredDPPCLK[i][j][k],
+ v->RequiredDISPCLK[i][j],
+ v->ProjectedDCFCLKDeepSleep[i][j],
+ v->PixelClock[k],
+ v->HTotal[k],
+ v->VTotal[k] - v->VActive[k],
+ v->DynamicMetadataTransmittedBytes[k],
+ v->DynamicMetadataLinesBeforeActiveRequired[k],
+ v->Interlace[k],
+ v->ProgressiveToInterlaceUnitInOPP,
&TSetupPipe,
&TdmbfPipe,
&TdmecPipe,
@@ -7425,31 +7272,31 @@ static void UseMinimumDCFCLK(
&dummy1,
&dummy2,
&dummy3);
- AllowedTimeForUrgentExtraLatency = MaximumVStartup[i][j][k] * HTotal[k] / PixelClock[k] - MinimumTWait - TSetupPipe - TdmbfPipe - TdmecPipe
+ AllowedTimeForUrgentExtraLatency = v->MaximumVStartup[i][j][k] * v->HTotal[k] / v->PixelClock[k] - MinimumTWait - TSetupPipe - TdmbfPipe - TdmecPipe
- TdmsksPipe - DynamicMetadataVMExtraLatency[k];
if (AllowedTimeForUrgentExtraLatency > 0) {
DCFCLKRequiredForPeakBandwidthPerPlane[k] = dml_max(
DCFCLKRequiredForPeakBandwidthPerPlane[k],
ExtraLatencyCycles / AllowedTimeForUrgentExtraLatency);
} else {
- DCFCLKRequiredForPeakBandwidthPerPlane[k] = DCFCLKPerState[i];
+ DCFCLKRequiredForPeakBandwidthPerPlane[k] = v->DCFCLKPerState[i];
}
}
}
DCFCLKRequiredForPeakBandwidth = 0;
- for (k = 0; k <= NumberOfActivePlanes - 1; ++k) {
+ for (k = 0; k <= v->NumberOfActivePlanes - 1; ++k) {
DCFCLKRequiredForPeakBandwidth = DCFCLKRequiredForPeakBandwidth + DCFCLKRequiredForPeakBandwidthPerPlane[k];
}
- MinimumTvmPlus2Tr0 = UrgLatency[i]
- * (GPUVMEnable == true ?
- (HostVMEnable == true ?
- (GPUVMMaxPageTableLevels + 2) * (HostVMMaxNonCachedPageTableLevels + 1) - 1 : GPUVMMaxPageTableLevels + 1) :
+ MinimumTvmPlus2Tr0 = v->UrgLatency[i]
+ * (v->GPUVMEnable == true ?
+ (v->HostVMEnable == true ?
+ (v->GPUVMMaxPageTableLevels + 2) * (v->HostVMMaxNonCachedPageTableLevels + 1) - 1 : v->GPUVMMaxPageTableLevels + 1) :
0);
- for (k = 0; k < NumberOfActivePlanes; ++k) {
+ for (k = 0; k < v->NumberOfActivePlanes; ++k) {
double MaximumTvmPlus2Tr0PlusTsw;
- MaximumTvmPlus2Tr0PlusTsw = (MaximumVStartup[i][j][k] - 2) * HTotal[k] / PixelClock[k] - MinimumTWait - DynamicMetadataVMExtraLatency[k];
+ MaximumTvmPlus2Tr0PlusTsw = (v->MaximumVStartup[i][j][k] - 2) * v->HTotal[k] / v->PixelClock[k] - MinimumTWait - DynamicMetadataVMExtraLatency[k];
if (MaximumTvmPlus2Tr0PlusTsw <= MinimumTvmPlus2Tr0 + PrefetchPixelLinesTime[k] / 4) {
- DCFCLKRequiredForPeakBandwidth = DCFCLKPerState[i];
+ DCFCLKRequiredForPeakBandwidth = v->DCFCLKPerState[i];
} else {
DCFCLKRequiredForPeakBandwidth = dml_max3(
DCFCLKRequiredForPeakBandwidth,
@@ -7457,7 +7304,7 @@ static void UseMinimumDCFCLK(
(2 * ExtraLatencyCycles + PixelDCFCLKCyclesRequiredInPrefetch[k]) / (MaximumTvmPlus2Tr0PlusTsw - MinimumTvmPlus2Tr0));
}
}
- DCFCLKState[i][j] = dml_min(DCFCLKPerState[i], 1.05 * dml_max(DCFCLKRequiredForAverageBandwidth, DCFCLKRequiredForPeakBandwidth));
+ v->DCFCLKState[i][j] = dml_min(v->DCFCLKPerState[i], 1.05 * dml_max(DCFCLKRequiredForAverageBandwidth, DCFCLKRequiredForPeakBandwidth));
}
}
}
diff --git a/drivers/gpu/drm/amd/display/dc/dml/display_mode_lib.h b/drivers/gpu/drm/amd/display/dc/dml/display_mode_lib.h
index 6905ef1..d76251f 100644
--- a/drivers/gpu/drm/amd/display/dc/dml/display_mode_lib.h
+++ b/drivers/gpu/drm/amd/display/dc/dml/display_mode_lib.h
@@ -73,6 +73,7 @@ struct display_mode_lib {
struct vba_vars_st vba;
struct dal_logger *logger;
struct dml_funcs funcs;
+ struct _vcs_dpi_display_e2e_pipe_params_st dml_pipe_state[6];
};
void dml_init_instance(struct display_mode_lib *lib,
diff --git a/drivers/gpu/drm/amd/display/dc/dml/dml_wrapper.c b/drivers/gpu/drm/amd/display/dc/dml/dml_wrapper.c
new file mode 100644
index 0000000..789f756
--- /dev/null
+++ b/drivers/gpu/drm/amd/display/dc/dml/dml_wrapper.c
@@ -0,0 +1,1889 @@
+/*
+ * Copyright 2017 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ * Authors: AMD
+ *
+ */
+
+#include "dml_wrapper.h"
+#include "resource.h"
+#include "core_types.h"
+#include "dsc.h"
+#include "clk_mgr.h"
+
+#ifndef DC_LOGGER_INIT
+#define DC_LOGGER_INIT
+#undef DC_LOG_WARNING
+#define DC_LOG_WARNING
+#endif
+
+#define DML_WRAPPER_TRANSLATION_
+#include "dml_wrapper_translation.c"
+#undef DML_WRAPPER_TRANSLATION_
+
+static bool is_dual_plane(enum surface_pixel_format format)
+{
+ return format >= SURFACE_PIXEL_FORMAT_VIDEO_BEGIN || format == SURFACE_PIXEL_FORMAT_GRPH_RGBE_ALPHA;
+}
+
+static void build_clamping_params(struct dc_stream_state *stream)
+{
+ stream->clamping.clamping_level = CLAMPING_FULL_RANGE;
+ stream->clamping.c_depth = stream->timing.display_color_depth;
+ stream->clamping.pixel_encoding = stream->timing.pixel_encoding;
+}
+
+static void get_pixel_clock_parameters(
+ const struct pipe_ctx *pipe_ctx,
+ struct pixel_clk_params *pixel_clk_params)
+{
+ const struct dc_stream_state *stream = pipe_ctx->stream;
+
+ /*TODO: is this halved for YCbCr 420? in that case we might want to move
+ * the pixel clock normalization for hdmi up to here instead of doing it
+ * in pll_adjust_pix_clk
+ */
+ pixel_clk_params->requested_pix_clk_100hz = stream->timing.pix_clk_100hz;
+ pixel_clk_params->encoder_object_id = stream->link->link_enc->id;
+ pixel_clk_params->signal_type = pipe_ctx->stream->signal;
+ pixel_clk_params->controller_id = pipe_ctx->stream_res.tg->inst + 1;
+ /* TODO: un-hardcode*/
+ pixel_clk_params->requested_sym_clk = LINK_RATE_LOW *
+ LINK_RATE_REF_FREQ_IN_KHZ;
+ pixel_clk_params->flags.ENABLE_SS = 0;
+ pixel_clk_params->color_depth =
+ stream->timing.display_color_depth;
+ pixel_clk_params->flags.DISPLAY_BLANKED = 1;
+ pixel_clk_params->flags.SUPPORT_YCBCR420 = (stream->timing.pixel_encoding ==
+ PIXEL_ENCODING_YCBCR420);
+ pixel_clk_params->pixel_encoding = stream->timing.pixel_encoding;
+ if (stream->timing.pixel_encoding == PIXEL_ENCODING_YCBCR422) {
+ pixel_clk_params->color_depth = COLOR_DEPTH_888;
+ }
+ if (stream->timing.pixel_encoding == PIXEL_ENCODING_YCBCR420) {
+ pixel_clk_params->requested_pix_clk_100hz = pixel_clk_params->requested_pix_clk_100hz / 2;
+ }
+ if (stream->timing.timing_3d_format == TIMING_3D_FORMAT_HW_FRAME_PACKING)
+ pixel_clk_params->requested_pix_clk_100hz *= 2;
+
+}
+
+static enum dc_status build_pipe_hw_param(struct pipe_ctx *pipe_ctx)
+{
+ get_pixel_clock_parameters(pipe_ctx, &pipe_ctx->stream_res.pix_clk_params);
+
+ if (pipe_ctx->clock_source)
+ pipe_ctx->clock_source->funcs->get_pix_clk_dividers(
+ pipe_ctx->clock_source,
+ &pipe_ctx->stream_res.pix_clk_params,
+ &pipe_ctx->pll_settings);
+
+ pipe_ctx->stream->clamping.pixel_encoding = pipe_ctx->stream->timing.pixel_encoding;
+
+ resource_build_bit_depth_reduction_params(pipe_ctx->stream,
+ &pipe_ctx->stream->bit_depth_params);
+ build_clamping_params(pipe_ctx->stream);
+
+ return DC_OK;
+}
+
+static void resource_build_bit_depth_reduction_params(struct dc_stream_state *stream,
+ struct bit_depth_reduction_params *fmt_bit_depth)
+{
+ enum dc_dither_option option = stream->dither_option;
+ enum dc_pixel_encoding pixel_encoding =
+ stream->timing.pixel_encoding;
+
+ memset(fmt_bit_depth, 0, sizeof(*fmt_bit_depth));
+
+ if (option == DITHER_OPTION_DEFAULT) {
+ switch (stream->timing.display_color_depth) {
+ case COLOR_DEPTH_666:
+ option = DITHER_OPTION_SPATIAL6;
+ break;
+ case COLOR_DEPTH_888:
+ option = DITHER_OPTION_SPATIAL8;
+ break;
+ case COLOR_DEPTH_101010:
+ option = DITHER_OPTION_SPATIAL10;
+ break;
+ default:
+ option = DITHER_OPTION_DISABLE;
+ }
+ }
+
+ if (option == DITHER_OPTION_DISABLE)
+ return;
+
+ if (option == DITHER_OPTION_TRUN6) {
+ fmt_bit_depth->flags.TRUNCATE_ENABLED = 1;
+ fmt_bit_depth->flags.TRUNCATE_DEPTH = 0;
+ } else if (option == DITHER_OPTION_TRUN8 ||
+ option == DITHER_OPTION_TRUN8_SPATIAL6 ||
+ option == DITHER_OPTION_TRUN8_FM6) {
+ fmt_bit_depth->flags.TRUNCATE_ENABLED = 1;
+ fmt_bit_depth->flags.TRUNCATE_DEPTH = 1;
+ } else if (option == DITHER_OPTION_TRUN10 ||
+ option == DITHER_OPTION_TRUN10_SPATIAL6 ||
+ option == DITHER_OPTION_TRUN10_SPATIAL8 ||
+ option == DITHER_OPTION_TRUN10_FM8 ||
+ option == DITHER_OPTION_TRUN10_FM6 ||
+ option == DITHER_OPTION_TRUN10_SPATIAL8_FM6) {
+ fmt_bit_depth->flags.TRUNCATE_ENABLED = 1;
+ fmt_bit_depth->flags.TRUNCATE_DEPTH = 2;
+ }
+
+ /* special case - Formatter can only reduce by 4 bits at most.
+ * When reducing from 12 to 6 bits,
+ * HW recommends we use trunc with round mode
+ * (if we did nothing, trunc to 10 bits would be used)
+ * note that any 12->10 bit reduction is ignored prior to DCE8,
+ * as the input was 10 bits.
+ */
+ if (option == DITHER_OPTION_SPATIAL6_FRAME_RANDOM ||
+ option == DITHER_OPTION_SPATIAL6 ||
+ option == DITHER_OPTION_FM6) {
+ fmt_bit_depth->flags.TRUNCATE_ENABLED = 1;
+ fmt_bit_depth->flags.TRUNCATE_DEPTH = 2;
+ fmt_bit_depth->flags.TRUNCATE_MODE = 1;
+ }
+
+ /* spatial dither
+ * note that spatial modes 1-3 are never used
+ */
+ if (option == DITHER_OPTION_SPATIAL6_FRAME_RANDOM ||
+ option == DITHER_OPTION_SPATIAL6 ||
+ option == DITHER_OPTION_TRUN10_SPATIAL6 ||
+ option == DITHER_OPTION_TRUN8_SPATIAL6) {
+ fmt_bit_depth->flags.SPATIAL_DITHER_ENABLED = 1;
+ fmt_bit_depth->flags.SPATIAL_DITHER_DEPTH = 0;
+ fmt_bit_depth->flags.HIGHPASS_RANDOM = 1;
+ fmt_bit_depth->flags.RGB_RANDOM =
+ (pixel_encoding == PIXEL_ENCODING_RGB) ? 1 : 0;
+ } else if (option == DITHER_OPTION_SPATIAL8_FRAME_RANDOM ||
+ option == DITHER_OPTION_SPATIAL8 ||
+ option == DITHER_OPTION_SPATIAL8_FM6 ||
+ option == DITHER_OPTION_TRUN10_SPATIAL8 ||
+ option == DITHER_OPTION_TRUN10_SPATIAL8_FM6) {
+ fmt_bit_depth->flags.SPATIAL_DITHER_ENABLED = 1;
+ fmt_bit_depth->flags.SPATIAL_DITHER_DEPTH = 1;
+ fmt_bit_depth->flags.HIGHPASS_RANDOM = 1;
+ fmt_bit_depth->flags.RGB_RANDOM =
+ (pixel_encoding == PIXEL_ENCODING_RGB) ? 1 : 0;
+ } else if (option == DITHER_OPTION_SPATIAL10_FRAME_RANDOM ||
+ option == DITHER_OPTION_SPATIAL10 ||
+ option == DITHER_OPTION_SPATIAL10_FM8 ||
+ option == DITHER_OPTION_SPATIAL10_FM6) {
+ fmt_bit_depth->flags.SPATIAL_DITHER_ENABLED = 1;
+ fmt_bit_depth->flags.SPATIAL_DITHER_DEPTH = 2;
+ fmt_bit_depth->flags.HIGHPASS_RANDOM = 1;
+ fmt_bit_depth->flags.RGB_RANDOM =
+ (pixel_encoding == PIXEL_ENCODING_RGB) ? 1 : 0;
+ }
+
+ if (option == DITHER_OPTION_SPATIAL6 ||
+ option == DITHER_OPTION_SPATIAL8 ||
+ option == DITHER_OPTION_SPATIAL10) {
+ fmt_bit_depth->flags.FRAME_RANDOM = 0;
+ } else {
+ fmt_bit_depth->flags.FRAME_RANDOM = 1;
+ }
+
+ //////////////////////
+ //// temporal dither
+ //////////////////////
+ if (option == DITHER_OPTION_FM6 ||
+ option == DITHER_OPTION_SPATIAL8_FM6 ||
+ option == DITHER_OPTION_SPATIAL10_FM6 ||
+ option == DITHER_OPTION_TRUN10_FM6 ||
+ option == DITHER_OPTION_TRUN8_FM6 ||
+ option == DITHER_OPTION_TRUN10_SPATIAL8_FM6) {
+ fmt_bit_depth->flags.FRAME_MODULATION_ENABLED = 1;
+ fmt_bit_depth->flags.FRAME_MODULATION_DEPTH = 0;
+ } else if (option == DITHER_OPTION_FM8 ||
+ option == DITHER_OPTION_SPATIAL10_FM8 ||
+ option == DITHER_OPTION_TRUN10_FM8) {
+ fmt_bit_depth->flags.FRAME_MODULATION_ENABLED = 1;
+ fmt_bit_depth->flags.FRAME_MODULATION_DEPTH = 1;
+ } else if (option == DITHER_OPTION_FM10) {
+ fmt_bit_depth->flags.FRAME_MODULATION_ENABLED = 1;
+ fmt_bit_depth->flags.FRAME_MODULATION_DEPTH = 2;
+ }
+
+ fmt_bit_depth->pixel_encoding = pixel_encoding;
+}
+
+bool dml_validate_dsc(struct dc *dc, struct dc_state *new_ctx)
+{
+ int i;
+
+ /* Validate DSC config, dsc count validation is already done */
+ for (i = 0; i < dc->res_pool->pipe_count; i++) {
+ struct pipe_ctx *pipe_ctx = &new_ctx->res_ctx.pipe_ctx[i];
+ struct dc_stream_state *stream = pipe_ctx->stream;
+ struct dsc_config dsc_cfg;
+ struct pipe_ctx *odm_pipe;
+ int opp_cnt = 1;
+
+ for (odm_pipe = pipe_ctx->next_odm_pipe; odm_pipe; odm_pipe = odm_pipe->next_odm_pipe)
+ opp_cnt++;
+
+ /* Only need to validate top pipe */
+ if (pipe_ctx->top_pipe || pipe_ctx->prev_odm_pipe || !stream || !stream->timing.flags.DSC)
+ continue;
+
+ dsc_cfg.pic_width = (stream->timing.h_addressable + stream->timing.h_border_left
+ + stream->timing.h_border_right) / opp_cnt;
+ dsc_cfg.pic_height = stream->timing.v_addressable + stream->timing.v_border_top
+ + stream->timing.v_border_bottom;
+ dsc_cfg.pixel_encoding = stream->timing.pixel_encoding;
+ dsc_cfg.color_depth = stream->timing.display_color_depth;
+ dsc_cfg.is_odm = pipe_ctx->next_odm_pipe ? true : false;
+ dsc_cfg.dc_dsc_cfg = stream->timing.dsc_cfg;
+ dsc_cfg.dc_dsc_cfg.num_slices_h /= opp_cnt;
+
+ if (pipe_ctx->stream_res.dsc && !pipe_ctx->stream_res.dsc->funcs->dsc_validate_stream(pipe_ctx->stream_res.dsc, &dsc_cfg))
+ return false;
+ }
+ return true;
+}
+
+enum dc_status dml_build_mapped_resource(const struct dc *dc, struct dc_state *context, struct dc_stream_state *stream)
+{
+ enum dc_status status = DC_OK;
+ struct pipe_ctx *pipe_ctx = resource_get_head_pipe_for_stream(&context->res_ctx, stream);
+
+ if (!pipe_ctx)
+ return DC_ERROR_UNEXPECTED;
+
+
+ status = build_pipe_hw_param(pipe_ctx);
+
+ return status;
+}
+
+void dml_acquire_dsc(const struct dc *dc,
+ struct resource_context *res_ctx,
+ struct display_stream_compressor **dsc,
+ int pipe_idx)
+{
+ int i;
+ const struct resource_pool *pool = dc->res_pool;
+ struct display_stream_compressor *dsc_old = dc->current_state->res_ctx.pipe_ctx[pipe_idx].stream_res.dsc;
+
+ ASSERT(*dsc == NULL); /* If this ASSERT fails, dsc was not released properly */
+ *dsc = NULL;
+
+ /* Always do 1-to-1 mapping when number of DSCs is same as number of pipes */
+ if (pool->res_cap->num_dsc == pool->res_cap->num_opp) {
+ *dsc = pool->dscs[pipe_idx];
+ res_ctx->is_dsc_acquired[pipe_idx] = true;
+ return;
+ }
+
+ /* Return old DSC to avoid the need for redo it */
+ if (dsc_old && !res_ctx->is_dsc_acquired[dsc_old->inst]) {
+ *dsc = dsc_old;
+ res_ctx->is_dsc_acquired[dsc_old->inst] = true;
+ return ;
+ }
+
+ /* Find first free DSC */
+ for (i = 0; i < pool->res_cap->num_dsc; i++)
+ if (!res_ctx->is_dsc_acquired[i]) {
+ *dsc = pool->dscs[i];
+ res_ctx->is_dsc_acquired[i] = true;
+ break;
+ }
+}
+
+static bool dml_split_stream_for_mpc_or_odm(
+ const struct dc *dc,
+ struct resource_context *res_ctx,
+ struct pipe_ctx *pri_pipe,
+ struct pipe_ctx *sec_pipe,
+ bool odm)
+{
+ int pipe_idx = sec_pipe->pipe_idx;
+ const struct resource_pool *pool = dc->res_pool;
+
+ *sec_pipe = *pri_pipe;
+
+ sec_pipe->pipe_idx = pipe_idx;
+ sec_pipe->plane_res.mi = pool->mis[pipe_idx];
+ sec_pipe->plane_res.hubp = pool->hubps[pipe_idx];
+ sec_pipe->plane_res.ipp = pool->ipps[pipe_idx];
+ sec_pipe->plane_res.xfm = pool->transforms[pipe_idx];
+ sec_pipe->plane_res.dpp = pool->dpps[pipe_idx];
+ sec_pipe->plane_res.mpcc_inst = pool->dpps[pipe_idx]->inst;
+ sec_pipe->stream_res.dsc = NULL;
+ if (odm) {
+ if (pri_pipe->next_odm_pipe) {
+ ASSERT(pri_pipe->next_odm_pipe != sec_pipe);
+ sec_pipe->next_odm_pipe = pri_pipe->next_odm_pipe;
+ sec_pipe->next_odm_pipe->prev_odm_pipe = sec_pipe;
+ }
+ if (pri_pipe->top_pipe && pri_pipe->top_pipe->next_odm_pipe) {
+ pri_pipe->top_pipe->next_odm_pipe->bottom_pipe = sec_pipe;
+ sec_pipe->top_pipe = pri_pipe->top_pipe->next_odm_pipe;
+ }
+ if (pri_pipe->bottom_pipe && pri_pipe->bottom_pipe->next_odm_pipe) {
+ pri_pipe->bottom_pipe->next_odm_pipe->top_pipe = sec_pipe;
+ sec_pipe->bottom_pipe = pri_pipe->bottom_pipe->next_odm_pipe;
+ }
+ pri_pipe->next_odm_pipe = sec_pipe;
+ sec_pipe->prev_odm_pipe = pri_pipe;
+ ASSERT(sec_pipe->top_pipe == NULL);
+
+ if (!sec_pipe->top_pipe)
+ sec_pipe->stream_res.opp = pool->opps[pipe_idx];
+ else
+ sec_pipe->stream_res.opp = sec_pipe->top_pipe->stream_res.opp;
+ if (sec_pipe->stream->timing.flags.DSC == 1) {
+ dml_acquire_dsc(dc, res_ctx, &sec_pipe->stream_res.dsc, pipe_idx);
+ ASSERT(sec_pipe->stream_res.dsc);
+ if (sec_pipe->stream_res.dsc == NULL)
+ return false;
+ }
+ } else {
+ if (pri_pipe->bottom_pipe) {
+ ASSERT(pri_pipe->bottom_pipe != sec_pipe);
+ sec_pipe->bottom_pipe = pri_pipe->bottom_pipe;
+ sec_pipe->bottom_pipe->top_pipe = sec_pipe;
+ }
+ pri_pipe->bottom_pipe = sec_pipe;
+ sec_pipe->top_pipe = pri_pipe;
+
+ ASSERT(pri_pipe->plane_state);
+ }
+
+ return true;
+}
+
+static struct pipe_ctx *dml_find_split_pipe(
+ struct dc *dc,
+ struct dc_state *context,
+ int old_index)
+{
+ struct pipe_ctx *pipe = NULL;
+ int i;
+
+ if (old_index >= 0 && context->res_ctx.pipe_ctx[old_index].stream == NULL) {
+ pipe = &context->res_ctx.pipe_ctx[old_index];
+ pipe->pipe_idx = old_index;
+ }
+
+ if (!pipe)
+ for (i = dc->res_pool->pipe_count - 1; i >= 0; i--) {
+ if (dc->current_state->res_ctx.pipe_ctx[i].top_pipe == NULL
+ && dc->current_state->res_ctx.pipe_ctx[i].prev_odm_pipe == NULL) {
+ if (context->res_ctx.pipe_ctx[i].stream == NULL) {
+ pipe = &context->res_ctx.pipe_ctx[i];
+ pipe->pipe_idx = i;
+ break;
+ }
+ }
+ }
+
+ /*
+ * May need to fix pipes getting tossed from 1 opp to another on flip
+ * Add for debugging transient underflow during topology updates:
+ * ASSERT(pipe);
+ */
+ if (!pipe)
+ for (i = dc->res_pool->pipe_count - 1; i >= 0; i--) {
+ if (context->res_ctx.pipe_ctx[i].stream == NULL) {
+ pipe = &context->res_ctx.pipe_ctx[i];
+ pipe->pipe_idx = i;
+ break;
+ }
+ }
+
+ return pipe;
+}
+
+static void dml_release_dsc(struct resource_context *res_ctx,
+ const struct resource_pool *pool,
+ struct display_stream_compressor **dsc)
+{
+ int i;
+
+ for (i = 0; i < pool->res_cap->num_dsc; i++)
+ if (pool->dscs[i] == *dsc) {
+ res_ctx->is_dsc_acquired[i] = false;
+ *dsc = NULL;
+ break;
+ }
+}
+
+static int dml_get_num_mpc_splits(struct pipe_ctx *pipe)
+{
+ int mpc_split_count = 0;
+ struct pipe_ctx *other_pipe = pipe->bottom_pipe;
+
+ while (other_pipe && other_pipe->plane_state == pipe->plane_state) {
+ mpc_split_count++;
+ other_pipe = other_pipe->bottom_pipe;
+ }
+ other_pipe = pipe->top_pipe;
+ while (other_pipe && other_pipe->plane_state == pipe->plane_state) {
+ mpc_split_count++;
+ other_pipe = other_pipe->top_pipe;
+ }
+
+ return mpc_split_count;
+}
+
+static bool dml_enough_pipes_for_subvp(struct dc *dc,
+ struct dc_state *context)
+{
+ int i = 0;
+ int num_pipes = 0;
+
+ for (i = 0; i < dc->res_pool->pipe_count; i++) {
+ struct pipe_ctx *pipe = &context->res_ctx.pipe_ctx[i];
+
+ if (pipe->stream && pipe->plane_state)
+ num_pipes++;
+ }
+
+ // Sub-VP only possible if the number of "real" pipes is
+ // less than or equal to half the number of available pipes
+ if (num_pipes * 2 > dc->res_pool->pipe_count)
+ return false;
+
+ return true;
+}
+
+static int dml_validate_apply_pipe_split_flags(
+ struct dc *dc,
+ struct dc_state *context,
+ int vlevel,
+ int *split,
+ bool *merge)
+{
+ int i, pipe_idx, vlevel_split;
+ int plane_count = 0;
+ bool force_split = false;
+ bool avoid_split = dc->debug.pipe_split_policy == MPC_SPLIT_AVOID;
+ struct vba_vars_st *v = &context->bw_ctx.dml.vba;
+ int max_mpc_comb = v->maxMpcComb;
+
+ if (context->stream_count > 1) {
+ if (dc->debug.pipe_split_policy == MPC_SPLIT_AVOID_MULT_DISP)
+ avoid_split = true;
+ } else if (dc->debug.force_single_disp_pipe_split)
+ force_split = true;
+
+ for (i = 0; i < dc->res_pool->pipe_count; i++) {
+ struct pipe_ctx *pipe = &context->res_ctx.pipe_ctx[i];
+
+ /**
+ * Workaround for avoiding pipe-split in cases where we'd split
+ * planes that are too small, resulting in splits that aren't
+ * valid for the scaler.
+ */
+ if (pipe->plane_state &&
+ (pipe->plane_state->dst_rect.width <= 16 ||
+ pipe->plane_state->dst_rect.height <= 16 ||
+ pipe->plane_state->src_rect.width <= 16 ||
+ pipe->plane_state->src_rect.height <= 16))
+ avoid_split = true;
+
+ /* TODO: fix dc bugs and remove this split threshold thing */
+ if (pipe->stream && !pipe->prev_odm_pipe &&
+ (!pipe->top_pipe || pipe->top_pipe->plane_state != pipe->plane_state))
+ ++plane_count;
+ }
+ if (plane_count > dc->res_pool->pipe_count / 2)
+ avoid_split = true;
+
+ /* W/A: Mode timing with borders may not work well with pipe split, avoid for this corner case */
+ for (i = 0; i < dc->res_pool->pipe_count; i++) {
+ struct pipe_ctx *pipe = &context->res_ctx.pipe_ctx[i];
+ struct dc_crtc_timing timing;
+
+ if (!pipe->stream)
+ continue;
+ else {
+ timing = pipe->stream->timing;
+ if (timing.h_border_left + timing.h_border_right
+ + timing.v_border_top + timing.v_border_bottom > 0) {
+ avoid_split = true;
+ break;
+ }
+ }
+ }
+
+ /* Avoid split loop looks for lowest voltage level that allows most unsplit pipes possible */
+ if (avoid_split) {
+ for (i = 0, pipe_idx = 0; i < dc->res_pool->pipe_count; i++) {
+ if (!context->res_ctx.pipe_ctx[i].stream)
+ continue;
+
+ for (vlevel_split = vlevel; vlevel <= context->bw_ctx.dml.soc.num_states; vlevel++)
+ if (v->NoOfDPP[vlevel][0][pipe_idx] == 1 &&
+ v->ModeSupport[vlevel][0])
+ break;
+ /* Impossible to not split this pipe */
+ if (vlevel > context->bw_ctx.dml.soc.num_states)
+ vlevel = vlevel_split;
+ else
+ max_mpc_comb = 0;
+ pipe_idx++;
+ }
+ v->maxMpcComb = max_mpc_comb;
+ }
+
+ /* Split loop sets which pipe should be split based on dml outputs and dc flags */
+ for (i = 0, pipe_idx = 0; i < dc->res_pool->pipe_count; i++) {
+ struct pipe_ctx *pipe = &context->res_ctx.pipe_ctx[i];
+ int pipe_plane = v->pipe_plane[pipe_idx];
+ bool split4mpc = context->stream_count == 1 && plane_count == 1
+ && dc->config.enable_4to1MPC && dc->res_pool->pipe_count >= 4;
+
+ if (!context->res_ctx.pipe_ctx[i].stream)
+ continue;
+
+ if (split4mpc || v->NoOfDPP[vlevel][max_mpc_comb][pipe_plane] == 4)
+ split[i] = 4;
+ else if (force_split || v->NoOfDPP[vlevel][max_mpc_comb][pipe_plane] == 2)
+ split[i] = 2;
+
+ if ((pipe->stream->view_format ==
+ VIEW_3D_FORMAT_SIDE_BY_SIDE ||
+ pipe->stream->view_format ==
+ VIEW_3D_FORMAT_TOP_AND_BOTTOM) &&
+ (pipe->stream->timing.timing_3d_format ==
+ TIMING_3D_FORMAT_TOP_AND_BOTTOM ||
+ pipe->stream->timing.timing_3d_format ==
+ TIMING_3D_FORMAT_SIDE_BY_SIDE))
+ split[i] = 2;
+ if (dc->debug.force_odm_combine & (1 << pipe->stream_res.tg->inst)) {
+ split[i] = 2;
+ v->ODMCombineEnablePerState[vlevel][pipe_plane] = dm_odm_combine_mode_2to1;
+ }
+ if (dc->debug.force_odm_combine_4to1 & (1 << pipe->stream_res.tg->inst)) {
+ split[i] = 4;
+ v->ODMCombineEnablePerState[vlevel][pipe_plane] = dm_odm_combine_mode_4to1;
+ }
+ /*420 format workaround*/
+ if (pipe->stream->timing.h_addressable > 7680 &&
+ pipe->stream->timing.pixel_encoding == PIXEL_ENCODING_YCBCR420) {
+ split[i] = 4;
+ }
+
+ v->ODMCombineEnabled[pipe_plane] =
+ v->ODMCombineEnablePerState[vlevel][pipe_plane];
+
+ if (v->ODMCombineEnabled[pipe_plane] == dm_odm_combine_mode_disabled) {
+ if (dml_get_num_mpc_splits(pipe) == 1) {
+ /*If need split for mpc but 2 way split already*/
+ if (split[i] == 4)
+ split[i] = 2; /* 2 -> 4 MPC */
+ else if (split[i] == 2)
+ split[i] = 0; /* 2 -> 2 MPC */
+ else if (pipe->top_pipe && pipe->top_pipe->plane_state == pipe->plane_state)
+ merge[i] = true; /* 2 -> 1 MPC */
+ } else if (dml_get_num_mpc_splits(pipe) == 3) {
+ /*If need split for mpc but 4 way split already*/
+ if (split[i] == 2 && ((pipe->top_pipe && !pipe->top_pipe->top_pipe)
+ || !pipe->bottom_pipe)) {
+ merge[i] = true; /* 4 -> 2 MPC */
+ } else if (split[i] == 0 && pipe->top_pipe &&
+ pipe->top_pipe->plane_state == pipe->plane_state)
+ merge[i] = true; /* 4 -> 1 MPC */
+ split[i] = 0;
+ } else if (dml_get_num_mpc_splits(pipe)) {
+ /* ODM -> MPC transition */
+ if (pipe->prev_odm_pipe) {
+ split[i] = 0;
+ merge[i] = true;
+ }
+ }
+ } else {
+ if (dml_get_num_mpc_splits(pipe) == 1) {
+ /*If need split for odm but 2 way split already*/
+ if (split[i] == 4)
+ split[i] = 2; /* 2 -> 4 ODM */
+ else if (split[i] == 2)
+ split[i] = 0; /* 2 -> 2 ODM */
+ else if (pipe->prev_odm_pipe) {
+ ASSERT(0); /* NOT expected yet */
+ merge[i] = true; /* exit ODM */
+ }
+ } else if (dml_get_num_mpc_splits(pipe) == 3) {
+ /*If need split for odm but 4 way split already*/
+ if (split[i] == 2 && ((pipe->prev_odm_pipe && !pipe->prev_odm_pipe->prev_odm_pipe)
+ || !pipe->next_odm_pipe)) {
+ ASSERT(0); /* NOT expected yet */
+ merge[i] = true; /* 4 -> 2 ODM */
+ } else if (split[i] == 0 && pipe->prev_odm_pipe) {
+ ASSERT(0); /* NOT expected yet */
+ merge[i] = true; /* exit ODM */
+ }
+ split[i] = 0;
+ } else if (dml_get_num_mpc_splits(pipe)) {
+ /* MPC -> ODM transition */
+ ASSERT(0); /* NOT expected yet */
+ if (pipe->top_pipe && pipe->top_pipe->plane_state == pipe->plane_state) {
+ split[i] = 0;
+ merge[i] = true;
+ }
+ }
+ }
+
+ /* Adjust dppclk when split is forced, do not bother with dispclk */
+ if (split[i] != 0 && v->NoOfDPP[vlevel][max_mpc_comb][pipe_idx] == 1)
+ v->RequiredDPPCLK[vlevel][max_mpc_comb][pipe_idx] /= 2;
+ pipe_idx++;
+ }
+
+ return vlevel;
+}
+
+static void dml_set_phantom_stream_timing(struct dc *dc,
+ struct dc_state *context,
+ struct pipe_ctx *ref_pipe,
+ struct dc_stream_state *phantom_stream)
+{
+ // phantom_vactive = blackout (latency + margin) + fw_processing_delays + pstate allow width
+ uint32_t phantom_vactive_us = context->bw_ctx.dml.soc.dram_clock_change_latency_us + 60 +
+ dc->caps.subvp_fw_processing_delay_us +
+ dc->caps.subvp_pstate_allow_width_us;
+ uint32_t phantom_vactive = ((double)phantom_vactive_us/1000000) *
+ (ref_pipe->stream->timing.pix_clk_100hz * 100) /
+ (double)ref_pipe->stream->timing.h_total;
+ uint32_t phantom_bp = ref_pipe->pipe_dlg_param.vstartup_start;
+
+ phantom_stream->dst.y = 0;
+ phantom_stream->dst.height = phantom_vactive;
+ phantom_stream->src.y = 0;
+ phantom_stream->src.height = phantom_vactive;
+
+ phantom_stream->timing.v_addressable = phantom_vactive;
+ phantom_stream->timing.v_front_porch = 1;
+ phantom_stream->timing.v_total = phantom_stream->timing.v_addressable +
+ phantom_stream->timing.v_front_porch +
+ phantom_stream->timing.v_sync_width +
+ phantom_bp;
+}
+
+static struct dc_stream_state *dml_enable_phantom_stream(struct dc *dc,
+ struct dc_state *context,
+ struct pipe_ctx *ref_pipe)
+{
+ struct dc_stream_state *phantom_stream = NULL;
+
+ phantom_stream = dc_create_stream_for_sink(ref_pipe->stream->sink);
+ phantom_stream->signal = SIGNAL_TYPE_VIRTUAL;
+ phantom_stream->dpms_off = true;
+ phantom_stream->mall_stream_config.type = SUBVP_PHANTOM;
+ phantom_stream->mall_stream_config.paired_stream = ref_pipe->stream;
+ ref_pipe->stream->mall_stream_config.type = SUBVP_MAIN;
+ ref_pipe->stream->mall_stream_config.paired_stream = phantom_stream;
+
+ /* stream has limited viewport and small timing */
+ memcpy(&phantom_stream->timing, &ref_pipe->stream->timing, sizeof(phantom_stream->timing));
+ memcpy(&phantom_stream->src, &ref_pipe->stream->src, sizeof(phantom_stream->src));
+ memcpy(&phantom_stream->dst, &ref_pipe->stream->dst, sizeof(phantom_stream->dst));
+ dml_set_phantom_stream_timing(dc, context, ref_pipe, phantom_stream);
+
+ dc_add_stream_to_ctx(dc, context, phantom_stream);
+ dc->hwss.apply_ctx_to_hw(dc, context);
+ return phantom_stream;
+}
+
+static void dml_enable_phantom_plane(struct dc *dc,
+ struct dc_state *context,
+ struct dc_stream_state *phantom_stream,
+ struct pipe_ctx *main_pipe)
+{
+ struct dc_plane_state *phantom_plane = NULL;
+ struct dc_plane_state *prev_phantom_plane = NULL;
+ struct pipe_ctx *curr_pipe = main_pipe;
+
+ while (curr_pipe) {
+ if (curr_pipe->top_pipe && curr_pipe->top_pipe->plane_state == curr_pipe->plane_state)
+ phantom_plane = prev_phantom_plane;
+ else
+ phantom_plane = dc_create_plane_state(dc);
+
+ memcpy(&phantom_plane->address, &curr_pipe->plane_state->address, sizeof(phantom_plane->address));
+ memcpy(&phantom_plane->scaling_quality, &curr_pipe->plane_state->scaling_quality,
+ sizeof(phantom_plane->scaling_quality));
+ memcpy(&phantom_plane->src_rect, &curr_pipe->plane_state->src_rect, sizeof(phantom_plane->src_rect));
+ memcpy(&phantom_plane->dst_rect, &curr_pipe->plane_state->dst_rect, sizeof(phantom_plane->dst_rect));
+ memcpy(&phantom_plane->clip_rect, &curr_pipe->plane_state->clip_rect, sizeof(phantom_plane->clip_rect));
+ memcpy(&phantom_plane->plane_size, &curr_pipe->plane_state->plane_size,
+ sizeof(phantom_plane->plane_size));
+ memcpy(&phantom_plane->tiling_info, &curr_pipe->plane_state->tiling_info,
+ sizeof(phantom_plane->tiling_info));
+ memcpy(&phantom_plane->dcc, &curr_pipe->plane_state->dcc, sizeof(phantom_plane->dcc));
+ /* Currently compat_level is undefined in dc_state
+ * phantom_plane->compat_level = curr_pipe->plane_state->compat_level;
+ */
+ phantom_plane->format = curr_pipe->plane_state->format;
+ phantom_plane->rotation = curr_pipe->plane_state->rotation;
+ phantom_plane->visible = curr_pipe->plane_state->visible;
+
+ /* Shadow pipe has small viewport. */
+ phantom_plane->clip_rect.y = 0;
+ phantom_plane->clip_rect.height = phantom_stream->timing.v_addressable;
+
+ dc_add_plane_to_context(dc, phantom_stream, phantom_plane, context);
+
+ curr_pipe = curr_pipe->bottom_pipe;
+ prev_phantom_plane = phantom_plane;
+ }
+}
+
+static void dml_add_phantom_pipes(struct dc *dc, struct dc_state *context)
+{
+ int i = 0;
+
+ for (i = 0; i < dc->res_pool->pipe_count; i++) {
+ struct pipe_ctx *pipe = &context->res_ctx.pipe_ctx[i];
+ struct dc_stream_state *ref_stream = pipe->stream;
+ // Only construct phantom stream for top pipes that have plane enabled
+ if (!pipe->top_pipe && pipe->plane_state && pipe->stream &&
+ pipe->stream->mall_stream_config.type == SUBVP_NONE) {
+ struct dc_stream_state *phantom_stream = NULL;
+
+ phantom_stream = dml_enable_phantom_stream(dc, context, pipe);
+ dml_enable_phantom_plane(dc, context, phantom_stream, pipe);
+ }
+ }
+
+ for (i = 0; i < dc->res_pool->pipe_count; i++) {
+ struct pipe_ctx *pipe = &context->res_ctx.pipe_ctx[i];
+
+ if (pipe->plane_state && pipe->stream &&
+ pipe->stream->mall_stream_config.type == SUBVP_PHANTOM) {
+ pipe->stream->use_dynamic_meta = false;
+ pipe->plane_state->flip_immediate = false;
+ if (!resource_build_scaling_params(pipe)) {
+ // Log / remove phantom pipes since failed to build scaling params
+ }
+ }
+ }
+}
+
+static void dml_remove_phantom_pipes(struct dc *dc, struct dc_state *context)
+{
+ int i;
+ bool removed_pipe = false;
+
+ for (i = 0; i < dc->res_pool->pipe_count; i++) {
+ struct pipe_ctx *pipe = &context->res_ctx.pipe_ctx[i];
+ // build scaling params for phantom pipes
+ if (pipe->plane_state && pipe->stream && pipe->stream->mall_stream_config.type == SUBVP_PHANTOM) {
+ dc_rem_all_planes_for_stream(dc, pipe->stream, context);
+ dc_remove_stream_from_ctx(dc, context, pipe->stream);
+ removed_pipe = true;
+ }
+
+ // Clear all phantom stream info
+ if (pipe->stream) {
+ pipe->stream->mall_stream_config.type = SUBVP_NONE;
+ pipe->stream->mall_stream_config.paired_stream = NULL;
+ }
+ }
+ if (removed_pipe)
+ dc->hwss.apply_ctx_to_hw(dc, context);
+}
+
+/*
+ * If the input state contains no upstream planes for a particular pipe (i.e. only timing)
+ * we need to populate some "conservative" plane information as DML cannot handle "no planes"
+ */
+static void populate_default_plane_from_timing(const struct dc_crtc_timing *timing, struct _vcs_dpi_display_pipe_params_st *pipe)
+{
+ pipe->src.is_hsplit = pipe->dest.odm_combine != dm_odm_combine_mode_disabled;
+ pipe->src.source_scan = dm_horz;
+ pipe->src.sw_mode = dm_sw_4kb_s;
+ pipe->src.macro_tile_size = dm_64k_tile;
+ pipe->src.viewport_width = timing->h_addressable;
+ if (pipe->src.viewport_width > 1920)
+ pipe->src.viewport_width = 1920;
+ pipe->src.viewport_height = timing->v_addressable;
+ if (pipe->src.viewport_height > 1080)
+ pipe->src.viewport_height = 1080;
+ pipe->src.surface_height_y = pipe->src.viewport_height;
+ pipe->src.surface_width_y = pipe->src.viewport_width;
+ pipe->src.surface_height_c = pipe->src.viewport_height;
+ pipe->src.surface_width_c = pipe->src.viewport_width;
+ pipe->src.data_pitch = ((pipe->src.viewport_width + 255) / 256) * 256;
+ pipe->src.source_format = dm_444_32;
+ pipe->dest.recout_width = pipe->src.viewport_width;
+ pipe->dest.recout_height = pipe->src.viewport_height;
+ pipe->dest.full_recout_width = pipe->dest.recout_width;
+ pipe->dest.full_recout_height = pipe->dest.recout_height;
+ pipe->scale_ratio_depth.lb_depth = dm_lb_16;
+ pipe->scale_ratio_depth.hscl_ratio = 1.0;
+ pipe->scale_ratio_depth.vscl_ratio = 1.0;
+ pipe->scale_ratio_depth.scl_enable = 0;
+ pipe->scale_taps.htaps = 1;
+ pipe->scale_taps.vtaps = 1;
+ pipe->dest.vtotal_min = timing->v_total;
+ pipe->dest.vtotal_max = timing->v_total;
+
+ if (pipe->dest.odm_combine == dm_odm_combine_mode_2to1) {
+ pipe->src.viewport_width /= 2;
+ pipe->dest.recout_width /= 2;
+ } else if (pipe->dest.odm_combine == dm_odm_combine_mode_4to1) {
+ pipe->src.viewport_width /= 4;
+ pipe->dest.recout_width /= 4;
+ }
+
+ pipe->src.dcc = false;
+ pipe->src.dcc_rate = 1;
+}
+
+/*
+ * If the pipe is not blending (i.e. pipe_ctx->top pipe == null) then its
+ * hsplit group is equal to its own pipe ID
+ * Otherwise, all pipes part of the same blending tree have the same hsplit group
+ * ID as the top most pipe
+ *
+ * If the pipe ctx is ODM combined, then similar logic follows
+ */
+static void populate_hsplit_group_from_dc_pipe_ctx (const struct pipe_ctx *dc_pipe_ctx, struct _vcs_dpi_display_e2e_pipe_params_st *e2e_pipe)
+{
+ e2e_pipe->pipe.src.hsplit_grp = dc_pipe_ctx->pipe_idx;
+
+ if (dc_pipe_ctx->top_pipe && dc_pipe_ctx->top_pipe->plane_state
+ == dc_pipe_ctx->plane_state) {
+ struct pipe_ctx *first_pipe = dc_pipe_ctx->top_pipe;
+ int split_idx = 0;
+
+ while (first_pipe->top_pipe && first_pipe->top_pipe->plane_state
+ == dc_pipe_ctx->plane_state) {
+ first_pipe = first_pipe->top_pipe;
+ split_idx++;
+ }
+
+ /* Treat 4to1 mpc combine as an mpo of 2 2-to-1 combines */
+ if (split_idx == 0)
+ e2e_pipe->pipe.src.hsplit_grp = first_pipe->pipe_idx;
+ else if (split_idx == 1)
+ e2e_pipe->pipe.src.hsplit_grp = dc_pipe_ctx->pipe_idx;
+ else if (split_idx == 2)
+ e2e_pipe->pipe.src.hsplit_grp = dc_pipe_ctx->top_pipe->pipe_idx;
+
+ } else if (dc_pipe_ctx->prev_odm_pipe) {
+ struct pipe_ctx *first_pipe = dc_pipe_ctx->prev_odm_pipe;
+
+ while (first_pipe->prev_odm_pipe)
+ first_pipe = first_pipe->prev_odm_pipe;
+ e2e_pipe->pipe.src.hsplit_grp = first_pipe->pipe_idx;
+ }
+}
+
+static void populate_dml_from_dc_pipe_ctx (const struct pipe_ctx *dc_pipe_ctx, struct _vcs_dpi_display_e2e_pipe_params_st *e2e_pipe, int always_scale)
+{
+ const struct dc_plane_state *pln = dc_pipe_ctx->plane_state;
+ const struct scaler_data *scl = &dc_pipe_ctx->plane_res.scl_data;
+
+ e2e_pipe->pipe.src.immediate_flip = pln->flip_immediate;
+ e2e_pipe->pipe.src.is_hsplit = (dc_pipe_ctx->bottom_pipe && dc_pipe_ctx->bottom_pipe->plane_state == pln)
+ || (dc_pipe_ctx->top_pipe && dc_pipe_ctx->top_pipe->plane_state == pln)
+ || e2e_pipe->pipe.dest.odm_combine != dm_odm_combine_mode_disabled;
+
+ /* stereo is not split */
+ if (pln->stereo_format == PLANE_STEREO_FORMAT_SIDE_BY_SIDE ||
+ pln->stereo_format == PLANE_STEREO_FORMAT_TOP_AND_BOTTOM) {
+ e2e_pipe->pipe.src.is_hsplit = false;
+ e2e_pipe->pipe.src.hsplit_grp = dc_pipe_ctx->pipe_idx;
+ }
+
+ e2e_pipe->pipe.src.source_scan = pln->rotation == ROTATION_ANGLE_90
+ || pln->rotation == ROTATION_ANGLE_270 ? dm_vert : dm_horz;
+ e2e_pipe->pipe.src.viewport_y_y = scl->viewport.y;
+ e2e_pipe->pipe.src.viewport_y_c = scl->viewport_c.y;
+ e2e_pipe->pipe.src.viewport_width = scl->viewport.width;
+ e2e_pipe->pipe.src.viewport_width_c = scl->viewport_c.width;
+ e2e_pipe->pipe.src.viewport_height = scl->viewport.height;
+ e2e_pipe->pipe.src.viewport_height_c = scl->viewport_c.height;
+ e2e_pipe->pipe.src.viewport_width_max = pln->src_rect.width;
+ e2e_pipe->pipe.src.viewport_height_max = pln->src_rect.height;
+ e2e_pipe->pipe.src.surface_width_y = pln->plane_size.surface_size.width;
+ e2e_pipe->pipe.src.surface_height_y = pln->plane_size.surface_size.height;
+ e2e_pipe->pipe.src.surface_width_c = pln->plane_size.chroma_size.width;
+ e2e_pipe->pipe.src.surface_height_c = pln->plane_size.chroma_size.height;
+
+ if (pln->format == SURFACE_PIXEL_FORMAT_GRPH_RGBE_ALPHA
+ || pln->format >= SURFACE_PIXEL_FORMAT_VIDEO_BEGIN) {
+ e2e_pipe->pipe.src.data_pitch = pln->plane_size.surface_pitch;
+ e2e_pipe->pipe.src.data_pitch_c = pln->plane_size.chroma_pitch;
+ e2e_pipe->pipe.src.meta_pitch = pln->dcc.meta_pitch;
+ e2e_pipe->pipe.src.meta_pitch_c = pln->dcc.meta_pitch_c;
+ } else {
+ e2e_pipe->pipe.src.data_pitch = pln->plane_size.surface_pitch;
+ e2e_pipe->pipe.src.meta_pitch = pln->dcc.meta_pitch;
+ }
+ e2e_pipe->pipe.src.dcc = pln->dcc.enable;
+ e2e_pipe->pipe.src.dcc_rate = 1;
+ e2e_pipe->pipe.dest.recout_width = scl->recout.width;
+ e2e_pipe->pipe.dest.recout_height = scl->recout.height;
+ e2e_pipe->pipe.dest.full_recout_height = scl->recout.height;
+ e2e_pipe->pipe.dest.full_recout_width = scl->recout.width;
+ if (e2e_pipe->pipe.dest.odm_combine == dm_odm_combine_mode_2to1)
+ e2e_pipe->pipe.dest.full_recout_width *= 2;
+ else if (e2e_pipe->pipe.dest.odm_combine == dm_odm_combine_mode_4to1)
+ e2e_pipe->pipe.dest.full_recout_width *= 4;
+ else {
+ struct pipe_ctx *split_pipe = dc_pipe_ctx->bottom_pipe;
+
+ while (split_pipe && split_pipe->plane_state == pln) {
+ e2e_pipe->pipe.dest.full_recout_width += split_pipe->plane_res.scl_data.recout.width;
+ split_pipe = split_pipe->bottom_pipe;
+ }
+ split_pipe = dc_pipe_ctx->top_pipe;
+ while (split_pipe && split_pipe->plane_state == pln) {
+ e2e_pipe->pipe.dest.full_recout_width += split_pipe->plane_res.scl_data.recout.width;
+ split_pipe = split_pipe->top_pipe;
+ }
+ }
+
+ e2e_pipe->pipe.scale_ratio_depth.lb_depth = dm_lb_16;
+ e2e_pipe->pipe.scale_ratio_depth.hscl_ratio = (double) scl->ratios.horz.value / (1ULL<<32);
+ e2e_pipe->pipe.scale_ratio_depth.hscl_ratio_c = (double) scl->ratios.horz_c.value / (1ULL<<32);
+ e2e_pipe->pipe.scale_ratio_depth.vscl_ratio = (double) scl->ratios.vert.value / (1ULL<<32);
+ e2e_pipe->pipe.scale_ratio_depth.vscl_ratio_c = (double) scl->ratios.vert_c.value / (1ULL<<32);
+ e2e_pipe->pipe.scale_ratio_depth.scl_enable =
+ scl->ratios.vert.value != dc_fixpt_one.value
+ || scl->ratios.horz.value != dc_fixpt_one.value
+ || scl->ratios.vert_c.value != dc_fixpt_one.value
+ || scl->ratios.horz_c.value != dc_fixpt_one.value /*Lb only or Full scl*/
+ || always_scale; /*support always scale*/
+ e2e_pipe->pipe.scale_taps.htaps = scl->taps.h_taps;
+ e2e_pipe->pipe.scale_taps.htaps_c = scl->taps.h_taps_c;
+ e2e_pipe->pipe.scale_taps.vtaps = scl->taps.v_taps;
+ e2e_pipe->pipe.scale_taps.vtaps_c = scl->taps.v_taps_c;
+
+ /* Currently compat_level is not defined. Commenting it until further resolution
+ * if (pln->compat_level == DC_LEGACY_TILING_ADDR_GEN_TWO) {
+ swizzle_to_dml_params(pln->tiling_info.gfx9.swizzle,
+ &e2e_pipe->pipe.src.sw_mode);
+ e2e_pipe->pipe.src.macro_tile_size =
+ swizzle_mode_to_macro_tile_size(pln->tiling_info.gfx9.swizzle);
+ } else {
+ gfx10array_mode_to_dml_params(pln->tiling_info.gfx10compatible.array_mode,
+ pln->compat_level,
+ &e2e_pipe->pipe.src.sw_mode);
+ e2e_pipe->pipe.src.macro_tile_size = dm_4k_tile;
+ }*/
+
+ e2e_pipe->pipe.src.source_format = dc_source_format_to_dml_source_format(pln->format);
+}
+
+static void populate_dml_cursor_parameters_from_dc_pipe_ctx (const struct pipe_ctx *dc_pipe_ctx, struct _vcs_dpi_display_e2e_pipe_params_st *e2e_pipe)
+{
+ /*
+ * For graphic plane, cursor number is 1, nv12 is 0
+ * bw calculations due to cursor on/off
+ */
+ if (dc_pipe_ctx->plane_state &&
+ (dc_pipe_ctx->plane_state->address.type == PLN_ADDR_TYPE_VIDEO_PROGRESSIVE ||
+ dc_pipe_ctx->stream->mall_stream_config.type == SUBVP_PHANTOM))
+ e2e_pipe->pipe.src.num_cursors = 0;
+ else
+ e2e_pipe->pipe.src.num_cursors = 1;
+
+ e2e_pipe->pipe.src.cur0_src_width = 256;
+ e2e_pipe->pipe.src.cur0_bpp = dm_cur_32bit;
+}
+
+static int populate_dml_pipes_from_context_base(
+ struct dc *dc,
+ struct dc_state *context,
+ display_e2e_pipe_params_st *pipes,
+ bool fast_validate)
+{
+ int pipe_cnt, i;
+ bool synchronized_vblank = true;
+ struct resource_context *res_ctx = &context->res_ctx;
+
+ for (i = 0, pipe_cnt = -1; i < dc->res_pool->pipe_count; i++) {
+ if (!res_ctx->pipe_ctx[i].stream)
+ continue;
+
+ if (pipe_cnt < 0) {
+ pipe_cnt = i;
+ continue;
+ }
+
+ if (res_ctx->pipe_ctx[pipe_cnt].stream == res_ctx->pipe_ctx[i].stream)
+ continue;
+
+ if (dc->debug.disable_timing_sync ||
+ (!resource_are_streams_timing_synchronizable(
+ res_ctx->pipe_ctx[pipe_cnt].stream,
+ res_ctx->pipe_ctx[i].stream) &&
+ !resource_are_vblanks_synchronizable(
+ res_ctx->pipe_ctx[pipe_cnt].stream,
+ res_ctx->pipe_ctx[i].stream))) {
+ synchronized_vblank = false;
+ break;
+ }
+ }
+
+ for (i = 0, pipe_cnt = 0; i < dc->res_pool->pipe_count; i++) {
+ struct dc_crtc_timing *timing = &res_ctx->pipe_ctx[i].stream->timing;
+
+ struct audio_check aud_check = {0};
+ if (!res_ctx->pipe_ctx[i].stream)
+ continue;
+
+ /* todo:
+ pipes[pipe_cnt].pipe.src.dynamic_metadata_enable = 0;
+ pipes[pipe_cnt].pipe.src.dcc = 0;
+ pipes[pipe_cnt].pipe.src.vm = 0;*/
+
+ pipes[pipe_cnt].clks_cfg.refclk_mhz = dc->res_pool->ref_clocks.dchub_ref_clock_inKhz / 1000.0;
+
+ pipes[pipe_cnt].dout.dsc_enable = res_ctx->pipe_ctx[i].stream->timing.flags.DSC;
+ /* todo: rotation?*/
+ pipes[pipe_cnt].dout.dsc_slices = res_ctx->pipe_ctx[i].stream->timing.dsc_cfg.num_slices_h;
+ if (res_ctx->pipe_ctx[i].stream->use_dynamic_meta) {
+ pipes[pipe_cnt].pipe.src.dynamic_metadata_enable = true;
+ /* 1/2 vblank */
+ pipes[pipe_cnt].pipe.src.dynamic_metadata_lines_before_active =
+ (timing->v_total - timing->v_addressable
+ - timing->v_border_top - timing->v_border_bottom) / 2;
+ /* 36 bytes dp, 32 hdmi */
+ pipes[pipe_cnt].pipe.src.dynamic_metadata_xmit_bytes =
+ dc_is_dp_signal(res_ctx->pipe_ctx[i].stream->signal) ? 36 : 32;
+ }
+ pipes[pipe_cnt].pipe.dest.synchronized_vblank_all_planes = synchronized_vblank;
+
+ dc_timing_to_dml_timing(timing, &pipes[pipe_cnt].pipe.dest);
+ pipes[pipe_cnt].pipe.dest.vtotal_min = res_ctx->pipe_ctx[i].stream->adjust.v_total_min;
+ pipes[pipe_cnt].pipe.dest.vtotal_max = res_ctx->pipe_ctx[i].stream->adjust.v_total_max;
+
+ pipes[pipe_cnt].pipe.dest.otg_inst = res_ctx->pipe_ctx[i].stream_res.tg->inst;
+
+ pipes[pipe_cnt].pipe.dest.odm_combine = get_dml_odm_combine(&res_ctx->pipe_ctx[i]);
+
+ populate_hsplit_group_from_dc_pipe_ctx(&res_ctx->pipe_ctx[i], &pipes[pipe_cnt]);
+
+ pipes[pipe_cnt].dout.dp_lanes = 4;
+ pipes[pipe_cnt].dout.is_virtual = 0;
+ pipes[pipe_cnt].dout.output_type = get_dml_output_type(res_ctx->pipe_ctx[i].stream->signal);
+ if (pipes[pipe_cnt].dout.output_type < 0) {
+ pipes[pipe_cnt].dout.output_type = dm_dp;
+ pipes[pipe_cnt].dout.is_virtual = 1;
+ }
+
+ populate_color_depth_and_encoding_from_timing(&res_ctx->pipe_ctx[i].stream->timing, &pipes[pipe_cnt].dout);
+
+ if (res_ctx->pipe_ctx[i].stream->timing.flags.DSC)
+ pipes[pipe_cnt].dout.output_bpp = res_ctx->pipe_ctx[i].stream->timing.dsc_cfg.bits_per_pixel / 16.0;
+
+ /* todo: default max for now, until there is logic reflecting this in dc*/
+ pipes[pipe_cnt].dout.dsc_input_bpc = 12;
+ /*fill up the audio sample rate (unit in kHz)*/
+ get_audio_check(&res_ctx->pipe_ctx[i].stream->audio_info, &aud_check);
+ pipes[pipe_cnt].dout.max_audio_sample_rate = aud_check.max_audiosample_rate / 1000;
+
+ populate_dml_cursor_parameters_from_dc_pipe_ctx(&res_ctx->pipe_ctx[i], &pipes[pipe_cnt]);
+
+ if (!res_ctx->pipe_ctx[i].plane_state) {
+ populate_default_plane_from_timing(timing, &pipes[pipe_cnt].pipe);
+ } else {
+ populate_dml_from_dc_pipe_ctx(&res_ctx->pipe_ctx[i], &pipes[pipe_cnt], dc->debug.always_scale);
+ }
+
+ pipe_cnt++;
+ }
+
+ /* populate writeback information */
+ if (dc->res_pool)
+ dc->res_pool->funcs->populate_dml_writeback_from_context(dc, res_ctx, pipes);
+
+ return pipe_cnt;
+}
+
+static int dml_populate_dml_pipes_from_context(
+ struct dc *dc, struct dc_state *context,
+ display_e2e_pipe_params_st *pipes,
+ bool fast_validate)
+{
+ int i, pipe_cnt;
+ struct resource_context *res_ctx = &context->res_ctx;
+ struct pipe_ctx *pipe;
+
+ populate_dml_pipes_from_context_base(dc, context, pipes, fast_validate);
+
+ for (i = 0, pipe_cnt = 0; i < dc->res_pool->pipe_count; i++) {
+ struct dc_crtc_timing *timing;
+
+ if (!res_ctx->pipe_ctx[i].stream)
+ continue;
+ pipe = &res_ctx->pipe_ctx[i];
+ timing = &pipe->stream->timing;
+
+ pipes[pipe_cnt].pipe.src.gpuvm = true;
+ pipes[pipe_cnt].pipe.src.dcc_fraction_of_zs_req_luma = 0;
+ pipes[pipe_cnt].pipe.src.dcc_fraction_of_zs_req_chroma = 0;
+ pipes[pipe_cnt].pipe.dest.vfront_porch = timing->v_front_porch;
+
+ pipes[pipe_cnt].dout.dsc_input_bpc = 0;
+ if (pipes[pipe_cnt].dout.dsc_enable) {
+ switch (timing->display_color_depth) {
+ case COLOR_DEPTH_888:
+ pipes[pipe_cnt].dout.dsc_input_bpc = 8;
+ break;
+ case COLOR_DEPTH_101010:
+ pipes[pipe_cnt].dout.dsc_input_bpc = 10;
+ break;
+ case COLOR_DEPTH_121212:
+ pipes[pipe_cnt].dout.dsc_input_bpc = 12;
+ break;
+ default:
+ ASSERT(0);
+ break;
+ }
+ }
+ pipe_cnt++;
+ }
+ dc->config.enable_4to1MPC = false;
+ if (pipe_cnt == 1 && pipe->plane_state && !dc->debug.disable_z9_mpc) {
+ if (is_dual_plane(pipe->plane_state->format)
+ && pipe->plane_state->src_rect.width <= 1920 && pipe->plane_state->src_rect.height <= 1080) {
+ dc->config.enable_4to1MPC = true;
+ } else if (!is_dual_plane(pipe->plane_state->format)) {
+ context->bw_ctx.dml.ip.det_buffer_size_kbytes = 192;
+ pipes[0].pipe.src.unbounded_req_mode = true;
+ }
+ }
+
+ return pipe_cnt;
+}
+
+static void dml_full_validate_bw_helper(struct dc *dc,
+ struct dc_state *context,
+ display_e2e_pipe_params_st *pipes,
+ int *vlevel,
+ int *split,
+ bool *merge,
+ int *pipe_cnt)
+{
+ struct vba_vars_st *vba = &context->bw_ctx.dml.vba;
+
+ /*
+ * DML favors voltage over p-state, but we're more interested in
+ * supporting p-state over voltage. We can't support p-state in
+ * prefetch mode > 0 so try capping the prefetch mode to start.
+ */
+ context->bw_ctx.dml.soc.allow_dram_self_refresh_or_dram_clock_change_in_vblank =
+ dm_allow_self_refresh_and_mclk_switch;
+ *vlevel = dml_get_voltage_level(&context->bw_ctx.dml, pipes, *pipe_cnt);
+ /* This may adjust vlevel and maxMpcComb */
+ if (*vlevel < context->bw_ctx.dml.soc.num_states)
+ *vlevel = dml_validate_apply_pipe_split_flags(dc, context, *vlevel, split, merge);
+
+ /* Conditions for setting up phantom pipes for SubVP:
+ * 1. Not force disable SubVP
+ * 2. Full update (i.e. !fast_validate)
+ * 3. Enough pipes are available to support SubVP (TODO: Which pipes will use VACTIVE / VBLANK / SUBVP?)
+ * 4. Display configuration passes validation
+ * 5. (Config doesn't support MCLK in VACTIVE/VBLANK || dc->debug.force_subvp_mclk_switch)
+ */
+ if (!dc->debug.force_disable_subvp &&
+ dml_enough_pipes_for_subvp(dc, context) &&
+ *vlevel < context->bw_ctx.dml.soc.num_states &&
+ (vba->DRAMClockChangeSupport[*vlevel][vba->maxMpcComb] == dm_dram_clock_change_unsupported ||
+ dc->debug.force_subvp_mclk_switch)) {
+
+ dml_add_phantom_pipes(dc, context);
+
+ /* Create input to DML based on new context which includes phantom pipes
+ * TODO: Input to DML should mark which pipes are phantom
+ */
+ *pipe_cnt = dml_populate_dml_pipes_from_context(dc, context, pipes, false);
+ *vlevel = dml_get_voltage_level(&context->bw_ctx.dml, pipes, *pipe_cnt);
+ if (*vlevel < context->bw_ctx.dml.soc.num_states) {
+ memset(split, 0, MAX_PIPES * sizeof(*split));
+ memset(merge, 0, MAX_PIPES * sizeof(*merge));
+ *vlevel = dml_validate_apply_pipe_split_flags(dc, context, *vlevel, split, merge);
+ }
+
+ // If SubVP pipe config is unsupported (or cannot be used for UCLK switching)
+ // remove phantom pipes and repopulate dml pipes
+ if (*vlevel == context->bw_ctx.dml.soc.num_states ||
+ vba->DRAMClockChangeSupport[*vlevel][vba->maxMpcComb] == dm_dram_clock_change_unsupported) {
+ dml_remove_phantom_pipes(dc, context);
+ *pipe_cnt = dml_populate_dml_pipes_from_context(dc, context, pipes, false);
+ }
+ }
+}
+
+static void dcn20_adjust_adaptive_sync_v_startup(
+ const struct dc_crtc_timing *dc_crtc_timing, int *vstartup_start)
+{
+ struct dc_crtc_timing patched_crtc_timing;
+ uint32_t asic_blank_end = 0;
+ uint32_t asic_blank_start = 0;
+ uint32_t newVstartup = 0;
+
+ patched_crtc_timing = *dc_crtc_timing;
+
+ if (patched_crtc_timing.flags.INTERLACE == 1) {
+ if (patched_crtc_timing.v_front_porch < 2)
+ patched_crtc_timing.v_front_porch = 2;
+ } else {
+ if (patched_crtc_timing.v_front_porch < 1)
+ patched_crtc_timing.v_front_porch = 1;
+ }
+
+ /* blank_start = frame end - front porch */
+ asic_blank_start = patched_crtc_timing.v_total -
+ patched_crtc_timing.v_front_porch;
+
+ /* blank_end = blank_start - active */
+ asic_blank_end = asic_blank_start -
+ patched_crtc_timing.v_border_bottom -
+ patched_crtc_timing.v_addressable -
+ patched_crtc_timing.v_border_top;
+
+ newVstartup = asic_blank_end + (patched_crtc_timing.v_total - asic_blank_start);
+
+ *vstartup_start = ((newVstartup > *vstartup_start) ? newVstartup : *vstartup_start);
+}
+
+static bool is_dp_128b_132b_signal(struct pipe_ctx *pipe_ctx)
+{
+ return (pipe_ctx->stream_res.hpo_dp_stream_enc &&
+ pipe_ctx->link_res.hpo_dp_link_enc &&
+ dc_is_dp_signal(pipe_ctx->stream->signal));
+}
+
+static bool is_dtbclk_required(struct dc *dc, struct dc_state *context)
+{
+ int i;
+ for (i = 0; i < dc->res_pool->pipe_count; i++) {
+ if (!context->res_ctx.pipe_ctx[i].stream)
+ continue;
+#if defined (CONFIG_DRM_AMD_DC_DP2_0)
+ if (is_dp_128b_132b_signal(&context->res_ctx.pipe_ctx[i]))
+ return true;
+#endif
+ }
+ return false;
+}
+
+static void dml_update_soc_for_wm_a(struct dc *dc, struct dc_state *context)
+{
+ if (dc->clk_mgr->bw_params->wm_table.nv_entries[WM_A].valid) {
+ context->bw_ctx.dml.soc.dram_clock_change_latency_us = dc->clk_mgr->bw_params->wm_table.nv_entries[WM_A].dml_input.pstate_latency_us;
+ context->bw_ctx.dml.soc.sr_enter_plus_exit_time_us = dc->clk_mgr->bw_params->wm_table.nv_entries[WM_A].dml_input.sr_enter_plus_exit_time_us;
+ context->bw_ctx.dml.soc.sr_exit_time_us = dc->clk_mgr->bw_params->wm_table.nv_entries[WM_A].dml_input.sr_exit_time_us;
+ }
+}
+
+static bool dml_internal_validate(
+ struct dc *dc,
+ struct dc_state *context,
+ display_e2e_pipe_params_st *pipes,
+ int *pipe_cnt_out,
+ int *vlevel_out,
+ bool fast_validate)
+{
+ bool out = false;
+ bool repopulate_pipes = false;
+ int split[MAX_PIPES] = { 0 };
+ bool merge[MAX_PIPES] = { false };
+ bool newly_split[MAX_PIPES] = { false };
+ int pipe_cnt, i, pipe_idx, vlevel;
+ struct vba_vars_st *vba = &context->bw_ctx.dml.vba;
+
+ ASSERT(pipes);
+ if (!pipes)
+ return false;
+
+ // For each full update, remove all existing phantom pipes first
+ dml_remove_phantom_pipes(dc, context);
+
+ dml_update_soc_for_wm_a(dc, context);
+
+ for (i = 0; i < dc->res_pool->pipe_count; i++) {
+ struct pipe_ctx *pipe = &context->res_ctx.pipe_ctx[i];
+
+ if (pipe->plane_state) {
+ // On initial pass through DML, we intend to use MALL for SS on all
+ // (non-PSR) surfaces with none using MALL for P-State
+ // 'mall_plane_config': is not a member of 'dc_plane_state' - commenting it out till mall_plane_config gets supported in dc_plant_state
+ //if (pipe->stream && pipe->stream->link->psr_settings.psr_version == DC_PSR_VERSION_UNSUPPORTED)
+ // pipe->plane_state->mall_plane_config.use_mall_for_ss = true;
+ }
+ }
+ pipe_cnt = dml_populate_dml_pipes_from_context(dc, context, pipes, fast_validate);
+
+ if (!pipe_cnt) {
+ out = true;
+ goto validate_out;
+ }
+
+ dml_log_pipe_params(&context->bw_ctx.dml, pipes, pipe_cnt);
+
+ if (!fast_validate) {
+ dml_full_validate_bw_helper(dc, context, pipes, &vlevel, split, merge, &pipe_cnt);
+ }
+
+ if (fast_validate || vlevel == context->bw_ctx.dml.soc.num_states ||
+ vba->DRAMClockChangeSupport[vlevel][vba->maxMpcComb] == dm_dram_clock_change_unsupported) {
+ /*
+ * If mode is unsupported or there's still no p-state support then
+ * fall back to favoring voltage.
+ *
+ * We don't actually support prefetch mode 2, so require that we
+ * at least support prefetch mode 1.
+ */
+ context->bw_ctx.dml.soc.allow_dram_self_refresh_or_dram_clock_change_in_vblank =
+ dm_allow_self_refresh;
+
+ vlevel = dml_get_voltage_level(&context->bw_ctx.dml, pipes, pipe_cnt);
+ if (vlevel < context->bw_ctx.dml.soc.num_states) {
+ memset(split, 0, sizeof(split));
+ memset(merge, 0, sizeof(merge));
+ vlevel = dml_validate_apply_pipe_split_flags(dc, context, vlevel, split, merge);
+ }
+ }
+
+ dml_log_mode_support_params(&context->bw_ctx.dml);
+
+ if (vlevel == context->bw_ctx.dml.soc.num_states)
+ goto validate_fail;
+
+ for (i = 0, pipe_idx = 0; i < dc->res_pool->pipe_count; i++) {
+ struct pipe_ctx *pipe = &context->res_ctx.pipe_ctx[i];
+ struct pipe_ctx *mpo_pipe = pipe->bottom_pipe;
+
+ if (!pipe->stream)
+ continue;
+
+ /* We only support full screen mpo with ODM */
+ if (vba->ODMCombineEnabled[vba->pipe_plane[pipe_idx]] != dm_odm_combine_mode_disabled
+ && pipe->plane_state && mpo_pipe
+ && memcmp(&mpo_pipe->plane_res.scl_data.recout,
+ &pipe->plane_res.scl_data.recout,
+ sizeof(struct rect)) != 0) {
+ ASSERT(mpo_pipe->plane_state != pipe->plane_state);
+ goto validate_fail;
+ }
+ pipe_idx++;
+ }
+
+ /* merge pipes if necessary */
+ for (i = 0; i < dc->res_pool->pipe_count; i++) {
+ struct pipe_ctx *pipe = &context->res_ctx.pipe_ctx[i];
+
+ /*skip pipes that don't need merging*/
+ if (!merge[i])
+ continue;
+
+ /* if ODM merge we ignore mpc tree, mpo pipes will have their own flags */
+ if (pipe->prev_odm_pipe) {
+ /*split off odm pipe*/
+ pipe->prev_odm_pipe->next_odm_pipe = pipe->next_odm_pipe;
+ if (pipe->next_odm_pipe)
+ pipe->next_odm_pipe->prev_odm_pipe = pipe->prev_odm_pipe;
+
+ pipe->bottom_pipe = NULL;
+ pipe->next_odm_pipe = NULL;
+ pipe->plane_state = NULL;
+ pipe->stream = NULL;
+ pipe->top_pipe = NULL;
+ pipe->prev_odm_pipe = NULL;
+ if (pipe->stream_res.dsc)
+ dml_release_dsc(&context->res_ctx, dc->res_pool, &pipe->stream_res.dsc);
+ memset(&pipe->plane_res, 0, sizeof(pipe->plane_res));
+ memset(&pipe->stream_res, 0, sizeof(pipe->stream_res));
+ repopulate_pipes = true;
+ } else if (pipe->top_pipe && pipe->top_pipe->plane_state == pipe->plane_state) {
+ struct pipe_ctx *top_pipe = pipe->top_pipe;
+ struct pipe_ctx *bottom_pipe = pipe->bottom_pipe;
+
+ top_pipe->bottom_pipe = bottom_pipe;
+ if (bottom_pipe)
+ bottom_pipe->top_pipe = top_pipe;
+
+ pipe->top_pipe = NULL;
+ pipe->bottom_pipe = NULL;
+ pipe->plane_state = NULL;
+ pipe->stream = NULL;
+ memset(&pipe->plane_res, 0, sizeof(pipe->plane_res));
+ memset(&pipe->stream_res, 0, sizeof(pipe->stream_res));
+ repopulate_pipes = true;
+ } else
+ ASSERT(0); /* Should never try to merge master pipe */
+
+ }
+
+ for (i = 0, pipe_idx = -1; i < dc->res_pool->pipe_count; i++) {
+ struct pipe_ctx *pipe = &context->res_ctx.pipe_ctx[i];
+ struct pipe_ctx *old_pipe = &dc->current_state->res_ctx.pipe_ctx[i];
+ struct pipe_ctx *hsplit_pipe = NULL;
+ bool odm;
+ int old_index = -1;
+
+ if (!pipe->stream || newly_split[i])
+ continue;
+
+ pipe_idx++;
+ odm = vba->ODMCombineEnabled[vba->pipe_plane[pipe_idx]] != dm_odm_combine_mode_disabled;
+
+ if (!pipe->plane_state && !odm)
+ continue;
+
+ if (split[i]) {
+ if (odm) {
+ if (split[i] == 4 && old_pipe->next_odm_pipe && old_pipe->next_odm_pipe->next_odm_pipe)
+ old_index = old_pipe->next_odm_pipe->next_odm_pipe->pipe_idx;
+ else if (old_pipe->next_odm_pipe)
+ old_index = old_pipe->next_odm_pipe->pipe_idx;
+ } else {
+ if (split[i] == 4 && old_pipe->bottom_pipe && old_pipe->bottom_pipe->bottom_pipe &&
+ old_pipe->bottom_pipe->bottom_pipe->plane_state == old_pipe->plane_state)
+ old_index = old_pipe->bottom_pipe->bottom_pipe->pipe_idx;
+ else if (old_pipe->bottom_pipe &&
+ old_pipe->bottom_pipe->plane_state == old_pipe->plane_state)
+ old_index = old_pipe->bottom_pipe->pipe_idx;
+ }
+ hsplit_pipe = dml_find_split_pipe(dc, context, old_index);
+ ASSERT(hsplit_pipe);
+ if (!hsplit_pipe)
+ goto validate_fail;
+
+ if (!dml_split_stream_for_mpc_or_odm(
+ dc, &context->res_ctx,
+ pipe, hsplit_pipe, odm))
+ goto validate_fail;
+
+ newly_split[hsplit_pipe->pipe_idx] = true;
+ repopulate_pipes = true;
+ }
+ if (split[i] == 4) {
+ struct pipe_ctx *pipe_4to1;
+
+ if (odm && old_pipe->next_odm_pipe)
+ old_index = old_pipe->next_odm_pipe->pipe_idx;
+ else if (!odm && old_pipe->bottom_pipe &&
+ old_pipe->bottom_pipe->plane_state == old_pipe->plane_state)
+ old_index = old_pipe->bottom_pipe->pipe_idx;
+ else
+ old_index = -1;
+ pipe_4to1 = dml_find_split_pipe(dc, context, old_index);
+ ASSERT(pipe_4to1);
+ if (!pipe_4to1)
+ goto validate_fail;
+ if (!dml_split_stream_for_mpc_or_odm(
+ dc, &context->res_ctx,
+ pipe, pipe_4to1, odm))
+ goto validate_fail;
+ newly_split[pipe_4to1->pipe_idx] = true;
+
+ if (odm && old_pipe->next_odm_pipe && old_pipe->next_odm_pipe->next_odm_pipe
+ && old_pipe->next_odm_pipe->next_odm_pipe->next_odm_pipe)
+ old_index = old_pipe->next_odm_pipe->next_odm_pipe->next_odm_pipe->pipe_idx;
+ else if (!odm && old_pipe->bottom_pipe && old_pipe->bottom_pipe->bottom_pipe &&
+ old_pipe->bottom_pipe->bottom_pipe->bottom_pipe &&
+ old_pipe->bottom_pipe->bottom_pipe->bottom_pipe->plane_state == old_pipe->plane_state)
+ old_index = old_pipe->bottom_pipe->bottom_pipe->bottom_pipe->pipe_idx;
+ else
+ old_index = -1;
+ pipe_4to1 = dml_find_split_pipe(dc, context, old_index);
+ ASSERT(pipe_4to1);
+ if (!pipe_4to1)
+ goto validate_fail;
+ if (!dml_split_stream_for_mpc_or_odm(
+ dc, &context->res_ctx,
+ hsplit_pipe, pipe_4to1, odm))
+ goto validate_fail;
+ newly_split[pipe_4to1->pipe_idx] = true;
+ }
+ if (odm)
+ dml_build_mapped_resource(dc, context, pipe->stream);
+ }
+
+ for (i = 0; i < dc->res_pool->pipe_count; i++) {
+ struct pipe_ctx *pipe = &context->res_ctx.pipe_ctx[i];
+
+ if (pipe->plane_state) {
+ if (!resource_build_scaling_params(pipe))
+ goto validate_fail;
+ }
+ }
+
+ /* Actual dsc count per stream dsc validation*/
+ if (!dml_validate_dsc(dc, context)) {
+ vba->ValidationStatus[vba->soc.num_states] = DML_FAIL_DSC_VALIDATION_FAILURE;
+ goto validate_fail;
+ }
+
+ if (repopulate_pipes)
+ pipe_cnt = dml_populate_dml_pipes_from_context(dc, context, pipes, fast_validate);
+ *vlevel_out = vlevel;
+ *pipe_cnt_out = pipe_cnt;
+
+ out = true;
+ goto validate_out;
+
+validate_fail:
+ out = false;
+
+validate_out:
+ return out;
+}
+
+static void dml_calculate_dlg_params(
+ struct dc *dc, struct dc_state *context,
+ display_e2e_pipe_params_st *pipes,
+ int pipe_cnt,
+ int vlevel)
+{
+ int i, pipe_idx;
+ int plane_count;
+
+ /* Writeback MCIF_WB arbitration parameters */
+ if (dc->res_pool)
+ dc->res_pool->funcs->set_mcif_arb_params(dc, context, pipes, pipe_cnt);
+
+ context->bw_ctx.bw.dcn.clk.dispclk_khz = context->bw_ctx.dml.vba.DISPCLK * 1000;
+ context->bw_ctx.bw.dcn.clk.dcfclk_khz = context->bw_ctx.dml.vba.DCFCLK * 1000;
+ context->bw_ctx.bw.dcn.clk.socclk_khz = context->bw_ctx.dml.vba.SOCCLK * 1000;
+ context->bw_ctx.bw.dcn.clk.dramclk_khz = context->bw_ctx.dml.vba.DRAMSpeed * 1000 / 16;
+ context->bw_ctx.bw.dcn.clk.dcfclk_deep_sleep_khz = context->bw_ctx.dml.vba.DCFCLKDeepSleep * 1000;
+ context->bw_ctx.bw.dcn.clk.fclk_khz = context->bw_ctx.dml.vba.FabricClock * 1000;
+ context->bw_ctx.bw.dcn.clk.p_state_change_support =
+ context->bw_ctx.dml.vba.DRAMClockChangeSupport[vlevel][context->bw_ctx.dml.vba.maxMpcComb]
+ != dm_dram_clock_change_unsupported;
+
+ context->bw_ctx.bw.dcn.clk.dppclk_khz = 0;
+ /* 'z9_support': is not a member of 'dc_clocks' - Commenting out till we have this support in dc_clocks
+ * context->bw_ctx.bw.dcn.clk.z9_support = (context->bw_ctx.dml.vba.StutterPeriod > 5000.0) ?
+ DCN_Z9_SUPPORT_ALLOW : DCN_Z9_SUPPORT_DISALLOW;
+ */
+ plane_count = 0;
+ for (i = 0; i < dc->res_pool->pipe_count; i++) {
+ if (context->res_ctx.pipe_ctx[i].plane_state)
+ plane_count++;
+ }
+
+ /* Commented out as per above error for now.
+ if (plane_count == 0)
+ context->bw_ctx.bw.dcn.clk.z9_support = DCN_Z9_SUPPORT_ALLOW;
+ */
+ context->bw_ctx.bw.dcn.clk.dtbclk_en = is_dtbclk_required(dc, context);
+ /* TODO : Uncomment the below line and make changes
+ * as per DML nomenclature once it is available.
+ * context->bw_ctx.bw.dcn.clk.fclk_p_state_change_support = context->bw_ctx.dml.vba.fclk_pstate_support;
+ */
+
+ if (context->bw_ctx.bw.dcn.clk.dispclk_khz < dc->debug.min_disp_clk_khz)
+ context->bw_ctx.bw.dcn.clk.dispclk_khz = dc->debug.min_disp_clk_khz;
+
+ for (i = 0, pipe_idx = 0; i < dc->res_pool->pipe_count; i++) {
+ if (!context->res_ctx.pipe_ctx[i].stream)
+ continue;
+ pipes[pipe_idx].pipe.dest.vstartup_start = get_vstartup(&context->bw_ctx.dml, pipes, pipe_cnt, pipe_idx);
+ pipes[pipe_idx].pipe.dest.vupdate_offset = get_vupdate_offset(&context->bw_ctx.dml, pipes, pipe_cnt, pipe_idx);
+ pipes[pipe_idx].pipe.dest.vupdate_width = get_vupdate_width(&context->bw_ctx.dml, pipes, pipe_cnt, pipe_idx);
+ pipes[pipe_idx].pipe.dest.vready_offset = get_vready_offset(&context->bw_ctx.dml, pipes, pipe_cnt, pipe_idx);
+ if (context->res_ctx.pipe_ctx[i].stream->mall_stream_config.type == SUBVP_PHANTOM) {
+ // Phantom pipe requires that DET_SIZE = 0 and no unbounded requests
+ context->res_ctx.pipe_ctx[i].det_buffer_size_kb = 0;
+ context->res_ctx.pipe_ctx[i].unbounded_req = false;
+ } else {
+ context->res_ctx.pipe_ctx[i].det_buffer_size_kb = context->bw_ctx.dml.ip.det_buffer_size_kbytes;
+ context->res_ctx.pipe_ctx[i].unbounded_req = pipes[pipe_idx].pipe.src.unbounded_req_mode;
+ }
+
+ if (context->bw_ctx.bw.dcn.clk.dppclk_khz < pipes[pipe_idx].clks_cfg.dppclk_mhz * 1000)
+ context->bw_ctx.bw.dcn.clk.dppclk_khz = pipes[pipe_idx].clks_cfg.dppclk_mhz * 1000;
+ context->res_ctx.pipe_ctx[i].plane_res.bw.dppclk_khz =
+ pipes[pipe_idx].clks_cfg.dppclk_mhz * 1000;
+ context->res_ctx.pipe_ctx[i].pipe_dlg_param = pipes[pipe_idx].pipe.dest;
+ pipe_idx++;
+ }
+ /*save a original dppclock copy*/
+ context->bw_ctx.bw.dcn.clk.bw_dppclk_khz = context->bw_ctx.bw.dcn.clk.dppclk_khz;
+ context->bw_ctx.bw.dcn.clk.bw_dispclk_khz = context->bw_ctx.bw.dcn.clk.dispclk_khz;
+ context->bw_ctx.bw.dcn.clk.max_supported_dppclk_khz = context->bw_ctx.dml.soc.clock_limits[vlevel].dppclk_mhz * 1000;
+ context->bw_ctx.bw.dcn.clk.max_supported_dispclk_khz = context->bw_ctx.dml.soc.clock_limits[vlevel].dispclk_mhz * 1000;
+ context->bw_ctx.bw.dcn.compbuf_size_kb = context->bw_ctx.dml.ip.config_return_buffer_size_in_kbytes
+ - context->bw_ctx.dml.ip.det_buffer_size_kbytes * pipe_idx;
+
+ for (i = 0, pipe_idx = 0; i < dc->res_pool->pipe_count; i++) {
+ bool cstate_en = context->bw_ctx.dml.vba.PrefetchMode[vlevel][context->bw_ctx.dml.vba.maxMpcComb] != 2;
+
+ if (!context->res_ctx.pipe_ctx[i].stream)
+ continue;
+
+ context->bw_ctx.dml.funcs.rq_dlg_get_dlg_reg(&context->bw_ctx.dml,
+ &context->res_ctx.pipe_ctx[i].dlg_regs,
+ &context->res_ctx.pipe_ctx[i].ttu_regs,
+ pipes,
+ pipe_cnt,
+ pipe_idx,
+ cstate_en,
+ context->bw_ctx.bw.dcn.clk.p_state_change_support,
+ false, false, true);
+
+ context->bw_ctx.dml.funcs.rq_dlg_get_rq_reg(&context->bw_ctx.dml,
+ &context->res_ctx.pipe_ctx[i].rq_regs,
+ &pipes[pipe_idx].pipe);
+ pipe_idx++;
+ }
+}
+
+static void dml_calculate_wm_and_dlg(
+ struct dc *dc, struct dc_state *context,
+ display_e2e_pipe_params_st *pipes,
+ int pipe_cnt,
+ int vlevel)
+{
+ int i, pipe_idx, vlevel_temp = 0;
+
+ double dcfclk = context->bw_ctx.dml.soc.clock_limits[0].dcfclk_mhz;
+ double dcfclk_from_validation = context->bw_ctx.dml.vba.DCFCLKState[vlevel][context->bw_ctx.dml.vba.maxMpcComb];
+ unsigned int min_dram_speed_mts = context->bw_ctx.dml.vba.DRAMSpeed;
+ bool pstate_en = context->bw_ctx.dml.vba.DRAMClockChangeSupport[vlevel][context->bw_ctx.dml.vba.maxMpcComb] !=
+ dm_dram_clock_change_unsupported;
+
+ /* Set B:
+ * For Set B calculations use clocks from clock_limits[2] when available i.e. when SMU is present,
+ * otherwise use arbitrary low value from spreadsheet for DCFCLK as lower is safer for watermark
+ * calculations to cover bootup clocks.
+ * DCFCLK: soc.clock_limits[2] when available
+ * UCLK: soc.clock_limits[2] when available
+ */
+ if (context->bw_ctx.dml.soc.num_states > 2) {
+ vlevel_temp = 2;
+ dcfclk = context->bw_ctx.dml.soc.clock_limits[2].dcfclk_mhz;
+ } else
+ dcfclk = 615; //DCFCLK Vmin_lv
+
+ pipes[0].clks_cfg.voltage = vlevel_temp;
+ pipes[0].clks_cfg.dcfclk_mhz = dcfclk;
+ pipes[0].clks_cfg.socclk_mhz = context->bw_ctx.dml.soc.clock_limits[vlevel_temp].socclk_mhz;
+
+ if (dc->clk_mgr->bw_params->wm_table.nv_entries[WM_B].valid) {
+ context->bw_ctx.dml.soc.dram_clock_change_latency_us = dc->clk_mgr->bw_params->wm_table.nv_entries[WM_B].dml_input.pstate_latency_us;
+ context->bw_ctx.dml.soc.sr_enter_plus_exit_time_us = dc->clk_mgr->bw_params->wm_table.nv_entries[WM_B].dml_input.sr_enter_plus_exit_time_us;
+ context->bw_ctx.dml.soc.sr_exit_time_us = dc->clk_mgr->bw_params->wm_table.nv_entries[WM_B].dml_input.sr_exit_time_us;
+ }
+ context->bw_ctx.bw.dcn.watermarks.b.urgent_ns = get_wm_urgent(&context->bw_ctx.dml, pipes, pipe_cnt) * 1000;
+ context->bw_ctx.bw.dcn.watermarks.b.cstate_pstate.cstate_enter_plus_exit_ns = get_wm_stutter_enter_exit(&context->bw_ctx.dml, pipes, pipe_cnt) * 1000;
+ context->bw_ctx.bw.dcn.watermarks.b.cstate_pstate.cstate_exit_ns = get_wm_stutter_exit(&context->bw_ctx.dml, pipes, pipe_cnt) * 1000;
+ context->bw_ctx.bw.dcn.watermarks.b.cstate_pstate.pstate_change_ns = get_wm_dram_clock_change(&context->bw_ctx.dml, pipes, pipe_cnt) * 1000;
+ context->bw_ctx.bw.dcn.watermarks.b.pte_meta_urgent_ns = get_wm_memory_trip(&context->bw_ctx.dml, pipes, pipe_cnt) * 1000;
+ context->bw_ctx.bw.dcn.watermarks.b.frac_urg_bw_nom = get_fraction_of_urgent_bandwidth(&context->bw_ctx.dml, pipes, pipe_cnt) * 1000;
+ context->bw_ctx.bw.dcn.watermarks.b.frac_urg_bw_flip = get_fraction_of_urgent_bandwidth_imm_flip(&context->bw_ctx.dml, pipes, pipe_cnt) * 1000;
+ context->bw_ctx.bw.dcn.watermarks.b.urgent_latency_ns = get_urgent_latency(&context->bw_ctx.dml, pipes, pipe_cnt) * 1000;
+ //context->bw_ctx.bw.dcn.watermarks.b.cstate_pstate.fclk_pstate_change_ns = get_wm_fclk_pstate(&context->bw_ctx.dml, pipes, pipe_cnt) * 1000;
+ //context->bw_ctx.bw.dcn.watermarks.b.usr_retraining_ns = get_wm_usr_retraining(&context->bw_ctx.dml, pipes, pipe_cnt) * 1000;
+
+ /* Temporary, to have some fclk_pstate_change_ns and usr_retraining_ns wm values until DML is implemented */
+ context->bw_ctx.bw.dcn.watermarks.b.cstate_pstate.fclk_pstate_change_ns = context->bw_ctx.bw.dcn.watermarks.b.cstate_pstate.pstate_change_ns / 4;
+ context->bw_ctx.bw.dcn.watermarks.b.usr_retraining_ns = context->bw_ctx.bw.dcn.watermarks.b.cstate_pstate.pstate_change_ns / 8;
+
+ /* Set D:
+ * All clocks min.
+ * DCFCLK: Min, as reported by PM FW when available
+ * UCLK : Min, as reported by PM FW when available
+ * sr_enter_exit/sr_exit should be lower than used for DRAM (TBD after bringup or later, use as decided in Clk Mgr)
+ */
+
+ if (context->bw_ctx.dml.soc.num_states > 2) {
+ vlevel_temp = 0;
+ dcfclk = dc->clk_mgr->bw_params->clk_table.entries[0].dcfclk_mhz;
+ } else
+ dcfclk = 615; //DCFCLK Vmin_lv
+
+ pipes[0].clks_cfg.voltage = vlevel_temp;
+ pipes[0].clks_cfg.dcfclk_mhz = dcfclk;
+ pipes[0].clks_cfg.socclk_mhz = context->bw_ctx.dml.soc.clock_limits[vlevel_temp].socclk_mhz;
+
+ if (dc->clk_mgr->bw_params->wm_table.nv_entries[WM_D].valid) {
+ context->bw_ctx.dml.soc.dram_clock_change_latency_us = dc->clk_mgr->bw_params->wm_table.nv_entries[WM_D].dml_input.pstate_latency_us;
+ context->bw_ctx.dml.soc.sr_enter_plus_exit_time_us = dc->clk_mgr->bw_params->wm_table.nv_entries[WM_D].dml_input.sr_enter_plus_exit_time_us;
+ context->bw_ctx.dml.soc.sr_exit_time_us = dc->clk_mgr->bw_params->wm_table.nv_entries[WM_D].dml_input.sr_exit_time_us;
+ }
+ context->bw_ctx.bw.dcn.watermarks.d.urgent_ns = get_wm_urgent(&context->bw_ctx.dml, pipes, pipe_cnt) * 1000;
+ context->bw_ctx.bw.dcn.watermarks.d.cstate_pstate.cstate_enter_plus_exit_ns = get_wm_stutter_enter_exit(&context->bw_ctx.dml, pipes, pipe_cnt) * 1000;
+ context->bw_ctx.bw.dcn.watermarks.d.cstate_pstate.cstate_exit_ns = get_wm_stutter_exit(&context->bw_ctx.dml, pipes, pipe_cnt) * 1000;
+ context->bw_ctx.bw.dcn.watermarks.d.cstate_pstate.pstate_change_ns = get_wm_dram_clock_change(&context->bw_ctx.dml, pipes, pipe_cnt) * 1000;
+ context->bw_ctx.bw.dcn.watermarks.d.pte_meta_urgent_ns = get_wm_memory_trip(&context->bw_ctx.dml, pipes, pipe_cnt) * 1000;
+ context->bw_ctx.bw.dcn.watermarks.d.frac_urg_bw_nom = get_fraction_of_urgent_bandwidth(&context->bw_ctx.dml, pipes, pipe_cnt) * 1000;
+ context->bw_ctx.bw.dcn.watermarks.d.frac_urg_bw_flip = get_fraction_of_urgent_bandwidth_imm_flip(&context->bw_ctx.dml, pipes, pipe_cnt) * 1000;
+ context->bw_ctx.bw.dcn.watermarks.d.urgent_latency_ns = get_urgent_latency(&context->bw_ctx.dml, pipes, pipe_cnt) * 1000;
+ //context->bw_ctx.bw.dcn.watermarks.d.cstate_pstate.fclk_pstate_change_ns = get_wm_fclk_pstate(&context->bw_ctx.dml, pipes, pipe_cnt) * 1000;
+ //context->bw_ctx.bw.dcn.watermarks.d.usr_retraining_ns = get_wm_usr_retraining(&context->bw_ctx.dml, pipes, pipe_cnt) * 1000;
+
+ /* Temporary, to have some fclk_pstate_change_ns and usr_retraining_ns wm values until DML is implemented */
+ context->bw_ctx.bw.dcn.watermarks.d.cstate_pstate.fclk_pstate_change_ns = context->bw_ctx.bw.dcn.watermarks.d.cstate_pstate.pstate_change_ns / 4;
+ context->bw_ctx.bw.dcn.watermarks.d.usr_retraining_ns = context->bw_ctx.bw.dcn.watermarks.d.cstate_pstate.pstate_change_ns / 8;
+
+ /* Set C, for Dummy P-State:
+ * All clocks min.
+ * DCFCLK: Min, as reported by PM FW, when available
+ * UCLK : Min, as reported by PM FW, when available
+ * pstate latency as per UCLK state dummy pstate latency
+ */
+ if (dc->clk_mgr->bw_params->wm_table.nv_entries[WM_C].valid) {
+ unsigned int min_dram_speed_mts_margin = 160;
+
+ if ((!pstate_en))
+ min_dram_speed_mts = dc->clk_mgr->bw_params->clk_table.entries[dc->clk_mgr->bw_params->clk_table.num_entries - 1].memclk_mhz * 16;
+
+ /* find largest table entry that is lower than dram speed, but lower than DPM0 still uses DPM0 */
+ for (i = 3; i > 0; i--)
+ if (min_dram_speed_mts + min_dram_speed_mts_margin > dc->clk_mgr->bw_params->dummy_pstate_table[i].dram_speed_mts)
+ break;
+
+ context->bw_ctx.dml.soc.dram_clock_change_latency_us = dc->clk_mgr->bw_params->dummy_pstate_table[i].dummy_pstate_latency_us;
+ context->bw_ctx.dml.soc.dummy_pstate_latency_us = dc->clk_mgr->bw_params->dummy_pstate_table[i].dummy_pstate_latency_us;
+ context->bw_ctx.dml.soc.sr_enter_plus_exit_time_us = dc->clk_mgr->bw_params->wm_table.nv_entries[WM_C].dml_input.sr_enter_plus_exit_time_us;
+ context->bw_ctx.dml.soc.sr_exit_time_us = dc->clk_mgr->bw_params->wm_table.nv_entries[WM_C].dml_input.sr_exit_time_us;
+ }
+ context->bw_ctx.bw.dcn.watermarks.c.urgent_ns = get_wm_urgent(&context->bw_ctx.dml, pipes, pipe_cnt) * 1000;
+ context->bw_ctx.bw.dcn.watermarks.c.cstate_pstate.cstate_enter_plus_exit_ns = get_wm_stutter_enter_exit(&context->bw_ctx.dml, pipes, pipe_cnt) * 1000;
+ context->bw_ctx.bw.dcn.watermarks.c.cstate_pstate.cstate_exit_ns = get_wm_stutter_exit(&context->bw_ctx.dml, pipes, pipe_cnt) * 1000;
+ context->bw_ctx.bw.dcn.watermarks.c.cstate_pstate.pstate_change_ns = get_wm_dram_clock_change(&context->bw_ctx.dml, pipes, pipe_cnt) * 1000;
+ context->bw_ctx.bw.dcn.watermarks.c.pte_meta_urgent_ns = get_wm_memory_trip(&context->bw_ctx.dml, pipes, pipe_cnt) * 1000;
+ context->bw_ctx.bw.dcn.watermarks.c.frac_urg_bw_nom = get_fraction_of_urgent_bandwidth(&context->bw_ctx.dml, pipes, pipe_cnt) * 1000;
+ context->bw_ctx.bw.dcn.watermarks.c.frac_urg_bw_flip = get_fraction_of_urgent_bandwidth_imm_flip(&context->bw_ctx.dml, pipes, pipe_cnt) * 1000;
+ context->bw_ctx.bw.dcn.watermarks.c.urgent_latency_ns = get_urgent_latency(&context->bw_ctx.dml, pipes, pipe_cnt) * 1000;
+ //context->bw_ctx.bw.dcn.watermarks.c.cstate_pstate.fclk_pstate_change_ns = get_wm_fclk_pstate(&context->bw_ctx.dml, pipes, pipe_cnt) * 1000;
+ //context->bw_ctx.bw.dcn.watermarks.c.usr_retraining_ns = get_wm_usr_retraining(&context->bw_ctx.dml, pipes, pipe_cnt) * 1000;
+
+ /* Temporary, to have some fclk_pstate_change_ns and usr_retraining_ns wm values until DML is implemented */
+ context->bw_ctx.bw.dcn.watermarks.c.cstate_pstate.fclk_pstate_change_ns = context->bw_ctx.bw.dcn.watermarks.c.cstate_pstate.pstate_change_ns / 4;
+ context->bw_ctx.bw.dcn.watermarks.c.usr_retraining_ns = context->bw_ctx.bw.dcn.watermarks.c.cstate_pstate.pstate_change_ns / 8;
+
+ if ((!pstate_en) && (dc->clk_mgr->bw_params->wm_table.nv_entries[WM_C].valid)) {
+ /* The only difference between A and C is p-state latency, if p-state is not supported
+ * with full p-state latency we want to calculate DLG based on dummy p-state latency,
+ * Set A p-state watermark set to 0 previously, when p-state unsupported, for now keep as previous implementation.
+ */
+ context->bw_ctx.bw.dcn.watermarks.a = context->bw_ctx.bw.dcn.watermarks.c;
+ context->bw_ctx.bw.dcn.watermarks.a.cstate_pstate.pstate_change_ns = 0;
+ } else {
+ /* Set A:
+ * All clocks min.
+ * DCFCLK: Min, as reported by PM FW, when available
+ * UCLK: Min, as reported by PM FW, when available
+ */
+ dml_update_soc_for_wm_a(dc, context);
+ context->bw_ctx.bw.dcn.watermarks.a.urgent_ns = get_wm_urgent(&context->bw_ctx.dml, pipes, pipe_cnt) * 1000;
+ context->bw_ctx.bw.dcn.watermarks.a.cstate_pstate.cstate_enter_plus_exit_ns = get_wm_stutter_enter_exit(&context->bw_ctx.dml, pipes, pipe_cnt) * 1000;
+ context->bw_ctx.bw.dcn.watermarks.a.cstate_pstate.cstate_exit_ns = get_wm_stutter_exit(&context->bw_ctx.dml, pipes, pipe_cnt) * 1000;
+ context->bw_ctx.bw.dcn.watermarks.a.cstate_pstate.pstate_change_ns = get_wm_dram_clock_change(&context->bw_ctx.dml, pipes, pipe_cnt) * 1000;
+ context->bw_ctx.bw.dcn.watermarks.a.pte_meta_urgent_ns = get_wm_memory_trip(&context->bw_ctx.dml, pipes, pipe_cnt) * 1000;
+ context->bw_ctx.bw.dcn.watermarks.a.frac_urg_bw_nom = get_fraction_of_urgent_bandwidth(&context->bw_ctx.dml, pipes, pipe_cnt) * 1000;
+ context->bw_ctx.bw.dcn.watermarks.a.frac_urg_bw_flip = get_fraction_of_urgent_bandwidth_imm_flip(&context->bw_ctx.dml, pipes, pipe_cnt) * 1000;
+ context->bw_ctx.bw.dcn.watermarks.a.urgent_latency_ns = get_urgent_latency(&context->bw_ctx.dml, pipes, pipe_cnt) * 1000;
+ }
+
+ pipes[0].clks_cfg.voltage = vlevel;
+ pipes[0].clks_cfg.dcfclk_mhz = dcfclk_from_validation;
+ pipes[0].clks_cfg.socclk_mhz = context->bw_ctx.dml.soc.clock_limits[vlevel].socclk_mhz;
+
+ for (i = 0, pipe_idx = 0; i < dc->res_pool->pipe_count; i++) {
+ if (!context->res_ctx.pipe_ctx[i].stream)
+ continue;
+
+ pipes[pipe_idx].clks_cfg.dispclk_mhz = get_dispclk_calculated(&context->bw_ctx.dml, pipes, pipe_cnt);
+ pipes[pipe_idx].clks_cfg.dppclk_mhz = get_dppclk_calculated(&context->bw_ctx.dml, pipes, pipe_cnt, pipe_idx);
+
+ if (dc->config.forced_clocks) {
+ pipes[pipe_idx].clks_cfg.dispclk_mhz = context->bw_ctx.dml.soc.clock_limits[0].dispclk_mhz;
+ pipes[pipe_idx].clks_cfg.dppclk_mhz = context->bw_ctx.dml.soc.clock_limits[0].dppclk_mhz;
+ }
+ if (dc->debug.min_disp_clk_khz > pipes[pipe_idx].clks_cfg.dispclk_mhz * 1000)
+ pipes[pipe_idx].clks_cfg.dispclk_mhz = dc->debug.min_disp_clk_khz / 1000.0;
+ if (dc->debug.min_dpp_clk_khz > pipes[pipe_idx].clks_cfg.dppclk_mhz * 1000)
+ pipes[pipe_idx].clks_cfg.dppclk_mhz = dc->debug.min_dpp_clk_khz / 1000.0;
+
+ pipe_idx++;
+ }
+
+ context->perf_params.stutter_period_us = context->bw_ctx.dml.vba.StutterPeriod;
+
+ dml_calculate_dlg_params(dc, context, pipes, pipe_cnt, vlevel);
+
+ if (!pstate_en)
+ /* Restore full p-state latency */
+ context->bw_ctx.dml.soc.dram_clock_change_latency_us =
+ dc->clk_mgr->bw_params->wm_table.nv_entries[WM_A].dml_input.pstate_latency_us;
+}
+
+bool dml_validate(struct dc *dc,
+ struct dc_state *context,
+ bool fast_validate)
+{
+ bool out = false;
+
+ BW_VAL_TRACE_SETUP();
+
+ int vlevel = 0;
+ int pipe_cnt = 0;
+ display_e2e_pipe_params_st *pipes = context->bw_ctx.dml.dml_pipe_state;
+ DC_LOGGER_INIT(dc->ctx->logger);
+
+ BW_VAL_TRACE_COUNT();
+
+ out = dml_internal_validate(dc, context, pipes, &pipe_cnt, &vlevel, fast_validate);
+
+ if (pipe_cnt == 0)
+ goto validate_out;
+
+ if (!out)
+ goto validate_fail;
+
+ BW_VAL_TRACE_END_VOLTAGE_LEVEL();
+
+ if (fast_validate) {
+ BW_VAL_TRACE_SKIP(fast);
+ goto validate_out;
+ }
+
+ dml_calculate_wm_and_dlg(dc, context, pipes, pipe_cnt, vlevel);
+
+ BW_VAL_TRACE_END_WATERMARKS();
+
+ goto validate_out;
+
+validate_fail:
+ DC_LOG_WARNING("Mode Validation Warning: %s failed validation.\n",
+ dml_get_status_message(context->bw_ctx.dml.vba.ValidationStatus[context->bw_ctx.dml.vba.soc.num_states]));
+
+ BW_VAL_TRACE_SKIP(fail);
+ out = false;
+
+validate_out:
+ BW_VAL_TRACE_FINISH();
+
+ return out;
+}
diff --git a/drivers/gpu/drm/amd/display/dc/dml/dml_wrapper_translation.c b/drivers/gpu/drm/amd/display/dc/dml/dml_wrapper_translation.c
new file mode 100644
index 0000000..4ec5310
--- /dev/null
+++ b/drivers/gpu/drm/amd/display/dc/dml/dml_wrapper_translation.c
@@ -0,0 +1,284 @@
+/*
+ * Copyright 2017 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ * Authors: AMD
+ *
+ */
+
+#ifdef DML_WRAPPER_TRANSLATION_
+
+static void gfx10array_mode_to_dml_params(
+ enum array_mode_values array_mode,
+ enum legacy_tiling_compat_level compat_level,
+ unsigned int *sw_mode)
+{
+ switch (array_mode) {
+ case DC_ARRAY_LINEAR_ALLIGNED:
+ case DC_ARRAY_LINEAR_GENERAL:
+ *sw_mode = dm_sw_linear;
+ break;
+ case DC_ARRAY_2D_TILED_THIN1:
+// DC_LEGACY_TILING_ADDR_GEN_ZERO - undefined as per current code hence removed
+#if 0
+ if (compat_level == DC_LEGACY_TILING_ADDR_GEN_ZERO)
+ *sw_mode = dm_sw_gfx7_2d_thin_l_vp;
+ else
+ *sw_mode = dm_sw_gfx7_2d_thin_gl;
+#endif
+ break;
+ default:
+ ASSERT(0); /* Not supported */
+ break;
+ }
+}
+
+static void swizzle_to_dml_params(
+ enum swizzle_mode_values swizzle,
+ unsigned int *sw_mode)
+{
+ switch (swizzle) {
+ case DC_SW_LINEAR:
+ *sw_mode = dm_sw_linear;
+ break;
+ case DC_SW_4KB_S:
+ *sw_mode = dm_sw_4kb_s;
+ break;
+ case DC_SW_4KB_S_X:
+ *sw_mode = dm_sw_4kb_s_x;
+ break;
+ case DC_SW_4KB_D:
+ *sw_mode = dm_sw_4kb_d;
+ break;
+ case DC_SW_4KB_D_X:
+ *sw_mode = dm_sw_4kb_d_x;
+ break;
+ case DC_SW_64KB_S:
+ *sw_mode = dm_sw_64kb_s;
+ break;
+ case DC_SW_64KB_S_X:
+ *sw_mode = dm_sw_64kb_s_x;
+ break;
+ case DC_SW_64KB_S_T:
+ *sw_mode = dm_sw_64kb_s_t;
+ break;
+ case DC_SW_64KB_D:
+ *sw_mode = dm_sw_64kb_d;
+ break;
+ case DC_SW_64KB_D_X:
+ *sw_mode = dm_sw_64kb_d_x;
+ break;
+ case DC_SW_64KB_D_T:
+ *sw_mode = dm_sw_64kb_d_t;
+ break;
+ case DC_SW_64KB_R_X:
+ *sw_mode = dm_sw_64kb_r_x;
+ break;
+ case DC_SW_VAR_S:
+ *sw_mode = dm_sw_var_s;
+ break;
+ case DC_SW_VAR_S_X:
+ *sw_mode = dm_sw_var_s_x;
+ break;
+ case DC_SW_VAR_D:
+ *sw_mode = dm_sw_var_d;
+ break;
+ case DC_SW_VAR_D_X:
+ *sw_mode = dm_sw_var_d_x;
+ break;
+
+ default:
+ ASSERT(0); /* Not supported */
+ break;
+ }
+}
+
+static void dc_timing_to_dml_timing(const struct dc_crtc_timing *timing, struct _vcs_dpi_display_pipe_dest_params_st *dest)
+{
+ dest->hblank_start = timing->h_total - timing->h_front_porch;
+ dest->hblank_end = dest->hblank_start
+ - timing->h_addressable
+ - timing->h_border_left
+ - timing->h_border_right;
+ dest->vblank_start = timing->v_total - timing->v_front_porch;
+ dest->vblank_end = dest->vblank_start
+ - timing->v_addressable
+ - timing->v_border_top
+ - timing->v_border_bottom;
+ dest->htotal = timing->h_total;
+ dest->vtotal = timing->v_total;
+ dest->hactive = timing->h_addressable;
+ dest->vactive = timing->v_addressable;
+ dest->interlaced = timing->flags.INTERLACE;
+ dest->pixel_rate_mhz = timing->pix_clk_100hz/10000.0;
+ if (timing->timing_3d_format == TIMING_3D_FORMAT_HW_FRAME_PACKING)
+ dest->pixel_rate_mhz *= 2;
+}
+
+static enum odm_combine_mode get_dml_odm_combine(const struct pipe_ctx *pipe)
+{
+ int odm_split_count = 0;
+ enum odm_combine_mode combine_mode = dm_odm_combine_mode_disabled;
+ struct pipe_ctx *next_pipe = pipe->next_odm_pipe;
+
+ // Traverse pipe tree to determine odm split count
+ while (next_pipe) {
+ odm_split_count++;
+ next_pipe = next_pipe->next_odm_pipe;
+ }
+ pipe = pipe->prev_odm_pipe;
+ while (pipe) {
+ odm_split_count++;
+ pipe = pipe->prev_odm_pipe;
+ }
+
+ // Translate split to DML odm combine factor
+ switch (odm_split_count) {
+ case 1:
+ combine_mode = dm_odm_combine_mode_2to1;
+ break;
+ case 3:
+ combine_mode = dm_odm_combine_mode_4to1;
+ break;
+ default:
+ combine_mode = dm_odm_combine_mode_disabled;
+ }
+
+ return combine_mode;
+}
+
+static int get_dml_output_type(enum signal_type dc_signal)
+{
+ int dml_output_type = -1;
+
+ switch (dc_signal) {
+ case SIGNAL_TYPE_DISPLAY_PORT_MST:
+ case SIGNAL_TYPE_DISPLAY_PORT:
+ dml_output_type = dm_dp;
+ break;
+ case SIGNAL_TYPE_EDP:
+ dml_output_type = dm_edp;
+ break;
+ case SIGNAL_TYPE_HDMI_TYPE_A:
+ case SIGNAL_TYPE_DVI_SINGLE_LINK:
+ case SIGNAL_TYPE_DVI_DUAL_LINK:
+ dml_output_type = dm_hdmi;
+ break;
+ default:
+ break;
+ }
+
+ return dml_output_type;
+}
+
+static void populate_color_depth_and_encoding_from_timing(const struct dc_crtc_timing *timing, struct _vcs_dpi_display_output_params_st *dout)
+{
+ int output_bpc = 0;
+
+ switch (timing->display_color_depth) {
+ case COLOR_DEPTH_666:
+ output_bpc = 6;
+ break;
+ case COLOR_DEPTH_888:
+ output_bpc = 8;
+ break;
+ case COLOR_DEPTH_101010:
+ output_bpc = 10;
+ break;
+ case COLOR_DEPTH_121212:
+ output_bpc = 12;
+ break;
+ case COLOR_DEPTH_141414:
+ output_bpc = 14;
+ break;
+ case COLOR_DEPTH_161616:
+ output_bpc = 16;
+ break;
+ case COLOR_DEPTH_999:
+ output_bpc = 9;
+ break;
+ case COLOR_DEPTH_111111:
+ output_bpc = 11;
+ break;
+ default:
+ output_bpc = 8;
+ break;
+ }
+
+ switch (timing->pixel_encoding) {
+ case PIXEL_ENCODING_RGB:
+ case PIXEL_ENCODING_YCBCR444:
+ dout->output_format = dm_444;
+ dout->output_bpp = output_bpc * 3;
+ break;
+ case PIXEL_ENCODING_YCBCR420:
+ dout->output_format = dm_420;
+ dout->output_bpp = (output_bpc * 3.0) / 2;
+ break;
+ case PIXEL_ENCODING_YCBCR422:
+ if (timing->flags.DSC && !timing->dsc_cfg.ycbcr422_simple)
+ dout->output_format = dm_n422;
+ else
+ dout->output_format = dm_s422;
+ dout->output_bpp = output_bpc * 2;
+ break;
+ default:
+ dout->output_format = dm_444;
+ dout->output_bpp = output_bpc * 3;
+ }
+}
+
+static enum source_format_class dc_source_format_to_dml_source_format(enum surface_pixel_format dc_format)
+{
+ enum source_format_class dml_format = dm_444_32;
+
+ switch (dc_format) {
+ case SURFACE_PIXEL_FORMAT_VIDEO_420_YCbCr:
+ case SURFACE_PIXEL_FORMAT_VIDEO_420_YCrCb:
+ dml_format = dm_420_8;
+ break;
+ case SURFACE_PIXEL_FORMAT_VIDEO_420_10bpc_YCbCr:
+ case SURFACE_PIXEL_FORMAT_VIDEO_420_10bpc_YCrCb:
+ dml_format = dm_420_10;
+ break;
+ case SURFACE_PIXEL_FORMAT_GRPH_ARGB16161616:
+ case SURFACE_PIXEL_FORMAT_GRPH_ARGB16161616F:
+ case SURFACE_PIXEL_FORMAT_GRPH_ABGR16161616F:
+ dml_format = dm_444_64;
+ break;
+ case SURFACE_PIXEL_FORMAT_GRPH_ARGB1555:
+ case SURFACE_PIXEL_FORMAT_GRPH_RGB565:
+ dml_format = dm_444_16;
+ break;
+ case SURFACE_PIXEL_FORMAT_GRPH_PALETA_256_COLORS:
+ dml_format = dm_444_8;
+ break;
+ case SURFACE_PIXEL_FORMAT_GRPH_RGBE_ALPHA:
+ dml_format = dm_rgbe_alpha;
+ break;
+ default:
+ dml_format = dm_444_32;
+ break;
+ }
+
+ return dml_format;
+}
+
+#endif
diff --git a/drivers/gpu/drm/amd/display/dc/dml/dsc/rc_calc_fpu.c b/drivers/gpu/drm/amd/display/dc/dml/dsc/rc_calc_fpu.c
index 3ee858f..ec636d0 100644
--- a/drivers/gpu/drm/amd/display/dc/dml/dsc/rc_calc_fpu.c
+++ b/drivers/gpu/drm/amd/display/dc/dml/dsc/rc_calc_fpu.c
@@ -61,16 +61,6 @@ static double dsc_roundf(double num)
return (int)(num);
}
-static double dsc_ceil(double num)
-{
- double retval = (int)num;
-
- if (retval != num && num > 0)
- retval = num + 1;
-
- return (int)retval;
-}
-
static void get_qp_set(qp_set qps, enum colour_mode cm, enum bits_per_comp bpc,
enum max_min max_min, float bpp)
{
@@ -103,7 +93,7 @@ static void get_qp_set(qp_set qps, enum colour_mode cm, enum bits_per_comp bpc,
TABLE_CASE(420, 12, min);
}
- if (table == 0)
+ if (!table)
return;
index = (bpp - table[0].bpp) * 2;
@@ -268,24 +258,3 @@ void _do_calc_rc_params(struct rc_params *rc,
rc->rc_buf_thresh[13] = 8064;
}
-u32 _do_bytes_per_pixel_calc(int slice_width,
- u16 drm_bpp,
- bool is_navite_422_or_420)
-{
- float bpp;
- u32 bytes_per_pixel;
- double d_bytes_per_pixel;
-
- dc_assert_fp_enabled();
-
- bpp = ((float)drm_bpp / 16.0);
- d_bytes_per_pixel = dsc_ceil(bpp * slice_width / 8.0) / slice_width;
- // TODO: Make sure the formula for calculating this is precise (ceiling
- // vs. floor, and at what point they should be applied)
- if (is_navite_422_or_420)
- d_bytes_per_pixel /= 2;
-
- bytes_per_pixel = (u32)dsc_ceil(d_bytes_per_pixel * 0x10000000);
-
- return bytes_per_pixel;
-}
diff --git a/drivers/gpu/drm/amd/display/dc/dml/dsc/rc_calc_fpu.h b/drivers/gpu/drm/amd/display/dc/dml/dsc/rc_calc_fpu.h
index b93b954..cad244c 100644
--- a/drivers/gpu/drm/amd/display/dc/dml/dsc/rc_calc_fpu.h
+++ b/drivers/gpu/drm/amd/display/dc/dml/dsc/rc_calc_fpu.h
@@ -78,10 +78,6 @@ struct qp_entry {
typedef struct qp_entry qp_table[];
-u32 _do_bytes_per_pixel_calc(int slice_width,
- u16 drm_bpp,
- bool is_navite_422_or_420);
-
void _do_calc_rc_params(struct rc_params *rc,
enum colour_mode cm,
enum bits_per_comp bpc,
diff --git a/drivers/gpu/drm/amd/display/dc/dsc/dc_dsc.c b/drivers/gpu/drm/amd/display/dc/dsc/dc_dsc.c
index 0321b44..9c74564 100644
--- a/drivers/gpu/drm/amd/display/dc/dsc/dc_dsc.c
+++ b/drivers/gpu/drm/amd/display/dc/dsc/dc_dsc.c
@@ -455,6 +455,7 @@ static bool intersect_dsc_caps(
if (pixel_encoding == PIXEL_ENCODING_YCBCR422 || pixel_encoding == PIXEL_ENCODING_YCBCR420)
dsc_common_caps->bpp_increment_div = min(dsc_common_caps->bpp_increment_div, (uint32_t)8);
+ dsc_common_caps->edp_sink_max_bits_per_pixel = dsc_sink_caps->edp_max_bits_per_pixel;
dsc_common_caps->is_dp = dsc_sink_caps->is_dp;
return true;
}
@@ -513,6 +514,13 @@ static bool decide_dsc_bandwidth_range(
range->min_target_bpp_x16 = preferred_bpp_x16;
}
}
+ /* TODO - make this value generic to all signal types */
+ else if (dsc_caps->edp_sink_max_bits_per_pixel) {
+ /* apply max bpp limitation from edp sink */
+ range->max_target_bpp_x16 = MIN(dsc_caps->edp_sink_max_bits_per_pixel,
+ max_bpp_x16);
+ range->min_target_bpp_x16 = min_bpp_x16;
+ }
else {
range->max_target_bpp_x16 = max_bpp_x16;
range->min_target_bpp_x16 = min_bpp_x16;
@@ -574,7 +582,7 @@ static bool decide_dsc_target_bpp_x16(
return *target_bpp_x16 != 0;
}
-#define MIN_AVAILABLE_SLICES_SIZE 4
+#define MIN_AVAILABLE_SLICES_SIZE 6
static int get_available_dsc_slices(union dsc_enc_slice_caps slice_caps, int *available_slices)
{
@@ -860,6 +868,10 @@ static bool setup_dsc_config(
min_slices_h = 0; // DSC TODO: Maybe try increasing the number of slices first?
is_dsc_possible = (min_slices_h <= max_slices_h);
+
+ if (min_slices_h == 0 && max_slices_h == 0)
+ is_dsc_possible = false;
+
if (!is_dsc_possible)
goto done;
diff --git a/drivers/gpu/drm/amd/display/dc/dsc/rc_calc.c b/drivers/gpu/drm/amd/display/dc/dsc/rc_calc.c
index b19d3ae..e97cf09 100644
--- a/drivers/gpu/drm/amd/display/dc/dsc/rc_calc.c
+++ b/drivers/gpu/drm/amd/display/dc/dsc/rc_calc.c
@@ -60,31 +60,3 @@ void calc_rc_params(struct rc_params *rc, const struct drm_dsc_config *pps)
pps->dsc_version_minor);
DC_FP_END();
}
-
-/**
- * calc_dsc_bytes_per_pixel - calculate bytes per pixel
- * @pps: DRM struct with all required DSC values
- *
- * Based on the information inside drm_dsc_config, this function calculates the
- * total of bytes per pixel.
- *
- * @note This calculation requires float point operation, most of it executes
- * under kernel_fpu_{begin,end}.
- *
- * Return:
- * Return the number of bytes per pixel
- */
-u32 calc_dsc_bytes_per_pixel(const struct drm_dsc_config *pps)
-
-{
- u32 ret;
- u16 drm_bpp = pps->bits_per_pixel;
- int slice_width = pps->slice_width;
- bool is_navite_422_or_420 = pps->native_422 || pps->native_420;
-
- DC_FP_START();
- ret = _do_bytes_per_pixel_calc(slice_width, drm_bpp,
- is_navite_422_or_420);
- DC_FP_END();
- return ret;
-}
diff --git a/drivers/gpu/drm/amd/display/dc/dsc/rc_calc.h b/drivers/gpu/drm/amd/display/dc/dsc/rc_calc.h
index c2340e0..80921c1 100644
--- a/drivers/gpu/drm/amd/display/dc/dsc/rc_calc.h
+++ b/drivers/gpu/drm/amd/display/dc/dsc/rc_calc.h
@@ -30,7 +30,6 @@
#include "dml/dsc/rc_calc_fpu.h"
void calc_rc_params(struct rc_params *rc, const struct drm_dsc_config *pps);
-u32 calc_dsc_bytes_per_pixel(const struct drm_dsc_config *pps);
#endif
diff --git a/drivers/gpu/drm/amd/display/dc/dsc/rc_calc_dpi.c b/drivers/gpu/drm/amd/display/dc/dsc/rc_calc_dpi.c
index 1e19dd6..7e306aa 100644
--- a/drivers/gpu/drm/amd/display/dc/dsc/rc_calc_dpi.c
+++ b/drivers/gpu/drm/amd/display/dc/dsc/rc_calc_dpi.c
@@ -100,8 +100,7 @@ int dscc_compute_dsc_parameters(const struct drm_dsc_config *pps, struct dsc_par
int ret;
struct rc_params rc;
struct drm_dsc_config dsc_cfg;
-
- dsc_params->bytes_per_pixel = calc_dsc_bytes_per_pixel(pps);
+ unsigned long long tmp;
calc_rc_params(&rc, pps);
dsc_params->pps = *pps;
@@ -113,6 +112,9 @@ int dscc_compute_dsc_parameters(const struct drm_dsc_config *pps, struct dsc_par
dsc_cfg.mux_word_size = dsc_params->pps.bits_per_component <= 10 ? 48 : 64;
ret = drm_dsc_compute_rc_parameters(&dsc_cfg);
+ tmp = (unsigned long long)dsc_cfg.slice_chunk_size * 0x10000000 + (dsc_cfg.slice_width - 1);
+ do_div(tmp, (uint32_t)dsc_cfg.slice_width); //ROUND-UP
+ dsc_params->bytes_per_pixel = (uint32_t)tmp;
copy_pps_fields(&dsc_params->pps, &dsc_cfg);
dsc_params->rc_buffer_model_size = dsc_cfg.rc_bits;
diff --git a/drivers/gpu/drm/amd/display/dc/inc/core_status.h b/drivers/gpu/drm/amd/display/dc/inc/core_status.h
index d34b0b0..444182a 100644
--- a/drivers/gpu/drm/amd/display/dc/inc/core_status.h
+++ b/drivers/gpu/drm/amd/display/dc/inc/core_status.h
@@ -53,6 +53,8 @@ enum dc_status {
DC_NOT_SUPPORTED = 24,
DC_UNSUPPORTED_VALUE = 25,
+ DC_NO_LINK_ENC_RESOURCE = 26,
+
DC_ERROR_UNEXPECTED = -1
};
diff --git a/drivers/gpu/drm/amd/display/dc/inc/core_types.h b/drivers/gpu/drm/amd/display/dc/inc/core_types.h
index 6fc6488..8902800 100644
--- a/drivers/gpu/drm/amd/display/dc/inc/core_types.h
+++ b/drivers/gpu/drm/amd/display/dc/inc/core_types.h
@@ -334,6 +334,20 @@ struct plane_resource {
struct dcn_fe_bandwidth bw;
};
+#if defined(CONFIG_DRM_AMD_DC_DCN)
+#define LINK_RES_HPO_DP_REC_MAP__MASK 0xFFFF
+#define LINK_RES_HPO_DP_REC_MAP__SHIFT 0
+#endif
+
+/* all mappable hardware resources used to enable a link */
+struct link_resource {
+#if defined(CONFIG_DRM_AMD_DC_DCN)
+ struct hpo_dp_link_encoder *hpo_dp_link_enc;
+#else
+ void *dummy;
+#endif
+};
+
union pipe_update_flags {
struct {
uint32_t enable : 1;
@@ -361,6 +375,7 @@ struct pipe_ctx {
struct plane_resource plane_res;
struct stream_resource stream_res;
+ struct link_resource link_res;
struct clock_source *clock_source;
@@ -411,6 +426,8 @@ struct resource_context {
struct link_enc_cfg_context link_enc_cfg_ctx;
#if defined(CONFIG_DRM_AMD_DC_DCN)
bool is_hpo_dp_stream_enc_acquired[MAX_HPO_DP2_ENCODERS];
+ unsigned int hpo_dp_link_enc_to_link_idx[MAX_HPO_DP2_LINK_ENCODERS];
+ int hpo_dp_link_enc_ref_cnts[MAX_HPO_DP2_LINK_ENCODERS];
#endif
#if defined(CONFIG_DRM_AMD_DC_DCN)
bool is_mpc_3dlut_acquired[MAX_PIPES];
diff --git a/drivers/gpu/drm/amd/display/dc/inc/dc_link_dp.h b/drivers/gpu/drm/amd/display/dc/inc/dc_link_dp.h
index a6d3d85..cd52813 100644
--- a/drivers/gpu/drm/amd/display/dc/inc/dc_link_dp.h
+++ b/drivers/gpu/drm/amd/display/dc/inc/dc_link_dp.h
@@ -56,16 +56,19 @@ enum {
bool dp_verify_link_cap(
struct dc_link *link,
+ const struct link_resource *link_res,
struct dc_link_settings *known_limit_link_setting,
int *fail_count);
bool dp_verify_link_cap_with_retries(
struct dc_link *link,
+ const struct link_resource *link_res,
struct dc_link_settings *known_limit_link_setting,
int attempts);
bool dp_verify_mst_link_cap(
- struct dc_link *link);
+ struct dc_link *link,
+ const struct link_resource *link_res);
bool dp_validate_mode_timing(
struct dc_link *link,
@@ -168,8 +171,9 @@ uint8_t dc_dp_initialize_scrambling_data_symbols(
struct dc_link *link,
enum dc_dp_training_pattern pattern);
-enum dc_status dp_set_fec_ready(struct dc_link *link, bool ready);
+enum dc_status dp_set_fec_ready(struct dc_link *link, const struct link_resource *link_res, bool ready);
void dp_set_fec_enable(struct dc_link *link, bool enable);
+struct link_encoder *dp_get_link_enc(struct dc_link *link);
bool dp_set_dsc_enable(struct pipe_ctx *pipe_ctx, bool enable);
bool dp_set_dsc_pps_sdp(struct pipe_ctx *pipe_ctx, bool enable, bool immediate_update);
void dp_set_dsc_on_stream(struct pipe_ctx *pipe_ctx, bool enable);
@@ -210,11 +214,16 @@ bool dpcd_poll_for_allocation_change_trigger(struct dc_link *link);
struct fixed31_32 calculate_sst_avg_time_slots_per_mtp(
const struct dc_stream_state *stream,
const struct dc_link *link);
-void enable_dp_hpo_output(struct dc_link *link, const struct dc_link_settings *link_settings);
-void disable_dp_hpo_output(struct dc_link *link, enum signal_type signal);
+void enable_dp_hpo_output(struct dc_link *link,
+ const struct link_resource *link_res,
+ const struct dc_link_settings *link_settings);
+void disable_dp_hpo_output(struct dc_link *link,
+ const struct link_resource *link_res,
+ enum signal_type signal);
void setup_dp_hpo_stream(struct pipe_ctx *pipe_ctx, bool enable);
bool is_dp_128b_132b_signal(struct pipe_ctx *pipe_ctx);
void reset_dp_hpo_stream_encoders_for_link(struct dc_link *link);
bool dp_retrieve_lttpr_cap(struct dc_link *link);
+void edp_panel_backlight_power_on(struct dc_link *link);
#endif /* __DC_LINK_DP_H__ */
diff --git a/drivers/gpu/drm/amd/display/dc/inc/dc_link_dpia.h b/drivers/gpu/drm/amd/display/dc/inc/dc_link_dpia.h
index 974d703..74dafd0f 100644
--- a/drivers/gpu/drm/amd/display/dc/inc/dc_link_dpia.h
+++ b/drivers/gpu/drm/amd/display/dc/inc/dc_link_dpia.h
@@ -91,8 +91,9 @@ enum dc_status dpcd_get_tunneling_device_data(struct dc_link *link);
* DPIA equivalent of dc_link_dp_perfrorm_link_training.
* Aborts link training upon detection of sink unplug.
*/
-enum link_training_result
-dc_link_dpia_perform_link_training(struct dc_link *link,
+enum link_training_result dc_link_dpia_perform_link_training(
+ struct dc_link *link,
+ const struct link_resource *link_res,
const struct dc_link_settings *link_setting,
bool skip_video_pattern);
diff --git a/drivers/gpu/drm/amd/display/dc/inc/dcn_calcs.h b/drivers/gpu/drm/amd/display/dc/inc/dcn_calcs.h
index 806f304..337c016 100644
--- a/drivers/gpu/drm/amd/display/dc/inc/dcn_calcs.h
+++ b/drivers/gpu/drm/amd/display/dc/inc/dcn_calcs.h
@@ -619,7 +619,7 @@ struct dcn_ip_params {
};
extern const struct dcn_ip_params dcn10_ip_defaults;
-bool dcn_validate_bandwidth(
+bool dcn10_validate_bandwidth(
struct dc *dc,
struct dc_state *context,
bool fast_validate);
diff --git a/drivers/gpu/drm/amd/display/dc/inc/dml_wrapper.h b/drivers/gpu/drm/amd/display/dc/inc/dml_wrapper.h
new file mode 100644
index 0000000..5dcfbd8
--- /dev/null
+++ b/drivers/gpu/drm/amd/display/dc/inc/dml_wrapper.h
@@ -0,0 +1,34 @@
+/*
+ * Copyright 2015 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ * Authors: AMD
+ *
+ */
+
+#ifndef DML_WRAPPER_H_
+#define DML_WRAPPER_H_
+
+#include "dc.h"
+#include "dml/display_mode_vba.h"
+
+bool dml_validate(struct dc *dc, struct dc_state *context, bool fast_validate);
+
+#endif
diff --git a/drivers/gpu/drm/amd/display/dc/inc/hw/clk_mgr.h b/drivers/gpu/drm/amd/display/dc/inc/hw/clk_mgr.h
index a17e5de..c920c4b 100644
--- a/drivers/gpu/drm/amd/display/dc/inc/hw/clk_mgr.h
+++ b/drivers/gpu/drm/amd/display/dc/inc/hw/clk_mgr.h
@@ -211,6 +211,8 @@ struct dummy_pstate_entry {
struct clk_bw_params {
unsigned int vram_type;
unsigned int num_channels;
+ unsigned int dispclk_vco_khz;
+ unsigned int dc_mode_softmax_memclk;
struct clk_limit_table clk_table;
struct wm_table wm_table;
struct dummy_pstate_entry dummy_pstate_table[4];
@@ -261,6 +263,10 @@ struct clk_mgr_funcs {
/* Send message to PMFW to set hard max memclk frequency to highest DPM */
void (*set_hard_max_memclk)(struct clk_mgr *clk_mgr);
+ /* Custom set a memclk freq range*/
+ void (*set_max_memclk)(struct clk_mgr *clk_mgr, unsigned int memclk_mhz);
+ void (*set_min_memclk)(struct clk_mgr *clk_mgr, unsigned int memclk_mhz);
+
/* Get current memclk states from PMFW, update relevant structures */
void (*get_memclk_states_from_smu)(struct clk_mgr *clk_mgr);
@@ -274,6 +280,7 @@ struct clk_mgr {
struct dc_clocks clks;
bool psr_allow_active_cache;
bool force_smu_not_present;
+ bool dc_mode_softmax_enabled;
int dprefclk_khz; // Used by program pixel clock in clock source funcs, need to figureout where this goes
int dentist_vco_freq_khz;
struct clk_state_registers_and_bypass boot_snapshot;
diff --git a/drivers/gpu/drm/amd/display/dc/inc/hw/dsc.h b/drivers/gpu/drm/amd/display/dc/inc/hw/dsc.h
index f94135c..346f0ba 100644
--- a/drivers/gpu/drm/amd/display/dc/inc/hw/dsc.h
+++ b/drivers/gpu/drm/amd/display/dc/inc/hw/dsc.h
@@ -61,6 +61,8 @@ struct dcn_dsc_state {
uint32_t dsc_pic_height;
uint32_t dsc_slice_bpg_offset;
uint32_t dsc_chunk_size;
+ uint32_t dsc_fw_en;
+ uint32_t dsc_opp_source;
};
@@ -88,6 +90,7 @@ struct dsc_enc_caps {
int32_t max_total_throughput_mps; /* Maximum total throughput with all the slices combined */
int32_t max_slice_width;
uint32_t bpp_increment_div; /* bpp increment divisor, e.g. if 16, it's 1/16th of a bit */
+ uint32_t edp_sink_max_bits_per_pixel;
bool is_dp;
};
diff --git a/drivers/gpu/drm/amd/display/dc/inc/hw/hubp.h b/drivers/gpu/drm/amd/display/dc/inc/hw/hubp.h
index 80e1a32..2c03158 100644
--- a/drivers/gpu/drm/amd/display/dc/inc/hw/hubp.h
+++ b/drivers/gpu/drm/amd/display/dc/inc/hw/hubp.h
@@ -139,6 +139,7 @@ struct hubp_funcs {
bool (*hubp_is_flip_pending)(struct hubp *hubp);
void (*set_blank)(struct hubp *hubp, bool blank);
+ void (*set_blank_regs)(struct hubp *hubp, bool blank);
void (*set_hubp_blank_en)(struct hubp *hubp, bool blank);
void (*set_cursor_attributes)(
diff --git a/drivers/gpu/drm/amd/display/dc/inc/hw/link_encoder.h b/drivers/gpu/drm/amd/display/dc/inc/hw/link_encoder.h
index bb0e917..2ce15cd 100644
--- a/drivers/gpu/drm/amd/display/dc/inc/hw/link_encoder.h
+++ b/drivers/gpu/drm/amd/display/dc/inc/hw/link_encoder.h
@@ -268,7 +268,8 @@ struct hpo_dp_link_encoder_funcs {
void (*enable_link_phy)(struct hpo_dp_link_encoder *enc,
const struct dc_link_settings *link_settings,
- enum transmitter transmitter);
+ enum transmitter transmitter,
+ enum hpd_source_id hpd_source);
void (*disable_link_phy)(struct hpo_dp_link_encoder *link_enc,
enum signal_type signal);
diff --git a/drivers/gpu/drm/amd/display/dc/inc/hw/stream_encoder.h b/drivers/gpu/drm/amd/display/dc/inc/hw/stream_encoder.h
index c88e113..073f8b6 100644
--- a/drivers/gpu/drm/amd/display/dc/inc/hw/stream_encoder.h
+++ b/drivers/gpu/drm/amd/display/dc/inc/hw/stream_encoder.h
@@ -164,6 +164,10 @@ struct stream_encoder_funcs {
void (*stop_dp_info_packets)(
struct stream_encoder *enc);
+ void (*reset_fifo)(
+ struct stream_encoder *enc
+ );
+
void (*dp_blank)(
struct dc_link *link,
struct stream_encoder *enc);
diff --git a/drivers/gpu/drm/amd/display/dc/inc/hw/timing_generator.h b/drivers/gpu/drm/amd/display/dc/inc/hw/timing_generator.h
index 7390baf..c29320b 100644
--- a/drivers/gpu/drm/amd/display/dc/inc/hw/timing_generator.h
+++ b/drivers/gpu/drm/amd/display/dc/inc/hw/timing_generator.h
@@ -290,6 +290,8 @@ struct timing_generator_funcs {
enum optc_dsc_mode dsc_mode,
uint32_t dsc_bytes_per_pixel,
uint32_t dsc_slice_width);
+ void (*get_dsc_status)(struct timing_generator *optc,
+ uint32_t *dsc_mode);
void (*set_odm_bypass)(struct timing_generator *optc, const struct dc_crtc_timing *dc_crtc_timing);
void (*set_odm_combine)(struct timing_generator *optc, int *opp_id, int opp_cnt,
struct dc_crtc_timing *timing);
diff --git a/drivers/gpu/drm/amd/display/dc/inc/hw_sequencer.h b/drivers/gpu/drm/amd/display/dc/inc/hw_sequencer.h
index d50f4bd..05053f3 100644
--- a/drivers/gpu/drm/amd/display/dc/inc/hw_sequencer.h
+++ b/drivers/gpu/drm/amd/display/dc/inc/hw_sequencer.h
@@ -64,6 +64,7 @@ struct hw_sequencer_funcs {
enum dc_status (*apply_ctx_to_hw)(struct dc *dc,
struct dc_state *context);
void (*disable_plane)(struct dc *dc, struct pipe_ctx *pipe_ctx);
+ void (*disable_pixel_data)(struct dc *dc, struct pipe_ctx *pipe_ctx, bool blank);
void (*apply_ctx_for_surface)(struct dc *dc,
const struct dc_stream_state *stream,
int num_planes, struct dc_state *context);
diff --git a/drivers/gpu/drm/amd/display/dc/inc/link_enc_cfg.h b/drivers/gpu/drm/amd/display/dc/inc/link_enc_cfg.h
index 10dcf6a..a4e43b4 100644
--- a/drivers/gpu/drm/amd/display/dc/inc/link_enc_cfg.h
+++ b/drivers/gpu/drm/amd/display/dc/inc/link_enc_cfg.h
@@ -36,7 +36,7 @@
* Initialise link encoder resource tracking.
*/
void link_enc_cfg_init(
- struct dc *dc,
+ const struct dc *dc,
struct dc_state *state);
/*
diff --git a/drivers/gpu/drm/amd/display/dc/inc/link_hwss.h b/drivers/gpu/drm/amd/display/dc/inc/link_hwss.h
index ba664bc..69d6376 100644
--- a/drivers/gpu/drm/amd/display/dc/inc/link_hwss.h
+++ b/drivers/gpu/drm/amd/display/dc/inc/link_hwss.h
@@ -32,6 +32,7 @@ struct gpio *get_hpd_gpio(struct dc_bios *dcb,
void dp_enable_link_phy(
struct dc_link *link,
+ const struct link_resource *link_res,
enum signal_type signal,
enum clock_source_id clock_source,
const struct dc_link_settings *link_settings);
@@ -42,22 +43,27 @@ void edp_add_delay_for_T9(struct dc_link *link);
bool edp_receiver_ready_T9(struct dc_link *link);
bool edp_receiver_ready_T7(struct dc_link *link);
-void dp_disable_link_phy(struct dc_link *link, enum signal_type signal);
+void dp_disable_link_phy(struct dc_link *link, const struct link_resource *link_res,
+ enum signal_type signal);
-void dp_disable_link_phy_mst(struct dc_link *link, enum signal_type signal);
+void dp_disable_link_phy_mst(struct dc_link *link, const struct link_resource *link_res,
+ enum signal_type signal);
bool dp_set_hw_training_pattern(
struct dc_link *link,
+ const struct link_resource *link_res,
enum dc_dp_training_pattern pattern,
uint32_t offset);
void dp_set_hw_lane_settings(
struct dc_link *link,
+ const struct link_resource *link_res,
const struct link_training_settings *link_settings,
uint32_t offset);
void dp_set_hw_test_pattern(
struct dc_link *link,
+ const struct link_resource *link_res,
enum dp_test_pattern test_pattern,
uint8_t *custom_pattern,
uint32_t custom_pattern_size);
diff --git a/drivers/gpu/drm/amd/display/dc/inc/resource.h b/drivers/gpu/drm/amd/display/dc/inc/resource.h
index 372c089..e589cbe 100644
--- a/drivers/gpu/drm/amd/display/dc/inc/resource.h
+++ b/drivers/gpu/drm/amd/display/dc/inc/resource.h
@@ -202,8 +202,10 @@ int get_num_mpc_splits(struct pipe_ctx *pipe);
int get_num_odm_splits(struct pipe_ctx *pipe);
#if defined(CONFIG_DRM_AMD_DC_DCN)
-struct hpo_dp_link_encoder *resource_get_unused_hpo_dp_link_encoder(
- const struct resource_pool *pool);
+struct hpo_dp_link_encoder *resource_get_hpo_dp_link_enc_for_det_lt(
+ const struct resource_context *res_ctx,
+ const struct resource_pool *pool,
+ const struct dc_link *link);
#endif
#endif /* DRIVERS_GPU_DRM_AMD_DC_DEV_DC_INC_RESOURCE_H_ */
diff --git a/drivers/gpu/drm/amd/display/dc/irq/dce110/irq_service_dce110.c b/drivers/gpu/drm/amd/display/dc/irq/dce110/irq_service_dce110.c
index 378cc11a..6b5fedd 100644
--- a/drivers/gpu/drm/amd/display/dc/irq/dce110/irq_service_dce110.c
+++ b/drivers/gpu/drm/amd/display/dc/irq/dce110/irq_service_dce110.c
@@ -185,16 +185,18 @@ bool dal_irq_service_dummy_set(struct irq_service *irq_service,
const struct irq_source_info *info,
bool enable)
{
- DC_LOG_ERROR("%s: called for non-implemented irq source\n",
- __func__);
+ DC_LOG_ERROR("%s: called for non-implemented irq source, src_id=%u, ext_id=%u\n",
+ __func__, info->src_id, info->ext_id);
+
return false;
}
bool dal_irq_service_dummy_ack(struct irq_service *irq_service,
const struct irq_source_info *info)
{
- DC_LOG_ERROR("%s: called for non-implemented irq source\n",
- __func__);
+ DC_LOG_ERROR("%s: called for non-implemented irq source, src_id=%u, ext_id=%u\n",
+ __func__, info->src_id, info->ext_id);
+
return false;
}
diff --git a/drivers/gpu/drm/amd/display/dc/irq/dcn10/irq_service_dcn10.c b/drivers/gpu/drm/amd/display/dc/irq/dcn10/irq_service_dcn10.c
index 34f43cb..cf072e2 100644
--- a/drivers/gpu/drm/amd/display/dc/irq/dcn10/irq_service_dcn10.c
+++ b/drivers/gpu/drm/amd/display/dc/irq/dcn10/irq_service_dcn10.c
@@ -40,10 +40,9 @@
#include "ivsrcid/dcn/irqsrcs_dcn_1_0.h"
-enum dc_irq_source to_dal_irq_source_dcn10(
- struct irq_service *irq_service,
- uint32_t src_id,
- uint32_t ext_id)
+static enum dc_irq_source to_dal_irq_source_dcn10(struct irq_service *irq_service,
+ uint32_t src_id,
+ uint32_t ext_id)
{
switch (src_id) {
case DCN_1_0__SRCID__DC_D1_OTG_VSTARTUP:
diff --git a/drivers/gpu/drm/amd/display/dc/irq/dcn201/irq_service_dcn201.c b/drivers/gpu/drm/amd/display/dc/irq/dcn201/irq_service_dcn201.c
index a47f686..aa708b6 100644
--- a/drivers/gpu/drm/amd/display/dc/irq/dcn201/irq_service_dcn201.c
+++ b/drivers/gpu/drm/amd/display/dc/irq/dcn201/irq_service_dcn201.c
@@ -39,10 +39,9 @@
#include "ivsrcid/dcn/irqsrcs_dcn_1_0.h"
-enum dc_irq_source to_dal_irq_source_dcn201(
- struct irq_service *irq_service,
- uint32_t src_id,
- uint32_t ext_id)
+static enum dc_irq_source to_dal_irq_source_dcn201(struct irq_service *irq_service,
+ uint32_t src_id,
+ uint32_t ext_id)
{
switch (src_id) {
case DCN_1_0__SRCID__DC_D1_OTG_VSTARTUP:
diff --git a/drivers/gpu/drm/amd/display/dc/irq/dcn21/irq_service_dcn21.c b/drivers/gpu/drm/amd/display/dc/irq/dcn21/irq_service_dcn21.c
index 78940cb..23529453 100644
--- a/drivers/gpu/drm/amd/display/dc/irq/dcn21/irq_service_dcn21.c
+++ b/drivers/gpu/drm/amd/display/dc/irq/dcn21/irq_service_dcn21.c
@@ -40,10 +40,9 @@
#include "ivsrcid/dcn/irqsrcs_dcn_1_0.h"
-enum dc_irq_source to_dal_irq_source_dcn21(
- struct irq_service *irq_service,
- uint32_t src_id,
- uint32_t ext_id)
+static enum dc_irq_source to_dal_irq_source_dcn21(struct irq_service *irq_service,
+ uint32_t src_id,
+ uint32_t ext_id)
{
switch (src_id) {
case DCN_1_0__SRCID__DC_D1_OTG_VSTARTUP:
diff --git a/drivers/gpu/drm/amd/display/dc/irq/dcn31/irq_service_dcn31.c b/drivers/gpu/drm/amd/display/dc/irq/dcn31/irq_service_dcn31.c
index 38e0ade..1b88e4e 100644
--- a/drivers/gpu/drm/amd/display/dc/irq/dcn31/irq_service_dcn31.c
+++ b/drivers/gpu/drm/amd/display/dc/irq/dcn31/irq_service_dcn31.c
@@ -36,10 +36,9 @@
#include "ivsrcid/dcn/irqsrcs_dcn_1_0.h"
-enum dc_irq_source to_dal_irq_source_dcn31(
- struct irq_service *irq_service,
- uint32_t src_id,
- uint32_t ext_id)
+static enum dc_irq_source to_dal_irq_source_dcn31(struct irq_service *irq_service,
+ uint32_t src_id,
+ uint32_t ext_id)
{
switch (src_id) {
case DCN_1_0__SRCID__DC_D1_OTG_VSTARTUP:
diff --git a/drivers/gpu/drm/amd/display/dmub/dmub_srv.h b/drivers/gpu/drm/amd/display/dmub/dmub_srv.h
index cd204ee..83855b8 100644
--- a/drivers/gpu/drm/amd/display/dmub/dmub_srv.h
+++ b/drivers/gpu/drm/amd/display/dmub/dmub_srv.h
@@ -360,6 +360,8 @@ struct dmub_srv_hw_funcs {
uint32_t (*get_gpint_dataout)(struct dmub_srv *dmub);
+ void (*clear_inbox0_ack_register)(struct dmub_srv *dmub);
+ uint32_t (*read_inbox0_ack_register)(struct dmub_srv *dmub);
void (*send_inbox0_cmd)(struct dmub_srv *dmub, union dmub_inbox0_data_register data);
uint32_t (*get_current_time)(struct dmub_srv *dmub);
@@ -409,6 +411,7 @@ struct dmub_srv {
struct dmub_srv_base_funcs funcs;
struct dmub_srv_hw_funcs hw_funcs;
struct dmub_rb inbox1_rb;
+ uint32_t inbox1_last_wptr;
/**
* outbox1_rb is accessed without locks (dal & dc)
* and to be used only in dmub_srv_stat_get_notification()
@@ -735,6 +738,45 @@ bool dmub_srv_get_diagnostic_data(struct dmub_srv *dmub, struct dmub_diagnostic_
bool dmub_srv_should_detect(struct dmub_srv *dmub);
+/**
+ * dmub_srv_send_inbox0_cmd() - Send command to DMUB using INBOX0
+ * @dmub: the dmub service
+ * @data: the data to be sent in the INBOX0 command
+ *
+ * Send command by writing directly to INBOX0 WPTR
+ *
+ * Return:
+ * DMUB_STATUS_OK - success
+ * DMUB_STATUS_INVALID - hw_init false or hw function does not exist
+ */
+enum dmub_status dmub_srv_send_inbox0_cmd(struct dmub_srv *dmub, union dmub_inbox0_data_register data);
+
+/**
+ * dmub_srv_wait_for_inbox0_ack() - wait for DMUB to ACK INBOX0 command
+ * @dmub: the dmub service
+ * @timeout_us: the maximum number of microseconds to wait
+ *
+ * Wait for DMUB to ACK the INBOX0 message
+ *
+ * Return:
+ * DMUB_STATUS_OK - success
+ * DMUB_STATUS_INVALID - hw_init false or hw function does not exist
+ * DMUB_STATUS_TIMEOUT - wait for ack timed out
+ */
+enum dmub_status dmub_srv_wait_for_inbox0_ack(struct dmub_srv *dmub, uint32_t timeout_us);
+
+/**
+ * dmub_srv_wait_for_inbox0_ack() - clear ACK register for INBOX0
+ * @dmub: the dmub service
+ *
+ * Clear ACK register for INBOX0
+ *
+ * Return:
+ * DMUB_STATUS_OK - success
+ * DMUB_STATUS_INVALID - hw_init false or hw function does not exist
+ */
+enum dmub_status dmub_srv_clear_inbox0_ack(struct dmub_srv *dmub);
+
#if defined(__cplusplus)
}
#endif
diff --git a/drivers/gpu/drm/amd/display/dmub/inc/dmub_cmd.h b/drivers/gpu/drm/amd/display/dmub/inc/dmub_cmd.h
index c29a67c..873ecd0 100644
--- a/drivers/gpu/drm/amd/display/dmub/inc/dmub_cmd.h
+++ b/drivers/gpu/drm/amd/display/dmub/inc/dmub_cmd.h
@@ -46,10 +46,10 @@
/* Firmware versioning. */
#ifdef DMUB_EXPOSE_VERSION
-#define DMUB_FW_VERSION_GIT_HASH 0x1d82d23e
+#define DMUB_FW_VERSION_GIT_HASH 0xbaf06b95
#define DMUB_FW_VERSION_MAJOR 0
#define DMUB_FW_VERSION_MINOR 0
-#define DMUB_FW_VERSION_REVISION 91
+#define DMUB_FW_VERSION_REVISION 98
#define DMUB_FW_VERSION_TEST 0
#define DMUB_FW_VERSION_VBIOS 0
#define DMUB_FW_VERSION_HOTFIX 0
@@ -173,13 +173,6 @@ extern "C" {
#endif
/**
- * Number of nanoseconds per DMUB tick.
- * DMCUB_TIMER_CURRENT increments in DMUB ticks, which are 10ns by default.
- * If DMCUB_TIMER_WINDOW is non-zero this will no longer be true.
- */
-#define NS_PER_DMUB_TICK 10
-
-/**
* union dmub_addr - DMUB physical/virtual 64-bit address.
*/
union dmub_addr {
@@ -208,10 +201,9 @@ union dmub_psr_debug_flags {
uint32_t use_hw_lock_mgr : 1;
/**
- * Unused.
- * TODO: Remove.
+ * Use TPS3 signal when restore main link.
*/
- uint32_t log_line_nums : 1;
+ uint32_t force_wakeup_by_tps3 : 1;
} bitfields;
/**
@@ -416,7 +408,14 @@ enum dmub_cmd_vbios_type {
* Enables or disables power gating.
*/
DMUB_CMD__VBIOS_ENABLE_DISP_POWER_GATING = 3,
+ /**
+ * Controls embedded panels.
+ */
DMUB_CMD__VBIOS_LVTMA_CONTROL = 15,
+ /**
+ * Query DP alt status on a transmitter.
+ */
+ DMUB_CMD__VBIOS_TRANSMITTER_QUERY_DP_ALT = 26,
};
//==============================================================================
@@ -1550,10 +1549,14 @@ struct dmub_cmd_psr_copy_settings_data {
* Currently the support is only for 0 or 1
*/
uint8_t panel_inst;
- /**
- * Explicit padding to 4 byte boundary.
+ /*
+ * DSC enable status in driver
*/
- uint8_t pad3[4];
+ uint8_t dsc_enable_status;
+ /**
+ * Explicit padding to 3 byte boundary.
+ */
+ uint8_t pad3[3];
};
/**
@@ -2398,6 +2401,24 @@ struct dmub_rb_cmd_lvtma_control {
};
/**
+ * Data passed in/out in a DMUB_CMD__VBIOS_TRANSMITTER_QUERY_DP_ALT command.
+ */
+struct dmub_rb_cmd_transmitter_query_dp_alt_data {
+ uint8_t phy_id; /**< 0=UNIPHYA, 1=UNIPHYB, 2=UNIPHYC, 3=UNIPHYD, 4=UNIPHYE, 5=UNIPHYF */
+ uint8_t is_usb; /**< is phy is usb */
+ uint8_t is_dp_alt_disable; /**< is dp alt disable */
+ uint8_t is_dp4; /**< is dp in 4 lane */
+};
+
+/**
+ * Definition of a DMUB_CMD__VBIOS_TRANSMITTER_QUERY_DP_ALT command.
+ */
+struct dmub_rb_cmd_transmitter_query_dp_alt {
+ struct dmub_cmd_header header; /**< header */
+ struct dmub_rb_cmd_transmitter_query_dp_alt_data data; /**< payload */
+};
+
+/**
* Maximum number of bytes a chunk sent to DMUB for parsing
*/
#define DMUB_EDID_CEA_DATA_CHUNK_BYTES 8
@@ -2408,7 +2429,7 @@ struct dmub_rb_cmd_lvtma_control {
struct dmub_cmd_send_edid_cea {
uint16_t offset; /**< offset into the CEA block */
uint8_t length; /**< number of bytes in payload to copy as part of CEA block */
- uint16_t total_length; /**< total length of the CEA block */
+ uint16_t cea_total_length; /**< total length of the CEA block */
uint8_t payload[DMUB_EDID_CEA_DATA_CHUNK_BYTES]; /**< data chunk of the CEA block */
uint8_t pad[3]; /**< padding and for future expansion */
};
@@ -2605,6 +2626,10 @@ union dmub_rb_cmd {
*/
struct dmub_rb_cmd_lvtma_control lvtma_control;
/**
+ * Definition of a DMUB_CMD__VBIOS_TRANSMITTER_QUERY_DP_ALT command.
+ */
+ struct dmub_rb_cmd_transmitter_query_dp_alt query_dp_alt;
+ /**
* Definition of a DMUB_CMD__DPIA_DIG1_CONTROL command.
*/
struct dmub_rb_cmd_dig1_dpia_control dig1_dpia_control;
@@ -2722,7 +2747,7 @@ static inline bool dmub_rb_full(struct dmub_rb *rb)
static inline bool dmub_rb_push_front(struct dmub_rb *rb,
const union dmub_rb_cmd *cmd)
{
- uint64_t volatile *dst = (uint64_t volatile *)(rb->base_address) + rb->wrpt / sizeof(uint64_t);
+ uint64_t volatile *dst = (uint64_t volatile *)((uint8_t *)(rb->base_address) + rb->wrpt);
const uint64_t *src = (const uint64_t *)cmd;
uint8_t i;
@@ -2840,7 +2865,7 @@ static inline bool dmub_rb_peek_offset(struct dmub_rb *rb,
static inline bool dmub_rb_out_front(struct dmub_rb *rb,
union dmub_rb_out_cmd *cmd)
{
- const uint64_t volatile *src = (const uint64_t volatile *)(rb->base_address) + rb->rptr / sizeof(uint64_t);
+ const uint64_t volatile *src = (const uint64_t volatile *)((uint8_t *)(rb->base_address) + rb->rptr);
uint64_t *dst = (uint64_t *)cmd;
uint8_t i;
@@ -2888,7 +2913,7 @@ static inline void dmub_rb_flush_pending(const struct dmub_rb *rb)
uint32_t wptr = rb->wrpt;
while (rptr != wptr) {
- uint64_t volatile *data = (uint64_t volatile *)rb->base_address + rptr / sizeof(uint64_t);
+ uint64_t volatile *data = (uint64_t volatile *)((uint8_t *)(rb->base_address) + rptr);
//uint64_t volatile *p = (uint64_t volatile *)data;
uint64_t temp;
uint8_t i;
diff --git a/drivers/gpu/drm/amd/display/dmub/src/dmub_srv.c b/drivers/gpu/drm/amd/display/dmub/src/dmub_srv.c
index 56d400f..9280f2a 100644
--- a/drivers/gpu/drm/amd/display/dmub/src/dmub_srv.c
+++ b/drivers/gpu/drm/amd/display/dmub/src/dmub_srv.c
@@ -100,24 +100,9 @@ void dmub_flush_buffer_mem(const struct dmub_fb *fb)
}
static const struct dmub_fw_meta_info *
-dmub_get_fw_meta_info(const struct dmub_srv_region_params *params)
+dmub_get_fw_meta_info_from_blob(const uint8_t *blob, uint32_t blob_size, uint32_t meta_offset)
{
const union dmub_fw_meta *meta;
- const uint8_t *blob = NULL;
- uint32_t blob_size = 0;
- uint32_t meta_offset = 0;
-
- if (params->fw_bss_data && params->bss_data_size) {
- /* Legacy metadata region. */
- blob = params->fw_bss_data;
- blob_size = params->bss_data_size;
- meta_offset = DMUB_FW_META_OFFSET;
- } else if (params->fw_inst_const && params->inst_const_size) {
- /* Combined metadata region. */
- blob = params->fw_inst_const;
- blob_size = params->inst_const_size;
- meta_offset = 0;
- }
if (!blob || !blob_size)
return NULL;
@@ -134,6 +119,32 @@ dmub_get_fw_meta_info(const struct dmub_srv_region_params *params)
return &meta->info;
}
+static const struct dmub_fw_meta_info *
+dmub_get_fw_meta_info(const struct dmub_srv_region_params *params)
+{
+ const struct dmub_fw_meta_info *info = NULL;
+
+ if (params->fw_bss_data && params->bss_data_size) {
+ /* Legacy metadata region. */
+ info = dmub_get_fw_meta_info_from_blob(params->fw_bss_data,
+ params->bss_data_size,
+ DMUB_FW_META_OFFSET);
+ } else if (params->fw_inst_const && params->inst_const_size) {
+ /* Combined metadata region - can be aligned to 16-bytes. */
+ uint32_t i;
+
+ for (i = 0; i < 16; ++i) {
+ info = dmub_get_fw_meta_info_from_blob(
+ params->fw_inst_const, params->inst_const_size, i);
+
+ if (info)
+ break;
+ }
+ }
+
+ return info;
+}
+
static bool dmub_srv_hw_setup(struct dmub_srv *dmub, enum dmub_asic asic)
{
struct dmub_srv_hw_funcs *funcs = &dmub->hw_funcs;
@@ -598,6 +609,8 @@ enum dmub_status dmub_srv_cmd_queue(struct dmub_srv *dmub,
enum dmub_status dmub_srv_cmd_execute(struct dmub_srv *dmub)
{
+ struct dmub_rb flush_rb;
+
if (!dmub->hw_init)
return DMUB_STATUS_INVALID;
@@ -606,9 +619,14 @@ enum dmub_status dmub_srv_cmd_execute(struct dmub_srv *dmub)
* been flushed to framebuffer memory. Otherwise DMCUB might
* read back stale, fully invalid or partially invalid data.
*/
- dmub_rb_flush_pending(&dmub->inbox1_rb);
+ flush_rb = dmub->inbox1_rb;
+ flush_rb.rptr = dmub->inbox1_last_wptr;
+ dmub_rb_flush_pending(&flush_rb);
- dmub->hw_funcs.set_inbox1_wptr(dmub, dmub->inbox1_rb.wrpt);
+ dmub->hw_funcs.set_inbox1_wptr(dmub, dmub->inbox1_rb.wrpt);
+
+ dmub->inbox1_last_wptr = dmub->inbox1_rb.wrpt;
+
return DMUB_STATUS_OK;
}
@@ -831,3 +849,38 @@ bool dmub_srv_should_detect(struct dmub_srv *dmub)
return dmub->hw_funcs.should_detect(dmub);
}
+
+enum dmub_status dmub_srv_clear_inbox0_ack(struct dmub_srv *dmub)
+{
+ if (!dmub->hw_init || !dmub->hw_funcs.clear_inbox0_ack_register)
+ return DMUB_STATUS_INVALID;
+
+ dmub->hw_funcs.clear_inbox0_ack_register(dmub);
+ return DMUB_STATUS_OK;
+}
+
+enum dmub_status dmub_srv_wait_for_inbox0_ack(struct dmub_srv *dmub, uint32_t timeout_us)
+{
+ uint32_t i = 0;
+ uint32_t ack = 0;
+
+ if (!dmub->hw_init || !dmub->hw_funcs.read_inbox0_ack_register)
+ return DMUB_STATUS_INVALID;
+
+ for (i = 0; i <= timeout_us; i++) {
+ ack = dmub->hw_funcs.read_inbox0_ack_register(dmub);
+ if (ack)
+ return DMUB_STATUS_OK;
+ }
+ return DMUB_STATUS_TIMEOUT;
+}
+
+enum dmub_status dmub_srv_send_inbox0_cmd(struct dmub_srv *dmub,
+ union dmub_inbox0_data_register data)
+{
+ if (!dmub->hw_init || !dmub->hw_funcs.send_inbox0_cmd)
+ return DMUB_STATUS_INVALID;
+
+ dmub->hw_funcs.send_inbox0_cmd(dmub, data);
+ return DMUB_STATUS_OK;
+}
diff --git a/drivers/gpu/drm/amd/display/include/ddc_service_types.h b/drivers/gpu/drm/amd/display/include/ddc_service_types.h
index 4de59b6..a2b8051 100644
--- a/drivers/gpu/drm/amd/display/include/ddc_service_types.h
+++ b/drivers/gpu/drm/amd/display/include/ddc_service_types.h
@@ -35,6 +35,7 @@
#define DP_BRANCH_DEVICE_ID_00E04C 0x00E04C
#define DP_BRANCH_DEVICE_ID_006037 0x006037
+#define DP_DEVICE_ID_38EC11 0x38EC11
enum ddc_result {
DDC_RESULT_UNKNOWN = 0,
DDC_RESULT_SUCESSFULL,
@@ -117,4 +118,7 @@ struct av_sync_data {
uint8_t aud_del_ins3;/* DPCD 0002Dh */
};
+static const uint8_t DP_SINK_DEVICE_STR_ID_1[] = {7, 1, 8, 7, 3, 0};
+static const uint8_t DP_SINK_DEVICE_STR_ID_2[] = {7, 1, 8, 7, 5, 0};
+
#endif /* __DAL_DDC_SERVICE_TYPES_H__ */
diff --git a/drivers/gpu/drm/amd/display/include/logger_types.h b/drivers/gpu/drm/amd/display/include/logger_types.h
index 370fad8..f093b49 100644
--- a/drivers/gpu/drm/amd/display/include/logger_types.h
+++ b/drivers/gpu/drm/amd/display/include/logger_types.h
@@ -72,9 +72,7 @@
#define DC_LOG_DSC(...) DRM_DEBUG_KMS(__VA_ARGS__)
#define DC_LOG_SMU(...) pr_debug("[SMU_MSG]:"__VA_ARGS__)
#define DC_LOG_DWB(...) DRM_DEBUG_KMS(__VA_ARGS__)
-#if defined(CONFIG_DRM_AMD_DC_DCN)
#define DC_LOG_DP2(...) DRM_DEBUG_KMS(__VA_ARGS__)
-#endif
struct dal_logger;
@@ -126,9 +124,7 @@ enum dc_log_type {
LOG_MAX_HW_POINTS,
LOG_ALL_TF_CHANNELS,
LOG_SAMPLE_1DLUT,
-#if defined(CONFIG_DRM_AMD_DC_DCN)
LOG_DP2,
-#endif
LOG_SECTION_TOTAL_COUNT
};
diff --git a/drivers/gpu/drm/amd/include/amd_shared.h b/drivers/gpu/drm/amd/include/amd_shared.h
index 4b9e68a..f57a147 100644
--- a/drivers/gpu/drm/amd/include/amd_shared.h
+++ b/drivers/gpu/drm/amd/include/amd_shared.h
@@ -231,6 +231,8 @@ enum DC_FEATURE_MASK {
DC_DISABLE_FRACTIONAL_PWM_MASK = (1 << 2), //0x4, disabled by default
DC_PSR_MASK = (1 << 3), //0x8, disabled by default for dcn < 3.1
DC_EDP_NO_POWER_SEQUENCING = (1 << 4), //0x10, disabled by default
+ DC_DISABLE_LTTPR_DP1_4A = (1 << 5), //0x20, disabled by default
+ DC_DISABLE_LTTPR_DP2_0 = (1 << 6), //0x40, disabled by default
};
enum DC_DEBUG_MASK {
diff --git a/drivers/gpu/drm/amd/include/asic_reg/mp/mp_11_0_offset.h b/drivers/gpu/drm/amd/include/asic_reg/mp/mp_11_0_offset.h
index 6d0052c..da6d380 100644
--- a/drivers/gpu/drm/amd/include/asic_reg/mp/mp_11_0_offset.h
+++ b/drivers/gpu/drm/amd/include/asic_reg/mp/mp_11_0_offset.h
@@ -354,5 +354,12 @@
#define mmMP1_SMN_EXT_SCRATCH7 0x03c7
#define mmMP1_SMN_EXT_SCRATCH7_BASE_IDX 0
+/*
+ * addressBlock: mp_SmuMp1Pub_MmuDec
+ * base address: 0x0
+ */
+#define smnMP1_PMI_3_START 0x3030204
+#define smnMP1_PMI_3_FIFO 0x3030208
+#define smnMP1_PMI_3 0x3030600
#endif
diff --git a/drivers/gpu/drm/amd/include/asic_reg/mp/mp_11_0_sh_mask.h b/drivers/gpu/drm/amd/include/asic_reg/mp/mp_11_0_sh_mask.h
index 136fb5d..a5ae2a8 100644
--- a/drivers/gpu/drm/amd/include/asic_reg/mp/mp_11_0_sh_mask.h
+++ b/drivers/gpu/drm/amd/include/asic_reg/mp/mp_11_0_sh_mask.h
@@ -959,5 +959,17 @@
#define MP1_SMN_EXT_SCRATCH7__DATA__SHIFT 0x0
#define MP1_SMN_EXT_SCRATCH7__DATA_MASK 0xFFFFFFFFL
+// MP1_PMI_3_START
+#define MP1_PMI_3_START__ENABLE_MASK 0x80000000L
+// MP1_PMI_3_FIFO
+#define MP1_PMI_3_FIFO__DEPTH_MASK 0x00000fffL
+
+// MP1_PMI_3_START
+#define MP1_PMI_3_START__ENABLE__SHIFT 0x0000001f
+// MP1_PMI_3_FIFO
+#define MP1_PMI_3_FIFO__DEPTH__SHIFT 0x00000000
+
+
+
#endif
diff --git a/drivers/gpu/drm/amd/include/asic_reg/nbio/nbio_7_2_0_offset.h b/drivers/gpu/drm/amd/include/asic_reg/nbio/nbio_7_2_0_offset.h
index 79eae02..8072b0a 100644
--- a/drivers/gpu/drm/amd/include/asic_reg/nbio/nbio_7_2_0_offset.h
+++ b/drivers/gpu/drm/amd/include/asic_reg/nbio/nbio_7_2_0_offset.h
@@ -20753,8 +20753,6 @@
// addressBlock: nbio_nbif0_gdc_GDCDEC
// base address: 0xd0000000
-#define regGDC1_LOGAN_FAST_WRITE_RESPONSE_CNTL 0x2ffc0eda
-#define regGDC1_LOGAN_FAST_WRITE_RESPONSE_CNTL_BASE_IDX 5
#define regGDC1_NGDC_SDP_PORT_CTRL 0x2ffc0ee2
#define regGDC1_NGDC_SDP_PORT_CTRL_BASE_IDX 5
#define regGDC1_SHUB_REGS_IF_CTL 0x2ffc0ee3
diff --git a/drivers/gpu/drm/amd/include/asic_reg/nbio/nbio_7_2_0_sh_mask.h b/drivers/gpu/drm/amd/include/asic_reg/nbio/nbio_7_2_0_sh_mask.h
index e27fdc0..54b0e46 100644
--- a/drivers/gpu/drm/amd/include/asic_reg/nbio/nbio_7_2_0_sh_mask.h
+++ b/drivers/gpu/drm/amd/include/asic_reg/nbio/nbio_7_2_0_sh_mask.h
@@ -1,4 +1,3 @@
-
/*
* Copyright (C) 2020 Advanced Micro Devices, Inc.
*
@@ -108541,17 +108540,6 @@
// addressBlock: nbio_nbif0_gdc_GDCDEC
-//GDC1_LOGAN_FAST_WRITE_RESPONSE_CNTL
-#define GDC1_LOGAN_FAST_WRITE_RESPONSE_CNTL__LOGAN0_FAST_WRITE_RESPONSE_EN__SHIFT 0x0
-#define GDC1_LOGAN_FAST_WRITE_RESPONSE_CNTL__LOGAN1_FAST_WRITE_RESPONSE_EN__SHIFT 0x1
-#define GDC1_LOGAN_FAST_WRITE_RESPONSE_CNTL__LOGAN2_FAST_WRITE_RESPONSE_EN__SHIFT 0x2
-#define GDC1_LOGAN_FAST_WRITE_RESPONSE_CNTL__LOGAN3_FAST_WRITE_RESPONSE_EN__SHIFT 0x3
-#define GDC1_LOGAN_FAST_WRITE_RESPONSE_CNTL__FWR_NORMAL_ARB_MODE__SHIFT 0x10
-#define GDC1_LOGAN_FAST_WRITE_RESPONSE_CNTL__LOGAN0_FAST_WRITE_RESPONSE_EN_MASK 0x00000001L
-#define GDC1_LOGAN_FAST_WRITE_RESPONSE_CNTL__LOGAN1_FAST_WRITE_RESPONSE_EN_MASK 0x00000002L
-#define GDC1_LOGAN_FAST_WRITE_RESPONSE_CNTL__LOGAN2_FAST_WRITE_RESPONSE_EN_MASK 0x00000004L
-#define GDC1_LOGAN_FAST_WRITE_RESPONSE_CNTL__LOGAN3_FAST_WRITE_RESPONSE_EN_MASK 0x00000008L
-#define GDC1_LOGAN_FAST_WRITE_RESPONSE_CNTL__FWR_NORMAL_ARB_MODE_MASK 0x00010000L
//GDC1_NGDC_SDP_PORT_CTRL
#define GDC1_NGDC_SDP_PORT_CTRL__SDP_DISCON_HYSTERESIS__SHIFT 0x0
#define GDC1_NGDC_SDP_PORT_CTRL__NGDC_OBFF_HW_URGENT_EARLY_WAKEUP_EN__SHIFT 0xf
diff --git a/drivers/gpu/drm/amd/include/cyan_skillfish_ip_offset.h b/drivers/gpu/drm/amd/include/cyan_skillfish_ip_offset.h
index 9cb5f36..ce79e5d 100644
--- a/drivers/gpu/drm/amd/include/cyan_skillfish_ip_offset.h
+++ b/drivers/gpu/drm/amd/include/cyan_skillfish_ip_offset.h
@@ -25,15 +25,15 @@
#define MAX_SEGMENT 5
-struct IP_BASE_INSTANCE
+struct IP_BASE_INSTANCE
{
unsigned int segment[MAX_SEGMENT];
-};
-
-struct IP_BASE
+} __maybe_unused;
+
+struct IP_BASE
{
struct IP_BASE_INSTANCE instance[MAX_INSTANCE];
-};
+} __maybe_unused;
static const struct IP_BASE ATHUB_BASE ={ { { { 0x00000C00, 0, 0, 0, 0 } },
diff --git a/drivers/gpu/drm/amd/include/discovery.h b/drivers/gpu/drm/amd/include/discovery.h
index 7ec4331..a486769 100644
--- a/drivers/gpu/drm/amd/include/discovery.h
+++ b/drivers/gpu/drm/amd/include/discovery.h
@@ -143,6 +143,55 @@ struct gc_info_v1_0 {
uint32_t gc_num_gl2a;
};
+struct gc_info_v1_1 {
+ struct gpu_info_header header;
+
+ uint32_t gc_num_se;
+ uint32_t gc_num_wgp0_per_sa;
+ uint32_t gc_num_wgp1_per_sa;
+ uint32_t gc_num_rb_per_se;
+ uint32_t gc_num_gl2c;
+ uint32_t gc_num_gprs;
+ uint32_t gc_num_max_gs_thds;
+ uint32_t gc_gs_table_depth;
+ uint32_t gc_gsprim_buff_depth;
+ uint32_t gc_parameter_cache_depth;
+ uint32_t gc_double_offchip_lds_buffer;
+ uint32_t gc_wave_size;
+ uint32_t gc_max_waves_per_simd;
+ uint32_t gc_max_scratch_slots_per_cu;
+ uint32_t gc_lds_size;
+ uint32_t gc_num_sc_per_se;
+ uint32_t gc_num_sa_per_se;
+ uint32_t gc_num_packer_per_sc;
+ uint32_t gc_num_gl2a;
+ uint32_t gc_num_tcp_per_sa;
+ uint32_t gc_num_sdp_interface;
+ uint32_t gc_num_tcps;
+};
+
+struct gc_info_v2_0 {
+ struct gpu_info_header header;
+
+ uint32_t gc_num_se;
+ uint32_t gc_num_cu_per_sh;
+ uint32_t gc_num_sh_per_se;
+ uint32_t gc_num_rb_per_se;
+ uint32_t gc_num_tccs;
+ uint32_t gc_num_gprs;
+ uint32_t gc_num_max_gs_thds;
+ uint32_t gc_gs_table_depth;
+ uint32_t gc_gsprim_buff_depth;
+ uint32_t gc_parameter_cache_depth;
+ uint32_t gc_double_offchip_lds_buffer;
+ uint32_t gc_wave_size;
+ uint32_t gc_max_waves_per_simd;
+ uint32_t gc_max_scratch_slots_per_cu;
+ uint32_t gc_lds_size;
+ uint32_t gc_num_sc_per_se;
+ uint32_t gc_num_packer_per_sc;
+};
+
typedef struct harvest_info_header {
uint32_t signature; /* Table Signature */
uint32_t version; /* Table Version */
diff --git a/drivers/gpu/drm/amd/include/kgd_kfd_interface.h b/drivers/gpu/drm/amd/include/kgd_kfd_interface.h
index c84bd7b..ac941f6 100644
--- a/drivers/gpu/drm/amd/include/kgd_kfd_interface.h
+++ b/drivers/gpu/drm/amd/include/kgd_kfd_interface.h
@@ -33,12 +33,11 @@
#include <linux/dma-fence.h>
struct pci_dev;
+struct amdgpu_device;
#define KGD_MAX_QUEUES 128
struct kfd_dev;
-struct kgd_dev;
-
struct kgd_mem;
enum kfd_preempt_type {
@@ -228,61 +227,61 @@ struct tile_config {
*/
struct kfd2kgd_calls {
/* Register access functions */
- void (*program_sh_mem_settings)(struct kgd_dev *kgd, uint32_t vmid,
+ void (*program_sh_mem_settings)(struct amdgpu_device *adev, uint32_t vmid,
uint32_t sh_mem_config, uint32_t sh_mem_ape1_base,
uint32_t sh_mem_ape1_limit, uint32_t sh_mem_bases);
- int (*set_pasid_vmid_mapping)(struct kgd_dev *kgd, u32 pasid,
+ int (*set_pasid_vmid_mapping)(struct amdgpu_device *adev, u32 pasid,
unsigned int vmid);
- int (*init_interrupts)(struct kgd_dev *kgd, uint32_t pipe_id);
+ int (*init_interrupts)(struct amdgpu_device *adev, uint32_t pipe_id);
- int (*hqd_load)(struct kgd_dev *kgd, void *mqd, uint32_t pipe_id,
+ int (*hqd_load)(struct amdgpu_device *adev, void *mqd, uint32_t pipe_id,
uint32_t queue_id, uint32_t __user *wptr,
uint32_t wptr_shift, uint32_t wptr_mask,
struct mm_struct *mm);
- int (*hiq_mqd_load)(struct kgd_dev *kgd, void *mqd,
+ int (*hiq_mqd_load)(struct amdgpu_device *adev, void *mqd,
uint32_t pipe_id, uint32_t queue_id,
uint32_t doorbell_off);
- int (*hqd_sdma_load)(struct kgd_dev *kgd, void *mqd,
+ int (*hqd_sdma_load)(struct amdgpu_device *adev, void *mqd,
uint32_t __user *wptr, struct mm_struct *mm);
- int (*hqd_dump)(struct kgd_dev *kgd,
+ int (*hqd_dump)(struct amdgpu_device *adev,
uint32_t pipe_id, uint32_t queue_id,
uint32_t (**dump)[2], uint32_t *n_regs);
- int (*hqd_sdma_dump)(struct kgd_dev *kgd,
+ int (*hqd_sdma_dump)(struct amdgpu_device *adev,
uint32_t engine_id, uint32_t queue_id,
uint32_t (**dump)[2], uint32_t *n_regs);
- bool (*hqd_is_occupied)(struct kgd_dev *kgd, uint64_t queue_address,
- uint32_t pipe_id, uint32_t queue_id);
-
- int (*hqd_destroy)(struct kgd_dev *kgd, void *mqd, uint32_t reset_type,
- unsigned int timeout, uint32_t pipe_id,
+ bool (*hqd_is_occupied)(struct amdgpu_device *adev,
+ uint64_t queue_address, uint32_t pipe_id,
uint32_t queue_id);
- bool (*hqd_sdma_is_occupied)(struct kgd_dev *kgd, void *mqd);
+ int (*hqd_destroy)(struct amdgpu_device *adev, void *mqd,
+ uint32_t reset_type, unsigned int timeout,
+ uint32_t pipe_id, uint32_t queue_id);
- int (*hqd_sdma_destroy)(struct kgd_dev *kgd, void *mqd,
+ bool (*hqd_sdma_is_occupied)(struct amdgpu_device *adev, void *mqd);
+
+ int (*hqd_sdma_destroy)(struct amdgpu_device *adev, void *mqd,
unsigned int timeout);
- int (*address_watch_disable)(struct kgd_dev *kgd);
- int (*address_watch_execute)(struct kgd_dev *kgd,
+ int (*address_watch_disable)(struct amdgpu_device *adev);
+ int (*address_watch_execute)(struct amdgpu_device *adev,
unsigned int watch_point_id,
uint32_t cntl_val,
uint32_t addr_hi,
uint32_t addr_lo);
- int (*wave_control_execute)(struct kgd_dev *kgd,
+ int (*wave_control_execute)(struct amdgpu_device *adev,
uint32_t gfx_index_val,
uint32_t sq_cmd);
- uint32_t (*address_watch_get_offset)(struct kgd_dev *kgd,
+ uint32_t (*address_watch_get_offset)(struct amdgpu_device *adev,
unsigned int watch_point_id,
unsigned int reg_offset);
- bool (*get_atc_vmid_pasid_mapping_info)(
- struct kgd_dev *kgd,
+ bool (*get_atc_vmid_pasid_mapping_info)(struct amdgpu_device *adev,
uint8_t vmid,
uint16_t *p_pasid);
@@ -290,16 +289,16 @@ struct kfd2kgd_calls {
* passed to the shader by the CP. It's the user mode driver's
* responsibility.
*/
- void (*set_scratch_backing_va)(struct kgd_dev *kgd,
+ void (*set_scratch_backing_va)(struct amdgpu_device *adev,
uint64_t va, uint32_t vmid);
- void (*set_vm_context_page_table_base)(struct kgd_dev *kgd,
+ void (*set_vm_context_page_table_base)(struct amdgpu_device *adev,
uint32_t vmid, uint64_t page_table_base);
- uint32_t (*read_vmid_from_vmfault_reg)(struct kgd_dev *kgd);
+ uint32_t (*read_vmid_from_vmfault_reg)(struct amdgpu_device *adev);
- void (*get_cu_occupancy)(struct kgd_dev *kgd, int pasid, int *wave_cnt,
- int *max_waves_per_cu);
- void (*program_trap_handler_settings)(struct kgd_dev *kgd,
+ void (*get_cu_occupancy)(struct amdgpu_device *adev, int pasid,
+ int *wave_cnt, int *max_waves_per_cu);
+ void (*program_trap_handler_settings)(struct amdgpu_device *adev,
uint32_t vmid, uint64_t tba_addr, uint64_t tma_addr);
};
diff --git a/drivers/gpu/drm/amd/include/kgd_pp_interface.h b/drivers/gpu/drm/amd/include/kgd_pp_interface.h
index bac15c4..5c0867e 100644
--- a/drivers/gpu/drm/amd/include/kgd_pp_interface.h
+++ b/drivers/gpu/drm/amd/include/kgd_pp_interface.h
@@ -153,6 +153,10 @@ enum PP_SMC_POWER_PROFILE {
PP_SMC_POWER_PROFILE_COUNT,
};
+extern const char * const amdgpu_pp_profile_name[PP_SMC_POWER_PROFILE_COUNT];
+
+
+
enum {
PP_GROUP_UNKNOWN = 0,
PP_GROUP_GFX = 1,
diff --git a/drivers/gpu/drm/amd/include/yellow_carp_offset.h b/drivers/gpu/drm/amd/include/yellow_carp_offset.h
index 76b9eb3..28a56b5 100644
--- a/drivers/gpu/drm/amd/include/yellow_carp_offset.h
+++ b/drivers/gpu/drm/amd/include/yellow_carp_offset.h
@@ -9,12 +9,12 @@
struct IP_BASE_INSTANCE
{
unsigned int segment[MAX_SEGMENT];
-};
+} __maybe_unused;
struct IP_BASE
{
struct IP_BASE_INSTANCE instance[MAX_INSTANCE];
-};
+} __maybe_unused;
static const struct IP_BASE ACP_BASE = { { { { 0x02403800, 0x00480000, 0, 0, 0, 0 } },
diff --git a/drivers/gpu/drm/amd/pm/amdgpu_pm.c b/drivers/gpu/drm/amd/pm/amdgpu_pm.c
index 41472ed..e2cae97 100644
--- a/drivers/gpu/drm/amd/pm/amdgpu_pm.c
+++ b/drivers/gpu/drm/amd/pm/amdgpu_pm.c
@@ -82,6 +82,16 @@ static const struct hwmon_temp_label {
{PP_TEMP_MEM, "mem"},
};
+const char * const amdgpu_pp_profile_name[] = {
+ "BOOTUP_DEFAULT",
+ "3D_FULL_SCREEN",
+ "POWER_SAVING",
+ "VIDEO",
+ "VR",
+ "COMPUTE",
+ "CUSTOM"
+};
+
/**
* DOC: power_dpm_state
*
@@ -2080,7 +2090,8 @@ static int default_attr_update(struct amdgpu_device *adev, struct amdgpu_device_
} else if (DEVICE_ATTR_IS(unique_id)) {
if (asic_type != CHIP_VEGA10 &&
asic_type != CHIP_VEGA20 &&
- asic_type != CHIP_ARCTURUS)
+ asic_type != CHIP_ARCTURUS &&
+ asic_type != CHIP_ALDEBARAN)
*states = ATTR_STATE_UNSUPPORTED;
} else if (DEVICE_ATTR_IS(pp_features)) {
if (adev->flags & AMD_IS_APU || asic_type < CHIP_VEGA10)
@@ -2123,6 +2134,12 @@ static int default_attr_update(struct amdgpu_device *adev, struct amdgpu_device_
}
}
+ /* setting should not be allowed from VF */
+ if (amdgpu_sriov_vf(adev)) {
+ dev_attr->attr.mode &= ~S_IWUGO;
+ dev_attr->store = NULL;
+ }
+
#undef DEVICE_ATTR_IS
return 0;
@@ -3759,5 +3776,7 @@ void amdgpu_debugfs_pm_init(struct amdgpu_device *adev)
adev,
&amdgpu_debugfs_pm_prv_buffer_fops,
adev->pm.smu_prv_buffer_size);
+
+ amdgpu_smu_stb_debug_fs_init(adev);
#endif
}
diff --git a/drivers/gpu/drm/amd/pm/inc/aldebaran_ppsmc.h b/drivers/gpu/drm/amd/pm/inc/aldebaran_ppsmc.h
index 35fa0d8..ab66a4b 100644
--- a/drivers/gpu/drm/amd/pm/inc/aldebaran_ppsmc.h
+++ b/drivers/gpu/drm/amd/pm/inc/aldebaran_ppsmc.h
@@ -102,7 +102,9 @@
#define PPSMC_MSG_GfxDriverResetRecovery 0x42
#define PPSMC_MSG_BoardPowerCalibration 0x43
-#define PPSMC_Message_Count 0x44
+#define PPSMC_MSG_HeavySBR 0x45
+#define PPSMC_Message_Count 0x46
+
//PPSMC Reset Types
#define PPSMC_RESET_TYPE_WARM_RESET 0x00
diff --git a/drivers/gpu/drm/amd/pm/inc/amdgpu_dpm.h b/drivers/gpu/drm/amd/pm/inc/amdgpu_dpm.h
index 16e3f72..c464a04 100644
--- a/drivers/gpu/drm/amd/pm/inc/amdgpu_dpm.h
+++ b/drivers/gpu/drm/amd/pm/inc/amdgpu_dpm.h
@@ -423,6 +423,9 @@ enum ip_power_state {
POWER_STATE_OFF,
};
+/* Used to mask smu debug modes */
+#define SMU_DEBUG_HALT_ON_ERROR 0x1
+
struct amdgpu_pm {
struct mutex mutex;
u32 current_sclk;
@@ -460,6 +463,11 @@ struct amdgpu_pm {
struct list_head pm_attr_list;
atomic_t pwr_state[AMD_IP_BLOCK_TYPE_NUM];
+
+ /*
+ * 0 = disabled (default), otherwise enable corresponding debug mode
+ */
+ uint32_t smu_debug_mask;
};
#define R600_SSTU_DFLT 0
diff --git a/drivers/gpu/drm/amd/pm/inc/amdgpu_smu.h b/drivers/gpu/drm/amd/pm/inc/amdgpu_smu.h
index 3557f4e..ba7565b 100644
--- a/drivers/gpu/drm/amd/pm/inc/amdgpu_smu.h
+++ b/drivers/gpu/drm/amd/pm/inc/amdgpu_smu.h
@@ -324,6 +324,7 @@ enum smu_table_id
SMU_TABLE_OVERDRIVE,
SMU_TABLE_I2C_COMMANDS,
SMU_TABLE_PACE,
+ SMU_TABLE_ECCINFO,
SMU_TABLE_COUNT,
};
@@ -340,6 +341,7 @@ struct smu_table_context
void *max_sustainable_clocks;
struct smu_bios_boot_up_values boot_values;
void *driver_pptable;
+ void *ecc_table;
struct smu_table tables[SMU_TABLE_COUNT];
/*
* The driver table is just a staging buffer for
@@ -472,7 +474,14 @@ struct cmn2asic_mapping {
int map_to;
};
+struct stb_context {
+ uint32_t stb_buf_size;
+ bool enabled;
+ spinlock_t lock;
+};
+
#define WORKLOAD_POLICY_MAX 7
+
struct smu_context
{
struct amdgpu_device *adev;
@@ -559,6 +568,8 @@ struct smu_context
uint16_t cpu_core_num;
struct smu_user_dpm_profile user_dpm_profile;
+
+ struct stb_context stb_context;
};
struct i2c_adapter;
@@ -1246,9 +1257,9 @@ struct pptable_funcs {
int (*set_fine_grain_gfx_freq_parameters)(struct smu_context *smu);
/**
- * @set_light_sbr: Set light sbr mode for the SMU.
+ * @smu_handle_passthrough_sbr: Send message to SMU about special handling for SBR.
*/
- int (*set_light_sbr)(struct smu_context *smu, bool enable);
+ int (*smu_handle_passthrough_sbr)(struct smu_context *smu, bool enable);
/**
* @wait_for_event: Wait for events from SMU.
@@ -1261,6 +1272,17 @@ struct pptable_funcs {
* of SMUBUS table.
*/
int (*send_hbm_bad_pages_num)(struct smu_context *smu, uint32_t size);
+
+ /**
+ * @get_ecc_table: message SMU to get ECC INFO table.
+ */
+ ssize_t (*get_ecc_info)(struct smu_context *smu, void *table);
+
+
+ /**
+ * @stb_collect_info: Collects Smart Trace Buffers data.
+ */
+ int (*stb_collect_info)(struct smu_context *smu, void *buf, uint32_t size);
};
typedef enum {
@@ -1393,10 +1415,13 @@ int smu_allow_xgmi_power_down(struct smu_context *smu, bool en);
int smu_get_status_gfxoff(struct amdgpu_device *adev, uint32_t *value);
-int smu_set_light_sbr(struct smu_context *smu, bool enable);
+int smu_handle_passthrough_sbr(struct smu_context *smu, bool enable);
int smu_wait_for_event(struct amdgpu_device *adev, enum smu_event_type event,
uint64_t event_arg);
+int smu_get_ecc_info(struct smu_context *smu, void *umc_ecc);
+int smu_stb_collect_info(struct smu_context *smu, void *buff, uint32_t size);
+void amdgpu_smu_stb_debug_fs_init(struct amdgpu_device *adev);
#endif
#endif
diff --git a/drivers/gpu/drm/amd/pm/inc/smu13_driver_if_aldebaran.h b/drivers/gpu/drm/amd/pm/inc/smu13_driver_if_aldebaran.h
index a017983..0f67c56 100644
--- a/drivers/gpu/drm/amd/pm/inc/smu13_driver_if_aldebaran.h
+++ b/drivers/gpu/drm/amd/pm/inc/smu13_driver_if_aldebaran.h
@@ -140,6 +140,8 @@
#define MAX_SW_I2C_COMMANDS 24
+#define ALDEBARAN_UMC_CHANNEL_NUM 32
+
typedef enum {
I2C_CONTROLLER_PORT_0, //CKSVII2C0
I2C_CONTROLLER_PORT_1, //CKSVII2C1
@@ -507,6 +509,19 @@ typedef struct {
uint32_t MmHubPadding[8]; // SMU internal use
} AvfsDebugTable_t;
+typedef struct {
+ uint64_t mca_umc_status;
+ uint64_t mca_umc_addr;
+ uint16_t ce_count_lo_chip;
+ uint16_t ce_count_hi_chip;
+
+ uint32_t eccPadding;
+} EccInfo_t;
+
+typedef struct {
+ EccInfo_t EccInfo[ALDEBARAN_UMC_CHANNEL_NUM];
+} EccInfoTable_t;
+
// These defines are used with the following messages:
// SMC_MSG_TransferTableDram2Smu
// SMC_MSG_TransferTableSmu2Dram
@@ -517,6 +532,7 @@ typedef struct {
#define TABLE_SMU_METRICS 4
#define TABLE_DRIVER_SMU_CONFIG 5
#define TABLE_I2C_COMMANDS 6
-#define TABLE_COUNT 7
+#define TABLE_ECCINFO 7
+#define TABLE_COUNT 8
#endif
diff --git a/drivers/gpu/drm/amd/pm/inc/smu_types.h b/drivers/gpu/drm/amd/pm/inc/smu_types.h
index 18b862a..ff8a0bc 100644
--- a/drivers/gpu/drm/amd/pm/inc/smu_types.h
+++ b/drivers/gpu/drm/amd/pm/inc/smu_types.h
@@ -229,7 +229,8 @@
__SMU_DUMMY_MAP(BoardPowerCalibration), \
__SMU_DUMMY_MAP(RequestGfxclk), \
__SMU_DUMMY_MAP(ForceGfxVid), \
- __SMU_DUMMY_MAP(UnforceGfxVid),
+ __SMU_DUMMY_MAP(UnforceGfxVid), \
+ __SMU_DUMMY_MAP(HeavySBR),
#undef __SMU_DUMMY_MAP
#define __SMU_DUMMY_MAP(type) SMU_MSG_##type
diff --git a/drivers/gpu/drm/amd/pm/inc/smu_v11_0.h b/drivers/gpu/drm/amd/pm/inc/smu_v11_0.h
index 2d422e6..acb3be2 100644
--- a/drivers/gpu/drm/amd/pm/inc/smu_v11_0.h
+++ b/drivers/gpu/drm/amd/pm/inc/smu_v11_0.h
@@ -312,7 +312,7 @@ int smu_v11_0_deep_sleep_control(struct smu_context *smu,
void smu_v11_0_interrupt_work(struct smu_context *smu);
-int smu_v11_0_set_light_sbr(struct smu_context *smu, bool enable);
+int smu_v11_0_handle_passthrough_sbr(struct smu_context *smu, bool enable);
int smu_v11_0_restore_user_od_settings(struct smu_context *smu);
diff --git a/drivers/gpu/drm/amd/pm/inc/smu_v13_0.h b/drivers/gpu/drm/amd/pm/inc/smu_v13_0.h
index e5d3b0d..44af23a 100644
--- a/drivers/gpu/drm/amd/pm/inc/smu_v13_0.h
+++ b/drivers/gpu/drm/amd/pm/inc/smu_v13_0.h
@@ -27,7 +27,9 @@
#define SMU13_DRIVER_IF_VERSION_INV 0xFFFFFFFF
#define SMU13_DRIVER_IF_VERSION_YELLOW_CARP 0x04
-#define SMU13_DRIVER_IF_VERSION_ALDE 0x07
+#define SMU13_DRIVER_IF_VERSION_ALDE 0x08
+
+#define SMU13_MODE1_RESET_WAIT_TIME_IN_MS 500 //500ms
/* MP Apertures */
#define MP0_Public 0x03800000
@@ -216,7 +218,6 @@ int smu_v13_0_baco_set_state(struct smu_context *smu, enum smu_baco_state state)
int smu_v13_0_baco_enter(struct smu_context *smu);
int smu_v13_0_baco_exit(struct smu_context *smu);
-int smu_v13_0_mode1_reset(struct smu_context *smu);
int smu_v13_0_mode2_reset(struct smu_context *smu);
int smu_v13_0_get_dpm_ultimate_freq(struct smu_context *smu, enum smu_clk_type clk_type,
diff --git a/drivers/gpu/drm/amd/pm/powerplay/amd_powerplay.c b/drivers/gpu/drm/amd/pm/powerplay/amd_powerplay.c
index 619f8d3..3ab67b2 100644
--- a/drivers/gpu/drm/amd/pm/powerplay/amd_powerplay.c
+++ b/drivers/gpu/drm/amd/pm/powerplay/amd_powerplay.c
@@ -1556,7 +1556,7 @@ static int pp_set_ppfeature_status(void *handle, uint64_t ppfeature_masks)
static int pp_asic_reset_mode_2(void *handle)
{
struct pp_hwmgr *hwmgr = handle;
- int ret = 0;
+ int ret = 0;
if (!hwmgr || !hwmgr->pm_en)
return -EINVAL;
diff --git a/drivers/gpu/drm/amd/pm/powerplay/hwmgr/smu10_hwmgr.c b/drivers/gpu/drm/amd/pm/powerplay/hwmgr/smu10_hwmgr.c
index 1f406f2..9ddd849 100644
--- a/drivers/gpu/drm/amd/pm/powerplay/hwmgr/smu10_hwmgr.c
+++ b/drivers/gpu/drm/amd/pm/powerplay/hwmgr/smu10_hwmgr.c
@@ -1439,13 +1439,6 @@ static int smu10_get_power_profile_mode(struct pp_hwmgr *hwmgr, char *buf)
{70, 90, 0, 0,},
{30, 60, 0, 6,},
};
- static const char *profile_name[6] = {
- "BOOTUP_DEFAULT",
- "3D_FULL_SCREEN",
- "POWER_SAVING",
- "VIDEO",
- "VR",
- "COMPUTE"};
static const char *title[6] = {"NUM",
"MODE_NAME",
"BUSY_SET_POINT",
@@ -1463,7 +1456,7 @@ static int smu10_get_power_profile_mode(struct pp_hwmgr *hwmgr, char *buf)
for (i = 0; i <= PP_SMC_POWER_PROFILE_COMPUTE; i++)
size += sysfs_emit_at(buf, size, "%3d %14s%s: %14d %3d %10d %14d\n",
- i, profile_name[i], (i == hwmgr->power_profile_mode) ? "*" : " ",
+ i, amdgpu_pp_profile_name[i], (i == hwmgr->power_profile_mode) ? "*" : " ",
profile_mode_setting[i][0], profile_mode_setting[i][1],
profile_mode_setting[i][2], profile_mode_setting[i][3]);
diff --git a/drivers/gpu/drm/amd/pm/powerplay/hwmgr/smu7_hwmgr.c b/drivers/gpu/drm/amd/pm/powerplay/hwmgr/smu7_hwmgr.c
index 611969b..cd99db0 100644
--- a/drivers/gpu/drm/amd/pm/powerplay/hwmgr/smu7_hwmgr.c
+++ b/drivers/gpu/drm/amd/pm/powerplay/hwmgr/smu7_hwmgr.c
@@ -5498,14 +5498,6 @@ static int smu7_get_power_profile_mode(struct pp_hwmgr *hwmgr, char *buf)
uint32_t i, size = 0;
uint32_t len;
- static const char *profile_name[7] = {"BOOTUP_DEFAULT",
- "3D_FULL_SCREEN",
- "POWER_SAVING",
- "VIDEO",
- "VR",
- "COMPUTE",
- "CUSTOM"};
-
static const char *title[8] = {"NUM",
"MODE_NAME",
"SCLK_UP_HYST",
@@ -5529,7 +5521,7 @@ static int smu7_get_power_profile_mode(struct pp_hwmgr *hwmgr, char *buf)
for (i = 0; i < len; i++) {
if (i == hwmgr->power_profile_mode) {
size += sysfs_emit_at(buf, size, "%3d %14s %s: %8d %16d %16d %16d %16d %16d\n",
- i, profile_name[i], "*",
+ i, amdgpu_pp_profile_name[i], "*",
data->current_profile_setting.sclk_up_hyst,
data->current_profile_setting.sclk_down_hyst,
data->current_profile_setting.sclk_activity,
@@ -5540,12 +5532,12 @@ static int smu7_get_power_profile_mode(struct pp_hwmgr *hwmgr, char *buf)
}
if (smu7_profiling[i].bupdate_sclk)
size += sysfs_emit_at(buf, size, "%3d %16s: %8d %16d %16d ",
- i, profile_name[i], smu7_profiling[i].sclk_up_hyst,
+ i, amdgpu_pp_profile_name[i], smu7_profiling[i].sclk_up_hyst,
smu7_profiling[i].sclk_down_hyst,
smu7_profiling[i].sclk_activity);
else
size += sysfs_emit_at(buf, size, "%3d %16s: %8s %16s %16s ",
- i, profile_name[i], "-", "-", "-");
+ i, amdgpu_pp_profile_name[i], "-", "-", "-");
if (smu7_profiling[i].bupdate_mclk)
size += sysfs_emit_at(buf, size, "%16d %16d %16d\n",
diff --git a/drivers/gpu/drm/amd/pm/powerplay/hwmgr/vega10_hwmgr.c b/drivers/gpu/drm/amd/pm/powerplay/hwmgr/vega10_hwmgr.c
index e633665..3f040be 100644
--- a/drivers/gpu/drm/amd/pm/powerplay/hwmgr/vega10_hwmgr.c
+++ b/drivers/gpu/drm/amd/pm/powerplay/hwmgr/vega10_hwmgr.c
@@ -5097,13 +5097,6 @@ static int vega10_get_power_profile_mode(struct pp_hwmgr *hwmgr, char *buf)
{70, 90, 0, 0,},
{30, 60, 0, 6,},
};
- static const char *profile_name[7] = {"BOOTUP_DEFAULT",
- "3D_FULL_SCREEN",
- "POWER_SAVING",
- "VIDEO",
- "VR",
- "COMPUTE",
- "CUSTOM"};
static const char *title[6] = {"NUM",
"MODE_NAME",
"BUSY_SET_POINT",
@@ -5121,11 +5114,12 @@ static int vega10_get_power_profile_mode(struct pp_hwmgr *hwmgr, char *buf)
for (i = 0; i < PP_SMC_POWER_PROFILE_CUSTOM; i++)
size += sysfs_emit_at(buf, size, "%3d %14s%s: %14d %3d %10d %14d\n",
- i, profile_name[i], (i == hwmgr->power_profile_mode) ? "*" : " ",
+ i, amdgpu_pp_profile_name[i], (i == hwmgr->power_profile_mode) ? "*" : " ",
profile_mode_setting[i][0], profile_mode_setting[i][1],
profile_mode_setting[i][2], profile_mode_setting[i][3]);
+
size += sysfs_emit_at(buf, size, "%3d %14s%s: %14d %3d %10d %14d\n", i,
- profile_name[i], (i == hwmgr->power_profile_mode) ? "*" : " ",
+ amdgpu_pp_profile_name[i], (i == hwmgr->power_profile_mode) ? "*" : " ",
data->custom_profile_mode[0], data->custom_profile_mode[1],
data->custom_profile_mode[2], data->custom_profile_mode[3]);
return size;
diff --git a/drivers/gpu/drm/amd/pm/powerplay/hwmgr/vega20_hwmgr.c b/drivers/gpu/drm/amd/pm/powerplay/hwmgr/vega20_hwmgr.c
index 85d55ab..97b3ad3 100644
--- a/drivers/gpu/drm/amd/pm/powerplay/hwmgr/vega20_hwmgr.c
+++ b/drivers/gpu/drm/amd/pm/powerplay/hwmgr/vega20_hwmgr.c
@@ -3980,14 +3980,6 @@ static int vega20_get_power_profile_mode(struct pp_hwmgr *hwmgr, char *buf)
DpmActivityMonitorCoeffInt_t activity_monitor;
uint32_t i, size = 0;
uint16_t workload_type = 0;
- static const char *profile_name[] = {
- "BOOTUP_DEFAULT",
- "3D_FULL_SCREEN",
- "POWER_SAVING",
- "VIDEO",
- "VR",
- "COMPUTE",
- "CUSTOM"};
static const char *title[] = {
"PROFILE_INDEX(NAME)",
"CLOCK_TYPE(NAME)",
@@ -4021,7 +4013,7 @@ static int vega20_get_power_profile_mode(struct pp_hwmgr *hwmgr, char *buf)
return result);
size += sysfs_emit_at(buf, size, "%2d %14s%s:\n",
- i, profile_name[i], (i == hwmgr->power_profile_mode) ? "*" : " ");
+ i, amdgpu_pp_profile_name[i], (i == hwmgr->power_profile_mode) ? "*" : " ");
size += sysfs_emit_at(buf, size, "%19s %d(%13s) %7d %7d %7d %7d %7d %7d %7d %7d %7d\n",
" ",
diff --git a/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c b/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c
index 8a32445..d93d28c 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c
+++ b/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c
@@ -277,8 +277,12 @@ static int smu_dpm_set_power_gate(void *handle,
struct smu_context *smu = handle;
int ret = 0;
- if (!smu->pm_enabled || !smu->adev->pm.dpm_enabled)
+ if (!smu->pm_enabled || !smu->adev->pm.dpm_enabled) {
+ dev_WARN(smu->adev->dev,
+ "SMU uninitialized but power %s requested for %u!\n",
+ gate ? "gate" : "ungate", block_type);
return -EOPNOTSUPP;
+ }
switch (block_type) {
/*
@@ -1153,6 +1157,8 @@ static int smu_smc_hw_setup(struct smu_context *smu)
case IP_VERSION(11, 5, 0):
case IP_VERSION(11, 0, 12):
ret = smu_system_features_control(smu, true);
+ if (ret)
+ dev_err(adev->dev, "Failed system features control!\n");
break;
default:
break;
@@ -1277,8 +1283,10 @@ static int smu_smc_hw_setup(struct smu_context *smu)
}
ret = smu_notify_display_change(smu);
- if (ret)
+ if (ret) {
+ dev_err(adev->dev, "Failed to notify display change!\n");
return ret;
+ }
/*
* Set min deep sleep dce fclk with bootup value from vbios via
@@ -1286,8 +1294,6 @@ static int smu_smc_hw_setup(struct smu_context *smu)
*/
ret = smu_set_min_dcef_deep_sleep(smu,
smu->smu_table.boot_values.dcefclk / 100);
- if (ret)
- return ret;
return ret;
}
@@ -1344,7 +1350,6 @@ static int smu_hw_init(void *handle)
}
if (smu->is_apu) {
- smu_powergate_sdma(&adev->smu, false);
smu_dpm_set_vcn_enable(smu, true);
smu_dpm_set_jpeg_enable(smu, true);
smu_set_gfx_cgpg(&adev->smu, true);
@@ -1400,8 +1405,14 @@ static int smu_disable_dpms(struct smu_context *smu)
{
struct amdgpu_device *adev = smu->adev;
int ret = 0;
+ /*
+ * TODO: (adev->in_suspend && !adev->in_s0ix) is added to pair
+ * the workaround which always reset the asic in suspend.
+ * It's likely that workaround will be dropped in the future.
+ * Then the change here should be dropped together.
+ */
bool use_baco = !smu->is_apu &&
- ((amdgpu_in_reset(adev) &&
+ (((amdgpu_in_reset(adev) || (adev->in_suspend && !adev->in_s0ix)) &&
(amdgpu_asic_reset_method(adev) == AMD_RESET_METHOD_BACO)) ||
((adev->in_runpm || adev->in_s4) && amdgpu_asic_supports_baco(adev)));
@@ -1506,10 +1517,6 @@ static int smu_hw_fini(void *handle)
if (amdgpu_sriov_vf(adev)&& !amdgpu_sriov_is_pp_one_vf(adev))
return 0;
- if (smu->is_apu) {
- smu_powergate_sdma(&adev->smu, true);
- }
-
smu_dpm_set_vcn_enable(smu, false);
smu_dpm_set_jpeg_enable(smu, false);
@@ -1568,9 +1575,7 @@ static int smu_suspend(void *handle)
smu->watermarks_bitmap &= ~(WATERMARKS_LOADED);
- /* skip CGPG when in S0ix */
- if (smu->is_apu && !adev->in_s0ix)
- smu_set_gfx_cgpg(&adev->smu, false);
+ smu_set_gfx_cgpg(&adev->smu, false);
return 0;
}
@@ -1601,8 +1606,7 @@ static int smu_resume(void *handle)
return ret;
}
- if (smu->is_apu)
- smu_set_gfx_cgpg(&adev->smu, true);
+ smu_set_gfx_cgpg(&adev->smu, true);
smu->disable_uclk_switch = 0;
@@ -3060,18 +3064,32 @@ static int smu_gfx_state_change_set(void *handle,
return ret;
}
-int smu_set_light_sbr(struct smu_context *smu, bool enable)
+int smu_handle_passthrough_sbr(struct smu_context *smu, bool enable)
{
int ret = 0;
mutex_lock(&smu->mutex);
- if (smu->ppt_funcs->set_light_sbr)
- ret = smu->ppt_funcs->set_light_sbr(smu, enable);
+ if (smu->ppt_funcs->smu_handle_passthrough_sbr)
+ ret = smu->ppt_funcs->smu_handle_passthrough_sbr(smu, enable);
mutex_unlock(&smu->mutex);
return ret;
}
+int smu_get_ecc_info(struct smu_context *smu, void *umc_ecc)
+{
+ int ret = -EOPNOTSUPP;
+
+ mutex_lock(&smu->mutex);
+ if (smu->ppt_funcs &&
+ smu->ppt_funcs->get_ecc_info)
+ ret = smu->ppt_funcs->get_ecc_info(smu, umc_ecc);
+ mutex_unlock(&smu->mutex);
+
+ return ret;
+
+}
+
static int smu_get_prv_buffer_details(void *handle, void **addr, size_t *size)
{
struct smu_context *smu = handle;
@@ -3161,3 +3179,107 @@ int smu_wait_for_event(struct amdgpu_device *adev, enum smu_event_type event,
return ret;
}
+
+int smu_stb_collect_info(struct smu_context *smu, void *buf, uint32_t size)
+{
+
+ if (!smu->ppt_funcs->stb_collect_info || !smu->stb_context.enabled)
+ return -EOPNOTSUPP;
+
+ /* Confirm the buffer allocated is of correct size */
+ if (size != smu->stb_context.stb_buf_size)
+ return -EINVAL;
+
+ /*
+ * No need to lock smu mutex as we access STB directly through MMIO
+ * and not going through SMU messaging route (for now at least).
+ * For registers access rely on implementation internal locking.
+ */
+ return smu->ppt_funcs->stb_collect_info(smu, buf, size);
+}
+
+#if defined(CONFIG_DEBUG_FS)
+
+static int smu_stb_debugfs_open(struct inode *inode, struct file *filp)
+{
+ struct amdgpu_device *adev = filp->f_inode->i_private;
+ struct smu_context *smu = &adev->smu;
+ unsigned char *buf;
+ int r;
+
+ buf = kvmalloc_array(smu->stb_context.stb_buf_size, sizeof(*buf), GFP_KERNEL);
+ if (!buf)
+ return -ENOMEM;
+
+ r = smu_stb_collect_info(smu, buf, smu->stb_context.stb_buf_size);
+ if (r)
+ goto out;
+
+ filp->private_data = buf;
+
+ return 0;
+
+out:
+ kvfree(buf);
+ return r;
+}
+
+static ssize_t smu_stb_debugfs_read(struct file *filp, char __user *buf, size_t size,
+ loff_t *pos)
+{
+ struct amdgpu_device *adev = filp->f_inode->i_private;
+ struct smu_context *smu = &adev->smu;
+
+
+ if (!filp->private_data)
+ return -EINVAL;
+
+ return simple_read_from_buffer(buf,
+ size,
+ pos, filp->private_data,
+ smu->stb_context.stb_buf_size);
+}
+
+static int smu_stb_debugfs_release(struct inode *inode, struct file *filp)
+{
+ kvfree(filp->private_data);
+ filp->private_data = NULL;
+
+ return 0;
+}
+
+/*
+ * We have to define not only read method but also
+ * open and release because .read takes up to PAGE_SIZE
+ * data each time so and so is invoked multiple times.
+ * We allocate the STB buffer in .open and release it
+ * in .release
+ */
+static const struct file_operations smu_stb_debugfs_fops = {
+ .owner = THIS_MODULE,
+ .open = smu_stb_debugfs_open,
+ .read = smu_stb_debugfs_read,
+ .release = smu_stb_debugfs_release,
+ .llseek = default_llseek,
+};
+
+#endif
+
+void amdgpu_smu_stb_debug_fs_init(struct amdgpu_device *adev)
+{
+#if defined(CONFIG_DEBUG_FS)
+
+ struct smu_context *smu = &adev->smu;
+
+ if (!smu->stb_context.stb_buf_size)
+ return;
+
+ debugfs_create_file_size("amdgpu_smu_stb_dump",
+ S_IRUSR,
+ adev_to_drm(adev)->primary->debugfs_root,
+ adev,
+ &smu_stb_debugfs_fops,
+ smu->stb_context.stb_buf_size);
+#endif
+
+}
diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu11/arcturus_ppt.c b/drivers/gpu/drm/amd/pm/swsmu/smu11/arcturus_ppt.c
index fd1d30a..505d2fb 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/smu11/arcturus_ppt.c
+++ b/drivers/gpu/drm/amd/pm/swsmu/smu11/arcturus_ppt.c
@@ -295,16 +295,6 @@ static int arcturus_allocate_dpm_context(struct smu_context *smu)
return -ENOMEM;
smu_dpm->dpm_context_size = sizeof(struct smu_11_0_dpm_context);
- smu_dpm->dpm_current_power_state = kzalloc(sizeof(struct smu_power_state),
- GFP_KERNEL);
- if (!smu_dpm->dpm_current_power_state)
- return -ENOMEM;
-
- smu_dpm->dpm_request_power_state = kzalloc(sizeof(struct smu_power_state),
- GFP_KERNEL);
- if (!smu_dpm->dpm_request_power_state)
- return -ENOMEM;
-
return 0;
}
@@ -1389,14 +1379,6 @@ static int arcturus_get_power_profile_mode(struct smu_context *smu,
char *buf)
{
DpmActivityMonitorCoeffInt_t activity_monitor;
- static const char *profile_name[] = {
- "BOOTUP_DEFAULT",
- "3D_FULL_SCREEN",
- "POWER_SAVING",
- "VIDEO",
- "VR",
- "COMPUTE",
- "CUSTOM"};
static const char *title[] = {
"PROFILE_INDEX(NAME)",
"CLOCK_TYPE(NAME)",
@@ -1453,7 +1435,7 @@ static int arcturus_get_power_profile_mode(struct smu_context *smu,
}
size += sysfs_emit_at(buf, size, "%2d %14s%s\n",
- i, profile_name[i], (i == smu->power_profile_mode) ? "*" : " ");
+ i, amdgpu_pp_profile_name[i], (i == smu->power_profile_mode) ? "*" : " ");
if (smu_version >= 0x360d00) {
size += sysfs_emit_at(buf, size, "%19s %d(%13s) %7d %7d %7d %7d %7d %7d %7d %7d %7d\n",
@@ -2490,7 +2472,7 @@ static const struct pptable_funcs arcturus_ppt_funcs = {
.deep_sleep_control = smu_v11_0_deep_sleep_control,
.get_fan_parameters = arcturus_get_fan_parameters,
.interrupt_work = smu_v11_0_interrupt_work,
- .set_light_sbr = smu_v11_0_set_light_sbr,
+ .smu_handle_passthrough_sbr = smu_v11_0_handle_passthrough_sbr,
.set_mp1_state = smu_cmn_set_mp1_state,
};
diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu11/navi10_ppt.c b/drivers/gpu/drm/amd/pm/swsmu/smu11/navi10_ppt.c
index 60a5570..2bb7816 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/smu11/navi10_ppt.c
+++ b/drivers/gpu/drm/amd/pm/swsmu/smu11/navi10_ppt.c
@@ -1713,14 +1713,6 @@ static int navi10_get_power_profile_mode(struct smu_context *smu, char *buf)
DpmActivityMonitorCoeffInt_t activity_monitor;
uint32_t i, size = 0;
int16_t workload_type = 0;
- static const char *profile_name[] = {
- "BOOTUP_DEFAULT",
- "3D_FULL_SCREEN",
- "POWER_SAVING",
- "VIDEO",
- "VR",
- "COMPUTE",
- "CUSTOM"};
static const char *title[] = {
"PROFILE_INDEX(NAME)",
"CLOCK_TYPE(NAME)",
@@ -1759,7 +1751,7 @@ static int navi10_get_power_profile_mode(struct smu_context *smu, char *buf)
}
size += sysfs_emit_at(buf, size, "%2d %14s%s:\n",
- i, profile_name[i], (i == smu->power_profile_mode) ? "*" : " ");
+ i, amdgpu_pp_profile_name[i], (i == smu->power_profile_mode) ? "*" : " ");
size += sysfs_emit_at(buf, size, "%19s %d(%13s) %7d %7d %7d %7d %7d %7d %7d %7d %7d\n",
" ",
diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu11/sienna_cichlid_ppt.c b/drivers/gpu/drm/amd/pm/swsmu/smu11/sienna_cichlid_ppt.c
index a410802..777f717 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/smu11/sienna_cichlid_ppt.c
+++ b/drivers/gpu/drm/amd/pm/swsmu/smu11/sienna_cichlid_ppt.c
@@ -80,6 +80,9 @@
(*member) = (smu->smu_table.driver_pptable + offsetof(PPTable_t, field));\
} while(0)
+/* STB FIFO depth is in 64bit units */
+#define SIENNA_CICHLID_STB_DEPTH_UNIT_BYTES 8
+
static int get_table_size(struct smu_context *smu)
{
if (smu->adev->ip_versions[MP1_HWIP][0] == IP_VERSION(11, 0, 13))
@@ -650,6 +653,8 @@ static int sienna_cichlid_allocate_dpm_context(struct smu_context *smu)
return 0;
}
+static void sienna_cichlid_stb_init(struct smu_context *smu);
+
static int sienna_cichlid_init_smc_tables(struct smu_context *smu)
{
int ret = 0;
@@ -662,6 +667,8 @@ static int sienna_cichlid_init_smc_tables(struct smu_context *smu)
if (ret)
return ret;
+ sienna_cichlid_stb_init(smu);
+
return smu_v11_0_init_smc_tables(smu);
}
@@ -1171,7 +1178,7 @@ static int sienna_cichlid_force_clk_levels(struct smu_context *smu,
enum smu_clk_type clk_type, uint32_t mask)
{
struct amdgpu_device *adev = smu->adev;
- int ret = 0, size = 0;
+ int ret = 0;
uint32_t soft_min_level = 0, soft_max_level = 0, min_freq = 0, max_freq = 0;
soft_min_level = mask ? (ffs(mask) - 1) : 0;
@@ -1216,7 +1223,7 @@ static int sienna_cichlid_force_clk_levels(struct smu_context *smu,
if ((clk_type == SMU_GFXCLK) || (clk_type == SMU_SCLK))
amdgpu_gfx_off_ctrl(adev, true);
- return size;
+ return 0;
}
static int sienna_cichlid_populate_umd_state_clk(struct smu_context *smu)
@@ -1342,14 +1349,6 @@ static int sienna_cichlid_get_power_profile_mode(struct smu_context *smu, char *
&(activity_monitor_external.DpmActivityMonitorCoeffInt);
uint32_t i, size = 0;
int16_t workload_type = 0;
- static const char *profile_name[] = {
- "BOOTUP_DEFAULT",
- "3D_FULL_SCREEN",
- "POWER_SAVING",
- "VIDEO",
- "VR",
- "COMPUTE",
- "CUSTOM"};
static const char *title[] = {
"PROFILE_INDEX(NAME)",
"CLOCK_TYPE(NAME)",
@@ -1388,7 +1387,7 @@ static int sienna_cichlid_get_power_profile_mode(struct smu_context *smu, char *
}
size += sysfs_emit_at(buf, size, "%2d %14s%s:\n",
- i, profile_name[i], (i == smu->power_profile_mode) ? "*" : " ");
+ i, amdgpu_pp_profile_name[i], (i == smu->power_profile_mode) ? "*" : " ");
size += sysfs_emit_at(buf, size, "%19s %d(%13s) %7d %7d %7d %7d %7d %7d %7d %7d %7d\n",
" ",
@@ -2135,7 +2134,13 @@ static int sienna_cichlid_od_edit_dpm_table(struct smu_context *smu,
static int sienna_cichlid_run_btc(struct smu_context *smu)
{
- return smu_cmn_send_smc_msg(smu, SMU_MSG_RunDcBtc, NULL);
+ int res;
+
+ res = smu_cmn_send_smc_msg(smu, SMU_MSG_RunDcBtc, NULL);
+ if (res)
+ dev_err(smu->adev->dev, "RunDcBtc failed!\n");
+
+ return res;
}
static int sienna_cichlid_baco_enter(struct smu_context *smu)
@@ -3619,6 +3624,16 @@ static ssize_t sienna_cichlid_get_gpu_metrics(struct smu_context *smu,
gpu_metrics->energy_accumulator =
use_metrics_v2 ? metrics_v2->EnergyAccumulator : metrics->EnergyAccumulator;
+ if (metrics->CurrGfxVoltageOffset)
+ gpu_metrics->voltage_gfx =
+ (155000 - 625 * metrics->CurrGfxVoltageOffset) / 100;
+ if (metrics->CurrMemVidOffset)
+ gpu_metrics->voltage_mem =
+ (155000 - 625 * metrics->CurrMemVidOffset) / 100;
+ if (metrics->CurrSocVoltageOffset)
+ gpu_metrics->voltage_soc =
+ (155000 - 625 * metrics->CurrSocVoltageOffset) / 100;
+
average_gfx_activity = use_metrics_v2 ? metrics_v2->AverageGfxActivity : metrics->AverageGfxActivity;
if (average_gfx_activity <= SMU_11_0_7_GFX_BUSY_THRESHOLD)
gpu_metrics->average_gfxclk_frequency =
@@ -3793,6 +3808,53 @@ static int sienna_cichlid_set_mp1_state(struct smu_context *smu,
return ret;
}
+static void sienna_cichlid_stb_init(struct smu_context *smu)
+{
+ struct amdgpu_device *adev = smu->adev;
+ uint32_t reg;
+
+ reg = RREG32_PCIE(MP1_Public | smnMP1_PMI_3_START);
+ smu->stb_context.enabled = REG_GET_FIELD(reg, MP1_PMI_3_START, ENABLE);
+
+ /* STB is disabled */
+ if (!smu->stb_context.enabled)
+ return;
+
+ spin_lock_init(&smu->stb_context.lock);
+
+ /* STB buffer size in bytes as function of FIFO depth */
+ reg = RREG32_PCIE(MP1_Public | smnMP1_PMI_3_FIFO);
+ smu->stb_context.stb_buf_size = 1 << REG_GET_FIELD(reg, MP1_PMI_3_FIFO, DEPTH);
+ smu->stb_context.stb_buf_size *= SIENNA_CICHLID_STB_DEPTH_UNIT_BYTES;
+
+ dev_info(smu->adev->dev, "STB initialized to %d entries",
+ smu->stb_context.stb_buf_size / SIENNA_CICHLID_STB_DEPTH_UNIT_BYTES);
+
+}
+
+int sienna_cichlid_stb_get_data_direct(struct smu_context *smu,
+ void *buf,
+ uint32_t size)
+{
+ uint32_t *p = buf;
+ struct amdgpu_device *adev = smu->adev;
+
+ /* No need to disable interrupts for now as we don't lock it yet from ISR */
+ spin_lock(&smu->stb_context.lock);
+
+ /*
+ * Read the STB FIFO in units of 32bit since this is the accessor window
+ * (register width) we have.
+ */
+ buf = ((char *) buf) + size;
+ while ((void *)p < buf)
+ *p++ = cpu_to_le32(RREG32_PCIE(MP1_Public | smnMP1_PMI_3));
+
+ spin_unlock(&smu->stb_context.lock);
+
+ return 0;
+}
+
static const struct pptable_funcs sienna_cichlid_ppt_funcs = {
.get_allowed_feature_mask = sienna_cichlid_get_allowed_feature_mask,
.set_default_dpm_table = sienna_cichlid_set_default_dpm_table,
@@ -3882,6 +3944,7 @@ static const struct pptable_funcs sienna_cichlid_ppt_funcs = {
.interrupt_work = smu_v11_0_interrupt_work,
.gpo_control = sienna_cichlid_gpo_control,
.set_mp1_state = sienna_cichlid_set_mp1_state,
+ .stb_collect_info = sienna_cichlid_stb_get_data_direct,
};
void sienna_cichlid_set_ppt_funcs(struct smu_context *smu)
diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu11/smu_v11_0.c b/drivers/gpu/drm/amd/pm/swsmu/smu11/smu_v11_0.c
index 28b7c05..4e9e2cf 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/smu11/smu_v11_0.c
+++ b/drivers/gpu/drm/amd/pm/swsmu/smu11/smu_v11_0.c
@@ -1724,7 +1724,7 @@ int smu_v11_0_mode1_reset(struct smu_context *smu)
return ret;
}
-int smu_v11_0_set_light_sbr(struct smu_context *smu, bool enable)
+int smu_v11_0_handle_passthrough_sbr(struct smu_context *smu, bool enable)
{
int ret = 0;
diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu11/vangogh_ppt.c b/drivers/gpu/drm/amd/pm/swsmu/smu11/vangogh_ppt.c
index c02ed65..5cb07ed 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/smu11/vangogh_ppt.c
+++ b/drivers/gpu/drm/amd/pm/swsmu/smu11/vangogh_ppt.c
@@ -1039,14 +1039,6 @@ static int vangogh_get_dpm_ultimate_freq(struct smu_context *smu,
static int vangogh_get_power_profile_mode(struct smu_context *smu,
char *buf)
{
- static const char *profile_name[] = {
- "BOOTUP_DEFAULT",
- "3D_FULL_SCREEN",
- "POWER_SAVING",
- "VIDEO",
- "VR",
- "COMPUTE",
- "CUSTOM"};
uint32_t i, size = 0;
int16_t workload_type = 0;
@@ -1066,7 +1058,7 @@ static int vangogh_get_power_profile_mode(struct smu_context *smu,
continue;
size += sysfs_emit_at(buf, size, "%2d %14s%s\n",
- i, profile_name[i], (i == smu->power_profile_mode) ? "*" : " ");
+ i, amdgpu_pp_profile_name[i], (i == smu->power_profile_mode) ? "*" : " ");
}
return size;
diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu12/renoir_ppt.c b/drivers/gpu/drm/amd/pm/swsmu/smu12/renoir_ppt.c
index 145f13b..25c4b13 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/smu12/renoir_ppt.c
+++ b/drivers/gpu/drm/amd/pm/swsmu/smu12/renoir_ppt.c
@@ -1095,14 +1095,6 @@ static int renoir_set_watermarks_table(
static int renoir_get_power_profile_mode(struct smu_context *smu,
char *buf)
{
- static const char *profile_name[] = {
- "BOOTUP_DEFAULT",
- "3D_FULL_SCREEN",
- "POWER_SAVING",
- "VIDEO",
- "VR",
- "COMPUTE",
- "CUSTOM"};
uint32_t i, size = 0;
int16_t workload_type = 0;
@@ -1121,7 +1113,7 @@ static int renoir_get_power_profile_mode(struct smu_context *smu,
continue;
size += sysfs_emit_at(buf, size, "%2d %14s%s\n",
- i, profile_name[i], (i == smu->power_profile_mode) ? "*" : " ");
+ i, amdgpu_pp_profile_name[i], (i == smu->power_profile_mode) ? "*" : " ");
}
return size;
diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu12/smu_v12_0.c b/drivers/gpu/drm/amd/pm/swsmu/smu12/smu_v12_0.c
index 43028f2..9c91e79 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/smu12/smu_v12_0.c
+++ b/drivers/gpu/drm/amd/pm/swsmu/smu12/smu_v12_0.c
@@ -120,7 +120,8 @@ int smu_v12_0_powergate_sdma(struct smu_context *smu, bool gate)
int smu_v12_0_set_gfx_cgpg(struct smu_context *smu, bool enable)
{
- if (!(smu->adev->pg_flags & AMD_PG_SUPPORT_GFX_PG))
+ /* Until now the SMU12 only implemented for Renoir series so here neen't do APU check. */
+ if (!(smu->adev->pg_flags & AMD_PG_SUPPORT_GFX_PG) || smu->adev->in_s0ix)
return 0;
return smu_cmn_send_smc_msg_with_param(smu,
diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu13/aldebaran_ppt.c b/drivers/gpu/drm/amd/pm/swsmu/smu13/aldebaran_ppt.c
index 59a7d27..380811b 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/smu13/aldebaran_ppt.c
+++ b/drivers/gpu/drm/amd/pm/swsmu/smu13/aldebaran_ppt.c
@@ -78,6 +78,12 @@
#define smnPCIE_ESM_CTRL 0x111003D0
+/*
+ * SMU support ECCTABLE since version 68.42.0,
+ * use this to check ECCTALE feature whether support
+ */
+#define SUPPORT_ECCTABLE_SMU_VERSION 0x00442a00
+
static const struct smu_temperature_range smu13_thermal_policy[] =
{
{-273150, 99000, 99000, -273150, 99000, 99000, -273150, 99000, 99000},
@@ -135,6 +141,7 @@ static const struct cmn2asic_msg_mapping aldebaran_message_map[SMU_MSG_MAX_COUNT
MSG_MAP(SetUclkDpmMode, PPSMC_MSG_SetUclkDpmMode, 0),
MSG_MAP(GfxDriverResetRecovery, PPSMC_MSG_GfxDriverResetRecovery, 0),
MSG_MAP(BoardPowerCalibration, PPSMC_MSG_BoardPowerCalibration, 0),
+ MSG_MAP(HeavySBR, PPSMC_MSG_HeavySBR, 0),
};
static const struct cmn2asic_mapping aldebaran_clk_map[SMU_CLK_COUNT] = {
@@ -190,6 +197,7 @@ static const struct cmn2asic_mapping aldebaran_table_map[SMU_TABLE_COUNT] = {
TAB_MAP(SMU_METRICS),
TAB_MAP(DRIVER_SMU_CONFIG),
TAB_MAP(I2C_COMMANDS),
+ TAB_MAP(ECCINFO),
};
static const uint8_t aldebaran_throttler_map[] = {
@@ -223,6 +231,9 @@ static int aldebaran_tables_init(struct smu_context *smu)
SMU_TABLE_INIT(tables, SMU_TABLE_I2C_COMMANDS, sizeof(SwI2cRequest_t),
PAGE_SIZE, AMDGPU_GEM_DOMAIN_VRAM);
+ SMU_TABLE_INIT(tables, SMU_TABLE_ECCINFO, sizeof(EccInfoTable_t),
+ PAGE_SIZE, AMDGPU_GEM_DOMAIN_VRAM);
+
smu_table->metrics_table = kzalloc(sizeof(SmuMetrics_t), GFP_KERNEL);
if (!smu_table->metrics_table)
return -ENOMEM;
@@ -235,6 +246,10 @@ static int aldebaran_tables_init(struct smu_context *smu)
return -ENOMEM;
}
+ smu_table->ecc_table = kzalloc(tables[SMU_TABLE_ECCINFO].size, GFP_KERNEL);
+ if (!smu_table->ecc_table)
+ return -ENOMEM;
+
return 0;
}
@@ -248,16 +263,6 @@ static int aldebaran_allocate_dpm_context(struct smu_context *smu)
return -ENOMEM;
smu_dpm->dpm_context_size = sizeof(struct smu_13_0_dpm_context);
- smu_dpm->dpm_current_power_state = kzalloc(sizeof(struct smu_power_state),
- GFP_KERNEL);
- if (!smu_dpm->dpm_current_power_state)
- return -ENOMEM;
-
- smu_dpm->dpm_request_power_state = kzalloc(sizeof(struct smu_power_state),
- GFP_KERNEL);
- if (!smu_dpm->dpm_request_power_state)
- return -ENOMEM;
-
return 0;
}
@@ -1601,7 +1606,8 @@ static void aldebaran_get_unique_id(struct smu_context *smu)
mutex_unlock(&smu->metrics_lock);
adev->unique_id = ((uint64_t)upper32 << 32) | lower32;
- sprintf(adev->serial, "%016llx", adev->unique_id);
+ if (adev->serial[0] == '\0')
+ sprintf(adev->serial, "%016llx", adev->unique_id);
}
static bool aldebaran_is_baco_supported(struct smu_context *smu)
@@ -1621,7 +1627,7 @@ static int aldebaran_allow_xgmi_power_down(struct smu_context *smu, bool en)
{
return smu_cmn_send_smc_msg_with_param(smu,
SMU_MSG_GmiPwrDnControl,
- en ? 1 : 0,
+ en ? 0 : 1,
NULL);
}
@@ -1765,6 +1771,98 @@ static ssize_t aldebaran_get_gpu_metrics(struct smu_context *smu,
return sizeof(struct gpu_metrics_v1_3);
}
+static int aldebaran_check_ecc_table_support(struct smu_context *smu)
+{
+ uint32_t if_version = 0xff, smu_version = 0xff;
+ int ret = 0;
+
+ ret = smu_cmn_get_smc_version(smu, &if_version, &smu_version);
+ if (ret) {
+ /* return not support if failed get smu_version */
+ ret = -EOPNOTSUPP;
+ }
+
+ if (smu_version < SUPPORT_ECCTABLE_SMU_VERSION)
+ ret = -EOPNOTSUPP;
+
+ return ret;
+}
+
+static ssize_t aldebaran_get_ecc_info(struct smu_context *smu,
+ void *table)
+{
+ struct smu_table_context *smu_table = &smu->smu_table;
+ EccInfoTable_t *ecc_table = NULL;
+ struct ecc_info_per_ch *ecc_info_per_channel = NULL;
+ int i, ret = 0;
+ struct umc_ecc_info *eccinfo = (struct umc_ecc_info *)table;
+
+ ret = aldebaran_check_ecc_table_support(smu);
+ if (ret)
+ return ret;
+
+ ret = smu_cmn_update_table(smu,
+ SMU_TABLE_ECCINFO,
+ 0,
+ smu_table->ecc_table,
+ false);
+ if (ret) {
+ dev_info(smu->adev->dev, "Failed to export SMU ecc table!\n");
+ return ret;
+ }
+
+ ecc_table = (EccInfoTable_t *)smu_table->ecc_table;
+
+ for (i = 0; i < ALDEBARAN_UMC_CHANNEL_NUM; i++) {
+ ecc_info_per_channel = &(eccinfo->ecc[i]);
+ ecc_info_per_channel->ce_count_lo_chip =
+ ecc_table->EccInfo[i].ce_count_lo_chip;
+ ecc_info_per_channel->ce_count_hi_chip =
+ ecc_table->EccInfo[i].ce_count_hi_chip;
+ ecc_info_per_channel->mca_umc_status =
+ ecc_table->EccInfo[i].mca_umc_status;
+ ecc_info_per_channel->mca_umc_addr =
+ ecc_table->EccInfo[i].mca_umc_addr;
+ }
+
+ return ret;
+}
+
+static int aldebaran_mode1_reset(struct smu_context *smu)
+{
+ u32 smu_version, fatal_err, param;
+ int ret = 0;
+ struct amdgpu_device *adev = smu->adev;
+ struct amdgpu_ras *ras = amdgpu_ras_get_context(adev);
+
+ fatal_err = 0;
+ param = SMU_RESET_MODE_1;
+
+ /*
+ * PM FW support SMU_MSG_GfxDeviceDriverReset from 68.07
+ */
+ smu_cmn_get_smc_version(smu, NULL, &smu_version);
+ if (smu_version < 0x00440700) {
+ ret = smu_cmn_send_smc_msg(smu, SMU_MSG_Mode1Reset, NULL);
+ }
+ else {
+ /* fatal error triggered by ras, PMFW supports the flag
+ from 68.44.0 */
+ if ((smu_version >= 0x00442c00) && ras &&
+ atomic_read(&ras->in_recovery))
+ fatal_err = 1;
+
+ param |= (fatal_err << 16);
+ ret = smu_cmn_send_smc_msg_with_param(smu,
+ SMU_MSG_GfxDeviceDriverReset, param, NULL);
+ }
+
+ if (!ret)
+ msleep(SMU13_MODE1_RESET_WAIT_TIME_IN_MS);
+
+ return ret;
+}
+
static int aldebaran_mode2_reset(struct smu_context *smu)
{
u32 smu_version;
@@ -1816,6 +1914,14 @@ static int aldebaran_mode2_reset(struct smu_context *smu)
return ret;
}
+static int aldebaran_smu_handle_passthrough_sbr(struct smu_context *smu, bool enable)
+{
+ int ret = 0;
+ ret = smu_cmn_send_smc_msg_with_param(smu, SMU_MSG_HeavySBR, enable ? 1 : 0, NULL);
+
+ return ret;
+}
+
static bool aldebaran_is_mode1_reset_supported(struct smu_context *smu)
{
#if 0
@@ -1925,13 +2031,15 @@ static const struct pptable_funcs aldebaran_ppt_funcs = {
.get_gpu_metrics = aldebaran_get_gpu_metrics,
.mode1_reset_is_support = aldebaran_is_mode1_reset_supported,
.mode2_reset_is_support = aldebaran_is_mode2_reset_supported,
- .mode1_reset = smu_v13_0_mode1_reset,
+ .smu_handle_passthrough_sbr = aldebaran_smu_handle_passthrough_sbr,
+ .mode1_reset = aldebaran_mode1_reset,
.set_mp1_state = aldebaran_set_mp1_state,
.mode2_reset = aldebaran_mode2_reset,
.wait_for_event = smu_v13_0_wait_for_event,
.i2c_init = aldebaran_i2c_control_init,
.i2c_fini = aldebaran_i2c_control_fini,
.send_hbm_bad_pages_num = aldebaran_smu_send_hbm_bad_page_num,
+ .get_ecc_info = aldebaran_get_ecc_info,
};
void aldebaran_set_ppt_funcs(struct smu_context *smu)
diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0.c b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0.c
index 19a5d2c..b54790d 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0.c
+++ b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0.c
@@ -60,8 +60,6 @@ MODULE_FIRMWARE("amdgpu/aldebaran_smc.bin");
#define SMU13_VOLTAGE_SCALE 4
-#define SMU13_MODE1_RESET_WAIT_TIME_IN_MS 500 //500ms
-
#define LINK_WIDTH_MAX 6
#define LINK_SPEED_MAX 3
@@ -214,7 +212,7 @@ int smu_v13_0_check_fw_version(struct smu_context *smu)
if (smu->is_apu)
adev->pm.fw_version = smu_version;
- switch (smu->adev->ip_versions[MP1_HWIP][0]) {
+ switch (adev->ip_versions[MP1_HWIP][0]) {
case IP_VERSION(13, 0, 2):
smu->smc_driver_if_version = SMU13_DRIVER_IF_VERSION_ALDE;
break;
@@ -223,13 +221,15 @@ int smu_v13_0_check_fw_version(struct smu_context *smu)
smu->smc_driver_if_version = SMU13_DRIVER_IF_VERSION_YELLOW_CARP;
break;
default:
- dev_err(smu->adev->dev, "smu unsupported IP version: 0x%x.\n",
- smu->adev->ip_versions[MP1_HWIP][0]);
+ dev_err(adev->dev, "smu unsupported IP version: 0x%x.\n",
+ adev->ip_versions[MP1_HWIP][0]);
smu->smc_driver_if_version = SMU13_DRIVER_IF_VERSION_INV;
break;
}
- dev_info(smu->adev->dev, "smu fw reported version = 0x%08x (%d.%d.%d)\n",
+ /* only for dGPU w/ SMU13*/
+ if (adev->pm.fw)
+ dev_dbg(adev->dev, "smu fw reported version = 0x%08x (%d.%d.%d)\n",
smu_version, smu_major, smu_minor, smu_debug);
/*
@@ -241,11 +241,11 @@ int smu_v13_0_check_fw_version(struct smu_context *smu)
* of halt driver loading.
*/
if (if_version != smu->smc_driver_if_version) {
- dev_info(smu->adev->dev, "smu driver if version = 0x%08x, smu fw if version = 0x%08x, "
+ dev_info(adev->dev, "smu driver if version = 0x%08x, smu fw if version = 0x%08x, "
"smu fw version = 0x%08x (%d.%d.%d)\n",
smu->smc_driver_if_version, if_version,
smu_version, smu_major, smu_minor, smu_debug);
- dev_warn(smu->adev->dev, "SMU driver if version not matched\n");
+ dev_warn(adev->dev, "SMU driver if version not matched\n");
}
return ret;
@@ -433,8 +433,10 @@ int smu_v13_0_fini_smc_tables(struct smu_context *smu)
kfree(smu_table->hardcode_pptable);
smu_table->hardcode_pptable = NULL;
+ kfree(smu_table->ecc_table);
kfree(smu_table->metrics_table);
kfree(smu_table->watermarks_table);
+ smu_table->ecc_table = NULL;
smu_table->metrics_table = NULL;
smu_table->watermarks_table = NULL;
smu_table->metrics_time = 0;
@@ -1427,25 +1429,6 @@ int smu_v13_0_set_azalia_d3_pme(struct smu_context *smu)
return ret;
}
-int smu_v13_0_mode1_reset(struct smu_context *smu)
-{
- u32 smu_version;
- int ret = 0;
- /*
- * PM FW support SMU_MSG_GfxDeviceDriverReset from 68.07
- */
- smu_cmn_get_smc_version(smu, NULL, &smu_version);
- if (smu_version < 0x00440700)
- ret = smu_cmn_send_smc_msg(smu, SMU_MSG_Mode1Reset, NULL);
- else
- ret = smu_cmn_send_smc_msg_with_param(smu, SMU_MSG_GfxDeviceDriverReset, SMU_RESET_MODE_1, NULL);
-
- if (!ret)
- msleep(SMU13_MODE1_RESET_WAIT_TIME_IN_MS);
-
- return ret;
-}
-
static int smu_v13_0_wait_for_reset_complete(struct smu_context *smu,
uint64_t event_arg)
{
diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu_cmn.c b/drivers/gpu/drm/amd/pm/swsmu/smu_cmn.c
index ea6f50c..ee1a312 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/smu_cmn.c
+++ b/drivers/gpu/drm/amd/pm/swsmu/smu_cmn.c
@@ -94,10 +94,10 @@ static void smu_cmn_read_arg(struct smu_context *smu,
/**
* __smu_cmn_poll_stat -- poll for a status from the SMU
- * smu: a pointer to SMU context
+ * @smu: a pointer to SMU context
*
* Returns the status of the SMU, which could be,
- * 0, the SMU is busy with your previous command;
+ * 0, the SMU is busy with your command;
* 1, execution status: success, execution result: success;
* 0xFF, execution status: success, execution result: failure;
* 0xFE, unknown command;
@@ -257,10 +257,11 @@ int smu_cmn_send_msg_without_waiting(struct smu_context *smu,
uint16_t msg_index,
uint32_t param)
{
+ struct amdgpu_device *adev = smu->adev;
u32 reg;
int res;
- if (smu->adev->no_hw_access)
+ if (adev->no_hw_access)
return 0;
reg = __smu_cmn_poll_stat(smu);
@@ -272,6 +273,12 @@ int smu_cmn_send_msg_without_waiting(struct smu_context *smu,
__smu_cmn_send_msg(smu, msg_index, param);
res = 0;
Out:
+ if (unlikely(adev->pm.smu_debug_mask & SMU_DEBUG_HALT_ON_ERROR) &&
+ res && (res != -ETIME)) {
+ amdgpu_device_halt(adev);
+ WARN_ON(1);
+ }
+
return res;
}
@@ -288,9 +295,18 @@ int smu_cmn_send_msg_without_waiting(struct smu_context *smu,
int smu_cmn_wait_for_response(struct smu_context *smu)
{
u32 reg;
+ int res;
reg = __smu_cmn_poll_stat(smu);
- return __smu_cmn_reg2errno(smu, reg);
+ res = __smu_cmn_reg2errno(smu, reg);
+
+ if (unlikely(smu->adev->pm.smu_debug_mask & SMU_DEBUG_HALT_ON_ERROR) &&
+ res && (res != -ETIME)) {
+ amdgpu_device_halt(smu->adev);
+ WARN_ON(1);
+ }
+
+ return res;
}
/**
@@ -328,10 +344,11 @@ int smu_cmn_send_smc_msg_with_param(struct smu_context *smu,
uint32_t param,
uint32_t *read_arg)
{
+ struct amdgpu_device *adev = smu->adev;
int res, index;
u32 reg;
- if (smu->adev->no_hw_access)
+ if (adev->no_hw_access)
return 0;
index = smu_cmn_to_asic_specific_index(smu,
@@ -352,11 +369,16 @@ int smu_cmn_send_smc_msg_with_param(struct smu_context *smu,
__smu_cmn_send_msg(smu, (uint16_t) index, param);
reg = __smu_cmn_poll_stat(smu);
res = __smu_cmn_reg2errno(smu, reg);
- if (res == -EREMOTEIO)
+ if (res != 0)
__smu_cmn_reg_print_error(smu, reg, index, param, msg);
if (read_arg)
smu_cmn_read_arg(smu, read_arg);
Out:
+ if (unlikely(adev->pm.smu_debug_mask & SMU_DEBUG_HALT_ON_ERROR) && res) {
+ amdgpu_device_halt(adev);
+ WARN_ON(1);
+ }
+
mutex_unlock(&smu->message_lock);
return res;
}
diff --git a/drivers/gpu/drm/arm/Kconfig b/drivers/gpu/drm/arm/Kconfig
index 3a9e966..58a2428 100644
--- a/drivers/gpu/drm/arm/Kconfig
+++ b/drivers/gpu/drm/arm/Kconfig
@@ -6,7 +6,6 @@
depends on DRM && OF && (ARM || ARM64 || COMPILE_TEST)
depends on COMMON_CLK
select DRM_KMS_HELPER
- select DRM_KMS_CMA_HELPER
help
Choose this option if you have an ARM High Definition Colour LCD
controller.
@@ -27,7 +26,6 @@
depends on DRM && OF && (ARM || ARM64 || COMPILE_TEST)
depends on COMMON_CLK
select DRM_KMS_HELPER
- select DRM_KMS_CMA_HELPER
select DRM_GEM_CMA_HELPER
select VIDEOMODE_HELPERS
help
diff --git a/drivers/gpu/drm/arm/display/Kconfig b/drivers/gpu/drm/arm/display/Kconfig
index cec0639..e91598b 100644
--- a/drivers/gpu/drm/arm/display/Kconfig
+++ b/drivers/gpu/drm/arm/display/Kconfig
@@ -4,7 +4,6 @@
depends on DRM && OF
depends on COMMON_CLK
select DRM_KMS_HELPER
- select DRM_KMS_CMA_HELPER
select DRM_GEM_CMA_HELPER
select VIDEOMODE_HELPERS
help
diff --git a/drivers/gpu/drm/aspeed/Kconfig b/drivers/gpu/drm/aspeed/Kconfig
index 5e95bce..024ccab 100644
--- a/drivers/gpu/drm/aspeed/Kconfig
+++ b/drivers/gpu/drm/aspeed/Kconfig
@@ -5,7 +5,7 @@
depends on (COMPILE_TEST || ARCH_ASPEED)
depends on MMU
select DRM_KMS_HELPER
- select DRM_KMS_CMA_HELPER
+ select DRM_GEM_CMA_HELPER
select DMA_CMA if HAVE_DMA_CONTIGUOUS
select CMA if HAVE_DMA_CONTIGUOUS
select MFD_SYSCON
diff --git a/drivers/gpu/drm/ast/Makefile b/drivers/gpu/drm/ast/Makefile
index 438a2d0..21f7116 100644
--- a/drivers/gpu/drm/ast/Makefile
+++ b/drivers/gpu/drm/ast/Makefile
@@ -3,6 +3,6 @@
# Makefile for the drm device driver. This driver provides support for the
# Direct Rendering Infrastructure (DRI) in XFree86 4.1.0 and higher.
-ast-y := ast_drv.o ast_main.o ast_mm.o ast_mode.o ast_post.o ast_dp501.o
+ast-y := ast_drv.o ast_i2c.o ast_main.o ast_mm.o ast_mode.o ast_post.o ast_dp501.o
obj-$(CONFIG_DRM_AST) := ast.o
diff --git a/drivers/gpu/drm/ast/ast_drv.c b/drivers/gpu/drm/ast/ast_drv.c
index 86d5cd7..6d8613f 100644
--- a/drivers/gpu/drm/ast/ast_drv.c
+++ b/drivers/gpu/drm/ast/ast_drv.c
@@ -26,7 +26,6 @@
* Authors: Dave Airlie <airlied@redhat.com>
*/
-#include <linux/console.h>
#include <linux/module.h>
#include <linux/pci.h>
@@ -233,7 +232,7 @@ static struct pci_driver ast_pci_driver = {
static int __init ast_init(void)
{
- if (vgacon_text_force() && ast_modeset == -1)
+ if (drm_firmware_drivers_only() && ast_modeset == -1)
return -EINVAL;
if (ast_modeset == 0)
diff --git a/drivers/gpu/drm/ast/ast_drv.h b/drivers/gpu/drm/ast/ast_drv.h
index 2cfce7d..00bfa41f 100644
--- a/drivers/gpu/drm/ast/ast_drv.h
+++ b/drivers/gpu/drm/ast/ast_drv.h
@@ -357,4 +357,7 @@ bool ast_dp501_read_edid(struct drm_device *dev, u8 *ediddata);
u8 ast_get_dp501_max_clk(struct drm_device *dev);
void ast_init_3rdtx(struct drm_device *dev);
+/* ast_i2c.c */
+struct ast_i2c_chan *ast_i2c_create(struct drm_device *dev);
+
#endif
diff --git a/drivers/gpu/drm/ast/ast_i2c.c b/drivers/gpu/drm/ast/ast_i2c.c
new file mode 100644
index 0000000..93e91c3
--- /dev/null
+++ b/drivers/gpu/drm/ast/ast_i2c.c
@@ -0,0 +1,152 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the
+ * "Software"), to deal in the Software without restriction, including
+ * without limitation the rights to use, copy, modify, merge, publish,
+ * distribute, sub license, and/or sell copies of the Software, and to
+ * permit persons to whom the Software is furnished to do so, subject to
+ * the following conditions:
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDERS, AUTHORS AND/OR ITS SUPPLIERS BE LIABLE FOR ANY CLAIM,
+ * DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
+ * OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
+ * USE OR OTHER DEALINGS IN THE SOFTWARE.
+ *
+ * The above copyright notice and this permission notice (including the
+ * next paragraph) shall be included in all copies or substantial portions
+ * of the Software.
+ */
+
+#include <drm/drm_managed.h>
+#include <drm/drm_print.h>
+
+#include "ast_drv.h"
+
+static void ast_i2c_setsda(void *i2c_priv, int data)
+{
+ struct ast_i2c_chan *i2c = i2c_priv;
+ struct ast_private *ast = to_ast_private(i2c->dev);
+ int i;
+ u8 ujcrb7, jtemp;
+
+ for (i = 0; i < 0x10000; i++) {
+ ujcrb7 = ((data & 0x01) ? 0 : 1) << 2;
+ ast_set_index_reg_mask(ast, AST_IO_CRTC_PORT, 0xb7, 0xf1, ujcrb7);
+ jtemp = ast_get_index_reg_mask(ast, AST_IO_CRTC_PORT, 0xb7, 0x04);
+ if (ujcrb7 == jtemp)
+ break;
+ }
+}
+
+static void ast_i2c_setscl(void *i2c_priv, int clock)
+{
+ struct ast_i2c_chan *i2c = i2c_priv;
+ struct ast_private *ast = to_ast_private(i2c->dev);
+ int i;
+ u8 ujcrb7, jtemp;
+
+ for (i = 0; i < 0x10000; i++) {
+ ujcrb7 = ((clock & 0x01) ? 0 : 1);
+ ast_set_index_reg_mask(ast, AST_IO_CRTC_PORT, 0xb7, 0xf4, ujcrb7);
+ jtemp = ast_get_index_reg_mask(ast, AST_IO_CRTC_PORT, 0xb7, 0x01);
+ if (ujcrb7 == jtemp)
+ break;
+ }
+}
+
+static int ast_i2c_getsda(void *i2c_priv)
+{
+ struct ast_i2c_chan *i2c = i2c_priv;
+ struct ast_private *ast = to_ast_private(i2c->dev);
+ uint32_t val, val2, count, pass;
+
+ count = 0;
+ pass = 0;
+ val = (ast_get_index_reg_mask(ast, AST_IO_CRTC_PORT, 0xb7, 0x20) >> 5) & 0x01;
+ do {
+ val2 = (ast_get_index_reg_mask(ast, AST_IO_CRTC_PORT, 0xb7, 0x20) >> 5) & 0x01;
+ if (val == val2) {
+ pass++;
+ } else {
+ pass = 0;
+ val = (ast_get_index_reg_mask(ast, AST_IO_CRTC_PORT, 0xb7, 0x20) >> 5) & 0x01;
+ }
+ } while ((pass < 5) && (count++ < 0x10000));
+
+ return val & 1 ? 1 : 0;
+}
+
+static int ast_i2c_getscl(void *i2c_priv)
+{
+ struct ast_i2c_chan *i2c = i2c_priv;
+ struct ast_private *ast = to_ast_private(i2c->dev);
+ uint32_t val, val2, count, pass;
+
+ count = 0;
+ pass = 0;
+ val = (ast_get_index_reg_mask(ast, AST_IO_CRTC_PORT, 0xb7, 0x10) >> 4) & 0x01;
+ do {
+ val2 = (ast_get_index_reg_mask(ast, AST_IO_CRTC_PORT, 0xb7, 0x10) >> 4) & 0x01;
+ if (val == val2) {
+ pass++;
+ } else {
+ pass = 0;
+ val = (ast_get_index_reg_mask(ast, AST_IO_CRTC_PORT, 0xb7, 0x10) >> 4) & 0x01;
+ }
+ } while ((pass < 5) && (count++ < 0x10000));
+
+ return val & 1 ? 1 : 0;
+}
+
+static void ast_i2c_release(struct drm_device *dev, void *res)
+{
+ struct ast_i2c_chan *i2c = res;
+
+ i2c_del_adapter(&i2c->adapter);
+ kfree(i2c);
+}
+
+struct ast_i2c_chan *ast_i2c_create(struct drm_device *dev)
+{
+ struct ast_i2c_chan *i2c;
+ int ret;
+
+ i2c = kzalloc(sizeof(struct ast_i2c_chan), GFP_KERNEL);
+ if (!i2c)
+ return NULL;
+
+ i2c->adapter.owner = THIS_MODULE;
+ i2c->adapter.class = I2C_CLASS_DDC;
+ i2c->adapter.dev.parent = dev->dev;
+ i2c->dev = dev;
+ i2c_set_adapdata(&i2c->adapter, i2c);
+ snprintf(i2c->adapter.name, sizeof(i2c->adapter.name),
+ "AST i2c bit bus");
+ i2c->adapter.algo_data = &i2c->bit;
+
+ i2c->bit.udelay = 20;
+ i2c->bit.timeout = 2;
+ i2c->bit.data = i2c;
+ i2c->bit.setsda = ast_i2c_setsda;
+ i2c->bit.setscl = ast_i2c_setscl;
+ i2c->bit.getsda = ast_i2c_getsda;
+ i2c->bit.getscl = ast_i2c_getscl;
+ ret = i2c_bit_add_bus(&i2c->adapter);
+ if (ret) {
+ drm_err(dev, "Failed to register bit i2c\n");
+ goto out_kfree;
+ }
+
+ ret = drmm_add_action_or_reset(dev, ast_i2c_release, i2c);
+ if (ret)
+ return NULL;
+ return i2c;
+
+out_kfree:
+ kfree(i2c);
+ return NULL;
+}
diff --git a/drivers/gpu/drm/ast/ast_mode.c b/drivers/gpu/drm/ast/ast_mode.c
index d5c98f7..956c898 100644
--- a/drivers/gpu/drm/ast/ast_mode.c
+++ b/drivers/gpu/drm/ast/ast_mode.c
@@ -47,9 +47,6 @@
#include "ast_drv.h"
#include "ast_tables.h"
-static struct ast_i2c_chan *ast_i2c_create(struct drm_device *dev);
-static void ast_i2c_destroy(struct ast_i2c_chan *i2c);
-
static inline void ast_load_palette_index(struct ast_private *ast,
u8 index, u8 red, u8 green,
u8 blue)
@@ -1213,9 +1210,9 @@ static int ast_get_modes(struct drm_connector *connector)
{
struct ast_connector *ast_connector = to_ast_connector(connector);
struct ast_private *ast = to_ast_private(connector->dev);
- struct edid *edid;
- int ret;
+ struct edid *edid = NULL;
bool flags = false;
+ int ret;
if (ast->tx_chip_type == AST_TX_DP501) {
ast->dp501_maxclk = 0xff;
@@ -1229,7 +1226,7 @@ static int ast_get_modes(struct drm_connector *connector)
else
kfree(edid);
}
- if (!flags)
+ if (!flags && ast_connector->i2c)
edid = drm_get_edid(connector, &ast_connector->i2c->adapter);
if (edid) {
drm_connector_update_edid_property(&ast_connector->base, edid);
@@ -1303,14 +1300,6 @@ static enum drm_mode_status ast_mode_valid(struct drm_connector *connector,
return flags;
}
-static void ast_connector_destroy(struct drm_connector *connector)
-{
- struct ast_connector *ast_connector = to_ast_connector(connector);
-
- ast_i2c_destroy(ast_connector->i2c);
- drm_connector_cleanup(connector);
-}
-
static const struct drm_connector_helper_funcs ast_connector_helper_funcs = {
.get_modes = ast_get_modes,
.mode_valid = ast_mode_valid,
@@ -1319,7 +1308,7 @@ static const struct drm_connector_helper_funcs ast_connector_helper_funcs = {
static const struct drm_connector_funcs ast_connector_funcs = {
.reset = drm_atomic_helper_connector_reset,
.fill_modes = drm_helper_probe_single_connector_modes,
- .destroy = ast_connector_destroy,
+ .destroy = drm_connector_cleanup,
.atomic_duplicate_state = drm_atomic_helper_connector_duplicate_state,
.atomic_destroy_state = drm_atomic_helper_connector_destroy_state,
};
@@ -1335,10 +1324,13 @@ static int ast_connector_init(struct drm_device *dev)
if (!ast_connector->i2c)
drm_err(dev, "failed to add ddc bus for connector\n");
- drm_connector_init_with_ddc(dev, connector,
- &ast_connector_funcs,
- DRM_MODE_CONNECTOR_VGA,
- &ast_connector->i2c->adapter);
+ if (ast_connector->i2c)
+ drm_connector_init_with_ddc(dev, connector, &ast_connector_funcs,
+ DRM_MODE_CONNECTOR_VGA,
+ &ast_connector->i2c->adapter);
+ else
+ drm_connector_init(dev, connector, &ast_connector_funcs,
+ DRM_MODE_CONNECTOR_VGA);
drm_connector_helper_add(connector, &ast_connector_helper_funcs);
@@ -1416,124 +1408,3 @@ int ast_mode_config_init(struct ast_private *ast)
return 0;
}
-
-static int get_clock(void *i2c_priv)
-{
- struct ast_i2c_chan *i2c = i2c_priv;
- struct ast_private *ast = to_ast_private(i2c->dev);
- uint32_t val, val2, count, pass;
-
- count = 0;
- pass = 0;
- val = (ast_get_index_reg_mask(ast, AST_IO_CRTC_PORT, 0xb7, 0x10) >> 4) & 0x01;
- do {
- val2 = (ast_get_index_reg_mask(ast, AST_IO_CRTC_PORT, 0xb7, 0x10) >> 4) & 0x01;
- if (val == val2) {
- pass++;
- } else {
- pass = 0;
- val = (ast_get_index_reg_mask(ast, AST_IO_CRTC_PORT, 0xb7, 0x10) >> 4) & 0x01;
- }
- } while ((pass < 5) && (count++ < 0x10000));
-
- return val & 1 ? 1 : 0;
-}
-
-static int get_data(void *i2c_priv)
-{
- struct ast_i2c_chan *i2c = i2c_priv;
- struct ast_private *ast = to_ast_private(i2c->dev);
- uint32_t val, val2, count, pass;
-
- count = 0;
- pass = 0;
- val = (ast_get_index_reg_mask(ast, AST_IO_CRTC_PORT, 0xb7, 0x20) >> 5) & 0x01;
- do {
- val2 = (ast_get_index_reg_mask(ast, AST_IO_CRTC_PORT, 0xb7, 0x20) >> 5) & 0x01;
- if (val == val2) {
- pass++;
- } else {
- pass = 0;
- val = (ast_get_index_reg_mask(ast, AST_IO_CRTC_PORT, 0xb7, 0x20) >> 5) & 0x01;
- }
- } while ((pass < 5) && (count++ < 0x10000));
-
- return val & 1 ? 1 : 0;
-}
-
-static void set_clock(void *i2c_priv, int clock)
-{
- struct ast_i2c_chan *i2c = i2c_priv;
- struct ast_private *ast = to_ast_private(i2c->dev);
- int i;
- u8 ujcrb7, jtemp;
-
- for (i = 0; i < 0x10000; i++) {
- ujcrb7 = ((clock & 0x01) ? 0 : 1);
- ast_set_index_reg_mask(ast, AST_IO_CRTC_PORT, 0xb7, 0xf4, ujcrb7);
- jtemp = ast_get_index_reg_mask(ast, AST_IO_CRTC_PORT, 0xb7, 0x01);
- if (ujcrb7 == jtemp)
- break;
- }
-}
-
-static void set_data(void *i2c_priv, int data)
-{
- struct ast_i2c_chan *i2c = i2c_priv;
- struct ast_private *ast = to_ast_private(i2c->dev);
- int i;
- u8 ujcrb7, jtemp;
-
- for (i = 0; i < 0x10000; i++) {
- ujcrb7 = ((data & 0x01) ? 0 : 1) << 2;
- ast_set_index_reg_mask(ast, AST_IO_CRTC_PORT, 0xb7, 0xf1, ujcrb7);
- jtemp = ast_get_index_reg_mask(ast, AST_IO_CRTC_PORT, 0xb7, 0x04);
- if (ujcrb7 == jtemp)
- break;
- }
-}
-
-static struct ast_i2c_chan *ast_i2c_create(struct drm_device *dev)
-{
- struct ast_i2c_chan *i2c;
- int ret;
-
- i2c = kzalloc(sizeof(struct ast_i2c_chan), GFP_KERNEL);
- if (!i2c)
- return NULL;
-
- i2c->adapter.owner = THIS_MODULE;
- i2c->adapter.class = I2C_CLASS_DDC;
- i2c->adapter.dev.parent = dev->dev;
- i2c->dev = dev;
- i2c_set_adapdata(&i2c->adapter, i2c);
- snprintf(i2c->adapter.name, sizeof(i2c->adapter.name),
- "AST i2c bit bus");
- i2c->adapter.algo_data = &i2c->bit;
-
- i2c->bit.udelay = 20;
- i2c->bit.timeout = 2;
- i2c->bit.data = i2c;
- i2c->bit.setsda = set_data;
- i2c->bit.setscl = set_clock;
- i2c->bit.getsda = get_data;
- i2c->bit.getscl = get_clock;
- ret = i2c_bit_add_bus(&i2c->adapter);
- if (ret) {
- drm_err(dev, "Failed to register bit i2c\n");
- goto out_free;
- }
-
- return i2c;
-out_free:
- kfree(i2c);
- return NULL;
-}
-
-static void ast_i2c_destroy(struct ast_i2c_chan *i2c)
-{
- if (!i2c)
- return;
- i2c_del_adapter(&i2c->adapter);
- kfree(i2c);
-}
diff --git a/drivers/gpu/drm/atmel-hlcdc/Kconfig b/drivers/gpu/drm/atmel-hlcdc/Kconfig
index 5f67f00..8ae679f 100644
--- a/drivers/gpu/drm/atmel-hlcdc/Kconfig
+++ b/drivers/gpu/drm/atmel-hlcdc/Kconfig
@@ -4,7 +4,6 @@
depends on DRM && OF && COMMON_CLK && MFD_ATMEL_HLCDC && ARM
select DRM_GEM_CMA_HELPER
select DRM_KMS_HELPER
- select DRM_KMS_CMA_HELPER
select DRM_PANEL
help
Choose this option if you have an ATMEL SoC with an HLCDC display
diff --git a/drivers/gpu/drm/bridge/Kconfig b/drivers/gpu/drm/bridge/Kconfig
index 431b6e1..61db5a6 100644
--- a/drivers/gpu/drm/bridge/Kconfig
+++ b/drivers/gpu/drm/bridge/Kconfig
@@ -182,6 +182,7 @@
config DRM_PARADE_PS8640
tristate "Parade PS8640 MIPI DSI to eDP Converter"
depends on OF
+ select DRM_DP_AUX_BUS
select DRM_KMS_HELPER
select DRM_MIPI_DSI
select DRM_PANEL
diff --git a/drivers/gpu/drm/bridge/adv7511/adv7511.h b/drivers/gpu/drm/bridge/adv7511/adv7511.h
index 05e3abb..592ecfc 100644
--- a/drivers/gpu/drm/bridge/adv7511/adv7511.h
+++ b/drivers/gpu/drm/bridge/adv7511/adv7511.h
@@ -401,7 +401,6 @@ void adv7533_mode_set(struct adv7511 *adv, const struct drm_display_mode *mode);
int adv7533_patch_registers(struct adv7511 *adv);
int adv7533_patch_cec_registers(struct adv7511 *adv);
int adv7533_attach_dsi(struct adv7511 *adv);
-void adv7533_detach_dsi(struct adv7511 *adv);
int adv7533_parse_dt(struct device_node *np, struct adv7511 *adv);
#ifdef CONFIG_DRM_I2C_ADV7511_AUDIO
diff --git a/drivers/gpu/drm/bridge/adv7511/adv7511_drv.c b/drivers/gpu/drm/bridge/adv7511/adv7511_drv.c
index 76555ae..f8e5da1 100644
--- a/drivers/gpu/drm/bridge/adv7511/adv7511_drv.c
+++ b/drivers/gpu/drm/bridge/adv7511/adv7511_drv.c
@@ -910,9 +910,6 @@ static int adv7511_bridge_attach(struct drm_bridge *bridge,
return ret;
}
- if (adv->type == ADV7533 || adv->type == ADV7535)
- ret = adv7533_attach_dsi(adv);
-
if (adv->i2c_main->irq)
regmap_write(adv->regmap, ADV7511_REG_INT_ENABLE(0),
ADV7511_INT0_HPD);
@@ -1288,8 +1285,18 @@ static int adv7511_probe(struct i2c_client *i2c, const struct i2c_device_id *id)
drm_bridge_add(&adv7511->bridge);
adv7511_audio_init(dev, adv7511);
+
+ if (adv7511->type == ADV7533 || adv7511->type == ADV7535) {
+ ret = adv7533_attach_dsi(adv7511);
+ if (ret)
+ goto err_unregister_audio;
+ }
+
return 0;
+err_unregister_audio:
+ adv7511_audio_exit(adv7511);
+ drm_bridge_remove(&adv7511->bridge);
err_unregister_cec:
i2c_unregister_device(adv7511->i2c_cec);
clk_disable_unprepare(adv7511->cec_clk);
@@ -1307,8 +1314,6 @@ static int adv7511_remove(struct i2c_client *i2c)
{
struct adv7511 *adv7511 = i2c_get_clientdata(i2c);
- if (adv7511->type == ADV7533 || adv7511->type == ADV7535)
- adv7533_detach_dsi(adv7511);
i2c_unregister_device(adv7511->i2c_cec);
clk_disable_unprepare(adv7511->cec_clk);
diff --git a/drivers/gpu/drm/bridge/adv7511/adv7533.c b/drivers/gpu/drm/bridge/adv7511/adv7533.c
index 59d718b..eb7579d 100644
--- a/drivers/gpu/drm/bridge/adv7511/adv7533.c
+++ b/drivers/gpu/drm/bridge/adv7511/adv7533.c
@@ -153,11 +153,10 @@ int adv7533_attach_dsi(struct adv7511 *adv)
return -EPROBE_DEFER;
}
- dsi = mipi_dsi_device_register_full(host, &info);
+ dsi = devm_mipi_dsi_device_register_full(dev, host, &info);
if (IS_ERR(dsi)) {
dev_err(dev, "failed to create dsi device\n");
- ret = PTR_ERR(dsi);
- goto err_dsi_device;
+ return PTR_ERR(dsi);
}
adv->dsi = dsi;
@@ -167,24 +166,13 @@ int adv7533_attach_dsi(struct adv7511 *adv)
dsi->mode_flags = MIPI_DSI_MODE_VIDEO | MIPI_DSI_MODE_VIDEO_SYNC_PULSE |
MIPI_DSI_MODE_NO_EOT_PACKET | MIPI_DSI_MODE_VIDEO_HSE;
- ret = mipi_dsi_attach(dsi);
+ ret = devm_mipi_dsi_attach(dev, dsi);
if (ret < 0) {
dev_err(dev, "failed to attach dsi to host\n");
- goto err_dsi_attach;
+ return ret;
}
return 0;
-
-err_dsi_attach:
- mipi_dsi_device_unregister(dsi);
-err_dsi_device:
- return ret;
-}
-
-void adv7533_detach_dsi(struct adv7511 *adv)
-{
- mipi_dsi_detach(adv->dsi);
- mipi_dsi_device_unregister(adv->dsi);
}
int adv7533_parse_dt(struct device_node *np, struct adv7511 *adv)
diff --git a/drivers/gpu/drm/bridge/analogix/analogix_dp_reg.c b/drivers/gpu/drm/bridge/analogix/analogix_dp_reg.c
index cab6c8b..6a4f20f 100644
--- a/drivers/gpu/drm/bridge/analogix/analogix_dp_reg.c
+++ b/drivers/gpu/drm/bridge/analogix/analogix_dp_reg.c
@@ -998,11 +998,21 @@ int analogix_dp_send_psr_spd(struct analogix_dp_device *dp,
if (!blocking)
return 0;
+ /*
+ * db[1]!=0: entering PSR, wait for fully active remote frame buffer.
+ * db[1]==0: exiting PSR, wait for either
+ * (a) ACTIVE_RESYNC - the sink "must display the
+ * incoming active frames from the Source device with no visible
+ * glitches and/or artifacts", even though timings may still be
+ * re-synchronizing; or
+ * (b) INACTIVE - the transition is fully complete.
+ */
ret = readx_poll_timeout(analogix_dp_get_psr_status, dp, psr_status,
psr_status >= 0 &&
((vsc->db[1] && psr_status == DP_PSR_SINK_ACTIVE_RFB) ||
- (!vsc->db[1] && psr_status == DP_PSR_SINK_INACTIVE)), 1500,
- DP_TIMEOUT_PSR_LOOP_MS * 1000);
+ (!vsc->db[1] && (psr_status == DP_PSR_SINK_ACTIVE_RESYNC ||
+ psr_status == DP_PSR_SINK_INACTIVE))),
+ 1500, DP_TIMEOUT_PSR_LOOP_MS * 1000);
if (ret) {
dev_warn(dp->dev, "Failed to apply PSR %d\n", ret);
return ret;
diff --git a/drivers/gpu/drm/bridge/analogix/anx7625.c b/drivers/gpu/drm/bridge/analogix/anx7625.c
index 1a871f6..2346dbc 100644
--- a/drivers/gpu/drm/bridge/analogix/anx7625.c
+++ b/drivers/gpu/drm/bridge/analogix/anx7625.c
@@ -32,6 +32,8 @@
#include <drm/drm_print.h>
#include <drm/drm_probe_helper.h>
+#include <media/v4l2-fwnode.h>
+#include <sound/hdmi-codec.h>
#include <video/display_timing.h>
#include "anx7625.h"
@@ -166,6 +168,20 @@ static int anx7625_write_and_or(struct anx7625_data *ctx,
offset, (val & and_mask) | (or_mask));
}
+static int anx7625_config_bit_matrix(struct anx7625_data *ctx)
+{
+ int i, ret;
+
+ ret = anx7625_write_or(ctx, ctx->i2c.tx_p2_client,
+ AUDIO_CONTROL_REGISTER, 0x80);
+ for (i = 0; i < 13; i++)
+ ret |= anx7625_reg_write(ctx, ctx->i2c.tx_p2_client,
+ VIDEO_BIT_MATRIX_12 + i,
+ 0x18 + i);
+
+ return ret;
+}
+
static int anx7625_read_ctrl_status_p0(struct anx7625_data *ctx)
{
return anx7625_reg_read(ctx, ctx->i2c.rx_p0_client, AP_AUX_CTRL_STATUS);
@@ -191,10 +207,10 @@ static int wait_aux_op_finish(struct anx7625_data *ctx)
AP_AUX_CTRL_STATUS);
if (val < 0 || (val & 0x0F)) {
DRM_DEV_ERROR(dev, "aux status %02x\n", val);
- val = -EIO;
+ return -EIO;
}
- return val;
+ return 0;
}
static int anx7625_video_mute_control(struct anx7625_data *ctx,
@@ -221,38 +237,6 @@ static int anx7625_video_mute_control(struct anx7625_data *ctx,
return ret;
}
-static int anx7625_config_audio_input(struct anx7625_data *ctx)
-{
- struct device *dev = &ctx->client->dev;
- int ret;
-
- /* Channel num */
- ret = anx7625_reg_write(ctx, ctx->i2c.tx_p2_client,
- AUDIO_CHANNEL_STATUS_6, I2S_CH_2 << 5);
-
- /* FS */
- ret |= anx7625_write_and_or(ctx, ctx->i2c.tx_p2_client,
- AUDIO_CHANNEL_STATUS_4,
- 0xf0, AUDIO_FS_48K);
- /* Word length */
- ret |= anx7625_write_and_or(ctx, ctx->i2c.tx_p2_client,
- AUDIO_CHANNEL_STATUS_5,
- 0xf0, AUDIO_W_LEN_24_24MAX);
- /* I2S */
- ret |= anx7625_write_or(ctx, ctx->i2c.tx_p2_client,
- AUDIO_CHANNEL_STATUS_6, I2S_SLAVE_MODE);
- ret |= anx7625_write_and(ctx, ctx->i2c.tx_p2_client,
- AUDIO_CONTROL_REGISTER, ~TDM_TIMING_MODE);
- /* Audio change flag */
- ret |= anx7625_write_or(ctx, ctx->i2c.rx_p0_client,
- AP_AV_STATUS, AP_AUDIO_CHG);
-
- if (ret < 0)
- DRM_DEV_ERROR(dev, "fail to config audio.\n");
-
- return ret;
-}
-
/* Reduction of fraction a/b */
static void anx7625_reduction_of_a_fraction(unsigned long *a, unsigned long *b)
{
@@ -431,7 +415,7 @@ static int anx7625_dsi_video_timing_config(struct anx7625_data *ctx)
ret |= anx7625_write_and(ctx, ctx->i2c.rx_p1_client,
MIPI_LANE_CTRL_0, 0xfc);
ret |= anx7625_write_or(ctx, ctx->i2c.rx_p1_client,
- MIPI_LANE_CTRL_0, 3);
+ MIPI_LANE_CTRL_0, ctx->pdata.mipi_lanes - 1);
/* Htotal */
htotal = ctx->dt.hactive.min + ctx->dt.hfront_porch.min +
@@ -615,6 +599,76 @@ static int anx7625_dsi_config(struct anx7625_data *ctx)
return ret;
}
+static int anx7625_api_dpi_config(struct anx7625_data *ctx)
+{
+ struct device *dev = &ctx->client->dev;
+ u16 freq = ctx->dt.pixelclock.min / 1000;
+ int ret;
+
+ /* configure pixel clock */
+ ret = anx7625_reg_write(ctx, ctx->i2c.rx_p0_client,
+ PIXEL_CLOCK_L, freq & 0xFF);
+ ret |= anx7625_reg_write(ctx, ctx->i2c.rx_p0_client,
+ PIXEL_CLOCK_H, (freq >> 8));
+
+ /* set DPI mode */
+ /* set to DPI PLL module sel */
+ ret |= anx7625_reg_write(ctx, ctx->i2c.rx_p1_client,
+ MIPI_DIGITAL_PLL_9, 0x20);
+ /* power down MIPI */
+ ret |= anx7625_reg_write(ctx, ctx->i2c.rx_p1_client,
+ MIPI_LANE_CTRL_10, 0x08);
+ /* enable DPI mode */
+ ret |= anx7625_reg_write(ctx, ctx->i2c.rx_p1_client,
+ MIPI_DIGITAL_PLL_18, 0x1C);
+ /* set first edge */
+ ret |= anx7625_reg_write(ctx, ctx->i2c.tx_p2_client,
+ VIDEO_CONTROL_0, 0x06);
+ if (ret < 0)
+ DRM_DEV_ERROR(dev, "IO error : dpi phy set failed.\n");
+
+ return ret;
+}
+
+static int anx7625_dpi_config(struct anx7625_data *ctx)
+{
+ struct device *dev = &ctx->client->dev;
+ int ret;
+
+ DRM_DEV_DEBUG_DRIVER(dev, "config dpi\n");
+
+ /* DSC disable */
+ ret = anx7625_write_and(ctx, ctx->i2c.rx_p0_client,
+ R_DSC_CTRL_0, ~DSC_EN);
+ if (ret < 0) {
+ DRM_DEV_ERROR(dev, "IO error : disable dsc failed.\n");
+ return ret;
+ }
+
+ ret = anx7625_config_bit_matrix(ctx);
+ if (ret < 0) {
+ DRM_DEV_ERROR(dev, "config bit matrix failed.\n");
+ return ret;
+ }
+
+ ret = anx7625_api_dpi_config(ctx);
+ if (ret < 0) {
+ DRM_DEV_ERROR(dev, "mipi phy(dpi) setup failed.\n");
+ return ret;
+ }
+
+ /* set MIPI RX EN */
+ ret = anx7625_write_or(ctx, ctx->i2c.rx_p0_client,
+ AP_AV_STATUS, AP_MIPI_RX_EN);
+ /* clear mute flag */
+ ret |= anx7625_write_and(ctx, ctx->i2c.rx_p0_client,
+ AP_AV_STATUS, (u8)~AP_MIPI_MUTE);
+ if (ret < 0)
+ DRM_DEV_ERROR(dev, "IO error : enable mipi rx failed.\n");
+
+ return ret;
+}
+
static void anx7625_dp_start(struct anx7625_data *ctx)
{
int ret;
@@ -625,9 +679,10 @@ static void anx7625_dp_start(struct anx7625_data *ctx)
return;
}
- anx7625_config_audio_input(ctx);
-
- ret = anx7625_dsi_config(ctx);
+ if (ctx->pdata.is_dpi)
+ ret = anx7625_dpi_config(ctx);
+ else
+ ret = anx7625_dsi_config(ctx);
if (ret < 0)
DRM_DEV_ERROR(dev, "MIPI phy setup error.\n");
@@ -795,7 +850,7 @@ static int sp_tx_edid_read(struct anx7625_data *ctx,
int count, blocks_num;
u8 pblock_buf[MAX_DPCD_BUFFER_SIZE];
u8 i, j;
- u8 g_edid_break = 0;
+ int g_edid_break = 0;
int ret;
struct device *dev = &ctx->client->dev;
@@ -826,7 +881,7 @@ static int sp_tx_edid_read(struct anx7625_data *ctx,
g_edid_break = edid_read(ctx, offset,
pblock_buf);
- if (g_edid_break)
+ if (g_edid_break < 0)
break;
memcpy(&pedid_blocks_buf[offset],
@@ -1075,6 +1130,7 @@ static void anx7625_start_dp_work(struct anx7625_data *ctx)
return;
}
+ ctx->hpd_status = 1;
ctx->hpd_high_cnt++;
/* Not support HDCP */
@@ -1084,8 +1140,10 @@ static void anx7625_start_dp_work(struct anx7625_data *ctx)
ret |= anx7625_write_or(ctx, ctx->i2c.rx_p1_client, 0xec, 0x10);
/* Interrupt for DRM */
ret |= anx7625_write_or(ctx, ctx->i2c.rx_p1_client, 0xff, 0x01);
- if (ret < 0)
+ if (ret < 0) {
+ DRM_DEV_ERROR(dev, "fail to setting HDCP/auth\n");
return;
+ }
ret = anx7625_reg_read(ctx, ctx->i2c.rx_p1_client, 0x86);
if (ret < 0)
@@ -1104,6 +1162,10 @@ static void anx7625_hpd_polling(struct anx7625_data *ctx)
int ret, val;
struct device *dev = &ctx->client->dev;
+ /* Interrupt mode, no need poll HPD status, just return */
+ if (ctx->pdata.intp_irq)
+ return;
+
ret = readx_poll_timeout(anx7625_read_hpd_status_p0,
ctx, val,
((val & HPD_STATUS) || (val < 0)),
@@ -1131,6 +1193,21 @@ static void anx7625_remove_edid(struct anx7625_data *ctx)
ctx->slimport_edid_p.edid_block_num = -1;
}
+static void anx7625_dp_adjust_swing(struct anx7625_data *ctx)
+{
+ int i;
+
+ for (i = 0; i < ctx->pdata.dp_lane0_swing_reg_cnt; i++)
+ anx7625_reg_write(ctx, ctx->i2c.tx_p1_client,
+ DP_TX_LANE0_SWING_REG0 + i,
+ ctx->pdata.lane0_reg_data[i] & 0xFF);
+
+ for (i = 0; i < ctx->pdata.dp_lane1_swing_reg_cnt; i++)
+ anx7625_reg_write(ctx, ctx->i2c.tx_p1_client,
+ DP_TX_LANE1_SWING_REG0 + i,
+ ctx->pdata.lane1_reg_data[i] & 0xFF);
+}
+
static void dp_hpd_change_handler(struct anx7625_data *ctx, bool on)
{
struct device *dev = &ctx->client->dev;
@@ -1146,9 +1223,8 @@ static void dp_hpd_change_handler(struct anx7625_data *ctx, bool on)
} else {
DRM_DEV_DEBUG_DRIVER(dev, " HPD high\n");
anx7625_start_dp_work(ctx);
+ anx7625_dp_adjust_swing(ctx);
}
-
- ctx->hpd_status = 1;
}
static int anx7625_hpd_change_detect(struct anx7625_data *ctx)
@@ -1225,20 +1301,75 @@ static irqreturn_t anx7625_intr_hpd_isr(int irq, void *data)
return IRQ_HANDLED;
}
+static int anx7625_get_swing_setting(struct device *dev,
+ struct anx7625_platform_data *pdata)
+{
+ int num_regs;
+
+ if (of_get_property(dev->of_node,
+ "analogix,lane0-swing", &num_regs)) {
+ if (num_regs > DP_TX_SWING_REG_CNT)
+ num_regs = DP_TX_SWING_REG_CNT;
+
+ pdata->dp_lane0_swing_reg_cnt = num_regs;
+ of_property_read_u32_array(dev->of_node, "analogix,lane0-swing",
+ pdata->lane0_reg_data, num_regs);
+ }
+
+ if (of_get_property(dev->of_node,
+ "analogix,lane1-swing", &num_regs)) {
+ if (num_regs > DP_TX_SWING_REG_CNT)
+ num_regs = DP_TX_SWING_REG_CNT;
+
+ pdata->dp_lane1_swing_reg_cnt = num_regs;
+ of_property_read_u32_array(dev->of_node, "analogix,lane1-swing",
+ pdata->lane1_reg_data, num_regs);
+ }
+
+ return 0;
+}
+
static int anx7625_parse_dt(struct device *dev,
struct anx7625_platform_data *pdata)
{
- struct device_node *np = dev->of_node;
+ struct device_node *np = dev->of_node, *ep0;
struct drm_panel *panel;
int ret;
+ int bus_type, mipi_lanes;
+ anx7625_get_swing_setting(dev, pdata);
+
+ pdata->is_dpi = 1; /* default dpi mode */
pdata->mipi_host_node = of_graph_get_remote_node(np, 0, 0);
if (!pdata->mipi_host_node) {
DRM_DEV_ERROR(dev, "fail to get internal panel.\n");
return -ENODEV;
}
- DRM_DEV_DEBUG_DRIVER(dev, "found dsi host node.\n");
+ bus_type = V4L2_FWNODE_BUS_TYPE_PARALLEL;
+ mipi_lanes = MAX_LANES_SUPPORT;
+ ep0 = of_graph_get_endpoint_by_regs(np, 0, 0);
+ if (ep0) {
+ if (of_property_read_u32(ep0, "bus-type", &bus_type))
+ bus_type = 0;
+
+ mipi_lanes = of_property_count_u32_elems(ep0, "data-lanes");
+ }
+
+ if (bus_type == V4L2_FWNODE_BUS_TYPE_PARALLEL) /* bus type is Parallel(DSI) */
+ pdata->is_dpi = 0;
+
+ pdata->mipi_lanes = mipi_lanes;
+ if (pdata->mipi_lanes > MAX_LANES_SUPPORT || pdata->mipi_lanes <= 0)
+ pdata->mipi_lanes = MAX_LANES_SUPPORT;
+
+ if (pdata->is_dpi)
+ DRM_DEV_DEBUG_DRIVER(dev, "found MIPI DPI host node.\n");
+ else
+ DRM_DEV_DEBUG_DRIVER(dev, "found MIPI DSI host node.\n");
+
+ if (of_property_read_bool(np, "analogix,audio-enable"))
+ pdata->audio_en = 1;
ret = drm_of_find_panel_or_bridge(np, 1, 0, &panel, NULL);
if (ret < 0) {
@@ -1301,9 +1432,215 @@ static enum drm_connector_status anx7625_sink_detect(struct anx7625_data *ctx)
{
struct device *dev = &ctx->client->dev;
- DRM_DEV_DEBUG_DRIVER(dev, "sink detect, return connected\n");
+ DRM_DEV_DEBUG_DRIVER(dev, "sink detect\n");
- return connector_status_connected;
+ if (ctx->pdata.panel_bridge)
+ return connector_status_connected;
+
+ return ctx->hpd_status ? connector_status_connected :
+ connector_status_disconnected;
+}
+
+static int anx7625_audio_hw_params(struct device *dev, void *data,
+ struct hdmi_codec_daifmt *fmt,
+ struct hdmi_codec_params *params)
+{
+ struct anx7625_data *ctx = dev_get_drvdata(dev);
+ int wl, ch, rate;
+ int ret = 0;
+
+ if (fmt->fmt != HDMI_DSP_A) {
+ DRM_DEV_ERROR(dev, "only supports DSP_A\n");
+ return -EINVAL;
+ }
+
+ DRM_DEV_DEBUG_DRIVER(dev, "setting %d Hz, %d bit, %d channels\n",
+ params->sample_rate, params->sample_width,
+ params->cea.channels);
+
+ ret |= anx7625_write_and_or(ctx, ctx->i2c.tx_p2_client,
+ AUDIO_CHANNEL_STATUS_6,
+ ~I2S_SLAVE_MODE,
+ TDM_SLAVE_MODE);
+
+ /* Word length */
+ switch (params->sample_width) {
+ case 16:
+ wl = AUDIO_W_LEN_16_20MAX;
+ break;
+ case 18:
+ wl = AUDIO_W_LEN_18_20MAX;
+ break;
+ case 20:
+ wl = AUDIO_W_LEN_20_20MAX;
+ break;
+ case 24:
+ wl = AUDIO_W_LEN_24_24MAX;
+ break;
+ default:
+ DRM_DEV_DEBUG_DRIVER(dev, "wordlength: %d bit not support",
+ params->sample_width);
+ return -EINVAL;
+ }
+ ret |= anx7625_write_and_or(ctx, ctx->i2c.tx_p2_client,
+ AUDIO_CHANNEL_STATUS_5,
+ 0xf0, wl);
+
+ /* Channel num */
+ switch (params->cea.channels) {
+ case 2:
+ ch = I2S_CH_2;
+ break;
+ case 4:
+ ch = TDM_CH_4;
+ break;
+ case 6:
+ ch = TDM_CH_6;
+ break;
+ case 8:
+ ch = TDM_CH_8;
+ break;
+ default:
+ DRM_DEV_DEBUG_DRIVER(dev, "channel number: %d not support",
+ params->cea.channels);
+ return -EINVAL;
+ }
+ ret |= anx7625_write_and_or(ctx, ctx->i2c.tx_p2_client,
+ AUDIO_CHANNEL_STATUS_6, 0x1f, ch << 5);
+ if (ch > I2S_CH_2)
+ ret |= anx7625_write_or(ctx, ctx->i2c.tx_p2_client,
+ AUDIO_CHANNEL_STATUS_6, AUDIO_LAYOUT);
+ else
+ ret |= anx7625_write_and(ctx, ctx->i2c.tx_p2_client,
+ AUDIO_CHANNEL_STATUS_6, ~AUDIO_LAYOUT);
+
+ /* FS */
+ switch (params->sample_rate) {
+ case 32000:
+ rate = AUDIO_FS_32K;
+ break;
+ case 44100:
+ rate = AUDIO_FS_441K;
+ break;
+ case 48000:
+ rate = AUDIO_FS_48K;
+ break;
+ case 88200:
+ rate = AUDIO_FS_882K;
+ break;
+ case 96000:
+ rate = AUDIO_FS_96K;
+ break;
+ case 176400:
+ rate = AUDIO_FS_1764K;
+ break;
+ case 192000:
+ rate = AUDIO_FS_192K;
+ break;
+ default:
+ DRM_DEV_DEBUG_DRIVER(dev, "sample rate: %d not support",
+ params->sample_rate);
+ return -EINVAL;
+ }
+ ret |= anx7625_write_and_or(ctx, ctx->i2c.tx_p2_client,
+ AUDIO_CHANNEL_STATUS_4,
+ 0xf0, rate);
+ ret |= anx7625_write_or(ctx, ctx->i2c.rx_p0_client,
+ AP_AV_STATUS, AP_AUDIO_CHG);
+ if (ret < 0) {
+ DRM_DEV_ERROR(dev, "IO error : config audio.\n");
+ return -EIO;
+ }
+
+ return 0;
+}
+
+static void anx7625_audio_shutdown(struct device *dev, void *data)
+{
+ DRM_DEV_DEBUG_DRIVER(dev, "stop audio\n");
+}
+
+static int anx7625_hdmi_i2s_get_dai_id(struct snd_soc_component *component,
+ struct device_node *endpoint)
+{
+ struct of_endpoint of_ep;
+ int ret;
+
+ ret = of_graph_parse_endpoint(endpoint, &of_ep);
+ if (ret < 0)
+ return ret;
+
+ /*
+ * HDMI sound should be located at external DPI port
+ * Didn't have good way to check where is internal(DSI)
+ * or external(DPI) bridge
+ */
+ return 0;
+}
+
+static void
+anx7625_audio_update_connector_status(struct anx7625_data *ctx,
+ enum drm_connector_status status)
+{
+ if (ctx->plugged_cb && ctx->codec_dev) {
+ ctx->plugged_cb(ctx->codec_dev,
+ status == connector_status_connected);
+ }
+}
+
+static int anx7625_audio_hook_plugged_cb(struct device *dev, void *data,
+ hdmi_codec_plugged_cb fn,
+ struct device *codec_dev)
+{
+ struct anx7625_data *ctx = data;
+
+ ctx->plugged_cb = fn;
+ ctx->codec_dev = codec_dev;
+ anx7625_audio_update_connector_status(ctx, anx7625_sink_detect(ctx));
+
+ return 0;
+}
+
+static const struct hdmi_codec_ops anx7625_codec_ops = {
+ .hw_params = anx7625_audio_hw_params,
+ .audio_shutdown = anx7625_audio_shutdown,
+ .get_dai_id = anx7625_hdmi_i2s_get_dai_id,
+ .hook_plugged_cb = anx7625_audio_hook_plugged_cb,
+};
+
+static void anx7625_unregister_audio(struct anx7625_data *ctx)
+{
+ struct device *dev = &ctx->client->dev;
+
+ if (ctx->audio_pdev) {
+ platform_device_unregister(ctx->audio_pdev);
+ ctx->audio_pdev = NULL;
+ }
+
+ DRM_DEV_DEBUG_DRIVER(dev, "unbound to %s", HDMI_CODEC_DRV_NAME);
+}
+
+static int anx7625_register_audio(struct device *dev, struct anx7625_data *ctx)
+{
+ struct hdmi_codec_pdata codec_data = {
+ .ops = &anx7625_codec_ops,
+ .max_i2s_channels = 8,
+ .i2s = 1,
+ .data = ctx,
+ };
+
+ ctx->audio_pdev = platform_device_register_data(dev,
+ HDMI_CODEC_DRV_NAME,
+ PLATFORM_DEVID_AUTO,
+ &codec_data,
+ sizeof(codec_data));
+
+ if (IS_ERR(ctx->audio_pdev))
+ return PTR_ERR(ctx->audio_pdev);
+
+ DRM_DEV_DEBUG_DRIVER(dev, "bound to %s", HDMI_CODEC_DRV_NAME);
+
+ return 0;
}
static int anx7625_attach_dsi(struct anx7625_data *ctx)
@@ -1316,6 +1653,7 @@ static int anx7625_attach_dsi(struct anx7625_data *ctx)
.channel = 0,
.node = NULL,
};
+ int ret;
DRM_DEV_DEBUG_DRIVER(dev, "attach dsi\n");
@@ -1325,22 +1663,22 @@ static int anx7625_attach_dsi(struct anx7625_data *ctx)
return -EINVAL;
}
- dsi = mipi_dsi_device_register_full(host, &info);
+ dsi = devm_mipi_dsi_device_register_full(dev, host, &info);
if (IS_ERR(dsi)) {
DRM_DEV_ERROR(dev, "fail to create dsi device.\n");
return -EINVAL;
}
- dsi->lanes = 4;
+ dsi->lanes = ctx->pdata.mipi_lanes;
dsi->format = MIPI_DSI_FMT_RGB888;
dsi->mode_flags = MIPI_DSI_MODE_VIDEO |
MIPI_DSI_MODE_VIDEO_SYNC_PULSE |
MIPI_DSI_MODE_VIDEO_HSE;
- if (mipi_dsi_attach(dsi) < 0) {
+ ret = devm_mipi_dsi_attach(dev, dsi);
+ if (ret) {
DRM_DEV_ERROR(dev, "fail to attach dsi to host.\n");
- mipi_dsi_device_unregister(dsi);
- return -EINVAL;
+ return ret;
}
ctx->dsi = dsi;
@@ -1350,16 +1688,6 @@ static int anx7625_attach_dsi(struct anx7625_data *ctx)
return 0;
}
-static void anx7625_bridge_detach(struct drm_bridge *bridge)
-{
- struct anx7625_data *ctx = bridge_to_anx7625(bridge);
-
- if (ctx->dsi) {
- mipi_dsi_detach(ctx->dsi);
- mipi_dsi_device_unregister(ctx->dsi);
- }
-}
-
static int anx7625_bridge_attach(struct drm_bridge *bridge,
enum drm_bridge_attach_flags flags)
{
@@ -1376,12 +1704,6 @@ static int anx7625_bridge_attach(struct drm_bridge *bridge,
return -ENODEV;
}
- err = anx7625_attach_dsi(ctx);
- if (err) {
- DRM_DEV_ERROR(dev, "Fail to attach to dsi : %d\n", err);
- return err;
- }
-
if (ctx->pdata.panel_bridge) {
err = drm_bridge_attach(bridge->encoder,
ctx->pdata.panel_bridge,
@@ -1475,6 +1797,10 @@ static bool anx7625_bridge_mode_fixup(struct drm_bridge *bridge,
DRM_DEV_DEBUG_DRIVER(dev, "drm mode fixup set\n");
+ /* No need fixup for external monitor */
+ if (!ctx->pdata.panel_bridge)
+ return true;
+
hsync = mode->hsync_end - mode->hsync_start;
hfp = mode->hsync_start - mode->hdisplay;
hbp = mode->htotal - mode->hsync_end;
@@ -1624,7 +1950,6 @@ static struct edid *anx7625_bridge_get_edid(struct drm_bridge *bridge,
static const struct drm_bridge_funcs anx7625_bridge_funcs = {
.attach = anx7625_bridge_attach,
- .detach = anx7625_bridge_detach,
.disable = anx7625_bridge_disable,
.mode_valid = anx7625_bridge_mode_valid,
.mode_set = anx7625_bridge_mode_set,
@@ -1851,14 +2176,39 @@ static int anx7625_i2c_probe(struct i2c_client *client,
platform->bridge.funcs = &anx7625_bridge_funcs;
platform->bridge.of_node = client->dev.of_node;
- platform->bridge.ops = DRM_BRIDGE_OP_EDID | DRM_BRIDGE_OP_HPD;
- platform->bridge.type = DRM_MODE_CONNECTOR_eDP;
+ platform->bridge.ops = DRM_BRIDGE_OP_EDID;
+ if (!platform->pdata.panel_bridge)
+ platform->bridge.ops |= DRM_BRIDGE_OP_HPD |
+ DRM_BRIDGE_OP_DETECT;
+ platform->bridge.type = platform->pdata.panel_bridge ?
+ DRM_MODE_CONNECTOR_eDP :
+ DRM_MODE_CONNECTOR_DisplayPort;
+
drm_bridge_add(&platform->bridge);
+ if (!platform->pdata.is_dpi) {
+ ret = anx7625_attach_dsi(platform);
+ if (ret) {
+ DRM_DEV_ERROR(dev, "Fail to attach to dsi : %d\n", ret);
+ goto unregister_bridge;
+ }
+ }
+
+ if (platform->pdata.audio_en)
+ anx7625_register_audio(dev, platform);
+
DRM_DEV_DEBUG_DRIVER(dev, "probe done\n");
return 0;
+unregister_bridge:
+ drm_bridge_remove(&platform->bridge);
+
+ if (!platform->pdata.low_power_mode)
+ pm_runtime_put_sync_suspend(&client->dev);
+
+ anx7625_unregister_i2c_dummy_clients(platform);
+
free_wq:
if (platform->workqueue)
destroy_workqueue(platform->workqueue);
@@ -1883,6 +2233,9 @@ static int anx7625_i2c_remove(struct i2c_client *client)
anx7625_unregister_i2c_dummy_clients(platform);
+ if (platform->pdata.audio_en)
+ anx7625_unregister_audio(platform);
+
kfree(platform);
return 0;
}
diff --git a/drivers/gpu/drm/bridge/analogix/anx7625.h b/drivers/gpu/drm/bridge/analogix/anx7625.h
index 6dcf64c..3d79b6f 100644
--- a/drivers/gpu/drm/bridge/analogix/anx7625.h
+++ b/drivers/gpu/drm/bridge/analogix/anx7625.h
@@ -111,6 +111,7 @@
#define AUDIO_CHANNEL_STATUS_6 0xd5
#define TDM_SLAVE_MODE 0x10
#define I2S_SLAVE_MODE 0x08
+#define AUDIO_LAYOUT 0x01
#define AUDIO_CONTROL_REGISTER 0xe6
#define TDM_TIMING_MODE 0x08
@@ -141,12 +142,20 @@
#define HORIZONTAL_BACK_PORCH_H 0x22 /* Bit[7:4] are reserved */
/******** END of I2C Address 0x72 *********/
+
+/***************************************************************/
+/* Register definition of device address 0x7a */
+#define DP_TX_SWING_REG_CNT 0x14
+#define DP_TX_LANE0_SWING_REG0 0x00
+#define DP_TX_LANE1_SWING_REG0 0x14
+/******** END of I2C Address 0x7a *********/
+
/***************************************************************/
/* Register definition of device address 0x7e */
#define I2C_ADDR_7E_FLASH_CONTROLLER 0x7E
-#define FLASH_LOAD_STA 0x05
+#define FLASH_LOAD_STA 0x05
#define FLASH_LOAD_STA_CHK BIT(7)
#define XTAL_FRQ_SEL 0x3F
@@ -349,12 +358,21 @@ struct s_edid_data {
/***************** Display End *****************/
+#define MAX_LANES_SUPPORT 4
+
struct anx7625_platform_data {
struct gpio_desc *gpio_p_on;
struct gpio_desc *gpio_reset;
struct regulator_bulk_data supplies[3];
struct drm_bridge *panel_bridge;
int intp_irq;
+ int is_dpi;
+ int mipi_lanes;
+ int audio_en;
+ int dp_lane0_swing_reg_cnt;
+ int lane0_reg_data[DP_TX_SWING_REG_CNT];
+ int dp_lane1_swing_reg_cnt;
+ int lane1_reg_data[DP_TX_SWING_REG_CNT];
u32 low_power_mode;
struct device_node *mipi_host_node;
};
@@ -371,6 +389,7 @@ struct anx7625_i2c_client {
struct anx7625_data {
struct anx7625_platform_data pdata;
+ struct platform_device *audio_pdev;
int hpd_status;
int hpd_high_cnt;
/* Lock for work queue */
@@ -379,6 +398,8 @@ struct anx7625_data {
struct anx7625_i2c_client i2c;
struct i2c_client *last_client;
struct s_edid_data slimport_edid_p;
+ struct device *codec_dev;
+ hdmi_codec_plugged_cb plugged_cb;
struct work_struct work;
struct workqueue_struct *workqueue;
char edid_block;
diff --git a/drivers/gpu/drm/bridge/display-connector.c b/drivers/gpu/drm/bridge/display-connector.c
index 05eb759..d24f5b9 100644
--- a/drivers/gpu/drm/bridge/display-connector.c
+++ b/drivers/gpu/drm/bridge/display-connector.c
@@ -13,6 +13,7 @@
#include <linux/platform_device.h>
#include <linux/regulator/consumer.h>
+#include <drm/drm_atomic_helper.h>
#include <drm/drm_bridge.h>
#include <drm/drm_edid.h>
@@ -87,10 +88,95 @@ static struct edid *display_connector_get_edid(struct drm_bridge *bridge,
return drm_get_edid(connector, conn->bridge.ddc);
}
+/*
+ * Since this bridge is tied to the connector, it acts like a passthrough,
+ * so concerning the output bus formats, either pass the bus formats from the
+ * previous bridge or return fallback data like done in the bridge function:
+ * drm_atomic_bridge_chain_select_bus_fmts().
+ * This supports negotiation if the bridge chain has all bits in place.
+ */
+static u32 *display_connector_get_output_bus_fmts(struct drm_bridge *bridge,
+ struct drm_bridge_state *bridge_state,
+ struct drm_crtc_state *crtc_state,
+ struct drm_connector_state *conn_state,
+ unsigned int *num_output_fmts)
+{
+ struct drm_bridge *prev_bridge = drm_bridge_get_prev_bridge(bridge);
+ struct drm_bridge_state *prev_bridge_state;
+
+ if (!prev_bridge || !prev_bridge->funcs->atomic_get_output_bus_fmts) {
+ struct drm_connector *conn = conn_state->connector;
+ u32 *out_bus_fmts;
+
+ *num_output_fmts = 1;
+ out_bus_fmts = kmalloc(sizeof(*out_bus_fmts), GFP_KERNEL);
+ if (!out_bus_fmts)
+ return NULL;
+
+ if (conn->display_info.num_bus_formats &&
+ conn->display_info.bus_formats)
+ out_bus_fmts[0] = conn->display_info.bus_formats[0];
+ else
+ out_bus_fmts[0] = MEDIA_BUS_FMT_FIXED;
+
+ return out_bus_fmts;
+ }
+
+ prev_bridge_state = drm_atomic_get_new_bridge_state(crtc_state->state,
+ prev_bridge);
+
+ return prev_bridge->funcs->atomic_get_output_bus_fmts(prev_bridge, prev_bridge_state,
+ crtc_state, conn_state,
+ num_output_fmts);
+}
+
+/*
+ * Since this bridge is tied to the connector, it acts like a passthrough,
+ * so concerning the input bus formats, either pass the bus formats from the
+ * previous bridge or MEDIA_BUS_FMT_FIXED (like select_bus_fmt_recursive())
+ * when atomic_get_input_bus_fmts is not supported.
+ * This supports negotiation if the bridge chain has all bits in place.
+ */
+static u32 *display_connector_get_input_bus_fmts(struct drm_bridge *bridge,
+ struct drm_bridge_state *bridge_state,
+ struct drm_crtc_state *crtc_state,
+ struct drm_connector_state *conn_state,
+ u32 output_fmt,
+ unsigned int *num_input_fmts)
+{
+ struct drm_bridge *prev_bridge = drm_bridge_get_prev_bridge(bridge);
+ struct drm_bridge_state *prev_bridge_state;
+
+ if (!prev_bridge || !prev_bridge->funcs->atomic_get_input_bus_fmts) {
+ u32 *in_bus_fmts;
+
+ *num_input_fmts = 1;
+ in_bus_fmts = kmalloc(sizeof(*in_bus_fmts), GFP_KERNEL);
+ if (!in_bus_fmts)
+ return NULL;
+
+ in_bus_fmts[0] = MEDIA_BUS_FMT_FIXED;
+
+ return in_bus_fmts;
+ }
+
+ prev_bridge_state = drm_atomic_get_new_bridge_state(crtc_state->state,
+ prev_bridge);
+
+ return prev_bridge->funcs->atomic_get_input_bus_fmts(prev_bridge, prev_bridge_state,
+ crtc_state, conn_state, output_fmt,
+ num_input_fmts);
+}
+
static const struct drm_bridge_funcs display_connector_bridge_funcs = {
.attach = display_connector_attach,
.detect = display_connector_detect,
.get_edid = display_connector_get_edid,
+ .atomic_get_output_bus_fmts = display_connector_get_output_bus_fmts,
+ .atomic_get_input_bus_fmts = display_connector_get_input_bus_fmts,
+ .atomic_duplicate_state = drm_atomic_helper_bridge_duplicate_state,
+ .atomic_destroy_state = drm_atomic_helper_bridge_destroy_state,
+ .atomic_reset = drm_atomic_helper_bridge_reset,
};
static irqreturn_t display_connector_hpd_irq(int irq, void *arg)
@@ -107,7 +193,7 @@ static int display_connector_probe(struct platform_device *pdev)
{
struct display_connector *conn;
unsigned int type;
- const char *label;
+ const char *label = NULL;
int ret;
conn = devm_kzalloc(&pdev->dev, sizeof(*conn), GFP_KERNEL);
diff --git a/drivers/gpu/drm/bridge/lontium-lt8912b.c b/drivers/gpu/drm/bridge/lontium-lt8912b.c
index 1b0c7ea..c642d1e 100644
--- a/drivers/gpu/drm/bridge/lontium-lt8912b.c
+++ b/drivers/gpu/drm/bridge/lontium-lt8912b.c
@@ -472,11 +472,11 @@ static int lt8912_attach_dsi(struct lt8912 *lt)
return -EPROBE_DEFER;
}
- dsi = mipi_dsi_device_register_full(host, &info);
+ dsi = devm_mipi_dsi_device_register_full(dev, host, &info);
if (IS_ERR(dsi)) {
ret = PTR_ERR(dsi);
dev_err(dev, "failed to create dsi device (%d)\n", ret);
- goto err_dsi_device;
+ return ret;
}
lt->dsi = dsi;
@@ -489,24 +489,13 @@ static int lt8912_attach_dsi(struct lt8912 *lt)
MIPI_DSI_MODE_LPM |
MIPI_DSI_MODE_NO_EOT_PACKET;
- ret = mipi_dsi_attach(dsi);
+ ret = devm_mipi_dsi_attach(dev, dsi);
if (ret < 0) {
dev_err(dev, "failed to attach dsi to host\n");
- goto err_dsi_attach;
+ return ret;
}
return 0;
-
-err_dsi_attach:
- mipi_dsi_device_unregister(dsi);
-err_dsi_device:
- return ret;
-}
-
-static void lt8912_detach_dsi(struct lt8912 *lt)
-{
- mipi_dsi_detach(lt->dsi);
- mipi_dsi_device_unregister(lt->dsi);
}
static int lt8912_bridge_connector_init(struct drm_bridge *bridge)
@@ -555,10 +544,6 @@ static int lt8912_bridge_attach(struct drm_bridge *bridge,
if (ret)
goto error;
- ret = lt8912_attach_dsi(lt);
- if (ret)
- goto error;
-
lt->is_attached = true;
return 0;
@@ -573,7 +558,6 @@ static void lt8912_bridge_detach(struct drm_bridge *bridge)
struct lt8912 *lt = bridge_to_lt8912(bridge);
if (lt->is_attached) {
- lt8912_detach_dsi(lt);
lt8912_hard_power_off(lt);
drm_connector_unregister(<->connector);
drm_connector_cleanup(<->connector);
@@ -718,8 +702,15 @@ static int lt8912_probe(struct i2c_client *client,
drm_bridge_add(<->bridge);
+ ret = lt8912_attach_dsi(lt);
+ if (ret)
+ goto err_attach;
+
return 0;
+err_attach:
+ drm_bridge_remove(<->bridge);
+ lt8912_free_i2c(lt);
err_i2c:
lt8912_put_dt(lt);
err_dt_parse:
diff --git a/drivers/gpu/drm/bridge/lontium-lt9611.c b/drivers/gpu/drm/bridge/lontium-lt9611.c
index 29b1ce2..dafb1b4 100644
--- a/drivers/gpu/drm/bridge/lontium-lt9611.c
+++ b/drivers/gpu/drm/bridge/lontium-lt9611.c
@@ -586,7 +586,7 @@ lt9611_connector_detect(struct drm_connector *connector, bool force)
int connected = 0;
regmap_read(lt9611->regmap, 0x825e, ®_val);
- connected = (reg_val & BIT(2));
+ connected = (reg_val & BIT(0));
lt9611->status = connected ? connector_status_connected :
connector_status_disconnected;
@@ -760,6 +760,7 @@ static struct mipi_dsi_device *lt9611_attach_dsi(struct lt9611 *lt9611,
const struct mipi_dsi_device_info info = { "lt9611", 0, NULL };
struct mipi_dsi_device *dsi;
struct mipi_dsi_host *host;
+ struct device *dev = lt9611->dev;
int ret;
host = of_find_mipi_dsi_host_by_node(dsi_node);
@@ -768,7 +769,7 @@ static struct mipi_dsi_device *lt9611_attach_dsi(struct lt9611 *lt9611,
return ERR_PTR(-EPROBE_DEFER);
}
- dsi = mipi_dsi_device_register_full(host, &info);
+ dsi = devm_mipi_dsi_device_register_full(dev, host, &info);
if (IS_ERR(dsi)) {
dev_err(lt9611->dev, "failed to create dsi device\n");
return dsi;
@@ -779,29 +780,15 @@ static struct mipi_dsi_device *lt9611_attach_dsi(struct lt9611 *lt9611,
dsi->mode_flags = MIPI_DSI_MODE_VIDEO | MIPI_DSI_MODE_VIDEO_SYNC_PULSE |
MIPI_DSI_MODE_VIDEO_HSE;
- ret = mipi_dsi_attach(dsi);
+ ret = devm_mipi_dsi_attach(dev, dsi);
if (ret < 0) {
- dev_err(lt9611->dev, "failed to attach dsi to host\n");
- mipi_dsi_device_unregister(dsi);
+ dev_err(dev, "failed to attach dsi to host\n");
return ERR_PTR(ret);
}
return dsi;
}
-static void lt9611_bridge_detach(struct drm_bridge *bridge)
-{
- struct lt9611 *lt9611 = bridge_to_lt9611(bridge);
-
- if (lt9611->dsi1) {
- mipi_dsi_detach(lt9611->dsi1);
- mipi_dsi_device_unregister(lt9611->dsi1);
- }
-
- mipi_dsi_detach(lt9611->dsi0);
- mipi_dsi_device_unregister(lt9611->dsi0);
-}
-
static int lt9611_connector_init(struct drm_bridge *bridge, struct lt9611 *lt9611)
{
int ret;
@@ -838,28 +825,7 @@ static int lt9611_bridge_attach(struct drm_bridge *bridge,
return ret;
}
- /* Attach primary DSI */
- lt9611->dsi0 = lt9611_attach_dsi(lt9611, lt9611->dsi0_node);
- if (IS_ERR(lt9611->dsi0))
- return PTR_ERR(lt9611->dsi0);
-
- /* Attach secondary DSI, if specified */
- if (lt9611->dsi1_node) {
- lt9611->dsi1 = lt9611_attach_dsi(lt9611, lt9611->dsi1_node);
- if (IS_ERR(lt9611->dsi1)) {
- ret = PTR_ERR(lt9611->dsi1);
- goto err_unregister_dsi0;
- }
- }
-
return 0;
-
-err_unregister_dsi0:
- lt9611_bridge_detach(bridge);
- drm_connector_cleanup(<9611->connector);
- mipi_dsi_device_unregister(lt9611->dsi0);
-
- return ret;
}
static enum drm_mode_status lt9611_bridge_mode_valid(struct drm_bridge *bridge,
@@ -926,7 +892,7 @@ static enum drm_connector_status lt9611_bridge_detect(struct drm_bridge *bridge)
int connected;
regmap_read(lt9611->regmap, 0x825e, ®_val);
- connected = reg_val & BIT(2);
+ connected = reg_val & BIT(0);
lt9611->status = connected ? connector_status_connected :
connector_status_disconnected;
@@ -952,7 +918,6 @@ static void lt9611_bridge_hpd_enable(struct drm_bridge *bridge)
static const struct drm_bridge_funcs lt9611_bridge_funcs = {
.attach = lt9611_bridge_attach,
- .detach = lt9611_bridge_detach,
.mode_valid = lt9611_bridge_mode_valid,
.enable = lt9611_bridge_enable,
.disable = lt9611_bridge_disable,
@@ -1181,10 +1146,29 @@ static int lt9611_probe(struct i2c_client *client,
drm_bridge_add(<9611->bridge);
+ /* Attach primary DSI */
+ lt9611->dsi0 = lt9611_attach_dsi(lt9611, lt9611->dsi0_node);
+ if (IS_ERR(lt9611->dsi0)) {
+ ret = PTR_ERR(lt9611->dsi0);
+ goto err_remove_bridge;
+ }
+
+ /* Attach secondary DSI, if specified */
+ if (lt9611->dsi1_node) {
+ lt9611->dsi1 = lt9611_attach_dsi(lt9611, lt9611->dsi1_node);
+ if (IS_ERR(lt9611->dsi1)) {
+ ret = PTR_ERR(lt9611->dsi1);
+ goto err_remove_bridge;
+ }
+ }
+
lt9611_enable_hpd_interrupts(lt9611);
return lt9611_audio_init(dev, lt9611);
+err_remove_bridge:
+ drm_bridge_remove(<9611->bridge);
+
err_disable_regulators:
regulator_bulk_disable(ARRAY_SIZE(lt9611->supplies), lt9611->supplies);
diff --git a/drivers/gpu/drm/bridge/lontium-lt9611uxc.c b/drivers/gpu/drm/bridge/lontium-lt9611uxc.c
index 010657e..33f9716 100644
--- a/drivers/gpu/drm/bridge/lontium-lt9611uxc.c
+++ b/drivers/gpu/drm/bridge/lontium-lt9611uxc.c
@@ -258,17 +258,18 @@ static struct mipi_dsi_device *lt9611uxc_attach_dsi(struct lt9611uxc *lt9611uxc,
const struct mipi_dsi_device_info info = { "lt9611uxc", 0, NULL };
struct mipi_dsi_device *dsi;
struct mipi_dsi_host *host;
+ struct device *dev = lt9611uxc->dev;
int ret;
host = of_find_mipi_dsi_host_by_node(dsi_node);
if (!host) {
- dev_err(lt9611uxc->dev, "failed to find dsi host\n");
+ dev_err(dev, "failed to find dsi host\n");
return ERR_PTR(-EPROBE_DEFER);
}
- dsi = mipi_dsi_device_register_full(host, &info);
+ dsi = devm_mipi_dsi_device_register_full(dev, host, &info);
if (IS_ERR(dsi)) {
- dev_err(lt9611uxc->dev, "failed to create dsi device\n");
+ dev_err(dev, "failed to create dsi device\n");
return dsi;
}
@@ -277,10 +278,9 @@ static struct mipi_dsi_device *lt9611uxc_attach_dsi(struct lt9611uxc *lt9611uxc,
dsi->mode_flags = MIPI_DSI_MODE_VIDEO | MIPI_DSI_MODE_VIDEO_SYNC_PULSE |
MIPI_DSI_MODE_VIDEO_HSE;
- ret = mipi_dsi_attach(dsi);
+ ret = devm_mipi_dsi_attach(dev, dsi);
if (ret < 0) {
- dev_err(lt9611uxc->dev, "failed to attach dsi to host\n");
- mipi_dsi_device_unregister(dsi);
+ dev_err(dev, "failed to attach dsi to host\n");
return ERR_PTR(ret);
}
@@ -355,19 +355,6 @@ static int lt9611uxc_connector_init(struct drm_bridge *bridge, struct lt9611uxc
return drm_connector_attach_encoder(<9611uxc->connector, bridge->encoder);
}
-static void lt9611uxc_bridge_detach(struct drm_bridge *bridge)
-{
- struct lt9611uxc *lt9611uxc = bridge_to_lt9611uxc(bridge);
-
- if (lt9611uxc->dsi1) {
- mipi_dsi_detach(lt9611uxc->dsi1);
- mipi_dsi_device_unregister(lt9611uxc->dsi1);
- }
-
- mipi_dsi_detach(lt9611uxc->dsi0);
- mipi_dsi_device_unregister(lt9611uxc->dsi0);
-}
-
static int lt9611uxc_bridge_attach(struct drm_bridge *bridge,
enum drm_bridge_attach_flags flags)
{
@@ -380,27 +367,7 @@ static int lt9611uxc_bridge_attach(struct drm_bridge *bridge,
return ret;
}
- /* Attach primary DSI */
- lt9611uxc->dsi0 = lt9611uxc_attach_dsi(lt9611uxc, lt9611uxc->dsi0_node);
- if (IS_ERR(lt9611uxc->dsi0))
- return PTR_ERR(lt9611uxc->dsi0);
-
- /* Attach secondary DSI, if specified */
- if (lt9611uxc->dsi1_node) {
- lt9611uxc->dsi1 = lt9611uxc_attach_dsi(lt9611uxc, lt9611uxc->dsi1_node);
- if (IS_ERR(lt9611uxc->dsi1)) {
- ret = PTR_ERR(lt9611uxc->dsi1);
- goto err_unregister_dsi0;
- }
- }
-
return 0;
-
-err_unregister_dsi0:
- mipi_dsi_detach(lt9611uxc->dsi0);
- mipi_dsi_device_unregister(lt9611uxc->dsi0);
-
- return ret;
}
static enum drm_mode_status
@@ -544,7 +511,6 @@ static struct edid *lt9611uxc_bridge_get_edid(struct drm_bridge *bridge,
static const struct drm_bridge_funcs lt9611uxc_bridge_funcs = {
.attach = lt9611uxc_bridge_attach,
- .detach = lt9611uxc_bridge_detach,
.mode_valid = lt9611uxc_bridge_mode_valid,
.mode_set = lt9611uxc_bridge_mode_set,
.detect = lt9611uxc_bridge_detect,
@@ -980,8 +946,27 @@ static int lt9611uxc_probe(struct i2c_client *client,
drm_bridge_add(<9611uxc->bridge);
+ /* Attach primary DSI */
+ lt9611uxc->dsi0 = lt9611uxc_attach_dsi(lt9611uxc, lt9611uxc->dsi0_node);
+ if (IS_ERR(lt9611uxc->dsi0)) {
+ ret = PTR_ERR(lt9611uxc->dsi0);
+ goto err_remove_bridge;
+ }
+
+ /* Attach secondary DSI, if specified */
+ if (lt9611uxc->dsi1_node) {
+ lt9611uxc->dsi1 = lt9611uxc_attach_dsi(lt9611uxc, lt9611uxc->dsi1_node);
+ if (IS_ERR(lt9611uxc->dsi1)) {
+ ret = PTR_ERR(lt9611uxc->dsi1);
+ goto err_remove_bridge;
+ }
+ }
+
return lt9611uxc_audio_init(dev, lt9611uxc);
+err_remove_bridge:
+ drm_bridge_remove(<9611uxc->bridge);
+
err_disable_regulators:
regulator_bulk_disable(ARRAY_SIZE(lt9611uxc->supplies), lt9611uxc->supplies);
diff --git a/drivers/gpu/drm/bridge/lvds-codec.c b/drivers/gpu/drm/bridge/lvds-codec.c
index ad460b9..702ea80 100644
--- a/drivers/gpu/drm/bridge/lvds-codec.c
+++ b/drivers/gpu/drm/bridge/lvds-codec.c
@@ -14,12 +14,14 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_bridge.h>
+#include <drm/drm_of.h>
#include <drm/drm_panel.h>
struct lvds_codec {
struct device *dev;
struct drm_bridge bridge;
struct drm_bridge *panel_bridge;
+ struct drm_bridge_timings timings;
struct regulator *vcc;
struct gpio_desc *powerdown_gpio;
u32 connector_type;
@@ -118,7 +120,7 @@ static int lvds_codec_probe(struct platform_device *pdev)
struct device_node *bus_node;
struct drm_panel *panel;
struct lvds_codec *lvds_codec;
- const char *mapping;
+ u32 val;
int ret;
lvds_codec = devm_kzalloc(dev, sizeof(*lvds_codec), GFP_KERNEL);
@@ -174,32 +176,38 @@ static int lvds_codec_probe(struct platform_device *pdev)
return -ENXIO;
}
- ret = of_property_read_string(bus_node, "data-mapping",
- &mapping);
+ ret = drm_of_lvds_get_data_mapping(bus_node);
of_node_put(bus_node);
- if (ret < 0) {
+ if (ret == -ENODEV) {
dev_warn(dev, "missing 'data-mapping' DT property\n");
+ } else if (ret) {
+ dev_err(dev, "invalid 'data-mapping' DT property\n");
+ return ret;
} else {
- if (!strcmp(mapping, "jeida-18")) {
- lvds_codec->bus_format = MEDIA_BUS_FMT_RGB666_1X7X3_SPWG;
- } else if (!strcmp(mapping, "jeida-24")) {
- lvds_codec->bus_format = MEDIA_BUS_FMT_RGB888_1X7X4_JEIDA;
- } else if (!strcmp(mapping, "vesa-24")) {
- lvds_codec->bus_format = MEDIA_BUS_FMT_RGB888_1X7X4_SPWG;
- } else {
- dev_err(dev, "invalid 'data-mapping' DT property\n");
- return -EINVAL;
- }
+ lvds_codec->bus_format = ret;
lvds_codec->bridge.funcs = &funcs_decoder;
}
}
/*
+ * Encoder might sample data on different clock edge than the display,
+ * for example OnSemi FIN3385 has a dedicated strapping pin to select
+ * the sampling edge.
+ */
+ if (lvds_codec->connector_type == DRM_MODE_CONNECTOR_LVDS &&
+ !of_property_read_u32(dev->of_node, "pclk-sample", &val)) {
+ lvds_codec->timings.input_bus_flags = val ?
+ DRM_BUS_FLAG_PIXDATA_SAMPLE_POSEDGE :
+ DRM_BUS_FLAG_PIXDATA_SAMPLE_NEGEDGE;
+ }
+
+ /*
* The panel_bridge bridge is attached to the panel's of_node,
* but we need a bridge attached to our of_node for our user
* to look up.
*/
lvds_codec->bridge.of_node = dev->of_node;
+ lvds_codec->bridge.timings = &lvds_codec->timings;
drm_bridge_add(&lvds_codec->bridge);
platform_set_drvdata(pdev, lvds_codec);
diff --git a/drivers/gpu/drm/bridge/megachips-stdpxxxx-ge-b850v3-fw.c b/drivers/gpu/drm/bridge/megachips-stdpxxxx-ge-b850v3-fw.c
index d2808c4..cce98bf 100644
--- a/drivers/gpu/drm/bridge/megachips-stdpxxxx-ge-b850v3-fw.c
+++ b/drivers/gpu/drm/bridge/megachips-stdpxxxx-ge-b850v3-fw.c
@@ -306,19 +306,10 @@ static void ge_b850v3_lvds_remove(void)
mutex_unlock(&ge_b850v3_lvds_dev_mutex);
}
-static int stdp4028_ge_b850v3_fw_probe(struct i2c_client *stdp4028_i2c,
- const struct i2c_device_id *id)
+static int ge_b850v3_register(void)
{
+ struct i2c_client *stdp4028_i2c = ge_b850v3_lvds_ptr->stdp4028_i2c;
struct device *dev = &stdp4028_i2c->dev;
- int ret;
-
- ret = ge_b850v3_lvds_init(dev);
-
- if (ret)
- return ret;
-
- ge_b850v3_lvds_ptr->stdp4028_i2c = stdp4028_i2c;
- i2c_set_clientdata(stdp4028_i2c, ge_b850v3_lvds_ptr);
/* drm bridge initialization */
ge_b850v3_lvds_ptr->bridge.funcs = &ge_b850v3_lvds_funcs;
@@ -343,6 +334,27 @@ static int stdp4028_ge_b850v3_fw_probe(struct i2c_client *stdp4028_i2c,
"ge-b850v3-lvds-dp", ge_b850v3_lvds_ptr);
}
+static int stdp4028_ge_b850v3_fw_probe(struct i2c_client *stdp4028_i2c,
+ const struct i2c_device_id *id)
+{
+ struct device *dev = &stdp4028_i2c->dev;
+ int ret;
+
+ ret = ge_b850v3_lvds_init(dev);
+
+ if (ret)
+ return ret;
+
+ ge_b850v3_lvds_ptr->stdp4028_i2c = stdp4028_i2c;
+ i2c_set_clientdata(stdp4028_i2c, ge_b850v3_lvds_ptr);
+
+ /* Only register after both bridges are probed */
+ if (!ge_b850v3_lvds_ptr->stdp2690_i2c)
+ return 0;
+
+ return ge_b850v3_register();
+}
+
static int stdp4028_ge_b850v3_fw_remove(struct i2c_client *stdp4028_i2c)
{
ge_b850v3_lvds_remove();
@@ -386,7 +398,11 @@ static int stdp2690_ge_b850v3_fw_probe(struct i2c_client *stdp2690_i2c,
ge_b850v3_lvds_ptr->stdp2690_i2c = stdp2690_i2c;
i2c_set_clientdata(stdp2690_i2c, ge_b850v3_lvds_ptr);
- return 0;
+ /* Only register after both bridges are probed */
+ if (!ge_b850v3_lvds_ptr->stdp4028_i2c)
+ return 0;
+
+ return ge_b850v3_register();
}
static int stdp2690_ge_b850v3_fw_remove(struct i2c_client *stdp2690_i2c)
diff --git a/drivers/gpu/drm/bridge/parade-ps8640.c b/drivers/gpu/drm/bridge/parade-ps8640.c
index 3aaa909..818704b 100644
--- a/drivers/gpu/drm/bridge/parade-ps8640.c
+++ b/drivers/gpu/drm/bridge/parade-ps8640.c
@@ -9,10 +9,12 @@
#include <linux/i2c.h>
#include <linux/module.h>
#include <linux/of_graph.h>
+#include <linux/pm_runtime.h>
#include <linux/regmap.h>
#include <linux/regulator/consumer.h>
#include <drm/drm_bridge.h>
+#include <drm/drm_dp_aux_bus.h>
#include <drm/drm_dp_helper.h>
#include <drm/drm_mipi_dsi.h>
#include <drm/drm_of.h>
@@ -100,7 +102,7 @@ struct ps8640 {
struct regulator_bulk_data supplies[2];
struct gpio_desc *gpio_reset;
struct gpio_desc *gpio_powerdown;
- bool powered;
+ bool pre_enabled;
};
static const struct regmap_config ps8640_regmap_config[] = {
@@ -148,8 +150,46 @@ static inline struct ps8640 *aux_to_ps8640(struct drm_dp_aux *aux)
return container_of(aux, struct ps8640, aux);
}
-static ssize_t ps8640_aux_transfer(struct drm_dp_aux *aux,
- struct drm_dp_aux_msg *msg)
+static bool ps8640_of_panel_on_aux_bus(struct device *dev)
+{
+ struct device_node *bus, *panel;
+
+ bus = of_get_child_by_name(dev->of_node, "aux-bus");
+ if (!bus)
+ return false;
+
+ panel = of_get_child_by_name(bus, "panel");
+ of_node_put(bus);
+ if (!panel)
+ return false;
+ of_node_put(panel);
+
+ return true;
+}
+
+static int ps8640_ensure_hpd(struct ps8640 *ps_bridge)
+{
+ struct regmap *map = ps_bridge->regmap[PAGE2_TOP_CNTL];
+ struct device *dev = &ps_bridge->page[PAGE2_TOP_CNTL]->dev;
+ int status;
+ int ret;
+
+ /*
+ * Apparently something about the firmware in the chip signals that
+ * HPD goes high by reporting GPIO9 as high (even though HPD isn't
+ * actually connected to GPIO9).
+ */
+ ret = regmap_read_poll_timeout(map, PAGE2_GPIO_H, status,
+ status & PS_GPIO9, 20 * 1000, 200 * 1000);
+
+ if (ret < 0)
+ dev_warn(dev, "HPD didn't go high: %d\n", ret);
+
+ return ret;
+}
+
+static ssize_t ps8640_aux_transfer_msg(struct drm_dp_aux *aux,
+ struct drm_dp_aux_msg *msg)
{
struct ps8640 *ps_bridge = aux_to_ps8640(aux);
struct regmap *map = ps_bridge->regmap[PAGE0_DP_CNTL];
@@ -274,38 +314,49 @@ static ssize_t ps8640_aux_transfer(struct drm_dp_aux *aux,
return len;
}
-static int ps8640_bridge_vdo_control(struct ps8640 *ps_bridge,
- const enum ps8640_vdo_control ctrl)
+static ssize_t ps8640_aux_transfer(struct drm_dp_aux *aux,
+ struct drm_dp_aux_msg *msg)
+{
+ struct ps8640 *ps_bridge = aux_to_ps8640(aux);
+ struct device *dev = &ps_bridge->page[PAGE0_DP_CNTL]->dev;
+ int ret;
+
+ pm_runtime_get_sync(dev);
+ ret = ps8640_ensure_hpd(ps_bridge);
+ if (!ret)
+ ret = ps8640_aux_transfer_msg(aux, msg);
+ pm_runtime_mark_last_busy(dev);
+ pm_runtime_put_autosuspend(dev);
+
+ return ret;
+}
+
+static void ps8640_bridge_vdo_control(struct ps8640 *ps_bridge,
+ const enum ps8640_vdo_control ctrl)
{
struct regmap *map = ps_bridge->regmap[PAGE3_DSI_CNTL1];
+ struct device *dev = &ps_bridge->page[PAGE3_DSI_CNTL1]->dev;
u8 vdo_ctrl_buf[] = { VDO_CTL_ADD, ctrl };
int ret;
ret = regmap_bulk_write(map, PAGE3_SET_ADD,
vdo_ctrl_buf, sizeof(vdo_ctrl_buf));
- if (ret < 0) {
- DRM_ERROR("failed to %sable VDO: %d\n",
- ctrl == ENABLE ? "en" : "dis", ret);
- return ret;
- }
-
- return 0;
+ if (ret < 0)
+ dev_err(dev, "failed to %sable VDO: %d\n",
+ ctrl == ENABLE ? "en" : "dis", ret);
}
-static void ps8640_bridge_poweron(struct ps8640 *ps_bridge)
+static int __maybe_unused ps8640_resume(struct device *dev)
{
- struct regmap *map = ps_bridge->regmap[PAGE2_TOP_CNTL];
- int ret, status;
-
- if (ps_bridge->powered)
- return;
+ struct ps8640 *ps_bridge = dev_get_drvdata(dev);
+ int ret;
ret = regulator_bulk_enable(ARRAY_SIZE(ps_bridge->supplies),
ps_bridge->supplies);
if (ret < 0) {
- DRM_ERROR("cannot enable regulators %d\n", ret);
- return;
+ dev_err(dev, "cannot enable regulators %d\n", ret);
+ return ret;
}
gpiod_set_value(ps_bridge->gpio_powerdown, 0);
@@ -314,20 +365,47 @@ static void ps8640_bridge_poweron(struct ps8640 *ps_bridge)
gpiod_set_value(ps_bridge->gpio_reset, 0);
/*
- * Wait for the ps8640 embedded MCU to be ready
- * First wait 200ms and then check the MCU ready flag every 20ms
+ * Mystery 200 ms delay for the "MCU to be ready". It's unclear if
+ * this is truly necessary since the MCU will already signal that
+ * things are "good to go" by signaling HPD on "gpio 9". See
+ * ps8640_ensure_hpd(). For now we'll keep this mystery delay just in
+ * case.
*/
msleep(200);
- ret = regmap_read_poll_timeout(map, PAGE2_GPIO_H, status,
- status & PS_GPIO9, 20 * 1000, 200 * 1000);
+ return 0;
+}
- if (ret < 0) {
- DRM_ERROR("failed read PAGE2_GPIO_H: %d\n", ret);
- goto err_regulators_disable;
- }
+static int __maybe_unused ps8640_suspend(struct device *dev)
+{
+ struct ps8640 *ps_bridge = dev_get_drvdata(dev);
+ int ret;
- msleep(50);
+ gpiod_set_value(ps_bridge->gpio_reset, 1);
+ gpiod_set_value(ps_bridge->gpio_powerdown, 1);
+ ret = regulator_bulk_disable(ARRAY_SIZE(ps_bridge->supplies),
+ ps_bridge->supplies);
+ if (ret < 0)
+ dev_err(dev, "cannot disable regulators %d\n", ret);
+
+ return ret;
+}
+
+static const struct dev_pm_ops ps8640_pm_ops = {
+ SET_RUNTIME_PM_OPS(ps8640_suspend, ps8640_resume, NULL)
+ SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend,
+ pm_runtime_force_resume)
+};
+
+static void ps8640_pre_enable(struct drm_bridge *bridge)
+{
+ struct ps8640 *ps_bridge = bridge_to_ps8640(bridge);
+ struct regmap *map = ps_bridge->regmap[PAGE2_TOP_CNTL];
+ struct device *dev = &ps_bridge->page[PAGE0_DP_CNTL]->dev;
+ int ret;
+
+ pm_runtime_get_sync(dev);
+ ps8640_ensure_hpd(ps_bridge);
/*
* The Manufacturer Command Set (MCS) is a device dependent interface
@@ -338,62 +416,27 @@ static void ps8640_bridge_poweron(struct ps8640 *ps_bridge)
*/
ret = regmap_update_bits(map, PAGE2_MCS_EN, MCS_EN, 0);
- if (ret < 0) {
- DRM_ERROR("failed write PAGE2_MCS_EN: %d\n", ret);
- goto err_regulators_disable;
- }
+ if (ret < 0)
+ dev_warn(dev, "failed write PAGE2_MCS_EN: %d\n", ret);
/* Switch access edp panel's edid through i2c */
ret = regmap_write(map, PAGE2_I2C_BYPASS, I2C_BYPASS_EN);
- if (ret < 0) {
- DRM_ERROR("failed write PAGE2_I2C_BYPASS: %d\n", ret);
- goto err_regulators_disable;
- }
-
- ps_bridge->powered = true;
-
- return;
-
-err_regulators_disable:
- regulator_bulk_disable(ARRAY_SIZE(ps_bridge->supplies),
- ps_bridge->supplies);
-}
-
-static void ps8640_bridge_poweroff(struct ps8640 *ps_bridge)
-{
- int ret;
-
- if (!ps_bridge->powered)
- return;
-
- gpiod_set_value(ps_bridge->gpio_reset, 1);
- gpiod_set_value(ps_bridge->gpio_powerdown, 1);
- ret = regulator_bulk_disable(ARRAY_SIZE(ps_bridge->supplies),
- ps_bridge->supplies);
if (ret < 0)
- DRM_ERROR("cannot disable regulators %d\n", ret);
+ dev_warn(dev, "failed write PAGE2_MCS_EN: %d\n", ret);
- ps_bridge->powered = false;
-}
+ ps8640_bridge_vdo_control(ps_bridge, ENABLE);
-static void ps8640_pre_enable(struct drm_bridge *bridge)
-{
- struct ps8640 *ps_bridge = bridge_to_ps8640(bridge);
- int ret;
-
- ps8640_bridge_poweron(ps_bridge);
-
- ret = ps8640_bridge_vdo_control(ps_bridge, ENABLE);
- if (ret < 0)
- ps8640_bridge_poweroff(ps_bridge);
+ ps_bridge->pre_enabled = true;
}
static void ps8640_post_disable(struct drm_bridge *bridge)
{
struct ps8640 *ps_bridge = bridge_to_ps8640(bridge);
+ ps_bridge->pre_enabled = false;
+
ps8640_bridge_vdo_control(ps_bridge, DISABLE);
- ps8640_bridge_poweroff(ps_bridge);
+ pm_runtime_put_sync_suspend(&ps_bridge->page[PAGE0_DP_CNTL]->dev);
}
static int ps8640_bridge_attach(struct drm_bridge *bridge,
@@ -401,68 +444,21 @@ static int ps8640_bridge_attach(struct drm_bridge *bridge,
{
struct ps8640 *ps_bridge = bridge_to_ps8640(bridge);
struct device *dev = &ps_bridge->page[0]->dev;
- struct device_node *in_ep, *dsi_node;
- struct mipi_dsi_device *dsi;
- struct mipi_dsi_host *host;
int ret;
- const struct mipi_dsi_device_info info = { .type = "ps8640",
- .channel = 0,
- .node = NULL,
- };
if (!(flags & DRM_BRIDGE_ATTACH_NO_CONNECTOR))
return -EINVAL;
- /* port@0 is ps8640 dsi input port */
- in_ep = of_graph_get_endpoint_by_regs(dev->of_node, 0, -1);
- if (!in_ep)
- return -ENODEV;
-
- dsi_node = of_graph_get_remote_port_parent(in_ep);
- of_node_put(in_ep);
- if (!dsi_node)
- return -ENODEV;
-
- host = of_find_mipi_dsi_host_by_node(dsi_node);
- of_node_put(dsi_node);
- if (!host)
- return -ENODEV;
-
- dsi = mipi_dsi_device_register_full(host, &info);
- if (IS_ERR(dsi)) {
- dev_err(dev, "failed to create dsi device\n");
- ret = PTR_ERR(dsi);
- return ret;
- }
-
- ps_bridge->dsi = dsi;
-
- dsi->host = host;
- dsi->mode_flags = MIPI_DSI_MODE_VIDEO |
- MIPI_DSI_MODE_VIDEO_SYNC_PULSE;
- dsi->format = MIPI_DSI_FMT_RGB888;
- dsi->lanes = NUM_MIPI_LANES;
- ret = mipi_dsi_attach(dsi);
- if (ret) {
- dev_err(dev, "failed to attach dsi device: %d\n", ret);
- goto err_dsi_attach;
- }
-
+ ps_bridge->aux.drm_dev = bridge->dev;
ret = drm_dp_aux_register(&ps_bridge->aux);
if (ret) {
dev_err(dev, "failed to register DP AUX channel: %d\n", ret);
- goto err_aux_register;
+ return ret;
}
/* Attach the panel-bridge to the dsi bridge */
return drm_bridge_attach(bridge->encoder, ps_bridge->panel_bridge,
&ps_bridge->bridge, flags);
-
-err_aux_register:
- mipi_dsi_detach(dsi);
-err_dsi_attach:
- mipi_dsi_device_unregister(dsi);
- return ret;
}
static void ps8640_bridge_detach(struct drm_bridge *bridge)
@@ -474,7 +470,7 @@ static struct edid *ps8640_bridge_get_edid(struct drm_bridge *bridge,
struct drm_connector *connector)
{
struct ps8640 *ps_bridge = bridge_to_ps8640(bridge);
- bool poweroff = !ps_bridge->powered;
+ bool poweroff = !ps_bridge->pre_enabled;
struct edid *edid;
/*
@@ -504,6 +500,12 @@ static struct edid *ps8640_bridge_get_edid(struct drm_bridge *bridge,
return edid;
}
+static void ps8640_runtime_disable(void *data)
+{
+ pm_runtime_dont_use_autosuspend(data);
+ pm_runtime_disable(data);
+}
+
static const struct drm_bridge_funcs ps8640_bridge_funcs = {
.attach = ps8640_bridge_attach,
.detach = ps8640_bridge_detach,
@@ -512,6 +514,53 @@ static const struct drm_bridge_funcs ps8640_bridge_funcs = {
.pre_enable = ps8640_pre_enable,
};
+static int ps8640_bridge_host_attach(struct device *dev, struct ps8640 *ps_bridge)
+{
+ struct device_node *in_ep, *dsi_node;
+ struct mipi_dsi_device *dsi;
+ struct mipi_dsi_host *host;
+ int ret;
+ const struct mipi_dsi_device_info info = { .type = "ps8640",
+ .channel = 0,
+ .node = NULL,
+ };
+
+ /* port@0 is ps8640 dsi input port */
+ in_ep = of_graph_get_endpoint_by_regs(dev->of_node, 0, -1);
+ if (!in_ep)
+ return -ENODEV;
+
+ dsi_node = of_graph_get_remote_port_parent(in_ep);
+ of_node_put(in_ep);
+ if (!dsi_node)
+ return -ENODEV;
+
+ host = of_find_mipi_dsi_host_by_node(dsi_node);
+ of_node_put(dsi_node);
+ if (!host)
+ return -EPROBE_DEFER;
+
+ dsi = devm_mipi_dsi_device_register_full(dev, host, &info);
+ if (IS_ERR(dsi)) {
+ dev_err(dev, "failed to create dsi device\n");
+ return PTR_ERR(dsi);
+ }
+
+ ps_bridge->dsi = dsi;
+
+ dsi->host = host;
+ dsi->mode_flags = MIPI_DSI_MODE_VIDEO |
+ MIPI_DSI_MODE_VIDEO_SYNC_PULSE;
+ dsi->format = MIPI_DSI_FMT_RGB888;
+ dsi->lanes = NUM_MIPI_LANES;
+
+ ret = devm_mipi_dsi_attach(dev, dsi);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
static int ps8640_probe(struct i2c_client *client)
{
struct device *dev = &client->dev;
@@ -525,17 +574,6 @@ static int ps8640_probe(struct i2c_client *client)
if (!ps_bridge)
return -ENOMEM;
- /* port@1 is ps8640 output port */
- ret = drm_of_find_panel_or_bridge(np, 1, 0, &panel, NULL);
- if (ret < 0)
- return ret;
- if (!panel)
- return -ENODEV;
-
- ps_bridge->panel_bridge = devm_drm_panel_bridge_add(dev, panel);
- if (IS_ERR(ps_bridge->panel_bridge))
- return PTR_ERR(ps_bridge->panel_bridge);
-
ps_bridge->supplies[0].supply = "vdd33";
ps_bridge->supplies[1].supply = "vdd12";
ret = devm_regulator_bulk_get(dev, ARRAY_SIZE(ps_bridge->supplies),
@@ -558,9 +596,16 @@ static int ps8640_probe(struct i2c_client *client)
ps_bridge->bridge.funcs = &ps8640_bridge_funcs;
ps_bridge->bridge.of_node = dev->of_node;
- ps_bridge->bridge.ops = DRM_BRIDGE_OP_EDID;
ps_bridge->bridge.type = DRM_MODE_CONNECTOR_eDP;
+ /*
+ * In the device tree, if panel is listed under aux-bus of the bridge
+ * node, panel driver should be able to retrieve EDID by itself using
+ * aux-bus. So let's not set DRM_BRIDGE_OP_EDID here.
+ */
+ if (!ps8640_of_panel_on_aux_bus(&client->dev))
+ ps_bridge->bridge.ops = DRM_BRIDGE_OP_EDID;
+
ps_bridge->page[PAGE0_DP_CNTL] = client;
ps_bridge->regmap[PAGE0_DP_CNTL] = devm_regmap_init_i2c(client, ps8640_regmap_config);
@@ -587,9 +632,46 @@ static int ps8640_probe(struct i2c_client *client)
ps_bridge->aux.transfer = ps8640_aux_transfer;
drm_dp_aux_init(&ps_bridge->aux);
+ pm_runtime_enable(dev);
+ /*
+ * Powering on ps8640 takes ~300ms. To avoid wasting time on power
+ * cycling ps8640 too often, set autosuspend_delay to 1000ms to ensure
+ * the bridge wouldn't suspend in between each _aux_transfer_msg() call
+ * during EDID read (~20ms in my experiment) and in between the last
+ * _aux_transfer_msg() call during EDID read and the _pre_enable() call
+ * (~100ms in my experiment).
+ */
+ pm_runtime_set_autosuspend_delay(dev, 1000);
+ pm_runtime_use_autosuspend(dev);
+ pm_suspend_ignore_children(dev, true);
+ ret = devm_add_action_or_reset(dev, ps8640_runtime_disable, dev);
+ if (ret)
+ return ret;
+
+ devm_of_dp_aux_populate_ep_devices(&ps_bridge->aux);
+
+ /* port@1 is ps8640 output port */
+ ret = drm_of_find_panel_or_bridge(np, 1, 0, &panel, NULL);
+ if (ret < 0)
+ return ret;
+ if (!panel)
+ return -ENODEV;
+
+ ps_bridge->panel_bridge = devm_drm_panel_bridge_add(dev, panel);
+ if (IS_ERR(ps_bridge->panel_bridge))
+ return PTR_ERR(ps_bridge->panel_bridge);
+
drm_bridge_add(&ps_bridge->bridge);
+ ret = ps8640_bridge_host_attach(dev, ps_bridge);
+ if (ret)
+ goto err_bridge_remove;
+
return 0;
+
+err_bridge_remove:
+ drm_bridge_remove(&ps_bridge->bridge);
+ return ret;
}
static int ps8640_remove(struct i2c_client *client)
@@ -613,6 +695,7 @@ static struct i2c_driver ps8640_driver = {
.driver = {
.name = "ps8640",
.of_match_table = ps8640_match,
+ .pm = &ps8640_pm_ops,
},
};
module_i2c_driver(ps8640_driver);
diff --git a/drivers/gpu/drm/bridge/synopsys/dw-hdmi-ahb-audio.c b/drivers/gpu/drm/bridge/synopsys/dw-hdmi-ahb-audio.c
index d0db1ac..7d2ed0e 100644
--- a/drivers/gpu/drm/bridge/synopsys/dw-hdmi-ahb-audio.c
+++ b/drivers/gpu/drm/bridge/synopsys/dw-hdmi-ahb-audio.c
@@ -320,13 +320,17 @@ static int dw_hdmi_open(struct snd_pcm_substream *substream)
struct snd_pcm_runtime *runtime = substream->runtime;
struct snd_dw_hdmi *dw = substream->private_data;
void __iomem *base = dw->data.base;
+ u8 *eld;
int ret;
runtime->hw = dw_hdmi_hw;
- ret = snd_pcm_hw_constraint_eld(runtime, dw->data.eld);
- if (ret < 0)
- return ret;
+ eld = dw->data.get_eld(dw->data.hdmi);
+ if (eld) {
+ ret = snd_pcm_hw_constraint_eld(runtime, eld);
+ if (ret < 0)
+ return ret;
+ }
ret = snd_pcm_limit_hw_rates(runtime);
if (ret < 0)
diff --git a/drivers/gpu/drm/bridge/synopsys/dw-hdmi-audio.h b/drivers/gpu/drm/bridge/synopsys/dw-hdmi-audio.h
index cb07dc0..f72d272 100644
--- a/drivers/gpu/drm/bridge/synopsys/dw-hdmi-audio.h
+++ b/drivers/gpu/drm/bridge/synopsys/dw-hdmi-audio.h
@@ -9,15 +9,15 @@ struct dw_hdmi_audio_data {
void __iomem *base;
int irq;
struct dw_hdmi *hdmi;
- u8 *eld;
+ u8 *(*get_eld)(struct dw_hdmi *hdmi);
};
struct dw_hdmi_i2s_audio_data {
struct dw_hdmi *hdmi;
- u8 *eld;
void (*write)(struct dw_hdmi *hdmi, u8 val, int offset);
u8 (*read)(struct dw_hdmi *hdmi, int offset);
+ u8 *(*get_eld)(struct dw_hdmi *hdmi);
};
#endif
diff --git a/drivers/gpu/drm/bridge/synopsys/dw-hdmi-i2s-audio.c b/drivers/gpu/drm/bridge/synopsys/dw-hdmi-i2s-audio.c
index feb04f1..f50b47a 100644
--- a/drivers/gpu/drm/bridge/synopsys/dw-hdmi-i2s-audio.c
+++ b/drivers/gpu/drm/bridge/synopsys/dw-hdmi-i2s-audio.c
@@ -135,8 +135,15 @@ static int dw_hdmi_i2s_get_eld(struct device *dev, void *data, uint8_t *buf,
size_t len)
{
struct dw_hdmi_i2s_audio_data *audio = data;
+ u8 *eld;
- memcpy(buf, audio->eld, min_t(size_t, MAX_ELD_BYTES, len));
+ eld = audio->get_eld(audio->hdmi);
+ if (eld)
+ memcpy(buf, eld, min_t(size_t, MAX_ELD_BYTES, len));
+ else
+ /* Pass en empty ELD if connector not available */
+ memset(buf, 0, len);
+
return 0;
}
diff --git a/drivers/gpu/drm/bridge/synopsys/dw-hdmi.c b/drivers/gpu/drm/bridge/synopsys/dw-hdmi.c
index f08d0fd..54d8fda 100644
--- a/drivers/gpu/drm/bridge/synopsys/dw-hdmi.c
+++ b/drivers/gpu/drm/bridge/synopsys/dw-hdmi.c
@@ -757,6 +757,14 @@ static void hdmi_enable_audio_clk(struct dw_hdmi *hdmi, bool enable)
hdmi_writeb(hdmi, hdmi->mc_clkdis, HDMI_MC_CLKDIS);
}
+static u8 *hdmi_audio_get_eld(struct dw_hdmi *hdmi)
+{
+ if (!hdmi->curr_conn)
+ return NULL;
+
+ return hdmi->curr_conn->eld;
+}
+
static void dw_hdmi_ahb_audio_enable(struct dw_hdmi *hdmi)
{
hdmi_set_cts_n(hdmi, hdmi->audio_cts, hdmi->audio_n);
@@ -3413,6 +3421,7 @@ struct dw_hdmi *dw_hdmi_probe(struct platform_device *pdev,
hdmi->bridge.funcs = &dw_hdmi_bridge_funcs;
hdmi->bridge.ops = DRM_BRIDGE_OP_DETECT | DRM_BRIDGE_OP_EDID
| DRM_BRIDGE_OP_HPD;
+ hdmi->bridge.interlace_allowed = true;
#ifdef CONFIG_OF
hdmi->bridge.of_node = pdev->dev.of_node;
#endif
@@ -3431,7 +3440,7 @@ struct dw_hdmi *dw_hdmi_probe(struct platform_device *pdev,
audio.base = hdmi->regs;
audio.irq = irq;
audio.hdmi = hdmi;
- audio.eld = hdmi->connector.eld;
+ audio.get_eld = hdmi_audio_get_eld;
hdmi->enable_audio = dw_hdmi_ahb_audio_enable;
hdmi->disable_audio = dw_hdmi_ahb_audio_disable;
@@ -3444,7 +3453,7 @@ struct dw_hdmi *dw_hdmi_probe(struct platform_device *pdev,
struct dw_hdmi_i2s_audio_data audio;
audio.hdmi = hdmi;
- audio.eld = hdmi->connector.eld;
+ audio.get_eld = hdmi_audio_get_eld;
audio.write = hdmi_writeb;
audio.read = hdmi_readb;
hdmi->enable_audio = dw_hdmi_i2s_audio_enable;
diff --git a/drivers/gpu/drm/bridge/tc358768.c b/drivers/gpu/drm/bridge/tc358768.c
index a3db532..fd585bf 100644
--- a/drivers/gpu/drm/bridge/tc358768.c
+++ b/drivers/gpu/drm/bridge/tc358768.c
@@ -237,6 +237,10 @@ static void tc358768_hw_enable(struct tc358768_priv *priv)
if (priv->enabled)
return;
+ ret = clk_prepare_enable(priv->refclk);
+ if (ret < 0)
+ dev_err(priv->dev, "error enabling refclk (%d)\n", ret);
+
ret = regulator_bulk_enable(ARRAY_SIZE(priv->supplies), priv->supplies);
if (ret < 0)
dev_err(priv->dev, "error enabling regulators (%d)\n", ret);
@@ -274,6 +278,8 @@ static void tc358768_hw_disable(struct tc358768_priv *priv)
if (ret < 0)
dev_err(priv->dev, "error disabling regulators (%d)\n", ret);
+ clk_disable_unprepare(priv->refclk);
+
priv->enabled = false;
}
@@ -625,12 +631,19 @@ static void tc358768_bridge_pre_enable(struct drm_bridge *bridge)
{
struct tc358768_priv *priv = bridge_to_tc358768(bridge);
struct mipi_dsi_device *dsi_dev = priv->output.dev;
+ unsigned long mode_flags = dsi_dev->mode_flags;
u32 val, val2, lptxcnt, hact, data_type;
const struct drm_display_mode *mode;
u32 dsibclk_nsk, dsiclk_nsk, ui_nsk, phy_delay_nsk;
- u32 dsiclk, dsibclk;
+ u32 dsiclk, dsibclk, video_start;
+ const u32 internal_delay = 40;
int ret, i;
+ if (mode_flags & MIPI_DSI_CLOCK_NON_CONTINUOUS) {
+ dev_warn_once(priv->dev, "Non-continuous mode unimplemented, falling back to continuous\n");
+ mode_flags &= ~MIPI_DSI_CLOCK_NON_CONTINUOUS;
+ }
+
tc358768_hw_enable(priv);
ret = tc358768_sw_reset(priv);
@@ -657,23 +670,27 @@ static void tc358768_bridge_pre_enable(struct drm_bridge *bridge)
case MIPI_DSI_FMT_RGB888:
val |= (0x3 << 4);
hact = mode->hdisplay * 3;
+ video_start = (mode->htotal - mode->hsync_start) * 3;
data_type = MIPI_DSI_PACKED_PIXEL_STREAM_24;
break;
case MIPI_DSI_FMT_RGB666:
val |= (0x4 << 4);
hact = mode->hdisplay * 3;
+ video_start = (mode->htotal - mode->hsync_start) * 3;
data_type = MIPI_DSI_PACKED_PIXEL_STREAM_18;
break;
case MIPI_DSI_FMT_RGB666_PACKED:
val |= (0x4 << 4) | BIT(3);
hact = mode->hdisplay * 18 / 8;
+ video_start = (mode->htotal - mode->hsync_start) * 18 / 8;
data_type = MIPI_DSI_PIXEL_STREAM_3BYTE_18;
break;
case MIPI_DSI_FMT_RGB565:
val |= (0x5 << 4);
hact = mode->hdisplay * 2;
+ video_start = (mode->htotal - mode->hsync_start) * 2;
data_type = MIPI_DSI_PACKED_PIXEL_STREAM_16;
break;
default:
@@ -684,7 +701,8 @@ static void tc358768_bridge_pre_enable(struct drm_bridge *bridge)
}
/* VSDly[9:0] */
- tc358768_write(priv, TC358768_VSDLY, 1);
+ video_start = max(video_start, internal_delay + 1) - internal_delay;
+ tc358768_write(priv, TC358768_VSDLY, video_start);
tc358768_write(priv, TC358768_DATAFMT, val);
tc358768_write(priv, TC358768_DSITX_DT, data_type);
@@ -764,7 +782,7 @@ static void tc358768_bridge_pre_enable(struct drm_bridge *bridge)
val |= BIT(i + 1);
tc358768_write(priv, TC358768_HSTXVREGEN, val);
- if (!(dsi_dev->mode_flags & MIPI_DSI_CLOCK_NON_CONTINUOUS))
+ if (!(mode_flags & MIPI_DSI_CLOCK_NON_CONTINUOUS))
tc358768_write(priv, TC358768_TXOPTIONCNTRL, 0x1);
/* TXTAGOCNT[26:16] RXTASURECNT[10:0] */
@@ -772,31 +790,61 @@ static void tc358768_bridge_pre_enable(struct drm_bridge *bridge)
val = tc358768_ns_to_cnt(val, dsibclk_nsk) - 1;
val2 = tc358768_ns_to_cnt(tc358768_to_ns((lptxcnt + 1) * dsibclk_nsk),
dsibclk_nsk) - 2;
- val |= val2 << 16;
+ val = val << 16 | val2;
dev_dbg(priv->dev, "BTACNTRL1: 0x%x\n", val);
tc358768_write(priv, TC358768_BTACNTRL1, val);
/* START[0] */
tc358768_write(priv, TC358768_STARTCNTRL, 1);
- /* Set event mode */
- tc358768_write(priv, TC358768_DSI_EVENT, 1);
+ if (dsi_dev->mode_flags & MIPI_DSI_MODE_VIDEO_SYNC_PULSE) {
+ /* Set pulse mode */
+ tc358768_write(priv, TC358768_DSI_EVENT, 0);
- /* vsw (+ vbp) */
- tc358768_write(priv, TC358768_DSI_VSW,
- mode->vtotal - mode->vsync_start);
- /* vbp (not used in event mode) */
- tc358768_write(priv, TC358768_DSI_VBPR, 0);
- /* vact */
- tc358768_write(priv, TC358768_DSI_VACT, mode->vdisplay);
+ /* vact */
+ tc358768_write(priv, TC358768_DSI_VACT, mode->vdisplay);
- /* (hsw + hbp) * byteclk * ndl / pclk */
- val = (u32)div_u64((mode->htotal - mode->hsync_start) *
- ((u64)priv->dsiclk / 4) * priv->dsi_lanes,
- mode->clock * 1000);
- tc358768_write(priv, TC358768_DSI_HSW, val);
- /* hbp (not used in event mode) */
- tc358768_write(priv, TC358768_DSI_HBPR, 0);
+ /* vsw */
+ tc358768_write(priv, TC358768_DSI_VSW,
+ mode->vsync_end - mode->vsync_start);
+ /* vbp */
+ tc358768_write(priv, TC358768_DSI_VBPR,
+ mode->vtotal - mode->vsync_end);
+
+ /* hsw * byteclk * ndl / pclk */
+ val = (u32)div_u64((mode->hsync_end - mode->hsync_start) *
+ ((u64)priv->dsiclk / 4) * priv->dsi_lanes,
+ mode->clock * 1000);
+ tc358768_write(priv, TC358768_DSI_HSW, val);
+
+ /* hbp * byteclk * ndl / pclk */
+ val = (u32)div_u64((mode->htotal - mode->hsync_end) *
+ ((u64)priv->dsiclk / 4) * priv->dsi_lanes,
+ mode->clock * 1000);
+ tc358768_write(priv, TC358768_DSI_HBPR, val);
+ } else {
+ /* Set event mode */
+ tc358768_write(priv, TC358768_DSI_EVENT, 1);
+
+ /* vact */
+ tc358768_write(priv, TC358768_DSI_VACT, mode->vdisplay);
+
+ /* vsw (+ vbp) */
+ tc358768_write(priv, TC358768_DSI_VSW,
+ mode->vtotal - mode->vsync_start);
+ /* vbp (not used in event mode) */
+ tc358768_write(priv, TC358768_DSI_VBPR, 0);
+
+ /* (hsw + hbp) * byteclk * ndl / pclk */
+ val = (u32)div_u64((mode->htotal - mode->hsync_start) *
+ ((u64)priv->dsiclk / 4) * priv->dsi_lanes,
+ mode->clock * 1000);
+ tc358768_write(priv, TC358768_DSI_HSW, val);
+
+ /* hbp (not used in event mode) */
+ tc358768_write(priv, TC358768_DSI_HBPR, 0);
+ }
+
/* hact (bytes) */
tc358768_write(priv, TC358768_DSI_HACT, hact);
@@ -822,7 +870,7 @@ static void tc358768_bridge_pre_enable(struct drm_bridge *bridge)
if (!(dsi_dev->mode_flags & MIPI_DSI_MODE_LPM))
val |= TC358768_DSI_CONTROL_TXMD;
- if (!(dsi_dev->mode_flags & MIPI_DSI_CLOCK_NON_CONTINUOUS))
+ if (!(mode_flags & MIPI_DSI_CLOCK_NON_CONTINUOUS))
val |= TC358768_DSI_CONTROL_HSCKMD;
if (dsi_dev->mode_flags & MIPI_DSI_MODE_NO_EOT_PACKET)
diff --git a/drivers/gpu/drm/bridge/tc358775.c b/drivers/gpu/drm/bridge/tc358775.c
index 2272adc..2c76331 100644
--- a/drivers/gpu/drm/bridge/tc358775.c
+++ b/drivers/gpu/drm/bridge/tc358775.c
@@ -594,11 +594,26 @@ static int tc_bridge_attach(struct drm_bridge *bridge,
enum drm_bridge_attach_flags flags)
{
struct tc_data *tc = bridge_to_tc(bridge);
+
+ /* Attach the panel-bridge to the dsi bridge */
+ return drm_bridge_attach(bridge->encoder, tc->panel_bridge,
+ &tc->bridge, flags);
+}
+
+static const struct drm_bridge_funcs tc_bridge_funcs = {
+ .attach = tc_bridge_attach,
+ .pre_enable = tc_bridge_pre_enable,
+ .enable = tc_bridge_enable,
+ .mode_valid = tc_mode_valid,
+ .post_disable = tc_bridge_post_disable,
+};
+
+static int tc_attach_host(struct tc_data *tc)
+{
struct device *dev = &tc->i2c->dev;
struct mipi_dsi_host *host;
struct mipi_dsi_device *dsi;
int ret;
-
const struct mipi_dsi_device_info info = { .type = "tc358775",
.channel = 0,
.node = NULL,
@@ -610,11 +625,10 @@ static int tc_bridge_attach(struct drm_bridge *bridge,
return -EPROBE_DEFER;
}
- dsi = mipi_dsi_device_register_full(host, &info);
+ dsi = devm_mipi_dsi_device_register_full(dev, host, &info);
if (IS_ERR(dsi)) {
dev_err(dev, "failed to create dsi device\n");
- ret = PTR_ERR(dsi);
- goto err_dsi_device;
+ return PTR_ERR(dsi);
}
tc->dsi = dsi;
@@ -623,29 +637,15 @@ static int tc_bridge_attach(struct drm_bridge *bridge,
dsi->format = MIPI_DSI_FMT_RGB888;
dsi->mode_flags = MIPI_DSI_MODE_VIDEO;
- ret = mipi_dsi_attach(dsi);
+ ret = devm_mipi_dsi_attach(dev, dsi);
if (ret < 0) {
dev_err(dev, "failed to attach dsi to host\n");
- goto err_dsi_attach;
+ return ret;
}
- /* Attach the panel-bridge to the dsi bridge */
- return drm_bridge_attach(bridge->encoder, tc->panel_bridge,
- &tc->bridge, flags);
-err_dsi_attach:
- mipi_dsi_device_unregister(dsi);
-err_dsi_device:
- return ret;
+ return 0;
}
-static const struct drm_bridge_funcs tc_bridge_funcs = {
- .attach = tc_bridge_attach,
- .pre_enable = tc_bridge_pre_enable,
- .enable = tc_bridge_enable,
- .mode_valid = tc_mode_valid,
- .post_disable = tc_bridge_post_disable,
-};
-
static int tc_probe(struct i2c_client *client, const struct i2c_device_id *id)
{
struct device *dev = &client->dev;
@@ -709,7 +709,15 @@ static int tc_probe(struct i2c_client *client, const struct i2c_device_id *id)
i2c_set_clientdata(client, tc);
+ ret = tc_attach_host(tc);
+ if (ret)
+ goto err_bridge_remove;
+
return 0;
+
+err_bridge_remove:
+ drm_bridge_remove(&tc->bridge);
+ return ret;
}
static int tc_remove(struct i2c_client *client)
diff --git a/drivers/gpu/drm/bridge/ti-sn65dsi83.c b/drivers/gpu/drm/bridge/ti-sn65dsi83.c
index ba1160e..945f08d 100644
--- a/drivers/gpu/drm/bridge/ti-sn65dsi83.c
+++ b/drivers/gpu/drm/bridge/ti-sn65dsi83.c
@@ -245,47 +245,9 @@ static int sn65dsi83_attach(struct drm_bridge *bridge,
enum drm_bridge_attach_flags flags)
{
struct sn65dsi83 *ctx = bridge_to_sn65dsi83(bridge);
- struct device *dev = ctx->dev;
- struct mipi_dsi_device *dsi;
- struct mipi_dsi_host *host;
- int ret = 0;
-
- const struct mipi_dsi_device_info info = {
- .type = "sn65dsi83",
- .channel = 0,
- .node = NULL,
- };
-
- host = of_find_mipi_dsi_host_by_node(ctx->host_node);
- if (!host) {
- dev_err(dev, "failed to find dsi host\n");
- return -EPROBE_DEFER;
- }
-
- dsi = mipi_dsi_device_register_full(host, &info);
- if (IS_ERR(dsi)) {
- return dev_err_probe(dev, PTR_ERR(dsi),
- "failed to create dsi device\n");
- }
-
- ctx->dsi = dsi;
-
- dsi->lanes = ctx->dsi_lanes;
- dsi->format = MIPI_DSI_FMT_RGB888;
- dsi->mode_flags = MIPI_DSI_MODE_VIDEO | MIPI_DSI_MODE_VIDEO_BURST;
-
- ret = mipi_dsi_attach(dsi);
- if (ret < 0) {
- dev_err(dev, "failed to attach dsi to host\n");
- goto err_dsi_attach;
- }
return drm_bridge_attach(bridge->encoder, ctx->panel_bridge,
&ctx->bridge, flags);
-
-err_dsi_attach:
- mipi_dsi_device_unregister(dsi);
- return ret;
}
static void sn65dsi83_detach(struct drm_bridge *bridge)
@@ -295,28 +257,9 @@ static void sn65dsi83_detach(struct drm_bridge *bridge)
if (!ctx->dsi)
return;
- mipi_dsi_detach(ctx->dsi);
- mipi_dsi_device_unregister(ctx->dsi);
- drm_bridge_remove(&ctx->bridge);
ctx->dsi = NULL;
}
-static void sn65dsi83_atomic_pre_enable(struct drm_bridge *bridge,
- struct drm_bridge_state *old_bridge_state)
-{
- struct sn65dsi83 *ctx = bridge_to_sn65dsi83(bridge);
-
- /*
- * Reset the chip, pull EN line low for t_reset=10ms,
- * then high for t_en=1ms.
- */
- regcache_mark_dirty(ctx->regmap);
- gpiod_set_value(ctx->enable_gpio, 0);
- usleep_range(10000, 11000);
- gpiod_set_value(ctx->enable_gpio, 1);
- usleep_range(1000, 1100);
-}
-
static u8 sn65dsi83_get_lvds_range(struct sn65dsi83 *ctx,
const struct drm_display_mode *mode)
{
@@ -394,6 +337,10 @@ static void sn65dsi83_atomic_enable(struct drm_bridge *bridge,
u16 val;
int ret;
+ /* Deassert reset */
+ gpiod_set_value(ctx->enable_gpio, 1);
+ usleep_range(1000, 1100);
+
/* Get the LVDS format from the bridge state. */
bridge_state = drm_atomic_get_new_bridge_state(state, bridge);
@@ -540,18 +487,11 @@ static void sn65dsi83_atomic_disable(struct drm_bridge *bridge,
{
struct sn65dsi83 *ctx = bridge_to_sn65dsi83(bridge);
- /* Clear reset, disable PLL */
- regmap_write(ctx->regmap, REG_RC_RESET, 0x00);
- regmap_write(ctx->regmap, REG_RC_PLL_EN, 0x00);
-}
-
-static void sn65dsi83_atomic_post_disable(struct drm_bridge *bridge,
- struct drm_bridge_state *old_bridge_state)
-{
- struct sn65dsi83 *ctx = bridge_to_sn65dsi83(bridge);
-
- /* Put the chip in reset, pull EN line low. */
+ /* Put the chip in reset, pull EN line low, and assure 10ms reset low timing. */
gpiod_set_value(ctx->enable_gpio, 0);
+ usleep_range(10000, 11000);
+
+ regcache_mark_dirty(ctx->regmap);
}
static enum drm_mode_status
@@ -597,10 +537,8 @@ sn65dsi83_atomic_get_input_bus_fmts(struct drm_bridge *bridge,
static const struct drm_bridge_funcs sn65dsi83_funcs = {
.attach = sn65dsi83_attach,
.detach = sn65dsi83_detach,
- .atomic_pre_enable = sn65dsi83_atomic_pre_enable,
.atomic_enable = sn65dsi83_atomic_enable,
.atomic_disable = sn65dsi83_atomic_disable,
- .atomic_post_disable = sn65dsi83_atomic_post_disable,
.mode_valid = sn65dsi83_mode_valid,
.atomic_duplicate_state = drm_atomic_helper_bridge_duplicate_state,
@@ -664,6 +602,44 @@ static int sn65dsi83_parse_dt(struct sn65dsi83 *ctx, enum sn65dsi83_model model)
return 0;
}
+static int sn65dsi83_host_attach(struct sn65dsi83 *ctx)
+{
+ struct device *dev = ctx->dev;
+ struct mipi_dsi_device *dsi;
+ struct mipi_dsi_host *host;
+ const struct mipi_dsi_device_info info = {
+ .type = "sn65dsi83",
+ .channel = 0,
+ .node = NULL,
+ };
+ int ret;
+
+ host = of_find_mipi_dsi_host_by_node(ctx->host_node);
+ if (!host) {
+ dev_err(dev, "failed to find dsi host\n");
+ return -EPROBE_DEFER;
+ }
+
+ dsi = devm_mipi_dsi_device_register_full(dev, host, &info);
+ if (IS_ERR(dsi))
+ return dev_err_probe(dev, PTR_ERR(dsi),
+ "failed to create dsi device\n");
+
+ ctx->dsi = dsi;
+
+ dsi->lanes = ctx->dsi_lanes;
+ dsi->format = MIPI_DSI_FMT_RGB888;
+ dsi->mode_flags = MIPI_DSI_MODE_VIDEO | MIPI_DSI_MODE_VIDEO_BURST;
+
+ ret = devm_mipi_dsi_attach(dev, dsi);
+ if (ret < 0) {
+ dev_err(dev, "failed to attach dsi to host: %d\n", ret);
+ return ret;
+ }
+
+ return 0;
+}
+
static int sn65dsi83_probe(struct i2c_client *client,
const struct i2c_device_id *id)
{
@@ -685,10 +661,13 @@ static int sn65dsi83_probe(struct i2c_client *client,
model = id->driver_data;
}
+ /* Put the chip in reset, pull EN line low, and assure 10ms reset low timing. */
ctx->enable_gpio = devm_gpiod_get(ctx->dev, "enable", GPIOD_OUT_LOW);
if (IS_ERR(ctx->enable_gpio))
return PTR_ERR(ctx->enable_gpio);
+ usleep_range(10000, 11000);
+
ret = sn65dsi83_parse_dt(ctx, model);
if (ret)
return ret;
@@ -704,13 +683,22 @@ static int sn65dsi83_probe(struct i2c_client *client,
ctx->bridge.of_node = dev->of_node;
drm_bridge_add(&ctx->bridge);
+ ret = sn65dsi83_host_attach(ctx);
+ if (ret)
+ goto err_remove_bridge;
+
return 0;
+
+err_remove_bridge:
+ drm_bridge_remove(&ctx->bridge);
+ return ret;
}
static int sn65dsi83_remove(struct i2c_client *client)
{
struct sn65dsi83 *ctx = i2c_get_clientdata(client);
+ drm_bridge_remove(&ctx->bridge);
of_node_put(ctx->host_node);
return 0;
diff --git a/drivers/gpu/drm/bridge/ti-sn65dsi86.c b/drivers/gpu/drm/bridge/ti-sn65dsi86.c
index 6154bed..dab8f76 100644
--- a/drivers/gpu/drm/bridge/ti-sn65dsi86.c
+++ b/drivers/gpu/drm/bridge/ti-sn65dsi86.c
@@ -4,7 +4,9 @@
* datasheet: https://www.ti.com/lit/ds/symlink/sn65dsi86.pdf
*/
+#include <linux/atomic.h>
#include <linux/auxiliary_bus.h>
+#include <linux/bitfield.h>
#include <linux/bits.h>
#include <linux/clk.h>
#include <linux/debugfs.h>
@@ -15,6 +17,7 @@
#include <linux/module.h>
#include <linux/of_graph.h>
#include <linux/pm_runtime.h>
+#include <linux/pwm.h>
#include <linux/regmap.h>
#include <linux/regulator/consumer.h>
@@ -91,6 +94,13 @@
#define SN_ML_TX_MODE_REG 0x96
#define ML_TX_MAIN_LINK_OFF 0
#define ML_TX_NORMAL_MODE BIT(0)
+#define SN_PWM_PRE_DIV_REG 0xA0
+#define SN_BACKLIGHT_SCALE_REG 0xA1
+#define BACKLIGHT_SCALE_MAX 0xFFFF
+#define SN_BACKLIGHT_REG 0xA3
+#define SN_PWM_EN_INV_REG 0xA5
+#define SN_PWM_INV_MASK BIT(0)
+#define SN_PWM_EN_MASK BIT(1)
#define SN_AUX_CMD_STATUS_REG 0xF4
#define AUX_IRQ_STATUS_AUX_RPLY_TOUT BIT(3)
#define AUX_IRQ_STATUS_AUX_SHORT BIT(5)
@@ -113,11 +123,14 @@
#define SN_LINK_TRAINING_TRIES 10
+#define SN_PWM_GPIO_IDX 3 /* 4th GPIO */
+
/**
* struct ti_sn65dsi86 - Platform data for ti-sn65dsi86 driver.
* @bridge_aux: AUX-bus sub device for MIPI-to-eDP bridge functionality.
* @gpio_aux: AUX-bus sub device for GPIO controller functionality.
* @aux_aux: AUX-bus sub device for eDP AUX channel functionality.
+ * @pwm_aux: AUX-bus sub device for PWM controller functionality.
*
* @dev: Pointer to the top level (i2c) device.
* @regmap: Regmap for accessing i2c.
@@ -145,11 +158,17 @@
* bitmap so we can do atomic ops on it without an extra
* lock so concurrent users of our 4 GPIOs don't stomp on
* each other's read-modify-write.
+ *
+ * @pchip: pwm_chip if the PWM is exposed.
+ * @pwm_enabled: Used to track if the PWM signal is currently enabled.
+ * @pwm_pin_busy: Track if GPIO4 is currently requested for GPIO or PWM.
+ * @pwm_refclk_freq: Cache for the reference clock input to the PWM.
*/
struct ti_sn65dsi86 {
struct auxiliary_device bridge_aux;
struct auxiliary_device gpio_aux;
struct auxiliary_device aux_aux;
+ struct auxiliary_device pwm_aux;
struct device *dev;
struct regmap *regmap;
@@ -172,6 +191,12 @@ struct ti_sn65dsi86 {
struct gpio_chip gchip;
DECLARE_BITMAP(gchip_output, SN_NUM_GPIOS);
#endif
+#if defined(CONFIG_PWM)
+ struct pwm_chip pchip;
+ bool pwm_enabled;
+ atomic_t pwm_pin_busy;
+#endif
+ unsigned int pwm_refclk_freq;
};
static const struct regmap_range ti_sn65dsi86_volatile_ranges[] = {
@@ -188,13 +213,30 @@ static const struct regmap_config ti_sn65dsi86_regmap_config = {
.val_bits = 8,
.volatile_table = &ti_sn_bridge_volatile_table,
.cache_type = REGCACHE_NONE,
+ .max_register = 0xFF,
};
+static int __maybe_unused ti_sn65dsi86_read_u16(struct ti_sn65dsi86 *pdata,
+ unsigned int reg, u16 *val)
+{
+ u8 buf[2];
+ int ret;
+
+ ret = regmap_bulk_read(pdata->regmap, reg, buf, ARRAY_SIZE(buf));
+ if (ret)
+ return ret;
+
+ *val = buf[0] | (buf[1] << 8);
+
+ return 0;
+}
+
static void ti_sn65dsi86_write_u16(struct ti_sn65dsi86 *pdata,
unsigned int reg, u16 val)
{
- regmap_write(pdata->regmap, reg, val & 0xFF);
- regmap_write(pdata->regmap, reg + 1, val >> 8);
+ u8 buf[2] = { val & 0xff, val >> 8 };
+
+ regmap_bulk_write(pdata->regmap, reg, buf, ARRAY_SIZE(buf));
}
static u32 ti_sn_bridge_get_dsi_freq(struct ti_sn65dsi86 *pdata)
@@ -253,6 +295,12 @@ static void ti_sn_bridge_set_refclk_freq(struct ti_sn65dsi86 *pdata)
regmap_update_bits(pdata->regmap, SN_DPPLL_SRC_REG, REFCLK_FREQ_MASK,
REFCLK_FREQ(i));
+
+ /*
+ * The PWM refclk is based on the value written to SN_DPPLL_SRC_REG,
+ * regardless of its actual sourcing.
+ */
+ pdata->pwm_refclk_freq = ti_sn_bridge_refclk_lut[i];
}
static void ti_sn65dsi86_enable_comms(struct ti_sn65dsi86 *pdata)
@@ -655,17 +703,47 @@ static struct ti_sn65dsi86 *bridge_to_ti_sn65dsi86(struct drm_bridge *bridge)
return container_of(bridge, struct ti_sn65dsi86, bridge);
}
-static int ti_sn_bridge_attach(struct drm_bridge *bridge,
- enum drm_bridge_attach_flags flags)
+static int ti_sn_attach_host(struct ti_sn65dsi86 *pdata)
{
- int ret, val;
- struct ti_sn65dsi86 *pdata = bridge_to_ti_sn65dsi86(bridge);
+ int val;
struct mipi_dsi_host *host;
struct mipi_dsi_device *dsi;
+ struct device *dev = pdata->dev;
const struct mipi_dsi_device_info info = { .type = "ti_sn_bridge",
.channel = 0,
.node = NULL,
- };
+ };
+
+ host = of_find_mipi_dsi_host_by_node(pdata->host_node);
+ if (!host)
+ return -EPROBE_DEFER;
+
+ dsi = devm_mipi_dsi_device_register_full(dev, host, &info);
+ if (IS_ERR(dsi))
+ return PTR_ERR(dsi);
+
+ /* TODO: setting to 4 MIPI lanes always for now */
+ dsi->lanes = 4;
+ dsi->format = MIPI_DSI_FMT_RGB888;
+ dsi->mode_flags = MIPI_DSI_MODE_VIDEO;
+
+ /* check if continuous dsi clock is required or not */
+ pm_runtime_get_sync(dev);
+ regmap_read(pdata->regmap, SN_DPPLL_SRC_REG, &val);
+ pm_runtime_put_autosuspend(dev);
+ if (!(val & DPPLL_CLK_SRC_DSICLK))
+ dsi->mode_flags |= MIPI_DSI_CLOCK_NON_CONTINUOUS;
+
+ pdata->dsi = dsi;
+
+ return devm_mipi_dsi_attach(dev, dsi);
+}
+
+static int ti_sn_bridge_attach(struct drm_bridge *bridge,
+ enum drm_bridge_attach_flags flags)
+{
+ struct ti_sn65dsi86 *pdata = bridge_to_ti_sn65dsi86(bridge);
+ int ret;
if (flags & DRM_BRIDGE_ATTACH_NO_CONNECTOR) {
DRM_ERROR("Fix bridge driver to make connector optional!");
@@ -683,50 +761,6 @@ static int ti_sn_bridge_attach(struct drm_bridge *bridge,
if (ret < 0)
goto err_conn_init;
- /*
- * TODO: ideally finding host resource and dsi dev registration needs
- * to be done in bridge probe. But some existing DSI host drivers will
- * wait for any of the drm_bridge/drm_panel to get added to the global
- * bridge/panel list, before completing their probe. So if we do the
- * dsi dev registration part in bridge probe, before populating in
- * the global bridge list, then it will cause deadlock as dsi host probe
- * will never complete, neither our bridge probe. So keeping it here
- * will satisfy most of the existing host drivers. Once the host driver
- * is fixed we can move the below code to bridge probe safely.
- */
- host = of_find_mipi_dsi_host_by_node(pdata->host_node);
- if (!host) {
- DRM_ERROR("failed to find dsi host\n");
- ret = -ENODEV;
- goto err_dsi_host;
- }
-
- dsi = mipi_dsi_device_register_full(host, &info);
- if (IS_ERR(dsi)) {
- DRM_ERROR("failed to create dsi device\n");
- ret = PTR_ERR(dsi);
- goto err_dsi_host;
- }
-
- /* TODO: setting to 4 MIPI lanes always for now */
- dsi->lanes = 4;
- dsi->format = MIPI_DSI_FMT_RGB888;
- dsi->mode_flags = MIPI_DSI_MODE_VIDEO;
-
- /* check if continuous dsi clock is required or not */
- pm_runtime_get_sync(pdata->dev);
- regmap_read(pdata->regmap, SN_DPPLL_SRC_REG, &val);
- pm_runtime_put_autosuspend(pdata->dev);
- if (!(val & DPPLL_CLK_SRC_DSICLK))
- dsi->mode_flags |= MIPI_DSI_CLOCK_NON_CONTINUOUS;
-
- ret = mipi_dsi_attach(dsi);
- if (ret < 0) {
- DRM_ERROR("failed to attach dsi to host\n");
- goto err_dsi_attach;
- }
- pdata->dsi = dsi;
-
/* We never want the next bridge to *also* create a connector: */
flags |= DRM_BRIDGE_ATTACH_NO_CONNECTOR;
@@ -734,14 +768,10 @@ static int ti_sn_bridge_attach(struct drm_bridge *bridge,
ret = drm_bridge_attach(bridge->encoder, pdata->next_bridge,
&pdata->bridge, flags);
if (ret < 0)
- goto err_dsi_detach;
+ goto err_dsi_host;
return 0;
-err_dsi_detach:
- mipi_dsi_detach(dsi);
-err_dsi_attach:
- mipi_dsi_device_unregister(dsi);
err_dsi_host:
drm_connector_cleanup(&pdata->connector);
err_conn_init:
@@ -1227,7 +1257,17 @@ static int ti_sn_bridge_probe(struct auxiliary_device *adev,
drm_bridge_add(&pdata->bridge);
+ ret = ti_sn_attach_host(pdata);
+ if (ret) {
+ dev_err_probe(pdata->dev, ret, "failed to attach dsi host\n");
+ goto err_remove_bridge;
+ }
+
return 0;
+
+err_remove_bridge:
+ drm_bridge_remove(&pdata->bridge);
+ return ret;
}
static void ti_sn_bridge_remove(struct auxiliary_device *adev)
@@ -1237,11 +1277,6 @@ static void ti_sn_bridge_remove(struct auxiliary_device *adev)
if (!pdata)
return;
- if (pdata->dsi) {
- mipi_dsi_detach(pdata->dsi);
- mipi_dsi_device_unregister(pdata->dsi);
- }
-
drm_bridge_remove(&pdata->bridge);
of_node_put(pdata->host_node);
@@ -1260,9 +1295,287 @@ static struct auxiliary_driver ti_sn_bridge_driver = {
};
/* -----------------------------------------------------------------------------
+ * PWM Controller
+ */
+#if defined(CONFIG_PWM)
+static int ti_sn_pwm_pin_request(struct ti_sn65dsi86 *pdata)
+{
+ return atomic_xchg(&pdata->pwm_pin_busy, 1) ? -EBUSY : 0;
+}
+
+static void ti_sn_pwm_pin_release(struct ti_sn65dsi86 *pdata)
+{
+ atomic_set(&pdata->pwm_pin_busy, 0);
+}
+
+static struct ti_sn65dsi86 *pwm_chip_to_ti_sn_bridge(struct pwm_chip *chip)
+{
+ return container_of(chip, struct ti_sn65dsi86, pchip);
+}
+
+static int ti_sn_pwm_request(struct pwm_chip *chip, struct pwm_device *pwm)
+{
+ struct ti_sn65dsi86 *pdata = pwm_chip_to_ti_sn_bridge(chip);
+
+ return ti_sn_pwm_pin_request(pdata);
+}
+
+static void ti_sn_pwm_free(struct pwm_chip *chip, struct pwm_device *pwm)
+{
+ struct ti_sn65dsi86 *pdata = pwm_chip_to_ti_sn_bridge(chip);
+
+ ti_sn_pwm_pin_release(pdata);
+}
+
+/*
+ * Limitations:
+ * - The PWM signal is not driven when the chip is powered down, or in its
+ * reset state and the driver does not implement the "suspend state"
+ * described in the documentation. In order to save power, state->enabled is
+ * interpreted as denoting if the signal is expected to be valid, and is used
+ * to determine if the chip needs to be kept powered.
+ * - Changing both period and duty_cycle is not done atomically, neither is the
+ * multi-byte register updates, so the output might briefly be undefined
+ * during update.
+ */
+static int ti_sn_pwm_apply(struct pwm_chip *chip, struct pwm_device *pwm,
+ const struct pwm_state *state)
+{
+ struct ti_sn65dsi86 *pdata = pwm_chip_to_ti_sn_bridge(chip);
+ unsigned int pwm_en_inv;
+ unsigned int backlight;
+ unsigned int pre_div;
+ unsigned int scale;
+ u64 period_max;
+ u64 period;
+ int ret;
+
+ if (!pdata->pwm_enabled) {
+ ret = pm_runtime_get_sync(pdata->dev);
+ if (ret < 0) {
+ pm_runtime_put_sync(pdata->dev);
+ return ret;
+ }
+ }
+
+ if (state->enabled) {
+ if (!pdata->pwm_enabled) {
+ /*
+ * The chip might have been powered down while we
+ * didn't hold a PM runtime reference, so mux in the
+ * PWM function on the GPIO pin again.
+ */
+ ret = regmap_update_bits(pdata->regmap, SN_GPIO_CTRL_REG,
+ SN_GPIO_MUX_MASK << (2 * SN_PWM_GPIO_IDX),
+ SN_GPIO_MUX_SPECIAL << (2 * SN_PWM_GPIO_IDX));
+ if (ret) {
+ dev_err(pdata->dev, "failed to mux in PWM function\n");
+ goto out;
+ }
+ }
+
+ /*
+ * Per the datasheet the PWM frequency is given by:
+ *
+ * REFCLK_FREQ
+ * PWM_FREQ = -----------------------------------
+ * PWM_PRE_DIV * BACKLIGHT_SCALE + 1
+ *
+ * However, after careful review the author is convinced that
+ * the documentation has lost some parenthesis around
+ * "BACKLIGHT_SCALE + 1".
+ *
+ * With the period T_pwm = 1/PWM_FREQ this can be written:
+ *
+ * T_pwm * REFCLK_FREQ = PWM_PRE_DIV * (BACKLIGHT_SCALE + 1)
+ *
+ * In order to keep BACKLIGHT_SCALE within its 16 bits,
+ * PWM_PRE_DIV must be:
+ *
+ * T_pwm * REFCLK_FREQ
+ * PWM_PRE_DIV >= -------------------------
+ * BACKLIGHT_SCALE_MAX + 1
+ *
+ * To simplify the search and to favour higher resolution of
+ * the duty cycle over accuracy of the period, the lowest
+ * possible PWM_PRE_DIV is used. Finally the scale is
+ * calculated as:
+ *
+ * T_pwm * REFCLK_FREQ
+ * BACKLIGHT_SCALE = ---------------------- - 1
+ * PWM_PRE_DIV
+ *
+ * Here T_pwm is represented in seconds, so appropriate scaling
+ * to nanoseconds is necessary.
+ */
+
+ /* Minimum T_pwm is 1 / REFCLK_FREQ */
+ if (state->period <= NSEC_PER_SEC / pdata->pwm_refclk_freq) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ /*
+ * Maximum T_pwm is 255 * (65535 + 1) / REFCLK_FREQ
+ * Limit period to this to avoid overflows
+ */
+ period_max = div_u64((u64)NSEC_PER_SEC * 255 * (65535 + 1),
+ pdata->pwm_refclk_freq);
+ period = min(state->period, period_max);
+
+ pre_div = DIV64_U64_ROUND_UP(period * pdata->pwm_refclk_freq,
+ (u64)NSEC_PER_SEC * (BACKLIGHT_SCALE_MAX + 1));
+ scale = div64_u64(period * pdata->pwm_refclk_freq, (u64)NSEC_PER_SEC * pre_div) - 1;
+
+ /*
+ * The documentation has the duty ratio given as:
+ *
+ * duty BACKLIGHT
+ * ------- = ---------------------
+ * period BACKLIGHT_SCALE + 1
+ *
+ * Solve for BACKLIGHT, substituting BACKLIGHT_SCALE according
+ * to definition above and adjusting for nanosecond
+ * representation of duty cycle gives us:
+ */
+ backlight = div64_u64(state->duty_cycle * pdata->pwm_refclk_freq,
+ (u64)NSEC_PER_SEC * pre_div);
+ if (backlight > scale)
+ backlight = scale;
+
+ ret = regmap_write(pdata->regmap, SN_PWM_PRE_DIV_REG, pre_div);
+ if (ret) {
+ dev_err(pdata->dev, "failed to update PWM_PRE_DIV\n");
+ goto out;
+ }
+
+ ti_sn65dsi86_write_u16(pdata, SN_BACKLIGHT_SCALE_REG, scale);
+ ti_sn65dsi86_write_u16(pdata, SN_BACKLIGHT_REG, backlight);
+ }
+
+ pwm_en_inv = FIELD_PREP(SN_PWM_EN_MASK, state->enabled) |
+ FIELD_PREP(SN_PWM_INV_MASK, state->polarity == PWM_POLARITY_INVERSED);
+ ret = regmap_write(pdata->regmap, SN_PWM_EN_INV_REG, pwm_en_inv);
+ if (ret) {
+ dev_err(pdata->dev, "failed to update PWM_EN/PWM_INV\n");
+ goto out;
+ }
+
+ pdata->pwm_enabled = state->enabled;
+out:
+
+ if (!pdata->pwm_enabled)
+ pm_runtime_put_sync(pdata->dev);
+
+ return ret;
+}
+
+static void ti_sn_pwm_get_state(struct pwm_chip *chip, struct pwm_device *pwm,
+ struct pwm_state *state)
+{
+ struct ti_sn65dsi86 *pdata = pwm_chip_to_ti_sn_bridge(chip);
+ unsigned int pwm_en_inv;
+ unsigned int pre_div;
+ u16 backlight;
+ u16 scale;
+ int ret;
+
+ ret = regmap_read(pdata->regmap, SN_PWM_EN_INV_REG, &pwm_en_inv);
+ if (ret)
+ return;
+
+ ret = ti_sn65dsi86_read_u16(pdata, SN_BACKLIGHT_SCALE_REG, &scale);
+ if (ret)
+ return;
+
+ ret = ti_sn65dsi86_read_u16(pdata, SN_BACKLIGHT_REG, &backlight);
+ if (ret)
+ return;
+
+ ret = regmap_read(pdata->regmap, SN_PWM_PRE_DIV_REG, &pre_div);
+ if (ret)
+ return;
+
+ state->enabled = FIELD_GET(SN_PWM_EN_MASK, pwm_en_inv);
+ if (FIELD_GET(SN_PWM_INV_MASK, pwm_en_inv))
+ state->polarity = PWM_POLARITY_INVERSED;
+ else
+ state->polarity = PWM_POLARITY_NORMAL;
+
+ state->period = DIV_ROUND_UP_ULL((u64)NSEC_PER_SEC * pre_div * (scale + 1),
+ pdata->pwm_refclk_freq);
+ state->duty_cycle = DIV_ROUND_UP_ULL((u64)NSEC_PER_SEC * pre_div * backlight,
+ pdata->pwm_refclk_freq);
+
+ if (state->duty_cycle > state->period)
+ state->duty_cycle = state->period;
+}
+
+static const struct pwm_ops ti_sn_pwm_ops = {
+ .request = ti_sn_pwm_request,
+ .free = ti_sn_pwm_free,
+ .apply = ti_sn_pwm_apply,
+ .get_state = ti_sn_pwm_get_state,
+ .owner = THIS_MODULE,
+};
+
+static int ti_sn_pwm_probe(struct auxiliary_device *adev,
+ const struct auxiliary_device_id *id)
+{
+ struct ti_sn65dsi86 *pdata = dev_get_drvdata(adev->dev.parent);
+
+ pdata->pchip.dev = pdata->dev;
+ pdata->pchip.ops = &ti_sn_pwm_ops;
+ pdata->pchip.npwm = 1;
+ pdata->pchip.of_xlate = of_pwm_single_xlate;
+ pdata->pchip.of_pwm_n_cells = 1;
+
+ return pwmchip_add(&pdata->pchip);
+}
+
+static void ti_sn_pwm_remove(struct auxiliary_device *adev)
+{
+ struct ti_sn65dsi86 *pdata = dev_get_drvdata(adev->dev.parent);
+
+ pwmchip_remove(&pdata->pchip);
+
+ if (pdata->pwm_enabled)
+ pm_runtime_put_sync(pdata->dev);
+}
+
+static const struct auxiliary_device_id ti_sn_pwm_id_table[] = {
+ { .name = "ti_sn65dsi86.pwm", },
+ {},
+};
+
+static struct auxiliary_driver ti_sn_pwm_driver = {
+ .name = "pwm",
+ .probe = ti_sn_pwm_probe,
+ .remove = ti_sn_pwm_remove,
+ .id_table = ti_sn_pwm_id_table,
+};
+
+static int __init ti_sn_pwm_register(void)
+{
+ return auxiliary_driver_register(&ti_sn_pwm_driver);
+}
+
+static void ti_sn_pwm_unregister(void)
+{
+ auxiliary_driver_unregister(&ti_sn_pwm_driver);
+}
+
+#else
+static inline int ti_sn_pwm_pin_request(struct ti_sn65dsi86 *pdata) { return 0; }
+static inline void ti_sn_pwm_pin_release(struct ti_sn65dsi86 *pdata) {}
+
+static inline int ti_sn_pwm_register(void) { return 0; }
+static inline void ti_sn_pwm_unregister(void) {}
+#endif
+
+/* -----------------------------------------------------------------------------
* GPIO Controller
*/
-
#if defined(CONFIG_OF_GPIO)
static int tn_sn_bridge_of_xlate(struct gpio_chip *chip,
@@ -1395,10 +1708,25 @@ static int ti_sn_bridge_gpio_direction_output(struct gpio_chip *chip,
return ret;
}
+static int ti_sn_bridge_gpio_request(struct gpio_chip *chip, unsigned int offset)
+{
+ struct ti_sn65dsi86 *pdata = gpiochip_get_data(chip);
+
+ if (offset == SN_PWM_GPIO_IDX)
+ return ti_sn_pwm_pin_request(pdata);
+
+ return 0;
+}
+
static void ti_sn_bridge_gpio_free(struct gpio_chip *chip, unsigned int offset)
{
+ struct ti_sn65dsi86 *pdata = gpiochip_get_data(chip);
+
/* We won't keep pm_runtime if we're input, so switch there on free */
ti_sn_bridge_gpio_direction_input(chip, offset);
+
+ if (offset == SN_PWM_GPIO_IDX)
+ ti_sn_pwm_pin_release(pdata);
}
static const char * const ti_sn_bridge_gpio_names[SN_NUM_GPIOS] = {
@@ -1420,6 +1748,7 @@ static int ti_sn_gpio_probe(struct auxiliary_device *adev,
pdata->gchip.owner = THIS_MODULE;
pdata->gchip.of_xlate = tn_sn_bridge_of_xlate;
pdata->gchip.of_gpio_n_cells = 2;
+ pdata->gchip.request = ti_sn_bridge_gpio_request;
pdata->gchip.free = ti_sn_bridge_gpio_free;
pdata->gchip.get_direction = ti_sn_bridge_gpio_get_direction;
pdata->gchip.direction_input = ti_sn_bridge_gpio_direction_input;
@@ -1546,10 +1875,9 @@ static int ti_sn65dsi86_probe(struct i2c_client *client,
* ordering. The bridge wants the panel to be there when it probes.
* The panel wants its HPD GPIO (provided by sn65dsi86 on some boards)
* when it probes. The panel and maybe backlight might want the DDC
- * bus. Soon the PWM provided by the bridge chip will have the same
- * problem. Having sub-devices allows the some sub devices to finish
- * probing even if others return -EPROBE_DEFER and gets us around the
- * problems.
+ * bus or the pwm_chip. Having sub-devices allows the some sub devices
+ * to finish probing even if others return -EPROBE_DEFER and gets us
+ * around the problems.
*/
if (IS_ENABLED(CONFIG_OF_GPIO)) {
@@ -1558,6 +1886,12 @@ static int ti_sn65dsi86_probe(struct i2c_client *client,
return ret;
}
+ if (IS_ENABLED(CONFIG_PWM)) {
+ ret = ti_sn65dsi86_add_aux_device(pdata, &pdata->pwm_aux, "pwm");
+ if (ret)
+ return ret;
+ }
+
/*
* NOTE: At the end of the AUX channel probe we'll add the aux device
* for the bridge. This is because the bridge can't be used until the
@@ -1601,10 +1935,14 @@ static int __init ti_sn65dsi86_init(void)
if (ret)
goto err_main_was_registered;
- ret = auxiliary_driver_register(&ti_sn_aux_driver);
+ ret = ti_sn_pwm_register();
if (ret)
goto err_gpio_was_registered;
+ ret = auxiliary_driver_register(&ti_sn_aux_driver);
+ if (ret)
+ goto err_pwm_was_registered;
+
ret = auxiliary_driver_register(&ti_sn_bridge_driver);
if (ret)
goto err_aux_was_registered;
@@ -1613,6 +1951,8 @@ static int __init ti_sn65dsi86_init(void)
err_aux_was_registered:
auxiliary_driver_unregister(&ti_sn_aux_driver);
+err_pwm_was_registered:
+ ti_sn_pwm_unregister();
err_gpio_was_registered:
ti_sn_gpio_unregister();
err_main_was_registered:
@@ -1626,6 +1966,7 @@ static void __exit ti_sn65dsi86_exit(void)
{
auxiliary_driver_unregister(&ti_sn_bridge_driver);
auxiliary_driver_unregister(&ti_sn_aux_driver);
+ ti_sn_pwm_unregister();
ti_sn_gpio_unregister();
i2c_del_driver(&ti_sn65dsi86_driver);
}
diff --git a/drivers/gpu/drm/drm_atomic.c b/drivers/gpu/drm/drm_atomic.c
index ff1416c..21174ef 100644
--- a/drivers/gpu/drm/drm_atomic.c
+++ b/drivers/gpu/drm/drm_atomic.c
@@ -74,7 +74,7 @@ int drm_crtc_commit_wait(struct drm_crtc_commit *commit)
ret = wait_for_completion_timeout(&commit->hw_done, timeout);
if (!ret) {
- DRM_ERROR("hw_done timed out\n");
+ drm_err(commit->crtc->dev, "hw_done timed out\n");
return -ETIMEDOUT;
}
@@ -84,7 +84,7 @@ int drm_crtc_commit_wait(struct drm_crtc_commit *commit)
*/
ret = wait_for_completion_timeout(&commit->flip_done, timeout);
if (!ret) {
- DRM_ERROR("flip_done timed out\n");
+ drm_err(commit->crtc->dev, "flip_done timed out\n");
return -ETIMEDOUT;
}
@@ -140,7 +140,7 @@ drm_atomic_state_init(struct drm_device *dev, struct drm_atomic_state *state)
state->dev = dev;
- DRM_DEBUG_ATOMIC("Allocated atomic state %p\n", state);
+ drm_dbg_atomic(dev, "Allocated atomic state %p\n", state);
return 0;
fail:
@@ -191,7 +191,7 @@ void drm_atomic_state_default_clear(struct drm_atomic_state *state)
struct drm_mode_config *config = &dev->mode_config;
int i;
- DRM_DEBUG_ATOMIC("Clearing atomic state %p\n", state);
+ drm_dbg_atomic(dev, "Clearing atomic state %p\n", state);
for (i = 0; i < state->num_connector; i++) {
struct drm_connector *connector = state->connectors[i].ptr;
@@ -301,7 +301,7 @@ void __drm_atomic_state_free(struct kref *ref)
drm_atomic_state_clear(state);
- DRM_DEBUG_ATOMIC("Freeing atomic state %p\n", state);
+ drm_dbg_atomic(state->dev, "Freeing atomic state %p\n", state);
if (config->funcs->atomic_state_free) {
config->funcs->atomic_state_free(state);
@@ -358,8 +358,8 @@ drm_atomic_get_crtc_state(struct drm_atomic_state *state,
state->crtcs[index].ptr = crtc;
crtc_state->state = state;
- DRM_DEBUG_ATOMIC("Added [CRTC:%d:%s] %p state to %p\n",
- crtc->base.id, crtc->name, crtc_state, state);
+ drm_dbg_atomic(state->dev, "Added [CRTC:%d:%s] %p state to %p\n",
+ crtc->base.id, crtc->name, crtc_state, state);
return crtc_state;
}
@@ -379,8 +379,9 @@ static int drm_atomic_crtc_check(const struct drm_crtc_state *old_crtc_state,
*/
if (new_crtc_state->active && !new_crtc_state->enable) {
- DRM_DEBUG_ATOMIC("[CRTC:%d:%s] active without enabled\n",
- crtc->base.id, crtc->name);
+ drm_dbg_atomic(crtc->dev,
+ "[CRTC:%d:%s] active without enabled\n",
+ crtc->base.id, crtc->name);
return -EINVAL;
}
@@ -390,15 +391,17 @@ static int drm_atomic_crtc_check(const struct drm_crtc_state *old_crtc_state,
*/
if (drm_core_check_feature(crtc->dev, DRIVER_ATOMIC) &&
WARN_ON(new_crtc_state->enable && !new_crtc_state->mode_blob)) {
- DRM_DEBUG_ATOMIC("[CRTC:%d:%s] enabled without mode blob\n",
- crtc->base.id, crtc->name);
+ drm_dbg_atomic(crtc->dev,
+ "[CRTC:%d:%s] enabled without mode blob\n",
+ crtc->base.id, crtc->name);
return -EINVAL;
}
if (drm_core_check_feature(crtc->dev, DRIVER_ATOMIC) &&
WARN_ON(!new_crtc_state->enable && new_crtc_state->mode_blob)) {
- DRM_DEBUG_ATOMIC("[CRTC:%d:%s] disabled with mode blob\n",
- crtc->base.id, crtc->name);
+ drm_dbg_atomic(crtc->dev,
+ "[CRTC:%d:%s] disabled with mode blob\n",
+ crtc->base.id, crtc->name);
return -EINVAL;
}
@@ -414,8 +417,9 @@ static int drm_atomic_crtc_check(const struct drm_crtc_state *old_crtc_state,
*/
if (new_crtc_state->event &&
!new_crtc_state->active && !old_crtc_state->active) {
- DRM_DEBUG_ATOMIC("[CRTC:%d:%s] requesting event but off\n",
- crtc->base.id, crtc->name);
+ drm_dbg_atomic(crtc->dev,
+ "[CRTC:%d:%s] requesting event but off\n",
+ crtc->base.id, crtc->name);
return -EINVAL;
}
@@ -460,8 +464,9 @@ static int drm_atomic_connector_check(struct drm_connector *connector,
return 0;
if (writeback_job->fb && !state->crtc) {
- DRM_DEBUG_ATOMIC("[CONNECTOR:%d:%s] framebuffer without CRTC\n",
- connector->base.id, connector->name);
+ drm_dbg_atomic(connector->dev,
+ "[CONNECTOR:%d:%s] framebuffer without CRTC\n",
+ connector->base.id, connector->name);
return -EINVAL;
}
@@ -470,16 +475,18 @@ static int drm_atomic_connector_check(struct drm_connector *connector,
state->crtc);
if (writeback_job->fb && !crtc_state->active) {
- DRM_DEBUG_ATOMIC("[CONNECTOR:%d:%s] has framebuffer, but [CRTC:%d] is off\n",
- connector->base.id, connector->name,
- state->crtc->base.id);
+ drm_dbg_atomic(connector->dev,
+ "[CONNECTOR:%d:%s] has framebuffer, but [CRTC:%d] is off\n",
+ connector->base.id, connector->name,
+ state->crtc->base.id);
return -EINVAL;
}
if (!writeback_job->fb) {
if (writeback_job->out_fence) {
- DRM_DEBUG_ATOMIC("[CONNECTOR:%d:%s] requesting out-fence without framebuffer\n",
- connector->base.id, connector->name);
+ drm_dbg_atomic(connector->dev,
+ "[CONNECTOR:%d:%s] requesting out-fence without framebuffer\n",
+ connector->base.id, connector->name);
return -EINVAL;
}
@@ -537,8 +544,8 @@ drm_atomic_get_plane_state(struct drm_atomic_state *state,
state->planes[index].new_state = plane_state;
plane_state->state = state;
- DRM_DEBUG_ATOMIC("Added [PLANE:%d:%s] %p state to %p\n",
- plane->base.id, plane->name, plane_state, state);
+ drm_dbg_atomic(plane->dev, "Added [PLANE:%d:%s] %p state to %p\n",
+ plane->base.id, plane->name, plane_state, state);
if (plane_state->crtc) {
struct drm_crtc_state *crtc_state;
@@ -594,12 +601,12 @@ static int drm_atomic_plane_check(const struct drm_plane_state *old_plane_state,
/* either *both* CRTC and FB must be set, or neither */
if (crtc && !fb) {
- DRM_DEBUG_ATOMIC("[PLANE:%d:%s] CRTC set but no FB\n",
- plane->base.id, plane->name);
+ drm_dbg_atomic(plane->dev, "[PLANE:%d:%s] CRTC set but no FB\n",
+ plane->base.id, plane->name);
return -EINVAL;
} else if (fb && !crtc) {
- DRM_DEBUG_ATOMIC("[PLANE:%d:%s] FB set but no CRTC\n",
- plane->base.id, plane->name);
+ drm_dbg_atomic(plane->dev, "[PLANE:%d:%s] FB set but no CRTC\n",
+ plane->base.id, plane->name);
return -EINVAL;
}
@@ -609,9 +616,10 @@ static int drm_atomic_plane_check(const struct drm_plane_state *old_plane_state,
/* Check whether this plane is usable on this CRTC */
if (!(plane->possible_crtcs & drm_crtc_mask(crtc))) {
- DRM_DEBUG_ATOMIC("Invalid [CRTC:%d:%s] for [PLANE:%d:%s]\n",
- crtc->base.id, crtc->name,
- plane->base.id, plane->name);
+ drm_dbg_atomic(plane->dev,
+ "Invalid [CRTC:%d:%s] for [PLANE:%d:%s]\n",
+ crtc->base.id, crtc->name,
+ plane->base.id, plane->name);
return -EINVAL;
}
@@ -619,9 +627,10 @@ static int drm_atomic_plane_check(const struct drm_plane_state *old_plane_state,
ret = drm_plane_check_pixel_format(plane, fb->format->format,
fb->modifier);
if (ret) {
- DRM_DEBUG_ATOMIC("[PLANE:%d:%s] invalid pixel format %p4cc, modifier 0x%llx\n",
- plane->base.id, plane->name,
- &fb->format->format, fb->modifier);
+ drm_dbg_atomic(plane->dev,
+ "[PLANE:%d:%s] invalid pixel format %p4cc, modifier 0x%llx\n",
+ plane->base.id, plane->name,
+ &fb->format->format, fb->modifier);
return ret;
}
@@ -630,10 +639,11 @@ static int drm_atomic_plane_check(const struct drm_plane_state *old_plane_state,
new_plane_state->crtc_x > INT_MAX - (int32_t) new_plane_state->crtc_w ||
new_plane_state->crtc_h > INT_MAX ||
new_plane_state->crtc_y > INT_MAX - (int32_t) new_plane_state->crtc_h) {
- DRM_DEBUG_ATOMIC("[PLANE:%d:%s] invalid CRTC coordinates %ux%u+%d+%d\n",
- plane->base.id, plane->name,
- new_plane_state->crtc_w, new_plane_state->crtc_h,
- new_plane_state->crtc_x, new_plane_state->crtc_y);
+ drm_dbg_atomic(plane->dev,
+ "[PLANE:%d:%s] invalid CRTC coordinates %ux%u+%d+%d\n",
+ plane->base.id, plane->name,
+ new_plane_state->crtc_w, new_plane_state->crtc_h,
+ new_plane_state->crtc_x, new_plane_state->crtc_y);
return -ERANGE;
}
@@ -645,18 +655,19 @@ static int drm_atomic_plane_check(const struct drm_plane_state *old_plane_state,
new_plane_state->src_x > fb_width - new_plane_state->src_w ||
new_plane_state->src_h > fb_height ||
new_plane_state->src_y > fb_height - new_plane_state->src_h) {
- DRM_DEBUG_ATOMIC("[PLANE:%d:%s] invalid source coordinates "
- "%u.%06ux%u.%06u+%u.%06u+%u.%06u (fb %ux%u)\n",
- plane->base.id, plane->name,
- new_plane_state->src_w >> 16,
- ((new_plane_state->src_w & 0xffff) * 15625) >> 10,
- new_plane_state->src_h >> 16,
- ((new_plane_state->src_h & 0xffff) * 15625) >> 10,
- new_plane_state->src_x >> 16,
- ((new_plane_state->src_x & 0xffff) * 15625) >> 10,
- new_plane_state->src_y >> 16,
- ((new_plane_state->src_y & 0xffff) * 15625) >> 10,
- fb->width, fb->height);
+ drm_dbg_atomic(plane->dev,
+ "[PLANE:%d:%s] invalid source coordinates "
+ "%u.%06ux%u.%06u+%u.%06u+%u.%06u (fb %ux%u)\n",
+ plane->base.id, plane->name,
+ new_plane_state->src_w >> 16,
+ ((new_plane_state->src_w & 0xffff) * 15625) >> 10,
+ new_plane_state->src_h >> 16,
+ ((new_plane_state->src_h & 0xffff) * 15625) >> 10,
+ new_plane_state->src_x >> 16,
+ ((new_plane_state->src_x & 0xffff) * 15625) >> 10,
+ new_plane_state->src_y >> 16,
+ ((new_plane_state->src_y & 0xffff) * 15625) >> 10,
+ fb->width, fb->height);
return -ENOSPC;
}
@@ -671,9 +682,10 @@ static int drm_atomic_plane_check(const struct drm_plane_state *old_plane_state,
clips->y1 < 0 ||
clips->x2 > fb_width ||
clips->y2 > fb_height) {
- DRM_DEBUG_ATOMIC("[PLANE:%d:%s] invalid damage clip %d %d %d %d\n",
- plane->base.id, plane->name, clips->x1,
- clips->y1, clips->x2, clips->y2);
+ drm_dbg_atomic(plane->dev,
+ "[PLANE:%d:%s] invalid damage clip %d %d %d %d\n",
+ plane->base.id, plane->name, clips->x1,
+ clips->y1, clips->x2, clips->y2);
return -EINVAL;
}
clips++;
@@ -681,8 +693,9 @@ static int drm_atomic_plane_check(const struct drm_plane_state *old_plane_state,
}
if (plane_switching_crtc(old_plane_state, new_plane_state)) {
- DRM_DEBUG_ATOMIC("[PLANE:%d:%s] switching CRTC directly\n",
- plane->base.id, plane->name);
+ drm_dbg_atomic(plane->dev,
+ "[PLANE:%d:%s] switching CRTC directly\n",
+ plane->base.id, plane->name);
return -EINVAL;
}
@@ -846,8 +859,9 @@ drm_atomic_get_private_obj_state(struct drm_atomic_state *state,
state->num_private_objs = num_objs;
- DRM_DEBUG_ATOMIC("Added new private object %p state %p to %p\n",
- obj, obj_state, state);
+ drm_dbg_atomic(state->dev,
+ "Added new private object %p state %p to %p\n",
+ obj, obj_state, state);
return obj_state;
}
@@ -1027,7 +1041,7 @@ drm_atomic_get_connector_state(struct drm_atomic_state *state,
state->connectors[index].ptr = connector;
connector_state->state = state;
- DRM_DEBUG_ATOMIC("Added [CONNECTOR:%d:%s] %p state to %p\n",
+ drm_dbg_atomic(connector->dev, "Added [CONNECTOR:%d:%s] %p state to %p\n",
connector->base.id, connector->name,
connector_state, state);
@@ -1160,8 +1174,9 @@ drm_atomic_add_encoder_bridges(struct drm_atomic_state *state,
if (!encoder)
return 0;
- DRM_DEBUG_ATOMIC("Adding all bridges for [encoder:%d:%s] to %p\n",
- encoder->base.id, encoder->name, state);
+ drm_dbg_atomic(encoder->dev,
+ "Adding all bridges for [encoder:%d:%s] to %p\n",
+ encoder->base.id, encoder->name, state);
drm_for_each_bridge_in_chain(encoder, bridge) {
/* Skip bridges that don't implement the atomic state hooks. */
@@ -1213,8 +1228,9 @@ drm_atomic_add_affected_connectors(struct drm_atomic_state *state,
if (ret)
return ret;
- DRM_DEBUG_ATOMIC("Adding all current connectors for [CRTC:%d:%s] to %p\n",
- crtc->base.id, crtc->name, state);
+ drm_dbg_atomic(crtc->dev,
+ "Adding all current connectors for [CRTC:%d:%s] to %p\n",
+ crtc->base.id, crtc->name, state);
/*
* Changed connectors are already in @state, so only need to look
@@ -1267,8 +1283,9 @@ drm_atomic_add_affected_planes(struct drm_atomic_state *state,
WARN_ON(!drm_atomic_get_new_crtc_state(state, crtc));
- DRM_DEBUG_ATOMIC("Adding all current planes for [CRTC:%d:%s] to %p\n",
- crtc->base.id, crtc->name, state);
+ drm_dbg_atomic(crtc->dev,
+ "Adding all current planes for [CRTC:%d:%s] to %p\n",
+ crtc->base.id, crtc->name, state);
drm_for_each_plane_mask(plane, state->dev, old_crtc_state->plane_mask) {
struct drm_plane_state *plane_state =
@@ -1308,7 +1325,7 @@ int drm_atomic_check_only(struct drm_atomic_state *state)
unsigned int affected_crtc = 0;
int i, ret = 0;
- DRM_DEBUG_ATOMIC("checking %p\n", state);
+ drm_dbg_atomic(dev, "checking %p\n", state);
for_each_new_crtc_in_state(state, crtc, new_crtc_state, i)
requested_crtc |= drm_crtc_mask(crtc);
@@ -1316,8 +1333,8 @@ int drm_atomic_check_only(struct drm_atomic_state *state)
for_each_oldnew_plane_in_state(state, plane, old_plane_state, new_plane_state, i) {
ret = drm_atomic_plane_check(old_plane_state, new_plane_state);
if (ret) {
- DRM_DEBUG_ATOMIC("[PLANE:%d:%s] atomic core check failed\n",
- plane->base.id, plane->name);
+ drm_dbg_atomic(dev, "[PLANE:%d:%s] atomic core check failed\n",
+ plane->base.id, plane->name);
return ret;
}
}
@@ -1325,8 +1342,8 @@ int drm_atomic_check_only(struct drm_atomic_state *state)
for_each_oldnew_crtc_in_state(state, crtc, old_crtc_state, new_crtc_state, i) {
ret = drm_atomic_crtc_check(old_crtc_state, new_crtc_state);
if (ret) {
- DRM_DEBUG_ATOMIC("[CRTC:%d:%s] atomic core check failed\n",
- crtc->base.id, crtc->name);
+ drm_dbg_atomic(dev, "[CRTC:%d:%s] atomic core check failed\n",
+ crtc->base.id, crtc->name);
return ret;
}
}
@@ -1334,8 +1351,8 @@ int drm_atomic_check_only(struct drm_atomic_state *state)
for_each_new_connector_in_state(state, conn, conn_state, i) {
ret = drm_atomic_connector_check(conn, conn_state);
if (ret) {
- DRM_DEBUG_ATOMIC("[CONNECTOR:%d:%s] atomic core check failed\n",
- conn->base.id, conn->name);
+ drm_dbg_atomic(dev, "[CONNECTOR:%d:%s] atomic core check failed\n",
+ conn->base.id, conn->name);
return ret;
}
}
@@ -1344,8 +1361,8 @@ int drm_atomic_check_only(struct drm_atomic_state *state)
ret = config->funcs->atomic_check(state->dev, state);
if (ret) {
- DRM_DEBUG_ATOMIC("atomic driver check for %p failed: %d\n",
- state, ret);
+ drm_dbg_atomic(dev, "atomic driver check for %p failed: %d\n",
+ state, ret);
return ret;
}
}
@@ -1353,8 +1370,8 @@ int drm_atomic_check_only(struct drm_atomic_state *state)
if (!state->allow_modeset) {
for_each_new_crtc_in_state(state, crtc, new_crtc_state, i) {
if (drm_atomic_crtc_needs_modeset(new_crtc_state)) {
- DRM_DEBUG_ATOMIC("[CRTC:%d:%s] requires full modeset\n",
- crtc->base.id, crtc->name);
+ drm_dbg_atomic(dev, "[CRTC:%d:%s] requires full modeset\n",
+ crtc->base.id, crtc->name);
return -EINVAL;
}
}
@@ -1374,8 +1391,9 @@ int drm_atomic_check_only(struct drm_atomic_state *state)
* so compositors know what's going on.
*/
if (affected_crtc != requested_crtc) {
- DRM_DEBUG_ATOMIC("driver added CRTC to commit: requested 0x%x, affected 0x%0x\n",
- requested_crtc, affected_crtc);
+ drm_dbg_atomic(dev,
+ "driver added CRTC to commit: requested 0x%x, affected 0x%0x\n",
+ requested_crtc, affected_crtc);
WARN(!state->allow_modeset, "adding CRTC not allowed without modesets: requested 0x%x, affected 0x%0x\n",
requested_crtc, affected_crtc);
}
@@ -1407,7 +1425,7 @@ int drm_atomic_commit(struct drm_atomic_state *state)
if (ret)
return ret;
- DRM_DEBUG_ATOMIC("committing %p\n", state);
+ drm_dbg_atomic(state->dev, "committing %p\n", state);
return config->funcs->atomic_commit(state->dev, state, false);
}
@@ -1436,7 +1454,7 @@ int drm_atomic_nonblocking_commit(struct drm_atomic_state *state)
if (ret)
return ret;
- DRM_DEBUG_ATOMIC("committing %p nonblocking\n", state);
+ drm_dbg_atomic(state->dev, "committing %p nonblocking\n", state);
return config->funcs->atomic_commit(state->dev, state, true);
}
@@ -1633,11 +1651,11 @@ void drm_atomic_print_new_state(const struct drm_atomic_state *state,
int i;
if (!p) {
- DRM_ERROR("invalid drm printer\n");
+ drm_err(state->dev, "invalid drm printer\n");
return;
}
- DRM_DEBUG_ATOMIC("checking %p\n", state);
+ drm_dbg_atomic(state->dev, "checking %p\n", state);
for_each_new_plane_in_state(state, plane, plane_state, i)
drm_atomic_plane_print_state(p, plane_state);
diff --git a/drivers/gpu/drm/drm_atomic_helper.c b/drivers/gpu/drm/drm_atomic_helper.c
index 2c0c6ec..aef2fbd 100644
--- a/drivers/gpu/drm/drm_atomic_helper.c
+++ b/drivers/gpu/drm/drm_atomic_helper.c
@@ -132,9 +132,10 @@ static int handle_conflicting_encoders(struct drm_atomic_state *state,
if (new_encoder) {
if (encoder_mask & drm_encoder_mask(new_encoder)) {
- DRM_DEBUG_ATOMIC("[ENCODER:%d:%s] on [CONNECTOR:%d:%s] already assigned\n",
- new_encoder->base.id, new_encoder->name,
- connector->base.id, connector->name);
+ drm_dbg_atomic(connector->dev,
+ "[ENCODER:%d:%s] on [CONNECTOR:%d:%s] already assigned\n",
+ new_encoder->base.id, new_encoder->name,
+ connector->base.id, connector->name);
return -EINVAL;
}
@@ -169,11 +170,12 @@ static int handle_conflicting_encoders(struct drm_atomic_state *state,
continue;
if (!disable_conflicting_encoders) {
- DRM_DEBUG_ATOMIC("[ENCODER:%d:%s] in use on [CRTC:%d:%s] by [CONNECTOR:%d:%s]\n",
- encoder->base.id, encoder->name,
- connector->state->crtc->base.id,
- connector->state->crtc->name,
- connector->base.id, connector->name);
+ drm_dbg_atomic(connector->dev,
+ "[ENCODER:%d:%s] in use on [CRTC:%d:%s] by [CONNECTOR:%d:%s]\n",
+ encoder->base.id, encoder->name,
+ connector->state->crtc->base.id,
+ connector->state->crtc->name,
+ connector->base.id, connector->name);
ret = -EINVAL;
goto out;
}
@@ -184,10 +186,11 @@ static int handle_conflicting_encoders(struct drm_atomic_state *state,
goto out;
}
- DRM_DEBUG_ATOMIC("[ENCODER:%d:%s] in use on [CRTC:%d:%s], disabling [CONNECTOR:%d:%s]\n",
- encoder->base.id, encoder->name,
- new_conn_state->crtc->base.id, new_conn_state->crtc->name,
- connector->base.id, connector->name);
+ drm_dbg_atomic(connector->dev,
+ "[ENCODER:%d:%s] in use on [CRTC:%d:%s], disabling [CONNECTOR:%d:%s]\n",
+ encoder->base.id, encoder->name,
+ new_conn_state->crtc->base.id, new_conn_state->crtc->name,
+ connector->base.id, connector->name);
crtc_state = drm_atomic_get_new_crtc_state(state, new_conn_state->crtc);
@@ -268,9 +271,10 @@ steal_encoder(struct drm_atomic_state *state,
encoder_crtc = old_connector_state->crtc;
- DRM_DEBUG_ATOMIC("[ENCODER:%d:%s] in use on [CRTC:%d:%s], stealing it\n",
- encoder->base.id, encoder->name,
- encoder_crtc->base.id, encoder_crtc->name);
+ drm_dbg_atomic(encoder->dev,
+ "[ENCODER:%d:%s] in use on [CRTC:%d:%s], stealing it\n",
+ encoder->base.id, encoder->name,
+ encoder_crtc->base.id, encoder_crtc->name);
set_best_encoder(state, new_connector_state, NULL);
@@ -291,9 +295,8 @@ update_connector_routing(struct drm_atomic_state *state,
struct drm_encoder *new_encoder;
struct drm_crtc_state *crtc_state;
- DRM_DEBUG_ATOMIC("Updating routing for [CONNECTOR:%d:%s]\n",
- connector->base.id,
- connector->name);
+ drm_dbg_atomic(connector->dev, "Updating routing for [CONNECTOR:%d:%s]\n",
+ connector->base.id, connector->name);
if (old_connector_state->crtc != new_connector_state->crtc) {
if (old_connector_state->crtc) {
@@ -308,9 +311,8 @@ update_connector_routing(struct drm_atomic_state *state,
}
if (!new_connector_state->crtc) {
- DRM_DEBUG_ATOMIC("Disabling [CONNECTOR:%d:%s]\n",
- connector->base.id,
- connector->name);
+ drm_dbg_atomic(connector->dev, "Disabling [CONNECTOR:%d:%s]\n",
+ connector->base.id, connector->name);
set_best_encoder(state, new_connector_state, NULL);
@@ -339,8 +341,9 @@ update_connector_routing(struct drm_atomic_state *state,
*/
if (!state->duplicated && drm_connector_is_unregistered(connector) &&
crtc_state->active) {
- DRM_DEBUG_ATOMIC("[CONNECTOR:%d:%s] is not registered\n",
- connector->base.id, connector->name);
+ drm_dbg_atomic(connector->dev,
+ "[CONNECTOR:%d:%s] is not registered\n",
+ connector->base.id, connector->name);
return -EINVAL;
}
@@ -354,31 +357,33 @@ update_connector_routing(struct drm_atomic_state *state,
new_encoder = drm_connector_get_single_encoder(connector);
if (!new_encoder) {
- DRM_DEBUG_ATOMIC("No suitable encoder found for [CONNECTOR:%d:%s]\n",
- connector->base.id,
- connector->name);
+ drm_dbg_atomic(connector->dev,
+ "No suitable encoder found for [CONNECTOR:%d:%s]\n",
+ connector->base.id, connector->name);
return -EINVAL;
}
if (!drm_encoder_crtc_ok(new_encoder, new_connector_state->crtc)) {
- DRM_DEBUG_ATOMIC("[ENCODER:%d:%s] incompatible with [CRTC:%d:%s]\n",
- new_encoder->base.id,
- new_encoder->name,
- new_connector_state->crtc->base.id,
- new_connector_state->crtc->name);
+ drm_dbg_atomic(connector->dev,
+ "[ENCODER:%d:%s] incompatible with [CRTC:%d:%s]\n",
+ new_encoder->base.id,
+ new_encoder->name,
+ new_connector_state->crtc->base.id,
+ new_connector_state->crtc->name);
return -EINVAL;
}
if (new_encoder == new_connector_state->best_encoder) {
set_best_encoder(state, new_connector_state, new_encoder);
- DRM_DEBUG_ATOMIC("[CONNECTOR:%d:%s] keeps [ENCODER:%d:%s], now on [CRTC:%d:%s]\n",
- connector->base.id,
- connector->name,
- new_encoder->base.id,
- new_encoder->name,
- new_connector_state->crtc->base.id,
- new_connector_state->crtc->name);
+ drm_dbg_atomic(connector->dev,
+ "[CONNECTOR:%d:%s] keeps [ENCODER:%d:%s], now on [CRTC:%d:%s]\n",
+ connector->base.id,
+ connector->name,
+ new_encoder->base.id,
+ new_encoder->name,
+ new_connector_state->crtc->base.id,
+ new_connector_state->crtc->name);
return 0;
}
@@ -389,13 +394,14 @@ update_connector_routing(struct drm_atomic_state *state,
crtc_state->connectors_changed = true;
- DRM_DEBUG_ATOMIC("[CONNECTOR:%d:%s] using [ENCODER:%d:%s] on [CRTC:%d:%s]\n",
- connector->base.id,
- connector->name,
- new_encoder->base.id,
- new_encoder->name,
- new_connector_state->crtc->base.id,
- new_connector_state->crtc->name);
+ drm_dbg_atomic(connector->dev,
+ "[CONNECTOR:%d:%s] using [ENCODER:%d:%s] on [CRTC:%d:%s]\n",
+ connector->base.id,
+ connector->name,
+ new_encoder->base.id,
+ new_encoder->name,
+ new_connector_state->crtc->base.id,
+ new_connector_state->crtc->name);
return 0;
}
@@ -443,7 +449,7 @@ mode_fixup(struct drm_atomic_state *state)
new_crtc_state,
new_conn_state);
if (ret) {
- DRM_DEBUG_ATOMIC("Bridge atomic check failed\n");
+ drm_dbg_atomic(encoder->dev, "Bridge atomic check failed\n");
return ret;
}
@@ -451,16 +457,18 @@ mode_fixup(struct drm_atomic_state *state)
ret = funcs->atomic_check(encoder, new_crtc_state,
new_conn_state);
if (ret) {
- DRM_DEBUG_ATOMIC("[ENCODER:%d:%s] check failed\n",
- encoder->base.id, encoder->name);
+ drm_dbg_atomic(encoder->dev,
+ "[ENCODER:%d:%s] check failed\n",
+ encoder->base.id, encoder->name);
return ret;
}
} else if (funcs && funcs->mode_fixup) {
ret = funcs->mode_fixup(encoder, &new_crtc_state->mode,
&new_crtc_state->adjusted_mode);
if (!ret) {
- DRM_DEBUG_ATOMIC("[ENCODER:%d:%s] fixup failed\n",
- encoder->base.id, encoder->name);
+ drm_dbg_atomic(encoder->dev,
+ "[ENCODER:%d:%s] fixup failed\n",
+ encoder->base.id, encoder->name);
return -EINVAL;
}
}
@@ -483,8 +491,8 @@ mode_fixup(struct drm_atomic_state *state)
ret = funcs->mode_fixup(crtc, &new_crtc_state->mode,
&new_crtc_state->adjusted_mode);
if (!ret) {
- DRM_DEBUG_ATOMIC("[CRTC:%d:%s] fixup failed\n",
- crtc->base.id, crtc->name);
+ drm_dbg_atomic(crtc->dev, "[CRTC:%d:%s] fixup failed\n",
+ crtc->base.id, crtc->name);
return -EINVAL;
}
}
@@ -502,8 +510,9 @@ static enum drm_mode_status mode_valid_path(struct drm_connector *connector,
ret = drm_encoder_mode_valid(encoder, mode);
if (ret != MODE_OK) {
- DRM_DEBUG_ATOMIC("[ENCODER:%d:%s] mode_valid() failed\n",
- encoder->base.id, encoder->name);
+ drm_dbg_atomic(encoder->dev,
+ "[ENCODER:%d:%s] mode_valid() failed\n",
+ encoder->base.id, encoder->name);
return ret;
}
@@ -511,14 +520,14 @@ static enum drm_mode_status mode_valid_path(struct drm_connector *connector,
ret = drm_bridge_chain_mode_valid(bridge, &connector->display_info,
mode);
if (ret != MODE_OK) {
- DRM_DEBUG_ATOMIC("[BRIDGE] mode_valid() failed\n");
+ drm_dbg_atomic(encoder->dev, "[BRIDGE] mode_valid() failed\n");
return ret;
}
ret = drm_crtc_mode_valid(crtc, mode);
if (ret != MODE_OK) {
- DRM_DEBUG_ATOMIC("[CRTC:%d:%s] mode_valid() failed\n",
- crtc->base.id, crtc->name);
+ drm_dbg_atomic(encoder->dev, "[CRTC:%d:%s] mode_valid() failed\n",
+ crtc->base.id, crtc->name);
return ret;
}
@@ -619,14 +628,14 @@ drm_atomic_helper_check_modeset(struct drm_device *dev,
WARN_ON(!drm_modeset_is_locked(&crtc->mutex));
if (!drm_mode_equal(&old_crtc_state->mode, &new_crtc_state->mode)) {
- DRM_DEBUG_ATOMIC("[CRTC:%d:%s] mode changed\n",
- crtc->base.id, crtc->name);
+ drm_dbg_atomic(dev, "[CRTC:%d:%s] mode changed\n",
+ crtc->base.id, crtc->name);
new_crtc_state->mode_changed = true;
}
if (old_crtc_state->enable != new_crtc_state->enable) {
- DRM_DEBUG_ATOMIC("[CRTC:%d:%s] enable changed\n",
- crtc->base.id, crtc->name);
+ drm_dbg_atomic(dev, "[CRTC:%d:%s] enable changed\n",
+ crtc->base.id, crtc->name);
/*
* For clarity this assignment is done here, but
@@ -641,14 +650,14 @@ drm_atomic_helper_check_modeset(struct drm_device *dev,
}
if (old_crtc_state->active != new_crtc_state->active) {
- DRM_DEBUG_ATOMIC("[CRTC:%d:%s] active changed\n",
- crtc->base.id, crtc->name);
+ drm_dbg_atomic(dev, "[CRTC:%d:%s] active changed\n",
+ crtc->base.id, crtc->name);
new_crtc_state->active_changed = true;
}
if (new_crtc_state->enable != has_connectors) {
- DRM_DEBUG_ATOMIC("[CRTC:%d:%s] enabled/connectors mismatch\n",
- crtc->base.id, crtc->name);
+ drm_dbg_atomic(dev, "[CRTC:%d:%s] enabled/connectors mismatch\n",
+ crtc->base.id, crtc->name);
return -EINVAL;
}
@@ -708,10 +717,11 @@ drm_atomic_helper_check_modeset(struct drm_device *dev,
if (!drm_atomic_crtc_needs_modeset(new_crtc_state))
continue;
- DRM_DEBUG_ATOMIC("[CRTC:%d:%s] needs all connectors, enable: %c, active: %c\n",
- crtc->base.id, crtc->name,
- new_crtc_state->enable ? 'y' : 'n',
- new_crtc_state->active ? 'y' : 'n');
+ drm_dbg_atomic(dev,
+ "[CRTC:%d:%s] needs all connectors, enable: %c, active: %c\n",
+ crtc->base.id, crtc->name,
+ new_crtc_state->enable ? 'y' : 'n',
+ new_crtc_state->active ? 'y' : 'n');
ret = drm_atomic_add_affected_connectors(state, crtc);
if (ret != 0)
@@ -818,7 +828,8 @@ int drm_atomic_helper_check_plane_state(struct drm_plane_state *plane_state,
}
if (!crtc_state->enable && !can_update_disabled) {
- DRM_DEBUG_KMS("Cannot update plane of a disabled CRTC.\n");
+ drm_dbg_kms(plane_state->crtc->dev,
+ "Cannot update plane of a disabled CRTC.\n");
return -EINVAL;
}
@@ -828,7 +839,8 @@ int drm_atomic_helper_check_plane_state(struct drm_plane_state *plane_state,
hscale = drm_rect_calc_hscale(src, dst, min_scale, max_scale);
vscale = drm_rect_calc_vscale(src, dst, min_scale, max_scale);
if (hscale < 0 || vscale < 0) {
- DRM_DEBUG_KMS("Invalid scaling of plane\n");
+ drm_dbg_kms(plane_state->crtc->dev,
+ "Invalid scaling of plane\n");
drm_rect_debug_print("src: ", &plane_state->src, true);
drm_rect_debug_print("dst: ", &plane_state->dst, false);
return -ERANGE;
@@ -852,7 +864,8 @@ int drm_atomic_helper_check_plane_state(struct drm_plane_state *plane_state,
return 0;
if (!can_position && !drm_rect_equals(dst, &clip)) {
- DRM_DEBUG_KMS("Plane must cover entire CRTC\n");
+ drm_dbg_kms(plane_state->crtc->dev,
+ "Plane must cover entire CRTC\n");
drm_rect_debug_print("dst: ", dst, false);
drm_rect_debug_print("clip: ", &clip, false);
return -EINVAL;
@@ -904,8 +917,9 @@ drm_atomic_helper_check_planes(struct drm_device *dev,
ret = funcs->atomic_check(plane, state);
if (ret) {
- DRM_DEBUG_ATOMIC("[PLANE:%d:%s] atomic driver check failed\n",
- plane->base.id, plane->name);
+ drm_dbg_atomic(plane->dev,
+ "[PLANE:%d:%s] atomic driver check failed\n",
+ plane->base.id, plane->name);
return ret;
}
}
@@ -920,8 +934,9 @@ drm_atomic_helper_check_planes(struct drm_device *dev,
ret = funcs->atomic_check(crtc, state);
if (ret) {
- DRM_DEBUG_ATOMIC("[CRTC:%d:%s] atomic driver check failed\n",
- crtc->base.id, crtc->name);
+ drm_dbg_atomic(crtc->dev,
+ "[CRTC:%d:%s] atomic driver check failed\n",
+ crtc->base.id, crtc->name);
return ret;
}
}
@@ -1049,8 +1064,8 @@ disable_outputs(struct drm_device *dev, struct drm_atomic_state *old_state)
funcs = encoder->helper_private;
- DRM_DEBUG_ATOMIC("disabling [ENCODER:%d:%s]\n",
- encoder->base.id, encoder->name);
+ drm_dbg_atomic(dev, "disabling [ENCODER:%d:%s]\n",
+ encoder->base.id, encoder->name);
/*
* Each encoder has at most one connector (since we always steal
@@ -1087,8 +1102,8 @@ disable_outputs(struct drm_device *dev, struct drm_atomic_state *old_state)
funcs = crtc->helper_private;
- DRM_DEBUG_ATOMIC("disabling [CRTC:%d:%s]\n",
- crtc->base.id, crtc->name);
+ drm_dbg_atomic(dev, "disabling [CRTC:%d:%s]\n",
+ crtc->base.id, crtc->name);
/* Right function depends upon target state. */
@@ -1229,8 +1244,8 @@ crtc_set_mode(struct drm_device *dev, struct drm_atomic_state *old_state)
funcs = crtc->helper_private;
if (new_crtc_state->enable && funcs->mode_set_nofb) {
- DRM_DEBUG_ATOMIC("modeset on [CRTC:%d:%s]\n",
- crtc->base.id, crtc->name);
+ drm_dbg_atomic(dev, "modeset on [CRTC:%d:%s]\n",
+ crtc->base.id, crtc->name);
funcs->mode_set_nofb(crtc);
}
@@ -1254,8 +1269,8 @@ crtc_set_mode(struct drm_device *dev, struct drm_atomic_state *old_state)
if (!new_crtc_state->mode_changed)
continue;
- DRM_DEBUG_ATOMIC("modeset on [ENCODER:%d:%s]\n",
- encoder->base.id, encoder->name);
+ drm_dbg_atomic(dev, "modeset on [ENCODER:%d:%s]\n",
+ encoder->base.id, encoder->name);
/*
* Each encoder has at most one connector (since we always steal
@@ -1357,8 +1372,8 @@ void drm_atomic_helper_commit_modeset_enables(struct drm_device *dev,
funcs = crtc->helper_private;
if (new_crtc_state->enable) {
- DRM_DEBUG_ATOMIC("enabling [CRTC:%d:%s]\n",
- crtc->base.id, crtc->name);
+ drm_dbg_atomic(dev, "enabling [CRTC:%d:%s]\n",
+ crtc->base.id, crtc->name);
if (funcs->atomic_enable)
funcs->atomic_enable(crtc, old_state);
else if (funcs->commit)
@@ -1381,8 +1396,8 @@ void drm_atomic_helper_commit_modeset_enables(struct drm_device *dev,
encoder = new_conn_state->best_encoder;
funcs = encoder->helper_private;
- DRM_DEBUG_ATOMIC("enabling [ENCODER:%d:%s]\n",
- encoder->base.id, encoder->name);
+ drm_dbg_atomic(dev, "enabling [ENCODER:%d:%s]\n",
+ encoder->base.id, encoder->name);
/*
* Each encoder has at most one connector (since we always steal
@@ -1551,8 +1566,8 @@ void drm_atomic_helper_wait_for_flip_done(struct drm_device *dev,
ret = wait_for_completion_timeout(&commit->flip_done, 10 * HZ);
if (ret == 0)
- DRM_ERROR("[CRTC:%d:%s] flip_done timed out\n",
- crtc->base.id, crtc->name);
+ drm_err(dev, "[CRTC:%d:%s] flip_done timed out\n",
+ crtc->base.id, crtc->name);
}
if (old_state->fake_commit)
@@ -1739,8 +1754,9 @@ int drm_atomic_helper_async_check(struct drm_device *dev,
*/
if (old_plane_state->commit &&
!try_wait_for_completion(&old_plane_state->commit->hw_done)) {
- DRM_DEBUG_ATOMIC("[PLANE:%d:%s] inflight previous commit preventing async commit\n",
- plane->base.id, plane->name);
+ drm_dbg_atomic(dev,
+ "[PLANE:%d:%s] inflight previous commit preventing async commit\n",
+ plane->base.id, plane->name);
return -EBUSY;
}
@@ -1962,8 +1978,9 @@ static int stall_checks(struct drm_crtc *crtc, bool nonblock)
*/
if (!completed && nonblock) {
spin_unlock(&crtc->commit_lock);
- DRM_DEBUG_ATOMIC("[CRTC:%d:%s] busy with a previous commit\n",
- crtc->base.id, crtc->name);
+ drm_dbg_atomic(crtc->dev,
+ "[CRTC:%d:%s] busy with a previous commit\n",
+ crtc->base.id, crtc->name);
return -EBUSY;
}
@@ -1985,8 +2002,8 @@ static int stall_checks(struct drm_crtc *crtc, bool nonblock)
ret = wait_for_completion_interruptible_timeout(&stall_commit->cleanup_done,
10*HZ);
if (ret == 0)
- DRM_ERROR("[CRTC:%d:%s] cleanup_done timed out\n",
- crtc->base.id, crtc->name);
+ drm_err(crtc->dev, "[CRTC:%d:%s] cleanup_done timed out\n",
+ crtc->base.id, crtc->name);
drm_crtc_commit_put(stall_commit);
@@ -2150,8 +2167,9 @@ int drm_atomic_helper_setup_commit(struct drm_atomic_state *state,
*/
if (nonblock && old_conn_state->commit &&
!try_wait_for_completion(&old_conn_state->commit->flip_done)) {
- DRM_DEBUG_ATOMIC("[CONNECTOR:%d:%s] busy with a previous commit\n",
- conn->base.id, conn->name);
+ drm_dbg_atomic(conn->dev,
+ "[CONNECTOR:%d:%s] busy with a previous commit\n",
+ conn->base.id, conn->name);
return -EBUSY;
}
@@ -2171,8 +2189,9 @@ int drm_atomic_helper_setup_commit(struct drm_atomic_state *state,
*/
if (nonblock && old_plane_state->commit &&
!try_wait_for_completion(&old_plane_state->commit->flip_done)) {
- DRM_DEBUG_ATOMIC("[PLANE:%d:%s] busy with a previous commit\n",
- plane->base.id, plane->name);
+ drm_dbg_atomic(plane->dev,
+ "[PLANE:%d:%s] busy with a previous commit\n",
+ plane->base.id, plane->name);
return -EBUSY;
}
@@ -2218,22 +2237,25 @@ void drm_atomic_helper_wait_for_dependencies(struct drm_atomic_state *old_state)
for_each_old_crtc_in_state(old_state, crtc, old_crtc_state, i) {
ret = drm_crtc_commit_wait(old_crtc_state->commit);
if (ret)
- DRM_ERROR("[CRTC:%d:%s] commit wait timed out\n",
- crtc->base.id, crtc->name);
+ drm_err(crtc->dev,
+ "[CRTC:%d:%s] commit wait timed out\n",
+ crtc->base.id, crtc->name);
}
for_each_old_connector_in_state(old_state, conn, old_conn_state, i) {
ret = drm_crtc_commit_wait(old_conn_state->commit);
if (ret)
- DRM_ERROR("[CONNECTOR:%d:%s] commit wait timed out\n",
- conn->base.id, conn->name);
+ drm_err(conn->dev,
+ "[CONNECTOR:%d:%s] commit wait timed out\n",
+ conn->base.id, conn->name);
}
for_each_old_plane_in_state(old_state, plane, old_plane_state, i) {
ret = drm_crtc_commit_wait(old_plane_state->commit);
if (ret)
- DRM_ERROR("[PLANE:%d:%s] commit wait timed out\n",
- plane->base.id, plane->name);
+ drm_err(plane->dev,
+ "[PLANE:%d:%s] commit wait timed out\n",
+ plane->base.id, plane->name);
}
}
EXPORT_SYMBOL(drm_atomic_helper_wait_for_dependencies);
@@ -3120,7 +3142,9 @@ void drm_atomic_helper_shutdown(struct drm_device *dev)
ret = drm_atomic_helper_disable_all(dev, &ctx);
if (ret)
- DRM_ERROR("Disabling all crtc's during unload failed with %i\n", ret);
+ drm_err(dev,
+ "Disabling all crtc's during unload failed with %i\n",
+ ret);
DRM_MODESET_LOCK_ALL_END(dev, ctx, ret);
}
@@ -3380,8 +3404,9 @@ static int page_flip_common(struct drm_atomic_state *state,
/* Make sure we don't accidentally do a full modeset. */
state->allow_modeset = false;
if (!crtc_state->active) {
- DRM_DEBUG_ATOMIC("[CRTC:%d:%s] disabled, rejecting legacy flip\n",
- crtc->base.id, crtc->name);
+ drm_dbg_atomic(crtc->dev,
+ "[CRTC:%d:%s] disabled, rejecting legacy flip\n",
+ crtc->base.id, crtc->name);
return -EINVAL;
}
diff --git a/drivers/gpu/drm/drm_atomic_uapi.c b/drivers/gpu/drm/drm_atomic_uapi.c
index 909f318..9781722 100644
--- a/drivers/gpu/drm/drm_atomic_uapi.c
+++ b/drivers/gpu/drm/drm_atomic_uapi.c
@@ -773,7 +773,7 @@ static int drm_atomic_connector_set_property(struct drm_connector *connector,
state->scaling_mode = val;
} else if (property == config->content_protection_property) {
if (val == DRM_MODE_CONTENT_PROTECTION_ENABLED) {
- DRM_DEBUG_KMS("only drivers can set CP Enabled\n");
+ drm_dbg_kms(dev, "only drivers can set CP Enabled\n");
return -EINVAL;
}
state->content_protection = val;
@@ -797,6 +797,8 @@ static int drm_atomic_connector_set_property(struct drm_connector *connector,
fence_ptr);
} else if (property == connector->max_bpc_property) {
state->max_requested_bpc = val;
+ } else if (property == connector->privacy_screen_sw_state_property) {
+ state->privacy_screen_sw_state = val;
} else if (connector->funcs->atomic_set_property) {
return connector->funcs->atomic_set_property(connector,
state, property, val);
@@ -874,6 +876,8 @@ drm_atomic_connector_get_property(struct drm_connector *connector,
*val = 0;
} else if (property == connector->max_bpc_property) {
*val = state->max_requested_bpc;
+ } else if (property == connector->privacy_screen_sw_state_property) {
+ *val = state->privacy_screen_sw_state;
} else if (connector->funcs->atomic_get_property) {
return connector->funcs->atomic_get_property(connector,
state, property, val);
diff --git a/drivers/gpu/drm/drm_auth.c b/drivers/gpu/drm/drm_auth.c
index 60a6b21..6e433d4 100644
--- a/drivers/gpu/drm/drm_auth.c
+++ b/drivers/gpu/drm/drm_auth.c
@@ -106,7 +106,7 @@ int drm_getmagic(struct drm_device *dev, void *data, struct drm_file *file_priv)
auth->magic = file_priv->magic;
mutex_unlock(&dev->master_mutex);
- DRM_DEBUG("%u\n", auth->magic);
+ drm_dbg_core(dev, "%u\n", auth->magic);
return ret < 0 ? ret : 0;
}
@@ -117,7 +117,7 @@ int drm_authmagic(struct drm_device *dev, void *data,
struct drm_auth *auth = data;
struct drm_file *file;
- DRM_DEBUG("%u\n", auth->magic);
+ drm_dbg_core(dev, "%u\n", auth->magic);
mutex_lock(&dev->master_mutex);
file = idr_find(&file_priv->master->magic_map, auth->magic);
@@ -274,7 +274,9 @@ int drm_setmaster_ioctl(struct drm_device *dev, void *data,
}
if (file_priv->master->lessor != NULL) {
- DRM_DEBUG_LEASE("Attempt to set lessee %d as master\n", file_priv->master->lessee_id);
+ drm_dbg_lease(dev,
+ "Attempt to set lessee %d as master\n",
+ file_priv->master->lessee_id);
ret = -EINVAL;
goto out_unlock;
}
@@ -315,7 +317,9 @@ int drm_dropmaster_ioctl(struct drm_device *dev, void *data,
}
if (file_priv->master->lessor != NULL) {
- DRM_DEBUG_LEASE("Attempt to drop lessee %d as master\n", file_priv->master->lessee_id);
+ drm_dbg_lease(dev,
+ "Attempt to drop lessee %d as master\n",
+ file_priv->master->lessee_id);
ret = -EINVAL;
goto out_unlock;
}
diff --git a/drivers/gpu/drm/drm_connector.c b/drivers/gpu/drm/drm_connector.c
index 52e20c6..a50c82b 100644
--- a/drivers/gpu/drm/drm_connector.c
+++ b/drivers/gpu/drm/drm_connector.c
@@ -28,6 +28,7 @@
#include <drm/drm_print.h>
#include <drm/drm_drv.h>
#include <drm/drm_file.h>
+#include <drm/drm_privacy_screen_consumer.h>
#include <drm/drm_sysfs.h>
#include <linux/uaccess.h>
@@ -462,6 +463,11 @@ void drm_connector_cleanup(struct drm_connector *connector)
DRM_CONNECTOR_REGISTERED))
drm_connector_unregister(connector);
+ if (connector->privacy_screen) {
+ drm_privacy_screen_put(connector->privacy_screen);
+ connector->privacy_screen = NULL;
+ }
+
if (connector->tile_group) {
drm_mode_put_tile_group(dev, connector->tile_group);
connector->tile_group = NULL;
@@ -541,7 +547,11 @@ int drm_connector_register(struct drm_connector *connector)
connector->registration_state = DRM_CONNECTOR_REGISTERED;
/* Let userspace know we have a new connector */
- drm_sysfs_hotplug_event(connector->dev);
+ drm_sysfs_connector_hotplug_event(connector);
+
+ if (connector->privacy_screen)
+ drm_privacy_screen_register_notifier(connector->privacy_screen,
+ &connector->privacy_screen_notifier);
mutex_lock(&connector_list_lock);
list_add_tail(&connector->global_connector_list_entry, &connector_list);
@@ -578,6 +588,11 @@ void drm_connector_unregister(struct drm_connector *connector)
list_del_init(&connector->global_connector_list_entry);
mutex_unlock(&connector_list_lock);
+ if (connector->privacy_screen)
+ drm_privacy_screen_unregister_notifier(
+ connector->privacy_screen,
+ &connector->privacy_screen_notifier);
+
if (connector->funcs->early_unregister)
connector->funcs->early_unregister(connector);
@@ -1271,6 +1286,46 @@ static const struct drm_prop_enum_list dp_colorspaces[] = {
* For DVI-I and TVout there is also a matching property "select subconnector"
* allowing to switch between signal types.
* DP subconnector corresponds to a downstream port.
+ *
+ * privacy-screen sw-state, privacy-screen hw-state:
+ * These 2 optional properties can be used to query the state of the
+ * electronic privacy screen that is available on some displays; and in
+ * some cases also control the state. If a driver implements these
+ * properties then both properties must be present.
+ *
+ * "privacy-screen hw-state" is read-only and reflects the actual state
+ * of the privacy-screen, possible values: "Enabled", "Disabled,
+ * "Enabled-locked", "Disabled-locked". The locked states indicate
+ * that the state cannot be changed through the DRM API. E.g. there
+ * might be devices where the firmware-setup options, or a hardware
+ * slider-switch, offer always on / off modes.
+ *
+ * "privacy-screen sw-state" can be set to change the privacy-screen state
+ * when not locked. In this case the driver must update the hw-state
+ * property to reflect the new state on completion of the commit of the
+ * sw-state property. Setting the sw-state property when the hw-state is
+ * locked must be interpreted by the driver as a request to change the
+ * state to the set state when the hw-state becomes unlocked. E.g. if
+ * "privacy-screen hw-state" is "Enabled-locked" and the sw-state
+ * gets set to "Disabled" followed by the user unlocking the state by
+ * changing the slider-switch position, then the driver must set the
+ * state to "Disabled" upon receiving the unlock event.
+ *
+ * In some cases the privacy-screen's actual state might change outside of
+ * control of the DRM code. E.g. there might be a firmware handled hotkey
+ * which toggles the actual state, or the actual state might be changed
+ * through another userspace API such as writing /proc/acpi/ibm/lcdshadow.
+ * In this case the driver must update both the hw-state and the sw-state
+ * to reflect the new value, overwriting any pending state requests in the
+ * sw-state. Any pending sw-state requests are thus discarded.
+ *
+ * Note that the ability for the state to change outside of control of
+ * the DRM master process means that userspace must not cache the value
+ * of the sw-state. Caching the sw-state value and including it in later
+ * atomic commits may lead to overriding a state change done through e.g.
+ * a firmware handled hotkey. Therefor userspace must not include the
+ * privacy-screen sw-state in an atomic commit unless it wants to change
+ * its value.
*/
int drm_connector_create_standard_properties(struct drm_device *dev)
@@ -2365,6 +2420,154 @@ int drm_connector_set_panel_orientation_with_quirk(
}
EXPORT_SYMBOL(drm_connector_set_panel_orientation_with_quirk);
+static const struct drm_prop_enum_list privacy_screen_enum[] = {
+ { PRIVACY_SCREEN_DISABLED, "Disabled" },
+ { PRIVACY_SCREEN_ENABLED, "Enabled" },
+ { PRIVACY_SCREEN_DISABLED_LOCKED, "Disabled-locked" },
+ { PRIVACY_SCREEN_ENABLED_LOCKED, "Enabled-locked" },
+};
+
+/**
+ * drm_connector_create_privacy_screen_properties - create the drm connecter's
+ * privacy-screen properties.
+ * @connector: connector for which to create the privacy-screen properties
+ *
+ * This function creates the "privacy-screen sw-state" and "privacy-screen
+ * hw-state" properties for the connector. They are not attached.
+ */
+void
+drm_connector_create_privacy_screen_properties(struct drm_connector *connector)
+{
+ if (connector->privacy_screen_sw_state_property)
+ return;
+
+ /* Note sw-state only supports the first 2 values of the enum */
+ connector->privacy_screen_sw_state_property =
+ drm_property_create_enum(connector->dev, DRM_MODE_PROP_ENUM,
+ "privacy-screen sw-state",
+ privacy_screen_enum, 2);
+
+ connector->privacy_screen_hw_state_property =
+ drm_property_create_enum(connector->dev,
+ DRM_MODE_PROP_IMMUTABLE | DRM_MODE_PROP_ENUM,
+ "privacy-screen hw-state",
+ privacy_screen_enum,
+ ARRAY_SIZE(privacy_screen_enum));
+}
+EXPORT_SYMBOL(drm_connector_create_privacy_screen_properties);
+
+/**
+ * drm_connector_attach_privacy_screen_properties - attach the drm connecter's
+ * privacy-screen properties.
+ * @connector: connector on which to attach the privacy-screen properties
+ *
+ * This function attaches the "privacy-screen sw-state" and "privacy-screen
+ * hw-state" properties to the connector. The initial state of both is set
+ * to "Disabled".
+ */
+void
+drm_connector_attach_privacy_screen_properties(struct drm_connector *connector)
+{
+ if (!connector->privacy_screen_sw_state_property)
+ return;
+
+ drm_object_attach_property(&connector->base,
+ connector->privacy_screen_sw_state_property,
+ PRIVACY_SCREEN_DISABLED);
+
+ drm_object_attach_property(&connector->base,
+ connector->privacy_screen_hw_state_property,
+ PRIVACY_SCREEN_DISABLED);
+}
+EXPORT_SYMBOL(drm_connector_attach_privacy_screen_properties);
+
+static void drm_connector_update_privacy_screen_properties(
+ struct drm_connector *connector, bool set_sw_state)
+{
+ enum drm_privacy_screen_status sw_state, hw_state;
+
+ drm_privacy_screen_get_state(connector->privacy_screen,
+ &sw_state, &hw_state);
+
+ if (set_sw_state)
+ connector->state->privacy_screen_sw_state = sw_state;
+ drm_object_property_set_value(&connector->base,
+ connector->privacy_screen_hw_state_property, hw_state);
+}
+
+static int drm_connector_privacy_screen_notifier(
+ struct notifier_block *nb, unsigned long action, void *data)
+{
+ struct drm_connector *connector =
+ container_of(nb, struct drm_connector, privacy_screen_notifier);
+ struct drm_device *dev = connector->dev;
+
+ drm_modeset_lock(&dev->mode_config.connection_mutex, NULL);
+ drm_connector_update_privacy_screen_properties(connector, true);
+ drm_modeset_unlock(&dev->mode_config.connection_mutex);
+
+ drm_sysfs_connector_status_event(connector,
+ connector->privacy_screen_sw_state_property);
+ drm_sysfs_connector_status_event(connector,
+ connector->privacy_screen_hw_state_property);
+
+ return NOTIFY_DONE;
+}
+
+/**
+ * drm_connector_attach_privacy_screen_provider - attach a privacy-screen to
+ * the connector
+ * @connector: connector to attach the privacy-screen to
+ * @priv: drm_privacy_screen to attach
+ *
+ * Create and attach the standard privacy-screen properties and register
+ * a generic notifier for generating sysfs-connector-status-events
+ * on external changes to the privacy-screen status.
+ * This function takes ownership of the passed in drm_privacy_screen and will
+ * call drm_privacy_screen_put() on it when the connector is destroyed.
+ */
+void drm_connector_attach_privacy_screen_provider(
+ struct drm_connector *connector, struct drm_privacy_screen *priv)
+{
+ connector->privacy_screen = priv;
+ connector->privacy_screen_notifier.notifier_call =
+ drm_connector_privacy_screen_notifier;
+
+ drm_connector_create_privacy_screen_properties(connector);
+ drm_connector_update_privacy_screen_properties(connector, true);
+ drm_connector_attach_privacy_screen_properties(connector);
+}
+EXPORT_SYMBOL(drm_connector_attach_privacy_screen_provider);
+
+/**
+ * drm_connector_update_privacy_screen - update connector's privacy-screen sw-state
+ * @connector_state: connector-state to update the privacy-screen for
+ *
+ * This function calls drm_privacy_screen_set_sw_state() on the connector's
+ * privacy-screen.
+ *
+ * If the connector has no privacy-screen, then this is a no-op.
+ */
+void drm_connector_update_privacy_screen(const struct drm_connector_state *connector_state)
+{
+ struct drm_connector *connector = connector_state->connector;
+ int ret;
+
+ if (!connector->privacy_screen)
+ return;
+
+ ret = drm_privacy_screen_set_sw_state(connector->privacy_screen,
+ connector_state->privacy_screen_sw_state);
+ if (ret) {
+ drm_err(connector->dev, "Error updating privacy-screen sw_state\n");
+ return;
+ }
+
+ /* The hw_state property value may have changed, update it. */
+ drm_connector_update_privacy_screen_properties(connector, false);
+}
+EXPORT_SYMBOL(drm_connector_update_privacy_screen);
+
int drm_connector_set_obj_prop(struct drm_mode_object *obj,
struct drm_property *property,
uint64_t value)
diff --git a/drivers/gpu/drm/drm_dp_helper.c b/drivers/gpu/drm/drm_dp_helper.c
index 4d0d1e8..23f9073 100644
--- a/drivers/gpu/drm/drm_dp_helper.c
+++ b/drivers/gpu/drm/drm_dp_helper.c
@@ -154,38 +154,155 @@ u8 drm_dp_get_adjust_request_post_cursor(const u8 link_status[DP_LINK_STATUS_SIZ
}
EXPORT_SYMBOL(drm_dp_get_adjust_request_post_cursor);
+static int __8b10b_clock_recovery_delay_us(const struct drm_dp_aux *aux, u8 rd_interval)
+{
+ if (rd_interval > 4)
+ drm_dbg_kms(aux->drm_dev, "%s: invalid AUX interval 0x%02x (max 4)\n",
+ aux->name, rd_interval);
+
+ if (rd_interval == 0)
+ return 100;
+
+ return rd_interval * 4 * USEC_PER_MSEC;
+}
+
+static int __8b10b_channel_eq_delay_us(const struct drm_dp_aux *aux, u8 rd_interval)
+{
+ if (rd_interval > 4)
+ drm_dbg_kms(aux->drm_dev, "%s: invalid AUX interval 0x%02x (max 4)\n",
+ aux->name, rd_interval);
+
+ if (rd_interval == 0)
+ return 400;
+
+ return rd_interval * 4 * USEC_PER_MSEC;
+}
+
+static int __128b132b_channel_eq_delay_us(const struct drm_dp_aux *aux, u8 rd_interval)
+{
+ switch (rd_interval) {
+ default:
+ drm_dbg_kms(aux->drm_dev, "%s: invalid AUX interval 0x%02x\n",
+ aux->name, rd_interval);
+ fallthrough;
+ case DP_128B132B_TRAINING_AUX_RD_INTERVAL_400_US:
+ return 400;
+ case DP_128B132B_TRAINING_AUX_RD_INTERVAL_4_MS:
+ return 4000;
+ case DP_128B132B_TRAINING_AUX_RD_INTERVAL_8_MS:
+ return 8000;
+ case DP_128B132B_TRAINING_AUX_RD_INTERVAL_12_MS:
+ return 12000;
+ case DP_128B132B_TRAINING_AUX_RD_INTERVAL_16_MS:
+ return 16000;
+ case DP_128B132B_TRAINING_AUX_RD_INTERVAL_32_MS:
+ return 32000;
+ case DP_128B132B_TRAINING_AUX_RD_INTERVAL_64_MS:
+ return 64000;
+ }
+}
+
+/*
+ * The link training delays are different for:
+ *
+ * - Clock recovery vs. channel equalization
+ * - DPRX vs. LTTPR
+ * - 128b/132b vs. 8b/10b
+ * - DPCD rev 1.3 vs. later
+ *
+ * Get the correct delay in us, reading DPCD if necessary.
+ */
+static int __read_delay(struct drm_dp_aux *aux, const u8 dpcd[DP_RECEIVER_CAP_SIZE],
+ enum drm_dp_phy dp_phy, bool uhbr, bool cr)
+{
+ int (*parse)(const struct drm_dp_aux *aux, u8 rd_interval);
+ unsigned int offset;
+ u8 rd_interval, mask;
+
+ if (dp_phy == DP_PHY_DPRX) {
+ if (uhbr) {
+ if (cr)
+ return 100;
+
+ offset = DP_128B132B_TRAINING_AUX_RD_INTERVAL;
+ mask = DP_128B132B_TRAINING_AUX_RD_INTERVAL_MASK;
+ parse = __128b132b_channel_eq_delay_us;
+ } else {
+ if (cr && dpcd[DP_DPCD_REV] >= DP_DPCD_REV_14)
+ return 100;
+
+ offset = DP_TRAINING_AUX_RD_INTERVAL;
+ mask = DP_TRAINING_AUX_RD_MASK;
+ if (cr)
+ parse = __8b10b_clock_recovery_delay_us;
+ else
+ parse = __8b10b_channel_eq_delay_us;
+ }
+ } else {
+ if (uhbr) {
+ offset = DP_128B132B_TRAINING_AUX_RD_INTERVAL_PHY_REPEATER(dp_phy);
+ mask = DP_128B132B_TRAINING_AUX_RD_INTERVAL_MASK;
+ parse = __128b132b_channel_eq_delay_us;
+ } else {
+ if (cr)
+ return 100;
+
+ offset = DP_TRAINING_AUX_RD_INTERVAL_PHY_REPEATER(dp_phy);
+ mask = DP_TRAINING_AUX_RD_MASK;
+ parse = __8b10b_channel_eq_delay_us;
+ }
+ }
+
+ if (offset < DP_RECEIVER_CAP_SIZE) {
+ rd_interval = dpcd[offset];
+ } else {
+ if (drm_dp_dpcd_readb(aux, offset, &rd_interval) != 1) {
+ drm_dbg_kms(aux->drm_dev, "%s: failed rd interval read\n",
+ aux->name);
+ /* arbitrary default delay */
+ return 400;
+ }
+ }
+
+ return parse(aux, rd_interval & mask);
+}
+
+int drm_dp_read_clock_recovery_delay(struct drm_dp_aux *aux, const u8 dpcd[DP_RECEIVER_CAP_SIZE],
+ enum drm_dp_phy dp_phy, bool uhbr)
+{
+ return __read_delay(aux, dpcd, dp_phy, uhbr, true);
+}
+EXPORT_SYMBOL(drm_dp_read_clock_recovery_delay);
+
+int drm_dp_read_channel_eq_delay(struct drm_dp_aux *aux, const u8 dpcd[DP_RECEIVER_CAP_SIZE],
+ enum drm_dp_phy dp_phy, bool uhbr)
+{
+ return __read_delay(aux, dpcd, dp_phy, uhbr, false);
+}
+EXPORT_SYMBOL(drm_dp_read_channel_eq_delay);
+
void drm_dp_link_train_clock_recovery_delay(const struct drm_dp_aux *aux,
const u8 dpcd[DP_RECEIVER_CAP_SIZE])
{
- unsigned long rd_interval = dpcd[DP_TRAINING_AUX_RD_INTERVAL] &
- DP_TRAINING_AUX_RD_MASK;
+ u8 rd_interval = dpcd[DP_TRAINING_AUX_RD_INTERVAL] &
+ DP_TRAINING_AUX_RD_MASK;
+ int delay_us;
- if (rd_interval > 4)
- drm_dbg_kms(aux->drm_dev, "%s: AUX interval %lu, out of range (max 4)\n",
- aux->name, rd_interval);
-
- if (rd_interval == 0 || dpcd[DP_DPCD_REV] >= DP_DPCD_REV_14)
- rd_interval = 100;
+ if (dpcd[DP_DPCD_REV] >= DP_DPCD_REV_14)
+ delay_us = 100;
else
- rd_interval *= 4 * USEC_PER_MSEC;
+ delay_us = __8b10b_clock_recovery_delay_us(aux, rd_interval);
- usleep_range(rd_interval, rd_interval * 2);
+ usleep_range(delay_us, delay_us * 2);
}
EXPORT_SYMBOL(drm_dp_link_train_clock_recovery_delay);
static void __drm_dp_link_train_channel_eq_delay(const struct drm_dp_aux *aux,
- unsigned long rd_interval)
+ u8 rd_interval)
{
- if (rd_interval > 4)
- drm_dbg_kms(aux->drm_dev, "%s: AUX interval %lu, out of range (max 4)\n",
- aux->name, rd_interval);
+ int delay_us = __8b10b_channel_eq_delay_us(aux, rd_interval);
- if (rd_interval == 0)
- rd_interval = 400;
- else
- rd_interval *= 4 * USEC_PER_MSEC;
-
- usleep_range(rd_interval, rd_interval * 2);
+ usleep_range(delay_us, delay_us * 2);
}
void drm_dp_link_train_channel_eq_delay(const struct drm_dp_aux *aux,
@@ -3173,6 +3290,10 @@ int drm_edp_backlight_set_level(struct drm_dp_aux *aux, const struct drm_edp_bac
int ret;
u8 buf[2] = { 0 };
+ /* The panel uses the PWM for controlling brightness levels */
+ if (!bl->aux_set)
+ return 0;
+
if (bl->lsb_reg_used) {
buf[0] = (level & 0xff00) >> 8;
buf[1] = (level & 0x00ff);
@@ -3199,7 +3320,7 @@ drm_edp_backlight_set_enable(struct drm_dp_aux *aux, const struct drm_edp_backli
int ret;
u8 buf;
- /* The panel uses something other then DPCD for enabling its backlight */
+ /* This panel uses the EDP_BL_PWR GPIO for enablement */
if (!bl->aux_enable)
return 0;
@@ -3234,11 +3355,11 @@ drm_edp_backlight_set_enable(struct drm_dp_aux *aux, const struct drm_edp_backli
* restoring any important backlight state such as the given backlight level, the brightness byte
* count, backlight frequency, etc.
*
- * Note that certain panels, while supporting brightness level controls over DPCD, may not support
- * having their backlights enabled via the standard %DP_EDP_DISPLAY_CONTROL_REGISTER. On such panels
- * &drm_edp_backlight_info.aux_enable will be set to %false, this function will skip the step of
- * programming the %DP_EDP_DISPLAY_CONTROL_REGISTER, and the driver must perform the required
- * implementation specific step for enabling the backlight after calling this function.
+ * Note that certain panels do not support being enabled or disabled via DPCD, but instead require
+ * that the driver handle enabling/disabling the panel through implementation-specific means using
+ * the EDP_BL_PWR GPIO. For such panels, &drm_edp_backlight_info.aux_enable will be set to %false,
+ * this function becomes a no-op, and the driver is expected to handle powering the panel on using
+ * the EDP_BL_PWR GPIO.
*
* Returns: %0 on success, negative error code on failure.
*/
@@ -3246,27 +3367,18 @@ int drm_edp_backlight_enable(struct drm_dp_aux *aux, const struct drm_edp_backli
const u16 level)
{
int ret;
- u8 dpcd_buf, new_dpcd_buf;
+ u8 dpcd_buf;
- ret = drm_dp_dpcd_readb(aux, DP_EDP_BACKLIGHT_MODE_SET_REGISTER, &dpcd_buf);
- if (ret != 1) {
- drm_dbg_kms(aux->drm_dev,
- "%s: Failed to read backlight mode: %d\n", aux->name, ret);
- return ret < 0 ? ret : -EIO;
- }
+ if (bl->aux_set)
+ dpcd_buf = DP_EDP_BACKLIGHT_CONTROL_MODE_DPCD;
+ else
+ dpcd_buf = DP_EDP_BACKLIGHT_CONTROL_MODE_PWM;
- new_dpcd_buf = dpcd_buf;
-
- if ((dpcd_buf & DP_EDP_BACKLIGHT_CONTROL_MODE_MASK) != DP_EDP_BACKLIGHT_CONTROL_MODE_DPCD) {
- new_dpcd_buf &= ~DP_EDP_BACKLIGHT_CONTROL_MODE_MASK;
- new_dpcd_buf |= DP_EDP_BACKLIGHT_CONTROL_MODE_DPCD;
-
- if (bl->pwmgen_bit_count) {
- ret = drm_dp_dpcd_writeb(aux, DP_EDP_PWMGEN_BIT_COUNT, bl->pwmgen_bit_count);
- if (ret != 1)
- drm_dbg_kms(aux->drm_dev, "%s: Failed to write aux pwmgen bit count: %d\n",
- aux->name, ret);
- }
+ if (bl->pwmgen_bit_count) {
+ ret = drm_dp_dpcd_writeb(aux, DP_EDP_PWMGEN_BIT_COUNT, bl->pwmgen_bit_count);
+ if (ret != 1)
+ drm_dbg_kms(aux->drm_dev, "%s: Failed to write aux pwmgen bit count: %d\n",
+ aux->name, ret);
}
if (bl->pwm_freq_pre_divider) {
@@ -3276,16 +3388,14 @@ int drm_edp_backlight_enable(struct drm_dp_aux *aux, const struct drm_edp_backli
"%s: Failed to write aux backlight frequency: %d\n",
aux->name, ret);
else
- new_dpcd_buf |= DP_EDP_BACKLIGHT_FREQ_AUX_SET_ENABLE;
+ dpcd_buf |= DP_EDP_BACKLIGHT_FREQ_AUX_SET_ENABLE;
}
- if (new_dpcd_buf != dpcd_buf) {
- ret = drm_dp_dpcd_writeb(aux, DP_EDP_BACKLIGHT_MODE_SET_REGISTER, new_dpcd_buf);
- if (ret != 1) {
- drm_dbg_kms(aux->drm_dev, "%s: Failed to write aux backlight mode: %d\n",
- aux->name, ret);
- return ret < 0 ? ret : -EIO;
- }
+ ret = drm_dp_dpcd_writeb(aux, DP_EDP_BACKLIGHT_MODE_SET_REGISTER, dpcd_buf);
+ if (ret != 1) {
+ drm_dbg_kms(aux->drm_dev, "%s: Failed to write aux backlight mode: %d\n",
+ aux->name, ret);
+ return ret < 0 ? ret : -EIO;
}
ret = drm_edp_backlight_set_level(aux, bl, level);
@@ -3304,12 +3414,13 @@ EXPORT_SYMBOL(drm_edp_backlight_enable);
* @aux: The DP AUX channel to use
* @bl: Backlight capability info from drm_edp_backlight_init()
*
- * This function handles disabling DPCD backlight controls on a panel over AUX. Note that some
- * panels have backlights that are enabled/disabled by other means, despite having their brightness
- * values controlled through DPCD. On such panels &drm_edp_backlight_info.aux_enable will be set to
- * %false, this function will become a no-op (and we will skip updating
- * %DP_EDP_DISPLAY_CONTROL_REGISTER), and the driver must take care to perform it's own
- * implementation specific step for disabling the backlight.
+ * This function handles disabling DPCD backlight controls on a panel over AUX.
+ *
+ * Note that certain panels do not support being enabled or disabled via DPCD, but instead require
+ * that the driver handle enabling/disabling the panel through implementation-specific means using
+ * the EDP_BL_PWR GPIO. For such panels, &drm_edp_backlight_info.aux_enable will be set to %false,
+ * this function becomes a no-op, and the driver is expected to handle powering the panel off using
+ * the EDP_BL_PWR GPIO.
*
* Returns: %0 on success or no-op, negative error code on failure.
*/
@@ -3333,6 +3444,9 @@ drm_edp_backlight_probe_max(struct drm_dp_aux *aux, struct drm_edp_backlight_inf
int ret;
u8 pn, pn_min, pn_max;
+ if (!bl->aux_set)
+ return 0;
+
ret = drm_dp_dpcd_readb(aux, DP_EDP_PWMGEN_BIT_COUNT, &pn);
if (ret != 1) {
drm_dbg_kms(aux->drm_dev, "%s: Failed to read pwmgen bit count cap: %d\n",
@@ -3418,7 +3532,7 @@ drm_edp_backlight_probe_max(struct drm_dp_aux *aux, struct drm_edp_backlight_inf
}
static inline int
-drm_edp_backlight_probe_level(struct drm_dp_aux *aux, struct drm_edp_backlight_info *bl,
+drm_edp_backlight_probe_state(struct drm_dp_aux *aux, struct drm_edp_backlight_info *bl,
u8 *current_mode)
{
int ret;
@@ -3433,6 +3547,9 @@ drm_edp_backlight_probe_level(struct drm_dp_aux *aux, struct drm_edp_backlight_i
}
*current_mode = (mode_reg & DP_EDP_BACKLIGHT_CONTROL_MODE_MASK);
+ if (!bl->aux_set)
+ return 0;
+
if (*current_mode == DP_EDP_BACKLIGHT_CONTROL_MODE_DPCD) {
int size = 1 + bl->lsb_reg_used;
@@ -3463,7 +3580,7 @@ drm_edp_backlight_probe_level(struct drm_dp_aux *aux, struct drm_edp_backlight_i
* @bl: The &drm_edp_backlight_info struct to fill out with information on the backlight
* @driver_pwm_freq_hz: Optional PWM frequency from the driver in hz
* @edp_dpcd: A cached copy of the eDP DPCD
- * @current_level: Where to store the probed brightness level
+ * @current_level: Where to store the probed brightness level, if any
* @current_mode: Where to store the currently set backlight control mode
*
* Initializes a &drm_edp_backlight_info struct by probing @aux for it's backlight capabilities,
@@ -3483,24 +3600,38 @@ drm_edp_backlight_init(struct drm_dp_aux *aux, struct drm_edp_backlight_info *bl
if (edp_dpcd[1] & DP_EDP_BACKLIGHT_AUX_ENABLE_CAP)
bl->aux_enable = true;
+ if (edp_dpcd[2] & DP_EDP_BACKLIGHT_BRIGHTNESS_AUX_SET_CAP)
+ bl->aux_set = true;
if (edp_dpcd[2] & DP_EDP_BACKLIGHT_BRIGHTNESS_BYTE_COUNT)
bl->lsb_reg_used = true;
+ /* Sanity check caps */
+ if (!bl->aux_set && !(edp_dpcd[2] & DP_EDP_BACKLIGHT_BRIGHTNESS_PWM_PIN_CAP)) {
+ drm_dbg_kms(aux->drm_dev,
+ "%s: Panel supports neither AUX or PWM brightness control? Aborting\n",
+ aux->name);
+ return -EINVAL;
+ }
+
ret = drm_edp_backlight_probe_max(aux, bl, driver_pwm_freq_hz, edp_dpcd);
if (ret < 0)
return ret;
- ret = drm_edp_backlight_probe_level(aux, bl, current_mode);
+ ret = drm_edp_backlight_probe_state(aux, bl, current_mode);
if (ret < 0)
return ret;
*current_level = ret;
drm_dbg_kms(aux->drm_dev,
- "%s: Found backlight level=%d/%d pwm_freq_pre_divider=%d mode=%x\n",
- aux->name, *current_level, bl->max, bl->pwm_freq_pre_divider, *current_mode);
- drm_dbg_kms(aux->drm_dev,
- "%s: Backlight caps: pwmgen_bit_count=%d lsb_reg_used=%d aux_enable=%d\n",
- aux->name, bl->pwmgen_bit_count, bl->lsb_reg_used, bl->aux_enable);
+ "%s: Found backlight: aux_set=%d aux_enable=%d mode=%d\n",
+ aux->name, bl->aux_set, bl->aux_enable, *current_mode);
+ if (bl->aux_set) {
+ drm_dbg_kms(aux->drm_dev,
+ "%s: Backlight caps: level=%d/%d pwm_freq_pre_divider=%d lsb_reg_used=%d\n",
+ aux->name, *current_level, bl->max, bl->pwm_freq_pre_divider,
+ bl->lsb_reg_used);
+ }
+
return 0;
}
EXPORT_SYMBOL(drm_edp_backlight_init);
diff --git a/drivers/gpu/drm/drm_drv.c b/drivers/gpu/drm/drm_drv.c
index 7a50974..8214a0b 100644
--- a/drivers/gpu/drm/drm_drv.c
+++ b/drivers/gpu/drm/drm_drv.c
@@ -43,6 +43,7 @@
#include <drm/drm_managed.h>
#include <drm/drm_mode_object.h>
#include <drm/drm_print.h>
+#include <drm/drm_privacy_screen_machine.h>
#include "drm_crtc_internal.h"
#include "drm_internal.h"
@@ -581,6 +582,7 @@ static int drm_dev_init(struct drm_device *dev,
const struct drm_driver *driver,
struct device *parent)
{
+ struct inode *inode;
int ret;
if (!drm_core_init_complete) {
@@ -617,13 +619,15 @@ static int drm_dev_init(struct drm_device *dev,
if (ret)
return ret;
- dev->anon_inode = drm_fs_inode_new();
- if (IS_ERR(dev->anon_inode)) {
- ret = PTR_ERR(dev->anon_inode);
+ inode = drm_fs_inode_new();
+ if (IS_ERR(inode)) {
+ ret = PTR_ERR(inode);
DRM_ERROR("Cannot allocate anonymous inode: %d\n", ret);
goto err;
}
+ dev->anon_inode = inode;
+
if (drm_core_check_feature(dev, DRIVER_RENDER)) {
ret = drm_minor_alloc(dev, DRM_MINOR_RENDER);
if (ret)
@@ -1029,6 +1033,7 @@ static const struct file_operations drm_stub_fops = {
static void drm_core_exit(void)
{
+ drm_privacy_screen_lookup_exit();
unregister_chrdev(DRM_MAJOR, "drm");
debugfs_remove(drm_debugfs_root);
drm_sysfs_destroy();
@@ -1056,6 +1061,8 @@ static int __init drm_core_init(void)
if (ret < 0)
goto error;
+ drm_privacy_screen_lookup_init();
+
drm_core_init_complete = true;
DRM_DEBUG("Initialized\n");
diff --git a/drivers/gpu/drm/drm_fb_helper.c b/drivers/gpu/drm/drm_fb_helper.c
index 22bf690..ed43b98 100644
--- a/drivers/gpu/drm/drm_fb_helper.c
+++ b/drivers/gpu/drm/drm_fb_helper.c
@@ -2344,7 +2344,7 @@ static int drm_fb_helper_generic_probe(struct drm_fb_helper *fb_helper,
return PTR_ERR(fbi);
fbi->fbops = &drm_fbdev_fb_ops;
- fbi->screen_size = fb->height * fb->pitches[0];
+ fbi->screen_size = sizes->surface_height * fb->pitches[0];
fbi->fix.smem_len = fbi->screen_size;
drm_fb_helper_fill_info(fbi, fb_helper, sizes);
diff --git a/drivers/gpu/drm/drm_format_helper.c b/drivers/gpu/drm/drm_format_helper.c
index 69fde60..0f28dd2 100644
--- a/drivers/gpu/drm/drm_format_helper.c
+++ b/drivers/gpu/drm/drm_format_helper.c
@@ -17,71 +17,91 @@
#include <drm/drm_fourcc.h>
#include <drm/drm_rect.h>
-static unsigned int clip_offset(struct drm_rect *clip,
- unsigned int pitch, unsigned int cpp)
+static unsigned int clip_offset(const struct drm_rect *clip, unsigned int pitch, unsigned int cpp)
{
return clip->y1 * pitch + clip->x1 * cpp;
}
/**
+ * drm_fb_clip_offset - Returns the clipping rectangles byte-offset in a framebuffer
+ * @pitch: Framebuffer line pitch in byte
+ * @format: Framebuffer format
+ * @clip: Clip rectangle
+ *
+ * Returns:
+ * The byte offset of the clip rectangle's top-left corner within the framebuffer.
+ */
+unsigned int drm_fb_clip_offset(unsigned int pitch, const struct drm_format_info *format,
+ const struct drm_rect *clip)
+{
+ return clip_offset(clip, pitch, format->cpp[0]);
+}
+EXPORT_SYMBOL(drm_fb_clip_offset);
+
+/**
* drm_fb_memcpy - Copy clip buffer
* @dst: Destination buffer
+ * @dst_pitch: Number of bytes between two consecutive scanlines within dst
* @vaddr: Source buffer
* @fb: DRM framebuffer
* @clip: Clip rectangle area to copy
*
* This function does not apply clipping on dst, i.e. the destination
- * is a small buffer containing the clip rect only.
+ * is at the top-left corner.
*/
-void drm_fb_memcpy(void *dst, void *vaddr, struct drm_framebuffer *fb,
- struct drm_rect *clip)
+void drm_fb_memcpy(void *dst, unsigned int dst_pitch, const void *vaddr,
+ const struct drm_framebuffer *fb, const struct drm_rect *clip)
{
unsigned int cpp = fb->format->cpp[0];
size_t len = (clip->x2 - clip->x1) * cpp;
unsigned int y, lines = clip->y2 - clip->y1;
+ if (!dst_pitch)
+ dst_pitch = len;
+
vaddr += clip_offset(clip, fb->pitches[0], cpp);
for (y = 0; y < lines; y++) {
memcpy(dst, vaddr, len);
vaddr += fb->pitches[0];
- dst += len;
+ dst += dst_pitch;
}
}
EXPORT_SYMBOL(drm_fb_memcpy);
/**
- * drm_fb_memcpy_dstclip - Copy clip buffer
+ * drm_fb_memcpy_toio - Copy clip buffer
* @dst: Destination buffer (iomem)
* @dst_pitch: Number of bytes between two consecutive scanlines within dst
* @vaddr: Source buffer
* @fb: DRM framebuffer
* @clip: Clip rectangle area to copy
*
- * This function applies clipping on dst, i.e. the destination is a
- * full (iomem) framebuffer but only the clip rect content is copied over.
+ * This function does not apply clipping on dst, i.e. the destination
+ * is at the top-left corner.
*/
-void drm_fb_memcpy_dstclip(void __iomem *dst, unsigned int dst_pitch,
- void *vaddr, struct drm_framebuffer *fb,
- struct drm_rect *clip)
+void drm_fb_memcpy_toio(void __iomem *dst, unsigned int dst_pitch, const void *vaddr,
+ const struct drm_framebuffer *fb, const struct drm_rect *clip)
{
unsigned int cpp = fb->format->cpp[0];
- unsigned int offset = clip_offset(clip, dst_pitch, cpp);
size_t len = (clip->x2 - clip->x1) * cpp;
unsigned int y, lines = clip->y2 - clip->y1;
- vaddr += offset;
- dst += offset;
+ if (!dst_pitch)
+ dst_pitch = len;
+
+ vaddr += clip_offset(clip, fb->pitches[0], cpp);
for (y = 0; y < lines; y++) {
memcpy_toio(dst, vaddr, len);
vaddr += fb->pitches[0];
dst += dst_pitch;
}
}
-EXPORT_SYMBOL(drm_fb_memcpy_dstclip);
+EXPORT_SYMBOL(drm_fb_memcpy_toio);
/**
* drm_fb_swab - Swap bytes into clip buffer
* @dst: Destination buffer
+ * @dst_pitch: Number of bytes between two consecutive scanlines within dst
* @src: Source buffer
* @fb: DRM framebuffer
* @clip: Clip rectangle area to copy
@@ -91,21 +111,27 @@ EXPORT_SYMBOL(drm_fb_memcpy_dstclip);
* time to speed up slow uncached reads.
*
* This function does not apply clipping on dst, i.e. the destination
- * is a small buffer containing the clip rect only.
+ * is at the top-left corner.
*/
-void drm_fb_swab(void *dst, void *src, struct drm_framebuffer *fb,
- struct drm_rect *clip, bool cached)
+void drm_fb_swab(void *dst, unsigned int dst_pitch, const void *src,
+ const struct drm_framebuffer *fb, const struct drm_rect *clip,
+ bool cached)
{
u8 cpp = fb->format->cpp[0];
size_t len = drm_rect_width(clip) * cpp;
- u16 *src16, *dst16 = dst;
- u32 *src32, *dst32 = dst;
+ const u16 *src16;
+ const u32 *src32;
+ u16 *dst16;
+ u32 *dst32;
unsigned int x, y;
void *buf = NULL;
if (WARN_ON_ONCE(cpp != 2 && cpp != 4))
return;
+ if (!dst_pitch)
+ dst_pitch = len;
+
if (!cached)
buf = kmalloc(len, GFP_KERNEL);
@@ -121,6 +147,9 @@ void drm_fb_swab(void *dst, void *src, struct drm_framebuffer *fb,
src32 = src;
}
+ dst16 = dst;
+ dst32 = dst;
+
for (x = clip->x1; x < clip->x2; x++) {
if (cpp == 4)
*dst32++ = swab32(*src32++);
@@ -129,13 +158,14 @@ void drm_fb_swab(void *dst, void *src, struct drm_framebuffer *fb,
}
src += fb->pitches[0];
+ dst += dst_pitch;
}
kfree(buf);
}
EXPORT_SYMBOL(drm_fb_swab);
-static void drm_fb_xrgb8888_to_rgb332_line(u8 *dbuf, __le32 *sbuf, unsigned int pixels)
+static void drm_fb_xrgb8888_to_rgb332_line(u8 *dbuf, const __le32 *sbuf, unsigned int pixels)
{
unsigned int x;
u32 pix;
@@ -151,23 +181,24 @@ static void drm_fb_xrgb8888_to_rgb332_line(u8 *dbuf, __le32 *sbuf, unsigned int
/**
* drm_fb_xrgb8888_to_rgb332 - Convert XRGB8888 to RGB332 clip buffer
* @dst: RGB332 destination buffer
+ * @dst_pitch: Number of bytes between two consecutive scanlines within dst
* @src: XRGB8888 source buffer
* @fb: DRM framebuffer
* @clip: Clip rectangle area to copy
*
* Drivers can use this function for RGB332 devices that don't natively support XRGB8888.
- *
- * This function does not apply clipping on dst, i.e. the destination is a small buffer
- * containing the clip rect only.
*/
-void drm_fb_xrgb8888_to_rgb332(void *dst, void *src, struct drm_framebuffer *fb,
- struct drm_rect *clip)
+void drm_fb_xrgb8888_to_rgb332(void *dst, unsigned int dst_pitch, const void *src,
+ const struct drm_framebuffer *fb, const struct drm_rect *clip)
{
size_t width = drm_rect_width(clip);
size_t src_len = width * sizeof(u32);
unsigned int y;
void *sbuf;
+ if (!dst_pitch)
+ dst_pitch = width;
+
/* Use a buffer to speed up access on buffers with uncached read mapping (i.e. WC) */
sbuf = kmalloc(src_len, GFP_KERNEL);
if (!sbuf)
@@ -178,14 +209,14 @@ void drm_fb_xrgb8888_to_rgb332(void *dst, void *src, struct drm_framebuffer *fb,
memcpy(sbuf, src, src_len);
drm_fb_xrgb8888_to_rgb332_line(dst, sbuf, width);
src += fb->pitches[0];
- dst += width;
+ dst += dst_pitch;
}
kfree(sbuf);
}
EXPORT_SYMBOL(drm_fb_xrgb8888_to_rgb332);
-static void drm_fb_xrgb8888_to_rgb565_line(u16 *dbuf, u32 *sbuf,
+static void drm_fb_xrgb8888_to_rgb565_line(u16 *dbuf, const u32 *sbuf,
unsigned int pixels,
bool swab)
{
@@ -206,6 +237,7 @@ static void drm_fb_xrgb8888_to_rgb565_line(u16 *dbuf, u32 *sbuf,
/**
* drm_fb_xrgb8888_to_rgb565 - Convert XRGB8888 to RGB565 clip buffer
* @dst: RGB565 destination buffer
+ * @dst_pitch: Number of bytes between two consecutive scanlines within dst
* @vaddr: XRGB8888 source buffer
* @fb: DRM framebuffer
* @clip: Clip rectangle area to copy
@@ -213,13 +245,10 @@ static void drm_fb_xrgb8888_to_rgb565_line(u16 *dbuf, u32 *sbuf,
*
* Drivers can use this function for RGB565 devices that don't natively
* support XRGB8888.
- *
- * This function does not apply clipping on dst, i.e. the destination
- * is a small buffer containing the clip rect only.
*/
-void drm_fb_xrgb8888_to_rgb565(void *dst, void *vaddr,
- struct drm_framebuffer *fb,
- struct drm_rect *clip, bool swab)
+void drm_fb_xrgb8888_to_rgb565(void *dst, unsigned int dst_pitch, const void *vaddr,
+ const struct drm_framebuffer *fb, const struct drm_rect *clip,
+ bool swab)
{
size_t linepixels = clip->x2 - clip->x1;
size_t src_len = linepixels * sizeof(u32);
@@ -227,6 +256,9 @@ void drm_fb_xrgb8888_to_rgb565(void *dst, void *vaddr,
unsigned y, lines = clip->y2 - clip->y1;
void *sbuf;
+ if (!dst_pitch)
+ dst_pitch = dst_len;
+
/*
* The cma memory is write-combined so reads are uncached.
* Speed up by fetching one line at a time.
@@ -240,7 +272,7 @@ void drm_fb_xrgb8888_to_rgb565(void *dst, void *vaddr,
memcpy(sbuf, vaddr, src_len);
drm_fb_xrgb8888_to_rgb565_line(dst, sbuf, linepixels, swab);
vaddr += fb->pitches[0];
- dst += dst_len;
+ dst += dst_pitch;
}
kfree(sbuf);
@@ -248,9 +280,9 @@ void drm_fb_xrgb8888_to_rgb565(void *dst, void *vaddr,
EXPORT_SYMBOL(drm_fb_xrgb8888_to_rgb565);
/**
- * drm_fb_xrgb8888_to_rgb565_dstclip - Convert XRGB8888 to RGB565 clip buffer
+ * drm_fb_xrgb8888_to_rgb565_toio - Convert XRGB8888 to RGB565 clip buffer
* @dst: RGB565 destination buffer (iomem)
- * @dst_pitch: destination buffer pitch
+ * @dst_pitch: Number of bytes between two consecutive scanlines within dst
* @vaddr: XRGB8888 source buffer
* @fb: DRM framebuffer
* @clip: Clip rectangle area to copy
@@ -258,37 +290,36 @@ EXPORT_SYMBOL(drm_fb_xrgb8888_to_rgb565);
*
* Drivers can use this function for RGB565 devices that don't natively
* support XRGB8888.
- *
- * This function applies clipping on dst, i.e. the destination is a
- * full (iomem) framebuffer but only the clip rect content is copied over.
*/
-void drm_fb_xrgb8888_to_rgb565_dstclip(void __iomem *dst, unsigned int dst_pitch,
- void *vaddr, struct drm_framebuffer *fb,
- struct drm_rect *clip, bool swab)
+void drm_fb_xrgb8888_to_rgb565_toio(void __iomem *dst, unsigned int dst_pitch,
+ const void *vaddr, const struct drm_framebuffer *fb,
+ const struct drm_rect *clip, bool swab)
{
size_t linepixels = clip->x2 - clip->x1;
size_t dst_len = linepixels * sizeof(u16);
unsigned y, lines = clip->y2 - clip->y1;
void *dbuf;
+ if (!dst_pitch)
+ dst_pitch = dst_len;
+
dbuf = kmalloc(dst_len, GFP_KERNEL);
if (!dbuf)
return;
vaddr += clip_offset(clip, fb->pitches[0], sizeof(u32));
- dst += clip_offset(clip, dst_pitch, sizeof(u16));
for (y = 0; y < lines; y++) {
drm_fb_xrgb8888_to_rgb565_line(dbuf, vaddr, linepixels, swab);
memcpy_toio(dst, dbuf, dst_len);
vaddr += fb->pitches[0];
- dst += dst_len;
+ dst += dst_pitch;
}
kfree(dbuf);
}
-EXPORT_SYMBOL(drm_fb_xrgb8888_to_rgb565_dstclip);
+EXPORT_SYMBOL(drm_fb_xrgb8888_to_rgb565_toio);
-static void drm_fb_xrgb8888_to_rgb888_line(u8 *dbuf, u32 *sbuf,
+static void drm_fb_xrgb8888_to_rgb888_line(u8 *dbuf, const u32 *sbuf,
unsigned int pixels)
{
unsigned int x;
@@ -303,24 +334,25 @@ static void drm_fb_xrgb8888_to_rgb888_line(u8 *dbuf, u32 *sbuf,
/**
* drm_fb_xrgb8888_to_rgb888 - Convert XRGB8888 to RGB888 clip buffer
* @dst: RGB888 destination buffer
+ * @dst_pitch: Number of bytes between two consecutive scanlines within dst
* @src: XRGB8888 source buffer
* @fb: DRM framebuffer
* @clip: Clip rectangle area to copy
*
* Drivers can use this function for RGB888 devices that don't natively
* support XRGB8888.
- *
- * This function does not apply clipping on dst, i.e. the destination
- * is a small buffer containing the clip rect only.
*/
-void drm_fb_xrgb8888_to_rgb888(void *dst, void *src, struct drm_framebuffer *fb,
- struct drm_rect *clip)
+void drm_fb_xrgb8888_to_rgb888(void *dst, unsigned int dst_pitch, const void *src,
+ const struct drm_framebuffer *fb, const struct drm_rect *clip)
{
size_t width = drm_rect_width(clip);
size_t src_len = width * sizeof(u32);
unsigned int y;
void *sbuf;
+ if (!dst_pitch)
+ dst_pitch = width * 3;
+
/* Use a buffer to speed up access on buffers with uncached read mapping (i.e. WC) */
sbuf = kmalloc(src_len, GFP_KERNEL);
if (!sbuf)
@@ -331,7 +363,7 @@ void drm_fb_xrgb8888_to_rgb888(void *dst, void *src, struct drm_framebuffer *fb,
memcpy(sbuf, src, src_len);
drm_fb_xrgb8888_to_rgb888_line(dst, sbuf, width);
src += fb->pitches[0];
- dst += width * 3;
+ dst += dst_pitch;
}
kfree(sbuf);
@@ -339,48 +371,103 @@ void drm_fb_xrgb8888_to_rgb888(void *dst, void *src, struct drm_framebuffer *fb,
EXPORT_SYMBOL(drm_fb_xrgb8888_to_rgb888);
/**
- * drm_fb_xrgb8888_to_rgb888_dstclip - Convert XRGB8888 to RGB888 clip buffer
+ * drm_fb_xrgb8888_to_rgb888_toio - Convert XRGB8888 to RGB888 clip buffer
* @dst: RGB565 destination buffer (iomem)
- * @dst_pitch: destination buffer pitch
+ * @dst_pitch: Number of bytes between two consecutive scanlines within dst
* @vaddr: XRGB8888 source buffer
* @fb: DRM framebuffer
* @clip: Clip rectangle area to copy
*
* Drivers can use this function for RGB888 devices that don't natively
* support XRGB8888.
- *
- * This function applies clipping on dst, i.e. the destination is a
- * full (iomem) framebuffer but only the clip rect content is copied over.
*/
-void drm_fb_xrgb8888_to_rgb888_dstclip(void __iomem *dst, unsigned int dst_pitch,
- void *vaddr, struct drm_framebuffer *fb,
- struct drm_rect *clip)
+void drm_fb_xrgb8888_to_rgb888_toio(void __iomem *dst, unsigned int dst_pitch,
+ const void *vaddr, const struct drm_framebuffer *fb,
+ const struct drm_rect *clip)
{
size_t linepixels = clip->x2 - clip->x1;
size_t dst_len = linepixels * 3;
unsigned y, lines = clip->y2 - clip->y1;
void *dbuf;
+ if (!dst_pitch)
+ dst_pitch = dst_len;
+
dbuf = kmalloc(dst_len, GFP_KERNEL);
if (!dbuf)
return;
vaddr += clip_offset(clip, fb->pitches[0], sizeof(u32));
- dst += clip_offset(clip, dst_pitch, sizeof(u16));
for (y = 0; y < lines; y++) {
drm_fb_xrgb8888_to_rgb888_line(dbuf, vaddr, linepixels);
memcpy_toio(dst, dbuf, dst_len);
vaddr += fb->pitches[0];
- dst += dst_len;
+ dst += dst_pitch;
}
kfree(dbuf);
}
-EXPORT_SYMBOL(drm_fb_xrgb8888_to_rgb888_dstclip);
+EXPORT_SYMBOL(drm_fb_xrgb8888_to_rgb888_toio);
+
+static void drm_fb_xrgb8888_to_xrgb2101010_line(u32 *dbuf, const u32 *sbuf,
+ unsigned int pixels)
+{
+ unsigned int x;
+ u32 val32;
+
+ for (x = 0; x < pixels; x++) {
+ val32 = ((sbuf[x] & 0x000000FF) << 2) |
+ ((sbuf[x] & 0x0000FF00) << 4) |
+ ((sbuf[x] & 0x00FF0000) << 6);
+ *dbuf++ = val32 | ((val32 >> 8) & 0x00300C03);
+ }
+}
+
+/**
+ * drm_fb_xrgb8888_to_xrgb2101010_toio - Convert XRGB8888 to XRGB2101010 clip
+ * buffer
+ * @dst: XRGB2101010 destination buffer (iomem)
+ * @dst_pitch: Number of bytes between two consecutive scanlines within dst
+ * @vaddr: XRGB8888 source buffer
+ * @fb: DRM framebuffer
+ * @clip: Clip rectangle area to copy
+ *
+ * Drivers can use this function for XRGB2101010 devices that don't natively
+ * support XRGB8888.
+ */
+void drm_fb_xrgb8888_to_xrgb2101010_toio(void __iomem *dst,
+ unsigned int dst_pitch, const void *vaddr,
+ const struct drm_framebuffer *fb,
+ const struct drm_rect *clip)
+{
+ size_t linepixels = clip->x2 - clip->x1;
+ size_t dst_len = linepixels * sizeof(u32);
+ unsigned int y, lines = clip->y2 - clip->y1;
+ void *dbuf;
+
+ if (!dst_pitch)
+ dst_pitch = dst_len;
+
+ dbuf = kmalloc(dst_len, GFP_KERNEL);
+ if (!dbuf)
+ return;
+
+ vaddr += clip_offset(clip, fb->pitches[0], sizeof(u32));
+ for (y = 0; y < lines; y++) {
+ drm_fb_xrgb8888_to_xrgb2101010_line(dbuf, vaddr, linepixels);
+ memcpy_toio(dst, dbuf, dst_len);
+ vaddr += fb->pitches[0];
+ dst += dst_pitch;
+ }
+
+ kfree(dbuf);
+}
+EXPORT_SYMBOL(drm_fb_xrgb8888_to_xrgb2101010_toio);
/**
* drm_fb_xrgb8888_to_gray8 - Convert XRGB8888 to grayscale
* @dst: 8-bit grayscale destination buffer
+ * @dst_pitch: Number of bytes between two consecutive scanlines within dst
* @vaddr: XRGB8888 source buffer
* @fb: DRM framebuffer
* @clip: Clip rectangle area to copy
@@ -394,16 +481,21 @@ EXPORT_SYMBOL(drm_fb_xrgb8888_to_rgb888_dstclip);
*
* ITU BT.601 is used for the RGB -> luma (brightness) conversion.
*/
-void drm_fb_xrgb8888_to_gray8(u8 *dst, void *vaddr, struct drm_framebuffer *fb,
- struct drm_rect *clip)
+void drm_fb_xrgb8888_to_gray8(void *dst, unsigned int dst_pitch, const void *vaddr,
+ const struct drm_framebuffer *fb, const struct drm_rect *clip)
{
unsigned int len = (clip->x2 - clip->x1) * sizeof(u32);
unsigned int x, y;
void *buf;
- u32 *src;
+ u8 *dst8;
+ u32 *src32;
if (WARN_ON(fb->format->format != DRM_FORMAT_XRGB8888))
return;
+
+ if (!dst_pitch)
+ dst_pitch = drm_rect_width(clip);
+
/*
* The cma memory is write-combined so reads are uncached.
* Speed up by fetching one line at a time.
@@ -412,20 +504,22 @@ void drm_fb_xrgb8888_to_gray8(u8 *dst, void *vaddr, struct drm_framebuffer *fb,
if (!buf)
return;
+ vaddr += clip_offset(clip, fb->pitches[0], sizeof(u32));
for (y = clip->y1; y < clip->y2; y++) {
- src = vaddr + (y * fb->pitches[0]);
- src += clip->x1;
- memcpy(buf, src, len);
- src = buf;
+ dst8 = dst;
+ src32 = memcpy(buf, vaddr, len);
for (x = clip->x1; x < clip->x2; x++) {
- u8 r = (*src & 0x00ff0000) >> 16;
- u8 g = (*src & 0x0000ff00) >> 8;
- u8 b = *src & 0x000000ff;
+ u8 r = (*src32 & 0x00ff0000) >> 16;
+ u8 g = (*src32 & 0x0000ff00) >> 8;
+ u8 b = *src32 & 0x000000ff;
/* ITU BT.601: Y = 0.299 R + 0.587 G + 0.114 B */
- *dst++ = (3 * r + 6 * g + b) / 10;
- src++;
+ *dst8++ = (3 * r + 6 * g + b) / 10;
+ src32++;
}
+
+ vaddr += fb->pitches[0];
+ dst += dst_pitch;
}
kfree(buf);
@@ -433,7 +527,7 @@ void drm_fb_xrgb8888_to_gray8(u8 *dst, void *vaddr, struct drm_framebuffer *fb,
EXPORT_SYMBOL(drm_fb_xrgb8888_to_gray8);
/**
- * drm_fb_blit_rect_dstclip - Copy parts of a framebuffer to display memory
+ * drm_fb_blit_toio - Copy parts of a framebuffer to display memory
* @dst: The display memory to copy to
* @dst_pitch: Number of bytes between two consecutive scanlines within dst
* @dst_format: FOURCC code of the display's color format
@@ -445,17 +539,14 @@ EXPORT_SYMBOL(drm_fb_xrgb8888_to_gray8);
* formats of the display and the framebuffer mismatch, the blit function
* will attempt to convert between them.
*
- * Use drm_fb_blit_dstclip() to copy the full framebuffer.
- *
* Returns:
* 0 on success, or
* -EINVAL if the color-format conversion failed, or
* a negative error code otherwise.
*/
-int drm_fb_blit_rect_dstclip(void __iomem *dst, unsigned int dst_pitch,
- uint32_t dst_format, void *vmap,
- struct drm_framebuffer *fb,
- struct drm_rect *clip)
+int drm_fb_blit_toio(void __iomem *dst, unsigned int dst_pitch, uint32_t dst_format,
+ const void *vmap, const struct drm_framebuffer *fb,
+ const struct drm_rect *clip)
{
uint32_t fb_format = fb->format->format;
@@ -464,58 +555,32 @@ int drm_fb_blit_rect_dstclip(void __iomem *dst, unsigned int dst_pitch,
fb_format = DRM_FORMAT_XRGB8888;
if (dst_format == DRM_FORMAT_ARGB8888)
dst_format = DRM_FORMAT_XRGB8888;
+ if (fb_format == DRM_FORMAT_ARGB2101010)
+ fb_format = DRM_FORMAT_XRGB2101010;
+ if (dst_format == DRM_FORMAT_ARGB2101010)
+ dst_format = DRM_FORMAT_XRGB2101010;
if (dst_format == fb_format) {
- drm_fb_memcpy_dstclip(dst, dst_pitch, vmap, fb, clip);
+ drm_fb_memcpy_toio(dst, dst_pitch, vmap, fb, clip);
return 0;
} else if (dst_format == DRM_FORMAT_RGB565) {
if (fb_format == DRM_FORMAT_XRGB8888) {
- drm_fb_xrgb8888_to_rgb565_dstclip(dst, dst_pitch,
- vmap, fb, clip,
- false);
+ drm_fb_xrgb8888_to_rgb565_toio(dst, dst_pitch, vmap, fb, clip, false);
return 0;
}
} else if (dst_format == DRM_FORMAT_RGB888) {
if (fb_format == DRM_FORMAT_XRGB8888) {
- drm_fb_xrgb8888_to_rgb888_dstclip(dst, dst_pitch,
- vmap, fb, clip);
+ drm_fb_xrgb8888_to_rgb888_toio(dst, dst_pitch, vmap, fb, clip);
+ return 0;
+ }
+ } else if (dst_format == DRM_FORMAT_XRGB2101010) {
+ if (fb_format == DRM_FORMAT_XRGB8888) {
+ drm_fb_xrgb8888_to_xrgb2101010_toio(dst, dst_pitch, vmap, fb, clip);
return 0;
}
}
return -EINVAL;
}
-EXPORT_SYMBOL(drm_fb_blit_rect_dstclip);
-
-/**
- * drm_fb_blit_dstclip - Copy framebuffer to display memory
- * @dst: The display memory to copy to
- * @dst_pitch: Number of bytes between two consecutive scanlines within dst
- * @dst_format: FOURCC code of the display's color format
- * @vmap: The framebuffer memory to copy from
- * @fb: The framebuffer to copy from
- *
- * This function copies a full framebuffer to display memory. If the formats
- * of the display and the framebuffer mismatch, the copy function will
- * attempt to convert between them.
- *
- * See drm_fb_blit_rect_dstclip() for more information.
- *
- * Returns:
- * 0 on success, or a negative error code otherwise.
- */
-int drm_fb_blit_dstclip(void __iomem *dst, unsigned int dst_pitch,
- uint32_t dst_format, void *vmap,
- struct drm_framebuffer *fb)
-{
- struct drm_rect fullscreen = {
- .x1 = 0,
- .x2 = fb->width,
- .y1 = 0,
- .y2 = fb->height,
- };
- return drm_fb_blit_rect_dstclip(dst, dst_pitch, dst_format, vmap, fb,
- &fullscreen);
-}
-EXPORT_SYMBOL(drm_fb_blit_dstclip);
+EXPORT_SYMBOL(drm_fb_blit_toio);
diff --git a/drivers/gpu/drm/drm_fourcc.c b/drivers/gpu/drm/drm_fourcc.c
index 25837b1..07741b6 100644
--- a/drivers/gpu/drm/drm_fourcc.c
+++ b/drivers/gpu/drm/drm_fourcc.c
@@ -269,6 +269,9 @@ const struct drm_format_info *__drm_format_info(u32 format)
.num_planes = 3, .char_per_block = { 2, 2, 2 },
.block_w = { 1, 1, 1 }, .block_h = { 1, 1, 1 }, .hsub = 0,
.vsub = 0, .is_yuv = true },
+ { .format = DRM_FORMAT_P030, .depth = 0, .num_planes = 2,
+ .char_per_block = { 4, 8, 0 }, .block_w = { 3, 3, 0 }, .block_h = { 1, 1, 0 },
+ .hsub = 2, .vsub = 2, .is_yuv = true},
};
unsigned int i;
diff --git a/drivers/gpu/drm/drm_gem_atomic_helper.c b/drivers/gpu/drm/drm_gem_atomic_helper.c
index e570398..c3189af 100644
--- a/drivers/gpu/drm/drm_gem_atomic_helper.c
+++ b/drivers/gpu/drm/drm_gem_atomic_helper.c
@@ -143,6 +143,7 @@
*/
int drm_gem_plane_helper_prepare_fb(struct drm_plane *plane, struct drm_plane_state *state)
{
+ struct dma_resv_iter cursor;
struct drm_gem_object *obj;
struct dma_fence *fence;
@@ -150,9 +151,18 @@ int drm_gem_plane_helper_prepare_fb(struct drm_plane *plane, struct drm_plane_st
return 0;
obj = drm_gem_fb_get_obj(state->fb, 0);
- fence = dma_resv_get_excl_unlocked(obj->resv);
- drm_atomic_set_fence_for_plane(state, fence);
+ dma_resv_iter_begin(&cursor, obj->resv, false);
+ dma_resv_for_each_fence_unlocked(&cursor, fence) {
+ /* TODO: Currently there should be only one write fence, so this
+ * here works fine. But drm_atomic_set_fence_for_plane() should
+ * be changed to be able to handle more fences in general for
+ * multiple BOs per fb anyway. */
+ dma_fence_get(fence);
+ break;
+ }
+ dma_resv_iter_end(&cursor);
+ drm_atomic_set_fence_for_plane(state, fence);
return 0;
}
EXPORT_SYMBOL_GPL(drm_gem_plane_helper_prepare_fb);
diff --git a/drivers/gpu/drm/drm_gem_cma_helper.c b/drivers/gpu/drm/drm_gem_cma_helper.c
index 9d05674..cefd0cb 100644
--- a/drivers/gpu/drm/drm_gem_cma_helper.c
+++ b/drivers/gpu/drm/drm_gem_cma_helper.c
@@ -13,6 +13,7 @@
#include <linux/dma-mapping.h>
#include <linux/export.h>
#include <linux/mm.h>
+#include <linux/module.h>
#include <linux/mutex.h>
#include <linux/slab.h>
@@ -31,14 +32,18 @@
* The DRM GEM/CMA helpers use this allocator as a means to provide buffer
* objects that are physically contiguous in memory. This is useful for
* display drivers that are unable to map scattered buffers via an IOMMU.
+ *
+ * For GEM callback helpers in struct &drm_gem_object functions, see likewise
+ * named functions with an _object_ infix (e.g., drm_gem_cma_object_vmap() wraps
+ * drm_gem_cma_vmap()). These helpers perform the necessary type conversion.
*/
static const struct drm_gem_object_funcs drm_gem_cma_default_funcs = {
- .free = drm_gem_cma_free_object,
- .print_info = drm_gem_cma_print_info,
- .get_sg_table = drm_gem_cma_get_sg_table,
- .vmap = drm_gem_cma_vmap,
- .mmap = drm_gem_cma_mmap,
+ .free = drm_gem_cma_object_free,
+ .print_info = drm_gem_cma_object_print_info,
+ .get_sg_table = drm_gem_cma_object_get_sg_table,
+ .vmap = drm_gem_cma_object_vmap,
+ .mmap = drm_gem_cma_object_mmap,
.vm_ops = &drm_gem_cma_vm_ops,
};
@@ -62,18 +67,21 @@ __drm_gem_cma_create(struct drm_device *drm, size_t size, bool private)
struct drm_gem_object *gem_obj;
int ret = 0;
- if (drm->driver->gem_create_object)
+ if (drm->driver->gem_create_object) {
gem_obj = drm->driver->gem_create_object(drm, size);
- else
- gem_obj = kzalloc(sizeof(*cma_obj), GFP_KERNEL);
- if (!gem_obj)
- return ERR_PTR(-ENOMEM);
+ if (IS_ERR(gem_obj))
+ return ERR_CAST(gem_obj);
+ cma_obj = to_drm_gem_cma_obj(gem_obj);
+ } else {
+ cma_obj = kzalloc(sizeof(*cma_obj), GFP_KERNEL);
+ if (!cma_obj)
+ return ERR_PTR(-ENOMEM);
+ gem_obj = &cma_obj->base;
+ }
if (!gem_obj->funcs)
gem_obj->funcs = &drm_gem_cma_default_funcs;
- cma_obj = container_of(gem_obj, struct drm_gem_cma_object, base);
-
if (private) {
drm_gem_private_object_init(drm, gem_obj, size);
@@ -191,18 +199,16 @@ drm_gem_cma_create_with_handle(struct drm_file *file_priv,
}
/**
- * drm_gem_cma_free_object - free resources associated with a CMA GEM object
- * @gem_obj: GEM object to free
+ * drm_gem_cma_free - free resources associated with a CMA GEM object
+ * @cma_obj: CMA GEM object to free
*
* This function frees the backing memory of the CMA GEM object, cleans up the
* GEM object state and frees the memory used to store the object itself.
* If the buffer is imported and the virtual address is set, it is released.
- * Drivers using the CMA helpers should set this as their
- * &drm_gem_object_funcs.free callback.
*/
-void drm_gem_cma_free_object(struct drm_gem_object *gem_obj)
+void drm_gem_cma_free(struct drm_gem_cma_object *cma_obj)
{
- struct drm_gem_cma_object *cma_obj = to_drm_gem_cma_obj(gem_obj);
+ struct drm_gem_object *gem_obj = &cma_obj->base;
struct dma_buf_map map = DMA_BUF_MAP_INIT_VADDR(cma_obj->vaddr);
if (gem_obj->import_attach) {
@@ -223,7 +229,7 @@ void drm_gem_cma_free_object(struct drm_gem_object *gem_obj)
kfree(cma_obj);
}
-EXPORT_SYMBOL_GPL(drm_gem_cma_free_object);
+EXPORT_SYMBOL_GPL(drm_gem_cma_free);
/**
* drm_gem_cma_dumb_create_internal - create a dumb buffer object
@@ -370,18 +376,15 @@ EXPORT_SYMBOL_GPL(drm_gem_cma_get_unmapped_area);
/**
* drm_gem_cma_print_info() - Print &drm_gem_cma_object info for debugfs
+ * @cma_obj: CMA GEM object
* @p: DRM printer
* @indent: Tab indentation level
- * @obj: GEM object
*
- * This function can be used as the &drm_driver->gem_print_info callback.
- * It prints paddr and vaddr for use in e.g. debugfs output.
+ * This function prints paddr and vaddr for use in e.g. debugfs output.
*/
-void drm_gem_cma_print_info(struct drm_printer *p, unsigned int indent,
- const struct drm_gem_object *obj)
+void drm_gem_cma_print_info(const struct drm_gem_cma_object *cma_obj,
+ struct drm_printer *p, unsigned int indent)
{
- const struct drm_gem_cma_object *cma_obj = to_drm_gem_cma_obj(obj);
-
drm_printf_indent(p, indent, "paddr=%pad\n", &cma_obj->paddr);
drm_printf_indent(p, indent, "vaddr=%p\n", cma_obj->vaddr);
}
@@ -390,18 +393,17 @@ EXPORT_SYMBOL(drm_gem_cma_print_info);
/**
* drm_gem_cma_get_sg_table - provide a scatter/gather table of pinned
* pages for a CMA GEM object
- * @obj: GEM object
+ * @cma_obj: CMA GEM object
*
- * This function exports a scatter/gather table by
- * calling the standard DMA mapping API. Drivers using the CMA helpers should
- * set this as their &drm_gem_object_funcs.get_sg_table callback.
+ * This function exports a scatter/gather table by calling the standard
+ * DMA mapping API.
*
* Returns:
* A pointer to the scatter/gather table of pinned pages or NULL on failure.
*/
-struct sg_table *drm_gem_cma_get_sg_table(struct drm_gem_object *obj)
+struct sg_table *drm_gem_cma_get_sg_table(struct drm_gem_cma_object *cma_obj)
{
- struct drm_gem_cma_object *cma_obj = to_drm_gem_cma_obj(obj);
+ struct drm_gem_object *obj = &cma_obj->base;
struct sg_table *sgt;
int ret;
@@ -467,23 +469,19 @@ EXPORT_SYMBOL_GPL(drm_gem_cma_prime_import_sg_table);
/**
* drm_gem_cma_vmap - map a CMA GEM object into the kernel's virtual
* address space
- * @obj: GEM object
+ * @cma_obj: CMA GEM object
* @map: Returns the kernel virtual address of the CMA GEM object's backing
* store.
*
- * This function maps a buffer into the kernel's
- * virtual address space. Since the CMA buffers are already mapped into the
- * kernel virtual address space this simply returns the cached virtual
- * address. Drivers using the CMA helpers should set this as their DRM
- * driver's &drm_gem_object_funcs.vmap callback.
+ * This function maps a buffer into the kernel's virtual address space.
+ * Since the CMA buffers are already mapped into the kernel virtual address
+ * space this simply returns the cached virtual address.
*
* Returns:
* 0 on success, or a negative error code otherwise.
*/
-int drm_gem_cma_vmap(struct drm_gem_object *obj, struct dma_buf_map *map)
+int drm_gem_cma_vmap(struct drm_gem_cma_object *cma_obj, struct dma_buf_map *map)
{
- struct drm_gem_cma_object *cma_obj = to_drm_gem_cma_obj(obj);
-
dma_buf_map_set_vaddr(map, cma_obj->vaddr);
return 0;
@@ -492,20 +490,19 @@ EXPORT_SYMBOL_GPL(drm_gem_cma_vmap);
/**
* drm_gem_cma_mmap - memory-map an exported CMA GEM object
- * @obj: GEM object
+ * @cma_obj: CMA GEM object
* @vma: VMA for the area to be mapped
*
* This function maps a buffer into a userspace process's address space.
* In addition to the usual GEM VMA setup it immediately faults in the entire
- * object instead of using on-demand faulting. Drivers that use the CMA
- * helpers should set this as their &drm_gem_object_funcs.mmap callback.
+ * object instead of using on-demand faulting.
*
* Returns:
* 0 on success or a negative error code on failure.
*/
-int drm_gem_cma_mmap(struct drm_gem_object *obj, struct vm_area_struct *vma)
+int drm_gem_cma_mmap(struct drm_gem_cma_object *cma_obj, struct vm_area_struct *vma)
{
- struct drm_gem_cma_object *cma_obj;
+ struct drm_gem_object *obj = &cma_obj->base;
int ret;
/*
@@ -516,8 +513,6 @@ int drm_gem_cma_mmap(struct drm_gem_object *obj, struct vm_area_struct *vma)
vma->vm_pgoff -= drm_vma_node_start(&obj->vma_node);
vma->vm_flags &= ~VM_PFNMAP;
- cma_obj = to_drm_gem_cma_obj(obj);
-
if (cma_obj->map_noncoherent) {
vma->vm_page_prot = vm_get_page_prot(vma->vm_flags);
@@ -583,3 +578,7 @@ drm_gem_cma_prime_import_sg_table_vmap(struct drm_device *dev,
return obj;
}
EXPORT_SYMBOL(drm_gem_cma_prime_import_sg_table_vmap);
+
+MODULE_DESCRIPTION("DRM CMA memory-management helpers");
+MODULE_IMPORT_NS(DMA_BUF);
+MODULE_LICENSE("GPL");
diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
index bca0de9..62192411 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -5,6 +5,7 @@
#include <linux/dma-buf.h>
#include <linux/export.h>
+#include <linux/module.h>
#include <linux/mutex.h>
#include <linux/shmem_fs.h>
#include <linux/slab.h>
@@ -29,17 +30,22 @@ MODULE_IMPORT_NS(DMA_BUF);
*
* This library provides helpers for GEM objects backed by shmem buffers
* allocated using anonymous pageable memory.
+ *
+ * Functions that operate on the GEM object receive struct &drm_gem_shmem_object.
+ * For GEM callback helpers in struct &drm_gem_object functions, see likewise
+ * named functions with an _object_ infix (e.g., drm_gem_shmem_object_vmap() wraps
+ * drm_gem_shmem_vmap()). These helpers perform the necessary type conversion.
*/
static const struct drm_gem_object_funcs drm_gem_shmem_funcs = {
- .free = drm_gem_shmem_free_object,
- .print_info = drm_gem_shmem_print_info,
- .pin = drm_gem_shmem_pin,
- .unpin = drm_gem_shmem_unpin,
- .get_sg_table = drm_gem_shmem_get_sg_table,
- .vmap = drm_gem_shmem_vmap,
- .vunmap = drm_gem_shmem_vunmap,
- .mmap = drm_gem_shmem_mmap,
+ .free = drm_gem_shmem_object_free,
+ .print_info = drm_gem_shmem_object_print_info,
+ .pin = drm_gem_shmem_object_pin,
+ .unpin = drm_gem_shmem_object_unpin,
+ .get_sg_table = drm_gem_shmem_object_get_sg_table,
+ .vmap = drm_gem_shmem_object_vmap,
+ .vunmap = drm_gem_shmem_object_vunmap,
+ .mmap = drm_gem_shmem_object_mmap,
};
static struct drm_gem_shmem_object *
@@ -51,14 +57,17 @@ __drm_gem_shmem_create(struct drm_device *dev, size_t size, bool private)
size = PAGE_ALIGN(size);
- if (dev->driver->gem_create_object)
+ if (dev->driver->gem_create_object) {
obj = dev->driver->gem_create_object(dev, size);
- else
- obj = kzalloc(sizeof(*shmem), GFP_KERNEL);
- if (!obj)
- return ERR_PTR(-ENOMEM);
-
- shmem = to_drm_gem_shmem_obj(obj);
+ if (IS_ERR(obj))
+ return ERR_CAST(obj);
+ shmem = to_drm_gem_shmem_obj(obj);
+ } else {
+ shmem = kzalloc(sizeof(*shmem), GFP_KERNEL);
+ if (!shmem)
+ return ERR_PTR(-ENOMEM);
+ obj = &shmem->base;
+ }
if (!obj->funcs)
obj->funcs = &drm_gem_shmem_funcs;
@@ -119,16 +128,15 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev, size_t
EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
/**
- * drm_gem_shmem_free_object - Free resources associated with a shmem GEM object
- * @obj: GEM object to free
+ * drm_gem_shmem_free - Free resources associated with a shmem GEM object
+ * @shmem: shmem GEM object to free
*
* This function cleans up the GEM object state and frees the memory used to
- * store the object itself. It should be used to implement
- * &drm_gem_object_funcs.free.
+ * store the object itself.
*/
-void drm_gem_shmem_free_object(struct drm_gem_object *obj)
+void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
{
- struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
+ struct drm_gem_object *obj = &shmem->base;
WARN_ON(shmem->vmap_use_count);
@@ -152,7 +160,7 @@ void drm_gem_shmem_free_object(struct drm_gem_object *obj)
mutex_destroy(&shmem->vmap_lock);
kfree(shmem);
}
-EXPORT_SYMBOL_GPL(drm_gem_shmem_free_object);
+EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
static int drm_gem_shmem_get_pages_locked(struct drm_gem_shmem_object *shmem)
{
@@ -247,19 +255,16 @@ EXPORT_SYMBOL(drm_gem_shmem_put_pages);
/**
* drm_gem_shmem_pin - Pin backing pages for a shmem GEM object
- * @obj: GEM object
+ * @shmem: shmem GEM object
*
* This function makes sure the backing pages are pinned in memory while the
- * buffer is exported. It should only be used to implement
- * &drm_gem_object_funcs.pin.
+ * buffer is exported.
*
* Returns:
* 0 on success or a negative error code on failure.
*/
-int drm_gem_shmem_pin(struct drm_gem_object *obj)
+int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
{
- struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
-
WARN_ON(shmem->base.import_attach);
return drm_gem_shmem_get_pages(shmem);
@@ -268,15 +273,13 @@ EXPORT_SYMBOL(drm_gem_shmem_pin);
/**
* drm_gem_shmem_unpin - Unpin backing pages for a shmem GEM object
- * @obj: GEM object
+ * @shmem: shmem GEM object
*
* This function removes the requirement that the backing pages are pinned in
- * memory. It should only be used to implement &drm_gem_object_funcs.unpin.
+ * memory.
*/
-void drm_gem_shmem_unpin(struct drm_gem_object *obj)
+void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
{
- struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
-
WARN_ON(shmem->base.import_attach);
drm_gem_shmem_put_pages(shmem);
@@ -342,20 +345,16 @@ static int drm_gem_shmem_vmap_locked(struct drm_gem_shmem_object *shmem, struct
* store.
*
* This function makes sure that a contiguous kernel virtual address mapping
- * exists for the buffer backing the shmem GEM object.
- *
- * This function can be used to implement &drm_gem_object_funcs.vmap. But it can
- * also be called by drivers directly, in which case it will hide the
- * differences between dma-buf imported and natively allocated objects.
+ * exists for the buffer backing the shmem GEM object. It hides the differences
+ * between dma-buf imported and natively allocated objects.
*
* Acquired mappings should be cleaned up by calling drm_gem_shmem_vunmap().
*
* Returns:
* 0 on success or a negative error code on failure.
*/
-int drm_gem_shmem_vmap(struct drm_gem_object *obj, struct dma_buf_map *map)
+int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem, struct dma_buf_map *map)
{
- struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
int ret;
ret = mutex_lock_interruptible(&shmem->vmap_lock);
@@ -398,21 +397,18 @@ static void drm_gem_shmem_vunmap_locked(struct drm_gem_shmem_object *shmem,
* drm_gem_shmem_vmap(). The mapping is only removed when the use count drops to
* zero.
*
- * This function can be used to implement &drm_gem_object_funcs.vmap. But it can
- * also be called by drivers directly, in which case it will hide the
- * differences between dma-buf imported and natively allocated objects.
+ * This function hides the differences between dma-buf imported and natively
+ * allocated objects.
*/
-void drm_gem_shmem_vunmap(struct drm_gem_object *obj, struct dma_buf_map *map)
+void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem, struct dma_buf_map *map)
{
- struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
-
mutex_lock(&shmem->vmap_lock);
drm_gem_shmem_vunmap_locked(shmem, map);
mutex_unlock(&shmem->vmap_lock);
}
EXPORT_SYMBOL(drm_gem_shmem_vunmap);
-struct drm_gem_shmem_object *
+static struct drm_gem_shmem_object *
drm_gem_shmem_create_with_handle(struct drm_file *file_priv,
struct drm_device *dev, size_t size,
uint32_t *handle)
@@ -436,15 +432,12 @@ drm_gem_shmem_create_with_handle(struct drm_file *file_priv,
return shmem;
}
-EXPORT_SYMBOL(drm_gem_shmem_create_with_handle);
/* Update madvise status, returns true if not purged, else
* false or -errno.
*/
-int drm_gem_shmem_madvise(struct drm_gem_object *obj, int madv)
+int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv)
{
- struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
-
mutex_lock(&shmem->pages_lock);
if (shmem->madv >= 0)
@@ -458,14 +451,14 @@ int drm_gem_shmem_madvise(struct drm_gem_object *obj, int madv)
}
EXPORT_SYMBOL(drm_gem_shmem_madvise);
-void drm_gem_shmem_purge_locked(struct drm_gem_object *obj)
+void drm_gem_shmem_purge_locked(struct drm_gem_shmem_object *shmem)
{
+ struct drm_gem_object *obj = &shmem->base;
struct drm_device *dev = obj->dev;
- struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
- dma_unmap_sgtable(obj->dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
+ dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
sg_free_table(shmem->sgt);
kfree(shmem->sgt);
shmem->sgt = NULL;
@@ -484,18 +477,15 @@ void drm_gem_shmem_purge_locked(struct drm_gem_object *obj)
*/
shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
- invalidate_mapping_pages(file_inode(obj->filp)->i_mapping,
- 0, (loff_t)-1);
+ invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
}
EXPORT_SYMBOL(drm_gem_shmem_purge_locked);
-bool drm_gem_shmem_purge(struct drm_gem_object *obj)
+bool drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
{
- struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
-
if (!mutex_trylock(&shmem->pages_lock))
return false;
- drm_gem_shmem_purge_locked(obj);
+ drm_gem_shmem_purge_locked(shmem);
mutex_unlock(&shmem->pages_lock);
return true;
@@ -603,19 +593,18 @@ static const struct vm_operations_struct drm_gem_shmem_vm_ops = {
/**
* drm_gem_shmem_mmap - Memory-map a shmem GEM object
- * @obj: gem object
+ * @shmem: shmem GEM object
* @vma: VMA for the area to be mapped
*
* This function implements an augmented version of the GEM DRM file mmap
- * operation for shmem objects. Drivers which employ the shmem helpers should
- * use this function as their &drm_gem_object_funcs.mmap handler.
+ * operation for shmem objects.
*
* Returns:
* 0 on success or a negative error code on failure.
*/
-int drm_gem_shmem_mmap(struct drm_gem_object *obj, struct vm_area_struct *vma)
+int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct *vma)
{
- struct drm_gem_shmem_object *shmem;
+ struct drm_gem_object *obj = &shmem->base;
int ret;
if (obj->import_attach) {
@@ -626,8 +615,6 @@ int drm_gem_shmem_mmap(struct drm_gem_object *obj, struct vm_area_struct *vma)
return dma_buf_mmap(obj->dma_buf, vma, 0);
}
- shmem = to_drm_gem_shmem_obj(obj);
-
ret = drm_gem_shmem_get_pages(shmem);
if (ret) {
drm_gem_vm_close(vma);
@@ -646,17 +633,13 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_mmap);
/**
* drm_gem_shmem_print_info() - Print &drm_gem_shmem_object info for debugfs
+ * @shmem: shmem GEM object
* @p: DRM printer
* @indent: Tab indentation level
- * @obj: GEM object
- *
- * This implements the &drm_gem_object_funcs.info callback.
*/
-void drm_gem_shmem_print_info(struct drm_printer *p, unsigned int indent,
- const struct drm_gem_object *obj)
+void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
+ struct drm_printer *p, unsigned int indent)
{
- const struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
-
drm_printf_indent(p, indent, "pages_use_count=%u\n", shmem->pages_use_count);
drm_printf_indent(p, indent, "vmap_use_count=%u\n", shmem->vmap_use_count);
drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
@@ -666,12 +649,10 @@ EXPORT_SYMBOL(drm_gem_shmem_print_info);
/**
* drm_gem_shmem_get_sg_table - Provide a scatter/gather table of pinned
* pages for a shmem GEM object
- * @obj: GEM object
+ * @shmem: shmem GEM object
*
* This function exports a scatter/gather table suitable for PRIME usage by
- * calling the standard DMA mapping API. Drivers should not call this function
- * directly, instead it should only be used as an implementation for
- * &drm_gem_object_funcs.get_sg_table.
+ * calling the standard DMA mapping API.
*
* Drivers who need to acquire an scatter/gather table for objects need to call
* drm_gem_shmem_get_pages_sgt() instead.
@@ -679,9 +660,9 @@ EXPORT_SYMBOL(drm_gem_shmem_print_info);
* Returns:
* A pointer to the scatter/gather table of pinned pages or NULL on failure.
*/
-struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_object *obj)
+struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem)
{
- struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
+ struct drm_gem_object *obj = &shmem->base;
WARN_ON(shmem->base.import_attach);
@@ -692,7 +673,7 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_get_sg_table);
/**
* drm_gem_shmem_get_pages_sgt - Pin pages, dma map them, and return a
* scatter/gather table for a shmem GEM object.
- * @obj: GEM object
+ * @shmem: shmem GEM object
*
* This function returns a scatter/gather table suitable for driver usage. If
* the sg table doesn't exist, the pages are pinned, dma-mapped, and a sg
@@ -705,10 +686,10 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_get_sg_table);
* Returns:
* A pointer to the scatter/gather table of pinned pages or errno on failure.
*/
-struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_object *obj)
+struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
{
+ struct drm_gem_object *obj = &shmem->base;
int ret;
- struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
struct sg_table *sgt;
if (shmem->sgt)
@@ -720,7 +701,7 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_object *obj)
if (ret)
return ERR_PTR(ret);
- sgt = drm_gem_shmem_get_sg_table(&shmem->base);
+ sgt = drm_gem_shmem_get_sg_table(shmem);
if (IS_ERR(sgt)) {
ret = PTR_ERR(sgt);
goto err_put_pages;
@@ -777,3 +758,7 @@ drm_gem_shmem_prime_import_sg_table(struct drm_device *dev,
return &shmem->base;
}
EXPORT_SYMBOL_GPL(drm_gem_shmem_prime_import_sg_table);
+
+MODULE_DESCRIPTION("DRM SHMEM memory-management helpers");
+MODULE_IMPORT_NS(DMA_BUF);
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/gpu/drm/drm_gem_vram_helper.c b/drivers/gpu/drm/drm_gem_vram_helper.c
index bfa386b..3f00192 100644
--- a/drivers/gpu/drm/drm_gem_vram_helper.c
+++ b/drivers/gpu/drm/drm_gem_vram_helper.c
@@ -197,8 +197,8 @@ struct drm_gem_vram_object *drm_gem_vram_create(struct drm_device *dev,
if (dev->driver->gem_create_object) {
gem = dev->driver->gem_create_object(dev, size);
- if (!gem)
- return ERR_PTR(-ENOMEM);
+ if (IS_ERR(gem))
+ return ERR_CAST(gem);
gbo = drm_gem_vram_of_gem(gem);
} else {
gbo = kzalloc(sizeof(*gbo), GFP_KERNEL);
diff --git a/drivers/gpu/drm/drm_hashtab.c b/drivers/gpu/drm/drm_hashtab.c
index c50fa6f..60afa18 100644
--- a/drivers/gpu/drm/drm_hashtab.c
+++ b/drivers/gpu/drm/drm_hashtab.c
@@ -32,16 +32,16 @@
* Thomas Hellström <thomas-at-tungstengraphics-dot-com>
*/
-#include <linux/export.h>
#include <linux/hash.h>
#include <linux/mm.h>
#include <linux/rculist.h>
#include <linux/slab.h>
#include <linux/vmalloc.h>
-#include <drm/drm_hashtab.h>
#include <drm/drm_print.h>
+#include "drm_legacy.h"
+
int drm_ht_create(struct drm_open_hash *ht, unsigned int order)
{
unsigned int size = 1 << order;
@@ -58,7 +58,6 @@ int drm_ht_create(struct drm_open_hash *ht, unsigned int order)
}
return 0;
}
-EXPORT_SYMBOL(drm_ht_create);
void drm_ht_verbose_list(struct drm_open_hash *ht, unsigned long key)
{
@@ -135,7 +134,6 @@ int drm_ht_insert_item(struct drm_open_hash *ht, struct drm_hash_item *item)
}
return 0;
}
-EXPORT_SYMBOL(drm_ht_insert_item);
/*
* Just insert an item and return any "bits" bit key that hasn't been
@@ -164,7 +162,6 @@ int drm_ht_just_insert_please(struct drm_open_hash *ht, struct drm_hash_item *it
}
return 0;
}
-EXPORT_SYMBOL(drm_ht_just_insert_please);
int drm_ht_find_item(struct drm_open_hash *ht, unsigned long key,
struct drm_hash_item **item)
@@ -178,7 +175,6 @@ int drm_ht_find_item(struct drm_open_hash *ht, unsigned long key,
*item = hlist_entry(list, struct drm_hash_item, head);
return 0;
}
-EXPORT_SYMBOL(drm_ht_find_item);
int drm_ht_remove_key(struct drm_open_hash *ht, unsigned long key)
{
@@ -197,7 +193,6 @@ int drm_ht_remove_item(struct drm_open_hash *ht, struct drm_hash_item *item)
hlist_del_init_rcu(&item->head);
return 0;
}
-EXPORT_SYMBOL(drm_ht_remove_item);
void drm_ht_remove(struct drm_open_hash *ht)
{
@@ -206,4 +201,3 @@ void drm_ht_remove(struct drm_open_hash *ht)
ht->table = NULL;
}
}
-EXPORT_SYMBOL(drm_ht_remove);
diff --git a/drivers/gpu/drm/drm_irq.c b/drivers/gpu/drm/drm_irq.c
index 13e1d5c..d327638 100644
--- a/drivers/gpu/drm/drm_irq.c
+++ b/drivers/gpu/drm/drm_irq.c
@@ -66,7 +66,6 @@
#include "drm_internal.h"
-#if IS_ENABLED(CONFIG_DRM_LEGACY)
static int drm_legacy_irq_install(struct drm_device *dev, int irq)
{
int ret;
@@ -203,4 +202,3 @@ int drm_legacy_irq_control(struct drm_device *dev, void *data,
return -EINVAL;
}
}
-#endif
diff --git a/drivers/gpu/drm/drm_legacy.h b/drivers/gpu/drm/drm_legacy.h
index c920684..70c9dba 100644
--- a/drivers/gpu/drm/drm_legacy.h
+++ b/drivers/gpu/drm/drm_legacy.h
@@ -35,9 +35,47 @@
#include <drm/drm_legacy.h>
struct agp_memory;
+struct drm_buf_desc;
struct drm_device;
struct drm_file;
-struct drm_buf_desc;
+struct drm_hash_item;
+struct drm_open_hash;
+
+/*
+ * Hash-table Support
+ */
+
+#define drm_hash_entry(_ptr, _type, _member) container_of(_ptr, _type, _member)
+
+/* drm_hashtab.c */
+#if IS_ENABLED(CONFIG_DRM_LEGACY)
+int drm_ht_create(struct drm_open_hash *ht, unsigned int order);
+int drm_ht_insert_item(struct drm_open_hash *ht, struct drm_hash_item *item);
+int drm_ht_just_insert_please(struct drm_open_hash *ht, struct drm_hash_item *item,
+ unsigned long seed, int bits, int shift,
+ unsigned long add);
+int drm_ht_find_item(struct drm_open_hash *ht, unsigned long key, struct drm_hash_item **item);
+
+void drm_ht_verbose_list(struct drm_open_hash *ht, unsigned long key);
+int drm_ht_remove_key(struct drm_open_hash *ht, unsigned long key);
+int drm_ht_remove_item(struct drm_open_hash *ht, struct drm_hash_item *item);
+void drm_ht_remove(struct drm_open_hash *ht);
+#endif
+
+/*
+ * RCU-safe interface
+ *
+ * The user of this API needs to make sure that two or more instances of the
+ * hash table manipulation functions are never run simultaneously.
+ * The lookup function drm_ht_find_item_rcu may, however, run simultaneously
+ * with any of the manipulation functions as long as it's called from within
+ * an RCU read-locked section.
+ */
+#define drm_ht_insert_item_rcu drm_ht_insert_item
+#define drm_ht_just_insert_please_rcu drm_ht_just_insert_please
+#define drm_ht_remove_key_rcu drm_ht_remove_key
+#define drm_ht_remove_item_rcu drm_ht_remove_item
+#define drm_ht_find_item_rcu drm_ht_find_item
/*
* Generic DRM Contexts
diff --git a/drivers/gpu/drm/drm_mipi_dbi.c b/drivers/gpu/drm/drm_mipi_dbi.c
index 71b646c..ded8968 100644
--- a/drivers/gpu/drm/drm_mipi_dbi.c
+++ b/drivers/gpu/drm/drm_mipi_dbi.c
@@ -15,9 +15,10 @@
#include <drm/drm_connector.h>
#include <drm/drm_damage_helper.h>
#include <drm/drm_drv.h>
-#include <drm/drm_gem_cma_helper.h>
+#include <drm/drm_file.h>
#include <drm/drm_format_helper.h>
#include <drm/drm_fourcc.h>
+#include <drm/drm_gem.h>
#include <drm/drm_gem_framebuffer_helper.h>
#include <drm/drm_mipi_dbi.h>
#include <drm/drm_modes.h>
@@ -200,30 +201,38 @@ int mipi_dbi_buf_copy(void *dst, struct drm_framebuffer *fb,
struct drm_rect *clip, bool swap)
{
struct drm_gem_object *gem = drm_gem_fb_get_obj(fb, 0);
- struct drm_gem_cma_object *cma_obj = to_drm_gem_cma_obj(gem);
- void *src = cma_obj->vaddr;
+ struct dma_buf_map map[DRM_FORMAT_MAX_PLANES];
+ struct dma_buf_map data[DRM_FORMAT_MAX_PLANES];
+ void *src;
int ret;
ret = drm_gem_fb_begin_cpu_access(fb, DMA_FROM_DEVICE);
if (ret)
return ret;
+ src = data[0].vaddr; /* TODO: Use mapping abstraction properly */
+
+ ret = drm_gem_fb_vmap(fb, map, data);
+ if (ret)
+ goto out_drm_gem_fb_end_cpu_access;
switch (fb->format->format) {
case DRM_FORMAT_RGB565:
if (swap)
- drm_fb_swab(dst, src, fb, clip, !gem->import_attach);
+ drm_fb_swab(dst, 0, src, fb, clip, !gem->import_attach);
else
- drm_fb_memcpy(dst, src, fb, clip);
+ drm_fb_memcpy(dst, 0, src, fb, clip);
break;
case DRM_FORMAT_XRGB8888:
- drm_fb_xrgb8888_to_rgb565(dst, src, fb, clip, swap);
+ drm_fb_xrgb8888_to_rgb565(dst, 0, src, fb, clip, swap);
break;
default:
drm_err_once(fb->dev, "Format is not supported: %p4cc\n",
&fb->format->format);
- return -EINVAL;
+ ret = -EINVAL;
}
+ drm_gem_fb_vunmap(fb, map);
+out_drm_gem_fb_end_cpu_access:
drm_gem_fb_end_cpu_access(fb, DMA_FROM_DEVICE);
return ret;
@@ -249,8 +258,8 @@ static void mipi_dbi_set_window_address(struct mipi_dbi_dev *dbidev,
static void mipi_dbi_fb_dirty(struct drm_framebuffer *fb, struct drm_rect *rect)
{
- struct drm_gem_object *gem = drm_gem_fb_get_obj(fb, 0);
- struct drm_gem_cma_object *cma_obj = to_drm_gem_cma_obj(gem);
+ struct dma_buf_map map[DRM_FORMAT_MAX_PLANES];
+ struct dma_buf_map data[DRM_FORMAT_MAX_PLANES];
struct mipi_dbi_dev *dbidev = drm_to_mipi_dbi_dev(fb->dev);
unsigned int height = rect->y2 - rect->y1;
unsigned int width = rect->x2 - rect->x1;
@@ -266,6 +275,10 @@ static void mipi_dbi_fb_dirty(struct drm_framebuffer *fb, struct drm_rect *rect)
if (!drm_dev_enter(fb->dev, &idx))
return;
+ ret = drm_gem_fb_vmap(fb, map, data);
+ if (ret)
+ goto err_drm_dev_exit;
+
full = width == fb->width && height == fb->height;
DRM_DEBUG_KMS("Flushing [FB:%d] " DRM_RECT_FMT "\n", fb->base.id, DRM_RECT_ARG(rect));
@@ -277,7 +290,7 @@ static void mipi_dbi_fb_dirty(struct drm_framebuffer *fb, struct drm_rect *rect)
if (ret)
goto err_msg;
} else {
- tr = cma_obj->vaddr;
+ tr = data[0].vaddr; /* TODO: Use mapping abstraction properly */
}
mipi_dbi_set_window_address(dbidev, rect->x1, rect->x2 - 1, rect->y1,
@@ -289,6 +302,9 @@ static void mipi_dbi_fb_dirty(struct drm_framebuffer *fb, struct drm_rect *rect)
if (ret)
drm_err_once(fb->dev, "Failed to update display %d\n", ret);
+ drm_gem_fb_vunmap(fb, map);
+
+err_drm_dev_exit:
drm_dev_exit(idx);
}
@@ -1117,8 +1133,8 @@ int mipi_dbi_spi_init(struct spi_device *spi, struct mipi_dbi *dbi,
/*
* Even though it's not the SPI device that does DMA (the master does),
- * the dma mask is necessary for the dma_alloc_wc() in
- * drm_gem_cma_create(). The dma_addr returned will be a physical
+ * the dma mask is necessary for the dma_alloc_wc() in the GEM code
+ * (e.g., drm_gem_cma_create()). The dma_addr returned will be a physical
* address which might be different from the bus address, but this is
* not a problem since the address will not be used.
* The virtual address is used in the transfer and the SPI core
diff --git a/drivers/gpu/drm/drm_nomodeset.c b/drivers/gpu/drm/drm_nomodeset.c
new file mode 100644
index 0000000..f3978d5
--- /dev/null
+++ b/drivers/gpu/drm/drm_nomodeset.c
@@ -0,0 +1,24 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <linux/module.h>
+#include <linux/types.h>
+
+static bool drm_nomodeset;
+
+bool drm_firmware_drivers_only(void)
+{
+ return drm_nomodeset;
+}
+EXPORT_SYMBOL(drm_firmware_drivers_only);
+
+static int __init disable_modeset(char *str)
+{
+ drm_nomodeset = true;
+
+ pr_warn("Booted with the nomodeset parameter. Only the system framebuffer will be available\n");
+
+ return 1;
+}
+
+/* Disable kernel modesetting */
+__setup("nomodeset", disable_modeset);
diff --git a/drivers/gpu/drm/drm_of.c b/drivers/gpu/drm/drm_of.c
index 37c3414..59d368e 100644
--- a/drivers/gpu/drm/drm_of.c
+++ b/drivers/gpu/drm/drm_of.c
@@ -402,3 +402,36 @@ int drm_of_lvds_get_dual_link_pixel_order(const struct device_node *port1,
DRM_LVDS_DUAL_LINK_ODD_EVEN_PIXELS;
}
EXPORT_SYMBOL_GPL(drm_of_lvds_get_dual_link_pixel_order);
+
+/**
+ * drm_of_lvds_get_data_mapping - Get LVDS data mapping
+ * @port: DT port node of the LVDS source or sink
+ *
+ * Convert DT "data-mapping" property string value into media bus format value.
+ *
+ * Return:
+ * * MEDIA_BUS_FMT_RGB666_1X7X3_SPWG - data-mapping is "jeida-18"
+ * * MEDIA_BUS_FMT_RGB888_1X7X4_JEIDA - data-mapping is "jeida-24"
+ * * MEDIA_BUS_FMT_RGB888_1X7X4_SPWG - data-mapping is "vesa-24"
+ * * -EINVAL - the "data-mapping" property is unsupported
+ * * -ENODEV - the "data-mapping" property is missing
+ */
+int drm_of_lvds_get_data_mapping(const struct device_node *port)
+{
+ const char *mapping;
+ int ret;
+
+ ret = of_property_read_string(port, "data-mapping", &mapping);
+ if (ret < 0)
+ return -ENODEV;
+
+ if (!strcmp(mapping, "jeida-18"))
+ return MEDIA_BUS_FMT_RGB666_1X7X3_SPWG;
+ if (!strcmp(mapping, "jeida-24"))
+ return MEDIA_BUS_FMT_RGB888_1X7X4_JEIDA;
+ if (!strcmp(mapping, "vesa-24"))
+ return MEDIA_BUS_FMT_RGB888_1X7X4_SPWG;
+
+ return -EINVAL;
+}
+EXPORT_SYMBOL_GPL(drm_of_lvds_get_data_mapping);
diff --git a/drivers/gpu/drm/drm_panel_orientation_quirks.c b/drivers/gpu/drm/drm_panel_orientation_quirks.c
index a935987..042bb80 100644
--- a/drivers/gpu/drm/drm_panel_orientation_quirks.c
+++ b/drivers/gpu/drm/drm_panel_orientation_quirks.c
@@ -262,6 +262,12 @@ static const struct dmi_system_id orientation_data[] = {
DMI_EXACT_MATCH(DMI_PRODUCT_VERSION, "Lenovo ideapad D330-10IGM"),
},
.driver_data = (void *)&lcd1200x1920_rightside_up,
+ }, { /* Lenovo Yoga Book X90F / X91F / X91L */
+ .matches = {
+ /* Non exact match to match all versions */
+ DMI_MATCH(DMI_PRODUCT_NAME, "Lenovo YB1-X9"),
+ },
+ .driver_data = (void *)&lcd1200x1920_rightside_up,
}, { /* OneGX1 Pro */
.matches = {
DMI_EXACT_MATCH(DMI_SYS_VENDOR, "SYSTEM_MANUFACTURER"),
diff --git a/drivers/gpu/drm/drm_privacy_screen.c b/drivers/gpu/drm/drm_privacy_screen.c
new file mode 100644
index 0000000..beaf99e
--- /dev/null
+++ b/drivers/gpu/drm/drm_privacy_screen.c
@@ -0,0 +1,467 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright (C) 2020 - 2021 Red Hat, Inc.
+ *
+ * Authors:
+ * Hans de Goede <hdegoede@redhat.com>
+ */
+
+#include <linux/device.h>
+#include <linux/kernel.h>
+#include <linux/list.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/slab.h>
+#include <drm/drm_privacy_screen_machine.h>
+#include <drm/drm_privacy_screen_consumer.h>
+#include <drm/drm_privacy_screen_driver.h>
+#include "drm_internal.h"
+
+/**
+ * DOC: overview
+ *
+ * This class allows non KMS drivers, from e.g. drivers/platform/x86 to
+ * register a privacy-screen device, which the KMS drivers can then use
+ * to implement the standard privacy-screen properties, see
+ * :ref:`Standard Connector Properties<standard_connector_properties>`.
+ *
+ * KMS drivers using a privacy-screen class device are advised to use the
+ * drm_connector_attach_privacy_screen_provider() and
+ * drm_connector_update_privacy_screen() helpers for dealing with this.
+ */
+
+#define to_drm_privacy_screen(dev) \
+ container_of(dev, struct drm_privacy_screen, dev)
+
+static DEFINE_MUTEX(drm_privacy_screen_lookup_lock);
+static LIST_HEAD(drm_privacy_screen_lookup_list);
+
+static DEFINE_MUTEX(drm_privacy_screen_devs_lock);
+static LIST_HEAD(drm_privacy_screen_devs);
+
+/*** drm_privacy_screen_machine.h functions ***/
+
+/**
+ * drm_privacy_screen_lookup_add - add an entry to the static privacy-screen
+ * lookup list
+ * @lookup: lookup list entry to add
+ *
+ * Add an entry to the static privacy-screen lookup list. Note the
+ * &struct list_head which is part of the &struct drm_privacy_screen_lookup
+ * gets added to a list owned by the privacy-screen core. So the passed in
+ * &struct drm_privacy_screen_lookup must not be free-ed until it is removed
+ * from the lookup list by calling drm_privacy_screen_lookup_remove().
+ */
+void drm_privacy_screen_lookup_add(struct drm_privacy_screen_lookup *lookup)
+{
+ mutex_lock(&drm_privacy_screen_lookup_lock);
+ list_add(&lookup->list, &drm_privacy_screen_lookup_list);
+ mutex_unlock(&drm_privacy_screen_lookup_lock);
+}
+EXPORT_SYMBOL(drm_privacy_screen_lookup_add);
+
+/**
+ * drm_privacy_screen_lookup_remove - remove an entry to the static
+ * privacy-screen lookup list
+ * @lookup: lookup list entry to remove
+ *
+ * Remove an entry previously added with drm_privacy_screen_lookup_add()
+ * from the static privacy-screen lookup list.
+ */
+void drm_privacy_screen_lookup_remove(struct drm_privacy_screen_lookup *lookup)
+{
+ mutex_lock(&drm_privacy_screen_lookup_lock);
+ list_del(&lookup->list);
+ mutex_unlock(&drm_privacy_screen_lookup_lock);
+}
+EXPORT_SYMBOL(drm_privacy_screen_lookup_remove);
+
+/*** drm_privacy_screen_consumer.h functions ***/
+
+static struct drm_privacy_screen *drm_privacy_screen_get_by_name(
+ const char *name)
+{
+ struct drm_privacy_screen *priv;
+ struct device *dev = NULL;
+
+ mutex_lock(&drm_privacy_screen_devs_lock);
+
+ list_for_each_entry(priv, &drm_privacy_screen_devs, list) {
+ if (strcmp(dev_name(&priv->dev), name) == 0) {
+ dev = get_device(&priv->dev);
+ break;
+ }
+ }
+
+ mutex_unlock(&drm_privacy_screen_devs_lock);
+
+ return dev ? to_drm_privacy_screen(dev) : NULL;
+}
+
+/**
+ * drm_privacy_screen_get - get a privacy-screen provider
+ * @dev: consumer-device for which to get a privacy-screen provider
+ * @con_id: (video)connector name for which to get a privacy-screen provider
+ *
+ * Get a privacy-screen provider for a privacy-screen attached to the
+ * display described by the @dev and @con_id parameters.
+ *
+ * Return:
+ * * A pointer to a &struct drm_privacy_screen on success.
+ * * ERR_PTR(-ENODEV) if no matching privacy-screen is found
+ * * ERR_PTR(-EPROBE_DEFER) if there is a matching privacy-screen,
+ * but it has not been registered yet.
+ */
+struct drm_privacy_screen *drm_privacy_screen_get(struct device *dev,
+ const char *con_id)
+{
+ const char *dev_id = dev ? dev_name(dev) : NULL;
+ struct drm_privacy_screen_lookup *l;
+ struct drm_privacy_screen *priv;
+ const char *provider = NULL;
+ int match, best = -1;
+
+ /*
+ * For now we only support using a static lookup table, which is
+ * populated by the drm_privacy_screen_arch_init() call. This should
+ * be extended with device-tree / fw_node lookup when support is added
+ * for device-tree using hardware with a privacy-screen.
+ *
+ * The lookup algorithm was shamelessly taken from the clock
+ * framework:
+ *
+ * We do slightly fuzzy matching here:
+ * An entry with a NULL ID is assumed to be a wildcard.
+ * If an entry has a device ID, it must match
+ * If an entry has a connection ID, it must match
+ * Then we take the most specific entry - with the following order
+ * of precedence: dev+con > dev only > con only.
+ */
+ mutex_lock(&drm_privacy_screen_lookup_lock);
+
+ list_for_each_entry(l, &drm_privacy_screen_lookup_list, list) {
+ match = 0;
+
+ if (l->dev_id) {
+ if (!dev_id || strcmp(l->dev_id, dev_id))
+ continue;
+
+ match += 2;
+ }
+
+ if (l->con_id) {
+ if (!con_id || strcmp(l->con_id, con_id))
+ continue;
+
+ match += 1;
+ }
+
+ if (match > best) {
+ provider = l->provider;
+ best = match;
+ }
+ }
+
+ mutex_unlock(&drm_privacy_screen_lookup_lock);
+
+ if (!provider)
+ return ERR_PTR(-ENODEV);
+
+ priv = drm_privacy_screen_get_by_name(provider);
+ if (!priv)
+ return ERR_PTR(-EPROBE_DEFER);
+
+ return priv;
+}
+EXPORT_SYMBOL(drm_privacy_screen_get);
+
+/**
+ * drm_privacy_screen_put - release a privacy-screen reference
+ * @priv: privacy screen reference to release
+ *
+ * Release a privacy-screen provider reference gotten through
+ * drm_privacy_screen_get(). May be called with a NULL or ERR_PTR,
+ * in which case it is a no-op.
+ */
+void drm_privacy_screen_put(struct drm_privacy_screen *priv)
+{
+ if (IS_ERR_OR_NULL(priv))
+ return;
+
+ put_device(&priv->dev);
+}
+EXPORT_SYMBOL(drm_privacy_screen_put);
+
+/**
+ * drm_privacy_screen_set_sw_state - set a privacy-screen's sw-state
+ * @priv: privacy screen to set the sw-state for
+ * @sw_state: new sw-state value to set
+ *
+ * Set the sw-state of a privacy screen. If the privacy-screen is not
+ * in a locked hw-state, then the actual and hw-state of the privacy-screen
+ * will be immediately updated to the new value. If the privacy-screen is
+ * in a locked hw-state, then the new sw-state will be remembered as the
+ * requested state to put the privacy-screen in when it becomes unlocked.
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+int drm_privacy_screen_set_sw_state(struct drm_privacy_screen *priv,
+ enum drm_privacy_screen_status sw_state)
+{
+ int ret = 0;
+
+ mutex_lock(&priv->lock);
+
+ if (!priv->ops) {
+ ret = -ENODEV;
+ goto out;
+ }
+
+ /*
+ * As per the DRM connector properties documentation, setting the
+ * sw_state while the hw_state is locked is allowed. In this case
+ * it is a no-op other then storing the new sw_state so that it
+ * can be honored when the state gets unlocked.
+ * Also skip the set if the hw already is in the desired state.
+ */
+ if (priv->hw_state >= PRIVACY_SCREEN_DISABLED_LOCKED ||
+ priv->hw_state == sw_state) {
+ priv->sw_state = sw_state;
+ goto out;
+ }
+
+ ret = priv->ops->set_sw_state(priv, sw_state);
+out:
+ mutex_unlock(&priv->lock);
+ return ret;
+}
+EXPORT_SYMBOL(drm_privacy_screen_set_sw_state);
+
+/**
+ * drm_privacy_screen_get_state - get privacy-screen's current state
+ * @priv: privacy screen to get the state for
+ * @sw_state_ret: address where to store the privacy-screens current sw-state
+ * @hw_state_ret: address where to store the privacy-screens current hw-state
+ *
+ * Get the current state of a privacy-screen, both the sw-state and the
+ * hw-state.
+ */
+void drm_privacy_screen_get_state(struct drm_privacy_screen *priv,
+ enum drm_privacy_screen_status *sw_state_ret,
+ enum drm_privacy_screen_status *hw_state_ret)
+{
+ mutex_lock(&priv->lock);
+ *sw_state_ret = priv->sw_state;
+ *hw_state_ret = priv->hw_state;
+ mutex_unlock(&priv->lock);
+}
+EXPORT_SYMBOL(drm_privacy_screen_get_state);
+
+/**
+ * drm_privacy_screen_register_notifier - register a notifier
+ * @priv: Privacy screen to register the notifier with
+ * @nb: Notifier-block for the notifier to register
+ *
+ * Register a notifier with the privacy-screen to be notified of changes made
+ * to the privacy-screen state from outside of the privacy-screen class.
+ * E.g. the state may be changed by the hardware itself in response to a
+ * hotkey press.
+ *
+ * The notifier is called with no locks held. The new hw_state and sw_state
+ * can be retrieved using the drm_privacy_screen_get_state() function.
+ * A pointer to the drm_privacy_screen's struct is passed as the void *data
+ * argument of the notifier_block's notifier_call.
+ *
+ * The notifier will NOT be called when changes are made through
+ * drm_privacy_screen_set_sw_state(). It is only called for external changes.
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+int drm_privacy_screen_register_notifier(struct drm_privacy_screen *priv,
+ struct notifier_block *nb)
+{
+ return blocking_notifier_chain_register(&priv->notifier_head, nb);
+}
+EXPORT_SYMBOL(drm_privacy_screen_register_notifier);
+
+/**
+ * drm_privacy_screen_unregister_notifier - unregister a notifier
+ * @priv: Privacy screen to register the notifier with
+ * @nb: Notifier-block for the notifier to register
+ *
+ * Unregister a notifier registered with drm_privacy_screen_register_notifier().
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+int drm_privacy_screen_unregister_notifier(struct drm_privacy_screen *priv,
+ struct notifier_block *nb)
+{
+ return blocking_notifier_chain_unregister(&priv->notifier_head, nb);
+}
+EXPORT_SYMBOL(drm_privacy_screen_unregister_notifier);
+
+/*** drm_privacy_screen_driver.h functions ***/
+
+static ssize_t sw_state_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct drm_privacy_screen *priv = to_drm_privacy_screen(dev);
+ const char * const sw_state_names[] = {
+ "Disabled",
+ "Enabled",
+ };
+ ssize_t ret;
+
+ mutex_lock(&priv->lock);
+
+ if (!priv->ops)
+ ret = -ENODEV;
+ else if (WARN_ON(priv->sw_state >= ARRAY_SIZE(sw_state_names)))
+ ret = -ENXIO;
+ else
+ ret = sprintf(buf, "%s\n", sw_state_names[priv->sw_state]);
+
+ mutex_unlock(&priv->lock);
+ return ret;
+}
+/*
+ * RO: Do not allow setting the sw_state through sysfs, this MUST be done
+ * through the drm_properties on the drm_connector.
+ */
+static DEVICE_ATTR_RO(sw_state);
+
+static ssize_t hw_state_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct drm_privacy_screen *priv = to_drm_privacy_screen(dev);
+ const char * const hw_state_names[] = {
+ "Disabled",
+ "Enabled",
+ "Disabled, locked",
+ "Enabled, locked",
+ };
+ ssize_t ret;
+
+ mutex_lock(&priv->lock);
+
+ if (!priv->ops)
+ ret = -ENODEV;
+ else if (WARN_ON(priv->hw_state >= ARRAY_SIZE(hw_state_names)))
+ ret = -ENXIO;
+ else
+ ret = sprintf(buf, "%s\n", hw_state_names[priv->hw_state]);
+
+ mutex_unlock(&priv->lock);
+ return ret;
+}
+static DEVICE_ATTR_RO(hw_state);
+
+static struct attribute *drm_privacy_screen_attrs[] = {
+ &dev_attr_sw_state.attr,
+ &dev_attr_hw_state.attr,
+ NULL
+};
+ATTRIBUTE_GROUPS(drm_privacy_screen);
+
+static struct device_type drm_privacy_screen_type = {
+ .name = "privacy_screen",
+ .groups = drm_privacy_screen_groups,
+};
+
+static void drm_privacy_screen_device_release(struct device *dev)
+{
+ struct drm_privacy_screen *priv = to_drm_privacy_screen(dev);
+
+ kfree(priv);
+}
+
+/**
+ * drm_privacy_screen_register - register a privacy-screen
+ * @parent: parent-device for the privacy-screen
+ * @ops: &struct drm_privacy_screen_ops pointer with ops for the privacy-screen
+ *
+ * Create and register a privacy-screen.
+ *
+ * Return:
+ * * A pointer to the created privacy-screen on success.
+ * * An ERR_PTR(errno) on failure.
+ */
+struct drm_privacy_screen *drm_privacy_screen_register(
+ struct device *parent, const struct drm_privacy_screen_ops *ops)
+{
+ struct drm_privacy_screen *priv;
+ int ret;
+
+ priv = kzalloc(sizeof(*priv), GFP_KERNEL);
+ if (!priv)
+ return ERR_PTR(-ENOMEM);
+
+ mutex_init(&priv->lock);
+ BLOCKING_INIT_NOTIFIER_HEAD(&priv->notifier_head);
+
+ priv->dev.class = drm_class;
+ priv->dev.type = &drm_privacy_screen_type;
+ priv->dev.parent = parent;
+ priv->dev.release = drm_privacy_screen_device_release;
+ dev_set_name(&priv->dev, "privacy_screen-%s", dev_name(parent));
+ priv->ops = ops;
+
+ priv->ops->get_hw_state(priv);
+
+ ret = device_register(&priv->dev);
+ if (ret) {
+ put_device(&priv->dev);
+ return ERR_PTR(ret);
+ }
+
+ mutex_lock(&drm_privacy_screen_devs_lock);
+ list_add(&priv->list, &drm_privacy_screen_devs);
+ mutex_unlock(&drm_privacy_screen_devs_lock);
+
+ return priv;
+}
+EXPORT_SYMBOL(drm_privacy_screen_register);
+
+/**
+ * drm_privacy_screen_unregister - unregister privacy-screen
+ * @priv: privacy-screen to unregister
+ *
+ * Unregister a privacy-screen registered with drm_privacy_screen_register().
+ * May be called with a NULL or ERR_PTR, in which case it is a no-op.
+ */
+void drm_privacy_screen_unregister(struct drm_privacy_screen *priv)
+{
+ if (IS_ERR_OR_NULL(priv))
+ return;
+
+ mutex_lock(&drm_privacy_screen_devs_lock);
+ list_del(&priv->list);
+ mutex_unlock(&drm_privacy_screen_devs_lock);
+
+ mutex_lock(&priv->lock);
+ priv->ops = NULL;
+ mutex_unlock(&priv->lock);
+
+ device_unregister(&priv->dev);
+}
+EXPORT_SYMBOL(drm_privacy_screen_unregister);
+
+/**
+ * drm_privacy_screen_call_notifier_chain - notify consumers of state change
+ * @priv: Privacy screen to register the notifier with
+ *
+ * A privacy-screen provider driver can call this functions upon external
+ * changes to the privacy-screen state. E.g. the state may be changed by the
+ * hardware itself in response to a hotkey press.
+ * This function must be called without holding the privacy-screen lock.
+ * the driver must update sw_state and hw_state to reflect the new state before
+ * calling this function.
+ * The expected behavior from the driver upon receiving an external state
+ * change event is: 1. Take the lock; 2. Update sw_state and hw_state;
+ * 3. Release the lock. 4. Call drm_privacy_screen_call_notifier_chain().
+ */
+void drm_privacy_screen_call_notifier_chain(struct drm_privacy_screen *priv)
+{
+ blocking_notifier_call_chain(&priv->notifier_head, 0, priv);
+}
+EXPORT_SYMBOL(drm_privacy_screen_call_notifier_chain);
diff --git a/drivers/gpu/drm/drm_privacy_screen_x86.c b/drivers/gpu/drm/drm_privacy_screen_x86.c
new file mode 100644
index 0000000..a2cafb2
--- /dev/null
+++ b/drivers/gpu/drm/drm_privacy_screen_x86.c
@@ -0,0 +1,86 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright (C) 2020 Red Hat, Inc.
+ *
+ * Authors:
+ * Hans de Goede <hdegoede@redhat.com>
+ */
+
+#include <linux/acpi.h>
+#include <drm/drm_privacy_screen_machine.h>
+
+#ifdef CONFIG_X86
+static struct drm_privacy_screen_lookup arch_lookup;
+
+struct arch_init_data {
+ struct drm_privacy_screen_lookup lookup;
+ bool (*detect)(void);
+};
+
+#if IS_ENABLED(CONFIG_THINKPAD_ACPI)
+static acpi_status __init acpi_set_handle(acpi_handle handle, u32 level,
+ void *context, void **return_value)
+{
+ *(acpi_handle *)return_value = handle;
+ return AE_CTRL_TERMINATE;
+}
+
+static bool __init detect_thinkpad_privacy_screen(void)
+{
+ union acpi_object obj = { .type = ACPI_TYPE_INTEGER };
+ struct acpi_object_list args = { .count = 1, .pointer = &obj, };
+ acpi_handle ec_handle = NULL;
+ unsigned long long output;
+ acpi_status status;
+
+ /* Get embedded-controller handle */
+ status = acpi_get_devices("PNP0C09", acpi_set_handle, NULL, &ec_handle);
+ if (ACPI_FAILURE(status) || !ec_handle)
+ return false;
+
+ /* And call the privacy-screen get-status method */
+ status = acpi_evaluate_integer(ec_handle, "HKEY.GSSS", &args, &output);
+ if (ACPI_FAILURE(status))
+ return false;
+
+ return (output & 0x10000) ? true : false;
+}
+#endif
+
+static const struct arch_init_data arch_init_data[] __initconst = {
+#if IS_ENABLED(CONFIG_THINKPAD_ACPI)
+ {
+ .lookup = {
+ .dev_id = NULL,
+ .con_id = NULL,
+ .provider = "privacy_screen-thinkpad_acpi",
+ },
+ .detect = detect_thinkpad_privacy_screen,
+ },
+#endif
+};
+
+void __init drm_privacy_screen_lookup_init(void)
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(arch_init_data); i++) {
+ if (!arch_init_data[i].detect())
+ continue;
+
+ pr_info("Found '%s' privacy-screen provider\n",
+ arch_init_data[i].lookup.provider);
+
+ /* Make a copy because arch_init_data is __initconst */
+ arch_lookup = arch_init_data[i].lookup;
+ drm_privacy_screen_lookup_add(&arch_lookup);
+ break;
+ }
+}
+
+void drm_privacy_screen_lookup_exit(void)
+{
+ if (arch_lookup.provider)
+ drm_privacy_screen_lookup_remove(&arch_lookup);
+}
+#endif /* ifdef CONFIG_X86 */
diff --git a/drivers/gpu/drm/drm_probe_helper.c b/drivers/gpu/drm/drm_probe_helper.c
index 61d5c57..6823595 100644
--- a/drivers/gpu/drm/drm_probe_helper.c
+++ b/drivers/gpu/drm/drm_probe_helper.c
@@ -604,6 +604,9 @@ EXPORT_SYMBOL(drm_helper_probe_single_connector_modes);
*
* This function must be called from process context with no mode
* setting locks held.
+ *
+ * If only a single connector has changed, consider calling
+ * drm_kms_helper_connector_hotplug_event() instead.
*/
void drm_kms_helper_hotplug_event(struct drm_device *dev)
{
@@ -616,6 +619,26 @@ void drm_kms_helper_hotplug_event(struct drm_device *dev)
}
EXPORT_SYMBOL(drm_kms_helper_hotplug_event);
+/**
+ * drm_kms_helper_connector_hotplug_event - fire off a KMS connector hotplug event
+ * @connector: drm_connector which has changed
+ *
+ * This is the same as drm_kms_helper_hotplug_event(), except it fires a more
+ * fine-grained uevent for a single connector.
+ */
+void drm_kms_helper_connector_hotplug_event(struct drm_connector *connector)
+{
+ struct drm_device *dev = connector->dev;
+
+ /* send a uevent + call fbdev */
+ drm_sysfs_connector_hotplug_event(connector);
+ if (dev->mode_config.funcs->output_poll_changed)
+ dev->mode_config.funcs->output_poll_changed(dev);
+
+ drm_client_dev_hotplug(dev);
+}
+EXPORT_SYMBOL(drm_kms_helper_connector_hotplug_event);
+
static void output_poll_execute(struct work_struct *work)
{
struct delayed_work *delayed_work = to_delayed_work(work);
@@ -865,7 +888,7 @@ bool drm_connector_helper_hpd_irq_event(struct drm_connector *connector)
mutex_unlock(&dev->mode_config.mutex);
if (changed) {
- drm_kms_helper_hotplug_event(dev);
+ drm_kms_helper_connector_hotplug_event(connector);
drm_dbg_kms(dev, "[CONNECTOR:%d:%s] Sent hotplug event\n",
connector->base.id,
connector->name);
@@ -904,9 +927,9 @@ EXPORT_SYMBOL(drm_connector_helper_hpd_irq_event);
*/
bool drm_helper_hpd_irq_event(struct drm_device *dev)
{
- struct drm_connector *connector;
+ struct drm_connector *connector, *first_changed_connector = NULL;
struct drm_connector_list_iter conn_iter;
- bool changed = false;
+ int changed = 0;
if (!dev->mode_config.poll_enabled)
return false;
@@ -918,16 +941,25 @@ bool drm_helper_hpd_irq_event(struct drm_device *dev)
if (!(connector->polled & DRM_CONNECTOR_POLL_HPD))
continue;
- if (check_connector_changed(connector))
- changed = true;
+ if (check_connector_changed(connector)) {
+ if (!first_changed_connector) {
+ drm_connector_get(connector);
+ first_changed_connector = connector;
+ }
+
+ changed++;
+ }
}
drm_connector_list_iter_end(&conn_iter);
mutex_unlock(&dev->mode_config.mutex);
- if (changed) {
+ if (changed == 1)
+ drm_kms_helper_connector_hotplug_event(first_changed_connector);
+ else if (changed > 0)
drm_kms_helper_hotplug_event(dev);
- DRM_DEBUG_KMS("Sent hotplug event\n");
- }
+
+ if (first_changed_connector)
+ drm_connector_put(first_changed_connector);
return changed;
}
diff --git a/drivers/gpu/drm/drm_sysfs.c b/drivers/gpu/drm/drm_sysfs.c
index 76ff6ec..430e00b 100644
--- a/drivers/gpu/drm/drm_sysfs.c
+++ b/drivers/gpu/drm/drm_sysfs.c
@@ -410,6 +410,31 @@ void drm_sysfs_hotplug_event(struct drm_device *dev)
EXPORT_SYMBOL(drm_sysfs_hotplug_event);
/**
+ * drm_sysfs_connector_hotplug_event - generate a DRM uevent for any connector
+ * change
+ * @connector: connector which has changed
+ *
+ * Send a uevent for the DRM connector specified by @connector. This will send
+ * a uevent with the properties HOTPLUG=1 and CONNECTOR.
+ */
+void drm_sysfs_connector_hotplug_event(struct drm_connector *connector)
+{
+ struct drm_device *dev = connector->dev;
+ char hotplug_str[] = "HOTPLUG=1", conn_id[21];
+ char *envp[] = { hotplug_str, conn_id, NULL };
+
+ snprintf(conn_id, sizeof(conn_id),
+ "CONNECTOR=%u", connector->base.id);
+
+ drm_dbg_kms(connector->dev,
+ "[CONNECTOR:%d:%s] generating connector hotplug event\n",
+ connector->base.id, connector->name);
+
+ kobject_uevent_env(&dev->primary->kdev->kobj, KOBJ_CHANGE, envp);
+}
+EXPORT_SYMBOL(drm_sysfs_connector_hotplug_event);
+
+/**
* drm_sysfs_connector_status_event - generate a DRM uevent for connector
* property status change
* @connector: connector on which property status changed
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_drv.c b/drivers/gpu/drm/etnaviv/etnaviv_drv.c
index 7dcc639..0b756ec 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_drv.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_drv.c
@@ -589,6 +589,7 @@ static int compare_str(struct device *dev, void *data)
static int etnaviv_pdev_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
+ struct device_node *first_node = NULL;
struct component_match *match = NULL;
if (!dev->platform_data) {
@@ -598,6 +599,9 @@ static int etnaviv_pdev_probe(struct platform_device *pdev)
if (!of_device_is_available(core_node))
continue;
+ if (!first_node)
+ first_node = core_node;
+
drm_of_component_match_add(&pdev->dev, &match,
compare_of, core_node);
}
@@ -609,6 +613,32 @@ static int etnaviv_pdev_probe(struct platform_device *pdev)
component_match_add(dev, &match, compare_str, names[i]);
}
+ /*
+ * PTA and MTLB can have 40 bit base addresses, but
+ * unfortunately, an entry in the MTLB can only point to a
+ * 32 bit base address of a STLB. Moreover, to initialize the
+ * MMU we need a command buffer with a 32 bit address because
+ * without an MMU there is only an indentity mapping between
+ * the internal 32 bit addresses and the bus addresses.
+ *
+ * To make things easy, we set the dma_coherent_mask to 32
+ * bit to make sure we are allocating the command buffers and
+ * TLBs in the lower 4 GiB address space.
+ */
+ if (dma_set_mask(&pdev->dev, DMA_BIT_MASK(40)) ||
+ dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(32))) {
+ dev_dbg(&pdev->dev, "No suitable DMA available\n");
+ return -ENODEV;
+ }
+
+ /*
+ * Apply the same DMA configuration to the virtual etnaviv
+ * device as the GPU we found. This assumes that all Vivante
+ * GPUs in the system share the same DMA constraints.
+ */
+ if (first_node)
+ of_dma_configure(&pdev->dev, first_node, true);
+
return component_master_add_with_match(dev, &etnaviv_master_ops, match);
}
@@ -653,21 +683,12 @@ static int __init etnaviv_init(void)
if (!of_device_is_available(np))
continue;
- pdev = platform_device_alloc("etnaviv", -1);
+ pdev = platform_device_alloc("etnaviv", PLATFORM_DEVID_NONE);
if (!pdev) {
ret = -ENOMEM;
of_node_put(np);
goto unregister_platform_driver;
}
- pdev->dev.coherent_dma_mask = DMA_BIT_MASK(40);
- pdev->dev.dma_mask = &pdev->dev.coherent_dma_mask;
-
- /*
- * Apply the same DMA configuration to the virtual etnaviv
- * device as the GPU we found. This assumes that all Vivante
- * GPUs in the system share the same DMA constraints.
- */
- of_dma_configure(&pdev->dev, np, true);
ret = platform_device_add(pdev);
if (ret) {
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gem.c b/drivers/gpu/drm/etnaviv/etnaviv_gem.c
index f0b2540..d5314aa 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_gem.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_gem.c
@@ -424,45 +424,24 @@ int etnaviv_gem_wait_bo(struct etnaviv_gpu *gpu, struct drm_gem_object *obj,
}
#ifdef CONFIG_DEBUG_FS
-static void etnaviv_gem_describe_fence(struct dma_fence *fence,
- const char *type, struct seq_file *m)
-{
- if (!test_bit(DMA_FENCE_FLAG_SIGNALED_BIT, &fence->flags))
- seq_printf(m, "\t%9s: %s %s seq %llu\n",
- type,
- fence->ops->get_driver_name(fence),
- fence->ops->get_timeline_name(fence),
- fence->seqno);
-}
-
static void etnaviv_gem_describe(struct drm_gem_object *obj, struct seq_file *m)
{
struct etnaviv_gem_object *etnaviv_obj = to_etnaviv_bo(obj);
struct dma_resv *robj = obj->resv;
- struct dma_resv_list *fobj;
- struct dma_fence *fence;
unsigned long off = drm_vma_node_start(&obj->vma_node);
+ int r;
seq_printf(m, "%08x: %c %2d (%2d) %08lx %p %zd\n",
etnaviv_obj->flags, is_active(etnaviv_obj) ? 'A' : 'I',
obj->name, kref_read(&obj->refcount),
off, etnaviv_obj->vaddr, obj->size);
- rcu_read_lock();
- fobj = dma_resv_shared_list(robj);
- if (fobj) {
- unsigned int i, shared_count = fobj->shared_count;
+ r = dma_resv_lock(robj, NULL);
+ if (r)
+ return;
- for (i = 0; i < shared_count; i++) {
- fence = rcu_dereference(fobj->shared[i]);
- etnaviv_gem_describe_fence(fence, "Shared", m);
- }
- }
-
- fence = dma_resv_excl_fence(robj);
- if (fence)
- etnaviv_gem_describe_fence(fence, "Exclusive", m);
- rcu_read_unlock();
+ dma_resv_describe(robj, m);
+ dma_resv_unlock(robj);
}
void etnaviv_gem_describe_objects(struct etnaviv_drm_private *priv,
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c b/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c
index 486259e..b03c20c 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c
@@ -189,13 +189,13 @@ static int submit_fence_sync(struct etnaviv_gem_submit *submit)
continue;
if (bo->flags & ETNA_SUBMIT_BO_WRITE) {
- ret = dma_resv_get_fences(robj, &bo->excl,
+ ret = dma_resv_get_fences(robj, NULL,
&bo->nr_shared,
&bo->shared);
if (ret)
return ret;
} else {
- bo->excl = dma_resv_get_excl_unlocked(robj);
+ bo->excl = dma_fence_get(dma_resv_excl_fence(robj));
}
}
@@ -469,6 +469,12 @@ int etnaviv_ioctl_gem_submit(struct drm_device *dev, void *data,
return -EINVAL;
}
+ if (args->stream_size > SZ_64K || args->nr_relocs > SZ_64K ||
+ args->nr_bos > SZ_64K || args->nr_pmrs > 128) {
+ DRM_ERROR("submit arguments out of size limits\n");
+ return -EINVAL;
+ }
+
/*
* Copy the command submission and bo array to kernel space in
* one go, and do this outside of any locks.
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gpu.c b/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
index 242a5fd..ba5fd01 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
@@ -1658,7 +1658,7 @@ etnaviv_gpu_cooling_set_cur_state(struct thermal_cooling_device *cdev,
return 0;
}
-static struct thermal_cooling_device_ops cooling_ops = {
+static const struct thermal_cooling_device_ops cooling_ops = {
.get_max_state = etnaviv_gpu_cooling_get_max_state,
.get_cur_state = etnaviv_gpu_cooling_get_cur_state,
.set_cur_state = etnaviv_gpu_cooling_set_cur_state,
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gpu.h b/drivers/gpu/drm/etnaviv/etnaviv_gpu.h
index 1c75c8e..85eddd4 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_gpu.h
+++ b/drivers/gpu/drm/etnaviv/etnaviv_gpu.h
@@ -130,6 +130,7 @@ struct etnaviv_gpu {
/* hang detection */
u32 hangcheck_dma_addr;
+ u32 hangcheck_fence;
void __iomem *mmio;
int irq;
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_sched.c b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
index 180bb63..58f593b 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_sched.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
@@ -107,8 +107,10 @@ static enum drm_gpu_sched_stat etnaviv_sched_timedout_job(struct drm_sched_job
*/
dma_addr = gpu_read(gpu, VIVS_FE_DMA_ADDRESS);
change = dma_addr - gpu->hangcheck_dma_addr;
- if (change < 0 || change > 16) {
+ if (gpu->completed_fence != gpu->hangcheck_fence ||
+ change < 0 || change > 16) {
gpu->hangcheck_dma_addr = dma_addr;
+ gpu->hangcheck_fence = gpu->completed_fence;
goto out_no_timeout;
}
diff --git a/drivers/gpu/drm/exynos/exynos_drm_drv.c b/drivers/gpu/drm/exynos/exynos_drm_drv.c
index d8f1cf4..9743b6b 100644
--- a/drivers/gpu/drm/exynos/exynos_drm_drv.c
+++ b/drivers/gpu/drm/exynos/exynos_drm_drv.c
@@ -102,16 +102,7 @@ static const struct drm_ioctl_desc exynos_ioctls[] = {
DRM_RENDER_ALLOW),
};
-static const struct file_operations exynos_drm_driver_fops = {
- .owner = THIS_MODULE,
- .open = drm_open,
- .mmap = exynos_drm_gem_mmap,
- .poll = drm_poll,
- .read = drm_read,
- .unlocked_ioctl = drm_ioctl,
- .compat_ioctl = drm_compat_ioctl,
- .release = drm_release,
-};
+DEFINE_DRM_GEM_FOPS(exynos_drm_driver_fops);
static const struct drm_driver exynos_drm_driver = {
.driver_features = DRIVER_MODESET | DRIVER_GEM
@@ -124,7 +115,7 @@ static const struct drm_driver exynos_drm_driver = {
.prime_fd_to_handle = drm_gem_prime_fd_to_handle,
.gem_prime_import = exynos_drm_gem_prime_import,
.gem_prime_import_sg_table = exynos_drm_gem_prime_import_sg_table,
- .gem_prime_mmap = exynos_drm_gem_prime_mmap,
+ .gem_prime_mmap = drm_gem_prime_mmap,
.ioctls = exynos_ioctls,
.num_ioctls = ARRAY_SIZE(exynos_ioctls),
.fops = &exynos_drm_driver_fops,
diff --git a/drivers/gpu/drm/exynos/exynos_drm_dsi.c b/drivers/gpu/drm/exynos/exynos_drm_dsi.c
index 8d13785..32a3657 100644
--- a/drivers/gpu/drm/exynos/exynos_drm_dsi.c
+++ b/drivers/gpu/drm/exynos/exynos_drm_dsi.c
@@ -13,7 +13,6 @@
#include <linux/gpio/consumer.h>
#include <linux/irq.h>
#include <linux/of_device.h>
-#include <linux/of_gpio.h>
#include <linux/of_graph.h>
#include <linux/phy/phy.h>
#include <linux/regulator/consumer.h>
@@ -265,7 +264,7 @@ struct exynos_dsi {
struct clk **clks;
struct regulator_bulk_data supplies[2];
int irq;
- int te_gpio;
+ struct gpio_desc *te_gpio;
u32 pll_clk_rate;
u32 burst_clk_rate;
@@ -1298,14 +1297,14 @@ static void exynos_dsi_enable_irq(struct exynos_dsi *dsi)
{
enable_irq(dsi->irq);
- if (gpio_is_valid(dsi->te_gpio))
- enable_irq(gpio_to_irq(dsi->te_gpio));
+ if (dsi->te_gpio)
+ enable_irq(gpiod_to_irq(dsi->te_gpio));
}
static void exynos_dsi_disable_irq(struct exynos_dsi *dsi)
{
- if (gpio_is_valid(dsi->te_gpio))
- disable_irq(gpio_to_irq(dsi->te_gpio));
+ if (dsi->te_gpio)
+ disable_irq(gpiod_to_irq(dsi->te_gpio));
disable_irq(dsi->irq);
}
@@ -1335,42 +1334,31 @@ static int exynos_dsi_register_te_irq(struct exynos_dsi *dsi,
int ret;
int te_gpio_irq;
- dsi->te_gpio = of_get_named_gpio(panel->of_node, "te-gpios", 0);
- if (dsi->te_gpio == -ENOENT)
- return 0;
-
- if (!gpio_is_valid(dsi->te_gpio)) {
- ret = dsi->te_gpio;
- dev_err(dsi->dev, "cannot get te-gpios, %d\n", ret);
- goto out;
+ dsi->te_gpio = devm_gpiod_get_optional(dsi->dev, "te", GPIOD_IN);
+ if (IS_ERR(dsi->te_gpio)) {
+ dev_err(dsi->dev, "gpio request failed with %ld\n",
+ PTR_ERR(dsi->te_gpio));
+ return PTR_ERR(dsi->te_gpio);
}
- ret = gpio_request(dsi->te_gpio, "te_gpio");
- if (ret) {
- dev_err(dsi->dev, "gpio request failed with %d\n", ret);
- goto out;
- }
-
- te_gpio_irq = gpio_to_irq(dsi->te_gpio);
+ te_gpio_irq = gpiod_to_irq(dsi->te_gpio);
ret = request_threaded_irq(te_gpio_irq, exynos_dsi_te_irq_handler, NULL,
IRQF_TRIGGER_RISING | IRQF_NO_AUTOEN, "TE", dsi);
if (ret) {
dev_err(dsi->dev, "request interrupt failed with %d\n", ret);
- gpio_free(dsi->te_gpio);
- goto out;
+ gpiod_put(dsi->te_gpio);
+ return ret;
}
-out:
- return ret;
+ return 0;
}
static void exynos_dsi_unregister_te_irq(struct exynos_dsi *dsi)
{
- if (gpio_is_valid(dsi->te_gpio)) {
- free_irq(gpio_to_irq(dsi->te_gpio), dsi);
- gpio_free(dsi->te_gpio);
- dsi->te_gpio = -ENOENT;
+ if (dsi->te_gpio) {
+ free_irq(gpiod_to_irq(dsi->te_gpio), dsi);
+ gpiod_put(dsi->te_gpio);
}
}
@@ -1745,9 +1733,6 @@ static int exynos_dsi_probe(struct platform_device *pdev)
if (!dsi)
return -ENOMEM;
- /* To be checked as invalid one */
- dsi->te_gpio = -ENOENT;
-
init_completion(&dsi->completed);
spin_lock_init(&dsi->transfer_lock);
INIT_LIST_HEAD(&dsi->transfer_list);
diff --git a/drivers/gpu/drm/exynos/exynos_drm_fbdev.c b/drivers/gpu/drm/exynos/exynos_drm_fbdev.c
index 5147f59..02c97b9 100644
--- a/drivers/gpu/drm/exynos/exynos_drm_fbdev.c
+++ b/drivers/gpu/drm/exynos/exynos_drm_fbdev.c
@@ -15,6 +15,7 @@
#include <drm/drm_crtc.h>
#include <drm/drm_fb_helper.h>
#include <drm/drm_fourcc.h>
+#include <drm/drm_prime.h>
#include <drm/drm_probe_helper.h>
#include <drm/exynos_drm.h>
@@ -39,25 +40,8 @@ static int exynos_drm_fb_mmap(struct fb_info *info,
struct drm_fb_helper *helper = info->par;
struct exynos_drm_fbdev *exynos_fbd = to_exynos_fbdev(helper);
struct exynos_drm_gem *exynos_gem = exynos_fbd->exynos_gem;
- unsigned long vm_size;
- int ret;
- vma->vm_flags |= VM_IO | VM_DONTEXPAND | VM_DONTDUMP;
-
- vm_size = vma->vm_end - vma->vm_start;
-
- if (vm_size > exynos_gem->size)
- return -EINVAL;
-
- ret = dma_mmap_attrs(to_dma_dev(helper->dev), vma, exynos_gem->cookie,
- exynos_gem->dma_addr, exynos_gem->size,
- exynos_gem->dma_attrs);
- if (ret < 0) {
- DRM_DEV_ERROR(to_dma_dev(helper->dev), "failed to mmap.\n");
- return ret;
- }
-
- return 0;
+ return drm_gem_prime_mmap(&exynos_gem->base, vma);
}
static const struct fb_ops exynos_drm_fb_ops = {
diff --git a/drivers/gpu/drm/exynos/exynos_drm_fimc.c b/drivers/gpu/drm/exynos/exynos_drm_fimc.c
index ecfd82d0..023f54e 100644
--- a/drivers/gpu/drm/exynos/exynos_drm_fimc.c
+++ b/drivers/gpu/drm/exynos/exynos_drm_fimc.c
@@ -782,8 +782,8 @@ static int fimc_set_prescaler(struct fimc_context *ctx, struct fimc_scaler *sc,
sc->hratio = (src_w << 14) / (dst_w << hfactor);
sc->vratio = (src_h << 14) / (dst_h << vfactor);
- sc->up_h = (dst_w >= src_w) ? true : false;
- sc->up_v = (dst_h >= src_h) ? true : false;
+ sc->up_h = (dst_w >= src_w);
+ sc->up_v = (dst_h >= src_h);
DRM_DEV_DEBUG_KMS(ctx->dev, "hratio[%d]vratio[%d]up_h[%d]up_v[%d]\n",
sc->hratio, sc->vratio, sc->up_h, sc->up_v);
diff --git a/drivers/gpu/drm/exynos/exynos_drm_gem.c b/drivers/gpu/drm/exynos/exynos_drm_gem.c
index 0a0c042..3e493f4 100644
--- a/drivers/gpu/drm/exynos/exynos_drm_gem.c
+++ b/drivers/gpu/drm/exynos/exynos_drm_gem.c
@@ -20,6 +20,8 @@
MODULE_IMPORT_NS(DMA_BUF);
+static int exynos_drm_gem_mmap(struct drm_gem_object *obj, struct vm_area_struct *vma);
+
static int exynos_drm_alloc_buf(struct exynos_drm_gem *exynos_gem, bool kvmap)
{
struct drm_device *dev = exynos_gem->base.dev;
@@ -138,6 +140,7 @@ static const struct vm_operations_struct exynos_drm_gem_vm_ops = {
static const struct drm_gem_object_funcs exynos_drm_gem_object_funcs = {
.free = exynos_drm_gem_free_object,
.get_sg_table = exynos_drm_gem_prime_get_sg_table,
+ .mmap = exynos_drm_gem_mmap,
.vm_ops = &exynos_drm_gem_vm_ops,
};
@@ -357,12 +360,16 @@ int exynos_drm_gem_dumb_create(struct drm_file *file_priv,
return 0;
}
-static int exynos_drm_gem_mmap_obj(struct drm_gem_object *obj,
- struct vm_area_struct *vma)
+static int exynos_drm_gem_mmap(struct drm_gem_object *obj, struct vm_area_struct *vma)
{
struct exynos_drm_gem *exynos_gem = to_exynos_gem(obj);
int ret;
+ if (obj->import_attach)
+ return dma_buf_mmap(obj->dma_buf, vma, 0);
+
+ vma->vm_flags |= VM_IO | VM_DONTEXPAND | VM_DONTDUMP;
+
DRM_DEV_DEBUG_KMS(to_dma_dev(obj->dev), "flags = 0x%x\n",
exynos_gem->flags);
@@ -388,26 +395,6 @@ static int exynos_drm_gem_mmap_obj(struct drm_gem_object *obj,
return ret;
}
-int exynos_drm_gem_mmap(struct file *filp, struct vm_area_struct *vma)
-{
- struct drm_gem_object *obj;
- int ret;
-
- /* set vm_area_struct. */
- ret = drm_gem_mmap(filp, vma);
- if (ret < 0) {
- DRM_ERROR("failed to mmap.\n");
- return ret;
- }
-
- obj = vma->vm_private_data;
-
- if (obj->import_attach)
- return dma_buf_mmap(obj->dma_buf, vma, 0);
-
- return exynos_drm_gem_mmap_obj(obj, vma);
-}
-
/* low-level interface prime helpers */
struct drm_gem_object *exynos_drm_gem_prime_import(struct drm_device *dev,
struct dma_buf *dma_buf)
@@ -469,15 +456,3 @@ exynos_drm_gem_prime_import_sg_table(struct drm_device *dev,
exynos_gem->sgt = sgt;
return &exynos_gem->base;
}
-
-int exynos_drm_gem_prime_mmap(struct drm_gem_object *obj,
- struct vm_area_struct *vma)
-{
- int ret;
-
- ret = drm_gem_mmap_obj(obj, obj->size, vma);
- if (ret < 0)
- return ret;
-
- return exynos_drm_gem_mmap_obj(obj, vma);
-}
diff --git a/drivers/gpu/drm/exynos/exynos_drm_gem.h b/drivers/gpu/drm/exynos/exynos_drm_gem.h
index a23272f..79d7e1a 100644
--- a/drivers/gpu/drm/exynos/exynos_drm_gem.h
+++ b/drivers/gpu/drm/exynos/exynos_drm_gem.h
@@ -96,9 +96,6 @@ int exynos_drm_gem_dumb_create(struct drm_file *file_priv,
struct drm_device *dev,
struct drm_mode_create_dumb *args);
-/* set vm_flags and we can change the vm attribute to other one at here. */
-int exynos_drm_gem_mmap(struct file *filp, struct vm_area_struct *vma);
-
/* low-level interface prime helpers */
struct drm_gem_object *exynos_drm_gem_prime_import(struct drm_device *dev,
struct dma_buf *dma_buf);
@@ -107,7 +104,5 @@ struct drm_gem_object *
exynos_drm_gem_prime_import_sg_table(struct drm_device *dev,
struct dma_buf_attachment *attach,
struct sg_table *sgt);
-int exynos_drm_gem_prime_mmap(struct drm_gem_object *obj,
- struct vm_area_struct *vma);
#endif
diff --git a/drivers/gpu/drm/fsl-dcu/Kconfig b/drivers/gpu/drm/fsl-dcu/Kconfig
index d7dd8ba..e95e96c 100644
--- a/drivers/gpu/drm/fsl-dcu/Kconfig
+++ b/drivers/gpu/drm/fsl-dcu/Kconfig
@@ -3,8 +3,8 @@
tristate "DRM Support for Freescale DCU"
depends on DRM && OF && ARM && COMMON_CLK
select BACKLIGHT_CLASS_DEVICE
+ select DRM_GEM_CMA_HELPER
select DRM_KMS_HELPER
- select DRM_KMS_CMA_HELPER
select DRM_PANEL
select REGMAP_MMIO
select VIDEOMODE_HELPERS
diff --git a/drivers/gpu/drm/gma500/framebuffer.c b/drivers/gpu/drm/gma500/framebuffer.c
index 321e416..45df9de 100644
--- a/drivers/gpu/drm/gma500/framebuffer.c
+++ b/drivers/gpu/drm/gma500/framebuffer.c
@@ -25,7 +25,6 @@
#include "framebuffer.h"
#include "gem.h"
-#include "gtt.h"
#include "psb_drv.h"
#include "psb_intel_drv.h"
#include "psb_intel_reg.h"
@@ -82,14 +81,13 @@ static vm_fault_t psbfb_vm_fault(struct vm_fault *vmf)
struct drm_framebuffer *fb = vma->vm_private_data;
struct drm_device *dev = fb->dev;
struct drm_psb_private *dev_priv = to_drm_psb_private(dev);
- struct gtt_range *gtt = to_gtt_range(fb->obj[0]);
+ struct psb_gem_object *pobj = to_psb_gem_object(fb->obj[0]);
int page_num;
int i;
unsigned long address;
vm_fault_t ret = VM_FAULT_SIGBUS;
unsigned long pfn;
- unsigned long phys_addr = (unsigned long)dev_priv->stolen_base +
- gtt->offset;
+ unsigned long phys_addr = (unsigned long)dev_priv->stolen_base + pobj->offset;
page_num = vma_pages(vma);
address = vmf->address - (vmf->pgoff << PAGE_SHIFT);
@@ -226,31 +224,6 @@ static struct drm_framebuffer *psb_framebuffer_create
}
/**
- * psbfb_alloc - allocate frame buffer memory
- * @dev: the DRM device
- * @aligned_size: space needed
- *
- * Allocate the frame buffer. In the usual case we get a GTT range that
- * is stolen memory backed and life is simple. If there isn't sufficient
- * we fail as we don't have the virtual mapping space to really vmap it
- * and the kernel console code can't handle non linear framebuffers.
- *
- * Re-address this as and if the framebuffer layer grows this ability.
- */
-static struct gtt_range *psbfb_alloc(struct drm_device *dev, int aligned_size)
-{
- struct gtt_range *backing;
- /* Begin by trying to use stolen memory backing */
- backing = psb_gtt_alloc_range(dev, aligned_size, "fb", 1, PAGE_SIZE);
- if (backing) {
- backing->gem.funcs = &psb_gem_object_funcs;
- drm_gem_private_object_init(dev, &backing->gem, aligned_size);
- return backing;
- }
- return NULL;
-}
-
-/**
* psbfb_create - create a framebuffer
* @fb_helper: the framebuffer helper
* @sizes: specification of the layout
@@ -268,7 +241,8 @@ static int psbfb_create(struct drm_fb_helper *fb_helper,
struct drm_mode_fb_cmd2 mode_cmd;
int size;
int ret;
- struct gtt_range *backing;
+ struct psb_gem_object *backing;
+ struct drm_gem_object *obj;
u32 bpp, depth;
mode_cmd.width = sizes->surface_width;
@@ -286,24 +260,25 @@ static int psbfb_create(struct drm_fb_helper *fb_helper,
size = ALIGN(size, PAGE_SIZE);
/* Allocate the framebuffer in the GTT with stolen page backing */
- backing = psbfb_alloc(dev, size);
- if (backing == NULL)
- return -ENOMEM;
+ backing = psb_gem_create(dev, size, "fb", true, PAGE_SIZE);
+ if (IS_ERR(backing))
+ return PTR_ERR(backing);
+ obj = &backing->base;
memset(dev_priv->vram_addr + backing->offset, 0, size);
info = drm_fb_helper_alloc_fbi(fb_helper);
if (IS_ERR(info)) {
ret = PTR_ERR(info);
- goto out;
+ goto err_drm_gem_object_put;
}
mode_cmd.pixel_format = drm_mode_legacy_fb_format(bpp, depth);
- fb = psb_framebuffer_create(dev, &mode_cmd, &backing->gem);
+ fb = psb_framebuffer_create(dev, &mode_cmd, obj);
if (IS_ERR(fb)) {
ret = PTR_ERR(fb);
- goto out;
+ goto err_drm_gem_object_put;
}
fb_helper->fb = fb;
@@ -334,8 +309,9 @@ static int psbfb_create(struct drm_fb_helper *fb_helper,
dev_dbg(dev->dev, "allocated %dx%d fb\n", fb->width, fb->height);
return 0;
-out:
- psb_gtt_free_range(dev, backing);
+
+err_drm_gem_object_put:
+ drm_gem_object_put(obj);
return ret;
}
diff --git a/drivers/gpu/drm/gma500/gem.c b/drivers/gpu/drm/gma500/gem.c
index 5ae54c9..8d65af8 100644
--- a/drivers/gpu/drm/gma500/gem.c
+++ b/drivers/gpu/drm/gma500/gem.c
@@ -13,24 +13,105 @@
#include <linux/pagemap.h>
+#include <asm/set_memory.h>
+
#include <drm/drm.h>
#include <drm/drm_vma_manager.h>
#include "gem.h"
#include "psb_drv.h"
+int psb_gem_pin(struct psb_gem_object *pobj)
+{
+ struct drm_gem_object *obj = &pobj->base;
+ struct drm_device *dev = obj->dev;
+ struct drm_psb_private *dev_priv = to_drm_psb_private(dev);
+ u32 gpu_base = dev_priv->gtt.gatt_start;
+ struct page **pages;
+ unsigned int npages;
+ int ret;
+
+ mutex_lock(&dev_priv->gtt_mutex);
+
+ if (pobj->in_gart || pobj->stolen)
+ goto out; /* already mapped */
+
+ pages = drm_gem_get_pages(obj);
+ if (IS_ERR(pages)) {
+ ret = PTR_ERR(pages);
+ goto err_mutex_unlock;
+ }
+
+ npages = obj->size / PAGE_SIZE;
+
+ set_pages_array_wc(pages, npages);
+
+ psb_gtt_insert_pages(dev_priv, &pobj->resource, pages);
+ psb_mmu_insert_pages(psb_mmu_get_default_pd(dev_priv->mmu), pages,
+ (gpu_base + pobj->offset), npages, 0, 0,
+ PSB_MMU_CACHED_MEMORY);
+
+ pobj->npage = npages;
+ pobj->pages = pages;
+
+out:
+ ++pobj->in_gart;
+ mutex_unlock(&dev_priv->gtt_mutex);
+
+ return 0;
+
+err_mutex_unlock:
+ mutex_unlock(&dev_priv->gtt_mutex);
+ return ret;
+}
+
+void psb_gem_unpin(struct psb_gem_object *pobj)
+{
+ struct drm_gem_object *obj = &pobj->base;
+ struct drm_device *dev = obj->dev;
+ struct drm_psb_private *dev_priv = to_drm_psb_private(dev);
+ u32 gpu_base = dev_priv->gtt.gatt_start;
+
+ mutex_lock(&dev_priv->gtt_mutex);
+
+ WARN_ON(!pobj->in_gart);
+
+ --pobj->in_gart;
+
+ if (pobj->in_gart || pobj->stolen)
+ goto out;
+
+ psb_mmu_remove_pages(psb_mmu_get_default_pd(dev_priv->mmu),
+ (gpu_base + pobj->offset), pobj->npage, 0, 0);
+ psb_gtt_remove_pages(dev_priv, &pobj->resource);
+
+ /* Reset caching flags */
+ set_pages_array_wb(pobj->pages, pobj->npage);
+
+ drm_gem_put_pages(obj, pobj->pages, true, false);
+ pobj->pages = NULL;
+ pobj->npage = 0;
+
+out:
+ mutex_unlock(&dev_priv->gtt_mutex);
+}
+
static vm_fault_t psb_gem_fault(struct vm_fault *vmf);
static void psb_gem_free_object(struct drm_gem_object *obj)
{
- struct gtt_range *gtt = container_of(obj, struct gtt_range, gem);
+ struct psb_gem_object *pobj = to_psb_gem_object(obj);
- /* Remove the list map if one is present */
- drm_gem_free_mmap_offset(obj);
drm_gem_object_release(obj);
- /* This must occur last as it frees up the memory of the GEM object */
- psb_gtt_free_range(obj->dev, gtt);
+ /* Undo the mmap pin if we are destroying the object */
+ if (pobj->mmapping)
+ psb_gem_unpin(pobj);
+
+ WARN_ON(pobj->in_gart && !pobj->stolen);
+
+ release_resource(&pobj->resource);
+ kfree(pobj);
}
static const struct vm_operations_struct psb_gem_vm_ops = {
@@ -39,63 +120,60 @@ static const struct vm_operations_struct psb_gem_vm_ops = {
.close = drm_gem_vm_close,
};
-const struct drm_gem_object_funcs psb_gem_object_funcs = {
+static const struct drm_gem_object_funcs psb_gem_object_funcs = {
.free = psb_gem_free_object,
.vm_ops = &psb_gem_vm_ops,
};
-/**
- * psb_gem_create - create a mappable object
- * @file: the DRM file of the client
- * @dev: our device
- * @size: the size requested
- * @handlep: returned handle (opaque number)
- * @stolen: unused
- * @align: unused
- *
- * Create a GEM object, fill in the boilerplate and attach a handle to
- * it so that userspace can speak about it. This does the core work
- * for the various methods that do/will create GEM objects for things
- */
-int psb_gem_create(struct drm_file *file, struct drm_device *dev, u64 size,
- u32 *handlep, int stolen, u32 align)
+struct psb_gem_object *
+psb_gem_create(struct drm_device *dev, u64 size, const char *name, bool stolen, u32 align)
{
- struct gtt_range *r;
+ struct drm_psb_private *dev_priv = to_drm_psb_private(dev);
+ struct psb_gem_object *pobj;
+ struct drm_gem_object *obj;
int ret;
- u32 handle;
size = roundup(size, PAGE_SIZE);
- /* Allocate our object - for now a direct gtt range which is not
- stolen memory backed */
- r = psb_gtt_alloc_range(dev, size, "gem", 0, PAGE_SIZE);
- if (r == NULL) {
- dev_err(dev->dev, "no memory for %lld byte GEM object\n", size);
- return -ENOSPC;
+ pobj = kzalloc(sizeof(*pobj), GFP_KERNEL);
+ if (!pobj)
+ return ERR_PTR(-ENOMEM);
+ obj = &pobj->base;
+
+ /* GTT resource */
+
+ ret = psb_gtt_allocate_resource(dev_priv, &pobj->resource, name, size, align, stolen,
+ &pobj->offset);
+ if (ret)
+ goto err_kfree;
+
+ if (stolen) {
+ pobj->stolen = true;
+ pobj->in_gart = 1;
}
- r->gem.funcs = &psb_gem_object_funcs;
- /* Initialize the extra goodies GEM needs to do all the hard work */
- if (drm_gem_object_init(dev, &r->gem, size) != 0) {
- psb_gtt_free_range(dev, r);
- /* GEM doesn't give an error code so use -ENOMEM */
- dev_err(dev->dev, "GEM init failed for %lld\n", size);
- return -ENOMEM;
+
+ /* GEM object */
+
+ obj->funcs = &psb_gem_object_funcs;
+
+ if (stolen) {
+ drm_gem_private_object_init(dev, obj, size);
+ } else {
+ ret = drm_gem_object_init(dev, obj, size);
+ if (ret)
+ goto err_release_resource;
+
+ /* Limit the object to 32-bit mappings */
+ mapping_set_gfp_mask(obj->filp->f_mapping, GFP_KERNEL | __GFP_DMA32);
}
- /* Limit the object to 32bit mappings */
- mapping_set_gfp_mask(r->gem.filp->f_mapping, GFP_KERNEL | __GFP_DMA32);
- /* Give the object a handle so we can carry it more easily */
- ret = drm_gem_handle_create(file, &r->gem, &handle);
- if (ret) {
- dev_err(dev->dev, "GEM handle failed for %p, %lld\n",
- &r->gem, size);
- drm_gem_object_release(&r->gem);
- psb_gtt_free_range(dev, r);
- return ret;
- }
- /* We have the initial and handle reference but need only one now */
- drm_gem_object_put(&r->gem);
- *handlep = handle;
- return 0;
+
+ return pobj;
+
+err_release_resource:
+ release_resource(&pobj->resource);
+err_kfree:
+ kfree(pobj);
+ return ERR_PTR(ret);
}
/**
@@ -111,10 +189,40 @@ int psb_gem_create(struct drm_file *file, struct drm_device *dev, u64 size,
int psb_gem_dumb_create(struct drm_file *file, struct drm_device *dev,
struct drm_mode_create_dumb *args)
{
- args->pitch = ALIGN(args->width * ((args->bpp + 7) / 8), 64);
- args->size = args->pitch * args->height;
- return psb_gem_create(file, dev, args->size, &args->handle, 0,
- PAGE_SIZE);
+ size_t pitch, size;
+ struct psb_gem_object *pobj;
+ struct drm_gem_object *obj;
+ u32 handle;
+ int ret;
+
+ pitch = args->width * DIV_ROUND_UP(args->bpp, 8);
+ pitch = ALIGN(pitch, 64);
+
+ size = pitch * args->height;
+ size = roundup(size, PAGE_SIZE);
+ if (!size)
+ return -EINVAL;
+
+ pobj = psb_gem_create(dev, size, "gem", false, PAGE_SIZE);
+ if (IS_ERR(pobj))
+ return PTR_ERR(pobj);
+ obj = &pobj->base;
+
+ ret = drm_gem_handle_create(file, obj, &handle);
+ if (ret)
+ goto err_drm_gem_object_put;
+
+ drm_gem_object_put(obj);
+
+ args->pitch = pitch;
+ args->size = size;
+ args->handle = handle;
+
+ return 0;
+
+err_drm_gem_object_put:
+ drm_gem_object_put(obj);
+ return ret;
}
/**
@@ -137,7 +245,7 @@ static vm_fault_t psb_gem_fault(struct vm_fault *vmf)
{
struct vm_area_struct *vma = vmf->vma;
struct drm_gem_object *obj;
- struct gtt_range *r;
+ struct psb_gem_object *pobj;
int err;
vm_fault_t ret;
unsigned long pfn;
@@ -149,7 +257,7 @@ static vm_fault_t psb_gem_fault(struct vm_fault *vmf)
dev = obj->dev;
dev_priv = to_drm_psb_private(dev);
- r = container_of(obj, struct gtt_range, gem); /* Get the gtt range */
+ pobj = to_psb_gem_object(obj);
/* Make sure we don't parallel update on a fault, nor move or remove
something from beneath our feet */
@@ -157,14 +265,14 @@ static vm_fault_t psb_gem_fault(struct vm_fault *vmf)
/* For now the mmap pins the object and it stays pinned. As things
stand that will do us no harm */
- if (r->mmapping == 0) {
- err = psb_gtt_pin(r);
+ if (pobj->mmapping == 0) {
+ err = psb_gem_pin(pobj);
if (err < 0) {
dev_err(dev->dev, "gma500: pin failed: %d\n", err);
ret = vmf_error(err);
goto fail;
}
- r->mmapping = 1;
+ pobj->mmapping = 1;
}
/* Page relative to the VMA start - we must calculate this ourselves
@@ -172,10 +280,10 @@ static vm_fault_t psb_gem_fault(struct vm_fault *vmf)
page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT;
/* CPU view of the page, don't go via the GART for CPU writes */
- if (r->stolen)
- pfn = (dev_priv->stolen_base + r->offset) >> PAGE_SHIFT;
+ if (pobj->stolen)
+ pfn = (dev_priv->stolen_base + pobj->offset) >> PAGE_SHIFT;
else
- pfn = page_to_pfn(r->pages[page_offset]);
+ pfn = page_to_pfn(pobj->pages[page_offset]);
ret = vmf_insert_pfn(vma, vmf->address, pfn);
fail:
mutex_unlock(&dev_priv->mmap_mutex);
diff --git a/drivers/gpu/drm/gma500/gem.h b/drivers/gpu/drm/gma500/gem.h
index bae6454..79cced4 100644
--- a/drivers/gpu/drm/gma500/gem.h
+++ b/drivers/gpu/drm/gma500/gem.h
@@ -8,11 +8,33 @@
#ifndef _GEM_H
#define _GEM_H
+#include <linux/kernel.h>
+
+#include <drm/drm_gem.h>
+
struct drm_device;
-extern const struct drm_gem_object_funcs psb_gem_object_funcs;
+struct psb_gem_object {
+ struct drm_gem_object base;
-extern int psb_gem_create(struct drm_file *file, struct drm_device *dev,
- u64 size, u32 *handlep, int stolen, u32 align);
+ struct resource resource; /* GTT resource for our allocation */
+ u32 offset; /* GTT offset of our object */
+ int in_gart; /* Currently in the GART (ref ct) */
+ bool stolen; /* Backed from stolen RAM */
+ bool mmapping; /* Is mmappable */
+ struct page **pages; /* Backing pages if present */
+ int npage; /* Number of backing pages */
+};
+
+static inline struct psb_gem_object *to_psb_gem_object(struct drm_gem_object *obj)
+{
+ return container_of(obj, struct psb_gem_object, base);
+}
+
+struct psb_gem_object *
+psb_gem_create(struct drm_device *dev, u64 size, const char *name, bool stolen, u32 align);
+
+int psb_gem_pin(struct psb_gem_object *pobj);
+void psb_gem_unpin(struct psb_gem_object *pobj);
#endif
diff --git a/drivers/gpu/drm/gma500/gma_display.c b/drivers/gpu/drm/gma500/gma_display.c
index cbcecba..99da311 100644
--- a/drivers/gpu/drm/gma500/gma_display.c
+++ b/drivers/gpu/drm/gma500/gma_display.c
@@ -15,6 +15,7 @@
#include <drm/drm_vblank.h>
#include "framebuffer.h"
+#include "gem.h"
#include "gma_display.h"
#include "psb_drv.h"
#include "psb_intel_drv.h"
@@ -54,7 +55,7 @@ int gma_pipe_set_base(struct drm_crtc *crtc, int x, int y,
struct drm_psb_private *dev_priv = to_drm_psb_private(dev);
struct gma_crtc *gma_crtc = to_gma_crtc(crtc);
struct drm_framebuffer *fb = crtc->primary->fb;
- struct gtt_range *gtt;
+ struct psb_gem_object *pobj;
int pipe = gma_crtc->pipe;
const struct psb_offset *map = &dev_priv->regmap[pipe];
unsigned long start, offset;
@@ -70,14 +71,14 @@ int gma_pipe_set_base(struct drm_crtc *crtc, int x, int y,
goto gma_pipe_cleaner;
}
- gtt = to_gtt_range(fb->obj[0]);
+ pobj = to_psb_gem_object(fb->obj[0]);
/* We are displaying this buffer, make sure it is actually loaded
into the GTT */
- ret = psb_gtt_pin(gtt);
+ ret = psb_gem_pin(pobj);
if (ret < 0)
goto gma_pipe_set_base_exit;
- start = gtt->offset;
+ start = pobj->offset;
offset = y * fb->pitches[0] + x * fb->format->cpp[0];
REG_WRITE(map->stride, fb->pitches[0]);
@@ -125,7 +126,7 @@ int gma_pipe_set_base(struct drm_crtc *crtc, int x, int y,
gma_pipe_cleaner:
/* If there was a previous display we can now unpin it */
if (old_fb)
- psb_gtt_unpin(to_gtt_range(old_fb->obj[0]));
+ psb_gem_unpin(to_psb_gem_object(old_fb->obj[0]));
gma_pipe_set_base_exit:
gma_power_end(dev);
@@ -331,8 +332,8 @@ int gma_crtc_cursor_set(struct drm_crtc *crtc,
uint32_t base = (pipe == 0) ? CURABASE : CURBBASE;
uint32_t temp;
size_t addr = 0;
- struct gtt_range *gt;
- struct gtt_range *cursor_gt = gma_crtc->cursor_gt;
+ struct psb_gem_object *pobj;
+ struct psb_gem_object *cursor_pobj = gma_crtc->cursor_pobj;
struct drm_gem_object *obj;
void *tmp_dst, *tmp_src;
int ret = 0, i, cursor_pages;
@@ -348,9 +349,8 @@ int gma_crtc_cursor_set(struct drm_crtc *crtc,
/* Unpin the old GEM object */
if (gma_crtc->cursor_obj) {
- gt = container_of(gma_crtc->cursor_obj,
- struct gtt_range, gem);
- psb_gtt_unpin(gt);
+ pobj = to_psb_gem_object(gma_crtc->cursor_obj);
+ psb_gem_unpin(pobj);
drm_gem_object_put(gma_crtc->cursor_obj);
gma_crtc->cursor_obj = NULL;
}
@@ -375,40 +375,40 @@ int gma_crtc_cursor_set(struct drm_crtc *crtc,
goto unref_cursor;
}
- gt = container_of(obj, struct gtt_range, gem);
+ pobj = to_psb_gem_object(obj);
/* Pin the memory into the GTT */
- ret = psb_gtt_pin(gt);
+ ret = psb_gem_pin(pobj);
if (ret) {
dev_err(dev->dev, "Can not pin down handle 0x%x\n", handle);
goto unref_cursor;
}
if (dev_priv->ops->cursor_needs_phys) {
- if (cursor_gt == NULL) {
+ if (!cursor_pobj) {
dev_err(dev->dev, "No hardware cursor mem available");
ret = -ENOMEM;
goto unref_cursor;
}
/* Prevent overflow */
- if (gt->npage > 4)
+ if (pobj->npage > 4)
cursor_pages = 4;
else
- cursor_pages = gt->npage;
+ cursor_pages = pobj->npage;
/* Copy the cursor to cursor mem */
- tmp_dst = dev_priv->vram_addr + cursor_gt->offset;
+ tmp_dst = dev_priv->vram_addr + cursor_pobj->offset;
for (i = 0; i < cursor_pages; i++) {
- tmp_src = kmap(gt->pages[i]);
+ tmp_src = kmap(pobj->pages[i]);
memcpy(tmp_dst, tmp_src, PAGE_SIZE);
- kunmap(gt->pages[i]);
+ kunmap(pobj->pages[i]);
tmp_dst += PAGE_SIZE;
}
addr = gma_crtc->cursor_addr;
} else {
- addr = gt->offset;
+ addr = pobj->offset;
gma_crtc->cursor_addr = addr;
}
@@ -425,8 +425,8 @@ int gma_crtc_cursor_set(struct drm_crtc *crtc,
/* unpin the old bo */
if (gma_crtc->cursor_obj) {
- gt = container_of(gma_crtc->cursor_obj, struct gtt_range, gem);
- psb_gtt_unpin(gt);
+ pobj = to_psb_gem_object(gma_crtc->cursor_obj);
+ psb_gem_unpin(pobj);
drm_gem_object_put(gma_crtc->cursor_obj);
}
@@ -483,14 +483,14 @@ void gma_crtc_commit(struct drm_crtc *crtc)
void gma_crtc_disable(struct drm_crtc *crtc)
{
- struct gtt_range *gt;
+ struct psb_gem_object *pobj;
const struct drm_crtc_helper_funcs *crtc_funcs = crtc->helper_private;
crtc_funcs->dpms(crtc, DRM_MODE_DPMS_OFF);
if (crtc->primary->fb) {
- gt = to_gtt_range(crtc->primary->fb->obj[0]);
- psb_gtt_unpin(gt);
+ pobj = to_psb_gem_object(crtc->primary->fb->obj[0]);
+ psb_gem_unpin(pobj);
}
}
@@ -498,6 +498,9 @@ void gma_crtc_destroy(struct drm_crtc *crtc)
{
struct gma_crtc *gma_crtc = to_gma_crtc(crtc);
+ if (gma_crtc->cursor_pobj)
+ drm_gem_object_put(&gma_crtc->cursor_pobj->base);
+
kfree(gma_crtc->crtc_state);
drm_crtc_cleanup(crtc);
kfree(gma_crtc);
diff --git a/drivers/gpu/drm/gma500/gtt.c b/drivers/gpu/drm/gma500/gtt.c
index 55a2a69..309ffe9 100644
--- a/drivers/gpu/drm/gma500/gtt.c
+++ b/drivers/gpu/drm/gma500/gtt.c
@@ -7,10 +7,7 @@
* Alan Cox <alan@linux.intel.com>
*/
-#include <linux/shmem_fs.h>
-
-#include <asm/set_memory.h>
-
+#include "gem.h" /* TODO: for struct psb_gem_object, see psb_gtt_restore() */
#include "psb_drv.h"
@@ -18,6 +15,33 @@
* GTT resource allocator - manage page mappings in GTT space
*/
+int psb_gtt_allocate_resource(struct drm_psb_private *pdev, struct resource *res,
+ const char *name, resource_size_t size, resource_size_t align,
+ bool stolen, u32 *offset)
+{
+ struct resource *root = pdev->gtt_mem;
+ resource_size_t start, end;
+ int ret;
+
+ if (stolen) {
+ /* The start of the GTT is backed by stolen pages. */
+ start = root->start;
+ end = root->start + pdev->gtt.stolen_size - 1;
+ } else {
+ /* The rest is backed by system pages. */
+ start = root->start + pdev->gtt.stolen_size;
+ end = root->end;
+ }
+
+ res->name = name;
+ ret = allocate_resource(root, res, size, start, end, align, NULL, NULL);
+ if (ret)
+ return ret;
+ *offset = res->start - root->start;
+
+ return 0;
+}
+
/**
* psb_gtt_mask_pte - generate GTT pte entry
* @pfn: page number to encode
@@ -43,281 +67,62 @@ static inline uint32_t psb_gtt_mask_pte(uint32_t pfn, int type)
return (pfn << PAGE_SHIFT) | mask;
}
-/**
- * psb_gtt_entry - find the GTT entries for a gtt_range
- * @dev: our DRM device
- * @r: our GTT range
- *
- * Given a gtt_range object return the GTT offset of the page table
- * entries for this gtt_range
- */
-static u32 __iomem *psb_gtt_entry(struct drm_device *dev, struct gtt_range *r)
+static u32 __iomem *psb_gtt_entry(struct drm_psb_private *pdev, const struct resource *res)
{
- struct drm_psb_private *dev_priv = to_drm_psb_private(dev);
- unsigned long offset;
+ unsigned long offset = res->start - pdev->gtt_mem->start;
- offset = r->resource.start - dev_priv->gtt_mem->start;
-
- return dev_priv->gtt_map + (offset >> PAGE_SHIFT);
+ return pdev->gtt_map + (offset >> PAGE_SHIFT);
}
-/**
- * psb_gtt_insert - put an object into the GTT
- * @dev: our DRM device
- * @r: our GTT range
- * @resume: on resume
- *
- * Take our preallocated GTT range and insert the GEM object into
- * the GTT. This is protected via the gtt mutex which the caller
- * must hold.
+/*
+ * Take our preallocated GTT range and insert the GEM object into
+ * the GTT. This is protected via the gtt mutex which the caller
+ * must hold.
*/
-static int psb_gtt_insert(struct drm_device *dev, struct gtt_range *r,
- int resume)
+void psb_gtt_insert_pages(struct drm_psb_private *pdev, const struct resource *res,
+ struct page **pages)
{
+ resource_size_t npages, i;
u32 __iomem *gtt_slot;
u32 pte;
- struct page **pages;
- int i;
-
- if (r->pages == NULL) {
- WARN_ON(1);
- return -EINVAL;
- }
-
- WARN_ON(r->stolen); /* refcount these maybe ? */
-
- gtt_slot = psb_gtt_entry(dev, r);
- pages = r->pages;
-
- if (!resume) {
- /* Make sure changes are visible to the GPU */
- set_pages_array_wc(pages, r->npage);
- }
/* Write our page entries into the GTT itself */
- for (i = 0; i < r->npage; i++) {
- pte = psb_gtt_mask_pte(page_to_pfn(r->pages[i]),
- PSB_MMU_CACHED_MEMORY);
- iowrite32(pte, gtt_slot++);
+
+ npages = resource_size(res) >> PAGE_SHIFT;
+ gtt_slot = psb_gtt_entry(pdev, res);
+
+ for (i = 0; i < npages; ++i, ++gtt_slot) {
+ pte = psb_gtt_mask_pte(page_to_pfn(pages[i]), PSB_MMU_CACHED_MEMORY);
+ iowrite32(pte, gtt_slot);
}
/* Make sure all the entries are set before we return */
ioread32(gtt_slot - 1);
-
- return 0;
-}
-
-/**
- * psb_gtt_remove - remove an object from the GTT
- * @dev: our DRM device
- * @r: our GTT range
- *
- * Remove a preallocated GTT range from the GTT. Overwrite all the
- * page table entries with the dummy page. This is protected via the gtt
- * mutex which the caller must hold.
- */
-static void psb_gtt_remove(struct drm_device *dev, struct gtt_range *r)
-{
- struct drm_psb_private *dev_priv = to_drm_psb_private(dev);
- u32 __iomem *gtt_slot;
- u32 pte;
- int i;
-
- WARN_ON(r->stolen);
-
- gtt_slot = psb_gtt_entry(dev, r);
- pte = psb_gtt_mask_pte(page_to_pfn(dev_priv->scratch_page),
- PSB_MMU_CACHED_MEMORY);
-
- for (i = 0; i < r->npage; i++)
- iowrite32(pte, gtt_slot++);
- ioread32(gtt_slot - 1);
- set_pages_array_wb(r->pages, r->npage);
-}
-
-/**
- * psb_gtt_attach_pages - attach and pin GEM pages
- * @gt: the gtt range
- *
- * Pin and build an in kernel list of the pages that back our GEM object.
- * While we hold this the pages cannot be swapped out. This is protected
- * via the gtt mutex which the caller must hold.
- */
-static int psb_gtt_attach_pages(struct gtt_range *gt)
-{
- struct page **pages;
-
- WARN_ON(gt->pages);
-
- pages = drm_gem_get_pages(>->gem);
- if (IS_ERR(pages))
- return PTR_ERR(pages);
-
- gt->npage = gt->gem.size / PAGE_SIZE;
- gt->pages = pages;
-
- return 0;
-}
-
-/**
- * psb_gtt_detach_pages - attach and pin GEM pages
- * @gt: the gtt range
- *
- * Undo the effect of psb_gtt_attach_pages. At this point the pages
- * must have been removed from the GTT as they could now be paged out
- * and move bus address. This is protected via the gtt mutex which the
- * caller must hold.
- */
-static void psb_gtt_detach_pages(struct gtt_range *gt)
-{
- drm_gem_put_pages(>->gem, gt->pages, true, false);
- gt->pages = NULL;
-}
-
-/**
- * psb_gtt_pin - pin pages into the GTT
- * @gt: range to pin
- *
- * Pin a set of pages into the GTT. The pins are refcounted so that
- * multiple pins need multiple unpins to undo.
- *
- * Non GEM backed objects treat this as a no-op as they are always GTT
- * backed objects.
- */
-int psb_gtt_pin(struct gtt_range *gt)
-{
- int ret = 0;
- struct drm_device *dev = gt->gem.dev;
- struct drm_psb_private *dev_priv = to_drm_psb_private(dev);
- u32 gpu_base = dev_priv->gtt.gatt_start;
-
- mutex_lock(&dev_priv->gtt_mutex);
-
- if (gt->in_gart == 0 && gt->stolen == 0) {
- ret = psb_gtt_attach_pages(gt);
- if (ret < 0)
- goto out;
- ret = psb_gtt_insert(dev, gt, 0);
- if (ret < 0) {
- psb_gtt_detach_pages(gt);
- goto out;
- }
- psb_mmu_insert_pages(psb_mmu_get_default_pd(dev_priv->mmu),
- gt->pages, (gpu_base + gt->offset),
- gt->npage, 0, 0, PSB_MMU_CACHED_MEMORY);
- }
- gt->in_gart++;
-out:
- mutex_unlock(&dev_priv->gtt_mutex);
- return ret;
-}
-
-/**
- * psb_gtt_unpin - Drop a GTT pin requirement
- * @gt: range to pin
- *
- * Undoes the effect of psb_gtt_pin. On the last drop the GEM object
- * will be removed from the GTT which will also drop the page references
- * and allow the VM to clean up or page stuff.
- *
- * Non GEM backed objects treat this as a no-op as they are always GTT
- * backed objects.
- */
-void psb_gtt_unpin(struct gtt_range *gt)
-{
- struct drm_device *dev = gt->gem.dev;
- struct drm_psb_private *dev_priv = to_drm_psb_private(dev);
- u32 gpu_base = dev_priv->gtt.gatt_start;
-
- mutex_lock(&dev_priv->gtt_mutex);
-
- WARN_ON(!gt->in_gart);
-
- gt->in_gart--;
- if (gt->in_gart == 0 && gt->stolen == 0) {
- psb_mmu_remove_pages(psb_mmu_get_default_pd(dev_priv->mmu),
- (gpu_base + gt->offset), gt->npage, 0, 0);
- psb_gtt_remove(dev, gt);
- psb_gtt_detach_pages(gt);
- }
-
- mutex_unlock(&dev_priv->gtt_mutex);
}
/*
- * GTT resource allocator - allocate and manage GTT address space
+ * Remove a preallocated GTT range from the GTT. Overwrite all the
+ * page table entries with the dummy page. This is protected via the gtt
+ * mutex which the caller must hold.
*/
-
-/**
- * psb_gtt_alloc_range - allocate GTT address space
- * @dev: Our DRM device
- * @len: length (bytes) of address space required
- * @name: resource name
- * @backed: resource should be backed by stolen pages
- * @align: requested alignment
- *
- * Ask the kernel core to find us a suitable range of addresses
- * to use for a GTT mapping.
- *
- * Returns a gtt_range structure describing the object, or NULL on
- * error. On successful return the resource is both allocated and marked
- * as in use.
- */
-struct gtt_range *psb_gtt_alloc_range(struct drm_device *dev, int len,
- const char *name, int backed, u32 align)
+void psb_gtt_remove_pages(struct drm_psb_private *pdev, const struct resource *res)
{
- struct drm_psb_private *dev_priv = to_drm_psb_private(dev);
- struct gtt_range *gt;
- struct resource *r = dev_priv->gtt_mem;
- int ret;
- unsigned long start, end;
+ resource_size_t npages, i;
+ u32 __iomem *gtt_slot;
+ u32 pte;
- if (backed) {
- /* The start of the GTT is the stolen pages */
- start = r->start;
- end = r->start + dev_priv->gtt.stolen_size - 1;
- } else {
- /* The rest we will use for GEM backed objects */
- start = r->start + dev_priv->gtt.stolen_size;
- end = r->end;
- }
+ /* Install scratch page for the resource */
- gt = kzalloc(sizeof(struct gtt_range), GFP_KERNEL);
- if (gt == NULL)
- return NULL;
- gt->resource.name = name;
- gt->stolen = backed;
- gt->in_gart = backed;
- /* Ensure this is set for non GEM objects */
- gt->gem.dev = dev;
- ret = allocate_resource(dev_priv->gtt_mem, >->resource,
- len, start, end, align, NULL, NULL);
- if (ret == 0) {
- gt->offset = gt->resource.start - r->start;
- return gt;
- }
- kfree(gt);
- return NULL;
-}
+ pte = psb_gtt_mask_pte(page_to_pfn(pdev->scratch_page), PSB_MMU_CACHED_MEMORY);
-/**
- * psb_gtt_free_range - release GTT address space
- * @dev: our DRM device
- * @gt: a mapping created with psb_gtt_alloc_range
- *
- * Release a resource that was allocated with psb_gtt_alloc_range. If the
- * object has been pinned by mmap users we clean this up here currently.
- */
-void psb_gtt_free_range(struct drm_device *dev, struct gtt_range *gt)
-{
- /* Undo the mmap pin if we are destroying the object */
- if (gt->mmapping) {
- psb_gtt_unpin(gt);
- gt->mmapping = 0;
- }
- WARN_ON(gt->in_gart && !gt->stolen);
- release_resource(>->resource);
- kfree(gt);
+ npages = resource_size(res) >> PAGE_SHIFT;
+ gtt_slot = psb_gtt_entry(pdev, res);
+
+ for (i = 0; i < npages; ++i, ++gtt_slot)
+ iowrite32(pte, gtt_slot);
+
+ /* Make sure all the entries are set before we return */
+ ioread32(gtt_slot - 1);
}
static void psb_gtt_alloc(struct drm_device *dev)
@@ -498,7 +303,7 @@ int psb_gtt_restore(struct drm_device *dev)
{
struct drm_psb_private *dev_priv = to_drm_psb_private(dev);
struct resource *r = dev_priv->gtt_mem->child;
- struct gtt_range *range;
+ struct psb_gem_object *pobj;
unsigned int restored = 0, total = 0, size = 0;
/* On resume, the gtt_mutex is already initialized */
@@ -506,10 +311,15 @@ int psb_gtt_restore(struct drm_device *dev)
psb_gtt_init(dev, 1);
while (r != NULL) {
- range = container_of(r, struct gtt_range, resource);
- if (range->pages) {
- psb_gtt_insert(dev, range, 1);
- size += range->resource.end - range->resource.start;
+ /*
+ * TODO: GTT restoration needs a refactoring, so that we don't have to touch
+ * struct psb_gem_object here. The type represents a GEM object and is
+ * not related to the GTT itself.
+ */
+ pobj = container_of(r, struct psb_gem_object, resource);
+ if (pobj->pages) {
+ psb_gtt_insert_pages(dev_priv, &pobj->resource, pobj->pages);
+ size += pobj->resource.end - pobj->resource.start;
restored++;
}
r = r->sibling;
diff --git a/drivers/gpu/drm/gma500/gtt.h b/drivers/gpu/drm/gma500/gtt.h
index 2bf1658..ff1dcdd 100644
--- a/drivers/gpu/drm/gma500/gtt.h
+++ b/drivers/gpu/drm/gma500/gtt.h
@@ -10,6 +10,8 @@
#include <drm/drm_gem.h>
+struct drm_psb_private;
+
/* This wants cleaning up with respect to the psb_dev and un-needed stuff */
struct psb_gtt {
uint32_t gatt_start;
@@ -26,27 +28,14 @@ struct psb_gtt {
/* Exported functions */
extern int psb_gtt_init(struct drm_device *dev, int resume);
extern void psb_gtt_takedown(struct drm_device *dev);
-
-/* Each gtt_range describes an allocation in the GTT area */
-struct gtt_range {
- struct resource resource; /* Resource for our allocation */
- u32 offset; /* GTT offset of our object */
- struct drm_gem_object gem; /* GEM high level stuff */
- int in_gart; /* Currently in the GART (ref ct) */
- bool stolen; /* Backed from stolen RAM */
- bool mmapping; /* Is mmappable */
- struct page **pages; /* Backing pages if present */
- int npage; /* Number of backing pages */
-};
-
-#define to_gtt_range(x) container_of(x, struct gtt_range, gem)
-
-extern struct gtt_range *psb_gtt_alloc_range(struct drm_device *dev, int len,
- const char *name, int backed,
- u32 align);
-extern void psb_gtt_kref_put(struct gtt_range *gt);
-extern void psb_gtt_free_range(struct drm_device *dev, struct gtt_range *gt);
-extern int psb_gtt_pin(struct gtt_range *gt);
-extern void psb_gtt_unpin(struct gtt_range *gt);
extern int psb_gtt_restore(struct drm_device *dev);
+
+int psb_gtt_allocate_resource(struct drm_psb_private *pdev, struct resource *res,
+ const char *name, resource_size_t size, resource_size_t align,
+ bool stolen, u32 *offset);
+
+void psb_gtt_insert_pages(struct drm_psb_private *pdev, const struct resource *res,
+ struct page **pages);
+void psb_gtt_remove_pages(struct drm_psb_private *pdev, const struct resource *res);
+
#endif
diff --git a/drivers/gpu/drm/gma500/oaktrail_crtc.c b/drivers/gpu/drm/gma500/oaktrail_crtc.c
index c6b1159..36c7c26 100644
--- a/drivers/gpu/drm/gma500/oaktrail_crtc.c
+++ b/drivers/gpu/drm/gma500/oaktrail_crtc.c
@@ -10,6 +10,7 @@
#include <drm/drm_fourcc.h>
#include "framebuffer.h"
+#include "gem.h"
#include "gma_display.h"
#include "power.h"
#include "psb_drv.h"
@@ -608,7 +609,7 @@ static int oaktrail_pipe_set_base(struct drm_crtc *crtc,
if (!gma_power_begin(dev, true))
return 0;
- start = to_gtt_range(fb->obj[0])->offset;
+ start = to_psb_gem_object(fb->obj[0])->offset;
offset = y * fb->pitches[0] + x * fb->format->cpp[0];
REG_WRITE(map->stride, fb->pitches[0]);
diff --git a/drivers/gpu/drm/gma500/psb_drv.c b/drivers/gpu/drm/gma500/psb_drv.c
index 7a10bb3..65cf1c7 100644
--- a/drivers/gpu/drm/gma500/psb_drv.c
+++ b/drivers/gpu/drm/gma500/psb_drv.c
@@ -19,6 +19,7 @@
#include <acpi/video.h>
#include <drm/drm.h>
+#include <drm/drm_aperture.h>
#include <drm/drm_drv.h>
#include <drm/drm_fb_helper.h>
#include <drm/drm_file.h>
@@ -448,6 +449,17 @@ static int psb_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
struct drm_device *dev;
int ret;
+ /*
+ * We cannot yet easily find the framebuffer's location in memory. So
+ * remove all framebuffers here.
+ *
+ * TODO: Refactor psb_driver_load() to map vdc_reg earlier. Then we
+ * might be able to read the framebuffer range from the device.
+ */
+ ret = drm_aperture_remove_framebuffers(true, &driver);
+ if (ret)
+ return ret;
+
ret = pcim_enable_device(pdev);
if (ret)
return ret;
diff --git a/drivers/gpu/drm/gma500/psb_intel_display.c b/drivers/gpu/drm/gma500/psb_intel_display.c
index f5f259f..d5f9521 100644
--- a/drivers/gpu/drm/gma500/psb_intel_display.c
+++ b/drivers/gpu/drm/gma500/psb_intel_display.c
@@ -12,6 +12,7 @@
#include <drm/drm_plane_helper.h>
#include "framebuffer.h"
+#include "gem.h"
#include "gma_display.h"
#include "power.h"
#include "psb_drv.h"
@@ -454,23 +455,21 @@ static void psb_intel_cursor_init(struct drm_device *dev,
struct drm_psb_private *dev_priv = to_drm_psb_private(dev);
u32 control[3] = { CURACNTR, CURBCNTR, CURCCNTR };
u32 base[3] = { CURABASE, CURBBASE, CURCBASE };
- struct gtt_range *cursor_gt;
+ struct psb_gem_object *cursor_pobj;
if (dev_priv->ops->cursor_needs_phys) {
/* Allocate 4 pages of stolen mem for a hardware cursor. That
* is enough for the 64 x 64 ARGB cursors we support.
*/
- cursor_gt = psb_gtt_alloc_range(dev, 4 * PAGE_SIZE, "cursor", 1,
- PAGE_SIZE);
- if (!cursor_gt) {
- gma_crtc->cursor_gt = NULL;
+ cursor_pobj = psb_gem_create(dev, 4 * PAGE_SIZE, "cursor", true, PAGE_SIZE);
+ if (IS_ERR(cursor_pobj)) {
+ gma_crtc->cursor_pobj = NULL;
goto out;
}
- gma_crtc->cursor_gt = cursor_gt;
- gma_crtc->cursor_addr = dev_priv->stolen_base +
- cursor_gt->offset;
+ gma_crtc->cursor_pobj = cursor_pobj;
+ gma_crtc->cursor_addr = dev_priv->stolen_base + cursor_pobj->offset;
} else {
- gma_crtc->cursor_gt = NULL;
+ gma_crtc->cursor_pobj = NULL;
}
out:
diff --git a/drivers/gpu/drm/gma500/psb_intel_drv.h b/drivers/gpu/drm/gma500/psb_intel_drv.h
index 5340225..db3e757 100644
--- a/drivers/gpu/drm/gma500/psb_intel_drv.h
+++ b/drivers/gpu/drm/gma500/psb_intel_drv.h
@@ -140,7 +140,7 @@ struct gma_crtc {
int pipe;
int plane;
uint32_t cursor_addr;
- struct gtt_range *cursor_gt;
+ struct psb_gem_object *cursor_pobj;
u8 lut_adj[256];
struct psb_intel_framebuffer *fbdev_fb;
/* a mode_set for fbdev users on this crtc */
diff --git a/drivers/gpu/drm/gud/gud_pipe.c b/drivers/gpu/drm/gud/gud_pipe.c
index daf75c1..a150a5a 100644
--- a/drivers/gpu/drm/gud/gud_pipe.c
+++ b/drivers/gpu/drm/gud/gud_pipe.c
@@ -74,7 +74,7 @@ static size_t gud_xrgb8888_to_r124(u8 *dst, const struct drm_format_info *format
if (!buf)
return 0;
- drm_fb_xrgb8888_to_gray8(buf, src, fb, rect);
+ drm_fb_xrgb8888_to_gray8(buf, 0, src, fb, rect);
pix8 = buf;
for (y = 0; y < height; y++) {
@@ -190,23 +190,23 @@ static int gud_prep_flush(struct gud_device *gdrm, struct drm_framebuffer *fb,
goto end_cpu_access;
}
} else if (format->format == DRM_FORMAT_R8) {
- drm_fb_xrgb8888_to_gray8(buf, vaddr, fb, rect);
+ drm_fb_xrgb8888_to_gray8(buf, 0, vaddr, fb, rect);
} else if (format->format == DRM_FORMAT_RGB332) {
- drm_fb_xrgb8888_to_rgb332(buf, vaddr, fb, rect);
+ drm_fb_xrgb8888_to_rgb332(buf, 0, vaddr, fb, rect);
} else if (format->format == DRM_FORMAT_RGB565) {
- drm_fb_xrgb8888_to_rgb565(buf, vaddr, fb, rect, gud_is_big_endian());
+ drm_fb_xrgb8888_to_rgb565(buf, 0, vaddr, fb, rect, gud_is_big_endian());
} else if (format->format == DRM_FORMAT_RGB888) {
- drm_fb_xrgb8888_to_rgb888(buf, vaddr, fb, rect);
+ drm_fb_xrgb8888_to_rgb888(buf, 0, vaddr, fb, rect);
} else {
len = gud_xrgb8888_to_color(buf, format, vaddr, fb, rect);
}
} else if (gud_is_big_endian() && format->cpp[0] > 1) {
- drm_fb_swab(buf, vaddr, fb, rect, !import_attach);
+ drm_fb_swab(buf, 0, vaddr, fb, rect, !import_attach);
} else if (compression && !import_attach && pitch == fb->pitches[0]) {
/* can compress directly from the framebuffer */
buf = vaddr + rect->y1 * pitch;
} else {
- drm_fb_memcpy(buf, vaddr, fb, rect);
+ drm_fb_memcpy(buf, 0, vaddr, fb, rect);
}
memset(req, 0, sizeof(*req));
diff --git a/drivers/gpu/drm/hisilicon/kirin/Kconfig b/drivers/gpu/drm/hisilicon/kirin/Kconfig
index 290553e..b770f76 100644
--- a/drivers/gpu/drm/hisilicon/kirin/Kconfig
+++ b/drivers/gpu/drm/hisilicon/kirin/Kconfig
@@ -4,7 +4,6 @@
depends on DRM && OF && ARM64
select DRM_KMS_HELPER
select DRM_GEM_CMA_HELPER
- select DRM_KMS_CMA_HELPER
select DRM_MIPI_DSI
help
Choose this option if you have a hisilicon Kirin chipsets(hi6220).
diff --git a/drivers/gpu/drm/hisilicon/kirin/dw_drm_dsi.c b/drivers/gpu/drm/hisilicon/kirin/dw_drm_dsi.c
index 952cfdb..1d55648 100644
--- a/drivers/gpu/drm/hisilicon/kirin/dw_drm_dsi.c
+++ b/drivers/gpu/drm/hisilicon/kirin/dw_drm_dsi.c
@@ -81,7 +81,7 @@ struct dsi_hw_ctx {
struct dw_dsi {
struct drm_encoder encoder;
- struct drm_bridge *bridge;
+ struct device *dev;
struct mipi_dsi_host host;
struct drm_display_mode cur_mode;
struct dsi_hw_ctx *ctx;
@@ -720,10 +720,13 @@ static int dw_drm_encoder_init(struct device *dev,
return 0;
}
+static const struct component_ops dsi_ops;
static int dsi_host_attach(struct mipi_dsi_host *host,
struct mipi_dsi_device *mdsi)
{
struct dw_dsi *dsi = host_to_dsi(host);
+ struct device *dev = host->dev;
+ int ret;
if (mdsi->lanes < 1 || mdsi->lanes > 4) {
DRM_ERROR("dsi device params invalid\n");
@@ -734,13 +737,20 @@ static int dsi_host_attach(struct mipi_dsi_host *host,
dsi->format = mdsi->format;
dsi->mode_flags = mdsi->mode_flags;
+ ret = component_add(dev, &dsi_ops);
+ if (ret)
+ return ret;
+
return 0;
}
static int dsi_host_detach(struct mipi_dsi_host *host,
struct mipi_dsi_device *mdsi)
{
- /* do nothing */
+ struct device *dev = host->dev;
+
+ component_del(dev, &dsi_ops);
+
return 0;
}
@@ -768,7 +778,17 @@ static int dsi_host_init(struct device *dev, struct dw_dsi *dsi)
static int dsi_bridge_init(struct drm_device *dev, struct dw_dsi *dsi)
{
struct drm_encoder *encoder = &dsi->encoder;
- struct drm_bridge *bridge = dsi->bridge;
+ struct drm_bridge *bridge;
+ struct device_node *np = dsi->dev->of_node;
+ int ret;
+
+ /*
+ * Get the endpoint node. In our case, dsi has one output port1
+ * to which the external HDMI bridge is connected.
+ */
+ ret = drm_of_find_panel_or_bridge(np, 1, 0, NULL, &bridge);
+ if (ret)
+ return ret;
/* associate the bridge to dsi encoder */
return drm_bridge_attach(encoder, bridge, NULL, 0);
@@ -785,10 +805,6 @@ static int dsi_bind(struct device *dev, struct device *master, void *data)
if (ret)
return ret;
- ret = dsi_host_init(dev, dsi);
- if (ret)
- return ret;
-
ret = dsi_bridge_init(drm_dev, dsi);
if (ret)
return ret;
@@ -809,17 +825,7 @@ static const struct component_ops dsi_ops = {
static int dsi_parse_dt(struct platform_device *pdev, struct dw_dsi *dsi)
{
struct dsi_hw_ctx *ctx = dsi->ctx;
- struct device_node *np = pdev->dev.of_node;
struct resource *res;
- int ret;
-
- /*
- * Get the endpoint node. In our case, dsi has one output port1
- * to which the external HDMI bridge is connected.
- */
- ret = drm_of_find_panel_or_bridge(np, 1, 0, NULL, &dsi->bridge);
- if (ret)
- return ret;
ctx->pclk = devm_clk_get(&pdev->dev, "pclk");
if (IS_ERR(ctx->pclk)) {
@@ -852,6 +858,7 @@ static int dsi_probe(struct platform_device *pdev)
dsi = &data->dsi;
ctx = &data->ctx;
dsi->ctx = ctx;
+ dsi->dev = &pdev->dev;
ret = dsi_parse_dt(pdev, dsi);
if (ret)
@@ -859,12 +866,19 @@ static int dsi_probe(struct platform_device *pdev)
platform_set_drvdata(pdev, data);
- return component_add(&pdev->dev, &dsi_ops);
+ ret = dsi_host_init(&pdev->dev, dsi);
+ if (ret)
+ return ret;
+
+ return 0;
}
static int dsi_remove(struct platform_device *pdev)
{
- component_del(&pdev->dev, &dsi_ops);
+ struct dsi_data *data = platform_get_drvdata(pdev);
+ struct dw_dsi *dsi = &data->dsi;
+
+ mipi_dsi_host_unregister(&dsi->host);
return 0;
}
diff --git a/drivers/gpu/drm/hyperv/hyperv_drm_modeset.c b/drivers/gpu/drm/hyperv/hyperv_drm_modeset.c
index 8c97a20..93f51e7 100644
--- a/drivers/gpu/drm/hyperv/hyperv_drm_modeset.c
+++ b/drivers/gpu/drm/hyperv/hyperv_drm_modeset.c
@@ -23,13 +23,16 @@ static int hyperv_blit_to_vram_rect(struct drm_framebuffer *fb,
struct drm_rect *rect)
{
struct hyperv_drm_device *hv = to_hv(fb->dev);
+ void __iomem *dst = hv->vram;
void *vmap = map->vaddr; /* TODO: Use mapping abstraction properly */
int idx;
if (!drm_dev_enter(&hv->dev, &idx))
return -ENODEV;
- drm_fb_memcpy_dstclip(hv->vram, fb->pitches[0], vmap, fb, rect);
+ dst += drm_fb_clip_offset(fb->pitches[0], fb->format, rect);
+ drm_fb_memcpy_toio(dst, fb->pitches[0], vmap, fb, rect);
+
drm_dev_exit(idx);
return 0;
diff --git a/drivers/gpu/drm/i915/Kconfig b/drivers/gpu/drm/i915/Kconfig
index 84b6fc7..a4c94dc 100644
--- a/drivers/gpu/drm/i915/Kconfig
+++ b/drivers/gpu/drm/i915/Kconfig
@@ -21,7 +21,7 @@
select ACPI_VIDEO if ACPI
select ACPI_BUTTON if ACPI
select SYNC_FILE
- select IOSF_MBI
+ select IOSF_MBI if X86
select CRC32
select SND_HDA_I915 if SND_HDA_CORE
select CEC_CORE if CEC_NOTIFIER
diff --git a/drivers/gpu/drm/i915/Makefile b/drivers/gpu/drm/i915/Makefile
index 660bb03..1b62b9f 100644
--- a/drivers/gpu/drm/i915/Makefile
+++ b/drivers/gpu/drm/i915/Makefile
@@ -30,7 +30,7 @@
# Please keep these build lists sorted!
# core driver code
-i915-y += i915_drv.o \
+i915-y += i915_driver.o \
i915_config.o \
i915_irq.o \
i915_getparam.o \
@@ -60,7 +60,6 @@
# core library code
i915-y += \
- dma_resv_utils.o \
i915_memcpy.o \
i915_mm.o \
i915_sw_fence.o \
@@ -154,6 +153,7 @@
gem/i915_gem_throttle.o \
gem/i915_gem_tiling.o \
gem/i915_gem_ttm.o \
+ gem/i915_gem_ttm_move.o \
gem/i915_gem_ttm_pm.o \
gem/i915_gem_userptr.o \
gem/i915_gem_wait.o \
@@ -163,6 +163,7 @@
i915_active.o \
i915_buddy.o \
i915_cmd_parser.o \
+ i915_deps.o \
i915_gem_evict.o \
i915_gem_gtt.o \
i915_gem_ww.o \
@@ -173,6 +174,7 @@
i915_trace_points.o \
i915_ttm_buddy_manager.o \
i915_vma.o \
+ i915_vma_snapshot.o \
intel_wopcm.o
# general-purpose microcontroller (GuC) support
@@ -226,6 +228,8 @@
display/intel_hotplug.o \
display/intel_lpe_audio.o \
display/intel_overlay.o \
+ display/intel_pch_display.o \
+ display/intel_pch_refclk.o \
display/intel_plane_initial.o \
display/intel_psr.o \
display/intel_quirks.o \
@@ -256,6 +260,7 @@
display/intel_crt.o \
display/intel_ddi.o \
display/intel_ddi_buf_trans.o \
+ display/intel_display_trace.o \
display/intel_dp.o \
display/intel_dp_aux.o \
display/intel_dp_aux_backlight.o \
diff --git a/drivers/gpu/drm/i915/display/g4x_dp.c b/drivers/gpu/drm/i915/display/g4x_dp.c
index dc41868..f37677d 100644
--- a/drivers/gpu/drm/i915/display/g4x_dp.c
+++ b/drivers/gpu/drm/i915/display/g4x_dp.c
@@ -9,6 +9,7 @@
#include "intel_audio.h"
#include "intel_backlight.h"
#include "intel_connector.h"
+#include "intel_crtc.h"
#include "intel_de.h"
#include "intel_display_types.h"
#include "intel_dp.h"
diff --git a/drivers/gpu/drm/i915/display/g4x_hdmi.c b/drivers/gpu/drm/i915/display/g4x_hdmi.c
index f5b4dd5..06e00b1 100644
--- a/drivers/gpu/drm/i915/display/g4x_hdmi.c
+++ b/drivers/gpu/drm/i915/display/g4x_hdmi.c
@@ -8,6 +8,7 @@
#include "g4x_hdmi.h"
#include "intel_audio.h"
#include "intel_connector.h"
+#include "intel_crtc.h"
#include "intel_de.h"
#include "intel_display_types.h"
#include "intel_dpio_phy.h"
diff --git a/drivers/gpu/drm/i915/display/i9xx_plane.c b/drivers/gpu/drm/i915/display/i9xx_plane.c
index b1439ba..85950ff 100644
--- a/drivers/gpu/drm/i915/display/i9xx_plane.c
+++ b/drivers/gpu/drm/i915/display/i9xx_plane.c
@@ -13,6 +13,7 @@
#include "intel_de.h"
#include "intel_display_types.h"
#include "intel_fb.h"
+#include "intel_fbc.h"
#include "intel_sprite.h"
#include "i9xx_plane.h"
@@ -60,22 +61,11 @@ static const u32 vlv_primary_formats[] = {
DRM_FORMAT_XBGR16161616F,
};
-static const u64 i9xx_format_modifiers[] = {
- I915_FORMAT_MOD_X_TILED,
- DRM_FORMAT_MOD_LINEAR,
- DRM_FORMAT_MOD_INVALID
-};
-
static bool i8xx_plane_format_mod_supported(struct drm_plane *_plane,
u32 format, u64 modifier)
{
- switch (modifier) {
- case DRM_FORMAT_MOD_LINEAR:
- case I915_FORMAT_MOD_X_TILED:
- break;
- default:
+ if (!intel_fb_plane_supports_modifier(to_intel_plane(_plane), modifier))
return false;
- }
switch (format) {
case DRM_FORMAT_C8:
@@ -92,13 +82,8 @@ static bool i8xx_plane_format_mod_supported(struct drm_plane *_plane,
static bool i965_plane_format_mod_supported(struct drm_plane *_plane,
u32 format, u64 modifier)
{
- switch (modifier) {
- case DRM_FORMAT_MOD_LINEAR:
- case I915_FORMAT_MOD_X_TILED:
- break;
- default:
+ if (!intel_fb_plane_supports_modifier(to_intel_plane(_plane), modifier))
return false;
- }
switch (format) {
case DRM_FORMAT_C8:
@@ -136,6 +121,15 @@ static bool i9xx_plane_has_fbc(struct drm_i915_private *dev_priv,
return i9xx_plane == PLANE_A;
}
+static struct intel_fbc *i9xx_plane_fbc(struct drm_i915_private *dev_priv,
+ enum i9xx_plane_id i9xx_plane)
+{
+ if (i9xx_plane_has_fbc(dev_priv, i9xx_plane))
+ return dev_priv->fbc;
+ else
+ return NULL;
+}
+
static bool i9xx_plane_has_windowing(struct intel_plane *plane)
{
struct drm_i915_private *dev_priv = to_i915(plane->base.dev);
@@ -272,7 +266,7 @@ int i9xx_check_plane_surface(struct intel_plane_state *plane_state)
u32 alignment = intel_surf_alignment(fb, 0);
int cpp = fb->format->cpp[0];
- while ((src_x + src_w) * cpp > plane_state->view.color_plane[0].stride) {
+ while ((src_x + src_w) * cpp > plane_state->view.color_plane[0].mapping_stride) {
if (offset == 0) {
drm_dbg_kms(&dev_priv->drm,
"Unable to find suitable display surface offset due to X-tiling\n");
@@ -418,22 +412,49 @@ static int i9xx_plane_min_cdclk(const struct intel_crtc_state *crtc_state,
return DIV_ROUND_UP(pixel_rate * num, den);
}
-static void i9xx_update_plane(struct intel_plane *plane,
- const struct intel_crtc_state *crtc_state,
- const struct intel_plane_state *plane_state)
+static void i9xx_plane_update_noarm(struct intel_plane *plane,
+ const struct intel_crtc_state *crtc_state,
+ const struct intel_plane_state *plane_state)
{
struct drm_i915_private *dev_priv = to_i915(plane->base.dev);
enum i9xx_plane_id i9xx_plane = plane->i9xx_plane;
- u32 linear_offset;
+ unsigned long irqflags;
+
+ spin_lock_irqsave(&dev_priv->uncore.lock, irqflags);
+
+ intel_de_write_fw(dev_priv, DSPSTRIDE(i9xx_plane),
+ plane_state->view.color_plane[0].mapping_stride);
+
+ if (DISPLAY_VER(dev_priv) < 4) {
+ int crtc_x = plane_state->uapi.dst.x1;
+ int crtc_y = plane_state->uapi.dst.y1;
+ int crtc_w = drm_rect_width(&plane_state->uapi.dst);
+ int crtc_h = drm_rect_height(&plane_state->uapi.dst);
+
+ /*
+ * PLANE_A doesn't actually have a full window
+ * generator but let's assume we still need to
+ * program whatever is there.
+ */
+ intel_de_write_fw(dev_priv, DSPPOS(i9xx_plane),
+ (crtc_y << 16) | crtc_x);
+ intel_de_write_fw(dev_priv, DSPSIZE(i9xx_plane),
+ ((crtc_h - 1) << 16) | (crtc_w - 1));
+ }
+
+ spin_unlock_irqrestore(&dev_priv->uncore.lock, irqflags);
+}
+
+static void i9xx_plane_update_arm(struct intel_plane *plane,
+ const struct intel_crtc_state *crtc_state,
+ const struct intel_plane_state *plane_state)
+{
+ struct drm_i915_private *dev_priv = to_i915(plane->base.dev);
+ enum i9xx_plane_id i9xx_plane = plane->i9xx_plane;
int x = plane_state->view.color_plane[0].x;
int y = plane_state->view.color_plane[0].y;
- int crtc_x = plane_state->uapi.dst.x1;
- int crtc_y = plane_state->uapi.dst.y1;
- int crtc_w = drm_rect_width(&plane_state->uapi.dst);
- int crtc_h = drm_rect_height(&plane_state->uapi.dst);
+ u32 dspcntr, dspaddr_offset, linear_offset;
unsigned long irqflags;
- u32 dspaddr_offset;
- u32 dspcntr;
dspcntr = plane_state->ctl | i9xx_plane_ctl_crtc(crtc_state);
@@ -446,20 +467,12 @@ static void i9xx_update_plane(struct intel_plane *plane,
spin_lock_irqsave(&dev_priv->uncore.lock, irqflags);
- intel_de_write_fw(dev_priv, DSPSTRIDE(i9xx_plane),
- plane_state->view.color_plane[0].stride);
+ if (IS_CHERRYVIEW(dev_priv) && i9xx_plane == PLANE_B) {
+ int crtc_x = plane_state->uapi.dst.x1;
+ int crtc_y = plane_state->uapi.dst.y1;
+ int crtc_w = drm_rect_width(&plane_state->uapi.dst);
+ int crtc_h = drm_rect_height(&plane_state->uapi.dst);
- if (DISPLAY_VER(dev_priv) < 4) {
- /*
- * PLANE_A doesn't actually have a full window
- * generator but let's assume we still need to
- * program whatever is there.
- */
- intel_de_write_fw(dev_priv, DSPPOS(i9xx_plane),
- (crtc_y << 16) | crtc_x);
- intel_de_write_fw(dev_priv, DSPSIZE(i9xx_plane),
- ((crtc_h - 1) << 16) | (crtc_w - 1));
- } else if (IS_CHERRYVIEW(dev_priv) && i9xx_plane == PLANE_B) {
intel_de_write_fw(dev_priv, PRIMPOS(i9xx_plane),
(crtc_y << 16) | crtc_x);
intel_de_write_fw(dev_priv, PRIMSIZE(i9xx_plane),
@@ -493,8 +506,22 @@ static void i9xx_update_plane(struct intel_plane *plane,
spin_unlock_irqrestore(&dev_priv->uncore.lock, irqflags);
}
-static void i9xx_disable_plane(struct intel_plane *plane,
- const struct intel_crtc_state *crtc_state)
+static void i830_plane_update_arm(struct intel_plane *plane,
+ const struct intel_crtc_state *crtc_state,
+ const struct intel_plane_state *plane_state)
+{
+ /*
+ * On i830/i845 all registers are self-arming [ALM040].
+ *
+ * Additional breakage on i830 causes register reads to return
+ * the last latched value instead of the last written value [ALM026].
+ */
+ i9xx_plane_update_noarm(plane, crtc_state, plane_state);
+ i9xx_plane_update_arm(plane, crtc_state, plane_state);
+}
+
+static void i9xx_plane_disable_arm(struct intel_plane *plane,
+ const struct intel_crtc_state *crtc_state)
{
struct drm_i915_private *dev_priv = to_i915(plane->base.dev);
enum i9xx_plane_id i9xx_plane = plane->i9xx_plane;
@@ -768,6 +795,7 @@ intel_primary_plane_create(struct drm_i915_private *dev_priv, enum pipe pipe)
struct intel_plane *plane;
const struct drm_plane_funcs *plane_funcs;
unsigned int supported_rotations;
+ const u64 *modifiers;
const u32 *formats;
int num_formats;
int ret, zpos;
@@ -789,12 +817,7 @@ intel_primary_plane_create(struct drm_i915_private *dev_priv, enum pipe pipe)
plane->id = PLANE_PRIMARY;
plane->frontbuffer_bit = INTEL_FRONTBUFFER(pipe, plane->id);
- plane->has_fbc = i9xx_plane_has_fbc(dev_priv, plane->i9xx_plane);
- if (plane->has_fbc) {
- struct intel_fbc *fbc = &dev_priv->fbc;
-
- fbc->possible_framebuffer_bits |= plane->frontbuffer_bit;
- }
+ intel_fbc_add_plane(i9xx_plane_fbc(dev_priv, plane->i9xx_plane), plane);
if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv)) {
formats = vlv_primary_formats;
@@ -851,8 +874,13 @@ intel_primary_plane_create(struct drm_i915_private *dev_priv, enum pipe pipe)
plane->max_stride = ilk_primary_max_stride;
}
- plane->update_plane = i9xx_update_plane;
- plane->disable_plane = i9xx_disable_plane;
+ if (IS_I830(dev_priv) || IS_I845G(dev_priv)) {
+ plane->update_arm = i830_plane_update_arm;
+ } else {
+ plane->update_noarm = i9xx_plane_update_noarm;
+ plane->update_arm = i9xx_plane_update_arm;
+ }
+ plane->disable_arm = i9xx_plane_disable_arm;
plane->get_hw_state = i9xx_plane_get_hw_state;
plane->check_plane = i9xx_plane_check;
@@ -875,21 +903,26 @@ intel_primary_plane_create(struct drm_i915_private *dev_priv, enum pipe pipe)
plane->disable_flip_done = ilk_primary_disable_flip_done;
}
+ modifiers = intel_fb_plane_get_modifiers(dev_priv, INTEL_PLANE_CAP_TILING_X);
+
if (DISPLAY_VER(dev_priv) >= 5 || IS_G4X(dev_priv))
ret = drm_universal_plane_init(&dev_priv->drm, &plane->base,
0, plane_funcs,
formats, num_formats,
- i9xx_format_modifiers,
+ modifiers,
DRM_PLANE_TYPE_PRIMARY,
"primary %c", pipe_name(pipe));
else
ret = drm_universal_plane_init(&dev_priv->drm, &plane->base,
0, plane_funcs,
formats, num_formats,
- i9xx_format_modifiers,
+ modifiers,
DRM_PLANE_TYPE_PRIMARY,
"plane %c",
plane_name(plane->i9xx_plane));
+
+ kfree(modifiers);
+
if (ret)
goto fail;
diff --git a/drivers/gpu/drm/i915/display/icl_dsi.c b/drivers/gpu/drm/i915/display/icl_dsi.c
index 71fbdcd..5781e9fa 100644
--- a/drivers/gpu/drm/i915/display/icl_dsi.c
+++ b/drivers/gpu/drm/i915/display/icl_dsi.c
@@ -28,6 +28,7 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_mipi_dsi.h>
+#include "icl_dsi.h"
#include "intel_atomic.h"
#include "intel_backlight.h"
#include "intel_combo_phy.h"
@@ -36,6 +37,7 @@
#include "intel_ddi.h"
#include "intel_de.h"
#include "intel_dsi.h"
+#include "intel_dsi_vbt.h"
#include "intel_panel.h"
#include "intel_vdsc.h"
#include "skl_scaler.h"
@@ -183,6 +185,8 @@ static int dsi_send_pkt_hdr(struct intel_dsi_host *host,
if (enable_lpdt)
tmp |= LP_DATA_TRANSFER;
+ else
+ tmp &= ~LP_DATA_TRANSFER;
tmp &= ~(PARAM_WC_MASK | VC_MASK | DT_MASK);
tmp |= ((packet->header[0] & VC_MASK) << VC_SHIFT);
@@ -1226,7 +1230,9 @@ static void gen11_dsi_pre_enable(struct intel_atomic_state *state,
/* step5: program and powerup panel */
gen11_dsi_powerup_panel(encoder);
- intel_dsc_enable(encoder, pipe_config);
+ intel_dsc_dsi_pps_write(encoder, pipe_config);
+
+ intel_dsc_enable(pipe_config);
/* step6c: configure transcoder timings */
gen11_dsi_set_transcoder_timings(encoder, pipe_config);
@@ -1623,7 +1629,7 @@ static int gen11_dsi_dsc_compute_config(struct intel_encoder *encoder,
/* FIXME: initialize from VBT */
vdsc_cfg->rc_model_size = DSC_RC_MODEL_SIZE_CONST;
- ret = intel_dsc_compute_params(encoder, crtc_state);
+ ret = intel_dsc_compute_params(crtc_state);
if (ret)
return ret;
diff --git a/drivers/gpu/drm/i915/display/icl_dsi.h b/drivers/gpu/drm/i915/display/icl_dsi.h
new file mode 100644
index 0000000..b4861b5
--- /dev/null
+++ b/drivers/gpu/drm/i915/display/icl_dsi.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2021 Intel Corporation
+ */
+
+#ifndef __ICL_DSI_H__
+#define __ICL_DSI_H__
+
+struct drm_i915_private;
+struct intel_crtc_state;
+
+void icl_dsi_init(struct drm_i915_private *i915);
+void icl_dsi_frame_update(struct intel_crtc_state *crtc_state);
+
+#endif /* __ICL_DSI_H__ */
diff --git a/drivers/gpu/drm/i915/display/intel_atomic.c b/drivers/gpu/drm/i915/display/intel_atomic.c
index b4e7ac5..a625507 100644
--- a/drivers/gpu/drm/i915/display/intel_atomic.c
+++ b/drivers/gpu/drm/i915/display/intel_atomic.c
@@ -139,6 +139,7 @@ int intel_digital_connector_atomic_check(struct drm_connector *conn,
new_conn_state->base.picture_aspect_ratio != old_conn_state->base.picture_aspect_ratio ||
new_conn_state->base.content_type != old_conn_state->base.content_type ||
new_conn_state->base.scaling_mode != old_conn_state->base.scaling_mode ||
+ new_conn_state->base.privacy_screen_sw_state != old_conn_state->base.privacy_screen_sw_state ||
!drm_connector_atomic_hdr_metadata_equal(old_state, new_state))
crtc_state->mode_changed = true;
diff --git a/drivers/gpu/drm/i915/display/intel_atomic_plane.c b/drivers/gpu/drm/i915/display/intel_atomic_plane.c
index 0be8c00..c2c512c 100644
--- a/drivers/gpu/drm/i915/display/intel_atomic_plane.c
+++ b/drivers/gpu/drm/i915/display/intel_atomic_plane.c
@@ -35,14 +35,16 @@
#include <drm/drm_fourcc.h>
#include <drm/drm_plane_helper.h>
-#include "i915_trace.h"
+#include "gt/intel_rps.h"
+
#include "intel_atomic_plane.h"
#include "intel_cdclk.h"
+#include "intel_display_trace.h"
#include "intel_display_types.h"
+#include "intel_fb.h"
#include "intel_fb_pin.h"
#include "intel_pm.h"
#include "intel_sprite.h"
-#include "gt/intel_rps.h"
static void intel_plane_state_reset(struct intel_plane_state *plane_state,
struct intel_plane *plane)
@@ -394,7 +396,7 @@ int intel_plane_atomic_check(struct intel_atomic_state *state,
const struct intel_plane_state *old_plane_state =
intel_atomic_get_old_plane_state(state, plane);
const struct intel_plane_state *new_master_plane_state;
- struct intel_crtc *crtc = intel_get_crtc_for_pipe(i915, plane->pipe);
+ struct intel_crtc *crtc = intel_crtc_for_pipe(i915, plane->pipe);
const struct intel_crtc_state *old_crtc_state =
intel_atomic_get_old_crtc_state(state, crtc);
struct intel_crtc_state *new_crtc_state =
@@ -469,31 +471,72 @@ skl_next_plane_to_commit(struct intel_atomic_state *state,
return NULL;
}
-void intel_update_plane(struct intel_plane *plane,
- const struct intel_crtc_state *crtc_state,
- const struct intel_plane_state *plane_state)
+void intel_plane_update_noarm(struct intel_plane *plane,
+ const struct intel_crtc_state *crtc_state,
+ const struct intel_plane_state *plane_state)
{
struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
- trace_intel_update_plane(&plane->base, crtc);
+ trace_intel_plane_update_noarm(&plane->base, crtc);
+
+ if (plane->update_noarm)
+ plane->update_noarm(plane, crtc_state, plane_state);
+}
+
+void intel_plane_update_arm(struct intel_plane *plane,
+ const struct intel_crtc_state *crtc_state,
+ const struct intel_plane_state *plane_state)
+{
+ struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
+
+ trace_intel_plane_update_arm(&plane->base, crtc);
if (crtc_state->uapi.async_flip && plane->async_flip)
plane->async_flip(plane, crtc_state, plane_state, true);
else
- plane->update_plane(plane, crtc_state, plane_state);
+ plane->update_arm(plane, crtc_state, plane_state);
}
-void intel_disable_plane(struct intel_plane *plane,
- const struct intel_crtc_state *crtc_state)
+void intel_plane_disable_arm(struct intel_plane *plane,
+ const struct intel_crtc_state *crtc_state)
{
struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
- trace_intel_disable_plane(&plane->base, crtc);
- plane->disable_plane(plane, crtc_state);
+ trace_intel_plane_disable_arm(&plane->base, crtc);
+ plane->disable_arm(plane, crtc_state);
}
-void skl_update_planes_on_crtc(struct intel_atomic_state *state,
- struct intel_crtc *crtc)
+void intel_update_planes_on_crtc(struct intel_atomic_state *state,
+ struct intel_crtc *crtc)
+{
+ struct intel_crtc_state *new_crtc_state =
+ intel_atomic_get_new_crtc_state(state, crtc);
+ u32 update_mask = new_crtc_state->update_planes;
+ struct intel_plane_state *new_plane_state;
+ struct intel_plane *plane;
+ int i;
+
+ if (new_crtc_state->uapi.async_flip)
+ return;
+
+ /*
+ * Since we only write non-arming registers here,
+ * the order does not matter even for skl+.
+ */
+ for_each_new_intel_plane_in_state(state, plane, new_plane_state, i) {
+ if (crtc->pipe != plane->pipe ||
+ !(update_mask & BIT(plane->id)))
+ continue;
+
+ /* TODO: for mailbox updates this should be skipped */
+ if (new_plane_state->uapi.visible ||
+ new_plane_state->planar_slave)
+ intel_plane_update_noarm(plane, new_crtc_state, new_plane_state);
+ }
+}
+
+void skl_arm_planes_on_crtc(struct intel_atomic_state *state,
+ struct intel_crtc *crtc)
{
struct intel_crtc_state *old_crtc_state =
intel_atomic_get_old_crtc_state(state, crtc);
@@ -515,17 +558,20 @@ void skl_update_planes_on_crtc(struct intel_atomic_state *state,
struct intel_plane_state *new_plane_state =
intel_atomic_get_new_plane_state(state, plane);
+ /*
+ * TODO: for mailbox updates intel_plane_update_noarm()
+ * would have to be called here as well.
+ */
if (new_plane_state->uapi.visible ||
- new_plane_state->planar_slave) {
- intel_update_plane(plane, new_crtc_state, new_plane_state);
- } else {
- intel_disable_plane(plane, new_crtc_state);
- }
+ new_plane_state->planar_slave)
+ intel_plane_update_arm(plane, new_crtc_state, new_plane_state);
+ else
+ intel_plane_disable_arm(plane, new_crtc_state);
}
}
-void i9xx_update_planes_on_crtc(struct intel_atomic_state *state,
- struct intel_crtc *crtc)
+void i9xx_arm_planes_on_crtc(struct intel_atomic_state *state,
+ struct intel_crtc *crtc)
{
struct intel_crtc_state *new_crtc_state =
intel_atomic_get_new_crtc_state(state, crtc);
@@ -539,10 +585,14 @@ void i9xx_update_planes_on_crtc(struct intel_atomic_state *state,
!(update_mask & BIT(plane->id)))
continue;
+ /*
+ * TODO: for mailbox updates intel_plane_update_noarm()
+ * would have to be called here as well.
+ */
if (new_plane_state->uapi.visible)
- intel_update_plane(plane, new_crtc_state, new_plane_state);
+ intel_plane_update_arm(plane, new_crtc_state, new_plane_state);
else
- intel_disable_plane(plane, new_crtc_state);
+ intel_plane_disable_arm(plane, new_crtc_state);
}
}
@@ -738,6 +788,7 @@ intel_prepare_plane_fb(struct drm_plane *_plane,
i915_gem_object_wait_priority(obj, 0, &attr);
if (!new_plane_state->uapi.fence) { /* implicit fencing */
+ struct dma_resv_iter cursor;
struct dma_fence *fence;
ret = i915_sw_fence_await_reservation(&state->commit_ready,
@@ -748,12 +799,12 @@ intel_prepare_plane_fb(struct drm_plane *_plane,
if (ret < 0)
goto unpin_fb;
- fence = dma_resv_get_excl_unlocked(obj->base.resv);
- if (fence) {
+ dma_resv_iter_begin(&cursor, obj->base.resv, false);
+ dma_resv_for_each_fence_unlocked(&cursor, fence) {
add_rps_boost_after_vblank(new_plane_state->hw.crtc,
fence);
- dma_fence_put(fence);
}
+ dma_resv_iter_end(&cursor);
} else {
add_rps_boost_after_vblank(new_plane_state->hw.crtc,
new_plane_state->uapi.fence);
@@ -768,7 +819,7 @@ intel_prepare_plane_fb(struct drm_plane *_plane,
* maximum clocks following a vblank miss (see do_rps_boost()).
*/
if (!state->rps_interactive) {
- intel_rps_mark_interactive(&dev_priv->gt.rps, true);
+ intel_rps_mark_interactive(&to_gt(dev_priv)->rps, true);
state->rps_interactive = true;
}
@@ -802,7 +853,7 @@ intel_cleanup_plane_fb(struct drm_plane *plane,
return;
if (state->rps_interactive) {
- intel_rps_mark_interactive(&dev_priv->gt.rps, false);
+ intel_rps_mark_interactive(&to_gt(dev_priv)->rps, false);
state->rps_interactive = false;
}
diff --git a/drivers/gpu/drm/i915/display/intel_atomic_plane.h b/drivers/gpu/drm/i915/display/intel_atomic_plane.h
index 62e5a2a..7907f60 100644
--- a/drivers/gpu/drm/i915/display/intel_atomic_plane.h
+++ b/drivers/gpu/drm/i915/display/intel_atomic_plane.h
@@ -30,20 +30,25 @@ void intel_plane_copy_uapi_to_hw_state(struct intel_plane_state *plane_state,
struct intel_crtc *crtc);
void intel_plane_copy_hw_state(struct intel_plane_state *plane_state,
const struct intel_plane_state *from_plane_state);
-void intel_update_plane(struct intel_plane *plane,
- const struct intel_crtc_state *crtc_state,
- const struct intel_plane_state *plane_state);
-void intel_disable_plane(struct intel_plane *plane,
- const struct intel_crtc_state *crtc_state);
+void intel_plane_update_noarm(struct intel_plane *plane,
+ const struct intel_crtc_state *crtc_state,
+ const struct intel_plane_state *plane_state);
+void intel_plane_update_arm(struct intel_plane *plane,
+ const struct intel_crtc_state *crtc_state,
+ const struct intel_plane_state *plane_state);
+void intel_plane_disable_arm(struct intel_plane *plane,
+ const struct intel_crtc_state *crtc_state);
struct intel_plane *intel_plane_alloc(void);
void intel_plane_free(struct intel_plane *plane);
struct drm_plane_state *intel_plane_duplicate_state(struct drm_plane *plane);
void intel_plane_destroy_state(struct drm_plane *plane,
struct drm_plane_state *state);
-void skl_update_planes_on_crtc(struct intel_atomic_state *state,
- struct intel_crtc *crtc);
-void i9xx_update_planes_on_crtc(struct intel_atomic_state *state,
- struct intel_crtc *crtc);
+void intel_update_planes_on_crtc(struct intel_atomic_state *state,
+ struct intel_crtc *crtc);
+void skl_arm_planes_on_crtc(struct intel_atomic_state *state,
+ struct intel_crtc *crtc);
+void i9xx_arm_planes_on_crtc(struct intel_atomic_state *state,
+ struct intel_crtc *crtc);
int intel_plane_atomic_check_with_state(const struct intel_crtc_state *old_crtc_state,
struct intel_crtc_state *crtc_state,
const struct intel_plane_state *old_plane_state,
diff --git a/drivers/gpu/drm/i915/display/intel_audio.c b/drivers/gpu/drm/i915/display/intel_audio.c
index 03e8c05..3bdca0f 100644
--- a/drivers/gpu/drm/i915/display/intel_audio.c
+++ b/drivers/gpu/drm/i915/display/intel_audio.c
@@ -31,6 +31,7 @@
#include "intel_atomic.h"
#include "intel_audio.h"
#include "intel_cdclk.h"
+#include "intel_crtc.h"
#include "intel_de.h"
#include "intel_display_types.h"
#include "intel_lpe_audio.h"
@@ -62,6 +63,15 @@
* struct &i915_audio_component_audio_ops @audio_ops is called from i915 driver.
*/
+struct intel_audio_funcs {
+ void (*audio_codec_enable)(struct intel_encoder *encoder,
+ const struct intel_crtc_state *crtc_state,
+ const struct drm_connector_state *conn_state);
+ void (*audio_codec_disable)(struct intel_encoder *encoder,
+ const struct intel_crtc_state *old_crtc_state,
+ const struct drm_connector_state *old_conn_state);
+};
+
/* DP N/M table */
#define LC_810M 810000
#define LC_540M 540000
@@ -388,7 +398,7 @@ hsw_dp_audio_config_update(struct intel_encoder *encoder,
const struct intel_crtc_state *crtc_state)
{
struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
- struct i915_audio_component *acomp = dev_priv->audio_component;
+ struct i915_audio_component *acomp = dev_priv->audio.component;
enum transcoder cpu_transcoder = crtc_state->cpu_transcoder;
enum port port = encoder->port;
const struct dp_aud_n_m *nm;
@@ -436,7 +446,7 @@ hsw_hdmi_audio_config_update(struct intel_encoder *encoder,
const struct intel_crtc_state *crtc_state)
{
struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
- struct i915_audio_component *acomp = dev_priv->audio_component;
+ struct i915_audio_component *acomp = dev_priv->audio.component;
enum transcoder cpu_transcoder = crtc_state->cpu_transcoder;
enum port port = encoder->port;
int n, rate;
@@ -494,7 +504,7 @@ static void hsw_audio_codec_disable(struct intel_encoder *encoder,
drm_dbg_kms(&dev_priv->drm, "Disable audio codec on transcoder %s\n",
transcoder_name(cpu_transcoder));
- mutex_lock(&dev_priv->av_mutex);
+ mutex_lock(&dev_priv->audio.mutex);
/* Disable timestamps */
tmp = intel_de_read(dev_priv, HSW_AUD_CFG(cpu_transcoder));
@@ -512,7 +522,7 @@ static void hsw_audio_codec_disable(struct intel_encoder *encoder,
tmp &= ~AUDIO_OUTPUT_ENABLE(cpu_transcoder);
intel_de_write(dev_priv, HSW_AUD_PIN_ELD_CP_VLD, tmp);
- mutex_unlock(&dev_priv->av_mutex);
+ mutex_unlock(&dev_priv->audio.mutex);
}
static unsigned int calc_hblank_early_prog(struct intel_encoder *encoder,
@@ -641,7 +651,7 @@ static void hsw_audio_codec_enable(struct intel_encoder *encoder,
"Enable audio codec on transcoder %s, %u bytes ELD\n",
transcoder_name(cpu_transcoder), drm_eld_size(eld));
- mutex_lock(&dev_priv->av_mutex);
+ mutex_lock(&dev_priv->audio.mutex);
/* Enable Audio WA for 4k DSC usecases */
if (intel_crtc_has_type(crtc_state, INTEL_OUTPUT_DP))
@@ -679,7 +689,7 @@ static void hsw_audio_codec_enable(struct intel_encoder *encoder,
/* Enable timestamps */
hsw_audio_config_update(encoder, crtc_state);
- mutex_unlock(&dev_priv->av_mutex);
+ mutex_unlock(&dev_priv->audio.mutex);
}
static void ilk_audio_codec_disable(struct intel_encoder *encoder,
@@ -826,7 +836,7 @@ void intel_audio_codec_enable(struct intel_encoder *encoder,
const struct drm_connector_state *conn_state)
{
struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
- struct i915_audio_component *acomp = dev_priv->audio_component;
+ struct i915_audio_component *acomp = dev_priv->audio.component;
struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
struct drm_connector *connector = conn_state->connector;
const struct drm_display_mode *adjusted_mode =
@@ -848,17 +858,17 @@ void intel_audio_codec_enable(struct intel_encoder *encoder,
connector->eld[6] = drm_av_sync_delay(connector, adjusted_mode) / 2;
- if (dev_priv->audio_funcs)
- dev_priv->audio_funcs->audio_codec_enable(encoder,
+ if (dev_priv->audio.funcs)
+ dev_priv->audio.funcs->audio_codec_enable(encoder,
crtc_state,
conn_state);
- mutex_lock(&dev_priv->av_mutex);
+ mutex_lock(&dev_priv->audio.mutex);
encoder->audio_connector = connector;
/* referred in audio callbacks */
- dev_priv->av_enc_map[pipe] = encoder;
- mutex_unlock(&dev_priv->av_mutex);
+ dev_priv->audio.encoder_map[pipe] = encoder;
+ mutex_unlock(&dev_priv->audio.mutex);
if (acomp && acomp->base.audio_ops &&
acomp->base.audio_ops->pin_eld_notify) {
@@ -888,20 +898,20 @@ void intel_audio_codec_disable(struct intel_encoder *encoder,
const struct drm_connector_state *old_conn_state)
{
struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
- struct i915_audio_component *acomp = dev_priv->audio_component;
+ struct i915_audio_component *acomp = dev_priv->audio.component;
struct intel_crtc *crtc = to_intel_crtc(old_crtc_state->uapi.crtc);
enum port port = encoder->port;
enum pipe pipe = crtc->pipe;
- if (dev_priv->audio_funcs)
- dev_priv->audio_funcs->audio_codec_disable(encoder,
+ if (dev_priv->audio.funcs)
+ dev_priv->audio.funcs->audio_codec_disable(encoder,
old_crtc_state,
old_conn_state);
- mutex_lock(&dev_priv->av_mutex);
+ mutex_lock(&dev_priv->audio.mutex);
encoder->audio_connector = NULL;
- dev_priv->av_enc_map[pipe] = NULL;
- mutex_unlock(&dev_priv->av_mutex);
+ dev_priv->audio.encoder_map[pipe] = NULL;
+ mutex_unlock(&dev_priv->audio.mutex);
if (acomp && acomp->base.audio_ops &&
acomp->base.audio_ops->pin_eld_notify) {
@@ -931,19 +941,53 @@ static const struct intel_audio_funcs hsw_audio_funcs = {
};
/**
- * intel_init_audio_hooks - Set up chip specific audio hooks
+ * intel_audio_hooks_init - Set up chip specific audio hooks
* @dev_priv: device private
*/
-void intel_init_audio_hooks(struct drm_i915_private *dev_priv)
+void intel_audio_hooks_init(struct drm_i915_private *dev_priv)
{
if (IS_G4X(dev_priv)) {
- dev_priv->audio_funcs = &g4x_audio_funcs;
+ dev_priv->audio.funcs = &g4x_audio_funcs;
} else if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv)) {
- dev_priv->audio_funcs = &ilk_audio_funcs;
+ dev_priv->audio.funcs = &ilk_audio_funcs;
} else if (IS_HASWELL(dev_priv) || DISPLAY_VER(dev_priv) >= 8) {
- dev_priv->audio_funcs = &hsw_audio_funcs;
+ dev_priv->audio.funcs = &hsw_audio_funcs;
} else if (HAS_PCH_SPLIT(dev_priv)) {
- dev_priv->audio_funcs = &ilk_audio_funcs;
+ dev_priv->audio.funcs = &ilk_audio_funcs;
+ }
+}
+
+struct aud_ts_cdclk_m_n {
+ u8 m;
+ u16 n;
+};
+
+void intel_audio_cdclk_change_pre(struct drm_i915_private *i915)
+{
+ if (DISPLAY_VER(i915) >= 13)
+ intel_de_rmw(i915, AUD_TS_CDCLK_M, AUD_TS_CDCLK_M_EN, 0);
+}
+
+static void get_aud_ts_cdclk_m_n(int refclk, int cdclk, struct aud_ts_cdclk_m_n *aud_ts)
+{
+ if (refclk == 24000)
+ aud_ts->m = 12;
+ else
+ aud_ts->m = 15;
+
+ aud_ts->n = cdclk * aud_ts->m / 24000;
+}
+
+void intel_audio_cdclk_change_post(struct drm_i915_private *i915)
+{
+ struct aud_ts_cdclk_m_n aud_ts;
+
+ if (DISPLAY_VER(i915) >= 13) {
+ get_aud_ts_cdclk_m_n(i915->cdclk.hw.ref, i915->cdclk.hw.cdclk, &aud_ts);
+
+ intel_de_write(i915, AUD_TS_CDCLK_N, aud_ts.n);
+ intel_de_write(i915, AUD_TS_CDCLK_M, aud_ts.m | AUD_TS_CDCLK_M_EN);
+ drm_dbg_kms(&i915->drm, "aud_ts_cdclk set to M=%u, N=%u\n", aud_ts.m, aud_ts.n);
}
}
@@ -976,7 +1020,7 @@ static void glk_force_audio_cdclk(struct drm_i915_private *dev_priv,
struct intel_crtc *crtc;
int ret;
- crtc = intel_get_first_crtc(dev_priv);
+ crtc = intel_first_crtc(dev_priv);
if (!crtc)
return;
@@ -1014,13 +1058,13 @@ static unsigned long i915_audio_component_get_power(struct device *kdev)
ret = intel_display_power_get(dev_priv, POWER_DOMAIN_AUDIO_PLAYBACK);
- if (dev_priv->audio_power_refcount++ == 0) {
+ if (dev_priv->audio.power_refcount++ == 0) {
if (DISPLAY_VER(dev_priv) >= 9) {
intel_de_write(dev_priv, AUD_FREQ_CNTRL,
- dev_priv->audio_freq_cntrl);
+ dev_priv->audio.freq_cntrl);
drm_dbg_kms(&dev_priv->drm,
"restored AUD_FREQ_CNTRL to 0x%x\n",
- dev_priv->audio_freq_cntrl);
+ dev_priv->audio.freq_cntrl);
}
/* Force CDCLK to 2*BCLK as long as we need audio powered. */
@@ -1041,7 +1085,7 @@ static void i915_audio_component_put_power(struct device *kdev,
struct drm_i915_private *dev_priv = kdev_to_i915(kdev);
/* Stop forcing CDCLK to 2*BCLK if no need for audio to be powered. */
- if (--dev_priv->audio_power_refcount == 0)
+ if (--dev_priv->audio.power_refcount == 0)
if (IS_GEMINILAKE(dev_priv))
glk_force_audio_cdclk(dev_priv, false);
@@ -1093,7 +1137,7 @@ static int i915_audio_component_get_cdclk_freq(struct device *kdev)
/*
* get the intel_encoder according to the parameter port and pipe
* intel_encoder is saved by the index of pipe
- * MST & (pipe >= 0): return the av_enc_map[pipe],
+ * MST & (pipe >= 0): return the audio.encoder_map[pipe],
* when port is matched
* MST & (pipe < 0): this is invalid
* Non-MST & (pipe >= 0): only pipe = 0 (the first device entry)
@@ -1108,10 +1152,10 @@ static struct intel_encoder *get_saved_enc(struct drm_i915_private *dev_priv,
/* MST */
if (pipe >= 0) {
if (drm_WARN_ON(&dev_priv->drm,
- pipe >= ARRAY_SIZE(dev_priv->av_enc_map)))
+ pipe >= ARRAY_SIZE(dev_priv->audio.encoder_map)))
return NULL;
- encoder = dev_priv->av_enc_map[pipe];
+ encoder = dev_priv->audio.encoder_map[pipe];
/*
* when bootup, audio driver may not know it is
* MST or not. So it will poll all the port & pipe
@@ -1127,7 +1171,7 @@ static struct intel_encoder *get_saved_enc(struct drm_i915_private *dev_priv,
return NULL;
for_each_pipe(dev_priv, pipe) {
- encoder = dev_priv->av_enc_map[pipe];
+ encoder = dev_priv->audio.encoder_map[pipe];
if (encoder == NULL)
continue;
@@ -1145,7 +1189,7 @@ static int i915_audio_component_sync_audio_rate(struct device *kdev, int port,
int pipe, int rate)
{
struct drm_i915_private *dev_priv = kdev_to_i915(kdev);
- struct i915_audio_component *acomp = dev_priv->audio_component;
+ struct i915_audio_component *acomp = dev_priv->audio.component;
struct intel_encoder *encoder;
struct intel_crtc *crtc;
unsigned long cookie;
@@ -1155,7 +1199,7 @@ static int i915_audio_component_sync_audio_rate(struct device *kdev, int port,
return 0;
cookie = i915_audio_component_get_power(kdev);
- mutex_lock(&dev_priv->av_mutex);
+ mutex_lock(&dev_priv->audio.mutex);
/* 1. get the pipe */
encoder = get_saved_enc(dev_priv, port, pipe);
@@ -1174,7 +1218,7 @@ static int i915_audio_component_sync_audio_rate(struct device *kdev, int port,
hsw_audio_config_update(encoder, crtc->config);
unlock:
- mutex_unlock(&dev_priv->av_mutex);
+ mutex_unlock(&dev_priv->audio.mutex);
i915_audio_component_put_power(kdev, cookie);
return err;
}
@@ -1188,13 +1232,13 @@ static int i915_audio_component_get_eld(struct device *kdev, int port,
const u8 *eld;
int ret = -EINVAL;
- mutex_lock(&dev_priv->av_mutex);
+ mutex_lock(&dev_priv->audio.mutex);
intel_encoder = get_saved_enc(dev_priv, port, pipe);
if (!intel_encoder) {
drm_dbg_kms(&dev_priv->drm, "Not valid for port %c\n",
port_name(port));
- mutex_unlock(&dev_priv->av_mutex);
+ mutex_unlock(&dev_priv->audio.mutex);
return ret;
}
@@ -1206,7 +1250,7 @@ static int i915_audio_component_get_eld(struct device *kdev, int port,
memcpy(buf, eld, min(max_bytes, ret));
}
- mutex_unlock(&dev_priv->av_mutex);
+ mutex_unlock(&dev_priv->audio.mutex);
return ret;
}
@@ -1241,7 +1285,7 @@ static int i915_audio_component_bind(struct device *i915_kdev,
BUILD_BUG_ON(MAX_PORTS != I915_MAX_PORTS);
for (i = 0; i < ARRAY_SIZE(acomp->aud_sample_rate); i++)
acomp->aud_sample_rate[i] = 0;
- dev_priv->audio_component = acomp;
+ dev_priv->audio.component = acomp;
drm_modeset_unlock_all(&dev_priv->drm);
return 0;
@@ -1256,14 +1300,14 @@ static void i915_audio_component_unbind(struct device *i915_kdev,
drm_modeset_lock_all(&dev_priv->drm);
acomp->base.ops = NULL;
acomp->base.dev = NULL;
- dev_priv->audio_component = NULL;
+ dev_priv->audio.component = NULL;
drm_modeset_unlock_all(&dev_priv->drm);
device_link_remove(hda_kdev, i915_kdev);
- if (dev_priv->audio_power_refcount)
+ if (dev_priv->audio.power_refcount)
drm_err(&dev_priv->drm, "audio power refcount %d after unbind\n",
- dev_priv->audio_power_refcount);
+ dev_priv->audio.power_refcount);
}
static const struct component_ops i915_audio_component_bind_ops = {
@@ -1327,10 +1371,13 @@ static void i915_audio_component_init(struct drm_i915_private *dev_priv)
drm_dbg_kms(&dev_priv->drm, "use AUD_FREQ_CNTRL of 0x%x (init value 0x%x)\n",
aud_freq, aud_freq_init);
- dev_priv->audio_freq_cntrl = aud_freq;
+ dev_priv->audio.freq_cntrl = aud_freq;
}
- dev_priv->audio_component_registered = true;
+ /* init with current cdclk */
+ intel_audio_cdclk_change_post(dev_priv);
+
+ dev_priv->audio.component_registered = true;
}
/**
@@ -1342,11 +1389,11 @@ static void i915_audio_component_init(struct drm_i915_private *dev_priv)
*/
static void i915_audio_component_cleanup(struct drm_i915_private *dev_priv)
{
- if (!dev_priv->audio_component_registered)
+ if (!dev_priv->audio.component_registered)
return;
component_del(dev_priv->drm.dev, &i915_audio_component_bind_ops);
- dev_priv->audio_component_registered = false;
+ dev_priv->audio.component_registered = false;
}
/**
@@ -1368,7 +1415,7 @@ void intel_audio_init(struct drm_i915_private *dev_priv)
*/
void intel_audio_deinit(struct drm_i915_private *dev_priv)
{
- if ((dev_priv)->lpe_audio.platdev != NULL)
+ if ((dev_priv)->audio.lpe.platdev != NULL)
intel_lpe_audio_teardown(dev_priv);
else
i915_audio_component_cleanup(dev_priv);
diff --git a/drivers/gpu/drm/i915/display/intel_audio.h b/drivers/gpu/drm/i915/display/intel_audio.h
index a3657c7..63b2213 100644
--- a/drivers/gpu/drm/i915/display/intel_audio.h
+++ b/drivers/gpu/drm/i915/display/intel_audio.h
@@ -11,13 +11,15 @@ struct drm_i915_private;
struct intel_crtc_state;
struct intel_encoder;
-void intel_init_audio_hooks(struct drm_i915_private *dev_priv);
+void intel_audio_hooks_init(struct drm_i915_private *dev_priv);
void intel_audio_codec_enable(struct intel_encoder *encoder,
const struct intel_crtc_state *crtc_state,
const struct drm_connector_state *conn_state);
void intel_audio_codec_disable(struct intel_encoder *encoder,
const struct intel_crtc_state *old_crtc_state,
const struct drm_connector_state *old_conn_state);
+void intel_audio_cdclk_change_pre(struct drm_i915_private *dev_priv);
+void intel_audio_cdclk_change_post(struct drm_i915_private *dev_priv);
void intel_audio_init(struct drm_i915_private *dev_priv);
void intel_audio_deinit(struct drm_i915_private *dev_priv);
diff --git a/drivers/gpu/drm/i915/display/intel_bios.c b/drivers/gpu/drm/i915/display/intel_bios.c
index 2b1423a..9d989c9f 100644
--- a/drivers/gpu/drm/i915/display/intel_bios.c
+++ b/drivers/gpu/drm/i915/display/intel_bios.c
@@ -1555,12 +1555,24 @@ static const u8 gen9bc_tgp_ddc_pin_map[] = {
[DDC_BUS_DDI_D] = GMBUS_PIN_10_TC2_ICP,
};
+static const u8 adlp_ddc_pin_map[] = {
+ [ICL_DDC_BUS_DDI_A] = GMBUS_PIN_1_BXT,
+ [ICL_DDC_BUS_DDI_B] = GMBUS_PIN_2_BXT,
+ [ADLP_DDC_BUS_PORT_TC1] = GMBUS_PIN_9_TC1_ICP,
+ [ADLP_DDC_BUS_PORT_TC2] = GMBUS_PIN_10_TC2_ICP,
+ [ADLP_DDC_BUS_PORT_TC3] = GMBUS_PIN_11_TC3_ICP,
+ [ADLP_DDC_BUS_PORT_TC4] = GMBUS_PIN_12_TC4_ICP,
+};
+
static u8 map_ddc_pin(struct drm_i915_private *i915, u8 vbt_pin)
{
const u8 *ddc_pin_map;
int n_entries;
- if (IS_ALDERLAKE_S(i915)) {
+ if (IS_ALDERLAKE_P(i915)) {
+ ddc_pin_map = adlp_ddc_pin_map;
+ n_entries = ARRAY_SIZE(adlp_ddc_pin_map);
+ } else if (IS_ALDERLAKE_S(i915)) {
ddc_pin_map = adls_ddc_pin_map;
n_entries = ARRAY_SIZE(adls_ddc_pin_map);
} else if (INTEL_PCH_TYPE(i915) >= PCH_DG1) {
diff --git a/drivers/gpu/drm/i915/display/intel_bw.c b/drivers/gpu/drm/i915/display/intel_bw.c
index 8d9d888..2da4aac 100644
--- a/drivers/gpu/drm/i915/display/intel_bw.c
+++ b/drivers/gpu/drm/i915/display/intel_bw.c
@@ -27,6 +27,9 @@ struct intel_qgv_info {
u8 num_points;
u8 num_psf_points;
u8 t_bl;
+ u8 max_numchannels;
+ u8 channel_width;
+ u8 deinterleave;
};
static int dg1_mchbar_read_qgv_point_info(struct drm_i915_private *dev_priv,
@@ -42,7 +45,7 @@ static int dg1_mchbar_read_qgv_point_info(struct drm_i915_private *dev_priv,
dclk_reference = 6; /* 6 * 16.666 MHz = 100 MHz */
else
dclk_reference = 8; /* 8 * 16.666 MHz = 133 MHz */
- sp->dclk = dclk_ratio * dclk_reference;
+ sp->dclk = DIV_ROUND_UP((16667 * dclk_ratio * dclk_reference) + 500, 1000);
val = intel_uncore_read(&dev_priv->uncore, SKL_MC_BIOS_DATA_0_0_0_MCHBAR_PCU);
if (val & DG1_GEAR_TYPE)
@@ -69,6 +72,7 @@ static int icl_pcode_read_qgv_point_info(struct drm_i915_private *dev_priv,
int point)
{
u32 val = 0, val2 = 0;
+ u16 dclk;
int ret;
ret = sandybridge_pcode_read(dev_priv,
@@ -78,7 +82,8 @@ static int icl_pcode_read_qgv_point_info(struct drm_i915_private *dev_priv,
if (ret)
return ret;
- sp->dclk = val & 0xffff;
+ dclk = val & 0xffff;
+ sp->dclk = DIV_ROUND_UP((16667 * dclk) + (DISPLAY_VER(dev_priv) > 11 ? 500 : 0), 1000);
sp->t_rp = (val & 0xff0000) >> 16;
sp->t_rcd = (val & 0xff000000) >> 24;
@@ -133,7 +138,8 @@ int icl_pcode_restrict_qgv_points(struct drm_i915_private *dev_priv,
}
static int icl_get_qgv_points(struct drm_i915_private *dev_priv,
- struct intel_qgv_info *qi)
+ struct intel_qgv_info *qi,
+ bool is_y_tile)
{
const struct dram_info *dram_info = &dev_priv->dram_info;
int i, ret;
@@ -141,20 +147,44 @@ static int icl_get_qgv_points(struct drm_i915_private *dev_priv,
qi->num_points = dram_info->num_qgv_points;
qi->num_psf_points = dram_info->num_psf_gv_points;
- if (DISPLAY_VER(dev_priv) == 12)
+ if (DISPLAY_VER(dev_priv) >= 12)
switch (dram_info->type) {
case INTEL_DRAM_DDR4:
- qi->t_bl = 4;
+ qi->t_bl = is_y_tile ? 8 : 4;
+ qi->max_numchannels = 2;
+ qi->channel_width = 64;
+ qi->deinterleave = is_y_tile ? 1 : 2;
break;
case INTEL_DRAM_DDR5:
- qi->t_bl = 8;
+ qi->t_bl = is_y_tile ? 16 : 8;
+ qi->max_numchannels = 4;
+ qi->channel_width = 32;
+ qi->deinterleave = is_y_tile ? 1 : 2;
+ break;
+ case INTEL_DRAM_LPDDR4:
+ if (IS_ROCKETLAKE(dev_priv)) {
+ qi->t_bl = 8;
+ qi->max_numchannels = 4;
+ qi->channel_width = 32;
+ qi->deinterleave = 2;
+ break;
+ }
+ fallthrough;
+ case INTEL_DRAM_LPDDR5:
+ qi->t_bl = 16;
+ qi->max_numchannels = 8;
+ qi->channel_width = 16;
+ qi->deinterleave = is_y_tile ? 2 : 4;
break;
default:
qi->t_bl = 16;
+ qi->max_numchannels = 1;
break;
}
- else if (DISPLAY_VER(dev_priv) == 11)
+ else if (DISPLAY_VER(dev_priv) == 11) {
qi->t_bl = dev_priv->dram_info.type == INTEL_DRAM_DDR4 ? 4 : 8;
+ qi->max_numchannels = 1;
+ }
if (drm_WARN_ON(&dev_priv->drm,
qi->num_points > ARRAY_SIZE(qi->points)))
@@ -193,12 +223,6 @@ static int icl_get_qgv_points(struct drm_i915_private *dev_priv,
return 0;
}
-static int icl_calc_bw(int dclk, int num, int den)
-{
- /* multiples of 16.666MHz (100/6) */
- return DIV_ROUND_CLOSEST(num * dclk * 100, den * 6);
-}
-
static int adl_calc_psf_bw(int clk)
{
/*
@@ -240,7 +264,7 @@ static const struct intel_sa_info tgl_sa_info = {
};
static const struct intel_sa_info rkl_sa_info = {
- .deburst = 16,
+ .deburst = 8,
.deprogbwlimit = 20, /* GB/s */
.displayrtids = 128,
.derating = 10,
@@ -265,34 +289,30 @@ static int icl_get_bw_info(struct drm_i915_private *dev_priv, const struct intel
struct intel_qgv_info qi = {};
bool is_y_tile = true; /* assume y tile may be used */
int num_channels = max_t(u8, 1, dev_priv->dram_info.num_channels);
- int deinterleave;
- int ipqdepth, ipqdepthpch;
+ int ipqdepth, ipqdepthpch = 16;
int dclk_max;
int maxdebw;
+ int num_groups = ARRAY_SIZE(dev_priv->max_bw);
int i, ret;
- ret = icl_get_qgv_points(dev_priv, &qi);
+ ret = icl_get_qgv_points(dev_priv, &qi, is_y_tile);
if (ret) {
drm_dbg_kms(&dev_priv->drm,
"Failed to get memory subsystem information, ignoring bandwidth limits");
return ret;
}
- deinterleave = DIV_ROUND_UP(num_channels, is_y_tile ? 4 : 2);
dclk_max = icl_sagv_max_dclk(&qi);
-
- ipqdepthpch = 16;
-
- maxdebw = min(sa->deprogbwlimit * 1000,
- icl_calc_bw(dclk_max, 16, 1) * 6 / 10); /* 60% */
+ maxdebw = min(sa->deprogbwlimit * 1000, dclk_max * 16 * 6 / 10);
ipqdepth = min(ipqdepthpch, sa->displayrtids / num_channels);
+ qi.deinterleave = DIV_ROUND_UP(num_channels, is_y_tile ? 4 : 2);
- for (i = 0; i < ARRAY_SIZE(dev_priv->max_bw); i++) {
+ for (i = 0; i < num_groups; i++) {
struct intel_bw_info *bi = &dev_priv->max_bw[i];
int clpchgroup;
int j;
- clpchgroup = (sa->deburst * deinterleave / num_channels) << i;
+ clpchgroup = (sa->deburst * qi.deinterleave / num_channels) << i;
bi->num_planes = (ipqdepth - clpchgroup) / clpchgroup + 1;
bi->num_qgv_points = qi.num_points;
@@ -310,7 +330,106 @@ static int icl_get_bw_info(struct drm_i915_private *dev_priv, const struct intel
*/
ct = max_t(int, sp->t_rc, sp->t_rp + sp->t_rcd +
(clpchgroup - 1) * qi.t_bl + sp->t_rdpre);
- bw = icl_calc_bw(sp->dclk, clpchgroup * 32 * num_channels, ct);
+ bw = DIV_ROUND_UP(sp->dclk * clpchgroup * 32 * num_channels, ct);
+
+ bi->deratedbw[j] = min(maxdebw,
+ bw * (100 - sa->derating) / 100);
+
+ drm_dbg_kms(&dev_priv->drm,
+ "BW%d / QGV %d: num_planes=%d deratedbw=%u\n",
+ i, j, bi->num_planes, bi->deratedbw[j]);
+ }
+ }
+ /*
+ * In case if SAGV is disabled in BIOS, we always get 1
+ * SAGV point, but we can't send PCode commands to restrict it
+ * as it will fail and pointless anyway.
+ */
+ if (qi.num_points == 1)
+ dev_priv->sagv_status = I915_SAGV_NOT_CONTROLLED;
+ else
+ dev_priv->sagv_status = I915_SAGV_ENABLED;
+
+ return 0;
+}
+
+static int tgl_get_bw_info(struct drm_i915_private *dev_priv, const struct intel_sa_info *sa)
+{
+ struct intel_qgv_info qi = {};
+ const struct dram_info *dram_info = &dev_priv->dram_info;
+ bool is_y_tile = true; /* assume y tile may be used */
+ int num_channels = max_t(u8, 1, dev_priv->dram_info.num_channels);
+ int ipqdepth, ipqdepthpch = 16;
+ int dclk_max;
+ int maxdebw, peakbw;
+ int clperchgroup;
+ int num_groups = ARRAY_SIZE(dev_priv->max_bw);
+ int i, ret;
+
+ ret = icl_get_qgv_points(dev_priv, &qi, is_y_tile);
+ if (ret) {
+ drm_dbg_kms(&dev_priv->drm,
+ "Failed to get memory subsystem information, ignoring bandwidth limits");
+ return ret;
+ }
+
+ if (dram_info->type == INTEL_DRAM_LPDDR4 || dram_info->type == INTEL_DRAM_LPDDR5)
+ num_channels *= 2;
+
+ qi.deinterleave = qi.deinterleave ? : DIV_ROUND_UP(num_channels, is_y_tile ? 4 : 2);
+
+ if (num_channels < qi.max_numchannels && DISPLAY_VER(dev_priv) >= 12)
+ qi.deinterleave = max(DIV_ROUND_UP(qi.deinterleave, 2), 1);
+
+ if (DISPLAY_VER(dev_priv) > 11 && num_channels > qi.max_numchannels)
+ drm_warn(&dev_priv->drm, "Number of channels exceeds max number of channels.");
+ if (qi.max_numchannels != 0)
+ num_channels = min_t(u8, num_channels, qi.max_numchannels);
+
+ dclk_max = icl_sagv_max_dclk(&qi);
+
+ peakbw = num_channels * DIV_ROUND_UP(qi.channel_width, 8) * dclk_max;
+ maxdebw = min(sa->deprogbwlimit * 1000, peakbw * 6 / 10); /* 60% */
+
+ ipqdepth = min(ipqdepthpch, sa->displayrtids / num_channels);
+ /*
+ * clperchgroup = 4kpagespermempage * clperchperblock,
+ * clperchperblock = 8 / num_channels * interleave
+ */
+ clperchgroup = 4 * DIV_ROUND_UP(8, num_channels) * qi.deinterleave;
+
+ for (i = 0; i < num_groups; i++) {
+ struct intel_bw_info *bi = &dev_priv->max_bw[i];
+ struct intel_bw_info *bi_next;
+ int clpchgroup;
+ int j;
+
+ if (i < num_groups - 1)
+ bi_next = &dev_priv->max_bw[i + 1];
+
+ clpchgroup = (sa->deburst * qi.deinterleave / num_channels) << i;
+
+ if (i < num_groups - 1 && clpchgroup < clperchgroup)
+ bi_next->num_planes = (ipqdepth - clpchgroup) / clpchgroup + 1;
+ else
+ bi_next->num_planes = 0;
+
+ bi->num_qgv_points = qi.num_points;
+ bi->num_psf_gv_points = qi.num_psf_points;
+
+ for (j = 0; j < qi.num_points; j++) {
+ const struct intel_qgv_point *sp = &qi.points[j];
+ int ct, bw;
+
+ /*
+ * Max row cycle time
+ *
+ * FIXME what is the logic behind the
+ * assumed burst length?
+ */
+ ct = max_t(int, sp->t_rc, sp->t_rp + sp->t_rcd +
+ (clpchgroup - 1) * qi.t_bl + sp->t_rdpre);
+ bw = DIV_ROUND_UP(sp->dclk * clpchgroup * 32 * num_channels, ct);
bi->deratedbw[j] = min(maxdebw,
bw * (100 - sa->derating) / 100);
@@ -329,9 +448,6 @@ static int icl_get_bw_info(struct drm_i915_private *dev_priv, const struct intel
"BW%d / PSF GV %d: num_planes=%d bw=%u\n",
i, j, bi->num_planes, bi->psf_bw[j]);
}
-
- if (bi->num_planes == 1)
- break;
}
/*
@@ -395,6 +511,34 @@ static unsigned int icl_max_bw(struct drm_i915_private *dev_priv,
return 0;
}
+static unsigned int tgl_max_bw(struct drm_i915_private *dev_priv,
+ int num_planes, int qgv_point)
+{
+ int i;
+
+ /*
+ * Let's return max bw for 0 planes
+ */
+ num_planes = max(1, num_planes);
+
+ for (i = ARRAY_SIZE(dev_priv->max_bw) - 1; i >= 0; i--) {
+ const struct intel_bw_info *bi =
+ &dev_priv->max_bw[i];
+
+ /*
+ * Pcode will not expose all QGV points when
+ * SAGV is forced to off/min/med/max.
+ */
+ if (qgv_point >= bi->num_qgv_points)
+ return UINT_MAX;
+
+ if (num_planes <= bi->num_planes)
+ return bi->deratedbw[qgv_point];
+ }
+
+ return dev_priv->max_bw[0].deratedbw[qgv_point];
+}
+
static unsigned int adl_psf_bw(struct drm_i915_private *dev_priv,
int psf_gv_point)
{
@@ -412,13 +556,13 @@ void intel_bw_init_hw(struct drm_i915_private *dev_priv)
if (IS_DG2(dev_priv))
dg2_get_bw_info(dev_priv);
else if (IS_ALDERLAKE_P(dev_priv))
- icl_get_bw_info(dev_priv, &adlp_sa_info);
+ tgl_get_bw_info(dev_priv, &adlp_sa_info);
else if (IS_ALDERLAKE_S(dev_priv))
- icl_get_bw_info(dev_priv, &adls_sa_info);
+ tgl_get_bw_info(dev_priv, &adls_sa_info);
else if (IS_ROCKETLAKE(dev_priv))
- icl_get_bw_info(dev_priv, &rkl_sa_info);
+ tgl_get_bw_info(dev_priv, &rkl_sa_info);
else if (DISPLAY_VER(dev_priv) == 12)
- icl_get_bw_info(dev_priv, &tgl_sa_info);
+ tgl_get_bw_info(dev_priv, &tgl_sa_info);
else if (DISPLAY_VER(dev_priv) == 11)
icl_get_bw_info(dev_priv, &icl_sa_info);
}
@@ -490,7 +634,7 @@ static unsigned int intel_bw_data_rate(struct drm_i915_private *dev_priv,
for_each_pipe(dev_priv, pipe)
data_rate += bw_state->data_rate[pipe];
- if (DISPLAY_VER(dev_priv) >= 13 && intel_vtd_active())
+ if (DISPLAY_VER(dev_priv) >= 13 && intel_vtd_active(dev_priv))
data_rate = data_rate * 105 / 100;
return data_rate;
@@ -746,7 +890,10 @@ int intel_bw_atomic_check(struct intel_atomic_state *state)
for (i = 0; i < num_qgv_points; i++) {
unsigned int max_data_rate;
- max_data_rate = icl_max_bw(dev_priv, num_active_planes, i);
+ if (DISPLAY_VER(dev_priv) > 11)
+ max_data_rate = tgl_max_bw(dev_priv, num_active_planes, i);
+ else
+ max_data_rate = icl_max_bw(dev_priv, num_active_planes, i);
/*
* We need to know which qgv point gives us
* maximum bandwidth in order to disable SAGV
diff --git a/drivers/gpu/drm/i915/display/intel_cdclk.c b/drivers/gpu/drm/i915/display/intel_cdclk.c
index 868dd43..c30cf8d 100644
--- a/drivers/gpu/drm/i915/display/intel_cdclk.c
+++ b/drivers/gpu/drm/i915/display/intel_cdclk.c
@@ -24,8 +24,11 @@
#include <linux/time.h>
#include "intel_atomic.h"
+#include "intel_atomic_plane.h"
+#include "intel_audio.h"
#include "intel_bw.h"
#include "intel_cdclk.h"
+#include "intel_crtc.h"
#include "intel_de.h"
#include "intel_display_types.h"
#include "intel_pcode.h"
@@ -66,7 +69,7 @@ void intel_cdclk_get_cdclk(struct drm_i915_private *dev_priv,
dev_priv->cdclk_funcs->get_cdclk(dev_priv, cdclk_config);
}
-int intel_cdclk_bw_calc_min_cdclk(struct intel_atomic_state *state)
+static int intel_cdclk_bw_calc_min_cdclk(struct intel_atomic_state *state)
{
struct drm_i915_private *dev_priv = to_i915(state->base.dev);
return dev_priv->cdclk_funcs->bw_calc_min_cdclk(state);
@@ -1211,6 +1214,19 @@ static void skl_cdclk_uninit_hw(struct drm_i915_private *dev_priv)
skl_set_cdclk(dev_priv, &cdclk_config, INVALID_PIPE);
}
+static bool has_cdclk_squasher(struct drm_i915_private *i915)
+{
+ return IS_DG2(i915);
+}
+
+struct intel_cdclk_vals {
+ u32 cdclk;
+ u16 refclk;
+ u16 waveform;
+ u8 divider; /* CD2X divider * 2 */
+ u8 ratio;
+};
+
static const struct intel_cdclk_vals bxt_cdclk_table[] = {
{ .refclk = 19200, .cdclk = 144000, .divider = 8, .ratio = 60 },
{ .refclk = 19200, .cdclk = 288000, .divider = 4, .ratio = 60 },
@@ -1312,12 +1328,19 @@ static const struct intel_cdclk_vals adlp_cdclk_table[] = {
};
static const struct intel_cdclk_vals dg2_cdclk_table[] = {
- { .refclk = 38400, .cdclk = 172800, .divider = 2, .ratio = 9 },
- { .refclk = 38400, .cdclk = 192000, .divider = 2, .ratio = 10 },
- { .refclk = 38400, .cdclk = 307200, .divider = 2, .ratio = 16 },
- { .refclk = 38400, .cdclk = 326400, .divider = 4, .ratio = 34 },
- { .refclk = 38400, .cdclk = 556800, .divider = 2, .ratio = 29 },
- { .refclk = 38400, .cdclk = 652800, .divider = 2, .ratio = 34 },
+ { .refclk = 38400, .cdclk = 163200, .divider = 2, .ratio = 34, .waveform = 0x8888 },
+ { .refclk = 38400, .cdclk = 204000, .divider = 2, .ratio = 34, .waveform = 0x9248 },
+ { .refclk = 38400, .cdclk = 244800, .divider = 2, .ratio = 34, .waveform = 0xa4a4 },
+ { .refclk = 38400, .cdclk = 285600, .divider = 2, .ratio = 34, .waveform = 0xa54a },
+ { .refclk = 38400, .cdclk = 326400, .divider = 2, .ratio = 34, .waveform = 0xaaaa },
+ { .refclk = 38400, .cdclk = 367200, .divider = 2, .ratio = 34, .waveform = 0xad5a },
+ { .refclk = 38400, .cdclk = 408000, .divider = 2, .ratio = 34, .waveform = 0xb6b6 },
+ { .refclk = 38400, .cdclk = 448800, .divider = 2, .ratio = 34, .waveform = 0xdbb6 },
+ { .refclk = 38400, .cdclk = 489600, .divider = 2, .ratio = 34, .waveform = 0xeeee },
+ { .refclk = 38400, .cdclk = 530400, .divider = 2, .ratio = 34, .waveform = 0xf7de },
+ { .refclk = 38400, .cdclk = 571200, .divider = 2, .ratio = 34, .waveform = 0xfefe },
+ { .refclk = 38400, .cdclk = 612000, .divider = 2, .ratio = 34, .waveform = 0xfffe },
+ { .refclk = 38400, .cdclk = 652800, .divider = 2, .ratio = 34, .waveform = 0xffff },
{}
};
@@ -1453,6 +1476,7 @@ static void bxt_de_pll_readout(struct drm_i915_private *dev_priv,
static void bxt_get_cdclk(struct drm_i915_private *dev_priv,
struct intel_cdclk_config *cdclk_config)
{
+ u32 squash_ctl = 0;
u32 divider;
int div;
@@ -1490,7 +1514,21 @@ static void bxt_get_cdclk(struct drm_i915_private *dev_priv,
return;
}
- cdclk_config->cdclk = DIV_ROUND_CLOSEST(cdclk_config->vco, div);
+ if (has_cdclk_squasher(dev_priv))
+ squash_ctl = intel_de_read(dev_priv, CDCLK_SQUASH_CTL);
+
+ if (squash_ctl & CDCLK_SQUASH_ENABLE) {
+ u16 waveform;
+ int size;
+
+ size = REG_FIELD_GET(CDCLK_SQUASH_WINDOW_SIZE_MASK, squash_ctl) + 1;
+ waveform = REG_FIELD_GET(CDCLK_SQUASH_WAVEFORM_MASK, squash_ctl) >> (16 - size);
+
+ cdclk_config->cdclk = DIV_ROUND_CLOSEST(hweight16(waveform) *
+ cdclk_config->vco, size * div);
+ } else {
+ cdclk_config->cdclk = DIV_ROUND_CLOSEST(cdclk_config->vco, div);
+ }
out:
/*
@@ -1625,6 +1663,26 @@ static u32 bxt_cdclk_cd2x_div_sel(struct drm_i915_private *dev_priv,
}
}
+static u32 cdclk_squash_waveform(struct drm_i915_private *dev_priv,
+ int cdclk)
+{
+ const struct intel_cdclk_vals *table = dev_priv->cdclk.table;
+ int i;
+
+ if (cdclk == dev_priv->cdclk.hw.bypass)
+ return 0;
+
+ for (i = 0; table[i].refclk; i++)
+ if (table[i].refclk == dev_priv->cdclk.hw.ref &&
+ table[i].cdclk == cdclk)
+ return table[i].waveform;
+
+ drm_WARN(&dev_priv->drm, 1, "cdclk %d not valid for refclk %u\n",
+ cdclk, dev_priv->cdclk.hw.ref);
+
+ return 0xffff;
+}
+
static void bxt_set_cdclk(struct drm_i915_private *dev_priv,
const struct intel_cdclk_config *cdclk_config,
enum pipe pipe)
@@ -1632,6 +1690,8 @@ static void bxt_set_cdclk(struct drm_i915_private *dev_priv,
int cdclk = cdclk_config->cdclk;
int vco = cdclk_config->vco;
u32 val;
+ u16 waveform;
+ int clock;
int ret;
/* Inform power controller of upcoming frequency change. */
@@ -1675,7 +1735,24 @@ static void bxt_set_cdclk(struct drm_i915_private *dev_priv,
bxt_de_pll_enable(dev_priv, vco);
}
- val = bxt_cdclk_cd2x_div_sel(dev_priv, cdclk, vco) |
+ waveform = cdclk_squash_waveform(dev_priv, cdclk);
+
+ if (waveform)
+ clock = vco / 2;
+ else
+ clock = cdclk;
+
+ if (has_cdclk_squasher(dev_priv)) {
+ u32 squash_ctl = 0;
+
+ if (waveform)
+ squash_ctl = CDCLK_SQUASH_ENABLE |
+ CDCLK_SQUASH_WINDOW_SIZE(0xf) | waveform;
+
+ intel_de_write(dev_priv, CDCLK_SQUASH_CTL, squash_ctl);
+ }
+
+ val = bxt_cdclk_cd2x_div_sel(dev_priv, clock, vco) |
bxt_cdclk_cd2x_pipe(dev_priv, pipe) |
skl_cdclk_decimal(cdclk);
@@ -1689,7 +1766,7 @@ static void bxt_set_cdclk(struct drm_i915_private *dev_priv,
intel_de_write(dev_priv, CDCLK_CTL, val);
if (pipe != INVALID_PIPE)
- intel_wait_for_vblank(dev_priv, pipe);
+ intel_crtc_wait_for_next_vblank(intel_crtc_for_pipe(dev_priv, pipe));
if (DISPLAY_VER(dev_priv) >= 11) {
ret = sandybridge_pcode_write(dev_priv, SKL_PCODE_CDCLK_CONTROL,
@@ -1727,7 +1804,7 @@ static void bxt_set_cdclk(struct drm_i915_private *dev_priv,
static void bxt_sanitize_cdclk(struct drm_i915_private *dev_priv)
{
u32 cdctl, expected;
- int cdclk, vco;
+ int cdclk, clock, vco;
intel_update_cdclk(dev_priv);
intel_dump_cdclk_config(&dev_priv->cdclk.hw, "Current CDCLK");
@@ -1763,8 +1840,12 @@ static void bxt_sanitize_cdclk(struct drm_i915_private *dev_priv)
expected = skl_cdclk_decimal(cdclk);
/* Figure out what CD2X divider we should be using for this cdclk */
- expected |= bxt_cdclk_cd2x_div_sel(dev_priv,
- dev_priv->cdclk.hw.cdclk,
+ if (has_cdclk_squasher(dev_priv))
+ clock = dev_priv->cdclk.hw.vco / 2;
+ else
+ clock = dev_priv->cdclk.hw.cdclk;
+
+ expected |= bxt_cdclk_cd2x_div_sel(dev_priv, clock,
dev_priv->cdclk.hw.vco);
/*
@@ -1880,6 +1961,25 @@ static bool intel_cdclk_can_crawl(struct drm_i915_private *dev_priv,
a->ref == b->ref;
}
+static bool intel_cdclk_can_squash(struct drm_i915_private *dev_priv,
+ const struct intel_cdclk_config *a,
+ const struct intel_cdclk_config *b)
+{
+ /*
+ * FIXME should store a bit more state in intel_cdclk_config
+ * to differentiate squasher vs. cd2x divider properly. For
+ * the moment all platforms with squasher use a fixed cd2x
+ * divider.
+ */
+ if (!has_cdclk_squasher(dev_priv))
+ return false;
+
+ return a->cdclk != b->cdclk &&
+ a->vco != 0 &&
+ a->vco == b->vco &&
+ a->ref == b->ref;
+}
+
/**
* intel_cdclk_needs_modeset - Determine if changong between the CDCLK
* configurations requires a modeset on all pipes
@@ -1917,7 +2017,17 @@ static bool intel_cdclk_can_cd2x_update(struct drm_i915_private *dev_priv,
if (DISPLAY_VER(dev_priv) < 10 && !IS_BROXTON(dev_priv))
return false;
+ /*
+ * FIXME should store a bit more state in intel_cdclk_config
+ * to differentiate squasher vs. cd2x divider properly. For
+ * the moment all platforms with squasher use a fixed cd2x
+ * divider.
+ */
+ if (has_cdclk_squasher(dev_priv))
+ return false;
+
return a->cdclk != b->cdclk &&
+ a->vco != 0 &&
a->vco == b->vco &&
a->ref == b->ref;
}
@@ -1975,6 +2085,8 @@ static void intel_set_cdclk(struct drm_i915_private *dev_priv,
intel_psr_pause(intel_dp);
}
+ intel_audio_cdclk_change_pre(dev_priv);
+
/*
* Lock aux/gmbus while we change cdclk in case those
* functions use cdclk. Not all platforms/ports do,
@@ -2003,6 +2115,8 @@ static void intel_set_cdclk(struct drm_i915_private *dev_priv,
intel_psr_resume(intel_dp);
}
+ intel_audio_cdclk_change_post(dev_priv);
+
if (drm_WARN(&dev_priv->drm,
intel_cdclk_changed(&dev_priv->cdclk.hw, cdclk_config),
"cdclk state doesn't match!\n")) {
@@ -2524,6 +2638,58 @@ intel_atomic_get_cdclk_state(struct intel_atomic_state *state)
return to_intel_cdclk_state(cdclk_state);
}
+int intel_cdclk_atomic_check(struct intel_atomic_state *state,
+ bool *need_cdclk_calc)
+{
+ struct drm_i915_private *i915 = to_i915(state->base.dev);
+ const struct intel_cdclk_state *old_cdclk_state;
+ const struct intel_cdclk_state *new_cdclk_state;
+ struct intel_plane_state *plane_state;
+ struct intel_bw_state *new_bw_state;
+ struct intel_plane *plane;
+ int min_cdclk = 0;
+ enum pipe pipe;
+ int ret;
+ int i;
+
+ /*
+ * active_planes bitmask has been updated, and potentially affected
+ * planes are part of the state. We can now compute the minimum cdclk
+ * for each plane.
+ */
+ for_each_new_intel_plane_in_state(state, plane, plane_state, i) {
+ ret = intel_plane_calc_min_cdclk(state, plane, need_cdclk_calc);
+ if (ret)
+ return ret;
+ }
+
+ old_cdclk_state = intel_atomic_get_old_cdclk_state(state);
+ new_cdclk_state = intel_atomic_get_new_cdclk_state(state);
+
+ if (new_cdclk_state &&
+ old_cdclk_state->force_min_cdclk != new_cdclk_state->force_min_cdclk)
+ *need_cdclk_calc = true;
+
+ ret = intel_cdclk_bw_calc_min_cdclk(state);
+ if (ret)
+ return ret;
+
+ new_bw_state = intel_atomic_get_new_bw_state(state);
+
+ if (!new_cdclk_state || !new_bw_state)
+ return 0;
+
+ for_each_pipe(i915, pipe) {
+ min_cdclk = max(new_cdclk_state->min_cdclk[pipe], min_cdclk);
+
+ /* Currently do this change only if we need to increase */
+ if (new_bw_state->min_cdclk > min_cdclk)
+ *need_cdclk_calc = true;
+ }
+
+ return 0;
+}
+
int intel_cdclk_init(struct drm_i915_private *dev_priv)
{
struct intel_cdclk_state *cdclk_state;
@@ -2587,7 +2753,7 @@ int intel_modeset_calc_cdclk(struct intel_atomic_state *state)
struct intel_crtc_state *crtc_state;
pipe = ilog2(new_cdclk_state->active_pipes);
- crtc = intel_get_crtc_for_pipe(dev_priv, pipe);
+ crtc = intel_crtc_for_pipe(dev_priv, pipe);
crtc_state = intel_atomic_get_crtc_state(&state->base, crtc);
if (IS_ERR(crtc_state))
@@ -2597,9 +2763,14 @@ int intel_modeset_calc_cdclk(struct intel_atomic_state *state)
pipe = INVALID_PIPE;
}
- if (intel_cdclk_can_crawl(dev_priv,
- &old_cdclk_state->actual,
- &new_cdclk_state->actual)) {
+ if (intel_cdclk_can_squash(dev_priv,
+ &old_cdclk_state->actual,
+ &new_cdclk_state->actual)) {
+ drm_dbg_kms(&dev_priv->drm,
+ "Can change cdclk via squasher\n");
+ } else if (intel_cdclk_can_crawl(dev_priv,
+ &old_cdclk_state->actual,
+ &new_cdclk_state->actual)) {
drm_dbg_kms(&dev_priv->drm,
"Can change cdclk via crawl\n");
} else if (pipe != INVALID_PIPE) {
diff --git a/drivers/gpu/drm/i915/display/intel_cdclk.h b/drivers/gpu/drm/i915/display/intel_cdclk.h
index 309b3f3..fc63852 100644
--- a/drivers/gpu/drm/i915/display/intel_cdclk.h
+++ b/drivers/gpu/drm/i915/display/intel_cdclk.h
@@ -16,13 +16,6 @@ struct drm_i915_private;
struct intel_atomic_state;
struct intel_crtc_state;
-struct intel_cdclk_vals {
- u32 cdclk;
- u16 refclk;
- u8 divider; /* CD2X divider * 2 */
- u8 ratio;
-};
-
struct intel_cdclk_state {
struct intel_global_state base;
@@ -70,7 +63,8 @@ void intel_dump_cdclk_config(const struct intel_cdclk_config *cdclk_config,
int intel_modeset_calc_cdclk(struct intel_atomic_state *state);
void intel_cdclk_get_cdclk(struct drm_i915_private *dev_priv,
struct intel_cdclk_config *cdclk_config);
-int intel_cdclk_bw_calc_min_cdclk(struct intel_atomic_state *state);
+int intel_cdclk_atomic_check(struct intel_atomic_state *state,
+ bool *need_cdclk_calc);
struct intel_cdclk_state *
intel_atomic_get_cdclk_state(struct intel_atomic_state *state);
diff --git a/drivers/gpu/drm/i915/display/intel_color.c b/drivers/gpu/drm/i915/display/intel_color.c
index 5359b73..de3ded1 100644
--- a/drivers/gpu/drm/i915/display/intel_color.c
+++ b/drivers/gpu/drm/i915/display/intel_color.c
@@ -26,7 +26,7 @@
#include "intel_de.h"
#include "intel_display_types.h"
#include "intel_dpll.h"
-#include "intel_dsi.h"
+#include "vlv_dsi_pll.h"
#define CTM_COEFF_SIGN (1ULL << 63)
@@ -552,8 +552,8 @@ static void i9xx_load_lut_8(struct intel_crtc *crtc,
lut = blob->data;
for (i = 0; i < 256; i++)
- intel_de_write(dev_priv, PALETTE(pipe, i),
- i9xx_lut_8(&lut[i]));
+ intel_de_write_fw(dev_priv, PALETTE(pipe, i),
+ i9xx_lut_8(&lut[i]));
}
static void i9xx_load_luts(const struct intel_crtc_state *crtc_state)
@@ -576,15 +576,15 @@ static void i965_load_lut_10p6(struct intel_crtc *crtc,
enum pipe pipe = crtc->pipe;
for (i = 0; i < lut_size - 1; i++) {
- intel_de_write(dev_priv, PALETTE(pipe, 2 * i + 0),
- i965_lut_10p6_ldw(&lut[i]));
- intel_de_write(dev_priv, PALETTE(pipe, 2 * i + 1),
- i965_lut_10p6_udw(&lut[i]));
+ intel_de_write_fw(dev_priv, PALETTE(pipe, 2 * i + 0),
+ i965_lut_10p6_ldw(&lut[i]));
+ intel_de_write_fw(dev_priv, PALETTE(pipe, 2 * i + 1),
+ i965_lut_10p6_udw(&lut[i]));
}
- intel_de_write(dev_priv, PIPEGCMAX(pipe, 0), lut[i].red);
- intel_de_write(dev_priv, PIPEGCMAX(pipe, 1), lut[i].green);
- intel_de_write(dev_priv, PIPEGCMAX(pipe, 2), lut[i].blue);
+ intel_de_write_fw(dev_priv, PIPEGCMAX(pipe, 0), lut[i].red);
+ intel_de_write_fw(dev_priv, PIPEGCMAX(pipe, 1), lut[i].green);
+ intel_de_write_fw(dev_priv, PIPEGCMAX(pipe, 2), lut[i].blue);
}
static void i965_load_luts(const struct intel_crtc_state *crtc_state)
@@ -618,8 +618,8 @@ static void ilk_load_lut_8(struct intel_crtc *crtc,
lut = blob->data;
for (i = 0; i < 256; i++)
- intel_de_write(dev_priv, LGC_PALETTE(pipe, i),
- i9xx_lut_8(&lut[i]));
+ intel_de_write_fw(dev_priv, LGC_PALETTE(pipe, i),
+ i9xx_lut_8(&lut[i]));
}
static void ilk_load_lut_10(struct intel_crtc *crtc,
@@ -631,8 +631,8 @@ static void ilk_load_lut_10(struct intel_crtc *crtc,
enum pipe pipe = crtc->pipe;
for (i = 0; i < lut_size; i++)
- intel_de_write(dev_priv, PREC_PALETTE(pipe, i),
- ilk_lut_10(&lut[i]));
+ intel_de_write_fw(dev_priv, PREC_PALETTE(pipe, i),
+ ilk_lut_10(&lut[i]));
}
static void ilk_load_luts(const struct intel_crtc_state *crtc_state)
@@ -681,16 +681,16 @@ static void ivb_load_lut_10(struct intel_crtc *crtc,
const struct drm_color_lut *entry =
&lut[i * (lut_size - 1) / (hw_lut_size - 1)];
- intel_de_write(dev_priv, PREC_PAL_INDEX(pipe), prec_index++);
- intel_de_write(dev_priv, PREC_PAL_DATA(pipe),
- ilk_lut_10(entry));
+ intel_de_write_fw(dev_priv, PREC_PAL_INDEX(pipe), prec_index++);
+ intel_de_write_fw(dev_priv, PREC_PAL_DATA(pipe),
+ ilk_lut_10(entry));
}
/*
* Reset the index, otherwise it prevents the legacy palette to be
* written properly.
*/
- intel_de_write(dev_priv, PREC_PAL_INDEX(pipe), 0);
+ intel_de_write_fw(dev_priv, PREC_PAL_INDEX(pipe), 0);
}
/* On BDW+ the index auto increment mode actually works */
@@ -704,23 +704,23 @@ static void bdw_load_lut_10(struct intel_crtc *crtc,
int i, lut_size = drm_color_lut_size(blob);
enum pipe pipe = crtc->pipe;
- intel_de_write(dev_priv, PREC_PAL_INDEX(pipe),
- prec_index | PAL_PREC_AUTO_INCREMENT);
+ intel_de_write_fw(dev_priv, PREC_PAL_INDEX(pipe),
+ prec_index | PAL_PREC_AUTO_INCREMENT);
for (i = 0; i < hw_lut_size; i++) {
/* We discard half the user entries in split gamma mode */
const struct drm_color_lut *entry =
&lut[i * (lut_size - 1) / (hw_lut_size - 1)];
- intel_de_write(dev_priv, PREC_PAL_DATA(pipe),
- ilk_lut_10(entry));
+ intel_de_write_fw(dev_priv, PREC_PAL_DATA(pipe),
+ ilk_lut_10(entry));
}
/*
* Reset the index, otherwise it prevents the legacy palette to be
* written properly.
*/
- intel_de_write(dev_priv, PREC_PAL_INDEX(pipe), 0);
+ intel_de_write_fw(dev_priv, PREC_PAL_INDEX(pipe), 0);
}
static void ivb_load_lut_ext_max(const struct intel_crtc_state *crtc_state)
@@ -808,6 +808,14 @@ static void bdw_load_luts(const struct intel_crtc_state *crtc_state)
}
}
+static int glk_degamma_lut_size(struct drm_i915_private *i915)
+{
+ if (DISPLAY_VER(i915) >= 13)
+ return 131;
+ else
+ return 35;
+}
+
static void glk_load_degamma_lut(const struct intel_crtc_state *crtc_state)
{
struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
@@ -821,14 +829,14 @@ static void glk_load_degamma_lut(const struct intel_crtc_state *crtc_state)
* ignore the index bits, so we need to reset it to index 0
* separately.
*/
- intel_de_write(dev_priv, PRE_CSC_GAMC_INDEX(pipe), 0);
- intel_de_write(dev_priv, PRE_CSC_GAMC_INDEX(pipe),
- PRE_CSC_GAMC_AUTO_INCREMENT);
+ intel_de_write_fw(dev_priv, PRE_CSC_GAMC_INDEX(pipe), 0);
+ intel_de_write_fw(dev_priv, PRE_CSC_GAMC_INDEX(pipe),
+ PRE_CSC_GAMC_AUTO_INCREMENT);
for (i = 0; i < lut_size; i++) {
/*
- * First 33 entries represent range from 0 to 1.0
- * 34th and 35th entry will represent extended range
+ * First lut_size entries represent range from 0 to 1.0
+ * 3 additional lut entries will represent extended range
* inputs 3.0 and 7.0 respectively, currently clamped
* at 1.0. Since the precision is 16bit, the user
* value can be directly filled to register.
@@ -839,15 +847,15 @@ static void glk_load_degamma_lut(const struct intel_crtc_state *crtc_state)
* ToDo: Extend to max 7.0. Enable 32 bit input value
* as compared to just 16 to achieve this.
*/
- intel_de_write(dev_priv, PRE_CSC_GAMC_DATA(pipe),
- lut[i].green);
+ intel_de_write_fw(dev_priv, PRE_CSC_GAMC_DATA(pipe),
+ lut[i].green);
}
/* Clamp values > 1.0. */
- while (i++ < 35)
- intel_de_write(dev_priv, PRE_CSC_GAMC_DATA(pipe), 1 << 16);
+ while (i++ < glk_degamma_lut_size(dev_priv))
+ intel_de_write_fw(dev_priv, PRE_CSC_GAMC_DATA(pipe), 1 << 16);
- intel_de_write(dev_priv, PRE_CSC_GAMC_INDEX(pipe), 0);
+ intel_de_write_fw(dev_priv, PRE_CSC_GAMC_INDEX(pipe), 0);
}
static void glk_load_degamma_lut_linear(const struct intel_crtc_state *crtc_state)
@@ -862,21 +870,21 @@ static void glk_load_degamma_lut_linear(const struct intel_crtc_state *crtc_stat
* ignore the index bits, so we need to reset it to index 0
* separately.
*/
- intel_de_write(dev_priv, PRE_CSC_GAMC_INDEX(pipe), 0);
- intel_de_write(dev_priv, PRE_CSC_GAMC_INDEX(pipe),
- PRE_CSC_GAMC_AUTO_INCREMENT);
+ intel_de_write_fw(dev_priv, PRE_CSC_GAMC_INDEX(pipe), 0);
+ intel_de_write_fw(dev_priv, PRE_CSC_GAMC_INDEX(pipe),
+ PRE_CSC_GAMC_AUTO_INCREMENT);
for (i = 0; i < lut_size; i++) {
u32 v = (i << 16) / (lut_size - 1);
- intel_de_write(dev_priv, PRE_CSC_GAMC_DATA(pipe), v);
+ intel_de_write_fw(dev_priv, PRE_CSC_GAMC_DATA(pipe), v);
}
/* Clamp values > 1.0. */
while (i++ < 35)
- intel_de_write(dev_priv, PRE_CSC_GAMC_DATA(pipe), 1 << 16);
+ intel_de_write_fw(dev_priv, PRE_CSC_GAMC_DATA(pipe), 1 << 16);
- intel_de_write(dev_priv, PRE_CSC_GAMC_INDEX(pipe), 0);
+ intel_de_write_fw(dev_priv, PRE_CSC_GAMC_INDEX(pipe), 0);
}
static void glk_load_luts(const struct intel_crtc_state *crtc_state)
@@ -1071,10 +1079,10 @@ static void chv_load_cgm_degamma(struct intel_crtc *crtc,
enum pipe pipe = crtc->pipe;
for (i = 0; i < lut_size; i++) {
- intel_de_write(dev_priv, CGM_PIPE_DEGAMMA(pipe, i, 0),
- chv_cgm_degamma_ldw(&lut[i]));
- intel_de_write(dev_priv, CGM_PIPE_DEGAMMA(pipe, i, 1),
- chv_cgm_degamma_udw(&lut[i]));
+ intel_de_write_fw(dev_priv, CGM_PIPE_DEGAMMA(pipe, i, 0),
+ chv_cgm_degamma_ldw(&lut[i]));
+ intel_de_write_fw(dev_priv, CGM_PIPE_DEGAMMA(pipe, i, 1),
+ chv_cgm_degamma_udw(&lut[i]));
}
}
@@ -1105,10 +1113,10 @@ static void chv_load_cgm_gamma(struct intel_crtc *crtc,
enum pipe pipe = crtc->pipe;
for (i = 0; i < lut_size; i++) {
- intel_de_write(dev_priv, CGM_PIPE_GAMMA(pipe, i, 0),
- chv_cgm_gamma_ldw(&lut[i]));
- intel_de_write(dev_priv, CGM_PIPE_GAMMA(pipe, i, 1),
- chv_cgm_gamma_udw(&lut[i]));
+ intel_de_write_fw(dev_priv, CGM_PIPE_GAMMA(pipe, i, 0),
+ chv_cgm_gamma_ldw(&lut[i]));
+ intel_de_write_fw(dev_priv, CGM_PIPE_GAMMA(pipe, i, 1),
+ chv_cgm_gamma_udw(&lut[i]));
}
}
@@ -1131,8 +1139,8 @@ static void chv_load_luts(const struct intel_crtc_state *crtc_state)
else
i965_load_luts(crtc_state);
- intel_de_write(dev_priv, CGM_PIPE_MODE(crtc->pipe),
- crtc_state->cgm_mode);
+ intel_de_write_fw(dev_priv, CGM_PIPE_MODE(crtc->pipe),
+ crtc_state->cgm_mode);
}
void intel_color_load_luts(const struct intel_crtc_state *crtc_state)
@@ -1574,6 +1582,8 @@ static int glk_color_check(struct intel_crtc_state *crtc_state)
static u32 icl_gamma_mode(const struct intel_crtc_state *crtc_state)
{
+ struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
+ struct drm_i915_private *i915 = to_i915(crtc->base.dev);
u32 gamma_mode = 0;
if (crtc_state->hw.degamma_lut)
@@ -1586,6 +1596,13 @@ static u32 icl_gamma_mode(const struct intel_crtc_state *crtc_state)
if (!crtc_state->hw.gamma_lut ||
crtc_state_is_legacy_gamma(crtc_state))
gamma_mode |= GAMMA_MODE_MODE_8BIT;
+ /*
+ * Enable 10bit gamma for D13
+ * ToDo: Extend to Logarithmic Gamma once the new UAPI
+ * is acccepted and implemented by a userspace consumer
+ */
+ else if (DISPLAY_VER(i915) >= 13)
+ gamma_mode |= GAMMA_MODE_MODE_10BIT;
else
gamma_mode |= GAMMA_MODE_MODE_12BIT_MULTI_SEGMENTED;
@@ -1808,7 +1825,7 @@ static struct drm_property_blob *i9xx_read_lut_8(struct intel_crtc *crtc)
lut = blob->data;
for (i = 0; i < LEGACY_LUT_LENGTH; i++) {
- u32 val = intel_de_read(dev_priv, PALETTE(pipe, i));
+ u32 val = intel_de_read_fw(dev_priv, PALETTE(pipe, i));
i9xx_lut_8_pack(&lut[i], val);
}
@@ -1843,15 +1860,15 @@ static struct drm_property_blob *i965_read_lut_10p6(struct intel_crtc *crtc)
lut = blob->data;
for (i = 0; i < lut_size - 1; i++) {
- u32 ldw = intel_de_read(dev_priv, PALETTE(pipe, 2 * i + 0));
- u32 udw = intel_de_read(dev_priv, PALETTE(pipe, 2 * i + 1));
+ u32 ldw = intel_de_read_fw(dev_priv, PALETTE(pipe, 2 * i + 0));
+ u32 udw = intel_de_read_fw(dev_priv, PALETTE(pipe, 2 * i + 1));
i965_lut_10p6_pack(&lut[i], ldw, udw);
}
- lut[i].red = i965_lut_11p6_max_pack(intel_de_read(dev_priv, PIPEGCMAX(pipe, 0)));
- lut[i].green = i965_lut_11p6_max_pack(intel_de_read(dev_priv, PIPEGCMAX(pipe, 1)));
- lut[i].blue = i965_lut_11p6_max_pack(intel_de_read(dev_priv, PIPEGCMAX(pipe, 2)));
+ lut[i].red = i965_lut_11p6_max_pack(intel_de_read_fw(dev_priv, PIPEGCMAX(pipe, 0)));
+ lut[i].green = i965_lut_11p6_max_pack(intel_de_read_fw(dev_priv, PIPEGCMAX(pipe, 1)));
+ lut[i].blue = i965_lut_11p6_max_pack(intel_de_read_fw(dev_priv, PIPEGCMAX(pipe, 2)));
return blob;
}
@@ -1886,8 +1903,8 @@ static struct drm_property_blob *chv_read_cgm_gamma(struct intel_crtc *crtc)
lut = blob->data;
for (i = 0; i < lut_size; i++) {
- u32 ldw = intel_de_read(dev_priv, CGM_PIPE_GAMMA(pipe, i, 0));
- u32 udw = intel_de_read(dev_priv, CGM_PIPE_GAMMA(pipe, i, 1));
+ u32 ldw = intel_de_read_fw(dev_priv, CGM_PIPE_GAMMA(pipe, i, 0));
+ u32 udw = intel_de_read_fw(dev_priv, CGM_PIPE_GAMMA(pipe, i, 1));
chv_cgm_gamma_pack(&lut[i], ldw, udw);
}
@@ -1922,7 +1939,7 @@ static struct drm_property_blob *ilk_read_lut_8(struct intel_crtc *crtc)
lut = blob->data;
for (i = 0; i < LEGACY_LUT_LENGTH; i++) {
- u32 val = intel_de_read(dev_priv, LGC_PALETTE(pipe, i));
+ u32 val = intel_de_read_fw(dev_priv, LGC_PALETTE(pipe, i));
i9xx_lut_8_pack(&lut[i], val);
}
@@ -1947,7 +1964,7 @@ static struct drm_property_blob *ilk_read_lut_10(struct intel_crtc *crtc)
lut = blob->data;
for (i = 0; i < lut_size; i++) {
- u32 val = intel_de_read(dev_priv, PREC_PALETTE(pipe, i));
+ u32 val = intel_de_read_fw(dev_priv, PREC_PALETTE(pipe, i));
ilk_lut_10_pack(&lut[i], val);
}
@@ -1999,16 +2016,16 @@ static struct drm_property_blob *bdw_read_lut_10(struct intel_crtc *crtc,
lut = blob->data;
- intel_de_write(dev_priv, PREC_PAL_INDEX(pipe),
- prec_index | PAL_PREC_AUTO_INCREMENT);
+ intel_de_write_fw(dev_priv, PREC_PAL_INDEX(pipe),
+ prec_index | PAL_PREC_AUTO_INCREMENT);
for (i = 0; i < lut_size; i++) {
- u32 val = intel_de_read(dev_priv, PREC_PAL_DATA(pipe));
+ u32 val = intel_de_read_fw(dev_priv, PREC_PAL_DATA(pipe));
ilk_lut_10_pack(&lut[i], val);
}
- intel_de_write(dev_priv, PREC_PAL_INDEX(pipe), 0);
+ intel_de_write_fw(dev_priv, PREC_PAL_INDEX(pipe), 0);
return blob;
}
@@ -2050,17 +2067,17 @@ icl_read_lut_multi_segment(struct intel_crtc *crtc)
lut = blob->data;
- intel_de_write(dev_priv, PREC_PAL_MULTI_SEG_INDEX(pipe),
- PAL_PREC_AUTO_INCREMENT);
+ intel_de_write_fw(dev_priv, PREC_PAL_MULTI_SEG_INDEX(pipe),
+ PAL_PREC_AUTO_INCREMENT);
for (i = 0; i < 9; i++) {
- u32 ldw = intel_de_read(dev_priv, PREC_PAL_MULTI_SEG_DATA(pipe));
- u32 udw = intel_de_read(dev_priv, PREC_PAL_MULTI_SEG_DATA(pipe));
+ u32 ldw = intel_de_read_fw(dev_priv, PREC_PAL_MULTI_SEG_DATA(pipe));
+ u32 udw = intel_de_read_fw(dev_priv, PREC_PAL_MULTI_SEG_DATA(pipe));
icl_lut_multi_seg_pack(&lut[i], ldw, udw);
}
- intel_de_write(dev_priv, PREC_PAL_MULTI_SEG_INDEX(pipe), 0);
+ intel_de_write_fw(dev_priv, PREC_PAL_MULTI_SEG_INDEX(pipe), 0);
/*
* FIXME readouts from PAL_PREC_DATA register aren't giving
diff --git a/drivers/gpu/drm/i915/display/intel_combo_phy.c b/drivers/gpu/drm/i915/display/intel_combo_phy.c
index 634e8d4..f628e05 100644
--- a/drivers/gpu/drm/i915/display/intel_combo_phy.c
+++ b/drivers/gpu/drm/i915/display/intel_combo_phy.c
@@ -301,7 +301,7 @@ void intel_combo_phy_power_up_lanes(struct drm_i915_private *dev_priv,
val = intel_de_read(dev_priv, ICL_PORT_CL_DW10(phy));
val &= ~PWR_DOWN_LN_MASK;
- val |= lane_mask << PWR_DOWN_LN_SHIFT;
+ val |= lane_mask;
intel_de_write(dev_priv, ICL_PORT_CL_DW10(phy), val);
}
diff --git a/drivers/gpu/drm/i915/display/intel_crt.c b/drivers/gpu/drm/i915/display/intel_crt.c
index 1c161ee..6a3893c 100644
--- a/drivers/gpu/drm/i915/display/intel_crt.c
+++ b/drivers/gpu/drm/i915/display/intel_crt.c
@@ -45,6 +45,7 @@
#include "intel_fifo_underrun.h"
#include "intel_gmbus.h"
#include "intel_hotplug.h"
+#include "intel_pch_display.h"
/* Here's the desired hotplug mode */
#define ADPA_HOTPLUG_BITS (ADPA_CRT_HOTPLUG_PERIOD_128 | \
@@ -143,7 +144,7 @@ static void intel_crt_get_config(struct intel_encoder *encoder,
static void hsw_crt_get_config(struct intel_encoder *encoder,
struct intel_crtc_state *pipe_config)
{
- struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
+ lpt_pch_get_config(pipe_config);
hsw_ddi_get_config(encoder, pipe_config);
@@ -152,8 +153,6 @@ static void hsw_crt_get_config(struct intel_encoder *encoder,
DRM_MODE_FLAG_PVSYNC |
DRM_MODE_FLAG_NVSYNC);
pipe_config->hw.adjusted_mode.flags |= intel_crt_get_flags(encoder);
-
- pipe_config->hw.adjusted_mode.crtc_clock = lpt_get_iclkip(dev_priv);
}
/* Note: The caller is required to filter out dpms modes not supported by the
@@ -247,6 +246,7 @@ static void hsw_post_disable_crt(struct intel_atomic_state *state,
const struct intel_crtc_state *old_crtc_state,
const struct drm_connector_state *old_conn_state)
{
+ struct intel_crtc *crtc = to_intel_crtc(old_crtc_state->uapi.crtc);
struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
intel_crtc_vblank_off(old_crtc_state);
@@ -261,10 +261,9 @@ static void hsw_post_disable_crt(struct intel_atomic_state *state,
pch_post_disable_crt(state, encoder, old_crtc_state, old_conn_state);
- lpt_disable_pch_transcoder(dev_priv);
- lpt_disable_iclkip(dev_priv);
+ lpt_pch_disable(state, crtc);
- intel_ddi_fdi_post_disable(state, encoder, old_crtc_state, old_conn_state);
+ hsw_fdi_disable(encoder);
drm_WARN_ON(&dev_priv->drm, !old_crtc_state->has_pch_encoder);
@@ -316,14 +315,14 @@ static void hsw_enable_crt(struct intel_atomic_state *state,
intel_enable_transcoder(crtc_state);
- lpt_pch_enable(crtc_state);
+ lpt_pch_enable(state, crtc);
intel_crtc_vblank_on(crtc_state);
intel_crt_set_dpms(encoder, crtc_state, DRM_MODE_DPMS_ON);
- intel_wait_for_vblank(dev_priv, pipe);
- intel_wait_for_vblank(dev_priv, pipe);
+ intel_crtc_wait_for_next_vblank(crtc);
+ intel_crtc_wait_for_next_vblank(crtc);
intel_set_cpu_fifo_underrun_reporting(dev_priv, pipe, true);
intel_set_pch_fifo_underrun_reporting(dev_priv, PIPE_A, true);
}
@@ -722,7 +721,7 @@ intel_crt_load_detect(struct intel_crt *crt, u32 pipe)
intel_uncore_posting_read(uncore, pipeconf_reg);
/* Wait for next Vblank to substitue
* border color for Color info */
- intel_wait_for_vblank(dev_priv, pipe);
+ intel_crtc_wait_for_next_vblank(intel_crtc_for_pipe(dev_priv, pipe));
st00 = intel_uncore_read8(uncore, _VGA_MSR_WRITE);
status = ((st00 & (1 << 4)) != 0) ?
connector_status_connected :
diff --git a/drivers/gpu/drm/i915/display/intel_crtc.c b/drivers/gpu/drm/i915/display/intel_crtc.c
index 254e671..16c3ca6 100644
--- a/drivers/gpu/drm/i915/display/intel_crtc.c
+++ b/drivers/gpu/drm/i915/display/intel_crtc.c
@@ -3,29 +3,31 @@
* Copyright © 2020 Intel Corporation
*/
#include <linux/kernel.h>
+#include <linux/pm_qos.h>
#include <linux/slab.h>
#include <drm/drm_atomic_helper.h>
#include <drm/drm_fourcc.h>
#include <drm/drm_plane.h>
#include <drm/drm_plane_helper.h>
+#include <drm/drm_vblank_work.h>
-#include "i915_trace.h"
#include "i915_vgpu.h"
-
+#include "i9xx_plane.h"
+#include "icl_dsi.h"
#include "intel_atomic.h"
#include "intel_atomic_plane.h"
#include "intel_color.h"
#include "intel_crtc.h"
#include "intel_cursor.h"
#include "intel_display_debugfs.h"
+#include "intel_display_trace.h"
#include "intel_display_types.h"
#include "intel_dsi.h"
#include "intel_pipe_crc.h"
#include "intel_psr.h"
#include "intel_sprite.h"
#include "intel_vrr.h"
-#include "i9xx_plane.h"
#include "skl_universal_plane.h"
static void assert_vblank_disabled(struct drm_crtc *crtc)
@@ -34,6 +36,38 @@ static void assert_vblank_disabled(struct drm_crtc *crtc)
drm_crtc_vblank_put(crtc);
}
+struct intel_crtc *intel_first_crtc(struct drm_i915_private *i915)
+{
+ return to_intel_crtc(drm_crtc_from_index(&i915->drm, 0));
+}
+
+struct intel_crtc *intel_crtc_for_pipe(struct drm_i915_private *i915,
+ enum pipe pipe)
+{
+ struct intel_crtc *crtc;
+
+ for_each_intel_crtc(&i915->drm, crtc) {
+ if (crtc->pipe == pipe)
+ return crtc;
+ }
+
+ return NULL;
+}
+
+void intel_crtc_wait_for_next_vblank(struct intel_crtc *crtc)
+{
+ drm_crtc_wait_one_vblank(&crtc->base);
+}
+
+void intel_wait_for_vblank_if_active(struct drm_i915_private *i915,
+ enum pipe pipe)
+{
+ struct intel_crtc *crtc = intel_crtc_for_pipe(i915, pipe);
+
+ if (crtc->active)
+ intel_crtc_wait_for_next_vblank(crtc);
+}
+
u32 intel_crtc_get_vblank_counter(struct intel_crtc *crtc)
{
struct drm_device *dev = crtc->base.dev;
@@ -167,6 +201,8 @@ static void intel_crtc_destroy(struct drm_crtc *_crtc)
{
struct intel_crtc *crtc = to_intel_crtc(_crtc);
+ cpu_latency_qos_remove_request(&crtc->vblank_pm_qos);
+
drm_crtc_cleanup(&crtc->base);
kfree(crtc);
}
@@ -323,18 +359,6 @@ int intel_crtc_init(struct drm_i915_private *dev_priv, enum pipe pipe)
if (ret)
goto fail;
- BUG_ON(pipe >= ARRAY_SIZE(dev_priv->pipe_to_crtc_mapping) ||
- dev_priv->pipe_to_crtc_mapping[pipe] != NULL);
- dev_priv->pipe_to_crtc_mapping[pipe] = crtc;
-
- if (DISPLAY_VER(dev_priv) < 9) {
- enum i9xx_plane_id i9xx_plane = primary->i9xx_plane;
-
- BUG_ON(i9xx_plane >= ARRAY_SIZE(dev_priv->plane_to_crtc_mapping) ||
- dev_priv->plane_to_crtc_mapping[i9xx_plane] != NULL);
- dev_priv->plane_to_crtc_mapping[i9xx_plane] = crtc;
- }
-
if (DISPLAY_VER(dev_priv) >= 11)
drm_crtc_create_scaling_filter_property(&crtc->base,
BIT(DRM_SCALING_FILTER_DEFAULT) |
@@ -344,6 +368,8 @@ int intel_crtc_init(struct drm_i915_private *dev_priv, enum pipe pipe)
intel_crtc_crc_init(crtc);
+ cpu_latency_qos_add_request(&crtc->vblank_pm_qos, PM_QOS_DEFAULT_VALUE);
+
drm_WARN_ON(&dev_priv->drm, drm_crtc_index(&crtc->base) != crtc->pipe);
return 0;
@@ -354,6 +380,65 @@ int intel_crtc_init(struct drm_i915_private *dev_priv, enum pipe pipe)
return ret;
}
+static bool intel_crtc_needs_vblank_work(const struct intel_crtc_state *crtc_state)
+{
+ return crtc_state->hw.active &&
+ !intel_crtc_needs_modeset(crtc_state) &&
+ !crtc_state->preload_luts &&
+ (crtc_state->uapi.color_mgmt_changed ||
+ crtc_state->update_pipe);
+}
+
+static void intel_crtc_vblank_work(struct kthread_work *base)
+{
+ struct drm_vblank_work *work = to_drm_vblank_work(base);
+ struct intel_crtc_state *crtc_state =
+ container_of(work, typeof(*crtc_state), vblank_work);
+ struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
+
+ trace_intel_crtc_vblank_work_start(crtc);
+
+ intel_color_load_luts(crtc_state);
+
+ if (crtc_state->uapi.event) {
+ spin_lock_irq(&crtc->base.dev->event_lock);
+ drm_crtc_send_vblank_event(&crtc->base, crtc_state->uapi.event);
+ crtc_state->uapi.event = NULL;
+ spin_unlock_irq(&crtc->base.dev->event_lock);
+ }
+
+ trace_intel_crtc_vblank_work_end(crtc);
+}
+
+static void intel_crtc_vblank_work_init(struct intel_crtc_state *crtc_state)
+{
+ struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
+
+ drm_vblank_work_init(&crtc_state->vblank_work, &crtc->base,
+ intel_crtc_vblank_work);
+ /*
+ * Interrupt latency is critical for getting the vblank
+ * work executed as early as possible during the vblank.
+ */
+ cpu_latency_qos_update_request(&crtc->vblank_pm_qos, 0);
+}
+
+void intel_wait_for_vblank_workers(struct intel_atomic_state *state)
+{
+ struct intel_crtc_state *crtc_state;
+ struct intel_crtc *crtc;
+ int i;
+
+ for_each_new_intel_crtc_in_state(state, crtc, crtc_state, i) {
+ if (!intel_crtc_needs_vblank_work(crtc_state))
+ continue;
+
+ drm_vblank_work_flush(&crtc_state->vblank_work);
+ cpu_latency_qos_update_request(&crtc->vblank_pm_qos,
+ PM_QOS_DEFAULT_VALUE);
+ }
+}
+
int intel_usecs_to_scanlines(const struct drm_display_mode *adjusted_mode,
int usecs)
{
@@ -387,7 +472,7 @@ static int intel_mode_vblank_start(const struct drm_display_mode *mode)
* until a subsequent call to intel_pipe_update_end(). That is done to
* avoid random delays.
*/
-void intel_pipe_update_start(const struct intel_crtc_state *new_crtc_state)
+void intel_pipe_update_start(struct intel_crtc_state *new_crtc_state)
{
struct intel_crtc *crtc = to_intel_crtc(new_crtc_state->uapi.crtc);
struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
@@ -402,10 +487,17 @@ void intel_pipe_update_start(const struct intel_crtc_state *new_crtc_state)
if (new_crtc_state->uapi.async_flip)
return;
- if (new_crtc_state->vrr.enable)
- vblank_start = intel_vrr_vmax_vblank_start(new_crtc_state);
- else
+ if (intel_crtc_needs_vblank_work(new_crtc_state))
+ intel_crtc_vblank_work_init(new_crtc_state);
+
+ if (new_crtc_state->vrr.enable) {
+ if (intel_vrr_is_push_sent(new_crtc_state))
+ vblank_start = intel_vrr_vmin_vblank_start(new_crtc_state);
+ else
+ vblank_start = intel_vrr_vmax_vblank_start(new_crtc_state);
+ } else {
vblank_start = intel_mode_vblank_start(adjusted_mode);
+ }
/* FIXME needs to be calibrated sensibly */
min = vblank_start - intel_usecs_to_scanlines(adjusted_mode,
@@ -554,7 +646,11 @@ void intel_pipe_update_end(struct intel_crtc_state *new_crtc_state)
* Would be slightly nice to just grab the vblank count and arm the
* event outside of the critical section - the spinlock might spin for a
* while ... */
- if (new_crtc_state->uapi.event) {
+ if (intel_crtc_needs_vblank_work(new_crtc_state)) {
+ drm_vblank_work_schedule(&new_crtc_state->vblank_work,
+ drm_crtc_accurate_vblank_count(&crtc->base) + 1,
+ false);
+ } else if (new_crtc_state->uapi.event) {
drm_WARN_ON(&dev_priv->drm,
drm_crtc_vblank_get(&crtc->base) != 0);
@@ -566,11 +662,24 @@ void intel_pipe_update_end(struct intel_crtc_state *new_crtc_state)
new_crtc_state->uapi.event = NULL;
}
- local_irq_enable();
-
- /* Send VRR Push to terminate Vblank */
+ /*
+ * Send VRR Push to terminate Vblank. If we are already in vblank
+ * this has to be done _after_ sampling the frame counter, as
+ * otherwise the push would immediately terminate the vblank and
+ * the sampled frame counter would correspond to the next frame
+ * instead of the current frame.
+ *
+ * There is a tiny race here (iff vblank evasion failed us) where
+ * we might sample the frame counter just before vmax vblank start
+ * but the push would be sent just after it. That would cause the
+ * push to affect the next frame instead of the current frame,
+ * which would cause the next frame to terminate already at vmin
+ * vblank start instead of vmax vblank start.
+ */
intel_vrr_send_push(new_crtc_state);
+ local_irq_enable();
+
if (intel_vgpu_active(dev_priv))
return;
diff --git a/drivers/gpu/drm/i915/display/intel_crtc.h b/drivers/gpu/drm/i915/display/intel_crtc.h
index a5ae997..7307713 100644
--- a/drivers/gpu/drm/i915/display/intel_crtc.h
+++ b/drivers/gpu/drm/i915/display/intel_crtc.h
@@ -8,11 +8,16 @@
#include <linux/types.h>
+enum i9xx_plane_id;
enum pipe;
+struct drm_display_mode;
struct drm_i915_private;
+struct intel_atomic_state;
struct intel_crtc;
struct intel_crtc_state;
+int intel_usecs_to_scanlines(const struct drm_display_mode *adjusted_mode,
+ int usecs);
u32 intel_crtc_max_vblank_count(const struct intel_crtc_state *crtc_state);
int intel_crtc_init(struct drm_i915_private *dev_priv, enum pipe pipe);
struct intel_crtc_state *intel_crtc_state_alloc(struct intel_crtc *crtc);
@@ -21,5 +26,14 @@ void intel_crtc_state_reset(struct intel_crtc_state *crtc_state,
u32 intel_crtc_get_vblank_counter(struct intel_crtc *crtc);
void intel_crtc_vblank_on(const struct intel_crtc_state *crtc_state);
void intel_crtc_vblank_off(const struct intel_crtc_state *crtc_state);
+void intel_pipe_update_start(struct intel_crtc_state *new_crtc_state);
+void intel_pipe_update_end(struct intel_crtc_state *new_crtc_state);
+void intel_wait_for_vblank_workers(struct intel_atomic_state *state);
+struct intel_crtc *intel_first_crtc(struct drm_i915_private *i915);
+struct intel_crtc *intel_crtc_for_pipe(struct drm_i915_private *i915,
+ enum pipe pipe);
+void intel_wait_for_vblank_if_active(struct drm_i915_private *i915,
+ enum pipe pipe);
+void intel_crtc_wait_for_next_vblank(struct intel_crtc *crtc);
#endif
diff --git a/drivers/gpu/drm/i915/display/intel_cursor.c b/drivers/gpu/drm/i915/display/intel_cursor.c
index 11842f2..16d3468 100644
--- a/drivers/gpu/drm/i915/display/intel_cursor.c
+++ b/drivers/gpu/drm/i915/display/intel_cursor.c
@@ -28,11 +28,6 @@ static const u32 intel_cursor_formats[] = {
DRM_FORMAT_ARGB8888,
};
-static const u64 cursor_format_modifiers[] = {
- DRM_FORMAT_MOD_LINEAR,
- DRM_FORMAT_MOD_INVALID
-};
-
static u32 intel_cursor_base(const struct intel_plane_state *plane_state)
{
struct drm_i915_private *dev_priv =
@@ -195,7 +190,7 @@ static u32 i845_cursor_ctl(const struct intel_crtc_state *crtc_state,
{
return CURSOR_ENABLE |
CURSOR_FORMAT_ARGB |
- CURSOR_STRIDE(plane_state->view.color_plane[0].stride);
+ CURSOR_STRIDE(plane_state->view.color_plane[0].mapping_stride);
}
static bool i845_cursor_size_ok(const struct intel_plane_state *plane_state)
@@ -234,7 +229,7 @@ static int i845_check_cursor(struct intel_crtc_state *crtc_state,
}
drm_WARN_ON(&i915->drm, plane_state->uapi.visible &&
- plane_state->view.color_plane[0].stride != fb->pitches[0]);
+ plane_state->view.color_plane[0].mapping_stride != fb->pitches[0]);
switch (fb->pitches[0]) {
case 256:
@@ -253,9 +248,10 @@ static int i845_check_cursor(struct intel_crtc_state *crtc_state,
return 0;
}
-static void i845_update_cursor(struct intel_plane *plane,
- const struct intel_crtc_state *crtc_state,
- const struct intel_plane_state *plane_state)
+/* TODO: split into noarm+arm pair */
+static void i845_cursor_update_arm(struct intel_plane *plane,
+ const struct intel_crtc_state *crtc_state,
+ const struct intel_plane_state *plane_state)
{
struct drm_i915_private *dev_priv = to_i915(plane->base.dev);
u32 cntl = 0, base = 0, pos = 0, size = 0;
@@ -298,10 +294,10 @@ static void i845_update_cursor(struct intel_plane *plane,
spin_unlock_irqrestore(&dev_priv->uncore.lock, irqflags);
}
-static void i845_disable_cursor(struct intel_plane *plane,
- const struct intel_crtc_state *crtc_state)
+static void i845_cursor_disable_arm(struct intel_plane *plane,
+ const struct intel_crtc_state *crtc_state)
{
- i845_update_cursor(plane, crtc_state, NULL);
+ i845_cursor_update_arm(plane, crtc_state, NULL);
}
static bool i845_cursor_get_hw_state(struct intel_plane *plane,
@@ -455,7 +451,7 @@ static int i9xx_check_cursor(struct intel_crtc_state *crtc_state,
}
drm_WARN_ON(&dev_priv->drm, plane_state->uapi.visible &&
- plane_state->view.color_plane[0].stride != fb->pitches[0]);
+ plane_state->view.color_plane[0].mapping_stride != fb->pitches[0]);
if (fb->pitches[0] !=
drm_rect_width(&plane_state->uapi.dst) * fb->format->cpp[0]) {
@@ -488,9 +484,10 @@ static int i9xx_check_cursor(struct intel_crtc_state *crtc_state,
return 0;
}
-static void i9xx_update_cursor(struct intel_plane *plane,
- const struct intel_crtc_state *crtc_state,
- const struct intel_plane_state *plane_state)
+/* TODO: split into noarm+arm pair */
+static void i9xx_cursor_update_arm(struct intel_plane *plane,
+ const struct intel_crtc_state *crtc_state,
+ const struct intel_plane_state *plane_state)
{
struct drm_i915_private *dev_priv = to_i915(plane->base.dev);
enum pipe pipe = plane->pipe;
@@ -562,10 +559,10 @@ static void i9xx_update_cursor(struct intel_plane *plane,
spin_unlock_irqrestore(&dev_priv->uncore.lock, irqflags);
}
-static void i9xx_disable_cursor(struct intel_plane *plane,
- const struct intel_crtc_state *crtc_state)
+static void i9xx_cursor_disable_arm(struct intel_plane *plane,
+ const struct intel_crtc_state *crtc_state)
{
- i9xx_update_cursor(plane, crtc_state, NULL);
+ i9xx_cursor_update_arm(plane, crtc_state, NULL);
}
static bool i9xx_cursor_get_hw_state(struct intel_plane *plane,
@@ -605,8 +602,10 @@ static bool i9xx_cursor_get_hw_state(struct intel_plane *plane,
static bool intel_cursor_format_mod_supported(struct drm_plane *_plane,
u32 format, u64 modifier)
{
- return modifier == DRM_FORMAT_MOD_LINEAR &&
- format == DRM_FORMAT_ARGB8888;
+ if (!intel_fb_plane_supports_modifier(to_intel_plane(_plane), modifier))
+ return false;
+
+ return format == DRM_FORMAT_ARGB8888;
}
static int
@@ -717,10 +716,12 @@ intel_legacy_cursor_update(struct drm_plane *_plane,
*/
crtc_state->active_planes = new_crtc_state->active_planes;
- if (new_plane_state->uapi.visible)
- intel_update_plane(plane, crtc_state, new_plane_state);
- else
- intel_disable_plane(plane, crtc_state);
+ if (new_plane_state->uapi.visible) {
+ intel_plane_update_noarm(plane, crtc_state, new_plane_state);
+ intel_plane_update_arm(plane, crtc_state, new_plane_state);
+ } else {
+ intel_plane_disable_arm(plane, crtc_state);
+ }
intel_plane_unpin_fb(old_plane_state);
@@ -754,6 +755,7 @@ intel_cursor_plane_create(struct drm_i915_private *dev_priv,
{
struct intel_plane *cursor;
int ret, zpos;
+ u64 *modifiers;
cursor = intel_plane_alloc();
if (IS_ERR(cursor))
@@ -766,14 +768,14 @@ intel_cursor_plane_create(struct drm_i915_private *dev_priv,
if (IS_I845G(dev_priv) || IS_I865G(dev_priv)) {
cursor->max_stride = i845_cursor_max_stride;
- cursor->update_plane = i845_update_cursor;
- cursor->disable_plane = i845_disable_cursor;
+ cursor->update_arm = i845_cursor_update_arm;
+ cursor->disable_arm = i845_cursor_disable_arm;
cursor->get_hw_state = i845_cursor_get_hw_state;
cursor->check_plane = i845_check_cursor;
} else {
cursor->max_stride = i9xx_cursor_max_stride;
- cursor->update_plane = i9xx_update_cursor;
- cursor->disable_plane = i9xx_disable_cursor;
+ cursor->update_arm = i9xx_cursor_update_arm;
+ cursor->disable_arm = i9xx_cursor_disable_arm;
cursor->get_hw_state = i9xx_cursor_get_hw_state;
cursor->check_plane = i9xx_check_cursor;
}
@@ -784,13 +786,18 @@ intel_cursor_plane_create(struct drm_i915_private *dev_priv,
if (IS_I845G(dev_priv) || IS_I865G(dev_priv) || HAS_CUR_FBC(dev_priv))
cursor->cursor.size = ~0;
+ modifiers = intel_fb_plane_get_modifiers(dev_priv, INTEL_PLANE_CAP_NONE);
+
ret = drm_universal_plane_init(&dev_priv->drm, &cursor->base,
0, &intel_cursor_plane_funcs,
intel_cursor_formats,
ARRAY_SIZE(intel_cursor_formats),
- cursor_format_modifiers,
+ modifiers,
DRM_PLANE_TYPE_CURSOR,
"cursor %c", pipe_name(pipe));
+
+ kfree(modifiers);
+
if (ret)
goto fail;
diff --git a/drivers/gpu/drm/i915/display/intel_ddi.c b/drivers/gpu/drm/i915/display/intel_ddi.c
index cfb567d..9c9d574 100644
--- a/drivers/gpu/drm/i915/display/intel_ddi.c
+++ b/drivers/gpu/drm/i915/display/intel_ddi.c
@@ -25,6 +25,7 @@
*
*/
+#include <drm/drm_privacy_screen_consumer.h>
#include <drm/drm_scdc_helper.h>
#include "i915_drv.h"
@@ -321,10 +322,11 @@ static void ddi_dotclock_get(struct intel_crtc_state *pipe_config)
{
int dotclock;
+ /* CRT dotclock is determined via other means */
if (pipe_config->has_pch_encoder)
- dotclock = intel_dotclock_calculate(pipe_config->port_clock,
- &pipe_config->fdi_m_n);
- else if (intel_crtc_has_dp_encoder(pipe_config))
+ return;
+
+ if (intel_crtc_has_dp_encoder(pipe_config))
dotclock = intel_dotclock_calculate(pipe_config->port_clock,
&pipe_config->dp_m_n);
else if (pipe_config->has_hdmi_sink && pipe_config->pipe_bpp > 24)
@@ -1039,7 +1041,6 @@ static void icl_ddi_combo_vswing_program(struct intel_encoder *encoder,
const struct intel_crtc_state *crtc_state)
{
struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
- int level = intel_ddi_level(encoder, crtc_state, 0);
const struct intel_ddi_buf_trans *trans;
enum phy phy = intel_port_to_phy(dev_priv, encoder->port);
int n_entries, ln;
@@ -1068,32 +1069,36 @@ static void icl_ddi_combo_vswing_program(struct intel_encoder *encoder,
intel_de_write(dev_priv, ICL_PORT_TX_DW5_GRP(phy), val);
/* Program PORT_TX_DW2 */
- val = intel_de_read(dev_priv, ICL_PORT_TX_DW2_LN(0, phy));
- val &= ~(SWING_SEL_LOWER_MASK | SWING_SEL_UPPER_MASK |
- RCOMP_SCALAR_MASK);
- val |= SWING_SEL_UPPER(trans->entries[level].icl.dw2_swing_sel);
- val |= SWING_SEL_LOWER(trans->entries[level].icl.dw2_swing_sel);
- /* Program Rcomp scalar for every table entry */
- val |= RCOMP_SCALAR(0x98);
- intel_de_write(dev_priv, ICL_PORT_TX_DW2_GRP(phy), val);
+ for (ln = 0; ln < 4; ln++) {
+ int level = intel_ddi_level(encoder, crtc_state, ln);
+
+ intel_de_rmw(dev_priv, ICL_PORT_TX_DW2_LN(ln, phy),
+ SWING_SEL_UPPER_MASK | SWING_SEL_LOWER_MASK | RCOMP_SCALAR_MASK,
+ SWING_SEL_UPPER(trans->entries[level].icl.dw2_swing_sel) |
+ SWING_SEL_LOWER(trans->entries[level].icl.dw2_swing_sel) |
+ RCOMP_SCALAR(0x98));
+ }
/* Program PORT_TX_DW4 */
/* We cannot write to GRP. It would overwrite individual loadgen. */
for (ln = 0; ln < 4; ln++) {
- val = intel_de_read(dev_priv, ICL_PORT_TX_DW4_LN(ln, phy));
- val &= ~(POST_CURSOR_1_MASK | POST_CURSOR_2_MASK |
- CURSOR_COEFF_MASK);
- val |= POST_CURSOR_1(trans->entries[level].icl.dw4_post_cursor_1);
- val |= POST_CURSOR_2(trans->entries[level].icl.dw4_post_cursor_2);
- val |= CURSOR_COEFF(trans->entries[level].icl.dw4_cursor_coeff);
- intel_de_write(dev_priv, ICL_PORT_TX_DW4_LN(ln, phy), val);
+ int level = intel_ddi_level(encoder, crtc_state, ln);
+
+ intel_de_rmw(dev_priv, ICL_PORT_TX_DW4_LN(ln, phy),
+ POST_CURSOR_1_MASK | POST_CURSOR_2_MASK | CURSOR_COEFF_MASK,
+ POST_CURSOR_1(trans->entries[level].icl.dw4_post_cursor_1) |
+ POST_CURSOR_2(trans->entries[level].icl.dw4_post_cursor_2) |
+ CURSOR_COEFF(trans->entries[level].icl.dw4_cursor_coeff));
}
/* Program PORT_TX_DW7 */
- val = intel_de_read(dev_priv, ICL_PORT_TX_DW7_LN(0, phy));
- val &= ~N_SCALAR_MASK;
- val |= N_SCALAR(trans->entries[level].icl.dw7_n_scalar);
- intel_de_write(dev_priv, ICL_PORT_TX_DW7_GRP(phy), val);
+ for (ln = 0; ln < 4; ln++) {
+ int level = intel_ddi_level(encoder, crtc_state, ln);
+
+ intel_de_rmw(dev_priv, ICL_PORT_TX_DW7_LN(ln, phy),
+ N_SCALAR_MASK,
+ N_SCALAR(trans->entries[level].icl.dw7_n_scalar));
+ }
}
static void icl_combo_phy_set_signal_levels(struct intel_encoder *encoder,
@@ -1124,16 +1129,14 @@ static void icl_combo_phy_set_signal_levels(struct intel_encoder *encoder,
* > 6 GHz (LN0=0, LN1=0, LN2=0, LN3=0)
*/
for (ln = 0; ln < 4; ln++) {
- val = intel_de_read(dev_priv, ICL_PORT_TX_DW4_LN(ln, phy));
- val &= ~LOADGEN_SELECT;
- val |= icl_combo_phy_loadgen_select(crtc_state, ln);
- intel_de_write(dev_priv, ICL_PORT_TX_DW4_LN(ln, phy), val);
+ intel_de_rmw(dev_priv, ICL_PORT_TX_DW4_LN(ln, phy),
+ LOADGEN_SELECT,
+ icl_combo_phy_loadgen_select(crtc_state, ln));
}
/* 3. Set PORT_CL_DW5 SUS Clock Config to 11b */
- val = intel_de_read(dev_priv, ICL_PORT_CL_DW5(phy));
- val |= SUS_CLOCK_CONFIG;
- intel_de_write(dev_priv, ICL_PORT_CL_DW5(phy), val);
+ intel_de_rmw(dev_priv, ICL_PORT_CL_DW5(phy),
+ 0, SUS_CLOCK_CONFIG);
/* 4. Clear training enable to change swing values */
val = intel_de_read(dev_priv, ICL_PORT_TX_DW5_LN(0, phy));
@@ -1154,10 +1157,8 @@ static void icl_mg_phy_set_signal_levels(struct intel_encoder *encoder,
{
struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
enum tc_port tc_port = intel_port_to_tc(dev_priv, encoder->port);
- int level = intel_ddi_level(encoder, crtc_state, 0);
const struct intel_ddi_buf_trans *trans;
int n_entries, ln;
- u32 val;
if (intel_tc_port_in_tbt_alt_mode(enc_to_dig_port(encoder)))
return;
@@ -1166,53 +1167,51 @@ static void icl_mg_phy_set_signal_levels(struct intel_encoder *encoder,
if (drm_WARN_ON_ONCE(&dev_priv->drm, !trans))
return;
- /* Set MG_TX_LINK_PARAMS cri_use_fs32 to 0. */
for (ln = 0; ln < 2; ln++) {
- val = intel_de_read(dev_priv, MG_TX1_LINK_PARAMS(ln, tc_port));
- val &= ~CRI_USE_FS32;
- intel_de_write(dev_priv, MG_TX1_LINK_PARAMS(ln, tc_port), val);
-
- val = intel_de_read(dev_priv, MG_TX2_LINK_PARAMS(ln, tc_port));
- val &= ~CRI_USE_FS32;
- intel_de_write(dev_priv, MG_TX2_LINK_PARAMS(ln, tc_port), val);
+ intel_de_rmw(dev_priv, MG_TX1_LINK_PARAMS(ln, tc_port),
+ CRI_USE_FS32, 0);
+ intel_de_rmw(dev_priv, MG_TX2_LINK_PARAMS(ln, tc_port),
+ CRI_USE_FS32, 0);
}
/* Program MG_TX_SWINGCTRL with values from vswing table */
for (ln = 0; ln < 2; ln++) {
- val = intel_de_read(dev_priv, MG_TX1_SWINGCTRL(ln, tc_port));
- val &= ~CRI_TXDEEMPH_OVERRIDE_17_12_MASK;
- val |= CRI_TXDEEMPH_OVERRIDE_17_12(
- trans->entries[level].mg.cri_txdeemph_override_17_12);
- intel_de_write(dev_priv, MG_TX1_SWINGCTRL(ln, tc_port), val);
+ int level;
- val = intel_de_read(dev_priv, MG_TX2_SWINGCTRL(ln, tc_port));
- val &= ~CRI_TXDEEMPH_OVERRIDE_17_12_MASK;
- val |= CRI_TXDEEMPH_OVERRIDE_17_12(
- trans->entries[level].mg.cri_txdeemph_override_17_12);
- intel_de_write(dev_priv, MG_TX2_SWINGCTRL(ln, tc_port), val);
+ level = intel_ddi_level(encoder, crtc_state, 2*ln+0);
+
+ intel_de_rmw(dev_priv, MG_TX1_SWINGCTRL(ln, tc_port),
+ CRI_TXDEEMPH_OVERRIDE_17_12_MASK,
+ CRI_TXDEEMPH_OVERRIDE_17_12(trans->entries[level].mg.cri_txdeemph_override_17_12));
+
+ level = intel_ddi_level(encoder, crtc_state, 2*ln+1);
+
+ intel_de_rmw(dev_priv, MG_TX2_SWINGCTRL(ln, tc_port),
+ CRI_TXDEEMPH_OVERRIDE_17_12_MASK,
+ CRI_TXDEEMPH_OVERRIDE_17_12(trans->entries[level].mg.cri_txdeemph_override_17_12));
}
/* Program MG_TX_DRVCTRL with values from vswing table */
for (ln = 0; ln < 2; ln++) {
- val = intel_de_read(dev_priv, MG_TX1_DRVCTRL(ln, tc_port));
- val &= ~(CRI_TXDEEMPH_OVERRIDE_11_6_MASK |
- CRI_TXDEEMPH_OVERRIDE_5_0_MASK);
- val |= CRI_TXDEEMPH_OVERRIDE_5_0(
- trans->entries[level].mg.cri_txdeemph_override_5_0) |
- CRI_TXDEEMPH_OVERRIDE_11_6(
- trans->entries[level].mg.cri_txdeemph_override_11_6) |
- CRI_TXDEEMPH_OVERRIDE_EN;
- intel_de_write(dev_priv, MG_TX1_DRVCTRL(ln, tc_port), val);
+ int level;
- val = intel_de_read(dev_priv, MG_TX2_DRVCTRL(ln, tc_port));
- val &= ~(CRI_TXDEEMPH_OVERRIDE_11_6_MASK |
- CRI_TXDEEMPH_OVERRIDE_5_0_MASK);
- val |= CRI_TXDEEMPH_OVERRIDE_5_0(
- trans->entries[level].mg.cri_txdeemph_override_5_0) |
- CRI_TXDEEMPH_OVERRIDE_11_6(
- trans->entries[level].mg.cri_txdeemph_override_11_6) |
- CRI_TXDEEMPH_OVERRIDE_EN;
- intel_de_write(dev_priv, MG_TX2_DRVCTRL(ln, tc_port), val);
+ level = intel_ddi_level(encoder, crtc_state, 2*ln+0);
+
+ intel_de_rmw(dev_priv, MG_TX1_DRVCTRL(ln, tc_port),
+ CRI_TXDEEMPH_OVERRIDE_11_6_MASK |
+ CRI_TXDEEMPH_OVERRIDE_5_0_MASK,
+ CRI_TXDEEMPH_OVERRIDE_11_6(trans->entries[level].mg.cri_txdeemph_override_11_6) |
+ CRI_TXDEEMPH_OVERRIDE_5_0(trans->entries[level].mg.cri_txdeemph_override_5_0) |
+ CRI_TXDEEMPH_OVERRIDE_EN);
+
+ level = intel_ddi_level(encoder, crtc_state, 2*ln+1);
+
+ intel_de_rmw(dev_priv, MG_TX2_DRVCTRL(ln, tc_port),
+ CRI_TXDEEMPH_OVERRIDE_11_6_MASK |
+ CRI_TXDEEMPH_OVERRIDE_5_0_MASK,
+ CRI_TXDEEMPH_OVERRIDE_11_6(trans->entries[level].mg.cri_txdeemph_override_11_6) |
+ CRI_TXDEEMPH_OVERRIDE_5_0(trans->entries[level].mg.cri_txdeemph_override_5_0) |
+ CRI_TXDEEMPH_OVERRIDE_EN);
/* FIXME: Program CRI_LOADGEN_SEL after the spec is updated */
}
@@ -1223,50 +1222,34 @@ static void icl_mg_phy_set_signal_levels(struct intel_encoder *encoder,
* values from table for which TX1 and TX2 enabled.
*/
for (ln = 0; ln < 2; ln++) {
- val = intel_de_read(dev_priv, MG_CLKHUB(ln, tc_port));
- if (crtc_state->port_clock < 300000)
- val |= CFG_LOW_RATE_LKREN_EN;
- else
- val &= ~CFG_LOW_RATE_LKREN_EN;
- intel_de_write(dev_priv, MG_CLKHUB(ln, tc_port), val);
+ intel_de_rmw(dev_priv, MG_CLKHUB(ln, tc_port),
+ CFG_LOW_RATE_LKREN_EN,
+ crtc_state->port_clock < 300000 ? CFG_LOW_RATE_LKREN_EN : 0);
}
/* Program the MG_TX_DCC<LN, port being used> based on the link frequency */
for (ln = 0; ln < 2; ln++) {
- val = intel_de_read(dev_priv, MG_TX1_DCC(ln, tc_port));
- val &= ~CFG_AMI_CK_DIV_OVERRIDE_VAL_MASK;
- if (crtc_state->port_clock <= 500000) {
- val &= ~CFG_AMI_CK_DIV_OVERRIDE_EN;
- } else {
- val |= CFG_AMI_CK_DIV_OVERRIDE_EN |
- CFG_AMI_CK_DIV_OVERRIDE_VAL(1);
- }
- intel_de_write(dev_priv, MG_TX1_DCC(ln, tc_port), val);
+ intel_de_rmw(dev_priv, MG_TX1_DCC(ln, tc_port),
+ CFG_AMI_CK_DIV_OVERRIDE_VAL_MASK |
+ CFG_AMI_CK_DIV_OVERRIDE_EN,
+ crtc_state->port_clock > 500000 ?
+ CFG_AMI_CK_DIV_OVERRIDE_VAL(1) |
+ CFG_AMI_CK_DIV_OVERRIDE_EN : 0);
- val = intel_de_read(dev_priv, MG_TX2_DCC(ln, tc_port));
- val &= ~CFG_AMI_CK_DIV_OVERRIDE_VAL_MASK;
- if (crtc_state->port_clock <= 500000) {
- val &= ~CFG_AMI_CK_DIV_OVERRIDE_EN;
- } else {
- val |= CFG_AMI_CK_DIV_OVERRIDE_EN |
- CFG_AMI_CK_DIV_OVERRIDE_VAL(1);
- }
- intel_de_write(dev_priv, MG_TX2_DCC(ln, tc_port), val);
+ intel_de_rmw(dev_priv, MG_TX2_DCC(ln, tc_port),
+ CFG_AMI_CK_DIV_OVERRIDE_VAL_MASK |
+ CFG_AMI_CK_DIV_OVERRIDE_EN,
+ crtc_state->port_clock > 500000 ?
+ CFG_AMI_CK_DIV_OVERRIDE_VAL(1) |
+ CFG_AMI_CK_DIV_OVERRIDE_EN : 0);
}
/* Program MG_TX_PISO_READLOAD with values from vswing table */
for (ln = 0; ln < 2; ln++) {
- val = intel_de_read(dev_priv,
- MG_TX1_PISO_READLOAD(ln, tc_port));
- val |= CRI_CALCINIT;
- intel_de_write(dev_priv, MG_TX1_PISO_READLOAD(ln, tc_port),
- val);
-
- val = intel_de_read(dev_priv,
- MG_TX2_PISO_READLOAD(ln, tc_port));
- val |= CRI_CALCINIT;
- intel_de_write(dev_priv, MG_TX2_PISO_READLOAD(ln, tc_port),
- val);
+ intel_de_rmw(dev_priv, MG_TX1_PISO_READLOAD(ln, tc_port),
+ 0, CRI_CALCINIT);
+ intel_de_rmw(dev_priv, MG_TX2_PISO_READLOAD(ln, tc_port),
+ 0, CRI_CALCINIT);
}
}
@@ -1275,9 +1258,7 @@ static void tgl_dkl_phy_set_signal_levels(struct intel_encoder *encoder,
{
struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
enum tc_port tc_port = intel_port_to_tc(dev_priv, encoder->port);
- int level = intel_ddi_level(encoder, crtc_state, 0);
const struct intel_ddi_buf_trans *trans;
- u32 val, dpcnt_mask, dpcnt_val;
int n_entries, ln;
if (intel_tc_port_in_tbt_alt_mode(enc_to_dig_port(encoder)))
@@ -1287,33 +1268,36 @@ static void tgl_dkl_phy_set_signal_levels(struct intel_encoder *encoder,
if (drm_WARN_ON_ONCE(&dev_priv->drm, !trans))
return;
- dpcnt_mask = (DKL_TX_PRESHOOT_COEFF_MASK |
- DKL_TX_DE_EMPAHSIS_COEFF_MASK |
- DKL_TX_VSWING_CONTROL_MASK);
- dpcnt_val = DKL_TX_VSWING_CONTROL(trans->entries[level].dkl.vswing);
- dpcnt_val |= DKL_TX_DE_EMPHASIS_COEFF(trans->entries[level].dkl.de_emphasis);
- dpcnt_val |= DKL_TX_PRESHOOT_COEFF(trans->entries[level].dkl.preshoot);
-
for (ln = 0; ln < 2; ln++) {
+ int level;
+
intel_de_write(dev_priv, HIP_INDEX_REG(tc_port),
HIP_INDEX_VAL(tc_port, ln));
intel_de_write(dev_priv, DKL_TX_PMD_LANE_SUS(tc_port), 0);
- /* All the registers are RMW */
- val = intel_de_read(dev_priv, DKL_TX_DPCNTL0(tc_port));
- val &= ~dpcnt_mask;
- val |= dpcnt_val;
- intel_de_write(dev_priv, DKL_TX_DPCNTL0(tc_port), val);
+ level = intel_ddi_level(encoder, crtc_state, 2*ln+0);
- val = intel_de_read(dev_priv, DKL_TX_DPCNTL1(tc_port));
- val &= ~dpcnt_mask;
- val |= dpcnt_val;
- intel_de_write(dev_priv, DKL_TX_DPCNTL1(tc_port), val);
+ intel_de_rmw(dev_priv, DKL_TX_DPCNTL0(tc_port),
+ DKL_TX_PRESHOOT_COEFF_MASK |
+ DKL_TX_DE_EMPAHSIS_COEFF_MASK |
+ DKL_TX_VSWING_CONTROL_MASK,
+ DKL_TX_PRESHOOT_COEFF(trans->entries[level].dkl.preshoot) |
+ DKL_TX_DE_EMPHASIS_COEFF(trans->entries[level].dkl.de_emphasis) |
+ DKL_TX_VSWING_CONTROL(trans->entries[level].dkl.vswing));
- val = intel_de_read(dev_priv, DKL_TX_DPCNTL2(tc_port));
- val &= ~DKL_TX_DP20BITMODE;
- intel_de_write(dev_priv, DKL_TX_DPCNTL2(tc_port), val);
+ level = intel_ddi_level(encoder, crtc_state, 2*ln+1);
+
+ intel_de_rmw(dev_priv, DKL_TX_DPCNTL1(tc_port),
+ DKL_TX_PRESHOOT_COEFF_MASK |
+ DKL_TX_DE_EMPAHSIS_COEFF_MASK |
+ DKL_TX_VSWING_CONTROL_MASK,
+ DKL_TX_PRESHOOT_COEFF(trans->entries[level].dkl.preshoot) |
+ DKL_TX_DE_EMPHASIS_COEFF(trans->entries[level].dkl.de_emphasis) |
+ DKL_TX_VSWING_CONTROL(trans->entries[level].dkl.vswing));
+
+ intel_de_rmw(dev_priv, DKL_TX_DPCNTL2(tc_port),
+ DKL_TX_DP20BITMODE, 0);
}
}
@@ -1938,7 +1922,7 @@ void intel_ddi_enable_clock(struct intel_encoder *encoder,
encoder->enable_clock(encoder, crtc_state);
}
-static void intel_ddi_disable_clock(struct intel_encoder *encoder)
+void intel_ddi_disable_clock(struct intel_encoder *encoder)
{
if (encoder->disable_clock)
encoder->disable_clock(encoder);
@@ -2385,7 +2369,10 @@ static void dg2_ddi_pre_enable_dp(struct intel_atomic_state *state,
/* 5.k Configure and enable FEC if needed */
intel_ddi_enable_fec(encoder, crtc_state);
- intel_dsc_enable(encoder, crtc_state);
+
+ intel_dsc_dp_pps_write(encoder, crtc_state);
+
+ intel_dsc_enable(crtc_state);
}
static void tgl_ddi_pre_enable_dp(struct intel_atomic_state *state,
@@ -2519,8 +2506,11 @@ static void tgl_ddi_pre_enable_dp(struct intel_atomic_state *state,
/* 7.l Configure and enable FEC if needed */
intel_ddi_enable_fec(encoder, crtc_state);
+
+ intel_dsc_dp_pps_write(encoder, crtc_state);
+
if (!crtc_state->bigjoiner)
- intel_dsc_enable(encoder, crtc_state);
+ intel_dsc_enable(crtc_state);
}
static void hsw_ddi_pre_enable_dp(struct intel_atomic_state *state,
@@ -2585,8 +2575,10 @@ static void hsw_ddi_pre_enable_dp(struct intel_atomic_state *state,
if (!is_mst)
intel_ddi_enable_pipe_clock(encoder, crtc_state);
+ intel_dsc_dp_pps_write(encoder, crtc_state);
+
if (!crtc_state->bigjoiner)
- intel_dsc_enable(encoder, crtc_state);
+ intel_dsc_enable(crtc_state);
}
static void intel_ddi_pre_enable_dp(struct intel_atomic_state *state,
@@ -2824,12 +2816,10 @@ static void intel_ddi_post_disable(struct intel_atomic_state *state,
}
if (old_crtc_state->bigjoiner_linked_crtc) {
- struct intel_atomic_state *state =
- to_intel_atomic_state(old_crtc_state->uapi.state);
- struct intel_crtc *slave =
+ struct intel_crtc *slave_crtc =
old_crtc_state->bigjoiner_linked_crtc;
const struct intel_crtc_state *old_slave_crtc_state =
- intel_atomic_get_old_crtc_state(state, slave);
+ intel_atomic_get_old_crtc_state(state, slave_crtc);
intel_crtc_vblank_off(old_slave_crtc_state);
@@ -2866,41 +2856,6 @@ static void intel_ddi_post_disable(struct intel_atomic_state *state,
intel_tc_port_put_link(dig_port);
}
-void intel_ddi_fdi_post_disable(struct intel_atomic_state *state,
- struct intel_encoder *encoder,
- const struct intel_crtc_state *old_crtc_state,
- const struct drm_connector_state *old_conn_state)
-{
- struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
- u32 val;
-
- /*
- * Bspec lists this as both step 13 (before DDI_BUF_CTL disable)
- * and step 18 (after clearing PORT_CLK_SEL). Based on a BUN,
- * step 13 is the correct place for it. Step 18 is where it was
- * originally before the BUN.
- */
- val = intel_de_read(dev_priv, FDI_RX_CTL(PIPE_A));
- val &= ~FDI_RX_ENABLE;
- intel_de_write(dev_priv, FDI_RX_CTL(PIPE_A), val);
-
- intel_disable_ddi_buf(encoder, old_crtc_state);
- intel_ddi_disable_clock(encoder);
-
- val = intel_de_read(dev_priv, FDI_RX_MISC(PIPE_A));
- val &= ~(FDI_RX_PWRDN_LANE1_MASK | FDI_RX_PWRDN_LANE0_MASK);
- val |= FDI_RX_PWRDN_LANE1_VAL(2) | FDI_RX_PWRDN_LANE0_VAL(2);
- intel_de_write(dev_priv, FDI_RX_MISC(PIPE_A), val);
-
- val = intel_de_read(dev_priv, FDI_RX_CTL(PIPE_A));
- val &= ~FDI_PCDCLK;
- intel_de_write(dev_priv, FDI_RX_CTL(PIPE_A), val);
-
- val = intel_de_read(dev_priv, FDI_RX_CTL(PIPE_A));
- val &= ~FDI_RX_PLL_ENABLE;
- intel_de_write(dev_priv, FDI_RX_CTL(PIPE_A), val);
-}
-
static void trans_port_sync_stop_link_train(struct intel_atomic_state *state,
struct intel_encoder *encoder,
const struct intel_crtc_state *crtc_state)
@@ -2951,6 +2906,7 @@ static void intel_enable_ddi_dp(struct intel_atomic_state *state,
if (port == PORT_A && DISPLAY_VER(dev_priv) < 9)
intel_dp_stop_link_train(intel_dp, crtc_state);
+ drm_connector_update_privacy_screen(conn_state);
intel_edp_backlight_on(crtc_state, conn_state);
if (!dig_port->lspcon.active || dig_port->dp.has_hdmi_sink)
@@ -3095,6 +3051,12 @@ static void intel_disable_ddi_dp(struct intel_atomic_state *state,
intel_dp->link_trained = false;
+ if (old_crtc_state->has_audio)
+ intel_audio_codec_disable(encoder,
+ old_crtc_state, old_conn_state);
+
+ intel_drrs_disable(intel_dp, old_crtc_state);
+ intel_psr_disable(intel_dp, old_crtc_state);
intel_edp_backlight_off(old_conn_state);
/* Disable the decompression in DP Sink */
intel_dp_sink_set_decompression_state(intel_dp, old_crtc_state,
@@ -3112,6 +3074,10 @@ static void intel_disable_ddi_hdmi(struct intel_atomic_state *state,
struct drm_i915_private *i915 = to_i915(encoder->base.dev);
struct drm_connector *connector = old_conn_state->connector;
+ if (old_crtc_state->has_audio)
+ intel_audio_codec_disable(encoder,
+ old_crtc_state, old_conn_state);
+
if (!intel_hdmi_handle_sink_scrambling(encoder, connector,
false, false))
drm_dbg_kms(&i915->drm,
@@ -3119,25 +3085,6 @@ static void intel_disable_ddi_hdmi(struct intel_atomic_state *state,
connector->base.id, connector->name);
}
-static void intel_pre_disable_ddi(struct intel_atomic_state *state,
- struct intel_encoder *encoder,
- const struct intel_crtc_state *old_crtc_state,
- const struct drm_connector_state *old_conn_state)
-{
- struct intel_dp *intel_dp;
-
- if (old_crtc_state->has_audio)
- intel_audio_codec_disable(encoder, old_crtc_state,
- old_conn_state);
-
- if (intel_crtc_has_type(old_crtc_state, INTEL_OUTPUT_HDMI))
- return;
-
- intel_dp = enc_to_intel_dp(encoder);
- intel_drrs_disable(intel_dp, old_crtc_state);
- intel_psr_disable(intel_dp, old_crtc_state);
-}
-
static void intel_disable_ddi(struct intel_atomic_state *state,
struct intel_encoder *encoder,
const struct intel_crtc_state *old_crtc_state,
@@ -3166,6 +3113,7 @@ static void intel_ddi_update_pipe_dp(struct intel_atomic_state *state,
intel_drrs_update(intel_dp, crtc_state);
intel_backlight_update(state, encoder, crtc_state, conn_state);
+ drm_connector_update_privacy_screen(conn_state);
}
void intel_ddi_update_pipe(struct intel_atomic_state *state,
@@ -3195,8 +3143,14 @@ intel_ddi_update_prepare(struct intel_atomic_state *state,
intel_tc_port_get_link(enc_to_dig_port(encoder),
required_lanes);
- if (crtc_state && crtc_state->hw.active)
+ if (crtc_state && crtc_state->hw.active) {
+ struct intel_crtc *slave_crtc = crtc_state->bigjoiner_linked_crtc;
+
intel_update_active_dpll(state, crtc, encoder);
+
+ if (slave_crtc)
+ intel_update_active_dpll(state, slave_crtc, encoder);
+ }
}
static void
@@ -3552,18 +3506,7 @@ static void intel_ddi_get_config(struct intel_encoder *encoder,
if (drm_WARN_ON(&dev_priv->drm, transcoder_is_dsi(cpu_transcoder)))
return;
- if (pipe_config->bigjoiner_slave) {
- /* read out pipe settings from master */
- enum transcoder save = pipe_config->cpu_transcoder;
-
- /* Our own transcoder needs to be disabled when reading it in intel_ddi_read_func_ctl() */
- WARN_ON(pipe_config->output_types);
- pipe_config->cpu_transcoder = (enum transcoder)pipe_config->bigjoiner_linked_crtc->pipe;
- intel_ddi_read_func_ctl(encoder, pipe_config);
- pipe_config->cpu_transcoder = save;
- } else {
- intel_ddi_read_func_ctl(encoder, pipe_config);
- }
+ intel_ddi_read_func_ctl(encoder, pipe_config);
intel_ddi_mso_get_config(encoder, pipe_config);
@@ -3591,8 +3534,7 @@ static void intel_ddi_get_config(struct intel_encoder *encoder,
dev_priv->vbt.edp.bpp = pipe_config->pipe_bpp;
}
- if (!pipe_config->bigjoiner_slave)
- ddi_dotclock_get(pipe_config);
+ ddi_dotclock_get(pipe_config);
if (IS_GEMINILAKE(dev_priv) || IS_BROXTON(dev_priv))
pipe_config->lane_lat_optim_mask =
@@ -3983,6 +3925,19 @@ intel_ddi_init_dp_connector(struct intel_digital_port *dig_port)
return NULL;
}
+ if (dig_port->base.type == INTEL_OUTPUT_EDP) {
+ struct drm_device *dev = dig_port->base.base.dev;
+ struct drm_privacy_screen *privacy_screen;
+
+ privacy_screen = drm_privacy_screen_get(dev->dev, NULL);
+ if (!IS_ERR(privacy_screen)) {
+ drm_connector_attach_privacy_screen_provider(&connector->base,
+ privacy_screen);
+ } else if (PTR_ERR(privacy_screen) != -ENODEV) {
+ drm_warn(dev, "Error getting privacy-screen\n");
+ }
+ }
+
return connector;
}
@@ -4472,7 +4427,6 @@ void intel_ddi_init(struct drm_i915_private *dev_priv, enum port port)
encoder->enable = intel_enable_ddi;
encoder->pre_pll_enable = intel_ddi_pre_pll_enable;
encoder->pre_enable = intel_ddi_pre_enable;
- encoder->pre_disable = intel_pre_disable_ddi;
encoder->disable = intel_disable_ddi;
encoder->post_disable = intel_ddi_post_disable;
encoder->update_pipe = intel_ddi_update_pipe;
diff --git a/drivers/gpu/drm/i915/display/intel_ddi.h b/drivers/gpu/drm/i915/display/intel_ddi.h
index d6971717..c2fea65 100644
--- a/drivers/gpu/drm/i915/display/intel_ddi.h
+++ b/drivers/gpu/drm/i915/display/intel_ddi.h
@@ -6,11 +6,11 @@
#ifndef __INTEL_DDI_H__
#define __INTEL_DDI_H__
-#include "intel_display.h"
#include "i915_reg.h"
struct drm_connector_state;
struct drm_i915_private;
+struct intel_atomic_state;
struct intel_connector;
struct intel_crtc;
struct intel_crtc_state;
@@ -18,6 +18,8 @@ struct intel_dp;
struct intel_dpll_hw_state;
struct intel_encoder;
struct intel_shared_dpll;
+enum pipe;
+enum port;
enum transcoder;
i915_reg_t dp_tp_ctl_reg(struct intel_encoder *encoder,
@@ -30,6 +32,7 @@ void intel_ddi_fdi_post_disable(struct intel_atomic_state *state,
const struct drm_connector_state *old_conn_state);
void intel_ddi_enable_clock(struct intel_encoder *encoder,
const struct intel_crtc_state *crtc_state);
+void intel_ddi_disable_clock(struct intel_encoder *encoder);
void intel_ddi_get_clock(struct intel_encoder *encoder,
struct intel_crtc_state *crtc_state,
struct intel_shared_dpll *pll);
diff --git a/drivers/gpu/drm/i915/display/intel_ddi_buf_trans.c b/drivers/gpu/drm/i915/display/intel_ddi_buf_trans.c
index 78cd8f7..1e689d5 100644
--- a/drivers/gpu/drm/i915/display/intel_ddi_buf_trans.c
+++ b/drivers/gpu/drm/i915/display/intel_ddi_buf_trans.c
@@ -1032,6 +1032,21 @@ bool is_hobl_buf_trans(const struct intel_ddi_buf_trans *table)
return table == &tgl_combo_phy_trans_edp_hbr2_hobl;
}
+static bool use_edp_hobl(struct intel_encoder *encoder)
+{
+ struct drm_i915_private *i915 = to_i915(encoder->base.dev);
+ struct intel_dp *intel_dp = enc_to_intel_dp(encoder);
+
+ return i915->vbt.edp.hobl && !intel_dp->hobl_failed;
+}
+
+static bool use_edp_low_vswing(struct intel_encoder *encoder)
+{
+ struct drm_i915_private *i915 = to_i915(encoder->base.dev);
+
+ return i915->vbt.edp.low_vswing;
+}
+
static const struct intel_ddi_buf_trans *
intel_get_buf_trans(const struct intel_ddi_buf_trans *trans, int *num_entries)
{
@@ -1057,14 +1072,12 @@ bdw_get_buf_trans(struct intel_encoder *encoder,
const struct intel_crtc_state *crtc_state,
int *n_entries)
{
- struct drm_i915_private *i915 = to_i915(encoder->base.dev);
-
if (intel_crtc_has_type(crtc_state, INTEL_OUTPUT_ANALOG))
return intel_get_buf_trans(&bdw_trans_fdi, n_entries);
else if (intel_crtc_has_type(crtc_state, INTEL_OUTPUT_HDMI))
return intel_get_buf_trans(&bdw_trans_hdmi, n_entries);
else if (intel_crtc_has_type(crtc_state, INTEL_OUTPUT_EDP) &&
- i915->vbt.edp.low_vswing)
+ use_edp_low_vswing(encoder))
return intel_get_buf_trans(&bdw_trans_edp, n_entries);
else
return intel_get_buf_trans(&bdw_trans_dp, n_entries);
@@ -1094,12 +1107,10 @@ skl_y_get_buf_trans(struct intel_encoder *encoder,
const struct intel_crtc_state *crtc_state,
int *n_entries)
{
- struct drm_i915_private *i915 = to_i915(encoder->base.dev);
-
if (intel_crtc_has_type(crtc_state, INTEL_OUTPUT_HDMI))
return intel_get_buf_trans(&skl_y_trans_hdmi, n_entries);
else if (intel_crtc_has_type(crtc_state, INTEL_OUTPUT_EDP) &&
- i915->vbt.edp.low_vswing)
+ use_edp_low_vswing(encoder))
return _skl_get_buf_trans_dp(encoder, &skl_y_trans_edp, n_entries);
else
return _skl_get_buf_trans_dp(encoder, &skl_y_trans_dp, n_entries);
@@ -1110,12 +1121,10 @@ skl_u_get_buf_trans(struct intel_encoder *encoder,
const struct intel_crtc_state *crtc_state,
int *n_entries)
{
- struct drm_i915_private *i915 = to_i915(encoder->base.dev);
-
if (intel_crtc_has_type(crtc_state, INTEL_OUTPUT_HDMI))
return intel_get_buf_trans(&skl_trans_hdmi, n_entries);
else if (intel_crtc_has_type(crtc_state, INTEL_OUTPUT_EDP) &&
- i915->vbt.edp.low_vswing)
+ use_edp_low_vswing(encoder))
return _skl_get_buf_trans_dp(encoder, &skl_u_trans_edp, n_entries);
else
return _skl_get_buf_trans_dp(encoder, &skl_u_trans_dp, n_entries);
@@ -1126,12 +1135,10 @@ skl_get_buf_trans(struct intel_encoder *encoder,
const struct intel_crtc_state *crtc_state,
int *n_entries)
{
- struct drm_i915_private *i915 = to_i915(encoder->base.dev);
-
if (intel_crtc_has_type(crtc_state, INTEL_OUTPUT_HDMI))
return intel_get_buf_trans(&skl_trans_hdmi, n_entries);
else if (intel_crtc_has_type(crtc_state, INTEL_OUTPUT_EDP) &&
- i915->vbt.edp.low_vswing)
+ use_edp_low_vswing(encoder))
return _skl_get_buf_trans_dp(encoder, &skl_trans_edp, n_entries);
else
return _skl_get_buf_trans_dp(encoder, &skl_trans_dp, n_entries);
@@ -1142,12 +1149,10 @@ kbl_y_get_buf_trans(struct intel_encoder *encoder,
const struct intel_crtc_state *crtc_state,
int *n_entries)
{
- struct drm_i915_private *i915 = to_i915(encoder->base.dev);
-
if (intel_crtc_has_type(crtc_state, INTEL_OUTPUT_HDMI))
return intel_get_buf_trans(&skl_y_trans_hdmi, n_entries);
else if (intel_crtc_has_type(crtc_state, INTEL_OUTPUT_EDP) &&
- i915->vbt.edp.low_vswing)
+ use_edp_low_vswing(encoder))
return _skl_get_buf_trans_dp(encoder, &skl_y_trans_edp, n_entries);
else
return _skl_get_buf_trans_dp(encoder, &kbl_y_trans_dp, n_entries);
@@ -1158,12 +1163,10 @@ kbl_u_get_buf_trans(struct intel_encoder *encoder,
const struct intel_crtc_state *crtc_state,
int *n_entries)
{
- struct drm_i915_private *i915 = to_i915(encoder->base.dev);
-
if (intel_crtc_has_type(crtc_state, INTEL_OUTPUT_HDMI))
return intel_get_buf_trans(&skl_trans_hdmi, n_entries);
else if (intel_crtc_has_type(crtc_state, INTEL_OUTPUT_EDP) &&
- i915->vbt.edp.low_vswing)
+ use_edp_low_vswing(encoder))
return _skl_get_buf_trans_dp(encoder, &skl_u_trans_edp, n_entries);
else
return _skl_get_buf_trans_dp(encoder, &kbl_u_trans_dp, n_entries);
@@ -1174,12 +1177,10 @@ kbl_get_buf_trans(struct intel_encoder *encoder,
const struct intel_crtc_state *crtc_state,
int *n_entries)
{
- struct drm_i915_private *i915 = to_i915(encoder->base.dev);
-
if (intel_crtc_has_type(crtc_state, INTEL_OUTPUT_HDMI))
return intel_get_buf_trans(&skl_trans_hdmi, n_entries);
else if (intel_crtc_has_type(crtc_state, INTEL_OUTPUT_EDP) &&
- i915->vbt.edp.low_vswing)
+ use_edp_low_vswing(encoder))
return _skl_get_buf_trans_dp(encoder, &skl_trans_edp, n_entries);
else
return _skl_get_buf_trans_dp(encoder, &kbl_trans_dp, n_entries);
@@ -1190,12 +1191,10 @@ bxt_get_buf_trans(struct intel_encoder *encoder,
const struct intel_crtc_state *crtc_state,
int *n_entries)
{
- struct drm_i915_private *i915 = to_i915(encoder->base.dev);
-
if (intel_crtc_has_type(crtc_state, INTEL_OUTPUT_HDMI))
return intel_get_buf_trans(&bxt_trans_hdmi, n_entries);
else if (intel_crtc_has_type(crtc_state, INTEL_OUTPUT_EDP) &&
- i915->vbt.edp.low_vswing)
+ use_edp_low_vswing(encoder))
return intel_get_buf_trans(&bxt_trans_edp, n_entries);
else
return intel_get_buf_trans(&bxt_trans_dp, n_entries);
@@ -1215,12 +1214,10 @@ icl_get_combo_buf_trans_edp(struct intel_encoder *encoder,
const struct intel_crtc_state *crtc_state,
int *n_entries)
{
- struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
-
if (crtc_state->port_clock > 540000) {
return intel_get_buf_trans(&icl_combo_phy_trans_dp_hbr2_edp_hbr3,
n_entries);
- } else if (dev_priv->vbt.edp.low_vswing) {
+ } else if (use_edp_low_vswing(encoder)) {
return intel_get_buf_trans(&icl_combo_phy_trans_edp_hbr2,
n_entries);
}
@@ -1282,12 +1279,10 @@ ehl_get_combo_buf_trans(struct intel_encoder *encoder,
const struct intel_crtc_state *crtc_state,
int *n_entries)
{
- struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
-
if (intel_crtc_has_type(crtc_state, INTEL_OUTPUT_HDMI))
return intel_get_buf_trans(&icl_combo_phy_trans_hdmi, n_entries);
else if (intel_crtc_has_type(crtc_state, INTEL_OUTPUT_EDP) &&
- dev_priv->vbt.edp.low_vswing)
+ use_edp_low_vswing(encoder))
return ehl_get_combo_buf_trans_edp(encoder, crtc_state, n_entries);
else
return intel_get_buf_trans(&ehl_combo_phy_trans_dp, n_entries);
@@ -1309,12 +1304,10 @@ jsl_get_combo_buf_trans(struct intel_encoder *encoder,
const struct intel_crtc_state *crtc_state,
int *n_entries)
{
- struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
-
if (intel_crtc_has_type(crtc_state, INTEL_OUTPUT_HDMI))
return intel_get_buf_trans(&icl_combo_phy_trans_hdmi, n_entries);
else if (intel_crtc_has_type(crtc_state, INTEL_OUTPUT_EDP) &&
- dev_priv->vbt.edp.low_vswing)
+ use_edp_low_vswing(encoder))
return jsl_get_combo_buf_trans_edp(encoder, crtc_state, n_entries);
else
return intel_get_buf_trans(&icl_combo_phy_trans_dp_hbr2_edp_hbr3, n_entries);
@@ -1346,16 +1339,13 @@ tgl_get_combo_buf_trans_edp(struct intel_encoder *encoder,
const struct intel_crtc_state *crtc_state,
int *n_entries)
{
- struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
- struct intel_dp *intel_dp = enc_to_intel_dp(encoder);
-
if (crtc_state->port_clock > 540000) {
return intel_get_buf_trans(&icl_combo_phy_trans_dp_hbr2_edp_hbr3,
n_entries);
- } else if (dev_priv->vbt.edp.hobl && !intel_dp->hobl_failed) {
+ } else if (use_edp_hobl(encoder)) {
return intel_get_buf_trans(&tgl_combo_phy_trans_edp_hbr2_hobl,
n_entries);
- } else if (dev_priv->vbt.edp.low_vswing) {
+ } else if (use_edp_low_vswing(encoder)) {
return intel_get_buf_trans(&icl_combo_phy_trans_edp_hbr2,
n_entries);
}
@@ -1394,16 +1384,13 @@ dg1_get_combo_buf_trans_edp(struct intel_encoder *encoder,
const struct intel_crtc_state *crtc_state,
int *n_entries)
{
- struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
- struct intel_dp *intel_dp = enc_to_intel_dp(encoder);
-
if (crtc_state->port_clock > 540000)
return intel_get_buf_trans(&icl_combo_phy_trans_dp_hbr2_edp_hbr3,
n_entries);
- else if (dev_priv->vbt.edp.hobl && !intel_dp->hobl_failed)
+ else if (use_edp_hobl(encoder))
return intel_get_buf_trans(&tgl_combo_phy_trans_edp_hbr2_hobl,
n_entries);
- else if (dev_priv->vbt.edp.low_vswing)
+ else if (use_edp_low_vswing(encoder))
return intel_get_buf_trans(&icl_combo_phy_trans_edp_hbr2,
n_entries);
else
@@ -1439,16 +1426,13 @@ rkl_get_combo_buf_trans_edp(struct intel_encoder *encoder,
const struct intel_crtc_state *crtc_state,
int *n_entries)
{
- struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
- struct intel_dp *intel_dp = enc_to_intel_dp(encoder);
-
if (crtc_state->port_clock > 540000) {
return intel_get_buf_trans(&icl_combo_phy_trans_dp_hbr2_edp_hbr3,
n_entries);
- } else if (dev_priv->vbt.edp.hobl && !intel_dp->hobl_failed) {
+ } else if (use_edp_hobl(encoder)) {
return intel_get_buf_trans(&tgl_combo_phy_trans_edp_hbr2_hobl,
n_entries);
- } else if (dev_priv->vbt.edp.low_vswing) {
+ } else if (use_edp_low_vswing(encoder)) {
return intel_get_buf_trans(&icl_combo_phy_trans_edp_hbr2,
n_entries);
}
@@ -1485,14 +1469,11 @@ adls_get_combo_buf_trans_edp(struct intel_encoder *encoder,
const struct intel_crtc_state *crtc_state,
int *n_entries)
{
- struct drm_i915_private *i915 = to_i915(encoder->base.dev);
- struct intel_dp *intel_dp = enc_to_intel_dp(encoder);
-
if (crtc_state->port_clock > 540000)
return intel_get_buf_trans(&adls_combo_phy_trans_edp_hbr3, n_entries);
- else if (i915->vbt.edp.hobl && !intel_dp->hobl_failed)
+ else if (use_edp_hobl(encoder))
return intel_get_buf_trans(&tgl_combo_phy_trans_edp_hbr2_hobl, n_entries);
- else if (i915->vbt.edp.low_vswing)
+ else if (use_edp_low_vswing(encoder))
return intel_get_buf_trans(&adls_combo_phy_trans_edp_hbr2, n_entries);
else
return adls_get_combo_buf_trans_dp(encoder, crtc_state, n_entries);
@@ -1527,16 +1508,13 @@ adlp_get_combo_buf_trans_edp(struct intel_encoder *encoder,
const struct intel_crtc_state *crtc_state,
int *n_entries)
{
- struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
- struct intel_dp *intel_dp = enc_to_intel_dp(encoder);
-
if (crtc_state->port_clock > 540000) {
return intel_get_buf_trans(&adlp_combo_phy_trans_edp_hbr3,
n_entries);
- } else if (dev_priv->vbt.edp.hobl && !intel_dp->hobl_failed) {
+ } else if (use_edp_hobl(encoder)) {
return intel_get_buf_trans(&tgl_combo_phy_trans_edp_hbr2_hobl,
n_entries);
- } else if (dev_priv->vbt.edp.low_vswing) {
+ } else if (use_edp_low_vswing(encoder)) {
return intel_get_buf_trans(&adlp_combo_phy_trans_edp_up_to_hbr2,
n_entries);
}
diff --git a/drivers/gpu/drm/i915/display/intel_display.c b/drivers/gpu/drm/i915/display/intel_display.c
index ec403e4..bf7ce68 100644
--- a/drivers/gpu/drm/i915/display/intel_display.c
+++ b/drivers/gpu/drm/i915/display/intel_display.c
@@ -32,6 +32,7 @@
#include <linux/module.h>
#include <linux/dma-resv.h>
#include <linux/slab.h>
+#include <linux/vga_switcheroo.h>
#include <drm/drm_atomic.h>
#include <drm/drm_atomic_helper.h>
@@ -41,6 +42,7 @@
#include <drm/drm_edid.h>
#include <drm/drm_fourcc.h>
#include <drm/drm_plane_helper.h>
+#include <drm/drm_privacy_screen_consumer.h>
#include <drm/drm_probe_helper.h>
#include <drm/drm_rect.h>
@@ -70,11 +72,10 @@
#include "gt/gen8_ppgtt.h"
-#include "pxp/intel_pxp.h"
-
#include "g4x_dp.h"
#include "g4x_hdmi.h"
#include "i915_drv.h"
+#include "icl_dsi.h"
#include "intel_acpi.h"
#include "intel_atomic.h"
#include "intel_atomic_plane.h"
@@ -96,6 +97,8 @@
#include "intel_hotplug.h"
#include "intel_overlay.h"
#include "intel_panel.h"
+#include "intel_pch_display.h"
+#include "intel_pch_refclk.h"
#include "intel_pcode.h"
#include "intel_pipe_crc.h"
#include "intel_plane_initial.h"
@@ -103,19 +106,15 @@
#include "intel_pps.h"
#include "intel_psr.h"
#include "intel_quirks.h"
-#include "intel_sbi.h"
#include "intel_sprite.h"
#include "intel_tc.h"
#include "intel_vga.h"
#include "i9xx_plane.h"
#include "skl_scaler.h"
#include "skl_universal_plane.h"
+#include "vlv_dsi_pll.h"
#include "vlv_sideband.h"
-
-static void i9xx_crtc_clock_get(struct intel_crtc *crtc,
- struct intel_crtc_state *pipe_config);
-static void ilk_pch_clock_get(struct intel_crtc *crtc,
- struct intel_crtc_state *pipe_config);
+#include "vlv_dsi.h"
static void intel_set_transcoder_timings(const struct intel_crtc_state *crtc_state);
static void intel_set_pipe_src_size(const struct intel_crtc_state *crtc_state);
@@ -341,6 +340,14 @@ is_trans_port_sync_mode(const struct intel_crtc_state *crtc_state)
is_trans_port_sync_slave(crtc_state);
}
+static struct intel_crtc *intel_master_crtc(const struct intel_crtc_state *crtc_state)
+{
+ if (crtc_state->bigjoiner_slave)
+ return crtc_state->bigjoiner_linked_crtc;
+ else
+ return to_intel_crtc(crtc_state->uapi.crtc);
+}
+
static bool pipe_scanline_is_moving(struct drm_i915_private *dev_priv,
enum pipe pipe)
{
@@ -454,80 +461,6 @@ static void assert_planes_disabled(struct intel_crtc *crtc)
assert_plane_disabled(plane);
}
-void assert_pch_transcoder_disabled(struct drm_i915_private *dev_priv,
- enum pipe pipe)
-{
- u32 val;
- bool enabled;
-
- val = intel_de_read(dev_priv, PCH_TRANSCONF(pipe));
- enabled = !!(val & TRANS_ENABLE);
- I915_STATE_WARN(enabled,
- "transcoder assertion failed, should be off on pipe %c but is still active\n",
- pipe_name(pipe));
-}
-
-static void assert_pch_dp_disabled(struct drm_i915_private *dev_priv,
- enum pipe pipe, enum port port,
- i915_reg_t dp_reg)
-{
- enum pipe port_pipe;
- bool state;
-
- state = g4x_dp_port_enabled(dev_priv, dp_reg, port, &port_pipe);
-
- I915_STATE_WARN(state && port_pipe == pipe,
- "PCH DP %c enabled on transcoder %c, should be disabled\n",
- port_name(port), pipe_name(pipe));
-
- I915_STATE_WARN(HAS_PCH_IBX(dev_priv) && !state && port_pipe == PIPE_B,
- "IBX PCH DP %c still using transcoder B\n",
- port_name(port));
-}
-
-static void assert_pch_hdmi_disabled(struct drm_i915_private *dev_priv,
- enum pipe pipe, enum port port,
- i915_reg_t hdmi_reg)
-{
- enum pipe port_pipe;
- bool state;
-
- state = intel_sdvo_port_enabled(dev_priv, hdmi_reg, &port_pipe);
-
- I915_STATE_WARN(state && port_pipe == pipe,
- "PCH HDMI %c enabled on transcoder %c, should be disabled\n",
- port_name(port), pipe_name(pipe));
-
- I915_STATE_WARN(HAS_PCH_IBX(dev_priv) && !state && port_pipe == PIPE_B,
- "IBX PCH HDMI %c still using transcoder B\n",
- port_name(port));
-}
-
-static void assert_pch_ports_disabled(struct drm_i915_private *dev_priv,
- enum pipe pipe)
-{
- enum pipe port_pipe;
-
- assert_pch_dp_disabled(dev_priv, pipe, PORT_B, PCH_DP_B);
- assert_pch_dp_disabled(dev_priv, pipe, PORT_C, PCH_DP_C);
- assert_pch_dp_disabled(dev_priv, pipe, PORT_D, PCH_DP_D);
-
- I915_STATE_WARN(intel_crt_port_enabled(dev_priv, PCH_ADPA, &port_pipe) &&
- port_pipe == pipe,
- "PCH VGA enabled on transcoder %c, should be disabled\n",
- pipe_name(pipe));
-
- I915_STATE_WARN(intel_lvds_port_enabled(dev_priv, PCH_LVDS, &port_pipe) &&
- port_pipe == pipe,
- "PCH LVDS enabled on transcoder %c, should be disabled\n",
- pipe_name(pipe));
-
- /* PCH SDVOB multiplex with HDMIB */
- assert_pch_hdmi_disabled(dev_priv, pipe, PORT_B, PCH_HDMIB);
- assert_pch_hdmi_disabled(dev_priv, pipe, PORT_C, PCH_HDMIC);
- assert_pch_hdmi_disabled(dev_priv, pipe, PORT_D, PCH_HDMID);
-}
-
void vlv_wait_port_ready(struct drm_i915_private *dev_priv,
struct intel_digital_port *dig_port,
unsigned int expected_mask)
@@ -562,154 +495,6 @@ void vlv_wait_port_ready(struct drm_i915_private *dev_priv,
expected_mask);
}
-static void ilk_enable_pch_transcoder(const struct intel_crtc_state *crtc_state)
-{
- struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
- enum pipe pipe = crtc->pipe;
- i915_reg_t reg;
- u32 val, pipeconf_val;
-
- /* Make sure PCH DPLL is enabled */
- assert_shared_dpll_enabled(dev_priv, crtc_state->shared_dpll);
-
- /* FDI must be feeding us bits for PCH ports */
- assert_fdi_tx_enabled(dev_priv, pipe);
- assert_fdi_rx_enabled(dev_priv, pipe);
-
- if (HAS_PCH_CPT(dev_priv)) {
- reg = TRANS_CHICKEN2(pipe);
- val = intel_de_read(dev_priv, reg);
- /*
- * Workaround: Set the timing override bit
- * before enabling the pch transcoder.
- */
- val |= TRANS_CHICKEN2_TIMING_OVERRIDE;
- /* Configure frame start delay to match the CPU */
- val &= ~TRANS_CHICKEN2_FRAME_START_DELAY_MASK;
- val |= TRANS_CHICKEN2_FRAME_START_DELAY(dev_priv->framestart_delay - 1);
- intel_de_write(dev_priv, reg, val);
- }
-
- reg = PCH_TRANSCONF(pipe);
- val = intel_de_read(dev_priv, reg);
- pipeconf_val = intel_de_read(dev_priv, PIPECONF(pipe));
-
- if (HAS_PCH_IBX(dev_priv)) {
- /* Configure frame start delay to match the CPU */
- val &= ~TRANS_FRAME_START_DELAY_MASK;
- val |= TRANS_FRAME_START_DELAY(dev_priv->framestart_delay - 1);
-
- /*
- * Make the BPC in transcoder be consistent with
- * that in pipeconf reg. For HDMI we must use 8bpc
- * here for both 8bpc and 12bpc.
- */
- val &= ~PIPECONF_BPC_MASK;
- if (intel_crtc_has_type(crtc_state, INTEL_OUTPUT_HDMI))
- val |= PIPECONF_8BPC;
- else
- val |= pipeconf_val & PIPECONF_BPC_MASK;
- }
-
- val &= ~TRANS_INTERLACE_MASK;
- if ((pipeconf_val & PIPECONF_INTERLACE_MASK) == PIPECONF_INTERLACED_ILK) {
- if (HAS_PCH_IBX(dev_priv) &&
- intel_crtc_has_type(crtc_state, INTEL_OUTPUT_SDVO))
- val |= TRANS_LEGACY_INTERLACED_ILK;
- else
- val |= TRANS_INTERLACED;
- } else {
- val |= TRANS_PROGRESSIVE;
- }
-
- intel_de_write(dev_priv, reg, val | TRANS_ENABLE);
- if (intel_de_wait_for_set(dev_priv, reg, TRANS_STATE_ENABLE, 100))
- drm_err(&dev_priv->drm, "failed to enable transcoder %c\n",
- pipe_name(pipe));
-}
-
-static void lpt_enable_pch_transcoder(struct drm_i915_private *dev_priv,
- enum transcoder cpu_transcoder)
-{
- u32 val, pipeconf_val;
-
- /* FDI must be feeding us bits for PCH ports */
- assert_fdi_tx_enabled(dev_priv, (enum pipe) cpu_transcoder);
- assert_fdi_rx_enabled(dev_priv, PIPE_A);
-
- val = intel_de_read(dev_priv, TRANS_CHICKEN2(PIPE_A));
- /* Workaround: set timing override bit. */
- val |= TRANS_CHICKEN2_TIMING_OVERRIDE;
- /* Configure frame start delay to match the CPU */
- val &= ~TRANS_CHICKEN2_FRAME_START_DELAY_MASK;
- val |= TRANS_CHICKEN2_FRAME_START_DELAY(dev_priv->framestart_delay - 1);
- intel_de_write(dev_priv, TRANS_CHICKEN2(PIPE_A), val);
-
- val = TRANS_ENABLE;
- pipeconf_val = intel_de_read(dev_priv, PIPECONF(cpu_transcoder));
-
- if ((pipeconf_val & PIPECONF_INTERLACE_MASK_HSW) ==
- PIPECONF_INTERLACED_ILK)
- val |= TRANS_INTERLACED;
- else
- val |= TRANS_PROGRESSIVE;
-
- intel_de_write(dev_priv, LPT_TRANSCONF, val);
- if (intel_de_wait_for_set(dev_priv, LPT_TRANSCONF,
- TRANS_STATE_ENABLE, 100))
- drm_err(&dev_priv->drm, "Failed to enable PCH transcoder\n");
-}
-
-static void ilk_disable_pch_transcoder(struct drm_i915_private *dev_priv,
- enum pipe pipe)
-{
- i915_reg_t reg;
- u32 val;
-
- /* FDI relies on the transcoder */
- assert_fdi_tx_disabled(dev_priv, pipe);
- assert_fdi_rx_disabled(dev_priv, pipe);
-
- /* Ports must be off as well */
- assert_pch_ports_disabled(dev_priv, pipe);
-
- reg = PCH_TRANSCONF(pipe);
- val = intel_de_read(dev_priv, reg);
- val &= ~TRANS_ENABLE;
- intel_de_write(dev_priv, reg, val);
- /* wait for PCH transcoder off, transcoder state */
- if (intel_de_wait_for_clear(dev_priv, reg, TRANS_STATE_ENABLE, 50))
- drm_err(&dev_priv->drm, "failed to disable transcoder %c\n",
- pipe_name(pipe));
-
- if (HAS_PCH_CPT(dev_priv)) {
- /* Workaround: Clear the timing override chicken bit again. */
- reg = TRANS_CHICKEN2(pipe);
- val = intel_de_read(dev_priv, reg);
- val &= ~TRANS_CHICKEN2_TIMING_OVERRIDE;
- intel_de_write(dev_priv, reg, val);
- }
-}
-
-void lpt_disable_pch_transcoder(struct drm_i915_private *dev_priv)
-{
- u32 val;
-
- val = intel_de_read(dev_priv, LPT_TRANSCONF);
- val &= ~TRANS_ENABLE;
- intel_de_write(dev_priv, LPT_TRANSCONF, val);
- /* wait for PCH transcoder off, transcoder state */
- if (intel_de_wait_for_clear(dev_priv, LPT_TRANSCONF,
- TRANS_STATE_ENABLE, 50))
- drm_err(&dev_priv->drm, "Failed to disable PCH transcoder\n");
-
- /* Workaround: clear timing override bit. */
- val = intel_de_read(dev_priv, TRANS_CHICKEN2(PIPE_A));
- val &= ~TRANS_CHICKEN2_TIMING_OVERRIDE;
- intel_de_write(dev_priv, TRANS_CHICKEN2(PIPE_A), val);
-}
-
enum pipe intel_crtc_pch_transcoder(struct intel_crtc *crtc)
{
struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
@@ -823,14 +608,6 @@ void intel_disable_transcoder(const struct intel_crtc_state *old_crtc_state)
intel_wait_for_pipe_off(old_crtc_state);
}
-bool
-intel_format_info_is_yuv_semiplanar(const struct drm_format_info *info,
- u64 modifier)
-{
- return info->is_yuv &&
- info->num_planes == (is_ccs_modifier(modifier) ? 4 : 2);
-}
-
unsigned int intel_rotation_info_size(const struct intel_rotation_info *rot_info)
{
unsigned int size = 0;
@@ -850,7 +627,11 @@ unsigned int intel_remapped_info_size(const struct intel_remapped_info *rem_info
for (i = 0 ; i < ARRAY_SIZE(rem_info->plane); i++) {
unsigned int plane_size;
- plane_size = rem_info->plane[i].dst_stride * rem_info->plane[i].height;
+ if (rem_info->plane[i].linear)
+ plane_size = rem_info->plane[i].size;
+ else
+ plane_size = rem_info->plane[i].dst_stride * rem_info->plane[i].height;
+
if (plane_size == 0)
continue;
@@ -869,7 +650,7 @@ bool intel_plane_uses_fence(const struct intel_plane_state *plane_state)
struct drm_i915_private *dev_priv = to_i915(plane->base.dev);
return DISPLAY_VER(dev_priv) < 4 ||
- (plane->has_fbc &&
+ (plane->fbc &&
plane_state->view.gtt.type == I915_GGTT_VIEW_NORMAL);
}
@@ -885,7 +666,7 @@ u32 intel_fb_xy_to_linear(int x, int y,
{
const struct drm_framebuffer *fb = state->hw.fb;
unsigned int cpp = fb->format->cpp[color_plane];
- unsigned int pitch = state->view.color_plane[color_plane].stride;
+ unsigned int pitch = state->view.color_plane[color_plane].mapping_stride;
return y * pitch + x * cpp;
}
@@ -904,136 +685,6 @@ void intel_add_fb_offsets(int *x, int *y,
*y += state->view.color_plane[color_plane].y;
}
-/*
- * From the Sky Lake PRM:
- * "The Color Control Surface (CCS) contains the compression status of
- * the cache-line pairs. The compression state of the cache-line pair
- * is specified by 2 bits in the CCS. Each CCS cache-line represents
- * an area on the main surface of 16 x16 sets of 128 byte Y-tiled
- * cache-line-pairs. CCS is always Y tiled."
- *
- * Since cache line pairs refers to horizontally adjacent cache lines,
- * each cache line in the CCS corresponds to an area of 32x16 cache
- * lines on the main surface. Since each pixel is 4 bytes, this gives
- * us a ratio of one byte in the CCS for each 8x16 pixels in the
- * main surface.
- */
-static const struct drm_format_info skl_ccs_formats[] = {
- { .format = DRM_FORMAT_XRGB8888, .depth = 24, .num_planes = 2,
- .cpp = { 4, 1, }, .hsub = 8, .vsub = 16, },
- { .format = DRM_FORMAT_XBGR8888, .depth = 24, .num_planes = 2,
- .cpp = { 4, 1, }, .hsub = 8, .vsub = 16, },
- { .format = DRM_FORMAT_ARGB8888, .depth = 32, .num_planes = 2,
- .cpp = { 4, 1, }, .hsub = 8, .vsub = 16, .has_alpha = true, },
- { .format = DRM_FORMAT_ABGR8888, .depth = 32, .num_planes = 2,
- .cpp = { 4, 1, }, .hsub = 8, .vsub = 16, .has_alpha = true, },
-};
-
-/*
- * Gen-12 compression uses 4 bits of CCS data for each cache line pair in the
- * main surface. And each 64B CCS cache line represents an area of 4x1 Y-tiles
- * in the main surface. With 4 byte pixels and each Y-tile having dimensions of
- * 32x32 pixels, the ratio turns out to 1B in the CCS for every 2x32 pixels in
- * the main surface.
- */
-static const struct drm_format_info gen12_ccs_formats[] = {
- { .format = DRM_FORMAT_XRGB8888, .depth = 24, .num_planes = 2,
- .char_per_block = { 4, 1 }, .block_w = { 1, 2 }, .block_h = { 1, 1 },
- .hsub = 1, .vsub = 1, },
- { .format = DRM_FORMAT_XBGR8888, .depth = 24, .num_planes = 2,
- .char_per_block = { 4, 1 }, .block_w = { 1, 2 }, .block_h = { 1, 1 },
- .hsub = 1, .vsub = 1, },
- { .format = DRM_FORMAT_ARGB8888, .depth = 32, .num_planes = 2,
- .char_per_block = { 4, 1 }, .block_w = { 1, 2 }, .block_h = { 1, 1 },
- .hsub = 1, .vsub = 1, .has_alpha = true },
- { .format = DRM_FORMAT_ABGR8888, .depth = 32, .num_planes = 2,
- .char_per_block = { 4, 1 }, .block_w = { 1, 2 }, .block_h = { 1, 1 },
- .hsub = 1, .vsub = 1, .has_alpha = true },
- { .format = DRM_FORMAT_YUYV, .num_planes = 2,
- .char_per_block = { 2, 1 }, .block_w = { 1, 2 }, .block_h = { 1, 1 },
- .hsub = 2, .vsub = 1, .is_yuv = true },
- { .format = DRM_FORMAT_YVYU, .num_planes = 2,
- .char_per_block = { 2, 1 }, .block_w = { 1, 2 }, .block_h = { 1, 1 },
- .hsub = 2, .vsub = 1, .is_yuv = true },
- { .format = DRM_FORMAT_UYVY, .num_planes = 2,
- .char_per_block = { 2, 1 }, .block_w = { 1, 2 }, .block_h = { 1, 1 },
- .hsub = 2, .vsub = 1, .is_yuv = true },
- { .format = DRM_FORMAT_VYUY, .num_planes = 2,
- .char_per_block = { 2, 1 }, .block_w = { 1, 2 }, .block_h = { 1, 1 },
- .hsub = 2, .vsub = 1, .is_yuv = true },
- { .format = DRM_FORMAT_XYUV8888, .num_planes = 2,
- .char_per_block = { 4, 1 }, .block_w = { 1, 2 }, .block_h = { 1, 1 },
- .hsub = 1, .vsub = 1, .is_yuv = true },
- { .format = DRM_FORMAT_NV12, .num_planes = 4,
- .char_per_block = { 1, 2, 1, 1 }, .block_w = { 1, 1, 4, 4 }, .block_h = { 1, 1, 1, 1 },
- .hsub = 2, .vsub = 2, .is_yuv = true },
- { .format = DRM_FORMAT_P010, .num_planes = 4,
- .char_per_block = { 2, 4, 1, 1 }, .block_w = { 1, 1, 2, 2 }, .block_h = { 1, 1, 1, 1 },
- .hsub = 2, .vsub = 2, .is_yuv = true },
- { .format = DRM_FORMAT_P012, .num_planes = 4,
- .char_per_block = { 2, 4, 1, 1 }, .block_w = { 1, 1, 2, 2 }, .block_h = { 1, 1, 1, 1 },
- .hsub = 2, .vsub = 2, .is_yuv = true },
- { .format = DRM_FORMAT_P016, .num_planes = 4,
- .char_per_block = { 2, 4, 1, 1 }, .block_w = { 1, 1, 2, 2 }, .block_h = { 1, 1, 1, 1 },
- .hsub = 2, .vsub = 2, .is_yuv = true },
-};
-
-/*
- * Same as gen12_ccs_formats[] above, but with additional surface used
- * to pass Clear Color information in plane 2 with 64 bits of data.
- */
-static const struct drm_format_info gen12_ccs_cc_formats[] = {
- { .format = DRM_FORMAT_XRGB8888, .depth = 24, .num_planes = 3,
- .char_per_block = { 4, 1, 0 }, .block_w = { 1, 2, 2 }, .block_h = { 1, 1, 1 },
- .hsub = 1, .vsub = 1, },
- { .format = DRM_FORMAT_XBGR8888, .depth = 24, .num_planes = 3,
- .char_per_block = { 4, 1, 0 }, .block_w = { 1, 2, 2 }, .block_h = { 1, 1, 1 },
- .hsub = 1, .vsub = 1, },
- { .format = DRM_FORMAT_ARGB8888, .depth = 32, .num_planes = 3,
- .char_per_block = { 4, 1, 0 }, .block_w = { 1, 2, 2 }, .block_h = { 1, 1, 1 },
- .hsub = 1, .vsub = 1, .has_alpha = true },
- { .format = DRM_FORMAT_ABGR8888, .depth = 32, .num_planes = 3,
- .char_per_block = { 4, 1, 0 }, .block_w = { 1, 2, 2 }, .block_h = { 1, 1, 1 },
- .hsub = 1, .vsub = 1, .has_alpha = true },
-};
-
-static const struct drm_format_info *
-lookup_format_info(const struct drm_format_info formats[],
- int num_formats, u32 format)
-{
- int i;
-
- for (i = 0; i < num_formats; i++) {
- if (formats[i].format == format)
- return &formats[i];
- }
-
- return NULL;
-}
-
-static const struct drm_format_info *
-intel_get_format_info(const struct drm_mode_fb_cmd2 *cmd)
-{
- switch (cmd->modifier[0]) {
- case I915_FORMAT_MOD_Y_TILED_CCS:
- case I915_FORMAT_MOD_Yf_TILED_CCS:
- return lookup_format_info(skl_ccs_formats,
- ARRAY_SIZE(skl_ccs_formats),
- cmd->pixel_format);
- case I915_FORMAT_MOD_Y_TILED_GEN12_RC_CCS:
- case I915_FORMAT_MOD_Y_TILED_GEN12_MC_CCS:
- return lookup_format_info(gen12_ccs_formats,
- ARRAY_SIZE(gen12_ccs_formats),
- cmd->pixel_format);
- case I915_FORMAT_MOD_Y_TILED_GEN12_RC_CCS_CC:
- return lookup_format_info(gen12_ccs_cc_formats,
- ARRAY_SIZE(gen12_ccs_cc_formats),
- cmd->pixel_format);
- default:
- return NULL;
- }
-}
-
u32 intel_plane_fb_max_stride(struct drm_i915_private *dev_priv,
u32 pixel_format, u64 modifier)
{
@@ -1048,7 +699,7 @@ u32 intel_plane_fb_max_stride(struct drm_i915_private *dev_priv,
* the highest stride limits of them all,
* if in case pipe A is disabled, use the first pipe from pipe_mask.
*/
- crtc = intel_get_first_crtc(dev_priv);
+ crtc = intel_first_crtc(dev_priv);
if (!crtc)
return 0;
@@ -1126,7 +777,7 @@ void intel_plane_disable_noatomic(struct intel_crtc *crtc,
*/
if (HAS_GMCH(dev_priv) &&
intel_set_memory_cxsr(dev_priv, false))
- intel_wait_for_vblank(dev_priv, crtc->pipe);
+ intel_crtc_wait_for_next_vblank(crtc);
/*
* Gen2 reports pipe underruns whenever all planes are disabled.
@@ -1135,8 +786,8 @@ void intel_plane_disable_noatomic(struct intel_crtc *crtc,
if (DISPLAY_VER(dev_priv) == 2 && !crtc_state->active_planes)
intel_set_cpu_fifo_underrun_reporting(dev_priv, crtc->pipe, false);
- intel_disable_plane(plane, crtc_state);
- intel_wait_for_vblank(dev_priv, crtc->pipe);
+ intel_plane_disable_arm(plane, crtc_state);
+ intel_crtc_wait_for_next_vblank(crtc);
}
unsigned int
@@ -1192,7 +843,7 @@ __intel_display_resume(struct drm_device *dev,
static bool gpu_reset_clobbers_display(struct drm_i915_private *dev_priv)
{
return (INTEL_INFO(dev_priv)->gpu_reset_clobbers_display &&
- intel_has_gpu_reset(&dev_priv->gt));
+ intel_has_gpu_reset(to_gt(dev_priv)));
}
void intel_display_prepare_reset(struct drm_i915_private *dev_priv)
@@ -1211,14 +862,14 @@ void intel_display_prepare_reset(struct drm_i915_private *dev_priv)
return;
/* We have a modeset vs reset deadlock, defensively unbreak it. */
- set_bit(I915_RESET_MODESET, &dev_priv->gt.reset.flags);
+ set_bit(I915_RESET_MODESET, &to_gt(dev_priv)->reset.flags);
smp_mb__after_atomic();
- wake_up_bit(&dev_priv->gt.reset.flags, I915_RESET_MODESET);
+ wake_up_bit(&to_gt(dev_priv)->reset.flags, I915_RESET_MODESET);
if (atomic_read(&dev_priv->gpu_error.pending_fb_pin)) {
drm_dbg_kms(&dev_priv->drm,
"Modeset potentially stuck, unbreaking through wedging\n");
- intel_gt_set_wedged(&dev_priv->gt);
+ intel_gt_set_wedged(to_gt(dev_priv));
}
/*
@@ -1269,7 +920,7 @@ void intel_display_finish_reset(struct drm_i915_private *dev_priv)
return;
/* reset doesn't touch the display */
- if (!test_bit(I915_RESET_MODESET, &dev_priv->gt.reset.flags))
+ if (!test_bit(I915_RESET_MODESET, &to_gt(dev_priv)->reset.flags))
return;
state = fetch_and_zero(&dev_priv->modeset_restore_state);
@@ -1307,27 +958,7 @@ void intel_display_finish_reset(struct drm_i915_private *dev_priv)
drm_modeset_acquire_fini(ctx);
mutex_unlock(&dev->mode_config.mutex);
- clear_bit_unlock(I915_RESET_MODESET, &dev_priv->gt.reset.flags);
-}
-
-static bool underrun_recovery_supported(const struct intel_crtc_state *crtc_state)
-{
- if (crtc_state->pch_pfit.enabled &&
- (crtc_state->pipe_src_w > drm_rect_width(&crtc_state->pch_pfit.dst) ||
- crtc_state->pipe_src_h > drm_rect_height(&crtc_state->pch_pfit.dst) ||
- crtc_state->output_format == INTEL_OUTPUT_FORMAT_YCBCR420))
- return false;
-
- if (crtc_state->dsc.compression_enable)
- return false;
-
- if (crtc_state->has_psr2)
- return false;
-
- if (crtc_state->splitter.enable)
- return false;
-
- return true;
+ clear_bit_unlock(I915_RESET_MODESET, &to_gt(dev_priv)->reset.flags);
}
static void icl_set_pipe_chicken(const struct intel_crtc_state *crtc_state)
@@ -1353,19 +984,18 @@ static void icl_set_pipe_chicken(const struct intel_crtc_state *crtc_state)
*/
tmp |= PIXEL_ROUNDING_TRUNC_FB_PASSTHRU;
- if (IS_DG2(dev_priv)) {
- /*
- * Underrun recovery must always be disabled on DG2. However
- * the chicken bit meaning is inverted compared to other
- * platforms.
- */
+ /*
+ * Underrun recovery must always be disabled on display 13+.
+ * DG2 chicken bit meaning is inverted compared to other platforms.
+ */
+ if (IS_DG2(dev_priv))
tmp &= ~UNDERRUN_RECOVERY_ENABLE_DG2;
- } else if (DISPLAY_VER(dev_priv) >= 13) {
- if (underrun_recovery_supported(crtc_state))
- tmp &= ~UNDERRUN_RECOVERY_DISABLE_ADLP;
- else
- tmp |= UNDERRUN_RECOVERY_DISABLE_ADLP;
- }
+ else if (DISPLAY_VER(dev_priv) >= 13)
+ tmp |= UNDERRUN_RECOVERY_DISABLE_ADLP;
+
+ /* Wa_14010547955:dg2 */
+ if (IS_DG2_DISPLAY_STEP(dev_priv, STEP_B0, STEP_FOREVER))
+ tmp |= DG2_RENDER_CCSTAG_4_3_EN;
intel_de_write(dev_priv, PIPE_CHICKEN(pipe), tmp);
}
@@ -1387,7 +1017,7 @@ bool intel_has_pending_fb_unpin(struct drm_i915_private *dev_priv)
if (cleanup_done)
continue;
- drm_crtc_wait_one_vblank(crtc);
+ intel_crtc_wait_for_next_vblank(to_intel_crtc(crtc));
return true;
}
@@ -1395,158 +1025,6 @@ bool intel_has_pending_fb_unpin(struct drm_i915_private *dev_priv)
return false;
}
-void lpt_disable_iclkip(struct drm_i915_private *dev_priv)
-{
- u32 temp;
-
- intel_de_write(dev_priv, PIXCLK_GATE, PIXCLK_GATE_GATE);
-
- mutex_lock(&dev_priv->sb_lock);
-
- temp = intel_sbi_read(dev_priv, SBI_SSCCTL6, SBI_ICLK);
- temp |= SBI_SSCCTL_DISABLE;
- intel_sbi_write(dev_priv, SBI_SSCCTL6, temp, SBI_ICLK);
-
- mutex_unlock(&dev_priv->sb_lock);
-}
-
-/* Program iCLKIP clock to the desired frequency */
-static void lpt_program_iclkip(const struct intel_crtc_state *crtc_state)
-{
- struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
- int clock = crtc_state->hw.adjusted_mode.crtc_clock;
- u32 divsel, phaseinc, auxdiv, phasedir = 0;
- u32 temp;
-
- lpt_disable_iclkip(dev_priv);
-
- /* The iCLK virtual clock root frequency is in MHz,
- * but the adjusted_mode->crtc_clock in in KHz. To get the
- * divisors, it is necessary to divide one by another, so we
- * convert the virtual clock precision to KHz here for higher
- * precision.
- */
- for (auxdiv = 0; auxdiv < 2; auxdiv++) {
- u32 iclk_virtual_root_freq = 172800 * 1000;
- u32 iclk_pi_range = 64;
- u32 desired_divisor;
-
- desired_divisor = DIV_ROUND_CLOSEST(iclk_virtual_root_freq,
- clock << auxdiv);
- divsel = (desired_divisor / iclk_pi_range) - 2;
- phaseinc = desired_divisor % iclk_pi_range;
-
- /*
- * Near 20MHz is a corner case which is
- * out of range for the 7-bit divisor
- */
- if (divsel <= 0x7f)
- break;
- }
-
- /* This should not happen with any sane values */
- drm_WARN_ON(&dev_priv->drm, SBI_SSCDIVINTPHASE_DIVSEL(divsel) &
- ~SBI_SSCDIVINTPHASE_DIVSEL_MASK);
- drm_WARN_ON(&dev_priv->drm, SBI_SSCDIVINTPHASE_DIR(phasedir) &
- ~SBI_SSCDIVINTPHASE_INCVAL_MASK);
-
- drm_dbg_kms(&dev_priv->drm,
- "iCLKIP clock: found settings for %dKHz refresh rate: auxdiv=%x, divsel=%x, phasedir=%x, phaseinc=%x\n",
- clock, auxdiv, divsel, phasedir, phaseinc);
-
- mutex_lock(&dev_priv->sb_lock);
-
- /* Program SSCDIVINTPHASE6 */
- temp = intel_sbi_read(dev_priv, SBI_SSCDIVINTPHASE6, SBI_ICLK);
- temp &= ~SBI_SSCDIVINTPHASE_DIVSEL_MASK;
- temp |= SBI_SSCDIVINTPHASE_DIVSEL(divsel);
- temp &= ~SBI_SSCDIVINTPHASE_INCVAL_MASK;
- temp |= SBI_SSCDIVINTPHASE_INCVAL(phaseinc);
- temp |= SBI_SSCDIVINTPHASE_DIR(phasedir);
- temp |= SBI_SSCDIVINTPHASE_PROPAGATE;
- intel_sbi_write(dev_priv, SBI_SSCDIVINTPHASE6, temp, SBI_ICLK);
-
- /* Program SSCAUXDIV */
- temp = intel_sbi_read(dev_priv, SBI_SSCAUXDIV6, SBI_ICLK);
- temp &= ~SBI_SSCAUXDIV_FINALDIV2SEL(1);
- temp |= SBI_SSCAUXDIV_FINALDIV2SEL(auxdiv);
- intel_sbi_write(dev_priv, SBI_SSCAUXDIV6, temp, SBI_ICLK);
-
- /* Enable modulator and associated divider */
- temp = intel_sbi_read(dev_priv, SBI_SSCCTL6, SBI_ICLK);
- temp &= ~SBI_SSCCTL_DISABLE;
- intel_sbi_write(dev_priv, SBI_SSCCTL6, temp, SBI_ICLK);
-
- mutex_unlock(&dev_priv->sb_lock);
-
- /* Wait for initialization time */
- udelay(24);
-
- intel_de_write(dev_priv, PIXCLK_GATE, PIXCLK_GATE_UNGATE);
-}
-
-int lpt_get_iclkip(struct drm_i915_private *dev_priv)
-{
- u32 divsel, phaseinc, auxdiv;
- u32 iclk_virtual_root_freq = 172800 * 1000;
- u32 iclk_pi_range = 64;
- u32 desired_divisor;
- u32 temp;
-
- if ((intel_de_read(dev_priv, PIXCLK_GATE) & PIXCLK_GATE_UNGATE) == 0)
- return 0;
-
- mutex_lock(&dev_priv->sb_lock);
-
- temp = intel_sbi_read(dev_priv, SBI_SSCCTL6, SBI_ICLK);
- if (temp & SBI_SSCCTL_DISABLE) {
- mutex_unlock(&dev_priv->sb_lock);
- return 0;
- }
-
- temp = intel_sbi_read(dev_priv, SBI_SSCDIVINTPHASE6, SBI_ICLK);
- divsel = (temp & SBI_SSCDIVINTPHASE_DIVSEL_MASK) >>
- SBI_SSCDIVINTPHASE_DIVSEL_SHIFT;
- phaseinc = (temp & SBI_SSCDIVINTPHASE_INCVAL_MASK) >>
- SBI_SSCDIVINTPHASE_INCVAL_SHIFT;
-
- temp = intel_sbi_read(dev_priv, SBI_SSCAUXDIV6, SBI_ICLK);
- auxdiv = (temp & SBI_SSCAUXDIV_FINALDIV2SEL_MASK) >>
- SBI_SSCAUXDIV_FINALDIV2SEL_SHIFT;
-
- mutex_unlock(&dev_priv->sb_lock);
-
- desired_divisor = (divsel + 2) * iclk_pi_range + phaseinc;
-
- return DIV_ROUND_CLOSEST(iclk_virtual_root_freq,
- desired_divisor << auxdiv);
-}
-
-static void ilk_pch_transcoder_set_timings(const struct intel_crtc_state *crtc_state,
- enum pipe pch_transcoder)
-{
- struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
- enum transcoder cpu_transcoder = crtc_state->cpu_transcoder;
-
- intel_de_write(dev_priv, PCH_TRANS_HTOTAL(pch_transcoder),
- intel_de_read(dev_priv, HTOTAL(cpu_transcoder)));
- intel_de_write(dev_priv, PCH_TRANS_HBLANK(pch_transcoder),
- intel_de_read(dev_priv, HBLANK(cpu_transcoder)));
- intel_de_write(dev_priv, PCH_TRANS_HSYNC(pch_transcoder),
- intel_de_read(dev_priv, HSYNC(cpu_transcoder)));
-
- intel_de_write(dev_priv, PCH_TRANS_VTOTAL(pch_transcoder),
- intel_de_read(dev_priv, VTOTAL(cpu_transcoder)));
- intel_de_write(dev_priv, PCH_TRANS_VBLANK(pch_transcoder),
- intel_de_read(dev_priv, VBLANK(cpu_transcoder)));
- intel_de_write(dev_priv, PCH_TRANS_VSYNC(pch_transcoder),
- intel_de_read(dev_priv, VSYNC(cpu_transcoder)));
- intel_de_write(dev_priv, PCH_TRANS_VSYNCSHIFT(pch_transcoder),
- intel_de_read(dev_priv, VSYNCSHIFT(cpu_transcoder)));
-}
-
/*
* Finds the encoder associated with the given CRTC. This can only be
* used when we know that the CRTC isn't feeding multiple encoders!
@@ -1555,15 +1033,17 @@ struct intel_encoder *
intel_get_crtc_new_encoder(const struct intel_atomic_state *state,
const struct intel_crtc_state *crtc_state)
{
- struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
const struct drm_connector_state *connector_state;
const struct drm_connector *connector;
struct intel_encoder *encoder = NULL;
+ struct intel_crtc *master_crtc;
int num_encoders = 0;
int i;
+ master_crtc = intel_master_crtc(crtc_state);
+
for_each_new_connector_in_state(&state->base, connector, connector_state, i) {
- if (connector_state->crtc != &crtc->base)
+ if (connector_state->crtc != &master_crtc->base)
continue;
encoder = to_intel_encoder(connector_state->best_encoder);
@@ -1572,111 +1052,11 @@ intel_get_crtc_new_encoder(const struct intel_atomic_state *state,
drm_WARN(encoder->base.dev, num_encoders != 1,
"%d encoders for pipe %c\n",
- num_encoders, pipe_name(crtc->pipe));
+ num_encoders, pipe_name(master_crtc->pipe));
return encoder;
}
-/*
- * Enable PCH resources required for PCH ports:
- * - PCH PLLs
- * - FDI training & RX/TX
- * - update transcoder timings
- * - DP transcoding bits
- * - transcoder
- */
-static void ilk_pch_enable(const struct intel_atomic_state *state,
- const struct intel_crtc_state *crtc_state)
-{
- struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
- struct drm_device *dev = crtc->base.dev;
- struct drm_i915_private *dev_priv = to_i915(dev);
- enum pipe pipe = crtc->pipe;
- u32 temp;
-
- assert_pch_transcoder_disabled(dev_priv, pipe);
-
- /* For PCH output, training FDI link */
- intel_fdi_link_train(crtc, crtc_state);
-
- /* We need to program the right clock selection before writing the pixel
- * mutliplier into the DPLL. */
- if (HAS_PCH_CPT(dev_priv)) {
- u32 sel;
-
- temp = intel_de_read(dev_priv, PCH_DPLL_SEL);
- temp |= TRANS_DPLL_ENABLE(pipe);
- sel = TRANS_DPLLB_SEL(pipe);
- if (crtc_state->shared_dpll ==
- intel_get_shared_dpll_by_id(dev_priv, DPLL_ID_PCH_PLL_B))
- temp |= sel;
- else
- temp &= ~sel;
- intel_de_write(dev_priv, PCH_DPLL_SEL, temp);
- }
-
- /* XXX: pch pll's can be enabled any time before we enable the PCH
- * transcoder, and we actually should do this to not upset any PCH
- * transcoder that already use the clock when we share it.
- *
- * Note that enable_shared_dpll tries to do the right thing, but
- * get_shared_dpll unconditionally resets the pll - we need that to have
- * the right LVDS enable sequence. */
- intel_enable_shared_dpll(crtc_state);
-
- /* set transcoder timing, panel must allow it */
- assert_pps_unlocked(dev_priv, pipe);
- ilk_pch_transcoder_set_timings(crtc_state, pipe);
-
- intel_fdi_normal_train(crtc);
-
- /* For PCH DP, enable TRANS_DP_CTL */
- if (HAS_PCH_CPT(dev_priv) &&
- intel_crtc_has_dp_encoder(crtc_state)) {
- const struct drm_display_mode *adjusted_mode =
- &crtc_state->hw.adjusted_mode;
- u32 bpc = (intel_de_read(dev_priv, PIPECONF(pipe)) & PIPECONF_BPC_MASK) >> 5;
- i915_reg_t reg = TRANS_DP_CTL(pipe);
- enum port port;
-
- temp = intel_de_read(dev_priv, reg);
- temp &= ~(TRANS_DP_PORT_SEL_MASK |
- TRANS_DP_SYNC_MASK |
- TRANS_DP_BPC_MASK);
- temp |= TRANS_DP_OUTPUT_ENABLE;
- temp |= bpc << 9; /* same format but at 11:9 */
-
- if (adjusted_mode->flags & DRM_MODE_FLAG_PHSYNC)
- temp |= TRANS_DP_HSYNC_ACTIVE_HIGH;
- if (adjusted_mode->flags & DRM_MODE_FLAG_PVSYNC)
- temp |= TRANS_DP_VSYNC_ACTIVE_HIGH;
-
- port = intel_get_crtc_new_encoder(state, crtc_state)->port;
- drm_WARN_ON(dev, port < PORT_B || port > PORT_D);
- temp |= TRANS_DP_PORT_SEL(port);
-
- intel_de_write(dev_priv, reg, temp);
- }
-
- ilk_enable_pch_transcoder(crtc_state);
-}
-
-void lpt_pch_enable(const struct intel_crtc_state *crtc_state)
-{
- struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
- enum transcoder cpu_transcoder = crtc_state->cpu_transcoder;
-
- assert_pch_transcoder_disabled(dev_priv, PIPE_A);
-
- lpt_program_iclkip(crtc_state);
-
- /* Set transcoder timing. */
- ilk_pch_transcoder_set_timings(crtc_state, PIPE_A);
-
- lpt_enable_pch_transcoder(dev_priv, cpu_transcoder);
-}
-
static void cpt_verify_modeset(struct drm_i915_private *dev_priv,
enum pipe pipe)
{
@@ -1784,7 +1164,7 @@ void hsw_disable_ips(const struct intel_crtc_state *crtc_state)
}
/* We need to wait for a vblank before we can disable the plane. */
- intel_wait_for_vblank(dev_priv, crtc->pipe);
+ intel_crtc_wait_for_next_vblank(crtc);
}
static void intel_crtc_dpms_overlay_disable(struct intel_crtc *crtc)
@@ -1919,7 +1299,7 @@ static bool needs_async_flip_vtd_wa(const struct intel_crtc_state *crtc_state)
{
struct drm_i915_private *i915 = to_i915(crtc_state->uapi.crtc->dev);
- return crtc_state->uapi.async_flip && intel_vtd_active() &&
+ return crtc_state->uapi.async_flip && intel_vtd_active(i915) &&
(DISPLAY_VER(i915) == 9 || IS_BROADWELL(i915) || IS_HASWELL(i915));
}
@@ -2015,7 +1395,6 @@ static void intel_crtc_disable_flip_done(struct intel_atomic_state *state,
static void intel_crtc_async_flip_disable_wa(struct intel_atomic_state *state,
struct intel_crtc *crtc)
{
- struct drm_i915_private *i915 = to_i915(state->base.dev);
const struct intel_crtc_state *old_crtc_state =
intel_atomic_get_old_crtc_state(state, crtc);
const struct intel_crtc_state *new_crtc_state =
@@ -2041,7 +1420,7 @@ static void intel_crtc_async_flip_disable_wa(struct intel_atomic_state *state,
}
if (need_vbl_wait)
- intel_wait_for_vblank(i915, crtc->pipe);
+ intel_crtc_wait_for_next_vblank(crtc);
}
static void intel_pre_plane_update(struct intel_atomic_state *state,
@@ -2054,11 +1433,13 @@ static void intel_pre_plane_update(struct intel_atomic_state *state,
intel_atomic_get_new_crtc_state(state, crtc);
enum pipe pipe = crtc->pipe;
+ intel_psr_pre_plane_update(state, crtc);
+
if (hsw_pre_update_disable_ips(old_crtc_state, new_crtc_state))
hsw_disable_ips(old_crtc_state);
if (intel_fbc_pre_update(state, crtc))
- intel_wait_for_vblank(dev_priv, pipe);
+ intel_crtc_wait_for_next_vblank(crtc);
if (!needs_async_flip_vtd_wa(old_crtc_state) &&
needs_async_flip_vtd_wa(new_crtc_state))
@@ -2090,7 +1471,7 @@ static void intel_pre_plane_update(struct intel_atomic_state *state,
*/
if (HAS_GMCH(dev_priv) && old_crtc_state->hw.active &&
new_crtc_state->disable_cxsr && intel_set_memory_cxsr(dev_priv, false))
- intel_wait_for_vblank(dev_priv, pipe);
+ intel_crtc_wait_for_next_vblank(crtc);
/*
* IVB workaround: must disable low power watermarks for at least
@@ -2101,7 +1482,7 @@ static void intel_pre_plane_update(struct intel_atomic_state *state,
*/
if (old_crtc_state->hw.active &&
new_crtc_state->disable_lp_wm && ilk_disable_lp_wm(dev_priv))
- intel_wait_for_vblank(dev_priv, pipe);
+ intel_crtc_wait_for_next_vblank(crtc);
/*
* If we're doing a modeset we don't need to do any
@@ -2165,7 +1546,7 @@ static void intel_crtc_disable_planes(struct intel_atomic_state *state,
!(update_mask & BIT(plane->id)))
continue;
- intel_disable_plane(plane, new_crtc_state);
+ intel_plane_disable_arm(plane, new_crtc_state);
if (old_plane_state->uapi.visible)
fb_bits |= plane->frontbuffer_bit;
@@ -2199,10 +1580,30 @@ intel_connector_primary_encoder(struct intel_connector *connector)
static void intel_encoders_update_prepare(struct intel_atomic_state *state)
{
+ struct drm_i915_private *i915 = to_i915(state->base.dev);
+ struct intel_crtc_state *new_crtc_state, *old_crtc_state;
+ struct intel_crtc *crtc;
struct drm_connector_state *new_conn_state;
struct drm_connector *connector;
int i;
+ /*
+ * Make sure the DPLL state is up-to-date for fastset TypeC ports after non-blocking commits.
+ * TODO: Update the DPLL state for all cases in the encoder->update_prepare() hook.
+ */
+ if (i915->dpll.mgr) {
+ for_each_oldnew_intel_crtc_in_state(state, crtc, old_crtc_state, new_crtc_state, i) {
+ if (intel_crtc_needs_modeset(new_crtc_state))
+ continue;
+
+ new_crtc_state->shared_dpll = old_crtc_state->shared_dpll;
+ new_crtc_state->dpll_hw_state = old_crtc_state->dpll_hw_state;
+ }
+ }
+
+ if (!state->modeset)
+ return;
+
for_each_new_connector_in_state(&state->base, connector, new_conn_state,
i) {
struct intel_connector *intel_connector;
@@ -2229,6 +1630,9 @@ static void intel_encoders_update_complete(struct intel_atomic_state *state)
struct drm_connector *connector;
int i;
+ if (!state->modeset)
+ return;
+
for_each_new_connector_in_state(&state->base, connector, new_conn_state,
i) {
struct intel_connector *intel_connector;
@@ -2316,28 +1720,6 @@ static void intel_encoders_enable(struct intel_atomic_state *state,
}
}
-static void intel_encoders_pre_disable(struct intel_atomic_state *state,
- struct intel_crtc *crtc)
-{
- const struct intel_crtc_state *old_crtc_state =
- intel_atomic_get_old_crtc_state(state, crtc);
- const struct drm_connector_state *old_conn_state;
- struct drm_connector *conn;
- int i;
-
- for_each_old_connector_in_state(&state->base, conn, old_conn_state, i) {
- struct intel_encoder *encoder =
- to_intel_encoder(old_conn_state->best_encoder);
-
- if (old_conn_state->crtc != &crtc->base)
- continue;
-
- if (encoder->pre_disable)
- encoder->pre_disable(state, encoder, old_crtc_state,
- old_conn_state);
- }
-}
-
static void intel_encoders_disable(struct intel_atomic_state *state,
struct intel_crtc *crtc)
{
@@ -2432,7 +1814,7 @@ static void intel_disable_primary_plane(const struct intel_crtc_state *crtc_stat
struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
struct intel_plane *plane = to_intel_plane(crtc->base.primary);
- plane->disable_plane(plane, crtc_state);
+ plane->disable_arm(plane, crtc_state);
}
static void ilk_crtc_enable(struct intel_atomic_state *state,
@@ -2500,7 +1882,7 @@ static void ilk_crtc_enable(struct intel_atomic_state *state,
intel_enable_transcoder(new_crtc_state);
if (new_crtc_state->has_pch_encoder)
- ilk_pch_enable(state, new_crtc_state);
+ ilk_pch_enable(state, crtc);
intel_crtc_vblank_on(new_crtc_state);
@@ -2516,8 +1898,8 @@ static void ilk_crtc_enable(struct intel_atomic_state *state,
* in case there are more corner cases we don't know about.
*/
if (new_crtc_state->has_pch_encoder) {
- intel_wait_for_vblank(dev_priv, pipe);
- intel_wait_for_vblank(dev_priv, pipe);
+ intel_crtc_wait_for_next_vblank(crtc);
+ intel_crtc_wait_for_next_vblank(crtc);
}
intel_set_cpu_fifo_underrun_reporting(dev_priv, pipe, true);
intel_set_pch_fifo_underrun_reporting(dev_priv, pipe, true);
@@ -2592,42 +1974,39 @@ static void hsw_set_frame_start_delay(const struct intel_crtc_state *crtc_state)
static void icl_ddi_bigjoiner_pre_enable(struct intel_atomic_state *state,
const struct intel_crtc_state *crtc_state)
{
- struct intel_crtc *master = to_intel_crtc(crtc_state->uapi.crtc);
- struct drm_i915_private *dev_priv = to_i915(master->base.dev);
+ struct drm_i915_private *dev_priv = to_i915(state->base.dev);
struct intel_crtc_state *master_crtc_state;
+ struct intel_crtc *master_crtc;
struct drm_connector_state *conn_state;
struct drm_connector *conn;
struct intel_encoder *encoder = NULL;
int i;
- if (crtc_state->bigjoiner_slave)
- master = crtc_state->bigjoiner_linked_crtc;
-
- master_crtc_state = intel_atomic_get_new_crtc_state(state, master);
+ master_crtc = intel_master_crtc(crtc_state);
+ master_crtc_state = intel_atomic_get_new_crtc_state(state, master_crtc);
for_each_new_connector_in_state(&state->base, conn, conn_state, i) {
- if (conn_state->crtc != &master->base)
+ if (conn_state->crtc != &master_crtc->base)
continue;
encoder = to_intel_encoder(conn_state->best_encoder);
break;
}
- if (!crtc_state->bigjoiner_slave) {
- /* need to enable VDSC, which we skipped in pre-enable */
- intel_dsc_enable(encoder, crtc_state);
- } else {
- /*
- * Enable sequence steps 1-7 on bigjoiner master
- */
- intel_encoders_pre_pll_enable(state, master);
- if (master_crtc_state->shared_dpll)
- intel_enable_shared_dpll(master_crtc_state);
- intel_encoders_pre_enable(state, master);
+ /*
+ * Enable sequence steps 1-7 on bigjoiner master
+ */
+ if (crtc_state->bigjoiner_slave)
+ intel_encoders_pre_pll_enable(state, master_crtc);
- /* and DSC on slave */
- intel_dsc_enable(NULL, crtc_state);
- }
+ if (crtc_state->shared_dpll)
+ intel_enable_shared_dpll(crtc_state);
+
+ if (crtc_state->bigjoiner_slave)
+ intel_encoders_pre_enable(state, master_crtc);
+
+ /* need to enable VDSC, which we skipped in pre-enable */
+ intel_dsc_enable(crtc_state);
if (DISPLAY_VER(dev_priv) >= 13)
intel_uncompressed_joiner_enable(crtc_state);
@@ -2720,7 +2099,7 @@ static void hsw_crtc_enable(struct intel_atomic_state *state,
intel_encoders_enable(state, crtc);
if (psl_clkgate_wa) {
- intel_wait_for_vblank(dev_priv, pipe);
+ intel_crtc_wait_for_next_vblank(crtc);
glk_pipe_scaler_clock_gating_wa(dev_priv, pipe, false);
}
@@ -2728,8 +2107,12 @@ static void hsw_crtc_enable(struct intel_atomic_state *state,
* to change the workaround. */
hsw_workaround_pipe = new_crtc_state->hsw_workaround_pipe;
if (IS_HASWELL(dev_priv) && hsw_workaround_pipe != INVALID_PIPE) {
- intel_wait_for_vblank(dev_priv, hsw_workaround_pipe);
- intel_wait_for_vblank(dev_priv, hsw_workaround_pipe);
+ struct intel_crtc *wa_crtc;
+
+ wa_crtc = intel_crtc_for_pipe(dev_priv, hsw_workaround_pipe);
+
+ intel_crtc_wait_for_next_vblank(wa_crtc);
+ intel_crtc_wait_for_next_vblank(wa_crtc);
}
}
@@ -2774,33 +2157,12 @@ static void ilk_crtc_disable(struct intel_atomic_state *state,
ilk_pfit_disable(old_crtc_state);
if (old_crtc_state->has_pch_encoder)
- ilk_fdi_disable(crtc);
+ ilk_pch_disable(state, crtc);
intel_encoders_post_disable(state, crtc);
- if (old_crtc_state->has_pch_encoder) {
- ilk_disable_pch_transcoder(dev_priv, pipe);
-
- if (HAS_PCH_CPT(dev_priv)) {
- i915_reg_t reg;
- u32 temp;
-
- /* disable TRANS_DP_CTL */
- reg = TRANS_DP_CTL(pipe);
- temp = intel_de_read(dev_priv, reg);
- temp &= ~(TRANS_DP_OUTPUT_ENABLE |
- TRANS_DP_PORT_SEL_MASK);
- temp |= TRANS_DP_PORT_SEL_NONE;
- intel_de_write(dev_priv, reg, temp);
-
- /* disable DPLL_SEL */
- temp = intel_de_read(dev_priv, PCH_DPLL_SEL);
- temp &= ~(TRANS_DPLL_ENABLE(pipe) | TRANS_DPLLB_SEL(pipe));
- intel_de_write(dev_priv, PCH_DPLL_SEL, temp);
- }
-
- ilk_fdi_pll_disable(crtc);
- }
+ if (old_crtc_state->has_pch_encoder)
+ ilk_pch_post_disable(state, crtc);
intel_set_cpu_fifo_underrun_reporting(dev_priv, pipe, true);
intel_set_pch_fifo_underrun_reporting(dev_priv, pipe, true);
@@ -2809,12 +2171,17 @@ static void ilk_crtc_disable(struct intel_atomic_state *state,
static void hsw_crtc_disable(struct intel_atomic_state *state,
struct intel_crtc *crtc)
{
+ const struct intel_crtc_state *old_crtc_state =
+ intel_atomic_get_old_crtc_state(state, crtc);
+
/*
* FIXME collapse everything to one hook.
* Need care with mst->ddi interactions.
*/
- intel_encoders_disable(state, crtc);
- intel_encoders_post_disable(state, crtc);
+ if (!old_crtc_state->bigjoiner_slave) {
+ intel_encoders_disable(state, crtc);
+ intel_encoders_post_disable(state, crtc);
+ }
}
static void i9xx_pfit_enable(const struct intel_crtc_state *crtc_state)
@@ -3171,7 +2538,7 @@ static void i9xx_crtc_enable(struct intel_atomic_state *state,
/* prevents spurious underruns */
if (DISPLAY_VER(dev_priv) == 2)
- intel_wait_for_vblank(dev_priv, pipe);
+ intel_crtc_wait_for_next_vblank(crtc);
}
static void i9xx_pfit_disable(const struct intel_crtc_state *old_crtc_state)
@@ -3202,7 +2569,7 @@ static void i9xx_crtc_disable(struct intel_atomic_state *state,
* wait for planes to fully turn off before disabling the pipe.
*/
if (DISPLAY_VER(dev_priv) == 2)
- intel_wait_for_vblank(dev_priv, pipe);
+ intel_crtc_wait_for_next_vblank(crtc);
intel_encoders_disable(state, crtc);
@@ -4306,414 +3673,6 @@ static bool i9xx_get_pipe_config(struct intel_crtc *crtc,
return ret;
}
-static void ilk_init_pch_refclk(struct drm_i915_private *dev_priv)
-{
- struct intel_encoder *encoder;
- int i;
- u32 val, final;
- bool has_lvds = false;
- bool has_cpu_edp = false;
- bool has_panel = false;
- bool has_ck505 = false;
- bool can_ssc = false;
- bool using_ssc_source = false;
-
- /* We need to take the global config into account */
- for_each_intel_encoder(&dev_priv->drm, encoder) {
- switch (encoder->type) {
- case INTEL_OUTPUT_LVDS:
- has_panel = true;
- has_lvds = true;
- break;
- case INTEL_OUTPUT_EDP:
- has_panel = true;
- if (encoder->port == PORT_A)
- has_cpu_edp = true;
- break;
- default:
- break;
- }
- }
-
- if (HAS_PCH_IBX(dev_priv)) {
- has_ck505 = dev_priv->vbt.display_clock_mode;
- can_ssc = has_ck505;
- } else {
- has_ck505 = false;
- can_ssc = true;
- }
-
- /* Check if any DPLLs are using the SSC source */
- for (i = 0; i < dev_priv->dpll.num_shared_dpll; i++) {
- u32 temp = intel_de_read(dev_priv, PCH_DPLL(i));
-
- if (!(temp & DPLL_VCO_ENABLE))
- continue;
-
- if ((temp & PLL_REF_INPUT_MASK) ==
- PLLB_REF_INPUT_SPREADSPECTRUMIN) {
- using_ssc_source = true;
- break;
- }
- }
-
- drm_dbg_kms(&dev_priv->drm,
- "has_panel %d has_lvds %d has_ck505 %d using_ssc_source %d\n",
- has_panel, has_lvds, has_ck505, using_ssc_source);
-
- /* Ironlake: try to setup display ref clock before DPLL
- * enabling. This is only under driver's control after
- * PCH B stepping, previous chipset stepping should be
- * ignoring this setting.
- */
- val = intel_de_read(dev_priv, PCH_DREF_CONTROL);
-
- /* As we must carefully and slowly disable/enable each source in turn,
- * compute the final state we want first and check if we need to
- * make any changes at all.
- */
- final = val;
- final &= ~DREF_NONSPREAD_SOURCE_MASK;
- if (has_ck505)
- final |= DREF_NONSPREAD_CK505_ENABLE;
- else
- final |= DREF_NONSPREAD_SOURCE_ENABLE;
-
- final &= ~DREF_SSC_SOURCE_MASK;
- final &= ~DREF_CPU_SOURCE_OUTPUT_MASK;
- final &= ~DREF_SSC1_ENABLE;
-
- if (has_panel) {
- final |= DREF_SSC_SOURCE_ENABLE;
-
- if (intel_panel_use_ssc(dev_priv) && can_ssc)
- final |= DREF_SSC1_ENABLE;
-
- if (has_cpu_edp) {
- if (intel_panel_use_ssc(dev_priv) && can_ssc)
- final |= DREF_CPU_SOURCE_OUTPUT_DOWNSPREAD;
- else
- final |= DREF_CPU_SOURCE_OUTPUT_NONSPREAD;
- } else
- final |= DREF_CPU_SOURCE_OUTPUT_DISABLE;
- } else if (using_ssc_source) {
- final |= DREF_SSC_SOURCE_ENABLE;
- final |= DREF_SSC1_ENABLE;
- }
-
- if (final == val)
- return;
-
- /* Always enable nonspread source */
- val &= ~DREF_NONSPREAD_SOURCE_MASK;
-
- if (has_ck505)
- val |= DREF_NONSPREAD_CK505_ENABLE;
- else
- val |= DREF_NONSPREAD_SOURCE_ENABLE;
-
- if (has_panel) {
- val &= ~DREF_SSC_SOURCE_MASK;
- val |= DREF_SSC_SOURCE_ENABLE;
-
- /* SSC must be turned on before enabling the CPU output */
- if (intel_panel_use_ssc(dev_priv) && can_ssc) {
- drm_dbg_kms(&dev_priv->drm, "Using SSC on panel\n");
- val |= DREF_SSC1_ENABLE;
- } else
- val &= ~DREF_SSC1_ENABLE;
-
- /* Get SSC going before enabling the outputs */
- intel_de_write(dev_priv, PCH_DREF_CONTROL, val);
- intel_de_posting_read(dev_priv, PCH_DREF_CONTROL);
- udelay(200);
-
- val &= ~DREF_CPU_SOURCE_OUTPUT_MASK;
-
- /* Enable CPU source on CPU attached eDP */
- if (has_cpu_edp) {
- if (intel_panel_use_ssc(dev_priv) && can_ssc) {
- drm_dbg_kms(&dev_priv->drm,
- "Using SSC on eDP\n");
- val |= DREF_CPU_SOURCE_OUTPUT_DOWNSPREAD;
- } else
- val |= DREF_CPU_SOURCE_OUTPUT_NONSPREAD;
- } else
- val |= DREF_CPU_SOURCE_OUTPUT_DISABLE;
-
- intel_de_write(dev_priv, PCH_DREF_CONTROL, val);
- intel_de_posting_read(dev_priv, PCH_DREF_CONTROL);
- udelay(200);
- } else {
- drm_dbg_kms(&dev_priv->drm, "Disabling CPU source output\n");
-
- val &= ~DREF_CPU_SOURCE_OUTPUT_MASK;
-
- /* Turn off CPU output */
- val |= DREF_CPU_SOURCE_OUTPUT_DISABLE;
-
- intel_de_write(dev_priv, PCH_DREF_CONTROL, val);
- intel_de_posting_read(dev_priv, PCH_DREF_CONTROL);
- udelay(200);
-
- if (!using_ssc_source) {
- drm_dbg_kms(&dev_priv->drm, "Disabling SSC source\n");
-
- /* Turn off the SSC source */
- val &= ~DREF_SSC_SOURCE_MASK;
- val |= DREF_SSC_SOURCE_DISABLE;
-
- /* Turn off SSC1 */
- val &= ~DREF_SSC1_ENABLE;
-
- intel_de_write(dev_priv, PCH_DREF_CONTROL, val);
- intel_de_posting_read(dev_priv, PCH_DREF_CONTROL);
- udelay(200);
- }
- }
-
- BUG_ON(val != final);
-}
-
-/* Implements 3 different sequences from BSpec chapter "Display iCLK
- * Programming" based on the parameters passed:
- * - Sequence to enable CLKOUT_DP
- * - Sequence to enable CLKOUT_DP without spread
- * - Sequence to enable CLKOUT_DP for FDI usage and configure PCH FDI I/O
- */
-static void lpt_enable_clkout_dp(struct drm_i915_private *dev_priv,
- bool with_spread, bool with_fdi)
-{
- u32 reg, tmp;
-
- if (drm_WARN(&dev_priv->drm, with_fdi && !with_spread,
- "FDI requires downspread\n"))
- with_spread = true;
- if (drm_WARN(&dev_priv->drm, HAS_PCH_LPT_LP(dev_priv) &&
- with_fdi, "LP PCH doesn't have FDI\n"))
- with_fdi = false;
-
- mutex_lock(&dev_priv->sb_lock);
-
- tmp = intel_sbi_read(dev_priv, SBI_SSCCTL, SBI_ICLK);
- tmp &= ~SBI_SSCCTL_DISABLE;
- tmp |= SBI_SSCCTL_PATHALT;
- intel_sbi_write(dev_priv, SBI_SSCCTL, tmp, SBI_ICLK);
-
- udelay(24);
-
- if (with_spread) {
- tmp = intel_sbi_read(dev_priv, SBI_SSCCTL, SBI_ICLK);
- tmp &= ~SBI_SSCCTL_PATHALT;
- intel_sbi_write(dev_priv, SBI_SSCCTL, tmp, SBI_ICLK);
-
- if (with_fdi)
- lpt_fdi_program_mphy(dev_priv);
- }
-
- reg = HAS_PCH_LPT_LP(dev_priv) ? SBI_GEN0 : SBI_DBUFF0;
- tmp = intel_sbi_read(dev_priv, reg, SBI_ICLK);
- tmp |= SBI_GEN0_CFG_BUFFENABLE_DISABLE;
- intel_sbi_write(dev_priv, reg, tmp, SBI_ICLK);
-
- mutex_unlock(&dev_priv->sb_lock);
-}
-
-/* Sequence to disable CLKOUT_DP */
-void lpt_disable_clkout_dp(struct drm_i915_private *dev_priv)
-{
- u32 reg, tmp;
-
- mutex_lock(&dev_priv->sb_lock);
-
- reg = HAS_PCH_LPT_LP(dev_priv) ? SBI_GEN0 : SBI_DBUFF0;
- tmp = intel_sbi_read(dev_priv, reg, SBI_ICLK);
- tmp &= ~SBI_GEN0_CFG_BUFFENABLE_DISABLE;
- intel_sbi_write(dev_priv, reg, tmp, SBI_ICLK);
-
- tmp = intel_sbi_read(dev_priv, SBI_SSCCTL, SBI_ICLK);
- if (!(tmp & SBI_SSCCTL_DISABLE)) {
- if (!(tmp & SBI_SSCCTL_PATHALT)) {
- tmp |= SBI_SSCCTL_PATHALT;
- intel_sbi_write(dev_priv, SBI_SSCCTL, tmp, SBI_ICLK);
- udelay(32);
- }
- tmp |= SBI_SSCCTL_DISABLE;
- intel_sbi_write(dev_priv, SBI_SSCCTL, tmp, SBI_ICLK);
- }
-
- mutex_unlock(&dev_priv->sb_lock);
-}
-
-#define BEND_IDX(steps) ((50 + (steps)) / 5)
-
-static const u16 sscdivintphase[] = {
- [BEND_IDX( 50)] = 0x3B23,
- [BEND_IDX( 45)] = 0x3B23,
- [BEND_IDX( 40)] = 0x3C23,
- [BEND_IDX( 35)] = 0x3C23,
- [BEND_IDX( 30)] = 0x3D23,
- [BEND_IDX( 25)] = 0x3D23,
- [BEND_IDX( 20)] = 0x3E23,
- [BEND_IDX( 15)] = 0x3E23,
- [BEND_IDX( 10)] = 0x3F23,
- [BEND_IDX( 5)] = 0x3F23,
- [BEND_IDX( 0)] = 0x0025,
- [BEND_IDX( -5)] = 0x0025,
- [BEND_IDX(-10)] = 0x0125,
- [BEND_IDX(-15)] = 0x0125,
- [BEND_IDX(-20)] = 0x0225,
- [BEND_IDX(-25)] = 0x0225,
- [BEND_IDX(-30)] = 0x0325,
- [BEND_IDX(-35)] = 0x0325,
- [BEND_IDX(-40)] = 0x0425,
- [BEND_IDX(-45)] = 0x0425,
- [BEND_IDX(-50)] = 0x0525,
-};
-
-/*
- * Bend CLKOUT_DP
- * steps -50 to 50 inclusive, in steps of 5
- * < 0 slow down the clock, > 0 speed up the clock, 0 == no bend (135MHz)
- * change in clock period = -(steps / 10) * 5.787 ps
- */
-static void lpt_bend_clkout_dp(struct drm_i915_private *dev_priv, int steps)
-{
- u32 tmp;
- int idx = BEND_IDX(steps);
-
- if (drm_WARN_ON(&dev_priv->drm, steps % 5 != 0))
- return;
-
- if (drm_WARN_ON(&dev_priv->drm, idx >= ARRAY_SIZE(sscdivintphase)))
- return;
-
- mutex_lock(&dev_priv->sb_lock);
-
- if (steps % 10 != 0)
- tmp = 0xAAAAAAAB;
- else
- tmp = 0x00000000;
- intel_sbi_write(dev_priv, SBI_SSCDITHPHASE, tmp, SBI_ICLK);
-
- tmp = intel_sbi_read(dev_priv, SBI_SSCDIVINTPHASE, SBI_ICLK);
- tmp &= 0xffff0000;
- tmp |= sscdivintphase[idx];
- intel_sbi_write(dev_priv, SBI_SSCDIVINTPHASE, tmp, SBI_ICLK);
-
- mutex_unlock(&dev_priv->sb_lock);
-}
-
-#undef BEND_IDX
-
-static bool spll_uses_pch_ssc(struct drm_i915_private *dev_priv)
-{
- u32 fuse_strap = intel_de_read(dev_priv, FUSE_STRAP);
- u32 ctl = intel_de_read(dev_priv, SPLL_CTL);
-
- if ((ctl & SPLL_PLL_ENABLE) == 0)
- return false;
-
- if ((ctl & SPLL_REF_MASK) == SPLL_REF_MUXED_SSC &&
- (fuse_strap & HSW_CPU_SSC_ENABLE) == 0)
- return true;
-
- if (IS_BROADWELL(dev_priv) &&
- (ctl & SPLL_REF_MASK) == SPLL_REF_PCH_SSC_BDW)
- return true;
-
- return false;
-}
-
-static bool wrpll_uses_pch_ssc(struct drm_i915_private *dev_priv,
- enum intel_dpll_id id)
-{
- u32 fuse_strap = intel_de_read(dev_priv, FUSE_STRAP);
- u32 ctl = intel_de_read(dev_priv, WRPLL_CTL(id));
-
- if ((ctl & WRPLL_PLL_ENABLE) == 0)
- return false;
-
- if ((ctl & WRPLL_REF_MASK) == WRPLL_REF_PCH_SSC)
- return true;
-
- if ((IS_BROADWELL(dev_priv) || IS_HSW_ULT(dev_priv)) &&
- (ctl & WRPLL_REF_MASK) == WRPLL_REF_MUXED_SSC_BDW &&
- (fuse_strap & HSW_CPU_SSC_ENABLE) == 0)
- return true;
-
- return false;
-}
-
-static void lpt_init_pch_refclk(struct drm_i915_private *dev_priv)
-{
- struct intel_encoder *encoder;
- bool has_fdi = false;
-
- for_each_intel_encoder(&dev_priv->drm, encoder) {
- switch (encoder->type) {
- case INTEL_OUTPUT_ANALOG:
- has_fdi = true;
- break;
- default:
- break;
- }
- }
-
- /*
- * The BIOS may have decided to use the PCH SSC
- * reference so we must not disable it until the
- * relevant PLLs have stopped relying on it. We'll
- * just leave the PCH SSC reference enabled in case
- * any active PLL is using it. It will get disabled
- * after runtime suspend if we don't have FDI.
- *
- * TODO: Move the whole reference clock handling
- * to the modeset sequence proper so that we can
- * actually enable/disable/reconfigure these things
- * safely. To do that we need to introduce a real
- * clock hierarchy. That would also allow us to do
- * clock bending finally.
- */
- dev_priv->pch_ssc_use = 0;
-
- if (spll_uses_pch_ssc(dev_priv)) {
- drm_dbg_kms(&dev_priv->drm, "SPLL using PCH SSC\n");
- dev_priv->pch_ssc_use |= BIT(DPLL_ID_SPLL);
- }
-
- if (wrpll_uses_pch_ssc(dev_priv, DPLL_ID_WRPLL1)) {
- drm_dbg_kms(&dev_priv->drm, "WRPLL1 using PCH SSC\n");
- dev_priv->pch_ssc_use |= BIT(DPLL_ID_WRPLL1);
- }
-
- if (wrpll_uses_pch_ssc(dev_priv, DPLL_ID_WRPLL2)) {
- drm_dbg_kms(&dev_priv->drm, "WRPLL2 using PCH SSC\n");
- dev_priv->pch_ssc_use |= BIT(DPLL_ID_WRPLL2);
- }
-
- if (dev_priv->pch_ssc_use)
- return;
-
- if (has_fdi) {
- lpt_bend_clkout_dp(dev_priv, 0);
- lpt_enable_clkout_dp(dev_priv, true, true);
- } else {
- lpt_disable_clkout_dp(dev_priv);
- }
-}
-
-/*
- * Initialize reference clocks when the driver loads
- */
-void intel_init_pch_refclk(struct drm_i915_private *dev_priv)
-{
- if (HAS_PCH_IBX(dev_priv) || HAS_PCH_CPT(dev_priv))
- ilk_init_pch_refclk(dev_priv);
- else if (HAS_PCH_LPT(dev_priv))
- lpt_init_pch_refclk(dev_priv);
-}
-
static void ilk_set_pipeconf(const struct intel_crtc_state *crtc_state)
{
struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
@@ -4978,8 +3937,8 @@ void intel_dp_get_m_n(struct intel_crtc *crtc,
&pipe_config->dp_m2_n2);
}
-static void ilk_get_fdi_m_n_config(struct intel_crtc *crtc,
- struct intel_crtc_state *pipe_config)
+void ilk_get_fdi_m_n_config(struct intel_crtc *crtc,
+ struct intel_crtc_state *pipe_config)
{
intel_cpu_transcoder_get_m_n(crtc, pipe_config->cpu_transcoder,
&pipe_config->fdi_m_n, NULL);
@@ -5116,50 +4075,9 @@ static bool ilk_get_pipe_config(struct intel_crtc *crtc,
i9xx_get_pipe_color_config(pipe_config);
intel_color_get_config(pipe_config);
- if (intel_de_read(dev_priv, PCH_TRANSCONF(crtc->pipe)) & TRANS_ENABLE) {
- struct intel_shared_dpll *pll;
- enum intel_dpll_id pll_id;
- bool pll_active;
+ pipe_config->pixel_multiplier = 1;
- pipe_config->has_pch_encoder = true;
-
- tmp = intel_de_read(dev_priv, FDI_RX_CTL(crtc->pipe));
- pipe_config->fdi_lanes = ((FDI_DP_PORT_WIDTH_MASK & tmp) >>
- FDI_DP_PORT_WIDTH_SHIFT) + 1;
-
- ilk_get_fdi_m_n_config(crtc, pipe_config);
-
- if (HAS_PCH_IBX(dev_priv)) {
- /*
- * The pipe->pch transcoder and pch transcoder->pll
- * mapping is fixed.
- */
- pll_id = (enum intel_dpll_id) crtc->pipe;
- } else {
- tmp = intel_de_read(dev_priv, PCH_DPLL_SEL);
- if (tmp & TRANS_DPLLB_SEL(crtc->pipe))
- pll_id = DPLL_ID_PCH_PLL_B;
- else
- pll_id= DPLL_ID_PCH_PLL_A;
- }
-
- pipe_config->shared_dpll =
- intel_get_shared_dpll_by_id(dev_priv, pll_id);
- pll = pipe_config->shared_dpll;
-
- pll_active = intel_dpll_get_hw_state(dev_priv, pll,
- &pipe_config->dpll_hw_state);
- drm_WARN_ON(dev, !pll_active);
-
- tmp = pipe_config->dpll_hw_state.dpll;
- pipe_config->pixel_multiplier =
- ((tmp & PLL_REF_SDVO_HDMI_MULTIPLIER_MASK)
- >> PLL_REF_SDVO_HDMI_MULTIPLIER_SHIFT) + 1;
-
- ilk_pch_clock_get(crtc, pipe_config);
- } else {
- pipe_config->pixel_multiplier = 1;
- }
+ ilk_pch_get_config(pipe_config);
intel_get_transcoder_timings(crtc, pipe_config);
intel_get_pipe_src_size(crtc, pipe_config);
@@ -5174,6 +4092,16 @@ static bool ilk_get_pipe_config(struct intel_crtc *crtc,
return ret;
}
+static u8 bigjoiner_pipes(struct drm_i915_private *i915)
+{
+ if (DISPLAY_VER(i915) >= 12)
+ return BIT(PIPE_A) | BIT(PIPE_B) | BIT(PIPE_C) | BIT(PIPE_D);
+ else if (DISPLAY_VER(i915) >= 11)
+ return BIT(PIPE_B) | BIT(PIPE_C);
+ else
+ return 0;
+}
+
static bool transcoder_ddi_func_is_enabled(struct drm_i915_private *dev_priv,
enum transcoder cpu_transcoder)
{
@@ -5189,6 +4117,54 @@ static bool transcoder_ddi_func_is_enabled(struct drm_i915_private *dev_priv,
return tmp & TRANS_DDI_FUNC_ENABLE;
}
+static u8 enabled_bigjoiner_pipes(struct drm_i915_private *dev_priv)
+{
+ u8 master_pipes = 0, slave_pipes = 0;
+ struct intel_crtc *crtc;
+
+ for_each_intel_crtc(&dev_priv->drm, crtc) {
+ enum intel_display_power_domain power_domain;
+ enum pipe pipe = crtc->pipe;
+ intel_wakeref_t wakeref;
+
+ if ((bigjoiner_pipes(dev_priv) & BIT(pipe)) == 0)
+ continue;
+
+ power_domain = intel_dsc_power_domain(crtc, (enum transcoder) pipe);
+ with_intel_display_power_if_enabled(dev_priv, power_domain, wakeref) {
+ u32 tmp = intel_de_read(dev_priv, ICL_PIPE_DSS_CTL1(pipe));
+
+ if (!(tmp & BIG_JOINER_ENABLE))
+ continue;
+
+ if (tmp & MASTER_BIG_JOINER_ENABLE)
+ master_pipes |= BIT(pipe);
+ else
+ slave_pipes |= BIT(pipe);
+ }
+
+ if (DISPLAY_VER(dev_priv) < 13)
+ continue;
+
+ power_domain = POWER_DOMAIN_PIPE(pipe);
+ with_intel_display_power_if_enabled(dev_priv, power_domain, wakeref) {
+ u32 tmp = intel_de_read(dev_priv, ICL_PIPE_DSS_CTL1(pipe));
+
+ if (tmp & UNCOMPRESSED_JOINER_MASTER)
+ master_pipes |= BIT(pipe);
+ if (tmp & UNCOMPRESSED_JOINER_SLAVE)
+ slave_pipes |= BIT(pipe);
+ }
+ }
+
+ /* Bigjoiner pipes should always be consecutive master and slave */
+ drm_WARN(&dev_priv->drm, slave_pipes != master_pipes << 1,
+ "Bigjoiner misconfigured (master pipes 0x%x, slave pipes 0x%x)\n",
+ master_pipes, slave_pipes);
+
+ return slave_pipes;
+}
+
static u8 hsw_panel_transcoders(struct drm_i915_private *i915)
{
u8 panel_transcoder_mask = BIT(TRANSCODER_EDP);
@@ -5250,10 +4226,18 @@ static u8 hsw_enabled_transcoders(struct intel_crtc *crtc)
enabled_transcoders |= BIT(cpu_transcoder);
}
+ /* single pipe or bigjoiner master */
cpu_transcoder = (enum transcoder) crtc->pipe;
if (transcoder_ddi_func_is_enabled(dev_priv, cpu_transcoder))
enabled_transcoders |= BIT(cpu_transcoder);
+ /* bigjoiner slave -> consider the master pipe's transcoder as well */
+ if (enabled_bigjoiner_pipes(dev_priv) & BIT(crtc->pipe)) {
+ cpu_transcoder = (enum transcoder) crtc->pipe - 1;
+ if (transcoder_ddi_func_is_enabled(dev_priv, cpu_transcoder))
+ enabled_transcoders |= BIT(cpu_transcoder);
+ }
+
return enabled_transcoders;
}
@@ -5374,45 +4358,6 @@ static bool bxt_get_dsi_transcoder_state(struct intel_crtc *crtc,
return transcoder_is_dsi(pipe_config->cpu_transcoder);
}
-static void hsw_get_ddi_port_state(struct intel_crtc *crtc,
- struct intel_crtc_state *pipe_config)
-{
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
- enum transcoder cpu_transcoder = pipe_config->cpu_transcoder;
- enum port port;
- u32 tmp;
-
- if (transcoder_is_dsi(cpu_transcoder)) {
- port = (cpu_transcoder == TRANSCODER_DSI_A) ?
- PORT_A : PORT_B;
- } else {
- tmp = intel_de_read(dev_priv,
- TRANS_DDI_FUNC_CTL(cpu_transcoder));
- if (!(tmp & TRANS_DDI_FUNC_ENABLE))
- return;
- if (DISPLAY_VER(dev_priv) >= 12)
- port = TGL_TRANS_DDI_FUNC_CTL_VAL_TO_PORT(tmp);
- else
- port = TRANS_DDI_FUNC_CTL_VAL_TO_PORT(tmp);
- }
-
- /*
- * Haswell has only FDI/PCH transcoder A. It is which is connected to
- * DDI E. So just check whether this pipe is wired to DDI E and whether
- * the PCH transcoder is on.
- */
- if (DISPLAY_VER(dev_priv) < 9 &&
- (port == PORT_E) && intel_de_read(dev_priv, LPT_TRANSCONF) & TRANS_ENABLE) {
- pipe_config->has_pch_encoder = true;
-
- tmp = intel_de_read(dev_priv, FDI_RX_CTL(PIPE_A));
- pipe_config->fdi_lanes = ((FDI_DP_PORT_WIDTH_MASK & tmp) >>
- FDI_DP_PORT_WIDTH_SHIFT) + 1;
-
- ilk_get_fdi_m_n_config(crtc, pipe_config);
- }
-}
-
static bool hsw_get_pipe_config(struct intel_crtc *crtc,
struct intel_crtc_state *pipe_config)
{
@@ -5439,21 +4384,12 @@ static bool hsw_get_pipe_config(struct intel_crtc *crtc,
if (DISPLAY_VER(dev_priv) >= 13 && !pipe_config->dsc.compression_enable)
intel_uncompressed_joiner_get_config(pipe_config);
- if (!active) {
- /* bigjoiner slave doesn't enable transcoder */
- if (!pipe_config->bigjoiner_slave)
- goto out;
+ if (!active)
+ goto out;
- active = true;
- pipe_config->pixel_multiplier = 1;
-
- /* we cannot read out most state, so don't bother.. */
- pipe_config->quirks |= PIPE_CONFIG_QUIRK_BIGJOINER_SLAVE;
- } else if (!transcoder_is_dsi(pipe_config->cpu_transcoder) ||
- DISPLAY_VER(dev_priv) >= 11) {
- hsw_get_ddi_port_state(crtc, pipe_config);
+ if (!transcoder_is_dsi(pipe_config->cpu_transcoder) ||
+ DISPLAY_VER(dev_priv) >= 11)
intel_get_transcoder_timings(crtc, pipe_config);
- }
if (HAS_VRR(dev_priv) && !transcoder_is_dsi(pipe_config->cpu_transcoder))
intel_vrr_get_config(crtc, pipe_config);
@@ -5521,10 +4457,7 @@ static bool hsw_get_pipe_config(struct intel_crtc *crtc,
}
}
- if (pipe_config->bigjoiner_slave) {
- /* Cannot be read out as a slave, set to 0. */
- pipe_config->pixel_multiplier = 0;
- } else if (pipe_config->cpu_transcoder != TRANSCODER_EDP &&
+ if (pipe_config->cpu_transcoder != TRANSCODER_EDP &&
!transcoder_is_dsi(pipe_config->cpu_transcoder)) {
pipe_config->pixel_multiplier =
intel_de_read(dev_priv,
@@ -5721,7 +4654,8 @@ int intel_get_load_detect_pipe(struct drm_connector *connector,
drm_atomic_state_put(state);
/* let the connector get through one full cycle before testing */
- intel_wait_for_vblank(dev_priv, crtc->pipe);
+ intel_crtc_wait_for_next_vblank(crtc);
+
return true;
fail:
@@ -5782,8 +4716,8 @@ static int i9xx_pll_refclk(struct drm_device *dev,
}
/* Returns the clock of the currently programmed mode of the given pipe. */
-static void i9xx_crtc_clock_get(struct intel_crtc *crtc,
- struct intel_crtc_state *pipe_config)
+void i9xx_crtc_clock_get(struct intel_crtc *crtc,
+ struct intel_crtc_state *pipe_config)
{
struct drm_device *dev = crtc->base.dev;
struct drm_i915_private *dev_priv = to_i915(dev);
@@ -5893,24 +4827,6 @@ int intel_dotclock_calculate(int link_freq,
return div_u64(mul_u32_u32(m_n->link_m, link_freq), m_n->link_n);
}
-static void ilk_pch_clock_get(struct intel_crtc *crtc,
- struct intel_crtc_state *pipe_config)
-{
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
-
- /* read out port_clock from the DPLL */
- i9xx_crtc_clock_get(crtc, pipe_config);
-
- /*
- * In case there is an active pipe without active ports,
- * we may need some idea for the dotclock anyway.
- * Calculate one based on the FDI configuration.
- */
- pipe_config->hw.adjusted_mode.crtc_clock =
- intel_dotclock_calculate(intel_fdi_link_freq(dev_priv, pipe_config),
- &pipe_config->fdi_m_n);
-}
-
/* Returns the currently programmed mode of the given encoder. */
struct drm_display_mode *
intel_encoder_current_mode(struct intel_encoder *encoder)
@@ -5924,7 +4840,7 @@ intel_encoder_current_mode(struct intel_encoder *encoder)
if (!encoder->get_hw_state(encoder, &pipe))
return NULL;
- crtc = intel_get_crtc_for_pipe(dev_priv, pipe);
+ crtc = intel_crtc_for_pipe(dev_priv, pipe);
mode = kzalloc(sizeof(*mode), GFP_KERNEL);
if (!mode)
@@ -6245,6 +5161,7 @@ static int icl_check_nv12_planes(struct intel_crtc_state *crtc_state)
linked_state->ctl = plane_state->ctl | PLANE_CTL_YUV420_Y_PLANE;
linked_state->color_ctl = plane_state->color_ctl;
linked_state->view = plane_state->view;
+ linked_state->decrypt = plane_state->decrypt;
intel_plane_copy_hw_state(linked_state, plane_state);
linked_state->uapi.src = plane_state->uapi.src;
@@ -6252,13 +5169,13 @@ static int icl_check_nv12_planes(struct intel_crtc_state *crtc_state)
if (icl_is_hdr_plane(dev_priv, plane->id)) {
if (linked->id == PLANE_SPRITE5)
- plane_state->cus_ctl |= PLANE_CUS_PLANE_7;
+ plane_state->cus_ctl |= PLANE_CUS_Y_PLANE_7_ICL;
else if (linked->id == PLANE_SPRITE4)
- plane_state->cus_ctl |= PLANE_CUS_PLANE_6;
+ plane_state->cus_ctl |= PLANE_CUS_Y_PLANE_6_ICL;
else if (linked->id == PLANE_SPRITE3)
- plane_state->cus_ctl |= PLANE_CUS_PLANE_5_RKL;
+ plane_state->cus_ctl |= PLANE_CUS_Y_PLANE_5_RKL;
else if (linked->id == PLANE_SPRITE2)
- plane_state->cus_ctl |= PLANE_CUS_PLANE_4_RKL;
+ plane_state->cus_ctl |= PLANE_CUS_Y_PLANE_4_RKL;
else
MISSING_CASE(linked->id);
}
@@ -6371,8 +5288,6 @@ static int intel_crtc_atomic_check(struct intel_atomic_state *state,
crtc_state->update_wm_post = true;
if (mode_changed && crtc_state->hw.enable &&
- dev_priv->dpll_funcs &&
- !crtc_state->bigjoiner_slave &&
!drm_WARN_ON(&dev_priv->drm, crtc_state->shared_dpll)) {
ret = dev_priv->dpll_funcs->crtc_compute_clock(crtc_state);
if (ret)
@@ -6928,18 +5843,15 @@ static void
intel_crtc_copy_uapi_to_hw_state_nomodeset(struct intel_atomic_state *state,
struct intel_crtc_state *crtc_state)
{
- const struct intel_crtc_state *from_crtc_state = crtc_state;
+ const struct intel_crtc_state *master_crtc_state;
+ struct intel_crtc *master_crtc;
- if (crtc_state->bigjoiner_slave) {
- from_crtc_state = intel_atomic_get_new_crtc_state(state,
- crtc_state->bigjoiner_linked_crtc);
+ master_crtc = intel_master_crtc(crtc_state);
+ master_crtc_state = intel_atomic_get_new_crtc_state(state, master_crtc);
- /* No need to copy state if the master state is unchanged */
- if (!from_crtc_state)
- return;
- }
-
- intel_crtc_copy_color_blobs(crtc_state, from_crtc_state);
+ /* No need to copy state if the master state is unchanged */
+ if (master_crtc_state)
+ intel_crtc_copy_color_blobs(crtc_state, master_crtc_state);
}
static void
@@ -6982,7 +5894,6 @@ copy_bigjoiner_crtc_state(struct intel_crtc_state *crtc_state,
const struct intel_crtc_state *from_crtc_state)
{
struct intel_crtc_state *saved_state;
- struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
saved_state = kmemdup(from_crtc_state, sizeof(*saved_state), GFP_KERNEL);
if (!saved_state)
@@ -7012,8 +5923,8 @@ copy_bigjoiner_crtc_state(struct intel_crtc_state *crtc_state,
crtc_state->nv12_planes = crtc_state->c8_planes = crtc_state->update_planes = 0;
crtc_state->bigjoiner_linked_crtc = to_intel_crtc(from_crtc_state->uapi.crtc);
crtc_state->bigjoiner_slave = true;
- crtc_state->cpu_transcoder = (enum transcoder)crtc->pipe;
- crtc_state->has_audio = false;
+ crtc_state->cpu_transcoder = from_crtc_state->cpu_transcoder;
+ crtc_state->has_audio = from_crtc_state->has_audio;
return 0;
}
@@ -7609,51 +6520,48 @@ intel_pipe_config_compare(const struct intel_crtc_state *current_config,
PIPE_CONF_CHECK_X(output_types);
- /* FIXME do the readout properly and get rid of this quirk */
- if (!PIPE_CONF_QUIRK(PIPE_CONFIG_QUIRK_BIGJOINER_SLAVE)) {
- PIPE_CONF_CHECK_I(hw.pipe_mode.crtc_hdisplay);
- PIPE_CONF_CHECK_I(hw.pipe_mode.crtc_htotal);
- PIPE_CONF_CHECK_I(hw.pipe_mode.crtc_hblank_start);
- PIPE_CONF_CHECK_I(hw.pipe_mode.crtc_hblank_end);
- PIPE_CONF_CHECK_I(hw.pipe_mode.crtc_hsync_start);
- PIPE_CONF_CHECK_I(hw.pipe_mode.crtc_hsync_end);
+ PIPE_CONF_CHECK_I(hw.pipe_mode.crtc_hdisplay);
+ PIPE_CONF_CHECK_I(hw.pipe_mode.crtc_htotal);
+ PIPE_CONF_CHECK_I(hw.pipe_mode.crtc_hblank_start);
+ PIPE_CONF_CHECK_I(hw.pipe_mode.crtc_hblank_end);
+ PIPE_CONF_CHECK_I(hw.pipe_mode.crtc_hsync_start);
+ PIPE_CONF_CHECK_I(hw.pipe_mode.crtc_hsync_end);
- PIPE_CONF_CHECK_I(hw.pipe_mode.crtc_vdisplay);
- PIPE_CONF_CHECK_I(hw.pipe_mode.crtc_vtotal);
- PIPE_CONF_CHECK_I(hw.pipe_mode.crtc_vblank_start);
- PIPE_CONF_CHECK_I(hw.pipe_mode.crtc_vblank_end);
- PIPE_CONF_CHECK_I(hw.pipe_mode.crtc_vsync_start);
- PIPE_CONF_CHECK_I(hw.pipe_mode.crtc_vsync_end);
+ PIPE_CONF_CHECK_I(hw.pipe_mode.crtc_vdisplay);
+ PIPE_CONF_CHECK_I(hw.pipe_mode.crtc_vtotal);
+ PIPE_CONF_CHECK_I(hw.pipe_mode.crtc_vblank_start);
+ PIPE_CONF_CHECK_I(hw.pipe_mode.crtc_vblank_end);
+ PIPE_CONF_CHECK_I(hw.pipe_mode.crtc_vsync_start);
+ PIPE_CONF_CHECK_I(hw.pipe_mode.crtc_vsync_end);
- PIPE_CONF_CHECK_I(hw.adjusted_mode.crtc_hdisplay);
- PIPE_CONF_CHECK_I(hw.adjusted_mode.crtc_htotal);
- PIPE_CONF_CHECK_I(hw.adjusted_mode.crtc_hblank_start);
- PIPE_CONF_CHECK_I(hw.adjusted_mode.crtc_hblank_end);
- PIPE_CONF_CHECK_I(hw.adjusted_mode.crtc_hsync_start);
- PIPE_CONF_CHECK_I(hw.adjusted_mode.crtc_hsync_end);
+ PIPE_CONF_CHECK_I(hw.adjusted_mode.crtc_hdisplay);
+ PIPE_CONF_CHECK_I(hw.adjusted_mode.crtc_htotal);
+ PIPE_CONF_CHECK_I(hw.adjusted_mode.crtc_hblank_start);
+ PIPE_CONF_CHECK_I(hw.adjusted_mode.crtc_hblank_end);
+ PIPE_CONF_CHECK_I(hw.adjusted_mode.crtc_hsync_start);
+ PIPE_CONF_CHECK_I(hw.adjusted_mode.crtc_hsync_end);
- PIPE_CONF_CHECK_I(hw.adjusted_mode.crtc_vdisplay);
- PIPE_CONF_CHECK_I(hw.adjusted_mode.crtc_vtotal);
- PIPE_CONF_CHECK_I(hw.adjusted_mode.crtc_vblank_start);
- PIPE_CONF_CHECK_I(hw.adjusted_mode.crtc_vblank_end);
- PIPE_CONF_CHECK_I(hw.adjusted_mode.crtc_vsync_start);
- PIPE_CONF_CHECK_I(hw.adjusted_mode.crtc_vsync_end);
+ PIPE_CONF_CHECK_I(hw.adjusted_mode.crtc_vdisplay);
+ PIPE_CONF_CHECK_I(hw.adjusted_mode.crtc_vtotal);
+ PIPE_CONF_CHECK_I(hw.adjusted_mode.crtc_vblank_start);
+ PIPE_CONF_CHECK_I(hw.adjusted_mode.crtc_vblank_end);
+ PIPE_CONF_CHECK_I(hw.adjusted_mode.crtc_vsync_start);
+ PIPE_CONF_CHECK_I(hw.adjusted_mode.crtc_vsync_end);
- PIPE_CONF_CHECK_I(pixel_multiplier);
+ PIPE_CONF_CHECK_I(pixel_multiplier);
+ PIPE_CONF_CHECK_FLAGS(hw.adjusted_mode.flags,
+ DRM_MODE_FLAG_INTERLACE);
+
+ if (!PIPE_CONF_QUIRK(PIPE_CONFIG_QUIRK_MODE_SYNC_FLAGS)) {
PIPE_CONF_CHECK_FLAGS(hw.adjusted_mode.flags,
- DRM_MODE_FLAG_INTERLACE);
-
- if (!PIPE_CONF_QUIRK(PIPE_CONFIG_QUIRK_MODE_SYNC_FLAGS)) {
- PIPE_CONF_CHECK_FLAGS(hw.adjusted_mode.flags,
- DRM_MODE_FLAG_PHSYNC);
- PIPE_CONF_CHECK_FLAGS(hw.adjusted_mode.flags,
- DRM_MODE_FLAG_NHSYNC);
- PIPE_CONF_CHECK_FLAGS(hw.adjusted_mode.flags,
- DRM_MODE_FLAG_PVSYNC);
- PIPE_CONF_CHECK_FLAGS(hw.adjusted_mode.flags,
- DRM_MODE_FLAG_NVSYNC);
- }
+ DRM_MODE_FLAG_PHSYNC);
+ PIPE_CONF_CHECK_FLAGS(hw.adjusted_mode.flags,
+ DRM_MODE_FLAG_NHSYNC);
+ PIPE_CONF_CHECK_FLAGS(hw.adjusted_mode.flags,
+ DRM_MODE_FLAG_PVSYNC);
+ PIPE_CONF_CHECK_FLAGS(hw.adjusted_mode.flags,
+ DRM_MODE_FLAG_NVSYNC);
}
PIPE_CONF_CHECK_I(output_format);
@@ -7665,9 +6573,7 @@ intel_pipe_config_compare(const struct intel_crtc_state *current_config,
PIPE_CONF_CHECK_BOOL(hdmi_scrambling);
PIPE_CONF_CHECK_BOOL(hdmi_high_tmds_clock_ratio);
PIPE_CONF_CHECK_BOOL(has_infoframe);
- /* FIXME do the readout properly and get rid of this quirk */
- if (!PIPE_CONF_QUIRK(PIPE_CONFIG_QUIRK_BIGJOINER_SLAVE))
- PIPE_CONF_CHECK_BOOL(fec_enable);
+ PIPE_CONF_CHECK_BOOL(fec_enable);
PIPE_CONF_CHECK_BOOL_INCOMPLETE(has_audio);
@@ -7696,9 +6602,7 @@ intel_pipe_config_compare(const struct intel_crtc_state *current_config,
}
PIPE_CONF_CHECK_I(scaler_state.scaler_id);
- /* FIXME do the readout properly and get rid of this quirk */
- if (!PIPE_CONF_QUIRK(PIPE_CONFIG_QUIRK_BIGJOINER_SLAVE))
- PIPE_CONF_CHECK_CLOCK_FUZZY(pixel_rate);
+ PIPE_CONF_CHECK_CLOCK_FUZZY(pixel_rate);
PIPE_CONF_CHECK_X(gamma_mode);
if (IS_CHERRYVIEW(dev_priv))
@@ -7725,11 +6629,9 @@ intel_pipe_config_compare(const struct intel_crtc_state *current_config,
PIPE_CONF_CHECK_BOOL(double_wide);
- if (dev_priv->dpll.mgr)
+ if (dev_priv->dpll.mgr) {
PIPE_CONF_CHECK_P(shared_dpll);
- /* FIXME do the readout properly and get rid of this quirk */
- if (dev_priv->dpll.mgr && !PIPE_CONF_QUIRK(PIPE_CONFIG_QUIRK_BIGJOINER_SLAVE)) {
PIPE_CONF_CHECK_X(dpll_hw_state.dpll);
PIPE_CONF_CHECK_X(dpll_hw_state.dpll_md);
PIPE_CONF_CHECK_X(dpll_hw_state.fp0);
@@ -7763,19 +6665,17 @@ intel_pipe_config_compare(const struct intel_crtc_state *current_config,
PIPE_CONF_CHECK_X(dpll_hw_state.mg_pll_tdc_coldst_bias);
}
- if (!PIPE_CONF_QUIRK(PIPE_CONFIG_QUIRK_BIGJOINER_SLAVE)) {
- PIPE_CONF_CHECK_X(dsi_pll.ctrl);
- PIPE_CONF_CHECK_X(dsi_pll.div);
+ PIPE_CONF_CHECK_X(dsi_pll.ctrl);
+ PIPE_CONF_CHECK_X(dsi_pll.div);
- if (IS_G4X(dev_priv) || DISPLAY_VER(dev_priv) >= 5)
- PIPE_CONF_CHECK_I(pipe_bpp);
+ if (IS_G4X(dev_priv) || DISPLAY_VER(dev_priv) >= 5)
+ PIPE_CONF_CHECK_I(pipe_bpp);
- PIPE_CONF_CHECK_CLOCK_FUZZY(hw.pipe_mode.crtc_clock);
- PIPE_CONF_CHECK_CLOCK_FUZZY(hw.adjusted_mode.crtc_clock);
- PIPE_CONF_CHECK_CLOCK_FUZZY(port_clock);
+ PIPE_CONF_CHECK_CLOCK_FUZZY(hw.pipe_mode.crtc_clock);
+ PIPE_CONF_CHECK_CLOCK_FUZZY(hw.adjusted_mode.crtc_clock);
+ PIPE_CONF_CHECK_CLOCK_FUZZY(port_clock);
- PIPE_CONF_CHECK_I(min_voltage_level);
- }
+ PIPE_CONF_CHECK_I(min_voltage_level);
if (current_config->has_psr || pipe_config->has_psr)
PIPE_CONF_CHECK_X_WITH_MASK(infoframes.enable,
@@ -8049,7 +6949,7 @@ verify_crtc_state(struct intel_crtc *crtc,
struct intel_encoder *encoder;
struct intel_crtc_state *pipe_config = old_crtc_state;
struct drm_atomic_state *state = old_crtc_state->uapi.state;
- struct intel_crtc *master = crtc;
+ struct intel_crtc *master_crtc;
__drm_atomic_helper_crtc_destroy_state(&old_crtc_state->uapi);
intel_crtc_free_hw_state(old_crtc_state);
@@ -8077,10 +6977,9 @@ verify_crtc_state(struct intel_crtc *crtc,
"(expected %i, found %i)\n",
new_crtc_state->hw.active, crtc->active);
- if (new_crtc_state->bigjoiner_slave)
- master = new_crtc_state->bigjoiner_linked_crtc;
+ master_crtc = intel_master_crtc(new_crtc_state);
- for_each_encoder_on_crtc(dev, &master->base, encoder) {
+ for_each_encoder_on_crtc(dev, &master_crtc->base, encoder) {
enum pipe pipe;
bool active;
@@ -8090,7 +6989,7 @@ verify_crtc_state(struct intel_crtc *crtc,
encoder->base.base.id, active,
new_crtc_state->hw.active);
- I915_STATE_WARN(active && master->pipe != pipe,
+ I915_STATE_WARN(active && master_crtc->pipe != pipe,
"Encoder connected to wrong pipe %c\n",
pipe_name(pipe));
@@ -8101,10 +7000,6 @@ verify_crtc_state(struct intel_crtc *crtc,
if (!new_crtc_state->hw.active)
return;
- if (new_crtc_state->bigjoiner_slave)
- /* No PLLs set for slave */
- pipe_config->shared_dpll = NULL;
-
intel_pipe_config_sanity_check(dev_priv, pipe_config);
if (!intel_pipe_config_compare(new_crtc_state,
@@ -8223,9 +7118,6 @@ verify_mpllb_state(struct intel_atomic_state *state,
if (!new_crtc_state->hw.active)
return;
- if (new_crtc_state->bigjoiner_slave)
- return;
-
encoder = intel_get_crtc_new_encoder(state, new_crtc_state);
intel_mpllb_readout_hw_state(encoder, &mpllb_hw_state);
@@ -8607,28 +7499,13 @@ static int intel_bigjoiner_add_affected_planes(struct intel_atomic_state *state)
return 0;
}
-static bool bo_has_valid_encryption(struct drm_i915_gem_object *obj)
-{
- struct drm_i915_private *i915 = to_i915(obj->base.dev);
-
- return intel_pxp_key_check(&i915->gt.pxp, obj, false) == 0;
-}
-
-static bool pxp_is_borked(struct drm_i915_gem_object *obj)
-{
- return i915_gem_object_is_protected(obj) && !bo_has_valid_encryption(obj);
-}
-
static int intel_atomic_check_planes(struct intel_atomic_state *state)
{
struct drm_i915_private *dev_priv = to_i915(state->base.dev);
struct intel_crtc_state *old_crtc_state, *new_crtc_state;
struct intel_plane_state *plane_state;
struct intel_plane *plane;
- struct intel_plane_state *new_plane_state;
- struct intel_plane_state *old_plane_state;
struct intel_crtc *crtc;
- const struct drm_framebuffer *fb;
int i, ret;
ret = icl_add_linked_planes(state);
@@ -8676,72 +7553,6 @@ static int intel_atomic_check_planes(struct intel_atomic_state *state)
return ret;
}
- for_each_new_intel_plane_in_state(state, plane, plane_state, i) {
- new_plane_state = intel_atomic_get_new_plane_state(state, plane);
- old_plane_state = intel_atomic_get_old_plane_state(state, plane);
- fb = new_plane_state->hw.fb;
- if (fb) {
- new_plane_state->decrypt = bo_has_valid_encryption(intel_fb_obj(fb));
- new_plane_state->force_black = pxp_is_borked(intel_fb_obj(fb));
- } else {
- new_plane_state->decrypt = old_plane_state->decrypt;
- new_plane_state->force_black = old_plane_state->force_black;
- }
- }
-
- return 0;
-}
-
-static int intel_atomic_check_cdclk(struct intel_atomic_state *state,
- bool *need_cdclk_calc)
-{
- struct drm_i915_private *dev_priv = to_i915(state->base.dev);
- const struct intel_cdclk_state *old_cdclk_state;
- const struct intel_cdclk_state *new_cdclk_state;
- struct intel_plane_state *plane_state;
- struct intel_bw_state *new_bw_state;
- struct intel_plane *plane;
- int min_cdclk = 0;
- enum pipe pipe;
- int ret;
- int i;
- /*
- * active_planes bitmask has been updated, and potentially
- * affected planes are part of the state. We can now
- * compute the minimum cdclk for each plane.
- */
- for_each_new_intel_plane_in_state(state, plane, plane_state, i) {
- ret = intel_plane_calc_min_cdclk(state, plane, need_cdclk_calc);
- if (ret)
- return ret;
- }
-
- old_cdclk_state = intel_atomic_get_old_cdclk_state(state);
- new_cdclk_state = intel_atomic_get_new_cdclk_state(state);
-
- if (new_cdclk_state &&
- old_cdclk_state->force_min_cdclk != new_cdclk_state->force_min_cdclk)
- *need_cdclk_calc = true;
-
- ret = intel_cdclk_bw_calc_min_cdclk(state);
- if (ret)
- return ret;
-
- new_bw_state = intel_atomic_get_new_bw_state(state);
-
- if (!new_cdclk_state || !new_bw_state)
- return 0;
-
- for_each_pipe(dev_priv, pipe) {
- min_cdclk = max(new_cdclk_state->min_cdclk[pipe], min_cdclk);
-
- /*
- * Currently do this change only if we need to increase
- */
- if (new_bw_state->min_cdclk > min_cdclk)
- *need_cdclk_calc = true;
- }
-
return 0;
}
@@ -8790,13 +7601,13 @@ static int intel_atomic_check_bigjoiner(struct intel_atomic_state *state,
struct intel_crtc_state *new_crtc_state)
{
struct intel_crtc_state *slave_crtc_state, *master_crtc_state;
- struct intel_crtc *slave, *master;
+ struct intel_crtc *slave_crtc, *master_crtc;
/* slave being enabled, is master is still claiming this crtc? */
if (old_crtc_state->bigjoiner_slave) {
- slave = crtc;
- master = old_crtc_state->bigjoiner_linked_crtc;
- master_crtc_state = intel_atomic_get_new_crtc_state(state, master);
+ slave_crtc = crtc;
+ master_crtc = old_crtc_state->bigjoiner_linked_crtc;
+ master_crtc_state = intel_atomic_get_new_crtc_state(state, master_crtc);
if (!master_crtc_state || !intel_crtc_needs_modeset(master_crtc_state))
goto claimed;
}
@@ -8804,17 +7615,17 @@ static int intel_atomic_check_bigjoiner(struct intel_atomic_state *state,
if (!new_crtc_state->bigjoiner)
return 0;
- slave = intel_dsc_get_bigjoiner_secondary(crtc);
- if (!slave) {
+ slave_crtc = intel_dsc_get_bigjoiner_secondary(crtc);
+ if (!slave_crtc) {
DRM_DEBUG_KMS("[CRTC:%d:%s] Big joiner configuration requires "
"CRTC + 1 to be used, doesn't exist\n",
crtc->base.base.id, crtc->base.name);
return -EINVAL;
}
- new_crtc_state->bigjoiner_linked_crtc = slave;
- slave_crtc_state = intel_atomic_get_crtc_state(&state->base, slave);
- master = crtc;
+ new_crtc_state->bigjoiner_linked_crtc = slave_crtc;
+ slave_crtc_state = intel_atomic_get_crtc_state(&state->base, slave_crtc);
+ master_crtc = crtc;
if (IS_ERR(slave_crtc_state))
return PTR_ERR(slave_crtc_state);
@@ -8823,15 +7634,15 @@ static int intel_atomic_check_bigjoiner(struct intel_atomic_state *state,
goto claimed;
DRM_DEBUG_KMS("[CRTC:%d:%s] Used as slave for big joiner\n",
- slave->base.base.id, slave->base.name);
+ slave_crtc->base.base.id, slave_crtc->base.name);
return copy_bigjoiner_crtc_state(slave_crtc_state, new_crtc_state);
claimed:
DRM_DEBUG_KMS("[CRTC:%d:%s] Slave is enabled as normal CRTC, but "
"[CRTC:%d:%s] claiming this CRTC for bigjoiner.\n",
- slave->base.base.id, slave->base.name,
- master->base.base.id, master->base.name);
+ slave_crtc->base.base.id, slave_crtc->base.name,
+ master_crtc->base.base.id, master_crtc->base.name);
return -EINVAL;
}
@@ -8865,35 +7676,37 @@ static void kill_bigjoiner_slave(struct intel_atomic_state *state,
* correspond to the last vblank and have no relation to the actual time when
* the flip done event was sent.
*/
-static int intel_atomic_check_async(struct intel_atomic_state *state)
+static int intel_atomic_check_async(struct intel_atomic_state *state, struct intel_crtc *crtc)
{
struct drm_i915_private *i915 = to_i915(state->base.dev);
const struct intel_crtc_state *old_crtc_state, *new_crtc_state;
const struct intel_plane_state *new_plane_state, *old_plane_state;
- struct intel_crtc *crtc;
struct intel_plane *plane;
int i;
- for_each_oldnew_intel_crtc_in_state(state, crtc, old_crtc_state,
- new_crtc_state, i) {
- if (intel_crtc_needs_modeset(new_crtc_state)) {
- drm_dbg_kms(&i915->drm, "Modeset Required. Async flip not supported\n");
- return -EINVAL;
- }
+ old_crtc_state = intel_atomic_get_old_crtc_state(state, crtc);
+ new_crtc_state = intel_atomic_get_new_crtc_state(state, crtc);
- if (!new_crtc_state->hw.active) {
- drm_dbg_kms(&i915->drm, "CRTC inactive\n");
- return -EINVAL;
- }
- if (old_crtc_state->active_planes != new_crtc_state->active_planes) {
- drm_dbg_kms(&i915->drm,
- "Active planes cannot be changed during async flip\n");
- return -EINVAL;
- }
+ if (intel_crtc_needs_modeset(new_crtc_state)) {
+ drm_dbg_kms(&i915->drm, "Modeset Required. Async flip not supported\n");
+ return -EINVAL;
+ }
+
+ if (!new_crtc_state->hw.active) {
+ drm_dbg_kms(&i915->drm, "CRTC inactive\n");
+ return -EINVAL;
+ }
+ if (old_crtc_state->active_planes != new_crtc_state->active_planes) {
+ drm_dbg_kms(&i915->drm,
+ "Active planes cannot be changed during async flip\n");
+ return -EINVAL;
}
for_each_oldnew_intel_plane_in_state(state, plane, old_plane_state,
new_plane_state, i) {
+ if (plane->pipe != crtc->pipe)
+ continue;
+
/*
* TODO: Async flip is only supported through the page flip IOCTL
* as of now. So support currently added for primary plane only.
@@ -8920,8 +7733,14 @@ static int intel_atomic_check_async(struct intel_atomic_state *state)
return -EINVAL;
}
- if (old_plane_state->view.color_plane[0].stride !=
- new_plane_state->view.color_plane[0].stride) {
+ if (new_plane_state->hw.fb->format->num_planes > 1) {
+ drm_dbg_kms(&i915->drm,
+ "Planar formats not supported with async flips\n");
+ return -EINVAL;
+ }
+
+ if (old_plane_state->view.color_plane[0].mapping_stride !=
+ new_plane_state->view.color_plane[0].mapping_stride) {
drm_dbg_kms(&i915->drm, "Stride cannot be changed in async flip\n");
return -EINVAL;
}
@@ -9177,7 +7996,6 @@ static int intel_atomic_check(struct drm_device *dev,
if (ret)
goto fail;
- intel_fbc_choose_crtc(dev_priv, state);
ret = intel_compute_global_watermarks(state);
if (ret)
goto fail;
@@ -9186,7 +8004,7 @@ static int intel_atomic_check(struct drm_device *dev,
if (ret)
goto fail;
- ret = intel_atomic_check_cdclk(state, &any_ms);
+ ret = intel_cdclk_atomic_check(state, &any_ms);
if (ret)
goto fail;
@@ -9209,10 +8027,14 @@ static int intel_atomic_check(struct drm_device *dev,
if (ret)
goto fail;
+ ret = intel_fbc_atomic_check(state);
+ if (ret)
+ goto fail;
+
for_each_oldnew_intel_crtc_in_state(state, crtc, old_crtc_state,
new_crtc_state, i) {
if (new_crtc_state->uapi.async_flip) {
- ret = intel_atomic_check_async(state);
+ ret = intel_atomic_check_async(state, crtc);
if (ret)
goto fail;
}
@@ -9417,15 +8239,17 @@ static void intel_update_crtc(struct intel_atomic_state *state,
intel_fbc_update(state, crtc);
+ intel_update_planes_on_crtc(state, crtc);
+
/* Perform vblank evasion around commit operation */
intel_pipe_update_start(new_crtc_state);
commit_pipe_pre_planes(state, crtc);
if (DISPLAY_VER(dev_priv) >= 9)
- skl_update_planes_on_crtc(state, crtc);
+ skl_arm_planes_on_crtc(state, crtc);
else
- i9xx_update_planes_on_crtc(state, crtc);
+ i9xx_arm_planes_on_crtc(state, crtc);
commit_pipe_post_planes(state, crtc);
@@ -9449,23 +8273,6 @@ static void intel_old_crtc_state_disables(struct intel_atomic_state *state,
{
struct drm_i915_private *dev_priv = to_i915(state->base.dev);
- drm_WARN_ON(&dev_priv->drm, old_crtc_state->bigjoiner_slave);
-
- intel_encoders_pre_disable(state, crtc);
-
- intel_crtc_disable_planes(state, crtc);
-
- /*
- * We still need special handling for disabling bigjoiner master
- * and slaves since for slave we do not have encoder or plls
- * so we dont need to disable those.
- */
- if (old_crtc_state->bigjoiner) {
- intel_crtc_disable_planes(state,
- old_crtc_state->bigjoiner_linked_crtc);
- old_crtc_state->bigjoiner_linked_crtc->active = false;
- }
-
/*
* We need to disable pipe CRC before disabling the pipe,
* or we race against vblank off.
@@ -9490,10 +8297,22 @@ static void intel_commit_modeset_disables(struct intel_atomic_state *state)
u32 handled = 0;
int i;
+ for_each_oldnew_intel_crtc_in_state(state, crtc, old_crtc_state,
+ new_crtc_state, i) {
+ if (!intel_crtc_needs_modeset(new_crtc_state))
+ continue;
+
+ if (!old_crtc_state->hw.active)
+ continue;
+
+ intel_pre_plane_update(state, crtc);
+ intel_crtc_disable_planes(state, crtc);
+ }
+
/* Only disable port sync and MST slaves */
for_each_oldnew_intel_crtc_in_state(state, crtc, old_crtc_state,
new_crtc_state, i) {
- if (!intel_crtc_needs_modeset(new_crtc_state) || old_crtc_state->bigjoiner)
+ if (!intel_crtc_needs_modeset(new_crtc_state))
continue;
if (!old_crtc_state->hw.active)
@@ -9505,10 +8324,10 @@ static void intel_commit_modeset_disables(struct intel_atomic_state *state)
* Slave vblanks are masked till Master Vblanks.
*/
if (!is_trans_port_sync_slave(old_crtc_state) &&
- !intel_dp_mst_is_slave_trans(old_crtc_state))
+ !intel_dp_mst_is_slave_trans(old_crtc_state) &&
+ !old_crtc_state->bigjoiner_slave)
continue;
- intel_pre_plane_update(state, crtc);
intel_old_crtc_state_disables(state, old_crtc_state,
new_crtc_state, crtc);
handled |= BIT(crtc->pipe);
@@ -9518,21 +8337,14 @@ static void intel_commit_modeset_disables(struct intel_atomic_state *state)
for_each_oldnew_intel_crtc_in_state(state, crtc, old_crtc_state,
new_crtc_state, i) {
if (!intel_crtc_needs_modeset(new_crtc_state) ||
- (handled & BIT(crtc->pipe)) ||
- old_crtc_state->bigjoiner_slave)
+ (handled & BIT(crtc->pipe)))
continue;
- intel_pre_plane_update(state, crtc);
- if (old_crtc_state->bigjoiner) {
- struct intel_crtc *slave =
- old_crtc_state->bigjoiner_linked_crtc;
+ if (!old_crtc_state->hw.active)
+ continue;
- intel_pre_plane_update(state, slave);
- }
-
- if (old_crtc_state->hw.active)
- intel_old_crtc_state_disables(state, old_crtc_state,
- new_crtc_state, crtc);
+ intel_old_crtc_state_disables(state, old_crtc_state,
+ new_crtc_state, crtc);
}
}
@@ -9610,7 +8422,7 @@ static void skl_commit_modeset_enables(struct intel_atomic_state *state)
if (!skl_ddb_entry_equal(&new_crtc_state->wm.skl.ddb,
&old_crtc_state->wm.skl.ddb) &&
(update_pipes | modeset_pipes))
- intel_wait_for_vblank(dev_priv, pipe);
+ intel_crtc_wait_for_next_vblank(crtc);
}
}
@@ -9701,19 +8513,19 @@ static void intel_atomic_commit_fence_wait(struct intel_atomic_state *intel_stat
for (;;) {
prepare_to_wait(&intel_state->commit_ready.wait,
&wait_fence, TASK_UNINTERRUPTIBLE);
- prepare_to_wait(bit_waitqueue(&dev_priv->gt.reset.flags,
+ prepare_to_wait(bit_waitqueue(&to_gt(dev_priv)->reset.flags,
I915_RESET_MODESET),
&wait_reset, TASK_UNINTERRUPTIBLE);
if (i915_sw_fence_done(&intel_state->commit_ready) ||
- test_bit(I915_RESET_MODESET, &dev_priv->gt.reset.flags))
+ test_bit(I915_RESET_MODESET, &to_gt(dev_priv)->reset.flags))
break;
schedule();
}
finish_wait(&intel_state->commit_ready.wait, &wait_fence);
- finish_wait(bit_waitqueue(&dev_priv->gt.reset.flags,
+ finish_wait(bit_waitqueue(&to_gt(dev_priv)->reset.flags,
I915_RESET_MODESET),
&wait_reset);
}
@@ -9752,10 +8564,14 @@ static void intel_atomic_prepare_plane_clear_colors(struct intel_atomic_state *s
for_each_new_intel_plane_in_state(state, plane, plane_state, i) {
struct drm_framebuffer *fb = plane_state->hw.fb;
+ int cc_plane;
int ret;
- if (!fb ||
- fb->modifier != I915_FORMAT_MOD_Y_TILED_GEN12_RC_CCS_CC)
+ if (!fb)
+ continue;
+
+ cc_plane = intel_fb_rc_ccs_cc_plane(fb);
+ if (cc_plane < 0)
continue;
/*
@@ -9772,7 +8588,7 @@ static void intel_atomic_prepare_plane_clear_colors(struct intel_atomic_state *s
* GPU write on it.
*/
ret = i915_gem_object_read_from_page(intel_fb_obj(fb),
- fb->offsets[2] + 16,
+ fb->offsets[cc_plane] + 16,
&plane_state->ccval,
sizeof(plane_state->ccval));
/* The above could only fail if the FB obj has an unexpected backing store type. */
@@ -9840,11 +8656,9 @@ static void intel_atomic_commit_tail(struct intel_atomic_state *state)
}
}
- if (state->modeset)
- intel_encoders_update_prepare(state);
+ intel_encoders_update_prepare(state);
intel_dbuf_pre_plane_update(state);
- intel_psr_pre_plane_update(state);
for_each_new_intel_crtc_in_state(state, crtc, new_crtc_state, i) {
if (new_crtc_state->uapi.async_flip)
@@ -9854,11 +8668,12 @@ static void intel_atomic_commit_tail(struct intel_atomic_state *state)
/* Now enable the clocks, plane, pipe, and connectors that we set up. */
dev_priv->display->commit_modeset_enables(state);
- if (state->modeset) {
- intel_encoders_update_complete(state);
+ intel_encoders_update_complete(state);
+ if (state->modeset)
intel_set_cdclk_post_plane_update(state);
- }
+
+ intel_wait_for_vblank_workers(state);
/* FIXME: We should call drm_atomic_helper_commit_hw_done() here
* already, but still need the state for the delayed optimization. To
@@ -9874,13 +8689,6 @@ static void intel_atomic_commit_tail(struct intel_atomic_state *state)
for_each_new_intel_crtc_in_state(state, crtc, new_crtc_state, i) {
if (new_crtc_state->uapi.async_flip)
intel_crtc_disable_flip_done(state, crtc);
-
- if (new_crtc_state->hw.active &&
- !intel_crtc_needs_modeset(new_crtc_state) &&
- !new_crtc_state->preload_luts &&
- (new_crtc_state->uapi.color_mgmt_changed ||
- new_crtc_state->update_pipe))
- intel_color_load_luts(new_crtc_state);
}
/*
@@ -9967,7 +8775,7 @@ static void intel_atomic_commit_work(struct work_struct *work)
intel_atomic_commit_tail(state);
}
-static int __i915_sw_fence_call
+static int
intel_atomic_commit_ready(struct i915_sw_fence *fence,
enum i915_sw_fence_notify notify)
{
@@ -10114,8 +8922,8 @@ static void intel_plane_possible_crtcs_init(struct drm_i915_private *dev_priv)
struct intel_plane *plane;
for_each_intel_plane(&dev_priv->drm, plane) {
- struct intel_crtc *crtc = intel_get_crtc_for_pipe(dev_priv,
- plane->pipe);
+ struct intel_crtc *crtc = intel_crtc_for_pipe(dev_priv,
+ plane->pipe);
plane->base.possible_crtcs = drm_crtc_mask(&crtc->base);
}
@@ -10580,7 +9388,7 @@ intel_mode_valid_max_plane_size(struct drm_i915_private *dev_priv,
static const struct drm_mode_config_funcs intel_mode_funcs = {
.fb_create = intel_user_framebuffer_create,
- .get_format_info = intel_get_format_info,
+ .get_format_info = intel_fb_get_format_info,
.output_poll_changed = intel_fbdev_output_poll_changed,
.mode_valid = intel_mode_valid,
.atomic_check = intel_atomic_check,
@@ -10640,7 +9448,7 @@ void intel_init_display_hooks(struct drm_i915_private *dev_priv)
return;
intel_init_cdclk_hooks(dev_priv);
- intel_init_audio_hooks(dev_priv);
+ intel_audio_hooks_init(dev_priv);
intel_dpll_init_clock_hook(dev_priv);
@@ -11108,7 +9916,7 @@ int intel_modeset_init(struct drm_i915_private *i915)
void i830_enable_pipe(struct drm_i915_private *dev_priv, enum pipe pipe)
{
- struct intel_crtc *crtc = intel_get_crtc_for_pipe(dev_priv, pipe);
+ struct intel_crtc *crtc = intel_crtc_for_pipe(dev_priv, pipe);
/* 640x480@60Hz, ~25175 kHz */
struct dpll clock = {
.m1 = 18,
@@ -11181,7 +9989,7 @@ void i830_enable_pipe(struct drm_i915_private *dev_priv, enum pipe pipe)
void i830_disable_pipe(struct drm_i915_private *dev_priv, enum pipe pipe)
{
- struct intel_crtc *crtc = intel_get_crtc_for_pipe(dev_priv, pipe);
+ struct intel_crtc *crtc = intel_crtc_for_pipe(dev_priv, pipe);
drm_dbg_kms(&dev_priv->drm, "disabling pipe %c due to force quirk\n",
pipe_name(pipe));
@@ -11233,7 +10041,7 @@ intel_sanitize_plane_mapping(struct drm_i915_private *dev_priv)
"[PLANE:%d:%s] attached to the wrong pipe, disabling plane\n",
plane->base.base.id, plane->base.name);
- plane_crtc = intel_get_crtc_for_pipe(dev_priv, pipe);
+ plane_crtc = intel_crtc_for_pipe(dev_priv, pipe);
intel_plane_disable_noatomic(plane_crtc, plane);
}
}
@@ -11486,7 +10294,7 @@ static void readout_plane_state(struct drm_i915_private *dev_priv)
visible = plane->get_hw_state(plane, &pipe);
- crtc = intel_get_crtc_for_pipe(dev_priv, pipe);
+ crtc = intel_crtc_for_pipe(dev_priv, pipe);
crtc_state = to_intel_crtc_state(crtc->base.state);
intel_set_plane_visible(crtc_state, plane_state, visible);
@@ -11553,7 +10361,7 @@ static void intel_modeset_readout_hw_state(struct drm_device *dev)
pipe = 0;
if (encoder->get_hw_state(encoder, &pipe)) {
- crtc = intel_get_crtc_for_pipe(dev_priv, pipe);
+ crtc = intel_crtc_for_pipe(dev_priv, pipe);
crtc_state = to_intel_crtc_state(crtc->base.state);
encoder->base.crtc = &crtc->base;
@@ -11628,9 +10436,6 @@ static void intel_modeset_readout_hw_state(struct drm_device *dev)
struct intel_plane *plane;
int min_cdclk = 0;
- if (crtc_state->bigjoiner_slave)
- continue;
-
if (crtc_state->hw.active) {
/*
* The initial mode needs to be set in order to keep
@@ -11690,39 +10495,6 @@ static void intel_modeset_readout_hw_state(struct drm_device *dev)
intel_bw_crtc_update(bw_state, crtc_state);
intel_pipe_config_sanity_check(dev_priv, crtc_state);
-
- /* discard our incomplete slave state, copy it from master */
- if (crtc_state->bigjoiner && crtc_state->hw.active) {
- struct intel_crtc *slave = crtc_state->bigjoiner_linked_crtc;
- struct intel_crtc_state *slave_crtc_state =
- to_intel_crtc_state(slave->base.state);
-
- copy_bigjoiner_crtc_state(slave_crtc_state, crtc_state);
- slave->base.mode = crtc->base.mode;
-
- cdclk_state->min_cdclk[slave->pipe] = min_cdclk;
- cdclk_state->min_voltage_level[slave->pipe] =
- crtc_state->min_voltage_level;
-
- for_each_intel_plane_on_crtc(&dev_priv->drm, slave, plane) {
- const struct intel_plane_state *plane_state =
- to_intel_plane_state(plane->base.state);
-
- /*
- * FIXME don't have the fb yet, so can't
- * use intel_plane_data_rate() :(
- */
- if (plane_state->uapi.visible)
- crtc_state->data_rate[plane->id] =
- 4 * crtc_state->pixel_rate;
- else
- crtc_state->data_rate[plane->id] = 0;
- }
-
- intel_bw_crtc_update(bw_state, slave_crtc_state);
- drm_calc_timestamping_constants(&slave->base,
- &slave_crtc_state->hw.adjusted_mode);
- }
}
}
@@ -12027,7 +10799,7 @@ void intel_modeset_driver_remove_noirq(struct drm_i915_private *i915)
destroy_workqueue(i915->flip_wq);
destroy_workqueue(i915->modeset_wq);
- intel_fbc_cleanup_cfb(i915);
+ intel_fbc_cleanup(i915);
}
/* part #3: call after gem init */
@@ -12042,6 +10814,27 @@ void intel_modeset_driver_remove_nogem(struct drm_i915_private *i915)
intel_bios_driver_remove(i915);
}
+bool intel_modeset_probe_defer(struct pci_dev *pdev)
+{
+ struct drm_privacy_screen *privacy_screen;
+
+ /*
+ * apple-gmux is needed on dual GPU MacBook Pro
+ * to probe the panel if we're the inactive GPU.
+ */
+ if (vga_switcheroo_client_probe_defer(pdev))
+ return true;
+
+ /* If the LCD panel has a privacy-screen, wait for it */
+ privacy_screen = drm_privacy_screen_get(&pdev->dev, NULL);
+ if (IS_ERR(privacy_screen) && PTR_ERR(privacy_screen) == -EPROBE_DEFER)
+ return true;
+
+ drm_privacy_screen_put(privacy_screen);
+
+ return false;
+}
+
void intel_display_driver_register(struct drm_i915_private *i915)
{
if (!HAS_DISPLAY(i915))
diff --git a/drivers/gpu/drm/i915/display/intel_display.h b/drivers/gpu/drm/i915/display/intel_display.h
index 0c76bf5..b61b752 100644
--- a/drivers/gpu/drm/i915/display/intel_display.h
+++ b/drivers/gpu/drm/i915/display/intel_display.h
@@ -57,6 +57,7 @@ struct intel_plane;
struct intel_plane_state;
struct intel_remapped_info;
struct intel_rotation_info;
+struct pci_dev;
enum i915_gpio {
GPIOA,
@@ -346,9 +347,33 @@ enum phy_fia {
FIA3,
};
+enum hpd_pin {
+ HPD_NONE = 0,
+ HPD_TV = HPD_NONE, /* TV is known to be unreliable */
+ HPD_CRT,
+ HPD_SDVO_B,
+ HPD_SDVO_C,
+ HPD_PORT_A,
+ HPD_PORT_B,
+ HPD_PORT_C,
+ HPD_PORT_D,
+ HPD_PORT_E,
+ HPD_PORT_TC1,
+ HPD_PORT_TC2,
+ HPD_PORT_TC3,
+ HPD_PORT_TC4,
+ HPD_PORT_TC5,
+ HPD_PORT_TC6,
+
+ HPD_NUM_PINS
+};
+
+#define for_each_hpd_pin(__pin) \
+ for ((__pin) = (HPD_NONE + 1); (__pin) < HPD_NUM_PINS; (__pin)++)
+
#define for_each_pipe(__dev_priv, __p) \
for ((__p) = 0; (__p) < I915_MAX_PIPES; (__p)++) \
- for_each_if(INTEL_INFO(__dev_priv)->pipe_mask & BIT(__p))
+ for_each_if(INTEL_INFO(__dev_priv)->display.pipe_mask & BIT(__p))
#define for_each_pipe_masked(__dev_priv, __p, __mask) \
for_each_pipe(__dev_priv, __p) \
@@ -356,7 +381,7 @@ enum phy_fia {
#define for_each_cpu_transcoder(__dev_priv, __t) \
for ((__t) = 0; (__t) < I915_MAX_TRANSCODERS; (__t)++) \
- for_each_if (INTEL_INFO(__dev_priv)->cpu_transcoder_mask & BIT(__t))
+ for_each_if (INTEL_INFO(__dev_priv)->display.cpu_transcoder_mask & BIT(__t))
#define for_each_cpu_transcoder_masked(__dev_priv, __t, __mask) \
for_each_cpu_transcoder(__dev_priv, __t) \
@@ -521,7 +546,6 @@ void intel_link_compute_m_n(u16 bpp, int nlanes,
int pixel_clock, int link_clock,
struct intel_link_m_n *m_n,
bool constant_n, bool fec_enable);
-void lpt_disable_clkout_dp(struct drm_i915_private *dev_priv);
u32 intel_plane_fb_max_stride(struct drm_i915_private *dev_priv,
u32 pixel_format, u64 modifier);
enum drm_mode_status
@@ -542,9 +566,6 @@ int vlv_get_cck_clock(struct drm_i915_private *dev_priv,
const char *name, u32 reg, int ref_freq);
int vlv_get_cck_clock_hpll(struct drm_i915_private *dev_priv,
const char *name, u32 reg);
-void lpt_pch_enable(const struct intel_crtc_state *crtc_state);
-void lpt_disable_pch_transcoder(struct drm_i915_private *dev_priv);
-void lpt_disable_iclkip(struct drm_i915_private *dev_priv);
void intel_init_display_hooks(struct drm_i915_private *dev_priv);
unsigned int intel_fb_xy_to_linear(int x, int y,
const struct intel_plane_state *state,
@@ -580,10 +601,6 @@ struct drm_framebuffer *
intel_framebuffer_create(struct drm_i915_gem_object *obj,
struct drm_mode_fb_cmd2 *mode_cmd);
-void assert_pch_transcoder_disabled(struct drm_i915_private *dev_priv,
- enum pipe pipe);
-
-int lpt_get_iclkip(struct drm_i915_private *dev_priv);
bool intel_fuzzy_clock_check(int clock1, int clock2);
void intel_display_prepare_reset(struct drm_i915_private *dev_priv);
@@ -592,8 +609,11 @@ void intel_dp_get_m_n(struct intel_crtc *crtc,
struct intel_crtc_state *pipe_config);
void intel_dp_set_m_n(const struct intel_crtc_state *crtc_state,
enum link_m_n_set m_n);
+void ilk_get_fdi_m_n_config(struct intel_crtc *crtc,
+ struct intel_crtc_state *pipe_config);
+void i9xx_crtc_clock_get(struct intel_crtc *crtc,
+ struct intel_crtc_state *pipe_config);
int intel_dotclock_calculate(int link_freq, const struct intel_link_m_n *m_n);
-
bool hsw_crtc_state_ips_capable(const struct intel_crtc_state *crtc_state);
void hsw_enable_ips(const struct intel_crtc_state *crtc_state);
void hsw_disable_ips(const struct intel_crtc_state *crtc_state);
@@ -610,9 +630,6 @@ int bdw_get_pipemisc_bpp(struct intel_crtc *crtc);
unsigned int intel_plane_fence_y_offset(const struct intel_plane_state *plane_state);
bool intel_plane_uses_fence(const struct intel_plane_state *plane_state);
-bool
-intel_format_info_is_yuv_semiplanar(const struct drm_format_info *info,
- u64 modifier);
struct intel_encoder *
intel_get_crtc_new_encoder(const struct intel_atomic_state *state,
@@ -624,6 +641,7 @@ void intel_display_driver_register(struct drm_i915_private *i915);
void intel_display_driver_unregister(struct drm_i915_private *i915);
/* modesetting */
+bool intel_modeset_probe_defer(struct pci_dev *pdev);
void intel_modeset_init_hw(struct drm_i915_private *i915);
int intel_modeset_init_noirq(struct drm_i915_private *i915);
int intel_modeset_init_nogem(struct drm_i915_private *i915);
@@ -632,7 +650,6 @@ void intel_modeset_driver_remove(struct drm_i915_private *i915);
void intel_modeset_driver_remove_noirq(struct drm_i915_private *i915);
void intel_modeset_driver_remove_nogem(struct drm_i915_private *i915);
void intel_display_resume(struct drm_device *dev);
-void intel_init_pch_refclk(struct drm_i915_private *dev_priv);
int intel_modeset_all_pipes(struct intel_atomic_state *state);
/* modesetting asserts */
diff --git a/drivers/gpu/drm/i915/display/intel_display_debugfs.c b/drivers/gpu/drm/i915/display/intel_display_debugfs.c
index e047676..5724452 100644
--- a/drivers/gpu/drm/i915/display/intel_display_debugfs.c
+++ b/drivers/gpu/drm/i915/display/intel_display_debugfs.c
@@ -40,83 +40,6 @@ static int i915_frontbuffer_tracking(struct seq_file *m, void *unused)
return 0;
}
-static int i915_fbc_status(struct seq_file *m, void *unused)
-{
- struct drm_i915_private *dev_priv = node_to_i915(m->private);
- struct intel_fbc *fbc = &dev_priv->fbc;
- intel_wakeref_t wakeref;
-
- if (!HAS_FBC(dev_priv))
- return -ENODEV;
-
- wakeref = intel_runtime_pm_get(&dev_priv->runtime_pm);
- mutex_lock(&fbc->lock);
-
- if (intel_fbc_is_active(dev_priv))
- seq_puts(m, "FBC enabled\n");
- else
- seq_printf(m, "FBC disabled: %s\n", fbc->no_fbc_reason);
-
- if (intel_fbc_is_active(dev_priv)) {
- u32 mask;
-
- if (DISPLAY_VER(dev_priv) >= 8)
- mask = intel_de_read(dev_priv, IVB_FBC_STATUS2) & BDW_FBC_COMP_SEG_MASK;
- else if (DISPLAY_VER(dev_priv) >= 7)
- mask = intel_de_read(dev_priv, IVB_FBC_STATUS2) & IVB_FBC_COMP_SEG_MASK;
- else if (DISPLAY_VER(dev_priv) >= 5)
- mask = intel_de_read(dev_priv, ILK_DPFC_STATUS) & ILK_DPFC_COMP_SEG_MASK;
- else if (IS_G4X(dev_priv))
- mask = intel_de_read(dev_priv, DPFC_STATUS) & DPFC_COMP_SEG_MASK;
- else
- mask = intel_de_read(dev_priv, FBC_STATUS) &
- (FBC_STAT_COMPRESSING | FBC_STAT_COMPRESSED);
-
- seq_printf(m, "Compressing: %s\n", yesno(mask));
- }
-
- mutex_unlock(&fbc->lock);
- intel_runtime_pm_put(&dev_priv->runtime_pm, wakeref);
-
- return 0;
-}
-
-static int i915_fbc_false_color_get(void *data, u64 *val)
-{
- struct drm_i915_private *dev_priv = data;
-
- if (DISPLAY_VER(dev_priv) < 7 || !HAS_FBC(dev_priv))
- return -ENODEV;
-
- *val = dev_priv->fbc.false_color;
-
- return 0;
-}
-
-static int i915_fbc_false_color_set(void *data, u64 val)
-{
- struct drm_i915_private *dev_priv = data;
- u32 reg;
-
- if (DISPLAY_VER(dev_priv) < 7 || !HAS_FBC(dev_priv))
- return -ENODEV;
-
- mutex_lock(&dev_priv->fbc.lock);
-
- reg = intel_de_read(dev_priv, ILK_DPFC_CONTROL);
- dev_priv->fbc.false_color = val;
-
- intel_de_write(dev_priv, ILK_DPFC_CONTROL,
- val ? (reg | FBC_CTL_FALSE_COLOR) : (reg & ~FBC_CTL_FALSE_COLOR));
-
- mutex_unlock(&dev_priv->fbc.lock);
- return 0;
-}
-
-DEFINE_SIMPLE_ATTRIBUTE(i915_fbc_false_color_fops,
- i915_fbc_false_color_get, i915_fbc_false_color_set,
- "%llu\n");
-
static int i915_ips_status(struct seq_file *m, void *unused)
{
struct drm_i915_private *dev_priv = node_to_i915(m->private);
@@ -303,8 +226,7 @@ psr_source_status(struct intel_dp *intel_dp, struct seq_file *m)
};
val = intel_de_read(dev_priv,
EDP_PSR2_STATUS(intel_dp->psr.transcoder));
- status_val = (val & EDP_PSR2_STATUS_STATE_MASK) >>
- EDP_PSR2_STATUS_STATE_SHIFT;
+ status_val = REG_FIELD_GET(EDP_PSR2_STATUS_STATE_MASK, val);
if (status_val < ARRAY_SIZE(live_status))
status = live_status[status_val];
} else {
@@ -503,28 +425,9 @@ DEFINE_SIMPLE_ATTRIBUTE(i915_edp_psr_debug_fops,
static int i915_power_domain_info(struct seq_file *m, void *unused)
{
- struct drm_i915_private *dev_priv = node_to_i915(m->private);
- struct i915_power_domains *power_domains = &dev_priv->power_domains;
- int i;
+ struct drm_i915_private *i915 = node_to_i915(m->private);
- mutex_lock(&power_domains->lock);
-
- seq_printf(m, "%-25s %s\n", "Power well/domain", "Use count");
- for (i = 0; i < power_domains->power_well_count; i++) {
- struct i915_power_well *power_well;
- enum intel_display_power_domain power_domain;
-
- power_well = &power_domains->power_wells[i];
- seq_printf(m, "%-25s %d\n", power_well->desc->name,
- power_well->count);
-
- for_each_power_domain(power_domain, power_well->desc->domains)
- seq_printf(m, " %-23s %d\n",
- intel_display_power_domain_str(power_domain),
- power_domains->domain_use_count[power_domain]);
- }
-
- mutex_unlock(&power_domains->lock);
+ intel_display_power_debug(i915, m);
return 0;
}
@@ -2095,9 +1998,7 @@ i915_fifo_underrun_reset_write(struct file *filp,
return ret;
}
- ret = intel_fbc_reset_underrun(dev_priv);
- if (ret)
- return ret;
+ intel_fbc_reset_underrun(dev_priv);
return cnt;
}
@@ -2111,7 +2012,6 @@ static const struct file_operations i915_fifo_underrun_reset_ops = {
static const struct drm_info_list intel_display_debugfs_list[] = {
{"i915_frontbuffer_tracking", i915_frontbuffer_tracking, 0},
- {"i915_fbc_status", i915_fbc_status, 0},
{"i915_ips_status", i915_ips_status, 0},
{"i915_sr_status", i915_sr_status, 0},
{"i915_opregion", i915_opregion, 0},
@@ -2136,7 +2036,6 @@ static const struct {
{"i915_pri_wm_latency", &i915_pri_wm_latency_fops},
{"i915_spr_wm_latency", &i915_spr_wm_latency_fops},
{"i915_cur_wm_latency", &i915_cur_wm_latency_fops},
- {"i915_fbc_false_color", &i915_fbc_false_color_fops},
{"i915_dp_test_data", &i915_displayport_test_data_fops},
{"i915_dp_test_type", &i915_displayport_test_type_fops},
{"i915_dp_test_active", &i915_displayport_test_active_fops},
@@ -2163,6 +2062,8 @@ void intel_display_debugfs_register(struct drm_i915_private *i915)
drm_debugfs_create_files(intel_display_debugfs_list,
ARRAY_SIZE(intel_display_debugfs_list),
minor->debugfs_root, minor);
+
+ intel_fbc_debugfs_register(i915);
}
static int i915_panel_show(struct seq_file *m, void *data)
diff --git a/drivers/gpu/drm/i915/display/intel_display_power.c b/drivers/gpu/drm/i915/display/intel_display_power.c
index 1672604..05babdc 100644
--- a/drivers/gpu/drm/i915/display/intel_display_power.c
+++ b/drivers/gpu/drm/i915/display/intel_display_power.c
@@ -15,6 +15,7 @@
#include "intel_dpio_phy.h"
#include "intel_dpll.h"
#include "intel_hotplug.h"
+#include "intel_pch_refclk.h"
#include "intel_pcode.h"
#include "intel_pm.h"
#include "intel_pps.h"
@@ -23,6 +24,98 @@
#include "intel_vga.h"
#include "vlv_sideband.h"
+struct i915_power_well_ops {
+ /*
+ * Synchronize the well's hw state to match the current sw state, for
+ * example enable/disable it based on the current refcount. Called
+ * during driver init and resume time, possibly after first calling
+ * the enable/disable handlers.
+ */
+ void (*sync_hw)(struct drm_i915_private *dev_priv,
+ struct i915_power_well *power_well);
+ /*
+ * Enable the well and resources that depend on it (for example
+ * interrupts located on the well). Called after the 0->1 refcount
+ * transition.
+ */
+ void (*enable)(struct drm_i915_private *dev_priv,
+ struct i915_power_well *power_well);
+ /*
+ * Disable the well and resources that depend on it. Called after
+ * the 1->0 refcount transition.
+ */
+ void (*disable)(struct drm_i915_private *dev_priv,
+ struct i915_power_well *power_well);
+ /* Returns the hw enabled state. */
+ bool (*is_enabled)(struct drm_i915_private *dev_priv,
+ struct i915_power_well *power_well);
+};
+
+struct i915_power_well_regs {
+ i915_reg_t bios;
+ i915_reg_t driver;
+ i915_reg_t kvmr;
+ i915_reg_t debug;
+};
+
+/* Power well structure for haswell */
+struct i915_power_well_desc {
+ const char *name;
+ bool always_on;
+ u64 domains;
+ /* unique identifier for this power well */
+ enum i915_power_well_id id;
+ /*
+ * Arbitraty data associated with this power well. Platform and power
+ * well specific.
+ */
+ union {
+ struct {
+ /*
+ * request/status flag index in the PUNIT power well
+ * control/status registers.
+ */
+ u8 idx;
+ } vlv;
+ struct {
+ enum dpio_phy phy;
+ } bxt;
+ struct {
+ const struct i915_power_well_regs *regs;
+ /*
+ * request/status flag index in the power well
+ * constrol/status registers.
+ */
+ u8 idx;
+ /* Mask of pipes whose IRQ logic is backed by the pw */
+ u8 irq_pipe_mask;
+ /*
+ * Instead of waiting for the status bit to ack enables,
+ * just wait a specific amount of time and then consider
+ * the well enabled.
+ */
+ u16 fixed_enable_delay;
+ /* The pw is backing the VGA functionality */
+ bool has_vga:1;
+ bool has_fuses:1;
+ /*
+ * The pw is for an ICL+ TypeC PHY port in
+ * Thunderbolt mode.
+ */
+ bool is_tc_tbt:1;
+ } hsw;
+ };
+ const struct i915_power_well_ops *ops;
+};
+
+struct i915_power_well {
+ const struct i915_power_well_desc *desc;
+ /* power well enable/disable usage count */
+ int count;
+ /* cached hw enabled state */
+ bool hw_enabled;
+};
+
bool intel_display_power_well_is_enabled(struct drm_i915_private *dev_priv,
enum i915_power_well_id power_well_id);
@@ -154,8 +247,8 @@ intel_display_power_domain_str(enum intel_display_power_domain domain)
return "MODESET";
case POWER_DOMAIN_GT_IRQ:
return "GT_IRQ";
- case POWER_DOMAIN_DPLL_DC_OFF:
- return "DPLL_DC_OFF";
+ case POWER_DOMAIN_DC_OFF:
+ return "DC_OFF";
case POWER_DOMAIN_TC_COLD_OFF:
return "TC_COLD_OFF";
default:
@@ -434,6 +527,11 @@ static void hsw_power_well_enable(struct drm_i915_private *dev_priv,
pg = DISPLAY_VER(dev_priv) >= 11 ? ICL_PW_CTL_IDX_TO_PG(pw_idx) :
SKL_PW_CTL_IDX_TO_PG(pw_idx);
+
+ /* Wa_16013190616:adlp */
+ if (IS_ALDERLAKE_P(dev_priv) && pg == SKL_PG1)
+ intel_de_rmw(dev_priv, GEN8_CHICKEN_DCPR_1, 0, DISABLE_FLR_SRC);
+
/*
* For PW1 we have to wait both for the PW0/PG0 fuse state
* before enabling the power well and PW1/PG1's own fuse
@@ -894,7 +992,7 @@ static u32
sanitize_target_dc_state(struct drm_i915_private *dev_priv,
u32 target_dc_state)
{
- u32 states[] = {
+ static const u32 states[] = {
DC_STATE_EN_UPTO_DC6,
DC_STATE_EN_UPTO_DC5,
DC_STATE_EN_DC3CO,
@@ -2802,7 +2900,7 @@ intel_display_power_put_mask_in_set(struct drm_i915_private *i915,
ICL_PW_2_POWER_DOMAINS | \
BIT_ULL(POWER_DOMAIN_MODESET) | \
BIT_ULL(POWER_DOMAIN_AUX_A) | \
- BIT_ULL(POWER_DOMAIN_DPLL_DC_OFF) | \
+ BIT_ULL(POWER_DOMAIN_DC_OFF) | \
BIT_ULL(POWER_DOMAIN_INIT))
#define ICL_DDI_IO_A_POWER_DOMAINS ( \
@@ -3105,6 +3203,7 @@ intel_display_power_put_mask_in_set(struct drm_i915_private *i915,
BIT_ULL(POWER_DOMAIN_MODESET) | \
BIT_ULL(POWER_DOMAIN_AUX_A) | \
BIT_ULL(POWER_DOMAIN_AUX_B) | \
+ BIT_ULL(POWER_DOMAIN_PORT_DSI) | \
BIT_ULL(POWER_DOMAIN_INIT))
#define XELPD_AUX_IO_D_XELPD_POWER_DOMAINS BIT_ULL(POWER_DOMAIN_AUX_D_XELPD)
@@ -5271,7 +5370,7 @@ static void gen12_dbuf_slices_config(struct drm_i915_private *dev_priv)
static void icl_mbus_init(struct drm_i915_private *dev_priv)
{
- unsigned long abox_regs = INTEL_INFO(dev_priv)->abox_mask;
+ unsigned long abox_regs = INTEL_INFO(dev_priv)->display.abox_mask;
u32 mask, val, i;
if (IS_ALDERLAKE_P(dev_priv))
@@ -5731,7 +5830,7 @@ static void tgl_bw_buddy_init(struct drm_i915_private *dev_priv)
enum intel_dram_type type = dev_priv->dram_info.type;
u8 num_channels = dev_priv->dram_info.num_channels;
const struct buddy_page_mask *table;
- unsigned long abox_mask = INTEL_INFO(dev_priv)->abox_mask;
+ unsigned long abox_mask = INTEL_INFO(dev_priv)->display.abox_mask;
int config, i;
/* BW_BUDDY registers are not used on dgpu's beyond DG1 */
@@ -6390,3 +6489,28 @@ void intel_display_power_resume(struct drm_i915_private *i915)
hsw_disable_pc8(i915);
}
}
+
+void intel_display_power_debug(struct drm_i915_private *i915, struct seq_file *m)
+{
+ struct i915_power_domains *power_domains = &i915->power_domains;
+ int i;
+
+ mutex_lock(&power_domains->lock);
+
+ seq_printf(m, "%-25s %s\n", "Power well/domain", "Use count");
+ for (i = 0; i < power_domains->power_well_count; i++) {
+ struct i915_power_well *power_well;
+ enum intel_display_power_domain power_domain;
+
+ power_well = &power_domains->power_wells[i];
+ seq_printf(m, "%-25s %d\n", power_well->desc->name,
+ power_well->count);
+
+ for_each_power_domain(power_domain, power_well->desc->domains)
+ seq_printf(m, " %-23s %d\n",
+ intel_display_power_domain_str(power_domain),
+ power_domains->domain_use_count[power_domain]);
+ }
+
+ mutex_unlock(&power_domains->lock);
+}
diff --git a/drivers/gpu/drm/i915/display/intel_display_power.h b/drivers/gpu/drm/i915/display/intel_display_power.h
index 0612e4b..686d18e 100644
--- a/drivers/gpu/drm/i915/display/intel_display_power.h
+++ b/drivers/gpu/drm/i915/display/intel_display_power.h
@@ -6,11 +6,13 @@
#ifndef __INTEL_DISPLAY_POWER_H__
#define __INTEL_DISPLAY_POWER_H__
-#include "intel_display.h"
#include "intel_runtime_pm.h"
#include "i915_reg.h"
+enum dpio_channel;
+enum dpio_phy;
struct drm_i915_private;
+struct i915_power_well;
struct intel_encoder;
enum intel_display_power_domain {
@@ -117,7 +119,7 @@ enum intel_display_power_domain {
POWER_DOMAIN_GMBUS,
POWER_DOMAIN_MODESET,
POWER_DOMAIN_GT_IRQ,
- POWER_DOMAIN_DPLL_DC_OFF,
+ POWER_DOMAIN_DC_OFF,
POWER_DOMAIN_TC_COLD_OFF,
POWER_DOMAIN_INIT,
@@ -155,100 +157,6 @@ enum i915_power_well_id {
((tran) == TRANSCODER_EDP ? POWER_DOMAIN_TRANSCODER_EDP : \
(tran) + POWER_DOMAIN_TRANSCODER_A)
-struct i915_power_well;
-
-struct i915_power_well_ops {
- /*
- * Synchronize the well's hw state to match the current sw state, for
- * example enable/disable it based on the current refcount. Called
- * during driver init and resume time, possibly after first calling
- * the enable/disable handlers.
- */
- void (*sync_hw)(struct drm_i915_private *dev_priv,
- struct i915_power_well *power_well);
- /*
- * Enable the well and resources that depend on it (for example
- * interrupts located on the well). Called after the 0->1 refcount
- * transition.
- */
- void (*enable)(struct drm_i915_private *dev_priv,
- struct i915_power_well *power_well);
- /*
- * Disable the well and resources that depend on it. Called after
- * the 1->0 refcount transition.
- */
- void (*disable)(struct drm_i915_private *dev_priv,
- struct i915_power_well *power_well);
- /* Returns the hw enabled state. */
- bool (*is_enabled)(struct drm_i915_private *dev_priv,
- struct i915_power_well *power_well);
-};
-
-struct i915_power_well_regs {
- i915_reg_t bios;
- i915_reg_t driver;
- i915_reg_t kvmr;
- i915_reg_t debug;
-};
-
-/* Power well structure for haswell */
-struct i915_power_well_desc {
- const char *name;
- bool always_on;
- u64 domains;
- /* unique identifier for this power well */
- enum i915_power_well_id id;
- /*
- * Arbitraty data associated with this power well. Platform and power
- * well specific.
- */
- union {
- struct {
- /*
- * request/status flag index in the PUNIT power well
- * control/status registers.
- */
- u8 idx;
- } vlv;
- struct {
- enum dpio_phy phy;
- } bxt;
- struct {
- const struct i915_power_well_regs *regs;
- /*
- * request/status flag index in the power well
- * constrol/status registers.
- */
- u8 idx;
- /* Mask of pipes whose IRQ logic is backed by the pw */
- u8 irq_pipe_mask;
- /*
- * Instead of waiting for the status bit to ack enables,
- * just wait a specific amount of time and then consider
- * the well enabled.
- */
- u16 fixed_enable_delay;
- /* The pw is backing the VGA functionality */
- bool has_vga:1;
- bool has_fuses:1;
- /*
- * The pw is for an ICL+ TypeC PHY port in
- * Thunderbolt mode.
- */
- bool is_tc_tbt:1;
- } hsw;
- };
- const struct i915_power_well_ops *ops;
-};
-
-struct i915_power_well {
- const struct i915_power_well_desc *desc;
- /* power well enable/disable usage count */
- int count;
- /* cached hw enabled state */
- bool hw_enabled;
-};
-
struct i915_power_domains {
/*
* Power wells needed for initialization at driver init and suspend
@@ -391,6 +299,8 @@ intel_display_power_put_all_in_set(struct drm_i915_private *i915,
intel_display_power_put_mask_in_set(i915, power_domain_set, power_domain_set->mask);
}
+void intel_display_power_debug(struct drm_i915_private *i915, struct seq_file *m);
+
/*
* FIXME: We should probably switch this to a 0-based scheme to be consistent
* with how we now name/number DBUF_CTL instances.
diff --git a/drivers/gpu/drm/i915/display/intel_display_trace.c b/drivers/gpu/drm/i915/display/intel_display_trace.c
new file mode 100644
index 0000000..737979a
--- /dev/null
+++ b/drivers/gpu/drm/i915/display/intel_display_trace.c
@@ -0,0 +1,9 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright © 2021 Intel Corporation
+ */
+
+#ifndef __CHECKER__
+#define CREATE_TRACE_POINTS
+#include "intel_display_trace.h"
+#endif
diff --git a/drivers/gpu/drm/i915/display/intel_display_trace.h b/drivers/gpu/drm/i915/display/intel_display_trace.h
new file mode 100644
index 0000000..4043e12
--- /dev/null
+++ b/drivers/gpu/drm/i915/display/intel_display_trace.h
@@ -0,0 +1,587 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright © 2021 Intel Corporation
+ */
+
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM i915
+
+#if !defined(__INTEL_DISPLAY_TRACE_H__) || defined(TRACE_HEADER_MULTI_READ)
+#define __INTEL_DISPLAY_TRACE_H__
+
+#include <linux/types.h>
+#include <linux/tracepoint.h>
+
+#include "i915_drv.h"
+#include "intel_crtc.h"
+#include "intel_display_types.h"
+
+TRACE_EVENT(intel_pipe_enable,
+ TP_PROTO(struct intel_crtc *crtc),
+ TP_ARGS(crtc),
+
+ TP_STRUCT__entry(
+ __array(u32, frame, 3)
+ __array(u32, scanline, 3)
+ __field(enum pipe, pipe)
+ ),
+ TP_fast_assign(
+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ struct intel_crtc *it__;
+ for_each_intel_crtc(&dev_priv->drm, it__) {
+ __entry->frame[it__->pipe] = intel_crtc_get_vblank_counter(it__);
+ __entry->scanline[it__->pipe] = intel_get_crtc_scanline(it__);
+ }
+ __entry->pipe = crtc->pipe;
+ ),
+
+ TP_printk("pipe %c enable, pipe A: frame=%u, scanline=%u, pipe B: frame=%u, scanline=%u, pipe C: frame=%u, scanline=%u",
+ pipe_name(__entry->pipe),
+ __entry->frame[PIPE_A], __entry->scanline[PIPE_A],
+ __entry->frame[PIPE_B], __entry->scanline[PIPE_B],
+ __entry->frame[PIPE_C], __entry->scanline[PIPE_C])
+);
+
+TRACE_EVENT(intel_pipe_disable,
+ TP_PROTO(struct intel_crtc *crtc),
+ TP_ARGS(crtc),
+
+ TP_STRUCT__entry(
+ __array(u32, frame, 3)
+ __array(u32, scanline, 3)
+ __field(enum pipe, pipe)
+ ),
+
+ TP_fast_assign(
+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ struct intel_crtc *it__;
+ for_each_intel_crtc(&dev_priv->drm, it__) {
+ __entry->frame[it__->pipe] = intel_crtc_get_vblank_counter(it__);
+ __entry->scanline[it__->pipe] = intel_get_crtc_scanline(it__);
+ }
+ __entry->pipe = crtc->pipe;
+ ),
+
+ TP_printk("pipe %c disable, pipe A: frame=%u, scanline=%u, pipe B: frame=%u, scanline=%u, pipe C: frame=%u, scanline=%u",
+ pipe_name(__entry->pipe),
+ __entry->frame[PIPE_A], __entry->scanline[PIPE_A],
+ __entry->frame[PIPE_B], __entry->scanline[PIPE_B],
+ __entry->frame[PIPE_C], __entry->scanline[PIPE_C])
+);
+
+TRACE_EVENT(intel_pipe_crc,
+ TP_PROTO(struct intel_crtc *crtc, const u32 *crcs),
+ TP_ARGS(crtc, crcs),
+
+ TP_STRUCT__entry(
+ __field(enum pipe, pipe)
+ __field(u32, frame)
+ __field(u32, scanline)
+ __array(u32, crcs, 5)
+ ),
+
+ TP_fast_assign(
+ __entry->pipe = crtc->pipe;
+ __entry->frame = intel_crtc_get_vblank_counter(crtc);
+ __entry->scanline = intel_get_crtc_scanline(crtc);
+ memcpy(__entry->crcs, crcs, sizeof(__entry->crcs));
+ ),
+
+ TP_printk("pipe %c, frame=%u, scanline=%u crc=%08x %08x %08x %08x %08x",
+ pipe_name(__entry->pipe), __entry->frame, __entry->scanline,
+ __entry->crcs[0], __entry->crcs[1], __entry->crcs[2],
+ __entry->crcs[3], __entry->crcs[4])
+);
+
+TRACE_EVENT(intel_cpu_fifo_underrun,
+ TP_PROTO(struct drm_i915_private *dev_priv, enum pipe pipe),
+ TP_ARGS(dev_priv, pipe),
+
+ TP_STRUCT__entry(
+ __field(enum pipe, pipe)
+ __field(u32, frame)
+ __field(u32, scanline)
+ ),
+
+ TP_fast_assign(
+ struct intel_crtc *crtc = intel_crtc_for_pipe(dev_priv, pipe);
+ __entry->pipe = pipe;
+ __entry->frame = intel_crtc_get_vblank_counter(crtc);
+ __entry->scanline = intel_get_crtc_scanline(crtc);
+ ),
+
+ TP_printk("pipe %c, frame=%u, scanline=%u",
+ pipe_name(__entry->pipe),
+ __entry->frame, __entry->scanline)
+);
+
+TRACE_EVENT(intel_pch_fifo_underrun,
+ TP_PROTO(struct drm_i915_private *dev_priv, enum pipe pch_transcoder),
+ TP_ARGS(dev_priv, pch_transcoder),
+
+ TP_STRUCT__entry(
+ __field(enum pipe, pipe)
+ __field(u32, frame)
+ __field(u32, scanline)
+ ),
+
+ TP_fast_assign(
+ enum pipe pipe = pch_transcoder;
+ struct intel_crtc *crtc = intel_crtc_for_pipe(dev_priv, pipe);
+ __entry->pipe = pipe;
+ __entry->frame = intel_crtc_get_vblank_counter(crtc);
+ __entry->scanline = intel_get_crtc_scanline(crtc);
+ ),
+
+ TP_printk("pch transcoder %c, frame=%u, scanline=%u",
+ pipe_name(__entry->pipe),
+ __entry->frame, __entry->scanline)
+);
+
+TRACE_EVENT(intel_memory_cxsr,
+ TP_PROTO(struct drm_i915_private *dev_priv, bool old, bool new),
+ TP_ARGS(dev_priv, old, new),
+
+ TP_STRUCT__entry(
+ __array(u32, frame, 3)
+ __array(u32, scanline, 3)
+ __field(bool, old)
+ __field(bool, new)
+ ),
+
+ TP_fast_assign(
+ struct intel_crtc *crtc;
+ for_each_intel_crtc(&dev_priv->drm, crtc) {
+ __entry->frame[crtc->pipe] = intel_crtc_get_vblank_counter(crtc);
+ __entry->scanline[crtc->pipe] = intel_get_crtc_scanline(crtc);
+ }
+ __entry->old = old;
+ __entry->new = new;
+ ),
+
+ TP_printk("%s->%s, pipe A: frame=%u, scanline=%u, pipe B: frame=%u, scanline=%u, pipe C: frame=%u, scanline=%u",
+ onoff(__entry->old), onoff(__entry->new),
+ __entry->frame[PIPE_A], __entry->scanline[PIPE_A],
+ __entry->frame[PIPE_B], __entry->scanline[PIPE_B],
+ __entry->frame[PIPE_C], __entry->scanline[PIPE_C])
+);
+
+TRACE_EVENT(g4x_wm,
+ TP_PROTO(struct intel_crtc *crtc, const struct g4x_wm_values *wm),
+ TP_ARGS(crtc, wm),
+
+ TP_STRUCT__entry(
+ __field(enum pipe, pipe)
+ __field(u32, frame)
+ __field(u32, scanline)
+ __field(u16, primary)
+ __field(u16, sprite)
+ __field(u16, cursor)
+ __field(u16, sr_plane)
+ __field(u16, sr_cursor)
+ __field(u16, sr_fbc)
+ __field(u16, hpll_plane)
+ __field(u16, hpll_cursor)
+ __field(u16, hpll_fbc)
+ __field(bool, cxsr)
+ __field(bool, hpll)
+ __field(bool, fbc)
+ ),
+
+ TP_fast_assign(
+ __entry->pipe = crtc->pipe;
+ __entry->frame = intel_crtc_get_vblank_counter(crtc);
+ __entry->scanline = intel_get_crtc_scanline(crtc);
+ __entry->primary = wm->pipe[crtc->pipe].plane[PLANE_PRIMARY];
+ __entry->sprite = wm->pipe[crtc->pipe].plane[PLANE_SPRITE0];
+ __entry->cursor = wm->pipe[crtc->pipe].plane[PLANE_CURSOR];
+ __entry->sr_plane = wm->sr.plane;
+ __entry->sr_cursor = wm->sr.cursor;
+ __entry->sr_fbc = wm->sr.fbc;
+ __entry->hpll_plane = wm->hpll.plane;
+ __entry->hpll_cursor = wm->hpll.cursor;
+ __entry->hpll_fbc = wm->hpll.fbc;
+ __entry->cxsr = wm->cxsr;
+ __entry->hpll = wm->hpll_en;
+ __entry->fbc = wm->fbc_en;
+ ),
+
+ TP_printk("pipe %c, frame=%u, scanline=%u, wm %d/%d/%d, sr %s/%d/%d/%d, hpll %s/%d/%d/%d, fbc %s",
+ pipe_name(__entry->pipe), __entry->frame, __entry->scanline,
+ __entry->primary, __entry->sprite, __entry->cursor,
+ yesno(__entry->cxsr), __entry->sr_plane, __entry->sr_cursor, __entry->sr_fbc,
+ yesno(__entry->hpll), __entry->hpll_plane, __entry->hpll_cursor, __entry->hpll_fbc,
+ yesno(__entry->fbc))
+);
+
+TRACE_EVENT(vlv_wm,
+ TP_PROTO(struct intel_crtc *crtc, const struct vlv_wm_values *wm),
+ TP_ARGS(crtc, wm),
+
+ TP_STRUCT__entry(
+ __field(enum pipe, pipe)
+ __field(u32, frame)
+ __field(u32, scanline)
+ __field(u32, level)
+ __field(u32, cxsr)
+ __field(u32, primary)
+ __field(u32, sprite0)
+ __field(u32, sprite1)
+ __field(u32, cursor)
+ __field(u32, sr_plane)
+ __field(u32, sr_cursor)
+ ),
+
+ TP_fast_assign(
+ __entry->pipe = crtc->pipe;
+ __entry->frame = intel_crtc_get_vblank_counter(crtc);
+ __entry->scanline = intel_get_crtc_scanline(crtc);
+ __entry->level = wm->level;
+ __entry->cxsr = wm->cxsr;
+ __entry->primary = wm->pipe[crtc->pipe].plane[PLANE_PRIMARY];
+ __entry->sprite0 = wm->pipe[crtc->pipe].plane[PLANE_SPRITE0];
+ __entry->sprite1 = wm->pipe[crtc->pipe].plane[PLANE_SPRITE1];
+ __entry->cursor = wm->pipe[crtc->pipe].plane[PLANE_CURSOR];
+ __entry->sr_plane = wm->sr.plane;
+ __entry->sr_cursor = wm->sr.cursor;
+ ),
+
+ TP_printk("pipe %c, frame=%u, scanline=%u, level=%d, cxsr=%d, wm %d/%d/%d/%d, sr %d/%d",
+ pipe_name(__entry->pipe), __entry->frame,
+ __entry->scanline, __entry->level, __entry->cxsr,
+ __entry->primary, __entry->sprite0, __entry->sprite1, __entry->cursor,
+ __entry->sr_plane, __entry->sr_cursor)
+);
+
+TRACE_EVENT(vlv_fifo_size,
+ TP_PROTO(struct intel_crtc *crtc, u32 sprite0_start, u32 sprite1_start, u32 fifo_size),
+ TP_ARGS(crtc, sprite0_start, sprite1_start, fifo_size),
+
+ TP_STRUCT__entry(
+ __field(enum pipe, pipe)
+ __field(u32, frame)
+ __field(u32, scanline)
+ __field(u32, sprite0_start)
+ __field(u32, sprite1_start)
+ __field(u32, fifo_size)
+ ),
+
+ TP_fast_assign(
+ __entry->pipe = crtc->pipe;
+ __entry->frame = intel_crtc_get_vblank_counter(crtc);
+ __entry->scanline = intel_get_crtc_scanline(crtc);
+ __entry->sprite0_start = sprite0_start;
+ __entry->sprite1_start = sprite1_start;
+ __entry->fifo_size = fifo_size;
+ ),
+
+ TP_printk("pipe %c, frame=%u, scanline=%u, %d/%d/%d",
+ pipe_name(__entry->pipe), __entry->frame,
+ __entry->scanline, __entry->sprite0_start,
+ __entry->sprite1_start, __entry->fifo_size)
+);
+
+TRACE_EVENT(intel_plane_update_noarm,
+ TP_PROTO(struct drm_plane *plane, struct intel_crtc *crtc),
+ TP_ARGS(plane, crtc),
+
+ TP_STRUCT__entry(
+ __field(enum pipe, pipe)
+ __field(u32, frame)
+ __field(u32, scanline)
+ __array(int, src, 4)
+ __array(int, dst, 4)
+ __string(name, plane->name)
+ ),
+
+ TP_fast_assign(
+ __assign_str(name, plane->name);
+ __entry->pipe = crtc->pipe;
+ __entry->frame = intel_crtc_get_vblank_counter(crtc);
+ __entry->scanline = intel_get_crtc_scanline(crtc);
+ memcpy(__entry->src, &plane->state->src, sizeof(__entry->src));
+ memcpy(__entry->dst, &plane->state->dst, sizeof(__entry->dst));
+ ),
+
+ TP_printk("pipe %c, plane %s, frame=%u, scanline=%u, " DRM_RECT_FP_FMT " -> " DRM_RECT_FMT,
+ pipe_name(__entry->pipe), __get_str(name),
+ __entry->frame, __entry->scanline,
+ DRM_RECT_FP_ARG((const struct drm_rect *)__entry->src),
+ DRM_RECT_ARG((const struct drm_rect *)__entry->dst))
+);
+
+TRACE_EVENT(intel_plane_update_arm,
+ TP_PROTO(struct drm_plane *plane, struct intel_crtc *crtc),
+ TP_ARGS(plane, crtc),
+
+ TP_STRUCT__entry(
+ __field(enum pipe, pipe)
+ __field(u32, frame)
+ __field(u32, scanline)
+ __array(int, src, 4)
+ __array(int, dst, 4)
+ __string(name, plane->name)
+ ),
+
+ TP_fast_assign(
+ __assign_str(name, plane->name);
+ __entry->pipe = crtc->pipe;
+ __entry->frame = intel_crtc_get_vblank_counter(crtc);
+ __entry->scanline = intel_get_crtc_scanline(crtc);
+ memcpy(__entry->src, &plane->state->src, sizeof(__entry->src));
+ memcpy(__entry->dst, &plane->state->dst, sizeof(__entry->dst));
+ ),
+
+ TP_printk("pipe %c, plane %s, frame=%u, scanline=%u, " DRM_RECT_FP_FMT " -> " DRM_RECT_FMT,
+ pipe_name(__entry->pipe), __get_str(name),
+ __entry->frame, __entry->scanline,
+ DRM_RECT_FP_ARG((const struct drm_rect *)__entry->src),
+ DRM_RECT_ARG((const struct drm_rect *)__entry->dst))
+);
+
+TRACE_EVENT(intel_plane_disable_arm,
+ TP_PROTO(struct drm_plane *plane, struct intel_crtc *crtc),
+ TP_ARGS(plane, crtc),
+
+ TP_STRUCT__entry(
+ __field(enum pipe, pipe)
+ __field(u32, frame)
+ __field(u32, scanline)
+ __string(name, plane->name)
+ ),
+
+ TP_fast_assign(
+ __assign_str(name, plane->name);
+ __entry->pipe = crtc->pipe;
+ __entry->frame = intel_crtc_get_vblank_counter(crtc);
+ __entry->scanline = intel_get_crtc_scanline(crtc);
+ ),
+
+ TP_printk("pipe %c, plane %s, frame=%u, scanline=%u",
+ pipe_name(__entry->pipe), __get_str(name),
+ __entry->frame, __entry->scanline)
+);
+
+TRACE_EVENT(intel_fbc_activate,
+ TP_PROTO(struct intel_plane *plane),
+ TP_ARGS(plane),
+
+ TP_STRUCT__entry(
+ __field(enum pipe, pipe)
+ __field(u32, frame)
+ __field(u32, scanline)
+ ),
+
+ TP_fast_assign(
+ struct intel_crtc *crtc = intel_crtc_for_pipe(to_i915(plane->base.dev),
+ plane->pipe);
+ __entry->pipe = crtc->pipe;
+ __entry->frame = intel_crtc_get_vblank_counter(crtc);
+ __entry->scanline = intel_get_crtc_scanline(crtc);
+ ),
+
+ TP_printk("pipe %c, frame=%u, scanline=%u",
+ pipe_name(__entry->pipe), __entry->frame, __entry->scanline)
+);
+
+TRACE_EVENT(intel_fbc_deactivate,
+ TP_PROTO(struct intel_plane *plane),
+ TP_ARGS(plane),
+
+ TP_STRUCT__entry(
+ __field(enum pipe, pipe)
+ __field(u32, frame)
+ __field(u32, scanline)
+ ),
+
+ TP_fast_assign(
+ struct intel_crtc *crtc = intel_crtc_for_pipe(to_i915(plane->base.dev),
+ plane->pipe);
+ __entry->pipe = crtc->pipe;
+ __entry->frame = intel_crtc_get_vblank_counter(crtc);
+ __entry->scanline = intel_get_crtc_scanline(crtc);
+ ),
+
+ TP_printk("pipe %c, frame=%u, scanline=%u",
+ pipe_name(__entry->pipe), __entry->frame, __entry->scanline)
+);
+
+TRACE_EVENT(intel_fbc_nuke,
+ TP_PROTO(struct intel_plane *plane),
+ TP_ARGS(plane),
+
+ TP_STRUCT__entry(
+ __field(enum pipe, pipe)
+ __field(u32, frame)
+ __field(u32, scanline)
+ ),
+
+ TP_fast_assign(
+ struct intel_crtc *crtc = intel_crtc_for_pipe(to_i915(plane->base.dev),
+ plane->pipe);
+ __entry->pipe = crtc->pipe;
+ __entry->frame = intel_crtc_get_vblank_counter(crtc);
+ __entry->scanline = intel_get_crtc_scanline(crtc);
+ ),
+
+ TP_printk("pipe %c, frame=%u, scanline=%u",
+ pipe_name(__entry->pipe), __entry->frame, __entry->scanline)
+);
+
+TRACE_EVENT(intel_crtc_vblank_work_start,
+ TP_PROTO(struct intel_crtc *crtc),
+ TP_ARGS(crtc),
+
+ TP_STRUCT__entry(
+ __field(enum pipe, pipe)
+ __field(u32, frame)
+ __field(u32, scanline)
+ ),
+
+ TP_fast_assign(
+ __entry->pipe = crtc->pipe;
+ __entry->frame = intel_crtc_get_vblank_counter(crtc);
+ __entry->scanline = intel_get_crtc_scanline(crtc);
+ ),
+
+ TP_printk("pipe %c, frame=%u, scanline=%u",
+ pipe_name(__entry->pipe), __entry->frame,
+ __entry->scanline)
+);
+
+TRACE_EVENT(intel_crtc_vblank_work_end,
+ TP_PROTO(struct intel_crtc *crtc),
+ TP_ARGS(crtc),
+
+ TP_STRUCT__entry(
+ __field(enum pipe, pipe)
+ __field(u32, frame)
+ __field(u32, scanline)
+ ),
+
+ TP_fast_assign(
+ __entry->pipe = crtc->pipe;
+ __entry->frame = intel_crtc_get_vblank_counter(crtc);
+ __entry->scanline = intel_get_crtc_scanline(crtc);
+ ),
+
+ TP_printk("pipe %c, frame=%u, scanline=%u",
+ pipe_name(__entry->pipe), __entry->frame,
+ __entry->scanline)
+);
+
+TRACE_EVENT(intel_pipe_update_start,
+ TP_PROTO(struct intel_crtc *crtc),
+ TP_ARGS(crtc),
+
+ TP_STRUCT__entry(
+ __field(enum pipe, pipe)
+ __field(u32, frame)
+ __field(u32, scanline)
+ __field(u32, min)
+ __field(u32, max)
+ ),
+
+ TP_fast_assign(
+ __entry->pipe = crtc->pipe;
+ __entry->frame = intel_crtc_get_vblank_counter(crtc);
+ __entry->scanline = intel_get_crtc_scanline(crtc);
+ __entry->min = crtc->debug.min_vbl;
+ __entry->max = crtc->debug.max_vbl;
+ ),
+
+ TP_printk("pipe %c, frame=%u, scanline=%u, min=%u, max=%u",
+ pipe_name(__entry->pipe), __entry->frame,
+ __entry->scanline, __entry->min, __entry->max)
+);
+
+TRACE_EVENT(intel_pipe_update_vblank_evaded,
+ TP_PROTO(struct intel_crtc *crtc),
+ TP_ARGS(crtc),
+
+ TP_STRUCT__entry(
+ __field(enum pipe, pipe)
+ __field(u32, frame)
+ __field(u32, scanline)
+ __field(u32, min)
+ __field(u32, max)
+ ),
+
+ TP_fast_assign(
+ __entry->pipe = crtc->pipe;
+ __entry->frame = crtc->debug.start_vbl_count;
+ __entry->scanline = crtc->debug.scanline_start;
+ __entry->min = crtc->debug.min_vbl;
+ __entry->max = crtc->debug.max_vbl;
+ ),
+
+ TP_printk("pipe %c, frame=%u, scanline=%u, min=%u, max=%u",
+ pipe_name(__entry->pipe), __entry->frame,
+ __entry->scanline, __entry->min, __entry->max)
+);
+
+TRACE_EVENT(intel_pipe_update_end,
+ TP_PROTO(struct intel_crtc *crtc, u32 frame, int scanline_end),
+ TP_ARGS(crtc, frame, scanline_end),
+
+ TP_STRUCT__entry(
+ __field(enum pipe, pipe)
+ __field(u32, frame)
+ __field(u32, scanline)
+ ),
+
+ TP_fast_assign(
+ __entry->pipe = crtc->pipe;
+ __entry->frame = frame;
+ __entry->scanline = scanline_end;
+ ),
+
+ TP_printk("pipe %c, frame=%u, scanline=%u",
+ pipe_name(__entry->pipe), __entry->frame,
+ __entry->scanline)
+);
+
+TRACE_EVENT(intel_frontbuffer_invalidate,
+ TP_PROTO(unsigned int frontbuffer_bits, unsigned int origin),
+ TP_ARGS(frontbuffer_bits, origin),
+
+ TP_STRUCT__entry(
+ __field(unsigned int, frontbuffer_bits)
+ __field(unsigned int, origin)
+ ),
+
+ TP_fast_assign(
+ __entry->frontbuffer_bits = frontbuffer_bits;
+ __entry->origin = origin;
+ ),
+
+ TP_printk("frontbuffer_bits=0x%08x, origin=%u",
+ __entry->frontbuffer_bits, __entry->origin)
+);
+
+TRACE_EVENT(intel_frontbuffer_flush,
+ TP_PROTO(unsigned int frontbuffer_bits, unsigned int origin),
+ TP_ARGS(frontbuffer_bits, origin),
+
+ TP_STRUCT__entry(
+ __field(unsigned int, frontbuffer_bits)
+ __field(unsigned int, origin)
+ ),
+
+ TP_fast_assign(
+ __entry->frontbuffer_bits = frontbuffer_bits;
+ __entry->origin = origin;
+ ),
+
+ TP_printk("frontbuffer_bits=0x%08x, origin=%u",
+ __entry->frontbuffer_bits, __entry->origin)
+);
+
+#endif /* __INTEL_DISPLAY_TRACE_H__ */
+
+/* This part must be outside protection */
+#undef TRACE_INCLUDE_PATH
+#undef TRACE_INCLUDE_FILE
+#define TRACE_INCLUDE_PATH ../../drivers/gpu/drm/i915/display
+#define TRACE_INCLUDE_FILE intel_display_trace
+#include <trace/define_trace.h>
diff --git a/drivers/gpu/drm/i915/display/intel_display_types.h b/drivers/gpu/drm/i915/display/intel_display_types.h
index aa72382..c9c6efa 100644
--- a/drivers/gpu/drm/i915/display/intel_display_types.h
+++ b/drivers/gpu/drm/i915/display/intel_display_types.h
@@ -28,6 +28,7 @@
#include <linux/async.h>
#include <linux/i2c.h>
+#include <linux/pm_qos.h>
#include <linux/pwm.h>
#include <linux/sched/clock.h>
@@ -35,20 +36,30 @@
#include <drm/drm_crtc.h>
#include <drm/drm_dp_dual_mode_helper.h>
#include <drm/drm_dp_mst_helper.h>
+#include <drm/drm_dsc.h>
#include <drm/drm_encoder.h>
#include <drm/drm_fb_helper.h>
#include <drm/drm_fourcc.h>
#include <drm/drm_probe_helper.h>
#include <drm/drm_rect.h>
#include <drm/drm_vblank.h>
+#include <drm/drm_vblank_work.h>
#include <drm/i915_mei_hdcp_interface.h>
#include <media/cec-notifier.h>
-#include "i915_drv.h"
+#include "i915_vma.h"
+#include "i915_vma_types.h"
+#include "intel_bios.h"
+#include "intel_display.h"
+#include "intel_display_power.h"
+#include "intel_dpll_mgr.h"
+#include "intel_pm_types.h"
struct drm_printer;
struct __intel_global_objs_state;
struct intel_ddi_buf_trans;
+struct intel_fbc;
+struct intel_connector;
/*
* Display related stuff
@@ -115,7 +126,8 @@ struct intel_fb_view {
* bytes for 0/180 degree rotation
* pixels for 90/270 degree rotation
*/
- unsigned int stride;
+ unsigned int mapping_stride;
+ unsigned int scanout_stride;
} color_plane[4];
};
@@ -194,10 +206,6 @@ struct intel_encoder {
void (*update_complete)(struct intel_atomic_state *,
struct intel_encoder *,
struct intel_crtc *);
- void (*pre_disable)(struct intel_atomic_state *,
- struct intel_encoder *,
- const struct intel_crtc_state *,
- const struct drm_connector_state *);
void (*disable)(struct intel_atomic_state *,
struct intel_encoder *,
const struct intel_crtc_state *,
@@ -687,6 +695,8 @@ struct intel_plane_state {
/* Clear Color Value */
u64 ccval;
+
+ const char *no_fbc_reason;
};
struct intel_initial_plane_config {
@@ -949,7 +959,6 @@ struct intel_crtc_state {
* accordingly.
*/
#define PIPE_CONFIG_QUIRK_MODE_SYNC_FLAGS (1<<0) /* unreliable sync mode.flags */
-#define PIPE_CONFIG_QUIRK_BIGJOINER_SLAVE (1<<1) /* bigjoiner slave, partial readout */
unsigned long quirks;
unsigned fb_bits; /* framebuffers to flip */
@@ -1118,8 +1127,6 @@ struct intel_crtc_state {
bool crc_enabled;
- bool enable_fbc;
-
bool double_wide;
int pbn;
@@ -1241,6 +1248,9 @@ struct intel_crtc_state {
u8 link_count;
u8 pixel_overlap;
} splitter;
+
+ /* for loading single buffered registers during vblank */
+ struct drm_vblank_work vblank_work;
};
enum intel_pipe_crc_source {
@@ -1325,6 +1335,9 @@ struct intel_crtc {
/* scalers available on this crtc */
int num_scalers;
+ /* for loading single buffered registers during vblank */
+ struct pm_qos_request vblank_pm_qos;
+
#ifdef CONFIG_DEBUG_FS
struct intel_pipe_crc pipe_crc;
#endif
@@ -1335,8 +1348,6 @@ struct intel_plane {
enum i9xx_plane_id i9xx_plane;
enum plane_id id;
enum pipe pipe;
- bool has_fbc;
- bool has_ccs;
bool need_async_flip_disable_wa;
u32 frontbuffer_bit;
@@ -1344,6 +1355,8 @@ struct intel_plane {
u32 base, cntl, size;
} cursor;
+ struct intel_fbc *fbc;
+
/*
* NOTE: Do not place new plane state fields here (e.g., when adding
* new plane properties). New runtime state should now be placed in
@@ -1362,11 +1375,17 @@ struct intel_plane {
unsigned int (*max_stride)(struct intel_plane *plane,
u32 pixel_format, u64 modifier,
unsigned int rotation);
- void (*update_plane)(struct intel_plane *plane,
+ /* Write all non-self arming plane registers */
+ void (*update_noarm)(struct intel_plane *plane,
const struct intel_crtc_state *crtc_state,
const struct intel_plane_state *plane_state);
- void (*disable_plane)(struct intel_plane *plane,
- const struct intel_crtc_state *crtc_state);
+ /* Write all self-arming plane registers */
+ void (*update_arm)(struct intel_plane *plane,
+ const struct intel_crtc_state *crtc_state,
+ const struct intel_plane_state *plane_state);
+ /* Disable the plane, must arm */
+ void (*disable_arm)(struct intel_plane *plane,
+ const struct intel_crtc_state *crtc_state);
bool (*get_hw_state)(struct intel_plane *plane, enum pipe *pipe);
int (*check_plane)(struct intel_crtc_state *crtc_state,
struct intel_plane_state *plane_state);
@@ -1563,6 +1582,8 @@ struct intel_dp {
int num_sink_rates;
int sink_rates[DP_MAX_SUPPORTED_RATES];
bool use_rate_select;
+ /* Max sink lane count as reported by DP_MAX_LANE_COUNT */
+ int max_sink_lane_count;
/* intersection of source and sink rates */
int num_common_rates;
int common_rates[DP_MAX_SUPPORTED_RATES];
@@ -1760,35 +1781,6 @@ vlv_pipe_to_channel(enum pipe pipe)
}
}
-static inline bool intel_pipe_valid(struct drm_i915_private *i915, enum pipe pipe)
-{
- return (pipe >= 0 &&
- pipe < ARRAY_SIZE(i915->pipe_to_crtc_mapping) &&
- INTEL_INFO(i915)->pipe_mask & BIT(pipe) &&
- i915->pipe_to_crtc_mapping[pipe]);
-}
-
-static inline struct intel_crtc *
-intel_get_first_crtc(struct drm_i915_private *dev_priv)
-{
- return to_intel_crtc(drm_crtc_from_index(&dev_priv->drm, 0));
-}
-
-static inline struct intel_crtc *
-intel_get_crtc_for_pipe(struct drm_i915_private *dev_priv, enum pipe pipe)
-{
- /* pipe_to_crtc_mapping may have hole on any of 3 display pipe system */
- drm_WARN_ON(&dev_priv->drm,
- !(INTEL_INFO(dev_priv)->pipe_mask & BIT(pipe)));
- return dev_priv->pipe_to_crtc_mapping[pipe];
-}
-
-static inline struct intel_crtc *
-intel_get_crtc_for_plane(struct drm_i915_private *dev_priv, enum i9xx_plane_id plane)
-{
- return dev_priv->plane_to_crtc_mapping[plane];
-}
-
struct intel_load_detect_pipe {
struct drm_atomic_state *restore_state;
};
@@ -1898,11 +1890,7 @@ dp_to_lspcon(struct intel_dp *intel_dp)
return &dp_to_dig_port(intel_dp)->lspcon;
}
-static inline struct drm_i915_private *
-dp_to_i915(struct intel_dp *intel_dp)
-{
- return to_i915(dp_to_dig_port(intel_dp)->base.base.dev);
-}
+#define dp_to_i915(__intel_dp) to_i915(dp_to_dig_port(__intel_dp)->base.base.dev)
#define CAN_PSR(intel_dp) ((intel_dp)->psr.sink_support && \
(intel_dp)->psr.source_support)
@@ -2006,33 +1994,6 @@ intel_crtc_needs_modeset(const struct intel_crtc_state *crtc_state)
return drm_atomic_crtc_needs_modeset(&crtc_state->uapi);
}
-static inline void
-intel_wait_for_vblank(struct drm_i915_private *dev_priv, enum pipe pipe)
-{
- struct intel_crtc *crtc = intel_get_crtc_for_pipe(dev_priv, pipe);
-
- drm_crtc_wait_one_vblank(&crtc->base);
-}
-
-static inline void
-intel_wait_for_vblank_if_active(struct drm_i915_private *dev_priv, enum pipe pipe)
-{
- const struct intel_crtc *crtc = intel_get_crtc_for_pipe(dev_priv, pipe);
-
- if (crtc->active)
- intel_wait_for_vblank(dev_priv, pipe);
-}
-
-static inline bool intel_modifier_uses_dpt(struct drm_i915_private *i915, u64 modifier)
-{
- return DISPLAY_VER(i915) >= 13 && modifier != DRM_FORMAT_MOD_LINEAR;
-}
-
-static inline bool intel_fb_uses_dpt(const struct drm_framebuffer *fb)
-{
- return fb && intel_modifier_uses_dpt(to_i915(fb->dev), fb->modifier);
-}
-
static inline u32 intel_plane_ggtt_offset(const struct intel_plane_state *plane_state)
{
return i915_ggtt_offset(plane_state->ggtt_vma);
@@ -2044,20 +2005,4 @@ to_intel_frontbuffer(struct drm_framebuffer *fb)
return fb ? to_intel_framebuffer(fb)->frontbuffer : NULL;
}
-static inline bool is_ccs_modifier(u64 modifier)
-{
- return modifier == I915_FORMAT_MOD_Y_TILED_GEN12_RC_CCS ||
- modifier == I915_FORMAT_MOD_Y_TILED_GEN12_RC_CCS_CC ||
- modifier == I915_FORMAT_MOD_Y_TILED_GEN12_MC_CCS ||
- modifier == I915_FORMAT_MOD_Y_TILED_CCS ||
- modifier == I915_FORMAT_MOD_Yf_TILED_CCS;
-}
-
-static inline bool is_gen12_ccs_modifier(u64 modifier)
-{
- return modifier == I915_FORMAT_MOD_Y_TILED_GEN12_RC_CCS ||
- modifier == I915_FORMAT_MOD_Y_TILED_GEN12_RC_CCS_CC ||
- modifier == I915_FORMAT_MOD_Y_TILED_GEN12_MC_CCS;
-}
-
#endif /* __INTEL_DISPLAY_TYPES_H__ */
diff --git a/drivers/gpu/drm/i915/display/intel_dmc.c b/drivers/gpu/drm/i915/display/intel_dmc.c
index aef6952..a69b28d 100644
--- a/drivers/gpu/drm/i915/display/intel_dmc.c
+++ b/drivers/gpu/drm/i915/display/intel_dmc.c
@@ -45,8 +45,10 @@
#define GEN12_DMC_MAX_FW_SIZE ICL_DMC_MAX_FW_SIZE
-#define ADLP_DMC_PATH DMC_PATH(adlp, 2, 12)
-#define ADLP_DMC_VERSION_REQUIRED DMC_VERSION(2, 12)
+#define GEN13_DMC_MAX_FW_SIZE 0x20000
+
+#define ADLP_DMC_PATH DMC_PATH(adlp, 2, 14)
+#define ADLP_DMC_VERSION_REQUIRED DMC_VERSION(2, 14)
MODULE_FIRMWARE(ADLP_DMC_PATH);
#define ADLS_DMC_PATH DMC_PATH(adls, 2, 01)
@@ -682,7 +684,7 @@ void intel_dmc_ucode_init(struct drm_i915_private *dev_priv)
if (IS_ALDERLAKE_P(dev_priv)) {
dmc->fw_path = ADLP_DMC_PATH;
dmc->required_version = ADLP_DMC_VERSION_REQUIRED;
- dmc->max_fw_size = GEN12_DMC_MAX_FW_SIZE;
+ dmc->max_fw_size = GEN13_DMC_MAX_FW_SIZE;
} else if (IS_ALDERLAKE_S(dev_priv)) {
dmc->fw_path = ADLS_DMC_PATH;
dmc->required_version = ADLS_DMC_VERSION_REQUIRED;
diff --git a/drivers/gpu/drm/i915/display/intel_dmc.h b/drivers/gpu/drm/i915/display/intel_dmc.h
index c3c00ff..b20f344 100644
--- a/drivers/gpu/drm/i915/display/intel_dmc.h
+++ b/drivers/gpu/drm/i915/display/intel_dmc.h
@@ -20,6 +20,8 @@ enum {
DMC_FW_MAIN = 0,
DMC_FW_PIPEA,
DMC_FW_PIPEB,
+ DMC_FW_PIPEC,
+ DMC_FW_PIPED,
DMC_FW_MAX
};
diff --git a/drivers/gpu/drm/i915/display/intel_dp.c b/drivers/gpu/drm/i915/display/intel_dp.c
index a552f05..b5e2508 100644
--- a/drivers/gpu/drm/i915/display/intel_dp.c
+++ b/drivers/gpu/drm/i915/display/intel_dp.c
@@ -47,6 +47,7 @@
#include "intel_audio.h"
#include "intel_backlight.h"
#include "intel_connector.h"
+#include "intel_crtc.h"
#include "intel_ddi.h"
#include "intel_de.h"
#include "intel_display_types.h"
@@ -128,7 +129,7 @@ static void intel_dp_set_default_sink_rates(struct intel_dp *intel_dp)
}
/* update sink rates from dpcd */
-static void intel_dp_set_sink_rates(struct intel_dp *intel_dp)
+static void intel_dp_set_dpcd_sink_rates(struct intel_dp *intel_dp)
{
static const int dp_rates[] = {
162000, 270000, 540000, 810000
@@ -198,6 +199,54 @@ static void intel_dp_set_sink_rates(struct intel_dp *intel_dp)
intel_dp->num_sink_rates = i;
}
+static void intel_dp_set_sink_rates(struct intel_dp *intel_dp)
+{
+ struct intel_connector *connector = intel_dp->attached_connector;
+ struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);
+ struct intel_encoder *encoder = &intel_dig_port->base;
+
+ intel_dp_set_dpcd_sink_rates(intel_dp);
+
+ if (intel_dp->num_sink_rates)
+ return;
+
+ drm_err(&dp_to_i915(intel_dp)->drm,
+ "[CONNECTOR:%d:%s][ENCODER:%d:%s] Invalid DPCD with no link rates, using defaults\n",
+ connector->base.base.id, connector->base.name,
+ encoder->base.base.id, encoder->base.name);
+
+ intel_dp_set_default_sink_rates(intel_dp);
+}
+
+static void intel_dp_set_default_max_sink_lane_count(struct intel_dp *intel_dp)
+{
+ intel_dp->max_sink_lane_count = 1;
+}
+
+static void intel_dp_set_max_sink_lane_count(struct intel_dp *intel_dp)
+{
+ struct intel_connector *connector = intel_dp->attached_connector;
+ struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);
+ struct intel_encoder *encoder = &intel_dig_port->base;
+
+ intel_dp->max_sink_lane_count = drm_dp_max_lane_count(intel_dp->dpcd);
+
+ switch (intel_dp->max_sink_lane_count) {
+ case 1:
+ case 2:
+ case 4:
+ return;
+ }
+
+ drm_err(&dp_to_i915(intel_dp)->drm,
+ "[CONNECTOR:%d:%s][ENCODER:%d:%s] Invalid DPCD max lane count (%d), using default\n",
+ connector->base.base.id, connector->base.name,
+ encoder->base.base.id, encoder->base.name,
+ intel_dp->max_sink_lane_count);
+
+ intel_dp_set_default_max_sink_lane_count(intel_dp);
+}
+
/* Get length of rates array potentially limited by max_rate. */
static int intel_dp_rate_limit_len(const int *rates, int len, int max_rate)
{
@@ -220,10 +269,19 @@ static int intel_dp_common_len_rate_limit(const struct intel_dp *intel_dp,
intel_dp->num_common_rates, max_rate);
}
+static int intel_dp_common_rate(struct intel_dp *intel_dp, int index)
+{
+ if (drm_WARN_ON(&dp_to_i915(intel_dp)->drm,
+ index < 0 || index >= intel_dp->num_common_rates))
+ return 162000;
+
+ return intel_dp->common_rates[index];
+}
+
/* Theoretical max between source and sink */
static int intel_dp_max_common_rate(struct intel_dp *intel_dp)
{
- return intel_dp->common_rates[intel_dp->num_common_rates - 1];
+ return intel_dp_common_rate(intel_dp, intel_dp->num_common_rates - 1);
}
/* Theoretical max between source and sink */
@@ -231,7 +289,7 @@ static int intel_dp_max_common_lane_count(struct intel_dp *intel_dp)
{
struct intel_digital_port *dig_port = dp_to_dig_port(intel_dp);
int source_max = dig_port->max_lanes;
- int sink_max = drm_dp_max_lane_count(intel_dp->dpcd);
+ int sink_max = intel_dp->max_sink_lane_count;
int fia_max = intel_tc_port_fia_max_lane_count(dig_port);
int lttpr_max = drm_dp_lttpr_max_lane_count(intel_dp->lttpr_common_caps);
@@ -243,7 +301,15 @@ static int intel_dp_max_common_lane_count(struct intel_dp *intel_dp)
int intel_dp_max_lane_count(struct intel_dp *intel_dp)
{
- return intel_dp->max_link_lane_count;
+ switch (intel_dp->max_link_lane_count) {
+ case 1:
+ case 2:
+ case 4:
+ return intel_dp->max_link_lane_count;
+ default:
+ MISSING_CASE(intel_dp->max_link_lane_count);
+ return 1;
+ }
}
/*
@@ -555,13 +621,13 @@ int intel_dp_get_link_train_fallback_values(struct intel_dp *intel_dp,
if (index > 0) {
if (intel_dp_is_edp(intel_dp) &&
!intel_dp_can_link_train_fallback_for_edp(intel_dp,
- intel_dp->common_rates[index - 1],
+ intel_dp_common_rate(intel_dp, index - 1),
lane_count)) {
drm_dbg_kms(&i915->drm,
"Retrying Link training for eDP with same parameters\n");
return 0;
}
- intel_dp->max_link_rate = intel_dp->common_rates[index - 1];
+ intel_dp->max_link_rate = intel_dp_common_rate(intel_dp, index - 1);
intel_dp->max_link_lane_count = lane_count;
} else if (lane_count > 1) {
if (intel_dp_is_edp(intel_dp) &&
@@ -1001,14 +1067,11 @@ static void intel_dp_print_rates(struct intel_dp *intel_dp)
int
intel_dp_max_link_rate(struct intel_dp *intel_dp)
{
- struct drm_i915_private *i915 = dp_to_i915(intel_dp);
int len;
len = intel_dp_common_len_rate_limit(intel_dp, intel_dp->max_link_rate);
- if (drm_WARN_ON(&i915->drm, len <= 0))
- return 162000;
- return intel_dp->common_rates[len - 1];
+ return intel_dp_common_rate(intel_dp, len - 1);
}
int intel_dp_rate_select(struct intel_dp *intel_dp, int rate)
@@ -1205,7 +1268,7 @@ intel_dp_compute_link_config_wide(struct intel_dp *intel_dp,
output_bpp);
for (i = 0; i < intel_dp->num_common_rates; i++) {
- link_rate = intel_dp->common_rates[i];
+ link_rate = intel_dp_common_rate(intel_dp, i);
if (link_rate < limits->min_rate ||
link_rate > limits->max_rate)
continue;
@@ -1284,7 +1347,7 @@ static int intel_dp_dsc_compute_params(struct intel_encoder *encoder,
else
vdsc_cfg->slice_height = 2;
- ret = intel_dsc_compute_params(encoder, crtc_state);
+ ret = intel_dsc_compute_params(crtc_state);
if (ret)
return ret;
@@ -1453,17 +1516,10 @@ intel_dp_compute_link_config(struct intel_encoder *encoder,
&pipe_config->hw.adjusted_mode;
struct intel_dp *intel_dp = enc_to_intel_dp(encoder);
struct link_config_limits limits;
- int common_len;
int ret;
- common_len = intel_dp_common_len_rate_limit(intel_dp,
- intel_dp->max_link_rate);
-
- /* No common link rates between source and sink */
- drm_WARN_ON(encoder->base.dev, common_len <= 0);
-
- limits.min_rate = intel_dp->common_rates[0];
- limits.max_rate = intel_dp->common_rates[common_len - 1];
+ limits.min_rate = intel_dp_common_rate(intel_dp, 0);
+ limits.max_rate = intel_dp_max_link_rate(intel_dp);
limits.min_lane_count = 1;
limits.max_lane_count = intel_dp_max_lane_count(intel_dp);
@@ -2154,6 +2210,18 @@ static int intel_dp_hdmi_sink_max_frl(struct intel_dp *intel_dp)
return max_frl_rate;
}
+static bool
+intel_dp_pcon_is_frl_trained(struct intel_dp *intel_dp,
+ u8 max_frl_bw_mask, u8 *frl_trained_mask)
+{
+ if (drm_dp_pcon_hdmi_link_active(&intel_dp->aux) &&
+ drm_dp_pcon_hdmi_link_mode(&intel_dp->aux, frl_trained_mask) == DP_PCON_HDMI_MODE_FRL &&
+ *frl_trained_mask >= max_frl_bw_mask)
+ return true;
+
+ return false;
+}
+
static int intel_dp_pcon_start_frl_training(struct intel_dp *intel_dp)
{
#define TIMEOUT_FRL_READY_MS 500
@@ -2164,10 +2232,6 @@ static int intel_dp_pcon_start_frl_training(struct intel_dp *intel_dp)
u8 max_frl_bw_mask = 0, frl_trained_mask;
bool is_active;
- ret = drm_dp_pcon_reset_frl_config(&intel_dp->aux);
- if (ret < 0)
- return ret;
-
max_pcon_frl_bw = intel_dp->dfp.pcon_max_frl_bw;
drm_dbg(&i915->drm, "PCON max rate = %d Gbps\n", max_pcon_frl_bw);
@@ -2179,6 +2243,12 @@ static int intel_dp_pcon_start_frl_training(struct intel_dp *intel_dp)
if (max_frl_bw <= 0)
return -EINVAL;
+ max_frl_bw_mask = intel_dp_pcon_set_frl_mask(max_frl_bw);
+ drm_dbg(&i915->drm, "MAX_FRL_BW_MASK = %u\n", max_frl_bw_mask);
+
+ if (intel_dp_pcon_is_frl_trained(intel_dp, max_frl_bw_mask, &frl_trained_mask))
+ goto frl_trained;
+
ret = drm_dp_pcon_frl_prepare(&intel_dp->aux, false);
if (ret < 0)
return ret;
@@ -2188,7 +2258,6 @@ static int intel_dp_pcon_start_frl_training(struct intel_dp *intel_dp)
if (!is_active)
return -ETIMEDOUT;
- max_frl_bw_mask = intel_dp_pcon_set_frl_mask(max_frl_bw);
ret = drm_dp_pcon_frl_configure_1(&intel_dp->aux, max_frl_bw,
DP_PCON_ENABLE_SEQUENTIAL_LINK);
if (ret < 0)
@@ -2204,19 +2273,15 @@ static int intel_dp_pcon_start_frl_training(struct intel_dp *intel_dp)
* Wait for FRL to be completed
* Check if the HDMI Link is up and active.
*/
- wait_for(is_active = drm_dp_pcon_hdmi_link_active(&intel_dp->aux) == true, TIMEOUT_HDMI_LINK_ACTIVE_MS);
+ wait_for(is_active =
+ intel_dp_pcon_is_frl_trained(intel_dp, max_frl_bw_mask, &frl_trained_mask),
+ TIMEOUT_HDMI_LINK_ACTIVE_MS);
if (!is_active)
return -ETIMEDOUT;
- /* Verify HDMI Link configuration shows FRL Mode */
- if (drm_dp_pcon_hdmi_link_mode(&intel_dp->aux, &frl_trained_mask) !=
- DP_PCON_HDMI_MODE_FRL) {
- drm_dbg(&i915->drm, "HDMI couldn't be trained in FRL Mode\n");
- return -EINVAL;
- }
- drm_dbg(&i915->drm, "MAX_FRL_MASK = %u, FRL_TRAINED_MASK = %u\n", max_frl_bw_mask, frl_trained_mask);
-
+frl_trained:
+ drm_dbg(&i915->drm, "FRL_TRAINED_MASK = %u\n", frl_trained_mask);
intel_dp->frl.trained_rate_gbps = intel_dp_pcon_get_frl_mask(frl_trained_mask);
intel_dp->frl.is_trained = true;
drm_dbg(&i915->drm, "FRL trained with : %d Gbps\n", intel_dp->frl.trained_rate_gbps);
@@ -2234,6 +2299,28 @@ static bool intel_dp_is_hdmi_2_1_sink(struct intel_dp *intel_dp)
return false;
}
+static
+int intel_dp_pcon_set_tmds_mode(struct intel_dp *intel_dp)
+{
+ int ret;
+ u8 buf = 0;
+
+ /* Set PCON source control mode */
+ buf |= DP_PCON_ENABLE_SOURCE_CTL_MODE;
+
+ ret = drm_dp_dpcd_writeb(&intel_dp->aux, DP_PCON_HDMI_LINK_CONFIG_1, buf);
+ if (ret < 0)
+ return ret;
+
+ /* Set HDMI LINK ENABLE */
+ buf |= DP_PCON_ENABLE_HDMI_LINK;
+ ret = drm_dp_dpcd_writeb(&intel_dp->aux, DP_PCON_HDMI_LINK_CONFIG_1, buf);
+ if (ret < 0)
+ return ret;
+
+ return 0;
+}
+
void intel_dp_check_frl_training(struct intel_dp *intel_dp)
{
struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
@@ -2252,7 +2339,7 @@ void intel_dp_check_frl_training(struct intel_dp *intel_dp)
int ret, mode;
drm_dbg(&dev_priv->drm, "Couldn't set FRL mode, continuing with TMDS mode\n");
- ret = drm_dp_pcon_reset_frl_config(&intel_dp->aux);
+ ret = intel_dp_pcon_set_tmds_mode(intel_dp);
mode = drm_dp_pcon_hdmi_link_mode(&intel_dp->aux, NULL);
if (ret < 0 || mode != DP_PCON_HDMI_MODE_TMDS)
@@ -2614,6 +2701,7 @@ intel_edp_init_dpcd(struct intel_dp *intel_dp)
intel_dp->use_rate_select = true;
else
intel_dp_set_sink_rates(intel_dp);
+ intel_dp_set_max_sink_lane_count(intel_dp);
intel_dp_set_common_rates(intel_dp);
intel_dp_reset_max_link_params(intel_dp);
@@ -2659,6 +2747,7 @@ intel_dp_get_dpcd(struct intel_dp *intel_dp)
drm_dp_is_branch(intel_dp->dpcd));
intel_dp_set_sink_rates(intel_dp);
+ intel_dp_set_max_sink_lane_count(intel_dp);
intel_dp_set_common_rates(intel_dp);
}
@@ -3817,7 +3906,7 @@ int intel_dp_retrain_link(struct intel_encoder *encoder,
to_intel_crtc_state(crtc->base.state);
/* Keep underrun reporting disabled until things are stable */
- intel_wait_for_vblank(dev_priv, crtc->pipe);
+ intel_crtc_wait_for_next_vblank(crtc);
intel_set_cpu_fifo_underrun_reporting(dev_priv, crtc->pipe, true);
if (crtc_state->has_pch_encoder)
@@ -4971,7 +5060,7 @@ static void intel_dp_modeset_retry_work_fn(struct work_struct *work)
DRM_MODE_LINK_STATUS_BAD);
mutex_unlock(&connector->dev->mode_config.mutex);
/* Send Hotplug uevent so userspace can reprobe */
- drm_kms_helper_hotplug_event(connector->dev);
+ drm_kms_helper_connector_hotplug_event(connector);
}
bool
@@ -5025,6 +5114,7 @@ intel_dp_init_connector(struct intel_digital_port *dig_port,
intel_dp_set_source_rates(intel_dp);
intel_dp_set_default_sink_rates(intel_dp);
+ intel_dp_set_default_max_sink_lane_count(intel_dp);
intel_dp_set_common_rates(intel_dp);
intel_dp_reset_max_link_params(intel_dp);
diff --git a/drivers/gpu/drm/i915/display/intel_dp_aux_backlight.c b/drivers/gpu/drm/i915/display/intel_dp_aux_backlight.c
index 3897468..97cf3ca 100644
--- a/drivers/gpu/drm/i915/display/intel_dp_aux_backlight.c
+++ b/drivers/gpu/drm/i915/display/intel_dp_aux_backlight.c
@@ -34,6 +34,7 @@
* for some reason.
*/
+#include "i915_drv.h"
#include "intel_backlight.h"
#include "intel_display_types.h"
#include "intel_dp.h"
@@ -287,6 +288,12 @@ intel_dp_aux_vesa_set_backlight(const struct drm_connector_state *conn_state, u3
struct intel_panel *panel = &connector->panel;
struct intel_dp *intel_dp = enc_to_intel_dp(connector->encoder);
+ if (!panel->backlight.edp.vesa.info.aux_set) {
+ const u32 pwm_level = intel_backlight_level_to_pwm(connector, level);
+
+ intel_backlight_set_pwm_level(conn_state, pwm_level);
+ }
+
drm_edp_backlight_set_level(&intel_dp->aux, &panel->backlight.edp.vesa.info, level);
}
@@ -299,8 +306,13 @@ intel_dp_aux_vesa_enable_backlight(const struct intel_crtc_state *crtc_state,
struct intel_dp *intel_dp = enc_to_intel_dp(connector->encoder);
if (!panel->backlight.edp.vesa.info.aux_enable) {
- u32 pwm_level = intel_backlight_invert_pwm_level(connector,
- panel->backlight.pwm_level_max);
+ u32 pwm_level;
+
+ if (!panel->backlight.edp.vesa.info.aux_set)
+ pwm_level = intel_backlight_level_to_pwm(connector, level);
+ else
+ pwm_level = intel_backlight_invert_pwm_level(connector,
+ panel->backlight.pwm_level_max);
panel->backlight.pwm_funcs->enable(crtc_state, conn_state, pwm_level);
}
@@ -337,7 +349,7 @@ static int intel_dp_aux_vesa_setup_backlight(struct intel_connector *connector,
if (ret < 0)
return ret;
- if (!panel->backlight.edp.vesa.info.aux_enable) {
+ if (!panel->backlight.edp.vesa.info.aux_set || !panel->backlight.edp.vesa.info.aux_enable) {
ret = panel->backlight.pwm_funcs->setup(connector, pipe);
if (ret < 0) {
drm_err(&i915->drm,
@@ -346,14 +358,27 @@ static int intel_dp_aux_vesa_setup_backlight(struct intel_connector *connector,
return ret;
}
}
- panel->backlight.max = panel->backlight.edp.vesa.info.max;
- panel->backlight.min = 0;
- if (current_mode == DP_EDP_BACKLIGHT_CONTROL_MODE_DPCD) {
- panel->backlight.level = current_level;
- panel->backlight.enabled = panel->backlight.level != 0;
+
+ if (panel->backlight.edp.vesa.info.aux_set) {
+ panel->backlight.max = panel->backlight.edp.vesa.info.max;
+ panel->backlight.min = 0;
+ if (current_mode == DP_EDP_BACKLIGHT_CONTROL_MODE_DPCD) {
+ panel->backlight.level = current_level;
+ panel->backlight.enabled = panel->backlight.level != 0;
+ } else {
+ panel->backlight.level = panel->backlight.max;
+ panel->backlight.enabled = false;
+ }
} else {
- panel->backlight.level = panel->backlight.max;
- panel->backlight.enabled = false;
+ panel->backlight.max = panel->backlight.pwm_level_max;
+ panel->backlight.min = panel->backlight.pwm_level_min;
+ if (current_mode == DP_EDP_BACKLIGHT_CONTROL_MODE_PWM) {
+ panel->backlight.level = panel->backlight.pwm_funcs->get(connector, pipe);
+ panel->backlight.enabled = panel->backlight.pwm_enabled;
+ } else {
+ panel->backlight.level = panel->backlight.max;
+ panel->backlight.enabled = false;
+ }
}
return 0;
@@ -437,11 +462,17 @@ int intel_dp_aux_init_backlight_funcs(struct intel_connector *connector)
}
/*
- * A lot of eDP panels in the wild will report supporting both the
- * Intel proprietary backlight control interface, and the VESA
- * backlight control interface. Many of these panels are liars though,
- * and will only work with the Intel interface. So, always probe for
- * that first.
+ * Since Intel has their own backlight control interface, the majority of machines out there
+ * using DPCD backlight controls with Intel GPUs will be using this interface as opposed to
+ * the VESA interface. However, other GPUs (such as Nvidia's) will always use the VESA
+ * interface. This means that there's quite a number of panels out there that will advertise
+ * support for both interfaces, primarily systems with Intel/Nvidia hybrid GPU setups.
+ *
+ * There's a catch to this though: on many panels that advertise support for both
+ * interfaces, the VESA backlight interface will stop working once we've programmed the
+ * panel with Intel's OUI - which is also required for us to be able to detect Intel's
+ * backlight interface at all. This means that the only sensible way for us to detect both
+ * interfaces is to probe for Intel's first, and VESA's second.
*/
if (try_intel_interface && intel_dp_aux_supports_hdr_backlight(connector)) {
drm_dbg_kms(dev, "Using Intel proprietary eDP backlight controls\n");
diff --git a/drivers/gpu/drm/i915/display/intel_dp_link_training.c b/drivers/gpu/drm/i915/display/intel_dp_link_training.c
index 85676c9..9451f33 100644
--- a/drivers/gpu/drm/i915/display/intel_dp_link_training.c
+++ b/drivers/gpu/drm/i915/display/intel_dp_link_training.c
@@ -21,11 +21,11 @@
* IN THE SOFTWARE.
*/
+#include "i915_drv.h"
#include "intel_display_types.h"
#include "intel_dp.h"
#include "intel_dp_link_training.h"
-
static void intel_dp_reset_lttpr_common_caps(struct intel_dp *intel_dp)
{
memset(intel_dp->lttpr_common_caps, 0, sizeof(intel_dp->lttpr_common_caps));
@@ -301,7 +301,10 @@ static u8 intel_dp_phy_preemph_max(struct intel_dp *intel_dp,
static bool has_per_lane_signal_levels(struct intel_dp *intel_dp,
enum drm_dp_phy dp_phy)
{
- return !intel_dp_phy_is_downstream_of_source(intel_dp, dp_phy);
+ struct drm_i915_private *i915 = dp_to_i915(intel_dp);
+
+ return !intel_dp_phy_is_downstream_of_source(intel_dp, dp_phy) ||
+ DISPLAY_VER(i915) >= 11;
}
/* 128b/132b */
@@ -683,15 +686,6 @@ intel_dp_prepare_link_train(struct intel_dp *intel_dp,
return true;
}
-static void intel_dp_link_training_clock_recovery_delay(struct intel_dp *intel_dp,
- enum drm_dp_phy dp_phy)
-{
- if (dp_phy == DP_PHY_DPRX)
- drm_dp_link_train_clock_recovery_delay(&intel_dp->aux, intel_dp->dpcd);
- else
- drm_dp_lttpr_link_train_clock_recovery_delay();
-}
-
static bool intel_dp_adjust_request_changed(const struct intel_crtc_state *crtc_state,
const u8 old_link_status[DP_LINK_STATUS_SIZE],
const u8 new_link_status[DP_LINK_STATUS_SIZE])
@@ -750,6 +744,11 @@ intel_dp_link_training_clock_recovery(struct intel_dp *intel_dp,
u8 link_status[DP_LINK_STATUS_SIZE];
bool max_vswing_reached = false;
char phy_name[10];
+ int delay_us;
+
+ delay_us = drm_dp_read_clock_recovery_delay(&intel_dp->aux,
+ intel_dp->dpcd, dp_phy,
+ intel_dp_is_uhbr(crtc_state));
intel_dp_phy_name(dp_phy, phy_name, sizeof(phy_name));
@@ -777,7 +776,7 @@ intel_dp_link_training_clock_recovery(struct intel_dp *intel_dp,
voltage_tries = 1;
for (cr_tries = 0; cr_tries < max_cr_tries; ++cr_tries) {
- intel_dp_link_training_clock_recovery_delay(intel_dp, dp_phy);
+ usleep_range(delay_us, 2 * delay_us);
if (drm_dp_dpcd_read_phy_link_status(&intel_dp->aux, dp_phy,
link_status) < 0) {
@@ -895,19 +894,6 @@ static u32 intel_dp_training_pattern(struct intel_dp *intel_dp,
return DP_TRAINING_PATTERN_2;
}
-static void
-intel_dp_link_training_channel_equalization_delay(struct intel_dp *intel_dp,
- enum drm_dp_phy dp_phy)
-{
- if (dp_phy == DP_PHY_DPRX) {
- drm_dp_link_train_channel_eq_delay(&intel_dp->aux, intel_dp->dpcd);
- } else {
- const u8 *phy_caps = intel_dp_lttpr_phy_caps(intel_dp, dp_phy);
-
- drm_dp_lttpr_link_train_channel_eq_delay(&intel_dp->aux, phy_caps);
- }
-}
-
/*
* Perform the link training channel equalization phase on the given DP PHY
* using one of training pattern 2, 3 or 4 depending on the source and
@@ -925,6 +911,11 @@ intel_dp_link_training_channel_equalization(struct intel_dp *intel_dp,
u8 link_status[DP_LINK_STATUS_SIZE];
bool channel_eq = false;
char phy_name[10];
+ int delay_us;
+
+ delay_us = drm_dp_read_channel_eq_delay(&intel_dp->aux,
+ intel_dp->dpcd, dp_phy,
+ intel_dp_is_uhbr(crtc_state));
intel_dp_phy_name(dp_phy, phy_name, sizeof(phy_name));
@@ -944,8 +935,8 @@ intel_dp_link_training_channel_equalization(struct intel_dp *intel_dp,
}
for (tries = 0; tries < 5; tries++) {
- intel_dp_link_training_channel_equalization_delay(intel_dp,
- dp_phy);
+ usleep_range(delay_us, 2 * delay_us);
+
if (drm_dp_dpcd_read_phy_link_status(&intel_dp->aux, dp_phy,
link_status) < 0) {
drm_err(&i915->drm,
diff --git a/drivers/gpu/drm/i915/display/intel_dp_mst.c b/drivers/gpu/drm/i915/display/intel_dp_mst.c
index 89d701e..b8bc7d3 100644
--- a/drivers/gpu/drm/i915/display/intel_dp_mst.c
+++ b/drivers/gpu/drm/i915/display/intel_dp_mst.c
@@ -231,6 +231,7 @@ intel_dp_mst_atomic_master_trans_check(struct intel_connector *connector,
struct drm_i915_private *dev_priv = to_i915(state->base.dev);
struct drm_connector_list_iter connector_list_iter;
struct intel_connector *connector_iter;
+ int ret = 0;
if (DISPLAY_VER(dev_priv) < 12)
return 0;
@@ -243,7 +244,6 @@ intel_dp_mst_atomic_master_trans_check(struct intel_connector *connector,
struct intel_digital_connector_state *conn_iter_state;
struct intel_crtc_state *crtc_state;
struct intel_crtc *crtc;
- int ret;
if (connector_iter->mst_port != connector->mst_port ||
connector_iter == connector)
@@ -252,8 +252,8 @@ intel_dp_mst_atomic_master_trans_check(struct intel_connector *connector,
conn_iter_state = intel_atomic_get_digital_connector_state(state,
connector_iter);
if (IS_ERR(conn_iter_state)) {
- drm_connector_list_iter_end(&connector_list_iter);
- return PTR_ERR(conn_iter_state);
+ ret = PTR_ERR(conn_iter_state);
+ break;
}
if (!conn_iter_state->base.crtc)
@@ -262,20 +262,18 @@ intel_dp_mst_atomic_master_trans_check(struct intel_connector *connector,
crtc = to_intel_crtc(conn_iter_state->base.crtc);
crtc_state = intel_atomic_get_crtc_state(&state->base, crtc);
if (IS_ERR(crtc_state)) {
- drm_connector_list_iter_end(&connector_list_iter);
- return PTR_ERR(crtc_state);
+ ret = PTR_ERR(crtc_state);
+ break;
}
ret = drm_atomic_add_affected_planes(&state->base, &crtc->base);
- if (ret) {
- drm_connector_list_iter_end(&connector_list_iter);
- return ret;
- }
+ if (ret)
+ break;
crtc_state->uapi.mode_changed = true;
}
drm_connector_list_iter_end(&connector_list_iter);
- return 0;
+ return ret;
}
static int
@@ -348,16 +346,6 @@ static void wait_for_act_sent(struct intel_encoder *encoder,
drm_dp_check_act_status(&intel_dp->mst_mgr);
}
-static void intel_mst_pre_disable_dp(struct intel_atomic_state *state,
- struct intel_encoder *encoder,
- const struct intel_crtc_state *old_crtc_state,
- const struct drm_connector_state *old_conn_state)
-{
- if (old_crtc_state->has_audio)
- intel_audio_codec_disable(encoder, old_crtc_state,
- old_conn_state);
-}
-
static void intel_mst_disable_dp(struct intel_atomic_state *state,
struct intel_encoder *encoder,
const struct intel_crtc_state *old_crtc_state,
@@ -382,6 +370,9 @@ static void intel_mst_disable_dp(struct intel_atomic_state *state,
if (ret) {
drm_dbg_kms(&i915->drm, "failed to update payload %d\n", ret);
}
+ if (old_crtc_state->has_audio)
+ intel_audio_codec_disable(encoder,
+ old_crtc_state, old_conn_state);
}
static void intel_mst_post_disable_dp(struct intel_atomic_state *state,
@@ -916,7 +907,6 @@ intel_dp_create_fake_mst_encoder(struct intel_digital_port *dig_port, enum pipe
intel_encoder->compute_config = intel_dp_mst_compute_config;
intel_encoder->compute_config_late = intel_dp_mst_compute_config_late;
- intel_encoder->pre_disable = intel_mst_pre_disable_dp;
intel_encoder->disable = intel_mst_disable_dp;
intel_encoder->post_disable = intel_mst_post_disable_dp;
intel_encoder->update_pipe = intel_ddi_update_pipe;
diff --git a/drivers/gpu/drm/i915/display/intel_dpll.c b/drivers/gpu/drm/i915/display/intel_dpll.c
index 04a7af834..1ce0c17 100644
--- a/drivers/gpu/drm/i915/display/intel_dpll.c
+++ b/drivers/gpu/drm/i915/display/intel_dpll.c
@@ -1823,7 +1823,7 @@ void chv_enable_pll(const struct intel_crtc_state *crtc_state)
int vlv_force_pll_on(struct drm_i915_private *dev_priv, enum pipe pipe,
const struct dpll *dpll)
{
- struct intel_crtc *crtc = intel_get_crtc_for_pipe(dev_priv, pipe);
+ struct intel_crtc *crtc = intel_crtc_for_pipe(dev_priv, pipe);
struct intel_crtc_state *crtc_state;
crtc_state = intel_crtc_state_alloc(crtc);
diff --git a/drivers/gpu/drm/i915/display/intel_dpll_mgr.c b/drivers/gpu/drm/i915/display/intel_dpll_mgr.c
index 0a7e04d..fc8fda7 100644
--- a/drivers/gpu/drm/i915/display/intel_dpll_mgr.c
+++ b/drivers/gpu/drm/i915/display/intel_dpll_mgr.c
@@ -26,6 +26,7 @@
#include "intel_dpio_phy.h"
#include "intel_dpll.h"
#include "intel_dpll_mgr.h"
+#include "intel_pch_refclk.h"
#include "intel_tc.h"
/**
@@ -3740,7 +3741,7 @@ static void combo_pll_enable(struct drm_i915_private *dev_priv,
* domain.
*/
pll->wakeref = intel_display_power_get(dev_priv,
- POWER_DOMAIN_DPLL_DC_OFF);
+ POWER_DOMAIN_DC_OFF);
}
icl_pll_power_enable(dev_priv, pll, enable_reg);
@@ -3847,7 +3848,7 @@ static void combo_pll_disable(struct drm_i915_private *dev_priv,
if (IS_JSL_EHL(dev_priv) &&
pll->info->id == DPLL_ID_EHL_DPLL4)
- intel_display_power_put(dev_priv, POWER_DOMAIN_DPLL_DC_OFF,
+ intel_display_power_put(dev_priv, POWER_DOMAIN_DC_OFF,
pll->wakeref);
}
@@ -4231,7 +4232,7 @@ static void readout_dpll_hw_state(struct drm_i915_private *i915,
if (IS_JSL_EHL(i915) && pll->on &&
pll->info->id == DPLL_ID_EHL_DPLL4) {
pll->wakeref = intel_display_power_get(i915,
- POWER_DOMAIN_DPLL_DC_OFF);
+ POWER_DOMAIN_DC_OFF);
}
pll->state.pipe_mask = 0;
diff --git a/drivers/gpu/drm/i915/display/intel_dpll_mgr.h b/drivers/gpu/drm/i915/display/intel_dpll_mgr.h
index 2f59d86..ef28897 100644
--- a/drivers/gpu/drm/i915/display/intel_dpll_mgr.h
+++ b/drivers/gpu/drm/i915/display/intel_dpll_mgr.h
@@ -27,7 +27,6 @@
#include <linux/types.h>
-#include "intel_display.h"
#include "intel_wakeref.h"
/*FIXME: Move this to a more appropriate place. */
@@ -37,6 +36,7 @@
(void) (&__a == &__b); \
__a > __b ? (__a - __b) : (__b - __a); })
+enum tc_port;
struct drm_device;
struct drm_i915_private;
struct intel_atomic_state;
diff --git a/drivers/gpu/drm/i915/display/intel_dpt.c b/drivers/gpu/drm/i915/display/intel_dpt.c
index 8f7b1f7..8f67474 100644
--- a/drivers/gpu/drm/i915/display/intel_dpt.c
+++ b/drivers/gpu/drm/i915/display/intel_dpt.c
@@ -167,6 +167,64 @@ void intel_dpt_unpin(struct i915_address_space *vm)
i915_vma_put(dpt->vma);
}
+/**
+ * intel_dpt_resume - restore the memory mapping for all DPT FBs during system resume
+ * @i915: device instance
+ *
+ * Restore the memory mapping during system resume for all framebuffers which
+ * are mapped to HW via a GGTT->DPT page table. The content of these page
+ * tables are not stored in the hibernation image during S4 and S3RST->S4
+ * transitions, so here we reprogram the PTE entries in those tables.
+ *
+ * This function must be called after the mappings in GGTT have been restored calling
+ * i915_ggtt_resume().
+ */
+void intel_dpt_resume(struct drm_i915_private *i915)
+{
+ struct drm_framebuffer *drm_fb;
+
+ if (!HAS_DISPLAY(i915))
+ return;
+
+ mutex_lock(&i915->drm.mode_config.fb_lock);
+ drm_for_each_fb(drm_fb, &i915->drm) {
+ struct intel_framebuffer *fb = to_intel_framebuffer(drm_fb);
+
+ if (fb->dpt_vm)
+ i915_ggtt_resume_vm(fb->dpt_vm);
+ }
+ mutex_unlock(&i915->drm.mode_config.fb_lock);
+}
+
+/**
+ * intel_dpt_suspend - suspend the memory mapping for all DPT FBs during system suspend
+ * @i915: device instance
+ *
+ * Suspend the memory mapping during system suspend for all framebuffers which
+ * are mapped to HW via a GGTT->DPT page table.
+ *
+ * This function must be called before the mappings in GGTT are suspended calling
+ * i915_ggtt_suspend().
+ */
+void intel_dpt_suspend(struct drm_i915_private *i915)
+{
+ struct drm_framebuffer *drm_fb;
+
+ if (!HAS_DISPLAY(i915))
+ return;
+
+ mutex_lock(&i915->drm.mode_config.fb_lock);
+
+ drm_for_each_fb(drm_fb, &i915->drm) {
+ struct intel_framebuffer *fb = to_intel_framebuffer(drm_fb);
+
+ if (fb->dpt_vm)
+ i915_ggtt_suspend_vm(fb->dpt_vm);
+ }
+
+ mutex_unlock(&i915->drm.mode_config.fb_lock);
+}
+
struct i915_address_space *
intel_dpt_create(struct intel_framebuffer *fb)
{
@@ -206,7 +264,7 @@ intel_dpt_create(struct intel_framebuffer *fb)
vm = &dpt->vm;
- vm->gt = &i915->gt;
+ vm->gt = to_gt(i915);
vm->i915 = i915;
vm->dma = i915->drm.dev;
vm->total = (size / sizeof(gen8_pte_t)) * I915_GTT_PAGE_SIZE;
@@ -221,8 +279,6 @@ intel_dpt_create(struct intel_framebuffer *fb)
vm->vma_ops.bind_vma = dpt_bind_vma;
vm->vma_ops.unbind_vma = dpt_unbind_vma;
- vm->vma_ops.set_pages = ggtt_set_pages;
- vm->vma_ops.clear_pages = clear_pages;
vm->pte_encode = gen8_ggtt_pte_encode;
diff --git a/drivers/gpu/drm/i915/display/intel_dpt.h b/drivers/gpu/drm/i915/display/intel_dpt.h
index 45142b8..e18a9f7 100644
--- a/drivers/gpu/drm/i915/display/intel_dpt.h
+++ b/drivers/gpu/drm/i915/display/intel_dpt.h
@@ -6,6 +6,8 @@
#ifndef __INTEL_DPT_H__
#define __INTEL_DPT_H__
+struct drm_i915_private;
+
struct i915_address_space;
struct i915_vma;
struct intel_framebuffer;
@@ -13,6 +15,8 @@ struct intel_framebuffer;
void intel_dpt_destroy(struct i915_address_space *vm);
struct i915_vma *intel_dpt_pin(struct i915_address_space *vm);
void intel_dpt_unpin(struct i915_address_space *vm);
+void intel_dpt_suspend(struct drm_i915_private *i915);
+void intel_dpt_resume(struct drm_i915_private *i915);
struct i915_address_space *
intel_dpt_create(struct intel_framebuffer *fb);
diff --git a/drivers/gpu/drm/i915/display/intel_dsb.c b/drivers/gpu/drm/i915/display/intel_dsb.c
index 62a8a69..83a69a4 100644
--- a/drivers/gpu/drm/i915/display/intel_dsb.c
+++ b/drivers/gpu/drm/i915/display/intel_dsb.c
@@ -100,7 +100,7 @@ void intel_dsb_indexed_reg_write(const struct intel_crtc_state *crtc_state,
u32 reg_val;
if (!dsb) {
- intel_de_write(dev_priv, reg, val);
+ intel_de_write_fw(dev_priv, reg, val);
return;
}
buf = dsb->cmd_buf;
@@ -177,7 +177,7 @@ void intel_dsb_reg_write(const struct intel_crtc_state *crtc_state,
dsb = crtc_state->dsb;
if (!dsb) {
- intel_de_write(dev_priv, reg, val);
+ intel_de_write_fw(dev_priv, reg, val);
return;
}
diff --git a/drivers/gpu/drm/i915/display/intel_dsi.c b/drivers/gpu/drm/i915/display/intel_dsi.c
index 6b0301b..a50422e 100644
--- a/drivers/gpu/drm/i915/display/intel_dsi.c
+++ b/drivers/gpu/drm/i915/display/intel_dsi.c
@@ -4,6 +4,8 @@
*/
#include <drm/drm_mipi_dsi.h>
+
+#include "i915_drv.h"
#include "intel_dsi.h"
#include "intel_panel.h"
diff --git a/drivers/gpu/drm/i915/display/intel_dsi.h b/drivers/gpu/drm/i915/display/intel_dsi.h
index fbc40ff..a3a906c 100644
--- a/drivers/gpu/drm/i915/display/intel_dsi.h
+++ b/drivers/gpu/drm/i915/display/intel_dsi.h
@@ -166,57 +166,15 @@ static inline u16 intel_dsi_encoder_ports(struct intel_encoder *encoder)
return enc_to_intel_dsi(encoder)->ports;
}
-/* icl_dsi.c */
-void icl_dsi_init(struct drm_i915_private *dev_priv);
-void icl_dsi_frame_update(struct intel_crtc_state *crtc_state);
-
-/* intel_dsi.c */
int intel_dsi_bitrate(const struct intel_dsi *intel_dsi);
int intel_dsi_tlpx_ns(const struct intel_dsi *intel_dsi);
enum drm_panel_orientation
intel_dsi_get_panel_orientation(struct intel_connector *connector);
-
-/* vlv_dsi.c */
-void vlv_dsi_wait_for_fifo_empty(struct intel_dsi *intel_dsi, enum port port);
-enum mipi_dsi_pixel_format pixel_format_from_register_bits(u32 fmt);
int intel_dsi_get_modes(struct drm_connector *connector);
enum drm_mode_status intel_dsi_mode_valid(struct drm_connector *connector,
struct drm_display_mode *mode);
struct intel_dsi_host *intel_dsi_host_init(struct intel_dsi *intel_dsi,
const struct mipi_dsi_host_ops *funcs,
enum port port);
-void vlv_dsi_init(struct drm_i915_private *dev_priv);
-
-/* vlv_dsi_pll.c */
-int vlv_dsi_pll_compute(struct intel_encoder *encoder,
- struct intel_crtc_state *config);
-void vlv_dsi_pll_enable(struct intel_encoder *encoder,
- const struct intel_crtc_state *config);
-void vlv_dsi_pll_disable(struct intel_encoder *encoder);
-u32 vlv_dsi_get_pclk(struct intel_encoder *encoder,
- struct intel_crtc_state *config);
-void vlv_dsi_reset_clocks(struct intel_encoder *encoder, enum port port);
-
-bool bxt_dsi_pll_is_enabled(struct drm_i915_private *dev_priv);
-int bxt_dsi_pll_compute(struct intel_encoder *encoder,
- struct intel_crtc_state *config);
-void bxt_dsi_pll_enable(struct intel_encoder *encoder,
- const struct intel_crtc_state *config);
-void bxt_dsi_pll_disable(struct intel_encoder *encoder);
-u32 bxt_dsi_get_pclk(struct intel_encoder *encoder,
- struct intel_crtc_state *config);
-void bxt_dsi_reset_clocks(struct intel_encoder *encoder, enum port port);
-
-void assert_dsi_pll_enabled(struct drm_i915_private *i915);
-void assert_dsi_pll_disabled(struct drm_i915_private *i915);
-
-/* intel_dsi_vbt.c */
-bool intel_dsi_vbt_init(struct intel_dsi *intel_dsi, u16 panel_id);
-void intel_dsi_vbt_gpio_init(struct intel_dsi *intel_dsi, bool panel_is_on);
-void intel_dsi_vbt_gpio_cleanup(struct intel_dsi *intel_dsi);
-void intel_dsi_vbt_exec_sequence(struct intel_dsi *intel_dsi,
- enum mipi_seq seq_id);
-void intel_dsi_msleep(struct intel_dsi *intel_dsi, int msec);
-void intel_dsi_log_params(struct intel_dsi *intel_dsi);
#endif /* _INTEL_DSI_H */
diff --git a/drivers/gpu/drm/i915/display/intel_dsi_dcs_backlight.c b/drivers/gpu/drm/i915/display/intel_dsi_dcs_backlight.c
index f61ed82e..7d23442 100644
--- a/drivers/gpu/drm/i915/display/intel_dsi_dcs_backlight.c
+++ b/drivers/gpu/drm/i915/display/intel_dsi_dcs_backlight.c
@@ -71,6 +71,7 @@ static void dcs_set_backlight(const struct drm_connector_state *conn_state, u32
u8 data[2] = {};
enum port port;
size_t len = panel->backlight.max > U8_MAX ? 2 : 1;
+ unsigned long mode_flags;
if (len == 1) {
data[0] = level;
@@ -81,8 +82,11 @@ static void dcs_set_backlight(const struct drm_connector_state *conn_state, u32
for_each_dsi_port(port, intel_dsi->dcs_backlight_ports) {
dsi_device = intel_dsi->dsi_hosts[port]->device;
+ mode_flags = dsi_device->mode_flags;
+ dsi_device->mode_flags &= ~MIPI_DSI_MODE_LPM;
mipi_dsi_dcs_write(dsi_device, MIPI_DCS_SET_DISPLAY_BRIGHTNESS,
&data, len);
+ dsi_device->mode_flags = mode_flags;
}
}
diff --git a/drivers/gpu/drm/i915/display/intel_dsi_vbt.c b/drivers/gpu/drm/i915/display/intel_dsi_vbt.c
index f241bed..0da9184 100644
--- a/drivers/gpu/drm/i915/display/intel_dsi_vbt.c
+++ b/drivers/gpu/drm/i915/display/intel_dsi_vbt.c
@@ -41,6 +41,8 @@
#include "i915_drv.h"
#include "intel_display_types.h"
#include "intel_dsi.h"
+#include "intel_dsi_vbt.h"
+#include "vlv_dsi.h"
#include "vlv_sideband.h"
#define MIPI_TRANSFER_MODE_SHIFT 0
diff --git a/drivers/gpu/drm/i915/display/intel_dsi_vbt.h b/drivers/gpu/drm/i915/display/intel_dsi_vbt.h
new file mode 100644
index 0000000..dc642c1
--- /dev/null
+++ b/drivers/gpu/drm/i915/display/intel_dsi_vbt.h
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2021 Intel Corporation
+ */
+
+#ifndef __INTEL_DSI_VBT_H__
+#define __INTEL_DSI_VBT_H__
+
+#include <linux/types.h>
+
+enum mipi_seq;
+struct intel_dsi;
+
+bool intel_dsi_vbt_init(struct intel_dsi *intel_dsi, u16 panel_id);
+void intel_dsi_vbt_gpio_init(struct intel_dsi *intel_dsi, bool panel_is_on);
+void intel_dsi_vbt_gpio_cleanup(struct intel_dsi *intel_dsi);
+void intel_dsi_vbt_exec_sequence(struct intel_dsi *intel_dsi,
+ enum mipi_seq seq_id);
+void intel_dsi_msleep(struct intel_dsi *intel_dsi, int msec);
+void intel_dsi_log_params(struct intel_dsi *intel_dsi);
+
+#endif /* __INTEL_DSI_VBT_H__ */
diff --git a/drivers/gpu/drm/i915/display/intel_fb.c b/drivers/gpu/drm/i915/display/intel_fb.c
index cb511b2..23cfe2e 100644
--- a/drivers/gpu/drm/i915/display/intel_fb.c
+++ b/drivers/gpu/drm/i915/display/intel_fb.c
@@ -6,6 +6,7 @@
#include <drm/drm_framebuffer.h>
#include <drm/drm_modeset_helper.h>
+#include "i915_drv.h"
#include "intel_display.h"
#include "intel_display_types.h"
#include "intel_dpt.h"
@@ -13,26 +14,465 @@
#define check_array_bounds(i915, a, i) drm_WARN_ON(&(i915)->drm, (i) >= ARRAY_SIZE(a))
-bool is_ccs_plane(const struct drm_framebuffer *fb, int plane)
+/*
+ * From the Sky Lake PRM:
+ * "The Color Control Surface (CCS) contains the compression status of
+ * the cache-line pairs. The compression state of the cache-line pair
+ * is specified by 2 bits in the CCS. Each CCS cache-line represents
+ * an area on the main surface of 16 x16 sets of 128 byte Y-tiled
+ * cache-line-pairs. CCS is always Y tiled."
+ *
+ * Since cache line pairs refers to horizontally adjacent cache lines,
+ * each cache line in the CCS corresponds to an area of 32x16 cache
+ * lines on the main surface. Since each pixel is 4 bytes, this gives
+ * us a ratio of one byte in the CCS for each 8x16 pixels in the
+ * main surface.
+ */
+static const struct drm_format_info skl_ccs_formats[] = {
+ { .format = DRM_FORMAT_XRGB8888, .depth = 24, .num_planes = 2,
+ .cpp = { 4, 1, }, .hsub = 8, .vsub = 16, },
+ { .format = DRM_FORMAT_XBGR8888, .depth = 24, .num_planes = 2,
+ .cpp = { 4, 1, }, .hsub = 8, .vsub = 16, },
+ { .format = DRM_FORMAT_ARGB8888, .depth = 32, .num_planes = 2,
+ .cpp = { 4, 1, }, .hsub = 8, .vsub = 16, .has_alpha = true, },
+ { .format = DRM_FORMAT_ABGR8888, .depth = 32, .num_planes = 2,
+ .cpp = { 4, 1, }, .hsub = 8, .vsub = 16, .has_alpha = true, },
+};
+
+/*
+ * Gen-12 compression uses 4 bits of CCS data for each cache line pair in the
+ * main surface. And each 64B CCS cache line represents an area of 4x1 Y-tiles
+ * in the main surface. With 4 byte pixels and each Y-tile having dimensions of
+ * 32x32 pixels, the ratio turns out to 1B in the CCS for every 2x32 pixels in
+ * the main surface.
+ */
+static const struct drm_format_info gen12_ccs_formats[] = {
+ { .format = DRM_FORMAT_XRGB8888, .depth = 24, .num_planes = 2,
+ .char_per_block = { 4, 1 }, .block_w = { 1, 2 }, .block_h = { 1, 1 },
+ .hsub = 1, .vsub = 1, },
+ { .format = DRM_FORMAT_XBGR8888, .depth = 24, .num_planes = 2,
+ .char_per_block = { 4, 1 }, .block_w = { 1, 2 }, .block_h = { 1, 1 },
+ .hsub = 1, .vsub = 1, },
+ { .format = DRM_FORMAT_ARGB8888, .depth = 32, .num_planes = 2,
+ .char_per_block = { 4, 1 }, .block_w = { 1, 2 }, .block_h = { 1, 1 },
+ .hsub = 1, .vsub = 1, .has_alpha = true },
+ { .format = DRM_FORMAT_ABGR8888, .depth = 32, .num_planes = 2,
+ .char_per_block = { 4, 1 }, .block_w = { 1, 2 }, .block_h = { 1, 1 },
+ .hsub = 1, .vsub = 1, .has_alpha = true },
+ { .format = DRM_FORMAT_YUYV, .num_planes = 2,
+ .char_per_block = { 2, 1 }, .block_w = { 1, 2 }, .block_h = { 1, 1 },
+ .hsub = 2, .vsub = 1, .is_yuv = true },
+ { .format = DRM_FORMAT_YVYU, .num_planes = 2,
+ .char_per_block = { 2, 1 }, .block_w = { 1, 2 }, .block_h = { 1, 1 },
+ .hsub = 2, .vsub = 1, .is_yuv = true },
+ { .format = DRM_FORMAT_UYVY, .num_planes = 2,
+ .char_per_block = { 2, 1 }, .block_w = { 1, 2 }, .block_h = { 1, 1 },
+ .hsub = 2, .vsub = 1, .is_yuv = true },
+ { .format = DRM_FORMAT_VYUY, .num_planes = 2,
+ .char_per_block = { 2, 1 }, .block_w = { 1, 2 }, .block_h = { 1, 1 },
+ .hsub = 2, .vsub = 1, .is_yuv = true },
+ { .format = DRM_FORMAT_XYUV8888, .num_planes = 2,
+ .char_per_block = { 4, 1 }, .block_w = { 1, 2 }, .block_h = { 1, 1 },
+ .hsub = 1, .vsub = 1, .is_yuv = true },
+ { .format = DRM_FORMAT_NV12, .num_planes = 4,
+ .char_per_block = { 1, 2, 1, 1 }, .block_w = { 1, 1, 4, 4 }, .block_h = { 1, 1, 1, 1 },
+ .hsub = 2, .vsub = 2, .is_yuv = true },
+ { .format = DRM_FORMAT_P010, .num_planes = 4,
+ .char_per_block = { 2, 4, 1, 1 }, .block_w = { 1, 1, 2, 2 }, .block_h = { 1, 1, 1, 1 },
+ .hsub = 2, .vsub = 2, .is_yuv = true },
+ { .format = DRM_FORMAT_P012, .num_planes = 4,
+ .char_per_block = { 2, 4, 1, 1 }, .block_w = { 1, 1, 2, 2 }, .block_h = { 1, 1, 1, 1 },
+ .hsub = 2, .vsub = 2, .is_yuv = true },
+ { .format = DRM_FORMAT_P016, .num_planes = 4,
+ .char_per_block = { 2, 4, 1, 1 }, .block_w = { 1, 1, 2, 2 }, .block_h = { 1, 1, 1, 1 },
+ .hsub = 2, .vsub = 2, .is_yuv = true },
+};
+
+/*
+ * Same as gen12_ccs_formats[] above, but with additional surface used
+ * to pass Clear Color information in plane 2 with 64 bits of data.
+ */
+static const struct drm_format_info gen12_ccs_cc_formats[] = {
+ { .format = DRM_FORMAT_XRGB8888, .depth = 24, .num_planes = 3,
+ .char_per_block = { 4, 1, 0 }, .block_w = { 1, 2, 2 }, .block_h = { 1, 1, 1 },
+ .hsub = 1, .vsub = 1, },
+ { .format = DRM_FORMAT_XBGR8888, .depth = 24, .num_planes = 3,
+ .char_per_block = { 4, 1, 0 }, .block_w = { 1, 2, 2 }, .block_h = { 1, 1, 1 },
+ .hsub = 1, .vsub = 1, },
+ { .format = DRM_FORMAT_ARGB8888, .depth = 32, .num_planes = 3,
+ .char_per_block = { 4, 1, 0 }, .block_w = { 1, 2, 2 }, .block_h = { 1, 1, 1 },
+ .hsub = 1, .vsub = 1, .has_alpha = true },
+ { .format = DRM_FORMAT_ABGR8888, .depth = 32, .num_planes = 3,
+ .char_per_block = { 4, 1, 0 }, .block_w = { 1, 2, 2 }, .block_h = { 1, 1, 1 },
+ .hsub = 1, .vsub = 1, .has_alpha = true },
+};
+
+struct intel_modifier_desc {
+ u64 modifier;
+ struct {
+ u8 from;
+ u8 until;
+ } display_ver;
+#define DISPLAY_VER_ALL { 0, -1 }
+
+ const struct drm_format_info *formats;
+ int format_count;
+#define FORMAT_OVERRIDE(format_list) \
+ .formats = format_list, \
+ .format_count = ARRAY_SIZE(format_list)
+
+ u8 plane_caps;
+
+ struct {
+ u8 cc_planes:3;
+ u8 packed_aux_planes:4;
+ u8 planar_aux_planes:4;
+ } ccs;
+};
+
+#define INTEL_PLANE_CAP_CCS_MASK (INTEL_PLANE_CAP_CCS_RC | \
+ INTEL_PLANE_CAP_CCS_RC_CC | \
+ INTEL_PLANE_CAP_CCS_MC)
+#define INTEL_PLANE_CAP_TILING_MASK (INTEL_PLANE_CAP_TILING_X | \
+ INTEL_PLANE_CAP_TILING_Y | \
+ INTEL_PLANE_CAP_TILING_Yf)
+#define INTEL_PLANE_CAP_TILING_NONE 0
+
+static const struct intel_modifier_desc intel_modifiers[] = {
+ {
+ .modifier = I915_FORMAT_MOD_Y_TILED_GEN12_MC_CCS,
+ .display_ver = { 12, 13 },
+ .plane_caps = INTEL_PLANE_CAP_TILING_Y | INTEL_PLANE_CAP_CCS_MC,
+
+ .ccs.packed_aux_planes = BIT(1),
+ .ccs.planar_aux_planes = BIT(2) | BIT(3),
+
+ FORMAT_OVERRIDE(gen12_ccs_formats),
+ }, {
+ .modifier = I915_FORMAT_MOD_Y_TILED_GEN12_RC_CCS,
+ .display_ver = { 12, 13 },
+ .plane_caps = INTEL_PLANE_CAP_TILING_Y | INTEL_PLANE_CAP_CCS_RC,
+
+ .ccs.packed_aux_planes = BIT(1),
+
+ FORMAT_OVERRIDE(gen12_ccs_formats),
+ }, {
+ .modifier = I915_FORMAT_MOD_Y_TILED_GEN12_RC_CCS_CC,
+ .display_ver = { 12, 13 },
+ .plane_caps = INTEL_PLANE_CAP_TILING_Y | INTEL_PLANE_CAP_CCS_RC_CC,
+
+ .ccs.cc_planes = BIT(2),
+ .ccs.packed_aux_planes = BIT(1),
+
+ FORMAT_OVERRIDE(gen12_ccs_cc_formats),
+ }, {
+ .modifier = I915_FORMAT_MOD_Yf_TILED_CCS,
+ .display_ver = { 9, 11 },
+ .plane_caps = INTEL_PLANE_CAP_TILING_Yf | INTEL_PLANE_CAP_CCS_RC,
+
+ .ccs.packed_aux_planes = BIT(1),
+
+ FORMAT_OVERRIDE(skl_ccs_formats),
+ }, {
+ .modifier = I915_FORMAT_MOD_Y_TILED_CCS,
+ .display_ver = { 9, 11 },
+ .plane_caps = INTEL_PLANE_CAP_TILING_Y | INTEL_PLANE_CAP_CCS_RC,
+
+ .ccs.packed_aux_planes = BIT(1),
+
+ FORMAT_OVERRIDE(skl_ccs_formats),
+ }, {
+ .modifier = I915_FORMAT_MOD_Yf_TILED,
+ .display_ver = { 9, 11 },
+ .plane_caps = INTEL_PLANE_CAP_TILING_Yf,
+ }, {
+ .modifier = I915_FORMAT_MOD_Y_TILED,
+ .display_ver = { 9, 13 },
+ .plane_caps = INTEL_PLANE_CAP_TILING_Y,
+ }, {
+ .modifier = I915_FORMAT_MOD_X_TILED,
+ .display_ver = DISPLAY_VER_ALL,
+ .plane_caps = INTEL_PLANE_CAP_TILING_X,
+ }, {
+ .modifier = DRM_FORMAT_MOD_LINEAR,
+ .display_ver = DISPLAY_VER_ALL,
+ },
+};
+
+static const struct intel_modifier_desc *lookup_modifier_or_null(u64 modifier)
{
- if (!is_ccs_modifier(fb->modifier))
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(intel_modifiers); i++)
+ if (intel_modifiers[i].modifier == modifier)
+ return &intel_modifiers[i];
+
+ return NULL;
+}
+
+static const struct intel_modifier_desc *lookup_modifier(u64 modifier)
+{
+ const struct intel_modifier_desc *md = lookup_modifier_or_null(modifier);
+
+ if (WARN_ON(!md))
+ return &intel_modifiers[0];
+
+ return md;
+}
+
+static const struct drm_format_info *
+lookup_format_info(const struct drm_format_info formats[],
+ int num_formats, u32 format)
+{
+ int i;
+
+ for (i = 0; i < num_formats; i++) {
+ if (formats[i].format == format)
+ return &formats[i];
+ }
+
+ return NULL;
+}
+
+/**
+ * intel_fb_get_format_info: Get a modifier specific format information
+ * @cmd: FB add command structure
+ *
+ * Returns:
+ * Returns the format information for @cmd->pixel_format specific to @cmd->modifier[0],
+ * or %NULL if the modifier doesn't override the format.
+ */
+const struct drm_format_info *
+intel_fb_get_format_info(const struct drm_mode_fb_cmd2 *cmd)
+{
+ const struct intel_modifier_desc *md = lookup_modifier_or_null(cmd->modifier[0]);
+
+ if (!md || !md->formats)
+ return NULL;
+
+ return lookup_format_info(md->formats, md->format_count, cmd->pixel_format);
+}
+
+static bool plane_caps_contain_any(u8 caps, u8 mask)
+{
+ return caps & mask;
+}
+
+static bool plane_caps_contain_all(u8 caps, u8 mask)
+{
+ return (caps & mask) == mask;
+}
+
+/**
+ * intel_fb_is_ccs_modifier: Check if a modifier is a CCS modifier type
+ * @modifier: Modifier to check
+ *
+ * Returns:
+ * Returns %true if @modifier is a render, render with color clear or
+ * media compression modifier.
+ */
+bool intel_fb_is_ccs_modifier(u64 modifier)
+{
+ return plane_caps_contain_any(lookup_modifier(modifier)->plane_caps,
+ INTEL_PLANE_CAP_CCS_MASK);
+}
+
+/**
+ * intel_fb_is_rc_ccs_cc_modifier: Check if a modifier is an RC CCS CC modifier type
+ * @modifier: Modifier to check
+ *
+ * Returns:
+ * Returns %true if @modifier is a render with color clear modifier.
+ */
+bool intel_fb_is_rc_ccs_cc_modifier(u64 modifier)
+{
+ return plane_caps_contain_any(lookup_modifier(modifier)->plane_caps,
+ INTEL_PLANE_CAP_CCS_RC_CC);
+}
+
+/**
+ * intel_fb_is_mc_ccs_modifier: Check if a modifier is an MC CCS modifier type
+ * @modifier: Modifier to check
+ *
+ * Returns:
+ * Returns %true if @modifier is a media compression modifier.
+ */
+bool intel_fb_is_mc_ccs_modifier(u64 modifier)
+{
+ return plane_caps_contain_any(lookup_modifier(modifier)->plane_caps,
+ INTEL_PLANE_CAP_CCS_MC);
+}
+
+static bool check_modifier_display_ver_range(const struct intel_modifier_desc *md,
+ u8 display_ver_from, u8 display_ver_until)
+{
+ return md->display_ver.from <= display_ver_until &&
+ display_ver_from <= md->display_ver.until;
+}
+
+static bool plane_has_modifier(struct drm_i915_private *i915,
+ u8 plane_caps,
+ const struct intel_modifier_desc *md)
+{
+ if (!IS_DISPLAY_VER(i915, md->display_ver.from, md->display_ver.until))
return false;
- return plane >= fb->format->num_planes / 2;
+ if (!plane_caps_contain_all(plane_caps, md->plane_caps))
+ return false;
+
+ return true;
}
-bool is_gen12_ccs_plane(const struct drm_framebuffer *fb, int plane)
+/**
+ * intel_fb_plane_get_modifiers: Get the modifiers for the given platform and plane capabilities
+ * @i915: i915 device instance
+ * @plane_caps: capabilities for the plane the modifiers are queried for
+ *
+ * Returns:
+ * Returns the list of modifiers allowed by the @i915 platform and @plane_caps.
+ * The caller must free the returned buffer.
+ */
+u64 *intel_fb_plane_get_modifiers(struct drm_i915_private *i915,
+ u8 plane_caps)
{
- return is_gen12_ccs_modifier(fb->modifier) && is_ccs_plane(fb, plane);
+ u64 *list, *p;
+ int count = 1; /* +1 for invalid modifier terminator */
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(intel_modifiers); i++) {
+ if (plane_has_modifier(i915, plane_caps, &intel_modifiers[i]))
+ count++;
+ }
+
+ list = kmalloc_array(count, sizeof(*list), GFP_KERNEL);
+ if (drm_WARN_ON(&i915->drm, !list))
+ return NULL;
+
+ p = list;
+ for (i = 0; i < ARRAY_SIZE(intel_modifiers); i++) {
+ if (plane_has_modifier(i915, plane_caps, &intel_modifiers[i]))
+ *p++ = intel_modifiers[i].modifier;
+ }
+ *p++ = DRM_FORMAT_MOD_INVALID;
+
+ return list;
}
-bool is_gen12_ccs_cc_plane(const struct drm_framebuffer *fb, int plane)
+/**
+ * intel_fb_plane_supports_modifier: Determine if a modifier is supported by the given plane
+ * @plane: Plane to check the modifier support for
+ * @modifier: The modifier to check the support for
+ *
+ * Returns:
+ * %true if the @modifier is supported on @plane.
+ */
+bool intel_fb_plane_supports_modifier(struct intel_plane *plane, u64 modifier)
{
- return fb->modifier == I915_FORMAT_MOD_Y_TILED_GEN12_RC_CCS_CC &&
- plane == 2;
+ int i;
+
+ for (i = 0; i < plane->base.modifier_count; i++)
+ if (plane->base.modifiers[i] == modifier)
+ return true;
+
+ return false;
}
-bool is_semiplanar_uv_plane(const struct drm_framebuffer *fb, int color_plane)
+static bool format_is_yuv_semiplanar(const struct intel_modifier_desc *md,
+ const struct drm_format_info *info)
+{
+ int yuv_planes;
+
+ if (!info->is_yuv)
+ return false;
+
+ if (plane_caps_contain_any(md->plane_caps, INTEL_PLANE_CAP_CCS_MASK))
+ yuv_planes = 4;
+ else
+ yuv_planes = 2;
+
+ return info->num_planes == yuv_planes;
+}
+
+/**
+ * intel_format_info_is_yuv_semiplanar: Check if the given format is YUV semiplanar
+ * @info: format to check
+ * @modifier: modifier used with the format
+ *
+ * Returns:
+ * %true if @info / @modifier is YUV semiplanar.
+ */
+bool intel_format_info_is_yuv_semiplanar(const struct drm_format_info *info,
+ u64 modifier)
+{
+ return format_is_yuv_semiplanar(lookup_modifier(modifier), info);
+}
+
+static u8 ccs_aux_plane_mask(const struct intel_modifier_desc *md,
+ const struct drm_format_info *format)
+{
+ if (format_is_yuv_semiplanar(md, format))
+ return md->ccs.planar_aux_planes;
+ else
+ return md->ccs.packed_aux_planes;
+}
+
+/**
+ * intel_fb_is_ccs_aux_plane: Check if a framebuffer color plane is a CCS AUX plane
+ * @fb: Framebuffer
+ * @color_plane: color plane index to check
+ *
+ * Returns:
+ * Returns %true if @fb's color plane at index @color_plane is a CCS AUX plane.
+ */
+bool intel_fb_is_ccs_aux_plane(const struct drm_framebuffer *fb, int color_plane)
+{
+ const struct intel_modifier_desc *md = lookup_modifier(fb->modifier);
+
+ return ccs_aux_plane_mask(md, fb->format) & BIT(color_plane);
+}
+
+/**
+ * intel_fb_is_gen12_ccs_aux_plane: Check if a framebuffer color plane is a GEN12 CCS AUX plane
+ * @fb: Framebuffer
+ * @color_plane: color plane index to check
+ *
+ * Returns:
+ * Returns %true if @fb's color plane at index @color_plane is a GEN12 CCS AUX plane.
+ */
+static bool intel_fb_is_gen12_ccs_aux_plane(const struct drm_framebuffer *fb, int color_plane)
+{
+ const struct intel_modifier_desc *md = lookup_modifier(fb->modifier);
+
+ return check_modifier_display_ver_range(md, 12, 13) &&
+ ccs_aux_plane_mask(md, fb->format) & BIT(color_plane);
+}
+
+/**
+ * intel_fb_rc_ccs_cc_plane: Get the CCS CC color plane index for a framebuffer
+ * @fb: Framebuffer
+ *
+ * Returns:
+ * Returns the index of the color clear plane for @fb, or -1 if @fb is not a
+ * framebuffer using a render compression/color clear modifier.
+ */
+int intel_fb_rc_ccs_cc_plane(const struct drm_framebuffer *fb)
+{
+ const struct intel_modifier_desc *md = lookup_modifier(fb->modifier);
+
+ if (!md->ccs.cc_planes)
+ return -1;
+
+ drm_WARN_ON_ONCE(fb->dev, hweight8(md->ccs.cc_planes) > 1);
+
+ return ilog2((int)md->ccs.cc_planes);
+}
+
+static bool is_gen12_ccs_cc_plane(const struct drm_framebuffer *fb, int color_plane)
+{
+ return intel_fb_rc_ccs_cc_plane(fb) == color_plane;
+}
+
+static bool is_semiplanar_uv_plane(const struct drm_framebuffer *fb, int color_plane)
{
return intel_format_info_is_yuv_semiplanar(fb->format, fb->modifier) &&
color_plane == 1;
@@ -41,12 +481,13 @@ bool is_semiplanar_uv_plane(const struct drm_framebuffer *fb, int color_plane)
bool is_surface_linear(const struct drm_framebuffer *fb, int color_plane)
{
return fb->modifier == DRM_FORMAT_MOD_LINEAR ||
- is_gen12_ccs_plane(fb, color_plane);
+ intel_fb_is_gen12_ccs_aux_plane(fb, color_plane) ||
+ is_gen12_ccs_cc_plane(fb, color_plane);
}
int main_to_ccs_plane(const struct drm_framebuffer *fb, int main_plane)
{
- drm_WARN_ON(fb->dev, !is_ccs_modifier(fb->modifier) ||
+ drm_WARN_ON(fb->dev, !intel_fb_is_ccs_modifier(fb->modifier) ||
(main_plane && main_plane >= fb->format->num_planes / 2));
return fb->format->num_planes / 2 + main_plane;
@@ -54,7 +495,7 @@ int main_to_ccs_plane(const struct drm_framebuffer *fb, int main_plane)
int skl_ccs_to_main_plane(const struct drm_framebuffer *fb, int ccs_plane)
{
- drm_WARN_ON(fb->dev, !is_ccs_modifier(fb->modifier) ||
+ drm_WARN_ON(fb->dev, !intel_fb_is_ccs_modifier(fb->modifier) ||
ccs_plane < fb->format->num_planes / 2);
if (is_gen12_ccs_cc_plane(fb, ccs_plane))
@@ -63,35 +504,12 @@ int skl_ccs_to_main_plane(const struct drm_framebuffer *fb, int ccs_plane)
return ccs_plane - fb->format->num_planes / 2;
}
-static unsigned int gen12_aligned_scanout_stride(const struct intel_framebuffer *fb,
- int color_plane)
-{
- struct drm_i915_private *i915 = to_i915(fb->base.dev);
- unsigned int stride = fb->base.pitches[color_plane];
-
- if (IS_ALDERLAKE_P(i915))
- return roundup_pow_of_two(max(stride,
- 8u * intel_tile_width_bytes(&fb->base, color_plane)));
-
- return stride;
-}
-
static unsigned int gen12_ccs_aux_stride(struct intel_framebuffer *fb, int ccs_plane)
{
- struct drm_i915_private *i915 = to_i915(fb->base.dev);
int main_plane = skl_ccs_to_main_plane(&fb->base, ccs_plane);
unsigned int main_stride = fb->base.pitches[main_plane];
unsigned int main_tile_width = intel_tile_width_bytes(&fb->base, main_plane);
- /*
- * On ADL-P the AUX stride must align with a power-of-two aligned main
- * surface stride. The stride of the allocated main surface object can
- * be less than this POT stride, which is then autopadded to the POT
- * size.
- */
- if (IS_ALDERLAKE_P(i915))
- main_stride = gen12_aligned_scanout_stride(fb, main_plane);
-
return DIV_ROUND_UP(main_stride, 4 * main_tile_width) * 64;
}
@@ -99,7 +517,7 @@ int skl_main_to_aux_plane(const struct drm_framebuffer *fb, int main_plane)
{
struct drm_i915_private *i915 = to_i915(fb->dev);
- if (is_ccs_modifier(fb->modifier))
+ if (intel_fb_is_ccs_modifier(fb->modifier))
return main_to_ccs_plane(fb, main_plane);
else if (DISPLAY_VER(i915) < 11 &&
intel_format_info_is_yuv_semiplanar(fb->format, fb->modifier))
@@ -128,13 +546,14 @@ intel_tile_width_bytes(const struct drm_framebuffer *fb, int color_plane)
else
return 512;
case I915_FORMAT_MOD_Y_TILED_CCS:
- if (is_ccs_plane(fb, color_plane))
+ if (intel_fb_is_ccs_aux_plane(fb, color_plane))
return 128;
fallthrough;
case I915_FORMAT_MOD_Y_TILED_GEN12_RC_CCS:
case I915_FORMAT_MOD_Y_TILED_GEN12_RC_CCS_CC:
case I915_FORMAT_MOD_Y_TILED_GEN12_MC_CCS:
- if (is_ccs_plane(fb, color_plane))
+ if (intel_fb_is_ccs_aux_plane(fb, color_plane) ||
+ is_gen12_ccs_cc_plane(fb, color_plane))
return 64;
fallthrough;
case I915_FORMAT_MOD_Y_TILED:
@@ -143,7 +562,7 @@ intel_tile_width_bytes(const struct drm_framebuffer *fb, int color_plane)
else
return 512;
case I915_FORMAT_MOD_Yf_TILED_CCS:
- if (is_ccs_plane(fb, color_plane))
+ if (intel_fb_is_ccs_aux_plane(fb, color_plane))
return 128;
fallthrough;
case I915_FORMAT_MOD_Yf_TILED:
@@ -199,7 +618,7 @@ static void intel_tile_block_dims(const struct drm_framebuffer *fb, int color_pl
{
intel_tile_dims(fb, color_plane, tile_width, tile_height);
- if (is_gen12_ccs_plane(fb, color_plane))
+ if (intel_fb_is_gen12_ccs_aux_plane(fb, color_plane))
*tile_height = 1;
}
@@ -223,20 +642,33 @@ intel_fb_align_height(const struct drm_framebuffer *fb,
static unsigned int intel_fb_modifier_to_tiling(u64 fb_modifier)
{
- switch (fb_modifier) {
- case I915_FORMAT_MOD_X_TILED:
- return I915_TILING_X;
- case I915_FORMAT_MOD_Y_TILED:
- case I915_FORMAT_MOD_Y_TILED_CCS:
- case I915_FORMAT_MOD_Y_TILED_GEN12_RC_CCS:
- case I915_FORMAT_MOD_Y_TILED_GEN12_RC_CCS_CC:
- case I915_FORMAT_MOD_Y_TILED_GEN12_MC_CCS:
+ u8 tiling_caps = lookup_modifier(fb_modifier)->plane_caps &
+ INTEL_PLANE_CAP_TILING_MASK;
+
+ switch (tiling_caps) {
+ case INTEL_PLANE_CAP_TILING_Y:
return I915_TILING_Y;
+ case INTEL_PLANE_CAP_TILING_X:
+ return I915_TILING_X;
+ case INTEL_PLANE_CAP_TILING_Yf:
+ case INTEL_PLANE_CAP_TILING_NONE:
+ return I915_TILING_NONE;
default:
+ MISSING_CASE(tiling_caps);
return I915_TILING_NONE;
}
}
+static bool intel_modifier_uses_dpt(struct drm_i915_private *i915, u64 modifier)
+{
+ return DISPLAY_VER(i915) >= 13 && modifier != DRM_FORMAT_MOD_LINEAR;
+}
+
+bool intel_fb_uses_dpt(const struct drm_framebuffer *fb)
+{
+ return fb && intel_modifier_uses_dpt(to_i915(fb->dev), fb->modifier);
+}
+
unsigned int intel_cursor_alignment(const struct drm_i915_private *i915)
{
if (IS_I830(i915))
@@ -271,7 +703,7 @@ unsigned int intel_surf_alignment(const struct drm_framebuffer *fb,
return 512 * 4096;
/* AUX_DIST needs only 4K alignment */
- if (is_ccs_plane(fb, color_plane))
+ if (intel_fb_is_ccs_aux_plane(fb, color_plane))
return 4096;
if (is_semiplanar_uv_plane(fb, color_plane)) {
@@ -330,7 +762,7 @@ void intel_fb_plane_get_subsampling(int *hsub, int *vsub,
* TODO: Deduct the subsampling from the char block for all CCS
* formats and planes.
*/
- if (!is_gen12_ccs_plane(fb, color_plane)) {
+ if (!intel_fb_is_gen12_ccs_aux_plane(fb, color_plane)) {
*hsub = fb->format->hsub;
*vsub = fb->format->vsub;
@@ -357,24 +789,13 @@ void intel_fb_plane_get_subsampling(int *hsub, int *vsub,
static void intel_fb_plane_dims(const struct intel_framebuffer *fb, int color_plane, int *w, int *h)
{
- struct drm_i915_private *i915 = to_i915(fb->base.dev);
- int main_plane = is_ccs_plane(&fb->base, color_plane) ?
+ int main_plane = intel_fb_is_ccs_aux_plane(&fb->base, color_plane) ?
skl_ccs_to_main_plane(&fb->base, color_plane) : 0;
unsigned int main_width = fb->base.width;
unsigned int main_height = fb->base.height;
int main_hsub, main_vsub;
int hsub, vsub;
- /*
- * On ADL-P the CCS AUX surface layout always aligns with the
- * power-of-two aligned main surface stride. The main surface
- * stride in the allocated FB object may not be power-of-two
- * sized, in which case it is auto-padded to the POT size.
- */
- if (IS_ALDERLAKE_P(i915) && is_ccs_plane(&fb->base, color_plane))
- main_width = gen12_aligned_scanout_stride(fb, 0) /
- fb->base.format->cpp[0];
-
intel_fb_plane_get_subsampling(&main_hsub, &main_vsub, &fb->base, main_plane);
intel_fb_plane_get_subsampling(&hsub, &vsub, &fb->base, color_plane);
@@ -409,6 +830,20 @@ static u32 intel_adjust_tile_offset(int *x, int *y,
return new_offset;
}
+static u32 intel_adjust_linear_offset(int *x, int *y,
+ unsigned int cpp,
+ unsigned int pitch,
+ u32 old_offset,
+ u32 new_offset)
+{
+ old_offset += *y * pitch + *x * cpp;
+
+ *y = (old_offset - new_offset) / pitch;
+ *x = ((old_offset - new_offset) - *y * pitch) / cpp;
+
+ return new_offset;
+}
+
static u32 intel_adjust_aligned_offset(int *x, int *y,
const struct drm_framebuffer *fb,
int color_plane,
@@ -439,10 +874,8 @@ static u32 intel_adjust_aligned_offset(int *x, int *y,
tile_size, pitch_tiles,
old_offset, new_offset);
} else {
- old_offset += *y * pitch + *x * cpp;
-
- *y = (old_offset - new_offset) / pitch;
- *x = ((old_offset - new_offset) - *y * pitch) / cpp;
+ intel_adjust_linear_offset(x, y, cpp, pitch,
+ old_offset, new_offset);
}
return new_offset;
@@ -459,7 +892,7 @@ u32 intel_plane_adjust_aligned_offset(int *x, int *y,
{
return intel_adjust_aligned_offset(x, y, state->hw.fb, color_plane,
state->hw.rotation,
- state->view.color_plane[color_plane].stride,
+ state->view.color_plane[color_plane].mapping_stride,
old_offset, new_offset);
}
@@ -540,7 +973,7 @@ u32 intel_plane_compute_aligned_offset(int *x, int *y,
struct drm_i915_private *i915 = to_i915(intel_plane->base.dev);
const struct drm_framebuffer *fb = state->hw.fb;
unsigned int rotation = state->hw.rotation;
- int pitch = state->view.color_plane[color_plane].stride;
+ int pitch = state->view.color_plane[color_plane].mapping_stride;
u32 alignment;
if (intel_plane->id == PLANE_CURSOR)
@@ -562,6 +995,7 @@ static int intel_fb_offset_to_xy(int *x, int *y,
u32 alignment;
if (DISPLAY_VER(i915) >= 12 &&
+ !intel_fb_needs_pot_stride_remap(to_intel_framebuffer(fb)) &&
is_semiplanar_uv_plane(fb, color_plane))
alignment = intel_tile_row_size(fb, color_plane);
else if (fb->modifier != DRM_FORMAT_MOD_LINEAR)
@@ -610,7 +1044,7 @@ static int intel_fb_check_ccs_xy(const struct drm_framebuffer *fb, int ccs_plane
int ccs_x, ccs_y;
int main_x, main_y;
- if (!is_ccs_plane(fb, ccs_plane) || is_gen12_ccs_cc_plane(fb, ccs_plane))
+ if (!intel_fb_is_ccs_aux_plane(fb, ccs_plane))
return 0;
/*
@@ -673,7 +1107,7 @@ static bool intel_plane_can_remap(const struct intel_plane_state *plane_state)
* The new CCS hash mode isn't compatible with remapping as
* the virtual address of the pages affects the compressed data.
*/
- if (is_ccs_modifier(fb->modifier))
+ if (intel_fb_is_ccs_modifier(fb->modifier))
return false;
/* Linear needs a page aligned stride for remapping */
@@ -699,11 +1133,11 @@ bool intel_fb_needs_pot_stride_remap(const struct intel_framebuffer *fb)
static int intel_fb_pitch(const struct intel_framebuffer *fb, int color_plane, unsigned int rotation)
{
if (drm_rotation_90_or_270(rotation))
- return fb->rotated_view.color_plane[color_plane].stride;
+ return fb->rotated_view.color_plane[color_plane].mapping_stride;
else if (intel_fb_needs_pot_stride_remap(fb))
- return fb->remapped_view.color_plane[color_plane].stride;
+ return fb->remapped_view.color_plane[color_plane].mapping_stride;
else
- return fb->normal_view.color_plane[color_plane].stride;
+ return fb->normal_view.color_plane[color_plane].mapping_stride;
}
static bool intel_plane_needs_remap(const struct intel_plane_state *plane_state)
@@ -814,18 +1248,32 @@ plane_view_dst_stride_tiles(const struct intel_framebuffer *fb, int color_plane,
unsigned int pitch_tiles)
{
if (intel_fb_needs_pot_stride_remap(fb)) {
- unsigned int min_stride = is_ccs_plane(&fb->base, color_plane) ? 2 : 8;
/*
* ADL_P, the only platform needing a POT stride has a minimum
- * of 8 main surface and 2 CCS AUX stride tiles.
+ * of 8 main surface tiles.
*/
- return roundup_pow_of_two(max(pitch_tiles, min_stride));
+ return roundup_pow_of_two(max(pitch_tiles, 8u));
} else {
return pitch_tiles;
}
}
static unsigned int
+plane_view_scanout_stride(const struct intel_framebuffer *fb, int color_plane,
+ unsigned int tile_width,
+ unsigned int src_stride_tiles, unsigned int dst_stride_tiles)
+{
+ unsigned int stride_tiles;
+
+ if (IS_ALDERLAKE_P(to_i915(fb->base.dev)))
+ stride_tiles = src_stride_tiles;
+ else
+ stride_tiles = dst_stride_tiles;
+
+ return stride_tiles * tile_width * fb->base.format->cpp[color_plane];
+}
+
+static unsigned int
plane_view_width_tiles(const struct intel_framebuffer *fb, int color_plane,
const struct fb_plane_view_dims *dims,
int x)
@@ -841,11 +1289,31 @@ plane_view_height_tiles(const struct intel_framebuffer *fb, int color_plane,
return DIV_ROUND_UP(y + dims->height, dims->tile_height);
}
+static unsigned int
+plane_view_linear_tiles(const struct intel_framebuffer *fb, int color_plane,
+ const struct fb_plane_view_dims *dims,
+ int x, int y)
+{
+ struct drm_i915_private *i915 = to_i915(fb->base.dev);
+ unsigned int size;
+
+ size = (y + dims->height) * fb->base.pitches[color_plane] +
+ x * fb->base.format->cpp[color_plane];
+
+ return DIV_ROUND_UP(size, intel_tile_size(i915));
+}
+
#define assign_chk_ovf(i915, var, val) ({ \
drm_WARN_ON(&(i915)->drm, overflows_type(val, var)); \
(var) = (val); \
})
+#define assign_bfld_chk_ovf(i915, var, val) ({ \
+ (var) = (val); \
+ drm_WARN_ON(&(i915)->drm, (var) != (val)); \
+ (var); \
+})
+
static u32 calc_plane_remap_info(const struct intel_framebuffer *fb, int color_plane,
const struct fb_plane_view_dims *dims,
u32 obj_offset, u32 gtt_offset, int x, int y,
@@ -860,12 +1328,26 @@ static u32 calc_plane_remap_info(const struct intel_framebuffer *fb, int color_p
struct drm_rect r;
u32 size = 0;
- assign_chk_ovf(i915, remap_info->offset, obj_offset);
- assign_chk_ovf(i915, remap_info->src_stride, plane_view_src_stride_tiles(fb, color_plane, dims));
- assign_chk_ovf(i915, remap_info->width, plane_view_width_tiles(fb, color_plane, dims, x));
- assign_chk_ovf(i915, remap_info->height, plane_view_height_tiles(fb, color_plane, dims, y));
+ assign_bfld_chk_ovf(i915, remap_info->offset, obj_offset);
+
+ if (intel_fb_is_gen12_ccs_aux_plane(&fb->base, color_plane)) {
+ remap_info->linear = 1;
+
+ assign_chk_ovf(i915, remap_info->size,
+ plane_view_linear_tiles(fb, color_plane, dims, x, y));
+ } else {
+ remap_info->linear = 0;
+
+ assign_chk_ovf(i915, remap_info->src_stride,
+ plane_view_src_stride_tiles(fb, color_plane, dims));
+ assign_chk_ovf(i915, remap_info->width,
+ plane_view_width_tiles(fb, color_plane, dims, x));
+ assign_chk_ovf(i915, remap_info->height,
+ plane_view_height_tiles(fb, color_plane, dims, y));
+ }
if (view->gtt.type == I915_GGTT_VIEW_ROTATED) {
+ drm_WARN_ON(&i915->drm, remap_info->linear);
check_array_bounds(i915, view->gtt.rotated.plane, color_plane);
assign_chk_ovf(i915, remap_info->dst_stride,
@@ -881,7 +1363,8 @@ static u32 calc_plane_remap_info(const struct intel_framebuffer *fb, int color_p
color_plane_info->x = r.x1;
color_plane_info->y = r.y1;
- color_plane_info->stride = remap_info->dst_stride * tile_height;
+ color_plane_info->mapping_stride = remap_info->dst_stride * tile_height;
+ color_plane_info->scanout_stride = color_plane_info->mapping_stride;
size += remap_info->dst_stride * remap_info->width;
@@ -900,16 +1383,29 @@ static u32 calc_plane_remap_info(const struct intel_framebuffer *fb, int color_p
gtt_offset = aligned_offset;
}
- assign_chk_ovf(i915, remap_info->dst_stride,
- plane_view_dst_stride_tiles(fb, color_plane, remap_info->width));
-
color_plane_info->x = x;
color_plane_info->y = y;
- color_plane_info->stride = remap_info->dst_stride * tile_width *
- fb->base.format->cpp[color_plane];
+ if (remap_info->linear) {
+ color_plane_info->mapping_stride = fb->base.pitches[color_plane];
+ color_plane_info->scanout_stride = color_plane_info->mapping_stride;
- size += remap_info->dst_stride * remap_info->height;
+ size += remap_info->size;
+ } else {
+ unsigned int dst_stride = plane_view_dst_stride_tiles(fb, color_plane,
+ remap_info->width);
+
+ assign_chk_ovf(i915, remap_info->dst_stride, dst_stride);
+ color_plane_info->mapping_stride = dst_stride *
+ tile_width *
+ fb->base.format->cpp[color_plane];
+ color_plane_info->scanout_stride =
+ plane_view_scanout_stride(fb, color_plane, tile_width,
+ remap_info->src_stride,
+ dst_stride);
+
+ size += dst_stride * remap_info->height;
+ }
}
/*
@@ -917,10 +1413,16 @@ static u32 calc_plane_remap_info(const struct intel_framebuffer *fb, int color_p
* the x/y offsets. x,y will hold the first pixel of the framebuffer
* plane from the start of the remapped/rotated gtt mapping.
*/
- intel_adjust_tile_offset(&color_plane_info->x, &color_plane_info->y,
- tile_width, tile_height,
- tile_size, remap_info->dst_stride,
- gtt_offset * tile_size, 0);
+ if (remap_info->linear)
+ intel_adjust_linear_offset(&color_plane_info->x, &color_plane_info->y,
+ fb->base.format->cpp[color_plane],
+ color_plane_info->mapping_stride,
+ gtt_offset * tile_size, 0);
+ else
+ intel_adjust_tile_offset(&color_plane_info->x, &color_plane_info->y,
+ tile_width, tile_height,
+ tile_size, remap_info->dst_stride,
+ gtt_offset * tile_size, 0);
return size;
}
@@ -933,15 +1435,10 @@ calc_plane_normal_size(const struct intel_framebuffer *fb, int color_plane,
const struct fb_plane_view_dims *dims,
int x, int y)
{
- struct drm_i915_private *i915 = to_i915(fb->base.dev);
unsigned int tiles;
if (is_surface_linear(&fb->base, color_plane)) {
- unsigned int size;
-
- size = (y + dims->height) * fb->base.pitches[color_plane] +
- x * fb->base.format->cpp[color_plane];
- tiles = DIV_ROUND_UP(size, intel_tile_size(i915));
+ tiles = plane_view_linear_tiles(fb, color_plane, dims, x, y);
} else {
tiles = plane_view_src_stride_tiles(fb, color_plane, dims) *
plane_view_height_tiles(fb, color_plane, dims, y);
@@ -1030,7 +1527,9 @@ int intel_fill_fb_info(struct drm_i915_private *i915, struct intel_framebuffer *
*/
fb->normal_view.color_plane[i].x = x;
fb->normal_view.color_plane[i].y = y;
- fb->normal_view.color_plane[i].stride = fb->base.pitches[i];
+ fb->normal_view.color_plane[i].mapping_stride = fb->base.pitches[i];
+ fb->normal_view.color_plane[i].scanout_stride =
+ fb->normal_view.color_plane[i].mapping_stride;
offset = calc_plane_aligned_offset(fb, i, &x, &y);
@@ -1080,7 +1579,7 @@ static void intel_plane_remap_gtt(struct intel_plane_state *plane_state)
src_w = drm_rect_width(&plane_state->uapi.src) >> 16;
src_h = drm_rect_height(&plane_state->uapi.src) >> 16;
- drm_WARN_ON(&i915->drm, is_ccs_modifier(fb->modifier));
+ drm_WARN_ON(&i915->drm, intel_fb_is_ccs_modifier(fb->modifier));
/* Make src coordinates relative to the viewport */
drm_rect_translate(&plane_state->uapi.src,
@@ -1143,7 +1642,7 @@ u32 intel_fb_max_stride(struct drm_i915_private *dev_priv,
*
* The new CCS hash mode makes remapping impossible
*/
- if (DISPLAY_VER(dev_priv) < 4 || is_ccs_modifier(modifier) ||
+ if (DISPLAY_VER(dev_priv) < 4 || intel_fb_is_ccs_modifier(modifier) ||
intel_modifier_uses_dpt(dev_priv, modifier))
return intel_plane_fb_max_stride(dev_priv, pixel_format, modifier);
else if (DISPLAY_VER(dev_priv) >= 7)
@@ -1168,27 +1667,19 @@ intel_fb_stride_alignment(const struct drm_framebuffer *fb, int color_plane)
* we need the stride to be page aligned.
*/
if (fb->pitches[color_plane] > max_stride &&
- !is_ccs_modifier(fb->modifier))
+ !intel_fb_is_ccs_modifier(fb->modifier))
return intel_tile_size(dev_priv);
else
return 64;
}
tile_width = intel_tile_width_bytes(fb, color_plane);
- if (is_ccs_modifier(fb->modifier)) {
- /*
- * On ADL-P the stride must be either 8 tiles or a stride
- * that is aligned to 16 tiles, required by the 16 tiles =
- * 64 kbyte CCS AUX PTE granularity, allowing CCS FBs to be
- * remapped.
- */
- if (IS_ALDERLAKE_P(dev_priv))
- tile_width *= fb->pitches[0] <= tile_width * 8 ? 8 : 16;
+ if (intel_fb_is_ccs_modifier(fb->modifier)) {
/*
* On TGL the surface stride must be 4 tile aligned, mapped by
* one 64 byte cacheline on the CCS AUX surface.
*/
- else if (DISPLAY_VER(dev_priv) >= 12)
+ if (DISPLAY_VER(dev_priv) >= 12)
tile_width *= 4;
/*
* Display WA #0531: skl,bxt,kbl,glk
@@ -1224,7 +1715,7 @@ static int intel_plane_check_stride(const struct intel_plane_state *plane_state)
return 0;
/* FIXME other color planes? */
- stride = plane_state->view.color_plane[0].stride;
+ stride = plane_state->view.color_plane[0].mapping_stride;
max_stride = plane->max_stride(plane, fb->format->format,
fb->modifier, rotation);
@@ -1430,7 +1921,7 @@ int intel_framebuffer_init(struct intel_framebuffer *intel_fb,
goto err;
}
- if (is_gen12_ccs_plane(fb, i) && !is_gen12_ccs_cc_plane(fb, i)) {
+ if (intel_fb_is_gen12_ccs_aux_plane(fb, i)) {
int ccs_aux_stride = gen12_ccs_aux_stride(intel_fb, i);
if (fb->pitches[i] != ccs_aux_stride) {
diff --git a/drivers/gpu/drm/i915/display/intel_fb.h b/drivers/gpu/drm/i915/display/intel_fb.h
index 1cbdd84..ba9df89 100644
--- a/drivers/gpu/drm/i915/display/intel_fb.h
+++ b/drivers/gpu/drm/i915/display/intel_fb.h
@@ -6,6 +6,7 @@
#ifndef __INTEL_FB_H__
#define __INTEL_FB_H__
+#include <linux/bits.h>
#include <linux/types.h>
struct drm_device;
@@ -16,12 +17,34 @@ struct drm_i915_private;
struct drm_mode_fb_cmd2;
struct intel_fb_view;
struct intel_framebuffer;
+struct intel_plane;
struct intel_plane_state;
-bool is_ccs_plane(const struct drm_framebuffer *fb, int plane);
-bool is_gen12_ccs_plane(const struct drm_framebuffer *fb, int plane);
-bool is_gen12_ccs_cc_plane(const struct drm_framebuffer *fb, int plane);
-bool is_semiplanar_uv_plane(const struct drm_framebuffer *fb, int color_plane);
+#define INTEL_PLANE_CAP_NONE 0
+#define INTEL_PLANE_CAP_CCS_RC BIT(0)
+#define INTEL_PLANE_CAP_CCS_RC_CC BIT(1)
+#define INTEL_PLANE_CAP_CCS_MC BIT(2)
+#define INTEL_PLANE_CAP_TILING_X BIT(3)
+#define INTEL_PLANE_CAP_TILING_Y BIT(4)
+#define INTEL_PLANE_CAP_TILING_Yf BIT(5)
+
+bool intel_fb_is_ccs_modifier(u64 modifier);
+bool intel_fb_is_rc_ccs_cc_modifier(u64 modifier);
+bool intel_fb_is_mc_ccs_modifier(u64 modifier);
+
+bool intel_fb_is_ccs_aux_plane(const struct drm_framebuffer *fb, int color_plane);
+int intel_fb_rc_ccs_cc_plane(const struct drm_framebuffer *fb);
+
+u64 *intel_fb_plane_get_modifiers(struct drm_i915_private *i915,
+ u8 plane_caps);
+bool intel_fb_plane_supports_modifier(struct intel_plane *plane, u64 modifier);
+
+const struct drm_format_info *
+intel_fb_get_format_info(const struct drm_mode_fb_cmd2 *cmd);
+
+bool
+intel_format_info_is_yuv_semiplanar(const struct drm_format_info *info,
+ u64 modifier);
bool is_surface_linear(const struct drm_framebuffer *fb, int color_plane);
@@ -67,4 +90,6 @@ intel_user_framebuffer_create(struct drm_device *dev,
struct drm_file *filp,
const struct drm_mode_fb_cmd2 *user_mode_cmd);
+bool intel_fb_uses_dpt(const struct drm_framebuffer *fb);
+
#endif /* __INTEL_FB_H__ */
diff --git a/drivers/gpu/drm/i915/display/intel_fb_pin.c b/drivers/gpu/drm/i915/display/intel_fb_pin.c
index 3f77f30..31c15e5 100644
--- a/drivers/gpu/drm/i915/display/intel_fb_pin.c
+++ b/drivers/gpu/drm/i915/display/intel_fb_pin.c
@@ -7,14 +7,14 @@
* DOC: display pinning helpers
*/
-#include "intel_display_types.h"
-#include "intel_fb_pin.h"
-#include "intel_fb.h"
-
-#include "intel_dpt.h"
-
#include "gem/i915_gem_object.h"
+#include "i915_drv.h"
+#include "intel_display_types.h"
+#include "intel_dpt.h"
+#include "intel_fb.h"
+#include "intel_fb_pin.h"
+
static struct i915_vma *
intel_pin_fb_obj_dpt(struct drm_framebuffer *fb,
const struct i915_ggtt_view *view,
@@ -142,13 +142,11 @@ intel_pin_and_fence_fb_obj(struct drm_framebuffer *fb,
if (ret)
goto err;
- if (!ret) {
- vma = i915_gem_object_pin_to_display_plane(obj, &ww, alignment,
- view, pinctl);
- if (IS_ERR(vma)) {
- ret = PTR_ERR(vma);
- goto err_unpin;
- }
+ vma = i915_gem_object_pin_to_display_plane(obj, &ww, alignment,
+ view, pinctl);
+ if (IS_ERR(vma)) {
+ ret = PTR_ERR(vma);
+ goto err_unpin;
}
if (uses_fence && i915_vma_is_map_and_fenceable(vma)) {
diff --git a/drivers/gpu/drm/i915/display/intel_fbc.c b/drivers/gpu/drm/i915/display/intel_fbc.c
index 1f66de7..160fd2b 100644
--- a/drivers/gpu/drm/i915/display/intel_fbc.c
+++ b/drivers/gpu/drm/i915/display/intel_fbc.c
@@ -41,26 +41,71 @@
#include <drm/drm_fourcc.h>
#include "i915_drv.h"
-#include "i915_trace.h"
#include "i915_vgpu.h"
+#include "intel_cdclk.h"
#include "intel_de.h"
+#include "intel_display_trace.h"
#include "intel_display_types.h"
#include "intel_fbc.h"
#include "intel_frontbuffer.h"
-/*
- * For SKL+, the plane source size used by the hardware is based on the value we
- * write to the PLANE_SIZE register. For BDW-, the hardware looks at the value
- * we wrote to PIPESRC.
- */
-static void intel_fbc_get_plane_source_size(const struct intel_fbc_state_cache *cache,
- int *width, int *height)
-{
- if (width)
- *width = cache->plane.src_w;
- if (height)
- *height = cache->plane.src_h;
-}
+struct intel_fbc_funcs {
+ void (*activate)(struct intel_fbc *fbc);
+ void (*deactivate)(struct intel_fbc *fbc);
+ bool (*is_active)(struct intel_fbc *fbc);
+ bool (*is_compressing)(struct intel_fbc *fbc);
+ void (*nuke)(struct intel_fbc *fbc);
+ void (*program_cfb)(struct intel_fbc *fbc);
+ void (*set_false_color)(struct intel_fbc *fbc, bool enable);
+};
+
+struct intel_fbc_state {
+ struct intel_plane *plane;
+ unsigned int cfb_stride;
+ unsigned int cfb_size;
+ unsigned int fence_y_offset;
+ u16 override_cfb_stride;
+ u16 interval;
+ s8 fence_id;
+};
+
+struct intel_fbc {
+ struct drm_i915_private *i915;
+ const struct intel_fbc_funcs *funcs;
+
+ /*
+ * This is always the inner lock when overlapping with
+ * struct_mutex and it's the outer lock when overlapping
+ * with stolen_lock.
+ */
+ struct mutex lock;
+ unsigned int possible_framebuffer_bits;
+ unsigned int busy_bits;
+
+ struct drm_mm_node compressed_fb;
+ struct drm_mm_node compressed_llb;
+
+ u8 limit;
+
+ bool false_color;
+
+ bool active;
+ bool activated;
+ bool flip_pending;
+
+ bool underrun_detected;
+ struct work_struct underrun_work;
+
+ /*
+ * This structure contains everything that's relevant to program the
+ * hardware registers. When we want to figure out if we need to disable
+ * and re-enable FBC for a new configuration we just check if there's
+ * something different in the struct. The genx_fbc_activate functions
+ * are supposed to read from it in order to program the registers.
+ */
+ struct intel_fbc_state state;
+ const char *no_fbc_reason;
+};
/* plane stride in pixels */
static unsigned int intel_fbc_plane_stride(const struct intel_plane_state *plane_state)
@@ -68,7 +113,7 @@ static unsigned int intel_fbc_plane_stride(const struct intel_plane_state *plane
const struct drm_framebuffer *fb = plane_state->hw.fb;
unsigned int stride;
- stride = plane_state->view.color_plane[0].stride;
+ stride = plane_state->view.color_plane[0].mapping_stride;
if (!drm_rotation_90_or_270(plane_state->hw.rotation))
stride /= fb->format->cpp[0];
@@ -76,24 +121,25 @@ static unsigned int intel_fbc_plane_stride(const struct intel_plane_state *plane
}
/* plane stride based cfb stride in bytes, assuming 1:1 compression limit */
-static unsigned int _intel_fbc_cfb_stride(const struct intel_fbc_state_cache *cache)
+static unsigned int _intel_fbc_cfb_stride(const struct intel_plane_state *plane_state)
{
unsigned int cpp = 4; /* FBC always 4 bytes per pixel */
- return cache->fb.stride * cpp;
+ return intel_fbc_plane_stride(plane_state) * cpp;
}
/* minimum acceptable cfb stride in bytes, assuming 1:1 compression limit */
-static unsigned int skl_fbc_min_cfb_stride(struct drm_i915_private *i915,
- const struct intel_fbc_state_cache *cache)
+static unsigned int skl_fbc_min_cfb_stride(const struct intel_plane_state *plane_state)
{
+ struct drm_i915_private *i915 = to_i915(plane_state->uapi.plane->dev);
unsigned int limit = 4; /* 1:4 compression limit is the worst case */
unsigned int cpp = 4; /* FBC always 4 bytes per pixel */
+ unsigned int width = drm_rect_width(&plane_state->uapi.src) >> 16;
unsigned int height = 4; /* FBC segment is 4 lines */
unsigned int stride;
/* minimum segment stride we can use */
- stride = cache->plane.src_w * cpp * height / limit;
+ stride = width * cpp * height / limit;
/*
* Wa_16011863758: icl+
@@ -113,10 +159,10 @@ static unsigned int skl_fbc_min_cfb_stride(struct drm_i915_private *i915,
}
/* properly aligned cfb stride in bytes, assuming 1:1 compression limit */
-static unsigned int intel_fbc_cfb_stride(struct drm_i915_private *i915,
- const struct intel_fbc_state_cache *cache)
+static unsigned int intel_fbc_cfb_stride(const struct intel_plane_state *plane_state)
{
- unsigned int stride = _intel_fbc_cfb_stride(cache);
+ struct drm_i915_private *i915 = to_i915(plane_state->uapi.plane->dev);
+ unsigned int stride = _intel_fbc_cfb_stride(plane_state);
/*
* At least some of the platforms require each 4 line segment to
@@ -124,98 +170,202 @@ static unsigned int intel_fbc_cfb_stride(struct drm_i915_private *i915,
* that regardless of the compression limit we choose later.
*/
if (DISPLAY_VER(i915) >= 9)
- return max(ALIGN(stride, 512), skl_fbc_min_cfb_stride(i915, cache));
+ return max(ALIGN(stride, 512), skl_fbc_min_cfb_stride(plane_state));
else
return stride;
}
-static unsigned int intel_fbc_cfb_size(struct drm_i915_private *dev_priv,
- const struct intel_fbc_state_cache *cache)
+static unsigned int intel_fbc_cfb_size(const struct intel_plane_state *plane_state)
{
- int lines = cache->plane.src_h;
+ struct drm_i915_private *i915 = to_i915(plane_state->uapi.plane->dev);
+ int lines = drm_rect_height(&plane_state->uapi.src) >> 16;
- if (DISPLAY_VER(dev_priv) == 7)
+ if (DISPLAY_VER(i915) == 7)
lines = min(lines, 2048);
- else if (DISPLAY_VER(dev_priv) >= 8)
+ else if (DISPLAY_VER(i915) >= 8)
lines = min(lines, 2560);
- return lines * intel_fbc_cfb_stride(dev_priv, cache);
+ return lines * intel_fbc_cfb_stride(plane_state);
}
-static void i8xx_fbc_deactivate(struct drm_i915_private *dev_priv)
+static u16 intel_fbc_override_cfb_stride(const struct intel_plane_state *plane_state)
{
+ struct drm_i915_private *i915 = to_i915(plane_state->uapi.plane->dev);
+ unsigned int stride_aligned = intel_fbc_cfb_stride(plane_state);
+ unsigned int stride = _intel_fbc_cfb_stride(plane_state);
+ const struct drm_framebuffer *fb = plane_state->hw.fb;
+
+ /*
+ * Override stride in 64 byte units per 4 line segment.
+ *
+ * Gen9 hw miscalculates cfb stride for linear as
+ * PLANE_STRIDE*512 instead of PLANE_STRIDE*64, so
+ * we always need to use the override there.
+ */
+ if (stride != stride_aligned ||
+ (DISPLAY_VER(i915) == 9 && fb->modifier == DRM_FORMAT_MOD_LINEAR))
+ return stride_aligned * 4 / 64;
+
+ return 0;
+}
+
+static u32 i8xx_fbc_ctl(struct intel_fbc *fbc)
+{
+ const struct intel_fbc_state *fbc_state = &fbc->state;
+ struct drm_i915_private *i915 = fbc->i915;
+ unsigned int cfb_stride;
+ u32 fbc_ctl;
+
+ cfb_stride = fbc_state->cfb_stride / fbc->limit;
+
+ /* FBC_CTL wants 32B or 64B units */
+ if (DISPLAY_VER(i915) == 2)
+ cfb_stride = (cfb_stride / 32) - 1;
+ else
+ cfb_stride = (cfb_stride / 64) - 1;
+
+ fbc_ctl = FBC_CTL_PERIODIC |
+ FBC_CTL_INTERVAL(fbc_state->interval) |
+ FBC_CTL_STRIDE(cfb_stride);
+
+ if (IS_I945GM(i915))
+ fbc_ctl |= FBC_CTL_C3_IDLE; /* 945 needs special SR handling */
+
+ if (fbc_state->fence_id >= 0)
+ fbc_ctl |= FBC_CTL_FENCENO(fbc_state->fence_id);
+
+ return fbc_ctl;
+}
+
+static u32 i965_fbc_ctl2(struct intel_fbc *fbc)
+{
+ const struct intel_fbc_state *fbc_state = &fbc->state;
+ u32 fbc_ctl2;
+
+ fbc_ctl2 = FBC_CTL_FENCE_DBL | FBC_CTL_IDLE_IMM |
+ FBC_CTL_PLANE(fbc_state->plane->i9xx_plane);
+
+ if (fbc_state->fence_id >= 0)
+ fbc_ctl2 |= FBC_CTL_CPU_FENCE_EN;
+
+ return fbc_ctl2;
+}
+
+static void i8xx_fbc_deactivate(struct intel_fbc *fbc)
+{
+ struct drm_i915_private *i915 = fbc->i915;
u32 fbc_ctl;
/* Disable compression */
- fbc_ctl = intel_de_read(dev_priv, FBC_CONTROL);
+ fbc_ctl = intel_de_read(i915, FBC_CONTROL);
if ((fbc_ctl & FBC_CTL_EN) == 0)
return;
fbc_ctl &= ~FBC_CTL_EN;
- intel_de_write(dev_priv, FBC_CONTROL, fbc_ctl);
+ intel_de_write(i915, FBC_CONTROL, fbc_ctl);
/* Wait for compressing bit to clear */
- if (intel_de_wait_for_clear(dev_priv, FBC_STATUS,
+ if (intel_de_wait_for_clear(i915, FBC_STATUS,
FBC_STAT_COMPRESSING, 10)) {
- drm_dbg_kms(&dev_priv->drm, "FBC idle timed out\n");
+ drm_dbg_kms(&i915->drm, "FBC idle timed out\n");
return;
}
}
-static void i8xx_fbc_activate(struct drm_i915_private *dev_priv)
+static void i8xx_fbc_activate(struct intel_fbc *fbc)
{
- struct intel_fbc *fbc = &dev_priv->fbc;
- const struct intel_fbc_reg_params *params = &fbc->params;
- int cfb_pitch;
+ const struct intel_fbc_state *fbc_state = &fbc->state;
+ struct drm_i915_private *i915 = fbc->i915;
int i;
- u32 fbc_ctl;
-
- cfb_pitch = params->cfb_stride / fbc->limit;
-
- /* FBC_CTL wants 32B or 64B units */
- if (DISPLAY_VER(dev_priv) == 2)
- cfb_pitch = (cfb_pitch / 32) - 1;
- else
- cfb_pitch = (cfb_pitch / 64) - 1;
/* Clear old tags */
for (i = 0; i < (FBC_LL_SIZE / 32) + 1; i++)
- intel_de_write(dev_priv, FBC_TAG(i), 0);
+ intel_de_write(i915, FBC_TAG(i), 0);
- if (DISPLAY_VER(dev_priv) == 4) {
- u32 fbc_ctl2;
-
- /* Set it up... */
- fbc_ctl2 = FBC_CTL_FENCE_DBL | FBC_CTL_IDLE_IMM;
- fbc_ctl2 |= FBC_CTL_PLANE(params->crtc.i9xx_plane);
- if (params->fence_id >= 0)
- fbc_ctl2 |= FBC_CTL_CPU_FENCE;
- intel_de_write(dev_priv, FBC_CONTROL2, fbc_ctl2);
- intel_de_write(dev_priv, FBC_FENCE_OFF,
- params->fence_y_offset);
+ if (DISPLAY_VER(i915) == 4) {
+ intel_de_write(i915, FBC_CONTROL2,
+ i965_fbc_ctl2(fbc));
+ intel_de_write(i915, FBC_FENCE_OFF,
+ fbc_state->fence_y_offset);
}
- /* enable it... */
- fbc_ctl = FBC_CTL_INTERVAL(params->interval);
- fbc_ctl |= FBC_CTL_EN | FBC_CTL_PERIODIC;
- if (IS_I945GM(dev_priv))
- fbc_ctl |= FBC_CTL_C3_IDLE; /* 945 needs special SR handling */
- fbc_ctl |= FBC_CTL_STRIDE(cfb_pitch & 0xff);
- if (params->fence_id >= 0)
- fbc_ctl |= FBC_CTL_FENCENO(params->fence_id);
- intel_de_write(dev_priv, FBC_CONTROL, fbc_ctl);
+ intel_de_write(i915, FBC_CONTROL,
+ FBC_CTL_EN | i8xx_fbc_ctl(fbc));
}
-static bool i8xx_fbc_is_active(struct drm_i915_private *dev_priv)
+static bool i8xx_fbc_is_active(struct intel_fbc *fbc)
{
- return intel_de_read(dev_priv, FBC_CONTROL) & FBC_CTL_EN;
+ return intel_de_read(fbc->i915, FBC_CONTROL) & FBC_CTL_EN;
}
-static u32 g4x_dpfc_ctl_limit(struct drm_i915_private *i915)
+static bool i8xx_fbc_is_compressing(struct intel_fbc *fbc)
{
- switch (i915->fbc.limit) {
+ return intel_de_read(fbc->i915, FBC_STATUS) &
+ (FBC_STAT_COMPRESSING | FBC_STAT_COMPRESSED);
+}
+
+static void i8xx_fbc_nuke(struct intel_fbc *fbc)
+{
+ struct intel_fbc_state *fbc_state = &fbc->state;
+ enum i9xx_plane_id i9xx_plane = fbc_state->plane->i9xx_plane;
+ struct drm_i915_private *dev_priv = fbc->i915;
+
+ spin_lock_irq(&dev_priv->uncore.lock);
+ intel_de_write_fw(dev_priv, DSPADDR(i9xx_plane),
+ intel_de_read_fw(dev_priv, DSPADDR(i9xx_plane)));
+ spin_unlock_irq(&dev_priv->uncore.lock);
+}
+
+static void i8xx_fbc_program_cfb(struct intel_fbc *fbc)
+{
+ struct drm_i915_private *i915 = fbc->i915;
+
+ GEM_BUG_ON(range_overflows_end_t(u64, i915->dsm.start,
+ fbc->compressed_fb.start, U32_MAX));
+ GEM_BUG_ON(range_overflows_end_t(u64, i915->dsm.start,
+ fbc->compressed_llb.start, U32_MAX));
+
+ intel_de_write(i915, FBC_CFB_BASE,
+ i915->dsm.start + fbc->compressed_fb.start);
+ intel_de_write(i915, FBC_LL_BASE,
+ i915->dsm.start + fbc->compressed_llb.start);
+}
+
+static const struct intel_fbc_funcs i8xx_fbc_funcs = {
+ .activate = i8xx_fbc_activate,
+ .deactivate = i8xx_fbc_deactivate,
+ .is_active = i8xx_fbc_is_active,
+ .is_compressing = i8xx_fbc_is_compressing,
+ .nuke = i8xx_fbc_nuke,
+ .program_cfb = i8xx_fbc_program_cfb,
+};
+
+static void i965_fbc_nuke(struct intel_fbc *fbc)
+{
+ struct intel_fbc_state *fbc_state = &fbc->state;
+ enum i9xx_plane_id i9xx_plane = fbc_state->plane->i9xx_plane;
+ struct drm_i915_private *dev_priv = fbc->i915;
+
+ spin_lock_irq(&dev_priv->uncore.lock);
+ intel_de_write_fw(dev_priv, DSPSURF(i9xx_plane),
+ intel_de_read_fw(dev_priv, DSPSURF(i9xx_plane)));
+ spin_unlock_irq(&dev_priv->uncore.lock);
+}
+
+static const struct intel_fbc_funcs i965_fbc_funcs = {
+ .activate = i8xx_fbc_activate,
+ .deactivate = i8xx_fbc_deactivate,
+ .is_active = i8xx_fbc_is_active,
+ .is_compressing = i8xx_fbc_is_compressing,
+ .nuke = i965_fbc_nuke,
+ .program_cfb = i8xx_fbc_program_cfb,
+};
+
+static u32 g4x_dpfc_ctl_limit(struct intel_fbc *fbc)
+{
+ switch (fbc->limit) {
default:
- MISSING_CASE(i915->fbc.limit);
+ MISSING_CASE(fbc->limit);
fallthrough;
case 1:
return DPFC_CTL_LIMIT_1X;
@@ -226,260 +376,306 @@ static u32 g4x_dpfc_ctl_limit(struct drm_i915_private *i915)
}
}
-static void g4x_fbc_activate(struct drm_i915_private *dev_priv)
+static u32 g4x_dpfc_ctl(struct intel_fbc *fbc)
{
- struct intel_fbc_reg_params *params = &dev_priv->fbc.params;
+ const struct intel_fbc_state *fbc_state = &fbc->state;
+ struct drm_i915_private *i915 = fbc->i915;
u32 dpfc_ctl;
- dpfc_ctl = DPFC_CTL_PLANE(params->crtc.i9xx_plane) | DPFC_SR_EN;
+ dpfc_ctl = g4x_dpfc_ctl_limit(fbc) |
+ DPFC_CTL_PLANE_G4X(fbc_state->plane->i9xx_plane);
- dpfc_ctl |= g4x_dpfc_ctl_limit(dev_priv);
+ if (IS_G4X(i915))
+ dpfc_ctl |= DPFC_CTL_SR_EN;
- if (params->fence_id >= 0) {
- dpfc_ctl |= DPFC_CTL_FENCE_EN | params->fence_id;
- intel_de_write(dev_priv, DPFC_FENCE_YOFF,
- params->fence_y_offset);
- } else {
- intel_de_write(dev_priv, DPFC_FENCE_YOFF, 0);
+ if (fbc_state->fence_id >= 0) {
+ dpfc_ctl |= DPFC_CTL_FENCE_EN_G4X;
+
+ if (DISPLAY_VER(i915) < 6)
+ dpfc_ctl |= DPFC_CTL_FENCENO(fbc_state->fence_id);
}
- /* enable it... */
- intel_de_write(dev_priv, DPFC_CONTROL, dpfc_ctl | DPFC_CTL_EN);
+ return dpfc_ctl;
}
-static void g4x_fbc_deactivate(struct drm_i915_private *dev_priv)
+static void g4x_fbc_activate(struct intel_fbc *fbc)
{
+ const struct intel_fbc_state *fbc_state = &fbc->state;
+ struct drm_i915_private *i915 = fbc->i915;
+
+ intel_de_write(i915, DPFC_FENCE_YOFF,
+ fbc_state->fence_y_offset);
+
+ intel_de_write(i915, DPFC_CONTROL,
+ DPFC_CTL_EN | g4x_dpfc_ctl(fbc));
+}
+
+static void g4x_fbc_deactivate(struct intel_fbc *fbc)
+{
+ struct drm_i915_private *i915 = fbc->i915;
u32 dpfc_ctl;
/* Disable compression */
- dpfc_ctl = intel_de_read(dev_priv, DPFC_CONTROL);
+ dpfc_ctl = intel_de_read(i915, DPFC_CONTROL);
if (dpfc_ctl & DPFC_CTL_EN) {
dpfc_ctl &= ~DPFC_CTL_EN;
- intel_de_write(dev_priv, DPFC_CONTROL, dpfc_ctl);
+ intel_de_write(i915, DPFC_CONTROL, dpfc_ctl);
}
}
-static bool g4x_fbc_is_active(struct drm_i915_private *dev_priv)
+static bool g4x_fbc_is_active(struct intel_fbc *fbc)
{
- return intel_de_read(dev_priv, DPFC_CONTROL) & DPFC_CTL_EN;
+ return intel_de_read(fbc->i915, DPFC_CONTROL) & DPFC_CTL_EN;
}
-static void i8xx_fbc_recompress(struct drm_i915_private *dev_priv)
+static bool g4x_fbc_is_compressing(struct intel_fbc *fbc)
{
- struct intel_fbc_reg_params *params = &dev_priv->fbc.params;
- enum i9xx_plane_id i9xx_plane = params->crtc.i9xx_plane;
-
- spin_lock_irq(&dev_priv->uncore.lock);
- intel_de_write_fw(dev_priv, DSPADDR(i9xx_plane),
- intel_de_read_fw(dev_priv, DSPADDR(i9xx_plane)));
- spin_unlock_irq(&dev_priv->uncore.lock);
+ return intel_de_read(fbc->i915, DPFC_STATUS) & DPFC_COMP_SEG_MASK;
}
-static void i965_fbc_recompress(struct drm_i915_private *dev_priv)
+static void g4x_fbc_program_cfb(struct intel_fbc *fbc)
{
- struct intel_fbc_reg_params *params = &dev_priv->fbc.params;
- enum i9xx_plane_id i9xx_plane = params->crtc.i9xx_plane;
+ struct drm_i915_private *i915 = fbc->i915;
- spin_lock_irq(&dev_priv->uncore.lock);
- intel_de_write_fw(dev_priv, DSPSURF(i9xx_plane),
- intel_de_read_fw(dev_priv, DSPSURF(i9xx_plane)));
- spin_unlock_irq(&dev_priv->uncore.lock);
+ intel_de_write(i915, DPFC_CB_BASE, fbc->compressed_fb.start);
}
-/* This function forces a CFB recompression through the nuke operation. */
-static void snb_fbc_recompress(struct drm_i915_private *dev_priv)
+static const struct intel_fbc_funcs g4x_fbc_funcs = {
+ .activate = g4x_fbc_activate,
+ .deactivate = g4x_fbc_deactivate,
+ .is_active = g4x_fbc_is_active,
+ .is_compressing = g4x_fbc_is_compressing,
+ .nuke = i965_fbc_nuke,
+ .program_cfb = g4x_fbc_program_cfb,
+};
+
+static void ilk_fbc_activate(struct intel_fbc *fbc)
{
- intel_de_write(dev_priv, MSG_FBC_REND_STATE, FBC_REND_NUKE);
- intel_de_posting_read(dev_priv, MSG_FBC_REND_STATE);
+ struct intel_fbc_state *fbc_state = &fbc->state;
+ struct drm_i915_private *i915 = fbc->i915;
+
+ intel_de_write(i915, ILK_DPFC_FENCE_YOFF,
+ fbc_state->fence_y_offset);
+
+ intel_de_write(i915, ILK_DPFC_CONTROL,
+ DPFC_CTL_EN | g4x_dpfc_ctl(fbc));
}
-static void intel_fbc_recompress(struct drm_i915_private *dev_priv)
+static void ilk_fbc_deactivate(struct intel_fbc *fbc)
{
- struct intel_fbc *fbc = &dev_priv->fbc;
-
- trace_intel_fbc_nuke(fbc->crtc);
-
- if (DISPLAY_VER(dev_priv) >= 6)
- snb_fbc_recompress(dev_priv);
- else if (DISPLAY_VER(dev_priv) >= 4)
- i965_fbc_recompress(dev_priv);
- else
- i8xx_fbc_recompress(dev_priv);
-}
-
-static void ilk_fbc_activate(struct drm_i915_private *dev_priv)
-{
- struct intel_fbc_reg_params *params = &dev_priv->fbc.params;
- u32 dpfc_ctl;
-
- dpfc_ctl = DPFC_CTL_PLANE(params->crtc.i9xx_plane);
-
- dpfc_ctl |= g4x_dpfc_ctl_limit(dev_priv);
-
- if (params->fence_id >= 0) {
- dpfc_ctl |= DPFC_CTL_FENCE_EN;
- if (IS_IRONLAKE(dev_priv))
- dpfc_ctl |= params->fence_id;
- if (IS_SANDYBRIDGE(dev_priv)) {
- intel_de_write(dev_priv, SNB_DPFC_CTL_SA,
- SNB_CPU_FENCE_ENABLE | params->fence_id);
- intel_de_write(dev_priv, DPFC_CPU_FENCE_OFFSET,
- params->fence_y_offset);
- }
- } else {
- if (IS_SANDYBRIDGE(dev_priv)) {
- intel_de_write(dev_priv, SNB_DPFC_CTL_SA, 0);
- intel_de_write(dev_priv, DPFC_CPU_FENCE_OFFSET, 0);
- }
- }
-
- intel_de_write(dev_priv, ILK_DPFC_FENCE_YOFF,
- params->fence_y_offset);
- /* enable it... */
- intel_de_write(dev_priv, ILK_DPFC_CONTROL, dpfc_ctl | DPFC_CTL_EN);
-}
-
-static void ilk_fbc_deactivate(struct drm_i915_private *dev_priv)
-{
+ struct drm_i915_private *i915 = fbc->i915;
u32 dpfc_ctl;
/* Disable compression */
- dpfc_ctl = intel_de_read(dev_priv, ILK_DPFC_CONTROL);
+ dpfc_ctl = intel_de_read(i915, ILK_DPFC_CONTROL);
if (dpfc_ctl & DPFC_CTL_EN) {
dpfc_ctl &= ~DPFC_CTL_EN;
- intel_de_write(dev_priv, ILK_DPFC_CONTROL, dpfc_ctl);
+ intel_de_write(i915, ILK_DPFC_CONTROL, dpfc_ctl);
}
}
-static bool ilk_fbc_is_active(struct drm_i915_private *dev_priv)
+static bool ilk_fbc_is_active(struct intel_fbc *fbc)
{
- return intel_de_read(dev_priv, ILK_DPFC_CONTROL) & DPFC_CTL_EN;
+ return intel_de_read(fbc->i915, ILK_DPFC_CONTROL) & DPFC_CTL_EN;
}
-static void gen7_fbc_activate(struct drm_i915_private *dev_priv)
+static bool ilk_fbc_is_compressing(struct intel_fbc *fbc)
{
- struct intel_fbc *fbc = &dev_priv->fbc;
- const struct intel_fbc_reg_params *params = &fbc->params;
+ return intel_de_read(fbc->i915, ILK_DPFC_STATUS) & DPFC_COMP_SEG_MASK;
+}
+
+static void ilk_fbc_program_cfb(struct intel_fbc *fbc)
+{
+ struct drm_i915_private *i915 = fbc->i915;
+
+ intel_de_write(i915, ILK_DPFC_CB_BASE, fbc->compressed_fb.start);
+}
+
+static const struct intel_fbc_funcs ilk_fbc_funcs = {
+ .activate = ilk_fbc_activate,
+ .deactivate = ilk_fbc_deactivate,
+ .is_active = ilk_fbc_is_active,
+ .is_compressing = ilk_fbc_is_compressing,
+ .nuke = i965_fbc_nuke,
+ .program_cfb = ilk_fbc_program_cfb,
+};
+
+static void snb_fbc_program_fence(struct intel_fbc *fbc)
+{
+ const struct intel_fbc_state *fbc_state = &fbc->state;
+ struct drm_i915_private *i915 = fbc->i915;
+ u32 ctl = 0;
+
+ if (fbc_state->fence_id >= 0)
+ ctl = SNB_DPFC_FENCE_EN | SNB_DPFC_FENCENO(fbc_state->fence_id);
+
+ intel_de_write(i915, SNB_DPFC_CTL_SA, ctl);
+ intel_de_write(i915, SNB_DPFC_CPU_FENCE_OFFSET, fbc_state->fence_y_offset);
+}
+
+static void snb_fbc_activate(struct intel_fbc *fbc)
+{
+ snb_fbc_program_fence(fbc);
+
+ ilk_fbc_activate(fbc);
+}
+
+static void snb_fbc_nuke(struct intel_fbc *fbc)
+{
+ struct drm_i915_private *i915 = fbc->i915;
+
+ intel_de_write(i915, MSG_FBC_REND_STATE, FBC_REND_NUKE);
+ intel_de_posting_read(i915, MSG_FBC_REND_STATE);
+}
+
+static const struct intel_fbc_funcs snb_fbc_funcs = {
+ .activate = snb_fbc_activate,
+ .deactivate = ilk_fbc_deactivate,
+ .is_active = ilk_fbc_is_active,
+ .is_compressing = ilk_fbc_is_compressing,
+ .nuke = snb_fbc_nuke,
+ .program_cfb = ilk_fbc_program_cfb,
+};
+
+static void glk_fbc_program_cfb_stride(struct intel_fbc *fbc)
+{
+ const struct intel_fbc_state *fbc_state = &fbc->state;
+ struct drm_i915_private *i915 = fbc->i915;
+ u32 val = 0;
+
+ if (fbc_state->override_cfb_stride)
+ val |= FBC_STRIDE_OVERRIDE |
+ FBC_STRIDE(fbc_state->override_cfb_stride / fbc->limit);
+
+ intel_de_write(i915, GLK_FBC_STRIDE, val);
+}
+
+static void skl_fbc_program_cfb_stride(struct intel_fbc *fbc)
+{
+ const struct intel_fbc_state *fbc_state = &fbc->state;
+ struct drm_i915_private *i915 = fbc->i915;
+ u32 val = 0;
+
+ /* Display WA #0529: skl, kbl, bxt. */
+ if (fbc_state->override_cfb_stride)
+ val |= CHICKEN_FBC_STRIDE_OVERRIDE |
+ CHICKEN_FBC_STRIDE(fbc_state->override_cfb_stride / fbc->limit);
+
+ intel_de_rmw(i915, CHICKEN_MISC_4,
+ CHICKEN_FBC_STRIDE_OVERRIDE |
+ CHICKEN_FBC_STRIDE_MASK, val);
+}
+
+static u32 ivb_dpfc_ctl(struct intel_fbc *fbc)
+{
+ const struct intel_fbc_state *fbc_state = &fbc->state;
+ struct drm_i915_private *i915 = fbc->i915;
u32 dpfc_ctl;
- if (DISPLAY_VER(dev_priv) >= 10) {
- u32 val = 0;
+ dpfc_ctl = g4x_dpfc_ctl_limit(fbc);
- if (params->override_cfb_stride)
- val |= FBC_STRIDE_OVERRIDE |
- FBC_STRIDE(params->override_cfb_stride / fbc->limit);
+ if (IS_IVYBRIDGE(i915))
+ dpfc_ctl |= DPFC_CTL_PLANE_IVB(fbc_state->plane->i9xx_plane);
- intel_de_write(dev_priv, GLK_FBC_STRIDE, val);
- } else if (DISPLAY_VER(dev_priv) == 9) {
- u32 val = 0;
+ if (fbc_state->fence_id >= 0)
+ dpfc_ctl |= DPFC_CTL_FENCE_EN_IVB;
- /* Display WA #0529: skl, kbl, bxt. */
- if (params->override_cfb_stride)
- val |= CHICKEN_FBC_STRIDE_OVERRIDE |
- CHICKEN_FBC_STRIDE(params->override_cfb_stride / fbc->limit);
+ if (fbc->false_color)
+ dpfc_ctl |= DPFC_CTL_FALSE_COLOR;
- intel_de_rmw(dev_priv, CHICKEN_MISC_4,
- CHICKEN_FBC_STRIDE_OVERRIDE |
- CHICKEN_FBC_STRIDE_MASK, val);
- }
-
- dpfc_ctl = 0;
- if (IS_IVYBRIDGE(dev_priv))
- dpfc_ctl |= IVB_DPFC_CTL_PLANE(params->crtc.i9xx_plane);
-
- dpfc_ctl |= g4x_dpfc_ctl_limit(dev_priv);
-
- if (params->fence_id >= 0) {
- dpfc_ctl |= IVB_DPFC_CTL_FENCE_EN;
- intel_de_write(dev_priv, SNB_DPFC_CTL_SA,
- SNB_CPU_FENCE_ENABLE | params->fence_id);
- intel_de_write(dev_priv, DPFC_CPU_FENCE_OFFSET,
- params->fence_y_offset);
- } else if (dev_priv->ggtt.num_fences) {
- intel_de_write(dev_priv, SNB_DPFC_CTL_SA, 0);
- intel_de_write(dev_priv, DPFC_CPU_FENCE_OFFSET, 0);
- }
-
- if (dev_priv->fbc.false_color)
- dpfc_ctl |= FBC_CTL_FALSE_COLOR;
-
- intel_de_write(dev_priv, ILK_DPFC_CONTROL, dpfc_ctl | DPFC_CTL_EN);
+ return dpfc_ctl;
}
-static bool intel_fbc_hw_is_active(struct drm_i915_private *dev_priv)
+static void ivb_fbc_activate(struct intel_fbc *fbc)
{
- if (DISPLAY_VER(dev_priv) >= 5)
- return ilk_fbc_is_active(dev_priv);
- else if (IS_GM45(dev_priv))
- return g4x_fbc_is_active(dev_priv);
- else
- return i8xx_fbc_is_active(dev_priv);
+ struct drm_i915_private *i915 = fbc->i915;
+
+ if (DISPLAY_VER(i915) >= 10)
+ glk_fbc_program_cfb_stride(fbc);
+ else if (DISPLAY_VER(i915) == 9)
+ skl_fbc_program_cfb_stride(fbc);
+
+ if (i915->ggtt.num_fences)
+ snb_fbc_program_fence(fbc);
+
+ intel_de_write(i915, ILK_DPFC_CONTROL,
+ DPFC_CTL_EN | ivb_dpfc_ctl(fbc));
}
-static void intel_fbc_hw_activate(struct drm_i915_private *dev_priv)
+static bool ivb_fbc_is_compressing(struct intel_fbc *fbc)
{
- struct intel_fbc *fbc = &dev_priv->fbc;
+ return intel_de_read(fbc->i915, ILK_DPFC_STATUS2) & DPFC_COMP_SEG_MASK_IVB;
+}
- trace_intel_fbc_activate(fbc->crtc);
+static void ivb_fbc_set_false_color(struct intel_fbc *fbc,
+ bool enable)
+{
+ intel_de_rmw(fbc->i915, ILK_DPFC_CONTROL,
+ DPFC_CTL_FALSE_COLOR, enable ? DPFC_CTL_FALSE_COLOR : 0);
+}
+
+static const struct intel_fbc_funcs ivb_fbc_funcs = {
+ .activate = ivb_fbc_activate,
+ .deactivate = ilk_fbc_deactivate,
+ .is_active = ilk_fbc_is_active,
+ .is_compressing = ivb_fbc_is_compressing,
+ .nuke = snb_fbc_nuke,
+ .program_cfb = ilk_fbc_program_cfb,
+ .set_false_color = ivb_fbc_set_false_color,
+};
+
+static bool intel_fbc_hw_is_active(struct intel_fbc *fbc)
+{
+ return fbc->funcs->is_active(fbc);
+}
+
+static void intel_fbc_hw_activate(struct intel_fbc *fbc)
+{
+ trace_intel_fbc_activate(fbc->state.plane);
fbc->active = true;
fbc->activated = true;
- if (DISPLAY_VER(dev_priv) >= 7)
- gen7_fbc_activate(dev_priv);
- else if (DISPLAY_VER(dev_priv) >= 5)
- ilk_fbc_activate(dev_priv);
- else if (IS_GM45(dev_priv))
- g4x_fbc_activate(dev_priv);
- else
- i8xx_fbc_activate(dev_priv);
+ fbc->funcs->activate(fbc);
}
-static void intel_fbc_hw_deactivate(struct drm_i915_private *dev_priv)
+static void intel_fbc_hw_deactivate(struct intel_fbc *fbc)
{
- struct intel_fbc *fbc = &dev_priv->fbc;
-
- trace_intel_fbc_deactivate(fbc->crtc);
+ trace_intel_fbc_deactivate(fbc->state.plane);
fbc->active = false;
- if (DISPLAY_VER(dev_priv) >= 5)
- ilk_fbc_deactivate(dev_priv);
- else if (IS_GM45(dev_priv))
- g4x_fbc_deactivate(dev_priv);
- else
- i8xx_fbc_deactivate(dev_priv);
+ fbc->funcs->deactivate(fbc);
}
-/**
- * intel_fbc_is_active - Is FBC active?
- * @dev_priv: i915 device instance
- *
- * This function is used to verify the current state of FBC.
- *
- * FIXME: This should be tracked in the plane config eventually
- * instead of queried at runtime for most callers.
- */
-bool intel_fbc_is_active(struct drm_i915_private *dev_priv)
+static bool intel_fbc_is_compressing(struct intel_fbc *fbc)
{
- return dev_priv->fbc.active;
+ return fbc->funcs->is_compressing(fbc);
}
-static void intel_fbc_activate(struct drm_i915_private *dev_priv)
+static void intel_fbc_nuke(struct intel_fbc *fbc)
{
- intel_fbc_hw_activate(dev_priv);
- intel_fbc_recompress(dev_priv);
+ trace_intel_fbc_nuke(fbc->state.plane);
+
+ fbc->funcs->nuke(fbc);
}
-static void intel_fbc_deactivate(struct drm_i915_private *dev_priv,
- const char *reason)
+static void intel_fbc_activate(struct intel_fbc *fbc)
{
- struct intel_fbc *fbc = &dev_priv->fbc;
+ intel_fbc_hw_activate(fbc);
+ intel_fbc_nuke(fbc);
- drm_WARN_ON(&dev_priv->drm, !mutex_is_locked(&fbc->lock));
+ fbc->no_fbc_reason = NULL;
+}
+
+static void intel_fbc_deactivate(struct intel_fbc *fbc, const char *reason)
+{
+ struct drm_i915_private *i915 = fbc->i915;
+
+ drm_WARN_ON(&i915->drm, !mutex_is_locked(&fbc->lock));
if (fbc->active)
- intel_fbc_hw_deactivate(dev_priv);
+ intel_fbc_hw_deactivate(fbc);
fbc->no_fbc_reason = reason;
}
@@ -492,7 +688,7 @@ static u64 intel_fbc_cfb_base_max(struct drm_i915_private *i915)
return BIT_ULL(32);
}
-static u64 intel_fbc_stolen_end(struct drm_i915_private *dev_priv)
+static u64 intel_fbc_stolen_end(struct drm_i915_private *i915)
{
u64 end;
@@ -500,24 +696,24 @@ static u64 intel_fbc_stolen_end(struct drm_i915_private *dev_priv)
* reserved range size, so it always assumes the maximum (8mb) is used.
* If we enable FBC using a CFB on that memory range we'll get FIFO
* underruns, even if that range is not reserved by the BIOS. */
- if (IS_BROADWELL(dev_priv) || (DISPLAY_VER(dev_priv) == 9 &&
- !IS_BROXTON(dev_priv)))
- end = resource_size(&dev_priv->dsm) - 8 * 1024 * 1024;
+ if (IS_BROADWELL(i915) ||
+ (DISPLAY_VER(i915) == 9 && !IS_BROXTON(i915)))
+ end = resource_size(&i915->dsm) - 8 * 1024 * 1024;
else
end = U64_MAX;
- return min(end, intel_fbc_cfb_base_max(dev_priv));
+ return min(end, intel_fbc_cfb_base_max(i915));
}
-static int intel_fbc_min_limit(int fb_cpp)
+static int intel_fbc_min_limit(const struct intel_plane_state *plane_state)
{
- return fb_cpp == 2 ? 2 : 1;
+ return plane_state->hw.fb->format->cpp[0] == 2 ? 2 : 1;
}
-static int intel_fbc_max_limit(struct drm_i915_private *dev_priv)
+static int intel_fbc_max_limit(struct drm_i915_private *i915)
{
/* WaFbcOnly1to1Ratio:ctg */
- if (IS_G4X(dev_priv))
+ if (IS_G4X(i915))
return 1;
/*
@@ -527,23 +723,23 @@ static int intel_fbc_max_limit(struct drm_i915_private *dev_priv)
return 4;
}
-static int find_compression_limit(struct drm_i915_private *dev_priv,
+static int find_compression_limit(struct intel_fbc *fbc,
unsigned int size, int min_limit)
{
- struct intel_fbc *fbc = &dev_priv->fbc;
- u64 end = intel_fbc_stolen_end(dev_priv);
+ struct drm_i915_private *i915 = fbc->i915;
+ u64 end = intel_fbc_stolen_end(i915);
int ret, limit = min_limit;
size /= limit;
/* Try to over-allocate to reduce reallocations and fragmentation. */
- ret = i915_gem_stolen_insert_node_in_range(dev_priv, &fbc->compressed_fb,
+ ret = i915_gem_stolen_insert_node_in_range(i915, &fbc->compressed_fb,
size <<= 1, 4096, 0, end);
if (ret == 0)
return limit;
- for (; limit <= intel_fbc_max_limit(dev_priv); limit <<= 1) {
- ret = i915_gem_stolen_insert_node_in_range(dev_priv, &fbc->compressed_fb,
+ for (; limit <= intel_fbc_max_limit(i915); limit <<= 1) {
+ ret = i915_gem_stolen_insert_node_in_range(i915, &fbc->compressed_fb,
size >>= 1, 4096, 0, end);
if (ret == 0)
return limit;
@@ -552,34 +748,34 @@ static int find_compression_limit(struct drm_i915_private *dev_priv,
return 0;
}
-static int intel_fbc_alloc_cfb(struct drm_i915_private *dev_priv,
+static int intel_fbc_alloc_cfb(struct intel_fbc *fbc,
unsigned int size, int min_limit)
{
- struct intel_fbc *fbc = &dev_priv->fbc;
+ struct drm_i915_private *i915 = fbc->i915;
int ret;
- drm_WARN_ON(&dev_priv->drm,
+ drm_WARN_ON(&i915->drm,
drm_mm_node_allocated(&fbc->compressed_fb));
- drm_WARN_ON(&dev_priv->drm,
+ drm_WARN_ON(&i915->drm,
drm_mm_node_allocated(&fbc->compressed_llb));
- if (DISPLAY_VER(dev_priv) < 5 && !IS_G4X(dev_priv)) {
- ret = i915_gem_stolen_insert_node(dev_priv, &fbc->compressed_llb,
+ if (DISPLAY_VER(i915) < 5 && !IS_G4X(i915)) {
+ ret = i915_gem_stolen_insert_node(i915, &fbc->compressed_llb,
4096, 4096);
if (ret)
goto err;
}
- ret = find_compression_limit(dev_priv, size, min_limit);
+ ret = find_compression_limit(fbc, size, min_limit);
if (!ret)
goto err_llb;
else if (ret > min_limit)
- drm_info_once(&dev_priv->drm,
+ drm_info_once(&i915->drm,
"Reducing the compressed framebuffer size. This may lead to less power savings than a non-reduced-size. Try to increase stolen memory size if available in BIOS.\n");
fbc->limit = ret;
- drm_dbg_kms(&dev_priv->drm,
+ drm_dbg_kms(&i915->drm,
"reserved %llu bytes of contiguous stolen space for FBC, limit: %d\n",
fbc->compressed_fb.size, fbc->limit);
@@ -587,83 +783,69 @@ static int intel_fbc_alloc_cfb(struct drm_i915_private *dev_priv,
err_llb:
if (drm_mm_node_allocated(&fbc->compressed_llb))
- i915_gem_stolen_remove_node(dev_priv, &fbc->compressed_llb);
+ i915_gem_stolen_remove_node(i915, &fbc->compressed_llb);
err:
- if (drm_mm_initialized(&dev_priv->mm.stolen))
- drm_info_once(&dev_priv->drm, "not enough stolen space for compressed buffer (need %d more bytes), disabling. Hint: you may be able to increase stolen memory size in the BIOS to avoid this.\n", size);
+ if (drm_mm_initialized(&i915->mm.stolen))
+ drm_info_once(&i915->drm, "not enough stolen space for compressed buffer (need %d more bytes), disabling. Hint: you may be able to increase stolen memory size in the BIOS to avoid this.\n", size);
return -ENOSPC;
}
-static void intel_fbc_program_cfb(struct drm_i915_private *dev_priv)
+static void intel_fbc_program_cfb(struct intel_fbc *fbc)
{
- struct intel_fbc *fbc = &dev_priv->fbc;
-
- if (DISPLAY_VER(dev_priv) >= 5) {
- intel_de_write(dev_priv, ILK_DPFC_CB_BASE,
- fbc->compressed_fb.start);
- } else if (IS_GM45(dev_priv)) {
- intel_de_write(dev_priv, DPFC_CB_BASE,
- fbc->compressed_fb.start);
- } else {
- GEM_BUG_ON(range_overflows_end_t(u64, dev_priv->dsm.start,
- fbc->compressed_fb.start,
- U32_MAX));
- GEM_BUG_ON(range_overflows_end_t(u64, dev_priv->dsm.start,
- fbc->compressed_llb.start,
- U32_MAX));
-
- intel_de_write(dev_priv, FBC_CFB_BASE,
- dev_priv->dsm.start + fbc->compressed_fb.start);
- intel_de_write(dev_priv, FBC_LL_BASE,
- dev_priv->dsm.start + fbc->compressed_llb.start);
- }
+ fbc->funcs->program_cfb(fbc);
}
-static void __intel_fbc_cleanup_cfb(struct drm_i915_private *dev_priv)
+static void __intel_fbc_cleanup_cfb(struct intel_fbc *fbc)
{
- struct intel_fbc *fbc = &dev_priv->fbc;
+ struct drm_i915_private *i915 = fbc->i915;
- if (WARN_ON(intel_fbc_hw_is_active(dev_priv)))
+ if (WARN_ON(intel_fbc_hw_is_active(fbc)))
return;
if (drm_mm_node_allocated(&fbc->compressed_llb))
- i915_gem_stolen_remove_node(dev_priv, &fbc->compressed_llb);
+ i915_gem_stolen_remove_node(i915, &fbc->compressed_llb);
if (drm_mm_node_allocated(&fbc->compressed_fb))
- i915_gem_stolen_remove_node(dev_priv, &fbc->compressed_fb);
+ i915_gem_stolen_remove_node(i915, &fbc->compressed_fb);
}
-void intel_fbc_cleanup_cfb(struct drm_i915_private *dev_priv)
+void intel_fbc_cleanup(struct drm_i915_private *i915)
{
- struct intel_fbc *fbc = &dev_priv->fbc;
+ struct intel_fbc *fbc = i915->fbc;
- if (!HAS_FBC(dev_priv))
+ if (!fbc)
return;
mutex_lock(&fbc->lock);
- __intel_fbc_cleanup_cfb(dev_priv);
+ __intel_fbc_cleanup_cfb(fbc);
mutex_unlock(&fbc->lock);
+
+ kfree(fbc);
}
-static bool stride_is_valid(struct drm_i915_private *dev_priv,
- u64 modifier, unsigned int stride)
+static bool stride_is_valid(const struct intel_plane_state *plane_state)
{
+ struct drm_i915_private *i915 = to_i915(plane_state->uapi.plane->dev);
+ const struct drm_framebuffer *fb = plane_state->hw.fb;
+ unsigned int stride = intel_fbc_plane_stride(plane_state) *
+ fb->format->cpp[0];
+
/* This should have been caught earlier. */
- if (drm_WARN_ON_ONCE(&dev_priv->drm, (stride & (64 - 1)) != 0))
+ if (drm_WARN_ON_ONCE(&i915->drm, (stride & (64 - 1)) != 0))
return false;
/* Below are the additional FBC restrictions. */
if (stride < 512)
return false;
- if (DISPLAY_VER(dev_priv) == 2 || DISPLAY_VER(dev_priv) == 3)
+ if (DISPLAY_VER(i915) == 2 || DISPLAY_VER(i915) == 3)
return stride == 4096 || stride == 8192;
- if (DISPLAY_VER(dev_priv) == 4 && !IS_G4X(dev_priv) && stride < 2048)
+ if (DISPLAY_VER(i915) == 4 && !IS_G4X(i915) && stride < 2048)
return false;
/* Display WA #1105: skl,bxt,kbl,cfl,glk */
- if ((DISPLAY_VER(dev_priv) == 9 || IS_GEMINILAKE(dev_priv)) &&
- modifier == DRM_FORMAT_MOD_LINEAR && stride & 511)
+ if ((DISPLAY_VER(i915) == 9 || IS_GEMINILAKE(i915)) &&
+ fb->modifier == DRM_FORMAT_MOD_LINEAR && stride & 511)
return false;
if (stride > 16384)
@@ -672,20 +854,22 @@ static bool stride_is_valid(struct drm_i915_private *dev_priv,
return true;
}
-static bool pixel_format_is_valid(struct drm_i915_private *dev_priv,
- u32 pixel_format)
+static bool pixel_format_is_valid(const struct intel_plane_state *plane_state)
{
- switch (pixel_format) {
+ struct drm_i915_private *i915 = to_i915(plane_state->uapi.plane->dev);
+ const struct drm_framebuffer *fb = plane_state->hw.fb;
+
+ switch (fb->format->format) {
case DRM_FORMAT_XRGB8888:
case DRM_FORMAT_XBGR8888:
return true;
case DRM_FORMAT_XRGB1555:
case DRM_FORMAT_RGB565:
/* 16bpp not supported on gen2 */
- if (DISPLAY_VER(dev_priv) == 2)
+ if (DISPLAY_VER(i915) == 2)
return false;
/* WaFbcOnly1to1Ratio:ctg */
- if (IS_G4X(dev_priv))
+ if (IS_G4X(i915))
return false;
return true;
default:
@@ -693,13 +877,16 @@ static bool pixel_format_is_valid(struct drm_i915_private *dev_priv,
}
}
-static bool rotation_is_valid(struct drm_i915_private *dev_priv,
- u32 pixel_format, unsigned int rotation)
+static bool rotation_is_valid(const struct intel_plane_state *plane_state)
{
- if (DISPLAY_VER(dev_priv) >= 9 && pixel_format == DRM_FORMAT_RGB565 &&
+ struct drm_i915_private *i915 = to_i915(plane_state->uapi.plane->dev);
+ const struct drm_framebuffer *fb = plane_state->hw.fb;
+ unsigned int rotation = plane_state->hw.rotation;
+
+ if (DISPLAY_VER(i915) >= 9 && fb->format->format == DRM_FORMAT_RGB565 &&
drm_rotation_90_or_270(rotation))
return false;
- else if (DISPLAY_VER(dev_priv) <= 4 && !IS_G4X(dev_priv) &&
+ else if (DISPLAY_VER(i915) <= 4 && !IS_G4X(i915) &&
rotation != DRM_MODE_ROTATE_0)
return false;
@@ -712,19 +899,18 @@ static bool rotation_is_valid(struct drm_i915_private *dev_priv,
* the X and Y offset registers. That's why we include the src x/y offsets
* instead of just looking at the plane size.
*/
-static bool intel_fbc_hw_tracking_covers_screen(struct intel_crtc *crtc)
+static bool intel_fbc_hw_tracking_covers_screen(const struct intel_plane_state *plane_state)
{
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
- struct intel_fbc *fbc = &dev_priv->fbc;
+ struct drm_i915_private *i915 = to_i915(plane_state->uapi.plane->dev);
unsigned int effective_w, effective_h, max_w, max_h;
- if (DISPLAY_VER(dev_priv) >= 10) {
+ if (DISPLAY_VER(i915) >= 10) {
max_w = 5120;
max_h = 4096;
- } else if (DISPLAY_VER(dev_priv) >= 8 || IS_HASWELL(dev_priv)) {
+ } else if (DISPLAY_VER(i915) >= 8 || IS_HASWELL(i915)) {
max_w = 4096;
max_h = 4096;
- } else if (IS_G4X(dev_priv) || DISPLAY_VER(dev_priv) >= 5) {
+ } else if (IS_G4X(i915) || DISPLAY_VER(i915) >= 5) {
max_w = 4096;
max_h = 2048;
} else {
@@ -732,22 +918,24 @@ static bool intel_fbc_hw_tracking_covers_screen(struct intel_crtc *crtc)
max_h = 1536;
}
- intel_fbc_get_plane_source_size(&fbc->state_cache, &effective_w,
- &effective_h);
- effective_w += fbc->state_cache.plane.adjusted_x;
- effective_h += fbc->state_cache.plane.adjusted_y;
+ effective_w = plane_state->view.color_plane[0].x +
+ (drm_rect_width(&plane_state->uapi.src) >> 16);
+ effective_h = plane_state->view.color_plane[0].y +
+ (drm_rect_height(&plane_state->uapi.src) >> 16);
return effective_w <= max_w && effective_h <= max_h;
}
-static bool tiling_is_valid(struct drm_i915_private *dev_priv,
- u64 modifier)
+static bool tiling_is_valid(const struct intel_plane_state *plane_state)
{
- switch (modifier) {
+ struct drm_i915_private *i915 = to_i915(plane_state->uapi.plane->dev);
+ const struct drm_framebuffer *fb = plane_state->hw.fb;
+
+ switch (fb->modifier) {
case DRM_FORMAT_MOD_LINEAR:
case I915_FORMAT_MOD_Y_TILED:
case I915_FORMAT_MOD_Yf_TILED:
- return DISPLAY_VER(dev_priv) >= 9;
+ return DISPLAY_VER(i915) >= 9;
case I915_FORMAT_MOD_X_TILED:
return true;
default:
@@ -755,139 +943,44 @@ static bool tiling_is_valid(struct drm_i915_private *dev_priv,
}
}
-static void intel_fbc_update_state_cache(struct intel_crtc *crtc,
- const struct intel_crtc_state *crtc_state,
- const struct intel_plane_state *plane_state)
+static void intel_fbc_update_state(struct intel_atomic_state *state,
+ struct intel_crtc *crtc,
+ struct intel_plane *plane)
{
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
- struct intel_fbc *fbc = &dev_priv->fbc;
- struct intel_fbc_state_cache *cache = &fbc->state_cache;
- struct drm_framebuffer *fb = plane_state->hw.fb;
+ struct drm_i915_private *i915 = to_i915(state->base.dev);
+ const struct intel_crtc_state *crtc_state =
+ intel_atomic_get_new_crtc_state(state, crtc);
+ const struct intel_plane_state *plane_state =
+ intel_atomic_get_new_plane_state(state, plane);
+ struct intel_fbc *fbc = plane->fbc;
+ struct intel_fbc_state *fbc_state = &fbc->state;
- cache->plane.visible = plane_state->uapi.visible;
- if (!cache->plane.visible)
- return;
+ WARN_ON(plane_state->no_fbc_reason);
- cache->crtc.mode_flags = crtc_state->hw.adjusted_mode.flags;
- if (IS_HASWELL(dev_priv) || IS_BROADWELL(dev_priv))
- cache->crtc.hsw_bdw_pixel_rate = crtc_state->pixel_rate;
-
- cache->plane.rotation = plane_state->hw.rotation;
- /*
- * Src coordinates are already rotated by 270 degrees for
- * the 90/270 degree plane rotation cases (to match the
- * GTT mapping), hence no need to account for rotation here.
- */
- cache->plane.src_w = drm_rect_width(&plane_state->uapi.src) >> 16;
- cache->plane.src_h = drm_rect_height(&plane_state->uapi.src) >> 16;
- cache->plane.adjusted_x = plane_state->view.color_plane[0].x;
- cache->plane.adjusted_y = plane_state->view.color_plane[0].y;
-
- cache->plane.pixel_blend_mode = plane_state->hw.pixel_blend_mode;
-
- cache->fb.format = fb->format;
- cache->fb.modifier = fb->modifier;
- cache->fb.stride = intel_fbc_plane_stride(plane_state);
+ fbc_state->plane = plane;
/* FBC1 compression interval: arbitrary choice of 1 second */
- cache->interval = drm_mode_vrefresh(&crtc_state->hw.adjusted_mode);
+ fbc_state->interval = drm_mode_vrefresh(&crtc_state->hw.adjusted_mode);
- cache->fence_y_offset = intel_plane_fence_y_offset(plane_state);
+ fbc_state->fence_y_offset = intel_plane_fence_y_offset(plane_state);
- drm_WARN_ON(&dev_priv->drm, plane_state->flags & PLANE_HAS_FENCE &&
+ drm_WARN_ON(&i915->drm, plane_state->flags & PLANE_HAS_FENCE &&
!plane_state->ggtt_vma->fence);
if (plane_state->flags & PLANE_HAS_FENCE &&
plane_state->ggtt_vma->fence)
- cache->fence_id = plane_state->ggtt_vma->fence->id;
+ fbc_state->fence_id = plane_state->ggtt_vma->fence->id;
else
- cache->fence_id = -1;
+ fbc_state->fence_id = -1;
- cache->psr2_active = crtc_state->has_psr2;
+ fbc_state->cfb_stride = intel_fbc_cfb_stride(plane_state);
+ fbc_state->cfb_size = intel_fbc_cfb_size(plane_state);
+ fbc_state->override_cfb_stride = intel_fbc_override_cfb_stride(plane_state);
}
-static bool intel_fbc_cfb_size_changed(struct drm_i915_private *dev_priv)
+static bool intel_fbc_is_fence_ok(const struct intel_plane_state *plane_state)
{
- struct intel_fbc *fbc = &dev_priv->fbc;
-
- return intel_fbc_cfb_size(dev_priv, &fbc->state_cache) >
- fbc->compressed_fb.size * fbc->limit;
-}
-
-static u16 intel_fbc_override_cfb_stride(struct drm_i915_private *dev_priv,
- const struct intel_fbc_state_cache *cache)
-{
- unsigned int stride = _intel_fbc_cfb_stride(cache);
- unsigned int stride_aligned = intel_fbc_cfb_stride(dev_priv, cache);
-
- /*
- * Override stride in 64 byte units per 4 line segment.
- *
- * Gen9 hw miscalculates cfb stride for linear as
- * PLANE_STRIDE*512 instead of PLANE_STRIDE*64, so
- * we always need to use the override there.
- */
- if (stride != stride_aligned ||
- (DISPLAY_VER(dev_priv) == 9 &&
- cache->fb.modifier == DRM_FORMAT_MOD_LINEAR))
- return stride_aligned * 4 / 64;
-
- return 0;
-}
-
-static bool intel_fbc_can_enable(struct drm_i915_private *dev_priv)
-{
- struct intel_fbc *fbc = &dev_priv->fbc;
-
- if (intel_vgpu_active(dev_priv)) {
- fbc->no_fbc_reason = "VGPU is active";
- return false;
- }
-
- if (!dev_priv->params.enable_fbc) {
- fbc->no_fbc_reason = "disabled per module param or by default";
- return false;
- }
-
- if (fbc->underrun_detected) {
- fbc->no_fbc_reason = "underrun detected";
- return false;
- }
-
- return true;
-}
-
-static bool intel_fbc_can_activate(struct intel_crtc *crtc)
-{
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
- struct intel_fbc *fbc = &dev_priv->fbc;
- struct intel_fbc_state_cache *cache = &fbc->state_cache;
-
- if (!intel_fbc_can_enable(dev_priv))
- return false;
-
- if (!cache->plane.visible) {
- fbc->no_fbc_reason = "primary plane not visible";
- return false;
- }
-
- /* We don't need to use a state cache here since this information is
- * global for all CRTC.
- */
- if (fbc->underrun_detected) {
- fbc->no_fbc_reason = "underrun detected";
- return false;
- }
-
- if (cache->crtc.mode_flags & DRM_MODE_FLAG_INTERLACE) {
- fbc->no_fbc_reason = "incompatible mode";
- return false;
- }
-
- if (!intel_fbc_hw_tracking_covers_screen(crtc)) {
- fbc->no_fbc_reason = "mode too large for compression";
- return false;
- }
+ struct drm_i915_private *i915 = to_i915(plane_state->uapi.plane->dev);
/* The use of a CPU fence is one of two ways to detect writes by the
* CPU to the scanout and trigger updates to the FBC.
@@ -906,77 +999,66 @@ static bool intel_fbc_can_activate(struct intel_crtc *crtc)
* For now this will effectively disable FBC with 90/270 degree
* rotation.
*/
- if (DISPLAY_VER(dev_priv) < 9 && cache->fence_id < 0) {
- fbc->no_fbc_reason = "framebuffer not tiled or fenced";
- return false;
+ return DISPLAY_VER(i915) >= 9 ||
+ (plane_state->flags & PLANE_HAS_FENCE &&
+ plane_state->ggtt_vma->fence);
+}
+
+static bool intel_fbc_is_cfb_ok(const struct intel_plane_state *plane_state)
+{
+ struct intel_plane *plane = to_intel_plane(plane_state->uapi.plane);
+ struct intel_fbc *fbc = plane->fbc;
+
+ return intel_fbc_min_limit(plane_state) <= fbc->limit &&
+ intel_fbc_cfb_size(plane_state) <= fbc->compressed_fb.size * fbc->limit;
+}
+
+static bool intel_fbc_is_ok(const struct intel_plane_state *plane_state)
+{
+ return !plane_state->no_fbc_reason &&
+ intel_fbc_is_fence_ok(plane_state) &&
+ intel_fbc_is_cfb_ok(plane_state);
+}
+
+static int intel_fbc_check_plane(struct intel_atomic_state *state,
+ struct intel_plane *plane)
+{
+ struct drm_i915_private *i915 = to_i915(state->base.dev);
+ struct intel_plane_state *plane_state =
+ intel_atomic_get_new_plane_state(state, plane);
+ const struct drm_framebuffer *fb = plane_state->hw.fb;
+ struct intel_crtc *crtc = to_intel_crtc(plane_state->uapi.crtc);
+ const struct intel_crtc_state *crtc_state;
+ struct intel_fbc *fbc = plane->fbc;
+
+ if (!fbc)
+ return 0;
+
+ if (intel_vgpu_active(i915)) {
+ plane_state->no_fbc_reason = "VGPU active";
+ return 0;
}
- if (!pixel_format_is_valid(dev_priv, cache->fb.format->format)) {
- fbc->no_fbc_reason = "pixel format is invalid";
- return false;
+ if (!i915->params.enable_fbc) {
+ plane_state->no_fbc_reason = "disabled per module param or by default";
+ return 0;
}
- if (!rotation_is_valid(dev_priv, cache->fb.format->format,
- cache->plane.rotation)) {
- fbc->no_fbc_reason = "rotation unsupported";
- return false;
+ if (!plane_state->uapi.visible) {
+ plane_state->no_fbc_reason = "plane not visible";
+ return 0;
}
- if (!tiling_is_valid(dev_priv, cache->fb.modifier)) {
- fbc->no_fbc_reason = "tiling unsupported";
- return false;
+ crtc_state = intel_atomic_get_new_crtc_state(state, crtc);
+
+ if (crtc_state->hw.adjusted_mode.flags & DRM_MODE_FLAG_INTERLACE) {
+ plane_state->no_fbc_reason = "interlaced mode not supported";
+ return 0;
}
- if (!stride_is_valid(dev_priv, cache->fb.modifier,
- cache->fb.stride * cache->fb.format->cpp[0])) {
- fbc->no_fbc_reason = "framebuffer stride not supported";
- return false;
- }
-
- if (cache->plane.pixel_blend_mode != DRM_MODE_BLEND_PIXEL_NONE &&
- cache->fb.format->has_alpha) {
- fbc->no_fbc_reason = "per-pixel alpha blending is incompatible with FBC";
- return false;
- }
-
- /* WaFbcExceedCdClockThreshold:hsw,bdw */
- if ((IS_HASWELL(dev_priv) || IS_BROADWELL(dev_priv)) &&
- cache->crtc.hsw_bdw_pixel_rate >= dev_priv->cdclk.hw.cdclk * 95 / 100) {
- fbc->no_fbc_reason = "pixel rate is too big";
- return false;
- }
-
- /* It is possible for the required CFB size change without a
- * crtc->disable + crtc->enable since it is possible to change the
- * stride without triggering a full modeset. Since we try to
- * over-allocate the CFB, there's a chance we may keep FBC enabled even
- * if this happens, but if we exceed the current CFB size we'll have to
- * disable FBC. Notice that it would be possible to disable FBC, wait
- * for a frame, free the stolen node, then try to reenable FBC in case
- * we didn't get any invalidate/deactivate calls, but this would require
- * a lot of tracking just for a specific case. If we conclude it's an
- * important case, we can implement it later. */
- if (intel_fbc_cfb_size_changed(dev_priv)) {
- fbc->no_fbc_reason = "CFB requirements changed";
- return false;
- }
-
- /*
- * Work around a problem on GEN9+ HW, where enabling FBC on a plane
- * having a Y offset that isn't divisible by 4 causes FIFO underrun
- * and screen flicker.
- */
- if (DISPLAY_VER(dev_priv) >= 9 &&
- (fbc->state_cache.plane.adjusted_y & 3)) {
- fbc->no_fbc_reason = "plane Y offset is misaligned";
- return false;
- }
-
- /* Wa_22010751166: icl, ehl, tgl, dg1, rkl */
- if (DISPLAY_VER(dev_priv) >= 11 &&
- (cache->plane.src_h + cache->plane.adjusted_y) % 4) {
- fbc->no_fbc_reason = "plane height + offset is non-modulo of 4";
- return false;
+ if (crtc_state->double_wide) {
+ plane_state->no_fbc_reason = "double wide pipe not supported";
+ return 0;
}
/*
@@ -984,220 +1066,252 @@ static bool intel_fbc_can_activate(struct intel_crtc *crtc)
* Recommendation is to keep this combination disabled
* Bspec: 50422 HSD: 14010260002
*/
- if (fbc->state_cache.psr2_active && DISPLAY_VER(dev_priv) >= 12) {
- fbc->no_fbc_reason = "not supported with PSR2";
+ if (DISPLAY_VER(i915) >= 12 && crtc_state->has_psr2) {
+ plane_state->no_fbc_reason = "PSR2 enabled";
return false;
}
+ if (!pixel_format_is_valid(plane_state)) {
+ plane_state->no_fbc_reason = "pixel format not supported";
+ return 0;
+ }
+
+ if (!tiling_is_valid(plane_state)) {
+ plane_state->no_fbc_reason = "tiling not supported";
+ return 0;
+ }
+
+ if (!rotation_is_valid(plane_state)) {
+ plane_state->no_fbc_reason = "rotation not supported";
+ return 0;
+ }
+
+ if (!stride_is_valid(plane_state)) {
+ plane_state->no_fbc_reason = "stride not supported";
+ return 0;
+ }
+
+ if (plane_state->hw.pixel_blend_mode != DRM_MODE_BLEND_PIXEL_NONE &&
+ fb->format->has_alpha) {
+ plane_state->no_fbc_reason = "per-pixel alpha not supported";
+ return false;
+ }
+
+ if (!intel_fbc_hw_tracking_covers_screen(plane_state)) {
+ plane_state->no_fbc_reason = "plane size too big";
+ return 0;
+ }
+
+ /*
+ * Work around a problem on GEN9+ HW, where enabling FBC on a plane
+ * having a Y offset that isn't divisible by 4 causes FIFO underrun
+ * and screen flicker.
+ */
+ if (DISPLAY_VER(i915) >= 9 &&
+ plane_state->view.color_plane[0].y & 3) {
+ plane_state->no_fbc_reason = "plane start Y offset misaligned";
+ return false;
+ }
+
+ /* Wa_22010751166: icl, ehl, tgl, dg1, rkl */
+ if (DISPLAY_VER(i915) >= 11 &&
+ (plane_state->view.color_plane[0].y + drm_rect_height(&plane_state->uapi.src)) & 3) {
+ plane_state->no_fbc_reason = "plane end Y offset misaligned";
+ return false;
+ }
+
+ /* WaFbcExceedCdClockThreshold:hsw,bdw */
+ if (IS_HASWELL(i915) || IS_BROADWELL(i915)) {
+ const struct intel_cdclk_state *cdclk_state;
+
+ cdclk_state = intel_atomic_get_cdclk_state(state);
+ if (IS_ERR(cdclk_state))
+ return PTR_ERR(cdclk_state);
+
+ if (crtc_state->pixel_rate >= cdclk_state->logical.cdclk * 95 / 100) {
+ plane_state->no_fbc_reason = "pixel rate too high";
+ return 0;
+ }
+ }
+
+ plane_state->no_fbc_reason = NULL;
+
+ return 0;
+}
+
+
+static bool intel_fbc_can_flip_nuke(struct intel_atomic_state *state,
+ struct intel_crtc *crtc,
+ struct intel_plane *plane)
+{
+ const struct intel_crtc_state *new_crtc_state =
+ intel_atomic_get_new_crtc_state(state, crtc);
+ const struct intel_plane_state *old_plane_state =
+ intel_atomic_get_old_plane_state(state, plane);
+ const struct intel_plane_state *new_plane_state =
+ intel_atomic_get_new_plane_state(state, plane);
+ const struct drm_framebuffer *old_fb = old_plane_state->hw.fb;
+ const struct drm_framebuffer *new_fb = new_plane_state->hw.fb;
+
+ if (drm_atomic_crtc_needs_modeset(&new_crtc_state->uapi))
+ return false;
+
+ if (!intel_fbc_is_ok(old_plane_state) ||
+ !intel_fbc_is_ok(new_plane_state))
+ return false;
+
+ if (old_fb->format->format != new_fb->format->format)
+ return false;
+
+ if (old_fb->modifier != new_fb->modifier)
+ return false;
+
+ if (intel_fbc_plane_stride(old_plane_state) !=
+ intel_fbc_plane_stride(new_plane_state))
+ return false;
+
+ if (intel_fbc_cfb_stride(old_plane_state) !=
+ intel_fbc_cfb_stride(new_plane_state))
+ return false;
+
+ if (intel_fbc_cfb_size(old_plane_state) !=
+ intel_fbc_cfb_size(new_plane_state))
+ return false;
+
+ if (intel_fbc_override_cfb_stride(old_plane_state) !=
+ intel_fbc_override_cfb_stride(new_plane_state))
+ return false;
+
return true;
}
-static void intel_fbc_get_reg_params(struct intel_crtc *crtc,
- struct intel_fbc_reg_params *params)
+static bool __intel_fbc_pre_update(struct intel_atomic_state *state,
+ struct intel_crtc *crtc,
+ struct intel_plane *plane)
{
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
- struct intel_fbc *fbc = &dev_priv->fbc;
- struct intel_fbc_state_cache *cache = &fbc->state_cache;
+ struct drm_i915_private *i915 = to_i915(state->base.dev);
+ struct intel_fbc *fbc = plane->fbc;
+ bool need_vblank_wait = false;
- /* Since all our fields are integer types, use memset here so the
- * comparison function can rely on memcmp because the padding will be
- * zero. */
- memset(params, 0, sizeof(*params));
+ fbc->flip_pending = true;
- params->fence_id = cache->fence_id;
- params->fence_y_offset = cache->fence_y_offset;
+ if (intel_fbc_can_flip_nuke(state, crtc, plane))
+ return need_vblank_wait;
- params->interval = cache->interval;
+ intel_fbc_deactivate(fbc, "update pending");
- params->crtc.pipe = crtc->pipe;
- params->crtc.i9xx_plane = to_intel_plane(crtc->base.primary)->i9xx_plane;
+ /*
+ * Display WA #1198: glk+
+ * Need an extra vblank wait between FBC disable and most plane
+ * updates. Bspec says this is only needed for plane disable, but
+ * that is not true. Touching most plane registers will cause the
+ * corruption to appear. Also SKL/derivatives do not seem to be
+ * affected.
+ *
+ * TODO: could optimize this a bit by sampling the frame
+ * counter when we disable FBC (if it was already done earlier)
+ * and skipping the extra vblank wait before the plane update
+ * if at least one frame has already passed.
+ */
+ if (fbc->activated && DISPLAY_VER(i915) >= 10)
+ need_vblank_wait = true;
+ fbc->activated = false;
- params->fb.format = cache->fb.format;
- params->fb.modifier = cache->fb.modifier;
- params->fb.stride = cache->fb.stride;
-
- params->cfb_stride = intel_fbc_cfb_stride(dev_priv, cache);
- params->cfb_size = intel_fbc_cfb_size(dev_priv, cache);
- params->override_cfb_stride = intel_fbc_override_cfb_stride(dev_priv, cache);
-
- params->plane_visible = cache->plane.visible;
-}
-
-static bool intel_fbc_can_flip_nuke(const struct intel_crtc_state *crtc_state)
-{
- struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
- const struct intel_fbc *fbc = &dev_priv->fbc;
- const struct intel_fbc_state_cache *cache = &fbc->state_cache;
- const struct intel_fbc_reg_params *params = &fbc->params;
-
- if (drm_atomic_crtc_needs_modeset(&crtc_state->uapi))
- return false;
-
- if (!params->plane_visible)
- return false;
-
- if (!intel_fbc_can_activate(crtc))
- return false;
-
- if (params->fb.format != cache->fb.format)
- return false;
-
- if (params->fb.modifier != cache->fb.modifier)
- return false;
-
- if (params->fb.stride != cache->fb.stride)
- return false;
-
- if (params->cfb_stride != intel_fbc_cfb_stride(dev_priv, cache))
- return false;
-
- if (params->cfb_size != intel_fbc_cfb_size(dev_priv, cache))
- return false;
-
- if (params->override_cfb_stride != intel_fbc_override_cfb_stride(dev_priv, cache))
- return false;
-
- return true;
+ return need_vblank_wait;
}
bool intel_fbc_pre_update(struct intel_atomic_state *state,
struct intel_crtc *crtc)
{
- struct intel_plane *plane = to_intel_plane(crtc->base.primary);
- const struct intel_crtc_state *crtc_state =
- intel_atomic_get_new_crtc_state(state, crtc);
- const struct intel_plane_state *plane_state =
- intel_atomic_get_new_plane_state(state, plane);
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
- struct intel_fbc *fbc = &dev_priv->fbc;
- const char *reason = "update pending";
+ const struct intel_plane_state *plane_state;
bool need_vblank_wait = false;
+ struct intel_plane *plane;
+ int i;
- if (!plane->has_fbc || !plane_state)
- return need_vblank_wait;
+ for_each_new_intel_plane_in_state(state, plane, plane_state, i) {
+ struct intel_fbc *fbc = plane->fbc;
- mutex_lock(&fbc->lock);
+ if (!fbc || plane->pipe != crtc->pipe)
+ continue;
- if (fbc->crtc != crtc)
- goto unlock;
+ mutex_lock(&fbc->lock);
- intel_fbc_update_state_cache(crtc, crtc_state, plane_state);
- fbc->flip_pending = true;
+ if (fbc->state.plane == plane)
+ need_vblank_wait |= __intel_fbc_pre_update(state, crtc, plane);
- if (!intel_fbc_can_flip_nuke(crtc_state)) {
- intel_fbc_deactivate(dev_priv, reason);
-
- /*
- * Display WA #1198: glk+
- * Need an extra vblank wait between FBC disable and most plane
- * updates. Bspec says this is only needed for plane disable, but
- * that is not true. Touching most plane registers will cause the
- * corruption to appear. Also SKL/derivatives do not seem to be
- * affected.
- *
- * TODO: could optimize this a bit by sampling the frame
- * counter when we disable FBC (if it was already done earlier)
- * and skipping the extra vblank wait before the plane update
- * if at least one frame has already passed.
- */
- if (fbc->activated &&
- DISPLAY_VER(dev_priv) >= 10)
- need_vblank_wait = true;
- fbc->activated = false;
+ mutex_unlock(&fbc->lock);
}
-unlock:
- mutex_unlock(&fbc->lock);
return need_vblank_wait;
}
-/**
- * __intel_fbc_disable - disable FBC
- * @dev_priv: i915 device instance
- *
- * This is the low level function that actually disables FBC. Callers should
- * grab the FBC lock.
- */
-static void __intel_fbc_disable(struct drm_i915_private *dev_priv)
+static void __intel_fbc_disable(struct intel_fbc *fbc)
{
- struct intel_fbc *fbc = &dev_priv->fbc;
- struct intel_crtc *crtc = fbc->crtc;
+ struct drm_i915_private *i915 = fbc->i915;
+ struct intel_plane *plane = fbc->state.plane;
- drm_WARN_ON(&dev_priv->drm, !mutex_is_locked(&fbc->lock));
- drm_WARN_ON(&dev_priv->drm, !fbc->crtc);
- drm_WARN_ON(&dev_priv->drm, fbc->active);
+ drm_WARN_ON(&i915->drm, !mutex_is_locked(&fbc->lock));
+ drm_WARN_ON(&i915->drm, fbc->active);
- drm_dbg_kms(&dev_priv->drm, "Disabling FBC on pipe %c\n",
- pipe_name(crtc->pipe));
+ drm_dbg_kms(&i915->drm, "Disabling FBC on [PLANE:%d:%s]\n",
+ plane->base.base.id, plane->base.name);
- __intel_fbc_cleanup_cfb(dev_priv);
+ __intel_fbc_cleanup_cfb(fbc);
- fbc->crtc = NULL;
+ fbc->state.plane = NULL;
}
-static void __intel_fbc_post_update(struct intel_crtc *crtc)
+static void __intel_fbc_post_update(struct intel_fbc *fbc)
{
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
- struct intel_fbc *fbc = &dev_priv->fbc;
+ struct drm_i915_private *i915 = fbc->i915;
- drm_WARN_ON(&dev_priv->drm, !mutex_is_locked(&fbc->lock));
-
- if (fbc->crtc != crtc)
- return;
-
- fbc->flip_pending = false;
-
- if (!dev_priv->params.enable_fbc) {
- intel_fbc_deactivate(dev_priv, "disabled at runtime per module param");
- __intel_fbc_disable(dev_priv);
-
- return;
- }
-
- intel_fbc_get_reg_params(crtc, &fbc->params);
-
- if (!intel_fbc_can_activate(crtc))
- return;
+ drm_WARN_ON(&i915->drm, !mutex_is_locked(&fbc->lock));
if (!fbc->busy_bits)
- intel_fbc_activate(dev_priv);
+ intel_fbc_activate(fbc);
else
- intel_fbc_deactivate(dev_priv, "frontbuffer write");
+ intel_fbc_deactivate(fbc, "frontbuffer write");
}
void intel_fbc_post_update(struct intel_atomic_state *state,
struct intel_crtc *crtc)
{
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
- struct intel_plane *plane = to_intel_plane(crtc->base.primary);
- const struct intel_plane_state *plane_state =
- intel_atomic_get_new_plane_state(state, plane);
- struct intel_fbc *fbc = &dev_priv->fbc;
+ const struct intel_plane_state *plane_state;
+ struct intel_plane *plane;
+ int i;
- if (!plane->has_fbc || !plane_state)
- return;
+ for_each_new_intel_plane_in_state(state, plane, plane_state, i) {
+ struct intel_fbc *fbc = plane->fbc;
- mutex_lock(&fbc->lock);
- __intel_fbc_post_update(crtc);
- mutex_unlock(&fbc->lock);
+ if (!fbc || plane->pipe != crtc->pipe)
+ continue;
+
+ mutex_lock(&fbc->lock);
+
+ if (fbc->state.plane == plane) {
+ fbc->flip_pending = false;
+ __intel_fbc_post_update(fbc);
+ }
+
+ mutex_unlock(&fbc->lock);
+ }
}
static unsigned int intel_fbc_get_frontbuffer_bit(struct intel_fbc *fbc)
{
- if (fbc->crtc)
- return to_intel_plane(fbc->crtc->base.primary)->frontbuffer_bit;
+ if (fbc->state.plane)
+ return fbc->state.plane->frontbuffer_bit;
else
return fbc->possible_framebuffer_bits;
}
-void intel_fbc_invalidate(struct drm_i915_private *dev_priv,
+void intel_fbc_invalidate(struct drm_i915_private *i915,
unsigned int frontbuffer_bits,
enum fb_op_origin origin)
{
- struct intel_fbc *fbc = &dev_priv->fbc;
+ struct intel_fbc *fbc = i915->fbc;
- if (!HAS_FBC(dev_priv))
+ if (!fbc)
return;
if (origin == ORIGIN_FLIP || origin == ORIGIN_CURSOR_UPDATE)
@@ -1207,18 +1321,18 @@ void intel_fbc_invalidate(struct drm_i915_private *dev_priv,
fbc->busy_bits |= intel_fbc_get_frontbuffer_bit(fbc) & frontbuffer_bits;
- if (fbc->crtc && fbc->busy_bits)
- intel_fbc_deactivate(dev_priv, "frontbuffer write");
+ if (fbc->state.plane && fbc->busy_bits)
+ intel_fbc_deactivate(fbc, "frontbuffer write");
mutex_unlock(&fbc->lock);
}
-void intel_fbc_flush(struct drm_i915_private *dev_priv,
+void intel_fbc_flush(struct drm_i915_private *i915,
unsigned int frontbuffer_bits, enum fb_op_origin origin)
{
- struct intel_fbc *fbc = &dev_priv->fbc;
+ struct intel_fbc *fbc = i915->fbc;
- if (!HAS_FBC(dev_priv))
+ if (!fbc)
return;
mutex_lock(&fbc->lock);
@@ -1228,143 +1342,85 @@ void intel_fbc_flush(struct drm_i915_private *dev_priv,
if (origin == ORIGIN_FLIP || origin == ORIGIN_CURSOR_UPDATE)
goto out;
- if (!fbc->busy_bits && fbc->crtc &&
+ if (!fbc->busy_bits && fbc->state.plane &&
(frontbuffer_bits & intel_fbc_get_frontbuffer_bit(fbc))) {
if (fbc->active)
- intel_fbc_recompress(dev_priv);
+ intel_fbc_nuke(fbc);
else if (!fbc->flip_pending)
- __intel_fbc_post_update(fbc->crtc);
+ __intel_fbc_post_update(fbc);
}
out:
mutex_unlock(&fbc->lock);
}
-/**
- * intel_fbc_choose_crtc - select a CRTC to enable FBC on
- * @dev_priv: i915 device instance
- * @state: the atomic state structure
- *
- * This function looks at the proposed state for CRTCs and planes, then chooses
- * which pipe is going to have FBC by setting intel_crtc_state->enable_fbc to
- * true.
- *
- * Later, intel_fbc_enable is going to look for state->enable_fbc and then maybe
- * enable FBC for the chosen CRTC. If it does, it will set dev_priv->fbc.crtc.
- */
-void intel_fbc_choose_crtc(struct drm_i915_private *dev_priv,
- struct intel_atomic_state *state)
+int intel_fbc_atomic_check(struct intel_atomic_state *state)
{
- struct intel_fbc *fbc = &dev_priv->fbc;
- struct intel_plane *plane;
struct intel_plane_state *plane_state;
- bool crtc_chosen = false;
+ struct intel_plane *plane;
int i;
- mutex_lock(&fbc->lock);
-
- /* Does this atomic commit involve the CRTC currently tied to FBC? */
- if (fbc->crtc &&
- !intel_atomic_get_new_crtc_state(state, fbc->crtc))
- goto out;
-
- if (!intel_fbc_can_enable(dev_priv))
- goto out;
-
- /* Simply choose the first CRTC that is compatible and has a visible
- * plane. We could go for fancier schemes such as checking the plane
- * size, but this would just affect the few platforms that don't tie FBC
- * to pipe or plane A. */
for_each_new_intel_plane_in_state(state, plane, plane_state, i) {
- struct intel_crtc_state *crtc_state;
- struct intel_crtc *crtc = to_intel_crtc(plane_state->hw.crtc);
+ int ret;
- if (!plane->has_fbc)
- continue;
-
- if (!plane_state->uapi.visible)
- continue;
-
- crtc_state = intel_atomic_get_new_crtc_state(state, crtc);
-
- crtc_state->enable_fbc = true;
- crtc_chosen = true;
- break;
+ ret = intel_fbc_check_plane(state, plane);
+ if (ret)
+ return ret;
}
- if (!crtc_chosen)
- fbc->no_fbc_reason = "no suitable CRTC for FBC";
-
-out:
- mutex_unlock(&fbc->lock);
+ return 0;
}
-/**
- * intel_fbc_enable: tries to enable FBC on the CRTC
- * @crtc: the CRTC
- * @state: corresponding &drm_crtc_state for @crtc
- *
- * This function checks if the given CRTC was chosen for FBC, then enables it if
- * possible. Notice that it doesn't activate FBC. It is valid to call
- * intel_fbc_enable multiple times for the same pipe without an
- * intel_fbc_disable in the middle, as long as it is deactivated.
- */
-static void intel_fbc_enable(struct intel_atomic_state *state,
- struct intel_crtc *crtc)
+static void __intel_fbc_enable(struct intel_atomic_state *state,
+ struct intel_crtc *crtc,
+ struct intel_plane *plane)
{
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
- struct intel_plane *plane = to_intel_plane(crtc->base.primary);
- const struct intel_crtc_state *crtc_state =
- intel_atomic_get_new_crtc_state(state, crtc);
+ struct drm_i915_private *i915 = to_i915(state->base.dev);
const struct intel_plane_state *plane_state =
intel_atomic_get_new_plane_state(state, plane);
- struct intel_fbc *fbc = &dev_priv->fbc;
- struct intel_fbc_state_cache *cache = &fbc->state_cache;
- int min_limit;
+ struct intel_fbc *fbc = plane->fbc;
- if (!plane->has_fbc || !plane_state)
+ if (fbc->state.plane) {
+ if (fbc->state.plane != plane)
+ return;
+
+ if (intel_fbc_is_ok(plane_state)) {
+ intel_fbc_update_state(state, crtc, plane);
+ return;
+ }
+
+ __intel_fbc_disable(fbc);
+ }
+
+ drm_WARN_ON(&i915->drm, fbc->active);
+
+ fbc->no_fbc_reason = plane_state->no_fbc_reason;
+ if (fbc->no_fbc_reason)
return;
- min_limit = intel_fbc_min_limit(plane_state->hw.fb ?
- plane_state->hw.fb->format->cpp[0] : 0);
-
- mutex_lock(&fbc->lock);
-
- if (fbc->crtc) {
- if (fbc->crtc != crtc)
- goto out;
-
- if (fbc->limit >= min_limit &&
- !intel_fbc_cfb_size_changed(dev_priv))
- goto out;
-
- __intel_fbc_disable(dev_priv);
+ if (!intel_fbc_is_fence_ok(plane_state)) {
+ fbc->no_fbc_reason = "framebuffer not fenced";
+ return;
}
- drm_WARN_ON(&dev_priv->drm, fbc->active);
+ if (fbc->underrun_detected) {
+ fbc->no_fbc_reason = "FIFO underrun";
+ return;
+ }
- intel_fbc_update_state_cache(crtc, crtc_state, plane_state);
-
- /* FIXME crtc_state->enable_fbc lies :( */
- if (!cache->plane.visible)
- goto out;
-
- if (intel_fbc_alloc_cfb(dev_priv,
- intel_fbc_cfb_size(dev_priv, cache), min_limit)) {
- cache->plane.visible = false;
+ if (intel_fbc_alloc_cfb(fbc, intel_fbc_cfb_size(plane_state),
+ intel_fbc_min_limit(plane_state))) {
fbc->no_fbc_reason = "not enough stolen memory";
- goto out;
+ return;
}
- drm_dbg_kms(&dev_priv->drm, "Enabling FBC on pipe %c\n",
- pipe_name(crtc->pipe));
+ drm_dbg_kms(&i915->drm, "Enabling FBC on [PLANE:%d:%s]\n",
+ plane->base.base.id, plane->base.name);
fbc->no_fbc_reason = "FBC enabled but not active yet\n";
- fbc->crtc = crtc;
+ intel_fbc_update_state(state, crtc, plane);
- intel_fbc_program_cfb(dev_priv);
-out:
- mutex_unlock(&fbc->lock);
+ intel_fbc_program_cfb(fbc);
}
/**
@@ -1375,114 +1431,122 @@ static void intel_fbc_enable(struct intel_atomic_state *state,
*/
void intel_fbc_disable(struct intel_crtc *crtc)
{
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
- struct intel_plane *plane = to_intel_plane(crtc->base.primary);
- struct intel_fbc *fbc = &dev_priv->fbc;
+ struct drm_i915_private *i915 = to_i915(crtc->base.dev);
+ struct intel_plane *plane;
- if (!plane->has_fbc)
- return;
+ for_each_intel_plane(&i915->drm, plane) {
+ struct intel_fbc *fbc = plane->fbc;
- mutex_lock(&fbc->lock);
- if (fbc->crtc == crtc)
- __intel_fbc_disable(dev_priv);
- mutex_unlock(&fbc->lock);
+ if (!fbc || plane->pipe != crtc->pipe)
+ continue;
+
+ mutex_lock(&fbc->lock);
+ if (fbc->state.plane == plane)
+ __intel_fbc_disable(fbc);
+ mutex_unlock(&fbc->lock);
+ }
}
-/**
- * intel_fbc_update: enable/disable FBC on the CRTC
- * @state: atomic state
- * @crtc: the CRTC
- *
- * This function checks if the given CRTC was chosen for FBC, then enables it if
- * possible. Notice that it doesn't activate FBC. It is valid to call
- * intel_fbc_update multiple times for the same pipe without an
- * intel_fbc_disable in the middle.
- */
void intel_fbc_update(struct intel_atomic_state *state,
struct intel_crtc *crtc)
{
const struct intel_crtc_state *crtc_state =
intel_atomic_get_new_crtc_state(state, crtc);
+ const struct intel_plane_state *plane_state;
+ struct intel_plane *plane;
+ int i;
- if (crtc_state->update_pipe && !crtc_state->enable_fbc)
- intel_fbc_disable(crtc);
- else
- intel_fbc_enable(state, crtc);
+ for_each_new_intel_plane_in_state(state, plane, plane_state, i) {
+ struct intel_fbc *fbc = plane->fbc;
+
+ if (!fbc || plane->pipe != crtc->pipe)
+ continue;
+
+ mutex_lock(&fbc->lock);
+
+ if (crtc_state->update_pipe && plane_state->no_fbc_reason) {
+ if (fbc->state.plane == plane)
+ __intel_fbc_disable(fbc);
+ } else {
+ __intel_fbc_enable(state, crtc, plane);
+ }
+
+ mutex_unlock(&fbc->lock);
+ }
}
/**
* intel_fbc_global_disable - globally disable FBC
- * @dev_priv: i915 device instance
+ * @i915: i915 device instance
*
* This function disables FBC regardless of which CRTC is associated with it.
*/
-void intel_fbc_global_disable(struct drm_i915_private *dev_priv)
+void intel_fbc_global_disable(struct drm_i915_private *i915)
{
- struct intel_fbc *fbc = &dev_priv->fbc;
+ struct intel_fbc *fbc = i915->fbc;
- if (!HAS_FBC(dev_priv))
+ if (!fbc)
return;
mutex_lock(&fbc->lock);
- if (fbc->crtc) {
- drm_WARN_ON(&dev_priv->drm, fbc->crtc->active);
- __intel_fbc_disable(dev_priv);
- }
+ if (fbc->state.plane)
+ __intel_fbc_disable(fbc);
mutex_unlock(&fbc->lock);
}
static void intel_fbc_underrun_work_fn(struct work_struct *work)
{
- struct drm_i915_private *dev_priv =
- container_of(work, struct drm_i915_private, fbc.underrun_work);
- struct intel_fbc *fbc = &dev_priv->fbc;
+ struct intel_fbc *fbc = container_of(work, typeof(*fbc), underrun_work);
+ struct drm_i915_private *i915 = fbc->i915;
mutex_lock(&fbc->lock);
/* Maybe we were scheduled twice. */
- if (fbc->underrun_detected || !fbc->crtc)
+ if (fbc->underrun_detected || !fbc->state.plane)
goto out;
- drm_dbg_kms(&dev_priv->drm, "Disabling FBC due to FIFO underrun.\n");
+ drm_dbg_kms(&i915->drm, "Disabling FBC due to FIFO underrun.\n");
fbc->underrun_detected = true;
- intel_fbc_deactivate(dev_priv, "FIFO underrun");
+ intel_fbc_deactivate(fbc, "FIFO underrun");
+ if (!fbc->flip_pending)
+ intel_crtc_wait_for_next_vblank(intel_crtc_for_pipe(i915, fbc->state.plane->pipe));
+ __intel_fbc_disable(fbc);
out:
mutex_unlock(&fbc->lock);
}
/*
* intel_fbc_reset_underrun - reset FBC fifo underrun status.
- * @dev_priv: i915 device instance
+ * @i915: the i915 device
*
* See intel_fbc_handle_fifo_underrun_irq(). For automated testing we
* want to re-enable FBC after an underrun to increase test coverage.
*/
-int intel_fbc_reset_underrun(struct drm_i915_private *dev_priv)
+void intel_fbc_reset_underrun(struct drm_i915_private *i915)
{
- int ret;
+ struct intel_fbc *fbc = i915->fbc;
- cancel_work_sync(&dev_priv->fbc.underrun_work);
+ if (!fbc)
+ return;
- ret = mutex_lock_interruptible(&dev_priv->fbc.lock);
- if (ret)
- return ret;
+ cancel_work_sync(&fbc->underrun_work);
- if (dev_priv->fbc.underrun_detected) {
- drm_dbg_kms(&dev_priv->drm,
+ mutex_lock(&fbc->lock);
+
+ if (fbc->underrun_detected) {
+ drm_dbg_kms(&i915->drm,
"Re-allowing FBC after fifo underrun\n");
- dev_priv->fbc.no_fbc_reason = "FIFO underrun cleared";
+ fbc->no_fbc_reason = "FIFO underrun cleared";
}
- dev_priv->fbc.underrun_detected = false;
- mutex_unlock(&dev_priv->fbc.lock);
-
- return 0;
+ fbc->underrun_detected = false;
+ mutex_unlock(&fbc->lock);
}
/**
* intel_fbc_handle_fifo_underrun_irq - disable FBC when we get a FIFO underrun
- * @dev_priv: i915 device instance
+ * @i915: i915 device
*
* Without FBC, most underruns are harmless and don't really cause too many
* problems, except for an annoying message on dmesg. With FBC, underruns can
@@ -1494,11 +1558,11 @@ int intel_fbc_reset_underrun(struct drm_i915_private *dev_priv)
*
* This function is called from the IRQ handler.
*/
-void intel_fbc_handle_fifo_underrun_irq(struct drm_i915_private *dev_priv)
+void intel_fbc_handle_fifo_underrun_irq(struct drm_i915_private *i915)
{
- struct intel_fbc *fbc = &dev_priv->fbc;
+ struct intel_fbc *fbc = i915->fbc;
- if (!HAS_FBC(dev_priv))
+ if (!fbc)
return;
/* There's no guarantee that underrun_detected won't be set to true
@@ -1522,26 +1586,26 @@ void intel_fbc_handle_fifo_underrun_irq(struct drm_i915_private *dev_priv)
* space to change the value during runtime without sanitizing it again. IGT
* relies on being able to change i915.enable_fbc at runtime.
*/
-static int intel_sanitize_fbc_option(struct drm_i915_private *dev_priv)
+static int intel_sanitize_fbc_option(struct drm_i915_private *i915)
{
- if (dev_priv->params.enable_fbc >= 0)
- return !!dev_priv->params.enable_fbc;
+ if (i915->params.enable_fbc >= 0)
+ return !!i915->params.enable_fbc;
- if (!HAS_FBC(dev_priv))
+ if (!HAS_FBC(i915))
return 0;
- if (IS_BROADWELL(dev_priv) || DISPLAY_VER(dev_priv) >= 9)
+ if (IS_BROADWELL(i915) || DISPLAY_VER(i915) >= 9)
return 1;
return 0;
}
-static bool need_fbc_vtd_wa(struct drm_i915_private *dev_priv)
+static bool need_fbc_vtd_wa(struct drm_i915_private *i915)
{
/* WaFbcTurnOffFbcWhenHyperVisorIsUsed:skl,bxt */
- if (intel_vtd_active() &&
- (IS_SKYLAKE(dev_priv) || IS_BROXTON(dev_priv))) {
- drm_info(&dev_priv->drm,
+ if (intel_vtd_active(i915) &&
+ (IS_SKYLAKE(i915) || IS_BROXTON(i915))) {
+ drm_info(&i915->drm,
"Disabling framebuffer compression (FBC) to prevent screen flicker with VT-d enabled\n");
return true;
}
@@ -1549,38 +1613,171 @@ static bool need_fbc_vtd_wa(struct drm_i915_private *dev_priv)
return false;
}
+void intel_fbc_add_plane(struct intel_fbc *fbc, struct intel_plane *plane)
+{
+ if (!fbc)
+ return;
+
+ plane->fbc = fbc;
+ fbc->possible_framebuffer_bits |= plane->frontbuffer_bit;
+}
+
+static struct intel_fbc *intel_fbc_create(struct drm_i915_private *i915)
+{
+ struct intel_fbc *fbc;
+
+ fbc = kzalloc(sizeof(*fbc), GFP_KERNEL);
+ if (!fbc)
+ return NULL;
+
+ fbc->i915 = i915;
+ INIT_WORK(&fbc->underrun_work, intel_fbc_underrun_work_fn);
+ mutex_init(&fbc->lock);
+
+ if (DISPLAY_VER(i915) >= 7)
+ fbc->funcs = &ivb_fbc_funcs;
+ else if (DISPLAY_VER(i915) == 6)
+ fbc->funcs = &snb_fbc_funcs;
+ else if (DISPLAY_VER(i915) == 5)
+ fbc->funcs = &ilk_fbc_funcs;
+ else if (IS_G4X(i915))
+ fbc->funcs = &g4x_fbc_funcs;
+ else if (DISPLAY_VER(i915) == 4)
+ fbc->funcs = &i965_fbc_funcs;
+ else
+ fbc->funcs = &i8xx_fbc_funcs;
+
+ return fbc;
+}
+
/**
* intel_fbc_init - Initialize FBC
- * @dev_priv: the i915 device
+ * @i915: the i915 device
*
* This function might be called during PM init process.
*/
-void intel_fbc_init(struct drm_i915_private *dev_priv)
+void intel_fbc_init(struct drm_i915_private *i915)
{
- struct intel_fbc *fbc = &dev_priv->fbc;
+ struct intel_fbc *fbc;
- INIT_WORK(&fbc->underrun_work, intel_fbc_underrun_work_fn);
- mutex_init(&fbc->lock);
- fbc->active = false;
+ if (!drm_mm_initialized(&i915->mm.stolen))
+ mkwrite_device_info(i915)->display.has_fbc = false;
- if (!drm_mm_initialized(&dev_priv->mm.stolen))
- mkwrite_device_info(dev_priv)->display.has_fbc = false;
+ if (need_fbc_vtd_wa(i915))
+ mkwrite_device_info(i915)->display.has_fbc = false;
- if (need_fbc_vtd_wa(dev_priv))
- mkwrite_device_info(dev_priv)->display.has_fbc = false;
+ i915->params.enable_fbc = intel_sanitize_fbc_option(i915);
+ drm_dbg_kms(&i915->drm, "Sanitized enable_fbc value: %d\n",
+ i915->params.enable_fbc);
- dev_priv->params.enable_fbc = intel_sanitize_fbc_option(dev_priv);
- drm_dbg_kms(&dev_priv->drm, "Sanitized enable_fbc value: %d\n",
- dev_priv->params.enable_fbc);
-
- if (!HAS_FBC(dev_priv)) {
- fbc->no_fbc_reason = "unsupported by this chipset";
+ if (!HAS_FBC(i915))
return;
- }
+
+ fbc = intel_fbc_create(i915);
+ if (!fbc)
+ return;
/* We still don't have any sort of hardware state readout for FBC, so
* deactivate it in case the BIOS activated it to make sure software
* matches the hardware state. */
- if (intel_fbc_hw_is_active(dev_priv))
- intel_fbc_hw_deactivate(dev_priv);
+ if (intel_fbc_hw_is_active(fbc))
+ intel_fbc_hw_deactivate(fbc);
+
+ i915->fbc = fbc;
+}
+
+static int intel_fbc_debugfs_status_show(struct seq_file *m, void *unused)
+{
+ struct intel_fbc *fbc = m->private;
+ struct drm_i915_private *i915 = fbc->i915;
+ struct intel_plane *plane;
+ intel_wakeref_t wakeref;
+
+ drm_modeset_lock_all(&i915->drm);
+
+ wakeref = intel_runtime_pm_get(&i915->runtime_pm);
+ mutex_lock(&fbc->lock);
+
+ if (fbc->active) {
+ seq_puts(m, "FBC enabled\n");
+ seq_printf(m, "Compressing: %s\n",
+ yesno(intel_fbc_is_compressing(fbc)));
+ } else {
+ seq_printf(m, "FBC disabled: %s\n", fbc->no_fbc_reason);
+ }
+
+ for_each_intel_plane(&i915->drm, plane) {
+ const struct intel_plane_state *plane_state =
+ to_intel_plane_state(plane->base.state);
+
+ if (plane->fbc != fbc)
+ continue;
+
+ seq_printf(m, "%c [PLANE:%d:%s]: %s\n",
+ fbc->state.plane == plane ? '*' : ' ',
+ plane->base.base.id, plane->base.name,
+ plane_state->no_fbc_reason ?: "FBC possible");
+ }
+
+ mutex_unlock(&fbc->lock);
+ intel_runtime_pm_put(&i915->runtime_pm, wakeref);
+
+ drm_modeset_unlock_all(&i915->drm);
+
+ return 0;
+}
+
+DEFINE_SHOW_ATTRIBUTE(intel_fbc_debugfs_status);
+
+static int intel_fbc_debugfs_false_color_get(void *data, u64 *val)
+{
+ struct intel_fbc *fbc = data;
+
+ *val = fbc->false_color;
+
+ return 0;
+}
+
+static int intel_fbc_debugfs_false_color_set(void *data, u64 val)
+{
+ struct intel_fbc *fbc = data;
+
+ mutex_lock(&fbc->lock);
+
+ fbc->false_color = val;
+
+ if (fbc->active)
+ fbc->funcs->set_false_color(fbc, fbc->false_color);
+
+ mutex_unlock(&fbc->lock);
+
+ return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(intel_fbc_debugfs_false_color_fops,
+ intel_fbc_debugfs_false_color_get,
+ intel_fbc_debugfs_false_color_set,
+ "%llu\n");
+
+static void intel_fbc_debugfs_add(struct intel_fbc *fbc)
+{
+ struct drm_i915_private *i915 = fbc->i915;
+ struct drm_minor *minor = i915->drm.primary;
+
+ debugfs_create_file("i915_fbc_status", 0444,
+ minor->debugfs_root, fbc,
+ &intel_fbc_debugfs_status_fops);
+
+ if (fbc->funcs->set_false_color)
+ debugfs_create_file("i915_fbc_false_color", 0644,
+ minor->debugfs_root, fbc,
+ &intel_fbc_debugfs_false_color_fops);
+}
+
+void intel_fbc_debugfs_register(struct drm_i915_private *i915)
+{
+ struct intel_fbc *fbc = i915->fbc;
+
+ if (fbc)
+ intel_fbc_debugfs_add(fbc);
}
diff --git a/drivers/gpu/drm/i915/display/intel_fbc.h b/drivers/gpu/drm/i915/display/intel_fbc.h
index b97d908..07ad041 100644
--- a/drivers/gpu/drm/i915/display/intel_fbc.h
+++ b/drivers/gpu/drm/i915/display/intel_fbc.h
@@ -8,22 +8,22 @@
#include <linux/types.h>
-#include "intel_frontbuffer.h"
-
+enum fb_op_origin;
struct drm_i915_private;
struct intel_atomic_state;
struct intel_crtc;
struct intel_crtc_state;
+struct intel_fbc;
+struct intel_plane;
struct intel_plane_state;
-void intel_fbc_choose_crtc(struct drm_i915_private *dev_priv,
- struct intel_atomic_state *state);
-bool intel_fbc_is_active(struct drm_i915_private *dev_priv);
+int intel_fbc_atomic_check(struct intel_atomic_state *state);
bool intel_fbc_pre_update(struct intel_atomic_state *state,
struct intel_crtc *crtc);
void intel_fbc_post_update(struct intel_atomic_state *state,
struct intel_crtc *crtc);
void intel_fbc_init(struct drm_i915_private *dev_priv);
+void intel_fbc_cleanup(struct drm_i915_private *dev_priv);
void intel_fbc_update(struct intel_atomic_state *state,
struct intel_crtc *crtc);
void intel_fbc_disable(struct intel_crtc *crtc);
@@ -33,8 +33,9 @@ void intel_fbc_invalidate(struct drm_i915_private *dev_priv,
enum fb_op_origin origin);
void intel_fbc_flush(struct drm_i915_private *dev_priv,
unsigned int frontbuffer_bits, enum fb_op_origin origin);
-void intel_fbc_cleanup_cfb(struct drm_i915_private *dev_priv);
-void intel_fbc_handle_fifo_underrun_irq(struct drm_i915_private *dev_priv);
-int intel_fbc_reset_underrun(struct drm_i915_private *dev_priv);
+void intel_fbc_add_plane(struct intel_fbc *fbc, struct intel_plane *plane);
+void intel_fbc_handle_fifo_underrun_irq(struct drm_i915_private *i915);
+void intel_fbc_reset_underrun(struct drm_i915_private *i915);
+void intel_fbc_debugfs_register(struct drm_i915_private *i915);
#endif /* __INTEL_FBC_H__ */
diff --git a/drivers/gpu/drm/i915/display/intel_fdi.c b/drivers/gpu/drm/i915/display/intel_fdi.c
index dd2cf0c..3d6e229 100644
--- a/drivers/gpu/drm/i915/display/intel_fdi.c
+++ b/drivers/gpu/drm/i915/display/intel_fdi.c
@@ -4,11 +4,11 @@
*/
#include "intel_atomic.h"
+#include "intel_crtc.h"
#include "intel_ddi.h"
#include "intel_de.h"
#include "intel_display_types.h"
#include "intel_fdi.h"
-#include "intel_sbi.h"
static void assert_fdi_tx(struct drm_i915_private *dev_priv,
enum pipe pipe, bool state)
@@ -158,7 +158,7 @@ static int ilk_check_fdi_lanes(struct drm_device *dev, enum pipe pipe,
if (pipe_config->fdi_lanes <= 2)
return 0;
- other_crtc = intel_get_crtc_for_pipe(dev_priv, PIPE_C);
+ other_crtc = intel_crtc_for_pipe(dev_priv, PIPE_C);
other_crtc_state =
intel_atomic_get_crtc_state(state, other_crtc);
if (IS_ERR(other_crtc_state))
@@ -179,7 +179,7 @@ static int ilk_check_fdi_lanes(struct drm_device *dev, enum pipe pipe,
return -EINVAL;
}
- other_crtc = intel_get_crtc_for_pipe(dev_priv, PIPE_B);
+ other_crtc = intel_crtc_for_pipe(dev_priv, PIPE_B);
other_crtc_state =
intel_atomic_get_crtc_state(state, other_crtc);
if (IS_ERR(other_crtc_state))
@@ -887,6 +887,43 @@ void hsw_fdi_link_train(struct intel_encoder *encoder,
DP_TP_CTL_ENABLE);
}
+void hsw_fdi_disable(struct intel_encoder *encoder)
+{
+ struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
+ u32 val;
+
+ /*
+ * Bspec lists this as both step 13 (before DDI_BUF_CTL disable)
+ * and step 18 (after clearing PORT_CLK_SEL). Based on a BUN,
+ * step 13 is the correct place for it. Step 18 is where it was
+ * originally before the BUN.
+ */
+ val = intel_de_read(dev_priv, FDI_RX_CTL(PIPE_A));
+ val &= ~FDI_RX_ENABLE;
+ intel_de_write(dev_priv, FDI_RX_CTL(PIPE_A), val);
+
+ val = intel_de_read(dev_priv, DDI_BUF_CTL(PORT_E));
+ val &= ~DDI_BUF_CTL_ENABLE;
+ intel_de_write(dev_priv, DDI_BUF_CTL(PORT_E), val);
+
+ intel_wait_ddi_buf_idle(dev_priv, PORT_E);
+
+ intel_ddi_disable_clock(encoder);
+
+ val = intel_de_read(dev_priv, FDI_RX_MISC(PIPE_A));
+ val &= ~(FDI_RX_PWRDN_LANE1_MASK | FDI_RX_PWRDN_LANE0_MASK);
+ val |= FDI_RX_PWRDN_LANE1_VAL(2) | FDI_RX_PWRDN_LANE0_VAL(2);
+ intel_de_write(dev_priv, FDI_RX_MISC(PIPE_A), val);
+
+ val = intel_de_read(dev_priv, FDI_RX_CTL(PIPE_A));
+ val &= ~FDI_PCDCLK;
+ intel_de_write(dev_priv, FDI_RX_CTL(PIPE_A), val);
+
+ val = intel_de_read(dev_priv, FDI_RX_CTL(PIPE_A));
+ val &= ~FDI_RX_PLL_ENABLE;
+ intel_de_write(dev_priv, FDI_RX_CTL(PIPE_A), val);
+}
+
void ilk_fdi_pll_enable(const struct intel_crtc_state *crtc_state)
{
struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
@@ -1006,104 +1043,6 @@ void ilk_fdi_disable(struct intel_crtc *crtc)
udelay(100);
}
-static void lpt_fdi_reset_mphy(struct drm_i915_private *dev_priv)
-{
- u32 tmp;
-
- tmp = intel_de_read(dev_priv, SOUTH_CHICKEN2);
- tmp |= FDI_MPHY_IOSFSB_RESET_CTL;
- intel_de_write(dev_priv, SOUTH_CHICKEN2, tmp);
-
- if (wait_for_us(intel_de_read(dev_priv, SOUTH_CHICKEN2) &
- FDI_MPHY_IOSFSB_RESET_STATUS, 100))
- drm_err(&dev_priv->drm, "FDI mPHY reset assert timeout\n");
-
- tmp = intel_de_read(dev_priv, SOUTH_CHICKEN2);
- tmp &= ~FDI_MPHY_IOSFSB_RESET_CTL;
- intel_de_write(dev_priv, SOUTH_CHICKEN2, tmp);
-
- if (wait_for_us((intel_de_read(dev_priv, SOUTH_CHICKEN2) &
- FDI_MPHY_IOSFSB_RESET_STATUS) == 0, 100))
- drm_err(&dev_priv->drm, "FDI mPHY reset de-assert timeout\n");
-}
-
-/* WaMPhyProgramming:hsw */
-void lpt_fdi_program_mphy(struct drm_i915_private *dev_priv)
-{
- u32 tmp;
-
- lpt_fdi_reset_mphy(dev_priv);
-
- tmp = intel_sbi_read(dev_priv, 0x8008, SBI_MPHY);
- tmp &= ~(0xFF << 24);
- tmp |= (0x12 << 24);
- intel_sbi_write(dev_priv, 0x8008, tmp, SBI_MPHY);
-
- tmp = intel_sbi_read(dev_priv, 0x2008, SBI_MPHY);
- tmp |= (1 << 11);
- intel_sbi_write(dev_priv, 0x2008, tmp, SBI_MPHY);
-
- tmp = intel_sbi_read(dev_priv, 0x2108, SBI_MPHY);
- tmp |= (1 << 11);
- intel_sbi_write(dev_priv, 0x2108, tmp, SBI_MPHY);
-
- tmp = intel_sbi_read(dev_priv, 0x206C, SBI_MPHY);
- tmp |= (1 << 24) | (1 << 21) | (1 << 18);
- intel_sbi_write(dev_priv, 0x206C, tmp, SBI_MPHY);
-
- tmp = intel_sbi_read(dev_priv, 0x216C, SBI_MPHY);
- tmp |= (1 << 24) | (1 << 21) | (1 << 18);
- intel_sbi_write(dev_priv, 0x216C, tmp, SBI_MPHY);
-
- tmp = intel_sbi_read(dev_priv, 0x2080, SBI_MPHY);
- tmp &= ~(7 << 13);
- tmp |= (5 << 13);
- intel_sbi_write(dev_priv, 0x2080, tmp, SBI_MPHY);
-
- tmp = intel_sbi_read(dev_priv, 0x2180, SBI_MPHY);
- tmp &= ~(7 << 13);
- tmp |= (5 << 13);
- intel_sbi_write(dev_priv, 0x2180, tmp, SBI_MPHY);
-
- tmp = intel_sbi_read(dev_priv, 0x208C, SBI_MPHY);
- tmp &= ~0xFF;
- tmp |= 0x1C;
- intel_sbi_write(dev_priv, 0x208C, tmp, SBI_MPHY);
-
- tmp = intel_sbi_read(dev_priv, 0x218C, SBI_MPHY);
- tmp &= ~0xFF;
- tmp |= 0x1C;
- intel_sbi_write(dev_priv, 0x218C, tmp, SBI_MPHY);
-
- tmp = intel_sbi_read(dev_priv, 0x2098, SBI_MPHY);
- tmp &= ~(0xFF << 16);
- tmp |= (0x1C << 16);
- intel_sbi_write(dev_priv, 0x2098, tmp, SBI_MPHY);
-
- tmp = intel_sbi_read(dev_priv, 0x2198, SBI_MPHY);
- tmp &= ~(0xFF << 16);
- tmp |= (0x1C << 16);
- intel_sbi_write(dev_priv, 0x2198, tmp, SBI_MPHY);
-
- tmp = intel_sbi_read(dev_priv, 0x20C4, SBI_MPHY);
- tmp |= (1 << 27);
- intel_sbi_write(dev_priv, 0x20C4, tmp, SBI_MPHY);
-
- tmp = intel_sbi_read(dev_priv, 0x21C4, SBI_MPHY);
- tmp |= (1 << 27);
- intel_sbi_write(dev_priv, 0x21C4, tmp, SBI_MPHY);
-
- tmp = intel_sbi_read(dev_priv, 0x20EC, SBI_MPHY);
- tmp &= ~(0xF << 28);
- tmp |= (4 << 28);
- intel_sbi_write(dev_priv, 0x20EC, tmp, SBI_MPHY);
-
- tmp = intel_sbi_read(dev_priv, 0x21EC, SBI_MPHY);
- tmp &= ~(0xF << 28);
- tmp |= (4 << 28);
- intel_sbi_write(dev_priv, 0x21EC, tmp, SBI_MPHY);
-}
-
static const struct intel_fdi_funcs ilk_funcs = {
.fdi_link_train = ilk_fdi_link_train,
};
diff --git a/drivers/gpu/drm/i915/display/intel_fdi.h b/drivers/gpu/drm/i915/display/intel_fdi.h
index 640d658..1cdb861 100644
--- a/drivers/gpu/drm/i915/display/intel_fdi.h
+++ b/drivers/gpu/drm/i915/display/intel_fdi.h
@@ -23,8 +23,8 @@ void ilk_fdi_pll_enable(const struct intel_crtc_state *crtc_state);
void intel_fdi_init_hook(struct drm_i915_private *dev_priv);
void hsw_fdi_link_train(struct intel_encoder *encoder,
const struct intel_crtc_state *crtc_state);
+void hsw_fdi_disable(struct intel_encoder *encoder);
void intel_fdi_pll_freq_update(struct drm_i915_private *i915);
-void lpt_fdi_program_mphy(struct drm_i915_private *i915);
void intel_fdi_link_train(struct intel_crtc *crtc,
const struct intel_crtc_state *crtc_state);
diff --git a/drivers/gpu/drm/i915/display/intel_fifo_underrun.c b/drivers/gpu/drm/i915/display/intel_fifo_underrun.c
index eb84196..d636d21 100644
--- a/drivers/gpu/drm/i915/display/intel_fifo_underrun.c
+++ b/drivers/gpu/drm/i915/display/intel_fifo_underrun.c
@@ -26,8 +26,8 @@
*/
#include "i915_drv.h"
-#include "i915_trace.h"
#include "intel_de.h"
+#include "intel_display_trace.h"
#include "intel_display_types.h"
#include "intel_fbc.h"
#include "intel_fifo_underrun.h"
@@ -61,7 +61,7 @@ static bool ivb_can_enable_err_int(struct drm_device *dev)
lockdep_assert_held(&dev_priv->irq_lock);
for_each_pipe(dev_priv, pipe) {
- crtc = intel_get_crtc_for_pipe(dev_priv, pipe);
+ crtc = intel_crtc_for_pipe(dev_priv, pipe);
if (crtc->cpu_fifo_underrun_disabled)
return false;
@@ -79,7 +79,7 @@ static bool cpt_can_enable_serr_int(struct drm_device *dev)
lockdep_assert_held(&dev_priv->irq_lock);
for_each_pipe(dev_priv, pipe) {
- crtc = intel_get_crtc_for_pipe(dev_priv, pipe);
+ crtc = intel_crtc_for_pipe(dev_priv, pipe);
if (crtc->pch_fifo_underrun_disabled)
return false;
@@ -279,7 +279,7 @@ static bool __intel_set_cpu_fifo_underrun_reporting(struct drm_device *dev,
enum pipe pipe, bool enable)
{
struct drm_i915_private *dev_priv = to_i915(dev);
- struct intel_crtc *crtc = intel_get_crtc_for_pipe(dev_priv, pipe);
+ struct intel_crtc *crtc = intel_crtc_for_pipe(dev_priv, pipe);
bool old;
lockdep_assert_held(&dev_priv->irq_lock);
@@ -348,7 +348,7 @@ bool intel_set_pch_fifo_underrun_reporting(struct drm_i915_private *dev_priv,
bool enable)
{
struct intel_crtc *crtc =
- intel_get_crtc_for_pipe(dev_priv, pch_transcoder);
+ intel_crtc_for_pipe(dev_priv, pch_transcoder);
unsigned long flags;
bool old;
@@ -391,7 +391,7 @@ bool intel_set_pch_fifo_underrun_reporting(struct drm_i915_private *dev_priv,
void intel_cpu_fifo_underrun_irq_handler(struct drm_i915_private *dev_priv,
enum pipe pipe)
{
- struct intel_crtc *crtc = intel_get_crtc_for_pipe(dev_priv, pipe);
+ struct intel_crtc *crtc = intel_crtc_for_pipe(dev_priv, pipe);
u32 underruns = 0;
/* We may be called too early in init, thanks BIOS! */
diff --git a/drivers/gpu/drm/i915/display/intel_frontbuffer.c b/drivers/gpu/drm/i915/display/intel_frontbuffer.c
index 0492446..791248f 100644
--- a/drivers/gpu/drm/i915/display/intel_frontbuffer.c
+++ b/drivers/gpu/drm/i915/display/intel_frontbuffer.c
@@ -55,14 +55,13 @@
* cancelled as soon as busyness is detected.
*/
-#include "display/intel_dp.h"
-
#include "i915_drv.h"
-#include "i915_trace.h"
+#include "intel_display_trace.h"
#include "intel_display_types.h"
+#include "intel_dp.h"
+#include "intel_drrs.h"
#include "intel_fbc.h"
#include "intel_frontbuffer.h"
-#include "intel_drrs.h"
#include "intel_psr.h"
/**
diff --git a/drivers/gpu/drm/i915/display/intel_frontbuffer.h b/drivers/gpu/drm/i915/display/intel_frontbuffer.h
index a88441e..ff0c37b 100644
--- a/drivers/gpu/drm/i915/display/intel_frontbuffer.h
+++ b/drivers/gpu/drm/i915/display/intel_frontbuffer.h
@@ -28,7 +28,7 @@
#include <linux/kref.h>
#include "gem/i915_gem_object_types.h"
-#include "i915_active.h"
+#include "i915_active_types.h"
struct drm_i915_private;
diff --git a/drivers/gpu/drm/i915/display/intel_gmbus.c b/drivers/gpu/drm/i915/display/intel_gmbus.c
index ceb1bf8..3b8b841 100644
--- a/drivers/gpu/drm/i915/display/intel_gmbus.c
+++ b/drivers/gpu/drm/i915/display/intel_gmbus.c
@@ -334,6 +334,15 @@ intel_gpio_setup(struct intel_gmbus *bus, unsigned int pin)
algo->data = bus;
}
+static bool has_gmbus_irq(struct drm_i915_private *i915)
+{
+ /*
+ * encoder->shutdown() may want to use GMBUS
+ * after irqs have already been disabled.
+ */
+ return HAS_GMBUS_IRQ(i915) && intel_irqs_enabled(i915);
+}
+
static int gmbus_wait(struct drm_i915_private *dev_priv, u32 status, u32 irq_en)
{
DEFINE_WAIT(wait);
@@ -344,7 +353,7 @@ static int gmbus_wait(struct drm_i915_private *dev_priv, u32 status, u32 irq_en)
* we also need to check for NAKs besides the hw ready/idle signal, we
* need to wake up periodically and check that ourselves.
*/
- if (!HAS_GMBUS_IRQ(dev_priv))
+ if (!has_gmbus_irq(dev_priv))
irq_en = 0;
add_wait_queue(&dev_priv->gmbus_wait_queue, &wait);
@@ -375,7 +384,7 @@ gmbus_wait_idle(struct drm_i915_private *dev_priv)
/* Important: The hw handles only the first bit, so set only one! */
irq_enable = 0;
- if (HAS_GMBUS_IRQ(dev_priv))
+ if (has_gmbus_irq(dev_priv))
irq_enable = GMBUS_IDLE_EN;
add_wait_queue(&dev_priv->gmbus_wait_queue, &wait);
diff --git a/drivers/gpu/drm/i915/display/intel_hdmi.c b/drivers/gpu/drm/i915/display/intel_hdmi.c
index 371736b..3b5b9e7 100644
--- a/drivers/gpu/drm/i915/display/intel_hdmi.c
+++ b/drivers/gpu/drm/i915/display/intel_hdmi.c
@@ -1800,6 +1800,11 @@ static bool intel_has_hdmi_sink(struct intel_hdmi *hdmi,
READ_ONCE(to_intel_digital_connector_state(conn_state)->force_audio) != HDMI_AUDIO_OFF_DVI;
}
+static bool intel_hdmi_is_ycbcr420(const struct intel_crtc_state *crtc_state)
+{
+ return crtc_state->output_format == INTEL_OUTPUT_FORMAT_YCBCR420;
+}
+
static int hdmi_port_clock_limit(struct intel_hdmi *hdmi,
bool respect_downstream_limits,
bool has_hdmi_sink)
@@ -1864,8 +1869,12 @@ hdmi_port_clock_valid(struct intel_hdmi *hdmi,
return MODE_OK;
}
-static int intel_hdmi_port_clock(int clock, int bpc)
+static int intel_hdmi_tmds_clock(int clock, int bpc, bool ycbcr420_output)
{
+ /* YCBCR420 TMDS rate requirement is half the pixel clock */
+ if (ycbcr420_output)
+ clock /= 2;
+
/*
* Need to adjust the port link by:
* 1.5x for 12bpc
@@ -1874,18 +1883,29 @@ static int intel_hdmi_port_clock(int clock, int bpc)
return clock * bpc / 8;
}
-static bool intel_hdmi_bpc_possible(struct drm_connector *connector,
- int bpc, bool has_hdmi_sink, bool ycbcr420_output)
+static bool intel_hdmi_source_bpc_possible(struct drm_i915_private *i915, int bpc)
{
- struct drm_i915_private *i915 = to_i915(connector->dev);
+ switch (bpc) {
+ case 12:
+ return !HAS_GMCH(i915);
+ case 10:
+ return DISPLAY_VER(i915) >= 11;
+ case 8:
+ return true;
+ default:
+ MISSING_CASE(bpc);
+ return false;
+ }
+}
+
+static bool intel_hdmi_sink_bpc_possible(struct drm_connector *connector,
+ int bpc, bool has_hdmi_sink, bool ycbcr420_output)
+{
const struct drm_display_info *info = &connector->display_info;
const struct drm_hdmi_info *hdmi = &info->hdmi;
switch (bpc) {
case 12:
- if (HAS_GMCH(i915))
- return false;
-
if (!has_hdmi_sink)
return false;
@@ -1894,9 +1914,6 @@ static bool intel_hdmi_bpc_possible(struct drm_connector *connector,
else
return info->edid_hdmi_dc_modes & DRM_EDID_HDMI_DC_36;
case 10:
- if (DISPLAY_VER(i915) < 11)
- return false;
-
if (!has_hdmi_sink)
return false;
@@ -1916,26 +1933,26 @@ static enum drm_mode_status
intel_hdmi_mode_clock_valid(struct drm_connector *connector, int clock,
bool has_hdmi_sink, bool ycbcr420_output)
{
+ struct drm_i915_private *i915 = to_i915(connector->dev);
struct intel_hdmi *hdmi = intel_attached_hdmi(to_intel_connector(connector));
enum drm_mode_status status;
- if (ycbcr420_output)
- clock /= 2;
-
/* check if we can do 8bpc */
- status = hdmi_port_clock_valid(hdmi, intel_hdmi_port_clock(clock, 8),
+ status = hdmi_port_clock_valid(hdmi, intel_hdmi_tmds_clock(clock, 8, ycbcr420_output),
true, has_hdmi_sink);
/* if we can't do 8bpc we may still be able to do 12bpc */
if (status != MODE_OK &&
- intel_hdmi_bpc_possible(connector, 12, has_hdmi_sink, ycbcr420_output))
- status = hdmi_port_clock_valid(hdmi, intel_hdmi_port_clock(clock, 12),
+ intel_hdmi_source_bpc_possible(i915, 12) &&
+ intel_hdmi_sink_bpc_possible(connector, 12, has_hdmi_sink, ycbcr420_output))
+ status = hdmi_port_clock_valid(hdmi, intel_hdmi_tmds_clock(clock, 12, ycbcr420_output),
true, has_hdmi_sink);
/* if we can't do 8,12bpc we may still be able to do 10bpc */
if (status != MODE_OK &&
- intel_hdmi_bpc_possible(connector, 10, has_hdmi_sink, ycbcr420_output))
- status = hdmi_port_clock_valid(hdmi, intel_hdmi_port_clock(clock, 10),
+ intel_hdmi_source_bpc_possible(i915, 10) &&
+ intel_hdmi_sink_bpc_possible(connector, 10, has_hdmi_sink, ycbcr420_output))
+ status = hdmi_port_clock_valid(hdmi, intel_hdmi_tmds_clock(clock, 10, ycbcr420_output),
true, has_hdmi_sink);
return status;
@@ -2000,7 +2017,7 @@ bool intel_hdmi_deep_color_possible(const struct intel_crtc_state *crtc_state,
if (connector_state->crtc != crtc_state->uapi.crtc)
continue;
- if (!intel_hdmi_bpc_possible(connector, bpc, has_hdmi_sink, ycbcr420_output))
+ if (!intel_hdmi_sink_bpc_possible(connector, bpc, has_hdmi_sink, ycbcr420_output))
return false;
}
@@ -2015,6 +2032,9 @@ static bool hdmi_deep_color_possible(const struct intel_crtc_state *crtc_state,
const struct drm_display_mode *adjusted_mode =
&crtc_state->hw.adjusted_mode;
+ if (!intel_hdmi_source_bpc_possible(dev_priv, bpc))
+ return false;
+
/*
* HDMI deep color affects the clocks, so it's only possible
* when not cloning with other encoder types.
@@ -2023,7 +2043,7 @@ static bool hdmi_deep_color_possible(const struct intel_crtc_state *crtc_state,
return false;
/* Display Wa_1405510057:icl,ehl */
- if (crtc_state->output_format == INTEL_OUTPUT_FORMAT_YCBCR420 &&
+ if (intel_hdmi_is_ycbcr420(crtc_state) &&
bpc == 10 && DISPLAY_VER(dev_priv) == 11 &&
(adjusted_mode->crtc_hblank_end -
adjusted_mode->crtc_hblank_start) % 8 == 2)
@@ -2031,8 +2051,7 @@ static bool hdmi_deep_color_possible(const struct intel_crtc_state *crtc_state,
return intel_hdmi_deep_color_possible(crtc_state, bpc,
crtc_state->has_hdmi_sink,
- crtc_state->output_format ==
- INTEL_OUTPUT_FORMAT_YCBCR420);
+ intel_hdmi_is_ycbcr420(crtc_state));
}
static int intel_hdmi_compute_bpc(struct intel_encoder *encoder,
@@ -2040,12 +2059,13 @@ static int intel_hdmi_compute_bpc(struct intel_encoder *encoder,
int clock)
{
struct intel_hdmi *intel_hdmi = enc_to_intel_hdmi(encoder);
+ bool ycbcr420_output = intel_hdmi_is_ycbcr420(crtc_state);
int bpc;
for (bpc = 12; bpc >= 10; bpc -= 2) {
if (hdmi_deep_color_possible(crtc_state, bpc) &&
hdmi_port_clock_valid(intel_hdmi,
- intel_hdmi_port_clock(clock, bpc),
+ intel_hdmi_tmds_clock(clock, bpc, ycbcr420_output),
true, crtc_state->has_hdmi_sink) == MODE_OK)
return bpc;
}
@@ -2065,13 +2085,10 @@ static int intel_hdmi_compute_clock(struct intel_encoder *encoder,
if (adjusted_mode->flags & DRM_MODE_FLAG_DBLCLK)
clock *= 2;
- /* YCBCR420 TMDS rate requirement is half the pixel clock */
- if (crtc_state->output_format == INTEL_OUTPUT_FORMAT_YCBCR420)
- clock /= 2;
-
bpc = intel_hdmi_compute_bpc(encoder, crtc_state, clock);
- crtc_state->port_clock = intel_hdmi_port_clock(clock, bpc);
+ crtc_state->port_clock = intel_hdmi_tmds_clock(clock, bpc,
+ intel_hdmi_is_ycbcr420(crtc_state));
/*
* pipe_bpp could already be below 8bpc due to
@@ -2141,34 +2158,44 @@ static bool intel_hdmi_has_audio(struct intel_encoder *encoder,
return intel_conn_state->force_audio == HDMI_AUDIO_ON;
}
+static enum intel_output_format
+intel_hdmi_output_format(struct intel_connector *connector,
+ bool ycbcr_420_output)
+{
+ if (connector->base.ycbcr_420_allowed && ycbcr_420_output)
+ return INTEL_OUTPUT_FORMAT_YCBCR420;
+ else
+ return INTEL_OUTPUT_FORMAT_RGB;
+}
+
static int intel_hdmi_compute_output_format(struct intel_encoder *encoder,
struct intel_crtc_state *crtc_state,
const struct drm_connector_state *conn_state)
{
- struct drm_connector *connector = conn_state->connector;
- struct drm_i915_private *i915 = to_i915(connector->dev);
+ struct intel_connector *connector = to_intel_connector(conn_state->connector);
const struct drm_display_mode *adjusted_mode = &crtc_state->hw.adjusted_mode;
+ const struct drm_display_info *info = &connector->base.display_info;
+ struct drm_i915_private *i915 = to_i915(connector->base.dev);
+ bool ycbcr_420_only = drm_mode_is_420_only(info, adjusted_mode);
int ret;
- bool ycbcr_420_only;
- ycbcr_420_only = drm_mode_is_420_only(&connector->display_info, adjusted_mode);
- if (connector->ycbcr_420_allowed && ycbcr_420_only) {
- crtc_state->output_format = INTEL_OUTPUT_FORMAT_YCBCR420;
- } else {
- if (!connector->ycbcr_420_allowed && ycbcr_420_only)
- drm_dbg_kms(&i915->drm,
- "YCbCr 4:2:0 mode but YCbCr 4:2:0 output not possible. Falling back to RGB.\n");
+ crtc_state->output_format = intel_hdmi_output_format(connector, ycbcr_420_only);
+
+ if (ycbcr_420_only && !intel_hdmi_is_ycbcr420(crtc_state)) {
+ drm_dbg_kms(&i915->drm,
+ "YCbCr 4:2:0 mode but YCbCr 4:2:0 output not possible. Falling back to RGB.\n");
crtc_state->output_format = INTEL_OUTPUT_FORMAT_RGB;
}
ret = intel_hdmi_compute_clock(encoder, crtc_state);
if (ret) {
- if (crtc_state->output_format != INTEL_OUTPUT_FORMAT_YCBCR420 &&
- connector->ycbcr_420_allowed &&
- drm_mode_is_420_also(&connector->display_info, adjusted_mode)) {
- crtc_state->output_format = INTEL_OUTPUT_FORMAT_YCBCR420;
- ret = intel_hdmi_compute_clock(encoder, crtc_state);
- }
+ if (intel_hdmi_is_ycbcr420(crtc_state) ||
+ !connector->base.ycbcr_420_allowed ||
+ !drm_mode_is_420_also(info, adjusted_mode))
+ return ret;
+
+ crtc_state->output_format = intel_hdmi_output_format(connector, true);
+ ret = intel_hdmi_compute_clock(encoder, crtc_state);
}
return ret;
@@ -2208,7 +2235,7 @@ int intel_hdmi_compute_config(struct intel_encoder *encoder,
if (ret)
return ret;
- if (pipe_config->output_format == INTEL_OUTPUT_FORMAT_YCBCR420) {
+ if (intel_hdmi_is_ycbcr420(pipe_config)) {
ret = intel_panel_fitting(pipe_config, conn_state);
if (ret)
return ret;
diff --git a/drivers/gpu/drm/i915/display/intel_lpe_audio.c b/drivers/gpu/drm/i915/display/intel_lpe_audio.c
index 7f3c638..4970bf1 100644
--- a/drivers/gpu/drm/i915/display/intel_lpe_audio.c
+++ b/drivers/gpu/drm/i915/display/intel_lpe_audio.c
@@ -74,7 +74,7 @@
#include "intel_de.h"
#include "intel_lpe_audio.h"
-#define HAS_LPE_AUDIO(dev_priv) ((dev_priv)->lpe_audio.platdev != NULL)
+#define HAS_LPE_AUDIO(dev_priv) ((dev_priv)->audio.lpe.platdev != NULL)
static struct platform_device *
lpe_audio_platdev_create(struct drm_i915_private *dev_priv)
@@ -96,7 +96,7 @@ lpe_audio_platdev_create(struct drm_i915_private *dev_priv)
return ERR_PTR(-ENOMEM);
}
- rsc[0].start = rsc[0].end = dev_priv->lpe_audio.irq;
+ rsc[0].start = rsc[0].end = dev_priv->audio.lpe.irq;
rsc[0].flags = IORESOURCE_IRQ;
rsc[0].name = "hdmi-lpe-audio-irq";
@@ -148,7 +148,7 @@ static void lpe_audio_platdev_destroy(struct drm_i915_private *dev_priv)
* than us fiddle with its internals.
*/
- platform_device_unregister(dev_priv->lpe_audio.platdev);
+ platform_device_unregister(dev_priv->audio.lpe.platdev);
}
static void lpe_audio_irq_unmask(struct irq_data *d)
@@ -167,7 +167,7 @@ static struct irq_chip lpe_audio_irqchip = {
static int lpe_audio_irq_init(struct drm_i915_private *dev_priv)
{
- int irq = dev_priv->lpe_audio.irq;
+ int irq = dev_priv->audio.lpe.irq;
drm_WARN_ON(&dev_priv->drm, !intel_irqs_enabled(dev_priv));
irq_set_chip_and_handler_name(irq,
@@ -204,15 +204,15 @@ static int lpe_audio_setup(struct drm_i915_private *dev_priv)
{
int ret;
- dev_priv->lpe_audio.irq = irq_alloc_desc(0);
- if (dev_priv->lpe_audio.irq < 0) {
+ dev_priv->audio.lpe.irq = irq_alloc_desc(0);
+ if (dev_priv->audio.lpe.irq < 0) {
drm_err(&dev_priv->drm, "Failed to allocate IRQ desc: %d\n",
- dev_priv->lpe_audio.irq);
- ret = dev_priv->lpe_audio.irq;
+ dev_priv->audio.lpe.irq);
+ ret = dev_priv->audio.lpe.irq;
goto err;
}
- drm_dbg(&dev_priv->drm, "irq = %d\n", dev_priv->lpe_audio.irq);
+ drm_dbg(&dev_priv->drm, "irq = %d\n", dev_priv->audio.lpe.irq);
ret = lpe_audio_irq_init(dev_priv);
@@ -223,10 +223,10 @@ static int lpe_audio_setup(struct drm_i915_private *dev_priv)
goto err_free_irq;
}
- dev_priv->lpe_audio.platdev = lpe_audio_platdev_create(dev_priv);
+ dev_priv->audio.lpe.platdev = lpe_audio_platdev_create(dev_priv);
- if (IS_ERR(dev_priv->lpe_audio.platdev)) {
- ret = PTR_ERR(dev_priv->lpe_audio.platdev);
+ if (IS_ERR(dev_priv->audio.lpe.platdev)) {
+ ret = PTR_ERR(dev_priv->audio.lpe.platdev);
drm_err(&dev_priv->drm,
"Failed to create lpe audio platform device: %d\n",
ret);
@@ -241,10 +241,10 @@ static int lpe_audio_setup(struct drm_i915_private *dev_priv)
return 0;
err_free_irq:
- irq_free_desc(dev_priv->lpe_audio.irq);
+ irq_free_desc(dev_priv->audio.lpe.irq);
err:
- dev_priv->lpe_audio.irq = -1;
- dev_priv->lpe_audio.platdev = NULL;
+ dev_priv->audio.lpe.irq = -1;
+ dev_priv->audio.lpe.platdev = NULL;
return ret;
}
@@ -262,7 +262,7 @@ void intel_lpe_audio_irq_handler(struct drm_i915_private *dev_priv)
if (!HAS_LPE_AUDIO(dev_priv))
return;
- ret = generic_handle_irq(dev_priv->lpe_audio.irq);
+ ret = generic_handle_irq(dev_priv->audio.lpe.irq);
if (ret)
drm_err_ratelimited(&dev_priv->drm,
"error handling LPE audio irq: %d\n", ret);
@@ -303,10 +303,10 @@ void intel_lpe_audio_teardown(struct drm_i915_private *dev_priv)
lpe_audio_platdev_destroy(dev_priv);
- irq_free_desc(dev_priv->lpe_audio.irq);
+ irq_free_desc(dev_priv->audio.lpe.irq);
- dev_priv->lpe_audio.irq = -1;
- dev_priv->lpe_audio.platdev = NULL;
+ dev_priv->audio.lpe.irq = -1;
+ dev_priv->audio.lpe.platdev = NULL;
}
/**
@@ -333,7 +333,7 @@ void intel_lpe_audio_notify(struct drm_i915_private *dev_priv,
if (!HAS_LPE_AUDIO(dev_priv))
return;
- pdata = dev_get_platdata(&dev_priv->lpe_audio.platdev->dev);
+ pdata = dev_get_platdata(&dev_priv->audio.lpe.platdev->dev);
ppdata = &pdata->port[port - PORT_B];
spin_lock_irqsave(&pdata->lpe_audio_slock, irqflags);
@@ -361,7 +361,7 @@ void intel_lpe_audio_notify(struct drm_i915_private *dev_priv,
}
if (pdata->notify_audio_lpe)
- pdata->notify_audio_lpe(dev_priv->lpe_audio.platdev, port - PORT_B);
+ pdata->notify_audio_lpe(dev_priv->audio.lpe.platdev, port - PORT_B);
spin_unlock_irqrestore(&pdata->lpe_audio_slock, irqflags);
}
diff --git a/drivers/gpu/drm/i915/display/intel_overlay.c b/drivers/gpu/drm/i915/display/intel_overlay.c
index 7e3f5c6..1a376e9 100644
--- a/drivers/gpu/drm/i915/display/intel_overlay.c
+++ b/drivers/gpu/drm/i915/display/intel_overlay.c
@@ -1382,7 +1382,7 @@ void intel_overlay_setup(struct drm_i915_private *dev_priv)
if (!HAS_OVERLAY(dev_priv))
return;
- engine = dev_priv->gt.engine[RCS0];
+ engine = to_gt(dev_priv)->engine[RCS0];
if (!engine || !engine->kernel_context)
return;
diff --git a/drivers/gpu/drm/i915/display/intel_pch_display.c b/drivers/gpu/drm/i915/display/intel_pch_display.c
new file mode 100644
index 0000000..a55c4bf
--- /dev/null
+++ b/drivers/gpu/drm/i915/display/intel_pch_display.c
@@ -0,0 +1,501 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright © 2021 Intel Corporation
+ */
+
+#include "g4x_dp.h"
+#include "intel_crt.h"
+#include "intel_de.h"
+#include "intel_display_types.h"
+#include "intel_fdi.h"
+#include "intel_lvds.h"
+#include "intel_pch_display.h"
+#include "intel_pch_refclk.h"
+#include "intel_pps.h"
+#include "intel_sdvo.h"
+
+static void assert_pch_dp_disabled(struct drm_i915_private *dev_priv,
+ enum pipe pipe, enum port port,
+ i915_reg_t dp_reg)
+{
+ enum pipe port_pipe;
+ bool state;
+
+ state = g4x_dp_port_enabled(dev_priv, dp_reg, port, &port_pipe);
+
+ I915_STATE_WARN(state && port_pipe == pipe,
+ "PCH DP %c enabled on transcoder %c, should be disabled\n",
+ port_name(port), pipe_name(pipe));
+
+ I915_STATE_WARN(HAS_PCH_IBX(dev_priv) && !state && port_pipe == PIPE_B,
+ "IBX PCH DP %c still using transcoder B\n",
+ port_name(port));
+}
+
+static void assert_pch_hdmi_disabled(struct drm_i915_private *dev_priv,
+ enum pipe pipe, enum port port,
+ i915_reg_t hdmi_reg)
+{
+ enum pipe port_pipe;
+ bool state;
+
+ state = intel_sdvo_port_enabled(dev_priv, hdmi_reg, &port_pipe);
+
+ I915_STATE_WARN(state && port_pipe == pipe,
+ "PCH HDMI %c enabled on transcoder %c, should be disabled\n",
+ port_name(port), pipe_name(pipe));
+
+ I915_STATE_WARN(HAS_PCH_IBX(dev_priv) && !state && port_pipe == PIPE_B,
+ "IBX PCH HDMI %c still using transcoder B\n",
+ port_name(port));
+}
+
+static void assert_pch_ports_disabled(struct drm_i915_private *dev_priv,
+ enum pipe pipe)
+{
+ enum pipe port_pipe;
+
+ assert_pch_dp_disabled(dev_priv, pipe, PORT_B, PCH_DP_B);
+ assert_pch_dp_disabled(dev_priv, pipe, PORT_C, PCH_DP_C);
+ assert_pch_dp_disabled(dev_priv, pipe, PORT_D, PCH_DP_D);
+
+ I915_STATE_WARN(intel_crt_port_enabled(dev_priv, PCH_ADPA, &port_pipe) &&
+ port_pipe == pipe,
+ "PCH VGA enabled on transcoder %c, should be disabled\n",
+ pipe_name(pipe));
+
+ I915_STATE_WARN(intel_lvds_port_enabled(dev_priv, PCH_LVDS, &port_pipe) &&
+ port_pipe == pipe,
+ "PCH LVDS enabled on transcoder %c, should be disabled\n",
+ pipe_name(pipe));
+
+ /* PCH SDVOB multiplex with HDMIB */
+ assert_pch_hdmi_disabled(dev_priv, pipe, PORT_B, PCH_HDMIB);
+ assert_pch_hdmi_disabled(dev_priv, pipe, PORT_C, PCH_HDMIC);
+ assert_pch_hdmi_disabled(dev_priv, pipe, PORT_D, PCH_HDMID);
+}
+
+static void assert_pch_transcoder_disabled(struct drm_i915_private *dev_priv,
+ enum pipe pipe)
+{
+ u32 val;
+ bool enabled;
+
+ val = intel_de_read(dev_priv, PCH_TRANSCONF(pipe));
+ enabled = !!(val & TRANS_ENABLE);
+ I915_STATE_WARN(enabled,
+ "transcoder assertion failed, should be off on pipe %c but is still active\n",
+ pipe_name(pipe));
+}
+
+static void ilk_pch_transcoder_set_timings(const struct intel_crtc_state *crtc_state,
+ enum pipe pch_transcoder)
+{
+ struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ enum transcoder cpu_transcoder = crtc_state->cpu_transcoder;
+
+ intel_de_write(dev_priv, PCH_TRANS_HTOTAL(pch_transcoder),
+ intel_de_read(dev_priv, HTOTAL(cpu_transcoder)));
+ intel_de_write(dev_priv, PCH_TRANS_HBLANK(pch_transcoder),
+ intel_de_read(dev_priv, HBLANK(cpu_transcoder)));
+ intel_de_write(dev_priv, PCH_TRANS_HSYNC(pch_transcoder),
+ intel_de_read(dev_priv, HSYNC(cpu_transcoder)));
+
+ intel_de_write(dev_priv, PCH_TRANS_VTOTAL(pch_transcoder),
+ intel_de_read(dev_priv, VTOTAL(cpu_transcoder)));
+ intel_de_write(dev_priv, PCH_TRANS_VBLANK(pch_transcoder),
+ intel_de_read(dev_priv, VBLANK(cpu_transcoder)));
+ intel_de_write(dev_priv, PCH_TRANS_VSYNC(pch_transcoder),
+ intel_de_read(dev_priv, VSYNC(cpu_transcoder)));
+ intel_de_write(dev_priv, PCH_TRANS_VSYNCSHIFT(pch_transcoder),
+ intel_de_read(dev_priv, VSYNCSHIFT(cpu_transcoder)));
+}
+
+static void ilk_enable_pch_transcoder(const struct intel_crtc_state *crtc_state)
+{
+ struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ enum pipe pipe = crtc->pipe;
+ i915_reg_t reg;
+ u32 val, pipeconf_val;
+
+ /* Make sure PCH DPLL is enabled */
+ assert_shared_dpll_enabled(dev_priv, crtc_state->shared_dpll);
+
+ /* FDI must be feeding us bits for PCH ports */
+ assert_fdi_tx_enabled(dev_priv, pipe);
+ assert_fdi_rx_enabled(dev_priv, pipe);
+
+ if (HAS_PCH_CPT(dev_priv)) {
+ reg = TRANS_CHICKEN2(pipe);
+ val = intel_de_read(dev_priv, reg);
+ /*
+ * Workaround: Set the timing override bit
+ * before enabling the pch transcoder.
+ */
+ val |= TRANS_CHICKEN2_TIMING_OVERRIDE;
+ /* Configure frame start delay to match the CPU */
+ val &= ~TRANS_CHICKEN2_FRAME_START_DELAY_MASK;
+ val |= TRANS_CHICKEN2_FRAME_START_DELAY(dev_priv->framestart_delay - 1);
+ intel_de_write(dev_priv, reg, val);
+ }
+
+ reg = PCH_TRANSCONF(pipe);
+ val = intel_de_read(dev_priv, reg);
+ pipeconf_val = intel_de_read(dev_priv, PIPECONF(pipe));
+
+ if (HAS_PCH_IBX(dev_priv)) {
+ /* Configure frame start delay to match the CPU */
+ val &= ~TRANS_FRAME_START_DELAY_MASK;
+ val |= TRANS_FRAME_START_DELAY(dev_priv->framestart_delay - 1);
+
+ /*
+ * Make the BPC in transcoder be consistent with
+ * that in pipeconf reg. For HDMI we must use 8bpc
+ * here for both 8bpc and 12bpc.
+ */
+ val &= ~PIPECONF_BPC_MASK;
+ if (intel_crtc_has_type(crtc_state, INTEL_OUTPUT_HDMI))
+ val |= PIPECONF_8BPC;
+ else
+ val |= pipeconf_val & PIPECONF_BPC_MASK;
+ }
+
+ val &= ~TRANS_INTERLACE_MASK;
+ if ((pipeconf_val & PIPECONF_INTERLACE_MASK) == PIPECONF_INTERLACED_ILK) {
+ if (HAS_PCH_IBX(dev_priv) &&
+ intel_crtc_has_type(crtc_state, INTEL_OUTPUT_SDVO))
+ val |= TRANS_LEGACY_INTERLACED_ILK;
+ else
+ val |= TRANS_INTERLACED;
+ } else {
+ val |= TRANS_PROGRESSIVE;
+ }
+
+ intel_de_write(dev_priv, reg, val | TRANS_ENABLE);
+ if (intel_de_wait_for_set(dev_priv, reg, TRANS_STATE_ENABLE, 100))
+ drm_err(&dev_priv->drm, "failed to enable transcoder %c\n",
+ pipe_name(pipe));
+}
+
+static void ilk_disable_pch_transcoder(struct intel_crtc *crtc)
+{
+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ enum pipe pipe = crtc->pipe;
+ i915_reg_t reg;
+ u32 val;
+
+ /* FDI relies on the transcoder */
+ assert_fdi_tx_disabled(dev_priv, pipe);
+ assert_fdi_rx_disabled(dev_priv, pipe);
+
+ /* Ports must be off as well */
+ assert_pch_ports_disabled(dev_priv, pipe);
+
+ reg = PCH_TRANSCONF(pipe);
+ val = intel_de_read(dev_priv, reg);
+ val &= ~TRANS_ENABLE;
+ intel_de_write(dev_priv, reg, val);
+ /* wait for PCH transcoder off, transcoder state */
+ if (intel_de_wait_for_clear(dev_priv, reg, TRANS_STATE_ENABLE, 50))
+ drm_err(&dev_priv->drm, "failed to disable transcoder %c\n",
+ pipe_name(pipe));
+
+ if (HAS_PCH_CPT(dev_priv)) {
+ /* Workaround: Clear the timing override chicken bit again. */
+ reg = TRANS_CHICKEN2(pipe);
+ val = intel_de_read(dev_priv, reg);
+ val &= ~TRANS_CHICKEN2_TIMING_OVERRIDE;
+ intel_de_write(dev_priv, reg, val);
+ }
+}
+
+/*
+ * Enable PCH resources required for PCH ports:
+ * - PCH PLLs
+ * - FDI training & RX/TX
+ * - update transcoder timings
+ * - DP transcoding bits
+ * - transcoder
+ */
+void ilk_pch_enable(struct intel_atomic_state *state,
+ struct intel_crtc *crtc)
+{
+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ const struct intel_crtc_state *crtc_state =
+ intel_atomic_get_new_crtc_state(state, crtc);
+ enum pipe pipe = crtc->pipe;
+ u32 temp;
+
+ assert_pch_transcoder_disabled(dev_priv, pipe);
+
+ /* For PCH output, training FDI link */
+ intel_fdi_link_train(crtc, crtc_state);
+
+ /*
+ * We need to program the right clock selection
+ * before writing the pixel multiplier into the DPLL.
+ */
+ if (HAS_PCH_CPT(dev_priv)) {
+ u32 sel;
+
+ temp = intel_de_read(dev_priv, PCH_DPLL_SEL);
+ temp |= TRANS_DPLL_ENABLE(pipe);
+ sel = TRANS_DPLLB_SEL(pipe);
+ if (crtc_state->shared_dpll ==
+ intel_get_shared_dpll_by_id(dev_priv, DPLL_ID_PCH_PLL_B))
+ temp |= sel;
+ else
+ temp &= ~sel;
+ intel_de_write(dev_priv, PCH_DPLL_SEL, temp);
+ }
+
+ /*
+ * XXX: pch pll's can be enabled any time before we enable the PCH
+ * transcoder, and we actually should do this to not upset any PCH
+ * transcoder that already use the clock when we share it.
+ *
+ * Note that enable_shared_dpll tries to do the right thing, but
+ * get_shared_dpll unconditionally resets the pll - we need that
+ * to have the right LVDS enable sequence.
+ */
+ intel_enable_shared_dpll(crtc_state);
+
+ /* set transcoder timing, panel must allow it */
+ assert_pps_unlocked(dev_priv, pipe);
+ ilk_pch_transcoder_set_timings(crtc_state, pipe);
+
+ intel_fdi_normal_train(crtc);
+
+ /* For PCH DP, enable TRANS_DP_CTL */
+ if (HAS_PCH_CPT(dev_priv) &&
+ intel_crtc_has_dp_encoder(crtc_state)) {
+ const struct drm_display_mode *adjusted_mode =
+ &crtc_state->hw.adjusted_mode;
+ u32 bpc = (intel_de_read(dev_priv, PIPECONF(pipe)) & PIPECONF_BPC_MASK) >> 5;
+ i915_reg_t reg = TRANS_DP_CTL(pipe);
+ enum port port;
+
+ temp = intel_de_read(dev_priv, reg);
+ temp &= ~(TRANS_DP_PORT_SEL_MASK |
+ TRANS_DP_SYNC_MASK |
+ TRANS_DP_BPC_MASK);
+ temp |= TRANS_DP_OUTPUT_ENABLE;
+ temp |= bpc << 9; /* same format but at 11:9 */
+
+ if (adjusted_mode->flags & DRM_MODE_FLAG_PHSYNC)
+ temp |= TRANS_DP_HSYNC_ACTIVE_HIGH;
+ if (adjusted_mode->flags & DRM_MODE_FLAG_PVSYNC)
+ temp |= TRANS_DP_VSYNC_ACTIVE_HIGH;
+
+ port = intel_get_crtc_new_encoder(state, crtc_state)->port;
+ drm_WARN_ON(&dev_priv->drm, port < PORT_B || port > PORT_D);
+ temp |= TRANS_DP_PORT_SEL(port);
+
+ intel_de_write(dev_priv, reg, temp);
+ }
+
+ ilk_enable_pch_transcoder(crtc_state);
+}
+
+void ilk_pch_disable(struct intel_atomic_state *state,
+ struct intel_crtc *crtc)
+{
+ ilk_fdi_disable(crtc);
+}
+
+void ilk_pch_post_disable(struct intel_atomic_state *state,
+ struct intel_crtc *crtc)
+{
+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ enum pipe pipe = crtc->pipe;
+
+ ilk_disable_pch_transcoder(crtc);
+
+ if (HAS_PCH_CPT(dev_priv)) {
+ i915_reg_t reg;
+ u32 temp;
+
+ /* disable TRANS_DP_CTL */
+ reg = TRANS_DP_CTL(pipe);
+ temp = intel_de_read(dev_priv, reg);
+ temp &= ~(TRANS_DP_OUTPUT_ENABLE |
+ TRANS_DP_PORT_SEL_MASK);
+ temp |= TRANS_DP_PORT_SEL_NONE;
+ intel_de_write(dev_priv, reg, temp);
+
+ /* disable DPLL_SEL */
+ temp = intel_de_read(dev_priv, PCH_DPLL_SEL);
+ temp &= ~(TRANS_DPLL_ENABLE(pipe) | TRANS_DPLLB_SEL(pipe));
+ intel_de_write(dev_priv, PCH_DPLL_SEL, temp);
+ }
+
+ ilk_fdi_pll_disable(crtc);
+}
+
+static void ilk_pch_clock_get(struct intel_crtc_state *crtc_state)
+{
+ struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+
+ /* read out port_clock from the DPLL */
+ i9xx_crtc_clock_get(crtc, crtc_state);
+
+ /*
+ * In case there is an active pipe without active ports,
+ * we may need some idea for the dotclock anyway.
+ * Calculate one based on the FDI configuration.
+ */
+ crtc_state->hw.adjusted_mode.crtc_clock =
+ intel_dotclock_calculate(intel_fdi_link_freq(dev_priv, crtc_state),
+ &crtc_state->fdi_m_n);
+}
+
+void ilk_pch_get_config(struct intel_crtc_state *crtc_state)
+{
+ struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ struct intel_shared_dpll *pll;
+ enum pipe pipe = crtc->pipe;
+ enum intel_dpll_id pll_id;
+ bool pll_active;
+ u32 tmp;
+
+ if ((intel_de_read(dev_priv, PCH_TRANSCONF(pipe)) & TRANS_ENABLE) == 0)
+ return;
+
+ crtc_state->has_pch_encoder = true;
+
+ tmp = intel_de_read(dev_priv, FDI_RX_CTL(pipe));
+ crtc_state->fdi_lanes = ((FDI_DP_PORT_WIDTH_MASK & tmp) >>
+ FDI_DP_PORT_WIDTH_SHIFT) + 1;
+
+ ilk_get_fdi_m_n_config(crtc, crtc_state);
+
+ if (HAS_PCH_IBX(dev_priv)) {
+ /*
+ * The pipe->pch transcoder and pch transcoder->pll
+ * mapping is fixed.
+ */
+ pll_id = (enum intel_dpll_id) pipe;
+ } else {
+ tmp = intel_de_read(dev_priv, PCH_DPLL_SEL);
+ if (tmp & TRANS_DPLLB_SEL(pipe))
+ pll_id = DPLL_ID_PCH_PLL_B;
+ else
+ pll_id = DPLL_ID_PCH_PLL_A;
+ }
+
+ crtc_state->shared_dpll = intel_get_shared_dpll_by_id(dev_priv, pll_id);
+ pll = crtc_state->shared_dpll;
+
+ pll_active = intel_dpll_get_hw_state(dev_priv, pll,
+ &crtc_state->dpll_hw_state);
+ drm_WARN_ON(&dev_priv->drm, !pll_active);
+
+ tmp = crtc_state->dpll_hw_state.dpll;
+ crtc_state->pixel_multiplier =
+ ((tmp & PLL_REF_SDVO_HDMI_MULTIPLIER_MASK)
+ >> PLL_REF_SDVO_HDMI_MULTIPLIER_SHIFT) + 1;
+
+ ilk_pch_clock_get(crtc_state);
+}
+
+static void lpt_enable_pch_transcoder(struct drm_i915_private *dev_priv,
+ enum transcoder cpu_transcoder)
+{
+ u32 val, pipeconf_val;
+
+ /* FDI must be feeding us bits for PCH ports */
+ assert_fdi_tx_enabled(dev_priv, (enum pipe) cpu_transcoder);
+ assert_fdi_rx_enabled(dev_priv, PIPE_A);
+
+ val = intel_de_read(dev_priv, TRANS_CHICKEN2(PIPE_A));
+ /* Workaround: set timing override bit. */
+ val |= TRANS_CHICKEN2_TIMING_OVERRIDE;
+ /* Configure frame start delay to match the CPU */
+ val &= ~TRANS_CHICKEN2_FRAME_START_DELAY_MASK;
+ val |= TRANS_CHICKEN2_FRAME_START_DELAY(dev_priv->framestart_delay - 1);
+ intel_de_write(dev_priv, TRANS_CHICKEN2(PIPE_A), val);
+
+ val = TRANS_ENABLE;
+ pipeconf_val = intel_de_read(dev_priv, PIPECONF(cpu_transcoder));
+
+ if ((pipeconf_val & PIPECONF_INTERLACE_MASK_HSW) ==
+ PIPECONF_INTERLACED_ILK)
+ val |= TRANS_INTERLACED;
+ else
+ val |= TRANS_PROGRESSIVE;
+
+ intel_de_write(dev_priv, LPT_TRANSCONF, val);
+ if (intel_de_wait_for_set(dev_priv, LPT_TRANSCONF,
+ TRANS_STATE_ENABLE, 100))
+ drm_err(&dev_priv->drm, "Failed to enable PCH transcoder\n");
+}
+
+static void lpt_disable_pch_transcoder(struct drm_i915_private *dev_priv)
+{
+ u32 val;
+
+ val = intel_de_read(dev_priv, LPT_TRANSCONF);
+ val &= ~TRANS_ENABLE;
+ intel_de_write(dev_priv, LPT_TRANSCONF, val);
+ /* wait for PCH transcoder off, transcoder state */
+ if (intel_de_wait_for_clear(dev_priv, LPT_TRANSCONF,
+ TRANS_STATE_ENABLE, 50))
+ drm_err(&dev_priv->drm, "Failed to disable PCH transcoder\n");
+
+ /* Workaround: clear timing override bit. */
+ val = intel_de_read(dev_priv, TRANS_CHICKEN2(PIPE_A));
+ val &= ~TRANS_CHICKEN2_TIMING_OVERRIDE;
+ intel_de_write(dev_priv, TRANS_CHICKEN2(PIPE_A), val);
+}
+
+void lpt_pch_enable(struct intel_atomic_state *state,
+ struct intel_crtc *crtc)
+{
+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ const struct intel_crtc_state *crtc_state =
+ intel_atomic_get_new_crtc_state(state, crtc);
+ enum transcoder cpu_transcoder = crtc_state->cpu_transcoder;
+
+ assert_pch_transcoder_disabled(dev_priv, PIPE_A);
+
+ lpt_program_iclkip(crtc_state);
+
+ /* Set transcoder timing. */
+ ilk_pch_transcoder_set_timings(crtc_state, PIPE_A);
+
+ lpt_enable_pch_transcoder(dev_priv, cpu_transcoder);
+}
+
+void lpt_pch_disable(struct intel_atomic_state *state,
+ struct intel_crtc *crtc)
+{
+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+
+ lpt_disable_pch_transcoder(dev_priv);
+
+ lpt_disable_iclkip(dev_priv);
+}
+
+void lpt_pch_get_config(struct intel_crtc_state *crtc_state)
+{
+ struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ u32 tmp;
+
+ if ((intel_de_read(dev_priv, LPT_TRANSCONF) & TRANS_ENABLE) == 0)
+ return;
+
+ crtc_state->has_pch_encoder = true;
+
+ tmp = intel_de_read(dev_priv, FDI_RX_CTL(PIPE_A));
+ crtc_state->fdi_lanes = ((FDI_DP_PORT_WIDTH_MASK & tmp) >>
+ FDI_DP_PORT_WIDTH_SHIFT) + 1;
+
+ ilk_get_fdi_m_n_config(crtc, crtc_state);
+
+ crtc_state->hw.adjusted_mode.crtc_clock = lpt_get_iclkip(dev_priv);
+}
diff --git a/drivers/gpu/drm/i915/display/intel_pch_display.h b/drivers/gpu/drm/i915/display/intel_pch_display.h
new file mode 100644
index 0000000..2c387fe
--- /dev/null
+++ b/drivers/gpu/drm/i915/display/intel_pch_display.h
@@ -0,0 +1,27 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2021 Intel Corporation
+ */
+
+#ifndef _INTEL_PCH_DISPLAY_H_
+#define _INTEL_PCH_DISPLAY_H_
+
+struct intel_atomic_state;
+struct intel_crtc;
+struct intel_crtc_state;
+
+void ilk_pch_enable(struct intel_atomic_state *state,
+ struct intel_crtc *crtc);
+void ilk_pch_disable(struct intel_atomic_state *state,
+ struct intel_crtc *crtc);
+void ilk_pch_post_disable(struct intel_atomic_state *state,
+ struct intel_crtc *crtc);
+void ilk_pch_get_config(struct intel_crtc_state *crtc_state);
+
+void lpt_pch_enable(struct intel_atomic_state *state,
+ struct intel_crtc *crtc);
+void lpt_pch_disable(struct intel_atomic_state *state,
+ struct intel_crtc *crtc);
+void lpt_pch_get_config(struct intel_crtc_state *crtc_state);
+
+#endif
diff --git a/drivers/gpu/drm/i915/display/intel_pch_refclk.c b/drivers/gpu/drm/i915/display/intel_pch_refclk.c
new file mode 100644
index 0000000..b688fd87
--- /dev/null
+++ b/drivers/gpu/drm/i915/display/intel_pch_refclk.c
@@ -0,0 +1,648 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright © 2021 Intel Corporation
+ */
+
+#include "intel_de.h"
+#include "intel_display_types.h"
+#include "intel_panel.h"
+#include "intel_pch_refclk.h"
+#include "intel_sbi.h"
+
+static void lpt_fdi_reset_mphy(struct drm_i915_private *dev_priv)
+{
+ u32 tmp;
+
+ tmp = intel_de_read(dev_priv, SOUTH_CHICKEN2);
+ tmp |= FDI_MPHY_IOSFSB_RESET_CTL;
+ intel_de_write(dev_priv, SOUTH_CHICKEN2, tmp);
+
+ if (wait_for_us(intel_de_read(dev_priv, SOUTH_CHICKEN2) &
+ FDI_MPHY_IOSFSB_RESET_STATUS, 100))
+ drm_err(&dev_priv->drm, "FDI mPHY reset assert timeout\n");
+
+ tmp = intel_de_read(dev_priv, SOUTH_CHICKEN2);
+ tmp &= ~FDI_MPHY_IOSFSB_RESET_CTL;
+ intel_de_write(dev_priv, SOUTH_CHICKEN2, tmp);
+
+ if (wait_for_us((intel_de_read(dev_priv, SOUTH_CHICKEN2) &
+ FDI_MPHY_IOSFSB_RESET_STATUS) == 0, 100))
+ drm_err(&dev_priv->drm, "FDI mPHY reset de-assert timeout\n");
+}
+
+/* WaMPhyProgramming:hsw */
+static void lpt_fdi_program_mphy(struct drm_i915_private *dev_priv)
+{
+ u32 tmp;
+
+ lpt_fdi_reset_mphy(dev_priv);
+
+ tmp = intel_sbi_read(dev_priv, 0x8008, SBI_MPHY);
+ tmp &= ~(0xFF << 24);
+ tmp |= (0x12 << 24);
+ intel_sbi_write(dev_priv, 0x8008, tmp, SBI_MPHY);
+
+ tmp = intel_sbi_read(dev_priv, 0x2008, SBI_MPHY);
+ tmp |= (1 << 11);
+ intel_sbi_write(dev_priv, 0x2008, tmp, SBI_MPHY);
+
+ tmp = intel_sbi_read(dev_priv, 0x2108, SBI_MPHY);
+ tmp |= (1 << 11);
+ intel_sbi_write(dev_priv, 0x2108, tmp, SBI_MPHY);
+
+ tmp = intel_sbi_read(dev_priv, 0x206C, SBI_MPHY);
+ tmp |= (1 << 24) | (1 << 21) | (1 << 18);
+ intel_sbi_write(dev_priv, 0x206C, tmp, SBI_MPHY);
+
+ tmp = intel_sbi_read(dev_priv, 0x216C, SBI_MPHY);
+ tmp |= (1 << 24) | (1 << 21) | (1 << 18);
+ intel_sbi_write(dev_priv, 0x216C, tmp, SBI_MPHY);
+
+ tmp = intel_sbi_read(dev_priv, 0x2080, SBI_MPHY);
+ tmp &= ~(7 << 13);
+ tmp |= (5 << 13);
+ intel_sbi_write(dev_priv, 0x2080, tmp, SBI_MPHY);
+
+ tmp = intel_sbi_read(dev_priv, 0x2180, SBI_MPHY);
+ tmp &= ~(7 << 13);
+ tmp |= (5 << 13);
+ intel_sbi_write(dev_priv, 0x2180, tmp, SBI_MPHY);
+
+ tmp = intel_sbi_read(dev_priv, 0x208C, SBI_MPHY);
+ tmp &= ~0xFF;
+ tmp |= 0x1C;
+ intel_sbi_write(dev_priv, 0x208C, tmp, SBI_MPHY);
+
+ tmp = intel_sbi_read(dev_priv, 0x218C, SBI_MPHY);
+ tmp &= ~0xFF;
+ tmp |= 0x1C;
+ intel_sbi_write(dev_priv, 0x218C, tmp, SBI_MPHY);
+
+ tmp = intel_sbi_read(dev_priv, 0x2098, SBI_MPHY);
+ tmp &= ~(0xFF << 16);
+ tmp |= (0x1C << 16);
+ intel_sbi_write(dev_priv, 0x2098, tmp, SBI_MPHY);
+
+ tmp = intel_sbi_read(dev_priv, 0x2198, SBI_MPHY);
+ tmp &= ~(0xFF << 16);
+ tmp |= (0x1C << 16);
+ intel_sbi_write(dev_priv, 0x2198, tmp, SBI_MPHY);
+
+ tmp = intel_sbi_read(dev_priv, 0x20C4, SBI_MPHY);
+ tmp |= (1 << 27);
+ intel_sbi_write(dev_priv, 0x20C4, tmp, SBI_MPHY);
+
+ tmp = intel_sbi_read(dev_priv, 0x21C4, SBI_MPHY);
+ tmp |= (1 << 27);
+ intel_sbi_write(dev_priv, 0x21C4, tmp, SBI_MPHY);
+
+ tmp = intel_sbi_read(dev_priv, 0x20EC, SBI_MPHY);
+ tmp &= ~(0xF << 28);
+ tmp |= (4 << 28);
+ intel_sbi_write(dev_priv, 0x20EC, tmp, SBI_MPHY);
+
+ tmp = intel_sbi_read(dev_priv, 0x21EC, SBI_MPHY);
+ tmp &= ~(0xF << 28);
+ tmp |= (4 << 28);
+ intel_sbi_write(dev_priv, 0x21EC, tmp, SBI_MPHY);
+}
+
+void lpt_disable_iclkip(struct drm_i915_private *dev_priv)
+{
+ u32 temp;
+
+ intel_de_write(dev_priv, PIXCLK_GATE, PIXCLK_GATE_GATE);
+
+ mutex_lock(&dev_priv->sb_lock);
+
+ temp = intel_sbi_read(dev_priv, SBI_SSCCTL6, SBI_ICLK);
+ temp |= SBI_SSCCTL_DISABLE;
+ intel_sbi_write(dev_priv, SBI_SSCCTL6, temp, SBI_ICLK);
+
+ mutex_unlock(&dev_priv->sb_lock);
+}
+
+/* Program iCLKIP clock to the desired frequency */
+void lpt_program_iclkip(const struct intel_crtc_state *crtc_state)
+{
+ struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ int clock = crtc_state->hw.adjusted_mode.crtc_clock;
+ u32 divsel, phaseinc, auxdiv, phasedir = 0;
+ u32 temp;
+
+ lpt_disable_iclkip(dev_priv);
+
+ /* The iCLK virtual clock root frequency is in MHz,
+ * but the adjusted_mode->crtc_clock in KHz. To get the
+ * divisors, it is necessary to divide one by another, so we
+ * convert the virtual clock precision to KHz here for higher
+ * precision.
+ */
+ for (auxdiv = 0; auxdiv < 2; auxdiv++) {
+ u32 iclk_virtual_root_freq = 172800 * 1000;
+ u32 iclk_pi_range = 64;
+ u32 desired_divisor;
+
+ desired_divisor = DIV_ROUND_CLOSEST(iclk_virtual_root_freq,
+ clock << auxdiv);
+ divsel = (desired_divisor / iclk_pi_range) - 2;
+ phaseinc = desired_divisor % iclk_pi_range;
+
+ /*
+ * Near 20MHz is a corner case which is
+ * out of range for the 7-bit divisor
+ */
+ if (divsel <= 0x7f)
+ break;
+ }
+
+ /* This should not happen with any sane values */
+ drm_WARN_ON(&dev_priv->drm, SBI_SSCDIVINTPHASE_DIVSEL(divsel) &
+ ~SBI_SSCDIVINTPHASE_DIVSEL_MASK);
+ drm_WARN_ON(&dev_priv->drm, SBI_SSCDIVINTPHASE_DIR(phasedir) &
+ ~SBI_SSCDIVINTPHASE_INCVAL_MASK);
+
+ drm_dbg_kms(&dev_priv->drm,
+ "iCLKIP clock: found settings for %dKHz refresh rate: auxdiv=%x, divsel=%x, phasedir=%x, phaseinc=%x\n",
+ clock, auxdiv, divsel, phasedir, phaseinc);
+
+ mutex_lock(&dev_priv->sb_lock);
+
+ /* Program SSCDIVINTPHASE6 */
+ temp = intel_sbi_read(dev_priv, SBI_SSCDIVINTPHASE6, SBI_ICLK);
+ temp &= ~SBI_SSCDIVINTPHASE_DIVSEL_MASK;
+ temp |= SBI_SSCDIVINTPHASE_DIVSEL(divsel);
+ temp &= ~SBI_SSCDIVINTPHASE_INCVAL_MASK;
+ temp |= SBI_SSCDIVINTPHASE_INCVAL(phaseinc);
+ temp |= SBI_SSCDIVINTPHASE_DIR(phasedir);
+ temp |= SBI_SSCDIVINTPHASE_PROPAGATE;
+ intel_sbi_write(dev_priv, SBI_SSCDIVINTPHASE6, temp, SBI_ICLK);
+
+ /* Program SSCAUXDIV */
+ temp = intel_sbi_read(dev_priv, SBI_SSCAUXDIV6, SBI_ICLK);
+ temp &= ~SBI_SSCAUXDIV_FINALDIV2SEL(1);
+ temp |= SBI_SSCAUXDIV_FINALDIV2SEL(auxdiv);
+ intel_sbi_write(dev_priv, SBI_SSCAUXDIV6, temp, SBI_ICLK);
+
+ /* Enable modulator and associated divider */
+ temp = intel_sbi_read(dev_priv, SBI_SSCCTL6, SBI_ICLK);
+ temp &= ~SBI_SSCCTL_DISABLE;
+ intel_sbi_write(dev_priv, SBI_SSCCTL6, temp, SBI_ICLK);
+
+ mutex_unlock(&dev_priv->sb_lock);
+
+ /* Wait for initialization time */
+ udelay(24);
+
+ intel_de_write(dev_priv, PIXCLK_GATE, PIXCLK_GATE_UNGATE);
+}
+
+int lpt_get_iclkip(struct drm_i915_private *dev_priv)
+{
+ u32 divsel, phaseinc, auxdiv;
+ u32 iclk_virtual_root_freq = 172800 * 1000;
+ u32 iclk_pi_range = 64;
+ u32 desired_divisor;
+ u32 temp;
+
+ if ((intel_de_read(dev_priv, PIXCLK_GATE) & PIXCLK_GATE_UNGATE) == 0)
+ return 0;
+
+ mutex_lock(&dev_priv->sb_lock);
+
+ temp = intel_sbi_read(dev_priv, SBI_SSCCTL6, SBI_ICLK);
+ if (temp & SBI_SSCCTL_DISABLE) {
+ mutex_unlock(&dev_priv->sb_lock);
+ return 0;
+ }
+
+ temp = intel_sbi_read(dev_priv, SBI_SSCDIVINTPHASE6, SBI_ICLK);
+ divsel = (temp & SBI_SSCDIVINTPHASE_DIVSEL_MASK) >>
+ SBI_SSCDIVINTPHASE_DIVSEL_SHIFT;
+ phaseinc = (temp & SBI_SSCDIVINTPHASE_INCVAL_MASK) >>
+ SBI_SSCDIVINTPHASE_INCVAL_SHIFT;
+
+ temp = intel_sbi_read(dev_priv, SBI_SSCAUXDIV6, SBI_ICLK);
+ auxdiv = (temp & SBI_SSCAUXDIV_FINALDIV2SEL_MASK) >>
+ SBI_SSCAUXDIV_FINALDIV2SEL_SHIFT;
+
+ mutex_unlock(&dev_priv->sb_lock);
+
+ desired_divisor = (divsel + 2) * iclk_pi_range + phaseinc;
+
+ return DIV_ROUND_CLOSEST(iclk_virtual_root_freq,
+ desired_divisor << auxdiv);
+}
+
+/* Implements 3 different sequences from BSpec chapter "Display iCLK
+ * Programming" based on the parameters passed:
+ * - Sequence to enable CLKOUT_DP
+ * - Sequence to enable CLKOUT_DP without spread
+ * - Sequence to enable CLKOUT_DP for FDI usage and configure PCH FDI I/O
+ */
+static void lpt_enable_clkout_dp(struct drm_i915_private *dev_priv,
+ bool with_spread, bool with_fdi)
+{
+ u32 reg, tmp;
+
+ if (drm_WARN(&dev_priv->drm, with_fdi && !with_spread,
+ "FDI requires downspread\n"))
+ with_spread = true;
+ if (drm_WARN(&dev_priv->drm, HAS_PCH_LPT_LP(dev_priv) &&
+ with_fdi, "LP PCH doesn't have FDI\n"))
+ with_fdi = false;
+
+ mutex_lock(&dev_priv->sb_lock);
+
+ tmp = intel_sbi_read(dev_priv, SBI_SSCCTL, SBI_ICLK);
+ tmp &= ~SBI_SSCCTL_DISABLE;
+ tmp |= SBI_SSCCTL_PATHALT;
+ intel_sbi_write(dev_priv, SBI_SSCCTL, tmp, SBI_ICLK);
+
+ udelay(24);
+
+ if (with_spread) {
+ tmp = intel_sbi_read(dev_priv, SBI_SSCCTL, SBI_ICLK);
+ tmp &= ~SBI_SSCCTL_PATHALT;
+ intel_sbi_write(dev_priv, SBI_SSCCTL, tmp, SBI_ICLK);
+
+ if (with_fdi)
+ lpt_fdi_program_mphy(dev_priv);
+ }
+
+ reg = HAS_PCH_LPT_LP(dev_priv) ? SBI_GEN0 : SBI_DBUFF0;
+ tmp = intel_sbi_read(dev_priv, reg, SBI_ICLK);
+ tmp |= SBI_GEN0_CFG_BUFFENABLE_DISABLE;
+ intel_sbi_write(dev_priv, reg, tmp, SBI_ICLK);
+
+ mutex_unlock(&dev_priv->sb_lock);
+}
+
+/* Sequence to disable CLKOUT_DP */
+void lpt_disable_clkout_dp(struct drm_i915_private *dev_priv)
+{
+ u32 reg, tmp;
+
+ mutex_lock(&dev_priv->sb_lock);
+
+ reg = HAS_PCH_LPT_LP(dev_priv) ? SBI_GEN0 : SBI_DBUFF0;
+ tmp = intel_sbi_read(dev_priv, reg, SBI_ICLK);
+ tmp &= ~SBI_GEN0_CFG_BUFFENABLE_DISABLE;
+ intel_sbi_write(dev_priv, reg, tmp, SBI_ICLK);
+
+ tmp = intel_sbi_read(dev_priv, SBI_SSCCTL, SBI_ICLK);
+ if (!(tmp & SBI_SSCCTL_DISABLE)) {
+ if (!(tmp & SBI_SSCCTL_PATHALT)) {
+ tmp |= SBI_SSCCTL_PATHALT;
+ intel_sbi_write(dev_priv, SBI_SSCCTL, tmp, SBI_ICLK);
+ udelay(32);
+ }
+ tmp |= SBI_SSCCTL_DISABLE;
+ intel_sbi_write(dev_priv, SBI_SSCCTL, tmp, SBI_ICLK);
+ }
+
+ mutex_unlock(&dev_priv->sb_lock);
+}
+
+#define BEND_IDX(steps) ((50 + (steps)) / 5)
+
+static const u16 sscdivintphase[] = {
+ [BEND_IDX( 50)] = 0x3B23,
+ [BEND_IDX( 45)] = 0x3B23,
+ [BEND_IDX( 40)] = 0x3C23,
+ [BEND_IDX( 35)] = 0x3C23,
+ [BEND_IDX( 30)] = 0x3D23,
+ [BEND_IDX( 25)] = 0x3D23,
+ [BEND_IDX( 20)] = 0x3E23,
+ [BEND_IDX( 15)] = 0x3E23,
+ [BEND_IDX( 10)] = 0x3F23,
+ [BEND_IDX( 5)] = 0x3F23,
+ [BEND_IDX( 0)] = 0x0025,
+ [BEND_IDX( -5)] = 0x0025,
+ [BEND_IDX(-10)] = 0x0125,
+ [BEND_IDX(-15)] = 0x0125,
+ [BEND_IDX(-20)] = 0x0225,
+ [BEND_IDX(-25)] = 0x0225,
+ [BEND_IDX(-30)] = 0x0325,
+ [BEND_IDX(-35)] = 0x0325,
+ [BEND_IDX(-40)] = 0x0425,
+ [BEND_IDX(-45)] = 0x0425,
+ [BEND_IDX(-50)] = 0x0525,
+};
+
+/*
+ * Bend CLKOUT_DP
+ * steps -50 to 50 inclusive, in steps of 5
+ * < 0 slow down the clock, > 0 speed up the clock, 0 == no bend (135MHz)
+ * change in clock period = -(steps / 10) * 5.787 ps
+ */
+static void lpt_bend_clkout_dp(struct drm_i915_private *dev_priv, int steps)
+{
+ u32 tmp;
+ int idx = BEND_IDX(steps);
+
+ if (drm_WARN_ON(&dev_priv->drm, steps % 5 != 0))
+ return;
+
+ if (drm_WARN_ON(&dev_priv->drm, idx >= ARRAY_SIZE(sscdivintphase)))
+ return;
+
+ mutex_lock(&dev_priv->sb_lock);
+
+ if (steps % 10 != 0)
+ tmp = 0xAAAAAAAB;
+ else
+ tmp = 0x00000000;
+ intel_sbi_write(dev_priv, SBI_SSCDITHPHASE, tmp, SBI_ICLK);
+
+ tmp = intel_sbi_read(dev_priv, SBI_SSCDIVINTPHASE, SBI_ICLK);
+ tmp &= 0xffff0000;
+ tmp |= sscdivintphase[idx];
+ intel_sbi_write(dev_priv, SBI_SSCDIVINTPHASE, tmp, SBI_ICLK);
+
+ mutex_unlock(&dev_priv->sb_lock);
+}
+
+#undef BEND_IDX
+
+static bool spll_uses_pch_ssc(struct drm_i915_private *dev_priv)
+{
+ u32 fuse_strap = intel_de_read(dev_priv, FUSE_STRAP);
+ u32 ctl = intel_de_read(dev_priv, SPLL_CTL);
+
+ if ((ctl & SPLL_PLL_ENABLE) == 0)
+ return false;
+
+ if ((ctl & SPLL_REF_MASK) == SPLL_REF_MUXED_SSC &&
+ (fuse_strap & HSW_CPU_SSC_ENABLE) == 0)
+ return true;
+
+ if (IS_BROADWELL(dev_priv) &&
+ (ctl & SPLL_REF_MASK) == SPLL_REF_PCH_SSC_BDW)
+ return true;
+
+ return false;
+}
+
+static bool wrpll_uses_pch_ssc(struct drm_i915_private *dev_priv,
+ enum intel_dpll_id id)
+{
+ u32 fuse_strap = intel_de_read(dev_priv, FUSE_STRAP);
+ u32 ctl = intel_de_read(dev_priv, WRPLL_CTL(id));
+
+ if ((ctl & WRPLL_PLL_ENABLE) == 0)
+ return false;
+
+ if ((ctl & WRPLL_REF_MASK) == WRPLL_REF_PCH_SSC)
+ return true;
+
+ if ((IS_BROADWELL(dev_priv) || IS_HSW_ULT(dev_priv)) &&
+ (ctl & WRPLL_REF_MASK) == WRPLL_REF_MUXED_SSC_BDW &&
+ (fuse_strap & HSW_CPU_SSC_ENABLE) == 0)
+ return true;
+
+ return false;
+}
+
+static void lpt_init_pch_refclk(struct drm_i915_private *dev_priv)
+{
+ struct intel_encoder *encoder;
+ bool has_fdi = false;
+
+ for_each_intel_encoder(&dev_priv->drm, encoder) {
+ switch (encoder->type) {
+ case INTEL_OUTPUT_ANALOG:
+ has_fdi = true;
+ break;
+ default:
+ break;
+ }
+ }
+
+ /*
+ * The BIOS may have decided to use the PCH SSC
+ * reference so we must not disable it until the
+ * relevant PLLs have stopped relying on it. We'll
+ * just leave the PCH SSC reference enabled in case
+ * any active PLL is using it. It will get disabled
+ * after runtime suspend if we don't have FDI.
+ *
+ * TODO: Move the whole reference clock handling
+ * to the modeset sequence proper so that we can
+ * actually enable/disable/reconfigure these things
+ * safely. To do that we need to introduce a real
+ * clock hierarchy. That would also allow us to do
+ * clock bending finally.
+ */
+ dev_priv->pch_ssc_use = 0;
+
+ if (spll_uses_pch_ssc(dev_priv)) {
+ drm_dbg_kms(&dev_priv->drm, "SPLL using PCH SSC\n");
+ dev_priv->pch_ssc_use |= BIT(DPLL_ID_SPLL);
+ }
+
+ if (wrpll_uses_pch_ssc(dev_priv, DPLL_ID_WRPLL1)) {
+ drm_dbg_kms(&dev_priv->drm, "WRPLL1 using PCH SSC\n");
+ dev_priv->pch_ssc_use |= BIT(DPLL_ID_WRPLL1);
+ }
+
+ if (wrpll_uses_pch_ssc(dev_priv, DPLL_ID_WRPLL2)) {
+ drm_dbg_kms(&dev_priv->drm, "WRPLL2 using PCH SSC\n");
+ dev_priv->pch_ssc_use |= BIT(DPLL_ID_WRPLL2);
+ }
+
+ if (dev_priv->pch_ssc_use)
+ return;
+
+ if (has_fdi) {
+ lpt_bend_clkout_dp(dev_priv, 0);
+ lpt_enable_clkout_dp(dev_priv, true, true);
+ } else {
+ lpt_disable_clkout_dp(dev_priv);
+ }
+}
+
+static void ilk_init_pch_refclk(struct drm_i915_private *dev_priv)
+{
+ struct intel_encoder *encoder;
+ int i;
+ u32 val, final;
+ bool has_lvds = false;
+ bool has_cpu_edp = false;
+ bool has_panel = false;
+ bool has_ck505 = false;
+ bool can_ssc = false;
+ bool using_ssc_source = false;
+
+ /* We need to take the global config into account */
+ for_each_intel_encoder(&dev_priv->drm, encoder) {
+ switch (encoder->type) {
+ case INTEL_OUTPUT_LVDS:
+ has_panel = true;
+ has_lvds = true;
+ break;
+ case INTEL_OUTPUT_EDP:
+ has_panel = true;
+ if (encoder->port == PORT_A)
+ has_cpu_edp = true;
+ break;
+ default:
+ break;
+ }
+ }
+
+ if (HAS_PCH_IBX(dev_priv)) {
+ has_ck505 = dev_priv->vbt.display_clock_mode;
+ can_ssc = has_ck505;
+ } else {
+ has_ck505 = false;
+ can_ssc = true;
+ }
+
+ /* Check if any DPLLs are using the SSC source */
+ for (i = 0; i < dev_priv->dpll.num_shared_dpll; i++) {
+ u32 temp = intel_de_read(dev_priv, PCH_DPLL(i));
+
+ if (!(temp & DPLL_VCO_ENABLE))
+ continue;
+
+ if ((temp & PLL_REF_INPUT_MASK) ==
+ PLLB_REF_INPUT_SPREADSPECTRUMIN) {
+ using_ssc_source = true;
+ break;
+ }
+ }
+
+ drm_dbg_kms(&dev_priv->drm,
+ "has_panel %d has_lvds %d has_ck505 %d using_ssc_source %d\n",
+ has_panel, has_lvds, has_ck505, using_ssc_source);
+
+ /* Ironlake: try to setup display ref clock before DPLL
+ * enabling. This is only under driver's control after
+ * PCH B stepping, previous chipset stepping should be
+ * ignoring this setting.
+ */
+ val = intel_de_read(dev_priv, PCH_DREF_CONTROL);
+
+ /* As we must carefully and slowly disable/enable each source in turn,
+ * compute the final state we want first and check if we need to
+ * make any changes at all.
+ */
+ final = val;
+ final &= ~DREF_NONSPREAD_SOURCE_MASK;
+ if (has_ck505)
+ final |= DREF_NONSPREAD_CK505_ENABLE;
+ else
+ final |= DREF_NONSPREAD_SOURCE_ENABLE;
+
+ final &= ~DREF_SSC_SOURCE_MASK;
+ final &= ~DREF_CPU_SOURCE_OUTPUT_MASK;
+ final &= ~DREF_SSC1_ENABLE;
+
+ if (has_panel) {
+ final |= DREF_SSC_SOURCE_ENABLE;
+
+ if (intel_panel_use_ssc(dev_priv) && can_ssc)
+ final |= DREF_SSC1_ENABLE;
+
+ if (has_cpu_edp) {
+ if (intel_panel_use_ssc(dev_priv) && can_ssc)
+ final |= DREF_CPU_SOURCE_OUTPUT_DOWNSPREAD;
+ else
+ final |= DREF_CPU_SOURCE_OUTPUT_NONSPREAD;
+ } else {
+ final |= DREF_CPU_SOURCE_OUTPUT_DISABLE;
+ }
+ } else if (using_ssc_source) {
+ final |= DREF_SSC_SOURCE_ENABLE;
+ final |= DREF_SSC1_ENABLE;
+ }
+
+ if (final == val)
+ return;
+
+ /* Always enable nonspread source */
+ val &= ~DREF_NONSPREAD_SOURCE_MASK;
+
+ if (has_ck505)
+ val |= DREF_NONSPREAD_CK505_ENABLE;
+ else
+ val |= DREF_NONSPREAD_SOURCE_ENABLE;
+
+ if (has_panel) {
+ val &= ~DREF_SSC_SOURCE_MASK;
+ val |= DREF_SSC_SOURCE_ENABLE;
+
+ /* SSC must be turned on before enabling the CPU output */
+ if (intel_panel_use_ssc(dev_priv) && can_ssc) {
+ drm_dbg_kms(&dev_priv->drm, "Using SSC on panel\n");
+ val |= DREF_SSC1_ENABLE;
+ } else {
+ val &= ~DREF_SSC1_ENABLE;
+ }
+
+ /* Get SSC going before enabling the outputs */
+ intel_de_write(dev_priv, PCH_DREF_CONTROL, val);
+ intel_de_posting_read(dev_priv, PCH_DREF_CONTROL);
+ udelay(200);
+
+ val &= ~DREF_CPU_SOURCE_OUTPUT_MASK;
+
+ /* Enable CPU source on CPU attached eDP */
+ if (has_cpu_edp) {
+ if (intel_panel_use_ssc(dev_priv) && can_ssc) {
+ drm_dbg_kms(&dev_priv->drm,
+ "Using SSC on eDP\n");
+ val |= DREF_CPU_SOURCE_OUTPUT_DOWNSPREAD;
+ } else {
+ val |= DREF_CPU_SOURCE_OUTPUT_NONSPREAD;
+ }
+ } else {
+ val |= DREF_CPU_SOURCE_OUTPUT_DISABLE;
+ }
+
+ intel_de_write(dev_priv, PCH_DREF_CONTROL, val);
+ intel_de_posting_read(dev_priv, PCH_DREF_CONTROL);
+ udelay(200);
+ } else {
+ drm_dbg_kms(&dev_priv->drm, "Disabling CPU source output\n");
+
+ val &= ~DREF_CPU_SOURCE_OUTPUT_MASK;
+
+ /* Turn off CPU output */
+ val |= DREF_CPU_SOURCE_OUTPUT_DISABLE;
+
+ intel_de_write(dev_priv, PCH_DREF_CONTROL, val);
+ intel_de_posting_read(dev_priv, PCH_DREF_CONTROL);
+ udelay(200);
+
+ if (!using_ssc_source) {
+ drm_dbg_kms(&dev_priv->drm, "Disabling SSC source\n");
+
+ /* Turn off the SSC source */
+ val &= ~DREF_SSC_SOURCE_MASK;
+ val |= DREF_SSC_SOURCE_DISABLE;
+
+ /* Turn off SSC1 */
+ val &= ~DREF_SSC1_ENABLE;
+
+ intel_de_write(dev_priv, PCH_DREF_CONTROL, val);
+ intel_de_posting_read(dev_priv, PCH_DREF_CONTROL);
+ udelay(200);
+ }
+ }
+
+ BUG_ON(val != final);
+}
+
+/*
+ * Initialize reference clocks when the driver loads
+ */
+void intel_init_pch_refclk(struct drm_i915_private *dev_priv)
+{
+ if (HAS_PCH_IBX(dev_priv) || HAS_PCH_CPT(dev_priv))
+ ilk_init_pch_refclk(dev_priv);
+ else if (HAS_PCH_LPT(dev_priv))
+ lpt_init_pch_refclk(dev_priv);
+}
diff --git a/drivers/gpu/drm/i915/display/intel_pch_refclk.h b/drivers/gpu/drm/i915/display/intel_pch_refclk.h
new file mode 100644
index 0000000..12ab2c7
--- /dev/null
+++ b/drivers/gpu/drm/i915/display/intel_pch_refclk.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2021 Intel Corporation
+ */
+
+#ifndef _INTEL_PCH_REFCLK_H_
+#define _INTEL_PCH_REFCLK_H_
+
+#include <linux/types.h>
+
+struct drm_i915_private;
+struct intel_crtc_state;
+
+void lpt_program_iclkip(const struct intel_crtc_state *crtc_state);
+void lpt_disable_iclkip(struct drm_i915_private *dev_priv);
+int lpt_get_iclkip(struct drm_i915_private *dev_priv);
+
+void intel_init_pch_refclk(struct drm_i915_private *dev_priv);
+void lpt_disable_clkout_dp(struct drm_i915_private *dev_priv);
+
+#endif
diff --git a/drivers/gpu/drm/i915/display/intel_plane_initial.c b/drivers/gpu/drm/i915/display/intel_plane_initial.c
index dcd698a..01ce1d7 100644
--- a/drivers/gpu/drm/i915/display/intel_plane_initial.c
+++ b/drivers/gpu/drm/i915/display/intel_plane_initial.c
@@ -3,11 +3,12 @@
* Copyright © 2021 Intel Corporation
*/
-#include "intel_display_types.h"
-#include "intel_plane_initial.h"
+#include "i915_drv.h"
#include "intel_atomic_plane.h"
#include "intel_display.h"
+#include "intel_display_types.h"
#include "intel_fb.h"
+#include "intel_plane_initial.h"
static bool
intel_reuse_initial_plane_obj(struct drm_i915_private *i915,
diff --git a/drivers/gpu/drm/i915/display/intel_psr.c b/drivers/gpu/drm/i915/display/intel_psr.c
index 7a205fd..a1a663f 100644
--- a/drivers/gpu/drm/i915/display/intel_psr.c
+++ b/drivers/gpu/drm/i915/display/intel_psr.c
@@ -28,13 +28,13 @@
#include "i915_drv.h"
#include "intel_atomic.h"
+#include "intel_crtc.h"
#include "intel_de.h"
#include "intel_display_types.h"
#include "intel_dp_aux.h"
#include "intel_hdmi.h"
#include "intel_psr.h"
#include "intel_snps_phy.h"
-#include "intel_sprite.h"
#include "skl_universal_plane.h"
/**
@@ -588,7 +588,9 @@ static void hsw_activate_psr2(struct intel_dp *intel_dp)
static bool
transcoder_has_psr2(struct drm_i915_private *dev_priv, enum transcoder trans)
{
- if (DISPLAY_VER(dev_priv) >= 12)
+ if (IS_ALDERLAKE_P(dev_priv))
+ return trans == TRANSCODER_A || trans == TRANSCODER_B;
+ else if (DISPLAY_VER(dev_priv) >= 12)
return trans == TRANSCODER_A;
else
return trans == TRANSCODER_EDP;
@@ -1346,6 +1348,7 @@ void intel_psr_disable(struct intel_dp *intel_dp,
*/
void intel_psr_pause(struct intel_dp *intel_dp)
{
+ struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
struct intel_psr *psr = &intel_dp->psr;
if (!CAN_PSR(intel_dp))
@@ -1358,6 +1361,9 @@ void intel_psr_pause(struct intel_dp *intel_dp)
return;
}
+ /* If we ever hit this, we will need to add refcount to pause/resume */
+ drm_WARN_ON(&dev_priv->drm, psr->paused);
+
intel_psr_exit(intel_dp);
intel_psr_wait_exit_locked(intel_dp);
psr->paused = true;
@@ -1463,10 +1469,19 @@ void intel_psr2_program_plane_sel_fetch(struct intel_plane *plane,
val |= plane_state->uapi.dst.x1;
intel_de_write_fw(dev_priv, PLANE_SEL_FETCH_POS(pipe, plane->id), val);
- /* TODO: consider auxiliary surfaces */
- x = plane_state->uapi.src.x1 >> 16;
- y = (plane_state->uapi.src.y1 >> 16) + clip->y1;
+ x = plane_state->view.color_plane[color_plane].x;
+
+ /*
+ * From Bspec: UV surface Start Y Position = half of Y plane Y
+ * start position.
+ */
+ if (!color_plane)
+ y = plane_state->view.color_plane[color_plane].y + clip->y1;
+ else
+ y = plane_state->view.color_plane[color_plane].y + clip->y1 / 2;
+
val = y << 16 | x;
+
intel_de_write_fw(dev_priv, PLANE_SEL_FETCH_OFFSET(pipe, plane->id),
val);
@@ -1558,9 +1573,6 @@ static void intel_psr2_sel_fetch_pipe_alignment(const struct intel_crtc_state *c
* also planes are not updated if they have a negative X
* position so for now doing a full update in this cases
*
- * TODO: We are missing multi-planar formats handling, until it is
- * implemented it will send full frame updates.
- *
* Plane scaling and rotation is not supported by selective fetch and both
* properties can change without a modeset, so need to be check at every
* atomic commmit.
@@ -1570,7 +1582,6 @@ static bool psr2_sel_fetch_plane_state_supported(const struct intel_plane_state
if (plane_state->uapi.dst.y1 < 0 ||
plane_state->uapi.dst.x1 < 0 ||
plane_state->scaler_id >= 0 ||
- plane_state->hw.fb->format->num_planes > 1 ||
plane_state->uapi.rotation != DRM_MODE_ROTATE_0)
return false;
@@ -1696,6 +1707,7 @@ int intel_psr2_sel_fetch_update(struct intel_atomic_state *state,
for_each_oldnew_intel_plane_in_state(state, plane, old_plane_state,
new_plane_state, i) {
struct drm_rect *sel_fetch_area, inter;
+ struct intel_plane *linked = new_plane_state->planar_linked_plane;
if (new_plane_state->uapi.crtc != crtc_state->uapi.crtc ||
!new_plane_state->uapi.visible)
@@ -1714,6 +1726,24 @@ int intel_psr2_sel_fetch_update(struct intel_atomic_state *state,
sel_fetch_area->y1 = inter.y1 - new_plane_state->uapi.dst.y1;
sel_fetch_area->y2 = inter.y2 - new_plane_state->uapi.dst.y1;
crtc_state->update_planes |= BIT(plane->id);
+
+ /*
+ * Sel_fetch_area is calculated for UV plane. Use
+ * same area for Y plane as well.
+ */
+ if (linked) {
+ struct intel_plane_state *linked_new_plane_state;
+ struct drm_rect *linked_sel_fetch_area;
+
+ linked_new_plane_state = intel_atomic_get_plane_state(state, linked);
+ if (IS_ERR(linked_new_plane_state))
+ return PTR_ERR(linked_new_plane_state);
+
+ linked_sel_fetch_area = &linked_new_plane_state->psr2_sel_fetch_area;
+ linked_sel_fetch_area->y1 = sel_fetch_area->y1;
+ linked_sel_fetch_area->y2 = sel_fetch_area->y2;
+ crtc_state->update_planes |= BIT(linked->id);
+ }
}
skip_sel_fetch_set_loop:
@@ -1721,11 +1751,17 @@ int intel_psr2_sel_fetch_update(struct intel_atomic_state *state,
return 0;
}
-static void _intel_psr_pre_plane_update(const struct intel_atomic_state *state,
- const struct intel_crtc_state *crtc_state)
+void intel_psr_pre_plane_update(struct intel_atomic_state *state,
+ struct intel_crtc *crtc)
{
+ struct drm_i915_private *i915 = to_i915(state->base.dev);
+ const struct intel_crtc_state *crtc_state =
+ intel_atomic_get_new_crtc_state(state, crtc);
struct intel_encoder *encoder;
+ if (!HAS_PSR(i915))
+ return;
+
for_each_intel_encoder_mask_with_psr(state->base.dev, encoder,
crtc_state->uapi.encoder_mask) {
struct intel_dp *intel_dp = enc_to_intel_dp(encoder);
@@ -1740,6 +1776,7 @@ static void _intel_psr_pre_plane_update(const struct intel_atomic_state *state,
* - All planes will go inactive
* - Changing between PSR versions
*/
+ needs_to_disable |= intel_crtc_needs_modeset(crtc_state);
needs_to_disable |= !crtc_state->has_psr;
needs_to_disable |= !crtc_state->active_planes;
needs_to_disable |= crtc_state->has_psr2 != psr->psr2_enabled;
@@ -1751,20 +1788,6 @@ static void _intel_psr_pre_plane_update(const struct intel_atomic_state *state,
}
}
-void intel_psr_pre_plane_update(const struct intel_atomic_state *state)
-{
- struct drm_i915_private *dev_priv = to_i915(state->base.dev);
- struct intel_crtc_state *crtc_state;
- struct intel_crtc *crtc;
- int i;
-
- if (!HAS_PSR(dev_priv))
- return;
-
- for_each_new_intel_crtc_in_state(state, crtc, crtc_state, i)
- _intel_psr_pre_plane_update(state, crtc_state);
-}
-
static void _intel_psr_post_plane_update(const struct intel_atomic_state *state,
const struct intel_crtc_state *crtc_state)
{
@@ -1809,15 +1832,21 @@ void intel_psr_post_plane_update(const struct intel_atomic_state *state)
_intel_psr_post_plane_update(state, crtc_state);
}
-/**
- * psr_wait_for_idle - wait for PSR1 to idle
- * @intel_dp: Intel DP
- * @out_value: PSR status in case of failure
- *
- * Returns: 0 on success or -ETIMEOUT if PSR status does not idle.
- *
- */
-static int psr_wait_for_idle(struct intel_dp *intel_dp, u32 *out_value)
+static int _psr2_ready_for_pipe_update_locked(struct intel_dp *intel_dp)
+{
+ struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
+
+ /*
+ * Any state lower than EDP_PSR2_STATUS_STATE_DEEP_SLEEP is enough.
+ * As all higher states has bit 4 of PSR2 state set we can just wait for
+ * EDP_PSR2_STATUS_STATE_DEEP_SLEEP to be cleared.
+ */
+ return intel_de_wait_for_clear(dev_priv,
+ EDP_PSR2_STATUS(intel_dp->psr.transcoder),
+ EDP_PSR2_STATUS_STATE_DEEP_SLEEP, 50);
+}
+
+static int _psr1_ready_for_pipe_update_locked(struct intel_dp *intel_dp)
{
struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
@@ -1827,15 +1856,13 @@ static int psr_wait_for_idle(struct intel_dp *intel_dp, u32 *out_value)
* exit training time + 1.5 ms of aux channel handshake. 50 ms is
* defensive enough to cover everything.
*/
- return __intel_wait_for_register(&dev_priv->uncore,
- EDP_PSR_STATUS(intel_dp->psr.transcoder),
- EDP_PSR_STATUS_STATE_MASK,
- EDP_PSR_STATUS_STATE_IDLE, 2, 50,
- out_value);
+ return intel_de_wait_for_clear(dev_priv,
+ EDP_PSR_STATUS(intel_dp->psr.transcoder),
+ EDP_PSR_STATUS_STATE_MASK, 50);
}
/**
- * intel_psr_wait_for_idle - wait for PSR1 to idle
+ * intel_psr_wait_for_idle - wait for PSR be ready for a pipe update
* @new_crtc_state: new CRTC state
*
* This function is expected to be called from pipe_update_start() where it is
@@ -1852,19 +1879,23 @@ void intel_psr_wait_for_idle(const struct intel_crtc_state *new_crtc_state)
for_each_intel_encoder_mask_with_psr(&dev_priv->drm, encoder,
new_crtc_state->uapi.encoder_mask) {
struct intel_dp *intel_dp = enc_to_intel_dp(encoder);
- u32 psr_status;
+ int ret;
mutex_lock(&intel_dp->psr.lock);
- if (!intel_dp->psr.enabled || intel_dp->psr.psr2_enabled) {
+
+ if (!intel_dp->psr.enabled) {
mutex_unlock(&intel_dp->psr.lock);
continue;
}
- /* when the PSR1 is enabled */
- if (psr_wait_for_idle(intel_dp, &psr_status))
- drm_err(&dev_priv->drm,
- "PSR idle timed out 0x%x, atomic update may fail\n",
- psr_status);
+ if (intel_dp->psr.psr2_enabled)
+ ret = _psr2_ready_for_pipe_update_locked(intel_dp);
+ else
+ ret = _psr1_ready_for_pipe_update_locked(intel_dp);
+
+ if (ret)
+ drm_err(&dev_priv->drm, "PSR wait timed out, atomic update may fail\n");
+
mutex_unlock(&intel_dp->psr.lock);
}
}
diff --git a/drivers/gpu/drm/i915/display/intel_psr.h b/drivers/gpu/drm/i915/display/intel_psr.h
index facffba..f6526d9 100644
--- a/drivers/gpu/drm/i915/display/intel_psr.h
+++ b/drivers/gpu/drm/i915/display/intel_psr.h
@@ -6,21 +6,23 @@
#ifndef __INTEL_PSR_H__
#define __INTEL_PSR_H__
-#include "intel_frontbuffer.h"
+#include <linux/types.h>
+enum fb_op_origin;
struct drm_connector;
struct drm_connector_state;
struct drm_i915_private;
+struct intel_atomic_state;
+struct intel_crtc;
struct intel_crtc_state;
struct intel_dp;
-struct intel_crtc;
-struct intel_atomic_state;
-struct intel_plane_state;
-struct intel_plane;
struct intel_encoder;
+struct intel_plane;
+struct intel_plane_state;
void intel_psr_init_dpcd(struct intel_dp *intel_dp);
-void intel_psr_pre_plane_update(const struct intel_atomic_state *state);
+void intel_psr_pre_plane_update(struct intel_atomic_state *state,
+ struct intel_crtc *crtc);
void intel_psr_post_plane_update(const struct intel_atomic_state *state);
void intel_psr_disable(struct intel_dp *intel_dp,
const struct intel_crtc_state *old_crtc_state);
diff --git a/drivers/gpu/drm/i915/display/intel_quirks.c b/drivers/gpu/drm/i915/display/intel_quirks.c
index 8a52b7a..c8488f5 100644
--- a/drivers/gpu/drm/i915/display/intel_quirks.c
+++ b/drivers/gpu/drm/i915/display/intel_quirks.c
@@ -5,6 +5,7 @@
#include <linux/dmi.h>
+#include "i915_drv.h"
#include "intel_display_types.h"
#include "intel_quirks.h"
diff --git a/drivers/gpu/drm/i915/display/intel_sdvo.c b/drivers/gpu/drm/i915/display/intel_sdvo.c
index 2dc6c37..76e1188 100644
--- a/drivers/gpu/drm/i915/display/intel_sdvo.c
+++ b/drivers/gpu/drm/i915/display/intel_sdvo.c
@@ -1842,7 +1842,7 @@ static void intel_enable_sdvo(struct intel_atomic_state *state,
intel_sdvo_write_sdvox(intel_sdvo, temp);
for (i = 0; i < 2; i++)
- intel_wait_for_vblank(dev_priv, crtc->pipe);
+ intel_crtc_wait_for_next_vblank(crtc);
success = intel_sdvo_get_trained_inputs(intel_sdvo, &input1, &input2);
/*
diff --git a/drivers/gpu/drm/i915/display/intel_snps_phy.c b/drivers/gpu/drm/i915/display/intel_snps_phy.c
index 5e20f34..09f405e 100644
--- a/drivers/gpu/drm/i915/display/intel_snps_phy.c
+++ b/drivers/gpu/drm/i915/display/intel_snps_phy.c
@@ -58,7 +58,6 @@ void intel_snps_phy_set_signal_levels(struct intel_encoder *encoder,
struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
const struct intel_ddi_buf_trans *trans;
enum phy phy = intel_port_to_phy(dev_priv, encoder->port);
- int level = intel_ddi_level(encoder, crtc_state, 0);
int n_entries, ln;
trans = encoder->get_buf_trans(encoder, crtc_state, &n_entries);
@@ -66,6 +65,7 @@ void intel_snps_phy_set_signal_levels(struct intel_encoder *encoder,
return;
for (ln = 0; ln < 4; ln++) {
+ int level = intel_ddi_level(encoder, crtc_state, ln);
u32 val = 0;
val |= REG_FIELD_PREP(SNPS_PHY_TX_EQ_MAIN, trans->entries[level].snps.vswing);
@@ -186,6 +186,7 @@ static const struct intel_mpllb_state dg2_dp_uhbr10_100 = {
REG_FIELD_PREP(SNPS_PHY_MPLLB_PMIX_EN, 1) |
REG_FIELD_PREP(SNPS_PHY_MPLLB_WORD_DIV2_EN, 1) |
REG_FIELD_PREP(SNPS_PHY_MPLLB_DP2_MODE, 1) |
+ REG_FIELD_PREP(SNPS_PHY_MPLLB_SHIM_DIV32_CLK_SEL, 1) |
REG_FIELD_PREP(SNPS_PHY_MPLLB_V2I, 2),
.mpllb_div2 =
REG_FIELD_PREP(SNPS_PHY_MPLLB_REF_CLK_DIV, 2) |
@@ -369,6 +370,7 @@ static const struct intel_mpllb_state dg2_dp_uhbr10_38_4 = {
REG_FIELD_PREP(SNPS_PHY_MPLLB_PMIX_EN, 1) |
REG_FIELD_PREP(SNPS_PHY_MPLLB_WORD_DIV2_EN, 1) |
REG_FIELD_PREP(SNPS_PHY_MPLLB_DP2_MODE, 1) |
+ REG_FIELD_PREP(SNPS_PHY_MPLLB_SHIM_DIV32_CLK_SEL, 1) |
REG_FIELD_PREP(SNPS_PHY_MPLLB_V2I, 2),
.mpllb_div2 =
REG_FIELD_PREP(SNPS_PHY_MPLLB_REF_CLK_DIV, 1) |
diff --git a/drivers/gpu/drm/i915/display/intel_sprite.c b/drivers/gpu/drm/i915/display/intel_sprite.c
index 08116f4..2357a13 100644
--- a/drivers/gpu/drm/i915/display/intel_sprite.c
+++ b/drivers/gpu/drm/i915/display/intel_sprite.c
@@ -40,14 +40,15 @@
#include <drm/drm_rect.h>
#include "i915_drv.h"
-#include "i915_trace.h"
#include "i915_vgpu.h"
+#include "i9xx_plane.h"
#include "intel_atomic_plane.h"
+#include "intel_crtc.h"
#include "intel_de.h"
#include "intel_display_types.h"
+#include "intel_fb.h"
#include "intel_frontbuffer.h"
#include "intel_sprite.h"
-#include "i9xx_plane.h"
#include "intel_vrr.h"
int intel_plane_check_src_coordinates(struct intel_plane_state *plane_state)
@@ -118,7 +119,7 @@ static void i9xx_plane_linear_gamma(u16 gamma[8])
}
static void
-chv_update_csc(const struct intel_plane_state *plane_state)
+chv_sprite_update_csc(const struct intel_plane_state *plane_state)
{
struct intel_plane *plane = to_intel_plane(plane_state->uapi.plane);
struct drm_i915_private *dev_priv = to_i915(plane->base.dev);
@@ -190,7 +191,7 @@ chv_update_csc(const struct intel_plane_state *plane_state)
#define COS_0 1
static void
-vlv_update_clrc(const struct intel_plane_state *plane_state)
+vlv_sprite_update_clrc(const struct intel_plane_state *plane_state)
{
struct intel_plane *plane = to_intel_plane(plane_state->uapi.plane);
struct drm_i915_private *dev_priv = to_i915(plane->base.dev);
@@ -393,7 +394,7 @@ static u32 vlv_sprite_ctl(const struct intel_crtc_state *crtc_state,
return sprctl;
}
-static void vlv_update_gamma(const struct intel_plane_state *plane_state)
+static void vlv_sprite_update_gamma(const struct intel_plane_state *plane_state)
{
struct intel_plane *plane = to_intel_plane(plane_state->uapi.plane);
struct drm_i915_private *dev_priv = to_i915(plane->base.dev);
@@ -417,45 +418,54 @@ static void vlv_update_gamma(const struct intel_plane_state *plane_state)
}
static void
-vlv_update_plane(struct intel_plane *plane,
- const struct intel_crtc_state *crtc_state,
- const struct intel_plane_state *plane_state)
+vlv_sprite_update_noarm(struct intel_plane *plane,
+ const struct intel_crtc_state *crtc_state,
+ const struct intel_plane_state *plane_state)
{
struct drm_i915_private *dev_priv = to_i915(plane->base.dev);
enum pipe pipe = plane->pipe;
enum plane_id plane_id = plane->id;
- u32 sprsurf_offset = plane_state->view.color_plane[0].offset;
- u32 linear_offset;
- const struct drm_intel_sprite_colorkey *key = &plane_state->ckey;
int crtc_x = plane_state->uapi.dst.x1;
int crtc_y = plane_state->uapi.dst.y1;
u32 crtc_w = drm_rect_width(&plane_state->uapi.dst);
u32 crtc_h = drm_rect_height(&plane_state->uapi.dst);
+ unsigned long irqflags;
+
+ spin_lock_irqsave(&dev_priv->uncore.lock, irqflags);
+
+ intel_de_write_fw(dev_priv, SPSTRIDE(pipe, plane_id),
+ plane_state->view.color_plane[0].mapping_stride);
+ intel_de_write_fw(dev_priv, SPPOS(pipe, plane_id),
+ (crtc_y << 16) | crtc_x);
+ intel_de_write_fw(dev_priv, SPSIZE(pipe, plane_id),
+ ((crtc_h - 1) << 16) | (crtc_w - 1));
+
+ spin_unlock_irqrestore(&dev_priv->uncore.lock, irqflags);
+}
+
+static void
+vlv_sprite_update_arm(struct intel_plane *plane,
+ const struct intel_crtc_state *crtc_state,
+ const struct intel_plane_state *plane_state)
+{
+ struct drm_i915_private *dev_priv = to_i915(plane->base.dev);
+ enum pipe pipe = plane->pipe;
+ enum plane_id plane_id = plane->id;
+ const struct drm_intel_sprite_colorkey *key = &plane_state->ckey;
+ u32 sprsurf_offset = plane_state->view.color_plane[0].offset;
u32 x = plane_state->view.color_plane[0].x;
u32 y = plane_state->view.color_plane[0].y;
+ u32 sprctl, linear_offset;
unsigned long irqflags;
- u32 sprctl;
sprctl = plane_state->ctl | vlv_sprite_ctl_crtc(crtc_state);
- /* Sizes are 0 based */
- crtc_w--;
- crtc_h--;
-
linear_offset = intel_fb_xy_to_linear(x, y, plane_state, 0);
spin_lock_irqsave(&dev_priv->uncore.lock, irqflags);
- intel_de_write_fw(dev_priv, SPSTRIDE(pipe, plane_id),
- plane_state->view.color_plane[0].stride);
- intel_de_write_fw(dev_priv, SPPOS(pipe, plane_id),
- (crtc_y << 16) | crtc_x);
- intel_de_write_fw(dev_priv, SPSIZE(pipe, plane_id),
- (crtc_h << 16) | crtc_w);
- intel_de_write_fw(dev_priv, SPCONSTALPHA(pipe, plane_id), 0);
-
if (IS_CHERRYVIEW(dev_priv) && pipe == PIPE_B)
- chv_update_csc(plane_state);
+ chv_sprite_update_csc(plane_state);
if (key->flags) {
intel_de_write_fw(dev_priv, SPKEYMINVAL(pipe, plane_id),
@@ -466,6 +476,8 @@ vlv_update_plane(struct intel_plane *plane,
key->max_value);
}
+ intel_de_write_fw(dev_priv, SPCONSTALPHA(pipe, plane_id), 0);
+
intel_de_write_fw(dev_priv, SPLINOFF(pipe, plane_id), linear_offset);
intel_de_write_fw(dev_priv, SPTILEOFF(pipe, plane_id), (y << 16) | x);
@@ -478,15 +490,15 @@ vlv_update_plane(struct intel_plane *plane,
intel_de_write_fw(dev_priv, SPSURF(pipe, plane_id),
intel_plane_ggtt_offset(plane_state) + sprsurf_offset);
- vlv_update_clrc(plane_state);
- vlv_update_gamma(plane_state);
+ vlv_sprite_update_clrc(plane_state);
+ vlv_sprite_update_gamma(plane_state);
spin_unlock_irqrestore(&dev_priv->uncore.lock, irqflags);
}
static void
-vlv_disable_plane(struct intel_plane *plane,
- const struct intel_crtc_state *crtc_state)
+vlv_sprite_disable_arm(struct intel_plane *plane,
+ const struct intel_crtc_state *crtc_state)
{
struct drm_i915_private *dev_priv = to_i915(plane->base.dev);
enum pipe pipe = plane->pipe;
@@ -502,8 +514,8 @@ vlv_disable_plane(struct intel_plane *plane,
}
static bool
-vlv_plane_get_hw_state(struct intel_plane *plane,
- enum pipe *pipe)
+vlv_sprite_get_hw_state(struct intel_plane *plane,
+ enum pipe *pipe)
{
struct drm_i915_private *dev_priv = to_i915(plane->base.dev);
enum intel_display_power_domain power_domain;
@@ -805,7 +817,7 @@ static void ivb_sprite_linear_gamma(const struct intel_plane_state *plane_state,
i++;
}
-static void ivb_update_gamma(const struct intel_plane_state *plane_state)
+static void ivb_sprite_update_gamma(const struct intel_plane_state *plane_state)
{
struct intel_plane *plane = to_intel_plane(plane_state->uapi.plane);
struct drm_i915_private *dev_priv = to_i915(plane->base.dev);
@@ -835,48 +847,56 @@ static void ivb_update_gamma(const struct intel_plane_state *plane_state)
}
static void
-ivb_update_plane(struct intel_plane *plane,
- const struct intel_crtc_state *crtc_state,
- const struct intel_plane_state *plane_state)
+ivb_sprite_update_noarm(struct intel_plane *plane,
+ const struct intel_crtc_state *crtc_state,
+ const struct intel_plane_state *plane_state)
{
struct drm_i915_private *dev_priv = to_i915(plane->base.dev);
enum pipe pipe = plane->pipe;
- u32 sprsurf_offset = plane_state->view.color_plane[0].offset;
- u32 linear_offset;
- const struct drm_intel_sprite_colorkey *key = &plane_state->ckey;
int crtc_x = plane_state->uapi.dst.x1;
int crtc_y = plane_state->uapi.dst.y1;
u32 crtc_w = drm_rect_width(&plane_state->uapi.dst);
u32 crtc_h = drm_rect_height(&plane_state->uapi.dst);
- u32 x = plane_state->view.color_plane[0].x;
- u32 y = plane_state->view.color_plane[0].y;
u32 src_w = drm_rect_width(&plane_state->uapi.src) >> 16;
u32 src_h = drm_rect_height(&plane_state->uapi.src) >> 16;
- u32 sprctl, sprscale = 0;
+ u32 sprscale = 0;
unsigned long irqflags;
- sprctl = plane_state->ctl | ivb_sprite_ctl_crtc(crtc_state);
-
- /* Sizes are 0 based */
- src_w--;
- src_h--;
- crtc_w--;
- crtc_h--;
-
if (crtc_w != src_w || crtc_h != src_h)
- sprscale = SPRITE_SCALE_ENABLE | (src_w << 16) | src_h;
-
- linear_offset = intel_fb_xy_to_linear(x, y, plane_state, 0);
+ sprscale = SPRITE_SCALE_ENABLE | ((src_w - 1) << 16) | (src_h - 1);
spin_lock_irqsave(&dev_priv->uncore.lock, irqflags);
intel_de_write_fw(dev_priv, SPRSTRIDE(pipe),
- plane_state->view.color_plane[0].stride);
+ plane_state->view.color_plane[0].mapping_stride);
intel_de_write_fw(dev_priv, SPRPOS(pipe), (crtc_y << 16) | crtc_x);
- intel_de_write_fw(dev_priv, SPRSIZE(pipe), (crtc_h << 16) | crtc_w);
+ intel_de_write_fw(dev_priv, SPRSIZE(pipe), ((crtc_h - 1) << 16) | (crtc_w - 1));
if (IS_IVYBRIDGE(dev_priv))
intel_de_write_fw(dev_priv, SPRSCALE(pipe), sprscale);
+ spin_unlock_irqrestore(&dev_priv->uncore.lock, irqflags);
+}
+
+static void
+ivb_sprite_update_arm(struct intel_plane *plane,
+ const struct intel_crtc_state *crtc_state,
+ const struct intel_plane_state *plane_state)
+{
+ struct drm_i915_private *dev_priv = to_i915(plane->base.dev);
+ enum pipe pipe = plane->pipe;
+ const struct drm_intel_sprite_colorkey *key = &plane_state->ckey;
+ u32 sprsurf_offset = plane_state->view.color_plane[0].offset;
+ u32 x = plane_state->view.color_plane[0].x;
+ u32 y = plane_state->view.color_plane[0].y;
+ u32 sprctl, linear_offset;
+ unsigned long irqflags;
+
+ sprctl = plane_state->ctl | ivb_sprite_ctl_crtc(crtc_state);
+
+ linear_offset = intel_fb_xy_to_linear(x, y, plane_state, 0);
+
+ spin_lock_irqsave(&dev_priv->uncore.lock, irqflags);
+
if (key->flags) {
intel_de_write_fw(dev_priv, SPRKEYVAL(pipe), key->min_value);
intel_de_write_fw(dev_priv, SPRKEYMSK(pipe),
@@ -902,14 +922,14 @@ ivb_update_plane(struct intel_plane *plane,
intel_de_write_fw(dev_priv, SPRSURF(pipe),
intel_plane_ggtt_offset(plane_state) + sprsurf_offset);
- ivb_update_gamma(plane_state);
+ ivb_sprite_update_gamma(plane_state);
spin_unlock_irqrestore(&dev_priv->uncore.lock, irqflags);
}
static void
-ivb_disable_plane(struct intel_plane *plane,
- const struct intel_crtc_state *crtc_state)
+ivb_sprite_disable_arm(struct intel_plane *plane,
+ const struct intel_crtc_state *crtc_state)
{
struct drm_i915_private *dev_priv = to_i915(plane->base.dev);
enum pipe pipe = plane->pipe;
@@ -927,8 +947,8 @@ ivb_disable_plane(struct intel_plane *plane,
}
static bool
-ivb_plane_get_hw_state(struct intel_plane *plane,
- enum pipe *pipe)
+ivb_sprite_get_hw_state(struct intel_plane *plane,
+ enum pipe *pipe)
{
struct drm_i915_private *dev_priv = to_i915(plane->base.dev);
enum intel_display_power_domain power_domain;
@@ -1106,7 +1126,7 @@ static u32 g4x_sprite_ctl(const struct intel_crtc_state *crtc_state,
return dvscntr;
}
-static void g4x_update_gamma(const struct intel_plane_state *plane_state)
+static void g4x_sprite_update_gamma(const struct intel_plane_state *plane_state)
{
struct intel_plane *plane = to_intel_plane(plane_state->uapi.plane);
struct drm_i915_private *dev_priv = to_i915(plane->base.dev);
@@ -1136,7 +1156,7 @@ static void ilk_sprite_linear_gamma(u16 gamma[17])
gamma[i] = (i << 10) / 16;
}
-static void ilk_update_gamma(const struct intel_plane_state *plane_state)
+static void ilk_sprite_update_gamma(const struct intel_plane_state *plane_state)
{
struct intel_plane *plane = to_intel_plane(plane_state->uapi.plane);
struct drm_i915_private *dev_priv = to_i915(plane->base.dev);
@@ -1163,47 +1183,55 @@ static void ilk_update_gamma(const struct intel_plane_state *plane_state)
}
static void
-g4x_update_plane(struct intel_plane *plane,
- const struct intel_crtc_state *crtc_state,
- const struct intel_plane_state *plane_state)
+g4x_sprite_update_noarm(struct intel_plane *plane,
+ const struct intel_crtc_state *crtc_state,
+ const struct intel_plane_state *plane_state)
{
struct drm_i915_private *dev_priv = to_i915(plane->base.dev);
enum pipe pipe = plane->pipe;
- u32 dvssurf_offset = plane_state->view.color_plane[0].offset;
- u32 linear_offset;
- const struct drm_intel_sprite_colorkey *key = &plane_state->ckey;
int crtc_x = plane_state->uapi.dst.x1;
int crtc_y = plane_state->uapi.dst.y1;
u32 crtc_w = drm_rect_width(&plane_state->uapi.dst);
u32 crtc_h = drm_rect_height(&plane_state->uapi.dst);
- u32 x = plane_state->view.color_plane[0].x;
- u32 y = plane_state->view.color_plane[0].y;
u32 src_w = drm_rect_width(&plane_state->uapi.src) >> 16;
u32 src_h = drm_rect_height(&plane_state->uapi.src) >> 16;
- u32 dvscntr, dvsscale = 0;
+ u32 dvsscale = 0;
unsigned long irqflags;
- dvscntr = plane_state->ctl | g4x_sprite_ctl_crtc(crtc_state);
-
- /* Sizes are 0 based */
- src_w--;
- src_h--;
- crtc_w--;
- crtc_h--;
-
if (crtc_w != src_w || crtc_h != src_h)
- dvsscale = DVS_SCALE_ENABLE | (src_w << 16) | src_h;
-
- linear_offset = intel_fb_xy_to_linear(x, y, plane_state, 0);
+ dvsscale = DVS_SCALE_ENABLE | ((src_w - 1) << 16) | (src_h - 1);
spin_lock_irqsave(&dev_priv->uncore.lock, irqflags);
intel_de_write_fw(dev_priv, DVSSTRIDE(pipe),
- plane_state->view.color_plane[0].stride);
+ plane_state->view.color_plane[0].mapping_stride);
intel_de_write_fw(dev_priv, DVSPOS(pipe), (crtc_y << 16) | crtc_x);
- intel_de_write_fw(dev_priv, DVSSIZE(pipe), (crtc_h << 16) | crtc_w);
+ intel_de_write_fw(dev_priv, DVSSIZE(pipe), ((crtc_h - 1) << 16) | (crtc_w - 1));
intel_de_write_fw(dev_priv, DVSSCALE(pipe), dvsscale);
+ spin_unlock_irqrestore(&dev_priv->uncore.lock, irqflags);
+}
+
+static void
+g4x_sprite_update_arm(struct intel_plane *plane,
+ const struct intel_crtc_state *crtc_state,
+ const struct intel_plane_state *plane_state)
+{
+ struct drm_i915_private *dev_priv = to_i915(plane->base.dev);
+ enum pipe pipe = plane->pipe;
+ const struct drm_intel_sprite_colorkey *key = &plane_state->ckey;
+ u32 dvssurf_offset = plane_state->view.color_plane[0].offset;
+ u32 x = plane_state->view.color_plane[0].x;
+ u32 y = plane_state->view.color_plane[0].y;
+ u32 dvscntr, linear_offset;
+ unsigned long irqflags;
+
+ dvscntr = plane_state->ctl | g4x_sprite_ctl_crtc(crtc_state);
+
+ linear_offset = intel_fb_xy_to_linear(x, y, plane_state, 0);
+
+ spin_lock_irqsave(&dev_priv->uncore.lock, irqflags);
+
if (key->flags) {
intel_de_write_fw(dev_priv, DVSKEYVAL(pipe), key->min_value);
intel_de_write_fw(dev_priv, DVSKEYMSK(pipe),
@@ -1224,16 +1252,16 @@ g4x_update_plane(struct intel_plane *plane,
intel_plane_ggtt_offset(plane_state) + dvssurf_offset);
if (IS_G4X(dev_priv))
- g4x_update_gamma(plane_state);
+ g4x_sprite_update_gamma(plane_state);
else
- ilk_update_gamma(plane_state);
+ ilk_sprite_update_gamma(plane_state);
spin_unlock_irqrestore(&dev_priv->uncore.lock, irqflags);
}
static void
-g4x_disable_plane(struct intel_plane *plane,
- const struct intel_crtc_state *crtc_state)
+g4x_sprite_disable_arm(struct intel_plane *plane,
+ const struct intel_crtc_state *crtc_state)
{
struct drm_i915_private *dev_priv = to_i915(plane->base.dev);
enum pipe pipe = plane->pipe;
@@ -1250,8 +1278,8 @@ g4x_disable_plane(struct intel_plane *plane,
}
static bool
-g4x_plane_get_hw_state(struct intel_plane *plane,
- enum pipe *pipe)
+g4x_sprite_get_hw_state(struct intel_plane *plane,
+ enum pipe *pipe)
{
struct drm_i915_private *dev_priv = to_i915(plane->base.dev);
enum intel_display_power_domain power_domain;
@@ -1299,7 +1327,7 @@ g4x_sprite_check_scaling(struct intel_crtc_state *crtc_state,
int src_x, src_w, src_h, crtc_w, crtc_h;
const struct drm_display_mode *adjusted_mode =
&crtc_state->hw.adjusted_mode;
- unsigned int stride = plane_state->view.color_plane[0].stride;
+ unsigned int stride = plane_state->view.color_plane[0].mapping_stride;
unsigned int cpp = fb->format->cpp[0];
unsigned int width_bytes;
int min_width, min_height;
@@ -1540,8 +1568,8 @@ int intel_sprite_set_colorkey_ioctl(struct drm_device *dev, void *data,
*/
if (!ret && has_dst_key_in_primary_plane(dev_priv)) {
struct intel_crtc *crtc =
- intel_get_crtc_for_pipe(dev_priv,
- to_intel_plane(plane)->pipe);
+ intel_crtc_for_pipe(dev_priv,
+ to_intel_plane(plane)->pipe);
plane_state = drm_atomic_get_plane_state(state,
crtc->base.primary);
@@ -1567,7 +1595,7 @@ int intel_sprite_set_colorkey_ioctl(struct drm_device *dev, void *data,
return ret;
}
-static const u32 g4x_plane_formats[] = {
+static const u32 g4x_sprite_formats[] = {
DRM_FORMAT_XRGB8888,
DRM_FORMAT_YUYV,
DRM_FORMAT_YVYU,
@@ -1575,13 +1603,7 @@ static const u32 g4x_plane_formats[] = {
DRM_FORMAT_VYUY,
};
-static const u64 i9xx_plane_format_modifiers[] = {
- I915_FORMAT_MOD_X_TILED,
- DRM_FORMAT_MOD_LINEAR,
- DRM_FORMAT_MOD_INVALID
-};
-
-static const u32 snb_plane_formats[] = {
+static const u32 snb_sprite_formats[] = {
DRM_FORMAT_XRGB8888,
DRM_FORMAT_XBGR8888,
DRM_FORMAT_XRGB2101010,
@@ -1594,7 +1616,7 @@ static const u32 snb_plane_formats[] = {
DRM_FORMAT_VYUY,
};
-static const u32 vlv_plane_formats[] = {
+static const u32 vlv_sprite_formats[] = {
DRM_FORMAT_C8,
DRM_FORMAT_RGB565,
DRM_FORMAT_XRGB8888,
@@ -1629,13 +1651,8 @@ static const u32 chv_pipe_b_sprite_formats[] = {
static bool g4x_sprite_format_mod_supported(struct drm_plane *_plane,
u32 format, u64 modifier)
{
- switch (modifier) {
- case DRM_FORMAT_MOD_LINEAR:
- case I915_FORMAT_MOD_X_TILED:
- break;
- default:
+ if (!intel_fb_plane_supports_modifier(to_intel_plane(_plane), modifier))
return false;
- }
switch (format) {
case DRM_FORMAT_XRGB8888:
@@ -1655,13 +1672,8 @@ static bool g4x_sprite_format_mod_supported(struct drm_plane *_plane,
static bool snb_sprite_format_mod_supported(struct drm_plane *_plane,
u32 format, u64 modifier)
{
- switch (modifier) {
- case DRM_FORMAT_MOD_LINEAR:
- case I915_FORMAT_MOD_X_TILED:
- break;
- default:
+ if (!intel_fb_plane_supports_modifier(to_intel_plane(_plane), modifier))
return false;
- }
switch (format) {
case DRM_FORMAT_XRGB8888:
@@ -1686,13 +1698,8 @@ static bool snb_sprite_format_mod_supported(struct drm_plane *_plane,
static bool vlv_sprite_format_mod_supported(struct drm_plane *_plane,
u32 format, u64 modifier)
{
- switch (modifier) {
- case DRM_FORMAT_MOD_LINEAR:
- case I915_FORMAT_MOD_X_TILED:
- break;
- default:
+ if (!intel_fb_plane_supports_modifier(to_intel_plane(_plane), modifier))
return false;
- }
switch (format) {
case DRM_FORMAT_C8:
@@ -1762,9 +1769,10 @@ intel_sprite_plane_create(struct drm_i915_private *dev_priv,
return plane;
if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv)) {
- plane->update_plane = vlv_update_plane;
- plane->disable_plane = vlv_disable_plane;
- plane->get_hw_state = vlv_plane_get_hw_state;
+ plane->update_noarm = vlv_sprite_update_noarm;
+ plane->update_arm = vlv_sprite_update_arm;
+ plane->disable_arm = vlv_sprite_disable_arm;
+ plane->get_hw_state = vlv_sprite_get_hw_state;
plane->check_plane = vlv_sprite_check;
plane->max_stride = i965_plane_max_stride;
plane->min_cdclk = vlv_plane_min_cdclk;
@@ -1773,16 +1781,16 @@ intel_sprite_plane_create(struct drm_i915_private *dev_priv,
formats = chv_pipe_b_sprite_formats;
num_formats = ARRAY_SIZE(chv_pipe_b_sprite_formats);
} else {
- formats = vlv_plane_formats;
- num_formats = ARRAY_SIZE(vlv_plane_formats);
+ formats = vlv_sprite_formats;
+ num_formats = ARRAY_SIZE(vlv_sprite_formats);
}
- modifiers = i9xx_plane_format_modifiers;
plane_funcs = &vlv_sprite_funcs;
} else if (DISPLAY_VER(dev_priv) >= 7) {
- plane->update_plane = ivb_update_plane;
- plane->disable_plane = ivb_disable_plane;
- plane->get_hw_state = ivb_plane_get_hw_state;
+ plane->update_noarm = ivb_sprite_update_noarm;
+ plane->update_arm = ivb_sprite_update_arm;
+ plane->disable_arm = ivb_sprite_disable_arm;
+ plane->get_hw_state = ivb_sprite_get_hw_state;
plane->check_plane = g4x_sprite_check;
if (IS_BROADWELL(dev_priv) || IS_HASWELL(dev_priv)) {
@@ -1793,28 +1801,27 @@ intel_sprite_plane_create(struct drm_i915_private *dev_priv,
plane->min_cdclk = ivb_sprite_min_cdclk;
}
- formats = snb_plane_formats;
- num_formats = ARRAY_SIZE(snb_plane_formats);
- modifiers = i9xx_plane_format_modifiers;
+ formats = snb_sprite_formats;
+ num_formats = ARRAY_SIZE(snb_sprite_formats);
plane_funcs = &snb_sprite_funcs;
} else {
- plane->update_plane = g4x_update_plane;
- plane->disable_plane = g4x_disable_plane;
- plane->get_hw_state = g4x_plane_get_hw_state;
+ plane->update_noarm = g4x_sprite_update_noarm;
+ plane->update_arm = g4x_sprite_update_arm;
+ plane->disable_arm = g4x_sprite_disable_arm;
+ plane->get_hw_state = g4x_sprite_get_hw_state;
plane->check_plane = g4x_sprite_check;
plane->max_stride = g4x_sprite_max_stride;
plane->min_cdclk = g4x_sprite_min_cdclk;
- modifiers = i9xx_plane_format_modifiers;
if (IS_SANDYBRIDGE(dev_priv)) {
- formats = snb_plane_formats;
- num_formats = ARRAY_SIZE(snb_plane_formats);
+ formats = snb_sprite_formats;
+ num_formats = ARRAY_SIZE(snb_sprite_formats);
plane_funcs = &snb_sprite_funcs;
} else {
- formats = g4x_plane_formats;
- num_formats = ARRAY_SIZE(g4x_plane_formats);
+ formats = g4x_sprite_formats;
+ num_formats = ARRAY_SIZE(g4x_sprite_formats);
plane_funcs = &g4x_sprite_funcs;
}
@@ -1833,11 +1840,15 @@ intel_sprite_plane_create(struct drm_i915_private *dev_priv,
plane->id = PLANE_SPRITE0 + sprite;
plane->frontbuffer_bit = INTEL_FRONTBUFFER(pipe, plane->id);
+ modifiers = intel_fb_plane_get_modifiers(dev_priv, INTEL_PLANE_CAP_TILING_X);
+
ret = drm_universal_plane_init(&dev_priv->drm, &plane->base,
0, plane_funcs,
formats, num_formats, modifiers,
DRM_PLANE_TYPE_OVERLAY,
"sprite %c", sprite_name(pipe, sprite));
+ kfree(modifiers);
+
if (ret)
goto fail;
diff --git a/drivers/gpu/drm/i915/display/intel_sprite.h b/drivers/gpu/drm/i915/display/intel_sprite.h
index c085eb8..4f63e49 100644
--- a/drivers/gpu/drm/i915/display/intel_sprite.h
+++ b/drivers/gpu/drm/i915/display/intel_sprite.h
@@ -27,14 +27,10 @@ struct intel_plane_state;
#define VBLANK_EVASION_TIME_US 100
#endif
-int intel_usecs_to_scanlines(const struct drm_display_mode *adjusted_mode,
- int usecs);
struct intel_plane *intel_sprite_plane_create(struct drm_i915_private *dev_priv,
enum pipe pipe, int plane);
int intel_sprite_set_colorkey_ioctl(struct drm_device *dev, void *data,
struct drm_file *file_priv);
-void intel_pipe_update_start(const struct intel_crtc_state *new_crtc_state);
-void intel_pipe_update_end(struct intel_crtc_state *new_crtc_state);
int intel_plane_check_src_coordinates(struct intel_plane_state *plane_state);
int chv_plane_check_rotation(const struct intel_plane_state *plane_state);
diff --git a/drivers/gpu/drm/i915/display/intel_tv.c b/drivers/gpu/drm/i915/display/intel_tv.c
index 88a398d..8a39989 100644
--- a/drivers/gpu/drm/i915/display/intel_tv.c
+++ b/drivers/gpu/drm/i915/display/intel_tv.c
@@ -36,6 +36,7 @@
#include "i915_drv.h"
#include "intel_connector.h"
+#include "intel_crtc.h"
#include "intel_de.h"
#include "intel_display_types.h"
#include "intel_hotplug.h"
@@ -924,8 +925,7 @@ intel_enable_tv(struct intel_atomic_state *state,
struct drm_i915_private *dev_priv = to_i915(dev);
/* Prevents vblank waits from timing out in intel_tv_detect_type() */
- intel_wait_for_vblank(dev_priv,
- to_intel_crtc(pipe_config->uapi.crtc)->pipe);
+ intel_crtc_wait_for_next_vblank(to_intel_crtc(pipe_config->uapi.crtc));
intel_de_write(dev_priv, TV_CTL,
intel_de_read(dev_priv, TV_CTL) | TV_ENC_ENABLE);
@@ -1618,7 +1618,7 @@ intel_tv_detect_type(struct intel_tv *intel_tv,
intel_de_write(dev_priv, TV_DAC, tv_dac);
intel_de_posting_read(dev_priv, TV_DAC);
- intel_wait_for_vblank(dev_priv, crtc->pipe);
+ intel_crtc_wait_for_next_vblank(crtc);
type = -1;
tv_dac = intel_de_read(dev_priv, TV_DAC);
@@ -1651,7 +1651,7 @@ intel_tv_detect_type(struct intel_tv *intel_tv,
intel_de_posting_read(dev_priv, TV_CTL);
/* For unknown reasons the hw barfs if we don't do this vblank wait. */
- intel_wait_for_vblank(dev_priv, crtc->pipe);
+ intel_crtc_wait_for_next_vblank(crtc);
/* Restore interrupt config */
if (connector->polled & DRM_CONNECTOR_POLL_HPD) {
diff --git a/drivers/gpu/drm/i915/display/intel_vbt_defs.h b/drivers/gpu/drm/i915/display/intel_vbt_defs.h
index a2108a8..f043d85 100644
--- a/drivers/gpu/drm/i915/display/intel_vbt_defs.h
+++ b/drivers/gpu/drm/i915/display/intel_vbt_defs.h
@@ -330,7 +330,12 @@ enum vbt_gmbus_ddi {
ADLS_DDC_BUS_PORT_TC1 = 0x2,
ADLS_DDC_BUS_PORT_TC2,
ADLS_DDC_BUS_PORT_TC3,
- ADLS_DDC_BUS_PORT_TC4
+ ADLS_DDC_BUS_PORT_TC4,
+ ADLP_DDC_BUS_PORT_TC1 = 0x3,
+ ADLP_DDC_BUS_PORT_TC2,
+ ADLP_DDC_BUS_PORT_TC3,
+ ADLP_DDC_BUS_PORT_TC4
+
};
#define DP_AUX_A 0x40
diff --git a/drivers/gpu/drm/i915/display/intel_vdsc.c b/drivers/gpu/drm/i915/display/intel_vdsc.c
index 2275f99..9b05f93 100644
--- a/drivers/gpu/drm/i915/display/intel_vdsc.c
+++ b/drivers/gpu/drm/i915/display/intel_vdsc.c
@@ -6,12 +6,14 @@
* Manasi Navare <manasi.d.navare@intel.com>
*/
#include <linux/limits.h>
+
#include "i915_drv.h"
+#include "intel_crtc.h"
#include "intel_de.h"
#include "intel_display_types.h"
#include "intel_dsi.h"
-#include "intel_vdsc.h"
#include "intel_qp_tables.h"
+#include "intel_vdsc.h"
enum ROW_INDEX_BPP {
ROW_INDEX_6BPP = 0,
@@ -442,10 +444,10 @@ calculate_rc_params(struct rc_parameters *rc,
}
}
-int intel_dsc_compute_params(struct intel_encoder *encoder,
- struct intel_crtc_state *pipe_config)
+int intel_dsc_compute_params(struct intel_crtc_state *pipe_config)
{
- struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
+ struct intel_crtc *crtc = to_intel_crtc(pipe_config->uapi.crtc);
+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
struct drm_dsc_config *vdsc_cfg = &pipe_config->dsc.config;
u16 compressed_bpp = pipe_config->dsc.compressed_bpp;
const struct rc_parameters *rc_params;
@@ -598,7 +600,7 @@ static void intel_dsc_pps_configure(const struct intel_crtc_state *crtc_state)
pps_val |= DSC_422_ENABLE;
if (vdsc_cfg->vbr_enable)
pps_val |= DSC_VBR_ENABLE;
- drm_info(&dev_priv->drm, "PPS0 = 0x%08x\n", pps_val);
+ drm_dbg_kms(&dev_priv->drm, "PPS0 = 0x%08x\n", pps_val);
if (!is_pipe_dsc(crtc, cpu_transcoder)) {
intel_de_write(dev_priv, DSCA_PICTURE_PARAMETER_SET_0,
pps_val);
@@ -622,7 +624,7 @@ static void intel_dsc_pps_configure(const struct intel_crtc_state *crtc_state)
/* Populate PICTURE_PARAMETER_SET_1 registers */
pps_val = 0;
pps_val |= DSC_BPP(vdsc_cfg->bits_per_pixel);
- drm_info(&dev_priv->drm, "PPS1 = 0x%08x\n", pps_val);
+ drm_dbg_kms(&dev_priv->drm, "PPS1 = 0x%08x\n", pps_val);
if (!is_pipe_dsc(crtc, cpu_transcoder)) {
intel_de_write(dev_priv, DSCA_PICTURE_PARAMETER_SET_1,
pps_val);
@@ -647,7 +649,7 @@ static void intel_dsc_pps_configure(const struct intel_crtc_state *crtc_state)
pps_val = 0;
pps_val |= DSC_PIC_HEIGHT(vdsc_cfg->pic_height) |
DSC_PIC_WIDTH(vdsc_cfg->pic_width / num_vdsc_instances);
- drm_info(&dev_priv->drm, "PPS2 = 0x%08x\n", pps_val);
+ drm_dbg_kms(&dev_priv->drm, "PPS2 = 0x%08x\n", pps_val);
if (!is_pipe_dsc(crtc, cpu_transcoder)) {
intel_de_write(dev_priv, DSCA_PICTURE_PARAMETER_SET_2,
pps_val);
@@ -672,7 +674,7 @@ static void intel_dsc_pps_configure(const struct intel_crtc_state *crtc_state)
pps_val = 0;
pps_val |= DSC_SLICE_HEIGHT(vdsc_cfg->slice_height) |
DSC_SLICE_WIDTH(vdsc_cfg->slice_width);
- drm_info(&dev_priv->drm, "PPS3 = 0x%08x\n", pps_val);
+ drm_dbg_kms(&dev_priv->drm, "PPS3 = 0x%08x\n", pps_val);
if (!is_pipe_dsc(crtc, cpu_transcoder)) {
intel_de_write(dev_priv, DSCA_PICTURE_PARAMETER_SET_3,
pps_val);
@@ -697,7 +699,7 @@ static void intel_dsc_pps_configure(const struct intel_crtc_state *crtc_state)
pps_val = 0;
pps_val |= DSC_INITIAL_XMIT_DELAY(vdsc_cfg->initial_xmit_delay) |
DSC_INITIAL_DEC_DELAY(vdsc_cfg->initial_dec_delay);
- drm_info(&dev_priv->drm, "PPS4 = 0x%08x\n", pps_val);
+ drm_dbg_kms(&dev_priv->drm, "PPS4 = 0x%08x\n", pps_val);
if (!is_pipe_dsc(crtc, cpu_transcoder)) {
intel_de_write(dev_priv, DSCA_PICTURE_PARAMETER_SET_4,
pps_val);
@@ -722,7 +724,7 @@ static void intel_dsc_pps_configure(const struct intel_crtc_state *crtc_state)
pps_val = 0;
pps_val |= DSC_SCALE_INC_INT(vdsc_cfg->scale_increment_interval) |
DSC_SCALE_DEC_INT(vdsc_cfg->scale_decrement_interval);
- drm_info(&dev_priv->drm, "PPS5 = 0x%08x\n", pps_val);
+ drm_dbg_kms(&dev_priv->drm, "PPS5 = 0x%08x\n", pps_val);
if (!is_pipe_dsc(crtc, cpu_transcoder)) {
intel_de_write(dev_priv, DSCA_PICTURE_PARAMETER_SET_5,
pps_val);
@@ -749,7 +751,7 @@ static void intel_dsc_pps_configure(const struct intel_crtc_state *crtc_state)
DSC_FIRST_LINE_BPG_OFFSET(vdsc_cfg->first_line_bpg_offset) |
DSC_FLATNESS_MIN_QP(vdsc_cfg->flatness_min_qp) |
DSC_FLATNESS_MAX_QP(vdsc_cfg->flatness_max_qp);
- drm_info(&dev_priv->drm, "PPS6 = 0x%08x\n", pps_val);
+ drm_dbg_kms(&dev_priv->drm, "PPS6 = 0x%08x\n", pps_val);
if (!is_pipe_dsc(crtc, cpu_transcoder)) {
intel_de_write(dev_priv, DSCA_PICTURE_PARAMETER_SET_6,
pps_val);
@@ -774,7 +776,7 @@ static void intel_dsc_pps_configure(const struct intel_crtc_state *crtc_state)
pps_val = 0;
pps_val |= DSC_SLICE_BPG_OFFSET(vdsc_cfg->slice_bpg_offset) |
DSC_NFL_BPG_OFFSET(vdsc_cfg->nfl_bpg_offset);
- drm_info(&dev_priv->drm, "PPS7 = 0x%08x\n", pps_val);
+ drm_dbg_kms(&dev_priv->drm, "PPS7 = 0x%08x\n", pps_val);
if (!is_pipe_dsc(crtc, cpu_transcoder)) {
intel_de_write(dev_priv, DSCA_PICTURE_PARAMETER_SET_7,
pps_val);
@@ -799,7 +801,7 @@ static void intel_dsc_pps_configure(const struct intel_crtc_state *crtc_state)
pps_val = 0;
pps_val |= DSC_FINAL_OFFSET(vdsc_cfg->final_offset) |
DSC_INITIAL_OFFSET(vdsc_cfg->initial_offset);
- drm_info(&dev_priv->drm, "PPS8 = 0x%08x\n", pps_val);
+ drm_dbg_kms(&dev_priv->drm, "PPS8 = 0x%08x\n", pps_val);
if (!is_pipe_dsc(crtc, cpu_transcoder)) {
intel_de_write(dev_priv, DSCA_PICTURE_PARAMETER_SET_8,
pps_val);
@@ -824,7 +826,7 @@ static void intel_dsc_pps_configure(const struct intel_crtc_state *crtc_state)
pps_val = 0;
pps_val |= DSC_RC_MODEL_SIZE(vdsc_cfg->rc_model_size) |
DSC_RC_EDGE_FACTOR(DSC_RC_EDGE_FACTOR_CONST);
- drm_info(&dev_priv->drm, "PPS9 = 0x%08x\n", pps_val);
+ drm_dbg_kms(&dev_priv->drm, "PPS9 = 0x%08x\n", pps_val);
if (!is_pipe_dsc(crtc, cpu_transcoder)) {
intel_de_write(dev_priv, DSCA_PICTURE_PARAMETER_SET_9,
pps_val);
@@ -851,7 +853,7 @@ static void intel_dsc_pps_configure(const struct intel_crtc_state *crtc_state)
DSC_RC_QUANT_INC_LIMIT1(vdsc_cfg->rc_quant_incr_limit1) |
DSC_RC_TARGET_OFF_HIGH(DSC_RC_TGT_OFFSET_HI_CONST) |
DSC_RC_TARGET_OFF_LOW(DSC_RC_TGT_OFFSET_LO_CONST);
- drm_info(&dev_priv->drm, "PPS10 = 0x%08x\n", pps_val);
+ drm_dbg_kms(&dev_priv->drm, "PPS10 = 0x%08x\n", pps_val);
if (!is_pipe_dsc(crtc, cpu_transcoder)) {
intel_de_write(dev_priv, DSCA_PICTURE_PARAMETER_SET_10,
pps_val);
@@ -879,7 +881,7 @@ static void intel_dsc_pps_configure(const struct intel_crtc_state *crtc_state)
vdsc_cfg->slice_width) |
DSC_SLICE_ROW_PER_FRAME(vdsc_cfg->pic_height /
vdsc_cfg->slice_height);
- drm_info(&dev_priv->drm, "PPS16 = 0x%08x\n", pps_val);
+ drm_dbg_kms(&dev_priv->drm, "PPS16 = 0x%08x\n", pps_val);
if (!is_pipe_dsc(crtc, cpu_transcoder)) {
intel_de_write(dev_priv, DSCA_PICTURE_PARAMETER_SET_16,
pps_val);
@@ -906,8 +908,8 @@ static void intel_dsc_pps_configure(const struct intel_crtc_state *crtc_state)
rc_buf_thresh_dword[i / 4] |=
(u32)(vdsc_cfg->rc_buf_thresh[i] <<
BITS_PER_BYTE * (i % 4));
- drm_info(&dev_priv->drm, " RC_BUF_THRESH%d = 0x%08x\n", i,
- rc_buf_thresh_dword[i / 4]);
+ drm_dbg_kms(&dev_priv->drm, "RC_BUF_THRESH_%d = 0x%08x\n", i,
+ rc_buf_thresh_dword[i / 4]);
}
if (!is_pipe_dsc(crtc, cpu_transcoder)) {
intel_de_write(dev_priv, DSCA_RC_BUF_THRESH_0,
@@ -963,8 +965,8 @@ static void intel_dsc_pps_configure(const struct intel_crtc_state *crtc_state)
RC_MAX_QP_SHIFT) |
(vdsc_cfg->rc_range_params[i].range_min_qp <<
RC_MIN_QP_SHIFT)) << 16 * (i % 2));
- drm_info(&dev_priv->drm, " RC_RANGE_PARAM_%d = 0x%08x\n", i,
- rc_range_params_dword[i / 2]);
+ drm_dbg_kms(&dev_priv->drm, "RC_RANGE_PARAM_%d = 0x%08x\n", i,
+ rc_range_params_dword[i / 2]);
}
if (!is_pipe_dsc(crtc, cpu_transcoder)) {
intel_de_write(dev_priv, DSCA_RC_RANGE_PARAMETERS_0,
@@ -1055,8 +1057,8 @@ static void intel_dsc_pps_configure(const struct intel_crtc_state *crtc_state)
}
}
-static void intel_dsc_dsi_pps_write(struct intel_encoder *encoder,
- const struct intel_crtc_state *crtc_state)
+void intel_dsc_dsi_pps_write(struct intel_encoder *encoder,
+ const struct intel_crtc_state *crtc_state)
{
const struct drm_dsc_config *vdsc_cfg = &crtc_state->dsc.config;
struct intel_dsi *intel_dsi = enc_to_intel_dsi(encoder);
@@ -1064,6 +1066,9 @@ static void intel_dsc_dsi_pps_write(struct intel_encoder *encoder,
struct drm_dsc_picture_parameter_set pps;
enum port port;
+ if (!crtc_state->dsc.compression_enable)
+ return;
+
drm_dsc_pps_payload_pack(&pps, vdsc_cfg);
for_each_dsi_port(port, intel_dsi->ports) {
@@ -1074,14 +1079,16 @@ static void intel_dsc_dsi_pps_write(struct intel_encoder *encoder,
}
}
-static void intel_dsc_dp_pps_write(struct intel_encoder *encoder,
- const struct intel_crtc_state *crtc_state)
+void intel_dsc_dp_pps_write(struct intel_encoder *encoder,
+ const struct intel_crtc_state *crtc_state)
{
- struct intel_dp *intel_dp = enc_to_intel_dp(encoder);
- struct intel_digital_port *dig_port = dp_to_dig_port(intel_dp);
+ struct intel_digital_port *dig_port = enc_to_dig_port(encoder);
const struct drm_dsc_config *vdsc_cfg = &crtc_state->dsc.config;
struct drm_dsc_pps_infoframe dp_dsc_pps_sdp;
+ if (!crtc_state->dsc.compression_enable)
+ return;
+
/* Prepare DP SDP PPS header as per DP 1.4 spec, Table 2-123 */
drm_dsc_dp_pps_header_init(&dp_dsc_pps_sdp.pps_header);
@@ -1105,25 +1112,16 @@ static i915_reg_t dss_ctl2_reg(struct intel_crtc *crtc, enum transcoder cpu_tran
ICL_PIPE_DSS_CTL2(crtc->pipe) : DSS_CTL2;
}
-static struct intel_crtc *
-_get_crtc_for_pipe(struct drm_i915_private *i915, enum pipe pipe)
-{
- if (!intel_pipe_valid(i915, pipe))
- return NULL;
-
- return intel_get_crtc_for_pipe(i915, pipe);
-}
-
struct intel_crtc *
intel_dsc_get_bigjoiner_secondary(const struct intel_crtc *primary_crtc)
{
- return _get_crtc_for_pipe(to_i915(primary_crtc->base.dev), primary_crtc->pipe + 1);
+ return intel_crtc_for_pipe(to_i915(primary_crtc->base.dev), primary_crtc->pipe + 1);
}
static struct intel_crtc *
intel_dsc_get_bigjoiner_primary(const struct intel_crtc *secondary_crtc)
{
- return _get_crtc_for_pipe(to_i915(secondary_crtc->base.dev), secondary_crtc->pipe - 1);
+ return intel_crtc_for_pipe(to_i915(secondary_crtc->base.dev), secondary_crtc->pipe - 1);
}
void intel_uncompressed_joiner_enable(const struct intel_crtc_state *crtc_state)
@@ -1142,8 +1140,7 @@ void intel_uncompressed_joiner_enable(const struct intel_crtc_state *crtc_state)
}
}
-void intel_dsc_enable(struct intel_encoder *encoder,
- const struct intel_crtc_state *crtc_state)
+void intel_dsc_enable(const struct intel_crtc_state *crtc_state)
{
struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
@@ -1155,13 +1152,6 @@ void intel_dsc_enable(struct intel_encoder *encoder,
intel_dsc_pps_configure(crtc_state);
- if (!crtc_state->bigjoiner_slave) {
- if (intel_crtc_has_type(crtc_state, INTEL_OUTPUT_DSI))
- intel_dsc_dsi_pps_write(encoder, crtc_state);
- else
- intel_dsc_dp_pps_write(encoder, crtc_state);
- }
-
dss_ctl2_val |= LEFT_BRANCH_VDSC_ENABLE;
if (crtc_state->dsc.dsc_split) {
dss_ctl2_val |= RIGHT_BRANCH_VDSC_ENABLE;
diff --git a/drivers/gpu/drm/i915/display/intel_vdsc.h b/drivers/gpu/drm/i915/display/intel_vdsc.h
index 0c5d80a..4ec75f7 100644
--- a/drivers/gpu/drm/i915/display/intel_vdsc.h
+++ b/drivers/gpu/drm/i915/display/intel_vdsc.h
@@ -15,15 +15,17 @@ struct intel_encoder;
bool intel_dsc_source_support(const struct intel_crtc_state *crtc_state);
void intel_uncompressed_joiner_enable(const struct intel_crtc_state *crtc_state);
-void intel_dsc_enable(struct intel_encoder *encoder,
- const struct intel_crtc_state *crtc_state);
+void intel_dsc_enable(const struct intel_crtc_state *crtc_state);
void intel_dsc_disable(const struct intel_crtc_state *crtc_state);
-int intel_dsc_compute_params(struct intel_encoder *encoder,
- struct intel_crtc_state *pipe_config);
+int intel_dsc_compute_params(struct intel_crtc_state *pipe_config);
void intel_uncompressed_joiner_get_config(struct intel_crtc_state *crtc_state);
void intel_dsc_get_config(struct intel_crtc_state *crtc_state);
enum intel_display_power_domain
intel_dsc_power_domain(struct intel_crtc *crtc, enum transcoder cpu_transcoder);
struct intel_crtc *intel_dsc_get_bigjoiner_secondary(const struct intel_crtc *primary_crtc);
+void intel_dsc_dsi_pps_write(struct intel_encoder *encoder,
+ const struct intel_crtc_state *crtc_state);
+void intel_dsc_dp_pps_write(struct intel_encoder *encoder,
+ const struct intel_crtc_state *crtc_state);
#endif /* __INTEL_VDSC_H__ */
diff --git a/drivers/gpu/drm/i915/display/intel_vrr.c b/drivers/gpu/drm/i915/display/intel_vrr.c
index c335b1d..139e893 100644
--- a/drivers/gpu/drm/i915/display/intel_vrr.c
+++ b/drivers/gpu/drm/i915/display/intel_vrr.c
@@ -60,7 +60,7 @@ intel_vrr_check_modeset(struct intel_atomic_state *state)
* Between those two points the vblank exit starts (and hence registers get
* latched) ASAP after a push is sent.
*
- * framestart_delay is programmable 0-3.
+ * framestart_delay is programmable 1-4.
*/
static int intel_vrr_vblank_exit_length(const struct intel_crtc_state *crtc_state)
{
@@ -138,13 +138,13 @@ intel_vrr_compute_config(struct intel_crtc_state *crtc_state,
i915->window2_delay;
else
/*
- * FIXME: s/4/framestart_delay+1/ to get consistent
+ * FIXME: s/4/framestart_delay/ to get consistent
* earliest/latest points for register latching regardless
* of the framestart_delay used?
*
* FIXME: this really needs the extra scanline to provide consistent
* behaviour for all framestart_delay values. Otherwise with
- * framestart_delay==3 we will end up extending the min vblank by
+ * framestart_delay==4 we will end up extending the min vblank by
* one extra line.
*/
crtc_state->vrr.pipeline_full =
@@ -193,6 +193,18 @@ void intel_vrr_send_push(const struct intel_crtc_state *crtc_state)
TRANS_PUSH_EN | TRANS_PUSH_SEND);
}
+bool intel_vrr_is_push_sent(const struct intel_crtc_state *crtc_state)
+{
+ struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ enum transcoder cpu_transcoder = crtc_state->cpu_transcoder;
+
+ if (!crtc_state->vrr.enable)
+ return false;
+
+ return intel_de_read(dev_priv, TRANS_PUSH(cpu_transcoder)) & TRANS_PUSH_SEND;
+}
+
void intel_vrr_disable(const struct intel_crtc_state *old_crtc_state)
{
struct intel_crtc *crtc = to_intel_crtc(old_crtc_state->uapi.crtc);
diff --git a/drivers/gpu/drm/i915/display/intel_vrr.h b/drivers/gpu/drm/i915/display/intel_vrr.h
index 96f9c9c..1c2da57 100644
--- a/drivers/gpu/drm/i915/display/intel_vrr.h
+++ b/drivers/gpu/drm/i915/display/intel_vrr.h
@@ -23,6 +23,7 @@ void intel_vrr_compute_config(struct intel_crtc_state *crtc_state,
void intel_vrr_enable(struct intel_encoder *encoder,
const struct intel_crtc_state *crtc_state);
void intel_vrr_send_push(const struct intel_crtc_state *crtc_state);
+bool intel_vrr_is_push_sent(const struct intel_crtc_state *crtc_state);
void intel_vrr_disable(const struct intel_crtc_state *old_crtc_state);
void intel_vrr_get_config(struct intel_crtc *crtc,
struct intel_crtc_state *crtc_state);
diff --git a/drivers/gpu/drm/i915/display/skl_scaler.c b/drivers/gpu/drm/i915/display/skl_scaler.c
index 37eabef..c2e9411 100644
--- a/drivers/gpu/drm/i915/display/skl_scaler.c
+++ b/drivers/gpu/drm/i915/display/skl_scaler.c
@@ -4,6 +4,7 @@
*/
#include "intel_de.h"
#include "intel_display_types.h"
+#include "intel_fb.h"
#include "skl_scaler.h"
#include "skl_universal_plane.h"
diff --git a/drivers/gpu/drm/i915/display/skl_universal_plane.c b/drivers/gpu/drm/i915/display/skl_universal_plane.c
index a0e53a3..93a3853 100644
--- a/drivers/gpu/drm/i915/display/skl_universal_plane.c
+++ b/drivers/gpu/drm/i915/display/skl_universal_plane.c
@@ -13,6 +13,7 @@
#include "intel_de.h"
#include "intel_display_types.h"
#include "intel_fb.h"
+#include "intel_fbc.h"
#include "intel_pm.h"
#include "intel_psr.h"
#include "intel_sprite.h"
@@ -163,50 +164,6 @@ static const u32 icl_hdr_plane_formats[] = {
DRM_FORMAT_XVYU16161616,
};
-static const u64 skl_plane_format_modifiers_noccs[] = {
- I915_FORMAT_MOD_Yf_TILED,
- I915_FORMAT_MOD_Y_TILED,
- I915_FORMAT_MOD_X_TILED,
- DRM_FORMAT_MOD_LINEAR,
- DRM_FORMAT_MOD_INVALID
-};
-
-static const u64 skl_plane_format_modifiers_ccs[] = {
- I915_FORMAT_MOD_Yf_TILED_CCS,
- I915_FORMAT_MOD_Y_TILED_CCS,
- I915_FORMAT_MOD_Yf_TILED,
- I915_FORMAT_MOD_Y_TILED,
- I915_FORMAT_MOD_X_TILED,
- DRM_FORMAT_MOD_LINEAR,
- DRM_FORMAT_MOD_INVALID
-};
-
-static const u64 gen12_plane_format_modifiers_mc_ccs[] = {
- I915_FORMAT_MOD_Y_TILED_GEN12_MC_CCS,
- I915_FORMAT_MOD_Y_TILED_GEN12_RC_CCS,
- I915_FORMAT_MOD_Y_TILED_GEN12_RC_CCS_CC,
- I915_FORMAT_MOD_Y_TILED,
- I915_FORMAT_MOD_X_TILED,
- DRM_FORMAT_MOD_LINEAR,
- DRM_FORMAT_MOD_INVALID
-};
-
-static const u64 gen12_plane_format_modifiers_rc_ccs[] = {
- I915_FORMAT_MOD_Y_TILED_GEN12_RC_CCS,
- I915_FORMAT_MOD_Y_TILED_GEN12_RC_CCS_CC,
- I915_FORMAT_MOD_Y_TILED,
- I915_FORMAT_MOD_X_TILED,
- DRM_FORMAT_MOD_LINEAR,
- DRM_FORMAT_MOD_INVALID
-};
-
-static const u64 adlp_step_a_plane_format_modifiers[] = {
- I915_FORMAT_MOD_Y_TILED,
- I915_FORMAT_MOD_X_TILED,
- DRM_FORMAT_MOD_LINEAR,
- DRM_FORMAT_MOD_INVALID
-};
-
int skl_format_to_fourcc(int format, bool rgb_order, bool alpha)
{
switch (format) {
@@ -464,9 +421,19 @@ static int icl_plane_min_width(const struct drm_framebuffer *fb,
}
}
-static int icl_plane_max_width(const struct drm_framebuffer *fb,
- int color_plane,
- unsigned int rotation)
+static int icl_hdr_plane_max_width(const struct drm_framebuffer *fb,
+ int color_plane,
+ unsigned int rotation)
+{
+ if (intel_format_info_is_yuv_semiplanar(fb->format, fb->modifier))
+ return 4096;
+ else
+ return 5120;
+}
+
+static int icl_sdr_plane_max_width(const struct drm_framebuffer *fb,
+ int color_plane,
+ unsigned int rotation)
{
return 5120;
}
@@ -633,7 +600,7 @@ static u32 skl_plane_stride(const struct intel_plane_state *plane_state,
{
const struct drm_framebuffer *fb = plane_state->hw.fb;
unsigned int rotation = plane_state->hw.rotation;
- u32 stride = plane_state->view.color_plane[color_plane].stride;
+ u32 stride = plane_state->view.color_plane[color_plane].scanout_stride;
if (color_plane >= fb->format->num_planes)
return 0;
@@ -642,8 +609,8 @@ static u32 skl_plane_stride(const struct intel_plane_state *plane_state,
}
static void
-skl_disable_plane(struct intel_plane *plane,
- const struct intel_crtc_state *crtc_state)
+skl_plane_disable_arm(struct intel_plane *plane,
+ const struct intel_crtc_state *crtc_state)
{
struct drm_i915_private *dev_priv = to_i915(plane->base.dev);
enum plane_id plane_id = plane->id;
@@ -716,13 +683,13 @@ static u32 skl_plane_ctl_format(u32 pixel_format)
case DRM_FORMAT_XYUV8888:
return PLANE_CTL_FORMAT_XYUV;
case DRM_FORMAT_YUYV:
- return PLANE_CTL_FORMAT_YUV422 | PLANE_CTL_YUV422_YUYV;
+ return PLANE_CTL_FORMAT_YUV422 | PLANE_CTL_YUV422_ORDER_YUYV;
case DRM_FORMAT_YVYU:
- return PLANE_CTL_FORMAT_YUV422 | PLANE_CTL_YUV422_YVYU;
+ return PLANE_CTL_FORMAT_YUV422 | PLANE_CTL_YUV422_ORDER_YVYU;
case DRM_FORMAT_UYVY:
- return PLANE_CTL_FORMAT_YUV422 | PLANE_CTL_YUV422_UYVY;
+ return PLANE_CTL_FORMAT_YUV422 | PLANE_CTL_YUV422_ORDER_UYVY;
case DRM_FORMAT_VYUY:
- return PLANE_CTL_FORMAT_YUV422 | PLANE_CTL_YUV422_VYUY;
+ return PLANE_CTL_FORMAT_YUV422 | PLANE_CTL_YUV422_ORDER_VYUY;
case DRM_FORMAT_NV12:
return PLANE_CTL_FORMAT_NV12;
case DRM_FORMAT_P010:
@@ -985,6 +952,9 @@ static u32 glk_plane_color_ctl(const struct intel_crtc_state *crtc_state,
plane_color_ctl |= PLANE_COLOR_YUV_RANGE_CORRECTION_DISABLE;
}
+ if (plane_state->force_black)
+ plane_color_ctl |= PLANE_COLOR_PLANE_CSC_ENABLE;
+
return plane_color_ctl;
}
@@ -1008,74 +978,60 @@ static u32 skl_surf_address(const struct intel_plane_state *plane_state,
}
}
-static void intel_load_plane_csc_black(struct intel_plane *intel_plane)
+static u32 skl_plane_surf(const struct intel_plane_state *plane_state,
+ int color_plane)
{
- struct drm_i915_private *dev_priv = to_i915(intel_plane->base.dev);
- enum pipe pipe = intel_plane->pipe;
- enum plane_id plane = intel_plane->id;
- u16 postoff = 0;
+ u32 plane_surf;
- drm_dbg_kms(&dev_priv->drm, "plane color CTM to black %s:%d\n",
- intel_plane->base.name, plane);
- intel_de_write_fw(dev_priv, PLANE_CSC_COEFF(pipe, plane, 0), 0);
- intel_de_write_fw(dev_priv, PLANE_CSC_COEFF(pipe, plane, 1), 0);
+ plane_surf = intel_plane_ggtt_offset(plane_state) +
+ skl_surf_address(plane_state, color_plane);
- intel_de_write_fw(dev_priv, PLANE_CSC_COEFF(pipe, plane, 2), 0);
- intel_de_write_fw(dev_priv, PLANE_CSC_COEFF(pipe, plane, 3), 0);
+ if (plane_state->decrypt)
+ plane_surf |= PLANE_SURF_DECRYPT;
- intel_de_write_fw(dev_priv, PLANE_CSC_COEFF(pipe, plane, 4), 0);
- intel_de_write_fw(dev_priv, PLANE_CSC_COEFF(pipe, plane, 5), 0);
+ return plane_surf;
+}
- intel_de_write_fw(dev_priv, PLANE_CSC_PREOFF(pipe, plane, 0), 0);
- intel_de_write_fw(dev_priv, PLANE_CSC_PREOFF(pipe, plane, 1), 0);
- intel_de_write_fw(dev_priv, PLANE_CSC_PREOFF(pipe, plane, 2), 0);
+static void icl_plane_csc_load_black(struct intel_plane *plane)
+{
+ struct drm_i915_private *i915 = to_i915(plane->base.dev);
+ enum plane_id plane_id = plane->id;
+ enum pipe pipe = plane->pipe;
- intel_de_write_fw(dev_priv, PLANE_CSC_POSTOFF(pipe, plane, 0), postoff);
- intel_de_write_fw(dev_priv, PLANE_CSC_POSTOFF(pipe, plane, 1), postoff);
- intel_de_write_fw(dev_priv, PLANE_CSC_POSTOFF(pipe, plane, 2), postoff);
+ intel_de_write_fw(i915, PLANE_CSC_COEFF(pipe, plane_id, 0), 0);
+ intel_de_write_fw(i915, PLANE_CSC_COEFF(pipe, plane_id, 1), 0);
+
+ intel_de_write_fw(i915, PLANE_CSC_COEFF(pipe, plane_id, 2), 0);
+ intel_de_write_fw(i915, PLANE_CSC_COEFF(pipe, plane_id, 3), 0);
+
+ intel_de_write_fw(i915, PLANE_CSC_COEFF(pipe, plane_id, 4), 0);
+ intel_de_write_fw(i915, PLANE_CSC_COEFF(pipe, plane_id, 5), 0);
+
+ intel_de_write_fw(i915, PLANE_CSC_PREOFF(pipe, plane_id, 0), 0);
+ intel_de_write_fw(i915, PLANE_CSC_PREOFF(pipe, plane_id, 1), 0);
+ intel_de_write_fw(i915, PLANE_CSC_PREOFF(pipe, plane_id, 2), 0);
+
+ intel_de_write_fw(i915, PLANE_CSC_POSTOFF(pipe, plane_id, 0), 0);
+ intel_de_write_fw(i915, PLANE_CSC_POSTOFF(pipe, plane_id, 1), 0);
+ intel_de_write_fw(i915, PLANE_CSC_POSTOFF(pipe, plane_id, 2), 0);
}
static void
-skl_program_plane(struct intel_plane *plane,
- const struct intel_crtc_state *crtc_state,
- const struct intel_plane_state *plane_state,
- int color_plane)
+skl_program_plane_noarm(struct intel_plane *plane,
+ const struct intel_crtc_state *crtc_state,
+ const struct intel_plane_state *plane_state,
+ int color_plane)
{
struct drm_i915_private *dev_priv = to_i915(plane->base.dev);
enum plane_id plane_id = plane->id;
enum pipe pipe = plane->pipe;
- const struct drm_intel_sprite_colorkey *key = &plane_state->ckey;
- u32 surf_addr = skl_surf_address(plane_state, color_plane);
u32 stride = skl_plane_stride(plane_state, color_plane);
const struct drm_framebuffer *fb = plane_state->hw.fb;
- int aux_plane = skl_main_to_aux_plane(fb, color_plane);
int crtc_x = plane_state->uapi.dst.x1;
int crtc_y = plane_state->uapi.dst.y1;
- u32 x = plane_state->view.color_plane[color_plane].x;
- u32 y = plane_state->view.color_plane[color_plane].y;
u32 src_w = drm_rect_width(&plane_state->uapi.src) >> 16;
u32 src_h = drm_rect_height(&plane_state->uapi.src) >> 16;
- u8 alpha = plane_state->hw.alpha >> 8;
- u32 plane_color_ctl = 0, aux_dist = 0;
unsigned long irqflags;
- u32 keymsk, keymax, plane_surf;
- u32 plane_ctl = plane_state->ctl;
-
- plane_ctl |= skl_plane_ctl_crtc(crtc_state);
-
- if (DISPLAY_VER(dev_priv) >= 10)
- plane_color_ctl = plane_state->color_ctl |
- glk_plane_color_ctl_crtc(crtc_state);
-
- /* Sizes are 0 based */
- src_w--;
- src_h--;
-
- keymax = (key->max_value & 0xffffff) | PLANE_KEYMAX_ALPHA(alpha);
-
- keymsk = key->channel_mask & 0x7ffffff;
- if (alpha < 0xff)
- keymsk |= PLANE_KEYMSK_ALPHA_ENABLE;
/* The scaler will handle the output position */
if (plane_state->scaler_id >= 0) {
@@ -1083,8 +1039,76 @@ skl_program_plane(struct intel_plane *plane,
crtc_y = 0;
}
+ spin_lock_irqsave(&dev_priv->uncore.lock, irqflags);
+
+ /*
+ * FIXME: pxp session invalidation can hit any time even at time of commit
+ * or after the commit, display content will be garbage.
+ */
+ if (plane_state->force_black)
+ icl_plane_csc_load_black(plane);
+
+ intel_de_write_fw(dev_priv, PLANE_STRIDE(pipe, plane_id), stride);
+ intel_de_write_fw(dev_priv, PLANE_POS(pipe, plane_id),
+ (crtc_y << 16) | crtc_x);
+ intel_de_write_fw(dev_priv, PLANE_SIZE(pipe, plane_id),
+ ((src_h - 1) << 16) | (src_w - 1));
+
+ if (intel_fb_is_rc_ccs_cc_modifier(fb->modifier)) {
+ intel_de_write_fw(dev_priv, PLANE_CC_VAL(pipe, plane_id, 0),
+ lower_32_bits(plane_state->ccval));
+ intel_de_write_fw(dev_priv, PLANE_CC_VAL(pipe, plane_id, 1),
+ upper_32_bits(plane_state->ccval));
+ }
+
+ if (icl_is_hdr_plane(dev_priv, plane_id))
+ intel_de_write_fw(dev_priv, PLANE_CUS_CTL(pipe, plane_id),
+ plane_state->cus_ctl);
+
+ if (fb->format->is_yuv && icl_is_hdr_plane(dev_priv, plane_id))
+ icl_program_input_csc(plane, crtc_state, plane_state);
+
+ skl_write_plane_wm(plane, crtc_state);
+
+ intel_psr2_program_plane_sel_fetch(plane, crtc_state, plane_state, color_plane);
+
+ spin_unlock_irqrestore(&dev_priv->uncore.lock, irqflags);
+}
+
+static void
+skl_program_plane_arm(struct intel_plane *plane,
+ const struct intel_crtc_state *crtc_state,
+ const struct intel_plane_state *plane_state,
+ int color_plane)
+{
+ struct drm_i915_private *dev_priv = to_i915(plane->base.dev);
+ enum plane_id plane_id = plane->id;
+ enum pipe pipe = plane->pipe;
+ const struct drm_intel_sprite_colorkey *key = &plane_state->ckey;
+ const struct drm_framebuffer *fb = plane_state->hw.fb;
+ int aux_plane = skl_main_to_aux_plane(fb, color_plane);
+ u32 x = plane_state->view.color_plane[color_plane].x;
+ u32 y = plane_state->view.color_plane[color_plane].y;
+ u32 keymsk, keymax, aux_dist = 0, plane_color_ctl = 0;
+ u8 alpha = plane_state->hw.alpha >> 8;
+ u32 plane_ctl = plane_state->ctl;
+ unsigned long irqflags;
+
+ plane_ctl |= skl_plane_ctl_crtc(crtc_state);
+
+ if (DISPLAY_VER(dev_priv) >= 10)
+ plane_color_ctl = plane_state->color_ctl |
+ glk_plane_color_ctl_crtc(crtc_state);
+
+ keymax = (key->max_value & 0xffffff) | PLANE_KEYMAX_ALPHA(alpha);
+
+ keymsk = key->channel_mask & 0x7ffffff;
+ if (alpha < 0xff)
+ keymsk |= PLANE_KEYMSK_ALPHA_ENABLE;
+
if (aux_plane) {
- aux_dist = skl_surf_address(plane_state, aux_plane) - surf_addr;
+ aux_dist = skl_surf_address(plane_state, aux_plane) -
+ skl_surf_address(plane_state, color_plane);
if (DISPLAY_VER(dev_priv) < 12)
aux_dist |= skl_plane_stride(plane_state, aux_plane);
@@ -1092,31 +1116,6 @@ skl_program_plane(struct intel_plane *plane,
spin_lock_irqsave(&dev_priv->uncore.lock, irqflags);
- intel_de_write_fw(dev_priv, PLANE_STRIDE(pipe, plane_id), stride);
- intel_de_write_fw(dev_priv, PLANE_POS(pipe, plane_id),
- (crtc_y << 16) | crtc_x);
- intel_de_write_fw(dev_priv, PLANE_SIZE(pipe, plane_id),
- (src_h << 16) | src_w);
-
- intel_de_write_fw(dev_priv, PLANE_AUX_DIST(pipe, plane_id), aux_dist);
-
- if (icl_is_hdr_plane(dev_priv, plane_id))
- intel_de_write_fw(dev_priv, PLANE_CUS_CTL(pipe, plane_id),
- plane_state->cus_ctl);
-
- if (DISPLAY_VER(dev_priv) >= 10)
- intel_de_write_fw(dev_priv, PLANE_COLOR_CTL(pipe, plane_id),
- plane_color_ctl);
-
- if (fb->format->is_yuv && icl_is_hdr_plane(dev_priv, plane_id))
- icl_program_input_csc(plane, crtc_state, plane_state);
-
- if (fb->modifier == I915_FORMAT_MOD_Y_TILED_GEN12_RC_CCS_CC)
- intel_uncore_write64_fw(&dev_priv->uncore,
- PLANE_CC_VAL(pipe, plane_id), plane_state->ccval);
-
- skl_write_plane_wm(plane, crtc_state);
-
intel_de_write_fw(dev_priv, PLANE_KEYVAL(pipe, plane_id),
key->min_value);
intel_de_write_fw(dev_priv, PLANE_KEYMSK(pipe, plane_id), keymsk);
@@ -1125,17 +1124,22 @@ skl_program_plane(struct intel_plane *plane,
intel_de_write_fw(dev_priv, PLANE_OFFSET(pipe, plane_id),
(y << 16) | x);
+ intel_de_write_fw(dev_priv, PLANE_AUX_DIST(pipe, plane_id), aux_dist);
+
if (DISPLAY_VER(dev_priv) < 11)
intel_de_write_fw(dev_priv, PLANE_AUX_OFFSET(pipe, plane_id),
(plane_state->view.color_plane[1].y << 16) |
plane_state->view.color_plane[1].x);
- intel_psr2_program_plane_sel_fetch(plane, crtc_state, plane_state, color_plane);
+ if (DISPLAY_VER(dev_priv) >= 10)
+ intel_de_write_fw(dev_priv, PLANE_COLOR_CTL(pipe, plane_id), plane_color_ctl);
/*
* Enable the scaler before the plane so that we don't
* get a catastrophic underrun even if the two operations
* end up happening in two different frames.
+ *
+ * TODO: split into noarm+arm pair
*/
if (plane_state->scaler_id >= 0)
skl_program_plane_scaler(plane, crtc_state, plane_state);
@@ -1146,23 +1150,8 @@ skl_program_plane(struct intel_plane *plane,
* the control register just before the surface register.
*/
intel_de_write_fw(dev_priv, PLANE_CTL(pipe, plane_id), plane_ctl);
- plane_surf = intel_plane_ggtt_offset(plane_state) + surf_addr;
- plane_color_ctl = intel_de_read_fw(dev_priv, PLANE_COLOR_CTL(pipe, plane_id));
-
- /*
- * FIXME: pxp session invalidation can hit any time even at time of commit
- * or after the commit, display content will be garbage.
- */
- if (plane_state->decrypt) {
- plane_surf |= PLANE_SURF_DECRYPT;
- } else if (plane_state->force_black) {
- intel_load_plane_csc_black(plane);
- plane_color_ctl |= PLANE_COLOR_PLANE_CSC_ENABLE;
- }
-
- intel_de_write_fw(dev_priv, PLANE_COLOR_CTL(pipe, plane_id),
- plane_color_ctl);
- intel_de_write_fw(dev_priv, PLANE_SURF(pipe, plane_id), plane_surf);
+ intel_de_write_fw(dev_priv, PLANE_SURF(pipe, plane_id),
+ skl_plane_surf(plane_state, color_plane));
spin_unlock_irqrestore(&dev_priv->uncore.lock, irqflags);
}
@@ -1177,7 +1166,6 @@ skl_plane_async_flip(struct intel_plane *plane,
unsigned long irqflags;
enum plane_id plane_id = plane->id;
enum pipe pipe = plane->pipe;
- u32 surf_addr = plane_state->view.color_plane[0].offset;
u32 plane_ctl = plane_state->ctl;
plane_ctl |= skl_plane_ctl_crtc(crtc_state);
@@ -1189,15 +1177,15 @@ skl_plane_async_flip(struct intel_plane *plane,
intel_de_write_fw(dev_priv, PLANE_CTL(pipe, plane_id), plane_ctl);
intel_de_write_fw(dev_priv, PLANE_SURF(pipe, plane_id),
- intel_plane_ggtt_offset(plane_state) + surf_addr);
+ skl_plane_surf(plane_state, 0));
spin_unlock_irqrestore(&dev_priv->uncore.lock, irqflags);
}
static void
-skl_update_plane(struct intel_plane *plane,
- const struct intel_crtc_state *crtc_state,
- const struct intel_plane_state *plane_state)
+skl_plane_update_noarm(struct intel_plane *plane,
+ const struct intel_crtc_state *crtc_state,
+ const struct intel_plane_state *plane_state)
{
int color_plane = 0;
@@ -1205,7 +1193,21 @@ skl_update_plane(struct intel_plane *plane,
/* Program the UV plane on planar master */
color_plane = 1;
- skl_program_plane(plane, crtc_state, plane_state, color_plane);
+ skl_program_plane_noarm(plane, crtc_state, plane_state, color_plane);
+}
+
+static void
+skl_plane_update_arm(struct intel_plane *plane,
+ const struct intel_crtc_state *crtc_state,
+ const struct intel_plane_state *plane_state)
+{
+ int color_plane = 0;
+
+ if (plane_state->planar_linked_plane && !plane_state->planar_slave)
+ /* Program the UV plane on planar master */
+ color_plane = 1;
+
+ skl_program_plane_arm(plane, crtc_state, plane_state, color_plane);
}
static bool intel_format_is_p01x(u32 format)
@@ -1232,7 +1234,7 @@ static int skl_plane_check_fb(const struct intel_crtc_state *crtc_state,
return 0;
if (rotation & ~(DRM_MODE_ROTATE_0 | DRM_MODE_ROTATE_180) &&
- is_ccs_modifier(fb->modifier)) {
+ intel_fb_is_ccs_modifier(fb->modifier)) {
drm_dbg_kms(&dev_priv->drm,
"RC support only with 0/180 degree rotation (%x)\n",
rotation);
@@ -1284,13 +1286,8 @@ static int skl_plane_check_fb(const struct intel_crtc_state *crtc_state,
/* Y-tiling is not supported in IF-ID Interlace mode */
if (crtc_state->hw.enable &&
crtc_state->hw.adjusted_mode.flags & DRM_MODE_FLAG_INTERLACE &&
- (fb->modifier == I915_FORMAT_MOD_Y_TILED ||
- fb->modifier == I915_FORMAT_MOD_Yf_TILED ||
- fb->modifier == I915_FORMAT_MOD_Y_TILED_CCS ||
- fb->modifier == I915_FORMAT_MOD_Yf_TILED_CCS ||
- fb->modifier == I915_FORMAT_MOD_Y_TILED_GEN12_RC_CCS ||
- fb->modifier == I915_FORMAT_MOD_Y_TILED_GEN12_MC_CCS ||
- fb->modifier == I915_FORMAT_MOD_Y_TILED_GEN12_RC_CCS_CC)) {
+ fb->modifier != DRM_FORMAT_MOD_LINEAR &&
+ fb->modifier != I915_FORMAT_MOD_X_TILED) {
drm_dbg_kms(&dev_priv->drm,
"Y/Yf tiling not supported in IF-ID mode\n");
return -EINVAL;
@@ -1487,7 +1484,7 @@ int skl_calc_main_surface_offset(const struct intel_plane_state *plane_state,
if (fb->modifier == I915_FORMAT_MOD_X_TILED) {
int cpp = fb->format->cpp[0];
- while ((*x + w) * cpp > plane_state->view.color_plane[0].stride) {
+ while ((*x + w) * cpp > plane_state->view.color_plane[0].mapping_stride) {
if (*offset == 0) {
drm_dbg_kms(&dev_priv->drm,
"Unable to find suitable display surface offset due to X-tiling\n");
@@ -1536,7 +1533,7 @@ static int skl_check_main_surface(struct intel_plane_state *plane_state)
* CCS AUX surface doesn't have its own x/y offsets, we must make sure
* they match with the main surface x/y offsets.
*/
- if (is_ccs_modifier(fb->modifier)) {
+ if (intel_fb_is_ccs_modifier(fb->modifier)) {
while (!skl_check_main_ccs_coordinates(plane_state, x, y,
offset, aux_plane)) {
if (offset == 0)
@@ -1600,7 +1597,7 @@ static int skl_check_nv12_aux_surface(struct intel_plane_state *plane_state)
offset = intel_plane_compute_aligned_offset(&x, &y,
plane_state, uv_plane);
- if (is_ccs_modifier(fb->modifier)) {
+ if (intel_fb_is_ccs_modifier(fb->modifier)) {
int ccs_plane = main_to_ccs_plane(fb, uv_plane);
u32 aux_offset = plane_state->view.color_plane[ccs_plane].offset;
u32 alignment = intel_surf_alignment(fb, uv_plane);
@@ -1656,8 +1653,7 @@ static int skl_check_ccs_aux_surface(struct intel_plane_state *plane_state)
int hsub, vsub;
int x, y;
- if (!is_ccs_plane(fb, ccs_plane) ||
- is_gen12_ccs_cc_plane(fb, ccs_plane))
+ if (!intel_fb_is_ccs_aux_plane(fb, ccs_plane))
continue;
intel_fb_plane_get_subsampling(&main_hsub, &main_vsub, fb,
@@ -1699,7 +1695,7 @@ static int skl_check_plane_surface(struct intel_plane_state *plane_state)
* Handle the AUX surface first since the main surface setup depends on
* it.
*/
- if (is_ccs_modifier(fb->modifier)) {
+ if (intel_fb_is_ccs_modifier(fb->modifier)) {
ret = skl_check_ccs_aux_surface(plane_state);
if (ret)
return ret;
@@ -1737,6 +1733,18 @@ static bool skl_fb_scalable(const struct drm_framebuffer *fb)
}
}
+static bool bo_has_valid_encryption(struct drm_i915_gem_object *obj)
+{
+ struct drm_i915_private *i915 = to_i915(obj->base.dev);
+
+ return intel_pxp_key_check(&to_gt(i915)->pxp, obj, false) == 0;
+}
+
+static bool pxp_is_borked(struct drm_i915_gem_object *obj)
+{
+ return i915_gem_object_is_protected(obj) && !bo_has_valid_encryption(obj);
+}
+
static int skl_plane_check(struct intel_crtc_state *crtc_state,
struct intel_plane_state *plane_state)
{
@@ -1781,6 +1789,11 @@ static int skl_plane_check(struct intel_crtc_state *crtc_state,
if (ret)
return ret;
+ if (DISPLAY_VER(dev_priv) >= 11) {
+ plane_state->decrypt = bo_has_valid_encryption(intel_fb_obj(fb));
+ plane_state->force_black = pxp_is_borked(intel_fb_obj(fb));
+ }
+
/* HW only has 8 bits pixel precision, disable plane if invisible */
if (!(plane_state->hw.alpha >> 8))
plane_state->uapi.visible = false;
@@ -1812,6 +1825,15 @@ static bool skl_plane_has_fbc(struct drm_i915_private *dev_priv,
return pipe == PIPE_A && plane_id == PLANE_PRIMARY;
}
+static struct intel_fbc *skl_plane_fbc(struct drm_i915_private *dev_priv,
+ enum pipe pipe, enum plane_id plane_id)
+{
+ if (skl_plane_has_fbc(dev_priv, pipe, plane_id))
+ return dev_priv->fbc;
+ else
+ return NULL;
+}
+
static bool skl_plane_has_planar(struct drm_i915_private *dev_priv,
enum pipe pipe, enum plane_id plane_id)
{
@@ -1870,49 +1892,20 @@ static const u32 *icl_get_plane_formats(struct drm_i915_private *dev_priv,
}
}
-static bool skl_plane_has_ccs(struct drm_i915_private *dev_priv,
- enum pipe pipe, enum plane_id plane_id)
-{
- if (plane_id == PLANE_CURSOR)
- return false;
-
- if (DISPLAY_VER(dev_priv) >= 11)
- return true;
-
- if (IS_GEMINILAKE(dev_priv))
- return pipe != PIPE_C;
-
- return pipe != PIPE_C &&
- (plane_id == PLANE_PRIMARY ||
- plane_id == PLANE_SPRITE0);
-}
-
static bool skl_plane_format_mod_supported(struct drm_plane *_plane,
u32 format, u64 modifier)
{
struct intel_plane *plane = to_intel_plane(_plane);
- switch (modifier) {
- case DRM_FORMAT_MOD_LINEAR:
- case I915_FORMAT_MOD_X_TILED:
- case I915_FORMAT_MOD_Y_TILED:
- case I915_FORMAT_MOD_Yf_TILED:
- break;
- case I915_FORMAT_MOD_Y_TILED_CCS:
- case I915_FORMAT_MOD_Yf_TILED_CCS:
- if (!plane->has_ccs)
- return false;
- break;
- default:
+ if (!intel_fb_plane_supports_modifier(plane, modifier))
return false;
- }
switch (format) {
case DRM_FORMAT_XRGB8888:
case DRM_FORMAT_XBGR8888:
case DRM_FORMAT_ARGB8888:
case DRM_FORMAT_ABGR8888:
- if (is_ccs_modifier(modifier))
+ if (intel_fb_is_ccs_modifier(modifier))
return true;
fallthrough;
case DRM_FORMAT_RGB565:
@@ -1953,52 +1946,20 @@ static bool skl_plane_format_mod_supported(struct drm_plane *_plane,
}
}
-static bool gen12_plane_supports_mc_ccs(struct drm_i915_private *dev_priv,
- enum plane_id plane_id)
-{
- /* Wa_14010477008:tgl[a0..c0],rkl[all],dg1[all] */
- if (IS_DG1(dev_priv) || IS_ROCKETLAKE(dev_priv) ||
- IS_TGL_DISPLAY_STEP(dev_priv, STEP_A0, STEP_D0))
- return false;
-
- /* Wa_22011186057 */
- if (IS_ADLP_DISPLAY_STEP(dev_priv, STEP_A0, STEP_B0))
- return false;
-
- return plane_id < PLANE_SPRITE4;
-}
-
static bool gen12_plane_format_mod_supported(struct drm_plane *_plane,
u32 format, u64 modifier)
{
- struct drm_i915_private *dev_priv = to_i915(_plane->dev);
struct intel_plane *plane = to_intel_plane(_plane);
- switch (modifier) {
- case I915_FORMAT_MOD_Y_TILED_GEN12_MC_CCS:
- if (!gen12_plane_supports_mc_ccs(dev_priv, plane->id))
- return false;
- fallthrough;
- case DRM_FORMAT_MOD_LINEAR:
- case I915_FORMAT_MOD_X_TILED:
- case I915_FORMAT_MOD_Y_TILED:
- break;
- case I915_FORMAT_MOD_Y_TILED_GEN12_RC_CCS:
- case I915_FORMAT_MOD_Y_TILED_GEN12_RC_CCS_CC:
- /* Wa_22011186057 */
- if (IS_ADLP_DISPLAY_STEP(dev_priv, STEP_A0, STEP_B0))
- return false;
- break;
- default:
+ if (!intel_fb_plane_supports_modifier(plane, modifier))
return false;
- }
switch (format) {
case DRM_FORMAT_XRGB8888:
case DRM_FORMAT_XBGR8888:
case DRM_FORMAT_ARGB8888:
case DRM_FORMAT_ABGR8888:
- if (is_ccs_modifier(modifier))
+ if (intel_fb_is_ccs_modifier(modifier))
return true;
fallthrough;
case DRM_FORMAT_YUYV:
@@ -2010,7 +1971,7 @@ static bool gen12_plane_format_mod_supported(struct drm_plane *_plane,
case DRM_FORMAT_P010:
case DRM_FORMAT_P012:
case DRM_FORMAT_P016:
- if (modifier == I915_FORMAT_MOD_Y_TILED_GEN12_MC_CCS)
+ if (intel_fb_is_mc_ccs_modifier(modifier))
return true;
fallthrough;
case DRM_FORMAT_RGB565:
@@ -2039,18 +2000,6 @@ static bool gen12_plane_format_mod_supported(struct drm_plane *_plane,
}
}
-static const u64 *gen12_get_plane_modifiers(struct drm_i915_private *dev_priv,
- enum plane_id plane_id)
-{
- /* Wa_22011186057 */
- if (IS_ADLP_DISPLAY_STEP(dev_priv, STEP_A0, STEP_B0))
- return adlp_step_a_plane_format_modifiers;
- else if (gen12_plane_supports_mc_ccs(dev_priv, plane_id))
- return gen12_plane_format_modifiers_mc_ccs;
- else
- return gen12_plane_format_modifiers_rc_ccs;
-}
-
static const struct drm_plane_funcs skl_plane_funcs = {
.update_plane = drm_atomic_helper_update_plane,
.disable_plane = drm_atomic_helper_disable_plane,
@@ -2091,6 +2040,64 @@ skl_plane_disable_flip_done(struct intel_plane *plane)
spin_unlock_irq(&i915->irq_lock);
}
+static bool skl_plane_has_rc_ccs(struct drm_i915_private *i915,
+ enum pipe pipe, enum plane_id plane_id)
+{
+ /* Wa_22011186057 */
+ if (IS_ADLP_DISPLAY_STEP(i915, STEP_A0, STEP_B0))
+ return false;
+
+ if (DISPLAY_VER(i915) >= 11)
+ return true;
+
+ if (IS_GEMINILAKE(i915))
+ return pipe != PIPE_C;
+
+ return pipe != PIPE_C &&
+ (plane_id == PLANE_PRIMARY ||
+ plane_id == PLANE_SPRITE0);
+}
+
+static bool gen12_plane_has_mc_ccs(struct drm_i915_private *i915,
+ enum plane_id plane_id)
+{
+ if (DISPLAY_VER(i915) < 12)
+ return false;
+
+ /* Wa_14010477008:tgl[a0..c0],rkl[all],dg1[all] */
+ if (IS_DG1(i915) || IS_ROCKETLAKE(i915) ||
+ IS_TGL_DISPLAY_STEP(i915, STEP_A0, STEP_D0))
+ return false;
+
+ /* Wa_22011186057 */
+ if (IS_ADLP_DISPLAY_STEP(i915, STEP_A0, STEP_B0))
+ return false;
+
+ return plane_id < PLANE_SPRITE4;
+}
+
+static u8 skl_get_plane_caps(struct drm_i915_private *i915,
+ enum pipe pipe, enum plane_id plane_id)
+{
+ u8 caps = INTEL_PLANE_CAP_TILING_X;
+
+ if (DISPLAY_VER(i915) < 13 || IS_ALDERLAKE_P(i915))
+ caps |= INTEL_PLANE_CAP_TILING_Y;
+ if (DISPLAY_VER(i915) < 12)
+ caps |= INTEL_PLANE_CAP_TILING_Yf;
+
+ if (skl_plane_has_rc_ccs(i915, pipe, plane_id)) {
+ caps |= INTEL_PLANE_CAP_CCS_RC;
+ if (DISPLAY_VER(i915) >= 12)
+ caps |= INTEL_PLANE_CAP_CCS_RC_CC;
+ }
+
+ if (gen12_plane_has_mc_ccs(i915, plane_id))
+ caps |= INTEL_PLANE_CAP_CCS_MC;
+
+ return caps;
+}
+
struct intel_plane *
skl_universal_plane_create(struct drm_i915_private *dev_priv,
enum pipe pipe, enum plane_id plane_id)
@@ -2113,16 +2120,14 @@ skl_universal_plane_create(struct drm_i915_private *dev_priv,
plane->id = plane_id;
plane->frontbuffer_bit = INTEL_FRONTBUFFER(pipe, plane_id);
- plane->has_fbc = skl_plane_has_fbc(dev_priv, pipe, plane_id);
- if (plane->has_fbc) {
- struct intel_fbc *fbc = &dev_priv->fbc;
-
- fbc->possible_framebuffer_bits |= plane->frontbuffer_bit;
- }
+ intel_fbc_add_plane(skl_plane_fbc(dev_priv, pipe, plane_id), plane);
if (DISPLAY_VER(dev_priv) >= 11) {
plane->min_width = icl_plane_min_width;
- plane->max_width = icl_plane_max_width;
+ if (icl_is_hdr_plane(dev_priv, plane_id))
+ plane->max_width = icl_hdr_plane_max_width;
+ else
+ plane->max_width = icl_sdr_plane_max_width;
plane->max_height = icl_plane_max_height;
plane->min_cdclk = icl_plane_min_cdclk;
} else if (DISPLAY_VER(dev_priv) >= 10) {
@@ -2136,8 +2141,9 @@ skl_universal_plane_create(struct drm_i915_private *dev_priv,
}
plane->max_stride = skl_plane_max_stride;
- plane->update_plane = skl_update_plane;
- plane->disable_plane = skl_disable_plane;
+ plane->update_noarm = skl_plane_update_noarm;
+ plane->update_arm = skl_plane_update_arm;
+ plane->disable_arm = skl_plane_disable_arm;
plane->get_hw_state = skl_plane_get_hw_state;
plane->check_plane = skl_plane_check;
@@ -2159,29 +2165,28 @@ skl_universal_plane_create(struct drm_i915_private *dev_priv,
formats = skl_get_plane_formats(dev_priv, pipe,
plane_id, &num_formats);
- plane->has_ccs = skl_plane_has_ccs(dev_priv, pipe, plane_id);
- if (DISPLAY_VER(dev_priv) >= 12) {
- modifiers = gen12_get_plane_modifiers(dev_priv, plane_id);
+ if (DISPLAY_VER(dev_priv) >= 12)
plane_funcs = &gen12_plane_funcs;
- } else {
- if (plane->has_ccs)
- modifiers = skl_plane_format_modifiers_ccs;
- else
- modifiers = skl_plane_format_modifiers_noccs;
+ else
plane_funcs = &skl_plane_funcs;
- }
if (plane_id == PLANE_PRIMARY)
plane_type = DRM_PLANE_TYPE_PRIMARY;
else
plane_type = DRM_PLANE_TYPE_OVERLAY;
+ modifiers = intel_fb_plane_get_modifiers(dev_priv,
+ skl_get_plane_caps(dev_priv, pipe, plane_id));
+
ret = drm_universal_plane_init(&dev_priv->drm, &plane->base,
0, plane_funcs,
formats, num_formats, modifiers,
plane_type,
"plane %d%c", plane_id + 1,
pipe_name(pipe));
+
+ kfree(modifiers);
+
if (ret)
goto fail;
diff --git a/drivers/gpu/drm/i915/display/vlv_dsi.c b/drivers/gpu/drm/i915/display/vlv_dsi.c
index 0758469..20141f3 100644
--- a/drivers/gpu/drm/i915/display/vlv_dsi.c
+++ b/drivers/gpu/drm/i915/display/vlv_dsi.c
@@ -38,9 +38,12 @@
#include "intel_de.h"
#include "intel_display_types.h"
#include "intel_dsi.h"
+#include "intel_dsi_vbt.h"
#include "intel_fifo_underrun.h"
#include "intel_panel.h"
#include "skl_scaler.h"
+#include "vlv_dsi.h"
+#include "vlv_dsi_pll.h"
#include "vlv_sideband.h"
/* return pixels in terms of txbyteclkhs */
@@ -1258,7 +1261,9 @@ static void intel_dsi_get_config(struct intel_encoder *encoder,
struct intel_crtc_state *pipe_config)
{
struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
+ struct intel_dsi *intel_dsi = enc_to_intel_dsi(encoder);
u32 pclk;
+
drm_dbg_kms(&dev_priv->drm, "\n");
pipe_config->output_types |= BIT(INTEL_OUTPUT_DSI);
@@ -1270,6 +1275,9 @@ static void intel_dsi_get_config(struct intel_encoder *encoder,
pclk = vlv_dsi_get_pclk(encoder, pipe_config);
}
+ if (intel_dsi->dual_link)
+ pclk *= 2;
+
if (pclk) {
pipe_config->hw.adjusted_mode.crtc_clock = pclk;
pipe_config->port_clock = pclk;
diff --git a/drivers/gpu/drm/i915/display/vlv_dsi.h b/drivers/gpu/drm/i915/display/vlv_dsi.h
new file mode 100644
index 0000000..0c2b279
--- /dev/null
+++ b/drivers/gpu/drm/i915/display/vlv_dsi.h
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2021 Intel Corporation
+ */
+
+#ifndef __VLV_DSI_H__
+#define __VLV_DSI_H__
+
+#include <linux/types.h>
+
+enum port;
+struct drm_i915_private;
+struct intel_dsi;
+
+void vlv_dsi_wait_for_fifo_empty(struct intel_dsi *intel_dsi, enum port port);
+enum mipi_dsi_pixel_format pixel_format_from_register_bits(u32 fmt);
+void vlv_dsi_init(struct drm_i915_private *dev_priv);
+
+#endif /* __VLV_DSI_H__ */
diff --git a/drivers/gpu/drm/i915/display/vlv_dsi_pll.c b/drivers/gpu/drm/i915/display/vlv_dsi_pll.c
index 5413b52..1b81797 100644
--- a/drivers/gpu/drm/i915/display/vlv_dsi_pll.c
+++ b/drivers/gpu/drm/i915/display/vlv_dsi_pll.c
@@ -31,6 +31,7 @@
#include "intel_de.h"
#include "intel_display_types.h"
#include "intel_dsi.h"
+#include "vlv_dsi_pll.h"
#include "vlv_sideband.h"
static const u16 lfsr_converts[] = {
diff --git a/drivers/gpu/drm/i915/display/vlv_dsi_pll.h b/drivers/gpu/drm/i915/display/vlv_dsi_pll.h
new file mode 100644
index 0000000..ab9291a
--- /dev/null
+++ b/drivers/gpu/drm/i915/display/vlv_dsi_pll.h
@@ -0,0 +1,38 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2021 Intel Corporation
+ */
+
+#ifndef __VLV_DSI_PLL_H__
+#define __VLV_DSI_PLL_H__
+
+#include <linux/types.h>
+
+enum port;
+struct drm_i915_private;
+struct intel_crtc_state;
+struct intel_encoder;
+
+int vlv_dsi_pll_compute(struct intel_encoder *encoder,
+ struct intel_crtc_state *config);
+void vlv_dsi_pll_enable(struct intel_encoder *encoder,
+ const struct intel_crtc_state *config);
+void vlv_dsi_pll_disable(struct intel_encoder *encoder);
+u32 vlv_dsi_get_pclk(struct intel_encoder *encoder,
+ struct intel_crtc_state *config);
+void vlv_dsi_reset_clocks(struct intel_encoder *encoder, enum port port);
+
+bool bxt_dsi_pll_is_enabled(struct drm_i915_private *dev_priv);
+int bxt_dsi_pll_compute(struct intel_encoder *encoder,
+ struct intel_crtc_state *config);
+void bxt_dsi_pll_enable(struct intel_encoder *encoder,
+ const struct intel_crtc_state *config);
+void bxt_dsi_pll_disable(struct intel_encoder *encoder);
+u32 bxt_dsi_get_pclk(struct intel_encoder *encoder,
+ struct intel_crtc_state *config);
+void bxt_dsi_reset_clocks(struct intel_encoder *encoder, enum port port);
+
+void assert_dsi_pll_enabled(struct drm_i915_private *i915);
+void assert_dsi_pll_disabled(struct drm_i915_private *i915);
+
+#endif /* __VLV_DSI_PLL_H__ */
diff --git a/drivers/gpu/drm/i915/dma_resv_utils.c b/drivers/gpu/drm/i915/dma_resv_utils.c
deleted file mode 100644
index 7df91b7..0000000
--- a/drivers/gpu/drm/i915/dma_resv_utils.c
+++ /dev/null
@@ -1,17 +0,0 @@
-// SPDX-License-Identifier: MIT
-/*
- * Copyright © 2020 Intel Corporation
- */
-
-#include <linux/dma-resv.h>
-
-#include "dma_resv_utils.h"
-
-void dma_resv_prune(struct dma_resv *resv)
-{
- if (dma_resv_trylock(resv)) {
- if (dma_resv_test_signaled(resv, true))
- dma_resv_add_excl_fence(resv, NULL);
- dma_resv_unlock(resv);
- }
-}
diff --git a/drivers/gpu/drm/i915/dma_resv_utils.h b/drivers/gpu/drm/i915/dma_resv_utils.h
deleted file mode 100644
index b9d8fb5..0000000
--- a/drivers/gpu/drm/i915/dma_resv_utils.h
+++ /dev/null
@@ -1,13 +0,0 @@
-/* SPDX-License-Identifier: MIT */
-/*
- * Copyright © 2020 Intel Corporation
- */
-
-#ifndef DMA_RESV_UTILS_H
-#define DMA_RESV_UTILS_H
-
-struct dma_resv;
-
-void dma_resv_prune(struct dma_resv *resv);
-
-#endif /* DMA_RESV_UTILS_H */
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_busy.c b/drivers/gpu/drm/i915/gem/i915_gem_busy.c
index 7358beb..470fdfd 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_busy.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_busy.c
@@ -115,8 +115,8 @@ i915_gem_busy_ioctl(struct drm_device *dev, void *data,
{
struct drm_i915_gem_busy *args = data;
struct drm_i915_gem_object *obj;
- struct dma_resv_list *list;
- unsigned int seq;
+ struct dma_resv_iter cursor;
+ struct dma_fence *fence;
int err;
err = -ENOENT;
@@ -142,27 +142,20 @@ i915_gem_busy_ioctl(struct drm_device *dev, void *data,
* to report the overall busyness. This is what the wait-ioctl does.
*
*/
-retry:
- seq = raw_read_seqcount(&obj->base.resv->seq);
+ args->busy = 0;
+ dma_resv_iter_begin(&cursor, obj->base.resv, true);
+ dma_resv_for_each_fence_unlocked(&cursor, fence) {
+ if (dma_resv_iter_is_restarted(&cursor))
+ args->busy = 0;
- /* Translate the exclusive fence to the READ *and* WRITE engine */
- args->busy = busy_check_writer(dma_resv_excl_fence(obj->base.resv));
-
- /* Translate shared fences to READ set of engines */
- list = dma_resv_shared_list(obj->base.resv);
- if (list) {
- unsigned int shared_count = list->shared_count, i;
-
- for (i = 0; i < shared_count; ++i) {
- struct dma_fence *fence =
- rcu_dereference(list->shared[i]);
-
+ if (dma_resv_iter_is_exclusive(&cursor))
+ /* Translate the exclusive fence to the READ *and* WRITE engine */
+ args->busy |= busy_check_writer(fence);
+ else
+ /* Translate shared fences to READ set of engines */
args->busy |= busy_check_reader(fence);
- }
}
-
- if (args->busy && read_seqcount_retry(&obj->base.resv->seq, seq))
- goto retry;
+ dma_resv_iter_end(&cursor);
err = 0;
out:
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_clflush.c b/drivers/gpu/drm/i915/gem/i915_gem_clflush.c
index f0435c6..8a24800 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_clflush.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_clflush.c
@@ -69,10 +69,16 @@ static struct clflush *clflush_work_create(struct drm_i915_gem_object *obj)
bool i915_gem_clflush_object(struct drm_i915_gem_object *obj,
unsigned int flags)
{
+ struct drm_i915_private *i915 = to_i915(obj->base.dev);
struct clflush *clflush;
assert_object_held(obj);
+ if (IS_DGFX(i915)) {
+ WARN_ON_ONCE(obj->cache_dirty);
+ return false;
+ }
+
/*
* Stolen memory is always coherent with the GPU as it is explicitly
* marked as wc by the system, or the system is cache-coherent.
@@ -105,16 +111,24 @@ bool i915_gem_clflush_object(struct drm_i915_gem_object *obj,
if (clflush) {
i915_sw_fence_await_reservation(&clflush->base.chain,
obj->base.resv, NULL, true,
- i915_fence_timeout(to_i915(obj->base.dev)),
+ i915_fence_timeout(i915),
I915_FENCE_GFP);
dma_resv_add_excl_fence(obj->base.resv, &clflush->base.dma);
dma_fence_work_commit(&clflush->base);
+ /*
+ * We must have successfully populated the pages(since we are
+ * holding a pin on the pages as per the flush worker) to reach
+ * this point, which must mean we have already done the required
+ * flush-on-acquire, hence resetting cache_dirty here should be
+ * safe.
+ */
+ obj->cache_dirty = false;
} else if (obj->mm.pages) {
__do_clflush(obj);
+ obj->cache_dirty = false;
} else {
GEM_BUG_ON(obj->write_domain != I915_GEM_DOMAIN_CPU);
}
- obj->cache_dirty = false;
return true;
}
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.c b/drivers/gpu/drm/i915/gem/i915_gem_context.c
index fb33d03..00327b7 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_context.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_context.c
@@ -237,7 +237,7 @@ static int proto_context_set_persistence(struct drm_i915_private *i915,
* colateral damage, and we should not pretend we can by
* exposing the interface.
*/
- if (!intel_has_reset_engine(&i915->gt))
+ if (!intel_has_reset_engine(to_gt(i915)))
return -ENODEV;
pc->user_flags &= ~BIT(UCONTEXT_PERSISTENCE);
@@ -254,7 +254,7 @@ static int proto_context_set_protected(struct drm_i915_private *i915,
if (!protected) {
pc->uses_protected_content = false;
- } else if (!intel_pxp_is_enabled(&i915->gt.pxp)) {
+ } else if (!intel_pxp_is_enabled(&to_gt(i915)->pxp)) {
ret = -ENODEV;
} else if ((pc->user_flags & BIT(UCONTEXT_RECOVERABLE)) ||
!(pc->user_flags & BIT(UCONTEXT_BANNABLE))) {
@@ -268,8 +268,8 @@ static int proto_context_set_protected(struct drm_i915_private *i915,
*/
pc->pxp_wakeref = intel_runtime_pm_get(&i915->runtime_pm);
- if (!intel_pxp_is_active(&i915->gt.pxp))
- ret = intel_pxp_start(&i915->gt.pxp);
+ if (!intel_pxp_is_active(&to_gt(i915)->pxp))
+ ret = intel_pxp_start(&to_gt(i915)->pxp);
}
return ret;
@@ -479,7 +479,7 @@ set_proto_ctx_engines_bond(struct i915_user_extension __user *base, void *data)
if (GRAPHICS_VER(i915) >= 12 && !IS_TIGERLAKE(i915) &&
!IS_ROCKETLAKE(i915) && !IS_ALDERLAKE_S(i915)) {
drm_dbg(&i915->drm,
- "Bonding on gen12+ aside from TGL, RKL, and ADL_S not supported\n");
+ "Bonding not supported on this platform\n");
return -ENODEV;
}
@@ -564,6 +564,7 @@ set_proto_ctx_engines_parallel_submit(struct i915_user_extension __user *base,
container_of_user(base, typeof(*ext), base);
const struct set_proto_ctx_engines *set = data;
struct drm_i915_private *i915 = set->i915;
+ struct i915_engine_class_instance prev_engine;
u64 flags;
int err = 0, n, i, j;
u16 slot, width, num_siblings;
@@ -571,7 +572,7 @@ set_proto_ctx_engines_parallel_submit(struct i915_user_extension __user *base,
intel_engine_mask_t prev_mask;
/* FIXME: This is NIY for execlists */
- if (!(intel_uc_uses_guc_submission(&i915->gt.uc)))
+ if (!(intel_uc_uses_guc_submission(&to_gt(i915)->uc)))
return -ENODEV;
if (get_user(slot, &ext->engine_index))
@@ -629,7 +630,6 @@ set_proto_ctx_engines_parallel_submit(struct i915_user_extension __user *base,
/* Create contexts / engines */
for (i = 0; i < width; ++i) {
intel_engine_mask_t current_mask = 0;
- struct i915_engine_class_instance prev_engine;
for (j = 0; j < num_siblings; ++j) {
struct i915_engine_class_instance ci;
@@ -833,7 +833,7 @@ static int set_proto_ctx_sseu(struct drm_i915_file_private *fpriv,
sseu = &pc->legacy_rcs_sseu;
}
- ret = i915_gem_user_to_context_sseu(&i915->gt, &user_sseu, sseu);
+ ret = i915_gem_user_to_context_sseu(to_gt(i915), &user_sseu, sseu);
if (ret)
return ret;
@@ -1001,7 +1001,7 @@ static void free_engines_rcu(struct rcu_head *rcu)
free_engines(engines);
}
-static int __i915_sw_fence_call
+static int
engines_notify(struct i915_sw_fence *fence, enum i915_sw_fence_notify state)
{
struct i915_gem_engines *engines =
@@ -1044,7 +1044,7 @@ static struct i915_gem_engines *alloc_engines(unsigned int count)
static struct i915_gem_engines *default_engines(struct i915_gem_context *ctx,
struct intel_sseu rcs_sseu)
{
- const struct intel_gt *gt = &ctx->i915->gt;
+ const struct intel_gt *gt = to_gt(ctx->i915);
struct intel_engine_cs *engine;
struct i915_gem_engines *e, *err;
enum intel_engine_id id;
@@ -1521,7 +1521,7 @@ static int __context_set_persistence(struct i915_gem_context *ctx, bool state)
* colateral damage, and we should not pretend we can by
* exposing the interface.
*/
- if (!intel_has_reset_engine(&ctx->i915->gt))
+ if (!intel_has_reset_engine(to_gt(ctx->i915)))
return -ENODEV;
i915_gem_context_clear_persistence(ctx);
@@ -1559,7 +1559,7 @@ i915_gem_create_context(struct drm_i915_private *i915,
} else if (HAS_FULL_PPGTT(i915)) {
struct i915_ppgtt *ppgtt;
- ppgtt = i915_ppgtt_create(&i915->gt, 0);
+ ppgtt = i915_ppgtt_create(to_gt(i915), 0);
if (IS_ERR(ppgtt)) {
drm_dbg(&i915->drm, "PPGTT setup failed (%ld)\n",
PTR_ERR(ppgtt));
@@ -1742,7 +1742,7 @@ int i915_gem_vm_create_ioctl(struct drm_device *dev, void *data,
if (args->flags)
return -EINVAL;
- ppgtt = i915_ppgtt_create(&i915->gt, 0);
+ ppgtt = i915_ppgtt_create(to_gt(i915), 0);
if (IS_ERR(ppgtt))
return PTR_ERR(ppgtt);
@@ -2194,7 +2194,7 @@ int i915_gem_context_create_ioctl(struct drm_device *dev, void *data,
if (args->flags & I915_CONTEXT_CREATE_FLAGS_UNKNOWN)
return -EINVAL;
- ret = intel_gt_terminally_wedged(&i915->gt);
+ ret = intel_gt_terminally_wedged(to_gt(i915));
if (ret)
return ret;
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_create.c b/drivers/gpu/drm/i915/gem/i915_gem_create.c
index 8955d6a..9402d4b 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_create.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_create.c
@@ -379,7 +379,7 @@ static int ext_set_protected(struct i915_user_extension __user *base, void *data
if (ext.flags)
return -EINVAL;
- if (!intel_pxp_is_enabled(&ext_data->i915->gt.pxp))
+ if (!intel_pxp_is_enabled(&to_gt(ext_data->i915)->pxp))
return -ENODEV;
ext_data->flags |= I915_BO_PROTECTED;
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c b/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c
index e8a58c9..1b52603 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c
@@ -248,8 +248,19 @@ static int i915_gem_object_get_pages_dmabuf(struct drm_i915_gem_object *obj)
if (IS_ERR(pages))
return PTR_ERR(pages);
- /* XXX: consider doing a vmap flush or something */
- if (!HAS_LLC(i915) || i915_gem_object_can_bypass_llc(obj))
+ /*
+ * DG1 is special here since it still snoops transactions even with
+ * CACHE_NONE. This is not the case with other HAS_SNOOP platforms. We
+ * might need to revisit this as we add new discrete platforms.
+ *
+ * XXX: Consider doing a vmap flush or something, where possible.
+ * Currently we just do a heavy handed wbinvd_on_all_cpus() here since
+ * the underlying sg_table might not even point to struct pages, so we
+ * can't just call drm_clflush_sg or similar, like we do elsewhere in
+ * the driver.
+ */
+ if (i915_gem_object_can_bypass_llc(obj) ||
+ (!HAS_LLC(i915) && !IS_DG1(i915)))
wbinvd_on_all_cpus();
sg_page_sizes = i915_sg_dma_sizes(pages->sgl);
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_domain.c b/drivers/gpu/drm/i915/gem/i915_gem_domain.c
index b684a62..26532c0 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_domain.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_domain.c
@@ -18,10 +18,32 @@
static bool gpu_write_needs_clflush(struct drm_i915_gem_object *obj)
{
+ struct drm_i915_private *i915 = to_i915(obj->base.dev);
+
+ if (IS_DGFX(i915))
+ return false;
+
return !(obj->cache_level == I915_CACHE_NONE ||
obj->cache_level == I915_CACHE_WT);
}
+bool i915_gem_cpu_write_needs_clflush(struct drm_i915_gem_object *obj)
+{
+ struct drm_i915_private *i915 = to_i915(obj->base.dev);
+
+ if (obj->cache_dirty)
+ return false;
+
+ if (!(obj->cache_coherent & I915_BO_CACHE_COHERENT_FOR_WRITE))
+ return true;
+
+ if (IS_DGFX(i915))
+ return false;
+
+ /* Currently in use by HW (display engine)? Keep flushed. */
+ return i915_gem_object_is_framebuffer(obj);
+}
+
static void
flush_write_domain(struct drm_i915_gem_object *obj, unsigned int flush_domains)
{
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
index 9b24d9b..3a5b247 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
@@ -29,6 +29,7 @@
#include "i915_gem_ioctls.h"
#include "i915_trace.h"
#include "i915_user_extensions.h"
+#include "i915_vma_snapshot.h"
struct eb_vma {
struct i915_vma *vma;
@@ -307,11 +308,15 @@ struct i915_execbuffer {
struct eb_fence *fences;
unsigned long num_fences;
+#if IS_ENABLED(CONFIG_DRM_I915_CAPTURE_ERROR)
+ struct i915_capture_list *capture_lists[MAX_ENGINE_INSTANCE + 1];
+#endif
};
static int eb_parse(struct i915_execbuffer *eb);
static int eb_pin_engine(struct i915_execbuffer *eb, bool throttle);
static void eb_unpin_engine(struct i915_execbuffer *eb);
+static void eb_capture_release(struct i915_execbuffer *eb);
static inline bool eb_use_cmdparser(const struct i915_execbuffer *eb)
{
@@ -990,7 +995,7 @@ static int eb_validate_vmas(struct i915_execbuffer *eb)
}
if (!(ev->flags & EXEC_OBJECT_WRITE)) {
- err = dma_resv_reserve_shared(vma->resv, 1);
+ err = dma_resv_reserve_shared(vma->obj->base.resv, 1);
if (err)
return err;
}
@@ -1043,6 +1048,7 @@ static void eb_release_vmas(struct i915_execbuffer *eb, bool final)
i915_vma_put(vma);
}
+ eb_capture_release(eb);
eb_unpin_engine(eb);
}
@@ -1092,6 +1098,47 @@ static inline struct i915_ggtt *cache_to_ggtt(struct reloc_cache *cache)
return &i915->ggtt;
}
+static void reloc_cache_unmap(struct reloc_cache *cache)
+{
+ void *vaddr;
+
+ if (!cache->vaddr)
+ return;
+
+ vaddr = unmask_page(cache->vaddr);
+ if (cache->vaddr & KMAP)
+ kunmap_atomic(vaddr);
+ else
+ io_mapping_unmap_atomic((void __iomem *)vaddr);
+}
+
+static void reloc_cache_remap(struct reloc_cache *cache,
+ struct drm_i915_gem_object *obj)
+{
+ void *vaddr;
+
+ if (!cache->vaddr)
+ return;
+
+ if (cache->vaddr & KMAP) {
+ struct page *page = i915_gem_object_get_page(obj, cache->page);
+
+ vaddr = kmap_atomic(page);
+ cache->vaddr = unmask_flags(cache->vaddr) |
+ (unsigned long)vaddr;
+ } else {
+ struct i915_ggtt *ggtt = cache_to_ggtt(cache);
+ unsigned long offset;
+
+ offset = cache->node.start;
+ if (!drm_mm_node_allocated(&cache->node))
+ offset += cache->page << PAGE_SHIFT;
+
+ cache->vaddr = (unsigned long)
+ io_mapping_map_atomic_wc(&ggtt->iomap, offset);
+ }
+}
+
static void reloc_cache_reset(struct reloc_cache *cache, struct i915_execbuffer *eb)
{
void *vaddr;
@@ -1356,10 +1403,17 @@ eb_relocate_entry(struct i915_execbuffer *eb,
* batchbuffers.
*/
if (reloc->write_domain == I915_GEM_DOMAIN_INSTRUCTION &&
- GRAPHICS_VER(eb->i915) == 6) {
+ GRAPHICS_VER(eb->i915) == 6 &&
+ !i915_vma_is_bound(target->vma, I915_VMA_GLOBAL_BIND)) {
+ struct i915_vma *vma = target->vma;
+
+ reloc_cache_unmap(&eb->reloc_cache);
+ mutex_lock(&vma->vm->mutex);
err = i915_vma_bind(target->vma,
target->vma->obj->cache_level,
PIN_GLOBAL, NULL);
+ mutex_unlock(&vma->vm->mutex);
+ reloc_cache_remap(&eb->reloc_cache, ev->vma->obj);
if (err)
return err;
}
@@ -1880,6 +1934,100 @@ eb_find_first_request_added(struct i915_execbuffer *eb)
return NULL;
}
+#if IS_ENABLED(CONFIG_DRM_I915_CAPTURE_ERROR)
+
+/* Stage with GFP_KERNEL allocations before we enter the signaling critical path */
+static void eb_capture_stage(struct i915_execbuffer *eb)
+{
+ const unsigned int count = eb->buffer_count;
+ unsigned int i = count, j;
+ struct i915_vma_snapshot *vsnap;
+
+ while (i--) {
+ struct eb_vma *ev = &eb->vma[i];
+ struct i915_vma *vma = ev->vma;
+ unsigned int flags = ev->flags;
+
+ if (!(flags & EXEC_OBJECT_CAPTURE))
+ continue;
+
+ vsnap = i915_vma_snapshot_alloc(GFP_KERNEL);
+ if (!vsnap)
+ continue;
+
+ i915_vma_snapshot_init(vsnap, vma, "user");
+ for_each_batch_create_order(eb, j) {
+ struct i915_capture_list *capture;
+
+ capture = kmalloc(sizeof(*capture), GFP_KERNEL);
+ if (!capture)
+ continue;
+
+ capture->next = eb->capture_lists[j];
+ capture->vma_snapshot = i915_vma_snapshot_get(vsnap);
+ eb->capture_lists[j] = capture;
+ }
+ i915_vma_snapshot_put(vsnap);
+ }
+}
+
+/* Commit once we're in the critical path */
+static void eb_capture_commit(struct i915_execbuffer *eb)
+{
+ unsigned int j;
+
+ for_each_batch_create_order(eb, j) {
+ struct i915_request *rq = eb->requests[j];
+
+ if (!rq)
+ break;
+
+ rq->capture_list = eb->capture_lists[j];
+ eb->capture_lists[j] = NULL;
+ }
+}
+
+/*
+ * Release anything that didn't get committed due to errors.
+ * The capture_list will otherwise be freed at request retire.
+ */
+static void eb_capture_release(struct i915_execbuffer *eb)
+{
+ unsigned int j;
+
+ for_each_batch_create_order(eb, j) {
+ if (eb->capture_lists[j]) {
+ i915_request_free_capture_list(eb->capture_lists[j]);
+ eb->capture_lists[j] = NULL;
+ }
+ }
+}
+
+static void eb_capture_list_clear(struct i915_execbuffer *eb)
+{
+ memset(eb->capture_lists, 0, sizeof(eb->capture_lists));
+}
+
+#else
+
+static void eb_capture_stage(struct i915_execbuffer *eb)
+{
+}
+
+static void eb_capture_commit(struct i915_execbuffer *eb)
+{
+}
+
+static void eb_capture_release(struct i915_execbuffer *eb)
+{
+}
+
+static void eb_capture_list_clear(struct i915_execbuffer *eb)
+{
+}
+
+#endif
+
static int eb_move_to_gpu(struct i915_execbuffer *eb)
{
const unsigned int count = eb->buffer_count;
@@ -1894,23 +2042,6 @@ static int eb_move_to_gpu(struct i915_execbuffer *eb)
assert_vma_held(vma);
- if (flags & EXEC_OBJECT_CAPTURE) {
- struct i915_capture_list *capture;
-
- for_each_batch_create_order(eb, j) {
- if (!eb->requests[j])
- break;
-
- capture = kmalloc(sizeof(*capture), GFP_KERNEL);
- if (capture) {
- capture->next =
- eb->requests[j]->capture_list;
- capture->vma = vma;
- eb->requests[j]->capture_list = capture;
- }
- }
- }
-
/*
* If the GPU is not _reading_ through the CPU cache, we need
* to make sure that any writes (both previous GPU writes from
@@ -1990,6 +2121,8 @@ static int eb_move_to_gpu(struct i915_execbuffer *eb)
/* Unconditionally flush any chipset caches (for streaming writes). */
intel_gt_chipset_flush(eb->gt);
+ eb_capture_commit(eb);
+
return 0;
err_skip:
@@ -2164,7 +2297,7 @@ static int eb_parse(struct i915_execbuffer *eb)
goto err_trampoline;
}
- err = dma_resv_reserve_shared(shadow->resv, 1);
+ err = dma_resv_reserve_shared(shadow->obj->base.resv, 1);
if (err)
goto err_trampoline;
@@ -2276,9 +2409,9 @@ static int eb_submit(struct i915_execbuffer *eb)
return err;
}
-static int num_vcs_engines(const struct drm_i915_private *i915)
+static int num_vcs_engines(struct drm_i915_private *i915)
{
- return hweight_long(VDBOX_MASK(&i915->gt));
+ return hweight_long(VDBOX_MASK(to_gt(i915)));
}
/*
@@ -3017,7 +3150,7 @@ eb_composite_fence_create(struct i915_execbuffer *eb, int out_fence_fd)
fence_array = dma_fence_array_create(eb->num_batches,
fences,
eb->context->parallel.fence_context,
- eb->context->parallel.seqno,
+ eb->context->parallel.seqno++,
false);
if (!fence_array) {
kfree(fences);
@@ -3114,7 +3247,7 @@ eb_requests_create(struct i915_execbuffer *eb, struct dma_fence *in_fence,
/* Allocate a request for this batch buffer nice and early. */
eb->requests[i] = i915_request_create(eb_find_context(eb, i));
if (IS_ERR(eb->requests[i])) {
- out_fence = ERR_PTR(PTR_ERR(eb->requests[i]));
+ out_fence = ERR_CAST(eb->requests[i]);
eb->requests[i] = NULL;
return out_fence;
}
@@ -3132,13 +3265,14 @@ eb_requests_create(struct i915_execbuffer *eb, struct dma_fence *in_fence,
}
/*
- * Whilst this request exists, batch_obj will be on the
- * active_list, and so will hold the active reference. Only when
- * this request is retired will the batch_obj be moved onto
- * the inactive_list and lose its active reference. Hence we do
- * not need to explicitly hold another reference here.
+ * Not really on stack, but we don't want to call
+ * kfree on the batch_snapshot when we put it, so use the
+ * _onstack interface.
*/
- eb->requests[i]->batch = eb->batches[i]->vma;
+ if (eb->batches[i]->vma)
+ i915_vma_snapshot_init_onstack(&eb->requests[i]->batch_snapshot,
+ eb->batches[i]->vma,
+ "batch");
if (eb->batch_pool) {
GEM_BUG_ON(intel_context_is_parallel(eb->context));
intel_gt_buffer_pool_mark_active(eb->batch_pool,
@@ -3187,6 +3321,8 @@ i915_gem_do_execbuffer(struct drm_device *dev,
eb.fences = NULL;
eb.num_fences = 0;
+ eb_capture_list_clear(&eb);
+
memset(eb.requests, 0, sizeof(struct i915_request *) *
ARRAY_SIZE(eb.requests));
eb.composite_fence = NULL;
@@ -3273,6 +3409,7 @@ i915_gem_do_execbuffer(struct drm_device *dev,
}
ww_acquire_done(&eb.ww.ctx);
+ eb_capture_stage(&eb);
out_fence = eb_requests_create(&eb, in_fence, out_fence_fd);
if (IS_ERR(out_fence)) {
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_internal.c b/drivers/gpu/drm/i915/gem/i915_gem_internal.c
index a57a6b7..c5150a1 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_internal.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_internal.c
@@ -145,6 +145,47 @@ static const struct drm_i915_gem_object_ops i915_gem_object_internal_ops = {
.put_pages = i915_gem_object_put_pages_internal,
};
+struct drm_i915_gem_object *
+__i915_gem_object_create_internal(struct drm_i915_private *i915,
+ const struct drm_i915_gem_object_ops *ops,
+ phys_addr_t size)
+{
+ static struct lock_class_key lock_class;
+ struct drm_i915_gem_object *obj;
+ unsigned int cache_level;
+
+ GEM_BUG_ON(!size);
+ GEM_BUG_ON(!IS_ALIGNED(size, PAGE_SIZE));
+
+ if (overflows_type(size, obj->base.size))
+ return ERR_PTR(-E2BIG);
+
+ obj = i915_gem_object_alloc();
+ if (!obj)
+ return ERR_PTR(-ENOMEM);
+
+ drm_gem_private_object_init(&i915->drm, &obj->base, size);
+ i915_gem_object_init(obj, ops, &lock_class, 0);
+ obj->mem_flags |= I915_BO_FLAG_STRUCT_PAGE;
+
+ /*
+ * Mark the object as volatile, such that the pages are marked as
+ * dontneed whilst they are still pinned. As soon as they are unpinned
+ * they are allowed to be reaped by the shrinker, and the caller is
+ * expected to repopulate - the contents of this object are only valid
+ * whilst active and pinned.
+ */
+ i915_gem_object_set_volatile(obj);
+
+ obj->read_domains = I915_GEM_DOMAIN_CPU;
+ obj->write_domain = I915_GEM_DOMAIN_CPU;
+
+ cache_level = HAS_LLC(i915) ? I915_CACHE_LLC : I915_CACHE_NONE;
+ i915_gem_object_set_cache_coherency(obj, cache_level);
+
+ return obj;
+}
+
/**
* i915_gem_object_create_internal: create an object with volatile pages
* @i915: the i915 device
@@ -164,38 +205,5 @@ struct drm_i915_gem_object *
i915_gem_object_create_internal(struct drm_i915_private *i915,
phys_addr_t size)
{
- static struct lock_class_key lock_class;
- struct drm_i915_gem_object *obj;
- unsigned int cache_level;
-
- GEM_BUG_ON(!size);
- GEM_BUG_ON(!IS_ALIGNED(size, PAGE_SIZE));
-
- if (overflows_type(size, obj->base.size))
- return ERR_PTR(-E2BIG);
-
- obj = i915_gem_object_alloc();
- if (!obj)
- return ERR_PTR(-ENOMEM);
-
- drm_gem_private_object_init(&i915->drm, &obj->base, size);
- i915_gem_object_init(obj, &i915_gem_object_internal_ops, &lock_class, 0);
- obj->mem_flags |= I915_BO_FLAG_STRUCT_PAGE;
-
- /*
- * Mark the object as volatile, such that the pages are marked as
- * dontneed whilst they are still pinned. As soon as they are unpinned
- * they are allowed to be reaped by the shrinker, and the caller is
- * expected to repopulate - the contents of this object are only valid
- * whilst active and pinned.
- */
- i915_gem_object_set_volatile(obj);
-
- obj->read_domains = I915_GEM_DOMAIN_CPU;
- obj->write_domain = I915_GEM_DOMAIN_CPU;
-
- cache_level = HAS_LLC(i915) ? I915_CACHE_LLC : I915_CACHE_NONE;
- i915_gem_object_set_cache_coherency(obj, cache_level);
-
- return obj;
+ return __i915_gem_object_create_internal(i915, &i915_gem_object_internal_ops, size);
}
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_mman.c b/drivers/gpu/drm/i915/gem/i915_gem_mman.c
index 65fc6ff..aaf970c 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_mman.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_mman.c
@@ -17,6 +17,7 @@
#include "i915_gem_ioctls.h"
#include "i915_gem_object.h"
#include "i915_gem_mman.h"
+#include "i915_mm.h"
#include "i915_trace.h"
#include "i915_user_extensions.h"
#include "i915_gem_ttm.h"
@@ -72,7 +73,7 @@ i915_gem_mmap_ioctl(struct drm_device *dev, void *data,
if (args->flags & ~(I915_MMAP_WC))
return -EINVAL;
- if (args->flags & I915_MMAP_WC && !boot_cpu_has(X86_FEATURE_PAT))
+ if (args->flags & I915_MMAP_WC && !pat_enabled())
return -ENODEV;
obj = i915_gem_object_lookup(file, args->handle);
@@ -645,7 +646,7 @@ mmap_offset_attach(struct drm_i915_gem_object *obj,
goto insert;
/* Attempt to reap some mmap space from dead objects */
- err = intel_gt_retire_requests_timeout(&i915->gt, MAX_SCHEDULE_TIMEOUT,
+ err = intel_gt_retire_requests_timeout(to_gt(i915), MAX_SCHEDULE_TIMEOUT,
NULL);
if (err)
goto err;
@@ -736,7 +737,7 @@ i915_gem_dumb_mmap_offset(struct drm_file *file,
if (HAS_LMEM(to_i915(dev)))
mmap_type = I915_MMAP_TYPE_FIXED;
- else if (boot_cpu_has(X86_FEATURE_PAT))
+ else if (pat_enabled())
mmap_type = I915_MMAP_TYPE_WC;
else if (!i915_ggtt_has_aperture(&to_i915(dev)->ggtt))
return -ENODEV;
@@ -792,7 +793,7 @@ i915_gem_mmap_offset_ioctl(struct drm_device *dev, void *data,
break;
case I915_MMAP_OFFSET_WC:
- if (!boot_cpu_has(X86_FEATURE_PAT))
+ if (!pat_enabled())
return -ENODEV;
type = I915_MMAP_TYPE_WC;
break;
@@ -802,7 +803,7 @@ i915_gem_mmap_offset_ioctl(struct drm_device *dev, void *data,
break;
case I915_MMAP_OFFSET_UC:
- if (!boot_cpu_has(X86_FEATURE_PAT))
+ if (!pat_enabled())
return -ENODEV;
type = I915_MMAP_TYPE_UC;
break;
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object.c b/drivers/gpu/drm/i915/gem/i915_gem_object.c
index 1e426a4..d87b508 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_object.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_object.c
@@ -31,6 +31,7 @@
#include "i915_gem_context.h"
#include "i915_gem_mman.h"
#include "i915_gem_object.h"
+#include "i915_gem_ttm.h"
#include "i915_memcpy.h"
#include "i915_trace.h"
@@ -91,7 +92,7 @@ void i915_gem_object_init(struct drm_i915_gem_object *obj,
}
/**
- * i915_gem_object_fini - Clean up a GEM object initialization
+ * __i915_gem_object_fini - Clean up a GEM object initialization
* @obj: The gem object to cleanup
*
* This function cleans up gem object fields that are set up by
@@ -107,25 +108,29 @@ void __i915_gem_object_fini(struct drm_i915_gem_object *obj)
}
/**
- * Mark up the object's coherency levels for a given cache_level
+ * i915_gem_object_set_cache_coherency - Mark up the object's coherency levels
+ * for a given cache_level
* @obj: #drm_i915_gem_object
* @cache_level: cache level
*/
void i915_gem_object_set_cache_coherency(struct drm_i915_gem_object *obj,
unsigned int cache_level)
{
+ struct drm_i915_private *i915 = to_i915(obj->base.dev);
+
obj->cache_level = cache_level;
if (cache_level != I915_CACHE_NONE)
obj->cache_coherent = (I915_BO_CACHE_COHERENT_FOR_READ |
I915_BO_CACHE_COHERENT_FOR_WRITE);
- else if (HAS_LLC(to_i915(obj->base.dev)))
+ else if (HAS_LLC(i915))
obj->cache_coherent = I915_BO_CACHE_COHERENT_FOR_READ;
else
obj->cache_coherent = 0;
obj->cache_dirty =
- !(obj->cache_coherent & I915_BO_CACHE_COHERENT_FOR_WRITE);
+ !(obj->cache_coherent & I915_BO_CACHE_COHERENT_FOR_WRITE) &&
+ !IS_DGFX(i915);
}
bool i915_gem_object_can_bypass_llc(struct drm_i915_gem_object *obj)
@@ -257,6 +262,8 @@ static void __i915_gem_object_free_mmaps(struct drm_i915_gem_object *obj)
*/
void __i915_gem_object_pages_fini(struct drm_i915_gem_object *obj)
{
+ assert_object_held(obj);
+
if (!list_empty(&obj->vma.list)) {
struct i915_vma *vma;
@@ -323,7 +330,16 @@ static void __i915_gem_free_objects(struct drm_i915_private *i915,
obj->ops->delayed_free(obj);
continue;
}
+
+ if (!i915_gem_object_trylock(obj, NULL)) {
+ /* busy, toss it back to the pile */
+ if (llist_add(&obj->freed, &i915->mm.free_list))
+ queue_delayed_work(i915->wq, &i915->mm.free_work, msecs_to_jiffies(10));
+ continue;
+ }
+
__i915_gem_object_pages_fini(obj);
+ i915_gem_object_unlock(obj);
__i915_gem_free_object(obj);
/* But keep the pointer alive for RCU-protected lookups */
@@ -343,7 +359,7 @@ void i915_gem_flush_free_objects(struct drm_i915_private *i915)
static void __i915_gem_free_work(struct work_struct *work)
{
struct drm_i915_private *i915 =
- container_of(work, struct drm_i915_private, mm.free_work);
+ container_of(work, struct drm_i915_private, mm.free_work.work);
i915_gem_flush_free_objects(i915);
}
@@ -364,15 +380,6 @@ static void i915_gem_free_object(struct drm_gem_object *gem_obj)
atomic_inc(&i915->mm.free_count);
/*
- * This serializes freeing with the shrinker. Since the free
- * is delayed, first by RCU then by the workqueue, we want the
- * shrinker to be able to free pages of unreferenced objects,
- * or else we may oom whilst there are plenty of deferred
- * freed objects.
- */
- i915_gem_object_make_unshrinkable(obj);
-
- /*
* Since we require blocking on struct_mutex to unbind the freed
* object from the GPU before releasing resources back to the
* system, we can not do that directly from the RCU callback (which may
@@ -384,7 +391,7 @@ static void i915_gem_free_object(struct drm_gem_object *gem_obj)
*/
if (llist_add(&obj->freed, &i915->mm.free_list))
- queue_work(i915->wq, &i915->mm.free_work);
+ queue_delayed_work(i915->wq, &i915->mm.free_work, 0);
}
void __i915_gem_object_flush_frontbuffer(struct drm_i915_gem_object *obj,
@@ -456,7 +463,7 @@ i915_gem_object_read_from_page_iomap(struct drm_i915_gem_object *obj, u64 offset
* from can't cross a page boundary. The caller must ensure that @obj pages
* are pinned and that @obj is synced wrt. any related writes.
*
- * Returns 0 on success or -ENODEV if the type of @obj's backing store is
+ * Return: %0 on success or -ENODEV if the type of @obj's backing store is
* unsupported.
*/
int i915_gem_object_read_from_page(struct drm_i915_gem_object *obj, u64 offset, void *dst, int size)
@@ -709,7 +716,7 @@ bool i915_gem_object_placement_possible(struct drm_i915_gem_object *obj,
void i915_gem_init__objects(struct drm_i915_private *i915)
{
- INIT_WORK(&i915->mm.free_work, __i915_gem_free_work);
+ INIT_DELAYED_WORK(&i915->mm.free_work, __i915_gem_free_work);
}
void i915_objects_module_exit(void)
@@ -732,6 +739,57 @@ static const struct drm_gem_object_funcs i915_gem_object_funcs = {
.export = i915_gem_prime_export,
};
+/**
+ * i915_gem_object_get_moving_fence - Get the object's moving fence if any
+ * @obj: The object whose moving fence to get.
+ *
+ * A non-signaled moving fence means that there is an async operation
+ * pending on the object that needs to be waited on before setting up
+ * any GPU- or CPU PTEs to the object's pages.
+ *
+ * Return: A refcounted pointer to the object's moving fence if any,
+ * NULL otherwise.
+ */
+struct dma_fence *
+i915_gem_object_get_moving_fence(struct drm_i915_gem_object *obj)
+{
+ return dma_fence_get(i915_gem_to_ttm(obj)->moving);
+}
+
+/**
+ * i915_gem_object_wait_moving_fence - Wait for the object's moving fence if any
+ * @obj: The object whose moving fence to wait for.
+ * @intr: Whether to wait interruptible.
+ *
+ * If the moving fence signaled without an error, it is detached from the
+ * object and put.
+ *
+ * Return: 0 if successful, -ERESTARTSYS if the wait was interrupted,
+ * negative error code if the async operation represented by the
+ * moving fence failed.
+ */
+int i915_gem_object_wait_moving_fence(struct drm_i915_gem_object *obj,
+ bool intr)
+{
+ struct dma_fence *fence = i915_gem_to_ttm(obj)->moving;
+ int ret;
+
+ assert_object_held(obj);
+ if (!fence)
+ return 0;
+
+ ret = dma_fence_wait(fence, intr);
+ if (ret)
+ return ret;
+
+ if (fence->error)
+ return fence->error;
+
+ i915_gem_to_ttm(obj)->moving = NULL;
+ dma_fence_put(fence);
+ return 0;
+}
+
#if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
#include "selftests/huge_gem_object.c"
#include "selftests/huge_pages.c"
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object.h b/drivers/gpu/drm/i915/gem/i915_gem_object.h
index 5920180..f66d468 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_object.h
+++ b/drivers/gpu/drm/i915/gem/i915_gem_object.h
@@ -93,7 +93,6 @@ void i915_gem_flush_free_objects(struct drm_i915_private *i915);
struct sg_table *
__i915_gem_object_unset_pages(struct drm_i915_gem_object *obj);
-void i915_gem_object_truncate(struct drm_i915_gem_object *obj);
/**
* i915_gem_object_lookup_rcu - look up a temporary GEM object from its handle
@@ -211,9 +210,13 @@ static inline int i915_gem_object_lock_interruptible(struct drm_i915_gem_object
return __i915_gem_object_lock(obj, ww, true);
}
-static inline bool i915_gem_object_trylock(struct drm_i915_gem_object *obj)
+static inline bool i915_gem_object_trylock(struct drm_i915_gem_object *obj,
+ struct i915_gem_ww_ctx *ww)
{
- return dma_resv_trylock(obj->base.resv);
+ if (!ww)
+ return dma_resv_trylock(obj->base.resv);
+ else
+ return ww_mutex_trylock(&obj->base.resv->lock, &ww->ctx);
}
static inline void i915_gem_object_unlock(struct drm_i915_gem_object *obj)
@@ -296,6 +299,12 @@ i915_gem_object_is_shrinkable(const struct drm_i915_gem_object *obj)
}
static inline bool
+i915_gem_object_has_self_managed_shrink_list(const struct drm_i915_gem_object *obj)
+{
+ return i915_gem_object_type_has(obj, I915_GEM_OBJECT_SELF_MANAGED_SHRINK_LIST);
+}
+
+static inline bool
i915_gem_object_is_proxy(const struct drm_i915_gem_object *obj)
{
return i915_gem_object_type_has(obj, I915_GEM_OBJECT_IS_PROXY);
@@ -449,7 +458,7 @@ i915_gem_object_unpin_pages(struct drm_i915_gem_object *obj)
}
int __i915_gem_object_put_pages(struct drm_i915_gem_object *obj);
-void i915_gem_object_truncate(struct drm_i915_gem_object *obj);
+int i915_gem_object_truncate(struct drm_i915_gem_object *obj);
void i915_gem_object_writeback(struct drm_i915_gem_object *obj);
/**
@@ -512,11 +521,18 @@ i915_gem_object_finish_access(struct drm_i915_gem_object *obj)
i915_gem_object_unpin_pages(obj);
}
+struct dma_fence *
+i915_gem_object_get_moving_fence(struct drm_i915_gem_object *obj);
+
+int i915_gem_object_wait_moving_fence(struct drm_i915_gem_object *obj,
+ bool intr);
+
void i915_gem_object_set_cache_coherency(struct drm_i915_gem_object *obj,
unsigned int cache_level);
bool i915_gem_object_can_bypass_llc(struct drm_i915_gem_object *obj);
void i915_gem_object_flush_if_display(struct drm_i915_gem_object *obj);
void i915_gem_object_flush_if_display_locked(struct drm_i915_gem_object *obj);
+bool i915_gem_cpu_write_needs_clflush(struct drm_i915_gem_object *obj);
int __must_check
i915_gem_object_set_to_wc_domain(struct drm_i915_gem_object *obj, bool write);
@@ -533,25 +549,15 @@ i915_gem_object_pin_to_display_plane(struct drm_i915_gem_object *obj,
void i915_gem_object_make_unshrinkable(struct drm_i915_gem_object *obj);
void i915_gem_object_make_shrinkable(struct drm_i915_gem_object *obj);
+void __i915_gem_object_make_shrinkable(struct drm_i915_gem_object *obj);
+void __i915_gem_object_make_purgeable(struct drm_i915_gem_object *obj);
void i915_gem_object_make_purgeable(struct drm_i915_gem_object *obj);
-static inline bool cpu_write_needs_clflush(struct drm_i915_gem_object *obj)
-{
- if (obj->cache_dirty)
- return false;
-
- if (!(obj->cache_coherent & I915_BO_CACHE_COHERENT_FOR_WRITE))
- return true;
-
- /* Currently in use by HW (display engine)? Keep flushed. */
- return i915_gem_object_is_framebuffer(obj);
-}
-
static inline void __start_cpu_write(struct drm_i915_gem_object *obj)
{
obj->read_domains = I915_GEM_DOMAIN_CPU;
obj->write_domain = I915_GEM_DOMAIN_CPU;
- if (cpu_write_needs_clflush(obj))
+ if (i915_gem_cpu_write_needs_clflush(obj))
obj->cache_dirty = true;
}
@@ -613,6 +619,14 @@ int i915_gem_object_wait_migration(struct drm_i915_gem_object *obj,
bool i915_gem_object_placement_possible(struct drm_i915_gem_object *obj,
enum intel_memory_type type);
+int shmem_sg_alloc_table(struct drm_i915_private *i915, struct sg_table *st,
+ size_t size, struct intel_memory_region *mr,
+ struct address_space *mapping,
+ unsigned int max_segment);
+void shmem_sg_free_table(struct sg_table *st, struct address_space *mapping,
+ bool dirty, bool backup);
+void __shmem_writeback(size_t size, struct address_space *mapping);
+
#ifdef CONFIG_MMU_NOTIFIER
static inline bool
i915_gem_object_is_userptr(struct drm_i915_gem_object *obj)
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object_types.h b/drivers/gpu/drm/i915/gem/i915_gem_object_types.h
index da85169..f9f7e44 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_object_types.h
+++ b/drivers/gpu/drm/i915/gem/i915_gem_object_types.h
@@ -34,9 +34,11 @@ struct i915_lut_handle {
struct drm_i915_gem_object_ops {
unsigned int flags;
-#define I915_GEM_OBJECT_IS_SHRINKABLE BIT(1)
-#define I915_GEM_OBJECT_IS_PROXY BIT(2)
-#define I915_GEM_OBJECT_NO_MMAP BIT(3)
+#define I915_GEM_OBJECT_IS_SHRINKABLE BIT(1)
+/* Skip the shrinker management in set_pages/unset_pages */
+#define I915_GEM_OBJECT_SELF_MANAGED_SHRINK_LIST BIT(2)
+#define I915_GEM_OBJECT_IS_PROXY BIT(3)
+#define I915_GEM_OBJECT_NO_MMAP BIT(4)
/* Interface between the GEM object and its backing storage.
* get_pages() is called once prior to the use of the associated set
@@ -54,8 +56,11 @@ struct drm_i915_gem_object_ops {
int (*get_pages)(struct drm_i915_gem_object *obj);
void (*put_pages)(struct drm_i915_gem_object *obj,
struct sg_table *pages);
- void (*truncate)(struct drm_i915_gem_object *obj);
+ int (*truncate)(struct drm_i915_gem_object *obj);
void (*writeback)(struct drm_i915_gem_object *obj);
+ int (*shrinker_release_pages)(struct drm_i915_gem_object *obj,
+ bool no_gpu_wait,
+ bool should_writeback);
int (*pread)(struct drm_i915_gem_object *obj,
const struct drm_i915_gem_pread *arg);
@@ -486,9 +491,37 @@ struct drm_i915_gem_object {
* instead go through the pin/unpin interfaces.
*/
atomic_t pages_pin_count;
+
+ /**
+ * @shrink_pin: Prevents the pages from being made visible to
+ * the shrinker, while the shrink_pin is non-zero. Most users
+ * should pretty much never have to care about this, outside of
+ * some special use cases.
+ *
+ * By default most objects will start out as visible to the
+ * shrinker(if I915_GEM_OBJECT_IS_SHRINKABLE) as soon as the
+ * backing pages are attached to the object, like in
+ * __i915_gem_object_set_pages(). They will then be removed the
+ * shrinker list once the pages are released.
+ *
+ * The @shrink_pin is incremented by calling
+ * i915_gem_object_make_unshrinkable(), which will also remove
+ * the object from the shrinker list, if the pin count was zero.
+ *
+ * Callers will then typically call
+ * i915_gem_object_make_shrinkable() or
+ * i915_gem_object_make_purgeable() to decrement the pin count,
+ * and make the pages visible again.
+ */
atomic_t shrink_pin;
/**
+ * @ttm_shrinkable: True when the object is using shmem pages
+ * underneath. Protected by the object lock.
+ */
+ bool ttm_shrinkable;
+
+ /**
* Priority list of potential placements for this object.
*/
struct intel_memory_region **placements;
@@ -512,6 +545,7 @@ struct drm_i915_gem_object {
*/
struct list_head region_link;
+ struct i915_refct_sgt *rsgt;
struct sg_table *pages;
void *mapping;
@@ -547,7 +581,7 @@ struct drm_i915_gem_object {
struct i915_gem_object_page_iter get_dma_page;
/**
- * Element within i915->mm.unbound_list or i915->mm.bound_list,
+ * Element within i915->mm.shrink_list or i915->mm.purge_list,
* locked by i915->mm.obj_lock.
*/
struct list_head link;
@@ -565,7 +599,7 @@ struct drm_i915_gem_object {
} mm;
struct {
- struct sg_table *cached_io_st;
+ struct i915_refct_sgt *cached_io_rsgt;
struct i915_gem_object_page_iter get_io_page;
struct drm_i915_gem_object *backup;
bool created:1;
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_pages.c b/drivers/gpu/drm/i915/gem/i915_gem_pages.c
index 8eb1c3a..89b70f5c 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_pages.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_pages.c
@@ -26,6 +26,7 @@ void __i915_gem_object_set_pages(struct drm_i915_gem_object *obj,
/* Make the pages coherent with the GPU (flushing any swapin). */
if (obj->cache_dirty) {
+ WARN_ON_ONCE(IS_DGFX(i915));
obj->write_domain = 0;
if (i915_gem_object_has_struct_page(obj))
drm_clflush_sg(pages);
@@ -68,7 +69,7 @@ void __i915_gem_object_set_pages(struct drm_i915_gem_object *obj,
shrinkable = false;
}
- if (shrinkable) {
+ if (shrinkable && !i915_gem_object_has_self_managed_shrink_list(obj)) {
struct list_head *list;
unsigned long flags;
@@ -158,11 +159,13 @@ int i915_gem_object_pin_pages_unlocked(struct drm_i915_gem_object *obj)
}
/* Immediately discard the backing storage */
-void i915_gem_object_truncate(struct drm_i915_gem_object *obj)
+int i915_gem_object_truncate(struct drm_i915_gem_object *obj)
{
drm_gem_free_mmap_offset(&obj->base);
if (obj->ops->truncate)
- obj->ops->truncate(obj);
+ return obj->ops->truncate(obj);
+
+ return 0;
}
/* Try to discard unwanted pages */
@@ -208,7 +211,8 @@ __i915_gem_object_unset_pages(struct drm_i915_gem_object *obj)
if (i915_gem_object_is_volatile(obj))
obj->mm.madv = I915_MADV_WILLNEED;
- i915_gem_object_make_unshrinkable(obj);
+ if (!i915_gem_object_has_self_managed_shrink_list(obj))
+ i915_gem_object_make_unshrinkable(obj);
if (obj->mm.mapping) {
unmap_object(obj, page_mask_bits(obj->mm.mapping));
@@ -414,8 +418,13 @@ void *i915_gem_object_pin_map(struct drm_i915_gem_object *obj,
}
if (!ptr) {
- if (GEM_WARN_ON(type == I915_MAP_WC &&
- !static_cpu_has(X86_FEATURE_PAT)))
+ err = i915_gem_object_wait_moving_fence(obj, true);
+ if (err) {
+ ptr = ERR_PTR(err);
+ goto err_unpin;
+ }
+
+ if (GEM_WARN_ON(type == I915_MAP_WC && !pat_enabled()))
ptr = ERR_PTR(-ENODEV);
else if (i915_gem_object_has_struct_page(obj))
ptr = i915_gem_object_map_page(obj, type);
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_phys.c b/drivers/gpu/drm/i915/gem/i915_gem_phys.c
index 7986612..ca6faff 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_phys.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_phys.c
@@ -19,6 +19,7 @@
static int i915_gem_object_get_pages_phys(struct drm_i915_gem_object *obj)
{
struct address_space *mapping = obj->base.filp->f_mapping;
+ struct drm_i915_private *i915 = to_i915(obj->base.dev);
struct scatterlist *sg;
struct sg_table *st;
dma_addr_t dma;
@@ -73,7 +74,7 @@ static int i915_gem_object_get_pages_phys(struct drm_i915_gem_object *obj)
dst += PAGE_SIZE;
}
- intel_gt_chipset_flush(&to_i915(obj->base.dev)->gt);
+ intel_gt_chipset_flush(to_gt(i915));
/* We're no longer struct page backed */
obj->mem_flags &= ~I915_BO_FLAG_STRUCT_PAGE;
@@ -140,6 +141,7 @@ int i915_gem_object_pwrite_phys(struct drm_i915_gem_object *obj,
{
void *vaddr = sg_page(obj->mm.pages->sgl) + args->offset;
char __user *user_data = u64_to_user_ptr(args->data_ptr);
+ struct drm_i915_private *i915 = to_i915(obj->base.dev);
int err;
err = i915_gem_object_wait(obj,
@@ -159,7 +161,7 @@ int i915_gem_object_pwrite_phys(struct drm_i915_gem_object *obj,
return -EFAULT;
drm_clflush_virt_range(vaddr, args->size);
- intel_gt_chipset_flush(&to_i915(obj->base.dev)->gt);
+ intel_gt_chipset_flush(to_gt(i915));
i915_gem_object_flush_frontbuffer(obj, ORIGIN_CPU);
return 0;
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_pm.c b/drivers/gpu/drm/i915/gem/i915_gem_pm.c
index 726b40e..ac56124 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_pm.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_pm.c
@@ -35,7 +35,7 @@ void i915_gem_suspend(struct drm_i915_private *i915)
* state. Fortunately, the kernel_context is disposable and we do
* not rely on its state.
*/
- intel_gt_suspend_prepare(&i915->gt);
+ intel_gt_suspend_prepare(to_gt(i915));
i915_gem_drain_freed_objects(i915);
}
@@ -153,7 +153,7 @@ void i915_gem_suspend_late(struct drm_i915_private *i915)
* machine in an unusable condition.
*/
- intel_gt_suspend_late(&i915->gt);
+ intel_gt_suspend_late(to_gt(i915));
spin_lock_irqsave(&i915->mm.obj_lock, flags);
for (phase = phases; *phase; phase++) {
@@ -223,7 +223,7 @@ void i915_gem_resume(struct drm_i915_private *i915)
* guarantee that the context image is complete. So let's just reset
* it and start again.
*/
- intel_gt_resume(&i915->gt);
+ intel_gt_resume(to_gt(i915));
ret = lmem_restore(i915, I915_TTM_BACKUP_ALLOW_GPU);
GEM_WARN_ON(ret);
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_region.c b/drivers/gpu/drm/i915/gem/i915_gem_region.c
index a016cce..a435022 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_region.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_region.c
@@ -11,7 +11,7 @@
void i915_gem_object_init_memory_region(struct drm_i915_gem_object *obj,
struct intel_memory_region *mem)
{
- obj->mm.region = intel_memory_region_get(mem);
+ obj->mm.region = mem;
mutex_lock(&mem->objects.lock);
list_add(&obj->mm.region_link, &mem->objects.list);
@@ -25,8 +25,6 @@ void i915_gem_object_release_memory_region(struct drm_i915_gem_object *obj)
mutex_lock(&mem->objects.lock);
list_del(&obj->mm.region_link);
mutex_unlock(&mem->objects.lock);
-
- intel_memory_region_put(mem);
}
struct drm_i915_gem_object *
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_shmem.c b/drivers/gpu/drm/i915/gem/i915_gem_shmem.c
index d77da59..cc9fe25 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_shmem.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_shmem.c
@@ -25,62 +25,67 @@ static void check_release_pagevec(struct pagevec *pvec)
cond_resched();
}
-static int shmem_get_pages(struct drm_i915_gem_object *obj)
+void shmem_sg_free_table(struct sg_table *st, struct address_space *mapping,
+ bool dirty, bool backup)
{
- struct drm_i915_private *i915 = to_i915(obj->base.dev);
- struct intel_memory_region *mem = obj->mm.region;
- const unsigned long page_count = obj->base.size / PAGE_SIZE;
- unsigned long i;
- struct address_space *mapping;
- struct sg_table *st;
- struct scatterlist *sg;
struct sgt_iter sgt_iter;
+ struct pagevec pvec;
+ struct page *page;
+
+ mapping_clear_unevictable(mapping);
+
+ pagevec_init(&pvec);
+ for_each_sgt_page(page, sgt_iter, st) {
+ if (dirty)
+ set_page_dirty(page);
+
+ if (backup)
+ mark_page_accessed(page);
+
+ if (!pagevec_add(&pvec, page))
+ check_release_pagevec(&pvec);
+ }
+ if (pagevec_count(&pvec))
+ check_release_pagevec(&pvec);
+
+ sg_free_table(st);
+}
+
+int shmem_sg_alloc_table(struct drm_i915_private *i915, struct sg_table *st,
+ size_t size, struct intel_memory_region *mr,
+ struct address_space *mapping,
+ unsigned int max_segment)
+{
+ const unsigned long page_count = size / PAGE_SIZE;
+ unsigned long i;
+ struct scatterlist *sg;
struct page *page;
unsigned long last_pfn = 0; /* suppress gcc warning */
- unsigned int max_segment = i915_sg_segment_size();
- unsigned int sg_page_sizes;
gfp_t noreclaim;
int ret;
/*
- * Assert that the object is not currently in any GPU domain. As it
- * wasn't in the GTT, there shouldn't be any way it could have been in
- * a GPU cache
- */
- GEM_BUG_ON(obj->read_domains & I915_GEM_GPU_DOMAINS);
- GEM_BUG_ON(obj->write_domain & I915_GEM_GPU_DOMAINS);
-
- /*
* If there's no chance of allocating enough pages for the whole
* object, bail early.
*/
- if (obj->base.size > resource_size(&mem->region))
+ if (size > resource_size(&mr->region))
return -ENOMEM;
- st = kmalloc(sizeof(*st), GFP_KERNEL);
- if (!st)
+ if (sg_alloc_table(st, page_count, GFP_KERNEL))
return -ENOMEM;
-rebuild_st:
- if (sg_alloc_table(st, page_count, GFP_KERNEL)) {
- kfree(st);
- return -ENOMEM;
- }
-
/*
* Get the list of pages out of our struct file. They'll be pinned
* at this point until we release them.
*
* Fail silently without starting the shrinker
*/
- mapping = obj->base.filp->f_mapping;
mapping_set_unevictable(mapping);
noreclaim = mapping_gfp_constraint(mapping, ~__GFP_RECLAIM);
noreclaim |= __GFP_NORETRY | __GFP_NOWARN;
sg = st->sgl;
st->nents = 0;
- sg_page_sizes = 0;
for (i = 0; i < page_count; i++) {
const unsigned int shrink[] = {
I915_SHRINK_BOUND | I915_SHRINK_UNBOUND,
@@ -135,10 +140,9 @@ static int shmem_get_pages(struct drm_i915_gem_object *obj)
if (!i ||
sg->length >= max_segment ||
page_to_pfn(page) != last_pfn + 1) {
- if (i) {
- sg_page_sizes |= sg->length;
+ if (i)
sg = sg_next(sg);
- }
+
st->nents++;
sg_set_page(sg, page, PAGE_SIZE, 0);
} else {
@@ -149,14 +153,67 @@ static int shmem_get_pages(struct drm_i915_gem_object *obj)
/* Check that the i965g/gm workaround works. */
GEM_BUG_ON(gfp & __GFP_DMA32 && last_pfn >= 0x00100000UL);
}
- if (sg) { /* loop terminated early; short sg table */
- sg_page_sizes |= sg->length;
+ if (sg) /* loop terminated early; short sg table */
sg_mark_end(sg);
- }
/* Trim unused sg entries to avoid wasting memory. */
i915_sg_trim(st);
+ return 0;
+err_sg:
+ sg_mark_end(sg);
+ if (sg != st->sgl) {
+ shmem_sg_free_table(st, mapping, false, false);
+ } else {
+ mapping_clear_unevictable(mapping);
+ sg_free_table(st);
+ }
+
+ /*
+ * shmemfs first checks if there is enough memory to allocate the page
+ * and reports ENOSPC should there be insufficient, along with the usual
+ * ENOMEM for a genuine allocation failure.
+ *
+ * We use ENOSPC in our driver to mean that we have run out of aperture
+ * space and so want to translate the error from shmemfs back to our
+ * usual understanding of ENOMEM.
+ */
+ if (ret == -ENOSPC)
+ ret = -ENOMEM;
+
+ return ret;
+}
+
+static int shmem_get_pages(struct drm_i915_gem_object *obj)
+{
+ struct drm_i915_private *i915 = to_i915(obj->base.dev);
+ struct intel_memory_region *mem = obj->mm.region;
+ struct address_space *mapping = obj->base.filp->f_mapping;
+ const unsigned long page_count = obj->base.size / PAGE_SIZE;
+ unsigned int max_segment = i915_sg_segment_size();
+ struct sg_table *st;
+ struct sgt_iter sgt_iter;
+ struct page *page;
+ int ret;
+
+ /*
+ * Assert that the object is not currently in any GPU domain. As it
+ * wasn't in the GTT, there shouldn't be any way it could have been in
+ * a GPU cache
+ */
+ GEM_BUG_ON(obj->read_domains & I915_GEM_GPU_DOMAINS);
+ GEM_BUG_ON(obj->write_domain & I915_GEM_GPU_DOMAINS);
+
+rebuild_st:
+ st = kmalloc(sizeof(*st), GFP_KERNEL);
+ if (!st)
+ return -ENOMEM;
+
+ ret = shmem_sg_alloc_table(i915, st, obj->base.size, mem, mapping,
+ max_segment);
+ if (ret)
+ goto err_st;
+
ret = i915_gem_gtt_prepare_pages(obj, st);
if (ret) {
/*
@@ -168,6 +225,7 @@ static int shmem_get_pages(struct drm_i915_gem_object *obj)
for_each_sgt_page(page, sgt_iter, st)
put_page(page);
sg_free_table(st);
+ kfree(st);
max_segment = PAGE_SIZE;
goto rebuild_st;
@@ -185,28 +243,12 @@ static int shmem_get_pages(struct drm_i915_gem_object *obj)
if (i915_gem_object_can_bypass_llc(obj))
obj->cache_dirty = true;
- __i915_gem_object_set_pages(obj, st, sg_page_sizes);
+ __i915_gem_object_set_pages(obj, st, i915_sg_dma_sizes(st->sgl));
return 0;
-err_sg:
- sg_mark_end(sg);
err_pages:
- mapping_clear_unevictable(mapping);
- if (sg != st->sgl) {
- struct pagevec pvec;
-
- pagevec_init(&pvec);
- for_each_sgt_page(page, sgt_iter, st) {
- if (!pagevec_add(&pvec, page))
- check_release_pagevec(&pvec);
- }
- if (pagevec_count(&pvec))
- check_release_pagevec(&pvec);
- }
- sg_free_table(st);
- kfree(st);
-
+ shmem_sg_free_table(st, mapping, false, false);
/*
* shmemfs first checks if there is enough memory to allocate the page
* and reports ENOSPC should there be insufficient, along with the usual
@@ -216,13 +258,16 @@ static int shmem_get_pages(struct drm_i915_gem_object *obj)
* space and so want to translate the error from shmemfs back to our
* usual understanding of ENOMEM.
*/
+err_st:
if (ret == -ENOSPC)
ret = -ENOMEM;
+ kfree(st);
+
return ret;
}
-static void
+static int
shmem_truncate(struct drm_i915_gem_object *obj)
{
/*
@@ -234,12 +279,12 @@ shmem_truncate(struct drm_i915_gem_object *obj)
shmem_truncate_range(file_inode(obj->base.filp), 0, (loff_t)-1);
obj->mm.madv = __I915_MADV_PURGED;
obj->mm.pages = ERR_PTR(-EFAULT);
+
+ return 0;
}
-static void
-shmem_writeback(struct drm_i915_gem_object *obj)
+void __shmem_writeback(size_t size, struct address_space *mapping)
{
- struct address_space *mapping;
struct writeback_control wbc = {
.sync_mode = WB_SYNC_NONE,
.nr_to_write = SWAP_CLUSTER_MAX,
@@ -255,10 +300,9 @@ shmem_writeback(struct drm_i915_gem_object *obj)
* instead of invoking writeback so they are aged and paged out
* as normal.
*/
- mapping = obj->base.filp->f_mapping;
/* Begin writeback on each dirty page */
- for (i = 0; i < obj->base.size >> PAGE_SHIFT; i++) {
+ for (i = 0; i < size >> PAGE_SHIFT; i++) {
struct page *page;
page = find_lock_page(mapping, i);
@@ -281,6 +325,12 @@ shmem_writeback(struct drm_i915_gem_object *obj)
}
}
+static void
+shmem_writeback(struct drm_i915_gem_object *obj)
+{
+ __shmem_writeback(obj->base.size, obj->base.filp->f_mapping);
+}
+
void
__i915_gem_object_release_shmem(struct drm_i915_gem_object *obj,
struct sg_table *pages,
@@ -313,11 +363,6 @@ __i915_gem_object_release_shmem(struct drm_i915_gem_object *obj,
void i915_gem_object_put_pages_shmem(struct drm_i915_gem_object *obj, struct sg_table *pages)
{
- struct sgt_iter sgt_iter;
- struct pagevec pvec;
- struct page *page;
-
- GEM_WARN_ON(IS_DGFX(to_i915(obj->base.dev)));
__i915_gem_object_release_shmem(obj, pages, true);
i915_gem_gtt_finish_pages(obj, pages);
@@ -325,25 +370,10 @@ void i915_gem_object_put_pages_shmem(struct drm_i915_gem_object *obj, struct sg_
if (i915_gem_object_needs_bit17_swizzle(obj))
i915_gem_object_save_bit_17_swizzle(obj, pages);
- mapping_clear_unevictable(file_inode(obj->base.filp)->i_mapping);
-
- pagevec_init(&pvec);
- for_each_sgt_page(page, sgt_iter, pages) {
- if (obj->mm.dirty)
- set_page_dirty(page);
-
- if (obj->mm.madv == I915_MADV_WILLNEED)
- mark_page_accessed(page);
-
- if (!pagevec_add(&pvec, page))
- check_release_pagevec(&pvec);
- }
- if (pagevec_count(&pvec))
- check_release_pagevec(&pvec);
- obj->mm.dirty = false;
-
- sg_free_table(pages);
+ shmem_sg_free_table(pages, file_inode(obj->base.filp)->i_mapping,
+ obj->mm.dirty, obj->mm.madv == I915_MADV_WILLNEED);
kfree(pages);
+ obj->mm.dirty = false;
}
static void
@@ -634,9 +664,10 @@ static int init_shmem(struct intel_memory_region *mem)
return 0; /* Don't error, we can simply fallback to the kernel mnt */
}
-static void release_shmem(struct intel_memory_region *mem)
+static int release_shmem(struct intel_memory_region *mem)
{
i915_gemfs_fini(mem->i915);
+ return 0;
}
static const struct intel_memory_region_ops shmem_region_ops = {
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_shrinker.c b/drivers/gpu/drm/i915/gem/i915_gem_shrinker.c
index 5ab136f..cc927e4 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_shrinker.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_shrinker.c
@@ -15,7 +15,6 @@
#include "gt/intel_gt_requests.h"
-#include "dma_resv_utils.h"
#include "i915_trace.h"
static bool swap_available(void)
@@ -37,8 +36,8 @@ static bool can_release_pages(struct drm_i915_gem_object *obj)
return swap_available() || obj->mm.madv == I915_MADV_DONTNEED;
}
-static bool unsafe_drop_pages(struct drm_i915_gem_object *obj,
- unsigned long shrink, bool trylock_vm)
+static int drop_pages(struct drm_i915_gem_object *obj,
+ unsigned long shrink, bool trylock_vm)
{
unsigned long flags;
@@ -56,19 +55,25 @@ static bool unsafe_drop_pages(struct drm_i915_gem_object *obj,
return false;
}
-static void try_to_writeback(struct drm_i915_gem_object *obj,
- unsigned int flags)
+static int try_to_writeback(struct drm_i915_gem_object *obj, unsigned int flags)
{
+ if (obj->ops->shrinker_release_pages)
+ return obj->ops->shrinker_release_pages(obj,
+ !(flags & I915_SHRINK_ACTIVE),
+ flags & I915_SHRINK_WRITEBACK);
+
switch (obj->mm.madv) {
case I915_MADV_DONTNEED:
i915_gem_object_truncate(obj);
- return;
+ return 0;
case __I915_MADV_PURGED:
- return;
+ return 0;
}
if (flags & I915_SHRINK_WRITEBACK)
i915_gem_object_writeback(obj);
+
+ return 0;
}
/**
@@ -148,7 +153,7 @@ i915_gem_shrink(struct i915_gem_ww_ctx *ww,
*/
if (shrink & I915_SHRINK_ACTIVE)
/* Retire requests to unpin all idle contexts */
- intel_gt_retire_requests(&i915->gt);
+ intel_gt_retire_requests(to_gt(i915));
/*
* As we may completely rewrite the (un)bound list whilst unbinding
@@ -209,27 +214,23 @@ i915_gem_shrink(struct i915_gem_ww_ctx *ww,
spin_unlock_irqrestore(&i915->mm.obj_lock, flags);
- err = 0;
- if (unsafe_drop_pages(obj, shrink, trylock_vm)) {
- /* May arrive from get_pages on another bo */
- if (!ww) {
- if (!i915_gem_object_trylock(obj))
- goto skip;
- } else {
- err = i915_gem_object_lock(obj, ww);
- if (err)
- goto skip;
- }
-
- if (!__i915_gem_object_put_pages(obj)) {
- try_to_writeback(obj, shrink);
- count += obj->base.size >> PAGE_SHIFT;
- }
- if (!ww)
- i915_gem_object_unlock(obj);
+ /* May arrive from get_pages on another bo */
+ if (!ww) {
+ if (!i915_gem_object_trylock(obj, NULL))
+ goto skip;
+ } else {
+ err = i915_gem_object_lock(obj, ww);
+ if (err)
+ goto skip;
}
- dma_resv_prune(obj->base.resv);
+ if (drop_pages(obj, shrink, trylock_vm) &&
+ !__i915_gem_object_put_pages(obj) &&
+ !try_to_writeback(obj, shrink))
+ count += obj->base.size >> PAGE_SHIFT;
+
+ if (!ww)
+ i915_gem_object_unlock(obj);
scanned += obj->base.size >> PAGE_SHIFT;
skip:
@@ -404,12 +405,18 @@ i915_gem_shrinker_vmap(struct notifier_block *nb, unsigned long event, void *ptr
list_for_each_entry_safe(vma, next,
&i915->ggtt.vm.bound_list, vm_link) {
unsigned long count = vma->node.size >> PAGE_SHIFT;
+ struct drm_i915_gem_object *obj = vma->obj;
if (!vma->iomap || i915_vma_is_active(vma))
continue;
+ if (!i915_gem_object_trylock(obj, NULL))
+ continue;
+
if (__i915_vma_unbind(vma) == 0)
freed_pages += count;
+
+ i915_gem_object_unlock(obj);
}
mutex_unlock(&i915->ggtt.vm.mutex);
@@ -458,6 +465,16 @@ void i915_gem_shrinker_taints_mutex(struct drm_i915_private *i915,
#define obj_to_i915(obj__) to_i915((obj__)->base.dev)
+/**
+ * i915_gem_object_make_unshrinkable - Hide the object from the shrinker. By
+ * default all object types that support shrinking(see IS_SHRINKABLE), will also
+ * make the object visible to the shrinker after allocating the system memory
+ * pages.
+ * @obj: The GEM object.
+ *
+ * This is typically used for special kernel internal objects that can't be
+ * easily processed by the shrinker, like if they are perma-pinned.
+ */
void i915_gem_object_make_unshrinkable(struct drm_i915_gem_object *obj)
{
struct drm_i915_private *i915 = obj_to_i915(obj);
@@ -482,13 +499,12 @@ void i915_gem_object_make_unshrinkable(struct drm_i915_gem_object *obj)
spin_unlock_irqrestore(&i915->mm.obj_lock, flags);
}
-static void __i915_gem_object_make_shrinkable(struct drm_i915_gem_object *obj,
- struct list_head *head)
+static void ___i915_gem_object_make_shrinkable(struct drm_i915_gem_object *obj,
+ struct list_head *head)
{
struct drm_i915_private *i915 = obj_to_i915(obj);
unsigned long flags;
- GEM_BUG_ON(!i915_gem_object_has_pages(obj));
if (!i915_gem_object_is_shrinkable(obj))
return;
@@ -508,14 +524,67 @@ static void __i915_gem_object_make_shrinkable(struct drm_i915_gem_object *obj,
spin_unlock_irqrestore(&i915->mm.obj_lock, flags);
}
-void i915_gem_object_make_shrinkable(struct drm_i915_gem_object *obj)
+/**
+ * __i915_gem_object_make_shrinkable - Move the object to the tail of the
+ * shrinkable list. Objects on this list might be swapped out. Used with
+ * WILLNEED objects.
+ * @obj: The GEM object.
+ *
+ * DO NOT USE. This is intended to be called on very special objects that don't
+ * yet have mm.pages, but are guaranteed to have potentially reclaimable pages
+ * underneath.
+ */
+void __i915_gem_object_make_shrinkable(struct drm_i915_gem_object *obj)
{
- __i915_gem_object_make_shrinkable(obj,
- &obj_to_i915(obj)->mm.shrink_list);
+ ___i915_gem_object_make_shrinkable(obj,
+ &obj_to_i915(obj)->mm.shrink_list);
}
+/**
+ * __i915_gem_object_make_purgeable - Move the object to the tail of the
+ * purgeable list. Objects on this list might be swapped out. Used with
+ * DONTNEED objects.
+ * @obj: The GEM object.
+ *
+ * DO NOT USE. This is intended to be called on very special objects that don't
+ * yet have mm.pages, but are guaranteed to have potentially reclaimable pages
+ * underneath.
+ */
+void __i915_gem_object_make_purgeable(struct drm_i915_gem_object *obj)
+{
+ ___i915_gem_object_make_shrinkable(obj,
+ &obj_to_i915(obj)->mm.purge_list);
+}
+
+/**
+ * i915_gem_object_make_shrinkable - Move the object to the tail of the
+ * shrinkable list. Objects on this list might be swapped out. Used with
+ * WILLNEED objects.
+ * @obj: The GEM object.
+ *
+ * MUST only be called on objects which have backing pages.
+ *
+ * MUST be balanced with previous call to i915_gem_object_make_unshrinkable().
+ */
+void i915_gem_object_make_shrinkable(struct drm_i915_gem_object *obj)
+{
+ GEM_BUG_ON(!i915_gem_object_has_pages(obj));
+ __i915_gem_object_make_shrinkable(obj);
+}
+
+/**
+ * i915_gem_object_make_purgeable - Move the object to the tail of the purgeable
+ * list. Used with DONTNEED objects. Unlike with shrinkable objects, the
+ * shrinker will attempt to discard the backing pages, instead of trying to swap
+ * them out.
+ * @obj: The GEM object.
+ *
+ * MUST only be called on objects which have backing pages.
+ *
+ * MUST be balanced with previous call to i915_gem_object_make_unshrinkable().
+ */
void i915_gem_object_make_purgeable(struct drm_i915_gem_object *obj)
{
- __i915_gem_object_make_shrinkable(obj,
- &obj_to_i915(obj)->mm.purge_list);
+ GEM_BUG_ON(!i915_gem_object_has_pages(obj));
+ __i915_gem_object_make_purgeable(obj);
}
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_stolen.c b/drivers/gpu/drm/i915/gem/i915_gem_stolen.c
index ddd37cc..7df50fd 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_stolen.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_stolen.c
@@ -399,7 +399,7 @@ static int i915_gem_init_stolen(struct intel_memory_region *mem)
return 0;
}
- if (intel_vtd_active() && GRAPHICS_VER(i915) < 8) {
+ if (intel_vtd_active(i915) && GRAPHICS_VER(i915) < 8) {
drm_notice(&i915->drm,
"%s, disabling use of stolen memory\n",
"DMAR active");
@@ -488,6 +488,9 @@ static int i915_gem_init_stolen(struct intel_memory_region *mem)
return 0;
}
+ /* Exclude the reserved region from driver use */
+ mem->region.end = reserved_base - 1;
+
/* It is possible for the reserved area to end before the end of stolen
* memory, so just consider the start. */
reserved_total = stolen_top - reserved_base;
@@ -653,7 +656,7 @@ static int __i915_gem_object_create_stolen(struct intel_memory_region *mem,
cache_level = HAS_LLC(mem->i915) ? I915_CACHE_LLC : I915_CACHE_NONE;
i915_gem_object_set_cache_coherency(obj, cache_level);
- if (WARN_ON(!i915_gem_object_trylock(obj)))
+ if (WARN_ON(!i915_gem_object_trylock(obj, NULL)))
return -EBUSY;
i915_gem_object_init_memory_region(obj, mem);
@@ -720,9 +723,10 @@ static int init_stolen_smem(struct intel_memory_region *mem)
return i915_gem_init_stolen(mem);
}
-static void release_stolen_smem(struct intel_memory_region *mem)
+static int release_stolen_smem(struct intel_memory_region *mem)
{
i915_gem_cleanup_stolen(mem->i915);
+ return 0;
}
static const struct intel_memory_region_ops i915_region_stolen_smem_ops = {
@@ -759,10 +763,11 @@ static int init_stolen_lmem(struct intel_memory_region *mem)
return err;
}
-static void release_stolen_lmem(struct intel_memory_region *mem)
+static int release_stolen_lmem(struct intel_memory_region *mem)
{
io_mapping_fini(&mem->iomap);
i915_gem_cleanup_stolen(mem->i915);
+ return 0;
}
static const struct intel_memory_region_ops i915_region_stolen_lmem_ops = {
@@ -778,6 +783,7 @@ i915_gem_stolen_lmem_setup(struct drm_i915_private *i915, u16 type,
struct intel_uncore *uncore = &i915->uncore;
struct pci_dev *pdev = to_pci_dev(i915->drm.dev);
struct intel_memory_region *mem;
+ resource_size_t min_page_size;
resource_size_t io_start;
resource_size_t lmem_size;
u64 lmem_base;
@@ -789,8 +795,11 @@ i915_gem_stolen_lmem_setup(struct drm_i915_private *i915, u16 type,
lmem_size = pci_resource_len(pdev, 2) - lmem_base;
io_start = pci_resource_start(pdev, 2) + lmem_base;
+ min_page_size = HAS_64K_PAGES(i915) ? I915_GTT_PAGE_SIZE_64K :
+ I915_GTT_PAGE_SIZE_4K;
+
mem = intel_memory_region_create(i915, lmem_base, lmem_size,
- I915_GTT_PAGE_SIZE_4K, io_start,
+ min_page_size, io_start,
type, instance,
&i915_region_stolen_lmem_ops);
if (IS_ERR(mem))
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_throttle.c b/drivers/gpu/drm/i915/gem/i915_gem_throttle.c
index 1929d6c..75501db 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_throttle.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_throttle.c
@@ -38,12 +38,13 @@ i915_gem_throttle_ioctl(struct drm_device *dev, void *data,
{
const unsigned long recent_enough = jiffies - DRM_I915_THROTTLE_JIFFIES;
struct drm_i915_file_private *file_priv = file->driver_priv;
+ struct drm_i915_private *i915 = to_i915(dev);
struct i915_gem_context *ctx;
unsigned long idx;
long ret;
/* ABI: return -EIO if already wedged */
- ret = intel_gt_terminally_wedged(&to_i915(dev)->gt);
+ ret = intel_gt_terminally_wedged(to_gt(i915));
if (ret)
return ret;
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_ttm.c b/drivers/gpu/drm/i915/gem/i915_gem_ttm.c
index 74a1ffd..923cc7a 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_ttm.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_ttm.c
@@ -14,13 +14,9 @@
#include "gem/i915_gem_object.h"
#include "gem/i915_gem_region.h"
#include "gem/i915_gem_ttm.h"
+#include "gem/i915_gem_ttm_move.h"
#include "gem/i915_gem_ttm_pm.h"
-
-#include "gt/intel_engine_pm.h"
-#include "gt/intel_gt.h"
-#include "gt/intel_migrate.h"
-
#define I915_TTM_PRIO_PURGE 0
#define I915_TTM_PRIO_NO_PAGES 1
#define I915_TTM_PRIO_HAS_PAGES 2
@@ -34,7 +30,9 @@
* struct i915_ttm_tt - TTM page vector with additional private information
* @ttm: The base TTM page vector.
* @dev: The struct device used for dma mapping and unmapping.
- * @cached_st: The cached scatter-gather table.
+ * @cached_rsgt: The cached scatter-gather table.
+ * @is_shmem: Set if using shmem.
+ * @filp: The shmem file, if using shmem backend.
*
* Note that DMA may be going on right up to the point where the page-
* vector is unpopulated in delayed destroy. Hence keep the
@@ -45,7 +43,10 @@
struct i915_ttm_tt {
struct ttm_tt ttm;
struct device *dev;
- struct sg_table *cached_st;
+ struct i915_refct_sgt cached_rsgt;
+
+ bool is_shmem;
+ struct file *filp;
};
static const struct ttm_place sys_placement_flags = {
@@ -103,37 +104,15 @@ static int i915_ttm_err_to_gem(int err)
return err;
}
-static bool gpu_binds_iomem(struct ttm_resource *mem)
-{
- return mem->mem_type != TTM_PL_SYSTEM;
-}
-
-static bool cpu_maps_iomem(struct ttm_resource *mem)
-{
- /* Once / if we support GGTT, this is also false for cached ttm_tts */
- return mem->mem_type != TTM_PL_SYSTEM;
-}
-
-static enum i915_cache_level
-i915_ttm_cache_level(struct drm_i915_private *i915, struct ttm_resource *res,
- struct ttm_tt *ttm)
-{
- return ((HAS_LLC(i915) || HAS_SNOOP(i915)) && !gpu_binds_iomem(res) &&
- ttm->caching == ttm_cached) ? I915_CACHE_LLC :
- I915_CACHE_NONE;
-}
-
-static void i915_ttm_adjust_lru(struct drm_i915_gem_object *obj);
-
static enum ttm_caching
i915_ttm_select_tt_caching(const struct drm_i915_gem_object *obj)
{
/*
- * Objects only allowed in system get cached cpu-mappings.
- * Other objects get WC mapping for now. Even if in system.
+ * Objects only allowed in system get cached cpu-mappings, or when
+ * evicting lmem-only buffers to system for swapping. Other objects get
+ * WC mapping for now. Even if in system.
*/
- if (obj->mm.region->type == INTEL_MEMORY_SYSTEM &&
- obj->mm.n_placements <= 1)
+ if (obj->mm.n_placements <= 1)
return ttm_cached;
return ttm_write_combined;
@@ -179,15 +158,103 @@ i915_ttm_placement_from_obj(const struct drm_i915_gem_object *obj,
placement->busy_placement = busy;
}
+static int i915_ttm_tt_shmem_populate(struct ttm_device *bdev,
+ struct ttm_tt *ttm,
+ struct ttm_operation_ctx *ctx)
+{
+ struct drm_i915_private *i915 = container_of(bdev, typeof(*i915), bdev);
+ struct intel_memory_region *mr = i915->mm.regions[INTEL_MEMORY_SYSTEM];
+ struct i915_ttm_tt *i915_tt = container_of(ttm, typeof(*i915_tt), ttm);
+ const unsigned int max_segment = i915_sg_segment_size();
+ const size_t size = (size_t)ttm->num_pages << PAGE_SHIFT;
+ struct file *filp = i915_tt->filp;
+ struct sgt_iter sgt_iter;
+ struct sg_table *st;
+ struct page *page;
+ unsigned long i;
+ int err;
+
+ if (!filp) {
+ struct address_space *mapping;
+ gfp_t mask;
+
+ filp = shmem_file_setup("i915-shmem-tt", size, VM_NORESERVE);
+ if (IS_ERR(filp))
+ return PTR_ERR(filp);
+
+ mask = GFP_HIGHUSER | __GFP_RECLAIMABLE;
+
+ mapping = filp->f_mapping;
+ mapping_set_gfp_mask(mapping, mask);
+ GEM_BUG_ON(!(mapping_gfp_mask(mapping) & __GFP_RECLAIM));
+
+ i915_tt->filp = filp;
+ }
+
+ st = &i915_tt->cached_rsgt.table;
+ err = shmem_sg_alloc_table(i915, st, size, mr, filp->f_mapping,
+ max_segment);
+ if (err)
+ return err;
+
+ err = dma_map_sgtable(i915_tt->dev, st, DMA_BIDIRECTIONAL,
+ DMA_ATTR_SKIP_CPU_SYNC);
+ if (err)
+ goto err_free_st;
+
+ i = 0;
+ for_each_sgt_page(page, sgt_iter, st)
+ ttm->pages[i++] = page;
+
+ if (ttm->page_flags & TTM_TT_FLAG_SWAPPED)
+ ttm->page_flags &= ~TTM_TT_FLAG_SWAPPED;
+
+ return 0;
+
+err_free_st:
+ shmem_sg_free_table(st, filp->f_mapping, false, false);
+
+ return err;
+}
+
+static void i915_ttm_tt_shmem_unpopulate(struct ttm_tt *ttm)
+{
+ struct i915_ttm_tt *i915_tt = container_of(ttm, typeof(*i915_tt), ttm);
+ bool backup = ttm->page_flags & TTM_TT_FLAG_SWAPPED;
+ struct sg_table *st = &i915_tt->cached_rsgt.table;
+
+ shmem_sg_free_table(st, file_inode(i915_tt->filp)->i_mapping,
+ backup, backup);
+}
+
+static void i915_ttm_tt_release(struct kref *ref)
+{
+ struct i915_ttm_tt *i915_tt =
+ container_of(ref, typeof(*i915_tt), cached_rsgt.kref);
+ struct sg_table *st = &i915_tt->cached_rsgt.table;
+
+ GEM_WARN_ON(st->sgl);
+
+ kfree(i915_tt);
+}
+
+static const struct i915_refct_sgt_ops tt_rsgt_ops = {
+ .release = i915_ttm_tt_release
+};
+
static struct ttm_tt *i915_ttm_tt_create(struct ttm_buffer_object *bo,
uint32_t page_flags)
{
struct ttm_resource_manager *man =
ttm_manager_type(bo->bdev, bo->resource->mem_type);
struct drm_i915_gem_object *obj = i915_ttm_to_gem(bo);
+ enum ttm_caching caching;
struct i915_ttm_tt *i915_tt;
int ret;
+ if (!obj)
+ return NULL;
+
i915_tt = kzalloc(sizeof(*i915_tt), GFP_KERNEL);
if (!i915_tt)
return NULL;
@@ -196,38 +263,66 @@ static struct ttm_tt *i915_ttm_tt_create(struct ttm_buffer_object *bo,
man->use_tt)
page_flags |= TTM_TT_FLAG_ZERO_ALLOC;
- ret = ttm_tt_init(&i915_tt->ttm, bo, page_flags,
- i915_ttm_select_tt_caching(obj));
- if (ret) {
- kfree(i915_tt);
- return NULL;
+ caching = i915_ttm_select_tt_caching(obj);
+ if (i915_gem_object_is_shrinkable(obj) && caching == ttm_cached) {
+ page_flags |= TTM_TT_FLAG_EXTERNAL |
+ TTM_TT_FLAG_EXTERNAL_MAPPABLE;
+ i915_tt->is_shmem = true;
}
+ ret = ttm_tt_init(&i915_tt->ttm, bo, page_flags, caching);
+ if (ret)
+ goto err_free;
+
+ __i915_refct_sgt_init(&i915_tt->cached_rsgt, bo->base.size,
+ &tt_rsgt_ops);
+
i915_tt->dev = obj->base.dev->dev;
return &i915_tt->ttm;
+
+err_free:
+ kfree(i915_tt);
+ return NULL;
+}
+
+static int i915_ttm_tt_populate(struct ttm_device *bdev,
+ struct ttm_tt *ttm,
+ struct ttm_operation_ctx *ctx)
+{
+ struct i915_ttm_tt *i915_tt = container_of(ttm, typeof(*i915_tt), ttm);
+
+ if (i915_tt->is_shmem)
+ return i915_ttm_tt_shmem_populate(bdev, ttm, ctx);
+
+ return ttm_pool_alloc(&bdev->pool, ttm, ctx);
}
static void i915_ttm_tt_unpopulate(struct ttm_device *bdev, struct ttm_tt *ttm)
{
struct i915_ttm_tt *i915_tt = container_of(ttm, typeof(*i915_tt), ttm);
+ struct sg_table *st = &i915_tt->cached_rsgt.table;
- if (i915_tt->cached_st) {
- dma_unmap_sgtable(i915_tt->dev, i915_tt->cached_st,
- DMA_BIDIRECTIONAL, 0);
- sg_free_table(i915_tt->cached_st);
- kfree(i915_tt->cached_st);
- i915_tt->cached_st = NULL;
+ if (st->sgl)
+ dma_unmap_sgtable(i915_tt->dev, st, DMA_BIDIRECTIONAL, 0);
+
+ if (i915_tt->is_shmem) {
+ i915_ttm_tt_shmem_unpopulate(ttm);
+ } else {
+ sg_free_table(st);
+ ttm_pool_free(&bdev->pool, ttm);
}
- ttm_pool_free(&bdev->pool, ttm);
}
static void i915_ttm_tt_destroy(struct ttm_device *bdev, struct ttm_tt *ttm)
{
struct i915_ttm_tt *i915_tt = container_of(ttm, typeof(*i915_tt), ttm);
+ if (i915_tt->filp)
+ fput(i915_tt->filp);
+
ttm_tt_fini(ttm);
- kfree(i915_tt);
+ i915_refct_sgt_put(&i915_tt->cached_rsgt);
}
static bool i915_ttm_eviction_valuable(struct ttm_buffer_object *bo,
@@ -235,6 +330,17 @@ static bool i915_ttm_eviction_valuable(struct ttm_buffer_object *bo,
{
struct drm_i915_gem_object *obj = i915_ttm_to_gem(bo);
+ if (!obj)
+ return false;
+
+ /*
+ * EXTERNAL objects should never be swapped out by TTM, instead we need
+ * to handle that ourselves. TTM will already skip such objects for us,
+ * but we would like to avoid grabbing locks for no good reason.
+ */
+ if (bo->ttm && bo->ttm->page_flags & TTM_TT_FLAG_EXTERNAL)
+ return false;
+
/* Will do for now. Our pinned objects are still on TTM's LRU lists */
return i915_gem_object_evictable(obj);
}
@@ -245,28 +351,19 @@ static void i915_ttm_evict_flags(struct ttm_buffer_object *bo,
*placement = i915_sys_placement;
}
-static int i915_ttm_move_notify(struct ttm_buffer_object *bo)
-{
- struct drm_i915_gem_object *obj = i915_ttm_to_gem(bo);
- int ret;
-
- ret = i915_gem_object_unbind(obj, I915_GEM_OBJECT_UNBIND_ACTIVE);
- if (ret)
- return ret;
-
- ret = __i915_gem_object_put_pages(obj);
- if (ret)
- return ret;
-
- return 0;
-}
-
-static void i915_ttm_free_cached_io_st(struct drm_i915_gem_object *obj)
+/**
+ * i915_ttm_free_cached_io_rsgt - Free object cached LMEM information
+ * @obj: The GEM object
+ * This function frees any LMEM-related information that is cached on
+ * the object. For example the radix tree for fast page lookup and the
+ * cached refcounted sg-table
+ */
+void i915_ttm_free_cached_io_rsgt(struct drm_i915_gem_object *obj)
{
struct radix_tree_iter iter;
void __rcu **slot;
- if (!obj->ttm.cached_io_st)
+ if (!obj->ttm.cached_io_rsgt)
return;
rcu_read_lock();
@@ -274,63 +371,24 @@ static void i915_ttm_free_cached_io_st(struct drm_i915_gem_object *obj)
radix_tree_delete(&obj->ttm.get_io_page.radix, iter.index);
rcu_read_unlock();
- sg_free_table(obj->ttm.cached_io_st);
- kfree(obj->ttm.cached_io_st);
- obj->ttm.cached_io_st = NULL;
+ i915_refct_sgt_put(obj->ttm.cached_io_rsgt);
+ obj->ttm.cached_io_rsgt = NULL;
}
-static void
-i915_ttm_adjust_domains_after_move(struct drm_i915_gem_object *obj)
+/**
+ * i915_ttm_purge - Clear an object of its memory
+ * @obj: The object
+ *
+ * This function is called to clear an object of it's memory when it is
+ * marked as not needed anymore.
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+int i915_ttm_purge(struct drm_i915_gem_object *obj)
{
struct ttm_buffer_object *bo = i915_gem_to_ttm(obj);
-
- if (cpu_maps_iomem(bo->resource) || bo->ttm->caching != ttm_cached) {
- obj->write_domain = I915_GEM_DOMAIN_WC;
- obj->read_domains = I915_GEM_DOMAIN_WC;
- } else {
- obj->write_domain = I915_GEM_DOMAIN_CPU;
- obj->read_domains = I915_GEM_DOMAIN_CPU;
- }
-}
-
-static void i915_ttm_adjust_gem_after_move(struct drm_i915_gem_object *obj)
-{
- struct ttm_buffer_object *bo = i915_gem_to_ttm(obj);
- unsigned int cache_level;
- unsigned int i;
-
- /*
- * If object was moved to an allowable region, update the object
- * region to consider it migrated. Note that if it's currently not
- * in an allowable region, it's evicted and we don't update the
- * object region.
- */
- if (intel_region_to_ttm_type(obj->mm.region) != bo->resource->mem_type) {
- for (i = 0; i < obj->mm.n_placements; ++i) {
- struct intel_memory_region *mr = obj->mm.placements[i];
-
- if (intel_region_to_ttm_type(mr) == bo->resource->mem_type &&
- mr != obj->mm.region) {
- i915_gem_object_release_memory_region(obj);
- i915_gem_object_init_memory_region(obj, mr);
- break;
- }
- }
- }
-
- obj->mem_flags &= ~(I915_BO_FLAG_STRUCT_PAGE | I915_BO_FLAG_IOMEM);
-
- obj->mem_flags |= cpu_maps_iomem(bo->resource) ? I915_BO_FLAG_IOMEM :
- I915_BO_FLAG_STRUCT_PAGE;
-
- cache_level = i915_ttm_cache_level(to_i915(bo->base.dev), bo->resource,
- bo->ttm);
- i915_gem_object_set_cache_coherency(obj, cache_level);
-}
-
-static void i915_ttm_purge(struct drm_i915_gem_object *obj)
-{
- struct ttm_buffer_object *bo = i915_gem_to_ttm(obj);
+ struct i915_ttm_tt *i915_tt =
+ container_of(bo->ttm, typeof(*i915_tt), ttm);
struct ttm_operation_ctx ctx = {
.interruptible = true,
.no_wait_gpu = false,
@@ -339,28 +397,80 @@ static void i915_ttm_purge(struct drm_i915_gem_object *obj)
int ret;
if (obj->mm.madv == __I915_MADV_PURGED)
- return;
+ return 0;
- /* TTM's purge interface. Note that we might be reentering. */
ret = ttm_bo_validate(bo, &place, &ctx);
- if (!ret) {
- obj->write_domain = 0;
- obj->read_domains = 0;
- i915_ttm_adjust_gem_after_move(obj);
- i915_ttm_free_cached_io_st(obj);
- obj->mm.madv = __I915_MADV_PURGED;
+ if (ret)
+ return ret;
+
+ if (bo->ttm && i915_tt->filp) {
+ /*
+ * The below fput(which eventually calls shmem_truncate) might
+ * be delayed by worker, so when directly called to purge the
+ * pages(like by the shrinker) we should try to be more
+ * aggressive and release the pages immediately.
+ */
+ shmem_truncate_range(file_inode(i915_tt->filp),
+ 0, (loff_t)-1);
+ fput(fetch_and_zero(&i915_tt->filp));
}
+
+ obj->write_domain = 0;
+ obj->read_domains = 0;
+ i915_ttm_adjust_gem_after_move(obj);
+ i915_ttm_free_cached_io_rsgt(obj);
+ obj->mm.madv = __I915_MADV_PURGED;
+
+ return 0;
}
-static void i915_ttm_swap_notify(struct ttm_buffer_object *bo)
+static int i915_ttm_shrinker_release_pages(struct drm_i915_gem_object *obj,
+ bool no_wait_gpu,
+ bool should_writeback)
{
- struct drm_i915_gem_object *obj = i915_ttm_to_gem(bo);
- int ret = i915_ttm_move_notify(bo);
+ struct ttm_buffer_object *bo = i915_gem_to_ttm(obj);
+ struct i915_ttm_tt *i915_tt =
+ container_of(bo->ttm, typeof(*i915_tt), ttm);
+ struct ttm_operation_ctx ctx = {
+ .interruptible = true,
+ .no_wait_gpu = no_wait_gpu,
+ };
+ struct ttm_placement place = {};
+ int ret;
- GEM_WARN_ON(ret);
- GEM_WARN_ON(obj->ttm.cached_io_st);
- if (!ret && obj->mm.madv != I915_MADV_WILLNEED)
- i915_ttm_purge(obj);
+ if (!bo->ttm || bo->resource->mem_type != TTM_PL_SYSTEM)
+ return 0;
+
+ GEM_BUG_ON(!i915_tt->is_shmem);
+
+ if (!i915_tt->filp)
+ return 0;
+
+ ret = ttm_bo_wait_ctx(bo, &ctx);
+ if (ret)
+ return ret;
+
+ switch (obj->mm.madv) {
+ case I915_MADV_DONTNEED:
+ return i915_ttm_purge(obj);
+ case __I915_MADV_PURGED:
+ return 0;
+ }
+
+ if (bo->ttm->page_flags & TTM_TT_FLAG_SWAPPED)
+ return 0;
+
+ bo->ttm->page_flags |= TTM_TT_FLAG_SWAPPED;
+ ret = ttm_bo_validate(bo, &place, &ctx);
+ if (ret) {
+ bo->ttm->page_flags &= ~TTM_TT_FLAG_SWAPPED;
+ return ret;
+ }
+
+ if (should_writeback)
+ __shmem_writeback(obj->base.size, i915_tt->filp->f_mapping);
+
+ return 0;
}
static void i915_ttm_delete_mem_notify(struct ttm_buffer_object *bo)
@@ -369,232 +479,101 @@ static void i915_ttm_delete_mem_notify(struct ttm_buffer_object *bo)
if (likely(obj)) {
__i915_gem_object_pages_fini(obj);
- i915_ttm_free_cached_io_st(obj);
+ i915_ttm_free_cached_io_rsgt(obj);
}
}
-static struct intel_memory_region *
-i915_ttm_region(struct ttm_device *bdev, int ttm_mem_type)
-{
- struct drm_i915_private *i915 = container_of(bdev, typeof(*i915), bdev);
-
- /* There's some room for optimization here... */
- GEM_BUG_ON(ttm_mem_type != I915_PL_SYSTEM &&
- ttm_mem_type < I915_PL_LMEM0);
- if (ttm_mem_type == I915_PL_SYSTEM)
- return intel_memory_region_lookup(i915, INTEL_MEMORY_SYSTEM,
- 0);
-
- return intel_memory_region_lookup(i915, INTEL_MEMORY_LOCAL,
- ttm_mem_type - I915_PL_LMEM0);
-}
-
-static struct sg_table *i915_ttm_tt_get_st(struct ttm_tt *ttm)
+static struct i915_refct_sgt *i915_ttm_tt_get_st(struct ttm_tt *ttm)
{
struct i915_ttm_tt *i915_tt = container_of(ttm, typeof(*i915_tt), ttm);
struct sg_table *st;
int ret;
- if (i915_tt->cached_st)
- return i915_tt->cached_st;
+ if (i915_tt->cached_rsgt.table.sgl)
+ return i915_refct_sgt_get(&i915_tt->cached_rsgt);
- st = kzalloc(sizeof(*st), GFP_KERNEL);
- if (!st)
- return ERR_PTR(-ENOMEM);
-
+ st = &i915_tt->cached_rsgt.table;
ret = sg_alloc_table_from_pages_segment(st,
ttm->pages, ttm->num_pages,
0, (unsigned long)ttm->num_pages << PAGE_SHIFT,
i915_sg_segment_size(), GFP_KERNEL);
if (ret) {
- kfree(st);
+ st->sgl = NULL;
return ERR_PTR(ret);
}
ret = dma_map_sgtable(i915_tt->dev, st, DMA_BIDIRECTIONAL, 0);
if (ret) {
sg_free_table(st);
- kfree(st);
return ERR_PTR(ret);
}
- i915_tt->cached_st = st;
- return st;
+ return i915_refct_sgt_get(&i915_tt->cached_rsgt);
}
-static struct sg_table *
+/**
+ * i915_ttm_resource_get_st - Get a refcounted sg-table pointing to the
+ * resource memory
+ * @obj: The GEM object used for sg-table caching
+ * @res: The struct ttm_resource for which an sg-table is requested.
+ *
+ * This function returns a refcounted sg-table representing the memory
+ * pointed to by @res. If @res is the object's current resource it may also
+ * cache the sg_table on the object or attempt to access an already cached
+ * sg-table. The refcounted sg-table needs to be put when no-longer in use.
+ *
+ * Return: A valid pointer to a struct i915_refct_sgt or error pointer on
+ * failure.
+ */
+struct i915_refct_sgt *
i915_ttm_resource_get_st(struct drm_i915_gem_object *obj,
struct ttm_resource *res)
{
struct ttm_buffer_object *bo = i915_gem_to_ttm(obj);
- if (!gpu_binds_iomem(res))
+ if (!i915_ttm_gtt_binds_lmem(res))
return i915_ttm_tt_get_st(bo->ttm);
/*
* If CPU mapping differs, we need to add the ttm_tt pages to
* the resulting st. Might make sense for GGTT.
*/
- GEM_WARN_ON(!cpu_maps_iomem(res));
- return intel_region_ttm_resource_to_st(obj->mm.region, res);
-}
+ GEM_WARN_ON(!i915_ttm_cpu_maps_iomem(res));
+ if (bo->resource == res) {
+ if (!obj->ttm.cached_io_rsgt) {
+ struct i915_refct_sgt *rsgt;
-static int i915_ttm_accel_move(struct ttm_buffer_object *bo,
- bool clear,
- struct ttm_resource *dst_mem,
- struct ttm_tt *dst_ttm,
- struct sg_table *dst_st)
-{
- struct drm_i915_private *i915 = container_of(bo->bdev, typeof(*i915),
- bdev);
- struct ttm_resource_manager *src_man =
- ttm_manager_type(bo->bdev, bo->resource->mem_type);
- struct drm_i915_gem_object *obj = i915_ttm_to_gem(bo);
- struct sg_table *src_st;
- struct i915_request *rq;
- struct ttm_tt *src_ttm = bo->ttm;
- enum i915_cache_level src_level, dst_level;
- int ret;
+ rsgt = intel_region_ttm_resource_to_rsgt(obj->mm.region,
+ res);
+ if (IS_ERR(rsgt))
+ return rsgt;
- if (!i915->gt.migrate.context || intel_gt_is_wedged(&i915->gt))
- return -EINVAL;
-
- dst_level = i915_ttm_cache_level(i915, dst_mem, dst_ttm);
- if (clear) {
- if (bo->type == ttm_bo_type_kernel)
- return -EINVAL;
-
- intel_engine_pm_get(i915->gt.migrate.context->engine);
- ret = intel_context_migrate_clear(i915->gt.migrate.context, NULL,
- dst_st->sgl, dst_level,
- gpu_binds_iomem(dst_mem),
- 0, &rq);
-
- if (!ret && rq) {
- i915_request_wait(rq, 0, MAX_SCHEDULE_TIMEOUT);
- i915_request_put(rq);
+ obj->ttm.cached_io_rsgt = rsgt;
}
- intel_engine_pm_put(i915->gt.migrate.context->engine);
- } else {
- src_st = src_man->use_tt ? i915_ttm_tt_get_st(src_ttm) :
- obj->ttm.cached_io_st;
-
- src_level = i915_ttm_cache_level(i915, bo->resource, src_ttm);
- intel_engine_pm_get(i915->gt.migrate.context->engine);
- ret = intel_context_migrate_copy(i915->gt.migrate.context,
- NULL, src_st->sgl, src_level,
- gpu_binds_iomem(bo->resource),
- dst_st->sgl, dst_level,
- gpu_binds_iomem(dst_mem),
- &rq);
- if (!ret && rq) {
- i915_request_wait(rq, 0, MAX_SCHEDULE_TIMEOUT);
- i915_request_put(rq);
- }
- intel_engine_pm_put(i915->gt.migrate.context->engine);
+ return i915_refct_sgt_get(obj->ttm.cached_io_rsgt);
}
- return ret;
+ return intel_region_ttm_resource_to_rsgt(obj->mm.region, res);
}
-static void __i915_ttm_move(struct ttm_buffer_object *bo, bool clear,
- struct ttm_resource *dst_mem,
- struct ttm_tt *dst_ttm,
- struct sg_table *dst_st,
- bool allow_accel)
-{
- int ret = -EINVAL;
-
- if (allow_accel)
- ret = i915_ttm_accel_move(bo, clear, dst_mem, dst_ttm, dst_st);
- if (ret) {
- struct drm_i915_gem_object *obj = i915_ttm_to_gem(bo);
- struct intel_memory_region *dst_reg, *src_reg;
- union {
- struct ttm_kmap_iter_tt tt;
- struct ttm_kmap_iter_iomap io;
- } _dst_iter, _src_iter;
- struct ttm_kmap_iter *dst_iter, *src_iter;
-
- dst_reg = i915_ttm_region(bo->bdev, dst_mem->mem_type);
- src_reg = i915_ttm_region(bo->bdev, bo->resource->mem_type);
- GEM_BUG_ON(!dst_reg || !src_reg);
-
- dst_iter = !cpu_maps_iomem(dst_mem) ?
- ttm_kmap_iter_tt_init(&_dst_iter.tt, dst_ttm) :
- ttm_kmap_iter_iomap_init(&_dst_iter.io, &dst_reg->iomap,
- dst_st, dst_reg->region.start);
-
- src_iter = !cpu_maps_iomem(bo->resource) ?
- ttm_kmap_iter_tt_init(&_src_iter.tt, bo->ttm) :
- ttm_kmap_iter_iomap_init(&_src_iter.io, &src_reg->iomap,
- obj->ttm.cached_io_st,
- src_reg->region.start);
-
- ttm_move_memcpy(clear, dst_mem->num_pages, dst_iter, src_iter);
- }
-}
-
-static int i915_ttm_move(struct ttm_buffer_object *bo, bool evict,
- struct ttm_operation_ctx *ctx,
- struct ttm_resource *dst_mem,
- struct ttm_place *hop)
+static void i915_ttm_swap_notify(struct ttm_buffer_object *bo)
{
struct drm_i915_gem_object *obj = i915_ttm_to_gem(bo);
- struct ttm_resource_manager *dst_man =
- ttm_manager_type(bo->bdev, dst_mem->mem_type);
- struct ttm_tt *ttm = bo->ttm;
- struct sg_table *dst_st;
- bool clear;
int ret;
- /* Sync for now. We could do the actual copy async. */
- ret = ttm_bo_wait_ctx(bo, ctx);
- if (ret)
- return ret;
+ if (!obj)
+ return;
ret = i915_ttm_move_notify(bo);
- if (ret)
- return ret;
-
- if (obj->mm.madv != I915_MADV_WILLNEED) {
+ GEM_WARN_ON(ret);
+ GEM_WARN_ON(obj->ttm.cached_io_rsgt);
+ if (!ret && obj->mm.madv != I915_MADV_WILLNEED)
i915_ttm_purge(obj);
- ttm_resource_free(bo, &dst_mem);
- return 0;
- }
-
- /* Populate ttm with pages if needed. Typically system memory. */
- if (ttm && (dst_man->use_tt || (ttm->page_flags & TTM_TT_FLAG_SWAPPED))) {
- ret = ttm_tt_populate(bo->bdev, ttm, ctx);
- if (ret)
- return ret;
- }
-
- dst_st = i915_ttm_resource_get_st(obj, dst_mem);
- if (IS_ERR(dst_st))
- return PTR_ERR(dst_st);
-
- clear = !cpu_maps_iomem(bo->resource) && (!ttm || !ttm_tt_is_populated(ttm));
- if (!(clear && ttm && !(ttm->page_flags & TTM_TT_FLAG_ZERO_ALLOC)))
- __i915_ttm_move(bo, clear, dst_mem, bo->ttm, dst_st, true);
-
- ttm_bo_move_sync_cleanup(bo, dst_mem);
- i915_ttm_adjust_domains_after_move(obj);
- i915_ttm_free_cached_io_st(obj);
-
- if (gpu_binds_iomem(dst_mem) || cpu_maps_iomem(dst_mem)) {
- obj->ttm.cached_io_st = dst_st;
- obj->ttm.get_io_page.sg_pos = dst_st->sgl;
- obj->ttm.get_io_page.sg_idx = 0;
- }
-
- i915_ttm_adjust_gem_after_move(obj);
- return 0;
}
static int i915_ttm_io_mem_reserve(struct ttm_device *bdev, struct ttm_resource *mem)
{
- if (!cpu_maps_iomem(mem))
+ if (!i915_ttm_cpu_maps_iomem(mem))
return 0;
mem->bus.caching = ttm_write_combined;
@@ -607,19 +586,26 @@ static unsigned long i915_ttm_io_mem_pfn(struct ttm_buffer_object *bo,
unsigned long page_offset)
{
struct drm_i915_gem_object *obj = i915_ttm_to_gem(bo);
- unsigned long base = obj->mm.region->iomap.base - obj->mm.region->region.start;
struct scatterlist *sg;
+ unsigned long base;
unsigned int ofs;
+ GEM_BUG_ON(!obj);
GEM_WARN_ON(bo->ttm);
+ base = obj->mm.region->iomap.base - obj->mm.region->region.start;
sg = __i915_gem_object_get_sg(obj, &obj->ttm.get_io_page, page_offset, &ofs, true);
return ((base + sg_dma_address(sg)) >> PAGE_SHIFT) + ofs;
}
+/*
+ * All callbacks need to take care not to downcast a struct ttm_buffer_object
+ * without checking its subclass, since it might be a TTM ghost object.
+ */
static struct ttm_device_funcs i915_ttm_bo_driver = {
.ttm_tt_create = i915_ttm_tt_create,
+ .ttm_tt_populate = i915_ttm_tt_populate,
.ttm_tt_unpopulate = i915_ttm_tt_unpopulate,
.ttm_tt_destroy = i915_ttm_tt_destroy,
.eviction_valuable = i915_ttm_eviction_valuable,
@@ -649,7 +635,6 @@ static int __i915_ttm_get_pages(struct drm_i915_gem_object *obj,
.interruptible = true,
.no_wait_gpu = false,
};
- struct sg_table *st;
int real_num_busy;
int ret;
@@ -676,7 +661,6 @@ static int __i915_ttm_get_pages(struct drm_i915_gem_object *obj,
return i915_ttm_err_to_gem(ret);
}
- i915_ttm_adjust_lru(obj);
if (bo->ttm && !ttm_tt_is_populated(bo->ttm)) {
ret = ttm_tt_populate(bo->bdev, bo->ttm, &ctx);
if (ret)
@@ -687,14 +671,19 @@ static int __i915_ttm_get_pages(struct drm_i915_gem_object *obj,
}
if (!i915_gem_object_has_pages(obj)) {
- /* Object either has a page vector or is an iomem object */
- st = bo->ttm ? i915_ttm_tt_get_st(bo->ttm) : obj->ttm.cached_io_st;
- if (IS_ERR(st))
- return PTR_ERR(st);
+ struct i915_refct_sgt *rsgt =
+ i915_ttm_resource_get_st(obj, bo->resource);
- __i915_gem_object_set_pages(obj, st, i915_sg_dma_sizes(st->sgl));
+ if (IS_ERR(rsgt))
+ return PTR_ERR(rsgt);
+
+ GEM_BUG_ON(obj->mm.rsgt);
+ obj->mm.rsgt = rsgt;
+ __i915_gem_object_set_pages(obj, &rsgt->table,
+ i915_sg_dma_sizes(rsgt->table.sgl));
}
+ i915_ttm_adjust_lru(obj);
return ret;
}
@@ -766,12 +755,21 @@ static void i915_ttm_put_pages(struct drm_i915_gem_object *obj,
* and shrinkers will move it out if needed.
*/
- i915_ttm_adjust_lru(obj);
+ if (obj->mm.rsgt)
+ i915_refct_sgt_put(fetch_and_zero(&obj->mm.rsgt));
}
-static void i915_ttm_adjust_lru(struct drm_i915_gem_object *obj)
+/**
+ * i915_ttm_adjust_lru - Adjust an object's position on relevant LRU lists.
+ * @obj: The object
+ */
+void i915_ttm_adjust_lru(struct drm_i915_gem_object *obj)
{
struct ttm_buffer_object *bo = i915_gem_to_ttm(obj);
+ struct i915_ttm_tt *i915_tt =
+ container_of(bo->ttm, typeof(*i915_tt), ttm);
+ bool shrinkable =
+ bo->ttm && i915_tt->filp && ttm_tt_is_populated(bo->ttm);
/*
* Don't manipulate the TTM LRUs while in TTM bo destruction.
@@ -781,10 +779,53 @@ static void i915_ttm_adjust_lru(struct drm_i915_gem_object *obj)
return;
/*
+ * We skip managing the shrinker LRU in set_pages() and just manage
+ * everything here. This does at least solve the issue with having
+ * temporary shmem mappings(like with evicted lmem) not being visible to
+ * the shrinker. Only our shmem objects are shrinkable, everything else
+ * we keep as unshrinkable.
+ *
+ * To make sure everything plays nice we keep an extra shrink pin in TTM
+ * if the underlying pages are not currently shrinkable. Once we release
+ * our pin, like when the pages are moved to shmem, the pages will then
+ * be added to the shrinker LRU, assuming the caller isn't also holding
+ * a pin.
+ *
+ * TODO: consider maybe also bumping the shrinker list here when we have
+ * already unpinned it, which should give us something more like an LRU.
+ *
+ * TODO: There is a small window of opportunity for this function to
+ * get called from eviction after we've dropped the last GEM refcount,
+ * but before the TTM deleted flag is set on the object. Avoid
+ * adjusting the shrinker list in such cases, since the object is
+ * not available to the shrinker anyway due to its zero refcount.
+ * To fix this properly we should move to a TTM shrinker LRU list for
+ * these objects.
+ */
+ if (kref_get_unless_zero(&obj->base.refcount)) {
+ if (shrinkable != obj->mm.ttm_shrinkable) {
+ if (shrinkable) {
+ if (obj->mm.madv == I915_MADV_WILLNEED)
+ __i915_gem_object_make_shrinkable(obj);
+ else
+ __i915_gem_object_make_purgeable(obj);
+ } else {
+ i915_gem_object_make_unshrinkable(obj);
+ }
+
+ obj->mm.ttm_shrinkable = shrinkable;
+ }
+ i915_gem_object_put(obj);
+ }
+
+ /*
* Put on the correct LRU list depending on the MADV status
*/
spin_lock(&bo->bdev->lru_lock);
- if (obj->mm.madv != I915_MADV_WILLNEED) {
+ if (shrinkable) {
+ /* Try to keep shmem_tt from being considered for shrinking. */
+ bo->priority = TTM_MAX_BO_PRIORITY - 1;
+ } else if (obj->mm.madv != I915_MADV_WILLNEED) {
bo->priority = I915_TTM_PRIO_PURGE;
} else if (!i915_gem_object_has_pages(obj)) {
if (bo->priority < I915_TTM_PRIO_HAS_PAGES)
@@ -823,15 +864,39 @@ static void i915_ttm_delayed_free(struct drm_i915_gem_object *obj)
static vm_fault_t vm_fault_ttm(struct vm_fault *vmf)
{
struct vm_area_struct *area = vmf->vma;
- struct drm_i915_gem_object *obj =
- i915_ttm_to_gem(area->vm_private_data);
+ struct ttm_buffer_object *bo = area->vm_private_data;
+ struct drm_device *dev = bo->base.dev;
+ struct drm_i915_gem_object *obj;
+ vm_fault_t ret;
+ int idx;
+
+ obj = i915_ttm_to_gem(bo);
+ if (!obj)
+ return VM_FAULT_SIGBUS;
/* Sanity check that we allow writing into this object */
if (unlikely(i915_gem_object_is_readonly(obj) &&
area->vm_flags & VM_WRITE))
return VM_FAULT_SIGBUS;
- return ttm_bo_vm_fault(vmf);
+ ret = ttm_bo_vm_reserve(bo, vmf);
+ if (ret)
+ return ret;
+
+ if (drm_dev_enter(dev, &idx)) {
+ ret = ttm_bo_vm_fault_reserved(vmf, vmf->vma->vm_page_prot,
+ TTM_BO_VM_NUM_PREFAULT);
+ drm_dev_exit(idx);
+ } else {
+ ret = ttm_bo_vm_dummy_page(vmf, vmf->vma->vm_page_prot);
+ }
+ if (ret == VM_FAULT_RETRY && !(vmf->flags & FAULT_FLAG_RETRY_NOWAIT))
+ return ret;
+
+ i915_ttm_adjust_lru(obj);
+
+ dma_resv_unlock(bo->base.resv);
+ return ret;
}
static int
@@ -882,13 +947,18 @@ static u64 i915_ttm_mmap_offset(struct drm_i915_gem_object *obj)
static const struct drm_i915_gem_object_ops i915_gem_ttm_obj_ops = {
.name = "i915_gem_object_ttm",
+ .flags = I915_GEM_OBJECT_IS_SHRINKABLE |
+ I915_GEM_OBJECT_SELF_MANAGED_SHRINK_LIST,
.get_pages = i915_ttm_get_pages,
.put_pages = i915_ttm_put_pages,
.truncate = i915_ttm_purge,
+ .shrinker_release_pages = i915_ttm_shrinker_release_pages,
+
.adjust_lru = i915_ttm_adjust_lru,
.delayed_free = i915_ttm_delayed_free,
.migrate = i915_ttm_migrate,
+
.mmap_offset = i915_ttm_mmap_offset,
.mmap_ops = &vm_ops_ttm,
};
@@ -901,6 +971,18 @@ void i915_ttm_bo_destroy(struct ttm_buffer_object *bo)
mutex_destroy(&obj->ttm.get_io_page.lock);
if (obj->ttm.created) {
+ /*
+ * We freely manage the shrinker LRU outide of the mm.pages life
+ * cycle. As a result when destroying the object we should be
+ * extra paranoid and ensure we remove it from the LRU, before
+ * we free the object.
+ *
+ * Touching the ttm_shrinkable outside of the object lock here
+ * should be safe now that the last GEM object ref was dropped.
+ */
+ if (obj->mm.ttm_shrinkable)
+ i915_gem_object_make_unshrinkable(obj);
+
i915_ttm_backup_free(obj);
/* This releases all gem object bindings to the backend. */
@@ -940,10 +1022,9 @@ int __i915_gem_ttm_object_init(struct intel_memory_region *mem,
i915_gem_object_init(obj, &i915_gem_ttm_obj_ops, &lock_class, flags);
/* Don't put on a region list until we're either locked or fully initialized. */
- obj->mm.region = intel_memory_region_get(mem);
+ obj->mm.region = mem;
INIT_LIST_HEAD(&obj->mm.region_link);
- i915_gem_object_make_unshrinkable(obj);
INIT_RADIX_TREE(&obj->ttm.get_io_page.radix, GFP_KERNEL | __GFP_NOWARN);
mutex_init(&obj->ttm.get_io_page.lock);
bo_type = (obj->flags & I915_BO_ALLOC_USER) ? ttm_bo_type_device :
@@ -955,6 +1036,14 @@ int __i915_gem_ttm_object_init(struct intel_memory_region *mem,
GEM_BUG_ON(page_size && obj->mm.n_placements);
/*
+ * Keep an extra shrink pin to prevent the object from being made
+ * shrinkable too early. If the ttm_tt is ever allocated in shmem, we
+ * drop the pin. The TTM backend manages the shrinker LRU itself,
+ * outside of the normal mm.pages life cycle.
+ */
+ i915_gem_object_make_unshrinkable(obj);
+
+ /*
* If this function fails, it will call the destructor, but
* our caller still owns the object. So no freeing in the
* destructor until obj->ttm.created is true.
@@ -980,6 +1069,7 @@ int __i915_gem_ttm_object_init(struct intel_memory_region *mem,
static const struct intel_memory_region_ops ttm_system_region_ops = {
.init_object = __i915_gem_ttm_object_init,
+ .release = intel_region_ttm_fini,
};
struct intel_memory_region *
@@ -999,50 +1089,3 @@ i915_gem_ttm_system_setup(struct drm_i915_private *i915,
intel_memory_region_set_name(mr, "system-ttm");
return mr;
}
-
-/**
- * i915_gem_obj_copy_ttm - Copy the contents of one ttm-based gem object to
- * another
- * @dst: The destination object
- * @src: The source object
- * @allow_accel: Allow using the blitter. Otherwise TTM memcpy is used.
- * @intr: Whether to perform waits interruptible:
- *
- * Note: The caller is responsible for assuring that the underlying
- * TTM objects are populated if needed and locked.
- *
- * Return: Zero on success. Negative error code on error. If @intr == true,
- * then it may return -ERESTARTSYS or -EINTR.
- */
-int i915_gem_obj_copy_ttm(struct drm_i915_gem_object *dst,
- struct drm_i915_gem_object *src,
- bool allow_accel, bool intr)
-{
- struct ttm_buffer_object *dst_bo = i915_gem_to_ttm(dst);
- struct ttm_buffer_object *src_bo = i915_gem_to_ttm(src);
- struct ttm_operation_ctx ctx = {
- .interruptible = intr,
- };
- struct sg_table *dst_st;
- int ret;
-
- assert_object_held(dst);
- assert_object_held(src);
-
- /*
- * Sync for now. This will change with async moves.
- */
- ret = ttm_bo_wait_ctx(dst_bo, &ctx);
- if (!ret)
- ret = ttm_bo_wait_ctx(src_bo, &ctx);
- if (ret)
- return ret;
-
- dst_st = gpu_binds_iomem(dst_bo->resource) ?
- dst->ttm.cached_io_st : i915_ttm_tt_get_st(dst_bo->ttm);
-
- __i915_ttm_move(src_bo, false, dst_bo->resource, dst_bo->ttm,
- dst_st, allow_accel);
-
- return 0;
-}
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_ttm.h b/drivers/gpu/drm/i915/gem/i915_gem_ttm.h
index 0b7291d..9d698ad 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_ttm.h
+++ b/drivers/gpu/drm/i915/gem/i915_gem_ttm.h
@@ -5,6 +5,8 @@
#ifndef _I915_GEM_TTM_H_
#define _I915_GEM_TTM_H_
+#include <drm/ttm/ttm_placement.h>
+
#include "gem/i915_gem_object_types.h"
/**
@@ -35,7 +37,7 @@ void i915_ttm_bo_destroy(struct ttm_buffer_object *bo);
static inline struct drm_i915_gem_object *
i915_ttm_to_gem(struct ttm_buffer_object *bo)
{
- if (GEM_WARN_ON(bo->destroy != i915_ttm_bo_destroy))
+ if (bo->destroy != i915_ttm_bo_destroy)
return NULL;
return container_of(bo, struct drm_i915_gem_object, __do_not_access);
@@ -47,10 +49,6 @@ int __i915_gem_ttm_object_init(struct intel_memory_region *mem,
resource_size_t page_size,
unsigned int flags);
-int i915_gem_obj_copy_ttm(struct drm_i915_gem_object *dst,
- struct drm_i915_gem_object *src,
- bool allow_accel, bool intr);
-
/* Internal I915 TTM declarations and definitions below. */
#define I915_PL_LMEM0 TTM_PL_PRIV
@@ -60,4 +58,37 @@ int i915_gem_obj_copy_ttm(struct drm_i915_gem_object *dst,
struct ttm_placement *i915_ttm_sys_placement(void);
+void i915_ttm_free_cached_io_rsgt(struct drm_i915_gem_object *obj);
+
+struct i915_refct_sgt *
+i915_ttm_resource_get_st(struct drm_i915_gem_object *obj,
+ struct ttm_resource *res);
+
+void i915_ttm_adjust_lru(struct drm_i915_gem_object *obj);
+
+int i915_ttm_purge(struct drm_i915_gem_object *obj);
+
+/**
+ * i915_ttm_gtt_binds_lmem - Should the memory be viewed as LMEM by the GTT?
+ * @mem: struct ttm_resource representing the memory.
+ *
+ * Return: true if memory should be viewed as LMEM for GTT binding purposes,
+ * false otherwise.
+ */
+static inline bool i915_ttm_gtt_binds_lmem(struct ttm_resource *mem)
+{
+ return mem->mem_type != I915_PL_SYSTEM;
+}
+
+/**
+ * i915_ttm_cpu_maps_iomem - Should the memory be viewed as IOMEM by the CPU?
+ * @mem: struct ttm_resource representing the memory.
+ *
+ * Return: true if memory should be viewed as IOMEM for CPU mapping purposes.
+ */
+static inline bool i915_ttm_cpu_maps_iomem(struct ttm_resource *mem)
+{
+ /* Once / if we support GGTT, this is also false for cached ttm_tts */
+ return mem->mem_type != I915_PL_SYSTEM;
+}
#endif
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c b/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c
new file mode 100644
index 0000000..ee9612a
--- /dev/null
+++ b/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c
@@ -0,0 +1,627 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright © 2021 Intel Corporation
+ */
+
+#include <drm/ttm/ttm_bo_driver.h>
+
+#include "i915_deps.h"
+#include "i915_drv.h"
+#include "intel_memory_region.h"
+#include "intel_region_ttm.h"
+
+#include "gem/i915_gem_object.h"
+#include "gem/i915_gem_region.h"
+#include "gem/i915_gem_ttm.h"
+#include "gem/i915_gem_ttm_move.h"
+
+#include "gt/intel_engine_pm.h"
+#include "gt/intel_gt.h"
+#include "gt/intel_migrate.h"
+
+/**
+ * DOC: Selftest failure modes for failsafe migration:
+ *
+ * For fail_gpu_migration, the gpu blit scheduled is always a clear blit
+ * rather than a copy blit, and then we force the failure paths as if
+ * the blit fence returned an error.
+ *
+ * For fail_work_allocation we fail the kmalloc of the async worker, we
+ * sync the gpu blit. If it then fails, or fail_gpu_migration is set to
+ * true, then a memcpy operation is performed sync.
+ */
+#if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
+static bool fail_gpu_migration;
+static bool fail_work_allocation;
+
+void i915_ttm_migrate_set_failure_modes(bool gpu_migration,
+ bool work_allocation)
+{
+ fail_gpu_migration = gpu_migration;
+ fail_work_allocation = work_allocation;
+}
+#endif
+
+static enum i915_cache_level
+i915_ttm_cache_level(struct drm_i915_private *i915, struct ttm_resource *res,
+ struct ttm_tt *ttm)
+{
+ return ((HAS_LLC(i915) || HAS_SNOOP(i915)) &&
+ !i915_ttm_gtt_binds_lmem(res) &&
+ ttm->caching == ttm_cached) ? I915_CACHE_LLC :
+ I915_CACHE_NONE;
+}
+
+static struct intel_memory_region *
+i915_ttm_region(struct ttm_device *bdev, int ttm_mem_type)
+{
+ struct drm_i915_private *i915 = container_of(bdev, typeof(*i915), bdev);
+
+ /* There's some room for optimization here... */
+ GEM_BUG_ON(ttm_mem_type != I915_PL_SYSTEM &&
+ ttm_mem_type < I915_PL_LMEM0);
+ if (ttm_mem_type == I915_PL_SYSTEM)
+ return intel_memory_region_lookup(i915, INTEL_MEMORY_SYSTEM,
+ 0);
+
+ return intel_memory_region_lookup(i915, INTEL_MEMORY_LOCAL,
+ ttm_mem_type - I915_PL_LMEM0);
+}
+
+/**
+ * i915_ttm_adjust_domains_after_move - Adjust the GEM domains after a
+ * TTM move
+ * @obj: The gem object
+ */
+void i915_ttm_adjust_domains_after_move(struct drm_i915_gem_object *obj)
+{
+ struct ttm_buffer_object *bo = i915_gem_to_ttm(obj);
+
+ if (i915_ttm_cpu_maps_iomem(bo->resource) || bo->ttm->caching != ttm_cached) {
+ obj->write_domain = I915_GEM_DOMAIN_WC;
+ obj->read_domains = I915_GEM_DOMAIN_WC;
+ } else {
+ obj->write_domain = I915_GEM_DOMAIN_CPU;
+ obj->read_domains = I915_GEM_DOMAIN_CPU;
+ }
+}
+
+/**
+ * i915_ttm_adjust_gem_after_move - Adjust the GEM state after a TTM move
+ * @obj: The gem object
+ *
+ * Adjusts the GEM object's region, mem_flags and cache coherency after a
+ * TTM move.
+ */
+void i915_ttm_adjust_gem_after_move(struct drm_i915_gem_object *obj)
+{
+ struct ttm_buffer_object *bo = i915_gem_to_ttm(obj);
+ unsigned int cache_level;
+ unsigned int i;
+
+ /*
+ * If object was moved to an allowable region, update the object
+ * region to consider it migrated. Note that if it's currently not
+ * in an allowable region, it's evicted and we don't update the
+ * object region.
+ */
+ if (intel_region_to_ttm_type(obj->mm.region) != bo->resource->mem_type) {
+ for (i = 0; i < obj->mm.n_placements; ++i) {
+ struct intel_memory_region *mr = obj->mm.placements[i];
+
+ if (intel_region_to_ttm_type(mr) == bo->resource->mem_type &&
+ mr != obj->mm.region) {
+ i915_gem_object_release_memory_region(obj);
+ i915_gem_object_init_memory_region(obj, mr);
+ break;
+ }
+ }
+ }
+
+ obj->mem_flags &= ~(I915_BO_FLAG_STRUCT_PAGE | I915_BO_FLAG_IOMEM);
+
+ obj->mem_flags |= i915_ttm_cpu_maps_iomem(bo->resource) ? I915_BO_FLAG_IOMEM :
+ I915_BO_FLAG_STRUCT_PAGE;
+
+ cache_level = i915_ttm_cache_level(to_i915(bo->base.dev), bo->resource,
+ bo->ttm);
+ i915_gem_object_set_cache_coherency(obj, cache_level);
+}
+
+/**
+ * i915_ttm_move_notify - Prepare an object for move
+ * @bo: The ttm buffer object.
+ *
+ * This function prepares an object for move by removing all GPU bindings,
+ * removing all CPU mapings and finally releasing the pages sg-table.
+ *
+ * Return: 0 if successful, negative error code on error.
+ */
+int i915_ttm_move_notify(struct ttm_buffer_object *bo)
+{
+ struct drm_i915_gem_object *obj = i915_ttm_to_gem(bo);
+ int ret;
+
+ ret = i915_gem_object_unbind(obj, I915_GEM_OBJECT_UNBIND_ACTIVE);
+ if (ret)
+ return ret;
+
+ ret = __i915_gem_object_put_pages(obj);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+static struct dma_fence *i915_ttm_accel_move(struct ttm_buffer_object *bo,
+ bool clear,
+ struct ttm_resource *dst_mem,
+ struct ttm_tt *dst_ttm,
+ struct sg_table *dst_st,
+ const struct i915_deps *deps)
+{
+ struct drm_i915_private *i915 = container_of(bo->bdev, typeof(*i915),
+ bdev);
+ struct drm_i915_gem_object *obj = i915_ttm_to_gem(bo);
+ struct i915_request *rq;
+ struct ttm_tt *src_ttm = bo->ttm;
+ enum i915_cache_level src_level, dst_level;
+ int ret;
+
+ if (!to_gt(i915)->migrate.context || intel_gt_is_wedged(to_gt(i915)))
+ return ERR_PTR(-EINVAL);
+
+ /* With fail_gpu_migration, we always perform a GPU clear. */
+ if (I915_SELFTEST_ONLY(fail_gpu_migration))
+ clear = true;
+
+ dst_level = i915_ttm_cache_level(i915, dst_mem, dst_ttm);
+ if (clear) {
+ if (bo->type == ttm_bo_type_kernel &&
+ !I915_SELFTEST_ONLY(fail_gpu_migration))
+ return ERR_PTR(-EINVAL);
+
+ intel_engine_pm_get(to_gt(i915)->migrate.context->engine);
+ ret = intel_context_migrate_clear(to_gt(i915)->migrate.context, deps,
+ dst_st->sgl, dst_level,
+ i915_ttm_gtt_binds_lmem(dst_mem),
+ 0, &rq);
+ } else {
+ struct i915_refct_sgt *src_rsgt =
+ i915_ttm_resource_get_st(obj, bo->resource);
+
+ if (IS_ERR(src_rsgt))
+ return ERR_CAST(src_rsgt);
+
+ src_level = i915_ttm_cache_level(i915, bo->resource, src_ttm);
+ intel_engine_pm_get(to_gt(i915)->migrate.context->engine);
+ ret = intel_context_migrate_copy(to_gt(i915)->migrate.context,
+ deps, src_rsgt->table.sgl,
+ src_level,
+ i915_ttm_gtt_binds_lmem(bo->resource),
+ dst_st->sgl, dst_level,
+ i915_ttm_gtt_binds_lmem(dst_mem),
+ &rq);
+
+ i915_refct_sgt_put(src_rsgt);
+ }
+
+ intel_engine_pm_put(to_gt(i915)->migrate.context->engine);
+
+ if (ret && rq) {
+ i915_request_wait(rq, 0, MAX_SCHEDULE_TIMEOUT);
+ i915_request_put(rq);
+ }
+
+ return ret ? ERR_PTR(ret) : &rq->fence;
+}
+
+/**
+ * struct i915_ttm_memcpy_arg - argument for the bo memcpy functionality.
+ * @_dst_iter: Storage space for the destination kmap iterator.
+ * @_src_iter: Storage space for the source kmap iterator.
+ * @dst_iter: Pointer to the destination kmap iterator.
+ * @src_iter: Pointer to the source kmap iterator.
+ * @clear: Whether to clear instead of copy.
+ * @src_rsgt: Refcounted scatter-gather list of source memory.
+ * @dst_rsgt: Refcounted scatter-gather list of destination memory.
+ */
+struct i915_ttm_memcpy_arg {
+ union {
+ struct ttm_kmap_iter_tt tt;
+ struct ttm_kmap_iter_iomap io;
+ } _dst_iter,
+ _src_iter;
+ struct ttm_kmap_iter *dst_iter;
+ struct ttm_kmap_iter *src_iter;
+ unsigned long num_pages;
+ bool clear;
+ struct i915_refct_sgt *src_rsgt;
+ struct i915_refct_sgt *dst_rsgt;
+};
+
+/**
+ * struct i915_ttm_memcpy_work - Async memcpy worker under a dma-fence.
+ * @fence: The dma-fence.
+ * @work: The work struct use for the memcpy work.
+ * @lock: The fence lock. Not used to protect anything else ATM.
+ * @irq_work: Low latency worker to signal the fence since it can't be done
+ * from the callback for lockdep reasons.
+ * @cb: Callback for the accelerated migration fence.
+ * @arg: The argument for the memcpy functionality.
+ */
+struct i915_ttm_memcpy_work {
+ struct dma_fence fence;
+ struct work_struct work;
+ /* The fence lock */
+ spinlock_t lock;
+ struct irq_work irq_work;
+ struct dma_fence_cb cb;
+ struct i915_ttm_memcpy_arg arg;
+};
+
+static void i915_ttm_move_memcpy(struct i915_ttm_memcpy_arg *arg)
+{
+ ttm_move_memcpy(arg->clear, arg->num_pages,
+ arg->dst_iter, arg->src_iter);
+}
+
+static void i915_ttm_memcpy_init(struct i915_ttm_memcpy_arg *arg,
+ struct ttm_buffer_object *bo, bool clear,
+ struct ttm_resource *dst_mem,
+ struct ttm_tt *dst_ttm,
+ struct i915_refct_sgt *dst_rsgt)
+{
+ struct drm_i915_gem_object *obj = i915_ttm_to_gem(bo);
+ struct intel_memory_region *dst_reg, *src_reg;
+
+ dst_reg = i915_ttm_region(bo->bdev, dst_mem->mem_type);
+ src_reg = i915_ttm_region(bo->bdev, bo->resource->mem_type);
+ GEM_BUG_ON(!dst_reg || !src_reg);
+
+ arg->dst_iter = !i915_ttm_cpu_maps_iomem(dst_mem) ?
+ ttm_kmap_iter_tt_init(&arg->_dst_iter.tt, dst_ttm) :
+ ttm_kmap_iter_iomap_init(&arg->_dst_iter.io, &dst_reg->iomap,
+ &dst_rsgt->table, dst_reg->region.start);
+
+ arg->src_iter = !i915_ttm_cpu_maps_iomem(bo->resource) ?
+ ttm_kmap_iter_tt_init(&arg->_src_iter.tt, bo->ttm) :
+ ttm_kmap_iter_iomap_init(&arg->_src_iter.io, &src_reg->iomap,
+ &obj->ttm.cached_io_rsgt->table,
+ src_reg->region.start);
+ arg->clear = clear;
+ arg->num_pages = bo->base.size >> PAGE_SHIFT;
+
+ arg->dst_rsgt = i915_refct_sgt_get(dst_rsgt);
+ arg->src_rsgt = clear ? NULL :
+ i915_ttm_resource_get_st(obj, bo->resource);
+}
+
+static void i915_ttm_memcpy_release(struct i915_ttm_memcpy_arg *arg)
+{
+ i915_refct_sgt_put(arg->src_rsgt);
+ i915_refct_sgt_put(arg->dst_rsgt);
+}
+
+static void __memcpy_work(struct work_struct *work)
+{
+ struct i915_ttm_memcpy_work *copy_work =
+ container_of(work, typeof(*copy_work), work);
+ struct i915_ttm_memcpy_arg *arg = ©_work->arg;
+ bool cookie = dma_fence_begin_signalling();
+
+ i915_ttm_move_memcpy(arg);
+ dma_fence_end_signalling(cookie);
+
+ dma_fence_signal(©_work->fence);
+
+ i915_ttm_memcpy_release(arg);
+ dma_fence_put(©_work->fence);
+}
+
+static void __memcpy_irq_work(struct irq_work *irq_work)
+{
+ struct i915_ttm_memcpy_work *copy_work =
+ container_of(irq_work, typeof(*copy_work), irq_work);
+ struct i915_ttm_memcpy_arg *arg = ©_work->arg;
+
+ dma_fence_signal(©_work->fence);
+ i915_ttm_memcpy_release(arg);
+ dma_fence_put(©_work->fence);
+}
+
+static void __memcpy_cb(struct dma_fence *fence, struct dma_fence_cb *cb)
+{
+ struct i915_ttm_memcpy_work *copy_work =
+ container_of(cb, typeof(*copy_work), cb);
+
+ if (unlikely(fence->error || I915_SELFTEST_ONLY(fail_gpu_migration))) {
+ INIT_WORK(©_work->work, __memcpy_work);
+ queue_work(system_unbound_wq, ©_work->work);
+ } else {
+ init_irq_work(©_work->irq_work, __memcpy_irq_work);
+ irq_work_queue(©_work->irq_work);
+ }
+}
+
+static const char *get_driver_name(struct dma_fence *fence)
+{
+ return "i915_ttm_memcpy_work";
+}
+
+static const char *get_timeline_name(struct dma_fence *fence)
+{
+ return "unbound";
+}
+
+static const struct dma_fence_ops dma_fence_memcpy_ops = {
+ .get_driver_name = get_driver_name,
+ .get_timeline_name = get_timeline_name,
+};
+
+static struct dma_fence *
+i915_ttm_memcpy_work_arm(struct i915_ttm_memcpy_work *work,
+ struct dma_fence *dep)
+{
+ int ret;
+
+ spin_lock_init(&work->lock);
+ dma_fence_init(&work->fence, &dma_fence_memcpy_ops, &work->lock, 0, 0);
+ dma_fence_get(&work->fence);
+ ret = dma_fence_add_callback(dep, &work->cb, __memcpy_cb);
+ if (ret) {
+ if (ret != -ENOENT)
+ dma_fence_wait(dep, false);
+
+ return ERR_PTR(I915_SELFTEST_ONLY(fail_gpu_migration) ? -EINVAL :
+ dep->error);
+ }
+
+ return &work->fence;
+}
+
+static struct dma_fence *
+__i915_ttm_move(struct ttm_buffer_object *bo,
+ const struct ttm_operation_ctx *ctx, bool clear,
+ struct ttm_resource *dst_mem, struct ttm_tt *dst_ttm,
+ struct i915_refct_sgt *dst_rsgt, bool allow_accel,
+ const struct i915_deps *move_deps)
+{
+ struct i915_ttm_memcpy_work *copy_work = NULL;
+ struct i915_ttm_memcpy_arg _arg, *arg = &_arg;
+ struct dma_fence *fence = ERR_PTR(-EINVAL);
+
+ if (allow_accel) {
+ fence = i915_ttm_accel_move(bo, clear, dst_mem, dst_ttm,
+ &dst_rsgt->table, move_deps);
+
+ /*
+ * We only need to intercept the error when moving to lmem.
+ * When moving to system, TTM or shmem will provide us with
+ * cleared pages.
+ */
+ if (!IS_ERR(fence) && !i915_ttm_gtt_binds_lmem(dst_mem) &&
+ !I915_SELFTEST_ONLY(fail_gpu_migration ||
+ fail_work_allocation))
+ goto out;
+ }
+
+ /* If we've scheduled gpu migration. Try to arm error intercept. */
+ if (!IS_ERR(fence)) {
+ struct dma_fence *dep = fence;
+
+ if (!I915_SELFTEST_ONLY(fail_work_allocation))
+ copy_work = kzalloc(sizeof(*copy_work), GFP_KERNEL);
+
+ if (copy_work) {
+ arg = ©_work->arg;
+ i915_ttm_memcpy_init(arg, bo, clear, dst_mem, dst_ttm,
+ dst_rsgt);
+ fence = i915_ttm_memcpy_work_arm(copy_work, dep);
+ } else {
+ dma_fence_wait(dep, false);
+ fence = ERR_PTR(I915_SELFTEST_ONLY(fail_gpu_migration) ?
+ -EINVAL : fence->error);
+ }
+ dma_fence_put(dep);
+
+ if (!IS_ERR(fence))
+ goto out;
+ } else if (move_deps) {
+ int err = i915_deps_sync(move_deps, ctx);
+
+ if (err)
+ return ERR_PTR(err);
+ }
+
+ /* Error intercept failed or no accelerated migration to start with */
+ if (!copy_work)
+ i915_ttm_memcpy_init(arg, bo, clear, dst_mem, dst_ttm,
+ dst_rsgt);
+ i915_ttm_move_memcpy(arg);
+ i915_ttm_memcpy_release(arg);
+ kfree(copy_work);
+
+ return NULL;
+out:
+ if (!fence && copy_work) {
+ i915_ttm_memcpy_release(arg);
+ kfree(copy_work);
+ }
+
+ return fence;
+}
+
+static int
+prev_deps(struct ttm_buffer_object *bo, struct ttm_operation_ctx *ctx,
+ struct i915_deps *deps)
+{
+ int ret;
+
+ ret = i915_deps_add_dependency(deps, bo->moving, ctx);
+ if (!ret)
+ ret = i915_deps_add_resv(deps, bo->base.resv, ctx);
+
+ return ret;
+}
+
+/**
+ * i915_ttm_move - The TTM move callback used by i915.
+ * @bo: The buffer object.
+ * @evict: Whether this is an eviction.
+ * @dst_mem: The destination ttm resource.
+ * @hop: If we need multihop, what temporary memory type to move to.
+ *
+ * Return: 0 if successful, negative error code otherwise.
+ */
+int i915_ttm_move(struct ttm_buffer_object *bo, bool evict,
+ struct ttm_operation_ctx *ctx,
+ struct ttm_resource *dst_mem,
+ struct ttm_place *hop)
+{
+ struct drm_i915_gem_object *obj = i915_ttm_to_gem(bo);
+ struct ttm_resource_manager *dst_man =
+ ttm_manager_type(bo->bdev, dst_mem->mem_type);
+ struct dma_fence *migration_fence = NULL;
+ struct ttm_tt *ttm = bo->ttm;
+ struct i915_refct_sgt *dst_rsgt;
+ bool clear;
+ int ret;
+
+ if (GEM_WARN_ON(!obj)) {
+ ttm_bo_move_null(bo, dst_mem);
+ return 0;
+ }
+
+ ret = i915_ttm_move_notify(bo);
+ if (ret)
+ return ret;
+
+ if (obj->mm.madv != I915_MADV_WILLNEED) {
+ i915_ttm_purge(obj);
+ ttm_resource_free(bo, &dst_mem);
+ return 0;
+ }
+
+ /* Populate ttm with pages if needed. Typically system memory. */
+ if (ttm && (dst_man->use_tt || (ttm->page_flags & TTM_TT_FLAG_SWAPPED))) {
+ ret = ttm_tt_populate(bo->bdev, ttm, ctx);
+ if (ret)
+ return ret;
+ }
+
+ dst_rsgt = i915_ttm_resource_get_st(obj, dst_mem);
+ if (IS_ERR(dst_rsgt))
+ return PTR_ERR(dst_rsgt);
+
+ clear = !i915_ttm_cpu_maps_iomem(bo->resource) && (!ttm || !ttm_tt_is_populated(ttm));
+ if (!(clear && ttm && !(ttm->page_flags & TTM_TT_FLAG_ZERO_ALLOC))) {
+ struct i915_deps deps;
+
+ i915_deps_init(&deps, GFP_KERNEL | __GFP_NORETRY | __GFP_NOWARN);
+ ret = prev_deps(bo, ctx, &deps);
+ if (ret) {
+ i915_refct_sgt_put(dst_rsgt);
+ return ret;
+ }
+
+ migration_fence = __i915_ttm_move(bo, ctx, clear, dst_mem, bo->ttm,
+ dst_rsgt, true, &deps);
+ i915_deps_fini(&deps);
+ }
+
+ /* We can possibly get an -ERESTARTSYS here */
+ if (IS_ERR(migration_fence)) {
+ i915_refct_sgt_put(dst_rsgt);
+ return PTR_ERR(migration_fence);
+ }
+
+ if (migration_fence) {
+ ret = ttm_bo_move_accel_cleanup(bo, migration_fence, evict,
+ true, dst_mem);
+ if (ret) {
+ dma_fence_wait(migration_fence, false);
+ ttm_bo_move_sync_cleanup(bo, dst_mem);
+ }
+ dma_fence_put(migration_fence);
+ } else {
+ ttm_bo_move_sync_cleanup(bo, dst_mem);
+ }
+
+ i915_ttm_adjust_domains_after_move(obj);
+ i915_ttm_free_cached_io_rsgt(obj);
+
+ if (i915_ttm_gtt_binds_lmem(dst_mem) || i915_ttm_cpu_maps_iomem(dst_mem)) {
+ obj->ttm.cached_io_rsgt = dst_rsgt;
+ obj->ttm.get_io_page.sg_pos = dst_rsgt->table.sgl;
+ obj->ttm.get_io_page.sg_idx = 0;
+ } else {
+ i915_refct_sgt_put(dst_rsgt);
+ }
+
+ i915_ttm_adjust_lru(obj);
+ i915_ttm_adjust_gem_after_move(obj);
+ return 0;
+}
+
+/**
+ * i915_gem_obj_copy_ttm - Copy the contents of one ttm-based gem object to
+ * another
+ * @dst: The destination object
+ * @src: The source object
+ * @allow_accel: Allow using the blitter. Otherwise TTM memcpy is used.
+ * @intr: Whether to perform waits interruptible:
+ *
+ * Note: The caller is responsible for assuring that the underlying
+ * TTM objects are populated if needed and locked.
+ *
+ * Return: Zero on success. Negative error code on error. If @intr == true,
+ * then it may return -ERESTARTSYS or -EINTR.
+ */
+int i915_gem_obj_copy_ttm(struct drm_i915_gem_object *dst,
+ struct drm_i915_gem_object *src,
+ bool allow_accel, bool intr)
+{
+ struct ttm_buffer_object *dst_bo = i915_gem_to_ttm(dst);
+ struct ttm_buffer_object *src_bo = i915_gem_to_ttm(src);
+ struct ttm_operation_ctx ctx = {
+ .interruptible = intr,
+ };
+ struct i915_refct_sgt *dst_rsgt;
+ struct dma_fence *copy_fence;
+ struct i915_deps deps;
+ int ret;
+
+ assert_object_held(dst);
+ assert_object_held(src);
+ i915_deps_init(&deps, GFP_KERNEL | __GFP_NORETRY | __GFP_NOWARN);
+
+ ret = dma_resv_reserve_shared(src_bo->base.resv, 1);
+ if (ret)
+ return ret;
+
+ ret = i915_deps_add_resv(&deps, dst_bo->base.resv, &ctx);
+ if (ret)
+ return ret;
+
+ ret = i915_deps_add_resv(&deps, src_bo->base.resv, &ctx);
+ if (ret)
+ return ret;
+
+ dst_rsgt = i915_ttm_resource_get_st(dst, dst_bo->resource);
+ copy_fence = __i915_ttm_move(src_bo, &ctx, false, dst_bo->resource,
+ dst_bo->ttm, dst_rsgt, allow_accel,
+ &deps);
+
+ i915_deps_fini(&deps);
+ i915_refct_sgt_put(dst_rsgt);
+ if (IS_ERR_OR_NULL(copy_fence))
+ return PTR_ERR_OR_ZERO(copy_fence);
+
+ dma_resv_add_excl_fence(dst_bo->base.resv, copy_fence);
+ dma_resv_add_shared_fence(src_bo->base.resv, copy_fence);
+
+ dma_fence_put(copy_fence);
+
+ return 0;
+}
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.h b/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.h
new file mode 100644
index 0000000..d2e7f14
--- /dev/null
+++ b/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.h
@@ -0,0 +1,41 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2021 Intel Corporation
+ */
+#ifndef _I915_GEM_TTM_MOVE_H_
+#define _I915_GEM_TTM_MOVE_H_
+
+#include <linux/types.h>
+
+#include "i915_selftest.h"
+
+struct ttm_buffer_object;
+struct ttm_operation_ctx;
+struct ttm_place;
+struct ttm_resource;
+struct ttm_tt;
+
+struct drm_i915_gem_object;
+struct i915_refct_sgt;
+
+int i915_ttm_move_notify(struct ttm_buffer_object *bo);
+
+I915_SELFTEST_DECLARE(void i915_ttm_migrate_set_failure_modes(bool gpu_migration,
+ bool work_allocation));
+
+int i915_gem_obj_copy_ttm(struct drm_i915_gem_object *dst,
+ struct drm_i915_gem_object *src,
+ bool allow_accel, bool intr);
+
+/* Internal I915 TTM declarations and definitions below. */
+
+int i915_ttm_move(struct ttm_buffer_object *bo, bool evict,
+ struct ttm_operation_ctx *ctx,
+ struct ttm_resource *dst_mem,
+ struct ttm_place *hop);
+
+void i915_ttm_adjust_domains_after_move(struct drm_i915_gem_object *obj);
+
+void i915_ttm_adjust_gem_after_move(struct drm_i915_gem_object *obj);
+
+#endif
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_ttm_pm.c b/drivers/gpu/drm/i915/gem/i915_gem_ttm_pm.c
index 3b6d14b..9aad8405 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_ttm_pm.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_ttm_pm.c
@@ -12,6 +12,7 @@
#include "gem/i915_gem_region.h"
#include "gem/i915_gem_ttm.h"
+#include "gem/i915_gem_ttm_move.h"
#include "gem/i915_gem_ttm_pm.h"
/**
@@ -79,6 +80,7 @@ static int i915_ttm_backup(struct i915_gem_apply_to_region *apply,
err = i915_gem_obj_copy_ttm(backup, obj, pm_apply->allow_gpu, false);
GEM_WARN_ON(err);
+ ttm_bo_wait_ctx(backup_bo, &ctx);
obj->ttm.backup = backup;
return 0;
@@ -169,6 +171,7 @@ static int i915_ttm_restore(struct i915_gem_apply_to_region *apply,
err = i915_gem_obj_copy_ttm(obj, backup, pm_apply->allow_gpu,
false);
GEM_WARN_ON(err);
+ ttm_bo_wait_ctx(backup_bo, &ctx);
obj->ttm.backup = NULL;
err = 0;
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_userptr.c b/drivers/gpu/drm/i915/gem/i915_gem_userptr.c
index 3173c9f..3cc01c3 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_userptr.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_userptr.c
@@ -529,7 +529,7 @@ i915_gem_userptr_ioctl(struct drm_device *dev,
* On almost all of the older hw, we cannot tell the GPU that
* a page is readonly.
*/
- if (!dev_priv->gt.vm->has_read_only)
+ if (!to_gt(dev_priv)->vm->has_read_only)
return -ENODEV;
}
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_wait.c b/drivers/gpu/drm/i915/gem/i915_gem_wait.c
index f909aaa..dab3d30c0 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_wait.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_wait.c
@@ -10,7 +10,6 @@
#include "gt/intel_engine.h"
-#include "dma_resv_utils.h"
#include "i915_gem_ioctls.h"
#include "i915_gem_object.h"
@@ -25,7 +24,7 @@ i915_gem_object_wait_fence(struct dma_fence *fence,
return timeout;
if (dma_fence_is_i915(fence))
- return i915_request_wait(to_request(fence), flags, timeout);
+ return i915_request_wait_timeout(to_request(fence), flags, timeout);
return dma_fence_wait_timeout(fence,
flags & I915_WAIT_INTERRUPTIBLE,
@@ -37,58 +36,22 @@ i915_gem_object_wait_reservation(struct dma_resv *resv,
unsigned int flags,
long timeout)
{
- struct dma_fence *excl;
- bool prune_fences = false;
+ struct dma_resv_iter cursor;
+ struct dma_fence *fence;
+ long ret = timeout ?: 1;
- if (flags & I915_WAIT_ALL) {
- struct dma_fence **shared;
- unsigned int count, i;
- int ret;
+ dma_resv_iter_begin(&cursor, resv, flags & I915_WAIT_ALL);
+ dma_resv_for_each_fence_unlocked(&cursor, fence) {
+ ret = i915_gem_object_wait_fence(fence, flags, timeout);
+ if (ret <= 0)
+ break;
- ret = dma_resv_get_fences(resv, &excl, &count, &shared);
- if (ret)
- return ret;
-
- for (i = 0; i < count; i++) {
- timeout = i915_gem_object_wait_fence(shared[i],
- flags, timeout);
- if (timeout < 0)
- break;
-
- dma_fence_put(shared[i]);
- }
-
- for (; i < count; i++)
- dma_fence_put(shared[i]);
- kfree(shared);
-
- /*
- * If both shared fences and an exclusive fence exist,
- * then by construction the shared fences must be later
- * than the exclusive fence. If we successfully wait for
- * all the shared fences, we know that the exclusive fence
- * must all be signaled. If all the shared fences are
- * signaled, we can prune the array and recover the
- * floating references on the fences/requests.
- */
- prune_fences = count && timeout >= 0;
- } else {
- excl = dma_resv_get_excl_unlocked(resv);
+ if (timeout)
+ timeout = ret;
}
+ dma_resv_iter_end(&cursor);
- if (excl && timeout >= 0)
- timeout = i915_gem_object_wait_fence(excl, flags, timeout);
-
- dma_fence_put(excl);
-
- /*
- * Opportunistically prune the fences iff we know they have *all* been
- * signaled.
- */
- if (prune_fences)
- dma_resv_prune(resv);
-
- return timeout;
+ return ret;
}
static void fence_set_priority(struct dma_fence *fence,
@@ -151,32 +114,13 @@ i915_gem_object_wait_priority(struct drm_i915_gem_object *obj,
unsigned int flags,
const struct i915_sched_attr *attr)
{
- struct dma_fence *excl;
+ struct dma_resv_iter cursor;
+ struct dma_fence *fence;
- if (flags & I915_WAIT_ALL) {
- struct dma_fence **shared;
- unsigned int count, i;
- int ret;
-
- ret = dma_resv_get_fences(obj->base.resv, &excl, &count,
- &shared);
- if (ret)
- return ret;
-
- for (i = 0; i < count; i++) {
- i915_gem_fence_wait_priority(shared[i], attr);
- dma_fence_put(shared[i]);
- }
-
- kfree(shared);
- } else {
- excl = dma_resv_get_excl_unlocked(obj->base.resv);
- }
-
- if (excl) {
- i915_gem_fence_wait_priority(excl, attr);
- dma_fence_put(excl);
- }
+ dma_resv_iter_begin(&cursor, obj->base.resv, flags & I915_WAIT_ALL);
+ dma_resv_for_each_fence_unlocked(&cursor, fence)
+ i915_gem_fence_wait_priority(fence, attr);
+ dma_resv_iter_end(&cursor);
return 0;
}
@@ -196,7 +140,11 @@ i915_gem_object_wait(struct drm_i915_gem_object *obj,
timeout = i915_gem_object_wait_reservation(obj->base.resv,
flags, timeout);
- return timeout < 0 ? timeout : 0;
+
+ if (timeout < 0)
+ return timeout;
+
+ return !timeout ? -ETIME : 0;
}
static inline unsigned long nsecs_to_jiffies_timeout(const u64 n)
@@ -306,6 +254,6 @@ int i915_gem_object_wait_migration(struct drm_i915_gem_object *obj,
unsigned int flags)
{
might_sleep();
- /* NOP for now. */
- return 0;
+
+ return i915_gem_object_wait_moving_fence(obj, !!(flags & I915_WAIT_INTERRUPTIBLE));
}
diff --git a/drivers/gpu/drm/i915/gem/i915_gemfs.c b/drivers/gpu/drm/i915/gem/i915_gemfs.c
index dbdbdc3..7271fbf 100644
--- a/drivers/gpu/drm/i915/gem/i915_gemfs.c
+++ b/drivers/gpu/drm/i915/gem/i915_gemfs.c
@@ -12,6 +12,7 @@
int i915_gemfs_init(struct drm_i915_private *i915)
{
+ char huge_opt[] = "huge=within_size"; /* r/w */
struct file_system_type *type;
struct vfsmount *gemfs;
char *opts;
@@ -31,10 +32,8 @@ int i915_gemfs_init(struct drm_i915_private *i915)
*/
opts = NULL;
- if (intel_vtd_active()) {
+ if (intel_vtd_active(i915)) {
if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE)) {
- static char huge_opt[] = "huge=within_size"; /* r/w */
-
opts = huge_opt;
drm_info(&i915->drm,
"Transparent Hugepage mode '%s'\n",
diff --git a/drivers/gpu/drm/i915/gem/selftests/huge_pages.c b/drivers/gpu/drm/i915/gem/selftests/huge_pages.c
index b200313..11f0aa6 100644
--- a/drivers/gpu/drm/i915/gem/selftests/huge_pages.c
+++ b/drivers/gpu/drm/i915/gem/selftests/huge_pages.c
@@ -22,6 +22,22 @@
#include "selftests/mock_region.h"
#include "selftests/i915_random.h"
+static struct i915_gem_context *hugepage_ctx(struct drm_i915_private *i915,
+ struct file *file)
+{
+ struct i915_gem_context *ctx = live_context(i915, file);
+ struct i915_address_space *vm;
+
+ if (IS_ERR(ctx))
+ return ctx;
+
+ vm = ctx->vm;
+ if (vm)
+ WRITE_ONCE(vm->scrub_64K, true);
+
+ return ctx;
+}
+
static const unsigned int page_sizes[] = {
I915_GTT_PAGE_SIZE_2M,
I915_GTT_PAGE_SIZE_64K,
@@ -552,7 +568,7 @@ static int igt_mock_memory_region_huge_pages(void *arg)
out_put:
i915_gem_object_put(obj);
out_region:
- intel_memory_region_put(mem);
+ intel_memory_region_destroy(mem);
return err;
}
@@ -959,6 +975,8 @@ static int igt_mock_ppgtt_64K(void *arg)
__i915_gem_object_put_pages(obj);
i915_gem_object_unlock(obj);
i915_gem_object_put(obj);
+
+ i915_gem_drain_freed_objects(i915);
}
}
@@ -1080,10 +1098,6 @@ static int __igt_write_huge(struct intel_context *ce,
if (IS_ERR(vma))
return PTR_ERR(vma);
- err = i915_vma_unbind(vma);
- if (err)
- return err;
-
err = i915_vma_pin(vma, size, 0, flags | offset);
if (err) {
/*
@@ -1117,7 +1131,7 @@ static int __igt_write_huge(struct intel_context *ce,
return err;
}
-static int igt_write_huge(struct i915_gem_context *ctx,
+static int igt_write_huge(struct drm_i915_private *i915,
struct drm_i915_gem_object *obj)
{
struct i915_gem_engines *engines;
@@ -1127,6 +1141,8 @@ static int igt_write_huge(struct i915_gem_context *ctx,
IGT_TIMEOUT(end_time);
unsigned int max_page_size;
unsigned int count;
+ struct i915_gem_context *ctx;
+ struct file *file;
u64 max;
u64 num;
u64 size;
@@ -1134,6 +1150,16 @@ static int igt_write_huge(struct i915_gem_context *ctx,
int i, n;
int err = 0;
+ file = mock_file(i915);
+ if (IS_ERR(file))
+ return PTR_ERR(file);
+
+ ctx = hugepage_ctx(i915, file);
+ if (IS_ERR(ctx)) {
+ err = PTR_ERR(ctx);
+ goto out;
+ }
+
GEM_BUG_ON(!i915_gem_object_has_pinned_pages(obj));
size = obj->base.size;
@@ -1153,7 +1179,7 @@ static int igt_write_huge(struct i915_gem_context *ctx,
}
i915_gem_context_unlock_engines(ctx);
if (!n)
- return 0;
+ goto out;
/*
* To keep things interesting when alternating between engines in our
@@ -1215,6 +1241,8 @@ static int igt_write_huge(struct i915_gem_context *ctx,
kfree(order);
+out:
+ fput(file);
return err;
}
@@ -1277,8 +1305,7 @@ static u32 igt_random_size(struct rnd_state *prng,
static int igt_ppgtt_smoke_huge(void *arg)
{
- struct i915_gem_context *ctx = arg;
- struct drm_i915_private *i915 = ctx->i915;
+ struct drm_i915_private *i915 = arg;
struct drm_i915_gem_object *obj;
I915_RND_STATE(prng);
struct {
@@ -1302,6 +1329,7 @@ static int igt_ppgtt_smoke_huge(void *arg)
u32 min = backends[i].min;
u32 max = backends[i].max;
u32 size = max;
+
try_again:
size = igt_random_size(&prng, min, rounddown_pow_of_two(size));
@@ -1336,7 +1364,7 @@ static int igt_ppgtt_smoke_huge(void *arg)
goto out_unpin;
}
- err = igt_write_huge(ctx, obj);
+ err = igt_write_huge(i915, obj);
if (err) {
pr_err("%s write-huge failed with size=%u, i=%d\n",
__func__, size, i);
@@ -1363,8 +1391,7 @@ static int igt_ppgtt_smoke_huge(void *arg)
static int igt_ppgtt_sanity_check(void *arg)
{
- struct i915_gem_context *ctx = arg;
- struct drm_i915_private *i915 = ctx->i915;
+ struct drm_i915_private *i915 = arg;
unsigned int supported = INTEL_INFO(i915)->page_sizes;
struct {
igt_create_fn fn;
@@ -1431,7 +1458,7 @@ static int igt_ppgtt_sanity_check(void *arg)
if (pages)
obj->mm.page_sizes.sg = pages;
- err = igt_write_huge(ctx, obj);
+ err = igt_write_huge(i915, obj);
i915_gem_object_lock(obj, NULL);
i915_gem_object_unpin_pages(obj);
@@ -1458,15 +1485,27 @@ static int igt_ppgtt_sanity_check(void *arg)
static int igt_tmpfs_fallback(void *arg)
{
- struct i915_gem_context *ctx = arg;
- struct drm_i915_private *i915 = ctx->i915;
+ struct drm_i915_private *i915 = arg;
+ struct i915_address_space *vm;
+ struct i915_gem_context *ctx;
struct vfsmount *gemfs = i915->mm.gemfs;
- struct i915_address_space *vm = i915_gem_context_get_eb_vm(ctx);
struct drm_i915_gem_object *obj;
struct i915_vma *vma;
+ struct file *file;
u32 *vaddr;
int err = 0;
+ file = mock_file(i915);
+ if (IS_ERR(file))
+ return PTR_ERR(file);
+
+ ctx = hugepage_ctx(i915, file);
+ if (IS_ERR(ctx)) {
+ err = PTR_ERR(ctx);
+ goto out;
+ }
+ vm = i915_gem_context_get_eb_vm(ctx);
+
/*
* Make sure that we don't burst into a ball of flames upon falling back
* to tmpfs, which we rely on if on the off-chance we encouter a failure
@@ -1510,33 +1549,47 @@ static int igt_tmpfs_fallback(void *arg)
i915->mm.gemfs = gemfs;
i915_vm_put(vm);
+out:
+ fput(file);
return err;
}
static int igt_shrink_thp(void *arg)
{
- struct i915_gem_context *ctx = arg;
- struct drm_i915_private *i915 = ctx->i915;
- struct i915_address_space *vm = i915_gem_context_get_eb_vm(ctx);
+ struct drm_i915_private *i915 = arg;
+ struct i915_address_space *vm;
+ struct i915_gem_context *ctx;
struct drm_i915_gem_object *obj;
struct i915_gem_engines_iter it;
struct intel_context *ce;
struct i915_vma *vma;
+ struct file *file;
unsigned int flags = PIN_USER;
unsigned int n;
bool should_swap;
- int err = 0;
+ int err;
+
+ if (!igt_can_allocate_thp(i915)) {
+ pr_info("missing THP support, skipping\n");
+ return 0;
+ }
+
+ file = mock_file(i915);
+ if (IS_ERR(file))
+ return PTR_ERR(file);
+
+ ctx = hugepage_ctx(i915, file);
+ if (IS_ERR(ctx)) {
+ err = PTR_ERR(ctx);
+ goto out;
+ }
+ vm = i915_gem_context_get_eb_vm(ctx);
/*
* Sanity check shrinking huge-paged object -- make sure nothing blows
* up.
*/
- if (!igt_can_allocate_thp(i915)) {
- pr_info("missing THP support, skipping\n");
- goto out_vm;
- }
-
obj = i915_gem_object_create_shmem(i915, SZ_2M);
if (IS_ERR(obj)) {
err = PTR_ERR(obj);
@@ -1626,7 +1679,8 @@ static int igt_shrink_thp(void *arg)
i915_gem_object_put(obj);
out_vm:
i915_vm_put(vm);
-
+out:
+ fput(file);
return err;
}
@@ -1651,7 +1705,7 @@ int i915_gem_huge_page_mock_selftests(void)
mkwrite_device_info(dev_priv)->ppgtt_type = INTEL_PPGTT_FULL;
mkwrite_device_info(dev_priv)->ppgtt_size = 48;
- ppgtt = i915_ppgtt_create(&dev_priv->gt, 0);
+ ppgtt = i915_ppgtt_create(to_gt(dev_priv), 0);
if (IS_ERR(ppgtt)) {
err = PTR_ERR(ppgtt);
goto out_unlock;
@@ -1687,36 +1741,14 @@ int i915_gem_huge_page_live_selftests(struct drm_i915_private *i915)
SUBTEST(igt_ppgtt_smoke_huge),
SUBTEST(igt_ppgtt_sanity_check),
};
- struct i915_gem_context *ctx;
- struct i915_address_space *vm;
- struct file *file;
- int err;
if (!HAS_PPGTT(i915)) {
pr_info("PPGTT not supported, skipping live-selftests\n");
return 0;
}
- if (intel_gt_is_wedged(&i915->gt))
+ if (intel_gt_is_wedged(to_gt(i915)))
return 0;
- file = mock_file(i915);
- if (IS_ERR(file))
- return PTR_ERR(file);
-
- ctx = live_context(i915, file);
- if (IS_ERR(ctx)) {
- err = PTR_ERR(ctx);
- goto out_file;
- }
-
- vm = ctx->vm;
- if (vm)
- WRITE_ONCE(vm->scrub_64K, true);
-
- err = i915_subtests(tests, ctx);
-
-out_file:
- fput(file);
- return err;
+ return i915_live_subtests(tests, i915);
}
diff --git a/drivers/gpu/drm/i915/gem/selftests/i915_gem_client_blt.c b/drivers/gpu/drm/i915/gem/selftests/i915_gem_client_blt.c
index 8402ed9..75947e9 100644
--- a/drivers/gpu/drm/i915/gem/selftests/i915_gem_client_blt.c
+++ b/drivers/gpu/drm/i915/gem/selftests/i915_gem_client_blt.c
@@ -592,7 +592,7 @@ int i915_gem_client_blt_live_selftests(struct drm_i915_private *i915)
SUBTEST(igt_client_tiled_blits),
};
- if (intel_gt_is_wedged(&i915->gt))
+ if (intel_gt_is_wedged(to_gt(i915)))
return 0;
return i915_live_subtests(tests, i915);
diff --git a/drivers/gpu/drm/i915/gem/selftests/i915_gem_context.c b/drivers/gpu/drm/i915/gem/selftests/i915_gem_context.c
index b32f7fe..3f41fe5 100644
--- a/drivers/gpu/drm/i915/gem/selftests/i915_gem_context.c
+++ b/drivers/gpu/drm/i915/gem/selftests/i915_gem_context.c
@@ -88,9 +88,9 @@ static int live_nop_switch(void *arg)
rq = i915_request_get(this);
i915_request_add(this);
}
- if (i915_request_wait(rq, 0, HZ) < 0) {
+ if (i915_request_wait(rq, 0, 10 * HZ) < 0) {
pr_err("Failed to populated %d contexts\n", nctx);
- intel_gt_set_wedged(&i915->gt);
+ intel_gt_set_wedged(to_gt(i915));
i915_request_put(rq);
err = -EIO;
goto out_file;
@@ -146,7 +146,7 @@ static int live_nop_switch(void *arg)
if (i915_request_wait(rq, 0, HZ / 5) < 0) {
pr_err("Switching between %ld contexts timed out\n",
prime);
- intel_gt_set_wedged(&i915->gt);
+ intel_gt_set_wedged(to_gt(i915));
i915_request_put(rq);
break;
}
@@ -1223,7 +1223,7 @@ __igt_ctx_sseu(struct drm_i915_private *i915,
return 0;
if (flags & TEST_RESET)
- igt_global_reset_lock(&i915->gt);
+ igt_global_reset_lock(to_gt(i915));
obj = i915_gem_object_create_internal(i915, PAGE_SIZE);
if (IS_ERR(obj)) {
@@ -1306,7 +1306,7 @@ __igt_ctx_sseu(struct drm_i915_private *i915,
out_unlock:
if (flags & TEST_RESET)
- igt_global_reset_unlock(&i915->gt);
+ igt_global_reset_unlock(to_gt(i915));
if (ret)
pr_err("%s: Failed with %d!\n", name, ret);
@@ -1481,10 +1481,10 @@ static int check_scratch(struct i915_address_space *vm, u64 offset)
static int write_to_scratch(struct i915_gem_context *ctx,
struct intel_engine_cs *engine,
+ struct drm_i915_gem_object *obj,
u64 offset, u32 value)
{
struct drm_i915_private *i915 = ctx->i915;
- struct drm_i915_gem_object *obj;
struct i915_address_space *vm;
struct i915_request *rq;
struct i915_vma *vma;
@@ -1497,15 +1497,9 @@ static int write_to_scratch(struct i915_gem_context *ctx,
if (err)
return err;
- obj = i915_gem_object_create_internal(i915, PAGE_SIZE);
- if (IS_ERR(obj))
- return PTR_ERR(obj);
-
cmd = i915_gem_object_pin_map_unlocked(obj, I915_MAP_WB);
- if (IS_ERR(cmd)) {
- err = PTR_ERR(cmd);
- goto out;
- }
+ if (IS_ERR(cmd))
+ return PTR_ERR(cmd);
*cmd++ = MI_STORE_DWORD_IMM_GEN4;
if (GRAPHICS_VER(i915) >= 8) {
@@ -1569,17 +1563,19 @@ static int write_to_scratch(struct i915_gem_context *ctx,
i915_vma_unpin(vma);
out_vm:
i915_vm_put(vm);
-out:
- i915_gem_object_put(obj);
+
+ if (!err)
+ err = i915_gem_object_wait(obj, 0, MAX_SCHEDULE_TIMEOUT);
+
return err;
}
static int read_from_scratch(struct i915_gem_context *ctx,
struct intel_engine_cs *engine,
+ struct drm_i915_gem_object *obj,
u64 offset, u32 *value)
{
struct drm_i915_private *i915 = ctx->i915;
- struct drm_i915_gem_object *obj;
struct i915_address_space *vm;
const u32 result = 0x100;
struct i915_request *rq;
@@ -1594,10 +1590,6 @@ static int read_from_scratch(struct i915_gem_context *ctx,
if (err)
return err;
- obj = i915_gem_object_create_internal(i915, PAGE_SIZE);
- if (IS_ERR(obj))
- return PTR_ERR(obj);
-
if (GRAPHICS_VER(i915) >= 8) {
const u32 GPR0 = engine->mmio_base + 0x600;
@@ -1615,7 +1607,7 @@ static int read_from_scratch(struct i915_gem_context *ctx,
cmd = i915_gem_object_pin_map_unlocked(obj, I915_MAP_WB);
if (IS_ERR(cmd)) {
err = PTR_ERR(cmd);
- goto out;
+ goto err_unpin;
}
memset(cmd, POISON_INUSE, PAGE_SIZE);
@@ -1651,7 +1643,7 @@ static int read_from_scratch(struct i915_gem_context *ctx,
cmd = i915_gem_object_pin_map_unlocked(obj, I915_MAP_WB);
if (IS_ERR(cmd)) {
err = PTR_ERR(cmd);
- goto out;
+ goto err_unpin;
}
memset(cmd, POISON_INUSE, PAGE_SIZE);
@@ -1722,8 +1714,10 @@ static int read_from_scratch(struct i915_gem_context *ctx,
i915_vma_unpin(vma);
out_vm:
i915_vm_put(vm);
-out:
- i915_gem_object_put(obj);
+
+ if (!err)
+ err = i915_gem_object_wait(obj, 0, MAX_SCHEDULE_TIMEOUT);
+
return err;
}
@@ -1757,6 +1751,7 @@ static int igt_vm_isolation(void *arg)
{
struct drm_i915_private *i915 = arg;
struct i915_gem_context *ctx_a, *ctx_b;
+ struct drm_i915_gem_object *obj_a, *obj_b;
unsigned long num_engines, count;
struct intel_engine_cs *engine;
struct igt_live_test t;
@@ -1810,6 +1805,18 @@ static int igt_vm_isolation(void *arg)
vm_total = ctx_a->vm->total;
GEM_BUG_ON(ctx_b->vm->total != vm_total);
+ obj_a = i915_gem_object_create_internal(i915, PAGE_SIZE);
+ if (IS_ERR(obj_a)) {
+ err = PTR_ERR(obj_a);
+ goto out_file;
+ }
+
+ obj_b = i915_gem_object_create_internal(i915, PAGE_SIZE);
+ if (IS_ERR(obj_b)) {
+ err = PTR_ERR(obj_b);
+ goto put_a;
+ }
+
count = 0;
num_engines = 0;
for_each_uabi_engine(engine, i915) {
@@ -1832,13 +1839,13 @@ static int igt_vm_isolation(void *arg)
I915_GTT_PAGE_SIZE, vm_total,
sizeof(u32), alignof_dword);
- err = write_to_scratch(ctx_a, engine,
+ err = write_to_scratch(ctx_a, engine, obj_a,
offset, 0xdeadbeef);
if (err == 0)
- err = read_from_scratch(ctx_b, engine,
+ err = read_from_scratch(ctx_b, engine, obj_b,
offset, &value);
if (err)
- goto out_file;
+ goto put_b;
if (value != expected) {
pr_err("%s: Read %08x from scratch (offset 0x%08x_%08x), after %lu reads!\n",
@@ -1847,7 +1854,7 @@ static int igt_vm_isolation(void *arg)
lower_32_bits(offset),
this);
err = -EINVAL;
- goto out_file;
+ goto put_b;
}
this++;
@@ -1858,6 +1865,10 @@ static int igt_vm_isolation(void *arg)
pr_info("Checked %lu scratch offsets across %lu engines\n",
count, num_engines);
+put_b:
+ i915_gem_object_put(obj_b);
+put_a:
+ i915_gem_object_put(obj_a);
out_file:
if (igt_live_test_end(&t))
err = -EIO;
@@ -1877,7 +1888,7 @@ int i915_gem_context_live_selftests(struct drm_i915_private *i915)
SUBTEST(igt_vm_isolation),
};
- if (intel_gt_is_wedged(&i915->gt))
+ if (intel_gt_is_wedged(to_gt(i915)))
return 0;
return i915_live_subtests(tests, i915);
diff --git a/drivers/gpu/drm/i915/gem/selftests/i915_gem_dmabuf.c b/drivers/gpu/drm/i915/gem/selftests/i915_gem_dmabuf.c
index 4a6bb64..3cc74b0 100644
--- a/drivers/gpu/drm/i915/gem/selftests/i915_gem_dmabuf.c
+++ b/drivers/gpu/drm/i915/gem/selftests/i915_gem_dmabuf.c
@@ -102,7 +102,7 @@ static int igt_dmabuf_import_same_driver_lmem(void *arg)
obj = __i915_gem_object_create_user(i915, PAGE_SIZE, &lmem, 1);
if (IS_ERR(obj)) {
pr_err("__i915_gem_object_create_user failed with err=%ld\n",
- PTR_ERR(dmabuf));
+ PTR_ERR(obj));
err = PTR_ERR(obj);
goto out_ret;
}
@@ -158,7 +158,7 @@ static int igt_dmabuf_import_same_driver(struct drm_i915_private *i915,
regions, num_regions);
if (IS_ERR(obj)) {
pr_err("__i915_gem_object_create_user failed with err=%ld\n",
- PTR_ERR(dmabuf));
+ PTR_ERR(obj));
err = PTR_ERR(obj);
goto out_ret;
}
diff --git a/drivers/gpu/drm/i915/gem/selftests/i915_gem_migrate.c b/drivers/gpu/drm/i915/gem/selftests/i915_gem_migrate.c
index 28a700f..ecb691c 100644
--- a/drivers/gpu/drm/i915/gem/selftests/i915_gem_migrate.c
+++ b/drivers/gpu/drm/i915/gem/selftests/i915_gem_migrate.c
@@ -4,6 +4,7 @@
*/
#include "gt/intel_migrate.h"
+#include "gem/i915_gem_ttm_move.h"
static int igt_fill_check_buffer(struct drm_i915_gem_object *obj,
bool fill)
@@ -227,17 +228,38 @@ static int igt_lmem_pages_migrate(void *arg)
return err;
}
+static int igt_lmem_pages_failsafe_migrate(void *arg)
+{
+ int fail_gpu, fail_alloc, ret;
+
+ for (fail_gpu = 0; fail_gpu < 2; ++fail_gpu) {
+ for (fail_alloc = 0; fail_alloc < 2; ++fail_alloc) {
+ pr_info("Simulated failure modes: gpu: %d, alloc: %d\n",
+ fail_gpu, fail_alloc);
+ i915_ttm_migrate_set_failure_modes(fail_gpu,
+ fail_alloc);
+ ret = igt_lmem_pages_migrate(arg);
+ if (ret)
+ goto out_err;
+ }
+ }
+
+out_err:
+ i915_ttm_migrate_set_failure_modes(false, false);
+ return ret;
+}
+
int i915_gem_migrate_live_selftests(struct drm_i915_private *i915)
{
static const struct i915_subtest tests[] = {
SUBTEST(igt_smem_create_migrate),
SUBTEST(igt_lmem_create_migrate),
SUBTEST(igt_same_create_migrate),
- SUBTEST(igt_lmem_pages_migrate),
+ SUBTEST(igt_lmem_pages_failsafe_migrate),
};
if (!HAS_LMEM(i915))
return 0;
- return intel_gt_live_subtests(tests, &i915->gt);
+ return intel_gt_live_subtests(tests, to_gt(i915));
}
diff --git a/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c b/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c
index 6d30cdf..743e6ab 100644
--- a/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c
+++ b/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c
@@ -84,6 +84,7 @@ static int check_partial_mapping(struct drm_i915_gem_object *obj,
struct rnd_state *prng)
{
const unsigned long npages = obj->base.size / PAGE_SIZE;
+ struct drm_i915_private *i915 = to_i915(obj->base.dev);
struct i915_ggtt_view view;
struct i915_vma *vma;
unsigned long page;
@@ -141,7 +142,7 @@ static int check_partial_mapping(struct drm_i915_gem_object *obj,
if (offset >= obj->base.size)
goto out;
- intel_gt_flush_ggtt_writes(&to_i915(obj->base.dev)->gt);
+ intel_gt_flush_ggtt_writes(to_gt(i915));
p = i915_gem_object_get_page(obj, offset >> PAGE_SHIFT);
cpu = kmap(p) + offset_in_page(offset);
@@ -175,6 +176,7 @@ static int check_partial_mappings(struct drm_i915_gem_object *obj,
{
const unsigned int nreal = obj->scratch / PAGE_SIZE;
const unsigned long npages = obj->base.size / PAGE_SIZE;
+ struct drm_i915_private *i915 = to_i915(obj->base.dev);
struct i915_vma *vma;
unsigned long page;
int err;
@@ -234,7 +236,7 @@ static int check_partial_mappings(struct drm_i915_gem_object *obj,
if (offset >= obj->base.size)
continue;
- intel_gt_flush_ggtt_writes(&to_i915(obj->base.dev)->gt);
+ intel_gt_flush_ggtt_writes(to_gt(i915));
p = i915_gem_object_get_page(obj, offset >> PAGE_SHIFT);
cpu = kmap(p) + offset_in_page(offset);
@@ -616,14 +618,14 @@ static bool assert_mmap_offset(struct drm_i915_private *i915,
static void disable_retire_worker(struct drm_i915_private *i915)
{
i915_gem_driver_unregister__shrinker(i915);
- intel_gt_pm_get(&i915->gt);
- cancel_delayed_work_sync(&i915->gt.requests.retire_work);
+ intel_gt_pm_get(to_gt(i915));
+ cancel_delayed_work_sync(&to_gt(i915)->requests.retire_work);
}
static void restore_retire_worker(struct drm_i915_private *i915)
{
igt_flush_test(i915);
- intel_gt_pm_put(&i915->gt);
+ intel_gt_pm_put(to_gt(i915));
i915_gem_driver_register__shrinker(i915);
}
@@ -651,8 +653,8 @@ static int igt_mmap_offset_exhaustion(void *arg)
/* Disable background reaper */
disable_retire_worker(i915);
- GEM_BUG_ON(!i915->gt.awake);
- intel_gt_retire_requests(&i915->gt);
+ GEM_BUG_ON(!to_gt(i915)->awake);
+ intel_gt_retire_requests(to_gt(i915));
i915_gem_drain_freed_objects(i915);
/* Trim the device mmap space to only a page */
@@ -728,7 +730,7 @@ static int igt_mmap_offset_exhaustion(void *arg)
/* Now fill with busy dead objects that we expect to reap */
for (loop = 0; loop < 3; loop++) {
- if (intel_gt_is_wedged(&i915->gt))
+ if (intel_gt_is_wedged(to_gt(i915)))
break;
obj = i915_gem_object_create_internal(i915, PAGE_SIZE);
@@ -942,7 +944,7 @@ static int __igt_mmap(struct drm_i915_private *i915,
}
if (type == I915_MMAP_TYPE_GTT)
- intel_gt_flush_ggtt_writes(&i915->gt);
+ intel_gt_flush_ggtt_writes(to_gt(i915));
err = wc_check(obj);
if (err == -ENXIO)
@@ -1049,7 +1051,7 @@ static int __igt_mmap_access(struct drm_i915_private *i915,
goto out_unmap;
}
- intel_gt_flush_ggtt_writes(&i915->gt);
+ intel_gt_flush_ggtt_writes(to_gt(i915));
err = access_process_vm(current, addr, &x, sizeof(x), 0);
if (err != sizeof(x)) {
@@ -1065,7 +1067,7 @@ static int __igt_mmap_access(struct drm_i915_private *i915,
goto out_unmap;
}
- intel_gt_flush_ggtt_writes(&i915->gt);
+ intel_gt_flush_ggtt_writes(to_gt(i915));
err = __get_user(y, ptr);
if (err) {
@@ -1165,7 +1167,7 @@ static int __igt_mmap_gpu(struct drm_i915_private *i915,
}
if (type == I915_MMAP_TYPE_GTT)
- intel_gt_flush_ggtt_writes(&i915->gt);
+ intel_gt_flush_ggtt_writes(to_gt(i915));
for_each_uabi_engine(engine, i915) {
struct i915_request *rq;
diff --git a/drivers/gpu/drm/i915/gt/gen6_ppgtt.c b/drivers/gpu/drm/i915/gt/gen6_ppgtt.c
index 890191f..6e92929 100644
--- a/drivers/gpu/drm/i915/gt/gen6_ppgtt.c
+++ b/drivers/gpu/drm/i915/gt/gen6_ppgtt.c
@@ -185,7 +185,6 @@ static void gen6_alloc_va_range(struct i915_address_space *vm,
pt = stash->pt[0];
__i915_gem_object_pin_pages(pt->base);
- i915_gem_object_make_unshrinkable(pt->base);
fill32_px(pt, vm->scratch[0]->encode);
@@ -262,30 +261,14 @@ static void gen6_ppgtt_cleanup(struct i915_address_space *vm)
{
struct gen6_ppgtt *ppgtt = to_gen6_ppgtt(i915_vm_to_ppgtt(vm));
- __i915_vma_put(ppgtt->vma);
-
gen6_ppgtt_free_pd(ppgtt);
free_scratch(vm);
mutex_destroy(&ppgtt->flush);
- mutex_destroy(&ppgtt->pin_mutex);
free_pd(&ppgtt->base.vm, ppgtt->base.pd);
}
-static int pd_vma_set_pages(struct i915_vma *vma)
-{
- vma->pages = ERR_PTR(-ENODEV);
- return 0;
-}
-
-static void pd_vma_clear_pages(struct i915_vma *vma)
-{
- GEM_BUG_ON(!vma->pages);
-
- vma->pages = NULL;
-}
-
static void pd_vma_bind(struct i915_address_space *vm,
struct i915_vm_pt_stash *stash,
struct i915_vma *vma,
@@ -325,43 +308,10 @@ static void pd_vma_unbind(struct i915_address_space *vm, struct i915_vma *vma)
}
static const struct i915_vma_ops pd_vma_ops = {
- .set_pages = pd_vma_set_pages,
- .clear_pages = pd_vma_clear_pages,
.bind_vma = pd_vma_bind,
.unbind_vma = pd_vma_unbind,
};
-static struct i915_vma *pd_vma_create(struct gen6_ppgtt *ppgtt, int size)
-{
- struct i915_ggtt *ggtt = ppgtt->base.vm.gt->ggtt;
- struct i915_vma *vma;
-
- GEM_BUG_ON(!IS_ALIGNED(size, I915_GTT_PAGE_SIZE));
- GEM_BUG_ON(size > ggtt->vm.total);
-
- vma = i915_vma_alloc();
- if (!vma)
- return ERR_PTR(-ENOMEM);
-
- i915_active_init(&vma->active, NULL, NULL, 0);
-
- kref_init(&vma->ref);
- mutex_init(&vma->pages_mutex);
- vma->vm = i915_vm_get(&ggtt->vm);
- vma->ops = &pd_vma_ops;
- vma->private = ppgtt;
-
- vma->size = size;
- vma->fence_size = size;
- atomic_set(&vma->flags, I915_VMA_GGTT);
- vma->ggtt_view.type = I915_GGTT_VIEW_ROTATED; /* prevent fencing */
-
- INIT_LIST_HEAD(&vma->obj_link);
- INIT_LIST_HEAD(&vma->closed_link);
-
- return vma;
-}
-
int gen6_ppgtt_pin(struct i915_ppgtt *base, struct i915_gem_ww_ctx *ww)
{
struct gen6_ppgtt *ppgtt = to_gen6_ppgtt(base);
@@ -378,24 +328,85 @@ int gen6_ppgtt_pin(struct i915_ppgtt *base, struct i915_gem_ww_ctx *ww)
if (atomic_add_unless(&ppgtt->pin_count, 1, 0))
return 0;
- if (mutex_lock_interruptible(&ppgtt->pin_mutex))
- return -EINTR;
+ /* grab the ppgtt resv to pin the object */
+ err = i915_vm_lock_objects(&ppgtt->base.vm, ww);
+ if (err)
+ return err;
/*
* PPGTT PDEs reside in the GGTT and consists of 512 entries. The
* allocator works in address space sizes, so it's multiplied by page
* size. We allocate at the top of the GTT to avoid fragmentation.
*/
- err = 0;
- if (!atomic_read(&ppgtt->pin_count))
+ if (!atomic_read(&ppgtt->pin_count)) {
err = i915_ggtt_pin(ppgtt->vma, ww, GEN6_PD_ALIGN, PIN_HIGH);
+
+ GEM_BUG_ON(ppgtt->vma->fence);
+ clear_bit(I915_VMA_CAN_FENCE_BIT, __i915_vma_flags(ppgtt->vma));
+ }
if (!err)
atomic_inc(&ppgtt->pin_count);
- mutex_unlock(&ppgtt->pin_mutex);
return err;
}
+static int pd_dummy_obj_get_pages(struct drm_i915_gem_object *obj)
+{
+ obj->mm.pages = ZERO_SIZE_PTR;
+ return 0;
+}
+
+static void pd_dummy_obj_put_pages(struct drm_i915_gem_object *obj,
+ struct sg_table *pages)
+{
+}
+
+static const struct drm_i915_gem_object_ops pd_dummy_obj_ops = {
+ .name = "pd_dummy_obj",
+ .get_pages = pd_dummy_obj_get_pages,
+ .put_pages = pd_dummy_obj_put_pages,
+};
+
+static struct i915_page_directory *
+gen6_alloc_top_pd(struct gen6_ppgtt *ppgtt)
+{
+ struct i915_ggtt * const ggtt = ppgtt->base.vm.gt->ggtt;
+ struct i915_page_directory *pd;
+ int err;
+
+ pd = __alloc_pd(I915_PDES);
+ if (unlikely(!pd))
+ return ERR_PTR(-ENOMEM);
+
+ pd->pt.base = __i915_gem_object_create_internal(ppgtt->base.vm.gt->i915,
+ &pd_dummy_obj_ops,
+ I915_PDES * SZ_4K);
+ if (IS_ERR(pd->pt.base)) {
+ err = PTR_ERR(pd->pt.base);
+ pd->pt.base = NULL;
+ goto err_pd;
+ }
+
+ pd->pt.base->base.resv = i915_vm_resv_get(&ppgtt->base.vm);
+ pd->pt.base->shares_resv_from = &ppgtt->base.vm;
+
+ ppgtt->vma = i915_vma_instance(pd->pt.base, &ggtt->vm, NULL);
+ if (IS_ERR(ppgtt->vma)) {
+ err = PTR_ERR(ppgtt->vma);
+ ppgtt->vma = NULL;
+ goto err_pd;
+ }
+
+ /* The dummy object we create is special, override ops.. */
+ ppgtt->vma->ops = &pd_vma_ops;
+ ppgtt->vma->private = ppgtt;
+ return pd;
+
+err_pd:
+ free_pd(&ppgtt->base.vm, pd);
+ return ERR_PTR(err);
+}
+
void gen6_ppgtt_unpin(struct i915_ppgtt *base)
{
struct gen6_ppgtt *ppgtt = to_gen6_ppgtt(base);
@@ -405,17 +416,6 @@ void gen6_ppgtt_unpin(struct i915_ppgtt *base)
i915_vma_unpin(ppgtt->vma);
}
-void gen6_ppgtt_unpin_all(struct i915_ppgtt *base)
-{
- struct gen6_ppgtt *ppgtt = to_gen6_ppgtt(base);
-
- if (!atomic_read(&ppgtt->pin_count))
- return;
-
- i915_vma_unpin(ppgtt->vma);
- atomic_set(&ppgtt->pin_count, 0);
-}
-
struct i915_ppgtt *gen6_ppgtt_create(struct intel_gt *gt)
{
struct i915_ggtt * const ggtt = gt->ggtt;
@@ -427,7 +427,6 @@ struct i915_ppgtt *gen6_ppgtt_create(struct intel_gt *gt)
return ERR_PTR(-ENOMEM);
mutex_init(&ppgtt->flush);
- mutex_init(&ppgtt->pin_mutex);
ppgtt_init(&ppgtt->base, gt, 0);
ppgtt->base.vm.pd_shift = ilog2(SZ_4K * SZ_4K / sizeof(gen6_pte_t));
@@ -440,21 +439,16 @@ struct i915_ppgtt *gen6_ppgtt_create(struct intel_gt *gt)
ppgtt->base.vm.cleanup = gen6_ppgtt_cleanup;
ppgtt->base.vm.alloc_pt_dma = alloc_pt_dma;
+ ppgtt->base.vm.alloc_scratch_dma = alloc_pt_dma;
ppgtt->base.vm.pte_encode = ggtt->vm.pte_encode;
- ppgtt->base.pd = __alloc_pd(I915_PDES);
- if (!ppgtt->base.pd) {
- err = -ENOMEM;
- goto err_free;
- }
-
err = gen6_ppgtt_init_scratch(ppgtt);
if (err)
- goto err_pd;
+ goto err_free;
- ppgtt->vma = pd_vma_create(ppgtt, GEN6_PD_SIZE);
- if (IS_ERR(ppgtt->vma)) {
- err = PTR_ERR(ppgtt->vma);
+ ppgtt->base.pd = gen6_alloc_top_pd(ppgtt);
+ if (IS_ERR(ppgtt->base.pd)) {
+ err = PTR_ERR(ppgtt->base.pd);
goto err_scratch;
}
@@ -462,10 +456,7 @@ struct i915_ppgtt *gen6_ppgtt_create(struct intel_gt *gt)
err_scratch:
free_scratch(&ppgtt->base.vm);
-err_pd:
- free_pd(&ppgtt->base.vm, ppgtt->base.pd);
err_free:
- mutex_destroy(&ppgtt->pin_mutex);
kfree(ppgtt);
return ERR_PTR(err);
}
diff --git a/drivers/gpu/drm/i915/gt/gen6_ppgtt.h b/drivers/gpu/drm/i915/gt/gen6_ppgtt.h
index 6a61a5c..5e5cf2e 100644
--- a/drivers/gpu/drm/i915/gt/gen6_ppgtt.h
+++ b/drivers/gpu/drm/i915/gt/gen6_ppgtt.h
@@ -19,7 +19,6 @@ struct gen6_ppgtt {
u32 pp_dir;
atomic_t pin_count;
- struct mutex pin_mutex;
bool scan_for_unused_pt;
};
@@ -71,7 +70,6 @@ static inline struct gen6_ppgtt *to_gen6_ppgtt(struct i915_ppgtt *base)
int gen6_ppgtt_pin(struct i915_ppgtt *base, struct i915_gem_ww_ctx *ww);
void gen6_ppgtt_unpin(struct i915_ppgtt *base);
-void gen6_ppgtt_unpin_all(struct i915_ppgtt *base);
void gen6_ppgtt_enable(struct intel_gt *gt);
void gen7_ppgtt_enable(struct intel_gt *gt);
struct i915_ppgtt *gen6_ppgtt_create(struct intel_gt *gt);
diff --git a/drivers/gpu/drm/i915/gt/gen8_engine_cs.c b/drivers/gpu/drm/i915/gt/gen8_engine_cs.c
index 461844d..e320610 100644
--- a/drivers/gpu/drm/i915/gt/gen8_engine_cs.c
+++ b/drivers/gpu/drm/i915/gt/gen8_engine_cs.c
@@ -42,7 +42,7 @@ int gen8_emit_flush_rcs(struct i915_request *rq, u32 mode)
vf_flush_wa = true;
/* WaForGAMHang:kbl */
- if (IS_KBL_GT_STEP(rq->engine->i915, 0, STEP_C0))
+ if (IS_KBL_GRAPHICS_STEP(rq->engine->i915, 0, STEP_C0))
dc_flush_wa = true;
}
diff --git a/drivers/gpu/drm/i915/gt/gen8_ppgtt.c b/drivers/gpu/drm/i915/gt/gen8_ppgtt.c
index 037a9a6..b012c50 100644
--- a/drivers/gpu/drm/i915/gt/gen8_ppgtt.c
+++ b/drivers/gpu/drm/i915/gt/gen8_ppgtt.c
@@ -18,7 +18,7 @@
static u64 gen8_pde_encode(const dma_addr_t addr,
const enum i915_cache_level level)
{
- u64 pde = addr | _PAGE_PRESENT | _PAGE_RW;
+ u64 pde = addr | GEN8_PAGE_PRESENT | GEN8_PAGE_RW;
if (level != I915_CACHE_NONE)
pde |= PPAT_CACHED_PDE;
@@ -32,10 +32,10 @@ static u64 gen8_pte_encode(dma_addr_t addr,
enum i915_cache_level level,
u32 flags)
{
- gen8_pte_t pte = addr | _PAGE_PRESENT | _PAGE_RW;
+ gen8_pte_t pte = addr | GEN8_PAGE_PRESENT | GEN8_PAGE_RW;
if (unlikely(flags & PTE_READ_ONLY))
- pte &= ~_PAGE_RW;
+ pte &= ~GEN8_PAGE_RW;
if (flags & PTE_LM)
pte |= GEN12_PPGTT_PTE_LM;
@@ -301,7 +301,6 @@ static void __gen8_ppgtt_alloc(struct i915_address_space * const vm,
pt = stash->pt[!!lvl];
__i915_gem_object_pin_pages(pt->base);
- i915_gem_object_make_unshrinkable(pt->base);
fill_px(pt, vm->scratch[lvl]->encode);
@@ -652,7 +651,7 @@ static int gen8_init_scratch(struct i915_address_space *vm)
vm->scratch[0]->encode =
gen8_pte_encode(px_dma(vm->scratch[0]),
- I915_CACHE_LLC, pte_flags);
+ I915_CACHE_NONE, pte_flags);
for (i = 1; i <= vm->top; i++) {
struct drm_i915_gem_object *obj;
@@ -668,7 +667,7 @@ static int gen8_init_scratch(struct i915_address_space *vm)
}
fill_px(obj, vm->scratch[i - 1]->encode);
- obj->encode = gen8_pde_encode(px_dma(obj), I915_CACHE_LLC);
+ obj->encode = gen8_pde_encode(px_dma(obj), I915_CACHE_NONE);
vm->scratch[i] = obj;
}
@@ -777,10 +776,29 @@ struct i915_ppgtt *gen8_ppgtt_create(struct intel_gt *gt,
*/
ppgtt->vm.has_read_only = !IS_GRAPHICS_VER(gt->i915, 11, 12);
- if (HAS_LMEM(gt->i915))
+ if (HAS_LMEM(gt->i915)) {
ppgtt->vm.alloc_pt_dma = alloc_pt_lmem;
- else
+
+ /*
+ * On some platforms the hw has dropped support for 4K GTT pages
+ * when dealing with LMEM, and due to the design of 64K GTT
+ * pages in the hw, we can only mark the *entire* page-table as
+ * operating in 64K GTT mode, since the enable bit is still on
+ * the pde, and not the pte. And since we still need to allow
+ * 4K GTT pages for SMEM objects, we can't have a "normal" 4K
+ * page-table with scratch pointing to LMEM, since that's
+ * undefined from the hw pov. The simplest solution is to just
+ * move the 64K scratch page to SMEM on such platforms and call
+ * it a day, since that should work for all configurations.
+ */
+ if (HAS_64K_PAGES(gt->i915))
+ ppgtt->vm.alloc_scratch_dma = alloc_pt_dma;
+ else
+ ppgtt->vm.alloc_scratch_dma = alloc_pt_lmem;
+ } else {
ppgtt->vm.alloc_pt_dma = alloc_pt_dma;
+ ppgtt->vm.alloc_scratch_dma = alloc_pt_dma;
+ }
err = gen8_init_scratch(&ppgtt->vm);
if (err)
diff --git a/drivers/gpu/drm/i915/gt/intel_context.c b/drivers/gpu/drm/i915/gt/intel_context.c
index 5634d14..ba083d8 100644
--- a/drivers/gpu/drm/i915/gt/intel_context.c
+++ b/drivers/gpu/drm/i915/gt/intel_context.c
@@ -219,7 +219,7 @@ int __intel_context_do_pin_ww(struct intel_context *ce,
*/
err = i915_gem_object_lock(ce->timeline->hwsp_ggtt->obj, ww);
- if (!err && ce->ring->vma->obj)
+ if (!err)
err = i915_gem_object_lock(ce->ring->vma->obj, ww);
if (!err && ce->state)
err = i915_gem_object_lock(ce->state->obj, ww);
@@ -228,17 +228,17 @@ int __intel_context_do_pin_ww(struct intel_context *ce,
if (err)
return err;
- err = i915_active_acquire(&ce->active);
+ err = ce->ops->pre_pin(ce, ww, &vaddr);
if (err)
goto err_ctx_unpin;
- err = ce->ops->pre_pin(ce, ww, &vaddr);
+ err = i915_active_acquire(&ce->active);
if (err)
- goto err_release;
+ goto err_post_unpin;
err = mutex_lock_interruptible(&ce->pin_mutex);
if (err)
- goto err_post_unpin;
+ goto err_release;
intel_engine_pm_might_get(ce->engine);
@@ -273,11 +273,11 @@ int __intel_context_do_pin_ww(struct intel_context *ce,
err_unlock:
mutex_unlock(&ce->pin_mutex);
+err_release:
+ i915_active_release(&ce->active);
err_post_unpin:
if (!handoff)
ce->ops->post_unpin(ce);
-err_release:
- i915_active_release(&ce->active);
err_ctx_unpin:
intel_context_post_unpin(ce);
@@ -364,7 +364,7 @@ static int __intel_context_active(struct i915_active *active)
return 0;
}
-static int __i915_sw_fence_call
+static int
sw_fence_dummy_notify(struct i915_sw_fence *sf,
enum i915_sw_fence_notify state)
{
diff --git a/drivers/gpu/drm/i915/gt/intel_context.h b/drivers/gpu/drm/i915/gt/intel_context.h
index 246c37d..d8c74bb 100644
--- a/drivers/gpu/drm/i915/gt/intel_context.h
+++ b/drivers/gpu/drm/i915/gt/intel_context.h
@@ -211,7 +211,8 @@ static inline void intel_context_enter(struct intel_context *ce)
static inline void intel_context_mark_active(struct intel_context *ce)
{
- lockdep_assert_held(&ce->timeline->mutex);
+ lockdep_assert(lockdep_is_held(&ce->timeline->mutex) ||
+ test_bit(CONTEXT_IS_PARKING, &ce->flags));
++ce->active_count;
}
diff --git a/drivers/gpu/drm/i915/gt/intel_context_types.h b/drivers/gpu/drm/i915/gt/intel_context_types.h
index 9e0177d..30cd81a 100644
--- a/drivers/gpu/drm/i915/gt/intel_context_types.h
+++ b/drivers/gpu/drm/i915/gt/intel_context_types.h
@@ -118,6 +118,7 @@ struct intel_context {
#define CONTEXT_LRCA_DIRTY 9
#define CONTEXT_GUC_INIT 10
#define CONTEXT_PERMA_PIN 11
+#define CONTEXT_IS_PARKING 12
struct {
u64 timeout_us;
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
index ff6753c..352254e 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
+++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
@@ -325,6 +325,38 @@ static int intel_engine_setup(struct intel_gt *gt, enum intel_engine_id id,
engine->id = id;
engine->legacy_idx = INVALID_ENGINE;
engine->mask = BIT(id);
+ if (GRAPHICS_VER(gt->i915) >= 11) {
+ static const u32 engine_reset_domains[] = {
+ [RCS0] = GEN11_GRDOM_RENDER,
+ [BCS0] = GEN11_GRDOM_BLT,
+ [VCS0] = GEN11_GRDOM_MEDIA,
+ [VCS1] = GEN11_GRDOM_MEDIA2,
+ [VCS2] = GEN11_GRDOM_MEDIA3,
+ [VCS3] = GEN11_GRDOM_MEDIA4,
+ [VCS4] = GEN11_GRDOM_MEDIA5,
+ [VCS5] = GEN11_GRDOM_MEDIA6,
+ [VCS6] = GEN11_GRDOM_MEDIA7,
+ [VCS7] = GEN11_GRDOM_MEDIA8,
+ [VECS0] = GEN11_GRDOM_VECS,
+ [VECS1] = GEN11_GRDOM_VECS2,
+ [VECS2] = GEN11_GRDOM_VECS3,
+ [VECS3] = GEN11_GRDOM_VECS4,
+ };
+ GEM_BUG_ON(id >= ARRAY_SIZE(engine_reset_domains) ||
+ !engine_reset_domains[id]);
+ engine->reset_domain = engine_reset_domains[id];
+ } else {
+ static const u32 engine_reset_domains[] = {
+ [RCS0] = GEN6_GRDOM_RENDER,
+ [BCS0] = GEN6_GRDOM_BLT,
+ [VCS0] = GEN6_GRDOM_MEDIA,
+ [VCS1] = GEN8_GRDOM_MEDIA2,
+ [VECS0] = GEN6_GRDOM_VECS,
+ };
+ GEM_BUG_ON(id >= ARRAY_SIZE(engine_reset_domains) ||
+ !engine_reset_domains[id]);
+ engine->reset_domain = engine_reset_domains[id];
+ }
engine->i915 = i915;
engine->gt = gt;
engine->uncore = gt->uncore;
@@ -363,7 +395,7 @@ static int intel_engine_setup(struct intel_gt *gt, enum intel_engine_id id,
DRIVER_CAPS(i915)->has_logical_contexts = true;
ewma__engine_latency_init(&engine->latency);
- seqcount_init(&engine->stats.lock);
+ seqcount_init(&engine->stats.execlists.lock);
ATOMIC_INIT_NOTIFIER_HEAD(&engine->context_status_notifier);
@@ -1676,14 +1708,18 @@ static void intel_engine_print_registers(struct intel_engine_cs *engine,
static void print_request_ring(struct drm_printer *m, struct i915_request *rq)
{
+ struct i915_vma_snapshot *vsnap = &rq->batch_snapshot;
void *ring;
int size;
+ if (!i915_vma_snapshot_present(vsnap))
+ vsnap = NULL;
+
drm_printf(m,
"[head %04x, postfix %04x, tail %04x, batch 0x%08x_%08x]:\n",
rq->head, rq->postfix, rq->tail,
- rq->batch ? upper_32_bits(rq->batch->node.start) : ~0u,
- rq->batch ? lower_32_bits(rq->batch->node.start) : ~0u);
+ vsnap ? upper_32_bits(vsnap->gtt_offset) : ~0u,
+ vsnap ? lower_32_bits(vsnap->gtt_offset) : ~0u);
size = rq->tail - rq->head;
if (rq->tail < rq->head)
@@ -1915,22 +1951,6 @@ void intel_engine_dump(struct intel_engine_cs *engine,
intel_engine_print_breadcrumbs(engine, m);
}
-static ktime_t __intel_engine_get_busy_time(struct intel_engine_cs *engine,
- ktime_t *now)
-{
- ktime_t total = engine->stats.total;
-
- /*
- * If the engine is executing something at the moment
- * add it to the total.
- */
- *now = ktime_get();
- if (READ_ONCE(engine->stats.active))
- total = ktime_add(total, ktime_sub(*now, engine->stats.start));
-
- return total;
-}
-
/**
* intel_engine_get_busy_time() - Return current accumulated engine busyness
* @engine: engine to report on
@@ -1940,15 +1960,7 @@ static ktime_t __intel_engine_get_busy_time(struct intel_engine_cs *engine,
*/
ktime_t intel_engine_get_busy_time(struct intel_engine_cs *engine, ktime_t *now)
{
- unsigned int seq;
- ktime_t total;
-
- do {
- seq = read_seqcount_begin(&engine->stats.lock);
- total = __intel_engine_get_busy_time(engine, now);
- } while (read_seqcount_retry(&engine->stats.lock, seq));
-
- return total;
+ return engine->busyness(engine, now);
}
struct intel_context *
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_pm.c b/drivers/gpu/drm/i915/gt/intel_engine_pm.c
index a1334b4..b0a4a2d 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_pm.c
+++ b/drivers/gpu/drm/i915/gt/intel_engine_pm.c
@@ -26,7 +26,7 @@ static void dbg_poison_ce(struct intel_context *ce)
int type = i915_coherent_map_type(ce->engine->i915, obj, true);
void *map;
- if (!i915_gem_object_trylock(obj))
+ if (!i915_gem_object_trylock(obj, NULL))
return;
map = i915_gem_object_pin_map(obj, type);
@@ -80,39 +80,6 @@ static int __engine_unpark(struct intel_wakeref *wf)
return 0;
}
-#if IS_ENABLED(CONFIG_LOCKDEP)
-
-static unsigned long __timeline_mark_lock(struct intel_context *ce)
-{
- unsigned long flags;
-
- local_irq_save(flags);
- mutex_acquire(&ce->timeline->mutex.dep_map, 2, 0, _THIS_IP_);
-
- return flags;
-}
-
-static void __timeline_mark_unlock(struct intel_context *ce,
- unsigned long flags)
-{
- mutex_release(&ce->timeline->mutex.dep_map, _THIS_IP_);
- local_irq_restore(flags);
-}
-
-#else
-
-static unsigned long __timeline_mark_lock(struct intel_context *ce)
-{
- return 0;
-}
-
-static void __timeline_mark_unlock(struct intel_context *ce,
- unsigned long flags)
-{
-}
-
-#endif /* !IS_ENABLED(CONFIG_LOCKDEP) */
-
static void duration(struct dma_fence *fence, struct dma_fence_cb *cb)
{
struct i915_request *rq = to_request(fence);
@@ -159,7 +126,6 @@ static bool switch_to_kernel_context(struct intel_engine_cs *engine)
{
struct intel_context *ce = engine->kernel_context;
struct i915_request *rq;
- unsigned long flags;
bool result = true;
/*
@@ -214,7 +180,7 @@ static bool switch_to_kernel_context(struct intel_engine_cs *engine)
* engine->wakeref.count, we may see the request completion and retire
* it causing an underflow of the engine->wakeref.
*/
- flags = __timeline_mark_lock(ce);
+ set_bit(CONTEXT_IS_PARKING, &ce->flags);
GEM_BUG_ON(atomic_read(&ce->timeline->active_count) < 0);
rq = __i915_request_create(ce, GFP_NOWAIT);
@@ -246,7 +212,7 @@ static bool switch_to_kernel_context(struct intel_engine_cs *engine)
result = false;
out_unlock:
- __timeline_mark_unlock(ce, flags);
+ clear_bit(CONTEXT_IS_PARKING, &ce->flags);
return result;
}
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_stats.h b/drivers/gpu/drm/i915/gt/intel_engine_stats.h
index 24fbdd9..8e762d6 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_stats.h
+++ b/drivers/gpu/drm/i915/gt/intel_engine_stats.h
@@ -15,45 +15,46 @@
static inline void intel_engine_context_in(struct intel_engine_cs *engine)
{
+ struct intel_engine_execlists_stats *stats = &engine->stats.execlists;
unsigned long flags;
- if (engine->stats.active) {
- engine->stats.active++;
+ if (stats->active) {
+ stats->active++;
return;
}
/* The writer is serialised; but the pmu reader may be from hardirq */
local_irq_save(flags);
- write_seqcount_begin(&engine->stats.lock);
+ write_seqcount_begin(&stats->lock);
- engine->stats.start = ktime_get();
- engine->stats.active++;
+ stats->start = ktime_get();
+ stats->active++;
- write_seqcount_end(&engine->stats.lock);
+ write_seqcount_end(&stats->lock);
local_irq_restore(flags);
- GEM_BUG_ON(!engine->stats.active);
+ GEM_BUG_ON(!stats->active);
}
static inline void intel_engine_context_out(struct intel_engine_cs *engine)
{
+ struct intel_engine_execlists_stats *stats = &engine->stats.execlists;
unsigned long flags;
- GEM_BUG_ON(!engine->stats.active);
- if (engine->stats.active > 1) {
- engine->stats.active--;
+ GEM_BUG_ON(!stats->active);
+ if (stats->active > 1) {
+ stats->active--;
return;
}
local_irq_save(flags);
- write_seqcount_begin(&engine->stats.lock);
+ write_seqcount_begin(&stats->lock);
- engine->stats.active--;
- engine->stats.total =
- ktime_add(engine->stats.total,
- ktime_sub(ktime_get(), engine->stats.start));
+ stats->active--;
+ stats->total = ktime_add(stats->total,
+ ktime_sub(ktime_get(), stats->start));
- write_seqcount_end(&engine->stats.lock);
+ write_seqcount_end(&stats->lock);
local_irq_restore(flags);
}
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_types.h b/drivers/gpu/drm/i915/gt/intel_engine_types.h
index e0f7735..36365bd 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_types.h
+++ b/drivers/gpu/drm/i915/gt/intel_engine_types.h
@@ -257,6 +257,55 @@ struct intel_engine_execlists {
#define INTEL_ENGINE_CS_MAX_NAME 8
+struct intel_engine_execlists_stats {
+ /**
+ * @active: Number of contexts currently scheduled in.
+ */
+ unsigned int active;
+
+ /**
+ * @lock: Lock protecting the below fields.
+ */
+ seqcount_t lock;
+
+ /**
+ * @total: Total time this engine was busy.
+ *
+ * Accumulated time not counting the most recent block in cases where
+ * engine is currently busy (active > 0).
+ */
+ ktime_t total;
+
+ /**
+ * @start: Timestamp of the last idle to active transition.
+ *
+ * Idle is defined as active == 0, active is active > 0.
+ */
+ ktime_t start;
+};
+
+struct intel_engine_guc_stats {
+ /**
+ * @running: Active state of the engine when busyness was last sampled.
+ */
+ bool running;
+
+ /**
+ * @prev_total: Previous value of total runtime clock cycles.
+ */
+ u32 prev_total;
+
+ /**
+ * @total_gt_clks: Total gt clock cycles this engine was busy.
+ */
+ u64 total_gt_clks;
+
+ /**
+ * @start_gt_clk: GT clock time of last idle to active transition.
+ */
+ u64 start_gt_clk;
+};
+
struct intel_engine_cs {
struct drm_i915_private *i915;
struct intel_gt *gt;
@@ -269,6 +318,7 @@ struct intel_engine_cs {
unsigned int guc_id;
intel_engine_mask_t mask;
+ u32 reset_domain;
/**
* @logical_mask: logical mask of engine, reported to user space via
* query IOCTL and used to communicate with the GuC in logical space.
@@ -439,6 +489,12 @@ struct intel_engine_cs {
void (*add_active_request)(struct i915_request *rq);
void (*remove_active_request)(struct i915_request *rq);
+ /*
+ * Get engine busyness and the time at which the busyness was sampled.
+ */
+ ktime_t (*busyness)(struct intel_engine_cs *engine,
+ ktime_t *now);
+
struct intel_engine_execlists execlists;
/*
@@ -488,30 +544,10 @@ struct intel_engine_cs {
u32 (*get_cmd_length_mask)(u32 cmd_header);
struct {
- /**
- * @active: Number of contexts currently scheduled in.
- */
- unsigned int active;
-
- /**
- * @lock: Lock protecting the below fields.
- */
- seqcount_t lock;
-
- /**
- * @total: Total time this engine was busy.
- *
- * Accumulated time not counting the most recent block in cases
- * where engine is currently busy (active > 0).
- */
- ktime_t total;
-
- /**
- * @start: Timestamp of the last idle to active transition.
- *
- * Idle is defined as active == 0, active is active > 0.
- */
- ktime_t start;
+ union {
+ struct intel_engine_execlists_stats execlists;
+ struct intel_engine_guc_stats guc;
+ };
/**
* @rps: Utilisation at last RPS sampling.
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_user.c b/drivers/gpu/drm/i915/gt/intel_engine_user.c
index 8f8bea0..9ce85a8 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_user.c
+++ b/drivers/gpu/drm/i915/gt/intel_engine_user.c
@@ -116,7 +116,7 @@ static void set_scheduler_caps(struct drm_i915_private *i915)
disabled |= (I915_SCHEDULER_CAP_ENABLED |
I915_SCHEDULER_CAP_PRIORITY);
- if (intel_uc_uses_guc_submission(&i915->gt.uc))
+ if (intel_uc_uses_guc_submission(&to_gt(i915)->uc))
enabled |= I915_SCHEDULER_CAP_STATIC_PRIORITY_MAP;
for (i = 0; i < ARRAY_SIZE(map); i++) {
diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
index bedb800..a69df5e 100644
--- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
+++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
@@ -2186,7 +2186,8 @@ struct execlists_capture {
static void execlists_capture_work(struct work_struct *work)
{
struct execlists_capture *cap = container_of(work, typeof(*cap), work);
- const gfp_t gfp = GFP_KERNEL | __GFP_RETRY_MAYFAIL | __GFP_NOWARN;
+ const gfp_t gfp = __GFP_KSWAPD_RECLAIM | __GFP_RETRY_MAYFAIL |
+ __GFP_NOWARN;
struct intel_engine_cs *engine = cap->rq->engine;
struct intel_gt_coredump *gt = cap->error->gt;
struct intel_engine_capture_vma *vma;
@@ -3293,6 +3294,38 @@ static void execlists_release(struct intel_engine_cs *engine)
lrc_fini_wa_ctx(engine);
}
+static ktime_t __execlists_engine_busyness(struct intel_engine_cs *engine,
+ ktime_t *now)
+{
+ struct intel_engine_execlists_stats *stats = &engine->stats.execlists;
+ ktime_t total = stats->total;
+
+ /*
+ * If the engine is executing something at the moment
+ * add it to the total.
+ */
+ *now = ktime_get();
+ if (READ_ONCE(stats->active))
+ total = ktime_add(total, ktime_sub(*now, stats->start));
+
+ return total;
+}
+
+static ktime_t execlists_engine_busyness(struct intel_engine_cs *engine,
+ ktime_t *now)
+{
+ struct intel_engine_execlists_stats *stats = &engine->stats.execlists;
+ unsigned int seq;
+ ktime_t total;
+
+ do {
+ seq = read_seqcount_begin(&stats->lock);
+ total = __execlists_engine_busyness(engine, now);
+ } while (read_seqcount_retry(&stats->lock, seq));
+
+ return total;
+}
+
static void
logical_ring_default_vfuncs(struct intel_engine_cs *engine)
{
@@ -3349,6 +3382,8 @@ logical_ring_default_vfuncs(struct intel_engine_cs *engine)
engine->emit_bb_start = gen8_emit_bb_start;
else
engine->emit_bb_start = gen8_emit_bb_start_noarb;
+
+ engine->busyness = execlists_engine_busyness;
}
static void logical_ring_default_irqs(struct intel_engine_cs *engine)
diff --git a/drivers/gpu/drm/i915/gt/intel_ggtt.c b/drivers/gpu/drm/i915/gt/intel_ggtt.c
index 57c9755..5263dda 100644
--- a/drivers/gpu/drm/i915/gt/intel_ggtt.c
+++ b/drivers/gpu/drm/i915/gt/intel_ggtt.c
@@ -3,12 +3,14 @@
* Copyright © 2020 Intel Corporation
*/
+#include <linux/agp_backend.h>
#include <linux/stop_machine.h>
#include <asm/set_memory.h>
#include <asm/smp.h>
#include <drm/i915_drm.h>
+#include <drm/intel-gtt.h>
#include "gem/i915_gem_lmem.h"
@@ -20,9 +22,6 @@
#include "intel_gtt.h"
#include "gen8_ppgtt.h"
-static int
-i915_get_ggtt_vma_pages(struct i915_vma *vma);
-
static void i915_ggtt_color_adjust(const struct drm_mm_node *node,
unsigned long color,
u64 *start,
@@ -104,7 +103,7 @@ static bool needs_idle_maps(struct drm_i915_private *i915)
* Query intel_iommu to see if we need the workaround. Presumably that
* was loaded first.
*/
- if (!intel_vtd_active())
+ if (!intel_vtd_active(i915))
return false;
if (GRAPHICS_VER(i915) == 5 && IS_MOBILE(i915))
@@ -116,17 +115,26 @@ static bool needs_idle_maps(struct drm_i915_private *i915)
return false;
}
-void i915_ggtt_suspend(struct i915_ggtt *ggtt)
+/**
+ * i915_ggtt_suspend_vm - Suspend the memory mappings for a GGTT or DPT VM
+ * @vm: The VM to suspend the mappings for
+ *
+ * Suspend the memory mappings for all objects mapped to HW via the GGTT or a
+ * DPT page table.
+ */
+void i915_ggtt_suspend_vm(struct i915_address_space *vm)
{
struct i915_vma *vma, *vn;
int open;
- mutex_lock(&ggtt->vm.mutex);
+ drm_WARN_ON(&vm->i915->drm, !vm->is_ggtt && !vm->is_dpt);
+
+ mutex_lock(&vm->mutex);
/* Skip rewriting PTE on VMA unbind. */
- open = atomic_xchg(&ggtt->vm.open, 0);
+ open = atomic_xchg(&vm->open, 0);
- list_for_each_entry_safe(vma, vn, &ggtt->vm.bound_list, vm_link) {
+ list_for_each_entry_safe(vma, vn, &vm->bound_list, vm_link) {
GEM_BUG_ON(!drm_mm_node_allocated(&vma->node));
i915_vma_wait_for_bind(vma);
@@ -139,11 +147,17 @@ void i915_ggtt_suspend(struct i915_ggtt *ggtt)
}
}
- ggtt->vm.clear_range(&ggtt->vm, 0, ggtt->vm.total);
- ggtt->invalidate(ggtt);
- atomic_set(&ggtt->vm.open, open);
+ vm->clear_range(vm, 0, vm->total);
- mutex_unlock(&ggtt->vm.mutex);
+ atomic_set(&vm->open, open);
+
+ mutex_unlock(&vm->mutex);
+}
+
+void i915_ggtt_suspend(struct i915_ggtt *ggtt)
+{
+ i915_ggtt_suspend_vm(&ggtt->vm);
+ ggtt->invalidate(ggtt);
intel_gt_check_and_clear_faults(ggtt->vm.gt);
}
@@ -192,7 +206,7 @@ u64 gen8_ggtt_pte_encode(dma_addr_t addr,
enum i915_cache_level level,
u32 flags)
{
- gen8_pte_t pte = addr | _PAGE_PRESENT;
+ gen8_pte_t pte = addr | GEN8_PAGE_PRESENT;
if (flags & PTE_LM)
pte |= GEN12_GGTT_PTE_LM;
@@ -875,21 +889,6 @@ static int ggtt_probe_common(struct i915_ggtt *ggtt, u64 size)
return 0;
}
-int ggtt_set_pages(struct i915_vma *vma)
-{
- int ret;
-
- GEM_BUG_ON(vma->pages);
-
- ret = i915_get_ggtt_vma_pages(vma);
- if (ret)
- return ret;
-
- vma->page_sizes = vma->obj->mm.page_sizes;
-
- return 0;
-}
-
static void gen6_gmch_remove(struct i915_address_space *vm)
{
struct i915_ggtt *ggtt = i915_vm_to_ggtt(vm);
@@ -924,6 +923,7 @@ static int gen8_gmch_probe(struct i915_ggtt *ggtt)
size = gen8_get_total_gtt_size(snb_gmch_ctl);
ggtt->vm.alloc_pt_dma = alloc_pt_dma;
+ ggtt->vm.alloc_scratch_dma = alloc_pt_dma;
ggtt->vm.lmem_pt_obj_flags = I915_BO_ALLOC_PM_EARLY;
ggtt->vm.total = (size / sizeof(gen8_pte_t)) * I915_GTT_PAGE_SIZE;
@@ -950,8 +950,6 @@ static int gen8_gmch_probe(struct i915_ggtt *ggtt)
ggtt->vm.vma_ops.bind_vma = ggtt_bind_vma;
ggtt->vm.vma_ops.unbind_vma = ggtt_unbind_vma;
- ggtt->vm.vma_ops.set_pages = ggtt_set_pages;
- ggtt->vm.vma_ops.clear_pages = clear_pages;
ggtt->vm.pte_encode = gen8_ggtt_pte_encode;
@@ -1077,6 +1075,7 @@ static int gen6_gmch_probe(struct i915_ggtt *ggtt)
ggtt->vm.total = (size / sizeof(gen6_pte_t)) * I915_GTT_PAGE_SIZE;
ggtt->vm.alloc_pt_dma = alloc_pt_dma;
+ ggtt->vm.alloc_scratch_dma = alloc_pt_dma;
ggtt->vm.clear_range = nop_clear_range;
if (!HAS_FULL_PPGTT(i915) || intel_scanout_needs_vtd_wa(i915))
@@ -1100,8 +1099,6 @@ static int gen6_gmch_probe(struct i915_ggtt *ggtt)
ggtt->vm.vma_ops.bind_vma = ggtt_bind_vma;
ggtt->vm.vma_ops.unbind_vma = ggtt_unbind_vma;
- ggtt->vm.vma_ops.set_pages = ggtt_set_pages;
- ggtt->vm.vma_ops.clear_pages = clear_pages;
return ggtt_probe_common(ggtt, size);
}
@@ -1129,6 +1126,7 @@ static int i915_gmch_probe(struct i915_ggtt *ggtt)
(struct resource)DEFINE_RES_MEM(gmadr_base, ggtt->mappable_end);
ggtt->vm.alloc_pt_dma = alloc_pt_dma;
+ ggtt->vm.alloc_scratch_dma = alloc_pt_dma;
if (needs_idle_maps(i915)) {
drm_notice(&i915->drm,
@@ -1145,8 +1143,6 @@ static int i915_gmch_probe(struct i915_ggtt *ggtt)
ggtt->vm.vma_ops.bind_vma = ggtt_bind_vma;
ggtt->vm.vma_ops.unbind_vma = ggtt_unbind_vma;
- ggtt->vm.vma_ops.set_pages = ggtt_set_pages;
- ggtt->vm.vma_ops.clear_pages = clear_pages;
if (unlikely(ggtt->do_idle_maps))
drm_notice(&i915->drm,
@@ -1212,11 +1208,11 @@ int i915_ggtt_probe_hw(struct drm_i915_private *i915)
{
int ret;
- ret = ggtt_probe_hw(&i915->ggtt, &i915->gt);
+ ret = ggtt_probe_hw(&i915->ggtt, to_gt(i915));
if (ret)
return ret;
- if (intel_vtd_active())
+ if (intel_vtd_active(i915))
drm_info(&i915->drm, "VT-d active for gfx access\n");
return 0;
@@ -1253,37 +1249,59 @@ void i915_ggtt_disable_guc(struct i915_ggtt *ggtt)
ggtt->invalidate(ggtt);
}
-void i915_ggtt_resume(struct i915_ggtt *ggtt)
+/**
+ * i915_ggtt_resume_vm - Restore the memory mappings for a GGTT or DPT VM
+ * @vm: The VM to restore the mappings for
+ *
+ * Restore the memory mappings for all objects mapped to HW via the GGTT or a
+ * DPT page table.
+ *
+ * Returns %true if restoring the mapping for any object that was in a write
+ * domain before suspend.
+ */
+bool i915_ggtt_resume_vm(struct i915_address_space *vm)
{
struct i915_vma *vma;
- bool flush = false;
+ bool write_domain_objs = false;
int open;
- intel_gt_check_and_clear_faults(ggtt->vm.gt);
+ drm_WARN_ON(&vm->i915->drm, !vm->is_ggtt && !vm->is_dpt);
/* First fill our portion of the GTT with scratch pages */
- ggtt->vm.clear_range(&ggtt->vm, 0, ggtt->vm.total);
+ vm->clear_range(vm, 0, vm->total);
/* Skip rewriting PTE on VMA unbind. */
- open = atomic_xchg(&ggtt->vm.open, 0);
+ open = atomic_xchg(&vm->open, 0);
/* clflush objects bound into the GGTT and rebind them. */
- list_for_each_entry(vma, &ggtt->vm.bound_list, vm_link) {
+ list_for_each_entry(vma, &vm->bound_list, vm_link) {
struct drm_i915_gem_object *obj = vma->obj;
unsigned int was_bound =
atomic_read(&vma->flags) & I915_VMA_BIND_MASK;
GEM_BUG_ON(!was_bound);
- vma->ops->bind_vma(&ggtt->vm, NULL, vma,
+ vma->ops->bind_vma(vm, NULL, vma,
obj ? obj->cache_level : 0,
was_bound);
if (obj) { /* only used during resume => exclusive access */
- flush |= fetch_and_zero(&obj->write_domain);
+ write_domain_objs |= fetch_and_zero(&obj->write_domain);
obj->read_domains |= I915_GEM_DOMAIN_GTT;
}
}
- atomic_set(&ggtt->vm.open, open);
+ atomic_set(&vm->open, open);
+
+ return write_domain_objs;
+}
+
+void i915_ggtt_resume(struct i915_ggtt *ggtt)
+{
+ bool flush;
+
+ intel_gt_check_and_clear_faults(ggtt->vm.gt);
+
+ flush = i915_ggtt_resume_vm(&ggtt->vm);
+
ggtt->invalidate(ggtt);
if (flush)
@@ -1294,327 +1312,3 @@ void i915_ggtt_resume(struct i915_ggtt *ggtt)
intel_ggtt_restore_fences(ggtt);
}
-
-static struct scatterlist *
-rotate_pages(struct drm_i915_gem_object *obj, unsigned int offset,
- unsigned int width, unsigned int height,
- unsigned int src_stride, unsigned int dst_stride,
- struct sg_table *st, struct scatterlist *sg)
-{
- unsigned int column, row;
- unsigned int src_idx;
-
- for (column = 0; column < width; column++) {
- unsigned int left;
-
- src_idx = src_stride * (height - 1) + column + offset;
- for (row = 0; row < height; row++) {
- st->nents++;
- /*
- * We don't need the pages, but need to initialize
- * the entries so the sg list can be happily traversed.
- * The only thing we need are DMA addresses.
- */
- sg_set_page(sg, NULL, I915_GTT_PAGE_SIZE, 0);
- sg_dma_address(sg) =
- i915_gem_object_get_dma_address(obj, src_idx);
- sg_dma_len(sg) = I915_GTT_PAGE_SIZE;
- sg = sg_next(sg);
- src_idx -= src_stride;
- }
-
- left = (dst_stride - height) * I915_GTT_PAGE_SIZE;
-
- if (!left)
- continue;
-
- st->nents++;
-
- /*
- * The DE ignores the PTEs for the padding tiles, the sg entry
- * here is just a conenience to indicate how many padding PTEs
- * to insert at this spot.
- */
- sg_set_page(sg, NULL, left, 0);
- sg_dma_address(sg) = 0;
- sg_dma_len(sg) = left;
- sg = sg_next(sg);
- }
-
- return sg;
-}
-
-static noinline struct sg_table *
-intel_rotate_pages(struct intel_rotation_info *rot_info,
- struct drm_i915_gem_object *obj)
-{
- unsigned int size = intel_rotation_info_size(rot_info);
- struct drm_i915_private *i915 = to_i915(obj->base.dev);
- struct sg_table *st;
- struct scatterlist *sg;
- int ret = -ENOMEM;
- int i;
-
- /* Allocate target SG list. */
- st = kmalloc(sizeof(*st), GFP_KERNEL);
- if (!st)
- goto err_st_alloc;
-
- ret = sg_alloc_table(st, size, GFP_KERNEL);
- if (ret)
- goto err_sg_alloc;
-
- st->nents = 0;
- sg = st->sgl;
-
- for (i = 0 ; i < ARRAY_SIZE(rot_info->plane); i++)
- sg = rotate_pages(obj, rot_info->plane[i].offset,
- rot_info->plane[i].width, rot_info->plane[i].height,
- rot_info->plane[i].src_stride,
- rot_info->plane[i].dst_stride,
- st, sg);
-
- return st;
-
-err_sg_alloc:
- kfree(st);
-err_st_alloc:
-
- drm_dbg(&i915->drm, "Failed to create rotated mapping for object size %zu! (%ux%u tiles, %u pages)\n",
- obj->base.size, rot_info->plane[0].width,
- rot_info->plane[0].height, size);
-
- return ERR_PTR(ret);
-}
-
-static struct scatterlist *
-remap_pages(struct drm_i915_gem_object *obj,
- unsigned int offset, unsigned int alignment_pad,
- unsigned int width, unsigned int height,
- unsigned int src_stride, unsigned int dst_stride,
- struct sg_table *st, struct scatterlist *sg)
-{
- unsigned int row;
-
- if (!width || !height)
- return sg;
-
- if (alignment_pad) {
- st->nents++;
-
- /*
- * The DE ignores the PTEs for the padding tiles, the sg entry
- * here is just a convenience to indicate how many padding PTEs
- * to insert at this spot.
- */
- sg_set_page(sg, NULL, alignment_pad * 4096, 0);
- sg_dma_address(sg) = 0;
- sg_dma_len(sg) = alignment_pad * 4096;
- sg = sg_next(sg);
- }
-
- for (row = 0; row < height; row++) {
- unsigned int left = width * I915_GTT_PAGE_SIZE;
-
- while (left) {
- dma_addr_t addr;
- unsigned int length;
-
- /*
- * We don't need the pages, but need to initialize
- * the entries so the sg list can be happily traversed.
- * The only thing we need are DMA addresses.
- */
-
- addr = i915_gem_object_get_dma_address_len(obj, offset, &length);
-
- length = min(left, length);
-
- st->nents++;
-
- sg_set_page(sg, NULL, length, 0);
- sg_dma_address(sg) = addr;
- sg_dma_len(sg) = length;
- sg = sg_next(sg);
-
- offset += length / I915_GTT_PAGE_SIZE;
- left -= length;
- }
-
- offset += src_stride - width;
-
- left = (dst_stride - width) * I915_GTT_PAGE_SIZE;
-
- if (!left)
- continue;
-
- st->nents++;
-
- /*
- * The DE ignores the PTEs for the padding tiles, the sg entry
- * here is just a conenience to indicate how many padding PTEs
- * to insert at this spot.
- */
- sg_set_page(sg, NULL, left, 0);
- sg_dma_address(sg) = 0;
- sg_dma_len(sg) = left;
- sg = sg_next(sg);
- }
-
- return sg;
-}
-
-static noinline struct sg_table *
-intel_remap_pages(struct intel_remapped_info *rem_info,
- struct drm_i915_gem_object *obj)
-{
- unsigned int size = intel_remapped_info_size(rem_info);
- struct drm_i915_private *i915 = to_i915(obj->base.dev);
- struct sg_table *st;
- struct scatterlist *sg;
- unsigned int gtt_offset = 0;
- int ret = -ENOMEM;
- int i;
-
- /* Allocate target SG list. */
- st = kmalloc(sizeof(*st), GFP_KERNEL);
- if (!st)
- goto err_st_alloc;
-
- ret = sg_alloc_table(st, size, GFP_KERNEL);
- if (ret)
- goto err_sg_alloc;
-
- st->nents = 0;
- sg = st->sgl;
-
- for (i = 0 ; i < ARRAY_SIZE(rem_info->plane); i++) {
- unsigned int alignment_pad = 0;
-
- if (rem_info->plane_alignment)
- alignment_pad = ALIGN(gtt_offset, rem_info->plane_alignment) - gtt_offset;
-
- sg = remap_pages(obj,
- rem_info->plane[i].offset, alignment_pad,
- rem_info->plane[i].width, rem_info->plane[i].height,
- rem_info->plane[i].src_stride, rem_info->plane[i].dst_stride,
- st, sg);
-
- gtt_offset += alignment_pad +
- rem_info->plane[i].dst_stride * rem_info->plane[i].height;
- }
-
- i915_sg_trim(st);
-
- return st;
-
-err_sg_alloc:
- kfree(st);
-err_st_alloc:
-
- drm_dbg(&i915->drm, "Failed to create remapped mapping for object size %zu! (%ux%u tiles, %u pages)\n",
- obj->base.size, rem_info->plane[0].width,
- rem_info->plane[0].height, size);
-
- return ERR_PTR(ret);
-}
-
-static noinline struct sg_table *
-intel_partial_pages(const struct i915_ggtt_view *view,
- struct drm_i915_gem_object *obj)
-{
- struct sg_table *st;
- struct scatterlist *sg, *iter;
- unsigned int count = view->partial.size;
- unsigned int offset;
- int ret = -ENOMEM;
-
- st = kmalloc(sizeof(*st), GFP_KERNEL);
- if (!st)
- goto err_st_alloc;
-
- ret = sg_alloc_table(st, count, GFP_KERNEL);
- if (ret)
- goto err_sg_alloc;
-
- iter = i915_gem_object_get_sg_dma(obj, view->partial.offset, &offset);
- GEM_BUG_ON(!iter);
-
- sg = st->sgl;
- st->nents = 0;
- do {
- unsigned int len;
-
- len = min(sg_dma_len(iter) - (offset << PAGE_SHIFT),
- count << PAGE_SHIFT);
- sg_set_page(sg, NULL, len, 0);
- sg_dma_address(sg) =
- sg_dma_address(iter) + (offset << PAGE_SHIFT);
- sg_dma_len(sg) = len;
-
- st->nents++;
- count -= len >> PAGE_SHIFT;
- if (count == 0) {
- sg_mark_end(sg);
- i915_sg_trim(st); /* Drop any unused tail entries. */
-
- return st;
- }
-
- sg = __sg_next(sg);
- iter = __sg_next(iter);
- offset = 0;
- } while (1);
-
-err_sg_alloc:
- kfree(st);
-err_st_alloc:
- return ERR_PTR(ret);
-}
-
-static int
-i915_get_ggtt_vma_pages(struct i915_vma *vma)
-{
- int ret;
-
- /*
- * The vma->pages are only valid within the lifespan of the borrowed
- * obj->mm.pages. When the obj->mm.pages sg_table is regenerated, so
- * must be the vma->pages. A simple rule is that vma->pages must only
- * be accessed when the obj->mm.pages are pinned.
- */
- GEM_BUG_ON(!i915_gem_object_has_pinned_pages(vma->obj));
-
- switch (vma->ggtt_view.type) {
- default:
- GEM_BUG_ON(vma->ggtt_view.type);
- fallthrough;
- case I915_GGTT_VIEW_NORMAL:
- vma->pages = vma->obj->mm.pages;
- return 0;
-
- case I915_GGTT_VIEW_ROTATED:
- vma->pages =
- intel_rotate_pages(&vma->ggtt_view.rotated, vma->obj);
- break;
-
- case I915_GGTT_VIEW_REMAPPED:
- vma->pages =
- intel_remap_pages(&vma->ggtt_view.remapped, vma->obj);
- break;
-
- case I915_GGTT_VIEW_PARTIAL:
- vma->pages = intel_partial_pages(&vma->ggtt_view, vma->obj);
- break;
- }
-
- ret = 0;
- if (IS_ERR(vma->pages)) {
- ret = PTR_ERR(vma->pages);
- vma->pages = NULL;
- drm_err(&vma->vm->i915->drm,
- "Failed to get pages for VMA view type %u (%d)!\n",
- vma->ggtt_view.type, ret);
- }
- return ret;
-}
diff --git a/drivers/gpu/drm/i915/gt/intel_gt.c b/drivers/gpu/drm/i915/gt/intel_gt.c
index 1cb1948..f98f0fb 100644
--- a/drivers/gpu/drm/i915/gt/intel_gt.c
+++ b/drivers/gpu/drm/i915/gt/intel_gt.c
@@ -3,6 +3,8 @@
* Copyright © 2019 Intel Corporation
*/
+#include <drm/intel-gtt.h>
+
#include "intel_gt_debugfs.h"
#include "gem/i915_gem_lmem.h"
@@ -23,11 +25,8 @@
#include "shmem_utils.h"
#include "pxp/intel_pxp.h"
-void intel_gt_init_early(struct intel_gt *gt, struct drm_i915_private *i915)
+void __intel_gt_init_early(struct intel_gt *gt, struct drm_i915_private *i915)
{
- gt->i915 = i915;
- gt->uncore = &i915->uncore;
-
spin_lock_init(>->irq_lock);
INIT_LIST_HEAD(>->closed_vma);
@@ -46,6 +45,12 @@ void intel_gt_init_early(struct intel_gt *gt, struct drm_i915_private *i915)
intel_rps_init_early(>->rps);
}
+void intel_gt_init_early(struct intel_gt *gt, struct drm_i915_private *i915)
+{
+ gt->i915 = i915;
+ gt->uncore = &i915->uncore;
+}
+
int intel_gt_probe_lmem(struct intel_gt *gt)
{
struct drm_i915_private *i915 = gt->i915;
diff --git a/drivers/gpu/drm/i915/gt/intel_gt.h b/drivers/gpu/drm/i915/gt/intel_gt.h
index 74e7718..3ace129 100644
--- a/drivers/gpu/drm/i915/gt/intel_gt.h
+++ b/drivers/gpu/drm/i915/gt/intel_gt.h
@@ -35,6 +35,7 @@ static inline struct intel_gt *huc_to_gt(struct intel_huc *huc)
}
void intel_gt_init_early(struct intel_gt *gt, struct drm_i915_private *i915);
+void __intel_gt_init_early(struct intel_gt *gt, struct drm_i915_private *i915);
void intel_gt_init_hw_early(struct intel_gt *gt, struct i915_ggtt *ggtt);
int intel_gt_probe_lmem(struct intel_gt *gt);
int intel_gt_init_mmio(struct intel_gt *gt);
diff --git a/drivers/gpu/drm/i915/gt/intel_gt_buffer_pool.c b/drivers/gpu/drm/i915/gt/intel_gt_buffer_pool.c
index acc49c5..9db3dcb 100644
--- a/drivers/gpu/drm/i915/gt/intel_gt_buffer_pool.c
+++ b/drivers/gpu/drm/i915/gt/intel_gt_buffer_pool.c
@@ -9,11 +9,6 @@
#include "intel_engine_pm.h"
#include "intel_gt_buffer_pool.h"
-static struct intel_gt *to_gt(struct intel_gt_buffer_pool *pool)
-{
- return container_of(pool, struct intel_gt, buffer_pool);
-}
-
static struct list_head *
bucket_for_size(struct intel_gt_buffer_pool *pool, size_t sz)
{
@@ -141,7 +136,7 @@ static struct intel_gt_buffer_pool_node *
node_create(struct intel_gt_buffer_pool *pool, size_t sz,
enum i915_map_type type)
{
- struct intel_gt *gt = to_gt(pool);
+ struct intel_gt *gt = container_of(pool, struct intel_gt, buffer_pool);
struct intel_gt_buffer_pool_node *node;
struct drm_i915_gem_object *obj;
diff --git a/drivers/gpu/drm/i915/gt/intel_gt_debugfs.h b/drivers/gpu/drm/i915/gt/intel_gt_debugfs.h
index e307ceb..17e79b7 100644
--- a/drivers/gpu/drm/i915/gt/intel_gt_debugfs.h
+++ b/drivers/gpu/drm/i915/gt/intel_gt_debugfs.h
@@ -10,11 +10,7 @@
struct intel_gt;
-#define DEFINE_INTEL_GT_DEBUGFS_ATTRIBUTE(__name) \
- static int __name ## _open(struct inode *inode, struct file *file) \
-{ \
- return single_open(file, __name ## _show, inode->i_private); \
-} \
+#define __GT_DEBUGFS_ATTRIBUTE_FOPS(__name) \
static const struct file_operations __name ## _fops = { \
.owner = THIS_MODULE, \
.open = __name ## _open, \
@@ -23,6 +19,21 @@ static const struct file_operations __name ## _fops = { \
.release = single_release, \
}
+#define DEFINE_INTEL_GT_DEBUGFS_ATTRIBUTE(__name) \
+static int __name ## _open(struct inode *inode, struct file *file) \
+{ \
+ return single_open(file, __name ## _show, inode->i_private); \
+} \
+__GT_DEBUGFS_ATTRIBUTE_FOPS(__name)
+
+#define DEFINE_INTEL_GT_DEBUGFS_ATTRIBUTE_WITH_SIZE(__name, __size_vf) \
+static int __name ## _open(struct inode *inode, struct file *file) \
+{ \
+ return single_open_size(file, __name ## _show, inode->i_private, \
+ __size_vf(inode->i_private)); \
+} \
+__GT_DEBUGFS_ATTRIBUTE_FOPS(__name)
+
void intel_gt_debugfs_register(struct intel_gt *gt);
struct intel_gt_debugfs_file {
diff --git a/drivers/gpu/drm/i915/gt/intel_gt_pm.c b/drivers/gpu/drm/i915/gt/intel_gt_pm.c
index 795689e..c0fa41e 100644
--- a/drivers/gpu/drm/i915/gt/intel_gt_pm.c
+++ b/drivers/gpu/drm/i915/gt/intel_gt_pm.c
@@ -86,6 +86,7 @@ static int __gt_unpark(struct intel_wakeref *wf)
intel_rc6_unpark(>->rc6);
intel_rps_unpark(>->rps);
i915_pmu_gt_unparked(i915);
+ intel_guc_busyness_unpark(gt);
intel_gt_unpark_requests(gt);
runtime_begin(gt);
@@ -104,6 +105,7 @@ static int __gt_park(struct intel_wakeref *wf)
runtime_end(gt);
intel_gt_park_requests(gt);
+ intel_guc_busyness_park(gt);
i915_vma_parked(gt);
i915_pmu_gt_parked(i915);
intel_rps_park(>->rps);
diff --git a/drivers/gpu/drm/i915/gt/intel_gtt.c b/drivers/gpu/drm/i915/gt/intel_gtt.c
index b67f620..a94be03 100644
--- a/drivers/gpu/drm/i915/gt/intel_gtt.c
+++ b/drivers/gpu/drm/i915/gt/intel_gtt.c
@@ -8,6 +8,8 @@
#include <linux/fault-inject.h>
#include <linux/sched/mm.h>
+#include <drm/drm_cache.h>
+
#include "gem/i915_gem_lmem.h"
#include "i915_trace.h"
#include "intel_gt.h"
@@ -222,19 +224,6 @@ void i915_address_space_init(struct i915_address_space *vm, int subclass)
INIT_LIST_HEAD(&vm->bound_list);
}
-void clear_pages(struct i915_vma *vma)
-{
- GEM_BUG_ON(!vma->pages);
-
- if (vma->pages != vma->obj->mm.pages) {
- sg_free_table(vma->pages);
- kfree(vma->pages);
- }
- vma->pages = NULL;
-
- memset(&vma->page_sizes, 0, sizeof(vma->page_sizes));
-}
-
void *__px_vaddr(struct drm_i915_gem_object *p)
{
enum i915_map_type type;
@@ -274,6 +263,7 @@ static void poison_scratch_page(struct drm_i915_gem_object *scratch)
val = POISON_FREE;
memset(vaddr, val, scratch->base.size);
+ drm_clflush_virt_range(vaddr, scratch->base.size);
}
int setup_scratch_page(struct i915_address_space *vm)
@@ -299,7 +289,7 @@ int setup_scratch_page(struct i915_address_space *vm)
do {
struct drm_i915_gem_object *obj;
- obj = vm->alloc_pt_dma(vm, size);
+ obj = vm->alloc_scratch_dma(vm, size);
if (IS_ERR(obj))
goto skip;
@@ -335,6 +325,18 @@ int setup_scratch_page(struct i915_address_space *vm)
if (size == I915_GTT_PAGE_SIZE_4K)
return -ENOMEM;
+ /*
+ * If we need 64K minimum GTT pages for device local-memory,
+ * like on XEHPSDV, then we need to fail the allocation here,
+ * otherwise we can't safely support the insertion of
+ * local-memory pages for this vm, since the HW expects the
+ * correct physical alignment and size when the page-table is
+ * operating in 64K GTT mode, which includes any scratch PTEs,
+ * since userspace can still touch them.
+ */
+ if (HAS_64K_PAGES(vm->i915))
+ return -ENOMEM;
+
size = I915_GTT_PAGE_SIZE_4K;
} while (1);
}
diff --git a/drivers/gpu/drm/i915/gt/intel_gtt.h b/drivers/gpu/drm/i915/gt/intel_gtt.h
index bc67502..177b42b 100644
--- a/drivers/gpu/drm/i915/gt/intel_gtt.h
+++ b/drivers/gpu/drm/i915/gt/intel_gtt.h
@@ -135,6 +135,9 @@ typedef u64 gen8_pte_t;
#define GEN8_PPAT_ELLC_OVERRIDE (0<<2)
#define GEN8_PPAT(i, x) ((u64)(x) << ((i) * 8))
+#define GEN8_PAGE_PRESENT BIT_ULL(0)
+#define GEN8_PAGE_RW BIT_ULL(1)
+
#define GEN8_PDE_IPS_64K BIT(11)
#define GEN8_PDE_PS_2M BIT(7)
@@ -206,9 +209,6 @@ struct i915_vma_ops {
*/
void (*unbind_vma)(struct i915_address_space *vm,
struct i915_vma *vma);
-
- int (*set_pages)(struct i915_vma *vma);
- void (*clear_pages)(struct i915_vma *vma);
};
struct i915_address_space {
@@ -265,6 +265,8 @@ struct i915_address_space {
struct drm_i915_gem_object *
(*alloc_pt_dma)(struct i915_address_space *vm, int sz);
+ struct drm_i915_gem_object *
+ (*alloc_scratch_dma)(struct i915_address_space *vm, int sz);
u64 (*pte_encode)(dma_addr_t addr,
enum i915_cache_level level,
@@ -544,6 +546,8 @@ int i915_ppgtt_init_hw(struct intel_gt *gt);
struct i915_ppgtt *i915_ppgtt_create(struct intel_gt *gt,
unsigned long lmem_pt_obj_flags);
+void i915_ggtt_suspend_vm(struct i915_address_space *vm);
+bool i915_ggtt_resume_vm(struct i915_address_space *vm);
void i915_ggtt_suspend(struct i915_ggtt *gtt);
void i915_ggtt_resume(struct i915_ggtt *ggtt);
@@ -594,10 +598,6 @@ release_pd_entry(struct i915_page_directory * const pd,
const struct drm_i915_gem_object * const scratch);
void gen6_ggtt_invalidate(struct i915_ggtt *ggtt);
-int ggtt_set_pages(struct i915_vma *vma);
-int ppgtt_set_pages(struct i915_vma *vma);
-void clear_pages(struct i915_vma *vma);
-
void ppgtt_bind_vma(struct i915_address_space *vm,
struct i915_vm_pt_stash *stash,
struct i915_vma *vma,
diff --git a/drivers/gpu/drm/i915/gt/intel_lrc.c b/drivers/gpu/drm/i915/gt/intel_lrc.c
index 56156cf..b348959 100644
--- a/drivers/gpu/drm/i915/gt/intel_lrc.c
+++ b/drivers/gpu/drm/i915/gt/intel_lrc.c
@@ -1167,6 +1167,11 @@ gen12_emit_indirect_ctx_rcs(const struct intel_context *ce, u32 *cs)
cs = gen12_emit_cmd_buf_wa(ce, cs);
cs = gen12_emit_restore_scratch(ce, cs);
+ /* Wa_16013000631:dg2 */
+ if (IS_DG2_GRAPHICS_STEP(ce->engine->i915, G10, STEP_B0, STEP_C0) ||
+ IS_DG2_G11(ce->engine->i915))
+ cs = gen8_emit_pipe_control(cs, PIPE_CONTROL_INSTRUCTION_CACHE_INVALIDATE, 0);
+
return cs;
}
diff --git a/drivers/gpu/drm/i915/gt/intel_migrate.c b/drivers/gpu/drm/i915/gt/intel_migrate.c
index afb1cce..18b44af 100644
--- a/drivers/gpu/drm/i915/gt/intel_migrate.c
+++ b/drivers/gpu/drm/i915/gt/intel_migrate.c
@@ -13,7 +13,6 @@
struct insert_pte_data {
u64 offset;
- bool is_lmem;
};
#define CHUNK_SZ SZ_8M /* ~1ms at 8GiB/s preemption delay */
@@ -40,7 +39,7 @@ static void insert_pte(struct i915_address_space *vm,
struct insert_pte_data *d = data;
vm->insert_page(vm, px_dma(pt), d->offset, I915_CACHE_NONE,
- d->is_lmem ? PTE_LM : 0);
+ i915_gem_object_is_lmem(pt->base) ? PTE_LM : 0);
d->offset += PAGE_SIZE;
}
@@ -134,8 +133,7 @@ static struct i915_address_space *migrate_vm(struct intel_gt *gt)
goto err_vm;
/* Now allow the GPU to rewrite the PTE via its own ppGTT */
- d.is_lmem = i915_gem_object_is_lmem(vm->vm.scratch[0]);
- vm->vm.foreach(&vm->vm, base, base + sz, insert_pte, &d);
+ vm->vm.foreach(&vm->vm, base, d.offset - base, insert_pte, &d);
}
return &vm->vm;
@@ -281,10 +279,10 @@ static int emit_pte(struct i915_request *rq,
GEM_BUG_ON(GRAPHICS_VER(rq->engine->i915) < 8);
/* Compute the page directory offset for the target address range */
- offset += (u64)rq->engine->instance << 32;
offset >>= 12;
offset *= sizeof(u64);
offset += 2 * CHUNK_SZ;
+ offset += (u64)rq->engine->instance << 32;
cs = intel_ring_begin(rq, 6);
if (IS_ERR(cs))
@@ -406,7 +404,7 @@ static int emit_copy(struct i915_request *rq, int size)
int
intel_context_migrate_copy(struct intel_context *ce,
- struct dma_fence *await,
+ const struct i915_deps *deps,
struct scatterlist *src,
enum i915_cache_level src_cache_level,
bool src_is_lmem,
@@ -433,8 +431,8 @@ intel_context_migrate_copy(struct intel_context *ce,
goto out_ce;
}
- if (await) {
- err = i915_request_await_dma_fence(rq, await);
+ if (deps) {
+ err = i915_request_await_deps(rq, deps);
if (err)
goto out_rq;
@@ -444,7 +442,7 @@ intel_context_migrate_copy(struct intel_context *ce,
goto out_rq;
}
- await = NULL;
+ deps = NULL;
}
/* The PTE updates + copy must not be interrupted. */
@@ -527,7 +525,7 @@ static int emit_clear(struct i915_request *rq, int size, u32 value)
int
intel_context_migrate_clear(struct intel_context *ce,
- struct dma_fence *await,
+ const struct i915_deps *deps,
struct scatterlist *sg,
enum i915_cache_level cache_level,
bool is_lmem,
@@ -552,8 +550,8 @@ intel_context_migrate_clear(struct intel_context *ce,
goto out_ce;
}
- if (await) {
- err = i915_request_await_dma_fence(rq, await);
+ if (deps) {
+ err = i915_request_await_deps(rq, deps);
if (err)
goto out_rq;
@@ -563,7 +561,7 @@ intel_context_migrate_clear(struct intel_context *ce,
goto out_rq;
}
- await = NULL;
+ deps = NULL;
}
/* The PTE updates + clear must not be interrupted. */
@@ -601,7 +599,7 @@ intel_context_migrate_clear(struct intel_context *ce,
int intel_migrate_copy(struct intel_migrate *m,
struct i915_gem_ww_ctx *ww,
- struct dma_fence *await,
+ const struct i915_deps *deps,
struct scatterlist *src,
enum i915_cache_level src_cache_level,
bool src_is_lmem,
@@ -626,7 +624,7 @@ int intel_migrate_copy(struct intel_migrate *m,
if (err)
goto out;
- err = intel_context_migrate_copy(ce, await,
+ err = intel_context_migrate_copy(ce, deps,
src, src_cache_level, src_is_lmem,
dst, dst_cache_level, dst_is_lmem,
out);
@@ -640,7 +638,7 @@ int intel_migrate_copy(struct intel_migrate *m,
int
intel_migrate_clear(struct intel_migrate *m,
struct i915_gem_ww_ctx *ww,
- struct dma_fence *await,
+ const struct i915_deps *deps,
struct scatterlist *sg,
enum i915_cache_level cache_level,
bool is_lmem,
@@ -663,7 +661,7 @@ intel_migrate_clear(struct intel_migrate *m,
if (err)
goto out;
- err = intel_context_migrate_clear(ce, await, sg, cache_level,
+ err = intel_context_migrate_clear(ce, deps, sg, cache_level,
is_lmem, value, out);
intel_context_unpin(ce);
diff --git a/drivers/gpu/drm/i915/gt/intel_migrate.h b/drivers/gpu/drm/i915/gt/intel_migrate.h
index 4e18e75..ccc677e 100644
--- a/drivers/gpu/drm/i915/gt/intel_migrate.h
+++ b/drivers/gpu/drm/i915/gt/intel_migrate.h
@@ -11,6 +11,7 @@
#include "intel_migrate_types.h"
struct dma_fence;
+struct i915_deps;
struct i915_request;
struct i915_gem_ww_ctx;
struct intel_gt;
@@ -23,7 +24,7 @@ struct intel_context *intel_migrate_create_context(struct intel_migrate *m);
int intel_migrate_copy(struct intel_migrate *m,
struct i915_gem_ww_ctx *ww,
- struct dma_fence *await,
+ const struct i915_deps *deps,
struct scatterlist *src,
enum i915_cache_level src_cache_level,
bool src_is_lmem,
@@ -33,7 +34,7 @@ int intel_migrate_copy(struct intel_migrate *m,
struct i915_request **out);
int intel_context_migrate_copy(struct intel_context *ce,
- struct dma_fence *await,
+ const struct i915_deps *deps,
struct scatterlist *src,
enum i915_cache_level src_cache_level,
bool src_is_lmem,
@@ -45,7 +46,7 @@ int intel_context_migrate_copy(struct intel_context *ce,
int
intel_migrate_clear(struct intel_migrate *m,
struct i915_gem_ww_ctx *ww,
- struct dma_fence *await,
+ const struct i915_deps *deps,
struct scatterlist *sg,
enum i915_cache_level cache_level,
bool is_lmem,
@@ -53,7 +54,7 @@ intel_migrate_clear(struct intel_migrate *m,
struct i915_request **out);
int
intel_context_migrate_clear(struct intel_context *ce,
- struct dma_fence *await,
+ const struct i915_deps *deps,
struct scatterlist *sg,
enum i915_cache_level cache_level,
bool is_lmem,
diff --git a/drivers/gpu/drm/i915/gt/intel_mocs.c b/drivers/gpu/drm/i915/gt/intel_mocs.c
index 15f9ada..9c253ba 100644
--- a/drivers/gpu/drm/i915/gt/intel_mocs.c
+++ b/drivers/gpu/drm/i915/gt/intel_mocs.c
@@ -424,7 +424,7 @@ static unsigned int get_mocs_settings(const struct drm_i915_private *i915,
table->unused_entries_index = I915_MOCS_PTE;
if (IS_DG2(i915)) {
- if (IS_DG2_GT_STEP(i915, G10, STEP_A0, STEP_B0)) {
+ if (IS_DG2_GRAPHICS_STEP(i915, G10, STEP_A0, STEP_B0)) {
table->size = ARRAY_SIZE(dg2_mocs_table_g10_ax);
table->table = dg2_mocs_table_g10_ax;
} else {
diff --git a/drivers/gpu/drm/i915/gt/intel_ppgtt.c b/drivers/gpu/drm/i915/gt/intel_ppgtt.c
index 4396bfd..083b309 100644
--- a/drivers/gpu/drm/i915/gt/intel_ppgtt.c
+++ b/drivers/gpu/drm/i915/gt/intel_ppgtt.c
@@ -289,16 +289,6 @@ void i915_vm_free_pt_stash(struct i915_address_space *vm,
}
}
-int ppgtt_set_pages(struct i915_vma *vma)
-{
- GEM_BUG_ON(vma->pages);
-
- vma->pages = vma->obj->mm.pages;
- vma->page_sizes = vma->obj->mm.page_sizes;
-
- return 0;
-}
-
void ppgtt_init(struct i915_ppgtt *ppgtt, struct intel_gt *gt,
unsigned long lmem_pt_obj_flags)
{
@@ -315,6 +305,4 @@ void ppgtt_init(struct i915_ppgtt *ppgtt, struct intel_gt *gt,
ppgtt->vm.vma_ops.bind_vma = ppgtt_bind_vma;
ppgtt->vm.vma_ops.unbind_vma = ppgtt_unbind_vma;
- ppgtt->vm.vma_ops.set_pages = ppgtt_set_pages;
- ppgtt->vm.vma_ops.clear_pages = clear_pages;
}
diff --git a/drivers/gpu/drm/i915/gt/intel_rc6.c b/drivers/gpu/drm/i915/gt/intel_rc6.c
index 43093dd..c3155ee 100644
--- a/drivers/gpu/drm/i915/gt/intel_rc6.c
+++ b/drivers/gpu/drm/i915/gt/intel_rc6.c
@@ -117,10 +117,17 @@ static void gen11_rc6_enable(struct intel_rc6 *rc6)
GEN6_RC_CTL_RC6_ENABLE |
GEN6_RC_CTL_EI_MODE(1);
- pg_enable =
- GEN9_RENDER_PG_ENABLE |
- GEN9_MEDIA_PG_ENABLE |
- GEN11_MEDIA_SAMPLER_PG_ENABLE;
+ /* Wa_16011777198 - Render powergating must remain disabled */
+ if (IS_DG2_GRAPHICS_STEP(gt->i915, G10, STEP_A0, STEP_C0) ||
+ IS_DG2_GRAPHICS_STEP(gt->i915, G11, STEP_A0, STEP_B0))
+ pg_enable =
+ GEN9_MEDIA_PG_ENABLE |
+ GEN11_MEDIA_SAMPLER_PG_ENABLE;
+ else
+ pg_enable =
+ GEN9_RENDER_PG_ENABLE |
+ GEN9_MEDIA_PG_ENABLE |
+ GEN11_MEDIA_SAMPLER_PG_ENABLE;
if (GRAPHICS_VER(gt->i915) >= 12) {
for (i = 0; i < I915_MAX_VCS; i++)
diff --git a/drivers/gpu/drm/i915/gt/intel_region_lmem.c b/drivers/gpu/drm/i915/gt/intel_region_lmem.c
index afb35d2..fde2dcb 100644
--- a/drivers/gpu/drm/i915/gt/intel_region_lmem.c
+++ b/drivers/gpu/drm/i915/gt/intel_region_lmem.c
@@ -66,12 +66,16 @@ static void release_fake_lmem_bar(struct intel_memory_region *mem)
DMA_ATTR_FORCE_CONTIGUOUS);
}
-static void
+static int
region_lmem_release(struct intel_memory_region *mem)
{
- intel_region_ttm_fini(mem);
+ int ret;
+
+ ret = intel_region_ttm_fini(mem);
io_mapping_fini(&mem->iomap);
release_fake_lmem_bar(mem);
+
+ return ret;
}
static int
@@ -158,7 +162,7 @@ intel_gt_setup_fake_lmem(struct intel_gt *gt)
static bool get_legacy_lowmem_region(struct intel_uncore *uncore,
u64 *start, u32 *size)
{
- if (!IS_DG1_GT_STEP(uncore->i915, STEP_A0, STEP_C0))
+ if (!IS_DG1_GRAPHICS_STEP(uncore->i915, STEP_A0, STEP_C0))
return false;
*start = 0;
@@ -193,6 +197,7 @@ static struct intel_memory_region *setup_lmem(struct intel_gt *gt)
struct intel_uncore *uncore = gt->uncore;
struct pci_dev *pdev = to_pci_dev(i915->drm.dev);
struct intel_memory_region *mem;
+ resource_size_t min_page_size;
resource_size_t io_start;
resource_size_t lmem_size;
int err;
@@ -207,10 +212,12 @@ static struct intel_memory_region *setup_lmem(struct intel_gt *gt)
if (GEM_WARN_ON(lmem_size > pci_resource_len(pdev, 2)))
return ERR_PTR(-ENODEV);
+ min_page_size = HAS_64K_PAGES(i915) ? I915_GTT_PAGE_SIZE_64K :
+ I915_GTT_PAGE_SIZE_4K;
mem = intel_memory_region_create(i915,
0,
lmem_size,
- I915_GTT_PAGE_SIZE_4K,
+ min_page_size,
io_start,
INTEL_MEMORY_LOCAL,
0,
@@ -231,7 +238,7 @@ static struct intel_memory_region *setup_lmem(struct intel_gt *gt)
return mem;
err_region_put:
- intel_memory_region_put(mem);
+ intel_memory_region_destroy(mem);
return ERR_PTR(err);
}
diff --git a/drivers/gpu/drm/i915/gt/intel_reset.c b/drivers/gpu/drm/i915/gt/intel_reset.c
index 91200c4..7be0002 100644
--- a/drivers/gpu/drm/i915/gt/intel_reset.c
+++ b/drivers/gpu/drm/i915/gt/intel_reset.c
@@ -6,7 +6,7 @@
#include <linux/sched/mm.h>
#include <linux/stop_machine.h>
-#include "display/intel_display_types.h"
+#include "display/intel_display.h"
#include "display/intel_overlay.h"
#include "gem/i915_gem_context.h"
@@ -297,13 +297,6 @@ static int gen6_reset_engines(struct intel_gt *gt,
intel_engine_mask_t engine_mask,
unsigned int retry)
{
- static const u32 hw_engine_mask[] = {
- [RCS0] = GEN6_GRDOM_RENDER,
- [BCS0] = GEN6_GRDOM_BLT,
- [VCS0] = GEN6_GRDOM_MEDIA,
- [VCS1] = GEN8_GRDOM_MEDIA2,
- [VECS0] = GEN6_GRDOM_VECS,
- };
struct intel_engine_cs *engine;
u32 hw_mask;
@@ -314,8 +307,7 @@ static int gen6_reset_engines(struct intel_gt *gt,
hw_mask = 0;
for_each_engine_masked(engine, gt, engine_mask, tmp) {
- GEM_BUG_ON(engine->id >= ARRAY_SIZE(hw_engine_mask));
- hw_mask |= hw_engine_mask[engine->id];
+ hw_mask |= engine->reset_domain;
}
}
@@ -492,22 +484,6 @@ static int gen11_reset_engines(struct intel_gt *gt,
intel_engine_mask_t engine_mask,
unsigned int retry)
{
- static const u32 hw_engine_mask[] = {
- [RCS0] = GEN11_GRDOM_RENDER,
- [BCS0] = GEN11_GRDOM_BLT,
- [VCS0] = GEN11_GRDOM_MEDIA,
- [VCS1] = GEN11_GRDOM_MEDIA2,
- [VCS2] = GEN11_GRDOM_MEDIA3,
- [VCS3] = GEN11_GRDOM_MEDIA4,
- [VCS4] = GEN11_GRDOM_MEDIA5,
- [VCS5] = GEN11_GRDOM_MEDIA6,
- [VCS6] = GEN11_GRDOM_MEDIA7,
- [VCS7] = GEN11_GRDOM_MEDIA8,
- [VECS0] = GEN11_GRDOM_VECS,
- [VECS1] = GEN11_GRDOM_VECS2,
- [VECS2] = GEN11_GRDOM_VECS3,
- [VECS3] = GEN11_GRDOM_VECS4,
- };
struct intel_engine_cs *engine;
intel_engine_mask_t tmp;
u32 reset_mask, unlock_mask = 0;
@@ -518,8 +494,7 @@ static int gen11_reset_engines(struct intel_gt *gt,
} else {
reset_mask = 0;
for_each_engine_masked(engine, gt, engine_mask, tmp) {
- GEM_BUG_ON(engine->id >= ARRAY_SIZE(hw_engine_mask));
- reset_mask |= hw_engine_mask[engine->id];
+ reset_mask |= engine->reset_domain;
ret = gen11_lock_sfc(engine, &reset_mask, &unlock_mask);
if (ret)
goto sfc_unlock;
@@ -1367,20 +1342,27 @@ void intel_gt_handle_error(struct intel_gt *gt,
/* Make sure i915_reset_trylock() sees the I915_RESET_BACKOFF */
synchronize_rcu_expedited();
- /* Prevent any other reset-engine attempt. */
- for_each_engine(engine, gt, tmp) {
- while (test_and_set_bit(I915_RESET_ENGINE + engine->id,
- >->reset.flags))
- wait_on_bit(>->reset.flags,
- I915_RESET_ENGINE + engine->id,
- TASK_UNINTERRUPTIBLE);
+ /*
+ * Prevent any other reset-engine attempt. We don't do this for GuC
+ * submission the GuC owns the per-engine reset, not the i915.
+ */
+ if (!intel_uc_uses_guc_submission(>->uc)) {
+ for_each_engine(engine, gt, tmp) {
+ while (test_and_set_bit(I915_RESET_ENGINE + engine->id,
+ >->reset.flags))
+ wait_on_bit(>->reset.flags,
+ I915_RESET_ENGINE + engine->id,
+ TASK_UNINTERRUPTIBLE);
+ }
}
intel_gt_reset_global(gt, engine_mask, msg);
- for_each_engine(engine, gt, tmp)
- clear_bit_unlock(I915_RESET_ENGINE + engine->id,
- >->reset.flags);
+ if (!intel_uc_uses_guc_submission(>->uc)) {
+ for_each_engine(engine, gt, tmp)
+ clear_bit_unlock(I915_RESET_ENGINE + engine->id,
+ >->reset.flags);
+ }
clear_bit_unlock(I915_RESET_BACKOFF, >->reset.flags);
smp_mb__after_atomic();
wake_up_all(>->reset.queue);
@@ -1441,6 +1423,7 @@ void intel_gt_set_wedged_on_init(struct intel_gt *gt)
BUILD_BUG_ON(I915_RESET_ENGINE + I915_NUM_ENGINES >
I915_WEDGED_ON_INIT);
intel_gt_set_wedged(gt);
+ i915_disable_error_state(gt->i915, -ENODEV);
set_bit(I915_WEDGED_ON_INIT, >->reset.flags);
/* Wedged on init is non-recoverable */
@@ -1450,6 +1433,7 @@ void intel_gt_set_wedged_on_init(struct intel_gt *gt)
void intel_gt_set_wedged_on_fini(struct intel_gt *gt)
{
intel_gt_set_wedged(gt);
+ i915_disable_error_state(gt->i915, -ENODEV);
set_bit(I915_WEDGED_ON_FINI, >->reset.flags);
intel_gt_retire_requests(gt); /* cleanup any wedged requests */
}
diff --git a/drivers/gpu/drm/i915/gt/intel_ring_submission.c b/drivers/gpu/drm/i915/gt/intel_ring_submission.c
index 586dca1..3e6fac0 100644
--- a/drivers/gpu/drm/i915/gt/intel_ring_submission.c
+++ b/drivers/gpu/drm/i915/gt/intel_ring_submission.c
@@ -1357,7 +1357,7 @@ int intel_ring_submission_setup(struct intel_engine_cs *engine)
err = i915_gem_object_lock(timeline->hwsp_ggtt->obj, &ww);
if (!err && gen7_wa_vma)
err = i915_gem_object_lock(gen7_wa_vma->obj, &ww);
- if (!err && engine->legacy.ring->vma->obj)
+ if (!err)
err = i915_gem_object_lock(engine->legacy.ring->vma->obj, &ww);
if (!err)
err = intel_timeline_pin(timeline, &ww);
diff --git a/drivers/gpu/drm/i915/gt/intel_rps.c b/drivers/gpu/drm/i915/gt/intel_rps.c
index 5e275f8..54e7df7 100644
--- a/drivers/gpu/drm/i915/gt/intel_rps.c
+++ b/drivers/gpu/drm/i915/gt/intel_rps.c
@@ -936,8 +936,70 @@ void intel_rps_park(struct intel_rps *rps)
GT_TRACE(rps_to_gt(rps), "park:%x\n", rps->cur_freq);
}
+u32 intel_rps_get_boost_frequency(struct intel_rps *rps)
+{
+ struct intel_guc_slpc *slpc;
+
+ if (rps_uses_slpc(rps)) {
+ slpc = rps_to_slpc(rps);
+
+ return slpc->boost_freq;
+ } else {
+ return intel_gpu_freq(rps, rps->boost_freq);
+ }
+}
+
+static int rps_set_boost_freq(struct intel_rps *rps, u32 val)
+{
+ bool boost = false;
+
+ /* Validate against (static) hardware limits */
+ val = intel_freq_opcode(rps, val);
+ if (val < rps->min_freq || val > rps->max_freq)
+ return -EINVAL;
+
+ mutex_lock(&rps->lock);
+ if (val != rps->boost_freq) {
+ rps->boost_freq = val;
+ boost = atomic_read(&rps->num_waiters);
+ }
+ mutex_unlock(&rps->lock);
+ if (boost)
+ schedule_work(&rps->work);
+
+ return 0;
+}
+
+int intel_rps_set_boost_frequency(struct intel_rps *rps, u32 freq)
+{
+ struct intel_guc_slpc *slpc;
+
+ if (rps_uses_slpc(rps)) {
+ slpc = rps_to_slpc(rps);
+
+ return intel_guc_slpc_set_boost_freq(slpc, freq);
+ } else {
+ return rps_set_boost_freq(rps, freq);
+ }
+}
+
+void intel_rps_dec_waiters(struct intel_rps *rps)
+{
+ struct intel_guc_slpc *slpc;
+
+ if (rps_uses_slpc(rps)) {
+ slpc = rps_to_slpc(rps);
+
+ intel_guc_slpc_dec_waiters(slpc);
+ } else {
+ atomic_dec(&rps->num_waiters);
+ }
+}
+
void intel_rps_boost(struct i915_request *rq)
{
+ struct intel_guc_slpc *slpc;
+
if (i915_request_signaled(rq) || i915_request_has_waitboost(rq))
return;
@@ -945,6 +1007,16 @@ void intel_rps_boost(struct i915_request *rq)
if (!test_and_set_bit(I915_FENCE_FLAG_BOOST, &rq->fence.flags)) {
struct intel_rps *rps = &READ_ONCE(rq->engine)->gt->rps;
+ if (rps_uses_slpc(rps)) {
+ slpc = rps_to_slpc(rps);
+
+ /* Return if old value is non zero */
+ if (!atomic_fetch_inc(&slpc->num_waiters))
+ schedule_work(&slpc->boost_work);
+
+ return;
+ }
+
if (atomic_fetch_inc(&rps->num_waiters))
return;
@@ -2154,6 +2226,65 @@ u32 intel_rps_read_state_cap(struct intel_rps *rps)
return intel_uncore_read(uncore, GEN6_RP_STATE_CAP);
}
+static void intel_rps_set_manual(struct intel_rps *rps, bool enable)
+{
+ struct intel_uncore *uncore = rps_to_uncore(rps);
+ u32 state = enable ? GEN9_RPSWCTL_ENABLE : GEN9_RPSWCTL_DISABLE;
+
+ /* Allow punit to process software requests */
+ intel_uncore_write(uncore, GEN6_RP_CONTROL, state);
+}
+
+void intel_rps_raise_unslice(struct intel_rps *rps)
+{
+ struct intel_uncore *uncore = rps_to_uncore(rps);
+ u32 rp0_unslice_req;
+
+ mutex_lock(&rps->lock);
+
+ if (rps_uses_slpc(rps)) {
+ /* RP limits have not been initialized yet for SLPC path */
+ rp0_unslice_req = ((intel_rps_read_state_cap(rps) >> 0)
+ & 0xff) * GEN9_FREQ_SCALER;
+
+ intel_rps_set_manual(rps, true);
+ intel_uncore_write(uncore, GEN6_RPNSWREQ,
+ ((rp0_unslice_req <<
+ GEN9_SW_REQ_UNSLICE_RATIO_SHIFT) |
+ GEN9_IGNORE_SLICE_RATIO));
+ intel_rps_set_manual(rps, false);
+ } else {
+ intel_rps_set(rps, rps->rp0_freq);
+ }
+
+ mutex_unlock(&rps->lock);
+}
+
+void intel_rps_lower_unslice(struct intel_rps *rps)
+{
+ struct intel_uncore *uncore = rps_to_uncore(rps);
+ u32 rpn_unslice_req;
+
+ mutex_lock(&rps->lock);
+
+ if (rps_uses_slpc(rps)) {
+ /* RP limits have not been initialized yet for SLPC path */
+ rpn_unslice_req = ((intel_rps_read_state_cap(rps) >> 16)
+ & 0xff) * GEN9_FREQ_SCALER;
+
+ intel_rps_set_manual(rps, true);
+ intel_uncore_write(uncore, GEN6_RPNSWREQ,
+ ((rpn_unslice_req <<
+ GEN9_SW_REQ_UNSLICE_RATIO_SHIFT) |
+ GEN9_IGNORE_SLICE_RATIO));
+ intel_rps_set_manual(rps, false);
+ } else {
+ intel_rps_set(rps, rps->min_freq);
+ }
+
+ mutex_unlock(&rps->lock);
+}
+
/* External interface for intel_ips.ko */
static struct drm_i915_private __rcu *ips_mchdev;
@@ -2230,7 +2361,7 @@ unsigned long i915_read_mch_val(void)
return 0;
with_intel_runtime_pm(&i915->runtime_pm, wakeref) {
- struct intel_ips *ips = &i915->gt.rps.ips;
+ struct intel_ips *ips = &to_gt(i915)->rps.ips;
spin_lock_irq(&mchdev_lock);
chipset_val = __ips_chipset_val(ips);
@@ -2257,7 +2388,7 @@ bool i915_gpu_raise(void)
if (!i915)
return false;
- rps = &i915->gt.rps;
+ rps = &to_gt(i915)->rps;
spin_lock_irq(&mchdev_lock);
if (rps->max_freq_softlimit < rps->max_freq)
@@ -2284,7 +2415,7 @@ bool i915_gpu_lower(void)
if (!i915)
return false;
- rps = &i915->gt.rps;
+ rps = &to_gt(i915)->rps;
spin_lock_irq(&mchdev_lock);
if (rps->max_freq_softlimit > rps->min_freq)
@@ -2310,7 +2441,7 @@ bool i915_gpu_busy(void)
if (!i915)
return false;
- ret = i915->gt.awake;
+ ret = to_gt(i915)->awake;
drm_dev_put(&i915->drm);
return ret;
@@ -2333,11 +2464,11 @@ bool i915_gpu_turbo_disable(void)
if (!i915)
return false;
- rps = &i915->gt.rps;
+ rps = &to_gt(i915)->rps;
spin_lock_irq(&mchdev_lock);
rps->max_freq_softlimit = rps->min_freq;
- ret = !__gen5_rps_set(&i915->gt.rps, rps->min_freq);
+ ret = !__gen5_rps_set(&to_gt(i915)->rps, rps->min_freq);
spin_unlock_irq(&mchdev_lock);
drm_dev_put(&i915->drm);
diff --git a/drivers/gpu/drm/i915/gt/intel_rps.h b/drivers/gpu/drm/i915/gt/intel_rps.h
index 11960d6..c6d76a3 100644
--- a/drivers/gpu/drm/i915/gt/intel_rps.h
+++ b/drivers/gpu/drm/i915/gt/intel_rps.h
@@ -23,6 +23,9 @@ void intel_rps_disable(struct intel_rps *rps);
void intel_rps_park(struct intel_rps *rps);
void intel_rps_unpark(struct intel_rps *rps);
void intel_rps_boost(struct i915_request *rq);
+void intel_rps_dec_waiters(struct intel_rps *rps);
+u32 intel_rps_get_boost_frequency(struct intel_rps *rps);
+int intel_rps_set_boost_frequency(struct intel_rps *rps, u32 freq);
int intel_rps_set(struct intel_rps *rps, u8 val);
void intel_rps_mark_interactive(struct intel_rps *rps, bool interactive);
@@ -42,6 +45,8 @@ u32 intel_rps_get_rpn_frequency(struct intel_rps *rps);
u32 intel_rps_read_punit_req(struct intel_rps *rps);
u32 intel_rps_read_punit_req_frequency(struct intel_rps *rps);
u32 intel_rps_read_state_cap(struct intel_rps *rps);
+void intel_rps_raise_unslice(struct intel_rps *rps);
+void intel_rps_lower_unslice(struct intel_rps *rps);
void gen5_rps_irq_handler(struct intel_rps *rps);
void gen6_rps_irq_handler(struct intel_rps *rps, u32 pm_iir);
diff --git a/drivers/gpu/drm/i915/gt/intel_workarounds.c b/drivers/gpu/drm/i915/gt/intel_workarounds.c
index 2400d64..ab3277a 100644
--- a/drivers/gpu/drm/i915/gt/intel_workarounds.c
+++ b/drivers/gpu/drm/i915/gt/intel_workarounds.c
@@ -482,7 +482,7 @@ static void kbl_ctx_workarounds_init(struct intel_engine_cs *engine,
gen9_ctx_workarounds_init(engine, wal);
/* WaToEnableHwFixForPushConstHWBug:kbl */
- if (IS_KBL_GT_STEP(i915, STEP_C0, STEP_FOREVER))
+ if (IS_KBL_GRAPHICS_STEP(i915, STEP_C0, STEP_FOREVER))
wa_masked_en(wal, COMMON_SLICE_CHICKEN2,
GEN8_SBE_DISABLE_REPLAY_BUF_OPTIMIZATION);
@@ -560,6 +560,22 @@ static void icl_ctx_workarounds_init(struct intel_engine_cs *engine,
/*
* These settings aren't actually workarounds, but general tuning settings that
+ * need to be programmed on dg2 platform.
+ */
+static void dg2_ctx_gt_tuning_init(struct intel_engine_cs *engine,
+ struct i915_wa_list *wal)
+{
+ wa_write_clr_set(wal, GEN11_L3SQCREG5, L3_PWM_TIMER_INIT_VAL_MASK,
+ REG_FIELD_PREP(L3_PWM_TIMER_INIT_VAL_MASK, 0x7f));
+ wa_add(wal,
+ FF_MODE2,
+ FF_MODE2_TDS_TIMER_MASK,
+ FF_MODE2_TDS_TIMER_128,
+ 0, false);
+}
+
+/*
+ * These settings aren't actually workarounds, but general tuning settings that
* need to be programmed on several platforms.
*/
static void gen12_ctx_gt_tuning_init(struct intel_engine_cs *engine,
@@ -637,6 +653,42 @@ static void dg1_ctx_workarounds_init(struct intel_engine_cs *engine,
DG1_HZ_READ_SUPPRESSION_OPTIMIZATION_DISABLE);
}
+static void dg2_ctx_workarounds_init(struct intel_engine_cs *engine,
+ struct i915_wa_list *wal)
+{
+ dg2_ctx_gt_tuning_init(engine, wal);
+
+ /* Wa_16011186671:dg2_g11 */
+ if (IS_DG2_GRAPHICS_STEP(engine->i915, G11, STEP_A0, STEP_B0)) {
+ wa_masked_dis(wal, VFLSKPD, DIS_MULT_MISS_RD_SQUASH);
+ wa_masked_en(wal, VFLSKPD, DIS_OVER_FETCH_CACHE);
+ }
+
+ if (IS_DG2_GRAPHICS_STEP(engine->i915, G10, STEP_A0, STEP_B0)) {
+ /* Wa_14010469329:dg2_g10 */
+ wa_masked_en(wal, GEN11_COMMON_SLICE_CHICKEN3,
+ XEHP_DUAL_SIMD8_SEQ_MERGE_DISABLE);
+
+ /*
+ * Wa_22010465075:dg2_g10
+ * Wa_22010613112:dg2_g10
+ * Wa_14010698770:dg2_g10
+ */
+ wa_masked_en(wal, GEN11_COMMON_SLICE_CHICKEN3,
+ GEN12_DISABLE_CPS_AWARE_COLOR_PIPE);
+ }
+
+ /* Wa_16013271637:dg2 */
+ wa_masked_en(wal, SLICE_COMMON_ECO_CHICKEN1,
+ MSC_MSAA_REODER_BUF_BYPASS_DISABLE);
+
+ /* Wa_22012532006:dg2 */
+ if (IS_DG2_GRAPHICS_STEP(engine->i915, G10, STEP_A0, STEP_C0) ||
+ IS_DG2_GRAPHICS_STEP(engine->i915, G11, STEP_A0, STEP_B0))
+ wa_masked_en(wal, GEN9_HALF_SLICE_CHICKEN7,
+ DG2_DISABLE_ROUND_ENABLE_ALLOW_FOR_SSLA);
+}
+
static void fakewa_disable_nestedbb_mode(struct intel_engine_cs *engine,
struct i915_wa_list *wal)
{
@@ -723,7 +775,11 @@ __intel_engine_init_ctx_wa(struct intel_engine_cs *engine,
if (engine->class != RENDER_CLASS)
goto done;
- if (IS_DG1(i915))
+ if (IS_DG2(i915))
+ dg2_ctx_workarounds_init(engine, wal);
+ else if (IS_XEHPSDV(i915))
+ ; /* noop; none at this time */
+ else if (IS_DG1(i915))
dg1_ctx_workarounds_init(engine, wal);
else if (GRAPHICS_VER(i915) == 12)
gen12_ctx_workarounds_init(engine, wal);
@@ -871,10 +927,51 @@ hsw_gt_workarounds_init(struct intel_gt *gt, struct i915_wa_list *wal)
}
static void
+gen9_wa_init_mcr(struct drm_i915_private *i915, struct i915_wa_list *wal)
+{
+ const struct sseu_dev_info *sseu = &to_gt(i915)->info.sseu;
+ unsigned int slice, subslice;
+ u32 mcr, mcr_mask;
+
+ GEM_BUG_ON(GRAPHICS_VER(i915) != 9);
+
+ /*
+ * WaProgramMgsrForCorrectSliceSpecificMmioReads:gen9,glk,kbl,cml
+ * Before any MMIO read into slice/subslice specific registers, MCR
+ * packet control register needs to be programmed to point to any
+ * enabled s/ss pair. Otherwise, incorrect values will be returned.
+ * This means each subsequent MMIO read will be forwarded to an
+ * specific s/ss combination, but this is OK since these registers
+ * are consistent across s/ss in almost all cases. In the rare
+ * occasions, such as INSTDONE, where this value is dependent
+ * on s/ss combo, the read should be done with read_subslice_reg.
+ */
+ slice = ffs(sseu->slice_mask) - 1;
+ GEM_BUG_ON(slice >= ARRAY_SIZE(sseu->subslice_mask));
+ subslice = ffs(intel_sseu_get_subslices(sseu, slice));
+ GEM_BUG_ON(!subslice);
+ subslice--;
+
+ /*
+ * We use GEN8_MCR..() macros to calculate the |mcr| value for
+ * Gen9 to address WaProgramMgsrForCorrectSliceSpecificMmioReads
+ */
+ mcr = GEN8_MCR_SLICE(slice) | GEN8_MCR_SUBSLICE(subslice);
+ mcr_mask = GEN8_MCR_SLICE_MASK | GEN8_MCR_SUBSLICE_MASK;
+
+ drm_dbg(&i915->drm, "MCR slice:%d/subslice:%d = %x\n", slice, subslice, mcr);
+
+ wa_write_clr_set(wal, GEN8_MCR_SELECTOR, mcr_mask, mcr);
+}
+
+static void
gen9_gt_workarounds_init(struct intel_gt *gt, struct i915_wa_list *wal)
{
struct drm_i915_private *i915 = gt->i915;
+ /* WaProgramMgsrForCorrectSliceSpecificMmioReads:glk,kbl,cml,gen9 */
+ gen9_wa_init_mcr(i915, wal);
+
/* WaDisableKillLogic:bxt,skl,kbl */
if (!IS_COFFEELAKE(i915) && !IS_COMETLAKE(i915))
wa_write_or(wal,
@@ -909,7 +1006,7 @@ skl_gt_workarounds_init(struct intel_gt *gt, struct i915_wa_list *wal)
GEN8_EU_GAUNIT_CLOCK_GATE_DISABLE);
/* WaInPlaceDecompressionHang:skl */
- if (IS_SKL_GT_STEP(gt->i915, STEP_A0, STEP_H0))
+ if (IS_SKL_GRAPHICS_STEP(gt->i915, STEP_A0, STEP_H0))
wa_write_or(wal,
GEN9_GAMT_ECO_REG_RW_IA,
GAMT_ECO_ENABLE_IN_PLACE_DECOMPRESS);
@@ -921,7 +1018,7 @@ kbl_gt_workarounds_init(struct intel_gt *gt, struct i915_wa_list *wal)
gen9_gt_workarounds_init(gt, wal);
/* WaDisableDynamicCreditSharing:kbl */
- if (IS_KBL_GT_STEP(gt->i915, 0, STEP_C0))
+ if (IS_KBL_GRAPHICS_STEP(gt->i915, 0, STEP_C0))
wa_write_or(wal,
GAMT_CHKN_BIT_REG,
GAMT_CHKN_DISABLE_DYNAMIC_CREDIT_SHARING);
@@ -1138,7 +1235,7 @@ icl_gt_workarounds_init(struct intel_gt *gt, struct i915_wa_list *wal)
/* Wa_1607087056:icl,ehl,jsl */
if (IS_ICELAKE(i915) ||
- IS_JSL_EHL_GT_STEP(i915, STEP_A0, STEP_B0))
+ IS_JSL_EHL_GRAPHICS_STEP(i915, STEP_A0, STEP_B0))
wa_write_or(wal,
SLICE_UNIT_LEVEL_CLKGATE,
L3_CLKGATE_DIS | L3_CR2X_CLKGATE_DIS);
@@ -1192,19 +1289,19 @@ tgl_gt_workarounds_init(struct intel_gt *gt, struct i915_wa_list *wal)
gen12_gt_workarounds_init(gt, wal);
/* Wa_1409420604:tgl */
- if (IS_TGL_UY_GT_STEP(i915, STEP_A0, STEP_B0))
+ if (IS_TGL_UY_GRAPHICS_STEP(i915, STEP_A0, STEP_B0))
wa_write_or(wal,
SUBSLICE_UNIT_LEVEL_CLKGATE2,
CPSSUNIT_CLKGATE_DIS);
/* Wa_1607087056:tgl also know as BUG:1409180338 */
- if (IS_TGL_UY_GT_STEP(i915, STEP_A0, STEP_B0))
+ if (IS_TGL_UY_GRAPHICS_STEP(i915, STEP_A0, STEP_B0))
wa_write_or(wal,
SLICE_UNIT_LEVEL_CLKGATE,
L3_CLKGATE_DIS | L3_CR2X_CLKGATE_DIS);
/* Wa_1408615072:tgl[a0] */
- if (IS_TGL_UY_GT_STEP(i915, STEP_A0, STEP_B0))
+ if (IS_TGL_UY_GRAPHICS_STEP(i915, STEP_A0, STEP_B0))
wa_write_or(wal, UNSLICE_UNIT_LEVEL_CLKGATE2,
VSUNIT_CLKGATE_DIS_TGL);
}
@@ -1217,7 +1314,7 @@ dg1_gt_workarounds_init(struct intel_gt *gt, struct i915_wa_list *wal)
gen12_gt_workarounds_init(gt, wal);
/* Wa_1607087056:dg1 */
- if (IS_DG1_GT_STEP(i915, STEP_A0, STEP_B0))
+ if (IS_DG1_GRAPHICS_STEP(i915, STEP_A0, STEP_B0))
wa_write_or(wal,
SLICE_UNIT_LEVEL_CLKGATE,
L3_CLKGATE_DIS | L3_CR2X_CLKGATE_DIS);
@@ -1238,7 +1335,179 @@ dg1_gt_workarounds_init(struct intel_gt *gt, struct i915_wa_list *wal)
static void
xehpsdv_gt_workarounds_init(struct intel_gt *gt, struct i915_wa_list *wal)
{
+ struct drm_i915_private *i915 = gt->i915;
+
xehp_init_mcr(gt, wal);
+
+ /* Wa_1409757795:xehpsdv */
+ wa_write_or(wal, SCCGCTL94DC, CG3DDISURB);
+
+ /* Wa_18011725039:xehpsdv */
+ if (IS_XEHPSDV_GRAPHICS_STEP(i915, STEP_A1, STEP_B0)) {
+ wa_masked_dis(wal, MLTICTXCTL, TDONRENDER);
+ wa_write_or(wal, L3SQCREG1_CCS0, FLUSHALLNONCOH);
+ }
+
+ /* Wa_16011155590:xehpsdv */
+ if (IS_XEHPSDV_GRAPHICS_STEP(i915, STEP_A0, STEP_B0))
+ wa_write_or(wal, UNSLICE_UNIT_LEVEL_CLKGATE,
+ TSGUNIT_CLKGATE_DIS);
+
+ /* Wa_14011780169:xehpsdv */
+ if (IS_XEHPSDV_GRAPHICS_STEP(i915, STEP_B0, STEP_FOREVER)) {
+ wa_write_or(wal, UNSLCGCTL9440, GAMTLBOACS_CLKGATE_DIS |
+ GAMTLBVDBOX7_CLKGATE_DIS |
+ GAMTLBVDBOX6_CLKGATE_DIS |
+ GAMTLBVDBOX5_CLKGATE_DIS |
+ GAMTLBVDBOX4_CLKGATE_DIS |
+ GAMTLBVDBOX3_CLKGATE_DIS |
+ GAMTLBVDBOX2_CLKGATE_DIS |
+ GAMTLBVDBOX1_CLKGATE_DIS |
+ GAMTLBVDBOX0_CLKGATE_DIS |
+ GAMTLBKCR_CLKGATE_DIS |
+ GAMTLBGUC_CLKGATE_DIS |
+ GAMTLBBLT_CLKGATE_DIS);
+ wa_write_or(wal, UNSLCGCTL9444, GAMTLBGFXA0_CLKGATE_DIS |
+ GAMTLBGFXA1_CLKGATE_DIS |
+ GAMTLBCOMPA0_CLKGATE_DIS |
+ GAMTLBCOMPA1_CLKGATE_DIS |
+ GAMTLBCOMPB0_CLKGATE_DIS |
+ GAMTLBCOMPB1_CLKGATE_DIS |
+ GAMTLBCOMPC0_CLKGATE_DIS |
+ GAMTLBCOMPC1_CLKGATE_DIS |
+ GAMTLBCOMPD0_CLKGATE_DIS |
+ GAMTLBCOMPD1_CLKGATE_DIS |
+ GAMTLBMERT_CLKGATE_DIS |
+ GAMTLBVEBOX3_CLKGATE_DIS |
+ GAMTLBVEBOX2_CLKGATE_DIS |
+ GAMTLBVEBOX1_CLKGATE_DIS |
+ GAMTLBVEBOX0_CLKGATE_DIS);
+ }
+
+ /* Wa_14012362059:xehpsdv */
+ wa_write_or(wal, GEN12_MERT_MOD_CTRL, FORCE_MISS_FTLB);
+
+ /* Wa_16012725990:xehpsdv */
+ if (IS_XEHPSDV_GRAPHICS_STEP(i915, STEP_A1, STEP_FOREVER))
+ wa_write_or(wal, UNSLICE_UNIT_LEVEL_CLKGATE, VFUNIT_CLKGATE_DIS);
+
+ /* Wa_14011060649:xehpsdv */
+ wa_14011060649(gt, wal);
+
+ /* Wa_14014368820:xehpsdv */
+ wa_write_or(wal, GEN12_GAMCNTRL_CTRL, INVALIDATION_BROADCAST_MODE_DIS |
+ GLOBAL_INVALIDATION_MODE);
+}
+
+static void
+dg2_gt_workarounds_init(struct intel_gt *gt, struct i915_wa_list *wal)
+{
+ struct intel_engine_cs *engine;
+ int id;
+
+ xehp_init_mcr(gt, wal);
+
+ /* Wa_14011060649:dg2 */
+ wa_14011060649(gt, wal);
+
+ /*
+ * Although there are per-engine instances of these registers,
+ * they technically exist outside the engine itself and are not
+ * impacted by engine resets. Furthermore, they're part of the
+ * GuC blacklist so trying to treat them as engine workarounds
+ * will result in GuC initialization failure and a wedged GPU.
+ */
+ for_each_engine(engine, gt, id) {
+ if (engine->class != VIDEO_DECODE_CLASS)
+ continue;
+
+ /* Wa_16010515920:dg2_g10 */
+ if (IS_DG2_GRAPHICS_STEP(gt->i915, G10, STEP_A0, STEP_B0))
+ wa_write_or(wal, VDBOX_CGCTL3F18(engine->mmio_base),
+ ALNUNIT_CLKGATE_DIS);
+ }
+
+ if (IS_DG2_G10(gt->i915)) {
+ /* Wa_22010523718:dg2 */
+ wa_write_or(wal, UNSLICE_UNIT_LEVEL_CLKGATE,
+ CG3DDISCFEG_CLKGATE_DIS);
+
+ /* Wa_14011006942:dg2 */
+ wa_write_or(wal, SUBSLICE_UNIT_LEVEL_CLKGATE,
+ DSS_ROUTER_CLKGATE_DIS);
+ }
+
+ if (IS_DG2_GRAPHICS_STEP(gt->i915, G10, STEP_A0, STEP_B0)) {
+ /* Wa_14010680813:dg2_g10 */
+ wa_write_or(wal, GEN12_GAMSTLB_CTRL, CONTROL_BLOCK_CLKGATE_DIS |
+ EGRESS_BLOCK_CLKGATE_DIS | TAG_BLOCK_CLKGATE_DIS);
+
+ /* Wa_14010948348:dg2_g10 */
+ wa_write_or(wal, UNSLCGCTL9430, MSQDUNIT_CLKGATE_DIS);
+
+ /* Wa_14011037102:dg2_g10 */
+ wa_write_or(wal, UNSLCGCTL9444, LTCDD_CLKGATE_DIS);
+
+ /* Wa_14011371254:dg2_g10 */
+ wa_write_or(wal, SLICE_UNIT_LEVEL_CLKGATE, NODEDSS_CLKGATE_DIS);
+
+ /* Wa_14011431319:dg2_g10 */
+ wa_write_or(wal, UNSLCGCTL9440, GAMTLBOACS_CLKGATE_DIS |
+ GAMTLBVDBOX7_CLKGATE_DIS |
+ GAMTLBVDBOX6_CLKGATE_DIS |
+ GAMTLBVDBOX5_CLKGATE_DIS |
+ GAMTLBVDBOX4_CLKGATE_DIS |
+ GAMTLBVDBOX3_CLKGATE_DIS |
+ GAMTLBVDBOX2_CLKGATE_DIS |
+ GAMTLBVDBOX1_CLKGATE_DIS |
+ GAMTLBVDBOX0_CLKGATE_DIS |
+ GAMTLBKCR_CLKGATE_DIS |
+ GAMTLBGUC_CLKGATE_DIS |
+ GAMTLBBLT_CLKGATE_DIS);
+ wa_write_or(wal, UNSLCGCTL9444, GAMTLBGFXA0_CLKGATE_DIS |
+ GAMTLBGFXA1_CLKGATE_DIS |
+ GAMTLBCOMPA0_CLKGATE_DIS |
+ GAMTLBCOMPA1_CLKGATE_DIS |
+ GAMTLBCOMPB0_CLKGATE_DIS |
+ GAMTLBCOMPB1_CLKGATE_DIS |
+ GAMTLBCOMPC0_CLKGATE_DIS |
+ GAMTLBCOMPC1_CLKGATE_DIS |
+ GAMTLBCOMPD0_CLKGATE_DIS |
+ GAMTLBCOMPD1_CLKGATE_DIS |
+ GAMTLBMERT_CLKGATE_DIS |
+ GAMTLBVEBOX3_CLKGATE_DIS |
+ GAMTLBVEBOX2_CLKGATE_DIS |
+ GAMTLBVEBOX1_CLKGATE_DIS |
+ GAMTLBVEBOX0_CLKGATE_DIS);
+
+ /* Wa_14010569222:dg2_g10 */
+ wa_write_or(wal, UNSLICE_UNIT_LEVEL_CLKGATE,
+ GAMEDIA_CLKGATE_DIS);
+
+ /* Wa_14011028019:dg2_g10 */
+ wa_write_or(wal, SSMCGCTL9530, RTFUNIT_CLKGATE_DIS);
+ }
+
+ if (IS_DG2_GRAPHICS_STEP(gt->i915, G10, STEP_A0, STEP_B0) ||
+ IS_DG2_GRAPHICS_STEP(gt->i915, G11, STEP_A0, STEP_B0)) {
+ /* Wa_14012362059:dg2 */
+ wa_write_or(wal, GEN12_MERT_MOD_CTRL, FORCE_MISS_FTLB);
+ }
+
+ /* Wa_1509235366:dg2 */
+ wa_write_or(wal, GEN12_GAMCNTRL_CTRL, INVALIDATION_BROADCAST_MODE_DIS |
+ GLOBAL_INVALIDATION_MODE);
+
+ /* Wa_14014830051:dg2 */
+ wa_write_clr(wal, SARB_CHICKEN1, COMP_CKN_IN);
+
+ /*
+ * The following are not actually "workarounds" but rather
+ * recommended tuning settings documented in the bspec's
+ * performance guide section.
+ */
+ wa_write_or(wal, XEHP_L3SCQREG7, BLEND_FILL_CACHING_OPT_DIS);
+ wa_write_or(wal, GEN12_SQCM, EN_32B_ACCESS);
}
static void
@@ -1246,7 +1515,9 @@ gt_init_workarounds(struct intel_gt *gt, struct i915_wa_list *wal)
{
struct drm_i915_private *i915 = gt->i915;
- if (IS_XEHPSDV(i915))
+ if (IS_DG2(i915))
+ dg2_gt_workarounds_init(gt, wal);
+ else if (IS_XEHPSDV(i915))
xehpsdv_gt_workarounds_init(gt, wal);
else if (IS_DG1(i915))
dg1_gt_workarounds_init(gt, wal);
@@ -1520,7 +1791,7 @@ static void cfl_whitelist_build(struct intel_engine_cs *engine)
RING_FORCE_TO_NONPRIV_RANGE_4);
}
-static void cml_whitelist_build(struct intel_engine_cs *engine)
+static void allow_read_ctx_timestamp(struct intel_engine_cs *engine)
{
struct i915_wa_list *w = &engine->whitelist;
@@ -1528,6 +1799,11 @@ static void cml_whitelist_build(struct intel_engine_cs *engine)
whitelist_reg_ext(w,
RING_CTX_TIMESTAMP(engine->mmio_base),
RING_FORCE_TO_NONPRIV_ACCESS_RD);
+}
+
+static void cml_whitelist_build(struct intel_engine_cs *engine)
+{
+ allow_read_ctx_timestamp(engine);
cfl_whitelist_build(engine);
}
@@ -1536,6 +1812,8 @@ static void icl_whitelist_build(struct intel_engine_cs *engine)
{
struct i915_wa_list *w = &engine->whitelist;
+ allow_read_ctx_timestamp(engine);
+
switch (engine->class) {
case RENDER_CLASS:
/* WaAllowUMDToModifyHalfSliceChicken7:icl */
@@ -1571,15 +1849,9 @@ static void icl_whitelist_build(struct intel_engine_cs *engine)
/* hucStatus2RegOffset */
whitelist_reg_ext(w, _MMIO(0x23B0 + engine->mmio_base),
RING_FORCE_TO_NONPRIV_ACCESS_RD);
- whitelist_reg_ext(w,
- RING_CTX_TIMESTAMP(engine->mmio_base),
- RING_FORCE_TO_NONPRIV_ACCESS_RD);
break;
default:
- whitelist_reg_ext(w,
- RING_CTX_TIMESTAMP(engine->mmio_base),
- RING_FORCE_TO_NONPRIV_ACCESS_RD);
break;
}
}
@@ -1588,6 +1860,8 @@ static void tgl_whitelist_build(struct intel_engine_cs *engine)
{
struct i915_wa_list *w = &engine->whitelist;
+ allow_read_ctx_timestamp(engine);
+
switch (engine->class) {
case RENDER_CLASS:
/*
@@ -1604,16 +1878,17 @@ static void tgl_whitelist_build(struct intel_engine_cs *engine)
RING_FORCE_TO_NONPRIV_ACCESS_RD |
RING_FORCE_TO_NONPRIV_RANGE_4);
- /* Wa_1808121037:tgl */
+ /*
+ * Wa_1808121037:tgl
+ * Wa_14012131227:dg1
+ * Wa_1508744258:tgl,rkl,dg1,adl-s,adl-p
+ */
whitelist_reg(w, GEN7_COMMON_SLICE_CHICKEN1);
/* Wa_1806527549:tgl */
whitelist_reg(w, HIZ_CHICKEN);
break;
default:
- whitelist_reg_ext(w,
- RING_CTX_TIMESTAMP(engine->mmio_base),
- RING_FORCE_TO_NONPRIV_ACCESS_RD);
break;
}
}
@@ -1625,13 +1900,46 @@ static void dg1_whitelist_build(struct intel_engine_cs *engine)
tgl_whitelist_build(engine);
/* GEN:BUG:1409280441:dg1 */
- if (IS_DG1_GT_STEP(engine->i915, STEP_A0, STEP_B0) &&
+ if (IS_DG1_GRAPHICS_STEP(engine->i915, STEP_A0, STEP_B0) &&
(engine->class == RENDER_CLASS ||
engine->class == COPY_ENGINE_CLASS))
whitelist_reg_ext(w, RING_ID(engine->mmio_base),
RING_FORCE_TO_NONPRIV_ACCESS_RD);
}
+static void xehpsdv_whitelist_build(struct intel_engine_cs *engine)
+{
+ allow_read_ctx_timestamp(engine);
+}
+
+static void dg2_whitelist_build(struct intel_engine_cs *engine)
+{
+ struct i915_wa_list *w = &engine->whitelist;
+
+ allow_read_ctx_timestamp(engine);
+
+ switch (engine->class) {
+ case RENDER_CLASS:
+ /*
+ * Wa_1507100340:dg2_g10
+ *
+ * This covers 4 registers which are next to one another :
+ * - PS_INVOCATION_COUNT
+ * - PS_INVOCATION_COUNT_UDW
+ * - PS_DEPTH_COUNT
+ * - PS_DEPTH_COUNT_UDW
+ */
+ if (IS_DG2_GRAPHICS_STEP(engine->i915, G10, STEP_A0, STEP_B0))
+ whitelist_reg_ext(w, PS_INVOCATION_COUNT,
+ RING_FORCE_TO_NONPRIV_ACCESS_RD |
+ RING_FORCE_TO_NONPRIV_RANGE_4);
+
+ break;
+ default:
+ break;
+ }
+}
+
void intel_engine_init_whitelist(struct intel_engine_cs *engine)
{
struct drm_i915_private *i915 = engine->i915;
@@ -1639,7 +1947,11 @@ void intel_engine_init_whitelist(struct intel_engine_cs *engine)
wa_init_start(w, "whitelist", engine->name);
- if (IS_DG1(i915))
+ if (IS_DG2(i915))
+ dg2_whitelist_build(engine);
+ else if (IS_XEHPSDV(i915))
+ xehpsdv_whitelist_build(engine);
+ else if (IS_DG1(i915))
dg1_whitelist_build(engine);
else if (GRAPHICS_VER(i915) == 12)
tgl_whitelist_build(engine);
@@ -1713,13 +2025,119 @@ engine_fake_wa_init(struct intel_engine_cs *engine, struct i915_wa_list *wal)
CMD_CCTL_MOCS_OVERRIDE(mocs, mocs));
}
}
+
+static bool needs_wa_1308578152(struct intel_engine_cs *engine)
+{
+ u64 dss_mask = intel_sseu_get_subslices(&engine->gt->info.sseu, 0);
+
+ return (dss_mask & GENMASK(GEN_DSS_PER_GSLICE - 1, 0)) == 0;
+}
+
static void
rcs_engine_wa_init(struct intel_engine_cs *engine, struct i915_wa_list *wal)
{
struct drm_i915_private *i915 = engine->i915;
- if (IS_DG1_GT_STEP(i915, STEP_A0, STEP_B0) ||
- IS_TGL_UY_GT_STEP(i915, STEP_A0, STEP_B0)) {
+ if (IS_DG2_GRAPHICS_STEP(engine->i915, G11, STEP_A0, STEP_B0)) {
+ /* Wa_14013392000:dg2_g11 */
+ wa_masked_en(wal, GEN7_ROW_CHICKEN2, GEN12_ENABLE_LARGE_GRF_MODE);
+
+ /* Wa_16011620976:dg2_g11 */
+ wa_write_or(wal, LSC_CHICKEN_BIT_0_UDW, DIS_CHAIN_2XSIMD8);
+ }
+
+ if (IS_DG2_GRAPHICS_STEP(engine->i915, G10, STEP_A0, STEP_B0) ||
+ IS_DG2_GRAPHICS_STEP(engine->i915, G11, STEP_A0, STEP_B0)) {
+ /* Wa_14012419201:dg2 */
+ wa_masked_en(wal, GEN9_ROW_CHICKEN4,
+ GEN12_DISABLE_HDR_PAST_PAYLOAD_HOLD_FIX);
+ }
+
+ if (IS_DG2_GRAPHICS_STEP(engine->i915, G10, STEP_B0, STEP_C0) ||
+ IS_DG2_G11(engine->i915)) {
+ /*
+ * Wa_22012826095:dg2
+ * Wa_22013059131:dg2
+ */
+ wa_write_clr_set(wal, LSC_CHICKEN_BIT_0_UDW,
+ MAXREQS_PER_BANK,
+ REG_FIELD_PREP(MAXREQS_PER_BANK, 2));
+
+ /* Wa_22013059131:dg2 */
+ wa_write_or(wal, LSC_CHICKEN_BIT_0,
+ FORCE_1_SUB_MESSAGE_PER_FRAGMENT);
+ }
+
+ /* Wa_1308578152:dg2_g10 when first gslice is fused off */
+ if (IS_DG2_GRAPHICS_STEP(engine->i915, G10, STEP_B0, STEP_C0) &&
+ needs_wa_1308578152(engine)) {
+ wa_masked_dis(wal, GEN12_CS_DEBUG_MODE1_CCCSUNIT_BE_COMMON,
+ GEN12_REPLAY_MODE_GRANULARITY);
+ }
+
+ if (IS_DG2_GRAPHICS_STEP(engine->i915, G10, STEP_B0, STEP_FOREVER) ||
+ IS_DG2_G11(engine->i915)) {
+ /* Wa_22013037850:dg2 */
+ wa_write_or(wal, LSC_CHICKEN_BIT_0_UDW,
+ DISABLE_128B_EVICTION_COMMAND_UDW);
+
+ /* Wa_22012856258:dg2 */
+ wa_masked_en(wal, GEN7_ROW_CHICKEN2,
+ GEN12_DISABLE_READ_SUPPRESSION);
+
+ /*
+ * Wa_22010960976:dg2
+ * Wa_14013347512:dg2
+ */
+ wa_masked_dis(wal, GEN12_HDC_CHICKEN0,
+ LSC_L1_FLUSH_CTL_3D_DATAPORT_FLUSH_EVENTS_MASK);
+ }
+
+ if (IS_DG2_GRAPHICS_STEP(engine->i915, G10, STEP_A0, STEP_B0)) {
+ /*
+ * Wa_1608949956:dg2_g10
+ * Wa_14010198302:dg2_g10
+ */
+ wa_masked_en(wal, GEN8_ROW_CHICKEN,
+ MDQ_ARBITRATION_MODE | UGM_BACKUP_MODE);
+
+ /*
+ * Wa_14010918519:dg2_g10
+ *
+ * LSC_CHICKEN_BIT_0 always reads back as 0 is this stepping,
+ * so ignoring verification.
+ */
+ wa_add(wal, LSC_CHICKEN_BIT_0_UDW, 0,
+ FORCE_SLM_FENCE_SCOPE_TO_TILE | FORCE_UGM_FENCE_SCOPE_TO_TILE,
+ 0, false);
+ }
+
+ if (IS_DG2_GRAPHICS_STEP(engine->i915, G10, STEP_A0, STEP_B0)) {
+ /* Wa_22010430635:dg2 */
+ wa_masked_en(wal,
+ GEN9_ROW_CHICKEN4,
+ GEN12_DISABLE_GRF_CLEAR);
+
+ /* Wa_14010648519:dg2 */
+ wa_write_or(wal, XEHP_L3NODEARBCFG, XEHP_LNESPARE);
+ }
+
+ if (IS_DG2_GRAPHICS_STEP(engine->i915, G10, STEP_A0, STEP_C0) ||
+ IS_DG2_G11(engine->i915)) {
+ /* Wa_22012654132:dg2 */
+ wa_add(wal, GEN10_CACHE_MODE_SS, 0,
+ _MASKED_BIT_ENABLE(ENABLE_PREFETCH_INTO_IC),
+ 0 /* write-only, so skip validation */,
+ true);
+ }
+
+ /* Wa_14013202645:dg2 */
+ if (IS_DG2_GRAPHICS_STEP(engine->i915, G10, STEP_B0, STEP_C0) ||
+ IS_DG2_GRAPHICS_STEP(engine->i915, G11, STEP_A0, STEP_B0))
+ wa_write_or(wal, RT_CTRL, DIS_NULL_QUERY);
+
+ if (IS_DG1_GRAPHICS_STEP(i915, STEP_A0, STEP_B0) ||
+ IS_TGL_UY_GRAPHICS_STEP(i915, STEP_A0, STEP_B0)) {
/*
* Wa_1607138336:tgl[a0],dg1[a0]
* Wa_1607063988:tgl[a0],dg1[a0]
@@ -1729,7 +2147,7 @@ rcs_engine_wa_init(struct intel_engine_cs *engine, struct i915_wa_list *wal)
GEN12_DISABLE_POSH_BUSY_FF_DOP_CG);
}
- if (IS_TGL_UY_GT_STEP(i915, STEP_A0, STEP_B0)) {
+ if (IS_TGL_UY_GRAPHICS_STEP(i915, STEP_A0, STEP_B0)) {
/*
* Wa_1606679103:tgl
* (see also Wa_1606682166:icl)
@@ -1764,7 +2182,7 @@ rcs_engine_wa_init(struct intel_engine_cs *engine, struct i915_wa_list *wal)
}
if (IS_ALDERLAKE_P(i915) || IS_ALDERLAKE_S(i915) ||
- IS_DG1_GT_STEP(i915, STEP_A0, STEP_B0) ||
+ IS_DG1_GRAPHICS_STEP(i915, STEP_A0, STEP_B0) ||
IS_ROCKETLAKE(i915) || IS_TIGERLAKE(i915)) {
/* Wa_1409804808:tgl,rkl,dg1[a0],adl-s,adl-p */
wa_masked_en(wal, GEN7_ROW_CHICKEN2,
@@ -1777,8 +2195,7 @@ rcs_engine_wa_init(struct intel_engine_cs *engine, struct i915_wa_list *wal)
wa_masked_en(wal, GEN9_ROW_CHICKEN4, GEN12_DISABLE_TDL_PUSH);
}
-
- if (IS_DG1_GT_STEP(i915, STEP_A0, STEP_B0) ||
+ if (IS_DG1_GRAPHICS_STEP(i915, STEP_A0, STEP_B0) ||
IS_ROCKETLAKE(i915) || IS_TIGERLAKE(i915)) {
/*
* Wa_1607030317:tgl
@@ -2131,7 +2548,7 @@ xcs_engine_wa_init(struct intel_engine_cs *engine, struct i915_wa_list *wal)
struct drm_i915_private *i915 = engine->i915;
/* WaKBLVECSSemaphoreWaitPoll:kbl */
- if (IS_KBL_GT_STEP(i915, STEP_A0, STEP_F0)) {
+ if (IS_KBL_GRAPHICS_STEP(i915, STEP_A0, STEP_F0)) {
wa_write(wal,
RING_SEMA_WAIT_POLL(engine->mmio_base),
1);
diff --git a/drivers/gpu/drm/i915/gt/mock_engine.c b/drivers/gpu/drm/i915/gt/mock_engine.c
index 8b89215..c0637bf 100644
--- a/drivers/gpu/drm/i915/gt/mock_engine.c
+++ b/drivers/gpu/drm/i915/gt/mock_engine.c
@@ -17,7 +17,7 @@ static int mock_timeline_pin(struct intel_timeline *tl)
{
int err;
- if (WARN_ON(!i915_gem_object_trylock(tl->hwsp_ggtt->obj)))
+ if (WARN_ON(!i915_gem_object_trylock(tl->hwsp_ggtt->obj, NULL)))
return -EBUSY;
err = intel_timeline_pin_map(tl);
@@ -35,9 +35,31 @@ static void mock_timeline_unpin(struct intel_timeline *tl)
atomic_dec(&tl->pin_count);
}
+static struct i915_vma *create_ring_vma(struct i915_ggtt *ggtt, int size)
+{
+ struct i915_address_space *vm = &ggtt->vm;
+ struct drm_i915_private *i915 = vm->i915;
+ struct drm_i915_gem_object *obj;
+ struct i915_vma *vma;
+
+ obj = i915_gem_object_create_internal(i915, size);
+ if (IS_ERR(obj))
+ return ERR_CAST(obj);
+
+ vma = i915_vma_instance(obj, vm, NULL);
+ if (IS_ERR(vma))
+ goto err;
+
+ return vma;
+
+err:
+ i915_gem_object_put(obj);
+ return vma;
+}
+
static struct intel_ring *mock_ring(struct intel_engine_cs *engine)
{
- const unsigned long sz = PAGE_SIZE / 2;
+ const unsigned long sz = PAGE_SIZE;
struct intel_ring *ring;
ring = kzalloc(sizeof(*ring) + sz, GFP_KERNEL);
@@ -50,15 +72,11 @@ static struct intel_ring *mock_ring(struct intel_engine_cs *engine)
ring->vaddr = (void *)(ring + 1);
atomic_set(&ring->pin_count, 1);
- ring->vma = i915_vma_alloc();
- if (!ring->vma) {
+ ring->vma = create_ring_vma(engine->gt->ggtt, PAGE_SIZE);
+ if (IS_ERR(ring->vma)) {
kfree(ring);
return NULL;
}
- i915_active_init(&ring->vma->active, NULL, NULL, 0);
- __set_bit(I915_VMA_GGTT_BIT, __i915_vma_flags(ring->vma));
- __set_bit(DRM_MM_NODE_ALLOCATED_BIT, &ring->vma->node.flags);
- ring->vma->node.size = sz;
intel_ring_update_space(ring);
@@ -67,8 +85,7 @@ static struct intel_ring *mock_ring(struct intel_engine_cs *engine)
static void mock_ring_free(struct intel_ring *ring)
{
- i915_active_fini(&ring->vma->active);
- i915_vma_free(ring->vma);
+ i915_vma_put(ring->vma);
kfree(ring);
}
@@ -125,6 +142,7 @@ static void mock_context_unpin(struct intel_context *ce)
static void mock_context_post_unpin(struct intel_context *ce)
{
+ i915_vma_unpin(ce->ring->vma);
}
static void mock_context_destroy(struct kref *ref)
@@ -169,7 +187,7 @@ static int mock_context_alloc(struct intel_context *ce)
static int mock_context_pre_pin(struct intel_context *ce,
struct i915_gem_ww_ctx *ww, void **unused)
{
- return 0;
+ return i915_vma_pin_ww(ce->ring->vma, ww, 0, 0, PIN_GLOBAL | PIN_HIGH);
}
static int mock_context_pin(struct intel_context *ce, void *unused)
@@ -327,7 +345,7 @@ struct intel_engine_cs *mock_engine(struct drm_i915_private *i915,
struct mock_engine *engine;
GEM_BUG_ON(id >= I915_NUM_ENGINES);
- GEM_BUG_ON(!i915->gt.uncore);
+ GEM_BUG_ON(!to_gt(i915)->uncore);
engine = kzalloc(sizeof(*engine) + PAGE_SIZE, GFP_KERNEL);
if (!engine)
@@ -335,8 +353,8 @@ struct intel_engine_cs *mock_engine(struct drm_i915_private *i915,
/* minimal engine setup for requests */
engine->base.i915 = i915;
- engine->base.gt = &i915->gt;
- engine->base.uncore = i915->gt.uncore;
+ engine->base.gt = to_gt(i915);
+ engine->base.uncore = to_gt(i915)->uncore;
snprintf(engine->base.name, sizeof(engine->base.name), "%s", name);
engine->base.id = id;
engine->base.mask = BIT(id);
@@ -359,8 +377,8 @@ struct intel_engine_cs *mock_engine(struct drm_i915_private *i915,
engine->base.release = mock_engine_release;
- i915->gt.engine[id] = &engine->base;
- i915->gt.engine_class[0][id] = &engine->base;
+ to_gt(i915)->engine[id] = &engine->base;
+ to_gt(i915)->engine_class[0][id] = &engine->base;
/* fake hw queue */
spin_lock_init(&engine->hw_lock);
diff --git a/drivers/gpu/drm/i915/gt/selftest_context.c b/drivers/gpu/drm/i915/gt/selftest_context.c
index fa7b99a..76fbae3 100644
--- a/drivers/gpu/drm/i915/gt/selftest_context.c
+++ b/drivers/gpu/drm/i915/gt/selftest_context.c
@@ -442,7 +442,7 @@ int intel_context_live_selftests(struct drm_i915_private *i915)
SUBTEST(live_active_context),
SUBTEST(live_remote_context),
};
- struct intel_gt *gt = &i915->gt;
+ struct intel_gt *gt = to_gt(i915);
if (intel_gt_is_wedged(gt))
return 0;
diff --git a/drivers/gpu/drm/i915/gt/selftest_engine.c b/drivers/gpu/drm/i915/gt/selftest_engine.c
index 262764f..57fea9e 100644
--- a/drivers/gpu/drm/i915/gt/selftest_engine.c
+++ b/drivers/gpu/drm/i915/gt/selftest_engine.c
@@ -12,7 +12,7 @@ int intel_engine_live_selftests(struct drm_i915_private *i915)
live_engine_pm_selftests,
NULL,
};
- struct intel_gt *gt = &i915->gt;
+ struct intel_gt *gt = to_gt(i915);
typeof(*tests) *fn;
for (fn = tests; *fn; fn++) {
diff --git a/drivers/gpu/drm/i915/gt/selftest_engine_cs.c b/drivers/gpu/drm/i915/gt/selftest_engine_cs.c
index 64abf5f..1b75f47 100644
--- a/drivers/gpu/drm/i915/gt/selftest_engine_cs.c
+++ b/drivers/gpu/drm/i915/gt/selftest_engine_cs.c
@@ -361,10 +361,10 @@ int intel_engine_cs_perf_selftests(struct drm_i915_private *i915)
SUBTEST(perf_mi_noop),
};
- if (intel_gt_is_wedged(&i915->gt))
+ if (intel_gt_is_wedged(to_gt(i915)))
return 0;
- return intel_gt_live_subtests(tests, &i915->gt);
+ return intel_gt_live_subtests(tests, to_gt(i915));
}
static int intel_mmio_bases_check(void *arg)
diff --git a/drivers/gpu/drm/i915/gt/selftest_engine_heartbeat.c b/drivers/gpu/drm/i915/gt/selftest_engine_heartbeat.c
index 6e6e4d7..273d440 100644
--- a/drivers/gpu/drm/i915/gt/selftest_engine_heartbeat.c
+++ b/drivers/gpu/drm/i915/gt/selftest_engine_heartbeat.c
@@ -378,13 +378,13 @@ int intel_heartbeat_live_selftests(struct drm_i915_private *i915)
int saved_hangcheck;
int err;
- if (intel_gt_is_wedged(&i915->gt))
+ if (intel_gt_is_wedged(to_gt(i915)))
return 0;
saved_hangcheck = i915->params.enable_hangcheck;
i915->params.enable_hangcheck = INT_MAX;
- err = intel_gt_live_subtests(tests, &i915->gt);
+ err = intel_gt_live_subtests(tests, to_gt(i915));
i915->params.enable_hangcheck = saved_hangcheck;
return err;
diff --git a/drivers/gpu/drm/i915/gt/selftest_engine_pm.c b/drivers/gpu/drm/i915/gt/selftest_engine_pm.c
index 7556966..8af2618 100644
--- a/drivers/gpu/drm/i915/gt/selftest_engine_pm.c
+++ b/drivers/gpu/drm/i915/gt/selftest_engine_pm.c
@@ -214,6 +214,31 @@ static int live_engine_timestamps(void *arg)
return 0;
}
+static int __spin_until_busier(struct intel_engine_cs *engine, ktime_t busyness)
+{
+ ktime_t start, unused, dt;
+
+ if (!intel_engine_uses_guc(engine))
+ return 0;
+
+ /*
+ * In GuC mode of submission, the busyness stats may get updated after
+ * the batch starts running. Poll for a change in busyness and timeout
+ * after 500 us.
+ */
+ start = ktime_get();
+ while (intel_engine_get_busy_time(engine, &unused) == busyness) {
+ dt = ktime_get() - start;
+ if (dt > 10000000) {
+ pr_err("active wait timed out %lld\n", dt);
+ ENGINE_TRACE(engine, "active wait time out %lld\n", dt);
+ return -ETIME;
+ }
+ }
+
+ return 0;
+}
+
static int live_engine_busy_stats(void *arg)
{
struct intel_gt *gt = arg;
@@ -232,6 +257,7 @@ static int live_engine_busy_stats(void *arg)
GEM_BUG_ON(intel_gt_pm_is_awake(gt));
for_each_engine(engine, gt, id) {
struct i915_request *rq;
+ ktime_t busyness, dummy;
ktime_t de, dt;
ktime_t t[2];
@@ -274,16 +300,23 @@ static int live_engine_busy_stats(void *arg)
}
i915_request_add(rq);
+ busyness = intel_engine_get_busy_time(engine, &dummy);
if (!igt_wait_for_spinner(&spin, rq)) {
intel_gt_set_wedged(engine->gt);
err = -ETIME;
goto end;
}
+ err = __spin_until_busier(engine, busyness);
+ if (err) {
+ GEM_TRACE_DUMP();
+ goto end;
+ }
+
ENGINE_TRACE(engine, "measuring busy time\n");
preempt_disable();
de = intel_engine_get_busy_time(engine, &t[0]);
- udelay(100);
+ mdelay(10);
de = ktime_sub(intel_engine_get_busy_time(engine, &t[1]), de);
preempt_enable();
dt = ktime_sub(t[1], t[0]);
diff --git a/drivers/gpu/drm/i915/gt/selftest_execlists.c b/drivers/gpu/drm/i915/gt/selftest_execlists.c
index b367ecf..e10da89 100644
--- a/drivers/gpu/drm/i915/gt/selftest_execlists.c
+++ b/drivers/gpu/drm/i915/gt/selftest_execlists.c
@@ -4502,11 +4502,11 @@ int intel_execlists_live_selftests(struct drm_i915_private *i915)
SUBTEST(live_virtual_reset),
};
- if (i915->gt.submission_method != INTEL_SUBMISSION_ELSP)
+ if (to_gt(i915)->submission_method != INTEL_SUBMISSION_ELSP)
return 0;
- if (intel_gt_is_wedged(&i915->gt))
+ if (intel_gt_is_wedged(to_gt(i915)))
return 0;
- return intel_gt_live_subtests(tests, &i915->gt);
+ return intel_gt_live_subtests(tests, to_gt(i915));
}
diff --git a/drivers/gpu/drm/i915/gt/selftest_gt_pm.c b/drivers/gpu/drm/i915/gt/selftest_gt_pm.c
index b944121..8bf62a5 100644
--- a/drivers/gpu/drm/i915/gt/selftest_gt_pm.c
+++ b/drivers/gpu/drm/i915/gt/selftest_gt_pm.c
@@ -43,7 +43,7 @@ static void measure_clocks(struct intel_engine_cs *engine,
int i;
for (i = 0; i < 5; i++) {
- preempt_disable();
+ local_irq_disable();
cycles[i] = -ENGINE_READ_FW(engine, RING_TIMESTAMP);
dt[i] = ktime_get();
@@ -51,7 +51,7 @@ static void measure_clocks(struct intel_engine_cs *engine,
dt[i] = ktime_sub(ktime_get(), dt[i]);
cycles[i] += ENGINE_READ_FW(engine, RING_TIMESTAMP);
- preempt_enable();
+ local_irq_enable();
}
/* Use the median of both cycle/dt; close enough */
@@ -193,10 +193,10 @@ int intel_gt_pm_live_selftests(struct drm_i915_private *i915)
SUBTEST(live_gt_resume),
};
- if (intel_gt_is_wedged(&i915->gt))
+ if (intel_gt_is_wedged(to_gt(i915)))
return 0;
- return intel_gt_live_subtests(tests, &i915->gt);
+ return intel_gt_live_subtests(tests, to_gt(i915));
}
int intel_gt_pm_late_selftests(struct drm_i915_private *i915)
@@ -210,8 +210,8 @@ int intel_gt_pm_late_selftests(struct drm_i915_private *i915)
SUBTEST(live_rc6_ctx_wa),
};
- if (intel_gt_is_wedged(&i915->gt))
+ if (intel_gt_is_wedged(to_gt(i915)))
return 0;
- return intel_gt_live_subtests(tests, &i915->gt);
+ return intel_gt_live_subtests(tests, to_gt(i915));
}
diff --git a/drivers/gpu/drm/i915/gt/selftest_hangcheck.c b/drivers/gpu/drm/i915/gt/selftest_hangcheck.c
index 7e2d99d..15d6343 100644
--- a/drivers/gpu/drm/i915/gt/selftest_hangcheck.c
+++ b/drivers/gpu/drm/i915/gt/selftest_hangcheck.c
@@ -471,7 +471,8 @@ static int igt_reset_nop_engine(void *arg)
count = 0;
st_engine_heartbeat_disable(engine);
- set_bit(I915_RESET_ENGINE + id, >->reset.flags);
+ GEM_BUG_ON(test_and_set_bit(I915_RESET_ENGINE + id,
+ >->reset.flags));
do {
int i;
@@ -528,7 +529,7 @@ static int igt_reset_nop_engine(void *arg)
break;
}
} while (time_before(jiffies, end_time));
- clear_bit(I915_RESET_ENGINE + id, >->reset.flags);
+ clear_and_wake_up_bit(I915_RESET_ENGINE + id, >->reset.flags);
st_engine_heartbeat_enable(engine);
pr_info("%s(%s): %d resets\n", __func__, engine->name, count);
@@ -582,7 +583,8 @@ static int igt_reset_fail_engine(void *arg)
}
st_engine_heartbeat_disable(engine);
- set_bit(I915_RESET_ENGINE + id, >->reset.flags);
+ GEM_BUG_ON(test_and_set_bit(I915_RESET_ENGINE + id,
+ >->reset.flags));
force_reset_timeout(engine);
err = intel_engine_reset(engine, NULL);
@@ -679,7 +681,7 @@ static int igt_reset_fail_engine(void *arg)
out:
pr_info("%s(%s): %d resets\n", __func__, engine->name, count);
skip:
- clear_bit(I915_RESET_ENGINE + id, >->reset.flags);
+ clear_and_wake_up_bit(I915_RESET_ENGINE + id, >->reset.flags);
st_engine_heartbeat_enable(engine);
intel_context_put(ce);
@@ -734,7 +736,8 @@ static int __igt_reset_engine(struct intel_gt *gt, bool active)
reset_engine_count = i915_reset_engine_count(global, engine);
st_engine_heartbeat_disable(engine);
- set_bit(I915_RESET_ENGINE + id, >->reset.flags);
+ GEM_BUG_ON(test_and_set_bit(I915_RESET_ENGINE + id,
+ >->reset.flags));
count = 0;
do {
struct i915_request *rq = NULL;
@@ -824,7 +827,7 @@ static int __igt_reset_engine(struct intel_gt *gt, bool active)
if (err)
break;
} while (time_before(jiffies, end_time));
- clear_bit(I915_RESET_ENGINE + id, >->reset.flags);
+ clear_and_wake_up_bit(I915_RESET_ENGINE + id, >->reset.flags);
st_engine_heartbeat_enable(engine);
pr_info("%s: Completed %lu %s resets\n",
engine->name, count, active ? "active" : "idle");
@@ -1042,7 +1045,8 @@ static int __igt_reset_engines(struct intel_gt *gt,
yield(); /* start all threads before we begin */
st_engine_heartbeat_disable_no_pm(engine);
- set_bit(I915_RESET_ENGINE + id, >->reset.flags);
+ GEM_BUG_ON(test_and_set_bit(I915_RESET_ENGINE + id,
+ >->reset.flags));
do {
struct i915_request *rq = NULL;
struct intel_selftest_saved_policy saved;
@@ -1165,7 +1169,7 @@ static int __igt_reset_engines(struct intel_gt *gt,
if (err)
break;
} while (time_before(jiffies, end_time));
- clear_bit(I915_RESET_ENGINE + id, >->reset.flags);
+ clear_and_wake_up_bit(I915_RESET_ENGINE + id, >->reset.flags);
st_engine_heartbeat_enable_no_pm(engine);
pr_info("i915_reset_engine(%s:%s): %lu resets\n",
@@ -2014,7 +2018,7 @@ int intel_hangcheck_live_selftests(struct drm_i915_private *i915)
SUBTEST(igt_reset_evict_fence),
SUBTEST(igt_handle_error),
};
- struct intel_gt *gt = &i915->gt;
+ struct intel_gt *gt = to_gt(i915);
intel_wakeref_t wakeref;
int err;
diff --git a/drivers/gpu/drm/i915/gt/selftest_lrc.c b/drivers/gpu/drm/i915/gt/selftest_lrc.c
index b0977a3..618c905 100644
--- a/drivers/gpu/drm/i915/gt/selftest_lrc.c
+++ b/drivers/gpu/drm/i915/gt/selftest_lrc.c
@@ -1847,5 +1847,5 @@ int intel_lrc_live_selftests(struct drm_i915_private *i915)
if (!HAS_LOGICAL_RING_CONTEXTS(i915))
return 0;
- return intel_gt_live_subtests(tests, &i915->gt);
+ return intel_gt_live_subtests(tests, to_gt(i915));
}
diff --git a/drivers/gpu/drm/i915/gt/selftest_migrate.c b/drivers/gpu/drm/i915/gt/selftest_migrate.c
index 12ef283..fa4293d 100644
--- a/drivers/gpu/drm/i915/gt/selftest_migrate.c
+++ b/drivers/gpu/drm/i915/gt/selftest_migrate.c
@@ -49,6 +49,7 @@ static int copy(struct intel_migrate *migrate,
if (IS_ERR(src))
return 0;
+ sz = src->base.size;
dst = i915_gem_object_create_internal(i915, sz);
if (IS_ERR(dst))
goto err_free_src;
@@ -441,7 +442,7 @@ int intel_migrate_live_selftests(struct drm_i915_private *i915)
SUBTEST(thread_global_copy),
SUBTEST(thread_global_clear),
};
- struct intel_gt *gt = &i915->gt;
+ struct intel_gt *gt = to_gt(i915);
if (!gt->migrate.context)
return 0;
@@ -464,7 +465,7 @@ create_init_lmem_internal(struct intel_gt *gt, size_t sz, bool try_lmem)
return obj;
}
- i915_gem_object_trylock(obj);
+ i915_gem_object_trylock(obj, NULL);
err = i915_gem_object_pin_pages(obj);
if (err) {
i915_gem_object_unlock(obj);
@@ -657,7 +658,7 @@ int intel_migrate_perf_selftests(struct drm_i915_private *i915)
SUBTEST(perf_clear_blt),
SUBTEST(perf_copy_blt),
};
- struct intel_gt *gt = &i915->gt;
+ struct intel_gt *gt = to_gt(i915);
if (intel_gt_is_wedged(gt))
return 0;
diff --git a/drivers/gpu/drm/i915/gt/selftest_mocs.c b/drivers/gpu/drm/i915/gt/selftest_mocs.c
index 13d25bf..c1d8613 100644
--- a/drivers/gpu/drm/i915/gt/selftest_mocs.c
+++ b/drivers/gpu/drm/i915/gt/selftest_mocs.c
@@ -451,5 +451,5 @@ int intel_mocs_live_selftests(struct drm_i915_private *i915)
if (!get_mocs_settings(i915, &table))
return 0;
- return intel_gt_live_subtests(tests, &i915->gt);
+ return intel_gt_live_subtests(tests, to_gt(i915));
}
diff --git a/drivers/gpu/drm/i915/gt/selftest_reset.c b/drivers/gpu/drm/i915/gt/selftest_reset.c
index 7a50c9f4..8a873f6 100644
--- a/drivers/gpu/drm/i915/gt/selftest_reset.c
+++ b/drivers/gpu/drm/i915/gt/selftest_reset.c
@@ -376,7 +376,7 @@ int intel_reset_live_selftests(struct drm_i915_private *i915)
SUBTEST(igt_atomic_reset),
SUBTEST(igt_atomic_engine_reset),
};
- struct intel_gt *gt = &i915->gt;
+ struct intel_gt *gt = to_gt(i915);
if (!intel_has_gpu_reset(gt))
return 0;
diff --git a/drivers/gpu/drm/i915/gt/selftest_ring_submission.c b/drivers/gpu/drm/i915/gt/selftest_ring_submission.c
index 0419544..70f9ac1 100644
--- a/drivers/gpu/drm/i915/gt/selftest_ring_submission.c
+++ b/drivers/gpu/drm/i915/gt/selftest_ring_submission.c
@@ -291,8 +291,8 @@ int intel_ring_submission_live_selftests(struct drm_i915_private *i915)
SUBTEST(live_ctx_switch_wa),
};
- if (i915->gt.submission_method > INTEL_SUBMISSION_RING)
+ if (to_gt(i915)->submission_method > INTEL_SUBMISSION_RING)
return 0;
- return intel_gt_live_subtests(tests, &i915->gt);
+ return intel_gt_live_subtests(tests, to_gt(i915));
}
diff --git a/drivers/gpu/drm/i915/gt/selftest_slpc.c b/drivers/gpu/drm/i915/gt/selftest_slpc.c
index 9334bad..b768cea 100644
--- a/drivers/gpu/drm/i915/gt/selftest_slpc.c
+++ b/drivers/gpu/drm/i915/gt/selftest_slpc.c
@@ -39,7 +39,7 @@ static int slpc_set_max_freq(struct intel_guc_slpc *slpc, u32 freq)
static int live_slpc_clamp_min(void *arg)
{
struct drm_i915_private *i915 = arg;
- struct intel_gt *gt = &i915->gt;
+ struct intel_gt *gt = to_gt(i915);
struct intel_guc_slpc *slpc = >->uc.guc.slpc;
struct intel_rps *rps = >->rps;
struct intel_engine_cs *engine;
@@ -166,7 +166,7 @@ static int live_slpc_clamp_min(void *arg)
static int live_slpc_clamp_max(void *arg)
{
struct drm_i915_private *i915 = arg;
- struct intel_gt *gt = &i915->gt;
+ struct intel_gt *gt = to_gt(i915);
struct intel_guc_slpc *slpc;
struct intel_rps *rps;
struct intel_engine_cs *engine;
@@ -304,7 +304,7 @@ int intel_slpc_live_selftests(struct drm_i915_private *i915)
SUBTEST(live_slpc_clamp_min),
};
- if (intel_gt_is_wedged(&i915->gt))
+ if (intel_gt_is_wedged(to_gt(i915)))
return 0;
return i915_live_subtests(tests, i915);
diff --git a/drivers/gpu/drm/i915/gt/selftest_timeline.c b/drivers/gpu/drm/i915/gt/selftest_timeline.c
index d0b6a3a..e2eb686 100644
--- a/drivers/gpu/drm/i915/gt/selftest_timeline.c
+++ b/drivers/gpu/drm/i915/gt/selftest_timeline.c
@@ -159,7 +159,7 @@ static int mock_hwsp_freelist(void *arg)
INIT_RADIX_TREE(&state.cachelines, GFP_KERNEL);
state.prng = I915_RND_STATE_INITIALIZER(i915_selftest.random_seed);
- state.gt = &i915->gt;
+ state.gt = to_gt(i915);
/*
* Create a bunch of timelines and check that their HWSP do not overlap.
@@ -1416,8 +1416,8 @@ int intel_timeline_live_selftests(struct drm_i915_private *i915)
SUBTEST(live_hwsp_rollover_user),
};
- if (intel_gt_is_wedged(&i915->gt))
+ if (intel_gt_is_wedged(to_gt(i915)))
return 0;
- return intel_gt_live_subtests(tests, &i915->gt);
+ return intel_gt_live_subtests(tests, to_gt(i915));
}
diff --git a/drivers/gpu/drm/i915/gt/selftest_workarounds.c b/drivers/gpu/drm/i915/gt/selftest_workarounds.c
index 962e91b..0287c25 100644
--- a/drivers/gpu/drm/i915/gt/selftest_workarounds.c
+++ b/drivers/gpu/drm/i915/gt/selftest_workarounds.c
@@ -1387,8 +1387,8 @@ int intel_workarounds_live_selftests(struct drm_i915_private *i915)
SUBTEST(live_engine_reset_workarounds),
};
- if (intel_gt_is_wedged(&i915->gt))
+ if (intel_gt_is_wedged(to_gt(i915)))
return 0;
- return intel_gt_live_subtests(tests, &i915->gt);
+ return intel_gt_live_subtests(tests, to_gt(i915));
}
diff --git a/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h b/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h
index ba10bd3..fe5d7d2 100644
--- a/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h
+++ b/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h
@@ -144,6 +144,7 @@ enum intel_guc_action {
INTEL_GUC_ACTION_DEREGISTER_CONTEXT_DONE = 0x4600,
INTEL_GUC_ACTION_REGISTER_CONTEXT_MULTI_LRC = 0x4601,
INTEL_GUC_ACTION_RESET_CLIENT = 0x5507,
+ INTEL_GUC_ACTION_SET_ENG_UTIL_BUFF = 0x550A,
INTEL_GUC_ACTION_LIMIT
};
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc.h b/drivers/gpu/drm/i915/gt/uc/intel_guc.h
index 31cf9fb..f9240d4 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc.h
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc.h
@@ -95,6 +95,11 @@ struct intel_guc {
*/
struct ida guc_ids;
/**
+ * @num_guc_ids: Number of guc_ids, selftest feature to be able
+ * to reduce this number while testing.
+ */
+ int num_guc_ids;
+ /**
* @guc_ids_bitmap: used to allocate new guc_ids, multi-lrc
*/
unsigned long *guc_ids_bitmap;
@@ -138,6 +143,8 @@ struct intel_guc {
u32 ads_regset_size;
/** @ads_golden_ctxt_size: size of the golden contexts in the ADS */
u32 ads_golden_ctxt_size;
+ /** @ads_engine_usage_size: size of engine usage in the ADS */
+ u32 ads_engine_usage_size;
/** @lrc_desc_pool: object allocated to hold the GuC LRC descriptor pool */
struct i915_vma *lrc_desc_pool;
@@ -172,6 +179,41 @@ struct intel_guc {
/** @send_mutex: used to serialize the intel_guc_send actions */
struct mutex send_mutex;
+
+ /**
+ * @timestamp: GT timestamp object that stores a copy of the timestamp
+ * and adjusts it for overflow using a worker.
+ */
+ struct {
+ /**
+ * @lock: Lock protecting the below fields and the engine stats.
+ */
+ spinlock_t lock;
+
+ /**
+ * @gt_stamp: 64 bit extended value of the GT timestamp.
+ */
+ u64 gt_stamp;
+
+ /**
+ * @ping_delay: Period for polling the GT timestamp for
+ * overflow.
+ */
+ unsigned long ping_delay;
+
+ /**
+ * @work: Periodic work to adjust GT timestamp, engine and
+ * context usage for overflows.
+ */
+ struct delayed_work work;
+ } timestamp;
+
+#ifdef CONFIG_DRM_I915_SELFTEST
+ /**
+ * @number_guc_id_stolen: The number of guc_ids that have been stolen
+ */
+ int number_guc_id_stolen;
+#endif
};
static inline struct intel_guc *log_to_guc(struct intel_guc_log *log)
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_ads.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_ads.c
index 621c893..1a1edae 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_ads.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_ads.c
@@ -26,6 +26,8 @@
* | guc_policies |
* +---------------------------------------+
* | guc_gt_system_info |
+ * +---------------------------------------+
+ * | guc_engine_usage |
* +---------------------------------------+ <== static
* | guc_mmio_reg[countA] (engine 0.0) |
* | guc_mmio_reg[countB] (engine 0.1) |
@@ -47,6 +49,7 @@ struct __guc_ads_blob {
struct guc_ads ads;
struct guc_policies policies;
struct guc_gt_system_info system_info;
+ struct guc_engine_usage engine_usage;
/* From here on, location is dynamic! Refer to above diagram. */
struct guc_mmio_reg regset[0];
} __packed;
@@ -628,3 +631,21 @@ void intel_guc_ads_reset(struct intel_guc *guc)
guc_ads_private_data_reset(guc);
}
+
+u32 intel_guc_engine_usage_offset(struct intel_guc *guc)
+{
+ struct __guc_ads_blob *blob = guc->ads_blob;
+ u32 base = intel_guc_ggtt_offset(guc, guc->ads_vma);
+ u32 offset = base + ptr_offset(blob, engine_usage);
+
+ return offset;
+}
+
+struct guc_engine_usage_record *intel_guc_engine_usage(struct intel_engine_cs *engine)
+{
+ struct intel_guc *guc = &engine->gt->uc.guc;
+ struct __guc_ads_blob *blob = guc->ads_blob;
+ u8 guc_class = engine_class_to_guc_class(engine->class);
+
+ return &blob->engine_usage.engines[guc_class][ilog2(engine->logical_mask)];
+}
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_ads.h b/drivers/gpu/drm/i915/gt/uc/intel_guc_ads.h
index 3d85051d..e74c110 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_ads.h
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_ads.h
@@ -6,8 +6,11 @@
#ifndef _INTEL_GUC_ADS_H_
#define _INTEL_GUC_ADS_H_
+#include <linux/types.h>
+
struct intel_guc;
struct drm_printer;
+struct intel_engine_cs;
int intel_guc_ads_create(struct intel_guc *guc);
void intel_guc_ads_destroy(struct intel_guc *guc);
@@ -15,5 +18,7 @@ void intel_guc_ads_init_late(struct intel_guc *guc);
void intel_guc_ads_reset(struct intel_guc *guc);
void intel_guc_ads_print_policy_info(struct intel_guc *guc,
struct drm_printer *p);
+struct guc_engine_usage_record *intel_guc_engine_usage(struct intel_engine_cs *engine);
+u32 intel_guc_engine_usage_offset(struct intel_guc *guc);
#endif
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c
index a0cc34b..aa6dd64 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c
@@ -523,6 +523,15 @@ static inline bool ct_deadlocked(struct intel_guc_ct *ct)
CT_ERROR(ct, "Communication stalled for %lld ms, desc status=%#x,%#x\n",
ktime_ms_delta(ktime_get(), ct->stall_time),
send->status, recv->status);
+ CT_ERROR(ct, "H2G Space: %u (Bytes)\n",
+ atomic_read(&ct->ctbs.send.space) * 4);
+ CT_ERROR(ct, "Head: %u (Dwords)\n", ct->ctbs.send.desc->head);
+ CT_ERROR(ct, "Tail: %u (Dwords)\n", ct->ctbs.send.desc->tail);
+ CT_ERROR(ct, "G2H Space: %u (Bytes)\n",
+ atomic_read(&ct->ctbs.recv.space) * 4);
+ CT_ERROR(ct, "Head: %u\n (Dwords)", ct->ctbs.recv.desc->head);
+ CT_ERROR(ct, "Tail: %u\n (Dwords)", ct->ctbs.recv.desc->tail);
+
ct->ctbs.send.broken = true;
}
@@ -582,12 +591,19 @@ static inline bool h2g_has_room(struct intel_guc_ct *ct, u32 len_dw)
static int has_room_nb(struct intel_guc_ct *ct, u32 h2g_dw, u32 g2h_dw)
{
+ bool h2g = h2g_has_room(ct, h2g_dw);
+ bool g2h = g2h_has_room(ct, g2h_dw);
+
lockdep_assert_held(&ct->ctbs.send.lock);
- if (unlikely(!h2g_has_room(ct, h2g_dw) || !g2h_has_room(ct, g2h_dw))) {
+ if (unlikely(!h2g || !g2h)) {
if (ct->stall_time == KTIME_MAX)
ct->stall_time = ktime_get();
+ /* Be paranoid and kick G2H tasklet to free credits */
+ if (!g2h)
+ tasklet_hi_schedule(&ct->receive_tasklet);
+
if (unlikely(ct_deadlocked(ct)))
return -EPIPE;
else
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_fw.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_fw.c
index 196424be..31420ce 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_fw.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_fw.c
@@ -40,9 +40,8 @@ static void guc_prepare_xfer(struct intel_uncore *uncore)
}
}
-/* Copy RSA signature from the fw image to HW for verification */
-static int guc_xfer_rsa(struct intel_uc_fw *guc_fw,
- struct intel_uncore *uncore)
+static int guc_xfer_rsa_mmio(struct intel_uc_fw *guc_fw,
+ struct intel_uncore *uncore)
{
u32 rsa[UOS_RSA_SCRATCH_COUNT];
size_t copied;
@@ -58,6 +57,27 @@ static int guc_xfer_rsa(struct intel_uc_fw *guc_fw,
return 0;
}
+static int guc_xfer_rsa_vma(struct intel_uc_fw *guc_fw,
+ struct intel_uncore *uncore)
+{
+ struct intel_guc *guc = container_of(guc_fw, struct intel_guc, fw);
+
+ intel_uncore_write(uncore, UOS_RSA_SCRATCH(0),
+ intel_guc_ggtt_offset(guc, guc_fw->rsa_data));
+
+ return 0;
+}
+
+/* Copy RSA signature from the fw image to HW for verification */
+static int guc_xfer_rsa(struct intel_uc_fw *guc_fw,
+ struct intel_uncore *uncore)
+{
+ if (guc_fw->rsa_data)
+ return guc_xfer_rsa_vma(guc_fw, uncore);
+ else
+ return guc_xfer_rsa_mmio(guc_fw, uncore);
+}
+
/*
* Read the GuC status register (GUC_STATUS) and store it in the
* specified location; then return a boolean indicating whether
@@ -142,7 +162,10 @@ int intel_guc_fw_upload(struct intel_guc *guc)
/*
* Note that GuC needs the CSS header plus uKernel code to be copied
* by the DMA engine in one operation, whereas the RSA signature is
- * loaded via MMIO.
+ * loaded separately, either by copying it to the UOS_RSA_SCRATCH
+ * register (if key size <= 256) or through a ggtt-pinned vma (if key
+ * size > 256). The RSA size and therefore the way we provide it to the
+ * HW is fixed for each platform and hard-coded in the bootrom.
*/
ret = guc_xfer_rsa(&guc->fw, uncore);
if (ret)
@@ -164,6 +187,6 @@ int intel_guc_fw_upload(struct intel_guc *guc)
return 0;
out:
- intel_uc_fw_change_status(&guc->fw, INTEL_UC_FIRMWARE_FAIL);
+ intel_uc_fw_change_status(&guc->fw, INTEL_UC_FIRMWARE_LOAD_FAIL);
return ret;
}
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h b/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
index 722933e..7072e30 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
@@ -294,6 +294,19 @@ struct guc_ads {
u32 reserved[15];
} __packed;
+/* Engine usage stats */
+struct guc_engine_usage_record {
+ u32 current_context_index;
+ u32 last_switch_in_stamp;
+ u32 reserved0;
+ u32 total_runtime;
+ u32 reserved1[4];
+} __packed;
+
+struct guc_engine_usage {
+ struct guc_engine_usage_record engines[GUC_MAX_ENGINE_CLASSES][GUC_MAX_INSTANCES_PER_CLASS];
+} __packed;
+
/* GuC logging structures */
enum guc_log_buffer_type {
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_log.h b/drivers/gpu/drm/i915/gt/uc/intel_guc_log.h
index ac1ee1d..fe6ab75 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_log.h
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_log.h
@@ -15,9 +15,12 @@
struct intel_guc;
-#ifdef CONFIG_DRM_I915_DEBUG_GUC
+#if defined(CONFIG_DRM_I915_DEBUG_GUC)
#define CRASH_BUFFER_SIZE SZ_2M
#define DEBUG_BUFFER_SIZE SZ_16M
+#elif defined(CONFIG_DRM_I915_DEBUG_GEM)
+#define CRASH_BUFFER_SIZE SZ_1M
+#define DEBUG_BUFFER_SIZE SZ_2M
#else
#define CRASH_BUFFER_SIZE SZ_8K
#define DEBUG_BUFFER_SIZE SZ_64K
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_log_debugfs.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_log_debugfs.c
index 46026c2..ddfbe33 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_log_debugfs.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_log_debugfs.c
@@ -10,28 +10,80 @@
#include "intel_guc.h"
#include "intel_guc_log.h"
#include "intel_guc_log_debugfs.h"
+#include "intel_uc.h"
+
+static u32 obj_to_guc_log_dump_size(struct drm_i915_gem_object *obj)
+{
+ u32 size;
+
+ if (!obj)
+ return PAGE_SIZE;
+
+ /* "0x%08x 0x%08x 0x%08x 0x%08x\n" => 16 bytes -> 44 chars => x2.75 */
+ size = ((obj->base.size * 11) + 3) / 4;
+
+ /* Add padding for final blank line, any extra header info, etc. */
+ size = PAGE_ALIGN(size + PAGE_SIZE);
+
+ return size;
+}
+
+static u32 guc_log_dump_size(struct intel_guc_log *log)
+{
+ struct intel_guc *guc = log_to_guc(log);
+
+ if (!intel_guc_is_supported(guc))
+ return PAGE_SIZE;
+
+ if (!log->vma)
+ return PAGE_SIZE;
+
+ return obj_to_guc_log_dump_size(log->vma->obj);
+}
static int guc_log_dump_show(struct seq_file *m, void *data)
{
struct drm_printer p = drm_seq_file_printer(m);
+ int ret;
- return intel_guc_log_dump(m->private, &p, false);
+ ret = intel_guc_log_dump(m->private, &p, false);
+
+ if (IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM) && seq_has_overflowed(m))
+ pr_warn_once("preallocated size:%zx for %s exceeded\n",
+ m->size, __func__);
+
+ return ret;
}
-DEFINE_INTEL_GT_DEBUGFS_ATTRIBUTE(guc_log_dump);
+DEFINE_INTEL_GT_DEBUGFS_ATTRIBUTE_WITH_SIZE(guc_log_dump, guc_log_dump_size);
+
+static u32 guc_load_err_dump_size(struct intel_guc_log *log)
+{
+ struct intel_guc *guc = log_to_guc(log);
+ struct intel_uc *uc = container_of(guc, struct intel_uc, guc);
+
+ if (!intel_guc_is_supported(guc))
+ return PAGE_SIZE;
+
+ return obj_to_guc_log_dump_size(uc->load_err_log);
+}
static int guc_load_err_log_dump_show(struct seq_file *m, void *data)
{
struct drm_printer p = drm_seq_file_printer(m);
+ if (IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM) && seq_has_overflowed(m))
+ pr_warn_once("preallocated size:%zx for %s exceeded\n",
+ m->size, __func__);
+
return intel_guc_log_dump(m->private, &p, true);
}
-DEFINE_INTEL_GT_DEBUGFS_ATTRIBUTE(guc_load_err_log_dump);
+DEFINE_INTEL_GT_DEBUGFS_ATTRIBUTE_WITH_SIZE(guc_load_err_log_dump, guc_load_err_dump_size);
static int guc_log_level_get(void *data, u64 *val)
{
struct intel_guc_log *log = data;
- if (!intel_guc_is_used(log_to_guc(log)))
+ if (!log->vma)
return -ENODEV;
*val = intel_guc_log_get_level(log);
@@ -43,7 +95,7 @@ static int guc_log_level_set(void *data, u64 val)
{
struct intel_guc_log *log = data;
- if (!intel_guc_is_used(log_to_guc(log)))
+ if (!log->vma)
return -ENODEV;
return intel_guc_log_set_level(log, val);
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_slpc.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_slpc.c
index 65a3e7f..13b27b8 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_slpc.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_slpc.c
@@ -79,29 +79,6 @@ static void slpc_mem_set_disabled(struct slpc_shared_data *data,
slpc_mem_set_param(data, enable_id, 0);
}
-int intel_guc_slpc_init(struct intel_guc_slpc *slpc)
-{
- struct intel_guc *guc = slpc_to_guc(slpc);
- struct drm_i915_private *i915 = slpc_to_i915(slpc);
- u32 size = PAGE_ALIGN(sizeof(struct slpc_shared_data));
- int err;
-
- GEM_BUG_ON(slpc->vma);
-
- err = intel_guc_allocate_and_map_vma(guc, size, &slpc->vma, (void **)&slpc->vaddr);
- if (unlikely(err)) {
- drm_err(&i915->drm,
- "Failed to allocate SLPC struct (err=%pe)\n",
- ERR_PTR(err));
- return err;
- }
-
- slpc->max_freq_softlimit = 0;
- slpc->min_freq_softlimit = 0;
-
- return err;
-}
-
static u32 slpc_get_state(struct intel_guc_slpc *slpc)
{
struct slpc_shared_data *data;
@@ -203,6 +180,86 @@ static int slpc_unset_param(struct intel_guc_slpc *slpc,
return guc_action_slpc_unset_param(guc, id);
}
+static int slpc_force_min_freq(struct intel_guc_slpc *slpc, u32 freq)
+{
+ struct drm_i915_private *i915 = slpc_to_i915(slpc);
+ struct intel_guc *guc = slpc_to_guc(slpc);
+ intel_wakeref_t wakeref;
+ int ret = 0;
+
+ lockdep_assert_held(&slpc->lock);
+
+ if (!intel_guc_is_ready(guc))
+ return -ENODEV;
+
+ /*
+ * This function is a little different as compared to
+ * intel_guc_slpc_set_min_freq(). Softlimit will not be updated
+ * here since this is used to temporarily change min freq,
+ * for example, during a waitboost. Caller is responsible for
+ * checking bounds.
+ */
+
+ with_intel_runtime_pm(&i915->runtime_pm, wakeref) {
+ ret = slpc_set_param(slpc,
+ SLPC_PARAM_GLOBAL_MIN_GT_UNSLICE_FREQ_MHZ,
+ freq);
+ if (ret)
+ drm_err(&i915->drm, "Unable to force min freq to %u: %d",
+ freq, ret);
+ }
+
+ return ret;
+}
+
+static void slpc_boost_work(struct work_struct *work)
+{
+ struct intel_guc_slpc *slpc = container_of(work, typeof(*slpc), boost_work);
+
+ /*
+ * Raise min freq to boost. It's possible that
+ * this is greater than current max. But it will
+ * certainly be limited by RP0. An error setting
+ * the min param is not fatal.
+ */
+ mutex_lock(&slpc->lock);
+ if (atomic_read(&slpc->num_waiters)) {
+ slpc_force_min_freq(slpc, slpc->boost_freq);
+ slpc->num_boosts++;
+ }
+ mutex_unlock(&slpc->lock);
+}
+
+int intel_guc_slpc_init(struct intel_guc_slpc *slpc)
+{
+ struct intel_guc *guc = slpc_to_guc(slpc);
+ struct drm_i915_private *i915 = slpc_to_i915(slpc);
+ u32 size = PAGE_ALIGN(sizeof(struct slpc_shared_data));
+ int err;
+
+ GEM_BUG_ON(slpc->vma);
+
+ err = intel_guc_allocate_and_map_vma(guc, size, &slpc->vma, (void **)&slpc->vaddr);
+ if (unlikely(err)) {
+ drm_err(&i915->drm,
+ "Failed to allocate SLPC struct (err=%pe)\n",
+ ERR_PTR(err));
+ return err;
+ }
+
+ slpc->max_freq_softlimit = 0;
+ slpc->min_freq_softlimit = 0;
+
+ slpc->boost_freq = 0;
+ atomic_set(&slpc->num_waiters, 0);
+ slpc->num_boosts = 0;
+
+ mutex_init(&slpc->lock);
+ INIT_WORK(&slpc->boost_work, slpc_boost_work);
+
+ return err;
+}
+
static const char *slpc_global_state_to_string(enum slpc_global_state state)
{
switch (state) {
@@ -393,7 +450,11 @@ int intel_guc_slpc_set_min_freq(struct intel_guc_slpc *slpc, u32 val)
val > slpc->max_freq_softlimit)
return -EINVAL;
+ /* Need a lock now since waitboost can be modifying min as well */
+ mutex_lock(&slpc->lock);
+
with_intel_runtime_pm(&i915->runtime_pm, wakeref) {
+
ret = slpc_set_param(slpc,
SLPC_PARAM_GLOBAL_MIN_GT_UNSLICE_FREQ_MHZ,
val);
@@ -406,6 +467,8 @@ int intel_guc_slpc_set_min_freq(struct intel_guc_slpc *slpc, u32 val)
if (!ret)
slpc->min_freq_softlimit = val;
+ mutex_unlock(&slpc->lock);
+
return ret;
}
@@ -522,6 +585,9 @@ static void slpc_get_rp_values(struct intel_guc_slpc *slpc)
GT_FREQUENCY_MULTIPLIER;
slpc->min_freq = REG_FIELD_GET(RPN_CAP_MASK, rp_state_cap) *
GT_FREQUENCY_MULTIPLIER;
+
+ if (!slpc->boost_freq)
+ slpc->boost_freq = slpc->rp0_freq;
}
/*
@@ -557,7 +623,7 @@ int intel_guc_slpc_enable(struct intel_guc_slpc *slpc)
if (unlikely(ret < 0))
return ret;
- intel_guc_pm_intrmsk_enable(&i915->gt);
+ intel_guc_pm_intrmsk_enable(to_gt(i915));
slpc_get_rp_values(slpc);
@@ -588,6 +654,47 @@ int intel_guc_slpc_enable(struct intel_guc_slpc *slpc)
return 0;
}
+int intel_guc_slpc_set_boost_freq(struct intel_guc_slpc *slpc, u32 val)
+{
+ int ret = 0;
+
+ if (val < slpc->min_freq || val > slpc->rp0_freq)
+ return -EINVAL;
+
+ mutex_lock(&slpc->lock);
+
+ if (slpc->boost_freq != val) {
+ /* Apply only if there are active waiters */
+ if (atomic_read(&slpc->num_waiters)) {
+ ret = slpc_force_min_freq(slpc, val);
+ if (ret) {
+ ret = -EIO;
+ goto done;
+ }
+ }
+
+ slpc->boost_freq = val;
+ }
+
+done:
+ mutex_unlock(&slpc->lock);
+ return ret;
+}
+
+void intel_guc_slpc_dec_waiters(struct intel_guc_slpc *slpc)
+{
+ /*
+ * Return min back to the softlimit.
+ * This is called during request retire,
+ * so we don't need to fail that if the
+ * set_param fails.
+ */
+ mutex_lock(&slpc->lock);
+ if (atomic_dec_and_test(&slpc->num_waiters))
+ slpc_force_min_freq(slpc, slpc->min_freq_softlimit);
+ mutex_unlock(&slpc->lock);
+}
+
int intel_guc_slpc_print_info(struct intel_guc_slpc *slpc, struct drm_printer *p)
{
struct drm_i915_private *i915 = slpc_to_i915(slpc);
@@ -611,6 +718,8 @@ int intel_guc_slpc_print_info(struct intel_guc_slpc *slpc, struct drm_printer *p
slpc_decode_max_freq(slpc));
drm_printf(p, "\tMin freq: %u MHz\n",
slpc_decode_min_freq(slpc));
+ drm_printf(p, "\twaitboosts: %u\n",
+ slpc->num_boosts);
}
}
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_slpc.h b/drivers/gpu/drm/i915/gt/uc/intel_guc_slpc.h
index e45054d..0caa8fe 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_slpc.h
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_slpc.h
@@ -34,9 +34,12 @@ int intel_guc_slpc_enable(struct intel_guc_slpc *slpc);
void intel_guc_slpc_fini(struct intel_guc_slpc *slpc);
int intel_guc_slpc_set_max_freq(struct intel_guc_slpc *slpc, u32 val);
int intel_guc_slpc_set_min_freq(struct intel_guc_slpc *slpc, u32 val);
+int intel_guc_slpc_set_boost_freq(struct intel_guc_slpc *slpc, u32 val);
int intel_guc_slpc_get_max_freq(struct intel_guc_slpc *slpc, u32 *val);
int intel_guc_slpc_get_min_freq(struct intel_guc_slpc *slpc, u32 *val);
int intel_guc_slpc_print_info(struct intel_guc_slpc *slpc, struct drm_printer *p);
void intel_guc_pm_intrmsk_enable(struct intel_gt *gt);
+void intel_guc_slpc_boost(struct intel_guc_slpc *slpc);
+void intel_guc_slpc_dec_waiters(struct intel_guc_slpc *slpc);
#endif
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_slpc_types.h b/drivers/gpu/drm/i915/gt/uc/intel_guc_slpc_types.h
index 41d1352..bf5b9a5 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_slpc_types.h
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_slpc_types.h
@@ -6,6 +6,9 @@
#ifndef _INTEL_GUC_SLPC_TYPES_H_
#define _INTEL_GUC_SLPC_TYPES_H_
+#include <linux/atomic.h>
+#include <linux/workqueue.h>
+#include <linux/mutex.h>
#include <linux/types.h>
#define SLPC_RESET_TIMEOUT_MS 5
@@ -20,10 +23,20 @@ struct intel_guc_slpc {
u32 min_freq;
u32 rp0_freq;
u32 rp1_freq;
+ u32 boost_freq;
/* frequency softlimits */
u32 min_freq_softlimit;
u32 max_freq_softlimit;
+
+ /* Protects set/reset of boost freq
+ * and value of num_waiters
+ */
+ struct mutex lock;
+
+ struct work_struct boost_work;
+ atomic_t num_waiters;
+ u32 num_boosts;
};
#endif
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index 302e9ff..e751720 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -13,6 +13,7 @@
#include "gt/intel_engine_heartbeat.h"
#include "gt/intel_gpu_commands.h"
#include "gt/intel_gt.h"
+#include "gt/intel_gt_clock_utils.h"
#include "gt/intel_gt_irq.h"
#include "gt/intel_gt_pm.h"
#include "gt/intel_gt_requests.h"
@@ -21,6 +22,7 @@
#include "gt/intel_mocs.h"
#include "gt/intel_ring.h"
+#include "intel_guc_ads.h"
#include "intel_guc_submission.h"
#include "i915_drv.h"
@@ -143,7 +145,8 @@ guc_create_parallel(struct intel_engine_cs **engines,
* use should be low and 1/16 should be sufficient. Minimum of 32 guc_ids for
* multi-lrc.
*/
-#define NUMBER_MULTI_LRC_GUC_ID (GUC_MAX_LRC_DESCRIPTORS / 16)
+#define NUMBER_MULTI_LRC_GUC_ID(guc) \
+ ((guc)->submission_state.num_guc_ids / 16)
/*
* Below is a set of functions which control the GuC scheduling state which
@@ -1038,8 +1041,6 @@ static void scrub_guc_desc_for_outstanding_g2h(struct intel_guc *guc)
spin_unlock(&ce->guc_state.lock);
- GEM_BUG_ON(!do_put && !destroyed);
-
if (pending_enable || destroyed || deregister) {
decr_outstanding_submission_g2h(guc);
if (deregister)
@@ -1077,6 +1078,271 @@ static void scrub_guc_desc_for_outstanding_g2h(struct intel_guc *guc)
xa_unlock_irqrestore(&guc->context_lookup, flags);
}
+/*
+ * GuC stores busyness stats for each engine at context in/out boundaries. A
+ * context 'in' logs execution start time, 'out' adds in -> out delta to total.
+ * i915/kmd accesses 'start', 'total' and 'context id' from memory shared with
+ * GuC.
+ *
+ * __i915_pmu_event_read samples engine busyness. When sampling, if context id
+ * is valid (!= ~0) and start is non-zero, the engine is considered to be
+ * active. For an active engine total busyness = total + (now - start), where
+ * 'now' is the time at which the busyness is sampled. For inactive engine,
+ * total busyness = total.
+ *
+ * All times are captured from GUCPMTIMESTAMP reg and are in gt clock domain.
+ *
+ * The start and total values provided by GuC are 32 bits and wrap around in a
+ * few minutes. Since perf pmu provides busyness as 64 bit monotonically
+ * increasing ns values, there is a need for this implementation to account for
+ * overflows and extend the GuC provided values to 64 bits before returning
+ * busyness to the user. In order to do that, a worker runs periodically at
+ * frequency = 1/8th the time it takes for the timestamp to wrap (i.e. once in
+ * 27 seconds for a gt clock frequency of 19.2 MHz).
+ */
+
+#define WRAP_TIME_CLKS U32_MAX
+#define POLL_TIME_CLKS (WRAP_TIME_CLKS >> 3)
+
+static void
+__extend_last_switch(struct intel_guc *guc, u64 *prev_start, u32 new_start)
+{
+ u32 gt_stamp_hi = upper_32_bits(guc->timestamp.gt_stamp);
+ u32 gt_stamp_last = lower_32_bits(guc->timestamp.gt_stamp);
+
+ if (new_start == lower_32_bits(*prev_start))
+ return;
+
+ if (new_start < gt_stamp_last &&
+ (new_start - gt_stamp_last) <= POLL_TIME_CLKS)
+ gt_stamp_hi++;
+
+ if (new_start > gt_stamp_last &&
+ (gt_stamp_last - new_start) <= POLL_TIME_CLKS && gt_stamp_hi)
+ gt_stamp_hi--;
+
+ *prev_start = ((u64)gt_stamp_hi << 32) | new_start;
+}
+
+static void guc_update_engine_gt_clks(struct intel_engine_cs *engine)
+{
+ struct guc_engine_usage_record *rec = intel_guc_engine_usage(engine);
+ struct intel_engine_guc_stats *stats = &engine->stats.guc;
+ struct intel_guc *guc = &engine->gt->uc.guc;
+ u32 last_switch = rec->last_switch_in_stamp;
+ u32 ctx_id = rec->current_context_index;
+ u32 total = rec->total_runtime;
+
+ lockdep_assert_held(&guc->timestamp.lock);
+
+ stats->running = ctx_id != ~0U && last_switch;
+ if (stats->running)
+ __extend_last_switch(guc, &stats->start_gt_clk, last_switch);
+
+ /*
+ * Instead of adjusting the total for overflow, just add the
+ * difference from previous sample stats->total_gt_clks
+ */
+ if (total && total != ~0U) {
+ stats->total_gt_clks += (u32)(total - stats->prev_total);
+ stats->prev_total = total;
+ }
+}
+
+static void guc_update_pm_timestamp(struct intel_guc *guc,
+ struct intel_engine_cs *engine,
+ ktime_t *now)
+{
+ u32 gt_stamp_now, gt_stamp_hi;
+
+ lockdep_assert_held(&guc->timestamp.lock);
+
+ gt_stamp_hi = upper_32_bits(guc->timestamp.gt_stamp);
+ gt_stamp_now = intel_uncore_read(engine->uncore,
+ RING_TIMESTAMP(engine->mmio_base));
+ *now = ktime_get();
+
+ if (gt_stamp_now < lower_32_bits(guc->timestamp.gt_stamp))
+ gt_stamp_hi++;
+
+ guc->timestamp.gt_stamp = ((u64)gt_stamp_hi << 32) | gt_stamp_now;
+}
+
+/*
+ * Unlike the execlist mode of submission total and active times are in terms of
+ * gt clocks. The *now parameter is retained to return the cpu time at which the
+ * busyness was sampled.
+ */
+static ktime_t guc_engine_busyness(struct intel_engine_cs *engine, ktime_t *now)
+{
+ struct intel_engine_guc_stats stats_saved, *stats = &engine->stats.guc;
+ struct i915_gpu_error *gpu_error = &engine->i915->gpu_error;
+ struct intel_gt *gt = engine->gt;
+ struct intel_guc *guc = >->uc.guc;
+ u64 total, gt_stamp_saved;
+ unsigned long flags;
+ u32 reset_count;
+ bool in_reset;
+
+ spin_lock_irqsave(&guc->timestamp.lock, flags);
+
+ /*
+ * If a reset happened, we risk reading partially updated engine
+ * busyness from GuC, so we just use the driver stored copy of busyness.
+ * Synchronize with gt reset using reset_count and the
+ * I915_RESET_BACKOFF flag. Note that reset flow updates the reset_count
+ * after I915_RESET_BACKOFF flag, so ensure that the reset_count is
+ * usable by checking the flag afterwards.
+ */
+ reset_count = i915_reset_count(gpu_error);
+ in_reset = test_bit(I915_RESET_BACKOFF, >->reset.flags);
+
+ *now = ktime_get();
+
+ /*
+ * The active busyness depends on start_gt_clk and gt_stamp.
+ * gt_stamp is updated by i915 only when gt is awake and the
+ * start_gt_clk is derived from GuC state. To get a consistent
+ * view of activity, we query the GuC state only if gt is awake.
+ */
+ if (!in_reset && intel_gt_pm_get_if_awake(gt)) {
+ stats_saved = *stats;
+ gt_stamp_saved = guc->timestamp.gt_stamp;
+ guc_update_engine_gt_clks(engine);
+ guc_update_pm_timestamp(guc, engine, now);
+ intel_gt_pm_put_async(gt);
+ if (i915_reset_count(gpu_error) != reset_count) {
+ *stats = stats_saved;
+ guc->timestamp.gt_stamp = gt_stamp_saved;
+ }
+ }
+
+ total = intel_gt_clock_interval_to_ns(gt, stats->total_gt_clks);
+ if (stats->running) {
+ u64 clk = guc->timestamp.gt_stamp - stats->start_gt_clk;
+
+ total += intel_gt_clock_interval_to_ns(gt, clk);
+ }
+
+ spin_unlock_irqrestore(&guc->timestamp.lock, flags);
+
+ return ns_to_ktime(total);
+}
+
+static void __reset_guc_busyness_stats(struct intel_guc *guc)
+{
+ struct intel_gt *gt = guc_to_gt(guc);
+ struct intel_engine_cs *engine;
+ enum intel_engine_id id;
+ unsigned long flags;
+ ktime_t unused;
+
+ cancel_delayed_work_sync(&guc->timestamp.work);
+
+ spin_lock_irqsave(&guc->timestamp.lock, flags);
+
+ for_each_engine(engine, gt, id) {
+ guc_update_pm_timestamp(guc, engine, &unused);
+ guc_update_engine_gt_clks(engine);
+ engine->stats.guc.prev_total = 0;
+ }
+
+ spin_unlock_irqrestore(&guc->timestamp.lock, flags);
+}
+
+static void __update_guc_busyness_stats(struct intel_guc *guc)
+{
+ struct intel_gt *gt = guc_to_gt(guc);
+ struct intel_engine_cs *engine;
+ enum intel_engine_id id;
+ unsigned long flags;
+ ktime_t unused;
+
+ spin_lock_irqsave(&guc->timestamp.lock, flags);
+ for_each_engine(engine, gt, id) {
+ guc_update_pm_timestamp(guc, engine, &unused);
+ guc_update_engine_gt_clks(engine);
+ }
+ spin_unlock_irqrestore(&guc->timestamp.lock, flags);
+}
+
+static void guc_timestamp_ping(struct work_struct *wrk)
+{
+ struct intel_guc *guc = container_of(wrk, typeof(*guc),
+ timestamp.work.work);
+ struct intel_uc *uc = container_of(guc, typeof(*uc), guc);
+ struct intel_gt *gt = guc_to_gt(guc);
+ intel_wakeref_t wakeref;
+ int srcu, ret;
+
+ /*
+ * Synchronize with gt reset to make sure the worker does not
+ * corrupt the engine/guc stats.
+ */
+ ret = intel_gt_reset_trylock(gt, &srcu);
+ if (ret)
+ return;
+
+ with_intel_runtime_pm(>->i915->runtime_pm, wakeref)
+ __update_guc_busyness_stats(guc);
+
+ intel_gt_reset_unlock(gt, srcu);
+
+ mod_delayed_work(system_highpri_wq, &guc->timestamp.work,
+ guc->timestamp.ping_delay);
+}
+
+static int guc_action_enable_usage_stats(struct intel_guc *guc)
+{
+ u32 offset = intel_guc_engine_usage_offset(guc);
+ u32 action[] = {
+ INTEL_GUC_ACTION_SET_ENG_UTIL_BUFF,
+ offset,
+ 0,
+ };
+
+ return intel_guc_send(guc, action, ARRAY_SIZE(action));
+}
+
+static void guc_init_engine_stats(struct intel_guc *guc)
+{
+ struct intel_gt *gt = guc_to_gt(guc);
+ intel_wakeref_t wakeref;
+
+ mod_delayed_work(system_highpri_wq, &guc->timestamp.work,
+ guc->timestamp.ping_delay);
+
+ with_intel_runtime_pm(>->i915->runtime_pm, wakeref) {
+ int ret = guc_action_enable_usage_stats(guc);
+
+ if (ret)
+ drm_err(>->i915->drm,
+ "Failed to enable usage stats: %d!\n", ret);
+ }
+}
+
+void intel_guc_busyness_park(struct intel_gt *gt)
+{
+ struct intel_guc *guc = >->uc.guc;
+
+ if (!guc_submission_initialized(guc))
+ return;
+
+ cancel_delayed_work(&guc->timestamp.work);
+ __update_guc_busyness_stats(guc);
+}
+
+void intel_guc_busyness_unpark(struct intel_gt *gt)
+{
+ struct intel_guc *guc = >->uc.guc;
+
+ if (!guc_submission_initialized(guc))
+ return;
+
+ mod_delayed_work(system_highpri_wq, &guc->timestamp.work,
+ guc->timestamp.ping_delay);
+}
+
static inline bool
submission_disabled(struct intel_guc *guc)
{
@@ -1138,6 +1404,7 @@ void intel_guc_submission_reset_prepare(struct intel_guc *guc)
intel_gt_park_heartbeats(guc_to_gt(guc));
disable_submission(guc);
guc->interrupts.disable(guc);
+ __reset_guc_busyness_stats(guc);
/* Flush IRQ handler */
spin_lock_irq(&guc_to_gt(guc)->irq_lock);
@@ -1484,6 +1751,7 @@ static void destroyed_worker_func(struct work_struct *w);
*/
int intel_guc_submission_init(struct intel_guc *guc)
{
+ struct intel_gt *gt = guc_to_gt(guc);
int ret;
if (guc->lrc_desc_pool)
@@ -1508,10 +1776,14 @@ int intel_guc_submission_init(struct intel_guc *guc)
destroyed_worker_func);
guc->submission_state.guc_ids_bitmap =
- bitmap_zalloc(NUMBER_MULTI_LRC_GUC_ID, GFP_KERNEL);
+ bitmap_zalloc(NUMBER_MULTI_LRC_GUC_ID(guc), GFP_KERNEL);
if (!guc->submission_state.guc_ids_bitmap)
return -ENOMEM;
+ spin_lock_init(&guc->timestamp.lock);
+ INIT_DELAYED_WORK(&guc->timestamp.work, guc_timestamp_ping);
+ guc->timestamp.ping_delay = (POLL_TIME_CLKS / gt->clock_frequency + 1) * HZ;
+
return 0;
}
@@ -1598,13 +1870,13 @@ static int new_guc_id(struct intel_guc *guc, struct intel_context *ce)
if (intel_context_is_parent(ce))
ret = bitmap_find_free_region(guc->submission_state.guc_ids_bitmap,
- NUMBER_MULTI_LRC_GUC_ID,
+ NUMBER_MULTI_LRC_GUC_ID(guc),
order_base_2(ce->parallel.number_children
+ 1));
else
ret = ida_simple_get(&guc->submission_state.guc_ids,
- NUMBER_MULTI_LRC_GUC_ID,
- GUC_MAX_LRC_DESCRIPTORS,
+ NUMBER_MULTI_LRC_GUC_ID(guc),
+ guc->submission_state.num_guc_ids,
GFP_KERNEL | __GFP_RETRY_MAYFAIL |
__GFP_NOWARN);
if (unlikely(ret < 0))
@@ -1670,6 +1942,10 @@ static int steal_guc_id(struct intel_guc *guc, struct intel_context *ce)
set_context_guc_id_invalid(cn);
+#ifdef CONFIG_DRM_I915_SELFTEST
+ guc->number_guc_id_stolen++;
+#endif
+
return 0;
} else {
return -EAGAIN;
@@ -2373,7 +2649,6 @@ static inline void guc_lrc_desc_unpin(struct intel_context *ce)
unsigned long flags;
bool disabled;
- lockdep_assert_held(&guc->submission_state.lock);
GEM_BUG_ON(!intel_gt_pm_is_awake(gt));
GEM_BUG_ON(!lrc_desc_registered(guc, ce->guc_id.id));
GEM_BUG_ON(ce != __get_context(guc, ce->guc_id.id));
@@ -2389,7 +2664,7 @@ static inline void guc_lrc_desc_unpin(struct intel_context *ce)
}
spin_unlock_irqrestore(&ce->guc_state.lock, flags);
if (unlikely(disabled)) {
- __release_guc_id(guc, ce);
+ release_guc_id(guc, ce);
__guc_context_destroy(ce);
return;
}
@@ -2423,36 +2698,48 @@ static void __guc_context_destroy(struct intel_context *ce)
static void guc_flush_destroyed_contexts(struct intel_guc *guc)
{
- struct intel_context *ce, *cn;
+ struct intel_context *ce;
unsigned long flags;
GEM_BUG_ON(!submission_disabled(guc) &&
guc_submission_initialized(guc));
- spin_lock_irqsave(&guc->submission_state.lock, flags);
- list_for_each_entry_safe(ce, cn,
- &guc->submission_state.destroyed_contexts,
- destroyed_link) {
- list_del_init(&ce->destroyed_link);
- __release_guc_id(guc, ce);
+ while (!list_empty(&guc->submission_state.destroyed_contexts)) {
+ spin_lock_irqsave(&guc->submission_state.lock, flags);
+ ce = list_first_entry_or_null(&guc->submission_state.destroyed_contexts,
+ struct intel_context,
+ destroyed_link);
+ if (ce)
+ list_del_init(&ce->destroyed_link);
+ spin_unlock_irqrestore(&guc->submission_state.lock, flags);
+
+ if (!ce)
+ break;
+
+ release_guc_id(guc, ce);
__guc_context_destroy(ce);
}
- spin_unlock_irqrestore(&guc->submission_state.lock, flags);
}
static void deregister_destroyed_contexts(struct intel_guc *guc)
{
- struct intel_context *ce, *cn;
+ struct intel_context *ce;
unsigned long flags;
- spin_lock_irqsave(&guc->submission_state.lock, flags);
- list_for_each_entry_safe(ce, cn,
- &guc->submission_state.destroyed_contexts,
- destroyed_link) {
- list_del_init(&ce->destroyed_link);
+ while (!list_empty(&guc->submission_state.destroyed_contexts)) {
+ spin_lock_irqsave(&guc->submission_state.lock, flags);
+ ce = list_first_entry_or_null(&guc->submission_state.destroyed_contexts,
+ struct intel_context,
+ destroyed_link);
+ if (ce)
+ list_del_init(&ce->destroyed_link);
+ spin_unlock_irqrestore(&guc->submission_state.lock, flags);
+
+ if (!ce)
+ break;
+
guc_lrc_desc_unpin(ce);
}
- spin_unlock_irqrestore(&guc->submission_state.lock, flags);
}
static void destroyed_worker_func(struct work_struct *w)
@@ -3369,7 +3656,9 @@ static void guc_default_vfuncs(struct intel_engine_cs *engine)
engine->emit_flush = gen12_emit_flush_xcs;
}
engine->set_default_submission = guc_set_default_submission;
+ engine->busyness = guc_engine_busyness;
+ engine->flags |= I915_ENGINE_SUPPORTS_STATS;
engine->flags |= I915_ENGINE_HAS_PREEMPTION;
engine->flags |= I915_ENGINE_HAS_TIMESLICES;
@@ -3468,6 +3757,7 @@ int intel_guc_submission_setup(struct intel_engine_cs *engine)
void intel_guc_submission_enable(struct intel_guc *guc)
{
guc_init_lrc_mapping(guc);
+ guc_init_engine_stats(guc);
}
void intel_guc_submission_disable(struct intel_guc *guc)
@@ -3494,6 +3784,7 @@ static bool __guc_submission_selected(struct intel_guc *guc)
void intel_guc_submission_init_early(struct intel_guc *guc)
{
+ guc->submission_state.num_guc_ids = GUC_MAX_LRC_DESCRIPTORS;
guc->submission_supported = __guc_submission_supported(guc);
guc->submission_selected = __guc_submission_selected(guc);
}
@@ -3695,6 +3986,7 @@ int intel_guc_context_reset_process_msg(struct intel_guc *guc,
const u32 *msg, u32 len)
{
struct intel_context *ce;
+ unsigned long flags;
int desc_idx;
if (unlikely(len != 1)) {
@@ -3703,11 +3995,24 @@ int intel_guc_context_reset_process_msg(struct intel_guc *guc,
}
desc_idx = msg[0];
+
+ /*
+ * The context lookup uses the xarray but lookups only require an RCU lock
+ * not the full spinlock. So take the lock explicitly and keep it until the
+ * context has been reference count locked to ensure it can't be destroyed
+ * asynchronously until the reset is done.
+ */
+ xa_lock_irqsave(&guc->context_lookup, flags);
ce = g2h_context_lookup(guc, desc_idx);
+ if (ce)
+ intel_context_get(ce);
+ xa_unlock_irqrestore(&guc->context_lookup, flags);
+
if (unlikely(!ce))
return -EPROTO;
guc_handle_context_reset(guc, ce);
+ intel_context_put(ce);
return 0;
}
@@ -3728,11 +4033,12 @@ int intel_guc_engine_failure_process_msg(struct intel_guc *guc,
const u32 *msg, u32 len)
{
struct intel_engine_cs *engine;
+ struct intel_gt *gt = guc_to_gt(guc);
u8 guc_class, instance;
u32 reason;
if (unlikely(len != 3)) {
- drm_err(&guc_to_gt(guc)->i915->drm, "Invalid length %u", len);
+ drm_err(>->i915->drm, "Invalid length %u", len);
return -EPROTO;
}
@@ -3742,12 +4048,19 @@ int intel_guc_engine_failure_process_msg(struct intel_guc *guc,
engine = guc_lookup_engine(guc, guc_class, instance);
if (unlikely(!engine)) {
- drm_err(&guc_to_gt(guc)->i915->drm,
+ drm_err(>->i915->drm,
"Invalid engine %d:%d", guc_class, instance);
return -EPROTO;
}
- intel_gt_handle_error(guc_to_gt(guc), engine->mask,
+ /*
+ * This is an unexpected failure of a hardware feature. So, log a real
+ * error message not just the informational that comes with the reset.
+ */
+ drm_err(>->i915->drm, "GuC engine reset request failed on %d:%d (%s) because 0x%08X",
+ guc_class, instance, engine->name, reason);
+
+ intel_gt_handle_error(gt, engine->mask,
I915_ERROR_CAPTURE,
"GuC failed to reset %s (reason=0x%08x)\n",
engine->name, reason);
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.h b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.h
index c7ef44f..5a95a9f 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.h
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.h
@@ -28,6 +28,8 @@ void intel_guc_submission_print_context_info(struct intel_guc *guc,
void intel_guc_dump_active_requests(struct intel_engine_cs *engine,
struct i915_request *hung_rq,
struct drm_printer *m);
+void intel_guc_busyness_park(struct intel_gt *gt);
+void intel_guc_busyness_unpark(struct intel_gt *gt);
bool intel_guc_virtual_engine_has_heartbeat(const struct intel_engine_cs *ve);
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_huc.c b/drivers/gpu/drm/i915/gt/uc/intel_huc.c
index ff4b686..d10b227 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_huc.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_huc.c
@@ -54,65 +54,6 @@ void intel_huc_init_early(struct intel_huc *huc)
}
}
-static int intel_huc_rsa_data_create(struct intel_huc *huc)
-{
- struct intel_gt *gt = huc_to_gt(huc);
- struct intel_guc *guc = >->uc.guc;
- struct i915_vma *vma;
- size_t copied;
- void *vaddr;
- int err;
-
- err = i915_inject_probe_error(gt->i915, -ENXIO);
- if (err)
- return err;
-
- /*
- * HuC firmware will sit above GUC_GGTT_TOP and will not map
- * through GTT. Unfortunately, this means GuC cannot perform
- * the HuC auth. as the rsa offset now falls within the GuC
- * inaccessible range. We resort to perma-pinning an additional
- * vma within the accessible range that only contains the rsa
- * signature. The GuC can use this extra pinning to perform
- * the authentication since its GGTT offset will be GuC
- * accessible.
- */
- GEM_BUG_ON(huc->fw.rsa_size > PAGE_SIZE);
- vma = intel_guc_allocate_vma(guc, PAGE_SIZE);
- if (IS_ERR(vma))
- return PTR_ERR(vma);
-
- vaddr = i915_gem_object_pin_map_unlocked(vma->obj,
- i915_coherent_map_type(gt->i915,
- vma->obj, true));
- if (IS_ERR(vaddr)) {
- i915_vma_unpin_and_release(&vma, 0);
- err = PTR_ERR(vaddr);
- goto unpin_out;
- }
-
- copied = intel_uc_fw_copy_rsa(&huc->fw, vaddr, vma->size);
- i915_gem_object_unpin_map(vma->obj);
-
- if (copied < huc->fw.rsa_size) {
- err = -ENOMEM;
- goto unpin_out;
- }
-
- huc->rsa_data = vma;
-
- return 0;
-
-unpin_out:
- i915_vma_unpin_and_release(&vma, 0);
- return err;
-}
-
-static void intel_huc_rsa_data_destroy(struct intel_huc *huc)
-{
- i915_vma_unpin_and_release(&huc->rsa_data, 0);
-}
-
int intel_huc_init(struct intel_huc *huc)
{
struct drm_i915_private *i915 = huc_to_gt(huc)->i915;
@@ -122,21 +63,10 @@ int intel_huc_init(struct intel_huc *huc)
if (err)
goto out;
- /*
- * HuC firmware image is outside GuC accessible range.
- * Copy the RSA signature out of the image into
- * a perma-pinned region set aside for it
- */
- err = intel_huc_rsa_data_create(huc);
- if (err)
- goto out_fini;
-
intel_uc_fw_change_status(&huc->fw, INTEL_UC_FIRMWARE_LOADABLE);
return 0;
-out_fini:
- intel_uc_fw_fini(&huc->fw);
out:
i915_probe_error(i915, "failed with %d\n", err);
return err;
@@ -147,7 +77,6 @@ void intel_huc_fini(struct intel_huc *huc)
if (!intel_uc_fw_is_loadable(&huc->fw))
return;
- intel_huc_rsa_data_destroy(huc);
intel_uc_fw_fini(&huc->fw);
}
@@ -177,7 +106,7 @@ int intel_huc_auth(struct intel_huc *huc)
goto fail;
ret = intel_guc_auth_huc(guc,
- intel_guc_ggtt_offset(guc, huc->rsa_data));
+ intel_guc_ggtt_offset(guc, huc->fw.rsa_data));
if (ret) {
DRM_ERROR("HuC: GuC did not ack Auth request %d\n", ret);
goto fail;
@@ -199,7 +128,7 @@ int intel_huc_auth(struct intel_huc *huc)
fail:
i915_probe_error(gt->i915, "HuC: Authentication failed %d\n", ret);
- intel_uc_fw_change_status(&huc->fw, INTEL_UC_FIRMWARE_FAIL);
+ intel_uc_fw_change_status(&huc->fw, INTEL_UC_FIRMWARE_LOAD_FAIL);
return ret;
}
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_huc.h b/drivers/gpu/drm/i915/gt/uc/intel_huc.h
index daee43b..ae8c8a6 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_huc.h
+++ b/drivers/gpu/drm/i915/gt/uc/intel_huc.h
@@ -15,8 +15,6 @@ struct intel_huc {
struct intel_uc_fw fw;
/* HuC-specific additions */
- struct i915_vma *rsa_data;
-
struct {
i915_reg_t reg;
u32 mask;
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_uc.c b/drivers/gpu/drm/i915/gt/uc/intel_uc.c
index 2fef3b0..09ed29d 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_uc.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_uc.c
@@ -8,6 +8,7 @@
#include "intel_guc.h"
#include "intel_guc_ads.h"
#include "intel_guc_submission.h"
+#include "gt/intel_rps.h"
#include "intel_uc.h"
#include "i915_drv.h"
@@ -35,7 +36,7 @@ static void uc_expand_default_options(struct intel_uc *uc)
}
/* Intermediate platforms are HuC authentication only */
- if (IS_ALDERLAKE_S(i915)) {
+ if (IS_ALDERLAKE_S(i915) && !IS_ADLS_RPLS(i915)) {
i915->params.enable_guc = ENABLE_GUC_LOAD_HUC;
return;
}
@@ -462,6 +463,8 @@ static int __uc_init_hw(struct intel_uc *uc)
else
attempts = 1;
+ intel_rps_raise_unslice(&uc_to_gt(uc)->rps);
+
while (attempts--) {
/*
* Always reset the GuC just before (re)loading, so
@@ -499,6 +502,9 @@ static int __uc_init_hw(struct intel_uc *uc)
ret = intel_guc_slpc_enable(&guc->slpc);
if (ret)
goto err_submission;
+ } else {
+ /* Restore GT back to RPn for non-SLPC path */
+ intel_rps_lower_unslice(&uc_to_gt(uc)->rps);
}
drm_info(&i915->drm, "%s firmware %s version %u.%u %s:%s\n",
@@ -529,6 +535,9 @@ static int __uc_init_hw(struct intel_uc *uc)
err_log_capture:
__uc_capture_load_err_log(uc);
err_out:
+ /* Return GT back to RPn */
+ intel_rps_lower_unslice(&uc_to_gt(uc)->rps);
+
__uc_sanitize(uc);
if (!ret) {
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c b/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c
index 3aa87be..a5af05b 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c
@@ -48,22 +48,39 @@ void intel_uc_fw_change_status(struct intel_uc_fw *uc_fw,
* Note that RKL and ADL-S have the same GuC/HuC device ID's and use the same
* firmware as TGL.
*/
-#define INTEL_UC_FIRMWARE_DEFS(fw_def, guc_def, huc_def) \
- fw_def(ALDERLAKE_P, 0, guc_def(adlp, 62, 0, 3), huc_def(tgl, 7, 9, 3)) \
- fw_def(ALDERLAKE_S, 0, guc_def(tgl, 62, 0, 0), huc_def(tgl, 7, 9, 3)) \
- fw_def(DG1, 0, guc_def(dg1, 62, 0, 0), huc_def(dg1, 7, 9, 3)) \
- fw_def(ROCKETLAKE, 0, guc_def(tgl, 62, 0, 0), huc_def(tgl, 7, 9, 3)) \
- fw_def(TIGERLAKE, 0, guc_def(tgl, 62, 0, 0), huc_def(tgl, 7, 9, 3)) \
- fw_def(JASPERLAKE, 0, guc_def(ehl, 62, 0, 0), huc_def(ehl, 9, 0, 0)) \
- fw_def(ELKHARTLAKE, 0, guc_def(ehl, 62, 0, 0), huc_def(ehl, 9, 0, 0)) \
- fw_def(ICELAKE, 0, guc_def(icl, 62, 0, 0), huc_def(icl, 9, 0, 0)) \
- fw_def(COMETLAKE, 5, guc_def(cml, 62, 0, 0), huc_def(cml, 4, 0, 0)) \
- fw_def(COMETLAKE, 0, guc_def(kbl, 62, 0, 0), huc_def(kbl, 4, 0, 0)) \
- fw_def(COFFEELAKE, 0, guc_def(kbl, 62, 0, 0), huc_def(kbl, 4, 0, 0)) \
- fw_def(GEMINILAKE, 0, guc_def(glk, 62, 0, 0), huc_def(glk, 4, 0, 0)) \
- fw_def(KABYLAKE, 0, guc_def(kbl, 62, 0, 0), huc_def(kbl, 4, 0, 0)) \
- fw_def(BROXTON, 0, guc_def(bxt, 62, 0, 0), huc_def(bxt, 2, 0, 0)) \
- fw_def(SKYLAKE, 0, guc_def(skl, 62, 0, 0), huc_def(skl, 2, 0, 0))
+#define INTEL_GUC_FIRMWARE_DEFS(fw_def, guc_def) \
+ fw_def(ALDERLAKE_P, 0, guc_def(adlp, 62, 0, 3)) \
+ fw_def(ALDERLAKE_S, 0, guc_def(tgl, 62, 0, 0)) \
+ fw_def(DG1, 0, guc_def(dg1, 62, 0, 0)) \
+ fw_def(ROCKETLAKE, 0, guc_def(tgl, 62, 0, 0)) \
+ fw_def(TIGERLAKE, 0, guc_def(tgl, 62, 0, 0)) \
+ fw_def(JASPERLAKE, 0, guc_def(ehl, 62, 0, 0)) \
+ fw_def(ELKHARTLAKE, 0, guc_def(ehl, 62, 0, 0)) \
+ fw_def(ICELAKE, 0, guc_def(icl, 62, 0, 0)) \
+ fw_def(COMETLAKE, 5, guc_def(cml, 62, 0, 0)) \
+ fw_def(COMETLAKE, 0, guc_def(kbl, 62, 0, 0)) \
+ fw_def(COFFEELAKE, 0, guc_def(kbl, 62, 0, 0)) \
+ fw_def(GEMINILAKE, 0, guc_def(glk, 62, 0, 0)) \
+ fw_def(KABYLAKE, 0, guc_def(kbl, 62, 0, 0)) \
+ fw_def(BROXTON, 0, guc_def(bxt, 62, 0, 0)) \
+ fw_def(SKYLAKE, 0, guc_def(skl, 62, 0, 0))
+
+#define INTEL_HUC_FIRMWARE_DEFS(fw_def, huc_def) \
+ fw_def(ALDERLAKE_P, 0, huc_def(tgl, 7, 9, 3)) \
+ fw_def(ALDERLAKE_S, 0, huc_def(tgl, 7, 9, 3)) \
+ fw_def(DG1, 0, huc_def(dg1, 7, 9, 3)) \
+ fw_def(ROCKETLAKE, 0, huc_def(tgl, 7, 9, 3)) \
+ fw_def(TIGERLAKE, 0, huc_def(tgl, 7, 9, 3)) \
+ fw_def(JASPERLAKE, 0, huc_def(ehl, 9, 0, 0)) \
+ fw_def(ELKHARTLAKE, 0, huc_def(ehl, 9, 0, 0)) \
+ fw_def(ICELAKE, 0, huc_def(icl, 9, 0, 0)) \
+ fw_def(COMETLAKE, 5, huc_def(cml, 4, 0, 0)) \
+ fw_def(COMETLAKE, 0, huc_def(kbl, 4, 0, 0)) \
+ fw_def(COFFEELAKE, 0, huc_def(kbl, 4, 0, 0)) \
+ fw_def(GEMINILAKE, 0, huc_def(glk, 4, 0, 0)) \
+ fw_def(KABYLAKE, 0, huc_def(kbl, 4, 0, 0)) \
+ fw_def(BROXTON, 0, huc_def(bxt, 2, 0, 0)) \
+ fw_def(SKYLAKE, 0, huc_def(skl, 2, 0, 0))
#define __MAKE_UC_FW_PATH(prefix_, name_, major_, minor_, patch_) \
"i915/" \
@@ -79,11 +96,11 @@ void intel_uc_fw_change_status(struct intel_uc_fw *uc_fw,
__MAKE_UC_FW_PATH(prefix_, "_huc_", major_, minor_, bld_num_)
/* All blobs need to be declared via MODULE_FIRMWARE() */
-#define INTEL_UC_MODULE_FW(platform_, revid_, guc_, huc_) \
- MODULE_FIRMWARE(guc_); \
- MODULE_FIRMWARE(huc_);
+#define INTEL_UC_MODULE_FW(platform_, revid_, uc_) \
+ MODULE_FIRMWARE(uc_);
-INTEL_UC_FIRMWARE_DEFS(INTEL_UC_MODULE_FW, MAKE_GUC_FW_PATH, MAKE_HUC_FW_PATH)
+INTEL_GUC_FIRMWARE_DEFS(INTEL_UC_MODULE_FW, MAKE_GUC_FW_PATH)
+INTEL_HUC_FIRMWARE_DEFS(INTEL_UC_MODULE_FW, MAKE_HUC_FW_PATH)
/* The below structs and macros are used to iterate across the list of blobs */
struct __packed uc_fw_blob {
@@ -106,31 +123,47 @@ struct __packed uc_fw_blob {
struct __packed uc_fw_platform_requirement {
enum intel_platform p;
u8 rev; /* first platform rev using this FW */
- const struct uc_fw_blob blobs[INTEL_UC_FW_NUM_TYPES];
+ const struct uc_fw_blob blob;
};
-#define MAKE_FW_LIST(platform_, revid_, guc_, huc_) \
+#define MAKE_FW_LIST(platform_, revid_, uc_) \
{ \
.p = INTEL_##platform_, \
.rev = revid_, \
- .blobs[INTEL_UC_FW_TYPE_GUC] = guc_, \
- .blobs[INTEL_UC_FW_TYPE_HUC] = huc_, \
+ .blob = uc_, \
},
+struct fw_blobs_by_type {
+ const struct uc_fw_platform_requirement *blobs;
+ u32 count;
+};
+
static void
__uc_fw_auto_select(struct drm_i915_private *i915, struct intel_uc_fw *uc_fw)
{
- static const struct uc_fw_platform_requirement fw_blobs[] = {
- INTEL_UC_FIRMWARE_DEFS(MAKE_FW_LIST, GUC_FW_BLOB, HUC_FW_BLOB)
+ static const struct uc_fw_platform_requirement blobs_guc[] = {
+ INTEL_GUC_FIRMWARE_DEFS(MAKE_FW_LIST, GUC_FW_BLOB)
};
+ static const struct uc_fw_platform_requirement blobs_huc[] = {
+ INTEL_HUC_FIRMWARE_DEFS(MAKE_FW_LIST, HUC_FW_BLOB)
+ };
+ static const struct fw_blobs_by_type blobs_all[INTEL_UC_FW_NUM_TYPES] = {
+ [INTEL_UC_FW_TYPE_GUC] = { blobs_guc, ARRAY_SIZE(blobs_guc) },
+ [INTEL_UC_FW_TYPE_HUC] = { blobs_huc, ARRAY_SIZE(blobs_huc) },
+ };
+ static const struct uc_fw_platform_requirement *fw_blobs;
enum intel_platform p = INTEL_INFO(i915)->platform;
+ u32 fw_count;
u8 rev = INTEL_REVID(i915);
int i;
- for (i = 0; i < ARRAY_SIZE(fw_blobs) && p <= fw_blobs[i].p; i++) {
+ GEM_BUG_ON(uc_fw->type >= ARRAY_SIZE(blobs_all));
+ fw_blobs = blobs_all[uc_fw->type].blobs;
+ fw_count = blobs_all[uc_fw->type].count;
+
+ for (i = 0; i < fw_count && p <= fw_blobs[i].p; i++) {
if (p == fw_blobs[i].p && rev >= fw_blobs[i].rev) {
- const struct uc_fw_blob *blob =
- &fw_blobs[i].blobs[uc_fw->type];
+ const struct uc_fw_blob *blob = &fw_blobs[i].blob;
uc_fw->path = blob->path;
uc_fw->major_ver_wanted = blob->major;
uc_fw->minor_ver_wanted = blob->minor;
@@ -140,7 +173,7 @@ __uc_fw_auto_select(struct drm_i915_private *i915, struct intel_uc_fw *uc_fw)
/* make sure the list is ordered as expected */
if (IS_ENABLED(CONFIG_DRM_I915_SELFTEST)) {
- for (i = 1; i < ARRAY_SIZE(fw_blobs); i++) {
+ for (i = 1; i < fw_count; i++) {
if (fw_blobs[i].p < fw_blobs[i - 1].p)
continue;
@@ -322,13 +355,6 @@ int intel_uc_fw_fetch(struct intel_uc_fw *uc_fw)
uc_fw->ucode_size = (css->size_dw - css->header_size_dw) * sizeof(u32);
/* now RSA */
- if (unlikely(css->key_size_dw != UOS_RSA_SCRATCH_COUNT)) {
- drm_warn(&i915->drm, "%s firmware %s: unexpected key size: %u != %u\n",
- intel_uc_fw_type_repr(uc_fw->type), uc_fw->path,
- css->key_size_dw, UOS_RSA_SCRATCH_COUNT);
- err = -EPROTO;
- goto fail;
- }
uc_fw->rsa_size = css->key_size_dw * sizeof(u32);
/* At least, it should have header, uCode and RSA. Size of all three. */
@@ -540,10 +566,79 @@ int intel_uc_fw_upload(struct intel_uc_fw *uc_fw, u32 dst_offset, u32 dma_flags)
i915_probe_error(gt->i915, "Failed to load %s firmware %s (%d)\n",
intel_uc_fw_type_repr(uc_fw->type), uc_fw->path,
err);
- intel_uc_fw_change_status(uc_fw, INTEL_UC_FIRMWARE_FAIL);
+ intel_uc_fw_change_status(uc_fw, INTEL_UC_FIRMWARE_LOAD_FAIL);
return err;
}
+static inline bool uc_fw_need_rsa_in_memory(struct intel_uc_fw *uc_fw)
+{
+ /*
+ * The HW reads the GuC RSA from memory if the key size is > 256 bytes,
+ * while it reads it from the 64 RSA registers if it is smaller.
+ * The HuC RSA is always read from memory.
+ */
+ return uc_fw->type == INTEL_UC_FW_TYPE_HUC || uc_fw->rsa_size > 256;
+}
+
+static int uc_fw_rsa_data_create(struct intel_uc_fw *uc_fw)
+{
+ struct intel_gt *gt = __uc_fw_to_gt(uc_fw);
+ struct i915_vma *vma;
+ size_t copied;
+ void *vaddr;
+ int err;
+
+ err = i915_inject_probe_error(gt->i915, -ENXIO);
+ if (err)
+ return err;
+
+ if (!uc_fw_need_rsa_in_memory(uc_fw))
+ return 0;
+
+ /*
+ * uC firmwares will sit above GUC_GGTT_TOP and will not map through
+ * GGTT. Unfortunately, this means that the GuC HW cannot perform the uC
+ * authentication from memory, as the RSA offset now falls within the
+ * GuC inaccessible range. We resort to perma-pinning an additional vma
+ * within the accessible range that only contains the RSA signature.
+ * The GuC HW can use this extra pinning to perform the authentication
+ * since its GGTT offset will be GuC accessible.
+ */
+ GEM_BUG_ON(uc_fw->rsa_size > PAGE_SIZE);
+ vma = intel_guc_allocate_vma(>->uc.guc, PAGE_SIZE);
+ if (IS_ERR(vma))
+ return PTR_ERR(vma);
+
+ vaddr = i915_gem_object_pin_map_unlocked(vma->obj,
+ i915_coherent_map_type(gt->i915, vma->obj, true));
+ if (IS_ERR(vaddr)) {
+ i915_vma_unpin_and_release(&vma, 0);
+ err = PTR_ERR(vaddr);
+ goto unpin_out;
+ }
+
+ copied = intel_uc_fw_copy_rsa(uc_fw, vaddr, vma->size);
+ i915_gem_object_unpin_map(vma->obj);
+
+ if (copied < uc_fw->rsa_size) {
+ err = -ENOMEM;
+ goto unpin_out;
+ }
+
+ uc_fw->rsa_data = vma;
+
+ return 0;
+
+unpin_out:
+ i915_vma_unpin_and_release(&vma, 0);
+ return err;
+}
+
+static void uc_fw_rsa_data_destroy(struct intel_uc_fw *uc_fw)
+{
+ i915_vma_unpin_and_release(&uc_fw->rsa_data, 0);
+}
+
int intel_uc_fw_init(struct intel_uc_fw *uc_fw)
{
int err;
@@ -558,14 +653,29 @@ int intel_uc_fw_init(struct intel_uc_fw *uc_fw)
if (err) {
DRM_DEBUG_DRIVER("%s fw pin-pages err=%d\n",
intel_uc_fw_type_repr(uc_fw->type), err);
- intel_uc_fw_change_status(uc_fw, INTEL_UC_FIRMWARE_FAIL);
+ goto out;
}
+ err = uc_fw_rsa_data_create(uc_fw);
+ if (err) {
+ DRM_DEBUG_DRIVER("%s fw rsa data creation failed, err=%d\n",
+ intel_uc_fw_type_repr(uc_fw->type), err);
+ goto out_unpin;
+ }
+
+ return 0;
+
+out_unpin:
+ i915_gem_object_unpin_pages(uc_fw->obj);
+out:
+ intel_uc_fw_change_status(uc_fw, INTEL_UC_FIRMWARE_INIT_FAIL);
return err;
}
void intel_uc_fw_fini(struct intel_uc_fw *uc_fw)
{
+ uc_fw_rsa_data_destroy(uc_fw);
+
if (i915_gem_object_has_pinned_pages(uc_fw->obj))
i915_gem_object_unpin_pages(uc_fw->obj);
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.h b/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.h
index 1e00bf6..d9d1dc0 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.h
+++ b/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.h
@@ -32,11 +32,12 @@ struct intel_gt;
* | | MISSING <--/ | \--> ERROR |
* | fetch | V |
* | | AVAILABLE |
- * +------------+- | -+
+ * +------------+- | \ -+
+ * | | | \--> INIT FAIL |
* | init | V |
* | | /------> LOADABLE <----<-----------\ |
* +------------+- \ / \ \ \ -+
- * | | FAIL <--< \--> TRANSFERRED \ |
+ * | | LOAD FAIL <--< \--> TRANSFERRED \ |
* | upload | \ / \ / |
* | | \---------/ \--> RUNNING |
* +------------+---------------------------------------------------+
@@ -50,8 +51,9 @@ enum intel_uc_fw_status {
INTEL_UC_FIRMWARE_MISSING, /* blob not found on the system */
INTEL_UC_FIRMWARE_ERROR, /* invalid format or version */
INTEL_UC_FIRMWARE_AVAILABLE, /* blob found and copied in mem */
+ INTEL_UC_FIRMWARE_INIT_FAIL, /* failed to prepare fw objects for load */
INTEL_UC_FIRMWARE_LOADABLE, /* all fw-required objects are ready */
- INTEL_UC_FIRMWARE_FAIL, /* failed to xfer or init/auth the fw */
+ INTEL_UC_FIRMWARE_LOAD_FAIL, /* failed to xfer or init/auth the fw */
INTEL_UC_FIRMWARE_TRANSFERRED, /* dma xfer done */
INTEL_UC_FIRMWARE_RUNNING /* init/auth done */
};
@@ -84,6 +86,7 @@ struct intel_uc_fw {
* or during a GT reset (mutex guarantees single threaded).
*/
struct i915_vma dummy;
+ struct i915_vma *rsa_data;
/*
* The firmware build process will generate a version header file with major and
@@ -130,10 +133,12 @@ const char *intel_uc_fw_status_repr(enum intel_uc_fw_status status)
return "ERROR";
case INTEL_UC_FIRMWARE_AVAILABLE:
return "AVAILABLE";
+ case INTEL_UC_FIRMWARE_INIT_FAIL:
+ return "INIT FAIL";
case INTEL_UC_FIRMWARE_LOADABLE:
return "LOADABLE";
- case INTEL_UC_FIRMWARE_FAIL:
- return "FAIL";
+ case INTEL_UC_FIRMWARE_LOAD_FAIL:
+ return "LOAD FAIL";
case INTEL_UC_FIRMWARE_TRANSFERRED:
return "TRANSFERRED";
case INTEL_UC_FIRMWARE_RUNNING:
@@ -155,7 +160,8 @@ static inline int intel_uc_fw_status_to_error(enum intel_uc_fw_status status)
return -ENOENT;
case INTEL_UC_FIRMWARE_ERROR:
return -ENOEXEC;
- case INTEL_UC_FIRMWARE_FAIL:
+ case INTEL_UC_FIRMWARE_INIT_FAIL:
+ case INTEL_UC_FIRMWARE_LOAD_FAIL:
return -EIO;
case INTEL_UC_FIRMWARE_SELECTED:
return -ESTALE;
diff --git a/drivers/gpu/drm/i915/gt/uc/selftest_guc.c b/drivers/gpu/drm/i915/gt/uc/selftest_guc.c
index fb0e4a7b..d3327b8 100644
--- a/drivers/gpu/drm/i915/gt/uc/selftest_guc.c
+++ b/drivers/gpu/drm/i915/gt/uc/selftest_guc.c
@@ -3,8 +3,21 @@
* Copyright �� 2021 Intel Corporation
*/
+#include "selftests/igt_spinner.h"
#include "selftests/intel_scheduler_helpers.h"
+static int request_add_spin(struct i915_request *rq, struct igt_spinner *spin)
+{
+ int err = 0;
+
+ i915_request_get(rq);
+ i915_request_add(rq);
+ if (spin && !igt_wait_for_spinner(spin, rq))
+ err = -ETIMEDOUT;
+
+ return err;
+}
+
static struct i915_request *nop_user_request(struct intel_context *ce,
struct i915_request *from)
{
@@ -110,12 +123,172 @@ static int intel_guc_scrub_ctbs(void *arg)
return ret;
}
+/*
+ * intel_guc_steal_guc_ids - Test to exhaust all guc_ids and then steal one
+ *
+ * This test creates a spinner which is used to block all subsequent submissions
+ * until it completes. Next, a loop creates a context and a NOP request each
+ * iteration until the guc_ids are exhausted (request creation returns -EAGAIN).
+ * The spinner is ended, unblocking all requests created in the loop. At this
+ * point all guc_ids are exhausted but are available to steal. Try to create
+ * another request which should successfully steal a guc_id. Wait on last
+ * request to complete, idle GPU, verify a guc_id was stolen via a counter, and
+ * exit the test. Test also artificially reduces the number of guc_ids so the
+ * test runs in a timely manner.
+ */
+static int intel_guc_steal_guc_ids(void *arg)
+{
+ struct intel_gt *gt = arg;
+ struct intel_guc *guc = >->uc.guc;
+ int ret, sv, context_index = 0;
+ intel_wakeref_t wakeref;
+ struct intel_engine_cs *engine;
+ struct intel_context **ce;
+ struct igt_spinner spin;
+ struct i915_request *spin_rq = NULL, *rq, *last = NULL;
+ int number_guc_id_stolen = guc->number_guc_id_stolen;
+
+ ce = kzalloc(sizeof(*ce) * GUC_MAX_LRC_DESCRIPTORS, GFP_KERNEL);
+ if (!ce) {
+ pr_err("Context array allocation failed\n");
+ return -ENOMEM;
+ }
+
+ wakeref = intel_runtime_pm_get(gt->uncore->rpm);
+ engine = intel_selftest_find_any_engine(gt);
+ sv = guc->submission_state.num_guc_ids;
+ guc->submission_state.num_guc_ids = 4096;
+
+ /* Create spinner to block requests in below loop */
+ ce[context_index] = intel_context_create(engine);
+ if (IS_ERR(ce[context_index])) {
+ ret = PTR_ERR(ce[context_index]);
+ ce[context_index] = NULL;
+ pr_err("Failed to create context: %d\n", ret);
+ goto err_wakeref;
+ }
+ ret = igt_spinner_init(&spin, engine->gt);
+ if (ret) {
+ pr_err("Failed to create spinner: %d\n", ret);
+ goto err_contexts;
+ }
+ spin_rq = igt_spinner_create_request(&spin, ce[context_index],
+ MI_ARB_CHECK);
+ if (IS_ERR(spin_rq)) {
+ ret = PTR_ERR(spin_rq);
+ pr_err("Failed to create spinner request: %d\n", ret);
+ goto err_contexts;
+ }
+ ret = request_add_spin(spin_rq, &spin);
+ if (ret) {
+ pr_err("Failed to add Spinner request: %d\n", ret);
+ goto err_spin_rq;
+ }
+
+ /* Use all guc_ids */
+ while (ret != -EAGAIN) {
+ ce[++context_index] = intel_context_create(engine);
+ if (IS_ERR(ce[context_index])) {
+ ret = PTR_ERR(ce[context_index--]);
+ ce[context_index] = NULL;
+ pr_err("Failed to create context: %d\n", ret);
+ goto err_spin_rq;
+ }
+
+ rq = nop_user_request(ce[context_index], spin_rq);
+ if (IS_ERR(rq)) {
+ ret = PTR_ERR(rq);
+ rq = NULL;
+ if (ret != -EAGAIN) {
+ pr_err("Failed to create request, %d: %d\n",
+ context_index, ret);
+ goto err_spin_rq;
+ }
+ } else {
+ if (last)
+ i915_request_put(last);
+ last = rq;
+ }
+ }
+
+ /* Release blocked requests */
+ igt_spinner_end(&spin);
+ ret = intel_selftest_wait_for_rq(spin_rq);
+ if (ret) {
+ pr_err("Spin request failed to complete: %d\n", ret);
+ i915_request_put(last);
+ goto err_spin_rq;
+ }
+ i915_request_put(spin_rq);
+ igt_spinner_fini(&spin);
+ spin_rq = NULL;
+
+ /* Wait for last request */
+ ret = i915_request_wait(last, 0, HZ * 30);
+ i915_request_put(last);
+ if (ret < 0) {
+ pr_err("Last request failed to complete: %d\n", ret);
+ goto err_spin_rq;
+ }
+
+ /* Try to steal guc_id */
+ rq = nop_user_request(ce[context_index], NULL);
+ if (IS_ERR(rq)) {
+ ret = PTR_ERR(rq);
+ pr_err("Failed to steal guc_id, %d: %d\n", context_index, ret);
+ goto err_spin_rq;
+ }
+
+ /* Wait for request with stolen guc_id */
+ ret = i915_request_wait(rq, 0, HZ);
+ i915_request_put(rq);
+ if (ret < 0) {
+ pr_err("Request with stolen guc_id failed to complete: %d\n",
+ ret);
+ goto err_spin_rq;
+ }
+
+ /* Wait for idle */
+ ret = intel_gt_wait_for_idle(gt, HZ * 30);
+ if (ret < 0) {
+ pr_err("GT failed to idle: %d\n", ret);
+ goto err_spin_rq;
+ }
+
+ /* Verify a guc_id was stolen */
+ if (guc->number_guc_id_stolen == number_guc_id_stolen) {
+ pr_err("No guc_id was stolen");
+ ret = -EINVAL;
+ } else {
+ ret = 0;
+ }
+
+err_spin_rq:
+ if (spin_rq) {
+ igt_spinner_end(&spin);
+ intel_selftest_wait_for_rq(spin_rq);
+ i915_request_put(spin_rq);
+ igt_spinner_fini(&spin);
+ intel_gt_wait_for_idle(gt, HZ * 30);
+ }
+err_contexts:
+ for (; context_index >= 0 && ce[context_index]; --context_index)
+ intel_context_put(ce[context_index]);
+err_wakeref:
+ intel_runtime_pm_put(gt->uncore->rpm, wakeref);
+ kfree(ce);
+ guc->submission_state.num_guc_ids = sv;
+
+ return ret;
+}
+
int intel_guc_live_selftests(struct drm_i915_private *i915)
{
static const struct i915_subtest tests[] = {
SUBTEST(intel_guc_scrub_ctbs),
+ SUBTEST(intel_guc_steal_guc_ids),
};
- struct intel_gt *gt = &i915->gt;
+ struct intel_gt *gt = to_gt(i915);
if (intel_gt_is_wedged(gt))
return 0;
diff --git a/drivers/gpu/drm/i915/gt/uc/selftest_guc_multi_lrc.c b/drivers/gpu/drm/i915/gt/uc/selftest_guc_multi_lrc.c
index 50953c8..1297ddb 100644
--- a/drivers/gpu/drm/i915/gt/uc/selftest_guc_multi_lrc.c
+++ b/drivers/gpu/drm/i915/gt/uc/selftest_guc_multi_lrc.c
@@ -167,7 +167,7 @@ int intel_guc_multi_lrc_live_selftests(struct drm_i915_private *i915)
static const struct i915_subtest tests[] = {
SUBTEST(intel_guc_multi_lrc_basic),
};
- struct intel_gt *gt = &i915->gt;
+ struct intel_gt *gt = to_gt(i915);
if (intel_gt_is_wedged(gt))
return 0;
diff --git a/drivers/gpu/drm/i915/gvt/fb_decoder.c b/drivers/gpu/drm/i915/gvt/fb_decoder.c
index 11a8bab..9ec0641 100644
--- a/drivers/gpu/drm/i915/gvt/fb_decoder.c
+++ b/drivers/gpu/drm/i915/gvt/fb_decoder.c
@@ -427,7 +427,7 @@ int intel_vgpu_decode_sprite_plane(struct intel_vgpu *vgpu,
plane->tiled = !!(val & SPRITE_TILED);
color_order = !!(val & SPRITE_RGB_ORDER_RGBX);
- yuv_order = (val & SPRITE_YUV_BYTE_ORDER_MASK) >>
+ yuv_order = (val & SPRITE_YUV_ORDER_MASK) >>
_SPRITE_YUV_ORDER_SHIFT;
fmt = (val & SPRITE_PIXFORMAT_MASK) >> _SPRITE_FMT_SHIFT;
diff --git a/drivers/gpu/drm/i915/gvt/gtt.c b/drivers/gpu/drm/i915/gvt/gtt.c
index 53d0cb3..99d1781 100644
--- a/drivers/gpu/drm/i915/gvt/gtt.c
+++ b/drivers/gpu/drm/i915/gvt/gtt.c
@@ -446,17 +446,17 @@ static bool gen8_gtt_test_present(struct intel_gvt_gtt_entry *e)
|| e->type == GTT_TYPE_PPGTT_ROOT_L4_ENTRY)
return (e->val64 != 0);
else
- return (e->val64 & _PAGE_PRESENT);
+ return (e->val64 & GEN8_PAGE_PRESENT);
}
static void gtt_entry_clear_present(struct intel_gvt_gtt_entry *e)
{
- e->val64 &= ~_PAGE_PRESENT;
+ e->val64 &= ~GEN8_PAGE_PRESENT;
}
static void gtt_entry_set_present(struct intel_gvt_gtt_entry *e)
{
- e->val64 |= _PAGE_PRESENT;
+ e->val64 |= GEN8_PAGE_PRESENT;
}
static bool gen8_gtt_test_64k_splited(struct intel_gvt_gtt_entry *e)
@@ -2439,7 +2439,7 @@ static int alloc_scratch_pages(struct intel_vgpu *vgpu,
/* The entry parameters like present/writeable/cache type
* set to the same as i915's scratch page tree.
*/
- se.val64 |= _PAGE_PRESENT | _PAGE_RW;
+ se.val64 |= GEN8_PAGE_PRESENT | GEN8_PAGE_RW;
if (type == GTT_TYPE_PPGTT_PDE_PT)
se.val64 |= PPAT_CACHED;
@@ -2896,7 +2896,7 @@ void intel_gvt_restore_ggtt(struct intel_gvt *gvt)
offset = vgpu_aperture_gmadr_base(vgpu) >> PAGE_SHIFT;
for (idx = 0; idx < num_low; idx++) {
pte = mm->ggtt_mm.host_ggtt_aperture[idx];
- if (pte & _PAGE_PRESENT)
+ if (pte & GEN8_PAGE_PRESENT)
write_pte64(vgpu->gvt->gt->ggtt, offset + idx, pte);
}
@@ -2904,7 +2904,7 @@ void intel_gvt_restore_ggtt(struct intel_gvt *gvt)
offset = vgpu_hidden_gmadr_base(vgpu) >> PAGE_SHIFT;
for (idx = 0; idx < num_hi; idx++) {
pte = mm->ggtt_mm.host_ggtt_hidden[idx];
- if (pte & _PAGE_PRESENT)
+ if (pte & GEN8_PAGE_PRESENT)
write_pte64(vgpu->gvt->gt->ggtt, offset + idx, pte);
}
}
diff --git a/drivers/gpu/drm/i915/gvt/gvt.c b/drivers/gpu/drm/i915/gvt/gvt.c
index cbac409..f0b69e4 100644
--- a/drivers/gpu/drm/i915/gvt/gvt.c
+++ b/drivers/gpu/drm/i915/gvt/gvt.c
@@ -205,7 +205,7 @@ int intel_gvt_init_device(struct drm_i915_private *i915)
spin_lock_init(&gvt->scheduler.mmio_context_lock);
mutex_init(&gvt->lock);
mutex_init(&gvt->sched_lock);
- gvt->gt = &i915->gt;
+ gvt->gt = to_gt(i915);
i915->gvt = gvt;
init_device_info(gvt);
diff --git a/drivers/gpu/drm/i915/gvt/scheduler.c b/drivers/gpu/drm/i915/gvt/scheduler.c
index 6c80410..42a0c9a 100644
--- a/drivers/gpu/drm/i915/gvt/scheduler.c
+++ b/drivers/gpu/drm/i915/gvt/scheduler.c
@@ -1386,7 +1386,7 @@ int intel_vgpu_setup_submission(struct intel_vgpu *vgpu)
enum intel_engine_id i;
int ret;
- ppgtt = i915_ppgtt_create(&i915->gt, I915_BO_ALLOC_PM_EARLY);
+ ppgtt = i915_ppgtt_create(to_gt(i915), I915_BO_ALLOC_PM_EARLY);
if (IS_ERR(ppgtt))
return PTR_ERR(ppgtt);
diff --git a/drivers/gpu/drm/i915/i915_active.c b/drivers/gpu/drm/i915/i915_active.c
index 3103c1e..ee2b3a3 100644
--- a/drivers/gpu/drm/i915/i915_active.c
+++ b/drivers/gpu/drm/i915/i915_active.c
@@ -426,8 +426,9 @@ replace_barrier(struct i915_active *ref, struct i915_active_fence *active)
return true;
}
-int i915_active_ref(struct i915_active *ref, u64 idx, struct dma_fence *fence)
+int i915_active_add_request(struct i915_active *ref, struct i915_request *rq)
{
+ struct dma_fence *fence = &rq->fence;
struct i915_active_fence *active;
int err;
@@ -436,7 +437,7 @@ int i915_active_ref(struct i915_active *ref, u64 idx, struct dma_fence *fence)
if (err)
return err;
- active = active_instance(ref, idx);
+ active = active_instance(ref, i915_request_timeline(rq)->fence_context);
if (!active) {
err = -ENOMEM;
goto out;
@@ -477,29 +478,6 @@ __i915_active_set_fence(struct i915_active *ref,
return prev;
}
-static struct i915_active_fence *
-__active_fence(struct i915_active *ref, u64 idx)
-{
- struct active_node *it;
-
- it = __active_lookup(ref, idx);
- if (unlikely(!it)) { /* Contention with parallel tree builders! */
- spin_lock_irq(&ref->tree_lock);
- it = __active_lookup(ref, idx);
- spin_unlock_irq(&ref->tree_lock);
- }
- GEM_BUG_ON(!it); /* slot must be preallocated */
-
- return &it->base;
-}
-
-struct dma_fence *
-__i915_active_ref(struct i915_active *ref, u64 idx, struct dma_fence *fence)
-{
- /* Only valid while active, see i915_active_acquire_for_context() */
- return __i915_active_set_fence(ref, __active_fence(ref, idx), fence);
-}
-
struct dma_fence *
i915_active_set_exclusive(struct i915_active *ref, struct dma_fence *f)
{
diff --git a/drivers/gpu/drm/i915/i915_active.h b/drivers/gpu/drm/i915/i915_active.h
index 5fcdb0e..7eb4413 100644
--- a/drivers/gpu/drm/i915/i915_active.h
+++ b/drivers/gpu/drm/i915/i915_active.h
@@ -164,26 +164,11 @@ void __i915_active_init(struct i915_active *ref,
__i915_active_init(ref, active, retire, flags, &__mkey, &__wkey); \
} while (0)
-struct dma_fence *
-__i915_active_ref(struct i915_active *ref, u64 idx, struct dma_fence *fence);
-int i915_active_ref(struct i915_active *ref, u64 idx, struct dma_fence *fence);
-
-static inline int
-i915_active_add_request(struct i915_active *ref, struct i915_request *rq)
-{
- return i915_active_ref(ref,
- i915_request_timeline(rq)->fence_context,
- &rq->fence);
-}
+int i915_active_add_request(struct i915_active *ref, struct i915_request *rq);
struct dma_fence *
i915_active_set_exclusive(struct i915_active *ref, struct dma_fence *f);
-static inline bool i915_active_has_exclusive(struct i915_active *ref)
-{
- return rcu_access_pointer(ref->excl.fence);
-}
-
int __i915_active_wait(struct i915_active *ref, int state);
static inline int i915_active_wait(struct i915_active *ref)
{
diff --git a/drivers/gpu/drm/i915/i915_active_types.h b/drivers/gpu/drm/i915/i915_active_types.h
index c149f34..b02a78a 100644
--- a/drivers/gpu/drm/i915/i915_active_types.h
+++ b/drivers/gpu/drm/i915/i915_active_types.h
@@ -15,8 +15,6 @@
#include <linux/rcupdate.h>
#include <linux/workqueue.h>
-#include "i915_utils.h"
-
struct i915_active_fence {
struct dma_fence __rcu *fence;
struct dma_fence_cb cb;
diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c
index fe638b5..e0e052c 100644
--- a/drivers/gpu/drm/i915/i915_debugfs.c
+++ b/drivers/gpu/drm/i915/i915_debugfs.c
@@ -48,7 +48,6 @@
#include "i915_debugfs_params.h"
#include "i915_irq.h"
#include "i915_scheduler.h"
-#include "i915_trace.h"
#include "intel_pm.h"
static inline struct drm_i915_private *node_to_i915(struct drm_info_node *node)
@@ -65,7 +64,8 @@ static int i915_capabilities(struct seq_file *m, void *data)
intel_device_info_print_static(INTEL_INFO(i915), &p);
intel_device_info_print_runtime(RUNTIME_INFO(i915), &p);
- intel_gt_info_print(&i915->gt.info, &p);
+ i915_print_iommu_status(i915, &p);
+ intel_gt_info_print(&to_gt(i915)->info, &p);
intel_driver_caps_print(&i915->caps, &p);
kernel_param_lock(THIS_MODULE);
@@ -293,7 +293,7 @@ static int i915_gpu_info_open(struct inode *inode, struct file *file)
gpu = NULL;
with_intel_runtime_pm(&i915->runtime_pm, wakeref)
- gpu = i915_gpu_coredump(&i915->gt, ALL_ENGINES);
+ gpu = i915_gpu_coredump(to_gt(i915), ALL_ENGINES);
if (IS_ERR(gpu))
return PTR_ERR(gpu);
@@ -351,7 +351,7 @@ static const struct file_operations i915_error_state_fops = {
static int i915_frequency_info(struct seq_file *m, void *unused)
{
struct drm_i915_private *i915 = node_to_i915(m->private);
- struct intel_gt *gt = &i915->gt;
+ struct intel_gt *gt = to_gt(i915);
struct drm_printer p = drm_seq_file_printer(m);
intel_gt_pm_frequency_dump(gt, &p);
@@ -439,11 +439,11 @@ static int i915_swizzle_info(struct seq_file *m, void *data)
static int i915_rps_boost_info(struct seq_file *m, void *data)
{
struct drm_i915_private *dev_priv = node_to_i915(m->private);
- struct intel_rps *rps = &dev_priv->gt.rps;
+ struct intel_rps *rps = &to_gt(dev_priv)->rps;
seq_printf(m, "RPS enabled? %s\n", yesno(intel_rps_is_enabled(rps)));
seq_printf(m, "RPS active? %s\n", yesno(intel_rps_is_active(rps)));
- seq_printf(m, "GPU busy? %s\n", yesno(dev_priv->gt.awake));
+ seq_printf(m, "GPU busy? %s\n", yesno(to_gt(dev_priv)->awake));
seq_printf(m, "Boosts outstanding? %d\n",
atomic_read(&rps->num_waiters));
seq_printf(m, "Interactive? %d\n", READ_ONCE(rps->power.interactive));
@@ -476,7 +476,7 @@ static int i915_runtime_pm_status(struct seq_file *m, void *unused)
seq_printf(m, "Runtime power status: %s\n",
enableddisabled(!dev_priv->power_domains.init_wakeref));
- seq_printf(m, "GPU idle: %s\n", yesno(!dev_priv->gt.awake));
+ seq_printf(m, "GPU idle: %s\n", yesno(!to_gt(dev_priv)->awake));
seq_printf(m, "IRQs disabled: %s\n",
yesno(!intel_irqs_enabled(dev_priv)));
#ifdef CONFIG_PM
@@ -508,18 +508,18 @@ static int i915_engine_info(struct seq_file *m, void *unused)
wakeref = intel_runtime_pm_get(&i915->runtime_pm);
seq_printf(m, "GT awake? %s [%d], %llums\n",
- yesno(i915->gt.awake),
- atomic_read(&i915->gt.wakeref.count),
- ktime_to_ms(intel_gt_get_awake_time(&i915->gt)));
+ yesno(to_gt(i915)->awake),
+ atomic_read(&to_gt(i915)->wakeref.count),
+ ktime_to_ms(intel_gt_get_awake_time(to_gt(i915))));
seq_printf(m, "CS timestamp frequency: %u Hz, %d ns\n",
- i915->gt.clock_frequency,
- i915->gt.clock_period_ns);
+ to_gt(i915)->clock_frequency,
+ to_gt(i915)->clock_period_ns);
p = drm_seq_file_printer(m);
for_each_uabi_engine(engine, i915)
intel_engine_dump(engine, &p, "%s\n", engine->name);
- intel_gt_show_timelines(&i915->gt, &p, i915_request_show_with_schedule);
+ intel_gt_show_timelines(to_gt(i915), &p, i915_request_show_with_schedule);
intel_runtime_pm_put(&i915->runtime_pm, wakeref);
@@ -558,14 +558,14 @@ static int i915_wedged_get(void *data, u64 *val)
{
struct drm_i915_private *i915 = data;
- return intel_gt_debugfs_reset_show(&i915->gt, val);
+ return intel_gt_debugfs_reset_show(to_gt(i915), val);
}
static int i915_wedged_set(void *data, u64 val)
{
struct drm_i915_private *i915 = data;
- return intel_gt_debugfs_reset_store(&i915->gt, val);
+ return intel_gt_debugfs_reset_store(to_gt(i915), val);
}
DEFINE_SIMPLE_ATTRIBUTE(i915_wedged_fops,
@@ -581,7 +581,7 @@ i915_perf_noa_delay_set(void *data, u64 val)
* This would lead to infinite waits as we're doing timestamp
* difference on the CS with only 32bits.
*/
- if (intel_gt_ns_to_clock_interval(&i915->gt, val) > U32_MAX)
+ if (intel_gt_ns_to_clock_interval(to_gt(i915), val) > U32_MAX)
return -EINVAL;
atomic64_set(&i915->perf.noa_programming_delay, val);
@@ -666,16 +666,18 @@ static int
i915_drop_caches_set(void *data, u64 val)
{
struct drm_i915_private *i915 = data;
+ unsigned int flags;
int ret;
DRM_DEBUG("Dropping caches: 0x%08llx [0x%08llx]\n",
val, val & DROP_ALL);
- ret = gt_drop_caches(&i915->gt, val);
+ ret = gt_drop_caches(to_gt(i915), val);
if (ret)
return ret;
fs_reclaim_acquire(GFP_KERNEL);
+ flags = memalloc_noreclaim_save();
if (val & DROP_BOUND)
i915_gem_shrink(NULL, i915, LONG_MAX, NULL, I915_SHRINK_BOUND);
@@ -684,6 +686,7 @@ i915_drop_caches_set(void *data, u64 val)
if (val & DROP_SHRINK_ALL)
i915_gem_shrink_all(i915);
+ memalloc_noreclaim_restore(flags);
fs_reclaim_release(GFP_KERNEL);
if (val & DROP_RCU)
@@ -702,7 +705,7 @@ DEFINE_SIMPLE_ATTRIBUTE(i915_drop_caches_fops,
static int i915_sseu_status(struct seq_file *m, void *unused)
{
struct drm_i915_private *i915 = node_to_i915(m->private);
- struct intel_gt *gt = &i915->gt;
+ struct intel_gt *gt = to_gt(i915);
return intel_sseu_status(m, gt);
}
@@ -711,14 +714,14 @@ static int i915_forcewake_open(struct inode *inode, struct file *file)
{
struct drm_i915_private *i915 = inode->i_private;
- return intel_gt_pm_debugfs_forcewake_user_open(&i915->gt);
+ return intel_gt_pm_debugfs_forcewake_user_open(to_gt(i915));
}
static int i915_forcewake_release(struct inode *inode, struct file *file)
{
struct drm_i915_private *i915 = inode->i_private;
- return intel_gt_pm_debugfs_forcewake_user_release(&i915->gt);
+ return intel_gt_pm_debugfs_forcewake_user_release(to_gt(i915));
}
static const struct file_operations i915_forcewake_fops = {
diff --git a/drivers/gpu/drm/i915/i915_debugfs_params.c b/drivers/gpu/drm/i915/i915_debugfs_params.c
index 2042427..783c867 100644
--- a/drivers/gpu/drm/i915/i915_debugfs_params.c
+++ b/drivers/gpu/drm/i915/i915_debugfs_params.c
@@ -40,8 +40,8 @@ static int notify_guc(struct drm_i915_private *i915)
{
int ret = 0;
- if (intel_uc_uses_guc_submission(&i915->gt.uc))
- ret = intel_guc_global_policies_update(&i915->gt.uc.guc);
+ if (intel_uc_uses_guc_submission(&to_gt(i915)->uc))
+ ret = intel_guc_global_policies_update(&to_gt(i915)->uc.guc);
return ret;
}
diff --git a/drivers/gpu/drm/i915/i915_deps.c b/drivers/gpu/drm/i915/i915_deps.c
new file mode 100644
index 0000000..999210b
--- /dev/null
+++ b/drivers/gpu/drm/i915/i915_deps.c
@@ -0,0 +1,237 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright © 2021 Intel Corporation
+ */
+
+#include <linux/dma-fence.h>
+#include <linux/slab.h>
+
+#include <drm/ttm/ttm_bo_api.h>
+
+#include "i915_deps.h"
+
+/**
+ * DOC: Set of utilities to dynamically collect dependencies into a
+ * structure which is fed into the GT migration code.
+ *
+ * Once we can do async unbinding, this is also needed to coalesce
+ * the migration fence with the unbind fences if these are coalesced
+ * post-migration.
+ *
+ * While collecting the individual dependencies, we store the refcounted
+ * struct dma_fence pointers in a realloc-managed pointer array, since
+ * that can be easily fed into a dma_fence_array. Other options are
+ * available, like for example an xarray for similarity with drm/sched.
+ * Can be changed easily if needed.
+ *
+ * A struct i915_deps need to be initialized using i915_deps_init().
+ * If i915_deps_add_dependency() or i915_deps_add_resv() return an
+ * error code they will internally call i915_deps_fini(), which frees
+ * all internal references and allocations.
+ */
+
+/* Min number of fence pointers in the array when an allocation occurs. */
+#define I915_DEPS_MIN_ALLOC_CHUNK 8U
+
+static void i915_deps_reset_fences(struct i915_deps *deps)
+{
+ if (deps->fences != &deps->single)
+ kfree(deps->fences);
+ deps->num_deps = 0;
+ deps->fences_size = 1;
+ deps->fences = &deps->single;
+}
+
+/**
+ * i915_deps_init - Initialize an i915_deps structure
+ * @deps: Pointer to the i915_deps structure to initialize.
+ * @gfp: The allocation mode for subsequenst allocations.
+ */
+void i915_deps_init(struct i915_deps *deps, gfp_t gfp)
+{
+ deps->fences = NULL;
+ deps->gfp = gfp;
+ i915_deps_reset_fences(deps);
+}
+
+/**
+ * i915_deps_fini - Finalize an i915_deps structure
+ * @deps: Pointer to the i915_deps structure to finalize.
+ *
+ * This function drops all fence references taken, conditionally frees and
+ * then resets the fences array.
+ */
+void i915_deps_fini(struct i915_deps *deps)
+{
+ unsigned int i;
+
+ for (i = 0; i < deps->num_deps; ++i)
+ dma_fence_put(deps->fences[i]);
+
+ if (deps->fences != &deps->single)
+ kfree(deps->fences);
+}
+
+static int i915_deps_grow(struct i915_deps *deps, struct dma_fence *fence,
+ const struct ttm_operation_ctx *ctx)
+{
+ int ret;
+
+ if (deps->num_deps >= deps->fences_size) {
+ unsigned int new_size = 2 * deps->fences_size;
+ struct dma_fence **new_fences;
+
+ new_size = max(new_size, I915_DEPS_MIN_ALLOC_CHUNK);
+ new_fences = kmalloc_array(new_size, sizeof(*new_fences), deps->gfp);
+ if (!new_fences)
+ goto sync;
+
+ memcpy(new_fences, deps->fences,
+ deps->fences_size * sizeof(*new_fences));
+ swap(new_fences, deps->fences);
+ if (new_fences != &deps->single)
+ kfree(new_fences);
+ deps->fences_size = new_size;
+ }
+ deps->fences[deps->num_deps++] = dma_fence_get(fence);
+ return 0;
+
+sync:
+ if (ctx->no_wait_gpu && !dma_fence_is_signaled(fence)) {
+ ret = -EBUSY;
+ goto unref;
+ }
+
+ ret = dma_fence_wait(fence, ctx->interruptible);
+ if (ret)
+ goto unref;
+
+ ret = fence->error;
+ if (ret)
+ goto unref;
+
+ return 0;
+
+unref:
+ i915_deps_fini(deps);
+ return ret;
+}
+
+/**
+ * i915_deps_sync - Wait for all the fences in the dependency collection
+ * @deps: Pointer to the i915_deps structure the fences of which to wait for.
+ * @ctx: Pointer to a struct ttm_operation_ctx indicating how the waits
+ * should be performed.
+ *
+ * This function waits for fences in the dependency collection. If it
+ * encounters an error during the wait or a fence error, the wait for
+ * further fences is aborted and the error returned.
+ *
+ * Return: Zero if successful, Negative error code on error.
+ */
+int i915_deps_sync(const struct i915_deps *deps, const struct ttm_operation_ctx *ctx)
+{
+ struct dma_fence **fences = deps->fences;
+ unsigned int i;
+ int ret = 0;
+
+ for (i = 0; i < deps->num_deps; ++i, ++fences) {
+ if (ctx->no_wait_gpu && !dma_fence_is_signaled(*fences)) {
+ ret = -EBUSY;
+ break;
+ }
+
+ ret = dma_fence_wait(*fences, ctx->interruptible);
+ if (!ret)
+ ret = (*fences)->error;
+ if (ret)
+ break;
+ }
+
+ return ret;
+}
+
+/**
+ * i915_deps_add_dependency - Add a fence to the dependency collection
+ * @deps: Pointer to the i915_deps structure a fence is to be added to.
+ * @fence: The fence to add.
+ * @ctx: Pointer to a struct ttm_operation_ctx indicating how waits are to
+ * be performed if waiting.
+ *
+ * Adds a fence to the dependency collection, and takes a reference on it.
+ * If the fence context is not zero and there was a later fence from the
+ * same fence context already added, then the fence is not added to the
+ * dependency collection. If the fence context is not zero and there was
+ * an earlier fence already added, then the fence will replace the older
+ * fence from the same context and the reference on the earlier fence will
+ * be dropped.
+ * If there is a failure to allocate memory to accommodate the new fence to
+ * be added, the new fence will instead be waited for and an error may
+ * be returned; depending on the value of @ctx, or if there was a fence
+ * error. If an error was returned, the dependency collection will be
+ * finalized and all fence reference dropped.
+ *
+ * Return: 0 if success. Negative error code on error.
+ */
+int i915_deps_add_dependency(struct i915_deps *deps,
+ struct dma_fence *fence,
+ const struct ttm_operation_ctx *ctx)
+{
+ unsigned int i;
+ int ret;
+
+ if (!fence)
+ return 0;
+
+ if (dma_fence_is_signaled(fence)) {
+ ret = fence->error;
+ if (ret)
+ i915_deps_fini(deps);
+ return ret;
+ }
+
+ for (i = 0; i < deps->num_deps; ++i) {
+ struct dma_fence *entry = deps->fences[i];
+
+ if (!entry->context || entry->context != fence->context)
+ continue;
+
+ if (dma_fence_is_later(fence, entry)) {
+ dma_fence_put(entry);
+ deps->fences[i] = dma_fence_get(fence);
+ }
+
+ return 0;
+ }
+
+ return i915_deps_grow(deps, fence, ctx);
+}
+
+/**
+ * i915_deps_add_resv - Add the fences of a reservation object to a dependency
+ * collection.
+ * @deps: Pointer to the i915_deps structure a fence is to be added to.
+ * @resv: The reservation object, then fences of which to add.
+ * @ctx: Pointer to a struct ttm_operation_ctx indicating how waits are to
+ * be performed if waiting.
+ *
+ * Calls i915_deps_add_depencency() on the indicated fences of @resv.
+ *
+ * Return: Zero on success. Negative error code on error.
+ */
+int i915_deps_add_resv(struct i915_deps *deps, struct dma_resv *resv,
+ const struct ttm_operation_ctx *ctx)
+{
+ struct dma_resv_iter iter;
+ struct dma_fence *fence;
+
+ dma_resv_assert_held(resv);
+ dma_resv_for_each_fence(&iter, resv, true, fence) {
+ int ret = i915_deps_add_dependency(deps, fence, ctx);
+
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
diff --git a/drivers/gpu/drm/i915/i915_deps.h b/drivers/gpu/drm/i915/i915_deps.h
new file mode 100644
index 0000000..d76c010
--- /dev/null
+++ b/drivers/gpu/drm/i915/i915_deps.h
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2021 Intel Corporation
+ */
+
+#ifndef _I915_DEPS_H_
+#define _I915_DEPS_H_
+
+#include <linux/types.h>
+
+struct ttm_operation_ctx;
+struct dma_fence;
+struct dma_resv;
+
+/**
+ * struct i915_deps - Collect dependencies into a single dma-fence
+ * @single: Storage for pointer if the collection is a single fence.
+ * @fences: Allocated array of fence pointers if more than a single fence;
+ * otherwise points to the address of @single.
+ * @num_deps: Current number of dependency fences.
+ * @fences_size: Size of the @fences array in number of pointers.
+ * @gfp: Allocation mode.
+ */
+struct i915_deps {
+ struct dma_fence *single;
+ struct dma_fence **fences;
+ unsigned int num_deps;
+ unsigned int fences_size;
+ gfp_t gfp;
+};
+
+void i915_deps_init(struct i915_deps *deps, gfp_t gfp);
+
+void i915_deps_fini(struct i915_deps *deps);
+
+int i915_deps_add_dependency(struct i915_deps *deps,
+ struct dma_fence *fence,
+ const struct ttm_operation_ctx *ctx);
+
+int i915_deps_add_resv(struct i915_deps *deps, struct dma_resv *resv,
+ const struct ttm_operation_ctx *ctx);
+
+int i915_deps_sync(const struct i915_deps *deps,
+ const struct ttm_operation_ctx *ctx);
+#endif
diff --git a/drivers/gpu/drm/i915/i915_driver.c b/drivers/gpu/drm/i915/i915_driver.c
new file mode 100644
index 0000000..9517493
--- /dev/null
+++ b/drivers/gpu/drm/i915/i915_driver.c
@@ -0,0 +1,1850 @@
+/* i915_drv.c -- i830,i845,i855,i865,i915 driver -*- linux-c -*-
+ */
+/*
+ *
+ * Copyright 2003 Tungsten Graphics, Inc., Cedar Park, Texas.
+ * All Rights Reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the
+ * "Software"), to deal in the Software without restriction, including
+ * without limitation the rights to use, copy, modify, merge, publish,
+ * distribute, sub license, and/or sell copies of the Software, and to
+ * permit persons to whom the Software is furnished to do so, subject to
+ * the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the
+ * next paragraph) shall be included in all copies or substantial portions
+ * of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
+ * OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT.
+ * IN NO EVENT SHALL TUNGSTEN GRAPHICS AND/OR ITS SUPPLIERS BE LIABLE FOR
+ * ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
+ * TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+ * SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+
+#include <linux/acpi.h>
+#include <linux/device.h>
+#include <linux/module.h>
+#include <linux/oom.h>
+#include <linux/pci.h>
+#include <linux/pm.h>
+#include <linux/pm_runtime.h>
+#include <linux/pnp.h>
+#include <linux/slab.h>
+#include <linux/vga_switcheroo.h>
+#include <linux/vt.h>
+
+#include <drm/drm_aperture.h>
+#include <drm/drm_atomic_helper.h>
+#include <drm/drm_ioctl.h>
+#include <drm/drm_managed.h>
+#include <drm/drm_probe_helper.h>
+
+#include "display/intel_acpi.h"
+#include "display/intel_bw.h"
+#include "display/intel_cdclk.h"
+#include "display/intel_display_types.h"
+#include "display/intel_dmc.h"
+#include "display/intel_dp.h"
+#include "display/intel_dpt.h"
+#include "display/intel_fbdev.h"
+#include "display/intel_hotplug.h"
+#include "display/intel_overlay.h"
+#include "display/intel_pch_refclk.h"
+#include "display/intel_pipe_crc.h"
+#include "display/intel_pps.h"
+#include "display/intel_sprite.h"
+#include "display/intel_vga.h"
+
+#include "gem/i915_gem_context.h"
+#include "gem/i915_gem_ioctls.h"
+#include "gem/i915_gem_mman.h"
+#include "gem/i915_gem_pm.h"
+#include "gt/intel_gt.h"
+#include "gt/intel_gt_pm.h"
+#include "gt/intel_rc6.h"
+
+#include "pxp/intel_pxp_pm.h"
+
+#include "i915_debugfs.h"
+#include "i915_driver.h"
+#include "i915_drv.h"
+#include "i915_ioc32.h"
+#include "i915_irq.h"
+#include "i915_memcpy.h"
+#include "i915_perf.h"
+#include "i915_query.h"
+#include "i915_suspend.h"
+#include "i915_switcheroo.h"
+#include "i915_sysfs.h"
+#include "i915_vgpu.h"
+#include "intel_dram.h"
+#include "intel_gvt.h"
+#include "intel_memory_region.h"
+#include "intel_pcode.h"
+#include "intel_pm.h"
+#include "intel_region_ttm.h"
+#include "vlv_suspend.h"
+
+static const struct drm_driver i915_drm_driver;
+
+static int i915_get_bridge_dev(struct drm_i915_private *dev_priv)
+{
+ int domain = pci_domain_nr(to_pci_dev(dev_priv->drm.dev)->bus);
+
+ dev_priv->bridge_dev =
+ pci_get_domain_bus_and_slot(domain, 0, PCI_DEVFN(0, 0));
+ if (!dev_priv->bridge_dev) {
+ drm_err(&dev_priv->drm, "bridge device not found\n");
+ return -EIO;
+ }
+ return 0;
+}
+
+/* Allocate space for the MCH regs if needed, return nonzero on error */
+static int
+intel_alloc_mchbar_resource(struct drm_i915_private *dev_priv)
+{
+ int reg = GRAPHICS_VER(dev_priv) >= 4 ? MCHBAR_I965 : MCHBAR_I915;
+ u32 temp_lo, temp_hi = 0;
+ u64 mchbar_addr;
+ int ret;
+
+ if (GRAPHICS_VER(dev_priv) >= 4)
+ pci_read_config_dword(dev_priv->bridge_dev, reg + 4, &temp_hi);
+ pci_read_config_dword(dev_priv->bridge_dev, reg, &temp_lo);
+ mchbar_addr = ((u64)temp_hi << 32) | temp_lo;
+
+ /* If ACPI doesn't have it, assume we need to allocate it ourselves */
+#ifdef CONFIG_PNP
+ if (mchbar_addr &&
+ pnp_range_reserved(mchbar_addr, mchbar_addr + MCHBAR_SIZE))
+ return 0;
+#endif
+
+ /* Get some space for it */
+ dev_priv->mch_res.name = "i915 MCHBAR";
+ dev_priv->mch_res.flags = IORESOURCE_MEM;
+ ret = pci_bus_alloc_resource(dev_priv->bridge_dev->bus,
+ &dev_priv->mch_res,
+ MCHBAR_SIZE, MCHBAR_SIZE,
+ PCIBIOS_MIN_MEM,
+ 0, pcibios_align_resource,
+ dev_priv->bridge_dev);
+ if (ret) {
+ drm_dbg(&dev_priv->drm, "failed bus alloc: %d\n", ret);
+ dev_priv->mch_res.start = 0;
+ return ret;
+ }
+
+ if (GRAPHICS_VER(dev_priv) >= 4)
+ pci_write_config_dword(dev_priv->bridge_dev, reg + 4,
+ upper_32_bits(dev_priv->mch_res.start));
+
+ pci_write_config_dword(dev_priv->bridge_dev, reg,
+ lower_32_bits(dev_priv->mch_res.start));
+ return 0;
+}
+
+/* Setup MCHBAR if possible, return true if we should disable it again */
+static void
+intel_setup_mchbar(struct drm_i915_private *dev_priv)
+{
+ int mchbar_reg = GRAPHICS_VER(dev_priv) >= 4 ? MCHBAR_I965 : MCHBAR_I915;
+ u32 temp;
+ bool enabled;
+
+ if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv))
+ return;
+
+ dev_priv->mchbar_need_disable = false;
+
+ if (IS_I915G(dev_priv) || IS_I915GM(dev_priv)) {
+ pci_read_config_dword(dev_priv->bridge_dev, DEVEN, &temp);
+ enabled = !!(temp & DEVEN_MCHBAR_EN);
+ } else {
+ pci_read_config_dword(dev_priv->bridge_dev, mchbar_reg, &temp);
+ enabled = temp & 1;
+ }
+
+ /* If it's already enabled, don't have to do anything */
+ if (enabled)
+ return;
+
+ if (intel_alloc_mchbar_resource(dev_priv))
+ return;
+
+ dev_priv->mchbar_need_disable = true;
+
+ /* Space is allocated or reserved, so enable it. */
+ if (IS_I915G(dev_priv) || IS_I915GM(dev_priv)) {
+ pci_write_config_dword(dev_priv->bridge_dev, DEVEN,
+ temp | DEVEN_MCHBAR_EN);
+ } else {
+ pci_read_config_dword(dev_priv->bridge_dev, mchbar_reg, &temp);
+ pci_write_config_dword(dev_priv->bridge_dev, mchbar_reg, temp | 1);
+ }
+}
+
+static void
+intel_teardown_mchbar(struct drm_i915_private *dev_priv)
+{
+ int mchbar_reg = GRAPHICS_VER(dev_priv) >= 4 ? MCHBAR_I965 : MCHBAR_I915;
+
+ if (dev_priv->mchbar_need_disable) {
+ if (IS_I915G(dev_priv) || IS_I915GM(dev_priv)) {
+ u32 deven_val;
+
+ pci_read_config_dword(dev_priv->bridge_dev, DEVEN,
+ &deven_val);
+ deven_val &= ~DEVEN_MCHBAR_EN;
+ pci_write_config_dword(dev_priv->bridge_dev, DEVEN,
+ deven_val);
+ } else {
+ u32 mchbar_val;
+
+ pci_read_config_dword(dev_priv->bridge_dev, mchbar_reg,
+ &mchbar_val);
+ mchbar_val &= ~1;
+ pci_write_config_dword(dev_priv->bridge_dev, mchbar_reg,
+ mchbar_val);
+ }
+ }
+
+ if (dev_priv->mch_res.start)
+ release_resource(&dev_priv->mch_res);
+}
+
+static int i915_workqueues_init(struct drm_i915_private *dev_priv)
+{
+ /*
+ * The i915 workqueue is primarily used for batched retirement of
+ * requests (and thus managing bo) once the task has been completed
+ * by the GPU. i915_retire_requests() is called directly when we
+ * need high-priority retirement, such as waiting for an explicit
+ * bo.
+ *
+ * It is also used for periodic low-priority events, such as
+ * idle-timers and recording error state.
+ *
+ * All tasks on the workqueue are expected to acquire the dev mutex
+ * so there is no point in running more than one instance of the
+ * workqueue at any time. Use an ordered one.
+ */
+ dev_priv->wq = alloc_ordered_workqueue("i915", 0);
+ if (dev_priv->wq == NULL)
+ goto out_err;
+
+ dev_priv->hotplug.dp_wq = alloc_ordered_workqueue("i915-dp", 0);
+ if (dev_priv->hotplug.dp_wq == NULL)
+ goto out_free_wq;
+
+ return 0;
+
+out_free_wq:
+ destroy_workqueue(dev_priv->wq);
+out_err:
+ drm_err(&dev_priv->drm, "Failed to allocate workqueues.\n");
+
+ return -ENOMEM;
+}
+
+static void i915_workqueues_cleanup(struct drm_i915_private *dev_priv)
+{
+ destroy_workqueue(dev_priv->hotplug.dp_wq);
+ destroy_workqueue(dev_priv->wq);
+}
+
+/*
+ * We don't keep the workarounds for pre-production hardware, so we expect our
+ * driver to fail on these machines in one way or another. A little warning on
+ * dmesg may help both the user and the bug triagers.
+ *
+ * Our policy for removing pre-production workarounds is to keep the
+ * current gen workarounds as a guide to the bring-up of the next gen
+ * (workarounds have a habit of persisting!). Anything older than that
+ * should be removed along with the complications they introduce.
+ */
+static void intel_detect_preproduction_hw(struct drm_i915_private *dev_priv)
+{
+ bool pre = false;
+
+ pre |= IS_HSW_EARLY_SDV(dev_priv);
+ pre |= IS_SKYLAKE(dev_priv) && INTEL_REVID(dev_priv) < 0x6;
+ pre |= IS_BROXTON(dev_priv) && INTEL_REVID(dev_priv) < 0xA;
+ pre |= IS_KABYLAKE(dev_priv) && INTEL_REVID(dev_priv) < 0x1;
+ pre |= IS_GEMINILAKE(dev_priv) && INTEL_REVID(dev_priv) < 0x3;
+ pre |= IS_ICELAKE(dev_priv) && INTEL_REVID(dev_priv) < 0x7;
+
+ if (pre) {
+ drm_err(&dev_priv->drm, "This is a pre-production stepping. "
+ "It may not be fully functional.\n");
+ add_taint(TAINT_MACHINE_CHECK, LOCKDEP_STILL_OK);
+ }
+}
+
+static void sanitize_gpu(struct drm_i915_private *i915)
+{
+ if (!INTEL_INFO(i915)->gpu_reset_clobbers_display)
+ __intel_gt_reset(to_gt(i915), ALL_ENGINES);
+}
+
+/**
+ * i915_driver_early_probe - setup state not requiring device access
+ * @dev_priv: device private
+ *
+ * Initialize everything that is a "SW-only" state, that is state not
+ * requiring accessing the device or exposing the driver via kernel internal
+ * or userspace interfaces. Example steps belonging here: lock initialization,
+ * system memory allocation, setting up device specific attributes and
+ * function hooks not requiring accessing the device.
+ */
+static int i915_driver_early_probe(struct drm_i915_private *dev_priv)
+{
+ int ret = 0;
+
+ if (i915_inject_probe_failure(dev_priv))
+ return -ENODEV;
+
+ intel_device_info_subplatform_init(dev_priv);
+ intel_step_init(dev_priv);
+
+ intel_gt_init_early(to_gt(dev_priv), dev_priv);
+ intel_uncore_mmio_debug_init_early(&dev_priv->mmio_debug);
+ intel_uncore_init_early(&dev_priv->uncore, to_gt(dev_priv));
+
+ spin_lock_init(&dev_priv->irq_lock);
+ spin_lock_init(&dev_priv->gpu_error.lock);
+ mutex_init(&dev_priv->backlight_lock);
+
+ mutex_init(&dev_priv->sb_lock);
+ cpu_latency_qos_add_request(&dev_priv->sb_qos, PM_QOS_DEFAULT_VALUE);
+
+ mutex_init(&dev_priv->audio.mutex);
+ mutex_init(&dev_priv->wm.wm_mutex);
+ mutex_init(&dev_priv->pps_mutex);
+ mutex_init(&dev_priv->hdcp_comp_mutex);
+
+ i915_memcpy_init_early(dev_priv);
+ intel_runtime_pm_init_early(&dev_priv->runtime_pm);
+
+ ret = i915_workqueues_init(dev_priv);
+ if (ret < 0)
+ return ret;
+
+ ret = vlv_suspend_init(dev_priv);
+ if (ret < 0)
+ goto err_workqueues;
+
+ ret = intel_region_ttm_device_init(dev_priv);
+ if (ret)
+ goto err_ttm;
+
+ intel_wopcm_init_early(&dev_priv->wopcm);
+
+ __intel_gt_init_early(to_gt(dev_priv), dev_priv);
+
+ i915_gem_init_early(dev_priv);
+
+ /* This must be called before any calls to HAS_PCH_* */
+ intel_detect_pch(dev_priv);
+
+ intel_pm_setup(dev_priv);
+ ret = intel_power_domains_init(dev_priv);
+ if (ret < 0)
+ goto err_gem;
+ intel_irq_init(dev_priv);
+ intel_init_display_hooks(dev_priv);
+ intel_init_clock_gating_hooks(dev_priv);
+
+ intel_detect_preproduction_hw(dev_priv);
+
+ return 0;
+
+err_gem:
+ i915_gem_cleanup_early(dev_priv);
+ intel_gt_driver_late_release(to_gt(dev_priv));
+ intel_region_ttm_device_fini(dev_priv);
+err_ttm:
+ vlv_suspend_cleanup(dev_priv);
+err_workqueues:
+ i915_workqueues_cleanup(dev_priv);
+ return ret;
+}
+
+/**
+ * i915_driver_late_release - cleanup the setup done in
+ * i915_driver_early_probe()
+ * @dev_priv: device private
+ */
+static void i915_driver_late_release(struct drm_i915_private *dev_priv)
+{
+ intel_irq_fini(dev_priv);
+ intel_power_domains_cleanup(dev_priv);
+ i915_gem_cleanup_early(dev_priv);
+ intel_gt_driver_late_release(to_gt(dev_priv));
+ intel_region_ttm_device_fini(dev_priv);
+ vlv_suspend_cleanup(dev_priv);
+ i915_workqueues_cleanup(dev_priv);
+
+ cpu_latency_qos_remove_request(&dev_priv->sb_qos);
+ mutex_destroy(&dev_priv->sb_lock);
+
+ i915_params_free(&dev_priv->params);
+}
+
+/**
+ * i915_driver_mmio_probe - setup device MMIO
+ * @dev_priv: device private
+ *
+ * Setup minimal device state necessary for MMIO accesses later in the
+ * initialization sequence. The setup here should avoid any other device-wide
+ * side effects or exposing the driver via kernel internal or user space
+ * interfaces.
+ */
+static int i915_driver_mmio_probe(struct drm_i915_private *dev_priv)
+{
+ int ret;
+
+ if (i915_inject_probe_failure(dev_priv))
+ return -ENODEV;
+
+ ret = i915_get_bridge_dev(dev_priv);
+ if (ret < 0)
+ return ret;
+
+ ret = intel_uncore_setup_mmio(&dev_priv->uncore);
+ if (ret < 0)
+ goto err_bridge;
+
+ ret = intel_uncore_init_mmio(&dev_priv->uncore);
+ if (ret)
+ goto err_mmio;
+
+ /* Try to make sure MCHBAR is enabled before poking at it */
+ intel_setup_mchbar(dev_priv);
+ intel_device_info_runtime_init(dev_priv);
+
+ ret = intel_gt_init_mmio(to_gt(dev_priv));
+ if (ret)
+ goto err_uncore;
+
+ /* As early as possible, scrub existing GPU state before clobbering */
+ sanitize_gpu(dev_priv);
+
+ return 0;
+
+err_uncore:
+ intel_teardown_mchbar(dev_priv);
+ intel_uncore_fini_mmio(&dev_priv->uncore);
+err_mmio:
+ intel_uncore_cleanup_mmio(&dev_priv->uncore);
+err_bridge:
+ pci_dev_put(dev_priv->bridge_dev);
+
+ return ret;
+}
+
+/**
+ * i915_driver_mmio_release - cleanup the setup done in i915_driver_mmio_probe()
+ * @dev_priv: device private
+ */
+static void i915_driver_mmio_release(struct drm_i915_private *dev_priv)
+{
+ intel_teardown_mchbar(dev_priv);
+ intel_uncore_fini_mmio(&dev_priv->uncore);
+ intel_uncore_cleanup_mmio(&dev_priv->uncore);
+ pci_dev_put(dev_priv->bridge_dev);
+}
+
+static void intel_sanitize_options(struct drm_i915_private *dev_priv)
+{
+ intel_gvt_sanitize_options(dev_priv);
+}
+
+/**
+ * i915_set_dma_info - set all relevant PCI dma info as configured for the
+ * platform
+ * @i915: valid i915 instance
+ *
+ * Set the dma max segment size, device and coherent masks. The dma mask set
+ * needs to occur before i915_ggtt_probe_hw.
+ *
+ * A couple of platforms have special needs. Address them as well.
+ *
+ */
+static int i915_set_dma_info(struct drm_i915_private *i915)
+{
+ unsigned int mask_size = INTEL_INFO(i915)->dma_mask_size;
+ int ret;
+
+ GEM_BUG_ON(!mask_size);
+
+ /*
+ * We don't have a max segment size, so set it to the max so sg's
+ * debugging layer doesn't complain
+ */
+ dma_set_max_seg_size(i915->drm.dev, UINT_MAX);
+
+ ret = dma_set_mask(i915->drm.dev, DMA_BIT_MASK(mask_size));
+ if (ret)
+ goto mask_err;
+
+ /* overlay on gen2 is broken and can't address above 1G */
+ if (GRAPHICS_VER(i915) == 2)
+ mask_size = 30;
+
+ /*
+ * 965GM sometimes incorrectly writes to hardware status page (HWS)
+ * using 32bit addressing, overwriting memory if HWS is located
+ * above 4GB.
+ *
+ * The documentation also mentions an issue with undefined
+ * behaviour if any general state is accessed within a page above 4GB,
+ * which also needs to be handled carefully.
+ */
+ if (IS_I965G(i915) || IS_I965GM(i915))
+ mask_size = 32;
+
+ ret = dma_set_coherent_mask(i915->drm.dev, DMA_BIT_MASK(mask_size));
+ if (ret)
+ goto mask_err;
+
+ return 0;
+
+mask_err:
+ drm_err(&i915->drm, "Can't set DMA mask/consistent mask (%d)\n", ret);
+ return ret;
+}
+
+/**
+ * i915_driver_hw_probe - setup state requiring device access
+ * @dev_priv: device private
+ *
+ * Setup state that requires accessing the device, but doesn't require
+ * exposing the driver via kernel internal or userspace interfaces.
+ */
+static int i915_driver_hw_probe(struct drm_i915_private *dev_priv)
+{
+ struct pci_dev *pdev = to_pci_dev(dev_priv->drm.dev);
+ int ret;
+
+ if (i915_inject_probe_failure(dev_priv))
+ return -ENODEV;
+
+ if (HAS_PPGTT(dev_priv)) {
+ if (intel_vgpu_active(dev_priv) &&
+ !intel_vgpu_has_full_ppgtt(dev_priv)) {
+ i915_report_error(dev_priv,
+ "incompatible vGPU found, support for isolated ppGTT required\n");
+ return -ENXIO;
+ }
+ }
+
+ if (HAS_EXECLISTS(dev_priv)) {
+ /*
+ * Older GVT emulation depends upon intercepting CSB mmio,
+ * which we no longer use, preferring to use the HWSP cache
+ * instead.
+ */
+ if (intel_vgpu_active(dev_priv) &&
+ !intel_vgpu_has_hwsp_emulation(dev_priv)) {
+ i915_report_error(dev_priv,
+ "old vGPU host found, support for HWSP emulation required\n");
+ return -ENXIO;
+ }
+ }
+
+ intel_sanitize_options(dev_priv);
+
+ /* needs to be done before ggtt probe */
+ intel_dram_edram_detect(dev_priv);
+
+ ret = i915_set_dma_info(dev_priv);
+ if (ret)
+ return ret;
+
+ i915_perf_init(dev_priv);
+
+ ret = i915_ggtt_probe_hw(dev_priv);
+ if (ret)
+ goto err_perf;
+
+ ret = drm_aperture_remove_conflicting_pci_framebuffers(pdev, dev_priv->drm.driver);
+ if (ret)
+ goto err_ggtt;
+
+ ret = i915_ggtt_init_hw(dev_priv);
+ if (ret)
+ goto err_ggtt;
+
+ ret = intel_memory_regions_hw_probe(dev_priv);
+ if (ret)
+ goto err_ggtt;
+
+ intel_gt_init_hw_early(to_gt(dev_priv), &dev_priv->ggtt);
+
+ ret = intel_gt_probe_lmem(to_gt(dev_priv));
+ if (ret)
+ goto err_mem_regions;
+
+ ret = i915_ggtt_enable_hw(dev_priv);
+ if (ret) {
+ drm_err(&dev_priv->drm, "failed to enable GGTT\n");
+ goto err_mem_regions;
+ }
+
+ pci_set_master(pdev);
+
+ /* On the 945G/GM, the chipset reports the MSI capability on the
+ * integrated graphics even though the support isn't actually there
+ * according to the published specs. It doesn't appear to function
+ * correctly in testing on 945G.
+ * This may be a side effect of MSI having been made available for PEG
+ * and the registers being closely associated.
+ *
+ * According to chipset errata, on the 965GM, MSI interrupts may
+ * be lost or delayed, and was defeatured. MSI interrupts seem to
+ * get lost on g4x as well, and interrupt delivery seems to stay
+ * properly dead afterwards. So we'll just disable them for all
+ * pre-gen5 chipsets.
+ *
+ * dp aux and gmbus irq on gen4 seems to be able to generate legacy
+ * interrupts even when in MSI mode. This results in spurious
+ * interrupt warnings if the legacy irq no. is shared with another
+ * device. The kernel then disables that interrupt source and so
+ * prevents the other device from working properly.
+ */
+ if (GRAPHICS_VER(dev_priv) >= 5) {
+ if (pci_enable_msi(pdev) < 0)
+ drm_dbg(&dev_priv->drm, "can't enable MSI");
+ }
+
+ ret = intel_gvt_init(dev_priv);
+ if (ret)
+ goto err_msi;
+
+ intel_opregion_setup(dev_priv);
+
+ ret = intel_pcode_init(dev_priv);
+ if (ret)
+ goto err_msi;
+
+ /*
+ * Fill the dram structure to get the system dram info. This will be
+ * used for memory latency calculation.
+ */
+ intel_dram_detect(dev_priv);
+
+ intel_bw_init_hw(dev_priv);
+
+ return 0;
+
+err_msi:
+ if (pdev->msi_enabled)
+ pci_disable_msi(pdev);
+err_mem_regions:
+ intel_memory_regions_driver_release(dev_priv);
+err_ggtt:
+ i915_ggtt_driver_release(dev_priv);
+ i915_gem_drain_freed_objects(dev_priv);
+ i915_ggtt_driver_late_release(dev_priv);
+err_perf:
+ i915_perf_fini(dev_priv);
+ return ret;
+}
+
+/**
+ * i915_driver_hw_remove - cleanup the setup done in i915_driver_hw_probe()
+ * @dev_priv: device private
+ */
+static void i915_driver_hw_remove(struct drm_i915_private *dev_priv)
+{
+ struct pci_dev *pdev = to_pci_dev(dev_priv->drm.dev);
+
+ i915_perf_fini(dev_priv);
+
+ if (pdev->msi_enabled)
+ pci_disable_msi(pdev);
+}
+
+/**
+ * i915_driver_register - register the driver with the rest of the system
+ * @dev_priv: device private
+ *
+ * Perform any steps necessary to make the driver available via kernel
+ * internal or userspace interfaces.
+ */
+static void i915_driver_register(struct drm_i915_private *dev_priv)
+{
+ struct drm_device *dev = &dev_priv->drm;
+
+ i915_gem_driver_register(dev_priv);
+ i915_pmu_register(dev_priv);
+
+ intel_vgpu_register(dev_priv);
+
+ /* Reveal our presence to userspace */
+ if (drm_dev_register(dev, 0)) {
+ drm_err(&dev_priv->drm,
+ "Failed to register driver for userspace access!\n");
+ return;
+ }
+
+ i915_debugfs_register(dev_priv);
+ i915_setup_sysfs(dev_priv);
+
+ /* Depends on sysfs having been initialized */
+ i915_perf_register(dev_priv);
+
+ intel_gt_driver_register(to_gt(dev_priv));
+
+ intel_display_driver_register(dev_priv);
+
+ intel_power_domains_enable(dev_priv);
+ intel_runtime_pm_enable(&dev_priv->runtime_pm);
+
+ intel_register_dsm_handler();
+
+ if (i915_switcheroo_register(dev_priv))
+ drm_err(&dev_priv->drm, "Failed to register vga switcheroo!\n");
+}
+
+/**
+ * i915_driver_unregister - cleanup the registration done in i915_driver_regiser()
+ * @dev_priv: device private
+ */
+static void i915_driver_unregister(struct drm_i915_private *dev_priv)
+{
+ i915_switcheroo_unregister(dev_priv);
+
+ intel_unregister_dsm_handler();
+
+ intel_runtime_pm_disable(&dev_priv->runtime_pm);
+ intel_power_domains_disable(dev_priv);
+
+ intel_display_driver_unregister(dev_priv);
+
+ intel_gt_driver_unregister(to_gt(dev_priv));
+
+ i915_perf_unregister(dev_priv);
+ i915_pmu_unregister(dev_priv);
+
+ i915_teardown_sysfs(dev_priv);
+ drm_dev_unplug(&dev_priv->drm);
+
+ i915_gem_driver_unregister(dev_priv);
+}
+
+void
+i915_print_iommu_status(struct drm_i915_private *i915, struct drm_printer *p)
+{
+ drm_printf(p, "iommu: %s\n", enableddisabled(intel_vtd_active(i915)));
+}
+
+static void i915_welcome_messages(struct drm_i915_private *dev_priv)
+{
+ if (drm_debug_enabled(DRM_UT_DRIVER)) {
+ struct drm_printer p = drm_debug_printer("i915 device info:");
+
+ drm_printf(&p, "pciid=0x%04x rev=0x%02x platform=%s (subplatform=0x%x) gen=%i\n",
+ INTEL_DEVID(dev_priv),
+ INTEL_REVID(dev_priv),
+ intel_platform_name(INTEL_INFO(dev_priv)->platform),
+ intel_subplatform(RUNTIME_INFO(dev_priv),
+ INTEL_INFO(dev_priv)->platform),
+ GRAPHICS_VER(dev_priv));
+
+ intel_device_info_print_static(INTEL_INFO(dev_priv), &p);
+ intel_device_info_print_runtime(RUNTIME_INFO(dev_priv), &p);
+ i915_print_iommu_status(dev_priv, &p);
+ intel_gt_info_print(&to_gt(dev_priv)->info, &p);
+ }
+
+ if (IS_ENABLED(CONFIG_DRM_I915_DEBUG))
+ drm_info(&dev_priv->drm, "DRM_I915_DEBUG enabled\n");
+ if (IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM))
+ drm_info(&dev_priv->drm, "DRM_I915_DEBUG_GEM enabled\n");
+ if (IS_ENABLED(CONFIG_DRM_I915_DEBUG_RUNTIME_PM))
+ drm_info(&dev_priv->drm,
+ "DRM_I915_DEBUG_RUNTIME_PM enabled\n");
+}
+
+static struct drm_i915_private *
+i915_driver_create(struct pci_dev *pdev, const struct pci_device_id *ent)
+{
+ const struct intel_device_info *match_info =
+ (struct intel_device_info *)ent->driver_data;
+ struct intel_device_info *device_info;
+ struct drm_i915_private *i915;
+
+ i915 = devm_drm_dev_alloc(&pdev->dev, &i915_drm_driver,
+ struct drm_i915_private, drm);
+ if (IS_ERR(i915))
+ return i915;
+
+ pci_set_drvdata(pdev, i915);
+
+ /* Device parameters start as a copy of module parameters. */
+ i915_params_copy(&i915->params, &i915_modparams);
+
+ /* Setup the write-once "constant" device info */
+ device_info = mkwrite_device_info(i915);
+ memcpy(device_info, match_info, sizeof(*device_info));
+ RUNTIME_INFO(i915)->device_id = pdev->device;
+
+ return i915;
+}
+
+/**
+ * i915_driver_probe - setup chip and create an initial config
+ * @pdev: PCI device
+ * @ent: matching PCI ID entry
+ *
+ * The driver probe routine has to do several things:
+ * - drive output discovery via intel_modeset_init()
+ * - initialize the memory manager
+ * - allocate initial config memory
+ * - setup the DRM framebuffer with the allocated memory
+ */
+int i915_driver_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
+{
+ const struct intel_device_info *match_info =
+ (struct intel_device_info *)ent->driver_data;
+ struct drm_i915_private *i915;
+ int ret;
+
+ i915 = i915_driver_create(pdev, ent);
+ if (IS_ERR(i915))
+ return PTR_ERR(i915);
+
+ /* Disable nuclear pageflip by default on pre-ILK */
+ if (!i915->params.nuclear_pageflip && match_info->graphics.ver < 5)
+ i915->drm.driver_features &= ~DRIVER_ATOMIC;
+
+ /*
+ * Check if we support fake LMEM -- for now we only unleash this for
+ * the live selftests(test-and-exit).
+ */
+#if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
+ if (IS_ENABLED(CONFIG_DRM_I915_UNSTABLE_FAKE_LMEM)) {
+ if (GRAPHICS_VER(i915) >= 9 && i915_selftest.live < 0 &&
+ i915->params.fake_lmem_start) {
+ mkwrite_device_info(i915)->memory_regions =
+ REGION_SMEM | REGION_LMEM | REGION_STOLEN_SMEM;
+ GEM_BUG_ON(!HAS_LMEM(i915));
+ }
+ }
+#endif
+
+ ret = pci_enable_device(pdev);
+ if (ret)
+ goto out_fini;
+
+ ret = i915_driver_early_probe(i915);
+ if (ret < 0)
+ goto out_pci_disable;
+
+ disable_rpm_wakeref_asserts(&i915->runtime_pm);
+
+ intel_vgpu_detect(i915);
+
+ ret = i915_driver_mmio_probe(i915);
+ if (ret < 0)
+ goto out_runtime_pm_put;
+
+ ret = i915_driver_hw_probe(i915);
+ if (ret < 0)
+ goto out_cleanup_mmio;
+
+ ret = intel_modeset_init_noirq(i915);
+ if (ret < 0)
+ goto out_cleanup_hw;
+
+ ret = intel_irq_install(i915);
+ if (ret)
+ goto out_cleanup_modeset;
+
+ ret = intel_modeset_init_nogem(i915);
+ if (ret)
+ goto out_cleanup_irq;
+
+ ret = i915_gem_init(i915);
+ if (ret)
+ goto out_cleanup_modeset2;
+
+ ret = intel_modeset_init(i915);
+ if (ret)
+ goto out_cleanup_gem;
+
+ i915_driver_register(i915);
+
+ enable_rpm_wakeref_asserts(&i915->runtime_pm);
+
+ i915_welcome_messages(i915);
+
+ i915->do_release = true;
+
+ return 0;
+
+out_cleanup_gem:
+ i915_gem_suspend(i915);
+ i915_gem_driver_remove(i915);
+ i915_gem_driver_release(i915);
+out_cleanup_modeset2:
+ /* FIXME clean up the error path */
+ intel_modeset_driver_remove(i915);
+ intel_irq_uninstall(i915);
+ intel_modeset_driver_remove_noirq(i915);
+ goto out_cleanup_modeset;
+out_cleanup_irq:
+ intel_irq_uninstall(i915);
+out_cleanup_modeset:
+ intel_modeset_driver_remove_nogem(i915);
+out_cleanup_hw:
+ i915_driver_hw_remove(i915);
+ intel_memory_regions_driver_release(i915);
+ i915_ggtt_driver_release(i915);
+ i915_gem_drain_freed_objects(i915);
+ i915_ggtt_driver_late_release(i915);
+out_cleanup_mmio:
+ i915_driver_mmio_release(i915);
+out_runtime_pm_put:
+ enable_rpm_wakeref_asserts(&i915->runtime_pm);
+ i915_driver_late_release(i915);
+out_pci_disable:
+ pci_disable_device(pdev);
+out_fini:
+ i915_probe_error(i915, "Device initialization failed (%d)\n", ret);
+ return ret;
+}
+
+void i915_driver_remove(struct drm_i915_private *i915)
+{
+ disable_rpm_wakeref_asserts(&i915->runtime_pm);
+
+ i915_driver_unregister(i915);
+
+ /* Flush any external code that still may be under the RCU lock */
+ synchronize_rcu();
+
+ i915_gem_suspend(i915);
+
+ intel_gvt_driver_remove(i915);
+
+ intel_modeset_driver_remove(i915);
+
+ intel_irq_uninstall(i915);
+
+ intel_modeset_driver_remove_noirq(i915);
+
+ i915_reset_error_state(i915);
+ i915_gem_driver_remove(i915);
+
+ intel_modeset_driver_remove_nogem(i915);
+
+ i915_driver_hw_remove(i915);
+
+ enable_rpm_wakeref_asserts(&i915->runtime_pm);
+}
+
+static void i915_driver_release(struct drm_device *dev)
+{
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct intel_runtime_pm *rpm = &dev_priv->runtime_pm;
+
+ if (!dev_priv->do_release)
+ return;
+
+ disable_rpm_wakeref_asserts(rpm);
+
+ i915_gem_driver_release(dev_priv);
+
+ intel_memory_regions_driver_release(dev_priv);
+ i915_ggtt_driver_release(dev_priv);
+ i915_gem_drain_freed_objects(dev_priv);
+ i915_ggtt_driver_late_release(dev_priv);
+
+ i915_driver_mmio_release(dev_priv);
+
+ enable_rpm_wakeref_asserts(rpm);
+ intel_runtime_pm_driver_release(rpm);
+
+ i915_driver_late_release(dev_priv);
+}
+
+static int i915_driver_open(struct drm_device *dev, struct drm_file *file)
+{
+ struct drm_i915_private *i915 = to_i915(dev);
+ int ret;
+
+ ret = i915_gem_open(i915, file);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+/**
+ * i915_driver_lastclose - clean up after all DRM clients have exited
+ * @dev: DRM device
+ *
+ * Take care of cleaning up after all DRM clients have exited. In the
+ * mode setting case, we want to restore the kernel's initial mode (just
+ * in case the last client left us in a bad state).
+ *
+ * Additionally, in the non-mode setting case, we'll tear down the GTT
+ * and DMA structures, since the kernel won't be using them, and clea
+ * up any GEM state.
+ */
+static void i915_driver_lastclose(struct drm_device *dev)
+{
+ struct drm_i915_private *i915 = to_i915(dev);
+
+ intel_fbdev_restore_mode(dev);
+
+ if (HAS_DISPLAY(i915))
+ vga_switcheroo_process_delayed_switch();
+}
+
+static void i915_driver_postclose(struct drm_device *dev, struct drm_file *file)
+{
+ struct drm_i915_file_private *file_priv = file->driver_priv;
+
+ i915_gem_context_close(file);
+
+ kfree_rcu(file_priv, rcu);
+
+ /* Catch up with all the deferred frees from "this" client */
+ i915_gem_flush_free_objects(to_i915(dev));
+}
+
+static void intel_suspend_encoders(struct drm_i915_private *dev_priv)
+{
+ struct drm_device *dev = &dev_priv->drm;
+ struct intel_encoder *encoder;
+
+ if (!HAS_DISPLAY(dev_priv))
+ return;
+
+ drm_modeset_lock_all(dev);
+ for_each_intel_encoder(dev, encoder)
+ if (encoder->suspend)
+ encoder->suspend(encoder);
+ drm_modeset_unlock_all(dev);
+}
+
+static void intel_shutdown_encoders(struct drm_i915_private *dev_priv)
+{
+ struct drm_device *dev = &dev_priv->drm;
+ struct intel_encoder *encoder;
+
+ if (!HAS_DISPLAY(dev_priv))
+ return;
+
+ drm_modeset_lock_all(dev);
+ for_each_intel_encoder(dev, encoder)
+ if (encoder->shutdown)
+ encoder->shutdown(encoder);
+ drm_modeset_unlock_all(dev);
+}
+
+void i915_driver_shutdown(struct drm_i915_private *i915)
+{
+ disable_rpm_wakeref_asserts(&i915->runtime_pm);
+ intel_runtime_pm_disable(&i915->runtime_pm);
+ intel_power_domains_disable(i915);
+
+ i915_gem_suspend(i915);
+
+ if (HAS_DISPLAY(i915)) {
+ drm_kms_helper_poll_disable(&i915->drm);
+
+ drm_atomic_helper_shutdown(&i915->drm);
+ }
+
+ intel_dp_mst_suspend(i915);
+
+ intel_runtime_pm_disable_interrupts(i915);
+ intel_hpd_cancel_work(i915);
+
+ intel_suspend_encoders(i915);
+ intel_shutdown_encoders(i915);
+
+ intel_dmc_ucode_suspend(i915);
+
+ /*
+ * The only requirement is to reboot with display DC states disabled,
+ * for now leaving all display power wells in the INIT power domain
+ * enabled.
+ *
+ * TODO:
+ * - unify the pci_driver::shutdown sequence here with the
+ * pci_driver.driver.pm.poweroff,poweroff_late sequence.
+ * - unify the driver remove and system/runtime suspend sequences with
+ * the above unified shutdown/poweroff sequence.
+ */
+ intel_power_domains_driver_remove(i915);
+ enable_rpm_wakeref_asserts(&i915->runtime_pm);
+
+ intel_runtime_pm_driver_release(&i915->runtime_pm);
+}
+
+static bool suspend_to_idle(struct drm_i915_private *dev_priv)
+{
+#if IS_ENABLED(CONFIG_ACPI_SLEEP)
+ if (acpi_target_system_state() < ACPI_STATE_S3)
+ return true;
+#endif
+ return false;
+}
+
+static int i915_drm_prepare(struct drm_device *dev)
+{
+ struct drm_i915_private *i915 = to_i915(dev);
+
+ /*
+ * NB intel_display_suspend() may issue new requests after we've
+ * ostensibly marked the GPU as ready-to-sleep here. We need to
+ * split out that work and pull it forward so that after point,
+ * the GPU is not woken again.
+ */
+ return i915_gem_backup_suspend(i915);
+}
+
+static int i915_drm_suspend(struct drm_device *dev)
+{
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct pci_dev *pdev = to_pci_dev(dev_priv->drm.dev);
+ pci_power_t opregion_target_state;
+
+ disable_rpm_wakeref_asserts(&dev_priv->runtime_pm);
+
+ /* We do a lot of poking in a lot of registers, make sure they work
+ * properly. */
+ intel_power_domains_disable(dev_priv);
+ if (HAS_DISPLAY(dev_priv))
+ drm_kms_helper_poll_disable(dev);
+
+ pci_save_state(pdev);
+
+ intel_display_suspend(dev);
+
+ intel_dp_mst_suspend(dev_priv);
+
+ intel_runtime_pm_disable_interrupts(dev_priv);
+ intel_hpd_cancel_work(dev_priv);
+
+ intel_suspend_encoders(dev_priv);
+
+ intel_suspend_hw(dev_priv);
+
+ /* Must be called before GGTT is suspended. */
+ intel_dpt_suspend(dev_priv);
+ i915_ggtt_suspend(&dev_priv->ggtt);
+
+ i915_save_display(dev_priv);
+
+ opregion_target_state = suspend_to_idle(dev_priv) ? PCI_D1 : PCI_D3cold;
+ intel_opregion_suspend(dev_priv, opregion_target_state);
+
+ intel_fbdev_set_suspend(dev, FBINFO_STATE_SUSPENDED, true);
+
+ dev_priv->suspend_count++;
+
+ intel_dmc_ucode_suspend(dev_priv);
+
+ enable_rpm_wakeref_asserts(&dev_priv->runtime_pm);
+
+ return 0;
+}
+
+static enum i915_drm_suspend_mode
+get_suspend_mode(struct drm_i915_private *dev_priv, bool hibernate)
+{
+ if (hibernate)
+ return I915_DRM_SUSPEND_HIBERNATE;
+
+ if (suspend_to_idle(dev_priv))
+ return I915_DRM_SUSPEND_IDLE;
+
+ return I915_DRM_SUSPEND_MEM;
+}
+
+static int i915_drm_suspend_late(struct drm_device *dev, bool hibernation)
+{
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct pci_dev *pdev = to_pci_dev(dev_priv->drm.dev);
+ struct intel_runtime_pm *rpm = &dev_priv->runtime_pm;
+ int ret;
+
+ disable_rpm_wakeref_asserts(rpm);
+
+ i915_gem_suspend_late(dev_priv);
+
+ intel_uncore_suspend(&dev_priv->uncore);
+
+ intel_power_domains_suspend(dev_priv,
+ get_suspend_mode(dev_priv, hibernation));
+
+ intel_display_power_suspend_late(dev_priv);
+
+ ret = vlv_suspend_complete(dev_priv);
+ if (ret) {
+ drm_err(&dev_priv->drm, "Suspend complete failed: %d\n", ret);
+ intel_power_domains_resume(dev_priv);
+
+ goto out;
+ }
+
+ /*
+ * FIXME: Temporary hammer to avoid freezing the machine on our DGFX
+ * This should be totally removed when we handle the pci states properly
+ * on runtime PM and on s2idle cases.
+ */
+ if (suspend_to_idle(dev_priv))
+ pci_d3cold_disable(pdev);
+
+ pci_disable_device(pdev);
+ /*
+ * During hibernation on some platforms the BIOS may try to access
+ * the device even though it's already in D3 and hang the machine. So
+ * leave the device in D0 on those platforms and hope the BIOS will
+ * power down the device properly. The issue was seen on multiple old
+ * GENs with different BIOS vendors, so having an explicit blacklist
+ * is inpractical; apply the workaround on everything pre GEN6. The
+ * platforms where the issue was seen:
+ * Lenovo Thinkpad X301, X61s, X60, T60, X41
+ * Fujitsu FSC S7110
+ * Acer Aspire 1830T
+ */
+ if (!(hibernation && GRAPHICS_VER(dev_priv) < 6))
+ pci_set_power_state(pdev, PCI_D3hot);
+
+out:
+ enable_rpm_wakeref_asserts(rpm);
+ if (!dev_priv->uncore.user_forcewake_count)
+ intel_runtime_pm_driver_release(rpm);
+
+ return ret;
+}
+
+int i915_driver_suspend_switcheroo(struct drm_i915_private *i915,
+ pm_message_t state)
+{
+ int error;
+
+ if (drm_WARN_ON_ONCE(&i915->drm, state.event != PM_EVENT_SUSPEND &&
+ state.event != PM_EVENT_FREEZE))
+ return -EINVAL;
+
+ if (i915->drm.switch_power_state == DRM_SWITCH_POWER_OFF)
+ return 0;
+
+ error = i915_drm_suspend(&i915->drm);
+ if (error)
+ return error;
+
+ return i915_drm_suspend_late(&i915->drm, false);
+}
+
+static int i915_drm_resume(struct drm_device *dev)
+{
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ int ret;
+
+ disable_rpm_wakeref_asserts(&dev_priv->runtime_pm);
+
+ ret = intel_pcode_init(dev_priv);
+ if (ret)
+ return ret;
+
+ sanitize_gpu(dev_priv);
+
+ ret = i915_ggtt_enable_hw(dev_priv);
+ if (ret)
+ drm_err(&dev_priv->drm, "failed to re-enable GGTT\n");
+
+ i915_ggtt_resume(&dev_priv->ggtt);
+ /* Must be called after GGTT is resumed. */
+ intel_dpt_resume(dev_priv);
+
+ intel_dmc_ucode_resume(dev_priv);
+
+ i915_restore_display(dev_priv);
+ intel_pps_unlock_regs_wa(dev_priv);
+
+ intel_init_pch_refclk(dev_priv);
+
+ /*
+ * Interrupts have to be enabled before any batches are run. If not the
+ * GPU will hang. i915_gem_init_hw() will initiate batches to
+ * update/restore the context.
+ *
+ * drm_mode_config_reset() needs AUX interrupts.
+ *
+ * Modeset enabling in intel_modeset_init_hw() also needs working
+ * interrupts.
+ */
+ intel_runtime_pm_enable_interrupts(dev_priv);
+
+ if (HAS_DISPLAY(dev_priv))
+ drm_mode_config_reset(dev);
+
+ i915_gem_resume(dev_priv);
+
+ intel_modeset_init_hw(dev_priv);
+ intel_init_clock_gating(dev_priv);
+ intel_hpd_init(dev_priv);
+
+ /* MST sideband requires HPD interrupts enabled */
+ intel_dp_mst_resume(dev_priv);
+ intel_display_resume(dev);
+
+ intel_hpd_poll_disable(dev_priv);
+ if (HAS_DISPLAY(dev_priv))
+ drm_kms_helper_poll_enable(dev);
+
+ intel_opregion_resume(dev_priv);
+
+ intel_fbdev_set_suspend(dev, FBINFO_STATE_RUNNING, false);
+
+ intel_power_domains_enable(dev_priv);
+
+ intel_gvt_resume(dev_priv);
+
+ enable_rpm_wakeref_asserts(&dev_priv->runtime_pm);
+
+ return 0;
+}
+
+static int i915_drm_resume_early(struct drm_device *dev)
+{
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct pci_dev *pdev = to_pci_dev(dev_priv->drm.dev);
+ int ret;
+
+ /*
+ * We have a resume ordering issue with the snd-hda driver also
+ * requiring our device to be power up. Due to the lack of a
+ * parent/child relationship we currently solve this with an early
+ * resume hook.
+ *
+ * FIXME: This should be solved with a special hdmi sink device or
+ * similar so that power domains can be employed.
+ */
+
+ /*
+ * Note that we need to set the power state explicitly, since we
+ * powered off the device during freeze and the PCI core won't power
+ * it back up for us during thaw. Powering off the device during
+ * freeze is not a hard requirement though, and during the
+ * suspend/resume phases the PCI core makes sure we get here with the
+ * device powered on. So in case we change our freeze logic and keep
+ * the device powered we can also remove the following set power state
+ * call.
+ */
+ ret = pci_set_power_state(pdev, PCI_D0);
+ if (ret) {
+ drm_err(&dev_priv->drm,
+ "failed to set PCI D0 power state (%d)\n", ret);
+ return ret;
+ }
+
+ /*
+ * Note that pci_enable_device() first enables any parent bridge
+ * device and only then sets the power state for this device. The
+ * bridge enabling is a nop though, since bridge devices are resumed
+ * first. The order of enabling power and enabling the device is
+ * imposed by the PCI core as described above, so here we preserve the
+ * same order for the freeze/thaw phases.
+ *
+ * TODO: eventually we should remove pci_disable_device() /
+ * pci_enable_enable_device() from suspend/resume. Due to how they
+ * depend on the device enable refcount we can't anyway depend on them
+ * disabling/enabling the device.
+ */
+ if (pci_enable_device(pdev))
+ return -EIO;
+
+ pci_set_master(pdev);
+
+ pci_d3cold_enable(pdev);
+
+ disable_rpm_wakeref_asserts(&dev_priv->runtime_pm);
+
+ ret = vlv_resume_prepare(dev_priv, false);
+ if (ret)
+ drm_err(&dev_priv->drm,
+ "Resume prepare failed: %d, continuing anyway\n", ret);
+
+ intel_uncore_resume_early(&dev_priv->uncore);
+
+ intel_gt_check_and_clear_faults(to_gt(dev_priv));
+
+ intel_display_power_resume_early(dev_priv);
+
+ intel_power_domains_resume(dev_priv);
+
+ enable_rpm_wakeref_asserts(&dev_priv->runtime_pm);
+
+ return ret;
+}
+
+int i915_driver_resume_switcheroo(struct drm_i915_private *i915)
+{
+ int ret;
+
+ if (i915->drm.switch_power_state == DRM_SWITCH_POWER_OFF)
+ return 0;
+
+ ret = i915_drm_resume_early(&i915->drm);
+ if (ret)
+ return ret;
+
+ return i915_drm_resume(&i915->drm);
+}
+
+static int i915_pm_prepare(struct device *kdev)
+{
+ struct drm_i915_private *i915 = kdev_to_i915(kdev);
+
+ if (!i915) {
+ dev_err(kdev, "DRM not initialized, aborting suspend.\n");
+ return -ENODEV;
+ }
+
+ if (i915->drm.switch_power_state == DRM_SWITCH_POWER_OFF)
+ return 0;
+
+ return i915_drm_prepare(&i915->drm);
+}
+
+static int i915_pm_suspend(struct device *kdev)
+{
+ struct drm_i915_private *i915 = kdev_to_i915(kdev);
+
+ if (!i915) {
+ dev_err(kdev, "DRM not initialized, aborting suspend.\n");
+ return -ENODEV;
+ }
+
+ if (i915->drm.switch_power_state == DRM_SWITCH_POWER_OFF)
+ return 0;
+
+ return i915_drm_suspend(&i915->drm);
+}
+
+static int i915_pm_suspend_late(struct device *kdev)
+{
+ struct drm_i915_private *i915 = kdev_to_i915(kdev);
+
+ /*
+ * We have a suspend ordering issue with the snd-hda driver also
+ * requiring our device to be power up. Due to the lack of a
+ * parent/child relationship we currently solve this with an late
+ * suspend hook.
+ *
+ * FIXME: This should be solved with a special hdmi sink device or
+ * similar so that power domains can be employed.
+ */
+ if (i915->drm.switch_power_state == DRM_SWITCH_POWER_OFF)
+ return 0;
+
+ return i915_drm_suspend_late(&i915->drm, false);
+}
+
+static int i915_pm_poweroff_late(struct device *kdev)
+{
+ struct drm_i915_private *i915 = kdev_to_i915(kdev);
+
+ if (i915->drm.switch_power_state == DRM_SWITCH_POWER_OFF)
+ return 0;
+
+ return i915_drm_suspend_late(&i915->drm, true);
+}
+
+static int i915_pm_resume_early(struct device *kdev)
+{
+ struct drm_i915_private *i915 = kdev_to_i915(kdev);
+
+ if (i915->drm.switch_power_state == DRM_SWITCH_POWER_OFF)
+ return 0;
+
+ return i915_drm_resume_early(&i915->drm);
+}
+
+static int i915_pm_resume(struct device *kdev)
+{
+ struct drm_i915_private *i915 = kdev_to_i915(kdev);
+
+ if (i915->drm.switch_power_state == DRM_SWITCH_POWER_OFF)
+ return 0;
+
+ return i915_drm_resume(&i915->drm);
+}
+
+/* freeze: before creating the hibernation_image */
+static int i915_pm_freeze(struct device *kdev)
+{
+ struct drm_i915_private *i915 = kdev_to_i915(kdev);
+ int ret;
+
+ if (i915->drm.switch_power_state != DRM_SWITCH_POWER_OFF) {
+ ret = i915_drm_suspend(&i915->drm);
+ if (ret)
+ return ret;
+ }
+
+ ret = i915_gem_freeze(i915);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+static int i915_pm_freeze_late(struct device *kdev)
+{
+ struct drm_i915_private *i915 = kdev_to_i915(kdev);
+ int ret;
+
+ if (i915->drm.switch_power_state != DRM_SWITCH_POWER_OFF) {
+ ret = i915_drm_suspend_late(&i915->drm, true);
+ if (ret)
+ return ret;
+ }
+
+ ret = i915_gem_freeze_late(i915);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+/* thaw: called after creating the hibernation image, but before turning off. */
+static int i915_pm_thaw_early(struct device *kdev)
+{
+ return i915_pm_resume_early(kdev);
+}
+
+static int i915_pm_thaw(struct device *kdev)
+{
+ return i915_pm_resume(kdev);
+}
+
+/* restore: called after loading the hibernation image. */
+static int i915_pm_restore_early(struct device *kdev)
+{
+ return i915_pm_resume_early(kdev);
+}
+
+static int i915_pm_restore(struct device *kdev)
+{
+ return i915_pm_resume(kdev);
+}
+
+static int intel_runtime_suspend(struct device *kdev)
+{
+ struct drm_i915_private *dev_priv = kdev_to_i915(kdev);
+ struct intel_runtime_pm *rpm = &dev_priv->runtime_pm;
+ struct pci_dev *pdev = to_pci_dev(dev_priv->drm.dev);
+ int ret;
+
+ if (drm_WARN_ON_ONCE(&dev_priv->drm, !HAS_RUNTIME_PM(dev_priv)))
+ return -ENODEV;
+
+ drm_dbg_kms(&dev_priv->drm, "Suspending device\n");
+
+ disable_rpm_wakeref_asserts(rpm);
+
+ /*
+ * We are safe here against re-faults, since the fault handler takes
+ * an RPM reference.
+ */
+ i915_gem_runtime_suspend(dev_priv);
+
+ intel_gt_runtime_suspend(to_gt(dev_priv));
+
+ intel_runtime_pm_disable_interrupts(dev_priv);
+
+ intel_uncore_suspend(&dev_priv->uncore);
+
+ intel_display_power_suspend(dev_priv);
+
+ ret = vlv_suspend_complete(dev_priv);
+ if (ret) {
+ drm_err(&dev_priv->drm,
+ "Runtime suspend failed, disabling it (%d)\n", ret);
+ intel_uncore_runtime_resume(&dev_priv->uncore);
+
+ intel_runtime_pm_enable_interrupts(dev_priv);
+
+ intel_gt_runtime_resume(to_gt(dev_priv));
+
+ enable_rpm_wakeref_asserts(rpm);
+
+ return ret;
+ }
+
+ enable_rpm_wakeref_asserts(rpm);
+ intel_runtime_pm_driver_release(rpm);
+
+ if (intel_uncore_arm_unclaimed_mmio_detection(&dev_priv->uncore))
+ drm_err(&dev_priv->drm,
+ "Unclaimed access detected prior to suspending\n");
+
+ /*
+ * FIXME: Temporary hammer to avoid freezing the machine on our DGFX
+ * This should be totally removed when we handle the pci states properly
+ * on runtime PM and on s2idle cases.
+ */
+ pci_d3cold_disable(pdev);
+ rpm->suspended = true;
+
+ /*
+ * FIXME: We really should find a document that references the arguments
+ * used below!
+ */
+ if (IS_BROADWELL(dev_priv)) {
+ /*
+ * On Broadwell, if we use PCI_D1 the PCH DDI ports will stop
+ * being detected, and the call we do at intel_runtime_resume()
+ * won't be able to restore them. Since PCI_D3hot matches the
+ * actual specification and appears to be working, use it.
+ */
+ intel_opregion_notify_adapter(dev_priv, PCI_D3hot);
+ } else {
+ /*
+ * current versions of firmware which depend on this opregion
+ * notification have repurposed the D1 definition to mean
+ * "runtime suspended" vs. what you would normally expect (D3)
+ * to distinguish it from notifications that might be sent via
+ * the suspend path.
+ */
+ intel_opregion_notify_adapter(dev_priv, PCI_D1);
+ }
+
+ assert_forcewakes_inactive(&dev_priv->uncore);
+
+ if (!IS_VALLEYVIEW(dev_priv) && !IS_CHERRYVIEW(dev_priv))
+ intel_hpd_poll_enable(dev_priv);
+
+ drm_dbg_kms(&dev_priv->drm, "Device suspended\n");
+ return 0;
+}
+
+static int intel_runtime_resume(struct device *kdev)
+{
+ struct drm_i915_private *dev_priv = kdev_to_i915(kdev);
+ struct intel_runtime_pm *rpm = &dev_priv->runtime_pm;
+ struct pci_dev *pdev = to_pci_dev(dev_priv->drm.dev);
+ int ret;
+
+ if (drm_WARN_ON_ONCE(&dev_priv->drm, !HAS_RUNTIME_PM(dev_priv)))
+ return -ENODEV;
+
+ drm_dbg_kms(&dev_priv->drm, "Resuming device\n");
+
+ drm_WARN_ON_ONCE(&dev_priv->drm, atomic_read(&rpm->wakeref_count));
+ disable_rpm_wakeref_asserts(rpm);
+
+ intel_opregion_notify_adapter(dev_priv, PCI_D0);
+ rpm->suspended = false;
+ pci_d3cold_enable(pdev);
+ if (intel_uncore_unclaimed_mmio(&dev_priv->uncore))
+ drm_dbg(&dev_priv->drm,
+ "Unclaimed access during suspend, bios?\n");
+
+ intel_display_power_resume(dev_priv);
+
+ ret = vlv_resume_prepare(dev_priv, true);
+
+ intel_uncore_runtime_resume(&dev_priv->uncore);
+
+ intel_runtime_pm_enable_interrupts(dev_priv);
+
+ /*
+ * No point of rolling back things in case of an error, as the best
+ * we can do is to hope that things will still work (and disable RPM).
+ */
+ intel_gt_runtime_resume(to_gt(dev_priv));
+
+ /*
+ * On VLV/CHV display interrupts are part of the display
+ * power well, so hpd is reinitialized from there. For
+ * everyone else do it here.
+ */
+ if (!IS_VALLEYVIEW(dev_priv) && !IS_CHERRYVIEW(dev_priv)) {
+ intel_hpd_init(dev_priv);
+ intel_hpd_poll_disable(dev_priv);
+ }
+
+ intel_enable_ipc(dev_priv);
+
+ enable_rpm_wakeref_asserts(rpm);
+
+ if (ret)
+ drm_err(&dev_priv->drm,
+ "Runtime resume failed, disabling it (%d)\n", ret);
+ else
+ drm_dbg_kms(&dev_priv->drm, "Device resumed\n");
+
+ return ret;
+}
+
+const struct dev_pm_ops i915_pm_ops = {
+ /*
+ * S0ix (via system suspend) and S3 event handlers [PMSG_SUSPEND,
+ * PMSG_RESUME]
+ */
+ .prepare = i915_pm_prepare,
+ .suspend = i915_pm_suspend,
+ .suspend_late = i915_pm_suspend_late,
+ .resume_early = i915_pm_resume_early,
+ .resume = i915_pm_resume,
+
+ /*
+ * S4 event handlers
+ * @freeze, @freeze_late : called (1) before creating the
+ * hibernation image [PMSG_FREEZE] and
+ * (2) after rebooting, before restoring
+ * the image [PMSG_QUIESCE]
+ * @thaw, @thaw_early : called (1) after creating the hibernation
+ * image, before writing it [PMSG_THAW]
+ * and (2) after failing to create or
+ * restore the image [PMSG_RECOVER]
+ * @poweroff, @poweroff_late: called after writing the hibernation
+ * image, before rebooting [PMSG_HIBERNATE]
+ * @restore, @restore_early : called after rebooting and restoring the
+ * hibernation image [PMSG_RESTORE]
+ */
+ .freeze = i915_pm_freeze,
+ .freeze_late = i915_pm_freeze_late,
+ .thaw_early = i915_pm_thaw_early,
+ .thaw = i915_pm_thaw,
+ .poweroff = i915_pm_suspend,
+ .poweroff_late = i915_pm_poweroff_late,
+ .restore_early = i915_pm_restore_early,
+ .restore = i915_pm_restore,
+
+ /* S0ix (via runtime suspend) event handlers */
+ .runtime_suspend = intel_runtime_suspend,
+ .runtime_resume = intel_runtime_resume,
+};
+
+static const struct file_operations i915_driver_fops = {
+ .owner = THIS_MODULE,
+ .open = drm_open,
+ .release = drm_release_noglobal,
+ .unlocked_ioctl = drm_ioctl,
+ .mmap = i915_gem_mmap,
+ .poll = drm_poll,
+ .read = drm_read,
+ .compat_ioctl = i915_ioc32_compat_ioctl,
+ .llseek = noop_llseek,
+};
+
+static int
+i915_gem_reject_pin_ioctl(struct drm_device *dev, void *data,
+ struct drm_file *file)
+{
+ return -ENODEV;
+}
+
+static const struct drm_ioctl_desc i915_ioctls[] = {
+ DRM_IOCTL_DEF_DRV(I915_INIT, drm_noop, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY),
+ DRM_IOCTL_DEF_DRV(I915_FLUSH, drm_noop, DRM_AUTH),
+ DRM_IOCTL_DEF_DRV(I915_FLIP, drm_noop, DRM_AUTH),
+ DRM_IOCTL_DEF_DRV(I915_BATCHBUFFER, drm_noop, DRM_AUTH),
+ DRM_IOCTL_DEF_DRV(I915_IRQ_EMIT, drm_noop, DRM_AUTH),
+ DRM_IOCTL_DEF_DRV(I915_IRQ_WAIT, drm_noop, DRM_AUTH),
+ DRM_IOCTL_DEF_DRV(I915_GETPARAM, i915_getparam_ioctl, DRM_RENDER_ALLOW),
+ DRM_IOCTL_DEF_DRV(I915_SETPARAM, drm_noop, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY),
+ DRM_IOCTL_DEF_DRV(I915_ALLOC, drm_noop, DRM_AUTH),
+ DRM_IOCTL_DEF_DRV(I915_FREE, drm_noop, DRM_AUTH),
+ DRM_IOCTL_DEF_DRV(I915_INIT_HEAP, drm_noop, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY),
+ DRM_IOCTL_DEF_DRV(I915_CMDBUFFER, drm_noop, DRM_AUTH),
+ DRM_IOCTL_DEF_DRV(I915_DESTROY_HEAP, drm_noop, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY),
+ DRM_IOCTL_DEF_DRV(I915_SET_VBLANK_PIPE, drm_noop, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY),
+ DRM_IOCTL_DEF_DRV(I915_GET_VBLANK_PIPE, drm_noop, DRM_AUTH),
+ DRM_IOCTL_DEF_DRV(I915_VBLANK_SWAP, drm_noop, DRM_AUTH),
+ DRM_IOCTL_DEF_DRV(I915_HWS_ADDR, drm_noop, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY),
+ DRM_IOCTL_DEF_DRV(I915_GEM_INIT, drm_noop, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY),
+ DRM_IOCTL_DEF_DRV(I915_GEM_EXECBUFFER, drm_invalid_op, DRM_AUTH),
+ DRM_IOCTL_DEF_DRV(I915_GEM_EXECBUFFER2_WR, i915_gem_execbuffer2_ioctl, DRM_RENDER_ALLOW),
+ DRM_IOCTL_DEF_DRV(I915_GEM_PIN, i915_gem_reject_pin_ioctl, DRM_AUTH|DRM_ROOT_ONLY),
+ DRM_IOCTL_DEF_DRV(I915_GEM_UNPIN, i915_gem_reject_pin_ioctl, DRM_AUTH|DRM_ROOT_ONLY),
+ DRM_IOCTL_DEF_DRV(I915_GEM_BUSY, i915_gem_busy_ioctl, DRM_RENDER_ALLOW),
+ DRM_IOCTL_DEF_DRV(I915_GEM_SET_CACHING, i915_gem_set_caching_ioctl, DRM_RENDER_ALLOW),
+ DRM_IOCTL_DEF_DRV(I915_GEM_GET_CACHING, i915_gem_get_caching_ioctl, DRM_RENDER_ALLOW),
+ DRM_IOCTL_DEF_DRV(I915_GEM_THROTTLE, i915_gem_throttle_ioctl, DRM_RENDER_ALLOW),
+ DRM_IOCTL_DEF_DRV(I915_GEM_ENTERVT, drm_noop, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY),
+ DRM_IOCTL_DEF_DRV(I915_GEM_LEAVEVT, drm_noop, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY),
+ DRM_IOCTL_DEF_DRV(I915_GEM_CREATE, i915_gem_create_ioctl, DRM_RENDER_ALLOW),
+ DRM_IOCTL_DEF_DRV(I915_GEM_CREATE_EXT, i915_gem_create_ext_ioctl, DRM_RENDER_ALLOW),
+ DRM_IOCTL_DEF_DRV(I915_GEM_PREAD, i915_gem_pread_ioctl, DRM_RENDER_ALLOW),
+ DRM_IOCTL_DEF_DRV(I915_GEM_PWRITE, i915_gem_pwrite_ioctl, DRM_RENDER_ALLOW),
+ DRM_IOCTL_DEF_DRV(I915_GEM_MMAP, i915_gem_mmap_ioctl, DRM_RENDER_ALLOW),
+ DRM_IOCTL_DEF_DRV(I915_GEM_MMAP_OFFSET, i915_gem_mmap_offset_ioctl, DRM_RENDER_ALLOW),
+ DRM_IOCTL_DEF_DRV(I915_GEM_SET_DOMAIN, i915_gem_set_domain_ioctl, DRM_RENDER_ALLOW),
+ DRM_IOCTL_DEF_DRV(I915_GEM_SW_FINISH, i915_gem_sw_finish_ioctl, DRM_RENDER_ALLOW),
+ DRM_IOCTL_DEF_DRV(I915_GEM_SET_TILING, i915_gem_set_tiling_ioctl, DRM_RENDER_ALLOW),
+ DRM_IOCTL_DEF_DRV(I915_GEM_GET_TILING, i915_gem_get_tiling_ioctl, DRM_RENDER_ALLOW),
+ DRM_IOCTL_DEF_DRV(I915_GEM_GET_APERTURE, i915_gem_get_aperture_ioctl, DRM_RENDER_ALLOW),
+ DRM_IOCTL_DEF_DRV(I915_GET_PIPE_FROM_CRTC_ID, intel_get_pipe_from_crtc_id_ioctl, 0),
+ DRM_IOCTL_DEF_DRV(I915_GEM_MADVISE, i915_gem_madvise_ioctl, DRM_RENDER_ALLOW),
+ DRM_IOCTL_DEF_DRV(I915_OVERLAY_PUT_IMAGE, intel_overlay_put_image_ioctl, DRM_MASTER),
+ DRM_IOCTL_DEF_DRV(I915_OVERLAY_ATTRS, intel_overlay_attrs_ioctl, DRM_MASTER),
+ DRM_IOCTL_DEF_DRV(I915_SET_SPRITE_COLORKEY, intel_sprite_set_colorkey_ioctl, DRM_MASTER),
+ DRM_IOCTL_DEF_DRV(I915_GET_SPRITE_COLORKEY, drm_noop, DRM_MASTER),
+ DRM_IOCTL_DEF_DRV(I915_GEM_WAIT, i915_gem_wait_ioctl, DRM_RENDER_ALLOW),
+ DRM_IOCTL_DEF_DRV(I915_GEM_CONTEXT_CREATE_EXT, i915_gem_context_create_ioctl, DRM_RENDER_ALLOW),
+ DRM_IOCTL_DEF_DRV(I915_GEM_CONTEXT_DESTROY, i915_gem_context_destroy_ioctl, DRM_RENDER_ALLOW),
+ DRM_IOCTL_DEF_DRV(I915_REG_READ, i915_reg_read_ioctl, DRM_RENDER_ALLOW),
+ DRM_IOCTL_DEF_DRV(I915_GET_RESET_STATS, i915_gem_context_reset_stats_ioctl, DRM_RENDER_ALLOW),
+ DRM_IOCTL_DEF_DRV(I915_GEM_USERPTR, i915_gem_userptr_ioctl, DRM_RENDER_ALLOW),
+ DRM_IOCTL_DEF_DRV(I915_GEM_CONTEXT_GETPARAM, i915_gem_context_getparam_ioctl, DRM_RENDER_ALLOW),
+ DRM_IOCTL_DEF_DRV(I915_GEM_CONTEXT_SETPARAM, i915_gem_context_setparam_ioctl, DRM_RENDER_ALLOW),
+ DRM_IOCTL_DEF_DRV(I915_PERF_OPEN, i915_perf_open_ioctl, DRM_RENDER_ALLOW),
+ DRM_IOCTL_DEF_DRV(I915_PERF_ADD_CONFIG, i915_perf_add_config_ioctl, DRM_RENDER_ALLOW),
+ DRM_IOCTL_DEF_DRV(I915_PERF_REMOVE_CONFIG, i915_perf_remove_config_ioctl, DRM_RENDER_ALLOW),
+ DRM_IOCTL_DEF_DRV(I915_QUERY, i915_query_ioctl, DRM_RENDER_ALLOW),
+ DRM_IOCTL_DEF_DRV(I915_GEM_VM_CREATE, i915_gem_vm_create_ioctl, DRM_RENDER_ALLOW),
+ DRM_IOCTL_DEF_DRV(I915_GEM_VM_DESTROY, i915_gem_vm_destroy_ioctl, DRM_RENDER_ALLOW),
+};
+
+static const struct drm_driver i915_drm_driver = {
+ /* Don't use MTRRs here; the Xserver or userspace app should
+ * deal with them for Intel hardware.
+ */
+ .driver_features =
+ DRIVER_GEM |
+ DRIVER_RENDER | DRIVER_MODESET | DRIVER_ATOMIC | DRIVER_SYNCOBJ |
+ DRIVER_SYNCOBJ_TIMELINE,
+ .release = i915_driver_release,
+ .open = i915_driver_open,
+ .lastclose = i915_driver_lastclose,
+ .postclose = i915_driver_postclose,
+
+ .prime_handle_to_fd = drm_gem_prime_handle_to_fd,
+ .prime_fd_to_handle = drm_gem_prime_fd_to_handle,
+ .gem_prime_import = i915_gem_prime_import,
+
+ .dumb_create = i915_gem_dumb_create,
+ .dumb_map_offset = i915_gem_dumb_mmap_offset,
+
+ .ioctls = i915_ioctls,
+ .num_ioctls = ARRAY_SIZE(i915_ioctls),
+ .fops = &i915_driver_fops,
+ .name = DRIVER_NAME,
+ .desc = DRIVER_DESC,
+ .date = DRIVER_DATE,
+ .major = DRIVER_MAJOR,
+ .minor = DRIVER_MINOR,
+ .patchlevel = DRIVER_PATCHLEVEL,
+};
diff --git a/drivers/gpu/drm/i915/i915_driver.h b/drivers/gpu/drm/i915/i915_driver.h
new file mode 100644
index 0000000..9ef8db4
--- /dev/null
+++ b/drivers/gpu/drm/i915/i915_driver.h
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2019 Intel Corporation
+ */
+
+#ifndef __I915_DRIVER_H__
+#define __I915_DRIVER_H__
+
+#include <linux/pm.h>
+
+struct pci_dev;
+struct pci_device_id;
+struct drm_i915_private;
+
+extern const struct dev_pm_ops i915_pm_ops;
+
+int i915_driver_probe(struct pci_dev *pdev, const struct pci_device_id *ent);
+void i915_driver_remove(struct drm_i915_private *i915);
+void i915_driver_shutdown(struct drm_i915_private *i915);
+
+int i915_driver_resume_switcheroo(struct drm_i915_private *i915);
+int i915_driver_suspend_switcheroo(struct drm_i915_private *i915, pm_message_t state);
+
+#endif /* __I915_DRIVER_H__ */
diff --git a/drivers/gpu/drm/i915/i915_drv.c b/drivers/gpu/drm/i915/i915_drv.c
deleted file mode 100644
index b18a250..0000000
--- a/drivers/gpu/drm/i915/i915_drv.c
+++ /dev/null
@@ -1,1809 +0,0 @@
-/* i915_drv.c -- i830,i845,i855,i865,i915 driver -*- linux-c -*-
- */
-/*
- *
- * Copyright 2003 Tungsten Graphics, Inc., Cedar Park, Texas.
- * All Rights Reserved.
- *
- * Permission is hereby granted, free of charge, to any person obtaining a
- * copy of this software and associated documentation files (the
- * "Software"), to deal in the Software without restriction, including
- * without limitation the rights to use, copy, modify, merge, publish,
- * distribute, sub license, and/or sell copies of the Software, and to
- * permit persons to whom the Software is furnished to do so, subject to
- * the following conditions:
- *
- * The above copyright notice and this permission notice (including the
- * next paragraph) shall be included in all copies or substantial portions
- * of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
- * OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
- * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT.
- * IN NO EVENT SHALL TUNGSTEN GRAPHICS AND/OR ITS SUPPLIERS BE LIABLE FOR
- * ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
- * TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
- * SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
- *
- */
-
-#include <linux/acpi.h>
-#include <linux/device.h>
-#include <linux/oom.h>
-#include <linux/module.h>
-#include <linux/pci.h>
-#include <linux/pm.h>
-#include <linux/pm_runtime.h>
-#include <linux/pnp.h>
-#include <linux/slab.h>
-#include <linux/vga_switcheroo.h>
-#include <linux/vt.h>
-
-#include <drm/drm_aperture.h>
-#include <drm/drm_atomic_helper.h>
-#include <drm/drm_ioctl.h>
-#include <drm/drm_managed.h>
-#include <drm/drm_probe_helper.h>
-
-#include "display/intel_acpi.h"
-#include "display/intel_bw.h"
-#include "display/intel_cdclk.h"
-#include "display/intel_dmc.h"
-#include "display/intel_display_types.h"
-#include "display/intel_dp.h"
-#include "display/intel_fbdev.h"
-#include "display/intel_hotplug.h"
-#include "display/intel_overlay.h"
-#include "display/intel_pipe_crc.h"
-#include "display/intel_pps.h"
-#include "display/intel_sprite.h"
-#include "display/intel_vga.h"
-
-#include "gem/i915_gem_context.h"
-#include "gem/i915_gem_ioctls.h"
-#include "gem/i915_gem_mman.h"
-#include "gem/i915_gem_pm.h"
-#include "gt/intel_gt.h"
-#include "gt/intel_gt_pm.h"
-#include "gt/intel_rc6.h"
-
-#include "pxp/intel_pxp_pm.h"
-
-#include "i915_debugfs.h"
-#include "i915_drv.h"
-#include "i915_ioc32.h"
-#include "i915_irq.h"
-#include "i915_memcpy.h"
-#include "i915_perf.h"
-#include "i915_query.h"
-#include "i915_suspend.h"
-#include "i915_switcheroo.h"
-#include "i915_sysfs.h"
-#include "i915_trace.h"
-#include "i915_vgpu.h"
-#include "intel_dram.h"
-#include "intel_gvt.h"
-#include "intel_memory_region.h"
-#include "intel_pcode.h"
-#include "intel_pm.h"
-#include "intel_region_ttm.h"
-#include "vlv_suspend.h"
-
-static const struct drm_driver driver;
-
-static int i915_get_bridge_dev(struct drm_i915_private *dev_priv)
-{
- int domain = pci_domain_nr(to_pci_dev(dev_priv->drm.dev)->bus);
-
- dev_priv->bridge_dev =
- pci_get_domain_bus_and_slot(domain, 0, PCI_DEVFN(0, 0));
- if (!dev_priv->bridge_dev) {
- drm_err(&dev_priv->drm, "bridge device not found\n");
- return -EIO;
- }
- return 0;
-}
-
-/* Allocate space for the MCH regs if needed, return nonzero on error */
-static int
-intel_alloc_mchbar_resource(struct drm_i915_private *dev_priv)
-{
- int reg = GRAPHICS_VER(dev_priv) >= 4 ? MCHBAR_I965 : MCHBAR_I915;
- u32 temp_lo, temp_hi = 0;
- u64 mchbar_addr;
- int ret;
-
- if (GRAPHICS_VER(dev_priv) >= 4)
- pci_read_config_dword(dev_priv->bridge_dev, reg + 4, &temp_hi);
- pci_read_config_dword(dev_priv->bridge_dev, reg, &temp_lo);
- mchbar_addr = ((u64)temp_hi << 32) | temp_lo;
-
- /* If ACPI doesn't have it, assume we need to allocate it ourselves */
-#ifdef CONFIG_PNP
- if (mchbar_addr &&
- pnp_range_reserved(mchbar_addr, mchbar_addr + MCHBAR_SIZE))
- return 0;
-#endif
-
- /* Get some space for it */
- dev_priv->mch_res.name = "i915 MCHBAR";
- dev_priv->mch_res.flags = IORESOURCE_MEM;
- ret = pci_bus_alloc_resource(dev_priv->bridge_dev->bus,
- &dev_priv->mch_res,
- MCHBAR_SIZE, MCHBAR_SIZE,
- PCIBIOS_MIN_MEM,
- 0, pcibios_align_resource,
- dev_priv->bridge_dev);
- if (ret) {
- drm_dbg(&dev_priv->drm, "failed bus alloc: %d\n", ret);
- dev_priv->mch_res.start = 0;
- return ret;
- }
-
- if (GRAPHICS_VER(dev_priv) >= 4)
- pci_write_config_dword(dev_priv->bridge_dev, reg + 4,
- upper_32_bits(dev_priv->mch_res.start));
-
- pci_write_config_dword(dev_priv->bridge_dev, reg,
- lower_32_bits(dev_priv->mch_res.start));
- return 0;
-}
-
-/* Setup MCHBAR if possible, return true if we should disable it again */
-static void
-intel_setup_mchbar(struct drm_i915_private *dev_priv)
-{
- int mchbar_reg = GRAPHICS_VER(dev_priv) >= 4 ? MCHBAR_I965 : MCHBAR_I915;
- u32 temp;
- bool enabled;
-
- if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv))
- return;
-
- dev_priv->mchbar_need_disable = false;
-
- if (IS_I915G(dev_priv) || IS_I915GM(dev_priv)) {
- pci_read_config_dword(dev_priv->bridge_dev, DEVEN, &temp);
- enabled = !!(temp & DEVEN_MCHBAR_EN);
- } else {
- pci_read_config_dword(dev_priv->bridge_dev, mchbar_reg, &temp);
- enabled = temp & 1;
- }
-
- /* If it's already enabled, don't have to do anything */
- if (enabled)
- return;
-
- if (intel_alloc_mchbar_resource(dev_priv))
- return;
-
- dev_priv->mchbar_need_disable = true;
-
- /* Space is allocated or reserved, so enable it. */
- if (IS_I915G(dev_priv) || IS_I915GM(dev_priv)) {
- pci_write_config_dword(dev_priv->bridge_dev, DEVEN,
- temp | DEVEN_MCHBAR_EN);
- } else {
- pci_read_config_dword(dev_priv->bridge_dev, mchbar_reg, &temp);
- pci_write_config_dword(dev_priv->bridge_dev, mchbar_reg, temp | 1);
- }
-}
-
-static void
-intel_teardown_mchbar(struct drm_i915_private *dev_priv)
-{
- int mchbar_reg = GRAPHICS_VER(dev_priv) >= 4 ? MCHBAR_I965 : MCHBAR_I915;
-
- if (dev_priv->mchbar_need_disable) {
- if (IS_I915G(dev_priv) || IS_I915GM(dev_priv)) {
- u32 deven_val;
-
- pci_read_config_dword(dev_priv->bridge_dev, DEVEN,
- &deven_val);
- deven_val &= ~DEVEN_MCHBAR_EN;
- pci_write_config_dword(dev_priv->bridge_dev, DEVEN,
- deven_val);
- } else {
- u32 mchbar_val;
-
- pci_read_config_dword(dev_priv->bridge_dev, mchbar_reg,
- &mchbar_val);
- mchbar_val &= ~1;
- pci_write_config_dword(dev_priv->bridge_dev, mchbar_reg,
- mchbar_val);
- }
- }
-
- if (dev_priv->mch_res.start)
- release_resource(&dev_priv->mch_res);
-}
-
-static int i915_workqueues_init(struct drm_i915_private *dev_priv)
-{
- /*
- * The i915 workqueue is primarily used for batched retirement of
- * requests (and thus managing bo) once the task has been completed
- * by the GPU. i915_retire_requests() is called directly when we
- * need high-priority retirement, such as waiting for an explicit
- * bo.
- *
- * It is also used for periodic low-priority events, such as
- * idle-timers and recording error state.
- *
- * All tasks on the workqueue are expected to acquire the dev mutex
- * so there is no point in running more than one instance of the
- * workqueue at any time. Use an ordered one.
- */
- dev_priv->wq = alloc_ordered_workqueue("i915", 0);
- if (dev_priv->wq == NULL)
- goto out_err;
-
- dev_priv->hotplug.dp_wq = alloc_ordered_workqueue("i915-dp", 0);
- if (dev_priv->hotplug.dp_wq == NULL)
- goto out_free_wq;
-
- return 0;
-
-out_free_wq:
- destroy_workqueue(dev_priv->wq);
-out_err:
- drm_err(&dev_priv->drm, "Failed to allocate workqueues.\n");
-
- return -ENOMEM;
-}
-
-static void i915_workqueues_cleanup(struct drm_i915_private *dev_priv)
-{
- destroy_workqueue(dev_priv->hotplug.dp_wq);
- destroy_workqueue(dev_priv->wq);
-}
-
-/*
- * We don't keep the workarounds for pre-production hardware, so we expect our
- * driver to fail on these machines in one way or another. A little warning on
- * dmesg may help both the user and the bug triagers.
- *
- * Our policy for removing pre-production workarounds is to keep the
- * current gen workarounds as a guide to the bring-up of the next gen
- * (workarounds have a habit of persisting!). Anything older than that
- * should be removed along with the complications they introduce.
- */
-static void intel_detect_preproduction_hw(struct drm_i915_private *dev_priv)
-{
- bool pre = false;
-
- pre |= IS_HSW_EARLY_SDV(dev_priv);
- pre |= IS_SKYLAKE(dev_priv) && INTEL_REVID(dev_priv) < 0x6;
- pre |= IS_BROXTON(dev_priv) && INTEL_REVID(dev_priv) < 0xA;
- pre |= IS_KABYLAKE(dev_priv) && INTEL_REVID(dev_priv) < 0x1;
- pre |= IS_GEMINILAKE(dev_priv) && INTEL_REVID(dev_priv) < 0x3;
- pre |= IS_ICELAKE(dev_priv) && INTEL_REVID(dev_priv) < 0x7;
-
- if (pre) {
- drm_err(&dev_priv->drm, "This is a pre-production stepping. "
- "It may not be fully functional.\n");
- add_taint(TAINT_MACHINE_CHECK, LOCKDEP_STILL_OK);
- }
-}
-
-static void sanitize_gpu(struct drm_i915_private *i915)
-{
- if (!INTEL_INFO(i915)->gpu_reset_clobbers_display)
- __intel_gt_reset(&i915->gt, ALL_ENGINES);
-}
-
-/**
- * i915_driver_early_probe - setup state not requiring device access
- * @dev_priv: device private
- *
- * Initialize everything that is a "SW-only" state, that is state not
- * requiring accessing the device or exposing the driver via kernel internal
- * or userspace interfaces. Example steps belonging here: lock initialization,
- * system memory allocation, setting up device specific attributes and
- * function hooks not requiring accessing the device.
- */
-static int i915_driver_early_probe(struct drm_i915_private *dev_priv)
-{
- int ret = 0;
-
- if (i915_inject_probe_failure(dev_priv))
- return -ENODEV;
-
- intel_device_info_subplatform_init(dev_priv);
- intel_step_init(dev_priv);
-
- intel_uncore_mmio_debug_init_early(&dev_priv->mmio_debug);
- intel_uncore_init_early(&dev_priv->uncore, dev_priv);
-
- spin_lock_init(&dev_priv->irq_lock);
- spin_lock_init(&dev_priv->gpu_error.lock);
- mutex_init(&dev_priv->backlight_lock);
-
- mutex_init(&dev_priv->sb_lock);
- cpu_latency_qos_add_request(&dev_priv->sb_qos, PM_QOS_DEFAULT_VALUE);
-
- mutex_init(&dev_priv->av_mutex);
- mutex_init(&dev_priv->wm.wm_mutex);
- mutex_init(&dev_priv->pps_mutex);
- mutex_init(&dev_priv->hdcp_comp_mutex);
-
- i915_memcpy_init_early(dev_priv);
- intel_runtime_pm_init_early(&dev_priv->runtime_pm);
-
- ret = i915_workqueues_init(dev_priv);
- if (ret < 0)
- return ret;
-
- ret = vlv_suspend_init(dev_priv);
- if (ret < 0)
- goto err_workqueues;
-
- ret = intel_region_ttm_device_init(dev_priv);
- if (ret)
- goto err_ttm;
-
- intel_wopcm_init_early(&dev_priv->wopcm);
-
- intel_gt_init_early(&dev_priv->gt, dev_priv);
-
- i915_gem_init_early(dev_priv);
-
- /* This must be called before any calls to HAS_PCH_* */
- intel_detect_pch(dev_priv);
-
- intel_pm_setup(dev_priv);
- ret = intel_power_domains_init(dev_priv);
- if (ret < 0)
- goto err_gem;
- intel_irq_init(dev_priv);
- intel_init_display_hooks(dev_priv);
- intel_init_clock_gating_hooks(dev_priv);
-
- intel_detect_preproduction_hw(dev_priv);
-
- return 0;
-
-err_gem:
- i915_gem_cleanup_early(dev_priv);
- intel_gt_driver_late_release(&dev_priv->gt);
- intel_region_ttm_device_fini(dev_priv);
-err_ttm:
- vlv_suspend_cleanup(dev_priv);
-err_workqueues:
- i915_workqueues_cleanup(dev_priv);
- return ret;
-}
-
-/**
- * i915_driver_late_release - cleanup the setup done in
- * i915_driver_early_probe()
- * @dev_priv: device private
- */
-static void i915_driver_late_release(struct drm_i915_private *dev_priv)
-{
- intel_irq_fini(dev_priv);
- intel_power_domains_cleanup(dev_priv);
- i915_gem_cleanup_early(dev_priv);
- intel_gt_driver_late_release(&dev_priv->gt);
- intel_region_ttm_device_fini(dev_priv);
- vlv_suspend_cleanup(dev_priv);
- i915_workqueues_cleanup(dev_priv);
-
- cpu_latency_qos_remove_request(&dev_priv->sb_qos);
- mutex_destroy(&dev_priv->sb_lock);
-
- i915_params_free(&dev_priv->params);
-}
-
-/**
- * i915_driver_mmio_probe - setup device MMIO
- * @dev_priv: device private
- *
- * Setup minimal device state necessary for MMIO accesses later in the
- * initialization sequence. The setup here should avoid any other device-wide
- * side effects or exposing the driver via kernel internal or user space
- * interfaces.
- */
-static int i915_driver_mmio_probe(struct drm_i915_private *dev_priv)
-{
- int ret;
-
- if (i915_inject_probe_failure(dev_priv))
- return -ENODEV;
-
- ret = i915_get_bridge_dev(dev_priv);
- if (ret < 0)
- return ret;
-
- ret = intel_uncore_init_mmio(&dev_priv->uncore);
- if (ret < 0)
- goto err_bridge;
-
- /* Try to make sure MCHBAR is enabled before poking at it */
- intel_setup_mchbar(dev_priv);
- intel_device_info_runtime_init(dev_priv);
-
- ret = intel_gt_init_mmio(&dev_priv->gt);
- if (ret)
- goto err_uncore;
-
- /* As early as possible, scrub existing GPU state before clobbering */
- sanitize_gpu(dev_priv);
-
- return 0;
-
-err_uncore:
- intel_teardown_mchbar(dev_priv);
- intel_uncore_fini_mmio(&dev_priv->uncore);
-err_bridge:
- pci_dev_put(dev_priv->bridge_dev);
-
- return ret;
-}
-
-/**
- * i915_driver_mmio_release - cleanup the setup done in i915_driver_mmio_probe()
- * @dev_priv: device private
- */
-static void i915_driver_mmio_release(struct drm_i915_private *dev_priv)
-{
- intel_teardown_mchbar(dev_priv);
- intel_uncore_fini_mmio(&dev_priv->uncore);
- pci_dev_put(dev_priv->bridge_dev);
-}
-
-static void intel_sanitize_options(struct drm_i915_private *dev_priv)
-{
- intel_gvt_sanitize_options(dev_priv);
-}
-
-/**
- * i915_set_dma_info - set all relevant PCI dma info as configured for the
- * platform
- * @i915: valid i915 instance
- *
- * Set the dma max segment size, device and coherent masks. The dma mask set
- * needs to occur before i915_ggtt_probe_hw.
- *
- * A couple of platforms have special needs. Address them as well.
- *
- */
-static int i915_set_dma_info(struct drm_i915_private *i915)
-{
- unsigned int mask_size = INTEL_INFO(i915)->dma_mask_size;
- int ret;
-
- GEM_BUG_ON(!mask_size);
-
- /*
- * We don't have a max segment size, so set it to the max so sg's
- * debugging layer doesn't complain
- */
- dma_set_max_seg_size(i915->drm.dev, UINT_MAX);
-
- ret = dma_set_mask(i915->drm.dev, DMA_BIT_MASK(mask_size));
- if (ret)
- goto mask_err;
-
- /* overlay on gen2 is broken and can't address above 1G */
- if (GRAPHICS_VER(i915) == 2)
- mask_size = 30;
-
- /*
- * 965GM sometimes incorrectly writes to hardware status page (HWS)
- * using 32bit addressing, overwriting memory if HWS is located
- * above 4GB.
- *
- * The documentation also mentions an issue with undefined
- * behaviour if any general state is accessed within a page above 4GB,
- * which also needs to be handled carefully.
- */
- if (IS_I965G(i915) || IS_I965GM(i915))
- mask_size = 32;
-
- ret = dma_set_coherent_mask(i915->drm.dev, DMA_BIT_MASK(mask_size));
- if (ret)
- goto mask_err;
-
- return 0;
-
-mask_err:
- drm_err(&i915->drm, "Can't set DMA mask/consistent mask (%d)\n", ret);
- return ret;
-}
-
-/**
- * i915_driver_hw_probe - setup state requiring device access
- * @dev_priv: device private
- *
- * Setup state that requires accessing the device, but doesn't require
- * exposing the driver via kernel internal or userspace interfaces.
- */
-static int i915_driver_hw_probe(struct drm_i915_private *dev_priv)
-{
- struct pci_dev *pdev = to_pci_dev(dev_priv->drm.dev);
- int ret;
-
- if (i915_inject_probe_failure(dev_priv))
- return -ENODEV;
-
- if (HAS_PPGTT(dev_priv)) {
- if (intel_vgpu_active(dev_priv) &&
- !intel_vgpu_has_full_ppgtt(dev_priv)) {
- i915_report_error(dev_priv,
- "incompatible vGPU found, support for isolated ppGTT required\n");
- return -ENXIO;
- }
- }
-
- if (HAS_EXECLISTS(dev_priv)) {
- /*
- * Older GVT emulation depends upon intercepting CSB mmio,
- * which we no longer use, preferring to use the HWSP cache
- * instead.
- */
- if (intel_vgpu_active(dev_priv) &&
- !intel_vgpu_has_hwsp_emulation(dev_priv)) {
- i915_report_error(dev_priv,
- "old vGPU host found, support for HWSP emulation required\n");
- return -ENXIO;
- }
- }
-
- intel_sanitize_options(dev_priv);
-
- /* needs to be done before ggtt probe */
- intel_dram_edram_detect(dev_priv);
-
- ret = i915_set_dma_info(dev_priv);
- if (ret)
- return ret;
-
- i915_perf_init(dev_priv);
-
- ret = i915_ggtt_probe_hw(dev_priv);
- if (ret)
- goto err_perf;
-
- ret = drm_aperture_remove_conflicting_pci_framebuffers(pdev, dev_priv->drm.driver);
- if (ret)
- goto err_ggtt;
-
- ret = i915_ggtt_init_hw(dev_priv);
- if (ret)
- goto err_ggtt;
-
- ret = intel_memory_regions_hw_probe(dev_priv);
- if (ret)
- goto err_ggtt;
-
- intel_gt_init_hw_early(&dev_priv->gt, &dev_priv->ggtt);
-
- ret = intel_gt_probe_lmem(&dev_priv->gt);
- if (ret)
- goto err_mem_regions;
-
- ret = i915_ggtt_enable_hw(dev_priv);
- if (ret) {
- drm_err(&dev_priv->drm, "failed to enable GGTT\n");
- goto err_mem_regions;
- }
-
- pci_set_master(pdev);
-
- /* On the 945G/GM, the chipset reports the MSI capability on the
- * integrated graphics even though the support isn't actually there
- * according to the published specs. It doesn't appear to function
- * correctly in testing on 945G.
- * This may be a side effect of MSI having been made available for PEG
- * and the registers being closely associated.
- *
- * According to chipset errata, on the 965GM, MSI interrupts may
- * be lost or delayed, and was defeatured. MSI interrupts seem to
- * get lost on g4x as well, and interrupt delivery seems to stay
- * properly dead afterwards. So we'll just disable them for all
- * pre-gen5 chipsets.
- *
- * dp aux and gmbus irq on gen4 seems to be able to generate legacy
- * interrupts even when in MSI mode. This results in spurious
- * interrupt warnings if the legacy irq no. is shared with another
- * device. The kernel then disables that interrupt source and so
- * prevents the other device from working properly.
- */
- if (GRAPHICS_VER(dev_priv) >= 5) {
- if (pci_enable_msi(pdev) < 0)
- drm_dbg(&dev_priv->drm, "can't enable MSI");
- }
-
- ret = intel_gvt_init(dev_priv);
- if (ret)
- goto err_msi;
-
- intel_opregion_setup(dev_priv);
-
- ret = intel_pcode_init(dev_priv);
- if (ret)
- goto err_msi;
-
- /*
- * Fill the dram structure to get the system dram info. This will be
- * used for memory latency calculation.
- */
- intel_dram_detect(dev_priv);
-
- intel_bw_init_hw(dev_priv);
-
- return 0;
-
-err_msi:
- if (pdev->msi_enabled)
- pci_disable_msi(pdev);
-err_mem_regions:
- intel_memory_regions_driver_release(dev_priv);
-err_ggtt:
- i915_ggtt_driver_release(dev_priv);
- i915_gem_drain_freed_objects(dev_priv);
- i915_ggtt_driver_late_release(dev_priv);
-err_perf:
- i915_perf_fini(dev_priv);
- return ret;
-}
-
-/**
- * i915_driver_hw_remove - cleanup the setup done in i915_driver_hw_probe()
- * @dev_priv: device private
- */
-static void i915_driver_hw_remove(struct drm_i915_private *dev_priv)
-{
- struct pci_dev *pdev = to_pci_dev(dev_priv->drm.dev);
-
- i915_perf_fini(dev_priv);
-
- if (pdev->msi_enabled)
- pci_disable_msi(pdev);
-}
-
-/**
- * i915_driver_register - register the driver with the rest of the system
- * @dev_priv: device private
- *
- * Perform any steps necessary to make the driver available via kernel
- * internal or userspace interfaces.
- */
-static void i915_driver_register(struct drm_i915_private *dev_priv)
-{
- struct drm_device *dev = &dev_priv->drm;
-
- i915_gem_driver_register(dev_priv);
- i915_pmu_register(dev_priv);
-
- intel_vgpu_register(dev_priv);
-
- /* Reveal our presence to userspace */
- if (drm_dev_register(dev, 0)) {
- drm_err(&dev_priv->drm,
- "Failed to register driver for userspace access!\n");
- return;
- }
-
- i915_debugfs_register(dev_priv);
- i915_setup_sysfs(dev_priv);
-
- /* Depends on sysfs having been initialized */
- i915_perf_register(dev_priv);
-
- intel_gt_driver_register(&dev_priv->gt);
-
- intel_display_driver_register(dev_priv);
-
- intel_power_domains_enable(dev_priv);
- intel_runtime_pm_enable(&dev_priv->runtime_pm);
-
- intel_register_dsm_handler();
-
- if (i915_switcheroo_register(dev_priv))
- drm_err(&dev_priv->drm, "Failed to register vga switcheroo!\n");
-}
-
-/**
- * i915_driver_unregister - cleanup the registration done in i915_driver_regiser()
- * @dev_priv: device private
- */
-static void i915_driver_unregister(struct drm_i915_private *dev_priv)
-{
- i915_switcheroo_unregister(dev_priv);
-
- intel_unregister_dsm_handler();
-
- intel_runtime_pm_disable(&dev_priv->runtime_pm);
- intel_power_domains_disable(dev_priv);
-
- intel_display_driver_unregister(dev_priv);
-
- intel_gt_driver_unregister(&dev_priv->gt);
-
- i915_perf_unregister(dev_priv);
- i915_pmu_unregister(dev_priv);
-
- i915_teardown_sysfs(dev_priv);
- drm_dev_unplug(&dev_priv->drm);
-
- i915_gem_driver_unregister(dev_priv);
-}
-
-static void i915_welcome_messages(struct drm_i915_private *dev_priv)
-{
- if (drm_debug_enabled(DRM_UT_DRIVER)) {
- struct drm_printer p = drm_debug_printer("i915 device info:");
-
- drm_printf(&p, "pciid=0x%04x rev=0x%02x platform=%s (subplatform=0x%x) gen=%i\n",
- INTEL_DEVID(dev_priv),
- INTEL_REVID(dev_priv),
- intel_platform_name(INTEL_INFO(dev_priv)->platform),
- intel_subplatform(RUNTIME_INFO(dev_priv),
- INTEL_INFO(dev_priv)->platform),
- GRAPHICS_VER(dev_priv));
-
- intel_device_info_print_static(INTEL_INFO(dev_priv), &p);
- intel_device_info_print_runtime(RUNTIME_INFO(dev_priv), &p);
- intel_gt_info_print(&dev_priv->gt.info, &p);
- }
-
- if (IS_ENABLED(CONFIG_DRM_I915_DEBUG))
- drm_info(&dev_priv->drm, "DRM_I915_DEBUG enabled\n");
- if (IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM))
- drm_info(&dev_priv->drm, "DRM_I915_DEBUG_GEM enabled\n");
- if (IS_ENABLED(CONFIG_DRM_I915_DEBUG_RUNTIME_PM))
- drm_info(&dev_priv->drm,
- "DRM_I915_DEBUG_RUNTIME_PM enabled\n");
-}
-
-static struct drm_i915_private *
-i915_driver_create(struct pci_dev *pdev, const struct pci_device_id *ent)
-{
- const struct intel_device_info *match_info =
- (struct intel_device_info *)ent->driver_data;
- struct intel_device_info *device_info;
- struct drm_i915_private *i915;
-
- i915 = devm_drm_dev_alloc(&pdev->dev, &driver,
- struct drm_i915_private, drm);
- if (IS_ERR(i915))
- return i915;
-
- pci_set_drvdata(pdev, i915);
-
- /* Device parameters start as a copy of module parameters. */
- i915_params_copy(&i915->params, &i915_modparams);
-
- /* Setup the write-once "constant" device info */
- device_info = mkwrite_device_info(i915);
- memcpy(device_info, match_info, sizeof(*device_info));
- RUNTIME_INFO(i915)->device_id = pdev->device;
-
- return i915;
-}
-
-/**
- * i915_driver_probe - setup chip and create an initial config
- * @pdev: PCI device
- * @ent: matching PCI ID entry
- *
- * The driver probe routine has to do several things:
- * - drive output discovery via intel_modeset_init()
- * - initialize the memory manager
- * - allocate initial config memory
- * - setup the DRM framebuffer with the allocated memory
- */
-int i915_driver_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
-{
- const struct intel_device_info *match_info =
- (struct intel_device_info *)ent->driver_data;
- struct drm_i915_private *i915;
- int ret;
-
- i915 = i915_driver_create(pdev, ent);
- if (IS_ERR(i915))
- return PTR_ERR(i915);
-
- /* Disable nuclear pageflip by default on pre-ILK */
- if (!i915->params.nuclear_pageflip && match_info->graphics_ver < 5)
- i915->drm.driver_features &= ~DRIVER_ATOMIC;
-
- /*
- * Check if we support fake LMEM -- for now we only unleash this for
- * the live selftests(test-and-exit).
- */
-#if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
- if (IS_ENABLED(CONFIG_DRM_I915_UNSTABLE_FAKE_LMEM)) {
- if (GRAPHICS_VER(i915) >= 9 && i915_selftest.live < 0 &&
- i915->params.fake_lmem_start) {
- mkwrite_device_info(i915)->memory_regions =
- REGION_SMEM | REGION_LMEM | REGION_STOLEN_SMEM;
- GEM_BUG_ON(!HAS_LMEM(i915));
- }
- }
-#endif
-
- ret = pci_enable_device(pdev);
- if (ret)
- goto out_fini;
-
- ret = i915_driver_early_probe(i915);
- if (ret < 0)
- goto out_pci_disable;
-
- disable_rpm_wakeref_asserts(&i915->runtime_pm);
-
- intel_vgpu_detect(i915);
-
- ret = i915_driver_mmio_probe(i915);
- if (ret < 0)
- goto out_runtime_pm_put;
-
- ret = i915_driver_hw_probe(i915);
- if (ret < 0)
- goto out_cleanup_mmio;
-
- ret = intel_modeset_init_noirq(i915);
- if (ret < 0)
- goto out_cleanup_hw;
-
- ret = intel_irq_install(i915);
- if (ret)
- goto out_cleanup_modeset;
-
- ret = intel_modeset_init_nogem(i915);
- if (ret)
- goto out_cleanup_irq;
-
- ret = i915_gem_init(i915);
- if (ret)
- goto out_cleanup_modeset2;
-
- ret = intel_modeset_init(i915);
- if (ret)
- goto out_cleanup_gem;
-
- i915_driver_register(i915);
-
- enable_rpm_wakeref_asserts(&i915->runtime_pm);
-
- i915_welcome_messages(i915);
-
- i915->do_release = true;
-
- return 0;
-
-out_cleanup_gem:
- i915_gem_suspend(i915);
- i915_gem_driver_remove(i915);
- i915_gem_driver_release(i915);
-out_cleanup_modeset2:
- /* FIXME clean up the error path */
- intel_modeset_driver_remove(i915);
- intel_irq_uninstall(i915);
- intel_modeset_driver_remove_noirq(i915);
- goto out_cleanup_modeset;
-out_cleanup_irq:
- intel_irq_uninstall(i915);
-out_cleanup_modeset:
- intel_modeset_driver_remove_nogem(i915);
-out_cleanup_hw:
- i915_driver_hw_remove(i915);
- intel_memory_regions_driver_release(i915);
- i915_ggtt_driver_release(i915);
- i915_gem_drain_freed_objects(i915);
- i915_ggtt_driver_late_release(i915);
-out_cleanup_mmio:
- i915_driver_mmio_release(i915);
-out_runtime_pm_put:
- enable_rpm_wakeref_asserts(&i915->runtime_pm);
- i915_driver_late_release(i915);
-out_pci_disable:
- pci_disable_device(pdev);
-out_fini:
- i915_probe_error(i915, "Device initialization failed (%d)\n", ret);
- return ret;
-}
-
-void i915_driver_remove(struct drm_i915_private *i915)
-{
- disable_rpm_wakeref_asserts(&i915->runtime_pm);
-
- i915_driver_unregister(i915);
-
- /* Flush any external code that still may be under the RCU lock */
- synchronize_rcu();
-
- i915_gem_suspend(i915);
-
- intel_gvt_driver_remove(i915);
-
- intel_modeset_driver_remove(i915);
-
- intel_irq_uninstall(i915);
-
- intel_modeset_driver_remove_noirq(i915);
-
- i915_reset_error_state(i915);
- i915_gem_driver_remove(i915);
-
- intel_modeset_driver_remove_nogem(i915);
-
- i915_driver_hw_remove(i915);
-
- enable_rpm_wakeref_asserts(&i915->runtime_pm);
-}
-
-static void i915_driver_release(struct drm_device *dev)
-{
- struct drm_i915_private *dev_priv = to_i915(dev);
- struct intel_runtime_pm *rpm = &dev_priv->runtime_pm;
-
- if (!dev_priv->do_release)
- return;
-
- disable_rpm_wakeref_asserts(rpm);
-
- i915_gem_driver_release(dev_priv);
-
- intel_memory_regions_driver_release(dev_priv);
- i915_ggtt_driver_release(dev_priv);
- i915_gem_drain_freed_objects(dev_priv);
- i915_ggtt_driver_late_release(dev_priv);
-
- i915_driver_mmio_release(dev_priv);
-
- enable_rpm_wakeref_asserts(rpm);
- intel_runtime_pm_driver_release(rpm);
-
- i915_driver_late_release(dev_priv);
-}
-
-static int i915_driver_open(struct drm_device *dev, struct drm_file *file)
-{
- struct drm_i915_private *i915 = to_i915(dev);
- int ret;
-
- ret = i915_gem_open(i915, file);
- if (ret)
- return ret;
-
- return 0;
-}
-
-/**
- * i915_driver_lastclose - clean up after all DRM clients have exited
- * @dev: DRM device
- *
- * Take care of cleaning up after all DRM clients have exited. In the
- * mode setting case, we want to restore the kernel's initial mode (just
- * in case the last client left us in a bad state).
- *
- * Additionally, in the non-mode setting case, we'll tear down the GTT
- * and DMA structures, since the kernel won't be using them, and clea
- * up any GEM state.
- */
-static void i915_driver_lastclose(struct drm_device *dev)
-{
- struct drm_i915_private *i915 = to_i915(dev);
-
- intel_fbdev_restore_mode(dev);
-
- if (HAS_DISPLAY(i915))
- vga_switcheroo_process_delayed_switch();
-}
-
-static void i915_driver_postclose(struct drm_device *dev, struct drm_file *file)
-{
- struct drm_i915_file_private *file_priv = file->driver_priv;
-
- i915_gem_context_close(file);
-
- kfree_rcu(file_priv, rcu);
-
- /* Catch up with all the deferred frees from "this" client */
- i915_gem_flush_free_objects(to_i915(dev));
-}
-
-static void intel_suspend_encoders(struct drm_i915_private *dev_priv)
-{
- struct drm_device *dev = &dev_priv->drm;
- struct intel_encoder *encoder;
-
- if (!HAS_DISPLAY(dev_priv))
- return;
-
- drm_modeset_lock_all(dev);
- for_each_intel_encoder(dev, encoder)
- if (encoder->suspend)
- encoder->suspend(encoder);
- drm_modeset_unlock_all(dev);
-}
-
-static void intel_shutdown_encoders(struct drm_i915_private *dev_priv)
-{
- struct drm_device *dev = &dev_priv->drm;
- struct intel_encoder *encoder;
-
- if (!HAS_DISPLAY(dev_priv))
- return;
-
- drm_modeset_lock_all(dev);
- for_each_intel_encoder(dev, encoder)
- if (encoder->shutdown)
- encoder->shutdown(encoder);
- drm_modeset_unlock_all(dev);
-}
-
-void i915_driver_shutdown(struct drm_i915_private *i915)
-{
- disable_rpm_wakeref_asserts(&i915->runtime_pm);
- intel_runtime_pm_disable(&i915->runtime_pm);
- intel_power_domains_disable(i915);
-
- i915_gem_suspend(i915);
-
- if (HAS_DISPLAY(i915)) {
- drm_kms_helper_poll_disable(&i915->drm);
-
- drm_atomic_helper_shutdown(&i915->drm);
- }
-
- intel_dp_mst_suspend(i915);
-
- intel_runtime_pm_disable_interrupts(i915);
- intel_hpd_cancel_work(i915);
-
- intel_suspend_encoders(i915);
- intel_shutdown_encoders(i915);
-
- intel_dmc_ucode_suspend(i915);
-
- /*
- * The only requirement is to reboot with display DC states disabled,
- * for now leaving all display power wells in the INIT power domain
- * enabled.
- *
- * TODO:
- * - unify the pci_driver::shutdown sequence here with the
- * pci_driver.driver.pm.poweroff,poweroff_late sequence.
- * - unify the driver remove and system/runtime suspend sequences with
- * the above unified shutdown/poweroff sequence.
- */
- intel_power_domains_driver_remove(i915);
- enable_rpm_wakeref_asserts(&i915->runtime_pm);
-
- intel_runtime_pm_driver_release(&i915->runtime_pm);
-}
-
-static bool suspend_to_idle(struct drm_i915_private *dev_priv)
-{
-#if IS_ENABLED(CONFIG_ACPI_SLEEP)
- if (acpi_target_system_state() < ACPI_STATE_S3)
- return true;
-#endif
- return false;
-}
-
-static int i915_drm_prepare(struct drm_device *dev)
-{
- struct drm_i915_private *i915 = to_i915(dev);
-
- /*
- * NB intel_display_suspend() may issue new requests after we've
- * ostensibly marked the GPU as ready-to-sleep here. We need to
- * split out that work and pull it forward so that after point,
- * the GPU is not woken again.
- */
- return i915_gem_backup_suspend(i915);
-}
-
-static int i915_drm_suspend(struct drm_device *dev)
-{
- struct drm_i915_private *dev_priv = to_i915(dev);
- struct pci_dev *pdev = to_pci_dev(dev_priv->drm.dev);
- pci_power_t opregion_target_state;
-
- disable_rpm_wakeref_asserts(&dev_priv->runtime_pm);
-
- /* We do a lot of poking in a lot of registers, make sure they work
- * properly. */
- intel_power_domains_disable(dev_priv);
- if (HAS_DISPLAY(dev_priv))
- drm_kms_helper_poll_disable(dev);
-
- pci_save_state(pdev);
-
- intel_display_suspend(dev);
-
- intel_dp_mst_suspend(dev_priv);
-
- intel_runtime_pm_disable_interrupts(dev_priv);
- intel_hpd_cancel_work(dev_priv);
-
- intel_suspend_encoders(dev_priv);
-
- intel_suspend_hw(dev_priv);
-
- i915_ggtt_suspend(&dev_priv->ggtt);
-
- i915_save_display(dev_priv);
-
- opregion_target_state = suspend_to_idle(dev_priv) ? PCI_D1 : PCI_D3cold;
- intel_opregion_suspend(dev_priv, opregion_target_state);
-
- intel_fbdev_set_suspend(dev, FBINFO_STATE_SUSPENDED, true);
-
- dev_priv->suspend_count++;
-
- intel_dmc_ucode_suspend(dev_priv);
-
- enable_rpm_wakeref_asserts(&dev_priv->runtime_pm);
-
- return 0;
-}
-
-static enum i915_drm_suspend_mode
-get_suspend_mode(struct drm_i915_private *dev_priv, bool hibernate)
-{
- if (hibernate)
- return I915_DRM_SUSPEND_HIBERNATE;
-
- if (suspend_to_idle(dev_priv))
- return I915_DRM_SUSPEND_IDLE;
-
- return I915_DRM_SUSPEND_MEM;
-}
-
-static int i915_drm_suspend_late(struct drm_device *dev, bool hibernation)
-{
- struct drm_i915_private *dev_priv = to_i915(dev);
- struct pci_dev *pdev = to_pci_dev(dev_priv->drm.dev);
- struct intel_runtime_pm *rpm = &dev_priv->runtime_pm;
- int ret;
-
- disable_rpm_wakeref_asserts(rpm);
-
- i915_gem_suspend_late(dev_priv);
-
- intel_uncore_suspend(&dev_priv->uncore);
-
- intel_power_domains_suspend(dev_priv,
- get_suspend_mode(dev_priv, hibernation));
-
- intel_display_power_suspend_late(dev_priv);
-
- ret = vlv_suspend_complete(dev_priv);
- if (ret) {
- drm_err(&dev_priv->drm, "Suspend complete failed: %d\n", ret);
- intel_power_domains_resume(dev_priv);
-
- goto out;
- }
-
- pci_disable_device(pdev);
- /*
- * During hibernation on some platforms the BIOS may try to access
- * the device even though it's already in D3 and hang the machine. So
- * leave the device in D0 on those platforms and hope the BIOS will
- * power down the device properly. The issue was seen on multiple old
- * GENs with different BIOS vendors, so having an explicit blacklist
- * is inpractical; apply the workaround on everything pre GEN6. The
- * platforms where the issue was seen:
- * Lenovo Thinkpad X301, X61s, X60, T60, X41
- * Fujitsu FSC S7110
- * Acer Aspire 1830T
- */
- if (!(hibernation && GRAPHICS_VER(dev_priv) < 6))
- pci_set_power_state(pdev, PCI_D3hot);
-
-out:
- enable_rpm_wakeref_asserts(rpm);
- if (!dev_priv->uncore.user_forcewake_count)
- intel_runtime_pm_driver_release(rpm);
-
- return ret;
-}
-
-int i915_suspend_switcheroo(struct drm_i915_private *i915, pm_message_t state)
-{
- int error;
-
- if (drm_WARN_ON_ONCE(&i915->drm, state.event != PM_EVENT_SUSPEND &&
- state.event != PM_EVENT_FREEZE))
- return -EINVAL;
-
- if (i915->drm.switch_power_state == DRM_SWITCH_POWER_OFF)
- return 0;
-
- error = i915_drm_suspend(&i915->drm);
- if (error)
- return error;
-
- return i915_drm_suspend_late(&i915->drm, false);
-}
-
-static int i915_drm_resume(struct drm_device *dev)
-{
- struct drm_i915_private *dev_priv = to_i915(dev);
- int ret;
-
- disable_rpm_wakeref_asserts(&dev_priv->runtime_pm);
-
- ret = intel_pcode_init(dev_priv);
- if (ret)
- return ret;
-
- sanitize_gpu(dev_priv);
-
- ret = i915_ggtt_enable_hw(dev_priv);
- if (ret)
- drm_err(&dev_priv->drm, "failed to re-enable GGTT\n");
-
- i915_ggtt_resume(&dev_priv->ggtt);
-
- intel_dmc_ucode_resume(dev_priv);
-
- i915_restore_display(dev_priv);
- intel_pps_unlock_regs_wa(dev_priv);
-
- intel_init_pch_refclk(dev_priv);
-
- /*
- * Interrupts have to be enabled before any batches are run. If not the
- * GPU will hang. i915_gem_init_hw() will initiate batches to
- * update/restore the context.
- *
- * drm_mode_config_reset() needs AUX interrupts.
- *
- * Modeset enabling in intel_modeset_init_hw() also needs working
- * interrupts.
- */
- intel_runtime_pm_enable_interrupts(dev_priv);
-
- if (HAS_DISPLAY(dev_priv))
- drm_mode_config_reset(dev);
-
- i915_gem_resume(dev_priv);
-
- intel_modeset_init_hw(dev_priv);
- intel_init_clock_gating(dev_priv);
- intel_hpd_init(dev_priv);
-
- /* MST sideband requires HPD interrupts enabled */
- intel_dp_mst_resume(dev_priv);
- intel_display_resume(dev);
-
- intel_hpd_poll_disable(dev_priv);
- if (HAS_DISPLAY(dev_priv))
- drm_kms_helper_poll_enable(dev);
-
- intel_opregion_resume(dev_priv);
-
- intel_fbdev_set_suspend(dev, FBINFO_STATE_RUNNING, false);
-
- intel_power_domains_enable(dev_priv);
-
- intel_gvt_resume(dev_priv);
-
- enable_rpm_wakeref_asserts(&dev_priv->runtime_pm);
-
- return 0;
-}
-
-static int i915_drm_resume_early(struct drm_device *dev)
-{
- struct drm_i915_private *dev_priv = to_i915(dev);
- struct pci_dev *pdev = to_pci_dev(dev_priv->drm.dev);
- int ret;
-
- /*
- * We have a resume ordering issue with the snd-hda driver also
- * requiring our device to be power up. Due to the lack of a
- * parent/child relationship we currently solve this with an early
- * resume hook.
- *
- * FIXME: This should be solved with a special hdmi sink device or
- * similar so that power domains can be employed.
- */
-
- /*
- * Note that we need to set the power state explicitly, since we
- * powered off the device during freeze and the PCI core won't power
- * it back up for us during thaw. Powering off the device during
- * freeze is not a hard requirement though, and during the
- * suspend/resume phases the PCI core makes sure we get here with the
- * device powered on. So in case we change our freeze logic and keep
- * the device powered we can also remove the following set power state
- * call.
- */
- ret = pci_set_power_state(pdev, PCI_D0);
- if (ret) {
- drm_err(&dev_priv->drm,
- "failed to set PCI D0 power state (%d)\n", ret);
- return ret;
- }
-
- /*
- * Note that pci_enable_device() first enables any parent bridge
- * device and only then sets the power state for this device. The
- * bridge enabling is a nop though, since bridge devices are resumed
- * first. The order of enabling power and enabling the device is
- * imposed by the PCI core as described above, so here we preserve the
- * same order for the freeze/thaw phases.
- *
- * TODO: eventually we should remove pci_disable_device() /
- * pci_enable_enable_device() from suspend/resume. Due to how they
- * depend on the device enable refcount we can't anyway depend on them
- * disabling/enabling the device.
- */
- if (pci_enable_device(pdev))
- return -EIO;
-
- pci_set_master(pdev);
-
- disable_rpm_wakeref_asserts(&dev_priv->runtime_pm);
-
- ret = vlv_resume_prepare(dev_priv, false);
- if (ret)
- drm_err(&dev_priv->drm,
- "Resume prepare failed: %d, continuing anyway\n", ret);
-
- intel_uncore_resume_early(&dev_priv->uncore);
-
- intel_gt_check_and_clear_faults(&dev_priv->gt);
-
- intel_display_power_resume_early(dev_priv);
-
- intel_power_domains_resume(dev_priv);
-
- enable_rpm_wakeref_asserts(&dev_priv->runtime_pm);
-
- return ret;
-}
-
-int i915_resume_switcheroo(struct drm_i915_private *i915)
-{
- int ret;
-
- if (i915->drm.switch_power_state == DRM_SWITCH_POWER_OFF)
- return 0;
-
- ret = i915_drm_resume_early(&i915->drm);
- if (ret)
- return ret;
-
- return i915_drm_resume(&i915->drm);
-}
-
-static int i915_pm_prepare(struct device *kdev)
-{
- struct drm_i915_private *i915 = kdev_to_i915(kdev);
-
- if (!i915) {
- dev_err(kdev, "DRM not initialized, aborting suspend.\n");
- return -ENODEV;
- }
-
- if (i915->drm.switch_power_state == DRM_SWITCH_POWER_OFF)
- return 0;
-
- return i915_drm_prepare(&i915->drm);
-}
-
-static int i915_pm_suspend(struct device *kdev)
-{
- struct drm_i915_private *i915 = kdev_to_i915(kdev);
-
- if (!i915) {
- dev_err(kdev, "DRM not initialized, aborting suspend.\n");
- return -ENODEV;
- }
-
- if (i915->drm.switch_power_state == DRM_SWITCH_POWER_OFF)
- return 0;
-
- return i915_drm_suspend(&i915->drm);
-}
-
-static int i915_pm_suspend_late(struct device *kdev)
-{
- struct drm_i915_private *i915 = kdev_to_i915(kdev);
-
- /*
- * We have a suspend ordering issue with the snd-hda driver also
- * requiring our device to be power up. Due to the lack of a
- * parent/child relationship we currently solve this with an late
- * suspend hook.
- *
- * FIXME: This should be solved with a special hdmi sink device or
- * similar so that power domains can be employed.
- */
- if (i915->drm.switch_power_state == DRM_SWITCH_POWER_OFF)
- return 0;
-
- return i915_drm_suspend_late(&i915->drm, false);
-}
-
-static int i915_pm_poweroff_late(struct device *kdev)
-{
- struct drm_i915_private *i915 = kdev_to_i915(kdev);
-
- if (i915->drm.switch_power_state == DRM_SWITCH_POWER_OFF)
- return 0;
-
- return i915_drm_suspend_late(&i915->drm, true);
-}
-
-static int i915_pm_resume_early(struct device *kdev)
-{
- struct drm_i915_private *i915 = kdev_to_i915(kdev);
-
- if (i915->drm.switch_power_state == DRM_SWITCH_POWER_OFF)
- return 0;
-
- return i915_drm_resume_early(&i915->drm);
-}
-
-static int i915_pm_resume(struct device *kdev)
-{
- struct drm_i915_private *i915 = kdev_to_i915(kdev);
-
- if (i915->drm.switch_power_state == DRM_SWITCH_POWER_OFF)
- return 0;
-
- return i915_drm_resume(&i915->drm);
-}
-
-/* freeze: before creating the hibernation_image */
-static int i915_pm_freeze(struct device *kdev)
-{
- struct drm_i915_private *i915 = kdev_to_i915(kdev);
- int ret;
-
- if (i915->drm.switch_power_state != DRM_SWITCH_POWER_OFF) {
- ret = i915_drm_suspend(&i915->drm);
- if (ret)
- return ret;
- }
-
- ret = i915_gem_freeze(i915);
- if (ret)
- return ret;
-
- return 0;
-}
-
-static int i915_pm_freeze_late(struct device *kdev)
-{
- struct drm_i915_private *i915 = kdev_to_i915(kdev);
- int ret;
-
- if (i915->drm.switch_power_state != DRM_SWITCH_POWER_OFF) {
- ret = i915_drm_suspend_late(&i915->drm, true);
- if (ret)
- return ret;
- }
-
- ret = i915_gem_freeze_late(i915);
- if (ret)
- return ret;
-
- return 0;
-}
-
-/* thaw: called after creating the hibernation image, but before turning off. */
-static int i915_pm_thaw_early(struct device *kdev)
-{
- return i915_pm_resume_early(kdev);
-}
-
-static int i915_pm_thaw(struct device *kdev)
-{
- return i915_pm_resume(kdev);
-}
-
-/* restore: called after loading the hibernation image. */
-static int i915_pm_restore_early(struct device *kdev)
-{
- return i915_pm_resume_early(kdev);
-}
-
-static int i915_pm_restore(struct device *kdev)
-{
- return i915_pm_resume(kdev);
-}
-
-static int intel_runtime_suspend(struct device *kdev)
-{
- struct drm_i915_private *dev_priv = kdev_to_i915(kdev);
- struct intel_runtime_pm *rpm = &dev_priv->runtime_pm;
- int ret;
-
- if (drm_WARN_ON_ONCE(&dev_priv->drm, !HAS_RUNTIME_PM(dev_priv)))
- return -ENODEV;
-
- drm_dbg_kms(&dev_priv->drm, "Suspending device\n");
-
- disable_rpm_wakeref_asserts(rpm);
-
- /*
- * We are safe here against re-faults, since the fault handler takes
- * an RPM reference.
- */
- i915_gem_runtime_suspend(dev_priv);
-
- intel_gt_runtime_suspend(&dev_priv->gt);
-
- intel_runtime_pm_disable_interrupts(dev_priv);
-
- intel_uncore_suspend(&dev_priv->uncore);
-
- intel_display_power_suspend(dev_priv);
-
- ret = vlv_suspend_complete(dev_priv);
- if (ret) {
- drm_err(&dev_priv->drm,
- "Runtime suspend failed, disabling it (%d)\n", ret);
- intel_uncore_runtime_resume(&dev_priv->uncore);
-
- intel_runtime_pm_enable_interrupts(dev_priv);
-
- intel_gt_runtime_resume(&dev_priv->gt);
-
- enable_rpm_wakeref_asserts(rpm);
-
- return ret;
- }
-
- enable_rpm_wakeref_asserts(rpm);
- intel_runtime_pm_driver_release(rpm);
-
- if (intel_uncore_arm_unclaimed_mmio_detection(&dev_priv->uncore))
- drm_err(&dev_priv->drm,
- "Unclaimed access detected prior to suspending\n");
-
- rpm->suspended = true;
-
- /*
- * FIXME: We really should find a document that references the arguments
- * used below!
- */
- if (IS_BROADWELL(dev_priv)) {
- /*
- * On Broadwell, if we use PCI_D1 the PCH DDI ports will stop
- * being detected, and the call we do at intel_runtime_resume()
- * won't be able to restore them. Since PCI_D3hot matches the
- * actual specification and appears to be working, use it.
- */
- intel_opregion_notify_adapter(dev_priv, PCI_D3hot);
- } else {
- /*
- * current versions of firmware which depend on this opregion
- * notification have repurposed the D1 definition to mean
- * "runtime suspended" vs. what you would normally expect (D3)
- * to distinguish it from notifications that might be sent via
- * the suspend path.
- */
- intel_opregion_notify_adapter(dev_priv, PCI_D1);
- }
-
- assert_forcewakes_inactive(&dev_priv->uncore);
-
- if (!IS_VALLEYVIEW(dev_priv) && !IS_CHERRYVIEW(dev_priv))
- intel_hpd_poll_enable(dev_priv);
-
- drm_dbg_kms(&dev_priv->drm, "Device suspended\n");
- return 0;
-}
-
-static int intel_runtime_resume(struct device *kdev)
-{
- struct drm_i915_private *dev_priv = kdev_to_i915(kdev);
- struct intel_runtime_pm *rpm = &dev_priv->runtime_pm;
- int ret;
-
- if (drm_WARN_ON_ONCE(&dev_priv->drm, !HAS_RUNTIME_PM(dev_priv)))
- return -ENODEV;
-
- drm_dbg_kms(&dev_priv->drm, "Resuming device\n");
-
- drm_WARN_ON_ONCE(&dev_priv->drm, atomic_read(&rpm->wakeref_count));
- disable_rpm_wakeref_asserts(rpm);
-
- intel_opregion_notify_adapter(dev_priv, PCI_D0);
- rpm->suspended = false;
- if (intel_uncore_unclaimed_mmio(&dev_priv->uncore))
- drm_dbg(&dev_priv->drm,
- "Unclaimed access during suspend, bios?\n");
-
- intel_display_power_resume(dev_priv);
-
- ret = vlv_resume_prepare(dev_priv, true);
-
- intel_uncore_runtime_resume(&dev_priv->uncore);
-
- intel_runtime_pm_enable_interrupts(dev_priv);
-
- /*
- * No point of rolling back things in case of an error, as the best
- * we can do is to hope that things will still work (and disable RPM).
- */
- intel_gt_runtime_resume(&dev_priv->gt);
-
- /*
- * On VLV/CHV display interrupts are part of the display
- * power well, so hpd is reinitialized from there. For
- * everyone else do it here.
- */
- if (!IS_VALLEYVIEW(dev_priv) && !IS_CHERRYVIEW(dev_priv)) {
- intel_hpd_init(dev_priv);
- intel_hpd_poll_disable(dev_priv);
- }
-
- intel_enable_ipc(dev_priv);
-
- enable_rpm_wakeref_asserts(rpm);
-
- if (ret)
- drm_err(&dev_priv->drm,
- "Runtime resume failed, disabling it (%d)\n", ret);
- else
- drm_dbg_kms(&dev_priv->drm, "Device resumed\n");
-
- return ret;
-}
-
-const struct dev_pm_ops i915_pm_ops = {
- /*
- * S0ix (via system suspend) and S3 event handlers [PMSG_SUSPEND,
- * PMSG_RESUME]
- */
- .prepare = i915_pm_prepare,
- .suspend = i915_pm_suspend,
- .suspend_late = i915_pm_suspend_late,
- .resume_early = i915_pm_resume_early,
- .resume = i915_pm_resume,
-
- /*
- * S4 event handlers
- * @freeze, @freeze_late : called (1) before creating the
- * hibernation image [PMSG_FREEZE] and
- * (2) after rebooting, before restoring
- * the image [PMSG_QUIESCE]
- * @thaw, @thaw_early : called (1) after creating the hibernation
- * image, before writing it [PMSG_THAW]
- * and (2) after failing to create or
- * restore the image [PMSG_RECOVER]
- * @poweroff, @poweroff_late: called after writing the hibernation
- * image, before rebooting [PMSG_HIBERNATE]
- * @restore, @restore_early : called after rebooting and restoring the
- * hibernation image [PMSG_RESTORE]
- */
- .freeze = i915_pm_freeze,
- .freeze_late = i915_pm_freeze_late,
- .thaw_early = i915_pm_thaw_early,
- .thaw = i915_pm_thaw,
- .poweroff = i915_pm_suspend,
- .poweroff_late = i915_pm_poweroff_late,
- .restore_early = i915_pm_restore_early,
- .restore = i915_pm_restore,
-
- /* S0ix (via runtime suspend) event handlers */
- .runtime_suspend = intel_runtime_suspend,
- .runtime_resume = intel_runtime_resume,
-};
-
-static const struct file_operations i915_driver_fops = {
- .owner = THIS_MODULE,
- .open = drm_open,
- .release = drm_release_noglobal,
- .unlocked_ioctl = drm_ioctl,
- .mmap = i915_gem_mmap,
- .poll = drm_poll,
- .read = drm_read,
- .compat_ioctl = i915_ioc32_compat_ioctl,
- .llseek = noop_llseek,
-};
-
-static int
-i915_gem_reject_pin_ioctl(struct drm_device *dev, void *data,
- struct drm_file *file)
-{
- return -ENODEV;
-}
-
-static const struct drm_ioctl_desc i915_ioctls[] = {
- DRM_IOCTL_DEF_DRV(I915_INIT, drm_noop, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY),
- DRM_IOCTL_DEF_DRV(I915_FLUSH, drm_noop, DRM_AUTH),
- DRM_IOCTL_DEF_DRV(I915_FLIP, drm_noop, DRM_AUTH),
- DRM_IOCTL_DEF_DRV(I915_BATCHBUFFER, drm_noop, DRM_AUTH),
- DRM_IOCTL_DEF_DRV(I915_IRQ_EMIT, drm_noop, DRM_AUTH),
- DRM_IOCTL_DEF_DRV(I915_IRQ_WAIT, drm_noop, DRM_AUTH),
- DRM_IOCTL_DEF_DRV(I915_GETPARAM, i915_getparam_ioctl, DRM_RENDER_ALLOW),
- DRM_IOCTL_DEF_DRV(I915_SETPARAM, drm_noop, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY),
- DRM_IOCTL_DEF_DRV(I915_ALLOC, drm_noop, DRM_AUTH),
- DRM_IOCTL_DEF_DRV(I915_FREE, drm_noop, DRM_AUTH),
- DRM_IOCTL_DEF_DRV(I915_INIT_HEAP, drm_noop, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY),
- DRM_IOCTL_DEF_DRV(I915_CMDBUFFER, drm_noop, DRM_AUTH),
- DRM_IOCTL_DEF_DRV(I915_DESTROY_HEAP, drm_noop, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY),
- DRM_IOCTL_DEF_DRV(I915_SET_VBLANK_PIPE, drm_noop, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY),
- DRM_IOCTL_DEF_DRV(I915_GET_VBLANK_PIPE, drm_noop, DRM_AUTH),
- DRM_IOCTL_DEF_DRV(I915_VBLANK_SWAP, drm_noop, DRM_AUTH),
- DRM_IOCTL_DEF_DRV(I915_HWS_ADDR, drm_noop, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY),
- DRM_IOCTL_DEF_DRV(I915_GEM_INIT, drm_noop, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY),
- DRM_IOCTL_DEF_DRV(I915_GEM_EXECBUFFER, drm_invalid_op, DRM_AUTH),
- DRM_IOCTL_DEF_DRV(I915_GEM_EXECBUFFER2_WR, i915_gem_execbuffer2_ioctl, DRM_RENDER_ALLOW),
- DRM_IOCTL_DEF_DRV(I915_GEM_PIN, i915_gem_reject_pin_ioctl, DRM_AUTH|DRM_ROOT_ONLY),
- DRM_IOCTL_DEF_DRV(I915_GEM_UNPIN, i915_gem_reject_pin_ioctl, DRM_AUTH|DRM_ROOT_ONLY),
- DRM_IOCTL_DEF_DRV(I915_GEM_BUSY, i915_gem_busy_ioctl, DRM_RENDER_ALLOW),
- DRM_IOCTL_DEF_DRV(I915_GEM_SET_CACHING, i915_gem_set_caching_ioctl, DRM_RENDER_ALLOW),
- DRM_IOCTL_DEF_DRV(I915_GEM_GET_CACHING, i915_gem_get_caching_ioctl, DRM_RENDER_ALLOW),
- DRM_IOCTL_DEF_DRV(I915_GEM_THROTTLE, i915_gem_throttle_ioctl, DRM_RENDER_ALLOW),
- DRM_IOCTL_DEF_DRV(I915_GEM_ENTERVT, drm_noop, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY),
- DRM_IOCTL_DEF_DRV(I915_GEM_LEAVEVT, drm_noop, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY),
- DRM_IOCTL_DEF_DRV(I915_GEM_CREATE, i915_gem_create_ioctl, DRM_RENDER_ALLOW),
- DRM_IOCTL_DEF_DRV(I915_GEM_CREATE_EXT, i915_gem_create_ext_ioctl, DRM_RENDER_ALLOW),
- DRM_IOCTL_DEF_DRV(I915_GEM_PREAD, i915_gem_pread_ioctl, DRM_RENDER_ALLOW),
- DRM_IOCTL_DEF_DRV(I915_GEM_PWRITE, i915_gem_pwrite_ioctl, DRM_RENDER_ALLOW),
- DRM_IOCTL_DEF_DRV(I915_GEM_MMAP, i915_gem_mmap_ioctl, DRM_RENDER_ALLOW),
- DRM_IOCTL_DEF_DRV(I915_GEM_MMAP_OFFSET, i915_gem_mmap_offset_ioctl, DRM_RENDER_ALLOW),
- DRM_IOCTL_DEF_DRV(I915_GEM_SET_DOMAIN, i915_gem_set_domain_ioctl, DRM_RENDER_ALLOW),
- DRM_IOCTL_DEF_DRV(I915_GEM_SW_FINISH, i915_gem_sw_finish_ioctl, DRM_RENDER_ALLOW),
- DRM_IOCTL_DEF_DRV(I915_GEM_SET_TILING, i915_gem_set_tiling_ioctl, DRM_RENDER_ALLOW),
- DRM_IOCTL_DEF_DRV(I915_GEM_GET_TILING, i915_gem_get_tiling_ioctl, DRM_RENDER_ALLOW),
- DRM_IOCTL_DEF_DRV(I915_GEM_GET_APERTURE, i915_gem_get_aperture_ioctl, DRM_RENDER_ALLOW),
- DRM_IOCTL_DEF_DRV(I915_GET_PIPE_FROM_CRTC_ID, intel_get_pipe_from_crtc_id_ioctl, 0),
- DRM_IOCTL_DEF_DRV(I915_GEM_MADVISE, i915_gem_madvise_ioctl, DRM_RENDER_ALLOW),
- DRM_IOCTL_DEF_DRV(I915_OVERLAY_PUT_IMAGE, intel_overlay_put_image_ioctl, DRM_MASTER),
- DRM_IOCTL_DEF_DRV(I915_OVERLAY_ATTRS, intel_overlay_attrs_ioctl, DRM_MASTER),
- DRM_IOCTL_DEF_DRV(I915_SET_SPRITE_COLORKEY, intel_sprite_set_colorkey_ioctl, DRM_MASTER),
- DRM_IOCTL_DEF_DRV(I915_GET_SPRITE_COLORKEY, drm_noop, DRM_MASTER),
- DRM_IOCTL_DEF_DRV(I915_GEM_WAIT, i915_gem_wait_ioctl, DRM_RENDER_ALLOW),
- DRM_IOCTL_DEF_DRV(I915_GEM_CONTEXT_CREATE_EXT, i915_gem_context_create_ioctl, DRM_RENDER_ALLOW),
- DRM_IOCTL_DEF_DRV(I915_GEM_CONTEXT_DESTROY, i915_gem_context_destroy_ioctl, DRM_RENDER_ALLOW),
- DRM_IOCTL_DEF_DRV(I915_REG_READ, i915_reg_read_ioctl, DRM_RENDER_ALLOW),
- DRM_IOCTL_DEF_DRV(I915_GET_RESET_STATS, i915_gem_context_reset_stats_ioctl, DRM_RENDER_ALLOW),
- DRM_IOCTL_DEF_DRV(I915_GEM_USERPTR, i915_gem_userptr_ioctl, DRM_RENDER_ALLOW),
- DRM_IOCTL_DEF_DRV(I915_GEM_CONTEXT_GETPARAM, i915_gem_context_getparam_ioctl, DRM_RENDER_ALLOW),
- DRM_IOCTL_DEF_DRV(I915_GEM_CONTEXT_SETPARAM, i915_gem_context_setparam_ioctl, DRM_RENDER_ALLOW),
- DRM_IOCTL_DEF_DRV(I915_PERF_OPEN, i915_perf_open_ioctl, DRM_RENDER_ALLOW),
- DRM_IOCTL_DEF_DRV(I915_PERF_ADD_CONFIG, i915_perf_add_config_ioctl, DRM_RENDER_ALLOW),
- DRM_IOCTL_DEF_DRV(I915_PERF_REMOVE_CONFIG, i915_perf_remove_config_ioctl, DRM_RENDER_ALLOW),
- DRM_IOCTL_DEF_DRV(I915_QUERY, i915_query_ioctl, DRM_RENDER_ALLOW),
- DRM_IOCTL_DEF_DRV(I915_GEM_VM_CREATE, i915_gem_vm_create_ioctl, DRM_RENDER_ALLOW),
- DRM_IOCTL_DEF_DRV(I915_GEM_VM_DESTROY, i915_gem_vm_destroy_ioctl, DRM_RENDER_ALLOW),
-};
-
-static const struct drm_driver driver = {
- /* Don't use MTRRs here; the Xserver or userspace app should
- * deal with them for Intel hardware.
- */
- .driver_features =
- DRIVER_GEM |
- DRIVER_RENDER | DRIVER_MODESET | DRIVER_ATOMIC | DRIVER_SYNCOBJ |
- DRIVER_SYNCOBJ_TIMELINE,
- .release = i915_driver_release,
- .open = i915_driver_open,
- .lastclose = i915_driver_lastclose,
- .postclose = i915_driver_postclose,
-
- .prime_handle_to_fd = drm_gem_prime_handle_to_fd,
- .prime_fd_to_handle = drm_gem_prime_fd_to_handle,
- .gem_prime_import = i915_gem_prime_import,
-
- .dumb_create = i915_gem_dumb_create,
- .dumb_map_offset = i915_gem_dumb_mmap_offset,
-
- .ioctls = i915_ioctls,
- .num_ioctls = ARRAY_SIZE(i915_ioctls),
- .fops = &i915_driver_fops,
- .name = DRIVER_NAME,
- .desc = DRIVER_DESC,
- .date = DRIVER_DATE,
- .major = DRIVER_MAJOR,
- .minor = DRIVER_MINOR,
- .patchlevel = DRIVER_PATCHLEVEL,
-};
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index 1225621..0c70ab0 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -50,7 +50,6 @@
#include <linux/stackdepot.h>
#include <linux/xarray.h>
-#include <drm/intel-gtt.h>
#include <drm/drm_gem.h>
#include <drm/drm_auth.h>
#include <drm/drm_cache.h>
@@ -90,6 +89,7 @@
#include "intel_device_info.h"
#include "intel_memory_region.h"
#include "intel_pch.h"
+#include "intel_pm_types.h"
#include "intel_runtime_pm.h"
#include "intel_step.h"
#include "intel_uncore.h"
@@ -117,30 +117,6 @@
struct drm_i915_gem_object;
-enum hpd_pin {
- HPD_NONE = 0,
- HPD_TV = HPD_NONE, /* TV is known to be unreliable */
- HPD_CRT,
- HPD_SDVO_B,
- HPD_SDVO_C,
- HPD_PORT_A,
- HPD_PORT_B,
- HPD_PORT_C,
- HPD_PORT_D,
- HPD_PORT_E,
- HPD_PORT_TC1,
- HPD_PORT_TC2,
- HPD_PORT_TC3,
- HPD_PORT_TC4,
- HPD_PORT_TC5,
- HPD_PORT_TC6,
-
- HPD_NUM_PINS
-};
-
-#define for_each_hpd_pin(__pin) \
- for ((__pin) = (HPD_NONE + 1); (__pin) < HPD_NUM_PINS; (__pin)++)
-
/* Threshold == 5 for long IRQs, 50 for short */
#define HPD_STORM_DEFAULT_THRESHOLD 50
@@ -191,8 +167,6 @@ struct i915_hotplug {
I915_GEM_DOMAIN_VERTEX)
struct drm_i915_private;
-struct i915_mm_struct;
-struct i915_mmu_object;
struct drm_i915_file_private {
struct drm_i915_private *dev_priv;
@@ -364,15 +338,6 @@ struct intel_color_funcs {
void (*read_luts)(struct intel_crtc_state *crtc_state);
};
-struct intel_audio_funcs {
- void (*audio_codec_enable)(struct intel_encoder *encoder,
- const struct intel_crtc_state *crtc_state,
- const struct drm_connector_state *conn_state);
- void (*audio_codec_disable)(struct intel_encoder *encoder,
- const struct intel_crtc_state *old_crtc_state,
- const struct drm_connector_state *old_conn_state);
-};
-
struct intel_cdclk_funcs {
void (*get_cdclk)(struct drm_i915_private *dev_priv,
struct intel_cdclk_config *cdclk_config);
@@ -411,102 +376,8 @@ struct drm_i915_display_funcs {
void (*commit_modeset_enables)(struct intel_atomic_state *state);
};
-
#define I915_COLOR_UNEVICTABLE (-1) /* a non-vma sharing the address space */
-struct intel_fbc {
- /* This is always the inner lock when overlapping with struct_mutex and
- * it's the outer lock when overlapping with stolen_lock. */
- struct mutex lock;
- unsigned int possible_framebuffer_bits;
- unsigned int busy_bits;
- struct intel_crtc *crtc;
-
- struct drm_mm_node compressed_fb;
- struct drm_mm_node compressed_llb;
-
- u8 limit;
-
- bool false_color;
-
- bool active;
- bool activated;
- bool flip_pending;
-
- bool underrun_detected;
- struct work_struct underrun_work;
-
- /*
- * Due to the atomic rules we can't access some structures without the
- * appropriate locking, so we cache information here in order to avoid
- * these problems.
- */
- struct intel_fbc_state_cache {
- struct {
- unsigned int mode_flags;
- u32 hsw_bdw_pixel_rate;
- } crtc;
-
- struct {
- unsigned int rotation;
- int src_w;
- int src_h;
- bool visible;
- /*
- * Display surface base address adjustement for
- * pageflips. Note that on gen4+ this only adjusts up
- * to a tile, offsets within a tile are handled in
- * the hw itself (with the TILEOFF register).
- */
- int adjusted_x;
- int adjusted_y;
-
- u16 pixel_blend_mode;
- } plane;
-
- struct {
- const struct drm_format_info *format;
- unsigned int stride;
- u64 modifier;
- } fb;
-
- unsigned int fence_y_offset;
- u16 interval;
- s8 fence_id;
- bool psr2_active;
- } state_cache;
-
- /*
- * This structure contains everything that's relevant to program the
- * hardware registers. When we want to figure out if we need to disable
- * and re-enable FBC for a new configuration we just check if there's
- * something different in the struct. The genx_fbc_activate functions
- * are supposed to read from it in order to program the registers.
- */
- struct intel_fbc_reg_params {
- struct {
- enum pipe pipe;
- enum i9xx_plane_id i9xx_plane;
- } crtc;
-
- struct {
- const struct drm_format_info *format;
- unsigned int stride;
- u64 modifier;
- } fb;
-
- unsigned int cfb_stride;
- unsigned int cfb_size;
- unsigned int fence_y_offset;
- u16 override_cfb_stride;
- u16 interval;
- s8 fence_id;
- bool plane_visible;
- } params;
-
- const char *no_fbc_reason;
-};
-
/*
* HIGH_RR is the highest eDP panel refresh rate read from EDID
* LOW_RR is the lowest eDP panel refresh rate found from EDID
@@ -543,7 +414,6 @@ struct i915_drrs {
#define QUIRK_NO_PPS_BACKLIGHT_POWER_HOOK (1<<8)
struct intel_fbdev;
-struct intel_fbc_work;
struct intel_gmbus {
struct i2c_adapter adapter;
@@ -603,7 +473,7 @@ struct i915_gem_mm {
* List of objects which are pending destruction.
*/
struct llist_head free_list;
- struct work_struct free_work;
+ struct delayed_work free_work;
/**
* Count of objects pending destructions. Used to skip needlessly
* waiting on an RCU barrier if no objects are waiting to be freed.
@@ -738,69 +608,6 @@ struct intel_vbt_data {
struct sdvo_device_mapping sdvo_mappings[2];
};
-enum intel_ddb_partitioning {
- INTEL_DDB_PART_1_2,
- INTEL_DDB_PART_5_6, /* IVB+ */
-};
-
-struct ilk_wm_values {
- u32 wm_pipe[3];
- u32 wm_lp[3];
- u32 wm_lp_spr[3];
- bool enable_fbc_wm;
- enum intel_ddb_partitioning partitioning;
-};
-
-struct g4x_pipe_wm {
- u16 plane[I915_MAX_PLANES];
- u16 fbc;
-};
-
-struct g4x_sr_wm {
- u16 plane;
- u16 cursor;
- u16 fbc;
-};
-
-struct vlv_wm_ddl_values {
- u8 plane[I915_MAX_PLANES];
-};
-
-struct vlv_wm_values {
- struct g4x_pipe_wm pipe[3];
- struct g4x_sr_wm sr;
- struct vlv_wm_ddl_values ddl[3];
- u8 level;
- bool cxsr;
-};
-
-struct g4x_wm_values {
- struct g4x_pipe_wm pipe[2];
- struct g4x_sr_wm sr;
- struct g4x_sr_wm hpll;
- bool cxsr;
- bool hpll_en;
- bool fbc_en;
-};
-
-struct skl_ddb_entry {
- u16 start, end; /* in number of blocks, 'end' is exclusive */
-};
-
-static inline u16 skl_ddb_entry_size(const struct skl_ddb_entry *entry)
-{
- return entry->end - entry->start;
-}
-
-static inline bool skl_ddb_entry_equal(const struct skl_ddb_entry *e1,
- const struct skl_ddb_entry *e2)
-{
- if (e1->start == e2->start && e1->end == e2->end)
- return true;
-
- return false;
-}
-
struct i915_frontbuffer_tracking {
spinlock_t lock;
@@ -828,6 +635,30 @@ struct i915_selftest_stash {
struct ida mock_region_instances;
};
+/* intel_audio.c private */
+struct intel_audio_funcs;
+struct intel_audio_private {
+ /* Display internal audio functions */
+ const struct intel_audio_funcs *funcs;
+
+ /* hda/i915 audio component */
+ struct i915_audio_component *component;
+ bool component_registered;
+ /* mutex for audio/video sync */
+ struct mutex mutex;
+ int power_refcount;
+ u32 freq_cntrl;
+
+ /* Used to save the pipe-to-encoder mapping for audio */
+ struct intel_encoder *encoder_map[I915_MAX_PIPES];
+
+ /* necessary resource sharing with HDMI LPE audio driver. */
+ struct {
+ struct platform_device *platdev;
+ int irq;
+ } lpe;
+};
+
struct drm_i915_private {
struct drm_device drm;
@@ -918,7 +749,7 @@ struct drm_i915_private {
u32 pipestat_irq_mask[I915_MAX_PIPES];
struct i915_hotplug hotplug;
- struct intel_fbc fbc;
+ struct intel_fbc *fbc;
struct i915_drrs drrs;
struct intel_opregion opregion;
struct intel_vbt_data vbt;
@@ -995,9 +826,6 @@ struct drm_i915_private {
/* Display internal color functions */
const struct intel_color_funcs *color_funcs;
- /* Display internal audio functions */
- const struct intel_audio_funcs *audio_funcs;
-
/* Display CDCLK functions */
const struct intel_cdclk_funcs *cdclk_funcs;
@@ -1016,9 +844,6 @@ struct drm_i915_private {
/* Kernel Modesetting */
- struct intel_crtc *plane_to_crtc_mapping[I915_MAX_PIPES];
- struct intel_crtc *pipe_to_crtc_mapping[I915_MAX_PIPES];
-
/**
* dpll and cdclk state is protected by connection_mutex
* dpll.lock serializes intel_{prepare,enable,disable}_shared_dpll.
@@ -1084,17 +909,6 @@ struct drm_i915_private {
struct drm_property *broadcast_rgb_property;
struct drm_property *force_audio_property;
- /* hda/i915 audio component */
- struct i915_audio_component *audio_component;
- bool audio_component_registered;
- /**
- * av_mutex - mutex for audio/video sync
- *
- */
- struct mutex av_mutex;
- int audio_power_refcount;
- u32 audio_freq_cntrl;
-
u32 fdi_rx_config;
/* Shadow for DISPLAY_PHY_CONTROL which can't be safely read */
@@ -1191,7 +1005,7 @@ struct drm_i915_private {
struct i915_perf perf;
/* Abstract the submission mechanism (legacy ringbuffer or execlists) away */
- struct intel_gt gt;
+ struct intel_gt gt0;
struct {
struct i915_gem_contexts {
@@ -1227,14 +1041,7 @@ struct drm_i915_private {
bool ipc_enabled;
- /* Used to save the pipe-to-encoder mapping for audio */
- struct intel_encoder *av_enc_map[I915_MAX_PIPES];
-
- /* necessary resource sharing with HDMI LPE audio driver. */
- struct {
- struct platform_device *platdev;
- int irq;
- } lpe_audio;
+ struct intel_audio_private audio;
struct i915_pmu pmu;
@@ -1270,6 +1077,11 @@ static inline struct drm_i915_private *pdev_to_i915(struct pci_dev *pdev)
return pci_get_drvdata(pdev);
}
+static inline struct intel_gt *to_gt(struct drm_i915_private *i915)
+{
+ return &i915->gt0;
+}
+
/* Simple iterator over all initialised engines */
#define for_each_engine(engine__, dev_priv__, id__) \
for ((id__) = 0; \
@@ -1327,15 +1139,15 @@ static inline struct drm_i915_private *pdev_to_i915(struct pci_dev *pdev)
#define IP_VER(ver, rel) ((ver) << 8 | (rel))
-#define GRAPHICS_VER(i915) (INTEL_INFO(i915)->graphics_ver)
-#define GRAPHICS_VER_FULL(i915) IP_VER(INTEL_INFO(i915)->graphics_ver, \
- INTEL_INFO(i915)->graphics_rel)
+#define GRAPHICS_VER(i915) (INTEL_INFO(i915)->graphics.ver)
+#define GRAPHICS_VER_FULL(i915) IP_VER(INTEL_INFO(i915)->graphics.ver, \
+ INTEL_INFO(i915)->graphics.rel)
#define IS_GRAPHICS_VER(i915, from, until) \
(GRAPHICS_VER(i915) >= (from) && GRAPHICS_VER(i915) <= (until))
-#define MEDIA_VER(i915) (INTEL_INFO(i915)->media_ver)
-#define MEDIA_VER_FULL(i915) IP_VER(INTEL_INFO(i915)->media_ver, \
- INTEL_INFO(i915)->media_rel)
+#define MEDIA_VER(i915) (INTEL_INFO(i915)->media.ver)
+#define MEDIA_VER_FULL(i915) IP_VER(INTEL_INFO(i915)->media.arch, \
+ INTEL_INFO(i915)->media.rel)
#define IS_MEDIA_VER(i915, from, until) \
(MEDIA_VER(i915) >= (from) && MEDIA_VER(i915) <= (until))
@@ -1348,15 +1160,20 @@ static inline struct drm_i915_private *pdev_to_i915(struct pci_dev *pdev)
#define HAS_DSB(dev_priv) (INTEL_INFO(dev_priv)->display.has_dsb)
#define INTEL_DISPLAY_STEP(__i915) (RUNTIME_INFO(__i915)->step.display_step)
-#define INTEL_GT_STEP(__i915) (RUNTIME_INFO(__i915)->step.gt_step)
+#define INTEL_GRAPHICS_STEP(__i915) (RUNTIME_INFO(__i915)->step.graphics_step)
+#define INTEL_MEDIA_STEP(__i915) (RUNTIME_INFO(__i915)->step.media_step)
#define IS_DISPLAY_STEP(__i915, since, until) \
(drm_WARN_ON(&(__i915)->drm, INTEL_DISPLAY_STEP(__i915) == STEP_NONE), \
INTEL_DISPLAY_STEP(__i915) >= (since) && INTEL_DISPLAY_STEP(__i915) < (until))
-#define IS_GT_STEP(__i915, since, until) \
- (drm_WARN_ON(&(__i915)->drm, INTEL_GT_STEP(__i915) == STEP_NONE), \
- INTEL_GT_STEP(__i915) >= (since) && INTEL_GT_STEP(__i915) < (until))
+#define IS_GRAPHICS_STEP(__i915, since, until) \
+ (drm_WARN_ON(&(__i915)->drm, INTEL_GRAPHICS_STEP(__i915) == STEP_NONE), \
+ INTEL_GRAPHICS_STEP(__i915) >= (since) && INTEL_GRAPHICS_STEP(__i915) < (until))
+
+#define IS_MEDIA_STEP(__i915, since, until) \
+ (drm_WARN_ON(&(__i915)->drm, INTEL_MEDIA_STEP(__i915) == STEP_NONE), \
+ INTEL_MEDIA_STEP(__i915) >= (since) && INTEL_MEDIA_STEP(__i915) < (until))
static __always_inline unsigned int
__platform_mask_index(const struct intel_runtime_info *info,
@@ -1455,7 +1272,6 @@ IS_SUBPLATFORM(const struct drm_i915_private *i915,
#define IS_GEMINILAKE(dev_priv) IS_PLATFORM(dev_priv, INTEL_GEMINILAKE)
#define IS_COFFEELAKE(dev_priv) IS_PLATFORM(dev_priv, INTEL_COFFEELAKE)
#define IS_COMETLAKE(dev_priv) IS_PLATFORM(dev_priv, INTEL_COMETLAKE)
-#define IS_CANNONLAKE(dev_priv) 0
#define IS_ICELAKE(dev_priv) IS_PLATFORM(dev_priv, INTEL_ICELAKE)
#define IS_JSL_EHL(dev_priv) (IS_PLATFORM(dev_priv, INTEL_JASPERLAKE) || \
IS_PLATFORM(dev_priv, INTEL_ELKHARTLAKE))
@@ -1470,6 +1286,8 @@ IS_SUBPLATFORM(const struct drm_i915_private *i915,
IS_SUBPLATFORM(dev_priv, INTEL_DG2, INTEL_SUBPLATFORM_G10)
#define IS_DG2_G11(dev_priv) \
IS_SUBPLATFORM(dev_priv, INTEL_DG2, INTEL_SUBPLATFORM_G11)
+#define IS_ADLS_RPLS(dev_priv) \
+ IS_SUBPLATFORM(dev_priv, INTEL_ALDERLAKE_S, INTEL_SUBPLATFORM_RPL_S)
#define IS_HSW_EARLY_SDV(dev_priv) (IS_HASWELL(dev_priv) && \
(INTEL_DEVID(dev_priv) & 0xFF00) == 0x0C00)
#define IS_BDW_ULT(dev_priv) \
@@ -1530,15 +1348,15 @@ IS_SUBPLATFORM(const struct drm_i915_private *i915,
#define IS_TGL_Y(dev_priv) \
IS_SUBPLATFORM(dev_priv, INTEL_TIGERLAKE, INTEL_SUBPLATFORM_ULX)
-#define IS_SKL_GT_STEP(p, since, until) (IS_SKYLAKE(p) && IS_GT_STEP(p, since, until))
+#define IS_SKL_GRAPHICS_STEP(p, since, until) (IS_SKYLAKE(p) && IS_GRAPHICS_STEP(p, since, until))
-#define IS_KBL_GT_STEP(dev_priv, since, until) \
- (IS_KABYLAKE(dev_priv) && IS_GT_STEP(dev_priv, since, until))
+#define IS_KBL_GRAPHICS_STEP(dev_priv, since, until) \
+ (IS_KABYLAKE(dev_priv) && IS_GRAPHICS_STEP(dev_priv, since, until))
#define IS_KBL_DISPLAY_STEP(dev_priv, since, until) \
(IS_KABYLAKE(dev_priv) && IS_DISPLAY_STEP(dev_priv, since, until))
-#define IS_JSL_EHL_GT_STEP(p, since, until) \
- (IS_JSL_EHL(p) && IS_GT_STEP(p, since, until))
+#define IS_JSL_EHL_GRAPHICS_STEP(p, since, until) \
+ (IS_JSL_EHL(p) && IS_GRAPHICS_STEP(p, since, until))
#define IS_JSL_EHL_DISPLAY_STEP(p, since, until) \
(IS_JSL_EHL(p) && IS_DISPLAY_STEP(p, since, until))
@@ -1546,19 +1364,19 @@ IS_SUBPLATFORM(const struct drm_i915_private *i915,
(IS_TIGERLAKE(__i915) && \
IS_DISPLAY_STEP(__i915, since, until))
-#define IS_TGL_UY_GT_STEP(__i915, since, until) \
+#define IS_TGL_UY_GRAPHICS_STEP(__i915, since, until) \
((IS_TGL_U(__i915) || IS_TGL_Y(__i915)) && \
- IS_GT_STEP(__i915, since, until))
+ IS_GRAPHICS_STEP(__i915, since, until))
-#define IS_TGL_GT_STEP(__i915, since, until) \
+#define IS_TGL_GRAPHICS_STEP(__i915, since, until) \
(IS_TIGERLAKE(__i915) && !(IS_TGL_U(__i915) || IS_TGL_Y(__i915)) && \
- IS_GT_STEP(__i915, since, until))
+ IS_GRAPHICS_STEP(__i915, since, until))
#define IS_RKL_DISPLAY_STEP(p, since, until) \
(IS_ROCKETLAKE(p) && IS_DISPLAY_STEP(p, since, until))
-#define IS_DG1_GT_STEP(p, since, until) \
- (IS_DG1(p) && IS_GT_STEP(p, since, until))
+#define IS_DG1_GRAPHICS_STEP(p, since, until) \
+ (IS_DG1(p) && IS_GRAPHICS_STEP(p, since, until))
#define IS_DG1_DISPLAY_STEP(p, since, until) \
(IS_DG1(p) && IS_DISPLAY_STEP(p, since, until))
@@ -1566,20 +1384,20 @@ IS_SUBPLATFORM(const struct drm_i915_private *i915,
(IS_ALDERLAKE_S(__i915) && \
IS_DISPLAY_STEP(__i915, since, until))
-#define IS_ADLS_GT_STEP(__i915, since, until) \
+#define IS_ADLS_GRAPHICS_STEP(__i915, since, until) \
(IS_ALDERLAKE_S(__i915) && \
- IS_GT_STEP(__i915, since, until))
+ IS_GRAPHICS_STEP(__i915, since, until))
#define IS_ADLP_DISPLAY_STEP(__i915, since, until) \
(IS_ALDERLAKE_P(__i915) && \
IS_DISPLAY_STEP(__i915, since, until))
-#define IS_ADLP_GT_STEP(__i915, since, until) \
+#define IS_ADLP_GRAPHICS_STEP(__i915, since, until) \
(IS_ALDERLAKE_P(__i915) && \
- IS_GT_STEP(__i915, since, until))
+ IS_GRAPHICS_STEP(__i915, since, until))
-#define IS_XEHPSDV_GT_STEP(__i915, since, until) \
- (IS_XEHPSDV(__i915) && IS_GT_STEP(__i915, since, until))
+#define IS_XEHPSDV_GRAPHICS_STEP(__i915, since, until) \
+ (IS_XEHPSDV(__i915) && IS_GRAPHICS_STEP(__i915, since, until))
/*
* DG2 hardware steppings are a bit unusual. The hardware design was forked
@@ -1595,11 +1413,11 @@ IS_SUBPLATFORM(const struct drm_i915_private *i915,
* and stepping-specific logic will be applied with a general DG2-wide stepping
* number.
*/
-#define IS_DG2_GT_STEP(__i915, variant, since, until) \
+#define IS_DG2_GRAPHICS_STEP(__i915, variant, since, until) \
(IS_SUBPLATFORM(__i915, INTEL_DG2, INTEL_SUBPLATFORM_##variant) && \
- IS_GT_STEP(__i915, since, until))
+ IS_GRAPHICS_STEP(__i915, since, until))
-#define IS_DG2_DISP_STEP(__i915, since, until) \
+#define IS_DG2_DISPLAY_STEP(__i915, since, until) \
(IS_DG2(__i915) && \
IS_DISPLAY_STEP(__i915, since, until))
@@ -1696,7 +1514,7 @@ IS_SUBPLATFORM(const struct drm_i915_private *i915,
#define HAS_PSR_HW_TRACKING(dev_priv) \
(INTEL_INFO(dev_priv)->display.has_psr_hw_tracking)
#define HAS_PSR2_SEL_FETCH(dev_priv) (GRAPHICS_VER(dev_priv) >= 12)
-#define HAS_TRANSCODER(dev_priv, trans) ((INTEL_INFO(dev_priv)->cpu_transcoder_mask & BIT(trans)) != 0)
+#define HAS_TRANSCODER(dev_priv, trans) ((INTEL_INFO(dev_priv)->display.cpu_transcoder_mask & BIT(trans)) != 0)
#define HAS_RC6(dev_priv) (INTEL_INFO(dev_priv)->has_rc6)
#define HAS_RC6p(dev_priv) (INTEL_INFO(dev_priv)->has_rc6p)
@@ -1714,6 +1532,14 @@ IS_SUBPLATFORM(const struct drm_i915_private *i915,
#define HAS_MSLICES(dev_priv) \
(INTEL_INFO(dev_priv)->has_mslices)
+/*
+ * Set this flag, when platform requires 64K GTT page sizes or larger for
+ * device local memory access. Also this flag implies that we require or
+ * at least support the compact PT layout for the ppGTT when using the 64K
+ * GTT pages.
+ */
+#define HAS_64K_PAGES(dev_priv) (INTEL_INFO(dev_priv)->has_64k_pages)
+
#define HAS_IPC(dev_priv) (INTEL_INFO(dev_priv)->display.has_ipc)
#define HAS_REGION(i915, i) (INTEL_INFO(i915)->memory_regions & (i))
@@ -1727,7 +1553,7 @@ IS_SUBPLATFORM(const struct drm_i915_private *i915,
#define HAS_PXP(dev_priv) ((IS_ENABLED(CONFIG_DRM_I915_PXP) && \
INTEL_INFO(dev_priv)->has_pxp) && \
- VDBOX_MASK(&dev_priv->gt))
+ VDBOX_MASK(to_gt(dev_priv)))
#define HAS_GMCH(dev_priv) (INTEL_INFO(dev_priv)->display.has_gmch)
@@ -1741,11 +1567,11 @@ IS_SUBPLATFORM(const struct drm_i915_private *i915,
#define GT_FREQUENCY_MULTIPLIER 50
#define GEN9_FREQ_SCALER 3
-#define INTEL_NUM_PIPES(dev_priv) (hweight8(INTEL_INFO(dev_priv)->pipe_mask))
+#define INTEL_NUM_PIPES(dev_priv) (hweight8(INTEL_INFO(dev_priv)->display.pipe_mask))
-#define HAS_DISPLAY(dev_priv) (INTEL_INFO(dev_priv)->pipe_mask != 0)
+#define HAS_DISPLAY(dev_priv) (INTEL_INFO(dev_priv)->display.pipe_mask != 0)
-#define HAS_VRR(i915) (GRAPHICS_VER(i915) >= 12)
+#define HAS_VRR(i915) (GRAPHICS_VER(i915) >= 11)
#define HAS_ASYNC_FLIPS(i915) (DISPLAY_VER(i915) >= 5)
@@ -1761,26 +1587,27 @@ static inline bool run_as_guest(void)
#define HAS_D12_PLANE_MINIMIZATION(dev_priv) (IS_ROCKETLAKE(dev_priv) || \
IS_ALDERLAKE_S(dev_priv))
-static inline bool intel_vtd_active(void)
+static inline bool intel_vtd_active(struct drm_i915_private *i915)
{
-#ifdef CONFIG_INTEL_IOMMU
- if (intel_iommu_gfx_mapped)
+ if (device_iommu_mapped(i915->drm.dev))
return true;
-#endif
/* Running as a guest, we assume the host is enforcing VT'd */
return run_as_guest();
}
+void
+i915_print_iommu_status(struct drm_i915_private *i915, struct drm_printer *p);
+
static inline bool intel_scanout_needs_vtd_wa(struct drm_i915_private *dev_priv)
{
- return GRAPHICS_VER(dev_priv) >= 6 && intel_vtd_active();
+ return GRAPHICS_VER(dev_priv) >= 6 && intel_vtd_active(dev_priv);
}
static inline bool
intel_ggtt_update_needs_vtd_wa(struct drm_i915_private *i915)
{
- return IS_BROXTON(i915) && intel_vtd_active();
+ return IS_BROXTON(i915) && intel_vtd_active(i915);
}
static inline bool
@@ -1789,16 +1616,7 @@ intel_vm_no_concurrent_access_wa(struct drm_i915_private *i915)
return IS_CHERRYVIEW(i915) || intel_ggtt_update_needs_vtd_wa(i915);
}
-/* i915_drv.c */
-extern const struct dev_pm_ops i915_pm_ops;
-
-int i915_driver_probe(struct pci_dev *pdev, const struct pci_device_id *ent);
-void i915_driver_remove(struct drm_i915_private *i915);
-void i915_driver_shutdown(struct drm_i915_private *i915);
-
-int i915_resume_switcheroo(struct drm_i915_private *i915);
-int i915_suspend_switcheroo(struct drm_i915_private *i915, pm_message_t state);
-
+/* i915_getparam.c */
int i915_getparam_ioctl(struct drm_device *dev, void *data,
struct drm_file *file_priv);
@@ -1818,7 +1636,8 @@ static inline void i915_gem_drain_freed_objects(struct drm_i915_private *i915)
* armed the work again.
*/
while (atomic_read(&i915->mm.free_count)) {
- flush_work(&i915->mm.free_work);
+ flush_delayed_work(&i915->mm.free_work);
+ flush_delayed_work(&i915->bdev.wq);
rcu_barrier();
}
}
@@ -1851,13 +1670,10 @@ i915_gem_object_ggtt_pin_ww(struct drm_i915_gem_object *obj,
const struct i915_ggtt_view *view,
u64 size, u64 alignment, u64 flags);
-static inline struct i915_vma * __must_check
+struct i915_vma * __must_check
i915_gem_object_ggtt_pin(struct drm_i915_gem_object *obj,
const struct i915_ggtt_view *view,
- u64 size, u64 alignment, u64 flags)
-{
- return i915_gem_object_ggtt_pin_ww(obj, NULL, view, size, alignment, flags);
-}
+ u64 size, u64 alignment, u64 flags);
int i915_gem_object_unbind(struct drm_i915_gem_object *obj,
unsigned long flags);
@@ -1933,6 +1749,10 @@ int i915_gem_evict_vm(struct i915_address_space *vm);
struct drm_i915_gem_object *
i915_gem_object_create_internal(struct drm_i915_private *dev_priv,
phys_addr_t size);
+struct drm_i915_gem_object *
+__i915_gem_object_create_internal(struct drm_i915_private *dev_priv,
+ const struct drm_i915_gem_object_ops *ops,
+ phys_addr_t size);
/* i915_gem_tiling.c */
static inline bool i915_gem_object_needs_bit17_swizzle(struct drm_i915_gem_object *obj)
@@ -1972,14 +1792,6 @@ mkwrite_device_info(struct drm_i915_private *dev_priv)
int i915_reg_read_ioctl(struct drm_device *dev, void *data,
struct drm_file *file);
-/* i915_mm.c */
-int remap_io_mapping(struct vm_area_struct *vma,
- unsigned long addr, unsigned long pfn, unsigned long size,
- struct io_mapping *iomap);
-int remap_io_sg(struct vm_area_struct *vma,
- unsigned long addr, unsigned long size,
- struct scatterlist *sgl, resource_size_t iobase);
-
static inline int intel_hws_csb_write_index(struct drm_i915_private *i915)
{
if (GRAPHICS_VER(i915) >= 11)
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 981e383..915bf43 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -764,7 +764,7 @@ i915_gem_pwrite_ioctl(struct drm_device *dev, void *data,
* perspective, requiring manual detiling by the client.
*/
if (!i915_gem_object_has_struct_page(obj) ||
- cpu_write_needs_clflush(obj))
+ i915_gem_cpu_write_needs_clflush(obj))
/* Note that the gtt paths might fail with non-page-backed user
* pointers (e.g. gtt mappings when moving data between
* textures). Fallback to the shmem path in that case.
@@ -877,6 +877,8 @@ i915_gem_object_ggtt_pin_ww(struct drm_i915_gem_object *obj,
struct i915_vma *vma;
int ret;
+ GEM_WARN_ON(!ww);
+
if (flags & PIN_MAPPABLE &&
(!view || view->type == I915_GGTT_VIEW_NORMAL)) {
/*
@@ -936,10 +938,7 @@ i915_gem_object_ggtt_pin_ww(struct drm_i915_gem_object *obj,
return ERR_PTR(ret);
}
- if (ww)
- ret = i915_vma_pin_ww(vma, ww, size, alignment, flags | PIN_GLOBAL);
- else
- ret = i915_vma_pin(vma, size, alignment, flags | PIN_GLOBAL);
+ ret = i915_vma_pin_ww(vma, ww, size, alignment, flags | PIN_GLOBAL);
if (ret)
return ERR_PTR(ret);
@@ -959,6 +958,29 @@ i915_gem_object_ggtt_pin_ww(struct drm_i915_gem_object *obj,
return vma;
}
+struct i915_vma * __must_check
+i915_gem_object_ggtt_pin(struct drm_i915_gem_object *obj,
+ const struct i915_ggtt_view *view,
+ u64 size, u64 alignment, u64 flags)
+{
+ struct i915_gem_ww_ctx ww;
+ struct i915_vma *ret;
+ int err;
+
+ for_i915_gem_ww(&ww, err, true) {
+ err = i915_gem_object_lock(obj, &ww);
+ if (err)
+ continue;
+
+ ret = i915_gem_object_ggtt_pin_ww(obj, &ww, view, size,
+ alignment, flags);
+ if (IS_ERR(ret))
+ err = PTR_ERR(ret);
+ }
+
+ return err ? ERR_PTR(err) : ret;
+}
+
int
i915_gem_madvise_ioctl(struct drm_device *dev, void *data,
struct drm_file *file_priv)
@@ -1005,7 +1027,8 @@ i915_gem_madvise_ioctl(struct drm_device *dev, void *data,
obj->ops->adjust_lru(obj);
}
- if (i915_gem_object_has_pages(obj)) {
+ if (i915_gem_object_has_pages(obj) ||
+ i915_gem_object_has_self_managed_shrink_list(obj)) {
unsigned long flags;
spin_lock_irqsave(&i915->mm.obj_lock, flags);
@@ -1048,7 +1071,7 @@ int i915_gem_init(struct drm_i915_private *dev_priv)
if (ret)
return ret;
- intel_uc_fetch_firmwares(&dev_priv->gt.uc);
+ intel_uc_fetch_firmwares(&to_gt(dev_priv)->uc);
intel_wopcm_init(&dev_priv->wopcm);
ret = i915_init_ggtt(dev_priv);
@@ -1068,7 +1091,7 @@ int i915_gem_init(struct drm_i915_private *dev_priv)
*/
intel_init_clock_gating(dev_priv);
- ret = intel_gt_init(&dev_priv->gt);
+ ret = intel_gt_init(to_gt(dev_priv));
if (ret)
goto err_unlock;
@@ -1084,7 +1107,7 @@ int i915_gem_init(struct drm_i915_private *dev_priv)
i915_gem_drain_workqueue(dev_priv);
if (ret != -EIO)
- intel_uc_cleanup_firmwares(&dev_priv->gt.uc);
+ intel_uc_cleanup_firmwares(&to_gt(dev_priv)->uc);
if (ret == -EIO) {
/*
@@ -1092,10 +1115,10 @@ int i915_gem_init(struct drm_i915_private *dev_priv)
* as wedged. But we only want to do this when the GPU is angry,
* for all other failure, such as an allocation failure, bail.
*/
- if (!intel_gt_is_wedged(&dev_priv->gt)) {
+ if (!intel_gt_is_wedged(to_gt(dev_priv))) {
i915_probe_error(dev_priv,
"Failed to initialize GPU, declaring it wedged!\n");
- intel_gt_set_wedged(&dev_priv->gt);
+ intel_gt_set_wedged(to_gt(dev_priv));
}
/* Minimal basic recovery for KMS */
@@ -1126,7 +1149,7 @@ void i915_gem_driver_remove(struct drm_i915_private *dev_priv)
intel_wakeref_auto_fini(&dev_priv->ggtt.userfault_wakeref);
i915_gem_suspend_late(dev_priv);
- intel_gt_driver_remove(&dev_priv->gt);
+ intel_gt_driver_remove(to_gt(dev_priv));
dev_priv->uabi_engines = RB_ROOT;
/* Flush any outstanding unpin_work. */
@@ -1137,9 +1160,9 @@ void i915_gem_driver_remove(struct drm_i915_private *dev_priv)
void i915_gem_driver_release(struct drm_i915_private *dev_priv)
{
- intel_gt_driver_release(&dev_priv->gt);
+ intel_gt_driver_release(to_gt(dev_priv));
- intel_uc_cleanup_firmwares(&dev_priv->gt.uc);
+ intel_uc_cleanup_firmwares(&to_gt(dev_priv)->uc);
i915_gem_drain_freed_objects(dev_priv);
diff --git a/drivers/gpu/drm/i915/i915_getparam.c b/drivers/gpu/drm/i915/i915_getparam.c
index 77490cb..7f80ad2 100644
--- a/drivers/gpu/drm/i915/i915_getparam.c
+++ b/drivers/gpu/drm/i915/i915_getparam.c
@@ -13,7 +13,7 @@ int i915_getparam_ioctl(struct drm_device *dev, void *data,
{
struct drm_i915_private *i915 = to_i915(dev);
struct pci_dev *pdev = to_pci_dev(dev->dev);
- const struct sseu_dev_info *sseu = &i915->gt.info.sseu;
+ const struct sseu_dev_info *sseu = &to_gt(i915)->info.sseu;
drm_i915_getparam_t *param = data;
int value = 0;
@@ -82,8 +82,8 @@ int i915_getparam_ioctl(struct drm_device *dev, void *data,
break;
case I915_PARAM_HAS_GPU_RESET:
value = i915->params.enable_hangcheck &&
- intel_has_gpu_reset(&i915->gt);
- if (value && intel_has_reset_engine(&i915->gt))
+ intel_has_gpu_reset(to_gt(i915));
+ if (value && intel_has_reset_engine(to_gt(i915)))
value = 2;
break;
case I915_PARAM_HAS_RESOURCE_STREAMER:
@@ -96,7 +96,7 @@ int i915_getparam_ioctl(struct drm_device *dev, void *data,
value = sseu->min_eu_in_pool;
break;
case I915_PARAM_HUC_STATUS:
- value = intel_huc_check_status(&i915->gt.uc.huc);
+ value = intel_huc_check_status(&to_gt(i915)->uc.huc);
if (value < 0)
return value;
break;
@@ -158,7 +158,7 @@ int i915_getparam_ioctl(struct drm_device *dev, void *data,
return -ENODEV;
break;
case I915_PARAM_CS_TIMESTAMP_FREQUENCY:
- value = i915->gt.clock_frequency;
+ value = to_gt(i915)->clock_frequency;
break;
case I915_PARAM_MMAP_GTT_COHERENT:
value = INTEL_INFO(i915)->has_coherent_ggtt;
diff --git a/drivers/gpu/drm/i915/i915_gpu_error.c b/drivers/gpu/drm/i915/i915_gpu_error.c
index 2a2d764..5ae812d 100644
--- a/drivers/gpu/drm/i915/i915_gpu_error.c
+++ b/drivers/gpu/drm/i915/i915_gpu_error.c
@@ -48,8 +48,9 @@
#include "i915_gpu_error.h"
#include "i915_memcpy.h"
#include "i915_scatterlist.h"
+#include "i915_vma_snapshot.h"
-#define ALLOW_FAIL (GFP_KERNEL | __GFP_RETRY_MAYFAIL | __GFP_NOWARN)
+#define ALLOW_FAIL (__GFP_KSWAPD_RECLAIM | __GFP_RETRY_MAYFAIL | __GFP_NOWARN)
#define ATOMIC_MAYFAIL (GFP_ATOMIC | __GFP_NOWARN)
static void __sg_set_buf(struct scatterlist *sg,
@@ -275,16 +276,16 @@ static bool compress_start(struct i915_vma_compress *c)
static void *compress_next_page(struct i915_vma_compress *c,
struct i915_vma_coredump *dst)
{
- void *page;
+ void *page_addr;
+ struct page *page;
- if (dst->page_count >= dst->num_pages)
- return ERR_PTR(-ENOSPC);
-
- page = pool_alloc(&c->pool, ALLOW_FAIL);
- if (!page)
+ page_addr = pool_alloc(&c->pool, ALLOW_FAIL);
+ if (!page_addr)
return ERR_PTR(-ENOMEM);
- return dst->pages[dst->page_count++] = page;
+ page = virt_to_page(page_addr);
+ list_add_tail(&page->lru, &dst->page_list);
+ return page_addr;
}
static int compress_page(struct i915_vma_compress *c,
@@ -397,7 +398,7 @@ static int compress_page(struct i915_vma_compress *c,
if (!(wc && i915_memcpy_from_wc(ptr, src, PAGE_SIZE)))
memcpy(ptr, src, PAGE_SIZE);
- dst->pages[dst->page_count++] = ptr;
+ list_add_tail(&virt_to_page(ptr)->lru, &dst->page_list);
cond_resched();
return 0;
@@ -504,7 +505,7 @@ static void error_print_context(struct drm_i915_error_state_buf *m,
const char *header,
const struct i915_gem_context_coredump *ctx)
{
- const u32 period = m->i915->gt.clock_period_ns;
+ const u32 period = to_gt(m->i915)->clock_period_ns;
err_printf(m, "%s%s[%d] prio %d, guilty %d active %d, runtime total %lluns, avg %lluns\n",
header, ctx->comm, ctx->pid, ctx->sched_attr.priority,
@@ -614,7 +615,7 @@ static void print_error_vma(struct drm_i915_error_state_buf *m,
const struct i915_vma_coredump *vma)
{
char out[ASCII85_BUFSZ];
- int page;
+ struct page *page;
if (!vma)
return;
@@ -628,16 +629,17 @@ static void print_error_vma(struct drm_i915_error_state_buf *m,
err_printf(m, "gtt_page_sizes = 0x%08x\n", vma->gtt_page_sizes);
err_compression_marker(m);
- for (page = 0; page < vma->page_count; page++) {
+ list_for_each_entry(page, &vma->page_list, lru) {
int i, len;
+ const u32 *addr = page_address(page);
len = PAGE_SIZE;
- if (page == vma->page_count - 1)
+ if (page == list_last_entry(&vma->page_list, typeof(*page), lru))
len -= vma->unused;
len = ascii85_encode_len(len);
for (i = 0; i < len; i++)
- err_puts(m, ascii85_encode(vma->pages[page][i], out));
+ err_puts(m, ascii85_encode(addr[i], out));
}
err_puts(m, "\n");
}
@@ -946,10 +948,12 @@ static void i915_vma_coredump_free(struct i915_vma_coredump *vma)
{
while (vma) {
struct i915_vma_coredump *next = vma->next;
- int page;
+ struct page *page, *n;
- for (page = 0; page < vma->page_count; page++)
- free_page((unsigned long)vma->pages[page]);
+ list_for_each_entry_safe(page, n, &vma->page_list, lru) {
+ list_del_init(&page->lru);
+ __free_page(page);
+ }
kfree(vma);
vma = next;
@@ -1009,25 +1013,21 @@ void __i915_gpu_coredump_free(struct kref *error_ref)
static struct i915_vma_coredump *
i915_vma_coredump_create(const struct intel_gt *gt,
- const struct i915_vma *vma,
- const char *name,
+ const struct i915_vma_snapshot *vsnap,
struct i915_vma_compress *compress)
{
struct i915_ggtt *ggtt = gt->ggtt;
const u64 slot = ggtt->error_capture.start;
struct i915_vma_coredump *dst;
- unsigned long num_pages;
struct sgt_iter iter;
int ret;
might_sleep();
- if (!vma || !vma->pages || !compress)
+ if (!vsnap || !vsnap->pages || !compress)
return NULL;
- num_pages = min_t(u64, vma->size, vma->obj->base.size) >> PAGE_SHIFT;
- num_pages = DIV_ROUND_UP(10 * num_pages, 8); /* worstcase zlib growth */
- dst = kmalloc(sizeof(*dst) + num_pages * sizeof(u32 *), ALLOW_FAIL);
+ dst = kmalloc(sizeof(*dst), ALLOW_FAIL);
if (!dst)
return NULL;
@@ -1036,14 +1036,13 @@ i915_vma_coredump_create(const struct intel_gt *gt,
return NULL;
}
- strcpy(dst->name, name);
+ INIT_LIST_HEAD(&dst->page_list);
+ strcpy(dst->name, vsnap->name);
dst->next = NULL;
- dst->gtt_offset = vma->node.start;
- dst->gtt_size = vma->node.size;
- dst->gtt_page_sizes = vma->page_sizes.gtt;
- dst->num_pages = num_pages;
- dst->page_count = 0;
+ dst->gtt_offset = vsnap->gtt_offset;
+ dst->gtt_size = vsnap->gtt_size;
+ dst->gtt_page_sizes = vsnap->page_sizes;
dst->unused = 0;
ret = -EINVAL;
@@ -1051,7 +1050,7 @@ i915_vma_coredump_create(const struct intel_gt *gt,
void __iomem *s;
dma_addr_t dma;
- for_each_sgt_daddr(dma, iter, vma->pages) {
+ for_each_sgt_daddr(dma, iter, vsnap->pages) {
mutex_lock(&ggtt->error_mutex);
ggtt->vm.insert_page(&ggtt->vm, dma, slot,
I915_CACHE_NONE, 0);
@@ -1069,11 +1068,11 @@ i915_vma_coredump_create(const struct intel_gt *gt,
if (ret)
break;
}
- } else if (__i915_gem_object_is_lmem(vma->obj)) {
- struct intel_memory_region *mem = vma->obj->mm.region;
+ } else if (vsnap->mr && vsnap->mr->type != INTEL_MEMORY_SYSTEM) {
+ struct intel_memory_region *mem = vsnap->mr;
dma_addr_t dma;
- for_each_sgt_daddr(dma, iter, vma->pages) {
+ for_each_sgt_daddr(dma, iter, vsnap->pages) {
void __iomem *s;
s = io_mapping_map_wc(&mem->iomap,
@@ -1089,7 +1088,7 @@ i915_vma_coredump_create(const struct intel_gt *gt,
} else {
struct page *page;
- for_each_sgt_page(page, iter, vma->pages) {
+ for_each_sgt_page(page, iter, vsnap->pages) {
void *s;
drm_clflush_pages(&page, 1);
@@ -1106,8 +1105,13 @@ i915_vma_coredump_create(const struct intel_gt *gt,
}
if (ret || compress_flush(compress, dst)) {
- while (dst->page_count--)
- pool_free(&compress->pool, dst->pages[dst->page_count]);
+ struct page *page, *n;
+
+ list_for_each_entry_safe_reverse(page, n, &dst->page_list, lru) {
+ list_del_init(&page->lru);
+ pool_free(&compress->pool, page_address(page));
+ }
+
kfree(dst);
dst = NULL;
}
@@ -1320,35 +1324,69 @@ static bool record_context(struct i915_gem_context_coredump *e,
struct intel_engine_capture_vma {
struct intel_engine_capture_vma *next;
- struct i915_vma *vma;
+ struct i915_vma_snapshot *vsnap;
char name[16];
+ bool lockdep_cookie;
};
static struct intel_engine_capture_vma *
+capture_vma_snapshot(struct intel_engine_capture_vma *next,
+ struct i915_vma_snapshot *vsnap,
+ gfp_t gfp)
+{
+ struct intel_engine_capture_vma *c;
+
+ if (!i915_vma_snapshot_present(vsnap))
+ return next;
+
+ c = kmalloc(sizeof(*c), gfp);
+ if (!c)
+ return next;
+
+ if (!i915_vma_snapshot_resource_pin(vsnap, &c->lockdep_cookie)) {
+ kfree(c);
+ return next;
+ }
+
+ strcpy(c->name, vsnap->name);
+ c->vsnap = vsnap;
+ i915_vma_snapshot_get(vsnap);
+
+ c->next = next;
+ return c;
+}
+
+static struct intel_engine_capture_vma *
capture_vma(struct intel_engine_capture_vma *next,
struct i915_vma *vma,
const char *name,
gfp_t gfp)
{
- struct intel_engine_capture_vma *c;
+ struct i915_vma_snapshot *vsnap;
if (!vma)
return next;
- c = kmalloc(sizeof(*c), gfp);
- if (!c)
+ /*
+ * If the vma isn't pinned, then the vma should be snapshotted
+ * to a struct i915_vma_snapshot at command submission time.
+ * Not here.
+ */
+ GEM_WARN_ON(!i915_vma_is_pinned(vma));
+ if (!i915_vma_is_pinned(vma))
return next;
- if (!i915_active_acquire_if_busy(&vma->active)) {
- kfree(c);
+ vsnap = i915_vma_snapshot_alloc(gfp);
+ if (!vsnap)
return next;
- }
- strcpy(c->name, name);
- c->vma = vma; /* reference held while active */
+ i915_vma_snapshot_init(vsnap, vma, name);
+ next = capture_vma_snapshot(next, vsnap, gfp);
- c->next = next;
- return c;
+ /* FIXME: Replace on async unbind. */
+ i915_vma_snapshot_put(vsnap);
+
+ return next;
}
static struct intel_engine_capture_vma *
@@ -1359,7 +1397,7 @@ capture_user(struct intel_engine_capture_vma *capture,
struct i915_capture_list *c;
for (c = rq->capture_list; c; c = c->next)
- capture = capture_vma(capture, c->vma, "user", gfp);
+ capture = capture_vma_snapshot(capture, c->vma_snapshot, gfp);
return capture;
}
@@ -1373,6 +1411,33 @@ static void add_vma(struct intel_engine_coredump *ee,
}
}
+static struct i915_vma_coredump *
+create_vma_coredump(const struct intel_gt *gt, struct i915_vma *vma,
+ const char *name, struct i915_vma_compress *compress)
+{
+ struct i915_vma_coredump *ret;
+ struct i915_vma_snapshot tmp;
+
+ if (!vma)
+ return NULL;
+
+ GEM_WARN_ON(!i915_vma_is_pinned(vma));
+ i915_vma_snapshot_init_onstack(&tmp, vma, name);
+ ret = i915_vma_coredump_create(gt, &tmp, compress);
+ i915_vma_snapshot_put_onstack(&tmp);
+
+ return ret;
+}
+
+static void add_vma_coredump(struct intel_engine_coredump *ee,
+ const struct intel_gt *gt,
+ struct i915_vma *vma,
+ const char *name,
+ struct i915_vma_compress *compress)
+{
+ add_vma(ee, create_vma_coredump(gt, vma, name, compress));
+}
+
struct intel_engine_coredump *
intel_engine_coredump_alloc(struct intel_engine_cs *engine, gfp_t gfp)
{
@@ -1406,7 +1471,7 @@ intel_engine_coredump_add_request(struct intel_engine_coredump *ee,
* as the simplest method to avoid being overwritten
* by userspace.
*/
- vma = capture_vma(vma, rq->batch, "batch", gfp);
+ vma = capture_vma_snapshot(vma, &rq->batch_snapshot, gfp);
vma = capture_user(vma, rq, gfp);
vma = capture_vma(vma, rq->ring->vma, "ring", gfp);
vma = capture_vma(vma, rq->context->state, "HW context", gfp);
@@ -1427,30 +1492,24 @@ intel_engine_coredump_add_vma(struct intel_engine_coredump *ee,
while (capture) {
struct intel_engine_capture_vma *this = capture;
- struct i915_vma *vma = this->vma;
+ struct i915_vma_snapshot *vsnap = this->vsnap;
add_vma(ee,
i915_vma_coredump_create(engine->gt,
- vma, this->name,
- compress));
+ vsnap, compress));
- i915_active_release(&vma->active);
+ i915_vma_snapshot_resource_unpin(vsnap, this->lockdep_cookie);
+ i915_vma_snapshot_put(vsnap);
capture = this->next;
kfree(this);
}
- add_vma(ee,
- i915_vma_coredump_create(engine->gt,
- engine->status_page.vma,
- "HW Status",
- compress));
+ add_vma_coredump(ee, engine->gt, engine->status_page.vma,
+ "HW Status", compress);
- add_vma(ee,
- i915_vma_coredump_create(engine->gt,
- engine->wa_ctx.vma,
- "WA context",
- compress));
+ add_vma_coredump(ee, engine->gt, engine->wa_ctx.vma,
+ "WA context", compress);
}
static struct intel_engine_coredump *
@@ -1486,17 +1545,25 @@ capture_engine(struct intel_engine_cs *engine,
}
}
if (rq)
- capture = intel_engine_coredump_add_request(ee, rq,
- ATOMIC_MAYFAIL);
+ rq = i915_request_get_rcu(rq);
+
+ if (!rq)
+ goto no_request_capture;
+
+ capture = intel_engine_coredump_add_request(ee, rq, ATOMIC_MAYFAIL);
if (!capture) {
-no_request_capture:
- kfree(ee);
- return NULL;
+ i915_request_put(rq);
+ goto no_request_capture;
}
intel_engine_coredump_add_vma(ee, capture, compress);
+ i915_request_put(rq);
return ee;
+
+no_request_capture:
+ kfree(ee);
+ return NULL;
}
static void
@@ -1550,10 +1617,8 @@ gt_record_uc(struct intel_gt_coredump *gt,
*/
error_uc->guc_fw.path = kstrdup(uc->guc.fw.path, ALLOW_FAIL);
error_uc->huc_fw.path = kstrdup(uc->huc.fw.path, ALLOW_FAIL);
- error_uc->guc_log =
- i915_vma_coredump_create(gt->_gt,
- uc->guc.log.vma, "GuC log buffer",
- compress);
+ error_uc->guc_log = create_vma_coredump(gt->_gt, uc->guc.log.vma,
+ "GuC log buffer", compress);
return error_uc;
}
@@ -1750,10 +1815,7 @@ static void capture_gen(struct i915_gpu_coredump *error)
error->wakelock = atomic_read(&i915->runtime_pm.wakeref_count);
error->suspended = i915->runtime_pm.suspended;
- error->iommu = -1;
-#ifdef CONFIG_INTEL_IOMMU
- error->iommu = intel_iommu_gfx_mapped;
-#endif
+ error->iommu = intel_vtd_active(i915);
error->reset_count = i915_reset_count(&i915->gpu_error);
error->suspend_count = i915->suspend_count;
@@ -1784,7 +1846,7 @@ i915_gpu_coredump_alloc(struct drm_i915_private *i915, gfp_t gfp)
error->time = ktime_get_real();
error->boottime = ktime_get_boottime();
- error->uptime = ktime_sub(ktime_get(), i915->gt.last_init_time);
+ error->uptime = ktime_sub(ktime_get(), to_gt(i915)->last_init_time);
error->capture = jiffies;
capture_gen(error);
@@ -1839,8 +1901,8 @@ void i915_vma_capture_finish(struct intel_gt_coredump *gt,
kfree(compress);
}
-struct i915_gpu_coredump *
-i915_gpu_coredump(struct intel_gt *gt, intel_engine_mask_t engine_mask)
+static struct i915_gpu_coredump *
+__i915_gpu_coredump(struct intel_gt *gt, intel_engine_mask_t engine_mask)
{
struct drm_i915_private *i915 = gt->i915;
struct i915_gpu_coredump *error;
@@ -1881,6 +1943,22 @@ i915_gpu_coredump(struct intel_gt *gt, intel_engine_mask_t engine_mask)
return error;
}
+struct i915_gpu_coredump *
+i915_gpu_coredump(struct intel_gt *gt, intel_engine_mask_t engine_mask)
+{
+ static DEFINE_MUTEX(capture_mutex);
+ int ret = mutex_lock_interruptible(&capture_mutex);
+ struct i915_gpu_coredump *dump;
+
+ if (ret)
+ return ERR_PTR(ret);
+
+ dump = __i915_gpu_coredump(gt, engine_mask);
+ mutex_unlock(&capture_mutex);
+
+ return dump;
+}
+
void i915_error_state_store(struct i915_gpu_coredump *error)
{
struct drm_i915_private *i915;
diff --git a/drivers/gpu/drm/i915/i915_gpu_error.h b/drivers/gpu/drm/i915/i915_gpu_error.h
index b98d8cd..5aedf51 100644
--- a/drivers/gpu/drm/i915/i915_gpu_error.h
+++ b/drivers/gpu/drm/i915/i915_gpu_error.h
@@ -39,10 +39,8 @@ struct i915_vma_coredump {
u64 gtt_size;
u32 gtt_page_sizes;
- int num_pages;
- int page_count;
int unused;
- u32 *pages[];
+ struct list_head page_list;
};
struct i915_request_coredump {
diff --git a/drivers/gpu/drm/i915/i915_iosf_mbi.h b/drivers/gpu/drm/i915/i915_iosf_mbi.h
new file mode 100644
index 0000000..8f81b76
--- /dev/null
+++ b/drivers/gpu/drm/i915/i915_iosf_mbi.h
@@ -0,0 +1,42 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2021 Intel Corporation
+ */
+
+#ifndef __I915_IOSF_MBI_H__
+#define __I915_IOSF_MBI_H__
+
+#if IS_ENABLED(CONFIG_IOSF_MBI)
+#include <asm/iosf_mbi.h>
+#else
+
+/* Stubs to compile for all non-x86 archs */
+#define MBI_PMIC_BUS_ACCESS_BEGIN 1
+#define MBI_PMIC_BUS_ACCESS_END 2
+
+struct notifier_block;
+
+static inline void iosf_mbi_punit_acquire(void) {}
+static inline void iosf_mbi_punit_release(void) {}
+static inline void iosf_mbi_assert_punit_acquired(void) {}
+
+static inline
+int iosf_mbi_register_pmic_bus_access_notifier(struct notifier_block *nb)
+{
+ return 0;
+}
+
+static inline int
+iosf_mbi_unregister_pmic_bus_access_notifier_unlocked(struct notifier_block *nb)
+{
+ return 0;
+}
+
+static inline
+int iosf_mbi_unregister_pmic_bus_access_notifier(struct notifier_block *nb)
+{
+ return 0;
+}
+#endif
+
+#endif /* __I915_IOSF_MBI_H__ */
diff --git a/drivers/gpu/drm/i915/i915_irq.c b/drivers/gpu/drm/i915/i915_irq.c
index 77680bc..21f75b0 100644
--- a/drivers/gpu/drm/i915/i915_irq.c
+++ b/drivers/gpu/drm/i915/i915_irq.c
@@ -35,6 +35,7 @@
#include <drm/drm_drv.h>
#include "display/intel_de.h"
+#include "display/intel_display_trace.h"
#include "display/intel_display_types.h"
#include "display/intel_fifo_underrun.h"
#include "display/intel_hotplug.h"
@@ -49,7 +50,6 @@
#include "i915_drv.h"
#include "i915_irq.h"
-#include "i915_trace.h"
#include "intel_pm.h"
/**
@@ -224,7 +224,7 @@ static void intel_hpd_init_pins(struct drm_i915_private *dev_priv)
static void
intel_handle_vblank(struct drm_i915_private *dev_priv, enum pipe pipe)
{
- struct intel_crtc *crtc = intel_get_crtc_for_pipe(dev_priv, pipe);
+ struct intel_crtc *crtc = intel_crtc_for_pipe(dev_priv, pipe);
drm_crtc_handle_vblank(&crtc->base);
}
@@ -1040,7 +1040,7 @@ static void ivb_parity_work(struct work_struct *work)
{
struct drm_i915_private *dev_priv =
container_of(work, typeof(*dev_priv), l3_parity.error_work);
- struct intel_gt *gt = &dev_priv->gt;
+ struct intel_gt *gt = to_gt(dev_priv);
u32 error_status, row, bank, subbank;
char *parity_event[6];
u32 misccpctl;
@@ -1318,7 +1318,7 @@ static void display_pipe_crc_irq_handler(struct drm_i915_private *dev_priv,
u32 crc2, u32 crc3,
u32 crc4)
{
- struct intel_crtc *crtc = intel_get_crtc_for_pipe(dev_priv, pipe);
+ struct intel_crtc *crtc = intel_crtc_for_pipe(dev_priv, pipe);
struct intel_pipe_crc *pipe_crc = &crtc->pipe_crc;
u32 crcs[5] = { crc0, crc1, crc2, crc3, crc4 };
@@ -1357,7 +1357,7 @@ display_pipe_crc_irq_handler(struct drm_i915_private *dev_priv,
static void flip_done_handler(struct drm_i915_private *i915,
enum pipe pipe)
{
- struct intel_crtc *crtc = intel_get_crtc_for_pipe(i915, pipe);
+ struct intel_crtc *crtc = intel_crtc_for_pipe(i915, pipe);
struct drm_crtc_state *crtc_state = crtc->base.state;
struct drm_pending_vblank_event *e = crtc_state->event;
struct drm_device *dev = &i915->drm;
@@ -1718,9 +1718,9 @@ static irqreturn_t valleyview_irq_handler(int irq, void *arg)
intel_uncore_write(&dev_priv->uncore, VLV_MASTER_IER, MASTER_INTERRUPT_ENABLE);
if (gt_iir)
- gen6_gt_irq_handler(&dev_priv->gt, gt_iir);
+ gen6_gt_irq_handler(to_gt(dev_priv), gt_iir);
if (pm_iir)
- gen6_rps_irq_handler(&dev_priv->gt.rps, pm_iir);
+ gen6_rps_irq_handler(&to_gt(dev_priv)->rps, pm_iir);
if (hotplug_status)
i9xx_hpd_irq_handler(dev_priv, hotplug_status);
@@ -1777,7 +1777,7 @@ static irqreturn_t cherryview_irq_handler(int irq, void *arg)
ier = intel_uncore_read(&dev_priv->uncore, VLV_IER);
intel_uncore_write(&dev_priv->uncore, VLV_IER, 0);
- gen8_gt_irq_handler(&dev_priv->gt, master_ctl);
+ gen8_gt_irq_handler(to_gt(dev_priv), master_ctl);
if (iir & I915_DISPLAY_PORT_INTERRUPT)
hotplug_status = i9xx_hpd_irq_ack(dev_priv);
@@ -2108,7 +2108,7 @@ static void ilk_display_irq_handler(struct drm_i915_private *dev_priv,
}
if (DISPLAY_VER(dev_priv) == 5 && de_iir & DE_PCU_EVENT)
- gen5_rps_irq_handler(&dev_priv->gt.rps);
+ gen5_rps_irq_handler(&to_gt(dev_priv)->rps);
}
static void ivb_display_irq_handler(struct drm_i915_private *dev_priv,
@@ -2189,9 +2189,9 @@ static irqreturn_t ilk_irq_handler(int irq, void *arg)
if (gt_iir) {
raw_reg_write(regs, GTIIR, gt_iir);
if (GRAPHICS_VER(i915) >= 6)
- gen6_gt_irq_handler(&i915->gt, gt_iir);
+ gen6_gt_irq_handler(to_gt(i915), gt_iir);
else
- gen5_gt_irq_handler(&i915->gt, gt_iir);
+ gen5_gt_irq_handler(to_gt(i915), gt_iir);
ret = IRQ_HANDLED;
}
@@ -2209,7 +2209,7 @@ static irqreturn_t ilk_irq_handler(int irq, void *arg)
u32 pm_iir = raw_reg_read(regs, GEN6_PMIIR);
if (pm_iir) {
raw_reg_write(regs, GEN6_PMIIR, pm_iir);
- gen6_rps_irq_handler(&i915->gt.rps, pm_iir);
+ gen6_rps_irq_handler(&to_gt(i915)->rps, pm_iir);
ret = IRQ_HANDLED;
}
}
@@ -2635,7 +2635,7 @@ static irqreturn_t gen8_irq_handler(int irq, void *arg)
}
/* Find, queue (onto bottom-halves), then clear each source */
- gen8_gt_irq_handler(&dev_priv->gt, master_ctl);
+ gen8_gt_irq_handler(to_gt(dev_priv), master_ctl);
/* IRQs are synced during runtime_suspend, we don't require a wakeref */
if (master_ctl & ~GEN8_GT_IRQS) {
@@ -2715,7 +2715,7 @@ static irqreturn_t gen11_irq_handler(int irq, void *arg)
{
struct drm_i915_private *i915 = arg;
void __iomem * const regs = i915->uncore.regs;
- struct intel_gt *gt = &i915->gt;
+ struct intel_gt *gt = to_gt(i915);
u32 master_ctl;
u32 gu_misc_iir;
@@ -2771,8 +2771,8 @@ static inline void dg1_master_intr_enable(void __iomem * const regs)
static irqreturn_t dg1_irq_handler(int irq, void *arg)
{
struct drm_i915_private * const i915 = arg;
- struct intel_gt *gt = &i915->gt;
- void __iomem * const regs = i915->uncore.regs;
+ struct intel_gt *gt = to_gt(i915);
+ void __iomem * const regs = gt->uncore->regs;
u32 master_tile_ctl, master_ctl;
u32 gu_misc_iir;
@@ -3016,7 +3016,7 @@ static void vlv_display_irq_reset(struct drm_i915_private *dev_priv)
if (IS_CHERRYVIEW(dev_priv))
intel_uncore_write(uncore, DPINVGTT, DPINVGTT_STATUS_MASK_CHV);
else
- intel_uncore_write(uncore, DPINVGTT, DPINVGTT_STATUS_MASK);
+ intel_uncore_write(uncore, DPINVGTT, DPINVGTT_STATUS_MASK_VLV);
i915_hotplug_interrupt_update_locked(dev_priv, 0xffffffff, 0);
intel_uncore_write(uncore, PORT_HOTPLUG_STAT, intel_uncore_read(&dev_priv->uncore, PORT_HOTPLUG_STAT));
@@ -3075,7 +3075,7 @@ static void ilk_irq_reset(struct drm_i915_private *dev_priv)
intel_uncore_write(uncore, EDP_PSR_IIR, 0xffffffff);
}
- gen5_gt_irq_reset(&dev_priv->gt);
+ gen5_gt_irq_reset(to_gt(dev_priv));
ibx_irq_reset(dev_priv);
}
@@ -3085,7 +3085,7 @@ static void valleyview_irq_reset(struct drm_i915_private *dev_priv)
intel_uncore_write(&dev_priv->uncore, VLV_MASTER_IER, 0);
intel_uncore_posting_read(&dev_priv->uncore, VLV_MASTER_IER);
- gen5_gt_irq_reset(&dev_priv->gt);
+ gen5_gt_irq_reset(to_gt(dev_priv));
spin_lock_irq(&dev_priv->irq_lock);
if (dev_priv->display_irqs_enabled)
@@ -3119,7 +3119,7 @@ static void gen8_irq_reset(struct drm_i915_private *dev_priv)
gen8_master_intr_disable(dev_priv->uncore.regs);
- gen8_gt_irq_reset(&dev_priv->gt);
+ gen8_gt_irq_reset(to_gt(dev_priv));
gen8_display_irq_reset(dev_priv);
GEN3_IRQ_RESET(uncore, GEN8_PCU_);
@@ -3173,11 +3173,12 @@ static void gen11_display_irq_reset(struct drm_i915_private *dev_priv)
static void gen11_irq_reset(struct drm_i915_private *dev_priv)
{
- struct intel_uncore *uncore = &dev_priv->uncore;
+ struct intel_gt *gt = to_gt(dev_priv);
+ struct intel_uncore *uncore = gt->uncore;
gen11_master_intr_disable(dev_priv->uncore.regs);
- gen11_gt_irq_reset(&dev_priv->gt);
+ gen11_gt_irq_reset(gt);
gen11_display_irq_reset(dev_priv);
GEN3_IRQ_RESET(uncore, GEN11_GU_MISC_);
@@ -3186,11 +3187,12 @@ static void gen11_irq_reset(struct drm_i915_private *dev_priv)
static void dg1_irq_reset(struct drm_i915_private *dev_priv)
{
- struct intel_uncore *uncore = &dev_priv->uncore;
+ struct intel_gt *gt = to_gt(dev_priv);
+ struct intel_uncore *uncore = gt->uncore;
dg1_master_intr_disable(dev_priv->uncore.regs);
- gen11_gt_irq_reset(&dev_priv->gt);
+ gen11_gt_irq_reset(gt);
gen11_display_irq_reset(dev_priv);
GEN3_IRQ_RESET(uncore, GEN11_GU_MISC_);
@@ -3250,7 +3252,7 @@ static void cherryview_irq_reset(struct drm_i915_private *dev_priv)
intel_uncore_write(&dev_priv->uncore, GEN8_MASTER_IRQ, 0);
intel_uncore_posting_read(&dev_priv->uncore, GEN8_MASTER_IRQ);
- gen8_gt_irq_reset(&dev_priv->gt);
+ gen8_gt_irq_reset(to_gt(dev_priv));
GEN3_IRQ_RESET(uncore, GEN8_PCU_);
@@ -3707,7 +3709,7 @@ static void ilk_irq_postinstall(struct drm_i915_private *dev_priv)
ibx_irq_postinstall(dev_priv);
- gen5_gt_irq_postinstall(&dev_priv->gt);
+ gen5_gt_irq_postinstall(to_gt(dev_priv));
GEN3_IRQ_INIT(uncore, DE, dev_priv->irq_mask,
display_mask | extra_mask);
@@ -3744,7 +3746,7 @@ void valleyview_disable_display_irqs(struct drm_i915_private *dev_priv)
static void valleyview_irq_postinstall(struct drm_i915_private *dev_priv)
{
- gen5_gt_irq_postinstall(&dev_priv->gt);
+ gen5_gt_irq_postinstall(to_gt(dev_priv));
spin_lock_irq(&dev_priv->irq_lock);
if (dev_priv->display_irqs_enabled)
@@ -3850,7 +3852,7 @@ static void gen8_irq_postinstall(struct drm_i915_private *dev_priv)
else if (HAS_PCH_SPLIT(dev_priv))
ibx_irq_postinstall(dev_priv);
- gen8_gt_irq_postinstall(&dev_priv->gt);
+ gen8_gt_irq_postinstall(to_gt(dev_priv));
gen8_de_irq_postinstall(dev_priv);
gen8_master_intr_enable(dev_priv->uncore.regs);
@@ -3869,13 +3871,14 @@ static void gen11_de_irq_postinstall(struct drm_i915_private *dev_priv)
static void gen11_irq_postinstall(struct drm_i915_private *dev_priv)
{
- struct intel_uncore *uncore = &dev_priv->uncore;
+ struct intel_gt *gt = to_gt(dev_priv);
+ struct intel_uncore *uncore = gt->uncore;
u32 gu_misc_masked = GEN11_GU_MISC_GSE;
if (INTEL_PCH_TYPE(dev_priv) >= PCH_ICP)
icp_irq_postinstall(dev_priv);
- gen11_gt_irq_postinstall(&dev_priv->gt);
+ gen11_gt_irq_postinstall(gt);
gen11_de_irq_postinstall(dev_priv);
GEN3_IRQ_INIT(uncore, GEN11_GU_MISC_, ~gu_misc_masked, gu_misc_masked);
@@ -3886,10 +3889,11 @@ static void gen11_irq_postinstall(struct drm_i915_private *dev_priv)
static void dg1_irq_postinstall(struct drm_i915_private *dev_priv)
{
- struct intel_uncore *uncore = &dev_priv->uncore;
+ struct intel_gt *gt = to_gt(dev_priv);
+ struct intel_uncore *uncore = gt->uncore;
u32 gu_misc_masked = GEN11_GU_MISC_GSE;
- gen11_gt_irq_postinstall(&dev_priv->gt);
+ gen11_gt_irq_postinstall(gt);
GEN3_IRQ_INIT(uncore, GEN11_GU_MISC_, ~gu_misc_masked, gu_misc_masked);
@@ -3900,13 +3904,13 @@ static void dg1_irq_postinstall(struct drm_i915_private *dev_priv)
GEN11_DISPLAY_IRQ_ENABLE);
}
- dg1_master_intr_enable(dev_priv->uncore.regs);
- intel_uncore_posting_read(&dev_priv->uncore, DG1_MSTR_TILE_INTR);
+ dg1_master_intr_enable(uncore->regs);
+ intel_uncore_posting_read(uncore, DG1_MSTR_TILE_INTR);
}
static void cherryview_irq_postinstall(struct drm_i915_private *dev_priv)
{
- gen8_gt_irq_postinstall(&dev_priv->gt);
+ gen8_gt_irq_postinstall(to_gt(dev_priv));
spin_lock_irq(&dev_priv->irq_lock);
if (dev_priv->display_irqs_enabled)
@@ -4069,7 +4073,7 @@ static irqreturn_t i8xx_irq_handler(int irq, void *arg)
intel_uncore_write16(&dev_priv->uncore, GEN2_IIR, iir);
if (iir & I915_USER_INTERRUPT)
- intel_engine_cs_irq(dev_priv->gt.engine[RCS0], iir);
+ intel_engine_cs_irq(to_gt(dev_priv)->engine[RCS0], iir);
if (iir & I915_MASTER_ERROR_INTERRUPT)
i8xx_error_irq_handler(dev_priv, eir, eir_stuck);
@@ -4177,7 +4181,7 @@ static irqreturn_t i915_irq_handler(int irq, void *arg)
intel_uncore_write(&dev_priv->uncore, GEN2_IIR, iir);
if (iir & I915_USER_INTERRUPT)
- intel_engine_cs_irq(dev_priv->gt.engine[RCS0], iir);
+ intel_engine_cs_irq(to_gt(dev_priv)->engine[RCS0], iir);
if (iir & I915_MASTER_ERROR_INTERRUPT)
i9xx_error_irq_handler(dev_priv, eir, eir_stuck);
@@ -4322,11 +4326,11 @@ static irqreturn_t i965_irq_handler(int irq, void *arg)
intel_uncore_write(&dev_priv->uncore, GEN2_IIR, iir);
if (iir & I915_USER_INTERRUPT)
- intel_engine_cs_irq(dev_priv->gt.engine[RCS0],
+ intel_engine_cs_irq(to_gt(dev_priv)->engine[RCS0],
iir);
if (iir & I915_BSD_USER_INTERRUPT)
- intel_engine_cs_irq(dev_priv->gt.engine[VCS0],
+ intel_engine_cs_irq(to_gt(dev_priv)->engine[VCS0],
iir >> 25);
if (iir & I915_MASTER_ERROR_INTERRUPT)
@@ -4377,7 +4381,7 @@ void intel_irq_init(struct drm_i915_private *dev_priv)
/* pre-gen11 the guc irqs bits are in the upper 16 bits of the pm reg */
if (HAS_GT_UC(dev_priv) && GRAPHICS_VER(dev_priv) < 11)
- dev_priv->gt.pm_guc_events = GUC_INTR_GUC2HOST << 16;
+ to_gt(dev_priv)->pm_guc_events = GUC_INTR_GUC2HOST << 16;
if (!HAS_DISPLAY(dev_priv))
return;
diff --git a/drivers/gpu/drm/i915/i915_mm.c b/drivers/gpu/drm/i915/i915_mm.c
index 666808c..7998bc7 100644
--- a/drivers/gpu/drm/i915/i915_mm.c
+++ b/drivers/gpu/drm/i915/i915_mm.c
@@ -27,6 +27,7 @@
#include "i915_drv.h"
+#include "i915_mm.h"
struct remap_pfn {
struct mm_struct *mm;
@@ -37,17 +38,6 @@ struct remap_pfn {
resource_size_t iobase;
};
-static int remap_pfn(pte_t *pte, unsigned long addr, void *data)
-{
- struct remap_pfn *r = data;
-
- /* Special PTE are not associated with any struct page */
- set_pte_at(r->mm, addr, pte, pte_mkspecial(pfn_pte(r->pfn, r->prot)));
- r->pfn++;
-
- return 0;
-}
-
#define use_dma(io) ((io) != -1)
static inline unsigned long sgt_pfn(const struct remap_pfn *r)
@@ -77,6 +67,20 @@ static int remap_sg(pte_t *pte, unsigned long addr, void *data)
return 0;
}
+#define EXPECTED_FLAGS (VM_PFNMAP | VM_DONTEXPAND | VM_DONTDUMP)
+
+#if IS_ENABLED(CONFIG_X86)
+static int remap_pfn(pte_t *pte, unsigned long addr, void *data)
+{
+ struct remap_pfn *r = data;
+
+ /* Special PTE are not associated with any struct page */
+ set_pte_at(r->mm, addr, pte, pte_mkspecial(pfn_pte(r->pfn, r->prot)));
+ r->pfn++;
+
+ return 0;
+}
+
/**
* remap_io_mapping - remap an IO mapping to userspace
* @vma: user vma to map to
@@ -94,7 +98,6 @@ int remap_io_mapping(struct vm_area_struct *vma,
struct remap_pfn r;
int err;
-#define EXPECTED_FLAGS (VM_PFNMAP | VM_DONTEXPAND | VM_DONTDUMP)
GEM_BUG_ON((vma->vm_flags & EXPECTED_FLAGS) != EXPECTED_FLAGS);
/* We rely on prevalidation of the io-mapping to skip track_pfn(). */
@@ -111,6 +114,7 @@ int remap_io_mapping(struct vm_area_struct *vma,
return 0;
}
+#endif
/**
* remap_io_sg - remap an IO mapping to userspace
diff --git a/drivers/gpu/drm/i915/i915_mm.h b/drivers/gpu/drm/i915/i915_mm.h
new file mode 100644
index 0000000..76f1d53
--- /dev/null
+++ b/drivers/gpu/drm/i915/i915_mm.h
@@ -0,0 +1,35 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2021 Intel Corporation
+ */
+
+#ifndef __I915_MM_H__
+#define __I915_MM_H__
+
+#include <linux/types.h>
+
+struct vm_area_struct;
+struct io_mapping;
+struct scatterlist;
+
+#if IS_ENABLED(CONFIG_X86)
+int remap_io_mapping(struct vm_area_struct *vma,
+ unsigned long addr, unsigned long pfn, unsigned long size,
+ struct io_mapping *iomap);
+#else
+static inline
+int remap_io_mapping(struct vm_area_struct *vma,
+ unsigned long addr, unsigned long pfn, unsigned long size,
+ struct io_mapping *iomap)
+{
+ pr_err("Architecture has no %s() and shouldn't be calling this function\n", __func__);
+ WARN_ON_ONCE(1);
+ return 0;
+}
+#endif
+
+int remap_io_sg(struct vm_area_struct *vma,
+ unsigned long addr, unsigned long size,
+ struct scatterlist *sgl, resource_size_t iobase);
+
+#endif /* __I915_MM_H__ */
diff --git a/drivers/gpu/drm/i915/i915_module.c b/drivers/gpu/drm/i915/i915_module.c
index ab2295d..f6bcd2f 100644
--- a/drivers/gpu/drm/i915/i915_module.c
+++ b/drivers/gpu/drm/i915/i915_module.c
@@ -4,7 +4,7 @@
* Copyright © 2021 Intel Corporation
*/
-#include <linux/console.h>
+#include <drm/drm_drv.h>
#include "gem/i915_gem_context.h"
#include "gem/i915_gem_object.h"
@@ -24,14 +24,14 @@ static int i915_check_nomodeset(void)
/*
* Enable KMS by default, unless explicitly overriden by
- * either the i915.modeset prarameter or by the
- * vga_text_mode_force boot option.
+ * either the i915.modeset parameter or by the
+ * nomodeset boot option.
*/
if (i915_modparams.modeset == 0)
use_kms = false;
- if (vgacon_text_force() && i915_modparams.modeset == -1)
+ if (drm_firmware_drivers_only() && i915_modparams.modeset == -1)
use_kms = false;
if (!use_kms) {
diff --git a/drivers/gpu/drm/i915/i915_params.c b/drivers/gpu/drm/i915/i915_params.c
index e07f4cf..525ae83 100644
--- a/drivers/gpu/drm/i915/i915_params.c
+++ b/drivers/gpu/drm/i915/i915_params.c
@@ -140,6 +140,9 @@ i915_param_named_unsafe(invert_brightness, int, 0400,
i915_param_named(disable_display, bool, 0400,
"Disable display (default: false)");
+i915_param_named(memtest, bool, 0400,
+ "Perform a read/write test of all device memory on module load (default: off)");
+
i915_param_named(mmio_debug, int, 0400,
"Enable the MMIO debug code for the first N failures (default: off). "
"This may negatively affect performance.");
diff --git a/drivers/gpu/drm/i915/i915_params.h b/drivers/gpu/drm/i915/i915_params.h
index 8d725b6..c9d53ff9 100644
--- a/drivers/gpu/drm/i915/i915_params.h
+++ b/drivers/gpu/drm/i915/i915_params.h
@@ -64,6 +64,7 @@ struct drm_printer;
param(char *, guc_firmware_path, NULL, 0400) \
param(char *, huc_firmware_path, NULL, 0400) \
param(char *, dmc_firmware_path, NULL, 0400) \
+ param(bool, memtest, false, 0400) \
param(int, mmio_debug, -IS_ENABLED(CONFIG_DRM_I915_DEBUG_MMIO), 0600) \
param(int, edp_vswing, 0, 0400) \
param(unsigned int, reset, 3, 0600) \
diff --git a/drivers/gpu/drm/i915/i915_pci.c b/drivers/gpu/drm/i915/i915_pci.c
index 169837d..261294d 100644
--- a/drivers/gpu/drm/i915/i915_pci.c
+++ b/drivers/gpu/drm/i915/i915_pci.c
@@ -22,18 +22,17 @@
*
*/
-#include <linux/vga_switcheroo.h>
-
#include <drm/drm_drv.h>
#include <drm/i915_pciids.h>
+#include "i915_driver.h"
#include "i915_drv.h"
#include "i915_pci.h"
#define PLATFORM(x) .platform = (x)
#define GEN(x) \
- .graphics_ver = (x), \
- .media_ver = (x), \
+ .graphics.ver = (x), \
+ .media.ver = (x), \
.display.ver = (x)
#define I845_PIPE_OFFSETS \
@@ -145,6 +144,12 @@
.degamma_lut_tests = DRM_COLOR_LUT_NON_DECREASING | \
DRM_COLOR_LUT_EQUAL_CHANNELS, \
}
+#define ICL_COLORS \
+ .color = { .degamma_lut_size = 33, .gamma_lut_size = 262145, \
+ .degamma_lut_tests = DRM_COLOR_LUT_NON_DECREASING | \
+ DRM_COLOR_LUT_EQUAL_CHANNELS, \
+ .gamma_lut_tests = DRM_COLOR_LUT_NON_DECREASING, \
+ }
/* Keep in gen based order, and chronological order within a gen */
@@ -157,8 +162,8 @@
#define I830_FEATURES \
GEN(2), \
.is_mobile = 1, \
- .pipe_mask = BIT(PIPE_A) | BIT(PIPE_B), \
- .cpu_transcoder_mask = BIT(TRANSCODER_A) | BIT(TRANSCODER_B), \
+ .display.pipe_mask = BIT(PIPE_A) | BIT(PIPE_B), \
+ .display.cpu_transcoder_mask = BIT(TRANSCODER_A) | BIT(TRANSCODER_B), \
.display.has_overlay = 1, \
.display.cursor_needs_physical = 1, \
.display.overlay_needs_physical = 1, \
@@ -178,8 +183,8 @@
#define I845_FEATURES \
GEN(2), \
- .pipe_mask = BIT(PIPE_A), \
- .cpu_transcoder_mask = BIT(TRANSCODER_A), \
+ .display.pipe_mask = BIT(PIPE_A), \
+ .display.cpu_transcoder_mask = BIT(TRANSCODER_A), \
.display.has_overlay = 1, \
.display.overlay_needs_physical = 1, \
.display.has_gmch = 1, \
@@ -220,8 +225,8 @@ static const struct intel_device_info i865g_info = {
#define GEN3_FEATURES \
GEN(3), \
- .pipe_mask = BIT(PIPE_A) | BIT(PIPE_B), \
- .cpu_transcoder_mask = BIT(TRANSCODER_A) | BIT(TRANSCODER_B), \
+ .display.pipe_mask = BIT(PIPE_A) | BIT(PIPE_B), \
+ .display.cpu_transcoder_mask = BIT(TRANSCODER_A) | BIT(TRANSCODER_B), \
.display.has_gmch = 1, \
.gpu_reset_clobbers_display = true, \
.platform_engine_mask = BIT(RCS0), \
@@ -310,8 +315,8 @@ static const struct intel_device_info pnv_m_info = {
#define GEN4_FEATURES \
GEN(4), \
- .pipe_mask = BIT(PIPE_A) | BIT(PIPE_B), \
- .cpu_transcoder_mask = BIT(TRANSCODER_A) | BIT(TRANSCODER_B), \
+ .display.pipe_mask = BIT(PIPE_A) | BIT(PIPE_B), \
+ .display.cpu_transcoder_mask = BIT(TRANSCODER_A) | BIT(TRANSCODER_B), \
.display.has_hotplug = 1, \
.display.has_gmch = 1, \
.gpu_reset_clobbers_display = true, \
@@ -363,8 +368,8 @@ static const struct intel_device_info gm45_info = {
#define GEN5_FEATURES \
GEN(5), \
- .pipe_mask = BIT(PIPE_A) | BIT(PIPE_B), \
- .cpu_transcoder_mask = BIT(TRANSCODER_A) | BIT(TRANSCODER_B), \
+ .display.pipe_mask = BIT(PIPE_A) | BIT(PIPE_B), \
+ .display.cpu_transcoder_mask = BIT(TRANSCODER_A) | BIT(TRANSCODER_B), \
.display.has_hotplug = 1, \
.platform_engine_mask = BIT(RCS0) | BIT(VCS0), \
.has_snoop = true, \
@@ -393,8 +398,8 @@ static const struct intel_device_info ilk_m_info = {
#define GEN6_FEATURES \
GEN(6), \
- .pipe_mask = BIT(PIPE_A) | BIT(PIPE_B), \
- .cpu_transcoder_mask = BIT(TRANSCODER_A) | BIT(TRANSCODER_B), \
+ .display.pipe_mask = BIT(PIPE_A) | BIT(PIPE_B), \
+ .display.cpu_transcoder_mask = BIT(TRANSCODER_A) | BIT(TRANSCODER_B), \
.display.has_hotplug = 1, \
.display.has_fbc = 1, \
.platform_engine_mask = BIT(RCS0) | BIT(VCS0) | BIT(BCS0), \
@@ -444,8 +449,8 @@ static const struct intel_device_info snb_m_gt2_info = {
#define GEN7_FEATURES \
GEN(7), \
- .pipe_mask = BIT(PIPE_A) | BIT(PIPE_B) | BIT(PIPE_C), \
- .cpu_transcoder_mask = BIT(TRANSCODER_A) | BIT(TRANSCODER_B) | BIT(TRANSCODER_C), \
+ .display.pipe_mask = BIT(PIPE_A) | BIT(PIPE_B) | BIT(PIPE_C), \
+ .display.cpu_transcoder_mask = BIT(TRANSCODER_A) | BIT(TRANSCODER_B) | BIT(TRANSCODER_C), \
.display.has_hotplug = 1, \
.display.has_fbc = 1, \
.platform_engine_mask = BIT(RCS0) | BIT(VCS0) | BIT(BCS0), \
@@ -499,8 +504,8 @@ static const struct intel_device_info ivb_q_info = {
GEN7_FEATURES,
PLATFORM(INTEL_IVYBRIDGE),
.gt = 2,
- .pipe_mask = 0, /* legal, last one wins */
- .cpu_transcoder_mask = 0,
+ .display.pipe_mask = 0, /* legal, last one wins */
+ .display.cpu_transcoder_mask = 0,
.has_l3_dpf = 1,
};
@@ -508,8 +513,8 @@ static const struct intel_device_info vlv_info = {
PLATFORM(INTEL_VALLEYVIEW),
GEN(7),
.is_lp = 1,
- .pipe_mask = BIT(PIPE_A) | BIT(PIPE_B),
- .cpu_transcoder_mask = BIT(TRANSCODER_A) | BIT(TRANSCODER_B),
+ .display.pipe_mask = BIT(PIPE_A) | BIT(PIPE_B),
+ .display.cpu_transcoder_mask = BIT(TRANSCODER_A) | BIT(TRANSCODER_B),
.has_runtime_pm = 1,
.has_rc6 = 1,
.has_reset_engine = true,
@@ -533,7 +538,7 @@ static const struct intel_device_info vlv_info = {
#define G75_FEATURES \
GEN7_FEATURES, \
.platform_engine_mask = BIT(RCS0) | BIT(VCS0) | BIT(BCS0) | BIT(VECS0), \
- .cpu_transcoder_mask = BIT(TRANSCODER_A) | BIT(TRANSCODER_B) | \
+ .display.cpu_transcoder_mask = BIT(TRANSCODER_A) | BIT(TRANSCODER_B) | \
BIT(TRANSCODER_C) | BIT(TRANSCODER_EDP), \
.display.has_ddi = 1, \
.display.has_fpga_dbg = 1, \
@@ -603,8 +608,8 @@ static const struct intel_device_info bdw_gt3_info = {
static const struct intel_device_info chv_info = {
PLATFORM(INTEL_CHERRYVIEW),
GEN(8),
- .pipe_mask = BIT(PIPE_A) | BIT(PIPE_B) | BIT(PIPE_C),
- .cpu_transcoder_mask = BIT(TRANSCODER_A) | BIT(TRANSCODER_B) | BIT(TRANSCODER_C),
+ .display.pipe_mask = BIT(PIPE_A) | BIT(PIPE_B) | BIT(PIPE_C),
+ .display.cpu_transcoder_mask = BIT(TRANSCODER_A) | BIT(TRANSCODER_B) | BIT(TRANSCODER_C),
.display.has_hotplug = 1,
.is_lp = 1,
.platform_engine_mask = BIT(RCS0) | BIT(VCS0) | BIT(BCS0) | BIT(VECS0),
@@ -681,8 +686,8 @@ static const struct intel_device_info skl_gt4_info = {
.dbuf.slice_mask = BIT(DBUF_S1), \
.display.has_hotplug = 1, \
.platform_engine_mask = BIT(RCS0) | BIT(VCS0) | BIT(BCS0) | BIT(VECS0), \
- .pipe_mask = BIT(PIPE_A) | BIT(PIPE_B) | BIT(PIPE_C), \
- .cpu_transcoder_mask = BIT(TRANSCODER_A) | BIT(TRANSCODER_B) | \
+ .display.pipe_mask = BIT(PIPE_A) | BIT(PIPE_B) | BIT(PIPE_C), \
+ .display.cpu_transcoder_mask = BIT(TRANSCODER_A) | BIT(TRANSCODER_B) | \
BIT(TRANSCODER_C) | BIT(TRANSCODER_EDP) | \
BIT(TRANSCODER_DSI_A) | BIT(TRANSCODER_DSI_C), \
.has_64bit_reloc = 1, \
@@ -790,8 +795,8 @@ static const struct intel_device_info cml_gt2_info = {
#define GEN11_FEATURES \
GEN9_FEATURES, \
GEN11_DEFAULT_PAGE_SIZES, \
- .abox_mask = BIT(0), \
- .cpu_transcoder_mask = BIT(TRANSCODER_A) | BIT(TRANSCODER_B) | \
+ .display.abox_mask = BIT(0), \
+ .display.cpu_transcoder_mask = BIT(TRANSCODER_A) | BIT(TRANSCODER_B) | \
BIT(TRANSCODER_C) | BIT(TRANSCODER_EDP) | \
BIT(TRANSCODER_DSI_0) | BIT(TRANSCODER_DSI_1), \
.pipe_offsets = { \
@@ -811,7 +816,7 @@ static const struct intel_device_info cml_gt2_info = {
[TRANSCODER_DSI_1] = TRANSCODER_DSI1_OFFSET, \
}, \
GEN(11), \
- .color = { .degamma_lut_size = 33, .gamma_lut_size = 262145 }, \
+ ICL_COLORS, \
.dbuf.size = 2048, \
.dbuf.slice_mask = BIT(DBUF_S1) | BIT(DBUF_S2), \
.display.has_dsc = 1, \
@@ -842,9 +847,9 @@ static const struct intel_device_info jsl_info = {
#define GEN12_FEATURES \
GEN11_FEATURES, \
GEN(12), \
- .abox_mask = GENMASK(2, 1), \
- .pipe_mask = BIT(PIPE_A) | BIT(PIPE_B) | BIT(PIPE_C) | BIT(PIPE_D), \
- .cpu_transcoder_mask = BIT(TRANSCODER_A) | BIT(TRANSCODER_B) | \
+ .display.abox_mask = GENMASK(2, 1), \
+ .display.pipe_mask = BIT(PIPE_A) | BIT(PIPE_B) | BIT(PIPE_C) | BIT(PIPE_D), \
+ .display.cpu_transcoder_mask = BIT(TRANSCODER_A) | BIT(TRANSCODER_B) | \
BIT(TRANSCODER_C) | BIT(TRANSCODER_D) | \
BIT(TRANSCODER_DSI_0) | BIT(TRANSCODER_DSI_1), \
.pipe_offsets = { \
@@ -866,7 +871,7 @@ static const struct intel_device_info jsl_info = {
TGL_CURSOR_OFFSETS, \
.has_global_mocs = 1, \
.has_pxp = 1, \
- .display.has_dsb = 1
+ .display.has_dsb = 0 /* FIXME: LUT load is broken with DSB */
static const struct intel_device_info tgl_info = {
GEN12_FEATURES,
@@ -879,9 +884,9 @@ static const struct intel_device_info tgl_info = {
static const struct intel_device_info rkl_info = {
GEN12_FEATURES,
PLATFORM(INTEL_ROCKETLAKE),
- .abox_mask = BIT(0),
- .pipe_mask = BIT(PIPE_A) | BIT(PIPE_B) | BIT(PIPE_C),
- .cpu_transcoder_mask = BIT(TRANSCODER_A) | BIT(TRANSCODER_B) |
+ .display.abox_mask = BIT(0),
+ .display.pipe_mask = BIT(PIPE_A) | BIT(PIPE_B) | BIT(PIPE_C),
+ .display.cpu_transcoder_mask = BIT(TRANSCODER_A) | BIT(TRANSCODER_B) |
BIT(TRANSCODER_C),
.display.has_hti = 1,
.display.has_psr_hw_tracking = 0,
@@ -899,9 +904,9 @@ static const struct intel_device_info rkl_info = {
static const struct intel_device_info dg1_info = {
GEN12_FEATURES,
DGFX_FEATURES,
- .graphics_rel = 10,
+ .graphics.rel = 10,
PLATFORM(INTEL_DG1),
- .pipe_mask = BIT(PIPE_A) | BIT(PIPE_B) | BIT(PIPE_C) | BIT(PIPE_D),
+ .display.pipe_mask = BIT(PIPE_A) | BIT(PIPE_B) | BIT(PIPE_C) | BIT(PIPE_D),
.require_force_probe = 1,
.platform_engine_mask =
BIT(RCS0) | BIT(BCS0) | BIT(VECS0) |
@@ -913,7 +918,7 @@ static const struct intel_device_info dg1_info = {
static const struct intel_device_info adl_s_info = {
GEN12_FEATURES,
PLATFORM(INTEL_ALDERLAKE_S),
- .pipe_mask = BIT(PIPE_A) | BIT(PIPE_B) | BIT(PIPE_C) | BIT(PIPE_D),
+ .display.pipe_mask = BIT(PIPE_A) | BIT(PIPE_B) | BIT(PIPE_C) | BIT(PIPE_D),
.display.has_hti = 1,
.display.has_psr_hw_tracking = 0,
.platform_engine_mask =
@@ -930,10 +935,11 @@ static const struct intel_device_info adl_s_info = {
}
#define XE_LPD_FEATURES \
- .abox_mask = GENMASK(1, 0), \
- .color = { .degamma_lut_size = 0, .gamma_lut_size = 0 }, \
- .cpu_transcoder_mask = BIT(TRANSCODER_A) | BIT(TRANSCODER_B) | \
- BIT(TRANSCODER_C) | BIT(TRANSCODER_D), \
+ .display.abox_mask = GENMASK(1, 0), \
+ .color = { .degamma_lut_size = 128, .gamma_lut_size = 1024, \
+ .degamma_lut_tests = DRM_COLOR_LUT_NON_DECREASING | \
+ DRM_COLOR_LUT_EQUAL_CHANNELS, \
+ }, \
.dbuf.size = 4096, \
.dbuf.slice_mask = BIT(DBUF_S1) | BIT(DBUF_S2) | BIT(DBUF_S3) | \
BIT(DBUF_S4), \
@@ -949,18 +955,22 @@ static const struct intel_device_info adl_s_info = {
.display.has_ipc = 1, \
.display.has_psr = 1, \
.display.ver = 13, \
- .pipe_mask = BIT(PIPE_A) | BIT(PIPE_B) | BIT(PIPE_C) | BIT(PIPE_D), \
+ .display.pipe_mask = BIT(PIPE_A) | BIT(PIPE_B) | BIT(PIPE_C) | BIT(PIPE_D), \
.pipe_offsets = { \
[TRANSCODER_A] = PIPE_A_OFFSET, \
[TRANSCODER_B] = PIPE_B_OFFSET, \
[TRANSCODER_C] = PIPE_C_OFFSET, \
[TRANSCODER_D] = PIPE_D_OFFSET, \
+ [TRANSCODER_DSI_0] = PIPE_DSI0_OFFSET, \
+ [TRANSCODER_DSI_1] = PIPE_DSI1_OFFSET, \
}, \
.trans_offsets = { \
[TRANSCODER_A] = TRANSCODER_A_OFFSET, \
[TRANSCODER_B] = TRANSCODER_B_OFFSET, \
[TRANSCODER_C] = TRANSCODER_C_OFFSET, \
[TRANSCODER_D] = TRANSCODER_D_OFFSET, \
+ [TRANSCODER_DSI_0] = TRANSCODER_DSI0_OFFSET, \
+ [TRANSCODER_DSI_1] = TRANSCODER_DSI1_OFFSET, \
}, \
XE_LPD_CURSOR_OFFSETS
@@ -968,7 +978,9 @@ static const struct intel_device_info adl_p_info = {
GEN12_FEATURES,
XE_LPD_FEATURES,
PLATFORM(INTEL_ALDERLAKE_P),
- .require_force_probe = 1,
+ .display.cpu_transcoder_mask = BIT(TRANSCODER_A) | BIT(TRANSCODER_B) |
+ BIT(TRANSCODER_C) | BIT(TRANSCODER_D) |
+ BIT(TRANSCODER_DSI_0) | BIT(TRANSCODER_DSI_1),
.display.has_cdclk_crawl = 1,
.display.has_modular_fia = 1,
.display.has_psr_hw_tracking = 0,
@@ -986,8 +998,8 @@ static const struct intel_device_info adl_p_info = {
I915_GTT_PAGE_SIZE_2M
#define XE_HP_FEATURES \
- .graphics_ver = 12, \
- .graphics_rel = 50, \
+ .graphics.ver = 12, \
+ .graphics.rel = 50, \
XE_HP_PAGE_SIZES, \
.dma_mask_size = 46, \
.has_64bit_reloc = 1, \
@@ -1005,8 +1017,8 @@ static const struct intel_device_info adl_p_info = {
.ppgtt_type = INTEL_PPGTT_FULL
#define XE_HPM_FEATURES \
- .media_ver = 12, \
- .media_rel = 50
+ .media.ver = 12, \
+ .media.rel = 50
__maybe_unused
static const struct intel_device_info xehpsdv_info = {
@@ -1015,7 +1027,7 @@ static const struct intel_device_info xehpsdv_info = {
DGFX_FEATURES,
PLATFORM(INTEL_XEHPSDV),
.display = { },
- .pipe_mask = 0,
+ .has_64k_pages = 1,
.platform_engine_mask =
BIT(RCS0) | BIT(BCS0) |
BIT(VECS0) | BIT(VECS1) | BIT(VECS2) | BIT(VECS3) |
@@ -1030,14 +1042,17 @@ static const struct intel_device_info dg2_info = {
XE_HPM_FEATURES,
XE_LPD_FEATURES,
DGFX_FEATURES,
- .graphics_rel = 55,
- .media_rel = 55,
+ .graphics.rel = 55,
+ .media.rel = 55,
PLATFORM(INTEL_DG2),
+ .has_64k_pages = 1,
.platform_engine_mask =
BIT(RCS0) | BIT(BCS0) |
BIT(VECS0) | BIT(VECS1) |
BIT(VCS0) | BIT(VCS2),
.require_force_probe = 1,
+ .display.cpu_transcoder_mask = BIT(TRANSCODER_A) | BIT(TRANSCODER_B) |
+ BIT(TRANSCODER_C) | BIT(TRANSCODER_D),
};
#undef PLATFORM
@@ -1117,6 +1132,7 @@ static const struct pci_device_id pciidlist[] = {
INTEL_ADLS_IDS(&adl_s_info),
INTEL_ADLP_IDS(&adl_p_info),
INTEL_DG1_IDS(&dg1_info),
+ INTEL_RPLS_IDS(&adl_s_info),
{0, 0, 0}
};
MODULE_DEVICE_TABLE(pci, pciidlist);
@@ -1189,11 +1205,8 @@ static int i915_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
if (PCI_FUNC(pdev->devfn))
return -ENODEV;
- /*
- * apple-gmux is needed on dual GPU MacBook Pro
- * to probe the panel if we're the inactive GPU.
- */
- if (vga_switcheroo_client_probe_defer(pdev))
+ /* Detect if we need to wait for other drivers early on */
+ if (intel_modeset_probe_defer(pdev))
return -EPROBE_DEFER;
err = i915_driver_probe(pdev, ent);
diff --git a/drivers/gpu/drm/i915/i915_perf.c b/drivers/gpu/drm/i915/i915_perf.c
index 2f01b8c..170bba9 100644
--- a/drivers/gpu/drm/i915/i915_perf.c
+++ b/drivers/gpu/drm/i915/i915_perf.c
@@ -4443,7 +4443,7 @@ void i915_perf_init(struct drm_i915_private *i915)
mutex_init(&perf->lock);
/* Choose a representative limit */
- oa_sample_rate_hard_limit = i915->gt.clock_frequency / 2;
+ oa_sample_rate_hard_limit = to_gt(i915)->clock_frequency / 2;
mutex_init(&perf->metrics_lock);
idr_init_base(&perf->metrics_idr, 1);
diff --git a/drivers/gpu/drm/i915/i915_pmu.c b/drivers/gpu/drm/i915/i915_pmu.c
index 0b488d4..ea65516 100644
--- a/drivers/gpu/drm/i915/i915_pmu.c
+++ b/drivers/gpu/drm/i915/i915_pmu.c
@@ -210,8 +210,8 @@ static void init_rc6(struct i915_pmu *pmu)
struct drm_i915_private *i915 = container_of(pmu, typeof(*i915), pmu);
intel_wakeref_t wakeref;
- with_intel_runtime_pm(i915->gt.uncore->rpm, wakeref) {
- pmu->sample[__I915_SAMPLE_RC6].cur = __get_rc6(&i915->gt);
+ with_intel_runtime_pm(to_gt(i915)->uncore->rpm, wakeref) {
+ pmu->sample[__I915_SAMPLE_RC6].cur = __get_rc6(to_gt(i915));
pmu->sample[__I915_SAMPLE_RC6_LAST_REPORTED].cur =
pmu->sample[__I915_SAMPLE_RC6].cur;
pmu->sleep_last = ktime_get_raw();
@@ -222,7 +222,7 @@ static void park_rc6(struct drm_i915_private *i915)
{
struct i915_pmu *pmu = &i915->pmu;
- pmu->sample[__I915_SAMPLE_RC6].cur = __get_rc6(&i915->gt);
+ pmu->sample[__I915_SAMPLE_RC6].cur = __get_rc6(to_gt(i915));
pmu->sleep_last = ktime_get_raw();
}
@@ -419,7 +419,7 @@ static enum hrtimer_restart i915_sample(struct hrtimer *hrtimer)
struct drm_i915_private *i915 =
container_of(hrtimer, struct drm_i915_private, pmu.timer);
struct i915_pmu *pmu = &i915->pmu;
- struct intel_gt *gt = &i915->gt;
+ struct intel_gt *gt = to_gt(i915);
unsigned int period_ns;
ktime_t now;
@@ -476,7 +476,7 @@ engine_event_status(struct intel_engine_cs *engine,
static int
config_status(struct drm_i915_private *i915, u64 config)
{
- struct intel_gt *gt = &i915->gt;
+ struct intel_gt *gt = to_gt(i915);
switch (config) {
case I915_PMU_ACTUAL_FREQUENCY:
@@ -601,10 +601,10 @@ static u64 __i915_pmu_event_read(struct perf_event *event)
val = READ_ONCE(pmu->irq_count);
break;
case I915_PMU_RC6_RESIDENCY:
- val = get_rc6(&i915->gt);
+ val = get_rc6(to_gt(i915));
break;
case I915_PMU_SOFTWARE_GT_AWAKE_TIME:
- val = ktime_to_ns(intel_gt_get_awake_time(&i915->gt));
+ val = ktime_to_ns(intel_gt_get_awake_time(to_gt(i915)));
break;
}
}
diff --git a/drivers/gpu/drm/i915/i915_query.c b/drivers/gpu/drm/i915/i915_query.c
index 51b368b..2dfbc22 100644
--- a/drivers/gpu/drm/i915/i915_query.c
+++ b/drivers/gpu/drm/i915/i915_query.c
@@ -31,7 +31,7 @@ static int copy_query_item(void *query_hdr, size_t query_sz,
static int query_topology_info(struct drm_i915_private *dev_priv,
struct drm_i915_query_item *query_item)
{
- const struct sseu_dev_info *sseu = &dev_priv->gt.info.sseu;
+ const struct sseu_dev_info *sseu = &to_gt(dev_priv)->info.sseu;
struct drm_i915_query_topology_info topo;
u32 slice_length, subslice_length, eu_length, total_length;
int ret;
diff --git a/drivers/gpu/drm/i915/i915_reg.h b/drivers/gpu/drm/i915/i915_reg.h
index bcee121..4c28dad 100644
--- a/drivers/gpu/drm/i915/i915_reg.h
+++ b/drivers/gpu/drm/i915/i915_reg.h
@@ -371,6 +371,9 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)
#define VLV_G3DCTL _MMIO(0x9024)
#define VLV_GSCKGCTL _MMIO(0x9028)
+#define FBC_LLC_READ_CTRL _MMIO(0x9044)
+#define FBC_LLC_FULLY_OPEN REG_BIT(30)
+
#define GEN6_MBCTL _MMIO(0x0907c)
#define GEN6_MBCTL_ENABLE_BOOT_FETCH (1 << 4)
#define GEN6_MBCTL_CTX_FETCH_NEEDED (1 << 3)
@@ -498,6 +501,18 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)
#define ECOBITS_PPGTT_CACHE64B (3 << 8)
#define ECOBITS_PPGTT_CACHE4B (0 << 8)
+#define GEN12_GAMCNTRL_CTRL _MMIO(0xcf54)
+#define INVALIDATION_BROADCAST_MODE_DIS REG_BIT(12)
+#define GLOBAL_INVALIDATION_MODE REG_BIT(2)
+
+#define GEN12_GAMSTLB_CTRL _MMIO(0xcf4c)
+#define CONTROL_BLOCK_CLKGATE_DIS REG_BIT(12)
+#define EGRESS_BLOCK_CLKGATE_DIS REG_BIT(11)
+#define TAG_BLOCK_CLKGATE_DIS REG_BIT(7)
+
+#define GEN12_MERT_MOD_CTRL _MMIO(0xcf28)
+#define FORCE_MISS_FTLB REG_BIT(3)
+
#define GAB_CTL _MMIO(0x24000)
#define GAB_CTL_CONT_AFTER_PAGEFAULT (1 << 8)
@@ -719,6 +734,9 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)
#define GEN12_OA_TLB_INV_CR _MMIO(0xceec)
+#define GEN12_SQCM _MMIO(0x8724)
+#define EN_32B_ACCESS REG_BIT(30)
+
/* Gen12 OAR unit */
#define GEN12_OAR_OACONTROL _MMIO(0x2960)
#define GEN12_OAR_OACONTROL_COUNTER_FORMAT_SHIFT 1
@@ -770,6 +788,9 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)
#define EU_PERF_CNTL5 _MMIO(0xe55c)
#define EU_PERF_CNTL6 _MMIO(0xe65c)
+#define RT_CTRL _MMIO(0xe530)
+#define DIS_NULL_QUERY REG_BIT(10)
+
/*
* OA Boolean state
*/
@@ -2244,6 +2265,7 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)
#define SNPS_PHY_MPLLB_DP2_MODE REG_BIT(9)
#define SNPS_PHY_MPLLB_WORD_DIV2_EN REG_BIT(8)
#define SNPS_PHY_MPLLB_TX_CLK_DIV REG_GENMASK(7, 5)
+#define SNPS_PHY_MPLLB_SHIM_DIV32_CLK_SEL REG_BIT(0)
#define SNPS_PHY_MPLLB_FRACN1(phy) _MMIO_SNPS(phy, 0x168008)
#define SNPS_PHY_MPLLB_FRACN_EN REG_BIT(31)
@@ -2662,6 +2684,8 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)
#define RING_WAIT (1 << 11) /* gen3+, PRBx_CTL */
#define RING_WAIT_SEMAPHORE (1 << 10) /* gen6+ */
+#define GUCPMTIMESTAMP _MMIO(0xC3E8)
+
/* There are 16 64-bit CS General Purpose Registers per-engine on Gen8+ */
#define GEN8_RING_CS_GPR(base, n) _MMIO((base) + 0x600 + (n) * 8)
#define GEN8_RING_CS_GPR_UDW(base, n) _MMIO((base) + 0x600 + (n) * 8 + 4)
@@ -2772,6 +2796,9 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)
#define VDBOX_CGCTL3F10(base) _MMIO((base) + 0x3f10)
#define IECPUNIT_CLKGATE_DIS REG_BIT(22)
+#define VDBOX_CGCTL3F18(base) _MMIO((base) + 0x3f18)
+#define ALNUNIT_CLKGATE_DIS REG_BIT(13)
+
#define ERROR_GEN6 _MMIO(0x40a0)
#define GEN7_ERR_INT _MMIO(0x44040)
#define ERR_INT_POISON (1 << 31)
@@ -2795,12 +2822,12 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)
#define GEN12_AUX_ERR_DBG _MMIO(0x43f4)
#define FPGA_DBG _MMIO(0x42300)
-#define FPGA_DBG_RM_NOCLAIM (1 << 31)
+#define FPGA_DBG_RM_NOCLAIM REG_BIT(31)
#define CLAIM_ER _MMIO(VLV_DISPLAY_BASE + 0x2028)
-#define CLAIM_ER_CLR (1 << 31)
-#define CLAIM_ER_OVERFLOW (1 << 16)
-#define CLAIM_ER_CTR_MASK 0xffff
+#define CLAIM_ER_CLR REG_BIT(31)
+#define CLAIM_ER_OVERFLOW REG_BIT(16)
+#define CLAIM_ER_CTR_MASK REG_GENMASK(15, 0)
#define DERRMR _MMIO(0x44050)
/* Note that HBLANK events are reserved on bdw+ */
@@ -2870,6 +2897,15 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)
#define GEN9_PREEMPT_GPGPU_SYNC_SWITCH_DISABLE (1 << 2)
#define GEN11_ENABLE_32_PLANE_MODE (1 << 7)
+#define SCCGCTL94DC _MMIO(0x94dc)
+#define CG3DDISURB REG_BIT(14)
+
+#define MLTICTXCTL _MMIO(0xb170)
+#define TDONRENDER REG_BIT(2)
+
+#define L3SQCREG1_CCS0 _MMIO(0xb200)
+#define FLUSHALLNONCOH REG_BIT(5)
+
/* WaClearTdlStateAckDirtyBits */
#define GEN8_STATE_ACK _MMIO(0x20F0)
#define GEN9_STATE_ACK_SLICE1 _MMIO(0x20F8)
@@ -3106,7 +3142,8 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)
#define GEN9_RCS_FE_FSM2 _MMIO(0x22a4)
#define GEN10_CACHE_MODE_SS _MMIO(0xe420)
-#define FLOAT_BLEND_OPTIMIZATION_ENABLE (1 << 4)
+#define ENABLE_PREFETCH_INTO_IC REG_BIT(3)
+#define FLOAT_BLEND_OPTIMIZATION_ENABLE REG_BIT(4)
/* Fuse readout registers for GT */
#define HSW_PAVP_FUSE1 _MMIO(0x911C)
@@ -3307,93 +3344,98 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)
#define FBC_CFB_BASE _MMIO(0x3200) /* 4k page aligned */
#define FBC_LL_BASE _MMIO(0x3204) /* 4k page aligned */
#define FBC_CONTROL _MMIO(0x3208)
-#define FBC_CTL_EN REG_BIT(31)
-#define FBC_CTL_PERIODIC REG_BIT(30)
-#define FBC_CTL_INTERVAL_MASK REG_GENMASK(29, 16)
-#define FBC_CTL_INTERVAL(x) REG_FIELD_PREP(FBC_CTL_INTERVAL_MASK, (x))
-#define FBC_CTL_STOP_ON_MOD REG_BIT(15)
-#define FBC_CTL_UNCOMPRESSIBLE REG_BIT(14) /* i915+ */
-#define FBC_CTL_C3_IDLE REG_BIT(13) /* i945gm */
-#define FBC_CTL_STRIDE_MASK REG_GENMASK(12, 5)
-#define FBC_CTL_STRIDE(x) REG_FIELD_PREP(FBC_CTL_STRIDE_MASK, (x))
-#define FBC_CTL_FENCENO_MASK REG_GENMASK(3, 0)
-#define FBC_CTL_FENCENO(x) REG_FIELD_PREP(FBC_CTL_FENCENO_MASK, (x))
+#define FBC_CTL_EN REG_BIT(31)
+#define FBC_CTL_PERIODIC REG_BIT(30)
+#define FBC_CTL_INTERVAL_MASK REG_GENMASK(29, 16)
+#define FBC_CTL_INTERVAL(x) REG_FIELD_PREP(FBC_CTL_INTERVAL_MASK, (x))
+#define FBC_CTL_STOP_ON_MOD REG_BIT(15)
+#define FBC_CTL_UNCOMPRESSIBLE REG_BIT(14) /* i915+ */
+#define FBC_CTL_C3_IDLE REG_BIT(13) /* i945gm only */
+#define FBC_CTL_STRIDE_MASK REG_GENMASK(12, 5)
+#define FBC_CTL_STRIDE(x) REG_FIELD_PREP(FBC_CTL_STRIDE_MASK, (x))
+#define FBC_CTL_FENCENO_MASK REG_GENMASK(3, 0)
+#define FBC_CTL_FENCENO(x) REG_FIELD_PREP(FBC_CTL_FENCENO_MASK, (x))
#define FBC_COMMAND _MMIO(0x320c)
-#define FBC_CMD_COMPRESS (1 << 0)
+#define FBC_CMD_COMPRESS REG_BIT(0)
#define FBC_STATUS _MMIO(0x3210)
-#define FBC_STAT_COMPRESSING (1 << 31)
-#define FBC_STAT_COMPRESSED (1 << 30)
-#define FBC_STAT_MODIFIED (1 << 29)
-#define FBC_STAT_CURRENT_LINE_SHIFT (0)
-#define FBC_CONTROL2 _MMIO(0x3214)
-#define FBC_CTL_FENCE_DBL (0 << 4)
-#define FBC_CTL_IDLE_IMM (0 << 2)
-#define FBC_CTL_IDLE_FULL (1 << 2)
-#define FBC_CTL_IDLE_LINE (2 << 2)
-#define FBC_CTL_IDLE_DEBUG (3 << 2)
-#define FBC_CTL_CPU_FENCE (1 << 1)
-#define FBC_CTL_PLANE(plane) ((plane) << 0)
-#define FBC_FENCE_OFF _MMIO(0x3218) /* BSpec typo has 321Bh */
-#define FBC_TAG(i) _MMIO(0x3300 + (i) * 4)
+#define FBC_STAT_COMPRESSING REG_BIT(31)
+#define FBC_STAT_COMPRESSED REG_BIT(30)
+#define FBC_STAT_MODIFIED REG_BIT(29)
+#define FBC_STAT_CURRENT_LINE_MASK REG_GENMASK(10, 0)
+#define FBC_CONTROL2 _MMIO(0x3214) /* i965gm only */
+#define FBC_CTL_FENCE_DBL REG_BIT(4)
+#define FBC_CTL_IDLE_MASK REG_GENMASK(3, 2)
+#define FBC_CTL_IDLE_IMM REG_FIELD_PREP(FBC_CTL_IDLE_MASK, 0)
+#define FBC_CTL_IDLE_FULL REG_FIELD_PREP(FBC_CTL_IDLE_MASK, 1)
+#define FBC_CTL_IDLE_LINE REG_FIELD_PREP(FBC_CTL_IDLE_MASK, 2)
+#define FBC_CTL_IDLE_DEBUG REG_FIELD_PREP(FBC_CTL_IDLE_MASK, 3)
+#define FBC_CTL_CPU_FENCE_EN REG_BIT(1)
+#define FBC_CTL_PLANE_MASK REG_GENMASK(1, 0)
+#define FBC_CTL_PLANE(i9xx_plane) REG_FIELD_PREP(FBC_CTL_PLANE_MASK, (i9xx_plane))
+#define FBC_FENCE_OFF _MMIO(0x3218) /* i965gm only, BSpec typo has 321Bh */
+#define FBC_MOD_NUM _MMIO(0x3220) /* i965gm only */
+#define FBC_MOD_NUM_MASK REG_GENMASK(31, 1)
+#define FBC_MOD_NUM_VALID REG_BIT(0)
+#define FBC_TAG(i) _MMIO(0x3300 + (i) * 4) /* 49 reisters */
+#define FBC_TAG_MASK REG_GENMASK(1, 0) /* 16 tags per register */
+#define FBC_TAG_MODIFIED REG_FIELD_PREP(FBC_TAG_MASK, 0)
+#define FBC_TAG_UNCOMPRESSED REG_FIELD_PREP(FBC_TAG_MASK, 1)
+#define FBC_TAG_UNCOMPRESSIBLE REG_FIELD_PREP(FBC_TAG_MASK, 2)
+#define FBC_TAG_COMPRESSED REG_FIELD_PREP(FBC_TAG_MASK, 3)
#define FBC_LL_SIZE (1536)
-#define FBC_LLC_READ_CTRL _MMIO(0x9044)
-#define FBC_LLC_FULLY_OPEN (1 << 30)
-
/* Framebuffer compression for GM45+ */
#define DPFC_CB_BASE _MMIO(0x3200)
-#define DPFC_CONTROL _MMIO(0x3208)
-#define DPFC_CTL_EN (1 << 31)
-#define DPFC_CTL_PLANE(plane) ((plane) << 30)
-#define IVB_DPFC_CTL_PLANE(plane) ((plane) << 29)
-#define DPFC_CTL_FENCE_EN (1 << 29)
-#define IVB_DPFC_CTL_FENCE_EN (1 << 28)
-#define DPFC_CTL_PERSISTENT_MODE (1 << 25)
-#define DPFC_SR_EN (1 << 10)
-#define DPFC_CTL_LIMIT_1X (0 << 6)
-#define DPFC_CTL_LIMIT_2X (1 << 6)
-#define DPFC_CTL_LIMIT_4X (2 << 6)
-#define DPFC_RECOMP_CTL _MMIO(0x320c)
-#define DPFC_RECOMP_STALL_EN (1 << 27)
-#define DPFC_RECOMP_STALL_WM_SHIFT (16)
-#define DPFC_RECOMP_STALL_WM_MASK (0x07ff0000)
-#define DPFC_RECOMP_TIMER_COUNT_SHIFT (0)
-#define DPFC_RECOMP_TIMER_COUNT_MASK (0x0000003f)
-#define DPFC_STATUS _MMIO(0x3210)
-#define DPFC_INVAL_SEG_SHIFT (16)
-#define DPFC_INVAL_SEG_MASK (0x07ff0000)
-#define DPFC_COMP_SEG_SHIFT (0)
-#define DPFC_COMP_SEG_MASK (0x000007ff)
-#define DPFC_STATUS2 _MMIO(0x3214)
-#define DPFC_FENCE_YOFF _MMIO(0x3218)
-#define DPFC_CHICKEN _MMIO(0x3224)
-#define DPFC_HT_MODIFY (1 << 31)
-
-/* Framebuffer compression for Ironlake */
#define ILK_DPFC_CB_BASE _MMIO(0x43200)
+#define DPFC_CONTROL _MMIO(0x3208)
#define ILK_DPFC_CONTROL _MMIO(0x43208)
-#define FBC_CTL_FALSE_COLOR (1 << 10)
-/* The bit 28-8 is reserved */
-#define DPFC_RESERVED (0x1FFFFF00)
+#define DPFC_CTL_EN REG_BIT(31)
+#define DPFC_CTL_PLANE_MASK_G4X REG_BIT(30) /* g4x-snb */
+#define DPFC_CTL_PLANE_G4X(i9xx_plane) REG_FIELD_PREP(DPFC_CTL_PLANE_MASK_G4X, (i9xx_plane))
+#define DPFC_CTL_FENCE_EN_G4X REG_BIT(29) /* g4x-snb */
+#define DPFC_CTL_PLANE_MASK_IVB REG_GENMASK(30, 29) /* ivb only */
+#define DPFC_CTL_PLANE_IVB(i9xx_plane) REG_FIELD_PREP(DPFC_CTL_PLANE_MASK_IVB, (i9xx_plane))
+#define DPFC_CTL_FENCE_EN_IVB REG_BIT(28) /* ivb+ */
+#define DPFC_CTL_PERSISTENT_MODE REG_BIT(25) /* g4x-snb */
+#define DPFC_CTL_FALSE_COLOR REG_BIT(10) /* ivb+ */
+#define DPFC_CTL_SR_EN REG_BIT(10) /* g4x only */
+#define DPFC_CTL_SR_EXIT_DIS REG_BIT(9) /* g4x only */
+#define DPFC_CTL_LIMIT_MASK REG_GENMASK(7, 6)
+#define DPFC_CTL_LIMIT_1X REG_FIELD_PREP(DPFC_CTL_LIMIT_MASK, 0)
+#define DPFC_CTL_LIMIT_2X REG_FIELD_PREP(DPFC_CTL_LIMIT_MASK, 1)
+#define DPFC_CTL_LIMIT_4X REG_FIELD_PREP(DPFC_CTL_LIMIT_MASK, 2)
+#define DPFC_CTL_FENCENO_MASK REG_GENMASK(3, 0)
+#define DPFC_CTL_FENCENO(fence) REG_FIELD_PREP(DPFC_CTL_FENCENO_MASK, (fence))
+#define DPFC_RECOMP_CTL _MMIO(0x320c)
#define ILK_DPFC_RECOMP_CTL _MMIO(0x4320c)
+#define DPFC_RECOMP_STALL_EN REG_BIT(27)
+#define DPFC_RECOMP_STALL_WM_MASK REG_GENMASK(26, 16)
+#define DPFC_RECOMP_TIMER_COUNT_MASK REG_GENMASK(5, 0)
+#define DPFC_STATUS _MMIO(0x3210)
#define ILK_DPFC_STATUS _MMIO(0x43210)
-#define ILK_DPFC_COMP_SEG_MASK 0x7ff
-#define IVB_FBC_STATUS2 _MMIO(0x43214)
-#define IVB_FBC_COMP_SEG_MASK 0x7ff
-#define BDW_FBC_COMP_SEG_MASK 0xfff
+#define DPFC_INVAL_SEG_MASK REG_GENMASK(26, 16)
+#define DPFC_COMP_SEG_MASK REG_GENMASK(10, 0)
+#define DPFC_STATUS2 _MMIO(0x3214)
+#define ILK_DPFC_STATUS2 _MMIO(0x43214)
+#define DPFC_COMP_SEG_MASK_IVB REG_GENMASK(11, 0)
+#define DPFC_FENCE_YOFF _MMIO(0x3218)
#define ILK_DPFC_FENCE_YOFF _MMIO(0x43218)
+#define DPFC_CHICKEN _MMIO(0x3224)
#define ILK_DPFC_CHICKEN _MMIO(0x43224)
-#define ILK_DPFC_DISABLE_DUMMY0 (1 << 8)
-#define ILK_DPFC_CHICKEN_COMP_DUMMY_PIXEL (1 << 14)
-#define ILK_DPFC_NUKE_ON_ANY_MODIFICATION (1 << 23)
+#define DPFC_HT_MODIFY REG_BIT(31) /* pre-ivb */
+#define DPFC_NUKE_ON_ANY_MODIFICATION REG_BIT(23) /* bdw+ */
+#define DPFC_CHICKEN_COMP_DUMMY_PIXEL REG_BIT(14) /* glk+ */
+#define DPFC_DISABLE_DUMMY0 REG_BIT(8) /* ivb+ */
+
#define GLK_FBC_STRIDE _MMIO(0x43228)
#define FBC_STRIDE_OVERRIDE REG_BIT(15)
#define FBC_STRIDE_MASK REG_GENMASK(14, 0)
#define FBC_STRIDE(x) REG_FIELD_PREP(FBC_STRIDE_MASK, (x))
+
#define ILK_FBC_RT_BASE _MMIO(0x2128)
-#define ILK_FBC_RT_VALID (1 << 0)
-#define SNB_FBC_FRONT_BUFFER (1 << 1)
+#define ILK_FBC_RT_VALID REG_BIT(0)
+#define SNB_FBC_FRONT_BUFFER REG_BIT(1)
#define ILK_DISPLAY_CHICKEN1 _MMIO(0x42000)
#define ILK_FBCQ_DIS (1 << 22)
@@ -3417,8 +3459,10 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)
* The following two registers are of type GTTMMADR
*/
#define SNB_DPFC_CTL_SA _MMIO(0x100100)
-#define SNB_CPU_FENCE_ENABLE (1 << 29)
-#define DPFC_CPU_FENCE_OFFSET _MMIO(0x100104)
+#define SNB_DPFC_FENCE_EN REG_BIT(29)
+#define SNB_DPFC_FENCENO_MASK REG_GENMASK(4, 0)
+#define SNB_DPFC_FENCENO(fence) REG_FIELD_PREP(SNB_DPFC_FENCENO_MASK, (fence))
+#define SNB_DPFC_CPU_FENCE_OFFSET _MMIO(0x100104)
/* Framebuffer compression for Ivybridge */
#define IVB_FBC_RT_BASE _MMIO(0x7020)
@@ -3428,8 +3472,8 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)
#define IPS_ENABLE (1 << 31)
#define MSG_FBC_REND_STATE _MMIO(0x50380)
-#define FBC_REND_NUKE (1 << 2)
-#define FBC_REND_CACHE_CLEAN (1 << 1)
+#define FBC_REND_NUKE REG_BIT(2)
+#define FBC_REND_CACHE_CLEAN REG_BIT(1)
/*
* GPIO regs
@@ -4278,21 +4322,62 @@ enum {
/*
* GEN10 clock gating regs
*/
+
+#define UNSLCGCTL9440 _MMIO(0x9440)
+#define GAMTLBOACS_CLKGATE_DIS REG_BIT(28)
+#define GAMTLBVDBOX5_CLKGATE_DIS REG_BIT(27)
+#define GAMTLBVDBOX6_CLKGATE_DIS REG_BIT(26)
+#define GAMTLBVDBOX3_CLKGATE_DIS REG_BIT(24)
+#define GAMTLBVDBOX4_CLKGATE_DIS REG_BIT(23)
+#define GAMTLBVDBOX7_CLKGATE_DIS REG_BIT(22)
+#define GAMTLBVDBOX2_CLKGATE_DIS REG_BIT(21)
+#define GAMTLBVDBOX0_CLKGATE_DIS REG_BIT(17)
+#define GAMTLBKCR_CLKGATE_DIS REG_BIT(16)
+#define GAMTLBGUC_CLKGATE_DIS REG_BIT(15)
+#define GAMTLBBLT_CLKGATE_DIS REG_BIT(14)
+#define GAMTLBVDBOX1_CLKGATE_DIS REG_BIT(6)
+
+#define UNSLCGCTL9444 _MMIO(0x9444)
+#define GAMTLBGFXA0_CLKGATE_DIS REG_BIT(30)
+#define GAMTLBGFXA1_CLKGATE_DIS REG_BIT(29)
+#define GAMTLBCOMPA0_CLKGATE_DIS REG_BIT(28)
+#define GAMTLBCOMPA1_CLKGATE_DIS REG_BIT(27)
+#define GAMTLBCOMPB0_CLKGATE_DIS REG_BIT(26)
+#define GAMTLBCOMPB1_CLKGATE_DIS REG_BIT(25)
+#define GAMTLBCOMPC0_CLKGATE_DIS REG_BIT(24)
+#define GAMTLBCOMPC1_CLKGATE_DIS REG_BIT(23)
+#define GAMTLBCOMPD0_CLKGATE_DIS REG_BIT(22)
+#define GAMTLBCOMPD1_CLKGATE_DIS REG_BIT(21)
+#define GAMTLBMERT_CLKGATE_DIS REG_BIT(20)
+#define GAMTLBVEBOX3_CLKGATE_DIS REG_BIT(19)
+#define GAMTLBVEBOX2_CLKGATE_DIS REG_BIT(18)
+#define GAMTLBVEBOX1_CLKGATE_DIS REG_BIT(17)
+#define GAMTLBVEBOX0_CLKGATE_DIS REG_BIT(16)
+#define LTCDD_CLKGATE_DIS REG_BIT(10)
+
#define SLICE_UNIT_LEVEL_CLKGATE _MMIO(0x94d4)
#define SARBUNIT_CLKGATE_DIS (1 << 5)
#define RCCUNIT_CLKGATE_DIS (1 << 7)
#define MSCUNIT_CLKGATE_DIS (1 << 10)
+#define NODEDSS_CLKGATE_DIS REG_BIT(12)
#define L3_CLKGATE_DIS REG_BIT(16)
#define L3_CR2X_CLKGATE_DIS REG_BIT(17)
#define SUBSLICE_UNIT_LEVEL_CLKGATE _MMIO(0x9524)
-#define GWUNIT_CLKGATE_DIS (1 << 16)
+#define DSS_ROUTER_CLKGATE_DIS REG_BIT(28)
+#define GWUNIT_CLKGATE_DIS REG_BIT(16)
#define SUBSLICE_UNIT_LEVEL_CLKGATE2 _MMIO(0x9528)
#define CPSSUNIT_CLKGATE_DIS REG_BIT(9)
+#define SSMCGCTL9530 _MMIO(0x9530)
+#define RTFUNIT_CLKGATE_DIS REG_BIT(18)
+
#define UNSLICE_UNIT_LEVEL_CLKGATE _MMIO(0x9434)
#define VFUNIT_CLKGATE_DIS REG_BIT(20)
+#define TSGUNIT_CLKGATE_DIS REG_BIT(17) /* XEHPSDV */
+#define CG3DDISCFEG_CLKGATE_DIS REG_BIT(17) /* DG2 */
+#define GAMEDIA_CLKGATE_DIS REG_BIT(11)
#define HSUNIT_CLKGATE_DIS REG_BIT(8)
#define VSUNIT_CLKGATE_DIS REG_BIT(3)
@@ -4309,47 +4394,52 @@ enum {
/* Pipe A CRC regs */
#define _PIPE_CRC_CTL_A 0x60050
-#define PIPE_CRC_ENABLE (1 << 31)
+#define PIPE_CRC_ENABLE REG_BIT(31)
/* skl+ source selection */
-#define PIPE_CRC_SOURCE_PLANE_1_SKL (0 << 28)
-#define PIPE_CRC_SOURCE_PLANE_2_SKL (2 << 28)
-#define PIPE_CRC_SOURCE_DMUX_SKL (4 << 28)
-#define PIPE_CRC_SOURCE_PLANE_3_SKL (6 << 28)
-#define PIPE_CRC_SOURCE_PLANE_4_SKL (7 << 28)
-#define PIPE_CRC_SOURCE_PLANE_5_SKL (5 << 28)
-#define PIPE_CRC_SOURCE_PLANE_6_SKL (3 << 28)
-#define PIPE_CRC_SOURCE_PLANE_7_SKL (1 << 28)
+#define PIPE_CRC_SOURCE_MASK_SKL REG_GENMASK(30, 28)
+#define PIPE_CRC_SOURCE_PLANE_1_SKL REG_FIELD_PREP(PIPE_CRC_SOURCE_MASK_SKL, 0)
+#define PIPE_CRC_SOURCE_PLANE_2_SKL REG_FIELD_PREP(PIPE_CRC_SOURCE_MASK_SKL, 2)
+#define PIPE_CRC_SOURCE_DMUX_SKL REG_FIELD_PREP(PIPE_CRC_SOURCE_MASK_SKL, 4)
+#define PIPE_CRC_SOURCE_PLANE_3_SKL REG_FIELD_PREP(PIPE_CRC_SOURCE_MASK_SKL, 6)
+#define PIPE_CRC_SOURCE_PLANE_4_SKL REG_FIELD_PREP(PIPE_CRC_SOURCE_MASK_SKL, 7)
+#define PIPE_CRC_SOURCE_PLANE_5_SKL REG_FIELD_PREP(PIPE_CRC_SOURCE_MASK_SKL, 5)
+#define PIPE_CRC_SOURCE_PLANE_6_SKL REG_FIELD_PREP(PIPE_CRC_SOURCE_MASK_SKL, 3)
+#define PIPE_CRC_SOURCE_PLANE_7_SKL REG_FIELD_PREP(PIPE_CRC_SOURCE_MASK_SKL, 1)
/* ivb+ source selection */
-#define PIPE_CRC_SOURCE_PRIMARY_IVB (0 << 29)
-#define PIPE_CRC_SOURCE_SPRITE_IVB (1 << 29)
-#define PIPE_CRC_SOURCE_PF_IVB (2 << 29)
+#define PIPE_CRC_SOURCE_MASK_IVB REG_GENMASK(30, 29)
+#define PIPE_CRC_SOURCE_PRIMARY_IVB REG_FIELD_PREP(PIPE_CRC_SOURCE_MASK_IVB, 0)
+#define PIPE_CRC_SOURCE_SPRITE_IVB REG_FIELD_PREP(PIPE_CRC_SOURCE_MASK_IVB, 1)
+#define PIPE_CRC_SOURCE_PF_IVB REG_FIELD_PREP(PIPE_CRC_SOURCE_MASK_IVB, 2)
/* ilk+ source selection */
-#define PIPE_CRC_SOURCE_PRIMARY_ILK (0 << 28)
-#define PIPE_CRC_SOURCE_SPRITE_ILK (1 << 28)
-#define PIPE_CRC_SOURCE_PIPE_ILK (2 << 28)
-/* embedded DP port on the north display block, reserved on ivb */
-#define PIPE_CRC_SOURCE_PORT_A_ILK (4 << 28)
-#define PIPE_CRC_SOURCE_FDI_ILK (5 << 28) /* reserved on ivb */
+#define PIPE_CRC_SOURCE_MASK_ILK REG_GENMASK(30, 28)
+#define PIPE_CRC_SOURCE_PRIMARY_ILK REG_FIELD_PREP(PIPE_CRC_SOURCE_MASK_ILK, 0)
+#define PIPE_CRC_SOURCE_SPRITE_ILK REG_FIELD_PREP(PIPE_CRC_SOURCE_MASK_ILK, 1)
+#define PIPE_CRC_SOURCE_PIPE_ILK REG_FIELD_PREP(PIPE_CRC_SOURCE_MASK_ILK, 2)
+/* embedded DP port on the north display block */
+#define PIPE_CRC_SOURCE_PORT_A_ILK REG_FIELD_PREP(PIPE_CRC_SOURCE_MASK_ILK, 4)
+#define PIPE_CRC_SOURCE_FDI_ILK REG_FIELD_PREP(PIPE_CRC_SOURCE_MASK_ILK, 5)
/* vlv source selection */
-#define PIPE_CRC_SOURCE_PIPE_VLV (0 << 27)
-#define PIPE_CRC_SOURCE_HDMIB_VLV (1 << 27)
-#define PIPE_CRC_SOURCE_HDMIC_VLV (2 << 27)
+#define PIPE_CRC_SOURCE_MASK_VLV REG_GENMASK(30, 27)
+#define PIPE_CRC_SOURCE_PIPE_VLV REG_FIELD_PREP(PIPE_CRC_SOURCE_MASK_VLV, 0)
+#define PIPE_CRC_SOURCE_HDMIB_VLV REG_FIELD_PREP(PIPE_CRC_SOURCE_MASK_VLV, 1)
+#define PIPE_CRC_SOURCE_HDMIC_VLV REG_FIELD_PREP(PIPE_CRC_SOURCE_MASK_VLV, 2)
/* with DP port the pipe source is invalid */
-#define PIPE_CRC_SOURCE_DP_D_VLV (3 << 27)
-#define PIPE_CRC_SOURCE_DP_B_VLV (6 << 27)
-#define PIPE_CRC_SOURCE_DP_C_VLV (7 << 27)
+#define PIPE_CRC_SOURCE_DP_D_VLV REG_FIELD_PREP(PIPE_CRC_SOURCE_MASK_VLV, 3)
+#define PIPE_CRC_SOURCE_DP_B_VLV REG_FIELD_PREP(PIPE_CRC_SOURCE_MASK_VLV, 6)
+#define PIPE_CRC_SOURCE_DP_C_VLV REG_FIELD_PREP(PIPE_CRC_SOURCE_MASK_VLV, 7)
/* gen3+ source selection */
-#define PIPE_CRC_SOURCE_PIPE_I9XX (0 << 28)
-#define PIPE_CRC_SOURCE_SDVOB_I9XX (1 << 28)
-#define PIPE_CRC_SOURCE_SDVOC_I9XX (2 << 28)
+#define PIPE_CRC_SOURCE_MASK_I9XX REG_GENMASK(30, 28)
+#define PIPE_CRC_SOURCE_PIPE_I9XX REG_FIELD_PREP(PIPE_CRC_SOURCE_MASK_I9XX, 0)
+#define PIPE_CRC_SOURCE_SDVOB_I9XX REG_FIELD_PREP(PIPE_CRC_SOURCE_MASK_I9XX, 1)
+#define PIPE_CRC_SOURCE_SDVOC_I9XX REG_FIELD_PREP(PIPE_CRC_SOURCE_MASK_I9XX, 2)
/* with DP/TV port the pipe source is invalid */
-#define PIPE_CRC_SOURCE_DP_D_G4X (3 << 28)
-#define PIPE_CRC_SOURCE_TV_PRE (4 << 28)
-#define PIPE_CRC_SOURCE_TV_POST (5 << 28)
-#define PIPE_CRC_SOURCE_DP_B_G4X (6 << 28)
-#define PIPE_CRC_SOURCE_DP_C_G4X (7 << 28)
+#define PIPE_CRC_SOURCE_DP_D_G4X REG_FIELD_PREP(PIPE_CRC_SOURCE_MASK_I9XX, 3)
+#define PIPE_CRC_SOURCE_TV_PRE REG_FIELD_PREP(PIPE_CRC_SOURCE_MASK_I9XX, 4)
+#define PIPE_CRC_SOURCE_TV_POST REG_FIELD_PREP(PIPE_CRC_SOURCE_MASK_I9XX, 5)
+#define PIPE_CRC_SOURCE_DP_B_G4X REG_FIELD_PREP(PIPE_CRC_SOURCE_MASK_I9XX, 6)
+#define PIPE_CRC_SOURCE_DP_C_G4X REG_FIELD_PREP(PIPE_CRC_SOURCE_MASK_I9XX, 7)
/* gen2 doesn't have source selection bits */
-#define PIPE_CRC_INCLUDE_BORDER_I8XX (1 << 30)
+#define PIPE_CRC_INCLUDE_BORDER_I8XX REG_BIT(30)
#define _PIPE_CRC_RES_1_A_IVB 0x60064
#define _PIPE_CRC_RES_2_A_IVB 0x60068
@@ -4698,11 +4788,11 @@ enum {
#define PSR_EVENT_LPSP_MODE_EXIT (1 << 1)
#define PSR_EVENT_PSR_DISABLE (1 << 0)
-#define _PSR2_STATUS_A 0x60940
-#define _PSR2_STATUS_EDP 0x6f940
-#define EDP_PSR2_STATUS(tran) _MMIO_TRANS2(tran, _PSR2_STATUS_A)
-#define EDP_PSR2_STATUS_STATE_MASK (0xf << 28)
-#define EDP_PSR2_STATUS_STATE_SHIFT 28
+#define _PSR2_STATUS_A 0x60940
+#define _PSR2_STATUS_EDP 0x6f940
+#define EDP_PSR2_STATUS(tran) _MMIO_TRANS2(tran, _PSR2_STATUS_A)
+#define EDP_PSR2_STATUS_STATE_MASK REG_GENMASK(31, 28)
+#define EDP_PSR2_STATUS_STATE_DEEP_SLEEP REG_FIELD_PREP(EDP_PSR2_STATUS_STATE_MASK, 0x8)
#define _PSR2_SU_STATUS_A 0x60914
#define _PSR2_SU_STATUS_EDP 0x6f914
@@ -4999,9 +5089,9 @@ enum {
#define PORT_DFT2_G4X _MMIO(DISPLAY_MMIO_BASE(dev_priv) + 0x61154)
#define DC_BALANCE_RESET_VLV (1 << 31)
#define PIPE_SCRAMBLE_RESET_MASK ((1 << 14) | (0x3 << 0))
-#define PIPE_C_SCRAMBLE_RESET (1 << 14) /* chv */
-#define PIPE_B_SCRAMBLE_RESET (1 << 1)
-#define PIPE_A_SCRAMBLE_RESET (1 << 0)
+#define PIPE_C_SCRAMBLE_RESET REG_BIT(14) /* chv */
+#define PIPE_B_SCRAMBLE_RESET REG_BIT(1)
+#define PIPE_A_SCRAMBLE_RESET REG_BIT(0)
/* Gen 3 SDVO bits: */
#define SDVO_ENABLE (1 << 31)
@@ -6266,55 +6356,55 @@ enum {
#define PIPE_STATUS_PORT_UNDERRUN_XELPD REG_BIT(26)
#define VLV_DPFLIPSTAT _MMIO(VLV_DISPLAY_BASE + 0x70028)
-#define PIPEB_LINE_COMPARE_INT_EN (1 << 29)
-#define PIPEB_HLINE_INT_EN (1 << 28)
-#define PIPEB_VBLANK_INT_EN (1 << 27)
-#define SPRITED_FLIP_DONE_INT_EN (1 << 26)
-#define SPRITEC_FLIP_DONE_INT_EN (1 << 25)
-#define PLANEB_FLIP_DONE_INT_EN (1 << 24)
-#define PIPE_PSR_INT_EN (1 << 22)
-#define PIPEA_LINE_COMPARE_INT_EN (1 << 21)
-#define PIPEA_HLINE_INT_EN (1 << 20)
-#define PIPEA_VBLANK_INT_EN (1 << 19)
-#define SPRITEB_FLIP_DONE_INT_EN (1 << 18)
-#define SPRITEA_FLIP_DONE_INT_EN (1 << 17)
-#define PLANEA_FLIPDONE_INT_EN (1 << 16)
-#define PIPEC_LINE_COMPARE_INT_EN (1 << 13)
-#define PIPEC_HLINE_INT_EN (1 << 12)
-#define PIPEC_VBLANK_INT_EN (1 << 11)
-#define SPRITEF_FLIPDONE_INT_EN (1 << 10)
-#define SPRITEE_FLIPDONE_INT_EN (1 << 9)
-#define PLANEC_FLIPDONE_INT_EN (1 << 8)
+#define PIPEB_LINE_COMPARE_INT_EN REG_BIT(29)
+#define PIPEB_HLINE_INT_EN REG_BIT(28)
+#define PIPEB_VBLANK_INT_EN REG_BIT(27)
+#define SPRITED_FLIP_DONE_INT_EN REG_BIT(26)
+#define SPRITEC_FLIP_DONE_INT_EN REG_BIT(25)
+#define PLANEB_FLIP_DONE_INT_EN REG_BIT(24)
+#define PIPE_PSR_INT_EN REG_BIT(22)
+#define PIPEA_LINE_COMPARE_INT_EN REG_BIT(21)
+#define PIPEA_HLINE_INT_EN REG_BIT(20)
+#define PIPEA_VBLANK_INT_EN REG_BIT(19)
+#define SPRITEB_FLIP_DONE_INT_EN REG_BIT(18)
+#define SPRITEA_FLIP_DONE_INT_EN REG_BIT(17)
+#define PLANEA_FLIPDONE_INT_EN REG_BIT(16)
+#define PIPEC_LINE_COMPARE_INT_EN REG_BIT(13)
+#define PIPEC_HLINE_INT_EN REG_BIT(12)
+#define PIPEC_VBLANK_INT_EN REG_BIT(11)
+#define SPRITEF_FLIPDONE_INT_EN REG_BIT(10)
+#define SPRITEE_FLIPDONE_INT_EN REG_BIT(9)
+#define PLANEC_FLIPDONE_INT_EN REG_BIT(8)
#define DPINVGTT _MMIO(VLV_DISPLAY_BASE + 0x7002c) /* VLV/CHV only */
-#define SPRITEF_INVALID_GTT_INT_EN (1 << 27)
-#define SPRITEE_INVALID_GTT_INT_EN (1 << 26)
-#define PLANEC_INVALID_GTT_INT_EN (1 << 25)
-#define CURSORC_INVALID_GTT_INT_EN (1 << 24)
-#define CURSORB_INVALID_GTT_INT_EN (1 << 23)
-#define CURSORA_INVALID_GTT_INT_EN (1 << 22)
-#define SPRITED_INVALID_GTT_INT_EN (1 << 21)
-#define SPRITEC_INVALID_GTT_INT_EN (1 << 20)
-#define PLANEB_INVALID_GTT_INT_EN (1 << 19)
-#define SPRITEB_INVALID_GTT_INT_EN (1 << 18)
-#define SPRITEA_INVALID_GTT_INT_EN (1 << 17)
-#define PLANEA_INVALID_GTT_INT_EN (1 << 16)
-#define DPINVGTT_EN_MASK 0xff0000
-#define DPINVGTT_EN_MASK_CHV 0xfff0000
-#define SPRITEF_INVALID_GTT_STATUS (1 << 11)
-#define SPRITEE_INVALID_GTT_STATUS (1 << 10)
-#define PLANEC_INVALID_GTT_STATUS (1 << 9)
-#define CURSORC_INVALID_GTT_STATUS (1 << 8)
-#define CURSORB_INVALID_GTT_STATUS (1 << 7)
-#define CURSORA_INVALID_GTT_STATUS (1 << 6)
-#define SPRITED_INVALID_GTT_STATUS (1 << 5)
-#define SPRITEC_INVALID_GTT_STATUS (1 << 4)
-#define PLANEB_INVALID_GTT_STATUS (1 << 3)
-#define SPRITEB_INVALID_GTT_STATUS (1 << 2)
-#define SPRITEA_INVALID_GTT_STATUS (1 << 1)
-#define PLANEA_INVALID_GTT_STATUS (1 << 0)
-#define DPINVGTT_STATUS_MASK 0xff
-#define DPINVGTT_STATUS_MASK_CHV 0xfff
+#define DPINVGTT_EN_MASK_CHV REG_GENMASK(27, 16)
+#define DPINVGTT_EN_MASK_VLV REG_GENMASK(23, 16)
+#define SPRITEF_INVALID_GTT_INT_EN REG_BIT(27)
+#define SPRITEE_INVALID_GTT_INT_EN REG_BIT(26)
+#define PLANEC_INVALID_GTT_INT_EN REG_BIT(25)
+#define CURSORC_INVALID_GTT_INT_EN REG_BIT(24)
+#define CURSORB_INVALID_GTT_INT_EN REG_BIT(23)
+#define CURSORA_INVALID_GTT_INT_EN REG_BIT(22)
+#define SPRITED_INVALID_GTT_INT_EN REG_BIT(21)
+#define SPRITEC_INVALID_GTT_INT_EN REG_BIT(20)
+#define PLANEB_INVALID_GTT_INT_EN REG_BIT(19)
+#define SPRITEB_INVALID_GTT_INT_EN REG_BIT(18)
+#define SPRITEA_INVALID_GTT_INT_EN REG_BIT(17)
+#define PLANEA_INVALID_GTT_INT_EN REG_BIT(16)
+#define DPINVGTT_STATUS_MASK_CHV REG_GENMASK(11, 0)
+#define DPINVGTT_STATUS_MASK_VLV REG_GENMASK(7, 0)
+#define SPRITEF_INVALID_GTT_STATUS REG_BIT(11)
+#define SPRITEE_INVALID_GTT_STATUS REG_BIT(10)
+#define PLANEC_INVALID_GTT_STATUS REG_BIT(9)
+#define CURSORC_INVALID_GTT_STATUS REG_BIT(8)
+#define CURSORB_INVALID_GTT_STATUS REG_BIT(7)
+#define CURSORA_INVALID_GTT_STATUS REG_BIT(6)
+#define SPRITED_INVALID_GTT_STATUS REG_BIT(5)
+#define SPRITEC_INVALID_GTT_STATUS REG_BIT(4)
+#define PLANEB_INVALID_GTT_STATUS REG_BIT(3)
+#define SPRITEB_INVALID_GTT_STATUS REG_BIT(2)
+#define SPRITEA_INVALID_GTT_STATUS REG_BIT(1)
+#define PLANEA_INVALID_GTT_STATUS REG_BIT(0)
#define DSPARB _MMIO(DISPLAY_MMIO_BASE(dev_priv) + 0x70030)
#define DSPARB_CSTART_MASK (0x7f << 7)
@@ -6877,7 +6967,7 @@ enum {
#define DVS_SOURCE_KEY (1 << 22)
#define DVS_RGB_ORDER_XBGR (1 << 20)
#define DVS_YUV_FORMAT_BT709 (1 << 18)
-#define DVS_YUV_BYTE_ORDER_MASK (3 << 16)
+#define DVS_YUV_ORDER_MASK (3 << 16)
#define DVS_YUV_ORDER_YUYV (0 << 16)
#define DVS_YUV_ORDER_UYVY (1 << 16)
#define DVS_YUV_ORDER_YVYU (2 << 16)
@@ -6956,7 +7046,7 @@ enum {
#define SPRITE_RGB_ORDER_RGBX (1 << 20) /* only for 888 and 161616 */
#define SPRITE_YUV_TO_RGB_CSC_DISABLE (1 << 19)
#define SPRITE_YUV_TO_RGB_CSC_FORMAT_BT709 (1 << 18) /* 0 is BT601 */
-#define SPRITE_YUV_BYTE_ORDER_MASK (3 << 16)
+#define SPRITE_YUV_ORDER_MASK (3 << 16)
#define SPRITE_YUV_ORDER_YUYV (0 << 16)
#define SPRITE_YUV_ORDER_UYVY (1 << 16)
#define SPRITE_YUV_ORDER_YVYU (2 << 16)
@@ -7041,7 +7131,7 @@ enum {
#define SP_ALPHA_PREMULTIPLY (1 << 23) /* CHV pipe B */
#define SP_SOURCE_KEY (1 << 22)
#define SP_YUV_FORMAT_BT709 (1 << 18)
-#define SP_YUV_BYTE_ORDER_MASK (3 << 16)
+#define SP_YUV_ORDER_MASK (3 << 16)
#define SP_YUV_ORDER_YUYV (0 << 16)
#define SP_YUV_ORDER_UYVY (1 << 16)
#define SP_YUV_ORDER_YVYU (2 << 16)
@@ -7182,10 +7272,10 @@ enum {
#define PLANE_CTL_YUV420_Y_PLANE (1 << 19)
#define PLANE_CTL_YUV_TO_RGB_CSC_FORMAT_BT709 (1 << 18)
#define PLANE_CTL_YUV422_ORDER_MASK (0x3 << 16)
-#define PLANE_CTL_YUV422_YUYV (0 << 16)
-#define PLANE_CTL_YUV422_UYVY (1 << 16)
-#define PLANE_CTL_YUV422_YVYU (2 << 16)
-#define PLANE_CTL_YUV422_VYUY (3 << 16)
+#define PLANE_CTL_YUV422_ORDER_YUYV (0 << 16)
+#define PLANE_CTL_YUV422_ORDER_UYVY (1 << 16)
+#define PLANE_CTL_YUV422_ORDER_YVYU (2 << 16)
+#define PLANE_CTL_YUV422_ORDER_VYUY (3 << 16)
#define PLANE_CTL_RENDER_DECOMPRESSION_ENABLE (1 << 15)
#define PLANE_CTL_TRICKLE_FEED_DISABLE (1 << 14)
#define PLANE_CTL_CLEAR_COLOR_DISABLE (1 << 13) /* TGL+ */
@@ -7239,10 +7329,10 @@ enum {
#define _PLANE_CUS_CTL_1_A 0x701c8
#define _PLANE_CUS_CTL_2_A 0x702c8
#define PLANE_CUS_ENABLE (1 << 31)
-#define PLANE_CUS_PLANE_4_RKL (0 << 30)
-#define PLANE_CUS_PLANE_5_RKL (1 << 30)
-#define PLANE_CUS_PLANE_6 (0 << 30)
-#define PLANE_CUS_PLANE_7 (1 << 30)
+#define PLANE_CUS_Y_PLANE_4_RKL (0 << 30)
+#define PLANE_CUS_Y_PLANE_5_RKL (1 << 30)
+#define PLANE_CUS_Y_PLANE_6_ICL (0 << 30)
+#define PLANE_CUS_Y_PLANE_7_ICL (1 << 30)
#define PLANE_CUS_HPHASE_SIGN_NEGATIVE (1 << 19)
#define PLANE_CUS_HPHASE_0 (0 << 16)
#define PLANE_CUS_HPHASE_0_25 (1 << 16)
@@ -7274,12 +7364,12 @@ enum {
#define _PLANE_NV12_BUF_CFG_1_A 0x70278
#define _PLANE_NV12_BUF_CFG_2_A 0x70378
-#define _PLANE_CC_VAL_1_B 0x711b4
-#define _PLANE_CC_VAL_2_B 0x712b4
-#define _PLANE_CC_VAL_1(pipe) _PIPE(pipe, _PLANE_CC_VAL_1_A, _PLANE_CC_VAL_1_B)
-#define _PLANE_CC_VAL_2(pipe) _PIPE(pipe, _PLANE_CC_VAL_2_A, _PLANE_CC_VAL_2_B)
-#define PLANE_CC_VAL(pipe, plane) \
- _MMIO_PLANE(plane, _PLANE_CC_VAL_1(pipe), _PLANE_CC_VAL_2(pipe))
+#define _PLANE_CC_VAL_1_B 0x711b4
+#define _PLANE_CC_VAL_2_B 0x712b4
+#define _PLANE_CC_VAL_1(pipe, dw) (_PIPE(pipe, _PLANE_CC_VAL_1_A, _PLANE_CC_VAL_1_B) + (dw) * 4)
+#define _PLANE_CC_VAL_2(pipe, dw) (_PIPE(pipe, _PLANE_CC_VAL_2_A, _PLANE_CC_VAL_2_B) + (dw) * 4)
+#define PLANE_CC_VAL(pipe, plane, dw) \
+ _MMIO_PLANE((plane), _PLANE_CC_VAL_1((pipe), (dw)), _PLANE_CC_VAL_2((pipe), (dw)))
/* Input CSC Register Definitions */
#define _PLANE_INPUT_CSC_RY_GY_1_A 0x701E0
@@ -8263,7 +8353,7 @@ enum {
/*
* The below are numbered starting from "S1" on gen11/gen12, but starting
- * with gen13 display, the bspec switches to a 0-based numbering scheme
+ * with display 13, the bspec switches to a 0-based numbering scheme
* (although the addresses stay the same so new S0 = old S1, new S1 = old S2).
* We'll just use the 0-based numbering here for all platforms since it's the
* way things will be named by the hardware team going forward, plus it's more
@@ -8308,9 +8398,10 @@ enum {
#define RESET_PCH_HANDSHAKE_ENABLE (1 << 4)
#define GEN8_CHICKEN_DCPR_1 _MMIO(0x46430)
-#define SKL_SELECT_ALTERNATE_DC_EXIT (1 << 30)
-#define ICL_DELAY_PMRSP (1 << 22)
-#define MASK_WAKEMEM (1 << 13)
+#define SKL_SELECT_ALTERNATE_DC_EXIT REG_BIT(30)
+#define ICL_DELAY_PMRSP REG_BIT(22)
+#define DISABLE_FLR_SRC REG_BIT(15)
+#define MASK_WAKEMEM REG_BIT(13)
#define GEN11_CHICKEN_DCPR_2 _MMIO(0x46434)
#define DCPR_MASK_MAXLATENCY_MEMUP_CLR REG_BIT(27)
@@ -8351,6 +8442,9 @@ enum {
#define GEN9_CTX_PREEMPT_REG _MMIO(0x2248)
#define GEN12_DISABLE_POSH_BUSY_FF_DOP_CG REG_BIT(11)
+#define GEN12_CS_DEBUG_MODE1_CCCSUNIT_BE_COMMON _MMIO(0x20EC)
+#define GEN12_REPLAY_MODE_GRANULARITY REG_BIT(0)
+
#define GEN8_CS_CHICKEN1 _MMIO(0x2580)
#define GEN9_PREEMPT_3D_OBJECT_LEVEL (1 << 0)
#define GEN9_PREEMPT_GPGPU_LEVEL(hi, lo) (((hi) << 2) | ((lo) << 1))
@@ -8374,9 +8468,10 @@ enum {
#define GEN8_ERRDETBCTRL (1 << 9)
#define GEN11_COMMON_SLICE_CHICKEN3 _MMIO(0x7304)
- #define DG1_FLOAT_POINT_BLEND_OPT_STRICT_MODE_EN REG_BIT(12)
- #define GEN11_BLEND_EMB_FIX_DISABLE_IN_RCC REG_BIT(11)
- #define GEN12_DISABLE_CPS_AWARE_COLOR_PIPE REG_BIT(9)
+#define DG1_FLOAT_POINT_BLEND_OPT_STRICT_MODE_EN REG_BIT(12)
+#define XEHP_DUAL_SIMD8_SEQ_MERGE_DISABLE REG_BIT(12)
+#define GEN11_BLEND_EMB_FIX_DISABLE_IN_RCC REG_BIT(11)
+#define GEN12_DISABLE_CPS_AWARE_COLOR_PIPE REG_BIT(9)
#define HIZ_CHICKEN _MMIO(0x7018)
# define CHV_HZ_8X8_MODE_IN_1X REG_BIT(15)
@@ -8430,6 +8525,12 @@ enum {
#define GEN8_LQSC_FLUSH_COHERENT_LINES (1 << 21)
#define GEN8_LQSQ_NONIA_COHERENT_ATOMICS_ENABLE REG_BIT(22)
+#define GEN11_L3SQCREG5 _MMIO(0xb158)
+#define L3_PWM_TIMER_INIT_VAL_MASK REG_GENMASK(9, 0)
+
+#define XEHP_L3SCQREG7 _MMIO(0xb188)
+#define BLEND_FILL_CACHING_OPT_DIS REG_BIT(3)
+
/* GEN8 chicken */
#define HDC_CHICKEN0 _MMIO(0x7300)
#define ICL_HDC_MODE _MMIO(0xE5F4)
@@ -8440,6 +8541,12 @@ enum {
#define HDC_FORCE_NON_COHERENT (1 << 4)
#define HDC_BARRIER_PERFORMANCE_DISABLE (1 << 10)
+#define GEN12_HDC_CHICKEN0 _MMIO(0xE5F0)
+#define LSC_L1_FLUSH_CTL_3D_DATAPORT_FLUSH_EVENTS_MASK REG_GENMASK(13, 11)
+
+#define SARB_CHICKEN1 _MMIO(0xe90c)
+#define COMP_CKN_IN REG_GENMASK(30, 29)
+
#define GEN8_HDC_CHICKEN1 _MMIO(0x7304)
/* GEN9 chicken */
@@ -8467,8 +8574,13 @@ enum {
_PIPEB_CHICKEN)
#define UNDERRUN_RECOVERY_DISABLE_ADLP REG_BIT(30)
#define UNDERRUN_RECOVERY_ENABLE_DG2 REG_BIT(30)
-#define PIXEL_ROUNDING_TRUNC_FB_PASSTHRU (1 << 15)
-#define PER_PIXEL_ALPHA_BYPASS_EN (1 << 7)
+#define PIXEL_ROUNDING_TRUNC_FB_PASSTHRU REG_BIT(15)
+#define DG2_RENDER_CCSTAG_4_3_EN REG_BIT(12)
+#define PER_PIXEL_ALPHA_BYPASS_EN REG_BIT(7)
+
+#define VFLSKPD _MMIO(0x62a8)
+#define DIS_OVER_FETCH_CACHE REG_BIT(1)
+#define DIS_MULT_MISS_RD_SQUASH REG_BIT(0)
#define FF_MODE2 _MMIO(0x6604)
#define FF_MODE2_GS_TIMER_MASK REG_GENMASK(31, 24)
@@ -9293,6 +9405,9 @@ enum {
#define GEN8_SDEUNIT_CLOCK_GATE_DISABLE (1 << 14)
#define GEN8_HDCUNIT_CLOCK_GATE_DISABLE_HDCREQ (1 << 28)
+#define UNSLCGCTL9430 _MMIO(0x9430)
+#define MSQDUNIT_CLKGATE_DIS REG_BIT(3)
+
#define GEN6_GFXPAUSE _MMIO(0xA000)
#define GEN6_RPNSWREQ _MMIO(0xA008)
#define GEN6_TURBO_DISABLE (1 << 31)
@@ -9302,6 +9417,7 @@ enum {
#define GEN6_OFFSET(x) ((x) << 19)
#define GEN6_AGGRESSIVE_TURBO (0 << 15)
#define GEN9_SW_REQ_UNSLICE_RATIO_SHIFT 23
+#define GEN9_IGNORE_SLICE_RATIO (0 << 0)
#define GEN6_RC_VIDEO_FREQ _MMIO(0xA00C)
#define GEN6_RC_CONTROL _MMIO(0xA090)
@@ -9337,6 +9453,9 @@ enum {
#define GEN6_RP_UP_BUSY_CONT (0x4 << 3)
#define GEN6_RP_DOWN_IDLE_AVG (0x2 << 0)
#define GEN6_RP_DOWN_IDLE_CONT (0x1 << 0)
+#define GEN6_RPSWCTL_SHIFT 9
+#define GEN9_RPSWCTL_ENABLE (0x2 << GEN6_RPSWCTL_SHIFT)
+#define GEN9_RPSWCTL_DISABLE (0x0 << GEN6_RPSWCTL_SHIFT)
#define GEN6_RP_UP_THRESHOLD _MMIO(0xA02C)
#define GEN6_RP_DOWN_THRESHOLD _MMIO(0xA030)
#define GEN6_RP_CUR_UP_EI _MMIO(0xA050)
@@ -9608,24 +9727,39 @@ enum {
#define GEN9_CCS_TLB_PREFETCH_ENABLE (1 << 3)
#define GEN8_ROW_CHICKEN _MMIO(0xe4f0)
-#define FLOW_CONTROL_ENABLE (1 << 15)
-#define PARTIAL_INSTRUCTION_SHOOTDOWN_DISABLE (1 << 8)
-#define STALL_DOP_GATING_DISABLE (1 << 5)
-#define THROTTLE_12_5 (7 << 2)
-#define DISABLE_EARLY_EOT (1 << 1)
+#define FLOW_CONTROL_ENABLE REG_BIT(15)
+#define UGM_BACKUP_MODE REG_BIT(13)
+#define MDQ_ARBITRATION_MODE REG_BIT(12)
+#define PARTIAL_INSTRUCTION_SHOOTDOWN_DISABLE REG_BIT(8)
+#define STALL_DOP_GATING_DISABLE REG_BIT(5)
+#define THROTTLE_12_5 REG_GENMASK(4, 2)
+#define DISABLE_EARLY_EOT REG_BIT(1)
#define GEN7_ROW_CHICKEN2 _MMIO(0xe4f4)
+#define GEN12_DISABLE_READ_SUPPRESSION REG_BIT(15)
#define GEN12_DISABLE_EARLY_READ REG_BIT(14)
+#define GEN12_ENABLE_LARGE_GRF_MODE REG_BIT(12)
#define GEN12_PUSH_CONST_DEREF_HOLD_DIS REG_BIT(8)
+#define LSC_CHICKEN_BIT_0 _MMIO(0xe7c8)
+#define FORCE_1_SUB_MESSAGE_PER_FRAGMENT REG_BIT(15)
+#define LSC_CHICKEN_BIT_0_UDW _MMIO(0xe7c8 + 4)
+#define DIS_CHAIN_2XSIMD8 REG_BIT(55 - 32)
+#define FORCE_SLM_FENCE_SCOPE_TO_TILE REG_BIT(42 - 32)
+#define FORCE_UGM_FENCE_SCOPE_TO_TILE REG_BIT(41 - 32)
+#define MAXREQS_PER_BANK REG_GENMASK(39 - 32, 37 - 32)
+#define DISABLE_128B_EVICTION_COMMAND_UDW REG_BIT(36 - 32)
+
#define GEN7_ROW_CHICKEN2_GT2 _MMIO(0xf4f4)
#define DOP_CLOCK_GATING_DISABLE (1 << 0)
#define PUSH_CONSTANT_DEREF_DISABLE (1 << 8)
#define GEN11_TDL_CLOCK_GATING_FIX_DISABLE (1 << 1)
-#define GEN9_ROW_CHICKEN4 _MMIO(0xe48c)
-#define GEN12_DISABLE_TDL_PUSH REG_BIT(9)
-#define GEN11_DIS_PICK_2ND_EU REG_BIT(7)
+#define GEN9_ROW_CHICKEN4 _MMIO(0xe48c)
+#define GEN12_DISABLE_GRF_CLEAR REG_BIT(13)
+#define GEN12_DISABLE_TDL_PUSH REG_BIT(9)
+#define GEN11_DIS_PICK_2ND_EU REG_BIT(7)
+#define GEN12_DISABLE_HDR_PAST_PAYLOAD_HOLD_FIX REG_BIT(4)
#define HSW_ROW_CHICKEN3 _MMIO(0xe49c)
#define HSW_ROW_CHICKEN3_L3_GLOBAL_ATOMICS_DISABLE (1 << 6)
@@ -9640,9 +9774,10 @@ enum {
#define GEN8_SAMPLER_POWER_BYPASS_DIS (1 << 1)
#define GEN9_HALF_SLICE_CHICKEN7 _MMIO(0xe194)
-#define GEN9_SAMPLER_HASH_COMPRESSED_READ_ADDR (1 << 8)
-#define GEN9_ENABLE_YV12_BUGFIX (1 << 4)
-#define GEN9_ENABLE_GPGPU_PREEMPTION (1 << 2)
+#define DG2_DISABLE_ROUND_ENABLE_ALLOW_FOR_SSLA REG_BIT(15)
+#define GEN9_SAMPLER_HASH_COMPRESSED_READ_ADDR REG_BIT(8)
+#define GEN9_ENABLE_YV12_BUGFIX REG_BIT(4)
+#define GEN9_ENABLE_GPGPU_PREEMPTION REG_BIT(2)
/* Audio */
#define G4X_AUD_VID_DID _MMIO(DISPLAY_MMIO_BASE(dev_priv) + 0x62020)
@@ -9781,6 +9916,10 @@ enum {
#define AUD_PIN_BUF_CTL _MMIO(0x48414)
#define AUD_PIN_BUF_ENABLE REG_BIT(31)
+#define AUD_TS_CDCLK_M _MMIO(0x65ea0)
+#define AUD_TS_CDCLK_M_EN REG_BIT(31)
+#define AUD_TS_CDCLK_N _MMIO(0x65ea4)
+
/* Display Audio Config Reg */
#define AUD_CONFIG_BE _MMIO(0x65ef0)
#define HBLANK_EARLY_ENABLE_ICL(pipe) (0x1 << (20 - (pipe)))
@@ -10212,8 +10351,6 @@ enum skl_power_gate {
#define TGL_TRANS_DDI_PORT_MASK (0xf << TGL_TRANS_DDI_PORT_SHIFT)
#define TRANS_DDI_SELECT_PORT(x) ((x) << TRANS_DDI_PORT_SHIFT)
#define TGL_TRANS_DDI_SELECT_PORT(x) (((x) + 1) << TGL_TRANS_DDI_PORT_SHIFT)
-#define TRANS_DDI_FUNC_CTL_VAL_TO_PORT(val) (((val) & TRANS_DDI_PORT_MASK) >> TRANS_DDI_PORT_SHIFT)
-#define TGL_TRANS_DDI_FUNC_CTL_VAL_TO_PORT(val) ((((val) & TGL_TRANS_DDI_PORT_MASK) >> TGL_TRANS_DDI_PORT_SHIFT) - 1)
#define TRANS_DDI_MODE_SELECT_MASK (7 << 24)
#define TRANS_DDI_MODE_SELECT_HDMI (0 << 24)
#define TRANS_DDI_MODE_SELECT_DVI (1 << 24)
@@ -10523,6 +10660,14 @@ enum skl_power_gate {
#define BXT_CDCLK_SSA_PRECHARGE_ENABLE (1 << 16)
#define CDCLK_FREQ_DECIMAL_MASK (0x7ff)
+/* CDCLK_SQUASH_CTL */
+#define CDCLK_SQUASH_CTL _MMIO(0x46008)
+#define CDCLK_SQUASH_ENABLE REG_BIT(31)
+#define CDCLK_SQUASH_WINDOW_SIZE_MASK REG_GENMASK(27, 24)
+#define CDCLK_SQUASH_WINDOW_SIZE(x) REG_FIELD_PREP(CDCLK_SQUASH_WINDOW_SIZE_MASK, (x))
+#define CDCLK_SQUASH_WAVEFORM_MASK REG_GENMASK(15, 0)
+#define CDCLK_SQUASH_WAVEFORM(x) REG_FIELD_PREP(CDCLK_SQUASH_WAVEFORM_MASK, (x))
+
/* LCPLL_CTL */
#define LCPLL1_CTL _MMIO(0x46010)
#define LCPLL2_CTL _MMIO(0x46014)
@@ -12466,11 +12611,19 @@ enum skl_power_gate {
#define PMFLUSH_GAPL3UNBLOCK (1 << 21)
#define PMFLUSHDONE_LNEBLK (1 << 22)
+#define XEHP_L3NODEARBCFG _MMIO(0xb0b4)
+#define XEHP_LNESPARE REG_BIT(19)
+
#define GEN12_GLOBAL_MOCS(i) _MMIO(0x4000 + (i) * 4) /* Global MOCS regs */
#define GEN12_GSMBASE _MMIO(0x108100)
#define GEN12_DSMBASE _MMIO(0x1080C0)
+#define XEHP_CLOCK_GATE_DIS _MMIO(0x101014)
+#define SGSI_SIDECLK_DIS REG_BIT(17)
+#define SGGI_DIS REG_BIT(15)
+#define SGR_DIS REG_BIT(13)
+
/* gamt regs */
#define GEN8_L3_LRA_1_GPGPU _MMIO(0x4dd4)
#define GEN8_L3_LRA_1_GPGPU_DEFAULT_VALUE_BDW 0x67F1427F /* max/min for LRA1/2 */
@@ -12847,4 +13000,7 @@ enum skl_power_gate {
#define CLKGATE_DIS_MISC _MMIO(0x46534)
#define CLKGATE_DIS_MISC_DMASC_GATING_DIS REG_BIT(21)
+#define SLICE_COMMON_ECO_CHICKEN1 _MMIO(0x731C)
+#define MSC_MSAA_REODER_BUF_BYPASS_DISABLE REG_BIT(14)
+
#endif /* _I915_REG_H_ */
diff --git a/drivers/gpu/drm/i915/i915_request.c b/drivers/gpu/drm/i915/i915_request.c
index 89cccef..76cf5ac 100644
--- a/drivers/gpu/drm/i915/i915_request.c
+++ b/drivers/gpu/drm/i915/i915_request.c
@@ -42,6 +42,7 @@
#include "gt/intel_rps.h"
#include "i915_active.h"
+#include "i915_deps.h"
#include "i915_drv.h"
#include "i915_trace.h"
#include "intel_pm.h"
@@ -97,9 +98,9 @@ static signed long i915_fence_wait(struct dma_fence *fence,
bool interruptible,
signed long timeout)
{
- return i915_request_wait(to_request(fence),
- interruptible | I915_WAIT_PRIORITY,
- timeout);
+ return i915_request_wait_timeout(to_request(fence),
+ interruptible | I915_WAIT_PRIORITY,
+ timeout);
}
struct kmem_cache *i915_request_slab_cache(void)
@@ -114,6 +115,10 @@ static void i915_fence_release(struct dma_fence *fence)
GEM_BUG_ON(rq->guc_prio != GUC_PRIO_INIT &&
rq->guc_prio != GUC_PRIO_FINI);
+ i915_request_free_capture_list(fetch_and_zero(&rq->capture_list));
+ if (i915_vma_snapshot_present(&rq->batch_snapshot))
+ i915_vma_snapshot_put_onstack(&rq->batch_snapshot);
+
/*
* The request is put onto a RCU freelist (i.e. the address
* is immediately reused), mark the fences as being freed now.
@@ -187,19 +192,6 @@ void i915_request_notify_execute_cb_imm(struct i915_request *rq)
__notify_execute_cb(rq, irq_work_imm);
}
-static void free_capture_list(struct i915_request *request)
-{
- struct i915_capture_list *capture;
-
- capture = fetch_and_zero(&request->capture_list);
- while (capture) {
- struct i915_capture_list *next = capture->next;
-
- kfree(capture);
- capture = next;
- }
-}
-
static void __i915_request_fill(struct i915_request *rq, u8 val)
{
void *vaddr = rq->ring->vaddr;
@@ -304,6 +296,38 @@ static void __rq_cancel_watchdog(struct i915_request *rq)
i915_request_put(rq);
}
+#if IS_ENABLED(CONFIG_DRM_I915_CAPTURE_ERROR)
+
+/**
+ * i915_request_free_capture_list - Free a capture list
+ * @capture: Pointer to the first list item or NULL
+ *
+ */
+void i915_request_free_capture_list(struct i915_capture_list *capture)
+{
+ while (capture) {
+ struct i915_capture_list *next = capture->next;
+
+ i915_vma_snapshot_put(capture->vma_snapshot);
+ kfree(capture);
+ capture = next;
+ }
+}
+
+#define assert_capture_list_is_null(_rq) GEM_BUG_ON((_rq)->capture_list)
+
+#define clear_capture_list(_rq) ((_rq)->capture_list = NULL)
+
+#else
+
+#define i915_request_free_capture_list(_a) do {} while (0)
+
+#define assert_capture_list_is_null(_a) do {} while (0)
+
+#define clear_capture_list(_rq) do {} while (0)
+
+#endif
+
bool i915_request_retire(struct i915_request *rq)
{
if (!__i915_request_is_complete(rq))
@@ -340,7 +364,7 @@ bool i915_request_retire(struct i915_request *rq)
}
if (test_and_set_bit(I915_FENCE_FLAG_BOOST, &rq->fence.flags))
- atomic_dec(&rq->engine->gt->rps.num_waiters);
+ intel_rps_dec_waiters(&rq->engine->gt->rps);
/*
* We only loosely track inflight requests across preemption,
@@ -360,7 +384,6 @@ bool i915_request_retire(struct i915_request *rq)
intel_context_exit(rq->context);
intel_context_unpin(rq->context);
- free_capture_list(rq);
i915_sched_node_fini(&rq->sched);
i915_request_put(rq);
@@ -720,7 +743,7 @@ void i915_request_cancel(struct i915_request *rq, int error)
intel_context_cancel_request(rq->context, rq);
}
-static int __i915_sw_fence_call
+static int
submit_notify(struct i915_sw_fence *fence, enum i915_sw_fence_notify state)
{
struct i915_request *request =
@@ -756,7 +779,7 @@ submit_notify(struct i915_sw_fence *fence, enum i915_sw_fence_notify state)
return NOTIFY_DONE;
}
-static int __i915_sw_fence_call
+static int
semaphore_notify(struct i915_sw_fence *fence, enum i915_sw_fence_notify state)
{
struct i915_request *rq = container_of(fence, typeof(*rq), semaphore);
@@ -830,11 +853,18 @@ static void __i915_request_ctor(void *arg)
i915_sw_fence_init(&rq->submit, submit_notify);
i915_sw_fence_init(&rq->semaphore, semaphore_notify);
- rq->capture_list = NULL;
+ clear_capture_list(rq);
+ rq->batch_snapshot.present = false;
init_llist_head(&rq->execute_cb);
}
+#if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
+#define clear_batch_ptr(_rq) ((_rq)->batch = NULL)
+#else
+#define clear_batch_ptr(_a) do {} while (0)
+#endif
+
struct i915_request *
__i915_request_create(struct intel_context *ce, gfp_t gfp)
{
@@ -926,10 +956,11 @@ __i915_request_create(struct intel_context *ce, gfp_t gfp)
i915_sched_node_reinit(&rq->sched);
/* No zalloc, everything must be cleared after use */
- rq->batch = NULL;
+ clear_batch_ptr(rq);
__rq_init_watchdog(rq);
- GEM_BUG_ON(rq->capture_list);
+ assert_capture_list_is_null(rq);
GEM_BUG_ON(!llist_empty(&rq->execute_cb));
+ GEM_BUG_ON(i915_vma_snapshot_present(&rq->batch_snapshot));
/*
* Reserve space in the ring buffer for all the commands required to
@@ -1514,6 +1545,27 @@ i915_request_await_dma_fence(struct i915_request *rq, struct dma_fence *fence)
}
/**
+ * i915_request_await_deps - set this request to (async) wait upon a struct
+ * i915_deps dma_fence collection
+ * @rq: request we are wishing to use
+ * @deps: The struct i915_deps containing the dependencies.
+ *
+ * Returns 0 if successful, negative error code on error.
+ */
+int i915_request_await_deps(struct i915_request *rq, const struct i915_deps *deps)
+{
+ int i, err;
+
+ for (i = 0; i < deps->num_deps; ++i) {
+ err = i915_request_await_dma_fence(rq, deps->fences[i]);
+ if (err)
+ return err;
+ }
+
+ return 0;
+}
+
+/**
* i915_request_await_object - set this request to (async) wait upon a bo
* @to: request we are wishing to use
* @obj: object which may be in use on another ring.
@@ -1858,23 +1910,27 @@ static void request_wait_wake(struct dma_fence *fence, struct dma_fence_cb *cb)
}
/**
- * i915_request_wait - wait until execution of request has finished
+ * i915_request_wait_timeout - wait until execution of request has finished
* @rq: the request to wait upon
* @flags: how to wait
* @timeout: how long to wait in jiffies
*
- * i915_request_wait() waits for the request to be completed, for a
+ * i915_request_wait_timeout() waits for the request to be completed, for a
* maximum of @timeout jiffies (with MAX_SCHEDULE_TIMEOUT implying an
* unbounded wait).
*
* Returns the remaining time (in jiffies) if the request completed, which may
- * be zero or -ETIME if the request is unfinished after the timeout expires.
+ * be zero if the request is unfinished after the timeout expires.
+ * If the timeout is 0, it will return 1 if the fence is signaled.
+ *
* May return -EINTR is called with I915_WAIT_INTERRUPTIBLE and a signal is
* pending before the request completes.
+ *
+ * NOTE: This function has the same wait semantics as dma-fence.
*/
-long i915_request_wait(struct i915_request *rq,
- unsigned int flags,
- long timeout)
+long i915_request_wait_timeout(struct i915_request *rq,
+ unsigned int flags,
+ long timeout)
{
const int state = flags & I915_WAIT_INTERRUPTIBLE ?
TASK_INTERRUPTIBLE : TASK_UNINTERRUPTIBLE;
@@ -1884,7 +1940,7 @@ long i915_request_wait(struct i915_request *rq,
GEM_BUG_ON(timeout < 0);
if (dma_fence_is_signaled(&rq->fence))
- return timeout;
+ return timeout ?: 1;
if (!timeout)
return -ETIME;
@@ -1993,6 +2049,39 @@ long i915_request_wait(struct i915_request *rq,
return timeout;
}
+/**
+ * i915_request_wait - wait until execution of request has finished
+ * @rq: the request to wait upon
+ * @flags: how to wait
+ * @timeout: how long to wait in jiffies
+ *
+ * i915_request_wait() waits for the request to be completed, for a
+ * maximum of @timeout jiffies (with MAX_SCHEDULE_TIMEOUT implying an
+ * unbounded wait).
+ *
+ * Returns the remaining time (in jiffies) if the request completed, which may
+ * be zero or -ETIME if the request is unfinished after the timeout expires.
+ * May return -EINTR is called with I915_WAIT_INTERRUPTIBLE and a signal is
+ * pending before the request completes.
+ *
+ * NOTE: This function behaves differently from dma-fence wait semantics for
+ * timeout = 0. It returns 0 on success, and -ETIME if not signaled.
+ */
+long i915_request_wait(struct i915_request *rq,
+ unsigned int flags,
+ long timeout)
+{
+ long ret = i915_request_wait_timeout(rq, flags, timeout);
+
+ if (!ret)
+ return -ETIME;
+
+ if (ret > 0 && !timeout)
+ return 0;
+
+ return ret;
+}
+
static int print_sched_attr(const struct i915_sched_attr *attr,
char *buf, int x, int len)
{
diff --git a/drivers/gpu/drm/i915/i915_request.h b/drivers/gpu/drm/i915/i915_request.h
index dc35924..170ee78 100644
--- a/drivers/gpu/drm/i915/i915_request.h
+++ b/drivers/gpu/drm/i915/i915_request.h
@@ -40,19 +40,27 @@
#include "i915_scheduler.h"
#include "i915_selftest.h"
#include "i915_sw_fence.h"
+#include "i915_vma_snapshot.h"
#include <uapi/drm/i915_drm.h>
struct drm_file;
struct drm_i915_gem_object;
struct drm_printer;
+struct i915_deps;
struct i915_request;
+#if IS_ENABLED(CONFIG_DRM_I915_CAPTURE_ERROR)
struct i915_capture_list {
+ struct i915_vma_snapshot *vma_snapshot;
struct i915_capture_list *next;
- struct i915_vma *vma;
};
+void i915_request_free_capture_list(struct i915_capture_list *capture);
+#else
+#define i915_request_free_capture_list(_a) do {} while (0)
+#endif
+
#define RQ_TRACE(rq, fmt, ...) do { \
const struct i915_request *rq__ = (rq); \
ENGINE_TRACE(rq__->engine, "fence %llx:%lld, current %d " fmt, \
@@ -289,10 +297,12 @@ struct i915_request {
/** Preallocate space in the ring for the emitting the request */
u32 reserved_space;
- /** Batch buffer related to this request if any (used for
- * error state dump only).
- */
- struct i915_vma *batch;
+ /** Batch buffer pointer for selftest internal use. */
+ I915_SELFTEST_DECLARE(struct i915_vma *batch);
+
+ struct i915_vma_snapshot batch_snapshot;
+
+#if IS_ENABLED(CONFIG_DRM_I915_CAPTURE_ERROR)
/**
* Additional buffers requested by userspace to be captured upon
* a GPU hang. The vma/obj on this list are protected by their
@@ -300,6 +310,7 @@ struct i915_request {
* on the active_list (of their final request).
*/
struct i915_capture_list *capture_list;
+#endif
/** Time at which this request was emitted, in jiffies. */
unsigned long emitted_jiffies;
@@ -401,6 +412,7 @@ int i915_request_await_object(struct i915_request *to,
bool write);
int i915_request_await_dma_fence(struct i915_request *rq,
struct dma_fence *fence);
+int i915_request_await_deps(struct i915_request *rq, const struct i915_deps *deps);
int i915_request_await_execution(struct i915_request *rq,
struct dma_fence *fence);
@@ -414,6 +426,11 @@ void i915_request_unsubmit(struct i915_request *request);
void i915_request_cancel(struct i915_request *rq, int error);
+long i915_request_wait_timeout(struct i915_request *rq,
+ unsigned int flags,
+ long timeout)
+ __attribute__((nonnull(1)));
+
long i915_request_wait(struct i915_request *rq,
unsigned int flags,
long timeout)
@@ -642,7 +659,8 @@ i915_request_timeline(const struct i915_request *rq)
{
/* Valid only while the request is being constructed (or retired). */
return rcu_dereference_protected(rq->timeline,
- lockdep_is_held(&rcu_access_pointer(rq->timeline)->mutex));
+ lockdep_is_held(&rcu_access_pointer(rq->timeline)->mutex) ||
+ test_bit(CONTEXT_IS_PARKING, &rq->context->flags));
}
static inline struct i915_gem_context *
diff --git a/drivers/gpu/drm/i915/i915_scatterlist.c b/drivers/gpu/drm/i915/i915_scatterlist.c
index 4a6712d..41f2adb 100644
--- a/drivers/gpu/drm/i915/i915_scatterlist.c
+++ b/drivers/gpu/drm/i915/i915_scatterlist.c
@@ -41,8 +41,32 @@ bool i915_sg_trim(struct sg_table *orig_st)
return true;
}
+static void i915_refct_sgt_release(struct kref *ref)
+{
+ struct i915_refct_sgt *rsgt =
+ container_of(ref, typeof(*rsgt), kref);
+
+ sg_free_table(&rsgt->table);
+ kfree(rsgt);
+}
+
+static const struct i915_refct_sgt_ops rsgt_ops = {
+ .release = i915_refct_sgt_release
+};
+
/**
- * i915_sg_from_mm_node - Create an sg_table from a struct drm_mm_node
+ * i915_refct_sgt_init - Initialize a struct i915_refct_sgt with default ops
+ * @rsgt: The struct i915_refct_sgt to initialize.
+ * size: The size of the underlying memory buffer.
+ */
+void i915_refct_sgt_init(struct i915_refct_sgt *rsgt, size_t size)
+{
+ __i915_refct_sgt_init(rsgt, size, &rsgt_ops);
+}
+
+/**
+ * i915_rsgt_from_mm_node - Create a refcounted sg_table from a struct
+ * drm_mm_node
* @node: The drm_mm_node.
* @region_start: An offset to add to the dma addresses of the sg list.
*
@@ -50,25 +74,28 @@ bool i915_sg_trim(struct sg_table *orig_st)
* taking a maximum segment length into account, splitting into segments
* if necessary.
*
- * Return: A pointer to a kmalloced struct sg_table on success, negative
+ * Return: A pointer to a kmalloced struct i915_refct_sgt on success, negative
* error code cast to an error pointer on failure.
*/
-struct sg_table *i915_sg_from_mm_node(const struct drm_mm_node *node,
- u64 region_start)
+struct i915_refct_sgt *i915_rsgt_from_mm_node(const struct drm_mm_node *node,
+ u64 region_start)
{
const u64 max_segment = SZ_1G; /* Do we have a limit on this? */
u64 segment_pages = max_segment >> PAGE_SHIFT;
u64 block_size, offset, prev_end;
+ struct i915_refct_sgt *rsgt;
struct sg_table *st;
struct scatterlist *sg;
- st = kmalloc(sizeof(*st), GFP_KERNEL);
- if (!st)
+ rsgt = kmalloc(sizeof(*rsgt), GFP_KERNEL);
+ if (!rsgt)
return ERR_PTR(-ENOMEM);
+ i915_refct_sgt_init(rsgt, node->size << PAGE_SHIFT);
+ st = &rsgt->table;
if (sg_alloc_table(st, DIV_ROUND_UP(node->size, segment_pages),
GFP_KERNEL)) {
- kfree(st);
+ i915_refct_sgt_put(rsgt);
return ERR_PTR(-ENOMEM);
}
@@ -104,11 +131,11 @@ struct sg_table *i915_sg_from_mm_node(const struct drm_mm_node *node,
sg_mark_end(sg);
i915_sg_trim(st);
- return st;
+ return rsgt;
}
/**
- * i915_sg_from_buddy_resource - Create an sg_table from a struct
+ * i915_rsgt_from_buddy_resource - Create a refcounted sg_table from a struct
* i915_buddy_block list
* @res: The struct i915_ttm_buddy_resource.
* @region_start: An offset to add to the dma addresses of the sg list.
@@ -117,11 +144,11 @@ struct sg_table *i915_sg_from_mm_node(const struct drm_mm_node *node,
* taking a maximum segment length into account, splitting into segments
* if necessary.
*
- * Return: A pointer to a kmalloced struct sg_table on success, negative
+ * Return: A pointer to a kmalloced struct i915_refct_sgts on success, negative
* error code cast to an error pointer on failure.
*/
-struct sg_table *i915_sg_from_buddy_resource(struct ttm_resource *res,
- u64 region_start)
+struct i915_refct_sgt *i915_rsgt_from_buddy_resource(struct ttm_resource *res,
+ u64 region_start)
{
struct i915_ttm_buddy_resource *bman_res = to_ttm_buddy_resource(res);
const u64 size = res->num_pages << PAGE_SHIFT;
@@ -129,18 +156,21 @@ struct sg_table *i915_sg_from_buddy_resource(struct ttm_resource *res,
struct i915_buddy_mm *mm = bman_res->mm;
struct list_head *blocks = &bman_res->blocks;
struct i915_buddy_block *block;
+ struct i915_refct_sgt *rsgt;
struct scatterlist *sg;
struct sg_table *st;
resource_size_t prev_end;
GEM_BUG_ON(list_empty(blocks));
- st = kmalloc(sizeof(*st), GFP_KERNEL);
- if (!st)
+ rsgt = kmalloc(sizeof(*rsgt), GFP_KERNEL);
+ if (!rsgt)
return ERR_PTR(-ENOMEM);
+ i915_refct_sgt_init(rsgt, size);
+ st = &rsgt->table;
if (sg_alloc_table(st, res->num_pages, GFP_KERNEL)) {
- kfree(st);
+ i915_refct_sgt_put(rsgt);
return ERR_PTR(-ENOMEM);
}
@@ -181,7 +211,7 @@ struct sg_table *i915_sg_from_buddy_resource(struct ttm_resource *res,
sg_mark_end(sg);
i915_sg_trim(st);
- return st;
+ return rsgt;
}
#if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
diff --git a/drivers/gpu/drm/i915/i915_scatterlist.h b/drivers/gpu/drm/i915/i915_scatterlist.h
index b8bd592..12c6a16 100644
--- a/drivers/gpu/drm/i915/i915_scatterlist.h
+++ b/drivers/gpu/drm/i915/i915_scatterlist.h
@@ -144,10 +144,78 @@ static inline unsigned int i915_sg_segment_size(void)
bool i915_sg_trim(struct sg_table *orig_st);
-struct sg_table *i915_sg_from_mm_node(const struct drm_mm_node *node,
- u64 region_start);
+/**
+ * struct i915_refct_sgt_ops - Operations structure for struct i915_refct_sgt
+ */
+struct i915_refct_sgt_ops {
+ /**
+ * release() - Free the memory of the struct i915_refct_sgt
+ * @ref: struct kref that is embedded in the struct i915_refct_sgt
+ */
+ void (*release)(struct kref *ref);
+};
-struct sg_table *i915_sg_from_buddy_resource(struct ttm_resource *res,
- u64 region_start);
+/**
+ * struct i915_refct_sgt - A refcounted scatter-gather table
+ * @kref: struct kref for refcounting
+ * @table: struct sg_table holding the scatter-gather table itself. Note that
+ * @table->sgl = NULL can be used to determine whether a scatter-gather table
+ * is present or not.
+ * @size: The size in bytes of the underlying memory buffer
+ * @ops: The operations structure.
+ */
+struct i915_refct_sgt {
+ struct kref kref;
+ struct sg_table table;
+ size_t size;
+ const struct i915_refct_sgt_ops *ops;
+};
+
+/**
+ * i915_refct_sgt_put - Put a refcounted sg-table
+ * @rsgt the struct i915_refct_sgt to put.
+ */
+static inline void i915_refct_sgt_put(struct i915_refct_sgt *rsgt)
+{
+ if (rsgt)
+ kref_put(&rsgt->kref, rsgt->ops->release);
+}
+
+/**
+ * i915_refct_sgt_get - Get a refcounted sg-table
+ * @rsgt the struct i915_refct_sgt to get.
+ */
+static inline struct i915_refct_sgt *
+i915_refct_sgt_get(struct i915_refct_sgt *rsgt)
+{
+ kref_get(&rsgt->kref);
+ return rsgt;
+}
+
+/**
+ * __i915_refct_sgt_init - Initialize a refcounted sg-list with a custom
+ * operations structure
+ * @rsgt The struct i915_refct_sgt to initialize.
+ * @size: Size in bytes of the underlying memory buffer.
+ * @ops: A customized operations structure in case the refcounted sg-list
+ * is embedded into another structure.
+ */
+static inline void __i915_refct_sgt_init(struct i915_refct_sgt *rsgt,
+ size_t size,
+ const struct i915_refct_sgt_ops *ops)
+{
+ kref_init(&rsgt->kref);
+ rsgt->table.sgl = NULL;
+ rsgt->size = size;
+ rsgt->ops = ops;
+}
+
+void i915_refct_sgt_init(struct i915_refct_sgt *rsgt, size_t size);
+
+struct i915_refct_sgt *i915_rsgt_from_mm_node(const struct drm_mm_node *node,
+ u64 region_start);
+
+struct i915_refct_sgt *i915_rsgt_from_buddy_resource(struct ttm_resource *res,
+ u64 region_start);
#endif
diff --git a/drivers/gpu/drm/i915/i915_sw_fence.c b/drivers/gpu/drm/i915/i915_sw_fence.c
index c589a68..2a74a9a 100644
--- a/drivers/gpu/drm/i915/i915_sw_fence.c
+++ b/drivers/gpu/drm/i915/i915_sw_fence.c
@@ -18,7 +18,9 @@
#define I915_SW_FENCE_BUG_ON(expr) BUILD_BUG_ON_INVALID(expr)
#endif
+#ifdef CONFIG_DRM_I915_SW_FENCE_CHECK_DAG
static DEFINE_SPINLOCK(i915_sw_fence_lock);
+#endif
#define WQ_FLAG_BITS \
BITS_PER_TYPE(typeof_member(struct wait_queue_entry, flags))
@@ -34,7 +36,7 @@ enum {
static void *i915_sw_fence_debug_hint(void *addr)
{
- return (void *)(((struct i915_sw_fence *)addr)->flags & I915_SW_FENCE_MASK);
+ return (void *)(((struct i915_sw_fence *)addr)->fn);
}
#ifdef CONFIG_DRM_I915_SW_FENCE_DEBUG_OBJECTS
@@ -126,10 +128,7 @@ static inline void debug_fence_assert(struct i915_sw_fence *fence)
static int __i915_sw_fence_notify(struct i915_sw_fence *fence,
enum i915_sw_fence_notify state)
{
- i915_sw_fence_notify_t fn;
-
- fn = (i915_sw_fence_notify_t)(fence->flags & I915_SW_FENCE_MASK);
- return fn(fence, state);
+ return fence->fn(fence, state);
}
#ifdef CONFIG_DRM_I915_SW_FENCE_DEBUG_OBJECTS
@@ -242,10 +241,13 @@ void __i915_sw_fence_init(struct i915_sw_fence *fence,
const char *name,
struct lock_class_key *key)
{
- BUG_ON(!fn || (unsigned long)fn & ~I915_SW_FENCE_MASK);
+ BUG_ON(!fn);
__init_waitqueue_head(&fence->wait, name, key);
- fence->flags = (unsigned long)fn;
+ fence->fn = fn;
+#ifdef CONFIG_DRM_I915_SW_FENCE_CHECK_DAG
+ fence->flags = 0;
+#endif
i915_sw_fence_reinit(fence);
}
@@ -257,7 +259,6 @@ void i915_sw_fence_reinit(struct i915_sw_fence *fence)
atomic_set(&fence->pending, 1);
fence->error = 0;
- I915_SW_FENCE_BUG_ON(!fence->flags);
I915_SW_FENCE_BUG_ON(!list_empty(&fence->wait.head));
}
@@ -279,6 +280,7 @@ static int i915_sw_fence_wake(wait_queue_entry_t *wq, unsigned mode, int flags,
return 0;
}
+#ifdef CONFIG_DRM_I915_SW_FENCE_CHECK_DAG
static bool __i915_sw_fence_check_if_after(struct i915_sw_fence *fence,
const struct i915_sw_fence * const signaler)
{
@@ -322,9 +324,6 @@ static bool i915_sw_fence_check_if_after(struct i915_sw_fence *fence,
unsigned long flags;
bool err;
- if (!IS_ENABLED(CONFIG_DRM_I915_SW_FENCE_CHECK_DAG))
- return false;
-
spin_lock_irqsave(&i915_sw_fence_lock, flags);
err = __i915_sw_fence_check_if_after(fence, signaler);
__i915_sw_fence_clear_checked_bit(fence);
@@ -332,6 +331,13 @@ static bool i915_sw_fence_check_if_after(struct i915_sw_fence *fence,
return err;
}
+#else
+static bool i915_sw_fence_check_if_after(struct i915_sw_fence *fence,
+ const struct i915_sw_fence * const signaler)
+{
+ return false;
+}
+#endif
static int __i915_sw_fence_await_sw_fence(struct i915_sw_fence *fence,
struct i915_sw_fence *signaler,
@@ -572,56 +578,25 @@ int i915_sw_fence_await_reservation(struct i915_sw_fence *fence,
unsigned long timeout,
gfp_t gfp)
{
- struct dma_fence *excl;
+ struct dma_resv_iter cursor;
+ struct dma_fence *f;
int ret = 0, pending;
debug_fence_assert(fence);
might_sleep_if(gfpflags_allow_blocking(gfp));
- if (write) {
- struct dma_fence **shared;
- unsigned int count, i;
-
- ret = dma_resv_get_fences(resv, &excl, &count, &shared);
- if (ret)
- return ret;
-
- for (i = 0; i < count; i++) {
- if (shared[i]->ops == exclude)
- continue;
-
- pending = i915_sw_fence_await_dma_fence(fence,
- shared[i],
- timeout,
- gfp);
- if (pending < 0) {
- ret = pending;
- break;
- }
-
- ret |= pending;
+ dma_resv_iter_begin(&cursor, resv, write);
+ dma_resv_for_each_fence_unlocked(&cursor, f) {
+ pending = i915_sw_fence_await_dma_fence(fence, f, timeout,
+ gfp);
+ if (pending < 0) {
+ ret = pending;
+ break;
}
- for (i = 0; i < count; i++)
- dma_fence_put(shared[i]);
- kfree(shared);
- } else {
- excl = dma_resv_get_excl_unlocked(resv);
+ ret |= pending;
}
-
- if (ret >= 0 && excl && excl->ops != exclude) {
- pending = i915_sw_fence_await_dma_fence(fence,
- excl,
- timeout,
- gfp);
- if (pending < 0)
- ret = pending;
- else
- ret |= pending;
- }
-
- dma_fence_put(excl);
-
+ dma_resv_iter_end(&cursor);
return ret;
}
diff --git a/drivers/gpu/drm/i915/i915_sw_fence.h b/drivers/gpu/drm/i915/i915_sw_fence.h
index 30a8633..a7c603b 100644
--- a/drivers/gpu/drm/i915/i915_sw_fence.h
+++ b/drivers/gpu/drm/i915/i915_sw_fence.h
@@ -17,17 +17,7 @@
struct completion;
struct dma_resv;
-
-struct i915_sw_fence {
- wait_queue_head_t wait;
- unsigned long flags;
- atomic_t pending;
- int error;
-};
-
-#define I915_SW_FENCE_CHECKED_BIT 0 /* used internally for DAG checking */
-#define I915_SW_FENCE_PRIVATE_BIT 1 /* available for use by owner */
-#define I915_SW_FENCE_MASK (~3)
+struct i915_sw_fence;
enum i915_sw_fence_notify {
FENCE_COMPLETE,
@@ -36,7 +26,18 @@ enum i915_sw_fence_notify {
typedef int (*i915_sw_fence_notify_t)(struct i915_sw_fence *,
enum i915_sw_fence_notify state);
-#define __i915_sw_fence_call __aligned(4)
+
+struct i915_sw_fence {
+ wait_queue_head_t wait;
+ i915_sw_fence_notify_t fn;
+#ifdef CONFIG_DRM_I915_SW_FENCE_CHECK_DAG
+ unsigned long flags;
+#endif
+ atomic_t pending;
+ int error;
+};
+
+#define I915_SW_FENCE_CHECKED_BIT 0 /* used internally for DAG checking */
void __i915_sw_fence_init(struct i915_sw_fence *fence,
i915_sw_fence_notify_t fn,
diff --git a/drivers/gpu/drm/i915/i915_sw_fence_work.c b/drivers/gpu/drm/i915/i915_sw_fence_work.c
index 5b33ef2..d2e56b3 100644
--- a/drivers/gpu/drm/i915/i915_sw_fence_work.c
+++ b/drivers/gpu/drm/i915/i915_sw_fence_work.c
@@ -23,7 +23,7 @@ static void fence_work(struct work_struct *work)
dma_fence_put(&f->dma);
}
-static int __i915_sw_fence_call
+static int
fence_notify(struct i915_sw_fence *fence, enum i915_sw_fence_notify state)
{
struct dma_fence_work *f = container_of(fence, typeof(*f), chain);
diff --git a/drivers/gpu/drm/i915/i915_switcheroo.c b/drivers/gpu/drm/i915/i915_switcheroo.c
index de0e224b..23777d5 100644
--- a/drivers/gpu/drm/i915/i915_switcheroo.c
+++ b/drivers/gpu/drm/i915/i915_switcheroo.c
@@ -5,6 +5,7 @@
#include <linux/vga_switcheroo.h>
+#include "i915_driver.h"
#include "i915_drv.h"
#include "i915_switcheroo.h"
@@ -24,12 +25,12 @@ static void i915_switcheroo_set_state(struct pci_dev *pdev,
i915->drm.switch_power_state = DRM_SWITCH_POWER_CHANGING;
/* i915 resume handler doesn't set to D0 */
pci_set_power_state(pdev, PCI_D0);
- i915_resume_switcheroo(i915);
+ i915_driver_resume_switcheroo(i915);
i915->drm.switch_power_state = DRM_SWITCH_POWER_ON;
} else {
drm_info(&i915->drm, "switched off\n");
i915->drm.switch_power_state = DRM_SWITCH_POWER_CHANGING;
- i915_suspend_switcheroo(i915, pmm);
+ i915_driver_suspend_switcheroo(i915, pmm);
i915->drm.switch_power_state = DRM_SWITCH_POWER_OFF;
}
}
diff --git a/drivers/gpu/drm/i915/i915_sysfs.c b/drivers/gpu/drm/i915/i915_sysfs.c
index 1804f41..fae4d1f 100644
--- a/drivers/gpu/drm/i915/i915_sysfs.c
+++ b/drivers/gpu/drm/i915/i915_sysfs.c
@@ -52,7 +52,7 @@ static u32 calc_residency(struct drm_i915_private *dev_priv,
u64 res = 0;
with_intel_runtime_pm(&dev_priv->runtime_pm, wakeref)
- res = intel_rc6_residency_us(&dev_priv->gt.rc6, reg);
+ res = intel_rc6_residency_us(&to_gt(dev_priv)->rc6, reg);
return DIV_ROUND_CLOSEST_ULL(res, 1000);
}
@@ -260,7 +260,7 @@ static ssize_t gt_act_freq_mhz_show(struct device *kdev,
struct device_attribute *attr, char *buf)
{
struct drm_i915_private *i915 = kdev_minor_to_i915(kdev);
- struct intel_rps *rps = &i915->gt.rps;
+ struct intel_rps *rps = &to_gt(i915)->rps;
return sysfs_emit(buf, "%d\n", intel_rps_read_actual_frequency(rps));
}
@@ -269,7 +269,7 @@ static ssize_t gt_cur_freq_mhz_show(struct device *kdev,
struct device_attribute *attr, char *buf)
{
struct drm_i915_private *i915 = kdev_minor_to_i915(kdev);
- struct intel_rps *rps = &i915->gt.rps;
+ struct intel_rps *rps = &to_gt(i915)->rps;
return sysfs_emit(buf, "%d\n", intel_rps_get_requested_frequency(rps));
}
@@ -277,9 +277,9 @@ static ssize_t gt_cur_freq_mhz_show(struct device *kdev,
static ssize_t gt_boost_freq_mhz_show(struct device *kdev, struct device_attribute *attr, char *buf)
{
struct drm_i915_private *i915 = kdev_minor_to_i915(kdev);
- struct intel_rps *rps = &i915->gt.rps;
+ struct intel_rps *rps = &to_gt(i915)->rps;
- return sysfs_emit(buf, "%d\n", intel_gpu_freq(rps, rps->boost_freq));
+ return sysfs_emit(buf, "%d\n", intel_rps_get_boost_frequency(rps));
}
static ssize_t gt_boost_freq_mhz_store(struct device *kdev,
@@ -287,8 +287,7 @@ static ssize_t gt_boost_freq_mhz_store(struct device *kdev,
const char *buf, size_t count)
{
struct drm_i915_private *dev_priv = kdev_minor_to_i915(kdev);
- struct intel_rps *rps = &dev_priv->gt.rps;
- bool boost = false;
+ struct intel_rps *rps = &to_gt(dev_priv)->rps;
ssize_t ret;
u32 val;
@@ -296,28 +295,16 @@ static ssize_t gt_boost_freq_mhz_store(struct device *kdev,
if (ret)
return ret;
- /* Validate against (static) hardware limits */
- val = intel_freq_opcode(rps, val);
- if (val < rps->min_freq || val > rps->max_freq)
- return -EINVAL;
+ ret = intel_rps_set_boost_frequency(rps, val);
- mutex_lock(&rps->lock);
- if (val != rps->boost_freq) {
- rps->boost_freq = val;
- boost = atomic_read(&rps->num_waiters);
- }
- mutex_unlock(&rps->lock);
- if (boost)
- schedule_work(&rps->work);
-
- return count;
+ return ret ?: count;
}
static ssize_t vlv_rpe_freq_mhz_show(struct device *kdev,
struct device_attribute *attr, char *buf)
{
struct drm_i915_private *dev_priv = kdev_minor_to_i915(kdev);
- struct intel_rps *rps = &dev_priv->gt.rps;
+ struct intel_rps *rps = &to_gt(dev_priv)->rps;
return sysfs_emit(buf, "%d\n", intel_gpu_freq(rps, rps->efficient_freq));
}
@@ -325,7 +312,7 @@ static ssize_t vlv_rpe_freq_mhz_show(struct device *kdev,
static ssize_t gt_max_freq_mhz_show(struct device *kdev, struct device_attribute *attr, char *buf)
{
struct drm_i915_private *dev_priv = kdev_minor_to_i915(kdev);
- struct intel_gt *gt = &dev_priv->gt;
+ struct intel_gt *gt = to_gt(dev_priv);
struct intel_rps *rps = >->rps;
return sysfs_emit(buf, "%d\n", intel_rps_get_max_frequency(rps));
@@ -336,7 +323,7 @@ static ssize_t gt_max_freq_mhz_store(struct device *kdev,
const char *buf, size_t count)
{
struct drm_i915_private *dev_priv = kdev_minor_to_i915(kdev);
- struct intel_gt *gt = &dev_priv->gt;
+ struct intel_gt *gt = to_gt(dev_priv);
struct intel_rps *rps = >->rps;
ssize_t ret;
u32 val;
@@ -353,7 +340,7 @@ static ssize_t gt_max_freq_mhz_store(struct device *kdev,
static ssize_t gt_min_freq_mhz_show(struct device *kdev, struct device_attribute *attr, char *buf)
{
struct drm_i915_private *i915 = kdev_minor_to_i915(kdev);
- struct intel_gt *gt = &i915->gt;
+ struct intel_gt *gt = to_gt(i915);
struct intel_rps *rps = >->rps;
return sysfs_emit(buf, "%d\n", intel_rps_get_min_frequency(rps));
@@ -364,7 +351,7 @@ static ssize_t gt_min_freq_mhz_store(struct device *kdev,
const char *buf, size_t count)
{
struct drm_i915_private *i915 = kdev_minor_to_i915(kdev);
- struct intel_rps *rps = &i915->gt.rps;
+ struct intel_rps *rps = &to_gt(i915)->rps;
ssize_t ret;
u32 val;
@@ -394,7 +381,7 @@ static DEVICE_ATTR(gt_RPn_freq_mhz, S_IRUGO, gt_rp_mhz_show, NULL);
static ssize_t gt_rp_mhz_show(struct device *kdev, struct device_attribute *attr, char *buf)
{
struct drm_i915_private *dev_priv = kdev_minor_to_i915(kdev);
- struct intel_rps *rps = &dev_priv->gt.rps;
+ struct intel_rps *rps = &to_gt(dev_priv)->rps;
u32 val;
if (attr == &dev_attr_gt_RP0_freq_mhz)
diff --git a/drivers/gpu/drm/i915/i915_trace.h b/drivers/gpu/drm/i915/i915_trace.h
index 8104981..37b5c9e 100644
--- a/drivers/gpu/drm/i915/i915_trace.h
+++ b/drivers/gpu/drm/i915/i915_trace.h
@@ -1,4 +1,8 @@
/* SPDX-License-Identifier: GPL-2.0 */
+
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM i915
+
#if !defined(_I915_TRACE_H_) || defined(TRACE_HEADER_MULTI_READ)
#define _I915_TRACE_H_
@@ -8,511 +12,11 @@
#include <drm/drm_drv.h>
-#include "display/intel_crtc.h"
-#include "display/intel_display_types.h"
#include "gt/intel_engine.h"
#include "i915_drv.h"
#include "i915_irq.h"
-#undef TRACE_SYSTEM
-#define TRACE_SYSTEM i915
-#define TRACE_INCLUDE_FILE i915_trace
-
-/* watermark/fifo updates */
-
-TRACE_EVENT(intel_pipe_enable,
- TP_PROTO(struct intel_crtc *crtc),
- TP_ARGS(crtc),
-
- TP_STRUCT__entry(
- __array(u32, frame, 3)
- __array(u32, scanline, 3)
- __field(enum pipe, pipe)
- ),
- TP_fast_assign(
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
- struct intel_crtc *it__;
- for_each_intel_crtc(&dev_priv->drm, it__) {
- __entry->frame[it__->pipe] = intel_crtc_get_vblank_counter(it__);
- __entry->scanline[it__->pipe] = intel_get_crtc_scanline(it__);
- }
- __entry->pipe = crtc->pipe;
- ),
-
- TP_printk("pipe %c enable, pipe A: frame=%u, scanline=%u, pipe B: frame=%u, scanline=%u, pipe C: frame=%u, scanline=%u",
- pipe_name(__entry->pipe),
- __entry->frame[PIPE_A], __entry->scanline[PIPE_A],
- __entry->frame[PIPE_B], __entry->scanline[PIPE_B],
- __entry->frame[PIPE_C], __entry->scanline[PIPE_C])
-);
-
-TRACE_EVENT(intel_pipe_disable,
- TP_PROTO(struct intel_crtc *crtc),
- TP_ARGS(crtc),
-
- TP_STRUCT__entry(
- __array(u32, frame, 3)
- __array(u32, scanline, 3)
- __field(enum pipe, pipe)
- ),
-
- TP_fast_assign(
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
- struct intel_crtc *it__;
- for_each_intel_crtc(&dev_priv->drm, it__) {
- __entry->frame[it__->pipe] = intel_crtc_get_vblank_counter(it__);
- __entry->scanline[it__->pipe] = intel_get_crtc_scanline(it__);
- }
- __entry->pipe = crtc->pipe;
- ),
-
- TP_printk("pipe %c disable, pipe A: frame=%u, scanline=%u, pipe B: frame=%u, scanline=%u, pipe C: frame=%u, scanline=%u",
- pipe_name(__entry->pipe),
- __entry->frame[PIPE_A], __entry->scanline[PIPE_A],
- __entry->frame[PIPE_B], __entry->scanline[PIPE_B],
- __entry->frame[PIPE_C], __entry->scanline[PIPE_C])
-);
-
-TRACE_EVENT(intel_pipe_crc,
- TP_PROTO(struct intel_crtc *crtc, const u32 *crcs),
- TP_ARGS(crtc, crcs),
-
- TP_STRUCT__entry(
- __field(enum pipe, pipe)
- __field(u32, frame)
- __field(u32, scanline)
- __array(u32, crcs, 5)
- ),
-
- TP_fast_assign(
- __entry->pipe = crtc->pipe;
- __entry->frame = intel_crtc_get_vblank_counter(crtc);
- __entry->scanline = intel_get_crtc_scanline(crtc);
- memcpy(__entry->crcs, crcs, sizeof(__entry->crcs));
- ),
-
- TP_printk("pipe %c, frame=%u, scanline=%u crc=%08x %08x %08x %08x %08x",
- pipe_name(__entry->pipe), __entry->frame, __entry->scanline,
- __entry->crcs[0], __entry->crcs[1], __entry->crcs[2],
- __entry->crcs[3], __entry->crcs[4])
-);
-
-TRACE_EVENT(intel_cpu_fifo_underrun,
- TP_PROTO(struct drm_i915_private *dev_priv, enum pipe pipe),
- TP_ARGS(dev_priv, pipe),
-
- TP_STRUCT__entry(
- __field(enum pipe, pipe)
- __field(u32, frame)
- __field(u32, scanline)
- ),
-
- TP_fast_assign(
- struct intel_crtc *crtc = intel_get_crtc_for_pipe(dev_priv, pipe);
- __entry->pipe = pipe;
- __entry->frame = intel_crtc_get_vblank_counter(crtc);
- __entry->scanline = intel_get_crtc_scanline(crtc);
- ),
-
- TP_printk("pipe %c, frame=%u, scanline=%u",
- pipe_name(__entry->pipe),
- __entry->frame, __entry->scanline)
-);
-
-TRACE_EVENT(intel_pch_fifo_underrun,
- TP_PROTO(struct drm_i915_private *dev_priv, enum pipe pch_transcoder),
- TP_ARGS(dev_priv, pch_transcoder),
-
- TP_STRUCT__entry(
- __field(enum pipe, pipe)
- __field(u32, frame)
- __field(u32, scanline)
- ),
-
- TP_fast_assign(
- enum pipe pipe = pch_transcoder;
- struct intel_crtc *crtc = intel_get_crtc_for_pipe(dev_priv, pipe);
- __entry->pipe = pipe;
- __entry->frame = intel_crtc_get_vblank_counter(crtc);
- __entry->scanline = intel_get_crtc_scanline(crtc);
- ),
-
- TP_printk("pch transcoder %c, frame=%u, scanline=%u",
- pipe_name(__entry->pipe),
- __entry->frame, __entry->scanline)
-);
-
-TRACE_EVENT(intel_memory_cxsr,
- TP_PROTO(struct drm_i915_private *dev_priv, bool old, bool new),
- TP_ARGS(dev_priv, old, new),
-
- TP_STRUCT__entry(
- __array(u32, frame, 3)
- __array(u32, scanline, 3)
- __field(bool, old)
- __field(bool, new)
- ),
-
- TP_fast_assign(
- struct intel_crtc *crtc;
- for_each_intel_crtc(&dev_priv->drm, crtc) {
- __entry->frame[crtc->pipe] = intel_crtc_get_vblank_counter(crtc);
- __entry->scanline[crtc->pipe] = intel_get_crtc_scanline(crtc);
- }
- __entry->old = old;
- __entry->new = new;
- ),
-
- TP_printk("%s->%s, pipe A: frame=%u, scanline=%u, pipe B: frame=%u, scanline=%u, pipe C: frame=%u, scanline=%u",
- onoff(__entry->old), onoff(__entry->new),
- __entry->frame[PIPE_A], __entry->scanline[PIPE_A],
- __entry->frame[PIPE_B], __entry->scanline[PIPE_B],
- __entry->frame[PIPE_C], __entry->scanline[PIPE_C])
-);
-
-TRACE_EVENT(g4x_wm,
- TP_PROTO(struct intel_crtc *crtc, const struct g4x_wm_values *wm),
- TP_ARGS(crtc, wm),
-
- TP_STRUCT__entry(
- __field(enum pipe, pipe)
- __field(u32, frame)
- __field(u32, scanline)
- __field(u16, primary)
- __field(u16, sprite)
- __field(u16, cursor)
- __field(u16, sr_plane)
- __field(u16, sr_cursor)
- __field(u16, sr_fbc)
- __field(u16, hpll_plane)
- __field(u16, hpll_cursor)
- __field(u16, hpll_fbc)
- __field(bool, cxsr)
- __field(bool, hpll)
- __field(bool, fbc)
- ),
-
- TP_fast_assign(
- __entry->pipe = crtc->pipe;
- __entry->frame = intel_crtc_get_vblank_counter(crtc);
- __entry->scanline = intel_get_crtc_scanline(crtc);
- __entry->primary = wm->pipe[crtc->pipe].plane[PLANE_PRIMARY];
- __entry->sprite = wm->pipe[crtc->pipe].plane[PLANE_SPRITE0];
- __entry->cursor = wm->pipe[crtc->pipe].plane[PLANE_CURSOR];
- __entry->sr_plane = wm->sr.plane;
- __entry->sr_cursor = wm->sr.cursor;
- __entry->sr_fbc = wm->sr.fbc;
- __entry->hpll_plane = wm->hpll.plane;
- __entry->hpll_cursor = wm->hpll.cursor;
- __entry->hpll_fbc = wm->hpll.fbc;
- __entry->cxsr = wm->cxsr;
- __entry->hpll = wm->hpll_en;
- __entry->fbc = wm->fbc_en;
- ),
-
- TP_printk("pipe %c, frame=%u, scanline=%u, wm %d/%d/%d, sr %s/%d/%d/%d, hpll %s/%d/%d/%d, fbc %s",
- pipe_name(__entry->pipe), __entry->frame, __entry->scanline,
- __entry->primary, __entry->sprite, __entry->cursor,
- yesno(__entry->cxsr), __entry->sr_plane, __entry->sr_cursor, __entry->sr_fbc,
- yesno(__entry->hpll), __entry->hpll_plane, __entry->hpll_cursor, __entry->hpll_fbc,
- yesno(__entry->fbc))
-);
-
-TRACE_EVENT(vlv_wm,
- TP_PROTO(struct intel_crtc *crtc, const struct vlv_wm_values *wm),
- TP_ARGS(crtc, wm),
-
- TP_STRUCT__entry(
- __field(enum pipe, pipe)
- __field(u32, frame)
- __field(u32, scanline)
- __field(u32, level)
- __field(u32, cxsr)
- __field(u32, primary)
- __field(u32, sprite0)
- __field(u32, sprite1)
- __field(u32, cursor)
- __field(u32, sr_plane)
- __field(u32, sr_cursor)
- ),
-
- TP_fast_assign(
- __entry->pipe = crtc->pipe;
- __entry->frame = intel_crtc_get_vblank_counter(crtc);
- __entry->scanline = intel_get_crtc_scanline(crtc);
- __entry->level = wm->level;
- __entry->cxsr = wm->cxsr;
- __entry->primary = wm->pipe[crtc->pipe].plane[PLANE_PRIMARY];
- __entry->sprite0 = wm->pipe[crtc->pipe].plane[PLANE_SPRITE0];
- __entry->sprite1 = wm->pipe[crtc->pipe].plane[PLANE_SPRITE1];
- __entry->cursor = wm->pipe[crtc->pipe].plane[PLANE_CURSOR];
- __entry->sr_plane = wm->sr.plane;
- __entry->sr_cursor = wm->sr.cursor;
- ),
-
- TP_printk("pipe %c, frame=%u, scanline=%u, level=%d, cxsr=%d, wm %d/%d/%d/%d, sr %d/%d",
- pipe_name(__entry->pipe), __entry->frame,
- __entry->scanline, __entry->level, __entry->cxsr,
- __entry->primary, __entry->sprite0, __entry->sprite1, __entry->cursor,
- __entry->sr_plane, __entry->sr_cursor)
-);
-
-TRACE_EVENT(vlv_fifo_size,
- TP_PROTO(struct intel_crtc *crtc, u32 sprite0_start, u32 sprite1_start, u32 fifo_size),
- TP_ARGS(crtc, sprite0_start, sprite1_start, fifo_size),
-
- TP_STRUCT__entry(
- __field(enum pipe, pipe)
- __field(u32, frame)
- __field(u32, scanline)
- __field(u32, sprite0_start)
- __field(u32, sprite1_start)
- __field(u32, fifo_size)
- ),
-
- TP_fast_assign(
- __entry->pipe = crtc->pipe;
- __entry->frame = intel_crtc_get_vblank_counter(crtc);
- __entry->scanline = intel_get_crtc_scanline(crtc);
- __entry->sprite0_start = sprite0_start;
- __entry->sprite1_start = sprite1_start;
- __entry->fifo_size = fifo_size;
- ),
-
- TP_printk("pipe %c, frame=%u, scanline=%u, %d/%d/%d",
- pipe_name(__entry->pipe), __entry->frame,
- __entry->scanline, __entry->sprite0_start,
- __entry->sprite1_start, __entry->fifo_size)
-);
-
-/* plane updates */
-
-TRACE_EVENT(intel_update_plane,
- TP_PROTO(struct drm_plane *plane, struct intel_crtc *crtc),
- TP_ARGS(plane, crtc),
-
- TP_STRUCT__entry(
- __field(enum pipe, pipe)
- __field(u32, frame)
- __field(u32, scanline)
- __array(int, src, 4)
- __array(int, dst, 4)
- __string(name, plane->name)
- ),
-
- TP_fast_assign(
- __assign_str(name, plane->name);
- __entry->pipe = crtc->pipe;
- __entry->frame = intel_crtc_get_vblank_counter(crtc);
- __entry->scanline = intel_get_crtc_scanline(crtc);
- memcpy(__entry->src, &plane->state->src, sizeof(__entry->src));
- memcpy(__entry->dst, &plane->state->dst, sizeof(__entry->dst));
- ),
-
- TP_printk("pipe %c, plane %s, frame=%u, scanline=%u, " DRM_RECT_FP_FMT " -> " DRM_RECT_FMT,
- pipe_name(__entry->pipe), __get_str(name),
- __entry->frame, __entry->scanline,
- DRM_RECT_FP_ARG((const struct drm_rect *)__entry->src),
- DRM_RECT_ARG((const struct drm_rect *)__entry->dst))
-);
-
-TRACE_EVENT(intel_disable_plane,
- TP_PROTO(struct drm_plane *plane, struct intel_crtc *crtc),
- TP_ARGS(plane, crtc),
-
- TP_STRUCT__entry(
- __field(enum pipe, pipe)
- __field(u32, frame)
- __field(u32, scanline)
- __string(name, plane->name)
- ),
-
- TP_fast_assign(
- __assign_str(name, plane->name);
- __entry->pipe = crtc->pipe;
- __entry->frame = intel_crtc_get_vblank_counter(crtc);
- __entry->scanline = intel_get_crtc_scanline(crtc);
- ),
-
- TP_printk("pipe %c, plane %s, frame=%u, scanline=%u",
- pipe_name(__entry->pipe), __get_str(name),
- __entry->frame, __entry->scanline)
-);
-
-/* fbc */
-
-TRACE_EVENT(intel_fbc_activate,
- TP_PROTO(struct intel_crtc *crtc),
- TP_ARGS(crtc),
-
- TP_STRUCT__entry(
- __field(enum pipe, pipe)
- __field(u32, frame)
- __field(u32, scanline)
- ),
-
- TP_fast_assign(
- __entry->pipe = crtc->pipe;
- __entry->frame = intel_crtc_get_vblank_counter(crtc);
- __entry->scanline = intel_get_crtc_scanline(crtc);
- ),
-
- TP_printk("pipe %c, frame=%u, scanline=%u",
- pipe_name(__entry->pipe), __entry->frame, __entry->scanline)
-);
-
-TRACE_EVENT(intel_fbc_deactivate,
- TP_PROTO(struct intel_crtc *crtc),
- TP_ARGS(crtc),
-
- TP_STRUCT__entry(
- __field(enum pipe, pipe)
- __field(u32, frame)
- __field(u32, scanline)
- ),
-
- TP_fast_assign(
- __entry->pipe = crtc->pipe;
- __entry->frame = intel_crtc_get_vblank_counter(crtc);
- __entry->scanline = intel_get_crtc_scanline(crtc);
- ),
-
- TP_printk("pipe %c, frame=%u, scanline=%u",
- pipe_name(__entry->pipe), __entry->frame, __entry->scanline)
-);
-
-TRACE_EVENT(intel_fbc_nuke,
- TP_PROTO(struct intel_crtc *crtc),
- TP_ARGS(crtc),
-
- TP_STRUCT__entry(
- __field(enum pipe, pipe)
- __field(u32, frame)
- __field(u32, scanline)
- ),
-
- TP_fast_assign(
- __entry->pipe = crtc->pipe;
- __entry->frame = intel_crtc_get_vblank_counter(crtc);
- __entry->scanline = intel_get_crtc_scanline(crtc);
- ),
-
- TP_printk("pipe %c, frame=%u, scanline=%u",
- pipe_name(__entry->pipe), __entry->frame, __entry->scanline)
-);
-
-/* pipe updates */
-
-TRACE_EVENT(intel_pipe_update_start,
- TP_PROTO(struct intel_crtc *crtc),
- TP_ARGS(crtc),
-
- TP_STRUCT__entry(
- __field(enum pipe, pipe)
- __field(u32, frame)
- __field(u32, scanline)
- __field(u32, min)
- __field(u32, max)
- ),
-
- TP_fast_assign(
- __entry->pipe = crtc->pipe;
- __entry->frame = intel_crtc_get_vblank_counter(crtc);
- __entry->scanline = intel_get_crtc_scanline(crtc);
- __entry->min = crtc->debug.min_vbl;
- __entry->max = crtc->debug.max_vbl;
- ),
-
- TP_printk("pipe %c, frame=%u, scanline=%u, min=%u, max=%u",
- pipe_name(__entry->pipe), __entry->frame,
- __entry->scanline, __entry->min, __entry->max)
-);
-
-TRACE_EVENT(intel_pipe_update_vblank_evaded,
- TP_PROTO(struct intel_crtc *crtc),
- TP_ARGS(crtc),
-
- TP_STRUCT__entry(
- __field(enum pipe, pipe)
- __field(u32, frame)
- __field(u32, scanline)
- __field(u32, min)
- __field(u32, max)
- ),
-
- TP_fast_assign(
- __entry->pipe = crtc->pipe;
- __entry->frame = crtc->debug.start_vbl_count;
- __entry->scanline = crtc->debug.scanline_start;
- __entry->min = crtc->debug.min_vbl;
- __entry->max = crtc->debug.max_vbl;
- ),
-
- TP_printk("pipe %c, frame=%u, scanline=%u, min=%u, max=%u",
- pipe_name(__entry->pipe), __entry->frame,
- __entry->scanline, __entry->min, __entry->max)
-);
-
-TRACE_EVENT(intel_pipe_update_end,
- TP_PROTO(struct intel_crtc *crtc, u32 frame, int scanline_end),
- TP_ARGS(crtc, frame, scanline_end),
-
- TP_STRUCT__entry(
- __field(enum pipe, pipe)
- __field(u32, frame)
- __field(u32, scanline)
- ),
-
- TP_fast_assign(
- __entry->pipe = crtc->pipe;
- __entry->frame = frame;
- __entry->scanline = scanline_end;
- ),
-
- TP_printk("pipe %c, frame=%u, scanline=%u",
- pipe_name(__entry->pipe), __entry->frame,
- __entry->scanline)
-);
-
-/* frontbuffer tracking */
-
-TRACE_EVENT(intel_frontbuffer_invalidate,
- TP_PROTO(unsigned int frontbuffer_bits, unsigned int origin),
- TP_ARGS(frontbuffer_bits, origin),
-
- TP_STRUCT__entry(
- __field(unsigned int, frontbuffer_bits)
- __field(unsigned int, origin)
- ),
-
- TP_fast_assign(
- __entry->frontbuffer_bits = frontbuffer_bits;
- __entry->origin = origin;
- ),
-
- TP_printk("frontbuffer_bits=0x%08x, origin=%u",
- __entry->frontbuffer_bits, __entry->origin)
-);
-
-TRACE_EVENT(intel_frontbuffer_flush,
- TP_PROTO(unsigned int frontbuffer_bits, unsigned int origin),
- TP_ARGS(frontbuffer_bits, origin),
-
- TP_STRUCT__entry(
- __field(unsigned int, frontbuffer_bits)
- __field(unsigned int, origin)
- ),
-
- TP_fast_assign(
- __entry->frontbuffer_bits = frontbuffer_bits;
- __entry->origin = origin;
- ),
-
- TP_printk("frontbuffer_bits=0x%08x, origin=%u",
- __entry->frontbuffer_bits, __entry->origin)
-);
-
/* object tracking */
TRACE_EVENT(i915_gem_object_create,
@@ -1260,5 +764,7 @@ DEFINE_EVENT(i915_context, i915_context_free,
/* This part must be outside protection */
#undef TRACE_INCLUDE_PATH
+#undef TRACE_INCLUDE_FILE
#define TRACE_INCLUDE_PATH ../../drivers/gpu/drm/i915
+#define TRACE_INCLUDE_FILE i915_trace
#include <trace/define_trace.h>
diff --git a/drivers/gpu/drm/i915/i915_vma.c b/drivers/gpu/drm/i915/i915_vma.c
index bef795e..29a858c 100644
--- a/drivers/gpu/drm/i915/i915_vma.c
+++ b/drivers/gpu/drm/i915/i915_vma.c
@@ -40,12 +40,12 @@
static struct kmem_cache *slab_vmas;
-struct i915_vma *i915_vma_alloc(void)
+static struct i915_vma *i915_vma_alloc(void)
{
return kmem_cache_zalloc(slab_vmas, GFP_KERNEL);
}
-void i915_vma_free(struct i915_vma *vma)
+static void i915_vma_free(struct i915_vma *vma)
{
return kmem_cache_free(slab_vmas, vma);
}
@@ -109,11 +109,9 @@ vma_create(struct drm_i915_gem_object *obj,
return ERR_PTR(-ENOMEM);
kref_init(&vma->ref);
- mutex_init(&vma->pages_mutex);
vma->vm = i915_vm_get(vm);
vma->ops = &vm->vma_ops;
vma->obj = obj;
- vma->resv = obj->base.resv;
vma->size = obj->base.size;
vma->display_alignment = I915_GTT_MIN_ALIGNMENT;
@@ -346,7 +344,7 @@ int i915_vma_wait_for_bind(struct i915_vma *vma)
fence = dma_fence_get_rcu_safe(&vma->active.excl.fence);
rcu_read_unlock();
if (fence) {
- err = dma_fence_wait(fence, MAX_SCHEDULE_TIMEOUT);
+ err = dma_fence_wait(fence, true);
dma_fence_put(fence);
}
}
@@ -354,6 +352,28 @@ int i915_vma_wait_for_bind(struct i915_vma *vma)
return err;
}
+#if IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM)
+static int i915_vma_verify_bind_complete(struct i915_vma *vma)
+{
+ struct dma_fence *fence = i915_active_fence_get(&vma->active.excl);
+ int err;
+
+ if (!fence)
+ return 0;
+
+ if (dma_fence_is_signaled(fence))
+ err = fence->error;
+ else
+ err = -EBUSY;
+
+ dma_fence_put(fence);
+
+ return err;
+}
+#else
+#define i915_vma_verify_bind_complete(_vma) 0
+#endif
+
/**
* i915_vma_bind - Sets up PTEs for an VMA in it's corresponding address space.
* @vma: VMA to map
@@ -373,6 +393,7 @@ int i915_vma_bind(struct i915_vma *vma,
u32 bind_flags;
u32 vma_flags;
+ lockdep_assert_held(&vma->vm->mutex);
GEM_BUG_ON(!drm_mm_node_allocated(&vma->node));
GEM_BUG_ON(vma->size > vma->node.size);
@@ -394,7 +415,7 @@ int i915_vma_bind(struct i915_vma *vma,
if (bind_flags == 0)
return 0;
- GEM_BUG_ON(!vma->pages);
+ GEM_BUG_ON(!atomic_read(&vma->pages_count));
trace_i915_vma_bind(vma, bind_flags);
if (work && bind_flags & vma->vm->bind_async_flags) {
@@ -423,11 +444,16 @@ int i915_vma_bind(struct i915_vma *vma,
work->base.dma.error = 0; /* enable the queue_work() */
- if (vma->obj) {
- __i915_gem_object_pin_pages(vma->obj);
- work->pinned = i915_gem_object_get(vma->obj);
- }
+ __i915_gem_object_pin_pages(vma->obj);
+ work->pinned = i915_gem_object_get(vma->obj);
} else {
+ if (vma->obj) {
+ int ret;
+
+ ret = i915_gem_object_wait_moving_fence(vma->obj, true);
+ if (ret)
+ return ret;
+ }
vma->ops->bind_vma(vma->vm, NULL, vma, cache_level, bind_flags);
}
@@ -449,6 +475,7 @@ void __iomem *i915_vma_pin_iomap(struct i915_vma *vma)
GEM_BUG_ON(!i915_vma_is_ggtt(vma));
GEM_BUG_ON(!i915_vma_is_bound(vma, I915_VMA_GLOBAL_BIND));
+ GEM_BUG_ON(i915_vma_verify_bind_complete(vma));
ptr = READ_ONCE(vma->iomap);
if (ptr == NULL) {
@@ -667,7 +694,7 @@ i915_vma_insert(struct i915_vma *vma, u64 size, u64 alignment, u64 flags)
}
color = 0;
- if (vma->obj && i915_vm_has_cache_coloring(vma->vm))
+ if (i915_vm_has_cache_coloring(vma->vm))
color = vma->obj->cache_level;
if (flags & PIN_OFFSET_FIXED) {
@@ -789,40 +816,356 @@ static bool try_qad_pin(struct i915_vma *vma, unsigned int flags)
return pinned;
}
-static int vma_get_pages(struct i915_vma *vma)
+static struct scatterlist *
+rotate_pages(struct drm_i915_gem_object *obj, unsigned int offset,
+ unsigned int width, unsigned int height,
+ unsigned int src_stride, unsigned int dst_stride,
+ struct sg_table *st, struct scatterlist *sg)
{
- int err = 0;
- bool pinned_pages = false;
+ unsigned int column, row;
+ unsigned int src_idx;
+
+ for (column = 0; column < width; column++) {
+ unsigned int left;
+
+ src_idx = src_stride * (height - 1) + column + offset;
+ for (row = 0; row < height; row++) {
+ st->nents++;
+ /*
+ * We don't need the pages, but need to initialize
+ * the entries so the sg list can be happily traversed.
+ * The only thing we need are DMA addresses.
+ */
+ sg_set_page(sg, NULL, I915_GTT_PAGE_SIZE, 0);
+ sg_dma_address(sg) =
+ i915_gem_object_get_dma_address(obj, src_idx);
+ sg_dma_len(sg) = I915_GTT_PAGE_SIZE;
+ sg = sg_next(sg);
+ src_idx -= src_stride;
+ }
+
+ left = (dst_stride - height) * I915_GTT_PAGE_SIZE;
+
+ if (!left)
+ continue;
+
+ st->nents++;
+
+ /*
+ * The DE ignores the PTEs for the padding tiles, the sg entry
+ * here is just a conenience to indicate how many padding PTEs
+ * to insert at this spot.
+ */
+ sg_set_page(sg, NULL, left, 0);
+ sg_dma_address(sg) = 0;
+ sg_dma_len(sg) = left;
+ sg = sg_next(sg);
+ }
+
+ return sg;
+}
+
+static noinline struct sg_table *
+intel_rotate_pages(struct intel_rotation_info *rot_info,
+ struct drm_i915_gem_object *obj)
+{
+ unsigned int size = intel_rotation_info_size(rot_info);
+ struct drm_i915_private *i915 = to_i915(obj->base.dev);
+ struct sg_table *st;
+ struct scatterlist *sg;
+ int ret = -ENOMEM;
+ int i;
+
+ /* Allocate target SG list. */
+ st = kmalloc(sizeof(*st), GFP_KERNEL);
+ if (!st)
+ goto err_st_alloc;
+
+ ret = sg_alloc_table(st, size, GFP_KERNEL);
+ if (ret)
+ goto err_sg_alloc;
+
+ st->nents = 0;
+ sg = st->sgl;
+
+ for (i = 0 ; i < ARRAY_SIZE(rot_info->plane); i++)
+ sg = rotate_pages(obj, rot_info->plane[i].offset,
+ rot_info->plane[i].width, rot_info->plane[i].height,
+ rot_info->plane[i].src_stride,
+ rot_info->plane[i].dst_stride,
+ st, sg);
+
+ return st;
+
+err_sg_alloc:
+ kfree(st);
+err_st_alloc:
+
+ drm_dbg(&i915->drm, "Failed to create rotated mapping for object size %zu! (%ux%u tiles, %u pages)\n",
+ obj->base.size, rot_info->plane[0].width,
+ rot_info->plane[0].height, size);
+
+ return ERR_PTR(ret);
+}
+
+static struct scatterlist *
+remap_pages(struct drm_i915_gem_object *obj,
+ unsigned int offset, unsigned int alignment_pad,
+ unsigned int width, unsigned int height,
+ unsigned int src_stride, unsigned int dst_stride,
+ struct sg_table *st, struct scatterlist *sg)
+{
+ unsigned int row;
+
+ if (!width || !height)
+ return sg;
+
+ if (alignment_pad) {
+ st->nents++;
+
+ /*
+ * The DE ignores the PTEs for the padding tiles, the sg entry
+ * here is just a convenience to indicate how many padding PTEs
+ * to insert at this spot.
+ */
+ sg_set_page(sg, NULL, alignment_pad * 4096, 0);
+ sg_dma_address(sg) = 0;
+ sg_dma_len(sg) = alignment_pad * 4096;
+ sg = sg_next(sg);
+ }
+
+ for (row = 0; row < height; row++) {
+ unsigned int left = width * I915_GTT_PAGE_SIZE;
+
+ while (left) {
+ dma_addr_t addr;
+ unsigned int length;
+
+ /*
+ * We don't need the pages, but need to initialize
+ * the entries so the sg list can be happily traversed.
+ * The only thing we need are DMA addresses.
+ */
+
+ addr = i915_gem_object_get_dma_address_len(obj, offset, &length);
+
+ length = min(left, length);
+
+ st->nents++;
+
+ sg_set_page(sg, NULL, length, 0);
+ sg_dma_address(sg) = addr;
+ sg_dma_len(sg) = length;
+ sg = sg_next(sg);
+
+ offset += length / I915_GTT_PAGE_SIZE;
+ left -= length;
+ }
+
+ offset += src_stride - width;
+
+ left = (dst_stride - width) * I915_GTT_PAGE_SIZE;
+
+ if (!left)
+ continue;
+
+ st->nents++;
+
+ /*
+ * The DE ignores the PTEs for the padding tiles, the sg entry
+ * here is just a conenience to indicate how many padding PTEs
+ * to insert at this spot.
+ */
+ sg_set_page(sg, NULL, left, 0);
+ sg_dma_address(sg) = 0;
+ sg_dma_len(sg) = left;
+ sg = sg_next(sg);
+ }
+
+ return sg;
+}
+
+static noinline struct sg_table *
+intel_remap_pages(struct intel_remapped_info *rem_info,
+ struct drm_i915_gem_object *obj)
+{
+ unsigned int size = intel_remapped_info_size(rem_info);
+ struct drm_i915_private *i915 = to_i915(obj->base.dev);
+ struct sg_table *st;
+ struct scatterlist *sg;
+ unsigned int gtt_offset = 0;
+ int ret = -ENOMEM;
+ int i;
+
+ /* Allocate target SG list. */
+ st = kmalloc(sizeof(*st), GFP_KERNEL);
+ if (!st)
+ goto err_st_alloc;
+
+ ret = sg_alloc_table(st, size, GFP_KERNEL);
+ if (ret)
+ goto err_sg_alloc;
+
+ st->nents = 0;
+ sg = st->sgl;
+
+ for (i = 0 ; i < ARRAY_SIZE(rem_info->plane); i++) {
+ unsigned int alignment_pad = 0;
+
+ if (rem_info->plane_alignment)
+ alignment_pad = ALIGN(gtt_offset, rem_info->plane_alignment) - gtt_offset;
+
+ sg = remap_pages(obj,
+ rem_info->plane[i].offset, alignment_pad,
+ rem_info->plane[i].width, rem_info->plane[i].height,
+ rem_info->plane[i].src_stride, rem_info->plane[i].dst_stride,
+ st, sg);
+
+ gtt_offset += alignment_pad +
+ rem_info->plane[i].dst_stride * rem_info->plane[i].height;
+ }
+
+ i915_sg_trim(st);
+
+ return st;
+
+err_sg_alloc:
+ kfree(st);
+err_st_alloc:
+
+ drm_dbg(&i915->drm, "Failed to create remapped mapping for object size %zu! (%ux%u tiles, %u pages)\n",
+ obj->base.size, rem_info->plane[0].width,
+ rem_info->plane[0].height, size);
+
+ return ERR_PTR(ret);
+}
+
+static noinline struct sg_table *
+intel_partial_pages(const struct i915_ggtt_view *view,
+ struct drm_i915_gem_object *obj)
+{
+ struct sg_table *st;
+ struct scatterlist *sg, *iter;
+ unsigned int count = view->partial.size;
+ unsigned int offset;
+ int ret = -ENOMEM;
+
+ st = kmalloc(sizeof(*st), GFP_KERNEL);
+ if (!st)
+ goto err_st_alloc;
+
+ ret = sg_alloc_table(st, count, GFP_KERNEL);
+ if (ret)
+ goto err_sg_alloc;
+
+ iter = i915_gem_object_get_sg_dma(obj, view->partial.offset, &offset);
+ GEM_BUG_ON(!iter);
+
+ sg = st->sgl;
+ st->nents = 0;
+ do {
+ unsigned int len;
+
+ len = min(sg_dma_len(iter) - (offset << PAGE_SHIFT),
+ count << PAGE_SHIFT);
+ sg_set_page(sg, NULL, len, 0);
+ sg_dma_address(sg) =
+ sg_dma_address(iter) + (offset << PAGE_SHIFT);
+ sg_dma_len(sg) = len;
+
+ st->nents++;
+ count -= len >> PAGE_SHIFT;
+ if (count == 0) {
+ sg_mark_end(sg);
+ i915_sg_trim(st); /* Drop any unused tail entries. */
+
+ return st;
+ }
+
+ sg = __sg_next(sg);
+ iter = __sg_next(iter);
+ offset = 0;
+ } while (1);
+
+err_sg_alloc:
+ kfree(st);
+err_st_alloc:
+ return ERR_PTR(ret);
+}
+
+static int
+__i915_vma_get_pages(struct i915_vma *vma)
+{
+ struct sg_table *pages;
+ int ret;
+
+ /*
+ * The vma->pages are only valid within the lifespan of the borrowed
+ * obj->mm.pages. When the obj->mm.pages sg_table is regenerated, so
+ * must be the vma->pages. A simple rule is that vma->pages must only
+ * be accessed when the obj->mm.pages are pinned.
+ */
+ GEM_BUG_ON(!i915_gem_object_has_pinned_pages(vma->obj));
+
+ switch (vma->ggtt_view.type) {
+ default:
+ GEM_BUG_ON(vma->ggtt_view.type);
+ fallthrough;
+ case I915_GGTT_VIEW_NORMAL:
+ pages = vma->obj->mm.pages;
+ break;
+
+ case I915_GGTT_VIEW_ROTATED:
+ pages =
+ intel_rotate_pages(&vma->ggtt_view.rotated, vma->obj);
+ break;
+
+ case I915_GGTT_VIEW_REMAPPED:
+ pages =
+ intel_remap_pages(&vma->ggtt_view.remapped, vma->obj);
+ break;
+
+ case I915_GGTT_VIEW_PARTIAL:
+ pages = intel_partial_pages(&vma->ggtt_view, vma->obj);
+ break;
+ }
+
+ ret = 0;
+ if (IS_ERR(pages)) {
+ ret = PTR_ERR(pages);
+ pages = NULL;
+ drm_err(&vma->vm->i915->drm,
+ "Failed to get pages for VMA view type %u (%d)!\n",
+ vma->ggtt_view.type, ret);
+ }
+
+ vma->pages = pages;
+
+ return ret;
+}
+
+I915_SELFTEST_EXPORT int i915_vma_get_pages(struct i915_vma *vma)
+{
+ int err;
if (atomic_add_unless(&vma->pages_count, 1, 0))
return 0;
- if (vma->obj) {
- err = i915_gem_object_pin_pages(vma->obj);
- if (err)
- return err;
- pinned_pages = true;
- }
+ err = i915_gem_object_pin_pages(vma->obj);
+ if (err)
+ return err;
- /* Allocations ahoy! */
- if (mutex_lock_interruptible(&vma->pages_mutex)) {
- err = -EINTR;
- goto unpin;
- }
+ err = __i915_vma_get_pages(vma);
+ if (err)
+ goto err_unpin;
- if (!atomic_read(&vma->pages_count)) {
- err = vma->ops->set_pages(vma);
- if (err)
- goto unlock;
- pinned_pages = false;
- }
+ vma->page_sizes = vma->obj->mm.page_sizes;
atomic_inc(&vma->pages_count);
-unlock:
- mutex_unlock(&vma->pages_mutex);
-unpin:
- if (pinned_pages)
- __i915_gem_object_unpin_pages(vma->obj);
+ return 0;
+
+err_unpin:
+ __i915_gem_object_unpin_pages(vma->obj);
return err;
}
@@ -830,18 +1173,31 @@ static int vma_get_pages(struct i915_vma *vma)
static void __vma_put_pages(struct i915_vma *vma, unsigned int count)
{
/* We allocate under vma_get_pages, so beware the shrinker */
- mutex_lock_nested(&vma->pages_mutex, SINGLE_DEPTH_NESTING);
+ struct sg_table *pages = READ_ONCE(vma->pages);
+
GEM_BUG_ON(atomic_read(&vma->pages_count) < count);
+
if (atomic_sub_return(count, &vma->pages_count) == 0) {
- vma->ops->clear_pages(vma);
- GEM_BUG_ON(vma->pages);
- if (vma->obj)
- i915_gem_object_unpin_pages(vma->obj);
+ /*
+ * The atomic_sub_return is a read barrier for the READ_ONCE of
+ * vma->pages above.
+ *
+ * READ_ONCE is safe because this is either called from the same
+ * function (i915_vma_pin_ww), or guarded by vma->vm->mutex.
+ *
+ * TODO: We're leaving vma->pages dangling, until vma->obj->resv
+ * lock is required.
+ */
+ if (pages != vma->obj->mm.pages) {
+ sg_free_table(pages);
+ kfree(pages);
+ }
+
+ i915_gem_object_unpin_pages(vma->obj);
}
- mutex_unlock(&vma->pages_mutex);
}
-static void vma_put_pages(struct i915_vma *vma)
+I915_SELFTEST_EXPORT void i915_vma_put_pages(struct i915_vma *vma)
{
if (atomic_add_unless(&vma->pages_count, -1, 1))
return;
@@ -867,14 +1223,13 @@ int i915_vma_pin_ww(struct i915_vma *vma, struct i915_gem_ww_ctx *ww,
u64 size, u64 alignment, u64 flags)
{
struct i915_vma_work *work = NULL;
+ struct dma_fence *moving = NULL;
intel_wakeref_t wakeref = 0;
unsigned int bound;
int err;
-#ifdef CONFIG_PROVE_LOCKING
- if (debug_locks && !WARN_ON(!ww) && vma->resv)
- assert_vma_held(vma);
-#endif
+ assert_vma_held(vma);
+ GEM_BUG_ON(!ww);
BUILD_BUG_ON(PIN_GLOBAL != I915_VMA_GLOBAL_BIND);
BUILD_BUG_ON(PIN_USER != I915_VMA_LOCAL_BIND);
@@ -885,14 +1240,15 @@ int i915_vma_pin_ww(struct i915_vma *vma, struct i915_gem_ww_ctx *ww,
if (try_qad_pin(vma, flags & I915_VMA_BIND_MASK))
return 0;
- err = vma_get_pages(vma);
+ err = i915_vma_get_pages(vma);
if (err)
return err;
if (flags & PIN_GLOBAL)
wakeref = intel_runtime_pm_get(&vma->vm->i915->runtime_pm);
- if (flags & vma->vm->bind_async_flags) {
+ moving = vma->obj ? i915_gem_object_get_moving_fence(vma->obj) : NULL;
+ if (flags & vma->vm->bind_async_flags || moving) {
/* lock VM */
err = i915_vm_lock_objects(vma->vm, ww);
if (err)
@@ -906,6 +1262,8 @@ int i915_vma_pin_ww(struct i915_vma *vma, struct i915_gem_ww_ctx *ww,
work->vm = i915_vm_get(vma->vm);
+ dma_fence_work_chain(&work->base, moving);
+
/* Allocate enough page directories to used PTE */
if (vma->vm->allocate_va_range) {
err = i915_vm_alloc_pt_stash(vma->vm,
@@ -980,7 +1338,7 @@ int i915_vma_pin_ww(struct i915_vma *vma, struct i915_gem_ww_ctx *ww,
GEM_BUG_ON(!vma->pages);
err = i915_vma_bind(vma,
- vma->obj ? vma->obj->cache_level : 0,
+ vma->obj->cache_level,
flags, work);
if (err)
goto err_remove;
@@ -1010,7 +1368,11 @@ int i915_vma_pin_ww(struct i915_vma *vma, struct i915_gem_ww_ctx *ww,
err_rpm:
if (wakeref)
intel_runtime_pm_put(&vma->vm->i915->runtime_pm, wakeref);
- vma_put_pages(vma);
+
+ if (moving)
+ dma_fence_put(moving);
+
+ i915_vma_put_pages(vma);
return err;
}
@@ -1025,23 +1387,15 @@ static void flush_idle_contexts(struct intel_gt *gt)
intel_gt_wait_for_idle(gt, MAX_SCHEDULE_TIMEOUT);
}
-int i915_ggtt_pin(struct i915_vma *vma, struct i915_gem_ww_ctx *ww,
- u32 align, unsigned int flags)
+static int __i915_ggtt_pin(struct i915_vma *vma, struct i915_gem_ww_ctx *ww,
+ u32 align, unsigned int flags)
{
struct i915_address_space *vm = vma->vm;
int err;
- GEM_BUG_ON(!i915_vma_is_ggtt(vma));
-
-#ifdef CONFIG_LOCKDEP
- WARN_ON(!ww && vma->resv && dma_resv_held(vma->resv));
-#endif
-
do {
- if (ww)
- err = i915_vma_pin_ww(vma, ww, 0, align, flags | PIN_GLOBAL);
- else
- err = i915_vma_pin(vma, 0, align, flags | PIN_GLOBAL);
+ err = i915_vma_pin_ww(vma, ww, 0, align, flags | PIN_GLOBAL);
+
if (err != -ENOSPC) {
if (!err) {
err = i915_vma_wait_for_bind(vma);
@@ -1060,6 +1414,30 @@ int i915_ggtt_pin(struct i915_vma *vma, struct i915_gem_ww_ctx *ww,
} while (1);
}
+int i915_ggtt_pin(struct i915_vma *vma, struct i915_gem_ww_ctx *ww,
+ u32 align, unsigned int flags)
+{
+ struct i915_gem_ww_ctx _ww;
+ int err;
+
+ GEM_BUG_ON(!i915_vma_is_ggtt(vma));
+
+ if (ww)
+ return __i915_ggtt_pin(vma, ww, align, flags);
+
+#ifdef CONFIG_LOCKDEP
+ WARN_ON(dma_resv_held(vma->obj->base.resv));
+#endif
+
+ for_i915_gem_ww(&_ww, err, true) {
+ err = i915_gem_object_lock(vma->obj, &_ww);
+ if (!err)
+ err = __i915_ggtt_pin(vma, &_ww, align, flags);
+ }
+
+ return err;
+}
+
static void __vma_close(struct i915_vma *vma, struct intel_gt *gt)
{
/*
@@ -1113,6 +1491,7 @@ void i915_vma_reopen(struct i915_vma *vma)
void i915_vma_release(struct kref *ref)
{
struct i915_vma *vma = container_of(ref, typeof(*vma), ref);
+ struct drm_i915_gem_object *obj = vma->obj;
if (drm_mm_node_allocated(&vma->node)) {
mutex_lock(&vma->vm->mutex);
@@ -1123,15 +1502,11 @@ void i915_vma_release(struct kref *ref)
}
GEM_BUG_ON(i915_vma_is_active(vma));
- if (vma->obj) {
- struct drm_i915_gem_object *obj = vma->obj;
-
- spin_lock(&obj->vma.lock);
- list_del(&vma->obj_link);
- if (!RB_EMPTY_NODE(&vma->obj_node))
- rb_erase(&vma->obj_node, &obj->vma.tree);
- spin_unlock(&obj->vma.lock);
- }
+ spin_lock(&obj->vma.lock);
+ list_del(&vma->obj_link);
+ if (!RB_EMPTY_NODE(&vma->obj_node))
+ rb_erase(&vma->obj_node, &obj->vma.tree);
+ spin_unlock(&obj->vma.lock);
__i915_vma_remove_closed(vma);
i915_vm_put(vma->vm);
@@ -1217,7 +1592,7 @@ __i915_request_await_bind(struct i915_request *rq, struct i915_vma *vma)
return __i915_request_await_exclusive(rq, &vma->active);
}
-int __i915_vma_move_to_active(struct i915_vma *vma, struct i915_request *rq)
+static int __i915_vma_move_to_active(struct i915_vma *vma, struct i915_request *rq)
{
int err;
@@ -1256,19 +1631,19 @@ int _i915_vma_move_to_active(struct i915_vma *vma,
}
if (fence) {
- dma_resv_add_excl_fence(vma->resv, fence);
+ dma_resv_add_excl_fence(vma->obj->base.resv, fence);
obj->write_domain = I915_GEM_DOMAIN_RENDER;
obj->read_domains = 0;
}
} else {
if (!(flags & __EXEC_OBJECT_NO_RESERVE)) {
- err = dma_resv_reserve_shared(vma->resv, 1);
+ err = dma_resv_reserve_shared(vma->obj->base.resv, 1);
if (unlikely(err))
return err;
}
if (fence) {
- dma_resv_add_shared_fence(vma->resv, fence);
+ dma_resv_add_shared_fence(vma->obj->base.resv, fence);
obj->write_domain = 0;
}
}
diff --git a/drivers/gpu/drm/i915/i915_vma.h b/drivers/gpu/drm/i915/i915_vma.h
index 648dbe7..3271943 100644
--- a/drivers/gpu/drm/i915/i915_vma.h
+++ b/drivers/gpu/drm/i915/i915_vma.h
@@ -55,8 +55,6 @@ static inline bool i915_vma_is_active(const struct i915_vma *vma)
/* do not reserve memory to prevent deadlocks */
#define __EXEC_OBJECT_NO_RESERVE BIT(31)
-int __must_check __i915_vma_move_to_active(struct i915_vma *vma,
- struct i915_request *rq);
int __must_check _i915_vma_move_to_active(struct i915_vma *vma,
struct i915_request *rq,
struct dma_fence *fence,
@@ -234,16 +232,16 @@ static inline void __i915_vma_put(struct i915_vma *vma)
kref_put(&vma->ref, i915_vma_release);
}
-#define assert_vma_held(vma) dma_resv_assert_held((vma)->resv)
+#define assert_vma_held(vma) dma_resv_assert_held((vma)->obj->base.resv)
static inline void i915_vma_lock(struct i915_vma *vma)
{
- dma_resv_lock(vma->resv, NULL);
+ dma_resv_lock(vma->obj->base.resv, NULL);
}
static inline void i915_vma_unlock(struct i915_vma *vma)
{
- dma_resv_unlock(vma->resv);
+ dma_resv_unlock(vma->obj->base.resv);
}
int __must_check
@@ -418,9 +416,6 @@ static inline void i915_vma_clear_scanout(struct i915_vma *vma)
list_for_each_entry(V, &(OBJ)->vma.list, obj_link) \
for_each_until(!i915_vma_is_ggtt(V))
-struct i915_vma *i915_vma_alloc(void);
-void i915_vma_free(struct i915_vma *vma);
-
struct i915_vma *i915_vma_make_unshrinkable(struct i915_vma *vma);
void i915_vma_make_shrinkable(struct i915_vma *vma);
void i915_vma_make_purgeable(struct i915_vma *vma);
@@ -436,4 +431,7 @@ static inline int i915_vma_sync(struct i915_vma *vma)
void i915_vma_module_exit(void);
int i915_vma_module_init(void);
+I915_SELFTEST_DECLARE(int i915_vma_get_pages(struct i915_vma *vma));
+I915_SELFTEST_DECLARE(void i915_vma_put_pages(struct i915_vma *vma));
+
#endif
diff --git a/drivers/gpu/drm/i915/i915_vma_snapshot.c b/drivers/gpu/drm/i915/i915_vma_snapshot.c
new file mode 100644
index 0000000..2949cee
--- /dev/null
+++ b/drivers/gpu/drm/i915/i915_vma_snapshot.c
@@ -0,0 +1,134 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright © 2021 Intel Corporation
+ */
+
+#include "i915_vma_snapshot.h"
+#include "i915_vma_types.h"
+#include "i915_vma.h"
+
+/**
+ * i915_vma_snapshot_init - Initialize a struct i915_vma_snapshot from
+ * a struct i915_vma.
+ * @vsnap: The i915_vma_snapshot to init.
+ * @vma: A struct i915_vma used to initialize @vsnap.
+ * @name: Name associated with the snapshot. The character pointer needs to
+ * stay alive over the lifitime of the shapsot
+ */
+void i915_vma_snapshot_init(struct i915_vma_snapshot *vsnap,
+ struct i915_vma *vma,
+ const char *name)
+{
+ if (!i915_vma_is_pinned(vma))
+ assert_object_held(vma->obj);
+
+ vsnap->name = name;
+ vsnap->size = vma->size;
+ vsnap->obj_size = vma->obj->base.size;
+ vsnap->gtt_offset = vma->node.start;
+ vsnap->gtt_size = vma->node.size;
+ vsnap->page_sizes = vma->page_sizes.gtt;
+ vsnap->pages = vma->pages;
+ vsnap->pages_rsgt = NULL;
+ vsnap->mr = NULL;
+ if (vma->obj->mm.rsgt)
+ vsnap->pages_rsgt = i915_refct_sgt_get(vma->obj->mm.rsgt);
+ vsnap->mr = vma->obj->mm.region;
+ kref_init(&vsnap->kref);
+ vsnap->vma_resource = &vma->active;
+ vsnap->onstack = false;
+ vsnap->present = true;
+}
+
+/**
+ * i915_vma_snapshot_init_onstack - Initialize a struct i915_vma_snapshot from
+ * a struct i915_vma, but avoid kfreeing it on last put.
+ * @vsnap: The i915_vma_snapshot to init.
+ * @vma: A struct i915_vma used to initialize @vsnap.
+ * @name: Name associated with the snapshot. The character pointer needs to
+ * stay alive over the lifitime of the shapsot
+ */
+void i915_vma_snapshot_init_onstack(struct i915_vma_snapshot *vsnap,
+ struct i915_vma *vma,
+ const char *name)
+{
+ i915_vma_snapshot_init(vsnap, vma, name);
+ vsnap->onstack = true;
+}
+
+static void vma_snapshot_release(struct kref *ref)
+{
+ struct i915_vma_snapshot *vsnap =
+ container_of(ref, typeof(*vsnap), kref);
+
+ vsnap->present = false;
+ if (vsnap->pages_rsgt)
+ i915_refct_sgt_put(vsnap->pages_rsgt);
+ if (!vsnap->onstack)
+ kfree(vsnap);
+}
+
+/**
+ * i915_vma_snapshot_put - Put an i915_vma_snapshot pointer reference
+ * @vsnap: The pointer reference
+ */
+void i915_vma_snapshot_put(struct i915_vma_snapshot *vsnap)
+{
+ kref_put(&vsnap->kref, vma_snapshot_release);
+}
+
+/**
+ * i915_vma_snapshot_put_onstack - Put an onstcak i915_vma_snapshot pointer
+ * reference and varify that the structure is released
+ * @vsnap: The pointer reference
+ *
+ * This function is intended to be paired with a i915_vma_init_onstack()
+ * and should be called before exiting the scope that declared or
+ * freeing the structure that embedded @vsnap to verify that all references
+ * have been released.
+ */
+void i915_vma_snapshot_put_onstack(struct i915_vma_snapshot *vsnap)
+{
+ if (!kref_put(&vsnap->kref, vma_snapshot_release))
+ GEM_BUG_ON(1);
+}
+
+/**
+ * i915_vma_snapshot_resource_pin - Temporarily block the memory the
+ * vma snapshot is pointing to from being released.
+ * @vsnap: The vma snapshot.
+ * @lockdep_cookie: Pointer to bool needed for lockdep support. This needs
+ * to be passed to the paired i915_vma_snapshot_resource_unpin.
+ *
+ * This function will temporarily try to hold up a fence or similar structure
+ * and will therefore enter a fence signaling critical section.
+ *
+ * Return: true if we succeeded in blocking the memory from being released,
+ * false otherwise.
+ */
+bool i915_vma_snapshot_resource_pin(struct i915_vma_snapshot *vsnap,
+ bool *lockdep_cookie)
+{
+ bool pinned = i915_active_acquire_if_busy(vsnap->vma_resource);
+
+ if (pinned)
+ *lockdep_cookie = dma_fence_begin_signalling();
+
+ return pinned;
+}
+
+/**
+ * i915_vma_snapshot_resource_unpin - Unblock vma snapshot memory from
+ * being released.
+ * @vsnap: The vma snapshot.
+ * @lockdep_cookie: Cookie returned from matching i915_vma_resource_pin().
+ *
+ * Might leave a fence signalling critical section and signal a fence.
+ */
+void i915_vma_snapshot_resource_unpin(struct i915_vma_snapshot *vsnap,
+ bool lockdep_cookie)
+{
+ dma_fence_end_signalling(lockdep_cookie);
+
+ return i915_active_release(vsnap->vma_resource);
+}
diff --git a/drivers/gpu/drm/i915/i915_vma_snapshot.h b/drivers/gpu/drm/i915/i915_vma_snapshot.h
new file mode 100644
index 0000000..940581d
--- /dev/null
+++ b/drivers/gpu/drm/i915/i915_vma_snapshot.h
@@ -0,0 +1,112 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2021 Intel Corporation
+ */
+#ifndef _I915_VMA_SNAPSHOT_H_
+#define _I915_VMA_SNAPSHOT_H_
+
+#include <linux/kref.h>
+#include <linux/slab.h>
+#include <linux/types.h>
+
+struct i915_active;
+struct i915_refct_sgt;
+struct i915_vma;
+struct intel_memory_region;
+struct sg_table;
+
+/**
+ * DOC: Simple utilities for snapshotting GPU vma metadata, later used for
+ * error capture. Vi use a separate header for this to avoid issues due to
+ * recursive header includes.
+ */
+
+/**
+ * struct i915_vma_snapshot - Snapshot of vma metadata.
+ * @size: The vma size in bytes.
+ * @obj_size: The size of the underlying object in bytes.
+ * @gtt_offset: The gtt offset the vma is bound to.
+ * @gtt_size: The size in bytes allocated for the vma in the GTT.
+ * @pages: The struct sg_table pointing to the pages bound.
+ * @pages_rsgt: The refcounted sg_table holding the reference for @pages if any.
+ * @mr: The memory region pointed for the pages bound.
+ * @kref: Reference for this structure.
+ * @vma_resource: FIXME: A means to keep the unbind fence from signaling.
+ * Temporarily while we have only sync unbinds, and still use the vma
+ * active, we use that. With async unbinding we need a signaling refcount
+ * for the unbind fence.
+ * @page_sizes: The vma GTT page sizes information.
+ * @onstack: Whether the structure shouldn't be freed on final put.
+ * @present: Whether the structure is present and initialized.
+ */
+struct i915_vma_snapshot {
+ const char *name;
+ size_t size;
+ size_t obj_size;
+ size_t gtt_offset;
+ size_t gtt_size;
+ struct sg_table *pages;
+ struct i915_refct_sgt *pages_rsgt;
+ struct intel_memory_region *mr;
+ struct kref kref;
+ struct i915_active *vma_resource;
+ u32 page_sizes;
+ bool onstack:1;
+ bool present:1;
+};
+
+void i915_vma_snapshot_init(struct i915_vma_snapshot *vsnap,
+ struct i915_vma *vma,
+ const char *name);
+
+void i915_vma_snapshot_init_onstack(struct i915_vma_snapshot *vsnap,
+ struct i915_vma *vma,
+ const char *name);
+
+void i915_vma_snapshot_put(struct i915_vma_snapshot *vsnap);
+
+void i915_vma_snapshot_put_onstack(struct i915_vma_snapshot *vsnap);
+
+bool i915_vma_snapshot_resource_pin(struct i915_vma_snapshot *vsnap,
+ bool *lockdep_cookie);
+
+void i915_vma_snapshot_resource_unpin(struct i915_vma_snapshot *vsnap,
+ bool lockdep_cookie);
+
+/**
+ * i915_vma_snapshot_alloc - Allocate a struct i915_vma_snapshot
+ * @gfp: Allocation mode.
+ *
+ * Return: A pointer to a struct i915_vma_snapshot if successful.
+ * NULL otherwise.
+ */
+static inline struct i915_vma_snapshot *i915_vma_snapshot_alloc(gfp_t gfp)
+{
+ return kmalloc(sizeof(struct i915_vma_snapshot), gfp);
+}
+
+/**
+ * i915_vma_snapshot_get - Take a reference on a struct i915_vma_snapshot
+ *
+ * Return: A pointer to a struct i915_vma_snapshot.
+ */
+static inline struct i915_vma_snapshot *
+i915_vma_snapshot_get(struct i915_vma_snapshot *vsnap)
+{
+ kref_get(&vsnap->kref);
+ return vsnap;
+}
+
+/**
+ * i915_vma_snapshot_present - Whether a struct i915_vma_snapshot is
+ * present and initialized.
+ *
+ * Return: true if present and initialized; false otherwise.
+ */
+static inline bool
+i915_vma_snapshot_present(const struct i915_vma_snapshot *vsnap)
+{
+ return vsnap && vsnap->present;
+}
+
+#endif
diff --git a/drivers/gpu/drm/i915/i915_vma_types.h b/drivers/gpu/drm/i915/i915_vma_types.h
index 80e93bf..ca575e1 100644
--- a/drivers/gpu/drm/i915/i915_vma_types.h
+++ b/drivers/gpu/drm/i915/i915_vma_types.h
@@ -97,11 +97,20 @@ enum i915_cache_level;
struct intel_remapped_plane_info {
/* in gtt pages */
- u32 offset;
- u16 width;
- u16 height;
- u16 src_stride;
- u16 dst_stride;
+ u32 offset:31;
+ u32 linear:1;
+ union {
+ /* in gtt pages for !linear */
+ struct {
+ u16 width;
+ u16 height;
+ u16 src_stride;
+ u16 dst_stride;
+ };
+
+ /* in gtt pages for linear */
+ u32 size;
+ };
} __packed;
struct intel_remapped_info {
@@ -178,7 +187,6 @@ struct i915_vma {
const struct i915_vma_ops *ops;
struct drm_i915_gem_object *obj;
- struct dma_resv *resv; /** Alias of obj->resv */
struct sg_table *pages;
void __iomem *iomap;
@@ -262,7 +270,6 @@ struct i915_vma {
#define I915_VMA_PAGES_BIAS 24
#define I915_VMA_PAGES_ACTIVE (BIT(24) | 1)
atomic_t pages_count; /* number of active binds to the pages */
- struct mutex pages_mutex; /* protect acquire/release of backing pages */
/**
* Support different GGTT views into the same object.
diff --git a/drivers/gpu/drm/i915/intel_device_info.c b/drivers/gpu/drm/i915/intel_device_info.c
index 305face..04fd266 100644
--- a/drivers/gpu/drm/i915/intel_device_info.c
+++ b/drivers/gpu/drm/i915/intel_device_info.c
@@ -83,33 +83,26 @@ const char *intel_platform_name(enum intel_platform platform)
return platform_names[platform];
}
-static const char *iommu_name(void)
-{
- const char *msg = "n/a";
-
-#ifdef CONFIG_INTEL_IOMMU
- msg = enableddisabled(intel_iommu_gfx_mapped);
-#endif
-
- return msg;
-}
-
void intel_device_info_print_static(const struct intel_device_info *info,
struct drm_printer *p)
{
- if (info->graphics_rel)
- drm_printf(p, "graphics version: %u.%02u\n", info->graphics_ver, info->graphics_rel);
+ if (info->graphics.rel)
+ drm_printf(p, "graphics version: %u.%02u\n", info->graphics.ver,
+ info->graphics.rel);
else
- drm_printf(p, "graphics version: %u\n", info->graphics_ver);
+ drm_printf(p, "graphics version: %u\n", info->graphics.ver);
- if (info->media_rel)
- drm_printf(p, "media version: %u.%02u\n", info->media_ver, info->media_rel);
+ if (info->media.rel)
+ drm_printf(p, "media version: %u.%02u\n", info->media.ver, info->media.rel);
else
- drm_printf(p, "media version: %u\n", info->media_ver);
+ drm_printf(p, "media version: %u\n", info->media.ver);
- drm_printf(p, "display version: %u\n", info->display.ver);
+ if (info->display.rel)
+ drm_printf(p, "display version: %u.%02u\n", info->display.ver, info->display.rel);
+ else
+ drm_printf(p, "display version: %u\n", info->display.ver);
+
drm_printf(p, "gt: %d\n", info->gt);
- drm_printf(p, "iommu: %s\n", iommu_name());
drm_printf(p, "memory-regions: %x\n", info->memory_regions);
drm_printf(p, "page-sizes: %x\n", info->page_sizes);
drm_printf(p, "platform: %s\n", intel_platform_name(info->platform));
@@ -177,6 +170,10 @@ static const u16 subplatform_portf_ids[] = {
INTEL_ICL_PORT_F_IDS(0),
};
+static const u16 subplatform_rpls_ids[] = {
+ INTEL_RPLS_IDS(0),
+};
+
static bool find_devid(u16 id, const u16 *p, unsigned int num)
{
for (; num; num--, p++) {
@@ -213,6 +210,9 @@ void intel_device_info_subplatform_init(struct drm_i915_private *i915)
} else if (find_devid(devid, subplatform_portf_ids,
ARRAY_SIZE(subplatform_portf_ids))) {
mask = BIT(INTEL_SUBPLATFORM_PORTF);
+ } else if (find_devid(devid, subplatform_rpls_ids,
+ ARRAY_SIZE(subplatform_rpls_ids))) {
+ mask = BIT(INTEL_SUBPLATFORM_RPL_S);
}
if (IS_TIGERLAKE(i915)) {
@@ -326,33 +326,33 @@ void intel_device_info_runtime_init(struct drm_i915_private *dev_priv)
!(sfuse_strap & SFUSE_STRAP_FUSE_LOCK))) {
drm_info(&dev_priv->drm,
"Display fused off, disabling\n");
- info->pipe_mask = 0;
- info->cpu_transcoder_mask = 0;
+ info->display.pipe_mask = 0;
+ info->display.cpu_transcoder_mask = 0;
} else if (fuse_strap & IVB_PIPE_C_DISABLE) {
drm_info(&dev_priv->drm, "PipeC fused off\n");
- info->pipe_mask &= ~BIT(PIPE_C);
- info->cpu_transcoder_mask &= ~BIT(TRANSCODER_C);
+ info->display.pipe_mask &= ~BIT(PIPE_C);
+ info->display.cpu_transcoder_mask &= ~BIT(TRANSCODER_C);
}
} else if (HAS_DISPLAY(dev_priv) && DISPLAY_VER(dev_priv) >= 9) {
u32 dfsm = intel_de_read(dev_priv, SKL_DFSM);
if (dfsm & SKL_DFSM_PIPE_A_DISABLE) {
- info->pipe_mask &= ~BIT(PIPE_A);
- info->cpu_transcoder_mask &= ~BIT(TRANSCODER_A);
+ info->display.pipe_mask &= ~BIT(PIPE_A);
+ info->display.cpu_transcoder_mask &= ~BIT(TRANSCODER_A);
}
if (dfsm & SKL_DFSM_PIPE_B_DISABLE) {
- info->pipe_mask &= ~BIT(PIPE_B);
- info->cpu_transcoder_mask &= ~BIT(TRANSCODER_B);
+ info->display.pipe_mask &= ~BIT(PIPE_B);
+ info->display.cpu_transcoder_mask &= ~BIT(TRANSCODER_B);
}
if (dfsm & SKL_DFSM_PIPE_C_DISABLE) {
- info->pipe_mask &= ~BIT(PIPE_C);
- info->cpu_transcoder_mask &= ~BIT(TRANSCODER_C);
+ info->display.pipe_mask &= ~BIT(PIPE_C);
+ info->display.cpu_transcoder_mask &= ~BIT(TRANSCODER_C);
}
if (DISPLAY_VER(dev_priv) >= 12 &&
(dfsm & TGL_DFSM_PIPE_D_DISABLE)) {
- info->pipe_mask &= ~BIT(PIPE_D);
- info->cpu_transcoder_mask &= ~BIT(TRANSCODER_D);
+ info->display.pipe_mask &= ~BIT(PIPE_D);
+ info->display.cpu_transcoder_mask &= ~BIT(TRANSCODER_D);
}
if (dfsm & SKL_DFSM_DISPLAY_HDCP_DISABLE)
@@ -369,7 +369,7 @@ void intel_device_info_runtime_init(struct drm_i915_private *dev_priv)
info->display.has_dsc = 0;
}
- if (GRAPHICS_VER(dev_priv) == 6 && intel_vtd_active()) {
+ if (GRAPHICS_VER(dev_priv) == 6 && intel_vtd_active(dev_priv)) {
drm_info(&dev_priv->drm,
"Disabling ppGTT for VT-d support\n");
info->ppgtt_type = INTEL_PPGTT_NONE;
diff --git a/drivers/gpu/drm/i915/intel_device_info.h b/drivers/gpu/drm/i915/intel_device_info.h
index 8e6f48d..78597d3 100644
--- a/drivers/gpu/drm/i915/intel_device_info.h
+++ b/drivers/gpu/drm/i915/intel_device_info.h
@@ -110,6 +110,9 @@ enum intel_platform {
#define INTEL_SUBPLATFORM_G10 0
#define INTEL_SUBPLATFORM_G11 1
+/* ADL-S */
+#define INTEL_SUBPLATFORM_RPL_S 0
+
enum intel_ppgtt_type {
INTEL_PPGTT_NONE = I915_GEM_PPGTT_NONE,
INTEL_PPGTT_ALIASING = I915_GEM_PPGTT_ALIASING,
@@ -123,6 +126,7 @@ enum intel_ppgtt_type {
func(is_dgfx); \
/* Keep has_* in alphabetical order */ \
func(has_64bit_reloc); \
+ func(has_64k_pages); \
func(gpu_reset_clobbers_display); \
func(has_reset_engine); \
func(has_global_mocs); \
@@ -166,11 +170,14 @@ enum intel_ppgtt_type {
func(overlay_needs_physical); \
func(supports_tv);
+struct ip_version {
+ u8 ver;
+ u8 rel;
+};
+
struct intel_device_info {
- u8 graphics_ver;
- u8 graphics_rel;
- u8 media_ver;
- u8 media_rel;
+ struct ip_version graphics;
+ struct ip_version media;
intel_engine_mask_t platform_engine_mask; /* Engines supported by the HW */
@@ -189,17 +196,17 @@ struct intel_device_info {
u8 gt; /* GT number, 0 if undefined */
- u8 pipe_mask;
- u8 cpu_transcoder_mask;
-
- u8 abox_mask;
-
#define DEFINE_FLAG(name) u8 name:1
DEV_INFO_FOR_EACH_FLAG(DEFINE_FLAG);
#undef DEFINE_FLAG
struct {
u8 ver;
+ u8 rel;
+
+ u8 pipe_mask;
+ u8 cpu_transcoder_mask;
+ u8 abox_mask;
#define DEFINE_FLAG(name) u8 name:1
DEV_INFO_DISPLAY_FOR_EACH_FLAG(DEFINE_FLAG);
diff --git a/drivers/gpu/drm/i915/intel_gvt.c b/drivers/gpu/drm/i915/intel_gvt.c
index 4e70c1a..cf6e989 100644
--- a/drivers/gpu/drm/i915/intel_gvt.c
+++ b/drivers/gpu/drm/i915/intel_gvt.c
@@ -109,7 +109,7 @@ int intel_gvt_init(struct drm_i915_private *dev_priv)
return 0;
}
- if (intel_uc_wants_guc_submission(&dev_priv->gt.uc)) {
+ if (intel_uc_wants_guc_submission(&to_gt(dev_priv)->uc)) {
drm_err(&dev_priv->drm,
"i915 GVT-g loading failed due to Graphics virtualization is not yet supported with GuC submission\n");
return -EIO;
diff --git a/drivers/gpu/drm/i915/intel_memory_region.c b/drivers/gpu/drm/i915/intel_memory_region.c
index e7f7e66..c70d7e2 100644
--- a/drivers/gpu/drm/i915/intel_memory_region.c
+++ b/drivers/gpu/drm/i915/intel_memory_region.c
@@ -3,6 +3,8 @@
* Copyright © 2019 Intel Corporation
*/
+#include <linux/prandom.h>
+
#include "intel_memory_region.h"
#include "i915_drv.h"
#include "i915_ttm_buddy_manager.h"
@@ -29,6 +31,110 @@ static const struct {
},
};
+static int __iopagetest(struct intel_memory_region *mem,
+ u8 __iomem *va, int pagesize,
+ u8 value, resource_size_t offset,
+ const void *caller)
+{
+ int byte = prandom_u32_max(pagesize);
+ u8 result[3];
+
+ memset_io(va, value, pagesize); /* or GPF! */
+ wmb();
+
+ result[0] = ioread8(va);
+ result[1] = ioread8(va + byte);
+ result[2] = ioread8(va + pagesize - 1);
+ if (memchr_inv(result, value, sizeof(result))) {
+ dev_err(mem->i915->drm.dev,
+ "Failed to read back from memory region:%pR at [%pa + %pa] for %ps; wrote %x, read (%x, %x, %x)\n",
+ &mem->region, &mem->io_start, &offset, caller,
+ value, result[0], result[1], result[2]);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int iopagetest(struct intel_memory_region *mem,
+ resource_size_t offset,
+ const void *caller)
+{
+ const u8 val[] = { 0x0, 0xa5, 0xc3, 0xf0 };
+ void __iomem *va;
+ int err;
+ int i;
+
+ va = ioremap_wc(mem->io_start + offset, PAGE_SIZE);
+ if (!va) {
+ dev_err(mem->i915->drm.dev,
+ "Failed to ioremap memory region [%pa + %pa] for %ps\n",
+ &mem->io_start, &offset, caller);
+ return -EFAULT;
+ }
+
+ for (i = 0; i < ARRAY_SIZE(val); i++) {
+ err = __iopagetest(mem, va, PAGE_SIZE, val[i], offset, caller);
+ if (err)
+ break;
+
+ err = __iopagetest(mem, va, PAGE_SIZE, ~val[i], offset, caller);
+ if (err)
+ break;
+ }
+
+ iounmap(va);
+ return err;
+}
+
+static resource_size_t random_page(resource_size_t last)
+{
+ /* Limited to low 44b (16TiB), but should suffice for a spot check */
+ return prandom_u32_max(last >> PAGE_SHIFT) << PAGE_SHIFT;
+}
+
+static int iomemtest(struct intel_memory_region *mem,
+ bool test_all,
+ const void *caller)
+{
+ resource_size_t last = resource_size(&mem->region) - PAGE_SIZE;
+ resource_size_t page;
+ int err;
+
+ /*
+ * Quick test to check read/write access to the iomap (backing store).
+ *
+ * Write a byte, read it back. If the iomapping fails, we expect
+ * a GPF preventing further execution. If the backing store does not
+ * exist, the read back will return garbage. We check a couple of pages,
+ * the first and last of the specified region to confirm the backing
+ * store + iomap does cover the entire memory region; and we check
+ * a random offset within as a quick spot check for bad memory.
+ */
+
+ if (test_all) {
+ for (page = 0; page <= last; page += PAGE_SIZE) {
+ err = iopagetest(mem, page, caller);
+ if (err)
+ return err;
+ }
+ } else {
+ err = iopagetest(mem, 0, caller);
+ if (err)
+ return err;
+
+ err = iopagetest(mem, last, caller);
+ if (err)
+ return err;
+
+ err = iopagetest(mem, random_page(last), caller);
+ if (err)
+ return err;
+ }
+
+ return 0;
+}
+
struct intel_memory_region *
intel_memory_region_lookup(struct drm_i915_private *i915,
u16 class, u16 instance)
@@ -90,6 +196,21 @@ void intel_memory_region_debug(struct intel_memory_region *mr,
&mr->total, &mr->avail);
}
+static int intel_memory_region_memtest(struct intel_memory_region *mem,
+ void *caller)
+{
+ struct drm_i915_private *i915 = mem->i915;
+ int err = 0;
+
+ if (!mem->io_start)
+ return 0;
+
+ if (IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM) || i915->params.memtest)
+ err = iomemtest(mem, i915->params.memtest, caller);
+
+ return err;
+}
+
struct intel_memory_region *
intel_memory_region_create(struct drm_i915_private *i915,
resource_size_t start,
@@ -126,9 +247,15 @@ intel_memory_region_create(struct drm_i915_private *i915,
goto err_free;
}
- kref_init(&mem->kref);
+ err = intel_memory_region_memtest(mem, (void *)_RET_IP_);
+ if (err)
+ goto err_release;
+
return mem;
+err_release:
+ if (mem->ops->release)
+ mem->ops->release(mem);
err_free:
kfree(mem);
return ERR_PTR(err);
@@ -144,28 +271,17 @@ void intel_memory_region_set_name(struct intel_memory_region *mem,
va_end(ap);
}
-static void __intel_memory_region_destroy(struct kref *kref)
+void intel_memory_region_destroy(struct intel_memory_region *mem)
{
- struct intel_memory_region *mem =
- container_of(kref, typeof(*mem), kref);
+ int ret = 0;
if (mem->ops->release)
- mem->ops->release(mem);
+ ret = mem->ops->release(mem);
+ GEM_WARN_ON(!list_empty_careful(&mem->objects.list));
mutex_destroy(&mem->objects.lock);
- kfree(mem);
-}
-
-struct intel_memory_region *
-intel_memory_region_get(struct intel_memory_region *mem)
-{
- kref_get(&mem->kref);
- return mem;
-}
-
-void intel_memory_region_put(struct intel_memory_region *mem)
-{
- kref_put(&mem->kref, __intel_memory_region_destroy);
+ if (!ret)
+ kfree(mem);
}
/* Global memory region registration -- only slight layer inversions! */
@@ -234,7 +350,7 @@ void intel_memory_regions_driver_release(struct drm_i915_private *i915)
fetch_and_zero(&i915->mm.regions[i]);
if (region)
- intel_memory_region_put(region);
+ intel_memory_region_destroy(region);
}
}
diff --git a/drivers/gpu/drm/i915/intel_memory_region.h b/drivers/gpu/drm/i915/intel_memory_region.h
index 3feae33..5625c9c 100644
--- a/drivers/gpu/drm/i915/intel_memory_region.h
+++ b/drivers/gpu/drm/i915/intel_memory_region.h
@@ -6,7 +6,6 @@
#ifndef __INTEL_MEMORY_REGION_H__
#define __INTEL_MEMORY_REGION_H__
-#include <linux/kref.h>
#include <linux/ioport.h>
#include <linux/mutex.h>
#include <linux/io-mapping.h>
@@ -51,7 +50,7 @@ struct intel_memory_region_ops {
unsigned int flags;
int (*init)(struct intel_memory_region *mem);
- void (*release)(struct intel_memory_region *mem);
+ int (*release)(struct intel_memory_region *mem);
int (*init_object)(struct intel_memory_region *mem,
struct drm_i915_gem_object *obj,
@@ -71,8 +70,6 @@ struct intel_memory_region {
/* For fake LMEM */
struct drm_mm_node fake_mappable;
- struct kref kref;
-
resource_size_t io_start;
resource_size_t min_page_size;
resource_size_t total;
@@ -110,9 +107,7 @@ intel_memory_region_create(struct drm_i915_private *i915,
u16 instance,
const struct intel_memory_region_ops *ops);
-struct intel_memory_region *
-intel_memory_region_get(struct intel_memory_region *mem);
-void intel_memory_region_put(struct intel_memory_region *mem);
+void intel_memory_region_destroy(struct intel_memory_region *mem);
int intel_memory_regions_hw_probe(struct drm_i915_private *i915);
void intel_memory_regions_driver_release(struct drm_i915_private *i915);
diff --git a/drivers/gpu/drm/i915/intel_pch.c b/drivers/gpu/drm/i915/intel_pch.c
index d1d4b97..da8f82c 100644
--- a/drivers/gpu/drm/i915/intel_pch.c
+++ b/drivers/gpu/drm/i915/intel_pch.c
@@ -129,6 +129,7 @@ intel_pch_type(const struct drm_i915_private *dev_priv, unsigned short id)
return PCH_JSP;
case INTEL_PCH_ADP_DEVICE_ID_TYPE:
case INTEL_PCH_ADP2_DEVICE_ID_TYPE:
+ case INTEL_PCH_ADP3_DEVICE_ID_TYPE:
drm_dbg_kms(&dev_priv->drm, "Found Alder Lake PCH\n");
drm_WARN_ON(&dev_priv->drm, !IS_ALDERLAKE_S(dev_priv) &&
!IS_ALDERLAKE_P(dev_priv));
diff --git a/drivers/gpu/drm/i915/intel_pch.h b/drivers/gpu/drm/i915/intel_pch.h
index 7c0d83d..6bff775 100644
--- a/drivers/gpu/drm/i915/intel_pch.h
+++ b/drivers/gpu/drm/i915/intel_pch.h
@@ -57,6 +57,7 @@ enum intel_pch {
#define INTEL_PCH_JSP2_DEVICE_ID_TYPE 0x3880
#define INTEL_PCH_ADP_DEVICE_ID_TYPE 0x7A80
#define INTEL_PCH_ADP2_DEVICE_ID_TYPE 0x5180
+#define INTEL_PCH_ADP3_DEVICE_ID_TYPE 0x7A00
#define INTEL_PCH_P2X_DEVICE_ID_TYPE 0x7100
#define INTEL_PCH_P3X_DEVICE_ID_TYPE 0x7000
#define INTEL_PCH_QEMU_DEVICE_ID_TYPE 0x2900 /* qemu q35 has 2918 */
diff --git a/drivers/gpu/drm/i915/intel_pm.c b/drivers/gpu/drm/i915/intel_pm.c
index ecbb3d1..434b1f8 100644
--- a/drivers/gpu/drm/i915/intel_pm.c
+++ b/drivers/gpu/drm/i915/intel_pm.c
@@ -36,7 +36,9 @@
#include "display/intel_atomic_plane.h"
#include "display/intel_bw.h"
#include "display/intel_de.h"
+#include "display/intel_display_trace.h"
#include "display/intel_display_types.h"
+#include "display/intel_fb.h"
#include "display/intel_fbc.h"
#include "display/intel_sprite.h"
#include "display/skl_universal_plane.h"
@@ -46,7 +48,6 @@
#include "i915_drv.h"
#include "i915_fixed.h"
#include "i915_irq.h"
-#include "i915_trace.h"
#include "intel_pcode.h"
#include "intel_pm.h"
#include "vlv_sideband.h"
@@ -97,7 +98,7 @@ static void gen9_init_clock_gating(struct drm_i915_private *dev_priv)
* "Plane N strech max must be programmed to 11b (x1)
* when Async flips are enabled on that plane."
*/
- if (!IS_GEMINILAKE(dev_priv) && intel_vtd_active())
+ if (!IS_GEMINILAKE(dev_priv) && intel_vtd_active(dev_priv))
intel_uncore_rmw(&dev_priv->uncore, CHICKEN_PIPESL_1(pipe),
SKL_PLANE1_STRETCH_MAX_MASK, SKL_PLANE1_STRETCH_MAX_X1);
}
@@ -160,7 +161,7 @@ static void bxt_init_clock_gating(struct drm_i915_private *dev_priv)
* Display WA #0883: bxt
*/
intel_uncore_write(&dev_priv->uncore, ILK_DPFC_CHICKEN, intel_uncore_read(&dev_priv->uncore, ILK_DPFC_CHICKEN) |
- ILK_DPFC_DISABLE_DUMMY0);
+ DPFC_DISABLE_DUMMY0);
}
static void glk_init_clock_gating(struct drm_i915_private *dev_priv)
@@ -988,7 +989,7 @@ static void g4x_write_wm_values(struct drm_i915_private *dev_priv,
enum pipe pipe;
for_each_pipe(dev_priv, pipe)
- trace_g4x_wm(intel_get_crtc_for_pipe(dev_priv, pipe), wm);
+ trace_g4x_wm(intel_crtc_for_pipe(dev_priv, pipe), wm);
intel_uncore_write(&dev_priv->uncore, DSPFW1,
FW_WM(wm->sr.plane, SR) |
@@ -1020,7 +1021,7 @@ static void vlv_write_wm_values(struct drm_i915_private *dev_priv,
enum pipe pipe;
for_each_pipe(dev_priv, pipe) {
- trace_vlv_wm(intel_get_crtc_for_pipe(dev_priv, pipe), wm);
+ trace_vlv_wm(intel_crtc_for_pipe(dev_priv, pipe), wm);
intel_uncore_write(&dev_priv->uncore, VLV_DDL(pipe),
(wm->ddl[pipe].plane[PLANE_CURSOR] << DDL_CURSOR_SHIFT) |
@@ -2335,6 +2336,20 @@ static void i965_update_wm(struct drm_i915_private *dev_priv)
#undef FW_WM
+static struct intel_crtc *intel_crtc_for_plane(struct drm_i915_private *i915,
+ enum i9xx_plane_id i9xx_plane)
+{
+ struct intel_plane *plane;
+
+ for_each_intel_plane(&i915->drm, plane) {
+ if (plane->id == PLANE_PRIMARY &&
+ plane->i9xx_plane == i9xx_plane)
+ return intel_crtc_for_pipe(i915, plane->pipe);
+ }
+
+ return NULL;
+}
+
static void i9xx_update_wm(struct drm_i915_private *dev_priv)
{
const struct intel_watermark_params *wm_info;
@@ -2356,7 +2371,7 @@ static void i9xx_update_wm(struct drm_i915_private *dev_priv)
fifo_size = i830_get_fifo_size(dev_priv, PLANE_A);
else
fifo_size = i9xx_get_fifo_size(dev_priv, PLANE_A);
- crtc = intel_get_crtc_for_plane(dev_priv, PLANE_A);
+ crtc = intel_crtc_for_plane(dev_priv, PLANE_A);
if (intel_crtc_active(crtc)) {
const struct drm_display_mode *pipe_mode =
&crtc->config->hw.pipe_mode;
@@ -2386,7 +2401,7 @@ static void i9xx_update_wm(struct drm_i915_private *dev_priv)
fifo_size = i830_get_fifo_size(dev_priv, PLANE_B);
else
fifo_size = i9xx_get_fifo_size(dev_priv, PLANE_B);
- crtc = intel_get_crtc_for_plane(dev_priv, PLANE_B);
+ crtc = intel_crtc_for_plane(dev_priv, PLANE_B);
if (intel_crtc_active(crtc)) {
const struct drm_display_mode *pipe_mode =
&crtc->config->hw.pipe_mode;
@@ -3062,9 +3077,9 @@ static void snb_wm_latency_quirk(struct drm_i915_private *dev_priv)
* The BIOS provided WM memory latency values are often
* inadequate for high resolution displays. Adjust them.
*/
- changed = ilk_increase_wm_latency(dev_priv, dev_priv->wm.pri_latency, 12) |
- ilk_increase_wm_latency(dev_priv, dev_priv->wm.spr_latency, 12) |
- ilk_increase_wm_latency(dev_priv, dev_priv->wm.cur_latency, 12);
+ changed = ilk_increase_wm_latency(dev_priv, dev_priv->wm.pri_latency, 12);
+ changed |= ilk_increase_wm_latency(dev_priv, dev_priv->wm.spr_latency, 12);
+ changed |= ilk_increase_wm_latency(dev_priv, dev_priv->wm.cur_latency, 12);
if (!changed)
return;
@@ -3368,13 +3383,8 @@ static void ilk_wm_merge(struct drm_i915_private *dev_priv,
}
/* ILK: LP2+ must be disabled when FBC WM is disabled but FBC enabled */
- /*
- * FIXME this is racy. FBC might get enabled later.
- * What we should check here is whether FBC can be
- * enabled sometime later.
- */
- if (DISPLAY_VER(dev_priv) == 5 && !merged->fbc_wm_enabled &&
- intel_fbc_is_active(dev_priv)) {
+ if (DISPLAY_VER(dev_priv) == 5 && HAS_FBC(dev_priv) &&
+ dev_priv->params.enable_fbc && !merged->fbc_wm_enabled) {
for (level = 2; level <= max_level; level++) {
struct intel_wm_level *wm = &merged->wm[level];
@@ -5094,6 +5104,18 @@ skl_check_nv12_wm_level(struct skl_wm_level *wm, struct skl_wm_level *uv_wm,
}
}
+static bool icl_need_wm1_wa(struct drm_i915_private *i915,
+ enum plane_id plane_id)
+{
+ /*
+ * Wa_1408961008:icl, ehl
+ * Wa_14012656716:tgl, adl
+ * Underruns with WM1+ disabled
+ */
+ return DISPLAY_VER(i915) == 11 ||
+ (IS_DISPLAY_VER(i915, 12, 13) && plane_id == PLANE_CURSOR);
+}
+
static int
skl_allocate_plane_ddb(struct intel_atomic_state *state,
struct intel_crtc *crtc)
@@ -5264,11 +5286,7 @@ skl_allocate_plane_ddb(struct intel_atomic_state *state,
skl_check_nv12_wm_level(&wm->wm[level], &wm->uv_wm[level],
total[plane_id], uv_total[plane_id]);
- /*
- * Wa_1408961008:icl, ehl
- * Underruns with WM1+ disabled
- */
- if (DISPLAY_VER(dev_priv) == 11 &&
+ if (icl_need_wm1_wa(dev_priv, plane_id) &&
level == 1 && wm->wm[0].enable) {
wm->wm[level].blocks = wm->wm[0].blocks;
wm->wm[level].lines = wm->wm[0].lines;
@@ -6900,7 +6918,7 @@ void g4x_wm_sanitize(struct drm_i915_private *dev_priv)
for_each_intel_plane(&dev_priv->drm, plane) {
struct intel_crtc *crtc =
- intel_get_crtc_for_pipe(dev_priv, plane->pipe);
+ intel_crtc_for_pipe(dev_priv, plane->pipe);
struct intel_crtc_state *crtc_state =
to_intel_crtc_state(crtc->base.state);
struct intel_plane_state *plane_state =
@@ -7056,7 +7074,7 @@ void vlv_wm_sanitize(struct drm_i915_private *dev_priv)
for_each_intel_plane(&dev_priv->drm, plane) {
struct intel_crtc *crtc =
- intel_get_crtc_for_pipe(dev_priv, plane->pipe);
+ intel_crtc_for_pipe(dev_priv, plane->pipe);
struct intel_crtc_state *crtc_state =
to_intel_crtc_state(crtc->base.state);
struct intel_plane_state *plane_state =
@@ -7434,7 +7452,7 @@ static void icl_init_clock_gating(struct drm_i915_private *dev_priv)
{
/* Wa_1409120013:icl,ehl */
intel_uncore_write(&dev_priv->uncore, ILK_DPFC_CHICKEN,
- ILK_DPFC_CHICKEN_COMP_DUMMY_PIXEL);
+ DPFC_CHICKEN_COMP_DUMMY_PIXEL);
/*Wa_14010594013:icl, ehl */
intel_uncore_rmw(&dev_priv->uncore, GEN8_CHICKEN_DCPR_1,
@@ -7443,11 +7461,11 @@ static void icl_init_clock_gating(struct drm_i915_private *dev_priv)
static void gen12lp_init_clock_gating(struct drm_i915_private *dev_priv)
{
- /* Wa_1409120013:tgl,rkl,adl-s,dg1 */
+ /* Wa_1409120013:tgl,rkl,adl-s,dg1,dg2 */
if (IS_TIGERLAKE(dev_priv) || IS_ROCKETLAKE(dev_priv) ||
- IS_ALDERLAKE_S(dev_priv) || IS_DG1(dev_priv))
+ IS_ALDERLAKE_S(dev_priv) || IS_DG1(dev_priv) || IS_DG2(dev_priv))
intel_uncore_write(&dev_priv->uncore, ILK_DPFC_CHICKEN,
- ILK_DPFC_CHICKEN_COMP_DUMMY_PIXEL);
+ DPFC_CHICKEN_COMP_DUMMY_PIXEL);
/* Wa_1409825376:tgl (pre-prod)*/
if (IS_TGL_DISPLAY_STEP(dev_priv, STEP_A0, STEP_C0))
@@ -7473,11 +7491,34 @@ static void dg1_init_clock_gating(struct drm_i915_private *dev_priv)
gen12lp_init_clock_gating(dev_priv);
/* Wa_1409836686:dg1[a0] */
- if (IS_DG1_GT_STEP(dev_priv, STEP_A0, STEP_B0))
+ if (IS_DG1_GRAPHICS_STEP(dev_priv, STEP_A0, STEP_B0))
intel_uncore_write(&dev_priv->uncore, GEN9_CLKGATE_DIS_3, intel_uncore_read(&dev_priv->uncore, GEN9_CLKGATE_DIS_3) |
DPT_GATING_DIS);
}
+static void xehpsdv_init_clock_gating(struct drm_i915_private *dev_priv)
+{
+ /* Wa_22010146351:xehpsdv */
+ if (IS_XEHPSDV_GRAPHICS_STEP(dev_priv, STEP_A0, STEP_B0))
+ intel_uncore_rmw(&dev_priv->uncore, XEHP_CLOCK_GATE_DIS, 0, SGR_DIS);
+}
+
+static void dg2_init_clock_gating(struct drm_i915_private *i915)
+{
+ /* Wa_22010954014:dg2_g10 */
+ if (IS_DG2_G10(i915))
+ intel_uncore_rmw(&i915->uncore, XEHP_CLOCK_GATE_DIS, 0,
+ SGSI_SIDECLK_DIS);
+
+ /*
+ * Wa_14010733611:dg2_g10
+ * Wa_22010146351:dg2_g10
+ */
+ if (IS_DG2_GRAPHICS_STEP(i915, G10, STEP_A0, STEP_B0))
+ intel_uncore_rmw(&i915->uncore, XEHP_CLOCK_GATE_DIS, 0,
+ SGR_DIS | SGGI_DIS);
+}
+
static void cnp_init_clock_gating(struct drm_i915_private *dev_priv)
{
if (!HAS_PCH_CNP(dev_priv))
@@ -7509,7 +7550,7 @@ static void cfl_init_clock_gating(struct drm_i915_private *dev_priv)
* Display WA #0873: cfl
*/
intel_uncore_write(&dev_priv->uncore, ILK_DPFC_CHICKEN, intel_uncore_read(&dev_priv->uncore, ILK_DPFC_CHICKEN) |
- ILK_DPFC_NUKE_ON_ANY_MODIFICATION);
+ DPFC_NUKE_ON_ANY_MODIFICATION);
}
static void kbl_init_clock_gating(struct drm_i915_private *dev_priv)
@@ -7521,12 +7562,12 @@ static void kbl_init_clock_gating(struct drm_i915_private *dev_priv)
FBC_LLC_FULLY_OPEN);
/* WaDisableSDEUnitClockGating:kbl */
- if (IS_KBL_GT_STEP(dev_priv, 0, STEP_C0))
+ if (IS_KBL_GRAPHICS_STEP(dev_priv, 0, STEP_C0))
intel_uncore_write(&dev_priv->uncore, GEN8_UCGCTL6, intel_uncore_read(&dev_priv->uncore, GEN8_UCGCTL6) |
GEN8_SDEUNIT_CLOCK_GATE_DISABLE);
/* WaDisableGamClockGating:kbl */
- if (IS_KBL_GT_STEP(dev_priv, 0, STEP_C0))
+ if (IS_KBL_GRAPHICS_STEP(dev_priv, 0, STEP_C0))
intel_uncore_write(&dev_priv->uncore, GEN6_UCGCTL1, intel_uncore_read(&dev_priv->uncore, GEN6_UCGCTL1) |
GEN6_GAMUNIT_CLOCK_GATE_DISABLE);
@@ -7542,7 +7583,7 @@ static void kbl_init_clock_gating(struct drm_i915_private *dev_priv)
* Display WA #0873: kbl
*/
intel_uncore_write(&dev_priv->uncore, ILK_DPFC_CHICKEN, intel_uncore_read(&dev_priv->uncore, ILK_DPFC_CHICKEN) |
- ILK_DPFC_NUKE_ON_ANY_MODIFICATION);
+ DPFC_NUKE_ON_ANY_MODIFICATION);
}
static void skl_init_clock_gating(struct drm_i915_private *dev_priv)
@@ -7569,14 +7610,14 @@ static void skl_init_clock_gating(struct drm_i915_private *dev_priv)
* Display WA #0873: skl
*/
intel_uncore_write(&dev_priv->uncore, ILK_DPFC_CHICKEN, intel_uncore_read(&dev_priv->uncore, ILK_DPFC_CHICKEN) |
- ILK_DPFC_NUKE_ON_ANY_MODIFICATION);
+ DPFC_NUKE_ON_ANY_MODIFICATION);
/*
* WaFbcHighMemBwCorruptionAvoidance:skl
* Display WA #0883: skl
*/
intel_uncore_write(&dev_priv->uncore, ILK_DPFC_CHICKEN, intel_uncore_read(&dev_priv->uncore, ILK_DPFC_CHICKEN) |
- ILK_DPFC_DISABLE_DUMMY0);
+ DPFC_DISABLE_DUMMY0);
}
static void bdw_init_clock_gating(struct drm_i915_private *dev_priv)
@@ -7888,6 +7929,8 @@ static const struct drm_i915_clock_gating_funcs platform##_clock_gating_funcs =
.init_clock_gating = platform##_init_clock_gating, \
}
+CG_FUNCS(dg2);
+CG_FUNCS(xehpsdv);
CG_FUNCS(adlp);
CG_FUNCS(dg1);
CG_FUNCS(gen12lp);
@@ -7924,7 +7967,11 @@ CG_FUNCS(nop);
*/
void intel_init_clock_gating_hooks(struct drm_i915_private *dev_priv)
{
- if (IS_ALDERLAKE_P(dev_priv))
+ if (IS_DG2(dev_priv))
+ dev_priv->clock_gating_funcs = &dg2_clock_gating_funcs;
+ else if (IS_XEHPSDV(dev_priv))
+ dev_priv->clock_gating_funcs = &xehpsdv_clock_gating_funcs;
+ else if (IS_ALDERLAKE_P(dev_priv))
dev_priv->clock_gating_funcs = &adlp_clock_gating_funcs;
else if (IS_DG1(dev_priv))
dev_priv->clock_gating_funcs = &dg1_clock_gating_funcs;
diff --git a/drivers/gpu/drm/i915/intel_pm_types.h b/drivers/gpu/drm/i915/intel_pm_types.h
new file mode 100644
index 0000000..211632f
--- /dev/null
+++ b/drivers/gpu/drm/i915/intel_pm_types.h
@@ -0,0 +1,76 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2021 Intel Corporation
+ */
+
+#ifndef __INTEL_PM_TYPES_H__
+#define __INTEL_PM_TYPES_H__
+
+#include <linux/types.h>
+
+#include "display/intel_display.h"
+
+enum intel_ddb_partitioning {
+ INTEL_DDB_PART_1_2,
+ INTEL_DDB_PART_5_6, /* IVB+ */
+};
+
+struct ilk_wm_values {
+ u32 wm_pipe[3];
+ u32 wm_lp[3];
+ u32 wm_lp_spr[3];
+ bool enable_fbc_wm;
+ enum intel_ddb_partitioning partitioning;
+};
+
+struct g4x_pipe_wm {
+ u16 plane[I915_MAX_PLANES];
+ u16 fbc;
+};
+
+struct g4x_sr_wm {
+ u16 plane;
+ u16 cursor;
+ u16 fbc;
+};
+
+struct vlv_wm_ddl_values {
+ u8 plane[I915_MAX_PLANES];
+};
+
+struct vlv_wm_values {
+ struct g4x_pipe_wm pipe[3];
+ struct g4x_sr_wm sr;
+ struct vlv_wm_ddl_values ddl[3];
+ u8 level;
+ bool cxsr;
+};
+
+struct g4x_wm_values {
+ struct g4x_pipe_wm pipe[2];
+ struct g4x_sr_wm sr;
+ struct g4x_sr_wm hpll;
+ bool cxsr;
+ bool hpll_en;
+ bool fbc_en;
+};
+
+struct skl_ddb_entry {
+ u16 start, end; /* in number of blocks, 'end' is exclusive */
+};
+
+static inline u16 skl_ddb_entry_size(const struct skl_ddb_entry *entry)
+{
+ return entry->end - entry->start;
+}
+
+static inline bool skl_ddb_entry_equal(const struct skl_ddb_entry *e1,
+ const struct skl_ddb_entry *e2)
+{
+ if (e1->start == e2->start && e1->end == e2->end)
+ return true;
+
+ return false;
+}
+
+#endif /* __INTEL_PM_TYPES_H__ */
diff --git a/drivers/gpu/drm/i915/intel_region_ttm.c b/drivers/gpu/drm/i915/intel_region_ttm.c
index 98c7339..f2b888c 100644
--- a/drivers/gpu/drm/i915/intel_region_ttm.c
+++ b/drivers/gpu/drm/i915/intel_region_ttm.c
@@ -104,19 +104,50 @@ int intel_region_ttm_init(struct intel_memory_region *mem)
* memory region, and if it was registered with the TTM device,
* removes that registration.
*/
-void intel_region_ttm_fini(struct intel_memory_region *mem)
+int intel_region_ttm_fini(struct intel_memory_region *mem)
{
- int ret;
+ struct ttm_resource_manager *man = mem->region_private;
+ int ret = -EBUSY;
+ int count;
+
+ /*
+ * Put the region's move fences. This releases requests that
+ * may hold on to contexts and vms that may hold on to buffer
+ * objects placed in this region.
+ */
+ if (man)
+ ttm_resource_manager_cleanup(man);
+
+ /* Flush objects from region. */
+ for (count = 0; count < 10; ++count) {
+ i915_gem_flush_free_objects(mem->i915);
+
+ mutex_lock(&mem->objects.lock);
+ if (list_empty(&mem->objects.list))
+ ret = 0;
+ mutex_unlock(&mem->objects.lock);
+ if (!ret)
+ break;
+
+ msleep(20);
+ flush_delayed_work(&mem->i915->bdev.wq);
+ }
+
+ /* If we leaked objects, Don't free the region causing use after free */
+ if (ret || !man)
+ return ret;
ret = i915_ttm_buddy_man_fini(&mem->i915->bdev,
intel_region_to_ttm_type(mem));
GEM_WARN_ON(ret);
mem->region_private = NULL;
+
+ return ret;
}
/**
- * intel_region_ttm_resource_to_st - Convert an opaque TTM resource manager resource
- * to an sg_table.
+ * intel_region_ttm_resource_to_rsgt -
+ * Convert an opaque TTM resource manager resource to a refcounted sg_table.
* @mem: The memory region.
* @res: The resource manager resource obtained from the TTM resource manager.
*
@@ -126,17 +157,18 @@ void intel_region_ttm_fini(struct intel_memory_region *mem)
*
* Return: A malloced sg_table on success, an error pointer on failure.
*/
-struct sg_table *intel_region_ttm_resource_to_st(struct intel_memory_region *mem,
- struct ttm_resource *res)
+struct i915_refct_sgt *
+intel_region_ttm_resource_to_rsgt(struct intel_memory_region *mem,
+ struct ttm_resource *res)
{
if (mem->is_range_manager) {
struct ttm_range_mgr_node *range_node =
to_ttm_range_mgr_node(res);
- return i915_sg_from_mm_node(&range_node->mm_nodes[0],
- mem->region.start);
+ return i915_rsgt_from_mm_node(&range_node->mm_nodes[0],
+ mem->region.start);
} else {
- return i915_sg_from_buddy_resource(res, mem->region.start);
+ return i915_rsgt_from_buddy_resource(res, mem->region.start);
}
}
diff --git a/drivers/gpu/drm/i915/intel_region_ttm.h b/drivers/gpu/drm/i915/intel_region_ttm.h
index 6f44075..fdee5e7 100644
--- a/drivers/gpu/drm/i915/intel_region_ttm.h
+++ b/drivers/gpu/drm/i915/intel_region_ttm.h
@@ -20,10 +20,11 @@ void intel_region_ttm_device_fini(struct drm_i915_private *dev_priv);
int intel_region_ttm_init(struct intel_memory_region *mem);
-void intel_region_ttm_fini(struct intel_memory_region *mem);
+int intel_region_ttm_fini(struct intel_memory_region *mem);
-struct sg_table *intel_region_ttm_resource_to_st(struct intel_memory_region *mem,
- struct ttm_resource *res);
+struct i915_refct_sgt *
+intel_region_ttm_resource_to_rsgt(struct intel_memory_region *mem,
+ struct ttm_resource *res);
void intel_region_ttm_resource_free(struct intel_memory_region *mem,
struct ttm_resource *res);
diff --git a/drivers/gpu/drm/i915/intel_runtime_pm.c b/drivers/gpu/drm/i915/intel_runtime_pm.c
index 0d85f3c..22dab36 100644
--- a/drivers/gpu/drm/i915/intel_runtime_pm.c
+++ b/drivers/gpu/drm/i915/intel_runtime_pm.c
@@ -590,6 +590,9 @@ void intel_runtime_pm_enable(struct intel_runtime_pm *rpm)
pm_runtime_use_autosuspend(kdev);
}
+ /* Enable by default */
+ pm_runtime_allow(kdev);
+
/*
* The core calls the driver load handler with an RPM reference held.
* We drop that here and will reacquire it during unloading in
diff --git a/drivers/gpu/drm/i915/intel_step.c b/drivers/gpu/drm/i915/intel_step.c
index 6cf9676..a4b16b9 100644
--- a/drivers/gpu/drm/i915/intel_step.c
+++ b/drivers/gpu/drm/i915/intel_step.c
@@ -23,7 +23,8 @@
* use a macro to define these to make it easier to identify the platforms
* where the two steppings can deviate.
*/
-#define COMMON_STEP(x) .gt_step = STEP_##x, .display_step = STEP_##x
+#define COMMON_STEP(x) .graphics_step = STEP_##x, .display_step = STEP_##x, .media_step = STEP_##x
+#define COMMON_GT_MEDIA_STEP(x) .graphics_step = STEP_##x, .media_step = STEP_##x
static const struct intel_step_info skl_revids[] = {
[0x6] = { COMMON_STEP(G0) },
@@ -33,13 +34,13 @@ static const struct intel_step_info skl_revids[] = {
};
static const struct intel_step_info kbl_revids[] = {
- [1] = { .gt_step = STEP_B0, .display_step = STEP_B0 },
- [2] = { .gt_step = STEP_C0, .display_step = STEP_B0 },
- [3] = { .gt_step = STEP_D0, .display_step = STEP_B0 },
- [4] = { .gt_step = STEP_F0, .display_step = STEP_C0 },
- [5] = { .gt_step = STEP_C0, .display_step = STEP_B1 },
- [6] = { .gt_step = STEP_D1, .display_step = STEP_B1 },
- [7] = { .gt_step = STEP_G0, .display_step = STEP_C0 },
+ [1] = { COMMON_GT_MEDIA_STEP(B0), .display_step = STEP_B0 },
+ [2] = { COMMON_GT_MEDIA_STEP(C0), .display_step = STEP_B0 },
+ [3] = { COMMON_GT_MEDIA_STEP(D0), .display_step = STEP_B0 },
+ [4] = { COMMON_GT_MEDIA_STEP(F0), .display_step = STEP_C0 },
+ [5] = { COMMON_GT_MEDIA_STEP(C0), .display_step = STEP_B1 },
+ [6] = { COMMON_GT_MEDIA_STEP(D1), .display_step = STEP_B1 },
+ [7] = { COMMON_GT_MEDIA_STEP(G0), .display_step = STEP_C0 },
};
static const struct intel_step_info bxt_revids[] = {
@@ -63,16 +64,16 @@ static const struct intel_step_info jsl_ehl_revids[] = {
};
static const struct intel_step_info tgl_uy_revids[] = {
- [0] = { .gt_step = STEP_A0, .display_step = STEP_A0 },
- [1] = { .gt_step = STEP_B0, .display_step = STEP_C0 },
- [2] = { .gt_step = STEP_B1, .display_step = STEP_C0 },
- [3] = { .gt_step = STEP_C0, .display_step = STEP_D0 },
+ [0] = { COMMON_GT_MEDIA_STEP(A0), .display_step = STEP_A0 },
+ [1] = { COMMON_GT_MEDIA_STEP(B0), .display_step = STEP_C0 },
+ [2] = { COMMON_GT_MEDIA_STEP(B1), .display_step = STEP_C0 },
+ [3] = { COMMON_GT_MEDIA_STEP(C0), .display_step = STEP_D0 },
};
/* Same GT stepping between tgl_uy_revids and tgl_revids don't mean the same HW */
static const struct intel_step_info tgl_revids[] = {
- [0] = { .gt_step = STEP_A0, .display_step = STEP_B0 },
- [1] = { .gt_step = STEP_B0, .display_step = STEP_D0 },
+ [0] = { COMMON_GT_MEDIA_STEP(A0), .display_step = STEP_B0 },
+ [1] = { COMMON_GT_MEDIA_STEP(B0), .display_step = STEP_D0 },
};
static const struct intel_step_info rkl_revids[] = {
@@ -87,38 +88,38 @@ static const struct intel_step_info dg1_revids[] = {
};
static const struct intel_step_info adls_revids[] = {
- [0x0] = { .gt_step = STEP_A0, .display_step = STEP_A0 },
- [0x1] = { .gt_step = STEP_A0, .display_step = STEP_A2 },
- [0x4] = { .gt_step = STEP_B0, .display_step = STEP_B0 },
- [0x8] = { .gt_step = STEP_C0, .display_step = STEP_B0 },
- [0xC] = { .gt_step = STEP_D0, .display_step = STEP_C0 },
+ [0x0] = { COMMON_GT_MEDIA_STEP(A0), .display_step = STEP_A0 },
+ [0x1] = { COMMON_GT_MEDIA_STEP(A0), .display_step = STEP_A2 },
+ [0x4] = { COMMON_GT_MEDIA_STEP(B0), .display_step = STEP_B0 },
+ [0x8] = { COMMON_GT_MEDIA_STEP(C0), .display_step = STEP_B0 },
+ [0xC] = { COMMON_GT_MEDIA_STEP(D0), .display_step = STEP_C0 },
};
static const struct intel_step_info adlp_revids[] = {
- [0x0] = { .gt_step = STEP_A0, .display_step = STEP_A0 },
- [0x4] = { .gt_step = STEP_B0, .display_step = STEP_B0 },
- [0x8] = { .gt_step = STEP_C0, .display_step = STEP_C0 },
- [0xC] = { .gt_step = STEP_C0, .display_step = STEP_D0 },
+ [0x0] = { COMMON_GT_MEDIA_STEP(A0), .display_step = STEP_A0 },
+ [0x4] = { COMMON_GT_MEDIA_STEP(B0), .display_step = STEP_B0 },
+ [0x8] = { COMMON_GT_MEDIA_STEP(C0), .display_step = STEP_C0 },
+ [0xC] = { COMMON_GT_MEDIA_STEP(C0), .display_step = STEP_D0 },
};
static const struct intel_step_info xehpsdv_revids[] = {
- [0x0] = { .gt_step = STEP_A0 },
- [0x1] = { .gt_step = STEP_A1 },
- [0x4] = { .gt_step = STEP_B0 },
- [0x8] = { .gt_step = STEP_C0 },
+ [0x0] = { COMMON_GT_MEDIA_STEP(A0) },
+ [0x1] = { COMMON_GT_MEDIA_STEP(A1) },
+ [0x4] = { COMMON_GT_MEDIA_STEP(B0) },
+ [0x8] = { COMMON_GT_MEDIA_STEP(C0) },
};
static const struct intel_step_info dg2_g10_revid_step_tbl[] = {
- [0x0] = { .gt_step = STEP_A0, .display_step = STEP_A0 },
- [0x1] = { .gt_step = STEP_A1, .display_step = STEP_A0 },
- [0x4] = { .gt_step = STEP_B0, .display_step = STEP_B0 },
- [0x8] = { .gt_step = STEP_C0, .display_step = STEP_C0 },
+ [0x0] = { COMMON_GT_MEDIA_STEP(A0), .display_step = STEP_A0 },
+ [0x1] = { COMMON_GT_MEDIA_STEP(A1), .display_step = STEP_A0 },
+ [0x4] = { COMMON_GT_MEDIA_STEP(B0), .display_step = STEP_B0 },
+ [0x8] = { COMMON_GT_MEDIA_STEP(C0), .display_step = STEP_C0 },
};
static const struct intel_step_info dg2_g11_revid_step_tbl[] = {
- [0x0] = { .gt_step = STEP_A0, .display_step = STEP_B0 },
- [0x4] = { .gt_step = STEP_B0, .display_step = STEP_C0 },
- [0x5] = { .gt_step = STEP_B1, .display_step = STEP_C0 },
+ [0x0] = { COMMON_GT_MEDIA_STEP(A0), .display_step = STEP_B0 },
+ [0x4] = { COMMON_GT_MEDIA_STEP(B0), .display_step = STEP_C0 },
+ [0x5] = { COMMON_GT_MEDIA_STEP(B1), .display_step = STEP_C0 },
};
void intel_step_init(struct drm_i915_private *i915)
@@ -179,7 +180,7 @@ void intel_step_init(struct drm_i915_private *i915)
if (!revids)
return;
- if (revid < size && revids[revid].gt_step != STEP_NONE) {
+ if (revid < size && revids[revid].graphics_step != STEP_NONE) {
step = revids[revid];
} else {
drm_warn(&i915->drm, "Unknown revid 0x%02x\n", revid);
@@ -192,7 +193,7 @@ void intel_step_init(struct drm_i915_private *i915)
* steppings in the array are not monotonically increasing, but
* it's better than defaulting to 0.
*/
- while (revid < size && revids[revid].gt_step == STEP_NONE)
+ while (revid < size && revids[revid].graphics_step == STEP_NONE)
revid++;
if (revid < size) {
@@ -201,12 +202,12 @@ void intel_step_init(struct drm_i915_private *i915)
step = revids[revid];
} else {
drm_dbg(&i915->drm, "Using future steppings\n");
- step.gt_step = STEP_FUTURE;
+ step.graphics_step = STEP_FUTURE;
step.display_step = STEP_FUTURE;
}
}
- if (drm_WARN_ON(&i915->drm, step.gt_step == STEP_NONE))
+ if (drm_WARN_ON(&i915->drm, step.graphics_step == STEP_NONE))
return;
RUNTIME_INFO(i915)->step = step;
diff --git a/drivers/gpu/drm/i915/intel_step.h b/drivers/gpu/drm/i915/intel_step.h
index f6641e2..d71a99b 100644
--- a/drivers/gpu/drm/i915/intel_step.h
+++ b/drivers/gpu/drm/i915/intel_step.h
@@ -11,8 +11,9 @@
struct drm_i915_private;
struct intel_step_info {
- u8 gt_step;
+ u8 graphics_step;
u8 display_step;
+ u8 media_step;
};
#define STEP_ENUM_VAL(name) STEP_##name,
diff --git a/drivers/gpu/drm/i915/intel_uncore.c b/drivers/gpu/drm/i915/intel_uncore.c
index e072054..fc25ebf 100644
--- a/drivers/gpu/drm/i915/intel_uncore.c
+++ b/drivers/gpu/drm/i915/intel_uncore.c
@@ -22,11 +22,11 @@
*/
#include <linux/pm_runtime.h>
-#include <asm/iosf_mbi.h>
#include "gt/intel_lrc_reg.h" /* for shadow reg list */
#include "i915_drv.h"
+#include "i915_iosf_mbi.h"
#include "i915_trace.h"
#include "i915_vgpu.h"
#include "intel_pm.h"
@@ -2020,7 +2020,7 @@ static int i915_pmic_bus_access_notifier(struct notifier_block *nb,
return NOTIFY_OK;
}
-static int uncore_mmio_setup(struct intel_uncore *uncore)
+int intel_uncore_setup_mmio(struct intel_uncore *uncore)
{
struct drm_i915_private *i915 = uncore->i915;
struct pci_dev *pdev = to_pci_dev(i915->drm.dev);
@@ -2053,7 +2053,7 @@ static int uncore_mmio_setup(struct intel_uncore *uncore)
return 0;
}
-static void uncore_mmio_cleanup(struct intel_uncore *uncore)
+void intel_uncore_cleanup_mmio(struct intel_uncore *uncore)
{
struct pci_dev *pdev = to_pci_dev(uncore->i915->drm.dev);
@@ -2061,12 +2061,13 @@ static void uncore_mmio_cleanup(struct intel_uncore *uncore)
}
void intel_uncore_init_early(struct intel_uncore *uncore,
- struct drm_i915_private *i915)
+ struct intel_gt *gt)
{
spin_lock_init(&uncore->lock);
- uncore->i915 = i915;
- uncore->rpm = &i915->runtime_pm;
- uncore->debug = &i915->mmio_debug;
+ uncore->i915 = gt->i915;
+ uncore->gt = gt;
+ uncore->rpm = >->i915->runtime_pm;
+ uncore->debug = >->i915->mmio_debug;
}
static void uncore_raw_init(struct intel_uncore *uncore)
@@ -2146,10 +2147,6 @@ int intel_uncore_init_mmio(struct intel_uncore *uncore)
struct drm_i915_private *i915 = uncore->i915;
int ret;
- ret = uncore_mmio_setup(uncore);
- if (ret)
- return ret;
-
/*
* The boot firmware initializes local memory and assesses its health.
* If memory training fails, the punit will have been instructed to
@@ -2170,7 +2167,7 @@ int intel_uncore_init_mmio(struct intel_uncore *uncore)
} else {
ret = uncore_forcewake_init(uncore);
if (ret)
- goto out_mmio_cleanup;
+ return ret;
}
/* make sure fw funcs are set if and only if we have fw*/
@@ -2192,11 +2189,6 @@ int intel_uncore_init_mmio(struct intel_uncore *uncore)
drm_dbg(&i915->drm, "unclaimed mmio detected on uncore init, clearing\n");
return 0;
-
-out_mmio_cleanup:
- uncore_mmio_cleanup(uncore);
-
- return ret;
}
/*
@@ -2261,8 +2253,6 @@ void intel_uncore_fini_mmio(struct intel_uncore *uncore)
intel_uncore_fw_domains_fini(uncore);
iosf_mbi_punit_release();
}
-
- uncore_mmio_cleanup(uncore);
}
static const struct reg_whitelist {
diff --git a/drivers/gpu/drm/i915/intel_uncore.h b/drivers/gpu/drm/i915/intel_uncore.h
index 3248e4e..210fe2a 100644
--- a/drivers/gpu/drm/i915/intel_uncore.h
+++ b/drivers/gpu/drm/i915/intel_uncore.h
@@ -129,6 +129,7 @@ struct intel_uncore {
void __iomem *regs;
struct drm_i915_private *i915;
+ struct intel_gt *gt;
struct intel_runtime_pm *rpm;
spinlock_t lock; /** lock is also taken in irq contexts. */
@@ -217,12 +218,14 @@ u32 intel_uncore_read_with_mcr_steering(struct intel_uncore *uncore,
void
intel_uncore_mmio_debug_init_early(struct intel_uncore_mmio_debug *mmio_debug);
void intel_uncore_init_early(struct intel_uncore *uncore,
- struct drm_i915_private *i915);
+ struct intel_gt *gt);
+int intel_uncore_setup_mmio(struct intel_uncore *uncore);
int intel_uncore_init_mmio(struct intel_uncore *uncore);
void intel_uncore_prune_engine_fw_domains(struct intel_uncore *uncore,
struct intel_gt *gt);
bool intel_uncore_unclaimed_mmio(struct intel_uncore *uncore);
bool intel_uncore_arm_unclaimed_mmio_detection(struct intel_uncore *uncore);
+void intel_uncore_cleanup_mmio(struct intel_uncore *uncore);
void intel_uncore_fini_mmio(struct intel_uncore *uncore);
void intel_uncore_suspend(struct intel_uncore *uncore);
void intel_uncore_resume_early(struct intel_uncore *uncore);
diff --git a/drivers/gpu/drm/i915/intel_wopcm.c b/drivers/gpu/drm/i915/intel_wopcm.c
index 5e511bb..f06d210 100644
--- a/drivers/gpu/drm/i915/intel_wopcm.c
+++ b/drivers/gpu/drm/i915/intel_wopcm.c
@@ -220,7 +220,7 @@ static bool __wopcm_regs_locked(struct intel_uncore *uncore,
void intel_wopcm_init(struct intel_wopcm *wopcm)
{
struct drm_i915_private *i915 = wopcm_to_i915(wopcm);
- struct intel_gt *gt = &i915->gt;
+ struct intel_gt *gt = to_gt(i915);
u32 guc_fw_size = intel_uc_fw_get_upload_size(>->uc.guc.fw);
u32 huc_fw_size = intel_uc_fw_get_upload_size(>->uc.huc.fw);
u32 ctx_rsvd = context_reserved_size(i915);
diff --git a/drivers/gpu/drm/i915/pxp/intel_pxp.c b/drivers/gpu/drm/i915/pxp/intel_pxp.c
index e2314ad..15311ea 100644
--- a/drivers/gpu/drm/i915/pxp/intel_pxp.c
+++ b/drivers/gpu/drm/i915/pxp/intel_pxp.c
@@ -44,6 +44,11 @@ struct intel_gt *pxp_to_gt(const struct intel_pxp *pxp)
return container_of(pxp, struct intel_gt, pxp);
}
+bool intel_pxp_is_enabled(const struct intel_pxp *pxp)
+{
+ return pxp->ce;
+}
+
bool intel_pxp_is_active(const struct intel_pxp *pxp)
{
return pxp->arb_is_valid;
diff --git a/drivers/gpu/drm/i915/pxp/intel_pxp.h b/drivers/gpu/drm/i915/pxp/intel_pxp.h
index aa26225..73847e5 100644
--- a/drivers/gpu/drm/i915/pxp/intel_pxp.h
+++ b/drivers/gpu/drm/i915/pxp/intel_pxp.h
@@ -6,17 +6,15 @@
#ifndef __INTEL_PXP_H__
#define __INTEL_PXP_H__
-#include "intel_pxp_types.h"
+#include <linux/errno.h>
+#include <linux/types.h>
+struct intel_pxp;
struct drm_i915_gem_object;
-static inline bool intel_pxp_is_enabled(const struct intel_pxp *pxp)
-{
- return pxp->ce;
-}
-
#ifdef CONFIG_DRM_I915_PXP
struct intel_gt *pxp_to_gt(const struct intel_pxp *pxp);
+bool intel_pxp_is_enabled(const struct intel_pxp *pxp);
bool intel_pxp_is_active(const struct intel_pxp *pxp);
void intel_pxp_init(struct intel_pxp *pxp);
@@ -48,6 +46,11 @@ static inline int intel_pxp_start(struct intel_pxp *pxp)
return -ENODEV;
}
+static inline bool intel_pxp_is_enabled(const struct intel_pxp *pxp)
+{
+ return false;
+}
+
static inline bool intel_pxp_is_active(const struct intel_pxp *pxp)
{
return false;
diff --git a/drivers/gpu/drm/i915/pxp/intel_pxp_session.c b/drivers/gpu/drm/i915/pxp/intel_pxp_session.c
index d02732f..598840b 100644
--- a/drivers/gpu/drm/i915/pxp/intel_pxp_session.c
+++ b/drivers/gpu/drm/i915/pxp/intel_pxp_session.c
@@ -3,7 +3,8 @@
* Copyright(c) 2020, Intel Corporation. All rights reserved.
*/
-#include "drm/i915_drm.h"
+#include <drm/i915_drm.h>
+
#include "i915_drv.h"
#include "intel_pxp.h"
diff --git a/drivers/gpu/drm/i915/pxp/intel_pxp_tee.c b/drivers/gpu/drm/i915/pxp/intel_pxp_tee.c
index 49508f3..195b232 100644
--- a/drivers/gpu/drm/i915/pxp/intel_pxp_tee.c
+++ b/drivers/gpu/drm/i915/pxp/intel_pxp_tee.c
@@ -4,8 +4,10 @@
*/
#include <linux/component.h>
-#include "drm/i915_pxp_tee_interface.h"
-#include "drm/i915_component.h"
+
+#include <drm/i915_pxp_tee_interface.h>
+#include <drm/i915_component.h>
+
#include "i915_drv.h"
#include "intel_pxp.h"
#include "intel_pxp_session.h"
@@ -14,7 +16,9 @@
static inline struct intel_pxp *i915_dev_to_pxp(struct device *i915_kdev)
{
- return &kdev_to_i915(i915_kdev)->gt.pxp;
+ struct drm_i915_private *i915 = kdev_to_i915(i915_kdev);
+
+ return &to_gt(i915)->pxp;
}
static int intel_pxp_tee_io_message(struct intel_pxp *pxp,
diff --git a/drivers/gpu/drm/i915/pxp/intel_pxp_types.h b/drivers/gpu/drm/i915/pxp/intel_pxp_types.h
index 73ef7d1..7ce5f37 100644
--- a/drivers/gpu/drm/i915/pxp/intel_pxp_types.h
+++ b/drivers/gpu/drm/i915/pxp/intel_pxp_types.h
@@ -7,9 +7,7 @@
#define __INTEL_PXP_TYPES_H__
#include <linux/completion.h>
-#include <linux/list.h>
#include <linux/mutex.h>
-#include <linux/spinlock.h>
#include <linux/types.h>
#include <linux/workqueue.h>
diff --git a/drivers/gpu/drm/i915/selftests/i915_active.c b/drivers/gpu/drm/i915/selftests/i915_active.c
index 61bf456..2dac9be 100644
--- a/drivers/gpu/drm/i915/selftests/i915_active.c
+++ b/drivers/gpu/drm/i915/selftests/i915_active.c
@@ -254,7 +254,7 @@ int i915_active_live_selftests(struct drm_i915_private *i915)
SUBTEST(live_active_barrier),
};
- if (intel_gt_is_wedged(&i915->gt))
+ if (intel_gt_is_wedged(to_gt(i915)))
return 0;
return i915_subtests(tests, i915);
diff --git a/drivers/gpu/drm/i915/selftests/i915_gem.c b/drivers/gpu/drm/i915/selftests/i915_gem.c
index 152d9ab..b557688 100644
--- a/drivers/gpu/drm/i915/selftests/i915_gem.c
+++ b/drivers/gpu/drm/i915/selftests/i915_gem.c
@@ -248,7 +248,7 @@ int i915_gem_live_selftests(struct drm_i915_private *i915)
SUBTEST(igt_gem_ww_ctx),
};
- if (intel_gt_is_wedged(&i915->gt))
+ if (intel_gt_is_wedged(to_gt(i915)))
return 0;
return i915_live_subtests(tests, i915);
diff --git a/drivers/gpu/drm/i915/selftests/i915_gem_evict.c b/drivers/gpu/drm/i915/selftests/i915_gem_evict.c
index f99bb01..75b709c 100644
--- a/drivers/gpu/drm/i915/selftests/i915_gem_evict.c
+++ b/drivers/gpu/drm/i915/selftests/i915_gem_evict.c
@@ -442,6 +442,7 @@ static int igt_evict_contexts(void *arg)
/* Overfill the GGTT with context objects and so try to evict one. */
for_each_engine(engine, gt, id) {
struct i915_sw_fence fence;
+ struct i915_request *last = NULL;
count = 0;
onstack_fence_init(&fence);
@@ -479,6 +480,9 @@ static int igt_evict_contexts(void *arg)
i915_request_add(rq);
count++;
+ if (last)
+ i915_request_put(last);
+ last = i915_request_get(rq);
err = 0;
} while(1);
onstack_fence_fini(&fence);
@@ -486,6 +490,21 @@ static int igt_evict_contexts(void *arg)
count, engine->name);
if (err)
break;
+ if (last) {
+ if (i915_request_wait(last, 0, HZ) < 0) {
+ err = -EIO;
+ i915_request_put(last);
+ pr_err("Failed waiting for last request (on %s)",
+ engine->name);
+ break;
+ }
+ i915_request_put(last);
+ }
+ err = intel_gt_wait_for_idle(engine->gt, HZ * 3);
+ if (err) {
+ pr_err("Failed to idle GT (on %s)", engine->name);
+ break;
+ }
}
mutex_lock(&ggtt->vm.mutex);
@@ -526,7 +545,7 @@ int i915_gem_evict_mock_selftests(void)
return -ENOMEM;
with_intel_runtime_pm(&i915->runtime_pm, wakeref)
- err = i915_subtests(tests, &i915->gt);
+ err = i915_subtests(tests, to_gt(i915));
mock_destroy_device(i915);
return err;
@@ -538,8 +557,8 @@ int i915_gem_evict_live_selftests(struct drm_i915_private *i915)
SUBTEST(igt_evict_contexts),
};
- if (intel_gt_is_wedged(&i915->gt))
+ if (intel_gt_is_wedged(to_gt(i915)))
return 0;
- return intel_gt_live_subtests(tests, &i915->gt);
+ return intel_gt_live_subtests(tests, to_gt(i915));
}
diff --git a/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c b/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c
index 46f4236..575705c 100644
--- a/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c
+++ b/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c
@@ -155,7 +155,7 @@ static int igt_ppgtt_alloc(void *arg)
if (!HAS_PPGTT(dev_priv))
return 0;
- ppgtt = i915_ppgtt_create(&dev_priv->gt, 0);
+ ppgtt = i915_ppgtt_create(to_gt(dev_priv), 0);
if (IS_ERR(ppgtt))
return PTR_ERR(ppgtt);
@@ -1053,7 +1053,7 @@ static int exercise_ppgtt(struct drm_i915_private *dev_priv,
if (IS_ERR(file))
return PTR_ERR(file);
- ppgtt = i915_ppgtt_create(&dev_priv->gt, 0);
+ ppgtt = i915_ppgtt_create(to_gt(dev_priv), 0);
if (IS_ERR(ppgtt)) {
err = PTR_ERR(ppgtt);
goto out_free;
@@ -1275,7 +1275,7 @@ static void track_vma_bind(struct i915_vma *vma)
__i915_gem_object_pin_pages(obj);
- GEM_BUG_ON(vma->pages);
+ GEM_BUG_ON(atomic_read(&vma->pages_count));
atomic_set(&vma->pages_count, I915_VMA_PAGES_ACTIVE);
__i915_gem_object_pin_pages(obj);
vma->pages = obj->mm.pages;
@@ -1953,7 +1953,9 @@ static int igt_cs_tlb(void *arg)
goto end;
}
- err = vma->ops->set_pages(vma);
+ i915_gem_object_lock(bbe, NULL);
+ err = i915_vma_get_pages(vma);
+ i915_gem_object_unlock(bbe);
if (err)
goto end;
@@ -1994,7 +1996,7 @@ static int igt_cs_tlb(void *arg)
i915_request_put(rq);
}
- vma->ops->clear_pages(vma);
+ i915_vma_put_pages(vma);
err = context_sync(ce);
if (err) {
@@ -2009,7 +2011,9 @@ static int igt_cs_tlb(void *arg)
goto end;
}
- err = vma->ops->set_pages(vma);
+ i915_gem_object_lock(act, NULL);
+ err = i915_vma_get_pages(vma);
+ i915_gem_object_unlock(act);
if (err)
goto end;
@@ -2047,7 +2051,7 @@ static int igt_cs_tlb(void *arg)
}
end_spin(batch, count - 1);
- vma->ops->clear_pages(vma);
+ i915_vma_put_pages(vma);
err = context_sync(ce);
if (err) {
diff --git a/drivers/gpu/drm/i915/selftests/i915_perf.c b/drivers/gpu/drm/i915/selftests/i915_perf.c
index 9e9a6cb..88db2e3 100644
--- a/drivers/gpu/drm/i915/selftests/i915_perf.c
+++ b/drivers/gpu/drm/i915/selftests/i915_perf.c
@@ -424,7 +424,7 @@ int i915_perf_live_selftests(struct drm_i915_private *i915)
if (!perf->metrics_kobj || !perf->ops.enable_metric_set)
return 0;
- if (intel_gt_is_wedged(&i915->gt))
+ if (intel_gt_is_wedged(to_gt(i915)))
return 0;
err = alloc_empty_config(&i915->perf);
diff --git a/drivers/gpu/drm/i915/selftests/i915_request.c b/drivers/gpu/drm/i915/selftests/i915_request.c
index d67710d..92a859b 100644
--- a/drivers/gpu/drm/i915/selftests/i915_request.c
+++ b/drivers/gpu/drm/i915/selftests/i915_request.c
@@ -209,6 +209,10 @@ static int igt_request_rewind(void *arg)
int err = -EINVAL;
ctx[0] = mock_context(i915, "A");
+ if (!ctx[0]) {
+ err = -ENOMEM;
+ goto err_ctx_0;
+ }
ce = i915_gem_context_get_engine(ctx[0], RCS0);
GEM_BUG_ON(IS_ERR(ce));
@@ -223,6 +227,10 @@ static int igt_request_rewind(void *arg)
i915_request_add(request);
ctx[1] = mock_context(i915, "B");
+ if (!ctx[1]) {
+ err = -ENOMEM;
+ goto err_ctx_1;
+ }
ce = i915_gem_context_get_engine(ctx[1], RCS0);
GEM_BUG_ON(IS_ERR(ce));
@@ -261,9 +269,11 @@ static int igt_request_rewind(void *arg)
i915_request_put(vip);
err_context_1:
mock_context_close(ctx[1]);
+err_ctx_1:
i915_request_put(request);
err_context_0:
mock_context_close(ctx[0]);
+err_ctx_0:
mock_device_flush(i915);
return err;
}
@@ -831,7 +841,7 @@ static struct i915_vma *empty_batch(struct drm_i915_private *i915)
__i915_gem_object_flush_map(obj, 0, 64);
i915_gem_object_unpin_map(obj);
- intel_gt_chipset_flush(&i915->gt);
+ intel_gt_chipset_flush(to_gt(i915));
vma = i915_vma_instance(obj, &i915->ggtt.vm, NULL);
if (IS_ERR(vma)) {
@@ -972,7 +982,7 @@ static struct i915_vma *recursive_batch(struct drm_i915_private *i915)
if (IS_ERR(obj))
return ERR_CAST(obj);
- vma = i915_vma_instance(obj, i915->gt.vm, NULL);
+ vma = i915_vma_instance(obj, to_gt(i915)->vm, NULL);
if (IS_ERR(vma)) {
err = PTR_ERR(vma);
goto err;
@@ -1004,7 +1014,7 @@ static struct i915_vma *recursive_batch(struct drm_i915_private *i915)
__i915_gem_object_flush_map(obj, 0, 64);
i915_gem_object_unpin_map(obj);
- intel_gt_chipset_flush(&i915->gt);
+ intel_gt_chipset_flush(to_gt(i915));
return vma;
@@ -1690,7 +1700,7 @@ int i915_request_live_selftests(struct drm_i915_private *i915)
SUBTEST(live_breadcrumbs_smoketest),
};
- if (intel_gt_is_wedged(&i915->gt))
+ if (intel_gt_is_wedged(to_gt(i915)))
return 0;
return i915_subtests(tests, i915);
@@ -2805,7 +2815,7 @@ static int p_sync0(void *arg)
i915_request_add(rq);
err = 0;
- if (i915_request_wait(rq, 0, HZ / 5) < 0)
+ if (i915_request_wait(rq, 0, HZ) < 0)
err = -ETIME;
i915_request_put(rq);
if (err)
@@ -2876,7 +2886,7 @@ static int p_sync1(void *arg)
i915_request_add(rq);
err = 0;
- if (prev && i915_request_wait(prev, 0, HZ / 5) < 0)
+ if (prev && i915_request_wait(prev, 0, HZ) < 0)
err = -ETIME;
i915_request_put(prev);
prev = rq;
@@ -3081,7 +3091,7 @@ int i915_request_perf_selftests(struct drm_i915_private *i915)
SUBTEST(perf_parallel_engines),
};
- if (intel_gt_is_wedged(&i915->gt))
+ if (intel_gt_is_wedged(to_gt(i915)))
return 0;
return i915_subtests(tests, i915);
diff --git a/drivers/gpu/drm/i915/selftests/i915_selftest.c b/drivers/gpu/drm/i915/selftests/i915_selftest.c
index 484759c..2d6d7bd 100644
--- a/drivers/gpu/drm/i915/selftests/i915_selftest.c
+++ b/drivers/gpu/drm/i915/selftests/i915_selftest.c
@@ -298,10 +298,10 @@ int __i915_live_setup(void *data)
struct drm_i915_private *i915 = data;
/* The selftests expect an idle system */
- if (intel_gt_pm_wait_for_idle(&i915->gt))
+ if (intel_gt_pm_wait_for_idle(to_gt(i915)))
return -EIO;
- return intel_gt_terminally_wedged(&i915->gt);
+ return intel_gt_terminally_wedged(to_gt(i915));
}
int __i915_live_teardown(int err, void *data)
diff --git a/drivers/gpu/drm/i915/selftests/i915_sw_fence.c b/drivers/gpu/drm/i915/selftests/i915_sw_fence.c
index cbf45d8..daa985e5 100644
--- a/drivers/gpu/drm/i915/selftests/i915_sw_fence.c
+++ b/drivers/gpu/drm/i915/selftests/i915_sw_fence.c
@@ -28,7 +28,7 @@
#include "../i915_selftest.h"
-static int __i915_sw_fence_call
+static int
fence_notify(struct i915_sw_fence *fence, enum i915_sw_fence_notify state)
{
switch (state) {
diff --git a/drivers/gpu/drm/i915/selftests/i915_vma.c b/drivers/gpu/drm/i915/selftests/i915_vma.c
index 1f10fe3..5c5809d 100644
--- a/drivers/gpu/drm/i915/selftests/i915_vma.c
+++ b/drivers/gpu/drm/i915/selftests/i915_vma.c
@@ -691,7 +691,11 @@ static int igt_vma_rotate_remap(void *arg)
}
i915_vma_unpin(vma);
-
+ err = i915_vma_unbind(vma);
+ if (err) {
+ pr_err("Unbinding returned %i\n", err);
+ goto out_object;
+ }
cond_resched();
}
}
@@ -848,6 +852,11 @@ static int igt_vma_partial(void *arg)
i915_vma_unpin(vma);
nvma++;
+ err = i915_vma_unbind(vma);
+ if (err) {
+ pr_err("Unbinding returned %i\n", err);
+ goto out_object;
+ }
cond_resched();
}
@@ -882,6 +891,12 @@ static int igt_vma_partial(void *arg)
i915_vma_unpin(vma);
+ err = i915_vma_unbind(vma);
+ if (err) {
+ pr_err("Unbinding returned %i\n", err);
+ goto out_object;
+ }
+
count = 0;
list_for_each_entry(vma, &obj->vma.list, obj_link)
count++;
diff --git a/drivers/gpu/drm/i915/selftests/igt_flush_test.c b/drivers/gpu/drm/i915/selftests/igt_flush_test.c
index a6c71fc..b845946 100644
--- a/drivers/gpu/drm/i915/selftests/igt_flush_test.c
+++ b/drivers/gpu/drm/i915/selftests/igt_flush_test.c
@@ -14,7 +14,7 @@
int igt_flush_test(struct drm_i915_private *i915)
{
- struct intel_gt *gt = &i915->gt;
+ struct intel_gt *gt = to_gt(i915);
int ret = intel_gt_is_wedged(gt) ? -EIO : 0;
cond_resched();
diff --git a/drivers/gpu/drm/i915/selftests/igt_live_test.c b/drivers/gpu/drm/i915/selftests/igt_live_test.c
index 1c72154..72b58b66 100644
--- a/drivers/gpu/drm/i915/selftests/igt_live_test.c
+++ b/drivers/gpu/drm/i915/selftests/igt_live_test.c
@@ -16,7 +16,7 @@ int igt_live_test_begin(struct igt_live_test *t,
const char *func,
const char *name)
{
- struct intel_gt *gt = &i915->gt;
+ struct intel_gt *gt = to_gt(i915);
struct intel_engine_cs *engine;
enum intel_engine_id id;
int err;
@@ -57,7 +57,7 @@ int igt_live_test_end(struct igt_live_test *t)
return -EIO;
}
- for_each_engine(engine, &i915->gt, id) {
+ for_each_engine(engine, to_gt(i915), id) {
if (t->reset_engine[id] ==
i915_reset_engine_count(&i915->gpu_error, engine))
continue;
diff --git a/drivers/gpu/drm/i915/selftests/igt_reset.c b/drivers/gpu/drm/i915/selftests/igt_reset.c
index 9f8590b..a2838c6 100644
--- a/drivers/gpu/drm/i915/selftests/igt_reset.c
+++ b/drivers/gpu/drm/i915/selftests/igt_reset.c
@@ -36,7 +36,7 @@ void igt_global_reset_unlock(struct intel_gt *gt)
enum intel_engine_id id;
for_each_engine(engine, gt, id)
- clear_bit(I915_RESET_ENGINE + id, >->reset.flags);
+ clear_and_wake_up_bit(I915_RESET_ENGINE + id, >->reset.flags);
clear_bit(I915_RESET_BACKOFF, >->reset.flags);
wake_up_all(>->reset.queue);
diff --git a/drivers/gpu/drm/i915/selftests/intel_memory_region.c b/drivers/gpu/drm/i915/selftests/intel_memory_region.c
index 418caae..8255561 100644
--- a/drivers/gpu/drm/i915/selftests/intel_memory_region.c
+++ b/drivers/gpu/drm/i915/selftests/intel_memory_region.c
@@ -225,7 +225,7 @@ static int igt_mock_reserve(void *arg)
out_close:
close_objects(mem, &objects);
- intel_memory_region_put(mem);
+ intel_memory_region_destroy(mem);
out_free_order:
kfree(order);
return err;
@@ -439,7 +439,7 @@ static int igt_mock_splintered_region(void *arg)
out_close:
close_objects(mem, &objects);
out_put:
- intel_memory_region_put(mem);
+ intel_memory_region_destroy(mem);
return err;
}
@@ -507,7 +507,7 @@ static int igt_mock_max_segment(void *arg)
out_close:
close_objects(mem, &objects);
out_put:
- intel_memory_region_put(mem);
+ intel_memory_region_destroy(mem);
return err;
}
@@ -1196,7 +1196,7 @@ int intel_memory_region_mock_selftests(void)
err = i915_subtests(tests, mem);
- intel_memory_region_put(mem);
+ intel_memory_region_destroy(mem);
out_unref:
mock_destroy_device(i915);
return err;
@@ -1217,7 +1217,7 @@ int intel_memory_region_live_selftests(struct drm_i915_private *i915)
return 0;
}
- if (intel_gt_is_wedged(&i915->gt))
+ if (intel_gt_is_wedged(to_gt(i915)))
return 0;
return i915_live_subtests(tests, i915);
@@ -1229,7 +1229,7 @@ int intel_memory_region_perf_selftests(struct drm_i915_private *i915)
SUBTEST(perf_memcpy),
};
- if (intel_gt_is_wedged(&i915->gt))
+ if (intel_gt_is_wedged(to_gt(i915)))
return 0;
return i915_live_subtests(tests, i915);
diff --git a/drivers/gpu/drm/i915/selftests/intel_uncore.c b/drivers/gpu/drm/i915/selftests/intel_uncore.c
index bc81281..cdd1967 100644
--- a/drivers/gpu/drm/i915/selftests/intel_uncore.c
+++ b/drivers/gpu/drm/i915/selftests/intel_uncore.c
@@ -344,5 +344,5 @@ int intel_uncore_live_selftests(struct drm_i915_private *i915)
SUBTEST(live_forcewake_domains),
};
- return intel_gt_live_subtests(tests, &i915->gt);
+ return intel_gt_live_subtests(tests, to_gt(i915));
}
diff --git a/drivers/gpu/drm/i915/selftests/lib_sw_fence.c b/drivers/gpu/drm/i915/selftests/lib_sw_fence.c
index 080b90b..bf2752c 100644
--- a/drivers/gpu/drm/i915/selftests/lib_sw_fence.c
+++ b/drivers/gpu/drm/i915/selftests/lib_sw_fence.c
@@ -26,7 +26,7 @@
/* Small library of different fence types useful for writing tests */
-static int __i915_sw_fence_call
+static int
nop_fence_notify(struct i915_sw_fence *fence, enum i915_sw_fence_notify state)
{
return NOTIFY_DONE;
@@ -41,12 +41,12 @@ void __onstack_fence_init(struct i915_sw_fence *fence,
__init_waitqueue_head(&fence->wait, name, key);
atomic_set(&fence->pending, 1);
fence->error = 0;
- fence->flags = (unsigned long)nop_fence_notify;
+ fence->fn = nop_fence_notify;
}
void onstack_fence_fini(struct i915_sw_fence *fence)
{
- if (!fence->flags)
+ if (!fence->fn)
return;
i915_sw_fence_commit(fence);
@@ -89,7 +89,7 @@ struct heap_fence {
};
};
-static int __i915_sw_fence_call
+static int
heap_fence_notify(struct i915_sw_fence *fence, enum i915_sw_fence_notify state)
{
struct heap_fence *h = container_of(fence, typeof(*h), fence);
diff --git a/drivers/gpu/drm/i915/selftests/mock_gem_device.c b/drivers/gpu/drm/i915/selftests/mock_gem_device.c
index 4f81801..8aa7b1d 100644
--- a/drivers/gpu/drm/i915/selftests/mock_gem_device.c
+++ b/drivers/gpu/drm/i915/selftests/mock_gem_device.c
@@ -45,7 +45,7 @@
void mock_device_flush(struct drm_i915_private *i915)
{
- struct intel_gt *gt = &i915->gt;
+ struct intel_gt *gt = to_gt(i915);
struct intel_engine_cs *engine;
enum intel_engine_id id;
@@ -64,7 +64,7 @@ static void mock_device_release(struct drm_device *dev)
goto out;
mock_device_flush(i915);
- intel_gt_driver_remove(&i915->gt);
+ intel_gt_driver_remove(to_gt(i915));
i915_gem_drain_workqueue(i915);
i915_gem_drain_freed_objects(i915);
@@ -73,7 +73,7 @@ static void mock_device_release(struct drm_device *dev)
destroy_workqueue(i915->wq);
intel_region_ttm_device_fini(i915);
- intel_gt_driver_late_release(&i915->gt);
+ intel_gt_driver_late_release(to_gt(i915));
intel_memory_regions_driver_release(i915);
drm_mode_config_cleanup(&i915->drm);
@@ -165,7 +165,7 @@ struct drm_i915_private *mock_gem_device(void)
/* Using the global GTT may ask questions about KMS users, so prepare */
drm_mode_config_init(&i915->drm);
- mkwrite_device_info(i915)->graphics_ver = -1;
+ mkwrite_device_info(i915)->graphics.ver = -1;
mkwrite_device_info(i915)->page_sizes =
I915_GTT_PAGE_SIZE_4K |
@@ -175,12 +175,14 @@ struct drm_i915_private *mock_gem_device(void)
mkwrite_device_info(i915)->memory_regions = REGION_SMEM;
intel_memory_regions_hw_probe(i915);
- mock_uncore_init(&i915->uncore, i915);
+ spin_lock_init(&i915->gpu_error.lock);
i915_gem_init__mm(i915);
- intel_gt_init_early(&i915->gt, i915);
- atomic_inc(&i915->gt.wakeref.count); /* disable; no hw support */
- i915->gt.awake = -ENODEV;
+ intel_gt_init_early(to_gt(i915), i915);
+ __intel_gt_init_early(to_gt(i915), i915);
+ mock_uncore_init(&i915->uncore, i915);
+ atomic_inc(&to_gt(i915)->wakeref.count); /* disable; no hw support */
+ to_gt(i915)->awake = -ENODEV;
ret = intel_region_ttm_device_init(i915);
if (ret)
@@ -193,19 +195,19 @@ struct drm_i915_private *mock_gem_device(void)
mock_init_contexts(i915);
mock_init_ggtt(i915, &i915->ggtt);
- i915->gt.vm = i915_vm_get(&i915->ggtt.vm);
+ to_gt(i915)->vm = i915_vm_get(&i915->ggtt.vm);
mkwrite_device_info(i915)->platform_engine_mask = BIT(0);
- i915->gt.info.engine_mask = BIT(0);
+ to_gt(i915)->info.engine_mask = BIT(0);
- i915->gt.engine[RCS0] = mock_engine(i915, "mock", RCS0);
- if (!i915->gt.engine[RCS0])
+ to_gt(i915)->engine[RCS0] = mock_engine(i915, "mock", RCS0);
+ if (!to_gt(i915)->engine[RCS0])
goto err_unlock;
- if (mock_engine_init(i915->gt.engine[RCS0]))
+ if (mock_engine_init(to_gt(i915)->engine[RCS0]))
goto err_context;
- __clear_bit(I915_WEDGED, &i915->gt.reset.flags);
+ __clear_bit(I915_WEDGED, &to_gt(i915)->reset.flags);
intel_engines_driver_register(i915);
i915->do_release = true;
@@ -214,13 +216,13 @@ struct drm_i915_private *mock_gem_device(void)
return i915;
err_context:
- intel_gt_driver_remove(&i915->gt);
+ intel_gt_driver_remove(to_gt(i915));
err_unlock:
destroy_workqueue(i915->wq);
err_drv:
intel_region_ttm_device_fini(i915);
err_ttm:
- intel_gt_driver_late_release(&i915->gt);
+ intel_gt_driver_late_release(to_gt(i915));
intel_memory_regions_driver_release(i915);
drm_mode_config_cleanup(&i915->drm);
mock_destroy_device(i915);
diff --git a/drivers/gpu/drm/i915/selftests/mock_gtt.c b/drivers/gpu/drm/i915/selftests/mock_gtt.c
index cc047ec..1802baf 100644
--- a/drivers/gpu/drm/i915/selftests/mock_gtt.c
+++ b/drivers/gpu/drm/i915/selftests/mock_gtt.c
@@ -70,7 +70,7 @@ struct i915_ppgtt *mock_ppgtt(struct drm_i915_private *i915, const char *name)
if (!ppgtt)
return NULL;
- ppgtt->vm.gt = &i915->gt;
+ ppgtt->vm.gt = to_gt(i915);
ppgtt->vm.i915 = i915;
ppgtt->vm.total = round_down(U64_MAX, PAGE_SIZE);
ppgtt->vm.dma = i915->drm.dev;
@@ -78,6 +78,7 @@ struct i915_ppgtt *mock_ppgtt(struct drm_i915_private *i915, const char *name)
i915_address_space_init(&ppgtt->vm, VM_CLASS_PPGTT);
ppgtt->vm.alloc_pt_dma = alloc_pt_dma;
+ ppgtt->vm.alloc_scratch_dma = alloc_pt_dma;
ppgtt->vm.clear_range = mock_clear_range;
ppgtt->vm.insert_page = mock_insert_page;
@@ -86,8 +87,6 @@ struct i915_ppgtt *mock_ppgtt(struct drm_i915_private *i915, const char *name)
ppgtt->vm.vma_ops.bind_vma = mock_bind_ppgtt;
ppgtt->vm.vma_ops.unbind_vma = mock_unbind_ppgtt;
- ppgtt->vm.vma_ops.set_pages = ppgtt_set_pages;
- ppgtt->vm.vma_ops.clear_pages = clear_pages;
return ppgtt;
}
@@ -109,7 +108,7 @@ void mock_init_ggtt(struct drm_i915_private *i915, struct i915_ggtt *ggtt)
{
memset(ggtt, 0, sizeof(*ggtt));
- ggtt->vm.gt = &i915->gt;
+ ggtt->vm.gt = to_gt(i915);
ggtt->vm.i915 = i915;
ggtt->vm.is_ggtt = true;
@@ -118,6 +117,7 @@ void mock_init_ggtt(struct drm_i915_private *i915, struct i915_ggtt *ggtt)
ggtt->vm.total = 4096 * PAGE_SIZE;
ggtt->vm.alloc_pt_dma = alloc_pt_dma;
+ ggtt->vm.alloc_scratch_dma = alloc_pt_dma;
ggtt->vm.clear_range = mock_clear_range;
ggtt->vm.insert_page = mock_insert_page;
@@ -126,11 +126,9 @@ void mock_init_ggtt(struct drm_i915_private *i915, struct i915_ggtt *ggtt)
ggtt->vm.vma_ops.bind_vma = mock_bind_ggtt;
ggtt->vm.vma_ops.unbind_vma = mock_unbind_ggtt;
- ggtt->vm.vma_ops.set_pages = ggtt_set_pages;
- ggtt->vm.vma_ops.clear_pages = clear_pages;
i915_address_space_init(&ggtt->vm, VM_CLASS_GGTT);
- i915->gt.ggtt = ggtt;
+ to_gt(i915)->ggtt = ggtt;
}
void mock_fini_ggtt(struct i915_ggtt *ggtt)
diff --git a/drivers/gpu/drm/i915/selftests/mock_region.c b/drivers/gpu/drm/i915/selftests/mock_region.c
index 7579300..19bff8a 100644
--- a/drivers/gpu/drm/i915/selftests/mock_region.c
+++ b/drivers/gpu/drm/i915/selftests/mock_region.c
@@ -15,9 +15,9 @@
static void mock_region_put_pages(struct drm_i915_gem_object *obj,
struct sg_table *pages)
{
+ i915_refct_sgt_put(obj->mm.rsgt);
+ obj->mm.rsgt = NULL;
intel_region_ttm_resource_free(obj->mm.region, obj->mm.res);
- sg_free_table(pages);
- kfree(pages);
}
static int mock_region_get_pages(struct drm_i915_gem_object *obj)
@@ -36,12 +36,14 @@ static int mock_region_get_pages(struct drm_i915_gem_object *obj)
if (IS_ERR(obj->mm.res))
return PTR_ERR(obj->mm.res);
- pages = intel_region_ttm_resource_to_st(obj->mm.region, obj->mm.res);
- if (IS_ERR(pages)) {
- err = PTR_ERR(pages);
+ obj->mm.rsgt = intel_region_ttm_resource_to_rsgt(obj->mm.region,
+ obj->mm.res);
+ if (IS_ERR(obj->mm.rsgt)) {
+ err = PTR_ERR(obj->mm.rsgt);
goto err_free_resource;
}
+ pages = &obj->mm.rsgt->table;
__i915_gem_object_set_pages(obj, pages, i915_sg_dma_sizes(pages->sgl));
return 0;
@@ -82,13 +84,16 @@ static int mock_object_init(struct intel_memory_region *mem,
return 0;
}
-static void mock_region_fini(struct intel_memory_region *mem)
+static int mock_region_fini(struct intel_memory_region *mem)
{
struct drm_i915_private *i915 = mem->i915;
int instance = mem->instance;
+ int ret;
- intel_region_ttm_fini(mem);
+ ret = intel_region_ttm_fini(mem);
ida_free(&i915->selftest.mock_region_instances, instance);
+
+ return ret;
}
static const struct intel_memory_region_ops mock_region_ops = {
diff --git a/drivers/gpu/drm/i915/selftests/mock_uncore.c b/drivers/gpu/drm/i915/selftests/mock_uncore.c
index ca57e40..f2d6be5 100644
--- a/drivers/gpu/drm/i915/selftests/mock_uncore.c
+++ b/drivers/gpu/drm/i915/selftests/mock_uncore.c
@@ -42,7 +42,7 @@ __nop_read(64)
void mock_uncore_init(struct intel_uncore *uncore,
struct drm_i915_private *i915)
{
- intel_uncore_init_early(uncore, i915);
+ intel_uncore_init_early(uncore, to_gt(i915));
ASSIGN_RAW_WRITE_MMIO_VFUNCS(uncore, nop);
ASSIGN_RAW_READ_MMIO_VFUNCS(uncore, nop);
diff --git a/drivers/gpu/drm/i915/vlv_sideband.c b/drivers/gpu/drm/i915/vlv_sideband.c
index 3538073..ed2ac57 100644
--- a/drivers/gpu/drm/i915/vlv_sideband.c
+++ b/drivers/gpu/drm/i915/vlv_sideband.c
@@ -3,9 +3,8 @@
* Copyright © 2013-2021 Intel Corporation
*/
-#include <asm/iosf_mbi.h>
-
#include "i915_drv.h"
+#include "i915_iosf_mbi.h"
#include "vlv_sideband.h"
/*
diff --git a/drivers/gpu/drm/imx/Kconfig b/drivers/gpu/drm/imx/Kconfig
index b5fa0e4..bb9738c 100644
--- a/drivers/gpu/drm/imx/Kconfig
+++ b/drivers/gpu/drm/imx/Kconfig
@@ -4,7 +4,7 @@
select DRM_KMS_HELPER
select VIDEOMODE_HELPERS
select DRM_GEM_CMA_HELPER
- select DRM_KMS_CMA_HELPER
+ select DRM_KMS_HELPER
depends on DRM && (ARCH_MXC || ARCH_MULTIPLATFORM || COMPILE_TEST)
depends on IMX_IPUV3_CORE
help
diff --git a/drivers/gpu/drm/imx/dcss/Kconfig b/drivers/gpu/drm/imx/dcss/Kconfig
index 2b17a96..7374f19 100644
--- a/drivers/gpu/drm/imx/dcss/Kconfig
+++ b/drivers/gpu/drm/imx/dcss/Kconfig
@@ -1,7 +1,7 @@
config DRM_IMX_DCSS
tristate "i.MX8MQ DCSS"
select IMX_IRQSTEER
- select DRM_KMS_CMA_HELPER
+ select DRM_KMS_HELPER
select VIDEOMODE_HELPERS
depends on DRM && ARCH_MXC && ARM64
help
diff --git a/drivers/gpu/drm/ingenic/Kconfig b/drivers/gpu/drm/ingenic/Kconfig
index 3b57f8b..001f59f 100644
--- a/drivers/gpu/drm/ingenic/Kconfig
+++ b/drivers/gpu/drm/ingenic/Kconfig
@@ -8,7 +8,6 @@
select DRM_BRIDGE
select DRM_PANEL_BRIDGE
select DRM_KMS_HELPER
- select DRM_KMS_CMA_HELPER
select DRM_GEM_CMA_HELPER
select VT_HW_CONSOLE_BINDING if FRAMEBUFFER_CONSOLE
help
diff --git a/drivers/gpu/drm/ingenic/ingenic-drm-drv.c b/drivers/gpu/drm/ingenic/ingenic-drm-drv.c
index a5df1c8..b4943a5 100644
--- a/drivers/gpu/drm/ingenic/ingenic-drm-drv.c
+++ b/drivers/gpu/drm/ingenic/ingenic-drm-drv.c
@@ -21,6 +21,7 @@
#include <drm/drm_atomic.h>
#include <drm/drm_atomic_helper.h>
#include <drm/drm_bridge.h>
+#include <drm/drm_bridge_connector.h>
#include <drm/drm_color_mgmt.h>
#include <drm/drm_crtc.h>
#include <drm/drm_crtc_helper.h>
@@ -41,6 +42,8 @@
#include <drm/drm_probe_helper.h>
#include <drm/drm_vblank.h>
+#define HWDESC_PALETTE 2
+
struct ingenic_dma_hwdesc {
u32 next;
u32 addr;
@@ -49,9 +52,7 @@ struct ingenic_dma_hwdesc {
} __aligned(16);
struct ingenic_dma_hwdescs {
- struct ingenic_dma_hwdesc hwdesc_f0;
- struct ingenic_dma_hwdesc hwdesc_f1;
- struct ingenic_dma_hwdesc hwdesc_pal;
+ struct ingenic_dma_hwdesc hwdesc[3];
u16 palette[256] __aligned(16);
};
@@ -64,6 +65,11 @@ struct jz_soc_info {
unsigned int num_formats_f0, num_formats_f1;
};
+struct ingenic_drm_private_state {
+ struct drm_private_state base;
+ bool use_palette;
+};
+
struct ingenic_drm {
struct drm_device drm;
/*
@@ -99,8 +105,53 @@ struct ingenic_drm {
struct mutex clk_mutex;
bool update_clk_rate;
struct notifier_block clock_nb;
+
+ struct drm_private_obj private_obj;
};
+struct ingenic_drm_bridge {
+ struct drm_encoder encoder;
+ struct drm_bridge bridge, *next_bridge;
+
+ struct drm_bus_cfg bus_cfg;
+};
+
+static inline struct ingenic_drm_bridge *
+to_ingenic_drm_bridge(struct drm_encoder *encoder)
+{
+ return container_of(encoder, struct ingenic_drm_bridge, encoder);
+}
+
+static inline struct ingenic_drm_private_state *
+to_ingenic_drm_priv_state(struct drm_private_state *state)
+{
+ return container_of(state, struct ingenic_drm_private_state, base);
+}
+
+static struct ingenic_drm_private_state *
+ingenic_drm_get_priv_state(struct ingenic_drm *priv, struct drm_atomic_state *state)
+{
+ struct drm_private_state *priv_state;
+
+ priv_state = drm_atomic_get_private_obj_state(state, &priv->private_obj);
+ if (IS_ERR(priv_state))
+ return ERR_CAST(priv_state);
+
+ return to_ingenic_drm_priv_state(priv_state);
+}
+
+static struct ingenic_drm_private_state *
+ingenic_drm_get_new_priv_state(struct ingenic_drm *priv, struct drm_atomic_state *state)
+{
+ struct drm_private_state *priv_state;
+
+ priv_state = drm_atomic_get_new_private_obj_state(state, &priv->private_obj);
+ if (!priv_state)
+ return NULL;
+
+ return to_ingenic_drm_priv_state(priv_state);
+}
+
static bool ingenic_drm_writeable_reg(struct device *dev, unsigned int reg)
{
switch (reg) {
@@ -141,6 +192,14 @@ static inline struct ingenic_drm *drm_nb_get_priv(struct notifier_block *nb)
return container_of(nb, struct ingenic_drm, clock_nb);
}
+static inline dma_addr_t dma_hwdesc_addr(const struct ingenic_drm *priv,
+ unsigned int idx)
+{
+ u32 offset = offsetof(struct ingenic_dma_hwdescs, hwdesc[idx]);
+
+ return priv->dma_hwdescs_phys + offset;
+}
+
static int ingenic_drm_update_pixclk(struct notifier_block *nb,
unsigned long action,
void *data)
@@ -163,9 +222,20 @@ static void ingenic_drm_crtc_atomic_enable(struct drm_crtc *crtc,
struct drm_atomic_state *state)
{
struct ingenic_drm *priv = drm_crtc_get_priv(crtc);
+ struct ingenic_drm_private_state *priv_state;
+ unsigned int next_id;
+
+ priv_state = ingenic_drm_get_priv_state(priv, state);
+ if (WARN_ON(IS_ERR(priv_state)))
+ return;
regmap_write(priv->map, JZ_REG_LCD_STATE, 0);
+ /* Set addresses of our DMA descriptor chains */
+ next_id = priv_state->use_palette ? HWDESC_PALETTE : 0;
+ regmap_write(priv->map, JZ_REG_LCD_DA0, dma_hwdesc_addr(priv, next_id));
+ regmap_write(priv->map, JZ_REG_LCD_DA1, dma_hwdesc_addr(priv, 1));
+
regmap_update_bits(priv->map, JZ_REG_LCD_CTRL,
JZ_LCD_CTRL_ENABLE | JZ_LCD_CTRL_DISABLE,
JZ_LCD_CTRL_ENABLE);
@@ -369,6 +439,7 @@ static int ingenic_drm_plane_atomic_check(struct drm_plane *plane,
struct drm_plane_state *new_plane_state = drm_atomic_get_new_plane_state(state,
plane);
struct ingenic_drm *priv = drm_device_get_priv(plane->dev);
+ struct ingenic_drm_private_state *priv_state;
struct drm_crtc_state *crtc_state;
struct drm_crtc *crtc = new_plane_state->crtc ?: old_plane_state->crtc;
int ret;
@@ -381,6 +452,10 @@ static int ingenic_drm_plane_atomic_check(struct drm_plane *plane,
if (WARN_ON(!crtc_state))
return -EINVAL;
+ priv_state = ingenic_drm_get_priv_state(priv, state);
+ if (IS_ERR(priv_state))
+ return PTR_ERR(priv_state);
+
ret = drm_atomic_helper_check_plane_state(new_plane_state, crtc_state,
DRM_PLANE_HELPER_NO_SCALING,
DRM_PLANE_HELPER_NO_SCALING,
@@ -399,6 +474,9 @@ static int ingenic_drm_plane_atomic_check(struct drm_plane *plane,
(new_plane_state->src_h >> 16) != new_plane_state->crtc_h))
return -EINVAL;
+ priv_state->use_palette = new_plane_state->fb &&
+ new_plane_state->fb->format->format == DRM_FORMAT_C8;
+
/*
* Require full modeset if enabling or disabling a plane, or changing
* its position, size or depth.
@@ -558,9 +636,10 @@ static void ingenic_drm_plane_atomic_update(struct drm_plane *plane,
struct ingenic_drm *priv = drm_device_get_priv(plane->dev);
struct drm_plane_state *newstate = drm_atomic_get_new_plane_state(state, plane);
struct drm_plane_state *oldstate = drm_atomic_get_old_plane_state(state, plane);
+ unsigned int width, height, cpp, next_id, plane_id;
+ struct ingenic_drm_private_state *priv_state;
struct drm_crtc_state *crtc_state;
struct ingenic_dma_hwdesc *hwdesc;
- unsigned int width, height, cpp, offset;
dma_addr_t addr;
u32 fourcc;
@@ -569,32 +648,26 @@ static void ingenic_drm_plane_atomic_update(struct drm_plane *plane,
drm_fb_cma_sync_non_coherent(&priv->drm, oldstate, newstate);
crtc_state = newstate->crtc->state;
+ plane_id = !!(priv->soc_info->has_osd && plane != &priv->f0);
addr = drm_fb_cma_get_gem_addr(newstate->fb, newstate, 0);
width = newstate->src_w >> 16;
height = newstate->src_h >> 16;
cpp = newstate->fb->format->cpp[0];
- if (!priv->soc_info->has_osd || plane == &priv->f0)
- hwdesc = &priv->dma_hwdescs->hwdesc_f0;
- else
- hwdesc = &priv->dma_hwdescs->hwdesc_f1;
+ priv_state = ingenic_drm_get_new_priv_state(priv, state);
+ next_id = (priv_state && priv_state->use_palette) ? HWDESC_PALETTE : plane_id;
+ hwdesc = &priv->dma_hwdescs->hwdesc[plane_id];
hwdesc->addr = addr;
hwdesc->cmd = JZ_LCD_CMD_EOF_IRQ | (width * height * cpp / 4);
+ hwdesc->next = dma_hwdesc_addr(priv, next_id);
if (drm_atomic_crtc_needs_modeset(crtc_state)) {
fourcc = newstate->fb->format->format;
ingenic_drm_plane_config(priv->dev, plane, fourcc);
- if (fourcc == DRM_FORMAT_C8)
- offset = offsetof(struct ingenic_dma_hwdescs, hwdesc_pal);
- else
- offset = offsetof(struct ingenic_dma_hwdescs, hwdesc_f0);
-
- priv->dma_hwdescs->hwdesc_f0.next = priv->dma_hwdescs_phys + offset;
-
crtc_state->color_mgmt_changed = fourcc == DRM_FORMAT_C8;
}
@@ -609,11 +682,10 @@ static void ingenic_drm_encoder_atomic_mode_set(struct drm_encoder *encoder,
{
struct ingenic_drm *priv = drm_device_get_priv(encoder->dev);
struct drm_display_mode *mode = &crtc_state->adjusted_mode;
- struct drm_connector *conn = conn_state->connector;
- struct drm_display_info *info = &conn->display_info;
+ struct ingenic_drm_bridge *bridge = to_ingenic_drm_bridge(encoder);
unsigned int cfg, rgbcfg = 0;
- priv->panel_is_sharp = info->bus_flags & DRM_BUS_FLAG_SHARP_SIGNALS;
+ priv->panel_is_sharp = bridge->bus_cfg.flags & DRM_BUS_FLAG_SHARP_SIGNALS;
if (priv->panel_is_sharp) {
cfg = JZ_LCD_CFG_MODE_SPECIAL_TFT_1 | JZ_LCD_CFG_REV_POLARITY;
@@ -626,19 +698,19 @@ static void ingenic_drm_encoder_atomic_mode_set(struct drm_encoder *encoder,
cfg |= JZ_LCD_CFG_HSYNC_ACTIVE_LOW;
if (mode->flags & DRM_MODE_FLAG_NVSYNC)
cfg |= JZ_LCD_CFG_VSYNC_ACTIVE_LOW;
- if (info->bus_flags & DRM_BUS_FLAG_DE_LOW)
+ if (bridge->bus_cfg.flags & DRM_BUS_FLAG_DE_LOW)
cfg |= JZ_LCD_CFG_DE_ACTIVE_LOW;
- if (info->bus_flags & DRM_BUS_FLAG_PIXDATA_DRIVE_NEGEDGE)
+ if (bridge->bus_cfg.flags & DRM_BUS_FLAG_PIXDATA_DRIVE_NEGEDGE)
cfg |= JZ_LCD_CFG_PCLK_FALLING_EDGE;
if (!priv->panel_is_sharp) {
- if (conn->connector_type == DRM_MODE_CONNECTOR_TV) {
+ if (conn_state->connector->connector_type == DRM_MODE_CONNECTOR_TV) {
if (mode->flags & DRM_MODE_FLAG_INTERLACE)
cfg |= JZ_LCD_CFG_MODE_TV_OUT_I;
else
cfg |= JZ_LCD_CFG_MODE_TV_OUT_P;
} else {
- switch (*info->bus_formats) {
+ switch (bridge->bus_cfg.format) {
case MEDIA_BUS_FMT_RGB565_1X16:
cfg |= JZ_LCD_CFG_MODE_GENERIC_16BIT;
break;
@@ -664,20 +736,29 @@ static void ingenic_drm_encoder_atomic_mode_set(struct drm_encoder *encoder,
regmap_write(priv->map, JZ_REG_LCD_RGBC, rgbcfg);
}
-static int ingenic_drm_encoder_atomic_check(struct drm_encoder *encoder,
- struct drm_crtc_state *crtc_state,
- struct drm_connector_state *conn_state)
+static int ingenic_drm_bridge_attach(struct drm_bridge *bridge,
+ enum drm_bridge_attach_flags flags)
{
- struct drm_display_info *info = &conn_state->connector->display_info;
- struct drm_display_mode *mode = &crtc_state->adjusted_mode;
+ struct ingenic_drm_bridge *ib = to_ingenic_drm_bridge(bridge->encoder);
- if (info->num_bus_formats != 1)
- return -EINVAL;
+ return drm_bridge_attach(bridge->encoder, ib->next_bridge,
+ &ib->bridge, flags);
+}
+
+static int ingenic_drm_bridge_atomic_check(struct drm_bridge *bridge,
+ struct drm_bridge_state *bridge_state,
+ struct drm_crtc_state *crtc_state,
+ struct drm_connector_state *conn_state)
+{
+ struct drm_display_mode *mode = &crtc_state->adjusted_mode;
+ struct ingenic_drm_bridge *ib = to_ingenic_drm_bridge(bridge->encoder);
+
+ ib->bus_cfg = bridge_state->output_bus_cfg;
if (conn_state->connector->connector_type == DRM_MODE_CONNECTOR_TV)
return 0;
- switch (*info->bus_formats) {
+ switch (bridge_state->output_bus_cfg.format) {
case MEDIA_BUS_FMT_RGB888_3X8:
case MEDIA_BUS_FMT_RGB888_3X8_DELTA:
/*
@@ -764,6 +845,28 @@ ingenic_drm_gem_create_object(struct drm_device *drm, size_t size)
return &obj->base;
}
+static struct drm_private_state *
+ingenic_drm_duplicate_state(struct drm_private_obj *obj)
+{
+ struct ingenic_drm_private_state *state = to_ingenic_drm_priv_state(obj->state);
+
+ state = kmemdup(state, sizeof(*state), GFP_KERNEL);
+ if (!state)
+ return NULL;
+
+ __drm_atomic_helper_private_obj_duplicate_state(obj, &state->base);
+
+ return &state->base;
+}
+
+static void ingenic_drm_destroy_state(struct drm_private_obj *obj,
+ struct drm_private_state *state)
+{
+ struct ingenic_drm_private_state *priv_state = to_ingenic_drm_priv_state(state);
+
+ kfree(priv_state);
+}
+
DEFINE_DRM_GEM_CMA_FOPS(ingenic_drm_fops);
static const struct drm_driver ingenic_drm_driver_data = {
@@ -819,8 +922,16 @@ static const struct drm_crtc_helper_funcs ingenic_drm_crtc_helper_funcs = {
};
static const struct drm_encoder_helper_funcs ingenic_drm_encoder_helper_funcs = {
- .atomic_mode_set = ingenic_drm_encoder_atomic_mode_set,
- .atomic_check = ingenic_drm_encoder_atomic_check,
+ .atomic_mode_set = ingenic_drm_encoder_atomic_mode_set,
+};
+
+static const struct drm_bridge_funcs ingenic_drm_bridge_funcs = {
+ .attach = ingenic_drm_bridge_attach,
+ .atomic_check = ingenic_drm_bridge_atomic_check,
+ .atomic_reset = drm_atomic_helper_bridge_reset,
+ .atomic_duplicate_state = drm_atomic_helper_bridge_duplicate_state,
+ .atomic_destroy_state = drm_atomic_helper_bridge_destroy_state,
+ .atomic_get_input_bus_fmts = drm_atomic_helper_bridge_propagate_bus_fmt,
};
static const struct drm_mode_config_funcs ingenic_drm_mode_config_funcs = {
@@ -834,6 +945,11 @@ static struct drm_mode_config_helper_funcs ingenic_drm_mode_config_helpers = {
.atomic_commit_tail = drm_atomic_helper_commit_tail,
};
+static const struct drm_private_state_funcs ingenic_drm_private_state_funcs = {
+ .atomic_duplicate_state = ingenic_drm_duplicate_state,
+ .atomic_destroy_state = ingenic_drm_destroy_state,
+};
+
static void ingenic_drm_unbind_all(void *d)
{
struct ingenic_drm *priv = d;
@@ -846,21 +962,57 @@ static void __maybe_unused ingenic_drm_release_rmem(void *d)
of_reserved_mem_device_release(d);
}
+static void ingenic_drm_configure_hwdesc(struct ingenic_drm *priv,
+ unsigned int hwdesc,
+ unsigned int next_hwdesc, u32 id)
+{
+ struct ingenic_dma_hwdesc *desc = &priv->dma_hwdescs->hwdesc[hwdesc];
+
+ desc->next = dma_hwdesc_addr(priv, next_hwdesc);
+ desc->id = id;
+}
+
+static void ingenic_drm_configure_hwdesc_palette(struct ingenic_drm *priv)
+{
+ struct ingenic_dma_hwdesc *desc;
+
+ ingenic_drm_configure_hwdesc(priv, HWDESC_PALETTE, 0, 0xc0);
+
+ desc = &priv->dma_hwdescs->hwdesc[HWDESC_PALETTE];
+ desc->addr = priv->dma_hwdescs_phys
+ + offsetof(struct ingenic_dma_hwdescs, palette);
+ desc->cmd = JZ_LCD_CMD_ENABLE_PAL
+ | (sizeof(priv->dma_hwdescs->palette) / 4);
+}
+
+static void ingenic_drm_configure_hwdesc_plane(struct ingenic_drm *priv,
+ unsigned int plane)
+{
+ ingenic_drm_configure_hwdesc(priv, plane, plane, 0xf0 | plane);
+}
+
+static void ingenic_drm_atomic_private_obj_fini(struct drm_device *drm, void *private_obj)
+{
+ drm_atomic_private_obj_fini(private_obj);
+}
+
static int ingenic_drm_bind(struct device *dev, bool has_components)
{
struct platform_device *pdev = to_platform_device(dev);
+ struct ingenic_drm_private_state *private_state;
const struct jz_soc_info *soc_info;
struct ingenic_drm *priv;
struct clk *parent_clk;
struct drm_plane *primary;
struct drm_bridge *bridge;
struct drm_panel *panel;
+ struct drm_connector *connector;
struct drm_encoder *encoder;
+ struct ingenic_drm_bridge *ib;
struct drm_device *drm;
void __iomem *base;
long parent_rate;
unsigned int i, clone_mask = 0;
- dma_addr_t dma_hwdesc_phys_f0, dma_hwdesc_phys_f1;
int ret, irq;
soc_info = of_device_get_match_data(dev);
@@ -942,27 +1094,14 @@ static int ingenic_drm_bind(struct device *dev, bool has_components)
if (!priv->dma_hwdescs)
return -ENOMEM;
-
/* Configure DMA hwdesc for foreground0 plane */
- dma_hwdesc_phys_f0 = priv->dma_hwdescs_phys
- + offsetof(struct ingenic_dma_hwdescs, hwdesc_f0);
- priv->dma_hwdescs->hwdesc_f0.next = dma_hwdesc_phys_f0;
- priv->dma_hwdescs->hwdesc_f0.id = 0xf0;
+ ingenic_drm_configure_hwdesc_plane(priv, 0);
/* Configure DMA hwdesc for foreground1 plane */
- dma_hwdesc_phys_f1 = priv->dma_hwdescs_phys
- + offsetof(struct ingenic_dma_hwdescs, hwdesc_f1);
- priv->dma_hwdescs->hwdesc_f1.next = dma_hwdesc_phys_f1;
- priv->dma_hwdescs->hwdesc_f1.id = 0xf1;
+ ingenic_drm_configure_hwdesc_plane(priv, 1);
/* Configure DMA hwdesc for palette */
- priv->dma_hwdescs->hwdesc_pal.next = priv->dma_hwdescs_phys
- + offsetof(struct ingenic_dma_hwdescs, hwdesc_f0);
- priv->dma_hwdescs->hwdesc_pal.id = 0xc0;
- priv->dma_hwdescs->hwdesc_pal.addr = priv->dma_hwdescs_phys
- + offsetof(struct ingenic_dma_hwdescs, palette);
- priv->dma_hwdescs->hwdesc_pal.cmd = JZ_LCD_CMD_ENABLE_PAL
- | (sizeof(priv->dma_hwdescs->palette) / 4);
+ ingenic_drm_configure_hwdesc_palette(priv);
primary = priv->soc_info->has_osd ? &priv->f1 : &priv->f0;
@@ -1046,20 +1185,36 @@ static int ingenic_drm_bind(struct device *dev, bool has_components)
bridge = devm_drm_panel_bridge_add_typed(dev, panel,
DRM_MODE_CONNECTOR_DPI);
- encoder = drmm_plain_encoder_alloc(drm, NULL, DRM_MODE_ENCODER_DPI, NULL);
- if (IS_ERR(encoder)) {
- ret = PTR_ERR(encoder);
+ ib = drmm_encoder_alloc(drm, struct ingenic_drm_bridge, encoder,
+ NULL, DRM_MODE_ENCODER_DPI, NULL);
+ if (IS_ERR(ib)) {
+ ret = PTR_ERR(ib);
dev_err(dev, "Failed to init encoder: %d\n", ret);
return ret;
}
- encoder->possible_crtcs = 1;
+ encoder = &ib->encoder;
+ encoder->possible_crtcs = drm_crtc_mask(&priv->crtc);
drm_encoder_helper_add(encoder, &ingenic_drm_encoder_helper_funcs);
- ret = drm_bridge_attach(encoder, bridge, NULL, 0);
- if (ret)
+ ib->bridge.funcs = &ingenic_drm_bridge_funcs;
+ ib->next_bridge = bridge;
+
+ ret = drm_bridge_attach(encoder, &ib->bridge, NULL,
+ DRM_BRIDGE_ATTACH_NO_CONNECTOR);
+ if (ret) {
+ dev_err(dev, "Unable to attach bridge\n");
return ret;
+ }
+
+ connector = drm_bridge_connector_init(drm, encoder);
+ if (IS_ERR(connector)) {
+ dev_err(dev, "Unable to init connector\n");
+ return PTR_ERR(connector);
+ }
+
+ drm_connector_attach_encoder(connector, encoder);
}
drm_for_each_encoder(encoder, drm) {
@@ -1112,10 +1267,6 @@ static int ingenic_drm_bind(struct device *dev, bool has_components)
}
}
- /* Set address of our DMA descriptor chain */
- regmap_write(priv->map, JZ_REG_LCD_DA0, dma_hwdesc_phys_f0);
- regmap_write(priv->map, JZ_REG_LCD_DA1, dma_hwdesc_phys_f1);
-
/* Enable OSD if available */
if (soc_info->has_osd)
regmap_write(priv->map, JZ_REG_LCD_OSDC, JZ_LCD_OSDC_OSDEN);
@@ -1130,6 +1281,20 @@ static int ingenic_drm_bind(struct device *dev, bool has_components)
goto err_devclk_disable;
}
+ private_state = kzalloc(sizeof(*private_state), GFP_KERNEL);
+ if (!private_state) {
+ ret = -ENOMEM;
+ goto err_clk_notifier_unregister;
+ }
+
+ drm_atomic_private_obj_init(drm, &priv->private_obj, &private_state->base,
+ &ingenic_drm_private_state_funcs);
+
+ ret = drmm_add_action_or_reset(drm, ingenic_drm_atomic_private_obj_fini,
+ &priv->private_obj);
+ if (ret)
+ goto err_private_state_free;
+
ret = drm_dev_register(drm, 0);
if (ret) {
dev_err(dev, "Failed to register DRM driver\n");
@@ -1140,6 +1305,8 @@ static int ingenic_drm_bind(struct device *dev, bool has_components)
return 0;
+err_private_state_free:
+ kfree(private_state);
err_clk_notifier_unregister:
clk_notifier_unregister(parent_clk, &priv->clock_nb);
err_devclk_disable:
diff --git a/drivers/gpu/drm/ingenic/ingenic-ipu.c b/drivers/gpu/drm/ingenic/ingenic-ipu.c
index aeb8a75..2737fc5 100644
--- a/drivers/gpu/drm/ingenic/ingenic-ipu.c
+++ b/drivers/gpu/drm/ingenic/ingenic-ipu.c
@@ -45,6 +45,12 @@ struct soc_info {
unsigned int weight, unsigned int offset);
};
+struct ingenic_ipu_private_state {
+ struct drm_private_state base;
+
+ unsigned int num_w, num_h, denom_w, denom_h;
+};
+
struct ingenic_ipu {
struct drm_plane plane;
struct drm_device *drm;
@@ -54,12 +60,12 @@ struct ingenic_ipu {
const struct soc_info *soc_info;
bool clk_enabled;
- unsigned int num_w, num_h, denom_w, denom_h;
-
dma_addr_t addr_y, addr_u, addr_v;
struct drm_property *sharpness_prop;
unsigned int sharpness;
+
+ struct drm_private_obj private_obj;
};
/* Signed 15.16 fixed-point math (for bicubic scaling coefficients) */
@@ -73,6 +79,36 @@ static inline struct ingenic_ipu *plane_to_ingenic_ipu(struct drm_plane *plane)
return container_of(plane, struct ingenic_ipu, plane);
}
+static inline struct ingenic_ipu_private_state *
+to_ingenic_ipu_priv_state(struct drm_private_state *state)
+{
+ return container_of(state, struct ingenic_ipu_private_state, base);
+}
+
+static struct ingenic_ipu_private_state *
+ingenic_ipu_get_priv_state(struct ingenic_ipu *priv, struct drm_atomic_state *state)
+{
+ struct drm_private_state *priv_state;
+
+ priv_state = drm_atomic_get_private_obj_state(state, &priv->private_obj);
+ if (IS_ERR(priv_state))
+ return ERR_CAST(priv_state);
+
+ return to_ingenic_ipu_priv_state(priv_state);
+}
+
+static struct ingenic_ipu_private_state *
+ingenic_ipu_get_new_priv_state(struct ingenic_ipu *priv, struct drm_atomic_state *state)
+{
+ struct drm_private_state *priv_state;
+
+ priv_state = drm_atomic_get_new_private_obj_state(state, &priv->private_obj);
+ if (!priv_state)
+ return NULL;
+
+ return to_ingenic_ipu_priv_state(priv_state);
+}
+
/*
* Apply conventional cubic convolution kernel. Both parameters
* and return value are 15.16 signed fixed-point.
@@ -293,11 +329,16 @@ static void ingenic_ipu_plane_atomic_update(struct drm_plane *plane,
const struct drm_format_info *finfo;
u32 ctrl, stride = 0, coef_index = 0, format = 0;
bool needs_modeset, upscaling_w, upscaling_h;
+ struct ingenic_ipu_private_state *ipu_state;
int err;
if (!newstate || !newstate->fb)
return;
+ ipu_state = ingenic_ipu_get_new_priv_state(ipu, state);
+ if (WARN_ON(!ipu_state))
+ return;
+
finfo = drm_format_info(newstate->fb->format->format);
if (!ipu->clk_enabled) {
@@ -470,27 +511,27 @@ static void ingenic_ipu_plane_atomic_update(struct drm_plane *plane,
if (ipu->soc_info->has_bicubic)
ctrl |= JZ_IPU_CTRL_ZOOM_SEL;
- upscaling_w = ipu->num_w > ipu->denom_w;
+ upscaling_w = ipu_state->num_w > ipu_state->denom_w;
if (upscaling_w)
ctrl |= JZ_IPU_CTRL_HSCALE;
- if (ipu->num_w != 1 || ipu->denom_w != 1) {
+ if (ipu_state->num_w != 1 || ipu_state->denom_w != 1) {
if (!ipu->soc_info->has_bicubic && !upscaling_w)
- coef_index |= (ipu->denom_w - 1) << 16;
+ coef_index |= (ipu_state->denom_w - 1) << 16;
else
- coef_index |= (ipu->num_w - 1) << 16;
+ coef_index |= (ipu_state->num_w - 1) << 16;
ctrl |= JZ_IPU_CTRL_HRSZ_EN;
}
- upscaling_h = ipu->num_h > ipu->denom_h;
+ upscaling_h = ipu_state->num_h > ipu_state->denom_h;
if (upscaling_h)
ctrl |= JZ_IPU_CTRL_VSCALE;
- if (ipu->num_h != 1 || ipu->denom_h != 1) {
+ if (ipu_state->num_h != 1 || ipu_state->denom_h != 1) {
if (!ipu->soc_info->has_bicubic && !upscaling_h)
- coef_index |= ipu->denom_h - 1;
+ coef_index |= ipu_state->denom_h - 1;
else
- coef_index |= ipu->num_h - 1;
+ coef_index |= ipu_state->num_h - 1;
ctrl |= JZ_IPU_CTRL_VRSZ_EN;
}
@@ -501,13 +542,13 @@ static void ingenic_ipu_plane_atomic_update(struct drm_plane *plane,
/* Set the LUT index register */
regmap_write(ipu->map, JZ_REG_IPU_RSZ_COEF_INDEX, coef_index);
- if (ipu->num_w != 1 || ipu->denom_w != 1)
+ if (ipu_state->num_w != 1 || ipu_state->denom_w != 1)
ingenic_ipu_set_coefs(ipu, JZ_REG_IPU_HRSZ_COEF_LUT,
- ipu->num_w, ipu->denom_w);
+ ipu_state->num_w, ipu_state->denom_w);
- if (ipu->num_h != 1 || ipu->denom_h != 1)
+ if (ipu_state->num_h != 1 || ipu_state->denom_h != 1)
ingenic_ipu_set_coefs(ipu, JZ_REG_IPU_VRSZ_COEF_LUT,
- ipu->num_h, ipu->denom_h);
+ ipu_state->num_h, ipu_state->denom_h);
/* Clear STATUS register */
regmap_write(ipu->map, JZ_REG_IPU_STATUS, 0);
@@ -519,7 +560,8 @@ static void ingenic_ipu_plane_atomic_update(struct drm_plane *plane,
dev_dbg(ipu->dev, "Scaling %ux%u to %ux%u (%u:%u horiz, %u:%u vert)\n",
newstate->src_w >> 16, newstate->src_h >> 16,
newstate->crtc_w, newstate->crtc_h,
- ipu->num_w, ipu->denom_w, ipu->num_h, ipu->denom_h);
+ ipu_state->num_w, ipu_state->denom_w,
+ ipu_state->num_h, ipu_state->denom_h);
}
static int ingenic_ipu_plane_atomic_check(struct drm_plane *plane,
@@ -533,6 +575,7 @@ static int ingenic_ipu_plane_atomic_check(struct drm_plane *plane,
struct ingenic_ipu *ipu = plane_to_ingenic_ipu(plane);
struct drm_crtc *crtc = new_plane_state->crtc ?: old_plane_state->crtc;
struct drm_crtc_state *crtc_state;
+ struct ingenic_ipu_private_state *ipu_state;
if (!crtc)
return 0;
@@ -541,6 +584,10 @@ static int ingenic_ipu_plane_atomic_check(struct drm_plane *plane,
if (WARN_ON(!crtc_state))
return -EINVAL;
+ ipu_state = ingenic_ipu_get_priv_state(ipu, state);
+ if (IS_ERR(ipu_state))
+ return PTR_ERR(ipu_state);
+
/* Request a full modeset if we are enabling or disabling the IPU. */
if (!old_plane_state->crtc ^ !new_plane_state->crtc)
crtc_state->mode_changed = true;
@@ -593,10 +640,10 @@ static int ingenic_ipu_plane_atomic_check(struct drm_plane *plane,
if (num_h > max_h)
return -EINVAL;
- ipu->num_w = num_w;
- ipu->num_h = num_h;
- ipu->denom_w = denom_w;
- ipu->denom_h = denom_h;
+ ipu_state->num_w = num_w;
+ ipu_state->num_h = num_h;
+ ipu_state->denom_w = denom_w;
+ ipu_state->denom_h = denom_h;
out_check_damage:
if (ingenic_drm_map_noncoherent(ipu->master))
@@ -679,6 +726,33 @@ static const struct drm_plane_funcs ingenic_ipu_plane_funcs = {
.atomic_set_property = ingenic_ipu_plane_atomic_set_property,
};
+static struct drm_private_state *
+ingenic_ipu_duplicate_state(struct drm_private_obj *obj)
+{
+ struct ingenic_ipu_private_state *state = to_ingenic_ipu_priv_state(obj->state);
+
+ state = kmemdup(state, sizeof(*state), GFP_KERNEL);
+ if (!state)
+ return NULL;
+
+ __drm_atomic_helper_private_obj_duplicate_state(obj, &state->base);
+
+ return &state->base;
+}
+
+static void ingenic_ipu_destroy_state(struct drm_private_obj *obj,
+ struct drm_private_state *state)
+{
+ struct ingenic_ipu_private_state *priv_state = to_ingenic_ipu_priv_state(state);
+
+ kfree(priv_state);
+}
+
+static const struct drm_private_state_funcs ingenic_ipu_private_state_funcs = {
+ .atomic_duplicate_state = ingenic_ipu_duplicate_state,
+ .atomic_destroy_state = ingenic_ipu_destroy_state,
+};
+
static irqreturn_t ingenic_ipu_irq_handler(int irq, void *arg)
{
struct ingenic_ipu *ipu = arg;
@@ -717,6 +791,7 @@ static const struct regmap_config ingenic_ipu_regmap_config = {
static int ingenic_ipu_bind(struct device *dev, struct device *master, void *d)
{
struct platform_device *pdev = to_platform_device(dev);
+ struct ingenic_ipu_private_state *private_state;
const struct soc_info *soc_info;
struct drm_device *drm = d;
struct drm_plane *plane;
@@ -810,7 +885,20 @@ static int ingenic_ipu_bind(struct device *dev, struct device *master, void *d)
return err;
}
+ private_state = kzalloc(sizeof(*private_state), GFP_KERNEL);
+ if (!private_state) {
+ err = -ENOMEM;
+ goto err_clk_unprepare;
+ }
+
+ drm_atomic_private_obj_init(drm, &ipu->private_obj, &private_state->base,
+ &ingenic_ipu_private_state_funcs);
+
return 0;
+
+err_clk_unprepare:
+ clk_unprepare(ipu->clk);
+ return err;
}
static void ingenic_ipu_unbind(struct device *dev,
@@ -818,6 +906,7 @@ static void ingenic_ipu_unbind(struct device *dev,
{
struct ingenic_ipu *ipu = dev_get_drvdata(dev);
+ drm_atomic_private_obj_fini(&ipu->private_obj);
clk_unprepare(ipu->clk);
}
diff --git a/drivers/gpu/drm/kmb/Kconfig b/drivers/gpu/drm/kmb/Kconfig
index bc4cb5e..5fdd43d 100644
--- a/drivers/gpu/drm/kmb/Kconfig
+++ b/drivers/gpu/drm/kmb/Kconfig
@@ -3,7 +3,6 @@
depends on DRM
depends on ARCH_KEEMBAY || COMPILE_TEST
select DRM_KMS_HELPER
- select DRM_KMS_CMA_HELPER
select DRM_GEM_CMA_HELPER
select DRM_MIPI_DSI
help
diff --git a/drivers/gpu/drm/kmb/kmb_drv.c b/drivers/gpu/drm/kmb/kmb_drv.c
index 961ac6f..ed24243 100644
--- a/drivers/gpu/drm/kmb/kmb_drv.c
+++ b/drivers/gpu/drm/kmb/kmb_drv.c
@@ -15,6 +15,7 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_drv.h>
+#include <drm/drm_fb_helper.h>
#include <drm/drm_gem_cma_helper.h>
#include <drm/drm_gem_framebuffer_helper.h>
#include <drm/drm_probe_helper.h>
@@ -176,6 +177,7 @@ static int kmb_setup_mode_config(struct drm_device *drm)
drm->mode_config.min_height = KMB_FB_MIN_HEIGHT;
drm->mode_config.max_width = KMB_FB_MAX_WIDTH;
drm->mode_config.max_height = KMB_FB_MAX_HEIGHT;
+ drm->mode_config.preferred_depth = 24;
drm->mode_config.funcs = &kmb_mode_config_funcs;
ret = kmb_setup_crtc(drm);
@@ -559,6 +561,8 @@ static int kmb_probe(struct platform_device *pdev)
if (ret)
goto err_register;
+ drm_fbdev_generic_setup(&kmb->drm, 0);
+
return 0;
err_register:
diff --git a/drivers/gpu/drm/lima/lima_device.c b/drivers/gpu/drm/lima/lima_device.c
index f74f804..02cef0c 100644
--- a/drivers/gpu/drm/lima/lima_device.c
+++ b/drivers/gpu/drm/lima/lima_device.c
@@ -358,6 +358,7 @@ int lima_device_init(struct lima_device *ldev)
int err, i;
dma_set_coherent_mask(ldev->dev, DMA_BIT_MASK(32));
+ dma_set_max_seg_size(ldev->dev, UINT_MAX);
err = lima_clk_init(ldev);
if (err)
diff --git a/drivers/gpu/drm/lima/lima_gem.c b/drivers/gpu/drm/lima/lima_gem.c
index 640acc0..f9a9198 100644
--- a/drivers/gpu/drm/lima/lima_gem.c
+++ b/drivers/gpu/drm/lima/lima_gem.c
@@ -127,7 +127,7 @@ int lima_gem_create_handle(struct drm_device *dev, struct drm_file *file,
if (err)
goto out;
} else {
- struct sg_table *sgt = drm_gem_shmem_get_pages_sgt(obj);
+ struct sg_table *sgt = drm_gem_shmem_get_pages_sgt(shmem);
if (IS_ERR(sgt)) {
err = PTR_ERR(sgt);
@@ -151,7 +151,7 @@ static void lima_gem_free_object(struct drm_gem_object *obj)
if (!list_empty(&bo->va))
dev_err(obj->dev->dev, "lima gem free bo still has va\n");
- drm_gem_shmem_free_object(obj);
+ drm_gem_shmem_free(&bo->base);
}
static int lima_gem_object_open(struct drm_gem_object *obj, struct drm_file *file)
@@ -179,7 +179,7 @@ static int lima_gem_pin(struct drm_gem_object *obj)
if (bo->heap_size)
return -EINVAL;
- return drm_gem_shmem_pin(obj);
+ return drm_gem_shmem_pin(&bo->base);
}
static int lima_gem_vmap(struct drm_gem_object *obj, struct dma_buf_map *map)
@@ -189,7 +189,7 @@ static int lima_gem_vmap(struct drm_gem_object *obj, struct dma_buf_map *map)
if (bo->heap_size)
return -EINVAL;
- return drm_gem_shmem_vmap(obj, map);
+ return drm_gem_shmem_vmap(&bo->base, map);
}
static int lima_gem_mmap(struct drm_gem_object *obj, struct vm_area_struct *vma)
@@ -199,19 +199,19 @@ static int lima_gem_mmap(struct drm_gem_object *obj, struct vm_area_struct *vma)
if (bo->heap_size)
return -EINVAL;
- return drm_gem_shmem_mmap(obj, vma);
+ return drm_gem_shmem_mmap(&bo->base, vma);
}
static const struct drm_gem_object_funcs lima_gem_funcs = {
.free = lima_gem_free_object,
.open = lima_gem_object_open,
.close = lima_gem_object_close,
- .print_info = drm_gem_shmem_print_info,
+ .print_info = drm_gem_shmem_object_print_info,
.pin = lima_gem_pin,
- .unpin = drm_gem_shmem_unpin,
- .get_sg_table = drm_gem_shmem_get_sg_table,
+ .unpin = drm_gem_shmem_object_unpin,
+ .get_sg_table = drm_gem_shmem_object_get_sg_table,
.vmap = lima_gem_vmap,
- .vunmap = drm_gem_shmem_vunmap,
+ .vunmap = drm_gem_shmem_object_vunmap,
.mmap = lima_gem_mmap,
};
@@ -221,7 +221,7 @@ struct drm_gem_object *lima_gem_create_object(struct drm_device *dev, size_t siz
bo = kzalloc(sizeof(*bo), GFP_KERNEL);
if (!bo)
- return NULL;
+ return ERR_PTR(-ENOMEM);
mutex_init(&bo->lock);
INIT_LIST_HEAD(&bo->va);
diff --git a/drivers/gpu/drm/lima/lima_sched.c b/drivers/gpu/drm/lima/lima_sched.c
index 99d5f6f..5612d73 100644
--- a/drivers/gpu/drm/lima/lima_sched.c
+++ b/drivers/gpu/drm/lima/lima_sched.c
@@ -371,7 +371,7 @@ static void lima_sched_build_error_task_list(struct lima_sched_task *task)
} else {
buffer_chunk->size = lima_bo_size(bo);
- ret = drm_gem_shmem_vmap(&bo->base.base, &map);
+ ret = drm_gem_shmem_vmap(&bo->base, &map);
if (ret) {
kvfree(et);
goto out;
@@ -379,7 +379,7 @@ static void lima_sched_build_error_task_list(struct lima_sched_task *task)
memcpy(buffer_chunk + 1, map.vaddr, buffer_chunk->size);
- drm_gem_shmem_vunmap(&bo->base.base, &map);
+ drm_gem_shmem_vunmap(&bo->base, &map);
}
buffer_chunk = (void *)(buffer_chunk + 1) + buffer_chunk->size;
diff --git a/drivers/gpu/drm/mcde/Kconfig b/drivers/gpu/drm/mcde/Kconfig
index 71c689b..d0bf1bc 100644
--- a/drivers/gpu/drm/mcde/Kconfig
+++ b/drivers/gpu/drm/mcde/Kconfig
@@ -10,7 +10,6 @@
select DRM_BRIDGE
select DRM_PANEL_BRIDGE
select DRM_KMS_HELPER
- select DRM_KMS_CMA_HELPER
select DRM_GEM_CMA_HELPER
select VT_HW_CONSOLE_BINDING if FRAMEBUFFER_CONSOLE
help
diff --git a/drivers/gpu/drm/mediatek/mtk_disp_ccorr.c b/drivers/gpu/drm/mediatek/mtk_disp_ccorr.c
index 141cb36..3a53ebc 100644
--- a/drivers/gpu/drm/mediatek/mtk_disp_ccorr.c
+++ b/drivers/gpu/drm/mediatek/mtk_disp_ccorr.c
@@ -205,9 +205,15 @@ static const struct mtk_disp_ccorr_data mt8183_ccorr_driver_data = {
.matrix_bits = 10,
};
+static const struct mtk_disp_ccorr_data mt8192_ccorr_driver_data = {
+ .matrix_bits = 11,
+};
+
static const struct of_device_id mtk_disp_ccorr_driver_dt_match[] = {
{ .compatible = "mediatek,mt8183-disp-ccorr",
.data = &mt8183_ccorr_driver_data},
+ { .compatible = "mediatek,mt8192-disp-ccorr",
+ .data = &mt8192_ccorr_driver_data},
{},
};
MODULE_DEVICE_TABLE(of, mtk_disp_ccorr_driver_dt_match);
diff --git a/drivers/gpu/drm/mediatek/mtk_disp_ovl.c b/drivers/gpu/drm/mediatek/mtk_disp_ovl.c
index 5326989..2146299 100644
--- a/drivers/gpu/drm/mediatek/mtk_disp_ovl.c
+++ b/drivers/gpu/drm/mediatek/mtk_disp_ovl.c
@@ -456,6 +456,22 @@ static const struct mtk_disp_ovl_data mt8183_ovl_2l_driver_data = {
.fmt_rgb565_is_0 = true,
};
+static const struct mtk_disp_ovl_data mt8192_ovl_driver_data = {
+ .addr = DISP_REG_OVL_ADDR_MT8173,
+ .gmc_bits = 10,
+ .layer_nr = 4,
+ .fmt_rgb565_is_0 = true,
+ .smi_id_en = true,
+};
+
+static const struct mtk_disp_ovl_data mt8192_ovl_2l_driver_data = {
+ .addr = DISP_REG_OVL_ADDR_MT8173,
+ .gmc_bits = 10,
+ .layer_nr = 2,
+ .fmt_rgb565_is_0 = true,
+ .smi_id_en = true,
+};
+
static const struct of_device_id mtk_disp_ovl_driver_dt_match[] = {
{ .compatible = "mediatek,mt2701-disp-ovl",
.data = &mt2701_ovl_driver_data},
@@ -465,6 +481,10 @@ static const struct of_device_id mtk_disp_ovl_driver_dt_match[] = {
.data = &mt8183_ovl_driver_data},
{ .compatible = "mediatek,mt8183-disp-ovl-2l",
.data = &mt8183_ovl_2l_driver_data},
+ { .compatible = "mediatek,mt8192-disp-ovl",
+ .data = &mt8192_ovl_driver_data},
+ { .compatible = "mediatek,mt8192-disp-ovl-2l",
+ .data = &mt8192_ovl_2l_driver_data},
{},
};
MODULE_DEVICE_TABLE(of, mtk_disp_ovl_driver_dt_match);
diff --git a/drivers/gpu/drm/mediatek/mtk_disp_rdma.c b/drivers/gpu/drm/mediatek/mtk_disp_rdma.c
index 75d7f45..d41a397 100644
--- a/drivers/gpu/drm/mediatek/mtk_disp_rdma.c
+++ b/drivers/gpu/drm/mediatek/mtk_disp_rdma.c
@@ -353,6 +353,10 @@ static const struct mtk_disp_rdma_data mt8183_rdma_driver_data = {
.fifo_size = 5 * SZ_1K,
};
+static const struct mtk_disp_rdma_data mt8192_rdma_driver_data = {
+ .fifo_size = 5 * SZ_1K,
+};
+
static const struct of_device_id mtk_disp_rdma_driver_dt_match[] = {
{ .compatible = "mediatek,mt2701-disp-rdma",
.data = &mt2701_rdma_driver_data},
@@ -360,6 +364,8 @@ static const struct of_device_id mtk_disp_rdma_driver_dt_match[] = {
.data = &mt8173_rdma_driver_data},
{ .compatible = "mediatek,mt8183-disp-rdma",
.data = &mt8183_rdma_driver_data},
+ { .compatible = "mediatek,mt8192-disp-rdma",
+ .data = &mt8192_rdma_driver_data},
{},
};
MODULE_DEVICE_TABLE(of, mtk_disp_rdma_driver_dt_match);
diff --git a/drivers/gpu/drm/mediatek/mtk_drm_crtc.c b/drivers/gpu/drm/mediatek/mtk_drm_crtc.c
index a4e80e4..d661edf 100644
--- a/drivers/gpu/drm/mediatek/mtk_drm_crtc.c
+++ b/drivers/gpu/drm/mediatek/mtk_drm_crtc.c
@@ -4,6 +4,8 @@
*/
#include <linux/clk.h>
+#include <linux/dma-mapping.h>
+#include <linux/mailbox_controller.h>
#include <linux/pm_runtime.h>
#include <linux/soc/mediatek/mtk-cmdq.h>
#include <linux/soc/mediatek/mtk-mmsys.h>
@@ -50,8 +52,10 @@ struct mtk_drm_crtc {
bool pending_async_planes;
#if IS_REACHABLE(CONFIG_MTK_CMDQ)
- struct cmdq_client *cmdq_client;
+ struct cmdq_client cmdq_client;
+ struct cmdq_pkt cmdq_handle;
u32 cmdq_event;
+ u32 cmdq_vblank_cnt;
#endif
struct device *mmsys_dev;
@@ -104,12 +108,60 @@ static void mtk_drm_finish_page_flip(struct mtk_drm_crtc *mtk_crtc)
}
}
+#if IS_REACHABLE(CONFIG_MTK_CMDQ)
+static int mtk_drm_cmdq_pkt_create(struct cmdq_client *client, struct cmdq_pkt *pkt,
+ size_t size)
+{
+ struct device *dev;
+ dma_addr_t dma_addr;
+
+ pkt->va_base = kzalloc(size, GFP_KERNEL);
+ if (!pkt->va_base) {
+ kfree(pkt);
+ return -ENOMEM;
+ }
+ pkt->buf_size = size;
+ pkt->cl = (void *)client;
+
+ dev = client->chan->mbox->dev;
+ dma_addr = dma_map_single(dev, pkt->va_base, pkt->buf_size,
+ DMA_TO_DEVICE);
+ if (dma_mapping_error(dev, dma_addr)) {
+ dev_err(dev, "dma map failed, size=%u\n", (u32)(u64)size);
+ kfree(pkt->va_base);
+ kfree(pkt);
+ return -ENOMEM;
+ }
+
+ pkt->pa_base = dma_addr;
+
+ return 0;
+}
+
+static void mtk_drm_cmdq_pkt_destroy(struct cmdq_pkt *pkt)
+{
+ struct cmdq_client *client = (struct cmdq_client *)pkt->cl;
+
+ dma_unmap_single(client->chan->mbox->dev, pkt->pa_base, pkt->buf_size,
+ DMA_TO_DEVICE);
+ kfree(pkt->va_base);
+ kfree(pkt);
+}
+#endif
+
static void mtk_drm_crtc_destroy(struct drm_crtc *crtc)
{
struct mtk_drm_crtc *mtk_crtc = to_mtk_crtc(crtc);
mtk_mutex_put(mtk_crtc->mutex);
+#if IS_REACHABLE(CONFIG_MTK_CMDQ)
+ mtk_drm_cmdq_pkt_destroy(&mtk_crtc->cmdq_handle);
+ if (mtk_crtc->cmdq_client.chan) {
+ mbox_free_channel(mtk_crtc->cmdq_client.chan);
+ mtk_crtc->cmdq_client.chan = NULL;
+ }
+#endif
drm_crtc_cleanup(crtc);
}
@@ -222,9 +274,46 @@ struct mtk_ddp_comp *mtk_drm_ddp_comp_for_plane(struct drm_crtc *crtc,
}
#if IS_REACHABLE(CONFIG_MTK_CMDQ)
-static void ddp_cmdq_cb(struct cmdq_cb_data data)
+static void ddp_cmdq_cb(struct mbox_client *cl, void *mssg)
{
- cmdq_pkt_destroy(data.data);
+ struct cmdq_cb_data *data = mssg;
+ struct cmdq_client *cmdq_cl = container_of(cl, struct cmdq_client, client);
+ struct mtk_drm_crtc *mtk_crtc = container_of(cmdq_cl, struct mtk_drm_crtc, cmdq_client);
+ struct mtk_crtc_state *state;
+ unsigned int i;
+
+ if (data->sta < 0)
+ return;
+
+ state = to_mtk_crtc_state(mtk_crtc->base.state);
+
+ state->pending_config = false;
+
+ if (mtk_crtc->pending_planes) {
+ for (i = 0; i < mtk_crtc->layer_nr; i++) {
+ struct drm_plane *plane = &mtk_crtc->planes[i];
+ struct mtk_plane_state *plane_state;
+
+ plane_state = to_mtk_plane_state(plane->state);
+
+ plane_state->pending.config = false;
+ }
+ mtk_crtc->pending_planes = false;
+ }
+
+ if (mtk_crtc->pending_async_planes) {
+ for (i = 0; i < mtk_crtc->layer_nr; i++) {
+ struct drm_plane *plane = &mtk_crtc->planes[i];
+ struct mtk_plane_state *plane_state;
+
+ plane_state = to_mtk_plane_state(plane->state);
+
+ plane_state->pending.async_config = false;
+ }
+ mtk_crtc->pending_async_planes = false;
+ }
+
+ mtk_crtc->cmdq_vblank_cnt = 0;
}
#endif
@@ -378,7 +467,8 @@ static void mtk_crtc_ddp_config(struct drm_crtc *crtc,
state->pending_vrefresh, 0,
cmdq_handle);
- state->pending_config = false;
+ if (!cmdq_handle)
+ state->pending_config = false;
}
if (mtk_crtc->pending_planes) {
@@ -398,9 +488,12 @@ static void mtk_crtc_ddp_config(struct drm_crtc *crtc,
mtk_ddp_comp_layer_config(comp, local_layer,
plane_state,
cmdq_handle);
- plane_state->pending.config = false;
+ if (!cmdq_handle)
+ plane_state->pending.config = false;
}
- mtk_crtc->pending_planes = false;
+
+ if (!cmdq_handle)
+ mtk_crtc->pending_planes = false;
}
if (mtk_crtc->pending_async_planes) {
@@ -420,9 +513,12 @@ static void mtk_crtc_ddp_config(struct drm_crtc *crtc,
mtk_ddp_comp_layer_config(comp, local_layer,
plane_state,
cmdq_handle);
- plane_state->pending.async_config = false;
+ if (!cmdq_handle)
+ plane_state->pending.async_config = false;
}
- mtk_crtc->pending_async_planes = false;
+
+ if (!cmdq_handle)
+ mtk_crtc->pending_async_planes = false;
}
}
@@ -430,7 +526,7 @@ static void mtk_drm_crtc_update_config(struct mtk_drm_crtc *mtk_crtc,
bool needs_vblank)
{
#if IS_REACHABLE(CONFIG_MTK_CMDQ)
- struct cmdq_pkt *cmdq_handle;
+ struct cmdq_pkt *cmdq_handle = &mtk_crtc->cmdq_handle;
#endif
struct drm_crtc *crtc = &mtk_crtc->base;
struct mtk_drm_private *priv = crtc->dev->dev_private;
@@ -468,14 +564,28 @@ static void mtk_drm_crtc_update_config(struct mtk_drm_crtc *mtk_crtc,
mtk_mutex_release(mtk_crtc->mutex);
}
#if IS_REACHABLE(CONFIG_MTK_CMDQ)
- if (mtk_crtc->cmdq_client) {
- mbox_flush(mtk_crtc->cmdq_client->chan, 2000);
- cmdq_handle = cmdq_pkt_create(mtk_crtc->cmdq_client, PAGE_SIZE);
+ if (mtk_crtc->cmdq_client.chan) {
+ mbox_flush(mtk_crtc->cmdq_client.chan, 2000);
+ cmdq_handle->cmd_buf_size = 0;
cmdq_pkt_clear_event(cmdq_handle, mtk_crtc->cmdq_event);
cmdq_pkt_wfe(cmdq_handle, mtk_crtc->cmdq_event, false);
mtk_crtc_ddp_config(crtc, cmdq_handle);
cmdq_pkt_finalize(cmdq_handle);
- cmdq_pkt_flush_async(cmdq_handle, ddp_cmdq_cb, cmdq_handle);
+ dma_sync_single_for_device(mtk_crtc->cmdq_client.chan->mbox->dev,
+ cmdq_handle->pa_base,
+ cmdq_handle->cmd_buf_size,
+ DMA_TO_DEVICE);
+ /*
+ * CMDQ command should execute in next 3 vblank.
+ * One vblank interrupt before send message (occasionally)
+ * and one vblank interrupt after cmdq done,
+ * so it's timeout after 3 vblank interrupt.
+ * If it fail to execute in next 3 vblank, timeout happen.
+ */
+ mtk_crtc->cmdq_vblank_cnt = 3;
+
+ mbox_send_message(mtk_crtc->cmdq_client.chan, cmdq_handle);
+ mbox_client_txdone(mtk_crtc->cmdq_client.chan, 0);
}
#endif
mtk_crtc->config_updating = false;
@@ -489,12 +599,15 @@ static void mtk_crtc_ddp_irq(void *data)
struct mtk_drm_private *priv = crtc->dev->dev_private;
#if IS_REACHABLE(CONFIG_MTK_CMDQ)
- if (!priv->data->shadow_register && !mtk_crtc->cmdq_client)
+ if (!priv->data->shadow_register && !mtk_crtc->cmdq_client.chan)
+ mtk_crtc_ddp_config(crtc, NULL);
+ else if (mtk_crtc->cmdq_vblank_cnt > 0 && --mtk_crtc->cmdq_vblank_cnt == 0)
+ DRM_ERROR("mtk_crtc %d CMDQ execute command timeout!\n",
+ drm_crtc_index(&mtk_crtc->base));
#else
if (!priv->data->shadow_register)
-#endif
mtk_crtc_ddp_config(crtc, NULL);
-
+#endif
mtk_drm_finish_page_flip(mtk_crtc);
}
@@ -829,16 +942,20 @@ int mtk_drm_crtc_create(struct drm_device *drm_dev,
mutex_init(&mtk_crtc->hw_lock);
#if IS_REACHABLE(CONFIG_MTK_CMDQ)
- mtk_crtc->cmdq_client =
- cmdq_mbox_create(mtk_crtc->mmsys_dev,
- drm_crtc_index(&mtk_crtc->base));
- if (IS_ERR(mtk_crtc->cmdq_client)) {
+ mtk_crtc->cmdq_client.client.dev = mtk_crtc->mmsys_dev;
+ mtk_crtc->cmdq_client.client.tx_block = false;
+ mtk_crtc->cmdq_client.client.knows_txdone = true;
+ mtk_crtc->cmdq_client.client.rx_callback = ddp_cmdq_cb;
+ mtk_crtc->cmdq_client.chan =
+ mbox_request_channel(&mtk_crtc->cmdq_client.client,
+ drm_crtc_index(&mtk_crtc->base));
+ if (IS_ERR(mtk_crtc->cmdq_client.chan)) {
dev_dbg(dev, "mtk_crtc %d failed to create mailbox client, writing register by CPU now\n",
drm_crtc_index(&mtk_crtc->base));
- mtk_crtc->cmdq_client = NULL;
+ mtk_crtc->cmdq_client.chan = NULL;
}
- if (mtk_crtc->cmdq_client) {
+ if (mtk_crtc->cmdq_client.chan) {
ret = of_property_read_u32_index(priv->mutex_node,
"mediatek,gce-events",
drm_crtc_index(&mtk_crtc->base),
@@ -846,8 +963,18 @@ int mtk_drm_crtc_create(struct drm_device *drm_dev,
if (ret) {
dev_dbg(dev, "mtk_crtc %d failed to get mediatek,gce-events property\n",
drm_crtc_index(&mtk_crtc->base));
- cmdq_mbox_destroy(mtk_crtc->cmdq_client);
- mtk_crtc->cmdq_client = NULL;
+ mbox_free_channel(mtk_crtc->cmdq_client.chan);
+ mtk_crtc->cmdq_client.chan = NULL;
+ } else {
+ ret = mtk_drm_cmdq_pkt_create(&mtk_crtc->cmdq_client,
+ &mtk_crtc->cmdq_handle,
+ PAGE_SIZE);
+ if (ret) {
+ dev_dbg(dev, "mtk_crtc %d failed to create cmdq packet\n",
+ drm_crtc_index(&mtk_crtc->base));
+ mbox_free_channel(mtk_crtc->cmdq_client.chan);
+ mtk_crtc->cmdq_client.chan = NULL;
+ }
}
}
#endif
diff --git a/drivers/gpu/drm/mediatek/mtk_drm_ddp_comp.c b/drivers/gpu/drm/mediatek/mtk_drm_ddp_comp.c
index 99cbf44..b4b682b 100644
--- a/drivers/gpu/drm/mediatek/mtk_drm_ddp_comp.c
+++ b/drivers/gpu/drm/mediatek/mtk_drm_ddp_comp.c
@@ -20,45 +20,39 @@
#include "mtk_drm_ddp_comp.h"
#include "mtk_drm_crtc.h"
-#define DISP_OD_EN 0x0000
-#define DISP_OD_INTEN 0x0008
-#define DISP_OD_INTSTA 0x000c
-#define DISP_OD_CFG 0x0020
-#define DISP_OD_SIZE 0x0030
-#define DISP_DITHER_5 0x0114
-#define DISP_DITHER_7 0x011c
-#define DISP_DITHER_15 0x013c
-#define DISP_DITHER_16 0x0140
-#define DISP_REG_UFO_START 0x0000
-
-#define DISP_DITHER_EN 0x0000
+#define DISP_REG_DITHER_EN 0x0000
#define DITHER_EN BIT(0)
-#define DISP_DITHER_CFG 0x0020
+#define DISP_REG_DITHER_CFG 0x0020
#define DITHER_RELAY_MODE BIT(0)
#define DITHER_ENGINE_EN BIT(1)
-#define DISP_DITHER_SIZE 0x0030
-
-#define LUT_10BIT_MASK 0x03ff
-
-#define OD_RELAYMODE BIT(0)
-
-#define UFO_BYPASS BIT(2)
-
#define DISP_DITHERING BIT(2)
+#define DISP_REG_DITHER_SIZE 0x0030
+#define DISP_REG_DITHER_5 0x0114
+#define DISP_REG_DITHER_7 0x011c
+#define DISP_REG_DITHER_15 0x013c
#define DITHER_LSB_ERR_SHIFT_R(x) (((x) & 0x7) << 28)
-#define DITHER_OVFLW_BIT_R(x) (((x) & 0x7) << 24)
#define DITHER_ADD_LSHIFT_R(x) (((x) & 0x7) << 20)
-#define DITHER_ADD_RSHIFT_R(x) (((x) & 0x7) << 16)
#define DITHER_NEW_BIT_MODE BIT(0)
+#define DISP_REG_DITHER_16 0x0140
#define DITHER_LSB_ERR_SHIFT_B(x) (((x) & 0x7) << 28)
-#define DITHER_OVFLW_BIT_B(x) (((x) & 0x7) << 24)
#define DITHER_ADD_LSHIFT_B(x) (((x) & 0x7) << 20)
-#define DITHER_ADD_RSHIFT_B(x) (((x) & 0x7) << 16)
#define DITHER_LSB_ERR_SHIFT_G(x) (((x) & 0x7) << 12)
-#define DITHER_OVFLW_BIT_G(x) (((x) & 0x7) << 8)
#define DITHER_ADD_LSHIFT_G(x) (((x) & 0x7) << 4)
-#define DITHER_ADD_RSHIFT_G(x) (((x) & 0x7) << 0)
+
+#define DISP_REG_OD_EN 0x0000
+#define DISP_REG_OD_CFG 0x0020
+#define OD_RELAYMODE BIT(0)
+#define DISP_REG_OD_SIZE 0x0030
+
+#define DISP_REG_POSTMASK_EN 0x0000
+#define POSTMASK_EN BIT(0)
+#define DISP_REG_POSTMASK_CFG 0x0020
+#define POSTMASK_RELAY_MODE BIT(0)
+#define DISP_REG_POSTMASK_SIZE 0x0030
+
+#define DISP_REG_UFO_START 0x0000
+#define UFO_BYPASS BIT(2)
struct mtk_ddp_comp_dev {
struct clk *clk;
@@ -134,25 +128,52 @@ void mtk_dither_set_common(void __iomem *regs, struct cmdq_client_reg *cmdq_reg,
return;
if (bpc >= MTK_MIN_BPC) {
- mtk_ddp_write(cmdq_pkt, 0, cmdq_reg, regs, DISP_DITHER_5);
- mtk_ddp_write(cmdq_pkt, 0, cmdq_reg, regs, DISP_DITHER_7);
+ mtk_ddp_write(cmdq_pkt, 0, cmdq_reg, regs, DISP_REG_DITHER_5);
+ mtk_ddp_write(cmdq_pkt, 0, cmdq_reg, regs, DISP_REG_DITHER_7);
mtk_ddp_write(cmdq_pkt,
DITHER_LSB_ERR_SHIFT_R(MTK_MAX_BPC - bpc) |
DITHER_ADD_LSHIFT_R(MTK_MAX_BPC - bpc) |
DITHER_NEW_BIT_MODE,
- cmdq_reg, regs, DISP_DITHER_15);
+ cmdq_reg, regs, DISP_REG_DITHER_15);
mtk_ddp_write(cmdq_pkt,
DITHER_LSB_ERR_SHIFT_B(MTK_MAX_BPC - bpc) |
DITHER_ADD_LSHIFT_B(MTK_MAX_BPC - bpc) |
DITHER_LSB_ERR_SHIFT_G(MTK_MAX_BPC - bpc) |
DITHER_ADD_LSHIFT_G(MTK_MAX_BPC - bpc),
- cmdq_reg, regs, DISP_DITHER_16);
+ cmdq_reg, regs, DISP_REG_DITHER_16);
mtk_ddp_write(cmdq_pkt, dither_en, cmdq_reg, regs, cfg);
}
}
+static void mtk_dither_config(struct device *dev, unsigned int w,
+ unsigned int h, unsigned int vrefresh,
+ unsigned int bpc, struct cmdq_pkt *cmdq_pkt)
+{
+ struct mtk_ddp_comp_dev *priv = dev_get_drvdata(dev);
+
+ mtk_ddp_write(cmdq_pkt, h << 16 | w, &priv->cmdq_reg, priv->regs, DISP_REG_DITHER_SIZE);
+ mtk_ddp_write(cmdq_pkt, DITHER_RELAY_MODE, &priv->cmdq_reg, priv->regs,
+ DISP_REG_DITHER_CFG);
+ mtk_dither_set_common(priv->regs, &priv->cmdq_reg, bpc, DISP_REG_DITHER_CFG,
+ DITHER_ENGINE_EN, cmdq_pkt);
+}
+
+static void mtk_dither_start(struct device *dev)
+{
+ struct mtk_ddp_comp_dev *priv = dev_get_drvdata(dev);
+
+ writel(DITHER_EN, priv->regs + DISP_REG_DITHER_EN);
+}
+
+static void mtk_dither_stop(struct device *dev)
+{
+ struct mtk_ddp_comp_dev *priv = dev_get_drvdata(dev);
+
+ writel_relaxed(0x0, priv->regs + DISP_REG_DITHER_EN);
+}
+
static void mtk_dither_set(struct device *dev, unsigned int bpc,
- unsigned int cfg, struct cmdq_pkt *cmdq_pkt)
+ unsigned int cfg, struct cmdq_pkt *cmdq_pkt)
{
struct mtk_ddp_comp_dev *priv = dev_get_drvdata(dev);
@@ -166,16 +187,42 @@ static void mtk_od_config(struct device *dev, unsigned int w,
{
struct mtk_ddp_comp_dev *priv = dev_get_drvdata(dev);
- mtk_ddp_write(cmdq_pkt, w << 16 | h, &priv->cmdq_reg, priv->regs, DISP_OD_SIZE);
- mtk_ddp_write(cmdq_pkt, OD_RELAYMODE, &priv->cmdq_reg, priv->regs, DISP_OD_CFG);
- mtk_dither_set(dev, bpc, DISP_OD_CFG, cmdq_pkt);
+ mtk_ddp_write(cmdq_pkt, w << 16 | h, &priv->cmdq_reg, priv->regs, DISP_REG_OD_SIZE);
+ mtk_ddp_write(cmdq_pkt, OD_RELAYMODE, &priv->cmdq_reg, priv->regs, DISP_REG_OD_CFG);
+ mtk_dither_set(dev, bpc, DISP_REG_OD_CFG, cmdq_pkt);
}
static void mtk_od_start(struct device *dev)
{
struct mtk_ddp_comp_dev *priv = dev_get_drvdata(dev);
- writel(1, priv->regs + DISP_OD_EN);
+ writel(1, priv->regs + DISP_REG_OD_EN);
+}
+
+static void mtk_postmask_config(struct device *dev, unsigned int w,
+ unsigned int h, unsigned int vrefresh,
+ unsigned int bpc, struct cmdq_pkt *cmdq_pkt)
+{
+ struct mtk_ddp_comp_dev *priv = dev_get_drvdata(dev);
+
+ mtk_ddp_write(cmdq_pkt, w << 16 | h, &priv->cmdq_reg, priv->regs,
+ DISP_REG_POSTMASK_SIZE);
+ mtk_ddp_write(cmdq_pkt, POSTMASK_RELAY_MODE, &priv->cmdq_reg,
+ priv->regs, DISP_REG_POSTMASK_CFG);
+}
+
+static void mtk_postmask_start(struct device *dev)
+{
+ struct mtk_ddp_comp_dev *priv = dev_get_drvdata(dev);
+
+ writel(POSTMASK_EN, priv->regs + DISP_REG_POSTMASK_EN);
+}
+
+static void mtk_postmask_stop(struct device *dev)
+{
+ struct mtk_ddp_comp_dev *priv = dev_get_drvdata(dev);
+
+ writel_relaxed(0x0, priv->regs + DISP_REG_POSTMASK_EN);
}
static void mtk_ufoe_start(struct device *dev)
@@ -185,32 +232,6 @@ static void mtk_ufoe_start(struct device *dev)
writel(UFO_BYPASS, priv->regs + DISP_REG_UFO_START);
}
-static void mtk_dither_config(struct device *dev, unsigned int w,
- unsigned int h, unsigned int vrefresh,
- unsigned int bpc, struct cmdq_pkt *cmdq_pkt)
-{
- struct mtk_ddp_comp_dev *priv = dev_get_drvdata(dev);
-
- mtk_ddp_write(cmdq_pkt, h << 16 | w, &priv->cmdq_reg, priv->regs, DISP_DITHER_SIZE);
- mtk_ddp_write(cmdq_pkt, DITHER_RELAY_MODE, &priv->cmdq_reg, priv->regs, DISP_DITHER_CFG);
- mtk_dither_set_common(priv->regs, &priv->cmdq_reg, bpc, DISP_DITHER_CFG,
- DITHER_ENGINE_EN, cmdq_pkt);
-}
-
-static void mtk_dither_start(struct device *dev)
-{
- struct mtk_ddp_comp_dev *priv = dev_get_drvdata(dev);
-
- writel(DITHER_EN, priv->regs + DISP_DITHER_EN);
-}
-
-static void mtk_dither_stop(struct device *dev)
-{
- struct mtk_ddp_comp_dev *priv = dev_get_drvdata(dev);
-
- writel_relaxed(0x0, priv->regs + DISP_DITHER_EN);
-}
-
static const struct mtk_ddp_comp_funcs ddp_aal = {
.clk_enable = mtk_aal_clk_enable,
.clk_disable = mtk_aal_clk_disable,
@@ -286,6 +307,14 @@ static const struct mtk_ddp_comp_funcs ddp_ovl = {
.bgclr_in_off = mtk_ovl_bgclr_in_off,
};
+static const struct mtk_ddp_comp_funcs ddp_postmask = {
+ .clk_enable = mtk_ddp_clk_enable,
+ .clk_disable = mtk_ddp_clk_disable,
+ .config = mtk_postmask_config,
+ .start = mtk_postmask_start,
+ .stop = mtk_postmask_stop,
+};
+
static const struct mtk_ddp_comp_funcs ddp_rdma = {
.clk_enable = mtk_rdma_clk_enable,
.clk_disable = mtk_rdma_clk_disable,
@@ -305,22 +334,23 @@ static const struct mtk_ddp_comp_funcs ddp_ufoe = {
};
static const char * const mtk_ddp_comp_stem[MTK_DDP_COMP_TYPE_MAX] = {
- [MTK_DISP_OVL] = "ovl",
- [MTK_DISP_OVL_2L] = "ovl-2l",
- [MTK_DISP_RDMA] = "rdma",
- [MTK_DISP_WDMA] = "wdma",
- [MTK_DISP_COLOR] = "color",
- [MTK_DISP_CCORR] = "ccorr",
[MTK_DISP_AAL] = "aal",
- [MTK_DISP_GAMMA] = "gamma",
+ [MTK_DISP_BLS] = "bls",
+ [MTK_DISP_CCORR] = "ccorr",
+ [MTK_DISP_COLOR] = "color",
[MTK_DISP_DITHER] = "dither",
- [MTK_DISP_UFOE] = "ufoe",
- [MTK_DSI] = "dsi",
- [MTK_DPI] = "dpi",
- [MTK_DISP_PWM] = "pwm",
+ [MTK_DISP_GAMMA] = "gamma",
[MTK_DISP_MUTEX] = "mutex",
[MTK_DISP_OD] = "od",
- [MTK_DISP_BLS] = "bls",
+ [MTK_DISP_OVL] = "ovl",
+ [MTK_DISP_OVL_2L] = "ovl-2l",
+ [MTK_DISP_POSTMASK] = "postmask",
+ [MTK_DISP_PWM] = "pwm",
+ [MTK_DISP_RDMA] = "rdma",
+ [MTK_DISP_UFOE] = "ufoe",
+ [MTK_DISP_WDMA] = "wdma",
+ [MTK_DPI] = "dpi",
+ [MTK_DSI] = "dsi",
};
struct mtk_ddp_comp_match {
@@ -330,35 +360,38 @@ struct mtk_ddp_comp_match {
};
static const struct mtk_ddp_comp_match mtk_ddp_matches[DDP_COMPONENT_ID_MAX] = {
- [DDP_COMPONENT_AAL0] = { MTK_DISP_AAL, 0, &ddp_aal },
- [DDP_COMPONENT_AAL1] = { MTK_DISP_AAL, 1, &ddp_aal },
- [DDP_COMPONENT_BLS] = { MTK_DISP_BLS, 0, NULL },
- [DDP_COMPONENT_CCORR] = { MTK_DISP_CCORR, 0, &ddp_ccorr },
- [DDP_COMPONENT_COLOR0] = { MTK_DISP_COLOR, 0, &ddp_color },
- [DDP_COMPONENT_COLOR1] = { MTK_DISP_COLOR, 1, &ddp_color },
- [DDP_COMPONENT_DITHER] = { MTK_DISP_DITHER, 0, &ddp_dither },
- [DDP_COMPONENT_DPI0] = { MTK_DPI, 0, &ddp_dpi },
- [DDP_COMPONENT_DPI1] = { MTK_DPI, 1, &ddp_dpi },
- [DDP_COMPONENT_DSI0] = { MTK_DSI, 0, &ddp_dsi },
- [DDP_COMPONENT_DSI1] = { MTK_DSI, 1, &ddp_dsi },
- [DDP_COMPONENT_DSI2] = { MTK_DSI, 2, &ddp_dsi },
- [DDP_COMPONENT_DSI3] = { MTK_DSI, 3, &ddp_dsi },
- [DDP_COMPONENT_GAMMA] = { MTK_DISP_GAMMA, 0, &ddp_gamma },
- [DDP_COMPONENT_OD0] = { MTK_DISP_OD, 0, &ddp_od },
- [DDP_COMPONENT_OD1] = { MTK_DISP_OD, 1, &ddp_od },
- [DDP_COMPONENT_OVL0] = { MTK_DISP_OVL, 0, &ddp_ovl },
- [DDP_COMPONENT_OVL1] = { MTK_DISP_OVL, 1, &ddp_ovl },
- [DDP_COMPONENT_OVL_2L0] = { MTK_DISP_OVL_2L, 0, &ddp_ovl },
- [DDP_COMPONENT_OVL_2L1] = { MTK_DISP_OVL_2L, 1, &ddp_ovl },
- [DDP_COMPONENT_PWM0] = { MTK_DISP_PWM, 0, NULL },
- [DDP_COMPONENT_PWM1] = { MTK_DISP_PWM, 1, NULL },
- [DDP_COMPONENT_PWM2] = { MTK_DISP_PWM, 2, NULL },
- [DDP_COMPONENT_RDMA0] = { MTK_DISP_RDMA, 0, &ddp_rdma },
- [DDP_COMPONENT_RDMA1] = { MTK_DISP_RDMA, 1, &ddp_rdma },
- [DDP_COMPONENT_RDMA2] = { MTK_DISP_RDMA, 2, &ddp_rdma },
- [DDP_COMPONENT_UFOE] = { MTK_DISP_UFOE, 0, &ddp_ufoe },
- [DDP_COMPONENT_WDMA0] = { MTK_DISP_WDMA, 0, NULL },
- [DDP_COMPONENT_WDMA1] = { MTK_DISP_WDMA, 1, NULL },
+ [DDP_COMPONENT_AAL0] = { MTK_DISP_AAL, 0, &ddp_aal },
+ [DDP_COMPONENT_AAL1] = { MTK_DISP_AAL, 1, &ddp_aal },
+ [DDP_COMPONENT_BLS] = { MTK_DISP_BLS, 0, NULL },
+ [DDP_COMPONENT_CCORR] = { MTK_DISP_CCORR, 0, &ddp_ccorr },
+ [DDP_COMPONENT_COLOR0] = { MTK_DISP_COLOR, 0, &ddp_color },
+ [DDP_COMPONENT_COLOR1] = { MTK_DISP_COLOR, 1, &ddp_color },
+ [DDP_COMPONENT_DITHER] = { MTK_DISP_DITHER, 0, &ddp_dither },
+ [DDP_COMPONENT_DPI0] = { MTK_DPI, 0, &ddp_dpi },
+ [DDP_COMPONENT_DPI1] = { MTK_DPI, 1, &ddp_dpi },
+ [DDP_COMPONENT_DSI0] = { MTK_DSI, 0, &ddp_dsi },
+ [DDP_COMPONENT_DSI1] = { MTK_DSI, 1, &ddp_dsi },
+ [DDP_COMPONENT_DSI2] = { MTK_DSI, 2, &ddp_dsi },
+ [DDP_COMPONENT_DSI3] = { MTK_DSI, 3, &ddp_dsi },
+ [DDP_COMPONENT_GAMMA] = { MTK_DISP_GAMMA, 0, &ddp_gamma },
+ [DDP_COMPONENT_OD0] = { MTK_DISP_OD, 0, &ddp_od },
+ [DDP_COMPONENT_OD1] = { MTK_DISP_OD, 1, &ddp_od },
+ [DDP_COMPONENT_OVL0] = { MTK_DISP_OVL, 0, &ddp_ovl },
+ [DDP_COMPONENT_OVL1] = { MTK_DISP_OVL, 1, &ddp_ovl },
+ [DDP_COMPONENT_OVL_2L0] = { MTK_DISP_OVL_2L, 0, &ddp_ovl },
+ [DDP_COMPONENT_OVL_2L1] = { MTK_DISP_OVL_2L, 1, &ddp_ovl },
+ [DDP_COMPONENT_OVL_2L2] = { MTK_DISP_OVL_2L, 2, &ddp_ovl },
+ [DDP_COMPONENT_POSTMASK0] = { MTK_DISP_POSTMASK, 0, &ddp_postmask },
+ [DDP_COMPONENT_PWM0] = { MTK_DISP_PWM, 0, NULL },
+ [DDP_COMPONENT_PWM1] = { MTK_DISP_PWM, 1, NULL },
+ [DDP_COMPONENT_PWM2] = { MTK_DISP_PWM, 2, NULL },
+ [DDP_COMPONENT_RDMA0] = { MTK_DISP_RDMA, 0, &ddp_rdma },
+ [DDP_COMPONENT_RDMA1] = { MTK_DISP_RDMA, 1, &ddp_rdma },
+ [DDP_COMPONENT_RDMA2] = { MTK_DISP_RDMA, 2, &ddp_rdma },
+ [DDP_COMPONENT_RDMA4] = { MTK_DISP_RDMA, 4, &ddp_rdma },
+ [DDP_COMPONENT_UFOE] = { MTK_DISP_UFOE, 0, &ddp_ufoe },
+ [DDP_COMPONENT_WDMA0] = { MTK_DISP_WDMA, 0, NULL },
+ [DDP_COMPONENT_WDMA1] = { MTK_DISP_WDMA, 1, NULL },
};
static bool mtk_drm_find_comp_in_ddp(struct device *dev,
@@ -475,12 +508,12 @@ int mtk_ddp_comp_init(struct device_node *node, struct mtk_ddp_comp *comp,
type == MTK_DISP_CCORR ||
type == MTK_DISP_COLOR ||
type == MTK_DISP_GAMMA ||
- type == MTK_DPI ||
- type == MTK_DSI ||
type == MTK_DISP_OVL ||
type == MTK_DISP_OVL_2L ||
type == MTK_DISP_PWM ||
- type == MTK_DISP_RDMA)
+ type == MTK_DISP_RDMA ||
+ type == MTK_DPI ||
+ type == MTK_DSI)
return 0;
priv = devm_kzalloc(comp->dev, sizeof(*priv), GFP_KERNEL);
diff --git a/drivers/gpu/drm/mediatek/mtk_drm_ddp_comp.h b/drivers/gpu/drm/mediatek/mtk_drm_ddp_comp.h
index bb914d9..4c6a986 100644
--- a/drivers/gpu/drm/mediatek/mtk_drm_ddp_comp.h
+++ b/drivers/gpu/drm/mediatek/mtk_drm_ddp_comp.h
@@ -18,22 +18,23 @@ struct mtk_plane_state;
struct drm_crtc_state;
enum mtk_ddp_comp_type {
- MTK_DISP_OVL,
- MTK_DISP_OVL_2L,
- MTK_DISP_RDMA,
- MTK_DISP_WDMA,
- MTK_DISP_COLOR,
- MTK_DISP_CCORR,
- MTK_DISP_DITHER,
MTK_DISP_AAL,
+ MTK_DISP_BLS,
+ MTK_DISP_CCORR,
+ MTK_DISP_COLOR,
+ MTK_DISP_DITHER,
MTK_DISP_GAMMA,
- MTK_DISP_UFOE,
- MTK_DSI,
- MTK_DPI,
- MTK_DISP_PWM,
MTK_DISP_MUTEX,
MTK_DISP_OD,
- MTK_DISP_BLS,
+ MTK_DISP_OVL,
+ MTK_DISP_OVL_2L,
+ MTK_DISP_POSTMASK,
+ MTK_DISP_PWM,
+ MTK_DISP_RDMA,
+ MTK_DISP_UFOE,
+ MTK_DISP_WDMA,
+ MTK_DPI,
+ MTK_DSI,
MTK_DDP_COMP_TYPE_MAX,
};
diff --git a/drivers/gpu/drm/mediatek/mtk_drm_drv.c b/drivers/gpu/drm/mediatek/mtk_drm_drv.c
index aec3972..56ff8c5 100644
--- a/drivers/gpu/drm/mediatek/mtk_drm_drv.c
+++ b/drivers/gpu/drm/mediatek/mtk_drm_drv.c
@@ -158,6 +158,25 @@ static const enum mtk_ddp_comp_id mt8183_mtk_ddp_ext[] = {
DDP_COMPONENT_DPI0,
};
+static const enum mtk_ddp_comp_id mt8192_mtk_ddp_main[] = {
+ DDP_COMPONENT_OVL0,
+ DDP_COMPONENT_OVL_2L0,
+ DDP_COMPONENT_RDMA0,
+ DDP_COMPONENT_COLOR0,
+ DDP_COMPONENT_CCORR,
+ DDP_COMPONENT_AAL0,
+ DDP_COMPONENT_GAMMA,
+ DDP_COMPONENT_POSTMASK0,
+ DDP_COMPONENT_DITHER,
+ DDP_COMPONENT_DSI0,
+};
+
+static const enum mtk_ddp_comp_id mt8192_mtk_ddp_ext[] = {
+ DDP_COMPONENT_OVL_2L2,
+ DDP_COMPONENT_RDMA4,
+ DDP_COMPONENT_DPI0,
+};
+
static const struct mtk_mmsys_driver_data mt2701_mmsys_driver_data = {
.main_path = mt2701_mtk_ddp_main,
.main_len = ARRAY_SIZE(mt2701_mtk_ddp_main),
@@ -202,6 +221,13 @@ static const struct mtk_mmsys_driver_data mt8183_mmsys_driver_data = {
.ext_len = ARRAY_SIZE(mt8183_mtk_ddp_ext),
};
+static const struct mtk_mmsys_driver_data mt8192_mmsys_driver_data = {
+ .main_path = mt8192_mtk_ddp_main,
+ .main_len = ARRAY_SIZE(mt8192_mtk_ddp_main),
+ .ext_path = mt8192_mtk_ddp_ext,
+ .ext_len = ARRAY_SIZE(mt8192_mtk_ddp_ext),
+};
+
static int mtk_drm_kms_init(struct drm_device *drm)
{
struct mtk_drm_private *private = drm->dev_private;
@@ -397,68 +423,36 @@ static const struct component_master_ops mtk_drm_ops = {
};
static const struct of_device_id mtk_ddp_comp_dt_ids[] = {
- { .compatible = "mediatek,mt2701-disp-ovl",
- .data = (void *)MTK_DISP_OVL },
- { .compatible = "mediatek,mt8167-disp-ovl",
- .data = (void *)MTK_DISP_OVL },
- { .compatible = "mediatek,mt8173-disp-ovl",
- .data = (void *)MTK_DISP_OVL },
- { .compatible = "mediatek,mt8183-disp-ovl",
- .data = (void *)MTK_DISP_OVL },
- { .compatible = "mediatek,mt8183-disp-ovl-2l",
- .data = (void *)MTK_DISP_OVL_2L },
- { .compatible = "mediatek,mt2701-disp-rdma",
- .data = (void *)MTK_DISP_RDMA },
- { .compatible = "mediatek,mt8167-disp-rdma",
- .data = (void *)MTK_DISP_RDMA },
- { .compatible = "mediatek,mt8173-disp-rdma",
- .data = (void *)MTK_DISP_RDMA },
- { .compatible = "mediatek,mt8183-disp-rdma",
- .data = (void *)MTK_DISP_RDMA },
- { .compatible = "mediatek,mt8173-disp-wdma",
- .data = (void *)MTK_DISP_WDMA },
+ { .compatible = "mediatek,mt8167-disp-aal",
+ .data = (void *)MTK_DISP_AAL},
+ { .compatible = "mediatek,mt8173-disp-aal",
+ .data = (void *)MTK_DISP_AAL},
+ { .compatible = "mediatek,mt8183-disp-aal",
+ .data = (void *)MTK_DISP_AAL},
+ { .compatible = "mediatek,mt8192-disp-aal",
+ .data = (void *)MTK_DISP_AAL},
{ .compatible = "mediatek,mt8167-disp-ccorr",
.data = (void *)MTK_DISP_CCORR },
{ .compatible = "mediatek,mt8183-disp-ccorr",
.data = (void *)MTK_DISP_CCORR },
+ { .compatible = "mediatek,mt8192-disp-ccorr",
+ .data = (void *)MTK_DISP_CCORR },
{ .compatible = "mediatek,mt2701-disp-color",
.data = (void *)MTK_DISP_COLOR },
{ .compatible = "mediatek,mt8167-disp-color",
.data = (void *)MTK_DISP_COLOR },
{ .compatible = "mediatek,mt8173-disp-color",
.data = (void *)MTK_DISP_COLOR },
- { .compatible = "mediatek,mt8167-disp-aal",
- .data = (void *)MTK_DISP_AAL},
- { .compatible = "mediatek,mt8173-disp-aal",
- .data = (void *)MTK_DISP_AAL},
- { .compatible = "mediatek,mt8183-disp-aal",
- .data = (void *)MTK_DISP_AAL},
+ { .compatible = "mediatek,mt8167-disp-dither",
+ .data = (void *)MTK_DISP_DITHER },
+ { .compatible = "mediatek,mt8183-disp-dither",
+ .data = (void *)MTK_DISP_DITHER },
{ .compatible = "mediatek,mt8167-disp-gamma",
.data = (void *)MTK_DISP_GAMMA, },
{ .compatible = "mediatek,mt8173-disp-gamma",
.data = (void *)MTK_DISP_GAMMA, },
{ .compatible = "mediatek,mt8183-disp-gamma",
.data = (void *)MTK_DISP_GAMMA, },
- { .compatible = "mediatek,mt8167-disp-dither",
- .data = (void *)MTK_DISP_DITHER },
- { .compatible = "mediatek,mt8183-disp-dither",
- .data = (void *)MTK_DISP_DITHER },
- { .compatible = "mediatek,mt8173-disp-ufoe",
- .data = (void *)MTK_DISP_UFOE },
- { .compatible = "mediatek,mt2701-dsi",
- .data = (void *)MTK_DSI },
- { .compatible = "mediatek,mt8167-dsi",
- .data = (void *)MTK_DSI },
- { .compatible = "mediatek,mt8173-dsi",
- .data = (void *)MTK_DSI },
- { .compatible = "mediatek,mt8183-dsi",
- .data = (void *)MTK_DSI },
- { .compatible = "mediatek,mt2701-dpi",
- .data = (void *)MTK_DPI },
- { .compatible = "mediatek,mt8173-dpi",
- .data = (void *)MTK_DPI },
- { .compatible = "mediatek,mt8183-dpi",
- .data = (void *)MTK_DPI },
{ .compatible = "mediatek,mt2701-disp-mutex",
.data = (void *)MTK_DISP_MUTEX },
{ .compatible = "mediatek,mt2712-disp-mutex",
@@ -469,14 +463,60 @@ static const struct of_device_id mtk_ddp_comp_dt_ids[] = {
.data = (void *)MTK_DISP_MUTEX },
{ .compatible = "mediatek,mt8183-disp-mutex",
.data = (void *)MTK_DISP_MUTEX },
+ { .compatible = "mediatek,mt8192-disp-mutex",
+ .data = (void *)MTK_DISP_MUTEX },
+ { .compatible = "mediatek,mt8173-disp-od",
+ .data = (void *)MTK_DISP_OD },
+ { .compatible = "mediatek,mt2701-disp-ovl",
+ .data = (void *)MTK_DISP_OVL },
+ { .compatible = "mediatek,mt8167-disp-ovl",
+ .data = (void *)MTK_DISP_OVL },
+ { .compatible = "mediatek,mt8173-disp-ovl",
+ .data = (void *)MTK_DISP_OVL },
+ { .compatible = "mediatek,mt8183-disp-ovl",
+ .data = (void *)MTK_DISP_OVL },
+ { .compatible = "mediatek,mt8192-disp-ovl",
+ .data = (void *)MTK_DISP_OVL },
+ { .compatible = "mediatek,mt8183-disp-ovl-2l",
+ .data = (void *)MTK_DISP_OVL_2L },
+ { .compatible = "mediatek,mt8192-disp-ovl-2l",
+ .data = (void *)MTK_DISP_OVL_2L },
+ { .compatible = "mediatek,mt8192-disp-postmask",
+ .data = (void *)MTK_DISP_POSTMASK },
{ .compatible = "mediatek,mt2701-disp-pwm",
.data = (void *)MTK_DISP_BLS },
{ .compatible = "mediatek,mt8167-disp-pwm",
.data = (void *)MTK_DISP_PWM },
{ .compatible = "mediatek,mt8173-disp-pwm",
.data = (void *)MTK_DISP_PWM },
- { .compatible = "mediatek,mt8173-disp-od",
- .data = (void *)MTK_DISP_OD },
+ { .compatible = "mediatek,mt2701-disp-rdma",
+ .data = (void *)MTK_DISP_RDMA },
+ { .compatible = "mediatek,mt8167-disp-rdma",
+ .data = (void *)MTK_DISP_RDMA },
+ { .compatible = "mediatek,mt8173-disp-rdma",
+ .data = (void *)MTK_DISP_RDMA },
+ { .compatible = "mediatek,mt8183-disp-rdma",
+ .data = (void *)MTK_DISP_RDMA },
+ { .compatible = "mediatek,mt8192-disp-rdma",
+ .data = (void *)MTK_DISP_RDMA },
+ { .compatible = "mediatek,mt8173-disp-ufoe",
+ .data = (void *)MTK_DISP_UFOE },
+ { .compatible = "mediatek,mt8173-disp-wdma",
+ .data = (void *)MTK_DISP_WDMA },
+ { .compatible = "mediatek,mt2701-dpi",
+ .data = (void *)MTK_DPI },
+ { .compatible = "mediatek,mt8167-dsi",
+ .data = (void *)MTK_DSI },
+ { .compatible = "mediatek,mt8173-dpi",
+ .data = (void *)MTK_DPI },
+ { .compatible = "mediatek,mt8183-dpi",
+ .data = (void *)MTK_DPI },
+ { .compatible = "mediatek,mt2701-dsi",
+ .data = (void *)MTK_DSI },
+ { .compatible = "mediatek,mt8173-dsi",
+ .data = (void *)MTK_DSI },
+ { .compatible = "mediatek,mt8183-dsi",
+ .data = (void *)MTK_DSI },
{ }
};
@@ -493,6 +533,8 @@ static const struct of_device_id mtk_drm_of_ids[] = {
.data = &mt8173_mmsys_driver_data},
{ .compatible = "mediatek,mt8183-mmsys",
.data = &mt8183_mmsys_driver_data},
+ { .compatible = "mediatek,mt8192-mmsys",
+ .data = &mt8192_mmsys_driver_data},
{ }
};
MODULE_DEVICE_TABLE(of, mtk_drm_of_ids);
@@ -568,8 +610,8 @@ static int mtk_drm_probe(struct platform_device *pdev)
comp_type == MTK_DISP_OVL ||
comp_type == MTK_DISP_OVL_2L ||
comp_type == MTK_DISP_RDMA ||
- comp_type == MTK_DSI ||
- comp_type == MTK_DPI) {
+ comp_type == MTK_DPI ||
+ comp_type == MTK_DSI) {
dev_info(dev, "Adding component match for %pOF\n",
node);
drm_of_component_match_add(dev, &match, compare_of,
diff --git a/drivers/gpu/drm/mediatek/mtk_drm_plane.c b/drivers/gpu/drm/mediatek/mtk_drm_plane.c
index 734a1fb..075747a 100644
--- a/drivers/gpu/drm/mediatek/mtk_drm_plane.c
+++ b/drivers/gpu/drm/mediatek/mtk_drm_plane.c
@@ -44,9 +44,10 @@ static void mtk_plane_reset(struct drm_plane *plane)
state = kzalloc(sizeof(*state), GFP_KERNEL);
if (!state)
return;
- plane->state = &state->base;
}
+ __drm_atomic_helper_plane_reset(plane, &state->base);
+
state->base.plane = plane;
state->pending.format = DRM_FORMAT_RGB565;
}
diff --git a/drivers/gpu/drm/meson/Kconfig b/drivers/gpu/drm/meson/Kconfig
index 9f9281d..6c70fc3 100644
--- a/drivers/gpu/drm/meson/Kconfig
+++ b/drivers/gpu/drm/meson/Kconfig
@@ -4,11 +4,12 @@
depends on DRM && OF && (ARM || ARM64)
depends on ARCH_MESON || COMPILE_TEST
select DRM_KMS_HELPER
- select DRM_KMS_CMA_HELPER
select DRM_GEM_CMA_HELPER
+ select DRM_DISPLAY_CONNECTOR
select VIDEOMODE_HELPERS
select REGMAP_MMIO
select MESON_CANVAS
+ select CEC_CORE if CEC_NOTIFIER
config DRM_MESON_DW_HDMI
tristate "HDMI Synopsys Controller support for Amlogic Meson Display"
diff --git a/drivers/gpu/drm/meson/Makefile b/drivers/gpu/drm/meson/Makefile
index 28a519c..3afa31b 100644
--- a/drivers/gpu/drm/meson/Makefile
+++ b/drivers/gpu/drm/meson/Makefile
@@ -1,7 +1,8 @@
# SPDX-License-Identifier: GPL-2.0-only
-meson-drm-y := meson_drv.o meson_plane.o meson_crtc.o meson_venc_cvbs.o
+meson-drm-y := meson_drv.o meson_plane.o meson_crtc.o meson_encoder_cvbs.o
meson-drm-y += meson_viu.o meson_vpp.o meson_venc.o meson_vclk.o meson_overlay.o
meson-drm-y += meson_rdma.o meson_osd_afbcd.o
+meson-drm-y += meson_encoder_hdmi.o
obj-$(CONFIG_DRM_MESON) += meson-drm.o
obj-$(CONFIG_DRM_MESON_DW_HDMI) += meson_dw_hdmi.o
diff --git a/drivers/gpu/drm/meson/meson_drv.c b/drivers/gpu/drm/meson/meson_drv.c
index 7f41a33..80f1d43 100644
--- a/drivers/gpu/drm/meson/meson_drv.c
+++ b/drivers/gpu/drm/meson/meson_drv.c
@@ -31,7 +31,8 @@
#include "meson_plane.h"
#include "meson_osd_afbcd.h"
#include "meson_registers.h"
-#include "meson_venc_cvbs.h"
+#include "meson_encoder_cvbs.h"
+#include "meson_encoder_hdmi.h"
#include "meson_viu.h"
#include "meson_vpp.h"
#include "meson_rdma.h"
@@ -306,7 +307,7 @@ static int meson_drv_bind_master(struct device *dev, bool has_components)
/* Encoder Initialization */
- ret = meson_venc_cvbs_create(priv);
+ ret = meson_encoder_cvbs_init(priv);
if (ret)
goto free_drm;
@@ -318,6 +319,10 @@ static int meson_drv_bind_master(struct device *dev, bool has_components)
}
}
+ ret = meson_encoder_hdmi_init(priv);
+ if (ret)
+ goto free_drm;
+
ret = meson_plane_create(priv);
if (ret)
goto free_drm;
@@ -426,46 +431,6 @@ static int compare_of(struct device *dev, void *data)
return dev->of_node == data;
}
-/* Possible connectors nodes to ignore */
-static const struct of_device_id connectors_match[] = {
- { .compatible = "composite-video-connector" },
- { .compatible = "svideo-connector" },
- { .compatible = "hdmi-connector" },
- { .compatible = "dvi-connector" },
- {}
-};
-
-static int meson_probe_remote(struct platform_device *pdev,
- struct component_match **match,
- struct device_node *parent,
- struct device_node *remote)
-{
- struct device_node *ep, *remote_node;
- int count = 1;
-
- /* If node is a connector, return and do not add to match table */
- if (of_match_node(connectors_match, remote))
- return 1;
-
- component_match_add(&pdev->dev, match, compare_of, remote);
-
- for_each_endpoint_of_node(remote, ep) {
- remote_node = of_graph_get_remote_port_parent(ep);
- if (!remote_node ||
- remote_node == parent || /* Ignore parent endpoint */
- !of_device_is_available(remote_node)) {
- of_node_put(remote_node);
- continue;
- }
-
- count += meson_probe_remote(pdev, match, remote, remote_node);
-
- of_node_put(remote_node);
- }
-
- return count;
-}
-
static void meson_drv_shutdown(struct platform_device *pdev)
{
struct meson_drm *priv = dev_get_drvdata(&pdev->dev);
@@ -477,6 +442,13 @@ static void meson_drv_shutdown(struct platform_device *pdev)
drm_atomic_helper_shutdown(priv->drm);
}
+/* Possible connectors nodes to ignore */
+static const struct of_device_id connectors_match[] = {
+ { .compatible = "composite-video-connector" },
+ { .compatible = "svideo-connector" },
+ {}
+};
+
static int meson_drv_probe(struct platform_device *pdev)
{
struct component_match *match = NULL;
@@ -491,8 +463,21 @@ static int meson_drv_probe(struct platform_device *pdev)
continue;
}
- count += meson_probe_remote(pdev, &match, np, remote);
+ /* If an analog connector is detected, count it as an output */
+ if (of_match_node(connectors_match, remote)) {
+ ++count;
+ of_node_put(remote);
+ continue;
+ }
+
+ dev_dbg(&pdev->dev, "parent %pOF remote match add %pOF parent %s\n",
+ np, remote, dev_name(&pdev->dev));
+
+ component_match_add(&pdev->dev, &match, compare_of, remote);
+
of_node_put(remote);
+
+ ++count;
}
if (count && !match)
diff --git a/drivers/gpu/drm/meson/meson_dw_hdmi.c b/drivers/gpu/drm/meson/meson_dw_hdmi.c
index 0afbd1e7..5cd2b2e 100644
--- a/drivers/gpu/drm/meson/meson_dw_hdmi.c
+++ b/drivers/gpu/drm/meson/meson_dw_hdmi.c
@@ -22,14 +22,11 @@
#include <drm/drm_probe_helper.h>
#include <drm/drm_print.h>
-#include <linux/media-bus-format.h>
#include <linux/videodev2.h>
#include "meson_drv.h"
#include "meson_dw_hdmi.h"
#include "meson_registers.h"
-#include "meson_vclk.h"
-#include "meson_venc.h"
#define DRIVER_NAME "meson-dw-hdmi"
#define DRIVER_DESC "Amlogic Meson HDMI-TX DRM driver"
@@ -135,8 +132,6 @@ struct meson_dw_hdmi_data {
};
struct meson_dw_hdmi {
- struct drm_encoder encoder;
- struct drm_bridge bridge;
struct dw_hdmi_plat_data dw_plat_data;
struct meson_drm *priv;
struct device *dev;
@@ -148,12 +143,8 @@ struct meson_dw_hdmi {
struct regulator *hdmi_supply;
u32 irq_stat;
struct dw_hdmi *hdmi;
- unsigned long output_bus_fmt;
+ struct drm_bridge *bridge;
};
-#define encoder_to_meson_dw_hdmi(x) \
- container_of(x, struct meson_dw_hdmi, encoder)
-#define bridge_to_meson_dw_hdmi(x) \
- container_of(x, struct meson_dw_hdmi, bridge)
static inline int dw_hdmi_is_compatible(struct meson_dw_hdmi *dw_hdmi,
const char *compat)
@@ -295,14 +286,14 @@ static inline void dw_hdmi_dwc_write_bits(struct meson_dw_hdmi *dw_hdmi,
/* Setup PHY bandwidth modes */
static void meson_hdmi_phy_setup_mode(struct meson_dw_hdmi *dw_hdmi,
- const struct drm_display_mode *mode)
+ const struct drm_display_mode *mode,
+ bool mode_is_420)
{
struct meson_drm *priv = dw_hdmi->priv;
unsigned int pixel_clock = mode->clock;
/* For 420, pixel clock is half unlike venc clock */
- if (dw_hdmi->output_bus_fmt == MEDIA_BUS_FMT_UYYVYY8_0_5X24)
- pixel_clock /= 2;
+ if (mode_is_420) pixel_clock /= 2;
if (dw_hdmi_is_compatible(dw_hdmi, "amlogic,meson-gxl-dw-hdmi") ||
dw_hdmi_is_compatible(dw_hdmi, "amlogic,meson-gxm-dw-hdmi")) {
@@ -374,68 +365,25 @@ static inline void meson_dw_hdmi_phy_reset(struct meson_dw_hdmi *dw_hdmi)
mdelay(2);
}
-static void dw_hdmi_set_vclk(struct meson_dw_hdmi *dw_hdmi,
- const struct drm_display_mode *mode)
-{
- struct meson_drm *priv = dw_hdmi->priv;
- int vic = drm_match_cea_mode(mode);
- unsigned int phy_freq;
- unsigned int vclk_freq;
- unsigned int venc_freq;
- unsigned int hdmi_freq;
-
- vclk_freq = mode->clock;
-
- /* For 420, pixel clock is half unlike venc clock */
- if (dw_hdmi->output_bus_fmt == MEDIA_BUS_FMT_UYYVYY8_0_5X24)
- vclk_freq /= 2;
-
- /* TMDS clock is pixel_clock * 10 */
- phy_freq = vclk_freq * 10;
-
- if (!vic) {
- meson_vclk_setup(priv, MESON_VCLK_TARGET_DMT, phy_freq,
- vclk_freq, vclk_freq, vclk_freq, false);
- return;
- }
-
- /* 480i/576i needs global pixel doubling */
- if (mode->flags & DRM_MODE_FLAG_DBLCLK)
- vclk_freq *= 2;
-
- venc_freq = vclk_freq;
- hdmi_freq = vclk_freq;
-
- /* VENC double pixels for 1080i, 720p and YUV420 modes */
- if (meson_venc_hdmi_venc_repeat(vic) ||
- dw_hdmi->output_bus_fmt == MEDIA_BUS_FMT_UYYVYY8_0_5X24)
- venc_freq *= 2;
-
- vclk_freq = max(venc_freq, hdmi_freq);
-
- if (mode->flags & DRM_MODE_FLAG_DBLCLK)
- venc_freq /= 2;
-
- DRM_DEBUG_DRIVER("vclk:%d phy=%d venc=%d hdmi=%d enci=%d\n",
- phy_freq, vclk_freq, venc_freq, hdmi_freq,
- priv->venc.hdmi_use_enci);
-
- meson_vclk_setup(priv, MESON_VCLK_TARGET_HDMI, phy_freq, vclk_freq,
- venc_freq, hdmi_freq, priv->venc.hdmi_use_enci);
-}
-
static int dw_hdmi_phy_init(struct dw_hdmi *hdmi, void *data,
const struct drm_display_info *display,
const struct drm_display_mode *mode)
{
struct meson_dw_hdmi *dw_hdmi = (struct meson_dw_hdmi *)data;
+ bool is_hdmi2_sink = display->hdmi.scdc.supported;
struct meson_drm *priv = dw_hdmi->priv;
unsigned int wr_clk =
readl_relaxed(priv->io_base + _REG(VPU_HDMI_SETTING));
+ bool mode_is_420 = false;
DRM_DEBUG_DRIVER("\"%s\" div%d\n", mode->name,
mode->clock > 340000 ? 40 : 10);
+ if (drm_mode_is_420_only(display, mode) ||
+ (!is_hdmi2_sink &&
+ drm_mode_is_420_also(display, mode)))
+ mode_is_420 = true;
+
/* Enable clocks */
regmap_update_bits(priv->hhi, HHI_HDMI_CLK_CNTL, 0xffff, 0x100);
@@ -457,8 +405,7 @@ static int dw_hdmi_phy_init(struct dw_hdmi *hdmi, void *data,
dw_hdmi->data->top_write(dw_hdmi, HDMITX_TOP_BIST_CNTL, BIT(12));
/* TMDS pattern setup */
- if (mode->clock > 340000 &&
- dw_hdmi->output_bus_fmt == MEDIA_BUS_FMT_YUV8_1X24) {
+ if (mode->clock > 340000 && !mode_is_420) {
dw_hdmi->data->top_write(dw_hdmi, HDMITX_TOP_TMDS_CLK_PTTN_01,
0);
dw_hdmi->data->top_write(dw_hdmi, HDMITX_TOP_TMDS_CLK_PTTN_23,
@@ -476,7 +423,7 @@ static int dw_hdmi_phy_init(struct dw_hdmi *hdmi, void *data,
dw_hdmi->data->top_write(dw_hdmi, HDMITX_TOP_TMDS_CLK_PTTN_CNTL, 0x2);
/* Setup PHY parameters */
- meson_hdmi_phy_setup_mode(dw_hdmi, mode);
+ meson_hdmi_phy_setup_mode(dw_hdmi, mode, mode_is_420);
/* Setup PHY */
regmap_update_bits(priv->hhi, HHI_HDMI_PHY_CNTL1,
@@ -622,214 +569,15 @@ static irqreturn_t dw_hdmi_top_thread_irq(int irq, void *dev_id)
dw_hdmi_setup_rx_sense(dw_hdmi->hdmi, hpd_connected,
hpd_connected);
- drm_helper_hpd_irq_event(dw_hdmi->encoder.dev);
+ drm_helper_hpd_irq_event(dw_hdmi->bridge->dev);
+ drm_bridge_hpd_notify(dw_hdmi->bridge,
+ hpd_connected ? connector_status_connected
+ : connector_status_disconnected);
}
return IRQ_HANDLED;
}
-static enum drm_mode_status
-dw_hdmi_mode_valid(struct dw_hdmi *hdmi, void *data,
- const struct drm_display_info *display_info,
- const struct drm_display_mode *mode)
-{
- struct meson_dw_hdmi *dw_hdmi = data;
- struct meson_drm *priv = dw_hdmi->priv;
- bool is_hdmi2_sink = display_info->hdmi.scdc.supported;
- unsigned int phy_freq;
- unsigned int vclk_freq;
- unsigned int venc_freq;
- unsigned int hdmi_freq;
- int vic = drm_match_cea_mode(mode);
- enum drm_mode_status status;
-
- DRM_DEBUG_DRIVER("Modeline " DRM_MODE_FMT "\n", DRM_MODE_ARG(mode));
-
- /* If sink does not support 540MHz, reject the non-420 HDMI2 modes */
- if (display_info->max_tmds_clock &&
- mode->clock > display_info->max_tmds_clock &&
- !drm_mode_is_420_only(display_info, mode) &&
- !drm_mode_is_420_also(display_info, mode))
- return MODE_BAD;
-
- /* Check against non-VIC supported modes */
- if (!vic) {
- status = meson_venc_hdmi_supported_mode(mode);
- if (status != MODE_OK)
- return status;
-
- return meson_vclk_dmt_supported_freq(priv, mode->clock);
- /* Check against supported VIC modes */
- } else if (!meson_venc_hdmi_supported_vic(vic))
- return MODE_BAD;
-
- vclk_freq = mode->clock;
-
- /* For 420, pixel clock is half unlike venc clock */
- if (drm_mode_is_420_only(display_info, mode) ||
- (!is_hdmi2_sink &&
- drm_mode_is_420_also(display_info, mode)))
- vclk_freq /= 2;
-
- /* TMDS clock is pixel_clock * 10 */
- phy_freq = vclk_freq * 10;
-
- /* 480i/576i needs global pixel doubling */
- if (mode->flags & DRM_MODE_FLAG_DBLCLK)
- vclk_freq *= 2;
-
- venc_freq = vclk_freq;
- hdmi_freq = vclk_freq;
-
- /* VENC double pixels for 1080i, 720p and YUV420 modes */
- if (meson_venc_hdmi_venc_repeat(vic) ||
- drm_mode_is_420_only(display_info, mode) ||
- (!is_hdmi2_sink &&
- drm_mode_is_420_also(display_info, mode)))
- venc_freq *= 2;
-
- vclk_freq = max(venc_freq, hdmi_freq);
-
- if (mode->flags & DRM_MODE_FLAG_DBLCLK)
- venc_freq /= 2;
-
- dev_dbg(dw_hdmi->dev, "%s: vclk:%d phy=%d venc=%d hdmi=%d\n",
- __func__, phy_freq, vclk_freq, venc_freq, hdmi_freq);
-
- return meson_vclk_vic_supported_freq(priv, phy_freq, vclk_freq);
-}
-
-/* Encoder */
-
-static const u32 meson_dw_hdmi_out_bus_fmts[] = {
- MEDIA_BUS_FMT_YUV8_1X24,
- MEDIA_BUS_FMT_UYYVYY8_0_5X24,
-};
-
-static void meson_venc_hdmi_encoder_destroy(struct drm_encoder *encoder)
-{
- drm_encoder_cleanup(encoder);
-}
-
-static const struct drm_encoder_funcs meson_venc_hdmi_encoder_funcs = {
- .destroy = meson_venc_hdmi_encoder_destroy,
-};
-
-static u32 *
-meson_venc_hdmi_encoder_get_inp_bus_fmts(struct drm_bridge *bridge,
- struct drm_bridge_state *bridge_state,
- struct drm_crtc_state *crtc_state,
- struct drm_connector_state *conn_state,
- u32 output_fmt,
- unsigned int *num_input_fmts)
-{
- u32 *input_fmts = NULL;
- int i;
-
- *num_input_fmts = 0;
-
- for (i = 0 ; i < ARRAY_SIZE(meson_dw_hdmi_out_bus_fmts) ; ++i) {
- if (output_fmt == meson_dw_hdmi_out_bus_fmts[i]) {
- *num_input_fmts = 1;
- input_fmts = kcalloc(*num_input_fmts,
- sizeof(*input_fmts),
- GFP_KERNEL);
- if (!input_fmts)
- return NULL;
-
- input_fmts[0] = output_fmt;
-
- break;
- }
- }
-
- return input_fmts;
-}
-
-static int meson_venc_hdmi_encoder_atomic_check(struct drm_bridge *bridge,
- struct drm_bridge_state *bridge_state,
- struct drm_crtc_state *crtc_state,
- struct drm_connector_state *conn_state)
-{
- struct meson_dw_hdmi *dw_hdmi = bridge_to_meson_dw_hdmi(bridge);
-
- dw_hdmi->output_bus_fmt = bridge_state->output_bus_cfg.format;
-
- DRM_DEBUG_DRIVER("output_bus_fmt %lx\n", dw_hdmi->output_bus_fmt);
-
- return 0;
-}
-
-static void meson_venc_hdmi_encoder_disable(struct drm_bridge *bridge)
-{
- struct meson_dw_hdmi *dw_hdmi = bridge_to_meson_dw_hdmi(bridge);
- struct meson_drm *priv = dw_hdmi->priv;
-
- DRM_DEBUG_DRIVER("\n");
-
- writel_bits_relaxed(0x3, 0,
- priv->io_base + _REG(VPU_HDMI_SETTING));
-
- writel_relaxed(0, priv->io_base + _REG(ENCI_VIDEO_EN));
- writel_relaxed(0, priv->io_base + _REG(ENCP_VIDEO_EN));
-}
-
-static void meson_venc_hdmi_encoder_enable(struct drm_bridge *bridge)
-{
- struct meson_dw_hdmi *dw_hdmi = bridge_to_meson_dw_hdmi(bridge);
- struct meson_drm *priv = dw_hdmi->priv;
-
- DRM_DEBUG_DRIVER("%s\n", priv->venc.hdmi_use_enci ? "VENCI" : "VENCP");
-
- if (priv->venc.hdmi_use_enci)
- writel_relaxed(1, priv->io_base + _REG(ENCI_VIDEO_EN));
- else
- writel_relaxed(1, priv->io_base + _REG(ENCP_VIDEO_EN));
-}
-
-static void meson_venc_hdmi_encoder_mode_set(struct drm_bridge *bridge,
- const struct drm_display_mode *mode,
- const struct drm_display_mode *adjusted_mode)
-{
- struct meson_dw_hdmi *dw_hdmi = bridge_to_meson_dw_hdmi(bridge);
- struct meson_drm *priv = dw_hdmi->priv;
- int vic = drm_match_cea_mode(mode);
- unsigned int ycrcb_map = VPU_HDMI_OUTPUT_CBYCR;
- bool yuv420_mode = false;
-
- DRM_DEBUG_DRIVER("\"%s\" vic %d\n", mode->name, vic);
-
- if (dw_hdmi->output_bus_fmt == MEDIA_BUS_FMT_UYYVYY8_0_5X24) {
- ycrcb_map = VPU_HDMI_OUTPUT_CRYCB;
- yuv420_mode = true;
- }
-
- /* VENC + VENC-DVI Mode setup */
- meson_venc_hdmi_mode_set(priv, vic, ycrcb_map, yuv420_mode, mode);
-
- /* VCLK Set clock */
- dw_hdmi_set_vclk(dw_hdmi, mode);
-
- if (dw_hdmi->output_bus_fmt == MEDIA_BUS_FMT_UYYVYY8_0_5X24)
- /* Setup YUV420 to HDMI-TX, no 10bit diphering */
- writel_relaxed(2 | (2 << 2),
- priv->io_base + _REG(VPU_HDMI_FMT_CTRL));
- else
- /* Setup YUV444 to HDMI-TX, no 10bit diphering */
- writel_relaxed(0, priv->io_base + _REG(VPU_HDMI_FMT_CTRL));
-}
-
-static const struct drm_bridge_funcs meson_venc_hdmi_encoder_bridge_funcs = {
- .atomic_duplicate_state = drm_atomic_helper_bridge_duplicate_state,
- .atomic_destroy_state = drm_atomic_helper_bridge_destroy_state,
- .atomic_get_input_bus_fmts = meson_venc_hdmi_encoder_get_inp_bus_fmts,
- .atomic_reset = drm_atomic_helper_bridge_reset,
- .atomic_check = meson_venc_hdmi_encoder_atomic_check,
- .enable = meson_venc_hdmi_encoder_enable,
- .disable = meson_venc_hdmi_encoder_disable,
- .mode_set = meson_venc_hdmi_encoder_mode_set,
-};
-
/* DW HDMI Regmap */
static int meson_dw_hdmi_reg_read(void *context, unsigned int reg,
@@ -876,28 +624,6 @@ static const struct meson_dw_hdmi_data meson_dw_hdmi_g12a_data = {
.dwc_write = dw_hdmi_g12a_dwc_write,
};
-static bool meson_hdmi_connector_is_available(struct device *dev)
-{
- struct device_node *ep, *remote;
-
- /* HDMI Connector is on the second port, first endpoint */
- ep = of_graph_get_endpoint_by_regs(dev->of_node, 1, 0);
- if (!ep)
- return false;
-
- /* If the endpoint node exists, consider it enabled */
- remote = of_graph_get_remote_port(ep);
- if (remote) {
- of_node_put(ep);
- return true;
- }
-
- of_node_put(ep);
- of_node_put(remote);
-
- return false;
-}
-
static void meson_dw_hdmi_init(struct meson_dw_hdmi *meson_dw_hdmi)
{
struct meson_drm *priv = meson_dw_hdmi->priv;
@@ -976,18 +702,11 @@ static int meson_dw_hdmi_bind(struct device *dev, struct device *master,
struct drm_device *drm = data;
struct meson_drm *priv = drm->dev_private;
struct dw_hdmi_plat_data *dw_plat_data;
- struct drm_bridge *next_bridge;
- struct drm_encoder *encoder;
int irq;
int ret;
DRM_DEBUG_DRIVER("\n");
- if (!meson_hdmi_connector_is_available(dev)) {
- dev_info(drm->dev, "HDMI Output connector not available\n");
- return -ENODEV;
- }
-
match = of_device_get_match_data(&pdev->dev);
if (!match) {
dev_err(&pdev->dev, "failed to get match data\n");
@@ -1003,7 +722,6 @@ static int meson_dw_hdmi_bind(struct device *dev, struct device *master,
meson_dw_hdmi->dev = dev;
meson_dw_hdmi->data = match;
dw_plat_data = &meson_dw_hdmi->dw_plat_data;
- encoder = &meson_dw_hdmi->encoder;
meson_dw_hdmi->hdmi_supply = devm_regulator_get_optional(dev, "hdmi");
if (IS_ERR(meson_dw_hdmi->hdmi_supply)) {
@@ -1074,34 +792,18 @@ static int meson_dw_hdmi_bind(struct device *dev, struct device *master,
return ret;
}
- /* Encoder */
-
- ret = drm_encoder_init(drm, encoder, &meson_venc_hdmi_encoder_funcs,
- DRM_MODE_ENCODER_TMDS, "meson_hdmi");
- if (ret) {
- dev_err(priv->dev, "Failed to init HDMI encoder\n");
- return ret;
- }
-
- meson_dw_hdmi->bridge.funcs = &meson_venc_hdmi_encoder_bridge_funcs;
- drm_bridge_attach(encoder, &meson_dw_hdmi->bridge, NULL, 0);
-
- encoder->possible_crtcs = BIT(0);
-
meson_dw_hdmi_init(meson_dw_hdmi);
- DRM_DEBUG_DRIVER("encoder initialized\n");
-
/* Bridge / Connector */
dw_plat_data->priv_data = meson_dw_hdmi;
- dw_plat_data->mode_valid = dw_hdmi_mode_valid;
dw_plat_data->phy_ops = &meson_dw_hdmi_phy_ops;
dw_plat_data->phy_name = "meson_dw_hdmi_phy";
dw_plat_data->phy_data = meson_dw_hdmi;
dw_plat_data->input_bus_encoding = V4L2_YCBCR_ENC_709;
dw_plat_data->ycbcr_420_allowed = true;
dw_plat_data->disable_cec = true;
+ dw_plat_data->output_port = 1;
if (dw_hdmi_is_compatible(meson_dw_hdmi, "amlogic,meson-gxl-dw-hdmi") ||
dw_hdmi_is_compatible(meson_dw_hdmi, "amlogic,meson-gxm-dw-hdmi") ||
@@ -1110,15 +812,11 @@ static int meson_dw_hdmi_bind(struct device *dev, struct device *master,
platform_set_drvdata(pdev, meson_dw_hdmi);
- meson_dw_hdmi->hdmi = dw_hdmi_probe(pdev,
- &meson_dw_hdmi->dw_plat_data);
+ meson_dw_hdmi->hdmi = dw_hdmi_probe(pdev, &meson_dw_hdmi->dw_plat_data);
if (IS_ERR(meson_dw_hdmi->hdmi))
return PTR_ERR(meson_dw_hdmi->hdmi);
- next_bridge = of_drm_find_bridge(pdev->dev.of_node);
- if (next_bridge)
- drm_bridge_attach(encoder, next_bridge,
- &meson_dw_hdmi->bridge, 0);
+ meson_dw_hdmi->bridge = of_drm_find_bridge(pdev->dev.of_node);
DRM_DEBUG_DRIVER("HDMI controller initialized\n");
diff --git a/drivers/gpu/drm/meson/meson_encoder_cvbs.c b/drivers/gpu/drm/meson/meson_encoder_cvbs.c
new file mode 100644
index 0000000..fd8db97
--- /dev/null
+++ b/drivers/gpu/drm/meson/meson_encoder_cvbs.c
@@ -0,0 +1,284 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (C) 2016 BayLibre, SAS
+ * Author: Neil Armstrong <narmstrong@baylibre.com>
+ * Copyright (C) 2015 Amlogic, Inc. All rights reserved.
+ * Copyright (C) 2014 Endless Mobile
+ *
+ * Written by:
+ * Jasper St. Pierre <jstpierre@mecheye.net>
+ */
+
+#include <linux/export.h>
+#include <linux/of_graph.h>
+
+#include <drm/drm_atomic_helper.h>
+#include <drm/drm_bridge.h>
+#include <drm/drm_bridge_connector.h>
+#include <drm/drm_device.h>
+#include <drm/drm_edid.h>
+#include <drm/drm_probe_helper.h>
+#include <drm/drm_simple_kms_helper.h>
+
+#include "meson_registers.h"
+#include "meson_vclk.h"
+#include "meson_encoder_cvbs.h"
+
+/* HHI VDAC Registers */
+#define HHI_VDAC_CNTL0 0x2F4 /* 0xbd offset in data sheet */
+#define HHI_VDAC_CNTL0_G12A 0x2EC /* 0xbd offset in data sheet */
+#define HHI_VDAC_CNTL1 0x2F8 /* 0xbe offset in data sheet */
+#define HHI_VDAC_CNTL1_G12A 0x2F0 /* 0xbe offset in data sheet */
+
+struct meson_encoder_cvbs {
+ struct drm_encoder encoder;
+ struct drm_bridge bridge;
+ struct drm_bridge *next_bridge;
+ struct meson_drm *priv;
+};
+
+#define bridge_to_meson_encoder_cvbs(x) \
+ container_of(x, struct meson_encoder_cvbs, bridge)
+
+/* Supported Modes */
+
+struct meson_cvbs_mode meson_cvbs_modes[MESON_CVBS_MODES_COUNT] = {
+ { /* PAL */
+ .enci = &meson_cvbs_enci_pal,
+ .mode = {
+ DRM_MODE("720x576i", DRM_MODE_TYPE_DRIVER, 13500,
+ 720, 732, 795, 864, 0, 576, 580, 586, 625, 0,
+ DRM_MODE_FLAG_INTERLACE),
+ .picture_aspect_ratio = HDMI_PICTURE_ASPECT_4_3,
+ },
+ },
+ { /* NTSC */
+ .enci = &meson_cvbs_enci_ntsc,
+ .mode = {
+ DRM_MODE("720x480i", DRM_MODE_TYPE_DRIVER, 13500,
+ 720, 739, 801, 858, 0, 480, 488, 494, 525, 0,
+ DRM_MODE_FLAG_INTERLACE),
+ .picture_aspect_ratio = HDMI_PICTURE_ASPECT_4_3,
+ },
+ },
+};
+
+static const struct meson_cvbs_mode *
+meson_cvbs_get_mode(const struct drm_display_mode *req_mode)
+{
+ int i;
+
+ for (i = 0; i < MESON_CVBS_MODES_COUNT; ++i) {
+ struct meson_cvbs_mode *meson_mode = &meson_cvbs_modes[i];
+
+ if (drm_mode_match(req_mode, &meson_mode->mode,
+ DRM_MODE_MATCH_TIMINGS |
+ DRM_MODE_MATCH_CLOCK |
+ DRM_MODE_MATCH_FLAGS |
+ DRM_MODE_MATCH_3D_FLAGS))
+ return meson_mode;
+ }
+
+ return NULL;
+}
+
+static int meson_encoder_cvbs_attach(struct drm_bridge *bridge,
+ enum drm_bridge_attach_flags flags)
+{
+ struct meson_encoder_cvbs *meson_encoder_cvbs =
+ bridge_to_meson_encoder_cvbs(bridge);
+
+ return drm_bridge_attach(bridge->encoder, meson_encoder_cvbs->next_bridge,
+ &meson_encoder_cvbs->bridge, flags);
+}
+
+static int meson_encoder_cvbs_get_modes(struct drm_bridge *bridge,
+ struct drm_connector *connector)
+{
+ struct meson_encoder_cvbs *meson_encoder_cvbs =
+ bridge_to_meson_encoder_cvbs(bridge);
+ struct meson_drm *priv = meson_encoder_cvbs->priv;
+ struct drm_display_mode *mode;
+ int i;
+
+ for (i = 0; i < MESON_CVBS_MODES_COUNT; ++i) {
+ struct meson_cvbs_mode *meson_mode = &meson_cvbs_modes[i];
+
+ mode = drm_mode_duplicate(priv->drm, &meson_mode->mode);
+ if (!mode) {
+ dev_err(priv->dev, "Failed to create a new display mode\n");
+ return 0;
+ }
+
+ drm_mode_probed_add(connector, mode);
+ }
+
+ return i;
+}
+
+static int meson_encoder_cvbs_mode_valid(struct drm_bridge *bridge,
+ const struct drm_display_info *display_info,
+ const struct drm_display_mode *mode)
+{
+ if (meson_cvbs_get_mode(mode))
+ return MODE_OK;
+
+ return MODE_BAD;
+}
+
+static int meson_encoder_cvbs_atomic_check(struct drm_bridge *bridge,
+ struct drm_bridge_state *bridge_state,
+ struct drm_crtc_state *crtc_state,
+ struct drm_connector_state *conn_state)
+{
+ if (meson_cvbs_get_mode(&crtc_state->mode))
+ return 0;
+
+ return -EINVAL;
+}
+
+static void meson_encoder_cvbs_atomic_enable(struct drm_bridge *bridge,
+ struct drm_bridge_state *bridge_state)
+{
+ struct meson_encoder_cvbs *encoder_cvbs = bridge_to_meson_encoder_cvbs(bridge);
+ struct drm_atomic_state *state = bridge_state->base.state;
+ struct meson_drm *priv = encoder_cvbs->priv;
+ const struct meson_cvbs_mode *meson_mode;
+ struct drm_connector_state *conn_state;
+ struct drm_crtc_state *crtc_state;
+ struct drm_connector *connector;
+
+ connector = drm_atomic_get_new_connector_for_encoder(state, bridge->encoder);
+ if (WARN_ON(!connector))
+ return;
+
+ conn_state = drm_atomic_get_new_connector_state(state, connector);
+ if (WARN_ON(!conn_state))
+ return;
+
+ crtc_state = drm_atomic_get_new_crtc_state(state, conn_state->crtc);
+ if (WARN_ON(!crtc_state))
+ return;
+
+ meson_mode = meson_cvbs_get_mode(&crtc_state->adjusted_mode);
+ if (WARN_ON(!meson_mode))
+ return;
+
+ meson_venci_cvbs_mode_set(priv, meson_mode->enci);
+
+ /* Setup 27MHz vclk2 for ENCI and VDAC */
+ meson_vclk_setup(priv, MESON_VCLK_TARGET_CVBS,
+ MESON_VCLK_CVBS, MESON_VCLK_CVBS,
+ MESON_VCLK_CVBS, MESON_VCLK_CVBS,
+ true);
+
+ /* VDAC0 source is not from ATV */
+ writel_bits_relaxed(VENC_VDAC_SEL_ATV_DMD, 0,
+ priv->io_base + _REG(VENC_VDAC_DACSEL0));
+
+ if (meson_vpu_is_compatible(priv, VPU_COMPATIBLE_GXBB)) {
+ regmap_write(priv->hhi, HHI_VDAC_CNTL0, 1);
+ regmap_write(priv->hhi, HHI_VDAC_CNTL1, 0);
+ } else if (meson_vpu_is_compatible(priv, VPU_COMPATIBLE_GXM) ||
+ meson_vpu_is_compatible(priv, VPU_COMPATIBLE_GXL)) {
+ regmap_write(priv->hhi, HHI_VDAC_CNTL0, 0xf0001);
+ regmap_write(priv->hhi, HHI_VDAC_CNTL1, 0);
+ } else if (meson_vpu_is_compatible(priv, VPU_COMPATIBLE_G12A)) {
+ regmap_write(priv->hhi, HHI_VDAC_CNTL0_G12A, 0x906001);
+ regmap_write(priv->hhi, HHI_VDAC_CNTL1_G12A, 0);
+ }
+}
+
+static void meson_encoder_cvbs_atomic_disable(struct drm_bridge *bridge,
+ struct drm_bridge_state *bridge_state)
+{
+ struct meson_encoder_cvbs *meson_encoder_cvbs =
+ bridge_to_meson_encoder_cvbs(bridge);
+ struct meson_drm *priv = meson_encoder_cvbs->priv;
+
+ /* Disable CVBS VDAC */
+ if (meson_vpu_is_compatible(priv, VPU_COMPATIBLE_G12A)) {
+ regmap_write(priv->hhi, HHI_VDAC_CNTL0_G12A, 0);
+ regmap_write(priv->hhi, HHI_VDAC_CNTL1_G12A, 0);
+ } else {
+ regmap_write(priv->hhi, HHI_VDAC_CNTL0, 0);
+ regmap_write(priv->hhi, HHI_VDAC_CNTL1, 8);
+ }
+}
+
+static const struct drm_bridge_funcs meson_encoder_cvbs_bridge_funcs = {
+ .attach = meson_encoder_cvbs_attach,
+ .mode_valid = meson_encoder_cvbs_mode_valid,
+ .get_modes = meson_encoder_cvbs_get_modes,
+ .atomic_enable = meson_encoder_cvbs_atomic_enable,
+ .atomic_disable = meson_encoder_cvbs_atomic_disable,
+ .atomic_check = meson_encoder_cvbs_atomic_check,
+ .atomic_duplicate_state = drm_atomic_helper_bridge_duplicate_state,
+ .atomic_destroy_state = drm_atomic_helper_bridge_destroy_state,
+ .atomic_reset = drm_atomic_helper_bridge_reset,
+};
+
+int meson_encoder_cvbs_init(struct meson_drm *priv)
+{
+ struct drm_device *drm = priv->drm;
+ struct meson_encoder_cvbs *meson_encoder_cvbs;
+ struct drm_connector *connector;
+ struct device_node *remote;
+ int ret;
+
+ meson_encoder_cvbs = devm_kzalloc(priv->dev, sizeof(*meson_encoder_cvbs), GFP_KERNEL);
+ if (!meson_encoder_cvbs)
+ return -ENOMEM;
+
+ /* CVBS Connector Bridge */
+ remote = of_graph_get_remote_node(priv->dev->of_node, 0, 0);
+ if (!remote) {
+ dev_info(drm->dev, "CVBS Output connector not available\n");
+ return 0;
+ }
+
+ meson_encoder_cvbs->next_bridge = of_drm_find_bridge(remote);
+ if (!meson_encoder_cvbs->next_bridge) {
+ dev_err(priv->dev, "Failed to find CVBS Connector bridge\n");
+ return -EPROBE_DEFER;
+ }
+
+ /* CVBS Encoder Bridge */
+ meson_encoder_cvbs->bridge.funcs = &meson_encoder_cvbs_bridge_funcs;
+ meson_encoder_cvbs->bridge.of_node = priv->dev->of_node;
+ meson_encoder_cvbs->bridge.type = DRM_MODE_CONNECTOR_Composite;
+ meson_encoder_cvbs->bridge.ops = DRM_BRIDGE_OP_MODES;
+ meson_encoder_cvbs->bridge.interlace_allowed = true;
+
+ drm_bridge_add(&meson_encoder_cvbs->bridge);
+
+ meson_encoder_cvbs->priv = priv;
+
+ /* Encoder */
+ ret = drm_simple_encoder_init(priv->drm, &meson_encoder_cvbs->encoder,
+ DRM_MODE_ENCODER_TVDAC);
+ if (ret) {
+ dev_err(priv->dev, "Failed to init CVBS encoder: %d\n", ret);
+ return ret;
+ }
+
+ meson_encoder_cvbs->encoder.possible_crtcs = BIT(0);
+
+ /* Attach CVBS Encoder Bridge to Encoder */
+ ret = drm_bridge_attach(&meson_encoder_cvbs->encoder, &meson_encoder_cvbs->bridge, NULL,
+ DRM_BRIDGE_ATTACH_NO_CONNECTOR);
+ if (ret) {
+ dev_err(priv->dev, "Failed to attach bridge: %d\n", ret);
+ return ret;
+ }
+
+ /* Initialize & attach Bridge Connector */
+ connector = drm_bridge_connector_init(priv->drm, &meson_encoder_cvbs->encoder);
+ if (IS_ERR(connector)) {
+ dev_err(priv->dev, "Unable to create CVBS bridge connector\n");
+ return PTR_ERR(connector);
+ }
+ drm_connector_attach_encoder(connector, &meson_encoder_cvbs->encoder);
+
+ return 0;
+}
diff --git a/drivers/gpu/drm/meson/meson_encoder_cvbs.h b/drivers/gpu/drm/meson/meson_encoder_cvbs.h
new file mode 100644
index 0000000..61d9d18
--- /dev/null
+++ b/drivers/gpu/drm/meson/meson_encoder_cvbs.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright (C) 2016 BayLibre, SAS
+ * Author: Neil Armstrong <narmstrong@baylibre.com>
+ * Copyright (C) 2014 Endless Mobile
+ *
+ * Written by:
+ * Jasper St. Pierre <jstpierre@mecheye.net>
+ */
+
+#ifndef __MESON_VENC_CVBS_H
+#define __MESON_VENC_CVBS_H
+
+#include "meson_drv.h"
+#include "meson_venc.h"
+
+struct meson_cvbs_mode {
+ struct meson_cvbs_enci_mode *enci;
+ struct drm_display_mode mode;
+};
+
+#define MESON_CVBS_MODES_COUNT 2
+
+/* Modes supported by the CVBS output */
+extern struct meson_cvbs_mode meson_cvbs_modes[MESON_CVBS_MODES_COUNT];
+
+int meson_encoder_cvbs_init(struct meson_drm *priv);
+
+#endif /* __MESON_VENC_CVBS_H */
diff --git a/drivers/gpu/drm/meson/meson_encoder_hdmi.c b/drivers/gpu/drm/meson/meson_encoder_hdmi.c
new file mode 100644
index 0000000..5e306de
--- /dev/null
+++ b/drivers/gpu/drm/meson/meson_encoder_hdmi.c
@@ -0,0 +1,447 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (C) 2016 BayLibre, SAS
+ * Author: Neil Armstrong <narmstrong@baylibre.com>
+ * Copyright (C) 2015 Amlogic, Inc. All rights reserved.
+ */
+
+#include <linux/clk.h>
+#include <linux/component.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/of_device.h>
+#include <linux/of_graph.h>
+#include <linux/regulator/consumer.h>
+#include <linux/reset.h>
+
+#include <media/cec-notifier.h>
+
+#include <drm/drm_atomic_helper.h>
+#include <drm/drm_bridge.h>
+#include <drm/drm_bridge_connector.h>
+#include <drm/drm_device.h>
+#include <drm/drm_edid.h>
+#include <drm/drm_probe_helper.h>
+#include <drm/drm_simple_kms_helper.h>
+
+#include <linux/media-bus-format.h>
+#include <linux/videodev2.h>
+
+#include "meson_drv.h"
+#include "meson_registers.h"
+#include "meson_vclk.h"
+#include "meson_venc.h"
+#include "meson_encoder_hdmi.h"
+
+struct meson_encoder_hdmi {
+ struct drm_encoder encoder;
+ struct drm_bridge bridge;
+ struct drm_bridge *next_bridge;
+ struct drm_connector *connector;
+ struct meson_drm *priv;
+ unsigned long output_bus_fmt;
+ struct cec_notifier *cec_notifier;
+};
+
+#define bridge_to_meson_encoder_hdmi(x) \
+ container_of(x, struct meson_encoder_hdmi, bridge)
+
+static int meson_encoder_hdmi_attach(struct drm_bridge *bridge,
+ enum drm_bridge_attach_flags flags)
+{
+ struct meson_encoder_hdmi *encoder_hdmi = bridge_to_meson_encoder_hdmi(bridge);
+
+ return drm_bridge_attach(bridge->encoder, encoder_hdmi->next_bridge,
+ &encoder_hdmi->bridge, flags);
+}
+
+static void meson_encoder_hdmi_detach(struct drm_bridge *bridge)
+{
+ struct meson_encoder_hdmi *encoder_hdmi = bridge_to_meson_encoder_hdmi(bridge);
+
+ cec_notifier_conn_unregister(encoder_hdmi->cec_notifier);
+ encoder_hdmi->cec_notifier = NULL;
+}
+
+static void meson_encoder_hdmi_set_vclk(struct meson_encoder_hdmi *encoder_hdmi,
+ const struct drm_display_mode *mode)
+{
+ struct meson_drm *priv = encoder_hdmi->priv;
+ int vic = drm_match_cea_mode(mode);
+ unsigned int phy_freq;
+ unsigned int vclk_freq;
+ unsigned int venc_freq;
+ unsigned int hdmi_freq;
+
+ vclk_freq = mode->clock;
+
+ /* For 420, pixel clock is half unlike venc clock */
+ if (encoder_hdmi->output_bus_fmt == MEDIA_BUS_FMT_UYYVYY8_0_5X24)
+ vclk_freq /= 2;
+
+ /* TMDS clock is pixel_clock * 10 */
+ phy_freq = vclk_freq * 10;
+
+ if (!vic) {
+ meson_vclk_setup(priv, MESON_VCLK_TARGET_DMT, phy_freq,
+ vclk_freq, vclk_freq, vclk_freq, false);
+ return;
+ }
+
+ /* 480i/576i needs global pixel doubling */
+ if (mode->flags & DRM_MODE_FLAG_DBLCLK)
+ vclk_freq *= 2;
+
+ venc_freq = vclk_freq;
+ hdmi_freq = vclk_freq;
+
+ /* VENC double pixels for 1080i, 720p and YUV420 modes */
+ if (meson_venc_hdmi_venc_repeat(vic) ||
+ encoder_hdmi->output_bus_fmt == MEDIA_BUS_FMT_UYYVYY8_0_5X24)
+ venc_freq *= 2;
+
+ vclk_freq = max(venc_freq, hdmi_freq);
+
+ if (mode->flags & DRM_MODE_FLAG_DBLCLK)
+ venc_freq /= 2;
+
+ dev_dbg(priv->dev, "vclk:%d phy=%d venc=%d hdmi=%d enci=%d\n",
+ phy_freq, vclk_freq, venc_freq, hdmi_freq,
+ priv->venc.hdmi_use_enci);
+
+ meson_vclk_setup(priv, MESON_VCLK_TARGET_HDMI, phy_freq, vclk_freq,
+ venc_freq, hdmi_freq, priv->venc.hdmi_use_enci);
+}
+
+static enum drm_mode_status meson_encoder_hdmi_mode_valid(struct drm_bridge *bridge,
+ const struct drm_display_info *display_info,
+ const struct drm_display_mode *mode)
+{
+ struct meson_encoder_hdmi *encoder_hdmi = bridge_to_meson_encoder_hdmi(bridge);
+ struct meson_drm *priv = encoder_hdmi->priv;
+ bool is_hdmi2_sink = display_info->hdmi.scdc.supported;
+ unsigned int phy_freq;
+ unsigned int vclk_freq;
+ unsigned int venc_freq;
+ unsigned int hdmi_freq;
+ int vic = drm_match_cea_mode(mode);
+ enum drm_mode_status status;
+
+ dev_dbg(priv->dev, "Modeline " DRM_MODE_FMT "\n", DRM_MODE_ARG(mode));
+
+ /* If sink does not support 540MHz, reject the non-420 HDMI2 modes */
+ if (display_info->max_tmds_clock &&
+ mode->clock > display_info->max_tmds_clock &&
+ !drm_mode_is_420_only(display_info, mode) &&
+ !drm_mode_is_420_also(display_info, mode))
+ return MODE_BAD;
+
+ /* Check against non-VIC supported modes */
+ if (!vic) {
+ status = meson_venc_hdmi_supported_mode(mode);
+ if (status != MODE_OK)
+ return status;
+
+ return meson_vclk_dmt_supported_freq(priv, mode->clock);
+ /* Check against supported VIC modes */
+ } else if (!meson_venc_hdmi_supported_vic(vic))
+ return MODE_BAD;
+
+ vclk_freq = mode->clock;
+
+ /* For 420, pixel clock is half unlike venc clock */
+ if (drm_mode_is_420_only(display_info, mode) ||
+ (!is_hdmi2_sink &&
+ drm_mode_is_420_also(display_info, mode)))
+ vclk_freq /= 2;
+
+ /* TMDS clock is pixel_clock * 10 */
+ phy_freq = vclk_freq * 10;
+
+ /* 480i/576i needs global pixel doubling */
+ if (mode->flags & DRM_MODE_FLAG_DBLCLK)
+ vclk_freq *= 2;
+
+ venc_freq = vclk_freq;
+ hdmi_freq = vclk_freq;
+
+ /* VENC double pixels for 1080i, 720p and YUV420 modes */
+ if (meson_venc_hdmi_venc_repeat(vic) ||
+ drm_mode_is_420_only(display_info, mode) ||
+ (!is_hdmi2_sink &&
+ drm_mode_is_420_also(display_info, mode)))
+ venc_freq *= 2;
+
+ vclk_freq = max(venc_freq, hdmi_freq);
+
+ if (mode->flags & DRM_MODE_FLAG_DBLCLK)
+ venc_freq /= 2;
+
+ dev_dbg(priv->dev, "%s: vclk:%d phy=%d venc=%d hdmi=%d\n",
+ __func__, phy_freq, vclk_freq, venc_freq, hdmi_freq);
+
+ return meson_vclk_vic_supported_freq(priv, phy_freq, vclk_freq);
+}
+
+static void meson_encoder_hdmi_atomic_enable(struct drm_bridge *bridge,
+ struct drm_bridge_state *bridge_state)
+{
+ struct meson_encoder_hdmi *encoder_hdmi = bridge_to_meson_encoder_hdmi(bridge);
+ struct drm_atomic_state *state = bridge_state->base.state;
+ unsigned int ycrcb_map = VPU_HDMI_OUTPUT_CBYCR;
+ struct meson_drm *priv = encoder_hdmi->priv;
+ struct drm_connector_state *conn_state;
+ const struct drm_display_mode *mode;
+ struct drm_crtc_state *crtc_state;
+ struct drm_connector *connector;
+ bool yuv420_mode = false;
+ int vic;
+
+ connector = drm_atomic_get_new_connector_for_encoder(state, bridge->encoder);
+ if (WARN_ON(!connector))
+ return;
+
+ conn_state = drm_atomic_get_new_connector_state(state, connector);
+ if (WARN_ON(!conn_state))
+ return;
+
+ crtc_state = drm_atomic_get_new_crtc_state(state, conn_state->crtc);
+ if (WARN_ON(!crtc_state))
+ return;
+
+ mode = &crtc_state->adjusted_mode;
+
+ vic = drm_match_cea_mode(mode);
+
+ dev_dbg(priv->dev, "\"%s\" vic %d\n", mode->name, vic);
+
+ if (encoder_hdmi->output_bus_fmt == MEDIA_BUS_FMT_UYYVYY8_0_5X24) {
+ ycrcb_map = VPU_HDMI_OUTPUT_CRYCB;
+ yuv420_mode = true;
+ }
+
+ /* VENC + VENC-DVI Mode setup */
+ meson_venc_hdmi_mode_set(priv, vic, ycrcb_map, yuv420_mode, mode);
+
+ /* VCLK Set clock */
+ meson_encoder_hdmi_set_vclk(encoder_hdmi, mode);
+
+ if (encoder_hdmi->output_bus_fmt == MEDIA_BUS_FMT_UYYVYY8_0_5X24)
+ /* Setup YUV420 to HDMI-TX, no 10bit diphering */
+ writel_relaxed(2 | (2 << 2),
+ priv->io_base + _REG(VPU_HDMI_FMT_CTRL));
+ else
+ /* Setup YUV444 to HDMI-TX, no 10bit diphering */
+ writel_relaxed(0, priv->io_base + _REG(VPU_HDMI_FMT_CTRL));
+
+ dev_dbg(priv->dev, "%s\n", priv->venc.hdmi_use_enci ? "VENCI" : "VENCP");
+
+ if (priv->venc.hdmi_use_enci)
+ writel_relaxed(1, priv->io_base + _REG(ENCI_VIDEO_EN));
+ else
+ writel_relaxed(1, priv->io_base + _REG(ENCP_VIDEO_EN));
+}
+
+static void meson_encoder_hdmi_atomic_disable(struct drm_bridge *bridge,
+ struct drm_bridge_state *bridge_state)
+{
+ struct meson_encoder_hdmi *encoder_hdmi = bridge_to_meson_encoder_hdmi(bridge);
+ struct meson_drm *priv = encoder_hdmi->priv;
+
+ writel_bits_relaxed(0x3, 0,
+ priv->io_base + _REG(VPU_HDMI_SETTING));
+
+ writel_relaxed(0, priv->io_base + _REG(ENCI_VIDEO_EN));
+ writel_relaxed(0, priv->io_base + _REG(ENCP_VIDEO_EN));
+}
+
+static const u32 meson_encoder_hdmi_out_bus_fmts[] = {
+ MEDIA_BUS_FMT_YUV8_1X24,
+ MEDIA_BUS_FMT_UYYVYY8_0_5X24,
+};
+
+static u32 *
+meson_encoder_hdmi_get_inp_bus_fmts(struct drm_bridge *bridge,
+ struct drm_bridge_state *bridge_state,
+ struct drm_crtc_state *crtc_state,
+ struct drm_connector_state *conn_state,
+ u32 output_fmt,
+ unsigned int *num_input_fmts)
+{
+ u32 *input_fmts = NULL;
+ int i;
+
+ *num_input_fmts = 0;
+
+ for (i = 0 ; i < ARRAY_SIZE(meson_encoder_hdmi_out_bus_fmts) ; ++i) {
+ if (output_fmt == meson_encoder_hdmi_out_bus_fmts[i]) {
+ *num_input_fmts = 1;
+ input_fmts = kcalloc(*num_input_fmts,
+ sizeof(*input_fmts),
+ GFP_KERNEL);
+ if (!input_fmts)
+ return NULL;
+
+ input_fmts[0] = output_fmt;
+
+ break;
+ }
+ }
+
+ return input_fmts;
+}
+
+static int meson_encoder_hdmi_atomic_check(struct drm_bridge *bridge,
+ struct drm_bridge_state *bridge_state,
+ struct drm_crtc_state *crtc_state,
+ struct drm_connector_state *conn_state)
+{
+ struct meson_encoder_hdmi *encoder_hdmi = bridge_to_meson_encoder_hdmi(bridge);
+ struct drm_connector_state *old_conn_state =
+ drm_atomic_get_old_connector_state(conn_state->state, conn_state->connector);
+ struct meson_drm *priv = encoder_hdmi->priv;
+
+ encoder_hdmi->output_bus_fmt = bridge_state->output_bus_cfg.format;
+
+ dev_dbg(priv->dev, "output_bus_fmt %lx\n", encoder_hdmi->output_bus_fmt);
+
+ if (!drm_connector_atomic_hdr_metadata_equal(old_conn_state, conn_state))
+ crtc_state->mode_changed = true;
+
+ return 0;
+}
+
+static void meson_encoder_hdmi_hpd_notify(struct drm_bridge *bridge,
+ enum drm_connector_status status)
+{
+ struct meson_encoder_hdmi *encoder_hdmi = bridge_to_meson_encoder_hdmi(bridge);
+ struct edid *edid;
+
+ if (!encoder_hdmi->cec_notifier)
+ return;
+
+ if (status == connector_status_connected) {
+ edid = drm_bridge_get_edid(encoder_hdmi->next_bridge, encoder_hdmi->connector);
+ if (!edid)
+ return;
+
+ cec_notifier_set_phys_addr_from_edid(encoder_hdmi->cec_notifier, edid);
+ } else
+ cec_notifier_phys_addr_invalidate(encoder_hdmi->cec_notifier);
+}
+
+static const struct drm_bridge_funcs meson_encoder_hdmi_bridge_funcs = {
+ .attach = meson_encoder_hdmi_attach,
+ .detach = meson_encoder_hdmi_detach,
+ .mode_valid = meson_encoder_hdmi_mode_valid,
+ .hpd_notify = meson_encoder_hdmi_hpd_notify,
+ .atomic_enable = meson_encoder_hdmi_atomic_enable,
+ .atomic_disable = meson_encoder_hdmi_atomic_disable,
+ .atomic_get_input_bus_fmts = meson_encoder_hdmi_get_inp_bus_fmts,
+ .atomic_check = meson_encoder_hdmi_atomic_check,
+ .atomic_duplicate_state = drm_atomic_helper_bridge_duplicate_state,
+ .atomic_destroy_state = drm_atomic_helper_bridge_destroy_state,
+ .atomic_reset = drm_atomic_helper_bridge_reset,
+};
+
+int meson_encoder_hdmi_init(struct meson_drm *priv)
+{
+ struct meson_encoder_hdmi *meson_encoder_hdmi;
+ struct platform_device *pdev;
+ struct device_node *remote;
+ int ret;
+
+ meson_encoder_hdmi = devm_kzalloc(priv->dev, sizeof(*meson_encoder_hdmi), GFP_KERNEL);
+ if (!meson_encoder_hdmi)
+ return -ENOMEM;
+
+ /* HDMI Transceiver Bridge */
+ remote = of_graph_get_remote_node(priv->dev->of_node, 1, 0);
+ if (!remote) {
+ dev_err(priv->dev, "HDMI transceiver device is disabled");
+ return 0;
+ }
+
+ meson_encoder_hdmi->next_bridge = of_drm_find_bridge(remote);
+ if (!meson_encoder_hdmi->next_bridge) {
+ dev_err(priv->dev, "Failed to find HDMI transceiver bridge\n");
+ return -EPROBE_DEFER;
+ }
+
+ /* HDMI Encoder Bridge */
+ meson_encoder_hdmi->bridge.funcs = &meson_encoder_hdmi_bridge_funcs;
+ meson_encoder_hdmi->bridge.of_node = priv->dev->of_node;
+ meson_encoder_hdmi->bridge.type = DRM_MODE_CONNECTOR_HDMIA;
+ meson_encoder_hdmi->bridge.interlace_allowed = true;
+
+ drm_bridge_add(&meson_encoder_hdmi->bridge);
+
+ meson_encoder_hdmi->priv = priv;
+
+ /* Encoder */
+ ret = drm_simple_encoder_init(priv->drm, &meson_encoder_hdmi->encoder,
+ DRM_MODE_ENCODER_TMDS);
+ if (ret) {
+ dev_err(priv->dev, "Failed to init HDMI encoder: %d\n", ret);
+ return ret;
+ }
+
+ meson_encoder_hdmi->encoder.possible_crtcs = BIT(0);
+
+ /* Attach HDMI Encoder Bridge to Encoder */
+ ret = drm_bridge_attach(&meson_encoder_hdmi->encoder, &meson_encoder_hdmi->bridge, NULL,
+ DRM_BRIDGE_ATTACH_NO_CONNECTOR);
+ if (ret) {
+ dev_err(priv->dev, "Failed to attach bridge: %d\n", ret);
+ return ret;
+ }
+
+ /* Initialize & attach Bridge Connector */
+ meson_encoder_hdmi->connector = drm_bridge_connector_init(priv->drm,
+ &meson_encoder_hdmi->encoder);
+ if (IS_ERR(meson_encoder_hdmi->connector)) {
+ dev_err(priv->dev, "Unable to create HDMI bridge connector\n");
+ return PTR_ERR(meson_encoder_hdmi->connector);
+ }
+ drm_connector_attach_encoder(meson_encoder_hdmi->connector,
+ &meson_encoder_hdmi->encoder);
+
+ /*
+ * We should have now in place:
+ * encoder->[hdmi encoder bridge]->[dw-hdmi bridge]->[display connector bridge]->[display connector]
+ */
+
+ /*
+ * drm_connector_attach_max_bpc_property() requires the
+ * connector to have a state.
+ */
+ drm_atomic_helper_connector_reset(meson_encoder_hdmi->connector);
+
+ if (meson_vpu_is_compatible(priv, VPU_COMPATIBLE_GXL) ||
+ meson_vpu_is_compatible(priv, VPU_COMPATIBLE_GXM) ||
+ meson_vpu_is_compatible(priv, VPU_COMPATIBLE_G12A))
+ drm_connector_attach_hdr_output_metadata_property(meson_encoder_hdmi->connector);
+
+ drm_connector_attach_max_bpc_property(meson_encoder_hdmi->connector, 8, 8);
+
+ /* Handle this here until handled by drm_bridge_connector_init() */
+ meson_encoder_hdmi->connector->ycbcr_420_allowed = true;
+
+ pdev = of_find_device_by_node(remote);
+ if (pdev) {
+ struct cec_connector_info conn_info;
+ struct cec_notifier *notifier;
+
+ cec_fill_conn_info_from_drm(&conn_info, meson_encoder_hdmi->connector);
+
+ notifier = cec_notifier_conn_register(&pdev->dev, NULL, &conn_info);
+ if (!notifier)
+ return -ENOMEM;
+
+ meson_encoder_hdmi->cec_notifier = notifier;
+ }
+
+ dev_dbg(priv->dev, "HDMI encoder initialized\n");
+
+ return 0;
+}
diff --git a/drivers/gpu/drm/meson/meson_encoder_hdmi.h b/drivers/gpu/drm/meson/meson_encoder_hdmi.h
new file mode 100644
index 0000000..ed19494
--- /dev/null
+++ b/drivers/gpu/drm/meson/meson_encoder_hdmi.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright (C) 2021 BayLibre, SAS
+ * Author: Neil Armstrong <narmstrong@baylibre.com>
+ */
+
+#ifndef __MESON_ENCODER_HDMI_H
+#define __MESON_ENCODER_HDMI_H
+
+int meson_encoder_hdmi_init(struct meson_drm *priv);
+
+#endif /* __MESON_ENCODER_HDMI_H */
diff --git a/drivers/gpu/drm/meson/meson_venc_cvbs.c b/drivers/gpu/drm/meson/meson_venc_cvbs.c
deleted file mode 100644
index f1747fde..0000000
--- a/drivers/gpu/drm/meson/meson_venc_cvbs.c
+++ /dev/null
@@ -1,293 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-or-later
-/*
- * Copyright (C) 2016 BayLibre, SAS
- * Author: Neil Armstrong <narmstrong@baylibre.com>
- * Copyright (C) 2015 Amlogic, Inc. All rights reserved.
- * Copyright (C) 2014 Endless Mobile
- *
- * Written by:
- * Jasper St. Pierre <jstpierre@mecheye.net>
- */
-
-#include <linux/export.h>
-#include <linux/of_graph.h>
-
-#include <drm/drm_atomic_helper.h>
-#include <drm/drm_device.h>
-#include <drm/drm_edid.h>
-#include <drm/drm_probe_helper.h>
-#include <drm/drm_print.h>
-
-#include "meson_registers.h"
-#include "meson_vclk.h"
-#include "meson_venc_cvbs.h"
-
-/* HHI VDAC Registers */
-#define HHI_VDAC_CNTL0 0x2F4 /* 0xbd offset in data sheet */
-#define HHI_VDAC_CNTL0_G12A 0x2EC /* 0xbd offset in data sheet */
-#define HHI_VDAC_CNTL1 0x2F8 /* 0xbe offset in data sheet */
-#define HHI_VDAC_CNTL1_G12A 0x2F0 /* 0xbe offset in data sheet */
-
-struct meson_venc_cvbs {
- struct drm_encoder encoder;
- struct drm_connector connector;
- struct meson_drm *priv;
-};
-#define encoder_to_meson_venc_cvbs(x) \
- container_of(x, struct meson_venc_cvbs, encoder)
-
-#define connector_to_meson_venc_cvbs(x) \
- container_of(x, struct meson_venc_cvbs, connector)
-
-/* Supported Modes */
-
-struct meson_cvbs_mode meson_cvbs_modes[MESON_CVBS_MODES_COUNT] = {
- { /* PAL */
- .enci = &meson_cvbs_enci_pal,
- .mode = {
- DRM_MODE("720x576i", DRM_MODE_TYPE_DRIVER, 13500,
- 720, 732, 795, 864, 0, 576, 580, 586, 625, 0,
- DRM_MODE_FLAG_INTERLACE),
- .picture_aspect_ratio = HDMI_PICTURE_ASPECT_4_3,
- },
- },
- { /* NTSC */
- .enci = &meson_cvbs_enci_ntsc,
- .mode = {
- DRM_MODE("720x480i", DRM_MODE_TYPE_DRIVER, 13500,
- 720, 739, 801, 858, 0, 480, 488, 494, 525, 0,
- DRM_MODE_FLAG_INTERLACE),
- .picture_aspect_ratio = HDMI_PICTURE_ASPECT_4_3,
- },
- },
-};
-
-static const struct meson_cvbs_mode *
-meson_cvbs_get_mode(const struct drm_display_mode *req_mode)
-{
- int i;
-
- for (i = 0; i < MESON_CVBS_MODES_COUNT; ++i) {
- struct meson_cvbs_mode *meson_mode = &meson_cvbs_modes[i];
-
- if (drm_mode_match(req_mode, &meson_mode->mode,
- DRM_MODE_MATCH_TIMINGS |
- DRM_MODE_MATCH_CLOCK |
- DRM_MODE_MATCH_FLAGS |
- DRM_MODE_MATCH_3D_FLAGS))
- return meson_mode;
- }
-
- return NULL;
-}
-
-/* Connector */
-
-static void meson_cvbs_connector_destroy(struct drm_connector *connector)
-{
- drm_connector_cleanup(connector);
-}
-
-static enum drm_connector_status
-meson_cvbs_connector_detect(struct drm_connector *connector, bool force)
-{
- /* FIXME: Add load-detect or jack-detect if possible */
- return connector_status_connected;
-}
-
-static int meson_cvbs_connector_get_modes(struct drm_connector *connector)
-{
- struct drm_device *dev = connector->dev;
- struct drm_display_mode *mode;
- int i;
-
- for (i = 0; i < MESON_CVBS_MODES_COUNT; ++i) {
- struct meson_cvbs_mode *meson_mode = &meson_cvbs_modes[i];
-
- mode = drm_mode_duplicate(dev, &meson_mode->mode);
- if (!mode) {
- DRM_ERROR("Failed to create a new display mode\n");
- return 0;
- }
-
- drm_mode_probed_add(connector, mode);
- }
-
- return i;
-}
-
-static int meson_cvbs_connector_mode_valid(struct drm_connector *connector,
- struct drm_display_mode *mode)
-{
- /* Validate the modes added in get_modes */
- return MODE_OK;
-}
-
-static const struct drm_connector_funcs meson_cvbs_connector_funcs = {
- .detect = meson_cvbs_connector_detect,
- .fill_modes = drm_helper_probe_single_connector_modes,
- .destroy = meson_cvbs_connector_destroy,
- .reset = drm_atomic_helper_connector_reset,
- .atomic_duplicate_state = drm_atomic_helper_connector_duplicate_state,
- .atomic_destroy_state = drm_atomic_helper_connector_destroy_state,
-};
-
-static const
-struct drm_connector_helper_funcs meson_cvbs_connector_helper_funcs = {
- .get_modes = meson_cvbs_connector_get_modes,
- .mode_valid = meson_cvbs_connector_mode_valid,
-};
-
-/* Encoder */
-
-static void meson_venc_cvbs_encoder_destroy(struct drm_encoder *encoder)
-{
- drm_encoder_cleanup(encoder);
-}
-
-static const struct drm_encoder_funcs meson_venc_cvbs_encoder_funcs = {
- .destroy = meson_venc_cvbs_encoder_destroy,
-};
-
-static int meson_venc_cvbs_encoder_atomic_check(struct drm_encoder *encoder,
- struct drm_crtc_state *crtc_state,
- struct drm_connector_state *conn_state)
-{
- if (meson_cvbs_get_mode(&crtc_state->mode))
- return 0;
-
- return -EINVAL;
-}
-
-static void meson_venc_cvbs_encoder_disable(struct drm_encoder *encoder)
-{
- struct meson_venc_cvbs *meson_venc_cvbs =
- encoder_to_meson_venc_cvbs(encoder);
- struct meson_drm *priv = meson_venc_cvbs->priv;
-
- /* Disable CVBS VDAC */
- if (meson_vpu_is_compatible(priv, VPU_COMPATIBLE_G12A)) {
- regmap_write(priv->hhi, HHI_VDAC_CNTL0_G12A, 0);
- regmap_write(priv->hhi, HHI_VDAC_CNTL1_G12A, 0);
- } else {
- regmap_write(priv->hhi, HHI_VDAC_CNTL0, 0);
- regmap_write(priv->hhi, HHI_VDAC_CNTL1, 8);
- }
-}
-
-static void meson_venc_cvbs_encoder_enable(struct drm_encoder *encoder)
-{
- struct meson_venc_cvbs *meson_venc_cvbs =
- encoder_to_meson_venc_cvbs(encoder);
- struct meson_drm *priv = meson_venc_cvbs->priv;
-
- /* VDAC0 source is not from ATV */
- writel_bits_relaxed(VENC_VDAC_SEL_ATV_DMD, 0,
- priv->io_base + _REG(VENC_VDAC_DACSEL0));
-
- if (meson_vpu_is_compatible(priv, VPU_COMPATIBLE_GXBB)) {
- regmap_write(priv->hhi, HHI_VDAC_CNTL0, 1);
- regmap_write(priv->hhi, HHI_VDAC_CNTL1, 0);
- } else if (meson_vpu_is_compatible(priv, VPU_COMPATIBLE_GXM) ||
- meson_vpu_is_compatible(priv, VPU_COMPATIBLE_GXL)) {
- regmap_write(priv->hhi, HHI_VDAC_CNTL0, 0xf0001);
- regmap_write(priv->hhi, HHI_VDAC_CNTL1, 0);
- } else if (meson_vpu_is_compatible(priv, VPU_COMPATIBLE_G12A)) {
- regmap_write(priv->hhi, HHI_VDAC_CNTL0_G12A, 0x906001);
- regmap_write(priv->hhi, HHI_VDAC_CNTL1_G12A, 0);
- }
-}
-
-static void meson_venc_cvbs_encoder_mode_set(struct drm_encoder *encoder,
- struct drm_display_mode *mode,
- struct drm_display_mode *adjusted_mode)
-{
- const struct meson_cvbs_mode *meson_mode = meson_cvbs_get_mode(mode);
- struct meson_venc_cvbs *meson_venc_cvbs =
- encoder_to_meson_venc_cvbs(encoder);
- struct meson_drm *priv = meson_venc_cvbs->priv;
-
- if (meson_mode) {
- meson_venci_cvbs_mode_set(priv, meson_mode->enci);
-
- /* Setup 27MHz vclk2 for ENCI and VDAC */
- meson_vclk_setup(priv, MESON_VCLK_TARGET_CVBS,
- MESON_VCLK_CVBS, MESON_VCLK_CVBS,
- MESON_VCLK_CVBS, MESON_VCLK_CVBS,
- true);
- }
-}
-
-static const struct drm_encoder_helper_funcs
- meson_venc_cvbs_encoder_helper_funcs = {
- .atomic_check = meson_venc_cvbs_encoder_atomic_check,
- .disable = meson_venc_cvbs_encoder_disable,
- .enable = meson_venc_cvbs_encoder_enable,
- .mode_set = meson_venc_cvbs_encoder_mode_set,
-};
-
-static bool meson_venc_cvbs_connector_is_available(struct meson_drm *priv)
-{
- struct device_node *remote;
-
- remote = of_graph_get_remote_node(priv->dev->of_node, 0, 0);
- if (!remote)
- return false;
-
- of_node_put(remote);
- return true;
-}
-
-int meson_venc_cvbs_create(struct meson_drm *priv)
-{
- struct drm_device *drm = priv->drm;
- struct meson_venc_cvbs *meson_venc_cvbs;
- struct drm_connector *connector;
- struct drm_encoder *encoder;
- int ret;
-
- if (!meson_venc_cvbs_connector_is_available(priv)) {
- dev_info(drm->dev, "CVBS Output connector not available\n");
- return 0;
- }
-
- meson_venc_cvbs = devm_kzalloc(priv->dev, sizeof(*meson_venc_cvbs),
- GFP_KERNEL);
- if (!meson_venc_cvbs)
- return -ENOMEM;
-
- meson_venc_cvbs->priv = priv;
- encoder = &meson_venc_cvbs->encoder;
- connector = &meson_venc_cvbs->connector;
-
- /* Connector */
-
- drm_connector_helper_add(connector,
- &meson_cvbs_connector_helper_funcs);
-
- ret = drm_connector_init(drm, connector, &meson_cvbs_connector_funcs,
- DRM_MODE_CONNECTOR_Composite);
- if (ret) {
- dev_err(priv->dev, "Failed to init CVBS connector\n");
- return ret;
- }
-
- connector->interlace_allowed = 1;
-
- /* Encoder */
-
- drm_encoder_helper_add(encoder, &meson_venc_cvbs_encoder_helper_funcs);
-
- ret = drm_encoder_init(drm, encoder, &meson_venc_cvbs_encoder_funcs,
- DRM_MODE_ENCODER_TVDAC, "meson_venc_cvbs");
- if (ret) {
- dev_err(priv->dev, "Failed to init CVBS encoder\n");
- return ret;
- }
-
- encoder->possible_crtcs = BIT(0);
-
- drm_connector_attach_encoder(connector, encoder);
-
- return 0;
-}
diff --git a/drivers/gpu/drm/meson/meson_venc_cvbs.h b/drivers/gpu/drm/meson/meson_venc_cvbs.h
deleted file mode 100644
index ab7f76b..0000000
--- a/drivers/gpu/drm/meson/meson_venc_cvbs.h
+++ /dev/null
@@ -1,29 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0-or-later */
-/*
- * Copyright (C) 2016 BayLibre, SAS
- * Author: Neil Armstrong <narmstrong@baylibre.com>
- * Copyright (C) 2014 Endless Mobile
- *
- * Written by:
- * Jasper St. Pierre <jstpierre@mecheye.net>
- */
-
-#ifndef __MESON_VENC_CVBS_H
-#define __MESON_VENC_CVBS_H
-
-#include "meson_drv.h"
-#include "meson_venc.h"
-
-struct meson_cvbs_mode {
- struct meson_cvbs_enci_mode *enci;
- struct drm_display_mode mode;
-};
-
-#define MESON_CVBS_MODES_COUNT 2
-
-/* Modes supported by the CVBS output */
-extern struct meson_cvbs_mode meson_cvbs_modes[MESON_CVBS_MODES_COUNT];
-
-int meson_venc_cvbs_create(struct meson_drm *priv);
-
-#endif /* __MESON_VENC_CVBS_H */
diff --git a/drivers/gpu/drm/mgag200/mgag200_drv.c b/drivers/gpu/drm/mgag200/mgag200_drv.c
index 6b92437..740108a 100644
--- a/drivers/gpu/drm/mgag200/mgag200_drv.c
+++ b/drivers/gpu/drm/mgag200/mgag200_drv.c
@@ -6,7 +6,6 @@
* Dave Airlie
*/
-#include <linux/console.h>
#include <linux/module.h>
#include <linux/pci.h>
#include <linux/vmalloc.h>
@@ -378,7 +377,7 @@ static struct pci_driver mgag200_pci_driver = {
static int __init mgag200_init(void)
{
- if (vgacon_text_force() && mgag200_modeset == -1)
+ if (drm_firmware_drivers_only() && mgag200_modeset == -1)
return -EINVAL;
if (mgag200_modeset == 0)
diff --git a/drivers/gpu/drm/mgag200/mgag200_mode.c b/drivers/gpu/drm/mgag200/mgag200_mode.c
index fd98e8b..b983541 100644
--- a/drivers/gpu/drm/mgag200/mgag200_mode.c
+++ b/drivers/gpu/drm/mgag200/mgag200_mode.c
@@ -847,9 +847,11 @@ static void
mgag200_handle_damage(struct mga_device *mdev, struct drm_framebuffer *fb,
struct drm_rect *clip, const struct dma_buf_map *map)
{
+ void __iomem *dst = mdev->vram;
void *vmap = map->vaddr; /* TODO: Use mapping abstraction properly */
- drm_fb_memcpy_dstclip(mdev->vram, fb->pitches[0], vmap, fb, clip);
+ dst += drm_fb_clip_offset(fb->pitches[0], fb->format, clip);
+ drm_fb_memcpy_toio(dst, fb->pitches[0], vmap, fb, clip);
/* Always scanout image at VRAM offset 0 */
mgag200_set_startadd(mdev, (u32)0);
diff --git a/drivers/gpu/drm/msm/Kconfig b/drivers/gpu/drm/msm/Kconfig
index 39197b4..1eae5a9 100644
--- a/drivers/gpu/drm/msm/Kconfig
+++ b/drivers/gpu/drm/msm/Kconfig
@@ -65,6 +65,7 @@
config DRM_MSM_DP
bool "Enable DisplayPort support in MSM DRM driver"
depends on DRM_MSM
+ select RATIONAL
default y
help
Compile in support for DP driver in MSM DRM driver. DP external
diff --git a/drivers/gpu/drm/msm/Makefile b/drivers/gpu/drm/msm/Makefile
index 0934544..03ab55c 100644
--- a/drivers/gpu/drm/msm/Makefile
+++ b/drivers/gpu/drm/msm/Makefile
@@ -19,7 +19,7 @@
hdmi/hdmi.o \
hdmi/hdmi_audio.o \
hdmi/hdmi_bridge.o \
- hdmi/hdmi_connector.o \
+ hdmi/hdmi_hpd.o \
hdmi/hdmi_i2c.o \
hdmi/hdmi_phy.o \
hdmi/hdmi_phy_8960.o \
@@ -27,12 +27,6 @@
hdmi/hdmi_phy_8x60.o \
hdmi/hdmi_phy_8x74.o \
hdmi/hdmi_pll_8960.o \
- edp/edp.o \
- edp/edp_aux.o \
- edp/edp_bridge.o \
- edp/edp_connector.o \
- edp/edp_ctrl.o \
- edp/edp_phy.o \
disp/mdp_format.o \
disp/mdp_kms.o \
disp/mdp4/mdp4_crtc.o \
diff --git a/drivers/gpu/drm/msm/adreno/a2xx_gpu.c b/drivers/gpu/drm/msm/adreno/a2xx_gpu.c
index bdc9891..22e8295 100644
--- a/drivers/gpu/drm/msm/adreno/a2xx_gpu.c
+++ b/drivers/gpu/drm/msm/adreno/a2xx_gpu.c
@@ -12,7 +12,6 @@ static bool a2xx_idle(struct msm_gpu *gpu);
static void a2xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit)
{
- struct msm_drm_private *priv = gpu->dev->dev_private;
struct msm_ringbuffer *ring = submit->ring;
unsigned int i;
@@ -23,7 +22,7 @@ static void a2xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit)
break;
case MSM_SUBMIT_CMD_CTX_RESTORE_BUF:
/* ignore if there has not been a ctx switch: */
- if (priv->lastctx == submit->queue->ctx)
+ if (gpu->cur_ctx_seqno == submit->queue->ctx->seqno)
break;
fallthrough;
case MSM_SUBMIT_CMD_BUF:
diff --git a/drivers/gpu/drm/msm/adreno/a3xx_gpu.c b/drivers/gpu/drm/msm/adreno/a3xx_gpu.c
index 8fb847c..2e481e2 100644
--- a/drivers/gpu/drm/msm/adreno/a3xx_gpu.c
+++ b/drivers/gpu/drm/msm/adreno/a3xx_gpu.c
@@ -30,7 +30,6 @@ static bool a3xx_idle(struct msm_gpu *gpu);
static void a3xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit)
{
- struct msm_drm_private *priv = gpu->dev->dev_private;
struct msm_ringbuffer *ring = submit->ring;
unsigned int i;
@@ -41,7 +40,7 @@ static void a3xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit)
break;
case MSM_SUBMIT_CMD_CTX_RESTORE_BUF:
/* ignore if there has not been a ctx switch: */
- if (priv->lastctx == submit->queue->ctx)
+ if (gpu->cur_ctx_seqno == submit->queue->ctx->seqno)
break;
fallthrough;
case MSM_SUBMIT_CMD_BUF:
diff --git a/drivers/gpu/drm/msm/adreno/a4xx_gpu.c b/drivers/gpu/drm/msm/adreno/a4xx_gpu.c
index a96ee79..c5524d6 100644
--- a/drivers/gpu/drm/msm/adreno/a4xx_gpu.c
+++ b/drivers/gpu/drm/msm/adreno/a4xx_gpu.c
@@ -24,7 +24,6 @@ static bool a4xx_idle(struct msm_gpu *gpu);
static void a4xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit)
{
- struct msm_drm_private *priv = gpu->dev->dev_private;
struct msm_ringbuffer *ring = submit->ring;
unsigned int i;
@@ -35,7 +34,7 @@ static void a4xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit)
break;
case MSM_SUBMIT_CMD_CTX_RESTORE_BUF:
/* ignore if there has not been a ctx switch: */
- if (priv->lastctx == submit->queue->ctx)
+ if (gpu->cur_ctx_seqno == submit->queue->ctx->seqno)
break;
fallthrough;
case MSM_SUBMIT_CMD_BUF:
diff --git a/drivers/gpu/drm/msm/adreno/a5xx_debugfs.c b/drivers/gpu/drm/msm/adreno/a5xx_debugfs.c
index dd593ec..6bd397a 100644
--- a/drivers/gpu/drm/msm/adreno/a5xx_debugfs.c
+++ b/drivers/gpu/drm/msm/adreno/a5xx_debugfs.c
@@ -107,7 +107,7 @@ reset_set(void *data, u64 val)
* try to reset an active GPU.
*/
- mutex_lock(&dev->struct_mutex);
+ mutex_lock(&gpu->lock);
release_firmware(adreno_gpu->fw[ADRENO_FW_PM4]);
adreno_gpu->fw[ADRENO_FW_PM4] = NULL;
@@ -133,7 +133,7 @@ reset_set(void *data, u64 val)
gpu->funcs->recover(gpu);
pm_runtime_put_sync(&gpu->pdev->dev);
- mutex_unlock(&dev->struct_mutex);
+ mutex_unlock(&gpu->lock);
return 0;
}
diff --git a/drivers/gpu/drm/msm/adreno/a5xx_gpu.c b/drivers/gpu/drm/msm/adreno/a5xx_gpu.c
index 5e2750e..3d28fcf 100644
--- a/drivers/gpu/drm/msm/adreno/a5xx_gpu.c
+++ b/drivers/gpu/drm/msm/adreno/a5xx_gpu.c
@@ -65,7 +65,6 @@ void a5xx_flush(struct msm_gpu *gpu, struct msm_ringbuffer *ring,
static void a5xx_submit_in_rb(struct msm_gpu *gpu, struct msm_gem_submit *submit)
{
- struct msm_drm_private *priv = gpu->dev->dev_private;
struct msm_ringbuffer *ring = submit->ring;
struct msm_gem_object *obj;
uint32_t *ptr, dwords;
@@ -76,7 +75,7 @@ static void a5xx_submit_in_rb(struct msm_gpu *gpu, struct msm_gem_submit *submit
case MSM_SUBMIT_CMD_IB_TARGET_BUF:
break;
case MSM_SUBMIT_CMD_CTX_RESTORE_BUF:
- if (priv->lastctx == submit->queue->ctx)
+ if (gpu->cur_ctx_seqno == submit->queue->ctx->seqno)
break;
fallthrough;
case MSM_SUBMIT_CMD_BUF:
@@ -126,12 +125,11 @@ static void a5xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit)
{
struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu);
struct a5xx_gpu *a5xx_gpu = to_a5xx_gpu(adreno_gpu);
- struct msm_drm_private *priv = gpu->dev->dev_private;
struct msm_ringbuffer *ring = submit->ring;
unsigned int i, ibs = 0;
if (IS_ENABLED(CONFIG_DRM_MSM_GPU_SUDO) && submit->in_rb) {
- priv->lastctx = NULL;
+ gpu->cur_ctx_seqno = 0;
a5xx_submit_in_rb(gpu, submit);
return;
}
@@ -166,7 +164,7 @@ static void a5xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit)
case MSM_SUBMIT_CMD_IB_TARGET_BUF:
break;
case MSM_SUBMIT_CMD_CTX_RESTORE_BUF:
- if (priv->lastctx == submit->queue->ctx)
+ if (gpu->cur_ctx_seqno == submit->queue->ctx->seqno)
break;
fallthrough;
case MSM_SUBMIT_CMD_BUF:
@@ -441,7 +439,7 @@ void a5xx_set_hwcg(struct msm_gpu *gpu, bool state)
const struct adreno_five_hwcg_regs *regs;
unsigned int i, sz;
- if (adreno_is_a508(adreno_gpu)) {
+ if (adreno_is_a506(adreno_gpu) || adreno_is_a508(adreno_gpu)) {
regs = a50x_hwcg;
sz = ARRAY_SIZE(a50x_hwcg);
} else if (adreno_is_a509(adreno_gpu) || adreno_is_a512(adreno_gpu)) {
@@ -485,7 +483,7 @@ static int a5xx_me_init(struct msm_gpu *gpu)
OUT_RING(ring, 0x00000000);
/* Specify workarounds for various microcode issues */
- if (adreno_is_a530(adreno_gpu)) {
+ if (adreno_is_a506(adreno_gpu) || adreno_is_a530(adreno_gpu)) {
/* Workaround for token end syncs
* Force a WFI after every direct-render 3D mode draw and every
* 2D mode 3 draw
@@ -620,8 +618,16 @@ static int a5xx_ucode_init(struct msm_gpu *gpu)
static int a5xx_zap_shader_resume(struct msm_gpu *gpu)
{
+ struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu);
int ret;
+ /*
+ * Adreno 506 have CPZ Retention feature and doesn't require
+ * to resume zap shader
+ */
+ if (adreno_is_a506(adreno_gpu))
+ return 0;
+
ret = qcom_scm_set_remote_state(SCM_GPU_ZAP_SHADER_RESUME, GPU_PAS_ID);
if (ret)
DRM_ERROR("%s: zap-shader resume failed: %d\n",
@@ -733,9 +739,10 @@ static int a5xx_hw_init(struct msm_gpu *gpu)
0x00100000 + adreno_gpu->gmem - 1);
gpu_write(gpu, REG_A5XX_UCHE_GMEM_RANGE_MAX_HI, 0x00000000);
- if (adreno_is_a508(adreno_gpu) || adreno_is_a510(adreno_gpu)) {
+ if (adreno_is_a506(adreno_gpu) || adreno_is_a508(adreno_gpu) ||
+ adreno_is_a510(adreno_gpu)) {
gpu_write(gpu, REG_A5XX_CP_MEQ_THRESHOLDS, 0x20);
- if (adreno_is_a508(adreno_gpu))
+ if (adreno_is_a506(adreno_gpu) || adreno_is_a508(adreno_gpu))
gpu_write(gpu, REG_A5XX_CP_MERCIU_SIZE, 0x400);
else
gpu_write(gpu, REG_A5XX_CP_MERCIU_SIZE, 0x20);
@@ -751,7 +758,7 @@ static int a5xx_hw_init(struct msm_gpu *gpu)
gpu_write(gpu, REG_A5XX_CP_ROQ_THRESHOLDS_1, 0x40201B16);
}
- if (adreno_is_a508(adreno_gpu))
+ if (adreno_is_a506(adreno_gpu) || adreno_is_a508(adreno_gpu))
gpu_write(gpu, REG_A5XX_PC_DBG_ECO_CNTL,
(0x100 << 11 | 0x100 << 22));
else if (adreno_is_a509(adreno_gpu) || adreno_is_a510(adreno_gpu) ||
@@ -769,8 +776,8 @@ static int a5xx_hw_init(struct msm_gpu *gpu)
* Disable the RB sampler datapath DP2 clock gating optimization
* for 1-SP GPUs, as it is enabled by default.
*/
- if (adreno_is_a508(adreno_gpu) || adreno_is_a509(adreno_gpu) ||
- adreno_is_a512(adreno_gpu))
+ if (adreno_is_a506(adreno_gpu) || adreno_is_a508(adreno_gpu) ||
+ adreno_is_a509(adreno_gpu) || adreno_is_a512(adreno_gpu))
gpu_rmw(gpu, REG_A5XX_RB_DBG_ECO_CNTL, 0, (1 << 9));
/* Disable UCHE global filter as SP can invalidate/flush independently */
@@ -851,10 +858,8 @@ static int a5xx_hw_init(struct msm_gpu *gpu)
/* UCHE */
gpu_write(gpu, REG_A5XX_CP_PROTECT(16), ADRENO_PROTECT_RW(0xE80, 16));
- if (adreno_is_a508(adreno_gpu) || adreno_is_a509(adreno_gpu) ||
- adreno_is_a510(adreno_gpu) || adreno_is_a512(adreno_gpu) ||
- adreno_is_a530(adreno_gpu))
- gpu_write(gpu, REG_A5XX_CP_PROTECT(17),
+ /* SMMU */
+ gpu_write(gpu, REG_A5XX_CP_PROTECT(17),
ADRENO_PROTECT_RW(0x10000, 0x8000));
gpu_write(gpu, REG_A5XX_RBBM_SECVID_TSB_CNTL, 0);
@@ -895,8 +900,7 @@ static int a5xx_hw_init(struct msm_gpu *gpu)
if (ret)
return ret;
- if (!(adreno_is_a508(adreno_gpu) || adreno_is_a509(adreno_gpu) ||
- adreno_is_a510(adreno_gpu) || adreno_is_a512(adreno_gpu)))
+ if (adreno_is_a530(adreno_gpu) || adreno_is_a540(adreno_gpu))
a5xx_gpmu_ucode_init(gpu);
ret = a5xx_ucode_init(gpu);
@@ -927,6 +931,8 @@ static int a5xx_hw_init(struct msm_gpu *gpu)
if (IS_ERR(a5xx_gpu->shadow))
return PTR_ERR(a5xx_gpu->shadow);
+
+ msm_gem_object_set_name(a5xx_gpu->shadow_bo, "shadow");
}
gpu_write64(gpu, REG_A5XX_CP_RB_RPTR_ADDR,
@@ -1254,6 +1260,7 @@ static void a5xx_fault_detect_irq(struct msm_gpu *gpu)
static irqreturn_t a5xx_irq(struct msm_gpu *gpu)
{
+ struct msm_drm_private *priv = gpu->dev->dev_private;
u32 status = gpu_read(gpu, REG_A5XX_RBBM_INT_0_STATUS);
/*
@@ -1263,6 +1270,11 @@ static irqreturn_t a5xx_irq(struct msm_gpu *gpu)
gpu_write(gpu, REG_A5XX_RBBM_INT_CLEAR_CMD,
status & ~A5XX_RBBM_INT_0_MASK_RBBM_AHB_ERROR);
+ if (priv->disable_err_irq) {
+ status &= A5XX_RBBM_INT_0_MASK_CP_CACHE_FLUSH_TS |
+ A5XX_RBBM_INT_0_MASK_CP_SW;
+ }
+
/* Pass status to a5xx_rbbm_err_irq because we've already cleared it */
if (status & RBBM_ERROR_MASK)
a5xx_rbbm_err_irq(gpu, status);
@@ -1338,7 +1350,7 @@ static int a5xx_pm_resume(struct msm_gpu *gpu)
if (ret)
return ret;
- /* Adreno 508, 509, 510, 512 needs manual RBBM sus/res control */
+ /* Adreno 506, 508, 509, 510, 512 needs manual RBBM sus/res control */
if (!(adreno_is_a530(adreno_gpu) || adreno_is_a540(adreno_gpu))) {
/* Halt the sp_input_clk at HM level */
gpu_write(gpu, REG_A5XX_RBBM_CLOCK_CNTL, 0x00000055);
@@ -1381,8 +1393,9 @@ static int a5xx_pm_suspend(struct msm_gpu *gpu)
u32 mask = 0xf;
int i, ret;
- /* A508, A510 have 3 XIN ports in VBIF */
- if (adreno_is_a508(adreno_gpu) || adreno_is_a510(adreno_gpu))
+ /* A506, A508, A510 have 3 XIN ports in VBIF */
+ if (adreno_is_a506(adreno_gpu) || adreno_is_a508(adreno_gpu) ||
+ adreno_is_a510(adreno_gpu))
mask = 0x7;
/* Clear the VBIF pipe before shutting down */
diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gmu.c b/drivers/gpu/drm/msm/adreno/a6xx_gmu.c
index 71e52b2..3e325e2 100644
--- a/drivers/gpu/drm/msm/adreno/a6xx_gmu.c
+++ b/drivers/gpu/drm/msm/adreno/a6xx_gmu.c
@@ -1146,7 +1146,7 @@ static void a6xx_gmu_memory_free(struct a6xx_gmu *gmu)
}
static int a6xx_gmu_memory_alloc(struct a6xx_gmu *gmu, struct a6xx_gmu_bo *bo,
- size_t size, u64 iova)
+ size_t size, u64 iova, const char *name)
{
struct a6xx_gpu *a6xx_gpu = container_of(gmu, struct a6xx_gpu, gmu);
struct drm_device *dev = a6xx_gpu->base.base.dev;
@@ -1181,6 +1181,8 @@ static int a6xx_gmu_memory_alloc(struct a6xx_gmu *gmu, struct a6xx_gmu_bo *bo,
bo->virt = msm_gem_get_vaddr(bo->obj);
bo->size = size;
+ msm_gem_object_set_name(bo->obj, name);
+
return 0;
}
@@ -1515,7 +1517,8 @@ int a6xx_gmu_init(struct a6xx_gpu *a6xx_gpu, struct device_node *node)
*/
gmu->dummy.size = SZ_4K;
if (adreno_is_a660_family(adreno_gpu)) {
- ret = a6xx_gmu_memory_alloc(gmu, &gmu->debug, SZ_4K * 7, 0x60400000);
+ ret = a6xx_gmu_memory_alloc(gmu, &gmu->debug, SZ_4K * 7,
+ 0x60400000, "debug");
if (ret)
goto err_memory;
@@ -1523,42 +1526,46 @@ int a6xx_gmu_init(struct a6xx_gpu *a6xx_gpu, struct device_node *node)
}
/* Allocate memory for the GMU dummy page */
- ret = a6xx_gmu_memory_alloc(gmu, &gmu->dummy, gmu->dummy.size, 0x60000000);
+ ret = a6xx_gmu_memory_alloc(gmu, &gmu->dummy, gmu->dummy.size,
+ 0x60000000, "dummy");
if (ret)
goto err_memory;
+ /* Note that a650 family also includes a660 family: */
if (adreno_is_a650_family(adreno_gpu)) {
ret = a6xx_gmu_memory_alloc(gmu, &gmu->icache,
- SZ_16M - SZ_16K, 0x04000);
+ SZ_16M - SZ_16K, 0x04000, "icache");
if (ret)
goto err_memory;
} else if (adreno_is_a640_family(adreno_gpu)) {
ret = a6xx_gmu_memory_alloc(gmu, &gmu->icache,
- SZ_256K - SZ_16K, 0x04000);
+ SZ_256K - SZ_16K, 0x04000, "icache");
if (ret)
goto err_memory;
ret = a6xx_gmu_memory_alloc(gmu, &gmu->dcache,
- SZ_256K - SZ_16K, 0x44000);
+ SZ_256K - SZ_16K, 0x44000, "dcache");
if (ret)
goto err_memory;
} else {
+ BUG_ON(adreno_is_a660_family(adreno_gpu));
+
/* HFI v1, has sptprac */
gmu->legacy = true;
/* Allocate memory for the GMU debug region */
- ret = a6xx_gmu_memory_alloc(gmu, &gmu->debug, SZ_16K, 0);
+ ret = a6xx_gmu_memory_alloc(gmu, &gmu->debug, SZ_16K, 0, "debug");
if (ret)
goto err_memory;
}
/* Allocate memory for for the HFI queues */
- ret = a6xx_gmu_memory_alloc(gmu, &gmu->hfi, SZ_16K, 0);
+ ret = a6xx_gmu_memory_alloc(gmu, &gmu->hfi, SZ_16K, 0, "hfi");
if (ret)
goto err_memory;
/* Allocate memory for the GMU log region */
- ret = a6xx_gmu_memory_alloc(gmu, &gmu->log, SZ_4K, 0);
+ ret = a6xx_gmu_memory_alloc(gmu, &gmu->log, SZ_4K, 0, "log");
if (ret)
goto err_memory;
diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c
index 78aad52..51b8377 100644
--- a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c
+++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c
@@ -106,7 +106,7 @@ static void a6xx_set_pagetable(struct a6xx_gpu *a6xx_gpu,
u32 asid;
u64 memptr = rbmemptr(ring, ttbr0);
- if (ctx->seqno == a6xx_gpu->cur_ctx_seqno)
+ if (ctx->seqno == a6xx_gpu->base.base.cur_ctx_seqno)
return;
if (msm_iommu_pagetable_params(ctx->aspace->mmu, &ttbr, &asid))
@@ -138,14 +138,11 @@ static void a6xx_set_pagetable(struct a6xx_gpu *a6xx_gpu,
OUT_PKT7(ring, CP_EVENT_WRITE, 1);
OUT_RING(ring, 0x31);
-
- a6xx_gpu->cur_ctx_seqno = ctx->seqno;
}
static void a6xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit)
{
unsigned int index = submit->seqno % MSM_GPU_SUBMIT_STATS_COUNT;
- struct msm_drm_private *priv = gpu->dev->dev_private;
struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu);
struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu);
struct msm_ringbuffer *ring = submit->ring;
@@ -177,7 +174,7 @@ static void a6xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit)
case MSM_SUBMIT_CMD_IB_TARGET_BUF:
break;
case MSM_SUBMIT_CMD_CTX_RESTORE_BUF:
- if (priv->lastctx == submit->queue->ctx)
+ if (gpu->cur_ctx_seqno == submit->queue->ctx->seqno)
break;
fallthrough;
case MSM_SUBMIT_CMD_BUF:
@@ -1071,6 +1068,8 @@ static int hw_init(struct msm_gpu *gpu)
if (IS_ERR(a6xx_gpu->shadow))
return PTR_ERR(a6xx_gpu->shadow);
+
+ msm_gem_object_set_name(a6xx_gpu->shadow_bo, "shadow");
}
gpu_write64(gpu, REG_A6XX_CP_RB_RPTR_ADDR_LO,
@@ -1081,7 +1080,7 @@ static int hw_init(struct msm_gpu *gpu)
/* Always come up on rb 0 */
a6xx_gpu->cur_ring = gpu->rb[0];
- a6xx_gpu->cur_ctx_seqno = 0;
+ gpu->cur_ctx_seqno = 0;
/* Enable the SQE_to start the CP engine */
gpu_write(gpu, REG_A6XX_CP_SQE_CNTL, 1);
@@ -1376,10 +1375,14 @@ static void a6xx_fault_detect_irq(struct msm_gpu *gpu)
static irqreturn_t a6xx_irq(struct msm_gpu *gpu)
{
+ struct msm_drm_private *priv = gpu->dev->dev_private;
u32 status = gpu_read(gpu, REG_A6XX_RBBM_INT_0_STATUS);
gpu_write(gpu, REG_A6XX_RBBM_INT_CLEAR_CMD, status);
+ if (priv->disable_err_irq)
+ status &= A6XX_RBBM_INT_0_MASK_CP_CACHE_FLUSH_TS;
+
if (status & A6XX_RBBM_INT_0_MASK_RBBM_HANG_DETECT)
a6xx_fault_detect_irq(gpu);
diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.h b/drivers/gpu/drm/msm/adreno/a6xx_gpu.h
index 8e5527c..86e0a7c 100644
--- a/drivers/gpu/drm/msm/adreno/a6xx_gpu.h
+++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.h
@@ -20,16 +20,6 @@ struct a6xx_gpu {
struct msm_ringbuffer *cur_ring;
- /**
- * cur_ctx_seqno:
- *
- * The ctx->seqno value of the context with current pgtables
- * installed. Tracked by seqno rather than pointer value to
- * avoid dangling pointers, and cases where a ctx can be freed
- * and a new one created with the same address.
- */
- int cur_ctx_seqno;
-
struct a6xx_gmu gmu;
struct drm_gem_object *shadow_bo;
diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu_state.c b/drivers/gpu/drm/msm/adreno/a6xx_gpu_state.c
index 6e90209..55f4433 100644
--- a/drivers/gpu/drm/msm/adreno/a6xx_gpu_state.c
+++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu_state.c
@@ -42,7 +42,15 @@ struct a6xx_gpu_state {
struct a6xx_gpu_state_obj *cx_debugbus;
int nr_cx_debugbus;
+ struct msm_gpu_state_bo *gmu_log;
+ struct msm_gpu_state_bo *gmu_hfi;
+ struct msm_gpu_state_bo *gmu_debug;
+
+ s32 hfi_queue_history[2][HFI_HISTORY_SZ];
+
struct list_head objs;
+
+ bool gpu_initialized;
};
static inline int CRASHDUMP_WRITE(u64 *in, u32 reg, u32 val)
@@ -800,6 +808,45 @@ static void a6xx_get_gmu_registers(struct msm_gpu *gpu,
&a6xx_state->gmu_registers[2], false);
}
+static struct msm_gpu_state_bo *a6xx_snapshot_gmu_bo(
+ struct a6xx_gpu_state *a6xx_state, struct a6xx_gmu_bo *bo)
+{
+ struct msm_gpu_state_bo *snapshot;
+
+ snapshot = state_kcalloc(a6xx_state, 1, sizeof(*snapshot));
+ if (!snapshot)
+ return NULL;
+
+ snapshot->iova = bo->iova;
+ snapshot->size = bo->size;
+ snapshot->data = kvzalloc(snapshot->size, GFP_KERNEL);
+ if (!snapshot->data)
+ return NULL;
+
+ memcpy(snapshot->data, bo->virt, bo->size);
+
+ return snapshot;
+}
+
+static void a6xx_snapshot_gmu_hfi_history(struct msm_gpu *gpu,
+ struct a6xx_gpu_state *a6xx_state)
+{
+ struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu);
+ struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu);
+ struct a6xx_gmu *gmu = &a6xx_gpu->gmu;
+ unsigned i, j;
+
+ BUILD_BUG_ON(ARRAY_SIZE(gmu->queues) != ARRAY_SIZE(a6xx_state->hfi_queue_history));
+
+ for (i = 0; i < ARRAY_SIZE(gmu->queues); i++) {
+ struct a6xx_hfi_queue *queue = &gmu->queues[i];
+ for (j = 0; j < HFI_HISTORY_SZ; j++) {
+ unsigned idx = (j + queue->history_idx) % HFI_HISTORY_SZ;
+ a6xx_state->hfi_queue_history[i][j] = queue->history[idx];
+ }
+ }
+}
+
#define A6XX_GBIF_REGLIST_SIZE 1
static void a6xx_get_registers(struct msm_gpu *gpu,
struct a6xx_gpu_state *a6xx_state,
@@ -937,6 +984,12 @@ struct msm_gpu_state *a6xx_gpu_state_get(struct msm_gpu *gpu)
a6xx_get_gmu_registers(gpu, a6xx_state);
+ a6xx_state->gmu_log = a6xx_snapshot_gmu_bo(a6xx_state, &a6xx_gpu->gmu.log);
+ a6xx_state->gmu_hfi = a6xx_snapshot_gmu_bo(a6xx_state, &a6xx_gpu->gmu.hfi);
+ a6xx_state->gmu_debug = a6xx_snapshot_gmu_bo(a6xx_state, &a6xx_gpu->gmu.debug);
+
+ a6xx_snapshot_gmu_hfi_history(gpu, a6xx_state);
+
/* If GX isn't on the rest of the data isn't going to be accessible */
if (!a6xx_gmu_gx_is_on(&a6xx_gpu->gmu))
return &a6xx_state->base;
@@ -950,7 +1003,8 @@ struct msm_gpu_state *a6xx_gpu_state_get(struct msm_gpu *gpu)
* write out GPU state, so we need to skip this when the SMMU is
* stalled in response to an iova fault
*/
- if (!stalled && !a6xx_crashdumper_init(gpu, &_dumper)) {
+ if (!stalled && !gpu->needs_hw_init &&
+ !a6xx_crashdumper_init(gpu, &_dumper)) {
dumper = &_dumper;
}
@@ -967,6 +1021,8 @@ struct msm_gpu_state *a6xx_gpu_state_get(struct msm_gpu *gpu)
if (snapshot_debugbus)
a6xx_get_debugbus(gpu, a6xx_state);
+ a6xx_state->gpu_initialized = !gpu->needs_hw_init;
+
return &a6xx_state->base;
}
@@ -978,6 +1034,12 @@ static void a6xx_gpu_state_destroy(struct kref *kref)
struct a6xx_gpu_state *a6xx_state = container_of(state,
struct a6xx_gpu_state, base);
+ if (a6xx_state->gmu_log)
+ kvfree(a6xx_state->gmu_log->data);
+
+ if (a6xx_state->gmu_hfi)
+ kvfree(a6xx_state->gmu_hfi->data);
+
list_for_each_entry_safe(obj, tmp, &a6xx_state->objs, node)
kfree(obj);
@@ -1189,8 +1251,48 @@ void a6xx_show(struct msm_gpu *gpu, struct msm_gpu_state *state,
if (IS_ERR_OR_NULL(state))
return;
+ drm_printf(p, "gpu-initialized: %d\n", a6xx_state->gpu_initialized);
+
adreno_show(gpu, state, p);
+ drm_puts(p, "gmu-log:\n");
+ if (a6xx_state->gmu_log) {
+ struct msm_gpu_state_bo *gmu_log = a6xx_state->gmu_log;
+
+ drm_printf(p, " iova: 0x%016llx\n", gmu_log->iova);
+ drm_printf(p, " size: %zu\n", gmu_log->size);
+ adreno_show_object(p, &gmu_log->data, gmu_log->size,
+ &gmu_log->encoded);
+ }
+
+ drm_puts(p, "gmu-hfi:\n");
+ if (a6xx_state->gmu_hfi) {
+ struct msm_gpu_state_bo *gmu_hfi = a6xx_state->gmu_hfi;
+ unsigned i, j;
+
+ drm_printf(p, " iova: 0x%016llx\n", gmu_hfi->iova);
+ drm_printf(p, " size: %zu\n", gmu_hfi->size);
+ for (i = 0; i < ARRAY_SIZE(a6xx_state->hfi_queue_history); i++) {
+ drm_printf(p, " queue-history[%u]:", i);
+ for (j = 0; j < HFI_HISTORY_SZ; j++) {
+ drm_printf(p, " %d", a6xx_state->hfi_queue_history[i][j]);
+ }
+ drm_printf(p, "\n");
+ }
+ adreno_show_object(p, &gmu_hfi->data, gmu_hfi->size,
+ &gmu_hfi->encoded);
+ }
+
+ drm_puts(p, "gmu-debug:\n");
+ if (a6xx_state->gmu_debug) {
+ struct msm_gpu_state_bo *gmu_debug = a6xx_state->gmu_debug;
+
+ drm_printf(p, " iova: 0x%016llx\n", gmu_debug->iova);
+ drm_printf(p, " size: %zu\n", gmu_debug->size);
+ adreno_show_object(p, &gmu_debug->data, gmu_debug->size,
+ &gmu_debug->encoded);
+ }
+
drm_puts(p, "registers:\n");
for (i = 0; i < a6xx_state->nr_registers; i++) {
struct a6xx_gpu_state_obj *obj = &a6xx_state->registers[i];
diff --git a/drivers/gpu/drm/msm/adreno/a6xx_hfi.c b/drivers/gpu/drm/msm/adreno/a6xx_hfi.c
index d4c65bf..d73fce5 100644
--- a/drivers/gpu/drm/msm/adreno/a6xx_hfi.c
+++ b/drivers/gpu/drm/msm/adreno/a6xx_hfi.c
@@ -36,6 +36,8 @@ static int a6xx_hfi_queue_read(struct a6xx_gmu *gmu,
hdr = queue->data[index];
+ queue->history[(queue->history_idx++) % HFI_HISTORY_SZ] = index;
+
/*
* If we are to assume that the GMU firmware is in fact a rational actor
* and is programmed to not send us a larger response than we expect
@@ -75,6 +77,8 @@ static int a6xx_hfi_queue_write(struct a6xx_gmu *gmu,
return -ENOSPC;
}
+ queue->history[(queue->history_idx++) % HFI_HISTORY_SZ] = index;
+
for (i = 0; i < dwords; i++) {
queue->data[index] = data[i];
index = (index + 1) % header->size;
@@ -600,6 +604,9 @@ void a6xx_hfi_stop(struct a6xx_gmu *gmu)
queue->header->read_index = 0;
queue->header->write_index = 0;
+
+ memset(&queue->history, 0xff, sizeof(queue->history));
+ queue->history_idx = 0;
}
}
@@ -612,6 +619,9 @@ static void a6xx_hfi_queue_init(struct a6xx_hfi_queue *queue,
queue->data = virt;
atomic_set(&queue->seqnum, 0);
+ memset(&queue->history, 0xff, sizeof(queue->history));
+ queue->history_idx = 0;
+
/* Set up the shared memory header */
header->iova = iova;
header->type = 10 << 8 | id;
diff --git a/drivers/gpu/drm/msm/adreno/a6xx_hfi.h b/drivers/gpu/drm/msm/adreno/a6xx_hfi.h
index 2bd670c..5281101 100644
--- a/drivers/gpu/drm/msm/adreno/a6xx_hfi.h
+++ b/drivers/gpu/drm/msm/adreno/a6xx_hfi.h
@@ -33,6 +33,17 @@ struct a6xx_hfi_queue {
spinlock_t lock;
u32 *data;
atomic_t seqnum;
+
+ /*
+ * Tracking for the start index of the last N messages in the
+ * queue, for the benefit of devcore dump / crashdec (since
+ * parsing in the reverse direction to decode the last N
+ * messages is difficult to do and would rely on heuristics
+ * which are not guaranteed to be correct)
+ */
+#define HFI_HISTORY_SZ 8
+ s32 history[HFI_HISTORY_SZ];
+ u8 history_idx;
};
/* This is the outgoing queue to the GMU */
diff --git a/drivers/gpu/drm/msm/adreno/adreno_device.c b/drivers/gpu/drm/msm/adreno/adreno_device.c
index 2a6ce76..9300583 100644
--- a/drivers/gpu/drm/msm/adreno/adreno_device.c
+++ b/drivers/gpu/drm/msm/adreno/adreno_device.c
@@ -132,6 +132,24 @@ static const struct adreno_info gpulist[] = {
.inactive_period = DRM_MSM_INACTIVE_PERIOD,
.init = a4xx_gpu_init,
}, {
+ .rev = ADRENO_REV(5, 0, 6, ANY_ID),
+ .revn = 506,
+ .name = "A506",
+ .fw = {
+ [ADRENO_FW_PM4] = "a530_pm4.fw",
+ [ADRENO_FW_PFP] = "a530_pfp.fw",
+ },
+ .gmem = (SZ_128K + SZ_8K),
+ /*
+ * Increase inactive period to 250 to avoid bouncing
+ * the GDSC which appears to make it grumpy
+ */
+ .inactive_period = 250,
+ .quirks = ADRENO_QUIRK_TWO_PASS_USE_WFI |
+ ADRENO_QUIRK_LMLOADKILL_DISABLE,
+ .init = a5xx_gpu_init,
+ .zapfw = "a506_zap.mdt",
+ }, {
.rev = ADRENO_REV(5, 0, 8, ANY_ID),
.revn = 508,
.name = "A508",
@@ -408,9 +426,9 @@ struct msm_gpu *adreno_load_gpu(struct drm_device *dev)
return NULL;
}
- mutex_lock(&dev->struct_mutex);
+ mutex_lock(&gpu->lock);
ret = msm_gpu_hw_init(gpu);
- mutex_unlock(&dev->struct_mutex);
+ mutex_unlock(&gpu->lock);
pm_runtime_put_autosuspend(&pdev->dev);
if (ret) {
DRM_DEV_ERROR(dev->dev, "gpu hw init failed: %d\n", ret);
@@ -427,13 +445,6 @@ struct msm_gpu *adreno_load_gpu(struct drm_device *dev)
return gpu;
}
-static void set_gpu_pdev(struct drm_device *dev,
- struct platform_device *pdev)
-{
- struct msm_drm_private *priv = dev->dev_private;
- priv->gpu_pdev = pdev;
-}
-
static int find_chipid(struct device *dev, struct adreno_rev *rev)
{
struct device_node *node = dev->of_node;
@@ -482,8 +493,8 @@ static int adreno_bind(struct device *dev, struct device *master, void *data)
{
static struct adreno_platform_config config = {};
const struct adreno_info *info;
- struct drm_device *drm = dev_get_drvdata(master);
- struct msm_drm_private *priv = drm->dev_private;
+ struct msm_drm_private *priv = dev_get_drvdata(master);
+ struct drm_device *drm = priv->dev;
struct msm_gpu *gpu;
int ret;
@@ -492,7 +503,7 @@ static int adreno_bind(struct device *dev, struct device *master, void *data)
return ret;
dev->platform_data = &config;
- set_gpu_pdev(drm, to_platform_device(dev));
+ priv->gpu_pdev = to_platform_device(dev);
info = adreno_info(config.rev);
@@ -521,12 +532,13 @@ static int adreno_bind(struct device *dev, struct device *master, void *data)
static void adreno_unbind(struct device *dev, struct device *master,
void *data)
{
+ struct msm_drm_private *priv = dev_get_drvdata(master);
struct msm_gpu *gpu = dev_to_gpu(dev);
pm_runtime_force_suspend(dev);
gpu->funcs->destroy(gpu);
- set_gpu_pdev(dev_get_drvdata(master), NULL);
+ priv->gpu_pdev = NULL;
}
static const struct component_ops a3xx_ops = {
diff --git a/drivers/gpu/drm/msm/adreno/adreno_gpu.c b/drivers/gpu/drm/msm/adreno/adreno_gpu.c
index 7486652..f33cfa4 100644
--- a/drivers/gpu/drm/msm/adreno/adreno_gpu.c
+++ b/drivers/gpu/drm/msm/adreno/adreno_gpu.c
@@ -504,6 +504,8 @@ int adreno_gpu_state_get(struct msm_gpu *gpu, struct msm_gpu_state *state)
struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu);
int i, count = 0;
+ WARN_ON(!mutex_is_locked(&gpu->lock));
+
kref_init(&state->ref);
ktime_get_real_ts64(&state->time);
@@ -630,7 +632,7 @@ static char *adreno_gpu_ascii85_encode(u32 *src, size_t len)
}
/* len is expected to be in bytes */
-static void adreno_show_object(struct drm_printer *p, void **ptr, int len,
+void adreno_show_object(struct drm_printer *p, void **ptr, int len,
bool *encoded)
{
if (!*ptr || !len)
diff --git a/drivers/gpu/drm/msm/adreno/adreno_gpu.h b/drivers/gpu/drm/msm/adreno/adreno_gpu.h
index 225c277..cffabe7 100644
--- a/drivers/gpu/drm/msm/adreno/adreno_gpu.h
+++ b/drivers/gpu/drm/msm/adreno/adreno_gpu.h
@@ -201,6 +201,11 @@ static inline int adreno_is_a430(struct adreno_gpu *gpu)
return gpu->revn == 430;
}
+static inline int adreno_is_a506(struct adreno_gpu *gpu)
+{
+ return gpu->revn == 506;
+}
+
static inline int adreno_is_a508(struct adreno_gpu *gpu)
{
return gpu->revn == 508;
@@ -306,6 +311,8 @@ void adreno_gpu_state_destroy(struct msm_gpu_state *state);
int adreno_gpu_state_get(struct msm_gpu *gpu, struct msm_gpu_state *state);
int adreno_gpu_state_put(struct msm_gpu_state *state);
+void adreno_show_object(struct drm_printer *p, void **ptr, int len,
+ bool *encoded);
/*
* Common helper function to initialize the default address space for arm-smmu
diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_crtc.c b/drivers/gpu/drm/msm/disp/dpu1/dpu_crtc.c
index 967245b..e7c9fe1 100644
--- a/drivers/gpu/drm/msm/disp/dpu1/dpu_crtc.c
+++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_crtc.c
@@ -337,7 +337,8 @@ static void _dpu_crtc_program_lm_output_roi(struct drm_crtc *crtc)
}
static void _dpu_crtc_blend_setup_mixer(struct drm_crtc *crtc,
- struct dpu_crtc *dpu_crtc, struct dpu_crtc_mixer *mixer)
+ struct dpu_crtc *dpu_crtc, struct dpu_crtc_mixer *mixer,
+ struct dpu_hw_stage_cfg *stage_cfg)
{
struct drm_plane *plane;
struct drm_framebuffer *fb;
@@ -346,7 +347,6 @@ static void _dpu_crtc_blend_setup_mixer(struct drm_crtc *crtc,
struct dpu_plane_state *pstate = NULL;
struct dpu_format *format;
struct dpu_hw_ctl *ctl = mixer->lm_ctl;
- struct dpu_hw_stage_cfg *stage_cfg = &dpu_crtc->stage_cfg;
u32 flush_mask;
uint32_t stage_idx, lm_idx;
@@ -422,6 +422,7 @@ static void _dpu_crtc_blend_setup(struct drm_crtc *crtc)
struct dpu_crtc_mixer *mixer = cstate->mixers;
struct dpu_hw_ctl *ctl;
struct dpu_hw_mixer *lm;
+ struct dpu_hw_stage_cfg stage_cfg;
int i;
DRM_DEBUG_ATOMIC("%s\n", dpu_crtc->name);
@@ -435,9 +436,9 @@ static void _dpu_crtc_blend_setup(struct drm_crtc *crtc)
}
/* initialize stage cfg */
- memset(&dpu_crtc->stage_cfg, 0, sizeof(struct dpu_hw_stage_cfg));
+ memset(&stage_cfg, 0, sizeof(struct dpu_hw_stage_cfg));
- _dpu_crtc_blend_setup_mixer(crtc, dpu_crtc, mixer);
+ _dpu_crtc_blend_setup_mixer(crtc, dpu_crtc, mixer, &stage_cfg);
for (i = 0; i < cstate->num_mixers; i++) {
ctl = mixer[i].lm_ctl;
@@ -458,7 +459,7 @@ static void _dpu_crtc_blend_setup(struct drm_crtc *crtc)
mixer[i].flush_mask);
ctl->ops.setup_blendstage(ctl, mixer[i].hw_lm->idx,
- &dpu_crtc->stage_cfg);
+ &stage_cfg);
}
}
@@ -923,6 +924,20 @@ static struct drm_crtc_state *dpu_crtc_duplicate_state(struct drm_crtc *crtc)
return &cstate->base;
}
+static void dpu_crtc_atomic_print_state(struct drm_printer *p,
+ const struct drm_crtc_state *state)
+{
+ const struct dpu_crtc_state *cstate = to_dpu_crtc_state(state);
+ int i;
+
+ for (i = 0; i < cstate->num_mixers; i++) {
+ drm_printf(p, "\tlm[%d]=%d\n", i, cstate->mixers[i].hw_lm->idx - LM_0);
+ drm_printf(p, "\tctl[%d]=%d\n", i, cstate->mixers[i].lm_ctl->idx - CTL_0);
+ if (cstate->mixers[i].hw_dspp)
+ drm_printf(p, "\tdspp[%d]=%d\n", i, cstate->mixers[i].hw_dspp->idx - DSPP_0);
+ }
+}
+
static void dpu_crtc_disable(struct drm_crtc *crtc,
struct drm_atomic_state *state)
{
@@ -1423,15 +1438,16 @@ DEFINE_SHOW_ATTRIBUTE(dpu_crtc_debugfs_state);
static int _dpu_crtc_init_debugfs(struct drm_crtc *crtc)
{
struct dpu_crtc *dpu_crtc = to_dpu_crtc(crtc);
+ struct dentry *debugfs_root;
- dpu_crtc->debugfs_root = debugfs_create_dir(dpu_crtc->name,
+ debugfs_root = debugfs_create_dir(dpu_crtc->name,
crtc->dev->primary->debugfs_root);
debugfs_create_file("status", 0400,
- dpu_crtc->debugfs_root,
+ debugfs_root,
dpu_crtc, &_dpu_debugfs_status_fops);
debugfs_create_file("state", 0600,
- dpu_crtc->debugfs_root,
+ debugfs_root,
&dpu_crtc->base,
&dpu_crtc_debugfs_state_fops);
@@ -1449,13 +1465,6 @@ static int dpu_crtc_late_register(struct drm_crtc *crtc)
return _dpu_crtc_init_debugfs(crtc);
}
-static void dpu_crtc_early_unregister(struct drm_crtc *crtc)
-{
- struct dpu_crtc *dpu_crtc = to_dpu_crtc(crtc);
-
- debugfs_remove_recursive(dpu_crtc->debugfs_root);
-}
-
static const struct drm_crtc_funcs dpu_crtc_funcs = {
.set_config = drm_atomic_helper_set_config,
.destroy = dpu_crtc_destroy,
@@ -1463,8 +1472,8 @@ static const struct drm_crtc_funcs dpu_crtc_funcs = {
.reset = dpu_crtc_reset,
.atomic_duplicate_state = dpu_crtc_duplicate_state,
.atomic_destroy_state = dpu_crtc_destroy_state,
+ .atomic_print_state = dpu_crtc_atomic_print_state,
.late_register = dpu_crtc_late_register,
- .early_unregister = dpu_crtc_early_unregister,
.verify_crc_source = dpu_crtc_verify_crc_source,
.set_crc_source = dpu_crtc_set_crc_source,
.enable_vblank = msm_crtc_enable_vblank,
diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_crtc.h b/drivers/gpu/drm/msm/disp/dpu1/dpu_crtc.h
index ae9546c..b8785c3 100644
--- a/drivers/gpu/drm/msm/disp/dpu1/dpu_crtc.h
+++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_crtc.h
@@ -129,8 +129,6 @@ struct dpu_crtc_frame_event {
* @drm_requested_vblank : Whether vblanks have been enabled in the encoder
* @property_info : Opaque structure for generic property support
* @property_defaults : Array of default values for generic property support
- * @stage_cfg : H/w mixer stage configuration
- * @debugfs_root : Parent of debugfs node
* @vblank_cb_count : count of vblank callback since last reset
* @play_count : frame count between crtc enable and disable
* @vblank_cb_time : ktime at vblank count reset
@@ -161,9 +159,6 @@ struct dpu_crtc {
struct drm_pending_vblank_event *event;
u32 vsync_count;
- struct dpu_hw_stage_cfg stage_cfg;
- struct dentry *debugfs_root;
-
u32 vblank_cb_count;
u64 play_count;
ktime_t vblank_cb_time;
diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_encoder.c b/drivers/gpu/drm/msm/disp/dpu1/dpu_encoder.c
index e7ee4cf..1e648db 100644
--- a/drivers/gpu/drm/msm/disp/dpu1/dpu_encoder.c
+++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_encoder.c
@@ -995,9 +995,6 @@ static void dpu_encoder_virt_mode_set(struct drm_encoder *drm_enc,
trace_dpu_enc_mode_set(DRMID(drm_enc));
- if (drm_enc->encoder_type == DRM_MODE_ENCODER_TMDS)
- msm_dp_display_mode_set(dpu_enc->dp, drm_enc, mode, adj_mode);
-
list_for_each_entry(conn_iter, connector_list, head)
if (conn_iter->encoder == drm_enc)
conn = conn_iter;
@@ -1148,10 +1145,6 @@ static void dpu_encoder_virt_enable(struct drm_encoder *drm_enc)
struct msm_drm_private *priv;
struct drm_display_mode *cur_mode = NULL;
- if (!drm_enc) {
- DPU_ERROR("invalid encoder\n");
- return;
- }
dpu_enc = to_dpu_encoder_virt(drm_enc);
mutex_lock(&dpu_enc->enc_lock);
@@ -1177,14 +1170,6 @@ static void dpu_encoder_virt_enable(struct drm_encoder *drm_enc)
_dpu_encoder_virt_enable_helper(drm_enc);
- if (drm_enc->encoder_type == DRM_MODE_ENCODER_TMDS) {
- ret = msm_dp_display_enable(dpu_enc->dp, drm_enc);
- if (ret) {
- DPU_ERROR_ENC(dpu_enc, "dp display enable failed: %d\n",
- ret);
- goto out;
- }
- }
dpu_enc->enabled = true;
out:
@@ -1197,14 +1182,6 @@ static void dpu_encoder_virt_disable(struct drm_encoder *drm_enc)
struct msm_drm_private *priv;
int i = 0;
- if (!drm_enc) {
- DPU_ERROR("invalid encoder\n");
- return;
- } else if (!drm_enc->dev) {
- DPU_ERROR("invalid dev\n");
- return;
- }
-
dpu_enc = to_dpu_encoder_virt(drm_enc);
DPU_DEBUG_ENC(dpu_enc, "\n");
@@ -1218,11 +1195,6 @@ static void dpu_encoder_virt_disable(struct drm_encoder *drm_enc)
/* wait for idle */
dpu_encoder_wait_for_event(drm_enc, MSM_ENC_TX_COMPLETE);
- if (drm_enc->encoder_type == DRM_MODE_ENCODER_TMDS) {
- if (msm_dp_display_pre_disable(dpu_enc->dp, drm_enc))
- DPU_ERROR_ENC(dpu_enc, "dp display push idle failed\n");
- }
-
dpu_encoder_resource_control(drm_enc, DPU_ENC_RC_EVENT_PRE_STOP);
for (i = 0; i < dpu_enc->num_phys_encs; i++) {
@@ -1247,11 +1219,6 @@ static void dpu_encoder_virt_disable(struct drm_encoder *drm_enc)
DPU_DEBUG_ENC(dpu_enc, "encoder disabled\n");
- if (drm_enc->encoder_type == DRM_MODE_ENCODER_TMDS) {
- if (msm_dp_display_disable(dpu_enc->dp, drm_enc))
- DPU_ERROR_ENC(dpu_enc, "dp display disable failed\n");
- }
-
mutex_unlock(&dpu_enc->enc_lock);
}
@@ -2128,11 +2095,8 @@ static void dpu_encoder_frame_done_timeout(struct timer_list *t)
static const struct drm_encoder_helper_funcs dpu_encoder_helper_funcs = {
.mode_set = dpu_encoder_virt_mode_set,
.disable = dpu_encoder_virt_disable,
- .enable = dpu_kms_encoder_enable,
+ .enable = dpu_encoder_virt_enable,
.atomic_check = dpu_encoder_virt_atomic_check,
-
- /* This is called by dpu_kms_encoder_enable */
- .commit = dpu_encoder_virt_enable,
};
static const struct drm_encoder_funcs dpu_encoder_funcs = {
diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_encoder_phys_vid.c b/drivers/gpu/drm/msm/disp/dpu1/dpu_encoder_phys_vid.c
index 185379b..ddd9d89 100644
--- a/drivers/gpu/drm/msm/disp/dpu1/dpu_encoder_phys_vid.c
+++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_encoder_phys_vid.c
@@ -698,17 +698,17 @@ struct dpu_encoder_phys *dpu_encoder_phys_vid_init(
{
struct dpu_encoder_phys *phys_enc = NULL;
struct dpu_encoder_irq *irq;
- int i, ret = 0;
+ int i;
if (!p) {
- ret = -EINVAL;
- goto fail;
+ DPU_ERROR("failed to create encoder due to invalid parameter\n");
+ return ERR_PTR(-EINVAL);
}
phys_enc = kzalloc(sizeof(*phys_enc), GFP_KERNEL);
if (!phys_enc) {
- ret = -ENOMEM;
- goto fail;
+ DPU_ERROR("failed to create encoder due to memory allocation error\n");
+ return ERR_PTR(-ENOMEM);
}
phys_enc->hw_mdptop = p->dpu_kms->hw_mdp;
@@ -748,11 +748,4 @@ struct dpu_encoder_phys *dpu_encoder_phys_vid_init(
DPU_DEBUG_VIDENC(phys_enc, "created intf idx:%d\n", p->intf_idx);
return phys_enc;
-
-fail:
- DPU_ERROR("failed to create encoder\n");
- if (phys_enc)
- dpu_encoder_phys_vid_destroy(phys_enc);
-
- return ERR_PTR(ret);
}
diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_catalog.c b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_catalog.c
index ce6f32a..aa75991 100644
--- a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_catalog.c
+++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_catalog.c
@@ -45,7 +45,7 @@
(PINGPONG_SDM845_MASK | BIT(DPU_PINGPONG_TE2))
#define CTL_SC7280_MASK \
- (BIT(DPU_CTL_ACTIVE_CFG) | BIT(DPU_CTL_FETCH_ACTIVE))
+ (BIT(DPU_CTL_ACTIVE_CFG) | BIT(DPU_CTL_FETCH_ACTIVE) | BIT(DPU_CTL_VM_CFG))
#define MERGE_3D_SM8150_MASK (0)
@@ -856,9 +856,9 @@ static const struct dpu_intf_cfg sm8150_intf[] = {
};
static const struct dpu_intf_cfg sc7280_intf[] = {
- INTF_BLK("intf_0", INTF_0, 0x34000, INTF_DP, 0, 24, INTF_SC7280_MASK, MDP_SSPP_TOP0_INTR, 24, 25),
+ INTF_BLK("intf_0", INTF_0, 0x34000, INTF_DP, MSM_DP_CONTROLLER_0, 24, INTF_SC7280_MASK, MDP_SSPP_TOP0_INTR, 24, 25),
INTF_BLK("intf_1", INTF_1, 0x35000, INTF_DSI, 0, 24, INTF_SC7280_MASK, MDP_SSPP_TOP0_INTR, 26, 27),
- INTF_BLK("intf_5", INTF_5, 0x39000, INTF_EDP, 0, 24, INTF_SC7280_MASK, MDP_SSPP_TOP0_INTR, 22, 23),
+ INTF_BLK("intf_5", INTF_5, 0x39000, INTF_DP, MSM_DP_CONTROLLER_1, 24, INTF_SC7280_MASK, MDP_SSPP_TOP0_INTR, 22, 23),
};
/*************************************************************
diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_catalog.h b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_catalog.h
index 4ade44b..31af04a 100644
--- a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_catalog.h
+++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_catalog.h
@@ -179,13 +179,16 @@ enum {
/**
* CTL sub-blocks
- * @DPU_CTL_SPLIT_DISPLAY CTL supports video mode split display
+ * @DPU_CTL_SPLIT_DISPLAY: CTL supports video mode split display
+ * @DPU_CTL_FETCH_ACTIVE: Active CTL for fetch HW (SSPPs)
+ * @DPU_CTL_VM_CFG: CTL config to support multiple VMs
* @DPU_CTL_MAX
*/
enum {
DPU_CTL_SPLIT_DISPLAY = 0x1,
DPU_CTL_ACTIVE_CFG,
DPU_CTL_FETCH_ACTIVE,
+ DPU_CTL_VM_CFG,
DPU_CTL_MAX
};
diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_ctl.c b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_ctl.c
index 64740ddb..02da9ec 100644
--- a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_ctl.c
+++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_ctl.c
@@ -36,6 +36,7 @@
#define MERGE_3D_IDX 23
#define INTF_IDX 31
#define CTL_INVALID_BIT 0xffff
+#define CTL_DEFAULT_GROUP_ID 0xf
static const u32 fetch_tbl[SSPP_MAX] = {CTL_INVALID_BIT, 16, 17, 18, 19,
CTL_INVALID_BIT, CTL_INVALID_BIT, CTL_INVALID_BIT, CTL_INVALID_BIT, 0,
@@ -498,6 +499,13 @@ static void dpu_hw_ctl_intf_cfg_v1(struct dpu_hw_ctl *ctx,
u32 intf_active = 0;
u32 mode_sel = 0;
+ /* CTL_TOP[31:28] carries group_id to collate CTL paths
+ * per VM. Explicitly disable it until VM support is
+ * added in SW. Power on reset value is not disable.
+ */
+ if ((test_bit(DPU_CTL_VM_CFG, &ctx->caps->features)))
+ mode_sel = CTL_DEFAULT_GROUP_ID << 28;
+
if (cfg->intf_mode_sel == DPU_CTL_MODE_SEL_CMD)
mode_sel |= BIT(17);
diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_interrupts.c b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_interrupts.c
index d2b6dca..a77a5ea 100644
--- a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_interrupts.c
+++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_interrupts.c
@@ -30,6 +30,9 @@
#define MDP_AD4_INTR_STATUS_OFF 0x420
#define MDP_INTF_0_OFF_REV_7xxx 0x34000
#define MDP_INTF_1_OFF_REV_7xxx 0x35000
+#define MDP_INTF_2_OFF_REV_7xxx 0x36000
+#define MDP_INTF_3_OFF_REV_7xxx 0x37000
+#define MDP_INTF_4_OFF_REV_7xxx 0x38000
#define MDP_INTF_5_OFF_REV_7xxx 0x39000
/**
@@ -111,6 +114,21 @@ static const struct dpu_intr_reg dpu_intr_set[] = {
MDP_INTF_1_OFF_REV_7xxx+INTF_INTR_STATUS
},
{
+ MDP_INTF_2_OFF_REV_7xxx+INTF_INTR_CLEAR,
+ MDP_INTF_2_OFF_REV_7xxx+INTF_INTR_EN,
+ MDP_INTF_2_OFF_REV_7xxx+INTF_INTR_STATUS
+ },
+ {
+ MDP_INTF_3_OFF_REV_7xxx+INTF_INTR_CLEAR,
+ MDP_INTF_3_OFF_REV_7xxx+INTF_INTR_EN,
+ MDP_INTF_3_OFF_REV_7xxx+INTF_INTR_STATUS
+ },
+ {
+ MDP_INTF_4_OFF_REV_7xxx+INTF_INTR_CLEAR,
+ MDP_INTF_4_OFF_REV_7xxx+INTF_INTR_EN,
+ MDP_INTF_4_OFF_REV_7xxx+INTF_INTR_STATUS
+ },
+ {
MDP_INTF_5_OFF_REV_7xxx+INTF_INTR_CLEAR,
MDP_INTF_5_OFF_REV_7xxx+INTF_INTR_EN,
MDP_INTF_5_OFF_REV_7xxx+INTF_INTR_STATUS
diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_interrupts.h b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_interrupts.h
index d50e78c..1ab75cc 100644
--- a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_interrupts.h
+++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_interrupts.h
@@ -26,6 +26,9 @@ enum dpu_hw_intr_reg {
MDP_AD4_1_INTR,
MDP_INTF0_7xxx_INTR,
MDP_INTF1_7xxx_INTR,
+ MDP_INTF2_7xxx_INTR,
+ MDP_INTF3_7xxx_INTR,
+ MDP_INTF4_7xxx_INTR,
MDP_INTF5_7xxx_INTR,
MDP_INTR_MAX,
};
diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_sspp.c b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_sspp.c
index f946067..09cdc35 100644
--- a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_sspp.c
+++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_sspp.c
@@ -8,6 +8,8 @@
#include "dpu_hw_sspp.h"
#include "dpu_kms.h"
+#include <drm/drm_file.h>
+
#define DPU_FETCH_CONFIG_RESET_VALUE 0x00000087
/* DPU_SSPP_SRC */
@@ -75,6 +77,7 @@
#define SSPP_TRAFFIC_SHAPER 0x130
#define SSPP_CDP_CNTL 0x134
#define SSPP_UBWC_ERROR_STATUS 0x138
+#define SSPP_CDP_CNTL_REC1 0x13c
#define SSPP_TRAFFIC_SHAPER_PREFILL 0x150
#define SSPP_TRAFFIC_SHAPER_REC1_PREFILL 0x154
#define SSPP_TRAFFIC_SHAPER_REC1 0x158
@@ -413,13 +416,11 @@ static void dpu_hw_sspp_setup_pe_config(struct dpu_hw_pipe *ctx,
static void _dpu_hw_sspp_setup_scaler3(struct dpu_hw_pipe *ctx,
struct dpu_hw_pipe_cfg *sspp,
- struct dpu_hw_pixel_ext *pe,
void *scaler_cfg)
{
u32 idx;
struct dpu_hw_scaler3_cfg *scaler3_cfg = scaler_cfg;
- (void)pe;
if (_sspp_subblk_offset(ctx, DPU_SSPP_SCALER_QSEED3, &idx) || !sspp
|| !scaler3_cfg)
return;
@@ -539,7 +540,7 @@ static void dpu_hw_sspp_setup_sourceaddress(struct dpu_hw_pipe *ctx,
}
static void dpu_hw_sspp_setup_csc(struct dpu_hw_pipe *ctx,
- struct dpu_csc_cfg *data)
+ const struct dpu_csc_cfg *data)
{
u32 idx;
bool csc10 = false;
@@ -571,19 +572,20 @@ static void dpu_hw_sspp_setup_solidfill(struct dpu_hw_pipe *ctx, u32 color, enum
}
static void dpu_hw_sspp_setup_danger_safe_lut(struct dpu_hw_pipe *ctx,
- struct dpu_hw_pipe_qos_cfg *cfg)
+ u32 danger_lut,
+ u32 safe_lut)
{
u32 idx;
if (_sspp_subblk_offset(ctx, DPU_SSPP_SRC, &idx))
return;
- DPU_REG_WRITE(&ctx->hw, SSPP_DANGER_LUT + idx, cfg->danger_lut);
- DPU_REG_WRITE(&ctx->hw, SSPP_SAFE_LUT + idx, cfg->safe_lut);
+ DPU_REG_WRITE(&ctx->hw, SSPP_DANGER_LUT + idx, danger_lut);
+ DPU_REG_WRITE(&ctx->hw, SSPP_SAFE_LUT + idx, safe_lut);
}
static void dpu_hw_sspp_setup_creq_lut(struct dpu_hw_pipe *ctx,
- struct dpu_hw_pipe_qos_cfg *cfg)
+ u64 creq_lut)
{
u32 idx;
@@ -591,11 +593,11 @@ static void dpu_hw_sspp_setup_creq_lut(struct dpu_hw_pipe *ctx,
return;
if (ctx->cap && test_bit(DPU_SSPP_QOS_8LVL, &ctx->cap->features)) {
- DPU_REG_WRITE(&ctx->hw, SSPP_CREQ_LUT_0 + idx, cfg->creq_lut);
+ DPU_REG_WRITE(&ctx->hw, SSPP_CREQ_LUT_0 + idx, creq_lut);
DPU_REG_WRITE(&ctx->hw, SSPP_CREQ_LUT_1 + idx,
- cfg->creq_lut >> 32);
+ creq_lut >> 32);
} else {
- DPU_REG_WRITE(&ctx->hw, SSPP_CREQ_LUT + idx, cfg->creq_lut);
+ DPU_REG_WRITE(&ctx->hw, SSPP_CREQ_LUT + idx, creq_lut);
}
}
@@ -625,10 +627,12 @@ static void dpu_hw_sspp_setup_qos_ctrl(struct dpu_hw_pipe *ctx,
}
static void dpu_hw_sspp_setup_cdp(struct dpu_hw_pipe *ctx,
- struct dpu_hw_pipe_cdp_cfg *cfg)
+ struct dpu_hw_pipe_cdp_cfg *cfg,
+ enum dpu_sspp_multirect_index index)
{
u32 idx;
u32 cdp_cntl = 0;
+ u32 cdp_cntl_offset = 0;
if (!ctx || !cfg)
return;
@@ -636,6 +640,11 @@ static void dpu_hw_sspp_setup_cdp(struct dpu_hw_pipe *ctx,
if (_sspp_subblk_offset(ctx, DPU_SSPP_SRC, &idx))
return;
+ if (index == DPU_SSPP_RECT_SOLO || index == DPU_SSPP_RECT_0)
+ cdp_cntl_offset = SSPP_CDP_CNTL;
+ else
+ cdp_cntl_offset = SSPP_CDP_CNTL_REC1;
+
if (cfg->enable)
cdp_cntl |= BIT(0);
if (cfg->ubwc_meta_enable)
@@ -645,7 +654,7 @@ static void dpu_hw_sspp_setup_cdp(struct dpu_hw_pipe *ctx,
if (cfg->preload_ahead == DPU_SSPP_CDP_PRELOAD_AHEAD_64)
cdp_cntl |= BIT(3);
- DPU_REG_WRITE(&ctx->hw, SSPP_CDP_CNTL, cdp_cntl);
+ DPU_REG_WRITE(&ctx->hw, cdp_cntl_offset, cdp_cntl);
}
static void _setup_layer_ops(struct dpu_hw_pipe *c,
@@ -685,6 +694,71 @@ static void _setup_layer_ops(struct dpu_hw_pipe *c,
c->ops.setup_cdp = dpu_hw_sspp_setup_cdp;
}
+#ifdef CONFIG_DEBUG_FS
+int _dpu_hw_sspp_init_debugfs(struct dpu_hw_pipe *hw_pipe, struct dpu_kms *kms, struct dentry *entry)
+{
+ const struct dpu_sspp_cfg *cfg = hw_pipe->cap;
+ const struct dpu_sspp_sub_blks *sblk = cfg->sblk;
+ struct dentry *debugfs_root;
+ char sspp_name[32];
+
+ snprintf(sspp_name, sizeof(sspp_name), "%d", hw_pipe->idx);
+
+ /* create overall sub-directory for the pipe */
+ debugfs_root =
+ debugfs_create_dir(sspp_name, entry);
+
+ /* don't error check these */
+ debugfs_create_xul("features", 0600,
+ debugfs_root, (unsigned long *)&hw_pipe->cap->features);
+
+ /* add register dump support */
+ dpu_debugfs_create_regset32("src_blk", 0400,
+ debugfs_root,
+ sblk->src_blk.base + cfg->base,
+ sblk->src_blk.len,
+ kms);
+
+ if (cfg->features & BIT(DPU_SSPP_SCALER_QSEED3) ||
+ cfg->features & BIT(DPU_SSPP_SCALER_QSEED3LITE) ||
+ cfg->features & BIT(DPU_SSPP_SCALER_QSEED2) ||
+ cfg->features & BIT(DPU_SSPP_SCALER_QSEED4))
+ dpu_debugfs_create_regset32("scaler_blk", 0400,
+ debugfs_root,
+ sblk->scaler_blk.base + cfg->base,
+ sblk->scaler_blk.len,
+ kms);
+
+ if (cfg->features & BIT(DPU_SSPP_CSC) ||
+ cfg->features & BIT(DPU_SSPP_CSC_10BIT))
+ dpu_debugfs_create_regset32("csc_blk", 0400,
+ debugfs_root,
+ sblk->csc_blk.base + cfg->base,
+ sblk->csc_blk.len,
+ kms);
+
+ debugfs_create_u32("xin_id",
+ 0400,
+ debugfs_root,
+ (u32 *) &cfg->xin_id);
+ debugfs_create_u32("clk_ctrl",
+ 0400,
+ debugfs_root,
+ (u32 *) &cfg->clk_ctrl);
+ debugfs_create_x32("creq_vblank",
+ 0600,
+ debugfs_root,
+ (u32 *) &sblk->creq_vblank);
+ debugfs_create_x32("danger_vblank",
+ 0600,
+ debugfs_root,
+ (u32 *) &sblk->danger_vblank);
+
+ return 0;
+}
+#endif
+
+
static const struct dpu_sspp_cfg *_sspp_offset(enum dpu_sspp sspp,
void __iomem *addr,
struct dpu_mdss_cfg *catalog,
diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_sspp.h b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_sspp.h
index fdfd4b4..92b071b 100644
--- a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_sspp.h
+++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_sspp.h
@@ -25,11 +25,17 @@ struct dpu_hw_pipe;
/**
* Define all scaler feature bits in catalog
*/
-#define DPU_SSPP_SCALER ((1UL << DPU_SSPP_SCALER_RGB) | \
- (1UL << DPU_SSPP_SCALER_QSEED2) | \
- (1UL << DPU_SSPP_SCALER_QSEED3) | \
- (1UL << DPU_SSPP_SCALER_QSEED3LITE) | \
- (1UL << DPU_SSPP_SCALER_QSEED4))
+#define DPU_SSPP_SCALER (BIT(DPU_SSPP_SCALER_RGB) | \
+ BIT(DPU_SSPP_SCALER_QSEED2) | \
+ BIT(DPU_SSPP_SCALER_QSEED3) | \
+ BIT(DPU_SSPP_SCALER_QSEED3LITE) | \
+ BIT(DPU_SSPP_SCALER_QSEED4))
+
+/*
+ * Define all CSC feature bits in catalog
+ */
+#define DPU_SSPP_CSC_ANY (BIT(DPU_SSPP_CSC) | \
+ BIT(DPU_SSPP_CSC_10BIT))
/**
* Component indices
@@ -166,18 +172,12 @@ struct dpu_hw_pipe_cfg {
/**
* struct dpu_hw_pipe_qos_cfg : Source pipe QoS configuration
- * @danger_lut: LUT for generate danger level based on fill level
- * @safe_lut: LUT for generate safe level based on fill level
- * @creq_lut: LUT for generate creq level based on fill level
* @creq_vblank: creq value generated to vbif during vertical blanking
* @danger_vblank: danger value generated during vertical blanking
* @vblank_en: enable creq_vblank and danger_vblank during vblank
* @danger_safe_en: enable danger safe generation
*/
struct dpu_hw_pipe_qos_cfg {
- u32 danger_lut;
- u32 safe_lut;
- u64 creq_lut;
u32 creq_vblank;
u32 danger_vblank;
bool vblank_en;
@@ -268,7 +268,7 @@ struct dpu_hw_sspp_ops {
* @ctx: Pointer to pipe context
* @data: Pointer to config structure
*/
- void (*setup_csc)(struct dpu_hw_pipe *ctx, struct dpu_csc_cfg *data);
+ void (*setup_csc)(struct dpu_hw_pipe *ctx, const struct dpu_csc_cfg *data);
/**
* setup_solidfill - enable/disable colorfill
@@ -302,20 +302,22 @@ struct dpu_hw_sspp_ops {
/**
* setup_danger_safe_lut - setup danger safe LUTs
* @ctx: Pointer to pipe context
- * @cfg: Pointer to pipe QoS configuration
+ * @danger_lut: LUT for generate danger level based on fill level
+ * @safe_lut: LUT for generate safe level based on fill level
*
*/
void (*setup_danger_safe_lut)(struct dpu_hw_pipe *ctx,
- struct dpu_hw_pipe_qos_cfg *cfg);
+ u32 danger_lut,
+ u32 safe_lut);
/**
* setup_creq_lut - setup CREQ LUT
* @ctx: Pointer to pipe context
- * @cfg: Pointer to pipe QoS configuration
+ * @creq_lut: LUT for generate creq level based on fill level
*
*/
void (*setup_creq_lut)(struct dpu_hw_pipe *ctx,
- struct dpu_hw_pipe_qos_cfg *cfg);
+ u64 creq_lut);
/**
* setup_qos_ctrl - setup QoS control
@@ -338,12 +340,10 @@ struct dpu_hw_sspp_ops {
* setup_scaler - setup scaler
* @ctx: Pointer to pipe context
* @pipe_cfg: Pointer to pipe configuration
- * @pe_cfg: Pointer to pixel extension configuration
* @scaler_cfg: Pointer to scaler configuration
*/
void (*setup_scaler)(struct dpu_hw_pipe *ctx,
struct dpu_hw_pipe_cfg *pipe_cfg,
- struct dpu_hw_pixel_ext *pe_cfg,
void *scaler_cfg);
/**
@@ -356,9 +356,11 @@ struct dpu_hw_sspp_ops {
* setup_cdp - setup client driven prefetch
* @ctx: Pointer to pipe context
* @cfg: Pointer to cdp configuration
+ * @index: rectangle index in multirect
*/
void (*setup_cdp)(struct dpu_hw_pipe *ctx,
- struct dpu_hw_pipe_cdp_cfg *cfg);
+ struct dpu_hw_pipe_cdp_cfg *cfg,
+ enum dpu_sspp_multirect_index index);
};
/**
@@ -385,6 +387,7 @@ struct dpu_hw_pipe {
struct dpu_hw_sspp_ops ops;
};
+struct dpu_kms;
/**
* dpu_hw_sspp_init - initializes the sspp hw driver object.
* Should be called once before accessing every pipe.
@@ -404,5 +407,8 @@ struct dpu_hw_pipe *dpu_hw_sspp_init(enum dpu_sspp idx,
*/
void dpu_hw_sspp_destroy(struct dpu_hw_pipe *ctx);
+void dpu_debugfs_sspp_init(struct dpu_kms *dpu_kms, struct dentry *debugfs_root);
+int _dpu_hw_sspp_init_debugfs(struct dpu_hw_pipe *hw_pipe, struct dpu_kms *kms, struct dentry *entry);
+
#endif /*_DPU_HW_SSPP_H */
diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_util.c b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_util.c
index f94584c..aad8511 100644
--- a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_util.c
+++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_util.c
@@ -374,7 +374,7 @@ u32 dpu_hw_get_scaler3_ver(struct dpu_hw_blk_reg_map *c,
void dpu_hw_csc_setup(struct dpu_hw_blk_reg_map *c,
u32 csc_reg_off,
- struct dpu_csc_cfg *data, bool csc10)
+ const struct dpu_csc_cfg *data, bool csc10)
{
static const u32 matrix_shift = 7;
u32 clamp_shift = csc10 ? 16 : 8;
diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_util.h b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_util.h
index 6d49119..3913475 100644
--- a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_util.h
+++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_util.h
@@ -322,6 +322,6 @@ u32 dpu_hw_get_scaler3_ver(struct dpu_hw_blk_reg_map *c,
void dpu_hw_csc_setup(struct dpu_hw_blk_reg_map *c,
u32 csc_reg_off,
- struct dpu_csc_cfg *data, bool csc10);
+ const struct dpu_csc_cfg *data, bool csc10);
#endif /* _DPU_HW_UTIL_H */
diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_kms.c b/drivers/gpu/drm/msm/disp/dpu1/dpu_kms.c
index a15b264..47fe11a 100644
--- a/drivers/gpu/drm/msm/disp/dpu1/dpu_kms.c
+++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_kms.c
@@ -21,14 +21,14 @@
#include "msm_gem.h"
#include "disp/msm_disp_snapshot.h"
-#include "dpu_kms.h"
#include "dpu_core_irq.h"
+#include "dpu_crtc.h"
+#include "dpu_encoder.h"
#include "dpu_formats.h"
#include "dpu_hw_vbif.h"
-#include "dpu_vbif.h"
-#include "dpu_encoder.h"
+#include "dpu_kms.h"
#include "dpu_plane.h"
-#include "dpu_crtc.h"
+#include "dpu_vbif.h"
#define CREATE_TRACE_POINTS
#include "dpu_trace.h"
@@ -73,8 +73,8 @@ static int _dpu_danger_signal_status(struct seq_file *s,
&status);
} else {
seq_puts(s, "\nSafe signal status:\n");
- if (kms->hw_mdp->ops.get_danger_status)
- kms->hw_mdp->ops.get_danger_status(kms->hw_mdp,
+ if (kms->hw_mdp->ops.get_safe_status)
+ kms->hw_mdp->ops.get_safe_status(kms->hw_mdp,
&status);
}
pm_runtime_put_sync(&kms->pdev->dev);
@@ -82,7 +82,7 @@ static int _dpu_danger_signal_status(struct seq_file *s,
seq_printf(s, "MDP : 0x%x\n", status.mdp);
for (i = SSPP_VIG0; i < SSPP_MAX; i++)
- seq_printf(s, "SSPP%d : 0x%x \t", i - SSPP_VIG0,
+ seq_printf(s, "SSPP%d : 0x%x \n", i - SSPP_VIG0,
status.sspp[i]);
seq_puts(s, "\n");
@@ -101,6 +101,73 @@ static int dpu_debugfs_safe_stats_show(struct seq_file *s, void *v)
}
DEFINE_SHOW_ATTRIBUTE(dpu_debugfs_safe_stats);
+static ssize_t _dpu_plane_danger_read(struct file *file,
+ char __user *buff, size_t count, loff_t *ppos)
+{
+ struct dpu_kms *kms = file->private_data;
+ int len;
+ char buf[40];
+
+ len = scnprintf(buf, sizeof(buf), "%d\n", !kms->has_danger_ctrl);
+
+ return simple_read_from_buffer(buff, count, ppos, buf, len);
+}
+
+static void _dpu_plane_set_danger_state(struct dpu_kms *kms, bool enable)
+{
+ struct drm_plane *plane;
+
+ drm_for_each_plane(plane, kms->dev) {
+ if (plane->fb && plane->state) {
+ dpu_plane_danger_signal_ctrl(plane, enable);
+ DPU_DEBUG("plane:%d img:%dx%d ",
+ plane->base.id, plane->fb->width,
+ plane->fb->height);
+ DPU_DEBUG("src[%d,%d,%d,%d] dst[%d,%d,%d,%d]\n",
+ plane->state->src_x >> 16,
+ plane->state->src_y >> 16,
+ plane->state->src_w >> 16,
+ plane->state->src_h >> 16,
+ plane->state->crtc_x, plane->state->crtc_y,
+ plane->state->crtc_w, plane->state->crtc_h);
+ } else {
+ DPU_DEBUG("Inactive plane:%d\n", plane->base.id);
+ }
+ }
+}
+
+static ssize_t _dpu_plane_danger_write(struct file *file,
+ const char __user *user_buf, size_t count, loff_t *ppos)
+{
+ struct dpu_kms *kms = file->private_data;
+ int disable_panic;
+ int ret;
+
+ ret = kstrtouint_from_user(user_buf, count, 0, &disable_panic);
+ if (ret)
+ return ret;
+
+ if (disable_panic) {
+ /* Disable panic signal for all active pipes */
+ DPU_DEBUG("Disabling danger:\n");
+ _dpu_plane_set_danger_state(kms, false);
+ kms->has_danger_ctrl = false;
+ } else {
+ /* Enable panic signal for all active pipes */
+ DPU_DEBUG("Enabling danger:\n");
+ kms->has_danger_ctrl = true;
+ _dpu_plane_set_danger_state(kms, true);
+ }
+
+ return count;
+}
+
+static const struct file_operations dpu_plane_danger_enable = {
+ .open = simple_open,
+ .read = _dpu_plane_danger_read,
+ .write = _dpu_plane_danger_write,
+};
+
static void dpu_debugfs_danger_init(struct dpu_kms *dpu_kms,
struct dentry *parent)
{
@@ -110,8 +177,20 @@ static void dpu_debugfs_danger_init(struct dpu_kms *dpu_kms,
dpu_kms, &dpu_debugfs_danger_stats_fops);
debugfs_create_file("safe_status", 0600, entry,
dpu_kms, &dpu_debugfs_safe_stats_fops);
+ debugfs_create_file("disable_danger", 0600, entry,
+ dpu_kms, &dpu_plane_danger_enable);
+
}
+/*
+ * Companion structure for dpu_debugfs_create_regset32.
+ */
+struct dpu_debugfs_regset32 {
+ uint32_t offset;
+ uint32_t blk_len;
+ struct dpu_kms *dpu_kms;
+};
+
static int _dpu_debugfs_show_regset32(struct seq_file *s, void *data)
{
struct dpu_debugfs_regset32 *regset = s->private;
@@ -159,24 +238,23 @@ static const struct file_operations dpu_fops_regset32 = {
.release = single_release,
};
-void dpu_debugfs_setup_regset32(struct dpu_debugfs_regset32 *regset,
+void dpu_debugfs_create_regset32(const char *name, umode_t mode,
+ void *parent,
uint32_t offset, uint32_t length, struct dpu_kms *dpu_kms)
{
- if (regset) {
- regset->offset = offset;
- regset->blk_len = length;
- regset->dpu_kms = dpu_kms;
- }
-}
+ struct dpu_debugfs_regset32 *regset;
-void dpu_debugfs_create_regset32(const char *name, umode_t mode,
- void *parent, struct dpu_debugfs_regset32 *regset)
-{
- if (!name || !regset || !regset->dpu_kms || !regset->blk_len)
+ if (WARN_ON(!name || !dpu_kms || !length))
+ return;
+
+ regset = devm_kzalloc(&dpu_kms->pdev->dev, sizeof(*regset), GFP_KERNEL);
+ if (!regset)
return;
/* make sure offset is a multiple of 4 */
- regset->offset = round_down(regset->offset, 4);
+ regset->offset = round_down(offset, 4);
+ regset->blk_len = length;
+ regset->dpu_kms = dpu_kms;
debugfs_create_file(name, mode, parent, regset, &dpu_fops_regset32);
}
@@ -203,6 +281,7 @@ static int dpu_kms_debugfs_init(struct msm_kms *kms, struct drm_minor *minor)
dpu_debugfs_danger_init(dpu_kms, entry);
dpu_debugfs_vbif_init(dpu_kms, entry);
dpu_debugfs_core_irq_init(dpu_kms, entry);
+ dpu_debugfs_sspp_init(dpu_kms, entry);
for (i = 0; i < ARRAY_SIZE(priv->dp); i++) {
if (priv->dp[i])
@@ -384,28 +463,6 @@ static void dpu_kms_flush_commit(struct msm_kms *kms, unsigned crtc_mask)
}
}
-/*
- * Override the encoder enable since we need to setup the inline rotator and do
- * some crtc magic before enabling any bridge that might be present.
- */
-void dpu_kms_encoder_enable(struct drm_encoder *encoder)
-{
- const struct drm_encoder_helper_funcs *funcs = encoder->helper_private;
- struct drm_device *dev = encoder->dev;
- struct drm_crtc *crtc;
-
- /* Forward this enable call to the commit hook */
- if (funcs && funcs->commit)
- funcs->commit(encoder);
-
- drm_for_each_crtc(crtc, dev) {
- if (!(crtc->state->encoder_mask & drm_encoder_mask(encoder)))
- continue;
-
- trace_dpu_kms_enc_enable(DRMID(crtc));
- }
-}
-
static void dpu_kms_complete_commit(struct msm_kms *kms, unsigned crtc_mask)
{
struct dpu_kms *dpu_kms = to_dpu_kms(kms);
@@ -863,6 +920,11 @@ static void dpu_kms_mdp_snapshot(struct msm_disp_state *disp_state, struct msm_k
msm_disp_snapshot_add_block(disp_state, cat->sspp[i].len,
dpu_kms->mmio + cat->sspp[i].base, "sspp_%d", i);
+ /* dump LM sub-blocks HW regs info */
+ for (i = 0; i < cat->mixer_count; i++)
+ msm_disp_snapshot_add_block(disp_state, cat->mixer[i].len,
+ dpu_kms->mmio + cat->mixer[i].base, "lm_%d", i);
+
msm_disp_snapshot_add_block(disp_state, top->hw.length,
dpu_kms->mmio + top->hw.blk_off, "top");
@@ -1153,9 +1215,9 @@ struct msm_kms *dpu_kms_init(struct drm_device *dev)
static int dpu_bind(struct device *dev, struct device *master, void *data)
{
- struct drm_device *ddev = dev_get_drvdata(master);
+ struct msm_drm_private *priv = dev_get_drvdata(master);
struct platform_device *pdev = to_platform_device(dev);
- struct msm_drm_private *priv = ddev->dev_private;
+ struct drm_device *ddev = priv->dev;
struct dpu_kms *dpu_kms;
struct dss_module_power *mp;
int ret = 0;
@@ -1285,7 +1347,7 @@ static const struct dev_pm_ops dpu_pm_ops = {
pm_runtime_force_resume)
};
-static const struct of_device_id dpu_dt_match[] = {
+const struct of_device_id dpu_dt_match[] = {
{ .compatible = "qcom,sdm845-dpu", },
{ .compatible = "qcom,sc7180-dpu", },
{ .compatible = "qcom,sc7280-dpu", },
diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_kms.h b/drivers/gpu/drm/msm/disp/dpu1/dpu_kms.h
index 775bcbd..2d385b4 100644
--- a/drivers/gpu/drm/msm/disp/dpu1/dpu_kms.h
+++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_kms.h
@@ -160,34 +160,10 @@ struct dpu_global_state
*
* Documentation/filesystems/debugfs.rst
*
- * @dpu_debugfs_setup_regset32: Initialize data for dpu_debugfs_create_regset32
* @dpu_debugfs_create_regset32: Create 32-bit register dump file
- * @dpu_debugfs_get_root: Get root dentry for DPU_KMS's debugfs node
*/
/**
- * Companion structure for dpu_debugfs_create_regset32. Do not initialize the
- * members of this structure explicitly; use dpu_debugfs_setup_regset32 instead.
- */
-struct dpu_debugfs_regset32 {
- uint32_t offset;
- uint32_t blk_len;
- struct dpu_kms *dpu_kms;
-};
-
-/**
- * dpu_debugfs_setup_regset32 - Initialize register block definition for debugfs
- * This function is meant to initialize dpu_debugfs_regset32 structures for use
- * with dpu_debugfs_create_regset32.
- * @regset: opaque register definition structure
- * @offset: sub-block offset
- * @length: sub-block length, in bytes
- * @dpu_kms: pointer to dpu kms structure
- */
-void dpu_debugfs_setup_regset32(struct dpu_debugfs_regset32 *regset,
- uint32_t offset, uint32_t length, struct dpu_kms *dpu_kms);
-
-/**
* dpu_debugfs_create_regset32 - Create register read back file for debugfs
*
* This function is almost identical to the standard debugfs_create_regset32()
@@ -195,20 +171,16 @@ void dpu_debugfs_setup_regset32(struct dpu_debugfs_regset32 *regset,
* names/offsets do not need to be provided. The 'read' function simply outputs
* sequential register values over a specified range.
*
- * Similar to the related debugfs_create_regset32 API, the structure pointed to
- * by regset needs to persist for the lifetime of the created file. The calling
- * code is responsible for initialization/management of this structure.
- *
- * The structure pointed to by regset is meant to be opaque. Please use
- * dpu_debugfs_setup_regset32 to initialize it.
- *
* @name: File name within debugfs
* @mode: File mode within debugfs
* @parent: Parent directory entry within debugfs, can be NULL
- * @regset: Pointer to persistent register block definition
+ * @offset: sub-block offset
+ * @length: sub-block length, in bytes
+ * @dpu_kms: pointer to dpu kms structure
*/
void dpu_debugfs_create_regset32(const char *name, umode_t mode,
- void *parent, struct dpu_debugfs_regset32 *regset);
+ void *parent,
+ uint32_t offset, uint32_t length, struct dpu_kms *dpu_kms);
/**
* dpu_debugfs_get_root - Return root directory entry for KMS's debugfs
@@ -235,8 +207,6 @@ void *dpu_debugfs_get_root(struct dpu_kms *dpu_kms);
int dpu_enable_vblank(struct msm_kms *kms, struct drm_crtc *crtc);
void dpu_disable_vblank(struct msm_kms *kms, struct drm_crtc *crtc);
-void dpu_kms_encoder_enable(struct drm_encoder *encoder);
-
/**
* dpu_kms_get_clk_rate() - get the clock rate
* @dpu_kms: pointer to dpu_kms structure
diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_mdss.c b/drivers/gpu/drm/msm/disp/dpu1/dpu_mdss.c
index b466784..131c1f1 100644
--- a/drivers/gpu/drm/msm/disp/dpu1/dpu_mdss.c
+++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_mdss.c
@@ -111,7 +111,7 @@ static int _dpu_mdss_irq_domain_add(struct dpu_mdss *dpu_mdss)
struct device *dev;
struct irq_domain *domain;
- dev = dpu_mdss->base.dev->dev;
+ dev = dpu_mdss->base.dev;
domain = irq_domain_add_linear(dev->of_node, 32,
&dpu_mdss_irqdomain_ops, dpu_mdss);
@@ -184,16 +184,15 @@ static int dpu_mdss_disable(struct msm_mdss *mdss)
return ret;
}
-static void dpu_mdss_destroy(struct drm_device *dev)
+static void dpu_mdss_destroy(struct msm_mdss *mdss)
{
- struct platform_device *pdev = to_platform_device(dev->dev);
- struct msm_drm_private *priv = dev->dev_private;
- struct dpu_mdss *dpu_mdss = to_dpu_mdss(priv->mdss);
+ struct platform_device *pdev = to_platform_device(mdss->dev);
+ struct dpu_mdss *dpu_mdss = to_dpu_mdss(mdss);
struct dss_module_power *mp = &dpu_mdss->mp;
int irq;
- pm_runtime_suspend(dev->dev);
- pm_runtime_disable(dev->dev);
+ pm_runtime_suspend(mdss->dev);
+ pm_runtime_disable(mdss->dev);
_dpu_mdss_irq_domain_fini(dpu_mdss);
irq = platform_get_irq(pdev, 0);
irq_set_chained_handler_and_data(irq, NULL, NULL);
@@ -203,7 +202,6 @@ static void dpu_mdss_destroy(struct drm_device *dev)
if (dpu_mdss->mmio)
devm_iounmap(&pdev->dev, dpu_mdss->mmio);
dpu_mdss->mmio = NULL;
- priv->mdss = NULL;
}
static const struct msm_mdss_funcs mdss_funcs = {
@@ -212,16 +210,15 @@ static const struct msm_mdss_funcs mdss_funcs = {
.destroy = dpu_mdss_destroy,
};
-int dpu_mdss_init(struct drm_device *dev)
+int dpu_mdss_init(struct platform_device *pdev)
{
- struct platform_device *pdev = to_platform_device(dev->dev);
- struct msm_drm_private *priv = dev->dev_private;
+ struct msm_drm_private *priv = platform_get_drvdata(pdev);
struct dpu_mdss *dpu_mdss;
struct dss_module_power *mp;
int ret;
int irq;
- dpu_mdss = devm_kzalloc(dev->dev, sizeof(*dpu_mdss), GFP_KERNEL);
+ dpu_mdss = devm_kzalloc(&pdev->dev, sizeof(*dpu_mdss), GFP_KERNEL);
if (!dpu_mdss)
return -ENOMEM;
@@ -238,7 +235,7 @@ int dpu_mdss_init(struct drm_device *dev)
goto clk_parse_err;
}
- dpu_mdss->base.dev = dev;
+ dpu_mdss->base.dev = &pdev->dev;
dpu_mdss->base.funcs = &mdss_funcs;
ret = _dpu_mdss_irq_domain_add(dpu_mdss);
@@ -256,7 +253,7 @@ int dpu_mdss_init(struct drm_device *dev)
priv->mdss = &dpu_mdss->base;
- pm_runtime_enable(dev->dev);
+ pm_runtime_enable(&pdev->dev);
return 0;
diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_plane.c b/drivers/gpu/drm/msm/disp/dpu1/dpu_plane.c
index a3e3b9d..ca75089 100644
--- a/drivers/gpu/drm/msm/disp/dpu1/dpu_plane.c
+++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_plane.c
@@ -13,7 +13,6 @@
#include <drm/drm_atomic.h>
#include <drm/drm_atomic_uapi.h>
#include <drm/drm_damage_helper.h>
-#include <drm/drm_file.h>
#include <drm/drm_gem_atomic_helper.h>
#include "msm_drv.h"
@@ -90,7 +89,6 @@ enum dpu_plane_qos {
/*
* struct dpu_plane - local dpu plane structure
* @aspace: address space pointer
- * @csc_ptr: Points to dpu_csc_cfg structure to use for current
* @mplane_list: List of multirect planes of the same pipe
* @catalog: Points to dpu catalog structure
* @revalidate: force revalidation of all the plane properties
@@ -101,29 +99,14 @@ struct dpu_plane {
struct mutex lock;
enum dpu_sspp pipe;
- uint32_t features; /* capabilities from catalog */
struct dpu_hw_pipe *pipe_hw;
- struct dpu_hw_pipe_cfg pipe_cfg;
- struct dpu_hw_pipe_qos_cfg pipe_qos_cfg;
uint32_t color_fill;
bool is_error;
bool is_rt_pipe;
bool is_virtual;
struct list_head mplane_list;
struct dpu_mdss_cfg *catalog;
-
- struct dpu_csc_cfg *csc_ptr;
-
- const struct dpu_sspp_sub_blks *pipe_sblk;
- char pipe_name[DPU_NAME_SIZE];
-
- /* debugfs related stuff */
- struct dentry *debugfs_root;
- struct dpu_debugfs_regset32 debugfs_src;
- struct dpu_debugfs_regset32 debugfs_scaler;
- struct dpu_debugfs_regset32 debugfs_csc;
- bool debugfs_default_scale;
};
static const uint64_t supported_format_modifiers[] = {
@@ -145,14 +128,15 @@ static struct dpu_kms *_dpu_plane_get_kms(struct drm_plane *plane)
* _dpu_plane_calc_bw - calculate bandwidth required for a plane
* @plane: Pointer to drm plane.
* @fb: Pointer to framebuffer associated with the given plane
+ * @pipe_cfg: Pointer to pipe configuration
* Result: Updates calculated bandwidth in the plane state.
* BW Equation: src_w * src_h * bpp * fps * (v_total / v_dest)
* Prefill BW Equation: line src bytes * line_time
*/
static void _dpu_plane_calc_bw(struct drm_plane *plane,
- struct drm_framebuffer *fb)
+ struct drm_framebuffer *fb,
+ struct dpu_hw_pipe_cfg *pipe_cfg)
{
- struct dpu_plane *pdpu = to_dpu_plane(plane);
struct dpu_plane_state *pstate;
struct drm_display_mode *mode;
const struct dpu_format *fmt = NULL;
@@ -169,9 +153,9 @@ static void _dpu_plane_calc_bw(struct drm_plane *plane,
fmt = dpu_get_dpu_format_ext(fb->format->format, fb->modifier);
- src_width = drm_rect_width(&pdpu->pipe_cfg.src_rect);
- src_height = drm_rect_height(&pdpu->pipe_cfg.src_rect);
- dst_height = drm_rect_height(&pdpu->pipe_cfg.dst_rect);
+ src_width = drm_rect_width(&pipe_cfg->src_rect);
+ src_height = drm_rect_height(&pipe_cfg->src_rect);
+ dst_height = drm_rect_height(&pipe_cfg->dst_rect);
fps = drm_mode_vrefresh(mode);
vbp = mode->vtotal - mode->vsync_end;
vpw = mode->vsync_end - mode->vsync_start;
@@ -202,12 +186,12 @@ static void _dpu_plane_calc_bw(struct drm_plane *plane,
/**
* _dpu_plane_calc_clk - calculate clock required for a plane
* @plane: Pointer to drm plane.
+ * @pipe_cfg: Pointer to pipe configuration
* Result: Updates calculated clock in the plane state.
* Clock equation: dst_w * v_total * fps * (src_h / dst_h)
*/
-static void _dpu_plane_calc_clk(struct drm_plane *plane)
+static void _dpu_plane_calc_clk(struct drm_plane *plane, struct dpu_hw_pipe_cfg *pipe_cfg)
{
- struct dpu_plane *pdpu = to_dpu_plane(plane);
struct dpu_plane_state *pstate;
struct drm_display_mode *mode;
int dst_width, src_height, dst_height, fps;
@@ -215,9 +199,9 @@ static void _dpu_plane_calc_clk(struct drm_plane *plane)
pstate = to_dpu_plane_state(plane->state);
mode = &plane->state->crtc->mode;
- src_height = drm_rect_height(&pdpu->pipe_cfg.src_rect);
- dst_width = drm_rect_width(&pdpu->pipe_cfg.dst_rect);
- dst_height = drm_rect_height(&pdpu->pipe_cfg.dst_rect);
+ src_height = drm_rect_height(&pipe_cfg->src_rect);
+ dst_width = drm_rect_width(&pipe_cfg->dst_rect);
+ dst_height = drm_rect_height(&pipe_cfg->dst_rect);
fps = drm_mode_vrefresh(mode);
pstate->plane_clk =
@@ -254,14 +238,17 @@ static int _dpu_plane_calc_fill_level(struct drm_plane *plane,
fixed_buff_size = pdpu->catalog->caps->pixel_ram_size;
list_for_each_entry(tmp, &pdpu->mplane_list, mplane_list) {
+ u32 tmp_width;
+
if (!tmp->base.state->visible)
continue;
+ tmp_width = drm_rect_width(&tmp->base.state->src) >> 16;
DPU_DEBUG("plane%d/%d src_width:%d/%d\n",
pdpu->base.base.id, tmp->base.base.id,
src_width,
- drm_rect_width(&tmp->pipe_cfg.src_rect));
+ tmp_width);
src_width = max_t(u32, src_width,
- drm_rect_width(&tmp->pipe_cfg.src_rect));
+ tmp_width);
}
if (fmt->fetch_planes == DPU_PLANE_PSEUDO_PLANAR) {
@@ -321,9 +308,10 @@ static u64 _dpu_plane_get_qos_lut(const struct dpu_qos_lut_tbl *tbl,
* _dpu_plane_set_qos_lut - set QoS LUT of the given plane
* @plane: Pointer to drm plane
* @fb: Pointer to framebuffer associated with the given plane
+ * @pipe_cfg: Pointer to pipe configuration
*/
static void _dpu_plane_set_qos_lut(struct drm_plane *plane,
- struct drm_framebuffer *fb)
+ struct drm_framebuffer *fb, struct dpu_hw_pipe_cfg *pipe_cfg)
{
struct dpu_plane *pdpu = to_dpu_plane(plane);
const struct dpu_format *fmt = NULL;
@@ -337,7 +325,7 @@ static void _dpu_plane_set_qos_lut(struct drm_plane *plane,
fb->format->format,
fb->modifier);
total_fl = _dpu_plane_calc_fill_level(plane, fmt,
- drm_rect_width(&pdpu->pipe_cfg.src_rect));
+ drm_rect_width(&pipe_cfg->src_rect));
if (fmt && DPU_FORMAT_IS_LINEAR(fmt))
lut_usage = DPU_QOS_LUT_USAGE_LINEAR;
@@ -348,8 +336,6 @@ static void _dpu_plane_set_qos_lut(struct drm_plane *plane,
qos_lut = _dpu_plane_get_qos_lut(
&pdpu->catalog->perf.qos_lut_tbl[lut_usage], total_fl);
- pdpu->pipe_qos_cfg.creq_lut = qos_lut;
-
trace_dpu_perf_set_qos_luts(pdpu->pipe - SSPP_VIG0,
(fmt) ? fmt->base.pixel_format : 0,
pdpu->is_rt_pipe, total_fl, qos_lut, lut_usage);
@@ -359,7 +345,7 @@ static void _dpu_plane_set_qos_lut(struct drm_plane *plane,
fmt ? (char *)&fmt->base.pixel_format : NULL,
pdpu->is_rt_pipe, total_fl, qos_lut);
- pdpu->pipe_hw->ops.setup_creq_lut(pdpu->pipe_hw, &pdpu->pipe_qos_cfg);
+ pdpu->pipe_hw->ops.setup_creq_lut(pdpu->pipe_hw, qos_lut);
}
/**
@@ -397,24 +383,21 @@ static void _dpu_plane_set_danger_lut(struct drm_plane *plane,
}
}
- pdpu->pipe_qos_cfg.danger_lut = danger_lut;
- pdpu->pipe_qos_cfg.safe_lut = safe_lut;
-
trace_dpu_perf_set_danger_luts(pdpu->pipe - SSPP_VIG0,
(fmt) ? fmt->base.pixel_format : 0,
(fmt) ? fmt->fetch_mode : 0,
- pdpu->pipe_qos_cfg.danger_lut,
- pdpu->pipe_qos_cfg.safe_lut);
+ danger_lut,
+ safe_lut);
DPU_DEBUG_PLANE(pdpu, "pnum:%d fmt: %4.4s mode:%d luts[0x%x, 0x%x]\n",
pdpu->pipe - SSPP_VIG0,
fmt ? (char *)&fmt->base.pixel_format : NULL,
fmt ? fmt->fetch_mode : -1,
- pdpu->pipe_qos_cfg.danger_lut,
- pdpu->pipe_qos_cfg.safe_lut);
+ danger_lut,
+ safe_lut);
pdpu->pipe_hw->ops.setup_danger_safe_lut(pdpu->pipe_hw,
- &pdpu->pipe_qos_cfg);
+ danger_lut, safe_lut);
}
/**
@@ -427,47 +410,51 @@ static void _dpu_plane_set_qos_ctrl(struct drm_plane *plane,
bool enable, u32 flags)
{
struct dpu_plane *pdpu = to_dpu_plane(plane);
+ struct dpu_hw_pipe_qos_cfg pipe_qos_cfg;
+
+ memset(&pipe_qos_cfg, 0, sizeof(pipe_qos_cfg));
if (flags & DPU_PLANE_QOS_VBLANK_CTRL) {
- pdpu->pipe_qos_cfg.creq_vblank = pdpu->pipe_sblk->creq_vblank;
- pdpu->pipe_qos_cfg.danger_vblank =
- pdpu->pipe_sblk->danger_vblank;
- pdpu->pipe_qos_cfg.vblank_en = enable;
+ pipe_qos_cfg.creq_vblank = pdpu->pipe_hw->cap->sblk->creq_vblank;
+ pipe_qos_cfg.danger_vblank =
+ pdpu->pipe_hw->cap->sblk->danger_vblank;
+ pipe_qos_cfg.vblank_en = enable;
}
if (flags & DPU_PLANE_QOS_VBLANK_AMORTIZE) {
/* this feature overrules previous VBLANK_CTRL */
- pdpu->pipe_qos_cfg.vblank_en = false;
- pdpu->pipe_qos_cfg.creq_vblank = 0; /* clear vblank bits */
+ pipe_qos_cfg.vblank_en = false;
+ pipe_qos_cfg.creq_vblank = 0; /* clear vblank bits */
}
if (flags & DPU_PLANE_QOS_PANIC_CTRL)
- pdpu->pipe_qos_cfg.danger_safe_en = enable;
+ pipe_qos_cfg.danger_safe_en = enable;
if (!pdpu->is_rt_pipe) {
- pdpu->pipe_qos_cfg.vblank_en = false;
- pdpu->pipe_qos_cfg.danger_safe_en = false;
+ pipe_qos_cfg.vblank_en = false;
+ pipe_qos_cfg.danger_safe_en = false;
}
DPU_DEBUG_PLANE(pdpu, "pnum:%d ds:%d vb:%d pri[0x%x, 0x%x] is_rt:%d\n",
pdpu->pipe - SSPP_VIG0,
- pdpu->pipe_qos_cfg.danger_safe_en,
- pdpu->pipe_qos_cfg.vblank_en,
- pdpu->pipe_qos_cfg.creq_vblank,
- pdpu->pipe_qos_cfg.danger_vblank,
+ pipe_qos_cfg.danger_safe_en,
+ pipe_qos_cfg.vblank_en,
+ pipe_qos_cfg.creq_vblank,
+ pipe_qos_cfg.danger_vblank,
pdpu->is_rt_pipe);
pdpu->pipe_hw->ops.setup_qos_ctrl(pdpu->pipe_hw,
- &pdpu->pipe_qos_cfg);
+ &pipe_qos_cfg);
}
/**
* _dpu_plane_set_ot_limit - set OT limit for the given plane
* @plane: Pointer to drm plane
* @crtc: Pointer to drm crtc
+ * @pipe_cfg: Pointer to pipe configuration
*/
static void _dpu_plane_set_ot_limit(struct drm_plane *plane,
- struct drm_crtc *crtc)
+ struct drm_crtc *crtc, struct dpu_hw_pipe_cfg *pipe_cfg)
{
struct dpu_plane *pdpu = to_dpu_plane(plane);
struct dpu_vbif_set_ot_params ot_params;
@@ -476,8 +463,8 @@ static void _dpu_plane_set_ot_limit(struct drm_plane *plane,
memset(&ot_params, 0, sizeof(ot_params));
ot_params.xin_id = pdpu->pipe_hw->cap->xin_id;
ot_params.num = pdpu->pipe_hw->idx - SSPP_NONE;
- ot_params.width = drm_rect_width(&pdpu->pipe_cfg.src_rect);
- ot_params.height = drm_rect_height(&pdpu->pipe_cfg.src_rect);
+ ot_params.width = drm_rect_width(&pipe_cfg->src_rect);
+ ot_params.height = drm_rect_height(&pipe_cfg->src_rect);
ot_params.is_wfd = !pdpu->is_rt_pipe;
ot_params.frame_rate = drm_mode_vrefresh(&crtc->mode);
ot_params.vbif_idx = VBIF_RT;
@@ -541,14 +528,12 @@ static void _dpu_plane_setup_scaler3(struct dpu_plane *pdpu,
struct dpu_plane_state *pstate,
uint32_t src_w, uint32_t src_h, uint32_t dst_w, uint32_t dst_h,
struct dpu_hw_scaler3_cfg *scale_cfg,
+ struct dpu_hw_pixel_ext *pixel_ext,
const struct dpu_format *fmt,
uint32_t chroma_subsmpl_h, uint32_t chroma_subsmpl_v)
{
uint32_t i;
- memset(scale_cfg, 0, sizeof(*scale_cfg));
- memset(&pstate->pixel_ext, 0, sizeof(struct dpu_hw_pixel_ext));
-
scale_cfg->phase_step_x[DPU_SSPP_COMP_0] =
mult_frac((1 << PHASE_STEP_SHIFT), src_w, dst_w);
scale_cfg->phase_step_y[DPU_SSPP_COMP_0] =
@@ -587,9 +572,9 @@ static void _dpu_plane_setup_scaler3(struct dpu_plane *pdpu,
scale_cfg->preload_y[i] = DPU_QSEED3_DEFAULT_PRELOAD_V;
}
- pstate->pixel_ext.num_ext_pxls_top[i] =
+ pixel_ext->num_ext_pxls_top[i] =
scale_cfg->src_height[i];
- pstate->pixel_ext.num_ext_pxls_left[i] =
+ pixel_ext->num_ext_pxls_left[i] =
scale_cfg->src_width[i];
}
if (!(DPU_FORMAT_IS_YUV(fmt)) && (src_h == dst_h)
@@ -606,68 +591,97 @@ static void _dpu_plane_setup_scaler3(struct dpu_plane *pdpu,
scale_cfg->enable = 1;
}
-static void _dpu_plane_setup_csc(struct dpu_plane *pdpu)
-{
- static const struct dpu_csc_cfg dpu_csc_YUV2RGB_601L = {
- {
- /* S15.16 format */
- 0x00012A00, 0x00000000, 0x00019880,
- 0x00012A00, 0xFFFF9B80, 0xFFFF3000,
- 0x00012A00, 0x00020480, 0x00000000,
+static const struct dpu_csc_cfg dpu_csc_YUV2RGB_601L = {
+ {
+ /* S15.16 format */
+ 0x00012A00, 0x00000000, 0x00019880,
+ 0x00012A00, 0xFFFF9B80, 0xFFFF3000,
+ 0x00012A00, 0x00020480, 0x00000000,
+ },
+ /* signed bias */
+ { 0xfff0, 0xff80, 0xff80,},
+ { 0x0, 0x0, 0x0,},
+ /* unsigned clamp */
+ { 0x10, 0xeb, 0x10, 0xf0, 0x10, 0xf0,},
+ { 0x00, 0xff, 0x00, 0xff, 0x00, 0xff,},
+};
+
+static const struct dpu_csc_cfg dpu_csc10_YUV2RGB_601L = {
+ {
+ /* S15.16 format */
+ 0x00012A00, 0x00000000, 0x00019880,
+ 0x00012A00, 0xFFFF9B80, 0xFFFF3000,
+ 0x00012A00, 0x00020480, 0x00000000,
},
- /* signed bias */
- { 0xfff0, 0xff80, 0xff80,},
- { 0x0, 0x0, 0x0,},
- /* unsigned clamp */
- { 0x10, 0xeb, 0x10, 0xf0, 0x10, 0xf0,},
- { 0x00, 0xff, 0x00, 0xff, 0x00, 0xff,},
- };
- static const struct dpu_csc_cfg dpu_csc10_YUV2RGB_601L = {
- {
- /* S15.16 format */
- 0x00012A00, 0x00000000, 0x00019880,
- 0x00012A00, 0xFFFF9B80, 0xFFFF3000,
- 0x00012A00, 0x00020480, 0x00000000,
- },
- /* signed bias */
- { 0xffc0, 0xfe00, 0xfe00,},
- { 0x0, 0x0, 0x0,},
- /* unsigned clamp */
- { 0x40, 0x3ac, 0x40, 0x3c0, 0x40, 0x3c0,},
- { 0x00, 0x3ff, 0x00, 0x3ff, 0x00, 0x3ff,},
- };
+ /* signed bias */
+ { 0xffc0, 0xfe00, 0xfe00,},
+ { 0x0, 0x0, 0x0,},
+ /* unsigned clamp */
+ { 0x40, 0x3ac, 0x40, 0x3c0, 0x40, 0x3c0,},
+ { 0x00, 0x3ff, 0x00, 0x3ff, 0x00, 0x3ff,},
+};
+
+static const struct dpu_csc_cfg *_dpu_plane_get_csc(struct dpu_plane *pdpu, const struct dpu_format *fmt)
+{
+ const struct dpu_csc_cfg *csc_ptr;
if (!pdpu) {
DPU_ERROR("invalid plane\n");
- return;
+ return NULL;
}
- if (BIT(DPU_SSPP_CSC_10BIT) & pdpu->features)
- pdpu->csc_ptr = (struct dpu_csc_cfg *)&dpu_csc10_YUV2RGB_601L;
+ if (!DPU_FORMAT_IS_YUV(fmt))
+ return NULL;
+
+ if (BIT(DPU_SSPP_CSC_10BIT) & pdpu->pipe_hw->cap->features)
+ csc_ptr = &dpu_csc10_YUV2RGB_601L;
else
- pdpu->csc_ptr = (struct dpu_csc_cfg *)&dpu_csc_YUV2RGB_601L;
+ csc_ptr = &dpu_csc_YUV2RGB_601L;
DPU_DEBUG_PLANE(pdpu, "using 0x%X 0x%X 0x%X...\n",
- pdpu->csc_ptr->csc_mv[0],
- pdpu->csc_ptr->csc_mv[1],
- pdpu->csc_ptr->csc_mv[2]);
+ csc_ptr->csc_mv[0],
+ csc_ptr->csc_mv[1],
+ csc_ptr->csc_mv[2]);
+
+ return csc_ptr;
}
static void _dpu_plane_setup_scaler(struct dpu_plane *pdpu,
struct dpu_plane_state *pstate,
- const struct dpu_format *fmt, bool color_fill)
+ const struct dpu_format *fmt, bool color_fill,
+ struct dpu_hw_pipe_cfg *pipe_cfg)
{
const struct drm_format_info *info = drm_format_info(fmt->base.pixel_format);
+ struct dpu_hw_scaler3_cfg scaler3_cfg;
+ struct dpu_hw_pixel_ext pixel_ext;
+
+ memset(&scaler3_cfg, 0, sizeof(scaler3_cfg));
+ memset(&pixel_ext, 0, sizeof(pixel_ext));
/* don't chroma subsample if decimating */
/* update scaler. calculate default config for QSEED3 */
_dpu_plane_setup_scaler3(pdpu, pstate,
- drm_rect_width(&pdpu->pipe_cfg.src_rect),
- drm_rect_height(&pdpu->pipe_cfg.src_rect),
- drm_rect_width(&pdpu->pipe_cfg.dst_rect),
- drm_rect_height(&pdpu->pipe_cfg.dst_rect),
- &pstate->scaler3_cfg, fmt,
+ drm_rect_width(&pipe_cfg->src_rect),
+ drm_rect_height(&pipe_cfg->src_rect),
+ drm_rect_width(&pipe_cfg->dst_rect),
+ drm_rect_height(&pipe_cfg->dst_rect),
+ &scaler3_cfg, &pixel_ext, fmt,
info->hsub, info->vsub);
+
+ if (pdpu->pipe_hw->ops.setup_pe)
+ pdpu->pipe_hw->ops.setup_pe(pdpu->pipe_hw,
+ &pixel_ext);
+
+ /**
+ * when programmed in multirect mode, scalar block will be
+ * bypassed. Still we need to update alpha and bitwidth
+ * ONLY for RECT0
+ */
+ if (pdpu->pipe_hw->ops.setup_scaler &&
+ pstate->multirect_index != DPU_SSPP_RECT_1)
+ pdpu->pipe_hw->ops.setup_scaler(pdpu->pipe_hw,
+ pipe_cfg,
+ &scaler3_cfg);
}
/**
@@ -683,6 +697,7 @@ static int _dpu_plane_color_fill(struct dpu_plane *pdpu,
const struct dpu_format *fmt;
const struct drm_plane *plane = &pdpu->base;
struct dpu_plane_state *pstate = to_dpu_plane_state(plane->state);
+ struct dpu_hw_pipe_cfg pipe_cfg;
DPU_DEBUG_PLANE(pdpu, "\n");
@@ -699,13 +714,14 @@ static int _dpu_plane_color_fill(struct dpu_plane *pdpu,
pstate->multirect_index);
/* override scaler/decimation if solid fill */
- pdpu->pipe_cfg.src_rect.x1 = 0;
- pdpu->pipe_cfg.src_rect.y1 = 0;
- pdpu->pipe_cfg.src_rect.x2 =
- drm_rect_width(&pdpu->pipe_cfg.dst_rect);
- pdpu->pipe_cfg.src_rect.y2 =
- drm_rect_height(&pdpu->pipe_cfg.dst_rect);
- _dpu_plane_setup_scaler(pdpu, pstate, fmt, true);
+ pipe_cfg.dst_rect = pstate->base.dst;
+
+ pipe_cfg.src_rect.x1 = 0;
+ pipe_cfg.src_rect.y1 = 0;
+ pipe_cfg.src_rect.x2 =
+ drm_rect_width(&pipe_cfg.dst_rect);
+ pipe_cfg.src_rect.y2 =
+ drm_rect_height(&pipe_cfg.dst_rect);
if (pdpu->pipe_hw->ops.setup_format)
pdpu->pipe_hw->ops.setup_format(pdpu->pipe_hw,
@@ -714,18 +730,10 @@ static int _dpu_plane_color_fill(struct dpu_plane *pdpu,
if (pdpu->pipe_hw->ops.setup_rects)
pdpu->pipe_hw->ops.setup_rects(pdpu->pipe_hw,
- &pdpu->pipe_cfg,
+ &pipe_cfg,
pstate->multirect_index);
- if (pdpu->pipe_hw->ops.setup_pe)
- pdpu->pipe_hw->ops.setup_pe(pdpu->pipe_hw,
- &pstate->pixel_ext);
-
- if (pdpu->pipe_hw->ops.setup_scaler &&
- pstate->multirect_index != DPU_SSPP_RECT_1)
- pdpu->pipe_hw->ops.setup_scaler(pdpu->pipe_hw,
- &pdpu->pipe_cfg, &pstate->pixel_ext,
- &pstate->scaler3_cfg);
+ _dpu_plane_setup_scaler(pdpu, pstate, fmt, true, &pipe_cfg);
}
return 0;
@@ -964,10 +972,10 @@ static int dpu_plane_atomic_check(struct drm_plane *plane,
crtc_state = drm_atomic_get_new_crtc_state(state,
new_plane_state->crtc);
- min_scale = FRAC_16_16(1, pdpu->pipe_sblk->maxupscale);
+ min_scale = FRAC_16_16(1, pdpu->pipe_hw->cap->sblk->maxupscale);
ret = drm_atomic_helper_check_plane_state(new_plane_state, crtc_state,
min_scale,
- pdpu->pipe_sblk->maxdwnscale << 16,
+ pdpu->pipe_hw->cap->sblk->maxdwnscale << 16,
true, true);
if (ret) {
DPU_DEBUG_PLANE(pdpu, "Check plane state failed (%d)\n", ret);
@@ -993,9 +1001,8 @@ static int dpu_plane_atomic_check(struct drm_plane *plane,
min_src_size = DPU_FORMAT_IS_YUV(fmt) ? 2 : 1;
if (DPU_FORMAT_IS_YUV(fmt) &&
- (!(pdpu->features & DPU_SSPP_SCALER) ||
- !(pdpu->features & (BIT(DPU_SSPP_CSC)
- | BIT(DPU_SSPP_CSC_10BIT))))) {
+ (!(pdpu->pipe_hw->cap->features & DPU_SSPP_SCALER) ||
+ !(pdpu->pipe_hw->cap->features & DPU_SSPP_CSC_ANY))) {
DPU_DEBUG_PLANE(pdpu,
"plane doesn't have scaler/csc for yuv\n");
return -EINVAL;
@@ -1056,8 +1063,13 @@ void dpu_plane_flush(struct drm_plane *plane)
else if (pdpu->color_fill & DPU_PLANE_COLOR_FILL_FLAG)
/* force 100% alpha */
_dpu_plane_color_fill(pdpu, pdpu->color_fill, 0xFF);
- else if (pdpu->pipe_hw && pdpu->csc_ptr && pdpu->pipe_hw->ops.setup_csc)
- pdpu->pipe_hw->ops.setup_csc(pdpu->pipe_hw, pdpu->csc_ptr);
+ else if (pdpu->pipe_hw && pdpu->pipe_hw->ops.setup_csc) {
+ const struct dpu_format *fmt = to_dpu_format(msm_framebuffer_format(plane->state->fb));
+ const struct dpu_csc_cfg *csc_ptr = _dpu_plane_get_csc(pdpu, fmt);
+
+ if (csc_ptr)
+ pdpu->pipe_hw->ops.setup_csc(pdpu->pipe_hw, csc_ptr);
+ }
/* flag h/w flush complete */
if (plane->state)
@@ -1091,10 +1103,11 @@ static void dpu_plane_sspp_atomic_update(struct drm_plane *plane)
bool is_rt_pipe, update_qos_remap;
const struct dpu_format *fmt =
to_dpu_format(msm_framebuffer_format(fb));
+ struct dpu_hw_pipe_cfg pipe_cfg;
- memset(&(pdpu->pipe_cfg), 0, sizeof(struct dpu_hw_pipe_cfg));
+ memset(&pipe_cfg, 0, sizeof(struct dpu_hw_pipe_cfg));
- _dpu_plane_set_scanout(plane, pstate, &pdpu->pipe_cfg, fb);
+ _dpu_plane_set_scanout(plane, pstate, &pipe_cfg, fb);
pstate->pending = true;
@@ -1106,17 +1119,15 @@ static void dpu_plane_sspp_atomic_update(struct drm_plane *plane)
crtc->base.id, DRM_RECT_ARG(&state->dst),
(char *)&fmt->base.pixel_format, DPU_FORMAT_IS_UBWC(fmt));
- pdpu->pipe_cfg.src_rect = state->src;
+ pipe_cfg.src_rect = state->src;
/* state->src is 16.16, src_rect is not */
- pdpu->pipe_cfg.src_rect.x1 >>= 16;
- pdpu->pipe_cfg.src_rect.x2 >>= 16;
- pdpu->pipe_cfg.src_rect.y1 >>= 16;
- pdpu->pipe_cfg.src_rect.y2 >>= 16;
+ pipe_cfg.src_rect.x1 >>= 16;
+ pipe_cfg.src_rect.x2 >>= 16;
+ pipe_cfg.src_rect.y1 >>= 16;
+ pipe_cfg.src_rect.y2 >>= 16;
- pdpu->pipe_cfg.dst_rect = state->dst;
-
- _dpu_plane_setup_scaler(pdpu, pstate, fmt, false);
+ pipe_cfg.dst_rect = state->dst;
/* override for color fill */
if (pdpu->color_fill & DPU_PLANE_COLOR_FILL_FLAG) {
@@ -1126,25 +1137,11 @@ static void dpu_plane_sspp_atomic_update(struct drm_plane *plane)
if (pdpu->pipe_hw->ops.setup_rects) {
pdpu->pipe_hw->ops.setup_rects(pdpu->pipe_hw,
- &pdpu->pipe_cfg,
+ &pipe_cfg,
pstate->multirect_index);
}
- if (pdpu->pipe_hw->ops.setup_pe &&
- (pstate->multirect_index != DPU_SSPP_RECT_1))
- pdpu->pipe_hw->ops.setup_pe(pdpu->pipe_hw,
- &pstate->pixel_ext);
-
- /**
- * when programmed in multirect mode, scalar block will be
- * bypassed. Still we need to update alpha and bitwidth
- * ONLY for RECT0
- */
- if (pdpu->pipe_hw->ops.setup_scaler &&
- pstate->multirect_index != DPU_SSPP_RECT_1)
- pdpu->pipe_hw->ops.setup_scaler(pdpu->pipe_hw,
- &pdpu->pipe_cfg, &pstate->pixel_ext,
- &pstate->scaler3_cfg);
+ _dpu_plane_setup_scaler(pdpu, pstate, fmt, false, &pipe_cfg);
if (pdpu->pipe_hw->ops.setup_multirect)
pdpu->pipe_hw->ops.setup_multirect(
@@ -1173,35 +1170,29 @@ static void dpu_plane_sspp_atomic_update(struct drm_plane *plane)
pstate->multirect_index);
if (pdpu->pipe_hw->ops.setup_cdp) {
- struct dpu_hw_pipe_cdp_cfg *cdp_cfg = &pstate->cdp_cfg;
+ struct dpu_hw_pipe_cdp_cfg cdp_cfg;
- memset(cdp_cfg, 0, sizeof(struct dpu_hw_pipe_cdp_cfg));
+ memset(&cdp_cfg, 0, sizeof(struct dpu_hw_pipe_cdp_cfg));
- cdp_cfg->enable = pdpu->catalog->perf.cdp_cfg
+ cdp_cfg.enable = pdpu->catalog->perf.cdp_cfg
[DPU_PERF_CDP_USAGE_RT].rd_enable;
- cdp_cfg->ubwc_meta_enable =
+ cdp_cfg.ubwc_meta_enable =
DPU_FORMAT_IS_UBWC(fmt);
- cdp_cfg->tile_amortize_enable =
+ cdp_cfg.tile_amortize_enable =
DPU_FORMAT_IS_UBWC(fmt) ||
DPU_FORMAT_IS_TILE(fmt);
- cdp_cfg->preload_ahead = DPU_SSPP_CDP_PRELOAD_AHEAD_64;
+ cdp_cfg.preload_ahead = DPU_SSPP_CDP_PRELOAD_AHEAD_64;
- pdpu->pipe_hw->ops.setup_cdp(pdpu->pipe_hw, cdp_cfg);
+ pdpu->pipe_hw->ops.setup_cdp(pdpu->pipe_hw, &cdp_cfg, pstate->multirect_index);
}
-
- /* update csc */
- if (DPU_FORMAT_IS_YUV(fmt))
- _dpu_plane_setup_csc(pdpu);
- else
- pdpu->csc_ptr = NULL;
}
- _dpu_plane_set_qos_lut(plane, fb);
+ _dpu_plane_set_qos_lut(plane, fb, &pipe_cfg);
_dpu_plane_set_danger_lut(plane, fb);
if (plane->type != DRM_PLANE_TYPE_CURSOR) {
_dpu_plane_set_qos_ctrl(plane, true, DPU_PLANE_QOS_PANIC_CTRL);
- _dpu_plane_set_ot_limit(plane, crtc);
+ _dpu_plane_set_ot_limit(plane, crtc, &pipe_cfg);
}
update_qos_remap = (is_rt_pipe != pdpu->is_rt_pipe) ||
@@ -1215,9 +1206,9 @@ static void dpu_plane_sspp_atomic_update(struct drm_plane *plane)
_dpu_plane_set_qos_remap(plane);
}
- _dpu_plane_calc_bw(plane, fb);
+ _dpu_plane_calc_bw(plane, fb, &pipe_cfg);
- _dpu_plane_calc_clk(plane);
+ _dpu_plane_calc_clk(plane, &pipe_cfg);
}
static void _dpu_plane_atomic_disable(struct drm_plane *plane)
@@ -1314,6 +1305,46 @@ dpu_plane_duplicate_state(struct drm_plane *plane)
return &pstate->base;
}
+static const char * const multirect_mode_name[] = {
+ [DPU_SSPP_MULTIRECT_NONE] = "none",
+ [DPU_SSPP_MULTIRECT_PARALLEL] = "parallel",
+ [DPU_SSPP_MULTIRECT_TIME_MX] = "time_mx",
+};
+
+static const char * const multirect_index_name[] = {
+ [DPU_SSPP_RECT_SOLO] = "solo",
+ [DPU_SSPP_RECT_0] = "rect_0",
+ [DPU_SSPP_RECT_1] = "rect_1",
+};
+
+static const char *dpu_get_multirect_mode(enum dpu_sspp_multirect_mode mode)
+{
+ if (WARN_ON(mode >= ARRAY_SIZE(multirect_mode_name)))
+ return "unknown";
+
+ return multirect_mode_name[mode];
+}
+
+static const char *dpu_get_multirect_index(enum dpu_sspp_multirect_index index)
+{
+ if (WARN_ON(index >= ARRAY_SIZE(multirect_index_name)))
+ return "unknown";
+
+ return multirect_index_name[index];
+}
+
+static void dpu_plane_atomic_print_state(struct drm_printer *p,
+ const struct drm_plane_state *state)
+{
+ const struct dpu_plane_state *pstate = to_dpu_plane_state(state);
+ const struct dpu_plane *pdpu = to_dpu_plane(state->plane);
+
+ drm_printf(p, "\tstage=%d\n", pstate->stage);
+ drm_printf(p, "\tsspp=%s\n", pdpu->pipe_hw->cap->name);
+ drm_printf(p, "\tmultirect_mode=%s\n", dpu_get_multirect_mode(pstate->multirect_mode));
+ drm_printf(p, "\tmultirect_index=%s\n", dpu_get_multirect_index(pstate->multirect_index));
+}
+
static void dpu_plane_reset(struct drm_plane *plane)
{
struct dpu_plane *pdpu;
@@ -1343,7 +1374,7 @@ static void dpu_plane_reset(struct drm_plane *plane)
}
#ifdef CONFIG_DEBUG_FS
-static void dpu_plane_danger_signal_ctrl(struct drm_plane *plane, bool enable)
+void dpu_plane_danger_signal_ctrl(struct drm_plane *plane, bool enable)
{
struct dpu_plane *pdpu = to_dpu_plane(plane);
struct dpu_kms *dpu_kms = _dpu_plane_get_kms(plane);
@@ -1356,167 +1387,23 @@ static void dpu_plane_danger_signal_ctrl(struct drm_plane *plane, bool enable)
pm_runtime_put_sync(&dpu_kms->pdev->dev);
}
-static ssize_t _dpu_plane_danger_read(struct file *file,
- char __user *buff, size_t count, loff_t *ppos)
-{
- struct dpu_kms *kms = file->private_data;
- int len;
- char buf[40];
-
- len = scnprintf(buf, sizeof(buf), "%d\n", !kms->has_danger_ctrl);
-
- return simple_read_from_buffer(buff, count, ppos, buf, len);
-}
-
-static void _dpu_plane_set_danger_state(struct dpu_kms *kms, bool enable)
+/* SSPP live inside dpu_plane private data only. Enumerate them here. */
+void dpu_debugfs_sspp_init(struct dpu_kms *dpu_kms, struct dentry *debugfs_root)
{
struct drm_plane *plane;
+ struct dentry *entry = debugfs_create_dir("sspp", debugfs_root);
- drm_for_each_plane(plane, kms->dev) {
- if (plane->fb && plane->state) {
- dpu_plane_danger_signal_ctrl(plane, enable);
- DPU_DEBUG("plane:%d img:%dx%d ",
- plane->base.id, plane->fb->width,
- plane->fb->height);
- DPU_DEBUG("src[%d,%d,%d,%d] dst[%d,%d,%d,%d]\n",
- plane->state->src_x >> 16,
- plane->state->src_y >> 16,
- plane->state->src_w >> 16,
- plane->state->src_h >> 16,
- plane->state->crtc_x, plane->state->crtc_y,
- plane->state->crtc_w, plane->state->crtc_h);
- } else {
- DPU_DEBUG("Inactive plane:%d\n", plane->base.id);
- }
+ if (IS_ERR(entry))
+ return;
+
+ drm_for_each_plane(plane, dpu_kms->dev) {
+ struct dpu_plane *pdpu = to_dpu_plane(plane);
+
+ _dpu_hw_sspp_init_debugfs(pdpu->pipe_hw, dpu_kms, entry);
}
}
-
-static ssize_t _dpu_plane_danger_write(struct file *file,
- const char __user *user_buf, size_t count, loff_t *ppos)
-{
- struct dpu_kms *kms = file->private_data;
- int disable_panic;
- int ret;
-
- ret = kstrtouint_from_user(user_buf, count, 0, &disable_panic);
- if (ret)
- return ret;
-
- if (disable_panic) {
- /* Disable panic signal for all active pipes */
- DPU_DEBUG("Disabling danger:\n");
- _dpu_plane_set_danger_state(kms, false);
- kms->has_danger_ctrl = false;
- } else {
- /* Enable panic signal for all active pipes */
- DPU_DEBUG("Enabling danger:\n");
- kms->has_danger_ctrl = true;
- _dpu_plane_set_danger_state(kms, true);
- }
-
- return count;
-}
-
-static const struct file_operations dpu_plane_danger_enable = {
- .open = simple_open,
- .read = _dpu_plane_danger_read,
- .write = _dpu_plane_danger_write,
-};
-
-static int _dpu_plane_init_debugfs(struct drm_plane *plane)
-{
- struct dpu_plane *pdpu = to_dpu_plane(plane);
- struct dpu_kms *kms = _dpu_plane_get_kms(plane);
- const struct dpu_sspp_cfg *cfg = pdpu->pipe_hw->cap;
- const struct dpu_sspp_sub_blks *sblk = cfg->sblk;
-
- /* create overall sub-directory for the pipe */
- pdpu->debugfs_root =
- debugfs_create_dir(pdpu->pipe_name,
- plane->dev->primary->debugfs_root);
-
- /* don't error check these */
- debugfs_create_x32("features", 0600,
- pdpu->debugfs_root, &pdpu->features);
-
- /* add register dump support */
- dpu_debugfs_setup_regset32(&pdpu->debugfs_src,
- sblk->src_blk.base + cfg->base,
- sblk->src_blk.len,
- kms);
- dpu_debugfs_create_regset32("src_blk", 0400,
- pdpu->debugfs_root, &pdpu->debugfs_src);
-
- if (cfg->features & BIT(DPU_SSPP_SCALER_QSEED3) ||
- cfg->features & BIT(DPU_SSPP_SCALER_QSEED3LITE) ||
- cfg->features & BIT(DPU_SSPP_SCALER_QSEED2) ||
- cfg->features & BIT(DPU_SSPP_SCALER_QSEED4)) {
- dpu_debugfs_setup_regset32(&pdpu->debugfs_scaler,
- sblk->scaler_blk.base + cfg->base,
- sblk->scaler_blk.len,
- kms);
- dpu_debugfs_create_regset32("scaler_blk", 0400,
- pdpu->debugfs_root,
- &pdpu->debugfs_scaler);
- debugfs_create_bool("default_scaling",
- 0600,
- pdpu->debugfs_root,
- &pdpu->debugfs_default_scale);
- }
-
- if (cfg->features & BIT(DPU_SSPP_CSC) ||
- cfg->features & BIT(DPU_SSPP_CSC_10BIT)) {
- dpu_debugfs_setup_regset32(&pdpu->debugfs_csc,
- sblk->csc_blk.base + cfg->base,
- sblk->csc_blk.len,
- kms);
- dpu_debugfs_create_regset32("csc_blk", 0400,
- pdpu->debugfs_root, &pdpu->debugfs_csc);
- }
-
- debugfs_create_u32("xin_id",
- 0400,
- pdpu->debugfs_root,
- (u32 *) &cfg->xin_id);
- debugfs_create_u32("clk_ctrl",
- 0400,
- pdpu->debugfs_root,
- (u32 *) &cfg->clk_ctrl);
- debugfs_create_x32("creq_vblank",
- 0600,
- pdpu->debugfs_root,
- (u32 *) &sblk->creq_vblank);
- debugfs_create_x32("danger_vblank",
- 0600,
- pdpu->debugfs_root,
- (u32 *) &sblk->danger_vblank);
-
- debugfs_create_file("disable_danger",
- 0600,
- pdpu->debugfs_root,
- kms, &dpu_plane_danger_enable);
-
- return 0;
-}
-#else
-static int _dpu_plane_init_debugfs(struct drm_plane *plane)
-{
- return 0;
-}
#endif
-static int dpu_plane_late_register(struct drm_plane *plane)
-{
- return _dpu_plane_init_debugfs(plane);
-}
-
-static void dpu_plane_early_unregister(struct drm_plane *plane)
-{
- struct dpu_plane *pdpu = to_dpu_plane(plane);
-
- debugfs_remove_recursive(pdpu->debugfs_root);
-}
-
static bool dpu_plane_format_mod_supported(struct drm_plane *plane,
uint32_t format, uint64_t modifier)
{
@@ -1541,8 +1428,7 @@ static const struct drm_plane_funcs dpu_plane_funcs = {
.reset = dpu_plane_reset,
.atomic_duplicate_state = dpu_plane_duplicate_state,
.atomic_destroy_state = dpu_plane_destroy_state,
- .late_register = dpu_plane_late_register,
- .early_unregister = dpu_plane_early_unregister,
+ .atomic_print_state = dpu_plane_atomic_print_state,
.format_mod_supported = dpu_plane_format_mod_supported,
};
@@ -1609,21 +1495,13 @@ struct drm_plane *dpu_plane_init(struct drm_device *dev,
goto clean_sspp;
}
- /* cache features mask for later */
- pdpu->features = pdpu->pipe_hw->cap->features;
- pdpu->pipe_sblk = pdpu->pipe_hw->cap->sblk;
- if (!pdpu->pipe_sblk) {
- DPU_ERROR("[%u]invalid sblk\n", pipe);
- goto clean_sspp;
- }
-
if (pdpu->is_virtual) {
- format_list = pdpu->pipe_sblk->virt_format_list;
- num_formats = pdpu->pipe_sblk->virt_num_formats;
+ format_list = pdpu->pipe_hw->cap->sblk->virt_format_list;
+ num_formats = pdpu->pipe_hw->cap->sblk->virt_num_formats;
}
else {
- format_list = pdpu->pipe_sblk->format_list;
- num_formats = pdpu->pipe_sblk->num_formats;
+ format_list = pdpu->pipe_hw->cap->sblk->format_list;
+ num_formats = pdpu->pipe_hw->cap->sblk->num_formats;
}
ret = drm_universal_plane_init(dev, plane, 0xff, &dpu_plane_funcs,
@@ -1663,12 +1541,9 @@ struct drm_plane *dpu_plane_init(struct drm_device *dev,
/* success! finalize initialization */
drm_plane_helper_add(plane, &dpu_plane_helper_funcs);
- /* save user friendly pipe name for later */
- snprintf(pdpu->pipe_name, DPU_NAME_SIZE, "plane%u", plane->base.id);
-
mutex_init(&pdpu->lock);
- DPU_DEBUG("%s created for pipe:%u id:%u virtual:%u\n", pdpu->pipe_name,
+ DPU_DEBUG("%s created for pipe:%u id:%u virtual:%u\n", plane->name,
pipe, plane->base.id, master_plane_id);
return plane;
@@ -1676,6 +1551,7 @@ struct drm_plane *dpu_plane_init(struct drm_device *dev,
if (pdpu && pdpu->pipe_hw)
dpu_hw_sspp_destroy(pdpu->pipe_hw);
clean_plane:
+ list_del(&pdpu->mplane_list);
kfree(pdpu);
return ERR_PTR(ret);
}
diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_plane.h b/drivers/gpu/drm/msm/disp/dpu1/dpu_plane.h
index 34e03ac..9d51dad 100644
--- a/drivers/gpu/drm/msm/disp/dpu1/dpu_plane.h
+++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_plane.h
@@ -23,9 +23,6 @@
* @multirect_index: index of the rectangle of SSPP
* @multirect_mode: parallel or time multiplex multirect mode
* @pending: whether the current update is still pending
- * @scaler3_cfg: configuration data for scaler3
- * @pixel_ext: configuration data for pixel extensions
- * @cdp_cfg: CDP configuration
* @plane_fetch_bw: calculated BW per plane
* @plane_clk: calculated clk per plane
*/
@@ -38,11 +35,6 @@ struct dpu_plane_state {
uint32_t multirect_mode;
bool pending;
- /* scaler configuration */
- struct dpu_hw_scaler3_cfg scaler3_cfg;
- struct dpu_hw_pixel_ext pixel_ext;
-
- struct dpu_hw_pipe_cdp_cfg cdp_cfg;
u64 plane_fetch_bw;
u64 plane_clk;
};
@@ -134,4 +126,10 @@ void dpu_plane_clear_multirect(const struct drm_plane_state *drm_state);
int dpu_plane_color_fill(struct drm_plane *plane,
uint32_t color, uint32_t alpha);
+#ifdef CONFIG_DEBUG_FS
+void dpu_plane_danger_signal_ctrl(struct drm_plane *plane, bool enable);
+#else
+static inline void dpu_plane_danger_signal_ctrl(struct drm_plane *plane, bool enable) {}
+#endif
+
#endif /* _DPU_PLANE_H_ */
diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_trace.h b/drivers/gpu/drm/msm/disp/dpu1/dpu_trace.h
index 37bba57..54d7434 100644
--- a/drivers/gpu/drm/msm/disp/dpu1/dpu_trace.h
+++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_trace.h
@@ -266,10 +266,6 @@ DEFINE_EVENT(dpu_drm_obj_template, dpu_crtc_complete_commit,
TP_PROTO(uint32_t drm_id),
TP_ARGS(drm_id)
);
-DEFINE_EVENT(dpu_drm_obj_template, dpu_kms_enc_enable,
- TP_PROTO(uint32_t drm_id),
- TP_ARGS(drm_id)
-);
DEFINE_EVENT(dpu_drm_obj_template, dpu_kms_commit,
TP_PROTO(uint32_t drm_id),
TP_ARGS(drm_id)
diff --git a/drivers/gpu/drm/msm/disp/mdp5/mdp5_kms.c b/drivers/gpu/drm/msm/disp/mdp5/mdp5_kms.c
index 7b24224..12a5f81 100644
--- a/drivers/gpu/drm/msm/disp/mdp5/mdp5_kms.c
+++ b/drivers/gpu/drm/msm/disp/mdp5/mdp5_kms.c
@@ -370,22 +370,7 @@ static int modeset_init_intf(struct mdp5_kms *mdp5_kms,
switch (intf->type) {
case INTF_eDP:
- if (!priv->edp)
- break;
-
- ctl = mdp5_ctlm_request(ctlm, intf->num);
- if (!ctl) {
- ret = -EINVAL;
- break;
- }
-
- encoder = construct_encoder(mdp5_kms, intf, ctl);
- if (IS_ERR(encoder)) {
- ret = PTR_ERR(encoder);
- break;
- }
-
- ret = msm_edp_modeset_init(priv->edp, dev, encoder);
+ DRM_DEV_INFO(dev->dev, "Skipping eDP interface %d\n", intf->num);
break;
case INTF_HDMI:
if (!priv->hdmi)
@@ -936,7 +921,8 @@ static int mdp5_init(struct platform_device *pdev, struct drm_device *dev)
static int mdp5_bind(struct device *dev, struct device *master, void *data)
{
- struct drm_device *ddev = dev_get_drvdata(master);
+ struct msm_drm_private *priv = dev_get_drvdata(master);
+ struct drm_device *ddev = priv->dev;
struct platform_device *pdev = to_platform_device(dev);
DBG("");
@@ -1031,7 +1017,7 @@ static const struct dev_pm_ops mdp5_pm_ops = {
SET_RUNTIME_PM_OPS(mdp5_runtime_suspend, mdp5_runtime_resume, NULL)
};
-static const struct of_device_id mdp5_dt_match[] = {
+const struct of_device_id mdp5_dt_match[] = {
{ .compatible = "qcom,mdp5", },
/* to support downstream DT files */
{ .compatible = "qcom,mdss_mdp", },
diff --git a/drivers/gpu/drm/msm/disp/mdp5/mdp5_mdss.c b/drivers/gpu/drm/msm/disp/mdp5/mdp5_mdss.c
index 0ea5342..b3f79c2 100644
--- a/drivers/gpu/drm/msm/disp/mdp5/mdp5_mdss.c
+++ b/drivers/gpu/drm/msm/disp/mdp5/mdp5_mdss.c
@@ -16,8 +16,6 @@ struct mdp5_mdss {
void __iomem *mmio, *vbif;
- struct regulator *vdd;
-
struct clk *ahb_clk;
struct clk *axi_clk;
struct clk *vsync_clk;
@@ -114,7 +112,7 @@ static const struct irq_domain_ops mdss_hw_irqdomain_ops = {
static int mdss_irq_domain_init(struct mdp5_mdss *mdp5_mdss)
{
- struct device *dev = mdp5_mdss->base.dev->dev;
+ struct device *dev = mdp5_mdss->base.dev;
struct irq_domain *d;
d = irq_domain_add_linear(dev->of_node, 32, &mdss_hw_irqdomain_ops,
@@ -157,7 +155,7 @@ static int mdp5_mdss_disable(struct msm_mdss *mdss)
static int msm_mdss_get_clocks(struct mdp5_mdss *mdp5_mdss)
{
struct platform_device *pdev =
- to_platform_device(mdp5_mdss->base.dev->dev);
+ to_platform_device(mdp5_mdss->base.dev);
mdp5_mdss->ahb_clk = msm_clk_get(pdev, "iface");
if (IS_ERR(mdp5_mdss->ahb_clk))
@@ -174,10 +172,9 @@ static int msm_mdss_get_clocks(struct mdp5_mdss *mdp5_mdss)
return 0;
}
-static void mdp5_mdss_destroy(struct drm_device *dev)
+static void mdp5_mdss_destroy(struct msm_mdss *mdss)
{
- struct msm_drm_private *priv = dev->dev_private;
- struct mdp5_mdss *mdp5_mdss = to_mdp5_mdss(priv->mdss);
+ struct mdp5_mdss *mdp5_mdss = to_mdp5_mdss(mdss);
if (!mdp5_mdss)
return;
@@ -185,9 +182,7 @@ static void mdp5_mdss_destroy(struct drm_device *dev)
irq_domain_remove(mdp5_mdss->irqcontroller.domain);
mdp5_mdss->irqcontroller.domain = NULL;
- regulator_disable(mdp5_mdss->vdd);
-
- pm_runtime_disable(dev->dev);
+ pm_runtime_disable(mdss->dev);
}
static const struct msm_mdss_funcs mdss_funcs = {
@@ -196,25 +191,24 @@ static const struct msm_mdss_funcs mdss_funcs = {
.destroy = mdp5_mdss_destroy,
};
-int mdp5_mdss_init(struct drm_device *dev)
+int mdp5_mdss_init(struct platform_device *pdev)
{
- struct platform_device *pdev = to_platform_device(dev->dev);
- struct msm_drm_private *priv = dev->dev_private;
+ struct msm_drm_private *priv = platform_get_drvdata(pdev);
struct mdp5_mdss *mdp5_mdss;
int ret;
DBG("");
- if (!of_device_is_compatible(dev->dev->of_node, "qcom,mdss"))
+ if (!of_device_is_compatible(pdev->dev.of_node, "qcom,mdss"))
return 0;
- mdp5_mdss = devm_kzalloc(dev->dev, sizeof(*mdp5_mdss), GFP_KERNEL);
+ mdp5_mdss = devm_kzalloc(&pdev->dev, sizeof(*mdp5_mdss), GFP_KERNEL);
if (!mdp5_mdss) {
ret = -ENOMEM;
goto fail;
}
- mdp5_mdss->base.dev = dev;
+ mdp5_mdss->base.dev = &pdev->dev;
mdp5_mdss->mmio = msm_ioremap(pdev, "mdss_phys", "MDSS");
if (IS_ERR(mdp5_mdss->mmio)) {
@@ -230,45 +224,29 @@ int mdp5_mdss_init(struct drm_device *dev)
ret = msm_mdss_get_clocks(mdp5_mdss);
if (ret) {
- DRM_DEV_ERROR(dev->dev, "failed to get clocks: %d\n", ret);
+ DRM_DEV_ERROR(&pdev->dev, "failed to get clocks: %d\n", ret);
goto fail;
}
- /* Regulator to enable GDSCs in downstream kernels */
- mdp5_mdss->vdd = devm_regulator_get(dev->dev, "vdd");
- if (IS_ERR(mdp5_mdss->vdd)) {
- ret = PTR_ERR(mdp5_mdss->vdd);
- goto fail;
- }
-
- ret = regulator_enable(mdp5_mdss->vdd);
- if (ret) {
- DRM_DEV_ERROR(dev->dev, "failed to enable regulator vdd: %d\n",
- ret);
- goto fail;
- }
-
- ret = devm_request_irq(dev->dev, platform_get_irq(pdev, 0),
+ ret = devm_request_irq(&pdev->dev, platform_get_irq(pdev, 0),
mdss_irq, 0, "mdss_isr", mdp5_mdss);
if (ret) {
- DRM_DEV_ERROR(dev->dev, "failed to init irq: %d\n", ret);
- goto fail_irq;
+ DRM_DEV_ERROR(&pdev->dev, "failed to init irq: %d\n", ret);
+ goto fail;
}
ret = mdss_irq_domain_init(mdp5_mdss);
if (ret) {
- DRM_DEV_ERROR(dev->dev, "failed to init sub-block irqs: %d\n", ret);
- goto fail_irq;
+ DRM_DEV_ERROR(&pdev->dev, "failed to init sub-block irqs: %d\n", ret);
+ goto fail;
}
mdp5_mdss->base.funcs = &mdss_funcs;
priv->mdss = &mdp5_mdss->base;
- pm_runtime_enable(dev->dev);
+ pm_runtime_enable(&pdev->dev);
return 0;
-fail_irq:
- regulator_disable(mdp5_mdss->vdd);
fail:
return ret;
}
diff --git a/drivers/gpu/drm/msm/disp/msm_disp_snapshot.c b/drivers/gpu/drm/msm/disp/msm_disp_snapshot.c
index a4a7cb0..e75b971 100644
--- a/drivers/gpu/drm/msm/disp/msm_disp_snapshot.c
+++ b/drivers/gpu/drm/msm/disp/msm_disp_snapshot.c
@@ -28,29 +28,42 @@ static ssize_t __maybe_unused disp_devcoredump_read(char *buffer, loff_t offset,
return count - iter.remain;
}
-static void _msm_disp_snapshot_work(struct kthread_work *work)
+struct msm_disp_state *
+msm_disp_snapshot_state_sync(struct msm_kms *kms)
{
- struct msm_kms *kms = container_of(work, struct msm_kms, dump_work);
struct drm_device *drm_dev = kms->dev;
struct msm_disp_state *disp_state;
- struct drm_printer p;
+
+ WARN_ON(!mutex_is_locked(&kms->dump_mutex));
disp_state = kzalloc(sizeof(struct msm_disp_state), GFP_KERNEL);
if (!disp_state)
- return;
+ return ERR_PTR(-ENOMEM);
disp_state->dev = drm_dev->dev;
disp_state->drm_dev = drm_dev;
INIT_LIST_HEAD(&disp_state->blocks);
- /* Serialize dumping here */
- mutex_lock(&kms->dump_mutex);
-
msm_disp_snapshot_capture_state(disp_state);
+ return disp_state;
+}
+
+static void _msm_disp_snapshot_work(struct kthread_work *work)
+{
+ struct msm_kms *kms = container_of(work, struct msm_kms, dump_work);
+ struct msm_disp_state *disp_state;
+ struct drm_printer p;
+
+ /* Serialize dumping here */
+ mutex_lock(&kms->dump_mutex);
+ disp_state = msm_disp_snapshot_state_sync(kms);
mutex_unlock(&kms->dump_mutex);
+ if (IS_ERR(disp_state))
+ return;
+
if (MSM_DISP_SNAPSHOT_DUMP_IN_CONSOLE) {
p = drm_info_printer(disp_state->drm_dev->dev);
msm_disp_state_print(disp_state, &p);
diff --git a/drivers/gpu/drm/msm/disp/msm_disp_snapshot.h b/drivers/gpu/drm/msm/disp/msm_disp_snapshot.h
index 4c61930..b5f452b 100644
--- a/drivers/gpu/drm/msm/disp/msm_disp_snapshot.h
+++ b/drivers/gpu/drm/msm/disp/msm_disp_snapshot.h
@@ -39,7 +39,7 @@
* @dev: device pointer
* @drm_dev: drm device pointer
* @atomic_state: atomic state duplicated at the time of the error
- * @timestamp: timestamp at which the coredump was captured
+ * @time: timestamp at which the coredump was captured
*/
struct msm_disp_state {
struct device *dev;
@@ -49,7 +49,7 @@ struct msm_disp_state {
struct drm_atomic_state *atomic_state;
- ktime_t timestamp;
+ struct timespec64 time;
};
/**
@@ -85,6 +85,16 @@ int msm_disp_snapshot_init(struct drm_device *drm_dev);
void msm_disp_snapshot_destroy(struct drm_device *drm_dev);
/**
+ * msm_disp_snapshot_state_sync - synchronously snapshot display state
+ * @kms: the kms object
+ *
+ * Returns state or error
+ *
+ * Must be called with &kms->dump_mutex held
+ */
+struct msm_disp_state *msm_disp_snapshot_state_sync(struct msm_kms *kms);
+
+/**
* msm_disp_snapshot_state - trigger to dump the display snapshot
* @drm_dev: handle to drm device
diff --git a/drivers/gpu/drm/msm/disp/msm_disp_snapshot_util.c b/drivers/gpu/drm/msm/disp/msm_disp_snapshot_util.c
index 2e1acb1..5d2ff67 100644
--- a/drivers/gpu/drm/msm/disp/msm_disp_snapshot_util.c
+++ b/drivers/gpu/drm/msm/disp/msm_disp_snapshot_util.c
@@ -5,6 +5,8 @@
#define pr_fmt(fmt) "[drm:%s:%d] " fmt, __func__, __LINE__
+#include <generated/utsrelease.h>
+
#include "msm_disp_snapshot.h"
static void msm_disp_state_dump_regs(u32 **reg, u32 aligned_len, void __iomem *base_addr)
@@ -79,10 +81,11 @@ void msm_disp_state_print(struct msm_disp_state *state, struct drm_printer *p)
}
drm_printf(p, "---\n");
-
+ drm_printf(p, "kernel: " UTS_RELEASE "\n");
drm_printf(p, "module: " KBUILD_MODNAME "\n");
drm_printf(p, "dpu devcoredump\n");
- drm_printf(p, "timestamp %lld\n", ktime_to_ns(state->timestamp));
+ drm_printf(p, "time: %lld.%09ld\n",
+ state->time.tv_sec, state->time.tv_nsec);
list_for_each_entry_safe(block, tmp, &state->blocks, node) {
drm_printf(p, "====================%s================\n", block->name);
@@ -100,7 +103,7 @@ static void msm_disp_capture_atomic_state(struct msm_disp_state *disp_state)
struct drm_device *ddev;
struct drm_modeset_acquire_ctx ctx;
- disp_state->timestamp = ktime_get();
+ ktime_get_real_ts64(&disp_state->time);
ddev = disp_state->drm_dev;
diff --git a/drivers/gpu/drm/msm/dp/dp_ctrl.c b/drivers/gpu/drm/msm/dp/dp_ctrl.c
index 62e75dc..c724cb0 100644
--- a/drivers/gpu/drm/msm/dp/dp_ctrl.c
+++ b/drivers/gpu/drm/msm/dp/dp_ctrl.c
@@ -119,13 +119,13 @@ void dp_ctrl_push_idle(struct dp_ctrl *dp_ctrl)
static void dp_ctrl_config_ctrl(struct dp_ctrl_private *ctrl)
{
u32 config = 0, tbd;
- u8 *dpcd = ctrl->panel->dpcd;
+ const u8 *dpcd = ctrl->panel->dpcd;
/* Default-> LSCLK DIV: 1/4 LCLK */
config |= (2 << DP_CONFIGURATION_CTRL_LSCLK_DIV_SHIFT);
/* Scrambler reset enable */
- if (dpcd[DP_EDP_CONFIGURATION_CAP] & DP_ALTERNATE_SCRAMBLER_RESET_CAP)
+ if (drm_dp_alternate_scrambler_reset_cap(dpcd))
config |= DP_CONFIGURATION_CTRL_ASSR;
tbd = dp_link_get_test_bits_depth(ctrl->link,
@@ -1228,7 +1228,10 @@ static int dp_ctrl_link_train(struct dp_ctrl_private *ctrl,
int *training_step)
{
int ret = 0;
+ const u8 *dpcd = ctrl->panel->dpcd;
u8 encoding = DP_SET_ANSI_8B10B;
+ u8 ssc;
+ u8 assr;
struct dp_link_info link_info = {0};
dp_ctrl_config_ctrl(ctrl);
@@ -1238,9 +1241,21 @@ static int dp_ctrl_link_train(struct dp_ctrl_private *ctrl,
link_info.capabilities = DP_LINK_CAP_ENHANCED_FRAMING;
dp_aux_link_configure(ctrl->aux, &link_info);
+
+ if (drm_dp_max_downspread(dpcd)) {
+ ssc = DP_SPREAD_AMP_0_5;
+ drm_dp_dpcd_write(ctrl->aux, DP_DOWNSPREAD_CTRL, &ssc, 1);
+ }
+
drm_dp_dpcd_write(ctrl->aux, DP_MAIN_LINK_CHANNEL_CODING_SET,
&encoding, 1);
+ if (drm_dp_alternate_scrambler_reset_cap(dpcd)) {
+ assr = DP_ALTERNATE_SCRAMBLER_RESET_ENABLE;
+ drm_dp_dpcd_write(ctrl->aux, DP_EDP_CONFIGURATION_SET,
+ &assr, 1);
+ }
+
ret = dp_ctrl_link_train_1(ctrl, training_step);
if (ret) {
DRM_ERROR("link training #1 failed. ret=%d\n", ret);
@@ -1312,9 +1327,11 @@ static int dp_ctrl_enable_mainlink_clocks(struct dp_ctrl_private *ctrl)
struct dp_io *dp_io = &ctrl->parser->io;
struct phy *phy = dp_io->phy;
struct phy_configure_opts_dp *opts_dp = &dp_io->phy_opts.dp;
+ const u8 *dpcd = ctrl->panel->dpcd;
opts_dp->lanes = ctrl->link->link_params.num_lanes;
opts_dp->link_rate = ctrl->link->link_params.rate / 100;
+ opts_dp->ssc = drm_dp_max_downspread(dpcd);
dp_ctrl_set_clock_rate(ctrl, DP_CTRL_PM, "ctrl_link",
ctrl->link->link_params.rate * 1000);
@@ -1406,7 +1423,7 @@ void dp_ctrl_host_deinit(struct dp_ctrl *dp_ctrl)
static bool dp_ctrl_use_fixed_nvid(struct dp_ctrl_private *ctrl)
{
- u8 *dpcd = ctrl->panel->dpcd;
+ const u8 *dpcd = ctrl->panel->dpcd;
/*
* For better interop experience, used a fixed NVID=0x8000
diff --git a/drivers/gpu/drm/msm/dp/dp_display.c b/drivers/gpu/drm/msm/dp/dp_display.c
index aba8aa4..7cc4d21 100644
--- a/drivers/gpu/drm/msm/dp/dp_display.c
+++ b/drivers/gpu/drm/msm/dp/dp_display.c
@@ -135,8 +135,18 @@ static const struct msm_dp_config sc7180_dp_cfg = {
.num_descs = 1,
};
+static const struct msm_dp_config sc7280_dp_cfg = {
+ .descs = (const struct msm_dp_desc[]) {
+ [MSM_DP_CONTROLLER_0] = { .io_start = 0x0ae90000, .connector_type = DRM_MODE_CONNECTOR_DisplayPort },
+ [MSM_DP_CONTROLLER_1] = { .io_start = 0x0aea0000, .connector_type = DRM_MODE_CONNECTOR_eDP },
+ },
+ .num_descs = 2,
+};
+
static const struct of_device_id dp_dt_match[] = {
{ .compatible = "qcom,sc7180-dp", .data = &sc7180_dp_cfg },
+ { .compatible = "qcom,sc7280-dp", .data = &sc7280_dp_cfg },
+ { .compatible = "qcom,sc7280-edp", .data = &sc7280_dp_cfg },
{}
};
@@ -224,13 +234,10 @@ static int dp_display_bind(struct device *dev, struct device *master,
{
int rc = 0;
struct dp_display_private *dp = dev_get_dp_display_private(dev);
- struct msm_drm_private *priv;
- struct drm_device *drm;
-
- drm = dev_get_drvdata(master);
+ struct msm_drm_private *priv = dev_get_drvdata(master);
+ struct drm_device *drm = priv->dev;
dp->dp_display.drm_dev = drm;
- priv = drm->dev_private;
priv->dp[dp->id] = &dp->dp_display;
rc = dp->parser->parse(dp->parser, dp->dp_display.connector_type);
@@ -266,8 +273,7 @@ static void dp_display_unbind(struct device *dev, struct device *master,
void *data)
{
struct dp_display_private *dp = dev_get_dp_display_private(dev);
- struct drm_device *drm = dev_get_drvdata(master);
- struct msm_drm_private *priv = drm->dev_private;
+ struct msm_drm_private *priv = dev_get_drvdata(master);
dp_power_client_deinit(dp->power);
dp_aux_unregister(dp->aux);
@@ -410,12 +416,11 @@ static int dp_display_usbpd_configure_cb(struct device *dev)
static int dp_display_usbpd_disconnect_cb(struct device *dev)
{
- int rc = 0;
struct dp_display_private *dp = dev_get_dp_display_private(dev);
dp_add_event(dp, EV_USER_NOTIFICATION, false, 0);
- return rc;
+ return 0;
}
static void dp_display_handle_video_request(struct dp_display_private *dp)
@@ -522,11 +527,8 @@ static int dp_hpd_plug_handle(struct dp_display_private *dp, u32 data)
dp->hpd_state = ST_CONNECT_PENDING;
- hpd->hpd_high = 1;
-
ret = dp_display_usbpd_configure_cb(&dp->pdev->dev);
if (ret) { /* link train failed */
- hpd->hpd_high = 0;
dp->hpd_state = ST_DISCONNECTED;
if (ret == -ECONNRESET) { /* cable unplugged */
@@ -603,7 +605,6 @@ static int dp_hpd_unplug_handle(struct dp_display_private *dp, u32 data)
/* triggered by irq_hdp with sink_count = 0 */
if (dp->link->sink_count == 0) {
dp_ctrl_off_phy(dp->ctrl);
- hpd->hpd_high = 0;
dp->core_initialized = false;
}
mutex_unlock(&dp->event_mutex);
@@ -627,8 +628,6 @@ static int dp_hpd_unplug_handle(struct dp_display_private *dp, u32 data)
/* disable HPD plug interrupts */
dp_catalog_hpd_config_intr(dp->catalog, DP_DP_HPD_PLUG_INT_MASK, false);
- hpd->hpd_high = 0;
-
/*
* We don't need separate work for disconnect as
* connect/attention interrupts are disabled
@@ -693,9 +692,15 @@ static int dp_irq_hpd_handle(struct dp_display_private *dp, u32 data)
return 0;
}
- ret = dp_display_usbpd_attention_cb(&dp->pdev->dev);
- if (ret == -ECONNRESET) { /* cable unplugged */
- dp->core_initialized = false;
+ /*
+ * dp core (ahb/aux clks) must be initialized before
+ * irq_hpd be handled
+ */
+ if (dp->core_initialized) {
+ ret = dp_display_usbpd_attention_cb(&dp->pdev->dev);
+ if (ret == -ECONNRESET) { /* cable unplugged */
+ dp->core_initialized = false;
+ }
}
DRM_DEBUG_DP("hpd_state=%d\n", state);
@@ -707,9 +712,9 @@ static int dp_irq_hpd_handle(struct dp_display_private *dp, u32 data)
static void dp_display_deinit_sub_modules(struct dp_display_private *dp)
{
dp_debug_put(dp->debug);
+ dp_audio_put(dp->audio);
dp_panel_put(dp->panel);
dp_aux_put(dp->aux);
- dp_audio_put(dp->audio);
}
static int dp_init_sub_modules(struct dp_display_private *dp)
@@ -1481,6 +1486,18 @@ int msm_dp_modeset_init(struct msm_dp *dp_display, struct drm_device *dev,
}
priv->connectors[priv->num_connectors++] = dp_display->connector;
+
+ dp_display->bridge = msm_dp_bridge_init(dp_display, dev, encoder);
+ if (IS_ERR(dp_display->bridge)) {
+ ret = PTR_ERR(dp_display->bridge);
+ DRM_DEV_ERROR(dev->dev,
+ "failed to create dp bridge: %d\n", ret);
+ dp_display->bridge = NULL;
+ return ret;
+ }
+
+ priv->bridges[priv->num_bridges++] = dp_display->bridge;
+
return 0;
}
@@ -1584,8 +1601,8 @@ int msm_dp_display_disable(struct msm_dp *dp, struct drm_encoder *encoder)
}
void msm_dp_display_mode_set(struct msm_dp *dp, struct drm_encoder *encoder,
- struct drm_display_mode *mode,
- struct drm_display_mode *adjusted_mode)
+ const struct drm_display_mode *mode,
+ const struct drm_display_mode *adjusted_mode)
{
struct dp_display_private *dp_display;
diff --git a/drivers/gpu/drm/msm/dp/dp_display.h b/drivers/gpu/drm/msm/dp/dp_display.h
index 8e80e3b..e3adcd5 100644
--- a/drivers/gpu/drm/msm/dp/dp_display.h
+++ b/drivers/gpu/drm/msm/dp/dp_display.h
@@ -13,6 +13,7 @@
struct msm_dp {
struct drm_device *drm_dev;
struct device *codec_dev;
+ struct drm_bridge *bridge;
struct drm_connector *connector;
struct drm_encoder *encoder;
struct drm_bridge *panel_bridge;
diff --git a/drivers/gpu/drm/msm/dp/dp_drm.c b/drivers/gpu/drm/msm/dp/dp_drm.c
index 76856c4..d4d360d 100644
--- a/drivers/gpu/drm/msm/dp/dp_drm.c
+++ b/drivers/gpu/drm/msm/dp/dp_drm.c
@@ -12,6 +12,14 @@
#include "msm_kms.h"
#include "dp_drm.h"
+
+struct msm_dp_bridge {
+ struct drm_bridge bridge;
+ struct msm_dp *dp_display;
+};
+
+#define to_dp_display(x) container_of((x), struct msm_dp_bridge, bridge)
+
struct dp_connector {
struct drm_connector base;
struct msm_dp *dp_display;
@@ -173,3 +181,70 @@ struct drm_connector *dp_drm_connector_init(struct msm_dp *dp_display)
return connector;
}
+
+static void dp_bridge_mode_set(struct drm_bridge *drm_bridge,
+ const struct drm_display_mode *mode,
+ const struct drm_display_mode *adjusted_mode)
+{
+ struct msm_dp_bridge *dp_bridge = to_dp_display(drm_bridge);
+ struct msm_dp *dp_display = dp_bridge->dp_display;
+
+ msm_dp_display_mode_set(dp_display, drm_bridge->encoder, mode, adjusted_mode);
+}
+
+static void dp_bridge_enable(struct drm_bridge *drm_bridge)
+{
+ struct msm_dp_bridge *dp_bridge = to_dp_display(drm_bridge);
+ struct msm_dp *dp_display = dp_bridge->dp_display;
+
+ msm_dp_display_enable(dp_display, drm_bridge->encoder);
+}
+
+static void dp_bridge_disable(struct drm_bridge *drm_bridge)
+{
+ struct msm_dp_bridge *dp_bridge = to_dp_display(drm_bridge);
+ struct msm_dp *dp_display = dp_bridge->dp_display;
+
+ msm_dp_display_pre_disable(dp_display, drm_bridge->encoder);
+}
+
+static void dp_bridge_post_disable(struct drm_bridge *drm_bridge)
+{
+ struct msm_dp_bridge *dp_bridge = to_dp_display(drm_bridge);
+ struct msm_dp *dp_display = dp_bridge->dp_display;
+
+ msm_dp_display_disable(dp_display, drm_bridge->encoder);
+}
+
+static const struct drm_bridge_funcs dp_bridge_ops = {
+ .enable = dp_bridge_enable,
+ .disable = dp_bridge_disable,
+ .post_disable = dp_bridge_post_disable,
+ .mode_set = dp_bridge_mode_set,
+};
+
+struct drm_bridge *msm_dp_bridge_init(struct msm_dp *dp_display, struct drm_device *dev,
+ struct drm_encoder *encoder)
+{
+ int rc;
+ struct msm_dp_bridge *dp_bridge;
+ struct drm_bridge *bridge;
+
+ dp_bridge = devm_kzalloc(dev->dev, sizeof(*dp_bridge), GFP_KERNEL);
+ if (!dp_bridge)
+ return ERR_PTR(-ENOMEM);
+
+ dp_bridge->dp_display = dp_display;
+
+ bridge = &dp_bridge->bridge;
+ bridge->funcs = &dp_bridge_ops;
+ bridge->encoder = encoder;
+
+ rc = drm_bridge_attach(encoder, bridge, NULL, DRM_BRIDGE_ATTACH_NO_CONNECTOR);
+ if (rc) {
+ DRM_ERROR("failed to attach bridge, rc=%d\n", rc);
+ return ERR_PTR(rc);
+ }
+
+ return bridge;
+}
diff --git a/drivers/gpu/drm/msm/dp/dp_hpd.c b/drivers/gpu/drm/msm/dp/dp_hpd.c
index e1c90fa..db98a1d 100644
--- a/drivers/gpu/drm/msm/dp/dp_hpd.c
+++ b/drivers/gpu/drm/msm/dp/dp_hpd.c
@@ -32,8 +32,6 @@ int dp_hpd_connect(struct dp_usbpd *dp_usbpd, bool hpd)
hpd_priv = container_of(dp_usbpd, struct dp_hpd_private,
dp_usbpd);
- dp_usbpd->hpd_high = hpd;
-
if (!hpd_priv->dp_cb || !hpd_priv->dp_cb->configure
|| !hpd_priv->dp_cb->disconnect) {
pr_err("hpd dp_cb not initialized\n");
diff --git a/drivers/gpu/drm/msm/dp/dp_hpd.h b/drivers/gpu/drm/msm/dp/dp_hpd.h
index 5bc5bb6..8feec5a 100644
--- a/drivers/gpu/drm/msm/dp/dp_hpd.h
+++ b/drivers/gpu/drm/msm/dp/dp_hpd.h
@@ -26,7 +26,6 @@ enum plug_orientation {
* @multi_func: multi-function preferred
* @usb_config_req: request to switch to usb
* @exit_dp_mode: request exit from displayport mode
- * @hpd_high: Hot Plug Detect signal is high.
* @hpd_irq: Change in the status since last message
* @alt_mode_cfg_done: bool to specify alt mode status
* @debug_en: bool to specify debug mode
@@ -39,7 +38,6 @@ struct dp_usbpd {
bool multi_func;
bool usb_config_req;
bool exit_dp_mode;
- bool hpd_high;
bool hpd_irq;
bool alt_mode_cfg_done;
bool debug_en;
diff --git a/drivers/gpu/drm/msm/dp/dp_link.c b/drivers/gpu/drm/msm/dp/dp_link.c
index a5bdfc5..d4d31e5 100644
--- a/drivers/gpu/drm/msm/dp/dp_link.c
+++ b/drivers/gpu/drm/msm/dp/dp_link.c
@@ -737,18 +737,25 @@ static int dp_link_parse_sink_count(struct dp_link *dp_link)
return 0;
}
-static void dp_link_parse_sink_status_field(struct dp_link_private *link)
+static int dp_link_parse_sink_status_field(struct dp_link_private *link)
{
int len = 0;
link->prev_sink_count = link->dp_link.sink_count;
- dp_link_parse_sink_count(&link->dp_link);
+ len = dp_link_parse_sink_count(&link->dp_link);
+ if (len < 0) {
+ DRM_ERROR("DP parse sink count failed\n");
+ return len;
+ }
len = drm_dp_dpcd_read_link_status(link->aux,
link->link_status);
- if (len < DP_LINK_STATUS_SIZE)
+ if (len < DP_LINK_STATUS_SIZE) {
DRM_ERROR("DP link status read failed\n");
- dp_link_parse_request(link);
+ return len;
+ }
+
+ return dp_link_parse_request(link);
}
/**
@@ -1023,7 +1030,9 @@ int dp_link_process_request(struct dp_link *dp_link)
dp_link_reset_data(link);
- dp_link_parse_sink_status_field(link);
+ ret = dp_link_parse_sink_status_field(link);
+ if (ret)
+ return ret;
if (link->request.test_requested == DP_TEST_LINK_EDID_READ) {
dp_link->sink_request |= DP_TEST_LINK_EDID_READ;
diff --git a/drivers/gpu/drm/msm/dsi/dsi.c b/drivers/gpu/drm/msm/dsi/dsi.c
index 75ae300..0525488 100644
--- a/drivers/gpu/drm/msm/dsi/dsi.c
+++ b/drivers/gpu/drm/msm/dsi/dsi.c
@@ -110,9 +110,40 @@ static struct msm_dsi *dsi_init(struct platform_device *pdev)
static int dsi_bind(struct device *dev, struct device *master, void *data)
{
- struct drm_device *drm = dev_get_drvdata(master);
- struct msm_drm_private *priv = drm->dev_private;
- struct platform_device *pdev = to_platform_device(dev);
+ struct msm_drm_private *priv = dev_get_drvdata(master);
+ struct msm_dsi *msm_dsi = dev_get_drvdata(dev);
+
+ priv->dsi[msm_dsi->id] = msm_dsi;
+
+ return 0;
+}
+
+static void dsi_unbind(struct device *dev, struct device *master,
+ void *data)
+{
+ struct msm_drm_private *priv = dev_get_drvdata(master);
+ struct msm_dsi *msm_dsi = dev_get_drvdata(dev);
+
+ priv->dsi[msm_dsi->id] = NULL;
+}
+
+static const struct component_ops dsi_ops = {
+ .bind = dsi_bind,
+ .unbind = dsi_unbind,
+};
+
+int dsi_dev_attach(struct platform_device *pdev)
+{
+ return component_add(&pdev->dev, &dsi_ops);
+}
+
+void dsi_dev_detach(struct platform_device *pdev)
+{
+ component_del(&pdev->dev, &dsi_ops);
+}
+
+static int dsi_dev_probe(struct platform_device *pdev)
+{
struct msm_dsi *msm_dsi;
DBG("");
@@ -125,39 +156,16 @@ static int dsi_bind(struct device *dev, struct device *master, void *data)
return PTR_ERR(msm_dsi);
}
- priv->dsi[msm_dsi->id] = msm_dsi;
-
return 0;
}
-static void dsi_unbind(struct device *dev, struct device *master,
- void *data)
-{
- struct drm_device *drm = dev_get_drvdata(master);
- struct msm_drm_private *priv = drm->dev_private;
- struct msm_dsi *msm_dsi = dev_get_drvdata(dev);
- int id = msm_dsi->id;
-
- if (priv->dsi[id]) {
- dsi_destroy(msm_dsi);
- priv->dsi[id] = NULL;
- }
-}
-
-static const struct component_ops dsi_ops = {
- .bind = dsi_bind,
- .unbind = dsi_unbind,
-};
-
-static int dsi_dev_probe(struct platform_device *pdev)
-{
- return component_add(&pdev->dev, &dsi_ops);
-}
-
static int dsi_dev_remove(struct platform_device *pdev)
{
+ struct msm_dsi *msm_dsi = platform_get_drvdata(pdev);
+
DBG("");
- component_del(&pdev->dev, &dsi_ops);
+ dsi_destroy(msm_dsi);
+
return 0;
}
@@ -215,9 +223,13 @@ int msm_dsi_modeset_init(struct msm_dsi *msm_dsi, struct drm_device *dev,
goto fail;
}
- if (!msm_dsi_manager_validate_current_config(msm_dsi->id)) {
- ret = -EINVAL;
- goto fail;
+ if (msm_dsi_is_bonded_dsi(msm_dsi) &&
+ !msm_dsi_is_master_dsi(msm_dsi)) {
+ /*
+ * Do not return an eror here,
+ * Just skip creating encoder/connector for the slave-DSI.
+ */
+ return 0;
}
msm_dsi->encoder = encoder;
diff --git a/drivers/gpu/drm/msm/dsi/dsi.h b/drivers/gpu/drm/msm/dsi/dsi.h
index 569c8ff..c8dedc9 100644
--- a/drivers/gpu/drm/msm/dsi/dsi.h
+++ b/drivers/gpu/drm/msm/dsi/dsi.h
@@ -82,7 +82,6 @@ int msm_dsi_manager_cmd_xfer(int id, const struct mipi_dsi_msg *msg);
bool msm_dsi_manager_cmd_xfer_trigger(int id, u32 dma_base, u32 len);
int msm_dsi_manager_register(struct msm_dsi *msm_dsi);
void msm_dsi_manager_unregister(struct msm_dsi *msm_dsi);
-bool msm_dsi_manager_validate_current_config(u8 id);
void msm_dsi_manager_tpg_enable(void);
/* msm dsi */
@@ -118,8 +117,10 @@ int msm_dsi_host_set_display_mode(struct mipi_dsi_host *host,
struct drm_panel *msm_dsi_host_get_panel(struct mipi_dsi_host *host);
unsigned long msm_dsi_host_get_mode_flags(struct mipi_dsi_host *host);
struct drm_bridge *msm_dsi_host_get_bridge(struct mipi_dsi_host *host);
-int msm_dsi_host_register(struct mipi_dsi_host *host, bool check_defer);
+int msm_dsi_host_register(struct mipi_dsi_host *host);
void msm_dsi_host_unregister(struct mipi_dsi_host *host);
+void msm_dsi_host_set_phy_mode(struct mipi_dsi_host *host,
+ struct msm_dsi_phy *src_phy);
int msm_dsi_host_set_src_pll(struct mipi_dsi_host *host,
struct msm_dsi_phy *src_phy);
void msm_dsi_host_reset_phy(struct mipi_dsi_host *host);
@@ -173,8 +174,6 @@ int msm_dsi_phy_enable(struct msm_dsi_phy *phy,
void msm_dsi_phy_disable(struct msm_dsi_phy *phy);
void msm_dsi_phy_set_usecase(struct msm_dsi_phy *phy,
enum msm_dsi_phy_usecase uc);
-int msm_dsi_phy_get_clk_provider(struct msm_dsi_phy *phy,
- struct clk **byte_clk_provider, struct clk **pixel_clk_provider);
void msm_dsi_phy_pll_save_state(struct msm_dsi_phy *phy);
int msm_dsi_phy_pll_restore_state(struct msm_dsi_phy *phy);
void msm_dsi_phy_snapshot(struct msm_disp_state *disp_state, struct msm_dsi_phy *phy);
diff --git a/drivers/gpu/drm/msm/dsi/dsi_host.c b/drivers/gpu/drm/msm/dsi/dsi_host.c
index 0afc3b7..6b3ced4 100644
--- a/drivers/gpu/drm/msm/dsi/dsi_host.c
+++ b/drivers/gpu/drm/msm/dsi/dsi_host.c
@@ -1586,6 +1586,10 @@ static int dsi_host_attach(struct mipi_dsi_host *host,
if (ret)
return ret;
+ ret = dsi_dev_attach(msm_host->pdev);
+ if (ret)
+ return ret;
+
DBG("id=%d", msm_host->id);
if (msm_host->dev)
queue_work(msm_host->workqueue, &msm_host->hpd_work);
@@ -1598,6 +1602,8 @@ static int dsi_host_detach(struct mipi_dsi_host *host,
{
struct msm_dsi_host *msm_host = to_msm_dsi_host(host);
+ dsi_dev_detach(msm_host->pdev);
+
msm_host->device_node = NULL;
DBG("id=%d", msm_host->id);
@@ -1933,7 +1939,7 @@ int msm_dsi_host_modeset_init(struct mipi_dsi_host *host,
return 0;
}
-int msm_dsi_host_register(struct mipi_dsi_host *host, bool check_defer)
+int msm_dsi_host_register(struct mipi_dsi_host *host)
{
struct msm_dsi_host *msm_host = to_msm_dsi_host(host);
int ret;
@@ -1947,20 +1953,6 @@ int msm_dsi_host_register(struct mipi_dsi_host *host, bool check_defer)
return ret;
msm_host->registered = true;
-
- /* If the panel driver has not been probed after host register,
- * we should defer the host's probe.
- * It makes sure panel is connected when fbcon detects
- * connector status and gets the proper display mode to
- * create framebuffer.
- * Don't try to defer if there is nothing connected to the dsi
- * output
- */
- if (check_defer && msm_host->device_node) {
- if (IS_ERR(of_drm_find_panel(msm_host->device_node)))
- if (!of_drm_find_bridge(msm_host->device_node))
- return -EPROBE_DEFER;
- }
}
return 0;
@@ -2028,7 +2020,7 @@ void msm_dsi_host_xfer_restore(struct mipi_dsi_host *host,
/* TODO: unvote for bus bandwidth */
cfg_hnd->ops->link_clk_disable(msm_host);
- pm_runtime_put_autosuspend(&msm_host->pdev->dev);
+ pm_runtime_put(&msm_host->pdev->dev);
}
int msm_dsi_host_cmd_tx(struct mipi_dsi_host *host,
@@ -2187,57 +2179,12 @@ void msm_dsi_host_cmd_xfer_commit(struct mipi_dsi_host *host, u32 dma_base,
wmb();
}
-int msm_dsi_host_set_src_pll(struct mipi_dsi_host *host,
+void msm_dsi_host_set_phy_mode(struct mipi_dsi_host *host,
struct msm_dsi_phy *src_phy)
{
struct msm_dsi_host *msm_host = to_msm_dsi_host(host);
- struct clk *byte_clk_provider, *pixel_clk_provider;
- int ret;
msm_host->cphy_mode = src_phy->cphy_mode;
-
- ret = msm_dsi_phy_get_clk_provider(src_phy,
- &byte_clk_provider, &pixel_clk_provider);
- if (ret) {
- pr_info("%s: can't get provider from pll, don't set parent\n",
- __func__);
- return 0;
- }
-
- ret = clk_set_parent(msm_host->byte_clk_src, byte_clk_provider);
- if (ret) {
- pr_err("%s: can't set parent to byte_clk_src. ret=%d\n",
- __func__, ret);
- goto exit;
- }
-
- ret = clk_set_parent(msm_host->pixel_clk_src, pixel_clk_provider);
- if (ret) {
- pr_err("%s: can't set parent to pixel_clk_src. ret=%d\n",
- __func__, ret);
- goto exit;
- }
-
- if (msm_host->dsi_clk_src) {
- ret = clk_set_parent(msm_host->dsi_clk_src, pixel_clk_provider);
- if (ret) {
- pr_err("%s: can't set parent to dsi_clk_src. ret=%d\n",
- __func__, ret);
- goto exit;
- }
- }
-
- if (msm_host->esc_clk_src) {
- ret = clk_set_parent(msm_host->esc_clk_src, byte_clk_provider);
- if (ret) {
- pr_err("%s: can't set parent to esc_clk_src. ret=%d\n",
- __func__, ret);
- goto exit;
- }
- }
-
-exit:
- return ret;
}
void msm_dsi_host_reset_phy(struct mipi_dsi_host *host)
@@ -2305,7 +2252,7 @@ int msm_dsi_host_enable(struct mipi_dsi_host *host)
*/
/* if (msm_panel->mode == MSM_DSI_CMD_MODE) {
* dsi_link_clk_disable(msm_host);
- * pm_runtime_put_autosuspend(&msm_host->pdev->dev);
+ * pm_runtime_put(&msm_host->pdev->dev);
* }
*/
msm_host->enabled = true;
@@ -2397,7 +2344,7 @@ int msm_dsi_host_power_on(struct mipi_dsi_host *host,
fail_disable_clk:
cfg_hnd->ops->link_clk_disable(msm_host);
- pm_runtime_put_autosuspend(&msm_host->pdev->dev);
+ pm_runtime_put(&msm_host->pdev->dev);
fail_disable_reg:
dsi_host_regulator_disable(msm_host);
unlock_ret:
@@ -2424,7 +2371,7 @@ int msm_dsi_host_power_off(struct mipi_dsi_host *host)
pinctrl_pm_select_sleep_state(&msm_host->pdev->dev);
cfg_hnd->ops->link_clk_disable(msm_host);
- pm_runtime_put_autosuspend(&msm_host->pdev->dev);
+ pm_runtime_put(&msm_host->pdev->dev);
dsi_host_regulator_disable(msm_host);
diff --git a/drivers/gpu/drm/msm/dsi/dsi_manager.c b/drivers/gpu/drm/msm/dsi/dsi_manager.c
index 20c4d65..f19bae4 100644
--- a/drivers/gpu/drm/msm/dsi/dsi_manager.c
+++ b/drivers/gpu/drm/msm/dsi/dsi_manager.c
@@ -74,15 +74,13 @@ static int dsi_mgr_setup_components(int id)
int ret;
if (!IS_BONDED_DSI()) {
- ret = msm_dsi_host_register(msm_dsi->host, true);
+ ret = msm_dsi_host_register(msm_dsi->host);
if (ret)
return ret;
msm_dsi_phy_set_usecase(msm_dsi->phy, MSM_DSI_PHY_STANDALONE);
- ret = msm_dsi_host_set_src_pll(msm_dsi->host, msm_dsi->phy);
- } else if (!other_dsi) {
- ret = 0;
- } else {
+ msm_dsi_host_set_phy_mode(msm_dsi->host, msm_dsi->phy);
+ } else if (other_dsi) {
struct msm_dsi *master_link_dsi = IS_MASTER_DSI_LINK(id) ?
msm_dsi : other_dsi;
struct msm_dsi *slave_link_dsi = IS_MASTER_DSI_LINK(id) ?
@@ -94,10 +92,10 @@ static int dsi_mgr_setup_components(int id)
* because only master DSI device adds the panel to global
* panel list. The panel's device is the master DSI device.
*/
- ret = msm_dsi_host_register(slave_link_dsi->host, false);
+ ret = msm_dsi_host_register(slave_link_dsi->host);
if (ret)
return ret;
- ret = msm_dsi_host_register(master_link_dsi->host, true);
+ ret = msm_dsi_host_register(master_link_dsi->host);
if (ret)
return ret;
@@ -106,13 +104,11 @@ static int dsi_mgr_setup_components(int id)
MSM_DSI_PHY_MASTER);
msm_dsi_phy_set_usecase(clk_slave_dsi->phy,
MSM_DSI_PHY_SLAVE);
- ret = msm_dsi_host_set_src_pll(msm_dsi->host, clk_master_dsi->phy);
- if (ret)
- return ret;
- ret = msm_dsi_host_set_src_pll(other_dsi->host, clk_master_dsi->phy);
+ msm_dsi_host_set_phy_mode(msm_dsi->host, msm_dsi->phy);
+ msm_dsi_host_set_phy_mode(other_dsi->host, other_dsi->phy);
}
- return ret;
+ return 0;
}
static int enable_phy(struct msm_dsi *msm_dsi,
@@ -649,23 +645,6 @@ struct drm_connector *msm_dsi_manager_connector_init(u8 id)
return ERR_PTR(ret);
}
-bool msm_dsi_manager_validate_current_config(u8 id)
-{
- bool is_bonded_dsi = IS_BONDED_DSI();
-
- /*
- * For bonded DSI, we only have one drm panel. For this
- * use case, we register only one bridge/connector.
- * Skip bridge/connector initialisation if it is
- * slave-DSI for bonded DSI configuration.
- */
- if (is_bonded_dsi && !IS_MASTER_DSI_LINK(id)) {
- DBG("Skip bridge registration for slave DSI->id: %d\n", id);
- return false;
- }
- return true;
-}
-
/* initialize bridge */
struct drm_bridge *msm_dsi_manager_bridge_init(u8 id)
{
diff --git a/drivers/gpu/drm/msm/dsi/phy/dsi_phy.c b/drivers/gpu/drm/msm/dsi/phy/dsi_phy.c
index 9842e04..c2ed177 100644
--- a/drivers/gpu/drm/msm/dsi/phy/dsi_phy.c
+++ b/drivers/gpu/drm/msm/dsi/phy/dsi_phy.c
@@ -602,7 +602,7 @@ static int dsi_phy_enable_resource(struct msm_dsi_phy *phy)
static void dsi_phy_disable_resource(struct msm_dsi_phy *phy)
{
clk_disable_unprepare(phy->ahb_clk);
- pm_runtime_put_autosuspend(&phy->pdev->dev);
+ pm_runtime_put(&phy->pdev->dev);
}
static const struct of_device_id dsi_phy_dt_match[] = {
@@ -892,17 +892,6 @@ bool msm_dsi_phy_set_continuous_clock(struct msm_dsi_phy *phy, bool enable)
return phy->cfg->ops.set_continuous_clock(phy, enable);
}
-int msm_dsi_phy_get_clk_provider(struct msm_dsi_phy *phy,
- struct clk **byte_clk_provider, struct clk **pixel_clk_provider)
-{
- if (byte_clk_provider)
- *byte_clk_provider = phy->provided_clocks->hws[DSI_BYTE_PLL_CLK]->clk;
- if (pixel_clk_provider)
- *pixel_clk_provider = phy->provided_clocks->hws[DSI_PIXEL_PLL_CLK]->clk;
-
- return 0;
-}
-
void msm_dsi_phy_pll_save_state(struct msm_dsi_phy *phy)
{
if (phy->cfg->ops.save_pll_state) {
diff --git a/drivers/gpu/drm/msm/edp/edp.c b/drivers/gpu/drm/msm/edp/edp.c
deleted file mode 100644
index 106a674..0000000
--- a/drivers/gpu/drm/msm/edp/edp.c
+++ /dev/null
@@ -1,198 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-only
-/*
- * Copyright (c) 2014-2015, The Linux Foundation. All rights reserved.
- */
-
-#include <linux/of_irq.h>
-#include "edp.h"
-
-static irqreturn_t edp_irq(int irq, void *dev_id)
-{
- struct msm_edp *edp = dev_id;
-
- /* Process eDP irq */
- return msm_edp_ctrl_irq(edp->ctrl);
-}
-
-static void edp_destroy(struct platform_device *pdev)
-{
- struct msm_edp *edp = platform_get_drvdata(pdev);
-
- if (!edp)
- return;
-
- if (edp->ctrl) {
- msm_edp_ctrl_destroy(edp->ctrl);
- edp->ctrl = NULL;
- }
-
- platform_set_drvdata(pdev, NULL);
-}
-
-/* construct eDP at bind/probe time, grab all the resources. */
-static struct msm_edp *edp_init(struct platform_device *pdev)
-{
- struct msm_edp *edp = NULL;
- int ret;
-
- if (!pdev) {
- pr_err("no eDP device\n");
- ret = -ENXIO;
- goto fail;
- }
-
- edp = devm_kzalloc(&pdev->dev, sizeof(*edp), GFP_KERNEL);
- if (!edp) {
- ret = -ENOMEM;
- goto fail;
- }
- DBG("eDP probed=%p", edp);
-
- edp->pdev = pdev;
- platform_set_drvdata(pdev, edp);
-
- ret = msm_edp_ctrl_init(edp);
- if (ret)
- goto fail;
-
- return edp;
-
-fail:
- if (edp)
- edp_destroy(pdev);
-
- return ERR_PTR(ret);
-}
-
-static int edp_bind(struct device *dev, struct device *master, void *data)
-{
- struct drm_device *drm = dev_get_drvdata(master);
- struct msm_drm_private *priv = drm->dev_private;
- struct msm_edp *edp;
-
- DBG("");
- edp = edp_init(to_platform_device(dev));
- if (IS_ERR(edp))
- return PTR_ERR(edp);
- priv->edp = edp;
-
- return 0;
-}
-
-static void edp_unbind(struct device *dev, struct device *master, void *data)
-{
- struct drm_device *drm = dev_get_drvdata(master);
- struct msm_drm_private *priv = drm->dev_private;
-
- DBG("");
- if (priv->edp) {
- edp_destroy(to_platform_device(dev));
- priv->edp = NULL;
- }
-}
-
-static const struct component_ops edp_ops = {
- .bind = edp_bind,
- .unbind = edp_unbind,
-};
-
-static int edp_dev_probe(struct platform_device *pdev)
-{
- DBG("");
- return component_add(&pdev->dev, &edp_ops);
-}
-
-static int edp_dev_remove(struct platform_device *pdev)
-{
- DBG("");
- component_del(&pdev->dev, &edp_ops);
- return 0;
-}
-
-static const struct of_device_id dt_match[] = {
- { .compatible = "qcom,mdss-edp" },
- {}
-};
-
-static struct platform_driver edp_driver = {
- .probe = edp_dev_probe,
- .remove = edp_dev_remove,
- .driver = {
- .name = "msm_edp",
- .of_match_table = dt_match,
- },
-};
-
-void __init msm_edp_register(void)
-{
- DBG("");
- platform_driver_register(&edp_driver);
-}
-
-void __exit msm_edp_unregister(void)
-{
- DBG("");
- platform_driver_unregister(&edp_driver);
-}
-
-/* Second part of initialization, the drm/kms level modeset_init */
-int msm_edp_modeset_init(struct msm_edp *edp, struct drm_device *dev,
- struct drm_encoder *encoder)
-{
- struct platform_device *pdev = edp->pdev;
- struct msm_drm_private *priv = dev->dev_private;
- int ret;
-
- edp->encoder = encoder;
- edp->dev = dev;
-
- edp->bridge = msm_edp_bridge_init(edp);
- if (IS_ERR(edp->bridge)) {
- ret = PTR_ERR(edp->bridge);
- DRM_DEV_ERROR(dev->dev, "failed to create eDP bridge: %d\n", ret);
- edp->bridge = NULL;
- goto fail;
- }
-
- edp->connector = msm_edp_connector_init(edp);
- if (IS_ERR(edp->connector)) {
- ret = PTR_ERR(edp->connector);
- DRM_DEV_ERROR(dev->dev, "failed to create eDP connector: %d\n", ret);
- edp->connector = NULL;
- goto fail;
- }
-
- edp->irq = irq_of_parse_and_map(pdev->dev.of_node, 0);
- if (edp->irq < 0) {
- ret = edp->irq;
- DRM_DEV_ERROR(dev->dev, "failed to get IRQ: %d\n", ret);
- goto fail;
- }
-
- ret = devm_request_irq(&pdev->dev, edp->irq,
- edp_irq, IRQF_TRIGGER_HIGH | IRQF_ONESHOT,
- "edp_isr", edp);
- if (ret < 0) {
- DRM_DEV_ERROR(dev->dev, "failed to request IRQ%u: %d\n",
- edp->irq, ret);
- goto fail;
- }
-
- priv->bridges[priv->num_bridges++] = edp->bridge;
- priv->connectors[priv->num_connectors++] = edp->connector;
-
- return 0;
-
-fail:
- /* bridge/connector are normally destroyed by drm */
- if (edp->bridge) {
- edp_bridge_destroy(edp->bridge);
- edp->bridge = NULL;
- }
- if (edp->connector) {
- edp->connector->funcs->destroy(edp->connector);
- edp->connector = NULL;
- }
-
- return ret;
-}
diff --git a/drivers/gpu/drm/msm/edp/edp.h b/drivers/gpu/drm/msm/edp/edp.h
deleted file mode 100644
index 8590f2c..0000000
--- a/drivers/gpu/drm/msm/edp/edp.h
+++ /dev/null
@@ -1,77 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0-only */
-/*
- * Copyright (c) 2014-2015, The Linux Foundation. All rights reserved.
- */
-
-#ifndef __EDP_CONNECTOR_H__
-#define __EDP_CONNECTOR_H__
-
-#include <linux/i2c.h>
-#include <linux/interrupt.h>
-#include <linux/kernel.h>
-#include <linux/platform_device.h>
-#include <drm/drm_bridge.h>
-#include <drm/drm_crtc.h>
-#include <drm/drm_dp_helper.h>
-
-#include "msm_drv.h"
-
-#define edp_read(offset) msm_readl((offset))
-#define edp_write(offset, data) msm_writel((data), (offset))
-
-struct edp_ctrl;
-struct edp_aux;
-struct edp_phy;
-
-struct msm_edp {
- struct drm_device *dev;
- struct platform_device *pdev;
-
- struct drm_connector *connector;
- struct drm_bridge *bridge;
-
- /* the encoder we are hooked to (outside of eDP block) */
- struct drm_encoder *encoder;
-
- struct edp_ctrl *ctrl;
-
- int irq;
-};
-
-/* eDP bridge */
-struct drm_bridge *msm_edp_bridge_init(struct msm_edp *edp);
-void edp_bridge_destroy(struct drm_bridge *bridge);
-
-/* eDP connector */
-struct drm_connector *msm_edp_connector_init(struct msm_edp *edp);
-
-/* AUX */
-void *msm_edp_aux_init(struct msm_edp *edp, void __iomem *regbase, struct drm_dp_aux **drm_aux);
-void msm_edp_aux_destroy(struct device *dev, struct edp_aux *aux);
-irqreturn_t msm_edp_aux_irq(struct edp_aux *aux, u32 isr);
-void msm_edp_aux_ctrl(struct edp_aux *aux, int enable);
-
-/* Phy */
-bool msm_edp_phy_ready(struct edp_phy *phy);
-void msm_edp_phy_ctrl(struct edp_phy *phy, int enable);
-void msm_edp_phy_vm_pe_init(struct edp_phy *phy);
-void msm_edp_phy_vm_pe_cfg(struct edp_phy *phy, u32 v0, u32 v1);
-void msm_edp_phy_lane_power_ctrl(struct edp_phy *phy, bool up, u32 max_lane);
-void *msm_edp_phy_init(struct device *dev, void __iomem *regbase);
-
-/* Ctrl */
-irqreturn_t msm_edp_ctrl_irq(struct edp_ctrl *ctrl);
-void msm_edp_ctrl_power(struct edp_ctrl *ctrl, bool on);
-int msm_edp_ctrl_init(struct msm_edp *edp);
-void msm_edp_ctrl_destroy(struct edp_ctrl *ctrl);
-bool msm_edp_ctrl_panel_connected(struct edp_ctrl *ctrl);
-int msm_edp_ctrl_get_panel_info(struct edp_ctrl *ctrl,
- struct drm_connector *connector, struct edid **edid);
-int msm_edp_ctrl_timing_cfg(struct edp_ctrl *ctrl,
- const struct drm_display_mode *mode,
- const struct drm_display_info *info);
-/* @pixel_rate is in kHz */
-bool msm_edp_ctrl_pixel_clock_valid(struct edp_ctrl *ctrl,
- u32 pixel_rate, u32 *pm, u32 *pn);
-
-#endif /* __EDP_CONNECTOR_H__ */
diff --git a/drivers/gpu/drm/msm/edp/edp.xml.h b/drivers/gpu/drm/msm/edp/edp.xml.h
deleted file mode 100644
index 7907e0f..0000000
--- a/drivers/gpu/drm/msm/edp/edp.xml.h
+++ /dev/null
@@ -1,388 +0,0 @@
-#ifndef EDP_XML
-#define EDP_XML
-
-/* Autogenerated file, DO NOT EDIT manually!
-
-This file was generated by the rules-ng-ng headergen tool in this git repository:
-http://github.com/freedreno/envytools/
-git clone https://github.com/freedreno/envytools.git
-
-The rules-ng-ng source files this header was generated from are:
-- /home/robclark/src/mesa/mesa/src/freedreno/registers/msm.xml ( 981 bytes, from 2021-06-05 21:37:42)
-- /home/robclark/src/mesa/mesa/src/freedreno/registers/freedreno_copyright.xml ( 1572 bytes, from 2021-02-18 16:45:44)
-- /home/robclark/src/mesa/mesa/src/freedreno/registers/mdp/mdp4.xml ( 20912 bytes, from 2021-02-18 16:45:44)
-- /home/robclark/src/mesa/mesa/src/freedreno/registers/mdp/mdp_common.xml ( 2849 bytes, from 2021-02-18 16:45:44)
-- /home/robclark/src/mesa/mesa/src/freedreno/registers/mdp/mdp5.xml ( 37461 bytes, from 2021-02-18 16:45:44)
-- /home/robclark/src/mesa/mesa/src/freedreno/registers/dsi/dsi.xml ( 15291 bytes, from 2021-06-15 22:36:13)
-- /home/robclark/src/mesa/mesa/src/freedreno/registers/dsi/dsi_phy_v2.xml ( 3236 bytes, from 2021-06-05 21:37:42)
-- /home/robclark/src/mesa/mesa/src/freedreno/registers/dsi/dsi_phy_28nm_8960.xml ( 4935 bytes, from 2021-05-21 19:18:08)
-- /home/robclark/src/mesa/mesa/src/freedreno/registers/dsi/dsi_phy_28nm.xml ( 7004 bytes, from 2021-05-21 19:18:08)
-- /home/robclark/src/mesa/mesa/src/freedreno/registers/dsi/dsi_phy_20nm.xml ( 3712 bytes, from 2021-05-21 19:18:08)
-- /home/robclark/src/mesa/mesa/src/freedreno/registers/dsi/dsi_phy_14nm.xml ( 5381 bytes, from 2021-05-21 19:18:08)
-- /home/robclark/src/mesa/mesa/src/freedreno/registers/dsi/dsi_phy_10nm.xml ( 4499 bytes, from 2021-05-21 19:18:08)
-- /home/robclark/src/mesa/mesa/src/freedreno/registers/dsi/dsi_phy_7nm.xml ( 10953 bytes, from 2021-05-21 19:18:08)
-- /home/robclark/src/mesa/mesa/src/freedreno/registers/dsi/dsi_phy_5nm.xml ( 10900 bytes, from 2021-05-21 19:18:08)
-- /home/robclark/src/mesa/mesa/src/freedreno/registers/dsi/sfpb.xml ( 602 bytes, from 2021-02-18 16:45:44)
-- /home/robclark/src/mesa/mesa/src/freedreno/registers/dsi/mmss_cc.xml ( 1686 bytes, from 2021-02-18 16:45:44)
-- /home/robclark/src/mesa/mesa/src/freedreno/registers/hdmi/qfprom.xml ( 600 bytes, from 2021-02-18 16:45:44)
-- /home/robclark/src/mesa/mesa/src/freedreno/registers/hdmi/hdmi.xml ( 41874 bytes, from 2021-02-18 16:45:44)
-- /home/robclark/src/mesa/mesa/src/freedreno/registers/edp/edp.xml ( 10416 bytes, from 2021-02-18 16:45:44)
-
-Copyright (C) 2013-2021 by the following authors:
-- Rob Clark <robdclark@gmail.com> (robclark)
-- Ilia Mirkin <imirkin@alum.mit.edu> (imirkin)
-
-Permission is hereby granted, free of charge, to any person obtaining
-a copy of this software and associated documentation files (the
-"Software"), to deal in the Software without restriction, including
-without limitation the rights to use, copy, modify, merge, publish,
-distribute, sublicense, and/or sell copies of the Software, and to
-permit persons to whom the Software is furnished to do so, subject to
-the following conditions:
-
-The above copyright notice and this permission notice (including the
-next paragraph) shall be included in all copies or substantial
-portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
-EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
-MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
-IN NO EVENT SHALL THE COPYRIGHT OWNER(S) AND/OR ITS SUPPLIERS BE
-LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
-OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
-WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
-*/
-
-
-enum edp_color_depth {
- EDP_6BIT = 0,
- EDP_8BIT = 1,
- EDP_10BIT = 2,
- EDP_12BIT = 3,
- EDP_16BIT = 4,
-};
-
-enum edp_component_format {
- EDP_RGB = 0,
- EDP_YUV422 = 1,
- EDP_YUV444 = 2,
-};
-
-#define REG_EDP_MAINLINK_CTRL 0x00000004
-#define EDP_MAINLINK_CTRL_ENABLE 0x00000001
-#define EDP_MAINLINK_CTRL_RESET 0x00000002
-
-#define REG_EDP_STATE_CTRL 0x00000008
-#define EDP_STATE_CTRL_TRAIN_PATTERN_1 0x00000001
-#define EDP_STATE_CTRL_TRAIN_PATTERN_2 0x00000002
-#define EDP_STATE_CTRL_TRAIN_PATTERN_3 0x00000004
-#define EDP_STATE_CTRL_SYMBOL_ERR_RATE_MEAS 0x00000008
-#define EDP_STATE_CTRL_PRBS7 0x00000010
-#define EDP_STATE_CTRL_CUSTOM_80_BIT_PATTERN 0x00000020
-#define EDP_STATE_CTRL_SEND_VIDEO 0x00000040
-#define EDP_STATE_CTRL_PUSH_IDLE 0x00000080
-
-#define REG_EDP_CONFIGURATION_CTRL 0x0000000c
-#define EDP_CONFIGURATION_CTRL_SYNC_CLK 0x00000001
-#define EDP_CONFIGURATION_CTRL_STATIC_MVID 0x00000002
-#define EDP_CONFIGURATION_CTRL_PROGRESSIVE 0x00000004
-#define EDP_CONFIGURATION_CTRL_LANES__MASK 0x00000030
-#define EDP_CONFIGURATION_CTRL_LANES__SHIFT 4
-static inline uint32_t EDP_CONFIGURATION_CTRL_LANES(uint32_t val)
-{
- return ((val) << EDP_CONFIGURATION_CTRL_LANES__SHIFT) & EDP_CONFIGURATION_CTRL_LANES__MASK;
-}
-#define EDP_CONFIGURATION_CTRL_ENHANCED_FRAMING 0x00000040
-#define EDP_CONFIGURATION_CTRL_COLOR__MASK 0x00000100
-#define EDP_CONFIGURATION_CTRL_COLOR__SHIFT 8
-static inline uint32_t EDP_CONFIGURATION_CTRL_COLOR(enum edp_color_depth val)
-{
- return ((val) << EDP_CONFIGURATION_CTRL_COLOR__SHIFT) & EDP_CONFIGURATION_CTRL_COLOR__MASK;
-}
-
-#define REG_EDP_SOFTWARE_MVID 0x00000014
-
-#define REG_EDP_SOFTWARE_NVID 0x00000018
-
-#define REG_EDP_TOTAL_HOR_VER 0x0000001c
-#define EDP_TOTAL_HOR_VER_HORIZ__MASK 0x0000ffff
-#define EDP_TOTAL_HOR_VER_HORIZ__SHIFT 0
-static inline uint32_t EDP_TOTAL_HOR_VER_HORIZ(uint32_t val)
-{
- return ((val) << EDP_TOTAL_HOR_VER_HORIZ__SHIFT) & EDP_TOTAL_HOR_VER_HORIZ__MASK;
-}
-#define EDP_TOTAL_HOR_VER_VERT__MASK 0xffff0000
-#define EDP_TOTAL_HOR_VER_VERT__SHIFT 16
-static inline uint32_t EDP_TOTAL_HOR_VER_VERT(uint32_t val)
-{
- return ((val) << EDP_TOTAL_HOR_VER_VERT__SHIFT) & EDP_TOTAL_HOR_VER_VERT__MASK;
-}
-
-#define REG_EDP_START_HOR_VER_FROM_SYNC 0x00000020
-#define EDP_START_HOR_VER_FROM_SYNC_HORIZ__MASK 0x0000ffff
-#define EDP_START_HOR_VER_FROM_SYNC_HORIZ__SHIFT 0
-static inline uint32_t EDP_START_HOR_VER_FROM_SYNC_HORIZ(uint32_t val)
-{
- return ((val) << EDP_START_HOR_VER_FROM_SYNC_HORIZ__SHIFT) & EDP_START_HOR_VER_FROM_SYNC_HORIZ__MASK;
-}
-#define EDP_START_HOR_VER_FROM_SYNC_VERT__MASK 0xffff0000
-#define EDP_START_HOR_VER_FROM_SYNC_VERT__SHIFT 16
-static inline uint32_t EDP_START_HOR_VER_FROM_SYNC_VERT(uint32_t val)
-{
- return ((val) << EDP_START_HOR_VER_FROM_SYNC_VERT__SHIFT) & EDP_START_HOR_VER_FROM_SYNC_VERT__MASK;
-}
-
-#define REG_EDP_HSYNC_VSYNC_WIDTH_POLARITY 0x00000024
-#define EDP_HSYNC_VSYNC_WIDTH_POLARITY_HORIZ__MASK 0x00007fff
-#define EDP_HSYNC_VSYNC_WIDTH_POLARITY_HORIZ__SHIFT 0
-static inline uint32_t EDP_HSYNC_VSYNC_WIDTH_POLARITY_HORIZ(uint32_t val)
-{
- return ((val) << EDP_HSYNC_VSYNC_WIDTH_POLARITY_HORIZ__SHIFT) & EDP_HSYNC_VSYNC_WIDTH_POLARITY_HORIZ__MASK;
-}
-#define EDP_HSYNC_VSYNC_WIDTH_POLARITY_NHSYNC 0x00008000
-#define EDP_HSYNC_VSYNC_WIDTH_POLARITY_VERT__MASK 0x7fff0000
-#define EDP_HSYNC_VSYNC_WIDTH_POLARITY_VERT__SHIFT 16
-static inline uint32_t EDP_HSYNC_VSYNC_WIDTH_POLARITY_VERT(uint32_t val)
-{
- return ((val) << EDP_HSYNC_VSYNC_WIDTH_POLARITY_VERT__SHIFT) & EDP_HSYNC_VSYNC_WIDTH_POLARITY_VERT__MASK;
-}
-#define EDP_HSYNC_VSYNC_WIDTH_POLARITY_NVSYNC 0x80000000
-
-#define REG_EDP_ACTIVE_HOR_VER 0x00000028
-#define EDP_ACTIVE_HOR_VER_HORIZ__MASK 0x0000ffff
-#define EDP_ACTIVE_HOR_VER_HORIZ__SHIFT 0
-static inline uint32_t EDP_ACTIVE_HOR_VER_HORIZ(uint32_t val)
-{
- return ((val) << EDP_ACTIVE_HOR_VER_HORIZ__SHIFT) & EDP_ACTIVE_HOR_VER_HORIZ__MASK;
-}
-#define EDP_ACTIVE_HOR_VER_VERT__MASK 0xffff0000
-#define EDP_ACTIVE_HOR_VER_VERT__SHIFT 16
-static inline uint32_t EDP_ACTIVE_HOR_VER_VERT(uint32_t val)
-{
- return ((val) << EDP_ACTIVE_HOR_VER_VERT__SHIFT) & EDP_ACTIVE_HOR_VER_VERT__MASK;
-}
-
-#define REG_EDP_MISC1_MISC0 0x0000002c
-#define EDP_MISC1_MISC0_MISC0__MASK 0x000000ff
-#define EDP_MISC1_MISC0_MISC0__SHIFT 0
-static inline uint32_t EDP_MISC1_MISC0_MISC0(uint32_t val)
-{
- return ((val) << EDP_MISC1_MISC0_MISC0__SHIFT) & EDP_MISC1_MISC0_MISC0__MASK;
-}
-#define EDP_MISC1_MISC0_SYNC 0x00000001
-#define EDP_MISC1_MISC0_COMPONENT_FORMAT__MASK 0x00000006
-#define EDP_MISC1_MISC0_COMPONENT_FORMAT__SHIFT 1
-static inline uint32_t EDP_MISC1_MISC0_COMPONENT_FORMAT(enum edp_component_format val)
-{
- return ((val) << EDP_MISC1_MISC0_COMPONENT_FORMAT__SHIFT) & EDP_MISC1_MISC0_COMPONENT_FORMAT__MASK;
-}
-#define EDP_MISC1_MISC0_CEA 0x00000008
-#define EDP_MISC1_MISC0_BT709_5 0x00000010
-#define EDP_MISC1_MISC0_COLOR__MASK 0x000000e0
-#define EDP_MISC1_MISC0_COLOR__SHIFT 5
-static inline uint32_t EDP_MISC1_MISC0_COLOR(enum edp_color_depth val)
-{
- return ((val) << EDP_MISC1_MISC0_COLOR__SHIFT) & EDP_MISC1_MISC0_COLOR__MASK;
-}
-#define EDP_MISC1_MISC0_MISC1__MASK 0x0000ff00
-#define EDP_MISC1_MISC0_MISC1__SHIFT 8
-static inline uint32_t EDP_MISC1_MISC0_MISC1(uint32_t val)
-{
- return ((val) << EDP_MISC1_MISC0_MISC1__SHIFT) & EDP_MISC1_MISC0_MISC1__MASK;
-}
-#define EDP_MISC1_MISC0_INTERLACED_ODD 0x00000100
-#define EDP_MISC1_MISC0_STEREO__MASK 0x00000600
-#define EDP_MISC1_MISC0_STEREO__SHIFT 9
-static inline uint32_t EDP_MISC1_MISC0_STEREO(uint32_t val)
-{
- return ((val) << EDP_MISC1_MISC0_STEREO__SHIFT) & EDP_MISC1_MISC0_STEREO__MASK;
-}
-
-#define REG_EDP_PHY_CTRL 0x00000074
-#define EDP_PHY_CTRL_SW_RESET_PLL 0x00000001
-#define EDP_PHY_CTRL_SW_RESET 0x00000004
-
-#define REG_EDP_MAINLINK_READY 0x00000084
-#define EDP_MAINLINK_READY_TRAIN_PATTERN_1_READY 0x00000008
-#define EDP_MAINLINK_READY_TRAIN_PATTERN_2_READY 0x00000010
-#define EDP_MAINLINK_READY_TRAIN_PATTERN_3_READY 0x00000020
-
-#define REG_EDP_AUX_CTRL 0x00000300
-#define EDP_AUX_CTRL_ENABLE 0x00000001
-#define EDP_AUX_CTRL_RESET 0x00000002
-
-#define REG_EDP_INTERRUPT_REG_1 0x00000308
-#define EDP_INTERRUPT_REG_1_HPD 0x00000001
-#define EDP_INTERRUPT_REG_1_HPD_ACK 0x00000002
-#define EDP_INTERRUPT_REG_1_HPD_EN 0x00000004
-#define EDP_INTERRUPT_REG_1_AUX_I2C_DONE 0x00000008
-#define EDP_INTERRUPT_REG_1_AUX_I2C_DONE_ACK 0x00000010
-#define EDP_INTERRUPT_REG_1_AUX_I2C_DONE_EN 0x00000020
-#define EDP_INTERRUPT_REG_1_WRONG_ADDR 0x00000040
-#define EDP_INTERRUPT_REG_1_WRONG_ADDR_ACK 0x00000080
-#define EDP_INTERRUPT_REG_1_WRONG_ADDR_EN 0x00000100
-#define EDP_INTERRUPT_REG_1_TIMEOUT 0x00000200
-#define EDP_INTERRUPT_REG_1_TIMEOUT_ACK 0x00000400
-#define EDP_INTERRUPT_REG_1_TIMEOUT_EN 0x00000800
-#define EDP_INTERRUPT_REG_1_NACK_DEFER 0x00001000
-#define EDP_INTERRUPT_REG_1_NACK_DEFER_ACK 0x00002000
-#define EDP_INTERRUPT_REG_1_NACK_DEFER_EN 0x00004000
-#define EDP_INTERRUPT_REG_1_WRONG_DATA_CNT 0x00008000
-#define EDP_INTERRUPT_REG_1_WRONG_DATA_CNT_ACK 0x00010000
-#define EDP_INTERRUPT_REG_1_WRONG_DATA_CNT_EN 0x00020000
-#define EDP_INTERRUPT_REG_1_I2C_NACK 0x00040000
-#define EDP_INTERRUPT_REG_1_I2C_NACK_ACK 0x00080000
-#define EDP_INTERRUPT_REG_1_I2C_NACK_EN 0x00100000
-#define EDP_INTERRUPT_REG_1_I2C_DEFER 0x00200000
-#define EDP_INTERRUPT_REG_1_I2C_DEFER_ACK 0x00400000
-#define EDP_INTERRUPT_REG_1_I2C_DEFER_EN 0x00800000
-#define EDP_INTERRUPT_REG_1_PLL_UNLOCK 0x01000000
-#define EDP_INTERRUPT_REG_1_PLL_UNLOCK_ACK 0x02000000
-#define EDP_INTERRUPT_REG_1_PLL_UNLOCK_EN 0x04000000
-#define EDP_INTERRUPT_REG_1_AUX_ERROR 0x08000000
-#define EDP_INTERRUPT_REG_1_AUX_ERROR_ACK 0x10000000
-#define EDP_INTERRUPT_REG_1_AUX_ERROR_EN 0x20000000
-
-#define REG_EDP_INTERRUPT_REG_2 0x0000030c
-#define EDP_INTERRUPT_REG_2_READY_FOR_VIDEO 0x00000001
-#define EDP_INTERRUPT_REG_2_READY_FOR_VIDEO_ACK 0x00000002
-#define EDP_INTERRUPT_REG_2_READY_FOR_VIDEO_EN 0x00000004
-#define EDP_INTERRUPT_REG_2_IDLE_PATTERNs_SENT 0x00000008
-#define EDP_INTERRUPT_REG_2_IDLE_PATTERNs_SENT_ACK 0x00000010
-#define EDP_INTERRUPT_REG_2_IDLE_PATTERNs_SENT_EN 0x00000020
-#define EDP_INTERRUPT_REG_2_FRAME_END 0x00000200
-#define EDP_INTERRUPT_REG_2_FRAME_END_ACK 0x00000080
-#define EDP_INTERRUPT_REG_2_FRAME_END_EN 0x00000100
-#define EDP_INTERRUPT_REG_2_CRC_UPDATED 0x00000200
-#define EDP_INTERRUPT_REG_2_CRC_UPDATED_ACK 0x00000400
-#define EDP_INTERRUPT_REG_2_CRC_UPDATED_EN 0x00000800
-
-#define REG_EDP_INTERRUPT_TRANS_NUM 0x00000310
-
-#define REG_EDP_AUX_DATA 0x00000314
-#define EDP_AUX_DATA_READ 0x00000001
-#define EDP_AUX_DATA_DATA__MASK 0x0000ff00
-#define EDP_AUX_DATA_DATA__SHIFT 8
-static inline uint32_t EDP_AUX_DATA_DATA(uint32_t val)
-{
- return ((val) << EDP_AUX_DATA_DATA__SHIFT) & EDP_AUX_DATA_DATA__MASK;
-}
-#define EDP_AUX_DATA_INDEX__MASK 0x00ff0000
-#define EDP_AUX_DATA_INDEX__SHIFT 16
-static inline uint32_t EDP_AUX_DATA_INDEX(uint32_t val)
-{
- return ((val) << EDP_AUX_DATA_INDEX__SHIFT) & EDP_AUX_DATA_INDEX__MASK;
-}
-#define EDP_AUX_DATA_INDEX_WRITE 0x80000000
-
-#define REG_EDP_AUX_TRANS_CTRL 0x00000318
-#define EDP_AUX_TRANS_CTRL_I2C 0x00000100
-#define EDP_AUX_TRANS_CTRL_GO 0x00000200
-
-#define REG_EDP_AUX_STATUS 0x00000324
-
-static inline uint32_t REG_EDP_PHY_LN(uint32_t i0) { return 0x00000400 + 0x40*i0; }
-
-static inline uint32_t REG_EDP_PHY_LN_PD_CTL(uint32_t i0) { return 0x00000404 + 0x40*i0; }
-
-#define REG_EDP_PHY_GLB_VM_CFG0 0x00000510
-
-#define REG_EDP_PHY_GLB_VM_CFG1 0x00000514
-
-#define REG_EDP_PHY_GLB_MISC9 0x00000518
-
-#define REG_EDP_PHY_GLB_CFG 0x00000528
-
-#define REG_EDP_PHY_GLB_PD_CTL 0x0000052c
-
-#define REG_EDP_PHY_GLB_PHY_STATUS 0x00000598
-
-#define REG_EDP_28nm_PHY_PLL_REFCLK_CFG 0x00000000
-
-#define REG_EDP_28nm_PHY_PLL_POSTDIV1_CFG 0x00000004
-
-#define REG_EDP_28nm_PHY_PLL_CHGPUMP_CFG 0x00000008
-
-#define REG_EDP_28nm_PHY_PLL_VCOLPF_CFG 0x0000000c
-
-#define REG_EDP_28nm_PHY_PLL_VREG_CFG 0x00000010
-
-#define REG_EDP_28nm_PHY_PLL_PWRGEN_CFG 0x00000014
-
-#define REG_EDP_28nm_PHY_PLL_DMUX_CFG 0x00000018
-
-#define REG_EDP_28nm_PHY_PLL_AMUX_CFG 0x0000001c
-
-#define REG_EDP_28nm_PHY_PLL_GLB_CFG 0x00000020
-#define EDP_28nm_PHY_PLL_GLB_CFG_PLL_PWRDN_B 0x00000001
-#define EDP_28nm_PHY_PLL_GLB_CFG_PLL_LDO_PWRDN_B 0x00000002
-#define EDP_28nm_PHY_PLL_GLB_CFG_PLL_PWRGEN_PWRDN_B 0x00000004
-#define EDP_28nm_PHY_PLL_GLB_CFG_PLL_ENABLE 0x00000008
-
-#define REG_EDP_28nm_PHY_PLL_POSTDIV2_CFG 0x00000024
-
-#define REG_EDP_28nm_PHY_PLL_POSTDIV3_CFG 0x00000028
-
-#define REG_EDP_28nm_PHY_PLL_LPFR_CFG 0x0000002c
-
-#define REG_EDP_28nm_PHY_PLL_LPFC1_CFG 0x00000030
-
-#define REG_EDP_28nm_PHY_PLL_LPFC2_CFG 0x00000034
-
-#define REG_EDP_28nm_PHY_PLL_SDM_CFG0 0x00000038
-
-#define REG_EDP_28nm_PHY_PLL_SDM_CFG1 0x0000003c
-
-#define REG_EDP_28nm_PHY_PLL_SDM_CFG2 0x00000040
-
-#define REG_EDP_28nm_PHY_PLL_SDM_CFG3 0x00000044
-
-#define REG_EDP_28nm_PHY_PLL_SDM_CFG4 0x00000048
-
-#define REG_EDP_28nm_PHY_PLL_SSC_CFG0 0x0000004c
-
-#define REG_EDP_28nm_PHY_PLL_SSC_CFG1 0x00000050
-
-#define REG_EDP_28nm_PHY_PLL_SSC_CFG2 0x00000054
-
-#define REG_EDP_28nm_PHY_PLL_SSC_CFG3 0x00000058
-
-#define REG_EDP_28nm_PHY_PLL_LKDET_CFG0 0x0000005c
-
-#define REG_EDP_28nm_PHY_PLL_LKDET_CFG1 0x00000060
-
-#define REG_EDP_28nm_PHY_PLL_LKDET_CFG2 0x00000064
-
-#define REG_EDP_28nm_PHY_PLL_TEST_CFG 0x00000068
-#define EDP_28nm_PHY_PLL_TEST_CFG_PLL_SW_RESET 0x00000001
-
-#define REG_EDP_28nm_PHY_PLL_CAL_CFG0 0x0000006c
-
-#define REG_EDP_28nm_PHY_PLL_CAL_CFG1 0x00000070
-
-#define REG_EDP_28nm_PHY_PLL_CAL_CFG2 0x00000074
-
-#define REG_EDP_28nm_PHY_PLL_CAL_CFG3 0x00000078
-
-#define REG_EDP_28nm_PHY_PLL_CAL_CFG4 0x0000007c
-
-#define REG_EDP_28nm_PHY_PLL_CAL_CFG5 0x00000080
-
-#define REG_EDP_28nm_PHY_PLL_CAL_CFG6 0x00000084
-
-#define REG_EDP_28nm_PHY_PLL_CAL_CFG7 0x00000088
-
-#define REG_EDP_28nm_PHY_PLL_CAL_CFG8 0x0000008c
-
-#define REG_EDP_28nm_PHY_PLL_CAL_CFG9 0x00000090
-
-#define REG_EDP_28nm_PHY_PLL_CAL_CFG10 0x00000094
-
-#define REG_EDP_28nm_PHY_PLL_CAL_CFG11 0x00000098
-
-#define REG_EDP_28nm_PHY_PLL_EFUSE_CFG 0x0000009c
-
-#define REG_EDP_28nm_PHY_PLL_DEBUG_BUS_SEL 0x000000a0
-
-
-#endif /* EDP_XML */
diff --git a/drivers/gpu/drm/msm/edp/edp_aux.c b/drivers/gpu/drm/msm/edp/edp_aux.c
deleted file mode 100644
index e3d85c6..0000000
--- a/drivers/gpu/drm/msm/edp/edp_aux.c
+++ /dev/null
@@ -1,265 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-only
-/*
- * Copyright (c) 2014-2015, The Linux Foundation. All rights reserved.
- */
-
-#include "edp.h"
-#include "edp.xml.h"
-
-#define AUX_CMD_FIFO_LEN 144
-#define AUX_CMD_NATIVE_MAX 16
-#define AUX_CMD_I2C_MAX 128
-
-#define EDP_INTR_AUX_I2C_ERR \
- (EDP_INTERRUPT_REG_1_WRONG_ADDR | EDP_INTERRUPT_REG_1_TIMEOUT | \
- EDP_INTERRUPT_REG_1_NACK_DEFER | EDP_INTERRUPT_REG_1_WRONG_DATA_CNT | \
- EDP_INTERRUPT_REG_1_I2C_NACK | EDP_INTERRUPT_REG_1_I2C_DEFER)
-#define EDP_INTR_TRANS_STATUS \
- (EDP_INTERRUPT_REG_1_AUX_I2C_DONE | EDP_INTR_AUX_I2C_ERR)
-
-struct edp_aux {
- void __iomem *base;
- bool msg_err;
-
- struct completion msg_comp;
-
- /* To prevent the message transaction routine from reentry. */
- struct mutex msg_mutex;
-
- struct drm_dp_aux drm_aux;
-};
-#define to_edp_aux(x) container_of(x, struct edp_aux, drm_aux)
-
-static int edp_msg_fifo_tx(struct edp_aux *aux, struct drm_dp_aux_msg *msg)
-{
- u32 data[4];
- u32 reg, len;
- bool native = msg->request & (DP_AUX_NATIVE_WRITE & DP_AUX_NATIVE_READ);
- bool read = msg->request & (DP_AUX_I2C_READ & DP_AUX_NATIVE_READ);
- u8 *msgdata = msg->buffer;
- int i;
-
- if (read)
- len = 4;
- else
- len = msg->size + 4;
-
- /*
- * cmd fifo only has depth of 144 bytes
- */
- if (len > AUX_CMD_FIFO_LEN)
- return -EINVAL;
-
- /* Pack cmd and write to HW */
- data[0] = (msg->address >> 16) & 0xf; /* addr[19:16] */
- if (read)
- data[0] |= BIT(4); /* R/W */
-
- data[1] = (msg->address >> 8) & 0xff; /* addr[15:8] */
- data[2] = msg->address & 0xff; /* addr[7:0] */
- data[3] = (msg->size - 1) & 0xff; /* len[7:0] */
-
- for (i = 0; i < len; i++) {
- reg = (i < 4) ? data[i] : msgdata[i - 4];
- reg = EDP_AUX_DATA_DATA(reg); /* index = 0, write */
- if (i == 0)
- reg |= EDP_AUX_DATA_INDEX_WRITE;
- edp_write(aux->base + REG_EDP_AUX_DATA, reg);
- }
-
- reg = 0; /* Transaction number is always 1 */
- if (!native) /* i2c */
- reg |= EDP_AUX_TRANS_CTRL_I2C;
-
- reg |= EDP_AUX_TRANS_CTRL_GO;
- edp_write(aux->base + REG_EDP_AUX_TRANS_CTRL, reg);
-
- return 0;
-}
-
-static int edp_msg_fifo_rx(struct edp_aux *aux, struct drm_dp_aux_msg *msg)
-{
- u32 data;
- u8 *dp;
- int i;
- u32 len = msg->size;
-
- edp_write(aux->base + REG_EDP_AUX_DATA,
- EDP_AUX_DATA_INDEX_WRITE | EDP_AUX_DATA_READ); /* index = 0 */
-
- dp = msg->buffer;
-
- /* discard first byte */
- data = edp_read(aux->base + REG_EDP_AUX_DATA);
- for (i = 0; i < len; i++) {
- data = edp_read(aux->base + REG_EDP_AUX_DATA);
- dp[i] = (u8)((data >> 8) & 0xff);
- }
-
- return 0;
-}
-
-/*
- * This function does the real job to process an AUX transaction.
- * It will call msm_edp_aux_ctrl() function to reset the AUX channel,
- * if the waiting is timeout.
- * The caller who triggers the transaction should avoid the
- * msm_edp_aux_ctrl() running concurrently in other threads, i.e.
- * start transaction only when AUX channel is fully enabled.
- */
-static ssize_t edp_aux_transfer(struct drm_dp_aux *drm_aux,
- struct drm_dp_aux_msg *msg)
-{
- struct edp_aux *aux = to_edp_aux(drm_aux);
- ssize_t ret;
- unsigned long time_left;
- bool native = msg->request & (DP_AUX_NATIVE_WRITE & DP_AUX_NATIVE_READ);
- bool read = msg->request & (DP_AUX_I2C_READ & DP_AUX_NATIVE_READ);
-
- /* Ignore address only message */
- if ((msg->size == 0) || (msg->buffer == NULL)) {
- msg->reply = native ?
- DP_AUX_NATIVE_REPLY_ACK : DP_AUX_I2C_REPLY_ACK;
- return msg->size;
- }
-
- /* msg sanity check */
- if ((native && (msg->size > AUX_CMD_NATIVE_MAX)) ||
- (msg->size > AUX_CMD_I2C_MAX)) {
- pr_err("%s: invalid msg: size(%zu), request(%x)\n",
- __func__, msg->size, msg->request);
- return -EINVAL;
- }
-
- mutex_lock(&aux->msg_mutex);
-
- aux->msg_err = false;
- reinit_completion(&aux->msg_comp);
-
- ret = edp_msg_fifo_tx(aux, msg);
- if (ret < 0)
- goto unlock_exit;
-
- DBG("wait_for_completion");
- time_left = wait_for_completion_timeout(&aux->msg_comp,
- msecs_to_jiffies(300));
- if (!time_left) {
- /*
- * Clear GO and reset AUX channel
- * to cancel the current transaction.
- */
- edp_write(aux->base + REG_EDP_AUX_TRANS_CTRL, 0);
- msm_edp_aux_ctrl(aux, 1);
- pr_err("%s: aux timeout,\n", __func__);
- ret = -ETIMEDOUT;
- goto unlock_exit;
- }
- DBG("completion");
-
- if (!aux->msg_err) {
- if (read) {
- ret = edp_msg_fifo_rx(aux, msg);
- if (ret < 0)
- goto unlock_exit;
- }
-
- msg->reply = native ?
- DP_AUX_NATIVE_REPLY_ACK : DP_AUX_I2C_REPLY_ACK;
- } else {
- /* Reply defer to retry */
- msg->reply = native ?
- DP_AUX_NATIVE_REPLY_DEFER : DP_AUX_I2C_REPLY_DEFER;
- /*
- * The sleep time in caller is not long enough to make sure
- * our H/W completes transactions. Add more defer time here.
- */
- msleep(100);
- }
-
- /* Return requested size for success or retry */
- ret = msg->size;
-
-unlock_exit:
- mutex_unlock(&aux->msg_mutex);
- return ret;
-}
-
-void *msm_edp_aux_init(struct msm_edp *edp, void __iomem *regbase, struct drm_dp_aux **drm_aux)
-{
- struct device *dev = &edp->pdev->dev;
- struct edp_aux *aux = NULL;
- int ret;
-
- DBG("");
- aux = devm_kzalloc(dev, sizeof(*aux), GFP_KERNEL);
- if (!aux)
- return NULL;
-
- aux->base = regbase;
- mutex_init(&aux->msg_mutex);
- init_completion(&aux->msg_comp);
-
- aux->drm_aux.name = "msm_edp_aux";
- aux->drm_aux.dev = dev;
- aux->drm_aux.drm_dev = edp->dev;
- aux->drm_aux.transfer = edp_aux_transfer;
- ret = drm_dp_aux_register(&aux->drm_aux);
- if (ret) {
- pr_err("%s: failed to register drm aux: %d\n", __func__, ret);
- mutex_destroy(&aux->msg_mutex);
- }
-
- if (drm_aux && aux)
- *drm_aux = &aux->drm_aux;
-
- return aux;
-}
-
-void msm_edp_aux_destroy(struct device *dev, struct edp_aux *aux)
-{
- if (aux) {
- drm_dp_aux_unregister(&aux->drm_aux);
- mutex_destroy(&aux->msg_mutex);
- }
-}
-
-irqreturn_t msm_edp_aux_irq(struct edp_aux *aux, u32 isr)
-{
- if (isr & EDP_INTR_TRANS_STATUS) {
- DBG("isr=%x", isr);
- edp_write(aux->base + REG_EDP_AUX_TRANS_CTRL, 0);
-
- if (isr & EDP_INTR_AUX_I2C_ERR)
- aux->msg_err = true;
- else
- aux->msg_err = false;
-
- complete(&aux->msg_comp);
- }
-
- return IRQ_HANDLED;
-}
-
-void msm_edp_aux_ctrl(struct edp_aux *aux, int enable)
-{
- u32 data;
-
- DBG("enable=%d", enable);
- data = edp_read(aux->base + REG_EDP_AUX_CTRL);
-
- if (enable) {
- data |= EDP_AUX_CTRL_RESET;
- edp_write(aux->base + REG_EDP_AUX_CTRL, data);
- /* Make sure full reset */
- wmb();
- usleep_range(500, 1000);
-
- data &= ~EDP_AUX_CTRL_RESET;
- data |= EDP_AUX_CTRL_ENABLE;
- edp_write(aux->base + REG_EDP_AUX_CTRL, data);
- } else {
- data &= ~EDP_AUX_CTRL_ENABLE;
- edp_write(aux->base + REG_EDP_AUX_CTRL, data);
- }
-}
-
diff --git a/drivers/gpu/drm/msm/edp/edp_bridge.c b/drivers/gpu/drm/msm/edp/edp_bridge.c
deleted file mode 100644
index c69a37e..0000000
--- a/drivers/gpu/drm/msm/edp/edp_bridge.c
+++ /dev/null
@@ -1,111 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-only
-/*
- * Copyright (c) 2014-2015, The Linux Foundation. All rights reserved.
- */
-
-#include "edp.h"
-
-struct edp_bridge {
- struct drm_bridge base;
- struct msm_edp *edp;
-};
-#define to_edp_bridge(x) container_of(x, struct edp_bridge, base)
-
-void edp_bridge_destroy(struct drm_bridge *bridge)
-{
-}
-
-static void edp_bridge_pre_enable(struct drm_bridge *bridge)
-{
- struct edp_bridge *edp_bridge = to_edp_bridge(bridge);
- struct msm_edp *edp = edp_bridge->edp;
-
- DBG("");
- msm_edp_ctrl_power(edp->ctrl, true);
-}
-
-static void edp_bridge_enable(struct drm_bridge *bridge)
-{
- DBG("");
-}
-
-static void edp_bridge_disable(struct drm_bridge *bridge)
-{
- DBG("");
-}
-
-static void edp_bridge_post_disable(struct drm_bridge *bridge)
-{
- struct edp_bridge *edp_bridge = to_edp_bridge(bridge);
- struct msm_edp *edp = edp_bridge->edp;
-
- DBG("");
- msm_edp_ctrl_power(edp->ctrl, false);
-}
-
-static void edp_bridge_mode_set(struct drm_bridge *bridge,
- const struct drm_display_mode *mode,
- const struct drm_display_mode *adjusted_mode)
-{
- struct drm_device *dev = bridge->dev;
- struct drm_connector *connector;
- struct edp_bridge *edp_bridge = to_edp_bridge(bridge);
- struct msm_edp *edp = edp_bridge->edp;
-
- DBG("set mode: " DRM_MODE_FMT, DRM_MODE_ARG(mode));
-
- list_for_each_entry(connector, &dev->mode_config.connector_list, head) {
- struct drm_encoder *encoder = connector->encoder;
- struct drm_bridge *first_bridge;
-
- if (!connector->encoder)
- continue;
-
- first_bridge = drm_bridge_chain_get_first_bridge(encoder);
- if (bridge == first_bridge) {
- msm_edp_ctrl_timing_cfg(edp->ctrl,
- adjusted_mode, &connector->display_info);
- break;
- }
- }
-}
-
-static const struct drm_bridge_funcs edp_bridge_funcs = {
- .pre_enable = edp_bridge_pre_enable,
- .enable = edp_bridge_enable,
- .disable = edp_bridge_disable,
- .post_disable = edp_bridge_post_disable,
- .mode_set = edp_bridge_mode_set,
-};
-
-/* initialize bridge */
-struct drm_bridge *msm_edp_bridge_init(struct msm_edp *edp)
-{
- struct drm_bridge *bridge = NULL;
- struct edp_bridge *edp_bridge;
- int ret;
-
- edp_bridge = devm_kzalloc(edp->dev->dev,
- sizeof(*edp_bridge), GFP_KERNEL);
- if (!edp_bridge) {
- ret = -ENOMEM;
- goto fail;
- }
-
- edp_bridge->edp = edp;
-
- bridge = &edp_bridge->base;
- bridge->funcs = &edp_bridge_funcs;
-
- ret = drm_bridge_attach(edp->encoder, bridge, NULL, 0);
- if (ret)
- goto fail;
-
- return bridge;
-
-fail:
- if (bridge)
- edp_bridge_destroy(bridge);
-
- return ERR_PTR(ret);
-}
diff --git a/drivers/gpu/drm/msm/edp/edp_connector.c b/drivers/gpu/drm/msm/edp/edp_connector.c
deleted file mode 100644
index 73cb5fd..0000000
--- a/drivers/gpu/drm/msm/edp/edp_connector.c
+++ /dev/null
@@ -1,132 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-only
-/*
- * Copyright (c) 2014-2015, The Linux Foundation. All rights reserved.
- */
-
-#include "drm/drm_edid.h"
-#include "msm_kms.h"
-#include "edp.h"
-
-struct edp_connector {
- struct drm_connector base;
- struct msm_edp *edp;
-};
-#define to_edp_connector(x) container_of(x, struct edp_connector, base)
-
-static enum drm_connector_status edp_connector_detect(
- struct drm_connector *connector, bool force)
-{
- struct edp_connector *edp_connector = to_edp_connector(connector);
- struct msm_edp *edp = edp_connector->edp;
-
- DBG("");
- return msm_edp_ctrl_panel_connected(edp->ctrl) ?
- connector_status_connected : connector_status_disconnected;
-}
-
-static void edp_connector_destroy(struct drm_connector *connector)
-{
- struct edp_connector *edp_connector = to_edp_connector(connector);
-
- DBG("");
-
- drm_connector_cleanup(connector);
-
- kfree(edp_connector);
-}
-
-static int edp_connector_get_modes(struct drm_connector *connector)
-{
- struct edp_connector *edp_connector = to_edp_connector(connector);
- struct msm_edp *edp = edp_connector->edp;
-
- struct edid *drm_edid = NULL;
- int ret = 0;
-
- DBG("");
- ret = msm_edp_ctrl_get_panel_info(edp->ctrl, connector, &drm_edid);
- if (ret)
- return ret;
-
- drm_connector_update_edid_property(connector, drm_edid);
- if (drm_edid)
- ret = drm_add_edid_modes(connector, drm_edid);
-
- return ret;
-}
-
-static int edp_connector_mode_valid(struct drm_connector *connector,
- struct drm_display_mode *mode)
-{
- struct edp_connector *edp_connector = to_edp_connector(connector);
- struct msm_edp *edp = edp_connector->edp;
- struct msm_drm_private *priv = connector->dev->dev_private;
- struct msm_kms *kms = priv->kms;
- long actual, requested;
-
- requested = 1000 * mode->clock;
- actual = kms->funcs->round_pixclk(kms,
- requested, edp_connector->edp->encoder);
-
- DBG("requested=%ld, actual=%ld", requested, actual);
- if (actual != requested)
- return MODE_CLOCK_RANGE;
-
- if (!msm_edp_ctrl_pixel_clock_valid(
- edp->ctrl, mode->clock, NULL, NULL))
- return MODE_CLOCK_RANGE;
-
- /* Invalidate all modes if color format is not supported */
- if (connector->display_info.bpc > 8)
- return MODE_BAD;
-
- return MODE_OK;
-}
-
-static const struct drm_connector_funcs edp_connector_funcs = {
- .detect = edp_connector_detect,
- .fill_modes = drm_helper_probe_single_connector_modes,
- .destroy = edp_connector_destroy,
- .reset = drm_atomic_helper_connector_reset,
- .atomic_duplicate_state = drm_atomic_helper_connector_duplicate_state,
- .atomic_destroy_state = drm_atomic_helper_connector_destroy_state,
-};
-
-static const struct drm_connector_helper_funcs edp_connector_helper_funcs = {
- .get_modes = edp_connector_get_modes,
- .mode_valid = edp_connector_mode_valid,
-};
-
-/* initialize connector */
-struct drm_connector *msm_edp_connector_init(struct msm_edp *edp)
-{
- struct drm_connector *connector = NULL;
- struct edp_connector *edp_connector;
- int ret;
-
- edp_connector = kzalloc(sizeof(*edp_connector), GFP_KERNEL);
- if (!edp_connector)
- return ERR_PTR(-ENOMEM);
-
- edp_connector->edp = edp;
-
- connector = &edp_connector->base;
-
- ret = drm_connector_init(edp->dev, connector, &edp_connector_funcs,
- DRM_MODE_CONNECTOR_eDP);
- if (ret)
- return ERR_PTR(ret);
-
- drm_connector_helper_add(connector, &edp_connector_helper_funcs);
-
- /* We don't support HPD, so only poll status until connected. */
- connector->polled = DRM_CONNECTOR_POLL_CONNECT;
-
- /* Display driver doesn't support interlace now. */
- connector->interlace_allowed = false;
- connector->doublescan_allowed = false;
-
- drm_connector_attach_encoder(connector, edp->encoder);
-
- return connector;
-}
diff --git a/drivers/gpu/drm/msm/edp/edp_ctrl.c b/drivers/gpu/drm/msm/edp/edp_ctrl.c
deleted file mode 100644
index a68a4a1..0000000
--- a/drivers/gpu/drm/msm/edp/edp_ctrl.c
+++ /dev/null
@@ -1,1373 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-only
-/*
- * Copyright (c) 2014-2015, The Linux Foundation. All rights reserved.
- */
-
-#include <linux/clk.h>
-#include <linux/gpio/consumer.h>
-#include <linux/regulator/consumer.h>
-#include <drm/drm_crtc.h>
-#include <drm/drm_dp_helper.h>
-#include <drm/drm_edid.h>
-
-#include "edp.h"
-#include "edp.xml.h"
-
-#define VDDA_UA_ON_LOAD 100000 /* uA units */
-#define VDDA_UA_OFF_LOAD 100 /* uA units */
-
-#define DPCD_LINK_VOLTAGE_MAX 4
-#define DPCD_LINK_PRE_EMPHASIS_MAX 4
-
-#define EDP_LINK_BW_MAX DP_LINK_BW_2_7
-
-/* Link training return value */
-#define EDP_TRAIN_FAIL -1
-#define EDP_TRAIN_SUCCESS 0
-#define EDP_TRAIN_RECONFIG 1
-
-#define EDP_CLK_MASK_AHB BIT(0)
-#define EDP_CLK_MASK_AUX BIT(1)
-#define EDP_CLK_MASK_LINK BIT(2)
-#define EDP_CLK_MASK_PIXEL BIT(3)
-#define EDP_CLK_MASK_MDP_CORE BIT(4)
-#define EDP_CLK_MASK_LINK_CHAN (EDP_CLK_MASK_LINK | EDP_CLK_MASK_PIXEL)
-#define EDP_CLK_MASK_AUX_CHAN \
- (EDP_CLK_MASK_AHB | EDP_CLK_MASK_AUX | EDP_CLK_MASK_MDP_CORE)
-#define EDP_CLK_MASK_ALL (EDP_CLK_MASK_AUX_CHAN | EDP_CLK_MASK_LINK_CHAN)
-
-#define EDP_BACKLIGHT_MAX 255
-
-#define EDP_INTR_STATUS1 \
- (EDP_INTERRUPT_REG_1_HPD | EDP_INTERRUPT_REG_1_AUX_I2C_DONE | \
- EDP_INTERRUPT_REG_1_WRONG_ADDR | EDP_INTERRUPT_REG_1_TIMEOUT | \
- EDP_INTERRUPT_REG_1_NACK_DEFER | EDP_INTERRUPT_REG_1_WRONG_DATA_CNT | \
- EDP_INTERRUPT_REG_1_I2C_NACK | EDP_INTERRUPT_REG_1_I2C_DEFER | \
- EDP_INTERRUPT_REG_1_PLL_UNLOCK | EDP_INTERRUPT_REG_1_AUX_ERROR)
-#define EDP_INTR_MASK1 (EDP_INTR_STATUS1 << 2)
-#define EDP_INTR_STATUS2 \
- (EDP_INTERRUPT_REG_2_READY_FOR_VIDEO | \
- EDP_INTERRUPT_REG_2_IDLE_PATTERNs_SENT | \
- EDP_INTERRUPT_REG_2_FRAME_END | EDP_INTERRUPT_REG_2_CRC_UPDATED)
-#define EDP_INTR_MASK2 (EDP_INTR_STATUS2 << 2)
-
-struct edp_ctrl {
- struct platform_device *pdev;
-
- void __iomem *base;
-
- /* regulators */
- struct regulator *vdda_vreg; /* 1.8 V */
- struct regulator *lvl_vreg;
-
- /* clocks */
- struct clk *aux_clk;
- struct clk *pixel_clk;
- struct clk *ahb_clk;
- struct clk *link_clk;
- struct clk *mdp_core_clk;
-
- /* gpios */
- struct gpio_desc *panel_en_gpio;
- struct gpio_desc *panel_hpd_gpio;
-
- /* completion and mutex */
- struct completion idle_comp;
- struct mutex dev_mutex; /* To protect device power status */
-
- /* work queue */
- struct work_struct on_work;
- struct work_struct off_work;
- struct workqueue_struct *workqueue;
-
- /* Interrupt register lock */
- spinlock_t irq_lock;
-
- bool edp_connected;
- bool power_on;
-
- /* edid raw data */
- struct edid *edid;
-
- struct drm_dp_aux *drm_aux;
-
- /* dpcd raw data */
- u8 dpcd[DP_RECEIVER_CAP_SIZE];
-
- /* Link status */
- u8 link_rate;
- u8 lane_cnt;
- u8 v_level;
- u8 p_level;
-
- /* Timing status */
- u8 interlaced;
- u32 pixel_rate; /* in kHz */
- u32 color_depth;
-
- struct edp_aux *aux;
- struct edp_phy *phy;
-};
-
-struct edp_pixel_clk_div {
- u32 rate; /* in kHz */
- u32 m;
- u32 n;
-};
-
-#define EDP_PIXEL_CLK_NUM 8
-static const struct edp_pixel_clk_div clk_divs[2][EDP_PIXEL_CLK_NUM] = {
- { /* Link clock = 162MHz, source clock = 810MHz */
- {119000, 31, 211}, /* WSXGA+ 1680x1050@60Hz CVT */
- {130250, 32, 199}, /* UXGA 1600x1200@60Hz CVT */
- {148500, 11, 60}, /* FHD 1920x1080@60Hz */
- {154000, 50, 263}, /* WUXGA 1920x1200@60Hz CVT */
- {209250, 31, 120}, /* QXGA 2048x1536@60Hz CVT */
- {268500, 119, 359}, /* WQXGA 2560x1600@60Hz CVT */
- {138530, 33, 193}, /* AUO B116HAN03.0 Panel */
- {141400, 48, 275}, /* AUO B133HTN01.2 Panel */
- },
- { /* Link clock = 270MHz, source clock = 675MHz */
- {119000, 52, 295}, /* WSXGA+ 1680x1050@60Hz CVT */
- {130250, 11, 57}, /* UXGA 1600x1200@60Hz CVT */
- {148500, 11, 50}, /* FHD 1920x1080@60Hz */
- {154000, 47, 206}, /* WUXGA 1920x1200@60Hz CVT */
- {209250, 31, 100}, /* QXGA 2048x1536@60Hz CVT */
- {268500, 107, 269}, /* WQXGA 2560x1600@60Hz CVT */
- {138530, 63, 307}, /* AUO B116HAN03.0 Panel */
- {141400, 53, 253}, /* AUO B133HTN01.2 Panel */
- },
-};
-
-static int edp_clk_init(struct edp_ctrl *ctrl)
-{
- struct platform_device *pdev = ctrl->pdev;
- int ret;
-
- ctrl->aux_clk = msm_clk_get(pdev, "core");
- if (IS_ERR(ctrl->aux_clk)) {
- ret = PTR_ERR(ctrl->aux_clk);
- pr_err("%s: Can't find core clock, %d\n", __func__, ret);
- ctrl->aux_clk = NULL;
- return ret;
- }
-
- ctrl->pixel_clk = msm_clk_get(pdev, "pixel");
- if (IS_ERR(ctrl->pixel_clk)) {
- ret = PTR_ERR(ctrl->pixel_clk);
- pr_err("%s: Can't find pixel clock, %d\n", __func__, ret);
- ctrl->pixel_clk = NULL;
- return ret;
- }
-
- ctrl->ahb_clk = msm_clk_get(pdev, "iface");
- if (IS_ERR(ctrl->ahb_clk)) {
- ret = PTR_ERR(ctrl->ahb_clk);
- pr_err("%s: Can't find iface clock, %d\n", __func__, ret);
- ctrl->ahb_clk = NULL;
- return ret;
- }
-
- ctrl->link_clk = msm_clk_get(pdev, "link");
- if (IS_ERR(ctrl->link_clk)) {
- ret = PTR_ERR(ctrl->link_clk);
- pr_err("%s: Can't find link clock, %d\n", __func__, ret);
- ctrl->link_clk = NULL;
- return ret;
- }
-
- /* need mdp core clock to receive irq */
- ctrl->mdp_core_clk = msm_clk_get(pdev, "mdp_core");
- if (IS_ERR(ctrl->mdp_core_clk)) {
- ret = PTR_ERR(ctrl->mdp_core_clk);
- pr_err("%s: Can't find mdp_core clock, %d\n", __func__, ret);
- ctrl->mdp_core_clk = NULL;
- return ret;
- }
-
- return 0;
-}
-
-static int edp_clk_enable(struct edp_ctrl *ctrl, u32 clk_mask)
-{
- int ret;
-
- DBG("mask=%x", clk_mask);
- /* ahb_clk should be enabled first */
- if (clk_mask & EDP_CLK_MASK_AHB) {
- ret = clk_prepare_enable(ctrl->ahb_clk);
- if (ret) {
- pr_err("%s: Failed to enable ahb clk\n", __func__);
- goto f0;
- }
- }
- if (clk_mask & EDP_CLK_MASK_AUX) {
- ret = clk_set_rate(ctrl->aux_clk, 19200000);
- if (ret) {
- pr_err("%s: Failed to set rate aux clk\n", __func__);
- goto f1;
- }
- ret = clk_prepare_enable(ctrl->aux_clk);
- if (ret) {
- pr_err("%s: Failed to enable aux clk\n", __func__);
- goto f1;
- }
- }
- /* Need to set rate and enable link_clk prior to pixel_clk */
- if (clk_mask & EDP_CLK_MASK_LINK) {
- DBG("edp->link_clk, set_rate %ld",
- (unsigned long)ctrl->link_rate * 27000000);
- ret = clk_set_rate(ctrl->link_clk,
- (unsigned long)ctrl->link_rate * 27000000);
- if (ret) {
- pr_err("%s: Failed to set rate to link clk\n",
- __func__);
- goto f2;
- }
-
- ret = clk_prepare_enable(ctrl->link_clk);
- if (ret) {
- pr_err("%s: Failed to enable link clk\n", __func__);
- goto f2;
- }
- }
- if (clk_mask & EDP_CLK_MASK_PIXEL) {
- DBG("edp->pixel_clk, set_rate %ld",
- (unsigned long)ctrl->pixel_rate * 1000);
- ret = clk_set_rate(ctrl->pixel_clk,
- (unsigned long)ctrl->pixel_rate * 1000);
- if (ret) {
- pr_err("%s: Failed to set rate to pixel clk\n",
- __func__);
- goto f3;
- }
-
- ret = clk_prepare_enable(ctrl->pixel_clk);
- if (ret) {
- pr_err("%s: Failed to enable pixel clk\n", __func__);
- goto f3;
- }
- }
- if (clk_mask & EDP_CLK_MASK_MDP_CORE) {
- ret = clk_prepare_enable(ctrl->mdp_core_clk);
- if (ret) {
- pr_err("%s: Failed to enable mdp core clk\n", __func__);
- goto f4;
- }
- }
-
- return 0;
-
-f4:
- if (clk_mask & EDP_CLK_MASK_PIXEL)
- clk_disable_unprepare(ctrl->pixel_clk);
-f3:
- if (clk_mask & EDP_CLK_MASK_LINK)
- clk_disable_unprepare(ctrl->link_clk);
-f2:
- if (clk_mask & EDP_CLK_MASK_AUX)
- clk_disable_unprepare(ctrl->aux_clk);
-f1:
- if (clk_mask & EDP_CLK_MASK_AHB)
- clk_disable_unprepare(ctrl->ahb_clk);
-f0:
- return ret;
-}
-
-static void edp_clk_disable(struct edp_ctrl *ctrl, u32 clk_mask)
-{
- if (clk_mask & EDP_CLK_MASK_MDP_CORE)
- clk_disable_unprepare(ctrl->mdp_core_clk);
- if (clk_mask & EDP_CLK_MASK_PIXEL)
- clk_disable_unprepare(ctrl->pixel_clk);
- if (clk_mask & EDP_CLK_MASK_LINK)
- clk_disable_unprepare(ctrl->link_clk);
- if (clk_mask & EDP_CLK_MASK_AUX)
- clk_disable_unprepare(ctrl->aux_clk);
- if (clk_mask & EDP_CLK_MASK_AHB)
- clk_disable_unprepare(ctrl->ahb_clk);
-}
-
-static int edp_regulator_init(struct edp_ctrl *ctrl)
-{
- struct device *dev = &ctrl->pdev->dev;
- int ret;
-
- DBG("");
- ctrl->vdda_vreg = devm_regulator_get(dev, "vdda");
- ret = PTR_ERR_OR_ZERO(ctrl->vdda_vreg);
- if (ret) {
- pr_err("%s: Could not get vdda reg, ret = %d\n", __func__,
- ret);
- ctrl->vdda_vreg = NULL;
- return ret;
- }
- ctrl->lvl_vreg = devm_regulator_get(dev, "lvl-vdd");
- ret = PTR_ERR_OR_ZERO(ctrl->lvl_vreg);
- if (ret) {
- pr_err("%s: Could not get lvl-vdd reg, ret = %d\n", __func__,
- ret);
- ctrl->lvl_vreg = NULL;
- return ret;
- }
-
- return 0;
-}
-
-static int edp_regulator_enable(struct edp_ctrl *ctrl)
-{
- int ret;
-
- ret = regulator_set_load(ctrl->vdda_vreg, VDDA_UA_ON_LOAD);
- if (ret < 0) {
- pr_err("%s: vdda_vreg set regulator mode failed.\n", __func__);
- goto vdda_set_fail;
- }
-
- ret = regulator_enable(ctrl->vdda_vreg);
- if (ret) {
- pr_err("%s: Failed to enable vdda_vreg regulator.\n", __func__);
- goto vdda_enable_fail;
- }
-
- ret = regulator_enable(ctrl->lvl_vreg);
- if (ret) {
- pr_err("Failed to enable lvl-vdd reg regulator, %d", ret);
- goto lvl_enable_fail;
- }
-
- DBG("exit");
- return 0;
-
-lvl_enable_fail:
- regulator_disable(ctrl->vdda_vreg);
-vdda_enable_fail:
- regulator_set_load(ctrl->vdda_vreg, VDDA_UA_OFF_LOAD);
-vdda_set_fail:
- return ret;
-}
-
-static void edp_regulator_disable(struct edp_ctrl *ctrl)
-{
- regulator_disable(ctrl->lvl_vreg);
- regulator_disable(ctrl->vdda_vreg);
- regulator_set_load(ctrl->vdda_vreg, VDDA_UA_OFF_LOAD);
-}
-
-static int edp_gpio_config(struct edp_ctrl *ctrl)
-{
- struct device *dev = &ctrl->pdev->dev;
- int ret;
-
- ctrl->panel_hpd_gpio = devm_gpiod_get(dev, "panel-hpd", GPIOD_IN);
- if (IS_ERR(ctrl->panel_hpd_gpio)) {
- ret = PTR_ERR(ctrl->panel_hpd_gpio);
- ctrl->panel_hpd_gpio = NULL;
- pr_err("%s: cannot get panel-hpd-gpios, %d\n", __func__, ret);
- return ret;
- }
-
- ctrl->panel_en_gpio = devm_gpiod_get(dev, "panel-en", GPIOD_OUT_LOW);
- if (IS_ERR(ctrl->panel_en_gpio)) {
- ret = PTR_ERR(ctrl->panel_en_gpio);
- ctrl->panel_en_gpio = NULL;
- pr_err("%s: cannot get panel-en-gpios, %d\n", __func__, ret);
- return ret;
- }
-
- DBG("gpio on");
-
- return 0;
-}
-
-static void edp_ctrl_irq_enable(struct edp_ctrl *ctrl, int enable)
-{
- unsigned long flags;
-
- DBG("%d", enable);
- spin_lock_irqsave(&ctrl->irq_lock, flags);
- if (enable) {
- edp_write(ctrl->base + REG_EDP_INTERRUPT_REG_1, EDP_INTR_MASK1);
- edp_write(ctrl->base + REG_EDP_INTERRUPT_REG_2, EDP_INTR_MASK2);
- } else {
- edp_write(ctrl->base + REG_EDP_INTERRUPT_REG_1, 0x0);
- edp_write(ctrl->base + REG_EDP_INTERRUPT_REG_2, 0x0);
- }
- spin_unlock_irqrestore(&ctrl->irq_lock, flags);
- DBG("exit");
-}
-
-static void edp_fill_link_cfg(struct edp_ctrl *ctrl)
-{
- u32 prate;
- u32 lrate;
- u32 bpp;
- u8 max_lane = drm_dp_max_lane_count(ctrl->dpcd);
- u8 lane;
-
- prate = ctrl->pixel_rate;
- bpp = ctrl->color_depth * 3;
-
- /*
- * By default, use the maximum link rate and minimum lane count,
- * so that we can do rate down shift during link training.
- */
- ctrl->link_rate = ctrl->dpcd[DP_MAX_LINK_RATE];
-
- prate *= bpp;
- prate /= 8; /* in kByte */
-
- lrate = 270000; /* in kHz */
- lrate *= ctrl->link_rate;
- lrate /= 10; /* in kByte, 10 bits --> 8 bits */
-
- for (lane = 1; lane <= max_lane; lane <<= 1) {
- if (lrate >= prate)
- break;
- lrate <<= 1;
- }
-
- ctrl->lane_cnt = lane;
- DBG("rate=%d lane=%d", ctrl->link_rate, ctrl->lane_cnt);
-}
-
-static void edp_config_ctrl(struct edp_ctrl *ctrl)
-{
- u32 data;
- enum edp_color_depth depth;
-
- data = EDP_CONFIGURATION_CTRL_LANES(ctrl->lane_cnt - 1);
-
- if (drm_dp_enhanced_frame_cap(ctrl->dpcd))
- data |= EDP_CONFIGURATION_CTRL_ENHANCED_FRAMING;
-
- depth = EDP_6BIT;
- if (ctrl->color_depth == 8)
- depth = EDP_8BIT;
-
- data |= EDP_CONFIGURATION_CTRL_COLOR(depth);
-
- if (!ctrl->interlaced) /* progressive */
- data |= EDP_CONFIGURATION_CTRL_PROGRESSIVE;
-
- data |= (EDP_CONFIGURATION_CTRL_SYNC_CLK |
- EDP_CONFIGURATION_CTRL_STATIC_MVID);
-
- edp_write(ctrl->base + REG_EDP_CONFIGURATION_CTRL, data);
-}
-
-static void edp_state_ctrl(struct edp_ctrl *ctrl, u32 state)
-{
- edp_write(ctrl->base + REG_EDP_STATE_CTRL, state);
- /* Make sure H/W status is set */
- wmb();
-}
-
-static int edp_lane_set_write(struct edp_ctrl *ctrl,
- u8 voltage_level, u8 pre_emphasis_level)
-{
- int i;
- u8 buf[4];
-
- if (voltage_level >= DPCD_LINK_VOLTAGE_MAX)
- voltage_level |= 0x04;
-
- if (pre_emphasis_level >= DPCD_LINK_PRE_EMPHASIS_MAX)
- pre_emphasis_level |= 0x04;
-
- pre_emphasis_level <<= 3;
-
- for (i = 0; i < 4; i++)
- buf[i] = voltage_level | pre_emphasis_level;
-
- DBG("%s: p|v=0x%x", __func__, voltage_level | pre_emphasis_level);
- if (drm_dp_dpcd_write(ctrl->drm_aux, 0x103, buf, 4) < 4) {
- pr_err("%s: Set sw/pe to panel failed\n", __func__);
- return -ENOLINK;
- }
-
- return 0;
-}
-
-static int edp_train_pattern_set_write(struct edp_ctrl *ctrl, u8 pattern)
-{
- u8 p = pattern;
-
- DBG("pattern=%x", p);
- if (drm_dp_dpcd_write(ctrl->drm_aux,
- DP_TRAINING_PATTERN_SET, &p, 1) < 1) {
- pr_err("%s: Set training pattern to panel failed\n", __func__);
- return -ENOLINK;
- }
-
- return 0;
-}
-
-static void edp_sink_train_set_adjust(struct edp_ctrl *ctrl,
- const u8 *link_status)
-{
- int i;
- u8 max = 0;
- u8 data;
-
- /* use the max level across lanes */
- for (i = 0; i < ctrl->lane_cnt; i++) {
- data = drm_dp_get_adjust_request_voltage(link_status, i);
- DBG("lane=%d req_voltage_swing=0x%x", i, data);
- if (max < data)
- max = data;
- }
-
- ctrl->v_level = max >> DP_TRAIN_VOLTAGE_SWING_SHIFT;
-
- /* use the max level across lanes */
- max = 0;
- for (i = 0; i < ctrl->lane_cnt; i++) {
- data = drm_dp_get_adjust_request_pre_emphasis(link_status, i);
- DBG("lane=%d req_pre_emphasis=0x%x", i, data);
- if (max < data)
- max = data;
- }
-
- ctrl->p_level = max >> DP_TRAIN_PRE_EMPHASIS_SHIFT;
- DBG("v_level=%d, p_level=%d", ctrl->v_level, ctrl->p_level);
-}
-
-static void edp_host_train_set(struct edp_ctrl *ctrl, u32 train)
-{
- int cnt = 10;
- u32 data;
- u32 shift = train - 1;
-
- DBG("train=%d", train);
-
- edp_state_ctrl(ctrl, EDP_STATE_CTRL_TRAIN_PATTERN_1 << shift);
- while (--cnt) {
- data = edp_read(ctrl->base + REG_EDP_MAINLINK_READY);
- if (data & (EDP_MAINLINK_READY_TRAIN_PATTERN_1_READY << shift))
- break;
- }
-
- if (cnt == 0)
- pr_err("%s: set link_train=%d failed\n", __func__, train);
-}
-
-static const u8 vm_pre_emphasis[4][4] = {
- {0x03, 0x06, 0x09, 0x0C}, /* pe0, 0 db */
- {0x03, 0x06, 0x09, 0xFF}, /* pe1, 3.5 db */
- {0x03, 0x06, 0xFF, 0xFF}, /* pe2, 6.0 db */
- {0x03, 0xFF, 0xFF, 0xFF} /* pe3, 9.5 db */
-};
-
-/* voltage swing, 0.2v and 1.0v are not support */
-static const u8 vm_voltage_swing[4][4] = {
- {0x14, 0x18, 0x1A, 0x1E}, /* sw0, 0.4v */
- {0x18, 0x1A, 0x1E, 0xFF}, /* sw1, 0.6 v */
- {0x1A, 0x1E, 0xFF, 0xFF}, /* sw1, 0.8 v */
- {0x1E, 0xFF, 0xFF, 0xFF} /* sw1, 1.2 v, optional */
-};
-
-static int edp_voltage_pre_emphasise_set(struct edp_ctrl *ctrl)
-{
- u32 value0;
- u32 value1;
-
- DBG("v=%d p=%d", ctrl->v_level, ctrl->p_level);
-
- value0 = vm_pre_emphasis[(int)(ctrl->v_level)][(int)(ctrl->p_level)];
- value1 = vm_voltage_swing[(int)(ctrl->v_level)][(int)(ctrl->p_level)];
-
- /* Configure host and panel only if both values are allowed */
- if (value0 != 0xFF && value1 != 0xFF) {
- msm_edp_phy_vm_pe_cfg(ctrl->phy, value0, value1);
- return edp_lane_set_write(ctrl, ctrl->v_level, ctrl->p_level);
- }
-
- return -EINVAL;
-}
-
-static int edp_start_link_train_1(struct edp_ctrl *ctrl)
-{
- u8 link_status[DP_LINK_STATUS_SIZE];
- u8 old_v_level;
- int tries;
- int ret;
- int rlen;
-
- DBG("");
-
- edp_host_train_set(ctrl, DP_TRAINING_PATTERN_1);
- ret = edp_voltage_pre_emphasise_set(ctrl);
- if (ret)
- return ret;
- ret = edp_train_pattern_set_write(ctrl,
- DP_TRAINING_PATTERN_1 | DP_RECOVERED_CLOCK_OUT_EN);
- if (ret)
- return ret;
-
- tries = 0;
- old_v_level = ctrl->v_level;
- while (1) {
- drm_dp_link_train_clock_recovery_delay(ctrl->drm_aux, ctrl->dpcd);
-
- rlen = drm_dp_dpcd_read_link_status(ctrl->drm_aux, link_status);
- if (rlen < DP_LINK_STATUS_SIZE) {
- pr_err("%s: read link status failed\n", __func__);
- return -ENOLINK;
- }
- if (drm_dp_clock_recovery_ok(link_status, ctrl->lane_cnt)) {
- ret = 0;
- break;
- }
-
- if (ctrl->v_level == DPCD_LINK_VOLTAGE_MAX) {
- ret = -1;
- break;
- }
-
- if (old_v_level == ctrl->v_level) {
- tries++;
- if (tries >= 5) {
- ret = -1;
- break;
- }
- } else {
- tries = 0;
- old_v_level = ctrl->v_level;
- }
-
- edp_sink_train_set_adjust(ctrl, link_status);
- ret = edp_voltage_pre_emphasise_set(ctrl);
- if (ret)
- return ret;
- }
-
- return ret;
-}
-
-static int edp_start_link_train_2(struct edp_ctrl *ctrl)
-{
- u8 link_status[DP_LINK_STATUS_SIZE];
- int tries = 0;
- int ret;
- int rlen;
-
- DBG("");
-
- edp_host_train_set(ctrl, DP_TRAINING_PATTERN_2);
- ret = edp_voltage_pre_emphasise_set(ctrl);
- if (ret)
- return ret;
-
- ret = edp_train_pattern_set_write(ctrl,
- DP_TRAINING_PATTERN_2 | DP_RECOVERED_CLOCK_OUT_EN);
- if (ret)
- return ret;
-
- while (1) {
- drm_dp_link_train_channel_eq_delay(ctrl->drm_aux, ctrl->dpcd);
-
- rlen = drm_dp_dpcd_read_link_status(ctrl->drm_aux, link_status);
- if (rlen < DP_LINK_STATUS_SIZE) {
- pr_err("%s: read link status failed\n", __func__);
- return -ENOLINK;
- }
- if (drm_dp_channel_eq_ok(link_status, ctrl->lane_cnt)) {
- ret = 0;
- break;
- }
-
- tries++;
- if (tries > 10) {
- ret = -1;
- break;
- }
-
- edp_sink_train_set_adjust(ctrl, link_status);
- ret = edp_voltage_pre_emphasise_set(ctrl);
- if (ret)
- return ret;
- }
-
- return ret;
-}
-
-static int edp_link_rate_down_shift(struct edp_ctrl *ctrl)
-{
- u32 prate, lrate, bpp;
- u8 rate, lane, max_lane;
- int changed = 0;
-
- rate = ctrl->link_rate;
- lane = ctrl->lane_cnt;
- max_lane = drm_dp_max_lane_count(ctrl->dpcd);
-
- bpp = ctrl->color_depth * 3;
- prate = ctrl->pixel_rate;
- prate *= bpp;
- prate /= 8; /* in kByte */
-
- if (rate > DP_LINK_BW_1_62 && rate <= EDP_LINK_BW_MAX) {
- rate -= 4; /* reduce rate */
- changed++;
- }
-
- if (changed) {
- if (lane >= 1 && lane < max_lane)
- lane <<= 1; /* increase lane */
-
- lrate = 270000; /* in kHz */
- lrate *= rate;
- lrate /= 10; /* kByte, 10 bits --> 8 bits */
- lrate *= lane;
-
- DBG("new lrate=%u prate=%u(kHz) rate=%d lane=%d p=%u b=%d",
- lrate, prate, rate, lane,
- ctrl->pixel_rate,
- bpp);
-
- if (lrate > prate) {
- ctrl->link_rate = rate;
- ctrl->lane_cnt = lane;
- DBG("new rate=%d %d", rate, lane);
- return 0;
- }
- }
-
- return -EINVAL;
-}
-
-static int edp_clear_training_pattern(struct edp_ctrl *ctrl)
-{
- int ret;
-
- ret = edp_train_pattern_set_write(ctrl, 0);
-
- drm_dp_link_train_channel_eq_delay(ctrl->drm_aux, ctrl->dpcd);
-
- return ret;
-}
-
-static int edp_do_link_train(struct edp_ctrl *ctrl)
-{
- u8 values[2];
- int ret;
-
- DBG("");
- /*
- * Set the current link rate and lane cnt to panel. They may have been
- * adjusted and the values are different from them in DPCD CAP
- */
- values[0] = ctrl->lane_cnt;
- values[1] = ctrl->link_rate;
-
- if (drm_dp_enhanced_frame_cap(ctrl->dpcd))
- values[1] |= DP_LANE_COUNT_ENHANCED_FRAME_EN;
-
- if (drm_dp_dpcd_write(ctrl->drm_aux, DP_LINK_BW_SET, values,
- sizeof(values)) < 0)
- return EDP_TRAIN_FAIL;
-
- ctrl->v_level = 0; /* start from default level */
- ctrl->p_level = 0;
-
- edp_state_ctrl(ctrl, 0);
- if (edp_clear_training_pattern(ctrl))
- return EDP_TRAIN_FAIL;
-
- ret = edp_start_link_train_1(ctrl);
- if (ret < 0) {
- if (edp_link_rate_down_shift(ctrl) == 0) {
- DBG("link reconfig");
- ret = EDP_TRAIN_RECONFIG;
- goto clear;
- } else {
- pr_err("%s: Training 1 failed", __func__);
- ret = EDP_TRAIN_FAIL;
- goto clear;
- }
- }
- DBG("Training 1 completed successfully");
-
- edp_state_ctrl(ctrl, 0);
- if (edp_clear_training_pattern(ctrl))
- return EDP_TRAIN_FAIL;
-
- ret = edp_start_link_train_2(ctrl);
- if (ret < 0) {
- if (edp_link_rate_down_shift(ctrl) == 0) {
- DBG("link reconfig");
- ret = EDP_TRAIN_RECONFIG;
- goto clear;
- } else {
- pr_err("%s: Training 2 failed", __func__);
- ret = EDP_TRAIN_FAIL;
- goto clear;
- }
- }
- DBG("Training 2 completed successfully");
-
- edp_state_ctrl(ctrl, EDP_STATE_CTRL_SEND_VIDEO);
-clear:
- edp_clear_training_pattern(ctrl);
-
- return ret;
-}
-
-static void edp_clock_synchrous(struct edp_ctrl *ctrl, int sync)
-{
- u32 data;
- enum edp_color_depth depth;
-
- data = edp_read(ctrl->base + REG_EDP_MISC1_MISC0);
-
- if (sync)
- data |= EDP_MISC1_MISC0_SYNC;
- else
- data &= ~EDP_MISC1_MISC0_SYNC;
-
- /* only legacy rgb mode supported */
- depth = EDP_6BIT; /* Default */
- if (ctrl->color_depth == 8)
- depth = EDP_8BIT;
- else if (ctrl->color_depth == 10)
- depth = EDP_10BIT;
- else if (ctrl->color_depth == 12)
- depth = EDP_12BIT;
- else if (ctrl->color_depth == 16)
- depth = EDP_16BIT;
-
- data |= EDP_MISC1_MISC0_COLOR(depth);
-
- edp_write(ctrl->base + REG_EDP_MISC1_MISC0, data);
-}
-
-static int edp_sw_mvid_nvid(struct edp_ctrl *ctrl, u32 m, u32 n)
-{
- u32 n_multi, m_multi = 5;
-
- if (ctrl->link_rate == DP_LINK_BW_1_62) {
- n_multi = 1;
- } else if (ctrl->link_rate == DP_LINK_BW_2_7) {
- n_multi = 2;
- } else {
- pr_err("%s: Invalid link rate, %d\n", __func__,
- ctrl->link_rate);
- return -EINVAL;
- }
-
- edp_write(ctrl->base + REG_EDP_SOFTWARE_MVID, m * m_multi);
- edp_write(ctrl->base + REG_EDP_SOFTWARE_NVID, n * n_multi);
-
- return 0;
-}
-
-static void edp_mainlink_ctrl(struct edp_ctrl *ctrl, int enable)
-{
- u32 data = 0;
-
- edp_write(ctrl->base + REG_EDP_MAINLINK_CTRL, EDP_MAINLINK_CTRL_RESET);
- /* Make sure fully reset */
- wmb();
- usleep_range(500, 1000);
-
- if (enable)
- data |= EDP_MAINLINK_CTRL_ENABLE;
-
- edp_write(ctrl->base + REG_EDP_MAINLINK_CTRL, data);
-}
-
-static void edp_ctrl_phy_aux_enable(struct edp_ctrl *ctrl, int enable)
-{
- if (enable) {
- edp_regulator_enable(ctrl);
- edp_clk_enable(ctrl, EDP_CLK_MASK_AUX_CHAN);
- msm_edp_phy_ctrl(ctrl->phy, 1);
- msm_edp_aux_ctrl(ctrl->aux, 1);
- gpiod_set_value(ctrl->panel_en_gpio, 1);
- } else {
- gpiod_set_value(ctrl->panel_en_gpio, 0);
- msm_edp_aux_ctrl(ctrl->aux, 0);
- msm_edp_phy_ctrl(ctrl->phy, 0);
- edp_clk_disable(ctrl, EDP_CLK_MASK_AUX_CHAN);
- edp_regulator_disable(ctrl);
- }
-}
-
-static void edp_ctrl_link_enable(struct edp_ctrl *ctrl, int enable)
-{
- u32 m, n;
-
- if (enable) {
- /* Enable link channel clocks */
- edp_clk_enable(ctrl, EDP_CLK_MASK_LINK_CHAN);
-
- msm_edp_phy_lane_power_ctrl(ctrl->phy, true, ctrl->lane_cnt);
-
- msm_edp_phy_vm_pe_init(ctrl->phy);
-
- /* Make sure phy is programed */
- wmb();
- msm_edp_phy_ready(ctrl->phy);
-
- edp_config_ctrl(ctrl);
- msm_edp_ctrl_pixel_clock_valid(ctrl, ctrl->pixel_rate, &m, &n);
- edp_sw_mvid_nvid(ctrl, m, n);
- edp_mainlink_ctrl(ctrl, 1);
- } else {
- edp_mainlink_ctrl(ctrl, 0);
-
- msm_edp_phy_lane_power_ctrl(ctrl->phy, false, 0);
- edp_clk_disable(ctrl, EDP_CLK_MASK_LINK_CHAN);
- }
-}
-
-static int edp_ctrl_training(struct edp_ctrl *ctrl)
-{
- int ret;
-
- /* Do link training only when power is on */
- if (!ctrl->power_on)
- return -EINVAL;
-
-train_start:
- ret = edp_do_link_train(ctrl);
- if (ret == EDP_TRAIN_RECONFIG) {
- /* Re-configure main link */
- edp_ctrl_irq_enable(ctrl, 0);
- edp_ctrl_link_enable(ctrl, 0);
- msm_edp_phy_ctrl(ctrl->phy, 0);
-
- /* Make sure link is fully disabled */
- wmb();
- usleep_range(500, 1000);
-
- msm_edp_phy_ctrl(ctrl->phy, 1);
- edp_ctrl_link_enable(ctrl, 1);
- edp_ctrl_irq_enable(ctrl, 1);
- goto train_start;
- }
-
- return ret;
-}
-
-static void edp_ctrl_on_worker(struct work_struct *work)
-{
- struct edp_ctrl *ctrl = container_of(
- work, struct edp_ctrl, on_work);
- u8 value;
- int ret;
-
- mutex_lock(&ctrl->dev_mutex);
-
- if (ctrl->power_on) {
- DBG("already on");
- goto unlock_ret;
- }
-
- edp_ctrl_phy_aux_enable(ctrl, 1);
- edp_ctrl_link_enable(ctrl, 1);
-
- edp_ctrl_irq_enable(ctrl, 1);
-
- /* DP_SET_POWER register is only available on DPCD v1.1 and later */
- if (ctrl->dpcd[DP_DPCD_REV] >= 0x11) {
- ret = drm_dp_dpcd_readb(ctrl->drm_aux, DP_SET_POWER, &value);
- if (ret < 0)
- goto fail;
-
- value &= ~DP_SET_POWER_MASK;
- value |= DP_SET_POWER_D0;
-
- ret = drm_dp_dpcd_writeb(ctrl->drm_aux, DP_SET_POWER, value);
- if (ret < 0)
- goto fail;
-
- /*
- * According to the DP 1.1 specification, a "Sink Device must
- * exit the power saving state within 1 ms" (Section 2.5.3.1,
- * Table 5-52, "Sink Control Field" (register 0x600).
- */
- usleep_range(1000, 2000);
- }
-
- ctrl->power_on = true;
-
- /* Start link training */
- ret = edp_ctrl_training(ctrl);
- if (ret != EDP_TRAIN_SUCCESS)
- goto fail;
-
- DBG("DONE");
- goto unlock_ret;
-
-fail:
- edp_ctrl_irq_enable(ctrl, 0);
- edp_ctrl_link_enable(ctrl, 0);
- edp_ctrl_phy_aux_enable(ctrl, 0);
- ctrl->power_on = false;
-unlock_ret:
- mutex_unlock(&ctrl->dev_mutex);
-}
-
-static void edp_ctrl_off_worker(struct work_struct *work)
-{
- struct edp_ctrl *ctrl = container_of(
- work, struct edp_ctrl, off_work);
- unsigned long time_left;
-
- mutex_lock(&ctrl->dev_mutex);
-
- if (!ctrl->power_on) {
- DBG("already off");
- goto unlock_ret;
- }
-
- reinit_completion(&ctrl->idle_comp);
- edp_state_ctrl(ctrl, EDP_STATE_CTRL_PUSH_IDLE);
-
- time_left = wait_for_completion_timeout(&ctrl->idle_comp,
- msecs_to_jiffies(500));
- if (!time_left)
- DBG("%s: idle pattern timedout\n", __func__);
-
- edp_state_ctrl(ctrl, 0);
-
- /* DP_SET_POWER register is only available on DPCD v1.1 and later */
- if (ctrl->dpcd[DP_DPCD_REV] >= 0x11) {
- u8 value;
- int ret;
-
- ret = drm_dp_dpcd_readb(ctrl->drm_aux, DP_SET_POWER, &value);
- if (ret > 0) {
- value &= ~DP_SET_POWER_MASK;
- value |= DP_SET_POWER_D3;
-
- drm_dp_dpcd_writeb(ctrl->drm_aux, DP_SET_POWER, value);
- }
- }
-
- edp_ctrl_irq_enable(ctrl, 0);
-
- edp_ctrl_link_enable(ctrl, 0);
-
- edp_ctrl_phy_aux_enable(ctrl, 0);
-
- ctrl->power_on = false;
-
-unlock_ret:
- mutex_unlock(&ctrl->dev_mutex);
-}
-
-irqreturn_t msm_edp_ctrl_irq(struct edp_ctrl *ctrl)
-{
- u32 isr1, isr2, mask1, mask2;
- u32 ack;
-
- DBG("");
- spin_lock(&ctrl->irq_lock);
- isr1 = edp_read(ctrl->base + REG_EDP_INTERRUPT_REG_1);
- isr2 = edp_read(ctrl->base + REG_EDP_INTERRUPT_REG_2);
-
- mask1 = isr1 & EDP_INTR_MASK1;
- mask2 = isr2 & EDP_INTR_MASK2;
-
- isr1 &= ~mask1; /* remove masks bit */
- isr2 &= ~mask2;
-
- DBG("isr=%x mask=%x isr2=%x mask2=%x",
- isr1, mask1, isr2, mask2);
-
- ack = isr1 & EDP_INTR_STATUS1;
- ack <<= 1; /* ack bits */
- ack |= mask1;
- edp_write(ctrl->base + REG_EDP_INTERRUPT_REG_1, ack);
-
- ack = isr2 & EDP_INTR_STATUS2;
- ack <<= 1; /* ack bits */
- ack |= mask2;
- edp_write(ctrl->base + REG_EDP_INTERRUPT_REG_2, ack);
- spin_unlock(&ctrl->irq_lock);
-
- if (isr1 & EDP_INTERRUPT_REG_1_HPD)
- DBG("edp_hpd");
-
- if (isr2 & EDP_INTERRUPT_REG_2_READY_FOR_VIDEO)
- DBG("edp_video_ready");
-
- if (isr2 & EDP_INTERRUPT_REG_2_IDLE_PATTERNs_SENT) {
- DBG("idle_patterns_sent");
- complete(&ctrl->idle_comp);
- }
-
- msm_edp_aux_irq(ctrl->aux, isr1);
-
- return IRQ_HANDLED;
-}
-
-void msm_edp_ctrl_power(struct edp_ctrl *ctrl, bool on)
-{
- if (on)
- queue_work(ctrl->workqueue, &ctrl->on_work);
- else
- queue_work(ctrl->workqueue, &ctrl->off_work);
-}
-
-int msm_edp_ctrl_init(struct msm_edp *edp)
-{
- struct edp_ctrl *ctrl = NULL;
- struct device *dev;
- int ret;
-
- if (!edp) {
- pr_err("%s: edp is NULL!\n", __func__);
- return -EINVAL;
- }
-
- dev = &edp->pdev->dev;
- ctrl = devm_kzalloc(dev, sizeof(*ctrl), GFP_KERNEL);
- if (!ctrl)
- return -ENOMEM;
-
- edp->ctrl = ctrl;
- ctrl->pdev = edp->pdev;
-
- ctrl->base = msm_ioremap(ctrl->pdev, "edp", "eDP");
- if (IS_ERR(ctrl->base))
- return PTR_ERR(ctrl->base);
-
- /* Get regulator, clock, gpio, pwm */
- ret = edp_regulator_init(ctrl);
- if (ret) {
- pr_err("%s:regulator init fail\n", __func__);
- return ret;
- }
- ret = edp_clk_init(ctrl);
- if (ret) {
- pr_err("%s:clk init fail\n", __func__);
- return ret;
- }
- ret = edp_gpio_config(ctrl);
- if (ret) {
- pr_err("%s:failed to configure GPIOs: %d", __func__, ret);
- return ret;
- }
-
- /* Init aux and phy */
- ctrl->aux = msm_edp_aux_init(edp, ctrl->base, &ctrl->drm_aux);
- if (!ctrl->aux || !ctrl->drm_aux) {
- pr_err("%s:failed to init aux\n", __func__);
- return -ENOMEM;
- }
-
- ctrl->phy = msm_edp_phy_init(dev, ctrl->base);
- if (!ctrl->phy) {
- pr_err("%s:failed to init phy\n", __func__);
- ret = -ENOMEM;
- goto err_destory_aux;
- }
-
- spin_lock_init(&ctrl->irq_lock);
- mutex_init(&ctrl->dev_mutex);
- init_completion(&ctrl->idle_comp);
-
- /* setup workqueue */
- ctrl->workqueue = alloc_ordered_workqueue("edp_drm_work", 0);
- INIT_WORK(&ctrl->on_work, edp_ctrl_on_worker);
- INIT_WORK(&ctrl->off_work, edp_ctrl_off_worker);
-
- return 0;
-
-err_destory_aux:
- msm_edp_aux_destroy(dev, ctrl->aux);
- ctrl->aux = NULL;
- return ret;
-}
-
-void msm_edp_ctrl_destroy(struct edp_ctrl *ctrl)
-{
- if (!ctrl)
- return;
-
- if (ctrl->workqueue) {
- destroy_workqueue(ctrl->workqueue);
- ctrl->workqueue = NULL;
- }
-
- if (ctrl->aux) {
- msm_edp_aux_destroy(&ctrl->pdev->dev, ctrl->aux);
- ctrl->aux = NULL;
- }
-
- kfree(ctrl->edid);
- ctrl->edid = NULL;
-
- mutex_destroy(&ctrl->dev_mutex);
-}
-
-bool msm_edp_ctrl_panel_connected(struct edp_ctrl *ctrl)
-{
- mutex_lock(&ctrl->dev_mutex);
- DBG("connect status = %d", ctrl->edp_connected);
- if (ctrl->edp_connected) {
- mutex_unlock(&ctrl->dev_mutex);
- return true;
- }
-
- if (!ctrl->power_on) {
- edp_ctrl_phy_aux_enable(ctrl, 1);
- edp_ctrl_irq_enable(ctrl, 1);
- }
-
- if (drm_dp_dpcd_read(ctrl->drm_aux, DP_DPCD_REV, ctrl->dpcd,
- DP_RECEIVER_CAP_SIZE) < DP_RECEIVER_CAP_SIZE) {
- pr_err("%s: AUX channel is NOT ready\n", __func__);
- memset(ctrl->dpcd, 0, DP_RECEIVER_CAP_SIZE);
- } else {
- ctrl->edp_connected = true;
- }
-
- if (!ctrl->power_on) {
- edp_ctrl_irq_enable(ctrl, 0);
- edp_ctrl_phy_aux_enable(ctrl, 0);
- }
-
- DBG("exit: connect status=%d", ctrl->edp_connected);
-
- mutex_unlock(&ctrl->dev_mutex);
-
- return ctrl->edp_connected;
-}
-
-int msm_edp_ctrl_get_panel_info(struct edp_ctrl *ctrl,
- struct drm_connector *connector, struct edid **edid)
-{
- mutex_lock(&ctrl->dev_mutex);
-
- if (ctrl->edid) {
- if (edid) {
- DBG("Just return edid buffer");
- *edid = ctrl->edid;
- }
- goto unlock_ret;
- }
-
- if (!ctrl->power_on) {
- edp_ctrl_phy_aux_enable(ctrl, 1);
- edp_ctrl_irq_enable(ctrl, 1);
- }
-
- /* Initialize link rate as panel max link rate */
- ctrl->link_rate = ctrl->dpcd[DP_MAX_LINK_RATE];
-
- ctrl->edid = drm_get_edid(connector, &ctrl->drm_aux->ddc);
- if (!ctrl->edid) {
- pr_err("%s: edid read fail\n", __func__);
- goto disable_ret;
- }
-
- if (edid)
- *edid = ctrl->edid;
-
-disable_ret:
- if (!ctrl->power_on) {
- edp_ctrl_irq_enable(ctrl, 0);
- edp_ctrl_phy_aux_enable(ctrl, 0);
- }
-unlock_ret:
- mutex_unlock(&ctrl->dev_mutex);
- return 0;
-}
-
-int msm_edp_ctrl_timing_cfg(struct edp_ctrl *ctrl,
- const struct drm_display_mode *mode,
- const struct drm_display_info *info)
-{
- u32 hstart_from_sync, vstart_from_sync;
- u32 data;
- int ret = 0;
-
- mutex_lock(&ctrl->dev_mutex);
- /*
- * Need to keep color depth, pixel rate and
- * interlaced information in ctrl context
- */
- ctrl->color_depth = info->bpc;
- ctrl->pixel_rate = mode->clock;
- ctrl->interlaced = !!(mode->flags & DRM_MODE_FLAG_INTERLACE);
-
- /* Fill initial link config based on passed in timing */
- edp_fill_link_cfg(ctrl);
-
- if (edp_clk_enable(ctrl, EDP_CLK_MASK_AHB)) {
- pr_err("%s, fail to prepare enable ahb clk\n", __func__);
- ret = -EINVAL;
- goto unlock_ret;
- }
- edp_clock_synchrous(ctrl, 1);
-
- /* Configure eDP timing to HW */
- edp_write(ctrl->base + REG_EDP_TOTAL_HOR_VER,
- EDP_TOTAL_HOR_VER_HORIZ(mode->htotal) |
- EDP_TOTAL_HOR_VER_VERT(mode->vtotal));
-
- vstart_from_sync = mode->vtotal - mode->vsync_start;
- hstart_from_sync = mode->htotal - mode->hsync_start;
- edp_write(ctrl->base + REG_EDP_START_HOR_VER_FROM_SYNC,
- EDP_START_HOR_VER_FROM_SYNC_HORIZ(hstart_from_sync) |
- EDP_START_HOR_VER_FROM_SYNC_VERT(vstart_from_sync));
-
- data = EDP_HSYNC_VSYNC_WIDTH_POLARITY_VERT(
- mode->vsync_end - mode->vsync_start);
- data |= EDP_HSYNC_VSYNC_WIDTH_POLARITY_HORIZ(
- mode->hsync_end - mode->hsync_start);
- if (mode->flags & DRM_MODE_FLAG_NVSYNC)
- data |= EDP_HSYNC_VSYNC_WIDTH_POLARITY_NVSYNC;
- if (mode->flags & DRM_MODE_FLAG_NHSYNC)
- data |= EDP_HSYNC_VSYNC_WIDTH_POLARITY_NHSYNC;
- edp_write(ctrl->base + REG_EDP_HSYNC_VSYNC_WIDTH_POLARITY, data);
-
- edp_write(ctrl->base + REG_EDP_ACTIVE_HOR_VER,
- EDP_ACTIVE_HOR_VER_HORIZ(mode->hdisplay) |
- EDP_ACTIVE_HOR_VER_VERT(mode->vdisplay));
-
- edp_clk_disable(ctrl, EDP_CLK_MASK_AHB);
-
-unlock_ret:
- mutex_unlock(&ctrl->dev_mutex);
- return ret;
-}
-
-bool msm_edp_ctrl_pixel_clock_valid(struct edp_ctrl *ctrl,
- u32 pixel_rate, u32 *pm, u32 *pn)
-{
- const struct edp_pixel_clk_div *divs;
- u32 err = 1; /* 1% error tolerance */
- u32 clk_err;
- int i;
-
- if (ctrl->link_rate == DP_LINK_BW_1_62) {
- divs = clk_divs[0];
- } else if (ctrl->link_rate == DP_LINK_BW_2_7) {
- divs = clk_divs[1];
- } else {
- pr_err("%s: Invalid link rate,%d\n", __func__, ctrl->link_rate);
- return false;
- }
-
- for (i = 0; i < EDP_PIXEL_CLK_NUM; i++) {
- clk_err = abs(divs[i].rate - pixel_rate);
- if ((divs[i].rate * err / 100) >= clk_err) {
- if (pm)
- *pm = divs[i].m;
- if (pn)
- *pn = divs[i].n;
- return true;
- }
- }
-
- DBG("pixel clock %d(kHz) not supported", pixel_rate);
-
- return false;
-}
-
diff --git a/drivers/gpu/drm/msm/edp/edp_phy.c b/drivers/gpu/drm/msm/edp/edp_phy.c
deleted file mode 100644
index fcaf7b7..0000000
--- a/drivers/gpu/drm/msm/edp/edp_phy.c
+++ /dev/null
@@ -1,98 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-only
-/*
- * Copyright (c) 2014-2015, The Linux Foundation. All rights reserved.
- */
-
-#include "edp.h"
-#include "edp.xml.h"
-
-#define EDP_MAX_LANE 4
-
-struct edp_phy {
- void __iomem *base;
-};
-
-bool msm_edp_phy_ready(struct edp_phy *phy)
-{
- u32 status;
- int cnt = 100;
-
- while (--cnt) {
- status = edp_read(phy->base +
- REG_EDP_PHY_GLB_PHY_STATUS);
- if (status & 0x01)
- break;
- usleep_range(500, 1000);
- }
-
- if (cnt == 0) {
- pr_err("%s: PHY NOT ready\n", __func__);
- return false;
- } else {
- return true;
- }
-}
-
-void msm_edp_phy_ctrl(struct edp_phy *phy, int enable)
-{
- DBG("enable=%d", enable);
- if (enable) {
- /* Reset */
- edp_write(phy->base + REG_EDP_PHY_CTRL,
- EDP_PHY_CTRL_SW_RESET | EDP_PHY_CTRL_SW_RESET_PLL);
- /* Make sure fully reset */
- wmb();
- usleep_range(500, 1000);
- edp_write(phy->base + REG_EDP_PHY_CTRL, 0x000);
- edp_write(phy->base + REG_EDP_PHY_GLB_PD_CTL, 0x3f);
- edp_write(phy->base + REG_EDP_PHY_GLB_CFG, 0x1);
- } else {
- edp_write(phy->base + REG_EDP_PHY_GLB_PD_CTL, 0xc0);
- }
-}
-
-/* voltage mode and pre emphasis cfg */
-void msm_edp_phy_vm_pe_init(struct edp_phy *phy)
-{
- edp_write(phy->base + REG_EDP_PHY_GLB_VM_CFG0, 0x3);
- edp_write(phy->base + REG_EDP_PHY_GLB_VM_CFG1, 0x64);
- edp_write(phy->base + REG_EDP_PHY_GLB_MISC9, 0x6c);
-}
-
-void msm_edp_phy_vm_pe_cfg(struct edp_phy *phy, u32 v0, u32 v1)
-{
- edp_write(phy->base + REG_EDP_PHY_GLB_VM_CFG0, v0);
- edp_write(phy->base + REG_EDP_PHY_GLB_VM_CFG1, v1);
-}
-
-void msm_edp_phy_lane_power_ctrl(struct edp_phy *phy, bool up, u32 max_lane)
-{
- u32 i;
- u32 data;
-
- if (up)
- data = 0; /* power up */
- else
- data = 0x7; /* power down */
-
- for (i = 0; i < max_lane; i++)
- edp_write(phy->base + REG_EDP_PHY_LN_PD_CTL(i) , data);
-
- /* power down unused lane */
- data = 0x7; /* power down */
- for (i = max_lane; i < EDP_MAX_LANE; i++)
- edp_write(phy->base + REG_EDP_PHY_LN_PD_CTL(i) , data);
-}
-
-void *msm_edp_phy_init(struct device *dev, void __iomem *regbase)
-{
- struct edp_phy *phy = NULL;
-
- phy = devm_kzalloc(dev, sizeof(*phy), GFP_KERNEL);
- if (!phy)
- return NULL;
-
- phy->base = regbase;
- return phy;
-}
-
diff --git a/drivers/gpu/drm/msm/hdmi/hdmi.c b/drivers/gpu/drm/msm/hdmi/hdmi.c
index 75b64e6..3acdeae 100644
--- a/drivers/gpu/drm/msm/hdmi/hdmi.c
+++ b/drivers/gpu/drm/msm/hdmi/hdmi.c
@@ -8,6 +8,8 @@
#include <linux/of_irq.h>
#include <linux/of_gpio.h>
+#include <drm/drm_bridge_connector.h>
+
#include <sound/hdmi-codec.h>
#include "hdmi.h"
@@ -41,7 +43,7 @@ static irqreturn_t msm_hdmi_irq(int irq, void *dev_id)
struct hdmi *hdmi = dev_id;
/* Process HPD: */
- msm_hdmi_connector_irq(hdmi->connector);
+ msm_hdmi_hpd_irq(hdmi->bridge);
/* Process DDC: */
msm_hdmi_i2c_irq(hdmi->i2c);
@@ -281,7 +283,7 @@ int msm_hdmi_modeset_init(struct hdmi *hdmi,
goto fail;
}
- hdmi->connector = msm_hdmi_connector_init(hdmi);
+ hdmi->connector = drm_bridge_connector_init(hdmi->dev, encoder);
if (IS_ERR(hdmi->connector)) {
ret = PTR_ERR(hdmi->connector);
DRM_DEV_ERROR(dev->dev, "failed to create HDMI connector: %d\n", ret);
@@ -289,6 +291,8 @@ int msm_hdmi_modeset_init(struct hdmi *hdmi,
goto fail;
}
+ drm_connector_attach_encoder(hdmi->connector, hdmi->encoder);
+
hdmi->irq = irq_of_parse_and_map(pdev->dev.of_node, 0);
if (hdmi->irq < 0) {
ret = hdmi->irq;
@@ -305,7 +309,9 @@ int msm_hdmi_modeset_init(struct hdmi *hdmi,
goto fail;
}
- ret = msm_hdmi_hpd_enable(hdmi->connector);
+ drm_bridge_connector_enable_hpd(hdmi->connector);
+
+ ret = msm_hdmi_hpd_enable(hdmi->bridge);
if (ret < 0) {
DRM_DEV_ERROR(&hdmi->pdev->dev, "failed to enable HPD: %d\n", ret);
goto fail;
@@ -514,8 +520,7 @@ static int msm_hdmi_register_audio_driver(struct hdmi *hdmi, struct device *dev)
static int msm_hdmi_bind(struct device *dev, struct device *master, void *data)
{
- struct drm_device *drm = dev_get_drvdata(master);
- struct msm_drm_private *priv = drm->dev_private;
+ struct msm_drm_private *priv = dev_get_drvdata(master);
struct hdmi_platform_config *hdmi_cfg;
struct hdmi *hdmi;
struct device_node *of_node = dev->of_node;
@@ -586,8 +591,8 @@ static int msm_hdmi_bind(struct device *dev, struct device *master, void *data)
static void msm_hdmi_unbind(struct device *dev, struct device *master,
void *data)
{
- struct drm_device *drm = dev_get_drvdata(master);
- struct msm_drm_private *priv = drm->dev_private;
+ struct msm_drm_private *priv = dev_get_drvdata(master);
+
if (priv->hdmi) {
if (priv->hdmi->audio_pdev)
platform_device_unregister(priv->hdmi->audio_pdev);
diff --git a/drivers/gpu/drm/msm/hdmi/hdmi.h b/drivers/gpu/drm/msm/hdmi/hdmi.h
index 8226107..736f348 100644
--- a/drivers/gpu/drm/msm/hdmi/hdmi.h
+++ b/drivers/gpu/drm/msm/hdmi/hdmi.h
@@ -114,6 +114,13 @@ struct hdmi_platform_config {
struct hdmi_gpio_data gpios[HDMI_MAX_NUM_GPIO];
};
+struct hdmi_bridge {
+ struct drm_bridge base;
+ struct hdmi *hdmi;
+ struct work_struct hpd_work;
+};
+#define to_hdmi_bridge(x) container_of(x, struct hdmi_bridge, base)
+
void msm_hdmi_set_mode(struct hdmi *hdmi, bool power_on);
static inline void hdmi_write(struct hdmi *hdmi, u32 reg, u32 data)
@@ -230,13 +237,11 @@ void msm_hdmi_audio_set_sample_rate(struct hdmi *hdmi, int rate);
struct drm_bridge *msm_hdmi_bridge_init(struct hdmi *hdmi);
void msm_hdmi_bridge_destroy(struct drm_bridge *bridge);
-/*
- * hdmi connector:
- */
-
-void msm_hdmi_connector_irq(struct drm_connector *connector);
-struct drm_connector *msm_hdmi_connector_init(struct hdmi *hdmi);
-int msm_hdmi_hpd_enable(struct drm_connector *connector);
+void msm_hdmi_hpd_irq(struct drm_bridge *bridge);
+enum drm_connector_status msm_hdmi_bridge_detect(
+ struct drm_bridge *bridge);
+int msm_hdmi_hpd_enable(struct drm_bridge *bridge);
+void msm_hdmi_hpd_disable(struct hdmi_bridge *hdmi_bridge);
/*
* i2c adapter for ddc:
diff --git a/drivers/gpu/drm/msm/hdmi/hdmi_bridge.c b/drivers/gpu/drm/msm/hdmi/hdmi_bridge.c
index f04eb4a..68fba4b 100644
--- a/drivers/gpu/drm/msm/hdmi/hdmi_bridge.c
+++ b/drivers/gpu/drm/msm/hdmi/hdmi_bridge.c
@@ -5,17 +5,16 @@
*/
#include <linux/delay.h>
+#include <drm/drm_bridge_connector.h>
+#include "msm_kms.h"
#include "hdmi.h"
-struct hdmi_bridge {
- struct drm_bridge base;
- struct hdmi *hdmi;
-};
-#define to_hdmi_bridge(x) container_of(x, struct hdmi_bridge, base)
-
void msm_hdmi_bridge_destroy(struct drm_bridge *bridge)
{
+ struct hdmi_bridge *hdmi_bridge = to_hdmi_bridge(bridge);
+
+ msm_hdmi_hpd_disable(hdmi_bridge);
}
static void msm_hdmi_power_on(struct drm_bridge *bridge)
@@ -70,7 +69,7 @@ static void power_off(struct drm_bridge *bridge)
if (ret)
DRM_DEV_ERROR(dev->dev, "failed to disable pwr regulator: %d\n", ret);
- pm_runtime_put_autosuspend(&hdmi->pdev->dev);
+ pm_runtime_put(&hdmi->pdev->dev);
}
#define AVI_IFRAME_LINE_NUMBER 1
@@ -251,14 +250,76 @@ static void msm_hdmi_bridge_mode_set(struct drm_bridge *bridge,
msm_hdmi_audio_update(hdmi);
}
+static struct edid *msm_hdmi_bridge_get_edid(struct drm_bridge *bridge,
+ struct drm_connector *connector)
+{
+ struct hdmi_bridge *hdmi_bridge = to_hdmi_bridge(bridge);
+ struct hdmi *hdmi = hdmi_bridge->hdmi;
+ struct edid *edid;
+ uint32_t hdmi_ctrl;
+
+ hdmi_ctrl = hdmi_read(hdmi, REG_HDMI_CTRL);
+ hdmi_write(hdmi, REG_HDMI_CTRL, hdmi_ctrl | HDMI_CTRL_ENABLE);
+
+ edid = drm_get_edid(connector, hdmi->i2c);
+
+ hdmi_write(hdmi, REG_HDMI_CTRL, hdmi_ctrl);
+
+ hdmi->hdmi_mode = drm_detect_hdmi_monitor(edid);
+
+ return edid;
+}
+
+static enum drm_mode_status msm_hdmi_bridge_mode_valid(struct drm_bridge *bridge,
+ const struct drm_display_info *info,
+ const struct drm_display_mode *mode)
+{
+ struct hdmi_bridge *hdmi_bridge = to_hdmi_bridge(bridge);
+ struct hdmi *hdmi = hdmi_bridge->hdmi;
+ const struct hdmi_platform_config *config = hdmi->config;
+ struct msm_drm_private *priv = bridge->dev->dev_private;
+ struct msm_kms *kms = priv->kms;
+ long actual, requested;
+
+ requested = 1000 * mode->clock;
+ actual = kms->funcs->round_pixclk(kms,
+ requested, hdmi_bridge->hdmi->encoder);
+
+ /* for mdp5/apq8074, we manage our own pixel clk (as opposed to
+ * mdp4/dtv stuff where pixel clk is assigned to mdp/encoder
+ * instead):
+ */
+ if (config->pwr_clk_cnt > 0)
+ actual = clk_round_rate(hdmi->pwr_clks[0], actual);
+
+ DBG("requested=%ld, actual=%ld", requested, actual);
+
+ if (actual != requested)
+ return MODE_CLOCK_RANGE;
+
+ return 0;
+}
+
static const struct drm_bridge_funcs msm_hdmi_bridge_funcs = {
.pre_enable = msm_hdmi_bridge_pre_enable,
.enable = msm_hdmi_bridge_enable,
.disable = msm_hdmi_bridge_disable,
.post_disable = msm_hdmi_bridge_post_disable,
.mode_set = msm_hdmi_bridge_mode_set,
+ .mode_valid = msm_hdmi_bridge_mode_valid,
+ .get_edid = msm_hdmi_bridge_get_edid,
+ .detect = msm_hdmi_bridge_detect,
};
+static void
+msm_hdmi_hotplug_work(struct work_struct *work)
+{
+ struct hdmi_bridge *hdmi_bridge =
+ container_of(work, struct hdmi_bridge, hpd_work);
+ struct drm_bridge *bridge = &hdmi_bridge->base;
+
+ drm_bridge_hpd_notify(bridge, drm_bridge_detect(bridge));
+}
/* initialize bridge */
struct drm_bridge *msm_hdmi_bridge_init(struct hdmi *hdmi)
@@ -275,11 +336,17 @@ struct drm_bridge *msm_hdmi_bridge_init(struct hdmi *hdmi)
}
hdmi_bridge->hdmi = hdmi;
+ INIT_WORK(&hdmi_bridge->hpd_work, msm_hdmi_hotplug_work);
bridge = &hdmi_bridge->base;
bridge->funcs = &msm_hdmi_bridge_funcs;
+ bridge->ddc = hdmi->i2c;
+ bridge->type = DRM_MODE_CONNECTOR_HDMIA;
+ bridge->ops = DRM_BRIDGE_OP_HPD |
+ DRM_BRIDGE_OP_DETECT |
+ DRM_BRIDGE_OP_EDID;
- ret = drm_bridge_attach(hdmi->encoder, bridge, NULL, 0);
+ ret = drm_bridge_attach(hdmi->encoder, bridge, NULL, DRM_BRIDGE_ATTACH_NO_CONNECTOR);
if (ret)
goto fail;
diff --git a/drivers/gpu/drm/msm/hdmi/hdmi_connector.c b/drivers/gpu/drm/msm/hdmi/hdmi_connector.c
deleted file mode 100644
index a7f729cd..0000000
--- a/drivers/gpu/drm/msm/hdmi/hdmi_connector.c
+++ /dev/null
@@ -1,445 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-only
-/*
- * Copyright (C) 2013 Red Hat
- * Author: Rob Clark <robdclark@gmail.com>
- */
-
-#include <linux/delay.h>
-#include <linux/gpio/consumer.h>
-#include <linux/pinctrl/consumer.h>
-
-#include "msm_kms.h"
-#include "hdmi.h"
-
-struct hdmi_connector {
- struct drm_connector base;
- struct hdmi *hdmi;
- struct work_struct hpd_work;
-};
-#define to_hdmi_connector(x) container_of(x, struct hdmi_connector, base)
-
-static void msm_hdmi_phy_reset(struct hdmi *hdmi)
-{
- unsigned int val;
-
- val = hdmi_read(hdmi, REG_HDMI_PHY_CTRL);
-
- if (val & HDMI_PHY_CTRL_SW_RESET_LOW) {
- /* pull low */
- hdmi_write(hdmi, REG_HDMI_PHY_CTRL,
- val & ~HDMI_PHY_CTRL_SW_RESET);
- } else {
- /* pull high */
- hdmi_write(hdmi, REG_HDMI_PHY_CTRL,
- val | HDMI_PHY_CTRL_SW_RESET);
- }
-
- if (val & HDMI_PHY_CTRL_SW_RESET_PLL_LOW) {
- /* pull low */
- hdmi_write(hdmi, REG_HDMI_PHY_CTRL,
- val & ~HDMI_PHY_CTRL_SW_RESET_PLL);
- } else {
- /* pull high */
- hdmi_write(hdmi, REG_HDMI_PHY_CTRL,
- val | HDMI_PHY_CTRL_SW_RESET_PLL);
- }
-
- msleep(100);
-
- if (val & HDMI_PHY_CTRL_SW_RESET_LOW) {
- /* pull high */
- hdmi_write(hdmi, REG_HDMI_PHY_CTRL,
- val | HDMI_PHY_CTRL_SW_RESET);
- } else {
- /* pull low */
- hdmi_write(hdmi, REG_HDMI_PHY_CTRL,
- val & ~HDMI_PHY_CTRL_SW_RESET);
- }
-
- if (val & HDMI_PHY_CTRL_SW_RESET_PLL_LOW) {
- /* pull high */
- hdmi_write(hdmi, REG_HDMI_PHY_CTRL,
- val | HDMI_PHY_CTRL_SW_RESET_PLL);
- } else {
- /* pull low */
- hdmi_write(hdmi, REG_HDMI_PHY_CTRL,
- val & ~HDMI_PHY_CTRL_SW_RESET_PLL);
- }
-}
-
-static int gpio_config(struct hdmi *hdmi, bool on)
-{
- const struct hdmi_platform_config *config = hdmi->config;
- int i;
-
- if (on) {
- for (i = 0; i < HDMI_MAX_NUM_GPIO; i++) {
- struct hdmi_gpio_data gpio = config->gpios[i];
-
- if (gpio.gpiod) {
- if (gpio.output) {
- gpiod_direction_output(gpio.gpiod,
- gpio.value);
- } else {
- gpiod_direction_input(gpio.gpiod);
- gpiod_set_value_cansleep(gpio.gpiod,
- gpio.value);
- }
- }
- }
-
- DBG("gpio on");
- } else {
- for (i = 0; i < HDMI_MAX_NUM_GPIO; i++) {
- struct hdmi_gpio_data gpio = config->gpios[i];
-
- if (!gpio.gpiod)
- continue;
-
- if (gpio.output) {
- int value = gpio.value ? 0 : 1;
-
- gpiod_set_value_cansleep(gpio.gpiod, value);
- }
- }
-
- DBG("gpio off");
- }
-
- return 0;
-}
-
-static void enable_hpd_clocks(struct hdmi *hdmi, bool enable)
-{
- const struct hdmi_platform_config *config = hdmi->config;
- struct device *dev = &hdmi->pdev->dev;
- int i, ret;
-
- if (enable) {
- for (i = 0; i < config->hpd_clk_cnt; i++) {
- if (config->hpd_freq && config->hpd_freq[i]) {
- ret = clk_set_rate(hdmi->hpd_clks[i],
- config->hpd_freq[i]);
- if (ret)
- dev_warn(dev,
- "failed to set clk %s (%d)\n",
- config->hpd_clk_names[i], ret);
- }
-
- ret = clk_prepare_enable(hdmi->hpd_clks[i]);
- if (ret) {
- DRM_DEV_ERROR(dev,
- "failed to enable hpd clk: %s (%d)\n",
- config->hpd_clk_names[i], ret);
- }
- }
- } else {
- for (i = config->hpd_clk_cnt - 1; i >= 0; i--)
- clk_disable_unprepare(hdmi->hpd_clks[i]);
- }
-}
-
-int msm_hdmi_hpd_enable(struct drm_connector *connector)
-{
- struct hdmi_connector *hdmi_connector = to_hdmi_connector(connector);
- struct hdmi *hdmi = hdmi_connector->hdmi;
- const struct hdmi_platform_config *config = hdmi->config;
- struct device *dev = &hdmi->pdev->dev;
- uint32_t hpd_ctrl;
- int ret;
- unsigned long flags;
-
- ret = regulator_bulk_enable(config->hpd_reg_cnt, hdmi->hpd_regs);
- if (ret) {
- DRM_DEV_ERROR(dev, "failed to enable hpd regulators: %d\n", ret);
- goto fail;
- }
-
- ret = pinctrl_pm_select_default_state(dev);
- if (ret) {
- DRM_DEV_ERROR(dev, "pinctrl state chg failed: %d\n", ret);
- goto fail;
- }
-
- ret = gpio_config(hdmi, true);
- if (ret) {
- DRM_DEV_ERROR(dev, "failed to configure GPIOs: %d\n", ret);
- goto fail;
- }
-
- pm_runtime_get_sync(dev);
- enable_hpd_clocks(hdmi, true);
-
- msm_hdmi_set_mode(hdmi, false);
- msm_hdmi_phy_reset(hdmi);
- msm_hdmi_set_mode(hdmi, true);
-
- hdmi_write(hdmi, REG_HDMI_USEC_REFTIMER, 0x0001001b);
-
- /* enable HPD events: */
- hdmi_write(hdmi, REG_HDMI_HPD_INT_CTRL,
- HDMI_HPD_INT_CTRL_INT_CONNECT |
- HDMI_HPD_INT_CTRL_INT_EN);
-
- /* set timeout to 4.1ms (max) for hardware debounce */
- spin_lock_irqsave(&hdmi->reg_lock, flags);
- hpd_ctrl = hdmi_read(hdmi, REG_HDMI_HPD_CTRL);
- hpd_ctrl |= HDMI_HPD_CTRL_TIMEOUT(0x1fff);
-
- /* Toggle HPD circuit to trigger HPD sense */
- hdmi_write(hdmi, REG_HDMI_HPD_CTRL,
- ~HDMI_HPD_CTRL_ENABLE & hpd_ctrl);
- hdmi_write(hdmi, REG_HDMI_HPD_CTRL,
- HDMI_HPD_CTRL_ENABLE | hpd_ctrl);
- spin_unlock_irqrestore(&hdmi->reg_lock, flags);
-
- return 0;
-
-fail:
- return ret;
-}
-
-static void hdp_disable(struct hdmi_connector *hdmi_connector)
-{
- struct hdmi *hdmi = hdmi_connector->hdmi;
- const struct hdmi_platform_config *config = hdmi->config;
- struct device *dev = &hdmi->pdev->dev;
- int ret;
-
- /* Disable HPD interrupt */
- hdmi_write(hdmi, REG_HDMI_HPD_INT_CTRL, 0);
-
- msm_hdmi_set_mode(hdmi, false);
-
- enable_hpd_clocks(hdmi, false);
- pm_runtime_put_autosuspend(dev);
-
- ret = gpio_config(hdmi, false);
- if (ret)
- dev_warn(dev, "failed to unconfigure GPIOs: %d\n", ret);
-
- ret = pinctrl_pm_select_sleep_state(dev);
- if (ret)
- dev_warn(dev, "pinctrl state chg failed: %d\n", ret);
-
- ret = regulator_bulk_disable(config->hpd_reg_cnt, hdmi->hpd_regs);
- if (ret)
- dev_warn(dev, "failed to disable hpd regulator: %d\n", ret);
-}
-
-static void
-msm_hdmi_hotplug_work(struct work_struct *work)
-{
- struct hdmi_connector *hdmi_connector =
- container_of(work, struct hdmi_connector, hpd_work);
- struct drm_connector *connector = &hdmi_connector->base;
- drm_helper_hpd_irq_event(connector->dev);
-}
-
-void msm_hdmi_connector_irq(struct drm_connector *connector)
-{
- struct hdmi_connector *hdmi_connector = to_hdmi_connector(connector);
- struct hdmi *hdmi = hdmi_connector->hdmi;
- uint32_t hpd_int_status, hpd_int_ctrl;
-
- /* Process HPD: */
- hpd_int_status = hdmi_read(hdmi, REG_HDMI_HPD_INT_STATUS);
- hpd_int_ctrl = hdmi_read(hdmi, REG_HDMI_HPD_INT_CTRL);
-
- if ((hpd_int_ctrl & HDMI_HPD_INT_CTRL_INT_EN) &&
- (hpd_int_status & HDMI_HPD_INT_STATUS_INT)) {
- bool detected = !!(hpd_int_status & HDMI_HPD_INT_STATUS_CABLE_DETECTED);
-
- /* ack & disable (temporarily) HPD events: */
- hdmi_write(hdmi, REG_HDMI_HPD_INT_CTRL,
- HDMI_HPD_INT_CTRL_INT_ACK);
-
- DBG("status=%04x, ctrl=%04x", hpd_int_status, hpd_int_ctrl);
-
- /* detect disconnect if we are connected or visa versa: */
- hpd_int_ctrl = HDMI_HPD_INT_CTRL_INT_EN;
- if (!detected)
- hpd_int_ctrl |= HDMI_HPD_INT_CTRL_INT_CONNECT;
- hdmi_write(hdmi, REG_HDMI_HPD_INT_CTRL, hpd_int_ctrl);
-
- queue_work(hdmi->workq, &hdmi_connector->hpd_work);
- }
-}
-
-static enum drm_connector_status detect_reg(struct hdmi *hdmi)
-{
- uint32_t hpd_int_status;
-
- pm_runtime_get_sync(&hdmi->pdev->dev);
- enable_hpd_clocks(hdmi, true);
-
- hpd_int_status = hdmi_read(hdmi, REG_HDMI_HPD_INT_STATUS);
-
- enable_hpd_clocks(hdmi, false);
- pm_runtime_put_autosuspend(&hdmi->pdev->dev);
-
- return (hpd_int_status & HDMI_HPD_INT_STATUS_CABLE_DETECTED) ?
- connector_status_connected : connector_status_disconnected;
-}
-
-#define HPD_GPIO_INDEX 2
-static enum drm_connector_status detect_gpio(struct hdmi *hdmi)
-{
- const struct hdmi_platform_config *config = hdmi->config;
- struct hdmi_gpio_data hpd_gpio = config->gpios[HPD_GPIO_INDEX];
-
- return gpiod_get_value(hpd_gpio.gpiod) ?
- connector_status_connected :
- connector_status_disconnected;
-}
-
-static enum drm_connector_status hdmi_connector_detect(
- struct drm_connector *connector, bool force)
-{
- struct hdmi_connector *hdmi_connector = to_hdmi_connector(connector);
- struct hdmi *hdmi = hdmi_connector->hdmi;
- const struct hdmi_platform_config *config = hdmi->config;
- struct hdmi_gpio_data hpd_gpio = config->gpios[HPD_GPIO_INDEX];
- enum drm_connector_status stat_gpio, stat_reg;
- int retry = 20;
-
- /*
- * some platforms may not have hpd gpio. Rely only on the status
- * provided by REG_HDMI_HPD_INT_STATUS in this case.
- */
- if (!hpd_gpio.gpiod)
- return detect_reg(hdmi);
-
- do {
- stat_gpio = detect_gpio(hdmi);
- stat_reg = detect_reg(hdmi);
-
- if (stat_gpio == stat_reg)
- break;
-
- mdelay(10);
- } while (--retry);
-
- /* the status we get from reading gpio seems to be more reliable,
- * so trust that one the most if we didn't manage to get hdmi and
- * gpio status to agree:
- */
- if (stat_gpio != stat_reg) {
- DBG("HDMI_HPD_INT_STATUS tells us: %d", stat_reg);
- DBG("hpd gpio tells us: %d", stat_gpio);
- }
-
- return stat_gpio;
-}
-
-static void hdmi_connector_destroy(struct drm_connector *connector)
-{
- struct hdmi_connector *hdmi_connector = to_hdmi_connector(connector);
-
- hdp_disable(hdmi_connector);
-
- drm_connector_cleanup(connector);
-
- kfree(hdmi_connector);
-}
-
-static int msm_hdmi_connector_get_modes(struct drm_connector *connector)
-{
- struct hdmi_connector *hdmi_connector = to_hdmi_connector(connector);
- struct hdmi *hdmi = hdmi_connector->hdmi;
- struct edid *edid;
- uint32_t hdmi_ctrl;
- int ret = 0;
-
- hdmi_ctrl = hdmi_read(hdmi, REG_HDMI_CTRL);
- hdmi_write(hdmi, REG_HDMI_CTRL, hdmi_ctrl | HDMI_CTRL_ENABLE);
-
- edid = drm_get_edid(connector, hdmi->i2c);
-
- hdmi_write(hdmi, REG_HDMI_CTRL, hdmi_ctrl);
-
- hdmi->hdmi_mode = drm_detect_hdmi_monitor(edid);
- drm_connector_update_edid_property(connector, edid);
-
- if (edid) {
- ret = drm_add_edid_modes(connector, edid);
- kfree(edid);
- }
-
- return ret;
-}
-
-static int msm_hdmi_connector_mode_valid(struct drm_connector *connector,
- struct drm_display_mode *mode)
-{
- struct hdmi_connector *hdmi_connector = to_hdmi_connector(connector);
- struct hdmi *hdmi = hdmi_connector->hdmi;
- const struct hdmi_platform_config *config = hdmi->config;
- struct msm_drm_private *priv = connector->dev->dev_private;
- struct msm_kms *kms = priv->kms;
- long actual, requested;
-
- requested = 1000 * mode->clock;
- actual = kms->funcs->round_pixclk(kms,
- requested, hdmi_connector->hdmi->encoder);
-
- /* for mdp5/apq8074, we manage our own pixel clk (as opposed to
- * mdp4/dtv stuff where pixel clk is assigned to mdp/encoder
- * instead):
- */
- if (config->pwr_clk_cnt > 0)
- actual = clk_round_rate(hdmi->pwr_clks[0], actual);
-
- DBG("requested=%ld, actual=%ld", requested, actual);
-
- if (actual != requested)
- return MODE_CLOCK_RANGE;
-
- return 0;
-}
-
-static const struct drm_connector_funcs hdmi_connector_funcs = {
- .detect = hdmi_connector_detect,
- .fill_modes = drm_helper_probe_single_connector_modes,
- .destroy = hdmi_connector_destroy,
- .reset = drm_atomic_helper_connector_reset,
- .atomic_duplicate_state = drm_atomic_helper_connector_duplicate_state,
- .atomic_destroy_state = drm_atomic_helper_connector_destroy_state,
-};
-
-static const struct drm_connector_helper_funcs msm_hdmi_connector_helper_funcs = {
- .get_modes = msm_hdmi_connector_get_modes,
- .mode_valid = msm_hdmi_connector_mode_valid,
-};
-
-/* initialize connector */
-struct drm_connector *msm_hdmi_connector_init(struct hdmi *hdmi)
-{
- struct drm_connector *connector = NULL;
- struct hdmi_connector *hdmi_connector;
-
- hdmi_connector = kzalloc(sizeof(*hdmi_connector), GFP_KERNEL);
- if (!hdmi_connector)
- return ERR_PTR(-ENOMEM);
-
- hdmi_connector->hdmi = hdmi;
- INIT_WORK(&hdmi_connector->hpd_work, msm_hdmi_hotplug_work);
-
- connector = &hdmi_connector->base;
-
- drm_connector_init_with_ddc(hdmi->dev, connector,
- &hdmi_connector_funcs,
- DRM_MODE_CONNECTOR_HDMIA,
- hdmi->i2c);
- drm_connector_helper_add(connector, &msm_hdmi_connector_helper_funcs);
-
- connector->polled = DRM_CONNECTOR_POLL_CONNECT |
- DRM_CONNECTOR_POLL_DISCONNECT;
-
- connector->interlace_allowed = 0;
- connector->doublescan_allowed = 0;
-
- drm_connector_attach_encoder(connector, hdmi->encoder);
-
- return connector;
-}
diff --git a/drivers/gpu/drm/msm/hdmi/hdmi_hpd.c b/drivers/gpu/drm/msm/hdmi/hdmi_hpd.c
new file mode 100644
index 0000000..75605dd
--- /dev/null
+++ b/drivers/gpu/drm/msm/hdmi/hdmi_hpd.c
@@ -0,0 +1,317 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2013 Red Hat
+ * Author: Rob Clark <robdclark@gmail.com>
+ */
+
+#include <linux/delay.h>
+#include <linux/gpio/consumer.h>
+#include <linux/pinctrl/consumer.h>
+
+#include "msm_kms.h"
+#include "hdmi.h"
+
+static void msm_hdmi_phy_reset(struct hdmi *hdmi)
+{
+ unsigned int val;
+
+ val = hdmi_read(hdmi, REG_HDMI_PHY_CTRL);
+
+ if (val & HDMI_PHY_CTRL_SW_RESET_LOW) {
+ /* pull low */
+ hdmi_write(hdmi, REG_HDMI_PHY_CTRL,
+ val & ~HDMI_PHY_CTRL_SW_RESET);
+ } else {
+ /* pull high */
+ hdmi_write(hdmi, REG_HDMI_PHY_CTRL,
+ val | HDMI_PHY_CTRL_SW_RESET);
+ }
+
+ if (val & HDMI_PHY_CTRL_SW_RESET_PLL_LOW) {
+ /* pull low */
+ hdmi_write(hdmi, REG_HDMI_PHY_CTRL,
+ val & ~HDMI_PHY_CTRL_SW_RESET_PLL);
+ } else {
+ /* pull high */
+ hdmi_write(hdmi, REG_HDMI_PHY_CTRL,
+ val | HDMI_PHY_CTRL_SW_RESET_PLL);
+ }
+
+ msleep(100);
+
+ if (val & HDMI_PHY_CTRL_SW_RESET_LOW) {
+ /* pull high */
+ hdmi_write(hdmi, REG_HDMI_PHY_CTRL,
+ val | HDMI_PHY_CTRL_SW_RESET);
+ } else {
+ /* pull low */
+ hdmi_write(hdmi, REG_HDMI_PHY_CTRL,
+ val & ~HDMI_PHY_CTRL_SW_RESET);
+ }
+
+ if (val & HDMI_PHY_CTRL_SW_RESET_PLL_LOW) {
+ /* pull high */
+ hdmi_write(hdmi, REG_HDMI_PHY_CTRL,
+ val | HDMI_PHY_CTRL_SW_RESET_PLL);
+ } else {
+ /* pull low */
+ hdmi_write(hdmi, REG_HDMI_PHY_CTRL,
+ val & ~HDMI_PHY_CTRL_SW_RESET_PLL);
+ }
+}
+
+static int gpio_config(struct hdmi *hdmi, bool on)
+{
+ const struct hdmi_platform_config *config = hdmi->config;
+ int i;
+
+ if (on) {
+ for (i = 0; i < HDMI_MAX_NUM_GPIO; i++) {
+ struct hdmi_gpio_data gpio = config->gpios[i];
+
+ if (gpio.gpiod) {
+ if (gpio.output) {
+ gpiod_direction_output(gpio.gpiod,
+ gpio.value);
+ } else {
+ gpiod_direction_input(gpio.gpiod);
+ gpiod_set_value_cansleep(gpio.gpiod,
+ gpio.value);
+ }
+ }
+ }
+
+ DBG("gpio on");
+ } else {
+ for (i = 0; i < HDMI_MAX_NUM_GPIO; i++) {
+ struct hdmi_gpio_data gpio = config->gpios[i];
+
+ if (!gpio.gpiod)
+ continue;
+
+ if (gpio.output) {
+ int value = gpio.value ? 0 : 1;
+
+ gpiod_set_value_cansleep(gpio.gpiod, value);
+ }
+ }
+
+ DBG("gpio off");
+ }
+
+ return 0;
+}
+
+static void enable_hpd_clocks(struct hdmi *hdmi, bool enable)
+{
+ const struct hdmi_platform_config *config = hdmi->config;
+ struct device *dev = &hdmi->pdev->dev;
+ int i, ret;
+
+ if (enable) {
+ for (i = 0; i < config->hpd_clk_cnt; i++) {
+ if (config->hpd_freq && config->hpd_freq[i]) {
+ ret = clk_set_rate(hdmi->hpd_clks[i],
+ config->hpd_freq[i]);
+ if (ret)
+ dev_warn(dev,
+ "failed to set clk %s (%d)\n",
+ config->hpd_clk_names[i], ret);
+ }
+
+ ret = clk_prepare_enable(hdmi->hpd_clks[i]);
+ if (ret) {
+ DRM_DEV_ERROR(dev,
+ "failed to enable hpd clk: %s (%d)\n",
+ config->hpd_clk_names[i], ret);
+ }
+ }
+ } else {
+ for (i = config->hpd_clk_cnt - 1; i >= 0; i--)
+ clk_disable_unprepare(hdmi->hpd_clks[i]);
+ }
+}
+
+int msm_hdmi_hpd_enable(struct drm_bridge *bridge)
+{
+ struct hdmi_bridge *hdmi_bridge = to_hdmi_bridge(bridge);
+ struct hdmi *hdmi = hdmi_bridge->hdmi;
+ const struct hdmi_platform_config *config = hdmi->config;
+ struct device *dev = &hdmi->pdev->dev;
+ uint32_t hpd_ctrl;
+ int ret;
+ unsigned long flags;
+
+ ret = regulator_bulk_enable(config->hpd_reg_cnt, hdmi->hpd_regs);
+ if (ret) {
+ DRM_DEV_ERROR(dev, "failed to enable hpd regulators: %d\n", ret);
+ goto fail;
+ }
+
+ ret = pinctrl_pm_select_default_state(dev);
+ if (ret) {
+ DRM_DEV_ERROR(dev, "pinctrl state chg failed: %d\n", ret);
+ goto fail;
+ }
+
+ ret = gpio_config(hdmi, true);
+ if (ret) {
+ DRM_DEV_ERROR(dev, "failed to configure GPIOs: %d\n", ret);
+ goto fail;
+ }
+
+ pm_runtime_get_sync(dev);
+ enable_hpd_clocks(hdmi, true);
+
+ msm_hdmi_set_mode(hdmi, false);
+ msm_hdmi_phy_reset(hdmi);
+ msm_hdmi_set_mode(hdmi, true);
+
+ hdmi_write(hdmi, REG_HDMI_USEC_REFTIMER, 0x0001001b);
+
+ /* enable HPD events: */
+ hdmi_write(hdmi, REG_HDMI_HPD_INT_CTRL,
+ HDMI_HPD_INT_CTRL_INT_CONNECT |
+ HDMI_HPD_INT_CTRL_INT_EN);
+
+ /* set timeout to 4.1ms (max) for hardware debounce */
+ spin_lock_irqsave(&hdmi->reg_lock, flags);
+ hpd_ctrl = hdmi_read(hdmi, REG_HDMI_HPD_CTRL);
+ hpd_ctrl |= HDMI_HPD_CTRL_TIMEOUT(0x1fff);
+
+ /* Toggle HPD circuit to trigger HPD sense */
+ hdmi_write(hdmi, REG_HDMI_HPD_CTRL,
+ ~HDMI_HPD_CTRL_ENABLE & hpd_ctrl);
+ hdmi_write(hdmi, REG_HDMI_HPD_CTRL,
+ HDMI_HPD_CTRL_ENABLE | hpd_ctrl);
+ spin_unlock_irqrestore(&hdmi->reg_lock, flags);
+
+ return 0;
+
+fail:
+ return ret;
+}
+
+void msm_hdmi_hpd_disable(struct hdmi_bridge *hdmi_bridge)
+{
+ struct hdmi *hdmi = hdmi_bridge->hdmi;
+ const struct hdmi_platform_config *config = hdmi->config;
+ struct device *dev = &hdmi->pdev->dev;
+ int ret;
+
+ /* Disable HPD interrupt */
+ hdmi_write(hdmi, REG_HDMI_HPD_INT_CTRL, 0);
+
+ msm_hdmi_set_mode(hdmi, false);
+
+ enable_hpd_clocks(hdmi, false);
+ pm_runtime_put(dev);
+
+ ret = gpio_config(hdmi, false);
+ if (ret)
+ dev_warn(dev, "failed to unconfigure GPIOs: %d\n", ret);
+
+ ret = pinctrl_pm_select_sleep_state(dev);
+ if (ret)
+ dev_warn(dev, "pinctrl state chg failed: %d\n", ret);
+
+ ret = regulator_bulk_disable(config->hpd_reg_cnt, hdmi->hpd_regs);
+ if (ret)
+ dev_warn(dev, "failed to disable hpd regulator: %d\n", ret);
+}
+
+void msm_hdmi_hpd_irq(struct drm_bridge *bridge)
+{
+ struct hdmi_bridge *hdmi_bridge = to_hdmi_bridge(bridge);
+ struct hdmi *hdmi = hdmi_bridge->hdmi;
+ uint32_t hpd_int_status, hpd_int_ctrl;
+
+ /* Process HPD: */
+ hpd_int_status = hdmi_read(hdmi, REG_HDMI_HPD_INT_STATUS);
+ hpd_int_ctrl = hdmi_read(hdmi, REG_HDMI_HPD_INT_CTRL);
+
+ if ((hpd_int_ctrl & HDMI_HPD_INT_CTRL_INT_EN) &&
+ (hpd_int_status & HDMI_HPD_INT_STATUS_INT)) {
+ bool detected = !!(hpd_int_status & HDMI_HPD_INT_STATUS_CABLE_DETECTED);
+
+ /* ack & disable (temporarily) HPD events: */
+ hdmi_write(hdmi, REG_HDMI_HPD_INT_CTRL,
+ HDMI_HPD_INT_CTRL_INT_ACK);
+
+ DBG("status=%04x, ctrl=%04x", hpd_int_status, hpd_int_ctrl);
+
+ /* detect disconnect if we are connected or visa versa: */
+ hpd_int_ctrl = HDMI_HPD_INT_CTRL_INT_EN;
+ if (!detected)
+ hpd_int_ctrl |= HDMI_HPD_INT_CTRL_INT_CONNECT;
+ hdmi_write(hdmi, REG_HDMI_HPD_INT_CTRL, hpd_int_ctrl);
+
+ queue_work(hdmi->workq, &hdmi_bridge->hpd_work);
+ }
+}
+
+static enum drm_connector_status detect_reg(struct hdmi *hdmi)
+{
+ uint32_t hpd_int_status;
+
+ pm_runtime_get_sync(&hdmi->pdev->dev);
+ enable_hpd_clocks(hdmi, true);
+
+ hpd_int_status = hdmi_read(hdmi, REG_HDMI_HPD_INT_STATUS);
+
+ enable_hpd_clocks(hdmi, false);
+ pm_runtime_put(&hdmi->pdev->dev);
+
+ return (hpd_int_status & HDMI_HPD_INT_STATUS_CABLE_DETECTED) ?
+ connector_status_connected : connector_status_disconnected;
+}
+
+#define HPD_GPIO_INDEX 2
+static enum drm_connector_status detect_gpio(struct hdmi *hdmi)
+{
+ const struct hdmi_platform_config *config = hdmi->config;
+ struct hdmi_gpio_data hpd_gpio = config->gpios[HPD_GPIO_INDEX];
+
+ return gpiod_get_value(hpd_gpio.gpiod) ?
+ connector_status_connected :
+ connector_status_disconnected;
+}
+
+enum drm_connector_status msm_hdmi_bridge_detect(
+ struct drm_bridge *bridge)
+{
+ struct hdmi_bridge *hdmi_bridge = to_hdmi_bridge(bridge);
+ struct hdmi *hdmi = hdmi_bridge->hdmi;
+ const struct hdmi_platform_config *config = hdmi->config;
+ struct hdmi_gpio_data hpd_gpio = config->gpios[HPD_GPIO_INDEX];
+ enum drm_connector_status stat_gpio, stat_reg;
+ int retry = 20;
+
+ /*
+ * some platforms may not have hpd gpio. Rely only on the status
+ * provided by REG_HDMI_HPD_INT_STATUS in this case.
+ */
+ if (!hpd_gpio.gpiod)
+ return detect_reg(hdmi);
+
+ do {
+ stat_gpio = detect_gpio(hdmi);
+ stat_reg = detect_reg(hdmi);
+
+ if (stat_gpio == stat_reg)
+ break;
+
+ mdelay(10);
+ } while (--retry);
+
+ /* the status we get from reading gpio seems to be more reliable,
+ * so trust that one the most if we didn't manage to get hdmi and
+ * gpio status to agree:
+ */
+ if (stat_gpio != stat_reg) {
+ DBG("HDMI_HPD_INT_STATUS tells us: %d", stat_reg);
+ DBG("hpd gpio tells us: %d", stat_gpio);
+ }
+
+ return stat_gpio;
+}
diff --git a/drivers/gpu/drm/msm/msm_debugfs.c b/drivers/gpu/drm/msm/msm_debugfs.c
index dee13fe..0804c31 100644
--- a/drivers/gpu/drm/msm/msm_debugfs.c
+++ b/drivers/gpu/drm/msm/msm_debugfs.c
@@ -15,6 +15,11 @@
#include "msm_gpu.h"
#include "msm_kms.h"
#include "msm_debugfs.h"
+#include "disp/msm_disp_snapshot.h"
+
+/*
+ * GPU Snapshot:
+ */
struct msm_gpu_show_priv {
struct msm_gpu_state *state;
@@ -29,14 +34,14 @@ static int msm_gpu_show(struct seq_file *m, void *arg)
struct msm_gpu *gpu = priv->gpu;
int ret;
- ret = mutex_lock_interruptible(&show_priv->dev->struct_mutex);
+ ret = mutex_lock_interruptible(&gpu->lock);
if (ret)
return ret;
drm_printf(&p, "%s Status:\n", gpu->name);
gpu->funcs->show(gpu, show_priv->state, &p);
- mutex_unlock(&show_priv->dev->struct_mutex);
+ mutex_unlock(&gpu->lock);
return 0;
}
@@ -48,9 +53,9 @@ static int msm_gpu_release(struct inode *inode, struct file *file)
struct msm_drm_private *priv = show_priv->dev->dev_private;
struct msm_gpu *gpu = priv->gpu;
- mutex_lock(&show_priv->dev->struct_mutex);
+ mutex_lock(&gpu->lock);
gpu->funcs->gpu_state_put(show_priv->state);
- mutex_unlock(&show_priv->dev->struct_mutex);
+ mutex_unlock(&gpu->lock);
kfree(show_priv);
@@ -72,7 +77,7 @@ static int msm_gpu_open(struct inode *inode, struct file *file)
if (!show_priv)
return -ENOMEM;
- ret = mutex_lock_interruptible(&dev->struct_mutex);
+ ret = mutex_lock_interruptible(&gpu->lock);
if (ret)
goto free_priv;
@@ -81,7 +86,7 @@ static int msm_gpu_open(struct inode *inode, struct file *file)
show_priv->state = gpu->funcs->gpu_state_get(gpu);
pm_runtime_put_sync(&gpu->pdev->dev);
- mutex_unlock(&dev->struct_mutex);
+ mutex_unlock(&gpu->lock);
if (IS_ERR(show_priv->state)) {
ret = PTR_ERR(show_priv->state);
@@ -109,6 +114,73 @@ static const struct file_operations msm_gpu_fops = {
.release = msm_gpu_release,
};
+/*
+ * Display Snapshot:
+ */
+
+static int msm_kms_show(struct seq_file *m, void *arg)
+{
+ struct drm_printer p = drm_seq_file_printer(m);
+ struct msm_disp_state *state = m->private;
+
+ msm_disp_state_print(state, &p);
+
+ return 0;
+}
+
+static int msm_kms_release(struct inode *inode, struct file *file)
+{
+ struct seq_file *m = file->private_data;
+ struct msm_disp_state *state = m->private;
+
+ msm_disp_state_free(state);
+
+ return single_release(inode, file);
+}
+
+static int msm_kms_open(struct inode *inode, struct file *file)
+{
+ struct drm_device *dev = inode->i_private;
+ struct msm_drm_private *priv = dev->dev_private;
+ struct msm_disp_state *state;
+ int ret;
+
+ if (!priv->kms)
+ return -ENODEV;
+
+ ret = mutex_lock_interruptible(&priv->kms->dump_mutex);
+ if (ret)
+ return ret;
+
+ state = msm_disp_snapshot_state_sync(priv->kms);
+
+ mutex_unlock(&priv->kms->dump_mutex);
+
+ if (IS_ERR(state)) {
+ return PTR_ERR(state);
+ }
+
+ ret = single_open(file, msm_kms_show, state);
+ if (ret) {
+ msm_disp_state_free(state);
+ return ret;
+ }
+
+ return 0;
+}
+
+static const struct file_operations msm_kms_fops = {
+ .owner = THIS_MODULE,
+ .open = msm_kms_open,
+ .read = seq_read,
+ .llseek = seq_lseek,
+ .release = msm_kms_release,
+};
+
+/*
+ * Other debugfs:
+ */
+
static unsigned long last_shrink_freed;
static int
@@ -134,8 +206,10 @@ DEFINE_SIMPLE_ATTRIBUTE(shrink_fops,
"0x%08llx\n");
-static int msm_gem_show(struct drm_device *dev, struct seq_file *m)
+static int msm_gem_show(struct seq_file *m, void *arg)
{
+ struct drm_info_node *node = (struct drm_info_node *) m->private;
+ struct drm_device *dev = node->minor->dev;
struct msm_drm_private *priv = dev->dev_private;
int ret;
@@ -150,8 +224,10 @@ static int msm_gem_show(struct drm_device *dev, struct seq_file *m)
return 0;
}
-static int msm_mm_show(struct drm_device *dev, struct seq_file *m)
+static int msm_mm_show(struct seq_file *m, void *arg)
{
+ struct drm_info_node *node = (struct drm_info_node *) m->private;
+ struct drm_device *dev = node->minor->dev;
struct drm_printer p = drm_seq_file_printer(m);
drm_mm_print(&dev->vma_offset_manager->vm_addr_space_mm, &p);
@@ -159,8 +235,10 @@ static int msm_mm_show(struct drm_device *dev, struct seq_file *m)
return 0;
}
-static int msm_fb_show(struct drm_device *dev, struct seq_file *m)
+static int msm_fb_show(struct seq_file *m, void *arg)
{
+ struct drm_info_node *node = (struct drm_info_node *) m->private;
+ struct drm_device *dev = node->minor->dev;
struct msm_drm_private *priv = dev->dev_private;
struct drm_framebuffer *fb, *fbdev_fb = NULL;
@@ -183,29 +261,10 @@ static int msm_fb_show(struct drm_device *dev, struct seq_file *m)
return 0;
}
-static int show_locked(struct seq_file *m, void *arg)
-{
- struct drm_info_node *node = (struct drm_info_node *) m->private;
- struct drm_device *dev = node->minor->dev;
- int (*show)(struct drm_device *dev, struct seq_file *m) =
- node->info_ent->data;
- int ret;
-
- ret = mutex_lock_interruptible(&dev->struct_mutex);
- if (ret)
- return ret;
-
- ret = show(dev, m);
-
- mutex_unlock(&dev->struct_mutex);
-
- return ret;
-}
-
static struct drm_info_list msm_debugfs_list[] = {
- {"gem", show_locked, 0, msm_gem_show},
- { "mm", show_locked, 0, msm_mm_show },
- { "fb", show_locked, 0, msm_fb_show },
+ {"gem", msm_gem_show},
+ { "mm", msm_mm_show },
+ { "fb", msm_fb_show },
};
static int late_init_minor(struct drm_minor *minor)
@@ -252,9 +311,15 @@ void msm_debugfs_init(struct drm_minor *minor)
debugfs_create_file("gpu", S_IRUSR, minor->debugfs_root,
dev, &msm_gpu_fops);
+ debugfs_create_file("kms", S_IRUSR, minor->debugfs_root,
+ dev, &msm_kms_fops);
+
debugfs_create_u32("hangcheck_period_ms", 0600, minor->debugfs_root,
&priv->hangcheck_period);
+ debugfs_create_bool("disable_err_irq", 0600, minor->debugfs_root,
+ &priv->disable_err_irq);
+
debugfs_create_file("shrink", S_IRWXU, minor->debugfs_root,
dev, &shrink_fops);
diff --git a/drivers/gpu/drm/msm/msm_drv.c b/drivers/gpu/drm/msm/msm_drv.c
index 892c043..ad35a5d 100644
--- a/drivers/gpu/drm/msm/msm_drv.c
+++ b/drivers/gpu/drm/msm/msm_drv.c
@@ -339,10 +339,9 @@ static int vblank_ctrl_queue_work(struct msm_drm_private *priv,
static int msm_drm_uninit(struct device *dev)
{
struct platform_device *pdev = to_platform_device(dev);
- struct drm_device *ddev = platform_get_drvdata(pdev);
- struct msm_drm_private *priv = ddev->dev_private;
+ struct msm_drm_private *priv = platform_get_drvdata(pdev);
+ struct drm_device *ddev = priv->dev;
struct msm_kms *kms = priv->kms;
- struct msm_mdss *mdss = priv->mdss;
int i;
/*
@@ -402,14 +401,10 @@ static int msm_drm_uninit(struct device *dev)
component_unbind_all(dev, ddev);
- if (mdss && mdss->funcs)
- mdss->funcs->destroy(ddev);
-
ddev->dev_private = NULL;
drm_dev_put(ddev);
destroy_workqueue(priv->wq);
- kfree(priv);
return 0;
}
@@ -512,8 +507,8 @@ static int msm_init_vram(struct drm_device *dev)
static int msm_drm_init(struct device *dev, const struct drm_driver *drv)
{
struct platform_device *pdev = to_platform_device(dev);
+ struct msm_drm_private *priv = dev_get_drvdata(dev);
struct drm_device *ddev;
- struct msm_drm_private *priv;
struct msm_kms *kms;
struct msm_mdss *mdss;
int ret, i;
@@ -523,32 +518,9 @@ static int msm_drm_init(struct device *dev, const struct drm_driver *drv)
DRM_DEV_ERROR(dev, "failed to allocate drm_device\n");
return PTR_ERR(ddev);
}
-
- platform_set_drvdata(pdev, ddev);
-
- priv = kzalloc(sizeof(*priv), GFP_KERNEL);
- if (!priv) {
- ret = -ENOMEM;
- goto err_put_drm_dev;
- }
-
ddev->dev_private = priv;
priv->dev = ddev;
- switch (get_mdp_ver(pdev)) {
- case KMS_MDP5:
- ret = mdp5_mdss_init(ddev);
- break;
- case KMS_DPU:
- ret = dpu_mdss_init(ddev);
- break;
- default:
- ret = 0;
- break;
- }
- if (ret)
- goto err_free_priv;
-
mdss = priv->mdss;
priv->wq = alloc_ordered_workqueue("msm", 0);
@@ -571,12 +543,12 @@ static int msm_drm_init(struct device *dev, const struct drm_driver *drv)
ret = msm_init_vram(ddev);
if (ret)
- goto err_destroy_mdss;
+ return ret;
/* Bind all our sub-components: */
ret = component_bind_all(dev, ddev);
if (ret)
- goto err_destroy_mdss;
+ return ret;
dma_set_max_seg_size(dev, UINT_MAX);
@@ -682,15 +654,6 @@ static int msm_drm_init(struct device *dev, const struct drm_driver *drv)
err_msm_uninit:
msm_drm_uninit(dev);
return ret;
-err_destroy_mdss:
- if (mdss && mdss->funcs)
- mdss->funcs->destroy(ddev);
-err_free_priv:
- kfree(priv);
-err_put_drm_dev:
- drm_dev_put(ddev);
- platform_set_drvdata(pdev, NULL);
- return ret;
}
/*
@@ -752,14 +715,8 @@ static void context_close(struct msm_file_private *ctx)
static void msm_postclose(struct drm_device *dev, struct drm_file *file)
{
- struct msm_drm_private *priv = dev->dev_private;
struct msm_file_private *ctx = file->driver_priv;
- mutex_lock(&dev->struct_mutex);
- if (ctx == priv->lastctx)
- priv->lastctx = NULL;
- mutex_unlock(&dev->struct_mutex);
-
context_close(ctx);
}
@@ -973,7 +930,7 @@ static int wait_fence(struct msm_gpu_submitqueue *queue, uint32_t fence_id,
struct dma_fence *fence;
int ret;
- if (fence_id > queue->last_fence) {
+ if (fence_after(fence_id, queue->last_fence)) {
DRM_ERROR_RATELIMITED("waiting on invalid fence: %u (of %u)\n",
fence_id, queue->last_fence);
return -EINVAL;
@@ -1142,8 +1099,7 @@ static const struct drm_driver msm_driver = {
static int __maybe_unused msm_runtime_suspend(struct device *dev)
{
- struct drm_device *ddev = dev_get_drvdata(dev);
- struct msm_drm_private *priv = ddev->dev_private;
+ struct msm_drm_private *priv = dev_get_drvdata(dev);
struct msm_mdss *mdss = priv->mdss;
DBG("");
@@ -1156,8 +1112,7 @@ static int __maybe_unused msm_runtime_suspend(struct device *dev)
static int __maybe_unused msm_runtime_resume(struct device *dev)
{
- struct drm_device *ddev = dev_get_drvdata(dev);
- struct msm_drm_private *priv = ddev->dev_private;
+ struct msm_drm_private *priv = dev_get_drvdata(dev);
struct msm_mdss *mdss = priv->mdss;
DBG("");
@@ -1187,8 +1142,8 @@ static int __maybe_unused msm_pm_resume(struct device *dev)
static int __maybe_unused msm_pm_prepare(struct device *dev)
{
- struct drm_device *ddev = dev_get_drvdata(dev);
- struct msm_drm_private *priv = ddev ? ddev->dev_private : NULL;
+ struct msm_drm_private *priv = dev_get_drvdata(dev);
+ struct drm_device *ddev = priv ? priv->dev : NULL;
if (!priv || !priv->kms)
return 0;
@@ -1198,8 +1153,8 @@ static int __maybe_unused msm_pm_prepare(struct device *dev)
static void __maybe_unused msm_pm_complete(struct device *dev)
{
- struct drm_device *ddev = dev_get_drvdata(dev);
- struct msm_drm_private *priv = ddev ? ddev->dev_private : NULL;
+ struct msm_drm_private *priv = dev_get_drvdata(dev);
+ struct drm_device *ddev = priv ? priv->dev : NULL;
if (!priv || !priv->kms)
return;
@@ -1292,9 +1247,10 @@ static int add_components_mdp(struct device *mdp_dev,
return 0;
}
-static int compare_name_mdp(struct device *dev, void *data)
+static int find_mdp_node(struct device *dev, void *data)
{
- return (strstr(dev_name(dev), "mdp") != NULL);
+ return of_match_node(dpu_dt_match, dev->of_node) ||
+ of_match_node(mdp5_dt_match, dev->of_node);
}
static int add_display_components(struct platform_device *pdev,
@@ -1319,7 +1275,7 @@ static int add_display_components(struct platform_device *pdev,
return ret;
}
- mdp_dev = device_find_child(dev, NULL, compare_name_mdp);
+ mdp_dev = device_find_child(dev, NULL, find_mdp_node);
if (!mdp_dev) {
DRM_DEV_ERROR(dev, "failed to find MDSS MDP node\n");
of_platform_depopulate(dev);
@@ -1397,12 +1353,35 @@ static const struct component_master_ops msm_drm_ops = {
static int msm_pdev_probe(struct platform_device *pdev)
{
struct component_match *match = NULL;
+ struct msm_drm_private *priv;
int ret;
+ priv = devm_kzalloc(&pdev->dev, sizeof(*priv), GFP_KERNEL);
+ if (!priv)
+ return -ENOMEM;
+
+ platform_set_drvdata(pdev, priv);
+
+ switch (get_mdp_ver(pdev)) {
+ case KMS_MDP5:
+ ret = mdp5_mdss_init(pdev);
+ break;
+ case KMS_DPU:
+ ret = dpu_mdss_init(pdev);
+ break;
+ default:
+ ret = 0;
+ break;
+ }
+ if (ret) {
+ platform_set_drvdata(pdev, NULL);
+ return ret;
+ }
+
if (get_mdp_ver(pdev)) {
ret = add_display_components(pdev, &match);
if (ret)
- return ret;
+ goto fail;
}
ret = add_gpu_components(&pdev->dev, &match);
@@ -1424,21 +1403,31 @@ static int msm_pdev_probe(struct platform_device *pdev)
fail:
of_platform_depopulate(&pdev->dev);
+
+ if (priv->mdss && priv->mdss->funcs)
+ priv->mdss->funcs->destroy(priv->mdss);
+
return ret;
}
static int msm_pdev_remove(struct platform_device *pdev)
{
+ struct msm_drm_private *priv = platform_get_drvdata(pdev);
+ struct msm_mdss *mdss = priv->mdss;
+
component_master_del(&pdev->dev, &msm_drm_ops);
of_platform_depopulate(&pdev->dev);
+ if (mdss && mdss->funcs)
+ mdss->funcs->destroy(mdss);
+
return 0;
}
static void msm_pdev_shutdown(struct platform_device *pdev)
{
- struct drm_device *drm = platform_get_drvdata(pdev);
- struct msm_drm_private *priv = drm ? drm->dev_private : NULL;
+ struct msm_drm_private *priv = platform_get_drvdata(pdev);
+ struct drm_device *drm = priv ? priv->dev : NULL;
if (!priv || !priv->kms)
return;
@@ -1478,7 +1467,6 @@ static int __init msm_drm_register(void)
msm_mdp_register();
msm_dpu_register();
msm_dsi_register();
- msm_edp_register();
msm_hdmi_register();
msm_dp_register();
adreno_register();
@@ -1492,7 +1480,6 @@ static void __exit msm_drm_unregister(void)
msm_dp_unregister();
msm_hdmi_unregister();
adreno_unregister();
- msm_edp_unregister();
msm_dsi_unregister();
msm_mdp_unregister();
msm_dpu_unregister();
diff --git a/drivers/gpu/drm/msm/msm_drv.h b/drivers/gpu/drm/msm/msm_drv.h
index 69952b23..d7574e6 100644
--- a/drivers/gpu/drm/msm/msm_drv.h
+++ b/drivers/gpu/drm/msm/msm_drv.h
@@ -151,12 +151,6 @@ struct msm_drm_private {
*/
struct hdmi *hdmi;
- /* eDP is for mdp5 only, but kms has not been created
- * when edp_bind() and edp_init() are called. Here is the only
- * place to keep the edp instance.
- */
- struct msm_edp *edp;
-
/* DSI is shared by mdp4 and mdp5 */
struct msm_dsi *dsi[2];
@@ -164,7 +158,7 @@ struct msm_drm_private {
/* when we have more than one 'msm_gpu' these need to be an array: */
struct msm_gpu *gpu;
- struct msm_file_private *lastctx;
+
/* gpu is only set on open(), but we need this info earlier */
bool is_a2xx;
bool has_cached_coherent;
@@ -246,6 +240,15 @@ struct msm_drm_private {
/* For hang detection, in ms */
unsigned int hangcheck_period;
+
+ /**
+ * disable_err_irq:
+ *
+ * Disable handling of GPU hw error interrupts, to force fallback to
+ * sw hangcheck timer. Written (via debugfs) by igt tests to test
+ * the sw hangcheck mechanism.
+ */
+ bool disable_err_irq;
};
struct msm_format {
@@ -335,14 +338,10 @@ int msm_hdmi_modeset_init(struct hdmi *hdmi, struct drm_device *dev,
void __init msm_hdmi_register(void);
void __exit msm_hdmi_unregister(void);
-struct msm_edp;
-void __init msm_edp_register(void);
-void __exit msm_edp_unregister(void);
-int msm_edp_modeset_init(struct msm_edp *edp, struct drm_device *dev,
- struct drm_encoder *encoder);
-
struct msm_dsi;
#ifdef CONFIG_DRM_MSM_DSI
+int dsi_dev_attach(struct platform_device *pdev);
+void dsi_dev_detach(struct platform_device *pdev);
void __init msm_dsi_register(void);
void __exit msm_dsi_unregister(void);
int msm_dsi_modeset_init(struct msm_dsi *msm_dsi, struct drm_device *dev,
@@ -390,8 +389,12 @@ int msm_dp_display_enable(struct msm_dp *dp, struct drm_encoder *encoder);
int msm_dp_display_disable(struct msm_dp *dp, struct drm_encoder *encoder);
int msm_dp_display_pre_disable(struct msm_dp *dp, struct drm_encoder *encoder);
void msm_dp_display_mode_set(struct msm_dp *dp, struct drm_encoder *encoder,
- struct drm_display_mode *mode,
- struct drm_display_mode *adjusted_mode);
+ const struct drm_display_mode *mode,
+ const struct drm_display_mode *adjusted_mode);
+
+struct drm_bridge *msm_dp_bridge_init(struct msm_dp *dp_display,
+ struct drm_device *dev,
+ struct drm_encoder *encoder);
void msm_dp_irq_postinstall(struct msm_dp *dp_display);
void msm_dp_snapshot(struct msm_disp_state *disp_state, struct msm_dp *dp_display);
@@ -428,8 +431,8 @@ static inline int msm_dp_display_pre_disable(struct msm_dp *dp,
}
static inline void msm_dp_display_mode_set(struct msm_dp *dp,
struct drm_encoder *encoder,
- struct drm_display_mode *mode,
- struct drm_display_mode *adjusted_mode)
+ const struct drm_display_mode *mode,
+ const struct drm_display_mode *adjusted_mode)
{
}
diff --git a/drivers/gpu/drm/msm/msm_fbdev.c b/drivers/gpu/drm/msm/msm_fbdev.c
index 0daaeb5..4c39ef9 100644
--- a/drivers/gpu/drm/msm/msm_fbdev.c
+++ b/drivers/gpu/drm/msm/msm_fbdev.c
@@ -81,8 +81,6 @@ static int msm_fbdev_create(struct drm_fb_helper *helper,
bo = msm_framebuffer_bo(fb, 0);
- mutex_lock(&dev->struct_mutex);
-
/*
* NOTE: if we can be guaranteed to be able to map buffer
* in panic (ie. lock-safe, etc) we could avoid pinning the
@@ -91,14 +89,14 @@ static int msm_fbdev_create(struct drm_fb_helper *helper,
ret = msm_gem_get_and_pin_iova(bo, priv->kms->aspace, &paddr);
if (ret) {
DRM_DEV_ERROR(dev->dev, "failed to get buffer obj iova: %d\n", ret);
- goto fail_unlock;
+ goto fail;
}
fbi = drm_fb_helper_alloc_fbi(helper);
if (IS_ERR(fbi)) {
DRM_DEV_ERROR(dev->dev, "failed to allocate fb info\n");
ret = PTR_ERR(fbi);
- goto fail_unlock;
+ goto fail;
}
DBG("fbi=%p, dev=%p", fbi, dev);
@@ -115,7 +113,7 @@ static int msm_fbdev_create(struct drm_fb_helper *helper,
fbi->screen_base = msm_gem_get_vaddr(bo);
if (IS_ERR(fbi->screen_base)) {
ret = PTR_ERR(fbi->screen_base);
- goto fail_unlock;
+ goto fail;
}
fbi->screen_size = bo->size;
fbi->fix.smem_start = paddr;
@@ -124,12 +122,9 @@ static int msm_fbdev_create(struct drm_fb_helper *helper,
DBG("par=%p, %dx%d", fbi->par, fbi->var.xres, fbi->var.yres);
DBG("allocated %dx%d fb", fbdev->fb->width, fbdev->fb->height);
- mutex_unlock(&dev->struct_mutex);
-
return 0;
-fail_unlock:
- mutex_unlock(&dev->struct_mutex);
+fail:
drm_framebuffer_remove(fb);
return ret;
}
diff --git a/drivers/gpu/drm/msm/msm_fence.h b/drivers/gpu/drm/msm/msm_fence.h
index 4783db5..17ee382 100644
--- a/drivers/gpu/drm/msm/msm_fence.h
+++ b/drivers/gpu/drm/msm/msm_fence.h
@@ -60,4 +60,16 @@ void msm_update_fence(struct msm_fence_context *fctx, uint32_t fence);
struct dma_fence * msm_fence_alloc(struct msm_fence_context *fctx);
+static inline bool
+fence_before(uint32_t a, uint32_t b)
+{
+ return (int32_t)(a - b) < 0;
+}
+
+static inline bool
+fence_after(uint32_t a, uint32_t b)
+{
+ return (int32_t)(a - b) > 0;
+}
+
#endif
diff --git a/drivers/gpu/drm/msm/msm_gem.c b/drivers/gpu/drm/msm/msm_gem.c
index 512d55e..02b9ae6 100644
--- a/drivers/gpu/drm/msm/msm_gem.c
+++ b/drivers/gpu/drm/msm/msm_gem.c
@@ -866,23 +866,11 @@ int msm_gem_cpu_fini(struct drm_gem_object *obj)
}
#ifdef CONFIG_DEBUG_FS
-static void describe_fence(struct dma_fence *fence, const char *type,
- struct seq_file *m)
-{
- if (!dma_fence_is_signaled(fence))
- seq_printf(m, "\t%9s: %s %s seq %llu\n", type,
- fence->ops->get_driver_name(fence),
- fence->ops->get_timeline_name(fence),
- fence->seqno);
-}
-
void msm_gem_describe(struct drm_gem_object *obj, struct seq_file *m,
struct msm_gem_stats *stats)
{
struct msm_gem_object *msm_obj = to_msm_bo(obj);
struct dma_resv *robj = obj->resv;
- struct dma_resv_list *fobj;
- struct dma_fence *fence;
struct msm_gem_vma *vma;
uint64_t off = drm_vma_node_start(&obj->vma_node);
const char *madv;
@@ -956,22 +944,7 @@ void msm_gem_describe(struct drm_gem_object *obj, struct seq_file *m,
seq_puts(m, "\n");
}
- rcu_read_lock();
- fobj = dma_resv_shared_list(robj);
- if (fobj) {
- unsigned int i, shared_count = fobj->shared_count;
-
- for (i = 0; i < shared_count; i++) {
- fence = rcu_dereference(fobj->shared[i]);
- describe_fence(fence, "Shared", m);
- }
- }
-
- fence = dma_resv_excl_fence(robj);
- if (fence)
- describe_fence(fence, "Exclusive", m);
- rcu_read_unlock();
-
+ dma_resv_describe(robj, m);
msm_gem_unlock(obj);
}
diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
index 282628d..6cfa984 100644
--- a/drivers/gpu/drm/msm/msm_gem_submit.c
+++ b/drivers/gpu/drm/msm/msm_gem_submit.c
@@ -881,7 +881,7 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
* to the underlying fence.
*/
submit->fence_id = idr_alloc_cyclic(&queue->fence_idr,
- submit->user_fence, 0, INT_MAX, GFP_KERNEL);
+ submit->user_fence, 1, INT_MAX, GFP_KERNEL);
if (submit->fence_id < 0) {
ret = submit->fence_id = 0;
submit->fence_id = 0;
diff --git a/drivers/gpu/drm/msm/msm_gpu.c b/drivers/gpu/drm/msm/msm_gpu.c
index 2c46cd9..0f78c26 100644
--- a/drivers/gpu/drm/msm/msm_gpu.c
+++ b/drivers/gpu/drm/msm/msm_gpu.c
@@ -150,7 +150,7 @@ int msm_gpu_hw_init(struct msm_gpu *gpu)
{
int ret;
- WARN_ON(!mutex_is_locked(&gpu->dev->struct_mutex));
+ WARN_ON(!mutex_is_locked(&gpu->lock));
if (!gpu->needs_hw_init)
return 0;
@@ -172,7 +172,7 @@ static void update_fences(struct msm_gpu *gpu, struct msm_ringbuffer *ring,
spin_lock_irqsave(&ring->submit_lock, flags);
list_for_each_entry(submit, &ring->submits, node) {
- if (submit->seqno > fence)
+ if (fence_after(submit->seqno, fence))
break;
msm_update_fence(submit->ring->fctx,
@@ -361,7 +361,7 @@ static void recover_worker(struct kthread_work *work)
char *comm = NULL, *cmd = NULL;
int i;
- mutex_lock(&dev->struct_mutex);
+ mutex_lock(&gpu->lock);
DRM_DEV_ERROR(dev->dev, "%s: hangcheck recover!\n", gpu->name);
@@ -442,7 +442,7 @@ static void recover_worker(struct kthread_work *work)
}
}
- mutex_unlock(&dev->struct_mutex);
+ mutex_unlock(&gpu->lock);
msm_gpu_retire(gpu);
}
@@ -450,12 +450,11 @@ static void recover_worker(struct kthread_work *work)
static void fault_worker(struct kthread_work *work)
{
struct msm_gpu *gpu = container_of(work, struct msm_gpu, fault_work);
- struct drm_device *dev = gpu->dev;
struct msm_gem_submit *submit;
struct msm_ringbuffer *cur_ring = gpu->funcs->active_ring(gpu);
char *comm = NULL, *cmd = NULL;
- mutex_lock(&dev->struct_mutex);
+ mutex_lock(&gpu->lock);
submit = find_submit(cur_ring, cur_ring->memptrs->fence + 1);
if (submit && submit->fault_dumped)
@@ -490,7 +489,7 @@ static void fault_worker(struct kthread_work *work)
memset(&gpu->fault_info, 0, sizeof(gpu->fault_info));
gpu->aspace->mmu->funcs->resume_translation(gpu->aspace->mmu);
- mutex_unlock(&dev->struct_mutex);
+ mutex_unlock(&gpu->lock);
}
static void hangcheck_timer_reset(struct msm_gpu *gpu)
@@ -510,7 +509,7 @@ static void hangcheck_handler(struct timer_list *t)
if (fence != ring->hangcheck_fence) {
/* some progress has been made.. ya! */
ring->hangcheck_fence = fence;
- } else if (fence < ring->seqno) {
+ } else if (fence_before(fence, ring->seqno)) {
/* no progress and not done.. hung! */
ring->hangcheck_fence = fence;
DRM_DEV_ERROR(dev->dev, "%s: hangcheck detected gpu lockup rb %d!\n",
@@ -524,7 +523,7 @@ static void hangcheck_handler(struct timer_list *t)
}
/* if still more pending work, reset the hangcheck timer: */
- if (ring->seqno > ring->hangcheck_fence)
+ if (fence_after(ring->seqno, ring->hangcheck_fence))
hangcheck_timer_reset(gpu);
/* workaround for missing irq: */
@@ -733,7 +732,7 @@ void msm_gpu_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit)
struct msm_ringbuffer *ring = submit->ring;
unsigned long flags;
- WARN_ON(!mutex_is_locked(&dev->struct_mutex));
+ WARN_ON(!mutex_is_locked(&gpu->lock));
pm_runtime_get_sync(&gpu->pdev->dev);
@@ -763,7 +762,7 @@ void msm_gpu_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit)
mutex_unlock(&gpu->active_lock);
gpu->funcs->submit(gpu, submit);
- priv->lastctx = submit->queue->ctx;
+ gpu->cur_ctx_seqno = submit->queue->ctx->seqno;
hangcheck_timer_reset(gpu);
}
@@ -848,6 +847,7 @@ int msm_gpu_init(struct drm_device *drm, struct platform_device *pdev,
INIT_LIST_HEAD(&gpu->active_list);
mutex_init(&gpu->active_lock);
+ mutex_init(&gpu->lock);
kthread_init_work(&gpu->retire_work, retire_worker);
kthread_init_work(&gpu->recover_work, recover_worker);
kthread_init_work(&gpu->fault_work, fault_worker);
diff --git a/drivers/gpu/drm/msm/msm_gpu.h b/drivers/gpu/drm/msm/msm_gpu.h
index 48ea2de..445c6bf 100644
--- a/drivers/gpu/drm/msm/msm_gpu.h
+++ b/drivers/gpu/drm/msm/msm_gpu.h
@@ -88,6 +88,21 @@ struct msm_gpu_devfreq {
struct devfreq *devfreq;
/**
+ * idle_constraint:
+ *
+ * A PM QoS constraint to limit max freq while the GPU is idle.
+ */
+ struct dev_pm_qos_request idle_freq;
+
+ /**
+ * boost_constraint:
+ *
+ * A PM QoS constraint to boost min freq for a period of time
+ * until the boost expires.
+ */
+ struct dev_pm_qos_request boost_freq;
+
+ /**
* busy_cycles:
*
* Used by implementation of gpu->gpu_busy() to track the last
@@ -103,22 +118,19 @@ struct msm_gpu_devfreq {
ktime_t idle_time;
/**
- * idle_freq:
- *
- * Shadow frequency used while the GPU is idle. From the PoV of
- * the devfreq governor, we are continuing to sample busyness and
- * adjust frequency while the GPU is idle, but we use this shadow
- * value as the GPU is actually clamped to minimum frequency while
- * it is inactive.
- */
- unsigned long idle_freq;
-
- /**
* idle_work:
*
* Used to delay clamping to idle freq on active->idle transition.
*/
struct msm_hrtimer_work idle_work;
+
+ /**
+ * boost_work:
+ *
+ * Used to reset the boost_constraint after the boost period has
+ * elapsed
+ */
+ struct msm_hrtimer_work boost_work;
};
struct msm_gpu {
@@ -144,6 +156,17 @@ struct msm_gpu {
struct msm_ringbuffer *rb[MSM_GPU_MAX_RINGS];
int nr_rings;
+ /**
+ * cur_ctx_seqno:
+ *
+ * The ctx->seqno value of the last context to submit rendering,
+ * and the one with current pgtables installed (for generations
+ * that support per-context pgtables). Tracked by seqno rather
+ * than pointer value to avoid dangling pointers, and cases where
+ * a ctx can be freed and a new one created with the same address.
+ */
+ int cur_ctx_seqno;
+
/*
* List of GEM active objects on this gpu. Protected by
* msm_drm_private::mm_lock
@@ -151,12 +174,22 @@ struct msm_gpu {
struct list_head active_list;
/**
+ * lock:
+ *
+ * General lock for serializing all the gpu things.
+ *
+ * TODO move to per-ring locking where feasible (ie. submit/retire
+ * path, etc)
+ */
+ struct mutex lock;
+
+ /**
* active_submits:
*
* The number of submitted but not yet retired submits, used to
* determine transitions between active and idle.
*
- * Protected by lock
+ * Protected by active_lock
*/
int active_submits;
@@ -241,7 +274,7 @@ static inline bool msm_gpu_active(struct msm_gpu *gpu)
for (i = 0; i < gpu->nr_rings; i++) {
struct msm_ringbuffer *ring = gpu->rb[i];
- if (ring->seqno > ring->memptrs->fence)
+ if (fence_after(ring->seqno, ring->memptrs->fence))
return true;
}
@@ -501,6 +534,7 @@ void msm_devfreq_init(struct msm_gpu *gpu);
void msm_devfreq_cleanup(struct msm_gpu *gpu);
void msm_devfreq_resume(struct msm_gpu *gpu);
void msm_devfreq_suspend(struct msm_gpu *gpu);
+void msm_devfreq_boost(struct msm_gpu *gpu, unsigned factor);
void msm_devfreq_active(struct msm_gpu *gpu);
void msm_devfreq_idle(struct msm_gpu *gpu);
@@ -537,28 +571,28 @@ static inline struct msm_gpu_state *msm_gpu_crashstate_get(struct msm_gpu *gpu)
{
struct msm_gpu_state *state = NULL;
- mutex_lock(&gpu->dev->struct_mutex);
+ mutex_lock(&gpu->lock);
if (gpu->crashstate) {
kref_get(&gpu->crashstate->ref);
state = gpu->crashstate;
}
- mutex_unlock(&gpu->dev->struct_mutex);
+ mutex_unlock(&gpu->lock);
return state;
}
static inline void msm_gpu_crashstate_put(struct msm_gpu *gpu)
{
- mutex_lock(&gpu->dev->struct_mutex);
+ mutex_lock(&gpu->lock);
if (gpu->crashstate) {
if (gpu->funcs->gpu_state_put(gpu->crashstate))
gpu->crashstate = NULL;
}
- mutex_unlock(&gpu->dev->struct_mutex);
+ mutex_unlock(&gpu->lock);
}
/*
diff --git a/drivers/gpu/drm/msm/msm_gpu_devfreq.c b/drivers/gpu/drm/msm/msm_gpu_devfreq.c
index 384e90c..62405e9 100644
--- a/drivers/gpu/drm/msm/msm_gpu_devfreq.c
+++ b/drivers/gpu/drm/msm/msm_gpu_devfreq.c
@@ -9,6 +9,7 @@
#include <linux/devfreq.h>
#include <linux/devfreq_cooling.h>
+#include <linux/units.h>
/*
* Power Management:
@@ -25,17 +26,6 @@ static int msm_devfreq_target(struct device *dev, unsigned long *freq,
* to something that actually is in the opp table:
*/
opp = devfreq_recommended_opp(dev, freq, flags);
-
- /*
- * If the GPU is idle, devfreq is not aware, so just ignore
- * it's requests
- */
- if (gpu->devfreq.idle_freq) {
- gpu->devfreq.idle_freq = *freq;
- dev_pm_opp_put(opp);
- return 0;
- }
-
if (IS_ERR(opp))
return PTR_ERR(opp);
@@ -53,9 +43,6 @@ static int msm_devfreq_target(struct device *dev, unsigned long *freq,
static unsigned long get_freq(struct msm_gpu *gpu)
{
- if (gpu->devfreq.idle_freq)
- return gpu->devfreq.idle_freq;
-
if (gpu->funcs->gpu_get_freq)
return gpu->funcs->gpu_get_freq(gpu);
@@ -93,6 +80,7 @@ static struct devfreq_dev_profile msm_devfreq_profile = {
.get_cur_freq = msm_devfreq_get_cur_freq,
};
+static void msm_devfreq_boost_work(struct kthread_work *work);
static void msm_devfreq_idle_work(struct kthread_work *work);
void msm_devfreq_init(struct msm_gpu *gpu)
@@ -103,6 +91,12 @@ void msm_devfreq_init(struct msm_gpu *gpu)
if (!gpu->funcs->gpu_busy)
return;
+ dev_pm_qos_add_request(&gpu->pdev->dev, &df->idle_freq,
+ DEV_PM_QOS_MAX_FREQUENCY,
+ PM_QOS_MAX_FREQUENCY_DEFAULT_VALUE);
+ dev_pm_qos_add_request(&gpu->pdev->dev, &df->boost_freq,
+ DEV_PM_QOS_MIN_FREQUENCY, 0);
+
msm_devfreq_profile.initial_freq = gpu->fast_rate;
/*
@@ -133,13 +127,19 @@ void msm_devfreq_init(struct msm_gpu *gpu)
gpu->cooling = NULL;
}
+ msm_hrtimer_work_init(&df->boost_work, gpu->worker, msm_devfreq_boost_work,
+ CLOCK_MONOTONIC, HRTIMER_MODE_REL);
msm_hrtimer_work_init(&df->idle_work, gpu->worker, msm_devfreq_idle_work,
CLOCK_MONOTONIC, HRTIMER_MODE_REL);
}
void msm_devfreq_cleanup(struct msm_gpu *gpu)
{
+ struct msm_gpu_devfreq *df = &gpu->devfreq;
+
devfreq_cooling_unregister(gpu->cooling);
+ dev_pm_qos_remove_request(&df->boost_freq);
+ dev_pm_qos_remove_request(&df->idle_freq);
}
void msm_devfreq_resume(struct msm_gpu *gpu)
@@ -155,12 +155,40 @@ void msm_devfreq_suspend(struct msm_gpu *gpu)
devfreq_suspend_device(gpu->devfreq.devfreq);
}
+static void msm_devfreq_boost_work(struct kthread_work *work)
+{
+ struct msm_gpu_devfreq *df = container_of(work,
+ struct msm_gpu_devfreq, boost_work.work);
+
+ dev_pm_qos_update_request(&df->boost_freq, 0);
+}
+
+void msm_devfreq_boost(struct msm_gpu *gpu, unsigned factor)
+{
+ struct msm_gpu_devfreq *df = &gpu->devfreq;
+ uint64_t freq;
+
+ freq = get_freq(gpu);
+ freq *= factor;
+
+ /*
+ * A nice little trap is that PM QoS operates in terms of KHz,
+ * while devfreq operates in terms of Hz:
+ */
+ do_div(freq, HZ_PER_KHZ);
+
+ dev_pm_qos_update_request(&df->boost_freq, freq);
+
+ msm_hrtimer_queue_work(&df->boost_work,
+ ms_to_ktime(msm_devfreq_profile.polling_ms),
+ HRTIMER_MODE_REL);
+}
+
void msm_devfreq_active(struct msm_gpu *gpu)
{
struct msm_gpu_devfreq *df = &gpu->devfreq;
struct devfreq_dev_status status;
unsigned int idle_time;
- unsigned long target_freq = df->idle_freq;
if (!df->devfreq)
return;
@@ -170,12 +198,6 @@ void msm_devfreq_active(struct msm_gpu *gpu)
*/
hrtimer_cancel(&df->idle_work.timer);
- /*
- * Hold devfreq lock to synchronize with get_dev_status()/
- * target() callbacks
- */
- mutex_lock(&df->devfreq->lock);
-
idle_time = ktime_to_ms(ktime_sub(ktime_get(), df->idle_time));
/*
@@ -183,21 +205,18 @@ void msm_devfreq_active(struct msm_gpu *gpu)
* interval, then we won't meet the threshold of busyness for
* the governor to ramp up the freq.. so give some boost
*/
- if (idle_time > msm_devfreq_profile.polling_ms/2) {
- target_freq *= 2;
+ if (idle_time > msm_devfreq_profile.polling_ms) {
+ msm_devfreq_boost(gpu, 2);
}
- df->idle_freq = 0;
-
- msm_devfreq_target(&gpu->pdev->dev, &target_freq, 0);
+ dev_pm_qos_update_request(&df->idle_freq,
+ PM_QOS_MAX_FREQUENCY_DEFAULT_VALUE);
/*
* Reset the polling interval so we aren't inconsistent
* about freq vs busy/total cycles
*/
msm_devfreq_get_dev_status(&gpu->pdev->dev, &status);
-
- mutex_unlock(&df->devfreq->lock);
}
@@ -206,23 +225,11 @@ static void msm_devfreq_idle_work(struct kthread_work *work)
struct msm_gpu_devfreq *df = container_of(work,
struct msm_gpu_devfreq, idle_work.work);
struct msm_gpu *gpu = container_of(df, struct msm_gpu, devfreq);
- unsigned long idle_freq, target_freq = 0;
-
- /*
- * Hold devfreq lock to synchronize with get_dev_status()/
- * target() callbacks
- */
- mutex_lock(&df->devfreq->lock);
-
- idle_freq = get_freq(gpu);
-
- if (gpu->clamp_to_idle)
- msm_devfreq_target(&gpu->pdev->dev, &target_freq, 0);
df->idle_time = ktime_get();
- df->idle_freq = idle_freq;
- mutex_unlock(&df->devfreq->lock);
+ if (gpu->clamp_to_idle)
+ dev_pm_qos_update_request(&df->idle_freq, 0);
}
void msm_devfreq_idle(struct msm_gpu *gpu)
diff --git a/drivers/gpu/drm/msm/msm_kms.h b/drivers/gpu/drm/msm/msm_kms.h
index 6a42b81..2a4f052 100644
--- a/drivers/gpu/drm/msm/msm_kms.h
+++ b/drivers/gpu/drm/msm/msm_kms.h
@@ -198,19 +198,22 @@ struct msm_kms *mdp4_kms_init(struct drm_device *dev);
struct msm_kms *mdp5_kms_init(struct drm_device *dev);
struct msm_kms *dpu_kms_init(struct drm_device *dev);
+extern const struct of_device_id dpu_dt_match[];
+extern const struct of_device_id mdp5_dt_match[];
+
struct msm_mdss_funcs {
int (*enable)(struct msm_mdss *mdss);
int (*disable)(struct msm_mdss *mdss);
- void (*destroy)(struct drm_device *dev);
+ void (*destroy)(struct msm_mdss *mdss);
};
struct msm_mdss {
- struct drm_device *dev;
+ struct device *dev;
const struct msm_mdss_funcs *funcs;
};
-int mdp5_mdss_init(struct drm_device *dev);
-int dpu_mdss_init(struct drm_device *dev);
+int mdp5_mdss_init(struct platform_device *dev);
+int dpu_mdss_init(struct platform_device *dev);
#define for_each_crtc_mask(dev, crtc, crtc_mask) \
drm_for_each_crtc(crtc, dev) \
diff --git a/drivers/gpu/drm/msm/msm_perf.c b/drivers/gpu/drm/msm/msm_perf.c
index 3a27153..3d3da79 100644
--- a/drivers/gpu/drm/msm/msm_perf.c
+++ b/drivers/gpu/drm/msm/msm_perf.c
@@ -155,9 +155,12 @@ static int perf_open(struct inode *inode, struct file *file)
struct msm_gpu *gpu = priv->gpu;
int ret = 0;
- mutex_lock(&dev->struct_mutex);
+ if (!gpu)
+ return -ENODEV;
- if (perf->open || !gpu) {
+ mutex_lock(&gpu->lock);
+
+ if (perf->open) {
ret = -EBUSY;
goto out;
}
@@ -171,7 +174,7 @@ static int perf_open(struct inode *inode, struct file *file)
perf->next_jiffies = jiffies + SAMPLE_TIME;
out:
- mutex_unlock(&dev->struct_mutex);
+ mutex_unlock(&gpu->lock);
return ret;
}
diff --git a/drivers/gpu/drm/msm/msm_rd.c b/drivers/gpu/drm/msm/msm_rd.c
index b55398a..81432ec 100644
--- a/drivers/gpu/drm/msm/msm_rd.c
+++ b/drivers/gpu/drm/msm/msm_rd.c
@@ -86,7 +86,7 @@ struct msm_rd_state {
struct msm_gem_submit *submit;
/* fifo access is synchronized on the producer side by
- * struct_mutex held by submit code (otherwise we could
+ * gpu->lock held by submit code (otherwise we could
* end up w/ cmds logged in different order than they
* were executed). And read_lock synchronizes the reads
*/
@@ -181,9 +181,12 @@ static int rd_open(struct inode *inode, struct file *file)
uint32_t gpu_id;
int ret = 0;
- mutex_lock(&dev->struct_mutex);
+ if (!gpu)
+ return -ENODEV;
- if (rd->open || !gpu) {
+ mutex_lock(&gpu->lock);
+
+ if (rd->open) {
ret = -EBUSY;
goto out;
}
@@ -200,7 +203,7 @@ static int rd_open(struct inode *inode, struct file *file)
rd_write_section(rd, RD_GPU_ID, &gpu_id, sizeof(gpu_id));
out:
- mutex_unlock(&dev->struct_mutex);
+ mutex_unlock(&gpu->lock);
return ret;
}
@@ -340,11 +343,10 @@ static void snapshot_buf(struct msm_rd_state *rd,
msm_gem_unlock(&obj->base);
}
-/* called under struct_mutex */
+/* called under gpu->lock */
void msm_rd_dump_submit(struct msm_rd_state *rd, struct msm_gem_submit *submit,
const char *fmt, ...)
{
- struct drm_device *dev = submit->dev;
struct task_struct *task;
char msg[256];
int i, n;
@@ -355,7 +357,7 @@ void msm_rd_dump_submit(struct msm_rd_state *rd, struct msm_gem_submit *submit,
/* writing into fifo is serialized by caller, and
* rd->read_lock is used to serialize the reads
*/
- WARN_ON(!mutex_is_locked(&dev->struct_mutex));
+ WARN_ON(!mutex_is_locked(&submit->gpu->lock));
if (fmt) {
va_list args;
diff --git a/drivers/gpu/drm/msm/msm_ringbuffer.c b/drivers/gpu/drm/msm/msm_ringbuffer.c
index 652b1de..3bbf574 100644
--- a/drivers/gpu/drm/msm/msm_ringbuffer.c
+++ b/drivers/gpu/drm/msm/msm_ringbuffer.c
@@ -21,11 +21,11 @@ static struct dma_fence *msm_job_run(struct drm_sched_job *job)
pm_runtime_get_sync(&gpu->pdev->dev);
/* TODO move submit path over to using a per-ring lock.. */
- mutex_lock(&gpu->dev->struct_mutex);
+ mutex_lock(&gpu->lock);
msm_gpu_submit(gpu, submit);
- mutex_unlock(&gpu->dev->struct_mutex);
+ mutex_unlock(&gpu->lock);
pm_runtime_put(&gpu->pdev->dev);
diff --git a/drivers/gpu/drm/mxsfb/Kconfig b/drivers/gpu/drm/mxsfb/Kconfig
index ee22cd2..987170e 100644
--- a/drivers/gpu/drm/mxsfb/Kconfig
+++ b/drivers/gpu/drm/mxsfb/Kconfig
@@ -10,7 +10,7 @@
depends on COMMON_CLK
select DRM_MXS
select DRM_KMS_HELPER
- select DRM_KMS_CMA_HELPER
+ select DRM_GEM_CMA_HELPER
select DRM_PANEL
select DRM_PANEL_BRIDGE
help
diff --git a/drivers/gpu/drm/nouveau/dispnv04/disp.c b/drivers/gpu/drm/nouveau/dispnv04/disp.c
index 7739f46..99fee4d 100644
--- a/drivers/gpu/drm/nouveau/dispnv04/disp.c
+++ b/drivers/gpu/drm/nouveau/dispnv04/disp.c
@@ -205,7 +205,7 @@ nv04_display_destroy(struct drm_device *dev)
nvif_notify_dtor(&disp->flip);
nouveau_display(dev)->priv = NULL;
- kfree(disp);
+ vfree(disp);
nvif_object_unmap(&drm->client.device.object);
}
@@ -223,7 +223,7 @@ nv04_display_create(struct drm_device *dev)
struct nv04_display *disp;
int i, ret;
- disp = kzalloc(sizeof(*disp), GFP_KERNEL);
+ disp = vzalloc(sizeof(*disp));
if (!disp)
return -ENOMEM;
diff --git a/drivers/gpu/drm/nouveau/dispnv50/Kbuild b/drivers/gpu/drm/nouveau/dispnv50/Kbuild
index 4488e1c..28be291 100644
--- a/drivers/gpu/drm/nouveau/dispnv50/Kbuild
+++ b/drivers/gpu/drm/nouveau/dispnv50/Kbuild
@@ -13,6 +13,7 @@
nouveau-$(CONFIG_DEBUG_FS) += dispnv50/crc.o
nouveau-$(CONFIG_DEBUG_FS) += dispnv50/crc907d.o
nouveau-$(CONFIG_DEBUG_FS) += dispnv50/crcc37d.o
+nouveau-$(CONFIG_DEBUG_FS) += dispnv50/crcc57d.o
nouveau-y += dispnv50/dac507d.o
nouveau-y += dispnv50/dac907d.o
diff --git a/drivers/gpu/drm/nouveau/dispnv50/base907c.c b/drivers/gpu/drm/nouveau/dispnv50/base907c.c
index 5396e37..e6b0417 100644
--- a/drivers/gpu/drm/nouveau/dispnv50/base907c.c
+++ b/drivers/gpu/drm/nouveau/dispnv50/base907c.c
@@ -103,12 +103,9 @@ base907c_xlut_set(struct nv50_wndw *wndw, struct nv50_wndw_atom *asyw)
return 0;
}
-static bool
+static void
base907c_ilut(struct nv50_wndw *wndw, struct nv50_wndw_atom *asyw, int size)
{
- if (size != 256 && size != 1024)
- return false;
-
if (size == 1024)
asyw->xlut.i.mode = NV907C_SET_BASE_LUT_LO_MODE_INTERPOLATE_1025_UNITY_RANGE;
else
@@ -116,7 +113,6 @@ base907c_ilut(struct nv50_wndw *wndw, struct nv50_wndw_atom *asyw, int size)
asyw->xlut.i.enable = NV907C_SET_BASE_LUT_LO_ENABLE_ENABLE;
asyw->xlut.i.load = head907d_olut_load;
- return true;
}
static inline u32
diff --git a/drivers/gpu/drm/nouveau/dispnv50/corec57d.c b/drivers/gpu/drm/nouveau/dispnv50/corec57d.c
index 7587654..53b1e2a 100644
--- a/drivers/gpu/drm/nouveau/dispnv50/corec57d.c
+++ b/drivers/gpu/drm/nouveau/dispnv50/corec57d.c
@@ -69,7 +69,7 @@ corec57d = {
.head = &headc57d,
.sor = &sorc37d,
#if IS_ENABLED(CONFIG_DEBUG_FS)
- .crc = &crcc37d,
+ .crc = &crcc57d,
#endif
};
diff --git a/drivers/gpu/drm/nouveau/dispnv50/crc.c b/drivers/gpu/drm/nouveau/dispnv50/crc.c
index 66f32d9..29428e7 100644
--- a/drivers/gpu/drm/nouveau/dispnv50/crc.c
+++ b/drivers/gpu/drm/nouveau/dispnv50/crc.c
@@ -84,7 +84,10 @@ static void nv50_crc_ctx_flip_work(struct kthread_work *base)
struct nv50_crc *crc = container_of(work, struct nv50_crc, flip_work);
struct nv50_head *head = container_of(crc, struct nv50_head, crc);
struct drm_crtc *crtc = &head->base.base;
- struct nv50_disp *disp = nv50_disp(crtc->dev);
+ struct drm_device *dev = crtc->dev;
+ struct nv50_disp *disp = nv50_disp(dev);
+ const uint64_t start_vbl = drm_crtc_vblank_count(crtc);
+ uint64_t end_vbl;
u8 new_idx = crc->ctx_idx ^ 1;
/*
@@ -92,23 +95,24 @@ static void nv50_crc_ctx_flip_work(struct kthread_work *base)
* try again for the next vblank if we don't grab the lock
*/
if (!mutex_trylock(&disp->mutex)) {
- DRM_DEV_DEBUG_KMS(crtc->dev->dev,
- "Lock contended, delaying CRC ctx flip for head-%d\n",
- head->base.index);
- drm_vblank_work_schedule(work,
- drm_crtc_vblank_count(crtc) + 1,
- true);
+ drm_dbg_kms(dev, "Lock contended, delaying CRC ctx flip for %s\n", crtc->name);
+ drm_vblank_work_schedule(work, start_vbl + 1, true);
return;
}
- DRM_DEV_DEBUG_KMS(crtc->dev->dev,
- "Flipping notifier ctx for head %d (%d -> %d)\n",
- drm_crtc_index(crtc), crc->ctx_idx, new_idx);
+ drm_dbg_kms(dev, "Flipping notifier ctx for %s (%d -> %d)\n",
+ crtc->name, crc->ctx_idx, new_idx);
nv50_crc_program_ctx(head, NULL);
nv50_crc_program_ctx(head, &crc->ctx[new_idx]);
mutex_unlock(&disp->mutex);
+ end_vbl = drm_crtc_vblank_count(crtc);
+ if (unlikely(end_vbl != start_vbl))
+ NV_ERROR(nouveau_drm(dev),
+ "Failed to flip CRC context on %s on time (%llu > %llu)\n",
+ crtc->name, end_vbl, start_vbl);
+
spin_lock_irq(&crc->lock);
crc->ctx_changed = true;
spin_unlock_irq(&crc->lock);
@@ -189,9 +193,9 @@ void nv50_crc_handle_vblank(struct nv50_head *head)
* updates back-to-back without waiting, we'll just be
* optimistic and assume we always miss exactly one frame.
*/
- DRM_DEV_DEBUG_KMS(head->base.base.dev->dev,
- "Notifier ctx flip for head-%d finished, lost CRC for frame %llu\n",
- head->base.index, crc->frame);
+ drm_dbg_kms(head->base.base.dev,
+ "Notifier ctx flip for head-%d finished, lost CRC for frame %llu\n",
+ head->base.index, crc->frame);
crc->frame++;
nv50_crc_reset_ctx(ctx);
@@ -347,8 +351,6 @@ int nv50_crc_atomic_check_head(struct nv50_head *head,
struct nv50_head_atom *armh)
{
struct nv50_atom *atom = nv50_atom(asyh->state.state);
- struct drm_device *dev = head->base.base.dev;
- struct nv50_disp *disp = nv50_disp(dev);
bool changed = armh->crc.src != asyh->crc.src;
if (!armh->crc.src && !asyh->crc.src) {
@@ -357,30 +359,7 @@ int nv50_crc_atomic_check_head(struct nv50_head *head,
return 0;
}
- /* While we don't care about entry tags, Volta+ hw always needs the
- * controlling wndw channel programmed to a wndw that's owned by our
- * head
- */
- if (asyh->crc.src && disp->disp->object.oclass >= GV100_DISP &&
- !(BIT(asyh->crc.wndw) & asyh->wndw.owned)) {
- if (!asyh->wndw.owned) {
- /* TODO: once we support flexible channel ownership,
- * we should write some code here to handle attempting
- * to "steal" a plane: e.g. take a plane that is
- * currently not-visible and owned by another head,
- * and reassign it to this head. If we fail to do so,
- * we shuld reject the mode outright as CRC capture
- * then becomes impossible.
- */
- NV_ATOMIC(nouveau_drm(dev),
- "No available wndws for CRC readback\n");
- return -EINVAL;
- }
- asyh->crc.wndw = ffs(asyh->wndw.owned) - 1;
- }
-
- if (drm_atomic_crtc_needs_modeset(&asyh->state) || changed ||
- armh->crc.wndw != asyh->crc.wndw) {
+ if (drm_atomic_crtc_needs_modeset(&asyh->state) || changed) {
asyh->clr.crc = armh->crc.src && armh->state.active;
asyh->set.crc = asyh->crc.src && asyh->state.active;
if (changed)
@@ -467,9 +446,8 @@ void nv50_crc_atomic_set(struct nv50_head *head,
struct nouveau_encoder *outp =
nv50_real_outp(nv50_head_atom_get_encoder(asyh));
- func->set_src(head, outp->or,
- nv50_crc_source_type(outp, asyh->crc.src),
- &crc->ctx[crc->ctx_idx], asyh->crc.wndw);
+ func->set_src(head, outp->or, nv50_crc_source_type(outp, asyh->crc.src),
+ &crc->ctx[crc->ctx_idx]);
}
void nv50_crc_atomic_clr(struct nv50_head *head)
@@ -477,7 +455,7 @@ void nv50_crc_atomic_clr(struct nv50_head *head)
const struct nv50_crc_func *func =
nv50_disp(head->base.base.dev)->core->func->crc;
- func->set_src(head, 0, NV50_CRC_SOURCE_TYPE_NONE, NULL, 0);
+ func->set_src(head, 0, NV50_CRC_SOURCE_TYPE_NONE, NULL);
}
static inline int
diff --git a/drivers/gpu/drm/nouveau/dispnv50/crc.h b/drivers/gpu/drm/nouveau/dispnv50/crc.h
index 4fce871..4823f1f 100644
--- a/drivers/gpu/drm/nouveau/dispnv50/crc.h
+++ b/drivers/gpu/drm/nouveau/dispnv50/crc.h
@@ -45,13 +45,11 @@ struct nv50_crc_notifier_ctx {
struct nv50_crc_atom {
enum nv50_crc_source src;
- /* Only used for gv100+ */
- u8 wndw : 4;
};
struct nv50_crc_func {
- int (*set_src)(struct nv50_head *, int or, enum nv50_crc_source_type,
- struct nv50_crc_notifier_ctx *, u32 wndw);
+ int (*set_src)(struct nv50_head *, int or, enum nv50_crc_source_type type,
+ struct nv50_crc_notifier_ctx *ctx);
int (*set_ctx)(struct nv50_head *, struct nv50_crc_notifier_ctx *);
u32 (*get_entry)(struct nv50_head *, struct nv50_crc_notifier_ctx *,
enum nv50_crc_source, int idx);
@@ -95,6 +93,7 @@ void nv50_crc_atomic_clr(struct nv50_head *);
extern const struct nv50_crc_func crc907d;
extern const struct nv50_crc_func crcc37d;
+extern const struct nv50_crc_func crcc57d;
#else /* IS_ENABLED(CONFIG_DEBUG_FS) */
struct nv50_crc {};
diff --git a/drivers/gpu/drm/nouveau/dispnv50/crc907d.c b/drivers/gpu/drm/nouveau/dispnv50/crc907d.c
index 0fb0fdb9..f9ad641 100644
--- a/drivers/gpu/drm/nouveau/dispnv50/crc907d.c
+++ b/drivers/gpu/drm/nouveau/dispnv50/crc907d.c
@@ -23,9 +23,8 @@ struct crc907d_notifier {
} __packed;
static int
-crc907d_set_src(struct nv50_head *head, int or,
- enum nv50_crc_source_type source,
- struct nv50_crc_notifier_ctx *ctx, u32 wndw)
+crc907d_set_src(struct nv50_head *head, int or, enum nv50_crc_source_type source,
+ struct nv50_crc_notifier_ctx *ctx)
{
struct nvif_push *push = nv50_disp(head->base.base.dev)->core->chan.push;
const int i = head->base.index;
@@ -33,7 +32,8 @@ crc907d_set_src(struct nv50_head *head, int or,
NVDEF(NV907D, HEAD_SET_CRC_CONTROL, EXPECT_BUFFER_COLLAPSE, FALSE) |
NVDEF(NV907D, HEAD_SET_CRC_CONTROL, TIMESTAMP_MODE, FALSE) |
NVDEF(NV907D, HEAD_SET_CRC_CONTROL, SECONDARY_OUTPUT, NONE) |
- NVDEF(NV907D, HEAD_SET_CRC_CONTROL, CRC_DURING_SNOOZE, DISABLE);
+ NVDEF(NV907D, HEAD_SET_CRC_CONTROL, CRC_DURING_SNOOZE, DISABLE) |
+ NVDEF(NV907D, HEAD_SET_CRC_CONTROL, WIDE_PIPE_CRC, ENABLE);
int ret;
switch (source) {
diff --git a/drivers/gpu/drm/nouveau/dispnv50/crcc37d.c b/drivers/gpu/drm/nouveau/dispnv50/crcc37d.c
index 814e5bd..f10f6c4 100644
--- a/drivers/gpu/drm/nouveau/dispnv50/crcc37d.c
+++ b/drivers/gpu/drm/nouveau/dispnv50/crcc37d.c
@@ -2,6 +2,7 @@
#include <drm/drm_crtc.h>
#include "crc.h"
+#include "crcc37d.h"
#include "core.h"
#include "disp.h"
#include "head.h"
@@ -10,38 +11,13 @@
#include <nvhw/class/clc37d.h>
-#define CRCC37D_MAX_ENTRIES 2047
-
-struct crcc37d_notifier {
- u32 status;
-
- /* reserved */
- u32 :32;
- u32 :32;
- u32 :32;
- u32 :32;
- u32 :32;
- u32 :32;
- u32 :32;
-
- struct crcc37d_entry {
- u32 status[2];
- u32 :32; /* reserved */
- u32 compositor_crc;
- u32 rg_crc;
- u32 output_crc[2];
- u32 :32; /* reserved */
- } entries[CRCC37D_MAX_ENTRIES];
-} __packed;
-
static int
-crcc37d_set_src(struct nv50_head *head, int or,
- enum nv50_crc_source_type source,
- struct nv50_crc_notifier_ctx *ctx, u32 wndw)
+crcc37d_set_src(struct nv50_head *head, int or, enum nv50_crc_source_type source,
+ struct nv50_crc_notifier_ctx *ctx)
{
struct nvif_push *push = nv50_disp(head->base.base.dev)->core->chan.push;
const int i = head->base.index;
- u32 crc_args = NVVAL(NVC37D, HEAD_SET_CRC_CONTROL, CONTROLLING_CHANNEL, wndw) |
+ u32 crc_args = NVVAL(NVC37D, HEAD_SET_CRC_CONTROL, CONTROLLING_CHANNEL, i * 4) |
NVDEF(NVC37D, HEAD_SET_CRC_CONTROL, EXPECT_BUFFER_COLLAPSE, FALSE) |
NVDEF(NVC37D, HEAD_SET_CRC_CONTROL, SECONDARY_CRC, NONE) |
NVDEF(NVC37D, HEAD_SET_CRC_CONTROL, CRC_DURING_SNOOZE, DISABLE);
@@ -75,8 +51,7 @@ crcc37d_set_src(struct nv50_head *head, int or,
return 0;
}
-static int
-crcc37d_set_ctx(struct nv50_head *head, struct nv50_crc_notifier_ctx *ctx)
+int crcc37d_set_ctx(struct nv50_head *head, struct nv50_crc_notifier_ctx *ctx)
{
struct nvif_push *push = nv50_disp(head->base.base.dev)->core->chan.push;
const int i = head->base.index;
@@ -89,9 +64,8 @@ crcc37d_set_ctx(struct nv50_head *head, struct nv50_crc_notifier_ctx *ctx)
return 0;
}
-static u32 crcc37d_get_entry(struct nv50_head *head,
- struct nv50_crc_notifier_ctx *ctx,
- enum nv50_crc_source source, int idx)
+u32 crcc37d_get_entry(struct nv50_head *head, struct nv50_crc_notifier_ctx *ctx,
+ enum nv50_crc_source source, int idx)
{
struct crcc37d_notifier __iomem *notifier = ctx->mem.object.map.ptr;
struct crcc37d_entry __iomem *entry = ¬ifier->entries[idx];
@@ -105,8 +79,7 @@ static u32 crcc37d_get_entry(struct nv50_head *head,
return ioread32_native(crc_addr);
}
-static bool crcc37d_ctx_finished(struct nv50_head *head,
- struct nv50_crc_notifier_ctx *ctx)
+bool crcc37d_ctx_finished(struct nv50_head *head, struct nv50_crc_notifier_ctx *ctx)
{
struct nouveau_drm *drm = nouveau_drm(head->base.base.dev);
struct crcc37d_notifier __iomem *notifier = ctx->mem.object.map.ptr;
@@ -148,7 +121,7 @@ const struct nv50_crc_func crcc37d = {
.set_ctx = crcc37d_set_ctx,
.get_entry = crcc37d_get_entry,
.ctx_finished = crcc37d_ctx_finished,
- .flip_threshold = CRCC37D_MAX_ENTRIES - 30,
+ .flip_threshold = CRCC37D_FLIP_THRESHOLD,
.num_entries = CRCC37D_MAX_ENTRIES,
.notifier_len = sizeof(struct crcc37d_notifier),
};
diff --git a/drivers/gpu/drm/nouveau/dispnv50/crcc37d.h b/drivers/gpu/drm/nouveau/dispnv50/crcc37d.h
new file mode 100644
index 0000000..5775137
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/dispnv50/crcc37d.h
@@ -0,0 +1,40 @@
+/* SPDX-License-Identifier: MIT */
+
+#ifndef __CRCC37D_H__
+#define __CRCC37D_H__
+
+#include <linux/types.h>
+
+#include "crc.h"
+
+#define CRCC37D_MAX_ENTRIES 2047
+#define CRCC37D_FLIP_THRESHOLD (CRCC37D_MAX_ENTRIES - 30)
+
+struct crcc37d_notifier {
+ u32 status;
+
+ /* reserved */
+ u32:32;
+ u32:32;
+ u32:32;
+ u32:32;
+ u32:32;
+ u32:32;
+ u32:32;
+
+ struct crcc37d_entry {
+ u32 status[2];
+ u32:32; /* reserved */
+ u32 compositor_crc;
+ u32 rg_crc;
+ u32 output_crc[2];
+ u32:32; /* reserved */
+ } entries[CRCC37D_MAX_ENTRIES];
+} __packed;
+
+int crcc37d_set_ctx(struct nv50_head *head, struct nv50_crc_notifier_ctx *ctx);
+u32 crcc37d_get_entry(struct nv50_head *head, struct nv50_crc_notifier_ctx *ctx,
+ enum nv50_crc_source source, int idx);
+bool crcc37d_ctx_finished(struct nv50_head *head, struct nv50_crc_notifier_ctx *ctx);
+
+#endif /* !__CRCC37D_H__ */
diff --git a/drivers/gpu/drm/nouveau/dispnv50/crcc57d.c b/drivers/gpu/drm/nouveau/dispnv50/crcc57d.c
new file mode 100644
index 0000000..cc0130e
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/dispnv50/crcc57d.c
@@ -0,0 +1,58 @@
+// SPDX-License-Identifier: MIT
+
+#include "crc.h"
+#include "crcc37d.h"
+#include "core.h"
+#include "disp.h"
+#include "head.h"
+
+#include <nvif/pushc37b.h>
+
+#include <nvhw/class/clc57d.h>
+
+static int crcc57d_set_src(struct nv50_head *head, int or, enum nv50_crc_source_type source,
+ struct nv50_crc_notifier_ctx *ctx)
+{
+ struct nvif_push *push = nv50_disp(head->base.base.dev)->core->chan.push;
+ const int i = head->base.index;
+ u32 crc_args = NVDEF(NVC57D, HEAD_SET_CRC_CONTROL, CONTROLLING_CHANNEL, CORE) |
+ NVDEF(NVC57D, HEAD_SET_CRC_CONTROL, EXPECT_BUFFER_COLLAPSE, FALSE) |
+ NVDEF(NVC57D, HEAD_SET_CRC_CONTROL, SECONDARY_CRC, NONE) |
+ NVDEF(NVC57D, HEAD_SET_CRC_CONTROL, CRC_DURING_SNOOZE, DISABLE);
+ int ret;
+
+ switch (source) {
+ case NV50_CRC_SOURCE_TYPE_SOR:
+ crc_args |= NVDEF(NVC57D, HEAD_SET_CRC_CONTROL, PRIMARY_CRC, SOR(or));
+ break;
+ case NV50_CRC_SOURCE_TYPE_SF:
+ crc_args |= NVDEF(NVC57D, HEAD_SET_CRC_CONTROL, PRIMARY_CRC, SF);
+ break;
+ default:
+ break;
+ }
+
+ ret = PUSH_WAIT(push, 4);
+ if (ret)
+ return ret;
+
+ if (source) {
+ PUSH_MTHD(push, NVC57D, HEAD_SET_CONTEXT_DMA_CRC(i), ctx->ntfy.handle);
+ PUSH_MTHD(push, NVC57D, HEAD_SET_CRC_CONTROL(i), crc_args);
+ } else {
+ PUSH_MTHD(push, NVC57D, HEAD_SET_CRC_CONTROL(i), 0);
+ PUSH_MTHD(push, NVC57D, HEAD_SET_CONTEXT_DMA_CRC(i), 0);
+ }
+
+ return 0;
+}
+
+const struct nv50_crc_func crcc57d = {
+ .set_src = crcc57d_set_src,
+ .set_ctx = crcc37d_set_ctx,
+ .get_entry = crcc37d_get_entry,
+ .ctx_finished = crcc37d_ctx_finished,
+ .flip_threshold = CRCC37D_FLIP_THRESHOLD,
+ .num_entries = CRCC37D_MAX_ENTRIES,
+ .notifier_len = sizeof(struct crcc37d_notifier),
+};
diff --git a/drivers/gpu/drm/nouveau/dispnv50/curs507a.c b/drivers/gpu/drm/nouveau/dispnv50/curs507a.c
index 54fbd6f..00e19fd 100644
--- a/drivers/gpu/drm/nouveau/dispnv50/curs507a.c
+++ b/drivers/gpu/drm/nouveau/dispnv50/curs507a.c
@@ -98,6 +98,7 @@ static int
curs507a_acquire(struct nv50_wndw *wndw, struct nv50_wndw_atom *asyw,
struct nv50_head_atom *asyh)
{
+ struct nouveau_drm *drm = nouveau_drm(wndw->plane.dev);
struct nv50_head *head = nv50_head(asyw->state.crtc);
int ret;
@@ -109,8 +110,20 @@ curs507a_acquire(struct nv50_wndw *wndw, struct nv50_wndw_atom *asyw,
if (ret || !asyh->curs.visible)
return ret;
- if (asyw->image.w != asyw->image.h)
+ if (asyw->state.crtc_w != asyw->state.crtc_h) {
+ NV_ATOMIC(drm, "Plane width/height must be equal for cursors\n");
return -EINVAL;
+ }
+
+ if (asyw->image.w != asyw->state.crtc_w) {
+ NV_ATOMIC(drm, "Plane width must be equal to fb width for cursors (height can be larger though)\n");
+ return -EINVAL;
+ }
+
+ if (asyw->state.src_x || asyw->state.src_y) {
+ NV_ATOMIC(drm, "Cursor planes do not support framebuffer offsets\n");
+ return -EINVAL;
+ }
ret = head->func->curs_layout(head, asyw, asyh);
if (ret)
diff --git a/drivers/gpu/drm/nouveau/dispnv50/disp.c b/drivers/gpu/drm/nouveau/dispnv50/disp.c
index 8e28403..ae1f412 100644
--- a/drivers/gpu/drm/nouveau/dispnv50/disp.c
+++ b/drivers/gpu/drm/nouveau/dispnv50/disp.c
@@ -852,6 +852,9 @@ nv50_hdmi_enable(struct drm_encoder *encoder, struct nouveau_crtc *nv_crtc,
ret = drm_hdmi_avi_infoframe_from_display_mode(&avi_frame.avi,
&nv_connector->base, mode);
if (!ret) {
+ drm_hdmi_avi_infoframe_quant_range(&avi_frame.avi,
+ &nv_connector->base, mode,
+ HDMI_QUANTIZATION_RANGE_FULL);
/* We have an AVI InfoFrame, populate it to the display */
args.pwr.avi_infoframe_length
= hdmi_infoframe_pack(&avi_frame, args.infoframes, 17);
@@ -1387,12 +1390,11 @@ nv50_mstm_cleanup(struct nv50_mstm *mstm)
{
struct nouveau_drm *drm = nouveau_drm(mstm->outp->base.base.dev);
struct drm_encoder *encoder;
- int ret;
NV_ATOMIC(drm, "%s: mstm cleanup\n", mstm->outp->base.base.name);
- ret = drm_dp_check_act_status(&mstm->mgr);
+ drm_dp_check_act_status(&mstm->mgr);
- ret = drm_dp_update_payload_part2(&mstm->mgr);
+ drm_dp_update_payload_part2(&mstm->mgr);
drm_for_each_encoder(encoder, mstm->outp->base.base.dev) {
if (encoder->encoder_type == DRM_MODE_ENCODER_DPMST) {
@@ -1411,10 +1413,9 @@ nv50_mstm_prepare(struct nv50_mstm *mstm)
{
struct nouveau_drm *drm = nouveau_drm(mstm->outp->base.base.dev);
struct drm_encoder *encoder;
- int ret;
NV_ATOMIC(drm, "%s: mstm prepare\n", mstm->outp->base.base.name);
- ret = drm_dp_update_payload_part1(&mstm->mgr, 1);
+ drm_dp_update_payload_part1(&mstm->mgr, 1);
drm_for_each_encoder(encoder, mstm->outp->base.base.dev) {
if (encoder->encoder_type == DRM_MODE_ENCODER_DPMST) {
diff --git a/drivers/gpu/drm/nouveau/dispnv50/head.c b/drivers/gpu/drm/nouveau/dispnv50/head.c
index 72099d1..c3c57be 100644
--- a/drivers/gpu/drm/nouveau/dispnv50/head.c
+++ b/drivers/gpu/drm/nouveau/dispnv50/head.c
@@ -226,10 +226,24 @@ static int
nv50_head_atomic_check_lut(struct nv50_head *head,
struct nv50_head_atom *asyh)
{
- struct nv50_disp *disp = nv50_disp(head->base.base.dev);
- struct drm_property_blob *olut = asyh->state.gamma_lut;
+ struct drm_device *dev = head->base.base.dev;
+ struct drm_crtc *crtc = &head->base.base;
+ struct nv50_disp *disp = nv50_disp(dev);
+ struct nouveau_drm *drm = nouveau_drm(dev);
+ struct drm_property_blob *olut = asyh->state.gamma_lut,
+ *ilut = asyh->state.degamma_lut;
int size;
+ /* Ensure that the ilut is valid */
+ if (ilut) {
+ size = drm_color_lut_size(ilut);
+ if (!head->func->ilut_check(size)) {
+ NV_ATOMIC(drm, "Invalid size %d for degamma on [CRTC:%d:%s]\n",
+ size, crtc->base.id, crtc->name);
+ return -EINVAL;
+ }
+ }
+
/* Determine whether core output LUT should be enabled. */
if (olut) {
/* Check if any window(s) have stolen the core output LUT
@@ -256,7 +270,8 @@ nv50_head_atomic_check_lut(struct nv50_head *head,
}
if (!head->func->olut(head, asyh, size)) {
- DRM_DEBUG_KMS("Invalid olut\n");
+ NV_ATOMIC(drm, "Invalid size %d for gamma on [CRTC:%d:%s]\n",
+ size, crtc->base.id, crtc->name);
return -EINVAL;
}
asyh->olut.handle = disp->core->chan.vram.handle;
@@ -330,8 +345,17 @@ nv50_head_atomic_check(struct drm_crtc *crtc, struct drm_atomic_state *state)
struct drm_connector_state *conns;
struct drm_connector *conn;
int i, ret;
+ bool check_lut = asyh->state.color_mgmt_changed ||
+ memcmp(&armh->wndw, &asyh->wndw, sizeof(asyh->wndw));
NV_ATOMIC(drm, "%s atomic_check %d\n", crtc->name, asyh->state.active);
+
+ if (check_lut) {
+ ret = nv50_head_atomic_check_lut(head, asyh);
+ if (ret)
+ return ret;
+ }
+
if (asyh->state.active) {
for_each_new_connector_in_state(asyh->state.state, conn, conns, i) {
if (conns->crtc == crtc) {
@@ -357,14 +381,8 @@ nv50_head_atomic_check(struct drm_crtc *crtc, struct drm_atomic_state *state)
if (asyh->state.mode_changed || asyh->state.connectors_changed)
nv50_head_atomic_check_mode(head, asyh);
- if (asyh->state.color_mgmt_changed ||
- memcmp(&armh->wndw, &asyh->wndw, sizeof(asyh->wndw))) {
- int ret = nv50_head_atomic_check_lut(head, asyh);
- if (ret)
- return ret;
-
+ if (check_lut)
asyh->olut.visible = asyh->olut.handle != 0;
- }
if (asyc) {
if (asyc->set.scaler)
diff --git a/drivers/gpu/drm/nouveau/dispnv50/head.h b/drivers/gpu/drm/nouveau/dispnv50/head.h
index 0bac6be..41c8788 100644
--- a/drivers/gpu/drm/nouveau/dispnv50/head.h
+++ b/drivers/gpu/drm/nouveau/dispnv50/head.h
@@ -29,6 +29,7 @@ struct nv50_head_func {
int (*view)(struct nv50_head *, struct nv50_head_atom *);
int (*mode)(struct nv50_head *, struct nv50_head_atom *);
bool (*olut)(struct nv50_head *, struct nv50_head_atom *, int);
+ bool (*ilut_check)(int size);
bool olut_identity;
int olut_size;
int (*olut_set)(struct nv50_head *, struct nv50_head_atom *);
@@ -71,6 +72,7 @@ extern const struct nv50_head_func head907d;
int head907d_view(struct nv50_head *, struct nv50_head_atom *);
int head907d_mode(struct nv50_head *, struct nv50_head_atom *);
bool head907d_olut(struct nv50_head *, struct nv50_head_atom *, int);
+bool head907d_ilut_check(int size);
int head907d_olut_set(struct nv50_head *, struct nv50_head_atom *);
int head907d_olut_clr(struct nv50_head *);
int head907d_core_set(struct nv50_head *, struct nv50_head_atom *);
diff --git a/drivers/gpu/drm/nouveau/dispnv50/head907d.c b/drivers/gpu/drm/nouveau/dispnv50/head907d.c
index 85648d7..18fe4c1 100644
--- a/drivers/gpu/drm/nouveau/dispnv50/head907d.c
+++ b/drivers/gpu/drm/nouveau/dispnv50/head907d.c
@@ -314,6 +314,11 @@ head907d_olut(struct nv50_head *head, struct nv50_head_atom *asyh, int size)
return true;
}
+bool head907d_ilut_check(int size)
+{
+ return size == 256 || size == 1024;
+}
+
int
head907d_mode(struct nv50_head *head, struct nv50_head_atom *asyh)
{
@@ -409,6 +414,7 @@ head907d = {
.view = head907d_view,
.mode = head907d_mode,
.olut = head907d_olut,
+ .ilut_check = head907d_ilut_check,
.olut_size = 1024,
.olut_set = head907d_olut_set,
.olut_clr = head907d_olut_clr,
diff --git a/drivers/gpu/drm/nouveau/dispnv50/head917d.c b/drivers/gpu/drm/nouveau/dispnv50/head917d.c
index ea9f866..4ce47b5 100644
--- a/drivers/gpu/drm/nouveau/dispnv50/head917d.c
+++ b/drivers/gpu/drm/nouveau/dispnv50/head917d.c
@@ -119,6 +119,7 @@ head917d = {
.view = head907d_view,
.mode = head907d_mode,
.olut = head907d_olut,
+ .ilut_check = head907d_ilut_check,
.olut_size = 1024,
.olut_set = head907d_olut_set,
.olut_clr = head907d_olut_clr,
diff --git a/drivers/gpu/drm/nouveau/dispnv50/headc37d.c b/drivers/gpu/drm/nouveau/dispnv50/headc37d.c
index 63adfeb..a4a3b78 100644
--- a/drivers/gpu/drm/nouveau/dispnv50/headc37d.c
+++ b/drivers/gpu/drm/nouveau/dispnv50/headc37d.c
@@ -285,6 +285,7 @@ headc37d = {
.view = headc37d_view,
.mode = headc37d_mode,
.olut = headc37d_olut,
+ .ilut_check = head907d_ilut_check,
.olut_size = 1024,
.olut_set = headc37d_olut_set,
.olut_clr = headc37d_olut_clr,
diff --git a/drivers/gpu/drm/nouveau/dispnv50/headc57d.c b/drivers/gpu/drm/nouveau/dispnv50/headc57d.c
index fd51527..543f08c 100644
--- a/drivers/gpu/drm/nouveau/dispnv50/headc57d.c
+++ b/drivers/gpu/drm/nouveau/dispnv50/headc57d.c
@@ -169,7 +169,7 @@ headc57d_olut_load(struct drm_color_lut *in, int size, void __iomem *mem)
writew(readw(mem - 4), mem + 4);
}
-bool
+static bool
headc57d_olut(struct nv50_head *head, struct nv50_head_atom *asyh, int size)
{
if (size != 0 && size != 256 && size != 1024)
@@ -236,6 +236,7 @@ headc57d = {
.view = headc37d_view,
.mode = headc57d_mode,
.olut = headc57d_olut,
+ .ilut_check = head907d_ilut_check,
.olut_identity = true,
.olut_size = 1024,
.olut_set = headc57d_olut_set,
diff --git a/drivers/gpu/drm/nouveau/dispnv50/wndw.c b/drivers/gpu/drm/nouveau/dispnv50/wndw.c
index 8d048ba..133c873 100644
--- a/drivers/gpu/drm/nouveau/dispnv50/wndw.c
+++ b/drivers/gpu/drm/nouveau/dispnv50/wndw.c
@@ -403,10 +403,7 @@ nv50_wndw_atomic_check_lut(struct nv50_wndw *wndw,
/* Recalculate LUT state. */
memset(&asyw->xlut, 0x00, sizeof(asyw->xlut));
if ((asyw->ilut = wndw->func->ilut ? ilut : NULL)) {
- if (!wndw->func->ilut(wndw, asyw, drm_color_lut_size(ilut))) {
- DRM_DEBUG_KMS("Invalid ilut\n");
- return -EINVAL;
- }
+ wndw->func->ilut(wndw, asyw, drm_color_lut_size(ilut));
asyw->xlut.handle = wndw->wndw.vram.handle;
asyw->xlut.i.buffer = !asyw->xlut.i.buffer;
asyw->set.xlut = true;
@@ -539,6 +536,8 @@ nv50_wndw_prepare_fb(struct drm_plane *plane, struct drm_plane_state *state)
struct nouveau_bo *nvbo;
struct nv50_head_atom *asyh;
struct nv50_wndw_ctxdma *ctxdma;
+ struct dma_resv_iter cursor;
+ struct dma_fence *fence;
int ret;
NV_ATOMIC(drm, "%s prepare: %p\n", plane->name, fb);
@@ -561,7 +560,13 @@ nv50_wndw_prepare_fb(struct drm_plane *plane, struct drm_plane_state *state)
asyw->image.handle[0] = ctxdma->object.handle;
}
- asyw->state.fence = dma_resv_get_excl_unlocked(nvbo->bo.base.resv);
+ dma_resv_iter_begin(&cursor, nvbo->bo.base.resv, false);
+ dma_resv_for_each_fence_unlocked(&cursor, fence) {
+ /* TODO: We only use the first writer here */
+ asyw->state.fence = dma_fence_get(fence);
+ break;
+ }
+ dma_resv_iter_end(&cursor);
asyw->image.offset[0] = nvbo->offset;
if (wndw->func->prepare) {
diff --git a/drivers/gpu/drm/nouveau/dispnv50/wndw.h b/drivers/gpu/drm/nouveau/dispnv50/wndw.h
index f4e0c508..9c9f2c2 100644
--- a/drivers/gpu/drm/nouveau/dispnv50/wndw.h
+++ b/drivers/gpu/drm/nouveau/dispnv50/wndw.h
@@ -64,7 +64,7 @@ struct nv50_wndw_func {
int (*ntfy_clr)(struct nv50_wndw *);
int (*ntfy_wait_begun)(struct nouveau_bo *, u32 offset,
struct nvif_device *);
- bool (*ilut)(struct nv50_wndw *, struct nv50_wndw_atom *, int);
+ void (*ilut)(struct nv50_wndw *wndw, struct nv50_wndw_atom *asyh, int size);
void (*csc)(struct nv50_wndw *, struct nv50_wndw_atom *,
const struct drm_color_ctm *);
int (*csc_set)(struct nv50_wndw *, struct nv50_wndw_atom *);
@@ -129,7 +129,7 @@ int wndwc37e_update(struct nv50_wndw *, u32 *);
int wndwc57e_new(struct nouveau_drm *, enum drm_plane_type, int, s32,
struct nv50_wndw **);
-bool wndwc57e_ilut(struct nv50_wndw *, struct nv50_wndw_atom *, int);
+void wndwc57e_ilut(struct nv50_wndw *, struct nv50_wndw_atom *, int);
int wndwc57e_ilut_set(struct nv50_wndw *, struct nv50_wndw_atom *);
int wndwc57e_ilut_clr(struct nv50_wndw *);
int wndwc57e_csc_set(struct nv50_wndw *, struct nv50_wndw_atom *);
diff --git a/drivers/gpu/drm/nouveau/dispnv50/wndwc37e.c b/drivers/gpu/drm/nouveau/dispnv50/wndwc37e.c
index 57df997..183d2c0 100644
--- a/drivers/gpu/drm/nouveau/dispnv50/wndwc37e.c
+++ b/drivers/gpu/drm/nouveau/dispnv50/wndwc37e.c
@@ -82,18 +82,14 @@ wndwc37e_ilut_set(struct nv50_wndw *wndw, struct nv50_wndw_atom *asyw)
return 0;
}
-static bool
+static void
wndwc37e_ilut(struct nv50_wndw *wndw, struct nv50_wndw_atom *asyw, int size)
{
- if (size != 256 && size != 1024)
- return false;
-
asyw->xlut.i.size = size == 1024 ? NVC37E_SET_CONTROL_INPUT_LUT_SIZE_SIZE_1025 :
NVC37E_SET_CONTROL_INPUT_LUT_SIZE_SIZE_257;
asyw->xlut.i.range = NVC37E_SET_CONTROL_INPUT_LUT_RANGE_UNITY;
asyw->xlut.i.output_mode = NVC37E_SET_CONTROL_INPUT_LUT_OUTPUT_MODE_INTERPOLATE;
asyw->xlut.i.load = head907d_olut_load;
- return true;
}
int
diff --git a/drivers/gpu/drm/nouveau/dispnv50/wndwc57e.c b/drivers/gpu/drm/nouveau/dispnv50/wndwc57e.c
index abdd3bb..37f6da8 100644
--- a/drivers/gpu/drm/nouveau/dispnv50/wndwc57e.c
+++ b/drivers/gpu/drm/nouveau/dispnv50/wndwc57e.c
@@ -179,11 +179,11 @@ wndwc57e_ilut_load(struct drm_color_lut *in, int size, void __iomem *mem)
writew(readw(mem - 4), mem + 4);
}
-bool
+void
wndwc57e_ilut(struct nv50_wndw *wndw, struct nv50_wndw_atom *asyw, int size)
{
- if (size = size ? size : 1024, size != 256 && size != 1024)
- return false;
+ if (!size)
+ size = 1024;
if (size == 256)
asyw->xlut.i.mode = NVC57E_SET_ILUT_CONTROL_MODE_DIRECT8;
@@ -193,7 +193,6 @@ wndwc57e_ilut(struct nv50_wndw *wndw, struct nv50_wndw_atom *asyw, int size)
asyw->xlut.i.size = 4 /* VSS header. */ + size + 1 /* Entries. */;
asyw->xlut.i.output_mode = NVC57E_SET_ILUT_CONTROL_INTERPOLATE_DISABLE;
asyw->xlut.i.load = wndwc57e_ilut_load;
- return true;
}
/****************************************************************
diff --git a/drivers/gpu/drm/nouveau/include/nvhw/class/cl907d.h b/drivers/gpu/drm/nouveau/include/nvhw/class/cl907d.h
index 79aff6f..f972ef1 100644
--- a/drivers/gpu/drm/nouveau/include/nvhw/class/cl907d.h
+++ b/drivers/gpu/drm/nouveau/include/nvhw/class/cl907d.h
@@ -246,6 +246,9 @@
#define NV907D_HEAD_SET_CRC_CONTROL_CRC_DURING_SNOOZE 5:5
#define NV907D_HEAD_SET_CRC_CONTROL_CRC_DURING_SNOOZE_DISABLE (0x00000000)
#define NV907D_HEAD_SET_CRC_CONTROL_CRC_DURING_SNOOZE_ENABLE (0x00000001)
+#define NV907D_HEAD_SET_CRC_CONTROL_WIDE_PIPE_CRC 6:6
+#define NV907D_HEAD_SET_CRC_CONTROL_WIDE_PIPE_CRC_DISABLE (0x00000000)
+#define NV907D_HEAD_SET_CRC_CONTROL_WIDE_PIPE_CRC_ENABLE (0x00000001)
#define NV907D_HEAD_SET_CONTEXT_DMA_CRC(a) (0x00000438 + (a)*0x00000300)
#define NV907D_HEAD_SET_CONTEXT_DMA_CRC_HANDLE 31:0
#define NV907D_HEAD_SET_OUTPUT_LUT_LO(a) (0x00000448 + (a)*0x00000300)
diff --git a/drivers/gpu/drm/nouveau/include/nvhw/class/clc57d.h b/drivers/gpu/drm/nouveau/include/nvhw/class/clc57d.h
index d83ac81..d4bad2d 100644
--- a/drivers/gpu/drm/nouveau/include/nvhw/class/clc57d.h
+++ b/drivers/gpu/drm/nouveau/include/nvhw/class/clc57d.h
@@ -265,6 +265,75 @@
#define NVC57D_HEAD_SET_RASTER_BLANK_START(a) (0x00002070 + (a)*0x00000400)
#define NVC57D_HEAD_SET_RASTER_BLANK_START_X 14:0
#define NVC57D_HEAD_SET_RASTER_BLANK_START_Y 30:16
+#define NVC57D_HEAD_SET_CONTEXT_DMA_CRC(a) (0x00002180 + (a)*0x00000400)
+#define NVC57D_HEAD_SET_CONTEXT_DMA_CRC_HANDLE 31:0
+#define NVC57D_HEAD_SET_CRC_CONTROL(a) (0x00002184 + (a)*0x00000400)
+#define NVC57D_HEAD_SET_CRC_CONTROL_CONTROLLING_CHANNEL 5:0
+#define NVC57D_HEAD_SET_CRC_CONTROL_CONTROLLING_CHANNEL_WIN_0 (0x00000000)
+#define NVC57D_HEAD_SET_CRC_CONTROL_CONTROLLING_CHANNEL_WIN_1 (0x00000001)
+#define NVC57D_HEAD_SET_CRC_CONTROL_CONTROLLING_CHANNEL_WIN_2 (0x00000002)
+#define NVC57D_HEAD_SET_CRC_CONTROL_CONTROLLING_CHANNEL_WIN_3 (0x00000003)
+#define NVC57D_HEAD_SET_CRC_CONTROL_CONTROLLING_CHANNEL_WIN_4 (0x00000004)
+#define NVC57D_HEAD_SET_CRC_CONTROL_CONTROLLING_CHANNEL_WIN_5 (0x00000005)
+#define NVC57D_HEAD_SET_CRC_CONTROL_CONTROLLING_CHANNEL_WIN_6 (0x00000006)
+#define NVC57D_HEAD_SET_CRC_CONTROL_CONTROLLING_CHANNEL_WIN_7 (0x00000007)
+#define NVC57D_HEAD_SET_CRC_CONTROL_CONTROLLING_CHANNEL_WIN_8 (0x00000008)
+#define NVC57D_HEAD_SET_CRC_CONTROL_CONTROLLING_CHANNEL_WIN_9 (0x00000009)
+#define NVC57D_HEAD_SET_CRC_CONTROL_CONTROLLING_CHANNEL_WIN_10 (0x0000000A)
+#define NVC57D_HEAD_SET_CRC_CONTROL_CONTROLLING_CHANNEL_WIN_11 (0x0000000B)
+#define NVC57D_HEAD_SET_CRC_CONTROL_CONTROLLING_CHANNEL_WIN_12 (0x0000000C)
+#define NVC57D_HEAD_SET_CRC_CONTROL_CONTROLLING_CHANNEL_WIN_13 (0x0000000D)
+#define NVC57D_HEAD_SET_CRC_CONTROL_CONTROLLING_CHANNEL_WIN_14 (0x0000000E)
+#define NVC57D_HEAD_SET_CRC_CONTROL_CONTROLLING_CHANNEL_WIN_15 (0x0000000F)
+#define NVC57D_HEAD_SET_CRC_CONTROL_CONTROLLING_CHANNEL_WIN_16 (0x00000010)
+#define NVC57D_HEAD_SET_CRC_CONTROL_CONTROLLING_CHANNEL_WIN_17 (0x00000011)
+#define NVC57D_HEAD_SET_CRC_CONTROL_CONTROLLING_CHANNEL_WIN_18 (0x00000012)
+#define NVC57D_HEAD_SET_CRC_CONTROL_CONTROLLING_CHANNEL_WIN_19 (0x00000013)
+#define NVC57D_HEAD_SET_CRC_CONTROL_CONTROLLING_CHANNEL_WIN_20 (0x00000014)
+#define NVC57D_HEAD_SET_CRC_CONTROL_CONTROLLING_CHANNEL_WIN_21 (0x00000015)
+#define NVC57D_HEAD_SET_CRC_CONTROL_CONTROLLING_CHANNEL_WIN_22 (0x00000016)
+#define NVC57D_HEAD_SET_CRC_CONTROL_CONTROLLING_CHANNEL_WIN_23 (0x00000017)
+#define NVC57D_HEAD_SET_CRC_CONTROL_CONTROLLING_CHANNEL_WIN_24 (0x00000018)
+#define NVC57D_HEAD_SET_CRC_CONTROL_CONTROLLING_CHANNEL_WIN_25 (0x00000019)
+#define NVC57D_HEAD_SET_CRC_CONTROL_CONTROLLING_CHANNEL_WIN_26 (0x0000001A)
+#define NVC57D_HEAD_SET_CRC_CONTROL_CONTROLLING_CHANNEL_WIN_27 (0x0000001B)
+#define NVC57D_HEAD_SET_CRC_CONTROL_CONTROLLING_CHANNEL_WIN_28 (0x0000001C)
+#define NVC57D_HEAD_SET_CRC_CONTROL_CONTROLLING_CHANNEL_WIN_29 (0x0000001D)
+#define NVC57D_HEAD_SET_CRC_CONTROL_CONTROLLING_CHANNEL_WIN_30 (0x0000001E)
+#define NVC57D_HEAD_SET_CRC_CONTROL_CONTROLLING_CHANNEL_WIN_31 (0x0000001F)
+#define NVC57D_HEAD_SET_CRC_CONTROL_CONTROLLING_CHANNEL_CORE (0x00000020)
+#define NVC57D_HEAD_SET_CRC_CONTROL_EXPECT_BUFFER_COLLAPSE 8:8
+#define NVC57D_HEAD_SET_CRC_CONTROL_EXPECT_BUFFER_COLLAPSE_FALSE (0x00000000)
+#define NVC57D_HEAD_SET_CRC_CONTROL_EXPECT_BUFFER_COLLAPSE_TRUE (0x00000001)
+#define NVC57D_HEAD_SET_CRC_CONTROL_PRIMARY_CRC 19:12
+#define NVC57D_HEAD_SET_CRC_CONTROL_PRIMARY_CRC_NONE (0x00000000)
+#define NVC57D_HEAD_SET_CRC_CONTROL_PRIMARY_CRC_SF (0x00000030)
+#define NVC57D_HEAD_SET_CRC_CONTROL_PRIMARY_CRC_SOR(i) (0x00000050 +(i))
+#define NVC57D_HEAD_SET_CRC_CONTROL_PRIMARY_CRC_SOR__SIZE_1 8
+#define NVC57D_HEAD_SET_CRC_CONTROL_PRIMARY_CRC_SOR0 (0x00000050)
+#define NVC57D_HEAD_SET_CRC_CONTROL_PRIMARY_CRC_SOR1 (0x00000051)
+#define NVC57D_HEAD_SET_CRC_CONTROL_PRIMARY_CRC_SOR2 (0x00000052)
+#define NVC57D_HEAD_SET_CRC_CONTROL_PRIMARY_CRC_SOR3 (0x00000053)
+#define NVC57D_HEAD_SET_CRC_CONTROL_PRIMARY_CRC_SOR4 (0x00000054)
+#define NVC57D_HEAD_SET_CRC_CONTROL_PRIMARY_CRC_SOR5 (0x00000055)
+#define NVC57D_HEAD_SET_CRC_CONTROL_PRIMARY_CRC_SOR6 (0x00000056)
+#define NVC57D_HEAD_SET_CRC_CONTROL_PRIMARY_CRC_SOR7 (0x00000057)
+#define NVC57D_HEAD_SET_CRC_CONTROL_SECONDARY_CRC 27:20
+#define NVC57D_HEAD_SET_CRC_CONTROL_SECONDARY_CRC_NONE (0x00000000)
+#define NVC57D_HEAD_SET_CRC_CONTROL_SECONDARY_CRC_SF (0x00000030)
+#define NVC57D_HEAD_SET_CRC_CONTROL_SECONDARY_CRC_SOR(i) (0x00000050 +(i))
+#define NVC57D_HEAD_SET_CRC_CONTROL_SECONDARY_CRC_SOR__SIZE_1 8
+#define NVC57D_HEAD_SET_CRC_CONTROL_SECONDARY_CRC_SOR0 (0x00000050)
+#define NVC57D_HEAD_SET_CRC_CONTROL_SECONDARY_CRC_SOR1 (0x00000051)
+#define NVC57D_HEAD_SET_CRC_CONTROL_SECONDARY_CRC_SOR2 (0x00000052)
+#define NVC57D_HEAD_SET_CRC_CONTROL_SECONDARY_CRC_SOR3 (0x00000053)
+#define NVC57D_HEAD_SET_CRC_CONTROL_SECONDARY_CRC_SOR4 (0x00000054)
+#define NVC57D_HEAD_SET_CRC_CONTROL_SECONDARY_CRC_SOR5 (0x00000055)
+#define NVC57D_HEAD_SET_CRC_CONTROL_SECONDARY_CRC_SOR6 (0x00000056)
+#define NVC57D_HEAD_SET_CRC_CONTROL_SECONDARY_CRC_SOR7 (0x00000057)
+#define NVC57D_HEAD_SET_CRC_CONTROL_CRC_DURING_SNOOZE 9:9
+#define NVC57D_HEAD_SET_CRC_CONTROL_CRC_DURING_SNOOZE_DISABLE (0x00000000)
+#define NVC57D_HEAD_SET_CRC_CONTROL_CRC_DURING_SNOOZE_ENABLE (0x00000001)
#define NVC57D_HEAD_SET_OLUT_CONTROL(a) (0x00002280 + (a)*0x00000400)
#define NVC57D_HEAD_SET_OLUT_CONTROL_INTERPOLATE 0:0
#define NVC57D_HEAD_SET_OLUT_CONTROL_INTERPOLATE_DISABLE (0x00000000)
diff --git a/drivers/gpu/drm/nouveau/nouveau_backlight.c b/drivers/gpu/drm/nouveau/nouveau_backlight.c
index 1cbd71a..ae2f2ab 100644
--- a/drivers/gpu/drm/nouveau/nouveau_backlight.c
+++ b/drivers/gpu/drm/nouveau/nouveau_backlight.c
@@ -308,7 +308,10 @@ nv50_backlight_init(struct nouveau_backlight *bl,
if (ret < 0)
return ret;
- if (drm_edp_backlight_supported(edp_dpcd)) {
+ /* TODO: Add support for hybrid PWM/DPCD panels */
+ if (drm_edp_backlight_supported(edp_dpcd) &&
+ (edp_dpcd[1] & DP_EDP_BACKLIGHT_AUX_ENABLE_CAP) &&
+ (edp_dpcd[2] & DP_EDP_BACKLIGHT_BRIGHTNESS_AUX_SET_CAP)) {
NV_DEBUG(drm, "DPCD backlight controls supported on %s\n",
nv_conn->base.name);
diff --git a/drivers/gpu/drm/nouveau/nouveau_bios.c b/drivers/gpu/drm/nouveau/nouveau_bios.c
index e8c445e..41b78e9 100644
--- a/drivers/gpu/drm/nouveau/nouveau_bios.c
+++ b/drivers/gpu/drm/nouveau/nouveau_bios.c
@@ -2045,7 +2045,6 @@ nouveau_run_vbios_init(struct drm_device *dev)
{
struct nouveau_drm *drm = nouveau_drm(dev);
struct nvbios *bios = &drm->vbios;
- int ret = 0;
/* Reset the BIOS head to 0. */
bios->state.crtchead = 0;
@@ -2058,7 +2057,7 @@ nouveau_run_vbios_init(struct drm_device *dev)
bios->fp.lvds_init_run = false;
}
- return ret;
+ return 0;
}
static bool
diff --git a/drivers/gpu/drm/nouveau/nouveau_display.c b/drivers/gpu/drm/nouveau/nouveau_display.c
index 929de41..2b46083 100644
--- a/drivers/gpu/drm/nouveau/nouveau_display.c
+++ b/drivers/gpu/drm/nouveau/nouveau_display.c
@@ -306,7 +306,7 @@ nouveau_framebuffer_new(struct drm_device *dev,
struct nouveau_bo *nvbo = nouveau_gem_object(gem);
struct drm_framebuffer *fb;
const struct drm_format_info *info;
- unsigned int width, height, i;
+ unsigned int height, i;
uint32_t tile_mode;
uint8_t kind;
int ret;
@@ -343,9 +343,6 @@ nouveau_framebuffer_new(struct drm_device *dev,
info = drm_get_format_info(dev, mode_cmd);
for (i = 0; i < info->num_planes; i++) {
- width = drm_format_info_plane_width(info,
- mode_cmd->width,
- i);
height = drm_format_info_plane_height(info,
mode_cmd->height,
i);
diff --git a/drivers/gpu/drm/nouveau/nouveau_drm.c b/drivers/gpu/drm/nouveau/nouveau_drm.c
index e7efd9e..561309d 100644
--- a/drivers/gpu/drm/nouveau/nouveau_drm.c
+++ b/drivers/gpu/drm/nouveau/nouveau_drm.c
@@ -22,7 +22,6 @@
* Authors: Ben Skeggs
*/
-#include <linux/console.h>
#include <linux/delay.h>
#include <linux/module.h>
#include <linux/pci.h>
@@ -32,6 +31,7 @@
#include <drm/drm_aperture.h>
#include <drm/drm_crtc_helper.h>
+#include <drm/drm_drv.h>
#include <drm/drm_gem_ttm_helper.h>
#include <drm/drm_ioctl.h>
#include <drm/drm_vblank.h>
@@ -1358,7 +1358,7 @@ nouveau_drm_init(void)
nouveau_display_options();
if (nouveau_modeset == -1) {
- if (vgacon_text_force())
+ if (drm_firmware_drivers_only())
nouveau_modeset = 0;
}
diff --git a/drivers/gpu/drm/nouveau/nouveau_fence.c b/drivers/gpu/drm/nouveau/nouveau_fence.c
index 05d0b3e..a3a04e0 100644
--- a/drivers/gpu/drm/nouveau/nouveau_fence.c
+++ b/drivers/gpu/drm/nouveau/nouveau_fence.c
@@ -339,68 +339,54 @@ nouveau_fence_wait(struct nouveau_fence *fence, bool lazy, bool intr)
}
int
-nouveau_fence_sync(struct nouveau_bo *nvbo, struct nouveau_channel *chan, bool exclusive, bool intr)
+nouveau_fence_sync(struct nouveau_bo *nvbo, struct nouveau_channel *chan,
+ bool exclusive, bool intr)
{
struct nouveau_fence_chan *fctx = chan->fence;
- struct dma_fence *fence;
struct dma_resv *resv = nvbo->bo.base.resv;
- struct dma_resv_list *fobj;
- struct nouveau_fence *f;
- int ret = 0, i;
+ int i, ret;
if (!exclusive) {
ret = dma_resv_reserve_shared(resv, 1);
-
if (ret)
return ret;
}
- fobj = dma_resv_shared_list(resv);
- fence = dma_resv_excl_fence(resv);
+ /* Waiting for the exclusive fence first causes performance regressions
+ * under some circumstances. So manually wait for the shared ones first.
+ */
+ for (i = 0; i < 2; ++i) {
+ struct dma_resv_iter cursor;
+ struct dma_fence *fence;
- if (fence) {
- struct nouveau_channel *prev = NULL;
- bool must_wait = true;
+ dma_resv_for_each_fence(&cursor, resv, exclusive, fence) {
+ struct nouveau_fence *f;
- f = nouveau_local_fence(fence, chan->drm);
- if (f) {
- rcu_read_lock();
- prev = rcu_dereference(f->channel);
- if (prev && (prev == chan || fctx->sync(f, prev, chan) == 0))
- must_wait = false;
- rcu_read_unlock();
- }
+ if (i == 0 && dma_resv_iter_is_exclusive(&cursor))
+ continue;
- if (must_wait)
+ f = nouveau_local_fence(fence, chan->drm);
+ if (f) {
+ struct nouveau_channel *prev;
+ bool must_wait = true;
+
+ rcu_read_lock();
+ prev = rcu_dereference(f->channel);
+ if (prev && (prev == chan ||
+ fctx->sync(f, prev, chan) == 0))
+ must_wait = false;
+ rcu_read_unlock();
+ if (!must_wait)
+ continue;
+ }
+
ret = dma_fence_wait(fence, intr);
-
- return ret;
+ if (ret)
+ return ret;
+ }
}
- if (!exclusive || !fobj)
- return ret;
-
- for (i = 0; i < fobj->shared_count && !ret; ++i) {
- struct nouveau_channel *prev = NULL;
- bool must_wait = true;
-
- fence = rcu_dereference_protected(fobj->shared[i],
- dma_resv_held(resv));
-
- f = nouveau_local_fence(fence, chan->drm);
- if (f) {
- rcu_read_lock();
- prev = rcu_dereference(f->channel);
- if (prev && (prev == chan || fctx->sync(f, prev, chan) == 0))
- must_wait = false;
- rcu_read_unlock();
- }
-
- if (must_wait)
- ret = dma_fence_wait(fence, intr);
- }
-
- return ret;
+ return 0;
}
void
diff --git a/drivers/gpu/drm/nouveau/nvkm/core/client.c b/drivers/gpu/drm/nouveau/nvkm/core/client.c
index ac67120..0c8c55c 100644
--- a/drivers/gpu/drm/nouveau/nvkm/core/client.c
+++ b/drivers/gpu/drm/nouveau/nvkm/core/client.c
@@ -60,7 +60,7 @@ nvkm_uclient_new(const struct nvkm_oclass *oclass, void *argv, u32 argc,
return 0;
}
-const struct nvkm_sclass
+static const struct nvkm_sclass
nvkm_uclient_sclass = {
.oclass = NVIF_CLASS_CLIENT,
.minver = 0,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/device/user.c b/drivers/gpu/drm/nouveau/nvkm/engine/device/user.c
index f28894f..113ddc1 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/device/user.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/device/user.c
@@ -161,8 +161,8 @@ nvkm_udevice_info(struct nvkm_udevice *udev, void *data, u32 size)
if (imem && args->v0.ram_size > 0)
args->v0.ram_user = args->v0.ram_user - imem->reserved;
- strncpy(args->v0.chip, device->chip->name, sizeof(args->v0.chip));
- strncpy(args->v0.name, device->name, sizeof(args->v0.name));
+ snprintf(args->v0.chip, sizeof(args->v0.chip), "%s", device->chip->name);
+ snprintf(args->v0.name, sizeof(args->v0.name), "%s", device->name);
return 0;
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/coregv100.c b/drivers/gpu/drm/nouveau/nvkm/engine/disp/coregv100.c
index e20a48f..448a515 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/coregv100.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/coregv100.c
@@ -106,6 +106,8 @@ gv100_disp_core_mthd_head = {
{ 0x20a4, 0x6820a4 },
{ 0x20a8, 0x6820a8 },
{ 0x20ac, 0x6820ac },
+ { 0x2180, 0x682180 },
+ { 0x2184, 0x682184 },
{ 0x218c, 0x68218c },
{ 0x2194, 0x682194 },
{ 0x2198, 0x682198 },
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/tu102.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/tu102.c
index e417044..260b197 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/tu102.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/tu102.c
@@ -49,7 +49,7 @@ tu102_fifo_runlist_commit(struct gk104_fifo *fifo, int runl,
/*XXX: how to wait? can you even wait? */
}
-const struct gk104_fifo_runlist_func
+static const struct gk104_fifo_runlist_func
tu102_fifo_runlist = {
.size = 16,
.cgrp = gv100_fifo_runlist_cgrp,
diff --git a/drivers/gpu/drm/nouveau/nvkm/falcon/base.c b/drivers/gpu/drm/nouveau/nvkm/falcon/base.c
index 262641a..c91130a 100644
--- a/drivers/gpu/drm/nouveau/nvkm/falcon/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/falcon/base.c
@@ -117,8 +117,12 @@ nvkm_falcon_disable(struct nvkm_falcon *falcon)
int
nvkm_falcon_reset(struct nvkm_falcon *falcon)
{
- nvkm_falcon_disable(falcon);
- return nvkm_falcon_enable(falcon);
+ if (!falcon->func->reset) {
+ nvkm_falcon_disable(falcon);
+ return nvkm_falcon_enable(falcon);
+ }
+
+ return falcon->func->reset(falcon);
}
int
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/bios/init.c b/drivers/gpu/drm/nouveau/nvkm/subdev/bios/init.c
index 9de74f4..1420794 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/bios/init.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/bios/init.c
@@ -401,7 +401,7 @@ init_table_(struct nvbios_init *init, u16 offset, const char *name)
#define init_macro_table(b) init_table_((b), 0x04, "macro table")
#define init_condition_table(b) init_table_((b), 0x06, "condition table")
#define init_io_condition_table(b) init_table_((b), 0x08, "io condition table")
-#define init_io_flag_condition_table(b) init_table_((b), 0x0a, "io flag conditon table")
+#define init_io_flag_condition_table(b) init_table_((b), 0x0a, "io flag condition table")
#define init_function_table(b) init_table_((b), 0x0c, "function table")
#define init_xlat_table(b) init_table_((b), 0x10, "xlat table");
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/mcp89.c b/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/mcp89.c
index fb90d47..a9cdf24 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/mcp89.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/mcp89.c
@@ -32,7 +32,6 @@ mcp89_devinit_disable(struct nvkm_devinit *init)
struct nvkm_device *device = init->subdev.device;
u32 r001540 = nvkm_rd32(device, 0x001540);
u32 r00154c = nvkm_rd32(device, 0x00154c);
- u64 disable = 0;
if (!(r001540 & 0x40000000)) {
nvkm_subdev_disable(device, NVKM_ENGINE_MSPDEC, 0);
@@ -48,7 +47,7 @@ mcp89_devinit_disable(struct nvkm_devinit *init)
if (!(r00154c & 0x00000200))
nvkm_subdev_disable(device, NVKM_ENGINE_CE, 0);
- return disable;
+ return 0;
}
static const struct nvkm_devinit_func
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/base.c b/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/base.c
index 2438287..455e95a 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/base.c
@@ -94,20 +94,13 @@ nvkm_pmu_fini(struct nvkm_subdev *subdev, bool suspend)
return 0;
}
-static int
+static void
nvkm_pmu_reset(struct nvkm_pmu *pmu)
{
struct nvkm_device *device = pmu->subdev.device;
if (!pmu->func->enabled(pmu))
- return 0;
-
- /* Inhibit interrupts, and wait for idle. */
- nvkm_wr32(device, 0x10a014, 0x0000ffff);
- nvkm_msec(device, 2000,
- if (!nvkm_rd32(device, 0x10a04c))
- break;
- );
+ return;
/* Reset. */
if (pmu->func->reset)
@@ -118,25 +111,37 @@ nvkm_pmu_reset(struct nvkm_pmu *pmu)
if (!(nvkm_rd32(device, 0x10a10c) & 0x00000006))
break;
);
-
- return 0;
}
static int
nvkm_pmu_preinit(struct nvkm_subdev *subdev)
{
struct nvkm_pmu *pmu = nvkm_pmu(subdev);
- return nvkm_pmu_reset(pmu);
+ nvkm_pmu_reset(pmu);
+ return 0;
}
static int
nvkm_pmu_init(struct nvkm_subdev *subdev)
{
struct nvkm_pmu *pmu = nvkm_pmu(subdev);
- int ret = nvkm_pmu_reset(pmu);
- if (ret == 0 && pmu->func->init)
- ret = pmu->func->init(pmu);
- return ret;
+ struct nvkm_device *device = pmu->subdev.device;
+
+ if (!pmu->func->init)
+ return 0;
+
+ if (pmu->func->enabled(pmu)) {
+ /* Inhibit interrupts, and wait for idle. */
+ nvkm_wr32(device, 0x10a014, 0x0000ffff);
+ nvkm_msec(device, 2000,
+ if (!nvkm_rd32(device, 0x10a04c))
+ break;
+ );
+
+ nvkm_pmu_reset(pmu);
+ }
+
+ return pmu->func->init(pmu);
}
static void *
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gm200.c b/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gm200.c
index 5968c76..40439e3 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gm200.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gm200.c
@@ -23,9 +23,38 @@
*/
#include "priv.h"
+static int
+gm200_pmu_flcn_reset(struct nvkm_falcon *falcon)
+{
+ struct nvkm_pmu *pmu = container_of(falcon, typeof(*pmu), falcon);
+
+ nvkm_falcon_wr32(falcon, 0x014, 0x0000ffff);
+ pmu->func->reset(pmu);
+ return nvkm_falcon_enable(falcon);
+}
+
+const struct nvkm_falcon_func
+gm200_pmu_flcn = {
+ .debug = 0xc08,
+ .fbif = 0xe00,
+ .load_imem = nvkm_falcon_v1_load_imem,
+ .load_dmem = nvkm_falcon_v1_load_dmem,
+ .read_dmem = nvkm_falcon_v1_read_dmem,
+ .bind_context = nvkm_falcon_v1_bind_context,
+ .wait_for_halt = nvkm_falcon_v1_wait_for_halt,
+ .clear_interrupt = nvkm_falcon_v1_clear_interrupt,
+ .set_start_addr = nvkm_falcon_v1_set_start_addr,
+ .start = nvkm_falcon_v1_start,
+ .enable = nvkm_falcon_v1_enable,
+ .disable = nvkm_falcon_v1_disable,
+ .reset = gm200_pmu_flcn_reset,
+ .cmdq = { 0x4a0, 0x4b0, 4 },
+ .msgq = { 0x4c8, 0x4cc, 0 },
+};
+
static const struct nvkm_pmu_func
gm200_pmu = {
- .flcn = >215_pmu_flcn,
+ .flcn = &gm200_pmu_flcn,
.enabled = gf100_pmu_enabled,
.reset = gf100_pmu_reset,
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gm20b.c b/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gm20b.c
index 1487069..e177221 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gm20b.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gm20b.c
@@ -211,7 +211,7 @@ gm20b_pmu_recv(struct nvkm_pmu *pmu)
static const struct nvkm_pmu_func
gm20b_pmu = {
- .flcn = >215_pmu_flcn,
+ .flcn = &gm200_pmu_flcn,
.enabled = gf100_pmu_enabled,
.intr = gt215_pmu_intr,
.recv = gm20b_pmu_recv,
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gp102.c b/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gp102.c
index 00da1b8..6bf7fc1 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gp102.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gp102.c
@@ -39,7 +39,7 @@ gp102_pmu_enabled(struct nvkm_pmu *pmu)
static const struct nvkm_pmu_func
gp102_pmu = {
- .flcn = >215_pmu_flcn,
+ .flcn = &gm200_pmu_flcn,
.enabled = gp102_pmu_enabled,
.reset = gp102_pmu_reset,
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gp10b.c b/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gp10b.c
index 461f722..ba1583bb 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gp10b.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gp10b.c
@@ -78,7 +78,7 @@ gp10b_pmu_acr = {
static const struct nvkm_pmu_func
gp10b_pmu = {
- .flcn = >215_pmu_flcn,
+ .flcn = &gm200_pmu_flcn,
.enabled = gf100_pmu_enabled,
.intr = gt215_pmu_intr,
.recv = gm20b_pmu_recv,
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/priv.h b/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/priv.h
index e7860d1..bcaade7 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/priv.h
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/priv.h
@@ -44,6 +44,8 @@ void gf100_pmu_reset(struct nvkm_pmu *);
void gk110_pmu_pgob(struct nvkm_pmu *, bool);
+extern const struct nvkm_falcon_func gm200_pmu_flcn;
+
void gm20b_pmu_acr_bld_patch(struct nvkm_acr *, u32, s64);
void gm20b_pmu_acr_bld_write(struct nvkm_acr *, u32, struct nvkm_acr_lsfw *);
int gm20b_pmu_acr_boot(struct nvkm_falcon *);
diff --git a/drivers/gpu/drm/omapdrm/Makefile b/drivers/gpu/drm/omapdrm/Makefile
index 21e8277..710b4e0 100644
--- a/drivers/gpu/drm/omapdrm/Makefile
+++ b/drivers/gpu/drm/omapdrm/Makefile
@@ -9,6 +9,7 @@
omap_debugfs.o \
omap_crtc.o \
omap_plane.o \
+ omap_overlay.o \
omap_encoder.o \
omap_fb.o \
omap_gem.o \
diff --git a/drivers/gpu/drm/omapdrm/dss/dispc.c b/drivers/gpu/drm/omapdrm/dss/dispc.c
index 5619420..c4de142 100644
--- a/drivers/gpu/drm/omapdrm/dss/dispc.c
+++ b/drivers/gpu/drm/omapdrm/dss/dispc.c
@@ -92,6 +92,8 @@ struct dispc_features {
u8 mgr_height_start;
u16 mgr_width_max;
u16 mgr_height_max;
+ u16 ovl_width_max;
+ u16 ovl_height_max;
unsigned long max_lcd_pclk;
unsigned long max_tv_pclk;
unsigned int max_downscale;
@@ -1279,8 +1281,8 @@ static u32 dispc_ovl_get_burst_size(struct dispc_device *dispc,
return dispc->feat->burst_size_unit * 8;
}
-static bool dispc_ovl_color_mode_supported(struct dispc_device *dispc,
- enum omap_plane_id plane, u32 fourcc)
+bool dispc_ovl_color_mode_supported(struct dispc_device *dispc,
+ enum omap_plane_id plane, u32 fourcc)
{
const u32 *modes;
unsigned int i;
@@ -2487,6 +2489,11 @@ static int dispc_ovl_calc_scaling_44xx(struct dispc_device *dispc,
return 0;
}
+enum omap_overlay_caps dispc_ovl_get_caps(struct dispc_device *dispc, enum omap_plane_id plane)
+{
+ return dispc->feat->overlay_caps[plane];
+}
+
#define DIV_FRAC(dividend, divisor) \
((dividend) * 100 / (divisor) - ((dividend) / (divisor) * 100))
@@ -2599,6 +2606,12 @@ static int dispc_ovl_calc_scaling(struct dispc_device *dispc,
return 0;
}
+void dispc_ovl_get_max_size(struct dispc_device *dispc, u16 *width, u16 *height)
+{
+ *width = dispc->feat->ovl_width_max;
+ *height = dispc->feat->ovl_height_max;
+}
+
static int dispc_ovl_setup_common(struct dispc_device *dispc,
enum omap_plane_id plane,
enum omap_overlay_caps caps,
@@ -4240,6 +4253,8 @@ static const struct dispc_features omap24xx_dispc_feats = {
.mgr_height_start = 26,
.mgr_width_max = 2048,
.mgr_height_max = 2048,
+ .ovl_width_max = 2048,
+ .ovl_height_max = 2048,
.max_lcd_pclk = 66500000,
.max_downscale = 2,
/*
@@ -4278,6 +4293,8 @@ static const struct dispc_features omap34xx_rev1_0_dispc_feats = {
.mgr_height_start = 26,
.mgr_width_max = 2048,
.mgr_height_max = 2048,
+ .ovl_width_max = 2048,
+ .ovl_height_max = 2048,
.max_lcd_pclk = 173000000,
.max_tv_pclk = 59000000,
.max_downscale = 4,
@@ -4313,6 +4330,8 @@ static const struct dispc_features omap34xx_rev3_0_dispc_feats = {
.mgr_height_start = 26,
.mgr_width_max = 2048,
.mgr_height_max = 2048,
+ .ovl_width_max = 2048,
+ .ovl_height_max = 2048,
.max_lcd_pclk = 173000000,
.max_tv_pclk = 59000000,
.max_downscale = 4,
@@ -4348,6 +4367,8 @@ static const struct dispc_features omap36xx_dispc_feats = {
.mgr_height_start = 26,
.mgr_width_max = 2048,
.mgr_height_max = 2048,
+ .ovl_width_max = 2048,
+ .ovl_height_max = 2048,
.max_lcd_pclk = 173000000,
.max_tv_pclk = 59000000,
.max_downscale = 4,
@@ -4383,6 +4404,8 @@ static const struct dispc_features am43xx_dispc_feats = {
.mgr_height_start = 26,
.mgr_width_max = 2048,
.mgr_height_max = 2048,
+ .ovl_width_max = 2048,
+ .ovl_height_max = 2048,
.max_lcd_pclk = 173000000,
.max_tv_pclk = 59000000,
.max_downscale = 4,
@@ -4418,6 +4441,8 @@ static const struct dispc_features omap44xx_dispc_feats = {
.mgr_height_start = 26,
.mgr_width_max = 2048,
.mgr_height_max = 2048,
+ .ovl_width_max = 2048,
+ .ovl_height_max = 2048,
.max_lcd_pclk = 170000000,
.max_tv_pclk = 185625000,
.max_downscale = 4,
@@ -4457,8 +4482,10 @@ static const struct dispc_features omap54xx_dispc_feats = {
.mgr_height_start = 27,
.mgr_width_max = 4096,
.mgr_height_max = 4096,
+ .ovl_width_max = 2048,
+ .ovl_height_max = 4096,
.max_lcd_pclk = 170000000,
- .max_tv_pclk = 186000000,
+ .max_tv_pclk = 192000000,
.max_downscale = 4,
.max_line_width = 2048,
.min_pcd = 1,
@@ -4725,7 +4752,6 @@ static int dispc_bind(struct device *dev, struct device *master, void *data)
struct dispc_device *dispc;
u32 rev;
int r = 0;
- struct resource *dispc_mem;
struct device_node *np = pdev->dev.of_node;
dispc = kzalloc(sizeof(*dispc), GFP_KERNEL);
@@ -4750,8 +4776,7 @@ static int dispc_bind(struct device *dev, struct device *master, void *data)
if (r)
goto err_free;
- dispc_mem = platform_get_resource(dispc->pdev, IORESOURCE_MEM, 0);
- dispc->base = devm_ioremap_resource(&pdev->dev, dispc_mem);
+ dispc->base = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(dispc->base)) {
r = PTR_ERR(dispc->base);
goto err_free;
@@ -4844,7 +4869,7 @@ static int dispc_remove(struct platform_device *pdev)
return 0;
}
-static int dispc_runtime_suspend(struct device *dev)
+static __maybe_unused int dispc_runtime_suspend(struct device *dev)
{
struct dispc_device *dispc = dev_get_drvdata(dev);
@@ -4859,7 +4884,7 @@ static int dispc_runtime_suspend(struct device *dev)
return 0;
}
-static int dispc_runtime_resume(struct device *dev)
+static __maybe_unused int dispc_runtime_resume(struct device *dev)
{
struct dispc_device *dispc = dev_get_drvdata(dev);
@@ -4887,8 +4912,7 @@ static int dispc_runtime_resume(struct device *dev)
}
static const struct dev_pm_ops dispc_pm_ops = {
- .runtime_suspend = dispc_runtime_suspend,
- .runtime_resume = dispc_runtime_resume,
+ SET_RUNTIME_PM_OPS(dispc_runtime_suspend, dispc_runtime_resume, NULL)
SET_LATE_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend, pm_runtime_force_resume)
};
diff --git a/drivers/gpu/drm/omapdrm/dss/dsi.c b/drivers/gpu/drm/omapdrm/dss/dsi.c
index 503b5d4..a684585 100644
--- a/drivers/gpu/drm/omapdrm/dss/dsi.c
+++ b/drivers/gpu/drm/omapdrm/dss/dsi.c
@@ -4884,7 +4884,6 @@ static int dsi_probe(struct platform_device *pdev)
struct device *dev = &pdev->dev;
struct dsi_data *dsi;
struct resource *dsi_mem;
- struct resource *res;
unsigned int i;
int r;
@@ -4921,13 +4920,11 @@ static int dsi_probe(struct platform_device *pdev)
if (IS_ERR(dsi->proto_base))
return PTR_ERR(dsi->proto_base);
- res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "phy");
- dsi->phy_base = devm_ioremap_resource(dev, res);
+ dsi->phy_base = devm_platform_ioremap_resource_byname(pdev, "phy");
if (IS_ERR(dsi->phy_base))
return PTR_ERR(dsi->phy_base);
- res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "pll");
- dsi->pll_base = devm_ioremap_resource(dev, res);
+ dsi->pll_base = devm_platform_ioremap_resource_byname(pdev, "pll");
if (IS_ERR(dsi->pll_base))
return PTR_ERR(dsi->pll_base);
@@ -5061,7 +5058,7 @@ static int dsi_remove(struct platform_device *pdev)
return 0;
}
-static int dsi_runtime_suspend(struct device *dev)
+static __maybe_unused int dsi_runtime_suspend(struct device *dev)
{
struct dsi_data *dsi = dev_get_drvdata(dev);
@@ -5074,7 +5071,7 @@ static int dsi_runtime_suspend(struct device *dev)
return 0;
}
-static int dsi_runtime_resume(struct device *dev)
+static __maybe_unused int dsi_runtime_resume(struct device *dev)
{
struct dsi_data *dsi = dev_get_drvdata(dev);
@@ -5086,8 +5083,7 @@ static int dsi_runtime_resume(struct device *dev)
}
static const struct dev_pm_ops dsi_pm_ops = {
- .runtime_suspend = dsi_runtime_suspend,
- .runtime_resume = dsi_runtime_resume,
+ SET_RUNTIME_PM_OPS(dsi_runtime_suspend, dsi_runtime_resume, NULL)
SET_LATE_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend, pm_runtime_force_resume)
};
diff --git a/drivers/gpu/drm/omapdrm/dss/dss.c b/drivers/gpu/drm/omapdrm/dss/dss.c
index d6a5862..69b3e15 100644
--- a/drivers/gpu/drm/omapdrm/dss/dss.c
+++ b/drivers/gpu/drm/omapdrm/dss/dss.c
@@ -1424,7 +1424,6 @@ static int dss_probe(struct platform_device *pdev)
const struct soc_device_attribute *soc;
struct dss_component_match_data cmatch;
struct component_match *match = NULL;
- struct resource *dss_mem;
struct dss_device *dss;
int r;
@@ -1452,8 +1451,7 @@ static int dss_probe(struct platform_device *pdev)
dss->feat = of_match_device(dss_of_match, &pdev->dev)->data;
/* Map I/O registers, get and setup clocks. */
- dss_mem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- dss->base = devm_ioremap_resource(&pdev->dev, dss_mem);
+ dss->base = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(dss->base)) {
r = PTR_ERR(dss->base);
goto err_free_dss;
@@ -1571,7 +1569,7 @@ static void dss_shutdown(struct platform_device *pdev)
DSSDBG("shutdown\n");
}
-static int dss_runtime_suspend(struct device *dev)
+static __maybe_unused int dss_runtime_suspend(struct device *dev)
{
struct dss_device *dss = dev_get_drvdata(dev);
@@ -1583,7 +1581,7 @@ static int dss_runtime_suspend(struct device *dev)
return 0;
}
-static int dss_runtime_resume(struct device *dev)
+static __maybe_unused int dss_runtime_resume(struct device *dev)
{
struct dss_device *dss = dev_get_drvdata(dev);
int r;
@@ -1606,8 +1604,7 @@ static int dss_runtime_resume(struct device *dev)
}
static const struct dev_pm_ops dss_pm_ops = {
- .runtime_suspend = dss_runtime_suspend,
- .runtime_resume = dss_runtime_resume,
+ SET_RUNTIME_PM_OPS(dss_runtime_suspend, dss_runtime_resume, NULL)
SET_LATE_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend, pm_runtime_force_resume)
};
diff --git a/drivers/gpu/drm/omapdrm/dss/dss.h b/drivers/gpu/drm/omapdrm/dss/dss.h
index a547527..4ff02fb 100644
--- a/drivers/gpu/drm/omapdrm/dss/dss.h
+++ b/drivers/gpu/drm/omapdrm/dss/dss.h
@@ -397,6 +397,11 @@ int dispc_get_num_mgrs(struct dispc_device *dispc);
const u32 *dispc_ovl_get_color_modes(struct dispc_device *dispc,
enum omap_plane_id plane);
+void dispc_ovl_get_max_size(struct dispc_device *dispc, u16 *width, u16 *height);
+bool dispc_ovl_color_mode_supported(struct dispc_device *dispc,
+ enum omap_plane_id plane, u32 fourcc);
+enum omap_overlay_caps dispc_ovl_get_caps(struct dispc_device *dispc, enum omap_plane_id plane);
+
u32 dispc_read_irqstatus(struct dispc_device *dispc);
void dispc_clear_irqstatus(struct dispc_device *dispc, u32 mask);
void dispc_write_irqenable(struct dispc_device *dispc, u32 mask);
diff --git a/drivers/gpu/drm/omapdrm/dss/hdmi4_cec.c b/drivers/gpu/drm/omapdrm/dss/hdmi4_cec.c
index 43592c1..852987e 100644
--- a/drivers/gpu/drm/omapdrm/dss/hdmi4_cec.c
+++ b/drivers/gpu/drm/omapdrm/dss/hdmi4_cec.c
@@ -1,3 +1,4 @@
+// SPDX-License-Identifier: GPL-2.0-only
/*
* HDMI CEC
*
@@ -10,19 +11,6 @@
* Heavily modified to use the linux CEC framework:
*
* Copyright 2016-2017 Cisco Systems, Inc. and/or its affiliates. All rights reserved.
- *
- * This program is free software; you may redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; version 2 of the License.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
- * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
- * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
- * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
- * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
- * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
- * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
- * SOFTWARE.
*/
#include <linux/kernel.h>
diff --git a/drivers/gpu/drm/omapdrm/dss/hdmi4_cec.h b/drivers/gpu/drm/omapdrm/dss/hdmi4_cec.h
index 0292337..1ca5b5c 100644
--- a/drivers/gpu/drm/omapdrm/dss/hdmi4_cec.h
+++ b/drivers/gpu/drm/omapdrm/dss/hdmi4_cec.h
@@ -1,20 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
/*
* HDMI header definition for OMAP4 HDMI CEC IP
*
* Copyright 2016-2017 Cisco Systems, Inc. and/or its affiliates. All rights reserved.
- *
- * This program is free software; you may redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; version 2 of the License.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
- * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
- * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
- * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
- * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
- * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
- * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
- * SOFTWARE.
*/
#ifndef _HDMI4_CEC_H_
diff --git a/drivers/gpu/drm/omapdrm/dss/hdmi4_core.c b/drivers/gpu/drm/omapdrm/dss/hdmi4_core.c
index 35faa7f..8720bf4 100644
--- a/drivers/gpu/drm/omapdrm/dss/hdmi4_core.c
+++ b/drivers/gpu/drm/omapdrm/dss/hdmi4_core.c
@@ -870,7 +870,6 @@ static const struct soc_device_attribute hdmi4_soc_devices[] = {
int hdmi4_core_init(struct platform_device *pdev, struct hdmi_core_data *core)
{
const struct hdmi4_features *features;
- struct resource *res;
const struct soc_device_attribute *soc;
soc = soc_device_match(hdmi4_soc_devices);
@@ -881,8 +880,7 @@ int hdmi4_core_init(struct platform_device *pdev, struct hdmi_core_data *core)
core->cts_swmode = features->cts_swmode;
core->audio_use_mclk = features->audio_use_mclk;
- res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "core");
- core->base = devm_ioremap_resource(&pdev->dev, res);
+ core->base = devm_platform_ioremap_resource_byname(pdev, "core");
if (IS_ERR(core->base))
return PTR_ERR(core->base);
diff --git a/drivers/gpu/drm/omapdrm/dss/hdmi5_core.c b/drivers/gpu/drm/omapdrm/dss/hdmi5_core.c
index 6cc2ad7..21564c3 100644
--- a/drivers/gpu/drm/omapdrm/dss/hdmi5_core.c
+++ b/drivers/gpu/drm/omapdrm/dss/hdmi5_core.c
@@ -872,10 +872,7 @@ int hdmi5_audio_config(struct hdmi_core_data *core, struct hdmi_wp_data *wp,
int hdmi5_core_init(struct platform_device *pdev, struct hdmi_core_data *core)
{
- struct resource *res;
-
- res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "core");
- core->base = devm_ioremap_resource(&pdev->dev, res);
+ core->base = devm_platform_ioremap_resource_byname(pdev, "core");
if (IS_ERR(core->base))
return PTR_ERR(core->base);
diff --git a/drivers/gpu/drm/omapdrm/dss/hdmi_phy.c b/drivers/gpu/drm/omapdrm/dss/hdmi_phy.c
index 5dc200f..060e8f7 100644
--- a/drivers/gpu/drm/omapdrm/dss/hdmi_phy.c
+++ b/drivers/gpu/drm/omapdrm/dss/hdmi_phy.c
@@ -182,15 +182,12 @@ static const struct hdmi_phy_features omap54xx_phy_feats = {
int hdmi_phy_init(struct platform_device *pdev, struct hdmi_phy_data *phy,
unsigned int version)
{
- struct resource *res;
-
if (version == 4)
phy->features = &omap44xx_phy_feats;
else
phy->features = &omap54xx_phy_feats;
- res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "phy");
- phy->base = devm_ioremap_resource(&pdev->dev, res);
+ phy->base = devm_platform_ioremap_resource_byname(pdev, "phy");
if (IS_ERR(phy->base))
return PTR_ERR(phy->base);
diff --git a/drivers/gpu/drm/omapdrm/dss/hdmi_pll.c b/drivers/gpu/drm/omapdrm/dss/hdmi_pll.c
index 13bf649..eea7192 100644
--- a/drivers/gpu/drm/omapdrm/dss/hdmi_pll.c
+++ b/drivers/gpu/drm/omapdrm/dss/hdmi_pll.c
@@ -162,13 +162,11 @@ int hdmi_pll_init(struct dss_device *dss, struct platform_device *pdev,
struct hdmi_pll_data *pll, struct hdmi_wp_data *wp)
{
int r;
- struct resource *res;
pll->pdev = pdev;
pll->wp = wp;
- res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "pll");
- pll->base = devm_ioremap_resource(&pdev->dev, res);
+ pll->base = devm_platform_ioremap_resource_byname(pdev, "pll");
if (IS_ERR(pll->base))
return PTR_ERR(pll->base);
diff --git a/drivers/gpu/drm/omapdrm/dss/venc.c b/drivers/gpu/drm/omapdrm/dss/venc.c
index e522c17..4480b69 100644
--- a/drivers/gpu/drm/omapdrm/dss/venc.c
+++ b/drivers/gpu/drm/omapdrm/dss/venc.c
@@ -806,7 +806,6 @@ static const struct soc_device_attribute venc_soc_devices[] = {
static int venc_probe(struct platform_device *pdev)
{
struct venc_device *venc;
- struct resource *venc_mem;
int r;
venc = kzalloc(sizeof(*venc), GFP_KERNEL);
@@ -823,8 +822,7 @@ static int venc_probe(struct platform_device *pdev)
venc->config = &venc_config_pal_trm;
- venc_mem = platform_get_resource(venc->pdev, IORESOURCE_MEM, 0);
- venc->base = devm_ioremap_resource(&pdev->dev, venc_mem);
+ venc->base = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(venc->base)) {
r = PTR_ERR(venc->base);
goto err_free;
@@ -881,7 +879,7 @@ static int venc_remove(struct platform_device *pdev)
return 0;
}
-static int venc_runtime_suspend(struct device *dev)
+static __maybe_unused int venc_runtime_suspend(struct device *dev)
{
struct venc_device *venc = dev_get_drvdata(dev);
@@ -891,7 +889,7 @@ static int venc_runtime_suspend(struct device *dev)
return 0;
}
-static int venc_runtime_resume(struct device *dev)
+static __maybe_unused int venc_runtime_resume(struct device *dev)
{
struct venc_device *venc = dev_get_drvdata(dev);
@@ -902,8 +900,7 @@ static int venc_runtime_resume(struct device *dev)
}
static const struct dev_pm_ops venc_pm_ops = {
- .runtime_suspend = venc_runtime_suspend,
- .runtime_resume = venc_runtime_resume,
+ SET_RUNTIME_PM_OPS(venc_runtime_suspend, venc_runtime_resume, NULL)
SET_LATE_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend, pm_runtime_force_resume)
};
diff --git a/drivers/gpu/drm/omapdrm/dss/video-pll.c b/drivers/gpu/drm/omapdrm/dss/video-pll.c
index b72c3ff..b6b5204 100644
--- a/drivers/gpu/drm/omapdrm/dss/video-pll.c
+++ b/drivers/gpu/drm/omapdrm/dss/video-pll.c
@@ -137,7 +137,6 @@ struct dss_pll *dss_video_pll_init(struct dss_device *dss,
const char * const clkctrl_name[] = { "pll1_clkctrl", "pll2_clkctrl" };
const char * const clkin_name[] = { "video1_clk", "video2_clk" };
- struct resource *res;
struct dss_video_pll *vpll;
void __iomem *pll_base, *clkctrl_base;
struct clk *clk;
@@ -146,16 +145,13 @@ struct dss_pll *dss_video_pll_init(struct dss_device *dss,
/* PLL CONTROL */
- res = platform_get_resource_byname(pdev, IORESOURCE_MEM, reg_name[id]);
- pll_base = devm_ioremap_resource(&pdev->dev, res);
+ pll_base = devm_platform_ioremap_resource_byname(pdev, reg_name[id]);
if (IS_ERR(pll_base))
return ERR_CAST(pll_base);
/* CLOCK CONTROL */
- res = platform_get_resource_byname(pdev, IORESOURCE_MEM,
- clkctrl_name[id]);
- clkctrl_base = devm_ioremap_resource(&pdev->dev, res);
+ clkctrl_base = devm_platform_ioremap_resource_byname(pdev, clkctrl_name[id]);
if (IS_ERR(clkctrl_base))
return ERR_CAST(clkctrl_base);
diff --git a/drivers/gpu/drm/omapdrm/omap_dmm_priv.h b/drivers/gpu/drm/omapdrm/omap_dmm_priv.h
index 58a8239..2288ed5 100644
--- a/drivers/gpu/drm/omapdrm/omap_dmm_priv.h
+++ b/drivers/gpu/drm/omapdrm/omap_dmm_priv.h
@@ -1,16 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
/*
* Copyright (C) 2011 Texas Instruments Incorporated - https://www.ti.com/
* Author: Rob Clark <rob@ti.com>
* Andy Gross <andy.gross@ti.com>
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation version 2.
- *
- * This program is distributed "as is" WITHOUT ANY WARRANTY of any
- * kind, whether express or implied; without even the implied warranty
- * of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
*/
#ifndef OMAP_DMM_PRIV_H
diff --git a/drivers/gpu/drm/omapdrm/omap_dmm_tiler.c b/drivers/gpu/drm/omapdrm/omap_dmm_tiler.c
index ed770ca..852e78a 100644
--- a/drivers/gpu/drm/omapdrm/omap_dmm_tiler.c
+++ b/drivers/gpu/drm/omapdrm/omap_dmm_tiler.c
@@ -1,18 +1,10 @@
+// SPDX-License-Identifier: GPL-2.0-only
/*
* DMM IOMMU driver support functions for TI OMAP processors.
*
* Copyright (C) 2011 Texas Instruments Incorporated - https://www.ti.com/
* Author: Rob Clark <rob@ti.com>
* Andy Gross <andy.gross@ti.com>
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation version 2.
- *
- * This program is distributed "as is" WITHOUT ANY WARRANTY of any
- * kind, whether express or implied; without even the implied warranty
- * of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
*/
#include <linux/completion.h>
diff --git a/drivers/gpu/drm/omapdrm/omap_dmm_tiler.h b/drivers/gpu/drm/omapdrm/omap_dmm_tiler.h
index 2f8918f..87a32b3 100644
--- a/drivers/gpu/drm/omapdrm/omap_dmm_tiler.h
+++ b/drivers/gpu/drm/omapdrm/omap_dmm_tiler.h
@@ -1,16 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
/*
* Copyright (C) 2011 Texas Instruments Incorporated - https://www.ti.com/
* Author: Rob Clark <rob@ti.com>
* Andy Gross <andy.gross@ti.com>
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation version 2.
- *
- * This program is distributed "as is" WITHOUT ANY WARRANTY of any
- * kind, whether express or implied; without even the implied warranty
- * of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
*/
#ifndef OMAP_DMM_TILER_H
#define OMAP_DMM_TILER_H
diff --git a/drivers/gpu/drm/omapdrm/omap_drv.c b/drivers/gpu/drm/omapdrm/omap_drv.c
index c05d397..2720a58 100644
--- a/drivers/gpu/drm/omapdrm/omap_drv.c
+++ b/drivers/gpu/drm/omapdrm/omap_drv.c
@@ -117,6 +117,102 @@ static void omap_atomic_commit_tail(struct drm_atomic_state *old_state)
dispc_runtime_put(priv->dispc);
}
+static int drm_atomic_state_normalized_zpos_cmp(const void *a, const void *b)
+{
+ const struct drm_plane_state *sa = *(struct drm_plane_state **)a;
+ const struct drm_plane_state *sb = *(struct drm_plane_state **)b;
+
+ if (sa->normalized_zpos != sb->normalized_zpos)
+ return sa->normalized_zpos - sb->normalized_zpos;
+ else
+ return sa->plane->base.id - sb->plane->base.id;
+}
+
+/*
+ * This replaces the drm_atomic_normalize_zpos to handle the dual overlay case.
+ *
+ * Since both halves need to be 'appear' side by side the zpos is
+ * recalculated when dealing with dual overlay cases so that the other
+ * planes zpos is consistent.
+ */
+static int omap_atomic_update_normalize_zpos(struct drm_device *dev,
+ struct drm_atomic_state *state)
+{
+ struct drm_crtc *crtc;
+ struct drm_crtc_state *old_state, *new_state;
+ struct drm_plane *plane;
+ int c, i, n, inc;
+ int total_planes = dev->mode_config.num_total_plane;
+ struct drm_plane_state **states;
+ int ret = 0;
+
+ states = kmalloc_array(total_planes, sizeof(*states), GFP_KERNEL);
+ if (!states)
+ return -ENOMEM;
+
+ for_each_oldnew_crtc_in_state(state, crtc, old_state, new_state, c) {
+ if (old_state->plane_mask == new_state->plane_mask &&
+ !new_state->zpos_changed)
+ continue;
+
+ /* Reset plane increment and index value for every crtc */
+ n = 0;
+
+ /*
+ * Normalization process might create new states for planes
+ * which normalized_zpos has to be recalculated.
+ */
+ drm_for_each_plane_mask(plane, dev, new_state->plane_mask) {
+ struct drm_plane_state *plane_state =
+ drm_atomic_get_plane_state(new_state->state,
+ plane);
+ if (IS_ERR(plane_state)) {
+ ret = PTR_ERR(plane_state);
+ goto done;
+ }
+ states[n++] = plane_state;
+ }
+
+ sort(states, n, sizeof(*states),
+ drm_atomic_state_normalized_zpos_cmp, NULL);
+
+ for (i = 0, inc = 0; i < n; i++) {
+ plane = states[i]->plane;
+
+ states[i]->normalized_zpos = i + inc;
+ DRM_DEBUG_ATOMIC("[PLANE:%d:%s] updated normalized zpos value %d\n",
+ plane->base.id, plane->name,
+ states[i]->normalized_zpos);
+
+ if (is_omap_plane_dual_overlay(states[i]))
+ inc++;
+ }
+ new_state->zpos_changed = true;
+ }
+
+done:
+ kfree(states);
+ return ret;
+}
+
+static int omap_atomic_check(struct drm_device *dev,
+ struct drm_atomic_state *state)
+{
+ int ret;
+
+ ret = drm_atomic_helper_check(dev, state);
+ if (ret)
+ return ret;
+
+ if (dev->mode_config.normalize_zpos) {
+ ret = omap_atomic_update_normalize_zpos(dev, state);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
static const struct drm_mode_config_helper_funcs omap_mode_config_helper_funcs = {
.atomic_commit_tail = omap_atomic_commit_tail,
};
@@ -124,10 +220,86 @@ static const struct drm_mode_config_helper_funcs omap_mode_config_helper_funcs =
static const struct drm_mode_config_funcs omap_mode_config_funcs = {
.fb_create = omap_framebuffer_create,
.output_poll_changed = drm_fb_helper_output_poll_changed,
- .atomic_check = drm_atomic_helper_check,
+ .atomic_check = omap_atomic_check,
.atomic_commit = drm_atomic_helper_commit,
};
+/* Global/shared object state funcs */
+
+/*
+ * This is a helper that returns the private state currently in operation.
+ * Note that this would return the "old_state" if called in the atomic check
+ * path, and the "new_state" after the atomic swap has been done.
+ */
+struct omap_global_state *
+omap_get_existing_global_state(struct omap_drm_private *priv)
+{
+ return to_omap_global_state(priv->glob_obj.state);
+}
+
+/*
+ * This acquires the modeset lock set aside for global state, creates
+ * a new duplicated private object state.
+ */
+struct omap_global_state *__must_check
+omap_get_global_state(struct drm_atomic_state *s)
+{
+ struct omap_drm_private *priv = s->dev->dev_private;
+ struct drm_private_state *priv_state;
+
+ priv_state = drm_atomic_get_private_obj_state(s, &priv->glob_obj);
+ if (IS_ERR(priv_state))
+ return ERR_CAST(priv_state);
+
+ return to_omap_global_state(priv_state);
+}
+
+static struct drm_private_state *
+omap_global_duplicate_state(struct drm_private_obj *obj)
+{
+ struct omap_global_state *state;
+
+ state = kmemdup(obj->state, sizeof(*state), GFP_KERNEL);
+ if (!state)
+ return NULL;
+
+ __drm_atomic_helper_private_obj_duplicate_state(obj, &state->base);
+
+ return &state->base;
+}
+
+static void omap_global_destroy_state(struct drm_private_obj *obj,
+ struct drm_private_state *state)
+{
+ struct omap_global_state *omap_state = to_omap_global_state(state);
+
+ kfree(omap_state);
+}
+
+static const struct drm_private_state_funcs omap_global_state_funcs = {
+ .atomic_duplicate_state = omap_global_duplicate_state,
+ .atomic_destroy_state = omap_global_destroy_state,
+};
+
+static int omap_global_obj_init(struct drm_device *dev)
+{
+ struct omap_drm_private *priv = dev->dev_private;
+ struct omap_global_state *state;
+
+ state = kzalloc(sizeof(*state), GFP_KERNEL);
+ if (!state)
+ return -ENOMEM;
+
+ drm_atomic_private_obj_init(dev, &priv->glob_obj, &state->base,
+ &omap_global_state_funcs);
+ return 0;
+}
+
+static void omap_global_obj_fini(struct omap_drm_private *priv)
+{
+ drm_atomic_private_obj_fini(&priv->glob_obj);
+}
+
static void omap_disconnect_pipelines(struct drm_device *ddev)
{
struct omap_drm_private *priv = ddev->dev_private;
@@ -231,8 +403,6 @@ static int omap_modeset_init(struct drm_device *dev)
if (!omapdss_stack_is_ready())
return -EPROBE_DEFER;
- drm_mode_config_init(dev);
-
ret = omap_modeset_init_properties(dev);
if (ret < 0)
return ret;
@@ -583,10 +753,20 @@ static int omapdrm_init(struct omap_drm_private *priv, struct device *dev)
omap_gem_init(ddev);
+ drm_mode_config_init(ddev);
+
+ ret = omap_global_obj_init(ddev);
+ if (ret)
+ goto err_gem_deinit;
+
+ ret = omap_hwoverlays_init(priv);
+ if (ret)
+ goto err_free_priv_obj;
+
ret = omap_modeset_init(ddev);
if (ret) {
dev_err(priv->dev, "omap_modeset_init failed: ret=%d\n", ret);
- goto err_gem_deinit;
+ goto err_free_overlays;
}
/* Initialize vblank handling, start with all CRTCs disabled. */
@@ -618,7 +798,12 @@ static int omapdrm_init(struct omap_drm_private *priv, struct device *dev)
omap_fbdev_fini(ddev);
err_cleanup_modeset:
omap_modeset_fini(ddev);
+err_free_overlays:
+ omap_hwoverlays_destroy(priv);
+err_free_priv_obj:
+ omap_global_obj_fini(priv);
err_gem_deinit:
+ drm_mode_config_cleanup(ddev);
omap_gem_deinit(ddev);
destroy_workqueue(priv->wq);
omap_disconnect_pipelines(ddev);
@@ -642,6 +827,9 @@ static void omapdrm_cleanup(struct omap_drm_private *priv)
drm_atomic_helper_shutdown(ddev);
omap_modeset_fini(ddev);
+ omap_hwoverlays_destroy(priv);
+ omap_global_obj_fini(priv);
+ drm_mode_config_cleanup(ddev);
omap_gem_deinit(ddev);
destroy_workqueue(priv->wq);
diff --git a/drivers/gpu/drm/omapdrm/omap_drv.h b/drivers/gpu/drm/omapdrm/omap_drv.h
index 591d4c2..825960f 100644
--- a/drivers/gpu/drm/omapdrm/omap_drv.h
+++ b/drivers/gpu/drm/omapdrm/omap_drv.h
@@ -14,6 +14,7 @@
#include "dss/omapdss.h"
#include "dss/dss.h"
+#include <drm/drm_atomic.h>
#include <drm/drm_gem.h>
#include <drm/omap_drm.h>
@@ -24,6 +25,7 @@
#include "omap_gem.h"
#include "omap_irq.h"
#include "omap_plane.h"
+#include "omap_overlay.h"
#define DBG(fmt, ...) DRM_DEBUG_DRIVER(fmt"\n", ##__VA_ARGS__)
#define VERB(fmt, ...) if (0) DRM_DEBUG_DRIVER(fmt, ##__VA_ARGS__) /* verbose debug */
@@ -40,6 +42,19 @@ struct omap_drm_pipeline {
unsigned int alias_id;
};
+/*
+ * Global private object state for tracking resources that are shared across
+ * multiple kms objects (planes/crtcs/etc).
+ */
+#define to_omap_global_state(x) container_of(x, struct omap_global_state, base)
+
+struct omap_global_state {
+ struct drm_private_state base;
+
+ /* global atomic state of assignment between overlays and planes */
+ struct drm_plane *hwoverlay_to_plane[8];
+};
+
struct omap_drm_private {
struct drm_device *ddev;
struct device *dev;
@@ -57,6 +72,11 @@ struct omap_drm_private {
unsigned int num_planes;
struct drm_plane *planes[8];
+ unsigned int num_ovls;
+ struct omap_hw_overlay *overlays[8];
+
+ struct drm_private_obj glob_obj;
+
struct drm_fb_helper *fbdev;
struct workqueue_struct *wq;
@@ -85,4 +105,8 @@ struct omap_drm_private {
void omap_debugfs_init(struct drm_minor *minor);
+struct omap_global_state * __must_check omap_get_global_state(struct drm_atomic_state *s);
+
+struct omap_global_state *omap_get_existing_global_state(struct omap_drm_private *priv);
+
#endif /* __OMAPDRM_DRV_H__ */
diff --git a/drivers/gpu/drm/omapdrm/omap_fb.c b/drivers/gpu/drm/omapdrm/omap_fb.c
index 190afc5..895e66b 100644
--- a/drivers/gpu/drm/omapdrm/omap_fb.c
+++ b/drivers/gpu/drm/omapdrm/omap_fb.c
@@ -131,7 +131,9 @@ static u32 drm_rotation_to_tiler(unsigned int drm_rot)
/* update ovl info for scanout, handles cases of multi-planar fb's, etc.
*/
void omap_framebuffer_update_scanout(struct drm_framebuffer *fb,
- struct drm_plane_state *state, struct omap_overlay_info *info)
+ struct drm_plane_state *state,
+ struct omap_overlay_info *info,
+ struct omap_overlay_info *r_info)
{
struct omap_framebuffer *omap_fb = to_omap_framebuffer(fb);
const struct drm_format_info *format = omap_fb->format;
@@ -218,6 +220,35 @@ void omap_framebuffer_update_scanout(struct drm_framebuffer *fb,
} else {
info->p_uv_addr = 0;
}
+
+ if (r_info) {
+ info->width /= 2;
+ info->out_width /= 2;
+
+ *r_info = *info;
+
+ if (fb->format->is_yuv) {
+ if (info->width & 1) {
+ info->width++;
+ r_info->width--;
+ }
+
+ if (info->out_width & 1) {
+ info->out_width++;
+ r_info->out_width--;
+ }
+ }
+
+ r_info->pos_x = info->pos_x + info->out_width;
+
+ r_info->paddr = get_linear_addr(fb, format, 0,
+ x + info->width, y);
+ if (fb->format->format == DRM_FORMAT_NV12) {
+ r_info->p_uv_addr =
+ get_linear_addr(fb, format, 1,
+ x + info->width, y);
+ }
+ }
}
/* pin, prepare for scanout: */
diff --git a/drivers/gpu/drm/omapdrm/omap_fb.h b/drivers/gpu/drm/omapdrm/omap_fb.h
index c0e19ae..b75f0b5 100644
--- a/drivers/gpu/drm/omapdrm/omap_fb.h
+++ b/drivers/gpu/drm/omapdrm/omap_fb.h
@@ -26,7 +26,9 @@ struct drm_framebuffer *omap_framebuffer_init(struct drm_device *dev,
int omap_framebuffer_pin(struct drm_framebuffer *fb);
void omap_framebuffer_unpin(struct drm_framebuffer *fb);
void omap_framebuffer_update_scanout(struct drm_framebuffer *fb,
- struct drm_plane_state *state, struct omap_overlay_info *info);
+ struct drm_plane_state *state,
+ struct omap_overlay_info *info,
+ struct omap_overlay_info *r_info);
bool omap_framebuffer_supports_rotation(struct drm_framebuffer *fb);
void omap_framebuffer_describe(struct drm_framebuffer *fb, struct seq_file *m);
diff --git a/drivers/gpu/drm/omapdrm/omap_gem.c b/drivers/gpu/drm/omapdrm/omap_gem.c
index 38af619..b0fa174 100644
--- a/drivers/gpu/drm/omapdrm/omap_gem.c
+++ b/drivers/gpu/drm/omapdrm/omap_gem.c
@@ -789,7 +789,7 @@ int omap_gem_pin(struct drm_gem_object *obj, dma_addr_t *dma_addr)
if (omap_obj->flags & OMAP_BO_TILED_MASK) {
block = tiler_reserve_2d(fmt,
omap_obj->width,
- omap_obj->height, 0);
+ omap_obj->height, PAGE_SIZE);
} else {
block = tiler_reserve_1d(obj->size);
}
@@ -851,6 +851,11 @@ static void omap_gem_unpin_locked(struct drm_gem_object *obj)
return;
if (refcount_dec_and_test(&omap_obj->dma_addr_cnt)) {
+ if (omap_obj->sgt) {
+ sg_free_table(omap_obj->sgt);
+ kfree(omap_obj->sgt);
+ omap_obj->sgt = NULL;
+ }
ret = tiler_unpin(omap_obj->block);
if (ret) {
dev_err(obj->dev->dev,
@@ -963,6 +968,78 @@ int omap_gem_put_pages(struct drm_gem_object *obj)
return 0;
}
+struct sg_table *omap_gem_get_sg(struct drm_gem_object *obj)
+{
+ struct omap_gem_object *omap_obj = to_omap_bo(obj);
+ dma_addr_t addr;
+ struct sg_table *sgt;
+ struct scatterlist *sg;
+ unsigned int count, len, stride, i;
+ int ret;
+
+ ret = omap_gem_pin(obj, &addr);
+ if (ret)
+ return ERR_PTR(ret);
+
+ mutex_lock(&omap_obj->lock);
+
+ sgt = omap_obj->sgt;
+ if (sgt)
+ goto out;
+
+ sgt = kzalloc(sizeof(*sgt), GFP_KERNEL);
+ if (!sgt) {
+ ret = -ENOMEM;
+ goto err_unpin;
+ }
+
+ if (omap_obj->flags & OMAP_BO_TILED_MASK) {
+ enum tiler_fmt fmt = gem2fmt(omap_obj->flags);
+
+ len = omap_obj->width << (int)fmt;
+ count = omap_obj->height;
+ stride = tiler_stride(fmt, 0);
+ } else {
+ len = obj->size;
+ count = 1;
+ stride = 0;
+ }
+
+ ret = sg_alloc_table(sgt, count, GFP_KERNEL);
+ if (ret)
+ goto err_free;
+
+ for_each_sg(sgt->sgl, sg, count, i) {
+ sg_set_page(sg, phys_to_page(addr), len, offset_in_page(addr));
+ sg_dma_address(sg) = addr;
+ sg_dma_len(sg) = len;
+
+ addr += stride;
+ }
+
+ omap_obj->sgt = sgt;
+out:
+ mutex_unlock(&omap_obj->lock);
+ return sgt;
+
+err_free:
+ kfree(sgt);
+err_unpin:
+ mutex_unlock(&omap_obj->lock);
+ omap_gem_unpin(obj);
+ return ERR_PTR(ret);
+}
+
+void omap_gem_put_sg(struct drm_gem_object *obj, struct sg_table *sgt)
+{
+ struct omap_gem_object *omap_obj = to_omap_bo(obj);
+
+ if (WARN_ON(omap_obj->sgt != sgt))
+ return;
+
+ omap_gem_unpin(obj);
+}
+
#ifdef CONFIG_DRM_FBDEV_EMULATION
/*
* Get kernel virtual address for CPU access.. this more or less only
diff --git a/drivers/gpu/drm/omapdrm/omap_gem.h b/drivers/gpu/drm/omapdrm/omap_gem.h
index eda9b48..19209e3 100644
--- a/drivers/gpu/drm/omapdrm/omap_gem.h
+++ b/drivers/gpu/drm/omapdrm/omap_gem.h
@@ -82,5 +82,7 @@ u32 omap_gem_flags(struct drm_gem_object *obj);
int omap_gem_rotated_dma_addr(struct drm_gem_object *obj, u32 orient,
int x, int y, dma_addr_t *dma_addr);
int omap_gem_tiled_stride(struct drm_gem_object *obj, u32 orient);
+struct sg_table *omap_gem_get_sg(struct drm_gem_object *obj);
+void omap_gem_put_sg(struct drm_gem_object *obj, struct sg_table *sgt);
#endif /* __OMAPDRM_GEM_H__ */
diff --git a/drivers/gpu/drm/omapdrm/omap_gem_dmabuf.c b/drivers/gpu/drm/omapdrm/omap_gem_dmabuf.c
index 809f86cf..57af3d9 100644
--- a/drivers/gpu/drm/omapdrm/omap_gem_dmabuf.c
+++ b/drivers/gpu/drm/omapdrm/omap_gem_dmabuf.c
@@ -23,45 +23,21 @@ static struct sg_table *omap_gem_map_dma_buf(
{
struct drm_gem_object *obj = attachment->dmabuf->priv;
struct sg_table *sg;
- dma_addr_t dma_addr;
- int ret;
-
- sg = kzalloc(sizeof(*sg), GFP_KERNEL);
- if (!sg)
- return ERR_PTR(-ENOMEM);
-
- /* camera, etc, need physically contiguous.. but we need a
- * better way to know this..
- */
- ret = omap_gem_pin(obj, &dma_addr);
- if (ret)
- goto out;
-
- ret = sg_alloc_table(sg, 1, GFP_KERNEL);
- if (ret)
- goto out;
-
- sg_init_table(sg->sgl, 1);
- sg_dma_len(sg->sgl) = obj->size;
- sg_set_page(sg->sgl, pfn_to_page(PFN_DOWN(dma_addr)), obj->size, 0);
- sg_dma_address(sg->sgl) = dma_addr;
+ sg = omap_gem_get_sg(obj);
+ if (IS_ERR(sg))
+ return sg;
/* this must be after omap_gem_pin() to ensure we have pages attached */
omap_gem_dma_sync_buffer(obj, dir);
return sg;
-out:
- kfree(sg);
- return ERR_PTR(ret);
}
static void omap_gem_unmap_dma_buf(struct dma_buf_attachment *attachment,
struct sg_table *sg, enum dma_data_direction dir)
{
struct drm_gem_object *obj = attachment->dmabuf->priv;
- omap_gem_unpin(obj);
- sg_free_table(sg);
- kfree(sg);
+ omap_gem_put_sg(obj, sg);
}
static int omap_gem_dmabuf_begin_cpu_access(struct dma_buf *buffer,
@@ -114,7 +90,7 @@ struct dma_buf *omap_gem_prime_export(struct drm_gem_object *obj, int flags)
DEFINE_DMA_BUF_EXPORT_INFO(exp_info);
exp_info.ops = &omap_dmabuf_ops;
- exp_info.size = obj->size;
+ exp_info.size = omap_gem_mmap_size(obj);
exp_info.flags = flags;
exp_info.priv = obj;
diff --git a/drivers/gpu/drm/omapdrm/omap_overlay.c b/drivers/gpu/drm/omapdrm/omap_overlay.c
new file mode 100644
index 0000000..10730c9
--- /dev/null
+++ b/drivers/gpu/drm/omapdrm/omap_overlay.c
@@ -0,0 +1,212 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2018 Texas Instruments Incorporated - http://www.ti.com/
+ * Author: Benoit Parrot <bparrot@ti.com>
+ */
+
+#include <drm/drm_atomic.h>
+#include <drm/drm_atomic_helper.h>
+#include <drm/drm_plane_helper.h>
+
+#include "omap_dmm_tiler.h"
+#include "omap_drv.h"
+
+/*
+ * overlay funcs
+ */
+static const char * const overlay_id_to_name[] = {
+ [OMAP_DSS_GFX] = "gfx",
+ [OMAP_DSS_VIDEO1] = "vid1",
+ [OMAP_DSS_VIDEO2] = "vid2",
+ [OMAP_DSS_VIDEO3] = "vid3",
+};
+
+/*
+ * Find a free overlay with the required caps and supported fourcc
+ */
+static struct omap_hw_overlay *
+omap_plane_find_free_overlay(struct drm_device *dev, struct drm_plane *hwoverlay_to_plane[],
+ u32 caps, u32 fourcc)
+{
+ struct omap_drm_private *priv = dev->dev_private;
+ int i;
+
+ DBG("caps: %x fourcc: %x", caps, fourcc);
+
+ for (i = 0; i < priv->num_ovls; i++) {
+ struct omap_hw_overlay *cur = priv->overlays[i];
+
+ DBG("%d: id: %d cur->caps: %x",
+ cur->idx, cur->id, cur->caps);
+
+ /* skip if already in-use */
+ if (hwoverlay_to_plane[cur->idx])
+ continue;
+
+ /* skip if doesn't support some required caps: */
+ if (caps & ~cur->caps)
+ continue;
+
+ /* check supported format */
+ if (!dispc_ovl_color_mode_supported(priv->dispc,
+ cur->id, fourcc))
+ continue;
+
+ return cur;
+ }
+
+ DBG("no match");
+ return NULL;
+}
+
+/*
+ * Assign a new overlay to a plane with the required caps and supported fourcc
+ * If a plane need a new overlay, the previous one should have been released
+ * with omap_overlay_release()
+ * This should be called from the plane atomic_check() in order to prepare the
+ * next global overlay_map to be enabled when atomic transaction is valid.
+ */
+int omap_overlay_assign(struct drm_atomic_state *s, struct drm_plane *plane,
+ u32 caps, u32 fourcc, struct omap_hw_overlay **overlay,
+ struct omap_hw_overlay **r_overlay)
+{
+ /* Get the global state of the current atomic transaction */
+ struct omap_global_state *state = omap_get_global_state(s);
+ struct drm_plane **overlay_map = state->hwoverlay_to_plane;
+ struct omap_hw_overlay *ovl, *r_ovl;
+
+ ovl = omap_plane_find_free_overlay(s->dev, overlay_map, caps, fourcc);
+ if (!ovl)
+ return -ENOMEM;
+
+ overlay_map[ovl->idx] = plane;
+ *overlay = ovl;
+
+ if (r_overlay) {
+ r_ovl = omap_plane_find_free_overlay(s->dev, overlay_map,
+ caps, fourcc);
+ if (!r_ovl) {
+ overlay_map[r_ovl->idx] = NULL;
+ *overlay = NULL;
+ return -ENOMEM;
+ }
+
+ overlay_map[r_ovl->idx] = plane;
+ *r_overlay = r_ovl;
+ }
+
+ DBG("%s: assign to plane %s caps %x", ovl->name, plane->name, caps);
+
+ if (r_overlay) {
+ DBG("%s: assign to right of plane %s caps %x",
+ r_ovl->name, plane->name, caps);
+ }
+
+ return 0;
+}
+
+/*
+ * Release an overlay from a plane if the plane gets not visible or the plane
+ * need a new overlay if overlay caps changes.
+ * This should be called from the plane atomic_check() in order to prepare the
+ * next global overlay_map to be enabled when atomic transaction is valid.
+ */
+void omap_overlay_release(struct drm_atomic_state *s, struct omap_hw_overlay *overlay)
+{
+ /* Get the global state of the current atomic transaction */
+ struct omap_global_state *state = omap_get_global_state(s);
+ struct drm_plane **overlay_map = state->hwoverlay_to_plane;
+
+ if (!overlay)
+ return;
+
+ if (WARN_ON(!overlay_map[overlay->idx]))
+ return;
+
+ DBG("%s: release from plane %s", overlay->name, overlay_map[overlay->idx]->name);
+
+ overlay_map[overlay->idx] = NULL;
+}
+
+/*
+ * Update an overlay state that was attached to a plane before the current atomic state.
+ * This should be called from the plane atomic_update() or atomic_disable(),
+ * where an overlay association to a plane could have changed between the old and current
+ * atomic state.
+ */
+void omap_overlay_update_state(struct omap_drm_private *priv,
+ struct omap_hw_overlay *overlay)
+{
+ struct omap_global_state *state = omap_get_existing_global_state(priv);
+ struct drm_plane **overlay_map = state->hwoverlay_to_plane;
+
+ /* Check if this overlay is not used anymore, then disable it */
+ if (!overlay_map[overlay->idx]) {
+ DBG("%s: disabled", overlay->name);
+
+ /* disable the overlay */
+ dispc_ovl_enable(priv->dispc, overlay->id, false);
+ }
+}
+
+static void omap_overlay_destroy(struct omap_hw_overlay *overlay)
+{
+ kfree(overlay);
+}
+
+static struct omap_hw_overlay *omap_overlay_init(enum omap_plane_id overlay_id,
+ enum omap_overlay_caps caps)
+{
+ struct omap_hw_overlay *overlay;
+
+ overlay = kzalloc(sizeof(*overlay), GFP_KERNEL);
+ if (!overlay)
+ return ERR_PTR(-ENOMEM);
+
+ overlay->name = overlay_id_to_name[overlay_id];
+ overlay->id = overlay_id;
+ overlay->caps = caps;
+
+ return overlay;
+}
+
+int omap_hwoverlays_init(struct omap_drm_private *priv)
+{
+ static const enum omap_plane_id hw_plane_ids[] = {
+ OMAP_DSS_GFX, OMAP_DSS_VIDEO1,
+ OMAP_DSS_VIDEO2, OMAP_DSS_VIDEO3,
+ };
+ u32 num_overlays = dispc_get_num_ovls(priv->dispc);
+ enum omap_overlay_caps caps;
+ int i, ret;
+
+ for (i = 0; i < num_overlays; i++) {
+ struct omap_hw_overlay *overlay;
+
+ caps = dispc_ovl_get_caps(priv->dispc, hw_plane_ids[i]);
+ overlay = omap_overlay_init(hw_plane_ids[i], caps);
+ if (IS_ERR(overlay)) {
+ ret = PTR_ERR(overlay);
+ dev_err(priv->dev, "failed to construct overlay for %s (%d)\n",
+ overlay_id_to_name[i], ret);
+ omap_hwoverlays_destroy(priv);
+ return ret;
+ }
+ overlay->idx = priv->num_ovls;
+ priv->overlays[priv->num_ovls++] = overlay;
+ }
+
+ return 0;
+}
+
+void omap_hwoverlays_destroy(struct omap_drm_private *priv)
+{
+ int i;
+
+ for (i = 0; i < priv->num_ovls; i++) {
+ omap_overlay_destroy(priv->overlays[i]);
+ priv->overlays[i] = NULL;
+ }
+
+ priv->num_ovls = 0;
+}
diff --git a/drivers/gpu/drm/omapdrm/omap_overlay.h b/drivers/gpu/drm/omapdrm/omap_overlay.h
new file mode 100644
index 0000000..e36a43f
--- /dev/null
+++ b/drivers/gpu/drm/omapdrm/omap_overlay.h
@@ -0,0 +1,35 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2018 Texas Instruments Incorporated - http://www.ti.com/
+ * Author: Benoit Parrot <bparrot@ti.com>
+ */
+
+#ifndef __OMAPDRM_OVERLAY_H__
+#define __OMAPDRM_OVERLAY_H__
+
+#include <linux/types.h>
+
+enum drm_plane_type;
+
+struct drm_device;
+struct drm_mode_object;
+struct drm_plane;
+
+/* Used to associate a HW overlay/plane to a plane */
+struct omap_hw_overlay {
+ unsigned int idx;
+
+ const char *name;
+ enum omap_plane_id id;
+
+ enum omap_overlay_caps caps;
+};
+
+int omap_hwoverlays_init(struct omap_drm_private *priv);
+void omap_hwoverlays_destroy(struct omap_drm_private *priv);
+int omap_overlay_assign(struct drm_atomic_state *s, struct drm_plane *plane,
+ u32 caps, u32 fourcc, struct omap_hw_overlay **overlay,
+ struct omap_hw_overlay **r_overlay);
+void omap_overlay_release(struct drm_atomic_state *s, struct omap_hw_overlay *overlay);
+void omap_overlay_update_state(struct omap_drm_private *priv, struct omap_hw_overlay *overlay);
+#endif /* __OMAPDRM_OVERLAY_H__ */
diff --git a/drivers/gpu/drm/omapdrm/omap_plane.c b/drivers/gpu/drm/omapdrm/omap_plane.c
index 512af97..b35205c 100644
--- a/drivers/gpu/drm/omapdrm/omap_plane.c
+++ b/drivers/gpu/drm/omapdrm/omap_plane.c
@@ -8,6 +8,7 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_gem_atomic_helper.h>
#include <drm/drm_plane_helper.h>
+#include <drm/drm_fourcc.h>
#include "omap_dmm_tiler.h"
#include "omap_drv.h"
@@ -16,14 +17,30 @@
* plane funcs
*/
+#define to_omap_plane_state(x) container_of(x, struct omap_plane_state, base)
+
+struct omap_plane_state {
+ /* Must be first. */
+ struct drm_plane_state base;
+
+ struct omap_hw_overlay *overlay;
+ struct omap_hw_overlay *r_overlay; /* right overlay */
+};
+
#define to_omap_plane(x) container_of(x, struct omap_plane, base)
struct omap_plane {
struct drm_plane base;
enum omap_plane_id id;
- const char *name;
};
+bool is_omap_plane_dual_overlay(struct drm_plane_state *state)
+{
+ struct omap_plane_state *omap_state = to_omap_plane_state(state);
+
+ return !!omap_state->r_overlay;
+}
+
static int omap_plane_prepare_fb(struct drm_plane *plane,
struct drm_plane_state *new_state)
{
@@ -46,13 +63,35 @@ static void omap_plane_atomic_update(struct drm_plane *plane,
struct drm_atomic_state *state)
{
struct omap_drm_private *priv = plane->dev->dev_private;
- struct omap_plane *omap_plane = to_omap_plane(plane);
struct drm_plane_state *new_state = drm_atomic_get_new_plane_state(state,
plane);
- struct omap_overlay_info info;
+ struct drm_plane_state *old_state = drm_atomic_get_old_plane_state(state,
+ plane);
+ struct omap_plane_state *new_omap_state;
+ struct omap_plane_state *old_omap_state;
+ struct omap_overlay_info info, r_info;
+ enum omap_plane_id ovl_id, r_ovl_id;
int ret;
+ bool dual_ovl;
- DBG("%s, crtc=%p fb=%p", omap_plane->name, new_state->crtc,
+ new_omap_state = to_omap_plane_state(new_state);
+ old_omap_state = to_omap_plane_state(old_state);
+
+ dual_ovl = is_omap_plane_dual_overlay(new_state);
+
+ /* Cleanup previously held overlay if needed */
+ if (old_omap_state->overlay)
+ omap_overlay_update_state(priv, old_omap_state->overlay);
+ if (old_omap_state->r_overlay)
+ omap_overlay_update_state(priv, old_omap_state->r_overlay);
+
+ if (!new_omap_state->overlay) {
+ DBG("[PLANE:%d:%s] no overlay attached", plane->base.id, plane->name);
+ return;
+ }
+
+ ovl_id = new_omap_state->overlay->id;
+ DBG("%s, crtc=%p fb=%p", plane->name, new_state->crtc,
new_state->fb);
memset(&info, 0, sizeof(info));
@@ -67,65 +106,155 @@ static void omap_plane_atomic_update(struct drm_plane *plane,
info.color_encoding = new_state->color_encoding;
info.color_range = new_state->color_range;
- /* update scanout: */
- omap_framebuffer_update_scanout(new_state->fb, new_state, &info);
+ r_info = info;
- DBG("%dx%d -> %dx%d (%d)", info.width, info.height,
- info.out_width, info.out_height,
- info.screen_width);
+ /* update scanout: */
+ omap_framebuffer_update_scanout(new_state->fb, new_state, &info,
+ dual_ovl ? &r_info : NULL);
+
+ DBG("%s: %dx%d -> %dx%d (%d)",
+ new_omap_state->overlay->name, info.width, info.height,
+ info.out_width, info.out_height, info.screen_width);
DBG("%d,%d %pad %pad", info.pos_x, info.pos_y,
&info.paddr, &info.p_uv_addr);
+ if (dual_ovl) {
+ r_ovl_id = new_omap_state->r_overlay->id;
+ /*
+ * If the current plane uses 2 hw planes the very next
+ * zorder is used by the r_overlay so we just use the
+ * main overlay zorder + 1
+ */
+ r_info.zorder = info.zorder + 1;
+
+ DBG("%s: %dx%d -> %dx%d (%d)",
+ new_omap_state->r_overlay->name,
+ r_info.width, r_info.height,
+ r_info.out_width, r_info.out_height, r_info.screen_width);
+ DBG("%d,%d %pad %pad", r_info.pos_x, r_info.pos_y,
+ &r_info.paddr, &r_info.p_uv_addr);
+ }
+
/* and finally, update omapdss: */
- ret = dispc_ovl_setup(priv->dispc, omap_plane->id, &info,
+ ret = dispc_ovl_setup(priv->dispc, ovl_id, &info,
omap_crtc_timings(new_state->crtc), false,
omap_crtc_channel(new_state->crtc));
if (ret) {
dev_err(plane->dev->dev, "Failed to setup plane %s\n",
- omap_plane->name);
- dispc_ovl_enable(priv->dispc, omap_plane->id, false);
+ plane->name);
+ dispc_ovl_enable(priv->dispc, ovl_id, false);
return;
}
- dispc_ovl_enable(priv->dispc, omap_plane->id, true);
+ dispc_ovl_enable(priv->dispc, ovl_id, true);
+
+ if (dual_ovl) {
+ ret = dispc_ovl_setup(priv->dispc, r_ovl_id, &r_info,
+ omap_crtc_timings(new_state->crtc), false,
+ omap_crtc_channel(new_state->crtc));
+ if (ret) {
+ dev_err(plane->dev->dev, "Failed to setup plane right-overlay %s\n",
+ plane->name);
+ dispc_ovl_enable(priv->dispc, r_ovl_id, false);
+ dispc_ovl_enable(priv->dispc, ovl_id, false);
+ return;
+ }
+
+ dispc_ovl_enable(priv->dispc, r_ovl_id, true);
+ }
}
static void omap_plane_atomic_disable(struct drm_plane *plane,
struct drm_atomic_state *state)
{
- struct drm_plane_state *new_state = drm_atomic_get_new_plane_state(state,
- plane);
struct omap_drm_private *priv = plane->dev->dev_private;
struct omap_plane *omap_plane = to_omap_plane(plane);
+ struct drm_plane_state *new_state = drm_atomic_get_new_plane_state(state,
+ plane);
+ struct drm_plane_state *old_state = drm_atomic_get_old_plane_state(state,
+ plane);
+ struct omap_plane_state *new_omap_state;
+ struct omap_plane_state *old_omap_state;
+
+ new_omap_state = to_omap_plane_state(new_state);
+ old_omap_state = to_omap_plane_state(old_state);
+
+ if (!old_omap_state->overlay)
+ return;
new_state->rotation = DRM_MODE_ROTATE_0;
new_state->zpos = plane->type == DRM_PLANE_TYPE_PRIMARY ? 0 : omap_plane->id;
- dispc_ovl_enable(priv->dispc, omap_plane->id, false);
+ omap_overlay_update_state(priv, old_omap_state->overlay);
+ new_omap_state->overlay = NULL;
+
+ if (is_omap_plane_dual_overlay(old_state)) {
+ omap_overlay_update_state(priv, old_omap_state->r_overlay);
+ new_omap_state->r_overlay = NULL;
+ }
}
+#define FRAC_16_16(mult, div) (((mult) << 16) / (div))
+
static int omap_plane_atomic_check(struct drm_plane *plane,
struct drm_atomic_state *state)
{
struct drm_plane_state *new_plane_state = drm_atomic_get_new_plane_state(state,
plane);
+ struct drm_plane_state *old_plane_state = drm_atomic_get_old_plane_state(state,
+ plane);
+ struct omap_drm_private *priv = plane->dev->dev_private;
+ struct omap_plane_state *omap_state = to_omap_plane_state(new_plane_state);
+ struct omap_global_state *omap_overlay_global_state;
struct drm_crtc_state *crtc_state;
+ bool new_r_hw_overlay = false;
+ bool new_hw_overlay = false;
+ u32 max_width, max_height;
+ struct drm_crtc *crtc;
+ u16 width, height;
+ u32 caps = 0;
+ u32 fourcc;
+ int ret;
- if (!new_plane_state->fb)
+ omap_overlay_global_state = omap_get_global_state(state);
+ if (IS_ERR(omap_overlay_global_state))
+ return PTR_ERR(omap_overlay_global_state);
+
+ dispc_ovl_get_max_size(priv->dispc, &width, &height);
+ max_width = width << 16;
+ max_height = height << 16;
+
+ crtc = new_plane_state->crtc ? new_plane_state->crtc : plane->state->crtc;
+ if (!crtc)
return 0;
- /* crtc should only be NULL when disabling (i.e., !new_plane_state->fb) */
- if (WARN_ON(!new_plane_state->crtc))
- return 0;
-
- crtc_state = drm_atomic_get_existing_crtc_state(state,
- new_plane_state->crtc);
+ crtc_state = drm_atomic_get_existing_crtc_state(state, crtc);
/* we should have a crtc state if the plane is attached to a crtc */
if (WARN_ON(!crtc_state))
return 0;
- if (!crtc_state->enable)
+ /*
+ * Note: these are just sanity checks to filter out totally bad scaling
+ * factors. The real limits must be calculated case by case, and
+ * unfortunately we currently do those checks only at the commit
+ * phase in dispc.
+ */
+ ret = drm_atomic_helper_check_plane_state(new_plane_state, crtc_state,
+ FRAC_16_16(1, 8), FRAC_16_16(8, 1),
+ true, true);
+ if (ret)
+ return ret;
+
+ DBG("%s: visible %d -> %d", plane->name,
+ old_plane_state->visible, new_plane_state->visible);
+
+ if (!new_plane_state->visible) {
+ omap_overlay_release(state, omap_state->overlay);
+ omap_overlay_release(state, omap_state->r_overlay);
+ omap_state->overlay = NULL;
+ omap_state->r_overlay = NULL;
return 0;
+ }
if (new_plane_state->crtc_x < 0 || new_plane_state->crtc_y < 0)
return -EINVAL;
@@ -136,10 +265,96 @@ static int omap_plane_atomic_check(struct drm_plane *plane,
if (new_plane_state->crtc_y + new_plane_state->crtc_h > crtc_state->adjusted_mode.vdisplay)
return -EINVAL;
+ /* Make sure dimensions are within bounds. */
+ if (new_plane_state->src_h > max_height || new_plane_state->crtc_h > height)
+ return -EINVAL;
+
+
+ if (new_plane_state->src_w > max_width || new_plane_state->crtc_w > width) {
+ bool is_fourcc_yuv = new_plane_state->fb->format->is_yuv;
+
+ if (is_fourcc_yuv && (((new_plane_state->src_w >> 16) / 2 & 1) ||
+ new_plane_state->crtc_w / 2 & 1)) {
+ /*
+ * When calculating the split overlay width
+ * and it yield an odd value we will need to adjust
+ * the indivual width +/- 1. So make sure it fits
+ */
+ if (new_plane_state->src_w <= ((2 * width - 1) << 16) &&
+ new_plane_state->crtc_w <= (2 * width - 1))
+ new_r_hw_overlay = true;
+ else
+ return -EINVAL;
+ } else {
+ if (new_plane_state->src_w <= (2 * max_width) &&
+ new_plane_state->crtc_w <= (2 * width))
+ new_r_hw_overlay = true;
+ else
+ return -EINVAL;
+ }
+ }
+
if (new_plane_state->rotation != DRM_MODE_ROTATE_0 &&
!omap_framebuffer_supports_rotation(new_plane_state->fb))
return -EINVAL;
+ if ((new_plane_state->src_w >> 16) != new_plane_state->crtc_w ||
+ (new_plane_state->src_h >> 16) != new_plane_state->crtc_h)
+ caps |= OMAP_DSS_OVL_CAP_SCALE;
+
+ fourcc = new_plane_state->fb->format->format;
+
+ /*
+ * (re)allocate hw overlay if we don't have one or
+ * there is a caps mismatch
+ */
+ if (!omap_state->overlay || (caps & ~omap_state->overlay->caps)) {
+ new_hw_overlay = true;
+ } else {
+ /* check supported format */
+ if (!dispc_ovl_color_mode_supported(priv->dispc, omap_state->overlay->id,
+ fourcc))
+ new_hw_overlay = true;
+ }
+
+ /*
+ * check if we need two overlays and only have 1 or
+ * if we had 2 overlays but will only need 1
+ */
+ if ((new_r_hw_overlay && !omap_state->r_overlay) ||
+ (!new_r_hw_overlay && omap_state->r_overlay))
+ new_hw_overlay = true;
+
+ if (new_hw_overlay) {
+ struct omap_hw_overlay *old_ovl = omap_state->overlay;
+ struct omap_hw_overlay *old_r_ovl = omap_state->r_overlay;
+ struct omap_hw_overlay *new_ovl = NULL;
+ struct omap_hw_overlay *new_r_ovl = NULL;
+
+ omap_overlay_release(state, old_ovl);
+ omap_overlay_release(state, old_r_ovl);
+
+ ret = omap_overlay_assign(state, plane, caps, fourcc, &new_ovl,
+ new_r_hw_overlay ? &new_r_ovl : NULL);
+ if (ret) {
+ DBG("%s: failed to assign hw_overlay", plane->name);
+ omap_state->overlay = NULL;
+ omap_state->r_overlay = NULL;
+ return ret;
+ }
+
+ omap_state->overlay = new_ovl;
+ if (new_r_hw_overlay)
+ omap_state->r_overlay = new_r_ovl;
+ else
+ omap_state->r_overlay = NULL;
+ }
+
+ DBG("plane: %s overlay_id: %d", plane->name, omap_state->overlay->id);
+
+ if (omap_state->r_overlay)
+ DBG("plane: %s r_overlay_id: %d", plane->name, omap_state->r_overlay->id);
+
return 0;
}
@@ -155,7 +370,7 @@ static void omap_plane_destroy(struct drm_plane *plane)
{
struct omap_plane *omap_plane = to_omap_plane(plane);
- DBG("%s", omap_plane->name);
+ DBG("%s", plane->name);
drm_plane_cleanup(plane);
@@ -189,11 +404,17 @@ void omap_plane_install_properties(struct drm_plane *plane,
static void omap_plane_reset(struct drm_plane *plane)
{
struct omap_plane *omap_plane = to_omap_plane(plane);
+ struct omap_plane_state *omap_state;
- drm_atomic_helper_plane_reset(plane);
- if (!plane->state)
+ if (plane->state)
+ drm_atomic_helper_plane_destroy_state(plane, plane->state);
+
+ omap_state = kzalloc(sizeof(*omap_state), GFP_KERNEL);
+ if (!omap_state)
return;
+ __drm_atomic_helper_plane_reset(plane, &omap_state->base);
+
/*
* Set the zpos default depending on whether we are a primary or overlay
* plane.
@@ -204,6 +425,47 @@ static void omap_plane_reset(struct drm_plane *plane)
plane->state->color_range = DRM_COLOR_YCBCR_FULL_RANGE;
}
+static struct drm_plane_state *
+omap_plane_atomic_duplicate_state(struct drm_plane *plane)
+{
+ struct omap_plane_state *state, *current_state;
+
+ if (WARN_ON(!plane->state))
+ return NULL;
+
+ current_state = to_omap_plane_state(plane->state);
+
+ state = kmalloc(sizeof(*state), GFP_KERNEL);
+ if (!state)
+ return NULL;
+
+ __drm_atomic_helper_plane_duplicate_state(plane, &state->base);
+
+ state->overlay = current_state->overlay;
+ state->r_overlay = current_state->r_overlay;
+
+ return &state->base;
+}
+
+static void omap_plane_atomic_print_state(struct drm_printer *p,
+ const struct drm_plane_state *state)
+{
+ struct omap_plane_state *omap_state = to_omap_plane_state(state);
+
+ if (omap_state->overlay)
+ drm_printf(p, "\toverlay=%s (caps=0x%x)\n",
+ omap_state->overlay->name,
+ omap_state->overlay->caps);
+ else
+ drm_printf(p, "\toverlay=None\n");
+ if (omap_state->r_overlay)
+ drm_printf(p, "\tr_overlay=%s (caps=0x%x)\n",
+ omap_state->r_overlay->name,
+ omap_state->r_overlay->caps);
+ else
+ drm_printf(p, "\tr_overlay=None\n");
+}
+
static int omap_plane_atomic_set_property(struct drm_plane *plane,
struct drm_plane_state *state,
struct drm_property *property,
@@ -239,10 +501,11 @@ static const struct drm_plane_funcs omap_plane_funcs = {
.disable_plane = drm_atomic_helper_disable_plane,
.reset = omap_plane_reset,
.destroy = omap_plane_destroy,
- .atomic_duplicate_state = drm_atomic_helper_plane_duplicate_state,
+ .atomic_duplicate_state = omap_plane_atomic_duplicate_state,
.atomic_destroy_state = drm_atomic_helper_plane_destroy_state,
.atomic_set_property = omap_plane_atomic_set_property,
.atomic_get_property = omap_plane_atomic_get_property,
+ .atomic_print_state = omap_plane_atomic_print_state,
};
static bool omap_plane_supports_yuv(struct drm_plane *plane)
@@ -261,20 +524,6 @@ static bool omap_plane_supports_yuv(struct drm_plane *plane)
return false;
}
-static const char *plane_id_to_name[] = {
- [OMAP_DSS_GFX] = "gfx",
- [OMAP_DSS_VIDEO1] = "vid1",
- [OMAP_DSS_VIDEO2] = "vid2",
- [OMAP_DSS_VIDEO3] = "vid3",
-};
-
-static const enum omap_plane_id plane_idx_to_id[] = {
- OMAP_DSS_GFX,
- OMAP_DSS_VIDEO1,
- OMAP_DSS_VIDEO2,
- OMAP_DSS_VIDEO3,
-};
-
/* initialize plane */
struct drm_plane *omap_plane_init(struct drm_device *dev,
int idx, enum drm_plane_type type,
@@ -284,27 +533,25 @@ struct drm_plane *omap_plane_init(struct drm_device *dev,
unsigned int num_planes = dispc_get_num_ovls(priv->dispc);
struct drm_plane *plane;
struct omap_plane *omap_plane;
- enum omap_plane_id id;
int ret;
u32 nformats;
const u32 *formats;
- if (WARN_ON(idx >= ARRAY_SIZE(plane_idx_to_id)))
+ if (WARN_ON(idx >= num_planes))
return ERR_PTR(-EINVAL);
- id = plane_idx_to_id[idx];
-
- DBG("%s: type=%d", plane_id_to_name[id], type);
-
omap_plane = kzalloc(sizeof(*omap_plane), GFP_KERNEL);
if (!omap_plane)
return ERR_PTR(-ENOMEM);
- formats = dispc_ovl_get_color_modes(priv->dispc, id);
+ omap_plane->id = idx;
+
+ DBG("%d: type=%d", omap_plane->id, type);
+ DBG(" crtc_mask: 0x%04x", possible_crtcs);
+
+ formats = dispc_ovl_get_color_modes(priv->dispc, omap_plane->id);
for (nformats = 0; formats[nformats]; ++nformats)
;
- omap_plane->id = id;
- omap_plane->name = plane_id_to_name[id];
plane = &omap_plane->base;
@@ -334,8 +581,8 @@ struct drm_plane *omap_plane_init(struct drm_device *dev,
return plane;
error:
- dev_err(dev->dev, "%s(): could not create plane: %s\n",
- __func__, plane_id_to_name[id]);
+ dev_err(dev->dev, "%s(): could not create plane: %d\n",
+ __func__, omap_plane->id);
kfree(omap_plane);
return NULL;
diff --git a/drivers/gpu/drm/omapdrm/omap_plane.h b/drivers/gpu/drm/omapdrm/omap_plane.h
index 0c28fe8..a9a33e1 100644
--- a/drivers/gpu/drm/omapdrm/omap_plane.h
+++ b/drivers/gpu/drm/omapdrm/omap_plane.h
@@ -22,5 +22,6 @@ struct drm_plane *omap_plane_init(struct drm_device *dev,
u32 possible_crtcs);
void omap_plane_install_properties(struct drm_plane *plane,
struct drm_mode_object *obj);
+bool is_omap_plane_dual_overlay(struct drm_plane_state *state);
#endif /* __OMAPDRM_PLANE_H__ */
diff --git a/drivers/gpu/drm/omapdrm/tcm-sita.c b/drivers/gpu/drm/omapdrm/tcm-sita.c
index 8338dc6..fde0208 100644
--- a/drivers/gpu/drm/omapdrm/tcm-sita.c
+++ b/drivers/gpu/drm/omapdrm/tcm-sita.c
@@ -1,3 +1,4 @@
+// SPDX-License-Identifier: GPL-2.0-only
/*
* SImple Tiler Allocator (SiTA): 2D and 1D allocation(reservation) algorithm
*
@@ -6,15 +7,6 @@
* Andy Gross <andy.gross@ti.com>
*
* Copyright (C) 2012 Texas Instruments Incorporated - https://www.ti.com/
- *
- * This package is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * THIS PACKAGE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR
- * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED
- * WARRANTIES OF MERCHANTIBILITY AND FITNESS FOR A PARTICULAR PURPOSE.
- *
*/
#include <linux/init.h>
#include <linux/module.h>
diff --git a/drivers/gpu/drm/panel/Kconfig b/drivers/gpu/drm/panel/Kconfig
index cfc8d64..434c286 100644
--- a/drivers/gpu/drm/panel/Kconfig
+++ b/drivers/gpu/drm/panel/Kconfig
@@ -37,6 +37,17 @@
NT35596 1080x1920 video mode panel as found in some Asus
Zenfone 2 Laser Z00T devices.
+config DRM_PANEL_BOE_BF060Y8M_AJ0
+ tristate "Boe BF060Y8M-AJ0 panel"
+ depends on OF
+ depends on DRM_MIPI_DSI
+ depends on BACKLIGHT_CLASS_DEVICE
+ help
+ Say Y here if you want to enable support for Boe BF060Y8M-AJ0
+ 5.99" AMOLED modules. The panel has a 1080x2160 resolution and
+ uses 24 bit RGB per pixel. It provides a MIPI DSI interface to
+ the host and backlight is controlled through DSI commands.
+
config DRM_PANEL_BOE_HIMAX8279D
tristate "Boe Himax8279d panel"
depends on OF
@@ -141,7 +152,7 @@
tristate "Ilitek ILI9341 240x320 QVGA panels"
depends on OF && SPI
depends on DRM_KMS_HELPER
- depends on DRM_KMS_CMA_HELPER
+ depends on DRM_GEM_CMA_HELPER
depends on BACKLIGHT_CLASS_DEVICE
select DRM_MIPI_DBI
help
@@ -189,6 +200,15 @@
The panel has a 1200(RGB)×1920 (WUXGA) resolution and uses
24 bit per pixel.
+config DRM_PANEL_JDI_R63452
+ tristate "JDI R63452 Full HD DSI panel"
+ depends on OF
+ depends on DRM_MIPI_DSI
+ depends on BACKLIGHT_CLASS_DEVICE
+ help
+ Say Y here if you want to enable support for the JDI R63452
+ DSI command mode panel as found in Xiaomi Mi 5 Devices.
+
config DRM_PANEL_KHADAS_TS050
tristate "Khadas TS050 panel"
depends on OF
@@ -272,6 +292,17 @@
around the Novatek NT35510 display controller, such as some
Hydis panels.
+config DRM_PANEL_NOVATEK_NT35950
+ tristate "Novatek NT35950 DSI panel"
+ depends on OF
+ depends on DRM_MIPI_DSI
+ depends on BACKLIGHT_CLASS_DEVICE
+ help
+ Say Y here if you want to enable support for the panels built
+ around the Novatek NT35950 display controller, such as some
+ Sharp panels used in Sony Xperia Z5 Premium and XZ Premium
+ mobile phones.
+
config DRM_PANEL_NOVATEK_NT36672A
tristate "Novatek NT36672A DSI panel"
depends on OF
@@ -580,6 +611,16 @@
Say Y here if you want to enable support for the Sony ACX565AKM
800x600 3.5" panel (found on the Nokia N900).
+config DRM_PANEL_SONY_TULIP_TRULY_NT35521
+ tristate "Sony Tulip Truly NT35521 panel"
+ depends on GPIOLIB && OF
+ depends on DRM_MIPI_DSI
+ depends on BACKLIGHT_CLASS_DEVICE
+ help
+ Say Y here if you want to enable support for the Sony Tulip
+ NT35521 1280x720 video mode panel as found on Sony Xperia M4
+ Aqua phone.
+
config DRM_PANEL_TDO_TL070WSH30
tristate "TDO TL070WSH30 DSI panel"
depends on OF
diff --git a/drivers/gpu/drm/panel/Makefile b/drivers/gpu/drm/panel/Makefile
index bca4cc1..d99fbbc 100644
--- a/drivers/gpu/drm/panel/Makefile
+++ b/drivers/gpu/drm/panel/Makefile
@@ -2,6 +2,7 @@
obj-$(CONFIG_DRM_PANEL_ABT_Y030XX067A) += panel-abt-y030xx067a.o
obj-$(CONFIG_DRM_PANEL_ARM_VERSATILE) += panel-arm-versatile.o
obj-$(CONFIG_DRM_PANEL_ASUS_Z00T_TM5P5_NT35596) += panel-asus-z00t-tm5p5-n35596.o
+obj-$(CONFIG_DRM_PANEL_BOE_BF060Y8M_AJ0) += panel-boe-bf060y8m-aj0.o
obj-$(CONFIG_DRM_PANEL_BOE_HIMAX8279D) += panel-boe-himax8279d.o
obj-$(CONFIG_DRM_PANEL_BOE_TV101WUM_NL6) += panel-boe-tv101wum-nl6.o
obj-$(CONFIG_DRM_PANEL_DSI_CM) += panel-dsi-cm.o
@@ -17,6 +18,7 @@
obj-$(CONFIG_DRM_PANEL_INNOLUX_EJ030NA) += panel-innolux-ej030na.o
obj-$(CONFIG_DRM_PANEL_INNOLUX_P079ZCA) += panel-innolux-p079zca.o
obj-$(CONFIG_DRM_PANEL_JDI_LT070ME05000) += panel-jdi-lt070me05000.o
+obj-$(CONFIG_DRM_PANEL_JDI_R63452) += panel-jdi-fhd-r63452.o
obj-$(CONFIG_DRM_PANEL_KHADAS_TS050) += panel-khadas-ts050.o
obj-$(CONFIG_DRM_PANEL_KINGDISPLAY_KD097D04) += panel-kingdisplay-kd097d04.o
obj-$(CONFIG_DRM_PANEL_LEADTEK_LTK050H3146W) += panel-leadtek-ltk050h3146w.o
@@ -25,6 +27,7 @@
obj-$(CONFIG_DRM_PANEL_LG_LG4573) += panel-lg-lg4573.o
obj-$(CONFIG_DRM_PANEL_NEC_NL8048HL11) += panel-nec-nl8048hl11.o
obj-$(CONFIG_DRM_PANEL_NOVATEK_NT35510) += panel-novatek-nt35510.o
+obj-$(CONFIG_DRM_PANEL_NOVATEK_NT35950) += panel-novatek-nt35950.o
obj-$(CONFIG_DRM_PANEL_NOVATEK_NT36672A) += panel-novatek-nt36672a.o
obj-$(CONFIG_DRM_PANEL_NOVATEK_NT39016) += panel-novatek-nt39016.o
obj-$(CONFIG_DRM_PANEL_MANTIX_MLAF057WE51) += panel-mantix-mlaf057we51.o
@@ -59,6 +62,7 @@
obj-$(CONFIG_DRM_PANEL_SITRONIX_ST7789V) += panel-sitronix-st7789v.o
obj-$(CONFIG_DRM_PANEL_SONY_ACX424AKP) += panel-sony-acx424akp.o
obj-$(CONFIG_DRM_PANEL_SONY_ACX565AKM) += panel-sony-acx565akm.o
+obj-$(CONFIG_DRM_PANEL_SONY_TULIP_TRULY_NT35521) += panel-sony-tulip-truly-nt35521.o
obj-$(CONFIG_DRM_PANEL_TDO_TL070WSH30) += panel-tdo-tl070wsh30.o
obj-$(CONFIG_DRM_PANEL_TPO_TD028TTEC1) += panel-tpo-td028ttec1.o
obj-$(CONFIG_DRM_PANEL_TPO_TD043MTEA1) += panel-tpo-td043mtea1.o
diff --git a/drivers/gpu/drm/panel/panel-abt-y030xx067a.c b/drivers/gpu/drm/panel/panel-abt-y030xx067a.c
index 3d8a9ab..f043b48 100644
--- a/drivers/gpu/drm/panel/panel-abt-y030xx067a.c
+++ b/drivers/gpu/drm/panel/panel-abt-y030xx067a.c
@@ -272,16 +272,14 @@ static int y030xx067a_probe(struct spi_device *spi)
return -EINVAL;
priv->supply = devm_regulator_get(dev, "power");
- if (IS_ERR(priv->supply)) {
- dev_err(dev, "Failed to get power supply\n");
- return PTR_ERR(priv->supply);
- }
+ if (IS_ERR(priv->supply))
+ return dev_err_probe(dev, PTR_ERR(priv->supply),
+ "Failed to get power supply\n");
priv->reset_gpio = devm_gpiod_get(dev, "reset", GPIOD_OUT_HIGH);
- if (IS_ERR(priv->reset_gpio)) {
- dev_err(dev, "Failed to get reset GPIO\n");
- return PTR_ERR(priv->reset_gpio);
- }
+ if (IS_ERR(priv->reset_gpio))
+ return dev_err_probe(dev, PTR_ERR(priv->reset_gpio),
+ "Failed to get reset GPIO\n");
drm_panel_init(&priv->panel, dev, &y030xx067a_funcs,
DRM_MODE_CONNECTOR_DPI);
diff --git a/drivers/gpu/drm/panel/panel-boe-bf060y8m-aj0.c b/drivers/gpu/drm/panel/panel-boe-bf060y8m-aj0.c
new file mode 100644
index 0000000..ef00cd6
--- /dev/null
+++ b/drivers/gpu/drm/panel/panel-boe-bf060y8m-aj0.c
@@ -0,0 +1,445 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * BOE BF060Y8M-AJ0 5.99" MIPI-DSI OLED Panel on SW43404 DriverIC
+ *
+ * Copyright (c) 2020 AngeloGioacchino Del Regno
+ * <angelogioacchino.delregno@somainline.org>
+ */
+
+#include <linux/backlight.h>
+#include <linux/delay.h>
+#include <linux/gpio/consumer.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/regulator/consumer.h>
+#include <video/mipi_display.h>
+#include <drm/drm_mipi_dsi.h>
+#include <drm/drm_modes.h>
+#include <drm/drm_panel.h>
+
+#define DCS_ALLOW_HBM_RANGE 0x0c
+#define DCS_DISALLOW_HBM_RANGE 0x08
+
+enum boe_bf060y8m_aj0_supplies {
+ BF060Y8M_VREG_VCC,
+ BF060Y8M_VREG_VDDIO,
+ BF060Y8M_VREG_VCI,
+ BF060Y8M_VREG_EL_VDD,
+ BF060Y8M_VREG_EL_VSS,
+ BF060Y8M_VREG_MAX
+};
+
+struct boe_bf060y8m_aj0 {
+ struct drm_panel panel;
+ struct mipi_dsi_device *dsi;
+ struct regulator_bulk_data vregs[BF060Y8M_VREG_MAX];
+ struct gpio_desc *reset_gpio;
+ bool prepared;
+};
+
+static inline
+struct boe_bf060y8m_aj0 *to_boe_bf060y8m_aj0(struct drm_panel *panel)
+{
+ return container_of(panel, struct boe_bf060y8m_aj0, panel);
+}
+
+#define dsi_dcs_write_seq(dsi, seq...) do { \
+ static const u8 d[] = { seq }; \
+ int ret; \
+ ret = mipi_dsi_dcs_write_buffer(dsi, d, ARRAY_SIZE(d)); \
+ if (ret < 0) \
+ return ret; \
+ } while (0)
+
+static void boe_bf060y8m_aj0_reset(struct boe_bf060y8m_aj0 *boe)
+{
+ gpiod_set_value_cansleep(boe->reset_gpio, 0);
+ usleep_range(2000, 3000);
+ gpiod_set_value_cansleep(boe->reset_gpio, 1);
+ usleep_range(15000, 16000);
+ gpiod_set_value_cansleep(boe->reset_gpio, 0);
+ usleep_range(5000, 6000);
+}
+
+static int boe_bf060y8m_aj0_on(struct boe_bf060y8m_aj0 *boe)
+{
+ struct mipi_dsi_device *dsi = boe->dsi;
+ struct device *dev = &dsi->dev;
+ int ret;
+
+ dsi_dcs_write_seq(dsi, 0xb0, 0xa5, 0x00);
+ dsi_dcs_write_seq(dsi, 0xb2, 0x00, 0x4c);
+ dsi_dcs_write_seq(dsi, MIPI_DCS_SET_3D_CONTROL, 0x10);
+ dsi_dcs_write_seq(dsi, MIPI_DCS_WRITE_POWER_SAVE, DCS_ALLOW_HBM_RANGE);
+ dsi_dcs_write_seq(dsi, 0xf8,
+ 0x00, 0x08, 0x10, 0x00, 0x22, 0x00, 0x00, 0x2d);
+
+ ret = mipi_dsi_dcs_exit_sleep_mode(dsi);
+ if (ret < 0) {
+ dev_err(dev, "Failed to exit sleep mode: %d\n", ret);
+ return ret;
+ }
+ msleep(30);
+
+ dsi_dcs_write_seq(dsi, 0xb0, 0xa5, 0x00);
+ dsi_dcs_write_seq(dsi, 0xc0,
+ 0x08, 0x48, 0x65, 0x33, 0x33, 0x33,
+ 0x2a, 0x31, 0x39, 0x20, 0x09);
+ dsi_dcs_write_seq(dsi, 0xc1, 0x00, 0x00, 0x00, 0x1f, 0x1f,
+ 0x1f, 0x1f, 0x1f, 0x1f, 0x1f, 0x1f, 0x1f,
+ 0x1f, 0x1f, 0x1f, 0x1f, 0x1f, 0x1f);
+ dsi_dcs_write_seq(dsi, 0xe2, 0x20, 0x04, 0x10, 0x12, 0x92,
+ 0x4f, 0x8f, 0x44, 0x84, 0x83, 0x83, 0x83,
+ 0x5c, 0x5c, 0x5c);
+ dsi_dcs_write_seq(dsi, 0xde, 0x01, 0x2c, 0x00, 0x77, 0x3e);
+
+ msleep(30);
+
+ ret = mipi_dsi_dcs_set_display_on(dsi);
+ if (ret < 0) {
+ dev_err(dev, "Failed to set display on: %d\n", ret);
+ return ret;
+ }
+ msleep(50);
+
+ return 0;
+}
+
+static int boe_bf060y8m_aj0_off(struct boe_bf060y8m_aj0 *boe)
+{
+ struct mipi_dsi_device *dsi = boe->dsi;
+ struct device *dev = &dsi->dev;
+ int ret;
+
+ /* OFF commands sent in HS mode */
+ dsi->mode_flags &= ~MIPI_DSI_MODE_LPM;
+ ret = mipi_dsi_dcs_set_display_off(dsi);
+ if (ret < 0) {
+ dev_err(dev, "Failed to set display off: %d\n", ret);
+ return ret;
+ }
+ msleep(20);
+
+ ret = mipi_dsi_dcs_enter_sleep_mode(dsi);
+ if (ret < 0) {
+ dev_err(dev, "Failed to enter sleep mode: %d\n", ret);
+ return ret;
+ }
+ usleep_range(1000, 2000);
+ dsi->mode_flags |= MIPI_DSI_MODE_LPM;
+
+ return 0;
+}
+
+static int boe_bf060y8m_aj0_prepare(struct drm_panel *panel)
+{
+ struct boe_bf060y8m_aj0 *boe = to_boe_bf060y8m_aj0(panel);
+ struct device *dev = &boe->dsi->dev;
+ int ret;
+
+ if (boe->prepared)
+ return 0;
+
+ /*
+ * Enable EL Driving Voltage first - doing that at the beginning
+ * or at the end of the power sequence doesn't matter, so enable
+ * it here to avoid yet another usleep at the end.
+ */
+ ret = regulator_enable(boe->vregs[BF060Y8M_VREG_EL_VDD].consumer);
+ if (ret)
+ return ret;
+ ret = regulator_enable(boe->vregs[BF060Y8M_VREG_EL_VSS].consumer);
+ if (ret)
+ goto err_elvss;
+
+ ret = regulator_enable(boe->vregs[BF060Y8M_VREG_VCC].consumer);
+ if (ret)
+ goto err_vcc;
+ usleep_range(1000, 2000);
+ ret = regulator_enable(boe->vregs[BF060Y8M_VREG_VDDIO].consumer);
+ if (ret)
+ goto err_vddio;
+ usleep_range(500, 1000);
+ ret = regulator_enable(boe->vregs[BF060Y8M_VREG_VCI].consumer);
+ if (ret)
+ goto err_vci;
+ usleep_range(2000, 3000);
+
+ boe_bf060y8m_aj0_reset(boe);
+
+ ret = boe_bf060y8m_aj0_on(boe);
+ if (ret < 0) {
+ dev_err(dev, "Failed to initialize panel: %d\n", ret);
+ gpiod_set_value_cansleep(boe->reset_gpio, 1);
+ return ret;
+ }
+
+ boe->prepared = true;
+ return 0;
+
+err_vci:
+ regulator_disable(boe->vregs[BF060Y8M_VREG_VDDIO].consumer);
+err_vddio:
+ regulator_disable(boe->vregs[BF060Y8M_VREG_VCC].consumer);
+err_vcc:
+ regulator_disable(boe->vregs[BF060Y8M_VREG_EL_VSS].consumer);
+err_elvss:
+ regulator_disable(boe->vregs[BF060Y8M_VREG_EL_VDD].consumer);
+ return ret;
+}
+
+static int boe_bf060y8m_aj0_unprepare(struct drm_panel *panel)
+{
+ struct boe_bf060y8m_aj0 *boe = to_boe_bf060y8m_aj0(panel);
+ struct device *dev = &boe->dsi->dev;
+ int ret;
+
+ if (!boe->prepared)
+ return 0;
+
+ ret = boe_bf060y8m_aj0_off(boe);
+ if (ret < 0)
+ dev_err(dev, "Failed to un-initialize panel: %d\n", ret);
+
+ gpiod_set_value_cansleep(boe->reset_gpio, 1);
+ ret = regulator_bulk_disable(ARRAY_SIZE(boe->vregs), boe->vregs);
+
+ boe->prepared = false;
+ return 0;
+}
+
+static const struct drm_display_mode boe_bf060y8m_aj0_mode = {
+ .clock = 165268,
+ .hdisplay = 1080,
+ .hsync_start = 1080 + 36,
+ .hsync_end = 1080 + 36 + 24,
+ .htotal = 1080 + 36 + 24 + 96,
+ .vdisplay = 2160,
+ .vsync_start = 2160 + 16,
+ .vsync_end = 2160 + 16 + 1,
+ .vtotal = 2160 + 16 + 1 + 15,
+ .width_mm = 68, /* 68.04 mm */
+ .height_mm = 136, /* 136.08 mm */
+};
+
+static int boe_bf060y8m_aj0_get_modes(struct drm_panel *panel,
+ struct drm_connector *connector)
+{
+ struct drm_display_mode *mode;
+
+ mode = drm_mode_duplicate(connector->dev, &boe_bf060y8m_aj0_mode);
+ if (!mode)
+ return -ENOMEM;
+
+ drm_mode_set_name(mode);
+
+ mode->type = DRM_MODE_TYPE_DRIVER | DRM_MODE_TYPE_PREFERRED;
+ connector->display_info.width_mm = mode->width_mm;
+ connector->display_info.height_mm = mode->height_mm;
+ drm_mode_probed_add(connector, mode);
+
+ return 1;
+}
+
+static const struct drm_panel_funcs boe_bf060y8m_aj0_panel_funcs = {
+ .prepare = boe_bf060y8m_aj0_prepare,
+ .unprepare = boe_bf060y8m_aj0_unprepare,
+ .get_modes = boe_bf060y8m_aj0_get_modes,
+};
+
+static int boe_bf060y8m_aj0_bl_update_status(struct backlight_device *bl)
+{
+ struct mipi_dsi_device *dsi = bl_get_data(bl);
+ u16 brightness = backlight_get_brightness(bl);
+ int ret;
+
+ ret = mipi_dsi_dcs_set_display_brightness(dsi, brightness);
+ if (ret < 0)
+ return ret;
+
+ return 0;
+}
+
+static int boe_bf060y8m_aj0_bl_get_brightness(struct backlight_device *bl)
+{
+ struct mipi_dsi_device *dsi = bl_get_data(bl);
+ u16 brightness;
+ int ret;
+
+ ret = mipi_dsi_dcs_get_display_brightness(dsi, &brightness);
+ if (ret < 0)
+ return ret;
+
+ return brightness & 0xff;
+}
+
+static const struct backlight_ops boe_bf060y8m_aj0_bl_ops = {
+ .update_status = boe_bf060y8m_aj0_bl_update_status,
+ .get_brightness = boe_bf060y8m_aj0_bl_get_brightness,
+};
+
+static struct backlight_device *
+boe_bf060y8m_aj0_create_backlight(struct mipi_dsi_device *dsi)
+{
+ struct device *dev = &dsi->dev;
+ const struct backlight_properties props = {
+ .type = BACKLIGHT_RAW,
+ .brightness = 127,
+ .max_brightness = 255,
+ .scale = BACKLIGHT_SCALE_NON_LINEAR,
+ };
+
+ return devm_backlight_device_register(dev, dev_name(dev), dev, dsi,
+ &boe_bf060y8m_aj0_bl_ops, &props);
+}
+
+static int boe_bf060y8m_aj0_init_vregs(struct boe_bf060y8m_aj0 *boe,
+ struct device *dev)
+{
+ struct regulator *vreg;
+ int ret;
+
+ boe->vregs[BF060Y8M_VREG_VCC].supply = "vcc";
+ boe->vregs[BF060Y8M_VREG_VDDIO].supply = "vddio";
+ boe->vregs[BF060Y8M_VREG_VCI].supply = "vci";
+ boe->vregs[BF060Y8M_VREG_EL_VDD].supply = "elvdd";
+ boe->vregs[BF060Y8M_VREG_EL_VSS].supply = "elvss";
+ ret = devm_regulator_bulk_get(dev, ARRAY_SIZE(boe->vregs),
+ boe->vregs);
+ if (ret < 0) {
+ dev_err(dev, "Failed to get regulators: %d\n", ret);
+ return ret;
+ }
+
+ vreg = boe->vregs[BF060Y8M_VREG_VCC].consumer;
+ ret = regulator_is_supported_voltage(vreg, 2700000, 3600000);
+ if (!ret)
+ return ret;
+
+ vreg = boe->vregs[BF060Y8M_VREG_VDDIO].consumer;
+ ret = regulator_is_supported_voltage(vreg, 1620000, 1980000);
+ if (!ret)
+ return ret;
+
+ vreg = boe->vregs[BF060Y8M_VREG_VCI].consumer;
+ ret = regulator_is_supported_voltage(vreg, 2600000, 3600000);
+ if (!ret)
+ return ret;
+
+ vreg = boe->vregs[BF060Y8M_VREG_EL_VDD].consumer;
+ ret = regulator_is_supported_voltage(vreg, 4400000, 4800000);
+ if (!ret)
+ return ret;
+
+ /* ELVSS is negative: -5.00V to -1.40V */
+ vreg = boe->vregs[BF060Y8M_VREG_EL_VSS].consumer;
+ ret = regulator_is_supported_voltage(vreg, 1400000, 5000000);
+ if (!ret)
+ return ret;
+
+ /*
+ * Set min/max rated current, known only for VCI and VDDIO and,
+ * in case of failure, just go on gracefully, as this step is not
+ * guaranteed to succeed on all regulator HW but do a debug print
+ * to inform the developer during debugging.
+ * In any case, these two supplies are also optional, so they may
+ * be fixed-regulator which, at the time of writing, does not
+ * support fake current limiting.
+ */
+ vreg = boe->vregs[BF060Y8M_VREG_VDDIO].consumer;
+ ret = regulator_set_current_limit(vreg, 1500, 2500);
+ if (ret)
+ dev_dbg(dev, "Current limit cannot be set on %s: %d\n",
+ boe->vregs[1].supply, ret);
+
+ vreg = boe->vregs[BF060Y8M_VREG_VCI].consumer;
+ ret = regulator_set_current_limit(vreg, 20000, 40000);
+ if (ret)
+ dev_dbg(dev, "Current limit cannot be set on %s: %d\n",
+ boe->vregs[2].supply, ret);
+
+ return 0;
+}
+
+static int boe_bf060y8m_aj0_probe(struct mipi_dsi_device *dsi)
+{
+ struct device *dev = &dsi->dev;
+ struct boe_bf060y8m_aj0 *boe;
+ int ret;
+
+ boe = devm_kzalloc(dev, sizeof(*boe), GFP_KERNEL);
+ if (!boe)
+ return -ENOMEM;
+
+ ret = boe_bf060y8m_aj0_init_vregs(boe, dev);
+ if (ret)
+ return dev_err_probe(dev, ret,
+ "Failed to initialize supplies.\n");
+
+ boe->reset_gpio = devm_gpiod_get_optional(dev, "reset", GPIOD_ASIS);
+ if (IS_ERR(boe->reset_gpio))
+ return dev_err_probe(dev, PTR_ERR(boe->reset_gpio),
+ "Failed to get reset-gpios\n");
+
+ boe->dsi = dsi;
+ mipi_dsi_set_drvdata(dsi, boe);
+
+ dsi->lanes = 4;
+ dsi->format = MIPI_DSI_FMT_RGB888;
+ dsi->mode_flags = MIPI_DSI_MODE_VIDEO | MIPI_DSI_MODE_NO_EOT_PACKET |
+ MIPI_DSI_MODE_VIDEO_SYNC_PULSE |
+ MIPI_DSI_CLOCK_NON_CONTINUOUS |
+ MIPI_DSI_MODE_LPM;
+
+ drm_panel_init(&boe->panel, dev, &boe_bf060y8m_aj0_panel_funcs,
+ DRM_MODE_CONNECTOR_DSI);
+
+ boe->panel.backlight = boe_bf060y8m_aj0_create_backlight(dsi);
+ if (IS_ERR(boe->panel.backlight))
+ return dev_err_probe(dev, PTR_ERR(boe->panel.backlight),
+ "Failed to create backlight\n");
+
+ drm_panel_add(&boe->panel);
+
+ ret = mipi_dsi_attach(dsi);
+ if (ret < 0) {
+ dev_err(dev, "Failed to attach to DSI host: %d\n", ret);
+ return ret;
+ }
+
+ return 0;
+}
+
+static int boe_bf060y8m_aj0_remove(struct mipi_dsi_device *dsi)
+{
+ struct boe_bf060y8m_aj0 *boe = mipi_dsi_get_drvdata(dsi);
+ int ret;
+
+ ret = mipi_dsi_detach(dsi);
+ if (ret < 0)
+ dev_err(&dsi->dev, "Failed to detach from DSI host: %d\n", ret);
+
+ drm_panel_remove(&boe->panel);
+
+ return 0;
+}
+
+static const struct of_device_id boe_bf060y8m_aj0_of_match[] = {
+ { .compatible = "boe,bf060y8m-aj0" },
+ { /* sentinel */ }
+};
+MODULE_DEVICE_TABLE(of, boe_bf060y8m_aj0_of_match);
+
+static struct mipi_dsi_driver boe_bf060y8m_aj0_driver = {
+ .probe = boe_bf060y8m_aj0_probe,
+ .remove = boe_bf060y8m_aj0_remove,
+ .driver = {
+ .name = "panel-sw43404-boe-fhd-amoled",
+ .of_match_table = boe_bf060y8m_aj0_of_match,
+ },
+};
+module_mipi_dsi_driver(boe_bf060y8m_aj0_driver);
+
+MODULE_AUTHOR("AngeloGioacchino Del Regno <angelogioacchino.delregno@somainline.org>");
+MODULE_DESCRIPTION("BOE BF060Y8M-AJ0 MIPI-DSI OLED panel");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/gpu/drm/panel/panel-boe-tv101wum-nl6.c b/drivers/gpu/drm/panel/panel-boe-tv101wum-nl6.c
index 529561b..5fcbde7 100644
--- a/drivers/gpu/drm/panel/panel-boe-tv101wum-nl6.c
+++ b/drivers/gpu/drm/panel/panel-boe-tv101wum-nl6.c
@@ -84,8 +84,8 @@ static const struct panel_init_cmd boe_tv110c9m_init_cmd[] = {
_INIT_DCS_CMD(0x0D, 0x63),
_INIT_DCS_CMD(0x0E, 0x91),
_INIT_DCS_CMD(0x0F, 0x73),
- _INIT_DCS_CMD(0x95, 0xEB),
- _INIT_DCS_CMD(0x96, 0xEB),
+ _INIT_DCS_CMD(0x95, 0xE6),
+ _INIT_DCS_CMD(0x96, 0xF0),
_INIT_DCS_CMD(0x30, 0x11),
_INIT_DCS_CMD(0x6D, 0x66),
_INIT_DCS_CMD(0x75, 0xA2),
@@ -111,18 +111,18 @@ static const struct panel_init_cmd boe_tv110c9m_init_cmd[] = {
_INIT_DCS_CMD(0xB0, 0x00, 0x00, 0x00, 0x1B, 0x00, 0x45, 0x00, 0x65, 0x00, 0x81, 0x00, 0x99, 0x00, 0xAE, 0x00, 0xC1),
_INIT_DCS_CMD(0xB1, 0x00, 0xD2, 0x01, 0x0B, 0x01, 0x34, 0x01, 0x76, 0x01, 0xA3, 0x01, 0xEF, 0x02, 0x27, 0x02, 0x29),
_INIT_DCS_CMD(0xB2, 0x02, 0x5F, 0x02, 0x9E, 0x02, 0xC9, 0x03, 0x00, 0x03, 0x26, 0x03, 0x53, 0x03, 0x63, 0x03, 0x73),
- _INIT_DCS_CMD(0xB3, 0x03, 0x86, 0x03, 0x9A, 0x03, 0xAF, 0x03, 0xDF, 0x03, 0xF5, 0x03, 0xF7),
+ _INIT_DCS_CMD(0xB3, 0x03, 0x86, 0x03, 0x9A, 0x03, 0xA7, 0x03, 0xCF, 0x03, 0xDE, 0x03, 0xE0),
_INIT_DCS_CMD(0xB4, 0x00, 0x00, 0x00, 0x1B, 0x00, 0x45, 0x00, 0x65, 0x00, 0x81, 0x00, 0x99, 0x00, 0xAE, 0x00, 0xC1),
_INIT_DCS_CMD(0xB5, 0x00, 0xD2, 0x01, 0x0B, 0x01, 0x34, 0x01, 0x76, 0x01, 0xA3, 0x01, 0xEF, 0x02, 0x27, 0x02, 0x29),
_INIT_DCS_CMD(0xB6, 0x02, 0x5F, 0x02, 0x9E, 0x02, 0xC9, 0x03, 0x00, 0x03, 0x26, 0x03, 0x53, 0x03, 0x63, 0x03, 0x73),
- _INIT_DCS_CMD(0xB7, 0x03, 0x86, 0x03, 0x9A, 0x03, 0xAF, 0x03, 0xDF, 0x03, 0xF5, 0x03, 0xF7),
+ _INIT_DCS_CMD(0xB7, 0x03, 0x86, 0x03, 0x9A, 0x03, 0xA7, 0x03, 0xCF, 0x03, 0xDE, 0x03, 0xE0),
_INIT_DCS_CMD(0xB8, 0x00, 0x00, 0x00, 0x1B, 0x00, 0x45, 0x00, 0x65, 0x00, 0x81, 0x00, 0x99, 0x00, 0xAE, 0x00, 0xC1),
_INIT_DCS_CMD(0xB9, 0x00, 0xD2, 0x01, 0x0B, 0x01, 0x34, 0x01, 0x76, 0x01, 0xA3, 0x01, 0xEF, 0x02, 0x27, 0x02, 0x29),
_INIT_DCS_CMD(0xBA, 0x02, 0x5F, 0x02, 0x9E, 0x02, 0xC9, 0x03, 0x00, 0x03, 0x26, 0x03, 0x53, 0x03, 0x63, 0x03, 0x73),
- _INIT_DCS_CMD(0xBB, 0x03, 0x86, 0x03, 0x9A, 0x03, 0xAF, 0x03, 0xDF, 0x03, 0xF5, 0x03, 0xF7),
+ _INIT_DCS_CMD(0xBB, 0x03, 0x86, 0x03, 0x9A, 0x03, 0xA7, 0x03, 0xCF, 0x03, 0xDE, 0x03, 0xE0),
_INIT_DCS_CMD(0xFF, 0x24),
_INIT_DCS_CMD(0xFB, 0x01),
@@ -225,6 +225,7 @@ static const struct panel_init_cmd boe_tv110c9m_init_cmd[] = {
_INIT_DCS_CMD(0x7F, 0x3C),
_INIT_DCS_CMD(0x82, 0x04),
_INIT_DCS_CMD(0x97, 0xC0),
+
_INIT_DCS_CMD(0xB6, 0x05, 0x00, 0x05, 0x00, 0x00, 0x00, 0x00, 0x00, 0x05, 0x05, 0x00, 0x00),
_INIT_DCS_CMD(0x91, 0x44),
_INIT_DCS_CMD(0x92, 0xA9),
@@ -332,12 +333,39 @@ static const struct panel_init_cmd boe_tv110c9m_init_cmd[] = {
_INIT_DCS_CMD(0x34, 0x78),
_INIT_DCS_CMD(0x35, 0x16),
_INIT_DCS_CMD(0xC8, 0x04),
- _INIT_DCS_CMD(0xC9, 0x80),
+ _INIT_DCS_CMD(0xC9, 0x9E),
_INIT_DCS_CMD(0xCA, 0x4E),
_INIT_DCS_CMD(0xCB, 0x00),
- _INIT_DCS_CMD(0xA9, 0x4C),
- _INIT_DCS_CMD(0xAA, 0x47),
+ _INIT_DCS_CMD(0xA9, 0x49),
+ _INIT_DCS_CMD(0xAA, 0x4B),
+ _INIT_DCS_CMD(0xAB, 0x48),
+ _INIT_DCS_CMD(0xAC, 0x43),
+ _INIT_DCS_CMD(0xAD, 0x40),
+ _INIT_DCS_CMD(0xAE, 0x50),
+ _INIT_DCS_CMD(0xAF, 0x44),
+ _INIT_DCS_CMD(0xB0, 0x54),
+ _INIT_DCS_CMD(0xB1, 0x4E),
+ _INIT_DCS_CMD(0xB2, 0x4D),
+ _INIT_DCS_CMD(0xB3, 0x4C),
+ _INIT_DCS_CMD(0xB4, 0x41),
+ _INIT_DCS_CMD(0xB5, 0x47),
+ _INIT_DCS_CMD(0xB6, 0x53),
+ _INIT_DCS_CMD(0xB7, 0x3E),
+ _INIT_DCS_CMD(0xB8, 0x51),
+ _INIT_DCS_CMD(0xB9, 0x3C),
+ _INIT_DCS_CMD(0xBA, 0x3B),
+ _INIT_DCS_CMD(0xBB, 0x46),
+ _INIT_DCS_CMD(0xBC, 0x45),
+ _INIT_DCS_CMD(0xBD, 0x55),
+ _INIT_DCS_CMD(0xBE, 0x3D),
+ _INIT_DCS_CMD(0xBF, 0x3F),
+ _INIT_DCS_CMD(0xC0, 0x52),
+ _INIT_DCS_CMD(0xC1, 0x4A),
+ _INIT_DCS_CMD(0xC2, 0x39),
+ _INIT_DCS_CMD(0xC3, 0x4F),
+ _INIT_DCS_CMD(0xC4, 0x3A),
+ _INIT_DCS_CMD(0xC5, 0x42),
_INIT_DCS_CMD(0xFF, 0x27),
_INIT_DCS_CMD(0xFB, 0x01),
@@ -419,7 +447,7 @@ static const struct panel_init_cmd boe_tv110c9m_init_cmd[] = {
{},
};
-static const struct panel_init_cmd inx_init_cmd[] = {
+static const struct panel_init_cmd inx_hj110iz_init_cmd[] = {
_INIT_DCS_CMD(0xFF, 0x20),
_INIT_DCS_CMD(0xFB, 0x01),
_INIT_DCS_CMD(0x05, 0xD1),
@@ -428,10 +456,10 @@ static const struct panel_init_cmd inx_init_cmd[] = {
_INIT_DCS_CMD(0x08, 0x4B),
_INIT_DCS_CMD(0x0E, 0x91),
_INIT_DCS_CMD(0x0F, 0x69),
- _INIT_DCS_CMD(0x95, 0xFF),
- _INIT_DCS_CMD(0x96, 0xFF),
- _INIT_DCS_CMD(0x9D, 0x0A),
- _INIT_DCS_CMD(0x9E, 0x0A),
+ _INIT_DCS_CMD(0x95, 0xF5),
+ _INIT_DCS_CMD(0x96, 0xF5),
+ _INIT_DCS_CMD(0x9D, 0x00),
+ _INIT_DCS_CMD(0x9E, 0x00),
_INIT_DCS_CMD(0x69, 0x98),
_INIT_DCS_CMD(0x75, 0xA2),
_INIT_DCS_CMD(0x77, 0xB3),
@@ -493,17 +521,17 @@ static const struct panel_init_cmd inx_init_cmd[] = {
_INIT_DCS_CMD(0x2A, 0x03),
_INIT_DCS_CMD(0x2B, 0x03),
- _INIT_DCS_CMD(0x2F, 0x06),
+ _INIT_DCS_CMD(0x2F, 0x05),
_INIT_DCS_CMD(0x30, 0x32),
_INIT_DCS_CMD(0x31, 0x43),
- _INIT_DCS_CMD(0x33, 0x06),
+ _INIT_DCS_CMD(0x33, 0x05),
_INIT_DCS_CMD(0x34, 0x32),
_INIT_DCS_CMD(0x35, 0x43),
_INIT_DCS_CMD(0x37, 0x44),
_INIT_DCS_CMD(0x38, 0x40),
_INIT_DCS_CMD(0x39, 0x00),
- _INIT_DCS_CMD(0x3A, 0x01),
- _INIT_DCS_CMD(0x3B, 0x48),
+ _INIT_DCS_CMD(0x3A, 0x18),
+ _INIT_DCS_CMD(0x3B, 0x00),
_INIT_DCS_CMD(0x3D, 0x93),
_INIT_DCS_CMD(0xAB, 0x44),
_INIT_DCS_CMD(0xAC, 0x40),
@@ -520,8 +548,8 @@ static const struct panel_init_cmd inx_init_cmd[] = {
_INIT_DCS_CMD(0x56, 0x08),
_INIT_DCS_CMD(0x58, 0x21),
_INIT_DCS_CMD(0x59, 0x40),
- _INIT_DCS_CMD(0x5A, 0x09),
- _INIT_DCS_CMD(0x5B, 0x48),
+ _INIT_DCS_CMD(0x5A, 0x00),
+ _INIT_DCS_CMD(0x5B, 0x2C),
_INIT_DCS_CMD(0x5E, 0x00, 0x10),
_INIT_DCS_CMD(0x5F, 0x00),
@@ -558,33 +586,36 @@ static const struct panel_init_cmd inx_init_cmd[] = {
_INIT_DCS_CMD(0xEF, 0x01),
_INIT_DCS_CMD(0xF0, 0x7A),
+ _INIT_DCS_CMD(0xB6, 0x05, 0x00, 0x05, 0x00, 0x00, 0x00, 0x00, 0x00, 0x05, 0x05, 0x00, 0x00),
_INIT_DCS_CMD(0xFF, 0x25),
_INIT_DCS_CMD(0xFB, 0x01),
_INIT_DCS_CMD(0x05, 0x00),
+ _INIT_DCS_CMD(0x13, 0x02),
+ _INIT_DCS_CMD(0x14, 0xDF),
_INIT_DCS_CMD(0xF1, 0x10),
_INIT_DCS_CMD(0x1E, 0x00),
- _INIT_DCS_CMD(0x1F, 0x09),
- _INIT_DCS_CMD(0x20, 0x46),
+ _INIT_DCS_CMD(0x1F, 0x00),
+ _INIT_DCS_CMD(0x20, 0x2C),
_INIT_DCS_CMD(0x25, 0x00),
- _INIT_DCS_CMD(0x26, 0x09),
- _INIT_DCS_CMD(0x27, 0x46),
+ _INIT_DCS_CMD(0x26, 0x00),
+ _INIT_DCS_CMD(0x27, 0x2C),
_INIT_DCS_CMD(0x3F, 0x80),
_INIT_DCS_CMD(0x40, 0x00),
_INIT_DCS_CMD(0x43, 0x00),
- _INIT_DCS_CMD(0x44, 0x09),
- _INIT_DCS_CMD(0x45, 0x46),
+ _INIT_DCS_CMD(0x44, 0x18),
+ _INIT_DCS_CMD(0x45, 0x00),
- _INIT_DCS_CMD(0x48, 0x09),
- _INIT_DCS_CMD(0x49, 0x46),
+ _INIT_DCS_CMD(0x48, 0x00),
+ _INIT_DCS_CMD(0x49, 0x2C),
_INIT_DCS_CMD(0x5B, 0x80),
_INIT_DCS_CMD(0x5C, 0x00),
- _INIT_DCS_CMD(0x5D, 0x01),
- _INIT_DCS_CMD(0x5E, 0x46),
- _INIT_DCS_CMD(0x61, 0x01),
- _INIT_DCS_CMD(0x62, 0x46),
+ _INIT_DCS_CMD(0x5D, 0x00),
+ _INIT_DCS_CMD(0x5E, 0x00),
+ _INIT_DCS_CMD(0x61, 0x00),
+ _INIT_DCS_CMD(0x62, 0x2C),
_INIT_DCS_CMD(0x68, 0x10),
_INIT_DCS_CMD(0xFF, 0x26),
_INIT_DCS_CMD(0xFB, 0x01),
@@ -700,16 +731,22 @@ static const struct panel_init_cmd inx_init_cmd[] = {
_INIT_DCS_CMD(0xA3, 0x30),
_INIT_DCS_CMD(0xA4, 0xC0),
_INIT_DCS_CMD(0xE8, 0x00),
+ _INIT_DCS_CMD(0x97, 0x3C),
+ _INIT_DCS_CMD(0x98, 0x02),
+ _INIT_DCS_CMD(0x99, 0x95),
+ _INIT_DCS_CMD(0x9A, 0x06),
+ _INIT_DCS_CMD(0x9B, 0x00),
+ _INIT_DCS_CMD(0x9C, 0x0B),
+ _INIT_DCS_CMD(0x9D, 0x0A),
+ _INIT_DCS_CMD(0x9E, 0x90),
_INIT_DCS_CMD(0xFF, 0xF0),
_INIT_DCS_CMD(0xFB, 0x01),
_INIT_DCS_CMD(0x3A, 0x08),
_INIT_DCS_CMD(0xFF, 0xD0),
_INIT_DCS_CMD(0xFB, 0x01),
_INIT_DCS_CMD(0x00, 0x33),
- _INIT_DCS_CMD(0x02, 0x77),
_INIT_DCS_CMD(0x08, 0x01),
_INIT_DCS_CMD(0x09, 0xBF),
- _INIT_DCS_CMD(0x28, 0x30),
_INIT_DCS_CMD(0x2F, 0x33),
_INIT_DCS_CMD(0xFF, 0x23),
_INIT_DCS_CMD(0xFB, 0x01),
@@ -718,6 +755,9 @@ static const struct panel_init_cmd inx_init_cmd[] = {
_INIT_DCS_CMD(0xFF, 0x20),
_INIT_DCS_CMD(0xFB, 0x01),
_INIT_DCS_CMD(0x30, 0x00),
+ _INIT_DCS_CMD(0xFF, 0x24),
+ _INIT_DCS_CMD(0x5C, 0x88),
+ _INIT_DCS_CMD(0x5D, 0x08),
_INIT_DCS_CMD(0xFF, 0x10),
_INIT_DCS_CMD(0xB9, 0x01),
_INIT_DCS_CMD(0xFF, 0x20),
@@ -1312,7 +1352,7 @@ static const struct panel_desc inx_hj110iz_desc = {
| MIPI_DSI_MODE_VIDEO_HSE
| MIPI_DSI_CLOCK_NON_CONTINUOUS
| MIPI_DSI_MODE_VIDEO_BURST,
- .init_cmds = inx_init_cmd,
+ .init_cmds = inx_hj110iz_init_cmd,
};
static const struct drm_display_mode boe_tv101wum_nl6_default_mode = {
diff --git a/drivers/gpu/drm/panel/panel-dsi-cm.c b/drivers/gpu/drm/panel/panel-dsi-cm.c
index da4a690..b58cb06 100644
--- a/drivers/gpu/drm/panel/panel-dsi-cm.c
+++ b/drivers/gpu/drm/panel/panel-dsi-cm.c
@@ -248,7 +248,7 @@ static ssize_t num_dsi_errors_show(struct device *dev,
if (r)
return r;
- return snprintf(buf, PAGE_SIZE, "%d\n", errors);
+ return sysfs_emit(buf, "%d\n", errors);
}
static ssize_t hw_revision_show(struct device *dev,
@@ -268,7 +268,7 @@ static ssize_t hw_revision_show(struct device *dev,
if (r)
return r;
- return snprintf(buf, PAGE_SIZE, "%02x.%02x.%02x\n", id1, id2, id3);
+ return sysfs_emit(buf, "%02x.%02x.%02x\n", id1, id2, id3);
}
static DEVICE_ATTR_RO(num_dsi_errors);
diff --git a/drivers/gpu/drm/panel/panel-edp.c b/drivers/gpu/drm/panel/panel-edp.c
index fc03046..176ef0c 100644
--- a/drivers/gpu/drm/panel/panel-edp.c
+++ b/drivers/gpu/drm/panel/panel-edp.c
@@ -196,10 +196,10 @@ struct edp_panel_entry {
/** @panel_id: 32-bit ID for panel, encoded with drm_edid_encode_panel_id(). */
u32 panel_id;
- /* @delay: The power sequencing delays needed for this panel. */
+ /** @delay: The power sequencing delays needed for this panel. */
const struct panel_delay *delay;
- /* @name: Name of this panel (for printing to logs). */
+ /** @name: Name of this panel (for printing to logs). */
const char *name;
};
diff --git a/drivers/gpu/drm/panel/panel-feixin-k101-im2ba02.c b/drivers/gpu/drm/panel/panel-feixin-k101-im2ba02.c
index 2a602ae..cb0bb30 100644
--- a/drivers/gpu/drm/panel/panel-feixin-k101-im2ba02.c
+++ b/drivers/gpu/drm/panel/panel-feixin-k101-im2ba02.c
@@ -456,16 +456,13 @@ static int k101_im2ba02_dsi_probe(struct mipi_dsi_device *dsi)
ret = devm_regulator_bulk_get(&dsi->dev, ARRAY_SIZE(ctx->supplies),
ctx->supplies);
- if (ret < 0) {
- dev_err(&dsi->dev, "Couldn't get regulators\n");
- return ret;
- }
+ if (ret < 0)
+ return dev_err_probe(&dsi->dev, ret, "Couldn't get regulators\n");
ctx->reset = devm_gpiod_get(&dsi->dev, "reset", GPIOD_OUT_LOW);
- if (IS_ERR(ctx->reset)) {
- dev_err(&dsi->dev, "Couldn't get our reset GPIO\n");
- return PTR_ERR(ctx->reset);
- }
+ if (IS_ERR(ctx->reset))
+ return dev_err_probe(&dsi->dev, PTR_ERR(ctx->reset),
+ "Couldn't get our reset GPIO\n");
drm_panel_init(&ctx->panel, &dsi->dev, &k101_im2ba02_funcs,
DRM_MODE_CONNECTOR_DSI);
diff --git a/drivers/gpu/drm/panel/panel-feiyang-fy07024di26a30d.c b/drivers/gpu/drm/panel/panel-feiyang-fy07024di26a30d.c
index 581661b..a9cd713 100644
--- a/drivers/gpu/drm/panel/panel-feiyang-fy07024di26a30d.c
+++ b/drivers/gpu/drm/panel/panel-feiyang-fy07024di26a30d.c
@@ -200,22 +200,19 @@ static int feiyang_dsi_probe(struct mipi_dsi_device *dsi)
DRM_MODE_CONNECTOR_DSI);
ctx->dvdd = devm_regulator_get(&dsi->dev, "dvdd");
- if (IS_ERR(ctx->dvdd)) {
- dev_err(&dsi->dev, "Couldn't get dvdd regulator\n");
- return PTR_ERR(ctx->dvdd);
- }
+ if (IS_ERR(ctx->dvdd))
+ return dev_err_probe(&dsi->dev, PTR_ERR(ctx->dvdd),
+ "Couldn't get dvdd regulator\n");
ctx->avdd = devm_regulator_get(&dsi->dev, "avdd");
- if (IS_ERR(ctx->avdd)) {
- dev_err(&dsi->dev, "Couldn't get avdd regulator\n");
- return PTR_ERR(ctx->avdd);
- }
+ if (IS_ERR(ctx->avdd))
+ return dev_err_probe(&dsi->dev, PTR_ERR(ctx->avdd),
+ "Couldn't get avdd regulator\n");
ctx->reset = devm_gpiod_get(&dsi->dev, "reset", GPIOD_OUT_LOW);
- if (IS_ERR(ctx->reset)) {
- dev_err(&dsi->dev, "Couldn't get our reset GPIO\n");
- return PTR_ERR(ctx->reset);
- }
+ if (IS_ERR(ctx->reset))
+ return dev_err_probe(&dsi->dev, PTR_ERR(ctx->reset),
+ "Couldn't get our reset GPIO\n");
ret = drm_panel_of_backlight(&ctx->panel);
if (ret)
@@ -227,7 +224,13 @@ static int feiyang_dsi_probe(struct mipi_dsi_device *dsi)
dsi->format = MIPI_DSI_FMT_RGB888;
dsi->lanes = 4;
- return mipi_dsi_attach(dsi);
+ ret = mipi_dsi_attach(dsi);
+ if (ret < 0) {
+ drm_panel_remove(&ctx->panel);
+ return ret;
+ }
+
+ return 0;
}
static int feiyang_dsi_remove(struct mipi_dsi_device *dsi)
diff --git a/drivers/gpu/drm/panel/panel-ilitek-ili9881c.c b/drivers/gpu/drm/panel/panel-ilitek-ili9881c.c
index 534dd741..ba30d11 100644
--- a/drivers/gpu/drm/panel/panel-ilitek-ili9881c.c
+++ b/drivers/gpu/drm/panel/panel-ilitek-ili9881c.c
@@ -42,6 +42,7 @@ struct ili9881c_desc {
const struct ili9881c_instr *init;
const size_t init_length;
const struct drm_display_mode *mode;
+ const unsigned long mode_flags;
};
struct ili9881c {
@@ -51,6 +52,8 @@ struct ili9881c {
struct regulator *power;
struct gpio_desc *reset;
+
+ enum drm_panel_orientation orientation;
};
#define ILI9881C_SWITCH_PAGE_INSTR(_page) \
@@ -453,6 +456,213 @@ static const struct ili9881c_instr k101_im2byl02_init[] = {
ILI9881C_COMMAND_INSTR(0xD3, 0x3F), /* VN0 */
};
+static const struct ili9881c_instr w552946ab_init[] = {
+ ILI9881C_SWITCH_PAGE_INSTR(3),
+ ILI9881C_COMMAND_INSTR(0x01, 0x00),
+ ILI9881C_COMMAND_INSTR(0x02, 0x00),
+ ILI9881C_COMMAND_INSTR(0x03, 0x53),
+ ILI9881C_COMMAND_INSTR(0x04, 0x53),
+ ILI9881C_COMMAND_INSTR(0x05, 0x13),
+ ILI9881C_COMMAND_INSTR(0x06, 0x04),
+ ILI9881C_COMMAND_INSTR(0x07, 0x02),
+ ILI9881C_COMMAND_INSTR(0x08, 0x02),
+ ILI9881C_COMMAND_INSTR(0x09, 0x00),
+ ILI9881C_COMMAND_INSTR(0x0A, 0x00),
+ ILI9881C_COMMAND_INSTR(0x0B, 0x00),
+ ILI9881C_COMMAND_INSTR(0x0C, 0x00),
+ ILI9881C_COMMAND_INSTR(0x0D, 0x00),
+ ILI9881C_COMMAND_INSTR(0x0E, 0x00),
+ ILI9881C_COMMAND_INSTR(0x0F, 0x00),
+
+ ILI9881C_COMMAND_INSTR(0x10, 0x00),
+ ILI9881C_COMMAND_INSTR(0x11, 0x00),
+ ILI9881C_COMMAND_INSTR(0x12, 0x00),
+ ILI9881C_COMMAND_INSTR(0x13, 0x00),
+ ILI9881C_COMMAND_INSTR(0x14, 0x00),
+ ILI9881C_COMMAND_INSTR(0x15, 0x08),
+ ILI9881C_COMMAND_INSTR(0x16, 0x10),
+ ILI9881C_COMMAND_INSTR(0x17, 0x00),
+ ILI9881C_COMMAND_INSTR(0x18, 0x08),
+ ILI9881C_COMMAND_INSTR(0x19, 0x00),
+ ILI9881C_COMMAND_INSTR(0x1A, 0x00),
+ ILI9881C_COMMAND_INSTR(0x1B, 0x00),
+ ILI9881C_COMMAND_INSTR(0x1C, 0x00),
+ ILI9881C_COMMAND_INSTR(0x1D, 0x00),
+ ILI9881C_COMMAND_INSTR(0x1E, 0xC0),
+ ILI9881C_COMMAND_INSTR(0x1F, 0x80),
+
+ ILI9881C_COMMAND_INSTR(0x20, 0x02),
+ ILI9881C_COMMAND_INSTR(0x21, 0x09),
+ ILI9881C_COMMAND_INSTR(0x22, 0x00),
+ ILI9881C_COMMAND_INSTR(0x23, 0x00),
+ ILI9881C_COMMAND_INSTR(0x24, 0x00),
+ ILI9881C_COMMAND_INSTR(0x25, 0x00),
+ ILI9881C_COMMAND_INSTR(0x26, 0x00),
+ ILI9881C_COMMAND_INSTR(0x27, 0x00),
+ ILI9881C_COMMAND_INSTR(0x28, 0x55),
+ ILI9881C_COMMAND_INSTR(0x29, 0x03),
+ ILI9881C_COMMAND_INSTR(0x2A, 0x00),
+ ILI9881C_COMMAND_INSTR(0x2B, 0x00),
+ ILI9881C_COMMAND_INSTR(0x2C, 0x00),
+ ILI9881C_COMMAND_INSTR(0x2D, 0x00),
+ ILI9881C_COMMAND_INSTR(0x2E, 0x00),
+ ILI9881C_COMMAND_INSTR(0x2F, 0x00),
+
+ ILI9881C_COMMAND_INSTR(0x30, 0x00),
+ ILI9881C_COMMAND_INSTR(0x31, 0x00),
+ ILI9881C_COMMAND_INSTR(0x32, 0x00),
+ ILI9881C_COMMAND_INSTR(0x33, 0x00),
+ ILI9881C_COMMAND_INSTR(0x34, 0x04),
+ ILI9881C_COMMAND_INSTR(0x35, 0x05),
+ ILI9881C_COMMAND_INSTR(0x36, 0x05),
+ ILI9881C_COMMAND_INSTR(0x37, 0x00),
+ ILI9881C_COMMAND_INSTR(0x38, 0x3C),
+ ILI9881C_COMMAND_INSTR(0x39, 0x35),
+ ILI9881C_COMMAND_INSTR(0x3A, 0x00),
+ ILI9881C_COMMAND_INSTR(0x3B, 0x40),
+ ILI9881C_COMMAND_INSTR(0x3C, 0x00),
+ ILI9881C_COMMAND_INSTR(0x3D, 0x00),
+ ILI9881C_COMMAND_INSTR(0x3E, 0x00),
+ ILI9881C_COMMAND_INSTR(0x3F, 0x00),
+
+ ILI9881C_COMMAND_INSTR(0x40, 0x00),
+ ILI9881C_COMMAND_INSTR(0x41, 0x88),
+ ILI9881C_COMMAND_INSTR(0x42, 0x00),
+ ILI9881C_COMMAND_INSTR(0x43, 0x00),
+ ILI9881C_COMMAND_INSTR(0x44, 0x1F),
+
+ ILI9881C_COMMAND_INSTR(0x50, 0x01),
+ ILI9881C_COMMAND_INSTR(0x51, 0x23),
+ ILI9881C_COMMAND_INSTR(0x52, 0x45),
+ ILI9881C_COMMAND_INSTR(0x53, 0x67),
+ ILI9881C_COMMAND_INSTR(0x54, 0x89),
+ ILI9881C_COMMAND_INSTR(0x55, 0xaB),
+ ILI9881C_COMMAND_INSTR(0x56, 0x01),
+ ILI9881C_COMMAND_INSTR(0x57, 0x23),
+ ILI9881C_COMMAND_INSTR(0x58, 0x45),
+ ILI9881C_COMMAND_INSTR(0x59, 0x67),
+ ILI9881C_COMMAND_INSTR(0x5A, 0x89),
+ ILI9881C_COMMAND_INSTR(0x5B, 0xAB),
+ ILI9881C_COMMAND_INSTR(0x5C, 0xCD),
+ ILI9881C_COMMAND_INSTR(0x5D, 0xEF),
+ ILI9881C_COMMAND_INSTR(0x5E, 0x03),
+ ILI9881C_COMMAND_INSTR(0x5F, 0x14),
+
+ ILI9881C_COMMAND_INSTR(0x60, 0x15),
+ ILI9881C_COMMAND_INSTR(0x61, 0x0C),
+ ILI9881C_COMMAND_INSTR(0x62, 0x0D),
+ ILI9881C_COMMAND_INSTR(0x63, 0x0E),
+ ILI9881C_COMMAND_INSTR(0x64, 0x0F),
+ ILI9881C_COMMAND_INSTR(0x65, 0x10),
+ ILI9881C_COMMAND_INSTR(0x66, 0x11),
+ ILI9881C_COMMAND_INSTR(0x67, 0x08),
+ ILI9881C_COMMAND_INSTR(0x68, 0x02),
+ ILI9881C_COMMAND_INSTR(0x69, 0x0A),
+ ILI9881C_COMMAND_INSTR(0x6A, 0x02),
+ ILI9881C_COMMAND_INSTR(0x6B, 0x02),
+ ILI9881C_COMMAND_INSTR(0x6C, 0x02),
+ ILI9881C_COMMAND_INSTR(0x6D, 0x02),
+ ILI9881C_COMMAND_INSTR(0x6E, 0x02),
+ ILI9881C_COMMAND_INSTR(0x6F, 0x02),
+
+ ILI9881C_COMMAND_INSTR(0x70, 0x02),
+ ILI9881C_COMMAND_INSTR(0x71, 0x02),
+ ILI9881C_COMMAND_INSTR(0x72, 0x06),
+ ILI9881C_COMMAND_INSTR(0x73, 0x02),
+ ILI9881C_COMMAND_INSTR(0x74, 0x02),
+ ILI9881C_COMMAND_INSTR(0x75, 0x14),
+ ILI9881C_COMMAND_INSTR(0x76, 0x15),
+ ILI9881C_COMMAND_INSTR(0x77, 0x0F),
+ ILI9881C_COMMAND_INSTR(0x78, 0x0E),
+ ILI9881C_COMMAND_INSTR(0x79, 0x0D),
+ ILI9881C_COMMAND_INSTR(0x7A, 0x0C),
+ ILI9881C_COMMAND_INSTR(0x7B, 0x11),
+ ILI9881C_COMMAND_INSTR(0x7C, 0x10),
+ ILI9881C_COMMAND_INSTR(0x7D, 0x06),
+ ILI9881C_COMMAND_INSTR(0x7E, 0x02),
+ ILI9881C_COMMAND_INSTR(0x7F, 0x0A),
+
+ ILI9881C_COMMAND_INSTR(0x80, 0x02),
+ ILI9881C_COMMAND_INSTR(0x81, 0x02),
+ ILI9881C_COMMAND_INSTR(0x82, 0x02),
+ ILI9881C_COMMAND_INSTR(0x83, 0x02),
+ ILI9881C_COMMAND_INSTR(0x84, 0x02),
+ ILI9881C_COMMAND_INSTR(0x85, 0x02),
+ ILI9881C_COMMAND_INSTR(0x86, 0x02),
+ ILI9881C_COMMAND_INSTR(0x87, 0x02),
+ ILI9881C_COMMAND_INSTR(0x88, 0x08),
+ ILI9881C_COMMAND_INSTR(0x89, 0x02),
+ ILI9881C_COMMAND_INSTR(0x8A, 0x02),
+
+ ILI9881C_SWITCH_PAGE_INSTR(4),
+ ILI9881C_COMMAND_INSTR(0x00, 0x80),
+ ILI9881C_COMMAND_INSTR(0x70, 0x00),
+ ILI9881C_COMMAND_INSTR(0x71, 0x00),
+ ILI9881C_COMMAND_INSTR(0x66, 0xFE),
+ ILI9881C_COMMAND_INSTR(0x82, 0x15),
+ ILI9881C_COMMAND_INSTR(0x84, 0x15),
+ ILI9881C_COMMAND_INSTR(0x85, 0x15),
+ ILI9881C_COMMAND_INSTR(0x3a, 0x24),
+ ILI9881C_COMMAND_INSTR(0x32, 0xAC),
+ ILI9881C_COMMAND_INSTR(0x8C, 0x80),
+ ILI9881C_COMMAND_INSTR(0x3C, 0xF5),
+ ILI9881C_COMMAND_INSTR(0x88, 0x33),
+
+ ILI9881C_SWITCH_PAGE_INSTR(1),
+ ILI9881C_COMMAND_INSTR(0x22, 0x0A),
+ ILI9881C_COMMAND_INSTR(0x31, 0x00),
+ ILI9881C_COMMAND_INSTR(0x53, 0x78),
+ ILI9881C_COMMAND_INSTR(0x50, 0x5B),
+ ILI9881C_COMMAND_INSTR(0x51, 0x5B),
+ ILI9881C_COMMAND_INSTR(0x60, 0x20),
+ ILI9881C_COMMAND_INSTR(0x61, 0x00),
+ ILI9881C_COMMAND_INSTR(0x62, 0x0D),
+ ILI9881C_COMMAND_INSTR(0x63, 0x00),
+
+ ILI9881C_COMMAND_INSTR(0xA0, 0x00),
+ ILI9881C_COMMAND_INSTR(0xA1, 0x10),
+ ILI9881C_COMMAND_INSTR(0xA2, 0x1C),
+ ILI9881C_COMMAND_INSTR(0xA3, 0x13),
+ ILI9881C_COMMAND_INSTR(0xA4, 0x15),
+ ILI9881C_COMMAND_INSTR(0xA5, 0x26),
+ ILI9881C_COMMAND_INSTR(0xA6, 0x1A),
+ ILI9881C_COMMAND_INSTR(0xA7, 0x1D),
+ ILI9881C_COMMAND_INSTR(0xA8, 0x67),
+ ILI9881C_COMMAND_INSTR(0xA9, 0x1C),
+ ILI9881C_COMMAND_INSTR(0xAA, 0x29),
+ ILI9881C_COMMAND_INSTR(0xAB, 0x5B),
+ ILI9881C_COMMAND_INSTR(0xAC, 0x26),
+ ILI9881C_COMMAND_INSTR(0xAD, 0x28),
+ ILI9881C_COMMAND_INSTR(0xAE, 0x5C),
+ ILI9881C_COMMAND_INSTR(0xAF, 0x30),
+ ILI9881C_COMMAND_INSTR(0xB0, 0x31),
+ ILI9881C_COMMAND_INSTR(0xB1, 0x2E),
+ ILI9881C_COMMAND_INSTR(0xB2, 0x32),
+ ILI9881C_COMMAND_INSTR(0xB3, 0x00),
+
+ ILI9881C_COMMAND_INSTR(0xC0, 0x00),
+ ILI9881C_COMMAND_INSTR(0xC1, 0x10),
+ ILI9881C_COMMAND_INSTR(0xC2, 0x1C),
+ ILI9881C_COMMAND_INSTR(0xC3, 0x13),
+ ILI9881C_COMMAND_INSTR(0xC4, 0x15),
+ ILI9881C_COMMAND_INSTR(0xC5, 0x26),
+ ILI9881C_COMMAND_INSTR(0xC6, 0x1A),
+ ILI9881C_COMMAND_INSTR(0xC7, 0x1D),
+ ILI9881C_COMMAND_INSTR(0xC8, 0x67),
+ ILI9881C_COMMAND_INSTR(0xC9, 0x1C),
+ ILI9881C_COMMAND_INSTR(0xCA, 0x29),
+ ILI9881C_COMMAND_INSTR(0xCB, 0x5B),
+ ILI9881C_COMMAND_INSTR(0xCC, 0x26),
+ ILI9881C_COMMAND_INSTR(0xCD, 0x28),
+ ILI9881C_COMMAND_INSTR(0xCE, 0x5C),
+ ILI9881C_COMMAND_INSTR(0xCF, 0x30),
+ ILI9881C_COMMAND_INSTR(0xD0, 0x31),
+ ILI9881C_COMMAND_INSTR(0xD1, 0x2E),
+ ILI9881C_COMMAND_INSTR(0xD2, 0x32),
+ ILI9881C_COMMAND_INSTR(0xD3, 0x00),
+ ILI9881C_SWITCH_PAGE_INSTR(0),
+};
+
static inline struct ili9881c *panel_to_ili9881c(struct drm_panel *panel)
{
return container_of(panel, struct ili9881c, panel);
@@ -603,6 +813,23 @@ static const struct drm_display_mode k101_im2byl02_default_mode = {
.height_mm = 217,
};
+static const struct drm_display_mode w552946aba_default_mode = {
+ .clock = 64000,
+
+ .hdisplay = 720,
+ .hsync_start = 720 + 40,
+ .hsync_end = 720 + 40 + 10,
+ .htotal = 720 + 40 + 10 + 40,
+
+ .vdisplay = 1280,
+ .vsync_start = 1280 + 22,
+ .vsync_end = 1280 + 22 + 4,
+ .vtotal = 1280 + 22 + 4 + 11,
+
+ .width_mm = 68,
+ .height_mm = 121,
+};
+
static int ili9881c_get_modes(struct drm_panel *panel,
struct drm_connector *connector)
{
@@ -626,6 +853,8 @@ static int ili9881c_get_modes(struct drm_panel *panel,
connector->display_info.width_mm = mode->width_mm;
connector->display_info.height_mm = mode->height_mm;
+ drm_connector_set_panel_orientation(connector, ctx->orientation);
+
return 1;
}
@@ -653,15 +882,20 @@ static int ili9881c_dsi_probe(struct mipi_dsi_device *dsi)
DRM_MODE_CONNECTOR_DSI);
ctx->power = devm_regulator_get(&dsi->dev, "power");
- if (IS_ERR(ctx->power)) {
- dev_err(&dsi->dev, "Couldn't get our power regulator\n");
- return PTR_ERR(ctx->power);
- }
+ if (IS_ERR(ctx->power))
+ return dev_err_probe(&dsi->dev, PTR_ERR(ctx->power),
+ "Couldn't get our power regulator\n");
- ctx->reset = devm_gpiod_get(&dsi->dev, "reset", GPIOD_OUT_LOW);
- if (IS_ERR(ctx->reset)) {
- dev_err(&dsi->dev, "Couldn't get our reset GPIO\n");
- return PTR_ERR(ctx->reset);
+ ctx->reset = devm_gpiod_get_optional(&dsi->dev, "reset", GPIOD_OUT_LOW);
+ if (IS_ERR(ctx->reset))
+ return dev_err_probe(&dsi->dev, PTR_ERR(ctx->reset),
+ "Couldn't get our reset GPIO\n");
+
+ ret = of_drm_get_panel_orientation(dsi->dev.of_node, &ctx->orientation);
+ if (ret) {
+ dev_err(&dsi->dev, "%pOF: failed to get orientation: %d\n",
+ dsi->dev.of_node, ret);
+ return ret;
}
ret = drm_panel_of_backlight(&ctx->panel);
@@ -670,7 +904,7 @@ static int ili9881c_dsi_probe(struct mipi_dsi_device *dsi)
drm_panel_add(&ctx->panel);
- dsi->mode_flags = MIPI_DSI_MODE_VIDEO_SYNC_PULSE;
+ dsi->mode_flags = ctx->desc->mode_flags;
dsi->format = MIPI_DSI_FMT_RGB888;
dsi->lanes = 4;
@@ -691,17 +925,28 @@ static const struct ili9881c_desc lhr050h41_desc = {
.init = lhr050h41_init,
.init_length = ARRAY_SIZE(lhr050h41_init),
.mode = &lhr050h41_default_mode,
+ .mode_flags = MIPI_DSI_MODE_VIDEO_SYNC_PULSE,
};
static const struct ili9881c_desc k101_im2byl02_desc = {
.init = k101_im2byl02_init,
.init_length = ARRAY_SIZE(k101_im2byl02_init),
.mode = &k101_im2byl02_default_mode,
+ .mode_flags = MIPI_DSI_MODE_VIDEO_SYNC_PULSE,
+};
+
+static const struct ili9881c_desc w552946aba_desc = {
+ .init = w552946ab_init,
+ .init_length = ARRAY_SIZE(w552946ab_init),
+ .mode = &w552946aba_default_mode,
+ .mode_flags = MIPI_DSI_MODE_VIDEO | MIPI_DSI_MODE_VIDEO_BURST |
+ MIPI_DSI_MODE_LPM | MIPI_DSI_MODE_NO_EOT_PACKET,
};
static const struct of_device_id ili9881c_of_match[] = {
{ .compatible = "bananapi,lhr050h41", .data = &lhr050h41_desc },
{ .compatible = "feixin,k101-im2byl02", .data = &k101_im2byl02_desc },
+ { .compatible = "wanchanglong,w552946aba", .data = &w552946aba_desc },
{ }
};
MODULE_DEVICE_TABLE(of, ili9881c_of_match);
diff --git a/drivers/gpu/drm/panel/panel-innolux-ej030na.c b/drivers/gpu/drm/panel/panel-innolux-ej030na.c
index 34b98f7..c558de3 100644
--- a/drivers/gpu/drm/panel/panel-innolux-ej030na.c
+++ b/drivers/gpu/drm/panel/panel-innolux-ej030na.c
@@ -198,16 +198,14 @@ static int ej030na_probe(struct spi_device *spi)
return -EINVAL;
priv->supply = devm_regulator_get(dev, "power");
- if (IS_ERR(priv->supply)) {
- dev_err(dev, "Failed to get power supply\n");
- return PTR_ERR(priv->supply);
- }
+ if (IS_ERR(priv->supply))
+ return dev_err_probe(dev, PTR_ERR(priv->supply),
+ "Failed to get power supply\n");
priv->reset_gpio = devm_gpiod_get(dev, "reset", GPIOD_OUT_HIGH);
- if (IS_ERR(priv->reset_gpio)) {
- dev_err(dev, "Failed to get reset GPIO\n");
- return PTR_ERR(priv->reset_gpio);
- }
+ if (IS_ERR(priv->reset_gpio))
+ return dev_err_probe(dev, PTR_ERR(priv->reset_gpio),
+ "Failed to get reset GPIO\n");
drm_panel_init(&priv->panel, dev, &ej030na_funcs,
DRM_MODE_CONNECTOR_DPI);
diff --git a/drivers/gpu/drm/panel/panel-innolux-p079zca.c b/drivers/gpu/drm/panel/panel-innolux-p079zca.c
index aea3162..f194b62 100644
--- a/drivers/gpu/drm/panel/panel-innolux-p079zca.c
+++ b/drivers/gpu/drm/panel/panel-innolux-p079zca.c
@@ -484,6 +484,7 @@ static void innolux_panel_del(struct innolux_panel *innolux)
static int innolux_panel_probe(struct mipi_dsi_device *dsi)
{
const struct panel_desc *desc;
+ struct innolux_panel *innolux;
int err;
desc = of_device_get_match_data(&dsi->dev);
@@ -495,7 +496,14 @@ static int innolux_panel_probe(struct mipi_dsi_device *dsi)
if (err < 0)
return err;
- return mipi_dsi_attach(dsi);
+ err = mipi_dsi_attach(dsi);
+ if (err < 0) {
+ innolux = mipi_dsi_get_drvdata(dsi);
+ innolux_panel_del(innolux);
+ return err;
+ }
+
+ return 0;
}
static int innolux_panel_remove(struct mipi_dsi_device *dsi)
diff --git a/drivers/gpu/drm/panel/panel-jdi-fhd-r63452.c b/drivers/gpu/drm/panel/panel-jdi-fhd-r63452.c
new file mode 100644
index 0000000..31eafbc
--- /dev/null
+++ b/drivers/gpu/drm/panel/panel-jdi-fhd-r63452.c
@@ -0,0 +1,323 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2021 Raffaele Tranquillini <raffaele.tranquillini@gmail.com>
+ *
+ * Generated using linux-mdss-dsi-panel-driver-generator from Lineage OS device tree:
+ * https://github.com/LineageOS/android_kernel_xiaomi_msm8996/blob/lineage-18.1/arch/arm/boot/dts/qcom/a1-msm8996-mtp.dtsi
+ */
+
+#include <linux/delay.h>
+#include <linux/gpio/consumer.h>
+#include <linux/module.h>
+#include <linux/of.h>
+
+#include <video/mipi_display.h>
+
+#include <drm/drm_mipi_dsi.h>
+#include <drm/drm_modes.h>
+#include <drm/drm_panel.h>
+
+struct jdi_fhd_r63452 {
+ struct drm_panel panel;
+ struct mipi_dsi_device *dsi;
+ struct gpio_desc *reset_gpio;
+ bool prepared;
+};
+
+static inline struct jdi_fhd_r63452 *to_jdi_fhd_r63452(struct drm_panel *panel)
+{
+ return container_of(panel, struct jdi_fhd_r63452, panel);
+}
+
+#define dsi_generic_write_seq(dsi, seq...) do { \
+ static const u8 d[] = { seq }; \
+ int ret; \
+ ret = mipi_dsi_generic_write(dsi, d, ARRAY_SIZE(d)); \
+ if (ret < 0) \
+ return ret; \
+ } while (0)
+
+#define dsi_dcs_write_seq(dsi, seq...) do { \
+ static const u8 d[] = { seq }; \
+ int ret; \
+ ret = mipi_dsi_dcs_write_buffer(dsi, d, ARRAY_SIZE(d)); \
+ if (ret < 0) \
+ return ret; \
+ } while (0)
+
+static void jdi_fhd_r63452_reset(struct jdi_fhd_r63452 *ctx)
+{
+ gpiod_set_value_cansleep(ctx->reset_gpio, 0);
+ usleep_range(10000, 11000);
+ gpiod_set_value_cansleep(ctx->reset_gpio, 1);
+ usleep_range(1000, 2000);
+ gpiod_set_value_cansleep(ctx->reset_gpio, 0);
+ usleep_range(10000, 11000);
+}
+
+static int jdi_fhd_r63452_on(struct jdi_fhd_r63452 *ctx)
+{
+ struct mipi_dsi_device *dsi = ctx->dsi;
+ struct device *dev = &dsi->dev;
+ int ret;
+
+ dsi->mode_flags |= MIPI_DSI_MODE_LPM;
+
+ dsi_generic_write_seq(dsi, 0xb0, 0x00);
+ dsi_generic_write_seq(dsi, 0xd6, 0x01);
+ dsi_generic_write_seq(dsi, 0xec,
+ 0x64, 0xdc, 0xec, 0x3b, 0x52, 0x00, 0x0b, 0x0b,
+ 0x13, 0x15, 0x68, 0x0b, 0xb5);
+ dsi_generic_write_seq(dsi, 0xb0, 0x03);
+
+ ret = mipi_dsi_dcs_set_tear_on(dsi, MIPI_DSI_DCS_TEAR_MODE_VBLANK);
+ if (ret < 0) {
+ dev_err(dev, "Failed to set tear on: %d\n", ret);
+ return ret;
+ }
+
+ dsi_dcs_write_seq(dsi, MIPI_DCS_SET_ADDRESS_MODE, 0x00);
+
+ ret = mipi_dsi_dcs_set_pixel_format(dsi, 0x77);
+ if (ret < 0) {
+ dev_err(dev, "Failed to set pixel format: %d\n", ret);
+ return ret;
+ }
+
+ ret = mipi_dsi_dcs_set_column_address(dsi, 0x0000, 0x0437);
+ if (ret < 0) {
+ dev_err(dev, "Failed to set column address: %d\n", ret);
+ return ret;
+ }
+
+ ret = mipi_dsi_dcs_set_page_address(dsi, 0x0000, 0x077f);
+ if (ret < 0) {
+ dev_err(dev, "Failed to set page address: %d\n", ret);
+ return ret;
+ }
+
+ ret = mipi_dsi_dcs_set_tear_scanline(dsi, 0x0000);
+ if (ret < 0) {
+ dev_err(dev, "Failed to set tear scanline: %d\n", ret);
+ return ret;
+ }
+
+ ret = mipi_dsi_dcs_set_display_brightness(dsi, 0x00ff);
+ if (ret < 0) {
+ dev_err(dev, "Failed to set display brightness: %d\n", ret);
+ return ret;
+ }
+
+ dsi_dcs_write_seq(dsi, MIPI_DCS_WRITE_CONTROL_DISPLAY, 0x24);
+ dsi_dcs_write_seq(dsi, MIPI_DCS_WRITE_POWER_SAVE, 0x00);
+ dsi_dcs_write_seq(dsi, MIPI_DCS_SET_CABC_MIN_BRIGHTNESS, 0x00);
+ dsi_dcs_write_seq(dsi, 0x84, 0x00);
+
+ ret = mipi_dsi_dcs_set_display_on(dsi);
+ if (ret < 0) {
+ dev_err(dev, "Failed to set display on: %d\n", ret);
+ return ret;
+ }
+ msleep(20);
+
+ ret = mipi_dsi_dcs_exit_sleep_mode(dsi);
+ if (ret < 0) {
+ dev_err(dev, "Failed to exit sleep mode: %d\n", ret);
+ return ret;
+ }
+ msleep(80);
+
+ dsi_generic_write_seq(dsi, 0xb0, 0x04);
+ dsi_dcs_write_seq(dsi, 0x84, 0x00);
+ dsi_generic_write_seq(dsi, 0xc8, 0x11);
+ dsi_generic_write_seq(dsi, 0xb0, 0x03);
+
+ return 0;
+}
+
+static int jdi_fhd_r63452_off(struct jdi_fhd_r63452 *ctx)
+{
+ struct mipi_dsi_device *dsi = ctx->dsi;
+ struct device *dev = &dsi->dev;
+ int ret;
+
+ dsi->mode_flags &= ~MIPI_DSI_MODE_LPM;
+
+ dsi_generic_write_seq(dsi, 0xb0, 0x00);
+ dsi_generic_write_seq(dsi, 0xd6, 0x01);
+ dsi_generic_write_seq(dsi, 0xec,
+ 0x64, 0xdc, 0xec, 0x3b, 0x52, 0x00, 0x0b, 0x0b,
+ 0x13, 0x15, 0x68, 0x0b, 0x95);
+ dsi_generic_write_seq(dsi, 0xb0, 0x03);
+
+ ret = mipi_dsi_dcs_set_display_off(dsi);
+ if (ret < 0) {
+ dev_err(dev, "Failed to set display off: %d\n", ret);
+ return ret;
+ }
+ usleep_range(2000, 3000);
+
+ ret = mipi_dsi_dcs_enter_sleep_mode(dsi);
+ if (ret < 0) {
+ dev_err(dev, "Failed to enter sleep mode: %d\n", ret);
+ return ret;
+ }
+ msleep(120);
+
+ return 0;
+}
+
+static int jdi_fhd_r63452_prepare(struct drm_panel *panel)
+{
+ struct jdi_fhd_r63452 *ctx = to_jdi_fhd_r63452(panel);
+ struct device *dev = &ctx->dsi->dev;
+ int ret;
+
+ if (ctx->prepared)
+ return 0;
+
+ jdi_fhd_r63452_reset(ctx);
+
+ ret = jdi_fhd_r63452_on(ctx);
+ if (ret < 0) {
+ dev_err(dev, "Failed to initialize panel: %d\n", ret);
+ gpiod_set_value_cansleep(ctx->reset_gpio, 1);
+ return ret;
+ }
+
+ ctx->prepared = true;
+ return 0;
+}
+
+static int jdi_fhd_r63452_unprepare(struct drm_panel *panel)
+{
+ struct jdi_fhd_r63452 *ctx = to_jdi_fhd_r63452(panel);
+ struct device *dev = &ctx->dsi->dev;
+ int ret;
+
+ if (!ctx->prepared)
+ return 0;
+
+ ret = jdi_fhd_r63452_off(ctx);
+ if (ret < 0)
+ dev_err(dev, "Failed to un-initialize panel: %d\n", ret);
+
+ gpiod_set_value_cansleep(ctx->reset_gpio, 1);
+
+ ctx->prepared = false;
+ return 0;
+}
+
+static const struct drm_display_mode jdi_fhd_r63452_mode = {
+ .clock = (1080 + 120 + 16 + 40) * (1920 + 4 + 2 + 4) * 60 / 1000,
+ .hdisplay = 1080,
+ .hsync_start = 1080 + 120,
+ .hsync_end = 1080 + 120 + 16,
+ .htotal = 1080 + 120 + 16 + 40,
+ .vdisplay = 1920,
+ .vsync_start = 1920 + 4,
+ .vsync_end = 1920 + 4 + 2,
+ .vtotal = 1920 + 4 + 2 + 4,
+ .width_mm = 64,
+ .height_mm = 114,
+};
+
+static int jdi_fhd_r63452_get_modes(struct drm_panel *panel,
+ struct drm_connector *connector)
+{
+ struct drm_display_mode *mode;
+
+ mode = drm_mode_duplicate(connector->dev, &jdi_fhd_r63452_mode);
+ if (!mode)
+ return -ENOMEM;
+
+ drm_mode_set_name(mode);
+
+ mode->type = DRM_MODE_TYPE_DRIVER | DRM_MODE_TYPE_PREFERRED;
+ connector->display_info.width_mm = mode->width_mm;
+ connector->display_info.height_mm = mode->height_mm;
+ drm_mode_probed_add(connector, mode);
+
+ return 1;
+}
+
+static const struct drm_panel_funcs jdi_fhd_r63452_panel_funcs = {
+ .prepare = jdi_fhd_r63452_prepare,
+ .unprepare = jdi_fhd_r63452_unprepare,
+ .get_modes = jdi_fhd_r63452_get_modes,
+};
+
+static int jdi_fhd_r63452_probe(struct mipi_dsi_device *dsi)
+{
+ struct device *dev = &dsi->dev;
+ struct jdi_fhd_r63452 *ctx;
+ int ret;
+
+ ctx = devm_kzalloc(dev, sizeof(*ctx), GFP_KERNEL);
+ if (!ctx)
+ return -ENOMEM;
+
+ ctx->reset_gpio = devm_gpiod_get(dev, "reset", GPIOD_OUT_HIGH);
+ if (IS_ERR(ctx->reset_gpio))
+ return dev_err_probe(dev, PTR_ERR(ctx->reset_gpio),
+ "Failed to get reset-gpios\n");
+
+ ctx->dsi = dsi;
+ mipi_dsi_set_drvdata(dsi, ctx);
+
+ dsi->lanes = 4;
+ dsi->format = MIPI_DSI_FMT_RGB888;
+ dsi->mode_flags = MIPI_DSI_MODE_VIDEO_BURST |
+ MIPI_DSI_CLOCK_NON_CONTINUOUS;
+
+ drm_panel_init(&ctx->panel, dev, &jdi_fhd_r63452_panel_funcs,
+ DRM_MODE_CONNECTOR_DSI);
+
+ ret = drm_panel_of_backlight(&ctx->panel);
+ if (ret)
+ return dev_err_probe(dev, ret, "Failed to get backlight\n");
+
+ drm_panel_add(&ctx->panel);
+
+ ret = mipi_dsi_attach(dsi);
+ if (ret < 0) {
+ dev_err(dev, "Failed to attach to DSI host: %d\n", ret);
+ return ret;
+ }
+
+ return 0;
+}
+
+static int jdi_fhd_r63452_remove(struct mipi_dsi_device *dsi)
+{
+ struct jdi_fhd_r63452 *ctx = mipi_dsi_get_drvdata(dsi);
+ int ret;
+
+ ret = mipi_dsi_detach(dsi);
+ if (ret < 0)
+ dev_err(&dsi->dev, "Failed to detach from DSI host: %d\n", ret);
+
+ drm_panel_remove(&ctx->panel);
+
+ return 0;
+}
+
+static const struct of_device_id jdi_fhd_r63452_of_match[] = {
+ { .compatible = "jdi,fhd-r63452" },
+ { /* sentinel */ }
+};
+MODULE_DEVICE_TABLE(of, jdi_fhd_r63452_of_match);
+
+static struct mipi_dsi_driver jdi_fhd_r63452_driver = {
+ .probe = jdi_fhd_r63452_probe,
+ .remove = jdi_fhd_r63452_remove,
+ .driver = {
+ .name = "panel-jdi-fhd-r63452",
+ .of_match_table = jdi_fhd_r63452_of_match,
+ },
+};
+module_mipi_dsi_driver(jdi_fhd_r63452_driver);
+
+MODULE_AUTHOR("Raffaele Tranquillini <raffaele.tranquillini@gmail.com>");
+MODULE_DESCRIPTION("DRM driver for JDI FHD R63452 DSI panel, command mode");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/gpu/drm/panel/panel-jdi-lt070me05000.c b/drivers/gpu/drm/panel/panel-jdi-lt070me05000.c
index 733010b..3c86ad2 100644
--- a/drivers/gpu/drm/panel/panel-jdi-lt070me05000.c
+++ b/drivers/gpu/drm/panel/panel-jdi-lt070me05000.c
@@ -473,7 +473,13 @@ static int jdi_panel_probe(struct mipi_dsi_device *dsi)
if (ret < 0)
return ret;
- return mipi_dsi_attach(dsi);
+ ret = mipi_dsi_attach(dsi);
+ if (ret < 0) {
+ jdi_panel_del(jdi);
+ return ret;
+ }
+
+ return 0;
}
static int jdi_panel_remove(struct mipi_dsi_device *dsi)
diff --git a/drivers/gpu/drm/panel/panel-kingdisplay-kd097d04.c b/drivers/gpu/drm/panel/panel-kingdisplay-kd097d04.c
index 86e4213..daccb1f 100644
--- a/drivers/gpu/drm/panel/panel-kingdisplay-kd097d04.c
+++ b/drivers/gpu/drm/panel/panel-kingdisplay-kd097d04.c
@@ -406,7 +406,13 @@ static int kingdisplay_panel_probe(struct mipi_dsi_device *dsi)
if (err < 0)
return err;
- return mipi_dsi_attach(dsi);
+ err = mipi_dsi_attach(dsi);
+ if (err < 0) {
+ kingdisplay_panel_del(kingdisplay);
+ return err;
+ }
+
+ return 0;
}
static int kingdisplay_panel_remove(struct mipi_dsi_device *dsi)
diff --git a/drivers/gpu/drm/panel/panel-lvds.c b/drivers/gpu/drm/panel/panel-lvds.c
index 59a8d99..27a1c99 100644
--- a/drivers/gpu/drm/panel/panel-lvds.c
+++ b/drivers/gpu/drm/panel/panel-lvds.c
@@ -20,6 +20,7 @@
#include <video/videomode.h>
#include <drm/drm_crtc.h>
+#include <drm/drm_of.h>
#include <drm/drm_panel.h>
struct panel_lvds {
@@ -116,7 +117,6 @@ static int panel_lvds_parse_dt(struct panel_lvds *lvds)
{
struct device_node *np = lvds->dev->of_node;
struct display_timing timing;
- const char *mapping;
int ret;
ret = of_drm_get_panel_orientation(np, &lvds->orientation);
@@ -149,24 +149,14 @@ static int panel_lvds_parse_dt(struct panel_lvds *lvds)
of_property_read_string(np, "label", &lvds->label);
- ret = of_property_read_string(np, "data-mapping", &mapping);
+ ret = drm_of_lvds_get_data_mapping(np);
if (ret < 0) {
dev_err(lvds->dev, "%pOF: invalid or missing %s DT property\n",
np, "data-mapping");
- return -ENODEV;
+ return ret;
}
- if (!strcmp(mapping, "jeida-18")) {
- lvds->bus_format = MEDIA_BUS_FMT_RGB666_1X7X3_SPWG;
- } else if (!strcmp(mapping, "jeida-24")) {
- lvds->bus_format = MEDIA_BUS_FMT_RGB888_1X7X4_JEIDA;
- } else if (!strcmp(mapping, "vesa-24")) {
- lvds->bus_format = MEDIA_BUS_FMT_RGB888_1X7X4_SPWG;
- } else {
- dev_err(lvds->dev, "%pOF: invalid or missing %s DT property\n",
- np, "data-mapping");
- return -EINVAL;
- }
+ lvds->bus_format = ret;
lvds->data_mirror = of_property_read_bool(np, "data-mirror");
diff --git a/drivers/gpu/drm/panel/panel-novatek-nt35950.c b/drivers/gpu/drm/panel/panel-novatek-nt35950.c
new file mode 100644
index 0000000..288c7fa8
--- /dev/null
+++ b/drivers/gpu/drm/panel/panel-novatek-nt35950.c
@@ -0,0 +1,702 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Novatek NT35950 DriverIC panels driver
+ *
+ * Copyright (c) 2021 AngeloGioacchino Del Regno
+ * <angelogioacchino.delregno@somainline.org>
+ */
+#include <linux/delay.h>
+#include <linux/gpio/consumer.h>
+#include <linux/module.h>
+#include <linux/of_device.h>
+#include <linux/of_graph.h>
+#include <linux/regulator/consumer.h>
+
+#include <drm/drm_connector.h>
+#include <drm/drm_crtc.h>
+#include <drm/drm_mipi_dsi.h>
+#include <drm/drm_modes.h>
+#include <drm/drm_panel.h>
+
+#define MCS_CMD_MAUCCTR 0xf0 /* Manufacturer command enable */
+#define MCS_PARAM_SCALER_FUNCTION 0x58 /* Scale-up function */
+#define MCS_PARAM_SCALEUP_MODE 0xc9
+ #define MCS_SCALEUP_SIMPLE 0x0
+ #define MCS_SCALEUP_BILINEAR BIT(0)
+ #define MCS_SCALEUP_DUPLICATE (BIT(0) | BIT(4))
+
+/* VESA Display Stream Compression param */
+#define MCS_PARAM_VESA_DSC_ON 0x03
+
+/* Data Compression mode */
+#define MCS_PARAM_DATA_COMPRESSION 0x90
+ #define MCS_DATA_COMPRESSION_NONE 0x00
+ #define MCS_DATA_COMPRESSION_FBC 0x02
+ #define MCS_DATA_COMPRESSION_DSC 0x03
+
+/* Display Output control */
+#define MCS_PARAM_DISP_OUTPUT_CTRL 0xb4
+ #define MCS_DISP_OUT_SRAM_EN BIT(0)
+ #define MCS_DISP_OUT_VIDEO_MODE BIT(4)
+
+/* VESA Display Stream Compression setting */
+#define MCS_PARAM_VESA_DSC_SETTING 0xc0
+
+/* SubPixel Rendering (SPR) */
+#define MCS_PARAM_SPR_EN 0xe3
+#define MCS_PARAM_SPR_MODE 0xef
+ #define MCS_SPR_MODE_YYG_RAINBOW_RGB 0x01
+
+#define NT35950_VREG_MAX 4
+
+struct nt35950 {
+ struct drm_panel panel;
+ struct drm_connector *connector;
+ struct mipi_dsi_device *dsi[2];
+ struct regulator_bulk_data vregs[NT35950_VREG_MAX];
+ struct gpio_desc *reset_gpio;
+ const struct nt35950_panel_desc *desc;
+
+ int cur_mode;
+ u8 last_page;
+ bool prepared;
+};
+
+struct nt35950_panel_mode {
+ const struct drm_display_mode mode;
+
+ bool enable_sram;
+ bool is_video_mode;
+ u8 scaler_on;
+ u8 scaler_mode;
+ u8 compression;
+ u8 spr_en;
+ u8 spr_mode;
+};
+
+struct nt35950_panel_desc {
+ const char *model_name;
+ const struct mipi_dsi_device_info dsi_info;
+ const struct nt35950_panel_mode *mode_data;
+
+ bool is_dual_dsi;
+ u8 num_lanes;
+ u8 num_modes;
+};
+
+static inline struct nt35950 *to_nt35950(struct drm_panel *panel)
+{
+ return container_of(panel, struct nt35950, panel);
+}
+
+#define dsi_dcs_write_seq(dsi, seq...) do { \
+ static const u8 d[] = { seq }; \
+ int ret; \
+ ret = mipi_dsi_dcs_write_buffer(dsi, d, ARRAY_SIZE(d)); \
+ if (ret < 0) \
+ return ret; \
+ } while (0)
+
+static void nt35950_reset(struct nt35950 *nt)
+{
+ gpiod_set_value_cansleep(nt->reset_gpio, 1);
+ usleep_range(12000, 13000);
+ gpiod_set_value_cansleep(nt->reset_gpio, 0);
+ usleep_range(300, 400);
+ gpiod_set_value_cansleep(nt->reset_gpio, 1);
+ usleep_range(12000, 13000);
+}
+
+/*
+ * nt35950_set_cmd2_page - Select manufacturer control (CMD2) page
+ * @nt: Main driver structure
+ * @page: Page number (0-7)
+ *
+ * Return: Number of transferred bytes or negative number on error
+ */
+static int nt35950_set_cmd2_page(struct nt35950 *nt, u8 page)
+{
+ const u8 mauc_cmd2_page[] = { MCS_CMD_MAUCCTR, 0x55, 0xaa, 0x52,
+ 0x08, page };
+ int ret;
+
+ ret = mipi_dsi_dcs_write_buffer(nt->dsi[0], mauc_cmd2_page,
+ ARRAY_SIZE(mauc_cmd2_page));
+ if (ret < 0)
+ return ret;
+
+ nt->last_page = page;
+ return 0;
+}
+
+/*
+ * nt35950_set_data_compression - Set data compression mode
+ * @nt: Main driver structure
+ * @comp_mode: Compression mode
+ *
+ * Return: Number of transferred bytes or negative number on error
+ */
+static int nt35950_set_data_compression(struct nt35950 *nt, u8 comp_mode)
+{
+ u8 cmd_data_compression[] = { MCS_PARAM_DATA_COMPRESSION, comp_mode };
+ u8 cmd_vesa_dsc_on[] = { MCS_PARAM_VESA_DSC_ON, !!comp_mode };
+ u8 cmd_vesa_dsc_setting[] = { MCS_PARAM_VESA_DSC_SETTING, 0x03 };
+ u8 last_page = nt->last_page;
+ int ret;
+
+ /* Set CMD2 Page 0 if we're not there yet */
+ if (last_page != 0) {
+ ret = nt35950_set_cmd2_page(nt, 0);
+ if (ret < 0)
+ return ret;
+ }
+
+ ret = mipi_dsi_dcs_write_buffer(nt->dsi[0], cmd_data_compression,
+ ARRAY_SIZE(cmd_data_compression));
+ if (ret < 0)
+ return ret;
+
+ ret = mipi_dsi_dcs_write_buffer(nt->dsi[0], cmd_vesa_dsc_on,
+ ARRAY_SIZE(cmd_vesa_dsc_on));
+ if (ret < 0)
+ return ret;
+
+ /* Set the vesa dsc setting on Page 4 */
+ ret = nt35950_set_cmd2_page(nt, 4);
+ if (ret < 0)
+ return ret;
+
+ /* Display Stream Compression setting, always 0x03 */
+ ret = mipi_dsi_dcs_write_buffer(nt->dsi[0], cmd_vesa_dsc_setting,
+ ARRAY_SIZE(cmd_vesa_dsc_setting));
+ if (ret < 0)
+ return ret;
+
+ /* Get back to the previously set page */
+ return nt35950_set_cmd2_page(nt, last_page);
+}
+
+/*
+ * nt35950_set_scaler - Enable/disable resolution upscaling
+ * @nt: Main driver structure
+ * @scale_up: Scale up function control
+ *
+ * Return: Number of transferred bytes or negative number on error
+ */
+static int nt35950_set_scaler(struct nt35950 *nt, u8 scale_up)
+{
+ u8 cmd_scaler[] = { MCS_PARAM_SCALER_FUNCTION, scale_up };
+
+ return mipi_dsi_dcs_write_buffer(nt->dsi[0], cmd_scaler,
+ ARRAY_SIZE(cmd_scaler));
+}
+
+/*
+ * nt35950_set_scale_mode - Resolution upscaling mode
+ * @nt: Main driver structure
+ * @mode: Scaler mode (MCS_DATA_COMPRESSION_*)
+ *
+ * Return: Number of transferred bytes or negative number on error
+ */
+static int nt35950_set_scale_mode(struct nt35950 *nt, u8 mode)
+{
+ u8 cmd_scaler[] = { MCS_PARAM_SCALEUP_MODE, mode };
+
+ return mipi_dsi_dcs_write_buffer(nt->dsi[0], cmd_scaler,
+ ARRAY_SIZE(cmd_scaler));
+}
+
+/*
+ * nt35950_inject_black_image - Display a completely black image
+ * @nt: Main driver structure
+ *
+ * After IC setup, the attached panel may show random data
+ * due to driveric behavior changes (resolution, compression,
+ * scaling, etc). This function, called after parameters setup,
+ * makes the driver ic to output a completely black image to
+ * the display.
+ * It makes sense to push a black image before sending the sleep-out
+ * and display-on commands.
+ *
+ * Return: Number of transferred bytes or negative number on error
+ */
+static int nt35950_inject_black_image(struct nt35950 *nt)
+{
+ const u8 cmd0_black_img[] = { 0x6f, 0x01 };
+ const u8 cmd1_black_img[] = { 0xf3, 0x10 };
+ u8 cmd_test[] = { 0xff, 0xaa, 0x55, 0xa5, 0x80 };
+ int ret;
+
+ /* Enable test command */
+ ret = mipi_dsi_dcs_write_buffer(nt->dsi[0], cmd_test, ARRAY_SIZE(cmd_test));
+ if (ret < 0)
+ return ret;
+
+ /* Send a black image */
+ ret = mipi_dsi_dcs_write_buffer(nt->dsi[0], cmd0_black_img,
+ ARRAY_SIZE(cmd0_black_img));
+ if (ret < 0)
+ return ret;
+ ret = mipi_dsi_dcs_write_buffer(nt->dsi[0], cmd1_black_img,
+ ARRAY_SIZE(cmd1_black_img));
+ if (ret < 0)
+ return ret;
+
+ /* Disable test command */
+ cmd_test[ARRAY_SIZE(cmd_test) - 1] = 0x00;
+ return mipi_dsi_dcs_write_buffer(nt->dsi[0], cmd_test, ARRAY_SIZE(cmd_test));
+}
+
+/*
+ * nt35950_set_dispout - Set Display Output register parameters
+ * @nt: Main driver structure
+ *
+ * Return: Number of transferred bytes or negative number on error
+ */
+static int nt35950_set_dispout(struct nt35950 *nt)
+{
+ u8 cmd_dispout[] = { MCS_PARAM_DISP_OUTPUT_CTRL, 0x00 };
+ const struct nt35950_panel_mode *mode_data = nt->desc->mode_data;
+
+ if (mode_data[nt->cur_mode].is_video_mode)
+ cmd_dispout[1] |= MCS_DISP_OUT_VIDEO_MODE;
+ if (mode_data[nt->cur_mode].enable_sram)
+ cmd_dispout[1] |= MCS_DISP_OUT_SRAM_EN;
+
+ return mipi_dsi_dcs_write_buffer(nt->dsi[0], cmd_dispout,
+ ARRAY_SIZE(cmd_dispout));
+}
+
+static int nt35950_get_current_mode(struct nt35950 *nt)
+{
+ struct drm_connector *connector = nt->connector;
+ struct drm_crtc_state *crtc_state;
+ int i;
+
+ /* Return the default (first) mode if no info available yet */
+ if (!connector->state || !connector->state->crtc)
+ return 0;
+
+ crtc_state = connector->state->crtc->state;
+
+ for (i = 0; i < nt->desc->num_modes; i++) {
+ if (drm_mode_match(&crtc_state->mode,
+ &nt->desc->mode_data[i].mode,
+ DRM_MODE_MATCH_TIMINGS | DRM_MODE_MATCH_CLOCK))
+ return i;
+ }
+
+ return 0;
+}
+
+static int nt35950_on(struct nt35950 *nt)
+{
+ const struct nt35950_panel_mode *mode_data = nt->desc->mode_data;
+ struct mipi_dsi_device *dsi = nt->dsi[0];
+ struct device *dev = &dsi->dev;
+ int ret;
+
+ nt->cur_mode = nt35950_get_current_mode(nt);
+ nt->dsi[0]->mode_flags |= MIPI_DSI_MODE_LPM;
+ nt->dsi[1]->mode_flags |= MIPI_DSI_MODE_LPM;
+
+ ret = nt35950_set_cmd2_page(nt, 0);
+ if (ret < 0)
+ return ret;
+
+ ret = nt35950_set_data_compression(nt, mode_data[nt->cur_mode].compression);
+ if (ret < 0)
+ return ret;
+
+ ret = nt35950_set_scale_mode(nt, mode_data[nt->cur_mode].scaler_mode);
+ if (ret < 0)
+ return ret;
+
+ ret = nt35950_set_scaler(nt, mode_data[nt->cur_mode].scaler_on);
+ if (ret < 0)
+ return ret;
+
+ ret = nt35950_set_dispout(nt);
+ if (ret < 0)
+ return ret;
+
+ ret = mipi_dsi_dcs_set_tear_on(dsi, MIPI_DSI_DCS_TEAR_MODE_VBLANK);
+ if (ret < 0) {
+ dev_err(dev, "Failed to set tear on: %d\n", ret);
+ return ret;
+ }
+
+ ret = mipi_dsi_dcs_set_tear_scanline(dsi, 0);
+ if (ret < 0) {
+ dev_err(dev, "Failed to set tear scanline: %d\n", ret);
+ return ret;
+ }
+
+ /* CMD2 Page 1 */
+ ret = nt35950_set_cmd2_page(nt, 1);
+ if (ret < 0)
+ return ret;
+
+ /* Unknown command */
+ dsi_dcs_write_seq(dsi, 0xd4, 0x88, 0x88);
+
+ /* CMD2 Page 7 */
+ ret = nt35950_set_cmd2_page(nt, 7);
+ if (ret < 0)
+ return ret;
+
+ /* Enable SubPixel Rendering */
+ dsi_dcs_write_seq(dsi, MCS_PARAM_SPR_EN, 0x01);
+
+ /* SPR Mode: YYG Rainbow-RGB */
+ dsi_dcs_write_seq(dsi, MCS_PARAM_SPR_MODE, MCS_SPR_MODE_YYG_RAINBOW_RGB);
+
+ /* CMD3 */
+ ret = nt35950_inject_black_image(nt);
+ if (ret < 0)
+ return ret;
+
+ ret = mipi_dsi_dcs_exit_sleep_mode(dsi);
+ if (ret < 0)
+ return ret;
+ msleep(120);
+
+ ret = mipi_dsi_dcs_set_display_on(dsi);
+ if (ret < 0)
+ return ret;
+ msleep(120);
+
+ nt->dsi[0]->mode_flags &= ~MIPI_DSI_MODE_LPM;
+ nt->dsi[1]->mode_flags &= ~MIPI_DSI_MODE_LPM;
+
+ return 0;
+}
+
+static int nt35950_off(struct nt35950 *nt)
+{
+ struct device *dev = &nt->dsi[0]->dev;
+ int ret;
+
+ ret = mipi_dsi_dcs_set_display_off(nt->dsi[0]);
+ if (ret < 0) {
+ dev_err(dev, "Failed to set display off: %d\n", ret);
+ goto set_lpm;
+ }
+ usleep_range(10000, 11000);
+
+ ret = mipi_dsi_dcs_enter_sleep_mode(nt->dsi[0]);
+ if (ret < 0) {
+ dev_err(dev, "Failed to enter sleep mode: %d\n", ret);
+ goto set_lpm;
+ }
+ msleep(150);
+
+set_lpm:
+ nt->dsi[0]->mode_flags |= MIPI_DSI_MODE_LPM;
+ nt->dsi[1]->mode_flags |= MIPI_DSI_MODE_LPM;
+
+ return 0;
+}
+
+static int nt35950_sharp_init_vregs(struct nt35950 *nt, struct device *dev)
+{
+ int ret;
+
+ nt->vregs[0].supply = "vddio";
+ nt->vregs[1].supply = "avdd";
+ nt->vregs[2].supply = "avee";
+ nt->vregs[3].supply = "dvdd";
+ ret = devm_regulator_bulk_get(dev, ARRAY_SIZE(nt->vregs),
+ nt->vregs);
+ if (ret < 0)
+ return ret;
+
+ ret = regulator_is_supported_voltage(nt->vregs[0].consumer,
+ 1750000, 1950000);
+ if (!ret)
+ return -EINVAL;
+ ret = regulator_is_supported_voltage(nt->vregs[1].consumer,
+ 5200000, 5900000);
+ if (!ret)
+ return -EINVAL;
+ /* AVEE is negative: -5.90V to -5.20V */
+ ret = regulator_is_supported_voltage(nt->vregs[2].consumer,
+ 5200000, 5900000);
+ if (!ret)
+ return -EINVAL;
+
+ ret = regulator_is_supported_voltage(nt->vregs[3].consumer,
+ 1300000, 1400000);
+ if (!ret)
+ return -EINVAL;
+
+ return 0;
+}
+
+static int nt35950_prepare(struct drm_panel *panel)
+{
+ struct nt35950 *nt = to_nt35950(panel);
+ struct device *dev = &nt->dsi[0]->dev;
+ int ret;
+
+ if (nt->prepared)
+ return 0;
+
+ ret = regulator_enable(nt->vregs[0].consumer);
+ if (ret)
+ return ret;
+ usleep_range(2000, 5000);
+
+ ret = regulator_enable(nt->vregs[3].consumer);
+ if (ret)
+ goto end;
+ usleep_range(15000, 18000);
+
+ ret = regulator_enable(nt->vregs[1].consumer);
+ if (ret)
+ goto end;
+
+ ret = regulator_enable(nt->vregs[2].consumer);
+ if (ret)
+ goto end;
+ usleep_range(12000, 13000);
+
+ nt35950_reset(nt);
+
+ ret = nt35950_on(nt);
+ if (ret < 0) {
+ dev_err(dev, "Failed to initialize panel: %d\n", ret);
+ goto end;
+ }
+ nt->prepared = true;
+
+end:
+ if (ret < 0) {
+ regulator_bulk_disable(ARRAY_SIZE(nt->vregs), nt->vregs);
+ return ret;
+ }
+
+ return 0;
+}
+
+static int nt35950_unprepare(struct drm_panel *panel)
+{
+ struct nt35950 *nt = to_nt35950(panel);
+ struct device *dev = &nt->dsi[0]->dev;
+ int ret;
+
+ if (!nt->prepared)
+ return 0;
+
+ ret = nt35950_off(nt);
+ if (ret < 0)
+ dev_err(dev, "Failed to deinitialize panel: %d\n", ret);
+
+ gpiod_set_value_cansleep(nt->reset_gpio, 0);
+ regulator_bulk_disable(ARRAY_SIZE(nt->vregs), nt->vregs);
+
+ nt->prepared = false;
+ return 0;
+}
+
+static int nt35950_get_modes(struct drm_panel *panel,
+ struct drm_connector *connector)
+{
+ struct nt35950 *nt = to_nt35950(panel);
+ int i;
+
+ for (i = 0; i < nt->desc->num_modes; i++) {
+ struct drm_display_mode *mode;
+
+ mode = drm_mode_duplicate(connector->dev,
+ &nt->desc->mode_data[i].mode);
+ if (!mode)
+ return -ENOMEM;
+
+ drm_mode_set_name(mode);
+
+ mode->type |= DRM_MODE_TYPE_DRIVER;
+ if (nt->desc->num_modes == 1)
+ mode->type |= DRM_MODE_TYPE_PREFERRED;
+
+ drm_mode_probed_add(connector, mode);
+ }
+
+ connector->display_info.bpc = 8;
+ connector->display_info.height_mm = nt->desc->mode_data[0].mode.height_mm;
+ connector->display_info.width_mm = nt->desc->mode_data[0].mode.width_mm;
+ nt->connector = connector;
+
+ return nt->desc->num_modes;
+}
+
+static const struct drm_panel_funcs nt35950_panel_funcs = {
+ .prepare = nt35950_prepare,
+ .unprepare = nt35950_unprepare,
+ .get_modes = nt35950_get_modes,
+};
+
+static int nt35950_probe(struct mipi_dsi_device *dsi)
+{
+ struct device *dev = &dsi->dev;
+ struct device_node *dsi_r;
+ struct mipi_dsi_host *dsi_r_host;
+ struct nt35950 *nt;
+ const struct mipi_dsi_device_info *info;
+ int i, num_dsis = 1, ret;
+
+ nt = devm_kzalloc(dev, sizeof(*nt), GFP_KERNEL);
+ if (!nt)
+ return -ENOMEM;
+
+ ret = nt35950_sharp_init_vregs(nt, dev);
+ if (ret)
+ return dev_err_probe(dev, ret, "Regulator init failure.\n");
+
+ nt->desc = of_device_get_match_data(dev);
+ if (!nt->desc)
+ return -ENODEV;
+
+ nt->reset_gpio = devm_gpiod_get(dev, "reset", GPIOD_ASIS);
+ if (IS_ERR(nt->reset_gpio)) {
+ return dev_err_probe(dev, PTR_ERR(nt->reset_gpio),
+ "Failed to get reset gpio\n");
+ }
+
+ /* If the panel is connected on two DSIs then DSI0 left, DSI1 right */
+ if (nt->desc->is_dual_dsi) {
+ info = &nt->desc->dsi_info;
+ dsi_r = of_graph_get_remote_node(dsi->dev.of_node, 1, -1);
+ if (!dsi_r) {
+ dev_err(dev, "Cannot get secondary DSI node.\n");
+ return -ENODEV;
+ }
+ dsi_r_host = of_find_mipi_dsi_host_by_node(dsi_r);
+ of_node_put(dsi_r);
+ if (!dsi_r_host) {
+ dev_err(dev, "Cannot get secondary DSI host\n");
+ return -EPROBE_DEFER;
+ }
+
+ nt->dsi[1] = mipi_dsi_device_register_full(dsi_r_host, info);
+ if (!nt->dsi[1]) {
+ dev_err(dev, "Cannot get secondary DSI node\n");
+ return -ENODEV;
+ }
+ num_dsis++;
+ }
+
+ nt->dsi[0] = dsi;
+ mipi_dsi_set_drvdata(dsi, nt);
+
+ drm_panel_init(&nt->panel, dev, &nt35950_panel_funcs,
+ DRM_MODE_CONNECTOR_DSI);
+
+ ret = drm_panel_of_backlight(&nt->panel);
+ if (ret)
+ return dev_err_probe(dev, ret, "Failed to get backlight\n");
+
+ drm_panel_add(&nt->panel);
+
+ for (i = 0; i < num_dsis; i++) {
+ nt->dsi[i]->lanes = nt->desc->num_lanes;
+ nt->dsi[i]->format = MIPI_DSI_FMT_RGB888;
+
+ nt->dsi[i]->mode_flags = MIPI_DSI_CLOCK_NON_CONTINUOUS |
+ MIPI_DSI_MODE_LPM;
+
+ if (nt->desc->mode_data[0].is_video_mode)
+ nt->dsi[i]->mode_flags |= MIPI_DSI_MODE_VIDEO;
+
+ ret = mipi_dsi_attach(nt->dsi[i]);
+ if (ret < 0) {
+ return dev_err_probe(dev, ret,
+ "Cannot attach to DSI%d host.\n", i);
+ }
+ }
+
+ /* Make sure to set RESX LOW before starting the power-on sequence */
+ gpiod_set_value_cansleep(nt->reset_gpio, 0);
+ return 0;
+}
+
+static int nt35950_remove(struct mipi_dsi_device *dsi)
+{
+ struct nt35950 *nt = mipi_dsi_get_drvdata(dsi);
+ int ret;
+
+ ret = mipi_dsi_detach(nt->dsi[0]);
+ if (ret < 0)
+ dev_err(&dsi->dev,
+ "Failed to detach from DSI0 host: %d\n", ret);
+
+ if (nt->dsi[1]) {
+ ret = mipi_dsi_detach(nt->dsi[1]);
+ if (ret < 0)
+ dev_err(&dsi->dev,
+ "Failed to detach from DSI1 host: %d\n", ret);
+ mipi_dsi_device_unregister(nt->dsi[1]);
+ }
+
+ drm_panel_remove(&nt->panel);
+
+ return 0;
+}
+
+static const struct nt35950_panel_mode sharp_ls055d1sx04_modes[] = {
+ {
+ /* 1920x1080 60Hz no compression */
+ .mode = {
+ .clock = 214537,
+ .hdisplay = 1080,
+ .hsync_start = 1080 + 400,
+ .hsync_end = 1080 + 400 + 40,
+ .htotal = 1080 + 400 + 40 + 300,
+ .vdisplay = 1920,
+ .vsync_start = 1920 + 12,
+ .vsync_end = 1920 + 12 + 2,
+ .vtotal = 1920 + 12 + 2 + 10,
+ .width_mm = 68,
+ .height_mm = 121,
+ },
+ .compression = MCS_DATA_COMPRESSION_NONE,
+ .enable_sram = true,
+ .is_video_mode = false,
+ .scaler_on = 1,
+ .scaler_mode = MCS_SCALEUP_DUPLICATE,
+ },
+ /* TODO: Add 2160x3840 60Hz when DSC is supported */
+};
+
+static const struct nt35950_panel_desc sharp_ls055d1sx04 = {
+ .model_name = "Sharp LS055D1SX04",
+ .dsi_info = {
+ .type = "LS055D1SX04",
+ .channel = 0,
+ .node = NULL,
+ },
+ .mode_data = sharp_ls055d1sx04_modes,
+ .num_modes = ARRAY_SIZE(sharp_ls055d1sx04_modes),
+ .is_dual_dsi = true,
+ .num_lanes = 4,
+};
+
+static const struct of_device_id nt35950_of_match[] = {
+ { .compatible = "sharp,ls055d1sx04", .data = &sharp_ls055d1sx04 },
+ { }
+};
+MODULE_DEVICE_TABLE(of, nt35950_of_match);
+
+static struct mipi_dsi_driver nt35950_driver = {
+ .probe = nt35950_probe,
+ .remove = nt35950_remove,
+ .driver = {
+ .name = "panel-novatek-nt35950",
+ .of_match_table = nt35950_of_match,
+ },
+};
+module_mipi_dsi_driver(nt35950_driver);
+
+MODULE_AUTHOR("AngeloGioacchino Del Regno <angelogioacchino.delregno@somainline.org>");
+MODULE_DESCRIPTION("Novatek NT35950 DriverIC panels driver");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/gpu/drm/panel/panel-novatek-nt36672a.c b/drivers/gpu/drm/panel/panel-novatek-nt36672a.c
index 533cd39..231f371 100644
--- a/drivers/gpu/drm/panel/panel-novatek-nt36672a.c
+++ b/drivers/gpu/drm/panel/panel-novatek-nt36672a.c
@@ -618,7 +618,7 @@ static int nt36672a_panel_add(struct nt36672a_panel *pinfo)
ret = regulator_set_load(pinfo->supplies[i].consumer,
nt36672a_regulator_enable_loads[i]);
if (ret)
- return dev_err_probe(dev, ret, "failed to set regulator enable loads\n");
+ return dev_err_probe(dev, ret, "failed to set regulator enable loads\n");
}
pinfo->reset_gpio = devm_gpiod_get(dev, "reset", GPIOD_OUT_LOW);
@@ -656,7 +656,13 @@ static int nt36672a_panel_probe(struct mipi_dsi_device *dsi)
if (err < 0)
return err;
- return mipi_dsi_attach(dsi);
+ err = mipi_dsi_attach(dsi);
+ if (err < 0) {
+ drm_panel_remove(&pinfo->base);
+ return err;
+ }
+
+ return 0;
}
static int nt36672a_panel_remove(struct mipi_dsi_device *dsi)
diff --git a/drivers/gpu/drm/panel/panel-novatek-nt39016.c b/drivers/gpu/drm/panel/panel-novatek-nt39016.c
index f8151fe..d036853 100644
--- a/drivers/gpu/drm/panel/panel-novatek-nt39016.c
+++ b/drivers/gpu/drm/panel/panel-novatek-nt39016.c
@@ -258,16 +258,13 @@ static int nt39016_probe(struct spi_device *spi)
return -EINVAL;
panel->supply = devm_regulator_get(dev, "power");
- if (IS_ERR(panel->supply)) {
- dev_err(dev, "Failed to get power supply\n");
- return PTR_ERR(panel->supply);
- }
+ if (IS_ERR(panel->supply))
+ return dev_err_probe(dev, PTR_ERR(panel->supply),
+ "Failed to get power supply\n");
panel->reset_gpio = devm_gpiod_get(dev, "reset", GPIOD_OUT_HIGH);
- if (IS_ERR(panel->reset_gpio)) {
- dev_err(dev, "Failed to get reset GPIO\n");
- return PTR_ERR(panel->reset_gpio);
- }
+ if (IS_ERR(panel->reset_gpio))
+ return dev_err_probe(dev, PTR_ERR(panel->reset_gpio), "Failed to get reset GPIO\n");
spi->bits_per_word = 8;
spi->mode = SPI_MODE_3 | SPI_3WIRE;
@@ -287,11 +284,8 @@ static int nt39016_probe(struct spi_device *spi)
DRM_MODE_CONNECTOR_DPI);
err = drm_panel_of_backlight(&panel->drm_panel);
- if (err) {
- if (err != -EPROBE_DEFER)
- dev_err(dev, "Failed to get backlight handle\n");
- return err;
- }
+ if (err)
+ return dev_err_probe(dev, err, "Failed to get backlight handle\n");
drm_panel_add(&panel->drm_panel);
diff --git a/drivers/gpu/drm/panel/panel-panasonic-vvx10f034n00.c b/drivers/gpu/drm/panel/panel-panasonic-vvx10f034n00.c
index 3c20bee..3991f5d 100644
--- a/drivers/gpu/drm/panel/panel-panasonic-vvx10f034n00.c
+++ b/drivers/gpu/drm/panel/panel-panasonic-vvx10f034n00.c
@@ -241,7 +241,13 @@ static int wuxga_nt_panel_probe(struct mipi_dsi_device *dsi)
if (ret < 0)
return ret;
- return mipi_dsi_attach(dsi);
+ ret = mipi_dsi_attach(dsi);
+ if (ret < 0) {
+ wuxga_nt_panel_del(wuxga_nt);
+ return ret;
+ }
+
+ return 0;
}
static int wuxga_nt_panel_remove(struct mipi_dsi_device *dsi)
diff --git a/drivers/gpu/drm/panel/panel-ronbo-rb070d30.c b/drivers/gpu/drm/panel/panel-ronbo-rb070d30.c
index a378283..1fb579a 100644
--- a/drivers/gpu/drm/panel/panel-ronbo-rb070d30.c
+++ b/drivers/gpu/drm/panel/panel-ronbo-rb070d30.c
@@ -199,7 +199,13 @@ static int rb070d30_panel_dsi_probe(struct mipi_dsi_device *dsi)
dsi->format = MIPI_DSI_FMT_RGB888;
dsi->lanes = 4;
- return mipi_dsi_attach(dsi);
+ ret = mipi_dsi_attach(dsi);
+ if (ret < 0) {
+ drm_panel_remove(&ctx->panel);
+ return ret;
+ }
+
+ return 0;
}
static int rb070d30_panel_dsi_remove(struct mipi_dsi_device *dsi)
diff --git a/drivers/gpu/drm/panel/panel-samsung-s6e63j0x03.c b/drivers/gpu/drm/panel/panel-samsung-s6e63j0x03.c
index ccc8ed6..e38262b 100644
--- a/drivers/gpu/drm/panel/panel-samsung-s6e63j0x03.c
+++ b/drivers/gpu/drm/panel/panel-samsung-s6e63j0x03.c
@@ -452,27 +452,22 @@ static int s6e63j0x03_probe(struct mipi_dsi_device *dsi)
ctx->supplies[1].supply = "vci";
ret = devm_regulator_bulk_get(dev, ARRAY_SIZE(ctx->supplies),
ctx->supplies);
- if (ret < 0) {
- dev_err(dev, "failed to get regulators: %d\n", ret);
- return ret;
- }
+ if (ret < 0)
+ return dev_err_probe(dev, ret, "failed to get regulators\n");
ctx->reset_gpio = devm_gpiod_get(dev, "reset", GPIOD_OUT_LOW);
- if (IS_ERR(ctx->reset_gpio)) {
- dev_err(dev, "cannot get reset-gpio: %ld\n",
- PTR_ERR(ctx->reset_gpio));
- return PTR_ERR(ctx->reset_gpio);
- }
+ if (IS_ERR(ctx->reset_gpio))
+ return dev_err_probe(dev, PTR_ERR(ctx->reset_gpio),
+ "cannot get reset-gpio\n");
drm_panel_init(&ctx->panel, dev, &s6e63j0x03_funcs,
DRM_MODE_CONNECTOR_DSI);
ctx->bl_dev = backlight_device_register("s6e63j0x03", dev, ctx,
&s6e63j0x03_bl_ops, NULL);
- if (IS_ERR(ctx->bl_dev)) {
- dev_err(dev, "failed to register backlight device\n");
- return PTR_ERR(ctx->bl_dev);
- }
+ if (IS_ERR(ctx->bl_dev))
+ return dev_err_probe(dev, PTR_ERR(ctx->bl_dev),
+ "failed to register backlight device\n");
ctx->bl_dev->props.max_brightness = MAX_BRIGHTNESS;
ctx->bl_dev->props.brightness = DEFAULT_BRIGHTNESS;
diff --git a/drivers/gpu/drm/panel/panel-samsung-s6e88a0-ams452ef01.c b/drivers/gpu/drm/panel/panel-samsung-s6e88a0-ams452ef01.c
index ea63799..29fde38 100644
--- a/drivers/gpu/drm/panel/panel-samsung-s6e88a0-ams452ef01.c
+++ b/drivers/gpu/drm/panel/panel-samsung-s6e88a0-ams452ef01.c
@@ -247,6 +247,7 @@ static int s6e88a0_ams452ef01_probe(struct mipi_dsi_device *dsi)
ret = mipi_dsi_attach(dsi);
if (ret < 0) {
dev_err(dev, "Failed to attach to DSI host: %d\n", ret);
+ drm_panel_remove(&ctx->panel);
return ret;
}
diff --git a/drivers/gpu/drm/panel/panel-samsung-sofef00.c b/drivers/gpu/drm/panel/panel-samsung-sofef00.c
index 8cb1853..1fb37fda 100644
--- a/drivers/gpu/drm/panel/panel-samsung-sofef00.c
+++ b/drivers/gpu/drm/panel/panel-samsung-sofef00.c
@@ -270,18 +270,14 @@ static int sofef00_panel_probe(struct mipi_dsi_device *dsi)
}
ctx->supply = devm_regulator_get(dev, "vddio");
- if (IS_ERR(ctx->supply)) {
- ret = PTR_ERR(ctx->supply);
- dev_err(dev, "Failed to get vddio regulator: %d\n", ret);
- return ret;
- }
+ if (IS_ERR(ctx->supply))
+ return dev_err_probe(dev, PTR_ERR(ctx->supply),
+ "Failed to get vddio regulator\n");
ctx->reset_gpio = devm_gpiod_get(dev, "reset", GPIOD_OUT_HIGH);
- if (IS_ERR(ctx->reset_gpio)) {
- ret = PTR_ERR(ctx->reset_gpio);
- dev_warn(dev, "Failed to get reset-gpios: %d\n", ret);
- return ret;
- }
+ if (IS_ERR(ctx->reset_gpio))
+ return dev_err_probe(dev, PTR_ERR(ctx->reset_gpio),
+ "Failed to get reset-gpios\n");
ctx->dsi = dsi;
mipi_dsi_set_drvdata(dsi, ctx);
@@ -302,6 +298,7 @@ static int sofef00_panel_probe(struct mipi_dsi_device *dsi)
ret = mipi_dsi_attach(dsi);
if (ret < 0) {
dev_err(dev, "Failed to attach to DSI host: %d\n", ret);
+ drm_panel_remove(&ctx->panel);
return ret;
}
diff --git a/drivers/gpu/drm/panel/panel-sharp-ls037v7dw01.c b/drivers/gpu/drm/panel/panel-sharp-ls037v7dw01.c
index 94992f4..a07d0f6 100644
--- a/drivers/gpu/drm/panel/panel-sharp-ls037v7dw01.c
+++ b/drivers/gpu/drm/panel/panel-sharp-ls037v7dw01.c
@@ -146,22 +146,19 @@ static int ls037v7dw01_probe(struct platform_device *pdev)
lcd->pdev = pdev;
lcd->vdd = devm_regulator_get(&pdev->dev, "envdd");
- if (IS_ERR(lcd->vdd)) {
- dev_err(&pdev->dev, "failed to get regulator\n");
- return PTR_ERR(lcd->vdd);
- }
+ if (IS_ERR(lcd->vdd))
+ return dev_err_probe(&pdev->dev, PTR_ERR(lcd->vdd),
+ "failed to get regulator\n");
lcd->ini_gpio = devm_gpiod_get(&pdev->dev, "enable", GPIOD_OUT_LOW);
- if (IS_ERR(lcd->ini_gpio)) {
- dev_err(&pdev->dev, "failed to get enable gpio\n");
- return PTR_ERR(lcd->ini_gpio);
- }
+ if (IS_ERR(lcd->ini_gpio))
+ return dev_err_probe(&pdev->dev, PTR_ERR(lcd->ini_gpio),
+ "failed to get enable gpio\n");
lcd->resb_gpio = devm_gpiod_get(&pdev->dev, "reset", GPIOD_OUT_LOW);
- if (IS_ERR(lcd->resb_gpio)) {
- dev_err(&pdev->dev, "failed to get reset gpio\n");
- return PTR_ERR(lcd->resb_gpio);
- }
+ if (IS_ERR(lcd->resb_gpio))
+ return dev_err_probe(&pdev->dev, PTR_ERR(lcd->resb_gpio),
+ "failed to get reset gpio\n");
lcd->mo_gpio = devm_gpiod_get_index(&pdev->dev, "mode", 0,
GPIOD_OUT_LOW);
diff --git a/drivers/gpu/drm/panel/panel-sharp-ls043t1le01.c b/drivers/gpu/drm/panel/panel-sharp-ls043t1le01.c
index b937e24..25829a0 100644
--- a/drivers/gpu/drm/panel/panel-sharp-ls043t1le01.c
+++ b/drivers/gpu/drm/panel/panel-sharp-ls043t1le01.c
@@ -296,7 +296,13 @@ static int sharp_nt_panel_probe(struct mipi_dsi_device *dsi)
if (ret < 0)
return ret;
- return mipi_dsi_attach(dsi);
+ ret = mipi_dsi_attach(dsi);
+ if (ret < 0) {
+ sharp_nt_panel_del(sharp_nt);
+ return ret;
+ }
+
+ return 0;
}
static int sharp_nt_panel_remove(struct mipi_dsi_device *dsi)
diff --git a/drivers/gpu/drm/panel/panel-simple.c b/drivers/gpu/drm/panel/panel-simple.c
index eb475a3..9e46db5 100644
--- a/drivers/gpu/drm/panel/panel-simple.c
+++ b/drivers/gpu/drm/panel/panel-simple.c
@@ -2031,6 +2031,31 @@ static const struct panel_desc innolux_g070y2_l01 = {
.connector_type = DRM_MODE_CONNECTOR_LVDS,
};
+static const struct drm_display_mode innolux_g070y2_t02_mode = {
+ .clock = 33333,
+ .hdisplay = 800,
+ .hsync_start = 800 + 210,
+ .hsync_end = 800 + 210 + 20,
+ .htotal = 800 + 210 + 20 + 46,
+ .vdisplay = 480,
+ .vsync_start = 480 + 22,
+ .vsync_end = 480 + 22 + 10,
+ .vtotal = 480 + 22 + 23 + 10,
+};
+
+static const struct panel_desc innolux_g070y2_t02 = {
+ .modes = &innolux_g070y2_t02_mode,
+ .num_modes = 1,
+ .bpc = 8,
+ .size = {
+ .width = 152,
+ .height = 92,
+ },
+ .bus_format = MEDIA_BUS_FMT_RGB888_1X24,
+ .bus_flags = DRM_BUS_FLAG_DE_HIGH | DRM_BUS_FLAG_PIXDATA_DRIVE_POSEDGE,
+ .connector_type = DRM_MODE_CONNECTOR_DPI,
+};
+
static const struct display_timing innolux_g101ice_l01_timing = {
.pixelclock = { 60400000, 71100000, 74700000 },
.hactive = { 1280, 1280, 1280 },
@@ -3227,6 +3252,33 @@ static const struct panel_desc starry_kr070pe2t = {
.connector_type = DRM_MODE_CONNECTOR_DPI,
};
+static const struct display_timing tsd_tst043015cmhx_timing = {
+ .pixelclock = { 5000000, 9000000, 12000000 },
+ .hactive = { 480, 480, 480 },
+ .hfront_porch = { 4, 5, 65 },
+ .hback_porch = { 36, 40, 255 },
+ .hsync_len = { 1, 1, 1 },
+ .vactive = { 272, 272, 272 },
+ .vfront_porch = { 2, 8, 97 },
+ .vback_porch = { 3, 8, 31 },
+ .vsync_len = { 1, 1, 1 },
+
+ .flags = DISPLAY_FLAGS_HSYNC_LOW | DISPLAY_FLAGS_VSYNC_LOW |
+ DISPLAY_FLAGS_DE_HIGH | DISPLAY_FLAGS_PIXDATA_POSEDGE,
+};
+
+static const struct panel_desc tsd_tst043015cmhx = {
+ .timings = &tsd_tst043015cmhx_timing,
+ .num_timings = 1,
+ .bpc = 8,
+ .size = {
+ .width = 105,
+ .height = 67,
+ },
+ .bus_format = MEDIA_BUS_FMT_RGB888_1X24,
+ .bus_flags = DRM_BUS_FLAG_DE_HIGH | DRM_BUS_FLAG_PIXDATA_SAMPLE_NEGEDGE,
+};
+
static const struct drm_display_mode tfc_s9700rtwv43tr_01b_mode = {
.clock = 30000,
.hdisplay = 800,
@@ -3473,6 +3525,31 @@ static const struct panel_desc urt_umsh_8596md_parallel = {
.bus_format = MEDIA_BUS_FMT_RGB666_1X18,
};
+static const struct drm_display_mode vivax_tpc9150_panel_mode = {
+ .clock = 60000,
+ .hdisplay = 1024,
+ .hsync_start = 1024 + 160,
+ .hsync_end = 1024 + 160 + 100,
+ .htotal = 1024 + 160 + 100 + 60,
+ .vdisplay = 600,
+ .vsync_start = 600 + 12,
+ .vsync_end = 600 + 12 + 10,
+ .vtotal = 600 + 12 + 10 + 13,
+};
+
+static const struct panel_desc vivax_tpc9150_panel = {
+ .modes = &vivax_tpc9150_panel_mode,
+ .num_modes = 1,
+ .bpc = 6,
+ .size = {
+ .width = 200,
+ .height = 115,
+ },
+ .bus_format = MEDIA_BUS_FMT_RGB666_1X7X3_SPWG,
+ .bus_flags = DRM_BUS_FLAG_DE_HIGH,
+ .connector_type = DRM_MODE_CONNECTOR_LVDS,
+};
+
static const struct drm_display_mode vl050_8048nt_c01_mode = {
.clock = 33333,
.hdisplay = 800,
@@ -3738,6 +3815,9 @@ static const struct of_device_id platform_of_match[] = {
.compatible = "innolux,g070y2-l01",
.data = &innolux_g070y2_l01,
}, {
+ .compatible = "innolux,g070y2-t02",
+ .data = &innolux_g070y2_t02,
+ }, {
.compatible = "innolux,g101ice-l01",
.data = &innolux_g101ice_l01
}, {
@@ -3876,6 +3956,9 @@ static const struct of_device_id platform_of_match[] = {
.compatible = "starry,kr070pe2t",
.data = &starry_kr070pe2t,
}, {
+ .compatible = "team-source-display,tst043015cmhx",
+ .data = &tsd_tst043015cmhx,
+ }, {
.compatible = "tfc,s9700rtwv43tr-01b",
.data = &tfc_s9700rtwv43tr_01b,
}, {
@@ -3921,6 +4004,9 @@ static const struct of_device_id platform_of_match[] = {
.compatible = "urt,umsh-8596md-20t",
.data = &urt_umsh_8596md_parallel,
}, {
+ .compatible = "vivax,tpc9150-panel",
+ .data = &vivax_tpc9150_panel,
+ }, {
.compatible = "vxt,vl050-8048nt-c01",
.data = &vl050_8048nt_c01,
}, {
diff --git a/drivers/gpu/drm/panel/panel-sony-tulip-truly-nt35521.c b/drivers/gpu/drm/panel/panel-sony-tulip-truly-nt35521.c
new file mode 100644
index 0000000..69f07b1
--- /dev/null
+++ b/drivers/gpu/drm/panel/panel-sony-tulip-truly-nt35521.c
@@ -0,0 +1,552 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2021, Linaro Limited
+ *
+ * Generated with linux-mdss-dsi-panel-driver-generator from vendor device tree:
+ * Copyright (c) 2013, The Linux Foundation. All rights reserved.
+ */
+
+#include <linux/backlight.h>
+#include <linux/delay.h>
+#include <linux/gpio/consumer.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/regulator/consumer.h>
+
+#include <drm/drm_mipi_dsi.h>
+#include <drm/drm_modes.h>
+#include <drm/drm_panel.h>
+
+struct truly_nt35521 {
+ struct drm_panel panel;
+ struct mipi_dsi_device *dsi;
+ struct regulator_bulk_data supplies[2];
+ struct gpio_desc *reset_gpio;
+ struct gpio_desc *blen_gpio;
+ bool prepared;
+ bool enabled;
+};
+
+static inline
+struct truly_nt35521 *to_truly_nt35521(struct drm_panel *panel)
+{
+ return container_of(panel, struct truly_nt35521, panel);
+}
+
+#define dsi_generic_write_seq(dsi, seq...) do { \
+ static const u8 d[] = { seq }; \
+ int ret; \
+ ret = mipi_dsi_generic_write(dsi, d, ARRAY_SIZE(d)); \
+ if (ret < 0) \
+ return ret; \
+ } while (0)
+
+static void truly_nt35521_reset(struct truly_nt35521 *ctx)
+{
+ gpiod_set_value_cansleep(ctx->reset_gpio, 1);
+ usleep_range(1000, 2000);
+ gpiod_set_value_cansleep(ctx->reset_gpio, 1);
+ usleep_range(10000, 11000);
+ gpiod_set_value_cansleep(ctx->reset_gpio, 0);
+ msleep(150);
+}
+
+static int truly_nt35521_on(struct truly_nt35521 *ctx)
+{
+ struct mipi_dsi_device *dsi = ctx->dsi;
+ struct device *dev = &dsi->dev;
+ int ret;
+
+ dsi->mode_flags |= MIPI_DSI_MODE_LPM;
+
+ dsi_generic_write_seq(dsi, 0xf0, 0x55, 0xaa, 0x52, 0x08, 0x00);
+ dsi_generic_write_seq(dsi, 0xff, 0xaa, 0x55, 0xa5, 0x80);
+ dsi_generic_write_seq(dsi, 0x6f, 0x11, 0x00);
+ dsi_generic_write_seq(dsi, 0xf7, 0x20, 0x00);
+ dsi_generic_write_seq(dsi, 0x6f, 0x01);
+ dsi_generic_write_seq(dsi, 0xb1, 0x21);
+ dsi_generic_write_seq(dsi, 0xbd, 0x01, 0xa0, 0x10, 0x08, 0x01);
+ dsi_generic_write_seq(dsi, 0xb8, 0x01, 0x02, 0x0c, 0x02);
+ dsi_generic_write_seq(dsi, 0xbb, 0x11, 0x11);
+ dsi_generic_write_seq(dsi, 0xbc, 0x00, 0x00);
+ dsi_generic_write_seq(dsi, 0xb6, 0x02);
+ dsi_generic_write_seq(dsi, 0xf0, 0x55, 0xaa, 0x52, 0x08, 0x01);
+ dsi_generic_write_seq(dsi, 0xb0, 0x09, 0x09);
+ dsi_generic_write_seq(dsi, 0xb1, 0x09, 0x09);
+ dsi_generic_write_seq(dsi, 0xbc, 0x8c, 0x00);
+ dsi_generic_write_seq(dsi, 0xbd, 0x8c, 0x00);
+ dsi_generic_write_seq(dsi, 0xca, 0x00);
+ dsi_generic_write_seq(dsi, 0xc0, 0x04);
+ dsi_generic_write_seq(dsi, 0xbe, 0xb5);
+ dsi_generic_write_seq(dsi, 0xb3, 0x35, 0x35);
+ dsi_generic_write_seq(dsi, 0xb4, 0x25, 0x25);
+ dsi_generic_write_seq(dsi, 0xb9, 0x43, 0x43);
+ dsi_generic_write_seq(dsi, 0xba, 0x24, 0x24);
+ dsi_generic_write_seq(dsi, 0xf0, 0x55, 0xaa, 0x52, 0x08, 0x02);
+ dsi_generic_write_seq(dsi, 0xee, 0x03);
+ dsi_generic_write_seq(dsi, 0xb0,
+ 0x00, 0xb2, 0x00, 0xb3, 0x00, 0xb6, 0x00, 0xc3,
+ 0x00, 0xce, 0x00, 0xe1, 0x00, 0xf3, 0x01, 0x11);
+ dsi_generic_write_seq(dsi, 0xb1,
+ 0x01, 0x2e, 0x01, 0x5c, 0x01, 0x82, 0x01, 0xc3,
+ 0x01, 0xfe, 0x02, 0x00, 0x02, 0x37, 0x02, 0x77);
+ dsi_generic_write_seq(dsi, 0xb2,
+ 0x02, 0xa1, 0x02, 0xd7, 0x02, 0xfe, 0x03, 0x2c,
+ 0x03, 0x4b, 0x03, 0x63, 0x03, 0x8f, 0x03, 0x90);
+ dsi_generic_write_seq(dsi, 0xb3, 0x03, 0x96, 0x03, 0x98);
+ dsi_generic_write_seq(dsi, 0xb4,
+ 0x00, 0x81, 0x00, 0x8b, 0x00, 0x9c, 0x00, 0xa9,
+ 0x00, 0xb5, 0x00, 0xcb, 0x00, 0xdf, 0x01, 0x02);
+ dsi_generic_write_seq(dsi, 0xb5,
+ 0x01, 0x1f, 0x01, 0x51, 0x01, 0x7a, 0x01, 0xbf,
+ 0x01, 0xfa, 0x01, 0xfc, 0x02, 0x34, 0x02, 0x76);
+ dsi_generic_write_seq(dsi, 0xb6,
+ 0x02, 0x9f, 0x02, 0xd7, 0x02, 0xfc, 0x03, 0x2c,
+ 0x03, 0x4a, 0x03, 0x63, 0x03, 0x8f, 0x03, 0xa2);
+ dsi_generic_write_seq(dsi, 0xb7, 0x03, 0xb8, 0x03, 0xba);
+ dsi_generic_write_seq(dsi, 0xb8,
+ 0x00, 0x01, 0x00, 0x02, 0x00, 0x0e, 0x00, 0x2a,
+ 0x00, 0x41, 0x00, 0x67, 0x00, 0x87, 0x00, 0xb9);
+ dsi_generic_write_seq(dsi, 0xb9,
+ 0x00, 0xe2, 0x01, 0x22, 0x01, 0x54, 0x01, 0xa3,
+ 0x01, 0xe6, 0x01, 0xe7, 0x02, 0x24, 0x02, 0x67);
+ dsi_generic_write_seq(dsi, 0xba,
+ 0x02, 0x93, 0x02, 0xcd, 0x02, 0xf6, 0x03, 0x31,
+ 0x03, 0x6c, 0x03, 0xe9, 0x03, 0xef, 0x03, 0xf4);
+ dsi_generic_write_seq(dsi, 0xbb, 0x03, 0xf6, 0x03, 0xf7);
+ dsi_generic_write_seq(dsi, 0xf0, 0x55, 0xaa, 0x52, 0x08, 0x03);
+ dsi_generic_write_seq(dsi, 0xb0, 0x22, 0x00);
+ dsi_generic_write_seq(dsi, 0xb1, 0x22, 0x00);
+ dsi_generic_write_seq(dsi, 0xb2, 0x05, 0x00, 0x60, 0x00, 0x00);
+ dsi_generic_write_seq(dsi, 0xb3, 0x05, 0x00, 0x60, 0x00, 0x00);
+ dsi_generic_write_seq(dsi, 0xb4, 0x05, 0x00, 0x60, 0x00, 0x00);
+ dsi_generic_write_seq(dsi, 0xb5, 0x05, 0x00, 0x60, 0x00, 0x00);
+ dsi_generic_write_seq(dsi, 0xba, 0x53, 0x00, 0x60, 0x00, 0x00);
+ dsi_generic_write_seq(dsi, 0xbb, 0x53, 0x00, 0x60, 0x00, 0x00);
+ dsi_generic_write_seq(dsi, 0xbc, 0x53, 0x00, 0x60, 0x00, 0x00);
+ dsi_generic_write_seq(dsi, 0xbd, 0x53, 0x00, 0x60, 0x00, 0x00);
+ dsi_generic_write_seq(dsi, 0xc0, 0x00, 0x34, 0x00, 0x00);
+ dsi_generic_write_seq(dsi, 0xc1, 0x00, 0x00, 0x34, 0x00);
+ dsi_generic_write_seq(dsi, 0xc2, 0x00, 0x00, 0x34, 0x00);
+ dsi_generic_write_seq(dsi, 0xc3, 0x00, 0x00, 0x34, 0x00);
+ dsi_generic_write_seq(dsi, 0xc4, 0x60);
+ dsi_generic_write_seq(dsi, 0xc5, 0xc0);
+ dsi_generic_write_seq(dsi, 0xc6, 0x00);
+ dsi_generic_write_seq(dsi, 0xc7, 0x00);
+ dsi_generic_write_seq(dsi, 0xf0, 0x55, 0xaa, 0x52, 0x08, 0x05);
+ dsi_generic_write_seq(dsi, 0xb0, 0x17, 0x06);
+ dsi_generic_write_seq(dsi, 0xb1, 0x17, 0x06);
+ dsi_generic_write_seq(dsi, 0xb2, 0x17, 0x06);
+ dsi_generic_write_seq(dsi, 0xb3, 0x17, 0x06);
+ dsi_generic_write_seq(dsi, 0xb4, 0x17, 0x06);
+ dsi_generic_write_seq(dsi, 0xb5, 0x17, 0x06);
+ dsi_generic_write_seq(dsi, 0xb6, 0x17, 0x06);
+ dsi_generic_write_seq(dsi, 0xb7, 0x17, 0x06);
+ dsi_generic_write_seq(dsi, 0xb8, 0x00);
+ dsi_generic_write_seq(dsi, 0xb9, 0x00, 0x03);
+ dsi_generic_write_seq(dsi, 0xba, 0x00, 0x00);
+ dsi_generic_write_seq(dsi, 0xbb, 0x02, 0x03);
+ dsi_generic_write_seq(dsi, 0xbc, 0x02, 0x03);
+ dsi_generic_write_seq(dsi, 0xbd, 0x03, 0x03, 0x00, 0x03, 0x03);
+ dsi_generic_write_seq(dsi, 0xc0, 0x0b);
+ dsi_generic_write_seq(dsi, 0xc1, 0x09);
+ dsi_generic_write_seq(dsi, 0xc2, 0xa6);
+ dsi_generic_write_seq(dsi, 0xc3, 0x05);
+ dsi_generic_write_seq(dsi, 0xc4, 0x00);
+ dsi_generic_write_seq(dsi, 0xc5, 0x02);
+ dsi_generic_write_seq(dsi, 0xc6, 0x22);
+ dsi_generic_write_seq(dsi, 0xc7, 0x03);
+ dsi_generic_write_seq(dsi, 0xc8, 0x07, 0x20);
+ dsi_generic_write_seq(dsi, 0xc9, 0x03, 0x20);
+ dsi_generic_write_seq(dsi, 0xca, 0x01, 0x60);
+ dsi_generic_write_seq(dsi, 0xcb, 0x01, 0x60);
+ dsi_generic_write_seq(dsi, 0xcc, 0x00, 0x00, 0x02);
+ dsi_generic_write_seq(dsi, 0xcd, 0x00, 0x00, 0x02);
+ dsi_generic_write_seq(dsi, 0xce, 0x00, 0x00, 0x02);
+ dsi_generic_write_seq(dsi, 0xcf, 0x00, 0x00, 0x02);
+ dsi_generic_write_seq(dsi, 0xd1, 0x00, 0x05, 0x01, 0x07, 0x10);
+ dsi_generic_write_seq(dsi, 0xd2, 0x10, 0x05, 0x05, 0x03, 0x10);
+ dsi_generic_write_seq(dsi, 0xd3, 0x20, 0x00, 0x43, 0x07, 0x10);
+ dsi_generic_write_seq(dsi, 0xd4, 0x30, 0x00, 0x43, 0x07, 0x10);
+ dsi_generic_write_seq(dsi, 0xd0,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00);
+ dsi_generic_write_seq(dsi, 0xd5,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00);
+ dsi_generic_write_seq(dsi, 0xd6,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00);
+ dsi_generic_write_seq(dsi, 0xd7,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00);
+ dsi_generic_write_seq(dsi, 0xd8, 0x00, 0x00, 0x00, 0x00, 0x00);
+ dsi_generic_write_seq(dsi, 0xe5, 0x06);
+ dsi_generic_write_seq(dsi, 0xe6, 0x06);
+ dsi_generic_write_seq(dsi, 0xe7, 0x00);
+ dsi_generic_write_seq(dsi, 0xe8, 0x06);
+ dsi_generic_write_seq(dsi, 0xe9, 0x06);
+ dsi_generic_write_seq(dsi, 0xea, 0x06);
+ dsi_generic_write_seq(dsi, 0xeb, 0x00);
+ dsi_generic_write_seq(dsi, 0xec, 0x00);
+ dsi_generic_write_seq(dsi, 0xed, 0x30);
+ dsi_generic_write_seq(dsi, 0xf0, 0x55, 0xaa, 0x52, 0x08, 0x06);
+ dsi_generic_write_seq(dsi, 0xb0, 0x31, 0x31);
+ dsi_generic_write_seq(dsi, 0xb1, 0x31, 0x31);
+ dsi_generic_write_seq(dsi, 0xb2, 0x2d, 0x2e);
+ dsi_generic_write_seq(dsi, 0xb3, 0x31, 0x34);
+ dsi_generic_write_seq(dsi, 0xb4, 0x29, 0x2a);
+ dsi_generic_write_seq(dsi, 0xb5, 0x12, 0x10);
+ dsi_generic_write_seq(dsi, 0xb6, 0x18, 0x16);
+ dsi_generic_write_seq(dsi, 0xb7, 0x00, 0x02);
+ dsi_generic_write_seq(dsi, 0xb8, 0x08, 0x31);
+ dsi_generic_write_seq(dsi, 0xb9, 0x31, 0x31);
+ dsi_generic_write_seq(dsi, 0xba, 0x31, 0x31);
+ dsi_generic_write_seq(dsi, 0xbb, 0x31, 0x08);
+ dsi_generic_write_seq(dsi, 0xbc, 0x03, 0x01);
+ dsi_generic_write_seq(dsi, 0xbd, 0x17, 0x19);
+ dsi_generic_write_seq(dsi, 0xbe, 0x11, 0x13);
+ dsi_generic_write_seq(dsi, 0xbf, 0x2a, 0x29);
+ dsi_generic_write_seq(dsi, 0xc0, 0x34, 0x31);
+ dsi_generic_write_seq(dsi, 0xc1, 0x2e, 0x2d);
+ dsi_generic_write_seq(dsi, 0xc2, 0x31, 0x31);
+ dsi_generic_write_seq(dsi, 0xc3, 0x31, 0x31);
+ dsi_generic_write_seq(dsi, 0xc4, 0x31, 0x31);
+ dsi_generic_write_seq(dsi, 0xc5, 0x31, 0x31);
+ dsi_generic_write_seq(dsi, 0xc6, 0x2e, 0x2d);
+ dsi_generic_write_seq(dsi, 0xc7, 0x31, 0x34);
+ dsi_generic_write_seq(dsi, 0xc8, 0x29, 0x2a);
+ dsi_generic_write_seq(dsi, 0xc9, 0x17, 0x19);
+ dsi_generic_write_seq(dsi, 0xca, 0x11, 0x13);
+ dsi_generic_write_seq(dsi, 0xcb, 0x03, 0x01);
+ dsi_generic_write_seq(dsi, 0xcc, 0x08, 0x31);
+ dsi_generic_write_seq(dsi, 0xcd, 0x31, 0x31);
+ dsi_generic_write_seq(dsi, 0xce, 0x31, 0x31);
+ dsi_generic_write_seq(dsi, 0xcf, 0x31, 0x08);
+ dsi_generic_write_seq(dsi, 0xd0, 0x00, 0x02);
+ dsi_generic_write_seq(dsi, 0xd1, 0x12, 0x10);
+ dsi_generic_write_seq(dsi, 0xd2, 0x18, 0x16);
+ dsi_generic_write_seq(dsi, 0xd3, 0x2a, 0x29);
+ dsi_generic_write_seq(dsi, 0xd4, 0x34, 0x31);
+ dsi_generic_write_seq(dsi, 0xd5, 0x2d, 0x2e);
+ dsi_generic_write_seq(dsi, 0xd6, 0x31, 0x31);
+ dsi_generic_write_seq(dsi, 0xd7, 0x31, 0x31);
+ dsi_generic_write_seq(dsi, 0xe5, 0x31, 0x31);
+ dsi_generic_write_seq(dsi, 0xe6, 0x31, 0x31);
+ dsi_generic_write_seq(dsi, 0xd8, 0x00, 0x00, 0x00, 0x00, 0x00);
+ dsi_generic_write_seq(dsi, 0xd9, 0x00, 0x00, 0x00, 0x00, 0x00);
+ dsi_generic_write_seq(dsi, 0xe7, 0x00);
+ dsi_generic_write_seq(dsi, 0x6f, 0x02);
+ dsi_generic_write_seq(dsi, 0xf7, 0x47);
+ dsi_generic_write_seq(dsi, 0x6f, 0x0a);
+ dsi_generic_write_seq(dsi, 0xf7, 0x02);
+ dsi_generic_write_seq(dsi, 0x6f, 0x17);
+ dsi_generic_write_seq(dsi, 0xf4, 0x60);
+ dsi_generic_write_seq(dsi, 0x6f, 0x01);
+ dsi_generic_write_seq(dsi, 0xf9, 0x46);
+ dsi_generic_write_seq(dsi, 0x6f, 0x11);
+ dsi_generic_write_seq(dsi, 0xf3, 0x01);
+ dsi_generic_write_seq(dsi, 0x35, 0x00);
+ dsi_generic_write_seq(dsi, 0xf0, 0x55, 0xaa, 0x52, 0x08, 0x00);
+ dsi_generic_write_seq(dsi, 0xd9, 0x02, 0x03, 0x00);
+ dsi_generic_write_seq(dsi, 0xf0, 0x55, 0xaa, 0x52, 0x00, 0x00);
+ dsi_generic_write_seq(dsi, 0xf0, 0x55, 0xaa, 0x52, 0x08, 0x00);
+ dsi_generic_write_seq(dsi, 0xb1, 0x6c, 0x21);
+ dsi_generic_write_seq(dsi, 0xf0, 0x55, 0xaa, 0x52, 0x00, 0x00);
+ dsi_generic_write_seq(dsi, 0x35, 0x00);
+
+ ret = mipi_dsi_dcs_exit_sleep_mode(dsi);
+ if (ret < 0) {
+ dev_err(dev, "Failed to exit sleep mode: %d\n", ret);
+ return ret;
+ }
+ msleep(120);
+
+ ret = mipi_dsi_dcs_set_display_on(dsi);
+ if (ret < 0) {
+ dev_err(dev, "Failed to set display on: %d\n", ret);
+ return ret;
+ }
+ usleep_range(1000, 2000);
+
+ dsi_generic_write_seq(dsi, 0x53, 0x24);
+
+ return 0;
+}
+
+static int truly_nt35521_off(struct truly_nt35521 *ctx)
+{
+ struct mipi_dsi_device *dsi = ctx->dsi;
+ struct device *dev = &dsi->dev;
+ int ret;
+
+ dsi->mode_flags &= ~MIPI_DSI_MODE_LPM;
+
+ ret = mipi_dsi_dcs_set_display_off(dsi);
+ if (ret < 0) {
+ dev_err(dev, "Failed to set display off: %d\n", ret);
+ return ret;
+ }
+ msleep(50);
+
+ ret = mipi_dsi_dcs_enter_sleep_mode(dsi);
+ if (ret < 0) {
+ dev_err(dev, "Failed to enter sleep mode: %d\n", ret);
+ return ret;
+ }
+ msleep(150);
+
+ return 0;
+}
+
+static int truly_nt35521_prepare(struct drm_panel *panel)
+{
+ struct truly_nt35521 *ctx = to_truly_nt35521(panel);
+ struct device *dev = &ctx->dsi->dev;
+ int ret;
+
+ if (ctx->prepared)
+ return 0;
+
+ ret = regulator_bulk_enable(ARRAY_SIZE(ctx->supplies), ctx->supplies);
+ if (ret < 0) {
+ dev_err(dev, "Failed to enable regulators: %d\n", ret);
+ return ret;
+ }
+
+ truly_nt35521_reset(ctx);
+
+ ret = truly_nt35521_on(ctx);
+ if (ret < 0) {
+ dev_err(dev, "Failed to initialize panel: %d\n", ret);
+ gpiod_set_value_cansleep(ctx->reset_gpio, 1);
+ return ret;
+ }
+
+ ctx->prepared = true;
+ return 0;
+}
+
+static int truly_nt35521_unprepare(struct drm_panel *panel)
+{
+ struct truly_nt35521 *ctx = to_truly_nt35521(panel);
+ struct device *dev = &ctx->dsi->dev;
+ int ret;
+
+ if (!ctx->prepared)
+ return 0;
+
+ ret = truly_nt35521_off(ctx);
+ if (ret < 0)
+ dev_err(dev, "Failed to un-initialize panel: %d\n", ret);
+
+ gpiod_set_value_cansleep(ctx->reset_gpio, 1);
+ regulator_bulk_disable(ARRAY_SIZE(ctx->supplies),
+ ctx->supplies);
+
+ ctx->prepared = false;
+ return 0;
+}
+
+static int truly_nt35521_enable(struct drm_panel *panel)
+{
+ struct truly_nt35521 *ctx = to_truly_nt35521(panel);
+
+ if (ctx->enabled)
+ return 0;
+
+ gpiod_set_value_cansleep(ctx->blen_gpio, 1);
+
+ ctx->enabled = true;
+ return 0;
+}
+
+static int truly_nt35521_disable(struct drm_panel *panel)
+{
+ struct truly_nt35521 *ctx = to_truly_nt35521(panel);
+
+ if (!ctx->enabled)
+ return 0;
+
+ gpiod_set_value_cansleep(ctx->blen_gpio, 0);
+
+ ctx->enabled = false;
+ return 0;
+}
+
+static const struct drm_display_mode truly_nt35521_mode = {
+ .clock = (720 + 232 + 20 + 112) * (1280 + 18 + 1 + 18) * 60 / 1000,
+ .hdisplay = 720,
+ .hsync_start = 720 + 232,
+ .hsync_end = 720 + 232 + 20,
+ .htotal = 720 + 232 + 20 + 112,
+ .vdisplay = 1280,
+ .vsync_start = 1280 + 18,
+ .vsync_end = 1280 + 18 + 1,
+ .vtotal = 1280 + 18 + 1 + 18,
+ .width_mm = 65,
+ .height_mm = 116,
+};
+
+static int truly_nt35521_get_modes(struct drm_panel *panel,
+ struct drm_connector *connector)
+{
+ struct drm_display_mode *mode;
+
+ mode = drm_mode_duplicate(connector->dev, &truly_nt35521_mode);
+ if (!mode)
+ return -ENOMEM;
+
+ drm_mode_set_name(mode);
+
+ mode->type = DRM_MODE_TYPE_DRIVER | DRM_MODE_TYPE_PREFERRED;
+ connector->display_info.width_mm = mode->width_mm;
+ connector->display_info.height_mm = mode->height_mm;
+ drm_mode_probed_add(connector, mode);
+
+ return 1;
+}
+
+static const struct drm_panel_funcs truly_nt35521_panel_funcs = {
+ .prepare = truly_nt35521_prepare,
+ .unprepare = truly_nt35521_unprepare,
+ .enable = truly_nt35521_enable,
+ .disable = truly_nt35521_disable,
+ .get_modes = truly_nt35521_get_modes,
+};
+
+static int truly_nt35521_bl_update_status(struct backlight_device *bl)
+{
+ struct mipi_dsi_device *dsi = bl_get_data(bl);
+ u16 brightness = backlight_get_brightness(bl);
+ int ret;
+
+ ret = mipi_dsi_dcs_set_display_brightness(dsi, brightness);
+ if (ret < 0)
+ return ret;
+
+ return 0;
+}
+
+static int truly_nt35521_bl_get_brightness(struct backlight_device *bl)
+{
+ struct mipi_dsi_device *dsi = bl_get_data(bl);
+ u16 brightness;
+ int ret;
+
+ ret = mipi_dsi_dcs_get_display_brightness(dsi, &brightness);
+ if (ret < 0)
+ return ret;
+
+ return brightness & 0xff;
+}
+
+static const struct backlight_ops truly_nt35521_bl_ops = {
+ .update_status = truly_nt35521_bl_update_status,
+ .get_brightness = truly_nt35521_bl_get_brightness,
+};
+
+static struct backlight_device *
+truly_nt35521_create_backlight(struct mipi_dsi_device *dsi)
+{
+ struct device *dev = &dsi->dev;
+ const struct backlight_properties props = {
+ .type = BACKLIGHT_RAW,
+ .brightness = 255,
+ .max_brightness = 255,
+ };
+
+ return devm_backlight_device_register(dev, dev_name(dev), dev, dsi,
+ &truly_nt35521_bl_ops, &props);
+}
+
+static int truly_nt35521_probe(struct mipi_dsi_device *dsi)
+{
+ struct device *dev = &dsi->dev;
+ struct truly_nt35521 *ctx;
+ int ret;
+
+ ctx = devm_kzalloc(dev, sizeof(*ctx), GFP_KERNEL);
+ if (!ctx)
+ return -ENOMEM;
+
+ ctx->supplies[0].supply = "positive5";
+ ctx->supplies[1].supply = "negative5";
+ ret = devm_regulator_bulk_get(dev, ARRAY_SIZE(ctx->supplies),
+ ctx->supplies);
+ if (ret < 0) {
+ dev_err(dev, "Failed to get regulators: %d\n", ret);
+ return ret;
+ }
+
+ ctx->reset_gpio = devm_gpiod_get(dev, "reset", GPIOD_OUT_HIGH);
+ if (IS_ERR(ctx->reset_gpio))
+ return dev_err_probe(dev, PTR_ERR(ctx->reset_gpio),
+ "Failed to get reset-gpios\n");
+
+ ctx->blen_gpio = devm_gpiod_get(dev, "backlight", GPIOD_OUT_LOW);
+ if (IS_ERR(ctx->blen_gpio))
+ return dev_err_probe(dev, PTR_ERR(ctx->blen_gpio),
+ "Failed to get backlight-gpios\n");
+
+ ctx->dsi = dsi;
+ mipi_dsi_set_drvdata(dsi, ctx);
+
+ dsi->lanes = 4;
+ dsi->format = MIPI_DSI_FMT_RGB888;
+ dsi->mode_flags = MIPI_DSI_MODE_VIDEO | MIPI_DSI_MODE_VIDEO_BURST |
+ MIPI_DSI_MODE_VIDEO_HSE | MIPI_DSI_MODE_NO_EOT_PACKET |
+ MIPI_DSI_CLOCK_NON_CONTINUOUS;
+
+ drm_panel_init(&ctx->panel, dev, &truly_nt35521_panel_funcs,
+ DRM_MODE_CONNECTOR_DSI);
+
+ ctx->panel.backlight = truly_nt35521_create_backlight(dsi);
+ if (IS_ERR(ctx->panel.backlight))
+ return dev_err_probe(dev, PTR_ERR(ctx->panel.backlight),
+ "Failed to create backlight\n");
+
+ drm_panel_add(&ctx->panel);
+
+ ret = mipi_dsi_attach(dsi);
+ if (ret < 0) {
+ dev_err(dev, "Failed to attach to DSI host: %d\n", ret);
+ drm_panel_remove(&ctx->panel);
+ return ret;
+ }
+
+ return 0;
+}
+
+static int truly_nt35521_remove(struct mipi_dsi_device *dsi)
+{
+ struct truly_nt35521 *ctx = mipi_dsi_get_drvdata(dsi);
+ int ret;
+
+ ret = mipi_dsi_detach(dsi);
+ if (ret < 0)
+ dev_err(&dsi->dev, "Failed to detach from DSI host: %d\n", ret);
+
+ drm_panel_remove(&ctx->panel);
+
+ return 0;
+}
+
+static const struct of_device_id truly_nt35521_of_match[] = {
+ { .compatible = "sony,tulip-truly-nt35521" },
+ { /* sentinel */ }
+};
+MODULE_DEVICE_TABLE(of, truly_nt35521_of_match);
+
+static struct mipi_dsi_driver truly_nt35521_driver = {
+ .probe = truly_nt35521_probe,
+ .remove = truly_nt35521_remove,
+ .driver = {
+ .name = "panel-truly-nt35521",
+ .of_match_table = truly_nt35521_of_match,
+ },
+};
+module_mipi_dsi_driver(truly_nt35521_driver);
+
+MODULE_AUTHOR("Shawn Guo <shawn.guo@linaro.org>");
+MODULE_DESCRIPTION("DRM driver for Sony Tulip Truly NT35521 panel");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/gpu/drm/panel/panel-tpo-td043mtea1.c b/drivers/gpu/drm/panel/panel-tpo-td043mtea1.c
index bacaf1b..1866cdb 100644
--- a/drivers/gpu/drm/panel/panel-tpo-td043mtea1.c
+++ b/drivers/gpu/drm/panel/panel-tpo-td043mtea1.c
@@ -431,16 +431,14 @@ static int td043mtea1_probe(struct spi_device *spi)
memcpy(lcd->gamma, td043mtea1_def_gamma, sizeof(lcd->gamma));
lcd->vcc_reg = devm_regulator_get(&spi->dev, "vcc");
- if (IS_ERR(lcd->vcc_reg)) {
- dev_err(&spi->dev, "failed to get VCC regulator\n");
- return PTR_ERR(lcd->vcc_reg);
- }
+ if (IS_ERR(lcd->vcc_reg))
+ return dev_err_probe(&spi->dev, PTR_ERR(lcd->vcc_reg),
+ "failed to get VCC regulator\n");
lcd->reset_gpio = devm_gpiod_get(&spi->dev, "reset", GPIOD_OUT_HIGH);
- if (IS_ERR(lcd->reset_gpio)) {
- dev_err(&spi->dev, "failed to get reset GPIO\n");
- return PTR_ERR(lcd->reset_gpio);
- }
+ if (IS_ERR(lcd->reset_gpio))
+ return dev_err_probe(&spi->dev, PTR_ERR(lcd->reset_gpio),
+ "failed to get reset GPIO\n");
spi->bits_per_word = 16;
spi->mode = SPI_MODE_0;
diff --git a/drivers/gpu/drm/panel/panel-xinpeng-xpp055c272.c b/drivers/gpu/drm/panel/panel-xinpeng-xpp055c272.c
index d17aae8..8177f5a 100644
--- a/drivers/gpu/drm/panel/panel-xinpeng-xpp055c272.c
+++ b/drivers/gpu/drm/panel/panel-xinpeng-xpp055c272.c
@@ -283,26 +283,19 @@ static int xpp055c272_probe(struct mipi_dsi_device *dsi)
return -ENOMEM;
ctx->reset_gpio = devm_gpiod_get_optional(dev, "reset", GPIOD_OUT_LOW);
- if (IS_ERR(ctx->reset_gpio)) {
- dev_err(dev, "cannot get reset gpio\n");
- return PTR_ERR(ctx->reset_gpio);
- }
+ if (IS_ERR(ctx->reset_gpio))
+ return dev_err_probe(dev, PTR_ERR(ctx->reset_gpio),
+ "cannot get reset gpio\n");
ctx->vci = devm_regulator_get(dev, "vci");
- if (IS_ERR(ctx->vci)) {
- ret = PTR_ERR(ctx->vci);
- if (ret != -EPROBE_DEFER)
- dev_err(dev, "Failed to request vci regulator: %d\n", ret);
- return ret;
- }
+ if (IS_ERR(ctx->vci))
+ return dev_err_probe(dev, PTR_ERR(ctx->vci),
+ "Failed to request vci regulator\n");
ctx->iovcc = devm_regulator_get(dev, "iovcc");
- if (IS_ERR(ctx->iovcc)) {
- ret = PTR_ERR(ctx->iovcc);
- if (ret != -EPROBE_DEFER)
- dev_err(dev, "Failed to request iovcc regulator: %d\n", ret);
- return ret;
- }
+ if (IS_ERR(ctx->iovcc))
+ return dev_err_probe(dev, PTR_ERR(ctx->iovcc),
+ "Failed to request iovcc regulator\n");
mipi_dsi_set_drvdata(dsi, ctx);
diff --git a/drivers/gpu/drm/panfrost/panfrost_drv.c b/drivers/gpu/drm/panfrost/panfrost_drv.c
index 82ad9a6..96bb5a4 100644
--- a/drivers/gpu/drm/panfrost/panfrost_drv.c
+++ b/drivers/gpu/drm/panfrost/panfrost_drv.c
@@ -427,7 +427,7 @@ static int panfrost_ioctl_madvise(struct drm_device *dev, void *data,
}
}
- args->retained = drm_gem_shmem_madvise(gem_obj, args->madv);
+ args->retained = drm_gem_shmem_madvise(&bo->base, args->madv);
if (args->retained) {
if (args->madv == PANFROST_MADV_DONTNEED)
diff --git a/drivers/gpu/drm/panfrost/panfrost_gem.c b/drivers/gpu/drm/panfrost/panfrost_gem.c
index 23377481..ead65f5 100644
--- a/drivers/gpu/drm/panfrost/panfrost_gem.c
+++ b/drivers/gpu/drm/panfrost/panfrost_gem.c
@@ -49,7 +49,7 @@ static void panfrost_gem_free_object(struct drm_gem_object *obj)
kvfree(bo->sgts);
}
- drm_gem_shmem_free_object(obj);
+ drm_gem_shmem_free(&bo->base);
}
struct panfrost_gem_mapping *
@@ -187,23 +187,25 @@ void panfrost_gem_close(struct drm_gem_object *obj, struct drm_file *file_priv)
static int panfrost_gem_pin(struct drm_gem_object *obj)
{
- if (to_panfrost_bo(obj)->is_heap)
+ struct panfrost_gem_object *bo = to_panfrost_bo(obj);
+
+ if (bo->is_heap)
return -EINVAL;
- return drm_gem_shmem_pin(obj);
+ return drm_gem_shmem_pin(&bo->base);
}
static const struct drm_gem_object_funcs panfrost_gem_funcs = {
.free = panfrost_gem_free_object,
.open = panfrost_gem_open,
.close = panfrost_gem_close,
- .print_info = drm_gem_shmem_print_info,
+ .print_info = drm_gem_shmem_object_print_info,
.pin = panfrost_gem_pin,
- .unpin = drm_gem_shmem_unpin,
- .get_sg_table = drm_gem_shmem_get_sg_table,
- .vmap = drm_gem_shmem_vmap,
- .vunmap = drm_gem_shmem_vunmap,
- .mmap = drm_gem_shmem_mmap,
+ .unpin = drm_gem_shmem_object_unpin,
+ .get_sg_table = drm_gem_shmem_object_get_sg_table,
+ .vmap = drm_gem_shmem_object_vmap,
+ .vunmap = drm_gem_shmem_object_vunmap,
+ .mmap = drm_gem_shmem_object_mmap,
};
/**
@@ -221,7 +223,7 @@ struct drm_gem_object *panfrost_gem_create_object(struct drm_device *dev, size_t
obj = kzalloc(sizeof(*obj), GFP_KERNEL);
if (!obj)
- return NULL;
+ return ERR_PTR(-ENOMEM);
INIT_LIST_HEAD(&obj->mappings.list);
mutex_init(&obj->mappings.lock);
diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
index 1b9f68d..b014234 100644
--- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
+++ b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
@@ -52,7 +52,7 @@ static bool panfrost_gem_purge(struct drm_gem_object *obj)
goto unlock_mappings;
panfrost_gem_teardown_mappings_locked(bo);
- drm_gem_shmem_purge_locked(obj);
+ drm_gem_shmem_purge_locked(&bo->base);
ret = true;
mutex_unlock(&shmem->pages_lock);
diff --git a/drivers/gpu/drm/panfrost/panfrost_mmu.c b/drivers/gpu/drm/panfrost/panfrost_mmu.c
index f51d3f7..39562f2 100644
--- a/drivers/gpu/drm/panfrost/panfrost_mmu.c
+++ b/drivers/gpu/drm/panfrost/panfrost_mmu.c
@@ -304,7 +304,8 @@ static int mmu_map_sg(struct panfrost_device *pfdev, struct panfrost_mmu *mmu,
int panfrost_mmu_map(struct panfrost_gem_mapping *mapping)
{
struct panfrost_gem_object *bo = mapping->obj;
- struct drm_gem_object *obj = &bo->base.base;
+ struct drm_gem_shmem_object *shmem = &bo->base;
+ struct drm_gem_object *obj = &shmem->base;
struct panfrost_device *pfdev = to_panfrost_device(obj->dev);
struct sg_table *sgt;
int prot = IOMMU_READ | IOMMU_WRITE;
@@ -315,7 +316,7 @@ int panfrost_mmu_map(struct panfrost_gem_mapping *mapping)
if (bo->noexec)
prot |= IOMMU_NOEXEC;
- sgt = drm_gem_shmem_get_pages_sgt(obj);
+ sgt = drm_gem_shmem_get_pages_sgt(shmem);
if (WARN_ON(IS_ERR(sgt)))
return PTR_ERR(sgt);
diff --git a/drivers/gpu/drm/panfrost/panfrost_perfcnt.c b/drivers/gpu/drm/panfrost/panfrost_perfcnt.c
index e116a4d..1d36df5 100644
--- a/drivers/gpu/drm/panfrost/panfrost_perfcnt.c
+++ b/drivers/gpu/drm/panfrost/panfrost_perfcnt.c
@@ -105,7 +105,7 @@ static int panfrost_perfcnt_enable_locked(struct panfrost_device *pfdev,
goto err_close_bo;
}
- ret = drm_gem_shmem_vmap(&bo->base, &map);
+ ret = drm_gem_shmem_vmap(bo, &map);
if (ret)
goto err_put_mapping;
perfcnt->buf = map.vaddr;
@@ -164,7 +164,7 @@ static int panfrost_perfcnt_enable_locked(struct panfrost_device *pfdev,
return 0;
err_vunmap:
- drm_gem_shmem_vunmap(&bo->base, &map);
+ drm_gem_shmem_vunmap(bo, &map);
err_put_mapping:
panfrost_gem_mapping_put(perfcnt->mapping);
err_close_bo:
@@ -194,7 +194,7 @@ static int panfrost_perfcnt_disable_locked(struct panfrost_device *pfdev,
GPU_PERFCNT_CFG_MODE(GPU_PERFCNT_CFG_MODE_OFF));
perfcnt->user = NULL;
- drm_gem_shmem_vunmap(&perfcnt->mapping->obj->base.base, &map);
+ drm_gem_shmem_vunmap(&perfcnt->mapping->obj->base, &map);
perfcnt->buf = NULL;
panfrost_gem_close(&perfcnt->mapping->obj->base.base, file_priv);
panfrost_mmu_as_put(pfdev, perfcnt->mapping->mmu);
diff --git a/drivers/gpu/drm/pl111/Kconfig b/drivers/gpu/drm/pl111/Kconfig
index 3aae387..91ee05b 100644
--- a/drivers/gpu/drm/pl111/Kconfig
+++ b/drivers/gpu/drm/pl111/Kconfig
@@ -6,7 +6,6 @@
depends on VEXPRESS_CONFIG || VEXPRESS_CONFIG=n
depends on COMMON_CLK
select DRM_KMS_HELPER
- select DRM_KMS_CMA_HELPER
select DRM_GEM_CMA_HELPER
select DRM_BRIDGE
select DRM_PANEL_BRIDGE
diff --git a/drivers/gpu/drm/qxl/qxl_debugfs.c b/drivers/gpu/drm/qxl/qxl_debugfs.c
index 1f9a596..6a36b0f 100644
--- a/drivers/gpu/drm/qxl/qxl_debugfs.c
+++ b/drivers/gpu/drm/qxl/qxl_debugfs.c
@@ -57,13 +57,16 @@ qxl_debugfs_buffers_info(struct seq_file *m, void *data)
struct qxl_bo *bo;
list_for_each_entry(bo, &qdev->gem.objects, list) {
- struct dma_resv_list *fobj;
- int rel;
+ struct dma_resv_iter cursor;
+ struct dma_fence *fence;
+ int rel = 0;
- rcu_read_lock();
- fobj = dma_resv_shared_list(bo->tbo.base.resv);
- rel = fobj ? fobj->shared_count : 0;
- rcu_read_unlock();
+ dma_resv_iter_begin(&cursor, bo->tbo.base.resv, true);
+ dma_resv_for_each_fence_unlocked(&cursor, fence) {
+ if (dma_resv_iter_is_restarted(&cursor))
+ rel = 0;
+ ++rel;
+ }
seq_printf(m, "size %ld, pc %d, num releases %d\n",
(unsigned long)bo->tbo.base.size,
diff --git a/drivers/gpu/drm/qxl/qxl_drv.c b/drivers/gpu/drm/qxl/qxl_drv.c
index fc47b0d..e4b1642 100644
--- a/drivers/gpu/drm/qxl/qxl_drv.c
+++ b/drivers/gpu/drm/qxl/qxl_drv.c
@@ -29,7 +29,6 @@
#include "qxl_drv.h"
-#include <linux/console.h>
#include <linux/module.h>
#include <linux/pci.h>
#include <linux/vgaarb.h>
@@ -295,7 +294,7 @@ static struct drm_driver qxl_driver = {
static int __init qxl_init(void)
{
- if (vgacon_text_force() && qxl_modeset == -1)
+ if (drm_firmware_drivers_only() && qxl_modeset == -1)
return -EINVAL;
if (qxl_modeset == 0)
diff --git a/drivers/gpu/drm/radeon/radeon_drv.c b/drivers/gpu/drm/radeon/radeon_drv.c
index b74cebc..956c72b 100644
--- a/drivers/gpu/drm/radeon/radeon_drv.c
+++ b/drivers/gpu/drm/radeon/radeon_drv.c
@@ -31,7 +31,6 @@
#include <linux/compat.h>
-#include <linux/console.h>
#include <linux/module.h>
#include <linux/pm_runtime.h>
#include <linux/vga_switcheroo.h>
@@ -637,15 +636,11 @@ static struct pci_driver radeon_kms_pci_driver = {
static int __init radeon_module_init(void)
{
- if (vgacon_text_force() && radeon_modeset == -1) {
- DRM_INFO("VGACON disable radeon kernel modesetting.\n");
+ if (drm_firmware_drivers_only() && radeon_modeset == -1)
radeon_modeset = 0;
- }
- if (radeon_modeset == 0) {
- DRM_ERROR("No UMS support in radeon module!\n");
+ if (radeon_modeset == 0)
return -EINVAL;
- }
DRM_INFO("radeon kernel modesetting enabled.\n");
radeon_register_atpx_handler();
diff --git a/drivers/gpu/drm/radeon/radeon_kms.c b/drivers/gpu/drm/radeon/radeon_kms.c
index 482fb0a..e248855 100644
--- a/drivers/gpu/drm/radeon/radeon_kms.c
+++ b/drivers/gpu/drm/radeon/radeon_kms.c
@@ -168,7 +168,7 @@ int radeon_driver_load_kms(struct drm_device *dev, unsigned long flags)
if (!r) {
acpi_status = radeon_acpi_init(rdev);
if (acpi_status)
- dev_dbg(dev->dev, "Error during ACPI methods call\n");
+ dev_dbg(dev->dev, "Error during ACPI methods call\n");
}
if (radeon_is_px(dev)) {
@@ -648,6 +648,8 @@ void radeon_driver_lastclose_kms(struct drm_device *dev)
int radeon_driver_open_kms(struct drm_device *dev, struct drm_file *file_priv)
{
struct radeon_device *rdev = dev->dev_private;
+ struct radeon_fpriv *fpriv;
+ struct radeon_vm *vm;
int r;
file_priv->driver_priv = NULL;
@@ -660,8 +662,6 @@ int radeon_driver_open_kms(struct drm_device *dev, struct drm_file *file_priv)
/* new gpu have virtual address space support */
if (rdev->family >= CHIP_CAYMAN) {
- struct radeon_fpriv *fpriv;
- struct radeon_vm *vm;
fpriv = kzalloc(sizeof(*fpriv), GFP_KERNEL);
if (unlikely(!fpriv)) {
@@ -672,35 +672,39 @@ int radeon_driver_open_kms(struct drm_device *dev, struct drm_file *file_priv)
if (rdev->accel_working) {
vm = &fpriv->vm;
r = radeon_vm_init(rdev, vm);
- if (r) {
- kfree(fpriv);
- goto out_suspend;
- }
+ if (r)
+ goto out_fpriv;
r = radeon_bo_reserve(rdev->ring_tmp_bo.bo, false);
- if (r) {
- radeon_vm_fini(rdev, vm);
- kfree(fpriv);
- goto out_suspend;
- }
+ if (r)
+ goto out_vm_fini;
/* map the ib pool buffer read only into
* virtual address space */
vm->ib_bo_va = radeon_vm_bo_add(rdev, vm,
rdev->ring_tmp_bo.bo);
+ if (!vm->ib_bo_va) {
+ r = -ENOMEM;
+ goto out_vm_fini;
+ }
+
r = radeon_vm_bo_set_addr(rdev, vm->ib_bo_va,
RADEON_VA_IB_OFFSET,
RADEON_VM_PAGE_READABLE |
RADEON_VM_PAGE_SNOOPED);
- if (r) {
- radeon_vm_fini(rdev, vm);
- kfree(fpriv);
- goto out_suspend;
- }
+ if (r)
+ goto out_vm_fini;
}
file_priv->driver_priv = fpriv;
}
+ if (!r)
+ goto out_suspend;
+
+out_vm_fini:
+ radeon_vm_fini(rdev, vm);
+out_fpriv:
+ kfree(fpriv);
out_suspend:
pm_runtime_mark_last_busy(dev->dev);
pm_runtime_put_autosuspend(dev->dev);
diff --git a/drivers/gpu/drm/radeon/radeon_sync.c b/drivers/gpu/drm/radeon/radeon_sync.c
index 9257b60..b991ba1b 100644
--- a/drivers/gpu/drm/radeon/radeon_sync.c
+++ b/drivers/gpu/drm/radeon/radeon_sync.c
@@ -91,33 +91,17 @@ int radeon_sync_resv(struct radeon_device *rdev,
struct dma_resv *resv,
bool shared)
{
- struct dma_resv_list *flist;
- struct dma_fence *f;
+ struct dma_resv_iter cursor;
struct radeon_fence *fence;
- unsigned i;
+ struct dma_fence *f;
int r = 0;
- /* always sync to the exclusive fence */
- f = dma_resv_excl_fence(resv);
- fence = f ? to_radeon_fence(f) : NULL;
- if (fence && fence->rdev == rdev)
- radeon_sync_fence(sync, fence);
- else if (f)
- r = dma_fence_wait(f, true);
-
- flist = dma_resv_shared_list(resv);
- if (shared || !flist || r)
- return r;
-
- for (i = 0; i < flist->shared_count; ++i) {
- f = rcu_dereference_protected(flist->shared[i],
- dma_resv_held(resv));
+ dma_resv_for_each_fence(&cursor, resv, shared, f) {
fence = to_radeon_fence(f);
if (fence && fence->rdev == rdev)
radeon_sync_fence(sync, fence);
else
r = dma_fence_wait(f, true);
-
if (r)
break;
}
diff --git a/drivers/gpu/drm/radeon/radeon_uvd.c b/drivers/gpu/drm/radeon/radeon_uvd.c
index 2ea8691..377f9cd 100644
--- a/drivers/gpu/drm/radeon/radeon_uvd.c
+++ b/drivers/gpu/drm/radeon/radeon_uvd.c
@@ -469,7 +469,6 @@ static int radeon_uvd_cs_msg(struct radeon_cs_parser *p, struct radeon_bo *bo,
{
int32_t *msg, msg_type, handle;
unsigned img_size = 0;
- struct dma_fence *f;
void *ptr;
int i, r;
@@ -479,13 +478,11 @@ static int radeon_uvd_cs_msg(struct radeon_cs_parser *p, struct radeon_bo *bo,
return -EINVAL;
}
- f = dma_resv_excl_fence(bo->tbo.base.resv);
- if (f) {
- r = radeon_fence_wait((struct radeon_fence *)f, false);
- if (r) {
- DRM_ERROR("Failed waiting for UVD message (%d)!\n", r);
- return r;
- }
+ r = dma_resv_wait_timeout(bo->tbo.base.resv, false, false,
+ MAX_SCHEDULE_TIMEOUT);
+ if (r <= 0) {
+ DRM_ERROR("Failed waiting for UVD message (%d)!\n", r);
+ return r ? r : -ETIME;
}
r = radeon_bo_kmap(bo, &ptr);
diff --git a/drivers/gpu/drm/radeon/radeon_vce.c b/drivers/gpu/drm/radeon/radeon_vce.c
index 511a942e..ca4a364 100644
--- a/drivers/gpu/drm/radeon/radeon_vce.c
+++ b/drivers/gpu/drm/radeon/radeon_vce.c
@@ -513,7 +513,7 @@ int radeon_vce_cs_reloc(struct radeon_cs_parser *p, int lo, int hi,
* @allocated: allocated a new handle?
*
* Validates the handle and return the found session index or -EINVAL
- * we we don't have another free session index.
+ * we don't have another free session index.
*/
static int radeon_vce_validate_handle(struct radeon_cs_parser *p,
uint32_t handle, bool *allocated)
diff --git a/drivers/gpu/drm/rcar-du/Kconfig b/drivers/gpu/drm/rcar-du/Kconfig
index b47e744..f6e6a6d 100644
--- a/drivers/gpu/drm/rcar-du/Kconfig
+++ b/drivers/gpu/drm/rcar-du/Kconfig
@@ -4,23 +4,24 @@
depends on DRM && OF
depends on ARM || ARM64
depends on ARCH_RENESAS || COMPILE_TEST
- imply DRM_RCAR_CMM
- imply DRM_RCAR_LVDS
select DRM_KMS_HELPER
- select DRM_KMS_CMA_HELPER
select DRM_GEM_CMA_HELPER
select VIDEOMODE_HELPERS
help
Choose this option if you have an R-Car chipset.
If M is selected the module will be called rcar-du-drm.
-config DRM_RCAR_CMM
- tristate "R-Car DU Color Management Module (CMM) Support"
- depends on DRM && OF
+config DRM_RCAR_USE_CMM
+ bool "R-Car DU Color Management Module (CMM) Support"
depends on DRM_RCAR_DU
+ default DRM_RCAR_DU
help
Enable support for R-Car Color Management Module (CMM).
+config DRM_RCAR_CMM
+ def_tristate DRM_RCAR_DU
+ depends on DRM_RCAR_USE_CMM
+
config DRM_RCAR_DW_HDMI
tristate "R-Car Gen3 and RZ/G2 DU HDMI Encoder Support"
depends on DRM && OF
@@ -28,15 +29,27 @@
help
Enable support for R-Car Gen3 or RZ/G2 internal HDMI encoder.
+config DRM_RCAR_USE_LVDS
+ bool "R-Car DU LVDS Encoder Support"
+ depends on DRM_BRIDGE && OF
+ default DRM_RCAR_DU
+ help
+ Enable support for the R-Car Display Unit embedded LVDS encoders.
+
config DRM_RCAR_LVDS
- tristate "R-Car DU LVDS Encoder Support"
- depends on DRM && DRM_BRIDGE && OF
+ def_tristate DRM_RCAR_DU
+ depends on DRM_RCAR_USE_LVDS
select DRM_KMS_HELPER
select DRM_PANEL
select OF_FLATTREE
select OF_OVERLAY
+
+config DRM_RCAR_MIPI_DSI
+ tristate "R-Car DU MIPI DSI Encoder Support"
+ depends on DRM && DRM_BRIDGE && OF
+ select DRM_MIPI_DSI
help
- Enable support for the R-Car Display Unit embedded LVDS encoders.
+ Enable support for the R-Car Display Unit embedded MIPI DSI encoders.
config DRM_RCAR_VSP
bool "R-Car DU VSP Compositor Support" if ARM
diff --git a/drivers/gpu/drm/rcar-du/Makefile b/drivers/gpu/drm/rcar-du/Makefile
index 4d1187c..286bc81 100644
--- a/drivers/gpu/drm/rcar-du/Makefile
+++ b/drivers/gpu/drm/rcar-du/Makefile
@@ -19,6 +19,7 @@
obj-$(CONFIG_DRM_RCAR_DU) += rcar-du-drm.o
obj-$(CONFIG_DRM_RCAR_DW_HDMI) += rcar_dw_hdmi.o
obj-$(CONFIG_DRM_RCAR_LVDS) += rcar_lvds.o
+obj-$(CONFIG_DRM_RCAR_MIPI_DSI) += rcar_mipi_dsi.o
# 'remote-endpoint' is fixed up at run-time
DTC_FLAGS_rcar_du_of_lvds_r8a7790 += -Wno-graph_endpoint
diff --git a/drivers/gpu/drm/rcar-du/rcar_du_crtc.c b/drivers/gpu/drm/rcar-du/rcar_du_crtc.c
index 5672830..f361a60 100644
--- a/drivers/gpu/drm/rcar-du/rcar_du_crtc.c
+++ b/drivers/gpu/drm/rcar-du/rcar_du_crtc.c
@@ -215,6 +215,7 @@ static void rcar_du_crtc_set_display_timing(struct rcar_du_crtc *rcrtc)
const struct drm_display_mode *mode = &rcrtc->crtc.state->adjusted_mode;
struct rcar_du_device *rcdu = rcrtc->dev;
unsigned long mode_clock = mode->clock * 1000;
+ unsigned int hdse_offset;
u32 dsmr;
u32 escr;
@@ -261,12 +262,13 @@ static void rcar_du_crtc_set_display_timing(struct rcar_du_crtc *rcrtc)
rcar_du_group_write(rcrtc->group, DPLLCR, dpllcr);
escr = ESCR_DCLKSEL_DCLKIN | div;
- } else if (rcdu->info->lvds_clk_mask & BIT(rcrtc->index)) {
+ } else if (rcdu->info->lvds_clk_mask & BIT(rcrtc->index) ||
+ rcdu->info->dsi_clk_mask & BIT(rcrtc->index)) {
/*
- * Use the LVDS PLL output as the dot clock when outputting to
- * the LVDS encoder on an SoC that supports this clock routing
- * option. We use the clock directly in that case, without any
- * additional divider.
+ * Use the external LVDS or DSI PLL output as the dot clock when
+ * outputting to the LVDS or DSI encoder on an SoC that supports
+ * this clock routing option. We use the clock directly in that
+ * case, without any additional divider.
*/
escr = ESCR_DCLKSEL_DCLKIN;
} else {
@@ -298,10 +300,15 @@ static void rcar_du_crtc_set_display_timing(struct rcar_du_crtc *rcrtc)
| DSMR_DIPM_DISP | DSMR_CSPM;
rcar_du_crtc_write(rcrtc, DSMR, dsmr);
+ hdse_offset = 19;
+ if (rcrtc->group->cmms_mask & BIT(rcrtc->index % 2))
+ hdse_offset += 25;
+
/* Display timings */
- rcar_du_crtc_write(rcrtc, HDSR, mode->htotal - mode->hsync_start - 19);
+ rcar_du_crtc_write(rcrtc, HDSR, mode->htotal - mode->hsync_start -
+ hdse_offset);
rcar_du_crtc_write(rcrtc, HDER, mode->htotal - mode->hsync_start +
- mode->hdisplay - 19);
+ mode->hdisplay - hdse_offset);
rcar_du_crtc_write(rcrtc, HSWR, mode->hsync_end -
mode->hsync_start - 1);
rcar_du_crtc_write(rcrtc, HCR, mode->htotal - 1);
@@ -836,6 +843,7 @@ rcar_du_crtc_mode_valid(struct drm_crtc *crtc,
struct rcar_du_crtc *rcrtc = to_rcar_crtc(crtc);
struct rcar_du_device *rcdu = rcrtc->dev;
bool interlaced = mode->flags & DRM_MODE_FLAG_INTERLACE;
+ unsigned int min_sync_porch;
unsigned int vbp;
if (interlaced && !rcar_du_has(rcdu, RCAR_DU_FEATURE_INTERLACED))
@@ -843,9 +851,14 @@ rcar_du_crtc_mode_valid(struct drm_crtc *crtc,
/*
* The hardware requires a minimum combined horizontal sync and back
- * porch of 20 pixels and a minimum vertical back porch of 3 lines.
+ * porch of 20 pixels (when CMM isn't used) or 45 pixels (when CMM is
+ * used), and a minimum vertical back porch of 3 lines.
*/
- if (mode->htotal - mode->hsync_start < 20)
+ min_sync_porch = 20;
+ if (rcrtc->group->cmms_mask & BIT(rcrtc->index % 2))
+ min_sync_porch += 25;
+
+ if (mode->htotal - mode->hsync_start < min_sync_porch)
return MODE_HBLANK_NARROW;
vbp = (mode->vtotal - mode->vsync_end) / (interlaced ? 2 : 1);
diff --git a/drivers/gpu/drm/rcar-du/rcar_du_drv.c b/drivers/gpu/drm/rcar-du/rcar_du_drv.c
index 5612a9e..5a8131ef 100644
--- a/drivers/gpu/drm/rcar-du/rcar_du_drv.c
+++ b/drivers/gpu/drm/rcar-du/rcar_du_drv.c
@@ -544,10 +544,12 @@ const char *rcar_du_output_name(enum rcar_du_output output)
static const char * const names[] = {
[RCAR_DU_OUTPUT_DPAD0] = "DPAD0",
[RCAR_DU_OUTPUT_DPAD1] = "DPAD1",
- [RCAR_DU_OUTPUT_LVDS0] = "LVDS0",
- [RCAR_DU_OUTPUT_LVDS1] = "LVDS1",
+ [RCAR_DU_OUTPUT_DSI0] = "DSI0",
+ [RCAR_DU_OUTPUT_DSI1] = "DSI1",
[RCAR_DU_OUTPUT_HDMI0] = "HDMI0",
[RCAR_DU_OUTPUT_HDMI1] = "HDMI1",
+ [RCAR_DU_OUTPUT_LVDS0] = "LVDS0",
+ [RCAR_DU_OUTPUT_LVDS1] = "LVDS1",
[RCAR_DU_OUTPUT_TCON] = "TCON",
};
diff --git a/drivers/gpu/drm/rcar-du/rcar_du_kms.c b/drivers/gpu/drm/rcar-du/rcar_du_kms.c
index eacb1f1..190dbb7 100644
--- a/drivers/gpu/drm/rcar-du/rcar_du_kms.c
+++ b/drivers/gpu/drm/rcar-du/rcar_du_kms.c
@@ -327,11 +327,11 @@ const struct rcar_du_format_info *rcar_du_format_info(u32 fourcc)
*/
static const struct drm_gem_object_funcs rcar_du_gem_funcs = {
- .free = drm_gem_cma_free_object,
- .print_info = drm_gem_cma_print_info,
- .get_sg_table = drm_gem_cma_get_sg_table,
- .vmap = drm_gem_cma_vmap,
- .mmap = drm_gem_cma_mmap,
+ .free = drm_gem_cma_object_free,
+ .print_info = drm_gem_cma_object_print_info,
+ .get_sg_table = drm_gem_cma_object_get_sg_table,
+ .vmap = drm_gem_cma_object_vmap,
+ .mmap = drm_gem_cma_object_mmap,
.vm_ops = &drm_gem_cma_vm_ops,
};
diff --git a/drivers/gpu/drm/rcar-du/rcar_mipi_dsi.c b/drivers/gpu/drm/rcar-du/rcar_mipi_dsi.c
new file mode 100644
index 0000000..891bb95
--- /dev/null
+++ b/drivers/gpu/drm/rcar-du/rcar_mipi_dsi.c
@@ -0,0 +1,819 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * rcar_mipi_dsi.c -- R-Car MIPI DSI Encoder
+ *
+ * Copyright (C) 2020 Renesas Electronics Corporation
+ */
+
+#include <linux/clk.h>
+#include <linux/delay.h>
+#include <linux/io.h>
+#include <linux/iopoll.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/of_device.h>
+#include <linux/of_graph.h>
+#include <linux/platform_device.h>
+#include <linux/reset.h>
+#include <linux/slab.h>
+
+#include <drm/drm_atomic.h>
+#include <drm/drm_atomic_helper.h>
+#include <drm/drm_bridge.h>
+#include <drm/drm_mipi_dsi.h>
+#include <drm/drm_of.h>
+#include <drm/drm_panel.h>
+#include <drm/drm_probe_helper.h>
+
+#include "rcar_mipi_dsi_regs.h"
+
+struct rcar_mipi_dsi {
+ struct device *dev;
+ const struct rcar_mipi_dsi_device_info *info;
+ struct reset_control *rstc;
+
+ struct mipi_dsi_host host;
+ struct drm_bridge bridge;
+ struct drm_bridge *next_bridge;
+ struct drm_connector connector;
+
+ void __iomem *mmio;
+ struct {
+ struct clk *mod;
+ struct clk *pll;
+ struct clk *dsi;
+ } clocks;
+
+ enum mipi_dsi_pixel_format format;
+ unsigned int num_data_lanes;
+ unsigned int lanes;
+};
+
+static inline struct rcar_mipi_dsi *
+bridge_to_rcar_mipi_dsi(struct drm_bridge *bridge)
+{
+ return container_of(bridge, struct rcar_mipi_dsi, bridge);
+}
+
+static inline struct rcar_mipi_dsi *
+host_to_rcar_mipi_dsi(struct mipi_dsi_host *host)
+{
+ return container_of(host, struct rcar_mipi_dsi, host);
+}
+
+static const u32 phtw[] = {
+ 0x01020114, 0x01600115, /* General testing */
+ 0x01030116, 0x0102011d, /* General testing */
+ 0x011101a4, 0x018601a4, /* 1Gbps testing */
+ 0x014201a0, 0x010001a3, /* 1Gbps testing */
+ 0x0101011f, /* 1Gbps testing */
+};
+
+static const u32 phtw2[] = {
+ 0x010c0130, 0x010c0140, /* General testing */
+ 0x010c0150, 0x010c0180, /* General testing */
+ 0x010c0190,
+ 0x010a0160, 0x010a0170,
+ 0x01800164, 0x01800174, /* 1Gbps testing */
+};
+
+static const u32 hsfreqrange_table[][2] = {
+ { 80000000U, 0x00 }, { 90000000U, 0x10 }, { 100000000U, 0x20 },
+ { 110000000U, 0x30 }, { 120000000U, 0x01 }, { 130000000U, 0x11 },
+ { 140000000U, 0x21 }, { 150000000U, 0x31 }, { 160000000U, 0x02 },
+ { 170000000U, 0x12 }, { 180000000U, 0x22 }, { 190000000U, 0x32 },
+ { 205000000U, 0x03 }, { 220000000U, 0x13 }, { 235000000U, 0x23 },
+ { 250000000U, 0x33 }, { 275000000U, 0x04 }, { 300000000U, 0x14 },
+ { 325000000U, 0x25 }, { 350000000U, 0x35 }, { 400000000U, 0x05 },
+ { 450000000U, 0x16 }, { 500000000U, 0x26 }, { 550000000U, 0x37 },
+ { 600000000U, 0x07 }, { 650000000U, 0x18 }, { 700000000U, 0x28 },
+ { 750000000U, 0x39 }, { 800000000U, 0x09 }, { 850000000U, 0x19 },
+ { 900000000U, 0x29 }, { 950000000U, 0x3a }, { 1000000000U, 0x0a },
+ { 1050000000U, 0x1a }, { 1100000000U, 0x2a }, { 1150000000U, 0x3b },
+ { 1200000000U, 0x0b }, { 1250000000U, 0x1b }, { 1300000000U, 0x2b },
+ { 1350000000U, 0x3c }, { 1400000000U, 0x0c }, { 1450000000U, 0x1c },
+ { 1500000000U, 0x2c }, { 1550000000U, 0x3d }, { 1600000000U, 0x0d },
+ { 1650000000U, 0x1d }, { 1700000000U, 0x2e }, { 1750000000U, 0x3e },
+ { 1800000000U, 0x0e }, { 1850000000U, 0x1e }, { 1900000000U, 0x2f },
+ { 1950000000U, 0x3f }, { 2000000000U, 0x0f }, { 2050000000U, 0x40 },
+ { 2100000000U, 0x41 }, { 2150000000U, 0x42 }, { 2200000000U, 0x43 },
+ { 2250000000U, 0x44 }, { 2300000000U, 0x45 }, { 2350000000U, 0x46 },
+ { 2400000000U, 0x47 }, { 2450000000U, 0x48 }, { 2500000000U, 0x49 },
+ { /* sentinel */ },
+};
+
+struct vco_cntrl_value {
+ u32 min_freq;
+ u32 max_freq;
+ u16 value;
+};
+
+static const struct vco_cntrl_value vco_cntrl_table[] = {
+ { .min_freq = 40000000U, .max_freq = 55000000U, .value = 0x3f },
+ { .min_freq = 52500000U, .max_freq = 80000000U, .value = 0x39 },
+ { .min_freq = 80000000U, .max_freq = 110000000U, .value = 0x2f },
+ { .min_freq = 105000000U, .max_freq = 160000000U, .value = 0x29 },
+ { .min_freq = 160000000U, .max_freq = 220000000U, .value = 0x1f },
+ { .min_freq = 210000000U, .max_freq = 320000000U, .value = 0x19 },
+ { .min_freq = 320000000U, .max_freq = 440000000U, .value = 0x0f },
+ { .min_freq = 420000000U, .max_freq = 660000000U, .value = 0x09 },
+ { .min_freq = 630000000U, .max_freq = 1149000000U, .value = 0x03 },
+ { .min_freq = 1100000000U, .max_freq = 1152000000U, .value = 0x01 },
+ { .min_freq = 1150000000U, .max_freq = 1250000000U, .value = 0x01 },
+ { /* sentinel */ },
+};
+
+static void rcar_mipi_dsi_write(struct rcar_mipi_dsi *dsi, u32 reg, u32 data)
+{
+ iowrite32(data, dsi->mmio + reg);
+}
+
+static u32 rcar_mipi_dsi_read(struct rcar_mipi_dsi *dsi, u32 reg)
+{
+ return ioread32(dsi->mmio + reg);
+}
+
+static void rcar_mipi_dsi_clr(struct rcar_mipi_dsi *dsi, u32 reg, u32 clr)
+{
+ rcar_mipi_dsi_write(dsi, reg, rcar_mipi_dsi_read(dsi, reg) & ~clr);
+}
+
+static void rcar_mipi_dsi_set(struct rcar_mipi_dsi *dsi, u32 reg, u32 set)
+{
+ rcar_mipi_dsi_write(dsi, reg, rcar_mipi_dsi_read(dsi, reg) | set);
+}
+
+static int rcar_mipi_dsi_phtw_test(struct rcar_mipi_dsi *dsi, u32 phtw)
+{
+ u32 status;
+ int ret;
+
+ rcar_mipi_dsi_write(dsi, PHTW, phtw);
+
+ ret = read_poll_timeout(rcar_mipi_dsi_read, status,
+ !(status & (PHTW_DWEN | PHTW_CWEN)),
+ 2000, 10000, false, dsi, PHTW);
+ if (ret < 0) {
+ dev_err(dsi->dev, "PHY test interface write timeout (0x%08x)\n",
+ phtw);
+ return ret;
+ }
+
+ return ret;
+}
+
+/* -----------------------------------------------------------------------------
+ * Hardware Setup
+ */
+
+struct dsi_setup_info {
+ unsigned long fout;
+ u16 vco_cntrl;
+ u16 prop_cntrl;
+ u16 hsfreqrange;
+ u16 div;
+ unsigned int m;
+ unsigned int n;
+};
+
+static void rcar_mipi_dsi_parameters_calc(struct rcar_mipi_dsi *dsi,
+ struct clk *clk, unsigned long target,
+ struct dsi_setup_info *setup_info)
+{
+
+ const struct vco_cntrl_value *vco_cntrl;
+ unsigned long fout_target;
+ unsigned long fin, fout;
+ unsigned long hsfreq;
+ unsigned int best_err = -1;
+ unsigned int divider;
+ unsigned int n;
+ unsigned int i;
+ unsigned int err;
+
+ /*
+ * Calculate Fout = dot clock * ColorDepth / (2 * Lane Count)
+ * The range out Fout is [40 - 1250] Mhz
+ */
+ fout_target = target * mipi_dsi_pixel_format_to_bpp(dsi->format)
+ / (2 * dsi->lanes);
+ if (fout_target < 40000000 || fout_target > 1250000000)
+ return;
+
+ /* Find vco_cntrl */
+ for (vco_cntrl = vco_cntrl_table; vco_cntrl->min_freq != 0; vco_cntrl++) {
+ if (fout_target > vco_cntrl->min_freq &&
+ fout_target <= vco_cntrl->max_freq) {
+ setup_info->vco_cntrl = vco_cntrl->value;
+ if (fout_target >= 1150000000)
+ setup_info->prop_cntrl = 0x0c;
+ else
+ setup_info->prop_cntrl = 0x0b;
+ break;
+ }
+ }
+
+ /* Add divider */
+ setup_info->div = (setup_info->vco_cntrl & 0x30) >> 4;
+
+ /* Find hsfreqrange */
+ hsfreq = fout_target * 2;
+ for (i = 0; i < ARRAY_SIZE(hsfreqrange_table); i++) {
+ if (hsfreqrange_table[i][0] >= hsfreq) {
+ setup_info->hsfreqrange = hsfreqrange_table[i][1];
+ break;
+ }
+ }
+
+ /*
+ * Calculate n and m for PLL clock
+ * Following the HW manual the ranges of n and m are
+ * n = [3-8] and m = [64-625]
+ */
+ fin = clk_get_rate(clk);
+ divider = 1 << setup_info->div;
+ for (n = 3; n < 9; n++) {
+ unsigned long fpfd;
+ unsigned int m;
+
+ fpfd = fin / n;
+
+ for (m = 64; m < 626; m++) {
+ fout = fpfd * m / divider;
+ err = abs((long)(fout - fout_target) * 10000 /
+ (long)fout_target);
+ if (err < best_err) {
+ setup_info->m = m - 2;
+ setup_info->n = n - 1;
+ setup_info->fout = fout;
+ best_err = err;
+ if (err == 0)
+ goto done;
+ }
+ }
+ }
+
+done:
+ dev_dbg(dsi->dev,
+ "%pC %lu Hz -> Fout %lu Hz (target %lu Hz, error %d.%02u%%), PLL M/N/DIV %u/%u/%u\n",
+ clk, fin, setup_info->fout, fout_target, best_err / 100,
+ best_err % 100, setup_info->m, setup_info->n, setup_info->div);
+ dev_dbg(dsi->dev,
+ "vco_cntrl = 0x%x\tprop_cntrl = 0x%x\thsfreqrange = 0x%x\n",
+ setup_info->vco_cntrl, setup_info->prop_cntrl,
+ setup_info->hsfreqrange);
+}
+
+static void rcar_mipi_dsi_set_display_timing(struct rcar_mipi_dsi *dsi,
+ const struct drm_display_mode *mode)
+{
+ u32 setr;
+ u32 vprmset0r;
+ u32 vprmset1r;
+ u32 vprmset2r;
+ u32 vprmset3r;
+ u32 vprmset4r;
+
+ /* Configuration for Pixel Stream and Packet Header */
+ if (mipi_dsi_pixel_format_to_bpp(dsi->format) == 24)
+ rcar_mipi_dsi_write(dsi, TXVMPSPHSETR, TXVMPSPHSETR_DT_RGB24);
+ else if (mipi_dsi_pixel_format_to_bpp(dsi->format) == 18)
+ rcar_mipi_dsi_write(dsi, TXVMPSPHSETR, TXVMPSPHSETR_DT_RGB18);
+ else if (mipi_dsi_pixel_format_to_bpp(dsi->format) == 16)
+ rcar_mipi_dsi_write(dsi, TXVMPSPHSETR, TXVMPSPHSETR_DT_RGB16);
+ else {
+ dev_warn(dsi->dev, "unsupported format");
+ return;
+ }
+
+ /* Configuration for Blanking sequence and Input Pixel */
+ setr = TXVMSETR_HSABPEN_EN | TXVMSETR_HBPBPEN_EN
+ | TXVMSETR_HFPBPEN_EN | TXVMSETR_SYNSEQ_PULSES
+ | TXVMSETR_PIXWDTH | TXVMSETR_VSTPM;
+ rcar_mipi_dsi_write(dsi, TXVMSETR, setr);
+
+ /* Configuration for Video Parameters */
+ vprmset0r = (mode->flags & DRM_MODE_FLAG_PVSYNC ?
+ TXVMVPRMSET0R_VSPOL_HIG : TXVMVPRMSET0R_VSPOL_LOW)
+ | (mode->flags & DRM_MODE_FLAG_PHSYNC ?
+ TXVMVPRMSET0R_HSPOL_HIG : TXVMVPRMSET0R_HSPOL_LOW)
+ | TXVMVPRMSET0R_CSPC_RGB | TXVMVPRMSET0R_BPP_24;
+
+ vprmset1r = TXVMVPRMSET1R_VACTIVE(mode->vdisplay)
+ | TXVMVPRMSET1R_VSA(mode->vsync_end - mode->vsync_start);
+
+ vprmset2r = TXVMVPRMSET2R_VFP(mode->vsync_start - mode->vdisplay)
+ | TXVMVPRMSET2R_VBP(mode->vtotal - mode->vsync_end);
+
+ vprmset3r = TXVMVPRMSET3R_HACTIVE(mode->hdisplay)
+ | TXVMVPRMSET3R_HSA(mode->hsync_end - mode->hsync_start);
+
+ vprmset4r = TXVMVPRMSET4R_HFP(mode->hsync_start - mode->hdisplay)
+ | TXVMVPRMSET4R_HBP(mode->htotal - mode->hsync_end);
+
+ rcar_mipi_dsi_write(dsi, TXVMVPRMSET0R, vprmset0r);
+ rcar_mipi_dsi_write(dsi, TXVMVPRMSET1R, vprmset1r);
+ rcar_mipi_dsi_write(dsi, TXVMVPRMSET2R, vprmset2r);
+ rcar_mipi_dsi_write(dsi, TXVMVPRMSET3R, vprmset3r);
+ rcar_mipi_dsi_write(dsi, TXVMVPRMSET4R, vprmset4r);
+}
+
+static int rcar_mipi_dsi_startup(struct rcar_mipi_dsi *dsi,
+ const struct drm_display_mode *mode)
+{
+ struct dsi_setup_info setup_info = {};
+ unsigned int timeout;
+ int ret, i;
+ int dsi_format;
+ u32 phy_setup;
+ u32 clockset2, clockset3;
+ u32 ppisetr;
+ u32 vclkset;
+
+ /* Checking valid format */
+ dsi_format = mipi_dsi_pixel_format_to_bpp(dsi->format);
+ if (dsi_format < 0) {
+ dev_warn(dsi->dev, "invalid format");
+ return -EINVAL;
+ }
+
+ /* Parameters Calculation */
+ rcar_mipi_dsi_parameters_calc(dsi, dsi->clocks.pll,
+ mode->clock * 1000, &setup_info);
+
+ /* LPCLK enable */
+ rcar_mipi_dsi_set(dsi, LPCLKSET, LPCLKSET_CKEN);
+
+ /* CFGCLK enabled */
+ rcar_mipi_dsi_set(dsi, CFGCLKSET, CFGCLKSET_CKEN);
+
+ rcar_mipi_dsi_clr(dsi, PHYSETUP, PHYSETUP_RSTZ);
+ rcar_mipi_dsi_clr(dsi, PHYSETUP, PHYSETUP_SHUTDOWNZ);
+
+ rcar_mipi_dsi_set(dsi, PHTC, PHTC_TESTCLR);
+ rcar_mipi_dsi_clr(dsi, PHTC, PHTC_TESTCLR);
+
+ /* PHY setting */
+ phy_setup = rcar_mipi_dsi_read(dsi, PHYSETUP);
+ phy_setup &= ~PHYSETUP_HSFREQRANGE_MASK;
+ phy_setup |= PHYSETUP_HSFREQRANGE(setup_info.hsfreqrange);
+ rcar_mipi_dsi_write(dsi, PHYSETUP, phy_setup);
+
+ for (i = 0; i < ARRAY_SIZE(phtw); i++) {
+ ret = rcar_mipi_dsi_phtw_test(dsi, phtw[i]);
+ if (ret < 0)
+ return ret;
+ }
+
+ /* PLL Clock Setting */
+ rcar_mipi_dsi_clr(dsi, CLOCKSET1, CLOCKSET1_SHADOW_CLEAR);
+ rcar_mipi_dsi_set(dsi, CLOCKSET1, CLOCKSET1_SHADOW_CLEAR);
+ rcar_mipi_dsi_clr(dsi, CLOCKSET1, CLOCKSET1_SHADOW_CLEAR);
+
+ clockset2 = CLOCKSET2_M(setup_info.m) | CLOCKSET2_N(setup_info.n)
+ | CLOCKSET2_VCO_CNTRL(setup_info.vco_cntrl);
+ clockset3 = CLOCKSET3_PROP_CNTRL(setup_info.prop_cntrl)
+ | CLOCKSET3_INT_CNTRL(0)
+ | CLOCKSET3_CPBIAS_CNTRL(0x10)
+ | CLOCKSET3_GMP_CNTRL(1);
+ rcar_mipi_dsi_write(dsi, CLOCKSET2, clockset2);
+ rcar_mipi_dsi_write(dsi, CLOCKSET3, clockset3);
+
+ rcar_mipi_dsi_clr(dsi, CLOCKSET1, CLOCKSET1_UPDATEPLL);
+ rcar_mipi_dsi_set(dsi, CLOCKSET1, CLOCKSET1_UPDATEPLL);
+ udelay(10);
+ rcar_mipi_dsi_clr(dsi, CLOCKSET1, CLOCKSET1_UPDATEPLL);
+
+ ppisetr = PPISETR_DLEN_3 | PPISETR_CLEN;
+ rcar_mipi_dsi_write(dsi, PPISETR, ppisetr);
+
+ rcar_mipi_dsi_set(dsi, PHYSETUP, PHYSETUP_SHUTDOWNZ);
+ rcar_mipi_dsi_set(dsi, PHYSETUP, PHYSETUP_RSTZ);
+ usleep_range(400, 500);
+
+ /* Checking PPI clock status register */
+ for (timeout = 10; timeout > 0; --timeout) {
+ if ((rcar_mipi_dsi_read(dsi, PPICLSR) & PPICLSR_STPST) &&
+ (rcar_mipi_dsi_read(dsi, PPIDLSR) & PPIDLSR_STPST) &&
+ (rcar_mipi_dsi_read(dsi, CLOCKSET1) & CLOCKSET1_LOCK))
+ break;
+
+ usleep_range(1000, 2000);
+ }
+
+ if (!timeout) {
+ dev_err(dsi->dev, "failed to enable PPI clock\n");
+ return -ETIMEDOUT;
+ }
+
+ for (i = 0; i < ARRAY_SIZE(phtw2); i++) {
+ ret = rcar_mipi_dsi_phtw_test(dsi, phtw2[i]);
+ if (ret < 0)
+ return ret;
+ }
+
+ /* Enable DOT clock */
+ vclkset = VCLKSET_CKEN;
+ rcar_mipi_dsi_set(dsi, VCLKSET, vclkset);
+
+ if (dsi_format == 24)
+ vclkset |= VCLKSET_BPP_24;
+ else if (dsi_format == 18)
+ vclkset |= VCLKSET_BPP_18;
+ else if (dsi_format == 16)
+ vclkset |= VCLKSET_BPP_16;
+ else {
+ dev_warn(dsi->dev, "unsupported format");
+ return -EINVAL;
+ }
+ vclkset |= VCLKSET_COLOR_RGB | VCLKSET_DIV(setup_info.div)
+ | VCLKSET_LANE(dsi->lanes - 1);
+
+ rcar_mipi_dsi_set(dsi, VCLKSET, vclkset);
+
+ /* After setting VCLKSET register, enable VCLKEN */
+ rcar_mipi_dsi_set(dsi, VCLKEN, VCLKEN_CKEN);
+
+ dev_dbg(dsi->dev, "DSI device is started\n");
+
+ return 0;
+}
+
+static void rcar_mipi_dsi_shutdown(struct rcar_mipi_dsi *dsi)
+{
+ rcar_mipi_dsi_clr(dsi, PHYSETUP, PHYSETUP_RSTZ);
+ rcar_mipi_dsi_clr(dsi, PHYSETUP, PHYSETUP_SHUTDOWNZ);
+
+ dev_dbg(dsi->dev, "DSI device is shutdown\n");
+}
+
+static int rcar_mipi_dsi_clk_enable(struct rcar_mipi_dsi *dsi)
+{
+ int ret;
+
+ reset_control_deassert(dsi->rstc);
+
+ ret = clk_prepare_enable(dsi->clocks.mod);
+ if (ret < 0)
+ goto err_reset;
+
+ ret = clk_prepare_enable(dsi->clocks.dsi);
+ if (ret < 0)
+ goto err_clock;
+
+ return 0;
+
+err_clock:
+ clk_disable_unprepare(dsi->clocks.mod);
+err_reset:
+ reset_control_assert(dsi->rstc);
+ return ret;
+}
+
+static void rcar_mipi_dsi_clk_disable(struct rcar_mipi_dsi *dsi)
+{
+ clk_disable_unprepare(dsi->clocks.dsi);
+ clk_disable_unprepare(dsi->clocks.mod);
+
+ reset_control_assert(dsi->rstc);
+}
+
+static int rcar_mipi_dsi_start_hs_clock(struct rcar_mipi_dsi *dsi)
+{
+ /*
+ * In HW manual, we need to check TxDDRClkHS-Q Stable? but it dont
+ * write how to check. So we skip this check in this patch
+ */
+ u32 status;
+ int ret;
+
+ /* Start HS clock. */
+ rcar_mipi_dsi_set(dsi, PPICLCR, PPICLCR_TXREQHS);
+
+ ret = read_poll_timeout(rcar_mipi_dsi_read, status,
+ status & PPICLSR_TOHS,
+ 2000, 10000, false, dsi, PPICLSR);
+ if (ret < 0) {
+ dev_err(dsi->dev, "failed to enable HS clock\n");
+ return ret;
+ }
+
+ rcar_mipi_dsi_set(dsi, PPICLSCR, PPICLSCR_TOHS);
+
+ return 0;
+}
+
+static int rcar_mipi_dsi_start_video(struct rcar_mipi_dsi *dsi)
+{
+ u32 status;
+ int ret;
+
+ /* Wait for the link to be ready. */
+ ret = read_poll_timeout(rcar_mipi_dsi_read, status,
+ !(status & (LINKSR_LPBUSY | LINKSR_HSBUSY)),
+ 2000, 10000, false, dsi, LINKSR);
+ if (ret < 0) {
+ dev_err(dsi->dev, "Link failed to become ready\n");
+ return ret;
+ }
+
+ /* De-assert video FIFO clear. */
+ rcar_mipi_dsi_clr(dsi, TXVMCR, TXVMCR_VFCLR);
+
+ ret = read_poll_timeout(rcar_mipi_dsi_read, status,
+ status & TXVMSR_VFRDY,
+ 2000, 10000, false, dsi, TXVMSR);
+ if (ret < 0) {
+ dev_err(dsi->dev, "Failed to de-assert video FIFO clear\n");
+ return ret;
+ }
+
+ /* Enable transmission in video mode. */
+ rcar_mipi_dsi_set(dsi, TXVMCR, TXVMCR_EN_VIDEO);
+
+ ret = read_poll_timeout(rcar_mipi_dsi_read, status,
+ status & TXVMSR_RDY,
+ 2000, 10000, false, dsi, TXVMSR);
+ if (ret < 0) {
+ dev_err(dsi->dev, "Failed to enable video transmission\n");
+ return ret;
+ }
+
+ return 0;
+}
+
+/* -----------------------------------------------------------------------------
+ * Bridge
+ */
+
+static int rcar_mipi_dsi_attach(struct drm_bridge *bridge,
+ enum drm_bridge_attach_flags flags)
+{
+ struct rcar_mipi_dsi *dsi = bridge_to_rcar_mipi_dsi(bridge);
+
+ return drm_bridge_attach(bridge->encoder, dsi->next_bridge, bridge,
+ flags);
+}
+
+static void rcar_mipi_dsi_atomic_enable(struct drm_bridge *bridge,
+ struct drm_bridge_state *old_bridge_state)
+{
+ struct drm_atomic_state *state = old_bridge_state->base.state;
+ struct rcar_mipi_dsi *dsi = bridge_to_rcar_mipi_dsi(bridge);
+ const struct drm_display_mode *mode;
+ struct drm_connector *connector;
+ struct drm_crtc *crtc;
+ int ret;
+
+ connector = drm_atomic_get_new_connector_for_encoder(state,
+ bridge->encoder);
+ crtc = drm_atomic_get_new_connector_state(state, connector)->crtc;
+ mode = &drm_atomic_get_new_crtc_state(state, crtc)->adjusted_mode;
+
+ ret = rcar_mipi_dsi_clk_enable(dsi);
+ if (ret < 0) {
+ dev_err(dsi->dev, "failed to enable DSI clocks\n");
+ return;
+ }
+
+ ret = rcar_mipi_dsi_startup(dsi, mode);
+ if (ret < 0)
+ goto err_dsi_startup;
+
+ rcar_mipi_dsi_set_display_timing(dsi, mode);
+
+ ret = rcar_mipi_dsi_start_hs_clock(dsi);
+ if (ret < 0)
+ goto err_dsi_start_hs;
+
+ rcar_mipi_dsi_start_video(dsi);
+
+ return;
+
+err_dsi_start_hs:
+ rcar_mipi_dsi_shutdown(dsi);
+err_dsi_startup:
+ rcar_mipi_dsi_clk_disable(dsi);
+}
+
+static void rcar_mipi_dsi_atomic_disable(struct drm_bridge *bridge,
+ struct drm_bridge_state *old_bridge_state)
+{
+ struct rcar_mipi_dsi *dsi = bridge_to_rcar_mipi_dsi(bridge);
+
+ rcar_mipi_dsi_shutdown(dsi);
+ rcar_mipi_dsi_clk_disable(dsi);
+}
+
+static enum drm_mode_status
+rcar_mipi_dsi_bridge_mode_valid(struct drm_bridge *bridge,
+ const struct drm_display_info *info,
+ const struct drm_display_mode *mode)
+{
+ if (mode->clock > 297000)
+ return MODE_CLOCK_HIGH;
+
+ return MODE_OK;
+}
+
+static const struct drm_bridge_funcs rcar_mipi_dsi_bridge_ops = {
+ .attach = rcar_mipi_dsi_attach,
+ .atomic_duplicate_state = drm_atomic_helper_bridge_duplicate_state,
+ .atomic_destroy_state = drm_atomic_helper_bridge_destroy_state,
+ .atomic_reset = drm_atomic_helper_bridge_reset,
+ .atomic_enable = rcar_mipi_dsi_atomic_enable,
+ .atomic_disable = rcar_mipi_dsi_atomic_disable,
+ .mode_valid = rcar_mipi_dsi_bridge_mode_valid,
+};
+
+/* -----------------------------------------------------------------------------
+ * Host setting
+ */
+
+static int rcar_mipi_dsi_host_attach(struct mipi_dsi_host *host,
+ struct mipi_dsi_device *device)
+{
+ struct rcar_mipi_dsi *dsi = host_to_rcar_mipi_dsi(host);
+ int ret;
+
+ if (device->lanes > dsi->num_data_lanes)
+ return -EINVAL;
+
+ dsi->lanes = device->lanes;
+ dsi->format = device->format;
+
+ dsi->next_bridge = devm_drm_of_get_bridge(dsi->dev, dsi->dev->of_node,
+ 1, 0);
+ if (IS_ERR(dsi->next_bridge)) {
+ ret = PTR_ERR(dsi->next_bridge);
+ dev_err(dsi->dev, "failed to get next bridge: %d\n", ret);
+ return ret;
+ }
+
+ /* Initialize the DRM bridge. */
+ dsi->bridge.funcs = &rcar_mipi_dsi_bridge_ops;
+ dsi->bridge.of_node = dsi->dev->of_node;
+ drm_bridge_add(&dsi->bridge);
+
+ return 0;
+}
+
+static int rcar_mipi_dsi_host_detach(struct mipi_dsi_host *host,
+ struct mipi_dsi_device *device)
+{
+ struct rcar_mipi_dsi *dsi = host_to_rcar_mipi_dsi(host);
+
+ drm_bridge_remove(&dsi->bridge);
+
+ return 0;
+}
+
+static const struct mipi_dsi_host_ops rcar_mipi_dsi_host_ops = {
+ .attach = rcar_mipi_dsi_host_attach,
+ .detach = rcar_mipi_dsi_host_detach,
+};
+
+/* -----------------------------------------------------------------------------
+ * Probe & Remove
+ */
+
+static int rcar_mipi_dsi_parse_dt(struct rcar_mipi_dsi *dsi)
+{
+ struct device_node *ep;
+ u32 data_lanes[4];
+ int ret;
+
+ ep = of_graph_get_endpoint_by_regs(dsi->dev->of_node, 1, 0);
+ if (!ep) {
+ dev_dbg(dsi->dev, "unconnected port@1\n");
+ return -ENODEV;
+ }
+
+ ret = of_property_read_variable_u32_array(ep, "data-lanes", data_lanes,
+ 1, 4);
+ of_node_put(ep);
+
+ if (ret < 0) {
+ dev_err(dsi->dev, "missing or invalid data-lanes property\n");
+ return -ENODEV;
+ }
+
+ dsi->num_data_lanes = ret;
+ return 0;
+}
+
+static struct clk *rcar_mipi_dsi_get_clock(struct rcar_mipi_dsi *dsi,
+ const char *name,
+ bool optional)
+{
+ struct clk *clk;
+
+ clk = devm_clk_get(dsi->dev, name);
+ if (!IS_ERR(clk))
+ return clk;
+
+ if (PTR_ERR(clk) == -ENOENT && optional)
+ return NULL;
+
+ dev_err_probe(dsi->dev, PTR_ERR(clk), "failed to get %s clock\n",
+ name ? name : "module");
+
+ return clk;
+}
+
+static int rcar_mipi_dsi_get_clocks(struct rcar_mipi_dsi *dsi)
+{
+ dsi->clocks.mod = rcar_mipi_dsi_get_clock(dsi, NULL, false);
+ if (IS_ERR(dsi->clocks.mod))
+ return PTR_ERR(dsi->clocks.mod);
+
+ dsi->clocks.pll = rcar_mipi_dsi_get_clock(dsi, "pll", true);
+ if (IS_ERR(dsi->clocks.pll))
+ return PTR_ERR(dsi->clocks.pll);
+
+ dsi->clocks.dsi = rcar_mipi_dsi_get_clock(dsi, "dsi", true);
+ if (IS_ERR(dsi->clocks.dsi))
+ return PTR_ERR(dsi->clocks.dsi);
+
+ if (!dsi->clocks.pll && !dsi->clocks.dsi) {
+ dev_err(dsi->dev, "no input clock (pll, dsi)\n");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int rcar_mipi_dsi_probe(struct platform_device *pdev)
+{
+ struct rcar_mipi_dsi *dsi;
+ struct resource *mem;
+ int ret;
+
+ dsi = devm_kzalloc(&pdev->dev, sizeof(*dsi), GFP_KERNEL);
+ if (dsi == NULL)
+ return -ENOMEM;
+
+ platform_set_drvdata(pdev, dsi);
+
+ dsi->dev = &pdev->dev;
+ dsi->info = of_device_get_match_data(&pdev->dev);
+
+ ret = rcar_mipi_dsi_parse_dt(dsi);
+ if (ret < 0)
+ return ret;
+
+ /* Acquire resources. */
+ mem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ dsi->mmio = devm_ioremap_resource(dsi->dev, mem);
+ if (IS_ERR(dsi->mmio))
+ return PTR_ERR(dsi->mmio);
+
+ ret = rcar_mipi_dsi_get_clocks(dsi);
+ if (ret < 0)
+ return ret;
+
+ dsi->rstc = devm_reset_control_get(dsi->dev, NULL);
+ if (IS_ERR(dsi->rstc)) {
+ dev_err(dsi->dev, "failed to get cpg reset\n");
+ return PTR_ERR(dsi->rstc);
+ }
+
+ /* Initialize the DSI host. */
+ dsi->host.dev = dsi->dev;
+ dsi->host.ops = &rcar_mipi_dsi_host_ops;
+ ret = mipi_dsi_host_register(&dsi->host);
+ if (ret < 0)
+ return ret;
+
+ return 0;
+}
+
+static int rcar_mipi_dsi_remove(struct platform_device *pdev)
+{
+ struct rcar_mipi_dsi *dsi = platform_get_drvdata(pdev);
+
+ mipi_dsi_host_unregister(&dsi->host);
+
+ return 0;
+}
+
+static const struct of_device_id rcar_mipi_dsi_of_table[] = {
+ { .compatible = "renesas,r8a779a0-dsi-csi2-tx" },
+ { }
+};
+
+MODULE_DEVICE_TABLE(of, rcar_mipi_dsi_of_table);
+
+static struct platform_driver rcar_mipi_dsi_platform_driver = {
+ .probe = rcar_mipi_dsi_probe,
+ .remove = rcar_mipi_dsi_remove,
+ .driver = {
+ .name = "rcar-mipi-dsi",
+ .of_match_table = rcar_mipi_dsi_of_table,
+ },
+};
+
+module_platform_driver(rcar_mipi_dsi_platform_driver);
+
+MODULE_DESCRIPTION("Renesas R-Car MIPI DSI Encoder Driver");
+MODULE_LICENSE("GPL");
diff --git a/drivers/gpu/drm/rcar-du/rcar_mipi_dsi_regs.h b/drivers/gpu/drm/rcar-du/rcar_mipi_dsi_regs.h
new file mode 100644
index 0000000..0e7a927
--- /dev/null
+++ b/drivers/gpu/drm/rcar-du/rcar_mipi_dsi_regs.h
@@ -0,0 +1,172 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * rcar_mipi_dsi_regs.h -- R-Car MIPI DSI Interface Registers Definitions
+ *
+ * Copyright (C) 2020 Renesas Electronics Corporation
+ */
+
+#ifndef __RCAR_MIPI_DSI_REGS_H__
+#define __RCAR_MIPI_DSI_REGS_H__
+
+#define LINKSR 0x010
+#define LINKSR_LPBUSY (1 << 1)
+#define LINKSR_HSBUSY (1 << 0)
+
+/*
+ * Video Mode Register
+ */
+#define TXVMSETR 0x180
+#define TXVMSETR_SYNSEQ_PULSES (0 << 16)
+#define TXVMSETR_SYNSEQ_EVENTS (1 << 16)
+#define TXVMSETR_VSTPM (1 << 15)
+#define TXVMSETR_PIXWDTH (1 << 8)
+#define TXVMSETR_VSEN_EN (1 << 4)
+#define TXVMSETR_VSEN_DIS (0 << 4)
+#define TXVMSETR_HFPBPEN_EN (1 << 2)
+#define TXVMSETR_HFPBPEN_DIS (0 << 2)
+#define TXVMSETR_HBPBPEN_EN (1 << 1)
+#define TXVMSETR_HBPBPEN_DIS (0 << 1)
+#define TXVMSETR_HSABPEN_EN (1 << 0)
+#define TXVMSETR_HSABPEN_DIS (0 << 0)
+
+#define TXVMCR 0x190
+#define TXVMCR_VFCLR (1 << 12)
+#define TXVMCR_EN_VIDEO (1 << 0)
+
+#define TXVMSR 0x1a0
+#define TXVMSR_STR (1 << 16)
+#define TXVMSR_VFRDY (1 << 12)
+#define TXVMSR_ACT (1 << 8)
+#define TXVMSR_RDY (1 << 0)
+
+#define TXVMSCR 0x1a4
+#define TXVMSCR_STR (1 << 16)
+
+#define TXVMPSPHSETR 0x1c0
+#define TXVMPSPHSETR_DT_RGB16 (0x0e << 16)
+#define TXVMPSPHSETR_DT_RGB18 (0x1e << 16)
+#define TXVMPSPHSETR_DT_RGB18_LS (0x2e << 16)
+#define TXVMPSPHSETR_DT_RGB24 (0x3e << 16)
+#define TXVMPSPHSETR_DT_YCBCR16 (0x2c << 16)
+
+#define TXVMVPRMSET0R 0x1d0
+#define TXVMVPRMSET0R_HSPOL_HIG (0 << 17)
+#define TXVMVPRMSET0R_HSPOL_LOW (1 << 17)
+#define TXVMVPRMSET0R_VSPOL_HIG (0 << 16)
+#define TXVMVPRMSET0R_VSPOL_LOW (1 << 16)
+#define TXVMVPRMSET0R_CSPC_RGB (0 << 4)
+#define TXVMVPRMSET0R_CSPC_YCbCr (1 << 4)
+#define TXVMVPRMSET0R_BPP_16 (0 << 0)
+#define TXVMVPRMSET0R_BPP_18 (1 << 0)
+#define TXVMVPRMSET0R_BPP_24 (2 << 0)
+
+#define TXVMVPRMSET1R 0x1d4
+#define TXVMVPRMSET1R_VACTIVE(x) (((x) & 0x7fff) << 16)
+#define TXVMVPRMSET1R_VSA(x) (((x) & 0xfff) << 0)
+
+#define TXVMVPRMSET2R 0x1d8
+#define TXVMVPRMSET2R_VFP(x) (((x) & 0x1fff) << 16)
+#define TXVMVPRMSET2R_VBP(x) (((x) & 0x1fff) << 0)
+
+#define TXVMVPRMSET3R 0x1dc
+#define TXVMVPRMSET3R_HACTIVE(x) (((x) & 0x7fff) << 16)
+#define TXVMVPRMSET3R_HSA(x) (((x) & 0xfff) << 0)
+
+#define TXVMVPRMSET4R 0x1e0
+#define TXVMVPRMSET4R_HFP(x) (((x) & 0x1fff) << 16)
+#define TXVMVPRMSET4R_HBP(x) (((x) & 0x1fff) << 0)
+
+/*
+ * PHY-Protocol Interface (PPI) Registers
+ */
+#define PPISETR 0x700
+#define PPISETR_DLEN_0 (0x1 << 0)
+#define PPISETR_DLEN_1 (0x3 << 0)
+#define PPISETR_DLEN_2 (0x7 << 0)
+#define PPISETR_DLEN_3 (0xf << 0)
+#define PPISETR_CLEN (1 << 8)
+
+#define PPICLCR 0x710
+#define PPICLCR_TXREQHS (1 << 8)
+#define PPICLCR_TXULPSEXT (1 << 1)
+#define PPICLCR_TXULPSCLK (1 << 0)
+
+#define PPICLSR 0x720
+#define PPICLSR_HSTOLP (1 << 27)
+#define PPICLSR_TOHS (1 << 26)
+#define PPICLSR_STPST (1 << 0)
+
+#define PPICLSCR 0x724
+#define PPICLSCR_HSTOLP (1 << 27)
+#define PPICLSCR_TOHS (1 << 26)
+
+#define PPIDLSR 0x760
+#define PPIDLSR_STPST (0xf << 0)
+
+/*
+ * Clocks registers
+ */
+#define LPCLKSET 0x1000
+#define LPCLKSET_CKEN (1 << 8)
+#define LPCLKSET_LPCLKDIV(x) (((x) & 0x3f) << 0)
+
+#define CFGCLKSET 0x1004
+#define CFGCLKSET_CKEN (1 << 8)
+#define CFGCLKSET_CFGCLKDIV(x) (((x) & 0x3f) << 0)
+
+#define DOTCLKDIV 0x1008
+#define DOTCLKDIV_CKEN (1 << 8)
+#define DOTCLKDIV_DOTCLKDIV(x) (((x) & 0x3f) << 0)
+
+#define VCLKSET 0x100c
+#define VCLKSET_CKEN (1 << 16)
+#define VCLKSET_COLOR_RGB (0 << 8)
+#define VCLKSET_COLOR_YCC (1 << 8)
+#define VCLKSET_DIV(x) (((x) & 0x3) << 4)
+#define VCLKSET_BPP_16 (0 << 2)
+#define VCLKSET_BPP_18 (1 << 2)
+#define VCLKSET_BPP_18L (2 << 2)
+#define VCLKSET_BPP_24 (3 << 2)
+#define VCLKSET_LANE(x) (((x) & 0x3) << 0)
+
+#define VCLKEN 0x1010
+#define VCLKEN_CKEN (1 << 0)
+
+#define PHYSETUP 0x1014
+#define PHYSETUP_HSFREQRANGE(x) (((x) & 0x7f) << 16)
+#define PHYSETUP_HSFREQRANGE_MASK (0x7f << 16)
+#define PHYSETUP_CFGCLKFREQRANGE(x) (((x) & 0x3f) << 8)
+#define PHYSETUP_SHUTDOWNZ (1 << 1)
+#define PHYSETUP_RSTZ (1 << 0)
+
+#define CLOCKSET1 0x101c
+#define CLOCKSET1_LOCK_PHY (1 << 17)
+#define CLOCKSET1_LOCK (1 << 16)
+#define CLOCKSET1_CLKSEL (1 << 8)
+#define CLOCKSET1_CLKINSEL_EXTAL (0 << 2)
+#define CLOCKSET1_CLKINSEL_DIG (1 << 2)
+#define CLOCKSET1_CLKINSEL_DU (1 << 3)
+#define CLOCKSET1_SHADOW_CLEAR (1 << 1)
+#define CLOCKSET1_UPDATEPLL (1 << 0)
+
+#define CLOCKSET2 0x1020
+#define CLOCKSET2_M(x) (((x) & 0xfff) << 16)
+#define CLOCKSET2_VCO_CNTRL(x) (((x) & 0x3f) << 8)
+#define CLOCKSET2_N(x) (((x) & 0xf) << 0)
+
+#define CLOCKSET3 0x1024
+#define CLOCKSET3_PROP_CNTRL(x) (((x) & 0x3f) << 24)
+#define CLOCKSET3_INT_CNTRL(x) (((x) & 0x3f) << 16)
+#define CLOCKSET3_CPBIAS_CNTRL(x) (((x) & 0x7f) << 8)
+#define CLOCKSET3_GMP_CNTRL(x) (((x) & 0x3) << 0)
+
+#define PHTW 0x1034
+#define PHTW_DWEN (1 << 24)
+#define PHTW_TESTDIN_DATA(x) (((x) & 0xff) << 16)
+#define PHTW_CWEN (1 << 8)
+#define PHTW_TESTDIN_CODE(x) (((x) & 0xff) << 0)
+
+#define PHTC 0x103c
+#define PHTC_TESTCLR (1 << 0)
+
+#endif /* __RCAR_MIPI_DSI_REGS_H__ */
diff --git a/drivers/gpu/drm/rockchip/Makefile b/drivers/gpu/drm/rockchip/Makefile
index 17a9e7e..1a56f69 100644
--- a/drivers/gpu/drm/rockchip/Makefile
+++ b/drivers/gpu/drm/rockchip/Makefile
@@ -5,7 +5,6 @@
rockchipdrm-y := rockchip_drm_drv.o rockchip_drm_fb.o \
rockchip_drm_gem.o rockchip_drm_vop.o rockchip_vop_reg.o
-rockchipdrm-$(CONFIG_DRM_FBDEV_EMULATION) += rockchip_drm_fbdev.o
rockchipdrm-$(CONFIG_ROCKCHIP_ANALOGIX_DP) += analogix_dp-rockchip.o
rockchipdrm-$(CONFIG_ROCKCHIP_CDN_DP) += cdn-dp-core.o cdn-dp-reg.o
diff --git a/drivers/gpu/drm/rockchip/dw-mipi-dsi-rockchip.c b/drivers/gpu/drm/rockchip/dw-mipi-dsi-rockchip.c
index a9acbcc..4ed7a68 100644
--- a/drivers/gpu/drm/rockchip/dw-mipi-dsi-rockchip.c
+++ b/drivers/gpu/drm/rockchip/dw-mipi-dsi-rockchip.c
@@ -267,6 +267,8 @@ struct dw_mipi_dsi_rockchip {
struct dw_mipi_dsi *dmd;
const struct rockchip_dw_dsi_chip_data *cdata;
struct dw_mipi_dsi_plat_data pdata;
+
+ bool dsi_bound;
};
struct dphy_pll_parameter_map {
@@ -772,10 +774,6 @@ static void dw_mipi_dsi_encoder_enable(struct drm_encoder *encoder)
if (mux < 0)
return;
- pm_runtime_get_sync(dsi->dev);
- if (dsi->slave)
- pm_runtime_get_sync(dsi->slave->dev);
-
/*
* For the RK3399, the clk of grf must be enabled before writing grf
* register. And for RK3288 or other soc, this grf_clk must be NULL,
@@ -794,20 +792,10 @@ static void dw_mipi_dsi_encoder_enable(struct drm_encoder *encoder)
clk_disable_unprepare(dsi->grf_clk);
}
-static void dw_mipi_dsi_encoder_disable(struct drm_encoder *encoder)
-{
- struct dw_mipi_dsi_rockchip *dsi = to_dsi(encoder);
-
- if (dsi->slave)
- pm_runtime_put(dsi->slave->dev);
- pm_runtime_put(dsi->dev);
-}
-
static const struct drm_encoder_helper_funcs
dw_mipi_dsi_encoder_helper_funcs = {
.atomic_check = dw_mipi_dsi_encoder_atomic_check,
.enable = dw_mipi_dsi_encoder_enable,
- .disable = dw_mipi_dsi_encoder_disable,
};
static int rockchip_dsi_drm_create_encoder(struct dw_mipi_dsi_rockchip *dsi,
@@ -937,10 +925,14 @@ static int dw_mipi_dsi_rockchip_bind(struct device *dev,
put_device(second);
}
+ pm_runtime_get_sync(dsi->dev);
+ if (dsi->slave)
+ pm_runtime_get_sync(dsi->slave->dev);
+
ret = clk_prepare_enable(dsi->pllref_clk);
if (ret) {
DRM_DEV_ERROR(dev, "Failed to enable pllref_clk: %d\n", ret);
- return ret;
+ goto out_pm_runtime;
}
/*
@@ -952,7 +944,7 @@ static int dw_mipi_dsi_rockchip_bind(struct device *dev,
ret = clk_prepare_enable(dsi->grf_clk);
if (ret) {
DRM_DEV_ERROR(dsi->dev, "Failed to enable grf_clk: %d\n", ret);
- return ret;
+ goto out_pll_clk;
}
dw_mipi_dsi_rockchip_config(dsi);
@@ -964,16 +956,27 @@ static int dw_mipi_dsi_rockchip_bind(struct device *dev,
ret = rockchip_dsi_drm_create_encoder(dsi, drm_dev);
if (ret) {
DRM_DEV_ERROR(dev, "Failed to create drm encoder\n");
- return ret;
+ goto out_pll_clk;
}
ret = dw_mipi_dsi_bind(dsi->dmd, &dsi->encoder);
if (ret) {
DRM_DEV_ERROR(dev, "Failed to bind: %d\n", ret);
- return ret;
+ goto out_pll_clk;
}
+ dsi->dsi_bound = true;
+
return 0;
+
+out_pll_clk:
+ clk_disable_unprepare(dsi->pllref_clk);
+out_pm_runtime:
+ pm_runtime_put(dsi->dev);
+ if (dsi->slave)
+ pm_runtime_put(dsi->slave->dev);
+
+ return ret;
}
static void dw_mipi_dsi_rockchip_unbind(struct device *dev,
@@ -985,9 +988,15 @@ static void dw_mipi_dsi_rockchip_unbind(struct device *dev,
if (dsi->is_slave)
return;
+ dsi->dsi_bound = false;
+
dw_mipi_dsi_unbind(dsi->dmd);
clk_disable_unprepare(dsi->pllref_clk);
+
+ pm_runtime_put(dsi->dev);
+ if (dsi->slave)
+ pm_runtime_put(dsi->slave->dev);
}
static const struct component_ops dw_mipi_dsi_rockchip_ops = {
@@ -1275,6 +1284,36 @@ static const struct phy_ops dw_mipi_dsi_dphy_ops = {
.exit = dw_mipi_dsi_dphy_exit,
};
+static int __maybe_unused dw_mipi_dsi_rockchip_resume(struct device *dev)
+{
+ struct dw_mipi_dsi_rockchip *dsi = dev_get_drvdata(dev);
+ int ret;
+
+ /*
+ * Re-configure DSI state, if we were previously initialized. We need
+ * to do this before rockchip_drm_drv tries to re-enable() any panels.
+ */
+ if (dsi->dsi_bound) {
+ ret = clk_prepare_enable(dsi->grf_clk);
+ if (ret) {
+ DRM_DEV_ERROR(dsi->dev, "Failed to enable grf_clk: %d\n", ret);
+ return ret;
+ }
+
+ dw_mipi_dsi_rockchip_config(dsi);
+ if (dsi->slave)
+ dw_mipi_dsi_rockchip_config(dsi->slave);
+
+ clk_disable_unprepare(dsi->grf_clk);
+ }
+
+ return 0;
+}
+
+static const struct dev_pm_ops dw_mipi_dsi_rockchip_pm_ops = {
+ SET_LATE_SYSTEM_SLEEP_PM_OPS(NULL, dw_mipi_dsi_rockchip_resume)
+};
+
static int dw_mipi_dsi_rockchip_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
@@ -1396,14 +1435,10 @@ static int dw_mipi_dsi_rockchip_probe(struct platform_device *pdev)
if (ret != -EPROBE_DEFER)
DRM_DEV_ERROR(dev,
"Failed to probe dw_mipi_dsi: %d\n", ret);
- goto err_clkdisable;
+ return ret;
}
return 0;
-
-err_clkdisable:
- clk_disable_unprepare(dsi->pllref_clk);
- return ret;
}
static int dw_mipi_dsi_rockchip_remove(struct platform_device *pdev)
@@ -1592,6 +1627,7 @@ struct platform_driver dw_mipi_dsi_rockchip_driver = {
.remove = dw_mipi_dsi_rockchip_remove,
.driver = {
.of_match_table = dw_mipi_dsi_rockchip_dt_ids,
+ .pm = &dw_mipi_dsi_rockchip_pm_ops,
.name = "dw-mipi-dsi-rockchip",
},
};
diff --git a/drivers/gpu/drm/rockchip/rockchip_drm_drv.c b/drivers/gpu/drm/rockchip/rockchip_drm_drv.c
index e4ebe60..bec207d 100644
--- a/drivers/gpu/drm/rockchip/rockchip_drm_drv.c
+++ b/drivers/gpu/drm/rockchip/rockchip_drm_drv.c
@@ -26,7 +26,6 @@
#include "rockchip_drm_drv.h"
#include "rockchip_drm_fb.h"
-#include "rockchip_drm_fbdev.h"
#include "rockchip_drm_gem.h"
#define DRIVER_NAME "rockchip"
@@ -159,10 +158,6 @@ static int rockchip_drm_bind(struct device *dev)
drm_mode_config_reset(drm_dev);
- ret = rockchip_drm_fbdev_init(drm_dev);
- if (ret)
- goto err_unbind_all;
-
/* init kms poll for handling hpd */
drm_kms_helper_poll_init(drm_dev);
@@ -170,10 +165,11 @@ static int rockchip_drm_bind(struct device *dev)
if (ret)
goto err_kms_helper_poll_fini;
+ drm_fbdev_generic_setup(drm_dev, 0);
+
return 0;
err_kms_helper_poll_fini:
drm_kms_helper_poll_fini(drm_dev);
- rockchip_drm_fbdev_fini(drm_dev);
err_unbind_all:
component_unbind_all(dev, drm_dev);
err_iommu_cleanup:
@@ -189,7 +185,6 @@ static void rockchip_drm_unbind(struct device *dev)
drm_dev_unregister(drm_dev);
- rockchip_drm_fbdev_fini(drm_dev);
drm_kms_helper_poll_fini(drm_dev);
drm_atomic_helper_shutdown(drm_dev);
@@ -199,25 +194,15 @@ static void rockchip_drm_unbind(struct device *dev)
drm_dev_put(drm_dev);
}
-static const struct file_operations rockchip_drm_driver_fops = {
- .owner = THIS_MODULE,
- .open = drm_open,
- .mmap = rockchip_gem_mmap,
- .poll = drm_poll,
- .read = drm_read,
- .unlocked_ioctl = drm_ioctl,
- .compat_ioctl = drm_compat_ioctl,
- .release = drm_release,
-};
+DEFINE_DRM_GEM_FOPS(rockchip_drm_driver_fops);
static const struct drm_driver rockchip_drm_driver = {
.driver_features = DRIVER_MODESET | DRIVER_GEM | DRIVER_ATOMIC,
- .lastclose = drm_fb_helper_lastclose,
.dumb_create = rockchip_gem_dumb_create,
.prime_handle_to_fd = drm_gem_prime_handle_to_fd,
.prime_fd_to_handle = drm_gem_prime_fd_to_handle,
.gem_prime_import_sg_table = rockchip_gem_prime_import_sg_table,
- .gem_prime_mmap = rockchip_gem_mmap_buf,
+ .gem_prime_mmap = drm_gem_prime_mmap,
.fops = &rockchip_drm_driver_fops,
.name = DRIVER_NAME,
.desc = DRIVER_DESC,
diff --git a/drivers/gpu/drm/rockchip/rockchip_drm_drv.h b/drivers/gpu/drm/rockchip/rockchip_drm_drv.h
index aa0909e..143a483 100644
--- a/drivers/gpu/drm/rockchip/rockchip_drm_drv.h
+++ b/drivers/gpu/drm/rockchip/rockchip_drm_drv.h
@@ -43,8 +43,6 @@ struct rockchip_crtc_state {
* @mm_lock: protect drm_mm on multi-threads.
*/
struct rockchip_drm_private {
- struct drm_fb_helper fbdev_helper;
- struct drm_gem_object *fbdev_bo;
struct iommu_domain *domain;
struct mutex mm_lock;
struct drm_mm mm;
diff --git a/drivers/gpu/drm/rockchip/rockchip_drm_fbdev.c b/drivers/gpu/drm/rockchip/rockchip_drm_fbdev.c
deleted file mode 100644
index 2fdc455..0000000
--- a/drivers/gpu/drm/rockchip/rockchip_drm_fbdev.c
+++ /dev/null
@@ -1,163 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-only
-/*
- * Copyright (C) Fuzhou Rockchip Electronics Co.Ltd
- * Author:Mark Yao <mark.yao@rock-chips.com>
- */
-
-#include <drm/drm.h>
-#include <drm/drm_fb_helper.h>
-#include <drm/drm_fourcc.h>
-#include <drm/drm_probe_helper.h>
-
-#include "rockchip_drm_drv.h"
-#include "rockchip_drm_gem.h"
-#include "rockchip_drm_fb.h"
-#include "rockchip_drm_fbdev.h"
-
-#define PREFERRED_BPP 32
-#define to_drm_private(x) \
- container_of(x, struct rockchip_drm_private, fbdev_helper)
-
-static int rockchip_fbdev_mmap(struct fb_info *info,
- struct vm_area_struct *vma)
-{
- struct drm_fb_helper *helper = info->par;
- struct rockchip_drm_private *private = to_drm_private(helper);
-
- return rockchip_gem_mmap_buf(private->fbdev_bo, vma);
-}
-
-static const struct fb_ops rockchip_drm_fbdev_ops = {
- .owner = THIS_MODULE,
- DRM_FB_HELPER_DEFAULT_OPS,
- .fb_mmap = rockchip_fbdev_mmap,
- .fb_fillrect = drm_fb_helper_cfb_fillrect,
- .fb_copyarea = drm_fb_helper_cfb_copyarea,
- .fb_imageblit = drm_fb_helper_cfb_imageblit,
-};
-
-static int rockchip_drm_fbdev_create(struct drm_fb_helper *helper,
- struct drm_fb_helper_surface_size *sizes)
-{
- struct rockchip_drm_private *private = to_drm_private(helper);
- struct drm_mode_fb_cmd2 mode_cmd = { 0 };
- struct drm_device *dev = helper->dev;
- struct rockchip_gem_object *rk_obj;
- struct drm_framebuffer *fb;
- unsigned int bytes_per_pixel;
- unsigned long offset;
- struct fb_info *fbi;
- size_t size;
- int ret;
-
- bytes_per_pixel = DIV_ROUND_UP(sizes->surface_bpp, 8);
-
- mode_cmd.width = sizes->surface_width;
- mode_cmd.height = sizes->surface_height;
- mode_cmd.pitches[0] = sizes->surface_width * bytes_per_pixel;
- mode_cmd.pixel_format = drm_mode_legacy_fb_format(sizes->surface_bpp,
- sizes->surface_depth);
-
- size = mode_cmd.pitches[0] * mode_cmd.height;
-
- rk_obj = rockchip_gem_create_object(dev, size, true);
- if (IS_ERR(rk_obj))
- return -ENOMEM;
-
- private->fbdev_bo = &rk_obj->base;
-
- fbi = drm_fb_helper_alloc_fbi(helper);
- if (IS_ERR(fbi)) {
- DRM_DEV_ERROR(dev->dev, "Failed to create framebuffer info.\n");
- ret = PTR_ERR(fbi);
- goto out;
- }
-
- helper->fb = rockchip_drm_framebuffer_init(dev, &mode_cmd,
- private->fbdev_bo);
- if (IS_ERR(helper->fb)) {
- DRM_DEV_ERROR(dev->dev,
- "Failed to allocate DRM framebuffer.\n");
- ret = PTR_ERR(helper->fb);
- goto out;
- }
-
- fbi->fbops = &rockchip_drm_fbdev_ops;
-
- fb = helper->fb;
- drm_fb_helper_fill_info(fbi, helper, sizes);
-
- offset = fbi->var.xoffset * bytes_per_pixel;
- offset += fbi->var.yoffset * fb->pitches[0];
-
- dev->mode_config.fb_base = 0;
- fbi->screen_base = rk_obj->kvaddr + offset;
- fbi->screen_size = rk_obj->base.size;
- fbi->fix.smem_len = rk_obj->base.size;
-
- DRM_DEBUG_KMS("FB [%dx%d]-%d kvaddr=%p offset=%ld size=%zu\n",
- fb->width, fb->height, fb->format->depth,
- rk_obj->kvaddr,
- offset, size);
-
- return 0;
-
-out:
- rockchip_gem_free_object(&rk_obj->base);
- return ret;
-}
-
-static const struct drm_fb_helper_funcs rockchip_drm_fb_helper_funcs = {
- .fb_probe = rockchip_drm_fbdev_create,
-};
-
-int rockchip_drm_fbdev_init(struct drm_device *dev)
-{
- struct rockchip_drm_private *private = dev->dev_private;
- struct drm_fb_helper *helper;
- int ret;
-
- if (!dev->mode_config.num_crtc || !dev->mode_config.num_connector)
- return -EINVAL;
-
- helper = &private->fbdev_helper;
-
- drm_fb_helper_prepare(dev, helper, &rockchip_drm_fb_helper_funcs);
-
- ret = drm_fb_helper_init(dev, helper);
- if (ret < 0) {
- DRM_DEV_ERROR(dev->dev,
- "Failed to initialize drm fb helper - %d.\n",
- ret);
- return ret;
- }
-
- ret = drm_fb_helper_initial_config(helper, PREFERRED_BPP);
- if (ret < 0) {
- DRM_DEV_ERROR(dev->dev,
- "Failed to set initial hw config - %d.\n",
- ret);
- goto err_drm_fb_helper_fini;
- }
-
- return 0;
-
-err_drm_fb_helper_fini:
- drm_fb_helper_fini(helper);
- return ret;
-}
-
-void rockchip_drm_fbdev_fini(struct drm_device *dev)
-{
- struct rockchip_drm_private *private = dev->dev_private;
- struct drm_fb_helper *helper;
-
- helper = &private->fbdev_helper;
-
- drm_fb_helper_unregister_fbi(helper);
-
- if (helper->fb)
- drm_framebuffer_put(helper->fb);
-
- drm_fb_helper_fini(helper);
-}
diff --git a/drivers/gpu/drm/rockchip/rockchip_drm_fbdev.h b/drivers/gpu/drm/rockchip/rockchip_drm_fbdev.h
deleted file mode 100644
index 5fb7ac2..0000000
--- a/drivers/gpu/drm/rockchip/rockchip_drm_fbdev.h
+++ /dev/null
@@ -1,24 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0-only */
-/*
- * Copyright (C) Fuzhou Rockchip Electronics Co.Ltd
- * Author:Mark Yao <mark.yao@rock-chips.com>
- */
-
-#ifndef _ROCKCHIP_DRM_FBDEV_H
-#define _ROCKCHIP_DRM_FBDEV_H
-
-#ifdef CONFIG_DRM_FBDEV_EMULATION
-int rockchip_drm_fbdev_init(struct drm_device *dev);
-void rockchip_drm_fbdev_fini(struct drm_device *dev);
-#else
-static inline int rockchip_drm_fbdev_init(struct drm_device *dev)
-{
- return 0;
-}
-
-static inline void rockchip_drm_fbdev_fini(struct drm_device *dev)
-{
-}
-#endif
-
-#endif /* _ROCKCHIP_DRM_FBDEV_H */
diff --git a/drivers/gpu/drm/rockchip/rockchip_drm_gem.c b/drivers/gpu/drm/rockchip/rockchip_drm_gem.c
index 7971f57..63eb73b 100644
--- a/drivers/gpu/drm/rockchip/rockchip_drm_gem.c
+++ b/drivers/gpu/drm/rockchip/rockchip_drm_gem.c
@@ -241,11 +241,21 @@ static int rockchip_drm_gem_object_mmap(struct drm_gem_object *obj,
struct rockchip_gem_object *rk_obj = to_rockchip_obj(obj);
/*
+ * Set vm_pgoff (used as a fake buffer offset by DRM) to 0 and map the
+ * whole buffer from the start.
+ */
+ vma->vm_pgoff = 0;
+
+ /*
* We allocated a struct page table for rk_obj, so clear
* VM_PFNMAP flag that was set by drm_gem_mmap_obj()/drm_gem_mmap().
*/
+ vma->vm_flags |= VM_IO | VM_DONTEXPAND | VM_DONTDUMP;
vma->vm_flags &= ~VM_PFNMAP;
+ vma->vm_page_prot = pgprot_writecombine(vm_get_page_prot(vma->vm_flags));
+ vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot);
+
if (rk_obj->pages)
ret = rockchip_drm_gem_object_mmap_iommu(obj, vma);
else
@@ -257,39 +267,6 @@ static int rockchip_drm_gem_object_mmap(struct drm_gem_object *obj,
return ret;
}
-int rockchip_gem_mmap_buf(struct drm_gem_object *obj,
- struct vm_area_struct *vma)
-{
- int ret;
-
- ret = drm_gem_mmap_obj(obj, obj->size, vma);
- if (ret)
- return ret;
-
- return rockchip_drm_gem_object_mmap(obj, vma);
-}
-
-/* drm driver mmap file operations */
-int rockchip_gem_mmap(struct file *filp, struct vm_area_struct *vma)
-{
- struct drm_gem_object *obj;
- int ret;
-
- ret = drm_gem_mmap(filp, vma);
- if (ret)
- return ret;
-
- /*
- * Set vm_pgoff (used as a fake buffer offset by DRM) to 0 and map the
- * whole buffer from the start.
- */
- vma->vm_pgoff = 0;
-
- obj = vma->vm_private_data;
-
- return rockchip_drm_gem_object_mmap(obj, vma);
-}
-
static void rockchip_gem_release_object(struct rockchip_gem_object *rk_obj)
{
drm_gem_object_release(&rk_obj->base);
@@ -301,6 +278,7 @@ static const struct drm_gem_object_funcs rockchip_gem_object_funcs = {
.get_sg_table = rockchip_gem_prime_get_sg_table,
.vmap = rockchip_gem_prime_vmap,
.vunmap = rockchip_gem_prime_vunmap,
+ .mmap = rockchip_drm_gem_object_mmap,
.vm_ops = &drm_gem_cma_vm_ops,
};
diff --git a/drivers/gpu/drm/rockchip/rockchip_drm_gem.h b/drivers/gpu/drm/rockchip/rockchip_drm_gem.h
index 5a70a56..47c1861 100644
--- a/drivers/gpu/drm/rockchip/rockchip_drm_gem.h
+++ b/drivers/gpu/drm/rockchip/rockchip_drm_gem.h
@@ -34,13 +34,6 @@ rockchip_gem_prime_import_sg_table(struct drm_device *dev,
int rockchip_gem_prime_vmap(struct drm_gem_object *obj, struct dma_buf_map *map);
void rockchip_gem_prime_vunmap(struct drm_gem_object *obj, struct dma_buf_map *map);
-/* drm driver mmap file operations */
-int rockchip_gem_mmap(struct file *filp, struct vm_area_struct *vma);
-
-/* mmap a gem object to userspace. */
-int rockchip_gem_mmap_buf(struct drm_gem_object *obj,
- struct vm_area_struct *vma);
-
struct rockchip_gem_object *
rockchip_gem_create_object(struct drm_device *drm, unsigned int size,
bool alloc_kmap);
diff --git a/drivers/gpu/drm/rockchip/rockchip_drm_vop.c b/drivers/gpu/drm/rockchip/rockchip_drm_vop.c
index a25b98b..3e8d9e2 100644
--- a/drivers/gpu/drm/rockchip/rockchip_drm_vop.c
+++ b/drivers/gpu/drm/rockchip/rockchip_drm_vop.c
@@ -726,7 +726,9 @@ static void vop_crtc_atomic_disable(struct drm_crtc *crtc,
spin_unlock(&vop->reg_lock);
- wait_for_completion(&vop->dsp_hold_completion);
+ if (!wait_for_completion_timeout(&vop->dsp_hold_completion,
+ msecs_to_jiffies(200)))
+ WARN(1, "%s: timed out waiting for DSP hold", crtc->name);
vop_dsp_hold_valid_irq_disable(vop);
diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c
index 27e1573..191c560 100644
--- a/drivers/gpu/drm/scheduler/sched_entity.c
+++ b/drivers/gpu/drm/scheduler/sched_entity.c
@@ -190,6 +190,16 @@ long drm_sched_entity_flush(struct drm_sched_entity *entity, long timeout)
}
EXPORT_SYMBOL(drm_sched_entity_flush);
+static void drm_sched_entity_kill_jobs_irq_work(struct irq_work *wrk)
+{
+ struct drm_sched_job *job = container_of(wrk, typeof(*job), work);
+
+ drm_sched_fence_finished(job->s_fence);
+ WARN_ON(job->s_fence->parent);
+ job->sched->ops->free_job(job);
+}
+
+
/* Signal the scheduler finished fence when the entity in question is killed. */
static void drm_sched_entity_kill_jobs_cb(struct dma_fence *f,
struct dma_fence_cb *cb)
@@ -197,9 +207,8 @@ static void drm_sched_entity_kill_jobs_cb(struct dma_fence *f,
struct drm_sched_job *job = container_of(cb, struct drm_sched_job,
finish_cb);
- drm_sched_fence_finished(job->s_fence);
- WARN_ON(job->s_fence->parent);
- job->sched->ops->free_job(job);
+ init_irq_work(&job->work, drm_sched_entity_kill_jobs_irq_work);
+ irq_work_queue(&job->work);
}
static struct dma_fence *
diff --git a/drivers/gpu/drm/shmobile/Kconfig b/drivers/gpu/drm/shmobile/Kconfig
index e2a6c82..288b838 100644
--- a/drivers/gpu/drm/shmobile/Kconfig
+++ b/drivers/gpu/drm/shmobile/Kconfig
@@ -5,7 +5,6 @@
depends on ARCH_SHMOBILE || COMPILE_TEST
select BACKLIGHT_CLASS_DEVICE
select DRM_KMS_HELPER
- select DRM_KMS_CMA_HELPER
select DRM_GEM_CMA_HELPER
help
Choose this option if you have an SH Mobile chipset.
diff --git a/drivers/gpu/drm/sprd/Kconfig b/drivers/gpu/drm/sprd/Kconfig
new file mode 100644
index 0000000..3edeaec
--- /dev/null
+++ b/drivers/gpu/drm/sprd/Kconfig
@@ -0,0 +1,13 @@
+config DRM_SPRD
+ tristate "DRM Support for Unisoc SoCs Platform"
+ depends on ARCH_SPRD || COMPILE_TEST
+ depends on DRM && OF
+ select DRM_GEM_CMA_HELPER
+ select DRM_KMS_CMA_HELPER
+ select DRM_KMS_HELPER
+ select DRM_MIPI_DSI
+ select VIDEOMODE_HELPERS
+ help
+ Choose this option if you have a Unisoc chipset.
+ If M is selected the module will be called sprd_drm.
+
diff --git a/drivers/gpu/drm/sprd/Makefile b/drivers/gpu/drm/sprd/Makefile
new file mode 100644
index 0000000..e82e6a6
--- /dev/null
+++ b/drivers/gpu/drm/sprd/Makefile
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: GPL-2.0
+
+sprd-drm-y := sprd_drm.o \
+ sprd_dpu.o \
+ sprd_dsi.o \
+ megacores_pll.o
+
+obj-$(CONFIG_DRM_SPRD) += sprd-drm.o
\ No newline at end of file
diff --git a/drivers/gpu/drm/sprd/megacores_pll.c b/drivers/gpu/drm/sprd/megacores_pll.c
new file mode 100644
index 0000000..3091dfd
--- /dev/null
+++ b/drivers/gpu/drm/sprd/megacores_pll.c
@@ -0,0 +1,305 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2020 Unisoc Inc.
+ */
+
+#include <asm/div64.h>
+#include <linux/delay.h>
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/regmap.h>
+#include <linux/string.h>
+
+#include "sprd_dsi.h"
+
+#define L 0
+#define H 1
+#define CLK 0
+#define DATA 1
+#define INFINITY 0xffffffff
+#define MIN_OUTPUT_FREQ (100)
+
+#define AVERAGE(a, b) (min(a, b) + abs((b) - (a)) / 2)
+
+/* sharkle */
+#define VCO_BAND_LOW 750
+#define VCO_BAND_MID 1100
+#define VCO_BAND_HIGH 1500
+#define PHY_REF_CLK 26000
+
+static int dphy_calc_pll_param(struct dphy_pll *pll)
+{
+ const u32 khz = 1000;
+ const u32 mhz = 1000000;
+ const unsigned long long factor = 100;
+ unsigned long long tmp;
+ int i;
+
+ pll->potential_fvco = pll->freq / khz;
+ pll->ref_clk = PHY_REF_CLK / khz;
+
+ for (i = 0; i < 4; ++i) {
+ if (pll->potential_fvco >= VCO_BAND_LOW &&
+ pll->potential_fvco <= VCO_BAND_HIGH) {
+ pll->fvco = pll->potential_fvco;
+ pll->out_sel = BIT(i);
+ break;
+ }
+ pll->potential_fvco <<= 1;
+ }
+ if (pll->fvco == 0)
+ return -EINVAL;
+
+ if (pll->fvco >= VCO_BAND_LOW && pll->fvco <= VCO_BAND_MID) {
+ /* vco band control */
+ pll->vco_band = 0x0;
+ /* low pass filter control */
+ pll->lpf_sel = 1;
+ } else if (pll->fvco > VCO_BAND_MID && pll->fvco <= VCO_BAND_HIGH) {
+ pll->vco_band = 0x1;
+ pll->lpf_sel = 0;
+ } else {
+ return -EINVAL;
+ }
+
+ pll->nint = pll->fvco / pll->ref_clk;
+ tmp = pll->fvco * factor * mhz;
+ do_div(tmp, pll->ref_clk);
+ tmp = tmp - pll->nint * factor * mhz;
+ tmp *= BIT(20);
+ do_div(tmp, 100000000);
+ pll->kint = (u32)tmp;
+ pll->refin = 3; /* pre-divider bypass */
+ pll->sdm_en = true; /* use fraction N PLL */
+ pll->fdk_s = 0x1; /* fraction */
+ pll->cp_s = 0x0;
+ pll->det_delay = 0x1;
+
+ return 0;
+}
+
+static void dphy_set_pll_reg(struct dphy_pll *pll, struct regmap *regmap)
+{
+ u8 reg_val[9] = {0};
+ int i;
+
+ u8 reg_addr[] = {
+ 0x03, 0x04, 0x06, 0x08, 0x09,
+ 0x0a, 0x0b, 0x0e, 0x0f
+ };
+
+ reg_val[0] = 1 | (1 << 1) | (pll->lpf_sel << 2);
+ reg_val[1] = pll->div | (1 << 3) | (pll->cp_s << 5) | (pll->fdk_s << 7);
+ reg_val[2] = pll->nint;
+ reg_val[3] = pll->vco_band | (pll->sdm_en << 1) | (pll->refin << 2);
+ reg_val[4] = pll->kint >> 12;
+ reg_val[5] = pll->kint >> 4;
+ reg_val[6] = pll->out_sel | ((pll->kint << 4) & 0xf);
+ reg_val[7] = 1 << 4;
+ reg_val[8] = pll->det_delay;
+
+ for (i = 0; i < sizeof(reg_addr); ++i) {
+ regmap_write(regmap, reg_addr[i], reg_val[i]);
+ DRM_DEBUG("%02x: %02x\n", reg_addr[i], reg_val[i]);
+ }
+}
+
+int dphy_pll_config(struct dsi_context *ctx)
+{
+ struct sprd_dsi *dsi = container_of(ctx, struct sprd_dsi, ctx);
+ struct regmap *regmap = ctx->regmap;
+ struct dphy_pll *pll = &ctx->pll;
+ int ret;
+
+ pll->freq = dsi->slave->hs_rate;
+
+ /* FREQ = 26M * (NINT + KINT / 2^20) / out_sel */
+ ret = dphy_calc_pll_param(pll);
+ if (ret) {
+ drm_err(dsi->drm, "failed to calculate dphy pll parameters\n");
+ return ret;
+ }
+ dphy_set_pll_reg(pll, regmap);
+
+ return 0;
+}
+
+static void dphy_set_timing_reg(struct regmap *regmap, int type, u8 val[])
+{
+ switch (type) {
+ case REQUEST_TIME:
+ regmap_write(regmap, 0x31, val[CLK]);
+ regmap_write(regmap, 0x41, val[DATA]);
+ regmap_write(regmap, 0x51, val[DATA]);
+ regmap_write(regmap, 0x61, val[DATA]);
+ regmap_write(regmap, 0x71, val[DATA]);
+
+ regmap_write(regmap, 0x90, val[CLK]);
+ regmap_write(regmap, 0xa0, val[DATA]);
+ regmap_write(regmap, 0xb0, val[DATA]);
+ regmap_write(regmap, 0xc0, val[DATA]);
+ regmap_write(regmap, 0xd0, val[DATA]);
+ break;
+ case PREPARE_TIME:
+ regmap_write(regmap, 0x32, val[CLK]);
+ regmap_write(regmap, 0x42, val[DATA]);
+ regmap_write(regmap, 0x52, val[DATA]);
+ regmap_write(regmap, 0x62, val[DATA]);
+ regmap_write(regmap, 0x72, val[DATA]);
+
+ regmap_write(regmap, 0x91, val[CLK]);
+ regmap_write(regmap, 0xa1, val[DATA]);
+ regmap_write(regmap, 0xb1, val[DATA]);
+ regmap_write(regmap, 0xc1, val[DATA]);
+ regmap_write(regmap, 0xd1, val[DATA]);
+ break;
+ case ZERO_TIME:
+ regmap_write(regmap, 0x33, val[CLK]);
+ regmap_write(regmap, 0x43, val[DATA]);
+ regmap_write(regmap, 0x53, val[DATA]);
+ regmap_write(regmap, 0x63, val[DATA]);
+ regmap_write(regmap, 0x73, val[DATA]);
+
+ regmap_write(regmap, 0x92, val[CLK]);
+ regmap_write(regmap, 0xa2, val[DATA]);
+ regmap_write(regmap, 0xb2, val[DATA]);
+ regmap_write(regmap, 0xc2, val[DATA]);
+ regmap_write(regmap, 0xd2, val[DATA]);
+ break;
+ case TRAIL_TIME:
+ regmap_write(regmap, 0x34, val[CLK]);
+ regmap_write(regmap, 0x44, val[DATA]);
+ regmap_write(regmap, 0x54, val[DATA]);
+ regmap_write(regmap, 0x64, val[DATA]);
+ regmap_write(regmap, 0x74, val[DATA]);
+
+ regmap_write(regmap, 0x93, val[CLK]);
+ regmap_write(regmap, 0xa3, val[DATA]);
+ regmap_write(regmap, 0xb3, val[DATA]);
+ regmap_write(regmap, 0xc3, val[DATA]);
+ regmap_write(regmap, 0xd3, val[DATA]);
+ break;
+ case EXIT_TIME:
+ regmap_write(regmap, 0x36, val[CLK]);
+ regmap_write(regmap, 0x46, val[DATA]);
+ regmap_write(regmap, 0x56, val[DATA]);
+ regmap_write(regmap, 0x66, val[DATA]);
+ regmap_write(regmap, 0x76, val[DATA]);
+
+ regmap_write(regmap, 0x95, val[CLK]);
+ regmap_write(regmap, 0xA5, val[DATA]);
+ regmap_write(regmap, 0xB5, val[DATA]);
+ regmap_write(regmap, 0xc5, val[DATA]);
+ regmap_write(regmap, 0xd5, val[DATA]);
+ break;
+ case CLKPOST_TIME:
+ regmap_write(regmap, 0x35, val[CLK]);
+ regmap_write(regmap, 0x94, val[CLK]);
+ break;
+
+ /* the following just use default value */
+ case SETTLE_TIME:
+ fallthrough;
+ case TA_GET:
+ fallthrough;
+ case TA_GO:
+ fallthrough;
+ case TA_SURE:
+ fallthrough;
+ default:
+ break;
+ }
+}
+
+void dphy_timing_config(struct dsi_context *ctx)
+{
+ struct regmap *regmap = ctx->regmap;
+ struct dphy_pll *pll = &ctx->pll;
+ const u32 factor = 2;
+ const u32 scale = 100;
+ u32 t_ui, t_byteck, t_half_byteck;
+ u32 range[2], constant;
+ u8 val[2];
+ u32 tmp = 0;
+
+ /* t_ui: 1 ui, byteck: 8 ui, half byteck: 4 ui */
+ t_ui = 1000 * scale / (pll->freq / 1000);
+ t_byteck = t_ui << 3;
+ t_half_byteck = t_ui << 2;
+ constant = t_ui << 1;
+
+ /* REQUEST_TIME: HS T-LPX: LP-01
+ * For T-LPX, mipi spec defined min value is 50ns,
+ * but maybe it shouldn't be too small, because BTA,
+ * LP-10, LP-00, LP-01, all of this is related to T-LPX.
+ */
+ range[L] = 50 * scale;
+ range[H] = INFINITY;
+ val[CLK] = DIV_ROUND_UP(range[L] * (factor << 1), t_byteck) - 2;
+ val[DATA] = val[CLK];
+ dphy_set_timing_reg(regmap, REQUEST_TIME, val);
+
+ /* PREPARE_TIME: HS sequence: LP-00 */
+ range[L] = 38 * scale;
+ range[H] = 95 * scale;
+ tmp = AVERAGE(range[L], range[H]);
+ val[CLK] = DIV_ROUND_UP(AVERAGE(range[L], range[H]), t_half_byteck) - 1;
+ range[L] = 40 * scale + 4 * t_ui;
+ range[H] = 85 * scale + 6 * t_ui;
+ tmp |= AVERAGE(range[L], range[H]) << 16;
+ val[DATA] = DIV_ROUND_UP(AVERAGE(range[L], range[H]), t_half_byteck) - 1;
+ dphy_set_timing_reg(regmap, PREPARE_TIME, val);
+
+ /* ZERO_TIME: HS-ZERO */
+ range[L] = 300 * scale;
+ range[H] = INFINITY;
+ val[CLK] = DIV_ROUND_UP(range[L] * factor + (tmp & 0xffff)
+ - 525 * t_byteck / 100, t_byteck) - 2;
+ range[L] = 145 * scale + 10 * t_ui;
+ val[DATA] = DIV_ROUND_UP(range[L] * factor
+ + ((tmp >> 16) & 0xffff) - 525 * t_byteck / 100,
+ t_byteck) - 2;
+ dphy_set_timing_reg(regmap, ZERO_TIME, val);
+
+ /* TRAIL_TIME: HS-TRAIL */
+ range[L] = 60 * scale;
+ range[H] = INFINITY;
+ val[CLK] = DIV_ROUND_UP(range[L] * factor - constant, t_half_byteck);
+ range[L] = max(8 * t_ui, 60 * scale + 4 * t_ui);
+ val[DATA] = DIV_ROUND_UP(range[L] * 3 / 2 - constant, t_half_byteck) - 2;
+ dphy_set_timing_reg(regmap, TRAIL_TIME, val);
+
+ /* EXIT_TIME: */
+ range[L] = 100 * scale;
+ range[H] = INFINITY;
+ val[CLK] = DIV_ROUND_UP(range[L] * factor, t_byteck) - 2;
+ val[DATA] = val[CLK];
+ dphy_set_timing_reg(regmap, EXIT_TIME, val);
+
+ /* CLKPOST_TIME: */
+ range[L] = 60 * scale + 52 * t_ui;
+ range[H] = INFINITY;
+ val[CLK] = DIV_ROUND_UP(range[L] * factor, t_byteck) - 2;
+ val[DATA] = val[CLK];
+ dphy_set_timing_reg(regmap, CLKPOST_TIME, val);
+
+ /* SETTLE_TIME:
+ * This time is used for receiver. So for transmitter,
+ * it can be ignored.
+ */
+
+ /* TA_GO:
+ * transmitter drives bridge state(LP-00) before releasing control,
+ * reg 0x1f default value: 0x04, which is good.
+ */
+
+ /* TA_SURE:
+ * After LP-10 state and before bridge state(LP-00),
+ * reg 0x20 default value: 0x01, which is good.
+ */
+
+ /* TA_GET:
+ * receiver drives Bridge state(LP-00) before releasing control
+ * reg 0x21 default value: 0x03, which is good.
+ */
+}
diff --git a/drivers/gpu/drm/sprd/sprd_dpu.c b/drivers/gpu/drm/sprd/sprd_dpu.c
new file mode 100644
index 0000000..06a3414
--- /dev/null
+++ b/drivers/gpu/drm/sprd/sprd_dpu.c
@@ -0,0 +1,880 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2020 Unisoc Inc.
+ */
+
+#include <linux/component.h>
+#include <linux/delay.h>
+#include <linux/dma-buf.h>
+#include <linux/io.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/of_device.h>
+#include <linux/of_graph.h>
+#include <linux/of_irq.h>
+#include <linux/wait.h>
+#include <linux/workqueue.h>
+
+#include <drm/drm_atomic_helper.h>
+#include <drm/drm_crtc_helper.h>
+#include <drm/drm_fb_cma_helper.h>
+#include <drm/drm_gem_cma_helper.h>
+#include <drm/drm_gem_framebuffer_helper.h>
+#include <drm/drm_plane_helper.h>
+
+#include "sprd_drm.h"
+#include "sprd_dpu.h"
+#include "sprd_dsi.h"
+
+/* Global control registers */
+#define REG_DPU_CTRL 0x04
+#define REG_DPU_CFG0 0x08
+#define REG_PANEL_SIZE 0x20
+#define REG_BLEND_SIZE 0x24
+#define REG_BG_COLOR 0x2C
+
+/* Layer0 control registers */
+#define REG_LAY_BASE_ADDR0 0x30
+#define REG_LAY_BASE_ADDR1 0x34
+#define REG_LAY_BASE_ADDR2 0x38
+#define REG_LAY_CTRL 0x40
+#define REG_LAY_SIZE 0x44
+#define REG_LAY_PITCH 0x48
+#define REG_LAY_POS 0x4C
+#define REG_LAY_ALPHA 0x50
+#define REG_LAY_CROP_START 0x5C
+
+/* Interrupt control registers */
+#define REG_DPU_INT_EN 0x1E0
+#define REG_DPU_INT_CLR 0x1E4
+#define REG_DPU_INT_STS 0x1E8
+
+/* DPI control registers */
+#define REG_DPI_CTRL 0x1F0
+#define REG_DPI_H_TIMING 0x1F4
+#define REG_DPI_V_TIMING 0x1F8
+
+/* MMU control registers */
+#define REG_MMU_EN 0x800
+#define REG_MMU_VPN_RANGE 0x80C
+#define REG_MMU_PPN1 0x83C
+#define REG_MMU_RANGE1 0x840
+#define REG_MMU_PPN2 0x844
+#define REG_MMU_RANGE2 0x848
+
+/* Global control bits */
+#define BIT_DPU_RUN BIT(0)
+#define BIT_DPU_STOP BIT(1)
+#define BIT_DPU_REG_UPDATE BIT(2)
+#define BIT_DPU_IF_EDPI BIT(0)
+
+/* Layer control bits */
+#define BIT_DPU_LAY_EN BIT(0)
+#define BIT_DPU_LAY_LAYER_ALPHA (0x01 << 2)
+#define BIT_DPU_LAY_COMBO_ALPHA (0x02 << 2)
+#define BIT_DPU_LAY_FORMAT_YUV422_2PLANE (0x00 << 4)
+#define BIT_DPU_LAY_FORMAT_YUV420_2PLANE (0x01 << 4)
+#define BIT_DPU_LAY_FORMAT_YUV420_3PLANE (0x02 << 4)
+#define BIT_DPU_LAY_FORMAT_ARGB8888 (0x03 << 4)
+#define BIT_DPU_LAY_FORMAT_RGB565 (0x04 << 4)
+#define BIT_DPU_LAY_DATA_ENDIAN_B0B1B2B3 (0x00 << 8)
+#define BIT_DPU_LAY_DATA_ENDIAN_B3B2B1B0 (0x01 << 8)
+#define BIT_DPU_LAY_NO_SWITCH (0x00 << 10)
+#define BIT_DPU_LAY_RB_OR_UV_SWITCH (0x01 << 10)
+#define BIT_DPU_LAY_MODE_BLEND_NORMAL (0x00 << 16)
+#define BIT_DPU_LAY_MODE_BLEND_PREMULT (0x01 << 16)
+#define BIT_DPU_LAY_ROTATION_0 (0x00 << 20)
+#define BIT_DPU_LAY_ROTATION_90 (0x01 << 20)
+#define BIT_DPU_LAY_ROTATION_180 (0x02 << 20)
+#define BIT_DPU_LAY_ROTATION_270 (0x03 << 20)
+#define BIT_DPU_LAY_ROTATION_0_M (0x04 << 20)
+#define BIT_DPU_LAY_ROTATION_90_M (0x05 << 20)
+#define BIT_DPU_LAY_ROTATION_180_M (0x06 << 20)
+#define BIT_DPU_LAY_ROTATION_270_M (0x07 << 20)
+
+/* Interrupt control & status bits */
+#define BIT_DPU_INT_DONE BIT(0)
+#define BIT_DPU_INT_TE BIT(1)
+#define BIT_DPU_INT_ERR BIT(2)
+#define BIT_DPU_INT_UPDATE_DONE BIT(4)
+#define BIT_DPU_INT_VSYNC BIT(5)
+
+/* DPI control bits */
+#define BIT_DPU_EDPI_TE_EN BIT(8)
+#define BIT_DPU_EDPI_FROM_EXTERNAL_PAD BIT(10)
+#define BIT_DPU_DPI_HALT_EN BIT(16)
+
+static const u32 layer_fmts[] = {
+ DRM_FORMAT_XRGB8888,
+ DRM_FORMAT_XBGR8888,
+ DRM_FORMAT_ARGB8888,
+ DRM_FORMAT_ABGR8888,
+ DRM_FORMAT_RGBA8888,
+ DRM_FORMAT_BGRA8888,
+ DRM_FORMAT_RGBX8888,
+ DRM_FORMAT_RGB565,
+ DRM_FORMAT_BGR565,
+ DRM_FORMAT_NV12,
+ DRM_FORMAT_NV21,
+ DRM_FORMAT_NV16,
+ DRM_FORMAT_NV61,
+ DRM_FORMAT_YUV420,
+ DRM_FORMAT_YVU420,
+};
+
+struct sprd_plane {
+ struct drm_plane base;
+};
+
+static int dpu_wait_stop_done(struct sprd_dpu *dpu)
+{
+ struct dpu_context *ctx = &dpu->ctx;
+ int rc;
+
+ if (ctx->stopped)
+ return 0;
+
+ rc = wait_event_interruptible_timeout(ctx->wait_queue, ctx->evt_stop,
+ msecs_to_jiffies(500));
+ ctx->evt_stop = false;
+
+ ctx->stopped = true;
+
+ if (!rc) {
+ drm_err(dpu->drm, "dpu wait for stop done time out!\n");
+ return -ETIMEDOUT;
+ }
+
+ return 0;
+}
+
+static int dpu_wait_update_done(struct sprd_dpu *dpu)
+{
+ struct dpu_context *ctx = &dpu->ctx;
+ int rc;
+
+ ctx->evt_update = false;
+
+ rc = wait_event_interruptible_timeout(ctx->wait_queue, ctx->evt_update,
+ msecs_to_jiffies(500));
+
+ if (!rc) {
+ drm_err(dpu->drm, "dpu wait for reg update done time out!\n");
+ return -ETIMEDOUT;
+ }
+
+ return 0;
+}
+
+static u32 drm_format_to_dpu(struct drm_framebuffer *fb)
+{
+ u32 format = 0;
+
+ switch (fb->format->format) {
+ case DRM_FORMAT_BGRA8888:
+ /* BGRA8888 -> ARGB8888 */
+ format |= BIT_DPU_LAY_DATA_ENDIAN_B3B2B1B0;
+ format |= BIT_DPU_LAY_FORMAT_ARGB8888;
+ break;
+ case DRM_FORMAT_RGBX8888:
+ case DRM_FORMAT_RGBA8888:
+ /* RGBA8888 -> ABGR8888 */
+ format |= BIT_DPU_LAY_DATA_ENDIAN_B3B2B1B0;
+ fallthrough;
+ case DRM_FORMAT_ABGR8888:
+ /* RB switch */
+ format |= BIT_DPU_LAY_RB_OR_UV_SWITCH;
+ fallthrough;
+ case DRM_FORMAT_ARGB8888:
+ format |= BIT_DPU_LAY_FORMAT_ARGB8888;
+ break;
+ case DRM_FORMAT_XBGR8888:
+ /* RB switch */
+ format |= BIT_DPU_LAY_RB_OR_UV_SWITCH;
+ fallthrough;
+ case DRM_FORMAT_XRGB8888:
+ format |= BIT_DPU_LAY_FORMAT_ARGB8888;
+ break;
+ case DRM_FORMAT_BGR565:
+ /* RB switch */
+ format |= BIT_DPU_LAY_RB_OR_UV_SWITCH;
+ fallthrough;
+ case DRM_FORMAT_RGB565:
+ format |= BIT_DPU_LAY_FORMAT_RGB565;
+ break;
+ case DRM_FORMAT_NV12:
+ /* 2-Lane: Yuv420 */
+ format |= BIT_DPU_LAY_FORMAT_YUV420_2PLANE;
+ /* Y endian */
+ format |= BIT_DPU_LAY_DATA_ENDIAN_B0B1B2B3;
+ /* UV endian */
+ format |= BIT_DPU_LAY_NO_SWITCH;
+ break;
+ case DRM_FORMAT_NV21:
+ /* 2-Lane: Yuv420 */
+ format |= BIT_DPU_LAY_FORMAT_YUV420_2PLANE;
+ /* Y endian */
+ format |= BIT_DPU_LAY_DATA_ENDIAN_B0B1B2B3;
+ /* UV endian */
+ format |= BIT_DPU_LAY_RB_OR_UV_SWITCH;
+ break;
+ case DRM_FORMAT_NV16:
+ /* 2-Lane: Yuv422 */
+ format |= BIT_DPU_LAY_FORMAT_YUV422_2PLANE;
+ /* Y endian */
+ format |= BIT_DPU_LAY_DATA_ENDIAN_B3B2B1B0;
+ /* UV endian */
+ format |= BIT_DPU_LAY_RB_OR_UV_SWITCH;
+ break;
+ case DRM_FORMAT_NV61:
+ /* 2-Lane: Yuv422 */
+ format |= BIT_DPU_LAY_FORMAT_YUV422_2PLANE;
+ /* Y endian */
+ format |= BIT_DPU_LAY_DATA_ENDIAN_B0B1B2B3;
+ /* UV endian */
+ format |= BIT_DPU_LAY_NO_SWITCH;
+ break;
+ case DRM_FORMAT_YUV420:
+ format |= BIT_DPU_LAY_FORMAT_YUV420_3PLANE;
+ /* Y endian */
+ format |= BIT_DPU_LAY_DATA_ENDIAN_B0B1B2B3;
+ /* UV endian */
+ format |= BIT_DPU_LAY_NO_SWITCH;
+ break;
+ case DRM_FORMAT_YVU420:
+ format |= BIT_DPU_LAY_FORMAT_YUV420_3PLANE;
+ /* Y endian */
+ format |= BIT_DPU_LAY_DATA_ENDIAN_B0B1B2B3;
+ /* UV endian */
+ format |= BIT_DPU_LAY_RB_OR_UV_SWITCH;
+ break;
+ default:
+ break;
+ }
+
+ return format;
+}
+
+static u32 drm_rotation_to_dpu(struct drm_plane_state *state)
+{
+ u32 rotation = 0;
+
+ switch (state->rotation) {
+ default:
+ case DRM_MODE_ROTATE_0:
+ rotation = BIT_DPU_LAY_ROTATION_0;
+ break;
+ case DRM_MODE_ROTATE_90:
+ rotation = BIT_DPU_LAY_ROTATION_90;
+ break;
+ case DRM_MODE_ROTATE_180:
+ rotation = BIT_DPU_LAY_ROTATION_180;
+ break;
+ case DRM_MODE_ROTATE_270:
+ rotation = BIT_DPU_LAY_ROTATION_270;
+ break;
+ case DRM_MODE_REFLECT_Y:
+ rotation = BIT_DPU_LAY_ROTATION_180_M;
+ break;
+ case (DRM_MODE_REFLECT_Y | DRM_MODE_ROTATE_90):
+ rotation = BIT_DPU_LAY_ROTATION_90_M;
+ break;
+ case DRM_MODE_REFLECT_X:
+ rotation = BIT_DPU_LAY_ROTATION_0_M;
+ break;
+ case (DRM_MODE_REFLECT_X | DRM_MODE_ROTATE_90):
+ rotation = BIT_DPU_LAY_ROTATION_270_M;
+ break;
+ }
+
+ return rotation;
+}
+
+static u32 drm_blend_to_dpu(struct drm_plane_state *state)
+{
+ u32 blend = 0;
+
+ switch (state->pixel_blend_mode) {
+ case DRM_MODE_BLEND_COVERAGE:
+ /* alpha mode select - combo alpha */
+ blend |= BIT_DPU_LAY_COMBO_ALPHA;
+ /* Normal mode */
+ blend |= BIT_DPU_LAY_MODE_BLEND_NORMAL;
+ break;
+ case DRM_MODE_BLEND_PREMULTI:
+ /* alpha mode select - combo alpha */
+ blend |= BIT_DPU_LAY_COMBO_ALPHA;
+ /* Pre-mult mode */
+ blend |= BIT_DPU_LAY_MODE_BLEND_PREMULT;
+ break;
+ case DRM_MODE_BLEND_PIXEL_NONE:
+ default:
+ /* don't do blending, maybe RGBX */
+ /* alpha mode select - layer alpha */
+ blend |= BIT_DPU_LAY_LAYER_ALPHA;
+ break;
+ }
+
+ return blend;
+}
+
+static void sprd_dpu_layer(struct sprd_dpu *dpu, struct drm_plane_state *state)
+{
+ struct dpu_context *ctx = &dpu->ctx;
+ struct drm_gem_cma_object *cma_obj;
+ struct drm_framebuffer *fb = state->fb;
+ u32 addr, size, offset, pitch, blend, format, rotation;
+ u32 src_x = state->src_x >> 16;
+ u32 src_y = state->src_y >> 16;
+ u32 src_w = state->src_w >> 16;
+ u32 src_h = state->src_h >> 16;
+ u32 dst_x = state->crtc_x;
+ u32 dst_y = state->crtc_y;
+ u32 alpha = state->alpha;
+ u32 index = state->zpos;
+ int i;
+
+ offset = (dst_x & 0xffff) | (dst_y << 16);
+ size = (src_w & 0xffff) | (src_h << 16);
+
+ for (i = 0; i < fb->format->num_planes; i++) {
+ cma_obj = drm_fb_cma_get_gem_obj(fb, i);
+ addr = cma_obj->paddr + fb->offsets[i];
+
+ if (i == 0)
+ layer_reg_wr(ctx, REG_LAY_BASE_ADDR0, addr, index);
+ else if (i == 1)
+ layer_reg_wr(ctx, REG_LAY_BASE_ADDR1, addr, index);
+ else
+ layer_reg_wr(ctx, REG_LAY_BASE_ADDR2, addr, index);
+ }
+
+ if (fb->format->num_planes == 3) {
+ /* UV pitch is 1/2 of Y pitch */
+ pitch = (fb->pitches[0] / fb->format->cpp[0]) |
+ (fb->pitches[0] / fb->format->cpp[0] << 15);
+ } else {
+ pitch = fb->pitches[0] / fb->format->cpp[0];
+ }
+
+ layer_reg_wr(ctx, REG_LAY_POS, offset, index);
+ layer_reg_wr(ctx, REG_LAY_SIZE, size, index);
+ layer_reg_wr(ctx, REG_LAY_CROP_START,
+ src_y << 16 | src_x, index);
+ layer_reg_wr(ctx, REG_LAY_ALPHA, alpha, index);
+ layer_reg_wr(ctx, REG_LAY_PITCH, pitch, index);
+
+ format = drm_format_to_dpu(fb);
+ blend = drm_blend_to_dpu(state);
+ rotation = drm_rotation_to_dpu(state);
+
+ layer_reg_wr(ctx, REG_LAY_CTRL, BIT_DPU_LAY_EN |
+ format |
+ blend |
+ rotation,
+ index);
+}
+
+static void sprd_dpu_flip(struct sprd_dpu *dpu)
+{
+ struct dpu_context *ctx = &dpu->ctx;
+
+ /*
+ * Make sure the dpu is in stop status. DPU has no shadow
+ * registers in EDPI mode. So the config registers can only be
+ * updated in the rising edge of DPU_RUN bit.
+ */
+ if (ctx->if_type == SPRD_DPU_IF_EDPI)
+ dpu_wait_stop_done(dpu);
+
+ /* update trigger and wait */
+ if (ctx->if_type == SPRD_DPU_IF_DPI) {
+ if (!ctx->stopped) {
+ dpu_reg_set(ctx, REG_DPU_CTRL, BIT_DPU_REG_UPDATE);
+ dpu_wait_update_done(dpu);
+ }
+
+ dpu_reg_set(ctx, REG_DPU_INT_EN, BIT_DPU_INT_ERR);
+ } else if (ctx->if_type == SPRD_DPU_IF_EDPI) {
+ dpu_reg_set(ctx, REG_DPU_CTRL, BIT_DPU_RUN);
+
+ ctx->stopped = false;
+ }
+}
+
+static void sprd_dpu_init(struct sprd_dpu *dpu)
+{
+ struct dpu_context *ctx = &dpu->ctx;
+ u32 int_mask = 0;
+
+ writel(0x00, ctx->base + REG_BG_COLOR);
+ writel(0x00, ctx->base + REG_MMU_EN);
+ writel(0x00, ctx->base + REG_MMU_PPN1);
+ writel(0xffff, ctx->base + REG_MMU_RANGE1);
+ writel(0x00, ctx->base + REG_MMU_PPN2);
+ writel(0xffff, ctx->base + REG_MMU_RANGE2);
+ writel(0x1ffff, ctx->base + REG_MMU_VPN_RANGE);
+
+ if (ctx->if_type == SPRD_DPU_IF_DPI) {
+ /* use dpi as interface */
+ dpu_reg_clr(ctx, REG_DPU_CFG0, BIT_DPU_IF_EDPI);
+ /* disable Halt function for SPRD DSI */
+ dpu_reg_clr(ctx, REG_DPI_CTRL, BIT_DPU_DPI_HALT_EN);
+ /* select te from external pad */
+ dpu_reg_set(ctx, REG_DPI_CTRL, BIT_DPU_EDPI_FROM_EXTERNAL_PAD);
+
+ /* enable dpu update done INT */
+ int_mask |= BIT_DPU_INT_UPDATE_DONE;
+ /* enable dpu done INT */
+ int_mask |= BIT_DPU_INT_DONE;
+ /* enable dpu dpi vsync */
+ int_mask |= BIT_DPU_INT_VSYNC;
+ /* enable dpu TE INT */
+ int_mask |= BIT_DPU_INT_TE;
+ /* enable underflow err INT */
+ int_mask |= BIT_DPU_INT_ERR;
+ } else if (ctx->if_type == SPRD_DPU_IF_EDPI) {
+ /* use edpi as interface */
+ dpu_reg_set(ctx, REG_DPU_CFG0, BIT_DPU_IF_EDPI);
+ /* use external te */
+ dpu_reg_set(ctx, REG_DPI_CTRL, BIT_DPU_EDPI_FROM_EXTERNAL_PAD);
+ /* enable te */
+ dpu_reg_set(ctx, REG_DPI_CTRL, BIT_DPU_EDPI_TE_EN);
+
+ /* enable stop done INT */
+ int_mask |= BIT_DPU_INT_DONE;
+ /* enable TE INT */
+ int_mask |= BIT_DPU_INT_TE;
+ }
+
+ writel(int_mask, ctx->base + REG_DPU_INT_EN);
+}
+
+static void sprd_dpu_fini(struct sprd_dpu *dpu)
+{
+ struct dpu_context *ctx = &dpu->ctx;
+
+ writel(0x00, ctx->base + REG_DPU_INT_EN);
+ writel(0xff, ctx->base + REG_DPU_INT_CLR);
+}
+
+static void sprd_dpi_init(struct sprd_dpu *dpu)
+{
+ struct dpu_context *ctx = &dpu->ctx;
+ u32 reg_val;
+ u32 size;
+
+ size = (ctx->vm.vactive << 16) | ctx->vm.hactive;
+ writel(size, ctx->base + REG_PANEL_SIZE);
+ writel(size, ctx->base + REG_BLEND_SIZE);
+
+ if (ctx->if_type == SPRD_DPU_IF_DPI) {
+ /* set dpi timing */
+ reg_val = ctx->vm.hsync_len << 0 |
+ ctx->vm.hback_porch << 8 |
+ ctx->vm.hfront_porch << 20;
+ writel(reg_val, ctx->base + REG_DPI_H_TIMING);
+
+ reg_val = ctx->vm.vsync_len << 0 |
+ ctx->vm.vback_porch << 8 |
+ ctx->vm.vfront_porch << 20;
+ writel(reg_val, ctx->base + REG_DPI_V_TIMING);
+ }
+}
+
+void sprd_dpu_run(struct sprd_dpu *dpu)
+{
+ struct dpu_context *ctx = &dpu->ctx;
+
+ dpu_reg_set(ctx, REG_DPU_CTRL, BIT_DPU_RUN);
+
+ ctx->stopped = false;
+}
+
+void sprd_dpu_stop(struct sprd_dpu *dpu)
+{
+ struct dpu_context *ctx = &dpu->ctx;
+
+ if (ctx->if_type == SPRD_DPU_IF_DPI)
+ dpu_reg_set(ctx, REG_DPU_CTRL, BIT_DPU_STOP);
+
+ dpu_wait_stop_done(dpu);
+}
+
+static int sprd_plane_atomic_check(struct drm_plane *plane,
+ struct drm_atomic_state *state)
+{
+ struct drm_plane_state *plane_state = drm_atomic_get_new_plane_state(state,
+ plane);
+ struct drm_crtc_state *crtc_state;
+ u32 fmt;
+
+ if (!plane_state->fb || !plane_state->crtc)
+ return 0;
+
+ fmt = drm_format_to_dpu(plane_state->fb);
+ if (!fmt)
+ return -EINVAL;
+
+ crtc_state = drm_atomic_get_crtc_state(plane_state->state, plane_state->crtc);
+ if (IS_ERR(crtc_state))
+ return PTR_ERR(crtc_state);
+
+ return drm_atomic_helper_check_plane_state(plane_state, crtc_state,
+ DRM_PLANE_HELPER_NO_SCALING,
+ DRM_PLANE_HELPER_NO_SCALING,
+ true, true);
+}
+
+static void sprd_plane_atomic_update(struct drm_plane *drm_plane,
+ struct drm_atomic_state *state)
+{
+ struct drm_plane_state *new_state = drm_atomic_get_new_plane_state(state,
+ drm_plane);
+ struct sprd_dpu *dpu = to_sprd_crtc(new_state->crtc);
+
+ /* start configure dpu layers */
+ sprd_dpu_layer(dpu, new_state);
+}
+
+static void sprd_plane_atomic_disable(struct drm_plane *drm_plane,
+ struct drm_atomic_state *state)
+{
+ struct drm_plane_state *old_state = drm_atomic_get_old_plane_state(state,
+ drm_plane);
+ struct sprd_dpu *dpu = to_sprd_crtc(old_state->crtc);
+
+ layer_reg_wr(&dpu->ctx, REG_LAY_CTRL, 0x00, old_state->zpos);
+}
+
+static void sprd_plane_create_properties(struct sprd_plane *plane, int index)
+{
+ unsigned int supported_modes = BIT(DRM_MODE_BLEND_PIXEL_NONE) |
+ BIT(DRM_MODE_BLEND_PREMULTI) |
+ BIT(DRM_MODE_BLEND_COVERAGE);
+
+ /* create rotation property */
+ drm_plane_create_rotation_property(&plane->base,
+ DRM_MODE_ROTATE_0,
+ DRM_MODE_ROTATE_MASK |
+ DRM_MODE_REFLECT_MASK);
+
+ /* create alpha property */
+ drm_plane_create_alpha_property(&plane->base);
+
+ /* create blend mode property */
+ drm_plane_create_blend_mode_property(&plane->base, supported_modes);
+
+ /* create zpos property */
+ drm_plane_create_zpos_immutable_property(&plane->base, index);
+}
+
+static const struct drm_plane_helper_funcs sprd_plane_helper_funcs = {
+ .atomic_check = sprd_plane_atomic_check,
+ .atomic_update = sprd_plane_atomic_update,
+ .atomic_disable = sprd_plane_atomic_disable,
+};
+
+static const struct drm_plane_funcs sprd_plane_funcs = {
+ .update_plane = drm_atomic_helper_update_plane,
+ .disable_plane = drm_atomic_helper_disable_plane,
+ .destroy = drm_plane_cleanup,
+ .reset = drm_atomic_helper_plane_reset,
+ .atomic_duplicate_state = drm_atomic_helper_plane_duplicate_state,
+ .atomic_destroy_state = drm_atomic_helper_plane_destroy_state,
+};
+
+static struct sprd_plane *sprd_planes_init(struct drm_device *drm)
+{
+ struct sprd_plane *plane, *primary;
+ enum drm_plane_type plane_type;
+ int i;
+
+ for (i = 0; i < 6; i++) {
+ plane_type = (i == 0) ? DRM_PLANE_TYPE_PRIMARY :
+ DRM_PLANE_TYPE_OVERLAY;
+
+ plane = drmm_universal_plane_alloc(drm, struct sprd_plane, base,
+ 1, &sprd_plane_funcs,
+ layer_fmts, ARRAY_SIZE(layer_fmts),
+ NULL, plane_type, NULL);
+ if (IS_ERR(plane)) {
+ drm_err(drm, "failed to init drm plane: %d\n", i);
+ return plane;
+ }
+
+ drm_plane_helper_add(&plane->base, &sprd_plane_helper_funcs);
+
+ sprd_plane_create_properties(plane, i);
+
+ if (i == 0)
+ primary = plane;
+ }
+
+ return primary;
+}
+
+static void sprd_crtc_mode_set_nofb(struct drm_crtc *crtc)
+{
+ struct sprd_dpu *dpu = to_sprd_crtc(crtc);
+ struct drm_display_mode *mode = &crtc->state->adjusted_mode;
+ struct drm_encoder *encoder;
+ struct sprd_dsi *dsi;
+
+ drm_display_mode_to_videomode(mode, &dpu->ctx.vm);
+
+ drm_for_each_encoder_mask(encoder, crtc->dev,
+ crtc->state->encoder_mask) {
+ dsi = encoder_to_dsi(encoder);
+
+ if (dsi->slave->mode_flags & MIPI_DSI_MODE_VIDEO)
+ dpu->ctx.if_type = SPRD_DPU_IF_DPI;
+ else
+ dpu->ctx.if_type = SPRD_DPU_IF_EDPI;
+ }
+
+ sprd_dpi_init(dpu);
+}
+
+static void sprd_crtc_atomic_enable(struct drm_crtc *crtc,
+ struct drm_atomic_state *state)
+{
+ struct sprd_dpu *dpu = to_sprd_crtc(crtc);
+
+ sprd_dpu_init(dpu);
+
+ drm_crtc_vblank_on(&dpu->base);
+}
+
+static void sprd_crtc_atomic_disable(struct drm_crtc *crtc,
+ struct drm_atomic_state *state)
+{
+ struct sprd_dpu *dpu = to_sprd_crtc(crtc);
+ struct drm_device *drm = dpu->base.dev;
+
+ drm_crtc_vblank_off(&dpu->base);
+
+ sprd_dpu_fini(dpu);
+
+ spin_lock_irq(&drm->event_lock);
+ if (crtc->state->event) {
+ drm_crtc_send_vblank_event(crtc, crtc->state->event);
+ crtc->state->event = NULL;
+ }
+ spin_unlock_irq(&drm->event_lock);
+}
+
+static void sprd_crtc_atomic_flush(struct drm_crtc *crtc,
+ struct drm_atomic_state *state)
+
+{
+ struct sprd_dpu *dpu = to_sprd_crtc(crtc);
+ struct drm_device *drm = dpu->base.dev;
+
+ sprd_dpu_flip(dpu);
+
+ spin_lock_irq(&drm->event_lock);
+ if (crtc->state->event) {
+ drm_crtc_send_vblank_event(crtc, crtc->state->event);
+ crtc->state->event = NULL;
+ }
+ spin_unlock_irq(&drm->event_lock);
+}
+
+static int sprd_crtc_enable_vblank(struct drm_crtc *crtc)
+{
+ struct sprd_dpu *dpu = to_sprd_crtc(crtc);
+
+ dpu_reg_set(&dpu->ctx, REG_DPU_INT_EN, BIT_DPU_INT_VSYNC);
+
+ return 0;
+}
+
+static void sprd_crtc_disable_vblank(struct drm_crtc *crtc)
+{
+ struct sprd_dpu *dpu = to_sprd_crtc(crtc);
+
+ dpu_reg_clr(&dpu->ctx, REG_DPU_INT_EN, BIT_DPU_INT_VSYNC);
+}
+
+static const struct drm_crtc_helper_funcs sprd_crtc_helper_funcs = {
+ .mode_set_nofb = sprd_crtc_mode_set_nofb,
+ .atomic_flush = sprd_crtc_atomic_flush,
+ .atomic_enable = sprd_crtc_atomic_enable,
+ .atomic_disable = sprd_crtc_atomic_disable,
+};
+
+static const struct drm_crtc_funcs sprd_crtc_funcs = {
+ .destroy = drm_crtc_cleanup,
+ .set_config = drm_atomic_helper_set_config,
+ .page_flip = drm_atomic_helper_page_flip,
+ .reset = drm_atomic_helper_crtc_reset,
+ .atomic_duplicate_state = drm_atomic_helper_crtc_duplicate_state,
+ .atomic_destroy_state = drm_atomic_helper_crtc_destroy_state,
+ .enable_vblank = sprd_crtc_enable_vblank,
+ .disable_vblank = sprd_crtc_disable_vblank,
+};
+
+static struct sprd_dpu *sprd_crtc_init(struct drm_device *drm,
+ struct drm_plane *primary, struct device *dev)
+{
+ struct device_node *port;
+ struct sprd_dpu *dpu;
+
+ dpu = drmm_crtc_alloc_with_planes(drm, struct sprd_dpu, base,
+ primary, NULL,
+ &sprd_crtc_funcs, NULL);
+ if (IS_ERR(dpu)) {
+ drm_err(drm, "failed to init crtc\n");
+ return dpu;
+ }
+ drm_crtc_helper_add(&dpu->base, &sprd_crtc_helper_funcs);
+
+ /*
+ * set crtc port so that drm_of_find_possible_crtcs call works
+ */
+ port = of_graph_get_port_by_id(dev->of_node, 0);
+ if (!port) {
+ drm_err(drm, "failed to found crtc output port for %s\n",
+ dev->of_node->full_name);
+ return ERR_PTR(-EINVAL);
+ }
+ dpu->base.port = port;
+ of_node_put(port);
+
+ return dpu;
+}
+
+static irqreturn_t sprd_dpu_isr(int irq, void *data)
+{
+ struct sprd_dpu *dpu = data;
+ struct dpu_context *ctx = &dpu->ctx;
+ u32 reg_val, int_mask = 0;
+
+ reg_val = readl(ctx->base + REG_DPU_INT_STS);
+
+ /* disable err interrupt */
+ if (reg_val & BIT_DPU_INT_ERR) {
+ int_mask |= BIT_DPU_INT_ERR;
+ drm_warn(dpu->drm, "Warning: dpu underflow!\n");
+ }
+
+ /* dpu update done isr */
+ if (reg_val & BIT_DPU_INT_UPDATE_DONE) {
+ ctx->evt_update = true;
+ wake_up_interruptible_all(&ctx->wait_queue);
+ }
+
+ /* dpu stop done isr */
+ if (reg_val & BIT_DPU_INT_DONE) {
+ ctx->evt_stop = true;
+ wake_up_interruptible_all(&ctx->wait_queue);
+ }
+
+ if (reg_val & BIT_DPU_INT_VSYNC)
+ drm_crtc_handle_vblank(&dpu->base);
+
+ writel(reg_val, ctx->base + REG_DPU_INT_CLR);
+ dpu_reg_clr(ctx, REG_DPU_INT_EN, int_mask);
+
+ return IRQ_HANDLED;
+}
+
+static int sprd_dpu_context_init(struct sprd_dpu *dpu,
+ struct device *dev)
+{
+ struct platform_device *pdev = to_platform_device(dev);
+ struct dpu_context *ctx = &dpu->ctx;
+ struct resource *res;
+ int ret;
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ ctx->base = devm_ioremap(dev, res->start, resource_size(res));
+ if (!ctx->base) {
+ dev_err(dev, "failed to map dpu registers\n");
+ return -EFAULT;
+ }
+
+ ctx->irq = platform_get_irq(pdev, 0);
+ if (ctx->irq < 0) {
+ dev_err(dev, "failed to get dpu irq\n");
+ return ctx->irq;
+ }
+
+ /* disable and clear interrupts before register dpu IRQ. */
+ writel(0x00, ctx->base + REG_DPU_INT_EN);
+ writel(0xff, ctx->base + REG_DPU_INT_CLR);
+
+ ret = devm_request_irq(dev, ctx->irq, sprd_dpu_isr,
+ IRQF_TRIGGER_NONE, "DPU", dpu);
+ if (ret) {
+ dev_err(dev, "failed to register dpu irq handler\n");
+ return ret;
+ }
+
+ init_waitqueue_head(&ctx->wait_queue);
+
+ return 0;
+}
+
+static int sprd_dpu_bind(struct device *dev, struct device *master, void *data)
+{
+ struct drm_device *drm = data;
+ struct sprd_dpu *dpu;
+ struct sprd_plane *plane;
+ int ret;
+
+ plane = sprd_planes_init(drm);
+ if (IS_ERR(plane))
+ return PTR_ERR(plane);
+
+ dpu = sprd_crtc_init(drm, &plane->base, dev);
+ if (IS_ERR(dpu))
+ return PTR_ERR(dpu);
+
+ dpu->drm = drm;
+ dev_set_drvdata(dev, dpu);
+
+ ret = sprd_dpu_context_init(dpu, dev);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+static const struct component_ops dpu_component_ops = {
+ .bind = sprd_dpu_bind,
+};
+
+static const struct of_device_id dpu_match_table[] = {
+ { .compatible = "sprd,sharkl3-dpu" },
+ { /* sentinel */ },
+};
+MODULE_DEVICE_TABLE(of, dpu_match_table);
+
+static int sprd_dpu_probe(struct platform_device *pdev)
+{
+ return component_add(&pdev->dev, &dpu_component_ops);
+}
+
+static int sprd_dpu_remove(struct platform_device *pdev)
+{
+ component_del(&pdev->dev, &dpu_component_ops);
+
+ return 0;
+}
+
+struct platform_driver sprd_dpu_driver = {
+ .probe = sprd_dpu_probe,
+ .remove = sprd_dpu_remove,
+ .driver = {
+ .name = "sprd-dpu-drv",
+ .of_match_table = dpu_match_table,
+ },
+};
+
+MODULE_AUTHOR("Leon He <leon.he@unisoc.com>");
+MODULE_AUTHOR("Kevin Tang <kevin.tang@unisoc.com>");
+MODULE_DESCRIPTION("Unisoc Display Controller Driver");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/gpu/drm/sprd/sprd_dpu.h b/drivers/gpu/drm/sprd/sprd_dpu.h
new file mode 100644
index 0000000..157a78f
--- /dev/null
+++ b/drivers/gpu/drm/sprd/sprd_dpu.h
@@ -0,0 +1,109 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020 Unisoc Inc.
+ */
+
+#ifndef __SPRD_DPU_H__
+#define __SPRD_DPU_H__
+
+#include <linux/bug.h>
+#include <linux/delay.h>
+#include <linux/device.h>
+#include <linux/kernel.h>
+#include <linux/platform_device.h>
+#include <linux/string.h>
+#include <video/videomode.h>
+
+#include <drm/drm_crtc.h>
+#include <drm/drm_fourcc.h>
+#include <drm/drm_print.h>
+#include <drm/drm_vblank.h>
+#include <uapi/drm/drm_mode.h>
+
+/* DPU Layer registers offset */
+#define DPU_LAY_REG_OFFSET 0x30
+
+enum {
+ SPRD_DPU_IF_DPI,
+ SPRD_DPU_IF_EDPI,
+ SPRD_DPU_IF_LIMIT
+};
+
+/**
+ * Sprd DPU context structure
+ *
+ * @base: DPU controller base address
+ * @irq: IRQ number to install the handler for
+ * @if_type: The type of DPI interface, default is DPI mode.
+ * @vm: videomode structure to use for DPU and DPI initialization
+ * @stopped: indicates whether DPU are stopped
+ * @wait_queue: wait queue, used to wait for DPU shadow register update done and
+ * DPU stop register done interrupt signal.
+ * @evt_update: wait queue condition for DPU shadow register
+ * @evt_stop: wait queue condition for DPU stop register
+ */
+struct dpu_context {
+ void __iomem *base;
+ int irq;
+ u8 if_type;
+ struct videomode vm;
+ bool stopped;
+ wait_queue_head_t wait_queue;
+ bool evt_update;
+ bool evt_stop;
+};
+
+/**
+ * Sprd DPU device structure
+ *
+ * @crtc: crtc object
+ * @drm: A point to drm device
+ * @ctx: DPU's implementation specific context object
+ */
+struct sprd_dpu {
+ struct drm_crtc base;
+ struct drm_device *drm;
+ struct dpu_context ctx;
+};
+
+static inline struct sprd_dpu *to_sprd_crtc(struct drm_crtc *crtc)
+{
+ return container_of(crtc, struct sprd_dpu, base);
+}
+
+static inline void
+dpu_reg_set(struct dpu_context *ctx, u32 offset, u32 set_bits)
+{
+ u32 bits = readl_relaxed(ctx->base + offset);
+
+ writel(bits | set_bits, ctx->base + offset);
+}
+
+static inline void
+dpu_reg_clr(struct dpu_context *ctx, u32 offset, u32 clr_bits)
+{
+ u32 bits = readl_relaxed(ctx->base + offset);
+
+ writel(bits & ~clr_bits, ctx->base + offset);
+}
+
+static inline u32
+layer_reg_rd(struct dpu_context *ctx, u32 offset, int index)
+{
+ u32 layer_offset = offset + index * DPU_LAY_REG_OFFSET;
+
+ return readl(ctx->base + layer_offset);
+}
+
+static inline void
+layer_reg_wr(struct dpu_context *ctx, u32 offset, u32 cfg_bits, int index)
+{
+ u32 layer_offset = offset + index * DPU_LAY_REG_OFFSET;
+
+ writel(cfg_bits, ctx->base + layer_offset);
+}
+
+void sprd_dpu_run(struct sprd_dpu *dpu);
+void sprd_dpu_stop(struct sprd_dpu *dpu);
+
+#endif
diff --git a/drivers/gpu/drm/sprd/sprd_drm.c b/drivers/gpu/drm/sprd/sprd_drm.c
new file mode 100644
index 0000000..a077e2d
--- /dev/null
+++ b/drivers/gpu/drm/sprd/sprd_drm.c
@@ -0,0 +1,205 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2020 Unisoc Inc.
+ */
+
+#include <linux/component.h>
+#include <linux/dma-mapping.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/of_graph.h>
+#include <linux/of_platform.h>
+
+#include <drm/drm_atomic_helper.h>
+#include <drm/drm_crtc_helper.h>
+#include <drm/drm_drv.h>
+#include <drm/drm_gem_cma_helper.h>
+#include <drm/drm_gem_framebuffer_helper.h>
+#include <drm/drm_of.h>
+#include <drm/drm_probe_helper.h>
+#include <drm/drm_vblank.h>
+
+#include "sprd_drm.h"
+
+#define DRIVER_NAME "sprd"
+#define DRIVER_DESC "Spreadtrum SoCs' DRM Driver"
+#define DRIVER_DATE "20200201"
+#define DRIVER_MAJOR 1
+#define DRIVER_MINOR 0
+
+static const struct drm_mode_config_helper_funcs sprd_drm_mode_config_helper = {
+ .atomic_commit_tail = drm_atomic_helper_commit_tail_rpm,
+};
+
+static const struct drm_mode_config_funcs sprd_drm_mode_config_funcs = {
+ .fb_create = drm_gem_fb_create,
+ .atomic_check = drm_atomic_helper_check,
+ .atomic_commit = drm_atomic_helper_commit,
+};
+
+static void sprd_drm_mode_config_init(struct drm_device *drm)
+{
+ drm->mode_config.min_width = 0;
+ drm->mode_config.min_height = 0;
+ drm->mode_config.max_width = 8192;
+ drm->mode_config.max_height = 8192;
+ drm->mode_config.allow_fb_modifiers = true;
+
+ drm->mode_config.funcs = &sprd_drm_mode_config_funcs;
+ drm->mode_config.helper_private = &sprd_drm_mode_config_helper;
+}
+
+DEFINE_DRM_GEM_CMA_FOPS(sprd_drm_fops);
+
+static struct drm_driver sprd_drm_drv = {
+ .driver_features = DRIVER_GEM | DRIVER_MODESET | DRIVER_ATOMIC,
+ .fops = &sprd_drm_fops,
+
+ /* GEM Operations */
+ DRM_GEM_CMA_DRIVER_OPS,
+
+ .name = DRIVER_NAME,
+ .desc = DRIVER_DESC,
+ .date = DRIVER_DATE,
+ .major = DRIVER_MAJOR,
+ .minor = DRIVER_MINOR,
+};
+
+static int sprd_drm_bind(struct device *dev)
+{
+ struct platform_device *pdev = to_platform_device(dev);
+ struct drm_device *drm;
+ struct sprd_drm *sprd;
+ int ret;
+
+ sprd = devm_drm_dev_alloc(dev, &sprd_drm_drv, struct sprd_drm, drm);
+ if (IS_ERR(sprd))
+ return PTR_ERR(sprd);
+
+ drm = &sprd->drm;
+ platform_set_drvdata(pdev, drm);
+
+ ret = drmm_mode_config_init(drm);
+ if (ret)
+ return ret;
+
+ sprd_drm_mode_config_init(drm);
+
+ /* bind and init sub drivers */
+ ret = component_bind_all(drm->dev, drm);
+ if (ret) {
+ drm_err(drm, "failed to bind all component.\n");
+ return ret;
+ }
+
+ /* vblank init */
+ ret = drm_vblank_init(drm, drm->mode_config.num_crtc);
+ if (ret) {
+ drm_err(drm, "failed to initialize vblank.\n");
+ goto err_unbind_all;
+ }
+
+ /* reset all the states of crtc/plane/encoder/connector */
+ drm_mode_config_reset(drm);
+
+ /* init kms poll for handling hpd */
+ drm_kms_helper_poll_init(drm);
+
+ ret = drm_dev_register(drm, 0);
+ if (ret < 0)
+ goto err_kms_helper_poll_fini;
+
+ return 0;
+
+err_kms_helper_poll_fini:
+ drm_kms_helper_poll_fini(drm);
+err_unbind_all:
+ component_unbind_all(drm->dev, drm);
+ return ret;
+}
+
+static void sprd_drm_unbind(struct device *dev)
+{
+ struct drm_device *drm = dev_get_drvdata(dev);
+
+ drm_dev_unregister(drm);
+
+ drm_kms_helper_poll_fini(drm);
+
+ component_unbind_all(drm->dev, drm);
+}
+
+static const struct component_master_ops drm_component_ops = {
+ .bind = sprd_drm_bind,
+ .unbind = sprd_drm_unbind,
+};
+
+static int compare_of(struct device *dev, void *data)
+{
+ return dev->of_node == data;
+}
+
+static int sprd_drm_probe(struct platform_device *pdev)
+{
+ return drm_of_component_probe(&pdev->dev, compare_of, &drm_component_ops);
+}
+
+static int sprd_drm_remove(struct platform_device *pdev)
+{
+ component_master_del(&pdev->dev, &drm_component_ops);
+ return 0;
+}
+
+static void sprd_drm_shutdown(struct platform_device *pdev)
+{
+ struct drm_device *drm = platform_get_drvdata(pdev);
+
+ if (!drm) {
+ drm_warn(drm, "drm device is not available, no shutdown\n");
+ return;
+ }
+
+ drm_atomic_helper_shutdown(drm);
+}
+
+static const struct of_device_id drm_match_table[] = {
+ { .compatible = "sprd,display-subsystem", },
+ { /* sentinel */ },
+};
+MODULE_DEVICE_TABLE(of, drm_match_table);
+
+static struct platform_driver sprd_drm_driver = {
+ .probe = sprd_drm_probe,
+ .remove = sprd_drm_remove,
+ .shutdown = sprd_drm_shutdown,
+ .driver = {
+ .name = "sprd-drm-drv",
+ .of_match_table = drm_match_table,
+ },
+};
+
+static struct platform_driver *sprd_drm_drivers[] = {
+ &sprd_drm_driver,
+ &sprd_dpu_driver,
+ &sprd_dsi_driver,
+};
+
+static int __init sprd_drm_init(void)
+{
+ return platform_register_drivers(sprd_drm_drivers,
+ ARRAY_SIZE(sprd_drm_drivers));
+}
+
+static void __exit sprd_drm_exit(void)
+{
+ platform_unregister_drivers(sprd_drm_drivers,
+ ARRAY_SIZE(sprd_drm_drivers));
+}
+
+module_init(sprd_drm_init);
+module_exit(sprd_drm_exit);
+
+MODULE_AUTHOR("Leon He <leon.he@unisoc.com>");
+MODULE_AUTHOR("Kevin Tang <kevin.tang@unisoc.com>");
+MODULE_DESCRIPTION("Unisoc DRM KMS Master Driver");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/gpu/drm/sprd/sprd_drm.h b/drivers/gpu/drm/sprd/sprd_drm.h
new file mode 100644
index 0000000..95d1b97
--- /dev/null
+++ b/drivers/gpu/drm/sprd/sprd_drm.h
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020 Unisoc Inc.
+ */
+
+#ifndef _SPRD_DRM_H_
+#define _SPRD_DRM_H_
+
+#include <drm/drm_atomic.h>
+#include <drm/drm_print.h>
+
+struct sprd_drm {
+ struct drm_device drm;
+};
+
+extern struct platform_driver sprd_dpu_driver;
+extern struct platform_driver sprd_dsi_driver;
+
+#endif /* _SPRD_DRM_H_ */
diff --git a/drivers/gpu/drm/sprd/sprd_dsi.c b/drivers/gpu/drm/sprd/sprd_dsi.c
new file mode 100644
index 0000000..911b3cd
--- /dev/null
+++ b/drivers/gpu/drm/sprd/sprd_dsi.c
@@ -0,0 +1,1073 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2020 Unisoc Inc.
+ */
+
+#include <linux/component.h>
+#include <linux/module.h>
+#include <linux/of_address.h>
+#include <linux/of_device.h>
+#include <linux/of_irq.h>
+#include <linux/of_graph.h>
+#include <video/mipi_display.h>
+
+#include <drm/drm_atomic_helper.h>
+#include <drm/drm_bridge.h>
+#include <drm/drm_crtc_helper.h>
+#include <drm/drm_of.h>
+#include <drm/drm_probe_helper.h>
+
+#include "sprd_drm.h"
+#include "sprd_dpu.h"
+#include "sprd_dsi.h"
+
+#define SOFT_RESET 0x04
+#define MASK_PROTOCOL_INT 0x0C
+#define MASK_INTERNAL_INT 0x14
+#define DSI_MODE_CFG 0x18
+
+#define VIRTUAL_CHANNEL_ID 0x1C
+#define GEN_RX_VCID GENMASK(1, 0)
+#define VIDEO_PKT_VCID GENMASK(3, 2)
+
+#define DPI_VIDEO_FORMAT 0x20
+#define DPI_VIDEO_MODE_FORMAT GENMASK(5, 0)
+#define LOOSELY18_EN BIT(6)
+
+#define VIDEO_PKT_CONFIG 0x24
+#define VIDEO_PKT_SIZE GENMASK(15, 0)
+#define VIDEO_LINE_CHUNK_NUM GENMASK(31, 16)
+
+#define VIDEO_LINE_HBLK_TIME 0x28
+#define VIDEO_LINE_HBP_TIME GENMASK(15, 0)
+#define VIDEO_LINE_HSA_TIME GENMASK(31, 16)
+
+#define VIDEO_LINE_TIME 0x2C
+
+#define VIDEO_VBLK_LINES 0x30
+#define VFP_LINES GENMASK(9, 0)
+#define VBP_LINES GENMASK(19, 10)
+#define VSA_LINES GENMASK(29, 20)
+
+#define VIDEO_VACTIVE_LINES 0x34
+
+#define VID_MODE_CFG 0x38
+#define VID_MODE_TYPE GENMASK(1, 0)
+#define LP_VSA_EN BIT(8)
+#define LP_VBP_EN BIT(9)
+#define LP_VFP_EN BIT(10)
+#define LP_VACT_EN BIT(11)
+#define LP_HBP_EN BIT(12)
+#define LP_HFP_EN BIT(13)
+#define FRAME_BTA_ACK_EN BIT(14)
+
+#define TIMEOUT_CNT_CLK_CONFIG 0x40
+#define HTX_TO_CONFIG 0x44
+#define LRX_H_TO_CONFIG 0x48
+
+#define TX_ESC_CLK_CONFIG 0x5C
+
+#define CMD_MODE_CFG 0x68
+#define TEAR_FX_EN BIT(0)
+
+#define GEN_HDR 0x6C
+#define GEN_DT GENMASK(5, 0)
+#define GEN_VC GENMASK(7, 6)
+
+#define GEN_PLD_DATA 0x70
+
+#define PHY_CLK_LANE_LP_CTRL 0x74
+#define PHY_CLKLANE_TX_REQ_HS BIT(0)
+#define AUTO_CLKLANE_CTRL_EN BIT(1)
+
+#define PHY_INTERFACE_CTRL 0x78
+#define RF_PHY_SHUTDOWN BIT(0)
+#define RF_PHY_RESET_N BIT(1)
+#define RF_PHY_CLK_EN BIT(2)
+
+#define CMD_MODE_STATUS 0x98
+#define GEN_CMD_RDATA_FIFO_EMPTY BIT(1)
+#define GEN_CMD_WDATA_FIFO_EMPTY BIT(3)
+#define GEN_CMD_CMD_FIFO_EMPTY BIT(5)
+#define GEN_CMD_RDCMD_DONE BIT(7)
+
+#define PHY_STATUS 0x9C
+#define PHY_LOCK BIT(1)
+
+#define PHY_MIN_STOP_TIME 0xA0
+#define PHY_LANE_NUM_CONFIG 0xA4
+
+#define PHY_CLKLANE_TIME_CONFIG 0xA8
+#define PHY_CLKLANE_LP_TO_HS_TIME GENMASK(15, 0)
+#define PHY_CLKLANE_HS_TO_LP_TIME GENMASK(31, 16)
+
+#define PHY_DATALANE_TIME_CONFIG 0xAC
+#define PHY_DATALANE_LP_TO_HS_TIME GENMASK(15, 0)
+#define PHY_DATALANE_HS_TO_LP_TIME GENMASK(31, 16)
+
+#define MAX_READ_TIME 0xB0
+
+#define RX_PKT_CHECK_CONFIG 0xB4
+#define RX_PKT_ECC_EN BIT(0)
+#define RX_PKT_CRC_EN BIT(1)
+
+#define TA_EN 0xB8
+
+#define EOTP_EN 0xBC
+#define TX_EOTP_EN BIT(0)
+#define RX_EOTP_EN BIT(1)
+
+#define VIDEO_NULLPKT_SIZE 0xC0
+#define DCS_WM_PKT_SIZE 0xC4
+
+#define VIDEO_SIG_DELAY_CONFIG 0xD0
+#define VIDEO_SIG_DELAY GENMASK(23, 0)
+
+#define PHY_TST_CTRL0 0xF0
+#define PHY_TESTCLR BIT(0)
+#define PHY_TESTCLK BIT(1)
+
+#define PHY_TST_CTRL1 0xF4
+#define PHY_TESTDIN GENMASK(7, 0)
+#define PHY_TESTDOUT GENMASK(15, 8)
+#define PHY_TESTEN BIT(16)
+
+#define host_to_dsi(host) \
+ container_of(host, struct sprd_dsi, host)
+
+static inline u32
+dsi_reg_rd(struct dsi_context *ctx, u32 offset, u32 mask,
+ u32 shift)
+{
+ return (readl(ctx->base + offset) & mask) >> shift;
+}
+
+static inline void
+dsi_reg_wr(struct dsi_context *ctx, u32 offset, u32 mask,
+ u32 shift, u32 val)
+{
+ u32 ret;
+
+ ret = readl(ctx->base + offset);
+ ret &= ~mask;
+ ret |= (val << shift) & mask;
+ writel(ret, ctx->base + offset);
+}
+
+static inline void
+dsi_reg_up(struct dsi_context *ctx, u32 offset, u32 mask,
+ u32 val)
+{
+ u32 ret = readl(ctx->base + offset);
+
+ writel((ret & ~mask) | (val & mask), ctx->base + offset);
+}
+
+static int regmap_tst_io_write(void *context, u32 reg, u32 val)
+{
+ struct sprd_dsi *dsi = context;
+ struct dsi_context *ctx = &dsi->ctx;
+
+ if (val > 0xff || reg > 0xff)
+ return -EINVAL;
+
+ drm_dbg(dsi->drm, "reg = 0x%02x, val = 0x%02x\n", reg, val);
+
+ dsi_reg_up(ctx, PHY_TST_CTRL1, PHY_TESTEN, PHY_TESTEN);
+ dsi_reg_wr(ctx, PHY_TST_CTRL1, PHY_TESTDIN, 0, reg);
+ dsi_reg_up(ctx, PHY_TST_CTRL0, PHY_TESTCLK, PHY_TESTCLK);
+ dsi_reg_up(ctx, PHY_TST_CTRL0, PHY_TESTCLK, 0);
+ dsi_reg_up(ctx, PHY_TST_CTRL1, PHY_TESTEN, 0);
+ dsi_reg_wr(ctx, PHY_TST_CTRL1, PHY_TESTDIN, 0, val);
+ dsi_reg_up(ctx, PHY_TST_CTRL0, PHY_TESTCLK, PHY_TESTCLK);
+ dsi_reg_up(ctx, PHY_TST_CTRL0, PHY_TESTCLK, 0);
+
+ return 0;
+}
+
+static int regmap_tst_io_read(void *context, u32 reg, u32 *val)
+{
+ struct sprd_dsi *dsi = context;
+ struct dsi_context *ctx = &dsi->ctx;
+ int ret;
+
+ if (reg > 0xff)
+ return -EINVAL;
+
+ dsi_reg_up(ctx, PHY_TST_CTRL1, PHY_TESTEN, PHY_TESTEN);
+ dsi_reg_wr(ctx, PHY_TST_CTRL1, PHY_TESTDIN, 0, reg);
+ dsi_reg_up(ctx, PHY_TST_CTRL0, PHY_TESTCLK, PHY_TESTCLK);
+ dsi_reg_up(ctx, PHY_TST_CTRL0, PHY_TESTCLK, 0);
+ dsi_reg_up(ctx, PHY_TST_CTRL1, PHY_TESTEN, 0);
+
+ udelay(1);
+
+ ret = dsi_reg_rd(ctx, PHY_TST_CTRL1, PHY_TESTDOUT, 8);
+ if (ret < 0)
+ return ret;
+
+ *val = ret;
+
+ drm_dbg(dsi->drm, "reg = 0x%02x, val = 0x%02x\n", reg, *val);
+ return 0;
+}
+
+static struct regmap_bus regmap_tst_io = {
+ .reg_write = regmap_tst_io_write,
+ .reg_read = regmap_tst_io_read,
+};
+
+static const struct regmap_config byte_config = {
+ .reg_bits = 8,
+ .val_bits = 8,
+};
+
+static int dphy_wait_pll_locked(struct dsi_context *ctx)
+{
+ struct sprd_dsi *dsi = container_of(ctx, struct sprd_dsi, ctx);
+ int i;
+
+ for (i = 0; i < 50000; i++) {
+ if (dsi_reg_rd(ctx, PHY_STATUS, PHY_LOCK, 1))
+ return 0;
+ udelay(3);
+ }
+
+ drm_err(dsi->drm, "dphy pll can not be locked\n");
+ return -ETIMEDOUT;
+}
+
+static int dsi_wait_tx_payload_fifo_empty(struct dsi_context *ctx)
+{
+ int i;
+
+ for (i = 0; i < 5000; i++) {
+ if (dsi_reg_rd(ctx, CMD_MODE_STATUS, GEN_CMD_WDATA_FIFO_EMPTY, 3))
+ return 0;
+ udelay(1);
+ }
+
+ return -ETIMEDOUT;
+}
+
+static int dsi_wait_tx_cmd_fifo_empty(struct dsi_context *ctx)
+{
+ int i;
+
+ for (i = 0; i < 5000; i++) {
+ if (dsi_reg_rd(ctx, CMD_MODE_STATUS, GEN_CMD_CMD_FIFO_EMPTY, 5))
+ return 0;
+ udelay(1);
+ }
+
+ return -ETIMEDOUT;
+}
+
+static int dsi_wait_rd_resp_completed(struct dsi_context *ctx)
+{
+ int i;
+
+ for (i = 0; i < 10000; i++) {
+ if (dsi_reg_rd(ctx, CMD_MODE_STATUS, GEN_CMD_RDCMD_DONE, 7))
+ return 0;
+ udelay(10);
+ }
+
+ return -ETIMEDOUT;
+}
+
+static u16 calc_bytes_per_pixel_x100(int coding)
+{
+ u16 bpp_x100;
+
+ switch (coding) {
+ case COLOR_CODE_16BIT_CONFIG1:
+ case COLOR_CODE_16BIT_CONFIG2:
+ case COLOR_CODE_16BIT_CONFIG3:
+ bpp_x100 = 200;
+ break;
+ case COLOR_CODE_18BIT_CONFIG1:
+ case COLOR_CODE_18BIT_CONFIG2:
+ bpp_x100 = 225;
+ break;
+ case COLOR_CODE_24BIT:
+ bpp_x100 = 300;
+ break;
+ case COLOR_CODE_COMPRESSTION:
+ bpp_x100 = 100;
+ break;
+ case COLOR_CODE_20BIT_YCC422_LOOSELY:
+ bpp_x100 = 250;
+ break;
+ case COLOR_CODE_24BIT_YCC422:
+ bpp_x100 = 300;
+ break;
+ case COLOR_CODE_16BIT_YCC422:
+ bpp_x100 = 200;
+ break;
+ case COLOR_CODE_30BIT:
+ bpp_x100 = 375;
+ break;
+ case COLOR_CODE_36BIT:
+ bpp_x100 = 450;
+ break;
+ case COLOR_CODE_12BIT_YCC420:
+ bpp_x100 = 150;
+ break;
+ default:
+ DRM_ERROR("invalid color coding");
+ bpp_x100 = 0;
+ break;
+ }
+
+ return bpp_x100;
+}
+
+static u8 calc_video_size_step(int coding)
+{
+ u8 video_size_step;
+
+ switch (coding) {
+ case COLOR_CODE_16BIT_CONFIG1:
+ case COLOR_CODE_16BIT_CONFIG2:
+ case COLOR_CODE_16BIT_CONFIG3:
+ case COLOR_CODE_18BIT_CONFIG1:
+ case COLOR_CODE_18BIT_CONFIG2:
+ case COLOR_CODE_24BIT:
+ case COLOR_CODE_COMPRESSTION:
+ return video_size_step = 1;
+ case COLOR_CODE_20BIT_YCC422_LOOSELY:
+ case COLOR_CODE_24BIT_YCC422:
+ case COLOR_CODE_16BIT_YCC422:
+ case COLOR_CODE_30BIT:
+ case COLOR_CODE_36BIT:
+ case COLOR_CODE_12BIT_YCC420:
+ return video_size_step = 2;
+ default:
+ DRM_ERROR("invalid color coding");
+ return 0;
+ }
+}
+
+static u16 round_video_size(int coding, u16 video_size)
+{
+ switch (coding) {
+ case COLOR_CODE_16BIT_YCC422:
+ case COLOR_CODE_24BIT_YCC422:
+ case COLOR_CODE_20BIT_YCC422_LOOSELY:
+ case COLOR_CODE_12BIT_YCC420:
+ /* round up active H pixels to a multiple of 2 */
+ if ((video_size % 2) != 0)
+ video_size += 1;
+ break;
+ default:
+ break;
+ }
+
+ return video_size;
+}
+
+#define SPRD_MIPI_DSI_FMT_DSC 0xff
+static u32 fmt_to_coding(u32 fmt)
+{
+ switch (fmt) {
+ case MIPI_DSI_FMT_RGB565:
+ return COLOR_CODE_16BIT_CONFIG1;
+ case MIPI_DSI_FMT_RGB666:
+ case MIPI_DSI_FMT_RGB666_PACKED:
+ return COLOR_CODE_18BIT_CONFIG1;
+ case MIPI_DSI_FMT_RGB888:
+ return COLOR_CODE_24BIT;
+ case SPRD_MIPI_DSI_FMT_DSC:
+ return COLOR_CODE_COMPRESSTION;
+ default:
+ DRM_ERROR("Unsupported format (%d)\n", fmt);
+ return COLOR_CODE_24BIT;
+ }
+}
+
+#define ns_to_cycle(ns, byte_clk) \
+ DIV_ROUND_UP((ns) * (byte_clk), 1000000)
+
+static void sprd_dsi_init(struct dsi_context *ctx)
+{
+ struct sprd_dsi *dsi = container_of(ctx, struct sprd_dsi, ctx);
+ u32 byte_clk = dsi->slave->hs_rate / 8;
+ u16 data_hs2lp, data_lp2hs, clk_hs2lp, clk_lp2hs;
+ u16 max_rd_time;
+ int div;
+
+ writel(0, ctx->base + SOFT_RESET);
+ writel(0xffffffff, ctx->base + MASK_PROTOCOL_INT);
+ writel(0xffffffff, ctx->base + MASK_INTERNAL_INT);
+ writel(1, ctx->base + DSI_MODE_CFG);
+ dsi_reg_up(ctx, EOTP_EN, RX_EOTP_EN, 0);
+ dsi_reg_up(ctx, EOTP_EN, TX_EOTP_EN, 0);
+ dsi_reg_up(ctx, RX_PKT_CHECK_CONFIG, RX_PKT_ECC_EN, RX_PKT_ECC_EN);
+ dsi_reg_up(ctx, RX_PKT_CHECK_CONFIG, RX_PKT_CRC_EN, RX_PKT_CRC_EN);
+ writel(1, ctx->base + TA_EN);
+ dsi_reg_up(ctx, VIRTUAL_CHANNEL_ID, VIDEO_PKT_VCID, 0);
+ dsi_reg_up(ctx, VIRTUAL_CHANNEL_ID, GEN_RX_VCID, 0);
+
+ div = DIV_ROUND_UP(byte_clk, dsi->slave->lp_rate);
+ writel(div, ctx->base + TX_ESC_CLK_CONFIG);
+
+ max_rd_time = ns_to_cycle(ctx->max_rd_time, byte_clk);
+ writel(max_rd_time, ctx->base + MAX_READ_TIME);
+
+ data_hs2lp = ns_to_cycle(ctx->data_hs2lp, byte_clk);
+ data_lp2hs = ns_to_cycle(ctx->data_lp2hs, byte_clk);
+ clk_hs2lp = ns_to_cycle(ctx->clk_hs2lp, byte_clk);
+ clk_lp2hs = ns_to_cycle(ctx->clk_lp2hs, byte_clk);
+ dsi_reg_wr(ctx, PHY_DATALANE_TIME_CONFIG,
+ PHY_DATALANE_HS_TO_LP_TIME, 16, data_hs2lp);
+ dsi_reg_wr(ctx, PHY_DATALANE_TIME_CONFIG,
+ PHY_DATALANE_LP_TO_HS_TIME, 0, data_lp2hs);
+ dsi_reg_wr(ctx, PHY_CLKLANE_TIME_CONFIG,
+ PHY_CLKLANE_HS_TO_LP_TIME, 16, clk_hs2lp);
+ dsi_reg_wr(ctx, PHY_CLKLANE_TIME_CONFIG,
+ PHY_CLKLANE_LP_TO_HS_TIME, 0, clk_lp2hs);
+
+ writel(1, ctx->base + SOFT_RESET);
+}
+
+/*
+ * Free up resources and shutdown host controller and PHY
+ */
+static void sprd_dsi_fini(struct dsi_context *ctx)
+{
+ writel(0xffffffff, ctx->base + MASK_PROTOCOL_INT);
+ writel(0xffffffff, ctx->base + MASK_INTERNAL_INT);
+ writel(0, ctx->base + SOFT_RESET);
+}
+
+/*
+ * If not in burst mode, it will compute the video and null packet sizes
+ * according to necessity.
+ * Configure timers for data lanes and/or clock lane to return to LP when
+ * bandwidth is not filled by data.
+ */
+static int sprd_dsi_dpi_video(struct dsi_context *ctx)
+{
+ struct sprd_dsi *dsi = container_of(ctx, struct sprd_dsi, ctx);
+ struct videomode *vm = &ctx->vm;
+ u32 byte_clk = dsi->slave->hs_rate / 8;
+ u16 bpp_x100;
+ u16 video_size;
+ u32 ratio_x1000;
+ u16 null_pkt_size = 0;
+ u8 video_size_step;
+ u32 hs_to;
+ u32 total_bytes;
+ u32 bytes_per_chunk;
+ u32 chunks = 0;
+ u32 bytes_left = 0;
+ u32 chunk_overhead;
+ const u8 pkt_header = 6;
+ u8 coding;
+ int div;
+ u16 hline;
+ u16 byte_cycle;
+
+ coding = fmt_to_coding(dsi->slave->format);
+ video_size = round_video_size(coding, vm->hactive);
+ bpp_x100 = calc_bytes_per_pixel_x100(coding);
+ video_size_step = calc_video_size_step(coding);
+ ratio_x1000 = byte_clk * 1000 / (vm->pixelclock / 1000);
+ hline = vm->hactive + vm->hsync_len + vm->hfront_porch +
+ vm->hback_porch;
+
+ writel(0, ctx->base + SOFT_RESET);
+ dsi_reg_wr(ctx, VID_MODE_CFG, FRAME_BTA_ACK_EN, 15, ctx->frame_ack_en);
+ dsi_reg_wr(ctx, DPI_VIDEO_FORMAT, DPI_VIDEO_MODE_FORMAT, 0, coding);
+ dsi_reg_wr(ctx, VID_MODE_CFG, VID_MODE_TYPE, 0, ctx->burst_mode);
+ byte_cycle = 95 * hline * ratio_x1000 / 100000;
+ dsi_reg_wr(ctx, VIDEO_SIG_DELAY_CONFIG, VIDEO_SIG_DELAY, 0, byte_cycle);
+ byte_cycle = hline * ratio_x1000 / 1000;
+ writel(byte_cycle, ctx->base + VIDEO_LINE_TIME);
+ byte_cycle = vm->hsync_len * ratio_x1000 / 1000;
+ dsi_reg_wr(ctx, VIDEO_LINE_HBLK_TIME, VIDEO_LINE_HSA_TIME, 16, byte_cycle);
+ byte_cycle = vm->hback_porch * ratio_x1000 / 1000;
+ dsi_reg_wr(ctx, VIDEO_LINE_HBLK_TIME, VIDEO_LINE_HBP_TIME, 0, byte_cycle);
+ writel(vm->vactive, ctx->base + VIDEO_VACTIVE_LINES);
+ dsi_reg_wr(ctx, VIDEO_VBLK_LINES, VFP_LINES, 0, vm->vfront_porch);
+ dsi_reg_wr(ctx, VIDEO_VBLK_LINES, VBP_LINES, 10, vm->vback_porch);
+ dsi_reg_wr(ctx, VIDEO_VBLK_LINES, VSA_LINES, 20, vm->vsync_len);
+ dsi_reg_up(ctx, VID_MODE_CFG, LP_HBP_EN | LP_HFP_EN | LP_VACT_EN |
+ LP_VFP_EN | LP_VBP_EN | LP_VSA_EN, LP_HBP_EN | LP_HFP_EN |
+ LP_VACT_EN | LP_VFP_EN | LP_VBP_EN | LP_VSA_EN);
+
+ hs_to = (hline * vm->vactive) + (2 * bpp_x100) / 100;
+ for (div = 0x80; (div < hs_to) && (div > 2); div--) {
+ if ((hs_to % div) == 0) {
+ writel(div, ctx->base + TIMEOUT_CNT_CLK_CONFIG);
+ writel(hs_to / div, ctx->base + LRX_H_TO_CONFIG);
+ writel(hs_to / div, ctx->base + HTX_TO_CONFIG);
+ break;
+ }
+ }
+
+ if (ctx->burst_mode == VIDEO_BURST_WITH_SYNC_PULSES) {
+ dsi_reg_wr(ctx, VIDEO_PKT_CONFIG, VIDEO_PKT_SIZE, 0, video_size);
+ writel(0, ctx->base + VIDEO_NULLPKT_SIZE);
+ dsi_reg_up(ctx, VIDEO_PKT_CONFIG, VIDEO_LINE_CHUNK_NUM, 0);
+ } else {
+ /* non burst transmission */
+ null_pkt_size = 0;
+
+ /* bytes to be sent - first as one chunk */
+ bytes_per_chunk = vm->hactive * bpp_x100 / 100 + pkt_header;
+
+ /* hline total bytes from the DPI interface */
+ total_bytes = (vm->hactive + vm->hfront_porch) *
+ ratio_x1000 / dsi->slave->lanes / 1000;
+
+ /* check if the pixels actually fit on the DSI link */
+ if (total_bytes < bytes_per_chunk) {
+ drm_err(dsi->drm, "current resolution can not be set\n");
+ return -EINVAL;
+ }
+
+ chunk_overhead = total_bytes - bytes_per_chunk;
+
+ /* overhead higher than 1 -> enable multi packets */
+ if (chunk_overhead > 1) {
+ /* multi packets */
+ for (video_size = video_size_step;
+ video_size < vm->hactive;
+ video_size += video_size_step) {
+ if (vm->hactive * 1000 / video_size % 1000)
+ continue;
+
+ chunks = vm->hactive / video_size;
+ bytes_per_chunk = bpp_x100 * video_size / 100
+ + pkt_header;
+ if (total_bytes >= (bytes_per_chunk * chunks)) {
+ bytes_left = total_bytes -
+ bytes_per_chunk * chunks;
+ break;
+ }
+ }
+
+ /* prevent overflow (unsigned - unsigned) */
+ if (bytes_left > (pkt_header * chunks)) {
+ null_pkt_size = (bytes_left -
+ pkt_header * chunks) / chunks;
+ /* avoid register overflow */
+ if (null_pkt_size > 1023)
+ null_pkt_size = 1023;
+ }
+
+ } else {
+ /* single packet */
+ chunks = 1;
+
+ /* must be a multiple of 4 except 18 loosely */
+ for (video_size = vm->hactive;
+ (video_size % video_size_step) != 0;
+ video_size++)
+ ;
+ }
+
+ dsi_reg_wr(ctx, VIDEO_PKT_CONFIG, VIDEO_PKT_SIZE, 0, video_size);
+ writel(null_pkt_size, ctx->base + VIDEO_NULLPKT_SIZE);
+ dsi_reg_wr(ctx, VIDEO_PKT_CONFIG, VIDEO_LINE_CHUNK_NUM, 16, chunks);
+ }
+
+ writel(ctx->int0_mask, ctx->base + MASK_PROTOCOL_INT);
+ writel(ctx->int1_mask, ctx->base + MASK_INTERNAL_INT);
+ writel(1, ctx->base + SOFT_RESET);
+
+ return 0;
+}
+
+static void sprd_dsi_edpi_video(struct dsi_context *ctx)
+{
+ struct sprd_dsi *dsi = container_of(ctx, struct sprd_dsi, ctx);
+ const u32 fifo_depth = 1096;
+ const u32 word_length = 4;
+ u32 hactive = ctx->vm.hactive;
+ u32 bpp_x100;
+ u32 max_fifo_len;
+ u8 coding;
+
+ coding = fmt_to_coding(dsi->slave->format);
+ bpp_x100 = calc_bytes_per_pixel_x100(coding);
+ max_fifo_len = word_length * fifo_depth * 100 / bpp_x100;
+
+ writel(0, ctx->base + SOFT_RESET);
+ dsi_reg_wr(ctx, DPI_VIDEO_FORMAT, DPI_VIDEO_MODE_FORMAT, 0, coding);
+ dsi_reg_wr(ctx, CMD_MODE_CFG, TEAR_FX_EN, 0, ctx->te_ack_en);
+
+ if (max_fifo_len > hactive)
+ writel(hactive, ctx->base + DCS_WM_PKT_SIZE);
+ else
+ writel(max_fifo_len, ctx->base + DCS_WM_PKT_SIZE);
+
+ writel(ctx->int0_mask, ctx->base + MASK_PROTOCOL_INT);
+ writel(ctx->int1_mask, ctx->base + MASK_INTERNAL_INT);
+ writel(1, ctx->base + SOFT_RESET);
+}
+
+/*
+ * Send a packet on the generic interface,
+ * this function has an active delay to wait for the buffer to clear.
+ * The delay is limited to:
+ * (param_length / 4) x DSIH_FIFO_ACTIVE_WAIT x register access time
+ * the controller restricts the sending of.
+ *
+ * This function will not be able to send Null and Blanking packets due to
+ * controller restriction
+ */
+static int sprd_dsi_wr_pkt(struct dsi_context *ctx, u8 vc, u8 type,
+ const u8 *param, u16 len)
+{
+ struct sprd_dsi *dsi = container_of(ctx, struct sprd_dsi, ctx);
+ u8 wc_lsbyte, wc_msbyte;
+ u32 payload;
+ int i, j, ret;
+
+ if (vc > 3)
+ return -EINVAL;
+
+ /* 1st: for long packet, must config payload first */
+ ret = dsi_wait_tx_payload_fifo_empty(ctx);
+ if (ret) {
+ drm_err(dsi->drm, "tx payload fifo is not empty\n");
+ return ret;
+ }
+
+ if (len > 2) {
+ for (i = 0, j = 0; i < len; i += j) {
+ payload = 0;
+ for (j = 0; (j < 4) && ((j + i) < (len)); j++)
+ payload |= param[i + j] << (j * 8);
+
+ writel(payload, ctx->base + GEN_PLD_DATA);
+ }
+ wc_lsbyte = len & 0xff;
+ wc_msbyte = len >> 8;
+ } else {
+ wc_lsbyte = (len > 0) ? param[0] : 0;
+ wc_msbyte = (len > 1) ? param[1] : 0;
+ }
+
+ /* 2nd: then set packet header */
+ ret = dsi_wait_tx_cmd_fifo_empty(ctx);
+ if (ret) {
+ drm_err(dsi->drm, "tx cmd fifo is not empty\n");
+ return ret;
+ }
+
+ writel(type | (vc << 6) | (wc_lsbyte << 8) | (wc_msbyte << 16),
+ ctx->base + GEN_HDR);
+
+ return 0;
+}
+
+/*
+ * Send READ packet to peripheral using the generic interface,
+ * this will force command mode and stop video mode (because of BTA).
+ *
+ * This function has an active delay to wait for the buffer to clear,
+ * the delay is limited to 2 x DSIH_FIFO_ACTIVE_WAIT
+ * (waiting for command buffer, and waiting for receiving)
+ * @note this function will enable BTA
+ */
+static int sprd_dsi_rd_pkt(struct dsi_context *ctx, u8 vc, u8 type,
+ u8 msb_byte, u8 lsb_byte,
+ u8 *buffer, u8 bytes_to_read)
+{
+ struct sprd_dsi *dsi = container_of(ctx, struct sprd_dsi, ctx);
+ int i, ret;
+ int count = 0;
+ u32 temp;
+
+ if (vc > 3)
+ return -EINVAL;
+
+ /* 1st: send read command to peripheral */
+ ret = dsi_reg_rd(ctx, CMD_MODE_STATUS, GEN_CMD_CMD_FIFO_EMPTY, 5);
+ if (!ret)
+ return -EIO;
+
+ writel(type | (vc << 6) | (lsb_byte << 8) | (msb_byte << 16),
+ ctx->base + GEN_HDR);
+
+ /* 2nd: wait peripheral response completed */
+ ret = dsi_wait_rd_resp_completed(ctx);
+ if (ret) {
+ drm_err(dsi->drm, "wait read response time out\n");
+ return ret;
+ }
+
+ /* 3rd: get data from rx payload fifo */
+ ret = dsi_reg_rd(ctx, CMD_MODE_STATUS, GEN_CMD_RDATA_FIFO_EMPTY, 1);
+ if (ret) {
+ drm_err(dsi->drm, "rx payload fifo empty\n");
+ return -EIO;
+ }
+
+ for (i = 0; i < 100; i++) {
+ temp = readl(ctx->base + GEN_PLD_DATA);
+
+ if (count < bytes_to_read)
+ buffer[count++] = temp & 0xff;
+ if (count < bytes_to_read)
+ buffer[count++] = (temp >> 8) & 0xff;
+ if (count < bytes_to_read)
+ buffer[count++] = (temp >> 16) & 0xff;
+ if (count < bytes_to_read)
+ buffer[count++] = (temp >> 24) & 0xff;
+
+ ret = dsi_reg_rd(ctx, CMD_MODE_STATUS, GEN_CMD_RDATA_FIFO_EMPTY, 1);
+ if (ret)
+ return count;
+ }
+
+ return 0;
+}
+
+static void sprd_dsi_set_work_mode(struct dsi_context *ctx, u8 mode)
+{
+ if (mode == DSI_MODE_CMD)
+ writel(1, ctx->base + DSI_MODE_CFG);
+ else
+ writel(0, ctx->base + DSI_MODE_CFG);
+}
+
+static void sprd_dsi_state_reset(struct dsi_context *ctx)
+{
+ writel(0, ctx->base + SOFT_RESET);
+ udelay(100);
+ writel(1, ctx->base + SOFT_RESET);
+}
+
+static int sprd_dphy_init(struct dsi_context *ctx)
+{
+ struct sprd_dsi *dsi = container_of(ctx, struct sprd_dsi, ctx);
+ int ret;
+
+ dsi_reg_up(ctx, PHY_INTERFACE_CTRL, RF_PHY_RESET_N, 0);
+ dsi_reg_up(ctx, PHY_INTERFACE_CTRL, RF_PHY_SHUTDOWN, 0);
+ dsi_reg_up(ctx, PHY_INTERFACE_CTRL, RF_PHY_CLK_EN, 0);
+
+ dsi_reg_up(ctx, PHY_TST_CTRL0, PHY_TESTCLR, 0);
+ dsi_reg_up(ctx, PHY_TST_CTRL0, PHY_TESTCLR, PHY_TESTCLR);
+ dsi_reg_up(ctx, PHY_TST_CTRL0, PHY_TESTCLR, 0);
+
+ dphy_pll_config(ctx);
+ dphy_timing_config(ctx);
+
+ dsi_reg_up(ctx, PHY_INTERFACE_CTRL, RF_PHY_SHUTDOWN, RF_PHY_SHUTDOWN);
+ dsi_reg_up(ctx, PHY_INTERFACE_CTRL, RF_PHY_RESET_N, RF_PHY_RESET_N);
+ writel(0x1C, ctx->base + PHY_MIN_STOP_TIME);
+ dsi_reg_up(ctx, PHY_INTERFACE_CTRL, RF_PHY_CLK_EN, RF_PHY_CLK_EN);
+ writel(dsi->slave->lanes - 1, ctx->base + PHY_LANE_NUM_CONFIG);
+
+ ret = dphy_wait_pll_locked(ctx);
+ if (ret) {
+ drm_err(dsi->drm, "dphy initial failed\n");
+ return ret;
+ }
+
+ return 0;
+}
+
+static void sprd_dphy_fini(struct dsi_context *ctx)
+{
+ dsi_reg_up(ctx, PHY_INTERFACE_CTRL, RF_PHY_RESET_N, 0);
+ dsi_reg_up(ctx, PHY_INTERFACE_CTRL, RF_PHY_SHUTDOWN, 0);
+ dsi_reg_up(ctx, PHY_INTERFACE_CTRL, RF_PHY_RESET_N, RF_PHY_RESET_N);
+}
+
+static void sprd_dsi_encoder_mode_set(struct drm_encoder *encoder,
+ struct drm_display_mode *mode,
+ struct drm_display_mode *adj_mode)
+{
+ struct sprd_dsi *dsi = encoder_to_dsi(encoder);
+
+ drm_display_mode_to_videomode(adj_mode, &dsi->ctx.vm);
+}
+
+static void sprd_dsi_encoder_enable(struct drm_encoder *encoder)
+{
+ struct sprd_dsi *dsi = encoder_to_dsi(encoder);
+ struct sprd_dpu *dpu = to_sprd_crtc(encoder->crtc);
+ struct dsi_context *ctx = &dsi->ctx;
+
+ if (ctx->enabled) {
+ drm_warn(dsi->drm, "dsi is initialized\n");
+ return;
+ }
+
+ sprd_dsi_init(ctx);
+ if (ctx->work_mode == DSI_MODE_VIDEO)
+ sprd_dsi_dpi_video(ctx);
+ else
+ sprd_dsi_edpi_video(ctx);
+
+ sprd_dphy_init(ctx);
+
+ sprd_dsi_set_work_mode(ctx, ctx->work_mode);
+ sprd_dsi_state_reset(ctx);
+
+ if (dsi->slave->mode_flags & MIPI_DSI_CLOCK_NON_CONTINUOUS) {
+ dsi_reg_up(ctx, PHY_CLK_LANE_LP_CTRL, AUTO_CLKLANE_CTRL_EN,
+ AUTO_CLKLANE_CTRL_EN);
+ } else {
+ dsi_reg_up(ctx, PHY_CLK_LANE_LP_CTRL, RF_PHY_CLK_EN, RF_PHY_CLK_EN);
+ dsi_reg_up(ctx, PHY_CLK_LANE_LP_CTRL, PHY_CLKLANE_TX_REQ_HS,
+ PHY_CLKLANE_TX_REQ_HS);
+ dphy_wait_pll_locked(ctx);
+ }
+
+ sprd_dpu_run(dpu);
+
+ ctx->enabled = true;
+}
+
+static void sprd_dsi_encoder_disable(struct drm_encoder *encoder)
+{
+ struct sprd_dsi *dsi = encoder_to_dsi(encoder);
+ struct sprd_dpu *dpu = to_sprd_crtc(encoder->crtc);
+ struct dsi_context *ctx = &dsi->ctx;
+
+ if (!ctx->enabled) {
+ drm_warn(dsi->drm, "dsi isn't initialized\n");
+ return;
+ }
+
+ sprd_dpu_stop(dpu);
+ sprd_dphy_fini(ctx);
+ sprd_dsi_fini(ctx);
+
+ ctx->enabled = false;
+}
+
+static const struct drm_encoder_helper_funcs sprd_encoder_helper_funcs = {
+ .mode_set = sprd_dsi_encoder_mode_set,
+ .enable = sprd_dsi_encoder_enable,
+ .disable = sprd_dsi_encoder_disable
+};
+
+static const struct drm_encoder_funcs sprd_encoder_funcs = {
+ .destroy = drm_encoder_cleanup,
+};
+
+static int sprd_dsi_encoder_init(struct sprd_dsi *dsi,
+ struct device *dev)
+{
+ struct drm_encoder *encoder = &dsi->encoder;
+ u32 crtc_mask;
+ int ret;
+
+ crtc_mask = drm_of_find_possible_crtcs(dsi->drm, dev->of_node);
+ if (!crtc_mask) {
+ drm_err(dsi->drm, "failed to find crtc mask\n");
+ return -EINVAL;
+ }
+
+ drm_dbg(dsi->drm, "find possible crtcs: 0x%08x\n", crtc_mask);
+
+ encoder->possible_crtcs = crtc_mask;
+ ret = drm_encoder_init(dsi->drm, encoder, &sprd_encoder_funcs,
+ DRM_MODE_ENCODER_DSI, NULL);
+ if (ret) {
+ drm_err(dsi->drm, "failed to init dsi encoder\n");
+ return ret;
+ }
+
+ drm_encoder_helper_add(encoder, &sprd_encoder_helper_funcs);
+
+ return 0;
+}
+
+static int sprd_dsi_bridge_init(struct sprd_dsi *dsi,
+ struct device *dev)
+{
+ int ret;
+
+ dsi->panel_bridge = devm_drm_of_get_bridge(dev, dev->of_node, 1, 0);
+ if (IS_ERR(dsi->panel_bridge))
+ return PTR_ERR(dsi->panel_bridge);
+
+ ret = drm_bridge_attach(&dsi->encoder, dsi->panel_bridge, NULL, 0);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+static int sprd_dsi_context_init(struct sprd_dsi *dsi,
+ struct device *dev)
+{
+ struct platform_device *pdev = to_platform_device(dev);
+ struct dsi_context *ctx = &dsi->ctx;
+ struct resource *res;
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ ctx->base = devm_ioremap(dev, res->start, resource_size(res));
+ if (!ctx->base) {
+ drm_err(dsi->drm, "failed to map dsi host registers\n");
+ return -ENXIO;
+ }
+
+ ctx->regmap = devm_regmap_init(dev, ®map_tst_io, dsi, &byte_config);
+ if (IS_ERR(ctx->regmap)) {
+ drm_err(dsi->drm, "dphy regmap init failed\n");
+ return PTR_ERR(ctx->regmap);
+ }
+
+ ctx->data_hs2lp = 120;
+ ctx->data_lp2hs = 500;
+ ctx->clk_hs2lp = 4;
+ ctx->clk_lp2hs = 15;
+ ctx->max_rd_time = 6000;
+ ctx->int0_mask = 0xffffffff;
+ ctx->int1_mask = 0xffffffff;
+ ctx->enabled = true;
+
+ return 0;
+}
+
+static int sprd_dsi_bind(struct device *dev, struct device *master, void *data)
+{
+ struct drm_device *drm = data;
+ struct sprd_dsi *dsi = dev_get_drvdata(dev);
+ int ret;
+
+ dsi->drm = drm;
+
+ ret = sprd_dsi_encoder_init(dsi, dev);
+ if (ret)
+ return ret;
+
+ ret = sprd_dsi_bridge_init(dsi, dev);
+ if (ret)
+ return ret;
+
+ ret = sprd_dsi_context_init(dsi, dev);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+static void sprd_dsi_unbind(struct device *dev,
+ struct device *master, void *data)
+{
+ struct sprd_dsi *dsi = dev_get_drvdata(dev);
+
+ drm_of_panel_bridge_remove(dev->of_node, 1, 0);
+
+ drm_encoder_cleanup(&dsi->encoder);
+}
+
+static const struct component_ops dsi_component_ops = {
+ .bind = sprd_dsi_bind,
+ .unbind = sprd_dsi_unbind,
+};
+
+static int sprd_dsi_host_attach(struct mipi_dsi_host *host,
+ struct mipi_dsi_device *slave)
+{
+ struct sprd_dsi *dsi = host_to_dsi(host);
+ struct dsi_context *ctx = &dsi->ctx;
+
+ dsi->slave = slave;
+
+ if (slave->mode_flags & MIPI_DSI_MODE_VIDEO)
+ ctx->work_mode = DSI_MODE_VIDEO;
+ else
+ ctx->work_mode = DSI_MODE_CMD;
+
+ if (slave->mode_flags & MIPI_DSI_MODE_VIDEO_BURST)
+ ctx->burst_mode = VIDEO_BURST_WITH_SYNC_PULSES;
+ else if (slave->mode_flags & MIPI_DSI_MODE_VIDEO_SYNC_PULSE)
+ ctx->burst_mode = VIDEO_NON_BURST_WITH_SYNC_PULSES;
+ else
+ ctx->burst_mode = VIDEO_NON_BURST_WITH_SYNC_EVENTS;
+
+ return component_add(host->dev, &dsi_component_ops);
+}
+
+static int sprd_dsi_host_detach(struct mipi_dsi_host *host,
+ struct mipi_dsi_device *slave)
+{
+ component_del(host->dev, &dsi_component_ops);
+
+ return 0;
+}
+
+static ssize_t sprd_dsi_host_transfer(struct mipi_dsi_host *host,
+ const struct mipi_dsi_msg *msg)
+{
+ struct sprd_dsi *dsi = host_to_dsi(host);
+ const u8 *tx_buf = msg->tx_buf;
+
+ if (msg->rx_buf && msg->rx_len) {
+ u8 lsb = (msg->tx_len > 0) ? tx_buf[0] : 0;
+ u8 msb = (msg->tx_len > 1) ? tx_buf[1] : 0;
+
+ return sprd_dsi_rd_pkt(&dsi->ctx, msg->channel, msg->type,
+ msb, lsb, msg->rx_buf, msg->rx_len);
+ }
+
+ if (msg->tx_buf && msg->tx_len)
+ return sprd_dsi_wr_pkt(&dsi->ctx, msg->channel, msg->type,
+ tx_buf, msg->tx_len);
+
+ return 0;
+}
+
+static const struct mipi_dsi_host_ops sprd_dsi_host_ops = {
+ .attach = sprd_dsi_host_attach,
+ .detach = sprd_dsi_host_detach,
+ .transfer = sprd_dsi_host_transfer,
+};
+
+static const struct of_device_id dsi_match_table[] = {
+ { .compatible = "sprd,sharkl3-dsi-host" },
+ { /* sentinel */ },
+};
+
+static int sprd_dsi_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct sprd_dsi *dsi;
+
+ dsi = devm_kzalloc(dev, sizeof(*dsi), GFP_KERNEL);
+ if (!dsi)
+ return -ENOMEM;
+
+ dev_set_drvdata(dev, dsi);
+
+ dsi->host.ops = &sprd_dsi_host_ops;
+ dsi->host.dev = dev;
+
+ return mipi_dsi_host_register(&dsi->host);
+}
+
+static int sprd_dsi_remove(struct platform_device *pdev)
+{
+ struct sprd_dsi *dsi = dev_get_drvdata(&pdev->dev);
+
+ mipi_dsi_host_unregister(&dsi->host);
+
+ return 0;
+}
+
+struct platform_driver sprd_dsi_driver = {
+ .probe = sprd_dsi_probe,
+ .remove = sprd_dsi_remove,
+ .driver = {
+ .name = "sprd-dsi-drv",
+ .of_match_table = dsi_match_table,
+ },
+};
+
+MODULE_AUTHOR("Leon He <leon.he@unisoc.com>");
+MODULE_AUTHOR("Kevin Tang <kevin.tang@unisoc.com>");
+MODULE_DESCRIPTION("Unisoc MIPI DSI HOST Controller Driver");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/gpu/drm/sprd/sprd_dsi.h b/drivers/gpu/drm/sprd/sprd_dsi.h
new file mode 100644
index 0000000..d858ebb
--- /dev/null
+++ b/drivers/gpu/drm/sprd/sprd_dsi.h
@@ -0,0 +1,126 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020 Unisoc Inc.
+ */
+
+#ifndef __SPRD_DSI_H__
+#define __SPRD_DSI_H__
+
+#include <linux/of.h>
+#include <linux/device.h>
+#include <linux/regmap.h>
+#include <video/videomode.h>
+
+#include <drm/drm_bridge.h>
+#include <drm/drm_connector.h>
+#include <drm/drm_encoder.h>
+#include <drm/drm_mipi_dsi.h>
+#include <drm/drm_print.h>
+#include <drm/drm_panel.h>
+
+#define encoder_to_dsi(encoder) \
+ container_of(encoder, struct sprd_dsi, encoder)
+
+enum dsi_work_mode {
+ DSI_MODE_CMD = 0,
+ DSI_MODE_VIDEO
+};
+
+enum video_burst_mode {
+ VIDEO_NON_BURST_WITH_SYNC_PULSES = 0,
+ VIDEO_NON_BURST_WITH_SYNC_EVENTS,
+ VIDEO_BURST_WITH_SYNC_PULSES
+};
+
+enum dsi_color_coding {
+ COLOR_CODE_16BIT_CONFIG1 = 0,
+ COLOR_CODE_16BIT_CONFIG2,
+ COLOR_CODE_16BIT_CONFIG3,
+ COLOR_CODE_18BIT_CONFIG1,
+ COLOR_CODE_18BIT_CONFIG2,
+ COLOR_CODE_24BIT,
+ COLOR_CODE_20BIT_YCC422_LOOSELY,
+ COLOR_CODE_24BIT_YCC422,
+ COLOR_CODE_16BIT_YCC422,
+ COLOR_CODE_30BIT,
+ COLOR_CODE_36BIT,
+ COLOR_CODE_12BIT_YCC420,
+ COLOR_CODE_COMPRESSTION,
+ COLOR_CODE_MAX
+};
+
+enum pll_timing {
+ NONE,
+ REQUEST_TIME,
+ PREPARE_TIME,
+ SETTLE_TIME,
+ ZERO_TIME,
+ TRAIL_TIME,
+ EXIT_TIME,
+ CLKPOST_TIME,
+ TA_GET,
+ TA_GO,
+ TA_SURE,
+ TA_WAIT,
+};
+
+struct dphy_pll {
+ u8 refin; /* Pre-divider control signal */
+ u8 cp_s; /* 00: SDM_EN=1, 10: SDM_EN=0 */
+ u8 fdk_s; /* PLL mode control: integer or fraction */
+ u8 sdm_en;
+ u8 div;
+ u8 int_n; /* integer N PLL */
+ u32 ref_clk; /* dphy reference clock, unit: MHz */
+ u32 freq; /* panel config, unit: KHz */
+ u32 fvco;
+ u32 potential_fvco;
+ u32 nint; /* sigma delta modulator NINT control */
+ u32 kint; /* sigma delta modulator KINT control */
+ u8 lpf_sel; /* low pass filter control */
+ u8 out_sel; /* post divider control */
+ u8 vco_band; /* vco range */
+ u8 det_delay;
+};
+
+struct dsi_context {
+ void __iomem *base;
+ struct regmap *regmap;
+ struct dphy_pll pll;
+ struct videomode vm;
+ bool enabled;
+
+ u8 work_mode;
+ u8 burst_mode;
+ u32 int0_mask;
+ u32 int1_mask;
+
+ /* maximum time (ns) for data lanes from HS to LP */
+ u16 data_hs2lp;
+ /* maximum time (ns) for data lanes from LP to HS */
+ u16 data_lp2hs;
+ /* maximum time (ns) for clk lanes from HS to LP */
+ u16 clk_hs2lp;
+ /* maximum time (ns) for clk lanes from LP to HS */
+ u16 clk_lp2hs;
+ /* maximum time (ns) for BTA operation - REQUIRED */
+ u16 max_rd_time;
+ /* enable receiving frame ack packets - for video mode */
+ bool frame_ack_en;
+ /* enable receiving tear effect ack packets - for cmd mode */
+ bool te_ack_en;
+};
+
+struct sprd_dsi {
+ struct drm_device *drm;
+ struct mipi_dsi_host host;
+ struct mipi_dsi_device *slave;
+ struct drm_encoder encoder;
+ struct drm_bridge *panel_bridge;
+ struct dsi_context ctx;
+};
+
+int dphy_pll_config(struct dsi_context *ctx);
+void dphy_timing_config(struct dsi_context *ctx);
+
+#endif /* __SPRD_DSI_H__ */
diff --git a/drivers/gpu/drm/sti/Kconfig b/drivers/gpu/drm/sti/Kconfig
index d0cfdd3..246a94a 100644
--- a/drivers/gpu/drm/sti/Kconfig
+++ b/drivers/gpu/drm/sti/Kconfig
@@ -5,7 +5,6 @@
select RESET_CONTROLLER
select DRM_KMS_HELPER
select DRM_GEM_CMA_HELPER
- select DRM_KMS_CMA_HELPER
select DRM_PANEL
select FW_LOADER
select SND_SOC_HDMI_CODEC if SND_SOC
diff --git a/drivers/gpu/drm/stm/Kconfig b/drivers/gpu/drm/stm/Kconfig
index b7d6691..e037948 100644
--- a/drivers/gpu/drm/stm/Kconfig
+++ b/drivers/gpu/drm/stm/Kconfig
@@ -4,7 +4,6 @@
depends on DRM && (ARCH_STM32 || ARCH_MULTIPLATFORM)
select DRM_KMS_HELPER
select DRM_GEM_CMA_HELPER
- select DRM_KMS_CMA_HELPER
select DRM_PANEL_BRIDGE
select VIDEOMODE_HELPERS
select FB_PROVIDE_GET_FB_UNMAPPED_AREA if FB
diff --git a/drivers/gpu/drm/sun4i/Kconfig b/drivers/gpu/drm/sun4i/Kconfig
index 8c796de..befc5a8 100644
--- a/drivers/gpu/drm/sun4i/Kconfig
+++ b/drivers/gpu/drm/sun4i/Kconfig
@@ -5,7 +5,6 @@
depends on ARCH_SUNXI || COMPILE_TEST
select DRM_GEM_CMA_HELPER
select DRM_KMS_HELPER
- select DRM_KMS_CMA_HELPER
select DRM_PANEL
select REGMAP_MMIO
select VIDEOMODE_HELPERS
diff --git a/drivers/gpu/drm/sun4i/sun4i_drv.c b/drivers/gpu/drm/sun4i/sun4i_drv.c
index 54dd562..b630614 100644
--- a/drivers/gpu/drm/sun4i/sun4i_drv.c
+++ b/drivers/gpu/drm/sun4i/sun4i_drv.c
@@ -53,7 +53,7 @@ static const struct drm_driver sun4i_drv_driver = {
.minor = 0,
/* GEM Operations */
- DRM_GEM_CMA_DRIVER_OPS_VMAP_WITH_DUMB_CREATE(drm_sun4i_gem_dumb_create),
+ DRM_GEM_CMA_DRIVER_OPS_WITH_DUMB_CREATE(drm_sun4i_gem_dumb_create),
};
static int sun4i_drv_bind(struct device *dev)
diff --git a/drivers/gpu/drm/tegra/Kconfig b/drivers/gpu/drm/tegra/Kconfig
index 1650a44..8cf5aeb 100644
--- a/drivers/gpu/drm/tegra/Kconfig
+++ b/drivers/gpu/drm/tegra/Kconfig
@@ -12,6 +12,9 @@
select INTERCONNECT
select IOMMU_IOVA
select CEC_CORE if CEC_NOTIFIER
+ select SND_SIMPLE_CARD if SND_SOC_TEGRA20_SPDIF
+ select SND_SOC_HDMI_CODEC if SND_SOC_TEGRA20_SPDIF
+ select SND_AUDIO_GRAPH_CARD if SND_SOC_TEGRA20_SPDIF
help
Choose this option if you have an NVIDIA Tegra SoC.
diff --git a/drivers/gpu/drm/tegra/Makefile b/drivers/gpu/drm/tegra/Makefile
index d801909..df6cc98 100644
--- a/drivers/gpu/drm/tegra/Makefile
+++ b/drivers/gpu/drm/tegra/Makefile
@@ -23,7 +23,8 @@
gr2d.o \
gr3d.o \
falcon.o \
- vic.o
+ vic.o \
+ nvdec.o
tegra-drm-y += trace.o
diff --git a/drivers/gpu/drm/tegra/dc.c b/drivers/gpu/drm/tegra/dc.c
index a29d64f..eb70eee 100644
--- a/drivers/gpu/drm/tegra/dc.c
+++ b/drivers/gpu/drm/tegra/dc.c
@@ -11,9 +11,12 @@
#include <linux/interconnect.h>
#include <linux/module.h>
#include <linux/of_device.h>
+#include <linux/pm_domain.h>
+#include <linux/pm_opp.h>
#include <linux/pm_runtime.h>
#include <linux/reset.h>
+#include <soc/tegra/common.h>
#include <soc/tegra/pmc.h>
#include <drm/drm_atomic.h>
@@ -890,11 +893,9 @@ static int tegra_cursor_atomic_check(struct drm_plane *plane,
return 0;
}
-static void tegra_cursor_atomic_update(struct drm_plane *plane,
- struct drm_atomic_state *state)
+static void __tegra_cursor_atomic_update(struct drm_plane *plane,
+ struct drm_plane_state *new_state)
{
- struct drm_plane_state *new_state = drm_atomic_get_new_plane_state(state,
- plane);
struct tegra_plane_state *tegra_plane_state = to_tegra_plane_state(new_state);
struct tegra_dc *dc = to_tegra_dc(new_state->crtc);
struct tegra_drm *tegra = plane->dev->dev_private;
@@ -990,6 +991,14 @@ static void tegra_cursor_atomic_update(struct drm_plane *plane,
tegra_dc_writel(dc, value, DC_DISP_CURSOR_POSITION);
}
+static void tegra_cursor_atomic_update(struct drm_plane *plane,
+ struct drm_atomic_state *state)
+{
+ struct drm_plane_state *new_state = drm_atomic_get_new_plane_state(state, plane);
+
+ __tegra_cursor_atomic_update(plane, new_state);
+}
+
static void tegra_cursor_atomic_disable(struct drm_plane *plane,
struct drm_atomic_state *state)
{
@@ -1009,12 +1018,78 @@ static void tegra_cursor_atomic_disable(struct drm_plane *plane,
tegra_dc_writel(dc, value, DC_DISP_DISP_WIN_OPTIONS);
}
+static int tegra_cursor_atomic_async_check(struct drm_plane *plane, struct drm_atomic_state *state)
+{
+ struct drm_plane_state *new_state = drm_atomic_get_new_plane_state(state, plane);
+ struct drm_crtc_state *crtc_state;
+ int min_scale, max_scale;
+ int err;
+
+ crtc_state = drm_atomic_get_existing_crtc_state(state, new_state->crtc);
+ if (WARN_ON(!crtc_state))
+ return -EINVAL;
+
+ if (!crtc_state->active)
+ return -EINVAL;
+
+ if (plane->state->crtc != new_state->crtc ||
+ plane->state->src_w != new_state->src_w ||
+ plane->state->src_h != new_state->src_h ||
+ plane->state->crtc_w != new_state->crtc_w ||
+ plane->state->crtc_h != new_state->crtc_h ||
+ plane->state->fb != new_state->fb ||
+ plane->state->fb == NULL)
+ return -EINVAL;
+
+ min_scale = (1 << 16) / 8;
+ max_scale = (8 << 16) / 1;
+
+ err = drm_atomic_helper_check_plane_state(new_state, crtc_state, min_scale, max_scale,
+ true, true);
+ if (err < 0)
+ return err;
+
+ if (new_state->visible != plane->state->visible)
+ return -EINVAL;
+
+ return 0;
+}
+
+static void tegra_cursor_atomic_async_update(struct drm_plane *plane,
+ struct drm_atomic_state *state)
+{
+ struct drm_plane_state *new_state = drm_atomic_get_new_plane_state(state, plane);
+ struct tegra_dc *dc = to_tegra_dc(new_state->crtc);
+
+ plane->state->src_x = new_state->src_x;
+ plane->state->src_y = new_state->src_y;
+ plane->state->crtc_x = new_state->crtc_x;
+ plane->state->crtc_y = new_state->crtc_y;
+
+ if (new_state->visible) {
+ struct tegra_plane *p = to_tegra_plane(plane);
+ u32 value;
+
+ __tegra_cursor_atomic_update(plane, new_state);
+
+ value = (WIN_A_ACT_REQ << p->index) << 8 | GENERAL_UPDATE;
+ tegra_dc_writel(dc, value, DC_CMD_STATE_CONTROL);
+ (void)tegra_dc_readl(dc, DC_CMD_STATE_CONTROL);
+
+ value = (WIN_A_ACT_REQ << p->index) | GENERAL_ACT_REQ;
+ tegra_dc_writel(dc, value, DC_CMD_STATE_CONTROL);
+ (void)tegra_dc_readl(dc, DC_CMD_STATE_CONTROL);
+ }
+}
+
static const struct drm_plane_helper_funcs tegra_cursor_plane_helper_funcs = {
.prepare_fb = tegra_plane_prepare_fb,
.cleanup_fb = tegra_plane_cleanup_fb,
.atomic_check = tegra_cursor_atomic_check,
.atomic_update = tegra_cursor_atomic_update,
.atomic_disable = tegra_cursor_atomic_disable,
+ .atomic_async_check = tegra_cursor_atomic_async_check,
+ .atomic_async_update = tegra_cursor_atomic_async_update,
};
static const uint64_t linear_modifiers[] = {
@@ -1267,9 +1342,9 @@ static struct drm_plane *tegra_dc_add_planes(struct drm_device *drm,
err = PTR_ERR(planes[i]);
while (i--)
- tegra_plane_funcs.destroy(planes[i]);
+ planes[i]->funcs->destroy(planes[i]);
- tegra_plane_funcs.destroy(primary);
+ primary->funcs->destroy(primary);
return ERR_PTR(err);
}
}
@@ -1762,10 +1837,55 @@ int tegra_dc_state_setup_clock(struct tegra_dc *dc,
return 0;
}
-static void tegra_dc_commit_state(struct tegra_dc *dc,
- struct tegra_dc_state *state)
+static void tegra_dc_update_voltage_state(struct tegra_dc *dc,
+ struct tegra_dc_state *state)
{
- u32 value;
+ unsigned long rate, pstate;
+ struct dev_pm_opp *opp;
+ int err;
+
+ if (!dc->has_opp_table)
+ return;
+
+ /* calculate actual pixel clock rate which depends on internal divider */
+ rate = DIV_ROUND_UP(clk_get_rate(dc->clk) * 2, state->div + 2);
+
+ /* find suitable OPP for the rate */
+ opp = dev_pm_opp_find_freq_ceil(dc->dev, &rate);
+
+ /*
+ * Very high resolution modes may results in a clock rate that is
+ * above the characterized maximum. In this case it's okay to fall
+ * back to the characterized maximum.
+ */
+ if (opp == ERR_PTR(-ERANGE))
+ opp = dev_pm_opp_find_freq_floor(dc->dev, &rate);
+
+ if (IS_ERR(opp)) {
+ dev_err(dc->dev, "failed to find OPP for %luHz: %pe\n",
+ rate, opp);
+ return;
+ }
+
+ pstate = dev_pm_opp_get_required_pstate(opp, 0);
+ dev_pm_opp_put(opp);
+
+ /*
+ * The minimum core voltage depends on the pixel clock rate (which
+ * depends on internal clock divider of the CRTC) and not on the
+ * rate of the display controller clock. This is why we're not using
+ * dev_pm_opp_set_rate() API and instead controlling the power domain
+ * directly.
+ */
+ err = dev_pm_genpd_set_performance_state(dc->dev, pstate);
+ if (err)
+ dev_err(dc->dev, "failed to set power domain state to %lu: %d\n",
+ pstate, err);
+}
+
+static void tegra_dc_set_clock_rate(struct tegra_dc *dc,
+ struct tegra_dc_state *state)
+{
int err;
err = clk_set_parent(dc->clk, state->clk);
@@ -1797,10 +1917,7 @@ static void tegra_dc_commit_state(struct tegra_dc *dc,
state->div);
DRM_DEBUG_KMS("pclk: %lu\n", state->pclk);
- if (!dc->soc->has_nvdisplay) {
- value = SHIFT_CLK_DIVIDER(state->div) | PIXEL_CLK_DIVIDER_PCD1;
- tegra_dc_writel(dc, value, DC_DISP_DISP_CLOCK_CONTROL);
- }
+ tegra_dc_update_voltage_state(dc, state);
}
static void tegra_dc_stop(struct tegra_dc *dc)
@@ -1991,6 +2108,13 @@ static void tegra_crtc_atomic_disable(struct drm_crtc *crtc,
err = host1x_client_suspend(&dc->client);
if (err < 0)
dev_err(dc->dev, "failed to suspend: %d\n", err);
+
+ if (dc->has_opp_table) {
+ err = dev_pm_genpd_set_performance_state(dc->dev, 0);
+ if (err)
+ dev_err(dc->dev,
+ "failed to clear power domain state: %d\n", err);
+ }
}
static void tegra_crtc_atomic_enable(struct drm_crtc *crtc,
@@ -2002,6 +2126,9 @@ static void tegra_crtc_atomic_enable(struct drm_crtc *crtc,
u32 value;
int err;
+ /* apply PLL changes */
+ tegra_dc_set_clock_rate(dc, crtc_state);
+
err = host1x_client_resume(&dc->client);
if (err < 0) {
dev_err(dc->dev, "failed to resume: %d\n", err);
@@ -2076,8 +2203,11 @@ static void tegra_crtc_atomic_enable(struct drm_crtc *crtc,
else
tegra_dc_writel(dc, 0, DC_DISP_BORDER_COLOR);
- /* apply PLL and pixel clock changes */
- tegra_dc_commit_state(dc, crtc_state);
+ /* apply pixel clock changes */
+ if (!dc->soc->has_nvdisplay) {
+ value = SHIFT_CLK_DIVIDER(crtc_state->div) | PIXEL_CLK_DIVIDER_PCD1;
+ tegra_dc_writel(dc, value, DC_DISP_DISP_CLOCK_CONTROL);
+ }
/* program display mode */
tegra_dc_set_timings(dc, mode);
@@ -2107,6 +2237,12 @@ static void tegra_crtc_atomic_enable(struct drm_crtc *crtc,
tegra_dc_writel(dc, value, DC_COM_RG_UNDERFLOW);
}
+ if (dc->rgb) {
+ /* XXX: parameterize? */
+ value = SC0_H_QUALIFIER_NONE | SC1_H_QUALIFIER_NONE;
+ tegra_dc_writel(dc, value, DC_DISP_SHIFT_CLOCK_OPTIONS);
+ }
+
tegra_dc_commit(dc);
drm_crtc_vblank_on(crtc);
@@ -2685,6 +2821,7 @@ static const struct tegra_dc_soc_info tegra20_dc_soc_info = {
.has_win_b_vfilter_mem_client = true,
.has_win_c_without_vert_filter = true,
.plane_tiled_memory_bandwidth_x2 = false,
+ .has_pll_d2_out0 = false,
};
static const struct tegra_dc_soc_info tegra30_dc_soc_info = {
@@ -2707,6 +2844,7 @@ static const struct tegra_dc_soc_info tegra30_dc_soc_info = {
.has_win_b_vfilter_mem_client = true,
.has_win_c_without_vert_filter = false,
.plane_tiled_memory_bandwidth_x2 = true,
+ .has_pll_d2_out0 = true,
};
static const struct tegra_dc_soc_info tegra114_dc_soc_info = {
@@ -2729,6 +2867,7 @@ static const struct tegra_dc_soc_info tegra114_dc_soc_info = {
.has_win_b_vfilter_mem_client = false,
.has_win_c_without_vert_filter = false,
.plane_tiled_memory_bandwidth_x2 = true,
+ .has_pll_d2_out0 = true,
};
static const struct tegra_dc_soc_info tegra124_dc_soc_info = {
@@ -2751,6 +2890,7 @@ static const struct tegra_dc_soc_info tegra124_dc_soc_info = {
.has_win_b_vfilter_mem_client = false,
.has_win_c_without_vert_filter = false,
.plane_tiled_memory_bandwidth_x2 = false,
+ .has_pll_d2_out0 = true,
};
static const struct tegra_dc_soc_info tegra210_dc_soc_info = {
@@ -2773,6 +2913,7 @@ static const struct tegra_dc_soc_info tegra210_dc_soc_info = {
.has_win_b_vfilter_mem_client = false,
.has_win_c_without_vert_filter = false,
.plane_tiled_memory_bandwidth_x2 = false,
+ .has_pll_d2_out0 = true,
};
static const struct tegra_windowgroup_soc tegra186_dc_wgrps[] = {
@@ -2823,6 +2964,7 @@ static const struct tegra_dc_soc_info tegra186_dc_soc_info = {
.wgrps = tegra186_dc_wgrps,
.num_wgrps = ARRAY_SIZE(tegra186_dc_wgrps),
.plane_tiled_memory_bandwidth_x2 = false,
+ .has_pll_d2_out0 = false,
};
static const struct tegra_windowgroup_soc tegra194_dc_wgrps[] = {
@@ -2873,6 +3015,7 @@ static const struct tegra_dc_soc_info tegra194_dc_soc_info = {
.wgrps = tegra194_dc_wgrps,
.num_wgrps = ARRAY_SIZE(tegra194_dc_wgrps),
.plane_tiled_memory_bandwidth_x2 = false,
+ .has_pll_d2_out0 = false,
};
static const struct of_device_id tegra_dc_of_match[] = {
@@ -2973,6 +3116,23 @@ static int tegra_dc_couple(struct tegra_dc *dc)
return 0;
}
+static int tegra_dc_init_opp_table(struct tegra_dc *dc)
+{
+ struct tegra_core_opp_params opp_params = {};
+ int err;
+
+ err = devm_tegra_core_dev_init_opp_table(dc->dev, &opp_params);
+ if (err && err != -ENODEV)
+ return err;
+
+ if (err)
+ dc->has_opp_table = false;
+ else
+ dc->has_opp_table = true;
+
+ return 0;
+}
+
static int tegra_dc_probe(struct platform_device *pdev)
{
u64 dma_mask = dma_get_mask(pdev->dev.parent);
@@ -3038,6 +3198,10 @@ static int tegra_dc_probe(struct platform_device *pdev)
tegra_powergate_power_off(dc->powergate);
}
+ err = tegra_dc_init_opp_table(dc);
+ if (err < 0)
+ return err;
+
dc->regs = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(dc->regs))
return PTR_ERR(dc->regs);
diff --git a/drivers/gpu/drm/tegra/dc.h b/drivers/gpu/drm/tegra/dc.h
index 4037830..3f91a10 100644
--- a/drivers/gpu/drm/tegra/dc.h
+++ b/drivers/gpu/drm/tegra/dc.h
@@ -76,6 +76,7 @@ struct tegra_dc_soc_info {
bool has_win_b_vfilter_mem_client;
bool has_win_c_without_vert_filter;
bool plane_tiled_memory_bandwidth_x2;
+ bool has_pll_d2_out0;
};
struct tegra_dc {
@@ -100,6 +101,8 @@ struct tegra_dc {
struct drm_info_list *debugfs_files;
const struct tegra_dc_soc_info *soc;
+
+ bool has_opp_table;
};
static inline struct tegra_dc *
diff --git a/drivers/gpu/drm/tegra/drm.c b/drivers/gpu/drm/tegra/drm.c
index 8d37d6b..e9de91a 100644
--- a/drivers/gpu/drm/tegra/drm.c
+++ b/drivers/gpu/drm/tegra/drm.c
@@ -10,6 +10,7 @@
#include <linux/iommu.h>
#include <linux/module.h>
#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
#include <drm/drm_aperture.h>
#include <drm/drm_atomic.h>
@@ -21,6 +22,10 @@
#include <drm/drm_prime.h>
#include <drm/drm_vblank.h>
+#if IS_ENABLED(CONFIG_ARM_DMA_USE_IOMMU)
+#include <asm/dma-iommu.h>
+#endif
+
#include "dc.h"
#include "drm.h"
#include "gem.h"
@@ -116,6 +121,7 @@ static int tegra_drm_open(struct drm_device *drm, struct drm_file *filp)
static void tegra_drm_context_free(struct tegra_drm_context *context)
{
context->client->ops->close_channel(context);
+ pm_runtime_put(context->client->base.dev);
kfree(context);
}
@@ -427,13 +433,20 @@ static int tegra_client_open(struct tegra_drm_file *fpriv,
{
int err;
- err = client->ops->open_channel(client, context);
- if (err < 0)
+ err = pm_runtime_resume_and_get(client->base.dev);
+ if (err)
return err;
+ err = client->ops->open_channel(client, context);
+ if (err < 0) {
+ pm_runtime_put(client->base.dev);
+ return err;
+ }
+
err = idr_alloc(&fpriv->legacy_contexts, context, 1, 0, GFP_KERNEL);
if (err < 0) {
client->ops->close_channel(context);
+ pm_runtime_put(client->base.dev);
return err;
}
@@ -936,6 +949,17 @@ int host1x_client_iommu_attach(struct host1x_client *client)
struct iommu_group *group = NULL;
int err;
+#if IS_ENABLED(CONFIG_ARM_DMA_USE_IOMMU)
+ if (client->dev->archdata.mapping) {
+ struct dma_iommu_mapping *mapping =
+ to_dma_iommu_mapping(client->dev);
+ arm_iommu_detach_device(client->dev);
+ arm_iommu_release_mapping(mapping);
+
+ domain = iommu_get_domain_for_dev(client->dev);
+ }
+#endif
+
/*
* If the host1x client is already attached to an IOMMU domain that is
* not the shared IOMMU domain, don't try to attach it to a different
@@ -1344,15 +1368,18 @@ static const struct of_device_id host1x_drm_subdevs[] = {
{ .compatible = "nvidia,tegra210-sor", },
{ .compatible = "nvidia,tegra210-sor1", },
{ .compatible = "nvidia,tegra210-vic", },
+ { .compatible = "nvidia,tegra210-nvdec", },
{ .compatible = "nvidia,tegra186-display", },
{ .compatible = "nvidia,tegra186-dc", },
{ .compatible = "nvidia,tegra186-sor", },
{ .compatible = "nvidia,tegra186-sor1", },
{ .compatible = "nvidia,tegra186-vic", },
+ { .compatible = "nvidia,tegra186-nvdec", },
{ .compatible = "nvidia,tegra194-display", },
{ .compatible = "nvidia,tegra194-dc", },
{ .compatible = "nvidia,tegra194-sor", },
{ .compatible = "nvidia,tegra194-vic", },
+ { .compatible = "nvidia,tegra194-nvdec", },
{ /* sentinel */ }
};
@@ -1376,6 +1403,7 @@ static struct platform_driver * const drivers[] = {
&tegra_gr2d_driver,
&tegra_gr3d_driver,
&tegra_vic_driver,
+ &tegra_nvdec_driver,
};
static int __init host1x_drm_init(void)
diff --git a/drivers/gpu/drm/tegra/drm.h b/drivers/gpu/drm/tegra/drm.h
index 8b28327..fc0a195 100644
--- a/drivers/gpu/drm/tegra/drm.h
+++ b/drivers/gpu/drm/tegra/drm.h
@@ -202,5 +202,6 @@ extern struct platform_driver tegra_sor_driver;
extern struct platform_driver tegra_gr2d_driver;
extern struct platform_driver tegra_gr3d_driver;
extern struct platform_driver tegra_vic_driver;
+extern struct platform_driver tegra_nvdec_driver;
#endif /* HOST1X_DRM_H */
diff --git a/drivers/gpu/drm/tegra/gem.c b/drivers/gpu/drm/tegra/gem.c
index d38fd7e..fce0e52 100644
--- a/drivers/gpu/drm/tegra/gem.c
+++ b/drivers/gpu/drm/tegra/gem.c
@@ -23,6 +23,31 @@
MODULE_IMPORT_NS(DMA_BUF);
+static unsigned int sg_dma_count_chunks(struct scatterlist *sgl, unsigned int nents)
+{
+ dma_addr_t next = ~(dma_addr_t)0;
+ unsigned int count = 0, i;
+ struct scatterlist *s;
+
+ for_each_sg(sgl, s, nents, i) {
+ /* sg_dma_address(s) is only valid for entries that have sg_dma_len(s) != 0. */
+ if (!sg_dma_len(s))
+ continue;
+
+ if (sg_dma_address(s) != next) {
+ next = sg_dma_address(s) + sg_dma_len(s);
+ count++;
+ }
+ }
+
+ return count;
+}
+
+static inline unsigned int sgt_dma_count_chunks(struct sg_table *sgt)
+{
+ return sg_dma_count_chunks(sgt->sgl, sgt->nents);
+}
+
static void tegra_bo_put(struct host1x_bo *bo)
{
struct tegra_bo *obj = host1x_to_tegra_bo(bo);
@@ -30,79 +55,65 @@ static void tegra_bo_put(struct host1x_bo *bo)
drm_gem_object_put(&obj->gem);
}
-/* XXX move this into lib/scatterlist.c? */
-static int sg_alloc_table_from_sg(struct sg_table *sgt, struct scatterlist *sg,
- unsigned int nents, gfp_t gfp_mask)
-{
- struct scatterlist *dst;
- unsigned int i;
- int err;
-
- err = sg_alloc_table(sgt, nents, gfp_mask);
- if (err < 0)
- return err;
-
- dst = sgt->sgl;
-
- for (i = 0; i < nents; i++) {
- sg_set_page(dst, sg_page(sg), sg->length, 0);
- dst = sg_next(dst);
- sg = sg_next(sg);
- }
-
- return 0;
-}
-
-static struct sg_table *tegra_bo_pin(struct device *dev, struct host1x_bo *bo,
- dma_addr_t *phys)
+static struct host1x_bo_mapping *tegra_bo_pin(struct device *dev, struct host1x_bo *bo,
+ enum dma_data_direction direction)
{
struct tegra_bo *obj = host1x_to_tegra_bo(bo);
- struct sg_table *sgt;
+ struct drm_gem_object *gem = &obj->gem;
+ struct host1x_bo_mapping *map;
int err;
+ map = kzalloc(sizeof(*map), GFP_KERNEL);
+ if (!map)
+ return ERR_PTR(-ENOMEM);
+
+ kref_init(&map->ref);
+ map->bo = host1x_bo_get(bo);
+ map->direction = direction;
+ map->dev = dev;
+
/*
- * If we've manually mapped the buffer object through the IOMMU, make
- * sure to return the IOVA address of our mapping.
- *
- * Similarly, for buffers that have been allocated by the DMA API the
- * physical address can be used for devices that are not attached to
- * an IOMMU. For these devices, callers must pass a valid pointer via
- * the @phys argument.
- *
- * Imported buffers were also already mapped at import time, so the
- * existing mapping can be reused.
+ * Imported buffers need special treatment to satisfy the semantics of DMA-BUF.
*/
- if (phys) {
- *phys = obj->iova;
- return NULL;
+ if (gem->import_attach) {
+ struct dma_buf *buf = gem->import_attach->dmabuf;
+
+ map->attach = dma_buf_attach(buf, dev);
+ if (IS_ERR(map->attach)) {
+ err = PTR_ERR(map->attach);
+ goto free;
+ }
+
+ map->sgt = dma_buf_map_attachment(map->attach, direction);
+ if (IS_ERR(map->sgt)) {
+ dma_buf_detach(buf, map->attach);
+ err = PTR_ERR(map->sgt);
+ goto free;
+ }
+
+ err = sgt_dma_count_chunks(map->sgt);
+ map->size = gem->size;
+
+ goto out;
}
/*
* If we don't have a mapping for this buffer yet, return an SG table
* so that host1x can do the mapping for us via the DMA API.
*/
- sgt = kzalloc(sizeof(*sgt), GFP_KERNEL);
- if (!sgt)
- return ERR_PTR(-ENOMEM);
+ map->sgt = kzalloc(sizeof(*map->sgt), GFP_KERNEL);
+ if (!map->sgt) {
+ err = -ENOMEM;
+ goto free;
+ }
if (obj->pages) {
/*
* If the buffer object was allocated from the explicit IOMMU
* API code paths, construct an SG table from the pages.
*/
- err = sg_alloc_table_from_pages(sgt, obj->pages, obj->num_pages,
- 0, obj->gem.size, GFP_KERNEL);
- if (err < 0)
- goto free;
- } else if (obj->sgt) {
- /*
- * If the buffer object already has an SG table but no pages
- * were allocated for it, it means the buffer was imported and
- * the SG table needs to be copied to avoid overwriting any
- * other potential users of the original SG table.
- */
- err = sg_alloc_table_from_sg(sgt, obj->sgt->sgl,
- obj->sgt->orig_nents, GFP_KERNEL);
+ err = sg_alloc_table_from_pages(map->sgt, obj->pages, obj->num_pages, 0, gem->size,
+ GFP_KERNEL);
if (err < 0)
goto free;
} else {
@@ -111,25 +122,53 @@ static struct sg_table *tegra_bo_pin(struct device *dev, struct host1x_bo *bo,
* not imported, it had to be allocated with the DMA API, so
* the DMA API helper can be used.
*/
- err = dma_get_sgtable(dev, sgt, obj->vaddr, obj->iova,
- obj->gem.size);
+ err = dma_get_sgtable(dev, map->sgt, obj->vaddr, obj->iova, gem->size);
if (err < 0)
goto free;
}
- return sgt;
+ err = dma_map_sgtable(dev, map->sgt, direction, 0);
+ if (err)
+ goto free_sgt;
+out:
+ /*
+ * If we've manually mapped the buffer object through the IOMMU, make sure to return the
+ * existing IOVA address of our mapping.
+ */
+ if (!obj->mm) {
+ map->phys = sg_dma_address(map->sgt->sgl);
+ map->chunks = err;
+ } else {
+ map->phys = obj->iova;
+ map->chunks = 1;
+ }
+
+ map->size = gem->size;
+
+ return map;
+
+free_sgt:
+ sg_free_table(map->sgt);
free:
- kfree(sgt);
+ kfree(map->sgt);
+ kfree(map);
return ERR_PTR(err);
}
-static void tegra_bo_unpin(struct device *dev, struct sg_table *sgt)
+static void tegra_bo_unpin(struct host1x_bo_mapping *map)
{
- if (sgt) {
- sg_free_table(sgt);
- kfree(sgt);
+ if (map->attach) {
+ dma_buf_unmap_attachment(map->attach, map->sgt, map->direction);
+ dma_buf_detach(map->attach->dmabuf, map->attach);
+ } else {
+ dma_unmap_sgtable(map->dev, map->sgt, map->direction, 0);
+ sg_free_table(map->sgt);
+ kfree(map->sgt);
}
+
+ host1x_bo_put(map->bo);
+ kfree(map);
}
static void *tegra_bo_mmap(struct host1x_bo *bo)
@@ -452,8 +491,18 @@ static struct tegra_bo *tegra_bo_import(struct drm_device *drm,
void tegra_bo_free_object(struct drm_gem_object *gem)
{
struct tegra_drm *tegra = gem->dev->dev_private;
+ struct host1x_bo_mapping *mapping, *tmp;
struct tegra_bo *bo = to_tegra_bo(gem);
+ /* remove all mappings of this buffer object from any caches */
+ list_for_each_entry_safe(mapping, tmp, &bo->base.mappings, list) {
+ if (mapping->cache)
+ host1x_bo_unpin(mapping);
+ else
+ dev_err(gem->dev->dev, "mapping %p stale for device %s\n", mapping,
+ dev_name(mapping->dev));
+ }
+
if (tegra->domain)
tegra_bo_iommu_unmap(tegra, bo);
diff --git a/drivers/gpu/drm/tegra/gr2d.c b/drivers/gpu/drm/tegra/gr2d.c
index de288cb..e3bb4c9 100644
--- a/drivers/gpu/drm/tegra/gr2d.c
+++ b/drivers/gpu/drm/tegra/gr2d.c
@@ -4,14 +4,25 @@
*/
#include <linux/clk.h>
+#include <linux/delay.h>
#include <linux/iommu.h>
#include <linux/module.h>
#include <linux/of_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/reset.h>
+
+#include <soc/tegra/common.h>
#include "drm.h"
#include "gem.h"
#include "gr2d.h"
+enum {
+ RST_MC,
+ RST_GR2D,
+ RST_GR2D_MAX,
+};
+
struct gr2d_soc {
unsigned int version;
};
@@ -21,6 +32,9 @@ struct gr2d {
struct host1x_channel *channel;
struct clk *clk;
+ struct reset_control_bulk_data resets[RST_GR2D_MAX];
+ unsigned int nresets;
+
const struct gr2d_soc *soc;
DECLARE_BITMAP(addr_regs, GR2D_NUM_REGS);
@@ -56,15 +70,22 @@ static int gr2d_init(struct host1x_client *client)
goto free;
}
+ pm_runtime_enable(client->dev);
+ pm_runtime_use_autosuspend(client->dev);
+ pm_runtime_set_autosuspend_delay(client->dev, 200);
+
err = tegra_drm_register_client(dev->dev_private, drm);
if (err < 0) {
dev_err(client->dev, "failed to register client: %d\n", err);
- goto detach;
+ goto disable_rpm;
}
return 0;
-detach:
+disable_rpm:
+ pm_runtime_dont_use_autosuspend(client->dev);
+ pm_runtime_force_suspend(client->dev);
+
host1x_client_iommu_detach(client);
free:
host1x_syncpt_put(client->syncpts[0]);
@@ -85,10 +106,15 @@ static int gr2d_exit(struct host1x_client *client)
if (err < 0)
return err;
+ pm_runtime_dont_use_autosuspend(client->dev);
+ pm_runtime_force_suspend(client->dev);
+
host1x_client_iommu_detach(client);
host1x_syncpt_put(client->syncpts[0]);
host1x_channel_put(gr2d->channel);
+ gr2d->channel = NULL;
+
return 0;
}
@@ -190,6 +216,27 @@ static const u32 gr2d_addr_regs[] = {
GR2D_VA_BASE_ADDR_SB,
};
+static int gr2d_get_resets(struct device *dev, struct gr2d *gr2d)
+{
+ int err;
+
+ gr2d->resets[RST_MC].id = "mc";
+ gr2d->resets[RST_GR2D].id = "2d";
+ gr2d->nresets = RST_GR2D_MAX;
+
+ err = devm_reset_control_bulk_get_optional_exclusive_released(
+ dev, gr2d->nresets, gr2d->resets);
+ if (err) {
+ dev_err(dev, "failed to get reset: %d\n", err);
+ return err;
+ }
+
+ if (WARN_ON(!gr2d->resets[RST_GR2D].rstc))
+ return -ENOENT;
+
+ return 0;
+}
+
static int gr2d_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
@@ -202,6 +249,8 @@ static int gr2d_probe(struct platform_device *pdev)
if (!gr2d)
return -ENOMEM;
+ platform_set_drvdata(pdev, gr2d);
+
gr2d->soc = of_device_get_match_data(dev);
syncpts = devm_kzalloc(dev, sizeof(*syncpts), GFP_KERNEL);
@@ -214,11 +263,9 @@ static int gr2d_probe(struct platform_device *pdev)
return PTR_ERR(gr2d->clk);
}
- err = clk_prepare_enable(gr2d->clk);
- if (err) {
- dev_err(dev, "cannot turn on clock\n");
+ err = gr2d_get_resets(dev, gr2d);
+ if (err)
return err;
- }
INIT_LIST_HEAD(&gr2d->client.base.list);
gr2d->client.base.ops = &gr2d_client_ops;
@@ -231,10 +278,13 @@ static int gr2d_probe(struct platform_device *pdev)
gr2d->client.version = gr2d->soc->version;
gr2d->client.ops = &gr2d_ops;
+ err = devm_tegra_core_dev_init_opp_table_common(dev);
+ if (err)
+ return err;
+
err = host1x_client_register(&gr2d->client.base);
if (err < 0) {
dev_err(dev, "failed to register host1x client: %d\n", err);
- clk_disable_unprepare(gr2d->clk);
return err;
}
@@ -242,8 +292,6 @@ static int gr2d_probe(struct platform_device *pdev)
for (i = 0; i < ARRAY_SIZE(gr2d_addr_regs); i++)
set_bit(gr2d_addr_regs[i], gr2d->addr_regs);
- platform_set_drvdata(pdev, gr2d);
-
return 0;
}
@@ -259,15 +307,100 @@ static int gr2d_remove(struct platform_device *pdev)
return err;
}
+ return 0;
+}
+
+static int __maybe_unused gr2d_runtime_suspend(struct device *dev)
+{
+ struct gr2d *gr2d = dev_get_drvdata(dev);
+ int err;
+
+ host1x_channel_stop(gr2d->channel);
+ reset_control_bulk_release(gr2d->nresets, gr2d->resets);
+
+ /*
+ * GR2D module shouldn't be reset while hardware is idling, otherwise
+ * host1x's cmdproc will stuck on trying to access any G2 register
+ * after reset. GR2D module could be either hot-reset or reset after
+ * power-gating of the HEG partition. Hence we will put in reset only
+ * the memory client part of the module, the HEG GENPD will take care
+ * of resetting GR2D module across power-gating.
+ *
+ * On Tegra20 there is no HEG partition, but it's okay to have
+ * undetermined h/w state since userspace is expected to reprogram
+ * the state on each job submission anyways.
+ */
+ err = reset_control_acquire(gr2d->resets[RST_MC].rstc);
+ if (err) {
+ dev_err(dev, "failed to acquire MC reset: %d\n", err);
+ goto acquire_reset;
+ }
+
+ err = reset_control_assert(gr2d->resets[RST_MC].rstc);
+ reset_control_release(gr2d->resets[RST_MC].rstc);
+ if (err) {
+ dev_err(dev, "failed to assert MC reset: %d\n", err);
+ goto acquire_reset;
+ }
+
clk_disable_unprepare(gr2d->clk);
return 0;
+
+acquire_reset:
+ reset_control_bulk_acquire(gr2d->nresets, gr2d->resets);
+ reset_control_bulk_deassert(gr2d->nresets, gr2d->resets);
+
+ return err;
}
+static int __maybe_unused gr2d_runtime_resume(struct device *dev)
+{
+ struct gr2d *gr2d = dev_get_drvdata(dev);
+ int err;
+
+ err = reset_control_bulk_acquire(gr2d->nresets, gr2d->resets);
+ if (err) {
+ dev_err(dev, "failed to acquire reset: %d\n", err);
+ return err;
+ }
+
+ err = clk_prepare_enable(gr2d->clk);
+ if (err) {
+ dev_err(dev, "failed to enable clock: %d\n", err);
+ goto release_reset;
+ }
+
+ usleep_range(2000, 4000);
+
+ /* this is a reset array which deasserts both 2D MC and 2D itself */
+ err = reset_control_bulk_deassert(gr2d->nresets, gr2d->resets);
+ if (err) {
+ dev_err(dev, "failed to deassert reset: %d\n", err);
+ goto disable_clk;
+ }
+
+ return 0;
+
+disable_clk:
+ clk_disable_unprepare(gr2d->clk);
+release_reset:
+ reset_control_bulk_release(gr2d->nresets, gr2d->resets);
+
+ return err;
+}
+
+static const struct dev_pm_ops tegra_gr2d_pm = {
+ SET_RUNTIME_PM_OPS(gr2d_runtime_suspend, gr2d_runtime_resume, NULL)
+ SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend,
+ pm_runtime_force_resume)
+};
+
struct platform_driver tegra_gr2d_driver = {
.driver = {
.name = "tegra-gr2d",
.of_match_table = gr2d_match,
+ .pm = &tegra_gr2d_pm,
},
.probe = gr2d_probe,
.remove = gr2d_remove,
diff --git a/drivers/gpu/drm/tegra/gr3d.c b/drivers/gpu/drm/tegra/gr3d.c
index 24442ad..a1fd311 100644
--- a/drivers/gpu/drm/tegra/gr3d.c
+++ b/drivers/gpu/drm/tegra/gr3d.c
@@ -5,32 +5,47 @@
*/
#include <linux/clk.h>
+#include <linux/delay.h>
#include <linux/host1x.h>
#include <linux/iommu.h>
#include <linux/module.h>
#include <linux/of_device.h>
#include <linux/platform_device.h>
+#include <linux/pm_domain.h>
+#include <linux/pm_opp.h>
+#include <linux/pm_runtime.h>
#include <linux/reset.h>
+#include <soc/tegra/common.h>
#include <soc/tegra/pmc.h>
#include "drm.h"
#include "gem.h"
#include "gr3d.h"
+enum {
+ RST_MC,
+ RST_GR3D,
+ RST_MC2,
+ RST_GR3D2,
+ RST_GR3D_MAX,
+};
+
struct gr3d_soc {
unsigned int version;
+ unsigned int num_clocks;
+ unsigned int num_resets;
};
struct gr3d {
struct tegra_drm_client client;
struct host1x_channel *channel;
- struct clk *clk_secondary;
- struct clk *clk;
- struct reset_control *rst_secondary;
- struct reset_control *rst;
const struct gr3d_soc *soc;
+ struct clk_bulk_data *clocks;
+ unsigned int nclocks;
+ struct reset_control_bulk_data resets[RST_GR3D_MAX];
+ unsigned int nresets;
DECLARE_BITMAP(addr_regs, GR3D_NUM_REGS);
};
@@ -65,15 +80,22 @@ static int gr3d_init(struct host1x_client *client)
goto free;
}
+ pm_runtime_enable(client->dev);
+ pm_runtime_use_autosuspend(client->dev);
+ pm_runtime_set_autosuspend_delay(client->dev, 200);
+
err = tegra_drm_register_client(dev->dev_private, drm);
if (err < 0) {
dev_err(client->dev, "failed to register client: %d\n", err);
- goto detach;
+ goto disable_rpm;
}
return 0;
-detach:
+disable_rpm:
+ pm_runtime_dont_use_autosuspend(client->dev);
+ pm_runtime_force_suspend(client->dev);
+
host1x_client_iommu_detach(client);
free:
host1x_syncpt_put(client->syncpts[0]);
@@ -93,10 +115,15 @@ static int gr3d_exit(struct host1x_client *client)
if (err < 0)
return err;
+ pm_runtime_dont_use_autosuspend(client->dev);
+ pm_runtime_force_suspend(client->dev);
+
host1x_client_iommu_detach(client);
host1x_syncpt_put(client->syncpts[0]);
host1x_channel_put(gr3d->channel);
+ gr3d->channel = NULL;
+
return 0;
}
@@ -155,14 +182,20 @@ static const struct tegra_drm_client_ops gr3d_ops = {
static const struct gr3d_soc tegra20_gr3d_soc = {
.version = 0x20,
+ .num_clocks = 1,
+ .num_resets = 2,
};
static const struct gr3d_soc tegra30_gr3d_soc = {
.version = 0x30,
+ .num_clocks = 2,
+ .num_resets = 4,
};
static const struct gr3d_soc tegra114_gr3d_soc = {
.version = 0x35,
+ .num_clocks = 1,
+ .num_resets = 2,
};
static const struct of_device_id tegra_gr3d_match[] = {
@@ -278,9 +311,187 @@ static const u32 gr3d_addr_regs[] = {
GR3D_GLOBAL_SAMP23SURFADDR(15),
};
+static int gr3d_power_up_legacy_domain(struct device *dev, const char *name,
+ unsigned int id)
+{
+ struct gr3d *gr3d = dev_get_drvdata(dev);
+ struct reset_control *reset;
+ struct clk *clk;
+ unsigned int i;
+ int err;
+
+ /*
+ * Tegra20 device-tree doesn't specify 3d clock name and there is only
+ * one clock for Tegra20. Tegra30+ device-trees always specified names
+ * for the clocks.
+ */
+ if (gr3d->nclocks == 1) {
+ if (id == TEGRA_POWERGATE_3D1)
+ return 0;
+
+ clk = gr3d->clocks[0].clk;
+ } else {
+ for (i = 0; i < gr3d->nclocks; i++) {
+ if (WARN_ON(!gr3d->clocks[i].id))
+ continue;
+
+ if (!strcmp(gr3d->clocks[i].id, name)) {
+ clk = gr3d->clocks[i].clk;
+ break;
+ }
+ }
+
+ if (WARN_ON(i == gr3d->nclocks))
+ return -EINVAL;
+ }
+
+ /*
+ * We use array of resets, which includes MC resets, and MC
+ * reset shouldn't be asserted while hardware is gated because
+ * MC flushing will fail for gated hardware. Hence for legacy
+ * PD we request the individual reset separately.
+ */
+ reset = reset_control_get_exclusive_released(dev, name);
+ if (IS_ERR(reset))
+ return PTR_ERR(reset);
+
+ err = reset_control_acquire(reset);
+ if (err) {
+ dev_err(dev, "failed to acquire %s reset: %d\n", name, err);
+ } else {
+ err = tegra_powergate_sequence_power_up(id, clk, reset);
+ reset_control_release(reset);
+ }
+
+ reset_control_put(reset);
+ if (err)
+ return err;
+
+ /*
+ * tegra_powergate_sequence_power_up() leaves clocks enabled,
+ * while GENPD not. Hence keep clock-enable balanced.
+ */
+ clk_disable_unprepare(clk);
+
+ return 0;
+}
+
+static void gr3d_del_link(void *link)
+{
+ device_link_del(link);
+}
+
+static int gr3d_init_power(struct device *dev, struct gr3d *gr3d)
+{
+ static const char * const opp_genpd_names[] = { "3d0", "3d1", NULL };
+ const u32 link_flags = DL_FLAG_STATELESS | DL_FLAG_PM_RUNTIME;
+ struct device **opp_virt_devs, *pd_dev;
+ struct device_link *link;
+ unsigned int i;
+ int err;
+
+ err = of_count_phandle_with_args(dev->of_node, "power-domains",
+ "#power-domain-cells");
+ if (err < 0) {
+ if (err != -ENOENT)
+ return err;
+
+ /*
+ * Older device-trees don't use GENPD. In this case we should
+ * toggle power domain manually.
+ */
+ err = gr3d_power_up_legacy_domain(dev, "3d",
+ TEGRA_POWERGATE_3D);
+ if (err)
+ return err;
+
+ err = gr3d_power_up_legacy_domain(dev, "3d2",
+ TEGRA_POWERGATE_3D1);
+ if (err)
+ return err;
+
+ return 0;
+ }
+
+ /*
+ * The PM domain core automatically attaches a single power domain,
+ * otherwise it skips attaching completely. We have a single domain
+ * on Tegra20 and two domains on Tegra30+.
+ */
+ if (dev->pm_domain)
+ return 0;
+
+ err = devm_pm_opp_attach_genpd(dev, opp_genpd_names, &opp_virt_devs);
+ if (err)
+ return err;
+
+ for (i = 0; opp_genpd_names[i]; i++) {
+ pd_dev = opp_virt_devs[i];
+ if (!pd_dev) {
+ dev_err(dev, "failed to get %s power domain\n",
+ opp_genpd_names[i]);
+ return -EINVAL;
+ }
+
+ link = device_link_add(dev, pd_dev, link_flags);
+ if (!link) {
+ dev_err(dev, "failed to link to %s\n", dev_name(pd_dev));
+ return -EINVAL;
+ }
+
+ err = devm_add_action_or_reset(dev, gr3d_del_link, link);
+ if (err)
+ return err;
+ }
+
+ return 0;
+}
+
+static int gr3d_get_clocks(struct device *dev, struct gr3d *gr3d)
+{
+ int err;
+
+ err = devm_clk_bulk_get_all(dev, &gr3d->clocks);
+ if (err < 0) {
+ dev_err(dev, "failed to get clock: %d\n", err);
+ return err;
+ }
+ gr3d->nclocks = err;
+
+ if (gr3d->nclocks != gr3d->soc->num_clocks) {
+ dev_err(dev, "invalid number of clocks: %u\n", gr3d->nclocks);
+ return -ENOENT;
+ }
+
+ return 0;
+}
+
+static int gr3d_get_resets(struct device *dev, struct gr3d *gr3d)
+{
+ int err;
+
+ gr3d->resets[RST_MC].id = "mc";
+ gr3d->resets[RST_MC2].id = "mc2";
+ gr3d->resets[RST_GR3D].id = "3d";
+ gr3d->resets[RST_GR3D2].id = "3d2";
+ gr3d->nresets = gr3d->soc->num_resets;
+
+ err = devm_reset_control_bulk_get_optional_exclusive_released(
+ dev, gr3d->nresets, gr3d->resets);
+ if (err) {
+ dev_err(dev, "failed to get reset: %d\n", err);
+ return err;
+ }
+
+ if (WARN_ON(!gr3d->resets[RST_GR3D].rstc) ||
+ WARN_ON(!gr3d->resets[RST_GR3D2].rstc && gr3d->nresets == 4))
+ return -ENOENT;
+
+ return 0;
+}
+
static int gr3d_probe(struct platform_device *pdev)
{
- struct device_node *np = pdev->dev.of_node;
struct host1x_syncpt **syncpts;
struct gr3d *gr3d;
unsigned int i;
@@ -290,56 +501,25 @@ static int gr3d_probe(struct platform_device *pdev)
if (!gr3d)
return -ENOMEM;
+ platform_set_drvdata(pdev, gr3d);
+
gr3d->soc = of_device_get_match_data(&pdev->dev);
syncpts = devm_kzalloc(&pdev->dev, sizeof(*syncpts), GFP_KERNEL);
if (!syncpts)
return -ENOMEM;
- gr3d->clk = devm_clk_get(&pdev->dev, NULL);
- if (IS_ERR(gr3d->clk)) {
- dev_err(&pdev->dev, "cannot get clock\n");
- return PTR_ERR(gr3d->clk);
- }
-
- gr3d->rst = devm_reset_control_get(&pdev->dev, "3d");
- if (IS_ERR(gr3d->rst)) {
- dev_err(&pdev->dev, "cannot get reset\n");
- return PTR_ERR(gr3d->rst);
- }
-
- if (of_device_is_compatible(np, "nvidia,tegra30-gr3d")) {
- gr3d->clk_secondary = devm_clk_get(&pdev->dev, "3d2");
- if (IS_ERR(gr3d->clk_secondary)) {
- dev_err(&pdev->dev, "cannot get secondary clock\n");
- return PTR_ERR(gr3d->clk_secondary);
- }
-
- gr3d->rst_secondary = devm_reset_control_get(&pdev->dev,
- "3d2");
- if (IS_ERR(gr3d->rst_secondary)) {
- dev_err(&pdev->dev, "cannot get secondary reset\n");
- return PTR_ERR(gr3d->rst_secondary);
- }
- }
-
- err = tegra_powergate_sequence_power_up(TEGRA_POWERGATE_3D, gr3d->clk,
- gr3d->rst);
- if (err < 0) {
- dev_err(&pdev->dev, "failed to power up 3D unit\n");
+ err = gr3d_get_clocks(&pdev->dev, gr3d);
+ if (err)
return err;
- }
- if (gr3d->clk_secondary) {
- err = tegra_powergate_sequence_power_up(TEGRA_POWERGATE_3D1,
- gr3d->clk_secondary,
- gr3d->rst_secondary);
- if (err < 0) {
- dev_err(&pdev->dev,
- "failed to power up secondary 3D unit\n");
- return err;
- }
- }
+ err = gr3d_get_resets(&pdev->dev, gr3d);
+ if (err)
+ return err;
+
+ err = gr3d_init_power(&pdev->dev, gr3d);
+ if (err)
+ return err;
INIT_LIST_HEAD(&gr3d->client.base.list);
gr3d->client.base.ops = &gr3d_client_ops;
@@ -352,6 +532,10 @@ static int gr3d_probe(struct platform_device *pdev)
gr3d->client.version = gr3d->soc->version;
gr3d->client.ops = &gr3d_ops;
+ err = devm_tegra_core_dev_init_opp_table_common(&pdev->dev);
+ if (err)
+ return err;
+
err = host1x_client_register(&gr3d->client.base);
if (err < 0) {
dev_err(&pdev->dev, "failed to register host1x client: %d\n",
@@ -363,8 +547,6 @@ static int gr3d_probe(struct platform_device *pdev)
for (i = 0; i < ARRAY_SIZE(gr3d_addr_regs); i++)
set_bit(gr3d_addr_regs[i], gr3d->addr_regs);
- platform_set_drvdata(pdev, gr3d);
-
return 0;
}
@@ -380,23 +562,80 @@ static int gr3d_remove(struct platform_device *pdev)
return err;
}
- if (gr3d->clk_secondary) {
- reset_control_assert(gr3d->rst_secondary);
- tegra_powergate_power_off(TEGRA_POWERGATE_3D1);
- clk_disable_unprepare(gr3d->clk_secondary);
+ return 0;
+}
+
+static int __maybe_unused gr3d_runtime_suspend(struct device *dev)
+{
+ struct gr3d *gr3d = dev_get_drvdata(dev);
+ int err;
+
+ host1x_channel_stop(gr3d->channel);
+
+ err = reset_control_bulk_assert(gr3d->nresets, gr3d->resets);
+ if (err) {
+ dev_err(dev, "failed to assert reset: %d\n", err);
+ return err;
}
- reset_control_assert(gr3d->rst);
- tegra_powergate_power_off(TEGRA_POWERGATE_3D);
- clk_disable_unprepare(gr3d->clk);
+ usleep_range(10, 20);
+
+ /*
+ * Older device-trees don't specify MC resets and power-gating can't
+ * be done safely in that case. Hence we will keep the power ungated
+ * for older DTBs. For newer DTBs, GENPD will perform the power-gating.
+ */
+
+ clk_bulk_disable_unprepare(gr3d->nclocks, gr3d->clocks);
+ reset_control_bulk_release(gr3d->nresets, gr3d->resets);
return 0;
}
+static int __maybe_unused gr3d_runtime_resume(struct device *dev)
+{
+ struct gr3d *gr3d = dev_get_drvdata(dev);
+ int err;
+
+ err = reset_control_bulk_acquire(gr3d->nresets, gr3d->resets);
+ if (err) {
+ dev_err(dev, "failed to acquire reset: %d\n", err);
+ return err;
+ }
+
+ err = clk_bulk_prepare_enable(gr3d->nclocks, gr3d->clocks);
+ if (err) {
+ dev_err(dev, "failed to enable clock: %d\n", err);
+ goto release_reset;
+ }
+
+ err = reset_control_bulk_deassert(gr3d->nresets, gr3d->resets);
+ if (err) {
+ dev_err(dev, "failed to deassert reset: %d\n", err);
+ goto disable_clk;
+ }
+
+ return 0;
+
+disable_clk:
+ clk_bulk_disable_unprepare(gr3d->nclocks, gr3d->clocks);
+release_reset:
+ reset_control_bulk_release(gr3d->nresets, gr3d->resets);
+
+ return err;
+}
+
+static const struct dev_pm_ops tegra_gr3d_pm = {
+ SET_RUNTIME_PM_OPS(gr3d_runtime_suspend, gr3d_runtime_resume, NULL)
+ SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend,
+ pm_runtime_force_resume)
+};
+
struct platform_driver tegra_gr3d_driver = {
.driver = {
.name = "tegra-gr3d",
.of_match_table = tegra_gr3d_match,
+ .pm = &tegra_gr3d_pm,
},
.probe = gr3d_probe,
.remove = gr3d_remove,
diff --git a/drivers/gpu/drm/tegra/hdmi.c b/drivers/gpu/drm/tegra/hdmi.c
index e5d2a40..8845af5 100644
--- a/drivers/gpu/drm/tegra/hdmi.c
+++ b/drivers/gpu/drm/tegra/hdmi.c
@@ -11,10 +11,14 @@
#include <linux/math64.h>
#include <linux/module.h>
#include <linux/of_device.h>
+#include <linux/pm_opp.h>
#include <linux/pm_runtime.h>
#include <linux/regulator/consumer.h>
#include <linux/reset.h>
+#include <soc/tegra/common.h>
+#include <sound/hdmi-codec.h>
+
#include <drm/drm_atomic_helper.h>
#include <drm/drm_crtc.h>
#include <drm/drm_debugfs.h>
@@ -78,6 +82,9 @@ struct tegra_hdmi {
bool dvi;
struct drm_info_list *debugfs_files;
+
+ struct platform_device *audio_pdev;
+ struct mutex audio_lock;
};
static inline struct tegra_hdmi *
@@ -360,6 +367,18 @@ static const struct tmds_config tegra124_tmds_config[] = {
},
};
+static void tegra_hdmi_audio_lock(struct tegra_hdmi *hdmi)
+{
+ mutex_lock(&hdmi->audio_lock);
+ disable_irq(hdmi->irq);
+}
+
+static void tegra_hdmi_audio_unlock(struct tegra_hdmi *hdmi)
+{
+ enable_irq(hdmi->irq);
+ mutex_unlock(&hdmi->audio_lock);
+}
+
static int
tegra_hdmi_get_audio_config(unsigned int audio_freq, unsigned int pix_clock,
struct tegra_hdmi_audio_config *config)
@@ -829,6 +848,23 @@ static void tegra_hdmi_setup_tmds(struct tegra_hdmi *hdmi,
HDMI_NV_PDISP_SOR_IO_PEAK_CURRENT);
}
+static int tegra_hdmi_reconfigure_audio(struct tegra_hdmi *hdmi)
+{
+ int err;
+
+ err = tegra_hdmi_setup_audio(hdmi);
+ if (err < 0) {
+ tegra_hdmi_disable_audio_infoframe(hdmi);
+ tegra_hdmi_disable_audio(hdmi);
+ } else {
+ tegra_hdmi_setup_audio_infoframe(hdmi);
+ tegra_hdmi_enable_audio_infoframe(hdmi);
+ tegra_hdmi_enable_audio(hdmi);
+ }
+
+ return err;
+}
+
static bool tegra_output_is_hdmi(struct tegra_output *output)
{
struct edid *edid;
@@ -1135,6 +1171,8 @@ static void tegra_hdmi_encoder_disable(struct drm_encoder *encoder)
u32 value;
int err;
+ tegra_hdmi_audio_lock(hdmi);
+
/*
* The following accesses registers of the display controller, so make
* sure it's only executed when the output is attached to one.
@@ -1159,6 +1197,10 @@ static void tegra_hdmi_encoder_disable(struct drm_encoder *encoder)
tegra_hdmi_writel(hdmi, 0, HDMI_NV_PDISP_INT_ENABLE);
tegra_hdmi_writel(hdmi, 0, HDMI_NV_PDISP_INT_MASK);
+ hdmi->pixel_clock = 0;
+
+ tegra_hdmi_audio_unlock(hdmi);
+
err = host1x_client_suspend(&hdmi->client);
if (err < 0)
dev_err(hdmi->dev, "failed to suspend: %d\n", err);
@@ -1182,6 +1224,8 @@ static void tegra_hdmi_encoder_enable(struct drm_encoder *encoder)
return;
}
+ tegra_hdmi_audio_lock(hdmi);
+
/*
* Enable and unmask the HDA codec SCRATCH0 register interrupt. This
* is used for interoperability between the HDA codec driver and the
@@ -1195,7 +1239,7 @@ static void tegra_hdmi_encoder_enable(struct drm_encoder *encoder)
h_back_porch = mode->htotal - mode->hsync_end;
h_front_porch = mode->hsync_start - mode->hdisplay;
- err = clk_set_rate(hdmi->clk, hdmi->pixel_clock);
+ err = dev_pm_opp_set_rate(hdmi->dev, hdmi->pixel_clock);
if (err < 0) {
dev_err(hdmi->dev, "failed to set HDMI clock frequency: %d\n",
err);
@@ -1387,6 +1431,8 @@ static void tegra_hdmi_encoder_enable(struct drm_encoder *encoder)
}
/* TODO: add HDCP support */
+
+ tegra_hdmi_audio_unlock(hdmi);
}
static int
@@ -1416,6 +1462,91 @@ static const struct drm_encoder_helper_funcs tegra_hdmi_encoder_helper_funcs = {
.atomic_check = tegra_hdmi_encoder_atomic_check,
};
+static int tegra_hdmi_hw_params(struct device *dev, void *data,
+ struct hdmi_codec_daifmt *fmt,
+ struct hdmi_codec_params *hparms)
+{
+ struct tegra_hdmi *hdmi = data;
+ int ret = 0;
+
+ tegra_hdmi_audio_lock(hdmi);
+
+ hdmi->format.sample_rate = hparms->sample_rate;
+ hdmi->format.channels = hparms->channels;
+
+ if (hdmi->pixel_clock && !hdmi->dvi)
+ ret = tegra_hdmi_reconfigure_audio(hdmi);
+
+ tegra_hdmi_audio_unlock(hdmi);
+
+ return ret;
+}
+
+static int tegra_hdmi_audio_startup(struct device *dev, void *data)
+{
+ struct tegra_hdmi *hdmi = data;
+ int ret;
+
+ ret = host1x_client_resume(&hdmi->client);
+ if (ret < 0)
+ dev_err(hdmi->dev, "failed to resume: %d\n", ret);
+
+ return ret;
+}
+
+static void tegra_hdmi_audio_shutdown(struct device *dev, void *data)
+{
+ struct tegra_hdmi *hdmi = data;
+ int ret;
+
+ tegra_hdmi_audio_lock(hdmi);
+
+ hdmi->format.sample_rate = 0;
+ hdmi->format.channels = 0;
+
+ tegra_hdmi_audio_unlock(hdmi);
+
+ ret = host1x_client_suspend(&hdmi->client);
+ if (ret < 0)
+ dev_err(hdmi->dev, "failed to suspend: %d\n", ret);
+}
+
+static const struct hdmi_codec_ops tegra_hdmi_codec_ops = {
+ .hw_params = tegra_hdmi_hw_params,
+ .audio_startup = tegra_hdmi_audio_startup,
+ .audio_shutdown = tegra_hdmi_audio_shutdown,
+};
+
+static int tegra_hdmi_codec_register(struct tegra_hdmi *hdmi)
+{
+ struct hdmi_codec_pdata codec_data = {};
+
+ if (hdmi->config->has_hda)
+ return 0;
+
+ codec_data.ops = &tegra_hdmi_codec_ops;
+ codec_data.data = hdmi;
+ codec_data.spdif = 1;
+
+ hdmi->audio_pdev = platform_device_register_data(hdmi->dev,
+ HDMI_CODEC_DRV_NAME,
+ PLATFORM_DEVID_AUTO,
+ &codec_data,
+ sizeof(codec_data));
+ if (IS_ERR(hdmi->audio_pdev))
+ return PTR_ERR(hdmi->audio_pdev);
+
+ hdmi->format.channels = 2;
+
+ return 0;
+}
+
+static void tegra_hdmi_codec_unregister(struct tegra_hdmi *hdmi)
+{
+ if (hdmi->audio_pdev)
+ platform_device_unregister(hdmi->audio_pdev);
+}
+
static int tegra_hdmi_init(struct host1x_client *client)
{
struct tegra_hdmi *hdmi = host1x_client_to_hdmi(client);
@@ -1453,28 +1584,47 @@ static int tegra_hdmi_init(struct host1x_client *client)
if (err < 0) {
dev_err(client->dev, "failed to enable HDMI regulator: %d\n",
err);
- return err;
+ goto output_exit;
}
err = regulator_enable(hdmi->pll);
if (err < 0) {
dev_err(hdmi->dev, "failed to enable PLL regulator: %d\n", err);
- return err;
+ goto disable_hdmi;
}
err = regulator_enable(hdmi->vdd);
if (err < 0) {
dev_err(hdmi->dev, "failed to enable VDD regulator: %d\n", err);
- return err;
+ goto disable_pll;
+ }
+
+ err = tegra_hdmi_codec_register(hdmi);
+ if (err < 0) {
+ dev_err(hdmi->dev, "failed to register audio codec: %d\n", err);
+ goto disable_vdd;
}
return 0;
+
+disable_vdd:
+ regulator_disable(hdmi->vdd);
+disable_pll:
+ regulator_disable(hdmi->pll);
+disable_hdmi:
+ regulator_disable(hdmi->hdmi);
+output_exit:
+ tegra_output_exit(&hdmi->output);
+
+ return err;
}
static int tegra_hdmi_exit(struct host1x_client *client)
{
struct tegra_hdmi *hdmi = host1x_client_to_hdmi(client);
+ tegra_hdmi_codec_unregister(hdmi);
+
tegra_output_exit(&hdmi->output);
regulator_disable(hdmi->vdd);
@@ -1599,7 +1749,6 @@ static irqreturn_t tegra_hdmi_irq(int irq, void *data)
{
struct tegra_hdmi *hdmi = data;
u32 value;
- int err;
value = tegra_hdmi_readl(hdmi, HDMI_NV_PDISP_INT_STATUS);
tegra_hdmi_writel(hdmi, value, HDMI_NV_PDISP_INT_STATUS);
@@ -1614,16 +1763,7 @@ static irqreturn_t tegra_hdmi_irq(int irq, void *data)
format = value & SOR_AUDIO_HDA_CODEC_SCRATCH0_FMT_MASK;
tegra_hda_parse_format(format, &hdmi->format);
-
- err = tegra_hdmi_setup_audio(hdmi);
- if (err < 0) {
- tegra_hdmi_disable_audio_infoframe(hdmi);
- tegra_hdmi_disable_audio(hdmi);
- } else {
- tegra_hdmi_setup_audio_infoframe(hdmi);
- tegra_hdmi_enable_audio_infoframe(hdmi);
- tegra_hdmi_enable_audio(hdmi);
- }
+ tegra_hdmi_reconfigure_audio(hdmi);
} else {
tegra_hdmi_disable_audio_infoframe(hdmi);
tegra_hdmi_disable_audio(hdmi);
@@ -1651,6 +1791,8 @@ static int tegra_hdmi_probe(struct platform_device *pdev)
hdmi->stereo = false;
hdmi->dvi = false;
+ mutex_init(&hdmi->audio_lock);
+
hdmi->clk = devm_clk_get(&pdev->dev, NULL);
if (IS_ERR(hdmi->clk)) {
dev_err(&pdev->dev, "failed to get clock\n");
@@ -1732,7 +1874,14 @@ static int tegra_hdmi_probe(struct platform_device *pdev)
}
platform_set_drvdata(pdev, hdmi);
- pm_runtime_enable(&pdev->dev);
+
+ err = devm_pm_runtime_enable(&pdev->dev);
+ if (err)
+ return err;
+
+ err = devm_tegra_core_dev_init_opp_table_common(&pdev->dev);
+ if (err)
+ return err;
INIT_LIST_HEAD(&hdmi->client.list);
hdmi->client.ops = &hdmi_client_ops;
@@ -1753,8 +1902,6 @@ static int tegra_hdmi_remove(struct platform_device *pdev)
struct tegra_hdmi *hdmi = platform_get_drvdata(pdev);
int err;
- pm_runtime_disable(&pdev->dev);
-
err = host1x_client_unregister(&hdmi->client);
if (err < 0) {
dev_err(&pdev->dev, "failed to unregister host1x client: %d\n",
diff --git a/drivers/gpu/drm/tegra/hub.h b/drivers/gpu/drm/tegra/hub.h
index 3efa1be..23c4b21 100644
--- a/drivers/gpu/drm/tegra/hub.h
+++ b/drivers/gpu/drm/tegra/hub.h
@@ -72,7 +72,6 @@ to_tegra_display_hub_state(struct drm_private_state *priv)
return container_of(priv, struct tegra_display_hub_state, base);
}
-struct tegra_dc;
struct tegra_plane;
int tegra_display_hub_prepare(struct tegra_display_hub *hub);
diff --git a/drivers/gpu/drm/tegra/nvdec.c b/drivers/gpu/drm/tegra/nvdec.c
new file mode 100644
index 0000000..79e1e88
--- /dev/null
+++ b/drivers/gpu/drm/tegra/nvdec.c
@@ -0,0 +1,466 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2015-2021, NVIDIA Corporation.
+ */
+
+#include <linux/clk.h>
+#include <linux/delay.h>
+#include <linux/host1x.h>
+#include <linux/iommu.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/of_device.h>
+#include <linux/of_platform.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/reset.h>
+
+#include <soc/tegra/pmc.h>
+
+#include "drm.h"
+#include "falcon.h"
+#include "vic.h"
+
+struct nvdec_config {
+ const char *firmware;
+ unsigned int version;
+ bool supports_sid;
+};
+
+struct nvdec {
+ struct falcon falcon;
+
+ void __iomem *regs;
+ struct tegra_drm_client client;
+ struct host1x_channel *channel;
+ struct device *dev;
+ struct clk *clk;
+
+ /* Platform configuration */
+ const struct nvdec_config *config;
+};
+
+static inline struct nvdec *to_nvdec(struct tegra_drm_client *client)
+{
+ return container_of(client, struct nvdec, client);
+}
+
+static inline void nvdec_writel(struct nvdec *nvdec, u32 value,
+ unsigned int offset)
+{
+ writel(value, nvdec->regs + offset);
+}
+
+static int nvdec_boot(struct nvdec *nvdec)
+{
+#ifdef CONFIG_IOMMU_API
+ struct iommu_fwspec *spec = dev_iommu_fwspec_get(nvdec->dev);
+#endif
+ int err;
+
+#ifdef CONFIG_IOMMU_API
+ if (nvdec->config->supports_sid && spec) {
+ u32 value;
+
+ value = TRANSCFG_ATT(1, TRANSCFG_SID_FALCON) | TRANSCFG_ATT(0, TRANSCFG_SID_HW);
+ nvdec_writel(nvdec, value, VIC_TFBIF_TRANSCFG);
+
+ if (spec->num_ids > 0) {
+ value = spec->ids[0] & 0xffff;
+
+ nvdec_writel(nvdec, value, VIC_THI_STREAMID0);
+ nvdec_writel(nvdec, value, VIC_THI_STREAMID1);
+ }
+ }
+#endif
+
+ err = falcon_boot(&nvdec->falcon);
+ if (err < 0)
+ return err;
+
+ err = falcon_wait_idle(&nvdec->falcon);
+ if (err < 0) {
+ dev_err(nvdec->dev, "falcon boot timed out\n");
+ return err;
+ }
+
+ return 0;
+}
+
+static int nvdec_init(struct host1x_client *client)
+{
+ struct tegra_drm_client *drm = host1x_to_drm_client(client);
+ struct drm_device *dev = dev_get_drvdata(client->host);
+ struct tegra_drm *tegra = dev->dev_private;
+ struct nvdec *nvdec = to_nvdec(drm);
+ int err;
+
+ err = host1x_client_iommu_attach(client);
+ if (err < 0 && err != -ENODEV) {
+ dev_err(nvdec->dev, "failed to attach to domain: %d\n", err);
+ return err;
+ }
+
+ nvdec->channel = host1x_channel_request(client);
+ if (!nvdec->channel) {
+ err = -ENOMEM;
+ goto detach;
+ }
+
+ client->syncpts[0] = host1x_syncpt_request(client, 0);
+ if (!client->syncpts[0]) {
+ err = -ENOMEM;
+ goto free_channel;
+ }
+
+ pm_runtime_enable(client->dev);
+ pm_runtime_use_autosuspend(client->dev);
+ pm_runtime_set_autosuspend_delay(client->dev, 500);
+
+ err = tegra_drm_register_client(tegra, drm);
+ if (err < 0)
+ goto disable_rpm;
+
+ /*
+ * Inherit the DMA parameters (such as maximum segment size) from the
+ * parent host1x device.
+ */
+ client->dev->dma_parms = client->host->dma_parms;
+
+ return 0;
+
+disable_rpm:
+ pm_runtime_dont_use_autosuspend(client->dev);
+ pm_runtime_force_suspend(client->dev);
+
+ host1x_syncpt_put(client->syncpts[0]);
+free_channel:
+ host1x_channel_put(nvdec->channel);
+detach:
+ host1x_client_iommu_detach(client);
+
+ return err;
+}
+
+static int nvdec_exit(struct host1x_client *client)
+{
+ struct tegra_drm_client *drm = host1x_to_drm_client(client);
+ struct drm_device *dev = dev_get_drvdata(client->host);
+ struct tegra_drm *tegra = dev->dev_private;
+ struct nvdec *nvdec = to_nvdec(drm);
+ int err;
+
+ /* avoid a dangling pointer just in case this disappears */
+ client->dev->dma_parms = NULL;
+
+ err = tegra_drm_unregister_client(tegra, drm);
+ if (err < 0)
+ return err;
+
+ pm_runtime_dont_use_autosuspend(client->dev);
+ pm_runtime_force_suspend(client->dev);
+
+ host1x_syncpt_put(client->syncpts[0]);
+ host1x_channel_put(nvdec->channel);
+ host1x_client_iommu_detach(client);
+
+ nvdec->channel = NULL;
+
+ if (client->group) {
+ dma_unmap_single(nvdec->dev, nvdec->falcon.firmware.phys,
+ nvdec->falcon.firmware.size, DMA_TO_DEVICE);
+ tegra_drm_free(tegra, nvdec->falcon.firmware.size,
+ nvdec->falcon.firmware.virt,
+ nvdec->falcon.firmware.iova);
+ } else {
+ dma_free_coherent(nvdec->dev, nvdec->falcon.firmware.size,
+ nvdec->falcon.firmware.virt,
+ nvdec->falcon.firmware.iova);
+ }
+
+ return 0;
+}
+
+static const struct host1x_client_ops nvdec_client_ops = {
+ .init = nvdec_init,
+ .exit = nvdec_exit,
+};
+
+static int nvdec_load_firmware(struct nvdec *nvdec)
+{
+ struct host1x_client *client = &nvdec->client.base;
+ struct tegra_drm *tegra = nvdec->client.drm;
+ dma_addr_t iova;
+ size_t size;
+ void *virt;
+ int err;
+
+ if (nvdec->falcon.firmware.virt)
+ return 0;
+
+ err = falcon_read_firmware(&nvdec->falcon, nvdec->config->firmware);
+ if (err < 0)
+ return err;
+
+ size = nvdec->falcon.firmware.size;
+
+ if (!client->group) {
+ virt = dma_alloc_coherent(nvdec->dev, size, &iova, GFP_KERNEL);
+
+ err = dma_mapping_error(nvdec->dev, iova);
+ if (err < 0)
+ return err;
+ } else {
+ virt = tegra_drm_alloc(tegra, size, &iova);
+ }
+
+ nvdec->falcon.firmware.virt = virt;
+ nvdec->falcon.firmware.iova = iova;
+
+ err = falcon_load_firmware(&nvdec->falcon);
+ if (err < 0)
+ goto cleanup;
+
+ /*
+ * In this case we have received an IOVA from the shared domain, so we
+ * need to make sure to get the physical address so that the DMA API
+ * knows what memory pages to flush the cache for.
+ */
+ if (client->group) {
+ dma_addr_t phys;
+
+ phys = dma_map_single(nvdec->dev, virt, size, DMA_TO_DEVICE);
+
+ err = dma_mapping_error(nvdec->dev, phys);
+ if (err < 0)
+ goto cleanup;
+
+ nvdec->falcon.firmware.phys = phys;
+ }
+
+ return 0;
+
+cleanup:
+ if (!client->group)
+ dma_free_coherent(nvdec->dev, size, virt, iova);
+ else
+ tegra_drm_free(tegra, size, virt, iova);
+
+ return err;
+}
+
+
+static __maybe_unused int nvdec_runtime_resume(struct device *dev)
+{
+ struct nvdec *nvdec = dev_get_drvdata(dev);
+ int err;
+
+ err = clk_prepare_enable(nvdec->clk);
+ if (err < 0)
+ return err;
+
+ usleep_range(10, 20);
+
+ err = nvdec_load_firmware(nvdec);
+ if (err < 0)
+ goto disable;
+
+ err = nvdec_boot(nvdec);
+ if (err < 0)
+ goto disable;
+
+ return 0;
+
+disable:
+ clk_disable_unprepare(nvdec->clk);
+ return err;
+}
+
+static __maybe_unused int nvdec_runtime_suspend(struct device *dev)
+{
+ struct nvdec *nvdec = dev_get_drvdata(dev);
+
+ host1x_channel_stop(nvdec->channel);
+
+ clk_disable_unprepare(nvdec->clk);
+
+ return 0;
+}
+
+static int nvdec_open_channel(struct tegra_drm_client *client,
+ struct tegra_drm_context *context)
+{
+ struct nvdec *nvdec = to_nvdec(client);
+
+ context->channel = host1x_channel_get(nvdec->channel);
+ if (!context->channel)
+ return -ENOMEM;
+
+ return 0;
+}
+
+static void nvdec_close_channel(struct tegra_drm_context *context)
+{
+ host1x_channel_put(context->channel);
+}
+
+static const struct tegra_drm_client_ops nvdec_ops = {
+ .open_channel = nvdec_open_channel,
+ .close_channel = nvdec_close_channel,
+ .submit = tegra_drm_submit,
+};
+
+#define NVIDIA_TEGRA_210_NVDEC_FIRMWARE "nvidia/tegra210/nvdec.bin"
+
+static const struct nvdec_config nvdec_t210_config = {
+ .firmware = NVIDIA_TEGRA_210_NVDEC_FIRMWARE,
+ .version = 0x21,
+ .supports_sid = false,
+};
+
+#define NVIDIA_TEGRA_186_NVDEC_FIRMWARE "nvidia/tegra186/nvdec.bin"
+
+static const struct nvdec_config nvdec_t186_config = {
+ .firmware = NVIDIA_TEGRA_186_NVDEC_FIRMWARE,
+ .version = 0x18,
+ .supports_sid = true,
+};
+
+#define NVIDIA_TEGRA_194_NVDEC_FIRMWARE "nvidia/tegra194/nvdec.bin"
+
+static const struct nvdec_config nvdec_t194_config = {
+ .firmware = NVIDIA_TEGRA_194_NVDEC_FIRMWARE,
+ .version = 0x19,
+ .supports_sid = true,
+};
+
+static const struct of_device_id tegra_nvdec_of_match[] = {
+ { .compatible = "nvidia,tegra210-nvdec", .data = &nvdec_t210_config },
+ { .compatible = "nvidia,tegra186-nvdec", .data = &nvdec_t186_config },
+ { .compatible = "nvidia,tegra194-nvdec", .data = &nvdec_t194_config },
+ { },
+};
+MODULE_DEVICE_TABLE(of, tegra_nvdec_of_match);
+
+static int nvdec_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct host1x_syncpt **syncpts;
+ struct nvdec *nvdec;
+ u32 host_class;
+ int err;
+
+ /* inherit DMA mask from host1x parent */
+ err = dma_coerce_mask_and_coherent(dev, *dev->parent->dma_mask);
+ if (err < 0) {
+ dev_err(&pdev->dev, "failed to set DMA mask: %d\n", err);
+ return err;
+ }
+
+ nvdec = devm_kzalloc(dev, sizeof(*nvdec), GFP_KERNEL);
+ if (!nvdec)
+ return -ENOMEM;
+
+ nvdec->config = of_device_get_match_data(dev);
+
+ syncpts = devm_kzalloc(dev, sizeof(*syncpts), GFP_KERNEL);
+ if (!syncpts)
+ return -ENOMEM;
+
+ nvdec->regs = devm_platform_get_and_ioremap_resource(pdev, 0, NULL);
+ if (IS_ERR(nvdec->regs))
+ return PTR_ERR(nvdec->regs);
+
+ nvdec->clk = devm_clk_get(dev, NULL);
+ if (IS_ERR(nvdec->clk)) {
+ dev_err(&pdev->dev, "failed to get clock\n");
+ return PTR_ERR(nvdec->clk);
+ }
+
+ err = clk_set_rate(nvdec->clk, ULONG_MAX);
+ if (err < 0) {
+ dev_err(&pdev->dev, "failed to set clock rate\n");
+ return err;
+ }
+
+ err = of_property_read_u32(dev->of_node, "nvidia,host1x-class", &host_class);
+ if (err < 0)
+ host_class = HOST1X_CLASS_NVDEC;
+
+ nvdec->falcon.dev = dev;
+ nvdec->falcon.regs = nvdec->regs;
+
+ err = falcon_init(&nvdec->falcon);
+ if (err < 0)
+ return err;
+
+ platform_set_drvdata(pdev, nvdec);
+
+ INIT_LIST_HEAD(&nvdec->client.base.list);
+ nvdec->client.base.ops = &nvdec_client_ops;
+ nvdec->client.base.dev = dev;
+ nvdec->client.base.class = host_class;
+ nvdec->client.base.syncpts = syncpts;
+ nvdec->client.base.num_syncpts = 1;
+ nvdec->dev = dev;
+
+ INIT_LIST_HEAD(&nvdec->client.list);
+ nvdec->client.version = nvdec->config->version;
+ nvdec->client.ops = &nvdec_ops;
+
+ err = host1x_client_register(&nvdec->client.base);
+ if (err < 0) {
+ dev_err(dev, "failed to register host1x client: %d\n", err);
+ goto exit_falcon;
+ }
+
+ return 0;
+
+exit_falcon:
+ falcon_exit(&nvdec->falcon);
+
+ return err;
+}
+
+static int nvdec_remove(struct platform_device *pdev)
+{
+ struct nvdec *nvdec = platform_get_drvdata(pdev);
+ int err;
+
+ err = host1x_client_unregister(&nvdec->client.base);
+ if (err < 0) {
+ dev_err(&pdev->dev, "failed to unregister host1x client: %d\n",
+ err);
+ return err;
+ }
+
+ falcon_exit(&nvdec->falcon);
+
+ return 0;
+}
+
+static const struct dev_pm_ops nvdec_pm_ops = {
+ SET_RUNTIME_PM_OPS(nvdec_runtime_suspend, nvdec_runtime_resume, NULL)
+ SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend,
+ pm_runtime_force_resume)
+};
+
+struct platform_driver tegra_nvdec_driver = {
+ .driver = {
+ .name = "tegra-nvdec",
+ .of_match_table = tegra_nvdec_of_match,
+ .pm = &nvdec_pm_ops
+ },
+ .probe = nvdec_probe,
+ .remove = nvdec_remove,
+};
+
+#if IS_ENABLED(CONFIG_ARCH_TEGRA_210_SOC)
+MODULE_FIRMWARE(NVIDIA_TEGRA_210_NVDEC_FIRMWARE);
+#endif
+#if IS_ENABLED(CONFIG_ARCH_TEGRA_186_SOC)
+MODULE_FIRMWARE(NVIDIA_TEGRA_186_NVDEC_FIRMWARE);
+#endif
+#if IS_ENABLED(CONFIG_ARCH_TEGRA_194_SOC)
+MODULE_FIRMWARE(NVIDIA_TEGRA_194_NVDEC_FIRMWARE);
+#endif
diff --git a/drivers/gpu/drm/tegra/plane.c b/drivers/gpu/drm/tegra/plane.c
index 16a1cdc..321cb1f 100644
--- a/drivers/gpu/drm/tegra/plane.c
+++ b/drivers/gpu/drm/tegra/plane.c
@@ -74,7 +74,7 @@ tegra_plane_atomic_duplicate_state(struct drm_plane *plane)
for (i = 0; i < 3; i++) {
copy->iova[i] = DMA_MAPPING_ERROR;
- copy->sgt[i] = NULL;
+ copy->map[i] = NULL;
}
return ©->base;
@@ -138,55 +138,37 @@ const struct drm_plane_funcs tegra_plane_funcs = {
static int tegra_dc_pin(struct tegra_dc *dc, struct tegra_plane_state *state)
{
- struct iommu_domain *domain = iommu_get_domain_for_dev(dc->dev);
unsigned int i;
int err;
for (i = 0; i < state->base.fb->format->num_planes; i++) {
struct tegra_bo *bo = tegra_fb_get_plane(state->base.fb, i);
- dma_addr_t phys_addr, *phys;
- struct sg_table *sgt;
+ struct host1x_bo_mapping *map;
- /*
- * If we're not attached to a domain, we already stored the
- * physical address when the buffer was allocated. If we're
- * part of a group that's shared between all display
- * controllers, we've also already mapped the framebuffer
- * through the SMMU. In both cases we can short-circuit the
- * code below and retrieve the stored IOV address.
- */
- if (!domain || dc->client.group)
- phys = &phys_addr;
- else
- phys = NULL;
-
- sgt = host1x_bo_pin(dc->dev, &bo->base, phys);
- if (IS_ERR(sgt)) {
- err = PTR_ERR(sgt);
+ map = host1x_bo_pin(dc->dev, &bo->base, DMA_TO_DEVICE, &dc->client.cache);
+ if (IS_ERR(map)) {
+ err = PTR_ERR(map);
goto unpin;
}
- if (sgt) {
- err = dma_map_sgtable(dc->dev, sgt, DMA_TO_DEVICE, 0);
- if (err)
- goto unpin;
-
+ if (!dc->client.group) {
/*
* The display controller needs contiguous memory, so
* fail if the buffer is discontiguous and we fail to
* map its SG table to a single contiguous chunk of
* I/O virtual memory.
*/
- if (sgt->nents > 1) {
+ if (map->chunks > 1) {
err = -EINVAL;
goto unpin;
}
- state->iova[i] = sg_dma_address(sgt->sgl);
- state->sgt[i] = sgt;
+ state->iova[i] = map->phys;
} else {
- state->iova[i] = phys_addr;
+ state->iova[i] = bo->iova;
}
+
+ state->map[i] = map;
}
return 0;
@@ -195,15 +177,9 @@ static int tegra_dc_pin(struct tegra_dc *dc, struct tegra_plane_state *state)
dev_err(dc->dev, "failed to map plane %u: %d\n", i, err);
while (i--) {
- struct tegra_bo *bo = tegra_fb_get_plane(state->base.fb, i);
- struct sg_table *sgt = state->sgt[i];
-
- if (sgt)
- dma_unmap_sgtable(dc->dev, sgt, DMA_TO_DEVICE, 0);
-
- host1x_bo_unpin(dc->dev, &bo->base, sgt);
+ host1x_bo_unpin(state->map[i]);
state->iova[i] = DMA_MAPPING_ERROR;
- state->sgt[i] = NULL;
+ state->map[i] = NULL;
}
return err;
@@ -214,15 +190,9 @@ static void tegra_dc_unpin(struct tegra_dc *dc, struct tegra_plane_state *state)
unsigned int i;
for (i = 0; i < state->base.fb->format->num_planes; i++) {
- struct tegra_bo *bo = tegra_fb_get_plane(state->base.fb, i);
- struct sg_table *sgt = state->sgt[i];
-
- if (sgt)
- dma_unmap_sgtable(dc->dev, sgt, DMA_TO_DEVICE, 0);
-
- host1x_bo_unpin(dc->dev, &bo->base, sgt);
+ host1x_bo_unpin(state->map[i]);
state->iova[i] = DMA_MAPPING_ERROR;
- state->sgt[i] = NULL;
+ state->map[i] = NULL;
}
}
@@ -230,11 +200,14 @@ int tegra_plane_prepare_fb(struct drm_plane *plane,
struct drm_plane_state *state)
{
struct tegra_dc *dc = to_tegra_dc(state->crtc);
+ int err;
if (!state->fb)
return 0;
- drm_gem_plane_helper_prepare_fb(plane, state);
+ err = drm_gem_plane_helper_prepare_fb(plane, state);
+ if (err < 0)
+ return err;
return tegra_dc_pin(dc, to_tegra_plane_state(state));
}
diff --git a/drivers/gpu/drm/tegra/plane.h b/drivers/gpu/drm/tegra/plane.h
index d947078..dfb2071 100644
--- a/drivers/gpu/drm/tegra/plane.h
+++ b/drivers/gpu/drm/tegra/plane.h
@@ -43,7 +43,7 @@ struct tegra_plane_legacy_blending_state {
struct tegra_plane_state {
struct drm_plane_state base;
- struct sg_table *sgt[3];
+ struct host1x_bo_mapping *map[3];
dma_addr_t iova[3];
struct tegra_bo_tiling tiling;
diff --git a/drivers/gpu/drm/tegra/rgb.c b/drivers/gpu/drm/tegra/rgb.c
index 606c78a..ff8fce3 100644
--- a/drivers/gpu/drm/tegra/rgb.c
+++ b/drivers/gpu/drm/tegra/rgb.c
@@ -17,6 +17,8 @@ struct tegra_rgb {
struct tegra_output output;
struct tegra_dc *dc;
+ struct clk *pll_d_out0;
+ struct clk *pll_d2_out0;
struct clk *clk_parent;
struct clk *clk;
};
@@ -116,13 +118,21 @@ static void tegra_rgb_encoder_enable(struct drm_encoder *encoder)
DISP_ORDER_RED_BLUE;
tegra_dc_writel(rgb->dc, value, DC_DISP_DISP_INTERFACE_CONTROL);
- /* XXX: parameterize? */
- value = SC0_H_QUALIFIER_NONE | SC1_H_QUALIFIER_NONE;
- tegra_dc_writel(rgb->dc, value, DC_DISP_SHIFT_CLOCK_OPTIONS);
-
tegra_dc_commit(rgb->dc);
}
+static bool tegra_rgb_pll_rate_change_allowed(struct tegra_rgb *rgb)
+{
+ if (!rgb->pll_d2_out0)
+ return false;
+
+ if (!clk_is_match(rgb->clk_parent, rgb->pll_d_out0) &&
+ !clk_is_match(rgb->clk_parent, rgb->pll_d2_out0))
+ return false;
+
+ return true;
+}
+
static int
tegra_rgb_encoder_atomic_check(struct drm_encoder *encoder,
struct drm_crtc_state *crtc_state,
@@ -151,8 +161,17 @@ tegra_rgb_encoder_atomic_check(struct drm_encoder *encoder,
* and hope that the desired frequency can be matched (or at least
* matched sufficiently close that the panel will still work).
*/
- div = ((clk_get_rate(rgb->clk) * 2) / pclk) - 2;
- pclk = 0;
+ if (tegra_rgb_pll_rate_change_allowed(rgb)) {
+ /*
+ * Set display controller clock to x2 of PCLK in order to
+ * produce higher resolution pulse positions.
+ */
+ div = 2;
+ pclk *= 2;
+ } else {
+ div = ((clk_get_rate(rgb->clk) * 2) / pclk) - 2;
+ pclk = 0;
+ }
err = tegra_dc_state_setup_clock(dc, crtc_state, rgb->clk_parent,
pclk, div);
@@ -210,6 +229,22 @@ int tegra_dc_rgb_probe(struct tegra_dc *dc)
return err;
}
+ rgb->pll_d_out0 = clk_get_sys(NULL, "pll_d_out0");
+ if (IS_ERR(rgb->pll_d_out0)) {
+ err = PTR_ERR(rgb->pll_d_out0);
+ dev_err(dc->dev, "failed to get pll_d_out0: %d\n", err);
+ return err;
+ }
+
+ if (dc->soc->has_pll_d2_out0) {
+ rgb->pll_d2_out0 = clk_get_sys(NULL, "pll_d2_out0");
+ if (IS_ERR(rgb->pll_d2_out0)) {
+ err = PTR_ERR(rgb->pll_d2_out0);
+ dev_err(dc->dev, "failed to get pll_d2_out0: %d\n", err);
+ return err;
+ }
+ }
+
dc->rgb = &rgb->output;
return 0;
@@ -217,9 +252,15 @@ int tegra_dc_rgb_probe(struct tegra_dc *dc)
int tegra_dc_rgb_remove(struct tegra_dc *dc)
{
+ struct tegra_rgb *rgb;
+
if (!dc->rgb)
return 0;
+ rgb = to_rgb(dc->rgb);
+ clk_put(rgb->pll_d2_out0);
+ clk_put(rgb->pll_d_out0);
+
tegra_output_remove(dc->rgb);
dc->rgb = NULL;
diff --git a/drivers/gpu/drm/tegra/submit.c b/drivers/gpu/drm/tegra/submit.c
index 776f825..6d6dd8c 100644
--- a/drivers/gpu/drm/tegra/submit.c
+++ b/drivers/gpu/drm/tegra/submit.c
@@ -64,33 +64,62 @@ static void gather_bo_put(struct host1x_bo *host_bo)
kref_put(&bo->ref, gather_bo_release);
}
-static struct sg_table *
-gather_bo_pin(struct device *dev, struct host1x_bo *host_bo, dma_addr_t *phys)
+static struct host1x_bo_mapping *
+gather_bo_pin(struct device *dev, struct host1x_bo *bo, enum dma_data_direction direction)
{
- struct gather_bo *bo = container_of(host_bo, struct gather_bo, base);
- struct sg_table *sgt;
+ struct gather_bo *gather = container_of(bo, struct gather_bo, base);
+ struct host1x_bo_mapping *map;
int err;
- sgt = kzalloc(sizeof(*sgt), GFP_KERNEL);
- if (!sgt)
+ map = kzalloc(sizeof(*map), GFP_KERNEL);
+ if (!map)
return ERR_PTR(-ENOMEM);
- err = dma_get_sgtable(bo->dev, sgt, bo->gather_data, bo->gather_data_dma,
- bo->gather_data_words * 4);
- if (err) {
- kfree(sgt);
- return ERR_PTR(err);
+ kref_init(&map->ref);
+ map->bo = host1x_bo_get(bo);
+ map->direction = direction;
+ map->dev = dev;
+
+ map->sgt = kzalloc(sizeof(*map->sgt), GFP_KERNEL);
+ if (!map->sgt) {
+ err = -ENOMEM;
+ goto free;
}
- return sgt;
+ err = dma_get_sgtable(gather->dev, map->sgt, gather->gather_data, gather->gather_data_dma,
+ gather->gather_data_words * 4);
+ if (err)
+ goto free_sgt;
+
+ err = dma_map_sgtable(dev, map->sgt, direction, 0);
+ if (err)
+ goto free_sgt;
+
+ map->phys = sg_dma_address(map->sgt->sgl);
+ map->size = gather->gather_data_words * 4;
+ map->chunks = err;
+
+ return map;
+
+free_sgt:
+ sg_free_table(map->sgt);
+ kfree(map->sgt);
+free:
+ kfree(map);
+ return ERR_PTR(err);
}
-static void gather_bo_unpin(struct device *dev, struct sg_table *sgt)
+static void gather_bo_unpin(struct host1x_bo_mapping *map)
{
- if (sgt) {
- sg_free_table(sgt);
- kfree(sgt);
- }
+ if (!map)
+ return;
+
+ dma_unmap_sgtable(map->dev, map->sgt, map->direction, 0);
+ sg_free_table(map->sgt);
+ kfree(map->sgt);
+ host1x_bo_put(map->bo);
+
+ kfree(map);
}
static void *gather_bo_mmap(struct host1x_bo *host_bo)
@@ -475,8 +504,8 @@ static void release_job(struct host1x_job *job)
kfree(job_data->used_mappings);
kfree(job_data);
- if (pm_runtime_enabled(client->base.dev))
- pm_runtime_put_autosuspend(client->base.dev);
+ pm_runtime_mark_last_busy(client->base.dev);
+ pm_runtime_put_autosuspend(client->base.dev);
}
int tegra_drm_ioctl_channel_submit(struct drm_device *drm, void *data,
@@ -560,12 +589,10 @@ int tegra_drm_ioctl_channel_submit(struct drm_device *drm, void *data,
}
/* Boot engine. */
- if (pm_runtime_enabled(context->client->base.dev)) {
- err = pm_runtime_resume_and_get(context->client->base.dev);
- if (err < 0) {
- SUBMIT_ERR(context, "could not power up engine: %d", err);
- goto unpin_job;
- }
+ err = pm_runtime_resume_and_get(context->client->base.dev);
+ if (err < 0) {
+ SUBMIT_ERR(context, "could not power up engine: %d", err);
+ goto unpin_job;
}
job->user_data = job_data;
diff --git a/drivers/gpu/drm/tegra/uapi.c b/drivers/gpu/drm/tegra/uapi.c
index 690a339..9ab9179 100644
--- a/drivers/gpu/drm/tegra/uapi.c
+++ b/drivers/gpu/drm/tegra/uapi.c
@@ -17,11 +17,7 @@ static void tegra_drm_mapping_release(struct kref *ref)
struct tegra_drm_mapping *mapping =
container_of(ref, struct tegra_drm_mapping, ref);
- if (mapping->sgt)
- dma_unmap_sgtable(mapping->dev, mapping->sgt, mapping->direction,
- DMA_ATTR_SKIP_CPU_SYNC);
-
- host1x_bo_unpin(mapping->dev, mapping->bo, mapping->sgt);
+ host1x_bo_unpin(mapping->map);
host1x_bo_put(mapping->bo);
kfree(mapping);
@@ -159,6 +155,7 @@ int tegra_drm_ioctl_channel_map(struct drm_device *drm, void *data, struct drm_f
struct drm_tegra_channel_map *args = data;
struct tegra_drm_mapping *mapping;
struct tegra_drm_context *context;
+ enum dma_data_direction direction;
int err = 0;
if (args->flags & ~DRM_TEGRA_CHANNEL_MAP_READ_WRITE)
@@ -180,68 +177,53 @@ int tegra_drm_ioctl_channel_map(struct drm_device *drm, void *data, struct drm_f
kref_init(&mapping->ref);
- mapping->dev = context->client->base.dev;
mapping->bo = tegra_gem_lookup(file, args->handle);
if (!mapping->bo) {
err = -EINVAL;
- goto unlock;
+ goto free;
}
- if (context->client->base.group) {
- /* IOMMU domain managed directly using IOMMU API */
- host1x_bo_pin(mapping->dev, mapping->bo, &mapping->iova);
- } else {
- switch (args->flags & DRM_TEGRA_CHANNEL_MAP_READ_WRITE) {
- case DRM_TEGRA_CHANNEL_MAP_READ_WRITE:
- mapping->direction = DMA_BIDIRECTIONAL;
- break;
+ switch (args->flags & DRM_TEGRA_CHANNEL_MAP_READ_WRITE) {
+ case DRM_TEGRA_CHANNEL_MAP_READ_WRITE:
+ direction = DMA_BIDIRECTIONAL;
+ break;
- case DRM_TEGRA_CHANNEL_MAP_WRITE:
- mapping->direction = DMA_FROM_DEVICE;
- break;
+ case DRM_TEGRA_CHANNEL_MAP_WRITE:
+ direction = DMA_FROM_DEVICE;
+ break;
- case DRM_TEGRA_CHANNEL_MAP_READ:
- mapping->direction = DMA_TO_DEVICE;
- break;
+ case DRM_TEGRA_CHANNEL_MAP_READ:
+ direction = DMA_TO_DEVICE;
+ break;
- default:
- return -EINVAL;
- }
-
- mapping->sgt = host1x_bo_pin(mapping->dev, mapping->bo, NULL);
- if (IS_ERR(mapping->sgt)) {
- err = PTR_ERR(mapping->sgt);
- goto put_gem;
- }
-
- err = dma_map_sgtable(mapping->dev, mapping->sgt, mapping->direction,
- DMA_ATTR_SKIP_CPU_SYNC);
- if (err)
- goto unpin;
-
- mapping->iova = sg_dma_address(mapping->sgt->sgl);
+ default:
+ err = -EINVAL;
+ goto put_gem;
}
+ mapping->map = host1x_bo_pin(context->client->base.dev, mapping->bo, direction, NULL);
+ if (IS_ERR(mapping->map)) {
+ err = PTR_ERR(mapping->map);
+ goto put_gem;
+ }
+
+ mapping->iova = mapping->map->phys;
mapping->iova_end = mapping->iova + host1x_to_tegra_bo(mapping->bo)->gem.size;
err = xa_alloc(&context->mappings, &args->mapping, mapping, XA_LIMIT(1, U32_MAX),
GFP_KERNEL);
if (err < 0)
- goto unmap;
+ goto unpin;
mutex_unlock(&fpriv->lock);
return 0;
-unmap:
- if (mapping->sgt) {
- dma_unmap_sgtable(mapping->dev, mapping->sgt, mapping->direction,
- DMA_ATTR_SKIP_CPU_SYNC);
- }
unpin:
- host1x_bo_unpin(mapping->dev, mapping->bo, mapping->sgt);
+ host1x_bo_unpin(mapping->map);
put_gem:
host1x_bo_put(mapping->bo);
+free:
kfree(mapping);
unlock:
mutex_unlock(&fpriv->lock);
diff --git a/drivers/gpu/drm/tegra/uapi.h b/drivers/gpu/drm/tegra/uapi.h
index 12adad7..92ff1e44f 100644
--- a/drivers/gpu/drm/tegra/uapi.h
+++ b/drivers/gpu/drm/tegra/uapi.h
@@ -27,10 +27,9 @@ struct tegra_drm_file {
struct tegra_drm_mapping {
struct kref ref;
- struct device *dev;
+ struct host1x_bo_mapping *map;
struct host1x_bo *bo;
- struct sg_table *sgt;
- enum dma_data_direction direction;
+
dma_addr_t iova;
dma_addr_t iova_end;
};
diff --git a/drivers/gpu/drm/tegra/vic.c b/drivers/gpu/drm/tegra/vic.c
index c02010f..1e342fa 100644
--- a/drivers/gpu/drm/tegra/vic.c
+++ b/drivers/gpu/drm/tegra/vic.c
@@ -5,6 +5,7 @@
#include <linux/clk.h>
#include <linux/delay.h>
+#include <linux/dma-mapping.h>
#include <linux/host1x.h>
#include <linux/iommu.h>
#include <linux/module.h>
@@ -151,9 +152,13 @@ static int vic_init(struct host1x_client *client)
goto free_channel;
}
+ pm_runtime_enable(client->dev);
+ pm_runtime_use_autosuspend(client->dev);
+ pm_runtime_set_autosuspend_delay(client->dev, 500);
+
err = tegra_drm_register_client(tegra, drm);
if (err < 0)
- goto free_syncpt;
+ goto disable_rpm;
/*
* Inherit the DMA parameters (such as maximum segment size) from the
@@ -163,7 +168,10 @@ static int vic_init(struct host1x_client *client)
return 0;
-free_syncpt:
+disable_rpm:
+ pm_runtime_dont_use_autosuspend(client->dev);
+ pm_runtime_force_suspend(client->dev);
+
host1x_syncpt_put(client->syncpts[0]);
free_channel:
host1x_channel_put(vic->channel);
@@ -188,10 +196,15 @@ static int vic_exit(struct host1x_client *client)
if (err < 0)
return err;
+ pm_runtime_dont_use_autosuspend(client->dev);
+ pm_runtime_force_suspend(client->dev);
+
host1x_syncpt_put(client->syncpts[0]);
host1x_channel_put(vic->channel);
host1x_client_iommu_detach(client);
+ vic->channel = NULL;
+
if (client->group) {
dma_unmap_single(vic->dev, vic->falcon.firmware.phys,
vic->falcon.firmware.size, DMA_TO_DEVICE);
@@ -232,12 +245,12 @@ static int vic_load_firmware(struct vic *vic)
if (!client->group) {
virt = dma_alloc_coherent(vic->dev, size, &iova, GFP_KERNEL);
-
- err = dma_mapping_error(vic->dev, iova);
- if (err < 0)
- return err;
+ if (!virt)
+ return -ENOMEM;
} else {
virt = tegra_drm_alloc(tegra, size, &iova);
+ if (IS_ERR(virt))
+ return PTR_ERR(virt);
}
vic->falcon.firmware.virt = virt;
@@ -315,6 +328,8 @@ static int vic_runtime_suspend(struct device *dev)
struct vic *vic = dev_get_drvdata(dev);
int err;
+ host1x_channel_stop(vic->channel);
+
err = reset_control_assert(vic->rst);
if (err < 0)
return err;
@@ -330,27 +345,17 @@ static int vic_open_channel(struct tegra_drm_client *client,
struct tegra_drm_context *context)
{
struct vic *vic = to_vic(client);
- int err;
-
- err = pm_runtime_resume_and_get(vic->dev);
- if (err < 0)
- return err;
context->channel = host1x_channel_get(vic->channel);
- if (!context->channel) {
- pm_runtime_put(vic->dev);
+ if (!context->channel)
return -ENOMEM;
- }
return 0;
}
static void vic_close_channel(struct tegra_drm_context *context)
{
- struct vic *vic = to_vic(context->client);
-
host1x_channel_put(context->channel);
- pm_runtime_put(vic->dev);
}
static const struct tegra_drm_client_ops vic_ops = {
@@ -441,6 +446,12 @@ static int vic_probe(struct platform_device *pdev)
return PTR_ERR(vic->clk);
}
+ err = clk_set_rate(vic->clk, ULONG_MAX);
+ if (err < 0) {
+ dev_err(&pdev->dev, "failed to set clock rate\n");
+ return err;
+ }
+
if (!dev->pm_domain) {
vic->rst = devm_reset_control_get(dev, "vic");
if (IS_ERR(vic->rst)) {
@@ -476,17 +487,8 @@ static int vic_probe(struct platform_device *pdev)
goto exit_falcon;
}
- pm_runtime_enable(&pdev->dev);
- if (!pm_runtime_enabled(&pdev->dev)) {
- err = vic_runtime_resume(&pdev->dev);
- if (err < 0)
- goto unregister_client;
- }
-
return 0;
-unregister_client:
- host1x_client_unregister(&vic->client.base);
exit_falcon:
falcon_exit(&vic->falcon);
@@ -505,11 +507,6 @@ static int vic_remove(struct platform_device *pdev)
return err;
}
- if (pm_runtime_enabled(&pdev->dev))
- pm_runtime_disable(&pdev->dev);
- else
- vic_runtime_suspend(&pdev->dev);
-
falcon_exit(&vic->falcon);
return 0;
@@ -517,6 +514,8 @@ static int vic_remove(struct platform_device *pdev)
static const struct dev_pm_ops vic_pm_ops = {
SET_RUNTIME_PM_OPS(vic_runtime_suspend, vic_runtime_resume, NULL)
+ SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend,
+ pm_runtime_force_resume)
};
struct platform_driver tegra_vic_driver = {
diff --git a/drivers/gpu/drm/tidss/Kconfig b/drivers/gpu/drm/tidss/Kconfig
index f790a52..bc4fa59 100644
--- a/drivers/gpu/drm/tidss/Kconfig
+++ b/drivers/gpu/drm/tidss/Kconfig
@@ -3,7 +3,6 @@
depends on DRM && OF
depends on ARM || ARM64 || COMPILE_TEST
select DRM_KMS_HELPER
- select DRM_KMS_CMA_HELPER
select DRM_GEM_CMA_HELPER
help
The TI Keystone family SoCs introduced a new generation of
diff --git a/drivers/gpu/drm/tidss/tidss_drv.c b/drivers/gpu/drm/tidss/tidss_drv.c
index d620f35..7c784e9 100644
--- a/drivers/gpu/drm/tidss/tidss_drv.c
+++ b/drivers/gpu/drm/tidss/tidss_drv.c
@@ -88,16 +88,11 @@ static int __maybe_unused tidss_resume(struct device *dev)
return drm_mode_config_helper_resume(&tidss->ddev);
}
-#ifdef CONFIG_PM
-
-static const struct dev_pm_ops tidss_pm_ops = {
- .runtime_suspend = tidss_pm_runtime_suspend,
- .runtime_resume = tidss_pm_runtime_resume,
+static __maybe_unused const struct dev_pm_ops tidss_pm_ops = {
SET_SYSTEM_SLEEP_PM_OPS(tidss_suspend, tidss_resume)
+ SET_RUNTIME_PM_OPS(tidss_pm_runtime_suspend, tidss_pm_runtime_resume, NULL)
};
-#endif /* CONFIG_PM */
-
/* DRM device Information */
static void tidss_release(struct drm_device *ddev)
@@ -250,9 +245,7 @@ static struct platform_driver tidss_platform_driver = {
.shutdown = tidss_shutdown,
.driver = {
.name = "tidss",
-#ifdef CONFIG_PM
- .pm = &tidss_pm_ops,
-#endif
+ .pm = pm_ptr(&tidss_pm_ops),
.of_match_table = tidss_of_table,
.suppress_bind_attrs = true,
},
diff --git a/drivers/gpu/drm/tilcdc/Kconfig b/drivers/gpu/drm/tilcdc/Kconfig
index 9f505a1..e315591 100644
--- a/drivers/gpu/drm/tilcdc/Kconfig
+++ b/drivers/gpu/drm/tilcdc/Kconfig
@@ -3,7 +3,6 @@
tristate "DRM Support for TI LCDC Display Controller"
depends on DRM && OF && ARM
select DRM_KMS_HELPER
- select DRM_KMS_CMA_HELPER
select DRM_GEM_CMA_HELPER
select DRM_BRIDGE
select DRM_PANEL_BRIDGE
diff --git a/drivers/gpu/drm/tiny/Kconfig b/drivers/gpu/drm/tiny/Kconfig
index 1ceb93f..712e000 100644
--- a/drivers/gpu/drm/tiny/Kconfig
+++ b/drivers/gpu/drm/tiny/Kconfig
@@ -3,7 +3,7 @@
config DRM_ARCPGU
tristate "ARC PGU"
depends on DRM && OF
- select DRM_KMS_CMA_HELPER
+ select DRM_GEM_CMA_HELPER
select DRM_KMS_HELPER
help
Choose this option if you have an ARC PGU controller.
@@ -71,7 +71,7 @@
tristate "DRM support for HX8357D display panels"
depends on DRM && SPI
select DRM_KMS_HELPER
- select DRM_KMS_CMA_HELPER
+ select DRM_GEM_CMA_HELPER
select DRM_MIPI_DBI
select BACKLIGHT_CLASS_DEVICE
help
@@ -80,11 +80,24 @@
If M is selected the module will be called hx8357d.
+config TINYDRM_ILI9163
+ tristate "DRM support for ILI9163 display panels"
+ depends on DRM && SPI
+ select BACKLIGHT_CLASS_DEVICE
+ select DRM_GEM_CMA_HELPER
+ select DRM_KMS_HELPER
+ select DRM_MIPI_DBI
+ help
+ DRM driver for the following Ilitek ILI9163 panels:
+ * NHD-1.8-128160EF 128x160 TFT
+
+ If M is selected the module will be called ili9163.
+
config TINYDRM_ILI9225
tristate "DRM support for ILI9225 display panels"
depends on DRM && SPI
select DRM_KMS_HELPER
- select DRM_KMS_CMA_HELPER
+ select DRM_GEM_CMA_HELPER
select DRM_MIPI_DBI
help
DRM driver for the following Ilitek ILI9225 panels:
@@ -96,7 +109,7 @@
tristate "DRM support for ILI9341 display panels"
depends on DRM && SPI
select DRM_KMS_HELPER
- select DRM_KMS_CMA_HELPER
+ select DRM_GEM_CMA_HELPER
select DRM_MIPI_DBI
select BACKLIGHT_CLASS_DEVICE
help
@@ -109,7 +122,7 @@
tristate "DRM support for ILI9486 display panels"
depends on DRM && SPI
select DRM_KMS_HELPER
- select DRM_KMS_CMA_HELPER
+ select DRM_GEM_CMA_HELPER
select DRM_MIPI_DBI
select BACKLIGHT_CLASS_DEVICE
help
@@ -123,7 +136,7 @@
tristate "DRM support for MI0283QT"
depends on DRM && SPI
select DRM_KMS_HELPER
- select DRM_KMS_CMA_HELPER
+ select DRM_GEM_CMA_HELPER
select DRM_MIPI_DBI
select BACKLIGHT_CLASS_DEVICE
help
@@ -134,7 +147,7 @@
tristate "DRM support for Pervasive Displays RePaper panels (V231)"
depends on DRM && SPI
select DRM_KMS_HELPER
- select DRM_KMS_CMA_HELPER
+ select DRM_GEM_CMA_HELPER
help
DRM driver for the following Pervasive Displays panels:
1.44" TFT EPD Panel (E1144CS021)
@@ -148,7 +161,7 @@
tristate "DRM support for Sitronix ST7586 display panels"
depends on DRM && SPI
select DRM_KMS_HELPER
- select DRM_KMS_CMA_HELPER
+ select DRM_GEM_CMA_HELPER
select DRM_MIPI_DBI
help
DRM driver for the following Sitronix ST7586 panels:
@@ -160,7 +173,7 @@
tristate "DRM support for Sitronix ST7715R/ST7735R display panels"
depends on DRM && SPI
select DRM_KMS_HELPER
- select DRM_KMS_CMA_HELPER
+ select DRM_GEM_CMA_HELPER
select DRM_MIPI_DBI
select BACKLIGHT_CLASS_DEVICE
help
diff --git a/drivers/gpu/drm/tiny/Makefile b/drivers/gpu/drm/tiny/Makefile
index e099428..5d5505d 100644
--- a/drivers/gpu/drm/tiny/Makefile
+++ b/drivers/gpu/drm/tiny/Makefile
@@ -6,6 +6,7 @@
obj-$(CONFIG_DRM_GM12U320) += gm12u320.o
obj-$(CONFIG_DRM_SIMPLEDRM) += simpledrm.o
obj-$(CONFIG_TINYDRM_HX8357D) += hx8357d.o
+obj-$(CONFIG_TINYDRM_ILI9163) += ili9163.o
obj-$(CONFIG_TINYDRM_ILI9225) += ili9225.o
obj-$(CONFIG_TINYDRM_ILI9341) += ili9341.o
obj-$(CONFIG_TINYDRM_ILI9486) += ili9486.o
diff --git a/drivers/gpu/drm/tiny/bochs.c b/drivers/gpu/drm/tiny/bochs.c
index 2ce3bd9..fc26a1c 100644
--- a/drivers/gpu/drm/tiny/bochs.c
+++ b/drivers/gpu/drm/tiny/bochs.c
@@ -1,6 +1,5 @@
// SPDX-License-Identifier: GPL-2.0-or-later
-#include <linux/console.h>
#include <linux/pci.h>
#include <drm/drm_aperture.h>
@@ -719,7 +718,7 @@ static struct pci_driver bochs_pci_driver = {
static int __init bochs_init(void)
{
- if (vgacon_text_force() && bochs_modeset == -1)
+ if (drm_firmware_drivers_only() && bochs_modeset == -1)
return -EINVAL;
if (bochs_modeset == 0)
diff --git a/drivers/gpu/drm/tiny/cirrus.c b/drivers/gpu/drm/tiny/cirrus.c
index 4611ec4..c95d9ff 100644
--- a/drivers/gpu/drm/tiny/cirrus.c
+++ b/drivers/gpu/drm/tiny/cirrus.c
@@ -16,7 +16,6 @@
* Copyright 1999-2001 Jeff Garzik <jgarzik@pobox.com>
*/
-#include <linux/console.h>
#include <linux/dma-buf-map.h>
#include <linux/module.h>
#include <linux/pci.h>
@@ -317,28 +316,28 @@ static int cirrus_fb_blit_rect(struct drm_framebuffer *fb, const struct dma_buf_
struct drm_rect *rect)
{
struct cirrus_device *cirrus = to_cirrus(fb->dev);
+ void __iomem *dst = cirrus->vram;
void *vmap = map->vaddr; /* TODO: Use mapping abstraction properly */
int idx;
if (!drm_dev_enter(&cirrus->dev, &idx))
return -ENODEV;
- if (cirrus->cpp == fb->format->cpp[0])
- drm_fb_memcpy_dstclip(cirrus->vram, fb->pitches[0],
- vmap, fb, rect);
+ if (cirrus->cpp == fb->format->cpp[0]) {
+ dst += drm_fb_clip_offset(fb->pitches[0], fb->format, rect);
+ drm_fb_memcpy_toio(dst, fb->pitches[0], vmap, fb, rect);
- else if (fb->format->cpp[0] == 4 && cirrus->cpp == 2)
- drm_fb_xrgb8888_to_rgb565_dstclip(cirrus->vram,
- cirrus->pitch,
- vmap, fb, rect, false);
+ } else if (fb->format->cpp[0] == 4 && cirrus->cpp == 2) {
+ dst += drm_fb_clip_offset(cirrus->pitch, fb->format, rect);
+ drm_fb_xrgb8888_to_rgb565_toio(dst, cirrus->pitch, vmap, fb, rect, false);
- else if (fb->format->cpp[0] == 4 && cirrus->cpp == 3)
- drm_fb_xrgb8888_to_rgb888_dstclip(cirrus->vram,
- cirrus->pitch,
- vmap, fb, rect);
+ } else if (fb->format->cpp[0] == 4 && cirrus->cpp == 3) {
+ dst += drm_fb_clip_offset(cirrus->pitch, fb->format, rect);
+ drm_fb_xrgb8888_to_rgb888_toio(dst, cirrus->pitch, vmap, fb, rect);
- else
+ } else {
WARN_ON_ONCE("cpp mismatch");
+ }
drm_dev_exit(idx);
@@ -636,8 +635,9 @@ static struct pci_driver cirrus_pci_driver = {
static int __init cirrus_init(void)
{
- if (vgacon_text_force())
+ if (drm_firmware_drivers_only())
return -EINVAL;
+
return pci_register_driver(&cirrus_pci_driver);
}
diff --git a/drivers/gpu/drm/tiny/ili9163.c b/drivers/gpu/drm/tiny/ili9163.c
new file mode 100644
index 0000000..bcc1813
--- /dev/null
+++ b/drivers/gpu/drm/tiny/ili9163.c
@@ -0,0 +1,225 @@
+// SPDX-License-Identifier: GPL-2.0+
+
+#include <linux/backlight.h>
+#include <linux/delay.h>
+#include <linux/gpio/consumer.h>
+#include <linux/module.h>
+#include <linux/property.h>
+#include <linux/spi/spi.h>
+
+#include <drm/drm_atomic_helper.h>
+#include <drm/drm_drv.h>
+#include <drm/drm_fb_helper.h>
+#include <drm/drm_gem_atomic_helper.h>
+#include <drm/drm_gem_cma_helper.h>
+#include <drm/drm_mipi_dbi.h>
+#include <drm/drm_modeset_helper.h>
+
+#include <video/mipi_display.h>
+
+#define ILI9163_FRMCTR1 0xb1
+
+#define ILI9163_PWCTRL1 0xc0
+#define ILI9163_PWCTRL2 0xc1
+#define ILI9163_VMCTRL1 0xc5
+#define ILI9163_VMCTRL2 0xc7
+#define ILI9163_PWCTRLA 0xcb
+#define ILI9163_PWCTRLB 0xcf
+
+#define ILI9163_EN3GAM 0xf2
+
+#define ILI9163_MADCTL_BGR BIT(3)
+#define ILI9163_MADCTL_MV BIT(5)
+#define ILI9163_MADCTL_MX BIT(6)
+#define ILI9163_MADCTL_MY BIT(7)
+
+static void yx240qv29_enable(struct drm_simple_display_pipe *pipe,
+ struct drm_crtc_state *crtc_state,
+ struct drm_plane_state *plane_state)
+{
+ struct mipi_dbi_dev *dbidev = drm_to_mipi_dbi_dev(pipe->crtc.dev);
+ struct mipi_dbi *dbi = &dbidev->dbi;
+ u8 addr_mode;
+ int ret, idx;
+
+ if (!drm_dev_enter(pipe->crtc.dev, &idx))
+ return;
+
+ DRM_DEBUG_KMS("\n");
+
+ ret = mipi_dbi_poweron_conditional_reset(dbidev);
+ if (ret < 0)
+ goto out_exit;
+ if (ret == 1)
+ goto out_enable;
+
+ /* Gamma */
+ mipi_dbi_command(dbi, MIPI_DCS_SET_GAMMA_CURVE, 0x04);
+ mipi_dbi_command(dbi, ILI9163_EN3GAM, 0x00);
+
+ /* Frame Rate */
+ mipi_dbi_command(dbi, ILI9163_FRMCTR1, 0x0a, 0x14);
+
+ /* Power Control */
+ mipi_dbi_command(dbi, ILI9163_PWCTRL1, 0x0a, 0x00);
+ mipi_dbi_command(dbi, ILI9163_PWCTRL2, 0x02);
+
+ /* VCOM */
+ mipi_dbi_command(dbi, ILI9163_VMCTRL1, 0x2f, 0x3e);
+ mipi_dbi_command(dbi, ILI9163_VMCTRL2, 0x40);
+
+ /* Memory Access Control */
+ mipi_dbi_command(dbi, MIPI_DCS_SET_PIXEL_FORMAT, MIPI_DCS_PIXEL_FMT_16BIT);
+
+ mipi_dbi_command(dbi, MIPI_DCS_EXIT_SLEEP_MODE);
+ msleep(100);
+
+ mipi_dbi_command(dbi, MIPI_DCS_SET_DISPLAY_ON);
+ msleep(100);
+
+out_enable:
+ switch (dbidev->rotation) {
+ default:
+ addr_mode = ILI9163_MADCTL_MX | ILI9163_MADCTL_MY;
+ break;
+ case 90:
+ addr_mode = ILI9163_MADCTL_MX | ILI9163_MADCTL_MV;
+ break;
+ case 180:
+ addr_mode = 0;
+ break;
+ case 270:
+ addr_mode = ILI9163_MADCTL_MY | ILI9163_MADCTL_MV;
+ break;
+ }
+ addr_mode |= ILI9163_MADCTL_BGR;
+ mipi_dbi_command(dbi, MIPI_DCS_SET_ADDRESS_MODE, addr_mode);
+ mipi_dbi_enable_flush(dbidev, crtc_state, plane_state);
+out_exit:
+ drm_dev_exit(idx);
+}
+
+static const struct drm_simple_display_pipe_funcs ili9163_pipe_funcs = {
+ .enable = yx240qv29_enable,
+ .disable = mipi_dbi_pipe_disable,
+ .update = mipi_dbi_pipe_update,
+ .prepare_fb = drm_gem_simple_display_pipe_prepare_fb,
+};
+
+static const struct drm_display_mode yx240qv29_mode = {
+ DRM_SIMPLE_MODE(128, 160, 28, 35),
+};
+
+DEFINE_DRM_GEM_CMA_FOPS(ili9163_fops);
+
+static struct drm_driver ili9163_driver = {
+ .driver_features = DRIVER_GEM | DRIVER_MODESET | DRIVER_ATOMIC,
+ .fops = &ili9163_fops,
+ DRM_GEM_CMA_DRIVER_OPS_VMAP,
+ .debugfs_init = mipi_dbi_debugfs_init,
+ .name = "ili9163",
+ .desc = "Ilitek ILI9163",
+ .date = "20210208",
+ .major = 1,
+ .minor = 0,
+};
+
+static const struct of_device_id ili9163_of_match[] = {
+ { .compatible = "newhaven,1.8-128160EF" },
+ { }
+};
+MODULE_DEVICE_TABLE(of, ili9163_of_match);
+
+static const struct spi_device_id ili9163_id[] = {
+ { "nhd-1.8-128160EF", 0 },
+ { }
+};
+MODULE_DEVICE_TABLE(spi, ili9163_id);
+
+static int ili9163_probe(struct spi_device *spi)
+{
+ struct device *dev = &spi->dev;
+ struct mipi_dbi_dev *dbidev;
+ struct drm_device *drm;
+ struct mipi_dbi *dbi;
+ struct gpio_desc *dc;
+ u32 rotation = 0;
+ int ret;
+
+ dbidev = devm_drm_dev_alloc(dev, &ili9163_driver,
+ struct mipi_dbi_dev, drm);
+ if (IS_ERR(dbidev))
+ return PTR_ERR(dbidev);
+
+ dbi = &dbidev->dbi;
+ drm = &dbidev->drm;
+
+ spi_set_drvdata(spi, drm);
+
+ dbi->reset = devm_gpiod_get_optional(dev, "reset", GPIOD_OUT_HIGH);
+ if (IS_ERR(dbi->reset)) {
+ DRM_DEV_ERROR(dev, "Failed to get gpio 'reset'\n");
+ return PTR_ERR(dbi->reset);
+ }
+
+ dc = devm_gpiod_get_optional(dev, "dc", GPIOD_OUT_LOW);
+ if (IS_ERR(dc)) {
+ DRM_DEV_ERROR(dev, "Failed to get gpio 'dc'\n");
+ return PTR_ERR(dc);
+ }
+
+ dbidev->backlight = devm_of_find_backlight(dev);
+ if (IS_ERR(dbidev->backlight))
+ return PTR_ERR(dbidev->backlight);
+
+ device_property_read_u32(dev, "rotation", &rotation);
+
+ ret = mipi_dbi_spi_init(spi, dbi, dc);
+ if (ret)
+ return ret;
+
+ ret = mipi_dbi_dev_init(dbidev, &ili9163_pipe_funcs, &yx240qv29_mode, rotation);
+ if (ret)
+ return ret;
+
+ drm_mode_config_reset(drm);
+
+ ret = drm_dev_register(drm, 0);
+ if (ret)
+ return ret;
+
+ drm_fbdev_generic_setup(drm, 0);
+
+ return 0;
+}
+
+static int ili9163_remove(struct spi_device *spi)
+{
+ struct drm_device *drm = spi_get_drvdata(spi);
+
+ drm_dev_unplug(drm);
+ drm_atomic_helper_shutdown(drm);
+
+ return 0;
+}
+
+static void ili9163_shutdown(struct spi_device *spi)
+{
+ drm_atomic_helper_shutdown(spi_get_drvdata(spi));
+}
+
+static struct spi_driver ili9163_spi_driver = {
+ .driver = {
+ .name = "ili9163",
+ .of_match_table = ili9163_of_match,
+ },
+ .id_table = ili9163_id,
+ .probe = ili9163_probe,
+ .remove = ili9163_remove,
+ .shutdown = ili9163_shutdown,
+};
+module_spi_driver(ili9163_spi_driver);
+
+MODULE_DESCRIPTION("Ilitek ILI9163 DRM driver");
+MODULE_AUTHOR("Daniel Mack <daniel@zonque.org>");
+MODULE_LICENSE("GPL");
diff --git a/drivers/gpu/drm/tiny/repaper.c b/drivers/gpu/drm/tiny/repaper.c
index 4d07b21..97a775c48 100644
--- a/drivers/gpu/drm/tiny/repaper.c
+++ b/drivers/gpu/drm/tiny/repaper.c
@@ -560,7 +560,7 @@ static int repaper_fb_dirty(struct drm_framebuffer *fb)
if (ret)
goto out_free;
- drm_fb_xrgb8888_to_gray8(buf, cma_obj->vaddr, fb, &clip);
+ drm_fb_xrgb8888_to_gray8(buf, 0, cma_obj->vaddr, fb, &clip);
drm_gem_fb_end_cpu_access(fb, DMA_FROM_DEVICE);
diff --git a/drivers/gpu/drm/tiny/simpledrm.c b/drivers/gpu/drm/tiny/simpledrm.c
index 5a6e898..04146da2 100644
--- a/drivers/gpu/drm/tiny/simpledrm.c
+++ b/drivers/gpu/drm/tiny/simpledrm.c
@@ -2,6 +2,7 @@
#include <linux/clk.h>
#include <linux/of_clk.h>
+#include <linux/minmax.h>
#include <linux/platform_data/simplefb.h>
#include <linux/platform_device.h>
#include <linux/regulator/consumer.h>
@@ -570,8 +571,8 @@ static const uint32_t simpledrm_default_formats[] = {
//DRM_FORMAT_XRGB1555,
//DRM_FORMAT_ARGB1555,
DRM_FORMAT_RGB888,
- //DRM_FORMAT_XRGB2101010,
- //DRM_FORMAT_ARGB2101010,
+ DRM_FORMAT_XRGB2101010,
+ DRM_FORMAT_ARGB2101010,
};
static const uint64_t simpledrm_format_modifiers[] = {
@@ -641,16 +642,25 @@ simpledrm_simple_display_pipe_enable(struct drm_simple_display_pipe *pipe,
struct drm_framebuffer *fb = plane_state->fb;
void *vmap = shadow_plane_state->data[0].vaddr; /* TODO: Use mapping abstraction */
struct drm_device *dev = &sdev->dev;
+ void __iomem *dst = sdev->screen_base;
+ struct drm_rect src_clip, dst_clip;
int idx;
if (!fb)
return;
+ drm_rect_fp_to_int(&src_clip, &plane_state->src);
+
+ dst_clip = plane_state->dst;
+ if (!drm_rect_intersect(&dst_clip, &src_clip))
+ return;
+
if (!drm_dev_enter(dev, &idx))
return;
- drm_fb_blit_dstclip(sdev->screen_base, sdev->pitch,
- sdev->format->format, vmap, fb);
+ dst += drm_fb_clip_offset(sdev->pitch, sdev->format, &dst_clip);
+ drm_fb_blit_toio(dst, sdev->pitch, sdev->format->format, vmap, fb, &src_clip);
+
drm_dev_exit(idx);
}
@@ -680,20 +690,25 @@ simpledrm_simple_display_pipe_update(struct drm_simple_display_pipe *pipe,
void *vmap = shadow_plane_state->data[0].vaddr; /* TODO: Use mapping abstraction */
struct drm_framebuffer *fb = plane_state->fb;
struct drm_device *dev = &sdev->dev;
- struct drm_rect clip;
+ void __iomem *dst = sdev->screen_base;
+ struct drm_rect src_clip, dst_clip;
int idx;
if (!fb)
return;
- if (!drm_atomic_helper_damage_merged(old_plane_state, plane_state, &clip))
+ if (!drm_atomic_helper_damage_merged(old_plane_state, plane_state, &src_clip))
+ return;
+
+ dst_clip = plane_state->dst;
+ if (!drm_rect_intersect(&dst_clip, &src_clip))
return;
if (!drm_dev_enter(dev, &idx))
return;
- drm_fb_blit_rect_dstclip(sdev->screen_base, sdev->pitch,
- sdev->format->format, vmap, fb, &clip);
+ dst += drm_fb_clip_offset(sdev->pitch, sdev->format, &dst_clip);
+ drm_fb_blit_toio(dst, sdev->pitch, sdev->format->format, vmap, fb, &src_clip);
drm_dev_exit(idx);
}
@@ -758,6 +773,7 @@ static int simpledrm_device_init_modeset(struct simpledrm_device *sdev)
struct drm_display_mode *mode = &sdev->mode;
struct drm_connector *connector = &sdev->connector;
struct drm_simple_display_pipe *pipe = &sdev->pipe;
+ unsigned long max_width, max_height;
const uint32_t *formats;
size_t nformats;
int ret;
@@ -766,10 +782,13 @@ static int simpledrm_device_init_modeset(struct simpledrm_device *sdev)
if (ret)
return ret;
+ max_width = max_t(unsigned long, mode->hdisplay, DRM_SHADOW_PLANE_MAX_WIDTH);
+ max_height = max_t(unsigned long, mode->vdisplay, DRM_SHADOW_PLANE_MAX_HEIGHT);
+
dev->mode_config.min_width = mode->hdisplay;
- dev->mode_config.max_width = mode->hdisplay;
+ dev->mode_config.max_width = max_width;
dev->mode_config.min_height = mode->vdisplay;
- dev->mode_config.max_height = mode->vdisplay;
+ dev->mode_config.max_height = max_height;
dev->mode_config.prefer_shadow_fbdev = true;
dev->mode_config.preferred_depth = sdev->format->cpp[0] * 8;
dev->mode_config.funcs = &simpledrm_mode_config_funcs;
@@ -788,6 +807,8 @@ static int simpledrm_device_init_modeset(struct simpledrm_device *sdev)
if (ret)
return ret;
+ drm_plane_enable_fb_damage_clips(&pipe->plane);
+
drm_mode_config_reset(dev);
return 0;
diff --git a/drivers/gpu/drm/tiny/st7586.c b/drivers/gpu/drm/tiny/st7586.c
index ad0faa8..51b9b9f 100644
--- a/drivers/gpu/drm/tiny/st7586.c
+++ b/drivers/gpu/drm/tiny/st7586.c
@@ -73,7 +73,7 @@ static void st7586_xrgb8888_to_gray332(u8 *dst, void *vaddr,
if (!buf)
return;
- drm_fb_xrgb8888_to_gray8(buf, vaddr, fb, clip);
+ drm_fb_xrgb8888_to_gray8(buf, 0, vaddr, fb, clip);
src = buf;
for (y = clip->y1; y < clip->y2; y++) {
diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c
index 047adc4..db3dc7e 100644
--- a/drivers/gpu/drm/ttm/ttm_bo.c
+++ b/drivers/gpu/drm/ttm/ttm_bo.c
@@ -727,6 +727,8 @@ int ttm_mem_evict_first(struct ttm_device *bdev,
ret = ttm_bo_evict(bo, ctx);
if (locked)
ttm_bo_unreserve(bo);
+ else
+ ttm_bo_move_to_lru_tail_unlocked(bo);
ttm_bo_put(bo);
return ret;
@@ -1084,7 +1086,6 @@ int ttm_bo_wait(struct ttm_buffer_object *bo,
if (timeout == 0)
return -EBUSY;
- dma_resv_add_excl_fence(bo->base.resv, NULL);
return 0;
}
EXPORT_SYMBOL(ttm_bo_wait);
diff --git a/drivers/gpu/drm/ttm/ttm_range_manager.c b/drivers/gpu/drm/ttm/ttm_range_manager.c
index 67d68a4..072e0baf 100644
--- a/drivers/gpu/drm/ttm/ttm_range_manager.c
+++ b/drivers/gpu/drm/ttm/ttm_range_manager.c
@@ -128,15 +128,17 @@ static const struct ttm_resource_manager_func ttm_range_manager_func = {
};
/**
- * ttm_range_man_init
+ * ttm_range_man_init_nocheck - Initialise a generic range manager for the
+ * selected memory type.
*
* @bdev: ttm device
* @type: memory manager type
* @use_tt: if the memory manager uses tt
* @p_size: size of area to be managed in pages.
*
- * Initialise a generic range manager for the selected memory type.
* The range manager is installed for this device in the type slot.
+ *
+ * Return: %0 on success or a negative error code on failure
*/
int ttm_range_man_init_nocheck(struct ttm_device *bdev,
unsigned type, bool use_tt,
@@ -166,12 +168,13 @@ int ttm_range_man_init_nocheck(struct ttm_device *bdev,
EXPORT_SYMBOL(ttm_range_man_init_nocheck);
/**
- * ttm_range_man_fini
+ * ttm_range_man_fini_nocheck - Remove the generic range manager from a slot
+ * and tear it down.
*
* @bdev: ttm device
* @type: memory manager type
*
- * Remove the generic range manager from a slot and tear it down.
+ * Return: %0 on success or a negative error code on failure
*/
int ttm_range_man_fini_nocheck(struct ttm_device *bdev,
unsigned type)
diff --git a/drivers/gpu/drm/tve200/Kconfig b/drivers/gpu/drm/tve200/Kconfig
index e2d163c..47a7dbe 100644
--- a/drivers/gpu/drm/tve200/Kconfig
+++ b/drivers/gpu/drm/tve200/Kconfig
@@ -8,7 +8,6 @@
select DRM_BRIDGE
select DRM_PANEL_BRIDGE
select DRM_KMS_HELPER
- select DRM_KMS_CMA_HELPER
select DRM_GEM_CMA_HELPER
select VT_HW_CONSOLE_BINDING if FRAMEBUFFER_CONSOLE
help
diff --git a/drivers/gpu/drm/v3d/v3d_bo.c b/drivers/gpu/drm/v3d/v3d_bo.c
index 6a8731a..6e3113f 100644
--- a/drivers/gpu/drm/v3d/v3d_bo.c
+++ b/drivers/gpu/drm/v3d/v3d_bo.c
@@ -47,18 +47,18 @@ void v3d_free_object(struct drm_gem_object *obj)
/* GPU execution may have dirtied any pages in the BO. */
bo->base.pages_mark_dirty_on_put = true;
- drm_gem_shmem_free_object(obj);
+ drm_gem_shmem_free(&bo->base);
}
static const struct drm_gem_object_funcs v3d_gem_funcs = {
.free = v3d_free_object,
- .print_info = drm_gem_shmem_print_info,
- .pin = drm_gem_shmem_pin,
- .unpin = drm_gem_shmem_unpin,
- .get_sg_table = drm_gem_shmem_get_sg_table,
- .vmap = drm_gem_shmem_vmap,
- .vunmap = drm_gem_shmem_vunmap,
- .mmap = drm_gem_shmem_mmap,
+ .print_info = drm_gem_shmem_object_print_info,
+ .pin = drm_gem_shmem_object_pin,
+ .unpin = drm_gem_shmem_object_unpin,
+ .get_sg_table = drm_gem_shmem_object_get_sg_table,
+ .vmap = drm_gem_shmem_object_vmap,
+ .vunmap = drm_gem_shmem_object_vunmap,
+ .mmap = drm_gem_shmem_object_mmap,
};
/* gem_create_object function for allocating a BO struct and doing
@@ -70,11 +70,11 @@ struct drm_gem_object *v3d_create_object(struct drm_device *dev, size_t size)
struct drm_gem_object *obj;
if (size == 0)
- return NULL;
+ return ERR_PTR(-EINVAL);
bo = kzalloc(sizeof(*bo), GFP_KERNEL);
if (!bo)
- return NULL;
+ return ERR_PTR(-ENOMEM);
obj = &bo->base.base;
obj->funcs = &v3d_gem_funcs;
@@ -95,7 +95,7 @@ v3d_bo_create_finish(struct drm_gem_object *obj)
/* So far we pin the BO in the MMU for its lifetime, so use
* shmem's helper for getting a lifetime sgt.
*/
- sgt = drm_gem_shmem_get_pages_sgt(&bo->base.base);
+ sgt = drm_gem_shmem_get_pages_sgt(&bo->base);
if (IS_ERR(sgt))
return PTR_ERR(sgt);
@@ -141,7 +141,7 @@ struct v3d_bo *v3d_bo_create(struct drm_device *dev, struct drm_file *file_priv,
return bo;
free_obj:
- drm_gem_shmem_free_object(&shmem_obj->base);
+ drm_gem_shmem_free(shmem_obj);
return ERR_PTR(ret);
}
@@ -159,7 +159,7 @@ v3d_prime_import_sg_table(struct drm_device *dev,
ret = v3d_bo_create_finish(obj);
if (ret) {
- drm_gem_shmem_free_object(obj);
+ drm_gem_shmem_free(&to_v3d_bo(obj)->base);
return ERR_PTR(ret);
}
diff --git a/drivers/gpu/drm/v3d/v3d_gem.c b/drivers/gpu/drm/v3d/v3d_gem.c
index e47ae40..c7ed2e1 100644
--- a/drivers/gpu/drm/v3d/v3d_gem.c
+++ b/drivers/gpu/drm/v3d/v3d_gem.c
@@ -774,7 +774,7 @@ v3d_submit_cl_ioctl(struct drm_device *dev, void *data,
if (args->flags & DRM_V3D_SUBMIT_CL_FLUSH_CACHE) {
ret = v3d_job_init(v3d, file_priv, (void *)&clean_job, sizeof(*clean_job),
- v3d_job_free, 0, 0, V3D_CACHE_CLEAN);
+ v3d_job_free, 0, NULL, V3D_CACHE_CLEAN);
if (ret)
goto fail;
@@ -1007,7 +1007,7 @@ v3d_submit_csd_ioctl(struct drm_device *dev, void *data,
goto fail;
ret = v3d_job_init(v3d, file_priv, (void *)&clean_job, sizeof(*clean_job),
- v3d_job_free, 0, 0, V3D_CACHE_CLEAN);
+ v3d_job_free, 0, NULL, V3D_CACHE_CLEAN);
if (ret)
goto fail;
diff --git a/drivers/gpu/drm/vboxvideo/vbox_drv.c b/drivers/gpu/drm/vboxvideo/vbox_drv.c
index a6c81af..f35d9e4 100644
--- a/drivers/gpu/drm/vboxvideo/vbox_drv.c
+++ b/drivers/gpu/drm/vboxvideo/vbox_drv.c
@@ -7,7 +7,6 @@
* Michael Thayer <michael.thayer@oracle.com,
* Hans de Goede <hdegoede@redhat.com>
*/
-#include <linux/console.h>
#include <linux/module.h>
#include <linux/pci.h>
#include <linux/vt_kern.h>
@@ -193,10 +192,8 @@ static const struct drm_driver driver = {
static int __init vbox_init(void)
{
-#ifdef CONFIG_VGA_CONSOLE
- if (vgacon_text_force() && vbox_modeset == -1)
+ if (drm_firmware_drivers_only() && vbox_modeset == -1)
return -EINVAL;
-#endif
if (vbox_modeset == 0)
return -EINVAL;
diff --git a/drivers/gpu/drm/vboxvideo/vbox_main.c b/drivers/gpu/drm/vboxvideo/vbox_main.c
index f287797..c9e8b3a 100644
--- a/drivers/gpu/drm/vboxvideo/vbox_main.c
+++ b/drivers/gpu/drm/vboxvideo/vbox_main.c
@@ -127,8 +127,8 @@ int vbox_hw_init(struct vbox_private *vbox)
/* Create guest-heap mem-pool use 2^4 = 16 byte chunks */
vbox->guest_pool = devm_gen_pool_create(vbox->ddev.dev, 4, -1,
"vboxvideo-accel");
- if (!vbox->guest_pool)
- return -ENOMEM;
+ if (IS_ERR(vbox->guest_pool))
+ return PTR_ERR(vbox->guest_pool);
ret = gen_pool_add_virt(vbox->guest_pool,
(unsigned long)vbox->guest_heap,
diff --git a/drivers/gpu/drm/vc4/Kconfig b/drivers/gpu/drm/vc4/Kconfig
index 345a557..de3424f 100644
--- a/drivers/gpu/drm/vc4/Kconfig
+++ b/drivers/gpu/drm/vc4/Kconfig
@@ -6,7 +6,6 @@
depends on SND && SND_SOC
depends on COMMON_CLK
select DRM_KMS_HELPER
- select DRM_KMS_CMA_HELPER
select DRM_GEM_CMA_HELPER
select DRM_PANEL_BRIDGE
select SND_PCM
diff --git a/drivers/gpu/drm/vc4/vc4_bo.c b/drivers/gpu/drm/vc4/vc4_bo.c
index f642bd6..6d1281a 100644
--- a/drivers/gpu/drm/vc4/vc4_bo.c
+++ b/drivers/gpu/drm/vc4/vc4_bo.c
@@ -177,7 +177,7 @@ static void vc4_bo_destroy(struct vc4_bo *bo)
bo->validated_shader = NULL;
}
- drm_gem_cma_free_object(obj);
+ drm_gem_cma_free(&bo->base);
}
static void vc4_bo_remove_from_cache(struct vc4_bo *bo)
@@ -391,7 +391,7 @@ struct drm_gem_object *vc4_create_object(struct drm_device *dev, size_t size)
bo = kzalloc(sizeof(*bo), GFP_KERNEL);
if (!bo)
- return NULL;
+ return ERR_PTR(-ENOMEM);
bo->madv = VC4_MADV_WILLNEED;
refcount_set(&bo->usecnt, 0);
@@ -720,7 +720,7 @@ static int vc4_gem_object_mmap(struct drm_gem_object *obj, struct vm_area_struct
return -EINVAL;
}
- return drm_gem_cma_mmap(obj, vma);
+ return drm_gem_cma_mmap(&bo->base, vma);
}
static const struct vm_operations_struct vc4_vm_ops = {
@@ -732,8 +732,8 @@ static const struct vm_operations_struct vc4_vm_ops = {
static const struct drm_gem_object_funcs vc4_gem_object_funcs = {
.free = vc4_free_object,
.export = vc4_prime_export,
- .get_sg_table = drm_gem_cma_get_sg_table,
- .vmap = drm_gem_cma_vmap,
+ .get_sg_table = drm_gem_cma_object_get_sg_table,
+ .vmap = drm_gem_cma_object_vmap,
.mmap = vc4_gem_object_mmap,
.vm_ops = &vc4_vm_ops,
};
diff --git a/drivers/gpu/drm/vc4/vc4_crtc.c b/drivers/gpu/drm/vc4/vc4_crtc.c
index 18f5009..287dbc8 100644
--- a/drivers/gpu/drm/vc4/vc4_crtc.c
+++ b/drivers/gpu/drm/vc4/vc4_crtc.c
@@ -32,6 +32,7 @@
#include <linux/clk.h>
#include <linux/component.h>
#include <linux/of_device.h>
+#include <linux/pm_runtime.h>
#include <drm/drm_atomic.h>
#include <drm/drm_atomic_helper.h>
@@ -42,6 +43,7 @@
#include <drm/drm_vblank.h>
#include "vc4_drv.h"
+#include "vc4_hdmi.h"
#include "vc4_regs.h"
#define HVS_FIFO_LATENCY_PIX 6
@@ -279,27 +281,15 @@ static u32 vc4_crtc_get_fifo_full_level_bits(struct vc4_crtc *vc4_crtc,
* allows drivers to push pixels to more than one encoder from the
* same CRTC.
*/
-static struct drm_encoder *vc4_get_crtc_encoder(struct drm_crtc *crtc,
- struct drm_atomic_state *state,
- struct drm_connector_state *(*get_state)(struct drm_atomic_state *state,
- struct drm_connector *connector))
+struct drm_encoder *vc4_get_crtc_encoder(struct drm_crtc *crtc,
+ struct drm_crtc_state *state)
{
- struct drm_connector *connector;
- struct drm_connector_list_iter conn_iter;
+ struct drm_encoder *encoder;
- drm_connector_list_iter_begin(crtc->dev, &conn_iter);
- drm_for_each_connector_iter(connector, &conn_iter) {
- struct drm_connector_state *conn_state = get_state(state, connector);
+ WARN_ON(hweight32(state->encoder_mask) > 1);
- if (!conn_state)
- continue;
-
- if (conn_state->crtc == crtc) {
- drm_connector_list_iter_end(&conn_iter);
- return connector->encoder;
- }
- }
- drm_connector_list_iter_end(&conn_iter);
+ drm_for_each_encoder_mask(encoder, crtc->dev, state->encoder_mask)
+ return encoder;
return NULL;
}
@@ -313,12 +303,11 @@ static void vc4_crtc_pixelvalve_reset(struct drm_crtc *crtc)
CRTC_WRITE(PV_CONTROL, CRTC_READ(PV_CONTROL) | PV_CONTROL_FIFO_CLR);
}
-static void vc4_crtc_config_pv(struct drm_crtc *crtc, struct drm_atomic_state *state)
+static void vc4_crtc_config_pv(struct drm_crtc *crtc, struct drm_encoder *encoder,
+ struct drm_atomic_state *state)
{
struct drm_device *dev = crtc->dev;
struct vc4_dev *vc4 = to_vc4_dev(dev);
- struct drm_encoder *encoder = vc4_get_crtc_encoder(crtc, state,
- drm_atomic_get_new_connector_state);
struct vc4_encoder *vc4_encoder = to_vc4_encoder(encoder);
struct vc4_crtc *vc4_crtc = to_vc4_crtc(crtc);
const struct vc4_pv_data *pv_data = vc4_crtc_to_vc4_pv_data(vc4_crtc);
@@ -496,8 +485,10 @@ int vc4_crtc_disable_at_boot(struct drm_crtc *crtc)
enum vc4_encoder_type encoder_type;
const struct vc4_pv_data *pv_data;
struct drm_encoder *encoder;
+ struct vc4_hdmi *vc4_hdmi;
unsigned encoder_sel;
int channel;
+ int ret;
if (!(of_device_is_compatible(vc4_crtc->pdev->dev.of_node,
"brcm,bcm2711-pixelvalve2") ||
@@ -525,7 +516,20 @@ int vc4_crtc_disable_at_boot(struct drm_crtc *crtc)
if (WARN_ON(!encoder))
return 0;
- return vc4_crtc_disable(crtc, encoder, NULL, channel);
+ vc4_hdmi = encoder_to_vc4_hdmi(encoder);
+ ret = pm_runtime_resume_and_get(&vc4_hdmi->pdev->dev);
+ if (ret)
+ return ret;
+
+ ret = vc4_crtc_disable(crtc, encoder, NULL, channel);
+ if (ret)
+ return ret;
+
+ ret = pm_runtime_put(&vc4_hdmi->pdev->dev);
+ if (ret)
+ return ret;
+
+ return 0;
}
static void vc4_crtc_atomic_disable(struct drm_crtc *crtc,
@@ -534,10 +538,12 @@ static void vc4_crtc_atomic_disable(struct drm_crtc *crtc,
struct drm_crtc_state *old_state = drm_atomic_get_old_crtc_state(state,
crtc);
struct vc4_crtc_state *old_vc4_state = to_vc4_crtc_state(old_state);
- struct drm_encoder *encoder = vc4_get_crtc_encoder(crtc, state,
- drm_atomic_get_old_connector_state);
+ struct drm_encoder *encoder = vc4_get_crtc_encoder(crtc, old_state);
struct drm_device *dev = crtc->dev;
+ drm_dbg(dev, "Disabling CRTC %s (%u) connected to Encoder %s (%u)",
+ crtc->name, crtc->base.id, encoder->name, encoder->base.id);
+
require_hvs_enabled(dev);
/* Disable vblank irq handling before crtc is disabled. */
@@ -562,12 +568,16 @@ static void vc4_crtc_atomic_disable(struct drm_crtc *crtc,
static void vc4_crtc_atomic_enable(struct drm_crtc *crtc,
struct drm_atomic_state *state)
{
+ struct drm_crtc_state *new_state = drm_atomic_get_new_crtc_state(state,
+ crtc);
struct drm_device *dev = crtc->dev;
struct vc4_crtc *vc4_crtc = to_vc4_crtc(crtc);
- struct drm_encoder *encoder = vc4_get_crtc_encoder(crtc, state,
- drm_atomic_get_new_connector_state);
+ struct drm_encoder *encoder = vc4_get_crtc_encoder(crtc, new_state);
struct vc4_encoder *vc4_encoder = to_vc4_encoder(encoder);
+ drm_dbg(dev, "Enabling CRTC %s (%u) connected to Encoder %s (%u)",
+ crtc->name, crtc->base.id, encoder->name, encoder->base.id);
+
require_hvs_enabled(dev);
/* Enable vblank irq handling before crtc is started otherwise
@@ -580,7 +590,7 @@ static void vc4_crtc_atomic_enable(struct drm_crtc *crtc,
if (vc4_encoder->pre_crtc_configure)
vc4_encoder->pre_crtc_configure(encoder, state);
- vc4_crtc_config_pv(crtc, state);
+ vc4_crtc_config_pv(crtc, encoder, state);
CRTC_WRITE(PV_CONTROL, CRTC_READ(PV_CONTROL) | PV_CONTROL_EN);
@@ -649,12 +659,27 @@ static int vc4_crtc_atomic_check(struct drm_crtc *crtc,
struct vc4_crtc_state *vc4_state = to_vc4_crtc_state(crtc_state);
struct drm_connector *conn;
struct drm_connector_state *conn_state;
+ struct drm_encoder *encoder;
int ret, i;
ret = vc4_hvs_atomic_check(crtc, state);
if (ret)
return ret;
+ encoder = vc4_get_crtc_encoder(crtc, crtc_state);
+ if (encoder) {
+ const struct drm_display_mode *mode = &crtc_state->adjusted_mode;
+ struct vc4_encoder *vc4_encoder = to_vc4_encoder(encoder);
+
+ mode = &crtc_state->adjusted_mode;
+ if (vc4_encoder->type == VC4_ENCODER_TYPE_HDMI0) {
+ vc4_state->hvs_load = max(mode->clock * mode->hdisplay / mode->htotal + 1000,
+ mode->clock * 9 / 10) * 1000;
+ } else {
+ vc4_state->hvs_load = mode->clock * 1000;
+ }
+ }
+
for_each_new_connector_in_state(state, conn, conn_state,
i) {
if (conn_state->crtc != crtc)
@@ -691,14 +716,14 @@ static void vc4_crtc_handle_page_flip(struct vc4_crtc *vc4_crtc)
struct drm_crtc *crtc = &vc4_crtc->base;
struct drm_device *dev = crtc->dev;
struct vc4_dev *vc4 = to_vc4_dev(dev);
- struct vc4_crtc_state *vc4_state = to_vc4_crtc_state(crtc->state);
- u32 chan = vc4_state->assigned_channel;
+ u32 chan = vc4_crtc->current_hvs_channel;
unsigned long flags;
spin_lock_irqsave(&dev->event_lock, flags);
+ spin_lock(&vc4_crtc->irq_lock);
if (vc4_crtc->event &&
- (vc4_state->mm.start == HVS_READ(SCALER_DISPLACTX(chan)) ||
- vc4_state->feed_txp)) {
+ (vc4_crtc->current_dlist == HVS_READ(SCALER_DISPLACTX(chan)) ||
+ vc4_crtc->feeds_txp)) {
drm_crtc_send_vblank_event(crtc, vc4_crtc->event);
vc4_crtc->event = NULL;
drm_crtc_vblank_put(crtc);
@@ -711,6 +736,7 @@ static void vc4_crtc_handle_page_flip(struct vc4_crtc *vc4_crtc)
*/
vc4_hvs_unmask_underrun(dev, chan);
}
+ spin_unlock(&vc4_crtc->irq_lock);
spin_unlock_irqrestore(&dev->event_lock, flags);
}
@@ -876,7 +902,6 @@ struct drm_crtc_state *vc4_crtc_duplicate_state(struct drm_crtc *crtc)
return NULL;
old_vc4_state = to_vc4_crtc_state(crtc->state);
- vc4_state->feed_txp = old_vc4_state->feed_txp;
vc4_state->margins = old_vc4_state->margins;
vc4_state->assigned_channel = old_vc4_state->assigned_channel;
@@ -937,6 +962,7 @@ static const struct drm_crtc_funcs vc4_crtc_funcs = {
static const struct drm_crtc_helper_funcs vc4_crtc_helper_funcs = {
.mode_valid = vc4_crtc_mode_valid,
.atomic_check = vc4_crtc_atomic_check,
+ .atomic_begin = vc4_hvs_atomic_begin,
.atomic_flush = vc4_hvs_atomic_flush,
.atomic_enable = vc4_crtc_atomic_enable,
.atomic_disable = vc4_crtc_atomic_disable,
@@ -1111,6 +1137,7 @@ int vc4_crtc_init(struct drm_device *drm, struct vc4_crtc *vc4_crtc,
return PTR_ERR(primary_plane);
}
+ spin_lock_init(&vc4_crtc->irq_lock);
drm_crtc_init_with_planes(drm, crtc, primary_plane, NULL,
crtc_funcs, NULL);
drm_crtc_helper_add(crtc, crtc_helper_funcs);
diff --git a/drivers/gpu/drm/vc4/vc4_debugfs.c b/drivers/gpu/drm/vc4/vc4_debugfs.c
index 6da22af..ba2d8ea 100644
--- a/drivers/gpu/drm/vc4/vc4_debugfs.c
+++ b/drivers/gpu/drm/vc4/vc4_debugfs.c
@@ -7,6 +7,7 @@
#include <linux/circ_buf.h>
#include <linux/ctype.h>
#include <linux/debugfs.h>
+#include <linux/platform_device.h>
#include "vc4_drv.h"
#include "vc4_regs.h"
@@ -26,8 +27,10 @@ vc4_debugfs_init(struct drm_minor *minor)
struct vc4_dev *vc4 = to_vc4_dev(minor->dev);
struct vc4_debugfs_info_entry *entry;
- debugfs_create_bool("hvs_load_tracker", S_IRUGO | S_IWUSR,
- minor->debugfs_root, &vc4->load_tracker_enabled);
+ if (!of_device_is_compatible(vc4->hvs->pdev->dev.of_node,
+ "brcm,bcm2711-vc5"))
+ debugfs_create_bool("hvs_load_tracker", S_IRUGO | S_IWUSR,
+ minor->debugfs_root, &vc4->load_tracker_enabled);
list_for_each_entry(entry, &vc4->debugfs_list, link) {
drm_debugfs_create_files(&entry->info, 1,
diff --git a/drivers/gpu/drm/vc4/vc4_drv.h b/drivers/gpu/drm/vc4/vc4_drv.h
index ef73e0a..4329e09 100644
--- a/drivers/gpu/drm/vc4/vc4_drv.h
+++ b/drivers/gpu/drm/vc4/vc4_drv.h
@@ -202,9 +202,6 @@ struct vc4_dev {
int power_refcount;
- /* Set to true when the load tracker is supported. */
- bool load_tracker_available;
-
/* Set to true when the load tracker is active. */
bool load_tracker_enabled;
@@ -495,6 +492,33 @@ struct vc4_crtc {
struct drm_pending_vblank_event *event;
struct debugfs_regset32 regset;
+
+ /**
+ * @feeds_txp: True if the CRTC feeds our writeback controller.
+ */
+ bool feeds_txp;
+
+ /**
+ * @irq_lock: Spinlock protecting the resources shared between
+ * the atomic code and our vblank handler.
+ */
+ spinlock_t irq_lock;
+
+ /**
+ * @current_dlist: Start offset of the display list currently
+ * set in the HVS for that CRTC. Protected by @irq_lock, and
+ * copied in vc4_hvs_update_dlist() for the CRTC interrupt
+ * handler to have access to that value.
+ */
+ unsigned int current_dlist;
+
+ /**
+ * @current_hvs_channel: HVS channel currently assigned to the
+ * CRTC. Protected by @irq_lock, and copied in
+ * vc4_hvs_atomic_begin() for the CRTC interrupt handler to have
+ * access to that value.
+ */
+ unsigned int current_hvs_channel;
};
static inline struct vc4_crtc *
@@ -517,11 +541,13 @@ vc4_crtc_to_vc4_pv_data(const struct vc4_crtc *crtc)
return container_of(data, struct vc4_pv_data, base);
}
+struct drm_encoder *vc4_get_crtc_encoder(struct drm_crtc *crtc,
+ struct drm_crtc_state *state);
+
struct vc4_crtc_state {
struct drm_crtc_state base;
/* Dlist area for this CRTC configuration. */
struct drm_mm_node mm;
- bool feed_txp;
bool txp_armed;
unsigned int assigned_channel;
@@ -532,6 +558,8 @@ struct vc4_crtc_state {
unsigned int bottom;
} margins;
+ unsigned long hvs_load;
+
/* Transitional state below, only valid during atomic commits */
bool update_muxing;
};
@@ -908,6 +936,7 @@ extern struct platform_driver vc4_hvs_driver;
void vc4_hvs_stop_channel(struct drm_device *dev, unsigned int output);
int vc4_hvs_get_fifo_from_output(struct drm_device *dev, unsigned int output);
int vc4_hvs_atomic_check(struct drm_crtc *crtc, struct drm_atomic_state *state);
+void vc4_hvs_atomic_begin(struct drm_crtc *crtc, struct drm_atomic_state *state);
void vc4_hvs_atomic_enable(struct drm_crtc *crtc, struct drm_atomic_state *state);
void vc4_hvs_atomic_disable(struct drm_crtc *crtc, struct drm_atomic_state *state);
void vc4_hvs_atomic_flush(struct drm_crtc *crtc, struct drm_atomic_state *state);
diff --git a/drivers/gpu/drm/vc4/vc4_hdmi.c b/drivers/gpu/drm/vc4/vc4_hdmi.c
index b284623..053fbaf 100644
--- a/drivers/gpu/drm/vc4/vc4_hdmi.c
+++ b/drivers/gpu/drm/vc4/vc4_hdmi.c
@@ -94,6 +94,7 @@
# define VC4_HD_M_SW_RST BIT(2)
# define VC4_HD_M_ENABLE BIT(0)
+#define HSM_MIN_CLOCK_FREQ 120000000
#define CEC_CLOCK_FREQ 40000
#define HDMI_14_MAX_TMDS_CLK (340 * 1000 * 1000)
@@ -117,6 +118,10 @@ static int vc4_hdmi_debugfs_regs(struct seq_file *m, void *unused)
static void vc4_hdmi_reset(struct vc4_hdmi *vc4_hdmi)
{
+ unsigned long flags;
+
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
+
HDMI_WRITE(HDMI_M_CTL, VC4_HD_M_SW_RST);
udelay(1);
HDMI_WRITE(HDMI_M_CTL, 0);
@@ -128,24 +133,36 @@ static void vc4_hdmi_reset(struct vc4_hdmi *vc4_hdmi)
VC4_HDMI_SW_RESET_FORMAT_DETECT);
HDMI_WRITE(HDMI_SW_RESET_CONTROL, 0);
+
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
}
static void vc5_hdmi_reset(struct vc4_hdmi *vc4_hdmi)
{
+ unsigned long flags;
+
reset_control_reset(vc4_hdmi->reset);
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
+
HDMI_WRITE(HDMI_DVP_CTL, 0);
HDMI_WRITE(HDMI_CLOCK_STOP,
HDMI_READ(HDMI_CLOCK_STOP) | VC4_DVP_HT_CLOCK_STOP_PIXEL);
+
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
}
#ifdef CONFIG_DRM_VC4_HDMI_CEC
static void vc4_hdmi_cec_update_clk_div(struct vc4_hdmi *vc4_hdmi)
{
+ unsigned long cec_rate = clk_get_rate(vc4_hdmi->cec_clock);
+ unsigned long flags;
u16 clk_cnt;
u32 value;
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
+
value = HDMI_READ(HDMI_CEC_CNTRL_1);
value &= ~VC4_HDMI_CEC_DIV_CLK_CNT_MASK;
@@ -153,27 +170,41 @@ static void vc4_hdmi_cec_update_clk_div(struct vc4_hdmi *vc4_hdmi)
* Set the clock divider: the hsm_clock rate and this divider
* setting will give a 40 kHz CEC clock.
*/
- clk_cnt = clk_get_rate(vc4_hdmi->cec_clock) / CEC_CLOCK_FREQ;
+ clk_cnt = cec_rate / CEC_CLOCK_FREQ;
value |= clk_cnt << VC4_HDMI_CEC_DIV_CLK_CNT_SHIFT;
HDMI_WRITE(HDMI_CEC_CNTRL_1, value);
+
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
}
#else
static void vc4_hdmi_cec_update_clk_div(struct vc4_hdmi *vc4_hdmi) {}
#endif
+static void vc4_hdmi_enable_scrambling(struct drm_encoder *encoder);
+
static enum drm_connector_status
vc4_hdmi_connector_detect(struct drm_connector *connector, bool force)
{
struct vc4_hdmi *vc4_hdmi = connector_to_vc4_hdmi(connector);
bool connected = false;
- if (vc4_hdmi->hpd_gpio &&
- gpiod_get_value_cansleep(vc4_hdmi->hpd_gpio)) {
- connected = true;
- } else if (drm_probe_ddc(vc4_hdmi->ddc)) {
- connected = true;
- } else if (HDMI_READ(HDMI_HOTPLUG) & VC4_HDMI_HOTPLUG_CONNECTED) {
- connected = true;
+ mutex_lock(&vc4_hdmi->mutex);
+
+ WARN_ON(pm_runtime_resume_and_get(&vc4_hdmi->pdev->dev));
+
+ if (vc4_hdmi->hpd_gpio) {
+ if (gpiod_get_value_cansleep(vc4_hdmi->hpd_gpio))
+ connected = true;
+ } else {
+ unsigned long flags;
+ u32 hotplug;
+
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
+ hotplug = HDMI_READ(HDMI_HOTPLUG);
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
+
+ if (hotplug & VC4_HDMI_HOTPLUG_CONNECTED)
+ connected = true;
}
if (connected) {
@@ -187,10 +218,15 @@ vc4_hdmi_connector_detect(struct drm_connector *connector, bool force)
}
}
+ vc4_hdmi_enable_scrambling(&vc4_hdmi->encoder.base.base);
+ pm_runtime_put(&vc4_hdmi->pdev->dev);
+ mutex_unlock(&vc4_hdmi->mutex);
return connector_status_connected;
}
cec_phys_addr_invalidate(vc4_hdmi->cec_adap);
+ pm_runtime_put(&vc4_hdmi->pdev->dev);
+ mutex_unlock(&vc4_hdmi->mutex);
return connector_status_disconnected;
}
@@ -207,10 +243,14 @@ static int vc4_hdmi_connector_get_modes(struct drm_connector *connector)
int ret = 0;
struct edid *edid;
+ mutex_lock(&vc4_hdmi->mutex);
+
edid = drm_get_edid(connector, vc4_hdmi->ddc);
cec_s_phys_addr_from_edid(vc4_hdmi->cec_adap, edid);
- if (!edid)
- return -ENODEV;
+ if (!edid) {
+ ret = -ENODEV;
+ goto out;
+ }
vc4_encoder->hdmi_monitor = drm_detect_hdmi_monitor(edid);
@@ -230,6 +270,9 @@ static int vc4_hdmi_connector_get_modes(struct drm_connector *connector)
}
}
+out:
+ mutex_unlock(&vc4_hdmi->mutex);
+
return ret;
}
@@ -364,9 +407,12 @@ static int vc4_hdmi_stop_packet(struct drm_encoder *encoder,
{
struct vc4_hdmi *vc4_hdmi = encoder_to_vc4_hdmi(encoder);
u32 packet_id = type - 0x80;
+ unsigned long flags;
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
HDMI_WRITE(HDMI_RAM_PACKET_CONFIG,
HDMI_READ(HDMI_RAM_PACKET_CONFIG) & ~BIT(packet_id));
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
if (!poll)
return 0;
@@ -386,6 +432,7 @@ static void vc4_hdmi_write_infoframe(struct drm_encoder *encoder,
void __iomem *base = __vc4_hdmi_get_field_base(vc4_hdmi,
ram_packet_start->reg);
uint8_t buffer[VC4_HDMI_PACKET_STRIDE];
+ unsigned long flags;
ssize_t len, i;
int ret;
@@ -403,6 +450,8 @@ static void vc4_hdmi_write_infoframe(struct drm_encoder *encoder,
return;
}
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
+
for (i = 0; i < len; i += 7) {
writel(buffer[i + 0] << 0 |
buffer[i + 1] << 8 |
@@ -420,6 +469,9 @@ static void vc4_hdmi_write_infoframe(struct drm_encoder *encoder,
HDMI_WRITE(HDMI_RAM_PACKET_CONFIG,
HDMI_READ(HDMI_RAM_PACKET_CONFIG) | BIT(packet_id));
+
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
+
ret = wait_for((HDMI_READ(HDMI_RAM_PACKET_STATUS) &
BIT(packet_id)), 100);
if (ret)
@@ -432,11 +484,12 @@ static void vc4_hdmi_set_avi_infoframe(struct drm_encoder *encoder)
struct vc4_hdmi_encoder *vc4_encoder = to_vc4_hdmi_encoder(encoder);
struct drm_connector *connector = &vc4_hdmi->connector;
struct drm_connector_state *cstate = connector->state;
- struct drm_crtc *crtc = encoder->crtc;
- const struct drm_display_mode *mode = &crtc->state->adjusted_mode;
+ const struct drm_display_mode *mode = &vc4_hdmi->saved_adjusted_mode;
union hdmi_infoframe frame;
int ret;
+ lockdep_assert_held(&vc4_hdmi->mutex);
+
ret = drm_hdmi_avi_infoframe_from_display_mode(&frame.avi,
connector, mode);
if (ret < 0) {
@@ -488,6 +541,8 @@ static void vc4_hdmi_set_hdr_infoframe(struct drm_encoder *encoder)
struct drm_connector_state *conn_state = connector->state;
union hdmi_infoframe frame;
+ lockdep_assert_held(&vc4_hdmi->mutex);
+
if (!vc4_hdmi->variant->supports_hdr)
return;
@@ -504,6 +559,8 @@ static void vc4_hdmi_set_infoframes(struct drm_encoder *encoder)
{
struct vc4_hdmi *vc4_hdmi = encoder_to_vc4_hdmi(encoder);
+ lockdep_assert_held(&vc4_hdmi->mutex);
+
vc4_hdmi_set_avi_infoframe(encoder);
vc4_hdmi_set_spd_infoframe(encoder);
/*
@@ -523,6 +580,8 @@ static bool vc4_hdmi_supports_scrambling(struct drm_encoder *encoder,
struct vc4_hdmi *vc4_hdmi = encoder_to_vc4_hdmi(encoder);
struct drm_display_info *display = &vc4_hdmi->connector.display_info;
+ lockdep_assert_held(&vc4_hdmi->mutex);
+
if (!vc4_encoder->hdmi_monitor)
return false;
@@ -537,8 +596,11 @@ static bool vc4_hdmi_supports_scrambling(struct drm_encoder *encoder,
static void vc4_hdmi_enable_scrambling(struct drm_encoder *encoder)
{
- struct drm_display_mode *mode = &encoder->crtc->state->adjusted_mode;
struct vc4_hdmi *vc4_hdmi = encoder_to_vc4_hdmi(encoder);
+ struct drm_display_mode *mode = &vc4_hdmi->saved_adjusted_mode;
+ unsigned long flags;
+
+ lockdep_assert_held(&vc4_hdmi->mutex);
if (!vc4_hdmi_supports_scrambling(encoder, mode))
return;
@@ -549,8 +611,12 @@ static void vc4_hdmi_enable_scrambling(struct drm_encoder *encoder)
drm_scdc_set_high_tmds_clock_ratio(vc4_hdmi->ddc, true);
drm_scdc_set_scrambling(vc4_hdmi->ddc, true);
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
HDMI_WRITE(HDMI_SCRAMBLER_CTL, HDMI_READ(HDMI_SCRAMBLER_CTL) |
VC5_HDMI_SCRAMBLER_CTL_ENABLE);
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
+
+ vc4_hdmi->scdc_enabled = true;
queue_delayed_work(system_wq, &vc4_hdmi->scrambling_work,
msecs_to_jiffies(SCRAMBLING_POLLING_DELAY_MS));
@@ -559,24 +625,22 @@ static void vc4_hdmi_enable_scrambling(struct drm_encoder *encoder)
static void vc4_hdmi_disable_scrambling(struct drm_encoder *encoder)
{
struct vc4_hdmi *vc4_hdmi = encoder_to_vc4_hdmi(encoder);
- struct drm_crtc *crtc = encoder->crtc;
+ unsigned long flags;
- /*
- * At boot, encoder->crtc will be NULL. Since we don't know the
- * state of the scrambler and in order to avoid any
- * inconsistency, let's disable it all the time.
- */
- if (crtc && !vc4_hdmi_supports_scrambling(encoder, &crtc->mode))
+ lockdep_assert_held(&vc4_hdmi->mutex);
+
+ if (!vc4_hdmi->scdc_enabled)
return;
- if (crtc && !vc4_hdmi_mode_needs_scrambling(&crtc->mode))
- return;
+ vc4_hdmi->scdc_enabled = false;
if (delayed_work_pending(&vc4_hdmi->scrambling_work))
cancel_delayed_work_sync(&vc4_hdmi->scrambling_work);
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
HDMI_WRITE(HDMI_SCRAMBLER_CTL, HDMI_READ(HDMI_SCRAMBLER_CTL) &
~VC5_HDMI_SCRAMBLER_CTL_ENABLE);
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
drm_scdc_set_scrambling(vc4_hdmi->ddc, false);
drm_scdc_set_high_tmds_clock_ratio(vc4_hdmi->ddc, false);
@@ -602,47 +666,73 @@ static void vc4_hdmi_encoder_post_crtc_disable(struct drm_encoder *encoder,
struct drm_atomic_state *state)
{
struct vc4_hdmi *vc4_hdmi = encoder_to_vc4_hdmi(encoder);
+ unsigned long flags;
+
+ mutex_lock(&vc4_hdmi->mutex);
+
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
HDMI_WRITE(HDMI_RAM_PACKET_CONFIG, 0);
HDMI_WRITE(HDMI_VID_CTL, HDMI_READ(HDMI_VID_CTL) | VC4_HD_VID_CTL_CLRRGB);
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
+
mdelay(1);
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
HDMI_WRITE(HDMI_VID_CTL,
HDMI_READ(HDMI_VID_CTL) & ~VC4_HD_VID_CTL_ENABLE);
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
+
vc4_hdmi_disable_scrambling(encoder);
+
+ mutex_unlock(&vc4_hdmi->mutex);
}
static void vc4_hdmi_encoder_post_crtc_powerdown(struct drm_encoder *encoder,
struct drm_atomic_state *state)
{
struct vc4_hdmi *vc4_hdmi = encoder_to_vc4_hdmi(encoder);
+ unsigned long flags;
int ret;
+ mutex_lock(&vc4_hdmi->mutex);
+
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
HDMI_WRITE(HDMI_VID_CTL,
HDMI_READ(HDMI_VID_CTL) | VC4_HD_VID_CTL_BLANKPIX);
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
if (vc4_hdmi->variant->phy_disable)
vc4_hdmi->variant->phy_disable(vc4_hdmi);
clk_disable_unprepare(vc4_hdmi->pixel_bvb_clock);
- clk_disable_unprepare(vc4_hdmi->hsm_clock);
clk_disable_unprepare(vc4_hdmi->pixel_clock);
ret = pm_runtime_put(&vc4_hdmi->pdev->dev);
if (ret < 0)
DRM_ERROR("Failed to release power domain: %d\n", ret);
+
+ mutex_unlock(&vc4_hdmi->mutex);
}
static void vc4_hdmi_encoder_disable(struct drm_encoder *encoder)
{
+ struct vc4_hdmi *vc4_hdmi = encoder_to_vc4_hdmi(encoder);
+
+ mutex_lock(&vc4_hdmi->mutex);
+ vc4_hdmi->output_enabled = false;
+ mutex_unlock(&vc4_hdmi->mutex);
}
static void vc4_hdmi_csc_setup(struct vc4_hdmi *vc4_hdmi, bool enable)
{
+ unsigned long flags;
u32 csc_ctl;
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
+
csc_ctl = VC4_SET_FIELD(VC4_HD_CSC_CTL_ORDER_BGR,
VC4_HD_CSC_CTL_ORDER);
@@ -672,14 +762,19 @@ static void vc4_hdmi_csc_setup(struct vc4_hdmi *vc4_hdmi, bool enable)
/* The RGB order applies even when CSC is disabled. */
HDMI_WRITE(HDMI_CSC_CTL, csc_ctl);
+
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
}
static void vc5_hdmi_csc_setup(struct vc4_hdmi *vc4_hdmi, bool enable)
{
+ unsigned long flags;
u32 csc_ctl;
csc_ctl = 0x07; /* RGB_CONVERT_MODE = custom matrix, || USE_RGB_TO_YCBCR */
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
+
if (enable) {
/* CEA VICs other than #1 requre limited range RGB
* output unless overridden by an AVI infoframe.
@@ -711,6 +806,8 @@ static void vc5_hdmi_csc_setup(struct vc4_hdmi *vc4_hdmi, bool enable)
}
HDMI_WRITE(HDMI_CSC_CTL, csc_ctl);
+
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
}
static void vc4_hdmi_set_timings(struct vc4_hdmi *vc4_hdmi,
@@ -734,6 +831,9 @@ static void vc4_hdmi_set_timings(struct vc4_hdmi *vc4_hdmi,
mode->crtc_vsync_end -
interlaced,
VC4_HDMI_VERTB_VBP));
+ unsigned long flags;
+
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
HDMI_WRITE(HDMI_HORZA,
(vsync_pos ? VC4_HDMI_HORZA_VPOS : 0) |
@@ -757,6 +857,8 @@ static void vc4_hdmi_set_timings(struct vc4_hdmi *vc4_hdmi,
HDMI_WRITE(HDMI_VERTB0, vertb_even);
HDMI_WRITE(HDMI_VERTB1, vertb);
+
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
}
static void vc5_hdmi_set_timings(struct vc4_hdmi *vc4_hdmi,
@@ -780,10 +882,13 @@ static void vc5_hdmi_set_timings(struct vc4_hdmi *vc4_hdmi,
mode->crtc_vsync_end -
interlaced,
VC4_HDMI_VERTB_VBP));
+ unsigned long flags;
unsigned char gcp;
bool gcp_en;
u32 reg;
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
+
HDMI_WRITE(HDMI_VEC_INTERFACE_XBAR, 0x354021);
HDMI_WRITE(HDMI_HORZA,
(vsync_pos ? VC5_HDMI_HORZA_VPOS : 0) |
@@ -842,13 +947,18 @@ static void vc5_hdmi_set_timings(struct vc4_hdmi *vc4_hdmi,
HDMI_WRITE(HDMI_GCP_CONFIG, reg);
HDMI_WRITE(HDMI_CLOCK_STOP, 0);
+
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
}
static void vc4_hdmi_recenter_fifo(struct vc4_hdmi *vc4_hdmi)
{
+ unsigned long flags;
u32 drift;
int ret;
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
+
drift = HDMI_READ(HDMI_FIFO_CTL);
drift &= VC4_HDMI_FIFO_VALID_WRITE_MASK;
@@ -856,12 +966,20 @@ static void vc4_hdmi_recenter_fifo(struct vc4_hdmi *vc4_hdmi)
drift & ~VC4_HDMI_FIFO_CTL_RECENTER);
HDMI_WRITE(HDMI_FIFO_CTL,
drift | VC4_HDMI_FIFO_CTL_RECENTER);
+
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
+
usleep_range(1000, 1100);
+
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
+
HDMI_WRITE(HDMI_FIFO_CTL,
drift & ~VC4_HDMI_FIFO_CTL_RECENTER);
HDMI_WRITE(HDMI_FIFO_CTL,
drift | VC4_HDMI_FIFO_CTL_RECENTER);
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
+
ret = wait_for(HDMI_READ(HDMI_FIFO_CTL) &
VC4_HDMI_FIFO_CTL_RECENTER_DONE, 1);
WARN_ONCE(ret, "Timeout waiting for "
@@ -891,29 +1009,14 @@ static void vc4_hdmi_encoder_pre_crtc_configure(struct drm_encoder *encoder,
vc4_hdmi_encoder_get_connector_state(encoder, state);
struct vc4_hdmi_connector_state *vc4_conn_state =
conn_state_to_vc4_hdmi_conn_state(conn_state);
- struct drm_display_mode *mode = &encoder->crtc->state->adjusted_mode;
struct vc4_hdmi *vc4_hdmi = encoder_to_vc4_hdmi(encoder);
- unsigned long bvb_rate, pixel_rate, hsm_rate;
+ struct drm_display_mode *mode = &vc4_hdmi->saved_adjusted_mode;
+ unsigned long pixel_rate = vc4_conn_state->pixel_rate;
+ unsigned long bvb_rate, hsm_rate;
+ unsigned long flags;
int ret;
- ret = pm_runtime_resume_and_get(&vc4_hdmi->pdev->dev);
- if (ret < 0) {
- DRM_ERROR("Failed to retain power domain: %d\n", ret);
- return;
- }
-
- pixel_rate = vc4_conn_state->pixel_rate;
- ret = clk_set_rate(vc4_hdmi->pixel_clock, pixel_rate);
- if (ret) {
- DRM_ERROR("Failed to set pixel clock rate: %d\n", ret);
- return;
- }
-
- ret = clk_prepare_enable(vc4_hdmi->pixel_clock);
- if (ret) {
- DRM_ERROR("Failed to turn on pixel clock: %d\n", ret);
- return;
- }
+ mutex_lock(&vc4_hdmi->mutex);
/*
* As stated in RPi's vc4 firmware "HDMI state machine (HSM) clock must
@@ -935,16 +1038,28 @@ static void vc4_hdmi_encoder_pre_crtc_configure(struct drm_encoder *encoder,
ret = clk_set_min_rate(vc4_hdmi->hsm_clock, hsm_rate);
if (ret) {
DRM_ERROR("Failed to set HSM clock rate: %d\n", ret);
- return;
+ goto out;
}
- ret = clk_prepare_enable(vc4_hdmi->hsm_clock);
- if (ret) {
- DRM_ERROR("Failed to turn on HSM clock: %d\n", ret);
- clk_disable_unprepare(vc4_hdmi->pixel_clock);
- return;
+ ret = pm_runtime_resume_and_get(&vc4_hdmi->pdev->dev);
+ if (ret < 0) {
+ DRM_ERROR("Failed to retain power domain: %d\n", ret);
+ goto out;
}
+ ret = clk_set_rate(vc4_hdmi->pixel_clock, pixel_rate);
+ if (ret) {
+ DRM_ERROR("Failed to set pixel clock rate: %d\n", ret);
+ goto err_put_runtime_pm;
+ }
+
+ ret = clk_prepare_enable(vc4_hdmi->pixel_clock);
+ if (ret) {
+ DRM_ERROR("Failed to turn on pixel clock: %d\n", ret);
+ goto err_put_runtime_pm;
+ }
+
+
vc4_hdmi_cec_update_clk_div(vc4_hdmi);
if (pixel_rate > 297000000)
@@ -957,37 +1072,52 @@ static void vc4_hdmi_encoder_pre_crtc_configure(struct drm_encoder *encoder,
ret = clk_set_min_rate(vc4_hdmi->pixel_bvb_clock, bvb_rate);
if (ret) {
DRM_ERROR("Failed to set pixel bvb clock rate: %d\n", ret);
- clk_disable_unprepare(vc4_hdmi->hsm_clock);
- clk_disable_unprepare(vc4_hdmi->pixel_clock);
- return;
+ goto err_disable_pixel_clock;
}
ret = clk_prepare_enable(vc4_hdmi->pixel_bvb_clock);
if (ret) {
DRM_ERROR("Failed to turn on pixel bvb clock: %d\n", ret);
- clk_disable_unprepare(vc4_hdmi->hsm_clock);
- clk_disable_unprepare(vc4_hdmi->pixel_clock);
- return;
+ goto err_disable_pixel_clock;
}
if (vc4_hdmi->variant->phy_init)
vc4_hdmi->variant->phy_init(vc4_hdmi, vc4_conn_state);
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
+
HDMI_WRITE(HDMI_SCHEDULER_CONTROL,
HDMI_READ(HDMI_SCHEDULER_CONTROL) |
VC4_HDMI_SCHEDULER_CONTROL_MANUAL_FORMAT |
VC4_HDMI_SCHEDULER_CONTROL_IGNORE_VSYNC_PREDICTS);
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
+
if (vc4_hdmi->variant->set_timings)
vc4_hdmi->variant->set_timings(vc4_hdmi, conn_state, mode);
+
+ mutex_unlock(&vc4_hdmi->mutex);
+
+ return;
+
+err_disable_pixel_clock:
+ clk_disable_unprepare(vc4_hdmi->pixel_clock);
+err_put_runtime_pm:
+ pm_runtime_put(&vc4_hdmi->pdev->dev);
+out:
+ mutex_unlock(&vc4_hdmi->mutex);
+ return;
}
static void vc4_hdmi_encoder_pre_crtc_enable(struct drm_encoder *encoder,
struct drm_atomic_state *state)
{
- struct drm_display_mode *mode = &encoder->crtc->state->adjusted_mode;
- struct vc4_hdmi_encoder *vc4_encoder = to_vc4_hdmi_encoder(encoder);
struct vc4_hdmi *vc4_hdmi = encoder_to_vc4_hdmi(encoder);
+ struct drm_display_mode *mode = &vc4_hdmi->saved_adjusted_mode;
+ struct vc4_hdmi_encoder *vc4_encoder = to_vc4_hdmi_encoder(encoder);
+ unsigned long flags;
+
+ mutex_lock(&vc4_hdmi->mutex);
if (vc4_encoder->hdmi_monitor &&
drm_default_rgb_quant_range(mode) == HDMI_QUANTIZATION_RANGE_LIMITED) {
@@ -1002,19 +1132,28 @@ static void vc4_hdmi_encoder_pre_crtc_enable(struct drm_encoder *encoder,
vc4_encoder->limited_rgb_range = false;
}
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
HDMI_WRITE(HDMI_FIFO_CTL, VC4_HDMI_FIFO_CTL_MASTER_SLAVE_N);
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
+
+ mutex_unlock(&vc4_hdmi->mutex);
}
static void vc4_hdmi_encoder_post_crtc_enable(struct drm_encoder *encoder,
struct drm_atomic_state *state)
{
- struct drm_display_mode *mode = &encoder->crtc->state->adjusted_mode;
struct vc4_hdmi *vc4_hdmi = encoder_to_vc4_hdmi(encoder);
+ struct drm_display_mode *mode = &vc4_hdmi->saved_adjusted_mode;
struct vc4_hdmi_encoder *vc4_encoder = to_vc4_hdmi_encoder(encoder);
bool hsync_pos = mode->flags & DRM_MODE_FLAG_PHSYNC;
bool vsync_pos = mode->flags & DRM_MODE_FLAG_PVSYNC;
+ unsigned long flags;
int ret;
+ mutex_lock(&vc4_hdmi->mutex);
+
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
+
HDMI_WRITE(HDMI_VID_CTL,
VC4_HD_VID_CTL_ENABLE |
VC4_HD_VID_CTL_CLRRGB |
@@ -1031,6 +1170,8 @@ static void vc4_hdmi_encoder_post_crtc_enable(struct drm_encoder *encoder,
HDMI_READ(HDMI_SCHEDULER_CONTROL) |
VC4_HDMI_SCHEDULER_CONTROL_MODE_HDMI);
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
+
ret = wait_for(HDMI_READ(HDMI_SCHEDULER_CONTROL) &
VC4_HDMI_SCHEDULER_CONTROL_HDMI_ACTIVE, 1000);
WARN_ONCE(ret, "Timeout waiting for "
@@ -1043,6 +1184,8 @@ static void vc4_hdmi_encoder_post_crtc_enable(struct drm_encoder *encoder,
HDMI_READ(HDMI_SCHEDULER_CONTROL) &
~VC4_HDMI_SCHEDULER_CONTROL_MODE_HDMI);
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
+
ret = wait_for(!(HDMI_READ(HDMI_SCHEDULER_CONTROL) &
VC4_HDMI_SCHEDULER_CONTROL_HDMI_ACTIVE), 1000);
WARN_ONCE(ret, "Timeout waiting for "
@@ -1050,6 +1193,8 @@ static void vc4_hdmi_encoder_post_crtc_enable(struct drm_encoder *encoder,
}
if (vc4_encoder->hdmi_monitor) {
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
+
WARN_ON(!(HDMI_READ(HDMI_SCHEDULER_CONTROL) &
VC4_HDMI_SCHEDULER_CONTROL_HDMI_ACTIVE));
HDMI_WRITE(HDMI_SCHEDULER_CONTROL,
@@ -1059,15 +1204,37 @@ static void vc4_hdmi_encoder_post_crtc_enable(struct drm_encoder *encoder,
HDMI_WRITE(HDMI_RAM_PACKET_CONFIG,
VC4_HDMI_RAM_PACKET_ENABLE);
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
+
vc4_hdmi_set_infoframes(encoder);
}
vc4_hdmi_recenter_fifo(vc4_hdmi);
vc4_hdmi_enable_scrambling(encoder);
+
+ mutex_unlock(&vc4_hdmi->mutex);
}
static void vc4_hdmi_encoder_enable(struct drm_encoder *encoder)
{
+ struct vc4_hdmi *vc4_hdmi = encoder_to_vc4_hdmi(encoder);
+
+ mutex_lock(&vc4_hdmi->mutex);
+ vc4_hdmi->output_enabled = true;
+ mutex_unlock(&vc4_hdmi->mutex);
+}
+
+static void vc4_hdmi_encoder_atomic_mode_set(struct drm_encoder *encoder,
+ struct drm_crtc_state *crtc_state,
+ struct drm_connector_state *conn_state)
+{
+ struct vc4_hdmi *vc4_hdmi = encoder_to_vc4_hdmi(encoder);
+
+ mutex_lock(&vc4_hdmi->mutex);
+ memcpy(&vc4_hdmi->saved_adjusted_mode,
+ &crtc_state->adjusted_mode,
+ sizeof(vc4_hdmi->saved_adjusted_mode));
+ mutex_unlock(&vc4_hdmi->mutex);
}
#define WIFI_2_4GHz_CH1_MIN_FREQ 2400000000ULL
@@ -1146,6 +1313,7 @@ vc4_hdmi_encoder_mode_valid(struct drm_encoder *encoder,
static const struct drm_encoder_helper_funcs vc4_hdmi_encoder_helper_funcs = {
.atomic_check = vc4_hdmi_encoder_atomic_check,
+ .atomic_mode_set = vc4_hdmi_encoder_atomic_mode_set,
.mode_valid = vc4_hdmi_encoder_mode_valid,
.disable = vc4_hdmi_encoder_disable,
.enable = vc4_hdmi_encoder_enable,
@@ -1180,6 +1348,7 @@ static void vc4_hdmi_audio_set_mai_clock(struct vc4_hdmi *vc4_hdmi,
unsigned int samplerate)
{
u32 hsm_clock = clk_get_rate(vc4_hdmi->audio_clock);
+ unsigned long flags;
unsigned long n, m;
rational_best_approximation(hsm_clock, samplerate,
@@ -1189,19 +1358,22 @@ static void vc4_hdmi_audio_set_mai_clock(struct vc4_hdmi *vc4_hdmi,
VC4_HD_MAI_SMP_M_SHIFT) + 1,
&n, &m);
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
HDMI_WRITE(HDMI_MAI_SMP,
VC4_SET_FIELD(n, VC4_HD_MAI_SMP_N) |
VC4_SET_FIELD(m - 1, VC4_HD_MAI_SMP_M));
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
}
static void vc4_hdmi_set_n_cts(struct vc4_hdmi *vc4_hdmi, unsigned int samplerate)
{
- struct drm_encoder *encoder = &vc4_hdmi->encoder.base.base;
- struct drm_crtc *crtc = encoder->crtc;
- const struct drm_display_mode *mode = &crtc->state->adjusted_mode;
+ const struct drm_display_mode *mode = &vc4_hdmi->saved_adjusted_mode;
u32 n, cts;
u64 tmp;
+ lockdep_assert_held(&vc4_hdmi->mutex);
+ lockdep_assert_held(&vc4_hdmi->hw_lock);
+
n = 128 * samplerate / 1000;
tmp = (u64)(mode->clock * 1000) * n;
do_div(tmp, 128 * samplerate);
@@ -1227,31 +1399,54 @@ static inline struct vc4_hdmi *dai_to_hdmi(struct snd_soc_dai *dai)
return snd_soc_card_get_drvdata(card);
}
+static bool vc4_hdmi_audio_can_stream(struct vc4_hdmi *vc4_hdmi)
+{
+ lockdep_assert_held(&vc4_hdmi->mutex);
+
+ /*
+ * If the controller is disabled, prevent any ALSA output.
+ */
+ if (!vc4_hdmi->output_enabled)
+ return false;
+
+ /*
+ * If the encoder is currently in DVI mode, treat the codec DAI
+ * as missing.
+ */
+ if (!(HDMI_READ(HDMI_RAM_PACKET_CONFIG) & VC4_HDMI_RAM_PACKET_ENABLE))
+ return false;
+
+ return true;
+}
+
static int vc4_hdmi_audio_startup(struct device *dev, void *data)
{
struct vc4_hdmi *vc4_hdmi = dev_get_drvdata(dev);
- struct drm_encoder *encoder = &vc4_hdmi->encoder.base.base;
+ unsigned long flags;
- /*
- * If the HDMI encoder hasn't probed, or the encoder is
- * currently in DVI mode, treat the codec dai as missing.
- */
- if (!encoder->crtc || !(HDMI_READ(HDMI_RAM_PACKET_CONFIG) &
- VC4_HDMI_RAM_PACKET_ENABLE))
+ mutex_lock(&vc4_hdmi->mutex);
+
+ if (!vc4_hdmi_audio_can_stream(vc4_hdmi)) {
+ mutex_unlock(&vc4_hdmi->mutex);
return -ENODEV;
+ }
vc4_hdmi->audio.streaming = true;
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
HDMI_WRITE(HDMI_MAI_CTL,
VC4_HD_MAI_CTL_RESET |
VC4_HD_MAI_CTL_FLUSH |
VC4_HD_MAI_CTL_DLATE |
VC4_HD_MAI_CTL_ERRORE |
VC4_HD_MAI_CTL_ERRORF);
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
if (vc4_hdmi->variant->phy_rng_enable)
vc4_hdmi->variant->phy_rng_enable(vc4_hdmi);
+ mutex_unlock(&vc4_hdmi->mutex);
+
return 0;
}
@@ -1259,32 +1454,48 @@ static void vc4_hdmi_audio_reset(struct vc4_hdmi *vc4_hdmi)
{
struct drm_encoder *encoder = &vc4_hdmi->encoder.base.base;
struct device *dev = &vc4_hdmi->pdev->dev;
+ unsigned long flags;
int ret;
+ lockdep_assert_held(&vc4_hdmi->mutex);
+
vc4_hdmi->audio.streaming = false;
ret = vc4_hdmi_stop_packet(encoder, HDMI_INFOFRAME_TYPE_AUDIO, false);
if (ret)
dev_err(dev, "Failed to stop audio infoframe: %d\n", ret);
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
+
HDMI_WRITE(HDMI_MAI_CTL, VC4_HD_MAI_CTL_RESET);
HDMI_WRITE(HDMI_MAI_CTL, VC4_HD_MAI_CTL_ERRORF);
HDMI_WRITE(HDMI_MAI_CTL, VC4_HD_MAI_CTL_FLUSH);
+
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
}
static void vc4_hdmi_audio_shutdown(struct device *dev, void *data)
{
struct vc4_hdmi *vc4_hdmi = dev_get_drvdata(dev);
+ unsigned long flags;
+
+ mutex_lock(&vc4_hdmi->mutex);
+
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
HDMI_WRITE(HDMI_MAI_CTL,
VC4_HD_MAI_CTL_DLATE |
VC4_HD_MAI_CTL_ERRORE |
VC4_HD_MAI_CTL_ERRORF);
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
+
if (vc4_hdmi->variant->phy_rng_disable)
vc4_hdmi->variant->phy_rng_disable(vc4_hdmi);
vc4_hdmi->audio.streaming = false;
vc4_hdmi_audio_reset(vc4_hdmi);
+
+ mutex_unlock(&vc4_hdmi->mutex);
}
static int sample_rate_to_mai_fmt(int samplerate)
@@ -1334,6 +1545,7 @@ static int vc4_hdmi_audio_prepare(struct device *dev, void *data,
struct drm_encoder *encoder = &vc4_hdmi->encoder.base.base;
unsigned int sample_rate = params->sample_rate;
unsigned int channels = params->channels;
+ unsigned long flags;
u32 audio_packet_config, channel_mask;
u32 channel_map;
u32 mai_audio_format;
@@ -1342,14 +1554,22 @@ static int vc4_hdmi_audio_prepare(struct device *dev, void *data,
dev_dbg(dev, "%s: %u Hz, %d bit, %d channels\n", __func__,
sample_rate, params->sample_width, channels);
+ mutex_lock(&vc4_hdmi->mutex);
+
+ if (!vc4_hdmi_audio_can_stream(vc4_hdmi)) {
+ mutex_unlock(&vc4_hdmi->mutex);
+ return -EINVAL;
+ }
+
+ vc4_hdmi_audio_set_mai_clock(vc4_hdmi, sample_rate);
+
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
HDMI_WRITE(HDMI_MAI_CTL,
VC4_SET_FIELD(channels, VC4_HD_MAI_CTL_CHNUM) |
VC4_HD_MAI_CTL_WHOLSMP |
VC4_HD_MAI_CTL_CHALIGN |
VC4_HD_MAI_CTL_ENABLE);
- vc4_hdmi_audio_set_mai_clock(vc4_hdmi, sample_rate);
-
mai_sample_rate = sample_rate_to_mai_fmt(sample_rate);
if (params->iec.status[0] & IEC958_AES0_NONAUDIO &&
params->channels == 8)
@@ -1387,11 +1607,16 @@ static int vc4_hdmi_audio_prepare(struct device *dev, void *data,
channel_map = vc4_hdmi->variant->channel_map(vc4_hdmi, channel_mask);
HDMI_WRITE(HDMI_MAI_CHANNEL_MAP, channel_map);
HDMI_WRITE(HDMI_AUDIO_PACKET_CONFIG, audio_packet_config);
+
vc4_hdmi_set_n_cts(vc4_hdmi, sample_rate);
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
+
memcpy(&vc4_hdmi->audio.infoframe, ¶ms->cea, sizeof(params->cea));
vc4_hdmi_set_audio_infoframe(encoder);
+ mutex_unlock(&vc4_hdmi->mutex);
+
return 0;
}
@@ -1434,7 +1659,9 @@ static int vc4_hdmi_audio_get_eld(struct device *dev, void *data,
struct vc4_hdmi *vc4_hdmi = dev_get_drvdata(dev);
struct drm_connector *connector = &vc4_hdmi->connector;
+ mutex_lock(&vc4_hdmi->mutex);
memcpy(buf, connector->eld, min(sizeof(connector->eld), len));
+ mutex_unlock(&vc4_hdmi->mutex);
return 0;
}
@@ -1656,6 +1883,8 @@ static void vc4_cec_read_msg(struct vc4_hdmi *vc4_hdmi, u32 cntrl1)
struct cec_msg *msg = &vc4_hdmi->cec_rx_msg;
unsigned int i;
+ lockdep_assert_held(&vc4_hdmi->hw_lock);
+
msg->len = 1 + ((cntrl1 & VC4_HDMI_CEC_REC_WRD_CNT_MASK) >>
VC4_HDMI_CEC_REC_WRD_CNT_SHIFT);
@@ -1674,11 +1903,12 @@ static void vc4_cec_read_msg(struct vc4_hdmi *vc4_hdmi, u32 cntrl1)
}
}
-static irqreturn_t vc4_cec_irq_handler_tx_bare(int irq, void *priv)
+static irqreturn_t vc4_cec_irq_handler_tx_bare_locked(struct vc4_hdmi *vc4_hdmi)
{
- struct vc4_hdmi *vc4_hdmi = priv;
u32 cntrl1;
+ lockdep_assert_held(&vc4_hdmi->hw_lock);
+
cntrl1 = HDMI_READ(HDMI_CEC_CNTRL_1);
vc4_hdmi->cec_tx_ok = cntrl1 & VC4_HDMI_CEC_TX_STATUS_GOOD;
cntrl1 &= ~VC4_HDMI_CEC_START_XMIT_BEGIN;
@@ -1687,11 +1917,24 @@ static irqreturn_t vc4_cec_irq_handler_tx_bare(int irq, void *priv)
return IRQ_WAKE_THREAD;
}
-static irqreturn_t vc4_cec_irq_handler_rx_bare(int irq, void *priv)
+static irqreturn_t vc4_cec_irq_handler_tx_bare(int irq, void *priv)
{
struct vc4_hdmi *vc4_hdmi = priv;
+ irqreturn_t ret;
+
+ spin_lock(&vc4_hdmi->hw_lock);
+ ret = vc4_cec_irq_handler_tx_bare_locked(vc4_hdmi);
+ spin_unlock(&vc4_hdmi->hw_lock);
+
+ return ret;
+}
+
+static irqreturn_t vc4_cec_irq_handler_rx_bare_locked(struct vc4_hdmi *vc4_hdmi)
+{
u32 cntrl1;
+ lockdep_assert_held(&vc4_hdmi->hw_lock);
+
vc4_hdmi->cec_rx_msg.len = 0;
cntrl1 = HDMI_READ(HDMI_CEC_CNTRL_1);
vc4_cec_read_msg(vc4_hdmi, cntrl1);
@@ -1704,6 +1947,18 @@ static irqreturn_t vc4_cec_irq_handler_rx_bare(int irq, void *priv)
return IRQ_WAKE_THREAD;
}
+static irqreturn_t vc4_cec_irq_handler_rx_bare(int irq, void *priv)
+{
+ struct vc4_hdmi *vc4_hdmi = priv;
+ irqreturn_t ret;
+
+ spin_lock(&vc4_hdmi->hw_lock);
+ ret = vc4_cec_irq_handler_rx_bare_locked(vc4_hdmi);
+ spin_unlock(&vc4_hdmi->hw_lock);
+
+ return ret;
+}
+
static irqreturn_t vc4_cec_irq_handler(int irq, void *priv)
{
struct vc4_hdmi *vc4_hdmi = priv;
@@ -1714,69 +1969,142 @@ static irqreturn_t vc4_cec_irq_handler(int irq, void *priv)
if (!(stat & VC4_HDMI_CPU_CEC))
return IRQ_NONE;
+ spin_lock(&vc4_hdmi->hw_lock);
cntrl5 = HDMI_READ(HDMI_CEC_CNTRL_5);
vc4_hdmi->cec_irq_was_rx = cntrl5 & VC4_HDMI_CEC_RX_CEC_INT;
if (vc4_hdmi->cec_irq_was_rx)
- ret = vc4_cec_irq_handler_rx_bare(irq, priv);
+ ret = vc4_cec_irq_handler_rx_bare_locked(vc4_hdmi);
else
- ret = vc4_cec_irq_handler_tx_bare(irq, priv);
+ ret = vc4_cec_irq_handler_tx_bare_locked(vc4_hdmi);
HDMI_WRITE(HDMI_CEC_CPU_CLEAR, VC4_HDMI_CPU_CEC);
+ spin_unlock(&vc4_hdmi->hw_lock);
+
return ret;
}
-static int vc4_hdmi_cec_adap_enable(struct cec_adapter *adap, bool enable)
+static int vc4_hdmi_cec_enable(struct cec_adapter *adap)
{
struct vc4_hdmi *vc4_hdmi = cec_get_drvdata(adap);
/* clock period in microseconds */
const u32 usecs = 1000000 / CEC_CLOCK_FREQ;
- u32 val = HDMI_READ(HDMI_CEC_CNTRL_5);
+ unsigned long flags;
+ u32 val;
+ int ret;
+ /*
+ * NOTE: This function should really take vc4_hdmi->mutex, but doing so
+ * results in a reentrancy since cec_s_phys_addr_from_edid() called in
+ * .detect or .get_modes might call .adap_enable, which leads to this
+ * function being called with that mutex held.
+ *
+ * Concurrency is not an issue for the moment since we don't share any
+ * state with KMS, so we can ignore the lock for now, but we need to
+ * keep it in mind if we were to change that assumption.
+ */
+
+ ret = pm_runtime_resume_and_get(&vc4_hdmi->pdev->dev);
+ if (ret)
+ return ret;
+
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
+
+ val = HDMI_READ(HDMI_CEC_CNTRL_5);
val &= ~(VC4_HDMI_CEC_TX_SW_RESET | VC4_HDMI_CEC_RX_SW_RESET |
VC4_HDMI_CEC_CNT_TO_4700_US_MASK |
VC4_HDMI_CEC_CNT_TO_4500_US_MASK);
val |= ((4700 / usecs) << VC4_HDMI_CEC_CNT_TO_4700_US_SHIFT) |
((4500 / usecs) << VC4_HDMI_CEC_CNT_TO_4500_US_SHIFT);
- if (enable) {
- HDMI_WRITE(HDMI_CEC_CNTRL_5, val |
- VC4_HDMI_CEC_TX_SW_RESET | VC4_HDMI_CEC_RX_SW_RESET);
- HDMI_WRITE(HDMI_CEC_CNTRL_5, val);
- HDMI_WRITE(HDMI_CEC_CNTRL_2,
- ((1500 / usecs) << VC4_HDMI_CEC_CNT_TO_1500_US_SHIFT) |
- ((1300 / usecs) << VC4_HDMI_CEC_CNT_TO_1300_US_SHIFT) |
- ((800 / usecs) << VC4_HDMI_CEC_CNT_TO_800_US_SHIFT) |
- ((600 / usecs) << VC4_HDMI_CEC_CNT_TO_600_US_SHIFT) |
- ((400 / usecs) << VC4_HDMI_CEC_CNT_TO_400_US_SHIFT));
- HDMI_WRITE(HDMI_CEC_CNTRL_3,
- ((2750 / usecs) << VC4_HDMI_CEC_CNT_TO_2750_US_SHIFT) |
- ((2400 / usecs) << VC4_HDMI_CEC_CNT_TO_2400_US_SHIFT) |
- ((2050 / usecs) << VC4_HDMI_CEC_CNT_TO_2050_US_SHIFT) |
- ((1700 / usecs) << VC4_HDMI_CEC_CNT_TO_1700_US_SHIFT));
- HDMI_WRITE(HDMI_CEC_CNTRL_4,
- ((4300 / usecs) << VC4_HDMI_CEC_CNT_TO_4300_US_SHIFT) |
- ((3900 / usecs) << VC4_HDMI_CEC_CNT_TO_3900_US_SHIFT) |
- ((3600 / usecs) << VC4_HDMI_CEC_CNT_TO_3600_US_SHIFT) |
- ((3500 / usecs) << VC4_HDMI_CEC_CNT_TO_3500_US_SHIFT));
+ HDMI_WRITE(HDMI_CEC_CNTRL_5, val |
+ VC4_HDMI_CEC_TX_SW_RESET | VC4_HDMI_CEC_RX_SW_RESET);
+ HDMI_WRITE(HDMI_CEC_CNTRL_5, val);
+ HDMI_WRITE(HDMI_CEC_CNTRL_2,
+ ((1500 / usecs) << VC4_HDMI_CEC_CNT_TO_1500_US_SHIFT) |
+ ((1300 / usecs) << VC4_HDMI_CEC_CNT_TO_1300_US_SHIFT) |
+ ((800 / usecs) << VC4_HDMI_CEC_CNT_TO_800_US_SHIFT) |
+ ((600 / usecs) << VC4_HDMI_CEC_CNT_TO_600_US_SHIFT) |
+ ((400 / usecs) << VC4_HDMI_CEC_CNT_TO_400_US_SHIFT));
+ HDMI_WRITE(HDMI_CEC_CNTRL_3,
+ ((2750 / usecs) << VC4_HDMI_CEC_CNT_TO_2750_US_SHIFT) |
+ ((2400 / usecs) << VC4_HDMI_CEC_CNT_TO_2400_US_SHIFT) |
+ ((2050 / usecs) << VC4_HDMI_CEC_CNT_TO_2050_US_SHIFT) |
+ ((1700 / usecs) << VC4_HDMI_CEC_CNT_TO_1700_US_SHIFT));
+ HDMI_WRITE(HDMI_CEC_CNTRL_4,
+ ((4300 / usecs) << VC4_HDMI_CEC_CNT_TO_4300_US_SHIFT) |
+ ((3900 / usecs) << VC4_HDMI_CEC_CNT_TO_3900_US_SHIFT) |
+ ((3600 / usecs) << VC4_HDMI_CEC_CNT_TO_3600_US_SHIFT) |
+ ((3500 / usecs) << VC4_HDMI_CEC_CNT_TO_3500_US_SHIFT));
- if (!vc4_hdmi->variant->external_irq_controller)
- HDMI_WRITE(HDMI_CEC_CPU_MASK_CLEAR, VC4_HDMI_CPU_CEC);
- } else {
- if (!vc4_hdmi->variant->external_irq_controller)
- HDMI_WRITE(HDMI_CEC_CPU_MASK_SET, VC4_HDMI_CPU_CEC);
- HDMI_WRITE(HDMI_CEC_CNTRL_5, val |
- VC4_HDMI_CEC_TX_SW_RESET | VC4_HDMI_CEC_RX_SW_RESET);
- }
+ if (!vc4_hdmi->variant->external_irq_controller)
+ HDMI_WRITE(HDMI_CEC_CPU_MASK_CLEAR, VC4_HDMI_CPU_CEC);
+
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
+
return 0;
}
+static int vc4_hdmi_cec_disable(struct cec_adapter *adap)
+{
+ struct vc4_hdmi *vc4_hdmi = cec_get_drvdata(adap);
+ unsigned long flags;
+
+ /*
+ * NOTE: This function should really take vc4_hdmi->mutex, but doing so
+ * results in a reentrancy since cec_s_phys_addr_from_edid() called in
+ * .detect or .get_modes might call .adap_enable, which leads to this
+ * function being called with that mutex held.
+ *
+ * Concurrency is not an issue for the moment since we don't share any
+ * state with KMS, so we can ignore the lock for now, but we need to
+ * keep it in mind if we were to change that assumption.
+ */
+
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
+
+ if (!vc4_hdmi->variant->external_irq_controller)
+ HDMI_WRITE(HDMI_CEC_CPU_MASK_SET, VC4_HDMI_CPU_CEC);
+
+ HDMI_WRITE(HDMI_CEC_CNTRL_5, HDMI_READ(HDMI_CEC_CNTRL_5) |
+ VC4_HDMI_CEC_TX_SW_RESET | VC4_HDMI_CEC_RX_SW_RESET);
+
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
+
+ pm_runtime_put(&vc4_hdmi->pdev->dev);
+
+ return 0;
+}
+
+static int vc4_hdmi_cec_adap_enable(struct cec_adapter *adap, bool enable)
+{
+ if (enable)
+ return vc4_hdmi_cec_enable(adap);
+ else
+ return vc4_hdmi_cec_disable(adap);
+}
+
static int vc4_hdmi_cec_adap_log_addr(struct cec_adapter *adap, u8 log_addr)
{
struct vc4_hdmi *vc4_hdmi = cec_get_drvdata(adap);
+ unsigned long flags;
+ /*
+ * NOTE: This function should really take vc4_hdmi->mutex, but doing so
+ * results in a reentrancy since cec_s_phys_addr_from_edid() called in
+ * .detect or .get_modes might call .adap_enable, which leads to this
+ * function being called with that mutex held.
+ *
+ * Concurrency is not an issue for the moment since we don't share any
+ * state with KMS, so we can ignore the lock for now, but we need to
+ * keep it in mind if we were to change that assumption.
+ */
+
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
HDMI_WRITE(HDMI_CEC_CNTRL_1,
(HDMI_READ(HDMI_CEC_CNTRL_1) & ~VC4_HDMI_CEC_ADDR_MASK) |
(log_addr & 0xf) << VC4_HDMI_CEC_ADDR_SHIFT);
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
+
return 0;
}
@@ -1785,14 +2113,28 @@ static int vc4_hdmi_cec_adap_transmit(struct cec_adapter *adap, u8 attempts,
{
struct vc4_hdmi *vc4_hdmi = cec_get_drvdata(adap);
struct drm_device *dev = vc4_hdmi->connector.dev;
+ unsigned long flags;
u32 val;
unsigned int i;
+ /*
+ * NOTE: This function should really take vc4_hdmi->mutex, but doing so
+ * results in a reentrancy since cec_s_phys_addr_from_edid() called in
+ * .detect or .get_modes might call .adap_enable, which leads to this
+ * function being called with that mutex held.
+ *
+ * Concurrency is not an issue for the moment since we don't share any
+ * state with KMS, so we can ignore the lock for now, but we need to
+ * keep it in mind if we were to change that assumption.
+ */
+
if (msg->len > 16) {
drm_err(dev, "Attempting to transmit too much data (%d)\n", msg->len);
return -ENOMEM;
}
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
+
for (i = 0; i < msg->len; i += 4)
HDMI_WRITE(HDMI_CEC_TX_DATA_1 + (i >> 2),
(msg->msg[i]) |
@@ -1808,6 +2150,9 @@ static int vc4_hdmi_cec_adap_transmit(struct cec_adapter *adap, u8 attempts,
val |= VC4_HDMI_CEC_START_XMIT_BEGIN;
HDMI_WRITE(HDMI_CEC_CNTRL_1, val);
+
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
+
return 0;
}
@@ -1822,6 +2167,7 @@ static int vc4_hdmi_cec_init(struct vc4_hdmi *vc4_hdmi)
struct cec_connector_info conn_info;
struct platform_device *pdev = vc4_hdmi->pdev;
struct device *dev = &pdev->dev;
+ unsigned long flags;
u32 value;
int ret;
@@ -1841,10 +2187,12 @@ static int vc4_hdmi_cec_init(struct vc4_hdmi *vc4_hdmi)
cec_fill_conn_info_from_drm(&conn_info, &vc4_hdmi->connector);
cec_s_conn_info(vc4_hdmi->cec_adap, &conn_info);
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
value = HDMI_READ(HDMI_CEC_CNTRL_1);
/* Set the logical address to Unregistered */
value |= VC4_HDMI_CEC_ADDR_MASK;
HDMI_WRITE(HDMI_CEC_CNTRL_1, value);
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
vc4_hdmi_cec_update_clk_div(vc4_hdmi);
@@ -1863,7 +2211,9 @@ static int vc4_hdmi_cec_init(struct vc4_hdmi *vc4_hdmi)
if (ret)
goto err_remove_cec_rx_handler;
} else {
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
HDMI_WRITE(HDMI_CEC_CPU_MASK_SET, 0xffffffff);
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
ret = request_threaded_irq(platform_get_irq(pdev, 0),
vc4_cec_irq_handler,
@@ -2099,6 +2449,27 @@ static int vc5_hdmi_init_resources(struct vc4_hdmi *vc4_hdmi)
return 0;
}
+static int __maybe_unused vc4_hdmi_runtime_suspend(struct device *dev)
+{
+ struct vc4_hdmi *vc4_hdmi = dev_get_drvdata(dev);
+
+ clk_disable_unprepare(vc4_hdmi->hsm_clock);
+
+ return 0;
+}
+
+static int vc4_hdmi_runtime_resume(struct device *dev)
+{
+ struct vc4_hdmi *vc4_hdmi = dev_get_drvdata(dev);
+ int ret;
+
+ ret = clk_prepare_enable(vc4_hdmi->hsm_clock);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
static int vc4_hdmi_bind(struct device *dev, struct device *master, void *data)
{
const struct vc4_hdmi_variant *variant = of_device_get_match_data(dev);
@@ -2112,6 +2483,8 @@ static int vc4_hdmi_bind(struct device *dev, struct device *master, void *data)
vc4_hdmi = devm_kzalloc(dev, sizeof(*vc4_hdmi), GFP_KERNEL);
if (!vc4_hdmi)
return -ENOMEM;
+ mutex_init(&vc4_hdmi->mutex);
+ spin_lock_init(&vc4_hdmi->hw_lock);
INIT_DELAYED_WORK(&vc4_hdmi->scrambling_work, vc4_hdmi_scrambling_wq);
dev_set_drvdata(dev, vc4_hdmi);
@@ -2125,6 +2498,14 @@ static int vc4_hdmi_bind(struct device *dev, struct device *master, void *data)
vc4_hdmi->pdev = pdev;
vc4_hdmi->variant = variant;
+ /*
+ * Since we don't know the state of the controller and its
+ * display (if any), let's assume it's always enabled.
+ * vc4_hdmi_disable_scrambling() will thus run at boot, make
+ * sure it's disabled, and avoid any inconsistency.
+ */
+ vc4_hdmi->scdc_enabled = true;
+
ret = variant->init_resources(vc4_hdmi);
if (ret)
return ret;
@@ -2162,6 +2543,31 @@ static int vc4_hdmi_bind(struct device *dev, struct device *master, void *data)
vc4_hdmi->disable_4kp60 = true;
}
+ /*
+ * If we boot without any cable connected to the HDMI connector,
+ * the firmware will skip the HSM initialization and leave it
+ * with a rate of 0, resulting in a bus lockup when we're
+ * accessing the registers even if it's enabled.
+ *
+ * Let's put a sensible default at runtime_resume so that we
+ * don't end up in this situation.
+ */
+ ret = clk_set_min_rate(vc4_hdmi->hsm_clock, HSM_MIN_CLOCK_FREQ);
+ if (ret)
+ goto err_put_ddc;
+
+ /*
+ * We need to have the device powered up at this point to call
+ * our reset hook and for the CEC init.
+ */
+ ret = vc4_hdmi_runtime_resume(dev);
+ if (ret)
+ goto err_put_ddc;
+
+ pm_runtime_get_noresume(dev);
+ pm_runtime_set_active(dev);
+ pm_runtime_enable(dev);
+
if (vc4_hdmi->variant->reset)
vc4_hdmi->variant->reset(vc4_hdmi);
@@ -2173,8 +2579,6 @@ static int vc4_hdmi_bind(struct device *dev, struct device *master, void *data)
clk_prepare_enable(vc4_hdmi->pixel_bvb_clock);
}
- pm_runtime_enable(dev);
-
drm_simple_encoder_init(drm, encoder, DRM_MODE_ENCODER_TMDS);
drm_encoder_helper_add(encoder, &vc4_hdmi_encoder_helper_funcs);
@@ -2198,6 +2602,8 @@ static int vc4_hdmi_bind(struct device *dev, struct device *master, void *data)
vc4_hdmi_debugfs_regs,
vc4_hdmi);
+ pm_runtime_put_sync(dev);
+
return 0;
err_free_cec:
@@ -2208,6 +2614,7 @@ static int vc4_hdmi_bind(struct device *dev, struct device *master, void *data)
vc4_hdmi_connector_destroy(&vc4_hdmi->connector);
err_destroy_encoder:
drm_encoder_cleanup(encoder);
+ pm_runtime_put_sync(dev);
pm_runtime_disable(dev);
err_put_ddc:
put_device(&vc4_hdmi->ddc->dev);
@@ -2294,7 +2701,7 @@ static const struct vc4_hdmi_variant bcm2711_hdmi0_variant = {
.encoder_type = VC4_ENCODER_TYPE_HDMI0,
.debugfs_name = "hdmi0_regs",
.card_name = "vc4-hdmi-0",
- .max_pixel_clock = HDMI_14_MAX_TMDS_CLK,
+ .max_pixel_clock = 600000000,
.registers = vc5_hdmi_hdmi0_fields,
.num_registers = ARRAY_SIZE(vc5_hdmi_hdmi0_fields),
.phy_lane_mapping = {
@@ -2353,11 +2760,18 @@ static const struct of_device_id vc4_hdmi_dt_match[] = {
{}
};
+static const struct dev_pm_ops vc4_hdmi_pm_ops = {
+ SET_RUNTIME_PM_OPS(vc4_hdmi_runtime_suspend,
+ vc4_hdmi_runtime_resume,
+ NULL)
+};
+
struct platform_driver vc4_hdmi_driver = {
.probe = vc4_hdmi_dev_probe,
.remove = vc4_hdmi_dev_remove,
.driver = {
.name = "vc4_hdmi",
.of_match_table = vc4_hdmi_dt_match,
+ .pm = &vc4_hdmi_pm_ops,
},
};
diff --git a/drivers/gpu/drm/vc4/vc4_hdmi.h b/drivers/gpu/drm/vc4/vc4_hdmi.h
index 33e9f66..36c0b08 100644
--- a/drivers/gpu/drm/vc4/vc4_hdmi.h
+++ b/drivers/gpu/drm/vc4/vc4_hdmi.h
@@ -178,6 +178,43 @@ struct vc4_hdmi {
struct debugfs_regset32 hdmi_regset;
struct debugfs_regset32 hd_regset;
+
+ /**
+ * @hw_lock: Spinlock protecting device register access.
+ */
+ spinlock_t hw_lock;
+
+ /**
+ * @mutex: Mutex protecting the driver access across multiple
+ * frameworks (KMS, ALSA).
+ *
+ * NOTE: While supported, CEC has been left out since
+ * cec_s_phys_addr_from_edid() might call .adap_enable and lead to a
+ * reentrancy issue between .get_modes (or .detect) and .adap_enable.
+ * Since we don't share any state between the CEC hooks and KMS', it's
+ * not a big deal. The only trouble might come from updating the CEC
+ * clock divider which might be affected by a modeset, but CEC should
+ * be resilient to that.
+ */
+ struct mutex mutex;
+
+ /**
+ * @saved_adjusted_mode: Copy of @drm_crtc_state.adjusted_mode
+ * for use by ALSA hooks and interrupt handlers. Protected by @mutex.
+ */
+ struct drm_display_mode saved_adjusted_mode;
+
+ /**
+ * @output_enabled: Is the HDMI controller currently active?
+ * Protected by @mutex.
+ */
+ bool output_enabled;
+
+ /**
+ * @scdc_enabled: Is the HDMI controller currently running with
+ * the scrambler on? Protected by @mutex.
+ */
+ bool scdc_enabled;
};
static inline struct vc4_hdmi *
diff --git a/drivers/gpu/drm/vc4/vc4_hdmi_phy.c b/drivers/gpu/drm/vc4/vc4_hdmi_phy.c
index 3653548..62148f0d 100644
--- a/drivers/gpu/drm/vc4/vc4_hdmi_phy.c
+++ b/drivers/gpu/drm/vc4/vc4_hdmi_phy.c
@@ -130,31 +130,49 @@
void vc4_hdmi_phy_init(struct vc4_hdmi *vc4_hdmi,
struct vc4_hdmi_connector_state *conn_state)
{
+ unsigned long flags;
+
/* PHY should be in reset, like
* vc4_hdmi_encoder_disable() does.
*/
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
+
HDMI_WRITE(HDMI_TX_PHY_RESET_CTL, 0xf << 16);
HDMI_WRITE(HDMI_TX_PHY_RESET_CTL, 0);
+
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
}
void vc4_hdmi_phy_disable(struct vc4_hdmi *vc4_hdmi)
{
+ unsigned long flags;
+
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
HDMI_WRITE(HDMI_TX_PHY_RESET_CTL, 0xf << 16);
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
}
void vc4_hdmi_phy_rng_enable(struct vc4_hdmi *vc4_hdmi)
{
+ unsigned long flags;
+
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
HDMI_WRITE(HDMI_TX_PHY_CTL_0,
HDMI_READ(HDMI_TX_PHY_CTL_0) &
~VC4_HDMI_TX_PHY_RNG_PWRDN);
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
}
void vc4_hdmi_phy_rng_disable(struct vc4_hdmi *vc4_hdmi)
{
+ unsigned long flags;
+
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
HDMI_WRITE(HDMI_TX_PHY_CTL_0,
HDMI_READ(HDMI_TX_PHY_CTL_0) |
VC4_HDMI_TX_PHY_RNG_PWRDN);
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
}
static unsigned long long
@@ -336,6 +354,8 @@ phy_get_channel_settings(enum vc4_hdmi_phy_channel chan,
static void vc5_hdmi_reset_phy(struct vc4_hdmi *vc4_hdmi)
{
+ lockdep_assert_held(&vc4_hdmi->hw_lock);
+
HDMI_WRITE(HDMI_TX_PHY_RESET_CTL, 0x0f);
HDMI_WRITE(HDMI_TX_PHY_POWERDOWN_CTL, BIT(10));
}
@@ -348,10 +368,13 @@ void vc5_hdmi_phy_init(struct vc4_hdmi *vc4_hdmi,
unsigned long long pixel_freq = conn_state->pixel_rate;
unsigned long long vco_freq;
unsigned char word_sel;
+ unsigned long flags;
u8 vco_sel, vco_div;
vco_freq = phy_get_vco_freq(pixel_freq, &vco_sel, &vco_div);
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
+
vc5_hdmi_reset_phy(vc4_hdmi);
HDMI_WRITE(HDMI_TX_PHY_POWERDOWN_CTL,
@@ -501,23 +524,37 @@ void vc5_hdmi_phy_init(struct vc4_hdmi *vc4_hdmi,
HDMI_READ(HDMI_TX_PHY_RESET_CTL) |
VC4_HDMI_TX_PHY_RESET_CTL_PLL_RESETB |
VC4_HDMI_TX_PHY_RESET_CTL_PLLDIV_RESETB);
+
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
}
void vc5_hdmi_phy_disable(struct vc4_hdmi *vc4_hdmi)
{
+ unsigned long flags;
+
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
vc5_hdmi_reset_phy(vc4_hdmi);
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
}
void vc5_hdmi_phy_rng_enable(struct vc4_hdmi *vc4_hdmi)
{
+ unsigned long flags;
+
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
HDMI_WRITE(HDMI_TX_PHY_POWERDOWN_CTL,
HDMI_READ(HDMI_TX_PHY_POWERDOWN_CTL) &
~VC4_HDMI_TX_PHY_POWERDOWN_CTL_RNDGEN_PWRDN);
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
}
void vc5_hdmi_phy_rng_disable(struct vc4_hdmi *vc4_hdmi)
{
+ unsigned long flags;
+
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
HDMI_WRITE(HDMI_TX_PHY_POWERDOWN_CTL,
HDMI_READ(HDMI_TX_PHY_POWERDOWN_CTL) |
VC4_HDMI_TX_PHY_POWERDOWN_CTL_RNDGEN_PWRDN);
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
}
diff --git a/drivers/gpu/drm/vc4/vc4_hdmi_regs.h b/drivers/gpu/drm/vc4/vc4_hdmi_regs.h
index 19d2fdc..fc97150 100644
--- a/drivers/gpu/drm/vc4/vc4_hdmi_regs.h
+++ b/drivers/gpu/drm/vc4/vc4_hdmi_regs.h
@@ -1,6 +1,8 @@
#ifndef _VC4_HDMI_REGS_H_
#define _VC4_HDMI_REGS_H_
+#include <linux/pm_runtime.h>
+
#include "vc4_hdmi.h"
#define VC4_HDMI_PACKET_STRIDE 0x24
@@ -412,6 +414,8 @@ static inline u32 vc4_hdmi_read(struct vc4_hdmi *hdmi,
const struct vc4_hdmi_variant *variant = hdmi->variant;
void __iomem *base;
+ WARN_ON(!pm_runtime_active(&hdmi->pdev->dev));
+
if (reg >= variant->num_registers) {
dev_warn(&hdmi->pdev->dev,
"Invalid register ID %u\n", reg);
@@ -438,6 +442,10 @@ static inline void vc4_hdmi_write(struct vc4_hdmi *hdmi,
const struct vc4_hdmi_variant *variant = hdmi->variant;
void __iomem *base;
+ lockdep_assert_held(&hdmi->hw_lock);
+
+ WARN_ON(!pm_runtime_active(&hdmi->pdev->dev));
+
if (reg >= variant->num_registers) {
dev_warn(&hdmi->pdev->dev,
"Invalid register ID %u\n", reg);
diff --git a/drivers/gpu/drm/vc4/vc4_hvs.c b/drivers/gpu/drm/vc4/vc4_hvs.c
index c239045..604933e 100644
--- a/drivers/gpu/drm/vc4/vc4_hvs.c
+++ b/drivers/gpu/drm/vc4/vc4_hvs.c
@@ -365,17 +365,16 @@ static void vc4_hvs_update_dlist(struct drm_crtc *crtc)
struct vc4_dev *vc4 = to_vc4_dev(dev);
struct vc4_crtc *vc4_crtc = to_vc4_crtc(crtc);
struct vc4_crtc_state *vc4_state = to_vc4_crtc_state(crtc->state);
+ unsigned long flags;
if (crtc->state->event) {
- unsigned long flags;
-
crtc->state->event->pipe = drm_crtc_index(crtc);
WARN_ON(drm_crtc_vblank_get(crtc) != 0);
spin_lock_irqsave(&dev->event_lock, flags);
- if (!vc4_state->feed_txp || vc4_state->txp_armed) {
+ if (!vc4_crtc->feeds_txp || vc4_state->txp_armed) {
vc4_crtc->event = crtc->state->event;
crtc->state->event = NULL;
}
@@ -388,6 +387,22 @@ static void vc4_hvs_update_dlist(struct drm_crtc *crtc)
HVS_WRITE(SCALER_DISPLISTX(vc4_state->assigned_channel),
vc4_state->mm.start);
}
+
+ spin_lock_irqsave(&vc4_crtc->irq_lock, flags);
+ vc4_crtc->current_dlist = vc4_state->mm.start;
+ spin_unlock_irqrestore(&vc4_crtc->irq_lock, flags);
+}
+
+void vc4_hvs_atomic_begin(struct drm_crtc *crtc,
+ struct drm_atomic_state *state)
+{
+ struct vc4_crtc *vc4_crtc = to_vc4_crtc(crtc);
+ struct vc4_crtc_state *vc4_state = to_vc4_crtc_state(crtc->state);
+ unsigned long flags;
+
+ spin_lock_irqsave(&vc4_crtc->irq_lock, flags);
+ vc4_crtc->current_hvs_channel = vc4_state->assigned_channel;
+ spin_unlock_irqrestore(&vc4_crtc->irq_lock, flags);
}
void vc4_hvs_atomic_enable(struct drm_crtc *crtc,
@@ -395,10 +410,9 @@ void vc4_hvs_atomic_enable(struct drm_crtc *crtc,
{
struct drm_device *dev = crtc->dev;
struct vc4_dev *vc4 = to_vc4_dev(dev);
- struct drm_crtc_state *new_crtc_state = drm_atomic_get_new_crtc_state(state, crtc);
- struct vc4_crtc_state *vc4_state = to_vc4_crtc_state(new_crtc_state);
struct drm_display_mode *mode = &crtc->state->adjusted_mode;
- bool oneshot = vc4_state->feed_txp;
+ struct vc4_crtc *vc4_crtc = to_vc4_crtc(crtc);
+ bool oneshot = vc4_crtc->feeds_txp;
vc4_hvs_update_dlist(crtc);
vc4_hvs_init_channel(vc4, crtc, mode, oneshot);
diff --git a/drivers/gpu/drm/vc4/vc4_kms.c b/drivers/gpu/drm/vc4/vc4_kms.c
index b61792d..24de29b 100644
--- a/drivers/gpu/drm/vc4/vc4_kms.c
+++ b/drivers/gpu/drm/vc4/vc4_kms.c
@@ -39,9 +39,11 @@ static struct vc4_ctm_state *to_vc4_ctm_state(struct drm_private_state *priv)
struct vc4_hvs_state {
struct drm_private_state base;
+ unsigned long core_clock_rate;
struct {
unsigned in_use: 1;
+ unsigned long fifo_load;
struct drm_crtc_commit *pending_commit;
} fifo_state[HVS_NUM_CHANNELS];
};
@@ -233,6 +235,7 @@ static void vc4_hvs_pv_muxing_commit(struct vc4_dev *vc4,
unsigned int i;
for_each_new_crtc_in_state(state, crtc, crtc_state, i) {
+ struct vc4_crtc *vc4_crtc = to_vc4_crtc(crtc);
struct vc4_crtc_state *vc4_state = to_vc4_crtc_state(crtc_state);
u32 dispctrl;
u32 dsp3_mux;
@@ -253,7 +256,7 @@ static void vc4_hvs_pv_muxing_commit(struct vc4_dev *vc4,
* TXP IP, and we need to disable the FIFO2 -> pixelvalve1
* route.
*/
- if (vc4_state->feed_txp)
+ if (vc4_crtc->feeds_txp)
dsp3_mux = VC4_SET_FIELD(3, SCALER_DISPCTRL_DSP3_MUX);
else
dsp3_mux = VC4_SET_FIELD(2, SCALER_DISPCTRL_DSP3_MUX);
@@ -338,11 +341,20 @@ static void vc4_atomic_commit_tail(struct drm_atomic_state *state)
struct vc4_dev *vc4 = to_vc4_dev(dev);
struct vc4_hvs *hvs = vc4->hvs;
struct drm_crtc_state *new_crtc_state;
+ struct vc4_hvs_state *new_hvs_state;
struct drm_crtc *crtc;
struct vc4_hvs_state *old_hvs_state;
unsigned int channel;
int i;
+ old_hvs_state = vc4_hvs_get_old_global_state(state);
+ if (WARN_ON(IS_ERR(old_hvs_state)))
+ return;
+
+ new_hvs_state = vc4_hvs_get_new_global_state(state);
+ if (WARN_ON(IS_ERR(new_hvs_state)))
+ return;
+
for_each_new_crtc_in_state(state, crtc, new_crtc_state, i) {
struct vc4_crtc_state *vc4_crtc_state;
@@ -353,10 +365,6 @@ static void vc4_atomic_commit_tail(struct drm_atomic_state *state)
vc4_hvs_mask_underrun(dev, vc4_crtc_state->assigned_channel);
}
- old_hvs_state = vc4_hvs_get_old_global_state(state);
- if (IS_ERR(old_hvs_state))
- return;
-
for (channel = 0; channel < HVS_NUM_CHANNELS; channel++) {
struct drm_crtc_commit *commit;
int ret;
@@ -376,9 +384,13 @@ static void vc4_atomic_commit_tail(struct drm_atomic_state *state)
old_hvs_state->fifo_state[channel].pending_commit = NULL;
}
- if (vc4->hvs->hvs5)
- clk_set_min_rate(hvs->core_clk, 500000000);
+ if (vc4->hvs->hvs5) {
+ unsigned long core_rate = max_t(unsigned long,
+ 500000000,
+ new_hvs_state->core_clock_rate);
+ clk_set_min_rate(hvs->core_clk, core_rate);
+ }
drm_atomic_helper_commit_modeset_disables(dev, state);
vc4_ctm_commit(vc4, state);
@@ -400,8 +412,12 @@ static void vc4_atomic_commit_tail(struct drm_atomic_state *state)
drm_atomic_helper_cleanup_planes(dev, state);
- if (vc4->hvs->hvs5)
- clk_set_min_rate(hvs->core_clk, 0);
+ if (vc4->hvs->hvs5) {
+ drm_dbg(dev, "Running the core clock at %lu Hz\n",
+ new_hvs_state->core_clock_rate);
+
+ clk_set_min_rate(hvs->core_clk, new_hvs_state->core_clock_rate);
+ }
}
static int vc4_atomic_commit_setup(struct drm_atomic_state *state)
@@ -553,9 +569,6 @@ static int vc4_load_tracker_atomic_check(struct drm_atomic_state *state)
struct drm_plane *plane;
int i;
- if (!vc4->load_tracker_available)
- return 0;
-
priv_state = drm_atomic_get_private_obj_state(state,
&vc4->load_tracker);
if (IS_ERR(priv_state))
@@ -630,9 +643,6 @@ static void vc4_load_tracker_obj_fini(struct drm_device *dev, void *unused)
{
struct vc4_dev *vc4 = to_vc4_dev(dev);
- if (!vc4->load_tracker_available)
- return;
-
drm_atomic_private_obj_fini(&vc4->load_tracker);
}
@@ -640,9 +650,6 @@ static int vc4_load_tracker_obj_init(struct vc4_dev *vc4)
{
struct vc4_load_tracker_state *load_state;
- if (!vc4->load_tracker_available)
- return 0;
-
load_state = kzalloc(sizeof(*load_state), GFP_KERNEL);
if (!load_state)
return -ENOMEM;
@@ -667,11 +674,13 @@ vc4_hvs_channels_duplicate_state(struct drm_private_obj *obj)
__drm_atomic_helper_private_obj_duplicate_state(obj, &state->base);
-
for (i = 0; i < HVS_NUM_CHANNELS; i++) {
state->fifo_state[i].in_use = old_state->fifo_state[i].in_use;
+ state->fifo_state[i].fifo_load = old_state->fifo_state[i].fifo_load;
}
+ state->core_clock_rate = old_state->core_clock_rate;
+
return &state->base;
}
@@ -827,6 +836,76 @@ static int vc4_pv_muxing_atomic_check(struct drm_device *dev,
}
static int
+vc4_core_clock_atomic_check(struct drm_atomic_state *state)
+{
+ struct vc4_dev *vc4 = to_vc4_dev(state->dev);
+ struct drm_private_state *priv_state;
+ struct vc4_hvs_state *hvs_new_state;
+ struct vc4_load_tracker_state *load_state;
+ struct drm_crtc_state *old_crtc_state, *new_crtc_state;
+ struct drm_crtc *crtc;
+ unsigned int num_outputs;
+ unsigned long pixel_rate;
+ unsigned long cob_rate;
+ unsigned int i;
+
+ priv_state = drm_atomic_get_private_obj_state(state,
+ &vc4->load_tracker);
+ if (IS_ERR(priv_state))
+ return PTR_ERR(priv_state);
+
+ load_state = to_vc4_load_tracker_state(priv_state);
+
+ hvs_new_state = vc4_hvs_get_global_state(state);
+ if (IS_ERR(hvs_new_state))
+ return PTR_ERR(hvs_new_state);
+
+ for_each_oldnew_crtc_in_state(state, crtc,
+ old_crtc_state,
+ new_crtc_state,
+ i) {
+ if (old_crtc_state->active) {
+ struct vc4_crtc_state *old_vc4_state =
+ to_vc4_crtc_state(old_crtc_state);
+ unsigned int channel = old_vc4_state->assigned_channel;
+
+ hvs_new_state->fifo_state[channel].fifo_load = 0;
+ }
+
+ if (new_crtc_state->active) {
+ struct vc4_crtc_state *new_vc4_state =
+ to_vc4_crtc_state(new_crtc_state);
+ unsigned int channel = new_vc4_state->assigned_channel;
+
+ hvs_new_state->fifo_state[channel].fifo_load =
+ new_vc4_state->hvs_load;
+ }
+ }
+
+ cob_rate = 0;
+ num_outputs = 0;
+ for (i = 0; i < HVS_NUM_CHANNELS; i++) {
+ if (!hvs_new_state->fifo_state[i].in_use)
+ continue;
+
+ num_outputs++;
+ cob_rate += hvs_new_state->fifo_state[i].fifo_load;
+ }
+
+ pixel_rate = load_state->hvs_load;
+ if (num_outputs > 1) {
+ pixel_rate = (pixel_rate * 40) / 100;
+ } else {
+ pixel_rate = (pixel_rate * 60) / 100;
+ }
+
+ hvs_new_state->core_clock_rate = max(cob_rate, pixel_rate);
+
+ return 0;
+}
+
+
+static int
vc4_atomic_check(struct drm_device *dev, struct drm_atomic_state *state)
{
int ret;
@@ -843,7 +922,11 @@ vc4_atomic_check(struct drm_device *dev, struct drm_atomic_state *state)
if (ret)
return ret;
- return vc4_load_tracker_atomic_check(state);
+ ret = vc4_load_tracker_atomic_check(state);
+ if (ret)
+ return ret;
+
+ return vc4_core_clock_atomic_check(state);
}
static struct drm_mode_config_helper_funcs vc4_mode_config_helpers = {
@@ -864,9 +947,12 @@ int vc4_kms_load(struct drm_device *dev)
"brcm,bcm2711-vc5");
int ret;
+ /*
+ * The limits enforced by the load tracker aren't relevant for
+ * the BCM2711, but the load tracker computations are used for
+ * the core clock rate calculation.
+ */
if (!is_vc5) {
- vc4->load_tracker_available = true;
-
/* Start with the load tracker enabled. Can be
* disabled through the debugfs load_tracker file.
*/
diff --git a/drivers/gpu/drm/vc4/vc4_plane.c b/drivers/gpu/drm/vc4/vc4_plane.c
index 19161b6..920a9ee 100644
--- a/drivers/gpu/drm/vc4/vc4_plane.c
+++ b/drivers/gpu/drm/vc4/vc4_plane.c
@@ -33,6 +33,7 @@ static const struct hvs_format {
u32 hvs; /* HVS_FORMAT_* */
u32 pixel_order;
u32 pixel_order_hvs5;
+ bool hvs5_only;
} hvs_formats[] = {
{
.drm = DRM_FORMAT_XRGB8888,
@@ -128,6 +129,12 @@ static const struct hvs_format {
.hvs = HVS_PIXEL_FORMAT_YCBCR_YUV422_2PLANE,
.pixel_order = HVS_PIXEL_ORDER_XYCRCB,
},
+ {
+ .drm = DRM_FORMAT_P030,
+ .hvs = HVS_PIXEL_FORMAT_YCBCR_10BIT,
+ .pixel_order = HVS_PIXEL_ORDER_XYCBCR,
+ .hvs5_only = true,
+ },
};
static const struct hvs_format *vc4_get_hvs_format(u32 drm_format)
@@ -529,11 +536,6 @@ static void vc4_plane_calc_load(struct drm_plane_state *state)
struct vc4_plane_state *vc4_state;
struct drm_crtc_state *crtc_state;
unsigned int vscale_factor;
- struct vc4_dev *vc4;
-
- vc4 = to_vc4_dev(state->plane->dev);
- if (!vc4->load_tracker_available)
- return;
vc4_state = to_vc4_plane_state(state);
crtc_state = drm_atomic_get_existing_crtc_state(state->state,
@@ -621,6 +623,51 @@ static int vc4_plane_allocate_lbm(struct drm_plane_state *state)
return 0;
}
+/*
+ * The colorspace conversion matrices are held in 3 entries in the dlist.
+ * Create an array of them, with entries for each full and limited mode, and
+ * each supported colorspace.
+ */
+static const u32 colorspace_coeffs[2][DRM_COLOR_ENCODING_MAX][3] = {
+ {
+ /* Limited range */
+ {
+ /* BT601 */
+ SCALER_CSC0_ITR_R_601_5,
+ SCALER_CSC1_ITR_R_601_5,
+ SCALER_CSC2_ITR_R_601_5,
+ }, {
+ /* BT709 */
+ SCALER_CSC0_ITR_R_709_3,
+ SCALER_CSC1_ITR_R_709_3,
+ SCALER_CSC2_ITR_R_709_3,
+ }, {
+ /* BT2020 */
+ SCALER_CSC0_ITR_R_2020,
+ SCALER_CSC1_ITR_R_2020,
+ SCALER_CSC2_ITR_R_2020,
+ }
+ }, {
+ /* Full range */
+ {
+ /* JFIF */
+ SCALER_CSC0_JPEG_JFIF,
+ SCALER_CSC1_JPEG_JFIF,
+ SCALER_CSC2_JPEG_JFIF,
+ }, {
+ /* BT709 */
+ SCALER_CSC0_ITR_R_709_3_FR,
+ SCALER_CSC1_ITR_R_709_3_FR,
+ SCALER_CSC2_ITR_R_709_3_FR,
+ }, {
+ /* BT2020 */
+ SCALER_CSC0_ITR_R_2020_FR,
+ SCALER_CSC1_ITR_R_2020_FR,
+ SCALER_CSC2_ITR_R_2020_FR,
+ }
+ }
+};
+
/* Writes out a full display list for an active plane to the plane's
* private dlist state.
*/
@@ -767,47 +814,90 @@ static int vc4_plane_mode_set(struct drm_plane *plane,
case DRM_FORMAT_MOD_BROADCOM_SAND128:
case DRM_FORMAT_MOD_BROADCOM_SAND256: {
uint32_t param = fourcc_mod_broadcom_param(fb->modifier);
- u32 tile_w, tile, x_off, pix_per_tile;
-
- hvs_format = HVS_PIXEL_FORMAT_H264;
-
- switch (base_format_mod) {
- case DRM_FORMAT_MOD_BROADCOM_SAND64:
- tiling = SCALER_CTL0_TILING_64B;
- tile_w = 64;
- break;
- case DRM_FORMAT_MOD_BROADCOM_SAND128:
- tiling = SCALER_CTL0_TILING_128B;
- tile_w = 128;
- break;
- case DRM_FORMAT_MOD_BROADCOM_SAND256:
- tiling = SCALER_CTL0_TILING_256B_OR_T;
- tile_w = 256;
- break;
- default:
- break;
- }
if (param > SCALER_TILE_HEIGHT_MASK) {
- DRM_DEBUG_KMS("SAND height too large (%d)\n", param);
+ DRM_DEBUG_KMS("SAND height too large (%d)\n",
+ param);
return -EINVAL;
}
- pix_per_tile = tile_w / fb->format->cpp[0];
- tile = vc4_state->src_x / pix_per_tile;
- x_off = vc4_state->src_x % pix_per_tile;
+ if (fb->format->format == DRM_FORMAT_P030) {
+ hvs_format = HVS_PIXEL_FORMAT_YCBCR_10BIT;
+ tiling = SCALER_CTL0_TILING_128B;
+ } else {
+ hvs_format = HVS_PIXEL_FORMAT_H264;
+
+ switch (base_format_mod) {
+ case DRM_FORMAT_MOD_BROADCOM_SAND64:
+ tiling = SCALER_CTL0_TILING_64B;
+ break;
+ case DRM_FORMAT_MOD_BROADCOM_SAND128:
+ tiling = SCALER_CTL0_TILING_128B;
+ break;
+ case DRM_FORMAT_MOD_BROADCOM_SAND256:
+ tiling = SCALER_CTL0_TILING_256B_OR_T;
+ break;
+ default:
+ return -EINVAL;
+ }
+ }
/* Adjust the base pointer to the first pixel to be scanned
* out.
+ *
+ * For P030, y_ptr [31:4] is the 128bit word for the start pixel
+ * y_ptr [3:0] is the pixel (0-11) contained within that 128bit
+ * word that should be taken as the first pixel.
+ * Ditto uv_ptr [31:4] vs [3:0], however [3:0] contains the
+ * element within the 128bit word, eg for pixel 3 the value
+ * should be 6.
*/
for (i = 0; i < num_planes; i++) {
+ u32 tile_w, tile, x_off, pix_per_tile;
+
+ if (fb->format->format == DRM_FORMAT_P030) {
+ /*
+ * Spec says: bits [31:4] of the given address
+ * should point to the 128-bit word containing
+ * the desired starting pixel, and bits[3:0]
+ * should be between 0 and 11, indicating which
+ * of the 12-pixels in that 128-bit word is the
+ * first pixel to be used
+ */
+ u32 remaining_pixels = vc4_state->src_x % 96;
+ u32 aligned = remaining_pixels / 12;
+ u32 last_bits = remaining_pixels % 12;
+
+ x_off = aligned * 16 + last_bits;
+ tile_w = 128;
+ pix_per_tile = 96;
+ } else {
+ switch (base_format_mod) {
+ case DRM_FORMAT_MOD_BROADCOM_SAND64:
+ tile_w = 64;
+ break;
+ case DRM_FORMAT_MOD_BROADCOM_SAND128:
+ tile_w = 128;
+ break;
+ case DRM_FORMAT_MOD_BROADCOM_SAND256:
+ tile_w = 256;
+ break;
+ default:
+ return -EINVAL;
+ }
+ pix_per_tile = tile_w / fb->format->cpp[0];
+ x_off = (vc4_state->src_x % pix_per_tile) /
+ (i ? h_subsample : 1) *
+ fb->format->cpp[i];
+ }
+
+ tile = vc4_state->src_x / pix_per_tile;
+
vc4_state->offsets[i] += param * tile_w * tile;
vc4_state->offsets[i] += src_y /
(i ? v_subsample : 1) *
tile_w;
- vc4_state->offsets[i] += x_off /
- (i ? h_subsample : 1) *
- fb->format->cpp[i];
+ vc4_state->offsets[i] += x_off & ~(i ? 1 : 0);
}
pitch0 = VC4_SET_FIELD(param, SCALER_TILE_HEIGHT);
@@ -960,7 +1050,8 @@ static int vc4_plane_mode_set(struct drm_plane *plane,
/* Pitch word 1/2 */
for (i = 1; i < num_planes; i++) {
- if (hvs_format != HVS_PIXEL_FORMAT_H264) {
+ if (hvs_format != HVS_PIXEL_FORMAT_H264 &&
+ hvs_format != HVS_PIXEL_FORMAT_YCBCR_10BIT) {
vc4_dlist_write(vc4_state,
VC4_SET_FIELD(fb->pitches[i],
SCALER_SRC_PITCH));
@@ -971,9 +1062,20 @@ static int vc4_plane_mode_set(struct drm_plane *plane,
/* Colorspace conversion words */
if (vc4_state->is_yuv) {
- vc4_dlist_write(vc4_state, SCALER_CSC0_ITR_R_601_5);
- vc4_dlist_write(vc4_state, SCALER_CSC1_ITR_R_601_5);
- vc4_dlist_write(vc4_state, SCALER_CSC2_ITR_R_601_5);
+ enum drm_color_encoding color_encoding = state->color_encoding;
+ enum drm_color_range color_range = state->color_range;
+ const u32 *ccm;
+
+ if (color_encoding >= DRM_COLOR_ENCODING_MAX)
+ color_encoding = DRM_COLOR_YCBCR_BT601;
+ if (color_range >= DRM_COLOR_RANGE_MAX)
+ color_range = DRM_COLOR_YCBCR_LIMITED_RANGE;
+
+ ccm = colorspace_coeffs[color_range][color_encoding];
+
+ vc4_dlist_write(vc4_state, ccm[0]);
+ vc4_dlist_write(vc4_state, ccm[1]);
+ vc4_dlist_write(vc4_state, ccm[2]);
}
vc4_state->lbm_offset = 0;
@@ -1320,6 +1422,13 @@ static bool vc4_format_mod_supported(struct drm_plane *plane,
default:
return false;
}
+ case DRM_FORMAT_P030:
+ switch (fourcc_mod_broadcom_mod(modifier)) {
+ case DRM_FORMAT_MOD_BROADCOM_SAND128:
+ return true;
+ default:
+ return false;
+ }
case DRM_FORMAT_RGBX1010102:
case DRM_FORMAT_BGRX1010102:
case DRM_FORMAT_RGBA1010102:
@@ -1352,8 +1461,11 @@ struct drm_plane *vc4_plane_init(struct drm_device *dev,
struct drm_plane *plane = NULL;
struct vc4_plane *vc4_plane;
u32 formats[ARRAY_SIZE(hvs_formats)];
+ int num_formats = 0;
int ret = 0;
unsigned i;
+ bool hvs5 = of_device_is_compatible(dev->dev->of_node,
+ "brcm,bcm2711-vc5");
static const uint64_t modifiers[] = {
DRM_FORMAT_MOD_BROADCOM_VC4_T_TILED,
DRM_FORMAT_MOD_BROADCOM_SAND128,
@@ -1368,13 +1480,17 @@ struct drm_plane *vc4_plane_init(struct drm_device *dev,
if (!vc4_plane)
return ERR_PTR(-ENOMEM);
- for (i = 0; i < ARRAY_SIZE(hvs_formats); i++)
- formats[i] = hvs_formats[i].drm;
+ for (i = 0; i < ARRAY_SIZE(hvs_formats); i++) {
+ if (!hvs_formats[i].hvs5_only || hvs5) {
+ formats[num_formats] = hvs_formats[i].drm;
+ num_formats++;
+ }
+ }
plane = &vc4_plane->base;
ret = drm_universal_plane_init(dev, plane, 0,
&vc4_plane_funcs,
- formats, ARRAY_SIZE(formats),
+ formats, num_formats,
modifiers, type, NULL);
if (ret)
return ERR_PTR(ret);
@@ -1388,6 +1504,15 @@ struct drm_plane *vc4_plane_init(struct drm_device *dev,
DRM_MODE_REFLECT_X |
DRM_MODE_REFLECT_Y);
+ drm_plane_create_color_properties(plane,
+ BIT(DRM_COLOR_YCBCR_BT601) |
+ BIT(DRM_COLOR_YCBCR_BT709) |
+ BIT(DRM_COLOR_YCBCR_BT2020),
+ BIT(DRM_COLOR_YCBCR_LIMITED_RANGE) |
+ BIT(DRM_COLOR_YCBCR_FULL_RANGE),
+ DRM_COLOR_YCBCR_BT709,
+ DRM_COLOR_YCBCR_LIMITED_RANGE);
+
return plane;
}
diff --git a/drivers/gpu/drm/vc4/vc4_regs.h b/drivers/gpu/drm/vc4/vc4_regs.h
index 489f921..7538b84 100644
--- a/drivers/gpu/drm/vc4/vc4_regs.h
+++ b/drivers/gpu/drm/vc4/vc4_regs.h
@@ -975,7 +975,10 @@ enum hvs_pixel_format {
#define SCALER_CSC0_COEF_CR_OFS_SHIFT 0
#define SCALER_CSC0_ITR_R_601_5 0x00f00000
#define SCALER_CSC0_ITR_R_709_3 0x00f00000
+#define SCALER_CSC0_ITR_R_2020 0x00f00000
#define SCALER_CSC0_JPEG_JFIF 0x00000000
+#define SCALER_CSC0_ITR_R_709_3_FR 0x00000000
+#define SCALER_CSC0_ITR_R_2020_FR 0x00000000
/* S2.8 contribution of Cb to Green */
#define SCALER_CSC1_COEF_CB_GRN_MASK VC4_MASK(31, 22)
@@ -990,8 +993,11 @@ enum hvs_pixel_format {
#define SCALER_CSC1_COEF_CR_BLU_MASK VC4_MASK(1, 0)
#define SCALER_CSC1_COEF_CR_BLU_SHIFT 0
#define SCALER_CSC1_ITR_R_601_5 0xe73304a8
-#define SCALER_CSC1_ITR_R_709_3 0xf2b784a8
-#define SCALER_CSC1_JPEG_JFIF 0xea34a400
+#define SCALER_CSC1_ITR_R_709_3 0xf27784a8
+#define SCALER_CSC1_ITR_R_2020 0xf43594a8
+#define SCALER_CSC1_JPEG_JFIF 0xea349400
+#define SCALER_CSC1_ITR_R_709_3_FR 0xf4388400
+#define SCALER_CSC1_ITR_R_2020_FR 0xf5b6d400
/* S2.8 contribution of Cb to Red */
#define SCALER_CSC2_COEF_CB_RED_MASK VC4_MASK(29, 20)
@@ -1002,9 +1008,12 @@ enum hvs_pixel_format {
/* S2.8 contribution of Cb to Blue */
#define SCALER_CSC2_COEF_CB_BLU_MASK VC4_MASK(19, 10)
#define SCALER_CSC2_COEF_CB_BLU_SHIFT 10
-#define SCALER_CSC2_ITR_R_601_5 0x00066204
-#define SCALER_CSC2_ITR_R_709_3 0x00072a1c
-#define SCALER_CSC2_JPEG_JFIF 0x000599c5
+#define SCALER_CSC2_ITR_R_601_5 0x00066604
+#define SCALER_CSC2_ITR_R_709_3 0x00072e1d
+#define SCALER_CSC2_ITR_R_2020 0x0006b624
+#define SCALER_CSC2_JPEG_JFIF 0x00059dc6
+#define SCALER_CSC2_ITR_R_709_3_FR 0x00064ddb
+#define SCALER_CSC2_ITR_R_2020_FR 0x0005e5e2
#define SCALER_TPZ0_VERT_RECALC BIT(31)
#define SCALER_TPZ0_SCALE_MASK VC4_MASK(28, 8)
diff --git a/drivers/gpu/drm/vc4/vc4_txp.c b/drivers/gpu/drm/vc4/vc4_txp.c
index 2fc7f4b..9809ca3 100644
--- a/drivers/gpu/drm/vc4/vc4_txp.c
+++ b/drivers/gpu/drm/vc4/vc4_txp.c
@@ -391,7 +391,6 @@ static int vc4_txp_atomic_check(struct drm_crtc *crtc,
{
struct drm_crtc_state *crtc_state = drm_atomic_get_new_crtc_state(state,
crtc);
- struct vc4_crtc_state *vc4_state = to_vc4_crtc_state(crtc_state);
int ret;
ret = vc4_hvs_atomic_check(crtc, state);
@@ -399,7 +398,6 @@ static int vc4_txp_atomic_check(struct drm_crtc *crtc,
return ret;
crtc_state->no_vblank = true;
- vc4_state->feed_txp = true;
return 0;
}
@@ -437,6 +435,7 @@ static void vc4_txp_atomic_disable(struct drm_crtc *crtc,
static const struct drm_crtc_helper_funcs vc4_txp_crtc_helper_funcs = {
.atomic_check = vc4_txp_atomic_check,
+ .atomic_begin = vc4_hvs_atomic_begin,
.atomic_flush = vc4_hvs_atomic_flush,
.atomic_enable = vc4_txp_atomic_enable,
.atomic_disable = vc4_txp_atomic_disable,
@@ -482,6 +481,7 @@ static int vc4_txp_bind(struct device *dev, struct device *master, void *data)
vc4_crtc->pdev = pdev;
vc4_crtc->data = &vc4_txp_crtc_data;
+ vc4_crtc->feeds_txp = true;
txp->pdev = pdev;
diff --git a/drivers/gpu/drm/vgem/vgem_drv.c b/drivers/gpu/drm/vgem/vgem_drv.c
index a87eafa..c5e3e54 100644
--- a/drivers/gpu/drm/vgem/vgem_drv.c
+++ b/drivers/gpu/drm/vgem/vgem_drv.c
@@ -97,7 +97,7 @@ static struct drm_gem_object *vgem_gem_create_object(struct drm_device *dev, siz
obj = kzalloc(sizeof(*obj), GFP_KERNEL);
if (!obj)
- return NULL;
+ return ERR_PTR(-ENOMEM);
/*
* vgem doesn't have any begin/end cpu access ioctls, therefore must use
diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.c b/drivers/gpu/drm/virtio/virtgpu_drv.c
index 5072dbb0..5f25a8d 100644
--- a/drivers/gpu/drm/virtio/virtgpu_drv.c
+++ b/drivers/gpu/drm/virtio/virtgpu_drv.c
@@ -27,7 +27,6 @@
*/
#include <linux/module.h>
-#include <linux/console.h>
#include <linux/pci.h>
#include <linux/poll.h>
#include <linux/wait.h>
@@ -104,7 +103,7 @@ static int virtio_gpu_probe(struct virtio_device *vdev)
struct drm_device *dev;
int ret;
- if (vgacon_text_force() && virtio_gpu_modeset == -1)
+ if (drm_firmware_drivers_only() && virtio_gpu_modeset == -1)
return -EINVAL;
if (virtio_gpu_modeset == 0)
diff --git a/drivers/gpu/drm/virtio/virtgpu_ioctl.c b/drivers/gpu/drm/virtio/virtgpu_ioctl.c
index 3607646..c708bab 100644
--- a/drivers/gpu/drm/virtio/virtgpu_ioctl.c
+++ b/drivers/gpu/drm/virtio/virtgpu_ioctl.c
@@ -774,7 +774,7 @@ static int virtio_gpu_context_init_ioctl(struct drm_device *dev,
goto out_unlock;
}
- if ((vgdev->capset_id_mask & (1 << value)) == 0) {
+ if ((vgdev->capset_id_mask & (1ULL << value)) == 0) {
ret = -EINVAL;
goto out_unlock;
}
@@ -819,7 +819,7 @@ static int virtio_gpu_context_init_ioctl(struct drm_device *dev,
if (vfpriv->ring_idx_mask) {
valid_ring_mask = 0;
for (i = 0; i < vfpriv->num_rings; i++)
- valid_ring_mask |= 1 << i;
+ valid_ring_mask |= 1ULL << i;
if (~valid_ring_mask & vfpriv->ring_idx_mask) {
ret = -EINVAL;
diff --git a/drivers/gpu/drm/virtio/virtgpu_object.c b/drivers/gpu/drm/virtio/virtgpu_object.c
index f648b0e..baef2c5 100644
--- a/drivers/gpu/drm/virtio/virtgpu_object.c
+++ b/drivers/gpu/drm/virtio/virtgpu_object.c
@@ -79,10 +79,10 @@ void virtio_gpu_cleanup_object(struct virtio_gpu_object *bo)
sg_free_table(shmem->pages);
kfree(shmem->pages);
shmem->pages = NULL;
- drm_gem_shmem_unpin(&bo->base.base);
+ drm_gem_shmem_unpin(&bo->base);
}
- drm_gem_shmem_free_object(&bo->base.base);
+ drm_gem_shmem_free(&bo->base);
} else if (virtio_gpu_is_vram(bo)) {
struct virtio_gpu_object_vram *vram = to_virtio_gpu_vram(bo);
@@ -116,15 +116,14 @@ static const struct drm_gem_object_funcs virtio_gpu_shmem_funcs = {
.free = virtio_gpu_free_object,
.open = virtio_gpu_gem_object_open,
.close = virtio_gpu_gem_object_close,
-
- .print_info = drm_gem_shmem_print_info,
+ .print_info = drm_gem_shmem_object_print_info,
.export = virtgpu_gem_prime_export,
- .pin = drm_gem_shmem_pin,
- .unpin = drm_gem_shmem_unpin,
- .get_sg_table = drm_gem_shmem_get_sg_table,
- .vmap = drm_gem_shmem_vmap,
- .vunmap = drm_gem_shmem_vunmap,
- .mmap = drm_gem_shmem_mmap,
+ .pin = drm_gem_shmem_object_pin,
+ .unpin = drm_gem_shmem_object_unpin,
+ .get_sg_table = drm_gem_shmem_object_get_sg_table,
+ .vmap = drm_gem_shmem_object_vmap,
+ .vunmap = drm_gem_shmem_object_vunmap,
+ .mmap = drm_gem_shmem_object_mmap,
};
bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo)
@@ -140,7 +139,7 @@ struct drm_gem_object *virtio_gpu_create_object(struct drm_device *dev,
shmem = kzalloc(sizeof(*shmem), GFP_KERNEL);
if (!shmem)
- return NULL;
+ return ERR_PTR(-ENOMEM);
dshmem = &shmem->base.base;
dshmem->base.funcs = &virtio_gpu_shmem_funcs;
@@ -157,7 +156,7 @@ static int virtio_gpu_object_shmem_init(struct virtio_gpu_device *vgdev,
struct scatterlist *sg;
int si, ret;
- ret = drm_gem_shmem_pin(&bo->base.base);
+ ret = drm_gem_shmem_pin(&bo->base);
if (ret < 0)
return -EINVAL;
@@ -167,9 +166,9 @@ static int virtio_gpu_object_shmem_init(struct virtio_gpu_device *vgdev,
* dma-ops. This is discouraged for other drivers, but should be fine
* since virtio_gpu doesn't support dma-buf import from other devices.
*/
- shmem->pages = drm_gem_shmem_get_sg_table(&bo->base.base);
+ shmem->pages = drm_gem_shmem_get_sg_table(&bo->base);
if (!shmem->pages) {
- drm_gem_shmem_unpin(&bo->base.base);
+ drm_gem_shmem_unpin(&bo->base);
return -EINVAL;
}
@@ -277,6 +276,6 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
err_put_id:
virtio_gpu_resource_id_put(vgdev, bo->hw_res_handle);
err_free_gem:
- drm_gem_shmem_free_object(&shmem_obj->base);
+ drm_gem_shmem_free(shmem_obj);
return ret;
}
diff --git a/drivers/gpu/drm/vmwgfx/Kconfig b/drivers/gpu/drm/vmwgfx/Kconfig
index c9ce47c..a4fabe2 100644
--- a/drivers/gpu/drm/vmwgfx/Kconfig
+++ b/drivers/gpu/drm/vmwgfx/Kconfig
@@ -4,6 +4,7 @@
depends on DRM && PCI && MMU
depends on X86 || ARM64
select DRM_TTM
+ select DRM_TTM_HELPER
select MAPPING_DIRTY_HELPERS
# Only needed for the transitional use of drm_crtc_init - can be removed
# again once vmwgfx sets up the primary plane itself.
diff --git a/drivers/gpu/drm/vmwgfx/Makefile b/drivers/gpu/drm/vmwgfx/Makefile
index bc323f7..eee73b9 100644
--- a/drivers/gpu/drm/vmwgfx/Makefile
+++ b/drivers/gpu/drm/vmwgfx/Makefile
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: GPL-2.0
-vmwgfx-y := vmwgfx_execbuf.o vmwgfx_gmr.o vmwgfx_kms.o vmwgfx_drv.o \
+vmwgfx-y := vmwgfx_execbuf.o vmwgfx_gmr.o vmwgfx_hashtab.o vmwgfx_kms.o vmwgfx_drv.o \
vmwgfx_ioctl.o vmwgfx_resource.o vmwgfx_ttm_buffer.o \
vmwgfx_cmd.o vmwgfx_irq.o vmwgfx_ldu.o vmwgfx_ttm_glue.o \
vmwgfx_overlay.o vmwgfx_gmrid_manager.o vmwgfx_fence.o \
@@ -9,9 +9,9 @@
vmwgfx_cotable.o vmwgfx_so.o vmwgfx_binding.o vmwgfx_msg.o \
vmwgfx_simple_resource.o vmwgfx_va.o vmwgfx_blit.o \
vmwgfx_validation.o vmwgfx_page_dirty.o vmwgfx_streamoutput.o \
- vmwgfx_devcaps.o ttm_object.o ttm_memory.o
+ vmwgfx_devcaps.o ttm_object.o vmwgfx_system_manager.o \
+ vmwgfx_gem.o
vmwgfx-$(CONFIG_DRM_FBDEV_EMULATION) += vmwgfx_fb.o
-vmwgfx-$(CONFIG_TRANSPARENT_HUGEPAGE) += vmwgfx_thp.o
obj-$(CONFIG_DRM_VMWGFX) := vmwgfx.o
diff --git a/drivers/gpu/drm/vmwgfx/device_include/svga3d_cmd.h b/drivers/gpu/drm/vmwgfx/device_include/svga3d_cmd.h
index 945c84b..d90d940 100644
--- a/drivers/gpu/drm/vmwgfx/device_include/svga3d_cmd.h
+++ b/drivers/gpu/drm/vmwgfx/device_include/svga3d_cmd.h
@@ -1,6 +1,6 @@
-/**********************************************************
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/*
* Copyright 2012-2021 VMware, Inc.
- * SPDX-License-Identifier: GPL-2.0 OR MIT
*
* Permission is hereby granted, free of charge, to any person
* obtaining a copy of this software and associated documentation
@@ -22,7 +22,7 @@
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*
- **********************************************************/
+ */
/*
* svga3d_cmd.h --
diff --git a/drivers/gpu/drm/vmwgfx/device_include/svga3d_devcaps.h b/drivers/gpu/drm/vmwgfx/device_include/svga3d_devcaps.h
index 379ec15..815d0ab 100644
--- a/drivers/gpu/drm/vmwgfx/device_include/svga3d_devcaps.h
+++ b/drivers/gpu/drm/vmwgfx/device_include/svga3d_devcaps.h
@@ -1,6 +1,6 @@
-/**********************************************************
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/*
* Copyright 1998-2021 VMware, Inc.
- * SPDX-License-Identifier: GPL-2.0 OR MIT
*
* Permission is hereby granted, free of charge, to any person
* obtaining a copy of this software and associated documentation
@@ -22,7 +22,7 @@
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*
- **********************************************************/
+ */
/*
* svga3d_devcaps.h --
@@ -347,6 +347,10 @@ typedef uint32 SVGA3dDevCapIndex;
#define SVGA3D_DEVCAP_SM5 258
#define SVGA3D_DEVCAP_MULTISAMPLE_8X 259
+#define SVGA3D_DEVCAP_MAX_FORCED_SAMPLE_COUNT 260
+
+#define SVGA3D_DEVCAP_GL43 261
+
#define SVGA3D_DEVCAP_MAX 262
#define SVGA3D_DXFMT_SUPPORTED (1 << 0)
diff --git a/drivers/gpu/drm/vmwgfx/device_include/svga3d_dx.h b/drivers/gpu/drm/vmwgfx/device_include/svga3d_dx.h
index 5af442d..925bf4b 100644
--- a/drivers/gpu/drm/vmwgfx/device_include/svga3d_dx.h
+++ b/drivers/gpu/drm/vmwgfx/device_include/svga3d_dx.h
@@ -1,6 +1,6 @@
-/**********************************************************
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/*
* Copyright 2012-2021 VMware, Inc.
- * SPDX-License-Identifier: GPL-2.0 OR MIT
*
* Permission is hereby granted, free of charge, to any person
* obtaining a copy of this software and associated documentation
@@ -22,7 +22,7 @@
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*
- **********************************************************/
+ */
/*
* svga3d_dx.h --
@@ -508,11 +508,11 @@ typedef struct SVGA3dCmdDXSetPredication {
#pragma pack(pop)
#pragma pack(push, 1)
-typedef struct MKS3dDXSOState {
+typedef struct SVGA3dDXSOState {
uint32 offset;
uint32 intOffset;
- uint32 vertexCount;
- uint32 dead;
+ uint32 dead1;
+ uint32 dead2;
} SVGA3dDXSOState;
#pragma pack(pop)
diff --git a/drivers/gpu/drm/vmwgfx/device_include/svga3d_limits.h b/drivers/gpu/drm/vmwgfx/device_include/svga3d_limits.h
index 35494a7..6103b41 100644
--- a/drivers/gpu/drm/vmwgfx/device_include/svga3d_limits.h
+++ b/drivers/gpu/drm/vmwgfx/device_include/svga3d_limits.h
@@ -1,6 +1,6 @@
-/**********************************************************
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/*
* Copyright 2012-2021 VMware, Inc.
- * SPDX-License-Identifier: GPL-2.0 OR MIT
*
* Permission is hereby granted, free of charge, to any person
* obtaining a copy of this software and associated documentation
@@ -22,7 +22,7 @@
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*
- **********************************************************/
+ */
/*
* svga3d_limits.h --
@@ -82,4 +82,6 @@
#define SVGA3D_MIN_SBX_DATA_SIZE (GBYTES_2_BYTES(1))
#define SVGA3D_MAX_SBX_DATA_SIZE (GBYTES_2_BYTES(4))
+#define SVGA3D_MIN_SBX_DATA_SIZE_DVM (MBYTES_2_BYTES(900))
+#define SVGA3D_MAX_SBX_DATA_SIZE_DVM (MBYTES_2_BYTES(910))
#endif
diff --git a/drivers/gpu/drm/vmwgfx/device_include/svga3d_reg.h b/drivers/gpu/drm/vmwgfx/device_include/svga3d_reg.h
index 988d850..b24b4f5 100644
--- a/drivers/gpu/drm/vmwgfx/device_include/svga3d_reg.h
+++ b/drivers/gpu/drm/vmwgfx/device_include/svga3d_reg.h
@@ -1,6 +1,6 @@
-/**********************************************************
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/*
* Copyright 1998-2015 VMware, Inc.
- * SPDX-License-Identifier: GPL-2.0 OR MIT
*
* Permission is hereby granted, free of charge, to any person
* obtaining a copy of this software and associated documentation
@@ -22,7 +22,7 @@
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*
- **********************************************************/
+ */
/*
* svga3d_reg.h --
diff --git a/drivers/gpu/drm/vmwgfx/device_include/svga3d_types.h b/drivers/gpu/drm/vmwgfx/device_include/svga3d_types.h
index 70b88ee..e9219eb 100644
--- a/drivers/gpu/drm/vmwgfx/device_include/svga3d_types.h
+++ b/drivers/gpu/drm/vmwgfx/device_include/svga3d_types.h
@@ -1,6 +1,6 @@
-/**********************************************************
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/*
* Copyright 2012-2021 VMware, Inc.
- * SPDX-License-Identifier: GPL-2.0 OR MIT
*
* Permission is hereby granted, free of charge, to any person
* obtaining a copy of this software and associated documentation
@@ -22,7 +22,7 @@
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*
- **********************************************************/
+ */
/*
* svga3d_types.h --
@@ -370,7 +370,6 @@ typedef enum SVGA3dSurfaceFormat {
#define SVGA3D_SURFACE_TRANSFER_FROM_BUFFER (CONST64U(1) << 30)
#define SVGA3D_SURFACE_RESERVED1 (CONST64U(1) << 31)
-#define SVGA3D_SURFACE_VADECODE SVGA3D_SURFACE_RESERVED1
#define SVGA3D_SURFACE_MULTISAMPLE (CONST64U(1) << 32)
diff --git a/drivers/gpu/drm/vmwgfx/device_include/svga_escape.h b/drivers/gpu/drm/vmwgfx/device_include/svga_escape.h
index bf242c2..405f20f 100644
--- a/drivers/gpu/drm/vmwgfx/device_include/svga_escape.h
+++ b/drivers/gpu/drm/vmwgfx/device_include/svga_escape.h
@@ -1,6 +1,6 @@
-/**********************************************************
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/*
* Copyright 2007,2020 VMware, Inc.
- * SPDX-License-Identifier: GPL-2.0 OR MIT
*
* Permission is hereby granted, free of charge, to any person
* obtaining a copy of this software and associated documentation
@@ -22,7 +22,7 @@
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*
- **********************************************************/
+ */
/*
* svga_escape.h --
diff --git a/drivers/gpu/drm/vmwgfx/device_include/svga_overlay.h b/drivers/gpu/drm/vmwgfx/device_include/svga_overlay.h
index aec17c3..691f48f 100644
--- a/drivers/gpu/drm/vmwgfx/device_include/svga_overlay.h
+++ b/drivers/gpu/drm/vmwgfx/device_include/svga_overlay.h
@@ -1,6 +1,6 @@
-/**********************************************************
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/*
* Copyright 2007-2021 VMware, Inc.
- * SPDX-License-Identifier: GPL-2.0 OR MIT
*
* Permission is hereby granted, free of charge, to any person
* obtaining a copy of this software and associated documentation
@@ -22,7 +22,7 @@
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*
- **********************************************************/
+ */
/*
* svga_overlay.h --
diff --git a/drivers/gpu/drm/vmwgfx/device_include/svga_reg.h b/drivers/gpu/drm/vmwgfx/device_include/svga_reg.h
index b360255..acabdb5 100644
--- a/drivers/gpu/drm/vmwgfx/device_include/svga_reg.h
+++ b/drivers/gpu/drm/vmwgfx/device_include/svga_reg.h
@@ -1,6 +1,6 @@
-/**********************************************************
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/*
* Copyright 1998-2021 VMware, Inc.
- * SPDX-License-Identifier: GPL-2.0 OR MIT
*
* Permission is hereby granted, free of charge, to any person
* obtaining a copy of this software and associated documentation
@@ -22,7 +22,7 @@
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*
- **********************************************************/
+ */
/*
* svga_reg.h --
@@ -442,6 +442,7 @@ typedef struct {
#define SVGA_CAP2_TRACE_FULL_FB 0x00002000
#define SVGA_CAP2_EXTRA_REGS 0x00004000
#define SVGA_CAP2_LO_STAGING 0x00008000
+#define SVGA_CAP2_VIDEO_BLT 0x00010000
#define SVGA_CAP2_RESERVED 0x80000000
typedef enum {
@@ -450,9 +451,10 @@ typedef enum {
SVGABackdoorCap3dHWVersion = 2,
SVGABackdoorCapDeviceCaps2 = 3,
SVGABackdoorCapDevelCaps = 4,
- SVGABackdoorDevelRenderer = 5,
- SVGABackdoorDevelUsingISB = 6,
- SVGABackdoorCapMax = 7,
+ SVGABackdoorCapDevCaps = 5,
+ SVGABackdoorDevelRenderer = 6,
+ SVGABackdoorDevelUsingISB = 7,
+ SVGABackdoorCapMax = 8,
} SVGABackdoorCapType;
enum {
diff --git a/drivers/gpu/drm/vmwgfx/ttm_memory.c b/drivers/gpu/drm/vmwgfx/ttm_memory.c
deleted file mode 100644
index 7f7fe35..0000000
--- a/drivers/gpu/drm/vmwgfx/ttm_memory.c
+++ /dev/null
@@ -1,683 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 OR MIT */
-/**************************************************************************
- *
- * Copyright (c) 2006-2009 VMware, Inc., Palo Alto, CA., USA
- * All Rights Reserved.
- *
- * Permission is hereby granted, free of charge, to any person obtaining a
- * copy of this software and associated documentation files (the
- * "Software"), to deal in the Software without restriction, including
- * without limitation the rights to use, copy, modify, merge, publish,
- * distribute, sub license, and/or sell copies of the Software, and to
- * permit persons to whom the Software is furnished to do so, subject to
- * the following conditions:
- *
- * The above copyright notice and this permission notice (including the
- * next paragraph) shall be included in all copies or substantial portions
- * of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. IN NO EVENT SHALL
- * THE COPYRIGHT HOLDERS, AUTHORS AND/OR ITS SUPPLIERS BE LIABLE FOR ANY CLAIM,
- * DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
- * OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
- * USE OR OTHER DEALINGS IN THE SOFTWARE.
- *
- **************************************************************************/
-
-#define pr_fmt(fmt) "[TTM] " fmt
-
-#include <linux/spinlock.h>
-#include <linux/sched.h>
-#include <linux/wait.h>
-#include <linux/mm.h>
-#include <linux/module.h>
-#include <linux/slab.h>
-#include <linux/swap.h>
-
-#include <drm/drm_device.h>
-#include <drm/drm_file.h>
-#include <drm/ttm/ttm_device.h>
-
-#include "ttm_memory.h"
-
-#define TTM_MEMORY_ALLOC_RETRIES 4
-
-struct ttm_mem_global ttm_mem_glob;
-EXPORT_SYMBOL(ttm_mem_glob);
-
-struct ttm_mem_zone {
- struct kobject kobj;
- struct ttm_mem_global *glob;
- const char *name;
- uint64_t zone_mem;
- uint64_t emer_mem;
- uint64_t max_mem;
- uint64_t swap_limit;
- uint64_t used_mem;
-};
-
-static struct attribute ttm_mem_sys = {
- .name = "zone_memory",
- .mode = S_IRUGO
-};
-static struct attribute ttm_mem_emer = {
- .name = "emergency_memory",
- .mode = S_IRUGO | S_IWUSR
-};
-static struct attribute ttm_mem_max = {
- .name = "available_memory",
- .mode = S_IRUGO | S_IWUSR
-};
-static struct attribute ttm_mem_swap = {
- .name = "swap_limit",
- .mode = S_IRUGO | S_IWUSR
-};
-static struct attribute ttm_mem_used = {
- .name = "used_memory",
- .mode = S_IRUGO
-};
-
-static void ttm_mem_zone_kobj_release(struct kobject *kobj)
-{
- struct ttm_mem_zone *zone =
- container_of(kobj, struct ttm_mem_zone, kobj);
-
- pr_info("Zone %7s: Used memory at exit: %llu KiB\n",
- zone->name, (unsigned long long)zone->used_mem >> 10);
- kfree(zone);
-}
-
-static ssize_t ttm_mem_zone_show(struct kobject *kobj,
- struct attribute *attr,
- char *buffer)
-{
- struct ttm_mem_zone *zone =
- container_of(kobj, struct ttm_mem_zone, kobj);
- uint64_t val = 0;
-
- spin_lock(&zone->glob->lock);
- if (attr == &ttm_mem_sys)
- val = zone->zone_mem;
- else if (attr == &ttm_mem_emer)
- val = zone->emer_mem;
- else if (attr == &ttm_mem_max)
- val = zone->max_mem;
- else if (attr == &ttm_mem_swap)
- val = zone->swap_limit;
- else if (attr == &ttm_mem_used)
- val = zone->used_mem;
- spin_unlock(&zone->glob->lock);
-
- return snprintf(buffer, PAGE_SIZE, "%llu\n",
- (unsigned long long) val >> 10);
-}
-
-static void ttm_check_swapping(struct ttm_mem_global *glob);
-
-static ssize_t ttm_mem_zone_store(struct kobject *kobj,
- struct attribute *attr,
- const char *buffer,
- size_t size)
-{
- struct ttm_mem_zone *zone =
- container_of(kobj, struct ttm_mem_zone, kobj);
- int chars;
- unsigned long val;
- uint64_t val64;
-
- chars = sscanf(buffer, "%lu", &val);
- if (chars == 0)
- return size;
-
- val64 = val;
- val64 <<= 10;
-
- spin_lock(&zone->glob->lock);
- if (val64 > zone->zone_mem)
- val64 = zone->zone_mem;
- if (attr == &ttm_mem_emer) {
- zone->emer_mem = val64;
- if (zone->max_mem > val64)
- zone->max_mem = val64;
- } else if (attr == &ttm_mem_max) {
- zone->max_mem = val64;
- if (zone->emer_mem < val64)
- zone->emer_mem = val64;
- } else if (attr == &ttm_mem_swap)
- zone->swap_limit = val64;
- spin_unlock(&zone->glob->lock);
-
- ttm_check_swapping(zone->glob);
-
- return size;
-}
-
-static struct attribute *ttm_mem_zone_attrs[] = {
- &ttm_mem_sys,
- &ttm_mem_emer,
- &ttm_mem_max,
- &ttm_mem_swap,
- &ttm_mem_used,
- NULL
-};
-
-static const struct sysfs_ops ttm_mem_zone_ops = {
- .show = &ttm_mem_zone_show,
- .store = &ttm_mem_zone_store
-};
-
-static struct kobj_type ttm_mem_zone_kobj_type = {
- .release = &ttm_mem_zone_kobj_release,
- .sysfs_ops = &ttm_mem_zone_ops,
- .default_attrs = ttm_mem_zone_attrs,
-};
-
-static struct attribute ttm_mem_global_lower_mem_limit = {
- .name = "lower_mem_limit",
- .mode = S_IRUGO | S_IWUSR
-};
-
-static ssize_t ttm_mem_global_show(struct kobject *kobj,
- struct attribute *attr,
- char *buffer)
-{
- struct ttm_mem_global *glob =
- container_of(kobj, struct ttm_mem_global, kobj);
- uint64_t val = 0;
-
- spin_lock(&glob->lock);
- val = glob->lower_mem_limit;
- spin_unlock(&glob->lock);
- /* convert from number of pages to KB */
- val <<= (PAGE_SHIFT - 10);
- return snprintf(buffer, PAGE_SIZE, "%llu\n",
- (unsigned long long) val);
-}
-
-static ssize_t ttm_mem_global_store(struct kobject *kobj,
- struct attribute *attr,
- const char *buffer,
- size_t size)
-{
- int chars;
- uint64_t val64;
- unsigned long val;
- struct ttm_mem_global *glob =
- container_of(kobj, struct ttm_mem_global, kobj);
-
- chars = sscanf(buffer, "%lu", &val);
- if (chars == 0)
- return size;
-
- val64 = val;
- /* convert from KB to number of pages */
- val64 >>= (PAGE_SHIFT - 10);
-
- spin_lock(&glob->lock);
- glob->lower_mem_limit = val64;
- spin_unlock(&glob->lock);
-
- return size;
-}
-
-static struct attribute *ttm_mem_global_attrs[] = {
- &ttm_mem_global_lower_mem_limit,
- NULL
-};
-
-static const struct sysfs_ops ttm_mem_global_ops = {
- .show = &ttm_mem_global_show,
- .store = &ttm_mem_global_store,
-};
-
-static struct kobj_type ttm_mem_glob_kobj_type = {
- .sysfs_ops = &ttm_mem_global_ops,
- .default_attrs = ttm_mem_global_attrs,
-};
-
-static bool ttm_zones_above_swap_target(struct ttm_mem_global *glob,
- bool from_wq, uint64_t extra)
-{
- unsigned int i;
- struct ttm_mem_zone *zone;
- uint64_t target;
-
- for (i = 0; i < glob->num_zones; ++i) {
- zone = glob->zones[i];
-
- if (from_wq)
- target = zone->swap_limit;
- else if (capable(CAP_SYS_ADMIN))
- target = zone->emer_mem;
- else
- target = zone->max_mem;
-
- target = (extra > target) ? 0ULL : target;
-
- if (zone->used_mem > target)
- return true;
- }
- return false;
-}
-
-/*
- * At this point we only support a single shrink callback.
- * Extend this if needed, perhaps using a linked list of callbacks.
- * Note that this function is reentrant:
- * many threads may try to swap out at any given time.
- */
-
-static void ttm_shrink(struct ttm_mem_global *glob, bool from_wq,
- uint64_t extra, struct ttm_operation_ctx *ctx)
-{
- int ret;
-
- spin_lock(&glob->lock);
-
- while (ttm_zones_above_swap_target(glob, from_wq, extra)) {
- spin_unlock(&glob->lock);
- ret = ttm_global_swapout(ctx, GFP_KERNEL);
- spin_lock(&glob->lock);
- if (unlikely(ret <= 0))
- break;
- }
-
- spin_unlock(&glob->lock);
-}
-
-static void ttm_shrink_work(struct work_struct *work)
-{
- struct ttm_operation_ctx ctx = {
- .interruptible = false,
- .no_wait_gpu = false
- };
- struct ttm_mem_global *glob =
- container_of(work, struct ttm_mem_global, work);
-
- ttm_shrink(glob, true, 0ULL, &ctx);
-}
-
-static int ttm_mem_init_kernel_zone(struct ttm_mem_global *glob,
- const struct sysinfo *si)
-{
- struct ttm_mem_zone *zone = kzalloc(sizeof(*zone), GFP_KERNEL);
- uint64_t mem;
- int ret;
-
- if (unlikely(!zone))
- return -ENOMEM;
-
- mem = si->totalram - si->totalhigh;
- mem *= si->mem_unit;
-
- zone->name = "kernel";
- zone->zone_mem = mem;
- zone->max_mem = mem >> 1;
- zone->emer_mem = (mem >> 1) + (mem >> 2);
- zone->swap_limit = zone->max_mem - (mem >> 3);
- zone->used_mem = 0;
- zone->glob = glob;
- glob->zone_kernel = zone;
- ret = kobject_init_and_add(
- &zone->kobj, &ttm_mem_zone_kobj_type, &glob->kobj, zone->name);
- if (unlikely(ret != 0)) {
- kobject_put(&zone->kobj);
- return ret;
- }
- glob->zones[glob->num_zones++] = zone;
- return 0;
-}
-
-#ifdef CONFIG_HIGHMEM
-static int ttm_mem_init_highmem_zone(struct ttm_mem_global *glob,
- const struct sysinfo *si)
-{
- struct ttm_mem_zone *zone;
- uint64_t mem;
- int ret;
-
- if (si->totalhigh == 0)
- return 0;
-
- zone = kzalloc(sizeof(*zone), GFP_KERNEL);
- if (unlikely(!zone))
- return -ENOMEM;
-
- mem = si->totalram;
- mem *= si->mem_unit;
-
- zone->name = "highmem";
- zone->zone_mem = mem;
- zone->max_mem = mem >> 1;
- zone->emer_mem = (mem >> 1) + (mem >> 2);
- zone->swap_limit = zone->max_mem - (mem >> 3);
- zone->used_mem = 0;
- zone->glob = glob;
- glob->zone_highmem = zone;
- ret = kobject_init_and_add(
- &zone->kobj, &ttm_mem_zone_kobj_type, &glob->kobj, "%s",
- zone->name);
- if (unlikely(ret != 0)) {
- kobject_put(&zone->kobj);
- return ret;
- }
- glob->zones[glob->num_zones++] = zone;
- return 0;
-}
-#else
-static int ttm_mem_init_dma32_zone(struct ttm_mem_global *glob,
- const struct sysinfo *si)
-{
- struct ttm_mem_zone *zone = kzalloc(sizeof(*zone), GFP_KERNEL);
- uint64_t mem;
- int ret;
-
- if (unlikely(!zone))
- return -ENOMEM;
-
- mem = si->totalram;
- mem *= si->mem_unit;
-
- /**
- * No special dma32 zone needed.
- */
-
- if (mem <= ((uint64_t) 1ULL << 32)) {
- kfree(zone);
- return 0;
- }
-
- /*
- * Limit max dma32 memory to 4GB for now
- * until we can figure out how big this
- * zone really is.
- */
-
- mem = ((uint64_t) 1ULL << 32);
- zone->name = "dma32";
- zone->zone_mem = mem;
- zone->max_mem = mem >> 1;
- zone->emer_mem = (mem >> 1) + (mem >> 2);
- zone->swap_limit = zone->max_mem - (mem >> 3);
- zone->used_mem = 0;
- zone->glob = glob;
- glob->zone_dma32 = zone;
- ret = kobject_init_and_add(
- &zone->kobj, &ttm_mem_zone_kobj_type, &glob->kobj, zone->name);
- if (unlikely(ret != 0)) {
- kobject_put(&zone->kobj);
- return ret;
- }
- glob->zones[glob->num_zones++] = zone;
- return 0;
-}
-#endif
-
-int ttm_mem_global_init(struct ttm_mem_global *glob, struct device *dev)
-{
- struct sysinfo si;
- int ret;
- int i;
- struct ttm_mem_zone *zone;
-
- spin_lock_init(&glob->lock);
- glob->swap_queue = create_singlethread_workqueue("ttm_swap");
- INIT_WORK(&glob->work, ttm_shrink_work);
-
- ret = kobject_init_and_add(&glob->kobj, &ttm_mem_glob_kobj_type,
- &dev->kobj, "memory_accounting");
- if (unlikely(ret != 0)) {
- kobject_put(&glob->kobj);
- return ret;
- }
-
- si_meminfo(&si);
-
- spin_lock(&glob->lock);
- /* set it as 0 by default to keep original behavior of OOM */
- glob->lower_mem_limit = 0;
- spin_unlock(&glob->lock);
-
- ret = ttm_mem_init_kernel_zone(glob, &si);
- if (unlikely(ret != 0))
- goto out_no_zone;
-#ifdef CONFIG_HIGHMEM
- ret = ttm_mem_init_highmem_zone(glob, &si);
- if (unlikely(ret != 0))
- goto out_no_zone;
-#else
- ret = ttm_mem_init_dma32_zone(glob, &si);
- if (unlikely(ret != 0))
- goto out_no_zone;
-#endif
- for (i = 0; i < glob->num_zones; ++i) {
- zone = glob->zones[i];
- pr_info("Zone %7s: Available graphics memory: %llu KiB\n",
- zone->name, (unsigned long long)zone->max_mem >> 10);
- }
- return 0;
-out_no_zone:
- ttm_mem_global_release(glob);
- return ret;
-}
-
-void ttm_mem_global_release(struct ttm_mem_global *glob)
-{
- struct ttm_mem_zone *zone;
- unsigned int i;
-
- destroy_workqueue(glob->swap_queue);
- glob->swap_queue = NULL;
- for (i = 0; i < glob->num_zones; ++i) {
- zone = glob->zones[i];
- kobject_del(&zone->kobj);
- kobject_put(&zone->kobj);
- }
- kobject_del(&glob->kobj);
- kobject_put(&glob->kobj);
- memset(glob, 0, sizeof(*glob));
-}
-
-static void ttm_check_swapping(struct ttm_mem_global *glob)
-{
- bool needs_swapping = false;
- unsigned int i;
- struct ttm_mem_zone *zone;
-
- spin_lock(&glob->lock);
- for (i = 0; i < glob->num_zones; ++i) {
- zone = glob->zones[i];
- if (zone->used_mem > zone->swap_limit) {
- needs_swapping = true;
- break;
- }
- }
-
- spin_unlock(&glob->lock);
-
- if (unlikely(needs_swapping))
- (void)queue_work(glob->swap_queue, &glob->work);
-
-}
-
-static void ttm_mem_global_free_zone(struct ttm_mem_global *glob,
- struct ttm_mem_zone *single_zone,
- uint64_t amount)
-{
- unsigned int i;
- struct ttm_mem_zone *zone;
-
- spin_lock(&glob->lock);
- for (i = 0; i < glob->num_zones; ++i) {
- zone = glob->zones[i];
- if (single_zone && zone != single_zone)
- continue;
- zone->used_mem -= amount;
- }
- spin_unlock(&glob->lock);
-}
-
-void ttm_mem_global_free(struct ttm_mem_global *glob,
- uint64_t amount)
-{
- return ttm_mem_global_free_zone(glob, glob->zone_kernel, amount);
-}
-EXPORT_SYMBOL(ttm_mem_global_free);
-
-/*
- * check if the available mem is under lower memory limit
- *
- * a. if no swap disk at all or free swap space is under swap_mem_limit
- * but available system mem is bigger than sys_mem_limit, allow TTM
- * allocation;
- *
- * b. if the available system mem is less than sys_mem_limit but free
- * swap disk is bigger than swap_mem_limit, allow TTM allocation.
- */
-bool
-ttm_check_under_lowerlimit(struct ttm_mem_global *glob,
- uint64_t num_pages,
- struct ttm_operation_ctx *ctx)
-{
- int64_t available;
-
- /* We allow over commit during suspend */
- if (ctx->force_alloc)
- return false;
-
- available = get_nr_swap_pages() + si_mem_available();
- available -= num_pages;
- if (available < glob->lower_mem_limit)
- return true;
-
- return false;
-}
-
-static int ttm_mem_global_reserve(struct ttm_mem_global *glob,
- struct ttm_mem_zone *single_zone,
- uint64_t amount, bool reserve)
-{
- uint64_t limit;
- int ret = -ENOMEM;
- unsigned int i;
- struct ttm_mem_zone *zone;
-
- spin_lock(&glob->lock);
- for (i = 0; i < glob->num_zones; ++i) {
- zone = glob->zones[i];
- if (single_zone && zone != single_zone)
- continue;
-
- limit = (capable(CAP_SYS_ADMIN)) ?
- zone->emer_mem : zone->max_mem;
-
- if (zone->used_mem > limit)
- goto out_unlock;
- }
-
- if (reserve) {
- for (i = 0; i < glob->num_zones; ++i) {
- zone = glob->zones[i];
- if (single_zone && zone != single_zone)
- continue;
- zone->used_mem += amount;
- }
- }
-
- ret = 0;
-out_unlock:
- spin_unlock(&glob->lock);
- ttm_check_swapping(glob);
-
- return ret;
-}
-
-
-static int ttm_mem_global_alloc_zone(struct ttm_mem_global *glob,
- struct ttm_mem_zone *single_zone,
- uint64_t memory,
- struct ttm_operation_ctx *ctx)
-{
- int count = TTM_MEMORY_ALLOC_RETRIES;
-
- while (unlikely(ttm_mem_global_reserve(glob,
- single_zone,
- memory, true)
- != 0)) {
- if (ctx->no_wait_gpu)
- return -ENOMEM;
- if (unlikely(count-- == 0))
- return -ENOMEM;
- ttm_shrink(glob, false, memory + (memory >> 2) + 16, ctx);
- }
-
- return 0;
-}
-
-int ttm_mem_global_alloc(struct ttm_mem_global *glob, uint64_t memory,
- struct ttm_operation_ctx *ctx)
-{
- /**
- * Normal allocations of kernel memory are registered in
- * the kernel zone.
- */
-
- return ttm_mem_global_alloc_zone(glob, glob->zone_kernel, memory, ctx);
-}
-EXPORT_SYMBOL(ttm_mem_global_alloc);
-
-int ttm_mem_global_alloc_page(struct ttm_mem_global *glob,
- struct page *page, uint64_t size,
- struct ttm_operation_ctx *ctx)
-{
- struct ttm_mem_zone *zone = NULL;
-
- /**
- * Page allocations may be registed in a single zone
- * only if highmem or !dma32.
- */
-
-#ifdef CONFIG_HIGHMEM
- if (PageHighMem(page) && glob->zone_highmem != NULL)
- zone = glob->zone_highmem;
-#else
- if (glob->zone_dma32 && page_to_pfn(page) > 0x00100000UL)
- zone = glob->zone_kernel;
-#endif
- return ttm_mem_global_alloc_zone(glob, zone, size, ctx);
-}
-
-void ttm_mem_global_free_page(struct ttm_mem_global *glob, struct page *page,
- uint64_t size)
-{
- struct ttm_mem_zone *zone = NULL;
-
-#ifdef CONFIG_HIGHMEM
- if (PageHighMem(page) && glob->zone_highmem != NULL)
- zone = glob->zone_highmem;
-#else
- if (glob->zone_dma32 && page_to_pfn(page) > 0x00100000UL)
- zone = glob->zone_kernel;
-#endif
- ttm_mem_global_free_zone(glob, zone, size);
-}
-
-size_t ttm_round_pot(size_t size)
-{
- if ((size & (size - 1)) == 0)
- return size;
- else if (size > PAGE_SIZE)
- return PAGE_ALIGN(size);
- else {
- size_t tmp_size = 4;
-
- while (tmp_size < size)
- tmp_size <<= 1;
-
- return tmp_size;
- }
- return 0;
-}
-EXPORT_SYMBOL(ttm_round_pot);
diff --git a/drivers/gpu/drm/vmwgfx/ttm_memory.h b/drivers/gpu/drm/vmwgfx/ttm_memory.h
deleted file mode 100644
index c50dba7..0000000
--- a/drivers/gpu/drm/vmwgfx/ttm_memory.h
+++ /dev/null
@@ -1,96 +0,0 @@
-/**************************************************************************
- *
- * Copyright (c) 2006-2009 VMware, Inc., Palo Alto, CA., USA
- * All Rights Reserved.
- *
- * Permission is hereby granted, free of charge, to any person obtaining a
- * copy of this software and associated documentation files (the
- * "Software"), to deal in the Software without restriction, including
- * without limitation the rights to use, copy, modify, merge, publish,
- * distribute, sub license, and/or sell copies of the Software, and to
- * permit persons to whom the Software is furnished to do so, subject to
- * the following conditions:
- *
- * The above copyright notice and this permission notice (including the
- * next paragraph) shall be included in all copies or substantial portions
- * of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. IN NO EVENT SHALL
- * THE COPYRIGHT HOLDERS, AUTHORS AND/OR ITS SUPPLIERS BE LIABLE FOR ANY CLAIM,
- * DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
- * OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
- * USE OR OTHER DEALINGS IN THE SOFTWARE.
- *
- **************************************************************************/
-
-#ifndef TTM_MEMORY_H
-#define TTM_MEMORY_H
-
-#include <linux/workqueue.h>
-#include <linux/spinlock.h>
-#include <linux/bug.h>
-#include <linux/wait.h>
-#include <linux/errno.h>
-#include <linux/kobject.h>
-#include <linux/mm.h>
-
-#include <drm/ttm/ttm_bo_api.h>
-
-/**
- * struct ttm_mem_global - Global memory accounting structure.
- *
- * @shrink: A single callback to shrink TTM memory usage. Extend this
- * to a linked list to be able to handle multiple callbacks when needed.
- * @swap_queue: A workqueue to handle shrinking in low memory situations. We
- * need a separate workqueue since it will spend a lot of time waiting
- * for the GPU, and this will otherwise block other workqueue tasks(?)
- * At this point we use only a single-threaded workqueue.
- * @work: The workqueue callback for the shrink queue.
- * @lock: Lock to protect the @shrink - and the memory accounting members,
- * that is, essentially the whole structure with some exceptions.
- * @lower_mem_limit: include lower limit of swap space and lower limit of
- * system memory.
- * @zones: Array of pointers to accounting zones.
- * @num_zones: Number of populated entries in the @zones array.
- * @zone_kernel: Pointer to the kernel zone.
- * @zone_highmem: Pointer to the highmem zone if there is one.
- * @zone_dma32: Pointer to the dma32 zone if there is one.
- *
- * Note that this structure is not per device. It should be global for all
- * graphics devices.
- */
-
-#define TTM_MEM_MAX_ZONES 2
-struct ttm_mem_zone;
-extern struct ttm_mem_global {
- struct kobject kobj;
- struct workqueue_struct *swap_queue;
- struct work_struct work;
- spinlock_t lock;
- uint64_t lower_mem_limit;
- struct ttm_mem_zone *zones[TTM_MEM_MAX_ZONES];
- unsigned int num_zones;
- struct ttm_mem_zone *zone_kernel;
-#ifdef CONFIG_HIGHMEM
- struct ttm_mem_zone *zone_highmem;
-#else
- struct ttm_mem_zone *zone_dma32;
-#endif
-} ttm_mem_glob;
-
-int ttm_mem_global_init(struct ttm_mem_global *glob, struct device *dev);
-void ttm_mem_global_release(struct ttm_mem_global *glob);
-int ttm_mem_global_alloc(struct ttm_mem_global *glob, uint64_t memory,
- struct ttm_operation_ctx *ctx);
-void ttm_mem_global_free(struct ttm_mem_global *glob, uint64_t amount);
-int ttm_mem_global_alloc_page(struct ttm_mem_global *glob,
- struct page *page, uint64_t size,
- struct ttm_operation_ctx *ctx);
-void ttm_mem_global_free_page(struct ttm_mem_global *glob,
- struct page *page, uint64_t size);
-size_t ttm_round_pot(size_t size);
-bool ttm_check_under_lowerlimit(struct ttm_mem_global *glob, uint64_t num_pages,
- struct ttm_operation_ctx *ctx);
-#endif
diff --git a/drivers/gpu/drm/vmwgfx/ttm_object.c b/drivers/gpu/drm/vmwgfx/ttm_object.c
index 899945f..26a55fe 100644
--- a/drivers/gpu/drm/vmwgfx/ttm_object.c
+++ b/drivers/gpu/drm/vmwgfx/ttm_object.c
@@ -50,6 +50,7 @@
#include <linux/atomic.h>
#include <linux/module.h>
#include "ttm_object.h"
+#include "vmwgfx_drv.h"
MODULE_IMPORT_NS(DMA_BUF);
@@ -73,7 +74,7 @@ struct ttm_object_file {
struct ttm_object_device *tdev;
spinlock_t lock;
struct list_head ref_list;
- struct drm_open_hash ref_hash[TTM_REF_NUM];
+ struct vmwgfx_open_hash ref_hash;
struct kref refcount;
};
@@ -91,12 +92,10 @@ struct ttm_object_file {
struct ttm_object_device {
spinlock_t object_lock;
- struct drm_open_hash object_hash;
+ struct vmwgfx_open_hash object_hash;
atomic_t object_count;
- struct ttm_mem_global *mem_glob;
struct dma_buf_ops ops;
void (*dmabuf_release)(struct dma_buf *dma_buf);
- size_t dma_buf_size;
struct idr idr;
};
@@ -123,10 +122,9 @@ struct ttm_object_device {
struct ttm_ref_object {
struct rcu_head rcu_head;
- struct drm_hash_item hash;
+ struct vmwgfx_hash_item hash;
struct list_head head;
struct kref kref;
- enum ttm_ref_type ref_type;
struct ttm_base_object *obj;
struct ttm_object_file *tfile;
};
@@ -162,9 +160,7 @@ int ttm_base_object_init(struct ttm_object_file *tfile,
struct ttm_base_object *base,
bool shareable,
enum ttm_object_type object_type,
- void (*refcount_release) (struct ttm_base_object **),
- void (*ref_obj_release) (struct ttm_base_object *,
- enum ttm_ref_type ref_type))
+ void (*refcount_release) (struct ttm_base_object **))
{
struct ttm_object_device *tdev = tfile->tdev;
int ret;
@@ -172,7 +168,6 @@ int ttm_base_object_init(struct ttm_object_file *tfile,
base->shareable = shareable;
base->tfile = ttm_object_file_ref(tfile);
base->refcount_release = refcount_release;
- base->ref_obj_release = ref_obj_release;
base->object_type = object_type;
kref_init(&base->refcount);
idr_preload(GFP_KERNEL);
@@ -184,7 +179,7 @@ int ttm_base_object_init(struct ttm_object_file *tfile,
return ret;
base->handle = ret;
- ret = ttm_ref_object_add(tfile, base, TTM_REF_USAGE, NULL, false);
+ ret = ttm_ref_object_add(tfile, base, NULL, false);
if (unlikely(ret != 0))
goto out_err1;
@@ -247,12 +242,12 @@ void ttm_base_object_unref(struct ttm_base_object **p_base)
struct ttm_base_object *
ttm_base_object_noref_lookup(struct ttm_object_file *tfile, uint32_t key)
{
- struct drm_hash_item *hash;
- struct drm_open_hash *ht = &tfile->ref_hash[TTM_REF_USAGE];
+ struct vmwgfx_hash_item *hash;
+ struct vmwgfx_open_hash *ht = &tfile->ref_hash;
int ret;
rcu_read_lock();
- ret = drm_ht_find_item_rcu(ht, key, &hash);
+ ret = vmwgfx_ht_find_item_rcu(ht, key, &hash);
if (ret) {
rcu_read_unlock();
return NULL;
@@ -267,12 +262,12 @@ struct ttm_base_object *ttm_base_object_lookup(struct ttm_object_file *tfile,
uint32_t key)
{
struct ttm_base_object *base = NULL;
- struct drm_hash_item *hash;
- struct drm_open_hash *ht = &tfile->ref_hash[TTM_REF_USAGE];
+ struct vmwgfx_hash_item *hash;
+ struct vmwgfx_open_hash *ht = &tfile->ref_hash;
int ret;
rcu_read_lock();
- ret = drm_ht_find_item_rcu(ht, key, &hash);
+ ret = vmwgfx_ht_find_item_rcu(ht, key, &hash);
if (likely(ret == 0)) {
base = drm_hash_entry(hash, struct ttm_ref_object, hash)->obj;
@@ -299,64 +294,14 @@ ttm_base_object_lookup_for_ref(struct ttm_object_device *tdev, uint32_t key)
return base;
}
-/**
- * ttm_ref_object_exists - Check whether a caller has a valid ref object
- * (has opened) a base object.
- *
- * @tfile: Pointer to a struct ttm_object_file identifying the caller.
- * @base: Pointer to a struct base object.
- *
- * Checks wether the caller identified by @tfile has put a valid USAGE
- * reference object on the base object identified by @base.
- */
-bool ttm_ref_object_exists(struct ttm_object_file *tfile,
- struct ttm_base_object *base)
-{
- struct drm_open_hash *ht = &tfile->ref_hash[TTM_REF_USAGE];
- struct drm_hash_item *hash;
- struct ttm_ref_object *ref;
-
- rcu_read_lock();
- if (unlikely(drm_ht_find_item_rcu(ht, base->handle, &hash) != 0))
- goto out_false;
-
- /*
- * Verify that the ref object is really pointing to our base object.
- * Our base object could actually be dead, and the ref object pointing
- * to another base object with the same handle.
- */
- ref = drm_hash_entry(hash, struct ttm_ref_object, hash);
- if (unlikely(base != ref->obj))
- goto out_false;
-
- /*
- * Verify that the ref->obj pointer was actually valid!
- */
- rmb();
- if (unlikely(kref_read(&ref->kref) == 0))
- goto out_false;
-
- rcu_read_unlock();
- return true;
-
- out_false:
- rcu_read_unlock();
- return false;
-}
-
int ttm_ref_object_add(struct ttm_object_file *tfile,
struct ttm_base_object *base,
- enum ttm_ref_type ref_type, bool *existed,
+ bool *existed,
bool require_existed)
{
- struct drm_open_hash *ht = &tfile->ref_hash[ref_type];
+ struct vmwgfx_open_hash *ht = &tfile->ref_hash;
struct ttm_ref_object *ref;
- struct drm_hash_item *hash;
- struct ttm_mem_global *mem_glob = tfile->tdev->mem_glob;
- struct ttm_operation_ctx ctx = {
- .interruptible = false,
- .no_wait_gpu = false
- };
+ struct vmwgfx_hash_item *hash;
int ret = -EINVAL;
if (base->tfile != tfile && !base->shareable)
@@ -367,7 +312,7 @@ int ttm_ref_object_add(struct ttm_object_file *tfile,
while (ret == -EINVAL) {
rcu_read_lock();
- ret = drm_ht_find_item_rcu(ht, base->handle, &hash);
+ ret = vmwgfx_ht_find_item_rcu(ht, base->handle, &hash);
if (ret == 0) {
ref = drm_hash_entry(hash, struct ttm_ref_object, hash);
@@ -381,24 +326,18 @@ int ttm_ref_object_add(struct ttm_object_file *tfile,
if (require_existed)
return -EPERM;
- ret = ttm_mem_global_alloc(mem_glob, sizeof(*ref),
- &ctx);
- if (unlikely(ret != 0))
- return ret;
ref = kmalloc(sizeof(*ref), GFP_KERNEL);
if (unlikely(ref == NULL)) {
- ttm_mem_global_free(mem_glob, sizeof(*ref));
return -ENOMEM;
}
ref->hash.key = base->handle;
ref->obj = base;
ref->tfile = tfile;
- ref->ref_type = ref_type;
kref_init(&ref->kref);
spin_lock(&tfile->lock);
- ret = drm_ht_insert_item_rcu(ht, &ref->hash);
+ ret = vmwgfx_ht_insert_item_rcu(ht, &ref->hash);
if (likely(ret == 0)) {
list_add_tail(&ref->head, &tfile->ref_list);
@@ -412,7 +351,6 @@ int ttm_ref_object_add(struct ttm_object_file *tfile,
spin_unlock(&tfile->lock);
BUG_ON(ret != -EINVAL);
- ttm_mem_global_free(mem_glob, sizeof(*ref));
kfree(ref);
}
@@ -424,35 +362,29 @@ ttm_ref_object_release(struct kref *kref)
{
struct ttm_ref_object *ref =
container_of(kref, struct ttm_ref_object, kref);
- struct ttm_base_object *base = ref->obj;
struct ttm_object_file *tfile = ref->tfile;
- struct drm_open_hash *ht;
- struct ttm_mem_global *mem_glob = tfile->tdev->mem_glob;
+ struct vmwgfx_open_hash *ht;
- ht = &tfile->ref_hash[ref->ref_type];
- (void)drm_ht_remove_item_rcu(ht, &ref->hash);
+ ht = &tfile->ref_hash;
+ (void)vmwgfx_ht_remove_item_rcu(ht, &ref->hash);
list_del(&ref->head);
spin_unlock(&tfile->lock);
- if (ref->ref_type != TTM_REF_USAGE && base->ref_obj_release)
- base->ref_obj_release(base, ref->ref_type);
-
ttm_base_object_unref(&ref->obj);
- ttm_mem_global_free(mem_glob, sizeof(*ref));
kfree_rcu(ref, rcu_head);
spin_lock(&tfile->lock);
}
int ttm_ref_object_base_unref(struct ttm_object_file *tfile,
- unsigned long key, enum ttm_ref_type ref_type)
+ unsigned long key)
{
- struct drm_open_hash *ht = &tfile->ref_hash[ref_type];
+ struct vmwgfx_open_hash *ht = &tfile->ref_hash;
struct ttm_ref_object *ref;
- struct drm_hash_item *hash;
+ struct vmwgfx_hash_item *hash;
int ret;
spin_lock(&tfile->lock);
- ret = drm_ht_find_item(ht, key, &hash);
+ ret = vmwgfx_ht_find_item(ht, key, &hash);
if (unlikely(ret != 0)) {
spin_unlock(&tfile->lock);
return -EINVAL;
@@ -467,7 +399,6 @@ void ttm_object_file_release(struct ttm_object_file **p_tfile)
{
struct ttm_ref_object *ref;
struct list_head *list;
- unsigned int i;
struct ttm_object_file *tfile = *p_tfile;
*p_tfile = NULL;
@@ -485,8 +416,7 @@ void ttm_object_file_release(struct ttm_object_file **p_tfile)
}
spin_unlock(&tfile->lock);
- for (i = 0; i < TTM_REF_NUM; ++i)
- drm_ht_remove(&tfile->ref_hash[i]);
+ vmwgfx_ht_remove(&tfile->ref_hash);
ttm_object_file_unref(&tfile);
}
@@ -495,8 +425,6 @@ struct ttm_object_file *ttm_object_file_init(struct ttm_object_device *tdev,
unsigned int hash_order)
{
struct ttm_object_file *tfile = kmalloc(sizeof(*tfile), GFP_KERNEL);
- unsigned int i;
- unsigned int j = 0;
int ret;
if (unlikely(tfile == NULL))
@@ -507,18 +435,13 @@ struct ttm_object_file *ttm_object_file_init(struct ttm_object_device *tdev,
kref_init(&tfile->refcount);
INIT_LIST_HEAD(&tfile->ref_list);
- for (i = 0; i < TTM_REF_NUM; ++i) {
- ret = drm_ht_create(&tfile->ref_hash[i], hash_order);
- if (ret) {
- j = i;
- goto out_err;
- }
- }
+ ret = vmwgfx_ht_create(&tfile->ref_hash, hash_order);
+ if (ret)
+ goto out_err;
return tfile;
out_err:
- for (i = 0; i < j; ++i)
- drm_ht_remove(&tfile->ref_hash[i]);
+ vmwgfx_ht_remove(&tfile->ref_hash);
kfree(tfile);
@@ -526,8 +449,7 @@ struct ttm_object_file *ttm_object_file_init(struct ttm_object_device *tdev,
}
struct ttm_object_device *
-ttm_object_device_init(struct ttm_mem_global *mem_glob,
- unsigned int hash_order,
+ttm_object_device_init(unsigned int hash_order,
const struct dma_buf_ops *ops)
{
struct ttm_object_device *tdev = kmalloc(sizeof(*tdev), GFP_KERNEL);
@@ -536,19 +458,24 @@ ttm_object_device_init(struct ttm_mem_global *mem_glob,
if (unlikely(tdev == NULL))
return NULL;
- tdev->mem_glob = mem_glob;
spin_lock_init(&tdev->object_lock);
atomic_set(&tdev->object_count, 0);
- ret = drm_ht_create(&tdev->object_hash, hash_order);
+ ret = vmwgfx_ht_create(&tdev->object_hash, hash_order);
if (ret != 0)
goto out_no_object_hash;
- idr_init_base(&tdev->idr, 1);
+ /*
+ * Our base is at VMWGFX_NUM_MOB + 1 because we want to create
+ * a seperate namespace for GEM handles (which are
+ * 1..VMWGFX_NUM_MOB) and the surface handles. Some ioctl's
+ * can take either handle as an argument so we want to
+ * easily be able to tell whether the handle refers to a
+ * GEM buffer or a surface.
+ */
+ idr_init_base(&tdev->idr, VMWGFX_NUM_MOB + 1);
tdev->ops = *ops;
tdev->dmabuf_release = tdev->ops.release;
tdev->ops.release = ttm_prime_dmabuf_release;
- tdev->dma_buf_size = ttm_round_pot(sizeof(struct dma_buf)) +
- ttm_round_pot(sizeof(struct file));
return tdev;
out_no_object_hash:
@@ -564,7 +491,7 @@ void ttm_object_device_release(struct ttm_object_device **p_tdev)
WARN_ON_ONCE(!idr_is_empty(&tdev->idr));
idr_destroy(&tdev->idr);
- drm_ht_remove(&tdev->object_hash);
+ vmwgfx_ht_remove(&tdev->object_hash);
kfree(tdev);
}
@@ -633,7 +560,6 @@ static void ttm_prime_dmabuf_release(struct dma_buf *dma_buf)
if (prime->dma_buf == dma_buf)
prime->dma_buf = NULL;
mutex_unlock(&prime->mutex);
- ttm_mem_global_free(tdev->mem_glob, tdev->dma_buf_size);
ttm_base_object_unref(&base);
}
@@ -667,7 +593,7 @@ int ttm_prime_fd_to_handle(struct ttm_object_file *tfile,
prime = (struct ttm_prime_object *) dma_buf->priv;
base = &prime->base;
*handle = base->handle;
- ret = ttm_ref_object_add(tfile, base, TTM_REF_USAGE, NULL, false);
+ ret = ttm_ref_object_add(tfile, base, NULL, false);
dma_buf_put(dma_buf);
@@ -715,30 +641,18 @@ int ttm_prime_handle_to_fd(struct ttm_object_file *tfile,
dma_buf = prime->dma_buf;
if (!dma_buf || !get_dma_buf_unless_doomed(dma_buf)) {
DEFINE_DMA_BUF_EXPORT_INFO(exp_info);
- struct ttm_operation_ctx ctx = {
- .interruptible = true,
- .no_wait_gpu = false
- };
exp_info.ops = &tdev->ops;
exp_info.size = prime->size;
exp_info.flags = flags;
exp_info.priv = prime;
/*
- * Need to create a new dma_buf, with memory accounting.
+ * Need to create a new dma_buf
*/
- ret = ttm_mem_global_alloc(tdev->mem_glob, tdev->dma_buf_size,
- &ctx);
- if (unlikely(ret != 0)) {
- mutex_unlock(&prime->mutex);
- goto out_unref;
- }
dma_buf = dma_buf_export(&exp_info);
if (IS_ERR(dma_buf)) {
ret = PTR_ERR(dma_buf);
- ttm_mem_global_free(tdev->mem_glob,
- tdev->dma_buf_size);
mutex_unlock(&prime->mutex);
goto out_unref;
}
@@ -773,7 +687,6 @@ int ttm_prime_handle_to_fd(struct ttm_object_file *tfile,
* @shareable: See ttm_base_object_init
* @type: See ttm_base_object_init
* @refcount_release: See ttm_base_object_init
- * @ref_obj_release: See ttm_base_object_init
*
* Initializes an object which is compatible with the drm_prime model
* for data sharing between processes and devices.
@@ -781,9 +694,7 @@ int ttm_prime_handle_to_fd(struct ttm_object_file *tfile,
int ttm_prime_object_init(struct ttm_object_file *tfile, size_t size,
struct ttm_prime_object *prime, bool shareable,
enum ttm_object_type type,
- void (*refcount_release) (struct ttm_base_object **),
- void (*ref_obj_release) (struct ttm_base_object *,
- enum ttm_ref_type ref_type))
+ void (*refcount_release) (struct ttm_base_object **))
{
mutex_init(&prime->mutex);
prime->size = PAGE_ALIGN(size);
@@ -792,6 +703,5 @@ int ttm_prime_object_init(struct ttm_object_file *tfile, size_t size,
prime->refcount_release = refcount_release;
return ttm_base_object_init(tfile, &prime->base, shareable,
ttm_prime_type,
- ttm_prime_refcount_release,
- ref_obj_release);
+ ttm_prime_refcount_release);
}
diff --git a/drivers/gpu/drm/vmwgfx/ttm_object.h b/drivers/gpu/drm/vmwgfx/ttm_object.h
index 49b064f..4c87000 100644
--- a/drivers/gpu/drm/vmwgfx/ttm_object.h
+++ b/drivers/gpu/drm/vmwgfx/ttm_object.h
@@ -42,31 +42,7 @@
#include <linux/list.h>
#include <linux/rcupdate.h>
-#include <drm/drm_hashtab.h>
-
-#include "ttm_memory.h"
-
-/**
- * enum ttm_ref_type
- *
- * Describes what type of reference a ref object holds.
- *
- * TTM_REF_USAGE is a simple refcount on a base object.
- *
- * TTM_REF_SYNCCPU_READ is a SYNCCPU_READ reference on a
- * buffer object.
- *
- * TTM_REF_SYNCCPU_WRITE is a SYNCCPU_WRITE reference on a
- * buffer object.
- *
- */
-
-enum ttm_ref_type {
- TTM_REF_USAGE,
- TTM_REF_SYNCCPU_READ,
- TTM_REF_SYNCCPU_WRITE,
- TTM_REF_NUM
-};
+#include "vmwgfx_hashtab.h"
/**
* enum ttm_object_type
@@ -78,7 +54,6 @@ enum ttm_ref_type {
enum ttm_object_type {
ttm_fence_type,
- ttm_buffer_type,
ttm_lock_type,
ttm_prime_type,
ttm_driver_type0 = 256,
@@ -129,8 +104,6 @@ struct ttm_base_object {
struct ttm_object_file *tfile;
struct kref refcount;
void (*refcount_release) (struct ttm_base_object **base);
- void (*ref_obj_release) (struct ttm_base_object *base,
- enum ttm_ref_type ref_type);
u32 handle;
enum ttm_object_type object_type;
u32 shareable;
@@ -179,11 +152,7 @@ extern int ttm_base_object_init(struct ttm_object_file *tfile,
bool shareable,
enum ttm_object_type type,
void (*refcount_release) (struct ttm_base_object
- **),
- void (*ref_obj_release) (struct ttm_base_object
- *,
- enum ttm_ref_type
- ref_type));
+ **));
/**
* ttm_base_object_lookup
@@ -247,12 +216,9 @@ extern void ttm_base_object_unref(struct ttm_base_object **p_base);
*/
extern int ttm_ref_object_add(struct ttm_object_file *tfile,
struct ttm_base_object *base,
- enum ttm_ref_type ref_type, bool *existed,
+ bool *existed,
bool require_existed);
-extern bool ttm_ref_object_exists(struct ttm_object_file *tfile,
- struct ttm_base_object *base);
-
/**
* ttm_ref_object_base_unref
*
@@ -265,8 +231,7 @@ extern bool ttm_ref_object_exists(struct ttm_object_file *tfile,
* will be unreferenced.
*/
extern int ttm_ref_object_base_unref(struct ttm_object_file *tfile,
- unsigned long key,
- enum ttm_ref_type ref_type);
+ unsigned long key);
/**
* ttm_object_file_init - initialize a struct ttm_object file
@@ -297,7 +262,6 @@ extern void ttm_object_file_release(struct ttm_object_file **p_tfile);
/**
* ttm_object device init - initialize a struct ttm_object_device
*
- * @mem_glob: struct ttm_mem_global for memory accounting.
* @hash_order: Order of hash table used to hash the base objects.
* @ops: DMA buf ops for prime objects of this device.
*
@@ -306,8 +270,7 @@ extern void ttm_object_file_release(struct ttm_object_file **p_tfile);
*/
extern struct ttm_object_device *
-ttm_object_device_init(struct ttm_mem_global *mem_glob,
- unsigned int hash_order,
+ttm_object_device_init(unsigned int hash_order,
const struct dma_buf_ops *ops);
/**
@@ -332,10 +295,7 @@ extern int ttm_prime_object_init(struct ttm_object_file *tfile,
bool shareable,
enum ttm_object_type type,
void (*refcount_release)
- (struct ttm_base_object **),
- void (*ref_obj_release)
- (struct ttm_base_object *,
- enum ttm_ref_type ref_type));
+ (struct ttm_base_object **));
static inline enum ttm_object_type
ttm_base_object_type(struct ttm_base_object *base)
@@ -353,13 +313,6 @@ extern int ttm_prime_handle_to_fd(struct ttm_object_file *tfile,
#define ttm_prime_object_kfree(__obj, __prime) \
kfree_rcu(__obj, __prime.base.rhead)
-/*
- * Extra memory required by the base object's idr storage, which is allocated
- * separately from the base object itself. We estimate an on-average 128 bytes
- * per idr.
- */
-#define TTM_OBJ_EXTRA_SIZE 128
-
struct ttm_base_object *
ttm_base_object_noref_lookup(struct ttm_object_file *tfile, uint32_t key);
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_binding.c b/drivers/gpu/drm/vmwgfx/vmwgfx_binding.c
index 6f27d69..ae2de91 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_binding.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_binding.c
@@ -354,6 +354,27 @@ void vmw_binding_add(struct vmw_ctx_binding_state *cbs,
}
/**
+ * vmw_binding_cb_offset_update: Update the offset of a cb binding
+ *
+ * @cbs: Pointer to the context binding state tracker.
+ * @shader_slot: The shader slot of the binding.
+ * @slot: The slot of the binding.
+ * @offsetInBytes: The new offset of the binding.
+ *
+ * Updates the offset of an existing cb binding in the context binding
+ * state structure @cbs.
+ */
+void vmw_binding_cb_offset_update(struct vmw_ctx_binding_state *cbs,
+ u32 shader_slot, u32 slot, u32 offsetInBytes)
+{
+ struct vmw_ctx_bindinfo *loc =
+ vmw_binding_loc(cbs, vmw_ctx_binding_cb, shader_slot, slot);
+ struct vmw_ctx_bindinfo_cb *loc_cb =
+ (struct vmw_ctx_bindinfo_cb *)((u8 *) loc);
+ loc_cb->offset = offsetInBytes;
+}
+
+/**
* vmw_binding_add_uav_index - Add UAV index for tracking.
* @cbs: Pointer to the context binding state tracker.
* @slot: UAV type to which bind this index.
@@ -1070,7 +1091,7 @@ static int vmw_emit_set_uav(struct vmw_ctx_binding_state *cbs)
size_t cmd_size, view_id_size;
const struct vmw_resource *ctx = vmw_cbs_context(cbs);
- vmw_collect_view_ids(cbs, loc, SVGA3D_MAX_UAVIEWS);
+ vmw_collect_view_ids(cbs, loc, vmw_max_num_uavs(cbs->dev_priv));
view_id_size = cbs->bind_cmd_count*sizeof(uint32);
cmd_size = sizeof(*cmd) + view_id_size;
cmd = VMW_CMD_CTX_RESERVE(ctx->dev_priv, cmd_size, ctx->id);
@@ -1100,7 +1121,7 @@ static int vmw_emit_set_cs_uav(struct vmw_ctx_binding_state *cbs)
size_t cmd_size, view_id_size;
const struct vmw_resource *ctx = vmw_cbs_context(cbs);
- vmw_collect_view_ids(cbs, loc, SVGA3D_MAX_UAVIEWS);
+ vmw_collect_view_ids(cbs, loc, vmw_max_num_uavs(cbs->dev_priv));
view_id_size = cbs->bind_cmd_count*sizeof(uint32);
cmd_size = sizeof(*cmd) + view_id_size;
cmd = VMW_CMD_CTX_RESERVE(ctx->dev_priv, cmd_size, ctx->id);
@@ -1327,8 +1348,7 @@ static int vmw_binding_scrub_so(struct vmw_ctx_bindinfo *bi, bool rebind)
}
/**
- * vmw_binding_state_alloc - Allocate a struct vmw_ctx_binding_state with
- * memory accounting.
+ * vmw_binding_state_alloc - Allocate a struct vmw_ctx_binding_state.
*
* @dev_priv: Pointer to a device private structure.
*
@@ -1338,20 +1358,9 @@ struct vmw_ctx_binding_state *
vmw_binding_state_alloc(struct vmw_private *dev_priv)
{
struct vmw_ctx_binding_state *cbs;
- struct ttm_operation_ctx ctx = {
- .interruptible = false,
- .no_wait_gpu = false
- };
- int ret;
-
- ret = ttm_mem_global_alloc(vmw_mem_glob(dev_priv), sizeof(*cbs),
- &ctx);
- if (ret)
- return ERR_PTR(ret);
cbs = vzalloc(sizeof(*cbs));
if (!cbs) {
- ttm_mem_global_free(vmw_mem_glob(dev_priv), sizeof(*cbs));
return ERR_PTR(-ENOMEM);
}
@@ -1362,17 +1371,13 @@ vmw_binding_state_alloc(struct vmw_private *dev_priv)
}
/**
- * vmw_binding_state_free - Free a struct vmw_ctx_binding_state and its
- * memory accounting info.
+ * vmw_binding_state_free - Free a struct vmw_ctx_binding_state.
*
* @cbs: Pointer to the struct vmw_ctx_binding_state to be freed.
*/
void vmw_binding_state_free(struct vmw_ctx_binding_state *cbs)
{
- struct vmw_private *dev_priv = cbs->dev_priv;
-
vfree(cbs);
- ttm_mem_global_free(vmw_mem_glob(dev_priv), sizeof(*cbs));
}
/**
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_binding.h b/drivers/gpu/drm/vmwgfx/vmwgfx_binding.h
index dcb71fd..85b90f7 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_binding.h
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_binding.h
@@ -200,7 +200,7 @@ struct vmw_dx_shader_bindings {
* @splice_index: The device splice index set by user-space.
*/
struct vmw_ctx_bindinfo_uav {
- struct vmw_ctx_bindinfo_view views[SVGA3D_MAX_UAVIEWS];
+ struct vmw_ctx_bindinfo_view views[SVGA3D_DX11_1_MAX_UAVIEWS];
uint32 index;
};
@@ -217,6 +217,8 @@ struct vmw_ctx_bindinfo_so {
extern void vmw_binding_add(struct vmw_ctx_binding_state *cbs,
const struct vmw_ctx_bindinfo *ci,
u32 shader_slot, u32 slot);
+extern void vmw_binding_cb_offset_update(struct vmw_ctx_binding_state *cbs,
+ u32 shader_slot, u32 slot, u32 offsetInBytes);
extern void vmw_binding_add_uav_index(struct vmw_ctx_binding_state *cbs,
uint32 slot, uint32 splice_index);
extern void
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_bo.c b/drivers/gpu/drm/vmwgfx/vmwgfx_bo.c
index fd007f1..31aecc4 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_bo.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_bo.c
@@ -33,18 +33,6 @@
/**
- * struct vmw_user_buffer_object - User-space-visible buffer object
- *
- * @prime: The prime object providing user visibility.
- * @vbo: The struct vmw_buffer_object
- */
-struct vmw_user_buffer_object {
- struct ttm_prime_object prime;
- struct vmw_buffer_object vbo;
-};
-
-
-/**
* vmw_buffer_object - Convert a struct ttm_buffer_object to a struct
* vmw_buffer_object.
*
@@ -60,23 +48,6 @@ vmw_buffer_object(struct ttm_buffer_object *bo)
/**
- * vmw_user_buffer_object - Convert a struct ttm_buffer_object to a struct
- * vmw_user_buffer_object.
- *
- * @bo: Pointer to the TTM buffer object.
- * Return: Pointer to the struct vmw_buffer_object embedding the TTM buffer
- * object.
- */
-static struct vmw_user_buffer_object *
-vmw_user_buffer_object(struct ttm_buffer_object *bo)
-{
- struct vmw_buffer_object *vmw_bo = vmw_buffer_object(bo);
-
- return container_of(vmw_bo, struct vmw_user_buffer_object, vbo);
-}
-
-
-/**
* vmw_bo_pin_in_placement - Validate a buffer to placement.
*
* @dev_priv: Driver private.
@@ -392,39 +363,6 @@ void vmw_bo_unmap(struct vmw_buffer_object *vbo)
/**
- * vmw_bo_acc_size - Calculate the pinned memory usage of buffers
- *
- * @dev_priv: Pointer to a struct vmw_private identifying the device.
- * @size: The requested buffer size.
- * @user: Whether this is an ordinary dma buffer or a user dma buffer.
- */
-static size_t vmw_bo_acc_size(struct vmw_private *dev_priv, size_t size,
- bool user)
-{
- static size_t struct_size, user_struct_size;
- size_t num_pages = PFN_UP(size);
- size_t page_array_size = ttm_round_pot(num_pages * sizeof(void *));
-
- if (unlikely(struct_size == 0)) {
- size_t backend_size = ttm_round_pot(vmw_tt_size);
-
- struct_size = backend_size +
- ttm_round_pot(sizeof(struct vmw_buffer_object));
- user_struct_size = backend_size +
- ttm_round_pot(sizeof(struct vmw_user_buffer_object)) +
- TTM_OBJ_EXTRA_SIZE;
- }
-
- if (dev_priv->map_mode == vmw_dma_alloc_coherent)
- page_array_size +=
- ttm_round_pot(num_pages * sizeof(dma_addr_t));
-
- return ((user) ? user_struct_size : struct_size) +
- page_array_size;
-}
-
-
-/**
* vmw_bo_bo_free - vmw buffer object destructor
*
* @bo: Pointer to the embedded struct ttm_buffer_object
@@ -436,27 +374,10 @@ void vmw_bo_bo_free(struct ttm_buffer_object *bo)
WARN_ON(vmw_bo->dirty);
WARN_ON(!RB_EMPTY_ROOT(&vmw_bo->res_tree));
vmw_bo_unmap(vmw_bo);
- dma_resv_fini(&bo->base._resv);
+ drm_gem_object_release(&bo->base);
kfree(vmw_bo);
}
-
-/**
- * vmw_user_bo_destroy - vmw buffer object destructor
- *
- * @bo: Pointer to the embedded struct ttm_buffer_object
- */
-static void vmw_user_bo_destroy(struct ttm_buffer_object *bo)
-{
- struct vmw_user_buffer_object *vmw_user_bo = vmw_user_buffer_object(bo);
- struct vmw_buffer_object *vbo = &vmw_user_bo->vbo;
-
- WARN_ON(vbo->dirty);
- WARN_ON(!RB_EMPTY_ROOT(&vbo->res_tree));
- vmw_bo_unmap(vbo);
- ttm_prime_object_kfree(vmw_user_bo, prime);
-}
-
/**
* vmw_bo_create_kernel - Create a pinned BO for internal kernel use.
*
@@ -471,33 +392,27 @@ int vmw_bo_create_kernel(struct vmw_private *dev_priv, unsigned long size,
struct ttm_placement *placement,
struct ttm_buffer_object **p_bo)
{
- struct ttm_operation_ctx ctx = { false, false };
+ struct ttm_operation_ctx ctx = {
+ .interruptible = false,
+ .no_wait_gpu = false
+ };
struct ttm_buffer_object *bo;
- size_t acc_size;
+ struct drm_device *vdev = &dev_priv->drm;
int ret;
bo = kzalloc(sizeof(*bo), GFP_KERNEL);
if (unlikely(!bo))
return -ENOMEM;
- acc_size = ttm_round_pot(sizeof(*bo));
- acc_size += ttm_round_pot(PFN_UP(size) * sizeof(void *));
- acc_size += ttm_round_pot(sizeof(struct ttm_tt));
+ size = ALIGN(size, PAGE_SIZE);
- ret = ttm_mem_global_alloc(&ttm_mem_glob, acc_size, &ctx);
- if (unlikely(ret))
- goto error_free;
-
-
- bo->base.size = size;
- dma_resv_init(&bo->base._resv);
- drm_vma_node_reset(&bo->base.vma_node);
+ drm_gem_private_object_init(vdev, &bo->base, size);
ret = ttm_bo_init_reserved(&dev_priv->bdev, bo, size,
- ttm_bo_type_device, placement, 0,
+ ttm_bo_type_kernel, placement, 0,
&ctx, NULL, NULL, NULL);
if (unlikely(ret))
- goto error_account;
+ goto error_free;
ttm_bo_pin(bo);
ttm_bo_unreserve(bo);
@@ -505,14 +420,38 @@ int vmw_bo_create_kernel(struct vmw_private *dev_priv, unsigned long size,
return 0;
-error_account:
- ttm_mem_global_free(&ttm_mem_glob, acc_size);
-
error_free:
kfree(bo);
return ret;
}
+int vmw_bo_create(struct vmw_private *vmw,
+ size_t size, struct ttm_placement *placement,
+ bool interruptible, bool pin,
+ void (*bo_free)(struct ttm_buffer_object *bo),
+ struct vmw_buffer_object **p_bo)
+{
+ int ret;
+
+ *p_bo = kmalloc(sizeof(**p_bo), GFP_KERNEL);
+ if (unlikely(!*p_bo)) {
+ DRM_ERROR("Failed to allocate a buffer.\n");
+ return -ENOMEM;
+ }
+
+ ret = vmw_bo_init(vmw, *p_bo, size,
+ placement, interruptible, pin,
+ bo_free);
+ if (unlikely(ret != 0))
+ goto out_error;
+
+ return ret;
+out_error:
+ kfree(*p_bo);
+ *p_bo = NULL;
+ return ret;
+}
+
/**
* vmw_bo_init - Initialize a vmw buffer object
*
@@ -533,192 +472,44 @@ int vmw_bo_init(struct vmw_private *dev_priv,
bool interruptible, bool pin,
void (*bo_free)(struct ttm_buffer_object *bo))
{
- struct ttm_operation_ctx ctx = { interruptible, false };
+ struct ttm_operation_ctx ctx = {
+ .interruptible = interruptible,
+ .no_wait_gpu = false
+ };
struct ttm_device *bdev = &dev_priv->bdev;
- size_t acc_size;
+ struct drm_device *vdev = &dev_priv->drm;
int ret;
- bool user = (bo_free == &vmw_user_bo_destroy);
- WARN_ON_ONCE(!bo_free && (!user && (bo_free != vmw_bo_bo_free)));
-
- acc_size = vmw_bo_acc_size(dev_priv, size, user);
+ WARN_ON_ONCE(!bo_free);
memset(vmw_bo, 0, sizeof(*vmw_bo));
BUILD_BUG_ON(TTM_MAX_BO_PRIORITY <= 3);
vmw_bo->base.priority = 3;
vmw_bo->res_tree = RB_ROOT;
- ret = ttm_mem_global_alloc(&ttm_mem_glob, acc_size, &ctx);
- if (unlikely(ret))
- return ret;
-
- vmw_bo->base.base.size = size;
- dma_resv_init(&vmw_bo->base.base._resv);
- drm_vma_node_reset(&vmw_bo->base.base.vma_node);
+ size = ALIGN(size, PAGE_SIZE);
+ drm_gem_private_object_init(vdev, &vmw_bo->base.base, size);
ret = ttm_bo_init_reserved(bdev, &vmw_bo->base, size,
- ttm_bo_type_device, placement,
+ ttm_bo_type_device,
+ placement,
0, &ctx, NULL, NULL, bo_free);
if (unlikely(ret)) {
- ttm_mem_global_free(&ttm_mem_glob, acc_size);
return ret;
}
if (pin)
ttm_bo_pin(&vmw_bo->base);
ttm_bo_unreserve(&vmw_bo->base);
+
return 0;
}
-
/**
- * vmw_user_bo_release - TTM reference base object release callback for
- * vmw user buffer objects
- *
- * @p_base: The TTM base object pointer about to be unreferenced.
- *
- * Clears the TTM base object pointer and drops the reference the
- * base object has on the underlying struct vmw_buffer_object.
- */
-static void vmw_user_bo_release(struct ttm_base_object **p_base)
-{
- struct vmw_user_buffer_object *vmw_user_bo;
- struct ttm_base_object *base = *p_base;
-
- *p_base = NULL;
-
- if (unlikely(base == NULL))
- return;
-
- vmw_user_bo = container_of(base, struct vmw_user_buffer_object,
- prime.base);
- ttm_bo_put(&vmw_user_bo->vbo.base);
-}
-
-
-/**
- * vmw_user_bo_ref_obj_release - TTM synccpu reference object release callback
- * for vmw user buffer objects
- *
- * @base: Pointer to the TTM base object
- * @ref_type: Reference type of the reference reaching zero.
- *
- * Called when user-space drops its last synccpu reference on the buffer
- * object, Either explicitly or as part of a cleanup file close.
- */
-static void vmw_user_bo_ref_obj_release(struct ttm_base_object *base,
- enum ttm_ref_type ref_type)
-{
- struct vmw_user_buffer_object *user_bo;
-
- user_bo = container_of(base, struct vmw_user_buffer_object, prime.base);
-
- switch (ref_type) {
- case TTM_REF_SYNCCPU_WRITE:
- atomic_dec(&user_bo->vbo.cpu_writers);
- break;
- default:
- WARN_ONCE(true, "Undefined buffer object reference release.\n");
- }
-}
-
-
-/**
- * vmw_user_bo_alloc - Allocate a user buffer object
- *
- * @dev_priv: Pointer to a struct device private.
- * @tfile: Pointer to a struct ttm_object_file on which to register the user
- * object.
- * @size: Size of the buffer object.
- * @shareable: Boolean whether the buffer is shareable with other open files.
- * @handle: Pointer to where the handle value should be assigned.
- * @p_vbo: Pointer to where the refcounted struct vmw_buffer_object pointer
- * should be assigned.
- * @p_base: The TTM base object pointer about to be allocated.
- * Return: Zero on success, negative error code on error.
- */
-int vmw_user_bo_alloc(struct vmw_private *dev_priv,
- struct ttm_object_file *tfile,
- uint32_t size,
- bool shareable,
- uint32_t *handle,
- struct vmw_buffer_object **p_vbo,
- struct ttm_base_object **p_base)
-{
- struct vmw_user_buffer_object *user_bo;
- int ret;
-
- user_bo = kzalloc(sizeof(*user_bo), GFP_KERNEL);
- if (unlikely(!user_bo)) {
- DRM_ERROR("Failed to allocate a buffer.\n");
- return -ENOMEM;
- }
-
- ret = vmw_bo_init(dev_priv, &user_bo->vbo, size,
- (dev_priv->has_mob) ?
- &vmw_sys_placement :
- &vmw_vram_sys_placement, true, false,
- &vmw_user_bo_destroy);
- if (unlikely(ret != 0))
- return ret;
-
- ttm_bo_get(&user_bo->vbo.base);
- ret = ttm_prime_object_init(tfile,
- size,
- &user_bo->prime,
- shareable,
- ttm_buffer_type,
- &vmw_user_bo_release,
- &vmw_user_bo_ref_obj_release);
- if (unlikely(ret != 0)) {
- ttm_bo_put(&user_bo->vbo.base);
- goto out_no_base_object;
- }
-
- *p_vbo = &user_bo->vbo;
- if (p_base) {
- *p_base = &user_bo->prime.base;
- kref_get(&(*p_base)->refcount);
- }
- *handle = user_bo->prime.base.handle;
-
-out_no_base_object:
- return ret;
-}
-
-
-/**
- * vmw_user_bo_verify_access - verify access permissions on this
- * buffer object.
- *
- * @bo: Pointer to the buffer object being accessed
- * @tfile: Identifying the caller.
- */
-int vmw_user_bo_verify_access(struct ttm_buffer_object *bo,
- struct ttm_object_file *tfile)
-{
- struct vmw_user_buffer_object *vmw_user_bo;
-
- if (unlikely(bo->destroy != vmw_user_bo_destroy))
- return -EPERM;
-
- vmw_user_bo = vmw_user_buffer_object(bo);
-
- /* Check that the caller has opened the object. */
- if (likely(ttm_ref_object_exists(tfile, &vmw_user_bo->prime.base)))
- return 0;
-
- DRM_ERROR("Could not grant buffer access.\n");
- return -EPERM;
-}
-
-
-/**
- * vmw_user_bo_synccpu_grab - Grab a struct vmw_user_buffer_object for cpu
+ * vmw_user_bo_synccpu_grab - Grab a struct vmw_buffer_object for cpu
* access, idling previous GPU operations on the buffer and optionally
* blocking it for further command submissions.
*
- * @user_bo: Pointer to the buffer object being grabbed for CPU access
- * @tfile: Identifying the caller.
+ * @vmw_bo: Pointer to the buffer object being grabbed for CPU access
* @flags: Flags indicating how the grab should be performed.
* Return: Zero on success, Negative error code on error. In particular,
* -EBUSY will be returned if a dontblock operation is requested and the
@@ -727,13 +518,11 @@ int vmw_user_bo_verify_access(struct ttm_buffer_object *bo,
*
* A blocking grab will be automatically released when @tfile is closed.
*/
-static int vmw_user_bo_synccpu_grab(struct vmw_user_buffer_object *user_bo,
- struct ttm_object_file *tfile,
+static int vmw_user_bo_synccpu_grab(struct vmw_buffer_object *vmw_bo,
uint32_t flags)
{
bool nonblock = !!(flags & drm_vmw_synccpu_dontblock);
- struct ttm_buffer_object *bo = &user_bo->vbo.base;
- bool existed;
+ struct ttm_buffer_object *bo = &vmw_bo->base;
int ret;
if (flags & drm_vmw_synccpu_allow_cs) {
@@ -755,17 +544,12 @@ static int vmw_user_bo_synccpu_grab(struct vmw_user_buffer_object *user_bo,
ret = ttm_bo_wait(bo, true, nonblock);
if (likely(ret == 0))
- atomic_inc(&user_bo->vbo.cpu_writers);
+ atomic_inc(&vmw_bo->cpu_writers);
ttm_bo_unreserve(bo);
if (unlikely(ret != 0))
return ret;
- ret = ttm_ref_object_add(tfile, &user_bo->prime.base,
- TTM_REF_SYNCCPU_WRITE, &existed, false);
- if (ret != 0 || existed)
- atomic_dec(&user_bo->vbo.cpu_writers);
-
return ret;
}
@@ -773,19 +557,25 @@ static int vmw_user_bo_synccpu_grab(struct vmw_user_buffer_object *user_bo,
* vmw_user_bo_synccpu_release - Release a previous grab for CPU access,
* and unblock command submission on the buffer if blocked.
*
+ * @filp: Identifying the caller.
* @handle: Handle identifying the buffer object.
- * @tfile: Identifying the caller.
* @flags: Flags indicating the type of release.
*/
-static int vmw_user_bo_synccpu_release(uint32_t handle,
- struct ttm_object_file *tfile,
- uint32_t flags)
+static int vmw_user_bo_synccpu_release(struct drm_file *filp,
+ uint32_t handle,
+ uint32_t flags)
{
- if (!(flags & drm_vmw_synccpu_allow_cs))
- return ttm_ref_object_base_unref(tfile, handle,
- TTM_REF_SYNCCPU_WRITE);
+ struct vmw_buffer_object *vmw_bo;
+ int ret = vmw_user_bo_lookup(filp, handle, &vmw_bo);
- return 0;
+ if (!ret) {
+ if (!(flags & drm_vmw_synccpu_allow_cs)) {
+ atomic_dec(&vmw_bo->cpu_writers);
+ }
+ ttm_bo_put(&vmw_bo->base);
+ }
+
+ return ret;
}
@@ -807,9 +597,6 @@ int vmw_user_bo_synccpu_ioctl(struct drm_device *dev, void *data,
struct drm_vmw_synccpu_arg *arg =
(struct drm_vmw_synccpu_arg *) data;
struct vmw_buffer_object *vbo;
- struct vmw_user_buffer_object *user_bo;
- struct ttm_object_file *tfile = vmw_fpriv(file_priv)->tfile;
- struct ttm_base_object *buffer_base;
int ret;
if ((arg->flags & (drm_vmw_synccpu_read | drm_vmw_synccpu_write)) == 0
@@ -822,16 +609,12 @@ int vmw_user_bo_synccpu_ioctl(struct drm_device *dev, void *data,
switch (arg->op) {
case drm_vmw_synccpu_grab:
- ret = vmw_user_bo_lookup(tfile, arg->handle, &vbo,
- &buffer_base);
+ ret = vmw_user_bo_lookup(file_priv, arg->handle, &vbo);
if (unlikely(ret != 0))
return ret;
- user_bo = container_of(vbo, struct vmw_user_buffer_object,
- vbo);
- ret = vmw_user_bo_synccpu_grab(user_bo, tfile, arg->flags);
+ ret = vmw_user_bo_synccpu_grab(vbo, arg->flags);
vmw_bo_unreference(&vbo);
- ttm_base_object_unref(&buffer_base);
if (unlikely(ret != 0 && ret != -ERESTARTSYS &&
ret != -EBUSY)) {
DRM_ERROR("Failed synccpu grab on handle 0x%08x.\n",
@@ -840,7 +623,8 @@ int vmw_user_bo_synccpu_ioctl(struct drm_device *dev, void *data,
}
break;
case drm_vmw_synccpu_release:
- ret = vmw_user_bo_synccpu_release(arg->handle, tfile,
+ ret = vmw_user_bo_synccpu_release(file_priv,
+ arg->handle,
arg->flags);
if (unlikely(ret != 0)) {
DRM_ERROR("Failed synccpu release on handle 0x%08x.\n",
@@ -856,50 +640,6 @@ int vmw_user_bo_synccpu_ioctl(struct drm_device *dev, void *data,
return 0;
}
-
-/**
- * vmw_bo_alloc_ioctl - ioctl function implementing the buffer object
- * allocation functionality.
- *
- * @dev: Identifies the drm device.
- * @data: Pointer to the ioctl argument.
- * @file_priv: Identifies the caller.
- * Return: Zero on success, negative error code on error.
- *
- * This function checks the ioctl arguments for validity and allocates a
- * struct vmw_user_buffer_object bo.
- */
-int vmw_bo_alloc_ioctl(struct drm_device *dev, void *data,
- struct drm_file *file_priv)
-{
- struct vmw_private *dev_priv = vmw_priv(dev);
- union drm_vmw_alloc_dmabuf_arg *arg =
- (union drm_vmw_alloc_dmabuf_arg *)data;
- struct drm_vmw_alloc_dmabuf_req *req = &arg->req;
- struct drm_vmw_dmabuf_rep *rep = &arg->rep;
- struct vmw_buffer_object *vbo;
- uint32_t handle;
- int ret;
-
- ret = vmw_user_bo_alloc(dev_priv, vmw_fpriv(file_priv)->tfile,
- req->size, false, &handle, &vbo,
- NULL);
- if (unlikely(ret != 0))
- goto out_no_bo;
-
- rep->handle = handle;
- rep->map_handle = drm_vma_node_offset_addr(&vbo->base.base.vma_node);
- rep->cur_gmr_id = handle;
- rep->cur_gmr_offset = 0;
-
- vmw_bo_unreference(&vbo);
-
-out_no_bo:
-
- return ret;
-}
-
-
/**
* vmw_bo_unref_ioctl - Generic handle close ioctl.
*
@@ -917,65 +657,48 @@ int vmw_bo_unref_ioctl(struct drm_device *dev, void *data,
struct drm_vmw_unref_dmabuf_arg *arg =
(struct drm_vmw_unref_dmabuf_arg *)data;
- return ttm_ref_object_base_unref(vmw_fpriv(file_priv)->tfile,
- arg->handle,
- TTM_REF_USAGE);
+ drm_gem_handle_delete(file_priv, arg->handle);
+ return 0;
}
/**
* vmw_user_bo_lookup - Look up a vmw user buffer object from a handle.
*
- * @tfile: The TTM object file the handle is registered with.
+ * @filp: The file the handle is registered with.
* @handle: The user buffer object handle
* @out: Pointer to a where a pointer to the embedded
* struct vmw_buffer_object should be placed.
- * @p_base: Pointer to where a pointer to the TTM base object should be
- * placed, or NULL if no such pointer is required.
* Return: Zero on success, Negative error code on error.
*
- * Both the output base object pointer and the vmw buffer object pointer
- * will be refcounted.
+ * The vmw buffer object pointer will be refcounted.
*/
-int vmw_user_bo_lookup(struct ttm_object_file *tfile,
- uint32_t handle, struct vmw_buffer_object **out,
- struct ttm_base_object **p_base)
+int vmw_user_bo_lookup(struct drm_file *filp,
+ uint32_t handle,
+ struct vmw_buffer_object **out)
{
- struct vmw_user_buffer_object *vmw_user_bo;
- struct ttm_base_object *base;
+ struct drm_gem_object *gobj;
- base = ttm_base_object_lookup(tfile, handle);
- if (unlikely(base == NULL)) {
+ gobj = drm_gem_object_lookup(filp, handle);
+ if (!gobj) {
DRM_ERROR("Invalid buffer object handle 0x%08lx.\n",
(unsigned long)handle);
return -ESRCH;
}
- if (unlikely(ttm_base_object_type(base) != ttm_buffer_type)) {
- ttm_base_object_unref(&base);
- DRM_ERROR("Invalid buffer object handle 0x%08lx.\n",
- (unsigned long)handle);
- return -EINVAL;
- }
-
- vmw_user_bo = container_of(base, struct vmw_user_buffer_object,
- prime.base);
- ttm_bo_get(&vmw_user_bo->vbo.base);
- if (p_base)
- *p_base = base;
- else
- ttm_base_object_unref(&base);
- *out = &vmw_user_bo->vbo;
+ *out = gem_to_vmw_bo(gobj);
+ ttm_bo_get(&(*out)->base);
+ drm_gem_object_put(gobj);
return 0;
}
/**
* vmw_user_bo_noref_lookup - Look up a vmw user buffer object without reference
- * @tfile: The TTM object file the handle is registered with.
+ * @filp: The TTM object file the handle is registered with.
* @handle: The user buffer object handle.
*
- * This function looks up a struct vmw_user_bo and returns a pointer to the
+ * This function looks up a struct vmw_bo and returns a pointer to the
* struct vmw_buffer_object it derives from without refcounting the pointer.
* The returned pointer is only valid until vmw_user_bo_noref_release() is
* called, and the object pointed to by the returned pointer may be doomed.
@@ -988,52 +711,23 @@ int vmw_user_bo_lookup(struct ttm_object_file *tfile,
* error pointer on failure.
*/
struct vmw_buffer_object *
-vmw_user_bo_noref_lookup(struct ttm_object_file *tfile, u32 handle)
+vmw_user_bo_noref_lookup(struct drm_file *filp, u32 handle)
{
- struct vmw_user_buffer_object *vmw_user_bo;
- struct ttm_base_object *base;
+ struct vmw_buffer_object *vmw_bo;
+ struct ttm_buffer_object *bo;
+ struct drm_gem_object *gobj = drm_gem_object_lookup(filp, handle);
- base = ttm_base_object_noref_lookup(tfile, handle);
- if (!base) {
+ if (!gobj) {
DRM_ERROR("Invalid buffer object handle 0x%08lx.\n",
(unsigned long)handle);
return ERR_PTR(-ESRCH);
}
+ vmw_bo = gem_to_vmw_bo(gobj);
+ bo = ttm_bo_get_unless_zero(&vmw_bo->base);
+ vmw_bo = vmw_buffer_object(bo);
+ drm_gem_object_put(gobj);
- if (unlikely(ttm_base_object_type(base) != ttm_buffer_type)) {
- ttm_base_object_noref_release();
- DRM_ERROR("Invalid buffer object handle 0x%08lx.\n",
- (unsigned long)handle);
- return ERR_PTR(-EINVAL);
- }
-
- vmw_user_bo = container_of(base, struct vmw_user_buffer_object,
- prime.base);
- return &vmw_user_bo->vbo;
-}
-
-/**
- * vmw_user_bo_reference - Open a handle to a vmw user buffer object.
- *
- * @tfile: The TTM object file to register the handle with.
- * @vbo: The embedded vmw buffer object.
- * @handle: Pointer to where the new handle should be placed.
- * Return: Zero on success, Negative error code on error.
- */
-int vmw_user_bo_reference(struct ttm_object_file *tfile,
- struct vmw_buffer_object *vbo,
- uint32_t *handle)
-{
- struct vmw_user_buffer_object *user_bo;
-
- if (vbo->base.destroy != vmw_user_bo_destroy)
- return -EINVAL;
-
- user_bo = container_of(vbo, struct vmw_user_buffer_object, vbo);
-
- *handle = user_bo->prime.base.handle;
- return ttm_ref_object_add(tfile, &user_bo->prime.base,
- TTM_REF_USAGE, NULL, false);
+ return vmw_bo;
}
@@ -1087,68 +781,15 @@ int vmw_dumb_create(struct drm_file *file_priv,
int ret;
args->pitch = args->width * ((args->bpp + 7) / 8);
- args->size = args->pitch * args->height;
+ args->size = ALIGN(args->pitch * args->height, PAGE_SIZE);
- ret = vmw_user_bo_alloc(dev_priv, vmw_fpriv(file_priv)->tfile,
- args->size, false, &args->handle,
- &vbo, NULL);
- if (unlikely(ret != 0))
- goto out_no_bo;
+ ret = vmw_gem_object_create_with_handle(dev_priv, file_priv,
+ args->size, &args->handle,
+ &vbo);
- vmw_bo_unreference(&vbo);
-out_no_bo:
return ret;
}
-
-/**
- * vmw_dumb_map_offset - Return the address space offset of a dumb buffer
- *
- * @file_priv: Pointer to a struct drm_file identifying the caller.
- * @dev: Pointer to the drm device.
- * @handle: Handle identifying the dumb buffer.
- * @offset: The address space offset returned.
- * Return: Zero on success, negative error code on failure.
- *
- * This is a driver callback for the core drm dumb_map_offset functionality.
- */
-int vmw_dumb_map_offset(struct drm_file *file_priv,
- struct drm_device *dev, uint32_t handle,
- uint64_t *offset)
-{
- struct ttm_object_file *tfile = vmw_fpriv(file_priv)->tfile;
- struct vmw_buffer_object *out_buf;
- int ret;
-
- ret = vmw_user_bo_lookup(tfile, handle, &out_buf, NULL);
- if (ret != 0)
- return -EINVAL;
-
- *offset = drm_vma_node_offset_addr(&out_buf->base.base.vma_node);
- vmw_bo_unreference(&out_buf);
- return 0;
-}
-
-
-/**
- * vmw_dumb_destroy - Destroy a dumb boffer
- *
- * @file_priv: Pointer to a struct drm_file identifying the caller.
- * @dev: Pointer to the drm device.
- * @handle: Handle identifying the dumb buffer.
- * Return: Zero on success, negative error code on failure.
- *
- * This is a driver callback for the core drm dumb_destroy functionality.
- */
-int vmw_dumb_destroy(struct drm_file *file_priv,
- struct drm_device *dev,
- uint32_t handle)
-{
- return ttm_ref_object_base_unref(vmw_fpriv(file_priv)->tfile,
- handle, TTM_REF_USAGE);
-}
-
-
/**
* vmw_bo_swap_notify - swapout notify callback.
*
@@ -1157,8 +798,7 @@ int vmw_dumb_destroy(struct drm_file *file_priv,
void vmw_bo_swap_notify(struct ttm_buffer_object *bo)
{
/* Is @bo embedded in a struct vmw_buffer_object? */
- if (bo->destroy != vmw_bo_bo_free &&
- bo->destroy != vmw_user_bo_destroy)
+ if (vmw_bo_is_vmw_bo(bo))
return;
/* Kill any cached kernel maps before swapout */
@@ -1182,8 +822,7 @@ void vmw_bo_move_notify(struct ttm_buffer_object *bo,
struct vmw_buffer_object *vbo;
/* Make sure @bo is embedded in a struct vmw_buffer_object? */
- if (bo->destroy != vmw_bo_bo_free &&
- bo->destroy != vmw_user_bo_destroy)
+ if (vmw_bo_is_vmw_bo(bo))
return;
vbo = container_of(bo, struct vmw_buffer_object, base);
@@ -1204,3 +843,22 @@ void vmw_bo_move_notify(struct ttm_buffer_object *bo,
if (mem->mem_type != VMW_PL_MOB && bo->resource->mem_type == VMW_PL_MOB)
vmw_resource_unbind_list(vbo);
}
+
+/**
+ * vmw_bo_is_vmw_bo - check if the buffer object is a &vmw_buffer_object
+ * @bo: buffer object to be checked
+ *
+ * Uses destroy function associated with the object to determine if this is
+ * a &vmw_buffer_object.
+ *
+ * Returns:
+ * true if the object is of &vmw_buffer_object type, false if not.
+ */
+bool vmw_bo_is_vmw_bo(struct ttm_buffer_object *bo)
+{
+ if (bo->destroy == &vmw_bo_bo_free ||
+ bo->destroy == &vmw_gem_destroy)
+ return true;
+
+ return false;
+}
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_cmd.c b/drivers/gpu/drm/vmwgfx/vmwgfx_cmd.c
index 67db472..a3bfbb6c 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_cmd.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_cmd.c
@@ -145,6 +145,13 @@ struct vmw_fifo_state *vmw_fifo_create(struct vmw_private *dev_priv)
(unsigned int) max,
(unsigned int) min,
(unsigned int) fifo->capabilities);
+
+ if (unlikely(min >= max)) {
+ drm_warn(&dev_priv->drm,
+ "FIFO memory is not usable. Driver failed to initialize.");
+ return ERR_PTR(-ENXIO);
+ }
+
return fifo;
}
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_cmdbuf_res.c b/drivers/gpu/drm/vmwgfx/vmwgfx_cmdbuf_res.c
index 8381750..415774f 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_cmdbuf_res.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_cmdbuf_res.c
@@ -42,7 +42,7 @@
*/
struct vmw_cmdbuf_res {
struct vmw_resource *res;
- struct drm_hash_item hash;
+ struct vmwgfx_hash_item hash;
struct list_head head;
enum vmw_cmdbuf_res_state state;
struct vmw_cmdbuf_res_manager *man;
@@ -59,7 +59,7 @@ struct vmw_cmdbuf_res {
* @resources and @list are protected by the cmdbuf mutex for now.
*/
struct vmw_cmdbuf_res_manager {
- struct drm_open_hash resources;
+ struct vmwgfx_open_hash resources;
struct list_head list;
struct vmw_private *dev_priv;
};
@@ -81,11 +81,11 @@ vmw_cmdbuf_res_lookup(struct vmw_cmdbuf_res_manager *man,
enum vmw_cmdbuf_res_type res_type,
u32 user_key)
{
- struct drm_hash_item *hash;
+ struct vmwgfx_hash_item *hash;
int ret;
unsigned long key = user_key | (res_type << 24);
- ret = drm_ht_find_item(&man->resources, key, &hash);
+ ret = vmwgfx_ht_find_item(&man->resources, key, &hash);
if (unlikely(ret != 0))
return ERR_PTR(ret);
@@ -105,7 +105,7 @@ static void vmw_cmdbuf_res_free(struct vmw_cmdbuf_res_manager *man,
struct vmw_cmdbuf_res *entry)
{
list_del(&entry->head);
- WARN_ON(drm_ht_remove_item(&man->resources, &entry->hash));
+ WARN_ON(vmwgfx_ht_remove_item(&man->resources, &entry->hash));
vmw_resource_unreference(&entry->res);
kfree(entry);
}
@@ -167,7 +167,7 @@ void vmw_cmdbuf_res_revert(struct list_head *list)
vmw_cmdbuf_res_free(entry->man, entry);
break;
case VMW_CMDBUF_RES_DEL:
- ret = drm_ht_insert_item(&entry->man->resources, &entry->hash);
+ ret = vmwgfx_ht_insert_item(&entry->man->resources, &entry->hash);
BUG_ON(ret);
list_move_tail(&entry->head, &entry->man->list);
entry->state = VMW_CMDBUF_RES_COMMITTED;
@@ -206,7 +206,7 @@ int vmw_cmdbuf_res_add(struct vmw_cmdbuf_res_manager *man,
return -ENOMEM;
cres->hash.key = user_key | (res_type << 24);
- ret = drm_ht_insert_item(&man->resources, &cres->hash);
+ ret = vmwgfx_ht_insert_item(&man->resources, &cres->hash);
if (unlikely(ret != 0)) {
kfree(cres);
goto out_invalid_key;
@@ -244,10 +244,10 @@ int vmw_cmdbuf_res_remove(struct vmw_cmdbuf_res_manager *man,
struct vmw_resource **res_p)
{
struct vmw_cmdbuf_res *entry;
- struct drm_hash_item *hash;
+ struct vmwgfx_hash_item *hash;
int ret;
- ret = drm_ht_find_item(&man->resources, user_key | (res_type << 24),
+ ret = vmwgfx_ht_find_item(&man->resources, user_key | (res_type << 24),
&hash);
if (likely(ret != 0))
return -EINVAL;
@@ -260,7 +260,7 @@ int vmw_cmdbuf_res_remove(struct vmw_cmdbuf_res_manager *man,
*res_p = NULL;
break;
case VMW_CMDBUF_RES_COMMITTED:
- (void) drm_ht_remove_item(&man->resources, &entry->hash);
+ (void) vmwgfx_ht_remove_item(&man->resources, &entry->hash);
list_del(&entry->head);
entry->state = VMW_CMDBUF_RES_DEL;
list_add_tail(&entry->head, list);
@@ -295,7 +295,7 @@ vmw_cmdbuf_res_man_create(struct vmw_private *dev_priv)
man->dev_priv = dev_priv;
INIT_LIST_HEAD(&man->list);
- ret = drm_ht_create(&man->resources, VMW_CMDBUF_RES_MAN_HT_ORDER);
+ ret = vmwgfx_ht_create(&man->resources, VMW_CMDBUF_RES_MAN_HT_ORDER);
if (ret == 0)
return man;
@@ -320,26 +320,7 @@ void vmw_cmdbuf_res_man_destroy(struct vmw_cmdbuf_res_manager *man)
list_for_each_entry_safe(entry, next, &man->list, head)
vmw_cmdbuf_res_free(man, entry);
- drm_ht_remove(&man->resources);
+ vmwgfx_ht_remove(&man->resources);
kfree(man);
}
-/**
- * vmw_cmdbuf_res_man_size - Return the size of a command buffer managed
- * resource manager
- *
- * Returns the approximate allocation size of a command buffer managed
- * resource manager.
- */
-size_t vmw_cmdbuf_res_man_size(void)
-{
- static size_t res_man_size;
-
- if (unlikely(res_man_size == 0))
- res_man_size =
- ttm_round_pot(sizeof(struct vmw_cmdbuf_res_manager)) +
- ttm_round_pot(sizeof(struct hlist_head) <<
- VMW_CMDBUF_RES_MAN_HT_ORDER);
-
- return res_man_size;
-}
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_context.c b/drivers/gpu/drm/vmwgfx/vmwgfx_context.c
index 4446758..e0f48cd 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_context.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_context.c
@@ -60,8 +60,6 @@ static int vmw_dx_context_unbind(struct vmw_resource *res,
struct ttm_validate_buffer *val_buf);
static int vmw_dx_context_destroy(struct vmw_resource *res);
-static uint64_t vmw_user_context_size;
-
static const struct vmw_user_resource_conv user_context_conv = {
.object_type = VMW_RES_CONTEXT,
.base_obj_to_res = vmw_user_context_base_to_res,
@@ -686,7 +684,6 @@ static void vmw_user_context_free(struct vmw_resource *res)
{
struct vmw_user_context *ctx =
container_of(res, struct vmw_user_context, res);
- struct vmw_private *dev_priv = res->dev_priv;
if (ctx->cbs)
vmw_binding_state_free(ctx->cbs);
@@ -694,8 +691,6 @@ static void vmw_user_context_free(struct vmw_resource *res)
(void) vmw_context_bind_dx_query(res, NULL);
ttm_base_object_kfree(ctx, base);
- ttm_mem_global_free(vmw_mem_glob(dev_priv),
- vmw_user_context_size);
}
/*
@@ -720,7 +715,7 @@ int vmw_context_destroy_ioctl(struct drm_device *dev, void *data,
struct drm_vmw_context_arg *arg = (struct drm_vmw_context_arg *)data;
struct ttm_object_file *tfile = vmw_fpriv(file_priv)->tfile;
- return ttm_ref_object_base_unref(tfile, arg->cid, TTM_REF_USAGE);
+ return ttm_ref_object_base_unref(tfile, arg->cid);
}
static int vmw_context_define(struct drm_device *dev, void *data,
@@ -732,10 +727,6 @@ static int vmw_context_define(struct drm_device *dev, void *data,
struct vmw_resource *tmp;
struct drm_vmw_context_arg *arg = (struct drm_vmw_context_arg *)data;
struct ttm_object_file *tfile = vmw_fpriv(file_priv)->tfile;
- struct ttm_operation_ctx ttm_opt_ctx = {
- .interruptible = true,
- .no_wait_gpu = false
- };
int ret;
if (!has_sm4_context(dev_priv) && dx) {
@@ -743,25 +734,8 @@ static int vmw_context_define(struct drm_device *dev, void *data,
return -EINVAL;
}
- if (unlikely(vmw_user_context_size == 0))
- vmw_user_context_size = ttm_round_pot(sizeof(*ctx)) +
- ((dev_priv->has_mob) ? vmw_cmdbuf_res_man_size() : 0) +
- + VMW_IDA_ACC_SIZE + TTM_OBJ_EXTRA_SIZE;
-
- ret = ttm_mem_global_alloc(vmw_mem_glob(dev_priv),
- vmw_user_context_size,
- &ttm_opt_ctx);
- if (unlikely(ret != 0)) {
- if (ret != -ERESTARTSYS)
- DRM_ERROR("Out of graphics memory for context"
- " creation.\n");
- goto out_ret;
- }
-
ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
if (unlikely(!ctx)) {
- ttm_mem_global_free(vmw_mem_glob(dev_priv),
- vmw_user_context_size);
ret = -ENOMEM;
goto out_ret;
}
@@ -780,7 +754,7 @@ static int vmw_context_define(struct drm_device *dev, void *data,
tmp = vmw_resource_reference(&ctx->res);
ret = ttm_base_object_init(tfile, &ctx->base, false, VMW_RES_CONTEXT,
- &vmw_user_context_base_release, NULL);
+ &vmw_user_context_base_release);
if (unlikely(ret != 0)) {
vmw_resource_unreference(&tmp);
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_cotable.c b/drivers/gpu/drm/vmwgfx/vmwgfx_cotable.c
index 17a98db..16f986b 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_cotable.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_cotable.c
@@ -407,12 +407,8 @@ static int vmw_cotable_resize(struct vmw_resource *res, size_t new_size)
* for the new COTable. Initially pin the buffer object to make sure
* we can use tryreserve without failure.
*/
- buf = kzalloc(sizeof(*buf), GFP_KERNEL);
- if (!buf)
- return -ENOMEM;
-
- ret = vmw_bo_init(dev_priv, buf, new_size, &vmw_mob_placement,
- true, true, vmw_bo_bo_free);
+ ret = vmw_bo_create(dev_priv, new_size, &vmw_mob_placement,
+ true, true, vmw_bo_bo_free, &buf);
if (ret) {
DRM_ERROR("Failed initializing new cotable MOB.\n");
return ret;
@@ -546,8 +542,6 @@ static void vmw_hw_cotable_destroy(struct vmw_resource *res)
(void) vmw_cotable_destroy(res);
}
-static size_t cotable_acc_size;
-
/**
* vmw_cotable_free - Cotable resource destructor
*
@@ -555,10 +549,7 @@ static size_t cotable_acc_size;
*/
static void vmw_cotable_free(struct vmw_resource *res)
{
- struct vmw_private *dev_priv = res->dev_priv;
-
kfree(res);
- ttm_mem_global_free(vmw_mem_glob(dev_priv), cotable_acc_size);
}
/**
@@ -574,21 +565,9 @@ struct vmw_resource *vmw_cotable_alloc(struct vmw_private *dev_priv,
u32 type)
{
struct vmw_cotable *vcotbl;
- struct ttm_operation_ctx ttm_opt_ctx = {
- .interruptible = true,
- .no_wait_gpu = false
- };
int ret;
u32 num_entries;
- if (unlikely(cotable_acc_size == 0))
- cotable_acc_size = ttm_round_pot(sizeof(struct vmw_cotable));
-
- ret = ttm_mem_global_alloc(vmw_mem_glob(dev_priv),
- cotable_acc_size, &ttm_opt_ctx);
- if (unlikely(ret))
- return ERR_PTR(ret);
-
vcotbl = kzalloc(sizeof(*vcotbl), GFP_KERNEL);
if (unlikely(!vcotbl)) {
ret = -ENOMEM;
@@ -622,7 +601,6 @@ struct vmw_resource *vmw_cotable_alloc(struct vmw_private *dev_priv,
out_no_init:
kfree(vcotbl);
out_no_alloc:
- ttm_mem_global_free(vmw_mem_glob(dev_priv), cotable_acc_size);
return ERR_PTR(ret);
}
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_drv.c b/drivers/gpu/drm/vmwgfx/vmwgfx_drv.c
index bfd71c8..fe36efd 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_drv.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_drv.c
@@ -25,7 +25,6 @@
*
**************************************************************************/
-#include <linux/console.h>
#include <linux/dma-mapping.h>
#include <linux/module.h>
#include <linux/pci.h>
@@ -35,6 +34,7 @@
#include <drm/drm_drv.h>
#include <drm/drm_ioctl.h>
#include <drm/drm_sysfs.h>
+#include <drm/drm_gem_ttm_helper.h>
#include <drm/ttm/ttm_bo_driver.h>
#include <drm/ttm/ttm_range_manager.h>
#include <drm/ttm/ttm_placement.h>
@@ -51,9 +51,6 @@
#define VMW_MIN_INITIAL_WIDTH 800
#define VMW_MIN_INITIAL_HEIGHT 600
-#define VMWGFX_VALIDATION_MEM_GRAN (16*PAGE_SIZE)
-
-
/*
* Fully encoded drm commands. Might move to vmw_drm.h
*/
@@ -166,7 +163,7 @@
static const struct drm_ioctl_desc vmw_ioctls[] = {
DRM_IOCTL_DEF_DRV(VMW_GET_PARAM, vmw_getparam_ioctl,
DRM_RENDER_ALLOW),
- DRM_IOCTL_DEF_DRV(VMW_ALLOC_DMABUF, vmw_bo_alloc_ioctl,
+ DRM_IOCTL_DEF_DRV(VMW_ALLOC_DMABUF, vmw_gem_object_create_ioctl,
DRM_RENDER_ALLOW),
DRM_IOCTL_DEF_DRV(VMW_UNREF_DMABUF, vmw_bo_unref_ioctl,
DRM_RENDER_ALLOW),
@@ -258,8 +255,8 @@ static const struct drm_ioctl_desc vmw_ioctls[] = {
};
static const struct pci_device_id vmw_pci_id_list[] = {
- { PCI_DEVICE(0x15ad, VMWGFX_PCI_ID_SVGA2) },
- { PCI_DEVICE(0x15ad, VMWGFX_PCI_ID_SVGA3) },
+ { PCI_DEVICE(PCI_VENDOR_ID_VMWARE, VMWGFX_PCI_ID_SVGA2) },
+ { PCI_DEVICE(PCI_VENDOR_ID_VMWARE, VMWGFX_PCI_ID_SVGA3) },
{ }
};
MODULE_DEVICE_TABLE(pci, vmw_pci_id_list);
@@ -367,6 +364,7 @@ static void vmw_print_sm_type(struct vmw_private *dev_priv)
[VMW_SM_4] = "SM4",
[VMW_SM_4_1] = "SM4_1",
[VMW_SM_5] = "SM_5",
+ [VMW_SM_5_1X] = "SM_5_1X",
[VMW_SM_MAX] = "Invalid"
};
BUILD_BUG_ON(ARRAY_SIZE(names) != (VMW_SM_MAX + 1));
@@ -400,13 +398,9 @@ static int vmw_dummy_query_bo_create(struct vmw_private *dev_priv)
* immediately succeed. This is because we're the only
* user of the bo currently.
*/
- vbo = kzalloc(sizeof(*vbo), GFP_KERNEL);
- if (!vbo)
- return -ENOMEM;
-
- ret = vmw_bo_init(dev_priv, vbo, PAGE_SIZE,
- &vmw_sys_placement, false, true,
- &vmw_bo_bo_free);
+ ret = vmw_bo_create(dev_priv, PAGE_SIZE,
+ &vmw_sys_placement, false, true,
+ &vmw_bo_bo_free, &vbo);
if (unlikely(ret != 0))
return ret;
@@ -707,23 +701,15 @@ static int vmw_dma_masks(struct vmw_private *dev_priv)
static int vmw_vram_manager_init(struct vmw_private *dev_priv)
{
int ret;
-#ifdef CONFIG_TRANSPARENT_HUGEPAGE
- ret = vmw_thp_init(dev_priv);
-#else
ret = ttm_range_man_init(&dev_priv->bdev, TTM_PL_VRAM, false,
dev_priv->vram_size >> PAGE_SHIFT);
-#endif
ttm_resource_manager_set_used(ttm_manager_type(&dev_priv->bdev, TTM_PL_VRAM), false);
return ret;
}
static void vmw_vram_manager_fini(struct vmw_private *dev_priv)
{
-#ifdef CONFIG_TRANSPARENT_HUGEPAGE
- vmw_thp_fini(dev_priv);
-#else
ttm_range_man_fini(&dev_priv->bdev, TTM_PL_VRAM);
-#endif
}
static int vmw_setup_pci_resources(struct vmw_private *dev,
@@ -987,8 +973,7 @@ static int vmw_driver_load(struct vmw_private *dev_priv, u32 pci_id)
goto out_err0;
}
- dev_priv->tdev = ttm_object_device_init(&ttm_mem_glob, 12,
- &vmw_prime_dmabuf_ops);
+ dev_priv->tdev = ttm_object_device_init(12, &vmw_prime_dmabuf_ops);
if (unlikely(dev_priv->tdev == NULL)) {
drm_err(&dev_priv->drm,
@@ -1071,6 +1056,12 @@ static int vmw_driver_load(struct vmw_private *dev_priv, u32 pci_id)
"3D will be disabled.\n");
dev_priv->has_mob = false;
}
+ if (vmw_sys_man_init(dev_priv) != 0) {
+ drm_info(&dev_priv->drm,
+ "No MOB page table memory available. "
+ "3D will be disabled.\n");
+ dev_priv->has_mob = false;
+ }
}
if (dev_priv->has_mob && (dev_priv->capabilities & SVGA_CAP_DX)) {
@@ -1078,8 +1069,6 @@ static int vmw_driver_load(struct vmw_private *dev_priv, u32 pci_id)
dev_priv->sm_type = VMW_SM_4;
}
- vmw_validation_mem_init_ttm(dev_priv, VMWGFX_VALIDATION_MEM_GRAN);
-
/* SVGA_CAP2_DX2 (DefineGBSurface_v3) is needed for SM4_1 support */
if (has_sm4_context(dev_priv) &&
(dev_priv->capabilities2 & SVGA_CAP2_DX2)) {
@@ -1087,8 +1076,11 @@ static int vmw_driver_load(struct vmw_private *dev_priv, u32 pci_id)
dev_priv->sm_type = VMW_SM_4_1;
if (has_sm4_1_context(dev_priv) &&
(dev_priv->capabilities2 & SVGA_CAP2_DX3)) {
- if (vmw_devcap_get(dev_priv, SVGA3D_DEVCAP_SM5))
+ if (vmw_devcap_get(dev_priv, SVGA3D_DEVCAP_SM5)) {
dev_priv->sm_type = VMW_SM_5;
+ if (vmw_devcap_get(dev_priv, SVGA3D_DEVCAP_GL43))
+ dev_priv->sm_type = VMW_SM_5_1X;
+ }
}
}
@@ -1121,8 +1113,10 @@ static int vmw_driver_load(struct vmw_private *dev_priv, u32 pci_id)
vmw_overlay_close(dev_priv);
vmw_kms_close(dev_priv);
out_no_kms:
- if (dev_priv->has_mob)
+ if (dev_priv->has_mob) {
vmw_gmrid_man_fini(dev_priv, VMW_PL_MOB);
+ vmw_sys_man_fini(dev_priv);
+ }
if (dev_priv->has_gmr)
vmw_gmrid_man_fini(dev_priv, VMW_PL_GMR);
vmw_devcaps_destroy(dev_priv);
@@ -1156,7 +1150,7 @@ static void vmw_driver_unload(struct drm_device *dev)
unregister_pm_notifier(&dev_priv->pm_nb);
if (dev_priv->ctx.res_ht_initialized)
- drm_ht_remove(&dev_priv->ctx.res_ht);
+ vmwgfx_ht_remove(&dev_priv->ctx.res_ht);
vfree(dev_priv->ctx.cmd_bounce);
if (dev_priv->enable_fb) {
vmw_fb_off(dev_priv);
@@ -1172,8 +1166,10 @@ static void vmw_driver_unload(struct drm_device *dev)
vmw_gmrid_man_fini(dev_priv, VMW_PL_GMR);
vmw_release_device_early(dev_priv);
- if (dev_priv->has_mob)
+ if (dev_priv->has_mob) {
vmw_gmrid_man_fini(dev_priv, VMW_PL_MOB);
+ vmw_sys_man_fini(dev_priv);
+ }
vmw_devcaps_destroy(dev_priv);
vmw_vram_manager_fini(dev_priv);
ttm_device_fini(&dev_priv->bdev);
@@ -1388,7 +1384,6 @@ static void vmw_remove(struct pci_dev *pdev)
{
struct drm_device *dev = pci_get_drvdata(pdev);
- ttm_mem_global_release(&ttm_mem_glob);
drm_dev_unregister(dev);
vmw_driver_unload(dev);
}
@@ -1576,7 +1571,7 @@ static const struct file_operations vmwgfx_driver_fops = {
static const struct drm_driver driver = {
.driver_features =
- DRIVER_MODESET | DRIVER_RENDER | DRIVER_ATOMIC,
+ DRIVER_MODESET | DRIVER_RENDER | DRIVER_ATOMIC | DRIVER_GEM,
.ioctls = vmw_ioctls,
.num_ioctls = ARRAY_SIZE(vmw_ioctls),
.master_set = vmw_master_set,
@@ -1585,8 +1580,7 @@ static const struct drm_driver driver = {
.postclose = vmw_postclose,
.dumb_create = vmw_dumb_create,
- .dumb_map_offset = vmw_dumb_map_offset,
- .dumb_destroy = vmw_dumb_destroy,
+ .dumb_map_offset = drm_gem_ttm_dumb_map_offset,
.prime_fd_to_handle = vmw_prime_fd_to_handle,
.prime_handle_to_fd = vmw_prime_handle_to_fd,
@@ -1617,41 +1611,43 @@ static int vmw_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
ret = drm_aperture_remove_conflicting_pci_framebuffers(pdev, &driver);
if (ret)
- return ret;
+ goto out_error;
ret = pcim_enable_device(pdev);
if (ret)
- return ret;
+ goto out_error;
vmw = devm_drm_dev_alloc(&pdev->dev, &driver,
struct vmw_private, drm);
- if (IS_ERR(vmw))
- return PTR_ERR(vmw);
+ if (IS_ERR(vmw)) {
+ ret = PTR_ERR(vmw);
+ goto out_error;
+ }
pci_set_drvdata(pdev, &vmw->drm);
- ret = ttm_mem_global_init(&ttm_mem_glob, &pdev->dev);
- if (ret)
- return ret;
-
ret = vmw_driver_load(vmw, ent->device);
if (ret)
- return ret;
+ goto out_error;
ret = drm_dev_register(&vmw->drm, 0);
- if (ret) {
- vmw_driver_unload(&vmw->drm);
- return ret;
- }
+ if (ret)
+ goto out_unload;
+
+ vmw_debugfs_gem_init(vmw);
return 0;
+out_unload:
+ vmw_driver_unload(&vmw->drm);
+out_error:
+ return ret;
}
static int __init vmwgfx_init(void)
{
int ret;
- if (vgacon_text_force())
+ if (drm_firmware_drivers_only())
return -EINVAL;
ret = pci_register_driver(&vmw_pci_driver);
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_drv.h b/drivers/gpu/drm/vmwgfx/vmwgfx_drv.h
index 858aff9..d6b6663 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_drv.h
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_drv.h
@@ -34,7 +34,6 @@
#include <drm/drm_auth.h>
#include <drm/drm_device.h>
#include <drm/drm_file.h>
-#include <drm/drm_hashtab.h>
#include <drm/drm_rect.h>
#include <drm/ttm/ttm_bo_driver.h>
@@ -43,6 +42,7 @@
#include "ttm_object.h"
#include "vmwgfx_fence.h"
+#include "vmwgfx_hashtab.h"
#include "vmwgfx_reg.h"
#include "vmwgfx_validation.h"
@@ -54,16 +54,13 @@
#define VMWGFX_DRIVER_NAME "vmwgfx"
-#define VMWGFX_DRIVER_DATE "20210722"
+#define VMWGFX_DRIVER_DATE "20211206"
#define VMWGFX_DRIVER_MAJOR 2
-#define VMWGFX_DRIVER_MINOR 19
+#define VMWGFX_DRIVER_MINOR 20
#define VMWGFX_DRIVER_PATCHLEVEL 0
#define VMWGFX_FIFO_STATIC_SIZE (1024*1024)
-#define VMWGFX_MAX_RELOCATIONS 2048
-#define VMWGFX_MAX_VALIDATIONS 2048
#define VMWGFX_MAX_DISPLAYS 16
#define VMWGFX_CMD_BOUNCE_INIT_SIZE 32768
-#define VMWGFX_ENABLE_SCREEN_TARGET_OTABLE 1
#define VMWGFX_PCI_ID_SVGA2 0x0405
#define VMWGFX_PCI_ID_SVGA3 0x0406
@@ -82,8 +79,9 @@
VMWGFX_NUM_GB_SURFACE +\
VMWGFX_NUM_GB_SCREEN_TARGET)
-#define VMW_PL_GMR (TTM_PL_PRIV + 0)
-#define VMW_PL_MOB (TTM_PL_PRIV + 1)
+#define VMW_PL_GMR (TTM_PL_PRIV + 0)
+#define VMW_PL_MOB (TTM_PL_PRIV + 1)
+#define VMW_PL_SYSTEM (TTM_PL_PRIV + 2)
#define VMW_RES_CONTEXT ttm_driver_type0
#define VMW_RES_SURFACE ttm_driver_type1
@@ -133,7 +131,7 @@ struct vmw_buffer_object {
*/
struct vmw_validate_buffer {
struct ttm_validate_buffer base;
- struct drm_hash_item hash;
+ struct vmwgfx_hash_item hash;
bool validate_as_mob;
};
@@ -332,7 +330,6 @@ struct vmw_sg_table {
struct page **pages;
const dma_addr_t *addrs;
struct sg_table *sgt;
- unsigned long num_regions;
unsigned long num_pages;
};
@@ -360,6 +357,19 @@ struct vmw_piter {
dma_addr_t (*dma_address)(struct vmw_piter *);
};
+
+struct vmw_ttm_tt {
+ struct ttm_tt dma_ttm;
+ struct vmw_private *dev_priv;
+ int gmr_id;
+ struct vmw_mob *mob;
+ int mem_type;
+ struct sg_table sgt;
+ struct vmw_sg_table vsgt;
+ bool mapped;
+ bool bound;
+};
+
/*
* enum vmw_display_unit_type - Describes the display unit
*/
@@ -406,10 +416,11 @@ struct vmw_ctx_validation_info;
* @ctx: The validation context
*/
struct vmw_sw_context{
- struct drm_open_hash res_ht;
+ struct vmwgfx_open_hash res_ht;
bool res_ht_initialized;
bool kernel;
struct vmw_fpriv *fp;
+ struct drm_file *filp;
uint32_t *cmd_bounce;
uint32_t cmd_bounce_size;
struct vmw_buffer_object *cur_query_bo;
@@ -473,6 +484,7 @@ enum {
* @VMW_SM_4: Context support upto SM4.
* @VMW_SM_4_1: Context support upto SM4_1.
* @VMW_SM_5: Context support up to SM5.
+ * @VMW_SM_5_1X: Adds support for sm5_1 and gl43 extensions.
* @VMW_SM_MAX: Should be the last.
*/
enum vmw_sm_type {
@@ -480,6 +492,7 @@ enum vmw_sm_type {
VMW_SM_4,
VMW_SM_4_1,
VMW_SM_5,
+ VMW_SM_5_1X,
VMW_SM_MAX
};
@@ -627,9 +640,6 @@ struct vmw_private {
struct vmw_cmdbuf_man *cman;
DECLARE_BITMAP(irqthread_pending, VMW_IRQTHREAD_MAX);
- /* Validation memory reservation */
- struct vmw_validation_mem vvm;
-
uint32 *devcaps;
/*
@@ -645,6 +655,11 @@ struct vmw_private {
#endif
};
+static inline struct vmw_buffer_object *gem_to_vmw_bo(struct drm_gem_object *gobj)
+{
+ return container_of((gobj), struct vmw_buffer_object, base.base);
+}
+
static inline struct vmw_surface *vmw_res_to_srf(struct vmw_resource *res)
{
return container_of(res, struct vmw_surface, res);
@@ -738,6 +753,24 @@ static inline bool has_sm5_context(const struct vmw_private *dev_priv)
return (dev_priv->sm_type >= VMW_SM_5);
}
+/**
+ * has_gl43_context - Does the device support GL43 context.
+ * @dev_priv: Device private.
+ *
+ * Return: Bool value if device support SM5 context or not.
+ */
+static inline bool has_gl43_context(const struct vmw_private *dev_priv)
+{
+ return (dev_priv->sm_type >= VMW_SM_5_1X);
+}
+
+
+static inline u32 vmw_max_num_uavs(struct vmw_private *dev_priv)
+{
+ return (has_gl43_context(dev_priv) ?
+ SVGA3D_DX11_1_MAX_UAVIEWS : SVGA3D_MAX_UAVIEWS);
+}
+
extern void vmw_svga_enable(struct vmw_private *dev_priv);
extern void vmw_svga_disable(struct vmw_private *dev_priv);
@@ -767,7 +800,7 @@ extern int vmw_resource_reserve(struct vmw_resource *res, bool interruptible,
bool no_backup);
extern bool vmw_resource_needs_backup(const struct vmw_resource *res);
extern int vmw_user_lookup_handle(struct vmw_private *dev_priv,
- struct ttm_object_file *tfile,
+ struct drm_file *filp,
uint32_t handle,
struct vmw_surface **out_surf,
struct vmw_buffer_object **out_buf);
@@ -833,6 +866,7 @@ static inline void vmw_user_resource_noref_release(void)
/**
* Buffer object helper functions - vmwgfx_bo.c
*/
+extern bool vmw_bo_is_vmw_bo(struct ttm_buffer_object *bo);
extern int vmw_bo_pin_in_placement(struct vmw_private *vmw_priv,
struct vmw_buffer_object *bo,
struct ttm_placement *placement,
@@ -857,32 +891,23 @@ extern int vmw_bo_create_kernel(struct vmw_private *dev_priv,
unsigned long size,
struct ttm_placement *placement,
struct ttm_buffer_object **p_bo);
+extern int vmw_bo_create(struct vmw_private *dev_priv,
+ size_t size, struct ttm_placement *placement,
+ bool interruptible, bool pin,
+ void (*bo_free)(struct ttm_buffer_object *bo),
+ struct vmw_buffer_object **p_bo);
extern int vmw_bo_init(struct vmw_private *dev_priv,
struct vmw_buffer_object *vmw_bo,
size_t size, struct ttm_placement *placement,
bool interruptible, bool pin,
void (*bo_free)(struct ttm_buffer_object *bo));
-extern int vmw_user_bo_verify_access(struct ttm_buffer_object *bo,
- struct ttm_object_file *tfile);
-extern int vmw_user_bo_alloc(struct vmw_private *dev_priv,
- struct ttm_object_file *tfile,
- uint32_t size,
- bool shareable,
- uint32_t *handle,
- struct vmw_buffer_object **p_dma_buf,
- struct ttm_base_object **p_base);
-extern int vmw_user_bo_reference(struct ttm_object_file *tfile,
- struct vmw_buffer_object *dma_buf,
- uint32_t *handle);
-extern int vmw_bo_alloc_ioctl(struct drm_device *dev, void *data,
- struct drm_file *file_priv);
extern int vmw_bo_unref_ioctl(struct drm_device *dev, void *data,
struct drm_file *file_priv);
extern int vmw_user_bo_synccpu_ioctl(struct drm_device *dev, void *data,
struct drm_file *file_priv);
-extern int vmw_user_bo_lookup(struct ttm_object_file *tfile,
- uint32_t id, struct vmw_buffer_object **out,
- struct ttm_base_object **base);
+extern int vmw_user_bo_lookup(struct drm_file *filp,
+ uint32_t handle,
+ struct vmw_buffer_object **out);
extern void vmw_bo_fence_single(struct ttm_buffer_object *bo,
struct vmw_fence_obj *fence);
extern void *vmw_bo_map_and_cache(struct vmw_buffer_object *vbo);
@@ -891,16 +916,7 @@ extern void vmw_bo_move_notify(struct ttm_buffer_object *bo,
struct ttm_resource *mem);
extern void vmw_bo_swap_notify(struct ttm_buffer_object *bo);
extern struct vmw_buffer_object *
-vmw_user_bo_noref_lookup(struct ttm_object_file *tfile, u32 handle);
-
-/**
- * vmw_user_bo_noref_release - release a buffer object pointer looked up
- * without reference
- */
-static inline void vmw_user_bo_noref_release(void)
-{
- ttm_base_object_noref_release();
-}
+vmw_user_bo_noref_lookup(struct drm_file *filp, u32 handle);
/**
* vmw_bo_adjust_prio - Adjust the buffer object eviction priority
@@ -952,6 +968,19 @@ static inline void vmw_bo_prio_del(struct vmw_buffer_object *vbo, int prio)
}
/**
+ * GEM related functionality - vmwgfx_gem.c
+ */
+extern int vmw_gem_object_create_with_handle(struct vmw_private *dev_priv,
+ struct drm_file *filp,
+ uint32_t size,
+ uint32_t *handle,
+ struct vmw_buffer_object **p_vbo);
+extern int vmw_gem_object_create_ioctl(struct drm_device *dev, void *data,
+ struct drm_file *filp);
+extern void vmw_gem_destroy(struct ttm_buffer_object *bo);
+extern void vmw_debugfs_gem_init(struct vmw_private *vdev);
+
+/**
* Misc Ioctl functionality - vmwgfx_ioctl.c
*/
@@ -1027,9 +1056,6 @@ vmw_is_cursor_bypass3_enabled(const struct vmw_private *dev_priv)
extern int vmw_mmap(struct file *filp, struct vm_area_struct *vma);
-extern void vmw_validation_mem_init_ttm(struct vmw_private *dev_priv,
- size_t gran);
-
/**
* TTM buffer object driver - vmwgfx_ttm_buffer.c
*/
@@ -1039,7 +1065,6 @@ extern struct ttm_placement vmw_vram_placement;
extern struct ttm_placement vmw_vram_sys_placement;
extern struct ttm_placement vmw_vram_gmr_placement;
extern struct ttm_placement vmw_sys_placement;
-extern struct ttm_placement vmw_evictable_placement;
extern struct ttm_placement vmw_srf_placement;
extern struct ttm_placement vmw_mob_placement;
extern struct ttm_placement vmw_nonfixed_placement;
@@ -1218,13 +1243,6 @@ void vmw_kms_lost_device(struct drm_device *dev);
int vmw_dumb_create(struct drm_file *file_priv,
struct drm_device *dev,
struct drm_mode_create_dumb *args);
-
-int vmw_dumb_map_offset(struct drm_file *file_priv,
- struct drm_device *dev, uint32_t handle,
- uint64_t *offset);
-int vmw_dumb_destroy(struct drm_file *file_priv,
- struct drm_device *dev,
- uint32_t handle);
extern int vmw_resource_pin(struct vmw_resource *res, bool interruptible);
extern void vmw_resource_unpin(struct vmw_resource *res);
extern enum vmw_res_type vmw_res_type(const struct vmw_resource *res);
@@ -1252,6 +1270,12 @@ int vmw_gmrid_man_init(struct vmw_private *dev_priv, int type);
void vmw_gmrid_man_fini(struct vmw_private *dev_priv, int type);
/**
+ * System memory manager
+ */
+int vmw_sys_man_init(struct vmw_private *dev_priv);
+void vmw_sys_man_fini(struct vmw_private *dev_priv);
+
+/**
* Prime - vmwgfx_prime.c
*/
@@ -1322,18 +1346,6 @@ extern int vmw_gb_surface_define_ioctl(struct drm_device *dev, void *data,
struct drm_file *file_priv);
extern int vmw_gb_surface_reference_ioctl(struct drm_device *dev, void *data,
struct drm_file *file_priv);
-int vmw_surface_gb_priv_define(struct drm_device *dev,
- uint32_t user_accounting_size,
- SVGA3dSurfaceAllFlags svga3d_flags,
- SVGA3dSurfaceFormat format,
- bool for_scanout,
- uint32_t num_mip_levels,
- uint32_t multisample_count,
- uint32_t array_size,
- struct drm_vmw_size size,
- SVGA3dMSPattern multisample_pattern,
- SVGA3dMSQualityLevel quality_level,
- struct vmw_surface **srf_out);
extern int vmw_gb_surface_define_ext_ioctl(struct drm_device *dev,
void *data,
struct drm_file *file_priv);
@@ -1342,7 +1354,6 @@ extern int vmw_gb_surface_reference_ext_ioctl(struct drm_device *dev,
struct drm_file *file_priv);
int vmw_gb_surface_define(struct vmw_private *dev_priv,
- uint32_t user_accounting_size,
const struct vmw_surface_metadata *req,
struct vmw_surface **srf_out);
@@ -1403,7 +1414,6 @@ void vmw_dx_streamoutput_cotable_list_scrub(struct vmw_private *dev_priv,
extern struct vmw_cmdbuf_res_manager *
vmw_cmdbuf_res_man_create(struct vmw_private *dev_priv);
extern void vmw_cmdbuf_res_man_destroy(struct vmw_cmdbuf_res_manager *man);
-extern size_t vmw_cmdbuf_res_man_size(void);
extern struct vmw_resource *
vmw_cmdbuf_res_lookup(struct vmw_cmdbuf_res_manager *man,
enum vmw_cmdbuf_res_type res_type,
@@ -1551,11 +1561,6 @@ void vmw_bo_dirty_unmap(struct vmw_buffer_object *vbo,
vm_fault_t vmw_bo_vm_fault(struct vm_fault *vmf);
vm_fault_t vmw_bo_vm_mkwrite(struct vm_fault *vmf);
-/* Transparent hugepage support - vmwgfx_thp.c */
-#ifdef CONFIG_TRANSPARENT_HUGEPAGE
-extern int vmw_thp_init(struct vmw_private *dev_priv);
-void vmw_thp_fini(struct vmw_private *dev_priv);
-#endif
/**
* VMW_DEBUG_KMS - Debug output for kernel mode-setting
@@ -1600,11 +1605,6 @@ vmw_bo_reference(struct vmw_buffer_object *buf)
return buf;
}
-static inline struct ttm_mem_global *vmw_mem_glob(struct vmw_private *dev_priv)
-{
- return &ttm_mem_glob;
-}
-
static inline void vmw_fifo_resource_inc(struct vmw_private *dev_priv)
{
atomic_inc(&dev_priv->num_fifo_resources);
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_execbuf.c b/drivers/gpu/drm/vmwgfx/vmwgfx_execbuf.c
index 5f2ffa9..44ca23b 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_execbuf.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_execbuf.c
@@ -1171,14 +1171,13 @@ static int vmw_translate_mob_ptr(struct vmw_private *dev_priv,
int ret;
vmw_validation_preload_bo(sw_context->ctx);
- vmw_bo = vmw_user_bo_noref_lookup(sw_context->fp->tfile, handle);
- if (IS_ERR(vmw_bo)) {
+ vmw_bo = vmw_user_bo_noref_lookup(sw_context->filp, handle);
+ if (IS_ERR_OR_NULL(vmw_bo)) {
VMW_DEBUG_USER("Could not find or use MOB buffer.\n");
return PTR_ERR(vmw_bo);
}
-
ret = vmw_validation_add_bo(sw_context->ctx, vmw_bo, true, false);
- vmw_user_bo_noref_release();
+ ttm_bo_put(&vmw_bo->base);
if (unlikely(ret != 0))
return ret;
@@ -1226,14 +1225,13 @@ static int vmw_translate_guest_ptr(struct vmw_private *dev_priv,
int ret;
vmw_validation_preload_bo(sw_context->ctx);
- vmw_bo = vmw_user_bo_noref_lookup(sw_context->fp->tfile, handle);
- if (IS_ERR(vmw_bo)) {
+ vmw_bo = vmw_user_bo_noref_lookup(sw_context->filp, handle);
+ if (IS_ERR_OR_NULL(vmw_bo)) {
VMW_DEBUG_USER("Could not find or use GMR region.\n");
return PTR_ERR(vmw_bo);
}
-
ret = vmw_validation_add_bo(sw_context->ctx, vmw_bo, false, false);
- vmw_user_bo_noref_release();
+ ttm_bo_put(&vmw_bo->base);
if (unlikely(ret != 0))
return ret;
@@ -2166,6 +2164,44 @@ vmw_cmd_dx_set_single_constant_buffer(struct vmw_private *dev_priv,
}
/**
+ * vmw_cmd_dx_set_constant_buffer_offset - Validate
+ * SVGA_3D_CMD_DX_SET_VS/PS/GS/HS/DS/CS_CONSTANT_BUFFER_OFFSET command.
+ *
+ * @dev_priv: Pointer to a device private struct.
+ * @sw_context: The software context being used for this batch.
+ * @header: Pointer to the command header in the command stream.
+ */
+static int
+vmw_cmd_dx_set_constant_buffer_offset(struct vmw_private *dev_priv,
+ struct vmw_sw_context *sw_context,
+ SVGA3dCmdHeader *header)
+{
+ VMW_DECLARE_CMD_VAR(*cmd, SVGA3dCmdDXSetConstantBufferOffset);
+
+ struct vmw_ctx_validation_info *ctx_node = VMW_GET_CTX_NODE(sw_context);
+ u32 shader_slot;
+
+ if (!has_sm5_context(dev_priv))
+ return -EINVAL;
+
+ if (!ctx_node)
+ return -EINVAL;
+
+ cmd = container_of(header, typeof(*cmd), header);
+ if (cmd->body.slot >= SVGA3D_DX_MAX_CONSTBUFFERS) {
+ VMW_DEBUG_USER("Illegal const buffer slot %u.\n",
+ (unsigned int) cmd->body.slot);
+ return -EINVAL;
+ }
+
+ shader_slot = cmd->header.id - SVGA_3D_CMD_DX_SET_VS_CONSTANT_BUFFER_OFFSET;
+ vmw_binding_cb_offset_update(ctx_node->staged, shader_slot,
+ cmd->body.slot, cmd->body.offsetInBytes);
+
+ return 0;
+}
+
+/**
* vmw_cmd_dx_set_shader_res - Validate SVGA_3D_CMD_DX_SET_SHADER_RESOURCES
* command
*
@@ -2918,7 +2954,7 @@ static int vmw_cmd_set_uav(struct vmw_private *dev_priv,
if (!has_sm5_context(dev_priv))
return -EINVAL;
- if (num_uav > SVGA3D_MAX_UAVIEWS) {
+ if (num_uav > vmw_max_num_uavs(dev_priv)) {
VMW_DEBUG_USER("Invalid UAV binding.\n");
return -EINVAL;
}
@@ -2950,7 +2986,7 @@ static int vmw_cmd_set_cs_uav(struct vmw_private *dev_priv,
if (!has_sm5_context(dev_priv))
return -EINVAL;
- if (num_uav > SVGA3D_MAX_UAVIEWS) {
+ if (num_uav > vmw_max_num_uavs(dev_priv)) {
VMW_DEBUG_USER("Invalid UAV binding.\n");
return -EINVAL;
}
@@ -3528,6 +3564,24 @@ static const struct vmw_cmd_entry vmw_cmd_entries[SVGA_3D_CMD_MAX] = {
VMW_CMD_DEF(SVGA_3D_CMD_DX_TRANSFER_FROM_BUFFER,
&vmw_cmd_dx_transfer_from_buffer,
true, false, true),
+ VMW_CMD_DEF(SVGA_3D_CMD_DX_SET_VS_CONSTANT_BUFFER_OFFSET,
+ &vmw_cmd_dx_set_constant_buffer_offset,
+ true, false, true),
+ VMW_CMD_DEF(SVGA_3D_CMD_DX_SET_PS_CONSTANT_BUFFER_OFFSET,
+ &vmw_cmd_dx_set_constant_buffer_offset,
+ true, false, true),
+ VMW_CMD_DEF(SVGA_3D_CMD_DX_SET_GS_CONSTANT_BUFFER_OFFSET,
+ &vmw_cmd_dx_set_constant_buffer_offset,
+ true, false, true),
+ VMW_CMD_DEF(SVGA_3D_CMD_DX_SET_HS_CONSTANT_BUFFER_OFFSET,
+ &vmw_cmd_dx_set_constant_buffer_offset,
+ true, false, true),
+ VMW_CMD_DEF(SVGA_3D_CMD_DX_SET_DS_CONSTANT_BUFFER_OFFSET,
+ &vmw_cmd_dx_set_constant_buffer_offset,
+ true, false, true),
+ VMW_CMD_DEF(SVGA_3D_CMD_DX_SET_CS_CONSTANT_BUFFER_OFFSET,
+ &vmw_cmd_dx_set_constant_buffer_offset,
+ true, false, true),
VMW_CMD_DEF(SVGA_3D_CMD_INTRA_SURFACE_COPY, &vmw_cmd_intra_surface_copy,
true, false, true),
@@ -3561,6 +3615,8 @@ static const struct vmw_cmd_entry vmw_cmd_entries[SVGA_3D_CMD_MAX] = {
&vmw_cmd_dx_define_streamoutput, true, false, true),
VMW_CMD_DEF(SVGA_3D_CMD_DX_BIND_STREAMOUTPUT,
&vmw_cmd_dx_bind_streamoutput, true, false, true),
+ VMW_CMD_DEF(SVGA_3D_CMD_DX_DEFINE_RASTERIZER_STATE_V2,
+ &vmw_cmd_dx_so_define, true, false, true),
};
bool vmw_cmd_describe(const void *buf, u32 *size, char const **cmd)
@@ -3869,8 +3925,7 @@ vmw_execbuf_copy_fence_user(struct vmw_private *dev_priv,
fence_rep.fd = -1;
}
- ttm_ref_object_base_unref(vmw_fp->tfile, fence_handle,
- TTM_REF_USAGE);
+ ttm_ref_object_base_unref(vmw_fp->tfile, fence_handle);
VMW_DEBUG_USER("Fence copy error. Syncing.\n");
(void) vmw_fence_obj_wait(fence, false, false,
VMW_FENCE_WAIT_TIMEOUT);
@@ -4054,8 +4109,6 @@ int vmw_execbuf_process(struct drm_file *file_priv,
struct sync_file *sync_file = NULL;
DECLARE_VAL_CONTEXT(val_ctx, &sw_context->res_ht, 1);
- vmw_validation_set_val_mem(&val_ctx, &dev_priv->vvm);
-
if (flags & DRM_VMW_EXECBUF_FLAG_EXPORT_FENCE_FD) {
out_fence_fd = get_unused_fd_flags(O_CLOEXEC);
if (out_fence_fd < 0) {
@@ -4101,6 +4154,7 @@ int vmw_execbuf_process(struct drm_file *file_priv,
sw_context->kernel = true;
}
+ sw_context->filp = file_priv;
sw_context->fp = vmw_fpriv(file_priv);
INIT_LIST_HEAD(&sw_context->ctx_list);
sw_context->cur_query_bo = dev_priv->pinned_bo;
@@ -4117,7 +4171,7 @@ int vmw_execbuf_process(struct drm_file *file_priv,
vmw_binding_state_reset(sw_context->staged_bindings);
if (!sw_context->res_ht_initialized) {
- ret = drm_ht_create(&sw_context->res_ht, VMW_RES_HT_ORDER);
+ ret = vmwgfx_ht_create(&sw_context->res_ht, VMW_RES_HT_ORDER);
if (unlikely(ret != 0))
goto out_unlock;
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_fb.c b/drivers/gpu/drm/vmwgfx/vmwgfx_fb.c
index d18c6a5..8ee3457 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_fb.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_fb.c
@@ -394,22 +394,15 @@ static int vmw_fb_create_bo(struct vmw_private *vmw_priv,
struct vmw_buffer_object *vmw_bo;
int ret;
- vmw_bo = kmalloc(sizeof(*vmw_bo), GFP_KERNEL);
- if (!vmw_bo) {
- ret = -ENOMEM;
- goto err_unlock;
- }
-
- ret = vmw_bo_init(vmw_priv, vmw_bo, size,
+ ret = vmw_bo_create(vmw_priv, size,
&vmw_sys_placement,
false, false,
- &vmw_bo_bo_free);
+ &vmw_bo_bo_free, &vmw_bo);
if (unlikely(ret != 0))
- goto err_unlock; /* init frees the buffer on failure */
+ return ret;
*out = vmw_bo;
-err_unlock:
return ret;
}
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_fence.c b/drivers/gpu/drm/vmwgfx/vmwgfx_fence.c
index 9fe1232..c60d395 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_fence.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_fence.c
@@ -37,9 +37,6 @@ struct vmw_fence_manager {
spinlock_t lock;
struct list_head fence_list;
struct work_struct work;
- u32 user_fence_size;
- u32 fence_size;
- u32 event_fence_action_size;
bool fifo_down;
struct list_head cleanup_list;
uint32_t pending_actions[VMW_ACTION_MAX];
@@ -304,11 +301,6 @@ struct vmw_fence_manager *vmw_fence_manager_init(struct vmw_private *dev_priv)
INIT_LIST_HEAD(&fman->cleanup_list);
INIT_WORK(&fman->work, &vmw_fence_work_func);
fman->fifo_down = true;
- fman->user_fence_size = ttm_round_pot(sizeof(struct vmw_user_fence)) +
- TTM_OBJ_EXTRA_SIZE;
- fman->fence_size = ttm_round_pot(sizeof(struct vmw_fence_obj));
- fman->event_fence_action_size =
- ttm_round_pot(sizeof(struct vmw_event_fence_action));
mutex_init(&fman->goal_irq_mutex);
fman->ctx = dma_fence_context_alloc(1);
@@ -560,14 +552,8 @@ static void vmw_user_fence_destroy(struct vmw_fence_obj *fence)
{
struct vmw_user_fence *ufence =
container_of(fence, struct vmw_user_fence, fence);
- struct vmw_fence_manager *fman = fman_from_fence(fence);
ttm_base_object_kfree(ufence, base);
- /*
- * Free kernel space accounting.
- */
- ttm_mem_global_free(vmw_mem_glob(fman->dev_priv),
- fman->user_fence_size);
}
static void vmw_user_fence_base_release(struct ttm_base_object **p_base)
@@ -590,23 +576,8 @@ int vmw_user_fence_create(struct drm_file *file_priv,
struct ttm_object_file *tfile = vmw_fpriv(file_priv)->tfile;
struct vmw_user_fence *ufence;
struct vmw_fence_obj *tmp;
- struct ttm_mem_global *mem_glob = vmw_mem_glob(fman->dev_priv);
- struct ttm_operation_ctx ctx = {
- .interruptible = false,
- .no_wait_gpu = false
- };
int ret;
- /*
- * Kernel memory space accounting, since this object may
- * be created by a user-space request.
- */
-
- ret = ttm_mem_global_alloc(mem_glob, fman->user_fence_size,
- &ctx);
- if (unlikely(ret != 0))
- return ret;
-
ufence = kzalloc(sizeof(*ufence), GFP_KERNEL);
if (unlikely(!ufence)) {
ret = -ENOMEM;
@@ -625,9 +596,10 @@ int vmw_user_fence_create(struct drm_file *file_priv,
* vmw_user_fence_base_release.
*/
tmp = vmw_fence_obj_reference(&ufence->fence);
+
ret = ttm_base_object_init(tfile, &ufence->base, false,
VMW_RES_FENCE,
- &vmw_user_fence_base_release, NULL);
+ &vmw_user_fence_base_release);
if (unlikely(ret != 0)) {
@@ -646,7 +618,6 @@ int vmw_user_fence_create(struct drm_file *file_priv,
tmp = &ufence->fence;
vmw_fence_obj_unreference(&tmp);
out_no_object:
- ttm_mem_global_free(mem_glob, fman->user_fence_size);
return ret;
}
@@ -831,8 +802,7 @@ int vmw_fence_obj_wait_ioctl(struct drm_device *dev, void *data,
*/
if (ret == 0 && (arg->wait_options & DRM_VMW_WAIT_OPTION_UNREF))
- return ttm_ref_object_base_unref(tfile, arg->handle,
- TTM_REF_USAGE);
+ return ttm_ref_object_base_unref(tfile, arg->handle);
return ret;
}
@@ -874,8 +844,7 @@ int vmw_fence_obj_unref_ioctl(struct drm_device *dev, void *data,
(struct drm_vmw_fence_arg *) data;
return ttm_ref_object_base_unref(vmw_fpriv(file_priv)->tfile,
- arg->handle,
- TTM_REF_USAGE);
+ arg->handle);
}
/**
@@ -1121,7 +1090,7 @@ int vmw_fence_event_ioctl(struct drm_device *dev, void *data,
if (user_fence_rep != NULL) {
ret = ttm_ref_object_add(vmw_fp->tfile, base,
- TTM_REF_USAGE, NULL, false);
+ NULL, false);
if (unlikely(ret != 0)) {
DRM_ERROR("Failed to reference a fence "
"object.\n");
@@ -1164,7 +1133,7 @@ int vmw_fence_event_ioctl(struct drm_device *dev, void *data,
return 0;
out_no_create:
if (user_fence_rep != NULL)
- ttm_ref_object_base_unref(tfile, handle, TTM_REF_USAGE);
+ ttm_ref_object_base_unref(tfile, handle);
out_no_ref_obj:
vmw_fence_obj_unreference(&fence);
return ret;
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_gem.c b/drivers/gpu/drm/vmwgfx/vmwgfx_gem.c
new file mode 100644
index 0000000..ce609e7
--- /dev/null
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_gem.c
@@ -0,0 +1,294 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/*
+ * Copyright 2021 VMware, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person
+ * obtaining a copy of this software and associated documentation
+ * files (the "Software"), to deal in the Software without
+ * restriction, including without limitation the rights to use, copy,
+ * modify, merge, publish, distribute, sublicense, and/or sell copies
+ * of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be
+ * included in all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ *
+ */
+
+#include "vmwgfx_drv.h"
+
+#include "drm/drm_prime.h"
+#include "drm/drm_gem_ttm_helper.h"
+
+/**
+ * vmw_buffer_object - Convert a struct ttm_buffer_object to a struct
+ * vmw_buffer_object.
+ *
+ * @bo: Pointer to the TTM buffer object.
+ * Return: Pointer to the struct vmw_buffer_object embedding the
+ * TTM buffer object.
+ */
+static struct vmw_buffer_object *
+vmw_buffer_object(struct ttm_buffer_object *bo)
+{
+ return container_of(bo, struct vmw_buffer_object, base);
+}
+
+static void vmw_gem_object_free(struct drm_gem_object *gobj)
+{
+ struct ttm_buffer_object *bo = drm_gem_ttm_of_gem(gobj);
+ if (bo) {
+ ttm_bo_put(bo);
+ }
+}
+
+static int vmw_gem_object_open(struct drm_gem_object *obj,
+ struct drm_file *file_priv)
+{
+ return 0;
+}
+
+static void vmw_gem_object_close(struct drm_gem_object *obj,
+ struct drm_file *file_priv)
+{
+}
+
+static int vmw_gem_pin_private(struct drm_gem_object *obj, bool do_pin)
+{
+ struct ttm_buffer_object *bo = drm_gem_ttm_of_gem(obj);
+ struct vmw_buffer_object *vbo = vmw_buffer_object(bo);
+ int ret;
+
+ ret = ttm_bo_reserve(bo, false, false, NULL);
+ if (unlikely(ret != 0))
+ goto err;
+
+ vmw_bo_pin_reserved(vbo, do_pin);
+
+ ttm_bo_unreserve(bo);
+
+err:
+ return ret;
+}
+
+
+static int vmw_gem_object_pin(struct drm_gem_object *obj)
+{
+ return vmw_gem_pin_private(obj, true);
+}
+
+static void vmw_gem_object_unpin(struct drm_gem_object *obj)
+{
+ vmw_gem_pin_private(obj, false);
+}
+
+static struct sg_table *vmw_gem_object_get_sg_table(struct drm_gem_object *obj)
+{
+ struct ttm_buffer_object *bo = drm_gem_ttm_of_gem(obj);
+ struct vmw_ttm_tt *vmw_tt =
+ container_of(bo->ttm, struct vmw_ttm_tt, dma_ttm);
+
+ if (vmw_tt->vsgt.sgt)
+ return vmw_tt->vsgt.sgt;
+
+ return drm_prime_pages_to_sg(obj->dev, vmw_tt->dma_ttm.pages, vmw_tt->dma_ttm.num_pages);
+}
+
+
+static const struct drm_gem_object_funcs vmw_gem_object_funcs = {
+ .free = vmw_gem_object_free,
+ .open = vmw_gem_object_open,
+ .close = vmw_gem_object_close,
+ .print_info = drm_gem_ttm_print_info,
+ .pin = vmw_gem_object_pin,
+ .unpin = vmw_gem_object_unpin,
+ .get_sg_table = vmw_gem_object_get_sg_table,
+ .vmap = drm_gem_ttm_vmap,
+ .vunmap = drm_gem_ttm_vunmap,
+ .mmap = drm_gem_ttm_mmap,
+};
+
+/**
+ * vmw_gem_destroy - vmw buffer object destructor
+ *
+ * @bo: Pointer to the embedded struct ttm_buffer_object
+ */
+void vmw_gem_destroy(struct ttm_buffer_object *bo)
+{
+ struct vmw_buffer_object *vbo = vmw_buffer_object(bo);
+
+ WARN_ON(vbo->dirty);
+ WARN_ON(!RB_EMPTY_ROOT(&vbo->res_tree));
+ vmw_bo_unmap(vbo);
+ drm_gem_object_release(&vbo->base.base);
+ kfree(vbo);
+}
+
+int vmw_gem_object_create_with_handle(struct vmw_private *dev_priv,
+ struct drm_file *filp,
+ uint32_t size,
+ uint32_t *handle,
+ struct vmw_buffer_object **p_vbo)
+{
+ int ret;
+
+ ret = vmw_bo_create(dev_priv, size,
+ (dev_priv->has_mob) ?
+ &vmw_sys_placement :
+ &vmw_vram_sys_placement,
+ true, false, &vmw_gem_destroy, p_vbo);
+
+ (*p_vbo)->base.base.funcs = &vmw_gem_object_funcs;
+ if (ret != 0)
+ goto out_no_bo;
+
+ ret = drm_gem_handle_create(filp, &(*p_vbo)->base.base, handle);
+ /* drop reference from allocate - handle holds it now */
+ drm_gem_object_put(&(*p_vbo)->base.base);
+out_no_bo:
+ return ret;
+}
+
+
+int vmw_gem_object_create_ioctl(struct drm_device *dev, void *data,
+ struct drm_file *filp)
+{
+ struct vmw_private *dev_priv = vmw_priv(dev);
+ union drm_vmw_alloc_dmabuf_arg *arg =
+ (union drm_vmw_alloc_dmabuf_arg *)data;
+ struct drm_vmw_alloc_dmabuf_req *req = &arg->req;
+ struct drm_vmw_dmabuf_rep *rep = &arg->rep;
+ struct vmw_buffer_object *vbo;
+ uint32_t handle;
+ int ret;
+
+ ret = vmw_gem_object_create_with_handle(dev_priv, filp,
+ req->size, &handle, &vbo);
+ if (ret)
+ goto out_no_bo;
+
+ rep->handle = handle;
+ rep->map_handle = drm_vma_node_offset_addr(&vbo->base.base.vma_node);
+ rep->cur_gmr_id = handle;
+ rep->cur_gmr_offset = 0;
+out_no_bo:
+ return ret;
+}
+
+#if defined(CONFIG_DEBUG_FS)
+
+static void vmw_bo_print_info(int id, struct vmw_buffer_object *bo, struct seq_file *m)
+{
+ const char *placement;
+ const char *type;
+
+ switch (bo->base.resource->mem_type) {
+ case TTM_PL_SYSTEM:
+ placement = " CPU";
+ break;
+ case VMW_PL_GMR:
+ placement = " GMR";
+ break;
+ case VMW_PL_MOB:
+ placement = " MOB";
+ break;
+ case VMW_PL_SYSTEM:
+ placement = "VCPU";
+ break;
+ case TTM_PL_VRAM:
+ placement = "VRAM";
+ break;
+ default:
+ placement = "None";
+ break;
+ }
+
+ switch (bo->base.type) {
+ case ttm_bo_type_device:
+ type = "device";
+ break;
+ case ttm_bo_type_kernel:
+ type = "kernel";
+ break;
+ case ttm_bo_type_sg:
+ type = "sg ";
+ break;
+ default:
+ type = "none ";
+ break;
+ }
+
+ seq_printf(m, "\t\t0x%08x: %12zu bytes %s, type = %s",
+ id, bo->base.base.size, placement, type);
+ seq_printf(m, ", priority = %u, pin_count = %u, GEM refs = %d, TTM refs = %d",
+ bo->base.priority,
+ bo->base.pin_count,
+ kref_read(&bo->base.base.refcount),
+ kref_read(&bo->base.kref));
+ seq_puts(m, "\n");
+}
+
+static int vmw_debugfs_gem_info_show(struct seq_file *m, void *unused)
+{
+ struct vmw_private *vdev = (struct vmw_private *)m->private;
+ struct drm_device *dev = &vdev->drm;
+ struct drm_file *file;
+ int r;
+
+ r = mutex_lock_interruptible(&dev->filelist_mutex);
+ if (r)
+ return r;
+
+ list_for_each_entry(file, &dev->filelist, lhead) {
+ struct task_struct *task;
+ struct drm_gem_object *gobj;
+ int id;
+
+ /*
+ * Although we have a valid reference on file->pid, that does
+ * not guarantee that the task_struct who called get_pid() is
+ * still alive (e.g. get_pid(current) => fork() => exit()).
+ * Therefore, we need to protect this ->comm access using RCU.
+ */
+ rcu_read_lock();
+ task = pid_task(file->pid, PIDTYPE_PID);
+ seq_printf(m, "pid %8d command %s:\n", pid_nr(file->pid),
+ task ? task->comm : "<unknown>");
+ rcu_read_unlock();
+
+ spin_lock(&file->table_lock);
+ idr_for_each_entry(&file->object_idr, gobj, id) {
+ struct vmw_buffer_object *bo = gem_to_vmw_bo(gobj);
+
+ vmw_bo_print_info(id, bo, m);
+ }
+ spin_unlock(&file->table_lock);
+ }
+
+ mutex_unlock(&dev->filelist_mutex);
+ return 0;
+}
+
+DEFINE_SHOW_ATTRIBUTE(vmw_debugfs_gem_info);
+
+#endif
+
+void vmw_debugfs_gem_init(struct vmw_private *vdev)
+{
+#if defined(CONFIG_DEBUG_FS)
+ struct drm_minor *minor = vdev->drm.primary;
+ struct dentry *root = minor->debugfs_root;
+
+ debugfs_create_file("vmwgfx_gem_info", 0444, root, vdev,
+ &vmw_debugfs_gem_info_fops);
+#endif
+}
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_gmrid_manager.c b/drivers/gpu/drm/vmwgfx/vmwgfx_gmrid_manager.c
index b2c4af3..ebb4505 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_gmrid_manager.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_gmrid_manager.c
@@ -42,6 +42,7 @@ struct vmwgfx_gmrid_man {
uint32_t max_gmr_ids;
uint32_t max_gmr_pages;
uint32_t used_gmr_pages;
+ uint8_t type;
};
static struct vmwgfx_gmrid_man *to_gmrid_manager(struct ttm_resource_manager *man)
@@ -132,6 +133,18 @@ static void vmw_gmrid_man_put_node(struct ttm_resource_manager *man,
kfree(res);
}
+static void vmw_gmrid_man_debug(struct ttm_resource_manager *man,
+ struct drm_printer *printer)
+{
+ struct vmwgfx_gmrid_man *gman = to_gmrid_manager(man);
+
+ BUG_ON(gman->type != VMW_PL_GMR && gman->type != VMW_PL_MOB);
+
+ drm_printf(printer, "%s's used: %u pages, max: %u pages, %u id's\n",
+ (gman->type == VMW_PL_MOB) ? "Mob" : "GMR",
+ gman->used_gmr_pages, gman->max_gmr_pages, gman->max_gmr_ids);
+}
+
static const struct ttm_resource_manager_func vmw_gmrid_manager_func;
int vmw_gmrid_man_init(struct vmw_private *dev_priv, int type)
@@ -146,12 +159,12 @@ int vmw_gmrid_man_init(struct vmw_private *dev_priv, int type)
man = &gman->manager;
man->func = &vmw_gmrid_manager_func;
- /* TODO: This is most likely not correct */
man->use_tt = true;
ttm_resource_manager_init(man, 0);
spin_lock_init(&gman->lock);
gman->used_gmr_pages = 0;
ida_init(&gman->gmr_ida);
+ gman->type = type;
switch (type) {
case VMW_PL_GMR:
@@ -190,4 +203,5 @@ void vmw_gmrid_man_fini(struct vmw_private *dev_priv, int type)
static const struct ttm_resource_manager_func vmw_gmrid_manager_func = {
.alloc = vmw_gmrid_man_get_node,
.free = vmw_gmrid_man_put_node,
+ .debug = vmw_gmrid_man_debug
};
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_hashtab.c b/drivers/gpu/drm/vmwgfx/vmwgfx_hashtab.c
new file mode 100644
index 0000000..06aebc1
--- /dev/null
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_hashtab.c
@@ -0,0 +1,199 @@
+/*
+ * Copyright 2006 Tungsten Graphics, Inc., Bismarck, ND. USA.
+ * All Rights Reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the
+ * "Software"), to deal in the Software without restriction, including
+ * without limitation the rights to use, copy, modify, merge, publish,
+ * distribute, sub license, and/or sell copies of the Software, and to
+ * permit persons to whom the Software is furnished to do so, subject to
+ * the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the
+ * next paragraph) shall be included in all copies or substantial portions
+ * of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDERS, AUTHORS AND/OR ITS SUPPLIERS BE LIABLE FOR ANY CLAIM,
+ * DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
+ * OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
+ * USE OR OTHER DEALINGS IN THE SOFTWARE.
+ */
+
+/*
+ * Simple open hash tab implementation.
+ *
+ * Authors:
+ * Thomas Hellström <thomas-at-tungstengraphics-dot-com>
+ */
+
+#include <linux/export.h>
+#include <linux/hash.h>
+#include <linux/mm.h>
+#include <linux/rculist.h>
+#include <linux/slab.h>
+#include <linux/vmalloc.h>
+
+#include <drm/drm_print.h>
+
+#include "vmwgfx_hashtab.h"
+
+int vmwgfx_ht_create(struct vmwgfx_open_hash *ht, unsigned int order)
+{
+ unsigned int size = 1 << order;
+
+ ht->order = order;
+ ht->table = NULL;
+ if (size <= PAGE_SIZE / sizeof(*ht->table))
+ ht->table = kcalloc(size, sizeof(*ht->table), GFP_KERNEL);
+ else
+ ht->table = vzalloc(array_size(size, sizeof(*ht->table)));
+ if (!ht->table) {
+ DRM_ERROR("Out of memory for hash table\n");
+ return -ENOMEM;
+ }
+ return 0;
+}
+
+void vmwgfx_ht_verbose_list(struct vmwgfx_open_hash *ht, unsigned long key)
+{
+ struct vmwgfx_hash_item *entry;
+ struct hlist_head *h_list;
+ unsigned int hashed_key;
+ int count = 0;
+
+ hashed_key = hash_long(key, ht->order);
+ DRM_DEBUG("Key is 0x%08lx, Hashed key is 0x%08x\n", key, hashed_key);
+ h_list = &ht->table[hashed_key];
+ hlist_for_each_entry(entry, h_list, head)
+ DRM_DEBUG("count %d, key: 0x%08lx\n", count++, entry->key);
+}
+
+static struct hlist_node *vmwgfx_ht_find_key(struct vmwgfx_open_hash *ht, unsigned long key)
+{
+ struct vmwgfx_hash_item *entry;
+ struct hlist_head *h_list;
+ unsigned int hashed_key;
+
+ hashed_key = hash_long(key, ht->order);
+ h_list = &ht->table[hashed_key];
+ hlist_for_each_entry(entry, h_list, head) {
+ if (entry->key == key)
+ return &entry->head;
+ if (entry->key > key)
+ break;
+ }
+ return NULL;
+}
+
+static struct hlist_node *vmwgfx_ht_find_key_rcu(struct vmwgfx_open_hash *ht, unsigned long key)
+{
+ struct vmwgfx_hash_item *entry;
+ struct hlist_head *h_list;
+ unsigned int hashed_key;
+
+ hashed_key = hash_long(key, ht->order);
+ h_list = &ht->table[hashed_key];
+ hlist_for_each_entry_rcu(entry, h_list, head) {
+ if (entry->key == key)
+ return &entry->head;
+ if (entry->key > key)
+ break;
+ }
+ return NULL;
+}
+
+int vmwgfx_ht_insert_item(struct vmwgfx_open_hash *ht, struct vmwgfx_hash_item *item)
+{
+ struct vmwgfx_hash_item *entry;
+ struct hlist_head *h_list;
+ struct hlist_node *parent;
+ unsigned int hashed_key;
+ unsigned long key = item->key;
+
+ hashed_key = hash_long(key, ht->order);
+ h_list = &ht->table[hashed_key];
+ parent = NULL;
+ hlist_for_each_entry(entry, h_list, head) {
+ if (entry->key == key)
+ return -EINVAL;
+ if (entry->key > key)
+ break;
+ parent = &entry->head;
+ }
+ if (parent)
+ hlist_add_behind_rcu(&item->head, parent);
+ else
+ hlist_add_head_rcu(&item->head, h_list);
+ return 0;
+}
+
+/*
+ * Just insert an item and return any "bits" bit key that hasn't been
+ * used before.
+ */
+int vmwgfx_ht_just_insert_please(struct vmwgfx_open_hash *ht, struct vmwgfx_hash_item *item,
+ unsigned long seed, int bits, int shift,
+ unsigned long add)
+{
+ int ret;
+ unsigned long mask = (1UL << bits) - 1;
+ unsigned long first, unshifted_key;
+
+ unshifted_key = hash_long(seed, bits);
+ first = unshifted_key;
+ do {
+ item->key = (unshifted_key << shift) + add;
+ ret = vmwgfx_ht_insert_item(ht, item);
+ if (ret)
+ unshifted_key = (unshifted_key + 1) & mask;
+ } while (ret && (unshifted_key != first));
+
+ if (ret) {
+ DRM_ERROR("Available key bit space exhausted\n");
+ return -EINVAL;
+ }
+ return 0;
+}
+
+int vmwgfx_ht_find_item(struct vmwgfx_open_hash *ht, unsigned long key,
+ struct vmwgfx_hash_item **item)
+{
+ struct hlist_node *list;
+
+ list = vmwgfx_ht_find_key_rcu(ht, key);
+ if (!list)
+ return -EINVAL;
+
+ *item = hlist_entry(list, struct vmwgfx_hash_item, head);
+ return 0;
+}
+
+int vmwgfx_ht_remove_key(struct vmwgfx_open_hash *ht, unsigned long key)
+{
+ struct hlist_node *list;
+
+ list = vmwgfx_ht_find_key(ht, key);
+ if (list) {
+ hlist_del_init_rcu(list);
+ return 0;
+ }
+ return -EINVAL;
+}
+
+int vmwgfx_ht_remove_item(struct vmwgfx_open_hash *ht, struct vmwgfx_hash_item *item)
+{
+ hlist_del_init_rcu(&item->head);
+ return 0;
+}
+
+void vmwgfx_ht_remove(struct vmwgfx_open_hash *ht)
+{
+ if (ht->table) {
+ kvfree(ht->table);
+ ht->table = NULL;
+ }
+}
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_hashtab.h b/drivers/gpu/drm/vmwgfx/vmwgfx_hashtab.h
new file mode 100644
index 0000000..a9ce129
--- /dev/null
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_hashtab.h
@@ -0,0 +1,83 @@
+/*
+ * Copyright 2006 Tungsten Graphics, Inc., Bismack, ND. USA.
+ * All Rights Reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the
+ * "Software"), to deal in the Software without restriction, including
+ * without limitation the rights to use, copy, modify, merge, publish,
+ * distribute, sub license, and/or sell copies of the Software, and to
+ * permit persons to whom the Software is furnished to do so, subject to
+ * the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the
+ * next paragraph) shall be included in all copies or substantial portions
+ * of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDERS, AUTHORS AND/OR ITS SUPPLIERS BE LIABLE FOR ANY CLAIM,
+ * DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
+ * OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
+ * USE OR OTHER DEALINGS IN THE SOFTWARE.
+ */
+
+/*
+ * Simple open hash tab implementation.
+ *
+ * Authors:
+ * Thomas Hellström <thomas-at-tungstengraphics-dot-com>
+ */
+
+/*
+ * TODO: Replace this hashtable with Linux' generic implementation
+ * from <linux/hashtable.h>.
+ */
+
+#ifndef VMWGFX_HASHTAB_H
+#define VMWGFX_HASHTAB_H
+
+#include <linux/list.h>
+
+#define drm_hash_entry(_ptr, _type, _member) container_of(_ptr, _type, _member)
+
+struct vmwgfx_hash_item {
+ struct hlist_node head;
+ unsigned long key;
+};
+
+struct vmwgfx_open_hash {
+ struct hlist_head *table;
+ u8 order;
+};
+
+int vmwgfx_ht_create(struct vmwgfx_open_hash *ht, unsigned int order);
+int vmwgfx_ht_insert_item(struct vmwgfx_open_hash *ht, struct vmwgfx_hash_item *item);
+int vmwgfx_ht_just_insert_please(struct vmwgfx_open_hash *ht, struct vmwgfx_hash_item *item,
+ unsigned long seed, int bits, int shift,
+ unsigned long add);
+int vmwgfx_ht_find_item(struct vmwgfx_open_hash *ht, unsigned long key,
+ struct vmwgfx_hash_item **item);
+
+void vmwgfx_ht_verbose_list(struct vmwgfx_open_hash *ht, unsigned long key);
+int vmwgfx_ht_remove_key(struct vmwgfx_open_hash *ht, unsigned long key);
+int vmwgfx_ht_remove_item(struct vmwgfx_open_hash *ht, struct vmwgfx_hash_item *item);
+void vmwgfx_ht_remove(struct vmwgfx_open_hash *ht);
+
+/*
+ * RCU-safe interface
+ *
+ * The user of this API needs to make sure that two or more instances of the
+ * hash table manipulation functions are never run simultaneously.
+ * The lookup function vmwgfx_ht_find_item_rcu may, however, run simultaneously
+ * with any of the manipulation functions as long as it's called from within
+ * an RCU read-locked section.
+ */
+#define vmwgfx_ht_insert_item_rcu vmwgfx_ht_insert_item
+#define vmwgfx_ht_just_insert_please_rcu vmwgfx_ht_just_insert_please
+#define vmwgfx_ht_remove_key_rcu vmwgfx_ht_remove_key
+#define vmwgfx_ht_remove_item_rcu vmwgfx_ht_remove_item
+#define vmwgfx_ht_find_item_rcu vmwgfx_ht_find_item
+
+#endif
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_ioctl.c b/drivers/gpu/drm/vmwgfx/vmwgfx_ioctl.c
index 28af34a..471da2b 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_ioctl.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_ioctl.c
@@ -105,6 +105,9 @@ int vmw_getparam_ioctl(struct drm_device *dev, void *data,
case DRM_VMW_PARAM_SM5:
param->value = has_sm5_context(dev_priv);
break;
+ case DRM_VMW_PARAM_GL43:
+ param->value = has_gl43_context(dev_priv);
+ break;
default:
return -EINVAL;
}
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_kms.c b/drivers/gpu/drm/vmwgfx/vmwgfx_kms.c
index 74fa419..4e693e8 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_kms.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_kms.c
@@ -843,8 +843,6 @@ static void vmw_framebuffer_surface_destroy(struct drm_framebuffer *framebuffer)
drm_framebuffer_cleanup(framebuffer);
vmw_surface_unreference(&vfbs->surface);
- if (vfbs->base.user_obj)
- ttm_base_object_unref(&vfbs->base.user_obj);
kfree(vfbs);
}
@@ -989,6 +987,16 @@ static int vmw_kms_new_framebuffer_surface(struct vmw_private *dev_priv,
* Buffer-object framebuffer code
*/
+static int vmw_framebuffer_bo_create_handle(struct drm_framebuffer *fb,
+ struct drm_file *file_priv,
+ unsigned int *handle)
+{
+ struct vmw_framebuffer_bo *vfbd =
+ vmw_framebuffer_to_vfbd(fb);
+
+ return drm_gem_handle_create(file_priv, &vfbd->buffer->base.base, handle);
+}
+
static void vmw_framebuffer_bo_destroy(struct drm_framebuffer *framebuffer)
{
struct vmw_framebuffer_bo *vfbd =
@@ -996,8 +1004,6 @@ static void vmw_framebuffer_bo_destroy(struct drm_framebuffer *framebuffer)
drm_framebuffer_cleanup(framebuffer);
vmw_bo_unreference(&vfbd->buffer);
- if (vfbd->base.user_obj)
- ttm_base_object_unref(&vfbd->base.user_obj);
kfree(vfbd);
}
@@ -1063,6 +1069,7 @@ static int vmw_framebuffer_bo_dirty_ext(struct drm_framebuffer *framebuffer,
}
static const struct drm_framebuffer_funcs vmw_framebuffer_bo_funcs = {
+ .create_handle = vmw_framebuffer_bo_create_handle,
.destroy = vmw_framebuffer_bo_destroy,
.dirty = vmw_framebuffer_bo_dirty_ext,
};
@@ -1188,7 +1195,7 @@ static int vmw_create_bo_proxy(struct drm_device *dev,
metadata.base_size.depth = 1;
metadata.scanout = true;
- ret = vmw_gb_surface_define(vmw_priv(dev), 0, &metadata, srf_out);
+ ret = vmw_gb_surface_define(vmw_priv(dev), &metadata, srf_out);
if (ret) {
DRM_ERROR("Failed to allocate proxy content buffer\n");
return ret;
@@ -1251,6 +1258,7 @@ static int vmw_kms_new_framebuffer_bo(struct vmw_private *dev_priv,
goto out_err1;
}
+ vfbd->base.base.obj[0] = &bo->base.base;
drm_helper_mode_fill_fb_struct(dev, &vfbd->base.base, mode_cmd);
vfbd->base.bo = true;
vfbd->buffer = vmw_bo_reference(bo);
@@ -1368,34 +1376,13 @@ static struct drm_framebuffer *vmw_kms_fb_create(struct drm_device *dev,
const struct drm_mode_fb_cmd2 *mode_cmd)
{
struct vmw_private *dev_priv = vmw_priv(dev);
- struct ttm_object_file *tfile = vmw_fpriv(file_priv)->tfile;
struct vmw_framebuffer *vfb = NULL;
struct vmw_surface *surface = NULL;
struct vmw_buffer_object *bo = NULL;
- struct ttm_base_object *user_obj;
int ret;
- /*
- * Take a reference on the user object of the resource
- * backing the kms fb. This ensures that user-space handle
- * lookups on that resource will always work as long as
- * it's registered with a kms framebuffer. This is important,
- * since vmw_execbuf_process identifies resources in the
- * command stream using user-space handles.
- */
-
- user_obj = ttm_base_object_lookup(tfile, mode_cmd->handles[0]);
- if (unlikely(user_obj == NULL)) {
- DRM_ERROR("Could not locate requested kms frame buffer.\n");
- return ERR_PTR(-ENOENT);
- }
-
- /**
- * End conditioned code.
- */
-
/* returns either a bo or surface */
- ret = vmw_user_lookup_handle(dev_priv, tfile,
+ ret = vmw_user_lookup_handle(dev_priv, file_priv,
mode_cmd->handles[0],
&surface, &bo);
if (ret)
@@ -1428,10 +1415,8 @@ static struct drm_framebuffer *vmw_kms_fb_create(struct drm_device *dev,
if (ret) {
DRM_ERROR("failed to create vmw_framebuffer: %i\n", ret);
- ttm_base_object_unref(&user_obj);
return ERR_PTR(ret);
- } else
- vfb->user_obj = user_obj;
+ }
return &vfb->base;
}
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_kms.h b/drivers/gpu/drm/vmwgfx/vmwgfx_kms.h
index bbc809f7..4d36e85 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_kms.h
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_kms.h
@@ -219,7 +219,6 @@ struct vmw_framebuffer {
int (*pin)(struct vmw_framebuffer *fb);
int (*unpin)(struct vmw_framebuffer *fb);
bool bo;
- struct ttm_base_object *user_obj;
uint32_t user_handle;
};
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_mob.c b/drivers/gpu/drm/vmwgfx/vmwgfx_mob.c
index f939420..0a8cc28 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_mob.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_mob.c
@@ -1,7 +1,7 @@
// SPDX-License-Identifier: GPL-2.0 OR MIT
/**************************************************************************
*
- * Copyright 2012-2015 VMware, Inc., Palo Alto, CA., USA
+ * Copyright 2012-2021 VMware, Inc., Palo Alto, CA., USA
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the
@@ -29,12 +29,6 @@
#include "vmwgfx_drv.h"
-/*
- * If we set up the screen target otable, screen objects stop working.
- */
-
-#define VMW_OTABLE_SETUP_SUB ((VMWGFX_ENABLE_SCREEN_TARGET_OTABLE ? 0 : 1))
-
#ifdef CONFIG_64BIT
#define VMW_PPN_SIZE 8
#define VMW_MOBFMT_PTDEPTH_0 SVGA3D_MOBFMT_PT64_0
@@ -75,7 +69,7 @@ static const struct vmw_otable pre_dx_tables[] = {
{VMWGFX_NUM_GB_CONTEXT * sizeof(SVGAOTableContextEntry), NULL, true},
{VMWGFX_NUM_GB_SHADER * sizeof(SVGAOTableShaderEntry), NULL, true},
{VMWGFX_NUM_GB_SCREEN_TARGET * sizeof(SVGAOTableScreenTargetEntry),
- NULL, VMWGFX_ENABLE_SCREEN_TARGET_OTABLE}
+ NULL, true}
};
static const struct vmw_otable dx_tables[] = {
@@ -84,7 +78,7 @@ static const struct vmw_otable dx_tables[] = {
{VMWGFX_NUM_GB_CONTEXT * sizeof(SVGAOTableContextEntry), NULL, true},
{VMWGFX_NUM_GB_SHADER * sizeof(SVGAOTableShaderEntry), NULL, true},
{VMWGFX_NUM_GB_SCREEN_TARGET * sizeof(SVGAOTableScreenTargetEntry),
- NULL, VMWGFX_ENABLE_SCREEN_TARGET_OTABLE},
+ NULL, true},
{VMWGFX_NUM_DXCONTEXT * sizeof(SVGAOTableDXContextEntry), NULL, true},
};
@@ -146,9 +140,6 @@ static int vmw_setup_otable_base(struct vmw_private *dev_priv,
if (otable->size <= PAGE_SIZE) {
mob->pt_level = VMW_MOBFMT_PTDEPTH_0;
mob->pt_root_page = vmw_piter_dma_addr(&iter);
- } else if (vsgt->num_regions == 1) {
- mob->pt_level = SVGA3D_MOBFMT_RANGE;
- mob->pt_root_page = vmw_piter_dma_addr(&iter);
} else {
ret = vmw_mob_pt_populate(dev_priv, mob);
if (unlikely(ret != 0))
@@ -413,10 +404,9 @@ struct vmw_mob *vmw_mob_create(unsigned long data_pages)
* @mob: Pointer to the mob the pagetable of which we want to
* populate.
*
- * This function allocates memory to be used for the pagetable, and
- * adjusts TTM memory accounting accordingly. Returns ENOMEM if
- * memory resources aren't sufficient and may cause TTM buffer objects
- * to be swapped out by using the TTM memory accounting function.
+ * This function allocates memory to be used for the pagetable.
+ * Returns ENOMEM if memory resources aren't sufficient and may
+ * cause TTM buffer objects to be swapped out.
*/
static int vmw_mob_pt_populate(struct vmw_private *dev_priv,
struct vmw_mob *mob)
@@ -624,9 +614,6 @@ int vmw_mob_bind(struct vmw_private *dev_priv,
if (likely(num_data_pages == 1)) {
mob->pt_level = VMW_MOBFMT_PTDEPTH_0;
mob->pt_root_page = vmw_piter_dma_addr(&data_iter);
- } else if (vsgt->num_regions == 1) {
- mob->pt_level = SVGA3D_MOBFMT_RANGE;
- mob->pt_root_page = vmw_piter_dma_addr(&data_iter);
} else if (unlikely(mob->pt_bo == NULL)) {
ret = vmw_mob_pt_populate(dev_priv, mob);
if (unlikely(ret != 0))
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_overlay.c b/drivers/gpu/drm/vmwgfx/vmwgfx_overlay.c
index 54c5d16..e9f5c89 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_overlay.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_overlay.c
@@ -451,7 +451,7 @@ int vmw_overlay_ioctl(struct drm_device *dev, void *data,
goto out_unlock;
}
- ret = vmw_user_bo_lookup(tfile, arg->handle, &buf, NULL);
+ ret = vmw_user_bo_lookup(file_priv, arg->handle, &buf);
if (ret)
goto out_unlock;
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_page_dirty.c b/drivers/gpu/drm/vmwgfx/vmwgfx_page_dirty.c
index 922317d..7bc99b1 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_page_dirty.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_page_dirty.c
@@ -57,7 +57,6 @@ enum vmw_bo_dirty_method {
* @ref_count: Reference count for this structure
* @bitmap_size: The size of the bitmap in bits. Typically equal to the
* nuber of pages in the bo.
- * @size: The accounting size for this struct.
* @bitmap: A bitmap where each bit represents a page. A set bit means a
* dirty page.
*/
@@ -68,7 +67,6 @@ struct vmw_bo_dirty {
unsigned int change_count;
unsigned int ref_count;
unsigned long bitmap_size;
- size_t size;
unsigned long bitmap[];
};
@@ -233,12 +231,8 @@ int vmw_bo_dirty_add(struct vmw_buffer_object *vbo)
{
struct vmw_bo_dirty *dirty = vbo->dirty;
pgoff_t num_pages = vbo->base.resource->num_pages;
- size_t size, acc_size;
+ size_t size;
int ret;
- static struct ttm_operation_ctx ctx = {
- .interruptible = false,
- .no_wait_gpu = false
- };
if (dirty) {
dirty->ref_count++;
@@ -246,20 +240,12 @@ int vmw_bo_dirty_add(struct vmw_buffer_object *vbo)
}
size = sizeof(*dirty) + BITS_TO_LONGS(num_pages) * sizeof(long);
- acc_size = ttm_round_pot(size);
- ret = ttm_mem_global_alloc(&ttm_mem_glob, acc_size, &ctx);
- if (ret) {
- VMW_DEBUG_USER("Out of graphics memory for buffer object "
- "dirty tracker.\n");
- return ret;
- }
dirty = kvzalloc(size, GFP_KERNEL);
if (!dirty) {
ret = -ENOMEM;
goto out_no_dirty;
}
- dirty->size = acc_size;
dirty->bitmap_size = num_pages;
dirty->start = dirty->bitmap_size;
dirty->end = 0;
@@ -285,7 +271,6 @@ int vmw_bo_dirty_add(struct vmw_buffer_object *vbo)
return 0;
out_no_dirty:
- ttm_mem_global_free(&ttm_mem_glob, acc_size);
return ret;
}
@@ -304,10 +289,7 @@ void vmw_bo_dirty_release(struct vmw_buffer_object *vbo)
struct vmw_bo_dirty *dirty = vbo->dirty;
if (dirty && --dirty->ref_count == 0) {
- size_t acc_size = dirty->size;
-
kvfree(dirty);
- ttm_mem_global_free(&ttm_mem_glob, acc_size);
vbo->dirty = NULL;
}
}
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_prime.c b/drivers/gpu/drm/vmwgfx/vmwgfx_prime.c
index d9552a1..2d72a5e 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_prime.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_prime.c
@@ -85,6 +85,5 @@ int vmw_prime_handle_to_fd(struct drm_device *dev,
int *prime_fd)
{
struct ttm_object_file *tfile = vmw_fpriv(file_priv)->tfile;
-
return ttm_prime_handle_to_fd(tfile, handle, flags, prime_fd);
}
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_resource.c b/drivers/gpu/drm/vmwgfx/vmwgfx_resource.c
index 8d1e869..708899b 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_resource.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_resource.c
@@ -320,11 +320,12 @@ vmw_user_resource_noref_lookup_handle(struct vmw_private *dev_priv,
* The pointer this pointed at by out_surf and out_buf needs to be null.
*/
int vmw_user_lookup_handle(struct vmw_private *dev_priv,
- struct ttm_object_file *tfile,
+ struct drm_file *filp,
uint32_t handle,
struct vmw_surface **out_surf,
struct vmw_buffer_object **out_buf)
{
+ struct ttm_object_file *tfile = vmw_fpriv(filp)->tfile;
struct vmw_resource *res;
int ret;
@@ -339,7 +340,7 @@ int vmw_user_lookup_handle(struct vmw_private *dev_priv,
}
*out_surf = NULL;
- ret = vmw_user_bo_lookup(tfile, handle, out_buf, NULL);
+ ret = vmw_user_bo_lookup(filp, handle, out_buf);
return ret;
}
@@ -362,14 +363,10 @@ static int vmw_resource_buf_alloc(struct vmw_resource *res,
return 0;
}
- backup = kzalloc(sizeof(*backup), GFP_KERNEL);
- if (unlikely(!backup))
- return -ENOMEM;
-
- ret = vmw_bo_init(res->dev_priv, backup, res->backup_size,
- res->func->backup_placement,
- interruptible, false,
- &vmw_bo_bo_free);
+ ret = vmw_bo_create(res->dev_priv, res->backup_size,
+ res->func->backup_placement,
+ interruptible, false,
+ &vmw_bo_bo_free, &backup);
if (unlikely(ret != 0))
goto out_no_bo;
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_scrn.c b/drivers/gpu/drm/vmwgfx/vmwgfx_scrn.c
index bd157fb2..3004c7a 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_scrn.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_scrn.c
@@ -442,19 +442,15 @@ vmw_sou_primary_plane_prepare_fb(struct drm_plane *plane,
vps->bo_size = 0;
}
- vps->bo = kzalloc(sizeof(*vps->bo), GFP_KERNEL);
- if (!vps->bo)
- return -ENOMEM;
-
vmw_svga_enable(dev_priv);
/* After we have alloced the backing store might not be able to
* resume the overlays, this is preferred to failing to alloc.
*/
vmw_overlay_pause_all(dev_priv);
- ret = vmw_bo_init(dev_priv, vps->bo, size,
- &vmw_vram_placement,
- false, true, &vmw_bo_bo_free);
+ ret = vmw_bo_create(dev_priv, size,
+ &vmw_vram_placement,
+ false, true, &vmw_bo_bo_free, &vps->bo);
vmw_overlay_resume_all(dev_priv);
if (ret) {
vps->bo = NULL; /* vmw_bo_init frees on error */
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_shader.c b/drivers/gpu/drm/vmwgfx/vmwgfx_shader.c
index b8dd625..108a496 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_shader.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_shader.c
@@ -53,10 +53,6 @@ struct vmw_dx_shader {
struct list_head cotable_head;
};
-static uint64_t vmw_user_shader_size;
-static uint64_t vmw_shader_size;
-static size_t vmw_shader_dx_size;
-
static void vmw_user_shader_free(struct vmw_resource *res);
static struct vmw_resource *
vmw_user_shader_base_to_res(struct ttm_base_object *base);
@@ -79,7 +75,6 @@ static void vmw_dx_shader_commit_notify(struct vmw_resource *res,
enum vmw_cmdbuf_res_state state);
static bool vmw_shader_id_ok(u32 user_key, SVGA3dShaderType shader_type);
static u32 vmw_shader_key(u32 user_key, SVGA3dShaderType shader_type);
-static uint64_t vmw_user_shader_size;
static const struct vmw_user_resource_conv user_shader_conv = {
.object_type = VMW_RES_SHADER,
@@ -563,16 +558,14 @@ void vmw_dx_shader_cotable_list_scrub(struct vmw_private *dev_priv,
*
* @res: The shader resource
*
- * Frees the DX shader resource and updates memory accounting.
+ * Frees the DX shader resource.
*/
static void vmw_dx_shader_res_free(struct vmw_resource *res)
{
- struct vmw_private *dev_priv = res->dev_priv;
struct vmw_dx_shader *shader = vmw_res_to_dx_shader(res);
vmw_resource_unreference(&shader->cotable);
kfree(shader);
- ttm_mem_global_free(vmw_mem_glob(dev_priv), vmw_shader_dx_size);
}
/**
@@ -594,30 +587,13 @@ int vmw_dx_shader_add(struct vmw_cmdbuf_res_manager *man,
struct vmw_dx_shader *shader;
struct vmw_resource *res;
struct vmw_private *dev_priv = ctx->dev_priv;
- struct ttm_operation_ctx ttm_opt_ctx = {
- .interruptible = true,
- .no_wait_gpu = false
- };
int ret;
- if (!vmw_shader_dx_size)
- vmw_shader_dx_size = ttm_round_pot(sizeof(*shader));
-
if (!vmw_shader_id_ok(user_key, shader_type))
return -EINVAL;
- ret = ttm_mem_global_alloc(vmw_mem_glob(dev_priv), vmw_shader_dx_size,
- &ttm_opt_ctx);
- if (ret) {
- if (ret != -ERESTARTSYS)
- DRM_ERROR("Out of graphics memory for shader "
- "creation.\n");
- return ret;
- }
-
shader = kmalloc(sizeof(*shader), GFP_KERNEL);
if (!shader) {
- ttm_mem_global_free(vmw_mem_glob(dev_priv), vmw_shader_dx_size);
return -ENOMEM;
}
@@ -669,21 +645,15 @@ static void vmw_user_shader_free(struct vmw_resource *res)
{
struct vmw_user_shader *ushader =
container_of(res, struct vmw_user_shader, shader.res);
- struct vmw_private *dev_priv = res->dev_priv;
ttm_base_object_kfree(ushader, base);
- ttm_mem_global_free(vmw_mem_glob(dev_priv),
- vmw_user_shader_size);
}
static void vmw_shader_free(struct vmw_resource *res)
{
struct vmw_shader *shader = vmw_res_to_shader(res);
- struct vmw_private *dev_priv = res->dev_priv;
kfree(shader);
- ttm_mem_global_free(vmw_mem_glob(dev_priv),
- vmw_shader_size);
}
/*
@@ -706,8 +676,7 @@ int vmw_shader_destroy_ioctl(struct drm_device *dev, void *data,
struct drm_vmw_shader_arg *arg = (struct drm_vmw_shader_arg *)data;
struct ttm_object_file *tfile = vmw_fpriv(file_priv)->tfile;
- return ttm_ref_object_base_unref(tfile, arg->handle,
- TTM_REF_USAGE);
+ return ttm_ref_object_base_unref(tfile, arg->handle);
}
static int vmw_user_shader_alloc(struct vmw_private *dev_priv,
@@ -722,31 +691,10 @@ static int vmw_user_shader_alloc(struct vmw_private *dev_priv,
{
struct vmw_user_shader *ushader;
struct vmw_resource *res, *tmp;
- struct ttm_operation_ctx ctx = {
- .interruptible = true,
- .no_wait_gpu = false
- };
int ret;
- if (unlikely(vmw_user_shader_size == 0))
- vmw_user_shader_size =
- ttm_round_pot(sizeof(struct vmw_user_shader)) +
- VMW_IDA_ACC_SIZE + TTM_OBJ_EXTRA_SIZE;
-
- ret = ttm_mem_global_alloc(vmw_mem_glob(dev_priv),
- vmw_user_shader_size,
- &ctx);
- if (unlikely(ret != 0)) {
- if (ret != -ERESTARTSYS)
- DRM_ERROR("Out of graphics memory for shader "
- "creation.\n");
- goto out;
- }
-
ushader = kzalloc(sizeof(*ushader), GFP_KERNEL);
if (unlikely(!ushader)) {
- ttm_mem_global_free(vmw_mem_glob(dev_priv),
- vmw_user_shader_size);
ret = -ENOMEM;
goto out;
}
@@ -769,7 +717,7 @@ static int vmw_user_shader_alloc(struct vmw_private *dev_priv,
tmp = vmw_resource_reference(res);
ret = ttm_base_object_init(tfile, &ushader->base, false,
VMW_RES_SHADER,
- &vmw_user_shader_base_release, NULL);
+ &vmw_user_shader_base_release);
if (unlikely(ret != 0)) {
vmw_resource_unreference(&tmp);
@@ -793,31 +741,10 @@ static struct vmw_resource *vmw_shader_alloc(struct vmw_private *dev_priv,
{
struct vmw_shader *shader;
struct vmw_resource *res;
- struct ttm_operation_ctx ctx = {
- .interruptible = true,
- .no_wait_gpu = false
- };
int ret;
- if (unlikely(vmw_shader_size == 0))
- vmw_shader_size =
- ttm_round_pot(sizeof(struct vmw_shader)) +
- VMW_IDA_ACC_SIZE;
-
- ret = ttm_mem_global_alloc(vmw_mem_glob(dev_priv),
- vmw_shader_size,
- &ctx);
- if (unlikely(ret != 0)) {
- if (ret != -ERESTARTSYS)
- DRM_ERROR("Out of graphics memory for shader "
- "creation.\n");
- goto out_err;
- }
-
shader = kzalloc(sizeof(*shader), GFP_KERNEL);
if (unlikely(!shader)) {
- ttm_mem_global_free(vmw_mem_glob(dev_priv),
- vmw_shader_size);
ret = -ENOMEM;
goto out_err;
}
@@ -849,8 +776,7 @@ static int vmw_shader_define(struct drm_device *dev, struct drm_file *file_priv,
int ret;
if (buffer_handle != SVGA3D_INVALID_ID) {
- ret = vmw_user_bo_lookup(tfile, buffer_handle,
- &buffer, NULL);
+ ret = vmw_user_bo_lookup(file_priv, buffer_handle, &buffer);
if (unlikely(ret != 0)) {
VMW_DEBUG_USER("Couldn't find buffer for shader creation.\n");
return ret;
@@ -966,13 +892,8 @@ int vmw_compat_shader_add(struct vmw_private *dev_priv,
if (!vmw_shader_id_ok(user_key, shader_type))
return -EINVAL;
- /* Allocate and pin a DMA buffer */
- buf = kzalloc(sizeof(*buf), GFP_KERNEL);
- if (unlikely(!buf))
- return -ENOMEM;
-
- ret = vmw_bo_init(dev_priv, buf, size, &vmw_sys_placement,
- true, true, vmw_bo_bo_free);
+ ret = vmw_bo_create(dev_priv, size, &vmw_sys_placement,
+ true, true, vmw_bo_bo_free, &buf);
if (unlikely(ret != 0))
goto out;
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_simple_resource.c b/drivers/gpu/drm/vmwgfx/vmwgfx_simple_resource.c
index 33b69a7..483ad54 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_simple_resource.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_simple_resource.c
@@ -32,12 +32,10 @@
* struct vmw_user_simple_resource - User-space simple resource struct
*
* @base: The TTM base object implementing user-space visibility.
- * @account_size: How much memory was accounted for this object.
* @simple: The embedded struct vmw_simple_resource.
*/
struct vmw_user_simple_resource {
struct ttm_base_object base;
- size_t account_size;
struct vmw_simple_resource simple;
/*
* Nothing to be placed after @simple, since size of @simple is
@@ -91,18 +89,15 @@ static int vmw_simple_resource_init(struct vmw_private *dev_priv,
*
* @res: The struct vmw_resource member of the simple resource object.
*
- * Frees memory and memory accounting for the object.
+ * Frees memory for the object.
*/
static void vmw_simple_resource_free(struct vmw_resource *res)
{
struct vmw_user_simple_resource *usimple =
container_of(res, struct vmw_user_simple_resource,
simple.res);
- struct vmw_private *dev_priv = res->dev_priv;
- size_t size = usimple->account_size;
ttm_base_object_kfree(usimple, base);
- ttm_mem_global_free(vmw_mem_glob(dev_priv), size);
}
/**
@@ -149,39 +144,19 @@ vmw_simple_resource_create_ioctl(struct drm_device *dev, void *data,
struct vmw_resource *res;
struct vmw_resource *tmp;
struct ttm_object_file *tfile = vmw_fpriv(file_priv)->tfile;
- struct ttm_operation_ctx ctx = {
- .interruptible = true,
- .no_wait_gpu = false
- };
size_t alloc_size;
- size_t account_size;
int ret;
alloc_size = offsetof(struct vmw_user_simple_resource, simple) +
func->size;
- account_size = ttm_round_pot(alloc_size) + VMW_IDA_ACC_SIZE +
- TTM_OBJ_EXTRA_SIZE;
-
- ret = ttm_mem_global_alloc(vmw_mem_glob(dev_priv), account_size,
- &ctx);
- if (ret) {
- if (ret != -ERESTARTSYS)
- DRM_ERROR("Out of graphics memory for %s"
- " creation.\n", func->res_func.type_name);
-
- goto out_ret;
- }
usimple = kzalloc(alloc_size, GFP_KERNEL);
if (!usimple) {
- ttm_mem_global_free(vmw_mem_glob(dev_priv),
- account_size);
ret = -ENOMEM;
goto out_ret;
}
usimple->simple.func = func;
- usimple->account_size = account_size;
res = &usimple->simple.res;
usimple->base.shareable = false;
usimple->base.tfile = NULL;
@@ -197,7 +172,7 @@ vmw_simple_resource_create_ioctl(struct drm_device *dev, void *data,
tmp = vmw_resource_reference(res);
ret = ttm_base_object_init(tfile, &usimple->base, false,
func->ttm_res_type,
- &vmw_simple_resource_base_release, NULL);
+ &vmw_simple_resource_base_release);
if (ret) {
vmw_resource_unreference(&tmp);
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_so.c b/drivers/gpu/drm/vmwgfx/vmwgfx_so.c
index 9efb446..4ea32b0 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_so.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_so.c
@@ -279,18 +279,15 @@ static bool vmw_view_id_ok(u32 user_key, enum vmw_view_type view_type)
*
* @res: Pointer to a struct vmw_resource
*
- * Frees memory and memory accounting held by a struct vmw_view.
+ * Frees memory held by the struct vmw_view.
*/
static void vmw_view_res_free(struct vmw_resource *res)
{
struct vmw_view *view = vmw_view(res);
- size_t size = offsetof(struct vmw_view, cmd) + view->cmd_size;
- struct vmw_private *dev_priv = res->dev_priv;
vmw_resource_unreference(&view->cotable);
vmw_resource_unreference(&view->srf);
kfree_rcu(view, rcu);
- ttm_mem_global_free(vmw_mem_glob(dev_priv), size);
}
/**
@@ -327,10 +324,6 @@ int vmw_view_add(struct vmw_cmdbuf_res_manager *man,
struct vmw_private *dev_priv = ctx->dev_priv;
struct vmw_resource *res;
struct vmw_view *view;
- struct ttm_operation_ctx ttm_opt_ctx = {
- .interruptible = true,
- .no_wait_gpu = false
- };
size_t size;
int ret;
@@ -347,16 +340,8 @@ int vmw_view_add(struct vmw_cmdbuf_res_manager *man,
size = offsetof(struct vmw_view, cmd) + cmd_size;
- ret = ttm_mem_global_alloc(vmw_mem_glob(dev_priv), size, &ttm_opt_ctx);
- if (ret) {
- if (ret != -ERESTARTSYS)
- DRM_ERROR("Out of graphics memory for view creation\n");
- return ret;
- }
-
view = kmalloc(size, GFP_KERNEL);
if (!view) {
- ttm_mem_global_free(vmw_mem_glob(dev_priv), size);
return -ENOMEM;
}
@@ -582,4 +567,8 @@ static void vmw_so_build_asserts(void)
offsetof(SVGA3dCmdDXDefineRenderTargetView, sid));
BUILD_BUG_ON(offsetof(struct vmw_view_define, sid) !=
offsetof(SVGA3dCmdDXDefineDepthStencilView, sid));
+ BUILD_BUG_ON(offsetof(struct vmw_view_define, sid) !=
+ offsetof(SVGA3dCmdDXDefineUAView, sid));
+ BUILD_BUG_ON(offsetof(struct vmw_view_define, sid) !=
+ offsetof(SVGA3dCmdDXDefineDepthStencilView_v2, sid));
}
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_so.h b/drivers/gpu/drm/vmwgfx/vmwgfx_so.h
index f48b84b..01c701e 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_so.h
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_so.h
@@ -93,7 +93,10 @@ static inline enum vmw_view_type vmw_view_cmd_to_type(u32 id)
id == SVGA_3D_CMD_DX_DESTROY_UA_VIEW)
return vmw_view_ua;
- if (tmp > (u32)vmw_view_max)
+ if (id == SVGA_3D_CMD_DX_DEFINE_DEPTHSTENCIL_VIEW_V2)
+ return vmw_view_ds;
+
+ if (tmp > (u32)vmw_view_ds)
return vmw_view_max;
return (enum vmw_view_type) tmp;
@@ -123,6 +126,7 @@ static inline enum vmw_so_type vmw_so_cmd_to_type(u32 id)
case SVGA_3D_CMD_DX_DESTROY_DEPTHSTENCIL_STATE:
return vmw_so_ds;
case SVGA_3D_CMD_DX_DEFINE_RASTERIZER_STATE:
+ case SVGA_3D_CMD_DX_DEFINE_RASTERIZER_STATE_V2:
case SVGA_3D_CMD_DX_DESTROY_RASTERIZER_STATE:
return vmw_so_rs;
case SVGA_3D_CMD_DX_DEFINE_SAMPLER_STATE:
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_stdu.c b/drivers/gpu/drm/vmwgfx/vmwgfx_stdu.c
index d85310b..ae9a604 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_stdu.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_stdu.c
@@ -1123,7 +1123,7 @@ vmw_stdu_primary_plane_prepare_fb(struct drm_plane *plane,
}
if (!vps->surf) {
- ret = vmw_gb_surface_define(dev_priv, 0, &metadata,
+ ret = vmw_gb_surface_define(dev_priv, &metadata,
&vps->surf);
if (ret != 0) {
DRM_ERROR("Couldn't allocate STDU surface.\n");
@@ -1872,8 +1872,8 @@ int vmw_kms_stdu_init_display(struct vmw_private *dev_priv)
int i, ret;
- /* Do nothing if Screen Target support is turned off */
- if (!VMWGFX_ENABLE_SCREEN_TARGET_OTABLE || !dev_priv->has_mob)
+ /* Do nothing if there's no support for MOBs */
+ if (!dev_priv->has_mob)
return -ENOSYS;
if (!(dev_priv->capabilities & SVGA_CAP_GBOBJECTS))
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_streamoutput.c b/drivers/gpu/drm/vmwgfx/vmwgfx_streamoutput.c
index c8efa4a..2de9741 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_streamoutput.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_streamoutput.c
@@ -60,8 +60,6 @@ static int vmw_dx_streamoutput_unbind(struct vmw_resource *res, bool readback,
static void vmw_dx_streamoutput_commit_notify(struct vmw_resource *res,
enum vmw_cmdbuf_res_state state);
-static size_t vmw_streamoutput_size;
-
static const struct vmw_res_func vmw_dx_streamoutput_func = {
.res_type = vmw_res_streamoutput,
.needs_backup = true,
@@ -254,12 +252,10 @@ vmw_dx_streamoutput_lookup(struct vmw_cmdbuf_res_manager *man,
static void vmw_dx_streamoutput_res_free(struct vmw_resource *res)
{
- struct vmw_private *dev_priv = res->dev_priv;
struct vmw_dx_streamoutput *so = vmw_res_to_dx_streamoutput(res);
vmw_resource_unreference(&so->cotable);
kfree(so);
- ttm_mem_global_free(vmw_mem_glob(dev_priv), vmw_streamoutput_size);
}
static void vmw_dx_streamoutput_hw_destroy(struct vmw_resource *res)
@@ -284,27 +280,10 @@ int vmw_dx_streamoutput_add(struct vmw_cmdbuf_res_manager *man,
struct vmw_dx_streamoutput *so;
struct vmw_resource *res;
struct vmw_private *dev_priv = ctx->dev_priv;
- struct ttm_operation_ctx ttm_opt_ctx = {
- .interruptible = true,
- .no_wait_gpu = false
- };
int ret;
- if (!vmw_streamoutput_size)
- vmw_streamoutput_size = ttm_round_pot(sizeof(*so));
-
- ret = ttm_mem_global_alloc(vmw_mem_glob(dev_priv),
- vmw_streamoutput_size, &ttm_opt_ctx);
- if (ret) {
- if (ret != -ERESTARTSYS)
- DRM_ERROR("Out of graphics memory for streamout.\n");
- return ret;
- }
-
so = kmalloc(sizeof(*so), GFP_KERNEL);
if (!so) {
- ttm_mem_global_free(vmw_mem_glob(dev_priv),
- vmw_streamoutput_size);
return -ENOMEM;
}
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_surface.c b/drivers/gpu/drm/vmwgfx/vmwgfx_surface.c
index 5d53a5f..00e8e27 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_surface.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_surface.c
@@ -45,16 +45,12 @@
* @prime: The TTM prime object.
* @base: The TTM base object handling user-space visibility.
* @srf: The surface metadata.
- * @size: TTM accounting size for the surface.
* @master: Master of the creating client. Used for security check.
- * @backup_base: The TTM base object of the backup buffer.
*/
struct vmw_user_surface {
struct ttm_prime_object prime;
struct vmw_surface srf;
- uint32_t size;
struct drm_master *master;
- struct ttm_base_object *backup_base;
};
/**
@@ -74,13 +70,11 @@ struct vmw_surface_offset {
/**
* struct vmw_surface_dirty - Surface dirty-tracker
* @cache: Cached layout information of the surface.
- * @size: Accounting size for the struct vmw_surface_dirty.
* @num_subres: Number of subresources.
* @boxes: Array of SVGA3dBoxes indicating dirty regions. One per subresource.
*/
struct vmw_surface_dirty {
struct vmw_surface_cache cache;
- size_t size;
u32 num_subres;
SVGA3dBox boxes[];
};
@@ -129,9 +123,6 @@ static const struct vmw_user_resource_conv user_surface_conv = {
const struct vmw_user_resource_conv *user_surface_converter =
&user_surface_conv;
-
-static uint64_t vmw_user_surface_size;
-
static const struct vmw_res_func vmw_legacy_surface_func = {
.res_type = vmw_res_surface,
.needs_backup = false,
@@ -359,7 +350,7 @@ static void vmw_surface_dma_encode(struct vmw_surface *srf,
* vmw_surface.
*
* Destroys a the device surface associated with a struct vmw_surface if
- * any, and adjusts accounting and resource count accordingly.
+ * any, and adjusts resource count accordingly.
*/
static void vmw_hw_surface_destroy(struct vmw_resource *res)
{
@@ -666,8 +657,6 @@ static void vmw_user_surface_free(struct vmw_resource *res)
struct vmw_surface *srf = vmw_res_to_srf(res);
struct vmw_user_surface *user_srf =
container_of(srf, struct vmw_user_surface, srf);
- struct vmw_private *dev_priv = srf->res.dev_priv;
- uint32_t size = user_srf->size;
WARN_ON_ONCE(res->dirty);
if (user_srf->master)
@@ -676,7 +665,6 @@ static void vmw_user_surface_free(struct vmw_resource *res)
kfree(srf->metadata.sizes);
kfree(srf->snooper.image);
ttm_prime_object_kfree(user_srf, prime);
- ttm_mem_global_free(vmw_mem_glob(dev_priv), size);
}
/**
@@ -696,8 +684,6 @@ static void vmw_user_surface_base_release(struct ttm_base_object **p_base)
struct vmw_resource *res = &user_srf->srf.res;
*p_base = NULL;
- if (user_srf->backup_base)
- ttm_base_object_unref(&user_srf->backup_base);
vmw_resource_unreference(&res);
}
@@ -715,7 +701,7 @@ int vmw_surface_destroy_ioctl(struct drm_device *dev, void *data,
struct drm_vmw_surface_arg *arg = (struct drm_vmw_surface_arg *)data;
struct ttm_object_file *tfile = vmw_fpriv(file_priv)->tfile;
- return ttm_ref_object_base_unref(tfile, arg->sid, TTM_REF_USAGE);
+ return ttm_ref_object_base_unref(tfile, arg->sid);
}
/**
@@ -740,23 +726,14 @@ int vmw_surface_define_ioctl(struct drm_device *dev, void *data,
struct drm_vmw_surface_create_req *req = &arg->req;
struct drm_vmw_surface_arg *rep = &arg->rep;
struct ttm_object_file *tfile = vmw_fpriv(file_priv)->tfile;
- struct ttm_operation_ctx ctx = {
- .interruptible = true,
- .no_wait_gpu = false
- };
int ret;
int i, j;
uint32_t cur_bo_offset;
struct drm_vmw_size *cur_size;
struct vmw_surface_offset *cur_offset;
uint32_t num_sizes;
- uint32_t size;
const SVGA3dSurfaceDesc *desc;
- if (unlikely(vmw_user_surface_size == 0))
- vmw_user_surface_size = ttm_round_pot(sizeof(*user_srf)) +
- VMW_IDA_ACC_SIZE + TTM_OBJ_EXTRA_SIZE;
-
num_sizes = 0;
for (i = 0; i < DRM_VMW_MAX_SURFACE_FACES; ++i) {
if (req->mip_levels[i] > DRM_VMW_MAX_MIP_LEVELS)
@@ -768,10 +745,6 @@ int vmw_surface_define_ioctl(struct drm_device *dev, void *data,
num_sizes == 0)
return -EINVAL;
- size = vmw_user_surface_size +
- ttm_round_pot(num_sizes * sizeof(struct drm_vmw_size)) +
- ttm_round_pot(num_sizes * sizeof(struct vmw_surface_offset));
-
desc = vmw_surface_get_desc(req->format);
if (unlikely(desc->blockDesc == SVGA3DBLOCKDESC_NONE)) {
VMW_DEBUG_USER("Invalid format %d for surface creation.\n",
@@ -779,18 +752,10 @@ int vmw_surface_define_ioctl(struct drm_device *dev, void *data,
return -EINVAL;
}
- ret = ttm_mem_global_alloc(vmw_mem_glob(dev_priv),
- size, &ctx);
- if (unlikely(ret != 0)) {
- if (ret != -ERESTARTSYS)
- DRM_ERROR("Out of graphics memory for surface.\n");
- goto out_unlock;
- }
-
user_srf = kzalloc(sizeof(*user_srf), GFP_KERNEL);
if (unlikely(!user_srf)) {
ret = -ENOMEM;
- goto out_no_user_srf;
+ goto out_unlock;
}
srf = &user_srf->srf;
@@ -805,7 +770,6 @@ int vmw_surface_define_ioctl(struct drm_device *dev, void *data,
memcpy(metadata->mip_levels, req->mip_levels,
sizeof(metadata->mip_levels));
metadata->num_sizes = num_sizes;
- user_srf->size = size;
metadata->sizes =
memdup_user((struct drm_vmw_size __user *)(unsigned long)
req->size_addr,
@@ -883,22 +847,22 @@ int vmw_surface_define_ioctl(struct drm_device *dev, void *data,
if (dev_priv->has_mob && req->shareable) {
uint32_t backup_handle;
- ret = vmw_user_bo_alloc(dev_priv, tfile,
- res->backup_size,
- true,
- &backup_handle,
- &res->backup,
- &user_srf->backup_base);
+ ret = vmw_gem_object_create_with_handle(dev_priv,
+ file_priv,
+ res->backup_size,
+ &backup_handle,
+ &res->backup);
if (unlikely(ret != 0)) {
vmw_resource_unreference(&res);
goto out_unlock;
}
+ vmw_bo_reference(res->backup);
}
tmp = vmw_resource_reference(&srf->res);
ret = ttm_prime_object_init(tfile, res->backup_size, &user_srf->prime,
req->shareable, VMW_RES_SURFACE,
- &vmw_user_surface_base_release, NULL);
+ &vmw_user_surface_base_release);
if (unlikely(ret != 0)) {
vmw_resource_unreference(&tmp);
@@ -916,8 +880,6 @@ int vmw_surface_define_ioctl(struct drm_device *dev, void *data,
kfree(metadata->sizes);
out_no_sizes:
ttm_prime_object_kfree(user_srf, prime);
-out_no_user_srf:
- ttm_mem_global_free(vmw_mem_glob(dev_priv), size);
out_unlock:
return ret;
}
@@ -955,7 +917,6 @@ vmw_surface_handle_reference(struct vmw_private *dev_priv,
VMW_DEBUG_USER("Referenced object is not a surface.\n");
goto out_bad_resource;
}
-
if (handle_type != DRM_VMW_HANDLE_PRIME) {
bool require_exist = false;
@@ -980,8 +941,7 @@ vmw_surface_handle_reference(struct vmw_private *dev_priv,
if (unlikely(drm_is_render_client(file_priv)))
require_exist = true;
- ret = ttm_ref_object_add(tfile, base, TTM_REF_USAGE, NULL,
- require_exist);
+ ret = ttm_ref_object_add(tfile, base, NULL, require_exist);
if (unlikely(ret != 0)) {
DRM_ERROR("Could not add a reference to a surface.\n");
goto out_bad_resource;
@@ -995,7 +955,7 @@ vmw_surface_handle_reference(struct vmw_private *dev_priv,
ttm_base_object_unref(&base);
out_no_lookup:
if (handle_type == DRM_VMW_HANDLE_PRIME)
- (void) ttm_ref_object_base_unref(tfile, handle, TTM_REF_USAGE);
+ (void) ttm_ref_object_base_unref(tfile, handle);
return ret;
}
@@ -1045,7 +1005,7 @@ int vmw_surface_reference_ioctl(struct drm_device *dev, void *data,
if (unlikely(ret != 0)) {
VMW_DEBUG_USER("copy_to_user failed %p %u\n", user_sizes,
srf->metadata.num_sizes);
- ttm_ref_object_base_unref(tfile, base->handle, TTM_REF_USAGE);
+ ttm_ref_object_base_unref(tfile, base->handle);
ret = -EFAULT;
}
@@ -1459,7 +1419,6 @@ vmw_gb_surface_define_internal(struct drm_device *dev,
struct vmw_resource *res;
struct vmw_resource *tmp;
int ret = 0;
- uint32_t size;
uint32_t backup_handle = 0;
SVGA3dSurfaceAllFlags svga3d_flags_64 =
SVGA3D_FLAGS_64(req->svga3d_flags_upper_32_bits,
@@ -1506,12 +1465,6 @@ vmw_gb_surface_define_internal(struct drm_device *dev,
return -EINVAL;
}
- if (unlikely(vmw_user_surface_size == 0))
- vmw_user_surface_size = ttm_round_pot(sizeof(*user_srf)) +
- VMW_IDA_ACC_SIZE + TTM_OBJ_EXTRA_SIZE;
-
- size = vmw_user_surface_size;
-
metadata.flags = svga3d_flags_64;
metadata.format = req->base.format;
metadata.mip_levels[0] = req->base.mip_levels;
@@ -1526,7 +1479,7 @@ vmw_gb_surface_define_internal(struct drm_device *dev,
drm_vmw_surface_flag_scanout;
/* Define a surface based on the parameters. */
- ret = vmw_gb_surface_define(dev_priv, size, &metadata, &srf);
+ ret = vmw_gb_surface_define(dev_priv, &metadata, &srf);
if (ret != 0) {
VMW_DEBUG_USER("Failed to define surface.\n");
return ret;
@@ -1539,9 +1492,8 @@ vmw_gb_surface_define_internal(struct drm_device *dev,
res = &user_srf->srf.res;
if (req->base.buffer_handle != SVGA3D_INVALID_ID) {
- ret = vmw_user_bo_lookup(tfile, req->base.buffer_handle,
- &res->backup,
- &user_srf->backup_base);
+ ret = vmw_user_bo_lookup(file_priv, req->base.buffer_handle,
+ &res->backup);
if (ret == 0) {
if (res->backup->base.base.size < res->backup_size) {
VMW_DEBUG_USER("Surface backup buffer too small.\n");
@@ -1554,14 +1506,15 @@ vmw_gb_surface_define_internal(struct drm_device *dev,
}
} else if (req->base.drm_surface_flags &
(drm_vmw_surface_flag_create_buffer |
- drm_vmw_surface_flag_coherent))
- ret = vmw_user_bo_alloc(dev_priv, tfile,
- res->backup_size,
- req->base.drm_surface_flags &
- drm_vmw_surface_flag_shareable,
- &backup_handle,
- &res->backup,
- &user_srf->backup_base);
+ drm_vmw_surface_flag_coherent)) {
+ ret = vmw_gem_object_create_with_handle(dev_priv, file_priv,
+ res->backup_size,
+ &backup_handle,
+ &res->backup);
+ if (ret == 0)
+ vmw_bo_reference(res->backup);
+
+ }
if (unlikely(ret != 0)) {
vmw_resource_unreference(&res);
@@ -1593,7 +1546,7 @@ vmw_gb_surface_define_internal(struct drm_device *dev,
req->base.drm_surface_flags &
drm_vmw_surface_flag_shareable,
VMW_RES_SURFACE,
- &vmw_user_surface_base_release, NULL);
+ &vmw_user_surface_base_release);
if (unlikely(ret != 0)) {
vmw_resource_unreference(&tmp);
@@ -1613,7 +1566,6 @@ vmw_gb_surface_define_internal(struct drm_device *dev,
rep->buffer_size = 0;
rep->buffer_handle = SVGA3D_INVALID_ID;
}
-
vmw_resource_unreference(&res);
out_unlock:
@@ -1636,12 +1588,11 @@ vmw_gb_surface_reference_internal(struct drm_device *dev,
struct drm_file *file_priv)
{
struct vmw_private *dev_priv = vmw_priv(dev);
- struct ttm_object_file *tfile = vmw_fpriv(file_priv)->tfile;
struct vmw_surface *srf;
struct vmw_user_surface *user_srf;
struct vmw_surface_metadata *metadata;
struct ttm_base_object *base;
- uint32_t backup_handle;
+ u32 backup_handle;
int ret;
ret = vmw_surface_handle_reference(dev_priv, file_priv, req->sid,
@@ -1658,14 +1609,12 @@ vmw_gb_surface_reference_internal(struct drm_device *dev,
metadata = &srf->metadata;
mutex_lock(&dev_priv->cmdbuf_mutex); /* Protect res->backup */
- ret = vmw_user_bo_reference(tfile, srf->res.backup, &backup_handle);
+ ret = drm_gem_handle_create(file_priv, &srf->res.backup->base.base,
+ &backup_handle);
mutex_unlock(&dev_priv->cmdbuf_mutex);
-
- if (unlikely(ret != 0)) {
- DRM_ERROR("Could not add a reference to a GB surface "
- "backup buffer.\n");
- (void) ttm_ref_object_base_unref(tfile, base->handle,
- TTM_REF_USAGE);
+ if (ret != 0) {
+ drm_err(dev, "Wasn't able to create a backing handle for surface sid = %u.\n",
+ req->sid);
goto out_bad_resource;
}
@@ -1955,11 +1904,7 @@ static int vmw_surface_dirty_alloc(struct vmw_resource *res)
u32 num_mip;
u32 num_subres;
u32 num_samples;
- size_t dirty_size, acc_size;
- static struct ttm_operation_ctx ctx = {
- .interruptible = false,
- .no_wait_gpu = false
- };
+ size_t dirty_size;
int ret;
if (metadata->array_size)
@@ -1973,14 +1918,6 @@ static int vmw_surface_dirty_alloc(struct vmw_resource *res)
num_subres = num_layers * num_mip;
dirty_size = struct_size(dirty, boxes, num_subres);
- acc_size = ttm_round_pot(dirty_size);
- ret = ttm_mem_global_alloc(vmw_mem_glob(res->dev_priv),
- acc_size, &ctx);
- if (ret) {
- VMW_DEBUG_USER("Out of graphics memory for surface "
- "dirty tracker.\n");
- return ret;
- }
dirty = kvzalloc(dirty_size, GFP_KERNEL);
if (!dirty) {
@@ -1990,13 +1927,12 @@ static int vmw_surface_dirty_alloc(struct vmw_resource *res)
num_samples = max_t(u32, 1, metadata->multisample_count);
ret = vmw_surface_setup_cache(&metadata->base_size, metadata->format,
- num_mip, num_layers, num_samples,
- &dirty->cache);
+ num_mip, num_layers, num_samples,
+ &dirty->cache);
if (ret)
goto out_no_cache;
dirty->num_subres = num_subres;
- dirty->size = acc_size;
res->dirty = (struct vmw_resource_dirty *) dirty;
return 0;
@@ -2004,7 +1940,6 @@ static int vmw_surface_dirty_alloc(struct vmw_resource *res)
out_no_cache:
kvfree(dirty);
out_no_dirty:
- ttm_mem_global_free(vmw_mem_glob(res->dev_priv), acc_size);
return ret;
}
@@ -2015,10 +1950,8 @@ static void vmw_surface_dirty_free(struct vmw_resource *res)
{
struct vmw_surface_dirty *dirty =
(struct vmw_surface_dirty *) res->dirty;
- size_t acc_size = dirty->size;
kvfree(dirty);
- ttm_mem_global_free(vmw_mem_glob(res->dev_priv), acc_size);
res->dirty = NULL;
}
@@ -2051,8 +1984,6 @@ static int vmw_surface_clean(struct vmw_resource *res)
* vmw_gb_surface_define - Define a private GB surface
*
* @dev_priv: Pointer to a device private.
- * @user_accounting_size: Used to track user-space memory usage, set
- * to 0 for kernel mode only memory
* @metadata: Metadata representing the surface to create.
* @user_srf_out: allocated user_srf. Set to NULL on failure.
*
@@ -2062,17 +1993,12 @@ static int vmw_surface_clean(struct vmw_resource *res)
* it available to user mode drivers.
*/
int vmw_gb_surface_define(struct vmw_private *dev_priv,
- uint32_t user_accounting_size,
const struct vmw_surface_metadata *req,
struct vmw_surface **srf_out)
{
struct vmw_surface_metadata *metadata;
struct vmw_user_surface *user_srf;
struct vmw_surface *srf;
- struct ttm_operation_ctx ctx = {
- .interruptible = true,
- .no_wait_gpu = false
- };
u32 sample_count = 1;
u32 num_layers = 1;
int ret;
@@ -2113,22 +2039,13 @@ int vmw_gb_surface_define(struct vmw_private *dev_priv,
if (req->sizes != NULL)
return -EINVAL;
- ret = ttm_mem_global_alloc(vmw_mem_glob(dev_priv),
- user_accounting_size, &ctx);
- if (ret != 0) {
- if (ret != -ERESTARTSYS)
- DRM_ERROR("Out of graphics memory for surface.\n");
- goto out_unlock;
- }
-
user_srf = kzalloc(sizeof(*user_srf), GFP_KERNEL);
if (unlikely(!user_srf)) {
ret = -ENOMEM;
- goto out_no_user_srf;
+ goto out_unlock;
}
*srf_out = &user_srf->srf;
- user_srf->size = user_accounting_size;
user_srf->prime.base.shareable = false;
user_srf->prime.base.tfile = NULL;
@@ -2179,9 +2096,6 @@ int vmw_gb_surface_define(struct vmw_private *dev_priv,
return ret;
-out_no_user_srf:
- ttm_mem_global_free(vmw_mem_glob(dev_priv), user_accounting_size);
-
out_unlock:
return ret;
}
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_system_manager.c b/drivers/gpu/drm/vmwgfx/vmwgfx_system_manager.c
new file mode 100644
index 0000000..b0005b0
--- /dev/null
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_system_manager.c
@@ -0,0 +1,90 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/*
+ * Copyright 2021 VMware, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person
+ * obtaining a copy of this software and associated documentation
+ * files (the "Software"), to deal in the Software without
+ * restriction, including without limitation the rights to use, copy,
+ * modify, merge, publish, distribute, sublicense, and/or sell copies
+ * of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be
+ * included in all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ *
+ */
+
+#include "vmwgfx_drv.h"
+
+#include <drm/ttm/ttm_bo_driver.h>
+#include <drm/ttm/ttm_device.h>
+#include <drm/ttm/ttm_placement.h>
+#include <drm/ttm/ttm_resource.h>
+#include <linux/slab.h>
+
+
+static int vmw_sys_man_alloc(struct ttm_resource_manager *man,
+ struct ttm_buffer_object *bo,
+ const struct ttm_place *place,
+ struct ttm_resource **res)
+{
+ *res = kzalloc(sizeof(**res), GFP_KERNEL);
+ if (!*res)
+ return -ENOMEM;
+
+ ttm_resource_init(bo, place, *res);
+ return 0;
+}
+
+static void vmw_sys_man_free(struct ttm_resource_manager *man,
+ struct ttm_resource *res)
+{
+ kfree(res);
+}
+
+static const struct ttm_resource_manager_func vmw_sys_manager_func = {
+ .alloc = vmw_sys_man_alloc,
+ .free = vmw_sys_man_free,
+};
+
+int vmw_sys_man_init(struct vmw_private *dev_priv)
+{
+ struct ttm_device *bdev = &dev_priv->bdev;
+ struct ttm_resource_manager *man =
+ kzalloc(sizeof(*man), GFP_KERNEL);
+
+ if (!man)
+ return -ENOMEM;
+
+ man->use_tt = true;
+ man->func = &vmw_sys_manager_func;
+
+ ttm_resource_manager_init(man, 0);
+ ttm_set_driver_manager(bdev, VMW_PL_SYSTEM, man);
+ ttm_resource_manager_set_used(man, true);
+ return 0;
+}
+
+void vmw_sys_man_fini(struct vmw_private *dev_priv)
+{
+ struct ttm_resource_manager *man = ttm_manager_type(&dev_priv->bdev,
+ VMW_PL_SYSTEM);
+
+ ttm_resource_manager_evict_all(&dev_priv->bdev, man);
+
+ ttm_resource_manager_set_used(man, false);
+ ttm_resource_manager_cleanup(man);
+
+ ttm_set_driver_manager(&dev_priv->bdev, VMW_PL_SYSTEM, NULL);
+ kfree(man);
+}
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_thp.c b/drivers/gpu/drm/vmwgfx/vmwgfx_thp.c
deleted file mode 100644
index 2a3d346..0000000
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_thp.c
+++ /dev/null
@@ -1,184 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0 OR MIT
-/*
- * Huge page-table-entry support for IO memory.
- *
- * Copyright (C) 2007-2019 Vmware, Inc. All rights reservedd.
- */
-#include "vmwgfx_drv.h"
-#include <drm/ttm/ttm_bo_driver.h>
-#include <drm/ttm/ttm_placement.h>
-#include <drm/ttm/ttm_range_manager.h>
-
-/**
- * struct vmw_thp_manager - Range manager implementing huge page alignment
- *
- * @manager: TTM resource manager.
- * @mm: The underlying range manager. Protected by @lock.
- * @lock: Manager lock.
- */
-struct vmw_thp_manager {
- struct ttm_resource_manager manager;
- struct drm_mm mm;
- spinlock_t lock;
-};
-
-static struct vmw_thp_manager *to_thp_manager(struct ttm_resource_manager *man)
-{
- return container_of(man, struct vmw_thp_manager, manager);
-}
-
-static const struct ttm_resource_manager_func vmw_thp_func;
-
-static int vmw_thp_insert_aligned(struct ttm_buffer_object *bo,
- struct drm_mm *mm, struct drm_mm_node *node,
- unsigned long align_pages,
- const struct ttm_place *place,
- struct ttm_resource *mem,
- unsigned long lpfn,
- enum drm_mm_insert_mode mode)
-{
- if (align_pages >= bo->page_alignment &&
- (!bo->page_alignment || align_pages % bo->page_alignment == 0)) {
- return drm_mm_insert_node_in_range(mm, node,
- mem->num_pages,
- align_pages, 0,
- place->fpfn, lpfn, mode);
- }
-
- return -ENOSPC;
-}
-
-static int vmw_thp_get_node(struct ttm_resource_manager *man,
- struct ttm_buffer_object *bo,
- const struct ttm_place *place,
- struct ttm_resource **res)
-{
- struct vmw_thp_manager *rman = to_thp_manager(man);
- struct drm_mm *mm = &rman->mm;
- struct ttm_range_mgr_node *node;
- unsigned long align_pages;
- unsigned long lpfn;
- enum drm_mm_insert_mode mode = DRM_MM_INSERT_BEST;
- int ret;
-
- node = kzalloc(struct_size(node, mm_nodes, 1), GFP_KERNEL);
- if (!node)
- return -ENOMEM;
-
- ttm_resource_init(bo, place, &node->base);
-
- lpfn = place->lpfn;
- if (!lpfn)
- lpfn = man->size;
-
- mode = DRM_MM_INSERT_BEST;
- if (place->flags & TTM_PL_FLAG_TOPDOWN)
- mode = DRM_MM_INSERT_HIGH;
-
- spin_lock(&rman->lock);
- if (IS_ENABLED(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD)) {
- align_pages = (HPAGE_PUD_SIZE >> PAGE_SHIFT);
- if (node->base.num_pages >= align_pages) {
- ret = vmw_thp_insert_aligned(bo, mm, &node->mm_nodes[0],
- align_pages, place,
- &node->base, lpfn, mode);
- if (!ret)
- goto found_unlock;
- }
- }
-
- align_pages = (HPAGE_PMD_SIZE >> PAGE_SHIFT);
- if (node->base.num_pages >= align_pages) {
- ret = vmw_thp_insert_aligned(bo, mm, &node->mm_nodes[0],
- align_pages, place, &node->base,
- lpfn, mode);
- if (!ret)
- goto found_unlock;
- }
-
- ret = drm_mm_insert_node_in_range(mm, &node->mm_nodes[0],
- node->base.num_pages,
- bo->page_alignment, 0,
- place->fpfn, lpfn, mode);
-found_unlock:
- spin_unlock(&rman->lock);
-
- if (unlikely(ret)) {
- kfree(node);
- } else {
- node->base.start = node->mm_nodes[0].start;
- *res = &node->base;
- }
-
- return ret;
-}
-
-static void vmw_thp_put_node(struct ttm_resource_manager *man,
- struct ttm_resource *res)
-{
- struct ttm_range_mgr_node *node = to_ttm_range_mgr_node(res);
- struct vmw_thp_manager *rman = to_thp_manager(man);
-
- spin_lock(&rman->lock);
- drm_mm_remove_node(&node->mm_nodes[0]);
- spin_unlock(&rman->lock);
-
- kfree(node);
-}
-
-int vmw_thp_init(struct vmw_private *dev_priv)
-{
- struct vmw_thp_manager *rman;
-
- rman = kzalloc(sizeof(*rman), GFP_KERNEL);
- if (!rman)
- return -ENOMEM;
-
- ttm_resource_manager_init(&rman->manager,
- dev_priv->vram_size >> PAGE_SHIFT);
-
- rman->manager.func = &vmw_thp_func;
- drm_mm_init(&rman->mm, 0, rman->manager.size);
- spin_lock_init(&rman->lock);
-
- ttm_set_driver_manager(&dev_priv->bdev, TTM_PL_VRAM, &rman->manager);
- ttm_resource_manager_set_used(&rman->manager, true);
- return 0;
-}
-
-void vmw_thp_fini(struct vmw_private *dev_priv)
-{
- struct ttm_resource_manager *man = ttm_manager_type(&dev_priv->bdev, TTM_PL_VRAM);
- struct vmw_thp_manager *rman = to_thp_manager(man);
- struct drm_mm *mm = &rman->mm;
- int ret;
-
- ttm_resource_manager_set_used(man, false);
-
- ret = ttm_resource_manager_evict_all(&dev_priv->bdev, man);
- if (ret)
- return;
- spin_lock(&rman->lock);
- drm_mm_clean(mm);
- drm_mm_takedown(mm);
- spin_unlock(&rman->lock);
- ttm_resource_manager_cleanup(man);
- ttm_set_driver_manager(&dev_priv->bdev, TTM_PL_VRAM, NULL);
- kfree(rman);
-}
-
-static void vmw_thp_debug(struct ttm_resource_manager *man,
- struct drm_printer *printer)
-{
- struct vmw_thp_manager *rman = to_thp_manager(man);
-
- spin_lock(&rman->lock);
- drm_mm_print(&rman->mm, printer);
- spin_unlock(&rman->lock);
-}
-
-static const struct ttm_resource_manager_func vmw_thp_func = {
- .alloc = vmw_thp_get_node,
- .free = vmw_thp_put_node,
- .debug = vmw_thp_debug
-};
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_ttm_buffer.c b/drivers/gpu/drm/vmwgfx/vmwgfx_ttm_buffer.c
index e899a936..b84ecc6 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_ttm_buffer.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_ttm_buffer.c
@@ -92,6 +92,13 @@ static const struct ttm_place gmr_vram_placement_flags[] = {
}
};
+static const struct ttm_place vmw_sys_placement_flags = {
+ .fpfn = 0,
+ .lpfn = 0,
+ .mem_type = VMW_PL_SYSTEM,
+ .flags = 0
+};
+
struct ttm_placement vmw_vram_gmr_placement = {
.num_placement = 2,
.placement = vram_gmr_placement_flags,
@@ -113,28 +120,11 @@ struct ttm_placement vmw_sys_placement = {
.busy_placement = &sys_placement_flags
};
-static const struct ttm_place evictable_placement_flags[] = {
- {
- .fpfn = 0,
- .lpfn = 0,
- .mem_type = TTM_PL_SYSTEM,
- .flags = 0
- }, {
- .fpfn = 0,
- .lpfn = 0,
- .mem_type = TTM_PL_VRAM,
- .flags = 0
- }, {
- .fpfn = 0,
- .lpfn = 0,
- .mem_type = VMW_PL_GMR,
- .flags = 0
- }, {
- .fpfn = 0,
- .lpfn = 0,
- .mem_type = VMW_PL_MOB,
- .flags = 0
- }
+struct ttm_placement vmw_pt_sys_placement = {
+ .num_placement = 1,
+ .placement = &vmw_sys_placement_flags,
+ .num_busy_placement = 1,
+ .busy_placement = &vmw_sys_placement_flags
};
static const struct ttm_place nonfixed_placement_flags[] = {
@@ -156,13 +146,6 @@ static const struct ttm_place nonfixed_placement_flags[] = {
}
};
-struct ttm_placement vmw_evictable_placement = {
- .num_placement = 4,
- .placement = evictable_placement_flags,
- .num_busy_placement = 1,
- .busy_placement = &sys_placement_flags
-};
-
struct ttm_placement vmw_srf_placement = {
.num_placement = 1,
.num_busy_placement = 2,
@@ -184,19 +167,6 @@ struct ttm_placement vmw_nonfixed_placement = {
.busy_placement = &sys_placement_flags
};
-struct vmw_ttm_tt {
- struct ttm_tt dma_ttm;
- struct vmw_private *dev_priv;
- int gmr_id;
- struct vmw_mob *mob;
- int mem_type;
- struct sg_table sgt;
- struct vmw_sg_table vsgt;
- uint64_t sg_alloc_size;
- bool mapped;
- bool bound;
-};
-
const size_t vmw_tt_size = sizeof(struct vmw_ttm_tt);
/**
@@ -317,17 +287,8 @@ static int vmw_ttm_map_for_dma(struct vmw_ttm_tt *vmw_tt)
static int vmw_ttm_map_dma(struct vmw_ttm_tt *vmw_tt)
{
struct vmw_private *dev_priv = vmw_tt->dev_priv;
- struct ttm_mem_global *glob = vmw_mem_glob(dev_priv);
struct vmw_sg_table *vsgt = &vmw_tt->vsgt;
- struct ttm_operation_ctx ctx = {
- .interruptible = true,
- .no_wait_gpu = false
- };
- struct vmw_piter iter;
- dma_addr_t old;
int ret = 0;
- static size_t sgl_size;
- static size_t sgt_size;
if (vmw_tt->mapped)
return 0;
@@ -336,20 +297,12 @@ static int vmw_ttm_map_dma(struct vmw_ttm_tt *vmw_tt)
vsgt->pages = vmw_tt->dma_ttm.pages;
vsgt->num_pages = vmw_tt->dma_ttm.num_pages;
vsgt->addrs = vmw_tt->dma_ttm.dma_address;
- vsgt->sgt = &vmw_tt->sgt;
+ vsgt->sgt = NULL;
switch (dev_priv->map_mode) {
case vmw_dma_map_bind:
case vmw_dma_map_populate:
- if (unlikely(!sgl_size)) {
- sgl_size = ttm_round_pot(sizeof(struct scatterlist));
- sgt_size = ttm_round_pot(sizeof(struct sg_table));
- }
- vmw_tt->sg_alloc_size = sgt_size + sgl_size * vsgt->num_pages;
- ret = ttm_mem_global_alloc(glob, vmw_tt->sg_alloc_size, &ctx);
- if (unlikely(ret != 0))
- return ret;
-
+ vsgt->sgt = &vmw_tt->sgt;
ret = sg_alloc_table_from_pages_segment(
&vmw_tt->sgt, vsgt->pages, vsgt->num_pages, 0,
(unsigned long)vsgt->num_pages << PAGE_SHIFT,
@@ -357,15 +310,6 @@ static int vmw_ttm_map_dma(struct vmw_ttm_tt *vmw_tt)
if (ret)
goto out_sg_alloc_fail;
- if (vsgt->num_pages > vmw_tt->sgt.orig_nents) {
- uint64_t over_alloc =
- sgl_size * (vsgt->num_pages -
- vmw_tt->sgt.orig_nents);
-
- ttm_mem_global_free(glob, over_alloc);
- vmw_tt->sg_alloc_size -= over_alloc;
- }
-
ret = vmw_ttm_map_for_dma(vmw_tt);
if (unlikely(ret != 0))
goto out_map_fail;
@@ -375,16 +319,6 @@ static int vmw_ttm_map_dma(struct vmw_ttm_tt *vmw_tt)
break;
}
- old = ~((dma_addr_t) 0);
- vmw_tt->vsgt.num_regions = 0;
- for (vmw_piter_start(&iter, vsgt, 0); vmw_piter_next(&iter);) {
- dma_addr_t cur = vmw_piter_dma_addr(&iter);
-
- if (cur != old + PAGE_SIZE)
- vmw_tt->vsgt.num_regions++;
- old = cur;
- }
-
vmw_tt->mapped = true;
return 0;
@@ -392,7 +326,6 @@ static int vmw_ttm_map_dma(struct vmw_ttm_tt *vmw_tt)
sg_free_table(vmw_tt->vsgt.sgt);
vmw_tt->vsgt.sgt = NULL;
out_sg_alloc_fail:
- ttm_mem_global_free(glob, vmw_tt->sg_alloc_size);
return ret;
}
@@ -418,8 +351,6 @@ static void vmw_ttm_unmap_dma(struct vmw_ttm_tt *vmw_tt)
vmw_ttm_unmap_from_dma(vmw_tt);
sg_free_table(vmw_tt->vsgt.sgt);
vmw_tt->vsgt.sgt = NULL;
- ttm_mem_global_free(vmw_mem_glob(dev_priv),
- vmw_tt->sg_alloc_size);
break;
default:
break;
@@ -484,6 +415,9 @@ static int vmw_ttm_bind(struct ttm_device *bdev,
&vmw_be->vsgt, ttm->num_pages,
vmw_be->gmr_id);
break;
+ case VMW_PL_SYSTEM:
+ /* Nothing to be done for a system bind */
+ break;
default:
BUG();
}
@@ -507,6 +441,8 @@ static void vmw_ttm_unbind(struct ttm_device *bdev,
case VMW_PL_MOB:
vmw_mob_unbind(vmw_be->dev_priv, vmw_be->mob);
break;
+ case VMW_PL_SYSTEM:
+ break;
default:
BUG();
}
@@ -534,7 +470,6 @@ static void vmw_ttm_destroy(struct ttm_device *bdev, struct ttm_tt *ttm)
static int vmw_ttm_populate(struct ttm_device *bdev,
struct ttm_tt *ttm, struct ttm_operation_ctx *ctx)
{
- unsigned int i;
int ret;
/* TODO: maybe completely drop this ? */
@@ -542,22 +477,7 @@ static int vmw_ttm_populate(struct ttm_device *bdev,
return 0;
ret = ttm_pool_alloc(&bdev->pool, ttm, ctx);
- if (ret)
- return ret;
- for (i = 0; i < ttm->num_pages; ++i) {
- ret = ttm_mem_global_alloc_page(&ttm_mem_glob, ttm->pages[i],
- PAGE_SIZE, ctx);
- if (ret)
- goto error;
- }
- return 0;
-
-error:
- while (i--)
- ttm_mem_global_free_page(&ttm_mem_glob, ttm->pages[i],
- PAGE_SIZE);
- ttm_pool_free(&bdev->pool, ttm);
return ret;
}
@@ -566,7 +486,6 @@ static void vmw_ttm_unpopulate(struct ttm_device *bdev,
{
struct vmw_ttm_tt *vmw_tt = container_of(ttm, struct vmw_ttm_tt,
dma_ttm);
- unsigned int i;
vmw_ttm_unbind(bdev, ttm);
@@ -577,10 +496,6 @@ static void vmw_ttm_unpopulate(struct ttm_device *bdev,
vmw_ttm_unmap_dma(vmw_tt);
- for (i = 0; i < ttm->num_pages; ++i)
- ttm_mem_global_free_page(&ttm_mem_glob, ttm->pages[i],
- PAGE_SIZE);
-
ttm_pool_free(&bdev->pool, ttm);
}
@@ -624,6 +539,7 @@ static int vmw_ttm_io_mem_reserve(struct ttm_device *bdev, struct ttm_resource *
switch (mem->mem_type) {
case TTM_PL_SYSTEM:
+ case VMW_PL_SYSTEM:
case VMW_PL_GMR:
case VMW_PL_MOB:
return 0;
@@ -670,6 +586,11 @@ static void vmw_swap_notify(struct ttm_buffer_object *bo)
(void) ttm_bo_wait(bo, false, false);
}
+static bool vmw_memtype_is_system(uint32_t mem_type)
+{
+ return mem_type == TTM_PL_SYSTEM || mem_type == VMW_PL_SYSTEM;
+}
+
static int vmw_move(struct ttm_buffer_object *bo,
bool evict,
struct ttm_operation_ctx *ctx,
@@ -680,7 +601,7 @@ static int vmw_move(struct ttm_buffer_object *bo,
struct ttm_resource_manager *new_man = ttm_manager_type(bo->bdev, new_mem->mem_type);
int ret;
- if (new_man->use_tt && new_mem->mem_type != TTM_PL_SYSTEM) {
+ if (new_man->use_tt && !vmw_memtype_is_system(new_mem->mem_type)) {
ret = vmw_ttm_bind(bo->bdev, bo->ttm, new_mem);
if (ret)
return ret;
@@ -689,7 +610,7 @@ static int vmw_move(struct ttm_buffer_object *bo,
vmw_move_notify(bo, bo->resource, new_mem);
if (old_man->use_tt && new_man->use_tt) {
- if (bo->resource->mem_type == TTM_PL_SYSTEM) {
+ if (vmw_memtype_is_system(bo->resource->mem_type)) {
ttm_bo_move_null(bo, new_mem);
return 0;
}
@@ -736,7 +657,7 @@ int vmw_bo_create_and_populate(struct vmw_private *dev_priv,
int ret;
ret = vmw_bo_create_kernel(dev_priv, bo_size,
- &vmw_sys_placement,
+ &vmw_pt_sys_placement,
&bo);
if (unlikely(ret != 0))
return ret;
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_ttm_glue.c b/drivers/gpu/drm/vmwgfx/vmwgfx_ttm_glue.c
index 0a4c340..265f7c4 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_ttm_glue.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_ttm_glue.c
@@ -27,30 +27,44 @@
#include "vmwgfx_drv.h"
-static struct ttm_buffer_object *vmw_bo_vm_lookup(struct ttm_device *bdev,
- unsigned long offset,
- unsigned long pages)
+static int vmw_bo_vm_lookup(struct ttm_device *bdev,
+ struct drm_file *filp,
+ unsigned long offset,
+ unsigned long pages,
+ struct ttm_buffer_object **p_bo)
{
struct vmw_private *dev_priv = container_of(bdev, struct vmw_private, bdev);
struct drm_device *drm = &dev_priv->drm;
struct drm_vma_offset_node *node;
- struct ttm_buffer_object *bo = NULL;
+ int ret;
+
+ *p_bo = NULL;
drm_vma_offset_lock_lookup(bdev->vma_manager);
node = drm_vma_offset_lookup_locked(bdev->vma_manager, offset, pages);
if (likely(node)) {
- bo = container_of(node, struct ttm_buffer_object,
+ *p_bo = container_of(node, struct ttm_buffer_object,
base.vma_node);
- bo = ttm_bo_get_unless_zero(bo);
+ *p_bo = ttm_bo_get_unless_zero(*p_bo);
}
drm_vma_offset_unlock_lookup(bdev->vma_manager);
- if (!bo)
+ if (!*p_bo) {
drm_err(drm, "Could not find buffer object to map\n");
+ return -EINVAL;
+ }
- return bo;
+ if (!drm_vma_node_is_allowed(node, filp)) {
+ ret = -EACCES;
+ goto out_no_access;
+ }
+
+ return 0;
+out_no_access:
+ ttm_bo_put(*p_bo);
+ return ret;
}
int vmw_mmap(struct file *filp, struct vm_area_struct *vma)
@@ -64,7 +78,6 @@ int vmw_mmap(struct file *filp, struct vm_area_struct *vma)
};
struct drm_file *file_priv = filp->private_data;
struct vmw_private *dev_priv = vmw_priv(file_priv->minor->dev);
- struct ttm_object_file *tfile = vmw_fpriv(file_priv)->tfile;
struct ttm_device *bdev = &dev_priv->bdev;
struct ttm_buffer_object *bo;
int ret;
@@ -72,13 +85,9 @@ int vmw_mmap(struct file *filp, struct vm_area_struct *vma)
if (unlikely(vma->vm_pgoff < DRM_FILE_PAGE_OFFSET_START))
return -EINVAL;
- bo = vmw_bo_vm_lookup(bdev, vma->vm_pgoff, vma_pages(vma));
- if (unlikely(!bo))
- return -EINVAL;
-
- ret = vmw_user_bo_verify_access(bo, tfile);
+ ret = vmw_bo_vm_lookup(bdev, file_priv, vma->vm_pgoff, vma_pages(vma), &bo);
if (unlikely(ret != 0))
- goto out_unref;
+ return ret;
ret = ttm_bo_mmap_obj(vma, bo);
if (unlikely(ret != 0))
@@ -99,38 +108,3 @@ int vmw_mmap(struct file *filp, struct vm_area_struct *vma)
return ret;
}
-/* struct vmw_validation_mem callback */
-static int vmw_vmt_reserve(struct vmw_validation_mem *m, size_t size)
-{
- static struct ttm_operation_ctx ctx = {.interruptible = false,
- .no_wait_gpu = false};
- struct vmw_private *dev_priv = container_of(m, struct vmw_private, vvm);
-
- return ttm_mem_global_alloc(vmw_mem_glob(dev_priv), size, &ctx);
-}
-
-/* struct vmw_validation_mem callback */
-static void vmw_vmt_unreserve(struct vmw_validation_mem *m, size_t size)
-{
- struct vmw_private *dev_priv = container_of(m, struct vmw_private, vvm);
-
- return ttm_mem_global_free(vmw_mem_glob(dev_priv), size);
-}
-
-/**
- * vmw_validation_mem_init_ttm - Interface the validation memory tracker
- * to ttm.
- * @dev_priv: Pointer to struct vmw_private. The reason we choose a vmw private
- * rather than a struct vmw_validation_mem is to make sure assumption in the
- * callbacks that struct vmw_private derives from struct vmw_validation_mem
- * holds true.
- * @gran: The recommended allocation granularity
- */
-void vmw_validation_mem_init_ttm(struct vmw_private *dev_priv, size_t gran)
-{
- struct vmw_validation_mem *vvm = &dev_priv->vvm;
-
- vvm->reserve_mem = vmw_vmt_reserve;
- vvm->unreserve_mem = vmw_vmt_unreserve;
- vvm->gran = gran;
-}
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_va.c b/drivers/gpu/drm/vmwgfx/vmwgfx_va.c
index ebc1d83..6ad744a 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_va.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_va.c
@@ -117,7 +117,7 @@ int vmw_stream_unref_ioctl(struct drm_device *dev, void *data,
struct drm_vmw_stream_arg *arg = (struct drm_vmw_stream_arg *)data;
return ttm_ref_object_base_unref(vmw_fpriv(file_priv)->tfile,
- arg->stream_id, TTM_REF_USAGE);
+ arg->stream_id);
}
/**
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_validation.c b/drivers/gpu/drm/vmwgfx/vmwgfx_validation.c
index b09094b..f468910 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_validation.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_validation.c
@@ -29,6 +29,9 @@
#include "vmwgfx_validation.h"
#include "vmwgfx_drv.h"
+
+#define VMWGFX_VALIDATION_MEM_GRAN (16*PAGE_SIZE)
+
/**
* struct vmw_validation_bo_node - Buffer object validation metadata.
* @base: Metadata used for TTM reservation- and validation.
@@ -43,7 +46,7 @@
*/
struct vmw_validation_bo_node {
struct ttm_validate_buffer base;
- struct drm_hash_item hash;
+ struct vmwgfx_hash_item hash;
unsigned int coherent_count;
u32 as_mob : 1;
u32 cpu_blit : 1;
@@ -72,7 +75,7 @@ struct vmw_validation_bo_node {
*/
struct vmw_validation_res_node {
struct list_head head;
- struct drm_hash_item hash;
+ struct vmwgfx_hash_item hash;
struct vmw_resource *res;
struct vmw_buffer_object *new_backup;
unsigned long new_backup_offset;
@@ -113,13 +116,8 @@ void *vmw_validation_mem_alloc(struct vmw_validation_context *ctx,
struct page *page;
if (ctx->vm && ctx->vm_size_left < PAGE_SIZE) {
- int ret = ctx->vm->reserve_mem(ctx->vm, ctx->vm->gran);
-
- if (ret)
- return NULL;
-
- ctx->vm_size_left += ctx->vm->gran;
- ctx->total_mem += ctx->vm->gran;
+ ctx->vm_size_left += VMWGFX_VALIDATION_MEM_GRAN;
+ ctx->total_mem += VMWGFX_VALIDATION_MEM_GRAN;
}
page = alloc_page(GFP_KERNEL | __GFP_ZERO);
@@ -159,7 +157,6 @@ static void vmw_validation_mem_free(struct vmw_validation_context *ctx)
ctx->mem_size_left = 0;
if (ctx->vm && ctx->total_mem) {
- ctx->vm->unreserve_mem(ctx->vm, ctx->total_mem);
ctx->total_mem = 0;
ctx->vm_size_left = 0;
}
@@ -184,9 +181,9 @@ vmw_validation_find_bo_dup(struct vmw_validation_context *ctx,
return NULL;
if (ctx->ht) {
- struct drm_hash_item *hash;
+ struct vmwgfx_hash_item *hash;
- if (!drm_ht_find_item(ctx->ht, (unsigned long) vbo, &hash))
+ if (!vmwgfx_ht_find_item(ctx->ht, (unsigned long) vbo, &hash))
bo_node = container_of(hash, typeof(*bo_node), hash);
} else {
struct vmw_validation_bo_node *entry;
@@ -221,9 +218,9 @@ vmw_validation_find_res_dup(struct vmw_validation_context *ctx,
return NULL;
if (ctx->ht) {
- struct drm_hash_item *hash;
+ struct vmwgfx_hash_item *hash;
- if (!drm_ht_find_item(ctx->ht, (unsigned long) res, &hash))
+ if (!vmwgfx_ht_find_item(ctx->ht, (unsigned long) res, &hash))
res_node = container_of(hash, typeof(*res_node), hash);
} else {
struct vmw_validation_res_node *entry;
@@ -280,7 +277,7 @@ int vmw_validation_add_bo(struct vmw_validation_context *ctx,
if (ctx->ht) {
bo_node->hash.key = (unsigned long) vbo;
- ret = drm_ht_insert_item(ctx->ht, &bo_node->hash);
+ ret = vmwgfx_ht_insert_item(ctx->ht, &bo_node->hash);
if (ret) {
DRM_ERROR("Failed to initialize a buffer "
"validation entry.\n");
@@ -335,7 +332,7 @@ int vmw_validation_add_resource(struct vmw_validation_context *ctx,
if (ctx->ht) {
node->hash.key = (unsigned long) res;
- ret = drm_ht_insert_item(ctx->ht, &node->hash);
+ ret = vmwgfx_ht_insert_item(ctx->ht, &node->hash);
if (ret) {
DRM_ERROR("Failed to initialize a resource validation "
"entry.\n");
@@ -688,13 +685,13 @@ void vmw_validation_drop_ht(struct vmw_validation_context *ctx)
return;
list_for_each_entry(entry, &ctx->bo_list, base.head)
- (void) drm_ht_remove_item(ctx->ht, &entry->hash);
+ (void) vmwgfx_ht_remove_item(ctx->ht, &entry->hash);
list_for_each_entry(val, &ctx->resource_list, head)
- (void) drm_ht_remove_item(ctx->ht, &val->hash);
+ (void) vmwgfx_ht_remove_item(ctx->ht, &val->hash);
list_for_each_entry(val, &ctx->resource_ctx_list, head)
- (void) drm_ht_remove_item(ctx->ht, &val->hash);
+ (void) vmwgfx_ht_remove_item(ctx->ht, &val->hash);
ctx->ht = NULL;
}
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_validation.h b/drivers/gpu/drm/vmwgfx/vmwgfx_validation.h
index 739906d..f21df05 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_validation.h
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_validation.h
@@ -31,29 +31,15 @@
#include <linux/list.h>
#include <linux/ww_mutex.h>
-#include <drm/drm_hashtab.h>
#include <drm/ttm/ttm_execbuf_util.h>
+#include "vmwgfx_hashtab.h"
+
#define VMW_RES_DIRTY_NONE 0
#define VMW_RES_DIRTY_SET BIT(0)
#define VMW_RES_DIRTY_CLEAR BIT(1)
/**
- * struct vmw_validation_mem - Custom interface to provide memory reservations
- * for the validation code.
- * @reserve_mem: Callback to reserve memory
- * @unreserve_mem: Callback to unreserve memory
- * @gran: Reservation granularity. Contains a hint how much memory should
- * be reserved in each call to @reserve_mem(). A slow implementation may want
- * reservation to be done in large batches.
- */
-struct vmw_validation_mem {
- int (*reserve_mem)(struct vmw_validation_mem *m, size_t size);
- void (*unreserve_mem)(struct vmw_validation_mem *m, size_t size);
- size_t gran;
-};
-
-/**
* struct vmw_validation_context - Per command submission validation context
* @ht: Hash table used to find resource- or buffer object duplicates
* @resource_list: List head for resource validation metadata
@@ -73,7 +59,7 @@ struct vmw_validation_mem {
* @total_mem: Amount of reserved memory.
*/
struct vmw_validation_context {
- struct drm_open_hash *ht;
+ struct vmwgfx_open_hash *ht;
struct list_head resource_list;
struct list_head resource_ctx_list;
struct list_head bo_list;
@@ -129,21 +115,6 @@ vmw_validation_has_bos(struct vmw_validation_context *ctx)
}
/**
- * vmw_validation_set_val_mem - Register a validation mem object for
- * validation memory reservation
- * @ctx: The validation context
- * @vm: Pointer to a struct vmw_validation_mem
- *
- * Must be set before the first attempt to allocate validation memory.
- */
-static inline void
-vmw_validation_set_val_mem(struct vmw_validation_context *ctx,
- struct vmw_validation_mem *vm)
-{
- ctx->vm = vm;
-}
-
-/**
* vmw_validation_set_ht - Register a hash table for duplicate finding
* @ctx: The validation context
* @ht: Pointer to a hash table to use for duplicate finding
@@ -151,7 +122,7 @@ vmw_validation_set_val_mem(struct vmw_validation_context *ctx,
* available at validation context declaration time
*/
static inline void vmw_validation_set_ht(struct vmw_validation_context *ctx,
- struct drm_open_hash *ht)
+ struct vmwgfx_open_hash *ht)
{
ctx->ht = ht;
}
@@ -190,22 +161,6 @@ vmw_validation_bo_fence(struct vmw_validation_context *ctx,
}
/**
- * vmw_validation_context_init - Initialize a validation context
- * @ctx: Pointer to the validation context to initialize
- *
- * This function initializes a validation context with @merge_dups set
- * to false
- */
-static inline void
-vmw_validation_context_init(struct vmw_validation_context *ctx)
-{
- memset(ctx, 0, sizeof(*ctx));
- INIT_LIST_HEAD(&ctx->resource_list);
- INIT_LIST_HEAD(&ctx->resource_ctx_list);
- INIT_LIST_HEAD(&ctx->bo_list);
-}
-
-/**
* vmw_validation_align - Align a validation memory allocation
* @val: The size to be aligned
*
diff --git a/drivers/gpu/drm/xen/xen_drm_front.c b/drivers/gpu/drm/xen/xen_drm_front.c
index bc76053..e63088c 100644
--- a/drivers/gpu/drm/xen/xen_drm_front.c
+++ b/drivers/gpu/drm/xen/xen_drm_front.c
@@ -469,19 +469,7 @@ static void xen_drm_drv_release(struct drm_device *dev)
kfree(drm_info);
}
-static const struct file_operations xen_drm_dev_fops = {
- .owner = THIS_MODULE,
- .open = drm_open,
- .release = drm_release,
- .unlocked_ioctl = drm_ioctl,
-#ifdef CONFIG_COMPAT
- .compat_ioctl = drm_compat_ioctl,
-#endif
- .poll = drm_poll,
- .read = drm_read,
- .llseek = no_llseek,
- .mmap = xen_drm_front_gem_mmap,
-};
+DEFINE_DRM_GEM_FOPS(xen_drm_dev_fops);
static const struct drm_driver xen_drm_driver = {
.driver_features = DRIVER_GEM | DRIVER_MODESET | DRIVER_ATOMIC,
@@ -489,7 +477,7 @@ static const struct drm_driver xen_drm_driver = {
.prime_handle_to_fd = drm_gem_prime_handle_to_fd,
.prime_fd_to_handle = drm_gem_prime_fd_to_handle,
.gem_prime_import_sg_table = xen_drm_front_gem_import_sg_table,
- .gem_prime_mmap = xen_drm_front_gem_prime_mmap,
+ .gem_prime_mmap = drm_gem_prime_mmap,
.dumb_create = xen_drm_drv_dumb_create,
.fops = &xen_drm_dev_fops,
.name = "xendrm-du",
diff --git a/drivers/gpu/drm/xen/xen_drm_front_gem.c b/drivers/gpu/drm/xen/xen_drm_front_gem.c
index b293c67..dd358ba 100644
--- a/drivers/gpu/drm/xen/xen_drm_front_gem.c
+++ b/drivers/gpu/drm/xen/xen_drm_front_gem.c
@@ -57,6 +57,47 @@ static void gem_free_pages_array(struct xen_gem_object *xen_obj)
xen_obj->pages = NULL;
}
+static int xen_drm_front_gem_object_mmap(struct drm_gem_object *gem_obj,
+ struct vm_area_struct *vma)
+{
+ struct xen_gem_object *xen_obj = to_xen_gem_obj(gem_obj);
+ int ret;
+
+ vma->vm_ops = gem_obj->funcs->vm_ops;
+
+ /*
+ * Clear the VM_PFNMAP flag that was set by drm_gem_mmap(), and set the
+ * vm_pgoff (used as a fake buffer offset by DRM) to 0 as we want to map
+ * the whole buffer.
+ */
+ vma->vm_flags &= ~VM_PFNMAP;
+ vma->vm_flags |= VM_MIXEDMAP;
+ vma->vm_pgoff = 0;
+
+ /*
+ * According to Xen on ARM ABI (xen/include/public/arch-arm.h):
+ * all memory which is shared with other entities in the system
+ * (including the hypervisor and other guests) must reside in memory
+ * which is mapped as Normal Inner Write-Back Outer Write-Back
+ * Inner-Shareable.
+ */
+ vma->vm_page_prot = vm_get_page_prot(vma->vm_flags);
+
+ /*
+ * vm_operations_struct.fault handler will be called if CPU access
+ * to VM is here. For GPUs this isn't the case, because CPU doesn't
+ * touch the memory. Insert pages now, so both CPU and GPU are happy.
+ *
+ * FIXME: as we insert all the pages now then no .fault handler must
+ * be called, so don't provide one
+ */
+ ret = vm_map_pages(vma, xen_obj->pages, xen_obj->num_pages);
+ if (ret < 0)
+ DRM_ERROR("Failed to map pages into vma: %d\n", ret);
+
+ return ret;
+}
+
static const struct vm_operations_struct xen_drm_drv_vm_ops = {
.open = drm_gem_vm_open,
.close = drm_gem_vm_close,
@@ -67,6 +108,7 @@ static const struct drm_gem_object_funcs xen_drm_front_gem_object_funcs = {
.get_sg_table = xen_drm_front_gem_get_sg_table,
.vmap = xen_drm_front_gem_prime_vmap,
.vunmap = xen_drm_front_gem_prime_vunmap,
+ .mmap = xen_drm_front_gem_object_mmap,
.vm_ops = &xen_drm_drv_vm_ops,
};
@@ -238,58 +280,6 @@ xen_drm_front_gem_import_sg_table(struct drm_device *dev,
return &xen_obj->base;
}
-static int gem_mmap_obj(struct xen_gem_object *xen_obj,
- struct vm_area_struct *vma)
-{
- int ret;
-
- /*
- * clear the VM_PFNMAP flag that was set by drm_gem_mmap(), and set the
- * vm_pgoff (used as a fake buffer offset by DRM) to 0 as we want to map
- * the whole buffer.
- */
- vma->vm_flags &= ~VM_PFNMAP;
- vma->vm_flags |= VM_MIXEDMAP;
- vma->vm_pgoff = 0;
- /*
- * According to Xen on ARM ABI (xen/include/public/arch-arm.h):
- * all memory which is shared with other entities in the system
- * (including the hypervisor and other guests) must reside in memory
- * which is mapped as Normal Inner Write-Back Outer Write-Back
- * Inner-Shareable.
- */
- vma->vm_page_prot = vm_get_page_prot(vma->vm_flags);
-
- /*
- * vm_operations_struct.fault handler will be called if CPU access
- * to VM is here. For GPUs this isn't the case, because CPU
- * doesn't touch the memory. Insert pages now, so both CPU and GPU are
- * happy.
- * FIXME: as we insert all the pages now then no .fault handler must
- * be called, so don't provide one
- */
- ret = vm_map_pages(vma, xen_obj->pages, xen_obj->num_pages);
- if (ret < 0)
- DRM_ERROR("Failed to map pages into vma: %d\n", ret);
-
- return ret;
-}
-
-int xen_drm_front_gem_mmap(struct file *filp, struct vm_area_struct *vma)
-{
- struct xen_gem_object *xen_obj;
- struct drm_gem_object *gem_obj;
- int ret;
-
- ret = drm_gem_mmap(filp, vma);
- if (ret < 0)
- return ret;
-
- gem_obj = vma->vm_private_data;
- xen_obj = to_xen_gem_obj(gem_obj);
- return gem_mmap_obj(xen_obj, vma);
-}
-
int xen_drm_front_gem_prime_vmap(struct drm_gem_object *gem_obj, struct dma_buf_map *map)
{
struct xen_gem_object *xen_obj = to_xen_gem_obj(gem_obj);
@@ -313,17 +303,3 @@ void xen_drm_front_gem_prime_vunmap(struct drm_gem_object *gem_obj,
{
vunmap(map->vaddr);
}
-
-int xen_drm_front_gem_prime_mmap(struct drm_gem_object *gem_obj,
- struct vm_area_struct *vma)
-{
- struct xen_gem_object *xen_obj;
- int ret;
-
- ret = drm_gem_mmap_obj(gem_obj, gem_obj->size, vma);
- if (ret < 0)
- return ret;
-
- xen_obj = to_xen_gem_obj(gem_obj);
- return gem_mmap_obj(xen_obj, vma);
-}
diff --git a/drivers/gpu/drm/xen/xen_drm_front_gem.h b/drivers/gpu/drm/xen/xen_drm_front_gem.h
index a4e67d0..eaea470 100644
--- a/drivers/gpu/drm/xen/xen_drm_front_gem.h
+++ b/drivers/gpu/drm/xen/xen_drm_front_gem.h
@@ -15,9 +15,7 @@ struct dma_buf_attachment;
struct dma_buf_map;
struct drm_device;
struct drm_gem_object;
-struct file;
struct sg_table;
-struct vm_area_struct;
struct drm_gem_object *xen_drm_front_gem_create(struct drm_device *dev,
size_t size);
@@ -33,15 +31,10 @@ struct page **xen_drm_front_gem_get_pages(struct drm_gem_object *obj);
void xen_drm_front_gem_free_object_unlocked(struct drm_gem_object *gem_obj);
-int xen_drm_front_gem_mmap(struct file *filp, struct vm_area_struct *vma);
-
int xen_drm_front_gem_prime_vmap(struct drm_gem_object *gem_obj,
struct dma_buf_map *map);
void xen_drm_front_gem_prime_vunmap(struct drm_gem_object *gem_obj,
struct dma_buf_map *map);
-int xen_drm_front_gem_prime_mmap(struct drm_gem_object *gem_obj,
- struct vm_area_struct *vma);
-
#endif /* __XEN_DRM_FRONT_GEM_H */
diff --git a/drivers/gpu/drm/xlnx/Kconfig b/drivers/gpu/drm/xlnx/Kconfig
index c3d0826..d8d38d8 100644
--- a/drivers/gpu/drm/xlnx/Kconfig
+++ b/drivers/gpu/drm/xlnx/Kconfig
@@ -7,7 +7,6 @@
depends on XILINX_ZYNQMP_DPDMA
select DMA_ENGINE
select DRM_GEM_CMA_HELPER
- select DRM_KMS_CMA_HELPER
select DRM_KMS_HELPER
select GENERIC_PHY
help
diff --git a/drivers/gpu/host1x/Kconfig b/drivers/gpu/host1x/Kconfig
index 6dab94a..6815b4d 100644
--- a/drivers/gpu/host1x/Kconfig
+++ b/drivers/gpu/host1x/Kconfig
@@ -2,6 +2,7 @@
config TEGRA_HOST1X
tristate "NVIDIA Tegra host1x driver"
depends on ARCH_TEGRA || (ARM && COMPILE_TEST)
+ select DMA_SHARED_BUFFER
select IOMMU_IOVA
help
Driver for the NVIDIA Tegra host1x hardware.
diff --git a/drivers/gpu/host1x/bus.c b/drivers/gpu/host1x/bus.c
index 218e371..bdee16a 100644
--- a/drivers/gpu/host1x/bus.c
+++ b/drivers/gpu/host1x/bus.c
@@ -5,6 +5,7 @@
*/
#include <linux/debugfs.h>
+#include <linux/dma-mapping.h>
#include <linux/host1x.h>
#include <linux/of.h>
#include <linux/seq_file.h>
@@ -742,6 +743,7 @@ EXPORT_SYMBOL(host1x_driver_unregister);
*/
void __host1x_client_init(struct host1x_client *client, struct lock_class_key *key)
{
+ host1x_bo_cache_init(&client->cache);
INIT_LIST_HEAD(&client->list);
__mutex_init(&client->lock, "host1x client lock", key);
client->usecount = 0;
@@ -761,7 +763,6 @@ EXPORT_SYMBOL(host1x_client_exit);
/**
* __host1x_client_register() - register a host1x client
* @client: host1x client
- * @key: lock class key for the client-specific mutex
*
* Registers a host1x client with each host1x controller instance. Note that
* each client will only match their parent host1x controller and will only be
@@ -830,6 +831,8 @@ int host1x_client_unregister(struct host1x_client *client)
mutex_unlock(&clients_lock);
+ host1x_bo_cache_destroy(&client->cache);
+
return 0;
}
EXPORT_SYMBOL(host1x_client_unregister);
@@ -904,3 +907,78 @@ int host1x_client_resume(struct host1x_client *client)
return err;
}
EXPORT_SYMBOL(host1x_client_resume);
+
+struct host1x_bo_mapping *host1x_bo_pin(struct device *dev, struct host1x_bo *bo,
+ enum dma_data_direction dir,
+ struct host1x_bo_cache *cache)
+{
+ struct host1x_bo_mapping *mapping;
+
+ if (cache) {
+ mutex_lock(&cache->lock);
+
+ list_for_each_entry(mapping, &cache->mappings, entry) {
+ if (mapping->bo == bo && mapping->direction == dir) {
+ kref_get(&mapping->ref);
+ goto unlock;
+ }
+ }
+ }
+
+ mapping = bo->ops->pin(dev, bo, dir);
+ if (IS_ERR(mapping))
+ goto unlock;
+
+ spin_lock(&mapping->bo->lock);
+ list_add_tail(&mapping->list, &bo->mappings);
+ spin_unlock(&mapping->bo->lock);
+
+ if (cache) {
+ INIT_LIST_HEAD(&mapping->entry);
+ mapping->cache = cache;
+
+ list_add_tail(&mapping->entry, &cache->mappings);
+
+ /* bump reference count to track the copy in the cache */
+ kref_get(&mapping->ref);
+ }
+
+unlock:
+ if (cache)
+ mutex_unlock(&cache->lock);
+
+ return mapping;
+}
+EXPORT_SYMBOL(host1x_bo_pin);
+
+static void __host1x_bo_unpin(struct kref *ref)
+{
+ struct host1x_bo_mapping *mapping = to_host1x_bo_mapping(ref);
+
+ /*
+ * When the last reference of the mapping goes away, make sure to remove the mapping from
+ * the cache.
+ */
+ if (mapping->cache)
+ list_del(&mapping->entry);
+
+ spin_lock(&mapping->bo->lock);
+ list_del(&mapping->list);
+ spin_unlock(&mapping->bo->lock);
+
+ mapping->bo->ops->unpin(mapping);
+}
+
+void host1x_bo_unpin(struct host1x_bo_mapping *mapping)
+{
+ struct host1x_bo_cache *cache = mapping->cache;
+
+ if (cache)
+ mutex_lock(&cache->lock);
+
+ kref_put(&mapping->ref, __host1x_bo_unpin);
+
+ if (cache)
+ mutex_unlock(&cache->lock);
+}
+EXPORT_SYMBOL(host1x_bo_unpin);
diff --git a/drivers/gpu/host1x/channel.c b/drivers/gpu/host1x/channel.c
index 4cd212b..2a9a3a8 100644
--- a/drivers/gpu/host1x/channel.c
+++ b/drivers/gpu/host1x/channel.c
@@ -75,6 +75,14 @@ struct host1x_channel *host1x_channel_get_index(struct host1x *host,
return ch;
}
+void host1x_channel_stop(struct host1x_channel *channel)
+{
+ struct host1x *host = dev_get_drvdata(channel->dev->parent);
+
+ host1x_hw_cdma_stop(host, &channel->cdma);
+}
+EXPORT_SYMBOL(host1x_channel_stop);
+
static void release_channel(struct kref *kref)
{
struct host1x_channel *channel =
diff --git a/drivers/gpu/host1x/debug.c b/drivers/gpu/host1x/debug.c
index 8a14880..18d9c8d 100644
--- a/drivers/gpu/host1x/debug.c
+++ b/drivers/gpu/host1x/debug.c
@@ -7,6 +7,7 @@
*/
#include <linux/debugfs.h>
+#include <linux/pm_runtime.h>
#include <linux/seq_file.h>
#include <linux/uaccess.h>
@@ -52,6 +53,11 @@ static int show_channel(struct host1x_channel *ch, void *data, bool show_fifo)
{
struct host1x *m = dev_get_drvdata(ch->dev->parent);
struct output *o = data;
+ int err;
+
+ err = pm_runtime_resume_and_get(m->dev);
+ if (err < 0)
+ return err;
mutex_lock(&ch->cdma.lock);
mutex_lock(&debug_lock);
@@ -64,6 +70,8 @@ static int show_channel(struct host1x_channel *ch, void *data, bool show_fifo)
mutex_unlock(&debug_lock);
mutex_unlock(&ch->cdma.lock);
+ pm_runtime_put(m->dev);
+
return 0;
}
@@ -71,9 +79,14 @@ static void show_syncpts(struct host1x *m, struct output *o)
{
struct list_head *pos;
unsigned int i;
+ int err;
host1x_debug_output(o, "---- syncpts ----\n");
+ err = pm_runtime_resume_and_get(m->dev);
+ if (err < 0)
+ return;
+
for (i = 0; i < host1x_syncpt_nb_pts(m); i++) {
u32 max = host1x_syncpt_read_max(m->syncpt + i);
u32 min = host1x_syncpt_load(m->syncpt + i);
@@ -101,6 +114,8 @@ static void show_syncpts(struct host1x *m, struct output *o)
base_val);
}
+ pm_runtime_put(m->dev);
+
host1x_debug_output(o, "\n");
}
diff --git a/drivers/gpu/host1x/dev.c b/drivers/gpu/host1x/dev.c
index fbb6447..6994f8c 100644
--- a/drivers/gpu/host1x/dev.c
+++ b/drivers/gpu/host1x/dev.c
@@ -6,18 +6,26 @@
*/
#include <linux/clk.h>
+#include <linux/delay.h>
#include <linux/dma-mapping.h>
#include <linux/io.h>
#include <linux/list.h>
#include <linux/module.h>
#include <linux/of_device.h>
#include <linux/of.h>
+#include <linux/pm_runtime.h>
#include <linux/slab.h>
+#include <soc/tegra/common.h>
+
#define CREATE_TRACE_POINTS
#include <trace/events/host1x.h>
#undef CREATE_TRACE_POINTS
+#if IS_ENABLED(CONFIG_ARM_DMA_USE_IOMMU)
+#include <asm/dma-iommu.h>
+#endif
+
#include "bus.h"
#include "channel.h"
#include "debug.h"
@@ -132,6 +140,12 @@ static const struct host1x_sid_entry tegra186_sid_table[] = {
.offset = 0x30,
.limit = 0x34
},
+ {
+ /* NVDEC */
+ .base = 0x1b00,
+ .offset = 0x30,
+ .limit = 0x34
+ },
};
static const struct host1x_info host1x06_info = {
@@ -156,6 +170,18 @@ static const struct host1x_sid_entry tegra194_sid_table[] = {
.offset = 0x30,
.limit = 0x34
},
+ {
+ /* NVDEC */
+ .base = 0x1b00,
+ .offset = 0x30,
+ .limit = 0x34
+ },
+ {
+ /* NVDEC1 */
+ .base = 0x1bc0,
+ .offset = 0x30,
+ .limit = 0x34
+ },
};
static const struct host1x_info host1x07_info = {
@@ -190,6 +216,9 @@ static void host1x_setup_sid_table(struct host1x *host)
const struct host1x_info *info = host->info;
unsigned int i;
+ if (!info->has_hypervisor)
+ return;
+
for (i = 0; i < info->num_sid_entries; i++) {
const struct host1x_sid_entry *entry = &info->sid_table[i];
@@ -238,6 +267,17 @@ static struct iommu_domain *host1x_iommu_attach(struct host1x *host)
struct iommu_domain *domain = iommu_get_domain_for_dev(host->dev);
int err;
+#if IS_ENABLED(CONFIG_ARM_DMA_USE_IOMMU)
+ if (host->dev->archdata.mapping) {
+ struct dma_iommu_mapping *mapping =
+ to_dma_iommu_mapping(host->dev);
+ arm_iommu_detach_device(host->dev);
+ arm_iommu_release_mapping(mapping);
+
+ domain = iommu_get_domain_for_dev(host->dev);
+ }
+#endif
+
/*
* We may not always want to enable IOMMU support (for example if the
* host1x firewall is already enabled and we don't support addressing
@@ -347,6 +387,27 @@ static void host1x_iommu_exit(struct host1x *host)
}
}
+static int host1x_get_resets(struct host1x *host)
+{
+ int err;
+
+ host->resets[0].id = "mc";
+ host->resets[1].id = "host1x";
+ host->nresets = ARRAY_SIZE(host->resets);
+
+ err = devm_reset_control_bulk_get_optional_exclusive_released(
+ host->dev, host->nresets, host->resets);
+ if (err) {
+ dev_err(host->dev, "failed to get reset: %d\n", err);
+ return err;
+ }
+
+ if (WARN_ON(!host->resets[1].rstc))
+ return -ENOENT;
+
+ return 0;
+}
+
static int host1x_probe(struct platform_device *pdev)
{
struct host1x *host;
@@ -386,6 +447,7 @@ static int host1x_probe(struct platform_device *pdev)
if (syncpt_irq < 0)
return syncpt_irq;
+ host1x_bo_cache_init(&host->cache);
mutex_init(&host->devices_lock);
INIT_LIST_HEAD(&host->devices);
INIT_LIST_HEAD(&host->list);
@@ -423,12 +485,9 @@ static int host1x_probe(struct platform_device *pdev)
return err;
}
- host->rst = devm_reset_control_get(&pdev->dev, "host1x");
- if (IS_ERR(host->rst)) {
- err = PTR_ERR(host->rst);
- dev_err(&pdev->dev, "failed to get reset: %d\n", err);
+ err = host1x_get_resets(host);
+ if (err)
return err;
- }
err = host1x_iommu_init(host);
if (err < 0) {
@@ -443,22 +502,10 @@ static int host1x_probe(struct platform_device *pdev)
goto iommu_exit;
}
- err = clk_prepare_enable(host->clk);
- if (err < 0) {
- dev_err(&pdev->dev, "failed to enable clock\n");
- goto free_channels;
- }
-
- err = reset_control_deassert(host->rst);
- if (err < 0) {
- dev_err(&pdev->dev, "failed to deassert reset: %d\n", err);
- goto unprepare_disable;
- }
-
err = host1x_syncpt_init(host);
if (err) {
dev_err(&pdev->dev, "failed to initialize syncpts\n");
- goto reset_assert;
+ goto free_channels;
}
err = host1x_intr_init(host, syncpt_irq);
@@ -467,10 +514,18 @@ static int host1x_probe(struct platform_device *pdev)
goto deinit_syncpt;
}
- host1x_debug_init(host);
+ pm_runtime_enable(&pdev->dev);
- if (host->info->has_hypervisor)
- host1x_setup_sid_table(host);
+ err = devm_tegra_core_dev_init_opp_table_common(&pdev->dev);
+ if (err)
+ goto pm_disable;
+
+ /* the driver's code isn't ready yet for the dynamic RPM */
+ err = pm_runtime_resume_and_get(&pdev->dev);
+ if (err)
+ goto pm_disable;
+
+ host1x_debug_init(host);
err = host1x_register(host);
if (err < 0)
@@ -486,13 +541,14 @@ static int host1x_probe(struct platform_device *pdev)
host1x_unregister(host);
deinit_debugfs:
host1x_debug_deinit(host);
+
+ pm_runtime_put_sync_suspend(&pdev->dev);
+pm_disable:
+ pm_runtime_disable(&pdev->dev);
+
host1x_intr_deinit(host);
deinit_syncpt:
host1x_syncpt_deinit(host);
-reset_assert:
- reset_control_assert(host->rst);
-unprepare_disable:
- clk_disable_unprepare(host->clk);
free_channels:
host1x_channel_list_free(&host->channel_list);
iommu_exit:
@@ -507,19 +563,94 @@ static int host1x_remove(struct platform_device *pdev)
host1x_unregister(host);
host1x_debug_deinit(host);
+
+ pm_runtime_force_suspend(&pdev->dev);
+
host1x_intr_deinit(host);
host1x_syncpt_deinit(host);
- reset_control_assert(host->rst);
- clk_disable_unprepare(host->clk);
host1x_iommu_exit(host);
+ host1x_bo_cache_destroy(&host->cache);
return 0;
}
+static int __maybe_unused host1x_runtime_suspend(struct device *dev)
+{
+ struct host1x *host = dev_get_drvdata(dev);
+ int err;
+
+ host1x_intr_stop(host);
+ host1x_syncpt_save(host);
+
+ err = reset_control_bulk_assert(host->nresets, host->resets);
+ if (err) {
+ dev_err(dev, "failed to assert reset: %d\n", err);
+ goto resume_host1x;
+ }
+
+ usleep_range(1000, 2000);
+
+ clk_disable_unprepare(host->clk);
+ reset_control_bulk_release(host->nresets, host->resets);
+
+ return 0;
+
+resume_host1x:
+ host1x_setup_sid_table(host);
+ host1x_syncpt_restore(host);
+ host1x_intr_start(host);
+
+ return err;
+}
+
+static int __maybe_unused host1x_runtime_resume(struct device *dev)
+{
+ struct host1x *host = dev_get_drvdata(dev);
+ int err;
+
+ err = reset_control_bulk_acquire(host->nresets, host->resets);
+ if (err) {
+ dev_err(dev, "failed to acquire reset: %d\n", err);
+ return err;
+ }
+
+ err = clk_prepare_enable(host->clk);
+ if (err) {
+ dev_err(dev, "failed to enable clock: %d\n", err);
+ goto release_reset;
+ }
+
+ err = reset_control_bulk_deassert(host->nresets, host->resets);
+ if (err < 0) {
+ dev_err(dev, "failed to deassert reset: %d\n", err);
+ goto disable_clk;
+ }
+
+ host1x_setup_sid_table(host);
+ host1x_syncpt_restore(host);
+ host1x_intr_start(host);
+
+ return 0;
+
+disable_clk:
+ clk_disable_unprepare(host->clk);
+release_reset:
+ reset_control_bulk_release(host->nresets, host->resets);
+
+ return err;
+}
+
+static const struct dev_pm_ops host1x_pm_ops = {
+ SET_RUNTIME_PM_OPS(host1x_runtime_suspend, host1x_runtime_resume,
+ NULL)
+ /* TODO: add system suspend-resume once driver will be ready for that */
+};
+
static struct platform_driver tegra_host1x_driver = {
.driver = {
.name = "tegra-host1x",
.of_match_table = host1x_of_match,
+ .pm = &host1x_pm_ops,
},
.probe = host1x_probe,
.remove = host1x_remove,
diff --git a/drivers/gpu/host1x/dev.h b/drivers/gpu/host1x/dev.h
index fa6d4bc..ca4b082 100644
--- a/drivers/gpu/host1x/dev.h
+++ b/drivers/gpu/host1x/dev.h
@@ -118,7 +118,8 @@ struct host1x {
struct host1x_syncpt_base *bases;
struct device *dev;
struct clk *clk;
- struct reset_control *rst;
+ struct reset_control_bulk_data resets[2];
+ unsigned int nresets;
struct iommu_group *group;
struct iommu_domain *domain;
@@ -149,6 +150,8 @@ struct host1x {
struct list_head list;
struct device_dma_parameters dma_parms;
+
+ struct host1x_bo_cache cache;
};
void host1x_hypervisor_writel(struct host1x *host1x, u32 r, u32 v);
diff --git a/drivers/gpu/host1x/hw/channel_hw.c b/drivers/gpu/host1x/hw/channel_hw.c
index 1999780..6b40e9a 100644
--- a/drivers/gpu/host1x/hw/channel_hw.c
+++ b/drivers/gpu/host1x/hw/channel_hw.c
@@ -159,6 +159,27 @@ static void host1x_channel_set_streamid(struct host1x_channel *channel)
#endif
}
+static void host1x_enable_gather_filter(struct host1x_channel *ch)
+{
+#if HOST1X_HW >= 6
+ struct host1x *host = dev_get_drvdata(ch->dev->parent);
+ u32 val;
+
+ if (!host->hv_regs)
+ return;
+
+ val = host1x_hypervisor_readl(
+ host, HOST1X_HV_CH_KERNEL_FILTER_GBUFFER(ch->id / 32));
+ val |= BIT(ch->id % 32);
+ host1x_hypervisor_writel(
+ host, val, HOST1X_HV_CH_KERNEL_FILTER_GBUFFER(ch->id / 32));
+#elif HOST1X_HW >= 4
+ host1x_ch_writel(ch,
+ HOST1X_CHANNEL_CHANNELCTRL_KERNEL_FILTER_GBUFFER(1),
+ HOST1X_CHANNEL_CHANNELCTRL);
+#endif
+}
+
static int channel_submit(struct host1x_job *job)
{
struct host1x_channel *ch = job->channel;
@@ -190,6 +211,7 @@ static int channel_submit(struct host1x_job *job)
}
host1x_channel_set_streamid(ch);
+ host1x_enable_gather_filter(ch);
/* begin a CDMA submit */
err = host1x_cdma_begin(&ch->cdma, job);
@@ -249,27 +271,6 @@ static int channel_submit(struct host1x_job *job)
return err;
}
-static void enable_gather_filter(struct host1x *host,
- struct host1x_channel *ch)
-{
-#if HOST1X_HW >= 6
- u32 val;
-
- if (!host->hv_regs)
- return;
-
- val = host1x_hypervisor_readl(
- host, HOST1X_HV_CH_KERNEL_FILTER_GBUFFER(ch->id / 32));
- val |= BIT(ch->id % 32);
- host1x_hypervisor_writel(
- host, val, HOST1X_HV_CH_KERNEL_FILTER_GBUFFER(ch->id / 32));
-#elif HOST1X_HW >= 4
- host1x_ch_writel(ch,
- HOST1X_CHANNEL_CHANNELCTRL_KERNEL_FILTER_GBUFFER(1),
- HOST1X_CHANNEL_CHANNELCTRL);
-#endif
-}
-
static int host1x_channel_init(struct host1x_channel *ch, struct host1x *dev,
unsigned int index)
{
@@ -278,7 +279,6 @@ static int host1x_channel_init(struct host1x_channel *ch, struct host1x *dev,
#else
ch->regs = dev->regs + index * 0x100;
#endif
- enable_gather_filter(dev, ch);
return 0;
}
diff --git a/drivers/gpu/host1x/intr.c b/drivers/gpu/host1x/intr.c
index 45b6be9..965ba21 100644
--- a/drivers/gpu/host1x/intr.c
+++ b/drivers/gpu/host1x/intr.c
@@ -297,14 +297,11 @@ int host1x_intr_init(struct host1x *host, unsigned int irq_sync)
"host1x_sp_%02u", id);
}
- host1x_intr_start(host);
-
return 0;
}
void host1x_intr_deinit(struct host1x *host)
{
- host1x_intr_stop(host);
}
void host1x_intr_start(struct host1x *host)
diff --git a/drivers/gpu/host1x/job.c b/drivers/gpu/host1x/job.c
index 0eef6df7..5e8c183 100644
--- a/drivers/gpu/host1x/job.c
+++ b/drivers/gpu/host1x/job.c
@@ -134,20 +134,20 @@ EXPORT_SYMBOL(host1x_job_add_wait);
static unsigned int pin_job(struct host1x *host, struct host1x_job *job)
{
+ unsigned long mask = HOST1X_RELOC_READ | HOST1X_RELOC_WRITE;
struct host1x_client *client = job->client;
struct device *dev = client->dev;
struct host1x_job_gather *g;
- struct iommu_domain *domain;
- struct sg_table *sgt;
unsigned int i;
int err;
- domain = iommu_get_domain_for_dev(dev);
job->num_unpins = 0;
for (i = 0; i < job->num_relocs; i++) {
struct host1x_reloc *reloc = &job->relocs[i];
- dma_addr_t phys_addr, *phys;
+ enum dma_data_direction direction;
+ struct host1x_bo_mapping *map;
+ struct host1x_bo *bo;
reloc->target.bo = host1x_bo_get(reloc->target.bo);
if (!reloc->target.bo) {
@@ -155,64 +155,44 @@ static unsigned int pin_job(struct host1x *host, struct host1x_job *job)
goto unpin;
}
- /*
- * If the client device is not attached to an IOMMU, the
- * physical address of the buffer object can be used.
- *
- * Similarly, when an IOMMU domain is shared between all
- * host1x clients, the IOVA is already available, so no
- * need to map the buffer object again.
- *
- * XXX Note that this isn't always safe to do because it
- * relies on an assumption that no cache maintenance is
- * needed on the buffer objects.
- */
- if (!domain || client->group)
- phys = &phys_addr;
- else
- phys = NULL;
+ bo = reloc->target.bo;
- sgt = host1x_bo_pin(dev, reloc->target.bo, phys);
- if (IS_ERR(sgt)) {
- err = PTR_ERR(sgt);
+ switch (reloc->flags & mask) {
+ case HOST1X_RELOC_READ:
+ direction = DMA_TO_DEVICE;
+ break;
+
+ case HOST1X_RELOC_WRITE:
+ direction = DMA_FROM_DEVICE;
+ break;
+
+ case HOST1X_RELOC_READ | HOST1X_RELOC_WRITE:
+ direction = DMA_BIDIRECTIONAL;
+ break;
+
+ default:
+ err = -EINVAL;
goto unpin;
}
- if (sgt) {
- unsigned long mask = HOST1X_RELOC_READ |
- HOST1X_RELOC_WRITE;
- enum dma_data_direction dir;
-
- switch (reloc->flags & mask) {
- case HOST1X_RELOC_READ:
- dir = DMA_TO_DEVICE;
- break;
-
- case HOST1X_RELOC_WRITE:
- dir = DMA_FROM_DEVICE;
- break;
-
- case HOST1X_RELOC_READ | HOST1X_RELOC_WRITE:
- dir = DMA_BIDIRECTIONAL;
- break;
-
- default:
- err = -EINVAL;
- goto unpin;
- }
-
- err = dma_map_sgtable(dev, sgt, dir, 0);
- if (err)
- goto unpin;
-
- job->unpins[job->num_unpins].dev = dev;
- job->unpins[job->num_unpins].dir = dir;
- phys_addr = sg_dma_address(sgt->sgl);
+ map = host1x_bo_pin(dev, bo, direction, &client->cache);
+ if (IS_ERR(map)) {
+ err = PTR_ERR(map);
+ goto unpin;
}
- job->addr_phys[job->num_unpins] = phys_addr;
- job->unpins[job->num_unpins].bo = reloc->target.bo;
- job->unpins[job->num_unpins].sgt = sgt;
+ /*
+ * host1x clients are generally not able to do scatter-gather themselves, so fail
+ * if the buffer is discontiguous and we fail to map its SG table to a single
+ * contiguous chunk of I/O virtual memory.
+ */
+ if (map->chunks > 1) {
+ err = -EINVAL;
+ goto unpin;
+ }
+
+ job->addr_phys[job->num_unpins] = map->phys;
+ job->unpins[job->num_unpins].map = map;
job->num_unpins++;
}
@@ -224,12 +204,11 @@ static unsigned int pin_job(struct host1x *host, struct host1x_job *job)
return 0;
for (i = 0; i < job->num_cmds; i++) {
+ struct host1x_bo_mapping *map;
size_t gather_size = 0;
struct scatterlist *sg;
- dma_addr_t phys_addr;
unsigned long shift;
struct iova *alloc;
- dma_addr_t *phys;
unsigned int j;
if (job->cmds[i].is_wait)
@@ -243,25 +222,16 @@ static unsigned int pin_job(struct host1x *host, struct host1x_job *job)
goto unpin;
}
- /**
- * If the host1x is not attached to an IOMMU, there is no need
- * to map the buffer object for the host1x, since the physical
- * address can simply be used.
- */
- if (!iommu_get_domain_for_dev(host->dev))
- phys = &phys_addr;
- else
- phys = NULL;
-
- sgt = host1x_bo_pin(host->dev, g->bo, phys);
- if (IS_ERR(sgt)) {
- err = PTR_ERR(sgt);
- goto put;
+ map = host1x_bo_pin(host->dev, g->bo, DMA_TO_DEVICE, &host->cache);
+ if (IS_ERR(map)) {
+ err = PTR_ERR(map);
+ goto unpin;
}
if (host->domain) {
- for_each_sgtable_sg(sgt, sg, j)
+ for_each_sgtable_sg(map->sgt, sg, j)
gather_size += sg->length;
+
gather_size = iova_align(&host->iova, gather_size);
shift = iova_shift(&host->iova);
@@ -272,33 +242,23 @@ static unsigned int pin_job(struct host1x *host, struct host1x_job *job)
goto put;
}
- err = iommu_map_sgtable(host->domain,
- iova_dma_addr(&host->iova, alloc),
- sgt, IOMMU_READ);
+ err = iommu_map_sgtable(host->domain, iova_dma_addr(&host->iova, alloc),
+ map->sgt, IOMMU_READ);
if (err == 0) {
__free_iova(&host->iova, alloc);
err = -EINVAL;
goto put;
}
- job->unpins[job->num_unpins].size = gather_size;
- phys_addr = iova_dma_addr(&host->iova, alloc);
- } else if (sgt) {
- err = dma_map_sgtable(host->dev, sgt, DMA_TO_DEVICE, 0);
- if (err)
- goto put;
-
- job->unpins[job->num_unpins].dir = DMA_TO_DEVICE;
- job->unpins[job->num_unpins].dev = host->dev;
- phys_addr = sg_dma_address(sgt->sgl);
+ map->phys = iova_dma_addr(&host->iova, alloc);
+ map->size = gather_size;
}
- job->addr_phys[job->num_unpins] = phys_addr;
- job->gather_addr_phys[i] = phys_addr;
-
- job->unpins[job->num_unpins].bo = g->bo;
- job->unpins[job->num_unpins].sgt = sgt;
+ job->addr_phys[job->num_unpins] = map->phys;
+ job->unpins[job->num_unpins].map = map;
job->num_unpins++;
+
+ job->gather_addr_phys[i] = map->phys;
}
return 0;
@@ -690,22 +650,16 @@ void host1x_job_unpin(struct host1x_job *job)
unsigned int i;
for (i = 0; i < job->num_unpins; i++) {
- struct host1x_job_unpin_data *unpin = &job->unpins[i];
- struct device *dev = unpin->dev ?: host->dev;
- struct sg_table *sgt = unpin->sgt;
+ struct host1x_bo_mapping *map = job->unpins[i].map;
+ struct host1x_bo *bo = map->bo;
- if (!job->enable_firewall && unpin->size && host->domain) {
- iommu_unmap(host->domain, job->addr_phys[i],
- unpin->size);
- free_iova(&host->iova,
- iova_pfn(&host->iova, job->addr_phys[i]));
+ if (!job->enable_firewall && map->size && host->domain) {
+ iommu_unmap(host->domain, job->addr_phys[i], map->size);
+ free_iova(&host->iova, iova_pfn(&host->iova, job->addr_phys[i]));
}
- if (unpin->dev && sgt)
- dma_unmap_sgtable(unpin->dev, sgt, unpin->dir, 0);
-
- host1x_bo_unpin(dev, unpin->bo, sgt);
- host1x_bo_put(unpin->bo);
+ host1x_bo_unpin(map);
+ host1x_bo_put(bo);
}
job->num_unpins = 0;
diff --git a/drivers/gpu/host1x/job.h b/drivers/gpu/host1x/job.h
index b4428c5..dad5a19 100644
--- a/drivers/gpu/host1x/job.h
+++ b/drivers/gpu/host1x/job.h
@@ -35,11 +35,7 @@ struct host1x_job_cmd {
};
struct host1x_job_unpin_data {
- struct host1x_bo *bo;
- struct sg_table *sgt;
- struct device *dev;
- size_t size;
- enum dma_data_direction dir;
+ struct host1x_bo_mapping *map;
};
/*
diff --git a/drivers/gpu/host1x/syncpt.c b/drivers/gpu/host1x/syncpt.c
index d198a10..e08e331 100644
--- a/drivers/gpu/host1x/syncpt.c
+++ b/drivers/gpu/host1x/syncpt.c
@@ -143,6 +143,8 @@ void host1x_syncpt_restore(struct host1x *host)
for (i = 0; i < host1x_syncpt_nb_bases(host); i++)
host1x_hw_syncpt_restore_wait_base(host, sp_base + i);
+ host1x_hw_syncpt_enable_protection(host);
+
wmb();
}
@@ -366,9 +368,6 @@ int host1x_syncpt_init(struct host1x *host)
host->syncpt = syncpt;
host->bases = bases;
- host1x_syncpt_restore(host);
- host1x_hw_syncpt_enable_protection(host);
-
/* Allocate sync point to use for clearing waits for expired fences */
host->nop_sp = host1x_syncpt_alloc(host, 0, "reserved-nop");
if (!host->nop_sp)
diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c
index 392c1ac..7ae04cc 100644
--- a/drivers/hv/vmbus_drv.c
+++ b/drivers/hv/vmbus_drv.c
@@ -1381,7 +1381,7 @@ static void vmbus_isr(void)
tasklet_schedule(&hv_cpu->msg_dpc);
}
- add_interrupt_randomness(vmbus_interrupt, 0);
+ add_interrupt_randomness(vmbus_interrupt);
}
static irqreturn_t vmbus_percpu_isr(int irq, void *dev_id)
diff --git a/drivers/i2c/busses/Kconfig b/drivers/i2c/busses/Kconfig
index dce3928..c6b854a 100644
--- a/drivers/i2c/busses/Kconfig
+++ b/drivers/i2c/busses/Kconfig
@@ -963,16 +963,10 @@
This driver can also be built as a module. If so, the module will
be called i2c-rk3x.
-config HAVE_S3C2410_I2C
- bool
- help
- This will include I2C support for Samsung SoCs. If you want to
- include I2C support for any machine, kindly select this in the
- respective Kconfig file.
-
config I2C_S3C2410
tristate "S3C/Exynos I2C Driver"
- depends on HAVE_S3C2410_I2C || COMPILE_TEST
+ depends on ARCH_EXYNOS || ARCH_S3C24XX || ARCH_S3C64XX || \
+ ARCH_S5PV210 || COMPILE_TEST
help
Say Y here to include support for I2C controller in the
Samsung SoCs (S3C, S5Pv210, Exynos).
diff --git a/drivers/i2c/busses/i2c-mpc.c b/drivers/i2c/busses/i2c-mpc.c
index 53b8da6d..db26cc3 100644
--- a/drivers/i2c/busses/i2c-mpc.c
+++ b/drivers/i2c/busses/i2c-mpc.c
@@ -492,7 +492,7 @@ static void mpc_i2c_finish(struct mpc_i2c *i2c, int rc)
static void mpc_i2c_do_action(struct mpc_i2c *i2c)
{
- struct i2c_msg *msg = &i2c->msgs[i2c->curr_msg];
+ struct i2c_msg *msg = NULL;
int dir = 0;
int recv_len = 0;
u8 byte;
@@ -501,10 +501,13 @@ static void mpc_i2c_do_action(struct mpc_i2c *i2c)
i2c->cntl_bits &= ~(CCR_RSTA | CCR_MTX | CCR_TXAK);
- if (msg->flags & I2C_M_RD)
- dir = 1;
- if (msg->flags & I2C_M_RECV_LEN)
- recv_len = 1;
+ if (i2c->action != MPC_I2C_ACTION_STOP) {
+ msg = &i2c->msgs[i2c->curr_msg];
+ if (msg->flags & I2C_M_RD)
+ dir = 1;
+ if (msg->flags & I2C_M_RECV_LEN)
+ recv_len = 1;
+ }
switch (i2c->action) {
case MPC_I2C_ACTION_RESTART:
@@ -581,7 +584,7 @@ static void mpc_i2c_do_action(struct mpc_i2c *i2c)
break;
}
- if (msg->len == i2c->byte_posn) {
+ if (msg && msg->len == i2c->byte_posn) {
i2c->curr_msg++;
i2c->byte_posn = 0;
diff --git a/drivers/i2c/i2c-core-acpi.c b/drivers/i2c/i2c-core-acpi.c
index 15332ba..85ed4c1 100644
--- a/drivers/i2c/i2c-core-acpi.c
+++ b/drivers/i2c/i2c-core-acpi.c
@@ -144,9 +144,12 @@ static int i2c_acpi_do_lookup(struct acpi_device *adev,
struct list_head resource_list;
int ret;
- if (acpi_bus_get_status(adev) || !adev->status.present)
+ if (acpi_bus_get_status(adev))
return -EINVAL;
+ if (!acpi_dev_ready_for_enumeration(adev))
+ return -ENODEV;
+
if (acpi_match_device_ids(adev, i2c_acpi_ignored_device_ids) == 0)
return -ENODEV;
@@ -480,8 +483,8 @@ struct notifier_block i2c_acpi_notifier = {
};
/**
- * i2c_acpi_new_device - Create i2c-client for the Nth I2cSerialBus resource
- * @dev: Device owning the ACPI resources to get the client from
+ * i2c_acpi_new_device_by_fwnode - Create i2c-client for the Nth I2cSerialBus resource
+ * @fwnode: fwnode with the ACPI resources to get the client from
* @index: Index of ACPI resource to get
* @info: describes the I2C device; note this is modified (addr gets set)
* Context: can sleep
@@ -497,15 +500,20 @@ struct notifier_block i2c_acpi_notifier = {
* Returns a pointer to the new i2c-client, or error pointer in case of failure.
* Specifically, -EPROBE_DEFER is returned if the adapter is not found.
*/
-struct i2c_client *i2c_acpi_new_device(struct device *dev, int index,
- struct i2c_board_info *info)
+struct i2c_client *i2c_acpi_new_device_by_fwnode(struct fwnode_handle *fwnode,
+ int index,
+ struct i2c_board_info *info)
{
- struct acpi_device *adev = ACPI_COMPANION(dev);
struct i2c_acpi_lookup lookup;
struct i2c_adapter *adapter;
+ struct acpi_device *adev;
LIST_HEAD(resource_list);
int ret;
+ adev = to_acpi_device_node(fwnode);
+ if (!adev)
+ return ERR_PTR(-ENODEV);
+
memset(&lookup, 0, sizeof(lookup));
lookup.info = info;
lookup.device_handle = acpi_device_handle(adev);
@@ -527,7 +535,7 @@ struct i2c_client *i2c_acpi_new_device(struct device *dev, int index,
return i2c_new_client_device(adapter, info);
}
-EXPORT_SYMBOL_GPL(i2c_acpi_new_device);
+EXPORT_SYMBOL_GPL(i2c_acpi_new_device_by_fwnode);
bool i2c_acpi_waive_d0_probe(struct device *dev)
{
diff --git a/drivers/i2c/i2c-core-base.c b/drivers/i2c/i2c-core-base.c
index f193f90..73253e6 100644
--- a/drivers/i2c/i2c-core-base.c
+++ b/drivers/i2c/i2c-core-base.c
@@ -466,14 +466,12 @@ static int i2c_smbus_host_notify_to_irq(const struct i2c_client *client)
static int i2c_device_probe(struct device *dev)
{
struct i2c_client *client = i2c_verify_client(dev);
- struct i2c_adapter *adap;
struct i2c_driver *driver;
int status;
if (!client)
return 0;
- adap = client->adapter;
client->irq = client->init_irq;
if (!client->irq) {
@@ -539,14 +537,6 @@ static int i2c_device_probe(struct device *dev)
dev_dbg(dev, "probe\n");
- if (adap->bus_regulator) {
- status = regulator_enable(adap->bus_regulator);
- if (status < 0) {
- dev_err(&adap->dev, "Failed to enable bus regulator\n");
- goto err_clear_wakeup_irq;
- }
- }
-
status = of_clk_set_defaults(dev->of_node, false);
if (status < 0)
goto err_clear_wakeup_irq;
@@ -605,10 +595,8 @@ static int i2c_device_probe(struct device *dev)
static void i2c_device_remove(struct device *dev)
{
struct i2c_client *client = to_i2c_client(dev);
- struct i2c_adapter *adap;
struct i2c_driver *driver;
- adap = client->adapter;
driver = to_i2c_driver(dev->driver);
if (driver->remove) {
int status;
@@ -623,8 +611,6 @@ static void i2c_device_remove(struct device *dev)
devres_release_group(&client->dev, client->devres_group_id);
dev_pm_domain_detach(&client->dev, !i2c_acpi_waive_d0_probe(dev));
- if (!pm_runtime_status_suspended(&client->dev) && adap->bus_regulator)
- regulator_disable(adap->bus_regulator);
dev_pm_clear_wake_irq(&client->dev);
device_init_wakeup(&client->dev, false);
@@ -634,86 +620,6 @@ static void i2c_device_remove(struct device *dev)
pm_runtime_put(&client->adapter->dev);
}
-#ifdef CONFIG_PM_SLEEP
-static int i2c_resume_early(struct device *dev)
-{
- struct i2c_client *client = i2c_verify_client(dev);
- int err;
-
- if (!client)
- return 0;
-
- if (pm_runtime_status_suspended(&client->dev) &&
- client->adapter->bus_regulator) {
- err = regulator_enable(client->adapter->bus_regulator);
- if (err)
- return err;
- }
-
- return pm_generic_resume_early(&client->dev);
-}
-
-static int i2c_suspend_late(struct device *dev)
-{
- struct i2c_client *client = i2c_verify_client(dev);
- int err;
-
- if (!client)
- return 0;
-
- err = pm_generic_suspend_late(&client->dev);
- if (err)
- return err;
-
- if (!pm_runtime_status_suspended(&client->dev) &&
- client->adapter->bus_regulator)
- return regulator_disable(client->adapter->bus_regulator);
-
- return 0;
-}
-#endif
-
-#ifdef CONFIG_PM
-static int i2c_runtime_resume(struct device *dev)
-{
- struct i2c_client *client = i2c_verify_client(dev);
- int err;
-
- if (!client)
- return 0;
-
- if (client->adapter->bus_regulator) {
- err = regulator_enable(client->adapter->bus_regulator);
- if (err)
- return err;
- }
-
- return pm_generic_runtime_resume(&client->dev);
-}
-
-static int i2c_runtime_suspend(struct device *dev)
-{
- struct i2c_client *client = i2c_verify_client(dev);
- int err;
-
- if (!client)
- return 0;
-
- err = pm_generic_runtime_suspend(&client->dev);
- if (err)
- return err;
-
- if (client->adapter->bus_regulator)
- return regulator_disable(client->adapter->bus_regulator);
- return 0;
-}
-#endif
-
-static const struct dev_pm_ops i2c_device_pm = {
- SET_LATE_SYSTEM_SLEEP_PM_OPS(i2c_suspend_late, i2c_resume_early)
- SET_RUNTIME_PM_OPS(i2c_runtime_suspend, i2c_runtime_resume, NULL)
-};
-
static void i2c_device_shutdown(struct device *dev)
{
struct i2c_client *client = i2c_verify_client(dev);
@@ -773,7 +679,6 @@ struct bus_type i2c_bus_type = {
.probe = i2c_device_probe,
.remove = i2c_device_remove,
.shutdown = i2c_device_shutdown,
- .pm = &i2c_device_pm,
};
EXPORT_SYMBOL_GPL(i2c_bus_type);
diff --git a/drivers/i2c/i2c-dev.c b/drivers/i2c/i2c-dev.c
index bce0e8b..cf5d049 100644
--- a/drivers/i2c/i2c-dev.c
+++ b/drivers/i2c/i2c-dev.c
@@ -535,6 +535,9 @@ static long compat_i2cdev_ioctl(struct file *file, unsigned int cmd, unsigned lo
sizeof(rdwr_arg)))
return -EFAULT;
+ if (!rdwr_arg.msgs || rdwr_arg.nmsgs == 0)
+ return -EINVAL;
+
if (rdwr_arg.nmsgs > I2C_RDWR_IOCTL_MAX_MSGS)
return -EINVAL;
diff --git a/drivers/infiniband/core/cache.c b/drivers/infiniband/core/cache.c
index 0c98dd3..b79f816 100644
--- a/drivers/infiniband/core/cache.c
+++ b/drivers/infiniband/core/cache.c
@@ -33,6 +33,7 @@
* SOFTWARE.
*/
+#include <linux/if_vlan.h>
#include <linux/module.h>
#include <linux/errno.h>
#include <linux/slab.h>
diff --git a/drivers/infiniband/core/uverbs_marshall.c b/drivers/infiniband/core/uverbs_marshall.c
index b8d715c..11a0806 100644
--- a/drivers/infiniband/core/uverbs_marshall.c
+++ b/drivers/infiniband/core/uverbs_marshall.c
@@ -66,7 +66,7 @@ void ib_copy_ah_attr_to_user(struct ib_device *device,
struct rdma_ah_attr *src = ah_attr;
struct rdma_ah_attr conv_ah;
- memset(&dst->grh.reserved, 0, sizeof(dst->grh.reserved));
+ memset(&dst->grh, 0, sizeof(dst->grh));
if ((ah_attr->type == RDMA_AH_ATTR_TYPE_OPA) &&
(rdma_ah_get_dlid(ah_attr) > be16_to_cpu(IB_LID_PERMISSIVE)) &&
diff --git a/drivers/infiniband/core/uverbs_uapi.c b/drivers/infiniband/core/uverbs_uapi.c
index 2f2c764..a02916a 100644
--- a/drivers/infiniband/core/uverbs_uapi.c
+++ b/drivers/infiniband/core/uverbs_uapi.c
@@ -447,6 +447,9 @@ static int uapi_finalize(struct uverbs_api *uapi)
uapi->num_write_ex = max_write_ex + 1;
data = kmalloc_array(uapi->num_write + uapi->num_write_ex,
sizeof(*uapi->write_methods), GFP_KERNEL);
+ if (!data)
+ return -ENOMEM;
+
for (i = 0; i != uapi->num_write + uapi->num_write_ex; i++)
data[i] = &uapi->notsupp_method;
uapi->write_methods = data;
diff --git a/drivers/infiniband/hw/irdma/ctrl.c b/drivers/infiniband/hw/irdma/ctrl.c
index 7264f8c..3141a9c 100644
--- a/drivers/infiniband/hw/irdma/ctrl.c
+++ b/drivers/infiniband/hw/irdma/ctrl.c
@@ -1,5 +1,7 @@
// SPDX-License-Identifier: GPL-2.0 or Linux-OpenIB
/* Copyright (c) 2015 - 2021 Intel Corporation */
+#include <linux/etherdevice.h>
+
#include "osdep.h"
#include "status.h"
#include "hmc.h"
diff --git a/drivers/infiniband/hw/irdma/main.c b/drivers/infiniband/hw/irdma/main.c
index 51a4135..3fda7b7 100644
--- a/drivers/infiniband/hw/irdma/main.c
+++ b/drivers/infiniband/hw/irdma/main.c
@@ -228,7 +228,8 @@ static void irdma_fill_device_info(struct irdma_device *iwdev, struct ice_pf *pf
rf->msix_count = pf->num_rdma_msix;
rf->msix_entries = &pf->msix_entries[pf->rdma_base_vector];
rf->default_vsi.vsi_idx = vsi->vsi_num;
- rf->protocol_used = IRDMA_ROCE_PROTOCOL_ONLY;
+ rf->protocol_used = pf->rdma_mode & IIDC_RDMA_PROTOCOL_ROCEV2 ?
+ IRDMA_ROCE_PROTOCOL_ONLY : IRDMA_IWARP_PROTOCOL_ONLY;
rf->rdma_ver = IRDMA_GEN_2;
rf->rsrc_profile = IRDMA_HMC_PROFILE_DEFAULT;
rf->rst_to = IRDMA_RST_TIMEOUT_HZ;
diff --git a/drivers/infiniband/hw/irdma/uda.c b/drivers/infiniband/hw/irdma/uda.c
index f5b1b61..7a9988d 100644
--- a/drivers/infiniband/hw/irdma/uda.c
+++ b/drivers/infiniband/hw/irdma/uda.c
@@ -1,5 +1,7 @@
// SPDX-License-Identifier: GPL-2.0 or Linux-OpenIB
/* Copyright (c) 2016 - 2021 Intel Corporation */
+#include <linux/etherdevice.h>
+
#include "osdep.h"
#include "status.h"
#include "hmc.h"
diff --git a/drivers/infiniband/hw/mlx5/doorbell.c b/drivers/infiniband/hw/mlx5/doorbell.c
index 6398e2f..e321111 100644
--- a/drivers/infiniband/hw/mlx5/doorbell.c
+++ b/drivers/infiniband/hw/mlx5/doorbell.c
@@ -32,6 +32,7 @@
#include <linux/kref.h>
#include <linux/slab.h>
+#include <linux/sched/mm.h>
#include <rdma/ib_umem.h>
#include "mlx5_ib.h"
diff --git a/drivers/infiniband/hw/mlx5/fs.c b/drivers/infiniband/hw/mlx5/fs.c
index b780185..661ed2b 100644
--- a/drivers/infiniband/hw/mlx5/fs.c
+++ b/drivers/infiniband/hw/mlx5/fs.c
@@ -1508,7 +1508,7 @@ _get_flow_table(struct mlx5_ib_dev *dev,
!esw_encap)
flags |= MLX5_FLOW_TABLE_TUNNEL_EN_REFORMAT;
break;
- case MLX5_FLOW_NAMESPACE_FDB:
+ case MLX5_FLOW_NAMESPACE_FDB_BYPASS:
max_table_size = BIT(
MLX5_CAP_ESW_FLOWTABLE_FDB(dev->mdev, log_max_ft_size));
if (MLX5_CAP_ESW_FLOWTABLE_FDB(dev->mdev, decap) && esw_encap)
@@ -1517,7 +1517,7 @@ _get_flow_table(struct mlx5_ib_dev *dev,
reformat_l3_tunnel_to_l2) &&
esw_encap)
flags |= MLX5_FLOW_TABLE_TUNNEL_EN_REFORMAT;
- priority = FDB_BYPASS_PATH;
+ priority = fs_matcher->priority;
break;
case MLX5_FLOW_NAMESPACE_RDMA_RX:
max_table_size = BIT(
@@ -1546,8 +1546,8 @@ _get_flow_table(struct mlx5_ib_dev *dev,
case MLX5_FLOW_NAMESPACE_EGRESS:
prio = &dev->flow_db->egress_prios[priority];
break;
- case MLX5_FLOW_NAMESPACE_FDB:
- prio = &dev->flow_db->fdb;
+ case MLX5_FLOW_NAMESPACE_FDB_BYPASS:
+ prio = &dev->flow_db->fdb[priority];
break;
case MLX5_FLOW_NAMESPACE_RDMA_RX:
prio = &dev->flow_db->rdma_rx[priority];
@@ -1937,7 +1937,7 @@ mlx5_ib_ft_type_to_namespace(enum mlx5_ib_uapi_flow_table_type table_type,
*namespace = MLX5_FLOW_NAMESPACE_EGRESS;
break;
case MLX5_IB_UAPI_FLOW_TABLE_TYPE_FDB:
- *namespace = MLX5_FLOW_NAMESPACE_FDB;
+ *namespace = MLX5_FLOW_NAMESPACE_FDB_BYPASS;
break;
case MLX5_IB_UAPI_FLOW_TABLE_TYPE_RDMA_RX:
*namespace = MLX5_FLOW_NAMESPACE_RDMA_RX;
@@ -2029,8 +2029,8 @@ static int get_dests(struct uverbs_attr_bundle *attrs,
}
/* Allow only DEVX object, drop as dest for FDB */
- if (fs_matcher->ns_type == MLX5_FLOW_NAMESPACE_FDB && !(dest_devx ||
- (*flags & MLX5_IB_ATTR_CREATE_FLOW_FLAGS_DROP)))
+ if (fs_matcher->ns_type == MLX5_FLOW_NAMESPACE_FDB_BYPASS &&
+ !(dest_devx || (*flags & MLX5_IB_ATTR_CREATE_FLOW_FLAGS_DROP)))
return -EINVAL;
/* Allow only DEVX object or QP as dest when inserting to RDMA_RX */
@@ -2050,7 +2050,7 @@ static int get_dests(struct uverbs_attr_bundle *attrs,
if (!is_flow_dest(devx_obj, dest_id, dest_type))
return -EINVAL;
/* Allow only flow table as dest when inserting to FDB or RDMA_RX */
- if ((fs_matcher->ns_type == MLX5_FLOW_NAMESPACE_FDB ||
+ if ((fs_matcher->ns_type == MLX5_FLOW_NAMESPACE_FDB_BYPASS ||
fs_matcher->ns_type == MLX5_FLOW_NAMESPACE_RDMA_RX) &&
*dest_type != MLX5_FLOW_DESTINATION_TYPE_FLOW_TABLE)
return -EINVAL;
@@ -2320,7 +2320,7 @@ static int UVERBS_HANDLER(MLX5_IB_METHOD_FLOW_MATCHER_CREATE)(
if (err)
goto end;
- if (obj->ns_type == MLX5_FLOW_NAMESPACE_FDB &&
+ if (obj->ns_type == MLX5_FLOW_NAMESPACE_FDB_BYPASS &&
mlx5_eswitch_mode(dev->mdev) != MLX5_ESWITCH_OFFLOADS) {
err = -EINVAL;
goto end;
diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h
index 4a7a56e..e3c33be 100644
--- a/drivers/infiniband/hw/mlx5/mlx5_ib.h
+++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h
@@ -232,6 +232,7 @@ enum {
#define MLX5_IB_NUM_FLOW_FT (MLX5_IB_FLOW_LEFTOVERS_PRIO + 1)
#define MLX5_IB_NUM_SNIFFER_FTS 2
#define MLX5_IB_NUM_EGRESS_FTS 1
+#define MLX5_IB_NUM_FDB_FTS MLX5_BY_PASS_NUM_REGULAR_PRIOS
struct mlx5_ib_flow_prio {
struct mlx5_flow_table *flow_table;
unsigned int refcount;
@@ -276,7 +277,7 @@ struct mlx5_ib_flow_db {
struct mlx5_ib_flow_prio egress_prios[MLX5_IB_NUM_FLOW_FT];
struct mlx5_ib_flow_prio sniffer[MLX5_IB_NUM_SNIFFER_FTS];
struct mlx5_ib_flow_prio egress[MLX5_IB_NUM_EGRESS_FTS];
- struct mlx5_ib_flow_prio fdb;
+ struct mlx5_ib_flow_prio fdb[MLX5_IB_NUM_FDB_FTS];
struct mlx5_ib_flow_prio rdma_rx[MLX5_IB_NUM_FLOW_FT];
struct mlx5_ib_flow_prio rdma_tx[MLX5_IB_NUM_FLOW_FT];
struct mlx5_ib_flow_prio opfcs[MLX5_IB_OPCOUNTER_MAX];
@@ -664,6 +665,7 @@ struct mlx5_ib_mr {
/* User MR data */
struct mlx5_cache_ent *cache_ent;
+ struct ib_umem *umem;
/* This is zero'd when the MR is allocated */
union {
@@ -675,7 +677,7 @@ struct mlx5_ib_mr {
struct list_head list;
};
- /* Used only by kernel MRs */
+ /* Used only by kernel MRs (umem == NULL) */
struct {
void *descs;
void *descs_alloc;
@@ -696,9 +698,8 @@ struct mlx5_ib_mr {
int data_length;
};
- /* Used only by User MRs */
+ /* Used only by User MRs (umem != NULL) */
struct {
- struct ib_umem *umem;
unsigned int page_shift;
/* Current access_flags */
int access_flags;
diff --git a/drivers/infiniband/hw/mlx5/mr.c b/drivers/infiniband/hw/mlx5/mr.c
index 63e2129..157d862 100644
--- a/drivers/infiniband/hw/mlx5/mr.c
+++ b/drivers/infiniband/hw/mlx5/mr.c
@@ -1904,18 +1904,19 @@ mlx5_alloc_priv_descs(struct ib_device *device,
return ret;
}
-static void mlx5_free_priv_descs(struct mlx5_ib_mr *mr)
+static void
+mlx5_free_priv_descs(struct mlx5_ib_mr *mr)
{
- struct mlx5_ib_dev *dev = to_mdev(mr->ibmr.device);
- int size = mr->max_descs * mr->desc_size;
+ if (!mr->umem && mr->descs) {
+ struct ib_device *device = mr->ibmr.device;
+ int size = mr->max_descs * mr->desc_size;
+ struct mlx5_ib_dev *dev = to_mdev(device);
- if (!mr->descs)
- return;
-
- dma_unmap_single(&dev->mdev->pdev->dev, mr->desc_map, size,
- DMA_TO_DEVICE);
- kfree(mr->descs_alloc);
- mr->descs = NULL;
+ dma_unmap_single(&dev->mdev->pdev->dev, mr->desc_map, size,
+ DMA_TO_DEVICE);
+ kfree(mr->descs_alloc);
+ mr->descs = NULL;
+ }
}
int mlx5_ib_dereg_mr(struct ib_mr *ibmr, struct ib_udata *udata)
@@ -1991,8 +1992,7 @@ int mlx5_ib_dereg_mr(struct ib_mr *ibmr, struct ib_udata *udata)
if (mr->cache_ent) {
mlx5_mr_cache_free(dev, mr);
} else {
- if (!udata)
- mlx5_free_priv_descs(mr);
+ mlx5_free_priv_descs(mr);
kfree(mr);
}
return 0;
@@ -2079,6 +2079,7 @@ static struct mlx5_ib_mr *mlx5_ib_alloc_pi_mr(struct ib_pd *pd,
if (err)
goto err_free_in;
+ mr->umem = NULL;
kfree(in);
return mr;
@@ -2205,6 +2206,7 @@ static struct ib_mr *__mlx5_ib_alloc_mr(struct ib_pd *pd,
}
mr->ibmr.device = pd->device;
+ mr->umem = NULL;
switch (mr_type) {
case IB_MR_TYPE_MEM_REG:
diff --git a/drivers/infiniband/hw/mlx5/odp.c b/drivers/infiniband/hw/mlx5/odp.c
index 91eb615..86842cd 100644
--- a/drivers/infiniband/hw/mlx5/odp.c
+++ b/drivers/infiniband/hw/mlx5/odp.c
@@ -1541,16 +1541,10 @@ int mlx5r_odp_create_eq(struct mlx5_ib_dev *dev, struct mlx5_ib_pf_eq *eq)
eq->irq_nb.notifier_call = mlx5_ib_eq_pf_int;
param = (struct mlx5_eq_param) {
- .irq_index = MLX5_IRQ_EQ_CTRL,
.nent = MLX5_IB_NUM_PF_EQE,
};
param.mask[0] = 1ull << MLX5_EVENT_TYPE_PAGE_FAULT;
- if (!zalloc_cpumask_var(¶m.affinity, GFP_KERNEL)) {
- err = -ENOMEM;
- goto err_wq;
- }
eq->core = mlx5_eq_create_generic(dev->mdev, ¶m);
- free_cpumask_var(param.affinity);
if (IS_ERR(eq->core)) {
err = PTR_ERR(eq->core);
goto err_wq;
diff --git a/drivers/infiniband/hw/mlx5/qp.c b/drivers/infiniband/hw/mlx5/qp.c
index e5abbcf..29475cf 100644
--- a/drivers/infiniband/hw/mlx5/qp.c
+++ b/drivers/infiniband/hw/mlx5/qp.c
@@ -30,6 +30,7 @@
* SOFTWARE.
*/
+#include <linux/etherdevice.h>
#include <linux/module.h>
#include <rdma/ib_umem.h>
#include <rdma/ib_cache.h>
diff --git a/drivers/infiniband/sw/rxe/rxe_mr.c b/drivers/infiniband/sw/rxe/rxe_mr.c
index 53271df..bcf717b 100644
--- a/drivers/infiniband/sw/rxe/rxe_mr.c
+++ b/drivers/infiniband/sw/rxe/rxe_mr.c
@@ -135,19 +135,19 @@ static int rxe_mr_alloc(struct rxe_mr *mr, int num_buf, int both)
ret = rxe_mr_alloc_map_set(num_map, &mr->cur_map_set);
if (ret)
- goto err_out;
+ return -ENOMEM;
if (both) {
ret = rxe_mr_alloc_map_set(num_map, &mr->next_map_set);
- if (ret) {
- rxe_mr_free_map_set(mr->num_map, mr->cur_map_set);
- goto err_out;
- }
+ if (ret)
+ goto err_free;
}
return 0;
-err_out:
+err_free:
+ rxe_mr_free_map_set(mr->num_map, mr->cur_map_set);
+ mr->cur_map_set = NULL;
return -ENOMEM;
}
@@ -214,7 +214,7 @@ int rxe_mr_init_user(struct rxe_pd *pd, u64 start, u64 length, u64 iova,
pr_warn("%s: Unable to get virtual address\n",
__func__);
err = -ENOMEM;
- goto err_cleanup_map;
+ goto err_release_umem;
}
buf->addr = (uintptr_t)vaddr;
@@ -237,8 +237,6 @@ int rxe_mr_init_user(struct rxe_pd *pd, u64 start, u64 length, u64 iova,
return 0;
-err_cleanup_map:
- rxe_mr_free_map_set(mr->num_map, mr->cur_map_set);
err_release_umem:
ib_umem_release(umem);
err_out:
diff --git a/drivers/input/joystick/spaceball.c b/drivers/input/joystick/spaceball.c
index 429411c..a85a4f3 100644
--- a/drivers/input/joystick/spaceball.c
+++ b/drivers/input/joystick/spaceball.c
@@ -19,6 +19,7 @@
#include <linux/module.h>
#include <linux/input.h>
#include <linux/serio.h>
+#include <asm/unaligned.h>
#define DRIVER_DESC "SpaceTec SpaceBall 2003/3003/4000 FLX driver"
@@ -75,9 +76,15 @@ static void spaceball_process_packet(struct spaceball* spaceball)
case 'D': /* Ball data */
if (spaceball->idx != 15) return;
- for (i = 0; i < 6; i++)
+ /*
+ * Skip first three bytes; read six axes worth of data.
+ * Axis values are signed 16-bit big-endian.
+ */
+ data += 3;
+ for (i = 0; i < ARRAY_SIZE(spaceball_axes); i++) {
input_report_abs(dev, spaceball_axes[i],
- (__s16)((data[2 * i + 3] << 8) | data[2 * i + 2]));
+ (__s16)get_unaligned_be16(&data[i * 2]));
+ }
break;
case 'K': /* Button data */
diff --git a/drivers/input/mouse/appletouch.c b/drivers/input/mouse/appletouch.c
index bfa2665..627048b 100644
--- a/drivers/input/mouse/appletouch.c
+++ b/drivers/input/mouse/appletouch.c
@@ -916,6 +916,8 @@ static int atp_probe(struct usb_interface *iface,
set_bit(BTN_TOOL_TRIPLETAP, input_dev->keybit);
set_bit(BTN_LEFT, input_dev->keybit);
+ INIT_WORK(&dev->work, atp_reinit);
+
error = input_register_device(dev->input);
if (error)
goto err_free_buffer;
@@ -923,8 +925,6 @@ static int atp_probe(struct usb_interface *iface,
/* save our data pointer in this interface device */
usb_set_intfdata(iface, dev);
- INIT_WORK(&dev->work, atp_reinit);
-
return 0;
err_free_buffer:
diff --git a/drivers/input/touchscreen/zinitix.c b/drivers/input/touchscreen/zinitix.c
index b8d9010..1e70b8d 100644
--- a/drivers/input/touchscreen/zinitix.c
+++ b/drivers/input/touchscreen/zinitix.c
@@ -488,6 +488,15 @@ static int zinitix_ts_probe(struct i2c_client *client)
return error;
}
+ error = devm_request_threaded_irq(&client->dev, client->irq,
+ NULL, zinitix_ts_irq_handler,
+ IRQF_ONESHOT | IRQF_NO_AUTOEN,
+ client->name, bt541);
+ if (error) {
+ dev_err(&client->dev, "Failed to request IRQ: %d\n", error);
+ return error;
+ }
+
error = zinitix_init_input_dev(bt541);
if (error) {
dev_err(&client->dev,
@@ -513,15 +522,6 @@ static int zinitix_ts_probe(struct i2c_client *client)
return -EINVAL;
}
- error = devm_request_threaded_irq(&client->dev, client->irq,
- NULL, zinitix_ts_irq_handler,
- IRQF_ONESHOT | IRQF_NO_AUTOEN,
- client->name, bt541);
- if (error) {
- dev_err(&client->dev, "Failed to request IRQ: %d\n", error);
- return error;
- }
-
return 0;
}
diff --git a/drivers/iommu/s390-iommu.c b/drivers/iommu/s390-iommu.c
index 83df387..50860eb 100644
--- a/drivers/iommu/s390-iommu.c
+++ b/drivers/iommu/s390-iommu.c
@@ -109,7 +109,7 @@ static int s390_iommu_attach_device(struct iommu_domain *domain,
zdev->dma_table = s390_domain->dma_table;
cc = zpci_register_ioat(zdev, 0, zdev->start_dma, zdev->end_dma,
- (u64) zdev->dma_table);
+ virt_to_phys(zdev->dma_table));
if (cc) {
rc = -EIO;
goto out_restore;
@@ -205,11 +205,11 @@ static void s390_iommu_release_device(struct device *dev)
}
static int s390_iommu_update_trans(struct s390_domain *s390_domain,
- unsigned long pa, dma_addr_t dma_addr,
+ phys_addr_t pa, dma_addr_t dma_addr,
size_t size, int flags)
{
struct s390_domain_device *domain_device;
- u8 *page_addr = (u8 *) (pa & PAGE_MASK);
+ phys_addr_t page_addr = pa & PAGE_MASK;
dma_addr_t start_dma_addr = dma_addr;
unsigned long irq_flags, nr_pages, i;
unsigned long *entry;
@@ -274,7 +274,7 @@ static int s390_iommu_map(struct iommu_domain *domain, unsigned long iova,
if (!(prot & IOMMU_WRITE))
flags |= ZPCI_TABLE_PROTECTED;
- rc = s390_iommu_update_trans(s390_domain, (unsigned long) paddr, iova,
+ rc = s390_iommu_update_trans(s390_domain, paddr, iova,
size, flags);
return rc;
@@ -324,7 +324,7 @@ static size_t s390_iommu_unmap(struct iommu_domain *domain,
if (!paddr)
return 0;
- rc = s390_iommu_update_trans(s390_domain, (unsigned long) paddr, iova,
+ rc = s390_iommu_update_trans(s390_domain, paddr, iova,
size, flags);
if (rc)
return 0;
diff --git a/drivers/isdn/capi/kcapi.c b/drivers/isdn/capi/kcapi.c
index 7313454..e69c4bf 100644
--- a/drivers/isdn/capi/kcapi.c
+++ b/drivers/isdn/capi/kcapi.c
@@ -32,7 +32,7 @@
#include <linux/mutex.h>
#include <linux/rcupdate.h>
-static int showcapimsgs = 0;
+static int showcapimsgs;
static struct workqueue_struct *kcapi_wq;
module_param(showcapimsgs, uint, 0);
diff --git a/drivers/isdn/mISDN/core.c b/drivers/isdn/mISDN/core.c
index 55891e4..a41b4b2 100644
--- a/drivers/isdn/mISDN/core.c
+++ b/drivers/isdn/mISDN/core.c
@@ -381,7 +381,7 @@ mISDNInit(void)
err = mISDN_inittimer(&debug);
if (err)
goto error2;
- err = l1_init(&debug);
+ err = Isdnl1_Init(&debug);
if (err)
goto error3;
err = Isdnl2_Init(&debug);
@@ -395,7 +395,7 @@ mISDNInit(void)
error5:
Isdnl2_cleanup();
error4:
- l1_cleanup();
+ Isdnl1_cleanup();
error3:
mISDN_timer_cleanup();
error2:
@@ -408,7 +408,7 @@ static void mISDN_cleanup(void)
{
misdn_sock_cleanup();
Isdnl2_cleanup();
- l1_cleanup();
+ Isdnl1_cleanup();
mISDN_timer_cleanup();
class_unregister(&mISDN_class);
diff --git a/drivers/isdn/mISDN/core.h b/drivers/isdn/mISDN/core.h
index 23b44d3..42599f4 100644
--- a/drivers/isdn/mISDN/core.h
+++ b/drivers/isdn/mISDN/core.h
@@ -60,8 +60,8 @@ struct Bprotocol *get_Bprotocol4id(u_int);
extern int mISDN_inittimer(u_int *);
extern void mISDN_timer_cleanup(void);
-extern int l1_init(u_int *);
-extern void l1_cleanup(void);
+extern int Isdnl1_Init(u_int *);
+extern void Isdnl1_cleanup(void);
extern int Isdnl2_Init(u_int *);
extern void Isdnl2_cleanup(void);
diff --git a/drivers/isdn/mISDN/layer1.c b/drivers/isdn/mISDN/layer1.c
index 98a3bc6..7b31c25 100644
--- a/drivers/isdn/mISDN/layer1.c
+++ b/drivers/isdn/mISDN/layer1.c
@@ -398,7 +398,7 @@ create_l1(struct dchannel *dch, dchannel_l1callback *dcb) {
EXPORT_SYMBOL(create_l1);
int
-l1_init(u_int *deb)
+Isdnl1_Init(u_int *deb)
{
debug = deb;
l1fsm_s.state_count = L1S_STATE_COUNT;
@@ -409,7 +409,7 @@ l1_init(u_int *deb)
}
void
-l1_cleanup(void)
+Isdnl1_cleanup(void)
{
mISDN_FsmFree(&l1fsm_s);
}
diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c
index 7dc8026..8550542 100644
--- a/drivers/md/raid1.c
+++ b/drivers/md/raid1.c
@@ -1496,12 +1496,13 @@ static void raid1_write_request(struct mddev *mddev, struct bio *bio,
if (!r1_bio->bios[i])
continue;
- if (first_clone && test_bit(WriteMostly, &rdev->flags)) {
+ if (first_clone) {
/* do behind I/O ?
* Not if there are too many, or cannot
* allocate memory, or a reader on WriteMostly
* is waiting for behind writes to flush */
if (bitmap &&
+ test_bit(WriteMostly, &rdev->flags) &&
(atomic_read(&bitmap->behind_writes)
< mddev->bitmap_info.max_write_behind) &&
!waitqueue_active(&bitmap->behind_wait)) {
diff --git a/drivers/media/Kconfig b/drivers/media/Kconfig
index b078126..f3f24c6 100644
--- a/drivers/media/Kconfig
+++ b/drivers/media/Kconfig
@@ -141,10 +141,10 @@
prompt "Test drivers" if MEDIA_SUPPORT_FILTER
default y if !MEDIA_SUPPORT_FILTER
help
- Those drivers should not be used on production Kernels, but
- can be useful on debug ones. It enables several dummy drivers
- that simulate a real hardware. Very useful to test userspace
- applications and to validate if the subsystem core is doesn't
+ These drivers should not be used on production kernels, but
+ can be useful on debug ones. This option enables several dummy drivers
+ that simulate real hardware. Very useful to test userspace
+ applications and to validate if the subsystem core doesn't
have regressions.
Say Y if you want to use some virtual test driver.
diff --git a/drivers/media/cec/core/cec-adap.c b/drivers/media/cec/core/cec-adap.c
index cd9cb35..2e12331 100644
--- a/drivers/media/cec/core/cec-adap.c
+++ b/drivers/media/cec/core/cec-adap.c
@@ -161,10 +161,10 @@ static void cec_queue_event(struct cec_adapter *adap,
u64 ts = ktime_get_ns();
struct cec_fh *fh;
- mutex_lock(&adap->devnode.lock);
+ mutex_lock(&adap->devnode.lock_fhs);
list_for_each_entry(fh, &adap->devnode.fhs, list)
cec_queue_event_fh(fh, ev, ts);
- mutex_unlock(&adap->devnode.lock);
+ mutex_unlock(&adap->devnode.lock_fhs);
}
/* Notify userspace that the CEC pin changed state at the given time. */
@@ -178,11 +178,12 @@ void cec_queue_pin_cec_event(struct cec_adapter *adap, bool is_high,
};
struct cec_fh *fh;
- mutex_lock(&adap->devnode.lock);
- list_for_each_entry(fh, &adap->devnode.fhs, list)
+ mutex_lock(&adap->devnode.lock_fhs);
+ list_for_each_entry(fh, &adap->devnode.fhs, list) {
if (fh->mode_follower == CEC_MODE_MONITOR_PIN)
cec_queue_event_fh(fh, &ev, ktime_to_ns(ts));
- mutex_unlock(&adap->devnode.lock);
+ }
+ mutex_unlock(&adap->devnode.lock_fhs);
}
EXPORT_SYMBOL_GPL(cec_queue_pin_cec_event);
@@ -195,10 +196,10 @@ void cec_queue_pin_hpd_event(struct cec_adapter *adap, bool is_high, ktime_t ts)
};
struct cec_fh *fh;
- mutex_lock(&adap->devnode.lock);
+ mutex_lock(&adap->devnode.lock_fhs);
list_for_each_entry(fh, &adap->devnode.fhs, list)
cec_queue_event_fh(fh, &ev, ktime_to_ns(ts));
- mutex_unlock(&adap->devnode.lock);
+ mutex_unlock(&adap->devnode.lock_fhs);
}
EXPORT_SYMBOL_GPL(cec_queue_pin_hpd_event);
@@ -211,10 +212,10 @@ void cec_queue_pin_5v_event(struct cec_adapter *adap, bool is_high, ktime_t ts)
};
struct cec_fh *fh;
- mutex_lock(&adap->devnode.lock);
+ mutex_lock(&adap->devnode.lock_fhs);
list_for_each_entry(fh, &adap->devnode.fhs, list)
cec_queue_event_fh(fh, &ev, ktime_to_ns(ts));
- mutex_unlock(&adap->devnode.lock);
+ mutex_unlock(&adap->devnode.lock_fhs);
}
EXPORT_SYMBOL_GPL(cec_queue_pin_5v_event);
@@ -286,12 +287,12 @@ static void cec_queue_msg_monitor(struct cec_adapter *adap,
u32 monitor_mode = valid_la ? CEC_MODE_MONITOR :
CEC_MODE_MONITOR_ALL;
- mutex_lock(&adap->devnode.lock);
+ mutex_lock(&adap->devnode.lock_fhs);
list_for_each_entry(fh, &adap->devnode.fhs, list) {
if (fh->mode_follower >= monitor_mode)
cec_queue_msg_fh(fh, msg);
}
- mutex_unlock(&adap->devnode.lock);
+ mutex_unlock(&adap->devnode.lock_fhs);
}
/*
@@ -302,12 +303,12 @@ static void cec_queue_msg_followers(struct cec_adapter *adap,
{
struct cec_fh *fh;
- mutex_lock(&adap->devnode.lock);
+ mutex_lock(&adap->devnode.lock_fhs);
list_for_each_entry(fh, &adap->devnode.fhs, list) {
if (fh->mode_follower == CEC_MODE_FOLLOWER)
cec_queue_msg_fh(fh, msg);
}
- mutex_unlock(&adap->devnode.lock);
+ mutex_unlock(&adap->devnode.lock_fhs);
}
/* Notify userspace of an adapter state change. */
@@ -342,7 +343,7 @@ static void cec_data_completed(struct cec_data *data)
* Without that we would be referring to a closed filehandle.
*/
if (data->fh)
- list_del(&data->xfer_list);
+ list_del_init(&data->xfer_list);
if (data->blocking) {
/*
@@ -898,6 +899,8 @@ int cec_transmit_msg_fh(struct cec_adapter *adap, struct cec_msg *msg,
if (fh)
list_add_tail(&data->xfer_list, &fh->xfer_list);
+ else
+ INIT_LIST_HEAD(&data->xfer_list);
list_add_tail(&data->list, &adap->transmit_queue);
adap->transmit_queue_sz++;
@@ -923,6 +926,10 @@ int cec_transmit_msg_fh(struct cec_adapter *adap, struct cec_msg *msg,
/* The transmit completed (possibly with an error) */
*msg = data->msg;
+ if (WARN_ON(!list_empty(&data->list)))
+ list_del(&data->list);
+ if (WARN_ON(!list_empty(&data->xfer_list)))
+ list_del(&data->xfer_list);
kfree(data);
return 0;
}
@@ -1573,6 +1580,7 @@ void __cec_s_phys_addr(struct cec_adapter *adap, u16 phys_addr, bool block)
/* Disabling monitor all mode should always succeed */
if (adap->monitor_all_cnt)
WARN_ON(call_op(adap, adap_monitor_all_enable, false));
+ /* serialize adap_enable */
mutex_lock(&adap->devnode.lock);
if (adap->needs_hpd || list_empty(&adap->devnode.fhs)) {
WARN_ON(adap->ops->adap_enable(adap, false));
@@ -1584,14 +1592,16 @@ void __cec_s_phys_addr(struct cec_adapter *adap, u16 phys_addr, bool block)
return;
}
+ /* serialize adap_enable */
mutex_lock(&adap->devnode.lock);
adap->last_initiator = 0xff;
adap->transmit_in_progress = false;
- if ((adap->needs_hpd || list_empty(&adap->devnode.fhs)) &&
- adap->ops->adap_enable(adap, true)) {
- mutex_unlock(&adap->devnode.lock);
- return;
+ if (adap->needs_hpd || list_empty(&adap->devnode.fhs)) {
+ if (adap->ops->adap_enable(adap, true)) {
+ mutex_unlock(&adap->devnode.lock);
+ return;
+ }
}
if (adap->monitor_all_cnt &&
diff --git a/drivers/media/cec/core/cec-api.c b/drivers/media/cec/core/cec-api.c
index 769e6b4..d72ad48 100644
--- a/drivers/media/cec/core/cec-api.c
+++ b/drivers/media/cec/core/cec-api.c
@@ -586,6 +586,7 @@ static int cec_open(struct inode *inode, struct file *filp)
return err;
}
+ /* serialize adap_enable */
mutex_lock(&devnode->lock);
if (list_empty(&devnode->fhs) &&
!adap->needs_hpd &&
@@ -624,7 +625,9 @@ static int cec_open(struct inode *inode, struct file *filp)
}
#endif
+ mutex_lock(&devnode->lock_fhs);
list_add(&fh->list, &devnode->fhs);
+ mutex_unlock(&devnode->lock_fhs);
mutex_unlock(&devnode->lock);
return 0;
@@ -653,8 +656,11 @@ static int cec_release(struct inode *inode, struct file *filp)
cec_monitor_all_cnt_dec(adap);
mutex_unlock(&adap->lock);
+ /* serialize adap_enable */
mutex_lock(&devnode->lock);
+ mutex_lock(&devnode->lock_fhs);
list_del(&fh->list);
+ mutex_unlock(&devnode->lock_fhs);
if (cec_is_registered(adap) && list_empty(&devnode->fhs) &&
!adap->needs_hpd && adap->phys_addr == CEC_PHYS_ADDR_INVALID) {
WARN_ON(adap->ops->adap_enable(adap, false));
@@ -669,7 +675,7 @@ static int cec_release(struct inode *inode, struct file *filp)
data->blocking = false;
data->fh = NULL;
- list_del(&data->xfer_list);
+ list_del_init(&data->xfer_list);
}
mutex_unlock(&adap->lock);
while (!list_empty(&fh->msgs)) {
diff --git a/drivers/media/cec/core/cec-core.c b/drivers/media/cec/core/cec-core.c
index 551689d..ec67065 100644
--- a/drivers/media/cec/core/cec-core.c
+++ b/drivers/media/cec/core/cec-core.c
@@ -169,8 +169,10 @@ static void cec_devnode_unregister(struct cec_adapter *adap)
devnode->registered = false;
devnode->unregistered = true;
+ mutex_lock(&devnode->lock_fhs);
list_for_each_entry(fh, &devnode->fhs, list)
wake_up_interruptible(&fh->wait);
+ mutex_unlock(&devnode->lock_fhs);
mutex_unlock(&devnode->lock);
@@ -272,6 +274,7 @@ struct cec_adapter *cec_allocate_adapter(const struct cec_adap_ops *ops,
/* adap->devnode initialization */
INIT_LIST_HEAD(&adap->devnode.fhs);
+ mutex_init(&adap->devnode.lock_fhs);
mutex_init(&adap->devnode.lock);
adap->kthread = kthread_run(cec_thread_func, adap, "cec-%s", name);
diff --git a/drivers/media/cec/core/cec-pin-priv.h b/drivers/media/cec/core/cec-pin-priv.h
index fb101f1..7bad5a0 100644
--- a/drivers/media/cec/core/cec-pin-priv.h
+++ b/drivers/media/cec/core/cec-pin-priv.h
@@ -170,7 +170,6 @@ struct cec_pin {
ktime_t ts;
unsigned int wait_usecs;
u16 la_mask;
- bool enabled;
bool monitor_all;
bool rx_eom;
bool enable_irq_failed;
diff --git a/drivers/media/cec/core/cec-pin.c b/drivers/media/cec/core/cec-pin.c
index a60b6f0..21f0f74 100644
--- a/drivers/media/cec/core/cec-pin.c
+++ b/drivers/media/cec/core/cec-pin.c
@@ -1033,6 +1033,7 @@ static int cec_pin_thread_func(void *_adap)
{
struct cec_adapter *adap = _adap;
struct cec_pin *pin = adap->pin;
+ bool irq_enabled = false;
for (;;) {
wait_event_interruptible(pin->kthread_waitq,
@@ -1060,6 +1061,7 @@ static int cec_pin_thread_func(void *_adap)
ns_to_ktime(pin->work_rx_msg.rx_ts));
msg->len = 0;
}
+
if (pin->work_tx_status) {
unsigned int tx_status = pin->work_tx_status;
@@ -1083,27 +1085,39 @@ static int cec_pin_thread_func(void *_adap)
switch (atomic_xchg(&pin->work_irq_change,
CEC_PIN_IRQ_UNCHANGED)) {
case CEC_PIN_IRQ_DISABLE:
- pin->ops->disable_irq(adap);
+ if (irq_enabled) {
+ pin->ops->disable_irq(adap);
+ irq_enabled = false;
+ }
cec_pin_high(pin);
cec_pin_to_idle(pin);
hrtimer_start(&pin->timer, ns_to_ktime(0),
HRTIMER_MODE_REL);
break;
case CEC_PIN_IRQ_ENABLE:
+ if (irq_enabled)
+ break;
pin->enable_irq_failed = !pin->ops->enable_irq(adap);
if (pin->enable_irq_failed) {
cec_pin_to_idle(pin);
hrtimer_start(&pin->timer, ns_to_ktime(0),
HRTIMER_MODE_REL);
+ } else {
+ irq_enabled = true;
}
break;
default:
break;
}
-
if (kthread_should_stop())
break;
}
+ if (pin->ops->disable_irq && irq_enabled)
+ pin->ops->disable_irq(adap);
+ hrtimer_cancel(&pin->timer);
+ cec_pin_read(pin);
+ cec_pin_to_idle(pin);
+ pin->state = CEC_ST_OFF;
return 0;
}
@@ -1111,7 +1125,6 @@ static int cec_pin_adap_enable(struct cec_adapter *adap, bool enable)
{
struct cec_pin *pin = adap->pin;
- pin->enabled = enable;
if (enable) {
atomic_set(&pin->work_pin_num_events, 0);
pin->work_pin_events_rd = pin->work_pin_events_wr = 0;
@@ -1130,13 +1143,7 @@ static int cec_pin_adap_enable(struct cec_adapter *adap, bool enable)
hrtimer_start(&pin->timer, ns_to_ktime(0),
HRTIMER_MODE_REL);
} else {
- if (pin->ops->disable_irq)
- pin->ops->disable_irq(adap);
- hrtimer_cancel(&pin->timer);
kthread_stop(pin->kthread);
- cec_pin_read(pin);
- cec_pin_to_idle(pin);
- pin->state = CEC_ST_OFF;
}
return 0;
}
@@ -1157,11 +1164,8 @@ void cec_pin_start_timer(struct cec_pin *pin)
if (pin->state != CEC_ST_RX_IRQ)
return;
- atomic_set(&pin->work_irq_change, CEC_PIN_IRQ_UNCHANGED);
- pin->ops->disable_irq(pin->adap);
- cec_pin_high(pin);
- cec_pin_to_idle(pin);
- hrtimer_start(&pin->timer, ns_to_ktime(0), HRTIMER_MODE_REL);
+ atomic_set(&pin->work_irq_change, CEC_PIN_IRQ_DISABLE);
+ wake_up_interruptible(&pin->kthread_waitq);
}
static int cec_pin_adap_transmit(struct cec_adapter *adap, u8 attempts,
diff --git a/drivers/media/common/b2c2/flexcop.c b/drivers/media/common/b2c2/flexcop.c
index cbaa61f..e7a88a2 100644
--- a/drivers/media/common/b2c2/flexcop.c
+++ b/drivers/media/common/b2c2/flexcop.c
@@ -1,3 +1,4 @@
+// SPDX-License-Identifier: LGPL-2.1-or-later
/*
* Linux driver for digital TV devices equipped with B2C2 FlexcopII(b)/III
* flexcop.c - main module part
@@ -15,16 +16,6 @@
* Uwe Bugla, uwe.bugla at gmx.de (doing tests, restyling code, writing docu)
* Niklas Peinecke, peinecke at gdv.uni-hannover.de (hardware pid/mac
* filtering)
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU Lesser General Public License
- * as published by the Free Software Foundation; either version 2.1
- * of the License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
*/
#include "flexcop.h"
diff --git a/drivers/media/common/saa7146/saa7146_fops.c b/drivers/media/common/saa7146/saa7146_fops.c
index baf5772..e9a15de 100644
--- a/drivers/media/common/saa7146/saa7146_fops.c
+++ b/drivers/media/common/saa7146/saa7146_fops.c
@@ -487,6 +487,7 @@ int saa7146_vv_init(struct saa7146_dev* dev, struct saa7146_ext_vv *ext_vv)
if (hdl->error) {
err = hdl->error;
v4l2_ctrl_handler_free(hdl);
+ v4l2_device_unregister(&dev->v4l2_dev);
return err;
}
dev->v4l2_dev.ctrl_handler = hdl;
@@ -495,6 +496,7 @@ int saa7146_vv_init(struct saa7146_dev* dev, struct saa7146_ext_vv *ext_vv)
if (vv == NULL) {
ERR("out of memory. aborting.\n");
v4l2_ctrl_handler_free(hdl);
+ v4l2_device_unregister(&dev->v4l2_dev);
return -ENOMEM;
}
ext_vv->vid_ops = saa7146_video_ioctl_ops;
@@ -521,7 +523,8 @@ int saa7146_vv_init(struct saa7146_dev* dev, struct saa7146_ext_vv *ext_vv)
ERR("out of memory. aborting.\n");
kfree(vv);
v4l2_ctrl_handler_free(hdl);
- return -1;
+ v4l2_device_unregister(&dev->v4l2_dev);
+ return -ENOMEM;
}
saa7146_video_uops.init(dev,vv);
diff --git a/drivers/media/common/videobuf2/frame_vector.c b/drivers/media/common/videobuf2/frame_vector.c
index ce879f6..542dde9 100644
--- a/drivers/media/common/videobuf2/frame_vector.c
+++ b/drivers/media/common/videobuf2/frame_vector.c
@@ -37,6 +37,7 @@ int get_vaddr_frames(unsigned long start, unsigned int nr_frames,
{
struct mm_struct *mm = current->mm;
struct vm_area_struct *vma;
+ int ret_pin_user_pages_fast = 0;
int ret = 0;
int err;
@@ -56,6 +57,7 @@ int get_vaddr_frames(unsigned long start, unsigned int nr_frames,
vec->is_pfns = false;
goto out_unlocked;
}
+ ret_pin_user_pages_fast = ret;
mmap_read_lock(mm);
vec->got_ref = false;
@@ -71,7 +73,18 @@ int get_vaddr_frames(unsigned long start, unsigned int nr_frames,
while (ret < nr_frames && start + PAGE_SIZE <= vma->vm_end) {
err = follow_pfn(vma, start, &nums[ret]);
if (err) {
- if (ret == 0)
+ if (ret)
+ goto out;
+ // If follow_pfn() returns -EINVAL, then this
+ // is not an IO mapping or a raw PFN mapping.
+ // In that case, return the original error from
+ // pin_user_pages_fast(). Otherwise this
+ // function would return -EINVAL when
+ // pin_user_pages_fast() returned -ENOMEM,
+ // which makes debugging hard.
+ if (err == -EINVAL && ret_pin_user_pages_fast)
+ ret = ret_pin_user_pages_fast;
+ else
ret = err;
goto out;
}
diff --git a/drivers/media/common/videobuf2/videobuf2-dma-contig.c b/drivers/media/common/videobuf2/videobuf2-dma-contig.c
index 556e42b..7c4096e 100644
--- a/drivers/media/common/videobuf2/videobuf2-dma-contig.c
+++ b/drivers/media/common/videobuf2/videobuf2-dma-contig.c
@@ -257,7 +257,7 @@ static void *vb2_dc_alloc(struct vb2_buffer *vb,
ret = vb2_dc_alloc_coherent(buf);
if (ret) {
- dev_err(dev, "dma alloc of size %ld failed\n", size);
+ dev_err(dev, "dma alloc of size %lu failed\n", size);
kfree(buf);
return ERR_PTR(-ENOMEM);
}
@@ -298,9 +298,9 @@ static int vb2_dc_mmap(void *buf_priv, struct vm_area_struct *vma)
vma->vm_ops->open(vma);
- pr_debug("%s: mapped dma addr 0x%08lx at 0x%08lx, size %ld\n",
- __func__, (unsigned long)buf->dma_addr, vma->vm_start,
- buf->size);
+ pr_debug("%s: mapped dma addr 0x%08lx at 0x%08lx, size %lu\n",
+ __func__, (unsigned long)buf->dma_addr, vma->vm_start,
+ buf->size);
return 0;
}
diff --git a/drivers/media/dvb-core/dmxdev.c b/drivers/media/dvb-core/dmxdev.c
index 5d5a484..f6ee678 100644
--- a/drivers/media/dvb-core/dmxdev.c
+++ b/drivers/media/dvb-core/dmxdev.c
@@ -1,19 +1,9 @@
+// SPDX-License-Identifier: LGPL-2.1-or-later
/*
* dmxdev.c - DVB demultiplexer device
*
* Copyright (C) 2000 Ralph Metzler & Marcus Metzler
* for convergence integrated media GmbH
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU Lesser General Public License
- * as published by the Free Software Foundation; either version 2.1
- * of the License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
*/
#define pr_fmt(fmt) "dmxdev: " fmt
@@ -1413,7 +1403,7 @@ static const struct dvb_device dvbdev_dvr = {
};
int dvb_dmxdev_init(struct dmxdev *dmxdev, struct dvb_adapter *dvb_adapter)
{
- int i;
+ int i, ret;
if (dmxdev->demux->open(dmxdev->demux) < 0)
return -EUSERS;
@@ -1432,14 +1422,26 @@ int dvb_dmxdev_init(struct dmxdev *dmxdev, struct dvb_adapter *dvb_adapter)
DMXDEV_STATE_FREE);
}
- dvb_register_device(dvb_adapter, &dmxdev->dvbdev, &dvbdev_demux, dmxdev,
+ ret = dvb_register_device(dvb_adapter, &dmxdev->dvbdev, &dvbdev_demux, dmxdev,
DVB_DEVICE_DEMUX, dmxdev->filternum);
- dvb_register_device(dvb_adapter, &dmxdev->dvr_dvbdev, &dvbdev_dvr,
+ if (ret < 0)
+ goto err_register_dvbdev;
+
+ ret = dvb_register_device(dvb_adapter, &dmxdev->dvr_dvbdev, &dvbdev_dvr,
dmxdev, DVB_DEVICE_DVR, dmxdev->filternum);
+ if (ret < 0)
+ goto err_register_dvr_dvbdev;
dvb_ringbuffer_init(&dmxdev->dvr_buffer, NULL, 8192);
return 0;
+
+err_register_dvr_dvbdev:
+ dvb_unregister_device(dmxdev->dvbdev);
+err_register_dvbdev:
+ vfree(dmxdev->filter);
+ dmxdev->filter = NULL;
+ return ret;
}
EXPORT_SYMBOL(dvb_dmxdev_init);
diff --git a/drivers/media/dvb-core/dvb_demux.c b/drivers/media/dvb-core/dvb_demux.c
index 5fde1d3..83cc32a 100644
--- a/drivers/media/dvb-core/dvb_demux.c
+++ b/drivers/media/dvb-core/dvb_demux.c
@@ -1,20 +1,10 @@
+// SPDX-License-Identifier: LGPL-2.1-or-later
/*
* dvb_demux.c - DVB kernel demux API
*
* Copyright (C) 2000-2001 Ralph Metzler <ralph@convergence.de>
* & Marcus Metzler <marcus@convergence.de>
* for convergence integrated media GmbH
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU Lesser General Public License
- * as published by the Free Software Foundation; either version 2.1
- * of the License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
*/
#define pr_fmt(fmt) "dvb_demux: " fmt
diff --git a/drivers/media/dvb-core/dvb_frontend.c b/drivers/media/dvb-core/dvb_frontend.c
index 258637d..48e735c 100644
--- a/drivers/media/dvb-core/dvb_frontend.c
+++ b/drivers/media/dvb-core/dvb_frontend.c
@@ -2554,8 +2554,7 @@ static int dvb_frontend_handle_ioctl(struct file *file,
case FE_DISEQC_SEND_BURST:
if (fe->ops.diseqc_send_burst) {
- err = fe->ops.diseqc_send_burst(fe,
- (enum fe_sec_mini_cmd)parg);
+ err = fe->ops.diseqc_send_burst(fe, (long)parg);
fepriv->state = FESTATE_DISEQC;
fepriv->status = 0;
}
@@ -2563,9 +2562,8 @@ static int dvb_frontend_handle_ioctl(struct file *file,
case FE_SET_TONE:
if (fe->ops.set_tone) {
- err = fe->ops.set_tone(fe,
- (enum fe_sec_tone_mode)parg);
- fepriv->tone = (enum fe_sec_tone_mode)parg;
+ fepriv->tone = (long)parg;
+ err = fe->ops.set_tone(fe, fepriv->tone);
fepriv->state = FESTATE_DISEQC;
fepriv->status = 0;
}
@@ -2573,9 +2571,8 @@ static int dvb_frontend_handle_ioctl(struct file *file,
case FE_SET_VOLTAGE:
if (fe->ops.set_voltage) {
- err = fe->ops.set_voltage(fe,
- (enum fe_sec_voltage)parg);
- fepriv->voltage = (enum fe_sec_voltage)parg;
+ fepriv->voltage = (long)parg;
+ err = fe->ops.set_voltage(fe, fepriv->voltage);
fepriv->state = FESTATE_DISEQC;
fepriv->status = 0;
}
@@ -2935,7 +2932,9 @@ int dvb_frontend_suspend(struct dvb_frontend *fe)
else if (fe->ops.tuner_ops.sleep)
ret = fe->ops.tuner_ops.sleep(fe);
- if (fe->ops.sleep)
+ if (fe->ops.suspend)
+ ret = fe->ops.suspend(fe);
+ else if (fe->ops.sleep)
ret = fe->ops.sleep(fe);
return ret;
@@ -2951,7 +2950,9 @@ int dvb_frontend_resume(struct dvb_frontend *fe)
fe->id);
fe->exit = DVB_FE_DEVICE_RESUME;
- if (fe->ops.init)
+ if (fe->ops.resume)
+ ret = fe->ops.resume(fe);
+ else if (fe->ops.init)
ret = fe->ops.init(fe);
if (fe->ops.tuner_ops.resume)
diff --git a/drivers/media/dvb-core/dvb_vb2.c b/drivers/media/dvb-core/dvb_vb2.c
index 959d110..a1bd6d9c 100644
--- a/drivers/media/dvb-core/dvb_vb2.c
+++ b/drivers/media/dvb-core/dvb_vb2.c
@@ -5,10 +5,6 @@
* Copyright (C) 2015 Samsung Electronics
*
* Author: jh1009.sung@samsung.com
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation.
*/
#include <linux/err.h>
diff --git a/drivers/media/dvb-core/dvbdev.c b/drivers/media/dvb-core/dvbdev.c
index 795d9bf..675d877 100644
--- a/drivers/media/dvb-core/dvbdev.c
+++ b/drivers/media/dvb-core/dvbdev.c
@@ -1,20 +1,10 @@
+// SPDX-License-Identifier: LGPL-2.1-or-later
/*
* dvbdev.c
*
* Copyright (C) 2000 Ralph Metzler <ralph@convergence.de>
* & Marcus Metzler <marcus@convergence.de>
* for convergence integrated media GmbH
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU Lesser General Public License
- * as published by the Free Software Foundation; either version 2.1
- * of the License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
*/
#define pr_fmt(fmt) "dvbdev: " fmt
diff --git a/drivers/media/dvb-frontends/cx24113.c b/drivers/media/dvb-frontends/cx24113.c
index 60a9f702..dd55d31 100644
--- a/drivers/media/dvb-frontends/cx24113.c
+++ b/drivers/media/dvb-frontends/cx24113.c
@@ -378,7 +378,7 @@ static void cx24113_set_nfr(struct cx24113_state *state, u16 n, s32 f, u8 r)
static int cx24113_set_frequency(struct cx24113_state *state, u32 frequency)
{
- u8 r = 1; /* or 2 */
+ u8 r;
u16 n = 6;
s32 f = 0;
diff --git a/drivers/media/dvb-frontends/dib8000.c b/drivers/media/dvb-frontends/dib8000.c
index bb02354..d67f2dd 100644
--- a/drivers/media/dvb-frontends/dib8000.c
+++ b/drivers/media/dvb-frontends/dib8000.c
@@ -4473,8 +4473,10 @@ static struct dvb_frontend *dib8000_init(struct i2c_adapter *i2c_adap, u8 i2c_ad
state->timf_default = cfg->pll->timf;
- if (dib8000_identify(&state->i2c) == 0)
+ if (dib8000_identify(&state->i2c) == 0) {
+ kfree(fe);
goto error;
+ }
dibx000_init_i2c_master(&state->i2c_master, DIB8000, state->i2c.adap, state->i2c.addr);
diff --git a/drivers/media/dvb-frontends/dib9000.c b/drivers/media/dvb-frontends/dib9000.c
index 04d92d6..914ca82 100644
--- a/drivers/media/dvb-frontends/dib9000.c
+++ b/drivers/media/dvb-frontends/dib9000.c
@@ -258,7 +258,7 @@ static int dib9000_read16_attr(struct dib9000_state *state, u16 reg, u8 *b, u32
state->i2c_write_buffer[0] |= (1 << 4);
do {
- l = len < chunk_size ? len : chunk_size;
+ l = min(len, chunk_size);
state->msg[1].len = l;
state->msg[1].buf = b;
ret = i2c_transfer(state->i2c.i2c_adap, state->msg, 2) != 2 ? -EREMOTEIO : 0;
@@ -342,7 +342,7 @@ static int dib9000_write16_attr(struct dib9000_state *state, u16 reg, const u8 *
state->i2c_write_buffer[0] |= (1 << 4);
do {
- l = len < chunk_size ? len : chunk_size;
+ l = min(len, chunk_size);
state->msg[0].len = l + 2;
memcpy(&state->i2c_write_buffer[2], buf, l);
diff --git a/drivers/media/dvb-frontends/drxd_hard.c b/drivers/media/dvb-frontends/drxd_hard.c
index a7eb81d..9860cae 100644
--- a/drivers/media/dvb-frontends/drxd_hard.c
+++ b/drivers/media/dvb-frontends/drxd_hard.c
@@ -914,44 +914,36 @@ static int DownloadMicrocode(struct drxd_state *state,
u32 Address;
u16 nBlocks;
u16 BlockSize;
- u32 offset = 0;
int i, status = 0;
pSrc = (u8 *) pMCImage;
/* We're not using Flags */
/* Flags = (pSrc[0] << 8) | pSrc[1]; */
pSrc += sizeof(u16);
- offset += sizeof(u16);
nBlocks = (pSrc[0] << 8) | pSrc[1];
pSrc += sizeof(u16);
- offset += sizeof(u16);
for (i = 0; i < nBlocks; i++) {
Address = (pSrc[0] << 24) | (pSrc[1] << 16) |
(pSrc[2] << 8) | pSrc[3];
pSrc += sizeof(u32);
- offset += sizeof(u32);
BlockSize = ((pSrc[0] << 8) | pSrc[1]) * sizeof(u16);
pSrc += sizeof(u16);
- offset += sizeof(u16);
/* We're not using Flags */
/* u16 Flags = (pSrc[0] << 8) | pSrc[1]; */
pSrc += sizeof(u16);
- offset += sizeof(u16);
/* We're not using BlockCRC */
/* u16 BlockCRC = (pSrc[0] << 8) | pSrc[1]; */
pSrc += sizeof(u16);
- offset += sizeof(u16);
status = WriteBlock(state, Address, BlockSize,
pSrc, DRX_I2C_CLEARCRC);
if (status < 0)
break;
pSrc += BlockSize;
- offset += BlockSize;
}
return status;
diff --git a/drivers/media/dvb-frontends/drxk_hard.c b/drivers/media/dvb-frontends/drxk_hard.c
index d7fc259..9430295 100644
--- a/drivers/media/dvb-frontends/drxk_hard.c
+++ b/drivers/media/dvb-frontends/drxk_hard.c
@@ -3720,7 +3720,6 @@ static int set_dvbt(struct drxk_state *state, u16 intermediate_freqk_hz,
{
u16 cmd_result = 0;
u16 transmission_params = 0;
- u16 operation_mode = 0;
u32 iqm_rc_rate_ofs = 0;
u32 bandwidth = 0;
u16 param1;
@@ -3759,10 +3758,8 @@ static int set_dvbt(struct drxk_state *state, u16 intermediate_freqk_hz,
/* mode */
switch (state->props.transmission_mode) {
case TRANSMISSION_MODE_AUTO:
- default:
- operation_mode |= OFDM_SC_RA_RAM_OP_AUTO_MODE__M;
- fallthrough; /* try first guess DRX_FFTMODE_8K */
case TRANSMISSION_MODE_8K:
+ default:
transmission_params |= OFDM_SC_RA_RAM_OP_PARAM_MODE_8K;
break;
case TRANSMISSION_MODE_2K:
@@ -3773,9 +3770,7 @@ static int set_dvbt(struct drxk_state *state, u16 intermediate_freqk_hz,
/* guard */
switch (state->props.guard_interval) {
default:
- case GUARD_INTERVAL_AUTO:
- operation_mode |= OFDM_SC_RA_RAM_OP_AUTO_GUARD__M;
- fallthrough; /* try first guess DRX_GUARD_1DIV4 */
+ case GUARD_INTERVAL_AUTO: /* try first guess DRX_GUARD_1DIV4 */
case GUARD_INTERVAL_1_4:
transmission_params |= OFDM_SC_RA_RAM_OP_PARAM_GUARD_4;
break;
@@ -3794,11 +3789,7 @@ static int set_dvbt(struct drxk_state *state, u16 intermediate_freqk_hz,
switch (state->props.hierarchy) {
case HIERARCHY_AUTO:
case HIERARCHY_NONE:
- default:
- operation_mode |= OFDM_SC_RA_RAM_OP_AUTO_HIER__M;
- /* try first guess SC_RA_RAM_OP_PARAM_HIER_NO */
- /* transmission_params |= OFDM_SC_RA_RAM_OP_PARAM_HIER_NO; */
- fallthrough;
+ default: /* try first guess SC_RA_RAM_OP_PARAM_HIER_NO */
case HIERARCHY_1:
transmission_params |= OFDM_SC_RA_RAM_OP_PARAM_HIER_A1;
break;
@@ -3814,9 +3805,7 @@ static int set_dvbt(struct drxk_state *state, u16 intermediate_freqk_hz,
/* modulation */
switch (state->props.modulation) {
case QAM_AUTO:
- default:
- operation_mode |= OFDM_SC_RA_RAM_OP_AUTO_CONST__M;
- fallthrough; /* try first guess DRX_CONSTELLATION_QAM64 */
+ default: /* try first guess DRX_CONSTELLATION_QAM64 */
case QAM_64:
transmission_params |= OFDM_SC_RA_RAM_OP_PARAM_CONST_QAM64;
break;
@@ -3857,9 +3846,7 @@ static int set_dvbt(struct drxk_state *state, u16 intermediate_freqk_hz,
/* coderate */
switch (state->props.code_rate_HP) {
case FEC_AUTO:
- default:
- operation_mode |= OFDM_SC_RA_RAM_OP_AUTO_RATE__M;
- fallthrough; /* try first guess DRX_CODERATE_2DIV3 */
+ default: /* try first guess DRX_CODERATE_2DIV3 */
case FEC_2_3:
transmission_params |= OFDM_SC_RA_RAM_OP_PARAM_RATE_2_3;
break;
diff --git a/drivers/media/dvb-frontends/m88ds3103.c b/drivers/media/dvb-frontends/m88ds3103.c
index 02e8aa1..bce0f42 100644
--- a/drivers/media/dvb-frontends/m88ds3103.c
+++ b/drivers/media/dvb-frontends/m88ds3103.c
@@ -451,7 +451,7 @@ static int m88ds3103b_select_mclk(struct m88ds3103_dev *dev)
static int m88ds3103b_set_mclk(struct m88ds3103_dev *dev, u32 mclk_khz)
{
- u8 reg11 = 0x0A, reg15, reg16, reg1D, reg1E, reg1F, tmp;
+ u8 reg15, reg16, reg1D, reg1E, reg1F, tmp;
u8 sm, f0 = 0, f1 = 0, f2 = 0, f3 = 0;
u16 pll_div_fb, N;
u32 div;
@@ -480,8 +480,6 @@ static int m88ds3103b_set_mclk(struct m88ds3103_dev *dev, u32 mclk_khz)
div /= mclk_khz;
if (dev->cfg->ts_mode == M88DS3103_TS_SERIAL) {
- reg11 |= 0x02;
-
if (div <= 32) {
N = 2;
@@ -532,8 +530,6 @@ static int m88ds3103b_set_mclk(struct m88ds3103_dev *dev, u32 mclk_khz)
else if ((f3 < 8) && (f3 != 0))
f3 = 8;
} else {
- reg11 &= ~0x02;
-
if (div <= 32) {
N = 2;
diff --git a/drivers/media/dvb-frontends/s5h1411.c b/drivers/media/dvb-frontends/s5h1411.c
index c1334d7..2563a72 100644
--- a/drivers/media/dvb-frontends/s5h1411.c
+++ b/drivers/media/dvb-frontends/s5h1411.c
@@ -150,7 +150,7 @@ static struct vsb_snr_tab {
{ 0x35b, 235, },
{ 0x353, 230, },
{ 0x349, 225, },
- { 0x340, 320, },
+ { 0x340, 220, },
{ 0x337, 215, },
{ 0x327, 210, },
{ 0x31b, 205, },
diff --git a/drivers/media/dvb-frontends/si2168.c b/drivers/media/dvb-frontends/si2168.c
index 14b93a7..196e028 100644
--- a/drivers/media/dvb-frontends/si2168.c
+++ b/drivers/media/dvb-frontends/si2168.c
@@ -448,23 +448,10 @@ static int si2168_init(struct dvb_frontend *fe)
/* request the firmware, this will block and timeout */
ret = request_firmware(&fw, dev->firmware_name, &client->dev);
if (ret) {
- /* fallback mechanism to handle old name for Si2168 B40 fw */
- if (dev->chip_id == SI2168_CHIP_ID_B40) {
- dev->firmware_name = SI2168_B40_FIRMWARE_FALLBACK;
- ret = request_firmware(&fw, dev->firmware_name,
- &client->dev);
- }
-
- if (ret == 0) {
- dev_notice(&client->dev,
- "please install firmware file '%s'\n",
- SI2168_B40_FIRMWARE);
- } else {
- dev_err(&client->dev,
- "firmware file '%s' not found\n",
- dev->firmware_name);
- goto err_release_firmware;
- }
+ dev_err(&client->dev,
+ "firmware file '%s' not found\n",
+ dev->firmware_name);
+ goto err_release_firmware;
}
dev_info(&client->dev, "downloading firmware from file '%s'\n",
@@ -527,6 +514,7 @@ static int si2168_init(struct dvb_frontend *fe)
goto err;
dev->warm = true;
+ dev->initialized = true;
warm:
/* Init stats here to indicate which stats are supported */
c->cnr.len = 1;
@@ -548,6 +536,26 @@ static int si2168_init(struct dvb_frontend *fe)
return ret;
}
+static int si2168_resume(struct dvb_frontend *fe)
+{
+ struct i2c_client *client = fe->demodulator_priv;
+ struct si2168_dev *dev = i2c_get_clientdata(client);
+
+ /*
+ * check whether si2168_init() has been called successfully
+ * outside of a resume cycle. Only call it (and load firmware)
+ * in this case. si2168_init() is only called during resume
+ * once the device has actually been used. Otherwise, leave the
+ * device untouched.
+ */
+ if (dev->initialized) {
+ dev_dbg(&client->dev, "previously initialized, call si2168_init()\n");
+ return si2168_init(fe);
+ }
+ dev_dbg(&client->dev, "not initialized yet, skipping init on resume\n");
+ return 0;
+}
+
static int si2168_sleep(struct dvb_frontend *fe)
{
struct i2c_client *client = fe->demodulator_priv;
@@ -657,6 +665,7 @@ static const struct dvb_frontend_ops si2168_ops = {
.init = si2168_init,
.sleep = si2168_sleep,
+ .resume = si2168_resume,
.set_frontend = si2168_set_frontend,
diff --git a/drivers/media/dvb-frontends/si2168_priv.h b/drivers/media/dvb-frontends/si2168_priv.h
index 18bea52..3a89762 100644
--- a/drivers/media/dvb-frontends/si2168_priv.h
+++ b/drivers/media/dvb-frontends/si2168_priv.h
@@ -18,7 +18,6 @@
#define SI2168_A30_FIRMWARE "dvb-demod-si2168-a30-01.fw"
#define SI2168_B40_FIRMWARE "dvb-demod-si2168-b40-01.fw"
#define SI2168_D60_FIRMWARE "dvb-demod-si2168-d60-01.fw"
-#define SI2168_B40_FIRMWARE_FALLBACK "dvb-demod-si2168-02.fw"
/* state struct */
struct si2168_dev {
@@ -37,6 +36,7 @@ struct si2168_dev {
u8 ts_mode;
unsigned int active:1;
unsigned int warm:1;
+ unsigned int initialized:1;
unsigned int ts_clock_inv:1;
unsigned int ts_clock_gapped:1;
unsigned int spectral_inversion:1;
diff --git a/drivers/media/dvb-frontends/si21xx.c b/drivers/media/dvb-frontends/si21xx.c
index e31eb2c..001b235 100644
--- a/drivers/media/dvb-frontends/si21xx.c
+++ b/drivers/media/dvb-frontends/si21xx.c
@@ -711,7 +711,7 @@ static int si21xx_set_frontend(struct dvb_frontend *fe)
int i;
bool inband_interferer_div2[ALLOWABLE_FS_COUNT];
bool inband_interferer_div4[ALLOWABLE_FS_COUNT];
- int status;
+ int status = 0;
/* allowable sample rates for ADC in MHz */
int afs[ALLOWABLE_FS_COUNT] = { 200, 192, 193, 194, 195,
@@ -747,8 +747,6 @@ static int si21xx_set_frontend(struct dvb_frontend *fe)
rf_freq = 10 * c->frequency ;
data_rate = c->symbol_rate / 100;
- status = PASS;
-
band_low = (rf_freq - lnb_lo) - ((lnb_uncertanity * 200)
+ (data_rate * 135)) / 200;
@@ -832,6 +830,9 @@ static int si21xx_set_frontend(struct dvb_frontend *fe)
state->fs = sample_rate;/*ADC MHz*/
si21xx_setacquire(fe, c->symbol_rate, c->fec_inner);
+ if (status)
+ return -EREMOTEIO;
+
return 0;
}
diff --git a/drivers/media/dvb-frontends/sp887x.c b/drivers/media/dvb-frontends/sp887x.c
index c89a91a..146e7f2 100644
--- a/drivers/media/dvb-frontends/sp887x.c
+++ b/drivers/media/dvb-frontends/sp887x.c
@@ -140,7 +140,7 @@ static int sp887x_initial_setup (struct dvb_frontend* fe, const struct firmware
u8 buf [BLOCKSIZE + 2];
int i;
int fw_size = fw->size;
- const unsigned char *mem = fw->data;
+ const unsigned char *mem = fw->data + 10;
dprintk("%s\n", __func__);
@@ -148,8 +148,6 @@ static int sp887x_initial_setup (struct dvb_frontend* fe, const struct firmware
if (fw_size < FW_SIZE + 10)
return -ENODEV;
- mem = fw->data + 10;
-
/* soft reset */
sp887x_writereg(state, 0xf1a, 0x000);
diff --git a/drivers/media/dvb-frontends/stb6100.c b/drivers/media/dvb-frontends/stb6100.c
index d541d66..698866c 100644
--- a/drivers/media/dvb-frontends/stb6100.c
+++ b/drivers/media/dvb-frontends/stb6100.c
@@ -110,7 +110,7 @@ static const struct stb6100_regmask stb6100_template[] = {
/*
* Currently unused. Some boards might need it in the future
*/
-static inline void stb6100_normalise_regs(u8 regs[])
+static __always_unused inline void stb6100_normalise_regs(u8 regs[])
{
int i;
diff --git a/drivers/media/dvb-frontends/stv0367.c b/drivers/media/dvb-frontends/stv0367.c
index 6c2b05f..95e376f 100644
--- a/drivers/media/dvb-frontends/stv0367.c
+++ b/drivers/media/dvb-frontends/stv0367.c
@@ -1797,11 +1797,7 @@ static u32 stv0367cab_get_mclk(struct dvb_frontend *fe, u32 ExtClk_Hz)
static u32 stv0367cab_get_adc_freq(struct dvb_frontend *fe, u32 ExtClk_Hz)
{
- u32 ADCClk_Hz = ExtClk_Hz;
-
- ADCClk_Hz = stv0367cab_get_mclk(fe, ExtClk_Hz);
-
- return ADCClk_Hz;
+ return stv0367cab_get_mclk(fe, ExtClk_Hz);
}
static enum stv0367cab_mod stv0367cab_SetQamSize(struct stv0367_state *state,
diff --git a/drivers/media/i2c/Kconfig b/drivers/media/i2c/Kconfig
index d6a5d4ca..69c56e2 100644
--- a/drivers/media/i2c/Kconfig
+++ b/drivers/media/i2c/Kconfig
@@ -469,6 +469,7 @@
config VIDEO_MAX9286
tristate "Maxim MAX9286 GMSL deserializer support"
depends on I2C && I2C_MUX
+ depends on VIDEO_V4L2
depends on OF_GPIO
select V4L2_FWNODE
select VIDEO_V4L2_SUBDEV_API
@@ -1058,6 +1059,17 @@
To compile this driver as a module, choose M here: the
module will be called ov5675.
+config VIDEO_OV5693
+ tristate "OmniVision OV5693 sensor support"
+ depends on I2C && VIDEO_V4L2
+ select V4L2_FWNODE
+ help
+ This is a Video4Linux2 sensor driver for the OmniVision
+ OV5693 camera.
+
+ To compile this driver as a module, choose M here: the
+ module will be called ov5693.
+
config VIDEO_OV5695
tristate "OmniVision OV5695 sensor support"
depends on I2C && VIDEO_V4L2
diff --git a/drivers/media/i2c/Makefile b/drivers/media/i2c/Makefile
index 4d4fe08..b01f6cd 100644
--- a/drivers/media/i2c/Makefile
+++ b/drivers/media/i2c/Makefile
@@ -75,6 +75,7 @@
obj-$(CONFIG_VIDEO_OV5648) += ov5648.o
obj-$(CONFIG_VIDEO_OV5670) += ov5670.o
obj-$(CONFIG_VIDEO_OV5675) += ov5675.o
+obj-$(CONFIG_VIDEO_OV5693) += ov5693.o
obj-$(CONFIG_VIDEO_OV5695) += ov5695.o
obj-$(CONFIG_VIDEO_OV6650) += ov6650.o
obj-$(CONFIG_VIDEO_OV7251) += ov7251.o
diff --git a/drivers/media/i2c/adv7511-v4l2.c b/drivers/media/i2c/adv7511-v4l2.c
index 41f4e74..8e13cae 100644
--- a/drivers/media/i2c/adv7511-v4l2.c
+++ b/drivers/media/i2c/adv7511-v4l2.c
@@ -270,28 +270,6 @@ static int adv7511_pktmem_rd(struct v4l2_subdev *sd, u8 reg)
return adv_smbus_read_byte_data(state->i2c_pktmem, reg);
}
-static int adv7511_pktmem_wr(struct v4l2_subdev *sd, u8 reg, u8 val)
-{
- struct adv7511_state *state = get_adv7511_state(sd);
- int ret;
- int i;
-
- for (i = 0; i < 3; i++) {
- ret = i2c_smbus_write_byte_data(state->i2c_pktmem, reg, val);
- if (ret == 0)
- return 0;
- }
- v4l2_err(sd, "%s: i2c write error\n", __func__);
- return ret;
-}
-
-/* To set specific bits in the register, a clear-mask is given (to be AND-ed),
- and then the value-mask (to be OR-ed). */
-static inline void adv7511_pktmem_wr_and_or(struct v4l2_subdev *sd, u8 reg, u8 clr_mask, u8 val_mask)
-{
- adv7511_pktmem_wr(sd, reg, (adv7511_pktmem_rd(sd, reg) & clr_mask) | val_mask);
-}
-
static inline bool adv7511_have_hotplug(struct v4l2_subdev *sd)
{
return adv7511_rd(sd, 0x42) & MASK_ADV7511_HPD_DETECT;
diff --git a/drivers/media/i2c/adv7604.c b/drivers/media/i2c/adv7604.c
index 44768b5..a2fa408 100644
--- a/drivers/media/i2c/adv7604.c
+++ b/drivers/media/i2c/adv7604.c
@@ -398,14 +398,14 @@ static inline int io_write_clr_set(struct v4l2_subdev *sd, u8 reg, u8 mask,
return io_write(sd, reg, (io_read(sd, reg) & ~mask) | val);
}
-static inline int avlink_read(struct v4l2_subdev *sd, u8 reg)
+static inline int __always_unused avlink_read(struct v4l2_subdev *sd, u8 reg)
{
struct adv76xx_state *state = to_state(sd);
return adv76xx_read_check(state, ADV7604_PAGE_AVLINK, reg);
}
-static inline int avlink_write(struct v4l2_subdev *sd, u8 reg, u8 val)
+static inline int __always_unused avlink_write(struct v4l2_subdev *sd, u8 reg, u8 val)
{
struct adv76xx_state *state = to_state(sd);
@@ -439,14 +439,14 @@ static inline int infoframe_read(struct v4l2_subdev *sd, u8 reg)
return adv76xx_read_check(state, ADV76XX_PAGE_INFOFRAME, reg);
}
-static inline int infoframe_write(struct v4l2_subdev *sd, u8 reg, u8 val)
+static inline int __always_unused infoframe_write(struct v4l2_subdev *sd, u8 reg, u8 val)
{
struct adv76xx_state *state = to_state(sd);
return regmap_write(state->regmap[ADV76XX_PAGE_INFOFRAME], reg, val);
}
-static inline int afe_read(struct v4l2_subdev *sd, u8 reg)
+static inline int __always_unused afe_read(struct v4l2_subdev *sd, u8 reg)
{
struct adv76xx_state *state = to_state(sd);
@@ -479,14 +479,14 @@ static inline int rep_write_clr_set(struct v4l2_subdev *sd, u8 reg, u8 mask, u8
return rep_write(sd, reg, (rep_read(sd, reg) & ~mask) | val);
}
-static inline int edid_read(struct v4l2_subdev *sd, u8 reg)
+static inline int __always_unused edid_read(struct v4l2_subdev *sd, u8 reg)
{
struct adv76xx_state *state = to_state(sd);
return adv76xx_read_check(state, ADV76XX_PAGE_EDID, reg);
}
-static inline int edid_write(struct v4l2_subdev *sd, u8 reg, u8 val)
+static inline int __always_unused edid_write(struct v4l2_subdev *sd, u8 reg, u8 val)
{
struct adv76xx_state *state = to_state(sd);
@@ -570,7 +570,7 @@ static inline int hdmi_write_clr_set(struct v4l2_subdev *sd, u8 reg, u8 mask, u8
return hdmi_write(sd, reg, (hdmi_read(sd, reg) & ~mask) | val);
}
-static inline int test_write(struct v4l2_subdev *sd, u8 reg, u8 val)
+static inline int __always_unused test_write(struct v4l2_subdev *sd, u8 reg, u8 val)
{
struct adv76xx_state *state = to_state(sd);
@@ -601,14 +601,14 @@ static inline int cp_write_clr_set(struct v4l2_subdev *sd, u8 reg, u8 mask, u8 v
return cp_write(sd, reg, (cp_read(sd, reg) & ~mask) | val);
}
-static inline int vdp_read(struct v4l2_subdev *sd, u8 reg)
+static inline int __always_unused vdp_read(struct v4l2_subdev *sd, u8 reg)
{
struct adv76xx_state *state = to_state(sd);
return adv76xx_read_check(state, ADV7604_PAGE_VDP, reg);
}
-static inline int vdp_write(struct v4l2_subdev *sd, u8 reg, u8 val)
+static inline int __always_unused vdp_write(struct v4l2_subdev *sd, u8 reg, u8 val)
{
struct adv76xx_state *state = to_state(sd);
diff --git a/drivers/media/i2c/adv7842.c b/drivers/media/i2c/adv7842.c
index 7f8acbd..9d6eed0 100644
--- a/drivers/media/i2c/adv7842.c
+++ b/drivers/media/i2c/adv7842.c
@@ -256,21 +256,11 @@ static inline struct v4l2_subdev *to_sd(struct v4l2_ctrl *ctrl)
return &container_of(ctrl->handler, struct adv7842_state, hdl)->sd;
}
-static inline unsigned hblanking(const struct v4l2_bt_timings *t)
-{
- return V4L2_DV_BT_BLANKING_WIDTH(t);
-}
-
static inline unsigned htotal(const struct v4l2_bt_timings *t)
{
return V4L2_DV_BT_FRAME_WIDTH(t);
}
-static inline unsigned vblanking(const struct v4l2_bt_timings *t)
-{
- return V4L2_DV_BT_BLANKING_HEIGHT(t);
-}
-
static inline unsigned vtotal(const struct v4l2_bt_timings *t)
{
return V4L2_DV_BT_FRAME_HEIGHT(t);
diff --git a/drivers/media/i2c/ccs/ccs-core.c b/drivers/media/i2c/ccs/ccs-core.c
index 5363f3b..9158d3c 100644
--- a/drivers/media/i2c/ccs/ccs-core.c
+++ b/drivers/media/i2c/ccs/ccs-core.c
@@ -2758,13 +2758,13 @@ ident_show(struct device *dev, struct device_attribute *attr, char *buf)
struct ccs_module_info *minfo = &sensor->minfo;
if (minfo->mipi_manufacturer_id)
- return snprintf(buf, PAGE_SIZE, "%4.4x%4.4x%2.2x\n",
- minfo->mipi_manufacturer_id, minfo->model_id,
- minfo->revision_number) + 1;
+ return sysfs_emit(buf, "%4.4x%4.4x%2.2x\n",
+ minfo->mipi_manufacturer_id, minfo->model_id,
+ minfo->revision_number) + 1;
else
- return snprintf(buf, PAGE_SIZE, "%2.2x%4.4x%2.2x\n",
- minfo->smia_manufacturer_id, minfo->model_id,
- minfo->revision_number) + 1;
+ return sysfs_emit(buf, "%2.2x%4.4x%2.2x\n",
+ minfo->smia_manufacturer_id, minfo->model_id,
+ minfo->revision_number) + 1;
}
static DEVICE_ATTR_RO(ident);
diff --git a/drivers/media/i2c/cx25840/cx25840-ir.c b/drivers/media/i2c/cx25840/cx25840-ir.c
index 2cf3e6a..9d7d1d1 100644
--- a/drivers/media/i2c/cx25840/cx25840-ir.c
+++ b/drivers/media/i2c/cx25840/cx25840-ir.c
@@ -136,19 +136,6 @@ static inline u16 count_to_clock_divider(unsigned int d)
return (u16) d;
}
-static inline u16 ns_to_clock_divider(unsigned int ns)
-{
- return count_to_clock_divider(
- DIV_ROUND_CLOSEST(CX25840_IR_REFCLK_FREQ / 1000000 * ns, 1000));
-}
-
-static inline unsigned int clock_divider_to_ns(unsigned int divider)
-{
- /* Period of the Rx or Tx clock in ns */
- return DIV_ROUND_CLOSEST((divider + 1) * 1000,
- CX25840_IR_REFCLK_FREQ / 1000000);
-}
-
static inline u16 carrier_freq_to_clock_divider(unsigned int freq)
{
return count_to_clock_divider(
@@ -160,13 +147,6 @@ static inline unsigned int clock_divider_to_carrier_freq(unsigned int divider)
return DIV_ROUND_CLOSEST(CX25840_IR_REFCLK_FREQ, (divider + 1) * 16);
}
-static inline u16 freq_to_clock_divider(unsigned int freq,
- unsigned int rollovers)
-{
- return count_to_clock_divider(
- DIV_ROUND_CLOSEST(CX25840_IR_REFCLK_FREQ, freq * rollovers));
-}
-
static inline unsigned int clock_divider_to_freq(unsigned int divider,
unsigned int rollovers)
{
diff --git a/drivers/media/i2c/dw9768.c b/drivers/media/i2c/dw9768.c
index c086580..65c6acf3 100644
--- a/drivers/media/i2c/dw9768.c
+++ b/drivers/media/i2c/dw9768.c
@@ -469,6 +469,11 @@ static int dw9768_probe(struct i2c_client *client)
dw9768->sd.entity.function = MEDIA_ENT_F_LENS;
+ /*
+ * Device is already turned on by i2c-core with ACPI domain PM.
+ * Attempt to turn off the device to satisfy the privacy LED concerns.
+ */
+ pm_runtime_set_active(dev);
pm_runtime_enable(dev);
if (!pm_runtime_enabled(dev)) {
ret = dw9768_runtime_resume(dev);
@@ -483,6 +488,7 @@ static int dw9768_probe(struct i2c_client *client)
dev_err(dev, "failed to register V4L2 subdev: %d", ret);
goto err_power_off;
}
+ pm_runtime_idle(dev);
return 0;
diff --git a/drivers/media/i2c/hi556.c b/drivers/media/i2c/hi556.c
index 8db1cbe..055d1aa 100644
--- a/drivers/media/i2c/hi556.c
+++ b/drivers/media/i2c/hi556.c
@@ -495,6 +495,9 @@ struct hi556 {
/* Streaming on/off */
bool streaming;
+
+ /* True if the device has been identified */
+ bool identified;
};
static u64 to_pixel_rate(u32 f_index)
@@ -757,12 +760,41 @@ static void hi556_assign_pad_format(const struct hi556_mode *mode,
fmt->field = V4L2_FIELD_NONE;
}
+static int hi556_identify_module(struct hi556 *hi556)
+{
+ struct i2c_client *client = v4l2_get_subdevdata(&hi556->sd);
+ int ret;
+ u32 val;
+
+ if (hi556->identified)
+ return 0;
+
+ ret = hi556_read_reg(hi556, HI556_REG_CHIP_ID,
+ HI556_REG_VALUE_16BIT, &val);
+ if (ret)
+ return ret;
+
+ if (val != HI556_CHIP_ID) {
+ dev_err(&client->dev, "chip id mismatch: %x!=%x",
+ HI556_CHIP_ID, val);
+ return -ENXIO;
+ }
+
+ hi556->identified = true;
+
+ return 0;
+}
+
static int hi556_start_streaming(struct hi556 *hi556)
{
struct i2c_client *client = v4l2_get_subdevdata(&hi556->sd);
const struct hi556_reg_list *reg_list;
int link_freq_index, ret;
+ ret = hi556_identify_module(hi556);
+ if (ret)
+ return ret;
+
link_freq_index = hi556->cur_mode->link_freq_index;
reg_list = &link_freq_configs[link_freq_index].reg_list;
ret = hi556_write_reg_list(hi556, reg_list);
@@ -1001,26 +1033,6 @@ static const struct v4l2_subdev_internal_ops hi556_internal_ops = {
.open = hi556_open,
};
-static int hi556_identify_module(struct hi556 *hi556)
-{
- struct i2c_client *client = v4l2_get_subdevdata(&hi556->sd);
- int ret;
- u32 val;
-
- ret = hi556_read_reg(hi556, HI556_REG_CHIP_ID,
- HI556_REG_VALUE_16BIT, &val);
- if (ret)
- return ret;
-
- if (val != HI556_CHIP_ID) {
- dev_err(&client->dev, "chip id mismatch: %x!=%x",
- HI556_CHIP_ID, val);
- return -ENXIO;
- }
-
- return 0;
-}
-
static int hi556_check_hwcfg(struct device *dev)
{
struct fwnode_handle *ep;
@@ -1106,6 +1118,7 @@ static int hi556_remove(struct i2c_client *client)
static int hi556_probe(struct i2c_client *client)
{
struct hi556 *hi556;
+ bool full_power;
int ret;
ret = hi556_check_hwcfg(&client->dev);
@@ -1120,10 +1133,14 @@ static int hi556_probe(struct i2c_client *client)
return -ENOMEM;
v4l2_i2c_subdev_init(&hi556->sd, client, &hi556_subdev_ops);
- ret = hi556_identify_module(hi556);
- if (ret) {
- dev_err(&client->dev, "failed to find sensor: %d", ret);
- return ret;
+
+ full_power = acpi_dev_state_d0(&client->dev);
+ if (full_power) {
+ ret = hi556_identify_module(hi556);
+ if (ret) {
+ dev_err(&client->dev, "failed to find sensor: %d", ret);
+ return ret;
+ }
}
mutex_init(&hi556->mutex);
@@ -1152,7 +1169,9 @@ static int hi556_probe(struct i2c_client *client)
goto probe_error_media_entity_cleanup;
}
- pm_runtime_set_active(&client->dev);
+ /* Set the device's state to active if it's in D0 state. */
+ if (full_power)
+ pm_runtime_set_active(&client->dev);
pm_runtime_enable(&client->dev);
pm_runtime_idle(&client->dev);
@@ -1189,6 +1208,7 @@ static struct i2c_driver hi556_i2c_driver = {
},
.probe_new = hi556_probe,
.remove = hi556_remove,
+ .flags = I2C_DRV_ACPI_WAIVE_D0_PROBE,
};
module_i2c_driver(hi556_i2c_driver);
diff --git a/drivers/media/i2c/hi846.c b/drivers/media/i2c/hi846.c
index 48909fa..ad35c3f 100644
--- a/drivers/media/i2c/hi846.c
+++ b/drivers/media/i2c/hi846.c
@@ -1656,7 +1656,7 @@ static int hi846_power_on(struct hi846 *hi846)
return ret;
}
-static void hi846_power_off(struct hi846 *hi846)
+static int hi846_power_off(struct hi846 *hi846)
{
if (hi846->rst_gpio)
gpiod_set_value_cansleep(hi846->rst_gpio, 1);
@@ -1665,7 +1665,7 @@ static void hi846_power_off(struct hi846 *hi846)
gpiod_set_value_cansleep(hi846->shutdown_gpio, 1);
clk_disable_unprepare(hi846->clock);
- regulator_bulk_disable(HI846_NUM_SUPPLIES, hi846->supplies);
+ return regulator_bulk_disable(HI846_NUM_SUPPLIES, hi846->supplies);
}
static int __maybe_unused hi846_suspend(struct device *dev)
@@ -1677,9 +1677,7 @@ static int __maybe_unused hi846_suspend(struct device *dev)
if (hi846->streaming)
hi846_stop_streaming(hi846);
- hi846_power_off(hi846);
-
- return 0;
+ return hi846_power_off(hi846);
}
static int __maybe_unused hi846_resume(struct device *dev)
@@ -2164,7 +2162,11 @@ static int hi846_remove(struct i2c_client *client)
return 0;
}
-static UNIVERSAL_DEV_PM_OPS(hi846_pm_ops, hi846_suspend, hi846_resume, NULL);
+static const struct dev_pm_ops hi846_pm_ops = {
+ SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend,
+ pm_runtime_force_resume)
+ SET_RUNTIME_PM_OPS(hi846_suspend, hi846_resume, NULL)
+};
static const struct of_device_id hi846_of_match[] = {
{ .compatible = "hynix,hi846", },
diff --git a/drivers/media/i2c/imx208.c b/drivers/media/i2c/imx208.c
index 6f3d9c1..b9516b2 100644
--- a/drivers/media/i2c/imx208.c
+++ b/drivers/media/i2c/imx208.c
@@ -296,6 +296,9 @@ struct imx208 {
/* OTP data */
bool otp_read;
char otp_data[IMX208_OTP_SIZE];
+
+ /* True if the device has been identified */
+ bool identified;
};
static inline struct imx208 *to_imx208(struct v4l2_subdev *_sd)
@@ -619,6 +622,34 @@ static int imx208_set_pad_format(struct v4l2_subdev *sd,
return 0;
}
+static int imx208_identify_module(struct imx208 *imx208)
+{
+ struct i2c_client *client = v4l2_get_subdevdata(&imx208->sd);
+ int ret;
+ u32 val;
+
+ if (imx208->identified)
+ return 0;
+
+ ret = imx208_read_reg(imx208, IMX208_REG_CHIP_ID,
+ 2, &val);
+ if (ret) {
+ dev_err(&client->dev, "failed to read chip id %x\n",
+ IMX208_CHIP_ID);
+ return ret;
+ }
+
+ if (val != IMX208_CHIP_ID) {
+ dev_err(&client->dev, "chip id mismatch: %x!=%x\n",
+ IMX208_CHIP_ID, val);
+ return -EIO;
+ }
+
+ imx208->identified = true;
+
+ return 0;
+}
+
/* Start streaming */
static int imx208_start_streaming(struct imx208 *imx208)
{
@@ -626,6 +657,10 @@ static int imx208_start_streaming(struct imx208 *imx208)
const struct imx208_reg_list *reg_list;
int ret, link_freq_index;
+ ret = imx208_identify_module(imx208);
+ if (ret)
+ return ret;
+
/* Setup PLL */
link_freq_index = imx208->cur_mode->link_freq_index;
reg_list = &link_freq_configs[link_freq_index].reg_list;
@@ -752,29 +787,6 @@ static int __maybe_unused imx208_resume(struct device *dev)
}
/* Verify chip ID */
-static int imx208_identify_module(struct imx208 *imx208)
-{
- struct i2c_client *client = v4l2_get_subdevdata(&imx208->sd);
- int ret;
- u32 val;
-
- ret = imx208_read_reg(imx208, IMX208_REG_CHIP_ID,
- 2, &val);
- if (ret) {
- dev_err(&client->dev, "failed to read chip id %x\n",
- IMX208_CHIP_ID);
- return ret;
- }
-
- if (val != IMX208_CHIP_ID) {
- dev_err(&client->dev, "chip id mismatch: %x!=%x\n",
- IMX208_CHIP_ID, val);
- return -EIO;
- }
-
- return 0;
-}
-
static const struct v4l2_subdev_video_ops imx208_video_ops = {
.s_stream = imx208_set_stream,
};
@@ -813,6 +825,10 @@ static int imx208_read_otp(struct imx208 *imx208)
goto out_unlock;
}
+ ret = imx208_identify_module(imx208);
+ if (ret)
+ goto out_pm_put;
+
/* Write register address */
msgs[0].addr = client->addr;
msgs[0].flags = 0;
@@ -831,6 +847,7 @@ static int imx208_read_otp(struct imx208 *imx208)
ret = 0;
}
+out_pm_put:
pm_runtime_put(&client->dev);
out_unlock:
@@ -961,6 +978,7 @@ static int imx208_probe(struct i2c_client *client)
{
struct imx208 *imx208;
int ret;
+ bool full_power;
u32 val = 0;
device_property_read_u32(&client->dev, "clock-frequency", &val);
@@ -978,11 +996,14 @@ static int imx208_probe(struct i2c_client *client)
/* Initialize subdev */
v4l2_i2c_subdev_init(&imx208->sd, client, &imx208_subdev_ops);
- /* Check module identity */
- ret = imx208_identify_module(imx208);
- if (ret) {
- dev_err(&client->dev, "failed to find sensor: %d", ret);
- goto error_probe;
+ full_power = acpi_dev_state_d0(&client->dev);
+ if (full_power) {
+ /* Check module identity */
+ ret = imx208_identify_module(imx208);
+ if (ret) {
+ dev_err(&client->dev, "failed to find sensor: %d", ret);
+ goto error_probe;
+ }
}
/* Set default mode to max resolution */
@@ -1017,7 +1038,9 @@ static int imx208_probe(struct i2c_client *client)
goto error_async_subdev;
}
- pm_runtime_set_active(&client->dev);
+ /* Set the device's state to active if it's in D0 state. */
+ if (full_power)
+ pm_runtime_set_active(&client->dev);
pm_runtime_enable(&client->dev);
pm_runtime_idle(&client->dev);
@@ -1077,6 +1100,7 @@ static struct i2c_driver imx208_i2c_driver = {
},
.probe_new = imx208_probe,
.remove = imx208_remove,
+ .flags = I2C_DRV_ACPI_WAIVE_D0_PROBE,
};
module_i2c_driver(imx208_i2c_driver);
diff --git a/drivers/media/i2c/imx274.c b/drivers/media/i2c/imx274.c
index 0dce928..2aa15b9 100644
--- a/drivers/media/i2c/imx274.c
+++ b/drivers/media/i2c/imx274.c
@@ -27,6 +27,7 @@
#include <media/v4l2-ctrls.h>
#include <media/v4l2-device.h>
+#include <media/v4l2-fwnode.h>
#include <media/v4l2-subdev.h>
/*
@@ -1367,6 +1368,10 @@ static int imx274_s_frame_interval(struct v4l2_subdev *sd,
int min, max, def;
int ret;
+ ret = pm_runtime_resume_and_get(&imx274->client->dev);
+ if (ret < 0)
+ return ret;
+
mutex_lock(&imx274->lock);
ret = imx274_set_frame_interval(imx274, fi->interval);
@@ -1398,6 +1403,7 @@ static int imx274_s_frame_interval(struct v4l2_subdev *sd,
unlock:
mutex_unlock(&imx274->lock);
+ pm_runtime_put(&imx274->client->dev);
return ret;
}
@@ -1457,7 +1463,7 @@ static int imx274_s_stream(struct v4l2_subdev *sd, int on)
goto fail;
/*
- * update frame rate & expsoure. if the last mode is different,
+ * update frame rate & exposure. if the last mode is different,
* HMAX could be changed. As the result, frame rate & exposure
* are changed.
* gain is not affected.
@@ -1494,7 +1500,7 @@ static int imx274_s_stream(struct v4l2_subdev *sd, int on)
/*
* imx274_get_frame_length - Function for obtaining current frame length
* @priv: Pointer to device structure
- * @val: Pointer to obainted value
+ * @val: Pointer to obtained value
*
* frame_length = vmax x (svr + 1), in unit of hmax.
*
@@ -1904,7 +1910,21 @@ static int imx274_set_frame_interval(struct stimx274 *priv,
return err;
}
+static int imx274_enum_mbus_code(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *sd_state,
+ struct v4l2_subdev_mbus_code_enum *code)
+{
+ if (code->index > 0)
+ return -EINVAL;
+
+ /* only supported format in the driver is Raw 10 bits SRGGB */
+ code->code = MEDIA_BUS_FMT_SRGGB10_1X10;
+
+ return 0;
+}
+
static const struct v4l2_subdev_pad_ops imx274_pad_ops = {
+ .enum_mbus_code = imx274_enum_mbus_code,
.get_fmt = imx274_get_fmt,
.set_fmt = imx274_set_fmt,
.get_selection = imx274_get_selection,
@@ -1938,27 +1958,66 @@ static const struct i2c_device_id imx274_id[] = {
};
MODULE_DEVICE_TABLE(i2c, imx274_id);
+static int imx274_fwnode_parse(struct device *dev)
+{
+ struct fwnode_handle *endpoint;
+ /* Only CSI2 is supported */
+ struct v4l2_fwnode_endpoint ep = {
+ .bus_type = V4L2_MBUS_CSI2_DPHY
+ };
+ int ret;
+
+ endpoint = fwnode_graph_get_next_endpoint(dev_fwnode(dev), NULL);
+ if (!endpoint) {
+ dev_err(dev, "Endpoint node not found\n");
+ return -EINVAL;
+ }
+
+ ret = v4l2_fwnode_endpoint_parse(endpoint, &ep);
+ fwnode_handle_put(endpoint);
+ if (ret == -ENXIO) {
+ dev_err(dev, "Unsupported bus type, should be CSI2\n");
+ return ret;
+ } else if (ret) {
+ dev_err(dev, "Parsing endpoint node failed %d\n", ret);
+ return ret;
+ }
+
+ /* Check number of data lanes, only 4 lanes supported */
+ if (ep.bus.mipi_csi2.num_data_lanes != 4) {
+ dev_err(dev, "Invalid data lanes: %d\n",
+ ep.bus.mipi_csi2.num_data_lanes);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
static int imx274_probe(struct i2c_client *client)
{
struct v4l2_subdev *sd;
struct stimx274 *imx274;
+ struct device *dev = &client->dev;
int ret;
/* initialize imx274 */
- imx274 = devm_kzalloc(&client->dev, sizeof(*imx274), GFP_KERNEL);
+ imx274 = devm_kzalloc(dev, sizeof(*imx274), GFP_KERNEL);
if (!imx274)
return -ENOMEM;
mutex_init(&imx274->lock);
- imx274->inck = devm_clk_get_optional(&client->dev, "inck");
+ ret = imx274_fwnode_parse(dev);
+ if (ret)
+ return ret;
+
+ imx274->inck = devm_clk_get_optional(dev, "inck");
if (IS_ERR(imx274->inck))
return PTR_ERR(imx274->inck);
- ret = imx274_regulators_get(&client->dev, imx274);
+ ret = imx274_regulators_get(dev, imx274);
if (ret) {
- dev_err(&client->dev,
- "Failed to get power regulators, err: %d\n", ret);
+ dev_err(dev, "Failed to get power regulators, err: %d\n", ret);
return ret;
}
@@ -1977,7 +2036,7 @@ static int imx274_probe(struct i2c_client *client)
/* initialize regmap */
imx274->regmap = devm_regmap_init_i2c(client, &imx274_regmap_config);
if (IS_ERR(imx274->regmap)) {
- dev_err(&client->dev,
+ dev_err(dev,
"regmap init failed: %ld\n", PTR_ERR(imx274->regmap));
ret = -ENODEV;
goto err_regmap;
@@ -1994,34 +2053,32 @@ static int imx274_probe(struct i2c_client *client)
sd->entity.function = MEDIA_ENT_F_CAM_SENSOR;
ret = media_entity_pads_init(&sd->entity, 1, &imx274->pad);
if (ret < 0) {
- dev_err(&client->dev,
+ dev_err(dev,
"%s : media entity init Failed %d\n", __func__, ret);
goto err_regmap;
}
/* initialize sensor reset gpio */
- imx274->reset_gpio = devm_gpiod_get_optional(&client->dev, "reset",
+ imx274->reset_gpio = devm_gpiod_get_optional(dev, "reset",
GPIOD_OUT_HIGH);
if (IS_ERR(imx274->reset_gpio)) {
if (PTR_ERR(imx274->reset_gpio) != -EPROBE_DEFER)
- dev_err(&client->dev, "Reset GPIO not setup in DT");
+ dev_err(dev, "Reset GPIO not setup in DT");
ret = PTR_ERR(imx274->reset_gpio);
goto err_me;
}
/* power on the sensor */
- ret = imx274_power_on(&client->dev);
+ ret = imx274_power_on(dev);
if (ret < 0) {
- dev_err(&client->dev,
- "%s : imx274 power on failed\n", __func__);
+ dev_err(dev, "%s : imx274 power on failed\n", __func__);
goto err_me;
}
/* initialize controls */
ret = v4l2_ctrl_handler_init(&imx274->ctrls.handler, 4);
if (ret < 0) {
- dev_err(&client->dev,
- "%s : ctrl handler init Failed\n", __func__);
+ dev_err(dev, "%s : ctrl handler init Failed\n", __func__);
goto err_power_off;
}
@@ -2064,23 +2121,22 @@ static int imx274_probe(struct i2c_client *client)
/* register subdevice */
ret = v4l2_async_register_subdev(sd);
if (ret < 0) {
- dev_err(&client->dev,
- "%s : v4l2_async_register_subdev failed %d\n",
+ dev_err(dev, "%s : v4l2_async_register_subdev failed %d\n",
__func__, ret);
goto err_ctrls;
}
- pm_runtime_set_active(&client->dev);
- pm_runtime_enable(&client->dev);
- pm_runtime_idle(&client->dev);
+ pm_runtime_set_active(dev);
+ pm_runtime_enable(dev);
+ pm_runtime_idle(dev);
- dev_info(&client->dev, "imx274 : imx274 probe success !\n");
+ dev_info(dev, "imx274 : imx274 probe success !\n");
return 0;
err_ctrls:
v4l2_ctrl_handler_free(&imx274->ctrls.handler);
err_power_off:
- imx274_power_off(&client->dev);
+ imx274_power_off(dev);
err_me:
media_entity_cleanup(&sd->entity);
err_regmap:
diff --git a/drivers/media/i2c/imx290.c b/drivers/media/i2c/imx290.c
index bf7a6c3..99f2a50 100644
--- a/drivers/media/i2c/imx290.c
+++ b/drivers/media/i2c/imx290.c
@@ -363,7 +363,7 @@ static inline struct imx290 *to_imx290(struct v4l2_subdev *_sd)
return container_of(_sd, struct imx290, sd);
}
-static inline int imx290_read_reg(struct imx290 *imx290, u16 addr, u8 *value)
+static inline int __always_unused imx290_read_reg(struct imx290 *imx290, u16 addr, u8 *value)
{
unsigned int regval;
int ret;
diff --git a/drivers/media/i2c/imx319.c b/drivers/media/i2c/imx319.c
index daa9768..a2b5a34 100644
--- a/drivers/media/i2c/imx319.c
+++ b/drivers/media/i2c/imx319.c
@@ -2565,6 +2565,6 @@ module_i2c_driver(imx319_i2c_driver);
MODULE_AUTHOR("Qiu, Tianshu <tian.shu.qiu@intel.com>");
MODULE_AUTHOR("Rapolu, Chiranjeevi <chiranjeevi.rapolu@intel.com>");
MODULE_AUTHOR("Bingbu Cao <bingbu.cao@intel.com>");
-MODULE_AUTHOR("Yang, Hyungwoo <hyungwoo.yang@intel.com>");
+MODULE_AUTHOR("Yang, Hyungwoo");
MODULE_DESCRIPTION("Sony imx319 sensor driver");
MODULE_LICENSE("GPL v2");
diff --git a/drivers/media/i2c/imx355.c b/drivers/media/i2c/imx355.c
index cb51c81..3922b93 100644
--- a/drivers/media/i2c/imx355.c
+++ b/drivers/media/i2c/imx355.c
@@ -1851,6 +1851,6 @@ module_i2c_driver(imx355_i2c_driver);
MODULE_AUTHOR("Qiu, Tianshu <tian.shu.qiu@intel.com>");
MODULE_AUTHOR("Rapolu, Chiranjeevi <chiranjeevi.rapolu@intel.com>");
MODULE_AUTHOR("Bingbu Cao <bingbu.cao@intel.com>");
-MODULE_AUTHOR("Yang, Hyungwoo <hyungwoo.yang@intel.com>");
+MODULE_AUTHOR("Yang, Hyungwoo");
MODULE_DESCRIPTION("Sony imx355 sensor driver");
MODULE_LICENSE("GPL v2");
diff --git a/drivers/media/i2c/max9286.c b/drivers/media/i2c/max9286.c
index 7c663fd5..eb2b8e4 100644
--- a/drivers/media/i2c/max9286.c
+++ b/drivers/media/i2c/max9286.c
@@ -1055,7 +1055,6 @@ static int max9286_register_gpio(struct max9286_priv *priv)
gpio->label = dev_name(dev);
gpio->parent = dev;
gpio->owner = THIS_MODULE;
- gpio->of_node = dev->of_node;
gpio->ngpio = 2;
gpio->base = -1;
gpio->set = max9286_gpio_set;
@@ -1295,11 +1294,9 @@ static int max9286_probe(struct i2c_client *client)
priv->regulator = devm_regulator_get(&client->dev, "poc");
if (IS_ERR(priv->regulator)) {
- if (PTR_ERR(priv->regulator) != -EPROBE_DEFER)
- dev_err(&client->dev,
- "Unable to get PoC regulator (%ld)\n",
- PTR_ERR(priv->regulator));
ret = PTR_ERR(priv->regulator);
+ dev_err_probe(&client->dev, ret,
+ "Unable to get PoC regulator\n");
goto err_powerdown;
}
diff --git a/drivers/media/i2c/ov13858.c b/drivers/media/i2c/ov13858.c
index b4d22f5..d5fe67c 100644
--- a/drivers/media/i2c/ov13858.c
+++ b/drivers/media/i2c/ov13858.c
@@ -1818,6 +1818,6 @@ module_i2c_driver(ov13858_i2c_driver);
MODULE_AUTHOR("Kan, Chris <chris.kan@intel.com>");
MODULE_AUTHOR("Rapolu, Chiranjeevi <chiranjeevi.rapolu@intel.com>");
-MODULE_AUTHOR("Yang, Hyungwoo <hyungwoo.yang@intel.com>");
+MODULE_AUTHOR("Yang, Hyungwoo");
MODULE_DESCRIPTION("Omnivision ov13858 sensor driver");
MODULE_LICENSE("GPL v2");
diff --git a/drivers/media/i2c/ov2740.c b/drivers/media/i2c/ov2740.c
index 934c9d6..bab720c 100644
--- a/drivers/media/i2c/ov2740.c
+++ b/drivers/media/i2c/ov2740.c
@@ -345,6 +345,9 @@ struct ov2740 {
/* NVM data inforamtion */
struct nvm_data *nvm;
+
+ /* True if the device has been identified */
+ bool identified;
};
static inline struct ov2740 *to_ov2740(struct v4l2_subdev *subdev)
@@ -440,6 +443,30 @@ static int ov2740_write_reg_list(struct ov2740 *ov2740,
return 0;
}
+static int ov2740_identify_module(struct ov2740 *ov2740)
+{
+ struct i2c_client *client = v4l2_get_subdevdata(&ov2740->sd);
+ int ret;
+ u32 val;
+
+ if (ov2740->identified)
+ return 0;
+
+ ret = ov2740_read_reg(ov2740, OV2740_REG_CHIP_ID, 3, &val);
+ if (ret)
+ return ret;
+
+ if (val != OV2740_CHIP_ID) {
+ dev_err(&client->dev, "chip id mismatch: %x!=%x",
+ OV2740_CHIP_ID, val);
+ return -ENXIO;
+ }
+
+ ov2740->identified = true;
+
+ return 0;
+}
+
static int ov2740_update_digital_gain(struct ov2740 *ov2740, u32 d_gain)
{
int ret = 0;
@@ -724,6 +751,10 @@ static int ov2740_start_streaming(struct ov2740 *ov2740)
int link_freq_index;
int ret = 0;
+ ret = ov2740_identify_module(ov2740);
+ if (ret)
+ return ret;
+
ov2740_load_otp_data(nvm);
link_freq_index = ov2740->cur_mode->link_freq_index;
@@ -956,25 +987,6 @@ static const struct v4l2_subdev_internal_ops ov2740_internal_ops = {
.open = ov2740_open,
};
-static int ov2740_identify_module(struct ov2740 *ov2740)
-{
- struct i2c_client *client = v4l2_get_subdevdata(&ov2740->sd);
- int ret;
- u32 val;
-
- ret = ov2740_read_reg(ov2740, OV2740_REG_CHIP_ID, 3, &val);
- if (ret)
- return ret;
-
- if (val != OV2740_CHIP_ID) {
- dev_err(&client->dev, "chip id mismatch: %x!=%x",
- OV2740_CHIP_ID, val);
- return -ENXIO;
- }
-
- return 0;
-}
-
static int ov2740_check_hwcfg(struct device *dev)
{
struct fwnode_handle *ep;
@@ -1137,6 +1149,7 @@ static int ov2740_probe(struct i2c_client *client)
{
struct ov2740 *ov2740;
int ret = 0;
+ bool full_power;
ret = ov2740_check_hwcfg(&client->dev);
if (ret) {
@@ -1149,6 +1162,15 @@ static int ov2740_probe(struct i2c_client *client)
if (!ov2740)
return -ENOMEM;
+ full_power = acpi_dev_state_d0(&client->dev);
+ if (full_power) {
+ ret = ov2740_identify_module(ov2740);
+ if (ret) {
+ dev_err(&client->dev, "failed to find sensor: %d", ret);
+ return ret;
+ }
+ }
+
v4l2_i2c_subdev_init(&ov2740->sd, client, &ov2740_subdev_ops);
ret = ov2740_identify_module(ov2740);
if (ret) {
@@ -1186,11 +1208,9 @@ static int ov2740_probe(struct i2c_client *client)
if (ret)
dev_warn(&client->dev, "register nvmem failed, ret %d\n", ret);
- /*
- * Device is already turned on by i2c-core with ACPI domain PM.
- * Enable runtime PM and turn off the device.
- */
- pm_runtime_set_active(&client->dev);
+ /* Set the device's state to active if it's in D0 state. */
+ if (full_power)
+ pm_runtime_set_active(&client->dev);
pm_runtime_enable(&client->dev);
pm_runtime_idle(&client->dev);
@@ -1225,6 +1245,7 @@ static struct i2c_driver ov2740_i2c_driver = {
},
.probe_new = ov2740_probe,
.remove = ov2740_remove,
+ .flags = I2C_DRV_ACPI_WAIVE_D0_PROBE,
};
module_i2c_driver(ov2740_i2c_driver);
diff --git a/drivers/media/i2c/ov5670.c b/drivers/media/i2c/ov5670.c
index 251f459..02f75c1 100644
--- a/drivers/media/i2c/ov5670.c
+++ b/drivers/media/i2c/ov5670.c
@@ -1833,6 +1833,8 @@ struct ov5670 {
/* Streaming on/off */
bool streaming;
+ /* True if the device has been identified */
+ bool identified;
};
#define to_ov5670(_sd) container_of(_sd, struct ov5670, sd)
@@ -2273,6 +2275,32 @@ static int ov5670_get_skip_frames(struct v4l2_subdev *sd, u32 *frames)
return 0;
}
+/* Verify chip ID */
+static int ov5670_identify_module(struct ov5670 *ov5670)
+{
+ struct i2c_client *client = v4l2_get_subdevdata(&ov5670->sd);
+ int ret;
+ u32 val;
+
+ if (ov5670->identified)
+ return 0;
+
+ ret = ov5670_read_reg(ov5670, OV5670_REG_CHIP_ID,
+ OV5670_REG_VALUE_24BIT, &val);
+ if (ret)
+ return ret;
+
+ if (val != OV5670_CHIP_ID) {
+ dev_err(&client->dev, "chip id mismatch: %x!=%x\n",
+ OV5670_CHIP_ID, val);
+ return -ENXIO;
+ }
+
+ ov5670->identified = true;
+
+ return 0;
+}
+
/* Prepare streaming by writing default values and customized values */
static int ov5670_start_streaming(struct ov5670 *ov5670)
{
@@ -2281,6 +2309,10 @@ static int ov5670_start_streaming(struct ov5670 *ov5670)
int link_freq_index;
int ret;
+ ret = ov5670_identify_module(ov5670);
+ if (ret)
+ return ret;
+
/* Get out of from software reset */
ret = ov5670_write_reg(ov5670, OV5670_REG_SOFTWARE_RST,
OV5670_REG_VALUE_08BIT, OV5670_SOFTWARE_RST);
@@ -2400,27 +2432,6 @@ static int __maybe_unused ov5670_resume(struct device *dev)
return 0;
}
-/* Verify chip ID */
-static int ov5670_identify_module(struct ov5670 *ov5670)
-{
- struct i2c_client *client = v4l2_get_subdevdata(&ov5670->sd);
- int ret;
- u32 val;
-
- ret = ov5670_read_reg(ov5670, OV5670_REG_CHIP_ID,
- OV5670_REG_VALUE_24BIT, &val);
- if (ret)
- return ret;
-
- if (val != OV5670_CHIP_ID) {
- dev_err(&client->dev, "chip id mismatch: %x!=%x\n",
- OV5670_CHIP_ID, val);
- return -ENXIO;
- }
-
- return 0;
-}
-
static const struct v4l2_subdev_core_ops ov5670_core_ops = {
.log_status = v4l2_ctrl_subdev_log_status,
.subscribe_event = v4l2_ctrl_subdev_subscribe_event,
@@ -2462,6 +2473,7 @@ static int ov5670_probe(struct i2c_client *client)
struct ov5670 *ov5670;
const char *err_msg;
u32 input_clk = 0;
+ bool full_power;
int ret;
device_property_read_u32(&client->dev, "clock-frequency", &input_clk);
@@ -2478,11 +2490,14 @@ static int ov5670_probe(struct i2c_client *client)
/* Initialize subdev */
v4l2_i2c_subdev_init(&ov5670->sd, client, &ov5670_subdev_ops);
- /* Check module identity */
- ret = ov5670_identify_module(ov5670);
- if (ret) {
- err_msg = "ov5670_identify_module() error";
- goto error_print;
+ full_power = acpi_dev_state_d0(&client->dev);
+ if (full_power) {
+ /* Check module identity */
+ ret = ov5670_identify_module(ov5670);
+ if (ret) {
+ err_msg = "ov5670_identify_module() error";
+ goto error_print;
+ }
}
mutex_init(&ov5670->mutex);
@@ -2519,11 +2534,9 @@ static int ov5670_probe(struct i2c_client *client)
ov5670->streaming = false;
- /*
- * Device is already turned on by i2c-core with ACPI domain PM.
- * Enable runtime PM and turn off the device.
- */
- pm_runtime_set_active(&client->dev);
+ /* Set the device's state to active if it's in D0 state. */
+ if (full_power)
+ pm_runtime_set_active(&client->dev);
pm_runtime_enable(&client->dev);
pm_runtime_idle(&client->dev);
@@ -2565,7 +2578,7 @@ static const struct dev_pm_ops ov5670_pm_ops = {
#ifdef CONFIG_ACPI
static const struct acpi_device_id ov5670_acpi_ids[] = {
- {"INT3479"},
+ { "INT3479" },
{ /* sentinel */ }
};
@@ -2580,11 +2593,12 @@ static struct i2c_driver ov5670_i2c_driver = {
},
.probe_new = ov5670_probe,
.remove = ov5670_remove,
+ .flags = I2C_DRV_ACPI_WAIVE_D0_PROBE,
};
module_i2c_driver(ov5670_i2c_driver);
MODULE_AUTHOR("Rapolu, Chiranjeevi <chiranjeevi.rapolu@intel.com>");
-MODULE_AUTHOR("Yang, Hyungwoo <hyungwoo.yang@intel.com>");
+MODULE_AUTHOR("Yang, Hyungwoo");
MODULE_DESCRIPTION("Omnivision ov5670 sensor driver");
MODULE_LICENSE("GPL v2");
diff --git a/drivers/media/i2c/ov5675.c b/drivers/media/i2c/ov5675.c
index da5850b..0092585 100644
--- a/drivers/media/i2c/ov5675.c
+++ b/drivers/media/i2c/ov5675.c
@@ -493,6 +493,9 @@ struct ov5675 {
/* Streaming on/off */
bool streaming;
+
+ /* True if the device has been identified */
+ bool identified;
};
static u64 to_pixel_rate(u32 f_index)
@@ -808,12 +811,41 @@ static void ov5675_update_pad_format(const struct ov5675_mode *mode,
fmt->field = V4L2_FIELD_NONE;
}
+static int ov5675_identify_module(struct ov5675 *ov5675)
+{
+ struct i2c_client *client = v4l2_get_subdevdata(&ov5675->sd);
+ int ret;
+ u32 val;
+
+ if (ov5675->identified)
+ return 0;
+
+ ret = ov5675_read_reg(ov5675, OV5675_REG_CHIP_ID,
+ OV5675_REG_VALUE_24BIT, &val);
+ if (ret)
+ return ret;
+
+ if (val != OV5675_CHIP_ID) {
+ dev_err(&client->dev, "chip id mismatch: %x!=%x",
+ OV5675_CHIP_ID, val);
+ return -ENXIO;
+ }
+
+ ov5675->identified = true;
+
+ return 0;
+}
+
static int ov5675_start_streaming(struct ov5675 *ov5675)
{
struct i2c_client *client = v4l2_get_subdevdata(&ov5675->sd);
const struct ov5675_reg_list *reg_list;
int link_freq_index, ret;
+ ret = ov5675_identify_module(ov5675);
+ if (ret)
+ return ret;
+
link_freq_index = ov5675->cur_mode->link_freq_index;
reg_list = &link_freq_configs[link_freq_index].reg_list;
ret = ov5675_write_reg_list(ov5675, reg_list);
@@ -1048,26 +1080,6 @@ static const struct v4l2_subdev_internal_ops ov5675_internal_ops = {
.open = ov5675_open,
};
-static int ov5675_identify_module(struct ov5675 *ov5675)
-{
- struct i2c_client *client = v4l2_get_subdevdata(&ov5675->sd);
- int ret;
- u32 val;
-
- ret = ov5675_read_reg(ov5675, OV5675_REG_CHIP_ID,
- OV5675_REG_VALUE_24BIT, &val);
- if (ret)
- return ret;
-
- if (val != OV5675_CHIP_ID) {
- dev_err(&client->dev, "chip id mismatch: %x!=%x",
- OV5675_CHIP_ID, val);
- return -ENXIO;
- }
-
- return 0;
-}
-
static int ov5675_check_hwcfg(struct device *dev)
{
struct fwnode_handle *ep;
@@ -1154,6 +1166,7 @@ static int ov5675_remove(struct i2c_client *client)
static int ov5675_probe(struct i2c_client *client)
{
struct ov5675 *ov5675;
+ bool full_power;
int ret;
ret = ov5675_check_hwcfg(&client->dev);
@@ -1168,10 +1181,14 @@ static int ov5675_probe(struct i2c_client *client)
return -ENOMEM;
v4l2_i2c_subdev_init(&ov5675->sd, client, &ov5675_subdev_ops);
- ret = ov5675_identify_module(ov5675);
- if (ret) {
- dev_err(&client->dev, "failed to find sensor: %d", ret);
- return ret;
+
+ full_power = acpi_dev_state_d0(&client->dev);
+ if (full_power) {
+ ret = ov5675_identify_module(ov5675);
+ if (ret) {
+ dev_err(&client->dev, "failed to find sensor: %d", ret);
+ return ret;
+ }
}
mutex_init(&ov5675->mutex);
@@ -1204,7 +1221,10 @@ static int ov5675_probe(struct i2c_client *client)
* Device is already turned on by i2c-core with ACPI domain PM.
* Enable runtime PM and turn off the device.
*/
- pm_runtime_set_active(&client->dev);
+
+ /* Set the device's state to active if it's in D0 state. */
+ if (full_power)
+ pm_runtime_set_active(&client->dev);
pm_runtime_enable(&client->dev);
pm_runtime_idle(&client->dev);
@@ -1241,6 +1261,7 @@ static struct i2c_driver ov5675_i2c_driver = {
},
.probe_new = ov5675_probe,
.remove = ov5675_remove,
+ .flags = I2C_DRV_ACPI_WAIVE_D0_PROBE,
};
module_i2c_driver(ov5675_i2c_driver);
diff --git a/drivers/media/i2c/ov5693.c b/drivers/media/i2c/ov5693.c
new file mode 100644
index 0000000..2784fcf
--- /dev/null
+++ b/drivers/media/i2c/ov5693.c
@@ -0,0 +1,1537 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2013 Intel Corporation. All Rights Reserved.
+ *
+ * Adapted from the atomisp-ov5693 driver, with contributions from:
+ *
+ * Daniel Scally
+ * Jean-Michel Hautbois
+ * Fabian Wuthrich
+ * Tsuchiya Yuto
+ * Jordan Hand
+ * Jake Day
+ */
+
+#include <asm/unaligned.h>
+#include <linux/acpi.h>
+#include <linux/clk.h>
+#include <linux/delay.h>
+#include <linux/device.h>
+#include <linux/i2c.h>
+#include <linux/module.h>
+#include <linux/pm_runtime.h>
+#include <linux/regulator/consumer.h>
+#include <linux/slab.h>
+#include <linux/types.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-fwnode.h>
+
+#define OV5693_REG_8BIT(n) ((1 << 16) | (n))
+#define OV5693_REG_16BIT(n) ((2 << 16) | (n))
+#define OV5693_REG_24BIT(n) ((3 << 16) | (n))
+#define OV5693_REG_SIZE_SHIFT 16
+#define OV5693_REG_ADDR_MASK 0xffff
+
+/* System Control */
+#define OV5693_SW_RESET_REG OV5693_REG_8BIT(0x0103)
+#define OV5693_SW_STREAM_REG OV5693_REG_8BIT(0x0100)
+#define OV5693_START_STREAMING 0x01
+#define OV5693_STOP_STREAMING 0x00
+#define OV5693_SW_RESET 0x01
+
+#define OV5693_REG_CHIP_ID OV5693_REG_16BIT(0x300a)
+/* Yes, this is right. The datasheet for the OV5693 gives its ID as 0x5690 */
+#define OV5693_CHIP_ID 0x5690
+
+/* Exposure */
+#define OV5693_EXPOSURE_CTRL_REG OV5693_REG_24BIT(0x3500)
+#define OV5693_EXPOSURE_CTRL_MASK GENMASK(19, 4)
+#define OV5693_INTEGRATION_TIME_MARGIN 8
+#define OV5693_EXPOSURE_MIN 1
+#define OV5693_EXPOSURE_STEP 1
+
+/* Analogue Gain */
+#define OV5693_GAIN_CTRL_REG OV5693_REG_16BIT(0x350a)
+#define OV5693_GAIN_CTRL_MASK GENMASK(10, 4)
+#define OV5693_GAIN_MIN 1
+#define OV5693_GAIN_MAX 127
+#define OV5693_GAIN_DEF 8
+#define OV5693_GAIN_STEP 1
+
+/* Digital Gain */
+#define OV5693_MWB_RED_GAIN_REG OV5693_REG_16BIT(0x3400)
+#define OV5693_MWB_GREEN_GAIN_REG OV5693_REG_16BIT(0x3402)
+#define OV5693_MWB_BLUE_GAIN_REG OV5693_REG_16BIT(0x3404)
+#define OV5693_MWB_GAIN_MASK GENMASK(11, 0)
+#define OV5693_MWB_GAIN_MAX 0x0fff
+#define OV5693_DIGITAL_GAIN_MIN 1
+#define OV5693_DIGITAL_GAIN_MAX 4095
+#define OV5693_DIGITAL_GAIN_DEF 1024
+#define OV5693_DIGITAL_GAIN_STEP 1
+
+/* Timing and Format */
+#define OV5693_CROP_START_X_REG OV5693_REG_16BIT(0x3800)
+#define OV5693_CROP_START_Y_REG OV5693_REG_16BIT(0x3802)
+#define OV5693_CROP_END_X_REG OV5693_REG_16BIT(0x3804)
+#define OV5693_CROP_END_Y_REG OV5693_REG_16BIT(0x3806)
+#define OV5693_OUTPUT_SIZE_X_REG OV5693_REG_16BIT(0x3808)
+#define OV5693_OUTPUT_SIZE_Y_REG OV5693_REG_16BIT(0x380a)
+
+#define OV5693_TIMING_HTS_REG OV5693_REG_16BIT(0x380c)
+#define OV5693_FIXED_PPL 2688U
+#define OV5693_TIMING_VTS_REG OV5693_REG_16BIT(0x380e)
+#define OV5693_TIMING_MAX_VTS 0xffff
+#define OV5693_TIMING_MIN_VTS 0x04
+
+#define OV5693_OFFSET_START_X_REG OV5693_REG_16BIT(0x3810)
+#define OV5693_OFFSET_START_Y_REG OV5693_REG_16BIT(0x3812)
+
+#define OV5693_SUB_INC_X_REG OV5693_REG_8BIT(0x3814)
+#define OV5693_SUB_INC_Y_REG OV5693_REG_8BIT(0x3815)
+
+#define OV5693_FORMAT1_REG OV5693_REG_8BIT(0x3820)
+#define OV5693_FORMAT1_FLIP_VERT_ISP_EN BIT(6)
+#define OV5693_FORMAT1_FLIP_VERT_SENSOR_EN BIT(1)
+#define OV5693_FORMAT1_VBIN_EN BIT(0)
+#define OV5693_FORMAT2_REG OV5693_REG_8BIT(0x3821)
+#define OV5693_FORMAT2_HDR_EN BIT(7)
+#define OV5693_FORMAT2_FLIP_HORZ_ISP_EN BIT(2)
+#define OV5693_FORMAT2_FLIP_HORZ_SENSOR_EN BIT(1)
+#define OV5693_FORMAT2_HBIN_EN BIT(0)
+
+#define OV5693_ISP_CTRL2_REG OV5693_REG_8BIT(0x5002)
+#define OV5693_ISP_SCALE_ENABLE BIT(7)
+
+/* Pixel Array */
+#define OV5693_NATIVE_WIDTH 2624
+#define OV5693_NATIVE_HEIGHT 1956
+#define OV5693_NATIVE_START_LEFT 0
+#define OV5693_NATIVE_START_TOP 0
+#define OV5693_ACTIVE_WIDTH 2592
+#define OV5693_ACTIVE_HEIGHT 1944
+#define OV5693_ACTIVE_START_LEFT 16
+#define OV5693_ACTIVE_START_TOP 6
+#define OV5693_MIN_CROP_WIDTH 2
+#define OV5693_MIN_CROP_HEIGHT 2
+
+/* Test Pattern */
+#define OV5693_TEST_PATTERN_REG OV5693_REG_8BIT(0x5e00)
+#define OV5693_TEST_PATTERN_ENABLE BIT(7)
+#define OV5693_TEST_PATTERN_ROLLING BIT(6)
+#define OV5693_TEST_PATTERN_RANDOM 0x01
+#define OV5693_TEST_PATTERN_BARS 0x00
+
+/* System Frequencies */
+#define OV5693_XVCLK_FREQ 19200000
+#define OV5693_LINK_FREQ_419_2MHZ 419200000
+#define OV5693_PIXEL_RATE 167680000
+
+/* Miscellaneous */
+#define OV5693_NUM_SUPPLIES 2
+
+#define to_ov5693_sensor(x) container_of(x, struct ov5693_device, sd)
+
+struct ov5693_reg {
+ u32 reg;
+ u8 val;
+};
+
+struct ov5693_reg_list {
+ u32 num_regs;
+ const struct ov5693_reg *regs;
+};
+
+struct ov5693_device {
+ struct i2c_client *client;
+ struct device *dev;
+
+ /* Protect against concurrent changes to controls */
+ struct mutex lock;
+
+ struct gpio_desc *reset;
+ struct gpio_desc *powerdown;
+ struct regulator_bulk_data supplies[OV5693_NUM_SUPPLIES];
+ struct clk *clk;
+
+ struct ov5693_mode {
+ struct v4l2_rect crop;
+ struct v4l2_mbus_framefmt format;
+ bool binning_x;
+ bool binning_y;
+ unsigned int inc_x_odd;
+ unsigned int inc_y_odd;
+ unsigned int vts;
+ } mode;
+ bool streaming;
+
+ struct v4l2_subdev sd;
+ struct media_pad pad;
+
+ struct ov5693_v4l2_ctrls {
+ struct v4l2_ctrl_handler handler;
+ struct v4l2_ctrl *link_freq;
+ struct v4l2_ctrl *pixel_rate;
+ struct v4l2_ctrl *exposure;
+ struct v4l2_ctrl *analogue_gain;
+ struct v4l2_ctrl *digital_gain;
+ struct v4l2_ctrl *hflip;
+ struct v4l2_ctrl *vflip;
+ struct v4l2_ctrl *hblank;
+ struct v4l2_ctrl *vblank;
+ struct v4l2_ctrl *test_pattern;
+ } ctrls;
+};
+
+static const struct ov5693_reg ov5693_global_regs[] = {
+ {OV5693_REG_8BIT(0x3016), 0xf0},
+ {OV5693_REG_8BIT(0x3017), 0xf0},
+ {OV5693_REG_8BIT(0x3018), 0xf0},
+ {OV5693_REG_8BIT(0x3022), 0x01},
+ {OV5693_REG_8BIT(0x3028), 0x44},
+ {OV5693_REG_8BIT(0x3098), 0x02},
+ {OV5693_REG_8BIT(0x3099), 0x19},
+ {OV5693_REG_8BIT(0x309a), 0x02},
+ {OV5693_REG_8BIT(0x309b), 0x01},
+ {OV5693_REG_8BIT(0x309c), 0x00},
+ {OV5693_REG_8BIT(0x30a0), 0xd2},
+ {OV5693_REG_8BIT(0x30a2), 0x01},
+ {OV5693_REG_8BIT(0x30b2), 0x00},
+ {OV5693_REG_8BIT(0x30b3), 0x83},
+ {OV5693_REG_8BIT(0x30b4), 0x03},
+ {OV5693_REG_8BIT(0x30b5), 0x04},
+ {OV5693_REG_8BIT(0x30b6), 0x01},
+ {OV5693_REG_8BIT(0x3080), 0x01},
+ {OV5693_REG_8BIT(0x3104), 0x21},
+ {OV5693_REG_8BIT(0x3106), 0x00},
+ {OV5693_REG_8BIT(0x3406), 0x01},
+ {OV5693_REG_8BIT(0x3503), 0x07},
+ {OV5693_REG_8BIT(0x350b), 0x40},
+ {OV5693_REG_8BIT(0x3601), 0x0a},
+ {OV5693_REG_8BIT(0x3602), 0x38},
+ {OV5693_REG_8BIT(0x3612), 0x80},
+ {OV5693_REG_8BIT(0x3620), 0x54},
+ {OV5693_REG_8BIT(0x3621), 0xc7},
+ {OV5693_REG_8BIT(0x3622), 0x0f},
+ {OV5693_REG_8BIT(0x3625), 0x10},
+ {OV5693_REG_8BIT(0x3630), 0x55},
+ {OV5693_REG_8BIT(0x3631), 0xf4},
+ {OV5693_REG_8BIT(0x3632), 0x00},
+ {OV5693_REG_8BIT(0x3633), 0x34},
+ {OV5693_REG_8BIT(0x3634), 0x02},
+ {OV5693_REG_8BIT(0x364d), 0x0d},
+ {OV5693_REG_8BIT(0x364f), 0xdd},
+ {OV5693_REG_8BIT(0x3660), 0x04},
+ {OV5693_REG_8BIT(0x3662), 0x10},
+ {OV5693_REG_8BIT(0x3663), 0xf1},
+ {OV5693_REG_8BIT(0x3665), 0x00},
+ {OV5693_REG_8BIT(0x3666), 0x20},
+ {OV5693_REG_8BIT(0x3667), 0x00},
+ {OV5693_REG_8BIT(0x366a), 0x80},
+ {OV5693_REG_8BIT(0x3680), 0xe0},
+ {OV5693_REG_8BIT(0x3681), 0x00},
+ {OV5693_REG_8BIT(0x3700), 0x42},
+ {OV5693_REG_8BIT(0x3701), 0x14},
+ {OV5693_REG_8BIT(0x3702), 0xa0},
+ {OV5693_REG_8BIT(0x3703), 0xd8},
+ {OV5693_REG_8BIT(0x3704), 0x78},
+ {OV5693_REG_8BIT(0x3705), 0x02},
+ {OV5693_REG_8BIT(0x370a), 0x00},
+ {OV5693_REG_8BIT(0x370b), 0x20},
+ {OV5693_REG_8BIT(0x370c), 0x0c},
+ {OV5693_REG_8BIT(0x370d), 0x11},
+ {OV5693_REG_8BIT(0x370e), 0x00},
+ {OV5693_REG_8BIT(0x370f), 0x40},
+ {OV5693_REG_8BIT(0x3710), 0x00},
+ {OV5693_REG_8BIT(0x371a), 0x1c},
+ {OV5693_REG_8BIT(0x371b), 0x05},
+ {OV5693_REG_8BIT(0x371c), 0x01},
+ {OV5693_REG_8BIT(0x371e), 0xa1},
+ {OV5693_REG_8BIT(0x371f), 0x0c},
+ {OV5693_REG_8BIT(0x3721), 0x00},
+ {OV5693_REG_8BIT(0x3724), 0x10},
+ {OV5693_REG_8BIT(0x3726), 0x00},
+ {OV5693_REG_8BIT(0x372a), 0x01},
+ {OV5693_REG_8BIT(0x3730), 0x10},
+ {OV5693_REG_8BIT(0x3738), 0x22},
+ {OV5693_REG_8BIT(0x3739), 0xe5},
+ {OV5693_REG_8BIT(0x373a), 0x50},
+ {OV5693_REG_8BIT(0x373b), 0x02},
+ {OV5693_REG_8BIT(0x373c), 0x41},
+ {OV5693_REG_8BIT(0x373f), 0x02},
+ {OV5693_REG_8BIT(0x3740), 0x42},
+ {OV5693_REG_8BIT(0x3741), 0x02},
+ {OV5693_REG_8BIT(0x3742), 0x18},
+ {OV5693_REG_8BIT(0x3743), 0x01},
+ {OV5693_REG_8BIT(0x3744), 0x02},
+ {OV5693_REG_8BIT(0x3747), 0x10},
+ {OV5693_REG_8BIT(0x374c), 0x04},
+ {OV5693_REG_8BIT(0x3751), 0xf0},
+ {OV5693_REG_8BIT(0x3752), 0x00},
+ {OV5693_REG_8BIT(0x3753), 0x00},
+ {OV5693_REG_8BIT(0x3754), 0xc0},
+ {OV5693_REG_8BIT(0x3755), 0x00},
+ {OV5693_REG_8BIT(0x3756), 0x1a},
+ {OV5693_REG_8BIT(0x3758), 0x00},
+ {OV5693_REG_8BIT(0x3759), 0x0f},
+ {OV5693_REG_8BIT(0x376b), 0x44},
+ {OV5693_REG_8BIT(0x375c), 0x04},
+ {OV5693_REG_8BIT(0x3774), 0x10},
+ {OV5693_REG_8BIT(0x3776), 0x00},
+ {OV5693_REG_8BIT(0x377f), 0x08},
+ {OV5693_REG_8BIT(0x3780), 0x22},
+ {OV5693_REG_8BIT(0x3781), 0x0c},
+ {OV5693_REG_8BIT(0x3784), 0x2c},
+ {OV5693_REG_8BIT(0x3785), 0x1e},
+ {OV5693_REG_8BIT(0x378f), 0xf5},
+ {OV5693_REG_8BIT(0x3791), 0xb0},
+ {OV5693_REG_8BIT(0x3795), 0x00},
+ {OV5693_REG_8BIT(0x3796), 0x64},
+ {OV5693_REG_8BIT(0x3797), 0x11},
+ {OV5693_REG_8BIT(0x3798), 0x30},
+ {OV5693_REG_8BIT(0x3799), 0x41},
+ {OV5693_REG_8BIT(0x379a), 0x07},
+ {OV5693_REG_8BIT(0x379b), 0xb0},
+ {OV5693_REG_8BIT(0x379c), 0x0c},
+ {OV5693_REG_8BIT(0x3a04), 0x06},
+ {OV5693_REG_8BIT(0x3a05), 0x14},
+ {OV5693_REG_8BIT(0x3e07), 0x20},
+ {OV5693_REG_8BIT(0x4000), 0x08},
+ {OV5693_REG_8BIT(0x4001), 0x04},
+ {OV5693_REG_8BIT(0x4004), 0x08},
+ {OV5693_REG_8BIT(0x4006), 0x20},
+ {OV5693_REG_8BIT(0x4008), 0x24},
+ {OV5693_REG_8BIT(0x4009), 0x10},
+ {OV5693_REG_8BIT(0x4058), 0x00},
+ {OV5693_REG_8BIT(0x4101), 0xb2},
+ {OV5693_REG_8BIT(0x4307), 0x31},
+ {OV5693_REG_8BIT(0x4511), 0x05},
+ {OV5693_REG_8BIT(0x4512), 0x01},
+ {OV5693_REG_8BIT(0x481f), 0x30},
+ {OV5693_REG_8BIT(0x4826), 0x2c},
+ {OV5693_REG_8BIT(0x4d02), 0xfd},
+ {OV5693_REG_8BIT(0x4d03), 0xf5},
+ {OV5693_REG_8BIT(0x4d04), 0x0c},
+ {OV5693_REG_8BIT(0x4d05), 0xcc},
+ {OV5693_REG_8BIT(0x4837), 0x0a},
+ {OV5693_REG_8BIT(0x5003), 0x20},
+ {OV5693_REG_8BIT(0x5013), 0x00},
+ {OV5693_REG_8BIT(0x5842), 0x01},
+ {OV5693_REG_8BIT(0x5843), 0x2b},
+ {OV5693_REG_8BIT(0x5844), 0x01},
+ {OV5693_REG_8BIT(0x5845), 0x92},
+ {OV5693_REG_8BIT(0x5846), 0x01},
+ {OV5693_REG_8BIT(0x5847), 0x8f},
+ {OV5693_REG_8BIT(0x5848), 0x01},
+ {OV5693_REG_8BIT(0x5849), 0x0c},
+ {OV5693_REG_8BIT(0x5e10), 0x0c},
+ {OV5693_REG_8BIT(0x3820), 0x00},
+ {OV5693_REG_8BIT(0x3821), 0x1e},
+ {OV5693_REG_8BIT(0x5041), 0x14}
+};
+
+static const struct ov5693_reg_list ov5693_global_setting = {
+ .num_regs = ARRAY_SIZE(ov5693_global_regs),
+ .regs = ov5693_global_regs,
+};
+
+static const struct v4l2_rect ov5693_default_crop = {
+ .left = OV5693_ACTIVE_START_LEFT,
+ .top = OV5693_ACTIVE_START_TOP,
+ .width = OV5693_ACTIVE_WIDTH,
+ .height = OV5693_ACTIVE_HEIGHT,
+};
+
+static const struct v4l2_mbus_framefmt ov5693_default_fmt = {
+ .width = OV5693_ACTIVE_WIDTH,
+ .height = OV5693_ACTIVE_HEIGHT,
+ .code = MEDIA_BUS_FMT_SBGGR10_1X10,
+};
+
+static const s64 link_freq_menu_items[] = {
+ OV5693_LINK_FREQ_419_2MHZ
+};
+
+static const char * const ov5693_supply_names[] = {
+ "avdd",
+ "dovdd",
+};
+
+static const char * const ov5693_test_pattern_menu[] = {
+ "Disabled",
+ "Random Data",
+ "Colour Bars",
+ "Colour Bars with Rolling Bar"
+};
+
+static const u8 ov5693_test_pattern_bits[] = {
+ 0,
+ OV5693_TEST_PATTERN_ENABLE | OV5693_TEST_PATTERN_RANDOM,
+ OV5693_TEST_PATTERN_ENABLE | OV5693_TEST_PATTERN_BARS,
+ OV5693_TEST_PATTERN_ENABLE | OV5693_TEST_PATTERN_BARS |
+ OV5693_TEST_PATTERN_ROLLING,
+};
+
+/* I2C I/O Operations */
+
+static int ov5693_read_reg(struct ov5693_device *ov5693, u32 addr, u32 *value)
+{
+ struct i2c_client *client = ov5693->client;
+ __be16 reg;
+ u8 val[4];
+ struct i2c_msg msg[] = {
+ {
+ .addr = client->addr,
+ .flags = 0,
+ .len = 2,
+ .buf = (u8 *)®,
+ },
+ {
+ .addr = client->addr,
+ .flags = I2C_M_RD,
+ .buf = (u8 *)&val,
+ },
+ };
+ unsigned int len = ((addr >> OV5693_REG_SIZE_SHIFT) & 3);
+ unsigned int i;
+ int ret;
+
+ reg = cpu_to_be16(addr & OV5693_REG_ADDR_MASK);
+
+ msg[1].len = len;
+
+ ret = i2c_transfer(client->adapter, msg, 2);
+ if (ret < 0)
+ return dev_err_probe(&client->dev, ret,
+ "Failed to read register 0x%04x: %d\n",
+ addr & OV5693_REG_ADDR_MASK, ret);
+
+ *value = 0;
+ for (i = 0; i < len; ++i) {
+ *value <<= 8;
+ *value |= val[i];
+ }
+
+ return 0;
+}
+
+static void ov5693_write_reg(struct ov5693_device *ov5693, u32 addr, u32 value,
+ int *error)
+{
+ struct i2c_client *client = ov5693->client;
+ struct {
+ __be16 reg;
+ u8 val[4];
+ } __packed buf;
+ struct i2c_msg msg = {
+ .addr = client->addr,
+ .buf = (u8 *)&buf,
+ };
+ unsigned int len = ((addr >> OV5693_REG_SIZE_SHIFT) & 3);
+ unsigned int i;
+ int ret;
+
+ if (*error < 0)
+ return;
+
+ buf.reg = cpu_to_be16(addr & OV5693_REG_ADDR_MASK);
+ for (i = 0; i < len; ++i) {
+ buf.val[len - i - 1] = value & 0xff;
+ value >>= 8;
+ }
+
+ msg.len = len + 2;
+
+ ret = i2c_transfer(client->adapter, &msg, 1);
+ if (ret < 0) {
+ dev_err(&client->dev, "Failed to write register 0x%04x: %d\n",
+ addr & OV5693_REG_ADDR_MASK, ret);
+ *error = ret;
+ }
+}
+
+static int ov5693_write_reg_array(struct ov5693_device *ov5693,
+ const struct ov5693_reg_list *reglist)
+{
+ unsigned int i;
+ int ret = 0;
+
+ for (i = 0; i < reglist->num_regs; i++)
+ ov5693_write_reg(ov5693, reglist->regs[i].reg,
+ reglist->regs[i].val, &ret);
+
+ return ret;
+}
+
+static int ov5693_update_bits(struct ov5693_device *ov5693, u32 address,
+ u32 mask, u32 bits)
+{
+ u32 value = 0;
+ int ret;
+
+ ret = ov5693_read_reg(ov5693, address, &value);
+ if (ret)
+ return ret;
+
+ value &= ~mask;
+ value |= bits;
+
+ ov5693_write_reg(ov5693, address, value, &ret);
+
+ return ret;
+}
+
+/* V4L2 Controls Functions */
+
+static int ov5693_flip_vert_configure(struct ov5693_device *ov5693,
+ bool enable)
+{
+ u8 bits = OV5693_FORMAT1_FLIP_VERT_ISP_EN |
+ OV5693_FORMAT1_FLIP_VERT_SENSOR_EN;
+ int ret;
+
+ ret = ov5693_update_bits(ov5693, OV5693_FORMAT1_REG, bits,
+ enable ? bits : 0);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+static int ov5693_flip_horz_configure(struct ov5693_device *ov5693,
+ bool enable)
+{
+ u8 bits = OV5693_FORMAT2_FLIP_HORZ_ISP_EN |
+ OV5693_FORMAT2_FLIP_HORZ_SENSOR_EN;
+ int ret;
+
+ ret = ov5693_update_bits(ov5693, OV5693_FORMAT2_REG, bits,
+ enable ? bits : 0);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+static int ov5693_get_exposure(struct ov5693_device *ov5693, s32 *value)
+{
+ u32 exposure;
+ int ret;
+
+ ret = ov5693_read_reg(ov5693, OV5693_EXPOSURE_CTRL_REG, &exposure);
+ if (ret)
+ return ret;
+
+ /* The lowest 4 bits are unsupported fractional bits */
+ *value = exposure >> 4;
+
+ return 0;
+}
+
+static int ov5693_exposure_configure(struct ov5693_device *ov5693,
+ u32 exposure)
+{
+ int ret = 0;
+
+ exposure = (exposure << 4) & OV5693_EXPOSURE_CTRL_MASK;
+
+ ov5693_write_reg(ov5693, OV5693_EXPOSURE_CTRL_REG, exposure, &ret);
+
+ return ret;
+}
+
+static int ov5693_get_gain(struct ov5693_device *ov5693, u32 *gain)
+{
+ u32 value;
+ int ret;
+
+ ret = ov5693_read_reg(ov5693, OV5693_GAIN_CTRL_REG, &value);
+ if (ret)
+ return ret;
+
+ /* As with exposure, the lowest 4 bits are fractional bits. */
+ *gain = value >> 4;
+
+ return ret;
+}
+
+static int ov5693_digital_gain_configure(struct ov5693_device *ov5693,
+ u32 gain)
+{
+ int ret = 0;
+
+ gain &= OV5693_MWB_GAIN_MASK;
+
+ ov5693_write_reg(ov5693, OV5693_MWB_RED_GAIN_REG, gain, &ret);
+ ov5693_write_reg(ov5693, OV5693_MWB_GREEN_GAIN_REG, gain, &ret);
+ ov5693_write_reg(ov5693, OV5693_MWB_BLUE_GAIN_REG, gain, &ret);
+
+ return ret;
+}
+
+static int ov5693_analog_gain_configure(struct ov5693_device *ov5693, u32 gain)
+{
+ int ret = 0;
+
+ gain = (gain << 4) & OV5693_GAIN_CTRL_MASK;
+
+ ov5693_write_reg(ov5693, OV5693_GAIN_CTRL_REG, gain, &ret);
+
+ return ret;
+}
+
+static int ov5693_vts_configure(struct ov5693_device *ov5693, u32 vblank)
+{
+ u16 vts = ov5693->mode.format.height + vblank;
+ int ret = 0;
+
+ ov5693_write_reg(ov5693, OV5693_TIMING_VTS_REG, vts, &ret);
+
+ return ret;
+}
+
+static int ov5693_test_pattern_configure(struct ov5693_device *ov5693, u32 idx)
+{
+ int ret = 0;
+
+ ov5693_write_reg(ov5693, OV5693_TEST_PATTERN_REG,
+ ov5693_test_pattern_bits[idx], &ret);
+
+ return ret;
+}
+
+static int ov5693_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+ struct ov5693_device *ov5693 =
+ container_of(ctrl->handler, struct ov5693_device, ctrls.handler);
+ int ret = 0;
+
+ /* If VBLANK is altered we need to update exposure to compensate */
+ if (ctrl->id == V4L2_CID_VBLANK) {
+ int exposure_max;
+
+ exposure_max = ov5693->mode.format.height + ctrl->val -
+ OV5693_INTEGRATION_TIME_MARGIN;
+ __v4l2_ctrl_modify_range(ov5693->ctrls.exposure,
+ ov5693->ctrls.exposure->minimum,
+ exposure_max,
+ ov5693->ctrls.exposure->step,
+ min(ov5693->ctrls.exposure->val,
+ exposure_max));
+ }
+
+ /* Only apply changes to the controls if the device is powered up */
+ if (!pm_runtime_get_if_in_use(ov5693->dev))
+ return 0;
+
+ switch (ctrl->id) {
+ case V4L2_CID_EXPOSURE:
+ ret = ov5693_exposure_configure(ov5693, ctrl->val);
+ break;
+ case V4L2_CID_ANALOGUE_GAIN:
+ ret = ov5693_analog_gain_configure(ov5693, ctrl->val);
+ break;
+ case V4L2_CID_DIGITAL_GAIN:
+ ret = ov5693_digital_gain_configure(ov5693, ctrl->val);
+ break;
+ case V4L2_CID_HFLIP:
+ ret = ov5693_flip_horz_configure(ov5693, !!ctrl->val);
+ break;
+ case V4L2_CID_VFLIP:
+ ret = ov5693_flip_vert_configure(ov5693, !!ctrl->val);
+ break;
+ case V4L2_CID_VBLANK:
+ ret = ov5693_vts_configure(ov5693, ctrl->val);
+ break;
+ case V4L2_CID_TEST_PATTERN:
+ ret = ov5693_test_pattern_configure(ov5693, ctrl->val);
+ break;
+ default:
+ ret = -EINVAL;
+ }
+
+ pm_runtime_put(ov5693->dev);
+
+ return ret;
+}
+
+static int ov5693_g_volatile_ctrl(struct v4l2_ctrl *ctrl)
+{
+ struct ov5693_device *ov5693 = container_of(ctrl->handler,
+ struct ov5693_device,
+ ctrls.handler);
+
+ switch (ctrl->id) {
+ case V4L2_CID_EXPOSURE_ABSOLUTE:
+ return ov5693_get_exposure(ov5693, &ctrl->val);
+ case V4L2_CID_AUTOGAIN:
+ return ov5693_get_gain(ov5693, &ctrl->val);
+ default:
+ return -EINVAL;
+ }
+}
+
+static const struct v4l2_ctrl_ops ov5693_ctrl_ops = {
+ .s_ctrl = ov5693_s_ctrl,
+ .g_volatile_ctrl = ov5693_g_volatile_ctrl
+};
+
+/* System Control Functions */
+
+static int ov5693_mode_configure(struct ov5693_device *ov5693)
+{
+ const struct ov5693_mode *mode = &ov5693->mode;
+ int ret = 0;
+
+ /* Crop Start X */
+ ov5693_write_reg(ov5693, OV5693_CROP_START_X_REG, mode->crop.left,
+ &ret);
+
+ /* Offset X */
+ ov5693_write_reg(ov5693, OV5693_OFFSET_START_X_REG, 0, &ret);
+
+ /* Output Size X */
+ ov5693_write_reg(ov5693, OV5693_OUTPUT_SIZE_X_REG, mode->format.width,
+ &ret);
+
+ /* Crop End X */
+ ov5693_write_reg(ov5693, OV5693_CROP_END_X_REG,
+ mode->crop.left + mode->crop.width, &ret);
+
+ /* Horizontal Total Size */
+ ov5693_write_reg(ov5693, OV5693_TIMING_HTS_REG, OV5693_FIXED_PPL,
+ &ret);
+
+ /* Crop Start Y */
+ ov5693_write_reg(ov5693, OV5693_CROP_START_Y_REG, mode->crop.top,
+ &ret);
+
+ /* Offset Y */
+ ov5693_write_reg(ov5693, OV5693_OFFSET_START_Y_REG, 0, &ret);
+
+ /* Output Size Y */
+ ov5693_write_reg(ov5693, OV5693_OUTPUT_SIZE_Y_REG, mode->format.height,
+ &ret);
+
+ /* Crop End Y */
+ ov5693_write_reg(ov5693, OV5693_CROP_END_Y_REG,
+ mode->crop.top + mode->crop.height, &ret);
+
+ /* Subsample X increase */
+ ov5693_write_reg(ov5693, OV5693_SUB_INC_X_REG,
+ ((mode->inc_x_odd << 4) & 0xf0) | 0x01, &ret);
+ /* Subsample Y increase */
+ ov5693_write_reg(ov5693, OV5693_SUB_INC_Y_REG,
+ ((mode->inc_y_odd << 4) & 0xf0) | 0x01, &ret);
+
+ if (ret)
+ return ret;
+
+ /* Binning */
+ ret = ov5693_update_bits(ov5693, OV5693_FORMAT1_REG,
+ OV5693_FORMAT1_VBIN_EN,
+ mode->binning_y ? OV5693_FORMAT1_VBIN_EN : 0);
+ if (ret)
+ return ret;
+
+ ret = ov5693_update_bits(ov5693, OV5693_FORMAT2_REG,
+ OV5693_FORMAT2_HBIN_EN,
+ mode->binning_x ? OV5693_FORMAT2_HBIN_EN : 0);
+
+ return ret;
+}
+
+static int ov5693_enable_streaming(struct ov5693_device *ov5693, bool enable)
+{
+ int ret = 0;
+
+ ov5693_write_reg(ov5693, OV5693_SW_STREAM_REG,
+ enable ? OV5693_START_STREAMING :
+ OV5693_STOP_STREAMING, &ret);
+
+ return ret;
+}
+
+static int ov5693_sw_reset(struct ov5693_device *ov5693)
+{
+ int ret = 0;
+
+ ov5693_write_reg(ov5693, OV5693_SW_RESET_REG, OV5693_SW_RESET, &ret);
+
+ return ret;
+}
+
+static int ov5693_sensor_init(struct ov5693_device *ov5693)
+{
+ int ret;
+
+ ret = ov5693_sw_reset(ov5693);
+ if (ret)
+ return dev_err_probe(ov5693->dev, ret,
+ "software reset error\n");
+
+ ret = ov5693_write_reg_array(ov5693, &ov5693_global_setting);
+ if (ret)
+ return dev_err_probe(ov5693->dev, ret,
+ "global settings error\n");
+
+ ret = ov5693_mode_configure(ov5693);
+ if (ret)
+ return dev_err_probe(ov5693->dev, ret,
+ "mode configure error\n");
+
+ ret = ov5693_enable_streaming(ov5693, false);
+ if (ret)
+ dev_err(ov5693->dev, "stop streaming error\n");
+
+ return ret;
+}
+
+static void ov5693_sensor_powerdown(struct ov5693_device *ov5693)
+{
+ gpiod_set_value_cansleep(ov5693->reset, 1);
+ gpiod_set_value_cansleep(ov5693->powerdown, 1);
+
+ regulator_bulk_disable(OV5693_NUM_SUPPLIES, ov5693->supplies);
+
+ clk_disable_unprepare(ov5693->clk);
+}
+
+static int ov5693_sensor_powerup(struct ov5693_device *ov5693)
+{
+ int ret;
+
+ gpiod_set_value_cansleep(ov5693->reset, 1);
+ gpiod_set_value_cansleep(ov5693->powerdown, 1);
+
+ ret = clk_prepare_enable(ov5693->clk);
+ if (ret) {
+ dev_err(ov5693->dev, "Failed to enable clk\n");
+ goto fail_power;
+ }
+
+ ret = regulator_bulk_enable(OV5693_NUM_SUPPLIES, ov5693->supplies);
+ if (ret) {
+ dev_err(ov5693->dev, "Failed to enable regulators\n");
+ goto fail_power;
+ }
+
+ gpiod_set_value_cansleep(ov5693->powerdown, 0);
+ gpiod_set_value_cansleep(ov5693->reset, 0);
+
+ usleep_range(5000, 7500);
+
+ return 0;
+
+fail_power:
+ ov5693_sensor_powerdown(ov5693);
+ return ret;
+}
+
+static int __maybe_unused ov5693_sensor_suspend(struct device *dev)
+{
+ struct v4l2_subdev *sd = dev_get_drvdata(dev);
+ struct ov5693_device *ov5693 = to_ov5693_sensor(sd);
+
+ ov5693_sensor_powerdown(ov5693);
+
+ return 0;
+}
+
+static int __maybe_unused ov5693_sensor_resume(struct device *dev)
+{
+ struct v4l2_subdev *sd = dev_get_drvdata(dev);
+ struct ov5693_device *ov5693 = to_ov5693_sensor(sd);
+ int ret;
+
+ mutex_lock(&ov5693->lock);
+
+ ret = ov5693_sensor_powerup(ov5693);
+ if (ret)
+ goto out_unlock;
+
+ ret = ov5693_sensor_init(ov5693);
+ if (ret) {
+ dev_err(dev, "ov5693 sensor init failure\n");
+ goto err_power;
+ }
+
+ goto out_unlock;
+
+err_power:
+ ov5693_sensor_powerdown(ov5693);
+out_unlock:
+ mutex_unlock(&ov5693->lock);
+ return ret;
+}
+
+static int ov5693_detect(struct ov5693_device *ov5693)
+{
+ int ret;
+ u32 id;
+
+ ret = ov5693_read_reg(ov5693, OV5693_REG_CHIP_ID, &id);
+ if (ret)
+ return ret;
+
+ if (id != OV5693_CHIP_ID)
+ return dev_err_probe(ov5693->dev, -ENODEV,
+ "sensor ID mismatch. Found 0x%04x\n", id);
+
+ return 0;
+}
+
+/* V4L2 Framework callbacks */
+
+static unsigned int __ov5693_calc_vts(u32 height)
+{
+ /*
+ * We need to set a sensible default VTS for whatever format height we
+ * happen to be given from set_fmt(). This function just targets
+ * an even multiple of 30fps.
+ */
+
+ unsigned int tgt_fps;
+
+ tgt_fps = rounddown(OV5693_PIXEL_RATE / OV5693_FIXED_PPL / height, 30);
+
+ return ALIGN_DOWN(OV5693_PIXEL_RATE / OV5693_FIXED_PPL / tgt_fps, 2);
+}
+
+static struct v4l2_mbus_framefmt *
+__ov5693_get_pad_format(struct ov5693_device *ov5693,
+ struct v4l2_subdev_state *state,
+ unsigned int pad, enum v4l2_subdev_format_whence which)
+{
+ switch (which) {
+ case V4L2_SUBDEV_FORMAT_TRY:
+ return v4l2_subdev_get_try_format(&ov5693->sd, state, pad);
+ case V4L2_SUBDEV_FORMAT_ACTIVE:
+ return &ov5693->mode.format;
+ default:
+ return NULL;
+ }
+}
+
+static struct v4l2_rect *
+__ov5693_get_pad_crop(struct ov5693_device *ov5693,
+ struct v4l2_subdev_state *state,
+ unsigned int pad, enum v4l2_subdev_format_whence which)
+{
+ switch (which) {
+ case V4L2_SUBDEV_FORMAT_TRY:
+ return v4l2_subdev_get_try_crop(&ov5693->sd, state, pad);
+ case V4L2_SUBDEV_FORMAT_ACTIVE:
+ return &ov5693->mode.crop;
+ }
+
+ return NULL;
+}
+
+static int ov5693_get_fmt(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_format *format)
+{
+ struct ov5693_device *ov5693 = to_ov5693_sensor(sd);
+
+ format->format = ov5693->mode.format;
+
+ return 0;
+}
+
+static int ov5693_set_fmt(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_format *format)
+{
+ struct ov5693_device *ov5693 = to_ov5693_sensor(sd);
+ const struct v4l2_rect *crop;
+ struct v4l2_mbus_framefmt *fmt;
+ unsigned int hratio, vratio;
+ unsigned int width, height;
+ unsigned int hblank;
+ int exposure_max;
+ int ret = 0;
+
+ crop = __ov5693_get_pad_crop(ov5693, state, format->pad, format->which);
+
+ /*
+ * Align to two to simplify the binning calculations below, and clamp
+ * the requested format at the crop rectangle
+ */
+ width = clamp_t(unsigned int, ALIGN(format->format.width, 2),
+ OV5693_MIN_CROP_WIDTH, crop->width);
+ height = clamp_t(unsigned int, ALIGN(format->format.height, 2),
+ OV5693_MIN_CROP_HEIGHT, crop->height);
+
+ /*
+ * We can only support setting either the dimensions of the crop rect
+ * or those dimensions binned (separately) by a factor of two.
+ */
+ hratio = clamp_t(unsigned int,
+ DIV_ROUND_CLOSEST(crop->width, width), 1, 2);
+ vratio = clamp_t(unsigned int,
+ DIV_ROUND_CLOSEST(crop->height, height), 1, 2);
+
+ fmt = __ov5693_get_pad_format(ov5693, state, format->pad,
+ format->which);
+
+ fmt->width = crop->width / hratio;
+ fmt->height = crop->height / vratio;
+ fmt->code = MEDIA_BUS_FMT_SBGGR10_1X10;
+
+ format->format = *fmt;
+
+ if (format->which == V4L2_SUBDEV_FORMAT_TRY)
+ return ret;
+
+ mutex_lock(&ov5693->lock);
+
+ ov5693->mode.binning_x = hratio > 1 ? true : false;
+ ov5693->mode.inc_x_odd = hratio > 1 ? 3 : 1;
+ ov5693->mode.binning_y = vratio > 1 ? true : false;
+ ov5693->mode.inc_y_odd = vratio > 1 ? 3 : 1;
+
+ ov5693->mode.vts = __ov5693_calc_vts(fmt->height);
+
+ __v4l2_ctrl_modify_range(ov5693->ctrls.vblank,
+ OV5693_TIMING_MIN_VTS,
+ OV5693_TIMING_MAX_VTS - fmt->height,
+ 1, ov5693->mode.vts - fmt->height);
+ __v4l2_ctrl_s_ctrl(ov5693->ctrls.vblank,
+ ov5693->mode.vts - fmt->height);
+
+ hblank = OV5693_FIXED_PPL - fmt->width;
+ __v4l2_ctrl_modify_range(ov5693->ctrls.hblank, hblank, hblank, 1,
+ hblank);
+
+ exposure_max = ov5693->mode.vts - OV5693_INTEGRATION_TIME_MARGIN;
+ __v4l2_ctrl_modify_range(ov5693->ctrls.exposure,
+ ov5693->ctrls.exposure->minimum, exposure_max,
+ ov5693->ctrls.exposure->step,
+ min(ov5693->ctrls.exposure->val,
+ exposure_max));
+
+ mutex_unlock(&ov5693->lock);
+ return ret;
+}
+
+static int ov5693_get_selection(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_selection *sel)
+{
+ struct ov5693_device *ov5693 = to_ov5693_sensor(sd);
+
+ switch (sel->target) {
+ case V4L2_SEL_TGT_CROP:
+ mutex_lock(&ov5693->lock);
+ sel->r = *__ov5693_get_pad_crop(ov5693, state, sel->pad,
+ sel->which);
+ mutex_unlock(&ov5693->lock);
+ break;
+ case V4L2_SEL_TGT_NATIVE_SIZE:
+ sel->r.top = 0;
+ sel->r.left = 0;
+ sel->r.width = OV5693_NATIVE_WIDTH;
+ sel->r.height = OV5693_NATIVE_HEIGHT;
+ break;
+ case V4L2_SEL_TGT_CROP_BOUNDS:
+ case V4L2_SEL_TGT_CROP_DEFAULT:
+ sel->r.top = OV5693_ACTIVE_START_TOP;
+ sel->r.left = OV5693_ACTIVE_START_LEFT;
+ sel->r.width = OV5693_ACTIVE_WIDTH;
+ sel->r.height = OV5693_ACTIVE_HEIGHT;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int ov5693_set_selection(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_selection *sel)
+{
+ struct ov5693_device *ov5693 = to_ov5693_sensor(sd);
+ struct v4l2_mbus_framefmt *format;
+ struct v4l2_rect *__crop;
+ struct v4l2_rect rect;
+
+ if (sel->target != V4L2_SEL_TGT_CROP)
+ return -EINVAL;
+
+ /*
+ * Clamp the boundaries of the crop rectangle to the size of the sensor
+ * pixel array. Align to multiples of 2 to ensure Bayer pattern isn't
+ * disrupted.
+ */
+ rect.left = clamp(ALIGN(sel->r.left, 2), OV5693_NATIVE_START_LEFT,
+ OV5693_NATIVE_WIDTH);
+ rect.top = clamp(ALIGN(sel->r.top, 2), OV5693_NATIVE_START_TOP,
+ OV5693_NATIVE_HEIGHT);
+ rect.width = clamp_t(unsigned int, ALIGN(sel->r.width, 2),
+ OV5693_MIN_CROP_WIDTH, OV5693_NATIVE_WIDTH);
+ rect.height = clamp_t(unsigned int, ALIGN(sel->r.height, 2),
+ OV5693_MIN_CROP_HEIGHT, OV5693_NATIVE_HEIGHT);
+
+ /* Make sure the crop rectangle isn't outside the bounds of the array */
+ rect.width = min_t(unsigned int, rect.width,
+ OV5693_NATIVE_WIDTH - rect.left);
+ rect.height = min_t(unsigned int, rect.height,
+ OV5693_NATIVE_HEIGHT - rect.top);
+
+ __crop = __ov5693_get_pad_crop(ov5693, state, sel->pad, sel->which);
+
+ if (rect.width != __crop->width || rect.height != __crop->height) {
+ /*
+ * Reset the output image size if the crop rectangle size has
+ * been modified.
+ */
+ format = __ov5693_get_pad_format(ov5693, state, sel->pad,
+ sel->which);
+ format->width = rect.width;
+ format->height = rect.height;
+ }
+
+ *__crop = rect;
+ sel->r = rect;
+
+ return 0;
+}
+
+static int ov5693_s_stream(struct v4l2_subdev *sd, int enable)
+{
+ struct ov5693_device *ov5693 = to_ov5693_sensor(sd);
+ int ret;
+
+ if (enable) {
+ ret = pm_runtime_get_sync(ov5693->dev);
+ if (ret < 0)
+ goto err_power_down;
+
+ mutex_lock(&ov5693->lock);
+ ret = __v4l2_ctrl_handler_setup(&ov5693->ctrls.handler);
+ if (ret) {
+ mutex_unlock(&ov5693->lock);
+ goto err_power_down;
+ }
+
+ ret = ov5693_enable_streaming(ov5693, true);
+ mutex_unlock(&ov5693->lock);
+ } else {
+ mutex_lock(&ov5693->lock);
+ ret = ov5693_enable_streaming(ov5693, false);
+ mutex_unlock(&ov5693->lock);
+ }
+ if (ret)
+ goto err_power_down;
+
+ ov5693->streaming = !!enable;
+
+ if (!enable)
+ pm_runtime_put(ov5693->dev);
+
+ return 0;
+err_power_down:
+ pm_runtime_put_noidle(ov5693->dev);
+ return ret;
+}
+
+static int ov5693_g_frame_interval(struct v4l2_subdev *sd,
+ struct v4l2_subdev_frame_interval *interval)
+{
+ struct ov5693_device *ov5693 = to_ov5693_sensor(sd);
+ unsigned int framesize = OV5693_FIXED_PPL * (ov5693->mode.format.height +
+ ov5693->ctrls.vblank->val);
+ unsigned int fps = DIV_ROUND_CLOSEST(OV5693_PIXEL_RATE, framesize);
+
+ interval->interval.numerator = 1;
+ interval->interval.denominator = fps;
+
+ return 0;
+}
+
+static int ov5693_enum_mbus_code(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_mbus_code_enum *code)
+{
+ /* Only a single mbus format is supported */
+ if (code->index > 0)
+ return -EINVAL;
+
+ code->code = MEDIA_BUS_FMT_SBGGR10_1X10;
+ return 0;
+}
+
+static int ov5693_enum_frame_size(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_frame_size_enum *fse)
+{
+ struct ov5693_device *ov5693 = to_ov5693_sensor(sd);
+ struct v4l2_rect *__crop;
+
+ if (fse->index > 1 || fse->code != MEDIA_BUS_FMT_SBGGR10_1X10)
+ return -EINVAL;
+
+ __crop = __ov5693_get_pad_crop(ov5693, state, fse->pad, fse->which);
+ if (!__crop)
+ return -EINVAL;
+
+ fse->min_width = __crop->width / (fse->index + 1);
+ fse->min_height = __crop->height / (fse->index + 1);
+ fse->max_width = fse->min_width;
+ fse->max_height = fse->min_height;
+
+ return 0;
+}
+
+static const struct v4l2_subdev_video_ops ov5693_video_ops = {
+ .s_stream = ov5693_s_stream,
+ .g_frame_interval = ov5693_g_frame_interval,
+};
+
+static const struct v4l2_subdev_pad_ops ov5693_pad_ops = {
+ .enum_mbus_code = ov5693_enum_mbus_code,
+ .enum_frame_size = ov5693_enum_frame_size,
+ .get_fmt = ov5693_get_fmt,
+ .set_fmt = ov5693_set_fmt,
+ .get_selection = ov5693_get_selection,
+ .set_selection = ov5693_set_selection,
+};
+
+static const struct v4l2_subdev_ops ov5693_ops = {
+ .video = &ov5693_video_ops,
+ .pad = &ov5693_pad_ops,
+};
+
+/* Sensor and Driver Configuration Functions */
+
+static int ov5693_init_controls(struct ov5693_device *ov5693)
+{
+ const struct v4l2_ctrl_ops *ops = &ov5693_ctrl_ops;
+ struct ov5693_v4l2_ctrls *ctrls = &ov5693->ctrls;
+ struct v4l2_fwnode_device_properties props;
+ int vblank_max, vblank_def;
+ int exposure_max;
+ int hblank;
+ int ret;
+
+ ret = v4l2_ctrl_handler_init(&ctrls->handler, 12);
+ if (ret)
+ return ret;
+
+ /* link freq */
+ ctrls->link_freq = v4l2_ctrl_new_int_menu(&ctrls->handler,
+ NULL, V4L2_CID_LINK_FREQ,
+ 0, 0, link_freq_menu_items);
+ if (ctrls->link_freq)
+ ctrls->link_freq->flags |= V4L2_CTRL_FLAG_READ_ONLY;
+
+ /* pixel rate */
+ ctrls->pixel_rate = v4l2_ctrl_new_std(&ctrls->handler, NULL,
+ V4L2_CID_PIXEL_RATE, 0,
+ OV5693_PIXEL_RATE, 1,
+ OV5693_PIXEL_RATE);
+
+ /* Exposure */
+ exposure_max = ov5693->mode.vts - OV5693_INTEGRATION_TIME_MARGIN;
+ ctrls->exposure = v4l2_ctrl_new_std(&ctrls->handler, ops,
+ V4L2_CID_EXPOSURE,
+ OV5693_EXPOSURE_MIN, exposure_max,
+ OV5693_EXPOSURE_STEP, exposure_max);
+
+ /* Gain */
+ ctrls->analogue_gain = v4l2_ctrl_new_std(&ctrls->handler,
+ ops, V4L2_CID_ANALOGUE_GAIN,
+ OV5693_GAIN_MIN,
+ OV5693_GAIN_MAX,
+ OV5693_GAIN_STEP,
+ OV5693_GAIN_DEF);
+
+ ctrls->digital_gain = v4l2_ctrl_new_std(&ctrls->handler, ops,
+ V4L2_CID_DIGITAL_GAIN,
+ OV5693_DIGITAL_GAIN_MIN,
+ OV5693_DIGITAL_GAIN_MAX,
+ OV5693_DIGITAL_GAIN_STEP,
+ OV5693_DIGITAL_GAIN_DEF);
+
+ /* Flip */
+ ctrls->hflip = v4l2_ctrl_new_std(&ctrls->handler, ops,
+ V4L2_CID_HFLIP, 0, 1, 1, 0);
+
+ ctrls->vflip = v4l2_ctrl_new_std(&ctrls->handler, ops,
+ V4L2_CID_VFLIP, 0, 1, 1, 0);
+
+ hblank = OV5693_FIXED_PPL - ov5693->mode.format.width;
+ ctrls->hblank = v4l2_ctrl_new_std(&ctrls->handler, ops,
+ V4L2_CID_HBLANK, hblank,
+ hblank, 1, hblank);
+
+ if (ctrls->hblank)
+ ctrls->hblank->flags |= V4L2_CTRL_FLAG_READ_ONLY;
+
+ vblank_max = OV5693_TIMING_MAX_VTS - ov5693->mode.format.height;
+ vblank_def = ov5693->mode.vts - ov5693->mode.format.height;
+ ctrls->vblank = v4l2_ctrl_new_std(&ctrls->handler, ops,
+ V4L2_CID_VBLANK,
+ OV5693_TIMING_MIN_VTS,
+ vblank_max, 1, vblank_def);
+
+ ctrls->test_pattern = v4l2_ctrl_new_std_menu_items(
+ &ctrls->handler, ops,
+ V4L2_CID_TEST_PATTERN,
+ ARRAY_SIZE(ov5693_test_pattern_menu) - 1,
+ 0, 0, ov5693_test_pattern_menu);
+
+ if (ctrls->handler.error) {
+ dev_err(ov5693->dev, "Error initialising v4l2 ctrls\n");
+ ret = ctrls->handler.error;
+ goto err_free_handler;
+ }
+
+ /* set properties from fwnode (e.g. rotation, orientation) */
+ ret = v4l2_fwnode_device_parse(ov5693->dev, &props);
+ if (ret)
+ goto err_free_handler;
+
+ ret = v4l2_ctrl_new_fwnode_properties(&ctrls->handler, ops,
+ &props);
+ if (ret)
+ goto err_free_handler;
+
+ /* Use same lock for controls as for everything else. */
+ ctrls->handler.lock = &ov5693->lock;
+ ov5693->sd.ctrl_handler = &ctrls->handler;
+
+ return 0;
+
+err_free_handler:
+ v4l2_ctrl_handler_free(&ctrls->handler);
+ return ret;
+}
+
+static int ov5693_configure_gpios(struct ov5693_device *ov5693)
+{
+ ov5693->reset = devm_gpiod_get_optional(ov5693->dev, "reset",
+ GPIOD_OUT_HIGH);
+ if (IS_ERR(ov5693->reset)) {
+ dev_err(ov5693->dev, "Error fetching reset GPIO\n");
+ return PTR_ERR(ov5693->reset);
+ }
+
+ ov5693->powerdown = devm_gpiod_get_optional(ov5693->dev, "powerdown",
+ GPIOD_OUT_HIGH);
+ if (IS_ERR(ov5693->powerdown)) {
+ dev_err(ov5693->dev, "Error fetching powerdown GPIO\n");
+ return PTR_ERR(ov5693->powerdown);
+ }
+
+ return 0;
+}
+
+static int ov5693_get_regulators(struct ov5693_device *ov5693)
+{
+ unsigned int i;
+
+ for (i = 0; i < OV5693_NUM_SUPPLIES; i++)
+ ov5693->supplies[i].supply = ov5693_supply_names[i];
+
+ return devm_regulator_bulk_get(ov5693->dev, OV5693_NUM_SUPPLIES,
+ ov5693->supplies);
+}
+
+static int ov5693_check_hwcfg(struct ov5693_device *ov5693)
+{
+ struct fwnode_handle *fwnode = dev_fwnode(ov5693->dev);
+ struct v4l2_fwnode_endpoint bus_cfg = {
+ .bus_type = V4L2_MBUS_CSI2_DPHY,
+ };
+ struct fwnode_handle *endpoint;
+ unsigned int i;
+ int ret;
+
+ endpoint = fwnode_graph_get_next_endpoint(fwnode, NULL);
+ if (!endpoint)
+ return -EPROBE_DEFER; /* Could be provided by cio2-bridge */
+
+ ret = v4l2_fwnode_endpoint_alloc_parse(endpoint, &bus_cfg);
+ fwnode_handle_put(endpoint);
+ if (ret)
+ return ret;
+
+ if (bus_cfg.bus.mipi_csi2.num_data_lanes != 2) {
+ dev_err(ov5693->dev, "only a 2-lane CSI2 config is supported");
+ ret = -EINVAL;
+ goto out_free_bus_cfg;
+ }
+
+ if (!bus_cfg.nr_of_link_frequencies) {
+ dev_err(ov5693->dev, "no link frequencies defined\n");
+ ret = -EINVAL;
+ goto out_free_bus_cfg;
+ }
+
+ for (i = 0; i < bus_cfg.nr_of_link_frequencies; i++)
+ if (bus_cfg.link_frequencies[i] == OV5693_LINK_FREQ_419_2MHZ)
+ break;
+
+ if (i == bus_cfg.nr_of_link_frequencies) {
+ dev_err(ov5693->dev, "supported link freq %ull not found\n",
+ OV5693_LINK_FREQ_419_2MHZ);
+ ret = -EINVAL;
+ goto out_free_bus_cfg;
+ }
+
+out_free_bus_cfg:
+ v4l2_fwnode_endpoint_free(&bus_cfg);
+
+ return ret;
+}
+
+static int ov5693_probe(struct i2c_client *client)
+{
+ struct ov5693_device *ov5693;
+ u32 clk_rate;
+ int ret = 0;
+
+ ov5693 = devm_kzalloc(&client->dev, sizeof(*ov5693), GFP_KERNEL);
+ if (!ov5693)
+ return -ENOMEM;
+
+ ov5693->client = client;
+ ov5693->dev = &client->dev;
+
+ ret = ov5693_check_hwcfg(ov5693);
+ if (ret)
+ return ret;
+
+ mutex_init(&ov5693->lock);
+
+ v4l2_i2c_subdev_init(&ov5693->sd, client, &ov5693_ops);
+
+ ov5693->clk = devm_clk_get(&client->dev, "xvclk");
+ if (IS_ERR(ov5693->clk)) {
+ dev_err(&client->dev, "Error getting clock\n");
+ return PTR_ERR(ov5693->clk);
+ }
+
+ clk_rate = clk_get_rate(ov5693->clk);
+ if (clk_rate != OV5693_XVCLK_FREQ)
+ dev_warn(&client->dev, "Found clk freq %u, expected %u\n",
+ clk_rate, OV5693_XVCLK_FREQ);
+
+ ret = ov5693_configure_gpios(ov5693);
+ if (ret)
+ return ret;
+
+ ret = ov5693_get_regulators(ov5693);
+ if (ret)
+ return dev_err_probe(&client->dev, ret,
+ "Error fetching regulators\n");
+
+ ov5693->sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
+ ov5693->pad.flags = MEDIA_PAD_FL_SOURCE;
+ ov5693->sd.entity.function = MEDIA_ENT_F_CAM_SENSOR;
+
+ ov5693->mode.crop = ov5693_default_crop;
+ ov5693->mode.format = ov5693_default_fmt;
+ ov5693->mode.vts = __ov5693_calc_vts(ov5693->mode.format.height);
+
+ ret = ov5693_init_controls(ov5693);
+ if (ret)
+ return ret;
+
+ ret = media_entity_pads_init(&ov5693->sd.entity, 1, &ov5693->pad);
+ if (ret)
+ goto err_ctrl_handler_free;
+
+ /*
+ * We need the driver to work in the event that pm runtime is disable in
+ * the kernel, so power up and verify the chip now. In the event that
+ * runtime pm is disabled this will leave the chip on, so that streaming
+ * will work.
+ */
+
+ ret = ov5693_sensor_powerup(ov5693);
+ if (ret)
+ goto err_media_entity_cleanup;
+
+ ret = ov5693_detect(ov5693);
+ if (ret)
+ goto err_powerdown;
+
+ pm_runtime_set_active(&client->dev);
+ pm_runtime_get_noresume(&client->dev);
+ pm_runtime_enable(&client->dev);
+
+ ret = v4l2_async_register_subdev_sensor(&ov5693->sd);
+ if (ret) {
+ dev_err(&client->dev, "failed to register V4L2 subdev: %d",
+ ret);
+ goto err_pm_runtime;
+ }
+
+ pm_runtime_set_autosuspend_delay(&client->dev, 1000);
+ pm_runtime_use_autosuspend(&client->dev);
+ pm_runtime_put_autosuspend(&client->dev);
+
+ return ret;
+
+err_pm_runtime:
+ pm_runtime_disable(&client->dev);
+ pm_runtime_put_noidle(&client->dev);
+err_powerdown:
+ ov5693_sensor_powerdown(ov5693);
+err_media_entity_cleanup:
+ media_entity_cleanup(&ov5693->sd.entity);
+err_ctrl_handler_free:
+ v4l2_ctrl_handler_free(&ov5693->ctrls.handler);
+
+ return ret;
+}
+
+static int ov5693_remove(struct i2c_client *client)
+{
+ struct v4l2_subdev *sd = i2c_get_clientdata(client);
+ struct ov5693_device *ov5693 = to_ov5693_sensor(sd);
+
+ v4l2_async_unregister_subdev(sd);
+ media_entity_cleanup(&ov5693->sd.entity);
+ v4l2_ctrl_handler_free(&ov5693->ctrls.handler);
+ mutex_destroy(&ov5693->lock);
+
+ /*
+ * Disable runtime PM. In case runtime PM is disabled in the kernel,
+ * make sure to turn power off manually.
+ */
+ pm_runtime_disable(&client->dev);
+ if (!pm_runtime_status_suspended(&client->dev))
+ ov5693_sensor_powerdown(ov5693);
+ pm_runtime_set_suspended(&client->dev);
+
+ return 0;
+}
+
+static const struct dev_pm_ops ov5693_pm_ops = {
+ SET_RUNTIME_PM_OPS(ov5693_sensor_suspend, ov5693_sensor_resume, NULL)
+};
+
+static const struct acpi_device_id ov5693_acpi_match[] = {
+ {"INT33BE"},
+ {},
+};
+MODULE_DEVICE_TABLE(acpi, ov5693_acpi_match);
+
+static struct i2c_driver ov5693_driver = {
+ .driver = {
+ .name = "ov5693",
+ .acpi_match_table = ov5693_acpi_match,
+ .pm = &ov5693_pm_ops,
+ },
+ .probe_new = ov5693_probe,
+ .remove = ov5693_remove,
+};
+module_i2c_driver(ov5693_driver);
+
+MODULE_DESCRIPTION("A low-level driver for OmniVision 5693 sensors");
+MODULE_LICENSE("GPL");
diff --git a/drivers/media/i2c/ov8856.c b/drivers/media/i2c/ov8856.c
index c6c6050..8785764 100644
--- a/drivers/media/i2c/ov8856.c
+++ b/drivers/media/i2c/ov8856.c
@@ -1445,6 +1445,9 @@ struct ov8856 {
const struct ov8856_lane_cfg *priv_lane;
u8 modes_size;
+
+ /* True if the device has been identified */
+ bool identified;
};
struct ov8856_lane_cfg {
@@ -1685,6 +1688,71 @@ static int ov8856_write_reg_list(struct ov8856 *ov8856,
return 0;
}
+static int ov8856_identify_module(struct ov8856 *ov8856)
+{
+ struct i2c_client *client = v4l2_get_subdevdata(&ov8856->sd);
+ int ret;
+ u32 val;
+
+ if (ov8856->identified)
+ return 0;
+
+ ret = ov8856_read_reg(ov8856, OV8856_REG_CHIP_ID,
+ OV8856_REG_VALUE_24BIT, &val);
+ if (ret)
+ return ret;
+
+ if (val != OV8856_CHIP_ID) {
+ dev_err(&client->dev, "chip id mismatch: %x!=%x",
+ OV8856_CHIP_ID, val);
+ return -ENXIO;
+ }
+
+ ret = ov8856_write_reg(ov8856, OV8856_REG_MODE_SELECT,
+ OV8856_REG_VALUE_08BIT, OV8856_MODE_STREAMING);
+ if (ret)
+ return ret;
+
+ ret = ov8856_write_reg(ov8856, OV8856_OTP_MODE_CTRL,
+ OV8856_REG_VALUE_08BIT, OV8856_OTP_MODE_AUTO);
+ if (ret) {
+ dev_err(&client->dev, "failed to set otp mode");
+ return ret;
+ }
+
+ ret = ov8856_write_reg(ov8856, OV8856_OTP_LOAD_CTRL,
+ OV8856_REG_VALUE_08BIT,
+ OV8856_OTP_LOAD_CTRL_ENABLE);
+ if (ret) {
+ dev_err(&client->dev, "failed to enable load control");
+ return ret;
+ }
+
+ ret = ov8856_read_reg(ov8856, OV8856_MODULE_REVISION,
+ OV8856_REG_VALUE_08BIT, &val);
+ if (ret) {
+ dev_err(&client->dev, "failed to read module revision");
+ return ret;
+ }
+
+ dev_info(&client->dev, "OV8856 revision %x (%s) at address 0x%02x\n",
+ val,
+ val == OV8856_2A_MODULE ? "2A" :
+ val == OV8856_1B_MODULE ? "1B" : "unknown revision",
+ client->addr);
+
+ ret = ov8856_write_reg(ov8856, OV8856_REG_MODE_SELECT,
+ OV8856_REG_VALUE_08BIT, OV8856_MODE_STANDBY);
+ if (ret) {
+ dev_err(&client->dev, "failed to exit streaming mode");
+ return ret;
+ }
+
+ ov8856->identified = true;
+
+ return 0;
+}
+
static int ov8856_update_digital_gain(struct ov8856 *ov8856, u32 d_gain)
{
int ret;
@@ -1969,6 +2037,10 @@ static int ov8856_start_streaming(struct ov8856 *ov8856)
const struct ov8856_reg_list *reg_list;
int link_freq_index, ret;
+ ret = ov8856_identify_module(ov8856);
+ if (ret)
+ return ret;
+
link_freq_index = ov8856->cur_mode->link_freq_index;
reg_list = &ov8856->priv_lane->link_freq_configs[link_freq_index].reg_list;
@@ -2276,65 +2348,6 @@ static const struct v4l2_subdev_internal_ops ov8856_internal_ops = {
.open = ov8856_open,
};
-static int ov8856_identify_module(struct ov8856 *ov8856)
-{
- struct i2c_client *client = v4l2_get_subdevdata(&ov8856->sd);
- int ret;
- u32 val;
-
- ret = ov8856_read_reg(ov8856, OV8856_REG_CHIP_ID,
- OV8856_REG_VALUE_24BIT, &val);
- if (ret)
- return ret;
-
- if (val != OV8856_CHIP_ID) {
- dev_err(&client->dev, "chip id mismatch: %x!=%x",
- OV8856_CHIP_ID, val);
- return -ENXIO;
- }
-
- ret = ov8856_write_reg(ov8856, OV8856_REG_MODE_SELECT,
- OV8856_REG_VALUE_08BIT, OV8856_MODE_STREAMING);
- if (ret)
- return ret;
-
- ret = ov8856_write_reg(ov8856, OV8856_OTP_MODE_CTRL,
- OV8856_REG_VALUE_08BIT, OV8856_OTP_MODE_AUTO);
- if (ret) {
- dev_err(&client->dev, "failed to set otp mode");
- return ret;
- }
-
- ret = ov8856_write_reg(ov8856, OV8856_OTP_LOAD_CTRL,
- OV8856_REG_VALUE_08BIT,
- OV8856_OTP_LOAD_CTRL_ENABLE);
- if (ret) {
- dev_err(&client->dev, "failed to enable load control");
- return ret;
- }
-
- ret = ov8856_read_reg(ov8856, OV8856_MODULE_REVISION,
- OV8856_REG_VALUE_08BIT, &val);
- if (ret) {
- dev_err(&client->dev, "failed to read module revision");
- return ret;
- }
-
- dev_info(&client->dev, "OV8856 revision %x (%s) at address 0x%02x\n",
- val,
- val == OV8856_2A_MODULE ? "2A" :
- val == OV8856_1B_MODULE ? "1B" : "unknown revision",
- client->addr);
-
- ret = ov8856_write_reg(ov8856, OV8856_REG_MODE_SELECT,
- OV8856_REG_VALUE_08BIT, OV8856_MODE_STANDBY);
- if (ret) {
- dev_err(&client->dev, "failed to exit streaming mode");
- return ret;
- }
-
- return 0;
-}
static int ov8856_get_hwcfg(struct ov8856 *ov8856, struct device *dev)
{
@@ -2458,6 +2471,7 @@ static int ov8856_probe(struct i2c_client *client)
{
struct ov8856 *ov8856;
int ret;
+ bool full_power;
ov8856 = devm_kzalloc(&client->dev, sizeof(*ov8856), GFP_KERNEL);
if (!ov8856)
@@ -2472,16 +2486,19 @@ static int ov8856_probe(struct i2c_client *client)
v4l2_i2c_subdev_init(&ov8856->sd, client, &ov8856_subdev_ops);
- ret = __ov8856_power_on(ov8856);
- if (ret) {
- dev_err(&client->dev, "failed to power on\n");
- return ret;
- }
+ full_power = acpi_dev_state_d0(&client->dev);
+ if (full_power) {
+ ret = __ov8856_power_on(ov8856);
+ if (ret) {
+ dev_err(&client->dev, "failed to power on\n");
+ return ret;
+ }
- ret = ov8856_identify_module(ov8856);
- if (ret) {
- dev_err(&client->dev, "failed to find sensor: %d", ret);
- goto probe_power_off;
+ ret = ov8856_identify_module(ov8856);
+ if (ret) {
+ dev_err(&client->dev, "failed to find sensor: %d", ret);
+ goto probe_power_off;
+ }
}
mutex_init(&ov8856->mutex);
@@ -2511,11 +2528,9 @@ static int ov8856_probe(struct i2c_client *client)
goto probe_error_media_entity_cleanup;
}
- /*
- * Device is already turned on by i2c-core with ACPI domain PM.
- * Enable runtime PM and turn off the device.
- */
- pm_runtime_set_active(&client->dev);
+ /* Set the device's state to active if it's in D0 state. */
+ if (full_power)
+ pm_runtime_set_active(&client->dev);
pm_runtime_enable(&client->dev);
pm_runtime_idle(&client->dev);
@@ -2562,6 +2577,7 @@ static struct i2c_driver ov8856_i2c_driver = {
},
.probe_new = ov8856_probe,
.remove = ov8856_remove,
+ .flags = I2C_DRV_ACPI_WAIVE_D0_PROBE,
};
module_i2c_driver(ov8856_i2c_driver);
diff --git a/drivers/media/i2c/ov8865.c b/drivers/media/i2c/ov8865.c
index ce50f3e..d9d016c 100644
--- a/drivers/media/i2c/ov8865.c
+++ b/drivers/media/i2c/ov8865.c
@@ -9,6 +9,7 @@
#include <linux/delay.h>
#include <linux/device.h>
#include <linux/i2c.h>
+#include <linux/mod_devicetable.h>
#include <linux/module.h>
#include <linux/of_graph.h>
#include <linux/pm_runtime.h>
@@ -20,10 +21,6 @@
#include <media/v4l2-image-sizes.h>
#include <media/v4l2-mediabus.h>
-/* Clock rate */
-
-#define OV8865_EXTCLK_RATE 24000000
-
/* Register definitions */
/* System */
@@ -146,6 +143,7 @@
#define OV8865_EXPOSURE_CTRL_L_REG 0x3502
#define OV8865_EXPOSURE_CTRL_L(v) ((v) & GENMASK(7, 0))
#define OV8865_EXPOSURE_GAIN_MANUAL_REG 0x3503
+#define OV8865_INTEGRATION_TIME_MARGIN 8
#define OV8865_GAIN_CTRL_H_REG 0x3508
#define OV8865_GAIN_CTRL_H(v) (((v) & GENMASK(12, 8)) >> 8)
@@ -186,6 +184,8 @@
#define OV8865_VTS_H(v) (((v) & GENMASK(11, 8)) >> 8)
#define OV8865_VTS_L_REG 0x380f
#define OV8865_VTS_L(v) ((v) & GENMASK(7, 0))
+#define OV8865_TIMING_MAX_VTS 0xffff
+#define OV8865_TIMING_MIN_VTS 0x04
#define OV8865_OFFSET_X_H_REG 0x3810
#define OV8865_OFFSET_X_H(v) (((v) & GENMASK(15, 8)) >> 8)
#define OV8865_OFFSET_X_L_REG 0x3811
@@ -453,6 +453,15 @@
#define OV8865_PRE_CTRL0_PATTERN_COLOR_SQUARES 2
#define OV8865_PRE_CTRL0_PATTERN_BLACK 3
+/* Pixel Array */
+
+#define OV8865_NATIVE_WIDTH 3296
+#define OV8865_NATIVE_HEIGHT 2528
+#define OV8865_ACTIVE_START_TOP 32
+#define OV8865_ACTIVE_START_LEFT 80
+#define OV8865_ACTIVE_WIDTH 3264
+#define OV8865_ACTIVE_HEIGHT 2448
+
/* Macros */
#define ov8865_subdev_sensor(s) \
@@ -566,6 +575,25 @@ struct ov8865_sclk_config {
unsigned int sclk_div;
};
+struct ov8865_pll_configs {
+ const struct ov8865_pll1_config *pll1_config;
+ const struct ov8865_pll2_config *pll2_config_native;
+ const struct ov8865_pll2_config *pll2_config_binning;
+};
+
+/* Clock rate */
+
+enum extclk_rate {
+ OV8865_19_2_MHZ,
+ OV8865_24_MHZ,
+ OV8865_NUM_SUPPORTED_RATES
+};
+
+static const unsigned long supported_extclk_rates[] = {
+ [OV8865_19_2_MHZ] = 19200000,
+ [OV8865_24_MHZ] = 24000000,
+};
+
/*
* General formulas for (array-centered) mode calculation:
* - photo_array_width = 3296
@@ -632,11 +660,7 @@ struct ov8865_mode {
unsigned int blc_anchor_right_start;
unsigned int blc_anchor_right_end;
- struct v4l2_fract frame_interval;
-
- const struct ov8865_pll1_config *pll1_config;
- const struct ov8865_pll2_config *pll2_config;
- const struct ov8865_sclk_config *sclk_config;
+ bool pll2_binning;
const struct ov8865_register_value *register_values;
unsigned int register_values_count;
@@ -652,6 +676,9 @@ struct ov8865_state {
struct ov8865_ctrls {
struct v4l2_ctrl *link_freq;
struct v4l2_ctrl *pixel_rate;
+ struct v4l2_ctrl *hblank;
+ struct v4l2_ctrl *vblank;
+ struct v4l2_ctrl *exposure;
struct v4l2_ctrl_handler handler;
};
@@ -664,6 +691,9 @@ struct ov8865_sensor {
struct regulator *avdd;
struct regulator *dvdd;
struct regulator *dovdd;
+
+ unsigned long extclk_rate;
+ const struct ov8865_pll_configs *pll_configs;
struct clk *extclk;
struct v4l2_fwnode_endpoint endpoint;
@@ -679,43 +709,70 @@ struct ov8865_sensor {
/* Static definitions */
/*
- * EXTCLK = 24 MHz
* PHY_SCLK = 720 MHz
* MIPI_PCLK = 90 MHz
*/
-static const struct ov8865_pll1_config ov8865_pll1_config_native = {
- .pll_pre_div_half = 1,
- .pll_pre_div = 0,
- .pll_mul = 30,
- .m_div = 1,
- .mipi_div = 3,
- .pclk_div = 1,
- .sys_pre_div = 1,
- .sys_div = 2,
+
+static const struct ov8865_pll1_config ov8865_pll1_config_native_19_2mhz = {
+ .pll_pre_div_half = 1,
+ .pll_pre_div = 2,
+ .pll_mul = 75,
+ .m_div = 1,
+ .mipi_div = 3,
+ .pclk_div = 1,
+ .sys_pre_div = 1,
+ .sys_div = 2,
+};
+
+static const struct ov8865_pll1_config ov8865_pll1_config_native_24mhz = {
+ .pll_pre_div_half = 1,
+ .pll_pre_div = 0,
+ .pll_mul = 30,
+ .m_div = 1,
+ .mipi_div = 3,
+ .pclk_div = 1,
+ .sys_pre_div = 1,
+ .sys_div = 2,
};
/*
- * EXTCLK = 24 MHz
* DAC_CLK = 360 MHz
* SCLK = 144 MHz
*/
-static const struct ov8865_pll2_config ov8865_pll2_config_native = {
- .pll_pre_div_half = 1,
- .pll_pre_div = 0,
- .pll_mul = 30,
- .dac_div = 2,
- .sys_pre_div = 5,
- .sys_div = 0,
+static const struct ov8865_pll2_config ov8865_pll2_config_native_19_2mhz = {
+ .pll_pre_div_half = 1,
+ .pll_pre_div = 5,
+ .pll_mul = 75,
+ .dac_div = 1,
+ .sys_pre_div = 1,
+ .sys_div = 3,
+};
+
+static const struct ov8865_pll2_config ov8865_pll2_config_native_24mhz = {
+ .pll_pre_div_half = 1,
+ .pll_pre_div = 0,
+ .pll_mul = 30,
+ .dac_div = 2,
+ .sys_pre_div = 5,
+ .sys_div = 0,
};
/*
- * EXTCLK = 24 MHz
* DAC_CLK = 360 MHz
- * SCLK = 80 MHz
+ * SCLK = 72 MHz
*/
-static const struct ov8865_pll2_config ov8865_pll2_config_binning = {
+static const struct ov8865_pll2_config ov8865_pll2_config_binning_19_2mhz = {
+ .pll_pre_div_half = 1,
+ .pll_pre_div = 2,
+ .pll_mul = 75,
+ .dac_div = 2,
+ .sys_pre_div = 10,
+ .sys_div = 0,
+};
+
+static const struct ov8865_pll2_config ov8865_pll2_config_binning_24mhz = {
.pll_pre_div_half = 1,
.pll_pre_div = 0,
.pll_mul = 30,
@@ -724,6 +781,23 @@ static const struct ov8865_pll2_config ov8865_pll2_config_binning = {
.sys_div = 0,
};
+static const struct ov8865_pll_configs ov8865_pll_configs_19_2mhz = {
+ .pll1_config = &ov8865_pll1_config_native_19_2mhz,
+ .pll2_config_native = &ov8865_pll2_config_native_19_2mhz,
+ .pll2_config_binning = &ov8865_pll2_config_binning_19_2mhz,
+};
+
+static const struct ov8865_pll_configs ov8865_pll_configs_24mhz = {
+ .pll1_config = &ov8865_pll1_config_native_24mhz,
+ .pll2_config_native = &ov8865_pll2_config_native_24mhz,
+ .pll2_config_binning = &ov8865_pll2_config_binning_24mhz,
+};
+
+static const struct ov8865_pll_configs *ov8865_pll_configs[] = {
+ &ov8865_pll_configs_19_2mhz,
+ &ov8865_pll_configs_24mhz,
+};
+
static const struct ov8865_sclk_config ov8865_sclk_config_native = {
.sys_sel = 1,
.sclk_sel = 0,
@@ -890,7 +964,7 @@ static const struct ov8865_mode ov8865_modes[] = {
{
/* Horizontal */
.output_size_x = 3264,
- .hts = 1944,
+ .hts = 3888,
/* Vertical */
.output_size_y = 2448,
@@ -929,13 +1003,8 @@ static const struct ov8865_mode ov8865_modes[] = {
.blc_anchor_right_start = 1984,
.blc_anchor_right_end = 2239,
- /* Frame Interval */
- .frame_interval = { 1, 30 },
-
/* PLL */
- .pll1_config = &ov8865_pll1_config_native,
- .pll2_config = &ov8865_pll2_config_native,
- .sclk_config = &ov8865_sclk_config_native,
+ .pll2_binning = false,
/* Registers */
.register_values = ov8865_register_values_native,
@@ -946,11 +1015,11 @@ static const struct ov8865_mode ov8865_modes[] = {
{
/* Horizontal */
.output_size_x = 3264,
- .hts = 2582,
+ .hts = 3888,
/* Vertical */
.output_size_y = 1836,
- .vts = 2002,
+ .vts = 2470,
.size_auto = true,
.size_auto_boundary_x = 8,
@@ -985,13 +1054,8 @@ static const struct ov8865_mode ov8865_modes[] = {
.blc_anchor_right_start = 1984,
.blc_anchor_right_end = 2239,
- /* Frame Interval */
- .frame_interval = { 1, 30 },
-
/* PLL */
- .pll1_config = &ov8865_pll1_config_native,
- .pll2_config = &ov8865_pll2_config_native,
- .sclk_config = &ov8865_sclk_config_native,
+ .pll2_binning = false,
/* Registers */
.register_values = ov8865_register_values_native,
@@ -1045,13 +1109,8 @@ static const struct ov8865_mode ov8865_modes[] = {
.blc_anchor_right_start = 992,
.blc_anchor_right_end = 1119,
- /* Frame Interval */
- .frame_interval = { 1, 30 },
-
/* PLL */
- .pll1_config = &ov8865_pll1_config_native,
- .pll2_config = &ov8865_pll2_config_binning,
- .sclk_config = &ov8865_sclk_config_native,
+ .pll2_binning = true,
/* Registers */
.register_values = ov8865_register_values_binning,
@@ -1111,13 +1170,8 @@ static const struct ov8865_mode ov8865_modes[] = {
.blc_anchor_right_start = 992,
.blc_anchor_right_end = 1119,
- /* Frame Interval */
- .frame_interval = { 1, 90 },
-
/* PLL */
- .pll1_config = &ov8865_pll1_config_native,
- .pll2_config = &ov8865_pll2_config_binning,
- .sclk_config = &ov8865_sclk_config_native,
+ .pll2_binning = true,
/* Registers */
.register_values = ov8865_register_values_binning,
@@ -1512,12 +1566,11 @@ static int ov8865_isp_configure(struct ov8865_sensor *sensor)
static unsigned long ov8865_mode_pll1_rate(struct ov8865_sensor *sensor,
const struct ov8865_mode *mode)
{
- const struct ov8865_pll1_config *config = mode->pll1_config;
- unsigned long extclk_rate;
+ const struct ov8865_pll1_config *config;
unsigned long pll1_rate;
- extclk_rate = clk_get_rate(sensor->extclk);
- pll1_rate = extclk_rate * config->pll_mul / config->pll_pre_div_half;
+ config = sensor->pll_configs->pll1_config;
+ pll1_rate = sensor->extclk_rate * config->pll_mul / config->pll_pre_div_half;
switch (config->pll_pre_div) {
case 0:
@@ -1551,10 +1604,12 @@ static int ov8865_mode_pll1_configure(struct ov8865_sensor *sensor,
const struct ov8865_mode *mode,
u32 mbus_code)
{
- const struct ov8865_pll1_config *config = mode->pll1_config;
+ const struct ov8865_pll1_config *config;
u8 value;
int ret;
+ config = sensor->pll_configs->pll1_config;
+
switch (mbus_code) {
case MEDIA_BUS_FMT_SBGGR10_1X10:
value = OV8865_MIPI_BIT_SEL(10);
@@ -1621,9 +1676,12 @@ static int ov8865_mode_pll1_configure(struct ov8865_sensor *sensor,
static int ov8865_mode_pll2_configure(struct ov8865_sensor *sensor,
const struct ov8865_mode *mode)
{
- const struct ov8865_pll2_config *config = mode->pll2_config;
+ const struct ov8865_pll2_config *config;
int ret;
+ config = mode->pll2_binning ? sensor->pll_configs->pll2_config_binning :
+ sensor->pll_configs->pll2_config_native;
+
ret = ov8865_write(sensor, OV8865_PLL_CTRL12_REG,
OV8865_PLL_CTRL12_PRE_DIV_HALF(config->pll_pre_div_half) |
OV8865_PLL_CTRL12_DAC_DIV(config->dac_div));
@@ -1657,7 +1715,7 @@ static int ov8865_mode_pll2_configure(struct ov8865_sensor *sensor,
static int ov8865_mode_sclk_configure(struct ov8865_sensor *sensor,
const struct ov8865_mode *mode)
{
- const struct ov8865_sclk_config *config = mode->sclk_config;
+ const struct ov8865_sclk_config *config = &ov8865_sclk_config_native;
int ret;
ret = ov8865_write(sensor, OV8865_CLK_SEL0_REG,
@@ -2052,9 +2110,11 @@ static int ov8865_mode_configure(struct ov8865_sensor *sensor,
static unsigned long ov8865_mode_mipi_clk_rate(struct ov8865_sensor *sensor,
const struct ov8865_mode *mode)
{
- const struct ov8865_pll1_config *config = mode->pll1_config;
+ const struct ov8865_pll1_config *config;
unsigned long pll1_rate;
+ config = sensor->pll_configs->pll1_config;
+
pll1_rate = ov8865_mode_pll1_rate(sensor, mode);
return pll1_rate / config->m_div / 2;
@@ -2066,6 +2126,9 @@ static int ov8865_exposure_configure(struct ov8865_sensor *sensor, u32 exposure)
{
int ret;
+ /* The sensor stores exposure in units of 1/16th of a line */
+ exposure *= 16;
+
ret = ov8865_write(sensor, OV8865_EXPOSURE_CTRL_HH_REG,
OV8865_EXPOSURE_CTRL_HH(exposure));
if (ret)
@@ -2082,7 +2145,7 @@ static int ov8865_exposure_configure(struct ov8865_sensor *sensor, u32 exposure)
/* Gain */
-static int ov8865_gain_configure(struct ov8865_sensor *sensor, u32 gain)
+static int ov8865_analog_gain_configure(struct ov8865_sensor *sensor, u32 gain)
{
int ret;
@@ -2157,6 +2220,20 @@ static int ov8865_test_pattern_configure(struct ov8865_sensor *sensor,
ov8865_test_pattern_bits[index]);
}
+/* Blanking */
+
+static int ov8865_vts_configure(struct ov8865_sensor *sensor, u32 vblank)
+{
+ u16 vts = sensor->state.mode->output_size_y + vblank;
+ int ret;
+
+ ret = ov8865_write(sensor, OV8865_VTS_H_REG, OV8865_VTS_H(vts));
+ if (ret)
+ return ret;
+
+ return ov8865_write(sensor, OV8865_VTS_L_REG, OV8865_VTS_L(vts));
+}
+
/* State */
static int ov8865_state_mipi_configure(struct ov8865_sensor *sensor,
@@ -2330,27 +2407,27 @@ static int ov8865_sensor_power(struct ov8865_sensor *sensor, bool on)
if (ret) {
dev_err(sensor->dev,
"failed to enable DOVDD regulator\n");
- goto disable;
+ return ret;
}
ret = regulator_enable(sensor->avdd);
if (ret) {
dev_err(sensor->dev,
"failed to enable AVDD regulator\n");
- goto disable;
+ goto disable_dovdd;
}
ret = regulator_enable(sensor->dvdd);
if (ret) {
dev_err(sensor->dev,
"failed to enable DVDD regulator\n");
- goto disable;
+ goto disable_avdd;
}
ret = clk_prepare_enable(sensor->extclk);
if (ret) {
dev_err(sensor->dev, "failed to enable EXTCLK clock\n");
- goto disable;
+ goto disable_dvdd;
}
gpiod_set_value_cansleep(sensor->reset, 0);
@@ -2359,14 +2436,16 @@ static int ov8865_sensor_power(struct ov8865_sensor *sensor, bool on)
/* Time to enter streaming mode according to power timings. */
usleep_range(10000, 12000);
} else {
-disable:
gpiod_set_value_cansleep(sensor->powerdown, 1);
gpiod_set_value_cansleep(sensor->reset, 1);
clk_disable_unprepare(sensor->extclk);
+disable_dvdd:
regulator_disable(sensor->dvdd);
+disable_avdd:
regulator_disable(sensor->avdd);
+disable_dovdd:
regulator_disable(sensor->dovdd);
}
@@ -2382,6 +2461,20 @@ static int ov8865_s_ctrl(struct v4l2_ctrl *ctrl)
unsigned int index;
int ret;
+ /* If VBLANK is altered we need to update exposure to compensate */
+ if (ctrl->id == V4L2_CID_VBLANK) {
+ int exposure_max;
+
+ exposure_max = sensor->state.mode->output_size_y + ctrl->val -
+ OV8865_INTEGRATION_TIME_MARGIN;
+ __v4l2_ctrl_modify_range(sensor->ctrls.exposure,
+ sensor->ctrls.exposure->minimum,
+ exposure_max,
+ sensor->ctrls.exposure->step,
+ min(sensor->ctrls.exposure->val,
+ exposure_max));
+ }
+
/* Wait for the sensor to be on before setting controls. */
if (pm_runtime_suspended(sensor->dev))
return 0;
@@ -2392,8 +2485,8 @@ static int ov8865_s_ctrl(struct v4l2_ctrl *ctrl)
if (ret)
return ret;
break;
- case V4L2_CID_GAIN:
- ret = ov8865_gain_configure(sensor, ctrl->val);
+ case V4L2_CID_ANALOGUE_GAIN:
+ ret = ov8865_analog_gain_configure(sensor, ctrl->val);
if (ret)
return ret;
break;
@@ -2408,6 +2501,8 @@ static int ov8865_s_ctrl(struct v4l2_ctrl *ctrl)
case V4L2_CID_TEST_PATTERN:
index = (unsigned int)ctrl->val;
return ov8865_test_pattern_configure(sensor, index);
+ case V4L2_CID_VBLANK:
+ return ov8865_vts_configure(sensor, ctrl->val);
default:
return -EINVAL;
}
@@ -2424,6 +2519,10 @@ static int ov8865_ctrls_init(struct ov8865_sensor *sensor)
struct ov8865_ctrls *ctrls = &sensor->ctrls;
struct v4l2_ctrl_handler *handler = &ctrls->handler;
const struct v4l2_ctrl_ops *ops = &ov8865_ctrl_ops;
+ const struct ov8865_mode *mode = &ov8865_modes[0];
+ struct v4l2_fwnode_device_properties props;
+ unsigned int vblank_max, vblank_def;
+ unsigned int hblank;
int ret;
v4l2_ctrl_handler_init(handler, 32);
@@ -2433,12 +2532,13 @@ static int ov8865_ctrls_init(struct ov8865_sensor *sensor)
/* Exposure */
- v4l2_ctrl_new_std(handler, ops, V4L2_CID_EXPOSURE, 16, 1048575, 16,
- 512);
+ ctrls->exposure = v4l2_ctrl_new_std(handler, ops, V4L2_CID_EXPOSURE, 2,
+ 65535, 1, 32);
/* Gain */
- v4l2_ctrl_new_std(handler, ops, V4L2_CID_GAIN, 128, 8191, 128, 128);
+ v4l2_ctrl_new_std(handler, ops, V4L2_CID_ANALOGUE_GAIN, 128, 2048, 128,
+ 128);
/* White Balance */
@@ -2459,6 +2559,20 @@ static int ov8865_ctrls_init(struct ov8865_sensor *sensor)
ARRAY_SIZE(ov8865_test_pattern_menu) - 1,
0, 0, ov8865_test_pattern_menu);
+ /* Blanking */
+ hblank = mode->hts - mode->output_size_x;
+ ctrls->hblank = v4l2_ctrl_new_std(handler, ops, V4L2_CID_HBLANK, hblank,
+ hblank, 1, hblank);
+
+ if (ctrls->hblank)
+ ctrls->hblank->flags |= V4L2_CTRL_FLAG_READ_ONLY;
+
+ vblank_max = OV8865_TIMING_MAX_VTS - mode->output_size_y;
+ vblank_def = mode->vts - mode->output_size_y;
+ ctrls->vblank = v4l2_ctrl_new_std(handler, ops, V4L2_CID_VBLANK,
+ OV8865_TIMING_MIN_VTS, vblank_max, 1,
+ vblank_def);
+
/* MIPI CSI-2 */
ctrls->link_freq =
@@ -2470,6 +2584,15 @@ static int ov8865_ctrls_init(struct ov8865_sensor *sensor)
v4l2_ctrl_new_std(handler, NULL, V4L2_CID_PIXEL_RATE, 1,
INT_MAX, 1, 1);
+ /* set properties from fwnode (e.g. rotation, orientation) */
+ ret = v4l2_fwnode_device_parse(sensor->dev, &props);
+ if (ret)
+ goto error_ctrls;
+
+ ret = v4l2_ctrl_new_fwnode_properties(handler, ops, &props);
+ if (ret)
+ goto error_ctrls;
+
if (handler->error) {
ret = handler->error;
goto error_ctrls;
@@ -2522,11 +2645,18 @@ static int ov8865_g_frame_interval(struct v4l2_subdev *subdev,
{
struct ov8865_sensor *sensor = ov8865_subdev_sensor(subdev);
const struct ov8865_mode *mode;
+ unsigned int framesize;
+ unsigned int fps;
mutex_lock(&sensor->mutex);
mode = sensor->state.mode;
- interval->interval = mode->frame_interval;
+ framesize = mode->hts * (mode->output_size_y +
+ sensor->ctrls.vblank->val);
+ fps = DIV_ROUND_CLOSEST(sensor->ctrls.pixel_rate->val, framesize);
+
+ interval->interval.numerator = 1;
+ interval->interval.denominator = fps;
mutex_unlock(&sensor->mutex);
@@ -2599,7 +2729,9 @@ static int ov8865_set_fmt(struct v4l2_subdev *subdev,
struct v4l2_mbus_framefmt *mbus_format = &format->format;
const struct ov8865_mode *mode;
u32 mbus_code = 0;
+ unsigned int hblank;
unsigned int index;
+ int exposure_max;
int ret = 0;
mutex_lock(&sensor->mutex);
@@ -2639,6 +2771,21 @@ static int ov8865_set_fmt(struct v4l2_subdev *subdev,
sensor->state.mbus_code != mbus_code)
ret = ov8865_state_configure(sensor, mode, mbus_code);
+ __v4l2_ctrl_modify_range(sensor->ctrls.vblank, OV8865_TIMING_MIN_VTS,
+ OV8865_TIMING_MAX_VTS - mode->output_size_y,
+ 1, mode->vts - mode->output_size_y);
+
+ hblank = mode->hts - mode->output_size_x;
+ __v4l2_ctrl_modify_range(sensor->ctrls.hblank, hblank, hblank, 1,
+ hblank);
+
+ exposure_max = mode->vts - OV8865_INTEGRATION_TIME_MARGIN;
+ __v4l2_ctrl_modify_range(sensor->ctrls.exposure,
+ sensor->ctrls.exposure->minimum, exposure_max,
+ sensor->ctrls.exposure->step,
+ min(sensor->ctrls.exposure->val,
+ exposure_max));
+
complete:
mutex_unlock(&sensor->mutex);
@@ -2662,37 +2809,55 @@ static int ov8865_enum_frame_size(struct v4l2_subdev *subdev,
return 0;
}
-static int ov8865_enum_frame_interval(struct v4l2_subdev *subdev,
- struct v4l2_subdev_state *sd_state,
- struct v4l2_subdev_frame_interval_enum *interval_enum)
+static void
+__ov8865_get_pad_crop(struct ov8865_sensor *sensor,
+ struct v4l2_subdev_state *state, unsigned int pad,
+ enum v4l2_subdev_format_whence which, struct v4l2_rect *r)
{
- const struct ov8865_mode *mode = NULL;
- unsigned int mode_index;
- unsigned int interval_index;
+ const struct ov8865_mode *mode = sensor->state.mode;
- if (interval_enum->index > 0)
- return -EINVAL;
- /*
- * Multiple modes with the same dimensions may have different frame
- * intervals, so look up each relevant mode.
- */
- for (mode_index = 0, interval_index = 0;
- mode_index < ARRAY_SIZE(ov8865_modes); mode_index++) {
- mode = &ov8865_modes[mode_index];
-
- if (mode->output_size_x == interval_enum->width &&
- mode->output_size_y == interval_enum->height) {
- if (interval_index == interval_enum->index)
- break;
-
- interval_index++;
- }
+ switch (which) {
+ case V4L2_SUBDEV_FORMAT_TRY:
+ *r = *v4l2_subdev_get_try_crop(&sensor->subdev, state, pad);
+ break;
+ case V4L2_SUBDEV_FORMAT_ACTIVE:
+ r->height = mode->output_size_y;
+ r->width = mode->output_size_x;
+ r->top = (OV8865_NATIVE_HEIGHT - mode->output_size_y) / 2;
+ r->left = (OV8865_NATIVE_WIDTH - mode->output_size_x) / 2;
+ break;
}
+}
- if (mode_index == ARRAY_SIZE(ov8865_modes))
+static int ov8865_get_selection(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_selection *sel)
+{
+ struct ov8865_sensor *sensor = ov8865_subdev_sensor(subdev);
+
+ switch (sel->target) {
+ case V4L2_SEL_TGT_CROP:
+ mutex_lock(&sensor->mutex);
+ __ov8865_get_pad_crop(sensor, state, sel->pad,
+ sel->which, &sel->r);
+ mutex_unlock(&sensor->mutex);
+ break;
+ case V4L2_SEL_TGT_NATIVE_SIZE:
+ sel->r.top = 0;
+ sel->r.left = 0;
+ sel->r.width = OV8865_NATIVE_WIDTH;
+ sel->r.height = OV8865_NATIVE_HEIGHT;
+ break;
+ case V4L2_SEL_TGT_CROP_BOUNDS:
+ case V4L2_SEL_TGT_CROP_DEFAULT:
+ sel->r.top = OV8865_ACTIVE_START_TOP;
+ sel->r.left = OV8865_ACTIVE_START_LEFT;
+ sel->r.width = OV8865_ACTIVE_WIDTH;
+ sel->r.height = OV8865_ACTIVE_HEIGHT;
+ break;
+ default:
return -EINVAL;
-
- interval_enum->interval = mode->frame_interval;
+ }
return 0;
}
@@ -2702,7 +2867,8 @@ static const struct v4l2_subdev_pad_ops ov8865_subdev_pad_ops = {
.get_fmt = ov8865_get_fmt,
.set_fmt = ov8865_set_fmt,
.enum_frame_size = ov8865_enum_frame_size,
- .enum_frame_interval = ov8865_enum_frame_interval,
+ .get_selection = ov8865_get_selection,
+ .set_selection = ov8865_get_selection,
};
static const struct v4l2_subdev_ops ov8865_subdev_ops = {
@@ -2782,7 +2948,8 @@ static int ov8865_probe(struct i2c_client *client)
struct ov8865_sensor *sensor;
struct v4l2_subdev *subdev;
struct media_pad *pad;
- unsigned long rate;
+ unsigned int rate = 0;
+ unsigned int i;
int ret;
sensor = devm_kzalloc(dev, sizeof(*sensor), GFP_KERNEL);
@@ -2792,13 +2959,31 @@ static int ov8865_probe(struct i2c_client *client)
sensor->dev = dev;
sensor->i2c_client = client;
+ /* Regulators */
+
+ /* DVDD: digital core */
+ sensor->dvdd = devm_regulator_get(dev, "dvdd");
+ if (IS_ERR(sensor->dvdd))
+ return dev_err_probe(dev, PTR_ERR(sensor->dvdd),
+ "cannot get DVDD regulator\n");
+
+ /* DOVDD: digital I/O */
+ sensor->dovdd = devm_regulator_get(dev, "dovdd");
+ if (IS_ERR(sensor->dovdd))
+ return dev_err_probe(dev, PTR_ERR(sensor->dovdd),
+ "cannot get DOVDD regulator\n");
+
+ /* AVDD: analog */
+ sensor->avdd = devm_regulator_get(dev, "avdd");
+ if (IS_ERR(sensor->avdd))
+ return dev_err_probe(dev, PTR_ERR(sensor->avdd),
+ "cannot get AVDD (analog) regulator\n");
+
/* Graph Endpoint */
handle = fwnode_graph_get_next_endpoint(dev_fwnode(dev), NULL);
- if (!handle) {
- dev_err(dev, "unable to find endpoint node\n");
- return -EINVAL;
- }
+ if (!handle)
+ return -EPROBE_DEFER;
sensor->endpoint.bus_type = V4L2_MBUS_CSI2_DPHY;
@@ -2824,48 +3009,54 @@ static int ov8865_probe(struct i2c_client *client)
goto error_endpoint;
}
- /* Regulators */
-
- /* DVDD: digital core */
- sensor->dvdd = devm_regulator_get(dev, "dvdd");
- if (IS_ERR(sensor->dvdd)) {
- dev_err(dev, "cannot get DVDD (digital core) regulator\n");
- ret = PTR_ERR(sensor->dvdd);
- goto error_endpoint;
- }
-
- /* DOVDD: digital I/O */
- sensor->dovdd = devm_regulator_get(dev, "dovdd");
- if (IS_ERR(sensor->dovdd)) {
- dev_err(dev, "cannot get DOVDD (digital I/O) regulator\n");
- ret = PTR_ERR(sensor->dovdd);
- goto error_endpoint;
- }
-
- /* AVDD: analog */
- sensor->avdd = devm_regulator_get(dev, "avdd");
- if (IS_ERR(sensor->avdd)) {
- dev_err(dev, "cannot get AVDD (analog) regulator\n");
- ret = PTR_ERR(sensor->avdd);
- goto error_endpoint;
- }
-
/* External Clock */
sensor->extclk = devm_clk_get(dev, NULL);
- if (IS_ERR(sensor->extclk)) {
+ if (PTR_ERR(sensor->extclk) == -ENOENT) {
+ dev_info(dev, "no external clock found, continuing...\n");
+ sensor->extclk = NULL;
+ } else if (IS_ERR(sensor->extclk)) {
dev_err(dev, "failed to get external clock\n");
ret = PTR_ERR(sensor->extclk);
goto error_endpoint;
}
- rate = clk_get_rate(sensor->extclk);
- if (rate != OV8865_EXTCLK_RATE) {
- dev_err(dev, "clock rate %lu Hz is unsupported\n", rate);
+ /*
+ * We could have either a 24MHz or 19.2MHz clock rate from either dt or
+ * ACPI...but we also need to support the weird IPU3 case which will
+ * have an external clock AND a clock-frequency property. Check for the
+ * clock-frequency property and if found, set that rate if we managed
+ * to acquire a clock. This should cover the ACPI case. If the system
+ * uses devicetree then the configured rate should already be set, so
+ * we can just read it.
+ */
+ ret = fwnode_property_read_u32(dev_fwnode(dev), "clock-frequency",
+ &rate);
+ if (!ret && sensor->extclk) {
+ ret = clk_set_rate(sensor->extclk, rate);
+ if (ret)
+ return dev_err_probe(dev, ret,
+ "failed to set clock rate\n");
+ } else if (ret && !sensor->extclk) {
+ return dev_err_probe(dev, ret, "invalid clock config\n");
+ }
+
+ sensor->extclk_rate = rate ? rate : clk_get_rate(sensor->extclk);
+
+ for (i = 0; i < ARRAY_SIZE(supported_extclk_rates); i++) {
+ if (sensor->extclk_rate == supported_extclk_rates[i])
+ break;
+ }
+
+ if (i == ARRAY_SIZE(supported_extclk_rates)) {
+ dev_err(dev, "clock rate %lu Hz is unsupported\n",
+ sensor->extclk_rate);
ret = -EINVAL;
goto error_endpoint;
}
+ sensor->pll_configs = ov8865_pll_configs[i];
+
/* Subdev, entity and pad */
subdev = &sensor->subdev;
@@ -2891,14 +3082,16 @@ static int ov8865_probe(struct i2c_client *client)
if (ret)
goto error_mutex;
+ mutex_lock(&sensor->mutex);
ret = ov8865_state_init(sensor);
+ mutex_unlock(&sensor->mutex);
if (ret)
goto error_ctrls;
/* Runtime PM */
- pm_runtime_enable(sensor->dev);
pm_runtime_set_suspended(sensor->dev);
+ pm_runtime_enable(sensor->dev);
/* V4L2 subdev register */
@@ -2946,6 +3139,12 @@ static const struct dev_pm_ops ov8865_pm_ops = {
SET_RUNTIME_PM_OPS(ov8865_suspend, ov8865_resume, NULL)
};
+static const struct acpi_device_id ov8865_acpi_match[] = {
+ {"INT347A"},
+ { }
+};
+MODULE_DEVICE_TABLE(acpi, ov8865_acpi_match);
+
static const struct of_device_id ov8865_of_match[] = {
{ .compatible = "ovti,ov8865" },
{ }
@@ -2956,6 +3155,7 @@ static struct i2c_driver ov8865_driver = {
.driver = {
.name = "ov8865",
.of_match_table = ov8865_of_match,
+ .acpi_match_table = ov8865_acpi_match,
.pm = &ov8865_pm_ops,
},
.probe_new = ov8865_probe,
diff --git a/drivers/media/i2c/s5c73m3/s5c73m3-spi.c b/drivers/media/i2c/s5c73m3/s5c73m3-spi.c
index c102c6b..7fe6118 100644
--- a/drivers/media/i2c/s5c73m3/s5c73m3-spi.c
+++ b/drivers/media/i2c/s5c73m3/s5c73m3-spi.c
@@ -130,16 +130,10 @@ static int s5c73m3_spi_probe(struct spi_device *spi)
return 0;
}
-static int s5c73m3_spi_remove(struct spi_device *spi)
-{
- return 0;
-}
-
int s5c73m3_register_spi_driver(struct s5c73m3 *state)
{
struct spi_driver *spidrv = &state->spidrv;
- spidrv->remove = s5c73m3_spi_remove;
spidrv->probe = s5c73m3_spi_probe;
spidrv->driver.name = S5C73M3_SPI_DRV_NAME;
spidrv->driver.of_match_table = s5c73m3_spi_ids;
diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
index f40f419..b411f979 100644
--- a/drivers/media/mc/mc-entity.c
+++ b/drivers/media/mc/mc-entity.c
@@ -14,22 +14,6 @@
#include <media/media-entity.h>
#include <media/media-device.h>
-static inline const char *gobj_type(enum media_gobj_type type)
-{
- switch (type) {
- case MEDIA_GRAPH_ENTITY:
- return "entity";
- case MEDIA_GRAPH_PAD:
- return "pad";
- case MEDIA_GRAPH_LINK:
- return "link";
- case MEDIA_GRAPH_INTF_DEVNODE:
- return "intf-devnode";
- default:
- return "unknown";
- }
-}
-
static inline const char *intf_type(struct media_interface *intf)
{
switch (intf->type) {
@@ -64,12 +48,10 @@ __must_check int __media_entity_enum_init(struct media_entity_enum *ent_enum,
int idx_max)
{
idx_max = ALIGN(idx_max, BITS_PER_LONG);
- ent_enum->bmap = kcalloc(idx_max / BITS_PER_LONG, sizeof(long),
- GFP_KERNEL);
+ ent_enum->bmap = bitmap_zalloc(idx_max, GFP_KERNEL);
if (!ent_enum->bmap)
return -ENOMEM;
- bitmap_zero(ent_enum->bmap, idx_max);
ent_enum->idx_max = idx_max;
return 0;
@@ -78,7 +60,7 @@ EXPORT_SYMBOL_GPL(__media_entity_enum_init);
void media_entity_enum_cleanup(struct media_entity_enum *ent_enum)
{
- kfree(ent_enum->bmap);
+ bitmap_free(ent_enum->bmap);
}
EXPORT_SYMBOL_GPL(media_entity_enum_cleanup);
diff --git a/drivers/media/pci/b2c2/flexcop-pci.c b/drivers/media/pci/b2c2/flexcop-pci.c
index 6a4c7cb..486c8ec 100644
--- a/drivers/media/pci/b2c2/flexcop-pci.c
+++ b/drivers/media/pci/b2c2/flexcop-pci.c
@@ -185,6 +185,8 @@ static irqreturn_t flexcop_pci_isr(int irq, void *dev_id)
dma_addr_t cur_addr =
fc->read_ibi_reg(fc,dma1_008).dma_0x8.dma_cur_addr << 2;
u32 cur_pos = cur_addr - fc_pci->dma[0].dma_addr0;
+ if (cur_pos > fc_pci->dma[0].size * 2)
+ goto error;
deb_irq("%u irq: %08x cur_addr: %llx: cur_pos: %08x, last_cur_pos: %08x ",
jiffies_to_usecs(jiffies - fc_pci->last_irq),
@@ -225,6 +227,7 @@ static irqreturn_t flexcop_pci_isr(int irq, void *dev_id)
ret = IRQ_NONE;
}
+error:
spin_unlock_irqrestore(&fc_pci->irq_lock, flags);
return ret;
}
diff --git a/drivers/media/pci/bt8xx/bttv-driver.c b/drivers/media/pci/bt8xx/bttv-driver.c
index 0e9df8b..8cc9bec 100644
--- a/drivers/media/pci/bt8xx/bttv-driver.c
+++ b/drivers/media/pci/bt8xx/bttv-driver.c
@@ -170,14 +170,14 @@ MODULE_VERSION(BTTV_VERSION);
/* ----------------------------------------------------------------------- */
/* sysfs */
-static ssize_t show_card(struct device *cd,
+static ssize_t card_show(struct device *cd,
struct device_attribute *attr, char *buf)
{
struct video_device *vfd = to_video_device(cd);
struct bttv *btv = video_get_drvdata(vfd);
return sprintf(buf, "%d\n", btv ? btv->c.type : UNSET);
}
-static DEVICE_ATTR(card, S_IRUGO, show_card, NULL);
+static DEVICE_ATTR_RO(card);
/* ----------------------------------------------------------------------- */
/* dvb auto-load setup */
diff --git a/drivers/media/pci/cobalt/cobalt-cpld.c b/drivers/media/pci/cobalt/cobalt-cpld.c
index 3d80264..fad8824 100644
--- a/drivers/media/pci/cobalt/cobalt-cpld.c
+++ b/drivers/media/pci/cobalt/cobalt-cpld.c
@@ -236,7 +236,6 @@ bool cobalt_cpld_set_freq(struct cobalt *cobalt, unsigned f_out)
u8 n1, hsdiv;
u8 regs[6];
int found = 0;
- u16 clock_ctrl;
int retries = 3;
for (i = 0; i < ARRAY_SIZE(multipliers); i++) {
@@ -260,9 +259,7 @@ bool cobalt_cpld_set_freq(struct cobalt *cobalt, unsigned f_out)
hsdiv = multipliers[i_best].hsdiv - 4;
rfreq = div_u64(dco << 28, f_xtal);
- clock_ctrl = cpld_read(cobalt, SI570_CLOCK_CTRL);
- clock_ctrl |= S01755_REG_CLOCK_CTRL_BITMAP_CLKHSMA_FPGA_CTRL;
- clock_ctrl |= S01755_REG_CLOCK_CTRL_BITMAP_CLKHSMA_EN;
+ cpld_read(cobalt, SI570_CLOCK_CTRL);
regs[0] = (hsdiv << 5) | (n1 >> 2);
regs[1] = ((n1 & 0x3) << 6) | (rfreq >> 32);
diff --git a/drivers/media/pci/cx18/cx18-alsa-main.c b/drivers/media/pci/cx18/cx18-alsa-main.c
index 9a82e68..9dc3618 100644
--- a/drivers/media/pci/cx18/cx18-alsa-main.c
+++ b/drivers/media/pci/cx18/cx18-alsa-main.c
@@ -51,12 +51,6 @@ struct snd_cx18_card *to_snd_cx18_card(struct v4l2_device *v4l2_dev)
return to_cx18(v4l2_dev)->alsa;
}
-static inline
-struct snd_cx18_card *p_to_snd_cx18_card(struct v4l2_device **v4l2_dev)
-{
- return container_of(v4l2_dev, struct snd_cx18_card, v4l2_dev);
-}
-
static void snd_cx18_card_free(struct snd_cx18_card *cxsc)
{
if (cxsc == NULL)
diff --git a/drivers/media/pci/cx25821/cx25821-core.c b/drivers/media/pci/cx25821/cx25821-core.c
index 40c10ca..3078a39 100644
--- a/drivers/media/pci/cx25821/cx25821-core.c
+++ b/drivers/media/pci/cx25821/cx25821-core.c
@@ -337,13 +337,6 @@ static int cx25821_risc_decode(u32 risc)
return incr[risc >> 28] ? incr[risc >> 28] : 1;
}
-static inline int i2c_slave_did_ack(struct i2c_adapter *i2c_adap)
-{
- struct cx25821_i2c *bus = i2c_adap->algo_data;
- struct cx25821_dev *dev = bus->dev;
- return cx_read(bus->reg_stat) & 0x01;
-}
-
static void cx25821_registers_init(struct cx25821_dev *dev)
{
u32 tmp;
diff --git a/drivers/media/pci/intel/ipu3/cio2-bridge.c b/drivers/media/pci/intel/ipu3/cio2-bridge.c
index 67c467d3..7ccb7b6 100644
--- a/drivers/media/pci/intel/ipu3/cio2-bridge.c
+++ b/drivers/media/pci/intel/ipu3/cio2-bridge.c
@@ -3,6 +3,7 @@
#include <linux/acpi.h>
#include <linux/device.h>
+#include <linux/i2c.h>
#include <linux/pci.h>
#include <linux/property.h>
#include <media/v4l2-fwnode.h>
@@ -21,7 +22,9 @@
*/
static const struct cio2_sensor_config cio2_supported_sensors[] = {
/* Omnivision OV5693 */
- CIO2_SENSOR_CONFIG("INT33BE", 0),
+ CIO2_SENSOR_CONFIG("INT33BE", 1, 419200000),
+ /* Omnivision OV8865 */
+ CIO2_SENSOR_CONFIG("INT347A", 1, 360000000),
/* Omnivision OV2680 */
CIO2_SENSOR_CONFIG("OVTI2680", 0),
};
@@ -36,6 +39,18 @@ static const struct cio2_property_names prop_names = {
.link_frequencies = "link-frequencies",
};
+static const char * const cio2_vcm_types[] = {
+ "ad5823",
+ "dw9714",
+ "ad5816",
+ "dw9719",
+ "dw9718",
+ "dw9806b",
+ "wv517s",
+ "lc898122xa",
+ "lc898212axb",
+};
+
static int cio2_bridge_read_acpi_buffer(struct acpi_device *adev, char *id,
void *data, u32 size)
{
@@ -132,6 +147,12 @@ static void cio2_bridge_create_fwnode_properties(
sensor->dev_properties[2] = PROPERTY_ENTRY_U32(
sensor->prop_names.orientation,
orientation);
+ if (sensor->ssdb.vcmtype) {
+ sensor->vcm_ref[0] =
+ SOFTWARE_NODE_REFERENCE(&sensor->swnodes[SWNODE_VCM]);
+ sensor->dev_properties[3] =
+ PROPERTY_ENTRY_REF_ARRAY("lens-focus", sensor->vcm_ref);
+ }
sensor->ep_properties[0] = PROPERTY_ENTRY_U32(
sensor->prop_names.bus_type,
@@ -193,6 +214,33 @@ static void cio2_bridge_create_connection_swnodes(struct cio2_bridge *bridge,
sensor->node_names.endpoint,
&nodes[SWNODE_CIO2_PORT],
sensor->cio2_properties);
+ if (sensor->ssdb.vcmtype)
+ nodes[SWNODE_VCM] =
+ NODE_VCM(cio2_vcm_types[sensor->ssdb.vcmtype - 1]);
+}
+
+static void cio2_bridge_instantiate_vcm_i2c_client(struct cio2_sensor *sensor)
+{
+ struct i2c_board_info board_info = { };
+ char name[16];
+
+ if (!sensor->ssdb.vcmtype)
+ return;
+
+ snprintf(name, sizeof(name), "%s-VCM", acpi_dev_name(sensor->adev));
+ board_info.dev_name = name;
+ strscpy(board_info.type, cio2_vcm_types[sensor->ssdb.vcmtype - 1],
+ ARRAY_SIZE(board_info.type));
+ board_info.swnode = &sensor->swnodes[SWNODE_VCM];
+
+ sensor->vcm_i2c_client =
+ i2c_acpi_new_device_by_fwnode(acpi_fwnode_handle(sensor->adev),
+ 1, &board_info);
+ if (IS_ERR(sensor->vcm_i2c_client)) {
+ dev_warn(&sensor->adev->dev, "Error instantiation VCM i2c-client: %ld\n",
+ PTR_ERR(sensor->vcm_i2c_client));
+ sensor->vcm_i2c_client = NULL;
+ }
}
static void cio2_bridge_unregister_sensors(struct cio2_bridge *bridge)
@@ -205,6 +253,7 @@ static void cio2_bridge_unregister_sensors(struct cio2_bridge *bridge)
software_node_unregister_nodes(sensor->swnodes);
ACPI_FREE(sensor->pld);
acpi_dev_put(sensor->adev);
+ i2c_unregister_device(sensor->vcm_i2c_client);
}
}
@@ -237,9 +286,17 @@ static int cio2_bridge_connect_sensor(const struct cio2_sensor_config *cfg,
if (ret)
goto err_put_adev;
+ if (sensor->ssdb.vcmtype > ARRAY_SIZE(cio2_vcm_types)) {
+ dev_warn(&adev->dev, "Unknown VCM type %d\n",
+ sensor->ssdb.vcmtype);
+ sensor->ssdb.vcmtype = 0;
+ }
+
status = acpi_get_physical_device_location(adev->handle, &sensor->pld);
- if (ACPI_FAILURE(status))
+ if (ACPI_FAILURE(status)) {
+ ret = -ENODEV;
goto err_put_adev;
+ }
if (sensor->ssdb.lanes > CIO2_MAX_LANES) {
dev_err(&adev->dev,
@@ -265,6 +322,8 @@ static int cio2_bridge_connect_sensor(const struct cio2_sensor_config *cfg,
sensor->adev = acpi_dev_get(adev);
adev->fwnode.secondary = fwnode;
+ cio2_bridge_instantiate_vcm_i2c_client(sensor);
+
dev_info(&cio2->dev, "Found supported sensor %s\n",
acpi_dev_name(adev));
@@ -304,6 +363,40 @@ static int cio2_bridge_connect_sensors(struct cio2_bridge *bridge,
return ret;
}
+/*
+ * The VCM cannot be probed until the PMIC is completely setup. We cannot rely
+ * on -EPROBE_DEFER for this, since the consumer<->supplier relations between
+ * the VCM and regulators/clks are not described in ACPI, instead they are
+ * passed as board-data to the PMIC drivers. Since -PROBE_DEFER does not work
+ * for the clks/regulators the VCM i2c-clients must not be instantiated until
+ * the PMIC is fully setup.
+ *
+ * The sensor/VCM ACPI device has an ACPI _DEP on the PMIC, check this using the
+ * acpi_dev_ready_for_enumeration() helper, like the i2c-core-acpi code does
+ * for the sensors.
+ */
+static int cio2_bridge_sensors_are_ready(void)
+{
+ struct acpi_device *adev;
+ bool ready = true;
+ unsigned int i;
+
+ for (i = 0; i < ARRAY_SIZE(cio2_supported_sensors); i++) {
+ const struct cio2_sensor_config *cfg =
+ &cio2_supported_sensors[i];
+
+ for_each_acpi_dev_match(adev, cfg->hid, NULL, -1) {
+ if (!adev->status.enabled)
+ continue;
+
+ if (!acpi_dev_ready_for_enumeration(adev))
+ ready = false;
+ }
+ }
+
+ return ready;
+}
+
int cio2_bridge_init(struct pci_dev *cio2)
{
struct device *dev = &cio2->dev;
@@ -312,6 +405,9 @@ int cio2_bridge_init(struct pci_dev *cio2)
unsigned int i;
int ret;
+ if (!cio2_bridge_sensors_are_ready())
+ return -EPROBE_DEFER;
+
bridge = kzalloc(sizeof(*bridge), GFP_KERNEL);
if (!bridge)
return -ENOMEM;
diff --git a/drivers/media/pci/intel/ipu3/cio2-bridge.h b/drivers/media/pci/intel/ipu3/cio2-bridge.h
index 202c7d4..4418cbd 100644
--- a/drivers/media/pci/intel/ipu3/cio2-bridge.h
+++ b/drivers/media/pci/intel/ipu3/cio2-bridge.h
@@ -8,6 +8,8 @@
#include "ipu3-cio2.h"
+struct i2c_client;
+
#define CIO2_HID "INT343E"
#define CIO2_MAX_LANES 4
#define MAX_NUM_LINK_FREQS 3
@@ -42,12 +44,19 @@
.properties = _PROPS, \
}
+#define NODE_VCM(_TYPE) \
+ (const struct software_node) { \
+ .name = _TYPE, \
+ }
+
enum cio2_sensor_swnodes {
SWNODE_SENSOR_HID,
SWNODE_SENSOR_PORT,
SWNODE_SENSOR_ENDPOINT,
SWNODE_CIO2_PORT,
SWNODE_CIO2_ENDPOINT,
+ /* Must be last because it is optional / maybe empty */
+ SWNODE_VCM,
SWNODE_COUNT
};
@@ -106,8 +115,10 @@ struct cio2_sensor_config {
struct cio2_sensor {
char name[ACPI_ID_LEN];
struct acpi_device *adev;
+ struct i2c_client *vcm_i2c_client;
- struct software_node swnodes[6];
+ /* SWNODE_COUNT + 1 for terminating empty node */
+ struct software_node swnodes[SWNODE_COUNT + 1];
struct cio2_node_names node_names;
struct cio2_sensor_ssdb ssdb;
@@ -115,10 +126,11 @@ struct cio2_sensor {
struct cio2_property_names prop_names;
struct property_entry ep_properties[5];
- struct property_entry dev_properties[4];
+ struct property_entry dev_properties[5];
struct property_entry cio2_properties[3];
struct software_node_ref_args local_ref[1];
struct software_node_ref_args remote_ref[1];
+ struct software_node_ref_args vcm_ref[1];
};
struct cio2_bridge {
diff --git a/drivers/media/pci/intel/ipu3/ipu3-cio2-main.c b/drivers/media/pci/intel/ipu3/ipu3-cio2-main.c
index 356ea96..0e9b050 100644
--- a/drivers/media/pci/intel/ipu3/ipu3-cio2-main.c
+++ b/drivers/media/pci/intel/ipu3/ipu3-cio2-main.c
@@ -1713,11 +1713,6 @@ static int cio2_pci_probe(struct pci_dev *pci_dev,
struct cio2_device *cio2;
int r;
- cio2 = devm_kzalloc(dev, sizeof(*cio2), GFP_KERNEL);
- if (!cio2)
- return -ENOMEM;
- cio2->pci_dev = pci_dev;
-
/*
* On some platforms no connections to sensors are defined in firmware,
* if the device has no endpoints then we can try to build those as
@@ -1735,6 +1730,11 @@ static int cio2_pci_probe(struct pci_dev *pci_dev,
return r;
}
+ cio2 = devm_kzalloc(dev, sizeof(*cio2), GFP_KERNEL);
+ if (!cio2)
+ return -ENOMEM;
+ cio2->pci_dev = pci_dev;
+
r = pcim_enable_device(pci_dev);
if (r) {
dev_err(dev, "failed to enable device (%d)\n", r);
@@ -1966,12 +1966,19 @@ static int __maybe_unused cio2_suspend(struct device *dev)
struct pci_dev *pci_dev = to_pci_dev(dev);
struct cio2_device *cio2 = pci_get_drvdata(pci_dev);
struct cio2_queue *q = cio2->cur_queue;
+ int r;
dev_dbg(dev, "cio2 suspend\n");
if (!cio2->streaming)
return 0;
/* Stop stream */
+ r = v4l2_subdev_call(q->sensor, video, s_stream, 0);
+ if (r) {
+ dev_err(dev, "failed to stop sensor streaming\n");
+ return r;
+ }
+
cio2_hw_exit(cio2, q);
synchronize_irq(pci_dev->irq);
@@ -2005,8 +2012,16 @@ static int __maybe_unused cio2_resume(struct device *dev)
}
r = cio2_hw_init(cio2, q);
- if (r)
+ if (r) {
dev_err(dev, "fail to init cio2 hw\n");
+ return r;
+ }
+
+ r = v4l2_subdev_call(q->sensor, video, s_stream, 1);
+ if (r) {
+ dev_err(dev, "fail to start sensor streaming\n");
+ cio2_hw_exit(cio2, q);
+ }
return r;
}
diff --git a/drivers/media/pci/ivtv/ivtv-alsa-main.c b/drivers/media/pci/ivtv/ivtv-alsa-main.c
index 4cefdb2..9e13a71 100644
--- a/drivers/media/pci/ivtv/ivtv-alsa-main.c
+++ b/drivers/media/pci/ivtv/ivtv-alsa-main.c
@@ -48,12 +48,6 @@ struct snd_ivtv_card *to_snd_ivtv_card(struct v4l2_device *v4l2_dev)
return to_ivtv(v4l2_dev)->alsa;
}
-static inline
-struct snd_ivtv_card *p_to_snd_ivtv_card(struct v4l2_device **v4l2_dev)
-{
- return container_of(v4l2_dev, struct snd_ivtv_card, v4l2_dev);
-}
-
static void snd_ivtv_card_free(struct snd_ivtv_card *itvsc)
{
if (itvsc == NULL)
diff --git a/drivers/media/pci/ivtv/ivtvfb.c b/drivers/media/pci/ivtv/ivtvfb.c
index 2c43ebf..00ac94d 100644
--- a/drivers/media/pci/ivtv/ivtvfb.c
+++ b/drivers/media/pci/ivtv/ivtvfb.c
@@ -42,7 +42,7 @@
/* card parameters */
static int ivtvfb_card_id = -1;
-static int ivtvfb_debug = 0;
+static int ivtvfb_debug;
static bool ivtvfb_force_pat = IS_ENABLED(CONFIG_VIDEO_FB_IVTV_FORCE_PAT);
static bool osd_laced;
static int osd_depth;
diff --git a/drivers/media/pci/pt3/pt3.c b/drivers/media/pci/pt3/pt3.c
index c0bc867..0d51bdf 100644
--- a/drivers/media/pci/pt3/pt3.c
+++ b/drivers/media/pci/pt3/pt3.c
@@ -685,12 +685,6 @@ static void pt3_remove(struct pci_dev *pdev)
for (i = PT3_NUM_FE - 1; i >= 0; i--)
pt3_cleanup_adapter(pt3, i);
i2c_del_adapter(&pt3->i2c_adap);
- kfree(pt3->i2c_buf);
- pci_iounmap(pt3->pdev, pt3->regs[0]);
- pci_iounmap(pt3->pdev, pt3->regs[1]);
- pci_release_regions(pdev);
- pci_disable_device(pdev);
- kfree(pt3);
}
static int pt3_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
@@ -704,14 +698,14 @@ static int pt3_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
if (pci_read_config_byte(pdev, PCI_REVISION_ID, &rev) || rev != 1)
return -ENODEV;
- ret = pci_enable_device(pdev);
+ ret = pcim_enable_device(pdev);
if (ret < 0)
return -ENODEV;
pci_set_master(pdev);
- ret = pci_request_regions(pdev, DRV_NAME);
+ ret = pcim_iomap_regions(pdev, BIT(0) | BIT(2), DRV_NAME);
if (ret < 0)
- goto err_disable_device;
+ return ret;
ret = dma_set_mask(&pdev->dev, DMA_BIT_MASK(64));
if (ret == 0)
@@ -722,42 +716,32 @@ static int pt3_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(32));
else {
dev_err(&pdev->dev, "Failed to set DMA mask\n");
- goto err_release_regions;
+ return ret;
}
dev_info(&pdev->dev, "Use 32bit DMA\n");
}
- pt3 = kzalloc(sizeof(*pt3), GFP_KERNEL);
- if (!pt3) {
- ret = -ENOMEM;
- goto err_release_regions;
- }
+ pt3 = devm_kzalloc(&pdev->dev, sizeof(*pt3), GFP_KERNEL);
+ if (!pt3)
+ return -ENOMEM;
pci_set_drvdata(pdev, pt3);
pt3->pdev = pdev;
mutex_init(&pt3->lock);
- pt3->regs[0] = pci_ioremap_bar(pdev, 0);
- pt3->regs[1] = pci_ioremap_bar(pdev, 2);
- if (pt3->regs[0] == NULL || pt3->regs[1] == NULL) {
- dev_err(&pdev->dev, "Failed to ioremap\n");
- ret = -ENOMEM;
- goto err_kfree;
- }
+ pt3->regs[0] = pcim_iomap_table(pdev)[0];
+ pt3->regs[1] = pcim_iomap_table(pdev)[2];
ver = ioread32(pt3->regs[0] + REG_VERSION);
if ((ver >> 16) != 0x0301) {
dev_warn(&pdev->dev, "PT%d, I/F-ver.:%d not supported\n",
ver >> 24, (ver & 0x00ff0000) >> 16);
- ret = -ENODEV;
- goto err_iounmap;
+ return -ENODEV;
}
pt3->num_bufs = clamp_val(num_bufs, MIN_DATA_BUFS, MAX_DATA_BUFS);
- pt3->i2c_buf = kmalloc(sizeof(*pt3->i2c_buf), GFP_KERNEL);
- if (pt3->i2c_buf == NULL) {
- ret = -ENOMEM;
- goto err_iounmap;
- }
+ pt3->i2c_buf = devm_kmalloc(&pdev->dev, sizeof(*pt3->i2c_buf), GFP_KERNEL);
+ if (!pt3->i2c_buf)
+ return -ENOMEM;
i2c = &pt3->i2c_adap;
i2c->owner = THIS_MODULE;
i2c->algo = &pt3_i2c_algo;
@@ -767,7 +751,7 @@ static int pt3_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
i2c_set_adapdata(i2c, pt3);
ret = i2c_add_adapter(i2c);
if (ret < 0)
- goto err_i2cbuf;
+ return ret;
for (i = 0; i < PT3_NUM_FE; i++) {
ret = pt3_alloc_adapter(pt3, i);
@@ -799,21 +783,7 @@ static int pt3_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
while (i >= 0)
pt3_cleanup_adapter(pt3, i--);
i2c_del_adapter(i2c);
-err_i2cbuf:
- kfree(pt3->i2c_buf);
-err_iounmap:
- if (pt3->regs[0])
- pci_iounmap(pdev, pt3->regs[0]);
- if (pt3->regs[1])
- pci_iounmap(pdev, pt3->regs[1]);
-err_kfree:
- kfree(pt3);
-err_release_regions:
- pci_release_regions(pdev);
-err_disable_device:
- pci_disable_device(pdev);
return ret;
-
}
static const struct pci_device_id pt3_id_table[] = {
diff --git a/drivers/media/pci/saa7134/saa7134-go7007.c b/drivers/media/pci/saa7134/saa7134-go7007.c
index f319edb..da83893 100644
--- a/drivers/media/pci/saa7134/saa7134-go7007.c
+++ b/drivers/media/pci/saa7134/saa7134-go7007.c
@@ -56,11 +56,6 @@ struct saa7134_go7007 {
dma_addr_t bottom_dma;
};
-static inline struct saa7134_go7007 *to_state(struct v4l2_subdev *sd)
-{
- return container_of(sd, struct saa7134_go7007, sd);
-}
-
static const struct go7007_board_info board_voyager = {
.flags = 0,
.sensor_flags = GO7007_SENSOR_656 |
@@ -385,7 +380,7 @@ MODULE_FIRMWARE("go7007/go7007tv.bin");
static int saa7134_go7007_s_std(struct v4l2_subdev *sd, v4l2_std_id norm)
{
#if 0
- struct saa7134_go7007 *saa = to_state(sd);
+ struct saa7134_go7007 *saa = container_of(sd, struct saa7134_go7007, sd);
struct saa7134_dev *dev = saa->dev;
return saa7134_s_std_internal(dev, NULL, norm);
diff --git a/drivers/media/pci/saa7146/hexium_gemini.c b/drivers/media/pci/saa7146/hexium_gemini.c
index 2214c74..3947701 100644
--- a/drivers/media/pci/saa7146/hexium_gemini.c
+++ b/drivers/media/pci/saa7146/hexium_gemini.c
@@ -284,7 +284,12 @@ static int hexium_attach(struct saa7146_dev *dev, struct saa7146_pci_extension_d
hexium_set_input(hexium, 0);
hexium->cur_input = 0;
- saa7146_vv_init(dev, &vv_data);
+ ret = saa7146_vv_init(dev, &vv_data);
+ if (ret) {
+ i2c_del_adapter(&hexium->i2c_adapter);
+ kfree(hexium);
+ return ret;
+ }
vv_data.vid_ops.vidioc_enum_input = vidioc_enum_input;
vv_data.vid_ops.vidioc_g_input = vidioc_g_input;
diff --git a/drivers/media/pci/saa7146/hexium_orion.c b/drivers/media/pci/saa7146/hexium_orion.c
index 39d14c1..2eb4bee 100644
--- a/drivers/media/pci/saa7146/hexium_orion.c
+++ b/drivers/media/pci/saa7146/hexium_orion.c
@@ -355,10 +355,16 @@ static struct saa7146_ext_vv vv_data;
static int hexium_attach(struct saa7146_dev *dev, struct saa7146_pci_extension_data *info)
{
struct hexium *hexium = (struct hexium *) dev->ext_priv;
+ int ret;
DEB_EE("\n");
- saa7146_vv_init(dev, &vv_data);
+ ret = saa7146_vv_init(dev, &vv_data);
+ if (ret) {
+ pr_err("Error in saa7146_vv_init()\n");
+ return ret;
+ }
+
vv_data.vid_ops.vidioc_enum_input = vidioc_enum_input;
vv_data.vid_ops.vidioc_g_input = vidioc_g_input;
vv_data.vid_ops.vidioc_s_input = vidioc_s_input;
diff --git a/drivers/media/pci/saa7146/mxb.c b/drivers/media/pci/saa7146/mxb.c
index 73fc901..7ded8f5 100644
--- a/drivers/media/pci/saa7146/mxb.c
+++ b/drivers/media/pci/saa7146/mxb.c
@@ -340,7 +340,7 @@ static int mxb_init_done(struct saa7146_dev* dev)
struct tuner_setup tun_setup;
v4l2_std_id std = V4L2_STD_PAL_BG;
- int i = 0, err = 0;
+ int i, err = 0;
/* mute audio on tea6420s */
tea6420_route(mxb, 6);
@@ -349,7 +349,6 @@ static int mxb_init_done(struct saa7146_dev* dev)
saa7111a_call(mxb, video, s_std, std);
/* select tuner-output on saa7111a */
- i = 0;
saa7111a_call(mxb, video, s_routing, SAA7115_COMPOSITE0,
SAA7111_FMT_CCIR, 0);
@@ -683,10 +682,16 @@ static struct saa7146_ext_vv vv_data;
static int mxb_attach(struct saa7146_dev *dev, struct saa7146_pci_extension_data *info)
{
struct mxb *mxb;
+ int ret;
DEB_EE("dev:%p\n", dev);
- saa7146_vv_init(dev, &vv_data);
+ ret = saa7146_vv_init(dev, &vv_data);
+ if (ret) {
+ ERR("Error in saa7146_vv_init()");
+ return ret;
+ }
+
if (mxb_probe(dev)) {
saa7146_vv_release(dev);
return -1;
diff --git a/drivers/media/pci/solo6x10/solo6x10-v4l2-enc.c b/drivers/media/pci/solo6x10/solo6x10-v4l2-enc.c
index 0abcad4..7766cad 100644
--- a/drivers/media/pci/solo6x10/solo6x10-v4l2-enc.c
+++ b/drivers/media/pci/solo6x10/solo6x10-v4l2-enc.c
@@ -391,12 +391,12 @@ static int solo_send_desc(struct solo_enc_dev *solo_enc, int skip,
}
/* Extract values from VOP header - VE_STATUSxx */
-static inline int vop_interlaced(const vop_header *vh)
+static inline __always_unused int vop_interlaced(const vop_header *vh)
{
return (__le32_to_cpu((*vh)[0]) >> 30) & 1;
}
-static inline u8 vop_channel(const vop_header *vh)
+static inline __always_unused u8 vop_channel(const vop_header *vh)
{
return (__le32_to_cpu((*vh)[0]) >> 24) & 0x1F;
}
@@ -411,12 +411,12 @@ static inline u32 vop_mpeg_size(const vop_header *vh)
return __le32_to_cpu((*vh)[0]) & 0xFFFFF;
}
-static inline u8 vop_hsize(const vop_header *vh)
+static inline u8 __always_unused vop_hsize(const vop_header *vh)
{
return (__le32_to_cpu((*vh)[1]) >> 8) & 0xFF;
}
-static inline u8 vop_vsize(const vop_header *vh)
+static inline u8 __always_unused vop_vsize(const vop_header *vh)
{
return __le32_to_cpu((*vh)[1]) & 0xFF;
}
@@ -436,12 +436,12 @@ static inline u32 vop_jpeg_size(const vop_header *vh)
return __le32_to_cpu((*vh)[4]) & 0xFFFFF;
}
-static inline u32 vop_sec(const vop_header *vh)
+static inline u32 __always_unused vop_sec(const vop_header *vh)
{
return __le32_to_cpu((*vh)[5]);
}
-static inline u32 vop_usec(const vop_header *vh)
+static inline __always_unused u32 vop_usec(const vop_header *vh)
{
return __le32_to_cpu((*vh)[6]);
}
diff --git a/drivers/media/pci/tw5864/tw5864-core.c b/drivers/media/pci/tw5864/tw5864-core.c
index 23d3cae..5cae73e 100644
--- a/drivers/media/pci/tw5864/tw5864-core.c
+++ b/drivers/media/pci/tw5864/tw5864-core.c
@@ -333,11 +333,10 @@ static void tw5864_finidev(struct pci_dev *pci_dev)
/* release resources */
iounmap(dev->mmio);
- release_mem_region(pci_resource_start(pci_dev, 0),
- pci_resource_len(pci_dev, 0));
+ pci_release_regions(pci_dev);
+ pci_disable_device(pci_dev);
v4l2_device_unregister(&dev->v4l2_dev);
- devm_kfree(&pci_dev->dev, dev);
}
static struct pci_driver tw5864_pci_driver = {
diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index cf4adc6..9fbdba0 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -492,6 +492,18 @@
endif # VIDEO_STI_DELTA
+config VIDEO_STM32_DMA2D
+ tristate "STM32 Chrom-Art Accelerator (DMA2D)"
+ depends on VIDEO_DEV && VIDEO_V4L2
+ depends on ARCH_STM32 || COMPILE_TEST
+ select VIDEOBUF2_DMA_CONTIG
+ select V4L2_MEM2MEM_DEV
+ help
+ Enables DMA2D hardware support on stm32.
+
+ The STM32 DMA2D is a memory-to-memory engine for pixel conversion
+ and specialized DMA dedicated to image manipulation.
+
config VIDEO_RENESAS_FDP1
tristate "Renesas Fine Display Processor"
depends on VIDEO_DEV && VIDEO_V4L2
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index a148553..19bcbce 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -71,6 +71,7 @@
obj-$(CONFIG_VIDEO_ATMEL_XISC) += atmel/
obj-$(CONFIG_VIDEO_STM32_DCMI) += stm32/
+obj-$(CONFIG_VIDEO_STM32_DMA2D) += stm32/
obj-$(CONFIG_VIDEO_MEDIATEK_VPU) += mtk-vpu/
diff --git a/drivers/media/platform/allegro-dvt/allegro-core.c b/drivers/media/platform/allegro-dvt/allegro-core.c
index c8156da..4a3d06c 100644
--- a/drivers/media/platform/allegro-dvt/allegro-core.c
+++ b/drivers/media/platform/allegro-dvt/allegro-core.c
@@ -2815,7 +2815,7 @@ static void allegro_buf_queue(struct vb2_buffer *vb)
unsigned int i;
for (i = 0; i < vb->num_planes; i++)
- vb->planes[i].bytesused = 0;
+ vb2_set_plane_payload(vb, i, 0);
vbuf->field = V4L2_FIELD_NONE;
vbuf->sequence = channel->csequence++;
diff --git a/drivers/media/platform/aspeed-video.c b/drivers/media/platform/aspeed-video.c
index cad3f97..7a24daf 100644
--- a/drivers/media/platform/aspeed-video.c
+++ b/drivers/media/platform/aspeed-video.c
@@ -539,6 +539,10 @@ static void aspeed_video_enable_mode_detect(struct aspeed_video *video)
aspeed_video_update(video, VE_INTERRUPT_CTRL, 0,
VE_INTERRUPT_MODE_DETECT);
+ /* Disable mode detect in order to re-trigger */
+ aspeed_video_update(video, VE_SEQ_CTRL,
+ VE_SEQ_CTRL_TRIG_MODE_DET, 0);
+
/* Trigger mode detect */
aspeed_video_update(video, VE_SEQ_CTRL, 0, VE_SEQ_CTRL_TRIG_MODE_DET);
}
@@ -591,6 +595,8 @@ static void aspeed_video_irq_res_change(struct aspeed_video *video, ulong delay)
set_bit(VIDEO_RES_CHANGE, &video->flags);
clear_bit(VIDEO_FRAME_INPRG, &video->flags);
+ video->v4l2_input_status = V4L2_IN_ST_NO_SIGNAL;
+
aspeed_video_off(video);
aspeed_video_bufs_done(video, VB2_BUF_STATE_ERROR);
@@ -824,10 +830,6 @@ static void aspeed_video_get_resolution(struct aspeed_video *video)
return;
}
- /* Disable mode detect in order to re-trigger */
- aspeed_video_update(video, VE_SEQ_CTRL,
- VE_SEQ_CTRL_TRIG_MODE_DET, 0);
-
aspeed_video_check_and_set_polarity(video);
aspeed_video_enable_mode_detect(video);
@@ -1375,7 +1377,6 @@ static void aspeed_video_resolution_work(struct work_struct *work)
struct delayed_work *dwork = to_delayed_work(work);
struct aspeed_video *video = container_of(dwork, struct aspeed_video,
res_work);
- u32 input_status = video->v4l2_input_status;
aspeed_video_on(video);
@@ -1388,8 +1389,7 @@ static void aspeed_video_resolution_work(struct work_struct *work)
aspeed_video_get_resolution(video);
if (video->detected_timings.width != video->active_timings.width ||
- video->detected_timings.height != video->active_timings.height ||
- input_status != video->v4l2_input_status) {
+ video->detected_timings.height != video->active_timings.height) {
static const struct v4l2_event ev = {
.type = V4L2_EVENT_SOURCE_CHANGE,
.u.src_change.changes = V4L2_EVENT_SRC_CH_RESOLUTION,
diff --git a/drivers/media/platform/coda/coda-common.c b/drivers/media/platform/coda/coda-common.c
index 0e312b0..3cd47ba 100644
--- a/drivers/media/platform/coda/coda-common.c
+++ b/drivers/media/platform/coda/coda-common.c
@@ -43,7 +43,7 @@
#define CODA_NAME "coda"
#define CODADX6_MAX_INSTANCES 4
-#define CODA_MAX_FORMATS 4
+#define CODA_MAX_FORMATS 5
#define CODA_ISRAM_SIZE (2048 * 2)
@@ -247,6 +247,7 @@ static const struct coda_video_device coda9_jpeg_encoder = {
V4L2_PIX_FMT_YUV420,
V4L2_PIX_FMT_YVU420,
V4L2_PIX_FMT_YUV422P,
+ V4L2_PIX_FMT_GREY,
},
.dst_formats = {
V4L2_PIX_FMT_JPEG,
@@ -626,6 +627,11 @@ static int coda_try_fmt(struct coda_ctx *ctx, const struct coda_codec *codec,
f->fmt.pix.sizeimage = f->fmt.pix.bytesperline *
f->fmt.pix.height * 2;
break;
+ case V4L2_PIX_FMT_GREY:
+ /* keep 16 pixel alignment of 8-bit pixel data */
+ f->fmt.pix.bytesperline = round_up(f->fmt.pix.width, 16);
+ f->fmt.pix.sizeimage = f->fmt.pix.bytesperline * f->fmt.pix.height;
+ break;
case V4L2_PIX_FMT_JPEG:
case V4L2_PIX_FMT_H264:
case V4L2_PIX_FMT_MPEG4:
@@ -1537,11 +1543,13 @@ static void coda_pic_run_work(struct work_struct *work)
if (!wait_for_completion_timeout(&ctx->completion,
msecs_to_jiffies(1000))) {
- dev_err(dev->dev, "CODA PIC_RUN timeout\n");
+ if (ctx->use_bit) {
+ dev_err(dev->dev, "CODA PIC_RUN timeout\n");
- ctx->hold = true;
+ ctx->hold = true;
- coda_hw_reset(ctx);
+ coda_hw_reset(ctx);
+ }
if (ctx->ops->run_timeout)
ctx->ops->run_timeout(ctx);
diff --git a/drivers/media/platform/coda/coda-jpeg.c b/drivers/media/platform/coda/coda-jpeg.c
index b11cfbe..a72f465 100644
--- a/drivers/media/platform/coda/coda-jpeg.c
+++ b/drivers/media/platform/coda/coda-jpeg.c
@@ -1127,7 +1127,8 @@ static int coda9_jpeg_prepare_encode(struct coda_ctx *ctx)
coda_write(dev, 0, CODA9_REG_JPEG_GBU_BT_PTR);
coda_write(dev, 0, CODA9_REG_JPEG_GBU_WD_PTR);
coda_write(dev, 0, CODA9_REG_JPEG_GBU_BBSR);
- coda_write(dev, 0, CODA9_REG_JPEG_BBC_STRM_CTRL);
+ coda_write(dev, BIT(31) | ((end_addr - start_addr - header_len) / 256),
+ CODA9_REG_JPEG_BBC_STRM_CTRL);
coda_write(dev, 0, CODA9_REG_JPEG_GBU_CTRL);
coda_write(dev, 0, CODA9_REG_JPEG_GBU_FF_RPTR);
coda_write(dev, 127, CODA9_REG_JPEG_GBU_BBER);
@@ -1257,6 +1258,23 @@ static void coda9_jpeg_finish_encode(struct coda_ctx *ctx)
coda_hw_reset(ctx);
}
+static void coda9_jpeg_encode_timeout(struct coda_ctx *ctx)
+{
+ struct coda_dev *dev = ctx->dev;
+ u32 end_addr, wr_ptr;
+
+ /* Handle missing BBC overflow interrupt via timeout */
+ end_addr = coda_read(dev, CODA9_REG_JPEG_BBC_END_ADDR);
+ wr_ptr = coda_read(dev, CODA9_REG_JPEG_BBC_WR_PTR);
+ if (wr_ptr >= end_addr - 256) {
+ v4l2_err(&dev->v4l2_dev, "JPEG too large for capture buffer\n");
+ coda9_jpeg_finish_encode(ctx);
+ return;
+ }
+
+ coda_hw_reset(ctx);
+}
+
static void coda9_jpeg_release(struct coda_ctx *ctx)
{
int i;
@@ -1276,6 +1294,7 @@ const struct coda_context_ops coda9_jpeg_encode_ops = {
.start_streaming = coda9_jpeg_start_encoding,
.prepare_run = coda9_jpeg_prepare_encode,
.finish_run = coda9_jpeg_finish_encode,
+ .run_timeout = coda9_jpeg_encode_timeout,
.release = coda9_jpeg_release,
};
diff --git a/drivers/media/platform/coda/imx-vdoa.c b/drivers/media/platform/coda/imx-vdoa.c
index 6996d45..00643f3 100644
--- a/drivers/media/platform/coda/imx-vdoa.c
+++ b/drivers/media/platform/coda/imx-vdoa.c
@@ -287,7 +287,11 @@ static int vdoa_probe(struct platform_device *pdev)
struct resource *res;
int ret;
- dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(32));
+ ret = dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(32));
+ if (ret) {
+ dev_err(&pdev->dev, "DMA enable failed\n");
+ return ret;
+ }
vdoa = devm_kzalloc(&pdev->dev, sizeof(*vdoa), GFP_KERNEL);
if (!vdoa)
diff --git a/drivers/media/platform/davinci/vpbe_osd.c b/drivers/media/platform/davinci/vpbe_osd.c
index 91b571a0..32f7ef5 100644
--- a/drivers/media/platform/davinci/vpbe_osd.c
+++ b/drivers/media/platform/davinci/vpbe_osd.c
@@ -45,7 +45,7 @@ static const struct platform_device_id vpbe_osd_devtype[] = {
MODULE_DEVICE_TABLE(platform, vpbe_osd_devtype);
/* register access routines */
-static inline u32 osd_read(struct osd_state *sd, u32 offset)
+static inline u32 __always_unused osd_read(struct osd_state *sd, u32 offset)
{
struct osd_state *osd = sd;
diff --git a/drivers/media/platform/davinci/vpif_capture.c b/drivers/media/platform/davinci/vpif_capture.c
index ae92e2c..8fe5537 100644
--- a/drivers/media/platform/davinci/vpif_capture.c
+++ b/drivers/media/platform/davinci/vpif_capture.c
@@ -619,17 +619,6 @@ static void vpif_calculate_offsets(struct channel_obj *ch)
}
/**
- * vpif_get_default_field() - Get default field type based on interface
- * @iface: ptr to vpif interface
- */
-static inline enum v4l2_field vpif_get_default_field(
- struct vpif_interface *iface)
-{
- return (iface->if_type == VPIF_IF_RAW_BAYER) ? V4L2_FIELD_NONE :
- V4L2_FIELD_INTERLACED;
-}
-
-/**
* vpif_config_addr() - function to configure buffer address in vpif
* @ch: channel ptr
* @muxmode: channel mux mode
@@ -1478,7 +1467,6 @@ static int vpif_probe_complete(void)
for (k = 0; k < j; k++) {
/* Get the pointer to the channel object */
ch = vpif_obj.dev[k];
- common = &ch->common[k];
/* Unregister video device */
video_unregister_device(&ch->video_dev);
}
diff --git a/drivers/media/platform/imx-pxp.c b/drivers/media/platform/imx-pxp.c
index 723b096..689ae5e 100644
--- a/drivers/media/platform/imx-pxp.c
+++ b/drivers/media/platform/imx-pxp.c
@@ -211,6 +211,7 @@ struct pxp_ctx {
/* Processing mode */
int mode;
u8 alpha_component;
+ u8 rotation;
enum v4l2_colorspace colorspace;
enum v4l2_xfer_func xfer_func;
@@ -767,14 +768,20 @@ static int pxp_start(struct pxp_ctx *ctx, struct vb2_v4l2_buffer *in_vb,
V4L2_BUF_FLAG_BFRAME |
V4L2_BUF_FLAG_TSTAMP_SRC_MASK);
- /* Rotation disabled, 8x8 block size */
+ /* 8x8 block size */
ctrl = BF_PXP_CTRL_VFLIP0(!!(ctx->mode & MEM2MEM_VFLIP)) |
- BF_PXP_CTRL_HFLIP0(!!(ctx->mode & MEM2MEM_HFLIP));
+ BF_PXP_CTRL_HFLIP0(!!(ctx->mode & MEM2MEM_HFLIP)) |
+ BF_PXP_CTRL_ROTATE0(ctx->rotation);
/* Always write alpha value as V4L2_CID_ALPHA_COMPONENT */
out_ctrl = BF_PXP_OUT_CTRL_ALPHA(ctx->alpha_component) |
BF_PXP_OUT_CTRL_ALPHA_OUTPUT(1) |
pxp_v4l2_pix_fmt_to_out_format(dst_fourcc);
out_buf = p_out;
+
+ if (ctx->rotation == BV_PXP_CTRL_ROTATE0__ROT_90 ||
+ ctx->rotation == BV_PXP_CTRL_ROTATE0__ROT_270)
+ swap(dst_width, dst_height);
+
switch (dst_fourcc) {
case V4L2_PIX_FMT_NV12:
case V4L2_PIX_FMT_NV21:
@@ -1297,6 +1304,21 @@ static int pxp_s_fmt_vid_out(struct file *file, void *priv,
return 0;
}
+static u8 pxp_degrees_to_rot_mode(u32 degrees)
+{
+ switch (degrees) {
+ case 90:
+ return BV_PXP_CTRL_ROTATE0__ROT_90;
+ case 180:
+ return BV_PXP_CTRL_ROTATE0__ROT_180;
+ case 270:
+ return BV_PXP_CTRL_ROTATE0__ROT_270;
+ case 0:
+ default:
+ return BV_PXP_CTRL_ROTATE0__ROT_0;
+ }
+}
+
static int pxp_s_ctrl(struct v4l2_ctrl *ctrl)
{
struct pxp_ctx *ctx =
@@ -1317,6 +1339,10 @@ static int pxp_s_ctrl(struct v4l2_ctrl *ctrl)
ctx->mode &= ~MEM2MEM_VFLIP;
break;
+ case V4L2_CID_ROTATE:
+ ctx->rotation = pxp_degrees_to_rot_mode(ctrl->val);
+ break;
+
case V4L2_CID_ALPHA_COMPONENT:
ctx->alpha_component = ctrl->val;
break;
@@ -1524,6 +1550,7 @@ static int pxp_open(struct file *file)
v4l2_ctrl_handler_init(hdl, 4);
v4l2_ctrl_new_std(hdl, &pxp_ctrl_ops, V4L2_CID_HFLIP, 0, 1, 1, 0);
v4l2_ctrl_new_std(hdl, &pxp_ctrl_ops, V4L2_CID_VFLIP, 0, 1, 1, 0);
+ v4l2_ctrl_new_std(hdl, &pxp_ctrl_ops, V4L2_CID_ROTATE, 0, 270, 90, 0);
v4l2_ctrl_new_std(hdl, &pxp_ctrl_ops, V4L2_CID_ALPHA_COMPONENT,
0, 255, 1, 255);
if (hdl->error) {
@@ -1659,6 +1686,8 @@ static int pxp_probe(struct platform_device *pdev)
if (irq < 0)
return irq;
+ spin_lock_init(&dev->irqlock);
+
ret = devm_request_threaded_irq(&pdev->dev, irq, NULL, pxp_irq_handler,
IRQF_ONESHOT, dev_name(&pdev->dev), dev);
if (ret < 0) {
@@ -1676,8 +1705,6 @@ static int pxp_probe(struct platform_device *pdev)
goto err_clk;
}
- spin_lock_init(&dev->irqlock);
-
ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev);
if (ret)
goto err_clk;
diff --git a/drivers/media/platform/marvell-ccic/cafe-driver.c b/drivers/media/platform/marvell-ccic/cafe-driver.c
index b61b9d9..03dcf8b 100644
--- a/drivers/media/platform/marvell-ccic/cafe-driver.c
+++ b/drivers/media/platform/marvell-ccic/cafe-driver.c
@@ -139,13 +139,6 @@ struct cafe_camera {
*/
#define CAFE_SMBUS_TIMEOUT (HZ) /* generous */
-static inline struct cafe_camera *to_cam(struct v4l2_device *dev)
-{
- struct mcam_camera *m = container_of(dev, struct mcam_camera, v4l2_dev);
- return container_of(m, struct cafe_camera, mcam);
-}
-
-
static int cafe_smbus_write_done(struct mcam_camera *mcam)
{
unsigned long flags;
diff --git a/drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c b/drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c
index af994b9..f332beb 100644
--- a/drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c
+++ b/drivers/media/platform/mtk-jpeg/mtk_jpeg_core.c
@@ -1361,10 +1361,8 @@ static int mtk_jpeg_probe(struct platform_device *pdev)
}
jpeg_irq = platform_get_irq(pdev, 0);
- if (jpeg_irq < 0) {
- dev_err(&pdev->dev, "Failed to get jpeg_irq %d.\n", jpeg_irq);
+ if (jpeg_irq < 0)
return jpeg_irq;
- }
ret = devm_request_irq(&pdev->dev, jpeg_irq,
jpeg->variant->irq_handler, 0, pdev->name, jpeg);
diff --git a/drivers/media/platform/mtk-mdp/mtk_mdp_core.c b/drivers/media/platform/mtk-mdp/mtk_mdp_core.c
index 976aa1f..3d38793 100644
--- a/drivers/media/platform/mtk-mdp/mtk_mdp_core.c
+++ b/drivers/media/platform/mtk-mdp/mtk_mdp_core.c
@@ -148,7 +148,7 @@ static int mtk_mdp_probe(struct platform_device *pdev)
continue;
}
- comp_type = (enum mtk_mdp_comp_type)of_id->data;
+ comp_type = (uintptr_t)of_id->data;
comp = devm_kzalloc(dev, sizeof(*comp), GFP_KERNEL);
if (!comp) {
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.h
index 4678351..e08886a 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.h
@@ -11,6 +11,7 @@
#include <media/videobuf2-core.h>
#include <media/v4l2-mem2mem.h>
+#define VCODEC_DEC_ALIGNED_64 64
#define VCODEC_CAPABILITY_4K_DISABLED 0x10
#define VCODEC_DEC_4K_CODED_WIDTH 4096U
#define VCODEC_DEC_4K_CODED_HEIGHT 2304U
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
index e6e6a82..40c39e1 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
@@ -28,9 +28,6 @@
#define VDEC_IRQ_CLR 0x10
#define VDEC_IRQ_CFG_REG 0xa4
-module_param(mtk_v4l2_dbg_level, int, 0644);
-module_param(mtk_vcodec_dbg, bool, 0644);
-
/* Wake up context wait_queue */
static void wake_up_ctx(struct mtk_vcodec_ctx *ctx)
{
@@ -358,6 +355,8 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
if (dev->vdec_pdata->uses_stateless_api)
v4l2_m2m_unregister_media_controller(dev->m2m_dev_dec);
err_reg_cont:
+ if (dev->vdec_pdata->uses_stateless_api)
+ media_device_cleanup(&dev->mdev_dec);
destroy_workqueue(dev->decode_workqueue);
err_event_workq:
v4l2_m2m_release(dev->m2m_dev_dec);
@@ -390,7 +389,6 @@ static int mtk_vcodec_dec_remove(struct platform_device *pdev)
{
struct mtk_vcodec_dev *dev = platform_get_drvdata(pdev);
- flush_workqueue(dev->decode_workqueue);
destroy_workqueue(dev->decode_workqueue);
if (media_devnode_is_registered(dev->mdev_dec.devnode)) {
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateless.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateless.c
index 8f4a1f0..3d9f475 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateless.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateless.c
@@ -133,7 +133,6 @@ static struct vdec_fb *vdec_get_cap_buffer(struct mtk_vcodec_ctx *ctx,
struct vdec_fb *pfb = &framebuf->frame_buffer;
struct vb2_buffer *dst_buf = &vb2_v4l2->vb2_buf;
- pfb = &framebuf->frame_buffer;
pfb->base_y.va = NULL;
pfb->base_y.dma_addr = vb2_dma_contig_plane_dma_addr(dst_buf, 0);
pfb->base_y.size = ctx->q_data[MTK_Q_DATA_DST].sizeimage[0];
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
index 7457451..ffb046e 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
@@ -104,6 +104,13 @@ static int vidioc_venc_s_ctrl(struct v4l2_ctrl *ctrl)
p->gop_size = ctrl->val;
ctx->param_change |= MTK_ENCODE_PARAM_GOP_SIZE;
break;
+ case V4L2_CID_MPEG_VIDEO_VP8_PROFILE:
+ /*
+ * FIXME - what vp8 profiles are actually supported?
+ * The ctrl is added (with only profile 0 supported) for now.
+ */
+ mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_VP8_PROFILE val = %d", ctrl->val);
+ break;
case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME:
mtk_v4l2_debug(2, "V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME");
p->force_intra = 1;
@@ -257,87 +264,82 @@ static struct mtk_q_data *mtk_venc_get_q_data(struct mtk_vcodec_ctx *ctx,
return &ctx->q_data[MTK_Q_DATA_DST];
}
+static void vidioc_try_fmt_cap(struct v4l2_format *f)
+{
+ f->fmt.pix_mp.field = V4L2_FIELD_NONE;
+ f->fmt.pix_mp.num_planes = 1;
+ f->fmt.pix_mp.plane_fmt[0].bytesperline = 0;
+ f->fmt.pix_mp.flags = 0;
+}
+
/* V4L2 specification suggests the driver corrects the format struct if any of
* the dimensions is unsupported
*/
-static int vidioc_try_fmt(struct mtk_vcodec_ctx *ctx, struct v4l2_format *f,
- const struct mtk_video_fmt *fmt)
+static int vidioc_try_fmt_out(struct mtk_vcodec_ctx *ctx, struct v4l2_format *f,
+ const struct mtk_video_fmt *fmt)
{
struct v4l2_pix_format_mplane *pix_fmt_mp = &f->fmt.pix_mp;
+ int tmp_w, tmp_h;
+ unsigned int max_width, max_height;
pix_fmt_mp->field = V4L2_FIELD_NONE;
- if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
- pix_fmt_mp->num_planes = 1;
- pix_fmt_mp->plane_fmt[0].bytesperline = 0;
- } else if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
- int tmp_w, tmp_h;
- unsigned int max_width, max_height;
+ if (ctx->dev->enc_capability & MTK_VENC_4K_CAPABILITY_ENABLE) {
+ max_width = MTK_VENC_4K_MAX_W;
+ max_height = MTK_VENC_4K_MAX_H;
+ } else {
+ max_width = MTK_VENC_HD_MAX_W;
+ max_height = MTK_VENC_HD_MAX_H;
+ }
- if (ctx->dev->enc_capability & MTK_VENC_4K_CAPABILITY_ENABLE) {
- max_width = MTK_VENC_4K_MAX_W;
- max_height = MTK_VENC_4K_MAX_H;
- } else {
- max_width = MTK_VENC_HD_MAX_W;
- max_height = MTK_VENC_HD_MAX_H;
- }
+ pix_fmt_mp->height = clamp(pix_fmt_mp->height, MTK_VENC_MIN_H, max_height);
+ pix_fmt_mp->width = clamp(pix_fmt_mp->width, MTK_VENC_MIN_W, max_width);
- pix_fmt_mp->height = clamp(pix_fmt_mp->height,
- MTK_VENC_MIN_H,
- max_height);
- pix_fmt_mp->width = clamp(pix_fmt_mp->width,
- MTK_VENC_MIN_W,
- max_width);
+ /* find next closer width align 16, heign align 32, size align
+ * 64 rectangle
+ */
+ tmp_w = pix_fmt_mp->width;
+ tmp_h = pix_fmt_mp->height;
+ v4l_bound_align_image(&pix_fmt_mp->width,
+ MTK_VENC_MIN_W,
+ max_width, 4,
+ &pix_fmt_mp->height,
+ MTK_VENC_MIN_H,
+ max_height, 5, 6);
- /* find next closer width align 16, heign align 32, size align
- * 64 rectangle
- */
- tmp_w = pix_fmt_mp->width;
- tmp_h = pix_fmt_mp->height;
- v4l_bound_align_image(&pix_fmt_mp->width,
- MTK_VENC_MIN_W,
- max_width, 4,
- &pix_fmt_mp->height,
- MTK_VENC_MIN_H,
- max_height, 5, 6);
+ if (pix_fmt_mp->width < tmp_w && (pix_fmt_mp->width + 16) <= max_width)
+ pix_fmt_mp->width += 16;
+ if (pix_fmt_mp->height < tmp_h && (pix_fmt_mp->height + 32) <= max_height)
+ pix_fmt_mp->height += 32;
- if (pix_fmt_mp->width < tmp_w &&
- (pix_fmt_mp->width + 16) <= max_width)
- pix_fmt_mp->width += 16;
- if (pix_fmt_mp->height < tmp_h &&
- (pix_fmt_mp->height + 32) <= max_height)
- pix_fmt_mp->height += 32;
+ mtk_v4l2_debug(0, "before resize w=%d, h=%d, after resize w=%d, h=%d, sizeimage=%d %d",
+ tmp_w, tmp_h, pix_fmt_mp->width,
+ pix_fmt_mp->height,
+ pix_fmt_mp->plane_fmt[0].sizeimage,
+ pix_fmt_mp->plane_fmt[1].sizeimage);
- mtk_v4l2_debug(0,
- "before resize width=%d, height=%d, after resize width=%d, height=%d, sizeimage=%d %d",
- tmp_w, tmp_h, pix_fmt_mp->width,
- pix_fmt_mp->height,
- pix_fmt_mp->plane_fmt[0].sizeimage,
- pix_fmt_mp->plane_fmt[1].sizeimage);
+ pix_fmt_mp->num_planes = fmt->num_planes;
+ pix_fmt_mp->plane_fmt[0].sizeimage =
+ pix_fmt_mp->width * pix_fmt_mp->height +
+ ((ALIGN(pix_fmt_mp->width, 16) * 2) * 16);
+ pix_fmt_mp->plane_fmt[0].bytesperline = pix_fmt_mp->width;
- pix_fmt_mp->num_planes = fmt->num_planes;
- pix_fmt_mp->plane_fmt[0].sizeimage =
- pix_fmt_mp->width * pix_fmt_mp->height +
- ((ALIGN(pix_fmt_mp->width, 16) * 2) * 16);
- pix_fmt_mp->plane_fmt[0].bytesperline = pix_fmt_mp->width;
-
- if (pix_fmt_mp->num_planes == 2) {
- pix_fmt_mp->plane_fmt[1].sizeimage =
- (pix_fmt_mp->width * pix_fmt_mp->height) / 2 +
- (ALIGN(pix_fmt_mp->width, 16) * 16);
- pix_fmt_mp->plane_fmt[2].sizeimage = 0;
- pix_fmt_mp->plane_fmt[1].bytesperline =
- pix_fmt_mp->width;
- pix_fmt_mp->plane_fmt[2].bytesperline = 0;
- } else if (pix_fmt_mp->num_planes == 3) {
- pix_fmt_mp->plane_fmt[1].sizeimage =
- pix_fmt_mp->plane_fmt[2].sizeimage =
- (pix_fmt_mp->width * pix_fmt_mp->height) / 4 +
- ((ALIGN(pix_fmt_mp->width, 16) / 2) * 16);
- pix_fmt_mp->plane_fmt[1].bytesperline =
- pix_fmt_mp->plane_fmt[2].bytesperline =
- pix_fmt_mp->width / 2;
- }
+ if (pix_fmt_mp->num_planes == 2) {
+ pix_fmt_mp->plane_fmt[1].sizeimage =
+ (pix_fmt_mp->width * pix_fmt_mp->height) / 2 +
+ (ALIGN(pix_fmt_mp->width, 16) * 16);
+ pix_fmt_mp->plane_fmt[2].sizeimage = 0;
+ pix_fmt_mp->plane_fmt[1].bytesperline =
+ pix_fmt_mp->width;
+ pix_fmt_mp->plane_fmt[2].bytesperline = 0;
+ } else if (pix_fmt_mp->num_planes == 3) {
+ pix_fmt_mp->plane_fmt[1].sizeimage =
+ pix_fmt_mp->plane_fmt[2].sizeimage =
+ (pix_fmt_mp->width * pix_fmt_mp->height) / 4 +
+ ((ALIGN(pix_fmt_mp->width, 16) / 2) * 16);
+ pix_fmt_mp->plane_fmt[1].bytesperline =
+ pix_fmt_mp->plane_fmt[2].bytesperline =
+ pix_fmt_mp->width / 2;
}
pix_fmt_mp->flags = 0;
@@ -398,7 +400,7 @@ static int vidioc_venc_s_fmt_cap(struct file *file, void *priv,
struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
const struct mtk_vcodec_enc_pdata *pdata = ctx->dev->venc_pdata;
struct vb2_queue *vq;
- struct mtk_q_data *q_data;
+ struct mtk_q_data *q_data = mtk_venc_get_q_data(ctx, f->type);
int i, ret;
const struct mtk_video_fmt *fmt;
@@ -413,12 +415,6 @@ static int vidioc_venc_s_fmt_cap(struct file *file, void *priv,
return -EBUSY;
}
- q_data = mtk_venc_get_q_data(ctx, f->type);
- if (!q_data) {
- mtk_v4l2_err("fail to get q data");
- return -EINVAL;
- }
-
fmt = mtk_venc_find_format(f->fmt.pix.pixelformat, pdata);
if (!fmt) {
fmt = &ctx->dev->venc_pdata->capture_formats[0];
@@ -426,9 +422,7 @@ static int vidioc_venc_s_fmt_cap(struct file *file, void *priv,
}
q_data->fmt = fmt;
- ret = vidioc_try_fmt(ctx, f, q_data->fmt);
- if (ret)
- return ret;
+ vidioc_try_fmt_cap(f);
q_data->coded_width = f->fmt.pix_mp.width;
q_data->coded_height = f->fmt.pix_mp.height;
@@ -461,7 +455,7 @@ static int vidioc_venc_s_fmt_out(struct file *file, void *priv,
struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
const struct mtk_vcodec_enc_pdata *pdata = ctx->dev->venc_pdata;
struct vb2_queue *vq;
- struct mtk_q_data *q_data;
+ struct mtk_q_data *q_data = mtk_venc_get_q_data(ctx, f->type);
int ret, i;
const struct mtk_video_fmt *fmt;
@@ -476,19 +470,13 @@ static int vidioc_venc_s_fmt_out(struct file *file, void *priv,
return -EBUSY;
}
- q_data = mtk_venc_get_q_data(ctx, f->type);
- if (!q_data) {
- mtk_v4l2_err("fail to get q data");
- return -EINVAL;
- }
-
fmt = mtk_venc_find_format(f->fmt.pix.pixelformat, pdata);
if (!fmt) {
fmt = &ctx->dev->venc_pdata->output_formats[0];
f->fmt.pix.pixelformat = fmt->fourcc;
}
- ret = vidioc_try_fmt(ctx, f, fmt);
+ ret = vidioc_try_fmt_out(ctx, f, fmt);
if (ret)
return ret;
@@ -521,14 +509,13 @@ static int vidioc_venc_g_fmt(struct file *file, void *priv,
struct v4l2_pix_format_mplane *pix = &f->fmt.pix_mp;
struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
struct vb2_queue *vq;
- struct mtk_q_data *q_data;
+ struct mtk_q_data *q_data = mtk_venc_get_q_data(ctx, f->type);
int i;
vq = v4l2_m2m_get_vq(ctx->m2m_ctx, f->type);
if (!vq)
return -EINVAL;
- q_data = mtk_venc_get_q_data(ctx, f->type);
pix->width = q_data->coded_width;
pix->height = q_data->coded_height;
@@ -566,7 +553,9 @@ static int vidioc_try_fmt_vid_cap_mplane(struct file *file, void *priv,
f->fmt.pix_mp.quantization = ctx->quantization;
f->fmt.pix_mp.xfer_func = ctx->xfer_func;
- return vidioc_try_fmt(ctx, f, fmt);
+ vidioc_try_fmt_cap(f);
+
+ return 0;
}
static int vidioc_try_fmt_vid_out_mplane(struct file *file, void *priv,
@@ -588,22 +577,18 @@ static int vidioc_try_fmt_vid_out_mplane(struct file *file, void *priv,
f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT;
}
- return vidioc_try_fmt(ctx, f, fmt);
+ return vidioc_try_fmt_out(ctx, f, fmt);
}
static int vidioc_venc_g_selection(struct file *file, void *priv,
struct v4l2_selection *s)
{
struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
- struct mtk_q_data *q_data;
+ struct mtk_q_data *q_data = mtk_venc_get_q_data(ctx, s->type);
if (s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT)
return -EINVAL;
- q_data = mtk_venc_get_q_data(ctx, s->type);
- if (!q_data)
- return -EINVAL;
-
switch (s->target) {
case V4L2_SEL_TGT_CROP_DEFAULT:
case V4L2_SEL_TGT_CROP_BOUNDS:
@@ -629,15 +614,11 @@ static int vidioc_venc_s_selection(struct file *file, void *priv,
struct v4l2_selection *s)
{
struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
- struct mtk_q_data *q_data;
+ struct mtk_q_data *q_data = mtk_venc_get_q_data(ctx, s->type);
if (s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT)
return -EINVAL;
- q_data = mtk_venc_get_q_data(ctx, s->type);
- if (!q_data)
- return -EINVAL;
-
switch (s->target) {
case V4L2_SEL_TGT_CROP:
/* Only support crop from (0,0) */
@@ -804,11 +785,9 @@ static int vb2ops_venc_queue_setup(struct vb2_queue *vq,
struct device *alloc_devs[])
{
struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vq);
- struct mtk_q_data *q_data;
+ struct mtk_q_data *q_data = mtk_venc_get_q_data(ctx, vq->type);
unsigned int i;
- q_data = mtk_venc_get_q_data(ctx, vq->type);
-
if (q_data == NULL)
return -EINVAL;
@@ -828,11 +807,9 @@ static int vb2ops_venc_queue_setup(struct vb2_queue *vq,
static int vb2ops_venc_buf_prepare(struct vb2_buffer *vb)
{
struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
- struct mtk_q_data *q_data;
+ struct mtk_q_data *q_data = mtk_venc_get_q_data(ctx, vb->vb2_queue->type);
int i;
- q_data = mtk_venc_get_q_data(ctx, vb->vb2_queue->type);
-
for (i = 0; i < q_data->fmt->num_planes; i++) {
if (vb2_plane_size(vb, i) < q_data->sizeimage[i]) {
mtk_v4l2_err("data will not fit into plane %d (%lu < %d)",
@@ -959,7 +936,7 @@ static void vb2ops_venc_stop_streaming(struct vb2_queue *q)
if (q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
while ((dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx))) {
- dst_buf->vb2_buf.planes[0].bytesused = 0;
+ vb2_set_plane_payload(&dst_buf->vb2_buf, 0, 0);
v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_ERROR);
}
/* STREAMOFF on the CAPTURE queue completes any ongoing flush */
@@ -1068,7 +1045,7 @@ static int mtk_venc_encode_header(void *priv)
NULL, &bs_buf, &enc_result);
if (ret) {
- dst_buf->vb2_buf.planes[0].bytesused = 0;
+ vb2_set_plane_payload(&dst_buf->vb2_buf, 0, 0);
ctx->state = MTK_STATE_ABORT;
v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_ERROR);
mtk_v4l2_err("venc_if_encode failed=%d", ret);
@@ -1083,7 +1060,7 @@ static int mtk_venc_encode_header(void *priv)
}
ctx->state = MTK_STATE_HEADER;
- dst_buf->vb2_buf.planes[0].bytesused = enc_result.bs_size;
+ vb2_set_plane_payload(&dst_buf->vb2_buf, 0, enc_result.bs_size);
v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_DONE);
return 0;
@@ -1232,12 +1209,12 @@ static void mtk_venc_worker(struct work_struct *work)
if (ret) {
v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_ERROR);
- dst_buf->vb2_buf.planes[0].bytesused = 0;
+ vb2_set_plane_payload(&dst_buf->vb2_buf, 0, 0);
v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_ERROR);
mtk_v4l2_err("venc_if_encode failed=%d", ret);
} else {
v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_DONE);
- dst_buf->vb2_buf.planes[0].bytesused = enc_result.bs_size;
+ vb2_set_plane_payload(&dst_buf->vb2_buf, 0, enc_result.bs_size);
v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_DONE);
mtk_v4l2_debug(2, "venc_if_encode bs size=%d",
enc_result.bs_size);
@@ -1395,6 +1372,9 @@ int mtk_vcodec_enc_ctrls_setup(struct mtk_vcodec_ctx *ctx)
v4l2_ctrl_new_std_menu(handler, ops, V4L2_CID_MPEG_VIDEO_H264_LEVEL,
h264_max_level,
0, V4L2_MPEG_VIDEO_H264_LEVEL_4_0);
+ v4l2_ctrl_new_std_menu(handler, ops, V4L2_CID_MPEG_VIDEO_VP8_PROFILE,
+ V4L2_MPEG_VIDEO_VP8_PROFILE_0, 0, V4L2_MPEG_VIDEO_VP8_PROFILE_0);
+
if (handler->error) {
mtk_v4l2_err("Init control handler fail %d",
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
index eed6739..aeaecb8 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
@@ -23,9 +23,6 @@
#include "mtk_vcodec_util.h"
#include "mtk_vcodec_fw.h"
-module_param(mtk_v4l2_dbg_level, int, S_IRUGO | S_IWUSR);
-module_param(mtk_vcodec_dbg, bool, S_IRUGO | S_IWUSR);
-
static const struct mtk_video_fmt mtk_video_formats_output[] = {
{
.fourcc = V4L2_PIX_FMT_NV12M,
@@ -214,11 +211,11 @@ static int fops_vcodec_release(struct file *file)
mtk_v4l2_debug(1, "[%d] encoder", ctx->id);
mutex_lock(&dev->dev_mutex);
+ v4l2_m2m_ctx_release(ctx->m2m_ctx);
mtk_vcodec_enc_release(ctx);
v4l2_fh_del(&ctx->fh);
v4l2_fh_exit(&ctx->fh);
v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
- v4l2_m2m_ctx_release(ctx->m2m_ctx);
list_del_init(&ctx->list);
kfree(ctx);
@@ -461,7 +458,6 @@ static int mtk_vcodec_enc_remove(struct platform_device *pdev)
struct mtk_vcodec_dev *dev = platform_get_drvdata(pdev);
mtk_v4l2_debug_enter();
- flush_workqueue(dev->encode_workqueue);
destroy_workqueue(dev->encode_workqueue);
if (dev->m2m_dev_enc)
v4l2_m2m_release(dev->m2m_dev_enc);
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
index ac5973b..5bac820 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
@@ -10,16 +10,6 @@
#include "mtk_vcodec_drv.h"
#include "mtk_vcodec_util.h"
-/* For encoder, this will enable logs in venc/*/
-bool mtk_vcodec_dbg;
-EXPORT_SYMBOL(mtk_vcodec_dbg);
-
-/* The log level of v4l2 encoder or decoder driver.
- * That is, files under mtk-vcodec/.
- */
-int mtk_v4l2_dbg_level;
-EXPORT_SYMBOL(mtk_v4l2_dbg_level);
-
void __iomem *mtk_vcodec_get_reg_addr(struct mtk_vcodec_ctx *data,
unsigned int reg_idx)
{
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
index b999d7b..87c3d6d 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
@@ -25,54 +25,29 @@ struct mtk_vcodec_fb {
struct mtk_vcodec_ctx;
struct mtk_vcodec_dev;
-extern int mtk_v4l2_dbg_level;
-extern bool mtk_vcodec_dbg;
-
+#undef pr_fmt
+#define pr_fmt(fmt) "%s(),%d: " fmt, __func__, __LINE__
#define mtk_v4l2_err(fmt, args...) \
- pr_err("[MTK_V4L2][ERROR] %s:%d: " fmt "\n", __func__, __LINE__, \
- ##args)
+ pr_err("[MTK_V4L2][ERROR] " fmt "\n", ##args)
-#define mtk_vcodec_err(h, fmt, args...) \
- pr_err("[MTK_VCODEC][ERROR][%d]: %s() " fmt "\n", \
- ((struct mtk_vcodec_ctx *)h->ctx)->id, __func__, ##args)
+#define mtk_vcodec_err(h, fmt, args...) \
+ pr_err("[MTK_VCODEC][ERROR][%d]: " fmt "\n", \
+ ((struct mtk_vcodec_ctx *)(h)->ctx)->id, ##args)
-#if defined(DEBUG)
-
-#define mtk_v4l2_debug(level, fmt, args...) \
- do { \
- if (mtk_v4l2_dbg_level >= level) \
- pr_info("[MTK_V4L2] level=%d %s(),%d: " fmt "\n",\
- level, __func__, __LINE__, ##args); \
- } while (0)
+#define mtk_v4l2_debug(level, fmt, args...) pr_debug(fmt, ##args)
#define mtk_v4l2_debug_enter() mtk_v4l2_debug(3, "+")
#define mtk_v4l2_debug_leave() mtk_v4l2_debug(3, "-")
-#define mtk_vcodec_debug(h, fmt, args...) \
- do { \
- if (mtk_vcodec_dbg) \
- pr_info("[MTK_VCODEC][%d]: %s() " fmt "\n", \
- ((struct mtk_vcodec_ctx *)h->ctx)->id, \
- __func__, ##args); \
- } while (0)
+#define mtk_vcodec_debug(h, fmt, args...) \
+ pr_debug("[MTK_VCODEC][%d]: " fmt "\n", \
+ ((struct mtk_vcodec_ctx *)(h)->ctx)->id, ##args)
#define mtk_vcodec_debug_enter(h) mtk_vcodec_debug(h, "+")
#define mtk_vcodec_debug_leave(h) mtk_vcodec_debug(h, "-")
-#else
-
-#define mtk_v4l2_debug(level, fmt, args...) {}
-#define mtk_v4l2_debug_enter() {}
-#define mtk_v4l2_debug_leave() {}
-
-#define mtk_vcodec_debug(h, fmt, args...) {}
-#define mtk_vcodec_debug_enter(h) {}
-#define mtk_vcodec_debug_leave(h) {}
-
-#endif
-
void __iomem *mtk_vcodec_get_reg_addr(struct mtk_vcodec_ctx *data,
unsigned int reg_idx);
int mtk_vcodec_mem_alloc(struct mtk_vcodec_ctx *data,
diff --git a/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c b/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c
index 946c230..fada4d1 100644
--- a/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c
+++ b/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c
@@ -527,7 +527,7 @@ static int alloc_mv_buf(struct vdec_h264_slice_inst *inst,
struct mtk_vcodec_mem *mem = NULL;
unsigned int buf_sz = get_mv_buf_size(pic->buf_w, pic->buf_h);
- mtk_v4l2_debug(3, "size = 0x%lx", buf_sz);
+ mtk_v4l2_debug(3, "size = 0x%x", buf_sz);
for (i = 0; i < H264_MAX_MV_NUM; i++) {
mem = &inst->mv_buf[i];
if (mem->va)
@@ -562,8 +562,8 @@ static void get_pic_info(struct vdec_h264_slice_inst *inst,
{
struct mtk_vcodec_ctx *ctx = inst->ctx;
- ctx->picinfo.buf_w = (ctx->picinfo.pic_w + 15) & 0xFFFFFFF0;
- ctx->picinfo.buf_h = (ctx->picinfo.pic_h + 31) & 0xFFFFFFE0;
+ ctx->picinfo.buf_w = ALIGN(ctx->picinfo.pic_w, VCODEC_DEC_ALIGNED_64);
+ ctx->picinfo.buf_h = ALIGN(ctx->picinfo.pic_h, VCODEC_DEC_ALIGNED_64);
ctx->picinfo.fb_sz[0] = ctx->picinfo.buf_w * ctx->picinfo.buf_h;
ctx->picinfo.fb_sz[1] = ctx->picinfo.fb_sz[0] >> 1;
inst->vsi_ctx.dec.cap_num_planes =
@@ -637,7 +637,7 @@ static int vdec_h264_slice_init(struct mtk_vcodec_ctx *ctx)
if (err)
goto error_deinit;
- mtk_vcodec_debug(inst, "struct size = %d,%d,%d,%d\n",
+ mtk_vcodec_debug(inst, "struct size = %zu,%zu,%zu,%zu\n",
sizeof(struct mtk_h264_sps_param),
sizeof(struct mtk_h264_pps_param),
sizeof(struct mtk_h264_dec_slice_param),
diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
index b6a4f20..bf03888 100644
--- a/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
@@ -367,8 +367,7 @@ static int h264_encode_sps(struct venc_h264_inst *inst,
mtk_vcodec_debug_enter(inst);
- ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_SPS, NULL,
- bs_buf, bs_size, NULL);
+ ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_SPS, NULL, bs_buf, NULL);
if (ret)
return ret;
@@ -394,8 +393,7 @@ static int h264_encode_pps(struct venc_h264_inst *inst,
mtk_vcodec_debug_enter(inst);
- ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_PPS, NULL,
- bs_buf, bs_size, NULL);
+ ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_PPS, NULL, bs_buf, NULL);
if (ret)
return ret;
@@ -451,8 +449,7 @@ static int h264_encode_frame(struct venc_h264_inst *inst,
mtk_vcodec_debug(inst, "frm_count = %d,skip_frm_count =%d,frm_type=%d.\n",
frame_info.frm_count, frame_info.skip_frm_count,
frame_info.frm_type);
- ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_FRAME, frm_buf,
- bs_buf, bs_size, &frame_info);
+ ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_FRAME, frm_buf, bs_buf, &frame_info);
if (ret)
return ret;
diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c
index 8267a9c..6b66957 100644
--- a/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c
@@ -302,8 +302,7 @@ static int vp8_enc_encode_frame(struct venc_vp8_inst *inst,
mtk_vcodec_debug(inst, "->frm_cnt=%d", inst->frm_cnt);
- ret = vpu_enc_encode(&inst->vpu_inst, 0, frm_buf, bs_buf, bs_size,
- NULL);
+ ret = vpu_enc_encode(&inst->vpu_inst, 0, frm_buf, bs_buf, NULL);
if (ret)
return ret;
diff --git a/drivers/media/platform/mtk-vcodec/venc_vpu_if.c b/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
index be6d879..e7899d8 100644
--- a/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
@@ -225,7 +225,6 @@ int vpu_enc_set_param(struct venc_vpu_inst *vpu,
int vpu_enc_encode(struct venc_vpu_inst *vpu, unsigned int bs_mode,
struct venc_frm_buf *frm_buf,
struct mtk_vcodec_mem *bs_buf,
- unsigned int *bs_size,
struct venc_frame_info *frame_info)
{
const bool is_ext = MTK_ENC_CTX_IS_EXT(vpu->ctx);
diff --git a/drivers/media/platform/mtk-vcodec/venc_vpu_if.h b/drivers/media/platform/mtk-vcodec/venc_vpu_if.h
index f9be9ca..f83bc1b 100644
--- a/drivers/media/platform/mtk-vcodec/venc_vpu_if.h
+++ b/drivers/media/platform/mtk-vcodec/venc_vpu_if.h
@@ -45,7 +45,6 @@ int vpu_enc_set_param(struct venc_vpu_inst *vpu,
int vpu_enc_encode(struct venc_vpu_inst *vpu, unsigned int bs_mode,
struct venc_frm_buf *frm_buf,
struct mtk_vcodec_mem *bs_buf,
- unsigned int *bs_size,
struct venc_frame_info *frame_info);
int vpu_enc_deinit(struct venc_vpu_inst *vpu);
diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu.c b/drivers/media/platform/mtk-vpu/mtk_vpu.c
index 7f1647d..7bd715fc 100644
--- a/drivers/media/platform/mtk-vpu/mtk_vpu.c
+++ b/drivers/media/platform/mtk-vpu/mtk_vpu.c
@@ -964,10 +964,8 @@ static int mtk_vpu_remove(struct platform_device *pdev)
#ifdef CONFIG_DEBUG_FS
debugfs_remove(vpu_debugfs);
#endif
- if (vpu->wdt.wq) {
- flush_workqueue(vpu->wdt.wq);
+ if (vpu->wdt.wq)
destroy_workqueue(vpu->wdt.wq);
- }
vpu_free_ext_mem(vpu, P_FW);
vpu_free_ext_mem(vpu, D_FW);
mutex_destroy(&vpu->vpu_mutex);
diff --git a/drivers/media/platform/omap3isp/isp.c b/drivers/media/platform/omap3isp/isp.c
index 6de377c..4c937f3 100644
--- a/drivers/media/platform/omap3isp/isp.c
+++ b/drivers/media/platform/omap3isp/isp.c
@@ -476,7 +476,8 @@ void omap3isp_hist_dma_done(struct isp_device *isp)
}
}
-static inline void isp_isr_dbg(struct isp_device *isp, u32 irqstatus)
+static inline void __maybe_unused isp_isr_dbg(struct isp_device *isp,
+ u32 irqstatus)
{
static const char *name[] = {
"CSIA_IRQ",
diff --git a/drivers/media/platform/omap3isp/isph3a_af.c b/drivers/media/platform/omap3isp/isph3a_af.c
index a65cfdf..de7b116 100644
--- a/drivers/media/platform/omap3isp/isph3a_af.c
+++ b/drivers/media/platform/omap3isp/isph3a_af.c
@@ -21,7 +21,7 @@
#include "ispstat.h"
#define IS_OUT_OF_BOUNDS(value, min, max) \
- (((value) < (min)) || ((value) > (max)))
+ ((((unsigned int)value) < (min)) || (((unsigned int)value) > (max)))
static void h3a_af_setup_regs(struct ispstat *af, void *priv)
{
diff --git a/drivers/media/platform/omap3isp/omap3isp.h b/drivers/media/platform/omap3isp/omap3isp.h
index 214f94c4..4a003c8 100644
--- a/drivers/media/platform/omap3isp/omap3isp.h
+++ b/drivers/media/platform/omap3isp/omap3isp.h
@@ -95,6 +95,8 @@ struct isp_csiphy_lanes_cfg {
* ISP_CCP2_PHY_DATA_CLOCK - Data/clock physical layer
* ISP_CCP2_PHY_DATA_STROBE - Data/strobe physical layer
* @vpclk_div: Video port output clock control
+ * @vp_clk_pol: Video port output clock polarity
+ * @lanecfg: CCP2/CSI2 lane configuration
*/
struct isp_ccp2_cfg {
unsigned int strobe_clk_pol:1;
diff --git a/drivers/media/platform/qcom/camss/camss-vfe-170.c b/drivers/media/platform/qcom/camss/camss-vfe-170.c
index 5c083d7..f524af7 100644
--- a/drivers/media/platform/qcom/camss/camss-vfe-170.c
+++ b/drivers/media/platform/qcom/camss/camss-vfe-170.c
@@ -191,13 +191,6 @@ static u32 vfe_hw_version(struct vfe_device *vfe)
return hw_version;
}
-static inline void vfe_reg_clr(struct vfe_device *vfe, u32 reg, u32 clr_bits)
-{
- u32 bits = readl_relaxed(vfe->base + reg);
-
- writel_relaxed(bits & ~clr_bits, vfe->base + reg);
-}
-
static inline void vfe_reg_set(struct vfe_device *vfe, u32 reg, u32 set_bits)
{
u32 bits = readl_relaxed(vfe->base + reg);
diff --git a/drivers/media/platform/qcom/venus/core.c b/drivers/media/platform/qcom/venus/core.c
index f5fa818..877eca1 100644
--- a/drivers/media/platform/qcom/venus/core.c
+++ b/drivers/media/platform/qcom/venus/core.c
@@ -350,11 +350,11 @@ static int venus_probe(struct platform_device *pdev)
ret = venus_firmware_init(core);
if (ret)
- goto err_runtime_disable;
+ goto err_of_depopulate;
ret = venus_boot(core);
if (ret)
- goto err_runtime_disable;
+ goto err_firmware_deinit;
ret = hfi_core_resume(core, true);
if (ret)
@@ -386,6 +386,10 @@ static int venus_probe(struct platform_device *pdev)
v4l2_device_unregister(&core->v4l2_dev);
err_venus_shutdown:
venus_shutdown(core);
+err_firmware_deinit:
+ venus_firmware_deinit(core);
+err_of_depopulate:
+ of_platform_depopulate(dev);
err_runtime_disable:
pm_runtime_put_noidle(dev);
pm_runtime_set_suspended(dev);
@@ -473,7 +477,8 @@ static __maybe_unused int venus_runtime_suspend(struct device *dev)
err_video_path:
icc_set_bw(core->cpucfg_path, kbps_to_icc(1000), 0);
err_cpucfg_path:
- pm_ops->core_power(core, POWER_ON);
+ if (pm_ops->core_power)
+ pm_ops->core_power(core, POWER_ON);
return ret;
}
diff --git a/drivers/media/platform/qcom/venus/pm_helpers.c b/drivers/media/platform/qcom/venus/pm_helpers.c
index cedc664..cb48c5f 100644
--- a/drivers/media/platform/qcom/venus/pm_helpers.c
+++ b/drivers/media/platform/qcom/venus/pm_helpers.c
@@ -163,14 +163,12 @@ static u32 load_per_type(struct venus_core *core, u32 session_type)
struct venus_inst *inst = NULL;
u32 mbs_per_sec = 0;
- mutex_lock(&core->lock);
list_for_each_entry(inst, &core->instances, list) {
if (inst->session_type != session_type)
continue;
mbs_per_sec += load_per_instance(inst);
}
- mutex_unlock(&core->lock);
return mbs_per_sec;
}
@@ -219,14 +217,12 @@ static int load_scale_bw(struct venus_core *core)
struct venus_inst *inst = NULL;
u32 mbs_per_sec, avg, peak, total_avg = 0, total_peak = 0;
- mutex_lock(&core->lock);
list_for_each_entry(inst, &core->instances, list) {
mbs_per_sec = load_per_instance(inst);
mbs_to_bw(inst, mbs_per_sec, &avg, &peak);
total_avg += avg;
total_peak += peak;
}
- mutex_unlock(&core->lock);
/*
* keep minimum bandwidth vote for "video-mem" path,
@@ -253,8 +249,9 @@ static int load_scale_v1(struct venus_inst *inst)
struct device *dev = core->dev;
u32 mbs_per_sec;
unsigned int i;
- int ret;
+ int ret = 0;
+ mutex_lock(&core->lock);
mbs_per_sec = load_per_type(core, VIDC_SESSION_TYPE_ENC) +
load_per_type(core, VIDC_SESSION_TYPE_DEC);
@@ -279,17 +276,19 @@ static int load_scale_v1(struct venus_inst *inst)
if (ret) {
dev_err(dev, "failed to set clock rate %lu (%d)\n",
freq, ret);
- return ret;
+ goto exit;
}
ret = load_scale_bw(core);
if (ret) {
dev_err(dev, "failed to set bandwidth (%d)\n",
ret);
- return ret;
+ goto exit;
}
- return 0;
+exit:
+ mutex_unlock(&core->lock);
+ return ret;
}
static int core_get_v1(struct venus_core *core)
@@ -587,8 +586,8 @@ min_loaded_core(struct venus_inst *inst, u32 *min_coreid, u32 *min_load, bool lo
if (inst->session_type == VIDC_SESSION_TYPE_DEC)
vpp_freq = inst_pos->clk_data.vpp_freq;
else if (inst->session_type == VIDC_SESSION_TYPE_ENC)
- vpp_freq = low_power ? inst_pos->clk_data.vpp_freq :
- inst_pos->clk_data.low_power_freq;
+ vpp_freq = low_power ? inst_pos->clk_data.low_power_freq :
+ inst_pos->clk_data.vpp_freq;
else
continue;
@@ -1116,13 +1115,13 @@ static int load_scale_v4(struct venus_inst *inst)
struct device *dev = core->dev;
unsigned long freq = 0, freq_core1 = 0, freq_core2 = 0;
unsigned long filled_len = 0;
- int i, ret;
+ int i, ret = 0;
for (i = 0; i < inst->num_input_bufs; i++)
filled_len = max(filled_len, inst->payloads[i]);
if (inst->session_type == VIDC_SESSION_TYPE_DEC && !filled_len)
- return 0;
+ return ret;
freq = calculate_inst_freq(inst, filled_len);
inst->clk_data.freq = freq;
@@ -1138,7 +1137,6 @@ static int load_scale_v4(struct venus_inst *inst)
freq_core2 += inst->clk_data.freq;
}
}
- mutex_unlock(&core->lock);
freq = max(freq_core1, freq_core2);
@@ -1163,17 +1161,19 @@ static int load_scale_v4(struct venus_inst *inst)
if (ret) {
dev_err(dev, "failed to set clock rate %lu (%d)\n",
freq, ret);
- return ret;
+ goto exit;
}
ret = load_scale_bw(core);
if (ret) {
dev_err(dev, "failed to set bandwidth (%d)\n",
ret);
- return ret;
+ goto exit;
}
- return 0;
+exit:
+ mutex_unlock(&core->lock);
+ return ret;
}
static const struct venus_pm_ops pm_ops_v4 = {
diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c
index 1d92cc8..0186ae2 100644
--- a/drivers/media/platform/rcar-vin/rcar-core.c
+++ b/drivers/media/platform/rcar-vin/rcar-core.c
@@ -349,12 +349,10 @@ static int rvin_group_parse_of(struct rvin_dev *vin, unsigned int port,
static void rvin_group_notifier_cleanup(struct rvin_dev *vin)
{
- mutex_lock(&vin->group->lock);
if (&vin->v4l2_dev == vin->group->notifier.v4l2_dev) {
v4l2_async_nf_unregister(&vin->group->notifier);
v4l2_async_nf_cleanup(&vin->group->notifier);
}
- mutex_unlock(&vin->group->lock);
}
static int rvin_group_notifier_init(struct rvin_dev *vin, unsigned int port,
@@ -1620,6 +1618,7 @@ static SIMPLE_DEV_PM_OPS(rvin_pm_ops, rvin_suspend, rvin_resume);
static struct platform_driver rcar_vin_driver = {
.driver = {
.name = "rcar-vin",
+ .suppress_bind_attrs = true,
.pm = &rvin_pm_ops,
.of_match_table = rvin_of_id_table,
},
diff --git a/drivers/media/platform/rcar-vin/rcar-csi2.c b/drivers/media/platform/rcar-vin/rcar-csi2.c
index 11848d0..8c939cb 100644
--- a/drivers/media/platform/rcar-vin/rcar-csi2.c
+++ b/drivers/media/platform/rcar-vin/rcar-csi2.c
@@ -542,16 +542,27 @@ static int rcsi2_wait_phy_start(struct rcar_csi2 *priv,
static int rcsi2_set_phypll(struct rcar_csi2 *priv, unsigned int mbps)
{
const struct rcsi2_mbps_reg *hsfreq;
+ const struct rcsi2_mbps_reg *hsfreq_prev = NULL;
- for (hsfreq = priv->info->hsfreqrange; hsfreq->mbps != 0; hsfreq++)
+ if (mbps < priv->info->hsfreqrange->mbps)
+ dev_warn(priv->dev, "%u Mbps less than min PHY speed %u Mbps",
+ mbps, priv->info->hsfreqrange->mbps);
+
+ for (hsfreq = priv->info->hsfreqrange; hsfreq->mbps != 0; hsfreq++) {
if (hsfreq->mbps >= mbps)
break;
+ hsfreq_prev = hsfreq;
+ }
if (!hsfreq->mbps) {
dev_err(priv->dev, "Unsupported PHY speed (%u Mbps)", mbps);
return -ERANGE;
}
+ if (hsfreq_prev &&
+ ((mbps - hsfreq_prev->mbps) <= (hsfreq->mbps - mbps)))
+ hsfreq = hsfreq_prev;
+
rcsi2_write(priv, PHYPLL_REG, PHYPLL_HSFREQRANGE(hsfreq->reg));
return 0;
@@ -1097,10 +1108,17 @@ static int rcsi2_phtw_write_mbps(struct rcar_csi2 *priv, unsigned int mbps,
const struct rcsi2_mbps_reg *values, u16 code)
{
const struct rcsi2_mbps_reg *value;
+ const struct rcsi2_mbps_reg *prev_value = NULL;
- for (value = values; value->mbps; value++)
+ for (value = values; value->mbps; value++) {
if (value->mbps >= mbps)
break;
+ prev_value = value;
+ }
+
+ if (prev_value &&
+ ((mbps - prev_value->mbps) <= (value->mbps - mbps)))
+ value = prev_value;
if (!value->mbps) {
dev_err(priv->dev, "Unsupported PHY speed (%u Mbps)", mbps);
@@ -1498,6 +1516,7 @@ static struct platform_driver rcar_csi2_pdrv = {
.probe = rcsi2_probe,
.driver = {
.name = "rcar-csi2",
+ .suppress_bind_attrs = true,
.of_match_table = rcar_csi2_of_table,
},
};
diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c b/drivers/media/platform/rcar-vin/rcar-dma.c
index 25ead93..8136bc7 100644
--- a/drivers/media/platform/rcar-vin/rcar-dma.c
+++ b/drivers/media/platform/rcar-vin/rcar-dma.c
@@ -111,6 +111,9 @@
#define VNIE_FIE (1 << 4)
#define VNIE_EFE (1 << 1)
+/* Video n Interrupt Status Register bits */
+#define VNINTS_FIS (1 << 4)
+
/* Video n Data Mode Register bits */
#define VNDMR_A8BIT(n) (((n) & 0xff) << 24)
#define VNDMR_A8BIT_MASK (0xff << 24)
@@ -1005,6 +1008,10 @@ static irqreturn_t rvin_irq(int irq, void *data)
rvin_ack_interrupt(vin);
handled = 1;
+ /* Nothing to do if nothing was captured. */
+ if (!(int_status & VNINTS_FIS))
+ goto done;
+
/* Nothing to do if capture status is 'STOPPED' */
if (vin->state == STOPPED) {
vin_dbg(vin, "IRQ while state stopped\n");
@@ -1371,6 +1378,16 @@ void rvin_stop_streaming(struct rvin_dev *vin)
spin_unlock_irqrestore(&vin->qlock, flags);
+ /* If something went wrong, free buffers with an error. */
+ if (!buffersFreed) {
+ return_unused_buffers(vin, VB2_BUF_STATE_ERROR);
+ for (i = 0; i < HW_BUFFER_NUM; i++) {
+ if (vin->buf_hw[i].buffer)
+ vb2_buffer_done(&vin->buf_hw[i].buffer->vb2_buf,
+ VB2_BUF_STATE_ERROR);
+ }
+ }
+
rvin_set_stream(vin, 0);
/* disable interrupts */
diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
index a5bfa76..2e60b9f 100644
--- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
+++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
@@ -179,20 +179,27 @@ static void rvin_format_align(struct rvin_dev *vin, struct v4l2_pix_format *pix)
break;
}
- /* HW limit width to a multiple of 32 (2^5) for NV12/16 else 2 (2^1) */
+ /* Hardware limits width alignment based on format. */
switch (pix->pixelformat) {
+ /* Multiple of 32 (2^5) for NV12/16. */
case V4L2_PIX_FMT_NV12:
case V4L2_PIX_FMT_NV16:
walign = 5;
break;
- default:
+ /* Multiple of 2 (2^1) for YUV. */
+ case V4L2_PIX_FMT_YUYV:
+ case V4L2_PIX_FMT_UYVY:
walign = 1;
break;
+ /* No multiple for RGB. */
+ default:
+ walign = 0;
+ break;
}
/* Limit to VIN capabilities */
- v4l_bound_align_image(&pix->width, 2, vin->info->max_width, walign,
- &pix->height, 4, vin->info->max_height, 2, 0);
+ v4l_bound_align_image(&pix->width, 5, vin->info->max_width, walign,
+ &pix->height, 2, vin->info->max_height, 0, 0);
pix->bytesperline = rvin_format_bytesperline(vin, pix);
pix->sizeimage = rvin_format_sizeimage(pix);
diff --git a/drivers/media/platform/rcar_fdp1.c b/drivers/media/platform/rcar_fdp1.c
index 19de3c1..37ecf48 100644
--- a/drivers/media/platform/rcar_fdp1.c
+++ b/drivers/media/platform/rcar_fdp1.c
@@ -2287,11 +2287,10 @@ static int fdp1_probe(struct platform_device *pdev)
return PTR_ERR(fdp1->regs);
/* Interrupt service routine registration */
- fdp1->irq = ret = platform_get_irq(pdev, 0);
- if (ret < 0) {
- dev_err(&pdev->dev, "cannot find IRQ\n");
+ ret = platform_get_irq(pdev, 0);
+ if (ret < 0)
return ret;
- }
+ fdp1->irq = ret;
ret = devm_request_irq(&pdev->dev, fdp1->irq, fdp1_irq_handler, 0,
dev_name(&pdev->dev), fdp1);
diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-dev.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-dev.c
index 50b166c..3f5cfa7 100644
--- a/drivers/media/platform/rockchip/rkisp1/rkisp1-dev.c
+++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-dev.c
@@ -462,7 +462,7 @@ static void rkisp1_debug_init(struct rkisp1_device *rkisp1)
{
struct rkisp1_debug *debug = &rkisp1->debug;
- debug->debugfs_dir = debugfs_create_dir(RKISP1_DRIVER_NAME, NULL);
+ debug->debugfs_dir = debugfs_create_dir(dev_name(rkisp1->dev), NULL);
debugfs_create_ulong("data_loss", 0444, debug->debugfs_dir,
&debug->data_loss);
debugfs_create_ulong("outform_size_err", 0444, debug->debugfs_dir,
diff --git a/drivers/media/platform/s3c-camif/camif-core.c b/drivers/media/platform/s3c-camif/camif-core.c
index 32892ab..6e8ef86 100644
--- a/drivers/media/platform/s3c-camif/camif-core.c
+++ b/drivers/media/platform/s3c-camif/camif-core.c
@@ -291,7 +291,6 @@ static void camif_unregister_media_entities(struct camif_dev *camif)
{
camif_unregister_video_nodes(camif);
camif_unregister_sensor(camif);
- s3c_camif_unregister_subdev(camif);
}
/*
@@ -520,6 +519,7 @@ static int s3c_camif_remove(struct platform_device *pdev)
pm_runtime_disable(&pdev->dev);
camif_clk_put(camif);
+ s3c_camif_unregister_subdev(camif);
pdata->gpio_put();
return 0;
diff --git a/drivers/media/platform/s5p-jpeg/jpeg-core.c b/drivers/media/platform/s5p-jpeg/jpeg-core.c
index ebdfd24..a8d9159 100644
--- a/drivers/media/platform/s5p-jpeg/jpeg-core.c
+++ b/drivers/media/platform/s5p-jpeg/jpeg-core.c
@@ -2425,17 +2425,17 @@ static int s5p_jpeg_job_ready(void *priv)
return 1;
}
-static struct v4l2_m2m_ops s5p_jpeg_m2m_ops = {
+static const struct v4l2_m2m_ops s5p_jpeg_m2m_ops = {
.device_run = s5p_jpeg_device_run,
.job_ready = s5p_jpeg_job_ready,
};
-static struct v4l2_m2m_ops exynos3250_jpeg_m2m_ops = {
+static const struct v4l2_m2m_ops exynos3250_jpeg_m2m_ops = {
.device_run = exynos3250_jpeg_device_run,
.job_ready = s5p_jpeg_job_ready,
};
-static struct v4l2_m2m_ops exynos4_jpeg_m2m_ops = {
+static const struct v4l2_m2m_ops exynos4_jpeg_m2m_ops = {
.device_run = exynos4_jpeg_device_run,
.job_ready = s5p_jpeg_job_ready,
};
diff --git a/drivers/media/platform/s5p-jpeg/jpeg-core.h b/drivers/media/platform/s5p-jpeg/jpeg-core.h
index 8473a01..4a5fb1b 100644
--- a/drivers/media/platform/s5p-jpeg/jpeg-core.h
+++ b/drivers/media/platform/s5p-jpeg/jpeg-core.h
@@ -142,7 +142,7 @@ struct s5p_jpeg_variant {
unsigned int hw3250_compat:1;
unsigned int htbl_reinit:1;
unsigned int hw_ex4_compat:1;
- struct v4l2_m2m_ops *m2m_ops;
+ const struct v4l2_m2m_ops *m2m_ops;
irqreturn_t (*jpeg_irq)(int irq, void *priv);
const char *clk_names[JPEG_MAX_CLOCKS];
int num_clocks;
diff --git a/drivers/media/platform/s5p-mfc/s5p_mfc.c b/drivers/media/platform/s5p-mfc/s5p_mfc.c
index fc85e4e..f6732f0 100644
--- a/drivers/media/platform/s5p-mfc/s5p_mfc.c
+++ b/drivers/media/platform/s5p-mfc/s5p_mfc.c
@@ -1185,7 +1185,6 @@ static int s5p_mfc_configure_common_memory(struct s5p_mfc_dev *mfc_dev)
{
struct device *dev = &mfc_dev->plat_dev->dev;
unsigned long mem_size = SZ_4M;
- unsigned int bitmap_size;
if (IS_ENABLED(CONFIG_DMA_CMA) || exynos_is_iommu_available(dev))
mem_size = SZ_8M;
@@ -1193,16 +1192,14 @@ static int s5p_mfc_configure_common_memory(struct s5p_mfc_dev *mfc_dev)
if (mfc_mem_size)
mem_size = memparse(mfc_mem_size, NULL);
- bitmap_size = BITS_TO_LONGS(mem_size >> PAGE_SHIFT) * sizeof(long);
-
- mfc_dev->mem_bitmap = kzalloc(bitmap_size, GFP_KERNEL);
+ mfc_dev->mem_bitmap = bitmap_zalloc(mem_size >> PAGE_SHIFT, GFP_KERNEL);
if (!mfc_dev->mem_bitmap)
return -ENOMEM;
mfc_dev->mem_virt = dma_alloc_coherent(dev, mem_size,
&mfc_dev->mem_base, GFP_KERNEL);
if (!mfc_dev->mem_virt) {
- kfree(mfc_dev->mem_bitmap);
+ bitmap_free(mfc_dev->mem_bitmap);
dev_err(dev, "failed to preallocate %ld MiB for the firmware and context buffers\n",
(mem_size / SZ_1M));
return -ENOMEM;
@@ -1241,7 +1238,7 @@ static void s5p_mfc_unconfigure_common_memory(struct s5p_mfc_dev *mfc_dev)
dma_free_coherent(dev, mfc_dev->mem_size, mfc_dev->mem_virt,
mfc_dev->mem_base);
- kfree(mfc_dev->mem_bitmap);
+ bitmap_free(mfc_dev->mem_bitmap);
vb2_dma_contig_clear_max_seg_size(dev);
}
diff --git a/drivers/media/platform/sti/bdisp/bdisp-v4l2.c b/drivers/media/platform/sti/bdisp/bdisp-v4l2.c
index 7d467f2..01ce7b7 100644
--- a/drivers/media/platform/sti/bdisp/bdisp-v4l2.c
+++ b/drivers/media/platform/sti/bdisp/bdisp-v4l2.c
@@ -1394,6 +1394,7 @@ static int bdisp_probe(struct platform_device *pdev)
err_pm:
pm_runtime_put(dev);
err_remove:
+ pm_runtime_disable(dev);
bdisp_debugfs_remove(bdisp);
v4l2_device_unregister(&bdisp->v4l2_dev);
err_clk:
diff --git a/drivers/media/platform/sti/c8sectpfe/c8sectpfe-core.c b/drivers/media/platform/sti/c8sectpfe/c8sectpfe-core.c
index 02dc78b..7bb1384 100644
--- a/drivers/media/platform/sti/c8sectpfe/c8sectpfe-core.c
+++ b/drivers/media/platform/sti/c8sectpfe/c8sectpfe-core.c
@@ -925,17 +925,12 @@ static int c8sectpfe_remove(struct platform_device *pdev)
static int configure_channels(struct c8sectpfei *fei)
{
int index = 0, ret;
- struct channel_info *tsin;
struct device_node *child, *np = fei->dev->of_node;
/* iterate round each tsin and configure memdma descriptor and IB hw */
for_each_child_of_node(np, child) {
-
- tsin = fei->channel_data[index];
-
ret = configure_memdma_and_inputblock(fei,
fei->channel_data[index]);
-
if (ret) {
dev_err(fei->dev,
"configure_memdma_and_inputblock failed\n");
@@ -947,10 +942,9 @@ static int configure_channels(struct c8sectpfei *fei)
return 0;
err_unmap:
- for (index = 0; index < fei->tsin_count; index++) {
- tsin = fei->channel_data[index];
- free_input_block(fei, tsin);
- }
+ while (--index >= 0)
+ free_input_block(fei, fei->channel_data[index]);
+
return ret;
}
diff --git a/drivers/media/platform/sti/hva/hva-hw.c b/drivers/media/platform/sti/hva/hva-hw.c
index 15e8f83..fe4ea2e 100644
--- a/drivers/media/platform/sti/hva/hva-hw.c
+++ b/drivers/media/platform/sti/hva/hva-hw.c
@@ -385,7 +385,7 @@ int hva_hw_probe(struct platform_device *pdev, struct hva_dev *hva)
ret = pm_runtime_resume_and_get(dev);
if (ret < 0) {
dev_err(dev, "%s failed to set PM\n", HVA_PREFIX);
- goto err_clk;
+ goto err_disable;
}
/* check IP hardware version */
@@ -403,6 +403,8 @@ int hva_hw_probe(struct platform_device *pdev, struct hva_dev *hva)
err_pm:
pm_runtime_put(dev);
+err_disable:
+ pm_runtime_disable(dev);
err_clk:
if (hva->clk)
clk_unprepare(hva->clk);
diff --git a/drivers/media/platform/stm32/Makefile b/drivers/media/platform/stm32/Makefile
index 48b36db..896ef98 100644
--- a/drivers/media/platform/stm32/Makefile
+++ b/drivers/media/platform/stm32/Makefile
@@ -1,2 +1,4 @@
# SPDX-License-Identifier: GPL-2.0-only
obj-$(CONFIG_VIDEO_STM32_DCMI) += stm32-dcmi.o
+stm32-dma2d-objs := dma2d/dma2d.o dma2d/dma2d-hw.o
+obj-$(CONFIG_VIDEO_STM32_DMA2D) += stm32-dma2d.o
diff --git a/drivers/media/platform/stm32/dma2d/dma2d-hw.c b/drivers/media/platform/stm32/dma2d/dma2d-hw.c
new file mode 100644
index 0000000..ea4cc84
--- /dev/null
+++ b/drivers/media/platform/stm32/dma2d/dma2d-hw.c
@@ -0,0 +1,133 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * ST stm32 Chrom-Art - 2D Graphics Accelerator Driver
+ *
+ * Copyright (c) 2021 Dillon Min
+ * Dillon Min, <dillon.minfei@gmail.com>
+ *
+ * based on s5p-g2d
+ *
+ * Copyright (c) 2011 Samsung Electronics Co., Ltd.
+ * Kamil Debski, <k.debski@samsung.com>
+ */
+
+#include <linux/io.h>
+
+#include "dma2d.h"
+#include "dma2d-regs.h"
+
+static inline u32 reg_read(void __iomem *base, u32 reg)
+{
+ return readl_relaxed(base + reg);
+}
+
+static inline void reg_write(void __iomem *base, u32 reg, u32 val)
+{
+ writel_relaxed(val, base + reg);
+}
+
+static inline void reg_update_bits(void __iomem *base, u32 reg, u32 mask,
+ u32 val)
+{
+ reg_write(base, reg, (reg_read(base, reg) & ~mask) | val);
+}
+
+void dma2d_start(struct dma2d_dev *d)
+{
+ reg_update_bits(d->regs, DMA2D_CR_REG, CR_START, CR_START);
+}
+
+u32 dma2d_get_int(struct dma2d_dev *d)
+{
+ return reg_read(d->regs, DMA2D_ISR_REG);
+}
+
+void dma2d_clear_int(struct dma2d_dev *d)
+{
+ u32 isr_val = reg_read(d->regs, DMA2D_ISR_REG);
+
+ reg_write(d->regs, DMA2D_IFCR_REG, isr_val & 0x003f);
+}
+
+void dma2d_config_common(struct dma2d_dev *d, enum dma2d_op_mode op_mode,
+ u16 width, u16 height)
+{
+ reg_update_bits(d->regs, DMA2D_CR_REG, CR_MODE_MASK,
+ op_mode << CR_MODE_SHIFT);
+
+ reg_write(d->regs, DMA2D_NLR_REG, (width << 16) | height);
+}
+
+void dma2d_config_out(struct dma2d_dev *d, struct dma2d_frame *frm,
+ dma_addr_t o_addr)
+{
+ reg_update_bits(d->regs, DMA2D_CR_REG, CR_CEIE, CR_CEIE);
+ reg_update_bits(d->regs, DMA2D_CR_REG, CR_CTCIE, CR_CTCIE);
+ reg_update_bits(d->regs, DMA2D_CR_REG, CR_CAEIE, CR_CAEIE);
+ reg_update_bits(d->regs, DMA2D_CR_REG, CR_TCIE, CR_TCIE);
+ reg_update_bits(d->regs, DMA2D_CR_REG, CR_TEIE, CR_TEIE);
+
+ if (frm->fmt->cmode >= CM_MODE_ARGB8888 &&
+ frm->fmt->cmode <= CM_MODE_ARGB4444)
+ reg_update_bits(d->regs, DMA2D_OPFCCR_REG, OPFCCR_CM_MASK,
+ frm->fmt->cmode);
+
+ reg_write(d->regs, DMA2D_OMAR_REG, o_addr);
+
+ reg_write(d->regs, DMA2D_OCOLR_REG,
+ (frm->a_rgb[3] << 24) |
+ (frm->a_rgb[2] << 16) |
+ (frm->a_rgb[1] << 8) |
+ frm->a_rgb[0]);
+
+ reg_update_bits(d->regs, DMA2D_OOR_REG, OOR_LO_MASK,
+ frm->line_offset & 0x3fff);
+}
+
+void dma2d_config_fg(struct dma2d_dev *d, struct dma2d_frame *frm,
+ dma_addr_t f_addr)
+{
+ reg_write(d->regs, DMA2D_FGMAR_REG, f_addr);
+ reg_update_bits(d->regs, DMA2D_FGOR_REG, FGOR_LO_MASK,
+ frm->line_offset);
+
+ if (frm->fmt->cmode >= CM_MODE_ARGB8888 &&
+ frm->fmt->cmode <= CM_MODE_A4)
+ reg_update_bits(d->regs, DMA2D_FGPFCCR_REG, FGPFCCR_CM_MASK,
+ frm->fmt->cmode);
+
+ reg_update_bits(d->regs, DMA2D_FGPFCCR_REG, FGPFCCR_AM_MASK,
+ (frm->a_mode << 16) & 0x03);
+
+ reg_update_bits(d->regs, DMA2D_FGPFCCR_REG, FGPFCCR_ALPHA_MASK,
+ frm->a_rgb[3] << 24);
+
+ reg_write(d->regs, DMA2D_FGCOLR_REG,
+ (frm->a_rgb[2] << 16) |
+ (frm->a_rgb[1] << 8) |
+ frm->a_rgb[0]);
+}
+
+void dma2d_config_bg(struct dma2d_dev *d, struct dma2d_frame *frm,
+ dma_addr_t b_addr)
+{
+ reg_write(d->regs, DMA2D_BGMAR_REG, b_addr);
+ reg_update_bits(d->regs, DMA2D_BGOR_REG, BGOR_LO_MASK,
+ frm->line_offset);
+
+ if (frm->fmt->cmode >= CM_MODE_ARGB8888 &&
+ frm->fmt->cmode <= CM_MODE_A4)
+ reg_update_bits(d->regs, DMA2D_BGPFCCR_REG, BGPFCCR_CM_MASK,
+ frm->fmt->cmode);
+
+ reg_update_bits(d->regs, DMA2D_BGPFCCR_REG, BGPFCCR_AM_MASK,
+ (frm->a_mode << 16) & 0x03);
+
+ reg_update_bits(d->regs, DMA2D_BGPFCCR_REG, BGPFCCR_ALPHA_MASK,
+ frm->a_rgb[3] << 24);
+
+ reg_write(d->regs, DMA2D_BGCOLR_REG,
+ (frm->a_rgb[2] << 16) |
+ (frm->a_rgb[1] << 8) |
+ frm->a_rgb[0]);
+}
diff --git a/drivers/media/platform/stm32/dma2d/dma2d-regs.h b/drivers/media/platform/stm32/dma2d/dma2d-regs.h
new file mode 100644
index 0000000..6444592
--- /dev/null
+++ b/drivers/media/platform/stm32/dma2d/dma2d-regs.h
@@ -0,0 +1,113 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * ST stm32 Chrom-Art - 2D Graphics Accelerator Driver
+ *
+ * Copyright (c) 2021 Dillon Min
+ * Dillon Min, <dillon.minfei@gmail.com>
+ *
+ * based on s5p-g2d
+ *
+ * Copyright (c) 2011 Samsung Electronics Co., Ltd.
+ * Kamil Debski, <k.debski@samsung.com>
+ */
+
+#ifndef __DMA2D_REGS_H__
+#define __DMA2D_REGS_H__
+
+#define DMA2D_CR_REG 0x0000
+#define CR_MODE_MASK GENMASK(17, 16)
+#define CR_MODE_SHIFT 16
+#define CR_M2M 0x0000
+#define CR_M2M_PFC BIT(16)
+#define CR_M2M_BLEND BIT(17)
+#define CR_R2M (BIT(17) | BIT(16))
+#define CR_CEIE BIT(13)
+#define CR_CTCIE BIT(12)
+#define CR_CAEIE BIT(11)
+#define CR_TWIE BIT(10)
+#define CR_TCIE BIT(9)
+#define CR_TEIE BIT(8)
+#define CR_ABORT BIT(2)
+#define CR_SUSP BIT(1)
+#define CR_START BIT(0)
+
+#define DMA2D_ISR_REG 0x0004
+#define ISR_CEIF BIT(5)
+#define ISR_CTCIF BIT(4)
+#define ISR_CAEIF BIT(3)
+#define ISR_TWIF BIT(2)
+#define ISR_TCIF BIT(1)
+#define ISR_TEIF BIT(0)
+
+#define DMA2D_IFCR_REG 0x0008
+#define IFCR_CCEIF BIT(5)
+#define IFCR_CCTCIF BIT(4)
+#define IFCR_CAECIF BIT(3)
+#define IFCR_CTWIF BIT(2)
+#define IFCR_CTCIF BIT(1)
+#define IFCR_CTEIF BIT(0)
+
+#define DMA2D_FGMAR_REG 0x000c
+#define DMA2D_FGOR_REG 0x0010
+#define FGOR_LO_MASK GENMASK(13, 0)
+
+#define DMA2D_BGMAR_REG 0x0014
+#define DMA2D_BGOR_REG 0x0018
+#define BGOR_LO_MASK GENMASK(13, 0)
+
+#define DMA2D_FGPFCCR_REG 0x001c
+#define FGPFCCR_ALPHA_MASK GENMASK(31, 24)
+#define FGPFCCR_AM_MASK GENMASK(17, 16)
+#define FGPFCCR_CS_MASK GENMASK(15, 8)
+#define FGPFCCR_START BIT(5)
+#define FGPFCCR_CCM_RGB888 BIT(4)
+#define FGPFCCR_CM_MASK GENMASK(3, 0)
+
+#define DMA2D_FGCOLR_REG 0x0020
+#define FGCOLR_REG_MASK GENMASK(23, 16)
+#define FGCOLR_GREEN_MASK GENMASK(15, 8)
+#define FGCOLR_BLUE_MASK GENMASK(7, 0)
+
+#define DMA2D_BGPFCCR_REG 0x0024
+#define BGPFCCR_ALPHA_MASK GENMASK(31, 24)
+#define BGPFCCR_AM_MASK GENMASK(17, 16)
+#define BGPFCCR_CS_MASK GENMASK(15, 8)
+#define BGPFCCR_START BIT(5)
+#define BGPFCCR_CCM_RGB888 BIT(4)
+#define BGPFCCR_CM_MASK GENMASK(3, 0)
+
+#define DMA2D_BGCOLR_REG 0x0028
+#define BGCOLR_REG_MASK GENMASK(23, 16)
+#define BGCOLR_GREEN_MASK GENMASK(15, 8)
+#define BGCOLR_BLUE_MASK GENMASK(7, 0)
+
+#define DMA2D_OPFCCR_REG 0x0034
+#define OPFCCR_CM_MASK GENMASK(2, 0)
+
+#define DMA2D_OCOLR_REG 0x0038
+#define OCOLR_ALPHA_MASK GENMASK(31, 24)
+#define OCOLR_RED_MASK GENMASK(23, 16)
+#define OCOLR_GREEN_MASK GENMASK(15, 8)
+#define OCOLR_BLUE_MASK GENMASK(7, 0)
+
+#define DMA2D_OMAR_REG 0x003c
+
+#define DMA2D_OOR_REG 0x0040
+#define OOR_LO_MASK GENMASK(13, 0)
+
+#define DMA2D_NLR_REG 0x0044
+#define NLR_PL_MASK GENMASK(29, 16)
+#define NLR_NL_MASK GENMASK(15, 0)
+
+/* Hardware limits */
+#define MAX_WIDTH 2592
+#define MAX_HEIGHT 2592
+
+#define DEFAULT_WIDTH 240
+#define DEFAULT_HEIGHT 320
+#define DEFAULT_SIZE 307200
+
+#define CM_MODE_ARGB8888 0x00
+#define CM_MODE_ARGB4444 0x04
+#define CM_MODE_A4 0x0a
+#endif /* __DMA2D_REGS_H__ */
diff --git a/drivers/media/platform/stm32/dma2d/dma2d.c b/drivers/media/platform/stm32/dma2d/dma2d.c
new file mode 100644
index 0000000..17af90d
--- /dev/null
+++ b/drivers/media/platform/stm32/dma2d/dma2d.c
@@ -0,0 +1,739 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * STM32 DMA2D - 2D Graphics Accelerator Driver
+ *
+ * Copyright (c) 2021 Dillon Min
+ * Dillon Min, <dillon.minfei@gmail.com>
+ *
+ * based on s5p-g2d
+ *
+ * Copyright (c) 2011 Samsung Electronics Co., Ltd.
+ * Kamil Debski, <k.debski@samsung.com>
+ */
+
+#include <linux/module.h>
+#include <linux/fs.h>
+#include <linux/timer.h>
+#include <linux/sched.h>
+#include <linux/slab.h>
+#include <linux/clk.h>
+#include <linux/interrupt.h>
+#include <linux/of.h>
+
+#include <linux/platform_device.h>
+#include <media/v4l2-mem2mem.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-event.h>
+#include <media/videobuf2-v4l2.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "dma2d.h"
+#include "dma2d-regs.h"
+
+/*
+ * This V4L2 subdev m2m driver enables Chrom-Art Accelerator unit
+ * of STMicroelectronics STM32 SoC series.
+ *
+ * Currently support r2m, m2m, m2m_pfc.
+ *
+ * - r2m, Filling a part or the whole of a destination image with a specific
+ * color.
+ * - m2m, Copying a part or the whole of a source image into a part or the
+ * whole of a destination.
+ * - m2m_pfc, Copying a part or the whole of a source image into a part or the
+ * whole of a destination image with a pixel format conversion.
+ */
+
+#define fh2ctx(__fh) container_of(__fh, struct dma2d_ctx, fh)
+
+static const struct dma2d_fmt formats[] = {
+ {
+ .fourcc = V4L2_PIX_FMT_ARGB32,
+ .cmode = DMA2D_CMODE_ARGB8888,
+ .depth = 32,
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_RGB24,
+ .cmode = DMA2D_CMODE_RGB888,
+ .depth = 24,
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_RGB565,
+ .cmode = DMA2D_CMODE_RGB565,
+ .depth = 16,
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_ARGB555,
+ .cmode = DMA2D_CMODE_ARGB1555,
+ .depth = 16,
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_ARGB444,
+ .cmode = DMA2D_CMODE_ARGB4444,
+ .depth = 16,
+ },
+};
+
+#define NUM_FORMATS ARRAY_SIZE(formats)
+
+static const struct dma2d_frame def_frame = {
+ .width = DEFAULT_WIDTH,
+ .height = DEFAULT_HEIGHT,
+ .line_offset = 0,
+ .a_rgb = {0x00, 0x00, 0x00, 0xff},
+ .a_mode = DMA2D_ALPHA_MODE_NO_MODIF,
+ .fmt = (struct dma2d_fmt *)&formats[0],
+ .size = DEFAULT_SIZE,
+};
+
+static struct dma2d_fmt *find_fmt(int pixelformat)
+{
+ unsigned int i;
+
+ for (i = 0; i < NUM_FORMATS; i++) {
+ if (formats[i].fourcc == pixelformat)
+ return (struct dma2d_fmt *)&formats[i];
+ }
+
+ return NULL;
+}
+
+static struct dma2d_frame *get_frame(struct dma2d_ctx *ctx,
+ enum v4l2_buf_type type)
+{
+ return V4L2_TYPE_IS_OUTPUT(type) ? &ctx->cap : &ctx->out;
+}
+
+static int dma2d_queue_setup(struct vb2_queue *vq,
+ unsigned int *nbuffers, unsigned int *nplanes,
+ unsigned int sizes[], struct device *alloc_devs[])
+{
+ struct dma2d_ctx *ctx = vb2_get_drv_priv(vq);
+ struct dma2d_frame *f = get_frame(ctx, vq->type);
+
+ if (*nplanes)
+ return sizes[0] < f->size ? -EINVAL : 0;
+
+ sizes[0] = f->size;
+ *nplanes = 1;
+
+ return 0;
+}
+
+static int dma2d_buf_out_validate(struct vb2_buffer *vb)
+{
+ struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+
+ if (vbuf->field == V4L2_FIELD_ANY)
+ vbuf->field = V4L2_FIELD_NONE;
+ if (vbuf->field != V4L2_FIELD_NONE)
+ return -EINVAL;
+
+ return 0;
+}
+
+static int dma2d_buf_prepare(struct vb2_buffer *vb)
+{
+ struct dma2d_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+ struct dma2d_frame *f = get_frame(ctx, vb->vb2_queue->type);
+
+ if (vb2_plane_size(vb, 0) < f->size)
+ return -EINVAL;
+
+ vb2_set_plane_payload(vb, 0, f->size);
+
+ return 0;
+}
+
+static void dma2d_buf_queue(struct vb2_buffer *vb)
+{
+ struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+ struct dma2d_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+
+ v4l2_m2m_buf_queue(ctx->fh.m2m_ctx, vbuf);
+}
+
+static int dma2d_start_streaming(struct vb2_queue *q, unsigned int count)
+{
+ struct dma2d_ctx *ctx = vb2_get_drv_priv(q);
+ struct dma2d_frame *f = get_frame(ctx, q->type);
+
+ f->sequence = 0;
+ return 0;
+}
+
+static void dma2d_stop_streaming(struct vb2_queue *q)
+{
+ struct dma2d_ctx *ctx = vb2_get_drv_priv(q);
+ struct vb2_v4l2_buffer *vbuf;
+
+ for (;;) {
+ if (V4L2_TYPE_IS_OUTPUT(q->type))
+ vbuf = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
+ else
+ vbuf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
+ if (!vbuf)
+ return;
+ v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_ERROR);
+ }
+}
+
+static const struct vb2_ops dma2d_qops = {
+ .queue_setup = dma2d_queue_setup,
+ .buf_out_validate = dma2d_buf_out_validate,
+ .buf_prepare = dma2d_buf_prepare,
+ .buf_queue = dma2d_buf_queue,
+ .start_streaming = dma2d_start_streaming,
+ .stop_streaming = dma2d_stop_streaming,
+ .wait_prepare = vb2_ops_wait_prepare,
+ .wait_finish = vb2_ops_wait_finish,
+};
+
+static int queue_init(void *priv, struct vb2_queue *src_vq,
+ struct vb2_queue *dst_vq)
+{
+ struct dma2d_ctx *ctx = priv;
+ int ret;
+
+ src_vq->type = V4L2_BUF_TYPE_VIDEO_OUTPUT;
+ src_vq->io_modes = VB2_MMAP | VB2_DMABUF;
+ src_vq->drv_priv = ctx;
+ src_vq->ops = &dma2d_qops;
+ src_vq->mem_ops = &vb2_dma_contig_memops;
+ src_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
+ src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+ src_vq->lock = &ctx->dev->mutex;
+ src_vq->dev = ctx->dev->v4l2_dev.dev;
+
+ ret = vb2_queue_init(src_vq);
+ if (ret)
+ return ret;
+
+ dst_vq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
+ dst_vq->io_modes = VB2_MMAP | VB2_DMABUF;
+ dst_vq->drv_priv = ctx;
+ dst_vq->ops = &dma2d_qops;
+ dst_vq->mem_ops = &vb2_dma_contig_memops;
+ dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
+ dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+ dst_vq->lock = &ctx->dev->mutex;
+ dst_vq->dev = ctx->dev->v4l2_dev.dev;
+
+ return vb2_queue_init(dst_vq);
+}
+
+static int dma2d_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+ struct dma2d_frame *frm;
+ struct dma2d_ctx *ctx = container_of(ctrl->handler, struct dma2d_ctx,
+ ctrl_handler);
+ unsigned long flags;
+
+ spin_lock_irqsave(&ctx->dev->ctrl_lock, flags);
+ switch (ctrl->id) {
+ case V4L2_CID_COLORFX:
+ if (ctrl->val == V4L2_COLORFX_SET_RGB)
+ ctx->op_mode = DMA2D_MODE_R2M;
+ else if (ctrl->val == V4L2_COLORFX_NONE)
+ ctx->op_mode = DMA2D_MODE_M2M;
+ break;
+ case V4L2_CID_COLORFX_RGB:
+ frm = get_frame(ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE);
+ frm->a_rgb[2] = (ctrl->val >> 16) & 0xff;
+ frm->a_rgb[1] = (ctrl->val >> 8) & 0xff;
+ frm->a_rgb[0] = (ctrl->val >> 0) & 0xff;
+ break;
+ default:
+ spin_unlock_irqrestore(&ctx->dev->ctrl_lock, flags);
+ return -EINVAL;
+ }
+ spin_unlock_irqrestore(&ctx->dev->ctrl_lock, flags);
+
+ return 0;
+}
+
+static const struct v4l2_ctrl_ops dma2d_ctrl_ops = {
+ .s_ctrl = dma2d_s_ctrl,
+};
+
+static int dma2d_setup_ctrls(struct dma2d_ctx *ctx)
+{
+ struct v4l2_ctrl_handler *handler = &ctx->ctrl_handler;
+
+ v4l2_ctrl_handler_init(handler, 2);
+
+ v4l2_ctrl_new_std_menu(handler, &dma2d_ctrl_ops, V4L2_CID_COLORFX,
+ V4L2_COLORFX_SET_RGB, ~0x10001,
+ V4L2_COLORFX_NONE);
+
+ v4l2_ctrl_new_std(handler, &dma2d_ctrl_ops, V4L2_CID_COLORFX_RGB, 0,
+ 0xffffff, 1, 0);
+
+ return 0;
+}
+
+static int dma2d_open(struct file *file)
+{
+ struct dma2d_dev *dev = video_drvdata(file);
+ struct dma2d_ctx *ctx = NULL;
+ int ret = 0;
+
+ ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
+ if (!ctx)
+ return -ENOMEM;
+ ctx->dev = dev;
+ /* Set default formats */
+ ctx->cap = def_frame;
+ ctx->bg = def_frame;
+ ctx->out = def_frame;
+ ctx->op_mode = DMA2D_MODE_M2M_FPC;
+ ctx->colorspace = V4L2_COLORSPACE_REC709;
+ if (mutex_lock_interruptible(&dev->mutex)) {
+ kfree(ctx);
+ return -ERESTARTSYS;
+ }
+
+ ctx->fh.m2m_ctx = v4l2_m2m_ctx_init(dev->m2m_dev, ctx, &queue_init);
+ if (IS_ERR(ctx->fh.m2m_ctx)) {
+ ret = PTR_ERR(ctx->fh.m2m_ctx);
+ mutex_unlock(&dev->mutex);
+ kfree(ctx);
+ return ret;
+ }
+
+ v4l2_fh_init(&ctx->fh, video_devdata(file));
+ file->private_data = &ctx->fh;
+ v4l2_fh_add(&ctx->fh);
+
+ dma2d_setup_ctrls(ctx);
+
+ /* Write the default values to the ctx struct */
+ v4l2_ctrl_handler_setup(&ctx->ctrl_handler);
+
+ ctx->fh.ctrl_handler = &ctx->ctrl_handler;
+ mutex_unlock(&dev->mutex);
+
+ return 0;
+}
+
+static int dma2d_release(struct file *file)
+{
+ struct dma2d_dev *dev = video_drvdata(file);
+ struct dma2d_ctx *ctx = fh2ctx(file->private_data);
+
+ mutex_lock(&dev->mutex);
+ v4l2_m2m_ctx_release(ctx->fh.m2m_ctx);
+ mutex_unlock(&dev->mutex);
+ v4l2_ctrl_handler_free(&ctx->ctrl_handler);
+ v4l2_fh_del(&ctx->fh);
+ v4l2_fh_exit(&ctx->fh);
+ kfree(ctx);
+
+ return 0;
+}
+
+static int vidioc_querycap(struct file *file, void *priv,
+ struct v4l2_capability *cap)
+{
+ strscpy(cap->driver, DMA2D_NAME, sizeof(cap->driver));
+ strscpy(cap->card, DMA2D_NAME, sizeof(cap->card));
+ strscpy(cap->bus_info, BUS_INFO, sizeof(cap->bus_info));
+
+ return 0;
+}
+
+static int vidioc_enum_fmt(struct file *file, void *prv, struct v4l2_fmtdesc *f)
+{
+ if (f->index >= NUM_FORMATS)
+ return -EINVAL;
+
+ f->pixelformat = formats[f->index].fourcc;
+ return 0;
+}
+
+static int vidioc_g_fmt(struct file *file, void *prv, struct v4l2_format *f)
+{
+ struct dma2d_ctx *ctx = prv;
+ struct vb2_queue *vq;
+ struct dma2d_frame *frm;
+
+ vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, f->type);
+ if (!vq)
+ return -EINVAL;
+
+ frm = get_frame(ctx, f->type);
+ f->fmt.pix.width = frm->width;
+ f->fmt.pix.height = frm->height;
+ f->fmt.pix.field = V4L2_FIELD_NONE;
+ f->fmt.pix.pixelformat = frm->fmt->fourcc;
+ f->fmt.pix.bytesperline = (frm->width * frm->fmt->depth) >> 3;
+ f->fmt.pix.sizeimage = frm->size;
+ f->fmt.pix.colorspace = ctx->colorspace;
+ f->fmt.pix.xfer_func = ctx->xfer_func;
+ f->fmt.pix.ycbcr_enc = ctx->ycbcr_enc;
+ f->fmt.pix.quantization = ctx->quant;
+
+ return 0;
+}
+
+static int vidioc_try_fmt(struct file *file, void *prv, struct v4l2_format *f)
+{
+ struct dma2d_ctx *ctx = prv;
+ struct dma2d_fmt *fmt;
+ enum v4l2_field *field;
+ u32 fourcc = f->fmt.pix.pixelformat;
+
+ fmt = find_fmt(fourcc);
+ if (!fmt) {
+ f->fmt.pix.pixelformat = formats[0].fourcc;
+ fmt = find_fmt(f->fmt.pix.pixelformat);
+ }
+
+ field = &f->fmt.pix.field;
+ if (*field == V4L2_FIELD_ANY)
+ *field = V4L2_FIELD_NONE;
+ else if (*field != V4L2_FIELD_NONE)
+ return -EINVAL;
+
+ if (f->fmt.pix.width > MAX_WIDTH)
+ f->fmt.pix.width = MAX_WIDTH;
+ if (f->fmt.pix.height > MAX_HEIGHT)
+ f->fmt.pix.height = MAX_HEIGHT;
+
+ if (f->fmt.pix.width < 1)
+ f->fmt.pix.width = 1;
+ if (f->fmt.pix.height < 1)
+ f->fmt.pix.height = 1;
+
+ if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT && !f->fmt.pix.colorspace) {
+ f->fmt.pix.colorspace = V4L2_COLORSPACE_REC709;
+ } else if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE) {
+ f->fmt.pix.colorspace = ctx->colorspace;
+ f->fmt.pix.xfer_func = ctx->xfer_func;
+ f->fmt.pix.ycbcr_enc = ctx->ycbcr_enc;
+ f->fmt.pix.quantization = ctx->quant;
+ }
+ f->fmt.pix.bytesperline = (f->fmt.pix.width * fmt->depth) >> 3;
+ f->fmt.pix.sizeimage = f->fmt.pix.height * f->fmt.pix.bytesperline;
+
+ return 0;
+}
+
+static int vidioc_s_fmt(struct file *file, void *prv, struct v4l2_format *f)
+{
+ struct dma2d_ctx *ctx = prv;
+ struct vb2_queue *vq;
+ struct dma2d_frame *frm;
+ struct dma2d_fmt *fmt;
+ int ret = 0;
+
+ /* Adjust all values accordingly to the hardware capabilities
+ * and chosen format.
+ */
+ ret = vidioc_try_fmt(file, prv, f);
+ if (ret)
+ return ret;
+
+ vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, f->type);
+ if (vb2_is_busy(vq))
+ return -EBUSY;
+
+ fmt = find_fmt(f->fmt.pix.pixelformat);
+ if (!fmt)
+ return -EINVAL;
+
+ if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT) {
+ ctx->colorspace = f->fmt.pix.colorspace;
+ ctx->xfer_func = f->fmt.pix.xfer_func;
+ ctx->ycbcr_enc = f->fmt.pix.ycbcr_enc;
+ ctx->quant = f->fmt.pix.quantization;
+ }
+
+ frm = get_frame(ctx, f->type);
+ frm->width = f->fmt.pix.width;
+ frm->height = f->fmt.pix.height;
+ frm->size = f->fmt.pix.sizeimage;
+ /* Reset crop settings */
+ frm->o_width = 0;
+ frm->o_height = 0;
+ frm->c_width = frm->width;
+ frm->c_height = frm->height;
+ frm->right = frm->width;
+ frm->bottom = frm->height;
+ frm->fmt = fmt;
+ frm->line_offset = 0;
+
+ return 0;
+}
+
+static void device_run(void *prv)
+{
+ struct dma2d_ctx *ctx = prv;
+ struct dma2d_dev *dev = ctx->dev;
+ struct dma2d_frame *frm_out, *frm_cap;
+ struct vb2_v4l2_buffer *src, *dst;
+ unsigned long flags;
+
+ spin_lock_irqsave(&dev->ctrl_lock, flags);
+ dev->curr = ctx;
+
+ src = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
+ dst = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
+ if (!dst || !src)
+ goto end;
+
+ frm_cap = get_frame(ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE);
+ frm_out = get_frame(ctx, V4L2_BUF_TYPE_VIDEO_OUTPUT);
+ if (!frm_cap || !frm_out)
+ goto end;
+
+ src->sequence = frm_out->sequence++;
+ dst->sequence = frm_cap->sequence++;
+ v4l2_m2m_buf_copy_metadata(src, dst, true);
+
+ clk_enable(dev->gate);
+
+ dma2d_config_fg(dev, frm_out,
+ vb2_dma_contig_plane_dma_addr(&src->vb2_buf, 0));
+
+ /* TODO: add M2M_BLEND handler here */
+
+ if (ctx->op_mode != DMA2D_MODE_R2M) {
+ if (frm_out->fmt->fourcc == frm_cap->fmt->fourcc)
+ ctx->op_mode = DMA2D_MODE_M2M;
+ else
+ ctx->op_mode = DMA2D_MODE_M2M_FPC;
+ }
+
+ dma2d_config_out(dev, frm_cap,
+ vb2_dma_contig_plane_dma_addr(&dst->vb2_buf, 0));
+ dma2d_config_common(dev, ctx->op_mode, frm_cap->width, frm_cap->height);
+
+ dma2d_start(dev);
+end:
+ spin_unlock_irqrestore(&dev->ctrl_lock, flags);
+}
+
+static irqreturn_t dma2d_isr(int irq, void *prv)
+{
+ struct dma2d_dev *dev = prv;
+ struct dma2d_ctx *ctx = dev->curr;
+ struct vb2_v4l2_buffer *src, *dst;
+ u32 s = dma2d_get_int(dev);
+
+ dma2d_clear_int(dev);
+ if (s & ISR_TCIF || s == 0) {
+ clk_disable(dev->gate);
+
+ WARN_ON(!ctx);
+
+ src = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
+ dst = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
+
+ WARN_ON(!dst);
+ WARN_ON(!src);
+
+ v4l2_m2m_buf_done(src, VB2_BUF_STATE_DONE);
+ v4l2_m2m_buf_done(dst, VB2_BUF_STATE_DONE);
+ v4l2_m2m_job_finish(dev->m2m_dev, ctx->fh.m2m_ctx);
+
+ dev->curr = NULL;
+ }
+
+ return IRQ_HANDLED;
+}
+
+static const struct v4l2_file_operations dma2d_fops = {
+ .owner = THIS_MODULE,
+ .open = dma2d_open,
+ .release = dma2d_release,
+ .poll = v4l2_m2m_fop_poll,
+ .unlocked_ioctl = video_ioctl2,
+ .mmap = v4l2_m2m_fop_mmap,
+#ifndef CONFIG_MMU
+ .get_unmapped_area = v4l2_m2m_get_unmapped_area,
+#endif
+};
+
+static const struct v4l2_ioctl_ops dma2d_ioctl_ops = {
+ .vidioc_querycap = vidioc_querycap,
+
+ .vidioc_enum_fmt_vid_cap = vidioc_enum_fmt,
+ .vidioc_g_fmt_vid_cap = vidioc_g_fmt,
+ .vidioc_try_fmt_vid_cap = vidioc_try_fmt,
+ .vidioc_s_fmt_vid_cap = vidioc_s_fmt,
+
+ .vidioc_enum_fmt_vid_out = vidioc_enum_fmt,
+ .vidioc_g_fmt_vid_out = vidioc_g_fmt,
+ .vidioc_try_fmt_vid_out = vidioc_try_fmt,
+ .vidioc_s_fmt_vid_out = vidioc_s_fmt,
+
+ .vidioc_reqbufs = v4l2_m2m_ioctl_reqbufs,
+ .vidioc_querybuf = v4l2_m2m_ioctl_querybuf,
+ .vidioc_qbuf = v4l2_m2m_ioctl_qbuf,
+ .vidioc_dqbuf = v4l2_m2m_ioctl_dqbuf,
+ .vidioc_prepare_buf = v4l2_m2m_ioctl_prepare_buf,
+ .vidioc_create_bufs = v4l2_m2m_ioctl_create_bufs,
+ .vidioc_expbuf = v4l2_m2m_ioctl_expbuf,
+
+ .vidioc_streamon = v4l2_m2m_ioctl_streamon,
+ .vidioc_streamoff = v4l2_m2m_ioctl_streamoff,
+
+ .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+};
+
+static const struct video_device dma2d_videodev = {
+ .name = DMA2D_NAME,
+ .fops = &dma2d_fops,
+ .ioctl_ops = &dma2d_ioctl_ops,
+ .minor = -1,
+ .release = video_device_release,
+ .vfl_dir = VFL_DIR_M2M,
+};
+
+static const struct v4l2_m2m_ops dma2d_m2m_ops = {
+ .device_run = device_run,
+};
+
+static const struct of_device_id stm32_dma2d_match[];
+
+static int dma2d_probe(struct platform_device *pdev)
+{
+ struct dma2d_dev *dev;
+ struct video_device *vfd;
+ struct resource *res;
+ int ret = 0;
+
+ dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
+ if (!dev)
+ return -ENOMEM;
+
+ spin_lock_init(&dev->ctrl_lock);
+ mutex_init(&dev->mutex);
+ atomic_set(&dev->num_inst, 0);
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+
+ dev->regs = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(dev->regs))
+ return PTR_ERR(dev->regs);
+
+ dev->gate = clk_get(&pdev->dev, "dma2d");
+ if (IS_ERR(dev->gate)) {
+ dev_err(&pdev->dev, "failed to get dma2d clock gate\n");
+ ret = -ENXIO;
+ return ret;
+ }
+
+ ret = clk_prepare(dev->gate);
+ if (ret) {
+ dev_err(&pdev->dev, "failed to prepare dma2d clock gate\n");
+ goto put_clk_gate;
+ }
+
+ res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
+ if (!res) {
+ dev_err(&pdev->dev, "failed to find IRQ\n");
+ ret = -ENXIO;
+ goto unprep_clk_gate;
+ }
+
+ dev->irq = res->start;
+
+ ret = devm_request_irq(&pdev->dev, dev->irq, dma2d_isr,
+ 0, pdev->name, dev);
+ if (ret) {
+ dev_err(&pdev->dev, "failed to install IRQ\n");
+ goto unprep_clk_gate;
+ }
+
+ ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev);
+ if (ret)
+ goto unprep_clk_gate;
+
+ vfd = video_device_alloc();
+ if (!vfd) {
+ v4l2_err(&dev->v4l2_dev, "Failed to allocate video device\n");
+ ret = -ENOMEM;
+ goto unreg_v4l2_dev;
+ }
+
+ *vfd = dma2d_videodev;
+ vfd->lock = &dev->mutex;
+ vfd->v4l2_dev = &dev->v4l2_dev;
+ vfd->device_caps = V4L2_CAP_VIDEO_M2M | V4L2_CAP_STREAMING;
+
+ platform_set_drvdata(pdev, dev);
+ dev->m2m_dev = v4l2_m2m_init(&dma2d_m2m_ops);
+ if (IS_ERR(dev->m2m_dev)) {
+ v4l2_err(&dev->v4l2_dev, "Failed to init mem2mem device\n");
+ ret = PTR_ERR(dev->m2m_dev);
+ goto rel_vdev;
+ }
+
+ ret = video_register_device(vfd, VFL_TYPE_VIDEO, 0);
+ if (ret) {
+ v4l2_err(&dev->v4l2_dev, "Failed to register video device\n");
+ goto free_m2m;
+ }
+
+ video_set_drvdata(vfd, dev);
+ dev->vfd = vfd;
+ v4l2_info(&dev->v4l2_dev, "device registered as /dev/video%d\n",
+ vfd->num);
+ return 0;
+
+free_m2m:
+ v4l2_m2m_release(dev->m2m_dev);
+rel_vdev:
+ video_device_release(vfd);
+unreg_v4l2_dev:
+ v4l2_device_unregister(&dev->v4l2_dev);
+unprep_clk_gate:
+ clk_unprepare(dev->gate);
+put_clk_gate:
+ clk_put(dev->gate);
+
+ return ret;
+}
+
+static int dma2d_remove(struct platform_device *pdev)
+{
+ struct dma2d_dev *dev = platform_get_drvdata(pdev);
+
+ v4l2_info(&dev->v4l2_dev, "Removing " DMA2D_NAME);
+ v4l2_m2m_release(dev->m2m_dev);
+ video_unregister_device(dev->vfd);
+ v4l2_device_unregister(&dev->v4l2_dev);
+ vb2_dma_contig_clear_max_seg_size(&pdev->dev);
+ clk_unprepare(dev->gate);
+ clk_put(dev->gate);
+
+ return 0;
+}
+
+static const struct of_device_id stm32_dma2d_match[] = {
+ {
+ .compatible = "st,stm32-dma2d",
+ .data = NULL,
+ },
+ {},
+};
+MODULE_DEVICE_TABLE(of, stm32_dma2d_match);
+
+static struct platform_driver dma2d_pdrv = {
+ .probe = dma2d_probe,
+ .remove = dma2d_remove,
+ .driver = {
+ .name = DMA2D_NAME,
+ .of_match_table = stm32_dma2d_match,
+ },
+};
+
+module_platform_driver(dma2d_pdrv);
+
+MODULE_AUTHOR("Dillon Min <dillon.minfei@gmail.com>");
+MODULE_DESCRIPTION("STM32 Chrom-Art Accelerator DMA2D driver");
+MODULE_LICENSE("GPL");
diff --git a/drivers/media/platform/stm32/dma2d/dma2d.h b/drivers/media/platform/stm32/dma2d/dma2d.h
new file mode 100644
index 0000000..3f03a7c
--- /dev/null
+++ b/drivers/media/platform/stm32/dma2d/dma2d.h
@@ -0,0 +1,135 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * ST stm32 DMA2D - 2D Graphics Accelerator Driver
+ *
+ * Copyright (c) 2021 Dillon Min
+ * Dillon Min, <dillon.minfei@gmail.com>
+ *
+ * based on s5p-g2d
+ *
+ * Copyright (c) 2011 Samsung Electronics Co., Ltd.
+ * Kamil Debski, <k.debski@samsung.com>
+ */
+
+#ifndef __DMA2D_H__
+#define __DMA2D_H__
+
+#include <linux/platform_device.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ctrls.h>
+
+#define DMA2D_NAME "stm-dma2d"
+#define BUS_INFO "platform:stm-dma2d"
+enum dma2d_op_mode {
+ DMA2D_MODE_M2M,
+ DMA2D_MODE_M2M_FPC,
+ DMA2D_MODE_M2M_BLEND,
+ DMA2D_MODE_R2M
+};
+
+enum dma2d_cmode {
+ /* output pfc cmode from ARGB888 to ARGB4444 */
+ DMA2D_CMODE_ARGB8888,
+ DMA2D_CMODE_RGB888,
+ DMA2D_CMODE_RGB565,
+ DMA2D_CMODE_ARGB1555,
+ DMA2D_CMODE_ARGB4444,
+ /* bg or fg pfc cmode from L8 to A4 */
+ DMA2D_CMODE_L8,
+ DMA2D_CMODE_AL44,
+ DMA2D_CMODE_AL88,
+ DMA2D_CMODE_L4,
+ DMA2D_CMODE_A8,
+ DMA2D_CMODE_A4
+};
+
+enum dma2d_alpha_mode {
+ DMA2D_ALPHA_MODE_NO_MODIF,
+ DMA2D_ALPHA_MODE_REPLACE,
+ DMA2D_ALPHA_MODE_COMBINE
+};
+
+struct dma2d_fmt {
+ u32 fourcc;
+ int depth;
+ enum dma2d_cmode cmode;
+};
+
+struct dma2d_frame {
+ /* Original dimensions */
+ u32 width;
+ u32 height;
+ /* Crop size */
+ u32 c_width;
+ u32 c_height;
+ /* Offset */
+ u32 o_width;
+ u32 o_height;
+ u32 bottom;
+ u32 right;
+ u16 line_offset;
+ /* Image format */
+ struct dma2d_fmt *fmt;
+ /* [0]: blue
+ * [1]: green
+ * [2]: red
+ * [3]: alpha
+ */
+ u8 a_rgb[4];
+ /*
+ * AM[1:0] of DMA2D_FGPFCCR
+ */
+ enum dma2d_alpha_mode a_mode;
+ u32 size;
+ unsigned int sequence;
+};
+
+struct dma2d_ctx {
+ struct v4l2_fh fh;
+ struct dma2d_dev *dev;
+ struct dma2d_frame cap;
+ struct dma2d_frame out;
+ struct dma2d_frame bg;
+ /* fb_buf always point to bg address */
+ struct v4l2_framebuffer fb_buf;
+ /*
+ * MODE[17:16] of DMA2D_CR
+ */
+ enum dma2d_op_mode op_mode;
+ struct v4l2_ctrl_handler ctrl_handler;
+ enum v4l2_colorspace colorspace;
+ enum v4l2_ycbcr_encoding ycbcr_enc;
+ enum v4l2_xfer_func xfer_func;
+ enum v4l2_quantization quant;
+};
+
+struct dma2d_dev {
+ struct v4l2_device v4l2_dev;
+ struct v4l2_m2m_dev *m2m_dev;
+ struct video_device *vfd;
+ /* for device open/close etc */
+ struct mutex mutex;
+ /* to avoid the conflict with device running and user setting
+ * at the same time
+ */
+ spinlock_t ctrl_lock;
+ atomic_t num_inst;
+ void __iomem *regs;
+ struct clk *gate;
+ struct dma2d_ctx *curr;
+ int irq;
+};
+
+void dma2d_start(struct dma2d_dev *d);
+u32 dma2d_get_int(struct dma2d_dev *d);
+void dma2d_clear_int(struct dma2d_dev *d);
+void dma2d_config_out(struct dma2d_dev *d, struct dma2d_frame *frm,
+ dma_addr_t o_addr);
+void dma2d_config_fg(struct dma2d_dev *d, struct dma2d_frame *frm,
+ dma_addr_t f_addr);
+void dma2d_config_bg(struct dma2d_dev *d, struct dma2d_frame *frm,
+ dma_addr_t b_addr);
+void dma2d_config_common(struct dma2d_dev *d, enum dma2d_op_mode op_mode,
+ u16 width, u16 height);
+
+#endif /* __DMA2D_H__ */
diff --git a/drivers/media/radio/radio-si476x.c b/drivers/media/radio/radio-si476x.c
index b39a68f..0bf99e1 100644
--- a/drivers/media/radio/radio-si476x.c
+++ b/drivers/media/radio/radio-si476x.c
@@ -313,12 +313,6 @@ struct si476x_radio {
};
static inline struct si476x_radio *
-v4l2_dev_to_radio(struct v4l2_device *d)
-{
- return container_of(d, struct si476x_radio, v4l2dev);
-}
-
-static inline struct si476x_radio *
v4l2_ctrl_handler_to_radio(struct v4l2_ctrl_handler *d)
{
return container_of(d, struct si476x_radio, ctrl_handler);
diff --git a/drivers/media/radio/si470x/radio-si470x-i2c.c b/drivers/media/radio/si470x/radio-si470x-i2c.c
index a972c07..59b3d77 100644
--- a/drivers/media/radio/si470x/radio-si470x-i2c.c
+++ b/drivers/media/radio/si470x/radio-si470x-i2c.c
@@ -334,7 +334,6 @@ static int si470x_i2c_probe(struct i2c_client *client)
{
struct si470x_device *radio;
int retval = 0;
- unsigned char version_warning = 0;
/* private data allocation and initialization */
radio = devm_kzalloc(&client->dev, sizeof(*radio), GFP_KERNEL);
@@ -368,7 +367,7 @@ static int si470x_i2c_probe(struct i2c_client *client)
if (radio->hdl.error) {
retval = radio->hdl.error;
dev_err(&client->dev, "couldn't register control\n");
- goto err_dev;
+ goto err_all;
}
/* video device initialization */
@@ -410,20 +409,10 @@ static int si470x_i2c_probe(struct i2c_client *client)
radio->registers[DEVICEID], radio->registers[SI_CHIPID]);
if ((radio->registers[SI_CHIPID] & SI_CHIPID_FIRMWARE) < RADIO_FW_VERSION) {
dev_warn(&client->dev,
- "This driver is known to work with firmware version %hu,\n",
- RADIO_FW_VERSION);
- dev_warn(&client->dev,
- "but the device has firmware version %hu.\n",
+ "This driver is known to work with firmware version %u, but the device has firmware version %u.\n"
+ "If you have some trouble using this driver, please report to V4L ML at linux-media@vger.kernel.org\n",
+ RADIO_FW_VERSION,
radio->registers[SI_CHIPID] & SI_CHIPID_FIRMWARE);
- version_warning = 1;
- }
-
- /* give out version warning */
- if (version_warning == 1) {
- dev_warn(&client->dev,
- "If you have some trouble using this driver,\n");
- dev_warn(&client->dev,
- "please report to V4L ML at linux-media@vger.kernel.org\n");
}
/* set initial frequency */
@@ -463,7 +452,6 @@ static int si470x_i2c_probe(struct i2c_client *client)
return 0;
err_all:
v4l2_ctrl_handler_free(&radio->hdl);
-err_dev:
v4l2_device_unregister(&radio->v4l2_dev);
err_initial:
return retval;
diff --git a/drivers/media/radio/si470x/radio-si470x-usb.c b/drivers/media/radio/si470x/radio-si470x-usb.c
index 3f8634a..6b27686 100644
--- a/drivers/media/radio/si470x/radio-si470x-usb.c
+++ b/drivers/media/radio/si470x/radio-si470x-usb.c
@@ -681,10 +681,8 @@ static int si470x_usb_driver_probe(struct usb_interface *intf,
radio->registers[DEVICEID], radio->registers[SI_CHIPID]);
if ((radio->registers[SI_CHIPID] & SI_CHIPID_FIRMWARE) < RADIO_FW_VERSION) {
dev_warn(&intf->dev,
- "This driver is known to work with firmware version %hu,\n",
- RADIO_FW_VERSION);
- dev_warn(&intf->dev,
- "but the device has firmware version %hu.\n",
+ "This driver is known to work with firmware version %u, but the device has firmware version %u.\n",
+ RADIO_FW_VERSION,
radio->registers[SI_CHIPID] & SI_CHIPID_FIRMWARE);
version_warning = 1;
}
@@ -698,10 +696,8 @@ static int si470x_usb_driver_probe(struct usb_interface *intf,
radio->software_version, radio->hardware_version);
if (radio->hardware_version < RADIO_HW_VERSION) {
dev_warn(&intf->dev,
- "This driver is known to work with hardware version %hu,\n",
- RADIO_HW_VERSION);
- dev_warn(&intf->dev,
- "but the device has hardware version %hu.\n",
+ "This driver is known to work with hardware version %u, but the device has hardware version %u.\n",
+ RADIO_HW_VERSION,
radio->hardware_version);
version_warning = 1;
}
@@ -709,9 +705,7 @@ static int si470x_usb_driver_probe(struct usb_interface *intf,
/* give out version warning */
if (version_warning == 1) {
dev_warn(&intf->dev,
- "If you have some trouble using this driver,\n");
- dev_warn(&intf->dev,
- "please report to V4L ML at linux-media@vger.kernel.org\n");
+ "If you have some trouble using this driver, please report to V4L ML at linux-media@vger.kernel.org\n");
}
/* set led to connect state */
diff --git a/drivers/media/rc/Kconfig b/drivers/media/rc/Kconfig
index 9506baf..c111af8 100644
--- a/drivers/media/rc/Kconfig
+++ b/drivers/media/rc/Kconfig
@@ -175,7 +175,7 @@
config IR_HIX5HD2
tristate "Hisilicon hix5hd2 IR remote control"
- depends on OF || COMPILE_TEST
+ depends on (OF && HAS_IOMEM) || COMPILE_TEST
help
Say Y here if you want to use hisilicon hix5hd2 remote control.
To compile this driver as a module, choose M here: the module will be
diff --git a/drivers/media/rc/igorplugusb.c b/drivers/media/rc/igorplugusb.c
index effaa57..3e9988e 100644
--- a/drivers/media/rc/igorplugusb.c
+++ b/drivers/media/rc/igorplugusb.c
@@ -64,9 +64,11 @@ static void igorplugusb_irdata(struct igorplugusb *ir, unsigned len)
if (start >= len) {
dev_err(ir->dev, "receive overflow invalid: %u", overflow);
} else {
- if (overflow > 0)
+ if (overflow > 0) {
dev_warn(ir->dev, "receive overflow, at least %u lost",
overflow);
+ ir_raw_event_reset(ir->rc);
+ }
do {
rawir.duration = ir->buf_in[i] * 85;
diff --git a/drivers/media/rc/iguanair.c b/drivers/media/rc/iguanair.c
index 84949ba..f8d080e 100644
--- a/drivers/media/rc/iguanair.c
+++ b/drivers/media/rc/iguanair.c
@@ -262,9 +262,6 @@ static int iguanair_receiver(struct iguanair *ir, bool enable)
ir->packet->header.direction = DIR_OUT;
ir->packet->header.cmd = enable ? CMD_RECEIVER_ON : CMD_RECEIVER_OFF;
- if (enable)
- ir_raw_event_reset(ir->rc);
-
return iguanair_send(ir, sizeof(ir->packet->header));
}
diff --git a/drivers/media/rc/ir-rx51.c b/drivers/media/rc/ir-rx51.c
index a0d9c02..a3b1451 100644
--- a/drivers/media/rc/ir-rx51.c
+++ b/drivers/media/rc/ir-rx51.c
@@ -19,6 +19,7 @@
struct ir_rx51 {
struct rc_dev *rcdev;
struct pwm_device *pwm;
+ struct pwm_state state;
struct hrtimer timer;
struct device *dev;
wait_queue_head_t wqueue;
@@ -32,22 +33,20 @@ struct ir_rx51 {
static inline void ir_rx51_on(struct ir_rx51 *ir_rx51)
{
- pwm_enable(ir_rx51->pwm);
+ ir_rx51->state.enabled = true;
+ pwm_apply_state(ir_rx51->pwm, &ir_rx51->state);
}
static inline void ir_rx51_off(struct ir_rx51 *ir_rx51)
{
- pwm_disable(ir_rx51->pwm);
+ ir_rx51->state.enabled = false;
+ pwm_apply_state(ir_rx51->pwm, &ir_rx51->state);
}
static int init_timing_params(struct ir_rx51 *ir_rx51)
{
- struct pwm_device *pwm = ir_rx51->pwm;
- int duty, period = DIV_ROUND_CLOSEST(NSEC_PER_SEC, ir_rx51->freq);
-
- duty = DIV_ROUND_CLOSEST(ir_rx51->duty_cycle * period, 100);
-
- pwm_config(pwm, duty, period);
+ ir_rx51->state.period = DIV_ROUND_CLOSEST(NSEC_PER_SEC, ir_rx51->freq);
+ pwm_set_relative_duty_cycle(&ir_rx51->state, ir_rx51->duty_cycle, 100);
return 0;
}
@@ -242,6 +241,7 @@ static int ir_rx51_probe(struct platform_device *dev)
/* Use default, in case userspace does not set the carrier */
ir_rx51.freq = DIV_ROUND_CLOSEST_ULL(pwm_get_period(pwm), NSEC_PER_SEC);
+ pwm_init_state(pwm, &ir_rx51.state);
pwm_put(pwm);
hrtimer_init(&ir_rx51.timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
diff --git a/drivers/media/rc/ir-spi.c b/drivers/media/rc/ir-spi.c
index c58f2d3..51aa55a 100644
--- a/drivers/media/rc/ir-spi.c
+++ b/drivers/media/rc/ir-spi.c
@@ -152,11 +152,6 @@ static int ir_spi_probe(struct spi_device *spi)
return devm_rc_register_device(&spi->dev, idata->rc);
}
-static int ir_spi_remove(struct spi_device *spi)
-{
- return 0;
-}
-
static const struct of_device_id ir_spi_of_match[] = {
{ .compatible = "ir-spi-led" },
{},
@@ -165,7 +160,6 @@ MODULE_DEVICE_TABLE(of, ir_spi_of_match);
static struct spi_driver ir_spi_driver = {
.probe = ir_spi_probe,
- .remove = ir_spi_remove,
.driver = {
.name = IR_SPI_DRIVER_NAME,
.of_match_table = ir_spi_of_match,
diff --git a/drivers/media/rc/lirc_dev.c b/drivers/media/rc/lirc_dev.c
index 7f591ff..c7c5157 100644
--- a/drivers/media/rc/lirc_dev.c
+++ b/drivers/media/rc/lirc_dev.c
@@ -102,8 +102,6 @@ void lirc_raw_event(struct rc_dev *dev, struct ir_raw_event ev)
spin_lock_irqsave(&dev->lirc_fh_lock, flags);
list_for_each_entry(fh, &dev->lirc_fh, list) {
- if (LIRC_IS_TIMEOUT(sample) && !fh->send_timeout_reports)
- continue;
if (kfifo_put(&fh->rawir, sample))
wake_up_poll(&fh->wait_poll, EPOLLIN | EPOLLRDNORM);
}
@@ -166,7 +164,6 @@ static int lirc_open(struct inode *inode, struct file *file)
fh->send_mode = LIRC_MODE_PULSE;
fh->rc = dev;
- fh->send_timeout_reports = true;
if (dev->driver_type == RC_DRIVER_SCANCODE)
fh->rec_mode = LIRC_MODE_SCANCODE;
@@ -570,8 +567,6 @@ static long lirc_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
case LIRC_SET_REC_TIMEOUT_REPORTS:
if (dev->driver_type != RC_DRIVER_IR_RAW)
ret = -ENOTTY;
- else
- fh->send_timeout_reports = !!val;
break;
default:
diff --git a/drivers/media/rc/mceusb.c b/drivers/media/rc/mceusb.c
index d09bee8..2dc810f 100644
--- a/drivers/media/rc/mceusb.c
+++ b/drivers/media/rc/mceusb.c
@@ -1430,7 +1430,7 @@ static void mceusb_gen1_init(struct mceusb_dev *ir)
*/
ret = usb_control_msg(ir->usbdev, usb_rcvctrlpipe(ir->usbdev, 0),
USB_REQ_SET_ADDRESS, USB_TYPE_VENDOR, 0, 0,
- data, USB_CTRL_MSG_SZ, HZ * 3);
+ data, USB_CTRL_MSG_SZ, 3000);
dev_dbg(dev, "set address - ret = %d", ret);
dev_dbg(dev, "set address - data[0] = %d, data[1] = %d",
data[0], data[1]);
@@ -1438,20 +1438,20 @@ static void mceusb_gen1_init(struct mceusb_dev *ir)
/* set feature: bit rate 38400 bps */
ret = usb_control_msg(ir->usbdev, usb_sndctrlpipe(ir->usbdev, 0),
USB_REQ_SET_FEATURE, USB_TYPE_VENDOR,
- 0xc04e, 0x0000, NULL, 0, HZ * 3);
+ 0xc04e, 0x0000, NULL, 0, 3000);
dev_dbg(dev, "set feature - ret = %d", ret);
/* bRequest 4: set char length to 8 bits */
ret = usb_control_msg(ir->usbdev, usb_sndctrlpipe(ir->usbdev, 0),
4, USB_TYPE_VENDOR,
- 0x0808, 0x0000, NULL, 0, HZ * 3);
+ 0x0808, 0x0000, NULL, 0, 3000);
dev_dbg(dev, "set char length - retB = %d", ret);
/* bRequest 2: set handshaking to use DTR/DSR */
ret = usb_control_msg(ir->usbdev, usb_sndctrlpipe(ir->usbdev, 0),
2, USB_TYPE_VENDOR,
- 0x0000, 0x0100, NULL, 0, HZ * 3);
+ 0x0000, 0x0100, NULL, 0, 3000);
dev_dbg(dev, "set handshake - retC = %d", ret);
/* device resume */
diff --git a/drivers/media/rc/pwm-ir-tx.c b/drivers/media/rc/pwm-ir-tx.c
index 4bc28d2..105a9c2 100644
--- a/drivers/media/rc/pwm-ir-tx.c
+++ b/drivers/media/rc/pwm-ir-tx.c
@@ -53,22 +53,21 @@ static int pwm_ir_tx(struct rc_dev *dev, unsigned int *txbuf,
{
struct pwm_ir *pwm_ir = dev->priv;
struct pwm_device *pwm = pwm_ir->pwm;
- int i, duty, period;
+ struct pwm_state state;
+ int i;
ktime_t edge;
long delta;
- period = DIV_ROUND_CLOSEST(NSEC_PER_SEC, pwm_ir->carrier);
- duty = DIV_ROUND_CLOSEST(pwm_ir->duty_cycle * period, 100);
+ pwm_init_state(pwm, &state);
- pwm_config(pwm, duty, period);
+ state.period = DIV_ROUND_CLOSEST(NSEC_PER_SEC, pwm_ir->carrier);
+ pwm_set_relative_duty_cycle(&state, pwm_ir->duty_cycle, 100);
edge = ktime_get();
for (i = 0; i < count; i++) {
- if (i % 2) // space
- pwm_disable(pwm);
- else
- pwm_enable(pwm);
+ state.enabled = !(i % 2);
+ pwm_apply_state(pwm, &state);
edge = ktime_add_us(edge, txbuf[i]);
delta = ktime_us_delta(edge, ktime_get());
@@ -76,7 +75,8 @@ static int pwm_ir_tx(struct rc_dev *dev, unsigned int *txbuf,
usleep_range(delta, delta + 10);
}
- pwm_disable(pwm);
+ state.enabled = false;
+ pwm_apply_state(pwm, &state);
return count;
}
diff --git a/drivers/media/rc/redrat3.c b/drivers/media/rc/redrat3.c
index ac85464..cb22316 100644
--- a/drivers/media/rc/redrat3.c
+++ b/drivers/media/rc/redrat3.c
@@ -404,7 +404,7 @@ static int redrat3_send_cmd(int cmd, struct redrat3_dev *rr3)
udev = rr3->udev;
res = usb_control_msg(udev, usb_rcvctrlpipe(udev, 0), cmd,
USB_TYPE_VENDOR | USB_RECIP_DEVICE | USB_DIR_IN,
- 0x0000, 0x0000, data, sizeof(u8), HZ * 10);
+ 0x0000, 0x0000, data, sizeof(u8), 10000);
if (res < 0) {
dev_err(rr3->dev, "%s: Error sending rr3 cmd res %d, data %d",
@@ -480,7 +480,7 @@ static u32 redrat3_get_timeout(struct redrat3_dev *rr3)
pipe = usb_rcvctrlpipe(rr3->udev, 0);
ret = usb_control_msg(rr3->udev, pipe, RR3_GET_IR_PARAM,
USB_TYPE_VENDOR | USB_RECIP_DEVICE | USB_DIR_IN,
- RR3_IR_IO_SIG_TIMEOUT, 0, tmp, len, HZ * 5);
+ RR3_IR_IO_SIG_TIMEOUT, 0, tmp, len, 5000);
if (ret != len)
dev_warn(rr3->dev, "Failed to read timeout from hardware\n");
else {
@@ -510,7 +510,7 @@ static int redrat3_set_timeout(struct rc_dev *rc_dev, unsigned int timeoutus)
ret = usb_control_msg(udev, usb_sndctrlpipe(udev, 0), RR3_SET_IR_PARAM,
USB_TYPE_VENDOR | USB_RECIP_DEVICE | USB_DIR_OUT,
RR3_IR_IO_SIG_TIMEOUT, 0, timeout, sizeof(*timeout),
- HZ * 25);
+ 25000);
dev_dbg(dev, "set ir parm timeout %d ret 0x%02x\n",
be32_to_cpu(*timeout), ret);
@@ -542,32 +542,32 @@ static void redrat3_reset(struct redrat3_dev *rr3)
*val = 0x01;
rc = usb_control_msg(udev, rxpipe, RR3_RESET,
USB_TYPE_VENDOR | USB_RECIP_DEVICE | USB_DIR_IN,
- RR3_CPUCS_REG_ADDR, 0, val, len, HZ * 25);
+ RR3_CPUCS_REG_ADDR, 0, val, len, 25000);
dev_dbg(dev, "reset returned 0x%02x\n", rc);
*val = length_fuzz;
rc = usb_control_msg(udev, txpipe, RR3_SET_IR_PARAM,
USB_TYPE_VENDOR | USB_RECIP_DEVICE | USB_DIR_OUT,
- RR3_IR_IO_LENGTH_FUZZ, 0, val, len, HZ * 25);
+ RR3_IR_IO_LENGTH_FUZZ, 0, val, len, 25000);
dev_dbg(dev, "set ir parm len fuzz %d rc 0x%02x\n", *val, rc);
*val = (65536 - (minimum_pause * 2000)) / 256;
rc = usb_control_msg(udev, txpipe, RR3_SET_IR_PARAM,
USB_TYPE_VENDOR | USB_RECIP_DEVICE | USB_DIR_OUT,
- RR3_IR_IO_MIN_PAUSE, 0, val, len, HZ * 25);
+ RR3_IR_IO_MIN_PAUSE, 0, val, len, 25000);
dev_dbg(dev, "set ir parm min pause %d rc 0x%02x\n", *val, rc);
*val = periods_measure_carrier;
rc = usb_control_msg(udev, txpipe, RR3_SET_IR_PARAM,
USB_TYPE_VENDOR | USB_RECIP_DEVICE | USB_DIR_OUT,
- RR3_IR_IO_PERIODS_MF, 0, val, len, HZ * 25);
+ RR3_IR_IO_PERIODS_MF, 0, val, len, 25000);
dev_dbg(dev, "set ir parm periods measure carrier %d rc 0x%02x", *val,
rc);
*val = RR3_DRIVER_MAXLENS;
rc = usb_control_msg(udev, txpipe, RR3_SET_IR_PARAM,
USB_TYPE_VENDOR | USB_RECIP_DEVICE | USB_DIR_OUT,
- RR3_IR_IO_MAX_LENGTHS, 0, val, len, HZ * 25);
+ RR3_IR_IO_MAX_LENGTHS, 0, val, len, 25000);
dev_dbg(dev, "set ir parm max lens %d rc 0x%02x\n", *val, rc);
kfree(val);
@@ -585,7 +585,7 @@ static void redrat3_get_firmware_rev(struct redrat3_dev *rr3)
rc = usb_control_msg(rr3->udev, usb_rcvctrlpipe(rr3->udev, 0),
RR3_FW_VERSION,
USB_TYPE_VENDOR | USB_RECIP_DEVICE | USB_DIR_IN,
- 0, 0, buffer, RR3_FW_VERSION_LEN, HZ * 5);
+ 0, 0, buffer, RR3_FW_VERSION_LEN, 5000);
if (rc >= 0)
dev_info(rr3->dev, "Firmware rev: %s", buffer);
@@ -825,14 +825,14 @@ static int redrat3_transmit_ir(struct rc_dev *rcdev, unsigned *txbuf,
pipe = usb_sndbulkpipe(rr3->udev, rr3->ep_out->bEndpointAddress);
ret = usb_bulk_msg(rr3->udev, pipe, irdata,
- sendbuf_len, &ret_len, 10 * HZ);
+ sendbuf_len, &ret_len, 10000);
dev_dbg(dev, "sent %d bytes, (ret %d)\n", ret_len, ret);
/* now tell the hardware to transmit what we sent it */
pipe = usb_rcvctrlpipe(rr3->udev, 0);
ret = usb_control_msg(rr3->udev, pipe, RR3_TX_SEND_SIGNAL,
USB_TYPE_VENDOR | USB_RECIP_DEVICE | USB_DIR_IN,
- 0, 0, irdata, 2, HZ * 10);
+ 0, 0, irdata, 2, 10000);
if (ret < 0)
dev_err(dev, "Error: control msg send failed, rc %d\n", ret);
diff --git a/drivers/media/rc/streamzap.c b/drivers/media/rc/streamzap.c
index 1cc5ebb..16ba85d 100644
--- a/drivers/media/rc/streamzap.c
+++ b/drivers/media/rc/streamzap.c
@@ -21,12 +21,10 @@
#include <linux/device.h>
#include <linux/module.h>
#include <linux/slab.h>
-#include <linux/ktime.h>
#include <linux/usb.h>
#include <linux/usb/input.h>
#include <media/rc-core.h>
-#define DRIVER_VERSION "1.61"
#define DRIVER_NAME "streamzap"
#define DRIVER_DESC "Streamzap Remote Control driver"
@@ -67,9 +65,6 @@ struct streamzap_ir {
struct device *dev;
/* usb */
- struct usb_device *usbdev;
- struct usb_interface *interface;
- struct usb_endpoint_descriptor *endpoint;
struct urb *urb_in;
/* buffer & dma */
@@ -79,16 +74,7 @@ struct streamzap_ir {
/* track what state we're in */
enum StreamzapDecoderState decoder_state;
- /* tracks whether we are currently receiving some signal */
- bool idle;
- /* sum of signal lengths received since signal start */
- unsigned long sum;
- /* start time of signal; necessary for gap tracking */
- ktime_t signal_last;
- ktime_t signal_start;
- bool timeout_enabled;
- char name[128];
char phys[64];
};
@@ -121,37 +107,11 @@ static void sz_push(struct streamzap_ir *sz, struct ir_raw_event rawir)
static void sz_push_full_pulse(struct streamzap_ir *sz,
unsigned char value)
{
- struct ir_raw_event rawir = {};
+ struct ir_raw_event rawir = {
+ .pulse = true,
+ .duration = value * SZ_RESOLUTION + SZ_RESOLUTION / 2,
+ };
- if (sz->idle) {
- int delta;
-
- sz->signal_last = sz->signal_start;
- sz->signal_start = ktime_get_real();
-
- delta = ktime_us_delta(sz->signal_start, sz->signal_last);
- rawir.pulse = false;
- if (delta > (15 * USEC_PER_SEC)) {
- /* really long time */
- rawir.duration = IR_MAX_DURATION;
- } else {
- rawir.duration = delta;
- rawir.duration -= sz->sum;
- rawir.duration = (rawir.duration > IR_MAX_DURATION) ?
- IR_MAX_DURATION : rawir.duration;
- }
- sz_push(sz, rawir);
-
- sz->idle = false;
- sz->sum = 0;
- }
-
- rawir.pulse = true;
- rawir.duration = ((int) value) * SZ_RESOLUTION;
- rawir.duration += SZ_RESOLUTION / 2;
- sz->sum += rawir.duration;
- rawir.duration = (rawir.duration > IR_MAX_DURATION) ?
- IR_MAX_DURATION : rawir.duration;
sz_push(sz, rawir);
}
@@ -164,12 +124,11 @@ static void sz_push_half_pulse(struct streamzap_ir *sz,
static void sz_push_full_space(struct streamzap_ir *sz,
unsigned char value)
{
- struct ir_raw_event rawir = {};
+ struct ir_raw_event rawir = {
+ .pulse = false,
+ .duration = value * SZ_RESOLUTION + SZ_RESOLUTION / 2,
+ };
- rawir.pulse = false;
- rawir.duration = ((int) value) * SZ_RESOLUTION;
- rawir.duration += SZ_RESOLUTION / 2;
- sz->sum += rawir.duration;
sz_push(sz, rawir);
}
@@ -241,11 +200,7 @@ static void streamzap_callback(struct urb *urb)
.pulse = false,
.duration = sz->rdev->timeout
};
- sz->idle = true;
- if (sz->timeout_enabled)
- sz_push(sz, rawir);
- ir_raw_event_handle(sz->rdev);
- ir_raw_event_reset(sz->rdev);
+ sz_push(sz, rawir);
} else {
sz_push_full_space(sz, sz->buf_in[i]);
}
@@ -267,27 +222,23 @@ static void streamzap_callback(struct urb *urb)
usb_submit_urb(urb, GFP_ATOMIC);
}
-static struct rc_dev *streamzap_init_rc_dev(struct streamzap_ir *sz)
+static struct rc_dev *streamzap_init_rc_dev(struct streamzap_ir *sz,
+ struct usb_device *usbdev)
{
struct rc_dev *rdev;
struct device *dev = sz->dev;
int ret;
rdev = rc_allocate_device(RC_DRIVER_IR_RAW);
- if (!rdev) {
- dev_err(dev, "remote dev allocation failed\n");
+ if (!rdev)
goto out;
- }
- snprintf(sz->name, sizeof(sz->name), "Streamzap PC Remote Infrared Receiver (%04x:%04x)",
- le16_to_cpu(sz->usbdev->descriptor.idVendor),
- le16_to_cpu(sz->usbdev->descriptor.idProduct));
- usb_make_path(sz->usbdev, sz->phys, sizeof(sz->phys));
+ usb_make_path(usbdev, sz->phys, sizeof(sz->phys));
strlcat(sz->phys, "/input0", sizeof(sz->phys));
- rdev->device_name = sz->name;
+ rdev->device_name = "Streamzap PC Remote Infrared Receiver";
rdev->input_phys = sz->phys;
- usb_to_input_id(sz->usbdev, &rdev->input_id);
+ usb_to_input_id(usbdev, &rdev->input_id);
rdev->dev.parent = dev;
rdev->priv = sz;
rdev->allowed_protocols = RC_PROTO_BIT_ALL_IR_DECODER;
@@ -319,9 +270,9 @@ static int streamzap_probe(struct usb_interface *intf,
const struct usb_device_id *id)
{
struct usb_device *usbdev = interface_to_usbdev(intf);
+ struct usb_endpoint_descriptor *endpoint;
struct usb_host_interface *iface_host;
struct streamzap_ir *sz = NULL;
- char buf[63], name[128] = "";
int retval = -ENOMEM;
int pipe, maxp;
@@ -330,9 +281,6 @@ static int streamzap_probe(struct usb_interface *intf,
if (!sz)
return -ENOMEM;
- sz->usbdev = usbdev;
- sz->interface = intf;
-
/* Check to ensure endpoint information matches requirements */
iface_host = intf->cur_altsetting;
@@ -343,22 +291,22 @@ static int streamzap_probe(struct usb_interface *intf,
goto free_sz;
}
- sz->endpoint = &(iface_host->endpoint[0].desc);
- if (!usb_endpoint_dir_in(sz->endpoint)) {
+ endpoint = &iface_host->endpoint[0].desc;
+ if (!usb_endpoint_dir_in(endpoint)) {
dev_err(&intf->dev, "%s: endpoint doesn't match input device 02%02x\n",
- __func__, sz->endpoint->bEndpointAddress);
+ __func__, endpoint->bEndpointAddress);
retval = -ENODEV;
goto free_sz;
}
- if (!usb_endpoint_xfer_int(sz->endpoint)) {
+ if (!usb_endpoint_xfer_int(endpoint)) {
dev_err(&intf->dev, "%s: endpoint attributes don't match xfer 02%02x\n",
- __func__, sz->endpoint->bmAttributes);
+ __func__, endpoint->bmAttributes);
retval = -ENODEV;
goto free_sz;
}
- pipe = usb_rcvintpipe(usbdev, sz->endpoint->bEndpointAddress);
+ pipe = usb_rcvintpipe(usbdev, endpoint->bEndpointAddress);
maxp = usb_maxpacket(usbdev, pipe, usb_pipeout(pipe));
if (maxp == 0) {
@@ -380,25 +328,12 @@ static int streamzap_probe(struct usb_interface *intf,
sz->dev = &intf->dev;
sz->buf_in_len = maxp;
- if (usbdev->descriptor.iManufacturer
- && usb_string(usbdev, usbdev->descriptor.iManufacturer,
- buf, sizeof(buf)) > 0)
- strscpy(name, buf, sizeof(name));
-
- if (usbdev->descriptor.iProduct
- && usb_string(usbdev, usbdev->descriptor.iProduct,
- buf, sizeof(buf)) > 0)
- snprintf(name + strlen(name), sizeof(name) - strlen(name),
- " %s", buf);
-
- sz->rdev = streamzap_init_rc_dev(sz);
+ sz->rdev = streamzap_init_rc_dev(sz, usbdev);
if (!sz->rdev)
goto rc_dev_fail;
- sz->idle = true;
sz->decoder_state = PulseSpace;
/* FIXME: don't yet have a way to set this */
- sz->timeout_enabled = true;
sz->rdev->timeout = SZ_TIMEOUT * SZ_RESOLUTION;
#if 0
/* not yet supported, depends on patches from maxim */
@@ -407,12 +342,9 @@ static int streamzap_probe(struct usb_interface *intf,
sz->max_timeout = SZ_TIMEOUT * SZ_RESOLUTION;
#endif
- sz->signal_start = ktime_get_real();
-
/* Complete final initialisations */
usb_fill_int_urb(sz->urb_in, usbdev, pipe, sz->buf_in,
- maxp, (usb_complete_t)streamzap_callback,
- sz, sz->endpoint->bInterval);
+ maxp, streamzap_callback, sz, endpoint->bInterval);
sz->urb_in->transfer_dma = sz->dma_in;
sz->urb_in->transfer_flags |= URB_NO_TRANSFER_DMA_MAP;
@@ -421,9 +353,6 @@ static int streamzap_probe(struct usb_interface *intf,
if (usb_submit_urb(sz->urb_in, GFP_ATOMIC))
dev_err(sz->dev, "urb submit failed\n");
- dev_info(sz->dev, "Registered %s on usb%d:%d\n", name,
- usbdev->bus->busnum, usbdev->devnum);
-
return 0;
rc_dev_fail:
@@ -456,7 +385,6 @@ static void streamzap_disconnect(struct usb_interface *interface)
if (!sz)
return;
- sz->usbdev = NULL;
rc_unregister_device(sz->rdev);
usb_kill_urb(sz->urb_in);
usb_free_urb(sz->urb_in);
diff --git a/drivers/media/rc/winbond-cir.c b/drivers/media/rc/winbond-cir.c
index aed23ca..94efb03 100644
--- a/drivers/media/rc/winbond-cir.c
+++ b/drivers/media/rc/winbond-cir.c
@@ -997,7 +997,6 @@ wbcir_resume(struct pnp_dev *device)
struct wbcir_data *data = pnp_get_drvdata(device);
wbcir_init_hw(data);
- ir_raw_event_reset(data->dev);
enable_irq(data->irq);
led_classdev_resume(&data->led);
diff --git a/drivers/media/spi/cxd2880-spi.c b/drivers/media/spi/cxd2880-spi.c
index 506f52c..6f2a66b 100644
--- a/drivers/media/spi/cxd2880-spi.c
+++ b/drivers/media/spi/cxd2880-spi.c
@@ -628,19 +628,8 @@ cxd2880_spi_probe(struct spi_device *spi)
static int
cxd2880_spi_remove(struct spi_device *spi)
{
- struct cxd2880_dvb_spi *dvb_spi;
+ struct cxd2880_dvb_spi *dvb_spi = spi_get_drvdata(spi);
- if (!spi) {
- pr_err("invalid arg\n");
- return -EINVAL;
- }
-
- dvb_spi = spi_get_drvdata(spi);
-
- if (!dvb_spi) {
- pr_err("failed\n");
- return -EINVAL;
- }
dvb_spi->demux.dmx.remove_frontend(&dvb_spi->demux.dmx,
&dvb_spi->dmx_fe);
dvb_dmxdev_release(&dvb_spi->dmxdev);
diff --git a/drivers/media/test-drivers/vicodec/vicodec-core.c b/drivers/media/test-drivers/vicodec/vicodec-core.c
index 33f1c89..be43f7d 100644
--- a/drivers/media/test-drivers/vicodec/vicodec-core.c
+++ b/drivers/media/test-drivers/vicodec/vicodec-core.c
@@ -1443,7 +1443,7 @@ static void vicodec_buf_queue(struct vb2_buffer *vb)
unsigned int i;
for (i = 0; i < vb->num_planes; i++)
- vb->planes[i].bytesused = 0;
+ vb2_set_plane_payload(vb, i, 0);
vbuf->field = V4L2_FIELD_NONE;
vbuf->sequence =
diff --git a/drivers/media/test-drivers/vidtv/vidtv_psi.c b/drivers/media/test-drivers/vidtv/vidtv_psi.c
index c11ac8d..a587538 100644
--- a/drivers/media/test-drivers/vidtv/vidtv_psi.c
+++ b/drivers/media/test-drivers/vidtv/vidtv_psi.c
@@ -94,34 +94,28 @@ static void vidtv_psi_update_version_num(struct vidtv_psi_table_header *h)
static u16 vidtv_psi_get_sec_len(struct vidtv_psi_table_header *h)
{
u16 mask;
- u16 ret;
mask = GENMASK(11, 0);
- ret = be16_to_cpu(h->bitfield) & mask;
- return ret;
+ return be16_to_cpu(h->bitfield) & mask;
}
u16 vidtv_psi_get_pat_program_pid(struct vidtv_psi_table_pat_program *p)
{
u16 mask;
- u16 ret;
mask = GENMASK(12, 0);
- ret = be16_to_cpu(p->bitfield) & mask;
- return ret;
+ return be16_to_cpu(p->bitfield) & mask;
}
u16 vidtv_psi_pmt_stream_get_elem_pid(struct vidtv_psi_table_pmt_stream *s)
{
u16 mask;
- u16 ret;
mask = GENMASK(12, 0);
- ret = be16_to_cpu(s->bitfield) & mask;
- return ret;
+ return be16_to_cpu(s->bitfield) & mask;
}
static void vidtv_psi_set_desc_loop_len(__be16 *bitfield, u16 new_len,
diff --git a/drivers/media/tuners/msi001.c b/drivers/media/tuners/msi001.c
index 78e6fd60..4424704 100644
--- a/drivers/media/tuners/msi001.c
+++ b/drivers/media/tuners/msi001.c
@@ -442,6 +442,13 @@ static int msi001_probe(struct spi_device *spi)
V4L2_CID_RF_TUNER_BANDWIDTH_AUTO, 0, 1, 1, 1);
dev->bandwidth = v4l2_ctrl_new_std(&dev->hdl, &msi001_ctrl_ops,
V4L2_CID_RF_TUNER_BANDWIDTH, 200000, 8000000, 1, 200000);
+ if (dev->hdl.error) {
+ ret = dev->hdl.error;
+ dev_err(&spi->dev, "Could not initialize controls\n");
+ /* control init failed, free handler */
+ goto err_ctrl_handler_free;
+ }
+
v4l2_ctrl_auto_cluster(2, &dev->bandwidth_auto, 0, false);
dev->lna_gain = v4l2_ctrl_new_std(&dev->hdl, &msi001_ctrl_ops,
V4L2_CID_RF_TUNER_LNA_GAIN, 0, 1, 1, 1);
diff --git a/drivers/media/tuners/mxl5005s.c b/drivers/media/tuners/mxl5005s.c
index f6e82a8..ab4c43d 100644
--- a/drivers/media/tuners/mxl5005s.c
+++ b/drivers/media/tuners/mxl5005s.c
@@ -3414,9 +3414,8 @@ static u16 MXL_ControlWrite_Group(struct dvb_frontend *fe, u16 controlNum,
u32 value, u16 controlGroup)
{
struct mxl5005s_state *state = fe->tuner_priv;
- u16 i, j, k;
+ u16 i, j;
u32 highLimit;
- u32 ctrlVal;
if (controlGroup == 1) /* Initial Control */ {
@@ -3432,9 +3431,6 @@ static u16 MXL_ControlWrite_Group(struct dvb_frontend *fe, u16 controlNum,
(u8)(state->Init_Ctrl[i].bit[j]),
(u8)((value>>j) & 0x01));
}
- ctrlVal = 0;
- for (k = 0; k < state->Init_Ctrl[i].size; k++)
- ctrlVal += state->Init_Ctrl[i].val[k] * (1 << k);
} else
return -1;
}
@@ -3454,9 +3450,6 @@ static u16 MXL_ControlWrite_Group(struct dvb_frontend *fe, u16 controlNum,
(u8)(state->CH_Ctrl[i].bit[j]),
(u8)((value>>j) & 0x01));
}
- ctrlVal = 0;
- for (k = 0; k < state->CH_Ctrl[i].size; k++)
- ctrlVal += state->CH_Ctrl[i].val[k] * (1 << k);
} else
return -1;
}
@@ -3477,11 +3470,6 @@ static u16 MXL_ControlWrite_Group(struct dvb_frontend *fe, u16 controlNum,
(u8)(state->MXL_Ctrl[i].bit[j]),
(u8)((value>>j) & 0x01));
}
- ctrlVal = 0;
- for (k = 0; k < state->MXL_Ctrl[i].size; k++)
- ctrlVal += state->
- MXL_Ctrl[i].val[k] *
- (1 << k);
} else
return -1;
}
diff --git a/drivers/media/tuners/r820t.c b/drivers/media/tuners/r820t.c
index aed2f13..b9a7590 100644
--- a/drivers/media/tuners/r820t.c
+++ b/drivers/media/tuners/r820t.c
@@ -326,6 +326,26 @@ static int r820t_xtal_capacitor[][2] = {
{ 0x10, XTAL_HIGH_CAP_0P },
};
+static const char *r820t_chip_enum_to_str(enum r820t_chip chip)
+{
+ switch (chip) {
+ case CHIP_R820T:
+ return "R820T";
+ case CHIP_R620D:
+ return "R620D";
+ case CHIP_R828D:
+ return "R828D";
+ case CHIP_R828:
+ return "R828";
+ case CHIP_R828S:
+ return "R828S";
+ case CHIP_R820C:
+ return "R820C";
+ default:
+ return "<unknown>";
+ }
+}
+
/*
* I2C read/write code and shadow registers logic
*/
@@ -2355,7 +2375,9 @@ struct dvb_frontend *r820t_attach(struct dvb_frontend *fe,
if (rc < 0)
goto err;
- tuner_info("Rafael Micro r820t successfully identified\n");
+ tuner_info(
+ "Rafael Micro r820t successfully identified, chip type: %s\n",
+ r820t_chip_enum_to_str(cfg->rafael_chip));
if (fe->ops.i2c_gate_ctrl)
fe->ops.i2c_gate_ctrl(fe, 0);
diff --git a/drivers/media/tuners/si2157.c b/drivers/media/tuners/si2157.c
index fefb262..481c5c3 100644
--- a/drivers/media/tuners/si2157.c
+++ b/drivers/media/tuners/si2157.c
@@ -76,129 +76,42 @@ static int si2157_cmd_execute(struct i2c_client *client, struct si2157_cmd *cmd)
return ret;
}
-static int si2157_init(struct dvb_frontend *fe)
+static const struct si2157_tuner_info si2157_tuners[] = {
+ { SI2141, false, 0x60, SI2141_60_FIRMWARE, SI2141_A10_FIRMWARE },
+ { SI2141, false, 0x61, SI2141_61_FIRMWARE, SI2141_A10_FIRMWARE },
+ { SI2146, false, 0x11, SI2146_11_FIRMWARE, NULL },
+ { SI2147, false, 0x50, SI2147_50_FIRMWARE, NULL },
+ { SI2148, true, 0x32, SI2148_32_FIRMWARE, SI2158_A20_FIRMWARE },
+ { SI2148, true, 0x33, SI2148_33_FIRMWARE, SI2158_A20_FIRMWARE },
+ { SI2157, false, 0x50, SI2157_50_FIRMWARE, SI2157_A30_FIRMWARE },
+ { SI2158, false, 0x50, SI2158_50_FIRMWARE, SI2158_A20_FIRMWARE },
+ { SI2158, false, 0x51, SI2158_51_FIRMWARE, SI2158_A20_FIRMWARE },
+ { SI2177, false, 0x50, SI2177_50_FIRMWARE, SI2157_A30_FIRMWARE },
+};
+
+static int si2157_load_firmware(struct dvb_frontend *fe,
+ const char *fw_name)
{
struct i2c_client *client = fe->tuner_priv;
- struct si2157_dev *dev = i2c_get_clientdata(client);
- struct dtv_frontend_properties *c = &fe->dtv_property_cache;
+ const struct firmware *fw;
int ret, len, remaining;
struct si2157_cmd cmd;
- const struct firmware *fw;
- const char *fw_name;
- unsigned int chip_id, xtal_trim;
-
- dev_dbg(&client->dev, "\n");
-
- /* Try to get Xtal trim property, to verify tuner still running */
- memcpy(cmd.args, "\x15\x00\x04\x02", 4);
- cmd.wlen = 4;
- cmd.rlen = 4;
- ret = si2157_cmd_execute(client, &cmd);
-
- xtal_trim = cmd.args[2] | (cmd.args[3] << 8);
-
- if (ret == 0 && xtal_trim < 16)
- goto warm;
-
- dev->if_frequency = 0; /* we no longer know current tuner state */
-
- /* power up */
- if (dev->chiptype == SI2157_CHIPTYPE_SI2146) {
- memcpy(cmd.args, "\xc0\x05\x01\x00\x00\x0b\x00\x00\x01", 9);
- cmd.wlen = 9;
- } else if (dev->chiptype == SI2157_CHIPTYPE_SI2141) {
- memcpy(cmd.args, "\xc0\x00\x0d\x0e\x00\x01\x01\x01\x01\x03", 10);
- cmd.wlen = 10;
- } else {
- memcpy(cmd.args, "\xc0\x00\x0c\x00\x00\x01\x01\x01\x01\x01\x01\x02\x00\x00\x01", 15);
- cmd.wlen = 15;
- }
- cmd.rlen = 1;
- ret = si2157_cmd_execute(client, &cmd);
- if (ret && (dev->chiptype != SI2157_CHIPTYPE_SI2141 || ret != -EAGAIN))
- goto err;
-
- /* Si2141 needs a second command before it answers the revision query */
- if (dev->chiptype == SI2157_CHIPTYPE_SI2141) {
- memcpy(cmd.args, "\xc0\x08\x01\x02\x00\x00\x01", 7);
- cmd.wlen = 7;
- ret = si2157_cmd_execute(client, &cmd);
- if (ret)
- goto err;
- }
-
- if (dev->dont_load_firmware) {
- dev_info(&client->dev, "device is buggy, skipping firmware download\n");
- goto skip_fw_download;
- }
-
- /* query chip revision */
- memcpy(cmd.args, "\x02", 1);
- cmd.wlen = 1;
- cmd.rlen = 13;
- ret = si2157_cmd_execute(client, &cmd);
- if (ret)
- goto err;
-
- chip_id = cmd.args[1] << 24 | cmd.args[2] << 16 | cmd.args[3] << 8 |
- cmd.args[4] << 0;
-
- #define SI2177_A30 ('A' << 24 | 77 << 16 | '3' << 8 | '0' << 0)
- #define SI2158_A20 ('A' << 24 | 58 << 16 | '2' << 8 | '0' << 0)
- #define SI2148_A20 ('A' << 24 | 48 << 16 | '2' << 8 | '0' << 0)
- #define SI2157_A30 ('A' << 24 | 57 << 16 | '3' << 8 | '0' << 0)
- #define SI2147_A30 ('A' << 24 | 47 << 16 | '3' << 8 | '0' << 0)
- #define SI2146_A10 ('A' << 24 | 46 << 16 | '1' << 8 | '0' << 0)
- #define SI2141_A10 ('A' << 24 | 41 << 16 | '1' << 8 | '0' << 0)
-
- switch (chip_id) {
- case SI2158_A20:
- case SI2148_A20:
- fw_name = SI2158_A20_FIRMWARE;
- break;
- case SI2141_A10:
- fw_name = SI2141_A10_FIRMWARE;
- break;
- case SI2177_A30:
- fw_name = SI2157_A30_FIRMWARE;
- break;
- case SI2157_A30:
- case SI2147_A30:
- case SI2146_A10:
- fw_name = NULL;
- break;
- default:
- dev_err(&client->dev, "unknown chip version Si21%d-%c%c%c\n",
- cmd.args[2], cmd.args[1],
- cmd.args[3], cmd.args[4]);
- ret = -EINVAL;
- goto err;
- }
-
- dev_info(&client->dev, "found a 'Silicon Labs Si21%d-%c%c%c'\n",
- cmd.args[2], cmd.args[1], cmd.args[3], cmd.args[4]);
-
- if (fw_name == NULL)
- goto skip_fw_download;
/* request the firmware, this will block and timeout */
- ret = request_firmware(&fw, fw_name, &client->dev);
- if (ret) {
- dev_err(&client->dev, "firmware file '%s' not found\n",
- fw_name);
- goto err;
- }
+ ret = firmware_request_nowarn(&fw, fw_name, &client->dev);
+ if (ret)
+ return ret;
/* firmware should be n chunks of 17 bytes */
if (fw->size % 17 != 0) {
dev_err(&client->dev, "firmware file '%s' is invalid\n",
- fw_name);
+ fw_name);
ret = -EINVAL;
goto err_release_firmware;
}
dev_info(&client->dev, "downloading firmware from file '%s'\n",
- fw_name);
+ fw_name);
for (remaining = fw->size; remaining > 0; remaining -= 17) {
len = fw->data[fw->size - remaining];
@@ -218,9 +131,144 @@ static int si2157_init(struct dvb_frontend *fe)
}
}
+err_release_firmware:
release_firmware(fw);
-skip_fw_download:
+ return ret;
+}
+
+static int si2157_find_and_load_firmware(struct dvb_frontend *fe)
+{
+ struct i2c_client *client = fe->tuner_priv;
+ struct si2157_dev *dev = i2c_get_clientdata(client);
+ const char *fw_alt_name = NULL;
+ unsigned char part_id, rom_id;
+ const char *fw_name = NULL;
+ struct si2157_cmd cmd;
+ bool required = true;
+ int ret, i;
+
+ if (dev->dont_load_firmware) {
+ dev_info(&client->dev,
+ "device is buggy, skipping firmware download\n");
+ return 0;
+ }
+
+ /* query chip revision */
+ memcpy(cmd.args, "\x02", 1);
+ cmd.wlen = 1;
+ cmd.rlen = 13;
+ ret = si2157_cmd_execute(client, &cmd);
+ if (ret)
+ return ret;
+
+ part_id = cmd.args[2];
+ rom_id = cmd.args[12];
+
+ for (i = 0; i < ARRAY_SIZE(si2157_tuners); i++) {
+ if (si2157_tuners[i].part_id != part_id)
+ continue;
+ required = si2157_tuners[i].required;
+ fw_alt_name = si2157_tuners[i].fw_alt_name;
+
+ /* Both part and rom ID match */
+ if (si2157_tuners[i].rom_id == rom_id) {
+ fw_name = si2157_tuners[i].fw_name;
+ break;
+ }
+ }
+
+ if (!fw_name && !fw_alt_name) {
+ dev_err(&client->dev,
+ "unknown chip version Si21%d-%c%c%c ROM 0x%02x\n",
+ part_id, cmd.args[1], cmd.args[3], cmd.args[4], rom_id);
+ return -EINVAL;
+ }
+
+ /* Update the part id based on device's report */
+ dev->part_id = part_id;
+
+ dev_info(&client->dev,
+ "found a 'Silicon Labs Si21%d-%c%c%c ROM 0x%02x'\n",
+ part_id, cmd.args[1], cmd.args[3], cmd.args[4], rom_id);
+
+ if (fw_name)
+ ret = si2157_load_firmware(fe, fw_name);
+ else
+ ret = -ENOENT;
+
+ /* Try alternate name, if any */
+ if (ret == -ENOENT && fw_alt_name)
+ ret = si2157_load_firmware(fe, fw_alt_name);
+
+ if (ret == -ENOENT) {
+ if (!required) {
+ dev_info(&client->dev, "Using ROM firmware.\n");
+ return 0;
+ }
+ dev_err(&client->dev, "Can't continue without a firmware.\n");
+ } else if (ret < 0) {
+ dev_err(&client->dev, "error %d when loading firmware\n", ret);
+ }
+ return ret;
+}
+
+static int si2157_init(struct dvb_frontend *fe)
+{
+ struct dtv_frontend_properties *c = &fe->dtv_property_cache;
+ struct i2c_client *client = fe->tuner_priv;
+ struct si2157_dev *dev = i2c_get_clientdata(client);
+ unsigned int xtal_trim;
+ struct si2157_cmd cmd;
+ int ret;
+
+ dev_dbg(&client->dev, "\n");
+
+ /* Try to get Xtal trim property, to verify tuner still running */
+ memcpy(cmd.args, "\x15\x00\x02\x04", 4);
+ cmd.wlen = 4;
+ cmd.rlen = 4;
+ ret = si2157_cmd_execute(client, &cmd);
+
+ xtal_trim = cmd.args[2] | (cmd.args[3] << 8);
+
+ if (ret == 0 && xtal_trim < 16)
+ goto warm;
+
+ dev->if_frequency = 0; /* we no longer know current tuner state */
+
+ /* power up */
+ if (dev->part_id == SI2146) {
+ /* clock_mode = XTAL, clock_freq = 24MHz */
+ memcpy(cmd.args, "\xc0\x05\x01\x00\x00\x0b\x00\x00\x01", 9);
+ cmd.wlen = 9;
+ } else if (dev->part_id == SI2141) {
+ /* clock_mode: XTAL, xout enabled */
+ memcpy(cmd.args, "\xc0\x00\x0d\x0e\x00\x01\x01\x01\x01\x03", 10);
+ cmd.wlen = 10;
+ } else {
+ memcpy(cmd.args, "\xc0\x00\x0c\x00\x00\x01\x01\x01\x01\x01\x01\x02\x00\x00\x01", 15);
+ cmd.wlen = 15;
+ }
+ cmd.rlen = 1;
+ ret = si2157_cmd_execute(client, &cmd);
+ if (ret && (dev->part_id != SI2141 || ret != -EAGAIN))
+ goto err;
+
+ /* Si2141 needs a wake up command */
+ if (dev->part_id == SI2141) {
+ memcpy(cmd.args, "\xc0\x08\x01\x02\x00\x00\x01", 7);
+ cmd.wlen = 7;
+ ret = si2157_cmd_execute(client, &cmd);
+ if (ret)
+ goto err;
+ }
+
+ /* Try to load the firmware */
+ ret = si2157_find_and_load_firmware(fe);
+ if (ret < 0)
+ goto err;
+
/* reboot the tuner with new firmware? */
memcpy(cmd.args, "\x01\x01", 2);
cmd.wlen = 2;
@@ -270,8 +318,7 @@ static int si2157_init(struct dvb_frontend *fe)
dev->active = true;
return 0;
-err_release_firmware:
- release_firmware(fw);
+
err:
dev_dbg(&client->dev, "failed=%d\n", ret);
return ret;
@@ -411,8 +458,12 @@ static int si2157_set_params(struct dvb_frontend *fe)
goto err;
}
+ if (SUPPORTS_1700KHz(dev) && c->bandwidth_hz <= 1700000)
+ bandwidth = 0x09;
if (c->bandwidth_hz <= 6000000)
bandwidth = 0x06;
+ if (SUPPORTS_1700KHz(dev) && c->bandwidth_hz <= 6100000)
+ bandwidth = 0x10;
else if (c->bandwidth_hz <= 7000000)
bandwidth = 0x07;
else if (c->bandwidth_hz <= 8000000)
@@ -436,6 +487,12 @@ static int si2157_set_params(struct dvb_frontend *fe)
case SYS_DVBC_ANNEX_A:
delivery_system = 0x30;
break;
+ case SYS_ISDBT:
+ delivery_system = 0x40;
+ break;
+ case SYS_DTMB:
+ delivery_system = 0x60;
+ break;
default:
ret = -EINVAL;
goto err;
@@ -451,7 +508,8 @@ static int si2157_set_params(struct dvb_frontend *fe)
if (ret)
goto err;
- if (dev->chiptype == SI2157_CHIPTYPE_SI2146)
+ /* On SI2146, set DTV AGC source to DLIF_AGC_3DB */
+ if (dev->part_id == SI2146)
memcpy(cmd.args, "\x14\x00\x02\x07\x00\x01", 6);
else
memcpy(cmd.args, "\x14\x00\x02\x07\x00\x00", 6);
@@ -518,9 +576,9 @@ static int si2157_set_analog_params(struct dvb_frontend *fe,
u8 color = 0; /* 0=NTSC/PAL, 0x10=SECAM */
u8 invert_analog = 1; /* analog tuner spectrum; 0=normal, 1=inverted */
- if (dev->chiptype != SI2157_CHIPTYPE_SI2157) {
- dev_info(&client->dev, "Analog tuning not supported for chiptype=%u\n",
- dev->chiptype);
+ if (!SUPPORTS_ATV_IF(dev)) {
+ dev_info(&client->dev, "Analog tuning not supported yet for Si21%d\n",
+ dev->part_id);
ret = -EINVAL;
goto err;
}
@@ -832,7 +890,7 @@ static int si2157_probe(struct i2c_client *client,
dev->inversion = cfg->inversion;
dev->dont_load_firmware = cfg->dont_load_firmware;
dev->if_port = cfg->if_port;
- dev->chiptype = (u8)id->driver_data;
+ dev->part_id = (u8)id->driver_data;
dev->if_frequency = 5000000; /* default value of property 0x0706 */
mutex_init(&dev->i2c_mutex);
INIT_DELAYED_WORK(&dev->stat_work, si2157_stat_work);
@@ -875,10 +933,8 @@ static int si2157_probe(struct i2c_client *client,
}
#endif
- dev_info(&client->dev, "Silicon Labs %s successfully attached\n",
- dev->chiptype == SI2157_CHIPTYPE_SI2141 ? "Si2141" :
- dev->chiptype == SI2157_CHIPTYPE_SI2146 ?
- "Si2146" : "Si2147/2148/2157/2158");
+ dev_info(&client->dev, "Silicon Labs Si21%d successfully attached\n",
+ dev->part_id);
return 0;
@@ -911,11 +967,16 @@ static int si2157_remove(struct i2c_client *client)
return 0;
}
+/*
+ * The part_id used here will only be used on buggy devices that don't
+ * accept firmware uploads. Non-buggy devices should just use "si2157" for
+ * all SiLabs TER tuners, as the driver should auto-detect it.
+ */
static const struct i2c_device_id si2157_id_table[] = {
- {"si2157", SI2157_CHIPTYPE_SI2157},
- {"si2146", SI2157_CHIPTYPE_SI2146},
- {"si2141", SI2157_CHIPTYPE_SI2141},
- {"si2177", SI2157_CHIPTYPE_SI2177},
+ {"si2157", SI2157},
+ {"si2146", SI2146},
+ {"si2141", SI2141},
+ {"si2177", SI2177},
{}
};
MODULE_DEVICE_TABLE(i2c, si2157_id_table);
@@ -938,3 +999,13 @@ MODULE_LICENSE("GPL");
MODULE_FIRMWARE(SI2158_A20_FIRMWARE);
MODULE_FIRMWARE(SI2141_A10_FIRMWARE);
MODULE_FIRMWARE(SI2157_A30_FIRMWARE);
+MODULE_FIRMWARE(SI2141_60_FIRMWARE);
+MODULE_FIRMWARE(SI2141_61_FIRMWARE);
+MODULE_FIRMWARE(SI2146_11_FIRMWARE);
+MODULE_FIRMWARE(SI2147_50_FIRMWARE);
+MODULE_FIRMWARE(SI2148_32_FIRMWARE);
+MODULE_FIRMWARE(SI2148_33_FIRMWARE);
+MODULE_FIRMWARE(SI2157_50_FIRMWARE);
+MODULE_FIRMWARE(SI2158_50_FIRMWARE);
+MODULE_FIRMWARE(SI2158_51_FIRMWARE);
+MODULE_FIRMWARE(SI2177_50_FIRMWARE);
diff --git a/drivers/media/tuners/si2157_priv.h b/drivers/media/tuners/si2157_priv.h
index ef47960..8579e80 100644
--- a/drivers/media/tuners/si2157_priv.h
+++ b/drivers/media/tuners/si2157_priv.h
@@ -26,7 +26,7 @@ struct si2157_dev {
unsigned int active:1;
unsigned int inversion:1;
unsigned int dont_load_firmware:1;
- u8 chiptype;
+ u8 part_id;
u8 if_port;
u32 if_frequency;
u32 bandwidth;
@@ -41,10 +41,22 @@ struct si2157_dev {
};
-#define SI2157_CHIPTYPE_SI2157 0
-#define SI2157_CHIPTYPE_SI2146 1
-#define SI2157_CHIPTYPE_SI2141 2
-#define SI2157_CHIPTYPE_SI2177 3
+enum si2157_part_id {
+ SI2141 = 41,
+ SI2146 = 46,
+ SI2147 = 47,
+ SI2148 = 48,
+ SI2157 = 57,
+ SI2158 = 58,
+ SI2177 = 77,
+};
+
+struct si2157_tuner_info {
+ enum si2157_part_id part_id;
+ unsigned char rom_id;
+ bool required;
+ const char *fw_name, *fw_alt_name;
+};
/* firmware command struct */
#define SI2157_ARGLEN 30
@@ -54,7 +66,29 @@ struct si2157_cmd {
unsigned rlen;
};
+#define SUPPORTS_1700KHz(dev) (((dev)->part_id == SI2141) || \
+ ((dev)->part_id == SI2147) || \
+ ((dev)->part_id == SI2157) || \
+ ((dev)->part_id == SI2177))
+
+#define SUPPORTS_ATV_IF(dev) (((dev)->part_id == SI2157) || \
+ ((dev)->part_id == SI2158))
+
+/* Old firmware namespace */
#define SI2158_A20_FIRMWARE "dvb-tuner-si2158-a20-01.fw"
#define SI2141_A10_FIRMWARE "dvb-tuner-si2141-a10-01.fw"
#define SI2157_A30_FIRMWARE "dvb-tuner-si2157-a30-01.fw"
+
+/* New firmware namespace */
+#define SI2141_60_FIRMWARE "dvb_driver_si2141_rom60.fw"
+#define SI2141_61_FIRMWARE "dvb_driver_si2141_rom61.fw"
+#define SI2146_11_FIRMWARE "dvb_driver_si2146_rom11.fw"
+#define SI2147_50_FIRMWARE "dvb_driver_si2147_rom50.fw"
+#define SI2148_32_FIRMWARE "dvb_driver_si2148_rom32.fw"
+#define SI2148_33_FIRMWARE "dvb_driver_si2148_rom33.fw"
+#define SI2157_50_FIRMWARE "dvb_driver_si2157_rom50.fw"
+#define SI2158_50_FIRMWARE "dvb_driver_si2178_rom50.fw"
+#define SI2158_51_FIRMWARE "dvb_driver_si2158_rom51.fw"
+#define SI2177_50_FIRMWARE "dvb_driver_si2177_rom50.fw"
+
#endif
diff --git a/drivers/media/tuners/tua9001.c b/drivers/media/tuners/tua9001.c
index 5e3625e..af7d5ea 100644
--- a/drivers/media/tuners/tua9001.c
+++ b/drivers/media/tuners/tua9001.c
@@ -240,14 +240,10 @@ static int tua9001_remove(struct i2c_client *client)
DVB_FRONTEND_COMPONENT_TUNER,
TUA9001_CMD_CEN, 0);
if (ret)
- goto err_kfree;
+ dev_err(&client->dev, "Tuner disable failed (%pe)\n", ERR_PTR(ret));
}
kfree(dev);
return 0;
-err_kfree:
- kfree(dev);
- dev_dbg(&client->dev, "failed=%d\n", ret);
- return ret;
}
static const struct i2c_device_id tua9001_id_table[] = {
diff --git a/drivers/media/usb/au0828/au0828-i2c.c b/drivers/media/usb/au0828/au0828-i2c.c
index 708f01a..749f90d 100644
--- a/drivers/media/usb/au0828/au0828-i2c.c
+++ b/drivers/media/usb/au0828/au0828-i2c.c
@@ -23,13 +23,6 @@ MODULE_PARM_DESC(i2c_scan, "scan i2c bus at insmod time");
#define I2C_WAIT_DELAY 25
#define I2C_WAIT_RETRY 1000
-static inline int i2c_slave_did_write_ack(struct i2c_adapter *i2c_adap)
-{
- struct au0828_dev *dev = i2c_adap->algo_data;
- return au0828_read(dev, AU0828_I2C_STATUS_201) &
- AU0828_I2C_STATUS_NO_WRITE_ACK ? 0 : 1;
-}
-
static inline int i2c_slave_did_read_ack(struct i2c_adapter *i2c_adap)
{
struct au0828_dev *dev = i2c_adap->algo_data;
diff --git a/drivers/media/usb/b2c2/flexcop-usb.c b/drivers/media/usb/b2c2/flexcop-usb.c
index 5d38171..7835bb0 100644
--- a/drivers/media/usb/b2c2/flexcop-usb.c
+++ b/drivers/media/usb/b2c2/flexcop-usb.c
@@ -87,7 +87,7 @@ static int flexcop_usb_readwrite_dw(struct flexcop_device *fc, u16 wRegOffsPCI,
0,
fc_usb->data,
sizeof(u32),
- B2C2_WAIT_FOR_OPERATION_RDW * HZ);
+ B2C2_WAIT_FOR_OPERATION_RDW);
if (ret != sizeof(u32)) {
err("error while %s dword from %d (%d).", read ? "reading" :
@@ -155,7 +155,7 @@ static int flexcop_usb_v8_memory_req(struct flexcop_usb *fc_usb,
wIndex,
fc_usb->data,
buflen,
- nWaitTime * HZ);
+ nWaitTime);
if (ret != buflen)
ret = -EIO;
@@ -171,7 +171,7 @@ static int flexcop_usb_v8_memory_req(struct flexcop_usb *fc_usb,
return ret;
}
-#define bytes_left_to_read_on_page(paddr,buflen) \
+#define bytes_left_to_read_on_page(paddr, buflen) \
((V8_MEMORY_PAGE_SIZE - (paddr & V8_MEMORY_PAGE_MASK)) > buflen \
? buflen : (V8_MEMORY_PAGE_SIZE - (paddr & V8_MEMORY_PAGE_MASK)))
@@ -179,11 +179,11 @@ static int flexcop_usb_memory_req(struct flexcop_usb *fc_usb,
flexcop_usb_request_t req, flexcop_usb_mem_page_t page_start,
u32 addr, int extended, u8 *buf, u32 len)
{
- int i,ret = 0;
+ int i, ret = 0;
u16 wMax;
u32 pagechunk = 0;
- switch(req) {
+ switch (req) {
case B2C2_USB_READ_V8_MEM:
wMax = USB_MEM_READ_MAX;
break;
@@ -248,13 +248,13 @@ static int flexcop_usb_i2c_req(struct flexcop_i2c_adapter *i2c,
/* DKT 020208 - add this to support special case of DiSEqC */
case USB_FUNC_I2C_CHECKWRITE:
pipe = B2C2_USB_CTRL_PIPE_OUT;
- nWaitTime = 2;
+ nWaitTime = 2000;
request_type |= USB_DIR_OUT;
break;
case USB_FUNC_I2C_READ:
case USB_FUNC_I2C_REPEATREAD:
pipe = B2C2_USB_CTRL_PIPE_IN;
- nWaitTime = 2;
+ nWaitTime = 2000;
request_type |= USB_DIR_IN;
break;
default:
@@ -281,7 +281,7 @@ static int flexcop_usb_i2c_req(struct flexcop_i2c_adapter *i2c,
wIndex,
fc_usb->data,
buflen,
- nWaitTime * HZ);
+ nWaitTime);
if (ret != buflen)
ret = -EIO;
@@ -341,8 +341,8 @@ static void flexcop_usb_process_frame(struct flexcop_usb *fc_usb,
b = fc_usb->tmp_buffer;
l = fc_usb->tmp_buffer_length;
} else {
- b=buffer;
- l=buffer_length;
+ b = buffer;
+ l = buffer_length;
}
while (l >= 190) {
@@ -368,7 +368,7 @@ static void flexcop_usb_process_frame(struct flexcop_usb *fc_usb,
}
}
- if (l>0)
+ if (l > 0)
memcpy(fc_usb->tmp_buffer, b, l);
fc_usb->tmp_buffer_length = l;
}
@@ -399,7 +399,7 @@ static void flexcop_usb_urb_complete(struct urb *urb)
urb->iso_frame_desc[i].status = 0;
urb->iso_frame_desc[i].actual_length = 0;
}
- usb_submit_urb(urb,GFP_ATOMIC);
+ usb_submit_urb(urb, GFP_ATOMIC);
}
static int flexcop_usb_stream_control(struct flexcop_device *fc, int onoff)
@@ -413,7 +413,7 @@ static void flexcop_usb_transfer_exit(struct flexcop_usb *fc_usb)
int i;
for (i = 0; i < B2C2_USB_NUM_ISO_URB; i++)
if (fc_usb->iso_urb[i] != NULL) {
- deb_ts("unlinking/killing urb no. %d\n",i);
+ deb_ts("unlinking/killing urb no. %d\n", i);
usb_kill_urb(fc_usb->iso_urb[i]);
usb_free_urb(fc_usb->iso_urb[i]);
}
@@ -483,7 +483,7 @@ static int flexcop_usb_transfer_init(struct flexcop_usb *fc_usb)
err("submitting urb %d failed with %d.", i, ret);
goto urb_error;
}
- deb_ts("submitted urb no. %d.\n",i);
+ deb_ts("submitted urb no. %d.\n", i);
}
/* SRAM */
diff --git a/drivers/media/usb/b2c2/flexcop-usb.h b/drivers/media/usb/b2c2/flexcop-usb.h
index 2f230bf..c7cca1a 100644
--- a/drivers/media/usb/b2c2/flexcop-usb.h
+++ b/drivers/media/usb/b2c2/flexcop-usb.h
@@ -91,13 +91,13 @@ typedef enum {
UTILITY_SRAM_TESTVERIFY = 0x16,
} flexcop_usb_utility_function_t;
-#define B2C2_WAIT_FOR_OPERATION_RW (1*HZ)
-#define B2C2_WAIT_FOR_OPERATION_RDW (3*HZ)
-#define B2C2_WAIT_FOR_OPERATION_WDW (1*HZ)
+#define B2C2_WAIT_FOR_OPERATION_RW 1000
+#define B2C2_WAIT_FOR_OPERATION_RDW 3000
+#define B2C2_WAIT_FOR_OPERATION_WDW 1000
-#define B2C2_WAIT_FOR_OPERATION_V8READ (3*HZ)
-#define B2C2_WAIT_FOR_OPERATION_V8WRITE (3*HZ)
-#define B2C2_WAIT_FOR_OPERATION_V8FLASH (3*HZ)
+#define B2C2_WAIT_FOR_OPERATION_V8READ 3000
+#define B2C2_WAIT_FOR_OPERATION_V8WRITE 3000
+#define B2C2_WAIT_FOR_OPERATION_V8FLASH 3000
typedef enum {
V8_MEMORY_PAGE_DVB_CI = 0x20,
diff --git a/drivers/media/usb/cpia2/cpia2_usb.c b/drivers/media/usb/cpia2/cpia2_usb.c
index 76aac06f..cba03b2 100644
--- a/drivers/media/usb/cpia2/cpia2_usb.c
+++ b/drivers/media/usb/cpia2/cpia2_usb.c
@@ -550,7 +550,7 @@ static int write_packet(struct usb_device *udev,
0, /* index */
buf, /* buffer */
size,
- HZ);
+ 1000);
kfree(buf);
return ret;
@@ -582,7 +582,7 @@ static int read_packet(struct usb_device *udev,
0, /* index */
buf, /* buffer */
size,
- HZ);
+ 1000);
if (ret >= 0)
memcpy(registers, buf, size);
diff --git a/drivers/media/usb/dvb-usb-v2/lmedm04.c b/drivers/media/usb/dvb-usb-v2/lmedm04.c
index fe4d886..8a34e6c 100644
--- a/drivers/media/usb/dvb-usb-v2/lmedm04.c
+++ b/drivers/media/usb/dvb-usb-v2/lmedm04.c
@@ -423,6 +423,9 @@ static int lme2510_pid_filter_ctrl(struct dvb_usb_adapter *adap, int onoff)
mutex_unlock(&d->i2c_mutex);
+ if (ret)
+ return -EREMOTEIO;
+
return 0;
}
diff --git a/drivers/media/usb/dvb-usb/dib0700_core.c b/drivers/media/usb/dvb-usb/dib0700_core.c
index 70219b3..1caabb5 100644
--- a/drivers/media/usb/dvb-usb/dib0700_core.c
+++ b/drivers/media/usb/dvb-usb/dib0700_core.c
@@ -583,7 +583,7 @@ int dib0700_download_firmware(struct usb_device *udev, const struct firmware *fw
int dib0700_streaming_ctrl(struct dvb_usb_adapter *adap, int onoff)
{
struct dib0700_state *st = adap->dev->priv;
- int ret;
+ int ret, adapt_nr;
if ((onoff != 0) && (st->fw_version >= 0x10201)) {
/* for firmware later than 1.20.1,
@@ -610,26 +610,24 @@ int dib0700_streaming_ctrl(struct dvb_usb_adapter *adap, int onoff)
st->buf[3] = 0x00;
- deb_info("modifying (%d) streaming state for %d\n", onoff, adap->id);
-
- st->channel_state &= ~0x3;
if ((adap->fe_adap[0].stream.props.endpoint != 2)
- && (adap->fe_adap[0].stream.props.endpoint != 3)) {
- deb_info("the endpoint number (%i) is not correct, use the adapter id instead", adap->fe_adap[0].stream.props.endpoint);
- if (onoff)
- st->channel_state |= 1 << (adap->id);
- else
- st->channel_state |= 1 << ~(adap->id);
+ && (adap->fe_adap[0].stream.props.endpoint != 3)) {
+ deb_info("the endpoint number (%i) is not correct, use the adapter id instead\n",
+ adap->fe_adap[0].stream.props.endpoint);
+ adapt_nr = adap->id;
} else {
- if (onoff)
- st->channel_state |= 1 << (adap->fe_adap[0].stream.props.endpoint-2);
- else
- st->channel_state |= 1 << (3-adap->fe_adap[0].stream.props.endpoint);
+ adapt_nr = adap->fe_adap[0].stream.props.endpoint - 2;
}
+ if (onoff)
+ st->channel_state |= 1 << adapt_nr;
+ else
+ st->channel_state &= ~(1 << adapt_nr);
+
st->buf[2] |= st->channel_state;
- deb_info("data for streaming: %x %x\n", st->buf[1], st->buf[2]);
+ deb_info("adapter %d, streaming %s: %*ph\n",
+ adapt_nr, onoff ? "ON" : "OFF", 3, st->buf);
ret = dib0700_ctrl_wr(adap->dev, st->buf, 4);
mutex_unlock(&adap->dev->usb_mutex);
diff --git a/drivers/media/usb/dvb-usb/dw2102.c b/drivers/media/usb/dvb-usb/dw2102.c
index f0e686b..ca75ebd 100644
--- a/drivers/media/usb/dvb-usb/dw2102.c
+++ b/drivers/media/usb/dvb-usb/dw2102.c
@@ -2150,46 +2150,153 @@ static struct dvb_usb_device_properties s6x0_properties = {
}
};
-static const struct dvb_usb_device_description d1100 = {
- "Prof 1100 USB ",
- {&dw2102_table[PROF_1100], NULL},
- {NULL},
+static struct dvb_usb_device_properties p1100_properties = {
+ .caps = DVB_USB_IS_AN_I2C_ADAPTER,
+ .usb_ctrl = DEVICE_SPECIFIC,
+ .size_of_priv = sizeof(struct dw2102_state),
+ .firmware = P1100_FIRMWARE,
+ .no_reconnect = 1,
+
+ .i2c_algo = &s6x0_i2c_algo,
+ .rc.core = {
+ .rc_interval = 150,
+ .rc_codes = RC_MAP_TBS_NEC,
+ .module_name = "dw2102",
+ .allowed_protos = RC_PROTO_BIT_NEC,
+ .rc_query = prof_rc_query,
+ },
+
+ .generic_bulk_ctrl_endpoint = 0x81,
+ .num_adapters = 1,
+ .download_firmware = dw2102_load_firmware,
+ .read_mac_address = s6x0_read_mac_address,
+ .adapter = {
+ {
+ .num_frontends = 1,
+ .fe = {{
+ .frontend_attach = stv0288_frontend_attach,
+ .stream = {
+ .type = USB_BULK,
+ .count = 8,
+ .endpoint = 0x82,
+ .u = {
+ .bulk = {
+ .buffersize = 4096,
+ }
+ }
+ },
+ } },
+ }
+ },
+ .num_device_descs = 1,
+ .devices = {
+ {"Prof 1100 USB ",
+ {&dw2102_table[PROF_1100], NULL},
+ {NULL},
+ },
+ }
};
-static const struct dvb_usb_device_description d660 = {
- "TeVii S660 USB",
- {&dw2102_table[TEVII_S660], NULL},
- {NULL},
+static struct dvb_usb_device_properties s660_properties = {
+ .caps = DVB_USB_IS_AN_I2C_ADAPTER,
+ .usb_ctrl = DEVICE_SPECIFIC,
+ .size_of_priv = sizeof(struct dw2102_state),
+ .firmware = S660_FIRMWARE,
+ .no_reconnect = 1,
+
+ .i2c_algo = &s6x0_i2c_algo,
+ .rc.core = {
+ .rc_interval = 150,
+ .rc_codes = RC_MAP_TEVII_NEC,
+ .module_name = "dw2102",
+ .allowed_protos = RC_PROTO_BIT_NEC,
+ .rc_query = dw2102_rc_query,
+ },
+
+ .generic_bulk_ctrl_endpoint = 0x81,
+ .num_adapters = 1,
+ .download_firmware = dw2102_load_firmware,
+ .read_mac_address = s6x0_read_mac_address,
+ .adapter = {
+ {
+ .num_frontends = 1,
+ .fe = {{
+ .frontend_attach = ds3000_frontend_attach,
+ .stream = {
+ .type = USB_BULK,
+ .count = 8,
+ .endpoint = 0x82,
+ .u = {
+ .bulk = {
+ .buffersize = 4096,
+ }
+ }
+ },
+ } },
+ }
+ },
+ .num_device_descs = 3,
+ .devices = {
+ {"TeVii S660 USB",
+ {&dw2102_table[TEVII_S660], NULL},
+ {NULL},
+ },
+ {"TeVii S480.1 USB",
+ {&dw2102_table[TEVII_S480_1], NULL},
+ {NULL},
+ },
+ {"TeVii S480.2 USB",
+ {&dw2102_table[TEVII_S480_2], NULL},
+ {NULL},
+ },
+ }
};
-static const struct dvb_usb_device_description d480_1 = {
- "TeVii S480.1 USB",
- {&dw2102_table[TEVII_S480_1], NULL},
- {NULL},
-};
+static struct dvb_usb_device_properties p7500_properties = {
+ .caps = DVB_USB_IS_AN_I2C_ADAPTER,
+ .usb_ctrl = DEVICE_SPECIFIC,
+ .size_of_priv = sizeof(struct dw2102_state),
+ .firmware = P7500_FIRMWARE,
+ .no_reconnect = 1,
-static const struct dvb_usb_device_description d480_2 = {
- "TeVii S480.2 USB",
- {&dw2102_table[TEVII_S480_2], NULL},
- {NULL},
-};
+ .i2c_algo = &s6x0_i2c_algo,
+ .rc.core = {
+ .rc_interval = 150,
+ .rc_codes = RC_MAP_TBS_NEC,
+ .module_name = "dw2102",
+ .allowed_protos = RC_PROTO_BIT_NEC,
+ .rc_query = prof_rc_query,
+ },
-static const struct dvb_usb_device_description d7500 = {
- "Prof 7500 USB DVB-S2",
- {&dw2102_table[PROF_7500], NULL},
- {NULL},
-};
-
-static const struct dvb_usb_device_description d421 = {
- "TeVii S421 PCI",
- {&dw2102_table[TEVII_S421], NULL},
- {NULL},
-};
-
-static const struct dvb_usb_device_description d632 = {
- "TeVii S632 USB",
- {&dw2102_table[TEVII_S632], NULL},
- {NULL},
+ .generic_bulk_ctrl_endpoint = 0x81,
+ .num_adapters = 1,
+ .download_firmware = dw2102_load_firmware,
+ .read_mac_address = s6x0_read_mac_address,
+ .adapter = {
+ {
+ .num_frontends = 1,
+ .fe = {{
+ .frontend_attach = prof_7500_frontend_attach,
+ .stream = {
+ .type = USB_BULK,
+ .count = 8,
+ .endpoint = 0x82,
+ .u = {
+ .bulk = {
+ .buffersize = 4096,
+ }
+ }
+ },
+ } },
+ }
+ },
+ .num_device_descs = 1,
+ .devices = {
+ {"Prof 7500 USB DVB-S2",
+ {&dw2102_table[PROF_7500], NULL},
+ {NULL},
+ },
+ }
};
static struct dvb_usb_device_properties su3000_properties = {
@@ -2273,6 +2380,59 @@ static struct dvb_usb_device_properties su3000_properties = {
}
};
+static struct dvb_usb_device_properties s421_properties = {
+ .caps = DVB_USB_IS_AN_I2C_ADAPTER,
+ .usb_ctrl = DEVICE_SPECIFIC,
+ .size_of_priv = sizeof(struct dw2102_state),
+ .power_ctrl = su3000_power_ctrl,
+ .num_adapters = 1,
+ .identify_state = su3000_identify_state,
+ .i2c_algo = &su3000_i2c_algo,
+
+ .rc.core = {
+ .rc_interval = 150,
+ .rc_codes = RC_MAP_SU3000,
+ .module_name = "dw2102",
+ .allowed_protos = RC_PROTO_BIT_RC5,
+ .rc_query = su3000_rc_query,
+ },
+
+ .read_mac_address = su3000_read_mac_address,
+
+ .generic_bulk_ctrl_endpoint = 0x01,
+
+ .adapter = {
+ {
+ .num_frontends = 1,
+ .fe = {{
+ .streaming_ctrl = su3000_streaming_ctrl,
+ .frontend_attach = m88rs2000_frontend_attach,
+ .stream = {
+ .type = USB_BULK,
+ .count = 8,
+ .endpoint = 0x82,
+ .u = {
+ .bulk = {
+ .buffersize = 4096,
+ }
+ }
+ }
+ } },
+ }
+ },
+ .num_device_descs = 2,
+ .devices = {
+ { "TeVii S421 PCI",
+ { &dw2102_table[TEVII_S421], NULL },
+ { NULL },
+ },
+ { "TeVii S632 USB",
+ { &dw2102_table[TEVII_S632], NULL },
+ { NULL },
+ },
+ }
+};
+
static struct dvb_usb_device_properties t220_properties = {
.caps = DVB_USB_IS_AN_I2C_ADAPTER,
.usb_ctrl = DEVICE_SPECIFIC,
@@ -2390,101 +2550,33 @@ static struct dvb_usb_device_properties tt_s2_4600_properties = {
static int dw2102_probe(struct usb_interface *intf,
const struct usb_device_id *id)
{
- int retval = -ENOMEM;
- struct dvb_usb_device_properties *p1100;
- struct dvb_usb_device_properties *s660;
- struct dvb_usb_device_properties *p7500;
- struct dvb_usb_device_properties *s421;
-
- p1100 = kmemdup(&s6x0_properties,
- sizeof(struct dvb_usb_device_properties), GFP_KERNEL);
- if (!p1100)
- goto err0;
-
- /* copy default structure */
- /* fill only different fields */
- p1100->firmware = P1100_FIRMWARE;
- p1100->devices[0] = d1100;
- p1100->rc.core.rc_query = prof_rc_query;
- p1100->rc.core.rc_codes = RC_MAP_TBS_NEC;
- p1100->adapter->fe[0].frontend_attach = stv0288_frontend_attach;
-
- s660 = kmemdup(&s6x0_properties,
- sizeof(struct dvb_usb_device_properties), GFP_KERNEL);
- if (!s660)
- goto err1;
-
- s660->firmware = S660_FIRMWARE;
- s660->num_device_descs = 3;
- s660->devices[0] = d660;
- s660->devices[1] = d480_1;
- s660->devices[2] = d480_2;
- s660->adapter->fe[0].frontend_attach = ds3000_frontend_attach;
-
- p7500 = kmemdup(&s6x0_properties,
- sizeof(struct dvb_usb_device_properties), GFP_KERNEL);
- if (!p7500)
- goto err2;
-
- p7500->firmware = P7500_FIRMWARE;
- p7500->devices[0] = d7500;
- p7500->rc.core.rc_query = prof_rc_query;
- p7500->rc.core.rc_codes = RC_MAP_TBS_NEC;
- p7500->adapter->fe[0].frontend_attach = prof_7500_frontend_attach;
-
-
- s421 = kmemdup(&su3000_properties,
- sizeof(struct dvb_usb_device_properties), GFP_KERNEL);
- if (!s421)
- goto err3;
-
- s421->num_device_descs = 2;
- s421->devices[0] = d421;
- s421->devices[1] = d632;
- s421->adapter->fe[0].frontend_attach = m88rs2000_frontend_attach;
-
- if (0 == dvb_usb_device_init(intf, &dw2102_properties,
- THIS_MODULE, NULL, adapter_nr) ||
- 0 == dvb_usb_device_init(intf, &dw2104_properties,
- THIS_MODULE, NULL, adapter_nr) ||
- 0 == dvb_usb_device_init(intf, &dw3101_properties,
- THIS_MODULE, NULL, adapter_nr) ||
- 0 == dvb_usb_device_init(intf, &s6x0_properties,
- THIS_MODULE, NULL, adapter_nr) ||
- 0 == dvb_usb_device_init(intf, p1100,
- THIS_MODULE, NULL, adapter_nr) ||
- 0 == dvb_usb_device_init(intf, s660,
- THIS_MODULE, NULL, adapter_nr) ||
- 0 == dvb_usb_device_init(intf, p7500,
- THIS_MODULE, NULL, adapter_nr) ||
- 0 == dvb_usb_device_init(intf, s421,
- THIS_MODULE, NULL, adapter_nr) ||
- 0 == dvb_usb_device_init(intf, &su3000_properties,
- THIS_MODULE, NULL, adapter_nr) ||
- 0 == dvb_usb_device_init(intf, &t220_properties,
- THIS_MODULE, NULL, adapter_nr) ||
- 0 == dvb_usb_device_init(intf, &tt_s2_4600_properties,
- THIS_MODULE, NULL, adapter_nr)) {
-
- /* clean up copied properties */
- kfree(s421);
- kfree(p7500);
- kfree(s660);
- kfree(p1100);
+ if (!(dvb_usb_device_init(intf, &dw2102_properties,
+ THIS_MODULE, NULL, adapter_nr) &&
+ dvb_usb_device_init(intf, &dw2104_properties,
+ THIS_MODULE, NULL, adapter_nr) &&
+ dvb_usb_device_init(intf, &dw3101_properties,
+ THIS_MODULE, NULL, adapter_nr) &&
+ dvb_usb_device_init(intf, &s6x0_properties,
+ THIS_MODULE, NULL, adapter_nr) &&
+ dvb_usb_device_init(intf, &p1100_properties,
+ THIS_MODULE, NULL, adapter_nr) &&
+ dvb_usb_device_init(intf, &s660_properties,
+ THIS_MODULE, NULL, adapter_nr) &&
+ dvb_usb_device_init(intf, &p7500_properties,
+ THIS_MODULE, NULL, adapter_nr) &&
+ dvb_usb_device_init(intf, &s421_properties,
+ THIS_MODULE, NULL, adapter_nr) &&
+ dvb_usb_device_init(intf, &su3000_properties,
+ THIS_MODULE, NULL, adapter_nr) &&
+ dvb_usb_device_init(intf, &t220_properties,
+ THIS_MODULE, NULL, adapter_nr) &&
+ dvb_usb_device_init(intf, &tt_s2_4600_properties,
+ THIS_MODULE, NULL, adapter_nr))) {
return 0;
}
- retval = -ENODEV;
- kfree(s421);
-err3:
- kfree(p7500);
-err2:
- kfree(s660);
-err1:
- kfree(p1100);
-err0:
- return retval;
+ return -ENODEV;
}
static void dw2102_disconnect(struct usb_interface *intf)
diff --git a/drivers/media/usb/dvb-usb/m920x.c b/drivers/media/usb/dvb-usb/m920x.c
index 4bb5b82..691e058 100644
--- a/drivers/media/usb/dvb-usb/m920x.c
+++ b/drivers/media/usb/dvb-usb/m920x.c
@@ -274,6 +274,13 @@ static int m920x_i2c_xfer(struct i2c_adapter *adap, struct i2c_msg msg[], int nu
/* Should check for ack here, if we knew how. */
}
if (msg[i].flags & I2C_M_RD) {
+ char *read = kmalloc(1, GFP_KERNEL);
+ if (!read) {
+ ret = -ENOMEM;
+ kfree(read);
+ goto unlock;
+ }
+
for (j = 0; j < msg[i].len; j++) {
/* Last byte of transaction?
* Send STOP, otherwise send ACK. */
@@ -281,9 +288,12 @@ static int m920x_i2c_xfer(struct i2c_adapter *adap, struct i2c_msg msg[], int nu
if ((ret = m920x_read(d->udev, M9206_I2C, 0x0,
0x20 | stop,
- &msg[i].buf[j], 1)) != 0)
+ read, 1)) != 0)
goto unlock;
+ msg[i].buf[j] = read[0];
}
+
+ kfree(read);
} else {
for (j = 0; j < msg[i].len; j++) {
/* Last byte of transaction? Then send STOP. */
diff --git a/drivers/media/usb/em28xx/em28xx-cards.c b/drivers/media/usb/em28xx/em28xx-cards.c
index b207f34..b451ce3 100644
--- a/drivers/media/usb/em28xx/em28xx-cards.c
+++ b/drivers/media/usb/em28xx/em28xx-cards.c
@@ -3630,8 +3630,10 @@ static int em28xx_init_dev(struct em28xx *dev, struct usb_device *udev,
if (dev->is_audio_only) {
retval = em28xx_audio_setup(dev);
- if (retval)
- return -ENODEV;
+ if (retval) {
+ retval = -ENODEV;
+ goto err_deinit_media;
+ }
em28xx_init_extension(dev);
return 0;
@@ -3650,7 +3652,7 @@ static int em28xx_init_dev(struct em28xx *dev, struct usb_device *udev,
dev_err(&dev->intf->dev,
"%s: em28xx_i2c_register bus 0 - error [%d]!\n",
__func__, retval);
- return retval;
+ goto err_deinit_media;
}
/* register i2c bus 1 */
@@ -3666,9 +3668,7 @@ static int em28xx_init_dev(struct em28xx *dev, struct usb_device *udev,
"%s: em28xx_i2c_register bus 1 - error [%d]!\n",
__func__, retval);
- em28xx_i2c_unregister(dev, 0);
-
- return retval;
+ goto err_unreg_i2c;
}
}
@@ -3676,6 +3676,12 @@ static int em28xx_init_dev(struct em28xx *dev, struct usb_device *udev,
em28xx_card_setup(dev);
return 0;
+
+err_unreg_i2c:
+ em28xx_i2c_unregister(dev, 0);
+err_deinit_media:
+ em28xx_unregister_media_device(dev);
+ return retval;
}
static int em28xx_duplicate_dev(struct em28xx *dev)
diff --git a/drivers/media/usb/em28xx/em28xx-core.c b/drivers/media/usb/em28xx/em28xx-core.c
index acc0bf7..c837cc5 100644
--- a/drivers/media/usb/em28xx/em28xx-core.c
+++ b/drivers/media/usb/em28xx/em28xx-core.c
@@ -89,7 +89,7 @@ int em28xx_read_reg_req_len(struct em28xx *dev, u8 req, u16 reg,
mutex_lock(&dev->ctrl_urb_lock);
ret = usb_control_msg(udev, pipe, req,
USB_DIR_IN | USB_TYPE_VENDOR | USB_RECIP_DEVICE,
- 0x0000, reg, dev->urb_buf, len, HZ);
+ 0x0000, reg, dev->urb_buf, len, 1000);
if (ret < 0) {
em28xx_regdbg("(pipe 0x%08x): IN: %02x %02x %02x %02x %02x %02x %02x %02x failed with error %i\n",
pipe,
@@ -158,7 +158,7 @@ int em28xx_write_regs_req(struct em28xx *dev, u8 req, u16 reg, char *buf,
memcpy(dev->urb_buf, buf, len);
ret = usb_control_msg(udev, pipe, req,
USB_DIR_OUT | USB_TYPE_VENDOR | USB_RECIP_DEVICE,
- 0x0000, reg, dev->urb_buf, len, HZ);
+ 0x0000, reg, dev->urb_buf, len, 1000);
mutex_unlock(&dev->ctrl_urb_lock);
if (ret < 0) {
diff --git a/drivers/media/usb/go7007/go7007-driver.c b/drivers/media/usb/go7007/go7007-driver.c
index 6650eab..0c24e29 100644
--- a/drivers/media/usb/go7007/go7007-driver.c
+++ b/drivers/media/usb/go7007/go7007-driver.c
@@ -516,7 +516,7 @@ void go7007_parse_video_stream(struct go7007 *go, u8 *buf, int length)
if (vb && vb->vb.vb2_buf.planes[0].bytesused >=
GO7007_BUF_SIZE - 3) {
v4l2_info(&go->v4l2_dev, "dropping oversized frame\n");
- vb->vb.vb2_buf.planes[0].bytesused = 0;
+ vb2_set_plane_payload(&vb->vb.vb2_buf, 0, 0);
vb->frame_offset = 0;
vb->modet_active = 0;
vb = go->active_buf = NULL;
diff --git a/drivers/media/usb/gspca/m5602/m5602_s5k83a.c b/drivers/media/usb/gspca/m5602/m5602_s5k83a.c
index bc4008d..8ef010a 100644
--- a/drivers/media/usb/gspca/m5602/m5602_s5k83a.c
+++ b/drivers/media/usb/gspca/m5602/m5602_s5k83a.c
@@ -339,14 +339,13 @@ int s5k83a_start(struct sd *sd)
/* Create another thread, polling the GPIO ports of the camera to check
if it got rotated. This is how the windows driver does it so we have
to assume that there is no better way of accomplishing this */
- sd->rotation_thread = kthread_create(rotation_thread_function,
- sd, "rotation thread");
+ sd->rotation_thread = kthread_run(rotation_thread_function,
+ sd, "rotation thread");
if (IS_ERR(sd->rotation_thread)) {
err = PTR_ERR(sd->rotation_thread);
sd->rotation_thread = NULL;
return err;
}
- wake_up_process(sd->rotation_thread);
/* Preinit the sensor */
for (i = 0; i < ARRAY_SIZE(start_s5k83a) && !err; i++) {
@@ -408,25 +407,21 @@ static int s5k83a_set_gain(struct gspca_dev *gspca_dev, __s32 val)
static int s5k83a_set_brightness(struct gspca_dev *gspca_dev, __s32 val)
{
- int err;
u8 data[1];
struct sd *sd = (struct sd *) gspca_dev;
data[0] = val;
- err = m5602_write_sensor(sd, S5K83A_BRIGHTNESS, data, 1);
- return err;
+ return m5602_write_sensor(sd, S5K83A_BRIGHTNESS, data, 1);
}
static int s5k83a_set_exposure(struct gspca_dev *gspca_dev, __s32 val)
{
- int err;
u8 data[2];
struct sd *sd = (struct sd *) gspca_dev;
data[0] = 0;
data[1] = val;
- err = m5602_write_sensor(sd, S5K83A_EXPOSURE, data, 2);
- return err;
+ return m5602_write_sensor(sd, S5K83A_EXPOSURE, data, 2);
}
static int s5k83a_set_flip_real(struct gspca_dev *gspca_dev,
diff --git a/drivers/media/usb/pvrusb2/pvrusb2-encoder.c b/drivers/media/usb/pvrusb2/pvrusb2-encoder.c
index f6005d1..c810277 100644
--- a/drivers/media/usb/pvrusb2/pvrusb2-encoder.c
+++ b/drivers/media/usb/pvrusb2/pvrusb2-encoder.c
@@ -188,8 +188,8 @@ static int pvr2_encoder_cmd(void *ctxt,
}
- LOCK_TAKE(hdw->ctl_lock); while (1) {
-
+ LOCK_TAKE(hdw->ctl_lock);
+ while (1) {
if (!hdw->state_encoder_ok) {
ret = -EIO;
break;
diff --git a/drivers/media/usb/pvrusb2/pvrusb2-hdw.c b/drivers/media/usb/pvrusb2/pvrusb2-hdw.c
index d38dee1..cd7b118 100644
--- a/drivers/media/usb/pvrusb2/pvrusb2-hdw.c
+++ b/drivers/media/usb/pvrusb2/pvrusb2-hdw.c
@@ -1467,7 +1467,7 @@ static int pvr2_upload_firmware1(struct pvr2_hdw *hdw)
for (address = 0; address < fwsize; address += 0x800) {
memcpy(fw_ptr, fw_entry->data + address, 0x800);
ret += usb_control_msg(hdw->usb_dev, pipe, 0xa0, 0x40, address,
- 0, fw_ptr, 0x800, HZ);
+ 0, fw_ptr, 0x800, 1000);
}
trace_firmware("Upload done, releasing device's CPU");
@@ -1605,7 +1605,7 @@ int pvr2_upload_firmware2(struct pvr2_hdw *hdw)
((u32 *)fw_ptr)[icnt] = swab32(((u32 *)fw_ptr)[icnt]);
ret |= usb_bulk_msg(hdw->usb_dev, pipe, fw_ptr,bcnt,
- &actual_length, HZ);
+ &actual_length, 1000);
ret |= (actual_length != bcnt);
if (ret) break;
fw_done += bcnt;
@@ -1718,16 +1718,16 @@ int pvr2_hdw_get_streaming(struct pvr2_hdw *hdw)
int pvr2_hdw_set_streaming(struct pvr2_hdw *hdw,int enable_flag)
{
int ret,st;
- LOCK_TAKE(hdw->big_lock); do {
- pvr2_hdw_untrip_unlocked(hdw);
- if ((!enable_flag) != !(hdw->state_pipeline_req)) {
- hdw->state_pipeline_req = enable_flag != 0;
- pvr2_trace(PVR2_TRACE_START_STOP,
- "/*--TRACE_STREAM--*/ %s",
- enable_flag ? "enable" : "disable");
- }
- pvr2_hdw_state_sched(hdw);
- } while (0); LOCK_GIVE(hdw->big_lock);
+ LOCK_TAKE(hdw->big_lock);
+ pvr2_hdw_untrip_unlocked(hdw);
+ if (!enable_flag != !hdw->state_pipeline_req) {
+ hdw->state_pipeline_req = enable_flag != 0;
+ pvr2_trace(PVR2_TRACE_START_STOP,
+ "/*--TRACE_STREAM--*/ %s",
+ enable_flag ? "enable" : "disable");
+ }
+ pvr2_hdw_state_sched(hdw);
+ LOCK_GIVE(hdw->big_lock);
if ((ret = pvr2_hdw_wait(hdw,0)) < 0) return ret;
if (enable_flag) {
while ((st = hdw->master_state) != PVR2_STATE_RUN) {
@@ -3394,7 +3394,8 @@ void pvr2_hdw_cpufw_set_enabled(struct pvr2_hdw *hdw,
int ret;
u16 address;
unsigned int pipe;
- LOCK_TAKE(hdw->big_lock); do {
+ LOCK_TAKE(hdw->big_lock);
+ do {
if ((hdw->fw_buffer == NULL) == !enable_flag) break;
if (!enable_flag) {
@@ -3438,7 +3439,7 @@ void pvr2_hdw_cpufw_set_enabled(struct pvr2_hdw *hdw,
0xa0,0xc0,
address,0,
hdw->fw_buffer+address,
- 0x800,HZ);
+ 0x800,1000);
if (ret < 0) break;
}
@@ -3457,8 +3458,8 @@ void pvr2_hdw_cpufw_set_enabled(struct pvr2_hdw *hdw,
pvr2_trace(PVR2_TRACE_FIRMWARE,
"Done sucking down EEPROM contents");
}
-
- } while (0); LOCK_GIVE(hdw->big_lock);
+ } while (0);
+ LOCK_GIVE(hdw->big_lock);
}
@@ -3473,7 +3474,8 @@ int pvr2_hdw_cpufw_get(struct pvr2_hdw *hdw,unsigned int offs,
char *buf,unsigned int cnt)
{
int ret = -EINVAL;
- LOCK_TAKE(hdw->big_lock); do {
+ LOCK_TAKE(hdw->big_lock);
+ do {
if (!buf) break;
if (!cnt) break;
@@ -3498,7 +3500,8 @@ int pvr2_hdw_cpufw_get(struct pvr2_hdw *hdw,unsigned int offs,
"Read firmware data offs=%d cnt=%d",
offs,cnt);
ret = cnt;
- } while (0); LOCK_GIVE(hdw->big_lock);
+ } while (0);
+ LOCK_GIVE(hdw->big_lock);
return ret;
}
@@ -3977,7 +3980,7 @@ void pvr2_hdw_cpureset_assert(struct pvr2_hdw *hdw,int val)
/* Write the CPUCS register on the 8051. The lsb of the register
is the reset bit; a 1 asserts reset while a 0 clears it. */
pipe = usb_sndctrlpipe(hdw->usb_dev, 0);
- ret = usb_control_msg(hdw->usb_dev,pipe,0xa0,0x40,0xe600,0,da,1,HZ);
+ ret = usb_control_msg(hdw->usb_dev,pipe,0xa0,0x40,0xe600,0,da,1,1000);
if (ret < 0) {
pvr2_trace(PVR2_TRACE_ERROR_LEGS,
"cpureset_assert(%d) error=%d",val,ret);
diff --git a/drivers/media/usb/s2255/s2255drv.c b/drivers/media/usb/s2255/s2255drv.c
index 3b0e4ed..acf18e2 100644
--- a/drivers/media/usb/s2255/s2255drv.c
+++ b/drivers/media/usb/s2255/s2255drv.c
@@ -1882,7 +1882,7 @@ static long s2255_vendor_req(struct s2255_dev *dev, unsigned char Request,
USB_TYPE_VENDOR | USB_RECIP_DEVICE |
USB_DIR_IN,
Value, Index, buf,
- TransferBufferLength, HZ * 5);
+ TransferBufferLength, USB_CTRL_SET_TIMEOUT);
if (r >= 0)
memcpy(TransferBuffer, buf, TransferBufferLength);
@@ -1891,7 +1891,7 @@ static long s2255_vendor_req(struct s2255_dev *dev, unsigned char Request,
r = usb_control_msg(dev->udev, usb_sndctrlpipe(dev->udev, 0),
Request, USB_TYPE_VENDOR | USB_RECIP_DEVICE,
Value, Index, buf,
- TransferBufferLength, HZ * 5);
+ TransferBufferLength, USB_CTRL_SET_TIMEOUT);
}
kfree(buf);
return r;
diff --git a/drivers/media/usb/siano/smsusb.c b/drivers/media/usb/siano/smsusb.c
index df4c5dcb..fe9c7b3 100644
--- a/drivers/media/usb/siano/smsusb.c
+++ b/drivers/media/usb/siano/smsusb.c
@@ -661,10 +661,6 @@ static const struct usb_device_id smsusb_id_table[] = {
.driver_info = SMS1XXX_BOARD_HAUPPAUGE_WINDHAM },
{ USB_DEVICE(0x2040, 0x5590),
.driver_info = SMS1XXX_BOARD_HAUPPAUGE_WINDHAM },
- { USB_DEVICE(0x187f, 0x0202),
- .driver_info = SMS1XXX_BOARD_SIANO_NICE },
- { USB_DEVICE(0x187f, 0x0301),
- .driver_info = SMS1XXX_BOARD_SIANO_VENICE },
{ USB_DEVICE(0x2040, 0xb900),
.driver_info = SMS1XXX_BOARD_HAUPPAUGE_WINDHAM },
{ USB_DEVICE(0x2040, 0xb910),
diff --git a/drivers/media/usb/stk1160/stk1160-core.c b/drivers/media/usb/stk1160/stk1160-core.c
index b4f8bc5..4e1698f 100644
--- a/drivers/media/usb/stk1160/stk1160-core.c
+++ b/drivers/media/usb/stk1160/stk1160-core.c
@@ -65,7 +65,7 @@ int stk1160_read_reg(struct stk1160 *dev, u16 reg, u8 *value)
return -ENOMEM;
ret = usb_control_msg(dev->udev, pipe, 0x00,
USB_DIR_IN | USB_TYPE_VENDOR | USB_RECIP_DEVICE,
- 0x00, reg, buf, sizeof(u8), HZ);
+ 0x00, reg, buf, sizeof(u8), 1000);
if (ret < 0) {
stk1160_err("read failed on reg 0x%x (%d)\n",
reg, ret);
@@ -85,7 +85,7 @@ int stk1160_write_reg(struct stk1160 *dev, u16 reg, u16 value)
ret = usb_control_msg(dev->udev, pipe, 0x01,
USB_DIR_OUT | USB_TYPE_VENDOR | USB_RECIP_DEVICE,
- value, reg, NULL, 0, HZ);
+ value, reg, NULL, 0, 1000);
if (ret < 0) {
stk1160_err("write failed on reg 0x%x (%d)\n",
reg, ret);
diff --git a/drivers/media/usb/uvc/uvc_ctrl.c b/drivers/media/usb/uvc/uvc_ctrl.c
index 30bfe90..b4f6edf 100644
--- a/drivers/media/usb/uvc/uvc_ctrl.c
+++ b/drivers/media/usb/uvc/uvc_ctrl.c
@@ -1638,8 +1638,8 @@ static int uvc_ctrl_find_ctrl_idx(struct uvc_entity *entity,
struct v4l2_ext_controls *ctrls,
struct uvc_control *uvc_control)
{
- struct uvc_control_mapping *mapping;
- struct uvc_control *ctrl_found;
+ struct uvc_control_mapping *mapping = NULL;
+ struct uvc_control *ctrl_found = NULL;
unsigned int i;
if (!entity)
diff --git a/drivers/media/usb/uvc/uvc_driver.c b/drivers/media/usb/uvc/uvc_driver.c
index 7c00742..5f394d4 100644
--- a/drivers/media/usb/uvc/uvc_driver.c
+++ b/drivers/media/usb/uvc/uvc_driver.c
@@ -256,7 +256,7 @@ static struct uvc_format_desc *uvc_format_by_guid(const u8 guid[16])
static enum v4l2_colorspace uvc_colorspace(const u8 primaries)
{
static const enum v4l2_colorspace colorprimaries[] = {
- V4L2_COLORSPACE_DEFAULT, /* Unspecified */
+ V4L2_COLORSPACE_SRGB, /* Unspecified */
V4L2_COLORSPACE_SRGB,
V4L2_COLORSPACE_470_SYSTEM_M,
V4L2_COLORSPACE_470_SYSTEM_BG,
@@ -267,7 +267,7 @@ static enum v4l2_colorspace uvc_colorspace(const u8 primaries)
if (primaries < ARRAY_SIZE(colorprimaries))
return colorprimaries[primaries];
- return V4L2_COLORSPACE_DEFAULT; /* Reserved */
+ return V4L2_COLORSPACE_SRGB; /* Reserved */
}
static enum v4l2_xfer_func uvc_xfer_func(const u8 transfer_characteristics)
@@ -769,6 +769,8 @@ static int uvc_parse_format(struct uvc_device *dev,
buflen -= buffer[0];
buffer += buffer[0];
+ } else {
+ format->colorspace = V4L2_COLORSPACE_SRGB;
}
return buffer - start;
@@ -2193,7 +2195,6 @@ int uvc_register_video_device(struct uvc_device *dev,
const struct v4l2_file_operations *fops,
const struct v4l2_ioctl_ops *ioctl_ops)
{
- const char *name;
int ret;
/* Initialize the video buffers queue. */
@@ -2222,20 +2223,16 @@ int uvc_register_video_device(struct uvc_device *dev,
case V4L2_BUF_TYPE_VIDEO_CAPTURE:
default:
vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING;
- name = "Video Capture";
break;
case V4L2_BUF_TYPE_VIDEO_OUTPUT:
vdev->device_caps = V4L2_CAP_VIDEO_OUTPUT | V4L2_CAP_STREAMING;
- name = "Video Output";
break;
case V4L2_BUF_TYPE_META_CAPTURE:
vdev->device_caps = V4L2_CAP_META_CAPTURE | V4L2_CAP_STREAMING;
- name = "Metadata";
break;
}
- snprintf(vdev->name, sizeof(vdev->name), "%s %u", name,
- stream->header.bTerminalLink);
+ strscpy(vdev->name, dev->name, sizeof(vdev->name));
/*
* Set the driver data before calling video_register_device, otherwise
diff --git a/drivers/media/usb/uvc/uvc_v4l2.c b/drivers/media/usb/uvc/uvc_v4l2.c
index f4e4aff..711556d 100644
--- a/drivers/media/usb/uvc/uvc_v4l2.c
+++ b/drivers/media/usb/uvc/uvc_v4l2.c
@@ -44,8 +44,10 @@ static int uvc_ioctl_ctrl_map(struct uvc_video_chain *chain,
if (v4l2_ctrl_get_name(map->id) == NULL) {
map->name = kmemdup(xmap->name, sizeof(xmap->name),
GFP_KERNEL);
- if (!map->name)
- return -ENOMEM;
+ if (!map->name) {
+ ret = -ENOMEM;
+ goto free_map;
+ }
}
memcpy(map->entity, xmap->entity, sizeof(map->entity));
map->selector = xmap->selector;
diff --git a/drivers/media/usb/uvc/uvc_video.c b/drivers/media/usb/uvc/uvc_video.c
index 9f37eaf..1b4cc93 100644
--- a/drivers/media/usb/uvc/uvc_video.c
+++ b/drivers/media/usb/uvc/uvc_video.c
@@ -1963,6 +1963,10 @@ static int uvc_video_start_transfer(struct uvc_streaming *stream,
if (ep == NULL)
return -EIO;
+ /* Reject broken descriptors. */
+ if (usb_endpoint_maxp(&ep->desc) == 0)
+ return -EIO;
+
ret = uvc_init_video_bulk(stream, ep, gfp_flags);
}
diff --git a/drivers/media/usb/uvc/uvcvideo.h b/drivers/media/usb/uvc/uvcvideo.h
index 2e53661..143230b 100644
--- a/drivers/media/usb/uvc/uvcvideo.h
+++ b/drivers/media/usb/uvc/uvcvideo.h
@@ -189,7 +189,7 @@
/* Maximum status buffer size in bytes of interrupt URB. */
#define UVC_MAX_STATUS_SIZE 16
-#define UVC_CTRL_CONTROL_TIMEOUT 500
+#define UVC_CTRL_CONTROL_TIMEOUT 5000
#define UVC_CTRL_STREAMING_TIMEOUT 5000
/* Maximum allowed number of control mappings per device */
diff --git a/drivers/media/v4l2-core/Kconfig b/drivers/media/v4l2-core/Kconfig
index 02dc178..6ee75c6 100644
--- a/drivers/media/v4l2-core/Kconfig
+++ b/drivers/media/v4l2-core/Kconfig
@@ -52,6 +52,10 @@
config V4L2_H264
tristate
+# Used by drivers that need v4l2-vp9.ko
+config V4L2_VP9
+ tristate
+
# Used by drivers that need v4l2-mem2mem.ko
config V4L2_MEM2MEM_DEV
tristate
diff --git a/drivers/media/v4l2-core/Makefile b/drivers/media/v4l2-core/Makefile
index 66a78c5..83fac5c 100644
--- a/drivers/media/v4l2-core/Makefile
+++ b/drivers/media/v4l2-core/Makefile
@@ -24,6 +24,7 @@
obj-$(CONFIG_V4L2_MEM2MEM_DEV) += v4l2-mem2mem.o
obj-$(CONFIG_V4L2_H264) += v4l2-h264.o
+obj-$(CONFIG_V4L2_VP9) += v4l2-vp9.o
obj-$(CONFIG_V4L2_FLASH_LED_CLASS) += v4l2-flash-led-class.o
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-core.c b/drivers/media/v4l2-core/v4l2-ctrls-core.c
index 70adfc1..54abe52 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-core.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-core.c
@@ -283,6 +283,12 @@ static void std_log(const struct v4l2_ctrl *ctrl)
case V4L2_CTRL_TYPE_MPEG2_PICTURE:
pr_cont("MPEG2_PICTURE");
break;
+ case V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR:
+ pr_cont("VP9_COMPRESSED_HDR");
+ break;
+ case V4L2_CTRL_TYPE_VP9_FRAME:
+ pr_cont("VP9_FRAME");
+ break;
default:
pr_cont("unknown type %d", ctrl->type);
break;
@@ -317,6 +323,168 @@ static void std_log(const struct v4l2_ctrl *ctrl)
#define zero_reserved(s) \
memset(&(s).reserved, 0, sizeof((s).reserved))
+static int
+validate_vp9_lf_params(struct v4l2_vp9_loop_filter *lf)
+{
+ unsigned int i;
+
+ if (lf->flags & ~(V4L2_VP9_LOOP_FILTER_FLAG_DELTA_ENABLED |
+ V4L2_VP9_LOOP_FILTER_FLAG_DELTA_UPDATE))
+ return -EINVAL;
+
+ /* That all values are in the accepted range. */
+ if (lf->level > GENMASK(5, 0))
+ return -EINVAL;
+
+ if (lf->sharpness > GENMASK(2, 0))
+ return -EINVAL;
+
+ for (i = 0; i < ARRAY_SIZE(lf->ref_deltas); i++)
+ if (lf->ref_deltas[i] < -63 || lf->ref_deltas[i] > 63)
+ return -EINVAL;
+
+ for (i = 0; i < ARRAY_SIZE(lf->mode_deltas); i++)
+ if (lf->mode_deltas[i] < -63 || lf->mode_deltas[i] > 63)
+ return -EINVAL;
+
+ zero_reserved(*lf);
+ return 0;
+}
+
+static int
+validate_vp9_quant_params(struct v4l2_vp9_quantization *quant)
+{
+ if (quant->delta_q_y_dc < -15 || quant->delta_q_y_dc > 15 ||
+ quant->delta_q_uv_dc < -15 || quant->delta_q_uv_dc > 15 ||
+ quant->delta_q_uv_ac < -15 || quant->delta_q_uv_ac > 15)
+ return -EINVAL;
+
+ zero_reserved(*quant);
+ return 0;
+}
+
+static int
+validate_vp9_seg_params(struct v4l2_vp9_segmentation *seg)
+{
+ unsigned int i, j;
+
+ if (seg->flags & ~(V4L2_VP9_SEGMENTATION_FLAG_ENABLED |
+ V4L2_VP9_SEGMENTATION_FLAG_UPDATE_MAP |
+ V4L2_VP9_SEGMENTATION_FLAG_TEMPORAL_UPDATE |
+ V4L2_VP9_SEGMENTATION_FLAG_UPDATE_DATA |
+ V4L2_VP9_SEGMENTATION_FLAG_ABS_OR_DELTA_UPDATE))
+ return -EINVAL;
+
+ for (i = 0; i < ARRAY_SIZE(seg->feature_enabled); i++) {
+ if (seg->feature_enabled[i] &
+ ~V4L2_VP9_SEGMENT_FEATURE_ENABLED_MASK)
+ return -EINVAL;
+ }
+
+ for (i = 0; i < ARRAY_SIZE(seg->feature_data); i++) {
+ const int range[] = { 255, 63, 3, 0 };
+
+ for (j = 0; j < ARRAY_SIZE(seg->feature_data[j]); j++) {
+ if (seg->feature_data[i][j] < -range[j] ||
+ seg->feature_data[i][j] > range[j])
+ return -EINVAL;
+ }
+ }
+
+ zero_reserved(*seg);
+ return 0;
+}
+
+static int
+validate_vp9_compressed_hdr(struct v4l2_ctrl_vp9_compressed_hdr *hdr)
+{
+ if (hdr->tx_mode > V4L2_VP9_TX_MODE_SELECT)
+ return -EINVAL;
+
+ return 0;
+}
+
+static int
+validate_vp9_frame(struct v4l2_ctrl_vp9_frame *frame)
+{
+ int ret;
+
+ /* Make sure we're not passed invalid flags. */
+ if (frame->flags & ~(V4L2_VP9_FRAME_FLAG_KEY_FRAME |
+ V4L2_VP9_FRAME_FLAG_SHOW_FRAME |
+ V4L2_VP9_FRAME_FLAG_ERROR_RESILIENT |
+ V4L2_VP9_FRAME_FLAG_INTRA_ONLY |
+ V4L2_VP9_FRAME_FLAG_ALLOW_HIGH_PREC_MV |
+ V4L2_VP9_FRAME_FLAG_REFRESH_FRAME_CTX |
+ V4L2_VP9_FRAME_FLAG_PARALLEL_DEC_MODE |
+ V4L2_VP9_FRAME_FLAG_X_SUBSAMPLING |
+ V4L2_VP9_FRAME_FLAG_Y_SUBSAMPLING |
+ V4L2_VP9_FRAME_FLAG_COLOR_RANGE_FULL_SWING))
+ return -EINVAL;
+
+ if (frame->flags & V4L2_VP9_FRAME_FLAG_ERROR_RESILIENT &&
+ frame->flags & V4L2_VP9_FRAME_FLAG_REFRESH_FRAME_CTX)
+ return -EINVAL;
+
+ if (frame->profile > V4L2_VP9_PROFILE_MAX)
+ return -EINVAL;
+
+ if (frame->reset_frame_context > V4L2_VP9_RESET_FRAME_CTX_ALL)
+ return -EINVAL;
+
+ if (frame->frame_context_idx >= V4L2_VP9_NUM_FRAME_CTX)
+ return -EINVAL;
+
+ /*
+ * Profiles 0 and 1 only support 8-bit depth, profiles 2 and 3 only 10
+ * and 12 bit depths.
+ */
+ if ((frame->profile < 2 && frame->bit_depth != 8) ||
+ (frame->profile >= 2 &&
+ (frame->bit_depth != 10 && frame->bit_depth != 12)))
+ return -EINVAL;
+
+ /* Profile 0 and 2 only accept YUV 4:2:0. */
+ if ((frame->profile == 0 || frame->profile == 2) &&
+ (!(frame->flags & V4L2_VP9_FRAME_FLAG_X_SUBSAMPLING) ||
+ !(frame->flags & V4L2_VP9_FRAME_FLAG_Y_SUBSAMPLING)))
+ return -EINVAL;
+
+ /* Profile 1 and 3 only accept YUV 4:2:2, 4:4:0 and 4:4:4. */
+ if ((frame->profile == 1 || frame->profile == 3) &&
+ ((frame->flags & V4L2_VP9_FRAME_FLAG_X_SUBSAMPLING) &&
+ (frame->flags & V4L2_VP9_FRAME_FLAG_Y_SUBSAMPLING)))
+ return -EINVAL;
+
+ if (frame->interpolation_filter > V4L2_VP9_INTERP_FILTER_SWITCHABLE)
+ return -EINVAL;
+
+ /*
+ * According to the spec, tile_cols_log2 shall be less than or equal
+ * to 6.
+ */
+ if (frame->tile_cols_log2 > 6)
+ return -EINVAL;
+
+ if (frame->reference_mode > V4L2_VP9_REFERENCE_MODE_SELECT)
+ return -EINVAL;
+
+ ret = validate_vp9_lf_params(&frame->lf);
+ if (ret)
+ return ret;
+
+ ret = validate_vp9_quant_params(&frame->quant);
+ if (ret)
+ return ret;
+
+ ret = validate_vp9_seg_params(&frame->seg);
+ if (ret)
+ return ret;
+
+ zero_reserved(*frame);
+ return 0;
+}
+
/*
* Compound controls validation requires setting unused fields/flags to zero
* in order to properly detect unchanged controls with std_equal's memcmp.
@@ -690,6 +858,12 @@ static int std_validate_compound(const struct v4l2_ctrl *ctrl, u32 idx,
case V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX:
break;
+ case V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR:
+ return validate_vp9_compressed_hdr(p);
+
+ case V4L2_CTRL_TYPE_VP9_FRAME:
+ return validate_vp9_frame(p);
+
case V4L2_CTRL_TYPE_AREA:
area = p;
if (!area->width || !area->height)
@@ -1255,6 +1429,12 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
case V4L2_CTRL_TYPE_HDR10_MASTERING_DISPLAY:
elem_size = sizeof(struct v4l2_ctrl_hdr10_mastering_display);
break;
+ case V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR:
+ elem_size = sizeof(struct v4l2_ctrl_vp9_compressed_hdr);
+ break;
+ case V4L2_CTRL_TYPE_VP9_FRAME:
+ elem_size = sizeof(struct v4l2_ctrl_vp9_frame);
+ break;
case V4L2_CTRL_TYPE_AREA:
elem_size = sizeof(struct v4l2_area);
break;
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
index ebe82b6..54ca4e6 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
@@ -785,6 +785,7 @@ const char *v4l2_ctrl_get_name(u32 id)
case V4L2_CID_MIN_BUFFERS_FOR_OUTPUT: return "Min Number of Output Buffers";
case V4L2_CID_ALPHA_COMPONENT: return "Alpha Component";
case V4L2_CID_COLORFX_CBCR: return "Color Effects, CbCr";
+ case V4L2_CID_COLORFX_RGB: return "Color Effects, RGB";
/*
* Codec controls
@@ -1177,6 +1178,8 @@ const char *v4l2_ctrl_get_name(u32 id)
case V4L2_CID_STATELESS_MPEG2_SEQUENCE: return "MPEG-2 Sequence Header";
case V4L2_CID_STATELESS_MPEG2_PICTURE: return "MPEG-2 Picture Header";
case V4L2_CID_STATELESS_MPEG2_QUANTISATION: return "MPEG-2 Quantisation Matrices";
+ case V4L2_CID_STATELESS_VP9_COMPRESSED_HDR: return "VP9 Probabilities Updates";
+ case V4L2_CID_STATELESS_VP9_FRAME: return "VP9 Frame Decode Parameters";
/* Colorimetry controls */
/* Keep the order of the 'case's the same as in v4l2-controls.h! */
@@ -1394,11 +1397,18 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
*min = *max = *step = *def = 0;
break;
case V4L2_CID_BG_COLOR:
+ case V4L2_CID_COLORFX_RGB:
*type = V4L2_CTRL_TYPE_INTEGER;
*step = 1;
*min = 0;
- /* Max is calculated as RGB888 that is 2^24 */
- *max = 0xFFFFFF;
+ /* Max is calculated as RGB888 that is 2^24 - 1 */
+ *max = 0xffffff;
+ break;
+ case V4L2_CID_COLORFX_CBCR:
+ *type = V4L2_CTRL_TYPE_INTEGER;
+ *step = 1;
+ *min = 0;
+ *max = 0xffff;
break;
case V4L2_CID_FLASH_FAULT:
case V4L2_CID_JPEG_ACTIVE_MARKER:
@@ -1498,6 +1508,12 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS:
*type = V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS;
break;
+ case V4L2_CID_STATELESS_VP9_COMPRESSED_HDR:
+ *type = V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR;
+ break;
+ case V4L2_CID_STATELESS_VP9_FRAME:
+ *type = V4L2_CTRL_TYPE_VP9_FRAME;
+ break;
case V4L2_CID_UNIT_CELL_SIZE:
*type = V4L2_CTRL_TYPE_AREA;
*flags |= V4L2_CTRL_FLAG_READ_ONLY;
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index 31d0109..9ac557b 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1413,6 +1413,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
case V4L2_PIX_FMT_VP8: descr = "VP8"; break;
case V4L2_PIX_FMT_VP8_FRAME: descr = "VP8 Frame"; break;
case V4L2_PIX_FMT_VP9: descr = "VP9"; break;
+ case V4L2_PIX_FMT_VP9_FRAME: descr = "VP9 Frame"; break;
case V4L2_PIX_FMT_HEVC: descr = "HEVC"; break; /* aka H.265 */
case V4L2_PIX_FMT_HEVC_SLICE: descr = "HEVC Parsed Slice Data"; break;
case V4L2_PIX_FMT_FWHT: descr = "FWHT"; break; /* used in vicodec */
@@ -2090,6 +2091,7 @@ static int v4l_prepare_buf(const struct v4l2_ioctl_ops *ops,
static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
struct file *file, void *fh, void *arg)
{
+ struct video_device *vfd = video_devdata(file);
struct v4l2_streamparm *p = arg;
v4l2_std_id std;
int ret = check_fmt(file, p->type);
@@ -2101,7 +2103,8 @@ static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
if (p->type != V4L2_BUF_TYPE_VIDEO_CAPTURE &&
p->type != V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
return -EINVAL;
- p->parm.capture.readbuffers = 2;
+ if (vfd->device_caps & V4L2_CAP_READWRITE)
+ p->parm.capture.readbuffers = 2;
ret = ops->vidioc_g_std(file, fh, &std);
if (ret == 0)
v4l2_video_std_frame_period(std, &p->parm.capture.timeperframe);
diff --git a/drivers/media/v4l2-core/v4l2-mem2mem.c b/drivers/media/v4l2-core/v4l2-mem2mem.c
index e7f4bf5..e2654b4 100644
--- a/drivers/media/v4l2-core/v4l2-mem2mem.c
+++ b/drivers/media/v4l2-core/v4l2-mem2mem.c
@@ -966,6 +966,27 @@ int v4l2_m2m_mmap(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
}
EXPORT_SYMBOL(v4l2_m2m_mmap);
+#ifndef CONFIG_MMU
+unsigned long v4l2_m2m_get_unmapped_area(struct file *file, unsigned long addr,
+ unsigned long len, unsigned long pgoff,
+ unsigned long flags)
+{
+ struct v4l2_fh *fh = file->private_data;
+ unsigned long offset = pgoff << PAGE_SHIFT;
+ struct vb2_queue *vq;
+
+ if (offset < DST_QUEUE_OFF_BASE) {
+ vq = v4l2_m2m_get_src_vq(fh->m2m_ctx);
+ } else {
+ vq = v4l2_m2m_get_dst_vq(fh->m2m_ctx);
+ pgoff -= (DST_QUEUE_OFF_BASE >> PAGE_SHIFT);
+ }
+
+ return vb2_get_unmapped_area(vq, addr, len, pgoff, flags);
+}
+EXPORT_SYMBOL_GPL(v4l2_m2m_get_unmapped_area);
+#endif
+
#if defined(CONFIG_MEDIA_CONTROLLER)
void v4l2_m2m_unregister_media_controller(struct v4l2_m2m_dev *m2m_dev)
{
diff --git a/drivers/media/v4l2-core/v4l2-vp9.c b/drivers/media/v4l2-core/v4l2-vp9.c
new file mode 100644
index 0000000..859589f
--- /dev/null
+++ b/drivers/media/v4l2-core/v4l2-vp9.c
@@ -0,0 +1,1850 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * V4L2 VP9 helpers.
+ *
+ * Copyright (C) 2021 Collabora, Ltd.
+ *
+ * Author: Andrzej Pietrasiewicz <andrzej.p@collabora.com>
+ */
+
+#include <linux/module.h>
+
+#include <media/v4l2-vp9.h>
+
+const u8 v4l2_vp9_kf_y_mode_prob[10][10][9] = {
+ {
+ /* above = dc */
+ { 137, 30, 42, 148, 151, 207, 70, 52, 91 }, /*left = dc */
+ { 92, 45, 102, 136, 116, 180, 74, 90, 100 }, /*left = v */
+ { 73, 32, 19, 187, 222, 215, 46, 34, 100 }, /*left = h */
+ { 91, 30, 32, 116, 121, 186, 93, 86, 94 }, /*left = d45 */
+ { 72, 35, 36, 149, 68, 206, 68, 63, 105 }, /*left = d135*/
+ { 73, 31, 28, 138, 57, 124, 55, 122, 151 }, /*left = d117*/
+ { 67, 23, 21, 140, 126, 197, 40, 37, 171 }, /*left = d153*/
+ { 86, 27, 28, 128, 154, 212, 45, 43, 53 }, /*left = d207*/
+ { 74, 32, 27, 107, 86, 160, 63, 134, 102 }, /*left = d63 */
+ { 59, 67, 44, 140, 161, 202, 78, 67, 119 }, /*left = tm */
+ }, { /* above = v */
+ { 63, 36, 126, 146, 123, 158, 60, 90, 96 }, /*left = dc */
+ { 43, 46, 168, 134, 107, 128, 69, 142, 92 }, /*left = v */
+ { 44, 29, 68, 159, 201, 177, 50, 57, 77 }, /*left = h */
+ { 58, 38, 76, 114, 97, 172, 78, 133, 92 }, /*left = d45 */
+ { 46, 41, 76, 140, 63, 184, 69, 112, 57 }, /*left = d135*/
+ { 38, 32, 85, 140, 46, 112, 54, 151, 133 }, /*left = d117*/
+ { 39, 27, 61, 131, 110, 175, 44, 75, 136 }, /*left = d153*/
+ { 52, 30, 74, 113, 130, 175, 51, 64, 58 }, /*left = d207*/
+ { 47, 35, 80, 100, 74, 143, 64, 163, 74 }, /*left = d63 */
+ { 36, 61, 116, 114, 128, 162, 80, 125, 82 }, /*left = tm */
+ }, { /* above = h */
+ { 82, 26, 26, 171, 208, 204, 44, 32, 105 }, /*left = dc */
+ { 55, 44, 68, 166, 179, 192, 57, 57, 108 }, /*left = v */
+ { 42, 26, 11, 199, 241, 228, 23, 15, 85 }, /*left = h */
+ { 68, 42, 19, 131, 160, 199, 55, 52, 83 }, /*left = d45 */
+ { 58, 50, 25, 139, 115, 232, 39, 52, 118 }, /*left = d135*/
+ { 50, 35, 33, 153, 104, 162, 64, 59, 131 }, /*left = d117*/
+ { 44, 24, 16, 150, 177, 202, 33, 19, 156 }, /*left = d153*/
+ { 55, 27, 12, 153, 203, 218, 26, 27, 49 }, /*left = d207*/
+ { 53, 49, 21, 110, 116, 168, 59, 80, 76 }, /*left = d63 */
+ { 38, 72, 19, 168, 203, 212, 50, 50, 107 }, /*left = tm */
+ }, { /* above = d45 */
+ { 103, 26, 36, 129, 132, 201, 83, 80, 93 }, /*left = dc */
+ { 59, 38, 83, 112, 103, 162, 98, 136, 90 }, /*left = v */
+ { 62, 30, 23, 158, 200, 207, 59, 57, 50 }, /*left = h */
+ { 67, 30, 29, 84, 86, 191, 102, 91, 59 }, /*left = d45 */
+ { 60, 32, 33, 112, 71, 220, 64, 89, 104 }, /*left = d135*/
+ { 53, 26, 34, 130, 56, 149, 84, 120, 103 }, /*left = d117*/
+ { 53, 21, 23, 133, 109, 210, 56, 77, 172 }, /*left = d153*/
+ { 77, 19, 29, 112, 142, 228, 55, 66, 36 }, /*left = d207*/
+ { 61, 29, 29, 93, 97, 165, 83, 175, 162 }, /*left = d63 */
+ { 47, 47, 43, 114, 137, 181, 100, 99, 95 }, /*left = tm */
+ }, { /* above = d135 */
+ { 69, 23, 29, 128, 83, 199, 46, 44, 101 }, /*left = dc */
+ { 53, 40, 55, 139, 69, 183, 61, 80, 110 }, /*left = v */
+ { 40, 29, 19, 161, 180, 207, 43, 24, 91 }, /*left = h */
+ { 60, 34, 19, 105, 61, 198, 53, 64, 89 }, /*left = d45 */
+ { 52, 31, 22, 158, 40, 209, 58, 62, 89 }, /*left = d135*/
+ { 44, 31, 29, 147, 46, 158, 56, 102, 198 }, /*left = d117*/
+ { 35, 19, 12, 135, 87, 209, 41, 45, 167 }, /*left = d153*/
+ { 55, 25, 21, 118, 95, 215, 38, 39, 66 }, /*left = d207*/
+ { 51, 38, 25, 113, 58, 164, 70, 93, 97 }, /*left = d63 */
+ { 47, 54, 34, 146, 108, 203, 72, 103, 151 }, /*left = tm */
+ }, { /* above = d117 */
+ { 64, 19, 37, 156, 66, 138, 49, 95, 133 }, /*left = dc */
+ { 46, 27, 80, 150, 55, 124, 55, 121, 135 }, /*left = v */
+ { 36, 23, 27, 165, 149, 166, 54, 64, 118 }, /*left = h */
+ { 53, 21, 36, 131, 63, 163, 60, 109, 81 }, /*left = d45 */
+ { 40, 26, 35, 154, 40, 185, 51, 97, 123 }, /*left = d135*/
+ { 35, 19, 34, 179, 19, 97, 48, 129, 124 }, /*left = d117*/
+ { 36, 20, 26, 136, 62, 164, 33, 77, 154 }, /*left = d153*/
+ { 45, 18, 32, 130, 90, 157, 40, 79, 91 }, /*left = d207*/
+ { 45, 26, 28, 129, 45, 129, 49, 147, 123 }, /*left = d63 */
+ { 38, 44, 51, 136, 74, 162, 57, 97, 121 }, /*left = tm */
+ }, { /* above = d153 */
+ { 75, 17, 22, 136, 138, 185, 32, 34, 166 }, /*left = dc */
+ { 56, 39, 58, 133, 117, 173, 48, 53, 187 }, /*left = v */
+ { 35, 21, 12, 161, 212, 207, 20, 23, 145 }, /*left = h */
+ { 56, 29, 19, 117, 109, 181, 55, 68, 112 }, /*left = d45 */
+ { 47, 29, 17, 153, 64, 220, 59, 51, 114 }, /*left = d135*/
+ { 46, 16, 24, 136, 76, 147, 41, 64, 172 }, /*left = d117*/
+ { 34, 17, 11, 108, 152, 187, 13, 15, 209 }, /*left = d153*/
+ { 51, 24, 14, 115, 133, 209, 32, 26, 104 }, /*left = d207*/
+ { 55, 30, 18, 122, 79, 179, 44, 88, 116 }, /*left = d63 */
+ { 37, 49, 25, 129, 168, 164, 41, 54, 148 }, /*left = tm */
+ }, { /* above = d207 */
+ { 82, 22, 32, 127, 143, 213, 39, 41, 70 }, /*left = dc */
+ { 62, 44, 61, 123, 105, 189, 48, 57, 64 }, /*left = v */
+ { 47, 25, 17, 175, 222, 220, 24, 30, 86 }, /*left = h */
+ { 68, 36, 17, 106, 102, 206, 59, 74, 74 }, /*left = d45 */
+ { 57, 39, 23, 151, 68, 216, 55, 63, 58 }, /*left = d135*/
+ { 49, 30, 35, 141, 70, 168, 82, 40, 115 }, /*left = d117*/
+ { 51, 25, 15, 136, 129, 202, 38, 35, 139 }, /*left = d153*/
+ { 68, 26, 16, 111, 141, 215, 29, 28, 28 }, /*left = d207*/
+ { 59, 39, 19, 114, 75, 180, 77, 104, 42 }, /*left = d63 */
+ { 40, 61, 26, 126, 152, 206, 61, 59, 93 }, /*left = tm */
+ }, { /* above = d63 */
+ { 78, 23, 39, 111, 117, 170, 74, 124, 94 }, /*left = dc */
+ { 48, 34, 86, 101, 92, 146, 78, 179, 134 }, /*left = v */
+ { 47, 22, 24, 138, 187, 178, 68, 69, 59 }, /*left = h */
+ { 56, 25, 33, 105, 112, 187, 95, 177, 129 }, /*left = d45 */
+ { 48, 31, 27, 114, 63, 183, 82, 116, 56 }, /*left = d135*/
+ { 43, 28, 37, 121, 63, 123, 61, 192, 169 }, /*left = d117*/
+ { 42, 17, 24, 109, 97, 177, 56, 76, 122 }, /*left = d153*/
+ { 58, 18, 28, 105, 139, 182, 70, 92, 63 }, /*left = d207*/
+ { 46, 23, 32, 74, 86, 150, 67, 183, 88 }, /*left = d63 */
+ { 36, 38, 48, 92, 122, 165, 88, 137, 91 }, /*left = tm */
+ }, { /* above = tm */
+ { 65, 70, 60, 155, 159, 199, 61, 60, 81 }, /*left = dc */
+ { 44, 78, 115, 132, 119, 173, 71, 112, 93 }, /*left = v */
+ { 39, 38, 21, 184, 227, 206, 42, 32, 64 }, /*left = h */
+ { 58, 47, 36, 124, 137, 193, 80, 82, 78 }, /*left = d45 */
+ { 49, 50, 35, 144, 95, 205, 63, 78, 59 }, /*left = d135*/
+ { 41, 53, 52, 148, 71, 142, 65, 128, 51 }, /*left = d117*/
+ { 40, 36, 28, 143, 143, 202, 40, 55, 137 }, /*left = d153*/
+ { 52, 34, 29, 129, 183, 227, 42, 35, 43 }, /*left = d207*/
+ { 42, 44, 44, 104, 105, 164, 64, 130, 80 }, /*left = d63 */
+ { 43, 81, 53, 140, 169, 204, 68, 84, 72 }, /*left = tm */
+ }
+};
+EXPORT_SYMBOL_GPL(v4l2_vp9_kf_y_mode_prob);
+
+const u8 v4l2_vp9_kf_partition_probs[16][3] = {
+ /* 8x8 -> 4x4 */
+ { 158, 97, 94 }, /* a/l both not split */
+ { 93, 24, 99 }, /* a split, l not split */
+ { 85, 119, 44 }, /* l split, a not split */
+ { 62, 59, 67 }, /* a/l both split */
+ /* 16x16 -> 8x8 */
+ { 149, 53, 53 }, /* a/l both not split */
+ { 94, 20, 48 }, /* a split, l not split */
+ { 83, 53, 24 }, /* l split, a not split */
+ { 52, 18, 18 }, /* a/l both split */
+ /* 32x32 -> 16x16 */
+ { 150, 40, 39 }, /* a/l both not split */
+ { 78, 12, 26 }, /* a split, l not split */
+ { 67, 33, 11 }, /* l split, a not split */
+ { 24, 7, 5 }, /* a/l both split */
+ /* 64x64 -> 32x32 */
+ { 174, 35, 49 }, /* a/l both not split */
+ { 68, 11, 27 }, /* a split, l not split */
+ { 57, 15, 9 }, /* l split, a not split */
+ { 12, 3, 3 }, /* a/l both split */
+};
+EXPORT_SYMBOL_GPL(v4l2_vp9_kf_partition_probs);
+
+const u8 v4l2_vp9_kf_uv_mode_prob[10][9] = {
+ { 144, 11, 54, 157, 195, 130, 46, 58, 108 }, /* y = dc */
+ { 118, 15, 123, 148, 131, 101, 44, 93, 131 }, /* y = v */
+ { 113, 12, 23, 188, 226, 142, 26, 32, 125 }, /* y = h */
+ { 120, 11, 50, 123, 163, 135, 64, 77, 103 }, /* y = d45 */
+ { 113, 9, 36, 155, 111, 157, 32, 44, 161 }, /* y = d135 */
+ { 116, 9, 55, 176, 76, 96, 37, 61, 149 }, /* y = d117 */
+ { 115, 9, 28, 141, 161, 167, 21, 25, 193 }, /* y = d153 */
+ { 120, 12, 32, 145, 195, 142, 32, 38, 86 }, /* y = d207 */
+ { 116, 12, 64, 120, 140, 125, 49, 115, 121 }, /* y = d63 */
+ { 102, 19, 66, 162, 182, 122, 35, 59, 128 } /* y = tm */
+};
+EXPORT_SYMBOL_GPL(v4l2_vp9_kf_uv_mode_prob);
+
+const struct v4l2_vp9_frame_context v4l2_vp9_default_probs = {
+ .tx8 = {
+ { 100 },
+ { 66 },
+ },
+ .tx16 = {
+ { 20, 152 },
+ { 15, 101 },
+ },
+ .tx32 = {
+ { 3, 136, 37 },
+ { 5, 52, 13 },
+ },
+ .coef = {
+ { /* tx = 4x4 */
+ { /* block Type 0 */
+ { /* Intra */
+ { /* Coeff Band 0 */
+ { 195, 29, 183 },
+ { 84, 49, 136 },
+ { 8, 42, 71 },
+ { 0, 0, 0 },
+ { 0, 0, 0 },
+ { 0, 0, 0 },
+ },
+ { /* Coeff Band 1 */
+ { 31, 107, 169 },
+ { 35, 99, 159 },
+ { 17, 82, 140 },
+ { 8, 66, 114 },
+ { 2, 44, 76 },
+ { 1, 19, 32 },
+ },
+ { /* Coeff Band 2 */
+ { 40, 132, 201 },
+ { 29, 114, 187 },
+ { 13, 91, 157 },
+ { 7, 75, 127 },
+ { 3, 58, 95 },
+ { 1, 28, 47 },
+ },
+ { /* Coeff Band 3 */
+ { 69, 142, 221 },
+ { 42, 122, 201 },
+ { 15, 91, 159 },
+ { 6, 67, 121 },
+ { 1, 42, 77 },
+ { 1, 17, 31 },
+ },
+ { /* Coeff Band 4 */
+ { 102, 148, 228 },
+ { 67, 117, 204 },
+ { 17, 82, 154 },
+ { 6, 59, 114 },
+ { 2, 39, 75 },
+ { 1, 15, 29 },
+ },
+ { /* Coeff Band 5 */
+ { 156, 57, 233 },
+ { 119, 57, 212 },
+ { 58, 48, 163 },
+ { 29, 40, 124 },
+ { 12, 30, 81 },
+ { 3, 12, 31 }
+ },
+ },
+ { /* Inter */
+ { /* Coeff Band 0 */
+ { 191, 107, 226 },
+ { 124, 117, 204 },
+ { 25, 99, 155 },
+ { 0, 0, 0 },
+ { 0, 0, 0 },
+ { 0, 0, 0 },
+ },
+ { /* Coeff Band 1 */
+ { 29, 148, 210 },
+ { 37, 126, 194 },
+ { 8, 93, 157 },
+ { 2, 68, 118 },
+ { 1, 39, 69 },
+ { 1, 17, 33 },
+ },
+ { /* Coeff Band 2 */
+ { 41, 151, 213 },
+ { 27, 123, 193 },
+ { 3, 82, 144 },
+ { 1, 58, 105 },
+ { 1, 32, 60 },
+ { 1, 13, 26 },
+ },
+ { /* Coeff Band 3 */
+ { 59, 159, 220 },
+ { 23, 126, 198 },
+ { 4, 88, 151 },
+ { 1, 66, 114 },
+ { 1, 38, 71 },
+ { 1, 18, 34 },
+ },
+ { /* Coeff Band 4 */
+ { 114, 136, 232 },
+ { 51, 114, 207 },
+ { 11, 83, 155 },
+ { 3, 56, 105 },
+ { 1, 33, 65 },
+ { 1, 17, 34 },
+ },
+ { /* Coeff Band 5 */
+ { 149, 65, 234 },
+ { 121, 57, 215 },
+ { 61, 49, 166 },
+ { 28, 36, 114 },
+ { 12, 25, 76 },
+ { 3, 16, 42 },
+ },
+ },
+ },
+ { /* block Type 1 */
+ { /* Intra */
+ { /* Coeff Band 0 */
+ { 214, 49, 220 },
+ { 132, 63, 188 },
+ { 42, 65, 137 },
+ { 0, 0, 0 },
+ { 0, 0, 0 },
+ { 0, 0, 0 },
+ },
+ { /* Coeff Band 1 */
+ { 85, 137, 221 },
+ { 104, 131, 216 },
+ { 49, 111, 192 },
+ { 21, 87, 155 },
+ { 2, 49, 87 },
+ { 1, 16, 28 },
+ },
+ { /* Coeff Band 2 */
+ { 89, 163, 230 },
+ { 90, 137, 220 },
+ { 29, 100, 183 },
+ { 10, 70, 135 },
+ { 2, 42, 81 },
+ { 1, 17, 33 },
+ },
+ { /* Coeff Band 3 */
+ { 108, 167, 237 },
+ { 55, 133, 222 },
+ { 15, 97, 179 },
+ { 4, 72, 135 },
+ { 1, 45, 85 },
+ { 1, 19, 38 },
+ },
+ { /* Coeff Band 4 */
+ { 124, 146, 240 },
+ { 66, 124, 224 },
+ { 17, 88, 175 },
+ { 4, 58, 122 },
+ { 1, 36, 75 },
+ { 1, 18, 37 },
+ },
+ { /* Coeff Band 5 */
+ { 141, 79, 241 },
+ { 126, 70, 227 },
+ { 66, 58, 182 },
+ { 30, 44, 136 },
+ { 12, 34, 96 },
+ { 2, 20, 47 },
+ },
+ },
+ { /* Inter */
+ { /* Coeff Band 0 */
+ { 229, 99, 249 },
+ { 143, 111, 235 },
+ { 46, 109, 192 },
+ { 0, 0, 0 },
+ { 0, 0, 0 },
+ { 0, 0, 0 },
+ },
+ { /* Coeff Band 1 */
+ { 82, 158, 236 },
+ { 94, 146, 224 },
+ { 25, 117, 191 },
+ { 9, 87, 149 },
+ { 3, 56, 99 },
+ { 1, 33, 57 },
+ },
+ { /* Coeff Band 2 */
+ { 83, 167, 237 },
+ { 68, 145, 222 },
+ { 10, 103, 177 },
+ { 2, 72, 131 },
+ { 1, 41, 79 },
+ { 1, 20, 39 },
+ },
+ { /* Coeff Band 3 */
+ { 99, 167, 239 },
+ { 47, 141, 224 },
+ { 10, 104, 178 },
+ { 2, 73, 133 },
+ { 1, 44, 85 },
+ { 1, 22, 47 },
+ },
+ { /* Coeff Band 4 */
+ { 127, 145, 243 },
+ { 71, 129, 228 },
+ { 17, 93, 177 },
+ { 3, 61, 124 },
+ { 1, 41, 84 },
+ { 1, 21, 52 },
+ },
+ { /* Coeff Band 5 */
+ { 157, 78, 244 },
+ { 140, 72, 231 },
+ { 69, 58, 184 },
+ { 31, 44, 137 },
+ { 14, 38, 105 },
+ { 8, 23, 61 },
+ },
+ },
+ },
+ },
+ { /* tx = 8x8 */
+ { /* block Type 0 */
+ { /* Intra */
+ { /* Coeff Band 0 */
+ { 125, 34, 187 },
+ { 52, 41, 133 },
+ { 6, 31, 56 },
+ { 0, 0, 0 },
+ { 0, 0, 0 },
+ { 0, 0, 0 },
+ },
+ { /* Coeff Band 1 */
+ { 37, 109, 153 },
+ { 51, 102, 147 },
+ { 23, 87, 128 },
+ { 8, 67, 101 },
+ { 1, 41, 63 },
+ { 1, 19, 29 },
+ },
+ { /* Coeff Band 2 */
+ { 31, 154, 185 },
+ { 17, 127, 175 },
+ { 6, 96, 145 },
+ { 2, 73, 114 },
+ { 1, 51, 82 },
+ { 1, 28, 45 },
+ },
+ { /* Coeff Band 3 */
+ { 23, 163, 200 },
+ { 10, 131, 185 },
+ { 2, 93, 148 },
+ { 1, 67, 111 },
+ { 1, 41, 69 },
+ { 1, 14, 24 },
+ },
+ { /* Coeff Band 4 */
+ { 29, 176, 217 },
+ { 12, 145, 201 },
+ { 3, 101, 156 },
+ { 1, 69, 111 },
+ { 1, 39, 63 },
+ { 1, 14, 23 },
+ },
+ { /* Coeff Band 5 */
+ { 57, 192, 233 },
+ { 25, 154, 215 },
+ { 6, 109, 167 },
+ { 3, 78, 118 },
+ { 1, 48, 69 },
+ { 1, 21, 29 },
+ },
+ },
+ { /* Inter */
+ { /* Coeff Band 0 */
+ { 202, 105, 245 },
+ { 108, 106, 216 },
+ { 18, 90, 144 },
+ { 0, 0, 0 },
+ { 0, 0, 0 },
+ { 0, 0, 0 },
+ },
+ { /* Coeff Band 1 */
+ { 33, 172, 219 },
+ { 64, 149, 206 },
+ { 14, 117, 177 },
+ { 5, 90, 141 },
+ { 2, 61, 95 },
+ { 1, 37, 57 },
+ },
+ { /* Coeff Band 2 */
+ { 33, 179, 220 },
+ { 11, 140, 198 },
+ { 1, 89, 148 },
+ { 1, 60, 104 },
+ { 1, 33, 57 },
+ { 1, 12, 21 },
+ },
+ { /* Coeff Band 3 */
+ { 30, 181, 221 },
+ { 8, 141, 198 },
+ { 1, 87, 145 },
+ { 1, 58, 100 },
+ { 1, 31, 55 },
+ { 1, 12, 20 },
+ },
+ { /* Coeff Band 4 */
+ { 32, 186, 224 },
+ { 7, 142, 198 },
+ { 1, 86, 143 },
+ { 1, 58, 100 },
+ { 1, 31, 55 },
+ { 1, 12, 22 },
+ },
+ { /* Coeff Band 5 */
+ { 57, 192, 227 },
+ { 20, 143, 204 },
+ { 3, 96, 154 },
+ { 1, 68, 112 },
+ { 1, 42, 69 },
+ { 1, 19, 32 },
+ },
+ },
+ },
+ { /* block Type 1 */
+ { /* Intra */
+ { /* Coeff Band 0 */
+ { 212, 35, 215 },
+ { 113, 47, 169 },
+ { 29, 48, 105 },
+ { 0, 0, 0 },
+ { 0, 0, 0 },
+ { 0, 0, 0 },
+ },
+ { /* Coeff Band 1 */
+ { 74, 129, 203 },
+ { 106, 120, 203 },
+ { 49, 107, 178 },
+ { 19, 84, 144 },
+ { 4, 50, 84 },
+ { 1, 15, 25 },
+ },
+ { /* Coeff Band 2 */
+ { 71, 172, 217 },
+ { 44, 141, 209 },
+ { 15, 102, 173 },
+ { 6, 76, 133 },
+ { 2, 51, 89 },
+ { 1, 24, 42 },
+ },
+ { /* Coeff Band 3 */
+ { 64, 185, 231 },
+ { 31, 148, 216 },
+ { 8, 103, 175 },
+ { 3, 74, 131 },
+ { 1, 46, 81 },
+ { 1, 18, 30 },
+ },
+ { /* Coeff Band 4 */
+ { 65, 196, 235 },
+ { 25, 157, 221 },
+ { 5, 105, 174 },
+ { 1, 67, 120 },
+ { 1, 38, 69 },
+ { 1, 15, 30 },
+ },
+ { /* Coeff Band 5 */
+ { 65, 204, 238 },
+ { 30, 156, 224 },
+ { 7, 107, 177 },
+ { 2, 70, 124 },
+ { 1, 42, 73 },
+ { 1, 18, 34 },
+ },
+ },
+ { /* Inter */
+ { /* Coeff Band 0 */
+ { 225, 86, 251 },
+ { 144, 104, 235 },
+ { 42, 99, 181 },
+ { 0, 0, 0 },
+ { 0, 0, 0 },
+ { 0, 0, 0 },
+ },
+ { /* Coeff Band 1 */
+ { 85, 175, 239 },
+ { 112, 165, 229 },
+ { 29, 136, 200 },
+ { 12, 103, 162 },
+ { 6, 77, 123 },
+ { 2, 53, 84 },
+ },
+ { /* Coeff Band 2 */
+ { 75, 183, 239 },
+ { 30, 155, 221 },
+ { 3, 106, 171 },
+ { 1, 74, 128 },
+ { 1, 44, 76 },
+ { 1, 17, 28 },
+ },
+ { /* Coeff Band 3 */
+ { 73, 185, 240 },
+ { 27, 159, 222 },
+ { 2, 107, 172 },
+ { 1, 75, 127 },
+ { 1, 42, 73 },
+ { 1, 17, 29 },
+ },
+ { /* Coeff Band 4 */
+ { 62, 190, 238 },
+ { 21, 159, 222 },
+ { 2, 107, 172 },
+ { 1, 72, 122 },
+ { 1, 40, 71 },
+ { 1, 18, 32 },
+ },
+ { /* Coeff Band 5 */
+ { 61, 199, 240 },
+ { 27, 161, 226 },
+ { 4, 113, 180 },
+ { 1, 76, 129 },
+ { 1, 46, 80 },
+ { 1, 23, 41 },
+ },
+ },
+ },
+ },
+ { /* tx = 16x16 */
+ { /* block Type 0 */
+ { /* Intra */
+ { /* Coeff Band 0 */
+ { 7, 27, 153 },
+ { 5, 30, 95 },
+ { 1, 16, 30 },
+ { 0, 0, 0 },
+ { 0, 0, 0 },
+ { 0, 0, 0 },
+ },
+ { /* Coeff Band 1 */
+ { 50, 75, 127 },
+ { 57, 75, 124 },
+ { 27, 67, 108 },
+ { 10, 54, 86 },
+ { 1, 33, 52 },
+ { 1, 12, 18 },
+ },
+ { /* Coeff Band 2 */
+ { 43, 125, 151 },
+ { 26, 108, 148 },
+ { 7, 83, 122 },
+ { 2, 59, 89 },
+ { 1, 38, 60 },
+ { 1, 17, 27 },
+ },
+ { /* Coeff Band 3 */
+ { 23, 144, 163 },
+ { 13, 112, 154 },
+ { 2, 75, 117 },
+ { 1, 50, 81 },
+ { 1, 31, 51 },
+ { 1, 14, 23 },
+ },
+ { /* Coeff Band 4 */
+ { 18, 162, 185 },
+ { 6, 123, 171 },
+ { 1, 78, 125 },
+ { 1, 51, 86 },
+ { 1, 31, 54 },
+ { 1, 14, 23 },
+ },
+ { /* Coeff Band 5 */
+ { 15, 199, 227 },
+ { 3, 150, 204 },
+ { 1, 91, 146 },
+ { 1, 55, 95 },
+ { 1, 30, 53 },
+ { 1, 11, 20 },
+ }
+ },
+ { /* Inter */
+ { /* Coeff Band 0 */
+ { 19, 55, 240 },
+ { 19, 59, 196 },
+ { 3, 52, 105 },
+ { 0, 0, 0 },
+ { 0, 0, 0 },
+ { 0, 0, 0 },
+ },
+ { /* Coeff Band 1 */
+ { 41, 166, 207 },
+ { 104, 153, 199 },
+ { 31, 123, 181 },
+ { 14, 101, 152 },
+ { 5, 72, 106 },
+ { 1, 36, 52 },
+ },
+ { /* Coeff Band 2 */
+ { 35, 176, 211 },
+ { 12, 131, 190 },
+ { 2, 88, 144 },
+ { 1, 60, 101 },
+ { 1, 36, 60 },
+ { 1, 16, 28 },
+ },
+ { /* Coeff Band 3 */
+ { 28, 183, 213 },
+ { 8, 134, 191 },
+ { 1, 86, 142 },
+ { 1, 56, 96 },
+ { 1, 30, 53 },
+ { 1, 12, 20 },
+ },
+ { /* Coeff Band 4 */
+ { 20, 190, 215 },
+ { 4, 135, 192 },
+ { 1, 84, 139 },
+ { 1, 53, 91 },
+ { 1, 28, 49 },
+ { 1, 11, 20 },
+ },
+ { /* Coeff Band 5 */
+ { 13, 196, 216 },
+ { 2, 137, 192 },
+ { 1, 86, 143 },
+ { 1, 57, 99 },
+ { 1, 32, 56 },
+ { 1, 13, 24 },
+ },
+ },
+ },
+ { /* block Type 1 */
+ { /* Intra */
+ { /* Coeff Band 0 */
+ { 211, 29, 217 },
+ { 96, 47, 156 },
+ { 22, 43, 87 },
+ { 0, 0, 0 },
+ { 0, 0, 0 },
+ { 0, 0, 0 },
+ },
+ { /* Coeff Band 1 */
+ { 78, 120, 193 },
+ { 111, 116, 186 },
+ { 46, 102, 164 },
+ { 15, 80, 128 },
+ { 2, 49, 76 },
+ { 1, 18, 28 },
+ },
+ { /* Coeff Band 2 */
+ { 71, 161, 203 },
+ { 42, 132, 192 },
+ { 10, 98, 150 },
+ { 3, 69, 109 },
+ { 1, 44, 70 },
+ { 1, 18, 29 },
+ },
+ { /* Coeff Band 3 */
+ { 57, 186, 211 },
+ { 30, 140, 196 },
+ { 4, 93, 146 },
+ { 1, 62, 102 },
+ { 1, 38, 65 },
+ { 1, 16, 27 },
+ },
+ { /* Coeff Band 4 */
+ { 47, 199, 217 },
+ { 14, 145, 196 },
+ { 1, 88, 142 },
+ { 1, 57, 98 },
+ { 1, 36, 62 },
+ { 1, 15, 26 },
+ },
+ { /* Coeff Band 5 */
+ { 26, 219, 229 },
+ { 5, 155, 207 },
+ { 1, 94, 151 },
+ { 1, 60, 104 },
+ { 1, 36, 62 },
+ { 1, 16, 28 },
+ }
+ },
+ { /* Inter */
+ { /* Coeff Band 0 */
+ { 233, 29, 248 },
+ { 146, 47, 220 },
+ { 43, 52, 140 },
+ { 0, 0, 0 },
+ { 0, 0, 0 },
+ { 0, 0, 0 },
+ },
+ { /* Coeff Band 1 */
+ { 100, 163, 232 },
+ { 179, 161, 222 },
+ { 63, 142, 204 },
+ { 37, 113, 174 },
+ { 26, 89, 137 },
+ { 18, 68, 97 },
+ },
+ { /* Coeff Band 2 */
+ { 85, 181, 230 },
+ { 32, 146, 209 },
+ { 7, 100, 164 },
+ { 3, 71, 121 },
+ { 1, 45, 77 },
+ { 1, 18, 30 },
+ },
+ { /* Coeff Band 3 */
+ { 65, 187, 230 },
+ { 20, 148, 207 },
+ { 2, 97, 159 },
+ { 1, 68, 116 },
+ { 1, 40, 70 },
+ { 1, 14, 29 },
+ },
+ { /* Coeff Band 4 */
+ { 40, 194, 227 },
+ { 8, 147, 204 },
+ { 1, 94, 155 },
+ { 1, 65, 112 },
+ { 1, 39, 66 },
+ { 1, 14, 26 },
+ },
+ { /* Coeff Band 5 */
+ { 16, 208, 228 },
+ { 3, 151, 207 },
+ { 1, 98, 160 },
+ { 1, 67, 117 },
+ { 1, 41, 74 },
+ { 1, 17, 31 },
+ },
+ },
+ },
+ },
+ { /* tx = 32x32 */
+ { /* block Type 0 */
+ { /* Intra */
+ { /* Coeff Band 0 */
+ { 17, 38, 140 },
+ { 7, 34, 80 },
+ { 1, 17, 29 },
+ { 0, 0, 0 },
+ { 0, 0, 0 },
+ { 0, 0, 0 },
+ },
+ { /* Coeff Band 1 */
+ { 37, 75, 128 },
+ { 41, 76, 128 },
+ { 26, 66, 116 },
+ { 12, 52, 94 },
+ { 2, 32, 55 },
+ { 1, 10, 16 },
+ },
+ { /* Coeff Band 2 */
+ { 50, 127, 154 },
+ { 37, 109, 152 },
+ { 16, 82, 121 },
+ { 5, 59, 85 },
+ { 1, 35, 54 },
+ { 1, 13, 20 },
+ },
+ { /* Coeff Band 3 */
+ { 40, 142, 167 },
+ { 17, 110, 157 },
+ { 2, 71, 112 },
+ { 1, 44, 72 },
+ { 1, 27, 45 },
+ { 1, 11, 17 },
+ },
+ { /* Coeff Band 4 */
+ { 30, 175, 188 },
+ { 9, 124, 169 },
+ { 1, 74, 116 },
+ { 1, 48, 78 },
+ { 1, 30, 49 },
+ { 1, 11, 18 },
+ },
+ { /* Coeff Band 5 */
+ { 10, 222, 223 },
+ { 2, 150, 194 },
+ { 1, 83, 128 },
+ { 1, 48, 79 },
+ { 1, 27, 45 },
+ { 1, 11, 17 },
+ },
+ },
+ { /* Inter */
+ { /* Coeff Band 0 */
+ { 36, 41, 235 },
+ { 29, 36, 193 },
+ { 10, 27, 111 },
+ { 0, 0, 0 },
+ { 0, 0, 0 },
+ { 0, 0, 0 },
+ },
+ { /* Coeff Band 1 */
+ { 85, 165, 222 },
+ { 177, 162, 215 },
+ { 110, 135, 195 },
+ { 57, 113, 168 },
+ { 23, 83, 120 },
+ { 10, 49, 61 },
+ },
+ { /* Coeff Band 2 */
+ { 85, 190, 223 },
+ { 36, 139, 200 },
+ { 5, 90, 146 },
+ { 1, 60, 103 },
+ { 1, 38, 65 },
+ { 1, 18, 30 },
+ },
+ { /* Coeff Band 3 */
+ { 72, 202, 223 },
+ { 23, 141, 199 },
+ { 2, 86, 140 },
+ { 1, 56, 97 },
+ { 1, 36, 61 },
+ { 1, 16, 27 },
+ },
+ { /* Coeff Band 4 */
+ { 55, 218, 225 },
+ { 13, 145, 200 },
+ { 1, 86, 141 },
+ { 1, 57, 99 },
+ { 1, 35, 61 },
+ { 1, 13, 22 },
+ },
+ { /* Coeff Band 5 */
+ { 15, 235, 212 },
+ { 1, 132, 184 },
+ { 1, 84, 139 },
+ { 1, 57, 97 },
+ { 1, 34, 56 },
+ { 1, 14, 23 },
+ },
+ },
+ },
+ { /* block Type 1 */
+ { /* Intra */
+ { /* Coeff Band 0 */
+ { 181, 21, 201 },
+ { 61, 37, 123 },
+ { 10, 38, 71 },
+ { 0, 0, 0 },
+ { 0, 0, 0 },
+ { 0, 0, 0 },
+ },
+ { /* Coeff Band 1 */
+ { 47, 106, 172 },
+ { 95, 104, 173 },
+ { 42, 93, 159 },
+ { 18, 77, 131 },
+ { 4, 50, 81 },
+ { 1, 17, 23 },
+ },
+ { /* Coeff Band 2 */
+ { 62, 147, 199 },
+ { 44, 130, 189 },
+ { 28, 102, 154 },
+ { 18, 75, 115 },
+ { 2, 44, 65 },
+ { 1, 12, 19 },
+ },
+ { /* Coeff Band 3 */
+ { 55, 153, 210 },
+ { 24, 130, 194 },
+ { 3, 93, 146 },
+ { 1, 61, 97 },
+ { 1, 31, 50 },
+ { 1, 10, 16 },
+ },
+ { /* Coeff Band 4 */
+ { 49, 186, 223 },
+ { 17, 148, 204 },
+ { 1, 96, 142 },
+ { 1, 53, 83 },
+ { 1, 26, 44 },
+ { 1, 11, 17 },
+ },
+ { /* Coeff Band 5 */
+ { 13, 217, 212 },
+ { 2, 136, 180 },
+ { 1, 78, 124 },
+ { 1, 50, 83 },
+ { 1, 29, 49 },
+ { 1, 14, 23 },
+ },
+ },
+ { /* Inter */
+ { /* Coeff Band 0 */
+ { 197, 13, 247 },
+ { 82, 17, 222 },
+ { 25, 17, 162 },
+ { 0, 0, 0 },
+ { 0, 0, 0 },
+ { 0, 0, 0 },
+ },
+ { /* Coeff Band 1 */
+ { 126, 186, 247 },
+ { 234, 191, 243 },
+ { 176, 177, 234 },
+ { 104, 158, 220 },
+ { 66, 128, 186 },
+ { 55, 90, 137 },
+ },
+ { /* Coeff Band 2 */
+ { 111, 197, 242 },
+ { 46, 158, 219 },
+ { 9, 104, 171 },
+ { 2, 65, 125 },
+ { 1, 44, 80 },
+ { 1, 17, 91 },
+ },
+ { /* Coeff Band 3 */
+ { 104, 208, 245 },
+ { 39, 168, 224 },
+ { 3, 109, 162 },
+ { 1, 79, 124 },
+ { 1, 50, 102 },
+ { 1, 43, 102 },
+ },
+ { /* Coeff Band 4 */
+ { 84, 220, 246 },
+ { 31, 177, 231 },
+ { 2, 115, 180 },
+ { 1, 79, 134 },
+ { 1, 55, 77 },
+ { 1, 60, 79 },
+ },
+ { /* Coeff Band 5 */
+ { 43, 243, 240 },
+ { 8, 180, 217 },
+ { 1, 115, 166 },
+ { 1, 84, 121 },
+ { 1, 51, 67 },
+ { 1, 16, 6 },
+ },
+ },
+ },
+ },
+ },
+
+ .skip = { 192, 128, 64 },
+ .inter_mode = {
+ { 2, 173, 34 },
+ { 7, 145, 85 },
+ { 7, 166, 63 },
+ { 7, 94, 66 },
+ { 8, 64, 46 },
+ { 17, 81, 31 },
+ { 25, 29, 30 },
+ },
+ .interp_filter = {
+ { 235, 162 },
+ { 36, 255 },
+ { 34, 3 },
+ { 149, 144 },
+ },
+ .is_inter = { 9, 102, 187, 225 },
+ .comp_mode = { 239, 183, 119, 96, 41 },
+ .single_ref = {
+ { 33, 16 },
+ { 77, 74 },
+ { 142, 142 },
+ { 172, 170 },
+ { 238, 247 },
+ },
+ .comp_ref = { 50, 126, 123, 221, 226 },
+ .y_mode = {
+ { 65, 32, 18, 144, 162, 194, 41, 51, 98 },
+ { 132, 68, 18, 165, 217, 196, 45, 40, 78 },
+ { 173, 80, 19, 176, 240, 193, 64, 35, 46 },
+ { 221, 135, 38, 194, 248, 121, 96, 85, 29 },
+ },
+ .uv_mode = {
+ { 120, 7, 76, 176, 208, 126, 28, 54, 103 } /* y = dc */,
+ { 48, 12, 154, 155, 139, 90, 34, 117, 119 } /* y = v */,
+ { 67, 6, 25, 204, 243, 158, 13, 21, 96 } /* y = h */,
+ { 97, 5, 44, 131, 176, 139, 48, 68, 97 } /* y = d45 */,
+ { 83, 5, 42, 156, 111, 152, 26, 49, 152 } /* y = d135 */,
+ { 80, 5, 58, 178, 74, 83, 33, 62, 145 } /* y = d117 */,
+ { 86, 5, 32, 154, 192, 168, 14, 22, 163 } /* y = d153 */,
+ { 85, 5, 32, 156, 216, 148, 19, 29, 73 } /* y = d207 */,
+ { 77, 7, 64, 116, 132, 122, 37, 126, 120 } /* y = d63 */,
+ { 101, 21, 107, 181, 192, 103, 19, 67, 125 } /* y = tm */
+ },
+ .partition = {
+ /* 8x8 -> 4x4 */
+ { 199, 122, 141 } /* a/l both not split */,
+ { 147, 63, 159 } /* a split, l not split */,
+ { 148, 133, 118 } /* l split, a not split */,
+ { 121, 104, 114 } /* a/l both split */,
+ /* 16x16 -> 8x8 */
+ { 174, 73, 87 } /* a/l both not split */,
+ { 92, 41, 83 } /* a split, l not split */,
+ { 82, 99, 50 } /* l split, a not split */,
+ { 53, 39, 39 } /* a/l both split */,
+ /* 32x32 -> 16x16 */
+ { 177, 58, 59 } /* a/l both not split */,
+ { 68, 26, 63 } /* a split, l not split */,
+ { 52, 79, 25 } /* l split, a not split */,
+ { 17, 14, 12 } /* a/l both split */,
+ /* 64x64 -> 32x32 */
+ { 222, 34, 30 } /* a/l both not split */,
+ { 72, 16, 44 } /* a split, l not split */,
+ { 58, 32, 12 } /* l split, a not split */,
+ { 10, 7, 6 } /* a/l both split */,
+ },
+
+ .mv = {
+ .joint = { 32, 64, 96 },
+ .sign = { 128, 128 },
+ .classes = {
+ { 224, 144, 192, 168, 192, 176, 192, 198, 198, 245 },
+ { 216, 128, 176, 160, 176, 176, 192, 198, 198, 208 },
+ },
+ .class0_bit = { 216, 208 },
+ .bits = {
+ { 136, 140, 148, 160, 176, 192, 224, 234, 234, 240},
+ { 136, 140, 148, 160, 176, 192, 224, 234, 234, 240},
+ },
+ .class0_fr = {
+ {
+ { 128, 128, 64 },
+ { 96, 112, 64 },
+ },
+ {
+ { 128, 128, 64 },
+ { 96, 112, 64 },
+ },
+ },
+ .fr = {
+ { 64, 96, 64 },
+ { 64, 96, 64 },
+ },
+ .class0_hp = { 160, 160 },
+ .hp = { 128, 128 },
+ },
+};
+EXPORT_SYMBOL_GPL(v4l2_vp9_default_probs);
+
+static u32 fastdiv(u32 dividend, u16 divisor)
+{
+#define DIV_INV(d) ((u32)(((1ULL << 32) + ((d) - 1)) / (d)))
+#define DIVS_INV(d0, d1, d2, d3, d4, d5, d6, d7, d8, d9) \
+ DIV_INV(d0), DIV_INV(d1), DIV_INV(d2), DIV_INV(d3), \
+ DIV_INV(d4), DIV_INV(d5), DIV_INV(d6), DIV_INV(d7), \
+ DIV_INV(d8), DIV_INV(d9)
+
+ static const u32 inv[] = {
+ DIV_INV(2), DIV_INV(3), DIV_INV(4), DIV_INV(5),
+ DIV_INV(6), DIV_INV(7), DIV_INV(8), DIV_INV(9),
+ DIVS_INV(10, 11, 12, 13, 14, 15, 16, 17, 18, 19),
+ DIVS_INV(20, 21, 22, 23, 24, 25, 26, 27, 28, 29),
+ DIVS_INV(30, 31, 32, 33, 34, 35, 36, 37, 38, 39),
+ DIVS_INV(40, 41, 42, 43, 44, 45, 46, 47, 48, 49),
+ DIVS_INV(50, 51, 52, 53, 54, 55, 56, 57, 58, 59),
+ DIVS_INV(60, 61, 62, 63, 64, 65, 66, 67, 68, 69),
+ DIVS_INV(70, 71, 72, 73, 74, 75, 76, 77, 78, 79),
+ DIVS_INV(80, 81, 82, 83, 84, 85, 86, 87, 88, 89),
+ DIVS_INV(90, 91, 92, 93, 94, 95, 96, 97, 98, 99),
+ DIVS_INV(100, 101, 102, 103, 104, 105, 106, 107, 108, 109),
+ DIVS_INV(110, 111, 112, 113, 114, 115, 116, 117, 118, 119),
+ DIVS_INV(120, 121, 122, 123, 124, 125, 126, 127, 128, 129),
+ DIVS_INV(130, 131, 132, 133, 134, 135, 136, 137, 138, 139),
+ DIVS_INV(140, 141, 142, 143, 144, 145, 146, 147, 148, 149),
+ DIVS_INV(150, 151, 152, 153, 154, 155, 156, 157, 158, 159),
+ DIVS_INV(160, 161, 162, 163, 164, 165, 166, 167, 168, 169),
+ DIVS_INV(170, 171, 172, 173, 174, 175, 176, 177, 178, 179),
+ DIVS_INV(180, 181, 182, 183, 184, 185, 186, 187, 188, 189),
+ DIVS_INV(190, 191, 192, 193, 194, 195, 196, 197, 198, 199),
+ DIVS_INV(200, 201, 202, 203, 204, 205, 206, 207, 208, 209),
+ DIVS_INV(210, 211, 212, 213, 214, 215, 216, 217, 218, 219),
+ DIVS_INV(220, 221, 222, 223, 224, 225, 226, 227, 228, 229),
+ DIVS_INV(230, 231, 232, 233, 234, 235, 236, 237, 238, 239),
+ DIVS_INV(240, 241, 242, 243, 244, 245, 246, 247, 248, 249),
+ DIV_INV(250), DIV_INV(251), DIV_INV(252), DIV_INV(253),
+ DIV_INV(254), DIV_INV(255), DIV_INV(256),
+ };
+
+ if (divisor == 0)
+ return 0;
+ else if (divisor == 1)
+ return dividend;
+
+ if (WARN_ON(divisor - 2 >= ARRAY_SIZE(inv)))
+ return dividend;
+
+ return ((u64)dividend * inv[divisor - 2]) >> 32;
+}
+
+/* 6.3.6 inv_recenter_nonneg(v, m) */
+static int inv_recenter_nonneg(int v, int m)
+{
+ if (v > 2 * m)
+ return v;
+
+ if (v & 1)
+ return m - ((v + 1) >> 1);
+
+ return m + (v >> 1);
+}
+
+/*
+ * part of 6.3.5 inv_remap_prob(deltaProb, prob)
+ * delta = inv_map_table[deltaProb] done by userspace
+ */
+static int update_prob(int delta, int prob)
+{
+ if (!delta)
+ return prob;
+
+ return prob <= 128 ?
+ 1 + inv_recenter_nonneg(delta, prob - 1) :
+ 255 - inv_recenter_nonneg(delta, 255 - prob);
+}
+
+/* Counterpart to 6.3.2 tx_mode_probs() */
+static void update_tx_probs(struct v4l2_vp9_frame_context *probs,
+ const struct v4l2_ctrl_vp9_compressed_hdr *deltas)
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(probs->tx8); i++) {
+ u8 *p8x8 = probs->tx8[i];
+ u8 *p16x16 = probs->tx16[i];
+ u8 *p32x32 = probs->tx32[i];
+ const u8 *d8x8 = deltas->tx8[i];
+ const u8 *d16x16 = deltas->tx16[i];
+ const u8 *d32x32 = deltas->tx32[i];
+
+ p8x8[0] = update_prob(d8x8[0], p8x8[0]);
+ p16x16[0] = update_prob(d16x16[0], p16x16[0]);
+ p16x16[1] = update_prob(d16x16[1], p16x16[1]);
+ p32x32[0] = update_prob(d32x32[0], p32x32[0]);
+ p32x32[1] = update_prob(d32x32[1], p32x32[1]);
+ p32x32[2] = update_prob(d32x32[2], p32x32[2]);
+ }
+}
+
+#define BAND_6(band) ((band) == 0 ? 3 : 6)
+
+static void update_coeff(const u8 deltas[6][6][3], u8 probs[6][6][3])
+{
+ int l, m, n;
+
+ for (l = 0; l < 6; l++)
+ for (m = 0; m < BAND_6(l); m++) {
+ u8 *p = probs[l][m];
+ const u8 *d = deltas[l][m];
+
+ for (n = 0; n < 3; n++)
+ p[n] = update_prob(d[n], p[n]);
+ }
+}
+
+/* Counterpart to 6.3.7 read_coef_probs() */
+static void update_coef_probs(struct v4l2_vp9_frame_context *probs,
+ const struct v4l2_ctrl_vp9_compressed_hdr *deltas,
+ const struct v4l2_ctrl_vp9_frame *dec_params)
+{
+ int i, j, k;
+
+ for (i = 0; i < ARRAY_SIZE(probs->coef); i++) {
+ for (j = 0; j < ARRAY_SIZE(probs->coef[0]); j++)
+ for (k = 0; k < ARRAY_SIZE(probs->coef[0][0]); k++)
+ update_coeff(deltas->coef[i][j][k], probs->coef[i][j][k]);
+
+ if (deltas->tx_mode == i)
+ break;
+ }
+}
+
+/* Counterpart to 6.3.8 read_skip_prob() */
+static void update_skip_probs(struct v4l2_vp9_frame_context *probs,
+ const struct v4l2_ctrl_vp9_compressed_hdr *deltas)
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(probs->skip); i++)
+ probs->skip[i] = update_prob(deltas->skip[i], probs->skip[i]);
+}
+
+/* Counterpart to 6.3.9 read_inter_mode_probs() */
+static void update_inter_mode_probs(struct v4l2_vp9_frame_context *probs,
+ const struct v4l2_ctrl_vp9_compressed_hdr *deltas)
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(probs->inter_mode); i++) {
+ u8 *p = probs->inter_mode[i];
+ const u8 *d = deltas->inter_mode[i];
+
+ p[0] = update_prob(d[0], p[0]);
+ p[1] = update_prob(d[1], p[1]);
+ p[2] = update_prob(d[2], p[2]);
+ }
+}
+
+/* Counterpart to 6.3.10 read_interp_filter_probs() */
+static void update_interp_filter_probs(struct v4l2_vp9_frame_context *probs,
+ const struct v4l2_ctrl_vp9_compressed_hdr *deltas)
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(probs->interp_filter); i++) {
+ u8 *p = probs->interp_filter[i];
+ const u8 *d = deltas->interp_filter[i];
+
+ p[0] = update_prob(d[0], p[0]);
+ p[1] = update_prob(d[1], p[1]);
+ }
+}
+
+/* Counterpart to 6.3.11 read_is_inter_probs() */
+static void update_is_inter_probs(struct v4l2_vp9_frame_context *probs,
+ const struct v4l2_ctrl_vp9_compressed_hdr *deltas)
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(probs->is_inter); i++)
+ probs->is_inter[i] = update_prob(deltas->is_inter[i], probs->is_inter[i]);
+}
+
+/* 6.3.12 frame_reference_mode() done entirely in userspace */
+
+/* Counterpart to 6.3.13 frame_reference_mode_probs() */
+static void
+update_frame_reference_mode_probs(unsigned int reference_mode,
+ struct v4l2_vp9_frame_context *probs,
+ const struct v4l2_ctrl_vp9_compressed_hdr *deltas)
+{
+ int i;
+
+ if (reference_mode == V4L2_VP9_REFERENCE_MODE_SELECT)
+ for (i = 0; i < ARRAY_SIZE(probs->comp_mode); i++)
+ probs->comp_mode[i] = update_prob(deltas->comp_mode[i],
+ probs->comp_mode[i]);
+
+ if (reference_mode != V4L2_VP9_REFERENCE_MODE_COMPOUND_REFERENCE)
+ for (i = 0; i < ARRAY_SIZE(probs->single_ref); i++) {
+ u8 *p = probs->single_ref[i];
+ const u8 *d = deltas->single_ref[i];
+
+ p[0] = update_prob(d[0], p[0]);
+ p[1] = update_prob(d[1], p[1]);
+ }
+
+ if (reference_mode != V4L2_VP9_REFERENCE_MODE_SINGLE_REFERENCE)
+ for (i = 0; i < ARRAY_SIZE(probs->comp_ref); i++)
+ probs->comp_ref[i] = update_prob(deltas->comp_ref[i], probs->comp_ref[i]);
+}
+
+/* Counterpart to 6.3.14 read_y_mode_probs() */
+static void update_y_mode_probs(struct v4l2_vp9_frame_context *probs,
+ const struct v4l2_ctrl_vp9_compressed_hdr *deltas)
+{
+ int i, j;
+
+ for (i = 0; i < ARRAY_SIZE(probs->y_mode); i++)
+ for (j = 0; j < ARRAY_SIZE(probs->y_mode[0]); ++j)
+ probs->y_mode[i][j] =
+ update_prob(deltas->y_mode[i][j], probs->y_mode[i][j]);
+}
+
+/* Counterpart to 6.3.15 read_partition_probs() */
+static void update_partition_probs(struct v4l2_vp9_frame_context *probs,
+ const struct v4l2_ctrl_vp9_compressed_hdr *deltas)
+{
+ int i, j;
+
+ for (i = 0; i < 4; i++)
+ for (j = 0; j < 4; j++) {
+ u8 *p = probs->partition[i * 4 + j];
+ const u8 *d = deltas->partition[i * 4 + j];
+
+ p[0] = update_prob(d[0], p[0]);
+ p[1] = update_prob(d[1], p[1]);
+ p[2] = update_prob(d[2], p[2]);
+ }
+}
+
+static inline int update_mv_prob(int delta, int prob)
+{
+ if (!delta)
+ return prob;
+
+ return delta;
+}
+
+/* Counterpart to 6.3.16 mv_probs() */
+static void update_mv_probs(struct v4l2_vp9_frame_context *probs,
+ const struct v4l2_ctrl_vp9_compressed_hdr *deltas,
+ const struct v4l2_ctrl_vp9_frame *dec_params)
+{
+ u8 *p = probs->mv.joint;
+ const u8 *d = deltas->mv.joint;
+ unsigned int i, j;
+
+ p[0] = update_mv_prob(d[0], p[0]);
+ p[1] = update_mv_prob(d[1], p[1]);
+ p[2] = update_mv_prob(d[2], p[2]);
+
+ for (i = 0; i < ARRAY_SIZE(probs->mv.sign); i++) {
+ p = probs->mv.sign;
+ d = deltas->mv.sign;
+ p[i] = update_mv_prob(d[i], p[i]);
+
+ p = probs->mv.classes[i];
+ d = deltas->mv.classes[i];
+ for (j = 0; j < ARRAY_SIZE(probs->mv.classes[0]); j++)
+ p[j] = update_mv_prob(d[j], p[j]);
+
+ p = probs->mv.class0_bit;
+ d = deltas->mv.class0_bit;
+ p[i] = update_mv_prob(d[i], p[i]);
+
+ p = probs->mv.bits[i];
+ d = deltas->mv.bits[i];
+ for (j = 0; j < ARRAY_SIZE(probs->mv.bits[0]); j++)
+ p[j] = update_mv_prob(d[j], p[j]);
+
+ for (j = 0; j < ARRAY_SIZE(probs->mv.class0_fr[0]); j++) {
+ p = probs->mv.class0_fr[i][j];
+ d = deltas->mv.class0_fr[i][j];
+
+ p[0] = update_mv_prob(d[0], p[0]);
+ p[1] = update_mv_prob(d[1], p[1]);
+ p[2] = update_mv_prob(d[2], p[2]);
+ }
+
+ p = probs->mv.fr[i];
+ d = deltas->mv.fr[i];
+ for (j = 0; j < ARRAY_SIZE(probs->mv.fr[i]); j++)
+ p[j] = update_mv_prob(d[j], p[j]);
+
+ if (dec_params->flags & V4L2_VP9_FRAME_FLAG_ALLOW_HIGH_PREC_MV) {
+ p = probs->mv.class0_hp;
+ d = deltas->mv.class0_hp;
+ p[i] = update_mv_prob(d[i], p[i]);
+
+ p = probs->mv.hp;
+ d = deltas->mv.hp;
+ p[i] = update_mv_prob(d[i], p[i]);
+ }
+ }
+}
+
+/* Counterpart to 6.3 compressed_header(), but parsing has been done in userspace. */
+void v4l2_vp9_fw_update_probs(struct v4l2_vp9_frame_context *probs,
+ const struct v4l2_ctrl_vp9_compressed_hdr *deltas,
+ const struct v4l2_ctrl_vp9_frame *dec_params)
+{
+ if (deltas->tx_mode == V4L2_VP9_TX_MODE_SELECT)
+ update_tx_probs(probs, deltas);
+
+ update_coef_probs(probs, deltas, dec_params);
+
+ update_skip_probs(probs, deltas);
+
+ if (dec_params->flags & V4L2_VP9_FRAME_FLAG_KEY_FRAME ||
+ dec_params->flags & V4L2_VP9_FRAME_FLAG_INTRA_ONLY)
+ return;
+
+ update_inter_mode_probs(probs, deltas);
+
+ if (dec_params->interpolation_filter == V4L2_VP9_INTERP_FILTER_SWITCHABLE)
+ update_interp_filter_probs(probs, deltas);
+
+ update_is_inter_probs(probs, deltas);
+
+ update_frame_reference_mode_probs(dec_params->reference_mode, probs, deltas);
+
+ update_y_mode_probs(probs, deltas);
+
+ update_partition_probs(probs, deltas);
+
+ update_mv_probs(probs, deltas, dec_params);
+}
+EXPORT_SYMBOL_GPL(v4l2_vp9_fw_update_probs);
+
+u8 v4l2_vp9_reset_frame_ctx(const struct v4l2_ctrl_vp9_frame *dec_params,
+ struct v4l2_vp9_frame_context *frame_context)
+{
+ int i;
+
+ u8 fctx_idx = dec_params->frame_context_idx;
+
+ if (dec_params->flags & V4L2_VP9_FRAME_FLAG_KEY_FRAME ||
+ dec_params->flags & V4L2_VP9_FRAME_FLAG_INTRA_ONLY ||
+ dec_params->flags & V4L2_VP9_FRAME_FLAG_ERROR_RESILIENT) {
+ /*
+ * setup_past_independence()
+ * We do nothing here. Instead of storing default probs in some intermediate
+ * location and then copying from that location to appropriate contexts
+ * in save_probs() below, we skip that step and save default probs directly
+ * to appropriate contexts.
+ */
+ if (dec_params->flags & V4L2_VP9_FRAME_FLAG_KEY_FRAME ||
+ dec_params->flags & V4L2_VP9_FRAME_FLAG_ERROR_RESILIENT ||
+ dec_params->reset_frame_context == V4L2_VP9_RESET_FRAME_CTX_ALL)
+ for (i = 0; i < 4; ++i)
+ /* save_probs(i) */
+ memcpy(&frame_context[i], &v4l2_vp9_default_probs,
+ sizeof(v4l2_vp9_default_probs));
+ else if (dec_params->reset_frame_context == V4L2_VP9_RESET_FRAME_CTX_SPEC)
+ /* save_probs(fctx_idx) */
+ memcpy(&frame_context[fctx_idx], &v4l2_vp9_default_probs,
+ sizeof(v4l2_vp9_default_probs));
+ fctx_idx = 0;
+ }
+
+ return fctx_idx;
+}
+EXPORT_SYMBOL_GPL(v4l2_vp9_reset_frame_ctx);
+
+/* 8.4.1 Merge prob process */
+static u8 merge_prob(u8 pre_prob, u32 ct0, u32 ct1, u16 count_sat, u32 max_update_factor)
+{
+ u32 den, prob, count, factor;
+
+ den = ct0 + ct1;
+ if (!den) {
+ /*
+ * prob = 128, count = 0, update_factor = 0
+ * Round2's argument: pre_prob * 256
+ * (pre_prob * 256 + 128) >> 8 == pre_prob
+ */
+ return pre_prob;
+ }
+
+ prob = clamp(((ct0 << 8) + (den >> 1)) / den, (u32)1, (u32)255);
+ count = min_t(u32, den, count_sat);
+ factor = fastdiv(max_update_factor * count, count_sat);
+
+ /*
+ * Round2(pre_prob * (256 - factor) + prob * factor, 8)
+ * Round2(pre_prob * 256 + (prob - pre_prob) * factor, 8)
+ * (pre_prob * 256 >> 8) + (((prob - pre_prob) * factor + 128) >> 8)
+ */
+ return pre_prob + (((prob - pre_prob) * factor + 128) >> 8);
+}
+
+static inline u8 noncoef_merge_prob(u8 pre_prob, u32 ct0, u32 ct1)
+{
+ return merge_prob(pre_prob, ct0, ct1, 20, 128);
+}
+
+/* 8.4.2 Merge probs process */
+/*
+ * merge_probs() is a recursive function in the spec. We avoid recursion in the kernel.
+ * That said, the "tree" parameter of merge_probs() controls how deep the recursion goes.
+ * It turns out that in all cases the recursive calls boil down to a short-ish series
+ * of merge_prob() invocations (note no "s").
+ *
+ * Variant A
+ * ---------
+ * merge_probs(small_token_tree, 2):
+ * merge_prob(p[1], c[0], c[1] + c[2])
+ * merge_prob(p[2], c[1], c[2])
+ *
+ * Variant B
+ * ---------
+ * merge_probs(binary_tree, 0) or
+ * merge_probs(tx_size_8_tree, 0):
+ * merge_prob(p[0], c[0], c[1])
+ *
+ * Variant C
+ * ---------
+ * merge_probs(inter_mode_tree, 0):
+ * merge_prob(p[0], c[2], c[1] + c[0] + c[3])
+ * merge_prob(p[1], c[0], c[1] + c[3])
+ * merge_prob(p[2], c[1], c[3])
+ *
+ * Variant D
+ * ---------
+ * merge_probs(intra_mode_tree, 0):
+ * merge_prob(p[0], c[0], c[1] + ... + c[9])
+ * merge_prob(p[1], c[9], c[1] + ... + c[8])
+ * merge_prob(p[2], c[1], c[2] + ... + c[8])
+ * merge_prob(p[3], c[2] + c[4] + c[5], c[3] + c[8] + c[6] + c[7])
+ * merge_prob(p[4], c[2], c[4] + c[5])
+ * merge_prob(p[5], c[4], c[5])
+ * merge_prob(p[6], c[3], c[8] + c[6] + c[7])
+ * merge_prob(p[7], c[8], c[6] + c[7])
+ * merge_prob(p[8], c[6], c[7])
+ *
+ * Variant E
+ * ---------
+ * merge_probs(partition_tree, 0) or
+ * merge_probs(tx_size_32_tree, 0) or
+ * merge_probs(mv_joint_tree, 0) or
+ * merge_probs(mv_fr_tree, 0):
+ * merge_prob(p[0], c[0], c[1] + c[2] + c[3])
+ * merge_prob(p[1], c[1], c[2] + c[3])
+ * merge_prob(p[2], c[2], c[3])
+ *
+ * Variant F
+ * ---------
+ * merge_probs(interp_filter_tree, 0) or
+ * merge_probs(tx_size_16_tree, 0):
+ * merge_prob(p[0], c[0], c[1] + c[2])
+ * merge_prob(p[1], c[1], c[2])
+ *
+ * Variant G
+ * ---------
+ * merge_probs(mv_class_tree, 0):
+ * merge_prob(p[0], c[0], c[1] + ... + c[10])
+ * merge_prob(p[1], c[1], c[2] + ... + c[10])
+ * merge_prob(p[2], c[2] + c[3], c[4] + ... + c[10])
+ * merge_prob(p[3], c[2], c[3])
+ * merge_prob(p[4], c[4] + c[5], c[6] + ... + c[10])
+ * merge_prob(p[5], c[4], c[5])
+ * merge_prob(p[6], c[6], c[7] + ... + c[10])
+ * merge_prob(p[7], c[7] + c[8], c[9] + c[10])
+ * merge_prob(p[8], c[7], c[8])
+ * merge_prob(p[9], c[9], [10])
+ */
+
+static inline void merge_probs_variant_a(u8 *p, const u32 *c, u16 count_sat, u32 update_factor)
+{
+ p[1] = merge_prob(p[1], c[0], c[1] + c[2], count_sat, update_factor);
+ p[2] = merge_prob(p[2], c[1], c[2], count_sat, update_factor);
+}
+
+static inline void merge_probs_variant_b(u8 *p, const u32 *c, u16 count_sat, u32 update_factor)
+{
+ p[0] = merge_prob(p[0], c[0], c[1], count_sat, update_factor);
+}
+
+static inline void merge_probs_variant_c(u8 *p, const u32 *c)
+{
+ p[0] = noncoef_merge_prob(p[0], c[2], c[1] + c[0] + c[3]);
+ p[1] = noncoef_merge_prob(p[1], c[0], c[1] + c[3]);
+ p[2] = noncoef_merge_prob(p[2], c[1], c[3]);
+}
+
+static void merge_probs_variant_d(u8 *p, const u32 *c)
+{
+ u32 sum = 0, s2;
+
+ sum = c[1] + c[2] + c[3] + c[4] + c[5] + c[6] + c[7] + c[8] + c[9];
+
+ p[0] = noncoef_merge_prob(p[0], c[0], sum);
+ sum -= c[9];
+ p[1] = noncoef_merge_prob(p[1], c[9], sum);
+ sum -= c[1];
+ p[2] = noncoef_merge_prob(p[2], c[1], sum);
+ s2 = c[2] + c[4] + c[5];
+ sum -= s2;
+ p[3] = noncoef_merge_prob(p[3], s2, sum);
+ s2 -= c[2];
+ p[4] = noncoef_merge_prob(p[4], c[2], s2);
+ p[5] = noncoef_merge_prob(p[5], c[4], c[5]);
+ sum -= c[3];
+ p[6] = noncoef_merge_prob(p[6], c[3], sum);
+ sum -= c[8];
+ p[7] = noncoef_merge_prob(p[7], c[8], sum);
+ p[8] = noncoef_merge_prob(p[8], c[6], c[7]);
+}
+
+static inline void merge_probs_variant_e(u8 *p, const u32 *c)
+{
+ p[0] = noncoef_merge_prob(p[0], c[0], c[1] + c[2] + c[3]);
+ p[1] = noncoef_merge_prob(p[1], c[1], c[2] + c[3]);
+ p[2] = noncoef_merge_prob(p[2], c[2], c[3]);
+}
+
+static inline void merge_probs_variant_f(u8 *p, const u32 *c)
+{
+ p[0] = noncoef_merge_prob(p[0], c[0], c[1] + c[2]);
+ p[1] = noncoef_merge_prob(p[1], c[1], c[2]);
+}
+
+static void merge_probs_variant_g(u8 *p, const u32 *c)
+{
+ u32 sum;
+
+ sum = c[1] + c[2] + c[3] + c[4] + c[5] + c[6] + c[7] + c[8] + c[9] + c[10];
+ p[0] = noncoef_merge_prob(p[0], c[0], sum);
+ sum -= c[1];
+ p[1] = noncoef_merge_prob(p[1], c[1], sum);
+ sum -= c[2] + c[3];
+ p[2] = noncoef_merge_prob(p[2], c[2] + c[3], sum);
+ p[3] = noncoef_merge_prob(p[3], c[2], c[3]);
+ sum -= c[4] + c[5];
+ p[4] = noncoef_merge_prob(p[4], c[4] + c[5], sum);
+ p[5] = noncoef_merge_prob(p[5], c[4], c[5]);
+ sum -= c[6];
+ p[6] = noncoef_merge_prob(p[6], c[6], sum);
+ p[7] = noncoef_merge_prob(p[7], c[7] + c[8], c[9] + c[10]);
+ p[8] = noncoef_merge_prob(p[8], c[7], c[8]);
+ p[9] = noncoef_merge_prob(p[9], c[9], c[10]);
+}
+
+/* 8.4.3 Coefficient probability adaptation process */
+static inline void adapt_probs_variant_a_coef(u8 *p, const u32 *c, u32 update_factor)
+{
+ merge_probs_variant_a(p, c, 24, update_factor);
+}
+
+static inline void adapt_probs_variant_b_coef(u8 *p, const u32 *c, u32 update_factor)
+{
+ merge_probs_variant_b(p, c, 24, update_factor);
+}
+
+static void _adapt_coeff(unsigned int i, unsigned int j, unsigned int k,
+ struct v4l2_vp9_frame_context *probs,
+ const struct v4l2_vp9_frame_symbol_counts *counts,
+ u32 uf)
+{
+ s32 l, m;
+
+ for (l = 0; l < ARRAY_SIZE(probs->coef[0][0][0]); l++) {
+ for (m = 0; m < BAND_6(l); m++) {
+ u8 *p = probs->coef[i][j][k][l][m];
+ const u32 counts_more_coefs[2] = {
+ *counts->eob[i][j][k][l][m][1],
+ *counts->eob[i][j][k][l][m][0] - *counts->eob[i][j][k][l][m][1],
+ };
+
+ adapt_probs_variant_a_coef(p, *counts->coeff[i][j][k][l][m], uf);
+ adapt_probs_variant_b_coef(p, counts_more_coefs, uf);
+ }
+ }
+}
+
+static void _adapt_coef_probs(struct v4l2_vp9_frame_context *probs,
+ const struct v4l2_vp9_frame_symbol_counts *counts,
+ unsigned int uf)
+{
+ unsigned int i, j, k;
+
+ for (i = 0; i < ARRAY_SIZE(probs->coef); i++)
+ for (j = 0; j < ARRAY_SIZE(probs->coef[0]); j++)
+ for (k = 0; k < ARRAY_SIZE(probs->coef[0][0]); k++)
+ _adapt_coeff(i, j, k, probs, counts, uf);
+}
+
+void v4l2_vp9_adapt_coef_probs(struct v4l2_vp9_frame_context *probs,
+ struct v4l2_vp9_frame_symbol_counts *counts,
+ bool use_128,
+ bool frame_is_intra)
+{
+ if (frame_is_intra) {
+ _adapt_coef_probs(probs, counts, 112);
+ } else {
+ if (use_128)
+ _adapt_coef_probs(probs, counts, 128);
+ else
+ _adapt_coef_probs(probs, counts, 112);
+ }
+}
+EXPORT_SYMBOL_GPL(v4l2_vp9_adapt_coef_probs);
+
+/* 8.4.4 Non coefficient probability adaptation process, adapt_probs() */
+static inline void adapt_probs_variant_b(u8 *p, const u32 *c)
+{
+ merge_probs_variant_b(p, c, 20, 128);
+}
+
+static inline void adapt_probs_variant_c(u8 *p, const u32 *c)
+{
+ merge_probs_variant_c(p, c);
+}
+
+static inline void adapt_probs_variant_d(u8 *p, const u32 *c)
+{
+ merge_probs_variant_d(p, c);
+}
+
+static inline void adapt_probs_variant_e(u8 *p, const u32 *c)
+{
+ merge_probs_variant_e(p, c);
+}
+
+static inline void adapt_probs_variant_f(u8 *p, const u32 *c)
+{
+ merge_probs_variant_f(p, c);
+}
+
+static inline void adapt_probs_variant_g(u8 *p, const u32 *c)
+{
+ merge_probs_variant_g(p, c);
+}
+
+/* 8.4.4 Non coefficient probability adaptation process, adapt_prob() */
+static inline u8 adapt_prob(u8 prob, const u32 counts[2])
+{
+ return noncoef_merge_prob(prob, counts[0], counts[1]);
+}
+
+/* 8.4.4 Non coefficient probability adaptation process */
+void v4l2_vp9_adapt_noncoef_probs(struct v4l2_vp9_frame_context *probs,
+ struct v4l2_vp9_frame_symbol_counts *counts,
+ u8 reference_mode, u8 interpolation_filter, u8 tx_mode,
+ u32 flags)
+{
+ unsigned int i, j;
+
+ for (i = 0; i < ARRAY_SIZE(probs->is_inter); i++)
+ probs->is_inter[i] = adapt_prob(probs->is_inter[i], (*counts->intra_inter)[i]);
+
+ for (i = 0; i < ARRAY_SIZE(probs->comp_mode); i++)
+ probs->comp_mode[i] = adapt_prob(probs->comp_mode[i], (*counts->comp)[i]);
+
+ for (i = 0; i < ARRAY_SIZE(probs->comp_ref); i++)
+ probs->comp_ref[i] = adapt_prob(probs->comp_ref[i], (*counts->comp_ref)[i]);
+
+ if (reference_mode != V4L2_VP9_REFERENCE_MODE_COMPOUND_REFERENCE)
+ for (i = 0; i < ARRAY_SIZE(probs->single_ref); i++)
+ for (j = 0; j < ARRAY_SIZE(probs->single_ref[0]); j++)
+ probs->single_ref[i][j] = adapt_prob(probs->single_ref[i][j],
+ (*counts->single_ref)[i][j]);
+
+ for (i = 0; i < ARRAY_SIZE(probs->inter_mode); i++)
+ adapt_probs_variant_c(probs->inter_mode[i], (*counts->mv_mode)[i]);
+
+ for (i = 0; i < ARRAY_SIZE(probs->y_mode); i++)
+ adapt_probs_variant_d(probs->y_mode[i], (*counts->y_mode)[i]);
+
+ for (i = 0; i < ARRAY_SIZE(probs->uv_mode); i++)
+ adapt_probs_variant_d(probs->uv_mode[i], (*counts->uv_mode)[i]);
+
+ for (i = 0; i < ARRAY_SIZE(probs->partition); i++)
+ adapt_probs_variant_e(probs->partition[i], (*counts->partition)[i]);
+
+ for (i = 0; i < ARRAY_SIZE(probs->skip); i++)
+ probs->skip[i] = adapt_prob(probs->skip[i], (*counts->skip)[i]);
+
+ if (interpolation_filter == V4L2_VP9_INTERP_FILTER_SWITCHABLE)
+ for (i = 0; i < ARRAY_SIZE(probs->interp_filter); i++)
+ adapt_probs_variant_f(probs->interp_filter[i], (*counts->filter)[i]);
+
+ if (tx_mode == V4L2_VP9_TX_MODE_SELECT)
+ for (i = 0; i < ARRAY_SIZE(probs->tx8); i++) {
+ adapt_probs_variant_b(probs->tx8[i], (*counts->tx8p)[i]);
+ adapt_probs_variant_f(probs->tx16[i], (*counts->tx16p)[i]);
+ adapt_probs_variant_e(probs->tx32[i], (*counts->tx32p)[i]);
+ }
+
+ adapt_probs_variant_e(probs->mv.joint, *counts->mv_joint);
+
+ for (i = 0; i < ARRAY_SIZE(probs->mv.sign); i++) {
+ probs->mv.sign[i] = adapt_prob(probs->mv.sign[i], (*counts->sign)[i]);
+
+ adapt_probs_variant_g(probs->mv.classes[i], (*counts->classes)[i]);
+
+ probs->mv.class0_bit[i] = adapt_prob(probs->mv.class0_bit[i], (*counts->class0)[i]);
+
+ for (j = 0; j < ARRAY_SIZE(probs->mv.bits[0]); j++)
+ probs->mv.bits[i][j] = adapt_prob(probs->mv.bits[i][j],
+ (*counts->bits)[i][j]);
+
+ for (j = 0; j < ARRAY_SIZE(probs->mv.class0_fr[0]); j++)
+ adapt_probs_variant_e(probs->mv.class0_fr[i][j],
+ (*counts->class0_fp)[i][j]);
+
+ adapt_probs_variant_e(probs->mv.fr[i], (*counts->fp)[i]);
+
+ if (!(flags & V4L2_VP9_FRAME_FLAG_ALLOW_HIGH_PREC_MV))
+ continue;
+
+ probs->mv.class0_hp[i] = adapt_prob(probs->mv.class0_hp[i],
+ (*counts->class0_hp)[i]);
+
+ probs->mv.hp[i] = adapt_prob(probs->mv.hp[i], (*counts->hp)[i]);
+ }
+}
+EXPORT_SYMBOL_GPL(v4l2_vp9_adapt_noncoef_probs);
+
+bool
+v4l2_vp9_seg_feat_enabled(const u8 *feature_enabled,
+ unsigned int feature,
+ unsigned int segid)
+{
+ u8 mask = V4L2_VP9_SEGMENT_FEATURE_ENABLED(feature);
+
+ return !!(feature_enabled[segid] & mask);
+}
+EXPORT_SYMBOL_GPL(v4l2_vp9_seg_feat_enabled);
+
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("V4L2 VP9 Helpers");
+MODULE_AUTHOR("Andrzej Pietrasiewicz <andrzej.p@collabora.com>");
diff --git a/drivers/memory/renesas-rpc-if.c b/drivers/memory/renesas-rpc-if.c
index 7435baa..e4cc64f 100644
--- a/drivers/memory/renesas-rpc-if.c
+++ b/drivers/memory/renesas-rpc-if.c
@@ -12,6 +12,7 @@
#include <linux/module.h>
#include <linux/platform_device.h>
#include <linux/of.h>
+#include <linux/of_device.h>
#include <linux/regmap.h>
#include <linux/reset.h>
@@ -19,19 +20,17 @@
#define RPCIF_CMNCR 0x0000 /* R/W */
#define RPCIF_CMNCR_MD BIT(31)
-#define RPCIF_CMNCR_SFDE BIT(24) /* undocumented but must be set */
#define RPCIF_CMNCR_MOIIO3(val) (((val) & 0x3) << 22)
#define RPCIF_CMNCR_MOIIO2(val) (((val) & 0x3) << 20)
#define RPCIF_CMNCR_MOIIO1(val) (((val) & 0x3) << 18)
#define RPCIF_CMNCR_MOIIO0(val) (((val) & 0x3) << 16)
-#define RPCIF_CMNCR_MOIIO_HIZ (RPCIF_CMNCR_MOIIO0(3) | \
- RPCIF_CMNCR_MOIIO1(3) | \
- RPCIF_CMNCR_MOIIO2(3) | RPCIF_CMNCR_MOIIO3(3))
-#define RPCIF_CMNCR_IO3FV(val) (((val) & 0x3) << 14) /* undocumented */
-#define RPCIF_CMNCR_IO2FV(val) (((val) & 0x3) << 12) /* undocumented */
+#define RPCIF_CMNCR_MOIIO(val) (RPCIF_CMNCR_MOIIO0(val) | RPCIF_CMNCR_MOIIO1(val) | \
+ RPCIF_CMNCR_MOIIO2(val) | RPCIF_CMNCR_MOIIO3(val))
+#define RPCIF_CMNCR_IO3FV(val) (((val) & 0x3) << 14) /* documented for RZ/G2L */
+#define RPCIF_CMNCR_IO2FV(val) (((val) & 0x3) << 12) /* documented for RZ/G2L */
#define RPCIF_CMNCR_IO0FV(val) (((val) & 0x3) << 8)
-#define RPCIF_CMNCR_IOFV_HIZ (RPCIF_CMNCR_IO0FV(3) | RPCIF_CMNCR_IO2FV(3) | \
- RPCIF_CMNCR_IO3FV(3))
+#define RPCIF_CMNCR_IOFV(val) (RPCIF_CMNCR_IO0FV(val) | RPCIF_CMNCR_IO2FV(val) | \
+ RPCIF_CMNCR_IO3FV(val))
#define RPCIF_CMNCR_BSZ(val) (((val) & 0x3) << 0)
#define RPCIF_SSLDR 0x0004 /* R/W */
@@ -126,6 +125,9 @@
#define RPCIF_SMDRENR_OPDRE BIT(4)
#define RPCIF_SMDRENR_SPIDRE BIT(0)
+#define RPCIF_PHYADD 0x0070 /* R/W available on R-Car E3/D3/V3M and RZ/G2{E,L} */
+#define RPCIF_PHYWR 0x0074 /* R/W available on R-Car E3/D3/V3M and RZ/G2{E,L} */
+
#define RPCIF_PHYCNT 0x007C /* R/W */
#define RPCIF_PHYCNT_CAL BIT(31)
#define RPCIF_PHYCNT_OCTA(v) (((v) & 0x3) << 22)
@@ -133,10 +135,12 @@
#define RPCIF_PHYCNT_OCT BIT(20)
#define RPCIF_PHYCNT_DDRCAL BIT(19)
#define RPCIF_PHYCNT_HS BIT(18)
-#define RPCIF_PHYCNT_STRTIM(v) (((v) & 0x7) << 15)
+#define RPCIF_PHYCNT_CKSEL(v) (((v) & 0x3) << 16) /* valid only for RZ/G2L */
+#define RPCIF_PHYCNT_STRTIM(v) (((v) & 0x7) << 15) /* valid for R-Car and RZ/G2{E,H,M,N} */
#define RPCIF_PHYCNT_WBUF2 BIT(4)
#define RPCIF_PHYCNT_WBUF BIT(2)
#define RPCIF_PHYCNT_PHYMEM(v) (((v) & 0x3) << 0)
+#define RPCIF_PHYCNT_PHYMEM_MASK GENMASK(1, 0)
#define RPCIF_PHYOFFSET1 0x0080 /* R/W */
#define RPCIF_PHYOFFSET1_DDRTMG(v) (((v) & 0x3) << 28)
@@ -147,8 +151,6 @@
#define RPCIF_PHYINT 0x0088 /* R/W */
#define RPCIF_PHYINT_WPVAL BIT(1)
-#define RPCIF_DIRMAP_SIZE 0x4000000
-
static const struct regmap_range rpcif_volatile_ranges[] = {
regmap_reg_range(RPCIF_SMRDR0, RPCIF_SMRDR1),
regmap_reg_range(RPCIF_SMWDR0, RPCIF_SMWDR1),
@@ -243,50 +245,74 @@ int rpcif_sw_init(struct rpcif *rpc, struct device *dev)
res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "dirmap");
rpc->dirmap = devm_ioremap_resource(&pdev->dev, res);
if (IS_ERR(rpc->dirmap))
- rpc->dirmap = NULL;
+ return PTR_ERR(rpc->dirmap);
rpc->size = resource_size(res);
+ rpc->type = (uintptr_t)of_device_get_match_data(dev);
rpc->rstc = devm_reset_control_get_exclusive(&pdev->dev, NULL);
return PTR_ERR_OR_ZERO(rpc->rstc);
}
EXPORT_SYMBOL(rpcif_sw_init);
-void rpcif_hw_init(struct rpcif *rpc, bool hyperflash)
+static void rpcif_rzg2l_timing_adjust_sdr(struct rpcif *rpc)
+{
+ regmap_write(rpc->regmap, RPCIF_PHYWR, 0xa5390000);
+ regmap_write(rpc->regmap, RPCIF_PHYADD, 0x80000000);
+ regmap_write(rpc->regmap, RPCIF_PHYWR, 0x00008080);
+ regmap_write(rpc->regmap, RPCIF_PHYADD, 0x80000022);
+ regmap_write(rpc->regmap, RPCIF_PHYWR, 0x00008080);
+ regmap_write(rpc->regmap, RPCIF_PHYADD, 0x80000024);
+ regmap_update_bits(rpc->regmap, RPCIF_PHYCNT, RPCIF_PHYCNT_CKSEL(3),
+ RPCIF_PHYCNT_CKSEL(3));
+ regmap_write(rpc->regmap, RPCIF_PHYWR, 0x00000030);
+ regmap_write(rpc->regmap, RPCIF_PHYADD, 0x80000032);
+}
+
+int rpcif_hw_init(struct rpcif *rpc, bool hyperflash)
{
u32 dummy;
pm_runtime_get_sync(rpc->dev);
- /*
- * NOTE: The 0x260 are undocumented bits, but they must be set.
- * RPCIF_PHYCNT_STRTIM is strobe timing adjustment bits,
- * 0x0 : the delay is biggest,
- * 0x1 : the delay is 2nd biggest,
- * On H3 ES1.x, the value should be 0, while on others,
- * the value should be 7.
- */
- regmap_write(rpc->regmap, RPCIF_PHYCNT, RPCIF_PHYCNT_STRTIM(7) |
- RPCIF_PHYCNT_PHYMEM(hyperflash ? 3 : 0) | 0x260);
+ if (rpc->type == RPCIF_RZ_G2L) {
+ int ret;
- /*
- * NOTE: The 0x1511144 are undocumented bits, but they must be set
- * for RPCIF_PHYOFFSET1.
- * The 0x31 are undocumented bits, but they must be set
- * for RPCIF_PHYOFFSET2.
- */
- regmap_write(rpc->regmap, RPCIF_PHYOFFSET1, 0x1511144 |
- RPCIF_PHYOFFSET1_DDRTMG(3));
- regmap_write(rpc->regmap, RPCIF_PHYOFFSET2, 0x31 |
- RPCIF_PHYOFFSET2_OCTTMG(4));
+ ret = reset_control_reset(rpc->rstc);
+ if (ret)
+ return ret;
+ usleep_range(200, 300);
+ rpcif_rzg2l_timing_adjust_sdr(rpc);
+ }
+
+ regmap_update_bits(rpc->regmap, RPCIF_PHYCNT, RPCIF_PHYCNT_PHYMEM_MASK,
+ RPCIF_PHYCNT_PHYMEM(hyperflash ? 3 : 0));
+
+ if (rpc->type == RPCIF_RCAR_GEN3)
+ regmap_update_bits(rpc->regmap, RPCIF_PHYCNT,
+ RPCIF_PHYCNT_STRTIM(7), RPCIF_PHYCNT_STRTIM(7));
+
+ regmap_update_bits(rpc->regmap, RPCIF_PHYOFFSET1, RPCIF_PHYOFFSET1_DDRTMG(3),
+ RPCIF_PHYOFFSET1_DDRTMG(3));
+ regmap_update_bits(rpc->regmap, RPCIF_PHYOFFSET2, RPCIF_PHYOFFSET2_OCTTMG(7),
+ RPCIF_PHYOFFSET2_OCTTMG(4));
if (hyperflash)
regmap_update_bits(rpc->regmap, RPCIF_PHYINT,
RPCIF_PHYINT_WPVAL, 0);
- regmap_write(rpc->regmap, RPCIF_CMNCR, RPCIF_CMNCR_SFDE |
- RPCIF_CMNCR_MOIIO_HIZ | RPCIF_CMNCR_IOFV_HIZ |
- RPCIF_CMNCR_BSZ(hyperflash ? 1 : 0));
+ if (rpc->type == RPCIF_RCAR_GEN3)
+ regmap_update_bits(rpc->regmap, RPCIF_CMNCR,
+ RPCIF_CMNCR_MOIIO(3) | RPCIF_CMNCR_BSZ(3),
+ RPCIF_CMNCR_MOIIO(3) |
+ RPCIF_CMNCR_BSZ(hyperflash ? 1 : 0));
+ else
+ regmap_update_bits(rpc->regmap, RPCIF_CMNCR,
+ RPCIF_CMNCR_MOIIO(3) | RPCIF_CMNCR_IOFV(3) |
+ RPCIF_CMNCR_BSZ(3),
+ RPCIF_CMNCR_MOIIO(1) | RPCIF_CMNCR_IOFV(2) |
+ RPCIF_CMNCR_BSZ(hyperflash ? 1 : 0));
+
/* Set RCF after BSZ update */
regmap_write(rpc->regmap, RPCIF_DRCR, RPCIF_DRCR_RCF);
/* Dummy read according to spec */
@@ -297,6 +323,8 @@ void rpcif_hw_init(struct rpcif *rpc, bool hyperflash)
pm_runtime_put(rpc->dev);
rpc->bus_size = hyperflash ? 2 : 1;
+
+ return 0;
}
EXPORT_SYMBOL(rpcif_hw_init);
@@ -588,8 +616,8 @@ static void memcpy_fromio_readw(void *to,
ssize_t rpcif_dirmap_read(struct rpcif *rpc, u64 offs, size_t len, void *buf)
{
- loff_t from = offs & (RPCIF_DIRMAP_SIZE - 1);
- size_t size = RPCIF_DIRMAP_SIZE - from;
+ loff_t from = offs & (rpc->size - 1);
+ size_t size = rpc->size - from;
if (len > size)
len = size;
@@ -659,7 +687,8 @@ static int rpcif_remove(struct platform_device *pdev)
}
static const struct of_device_id rpcif_of_match[] = {
- { .compatible = "renesas,rcar-gen3-rpc-if", },
+ { .compatible = "renesas,rcar-gen3-rpc-if", .data = (void *)RPCIF_RCAR_GEN3 },
+ { .compatible = "renesas,rzg2l-rpc-if", .data = (void *)RPCIF_RZ_G2L },
{},
};
MODULE_DEVICE_TABLE(of, rpcif_of_match);
diff --git a/drivers/misc/mei/bus.c b/drivers/misc/mei/bus.c
index 44bac4a..46aa355 100644
--- a/drivers/misc/mei/bus.c
+++ b/drivers/misc/mei/bus.c
@@ -643,6 +643,64 @@ static void mei_cl_bus_vtag_free(struct mei_cl_device *cldev)
kfree(cl_vtag);
}
+void *mei_cldev_dma_map(struct mei_cl_device *cldev, u8 buffer_id, size_t size)
+{
+ struct mei_device *bus;
+ struct mei_cl *cl;
+ int ret;
+
+ if (!cldev || !buffer_id || !size)
+ return ERR_PTR(-EINVAL);
+
+ if (!IS_ALIGNED(size, MEI_FW_PAGE_SIZE)) {
+ dev_err(&cldev->dev, "Map size should be aligned to %lu\n",
+ MEI_FW_PAGE_SIZE);
+ return ERR_PTR(-EINVAL);
+ }
+
+ cl = cldev->cl;
+ bus = cldev->bus;
+
+ mutex_lock(&bus->device_lock);
+ if (cl->state == MEI_FILE_UNINITIALIZED) {
+ ret = mei_cl_link(cl);
+ if (ret)
+ goto out;
+ /* update pointers */
+ cl->cldev = cldev;
+ }
+
+ ret = mei_cl_dma_alloc_and_map(cl, NULL, buffer_id, size);
+out:
+ mutex_unlock(&bus->device_lock);
+ if (ret)
+ return ERR_PTR(ret);
+ return cl->dma.vaddr;
+}
+EXPORT_SYMBOL_GPL(mei_cldev_dma_map);
+
+int mei_cldev_dma_unmap(struct mei_cl_device *cldev)
+{
+ struct mei_device *bus;
+ struct mei_cl *cl;
+ int ret;
+
+ if (!cldev)
+ return -EINVAL;
+
+ cl = cldev->cl;
+ bus = cldev->bus;
+
+ mutex_lock(&bus->device_lock);
+ ret = mei_cl_dma_unmap(cl, NULL);
+
+ mei_cl_flush_queues(cl, NULL);
+ mei_cl_unlink(cl);
+ mutex_unlock(&bus->device_lock);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(mei_cldev_dma_unmap);
+
/**
* mei_cldev_enable - enable me client device
* create connection with me client
@@ -753,9 +811,11 @@ int mei_cldev_disable(struct mei_cl_device *cldev)
dev_err(bus->dev, "Could not disconnect from the ME client\n");
out:
- /* Flush queues and remove any pending read */
- mei_cl_flush_queues(cl, NULL);
- mei_cl_unlink(cl);
+ /* Flush queues and remove any pending read unless we have mapped DMA */
+ if (!cl->dma_mapped) {
+ mei_cl_flush_queues(cl, NULL);
+ mei_cl_unlink(cl);
+ }
mutex_unlock(&bus->device_lock);
return err;
@@ -1052,6 +1112,7 @@ static void mei_cl_bus_dev_release(struct device *dev)
if (!cldev)
return;
+ mei_cl_flush_queues(cldev->cl, NULL);
mei_me_cl_put(cldev->me_cl);
mei_dev_bus_put(cldev->bus);
mei_cl_unlink(cldev->cl);
diff --git a/drivers/misc/mei/client.c b/drivers/misc/mei/client.c
index 96f4e59..0e90591 100644
--- a/drivers/misc/mei/client.c
+++ b/drivers/misc/mei/client.c
@@ -700,6 +700,9 @@ int mei_cl_unlink(struct mei_cl *cl)
cl_dbg(dev, cl, "unlink client");
+ if (cl->state == MEI_FILE_UNINITIALIZED)
+ return 0;
+
if (dev->open_handle_count > 0)
dev->open_handle_count--;
diff --git a/drivers/misc/mei/hw.h b/drivers/misc/mei/hw.h
index dfd60c9..b46077b 100644
--- a/drivers/misc/mei/hw.h
+++ b/drivers/misc/mei/hw.h
@@ -23,6 +23,11 @@
#define MEI_HBM_TIMEOUT 1 /* 1 second */
/*
+ * FW page size for DMA allocations
+ */
+#define MEI_FW_PAGE_SIZE 4096UL
+
+/*
* MEI Version
*/
#define HBM_MINOR_VERSION 2
diff --git a/drivers/mmc/host/sdhci-tegra.c b/drivers/mmc/host/sdhci-tegra.c
index 9762ffa..35ebba0 100644
--- a/drivers/mmc/host/sdhci-tegra.c
+++ b/drivers/mmc/host/sdhci-tegra.c
@@ -15,6 +15,8 @@
#include <linux/of.h>
#include <linux/of_device.h>
#include <linux/pinctrl/consumer.h>
+#include <linux/pm_opp.h>
+#include <linux/pm_runtime.h>
#include <linux/regulator/consumer.h>
#include <linux/reset.h>
#include <linux/mmc/card.h>
@@ -24,6 +26,8 @@
#include <linux/gpio/consumer.h>
#include <linux/ktime.h>
+#include <soc/tegra/common.h>
+
#include "sdhci-pltfm.h"
#include "cqhci.h"
@@ -743,7 +747,9 @@ static void tegra_sdhci_set_clock(struct sdhci_host *host, unsigned int clock)
{
struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
struct sdhci_tegra *tegra_host = sdhci_pltfm_priv(pltfm_host);
+ struct device *dev = mmc_dev(host->mmc);
unsigned long host_clk;
+ int err;
if (!clock)
return sdhci_set_clock(host, clock);
@@ -761,7 +767,12 @@ static void tegra_sdhci_set_clock(struct sdhci_host *host, unsigned int clock)
* from clk_get_rate() is used.
*/
host_clk = tegra_host->ddr_signaling ? clock * 2 : clock;
- clk_set_rate(pltfm_host->clk, host_clk);
+
+ err = dev_pm_opp_set_rate(dev, host_clk);
+ if (err)
+ dev_err(dev, "failed to set clk rate to %luHz: %d\n",
+ host_clk, err);
+
tegra_host->curr_clk_rate = host_clk;
if (tegra_host->ddr_signaling)
host->max_clk = host_clk;
@@ -1714,7 +1725,6 @@ static int sdhci_tegra_probe(struct platform_device *pdev)
"failed to get clock\n");
goto err_clk_get;
}
- clk_prepare_enable(clk);
pltfm_host->clk = clk;
tegra_host->rst = devm_reset_control_get_exclusive(&pdev->dev,
@@ -1725,15 +1735,24 @@ static int sdhci_tegra_probe(struct platform_device *pdev)
goto err_rst_get;
}
- rc = reset_control_assert(tegra_host->rst);
+ rc = devm_tegra_core_dev_init_opp_table_common(&pdev->dev);
if (rc)
goto err_rst_get;
+ pm_runtime_enable(&pdev->dev);
+ rc = pm_runtime_resume_and_get(&pdev->dev);
+ if (rc)
+ goto err_pm_get;
+
+ rc = reset_control_assert(tegra_host->rst);
+ if (rc)
+ goto err_rst_assert;
+
usleep_range(2000, 4000);
rc = reset_control_deassert(tegra_host->rst);
if (rc)
- goto err_rst_get;
+ goto err_rst_assert;
usleep_range(2000, 4000);
@@ -1745,8 +1764,11 @@ static int sdhci_tegra_probe(struct platform_device *pdev)
err_add_host:
reset_control_assert(tegra_host->rst);
+err_rst_assert:
+ pm_runtime_put_sync_suspend(&pdev->dev);
+err_pm_get:
+ pm_runtime_disable(&pdev->dev);
err_rst_get:
- clk_disable_unprepare(pltfm_host->clk);
err_clk_get:
clk_disable_unprepare(tegra_host->tmclk);
err_power_req:
@@ -1765,19 +1787,38 @@ static int sdhci_tegra_remove(struct platform_device *pdev)
reset_control_assert(tegra_host->rst);
usleep_range(2000, 4000);
- clk_disable_unprepare(pltfm_host->clk);
- clk_disable_unprepare(tegra_host->tmclk);
+ pm_runtime_put_sync_suspend(&pdev->dev);
+ pm_runtime_force_suspend(&pdev->dev);
+
+ clk_disable_unprepare(tegra_host->tmclk);
sdhci_pltfm_free(pdev);
return 0;
}
-#ifdef CONFIG_PM_SLEEP
-static int __maybe_unused sdhci_tegra_suspend(struct device *dev)
+static int __maybe_unused sdhci_tegra_runtime_suspend(struct device *dev)
{
struct sdhci_host *host = dev_get_drvdata(dev);
struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
+
+ clk_disable_unprepare(pltfm_host->clk);
+
+ return 0;
+}
+
+static int __maybe_unused sdhci_tegra_runtime_resume(struct device *dev)
+{
+ struct sdhci_host *host = dev_get_drvdata(dev);
+ struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
+
+ return clk_prepare_enable(pltfm_host->clk);
+}
+
+#ifdef CONFIG_PM_SLEEP
+static int sdhci_tegra_suspend(struct device *dev)
+{
+ struct sdhci_host *host = dev_get_drvdata(dev);
int ret;
if (host->mmc->caps2 & MMC_CAP2_CQE) {
@@ -1792,17 +1833,22 @@ static int __maybe_unused sdhci_tegra_suspend(struct device *dev)
return ret;
}
- clk_disable_unprepare(pltfm_host->clk);
+ ret = pm_runtime_force_suspend(dev);
+ if (ret) {
+ sdhci_resume_host(host);
+ cqhci_resume(host->mmc);
+ return ret;
+ }
+
return 0;
}
-static int __maybe_unused sdhci_tegra_resume(struct device *dev)
+static int sdhci_tegra_resume(struct device *dev)
{
struct sdhci_host *host = dev_get_drvdata(dev);
- struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
int ret;
- ret = clk_prepare_enable(pltfm_host->clk);
+ ret = pm_runtime_force_resume(dev);
if (ret)
return ret;
@@ -1821,13 +1867,16 @@ static int __maybe_unused sdhci_tegra_resume(struct device *dev)
suspend_host:
sdhci_suspend_host(host);
disable_clk:
- clk_disable_unprepare(pltfm_host->clk);
+ pm_runtime_force_suspend(dev);
return ret;
}
#endif
-static SIMPLE_DEV_PM_OPS(sdhci_tegra_dev_pm_ops, sdhci_tegra_suspend,
- sdhci_tegra_resume);
+static const struct dev_pm_ops sdhci_tegra_dev_pm_ops = {
+ SET_RUNTIME_PM_OPS(sdhci_tegra_runtime_suspend, sdhci_tegra_runtime_resume,
+ NULL)
+ SET_SYSTEM_SLEEP_PM_OPS(sdhci_tegra_suspend, sdhci_tegra_resume)
+};
static struct platform_driver sdhci_tegra_driver = {
.driver = {
diff --git a/drivers/mtd/hyperbus/rpc-if.c b/drivers/mtd/hyperbus/rpc-if.c
index ecb050b..8daa296 100644
--- a/drivers/mtd/hyperbus/rpc-if.c
+++ b/drivers/mtd/hyperbus/rpc-if.c
@@ -130,7 +130,9 @@ static int rpcif_hb_probe(struct platform_device *pdev)
rpcif_enable_rpm(&hyperbus->rpc);
- rpcif_hw_init(&hyperbus->rpc, true);
+ error = rpcif_hw_init(&hyperbus->rpc, true);
+ if (error)
+ return error;
hyperbus->hbdev.map.size = hyperbus->rpc.size;
hyperbus->hbdev.map.virt = hyperbus->rpc.dirmap;
diff --git a/drivers/mtd/nand/raw/tegra_nand.c b/drivers/mtd/nand/raw/tegra_nand.c
index 32431bb..b36e526 100644
--- a/drivers/mtd/nand/raw/tegra_nand.c
+++ b/drivers/mtd/nand/raw/tegra_nand.c
@@ -17,8 +17,11 @@
#include <linux/mtd/rawnand.h>
#include <linux/of.h>
#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
#include <linux/reset.h>
+#include <soc/tegra/common.h>
+
#define COMMAND 0x00
#define COMMAND_GO BIT(31)
#define COMMAND_CLE BIT(30)
@@ -1151,6 +1154,7 @@ static int tegra_nand_probe(struct platform_device *pdev)
return -ENOMEM;
ctrl->dev = &pdev->dev;
+ platform_set_drvdata(pdev, ctrl);
nand_controller_init(&ctrl->controller);
ctrl->controller.ops = &tegra_nand_controller_ops;
@@ -1166,14 +1170,23 @@ static int tegra_nand_probe(struct platform_device *pdev)
if (IS_ERR(ctrl->clk))
return PTR_ERR(ctrl->clk);
- err = clk_prepare_enable(ctrl->clk);
+ err = devm_tegra_core_dev_init_opp_table_common(&pdev->dev);
+ if (err)
+ return err;
+
+ /*
+ * This driver doesn't support active power management yet,
+ * so we will simply keep device resumed.
+ */
+ pm_runtime_enable(&pdev->dev);
+ err = pm_runtime_resume_and_get(&pdev->dev);
if (err)
return err;
err = reset_control_reset(rst);
if (err) {
dev_err(ctrl->dev, "Failed to reset HW: %d\n", err);
- goto err_disable_clk;
+ goto err_put_pm;
}
writel_relaxed(HWSTATUS_CMD_DEFAULT, ctrl->regs + HWSTATUS_CMD);
@@ -1188,21 +1201,20 @@ static int tegra_nand_probe(struct platform_device *pdev)
dev_name(&pdev->dev), ctrl);
if (err) {
dev_err(ctrl->dev, "Failed to get IRQ: %d\n", err);
- goto err_disable_clk;
+ goto err_put_pm;
}
writel_relaxed(DMA_MST_CTRL_IS_DONE, ctrl->regs + DMA_MST_CTRL);
err = tegra_nand_chips_init(ctrl->dev, ctrl);
if (err)
- goto err_disable_clk;
-
- platform_set_drvdata(pdev, ctrl);
+ goto err_put_pm;
return 0;
-err_disable_clk:
- clk_disable_unprepare(ctrl->clk);
+err_put_pm:
+ pm_runtime_put_sync_suspend(ctrl->dev);
+ pm_runtime_force_suspend(ctrl->dev);
return err;
}
@@ -1219,11 +1231,40 @@ static int tegra_nand_remove(struct platform_device *pdev)
nand_cleanup(chip);
+ pm_runtime_put_sync_suspend(ctrl->dev);
+ pm_runtime_force_suspend(ctrl->dev);
+
+ return 0;
+}
+
+static int __maybe_unused tegra_nand_runtime_resume(struct device *dev)
+{
+ struct tegra_nand_controller *ctrl = dev_get_drvdata(dev);
+ int err;
+
+ err = clk_prepare_enable(ctrl->clk);
+ if (err) {
+ dev_err(dev, "Failed to enable clock: %d\n", err);
+ return err;
+ }
+
+ return 0;
+}
+
+static int __maybe_unused tegra_nand_runtime_suspend(struct device *dev)
+{
+ struct tegra_nand_controller *ctrl = dev_get_drvdata(dev);
+
clk_disable_unprepare(ctrl->clk);
return 0;
}
+static const struct dev_pm_ops tegra_nand_pm = {
+ SET_RUNTIME_PM_OPS(tegra_nand_runtime_suspend, tegra_nand_runtime_resume,
+ NULL)
+};
+
static const struct of_device_id tegra_nand_of_match[] = {
{ .compatible = "nvidia,tegra20-nand" },
{ /* sentinel */ }
@@ -1234,6 +1275,7 @@ static struct platform_driver tegra_nand_driver = {
.driver = {
.name = "tegra-nand",
.of_match_table = tegra_nand_of_match,
+ .pm = &tegra_nand_pm,
},
.probe = tegra_nand_probe,
.remove = tegra_nand_remove,
diff --git a/drivers/net/Kconfig b/drivers/net/Kconfig
index 6cccc3d..b2a4f998 100644
--- a/drivers/net/Kconfig
+++ b/drivers/net/Kconfig
@@ -81,7 +81,6 @@
select CRYPTO
select CRYPTO_LIB_CURVE25519
select CRYPTO_LIB_CHACHA20POLY1305
- select CRYPTO_LIB_BLAKE2S
select CRYPTO_CHACHA20_X86_64 if X86 && 64BIT
select CRYPTO_POLY1305_X86_64 if X86 && 64BIT
select CRYPTO_BLAKE2S_X86 if X86 && 64BIT
diff --git a/drivers/net/amt.c b/drivers/net/amt.c
index b732ee9..f1a36d7 100644
--- a/drivers/net/amt.c
+++ b/drivers/net/amt.c
@@ -11,6 +11,7 @@
#include <linux/net.h>
#include <linux/igmp.h>
#include <linux/workqueue.h>
+#include <net/sch_generic.h>
#include <net/net_namespace.h>
#include <net/ip.h>
#include <net/udp.h>
@@ -1106,7 +1107,7 @@ static bool amt_send_membership_query(struct amt_dev *amt,
rt = ip_route_output_key(amt->net, &fl4);
if (IS_ERR(rt)) {
netdev_dbg(amt->dev, "no route to %pI4\n", &tunnel->ip4);
- return -1;
+ return true;
}
amtmq = skb_push(skb, sizeof(*amtmq));
diff --git a/drivers/net/appletalk/ipddp.c b/drivers/net/appletalk/ipddp.c
index 5566dae..d558535 100644
--- a/drivers/net/appletalk/ipddp.c
+++ b/drivers/net/appletalk/ipddp.c
@@ -23,6 +23,7 @@
* of the GNU General Public License, incorporated herein by reference.
*/
+#include <linux/compat.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/init.h>
diff --git a/drivers/net/bareudp.c b/drivers/net/bareudp.c
index edffc34..ba587e5f 100644
--- a/drivers/net/bareudp.c
+++ b/drivers/net/bareudp.c
@@ -38,6 +38,13 @@ struct bareudp_net {
struct list_head bareudp_list;
};
+struct bareudp_conf {
+ __be16 ethertype;
+ __be16 port;
+ u16 sport_min;
+ bool multi_proto_mode;
+};
+
/* Pseudo network device */
struct bareudp_dev {
struct net *net; /* netns for packet i/o */
@@ -602,7 +609,8 @@ static struct bareudp_dev *bareudp_find_dev(struct bareudp_net *bn,
}
static int bareudp_configure(struct net *net, struct net_device *dev,
- struct bareudp_conf *conf)
+ struct bareudp_conf *conf,
+ struct netlink_ext_ack *extack)
{
struct bareudp_net *bn = net_generic(net, bareudp_net_id);
struct bareudp_dev *t, *bareudp = netdev_priv(dev);
@@ -611,13 +619,17 @@ static int bareudp_configure(struct net *net, struct net_device *dev,
bareudp->net = net;
bareudp->dev = dev;
t = bareudp_find_dev(bn, conf);
- if (t)
+ if (t) {
+ NL_SET_ERR_MSG(extack, "Another bareudp device using the same port already exists");
return -EBUSY;
+ }
if (conf->multi_proto_mode &&
(conf->ethertype != htons(ETH_P_MPLS_UC) &&
- conf->ethertype != htons(ETH_P_IP)))
+ conf->ethertype != htons(ETH_P_IP))) {
+ NL_SET_ERR_MSG(extack, "Cannot set multiproto mode for this ethertype (only IPv4 and unicast MPLS are supported)");
return -EINVAL;
+ }
bareudp->port = conf->port;
bareudp->ethertype = conf->ethertype;
@@ -664,7 +676,7 @@ static int bareudp_newlink(struct net *net, struct net_device *dev,
if (err)
return err;
- err = bareudp_configure(net, dev, &conf);
+ err = bareudp_configure(net, dev, &conf, extack);
if (err)
return err;
@@ -721,40 +733,6 @@ static struct rtnl_link_ops bareudp_link_ops __read_mostly = {
.fill_info = bareudp_fill_info,
};
-struct net_device *bareudp_dev_create(struct net *net, const char *name,
- u8 name_assign_type,
- struct bareudp_conf *conf)
-{
- struct nlattr *tb[IFLA_MAX + 1];
- struct net_device *dev;
- int err;
-
- memset(tb, 0, sizeof(tb));
- dev = rtnl_create_link(net, name, name_assign_type,
- &bareudp_link_ops, tb, NULL);
- if (IS_ERR(dev))
- return dev;
-
- err = bareudp_configure(net, dev, conf);
- if (err) {
- free_netdev(dev);
- return ERR_PTR(err);
- }
- err = dev_set_mtu(dev, IP_MAX_MTU - BAREUDP_BASE_HLEN);
- if (err)
- goto err;
-
- err = rtnl_configure_link(dev, NULL);
- if (err < 0)
- goto err;
-
- return dev;
-err:
- bareudp_dellink(dev, NULL);
- return ERR_PTR(err);
-}
-EXPORT_SYMBOL_GPL(bareudp_dev_create);
-
static __net_init int bareudp_init_net(struct net *net)
{
struct bareudp_net *bn = net_generic(net, bareudp_net_id);
diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c
index ff8da72..07fc603 100644
--- a/drivers/net/bonding/bond_main.c
+++ b/drivers/net/bonding/bond_main.c
@@ -35,6 +35,7 @@
#include <linux/module.h>
#include <linux/types.h>
#include <linux/fcntl.h>
+#include <linux/filter.h>
#include <linux/interrupt.h>
#include <linux/ptrace.h>
#include <linux/ioport.h>
@@ -71,6 +72,7 @@
#include <linux/ethtool.h>
#include <linux/if_vlan.h>
#include <linux/if_bonding.h>
+#include <linux/phy.h>
#include <linux/jiffies.h>
#include <linux/preempt.h>
#include <net/route.h>
@@ -1096,9 +1098,6 @@ static bool bond_should_notify_peers(struct bonding *bond)
slave = rcu_dereference(bond->curr_active_slave);
rcu_read_unlock();
- netdev_dbg(bond->dev, "bond_should_notify_peers: slave %s\n",
- slave ? slave->dev->name : "NULL");
-
if (!slave || !bond->send_peer_notif ||
bond->send_peer_notif %
max(1, bond->params.peer_notif_delay) != 0 ||
@@ -1106,6 +1105,9 @@ static bool bond_should_notify_peers(struct bonding *bond)
test_bit(__LINK_STATE_LINKWATCH_PENDING, &slave->dev->state))
return false;
+ netdev_dbg(bond->dev, "bond_should_notify_peers: slave %s\n",
+ slave ? slave->dev->name : "NULL");
+
return true;
}
@@ -1460,7 +1462,7 @@ static void bond_compute_features(struct bonding *bond)
bond_dev->hw_enc_features |= xfrm_features;
#endif /* CONFIG_XFRM_OFFLOAD */
bond_dev->mpls_features = mpls_features;
- bond_dev->gso_max_segs = gso_max_segs;
+ netif_set_gso_max_segs(bond_dev, gso_max_segs);
netif_set_gso_max_size(bond_dev, gso_max_size);
bond_dev->priv_flags &= ~IFF_XMIT_DST_RELEASE;
@@ -3129,8 +3131,8 @@ static void bond_loadbalance_arp_mon(struct bonding *bond)
* when the source ip is 0, so don't take the link down
* if we don't know our ip yet
*/
- if (!bond_time_in_interval(bond, trans_start, 2) ||
- !bond_time_in_interval(bond, slave->last_rx, 2)) {
+ if (!bond_time_in_interval(bond, trans_start, bond->params.missed_max) ||
+ !bond_time_in_interval(bond, slave->last_rx, bond->params.missed_max)) {
bond_propose_link_state(slave, BOND_LINK_DOWN);
slave_state_changed = 1;
@@ -3224,7 +3226,7 @@ static int bond_ab_arp_inspect(struct bonding *bond)
/* Backup slave is down if:
* - No current_arp_slave AND
- * - more than 3*delta since last receive AND
+ * - more than (missed_max+1)*delta since last receive AND
* - the bond has an IP address
*
* Note: a non-null current_arp_slave indicates
@@ -3236,20 +3238,20 @@ static int bond_ab_arp_inspect(struct bonding *bond)
*/
if (!bond_is_active_slave(slave) &&
!rcu_access_pointer(bond->current_arp_slave) &&
- !bond_time_in_interval(bond, last_rx, 3)) {
+ !bond_time_in_interval(bond, last_rx, bond->params.missed_max + 1)) {
bond_propose_link_state(slave, BOND_LINK_DOWN);
commit++;
}
/* Active slave is down if:
- * - more than 2*delta since transmitting OR
- * - (more than 2*delta since receive AND
+ * - more than missed_max*delta since transmitting OR
+ * - (more than missed_max*delta since receive AND
* the bond has an IP address)
*/
trans_start = dev_trans_start(slave->dev);
if (bond_is_active_slave(slave) &&
- (!bond_time_in_interval(bond, trans_start, 2) ||
- !bond_time_in_interval(bond, last_rx, 2))) {
+ (!bond_time_in_interval(bond, trans_start, bond->params.missed_max) ||
+ !bond_time_in_interval(bond, last_rx, bond->params.missed_max))) {
bond_propose_link_state(slave, BOND_LINK_DOWN);
commit++;
}
@@ -4091,7 +4093,11 @@ static int bond_eth_ioctl(struct net_device *bond_dev, struct ifreq *ifr, int cm
{
struct bonding *bond = netdev_priv(bond_dev);
struct mii_ioctl_data *mii = NULL;
- int res;
+ const struct net_device_ops *ops;
+ struct net_device *real_dev;
+ struct hwtstamp_config cfg;
+ struct ifreq ifrr;
+ int res = 0;
netdev_dbg(bond_dev, "bond_eth_ioctl: cmd=%d\n", cmd);
@@ -4117,7 +4123,42 @@ static int bond_eth_ioctl(struct net_device *bond_dev, struct ifreq *ifr, int cm
mii->val_out = BMSR_LSTATUS;
}
- return 0;
+ break;
+ case SIOCSHWTSTAMP:
+ if (copy_from_user(&cfg, ifr->ifr_data, sizeof(cfg)))
+ return -EFAULT;
+
+ if (!(cfg.flags & HWTSTAMP_FLAG_BONDED_PHC_INDEX))
+ return -EOPNOTSUPP;
+
+ fallthrough;
+ case SIOCGHWTSTAMP:
+ rcu_read_lock();
+ real_dev = bond_option_active_slave_get_rcu(bond);
+ rcu_read_unlock();
+ if (!real_dev)
+ return -EOPNOTSUPP;
+
+ strscpy_pad(ifrr.ifr_name, real_dev->name, IFNAMSIZ);
+ ifrr.ifr_ifru = ifr->ifr_ifru;
+
+ ops = real_dev->netdev_ops;
+ if (netif_device_present(real_dev) && ops->ndo_eth_ioctl) {
+ res = ops->ndo_eth_ioctl(real_dev, &ifrr, cmd);
+ if (res)
+ return res;
+
+ ifr->ifr_ifru = ifrr.ifr_ifru;
+ if (copy_from_user(&cfg, ifr->ifr_data, sizeof(cfg)))
+ return -EFAULT;
+
+ /* Set the BOND_PHC_INDEX flag to notify user space */
+ cfg.flags |= HWTSTAMP_FLAG_BONDED_PHC_INDEX;
+
+ return copy_to_user(ifr->ifr_data, &cfg, sizeof(cfg)) ?
+ -EFAULT : 0;
+ }
+ fallthrough;
default:
res = -EOPNOTSUPP;
}
@@ -5319,10 +5360,40 @@ static void bond_ethtool_get_drvinfo(struct net_device *bond_dev,
BOND_ABI_VERSION);
}
+static int bond_ethtool_get_ts_info(struct net_device *bond_dev,
+ struct ethtool_ts_info *info)
+{
+ struct bonding *bond = netdev_priv(bond_dev);
+ const struct ethtool_ops *ops;
+ struct net_device *real_dev;
+ struct phy_device *phydev;
+
+ rcu_read_lock();
+ real_dev = bond_option_active_slave_get_rcu(bond);
+ rcu_read_unlock();
+ if (real_dev) {
+ ops = real_dev->ethtool_ops;
+ phydev = real_dev->phydev;
+
+ if (phy_has_tsinfo(phydev)) {
+ return phy_ts_info(phydev, info);
+ } else if (ops->get_ts_info) {
+ return ops->get_ts_info(real_dev, info);
+ }
+ }
+
+ info->so_timestamping = SOF_TIMESTAMPING_RX_SOFTWARE |
+ SOF_TIMESTAMPING_SOFTWARE;
+ info->phc_index = -1;
+
+ return 0;
+}
+
static const struct ethtool_ops bond_ethtool_ops = {
.get_drvinfo = bond_ethtool_get_drvinfo,
.get_link = ethtool_op_get_link,
.get_link_ksettings = bond_ethtool_get_link_ksettings,
+ .get_ts_info = bond_ethtool_get_ts_info,
};
static const struct net_device_ops bond_netdev_ops = {
@@ -5822,6 +5893,7 @@ static int bond_check_params(struct bond_params *params)
params->arp_interval = arp_interval;
params->arp_validate = arp_validate_value;
params->arp_all_targets = arp_all_targets_value;
+ params->missed_max = 2;
params->updelay = updelay;
params->downdelay = downdelay;
params->peer_notif_delay = 0;
diff --git a/drivers/net/bonding/bond_netlink.c b/drivers/net/bonding/bond_netlink.c
index 5d54e11..1007bf6 100644
--- a/drivers/net/bonding/bond_netlink.c
+++ b/drivers/net/bonding/bond_netlink.c
@@ -110,6 +110,7 @@ static const struct nla_policy bond_policy[IFLA_BOND_MAX + 1] = {
.len = ETH_ALEN },
[IFLA_BOND_TLB_DYNAMIC_LB] = { .type = NLA_U8 },
[IFLA_BOND_PEER_NOTIF_DELAY] = { .type = NLA_U32 },
+ [IFLA_BOND_MISSED_MAX] = { .type = NLA_U8 },
};
static const struct nla_policy bond_slave_policy[IFLA_BOND_SLAVE_MAX + 1] = {
@@ -453,6 +454,15 @@ static int bond_changelink(struct net_device *bond_dev, struct nlattr *tb[],
return err;
}
+ if (data[IFLA_BOND_MISSED_MAX]) {
+ int missed_max = nla_get_u8(data[IFLA_BOND_MISSED_MAX]);
+
+ bond_opt_initval(&newval, missed_max);
+ err = __bond_opt_set(bond, BOND_OPT_MISSED_MAX, &newval);
+ if (err)
+ return err;
+ }
+
return 0;
}
@@ -515,6 +525,7 @@ static size_t bond_get_size(const struct net_device *bond_dev)
nla_total_size(ETH_ALEN) + /* IFLA_BOND_AD_ACTOR_SYSTEM */
nla_total_size(sizeof(u8)) + /* IFLA_BOND_TLB_DYNAMIC_LB */
nla_total_size(sizeof(u32)) + /* IFLA_BOND_PEER_NOTIF_DELAY */
+ nla_total_size(sizeof(u8)) + /* IFLA_BOND_MISSED_MAX */
0;
}
@@ -650,6 +661,10 @@ static int bond_fill_info(struct sk_buff *skb,
bond->params.tlb_dynamic_lb))
goto nla_put_failure;
+ if (nla_put_u8(skb, IFLA_BOND_MISSED_MAX,
+ bond->params.missed_max))
+ goto nla_put_failure;
+
if (BOND_MODE(bond) == BOND_MODE_8023AD) {
struct ad_info info;
diff --git a/drivers/net/bonding/bond_options.c b/drivers/net/bonding/bond_options.c
index b93337b..2e8484a 100644
--- a/drivers/net/bonding/bond_options.c
+++ b/drivers/net/bonding/bond_options.c
@@ -78,6 +78,8 @@ static int bond_option_ad_actor_system_set(struct bonding *bond,
const struct bond_opt_value *newval);
static int bond_option_ad_user_port_key_set(struct bonding *bond,
const struct bond_opt_value *newval);
+static int bond_option_missed_max_set(struct bonding *bond,
+ const struct bond_opt_value *newval);
static const struct bond_opt_value bond_mode_tbl[] = {
@@ -213,6 +215,13 @@ static const struct bond_opt_value bond_ad_user_port_key_tbl[] = {
{ NULL, -1, 0},
};
+static const struct bond_opt_value bond_missed_max_tbl[] = {
+ { "minval", 1, BOND_VALFLAG_MIN},
+ { "maxval", 255, BOND_VALFLAG_MAX},
+ { "default", 2, BOND_VALFLAG_DEFAULT},
+ { NULL, -1, 0},
+};
+
static const struct bond_option bond_opts[BOND_OPT_LAST] = {
[BOND_OPT_MODE] = {
.id = BOND_OPT_MODE,
@@ -270,6 +279,15 @@ static const struct bond_option bond_opts[BOND_OPT_LAST] = {
.values = bond_intmax_tbl,
.set = bond_option_arp_interval_set
},
+ [BOND_OPT_MISSED_MAX] = {
+ .id = BOND_OPT_MISSED_MAX,
+ .name = "arp_missed_max",
+ .desc = "Maximum number of missed ARP interval",
+ .unsuppmodes = BIT(BOND_MODE_8023AD) | BIT(BOND_MODE_TLB) |
+ BIT(BOND_MODE_ALB),
+ .values = bond_missed_max_tbl,
+ .set = bond_option_missed_max_set
+ },
[BOND_OPT_ARP_TARGETS] = {
.id = BOND_OPT_ARP_TARGETS,
.name = "arp_ip_target",
@@ -1186,6 +1204,16 @@ static int bond_option_arp_all_targets_set(struct bonding *bond,
return 0;
}
+static int bond_option_missed_max_set(struct bonding *bond,
+ const struct bond_opt_value *newval)
+{
+ netdev_dbg(bond->dev, "Setting missed max to %s (%llu)\n",
+ newval->string, newval->value);
+ bond->params.missed_max = newval->value;
+
+ return 0;
+}
+
static int bond_option_primary_set(struct bonding *bond,
const struct bond_opt_value *newval)
{
diff --git a/drivers/net/bonding/bond_procfs.c b/drivers/net/bonding/bond_procfs.c
index f3e3bfd..2ec11af 100644
--- a/drivers/net/bonding/bond_procfs.c
+++ b/drivers/net/bonding/bond_procfs.c
@@ -115,6 +115,8 @@ static void bond_info_show_master(struct seq_file *seq)
seq_printf(seq, "ARP Polling Interval (ms): %d\n",
bond->params.arp_interval);
+ seq_printf(seq, "ARP Missed Max: %u\n",
+ bond->params.missed_max);
seq_printf(seq, "ARP IP target/s (n.n.n.n form):");
diff --git a/drivers/net/bonding/bond_sysfs.c b/drivers/net/bonding/bond_sysfs.c
index c48b771..9b5a5df 100644
--- a/drivers/net/bonding/bond_sysfs.c
+++ b/drivers/net/bonding/bond_sysfs.c
@@ -303,6 +303,18 @@ static ssize_t bonding_show_arp_targets(struct device *d,
static DEVICE_ATTR(arp_ip_target, 0644,
bonding_show_arp_targets, bonding_sysfs_store_option);
+/* Show the arp missed max. */
+static ssize_t bonding_show_missed_max(struct device *d,
+ struct device_attribute *attr,
+ char *buf)
+{
+ struct bonding *bond = to_bond(d);
+
+ return sprintf(buf, "%u\n", bond->params.missed_max);
+}
+static DEVICE_ATTR(arp_missed_max, 0644,
+ bonding_show_missed_max, bonding_sysfs_store_option);
+
/* Show the up and down delays. */
static ssize_t bonding_show_downdelay(struct device *d,
struct device_attribute *attr,
@@ -779,6 +791,7 @@ static struct attribute *per_bond_attrs[] = {
&dev_attr_ad_actor_sys_prio.attr,
&dev_attr_ad_actor_system.attr,
&dev_attr_ad_user_port_key.attr,
+ &dev_attr_arp_missed_max.attr,
NULL,
};
diff --git a/drivers/net/can/Makefile b/drivers/net/can/Makefile
index a2b4463..1e660af 100644
--- a/drivers/net/can/Makefile
+++ b/drivers/net/can/Makefile
@@ -16,7 +16,7 @@
obj-$(CONFIG_CAN_AT91) += at91_can.o
obj-$(CONFIG_CAN_CC770) += cc770/
obj-$(CONFIG_CAN_C_CAN) += c_can/
-obj-$(CONFIG_CAN_FLEXCAN) += flexcan.o
+obj-$(CONFIG_CAN_FLEXCAN) += flexcan/
obj-$(CONFIG_CAN_GRCAN) += grcan.o
obj-$(CONFIG_CAN_IFI_CANFD) += ifi_canfd/
obj-$(CONFIG_CAN_JANZ_ICAN3) += janz-ican3.o
diff --git a/drivers/net/can/at91_can.c b/drivers/net/can/at91_can.c
index 3aea32c..a00655c 100644
--- a/drivers/net/can/at91_can.c
+++ b/drivers/net/can/at91_can.c
@@ -448,7 +448,6 @@ static void at91_chip_stop(struct net_device *dev, enum can_state state)
static netdev_tx_t at91_start_xmit(struct sk_buff *skb, struct net_device *dev)
{
struct at91_priv *priv = netdev_priv(dev);
- struct net_device_stats *stats = &dev->stats;
struct can_frame *cf = (struct can_frame *)skb->data;
unsigned int mb, prio;
u32 reg_mid, reg_mcr;
@@ -480,8 +479,6 @@ static netdev_tx_t at91_start_xmit(struct sk_buff *skb, struct net_device *dev)
/* This triggers transmission */
at91_write(priv, AT91_MCR(mb), reg_mcr);
- stats->tx_bytes += cf->len;
-
/* _NOTE_: subtract AT91_MB_TX_FIRST offset from mb! */
can_put_echo_skb(skb, dev, mb - get_mb_tx_first(priv), 0);
@@ -553,8 +550,6 @@ static void at91_rx_overflow_err(struct net_device *dev)
cf->can_id |= CAN_ERR_CRTL;
cf->data[1] = CAN_ERR_CRTL_RX_OVERFLOW;
- stats->rx_packets++;
- stats->rx_bytes += cf->len;
netif_receive_skb(skb);
}
@@ -619,7 +614,9 @@ static void at91_read_msg(struct net_device *dev, unsigned int mb)
at91_read_mb(dev, mb, cf);
stats->rx_packets++;
- stats->rx_bytes += cf->len;
+ if (!(cf->can_id & CAN_RTR_FLAG))
+ stats->rx_bytes += cf->len;
+
netif_receive_skb(skb);
can_led_event(dev, CAN_LED_EVENT_RX);
@@ -779,8 +776,6 @@ static int at91_poll_err(struct net_device *dev, int quota, u32 reg_sr)
at91_poll_err_frame(dev, cf, reg_sr);
- dev->stats.rx_packets++;
- dev->stats.rx_bytes += cf->len;
netif_receive_skb(skb);
return 1;
@@ -854,7 +849,10 @@ static void at91_irq_tx(struct net_device *dev, u32 reg_sr)
if (likely(reg_msr & AT91_MSR_MRDY &&
~reg_msr & AT91_MSR_MABT)) {
/* _NOTE_: subtract AT91_MB_TX_FIRST offset from mb! */
- can_get_echo_skb(dev, mb - get_mb_tx_first(priv), NULL);
+ dev->stats.tx_bytes +=
+ can_get_echo_skb(dev,
+ mb - get_mb_tx_first(priv),
+ NULL);
dev->stats.tx_packets++;
can_led_event(dev, CAN_LED_EVENT_TX);
}
@@ -1037,8 +1035,6 @@ static void at91_irq_err(struct net_device *dev)
at91_irq_err_state(dev, cf, new_state);
- dev->stats.rx_packets++;
- dev->stats.rx_bytes += cf->len;
netif_rx(skb);
priv->can.state = new_state;
diff --git a/drivers/net/can/c_can/c_can.h b/drivers/net/can/c_can/c_can.h
index 08b6efa..bd2f6dc 100644
--- a/drivers/net/can/c_can/c_can.h
+++ b/drivers/net/can/c_can/c_can.h
@@ -211,7 +211,6 @@ struct c_can_priv {
struct c_can_raminit raminit_sys; /* RAMINIT via syscon regmap */
void (*raminit)(const struct c_can_priv *priv, bool enable);
u32 comm_rcv_high;
- u32 dlc[];
};
struct net_device *alloc_c_can_dev(int msg_obj_num);
diff --git a/drivers/net/can/c_can/c_can_ethtool.c b/drivers/net/can/c_can/c_can_ethtool.c
index 377c7d2..6655146 100644
--- a/drivers/net/can/c_can/c_can_ethtool.c
+++ b/drivers/net/can/c_can/c_can_ethtool.c
@@ -20,7 +20,9 @@ static void c_can_get_drvinfo(struct net_device *netdev,
}
static void c_can_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct c_can_priv *priv = netdev_priv(netdev);
diff --git a/drivers/net/can/c_can/c_can_main.c b/drivers/net/can/c_can/c_can_main.c
index 52671d1..faa217f 100644
--- a/drivers/net/can/c_can/c_can_main.c
+++ b/drivers/net/can/c_can/c_can_main.c
@@ -403,10 +403,10 @@ static int c_can_read_msg_object(struct net_device *dev, int iface, u32 ctrl)
frame->data[i + 1] = data >> 8;
}
}
- }
+ stats->rx_bytes += frame->len;
+ }
stats->rx_packets++;
- stats->rx_bytes += frame->len;
netif_receive_skb(skb);
return 0;
@@ -477,7 +477,6 @@ static netdev_tx_t c_can_start_xmit(struct sk_buff *skb,
* transmit as we might race against do_tx().
*/
c_can_setup_tx_object(dev, IF_TX, frame, idx);
- priv->dlc[idx] = frame->len;
can_put_echo_skb(skb, dev, idx, 0);
obj = idx + priv->msg_obj_tx_first;
c_can_object_put(dev, IF_TX, obj, cmd);
@@ -742,8 +741,7 @@ static void c_can_do_tx(struct net_device *dev)
* NAPI. We are not transmitting.
*/
c_can_inval_tx_object(dev, IF_NAPI, obj);
- can_get_echo_skb(dev, idx, NULL);
- bytes += priv->dlc[idx];
+ bytes += can_get_echo_skb(dev, idx, NULL);
pkts++;
}
@@ -920,7 +918,6 @@ static int c_can_handle_state_change(struct net_device *dev,
unsigned int reg_err_counter;
unsigned int rx_err_passive;
struct c_can_priv *priv = netdev_priv(dev);
- struct net_device_stats *stats = &dev->stats;
struct can_frame *cf;
struct sk_buff *skb;
struct can_berr_counter bec;
@@ -996,8 +993,6 @@ static int c_can_handle_state_change(struct net_device *dev,
break;
}
- stats->rx_packets++;
- stats->rx_bytes += cf->len;
netif_receive_skb(skb);
return 1;
@@ -1064,8 +1059,6 @@ static int c_can_handle_bus_err(struct net_device *dev,
break;
}
- stats->rx_packets++;
- stats->rx_bytes += cf->len;
netif_receive_skb(skb);
return 1;
}
@@ -1232,8 +1225,7 @@ struct net_device *alloc_c_can_dev(int msg_obj_num)
struct c_can_priv *priv;
int msg_obj_tx_num = msg_obj_num / 2;
- dev = alloc_candev(struct_size(priv, dlc, msg_obj_tx_num),
- msg_obj_tx_num);
+ dev = alloc_candev(sizeof(*priv), msg_obj_tx_num);
if (!dev)
return NULL;
diff --git a/drivers/net/can/cc770/cc770.c b/drivers/net/can/cc770/cc770.c
index f8a130f..bb7224c 100644
--- a/drivers/net/can/cc770/cc770.c
+++ b/drivers/net/can/cc770/cc770.c
@@ -489,17 +489,17 @@ static void cc770_rx(struct net_device *dev, unsigned int mo, u8 ctrl1)
cf->len = can_cc_dlc2len((config & 0xf0) >> 4);
for (i = 0; i < cf->len; i++)
cf->data[i] = cc770_read_reg(priv, msgobj[mo].data[i]);
- }
+ stats->rx_bytes += cf->len;
+ }
stats->rx_packets++;
- stats->rx_bytes += cf->len;
+
netif_rx(skb);
}
static int cc770_err(struct net_device *dev, u8 status)
{
struct cc770_priv *priv = netdev_priv(dev);
- struct net_device_stats *stats = &dev->stats;
struct can_frame *cf;
struct sk_buff *skb;
u8 lec;
@@ -571,8 +571,6 @@ static int cc770_err(struct net_device *dev, u8 status)
}
- stats->rx_packets++;
- stats->rx_bytes += cf->len;
netif_rx(skb);
return 0;
@@ -666,7 +664,6 @@ static void cc770_tx_interrupt(struct net_device *dev, unsigned int o)
struct cc770_priv *priv = netdev_priv(dev);
struct net_device_stats *stats = &dev->stats;
unsigned int mo = obj2msgobj(o);
- struct can_frame *cf;
u8 ctrl1;
ctrl1 = cc770_read_reg(priv, msgobj[mo].ctrl1);
@@ -698,12 +695,9 @@ static void cc770_tx_interrupt(struct net_device *dev, unsigned int o)
return;
}
- cf = (struct can_frame *)priv->tx_skb->data;
- stats->tx_bytes += cf->len;
- stats->tx_packets++;
-
can_put_echo_skb(priv->tx_skb, dev, 0, 0);
- can_get_echo_skb(dev, 0, NULL);
+ stats->tx_bytes += can_get_echo_skb(dev, 0, NULL);
+ stats->tx_packets++;
priv->tx_skb = NULL;
netif_wake_queue(dev);
diff --git a/drivers/net/can/dev/bittiming.c b/drivers/net/can/dev/bittiming.c
index 0509625..d5fca3b 100644
--- a/drivers/net/can/dev/bittiming.c
+++ b/drivers/net/can/dev/bittiming.c
@@ -4,6 +4,7 @@
* Copyright (C) 2008-2009 Wolfgang Grandegger <wg@grandegger.com>
*/
+#include <linux/units.h>
#include <linux/can/dev.h>
#ifdef CONFIG_CAN_CALC_BITTIMING
@@ -81,9 +82,9 @@ int can_calc_bittiming(struct net_device *dev, struct can_bittiming *bt,
if (bt->sample_point) {
sample_point_nominal = bt->sample_point;
} else {
- if (bt->bitrate > 800 * CAN_KBPS)
+ if (bt->bitrate > 800 * KILO /* BPS */)
sample_point_nominal = 750;
- else if (bt->bitrate > 500 * CAN_KBPS)
+ else if (bt->bitrate > 500 * KILO /* BPS */)
sample_point_nominal = 800;
else
sample_point_nominal = 875;
diff --git a/drivers/net/can/dev/dev.c b/drivers/net/can/dev/dev.c
index e3d840b..c192f25 100644
--- a/drivers/net/can/dev/dev.c
+++ b/drivers/net/can/dev/dev.c
@@ -136,7 +136,6 @@ EXPORT_SYMBOL_GPL(can_change_state);
static void can_restart(struct net_device *dev)
{
struct can_priv *priv = netdev_priv(dev);
- struct net_device_stats *stats = &dev->stats;
struct sk_buff *skb;
struct can_frame *cf;
int err;
@@ -155,9 +154,6 @@ static void can_restart(struct net_device *dev)
cf->can_id |= CAN_ERR_RESTARTED;
- stats->rx_packets++;
- stats->rx_bytes += cf->len;
-
netif_rx_ni(skb);
restart:
@@ -300,6 +296,7 @@ EXPORT_SYMBOL_GPL(free_candev);
int can_change_mtu(struct net_device *dev, int new_mtu)
{
struct can_priv *priv = netdev_priv(dev);
+ u32 ctrlmode_static = can_get_static_ctrlmode(priv);
/* Do not allow changing the MTU while running */
if (dev->flags & IFF_UP)
@@ -309,7 +306,7 @@ int can_change_mtu(struct net_device *dev, int new_mtu)
switch (new_mtu) {
case CAN_MTU:
/* 'CANFD-only' controllers can not switch to CAN_MTU */
- if (priv->ctrlmode_static & CAN_CTRLMODE_FD)
+ if (ctrlmode_static & CAN_CTRLMODE_FD)
return -EINVAL;
priv->ctrlmode &= ~CAN_CTRLMODE_FD;
@@ -318,7 +315,7 @@ int can_change_mtu(struct net_device *dev, int new_mtu)
case CANFD_MTU:
/* check for potential CANFD ability */
if (!(priv->ctrlmode_supported & CAN_CTRLMODE_FD) &&
- !(priv->ctrlmode_static & CAN_CTRLMODE_FD))
+ !(ctrlmode_static & CAN_CTRLMODE_FD))
return -EINVAL;
priv->ctrlmode |= CAN_CTRLMODE_FD;
diff --git a/drivers/net/can/dev/netlink.c b/drivers/net/can/dev/netlink.c
index 95cca4e..7633d98 100644
--- a/drivers/net/can/dev/netlink.c
+++ b/drivers/net/can/dev/netlink.c
@@ -21,6 +21,7 @@ static const struct nla_policy can_policy[IFLA_CAN_MAX + 1] = {
[IFLA_CAN_DATA_BITTIMING_CONST] = { .len = sizeof(struct can_bittiming_const) },
[IFLA_CAN_TERMINATION] = { .type = NLA_U16 },
[IFLA_CAN_TDC] = { .type = NLA_NESTED },
+ [IFLA_CAN_CTRLMODE_EXT] = { .type = NLA_NESTED },
};
static const struct nla_policy can_tdc_policy[IFLA_CAN_TDC_MAX + 1] = {
@@ -211,7 +212,7 @@ static int can_changelink(struct net_device *dev, struct nlattr *tb[],
if (dev->flags & IFF_UP)
return -EBUSY;
cm = nla_data(data[IFLA_CAN_CTRLMODE]);
- ctrlstatic = priv->ctrlmode_static;
+ ctrlstatic = can_get_static_ctrlmode(priv);
maskedflags = cm->flags & cm->mask;
/* check whether provided bits are allowed to be passed */
@@ -383,6 +384,12 @@ static size_t can_tdc_get_size(const struct net_device *dev)
return size;
}
+static size_t can_ctrlmode_ext_get_size(void)
+{
+ return nla_total_size(0) + /* nest IFLA_CAN_CTRLMODE_EXT */
+ nla_total_size(sizeof(u32)); /* IFLA_CAN_CTRLMODE_SUPPORTED */
+}
+
static size_t can_get_size(const struct net_device *dev)
{
struct can_priv *priv = netdev_priv(dev);
@@ -415,6 +422,7 @@ static size_t can_get_size(const struct net_device *dev)
priv->data_bitrate_const_cnt);
size += sizeof(priv->bitrate_max); /* IFLA_CAN_BITRATE_MAX */
size += can_tdc_get_size(dev); /* IFLA_CAN_TDC */
+ size += can_ctrlmode_ext_get_size(); /* IFLA_CAN_CTRLMODE_EXT */
return size;
}
@@ -472,6 +480,25 @@ static int can_tdc_fill_info(struct sk_buff *skb, const struct net_device *dev)
return -EMSGSIZE;
}
+static int can_ctrlmode_ext_fill_info(struct sk_buff *skb,
+ const struct can_priv *priv)
+{
+ struct nlattr *nest;
+
+ nest = nla_nest_start(skb, IFLA_CAN_CTRLMODE_EXT);
+ if (!nest)
+ return -EMSGSIZE;
+
+ if (nla_put_u32(skb, IFLA_CAN_CTRLMODE_SUPPORTED,
+ priv->ctrlmode_supported)) {
+ nla_nest_cancel(skb, nest);
+ return -EMSGSIZE;
+ }
+
+ nla_nest_end(skb, nest);
+ return 0;
+}
+
static int can_fill_info(struct sk_buff *skb, const struct net_device *dev)
{
struct can_priv *priv = netdev_priv(dev);
@@ -531,7 +558,9 @@ static int can_fill_info(struct sk_buff *skb, const struct net_device *dev)
sizeof(priv->bitrate_max),
&priv->bitrate_max)) ||
- (can_tdc_fill_info(skb, dev))
+ can_tdc_fill_info(skb, dev) ||
+
+ can_ctrlmode_ext_fill_info(skb, priv)
)
return -EMSGSIZE;
diff --git a/drivers/net/can/dev/rx-offload.c b/drivers/net/can/dev/rx-offload.c
index 37b0cc6..7f80d8e 100644
--- a/drivers/net/can/dev/rx-offload.c
+++ b/drivers/net/can/dev/rx-offload.c
@@ -54,8 +54,11 @@ static int can_rx_offload_napi_poll(struct napi_struct *napi, int quota)
struct can_frame *cf = (struct can_frame *)skb->data;
work_done++;
- stats->rx_packets++;
- stats->rx_bytes += cf->len;
+ if (!(cf->can_id & CAN_ERR_FLAG)) {
+ stats->rx_packets++;
+ if (!(cf->can_id & CAN_RTR_FLAG))
+ stats->rx_bytes += cf->len;
+ }
netif_receive_skb(skb);
}
diff --git a/drivers/net/can/flexcan.c b/drivers/net/can/flexcan.c
deleted file mode 100644
index 12b60ad..0000000
--- a/drivers/net/can/flexcan.c
+++ /dev/null
@@ -1,2419 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-//
-// flexcan.c - FLEXCAN CAN controller driver
-//
-// Copyright (c) 2005-2006 Varma Electronics Oy
-// Copyright (c) 2009 Sascha Hauer, Pengutronix
-// Copyright (c) 2010-2017 Pengutronix, Marc Kleine-Budde <kernel@pengutronix.de>
-// Copyright (c) 2014 David Jander, Protonic Holland
-//
-// Based on code originally by Andrey Volkov <avolkov@varma-el.com>
-
-#include <dt-bindings/firmware/imx/rsrc.h>
-#include <linux/bitfield.h>
-#include <linux/can.h>
-#include <linux/can/dev.h>
-#include <linux/can/error.h>
-#include <linux/can/led.h>
-#include <linux/can/rx-offload.h>
-#include <linux/clk.h>
-#include <linux/delay.h>
-#include <linux/firmware/imx/sci.h>
-#include <linux/interrupt.h>
-#include <linux/io.h>
-#include <linux/mfd/syscon.h>
-#include <linux/module.h>
-#include <linux/netdevice.h>
-#include <linux/of.h>
-#include <linux/of_device.h>
-#include <linux/pinctrl/consumer.h>
-#include <linux/platform_device.h>
-#include <linux/can/platform/flexcan.h>
-#include <linux/pm_runtime.h>
-#include <linux/regmap.h>
-#include <linux/regulator/consumer.h>
-
-#define DRV_NAME "flexcan"
-
-/* 8 for RX fifo and 2 error handling */
-#define FLEXCAN_NAPI_WEIGHT (8 + 2)
-
-/* FLEXCAN module configuration register (CANMCR) bits */
-#define FLEXCAN_MCR_MDIS BIT(31)
-#define FLEXCAN_MCR_FRZ BIT(30)
-#define FLEXCAN_MCR_FEN BIT(29)
-#define FLEXCAN_MCR_HALT BIT(28)
-#define FLEXCAN_MCR_NOT_RDY BIT(27)
-#define FLEXCAN_MCR_WAK_MSK BIT(26)
-#define FLEXCAN_MCR_SOFTRST BIT(25)
-#define FLEXCAN_MCR_FRZ_ACK BIT(24)
-#define FLEXCAN_MCR_SUPV BIT(23)
-#define FLEXCAN_MCR_SLF_WAK BIT(22)
-#define FLEXCAN_MCR_WRN_EN BIT(21)
-#define FLEXCAN_MCR_LPM_ACK BIT(20)
-#define FLEXCAN_MCR_WAK_SRC BIT(19)
-#define FLEXCAN_MCR_DOZE BIT(18)
-#define FLEXCAN_MCR_SRX_DIS BIT(17)
-#define FLEXCAN_MCR_IRMQ BIT(16)
-#define FLEXCAN_MCR_LPRIO_EN BIT(13)
-#define FLEXCAN_MCR_AEN BIT(12)
-#define FLEXCAN_MCR_FDEN BIT(11)
-/* MCR_MAXMB: maximum used MBs is MAXMB + 1 */
-#define FLEXCAN_MCR_MAXMB(x) ((x) & 0x7f)
-#define FLEXCAN_MCR_IDAM_A (0x0 << 8)
-#define FLEXCAN_MCR_IDAM_B (0x1 << 8)
-#define FLEXCAN_MCR_IDAM_C (0x2 << 8)
-#define FLEXCAN_MCR_IDAM_D (0x3 << 8)
-
-/* FLEXCAN control register (CANCTRL) bits */
-#define FLEXCAN_CTRL_PRESDIV(x) (((x) & 0xff) << 24)
-#define FLEXCAN_CTRL_RJW(x) (((x) & 0x03) << 22)
-#define FLEXCAN_CTRL_PSEG1(x) (((x) & 0x07) << 19)
-#define FLEXCAN_CTRL_PSEG2(x) (((x) & 0x07) << 16)
-#define FLEXCAN_CTRL_BOFF_MSK BIT(15)
-#define FLEXCAN_CTRL_ERR_MSK BIT(14)
-#define FLEXCAN_CTRL_CLK_SRC BIT(13)
-#define FLEXCAN_CTRL_LPB BIT(12)
-#define FLEXCAN_CTRL_TWRN_MSK BIT(11)
-#define FLEXCAN_CTRL_RWRN_MSK BIT(10)
-#define FLEXCAN_CTRL_SMP BIT(7)
-#define FLEXCAN_CTRL_BOFF_REC BIT(6)
-#define FLEXCAN_CTRL_TSYN BIT(5)
-#define FLEXCAN_CTRL_LBUF BIT(4)
-#define FLEXCAN_CTRL_LOM BIT(3)
-#define FLEXCAN_CTRL_PROPSEG(x) ((x) & 0x07)
-#define FLEXCAN_CTRL_ERR_BUS (FLEXCAN_CTRL_ERR_MSK)
-#define FLEXCAN_CTRL_ERR_STATE \
- (FLEXCAN_CTRL_TWRN_MSK | FLEXCAN_CTRL_RWRN_MSK | \
- FLEXCAN_CTRL_BOFF_MSK)
-#define FLEXCAN_CTRL_ERR_ALL \
- (FLEXCAN_CTRL_ERR_BUS | FLEXCAN_CTRL_ERR_STATE)
-
-/* FLEXCAN control register 2 (CTRL2) bits */
-#define FLEXCAN_CTRL2_ECRWRE BIT(29)
-#define FLEXCAN_CTRL2_WRMFRZ BIT(28)
-#define FLEXCAN_CTRL2_RFFN(x) (((x) & 0x0f) << 24)
-#define FLEXCAN_CTRL2_TASD(x) (((x) & 0x1f) << 19)
-#define FLEXCAN_CTRL2_MRP BIT(18)
-#define FLEXCAN_CTRL2_RRS BIT(17)
-#define FLEXCAN_CTRL2_EACEN BIT(16)
-#define FLEXCAN_CTRL2_ISOCANFDEN BIT(12)
-
-/* FLEXCAN memory error control register (MECR) bits */
-#define FLEXCAN_MECR_ECRWRDIS BIT(31)
-#define FLEXCAN_MECR_HANCEI_MSK BIT(19)
-#define FLEXCAN_MECR_FANCEI_MSK BIT(18)
-#define FLEXCAN_MECR_CEI_MSK BIT(16)
-#define FLEXCAN_MECR_HAERRIE BIT(15)
-#define FLEXCAN_MECR_FAERRIE BIT(14)
-#define FLEXCAN_MECR_EXTERRIE BIT(13)
-#define FLEXCAN_MECR_RERRDIS BIT(9)
-#define FLEXCAN_MECR_ECCDIS BIT(8)
-#define FLEXCAN_MECR_NCEFAFRZ BIT(7)
-
-/* FLEXCAN error and status register (ESR) bits */
-#define FLEXCAN_ESR_TWRN_INT BIT(17)
-#define FLEXCAN_ESR_RWRN_INT BIT(16)
-#define FLEXCAN_ESR_BIT1_ERR BIT(15)
-#define FLEXCAN_ESR_BIT0_ERR BIT(14)
-#define FLEXCAN_ESR_ACK_ERR BIT(13)
-#define FLEXCAN_ESR_CRC_ERR BIT(12)
-#define FLEXCAN_ESR_FRM_ERR BIT(11)
-#define FLEXCAN_ESR_STF_ERR BIT(10)
-#define FLEXCAN_ESR_TX_WRN BIT(9)
-#define FLEXCAN_ESR_RX_WRN BIT(8)
-#define FLEXCAN_ESR_IDLE BIT(7)
-#define FLEXCAN_ESR_TXRX BIT(6)
-#define FLEXCAN_EST_FLT_CONF_SHIFT (4)
-#define FLEXCAN_ESR_FLT_CONF_MASK (0x3 << FLEXCAN_EST_FLT_CONF_SHIFT)
-#define FLEXCAN_ESR_FLT_CONF_ACTIVE (0x0 << FLEXCAN_EST_FLT_CONF_SHIFT)
-#define FLEXCAN_ESR_FLT_CONF_PASSIVE (0x1 << FLEXCAN_EST_FLT_CONF_SHIFT)
-#define FLEXCAN_ESR_BOFF_INT BIT(2)
-#define FLEXCAN_ESR_ERR_INT BIT(1)
-#define FLEXCAN_ESR_WAK_INT BIT(0)
-#define FLEXCAN_ESR_ERR_BUS \
- (FLEXCAN_ESR_BIT1_ERR | FLEXCAN_ESR_BIT0_ERR | \
- FLEXCAN_ESR_ACK_ERR | FLEXCAN_ESR_CRC_ERR | \
- FLEXCAN_ESR_FRM_ERR | FLEXCAN_ESR_STF_ERR)
-#define FLEXCAN_ESR_ERR_STATE \
- (FLEXCAN_ESR_TWRN_INT | FLEXCAN_ESR_RWRN_INT | FLEXCAN_ESR_BOFF_INT)
-#define FLEXCAN_ESR_ERR_ALL \
- (FLEXCAN_ESR_ERR_BUS | FLEXCAN_ESR_ERR_STATE)
-#define FLEXCAN_ESR_ALL_INT \
- (FLEXCAN_ESR_TWRN_INT | FLEXCAN_ESR_RWRN_INT | \
- FLEXCAN_ESR_BOFF_INT | FLEXCAN_ESR_ERR_INT)
-
-/* FLEXCAN Bit Timing register (CBT) bits */
-#define FLEXCAN_CBT_BTF BIT(31)
-#define FLEXCAN_CBT_EPRESDIV_MASK GENMASK(30, 21)
-#define FLEXCAN_CBT_ERJW_MASK GENMASK(20, 16)
-#define FLEXCAN_CBT_EPROPSEG_MASK GENMASK(15, 10)
-#define FLEXCAN_CBT_EPSEG1_MASK GENMASK(9, 5)
-#define FLEXCAN_CBT_EPSEG2_MASK GENMASK(4, 0)
-
-/* FLEXCAN FD control register (FDCTRL) bits */
-#define FLEXCAN_FDCTRL_FDRATE BIT(31)
-#define FLEXCAN_FDCTRL_MBDSR1 GENMASK(20, 19)
-#define FLEXCAN_FDCTRL_MBDSR0 GENMASK(17, 16)
-#define FLEXCAN_FDCTRL_MBDSR_8 0x0
-#define FLEXCAN_FDCTRL_MBDSR_12 0x1
-#define FLEXCAN_FDCTRL_MBDSR_32 0x2
-#define FLEXCAN_FDCTRL_MBDSR_64 0x3
-#define FLEXCAN_FDCTRL_TDCEN BIT(15)
-#define FLEXCAN_FDCTRL_TDCFAIL BIT(14)
-#define FLEXCAN_FDCTRL_TDCOFF GENMASK(12, 8)
-#define FLEXCAN_FDCTRL_TDCVAL GENMASK(5, 0)
-
-/* FLEXCAN FD Bit Timing register (FDCBT) bits */
-#define FLEXCAN_FDCBT_FPRESDIV_MASK GENMASK(29, 20)
-#define FLEXCAN_FDCBT_FRJW_MASK GENMASK(18, 16)
-#define FLEXCAN_FDCBT_FPROPSEG_MASK GENMASK(14, 10)
-#define FLEXCAN_FDCBT_FPSEG1_MASK GENMASK(7, 5)
-#define FLEXCAN_FDCBT_FPSEG2_MASK GENMASK(2, 0)
-
-/* FLEXCAN interrupt flag register (IFLAG) bits */
-/* Errata ERR005829 step7: Reserve first valid MB */
-#define FLEXCAN_TX_MB_RESERVED_OFF_FIFO 8
-#define FLEXCAN_TX_MB_RESERVED_OFF_TIMESTAMP 0
-#define FLEXCAN_RX_MB_OFF_TIMESTAMP_FIRST (FLEXCAN_TX_MB_RESERVED_OFF_TIMESTAMP + 1)
-#define FLEXCAN_IFLAG_MB(x) BIT_ULL(x)
-#define FLEXCAN_IFLAG_RX_FIFO_OVERFLOW BIT(7)
-#define FLEXCAN_IFLAG_RX_FIFO_WARN BIT(6)
-#define FLEXCAN_IFLAG_RX_FIFO_AVAILABLE BIT(5)
-
-/* FLEXCAN message buffers */
-#define FLEXCAN_MB_CODE_MASK (0xf << 24)
-#define FLEXCAN_MB_CODE_RX_BUSY_BIT (0x1 << 24)
-#define FLEXCAN_MB_CODE_RX_INACTIVE (0x0 << 24)
-#define FLEXCAN_MB_CODE_RX_EMPTY (0x4 << 24)
-#define FLEXCAN_MB_CODE_RX_FULL (0x2 << 24)
-#define FLEXCAN_MB_CODE_RX_OVERRUN (0x6 << 24)
-#define FLEXCAN_MB_CODE_RX_RANSWER (0xa << 24)
-
-#define FLEXCAN_MB_CODE_TX_INACTIVE (0x8 << 24)
-#define FLEXCAN_MB_CODE_TX_ABORT (0x9 << 24)
-#define FLEXCAN_MB_CODE_TX_DATA (0xc << 24)
-#define FLEXCAN_MB_CODE_TX_TANSWER (0xe << 24)
-
-#define FLEXCAN_MB_CNT_EDL BIT(31)
-#define FLEXCAN_MB_CNT_BRS BIT(30)
-#define FLEXCAN_MB_CNT_ESI BIT(29)
-#define FLEXCAN_MB_CNT_SRR BIT(22)
-#define FLEXCAN_MB_CNT_IDE BIT(21)
-#define FLEXCAN_MB_CNT_RTR BIT(20)
-#define FLEXCAN_MB_CNT_LENGTH(x) (((x) & 0xf) << 16)
-#define FLEXCAN_MB_CNT_TIMESTAMP(x) ((x) & 0xffff)
-
-#define FLEXCAN_TIMEOUT_US (250)
-
-/* FLEXCAN hardware feature flags
- *
- * Below is some version info we got:
- * SOC Version IP-Version Glitch- [TR]WRN_INT IRQ Err Memory err RTR rece- FD Mode MB
- * Filter? connected? Passive detection ption in MB Supported?
- * MCF5441X FlexCAN2 ? no yes no no yes no 16
- * MX25 FlexCAN2 03.00.00.00 no no no no no no 64
- * MX28 FlexCAN2 03.00.04.00 yes yes no no no no 64
- * MX35 FlexCAN2 03.00.00.00 no no no no no no 64
- * MX53 FlexCAN2 03.00.00.00 yes no no no no no 64
- * MX6s FlexCAN3 10.00.12.00 yes yes no no yes no 64
- * MX8QM FlexCAN3 03.00.23.00 yes yes no no yes yes 64
- * MX8MP FlexCAN3 03.00.17.01 yes yes no yes yes yes 64
- * VF610 FlexCAN3 ? no yes no yes yes? no 64
- * LS1021A FlexCAN2 03.00.04.00 no yes no no yes no 64
- * LX2160A FlexCAN3 03.00.23.00 no yes no yes yes yes 64
- *
- * Some SOCs do not have the RX_WARN & TX_WARN interrupt line connected.
- */
-
-/* [TR]WRN_INT not connected */
-#define FLEXCAN_QUIRK_BROKEN_WERR_STATE BIT(1)
- /* Disable RX FIFO Global mask */
-#define FLEXCAN_QUIRK_DISABLE_RXFG BIT(2)
-/* Enable EACEN and RRS bit in ctrl2 */
-#define FLEXCAN_QUIRK_ENABLE_EACEN_RRS BIT(3)
-/* Disable non-correctable errors interrupt and freeze mode */
-#define FLEXCAN_QUIRK_DISABLE_MECR BIT(4)
-/* Use timestamp based offloading */
-#define FLEXCAN_QUIRK_USE_OFF_TIMESTAMP BIT(5)
-/* No interrupt for error passive */
-#define FLEXCAN_QUIRK_BROKEN_PERR_STATE BIT(6)
-/* default to BE register access */
-#define FLEXCAN_QUIRK_DEFAULT_BIG_ENDIAN BIT(7)
-/* Setup stop mode with GPR to support wakeup */
-#define FLEXCAN_QUIRK_SETUP_STOP_MODE_GPR BIT(8)
-/* Support CAN-FD mode */
-#define FLEXCAN_QUIRK_SUPPORT_FD BIT(9)
-/* support memory detection and correction */
-#define FLEXCAN_QUIRK_SUPPORT_ECC BIT(10)
-/* Setup stop mode with SCU firmware to support wakeup */
-#define FLEXCAN_QUIRK_SETUP_STOP_MODE_SCFW BIT(11)
-/* Setup 3 separate interrupts, main, boff and err */
-#define FLEXCAN_QUIRK_NR_IRQ_3 BIT(12)
-/* Setup 16 mailboxes */
-#define FLEXCAN_QUIRK_NR_MB_16 BIT(13)
-
-/* Structure of the message buffer */
-struct flexcan_mb {
- u32 can_ctrl;
- u32 can_id;
- u32 data[];
-};
-
-/* Structure of the hardware registers */
-struct flexcan_regs {
- u32 mcr; /* 0x00 */
- u32 ctrl; /* 0x04 - Not affected by Soft Reset */
- u32 timer; /* 0x08 */
- u32 tcr; /* 0x0c */
- u32 rxgmask; /* 0x10 - Not affected by Soft Reset */
- u32 rx14mask; /* 0x14 - Not affected by Soft Reset */
- u32 rx15mask; /* 0x18 - Not affected by Soft Reset */
- u32 ecr; /* 0x1c */
- u32 esr; /* 0x20 */
- u32 imask2; /* 0x24 */
- u32 imask1; /* 0x28 */
- u32 iflag2; /* 0x2c */
- u32 iflag1; /* 0x30 */
- union { /* 0x34 */
- u32 gfwr_mx28; /* MX28, MX53 */
- u32 ctrl2; /* MX6, VF610 - Not affected by Soft Reset */
- };
- u32 esr2; /* 0x38 */
- u32 imeur; /* 0x3c */
- u32 lrfr; /* 0x40 */
- u32 crcr; /* 0x44 */
- u32 rxfgmask; /* 0x48 */
- u32 rxfir; /* 0x4c - Not affected by Soft Reset */
- u32 cbt; /* 0x50 - Not affected by Soft Reset */
- u32 _reserved2; /* 0x54 */
- u32 dbg1; /* 0x58 */
- u32 dbg2; /* 0x5c */
- u32 _reserved3[8]; /* 0x60 */
- struct_group(init,
- u8 mb[2][512]; /* 0x80 - Not affected by Soft Reset */
- /* FIFO-mode:
- * MB
- * 0x080...0x08f 0 RX message buffer
- * 0x090...0x0df 1-5 reserved
- * 0x0e0...0x0ff 6-7 8 entry ID table
- * (mx25, mx28, mx35, mx53)
- * 0x0e0...0x2df 6-7..37 8..128 entry ID table
- * size conf'ed via ctrl2::RFFN
- * (mx6, vf610)
- */
- u32 _reserved4[256]; /* 0x480 */
- u32 rximr[64]; /* 0x880 - Not affected by Soft Reset */
- u32 _reserved5[24]; /* 0x980 */
- u32 gfwr_mx6; /* 0x9e0 - MX6 */
- u32 _reserved6[39]; /* 0x9e4 */
- u32 _rxfir[6]; /* 0xa80 */
- u32 _reserved8[2]; /* 0xa98 */
- u32 _rxmgmask; /* 0xaa0 */
- u32 _rxfgmask; /* 0xaa4 */
- u32 _rx14mask; /* 0xaa8 */
- u32 _rx15mask; /* 0xaac */
- u32 tx_smb[4]; /* 0xab0 */
- u32 rx_smb0[4]; /* 0xac0 */
- u32 rx_smb1[4]; /* 0xad0 */
- );
- u32 mecr; /* 0xae0 */
- u32 erriar; /* 0xae4 */
- u32 erridpr; /* 0xae8 */
- u32 errippr; /* 0xaec */
- u32 rerrar; /* 0xaf0 */
- u32 rerrdr; /* 0xaf4 */
- u32 rerrsynr; /* 0xaf8 */
- u32 errsr; /* 0xafc */
- u32 _reserved7[64]; /* 0xb00 */
- u32 fdctrl; /* 0xc00 - Not affected by Soft Reset */
- u32 fdcbt; /* 0xc04 - Not affected by Soft Reset */
- u32 fdcrc; /* 0xc08 */
- u32 _reserved9[199]; /* 0xc0c */
- struct_group(init_fd,
- u32 tx_smb_fd[18]; /* 0xf28 */
- u32 rx_smb0_fd[18]; /* 0xf70 */
- u32 rx_smb1_fd[18]; /* 0xfb8 */
- );
-};
-
-static_assert(sizeof(struct flexcan_regs) == 0x4 * 18 + 0xfb8);
-
-struct flexcan_devtype_data {
- u32 quirks; /* quirks needed for different IP cores */
-};
-
-struct flexcan_stop_mode {
- struct regmap *gpr;
- u8 req_gpr;
- u8 req_bit;
-};
-
-struct flexcan_priv {
- struct can_priv can;
- struct can_rx_offload offload;
- struct device *dev;
-
- struct flexcan_regs __iomem *regs;
- struct flexcan_mb __iomem *tx_mb;
- struct flexcan_mb __iomem *tx_mb_reserved;
- u8 tx_mb_idx;
- u8 mb_count;
- u8 mb_size;
- u8 clk_src; /* clock source of CAN Protocol Engine */
- u8 scu_idx;
-
- u64 rx_mask;
- u64 tx_mask;
- u32 reg_ctrl_default;
-
- struct clk *clk_ipg;
- struct clk *clk_per;
- const struct flexcan_devtype_data *devtype_data;
- struct regulator *reg_xceiver;
- struct flexcan_stop_mode stm;
-
- int irq_boff;
- int irq_err;
-
- /* IPC handle when setup stop mode by System Controller firmware(scfw) */
- struct imx_sc_ipc *sc_ipc_handle;
-
- /* Read and Write APIs */
- u32 (*read)(void __iomem *addr);
- void (*write)(u32 val, void __iomem *addr);
-};
-
-static const struct flexcan_devtype_data fsl_mcf5441x_devtype_data = {
- .quirks = FLEXCAN_QUIRK_BROKEN_PERR_STATE |
- FLEXCAN_QUIRK_NR_IRQ_3 | FLEXCAN_QUIRK_NR_MB_16,
-};
-
-static const struct flexcan_devtype_data fsl_p1010_devtype_data = {
- .quirks = FLEXCAN_QUIRK_BROKEN_WERR_STATE |
- FLEXCAN_QUIRK_BROKEN_PERR_STATE |
- FLEXCAN_QUIRK_DEFAULT_BIG_ENDIAN,
-};
-
-static const struct flexcan_devtype_data fsl_imx25_devtype_data = {
- .quirks = FLEXCAN_QUIRK_BROKEN_WERR_STATE |
- FLEXCAN_QUIRK_BROKEN_PERR_STATE,
-};
-
-static const struct flexcan_devtype_data fsl_imx28_devtype_data = {
- .quirks = FLEXCAN_QUIRK_BROKEN_PERR_STATE,
-};
-
-static const struct flexcan_devtype_data fsl_imx6q_devtype_data = {
- .quirks = FLEXCAN_QUIRK_DISABLE_RXFG | FLEXCAN_QUIRK_ENABLE_EACEN_RRS |
- FLEXCAN_QUIRK_USE_OFF_TIMESTAMP | FLEXCAN_QUIRK_BROKEN_PERR_STATE |
- FLEXCAN_QUIRK_SETUP_STOP_MODE_GPR,
-};
-
-static const struct flexcan_devtype_data fsl_imx8qm_devtype_data = {
- .quirks = FLEXCAN_QUIRK_DISABLE_RXFG | FLEXCAN_QUIRK_ENABLE_EACEN_RRS |
- FLEXCAN_QUIRK_USE_OFF_TIMESTAMP | FLEXCAN_QUIRK_BROKEN_PERR_STATE |
- FLEXCAN_QUIRK_SUPPORT_FD | FLEXCAN_QUIRK_SETUP_STOP_MODE_SCFW,
-};
-
-static struct flexcan_devtype_data fsl_imx8mp_devtype_data = {
- .quirks = FLEXCAN_QUIRK_DISABLE_RXFG | FLEXCAN_QUIRK_ENABLE_EACEN_RRS |
- FLEXCAN_QUIRK_DISABLE_MECR | FLEXCAN_QUIRK_USE_OFF_TIMESTAMP |
- FLEXCAN_QUIRK_BROKEN_PERR_STATE | FLEXCAN_QUIRK_SETUP_STOP_MODE_GPR |
- FLEXCAN_QUIRK_SUPPORT_FD | FLEXCAN_QUIRK_SUPPORT_ECC,
-};
-
-static const struct flexcan_devtype_data fsl_vf610_devtype_data = {
- .quirks = FLEXCAN_QUIRK_DISABLE_RXFG | FLEXCAN_QUIRK_ENABLE_EACEN_RRS |
- FLEXCAN_QUIRK_DISABLE_MECR | FLEXCAN_QUIRK_USE_OFF_TIMESTAMP |
- FLEXCAN_QUIRK_BROKEN_PERR_STATE | FLEXCAN_QUIRK_SUPPORT_ECC,
-};
-
-static const struct flexcan_devtype_data fsl_ls1021a_r2_devtype_data = {
- .quirks = FLEXCAN_QUIRK_DISABLE_RXFG | FLEXCAN_QUIRK_ENABLE_EACEN_RRS |
- FLEXCAN_QUIRK_BROKEN_PERR_STATE | FLEXCAN_QUIRK_USE_OFF_TIMESTAMP,
-};
-
-static const struct flexcan_devtype_data fsl_lx2160a_r1_devtype_data = {
- .quirks = FLEXCAN_QUIRK_DISABLE_RXFG | FLEXCAN_QUIRK_ENABLE_EACEN_RRS |
- FLEXCAN_QUIRK_DISABLE_MECR | FLEXCAN_QUIRK_BROKEN_PERR_STATE |
- FLEXCAN_QUIRK_USE_OFF_TIMESTAMP | FLEXCAN_QUIRK_SUPPORT_FD |
- FLEXCAN_QUIRK_SUPPORT_ECC,
-};
-
-static const struct can_bittiming_const flexcan_bittiming_const = {
- .name = DRV_NAME,
- .tseg1_min = 4,
- .tseg1_max = 16,
- .tseg2_min = 2,
- .tseg2_max = 8,
- .sjw_max = 4,
- .brp_min = 1,
- .brp_max = 256,
- .brp_inc = 1,
-};
-
-static const struct can_bittiming_const flexcan_fd_bittiming_const = {
- .name = DRV_NAME,
- .tseg1_min = 2,
- .tseg1_max = 96,
- .tseg2_min = 2,
- .tseg2_max = 32,
- .sjw_max = 16,
- .brp_min = 1,
- .brp_max = 1024,
- .brp_inc = 1,
-};
-
-static const struct can_bittiming_const flexcan_fd_data_bittiming_const = {
- .name = DRV_NAME,
- .tseg1_min = 2,
- .tseg1_max = 39,
- .tseg2_min = 2,
- .tseg2_max = 8,
- .sjw_max = 4,
- .brp_min = 1,
- .brp_max = 1024,
- .brp_inc = 1,
-};
-
-/* FlexCAN module is essentially modelled as a little-endian IP in most
- * SoCs, i.e the registers as well as the message buffer areas are
- * implemented in a little-endian fashion.
- *
- * However there are some SoCs (e.g. LS1021A) which implement the FlexCAN
- * module in a big-endian fashion (i.e the registers as well as the
- * message buffer areas are implemented in a big-endian way).
- *
- * In addition, the FlexCAN module can be found on SoCs having ARM or
- * PPC cores. So, we need to abstract off the register read/write
- * functions, ensuring that these cater to all the combinations of module
- * endianness and underlying CPU endianness.
- */
-static inline u32 flexcan_read_be(void __iomem *addr)
-{
- return ioread32be(addr);
-}
-
-static inline void flexcan_write_be(u32 val, void __iomem *addr)
-{
- iowrite32be(val, addr);
-}
-
-static inline u32 flexcan_read_le(void __iomem *addr)
-{
- return ioread32(addr);
-}
-
-static inline void flexcan_write_le(u32 val, void __iomem *addr)
-{
- iowrite32(val, addr);
-}
-
-static struct flexcan_mb __iomem *flexcan_get_mb(const struct flexcan_priv *priv,
- u8 mb_index)
-{
- u8 bank_size;
- bool bank;
-
- if (WARN_ON(mb_index >= priv->mb_count))
- return NULL;
-
- bank_size = sizeof(priv->regs->mb[0]) / priv->mb_size;
-
- bank = mb_index >= bank_size;
- if (bank)
- mb_index -= bank_size;
-
- return (struct flexcan_mb __iomem *)
- (&priv->regs->mb[bank][priv->mb_size * mb_index]);
-}
-
-static int flexcan_low_power_enter_ack(struct flexcan_priv *priv)
-{
- struct flexcan_regs __iomem *regs = priv->regs;
- unsigned int timeout = FLEXCAN_TIMEOUT_US / 10;
-
- while (timeout-- && !(priv->read(®s->mcr) & FLEXCAN_MCR_LPM_ACK))
- udelay(10);
-
- if (!(priv->read(®s->mcr) & FLEXCAN_MCR_LPM_ACK))
- return -ETIMEDOUT;
-
- return 0;
-}
-
-static int flexcan_low_power_exit_ack(struct flexcan_priv *priv)
-{
- struct flexcan_regs __iomem *regs = priv->regs;
- unsigned int timeout = FLEXCAN_TIMEOUT_US / 10;
-
- while (timeout-- && (priv->read(®s->mcr) & FLEXCAN_MCR_LPM_ACK))
- udelay(10);
-
- if (priv->read(®s->mcr) & FLEXCAN_MCR_LPM_ACK)
- return -ETIMEDOUT;
-
- return 0;
-}
-
-static void flexcan_enable_wakeup_irq(struct flexcan_priv *priv, bool enable)
-{
- struct flexcan_regs __iomem *regs = priv->regs;
- u32 reg_mcr;
-
- reg_mcr = priv->read(®s->mcr);
-
- if (enable)
- reg_mcr |= FLEXCAN_MCR_WAK_MSK;
- else
- reg_mcr &= ~FLEXCAN_MCR_WAK_MSK;
-
- priv->write(reg_mcr, ®s->mcr);
-}
-
-static int flexcan_stop_mode_enable_scfw(struct flexcan_priv *priv, bool enabled)
-{
- u8 idx = priv->scu_idx;
- u32 rsrc_id, val;
-
- rsrc_id = IMX_SC_R_CAN(idx);
-
- if (enabled)
- val = 1;
- else
- val = 0;
-
- /* stop mode request via scu firmware */
- return imx_sc_misc_set_control(priv->sc_ipc_handle, rsrc_id,
- IMX_SC_C_IPG_STOP, val);
-}
-
-static inline int flexcan_enter_stop_mode(struct flexcan_priv *priv)
-{
- struct flexcan_regs __iomem *regs = priv->regs;
- u32 reg_mcr;
- int ret;
-
- reg_mcr = priv->read(®s->mcr);
- reg_mcr |= FLEXCAN_MCR_SLF_WAK;
- priv->write(reg_mcr, ®s->mcr);
-
- /* enable stop request */
- if (priv->devtype_data->quirks & FLEXCAN_QUIRK_SETUP_STOP_MODE_SCFW) {
- ret = flexcan_stop_mode_enable_scfw(priv, true);
- if (ret < 0)
- return ret;
- } else {
- regmap_update_bits(priv->stm.gpr, priv->stm.req_gpr,
- 1 << priv->stm.req_bit, 1 << priv->stm.req_bit);
- }
-
- return flexcan_low_power_enter_ack(priv);
-}
-
-static inline int flexcan_exit_stop_mode(struct flexcan_priv *priv)
-{
- struct flexcan_regs __iomem *regs = priv->regs;
- u32 reg_mcr;
- int ret;
-
- /* remove stop request */
- if (priv->devtype_data->quirks & FLEXCAN_QUIRK_SETUP_STOP_MODE_SCFW) {
- ret = flexcan_stop_mode_enable_scfw(priv, false);
- if (ret < 0)
- return ret;
- } else {
- regmap_update_bits(priv->stm.gpr, priv->stm.req_gpr,
- 1 << priv->stm.req_bit, 0);
- }
-
- reg_mcr = priv->read(®s->mcr);
- reg_mcr &= ~FLEXCAN_MCR_SLF_WAK;
- priv->write(reg_mcr, ®s->mcr);
-
- return flexcan_low_power_exit_ack(priv);
-}
-
-static inline void flexcan_error_irq_enable(const struct flexcan_priv *priv)
-{
- struct flexcan_regs __iomem *regs = priv->regs;
- u32 reg_ctrl = (priv->reg_ctrl_default | FLEXCAN_CTRL_ERR_MSK);
-
- priv->write(reg_ctrl, ®s->ctrl);
-}
-
-static inline void flexcan_error_irq_disable(const struct flexcan_priv *priv)
-{
- struct flexcan_regs __iomem *regs = priv->regs;
- u32 reg_ctrl = (priv->reg_ctrl_default & ~FLEXCAN_CTRL_ERR_MSK);
-
- priv->write(reg_ctrl, ®s->ctrl);
-}
-
-static int flexcan_clks_enable(const struct flexcan_priv *priv)
-{
- int err = 0;
-
- if (priv->clk_ipg) {
- err = clk_prepare_enable(priv->clk_ipg);
- if (err)
- return err;
- }
-
- if (priv->clk_per) {
- err = clk_prepare_enable(priv->clk_per);
- if (err)
- clk_disable_unprepare(priv->clk_ipg);
- }
-
- return err;
-}
-
-static void flexcan_clks_disable(const struct flexcan_priv *priv)
-{
- clk_disable_unprepare(priv->clk_per);
- clk_disable_unprepare(priv->clk_ipg);
-}
-
-static inline int flexcan_transceiver_enable(const struct flexcan_priv *priv)
-{
- if (!priv->reg_xceiver)
- return 0;
-
- return regulator_enable(priv->reg_xceiver);
-}
-
-static inline int flexcan_transceiver_disable(const struct flexcan_priv *priv)
-{
- if (!priv->reg_xceiver)
- return 0;
-
- return regulator_disable(priv->reg_xceiver);
-}
-
-static int flexcan_chip_enable(struct flexcan_priv *priv)
-{
- struct flexcan_regs __iomem *regs = priv->regs;
- u32 reg;
-
- reg = priv->read(®s->mcr);
- reg &= ~FLEXCAN_MCR_MDIS;
- priv->write(reg, ®s->mcr);
-
- return flexcan_low_power_exit_ack(priv);
-}
-
-static int flexcan_chip_disable(struct flexcan_priv *priv)
-{
- struct flexcan_regs __iomem *regs = priv->regs;
- u32 reg;
-
- reg = priv->read(®s->mcr);
- reg |= FLEXCAN_MCR_MDIS;
- priv->write(reg, ®s->mcr);
-
- return flexcan_low_power_enter_ack(priv);
-}
-
-static int flexcan_chip_freeze(struct flexcan_priv *priv)
-{
- struct flexcan_regs __iomem *regs = priv->regs;
- unsigned int timeout;
- u32 bitrate = priv->can.bittiming.bitrate;
- u32 reg;
-
- if (bitrate)
- timeout = 1000 * 1000 * 10 / bitrate;
- else
- timeout = FLEXCAN_TIMEOUT_US / 10;
-
- reg = priv->read(®s->mcr);
- reg |= FLEXCAN_MCR_FRZ | FLEXCAN_MCR_HALT;
- priv->write(reg, ®s->mcr);
-
- while (timeout-- && !(priv->read(®s->mcr) & FLEXCAN_MCR_FRZ_ACK))
- udelay(100);
-
- if (!(priv->read(®s->mcr) & FLEXCAN_MCR_FRZ_ACK))
- return -ETIMEDOUT;
-
- return 0;
-}
-
-static int flexcan_chip_unfreeze(struct flexcan_priv *priv)
-{
- struct flexcan_regs __iomem *regs = priv->regs;
- unsigned int timeout = FLEXCAN_TIMEOUT_US / 10;
- u32 reg;
-
- reg = priv->read(®s->mcr);
- reg &= ~FLEXCAN_MCR_HALT;
- priv->write(reg, ®s->mcr);
-
- while (timeout-- && (priv->read(®s->mcr) & FLEXCAN_MCR_FRZ_ACK))
- udelay(10);
-
- if (priv->read(®s->mcr) & FLEXCAN_MCR_FRZ_ACK)
- return -ETIMEDOUT;
-
- return 0;
-}
-
-static int flexcan_chip_softreset(struct flexcan_priv *priv)
-{
- struct flexcan_regs __iomem *regs = priv->regs;
- unsigned int timeout = FLEXCAN_TIMEOUT_US / 10;
-
- priv->write(FLEXCAN_MCR_SOFTRST, ®s->mcr);
- while (timeout-- && (priv->read(®s->mcr) & FLEXCAN_MCR_SOFTRST))
- udelay(10);
-
- if (priv->read(®s->mcr) & FLEXCAN_MCR_SOFTRST)
- return -ETIMEDOUT;
-
- return 0;
-}
-
-static int __flexcan_get_berr_counter(const struct net_device *dev,
- struct can_berr_counter *bec)
-{
- const struct flexcan_priv *priv = netdev_priv(dev);
- struct flexcan_regs __iomem *regs = priv->regs;
- u32 reg = priv->read(®s->ecr);
-
- bec->txerr = (reg >> 0) & 0xff;
- bec->rxerr = (reg >> 8) & 0xff;
-
- return 0;
-}
-
-static int flexcan_get_berr_counter(const struct net_device *dev,
- struct can_berr_counter *bec)
-{
- const struct flexcan_priv *priv = netdev_priv(dev);
- int err;
-
- err = pm_runtime_get_sync(priv->dev);
- if (err < 0) {
- pm_runtime_put_noidle(priv->dev);
- return err;
- }
-
- err = __flexcan_get_berr_counter(dev, bec);
-
- pm_runtime_put(priv->dev);
-
- return err;
-}
-
-static netdev_tx_t flexcan_start_xmit(struct sk_buff *skb, struct net_device *dev)
-{
- const struct flexcan_priv *priv = netdev_priv(dev);
- struct canfd_frame *cfd = (struct canfd_frame *)skb->data;
- u32 can_id;
- u32 data;
- u32 ctrl = FLEXCAN_MB_CODE_TX_DATA | ((can_fd_len2dlc(cfd->len)) << 16);
- int i;
-
- if (can_dropped_invalid_skb(dev, skb))
- return NETDEV_TX_OK;
-
- netif_stop_queue(dev);
-
- if (cfd->can_id & CAN_EFF_FLAG) {
- can_id = cfd->can_id & CAN_EFF_MASK;
- ctrl |= FLEXCAN_MB_CNT_IDE | FLEXCAN_MB_CNT_SRR;
- } else {
- can_id = (cfd->can_id & CAN_SFF_MASK) << 18;
- }
-
- if (cfd->can_id & CAN_RTR_FLAG)
- ctrl |= FLEXCAN_MB_CNT_RTR;
-
- if (can_is_canfd_skb(skb)) {
- ctrl |= FLEXCAN_MB_CNT_EDL;
-
- if (cfd->flags & CANFD_BRS)
- ctrl |= FLEXCAN_MB_CNT_BRS;
- }
-
- for (i = 0; i < cfd->len; i += sizeof(u32)) {
- data = be32_to_cpup((__be32 *)&cfd->data[i]);
- priv->write(data, &priv->tx_mb->data[i / sizeof(u32)]);
- }
-
- can_put_echo_skb(skb, dev, 0, 0);
-
- priv->write(can_id, &priv->tx_mb->can_id);
- priv->write(ctrl, &priv->tx_mb->can_ctrl);
-
- /* Errata ERR005829 step8:
- * Write twice INACTIVE(0x8) code to first MB.
- */
- priv->write(FLEXCAN_MB_CODE_TX_INACTIVE,
- &priv->tx_mb_reserved->can_ctrl);
- priv->write(FLEXCAN_MB_CODE_TX_INACTIVE,
- &priv->tx_mb_reserved->can_ctrl);
-
- return NETDEV_TX_OK;
-}
-
-static void flexcan_irq_bus_err(struct net_device *dev, u32 reg_esr)
-{
- struct flexcan_priv *priv = netdev_priv(dev);
- struct flexcan_regs __iomem *regs = priv->regs;
- struct sk_buff *skb;
- struct can_frame *cf;
- bool rx_errors = false, tx_errors = false;
- u32 timestamp;
- int err;
-
- timestamp = priv->read(®s->timer) << 16;
-
- skb = alloc_can_err_skb(dev, &cf);
- if (unlikely(!skb))
- return;
-
- cf->can_id |= CAN_ERR_PROT | CAN_ERR_BUSERROR;
-
- if (reg_esr & FLEXCAN_ESR_BIT1_ERR) {
- netdev_dbg(dev, "BIT1_ERR irq\n");
- cf->data[2] |= CAN_ERR_PROT_BIT1;
- tx_errors = true;
- }
- if (reg_esr & FLEXCAN_ESR_BIT0_ERR) {
- netdev_dbg(dev, "BIT0_ERR irq\n");
- cf->data[2] |= CAN_ERR_PROT_BIT0;
- tx_errors = true;
- }
- if (reg_esr & FLEXCAN_ESR_ACK_ERR) {
- netdev_dbg(dev, "ACK_ERR irq\n");
- cf->can_id |= CAN_ERR_ACK;
- cf->data[3] = CAN_ERR_PROT_LOC_ACK;
- tx_errors = true;
- }
- if (reg_esr & FLEXCAN_ESR_CRC_ERR) {
- netdev_dbg(dev, "CRC_ERR irq\n");
- cf->data[2] |= CAN_ERR_PROT_BIT;
- cf->data[3] = CAN_ERR_PROT_LOC_CRC_SEQ;
- rx_errors = true;
- }
- if (reg_esr & FLEXCAN_ESR_FRM_ERR) {
- netdev_dbg(dev, "FRM_ERR irq\n");
- cf->data[2] |= CAN_ERR_PROT_FORM;
- rx_errors = true;
- }
- if (reg_esr & FLEXCAN_ESR_STF_ERR) {
- netdev_dbg(dev, "STF_ERR irq\n");
- cf->data[2] |= CAN_ERR_PROT_STUFF;
- rx_errors = true;
- }
-
- priv->can.can_stats.bus_error++;
- if (rx_errors)
- dev->stats.rx_errors++;
- if (tx_errors)
- dev->stats.tx_errors++;
-
- err = can_rx_offload_queue_sorted(&priv->offload, skb, timestamp);
- if (err)
- dev->stats.rx_fifo_errors++;
-}
-
-static void flexcan_irq_state(struct net_device *dev, u32 reg_esr)
-{
- struct flexcan_priv *priv = netdev_priv(dev);
- struct flexcan_regs __iomem *regs = priv->regs;
- struct sk_buff *skb;
- struct can_frame *cf;
- enum can_state new_state, rx_state, tx_state;
- int flt;
- struct can_berr_counter bec;
- u32 timestamp;
- int err;
-
- flt = reg_esr & FLEXCAN_ESR_FLT_CONF_MASK;
- if (likely(flt == FLEXCAN_ESR_FLT_CONF_ACTIVE)) {
- tx_state = unlikely(reg_esr & FLEXCAN_ESR_TX_WRN) ?
- CAN_STATE_ERROR_WARNING : CAN_STATE_ERROR_ACTIVE;
- rx_state = unlikely(reg_esr & FLEXCAN_ESR_RX_WRN) ?
- CAN_STATE_ERROR_WARNING : CAN_STATE_ERROR_ACTIVE;
- new_state = max(tx_state, rx_state);
- } else {
- __flexcan_get_berr_counter(dev, &bec);
- new_state = flt == FLEXCAN_ESR_FLT_CONF_PASSIVE ?
- CAN_STATE_ERROR_PASSIVE : CAN_STATE_BUS_OFF;
- rx_state = bec.rxerr >= bec.txerr ? new_state : 0;
- tx_state = bec.rxerr <= bec.txerr ? new_state : 0;
- }
-
- /* state hasn't changed */
- if (likely(new_state == priv->can.state))
- return;
-
- timestamp = priv->read(®s->timer) << 16;
-
- skb = alloc_can_err_skb(dev, &cf);
- if (unlikely(!skb))
- return;
-
- can_change_state(dev, cf, tx_state, rx_state);
-
- if (unlikely(new_state == CAN_STATE_BUS_OFF))
- can_bus_off(dev);
-
- err = can_rx_offload_queue_sorted(&priv->offload, skb, timestamp);
- if (err)
- dev->stats.rx_fifo_errors++;
-}
-
-static inline u64 flexcan_read64_mask(struct flexcan_priv *priv, void __iomem *addr, u64 mask)
-{
- u64 reg = 0;
-
- if (upper_32_bits(mask))
- reg = (u64)priv->read(addr - 4) << 32;
- if (lower_32_bits(mask))
- reg |= priv->read(addr);
-
- return reg & mask;
-}
-
-static inline void flexcan_write64(struct flexcan_priv *priv, u64 val, void __iomem *addr)
-{
- if (upper_32_bits(val))
- priv->write(upper_32_bits(val), addr - 4);
- if (lower_32_bits(val))
- priv->write(lower_32_bits(val), addr);
-}
-
-static inline u64 flexcan_read_reg_iflag_rx(struct flexcan_priv *priv)
-{
- return flexcan_read64_mask(priv, &priv->regs->iflag1, priv->rx_mask);
-}
-
-static inline u64 flexcan_read_reg_iflag_tx(struct flexcan_priv *priv)
-{
- return flexcan_read64_mask(priv, &priv->regs->iflag1, priv->tx_mask);
-}
-
-static inline struct flexcan_priv *rx_offload_to_priv(struct can_rx_offload *offload)
-{
- return container_of(offload, struct flexcan_priv, offload);
-}
-
-static struct sk_buff *flexcan_mailbox_read(struct can_rx_offload *offload,
- unsigned int n, u32 *timestamp,
- bool drop)
-{
- struct flexcan_priv *priv = rx_offload_to_priv(offload);
- struct flexcan_regs __iomem *regs = priv->regs;
- struct flexcan_mb __iomem *mb;
- struct sk_buff *skb;
- struct canfd_frame *cfd;
- u32 reg_ctrl, reg_id, reg_iflag1;
- int i;
-
- if (unlikely(drop)) {
- skb = ERR_PTR(-ENOBUFS);
- goto mark_as_read;
- }
-
- mb = flexcan_get_mb(priv, n);
-
- if (priv->devtype_data->quirks & FLEXCAN_QUIRK_USE_OFF_TIMESTAMP) {
- u32 code;
-
- do {
- reg_ctrl = priv->read(&mb->can_ctrl);
- } while (reg_ctrl & FLEXCAN_MB_CODE_RX_BUSY_BIT);
-
- /* is this MB empty? */
- code = reg_ctrl & FLEXCAN_MB_CODE_MASK;
- if ((code != FLEXCAN_MB_CODE_RX_FULL) &&
- (code != FLEXCAN_MB_CODE_RX_OVERRUN))
- return NULL;
-
- if (code == FLEXCAN_MB_CODE_RX_OVERRUN) {
- /* This MB was overrun, we lost data */
- offload->dev->stats.rx_over_errors++;
- offload->dev->stats.rx_errors++;
- }
- } else {
- reg_iflag1 = priv->read(®s->iflag1);
- if (!(reg_iflag1 & FLEXCAN_IFLAG_RX_FIFO_AVAILABLE))
- return NULL;
-
- reg_ctrl = priv->read(&mb->can_ctrl);
- }
-
- if (reg_ctrl & FLEXCAN_MB_CNT_EDL)
- skb = alloc_canfd_skb(offload->dev, &cfd);
- else
- skb = alloc_can_skb(offload->dev, (struct can_frame **)&cfd);
- if (unlikely(!skb)) {
- skb = ERR_PTR(-ENOMEM);
- goto mark_as_read;
- }
-
- /* increase timstamp to full 32 bit */
- *timestamp = reg_ctrl << 16;
-
- reg_id = priv->read(&mb->can_id);
- if (reg_ctrl & FLEXCAN_MB_CNT_IDE)
- cfd->can_id = ((reg_id >> 0) & CAN_EFF_MASK) | CAN_EFF_FLAG;
- else
- cfd->can_id = (reg_id >> 18) & CAN_SFF_MASK;
-
- if (reg_ctrl & FLEXCAN_MB_CNT_EDL) {
- cfd->len = can_fd_dlc2len((reg_ctrl >> 16) & 0xf);
-
- if (reg_ctrl & FLEXCAN_MB_CNT_BRS)
- cfd->flags |= CANFD_BRS;
- } else {
- cfd->len = can_cc_dlc2len((reg_ctrl >> 16) & 0xf);
-
- if (reg_ctrl & FLEXCAN_MB_CNT_RTR)
- cfd->can_id |= CAN_RTR_FLAG;
- }
-
- if (reg_ctrl & FLEXCAN_MB_CNT_ESI)
- cfd->flags |= CANFD_ESI;
-
- for (i = 0; i < cfd->len; i += sizeof(u32)) {
- __be32 data = cpu_to_be32(priv->read(&mb->data[i / sizeof(u32)]));
- *(__be32 *)(cfd->data + i) = data;
- }
-
- mark_as_read:
- if (priv->devtype_data->quirks & FLEXCAN_QUIRK_USE_OFF_TIMESTAMP)
- flexcan_write64(priv, FLEXCAN_IFLAG_MB(n), ®s->iflag1);
- else
- priv->write(FLEXCAN_IFLAG_RX_FIFO_AVAILABLE, ®s->iflag1);
-
- /* Read the Free Running Timer. It is optional but recommended
- * to unlock Mailbox as soon as possible and make it available
- * for reception.
- */
- priv->read(®s->timer);
-
- return skb;
-}
-
-static irqreturn_t flexcan_irq(int irq, void *dev_id)
-{
- struct net_device *dev = dev_id;
- struct net_device_stats *stats = &dev->stats;
- struct flexcan_priv *priv = netdev_priv(dev);
- struct flexcan_regs __iomem *regs = priv->regs;
- irqreturn_t handled = IRQ_NONE;
- u64 reg_iflag_tx;
- u32 reg_esr;
- enum can_state last_state = priv->can.state;
-
- /* reception interrupt */
- if (priv->devtype_data->quirks & FLEXCAN_QUIRK_USE_OFF_TIMESTAMP) {
- u64 reg_iflag_rx;
- int ret;
-
- while ((reg_iflag_rx = flexcan_read_reg_iflag_rx(priv))) {
- handled = IRQ_HANDLED;
- ret = can_rx_offload_irq_offload_timestamp(&priv->offload,
- reg_iflag_rx);
- if (!ret)
- break;
- }
- } else {
- u32 reg_iflag1;
-
- reg_iflag1 = priv->read(®s->iflag1);
- if (reg_iflag1 & FLEXCAN_IFLAG_RX_FIFO_AVAILABLE) {
- handled = IRQ_HANDLED;
- can_rx_offload_irq_offload_fifo(&priv->offload);
- }
-
- /* FIFO overflow interrupt */
- if (reg_iflag1 & FLEXCAN_IFLAG_RX_FIFO_OVERFLOW) {
- handled = IRQ_HANDLED;
- priv->write(FLEXCAN_IFLAG_RX_FIFO_OVERFLOW,
- ®s->iflag1);
- dev->stats.rx_over_errors++;
- dev->stats.rx_errors++;
- }
- }
-
- reg_iflag_tx = flexcan_read_reg_iflag_tx(priv);
-
- /* transmission complete interrupt */
- if (reg_iflag_tx & priv->tx_mask) {
- u32 reg_ctrl = priv->read(&priv->tx_mb->can_ctrl);
-
- handled = IRQ_HANDLED;
- stats->tx_bytes +=
- can_rx_offload_get_echo_skb(&priv->offload, 0,
- reg_ctrl << 16, NULL);
- stats->tx_packets++;
- can_led_event(dev, CAN_LED_EVENT_TX);
-
- /* after sending a RTR frame MB is in RX mode */
- priv->write(FLEXCAN_MB_CODE_TX_INACTIVE,
- &priv->tx_mb->can_ctrl);
- flexcan_write64(priv, priv->tx_mask, ®s->iflag1);
- netif_wake_queue(dev);
- }
-
- reg_esr = priv->read(®s->esr);
-
- /* ACK all bus error, state change and wake IRQ sources */
- if (reg_esr & (FLEXCAN_ESR_ALL_INT | FLEXCAN_ESR_WAK_INT)) {
- handled = IRQ_HANDLED;
- priv->write(reg_esr & (FLEXCAN_ESR_ALL_INT | FLEXCAN_ESR_WAK_INT), ®s->esr);
- }
-
- /* state change interrupt or broken error state quirk fix is enabled */
- if ((reg_esr & FLEXCAN_ESR_ERR_STATE) ||
- (priv->devtype_data->quirks & (FLEXCAN_QUIRK_BROKEN_WERR_STATE |
- FLEXCAN_QUIRK_BROKEN_PERR_STATE)))
- flexcan_irq_state(dev, reg_esr);
-
- /* bus error IRQ - handle if bus error reporting is activated */
- if ((reg_esr & FLEXCAN_ESR_ERR_BUS) &&
- (priv->can.ctrlmode & CAN_CTRLMODE_BERR_REPORTING))
- flexcan_irq_bus_err(dev, reg_esr);
-
- /* availability of error interrupt among state transitions in case
- * bus error reporting is de-activated and
- * FLEXCAN_QUIRK_BROKEN_PERR_STATE is enabled:
- * +--------------------------------------------------------------+
- * | +----------------------------------------------+ [stopped / |
- * | | | sleeping] -+
- * +-+-> active <-> warning <-> passive -> bus off -+
- * ___________^^^^^^^^^^^^_______________________________
- * disabled(1) enabled disabled
- *
- * (1): enabled if FLEXCAN_QUIRK_BROKEN_WERR_STATE is enabled
- */
- if ((last_state != priv->can.state) &&
- (priv->devtype_data->quirks & FLEXCAN_QUIRK_BROKEN_PERR_STATE) &&
- !(priv->can.ctrlmode & CAN_CTRLMODE_BERR_REPORTING)) {
- switch (priv->can.state) {
- case CAN_STATE_ERROR_ACTIVE:
- if (priv->devtype_data->quirks &
- FLEXCAN_QUIRK_BROKEN_WERR_STATE)
- flexcan_error_irq_enable(priv);
- else
- flexcan_error_irq_disable(priv);
- break;
-
- case CAN_STATE_ERROR_WARNING:
- flexcan_error_irq_enable(priv);
- break;
-
- case CAN_STATE_ERROR_PASSIVE:
- case CAN_STATE_BUS_OFF:
- flexcan_error_irq_disable(priv);
- break;
-
- default:
- break;
- }
- }
-
- if (handled)
- can_rx_offload_irq_finish(&priv->offload);
-
- return handled;
-}
-
-static void flexcan_set_bittiming_ctrl(const struct net_device *dev)
-{
- const struct flexcan_priv *priv = netdev_priv(dev);
- const struct can_bittiming *bt = &priv->can.bittiming;
- struct flexcan_regs __iomem *regs = priv->regs;
- u32 reg;
-
- reg = priv->read(®s->ctrl);
- reg &= ~(FLEXCAN_CTRL_PRESDIV(0xff) |
- FLEXCAN_CTRL_RJW(0x3) |
- FLEXCAN_CTRL_PSEG1(0x7) |
- FLEXCAN_CTRL_PSEG2(0x7) |
- FLEXCAN_CTRL_PROPSEG(0x7));
-
- reg |= FLEXCAN_CTRL_PRESDIV(bt->brp - 1) |
- FLEXCAN_CTRL_PSEG1(bt->phase_seg1 - 1) |
- FLEXCAN_CTRL_PSEG2(bt->phase_seg2 - 1) |
- FLEXCAN_CTRL_RJW(bt->sjw - 1) |
- FLEXCAN_CTRL_PROPSEG(bt->prop_seg - 1);
-
- netdev_dbg(dev, "writing ctrl=0x%08x\n", reg);
- priv->write(reg, ®s->ctrl);
-
- /* print chip status */
- netdev_dbg(dev, "%s: mcr=0x%08x ctrl=0x%08x\n", __func__,
- priv->read(®s->mcr), priv->read(®s->ctrl));
-}
-
-static void flexcan_set_bittiming_cbt(const struct net_device *dev)
-{
- struct flexcan_priv *priv = netdev_priv(dev);
- struct can_bittiming *bt = &priv->can.bittiming;
- struct can_bittiming *dbt = &priv->can.data_bittiming;
- struct flexcan_regs __iomem *regs = priv->regs;
- u32 reg_cbt, reg_fdctrl;
-
- /* CBT */
- /* CBT[EPSEG1] is 5 bit long and CBT[EPROPSEG] is 6 bit
- * long. The can_calc_bittiming() tries to divide the tseg1
- * equally between phase_seg1 and prop_seg, which may not fit
- * in CBT register. Therefore, if phase_seg1 is more than
- * possible value, increase prop_seg and decrease phase_seg1.
- */
- if (bt->phase_seg1 > 0x20) {
- bt->prop_seg += (bt->phase_seg1 - 0x20);
- bt->phase_seg1 = 0x20;
- }
-
- reg_cbt = FLEXCAN_CBT_BTF |
- FIELD_PREP(FLEXCAN_CBT_EPRESDIV_MASK, bt->brp - 1) |
- FIELD_PREP(FLEXCAN_CBT_ERJW_MASK, bt->sjw - 1) |
- FIELD_PREP(FLEXCAN_CBT_EPROPSEG_MASK, bt->prop_seg - 1) |
- FIELD_PREP(FLEXCAN_CBT_EPSEG1_MASK, bt->phase_seg1 - 1) |
- FIELD_PREP(FLEXCAN_CBT_EPSEG2_MASK, bt->phase_seg2 - 1);
-
- netdev_dbg(dev, "writing cbt=0x%08x\n", reg_cbt);
- priv->write(reg_cbt, ®s->cbt);
-
- if (priv->can.ctrlmode & CAN_CTRLMODE_FD) {
- u32 reg_fdcbt, reg_ctrl2;
-
- if (bt->brp != dbt->brp)
- netdev_warn(dev, "Data brp=%d and brp=%d don't match, this may result in a phase error. Consider using different bitrate and/or data bitrate.\n",
- dbt->brp, bt->brp);
-
- /* FDCBT */
- /* FDCBT[FPSEG1] is 3 bit long and FDCBT[FPROPSEG] is
- * 5 bit long. The can_calc_bittiming tries to divide
- * the tseg1 equally between phase_seg1 and prop_seg,
- * which may not fit in FDCBT register. Therefore, if
- * phase_seg1 is more than possible value, increase
- * prop_seg and decrease phase_seg1
- */
- if (dbt->phase_seg1 > 0x8) {
- dbt->prop_seg += (dbt->phase_seg1 - 0x8);
- dbt->phase_seg1 = 0x8;
- }
-
- reg_fdcbt = priv->read(®s->fdcbt);
- reg_fdcbt &= ~(FIELD_PREP(FLEXCAN_FDCBT_FPRESDIV_MASK, 0x3ff) |
- FIELD_PREP(FLEXCAN_FDCBT_FRJW_MASK, 0x7) |
- FIELD_PREP(FLEXCAN_FDCBT_FPROPSEG_MASK, 0x1f) |
- FIELD_PREP(FLEXCAN_FDCBT_FPSEG1_MASK, 0x7) |
- FIELD_PREP(FLEXCAN_FDCBT_FPSEG2_MASK, 0x7));
-
- reg_fdcbt |= FIELD_PREP(FLEXCAN_FDCBT_FPRESDIV_MASK, dbt->brp - 1) |
- FIELD_PREP(FLEXCAN_FDCBT_FRJW_MASK, dbt->sjw - 1) |
- FIELD_PREP(FLEXCAN_FDCBT_FPROPSEG_MASK, dbt->prop_seg) |
- FIELD_PREP(FLEXCAN_FDCBT_FPSEG1_MASK, dbt->phase_seg1 - 1) |
- FIELD_PREP(FLEXCAN_FDCBT_FPSEG2_MASK, dbt->phase_seg2 - 1);
-
- netdev_dbg(dev, "writing fdcbt=0x%08x\n", reg_fdcbt);
- priv->write(reg_fdcbt, ®s->fdcbt);
-
- /* CTRL2 */
- reg_ctrl2 = priv->read(®s->ctrl2);
- reg_ctrl2 &= ~FLEXCAN_CTRL2_ISOCANFDEN;
- if (!(priv->can.ctrlmode & CAN_CTRLMODE_FD_NON_ISO))
- reg_ctrl2 |= FLEXCAN_CTRL2_ISOCANFDEN;
-
- netdev_dbg(dev, "writing ctrl2=0x%08x\n", reg_ctrl2);
- priv->write(reg_ctrl2, ®s->ctrl2);
- }
-
- /* FDCTRL */
- reg_fdctrl = priv->read(®s->fdctrl);
- reg_fdctrl &= ~(FLEXCAN_FDCTRL_FDRATE |
- FIELD_PREP(FLEXCAN_FDCTRL_TDCOFF, 0x1f));
-
- if (priv->can.ctrlmode & CAN_CTRLMODE_FD) {
- reg_fdctrl |= FLEXCAN_FDCTRL_FDRATE;
-
- if (priv->can.ctrlmode & CAN_CTRLMODE_LOOPBACK) {
- /* TDC must be disabled for Loop Back mode */
- reg_fdctrl &= ~FLEXCAN_FDCTRL_TDCEN;
- } else {
- reg_fdctrl |= FLEXCAN_FDCTRL_TDCEN |
- FIELD_PREP(FLEXCAN_FDCTRL_TDCOFF,
- ((dbt->phase_seg1 - 1) +
- dbt->prop_seg + 2) *
- ((dbt->brp - 1 ) + 1));
- }
- }
-
- netdev_dbg(dev, "writing fdctrl=0x%08x\n", reg_fdctrl);
- priv->write(reg_fdctrl, ®s->fdctrl);
-
- netdev_dbg(dev, "%s: mcr=0x%08x ctrl=0x%08x ctrl2=0x%08x fdctrl=0x%08x cbt=0x%08x fdcbt=0x%08x\n",
- __func__,
- priv->read(®s->mcr), priv->read(®s->ctrl),
- priv->read(®s->ctrl2), priv->read(®s->fdctrl),
- priv->read(®s->cbt), priv->read(®s->fdcbt));
-}
-
-static void flexcan_set_bittiming(struct net_device *dev)
-{
- const struct flexcan_priv *priv = netdev_priv(dev);
- struct flexcan_regs __iomem *regs = priv->regs;
- u32 reg;
-
- reg = priv->read(®s->ctrl);
- reg &= ~(FLEXCAN_CTRL_LPB | FLEXCAN_CTRL_SMP |
- FLEXCAN_CTRL_LOM);
-
- if (priv->can.ctrlmode & CAN_CTRLMODE_LOOPBACK)
- reg |= FLEXCAN_CTRL_LPB;
- if (priv->can.ctrlmode & CAN_CTRLMODE_LISTENONLY)
- reg |= FLEXCAN_CTRL_LOM;
- if (priv->can.ctrlmode & CAN_CTRLMODE_3_SAMPLES)
- reg |= FLEXCAN_CTRL_SMP;
-
- netdev_dbg(dev, "writing ctrl=0x%08x\n", reg);
- priv->write(reg, ®s->ctrl);
-
- if (priv->can.ctrlmode_supported & CAN_CTRLMODE_FD)
- return flexcan_set_bittiming_cbt(dev);
- else
- return flexcan_set_bittiming_ctrl(dev);
-}
-
-static void flexcan_ram_init(struct net_device *dev)
-{
- struct flexcan_priv *priv = netdev_priv(dev);
- struct flexcan_regs __iomem *regs = priv->regs;
- u32 reg_ctrl2;
-
- /* 11.8.3.13 Detection and correction of memory errors:
- * CTRL2[WRMFRZ] grants write access to all memory positions
- * that require initialization, ranging from 0x080 to 0xADF
- * and from 0xF28 to 0xFFF when the CAN FD feature is enabled.
- * The RXMGMASK, RX14MASK, RX15MASK, and RXFGMASK registers
- * need to be initialized as well. MCR[RFEN] must not be set
- * during memory initialization.
- */
- reg_ctrl2 = priv->read(®s->ctrl2);
- reg_ctrl2 |= FLEXCAN_CTRL2_WRMFRZ;
- priv->write(reg_ctrl2, ®s->ctrl2);
-
- memset_io(®s->init, 0, sizeof(regs->init));
-
- if (priv->can.ctrlmode & CAN_CTRLMODE_FD)
- memset_io(®s->init_fd, 0, sizeof(regs->init_fd));
-
- reg_ctrl2 &= ~FLEXCAN_CTRL2_WRMFRZ;
- priv->write(reg_ctrl2, ®s->ctrl2);
-}
-
-static int flexcan_rx_offload_setup(struct net_device *dev)
-{
- struct flexcan_priv *priv = netdev_priv(dev);
- int err;
-
- if (priv->can.ctrlmode & CAN_CTRLMODE_FD)
- priv->mb_size = sizeof(struct flexcan_mb) + CANFD_MAX_DLEN;
- else
- priv->mb_size = sizeof(struct flexcan_mb) + CAN_MAX_DLEN;
-
- if (priv->devtype_data->quirks & FLEXCAN_QUIRK_NR_MB_16)
- priv->mb_count = 16;
- else
- priv->mb_count = (sizeof(priv->regs->mb[0]) / priv->mb_size) +
- (sizeof(priv->regs->mb[1]) / priv->mb_size);
-
- if (priv->devtype_data->quirks & FLEXCAN_QUIRK_USE_OFF_TIMESTAMP)
- priv->tx_mb_reserved =
- flexcan_get_mb(priv, FLEXCAN_TX_MB_RESERVED_OFF_TIMESTAMP);
- else
- priv->tx_mb_reserved =
- flexcan_get_mb(priv, FLEXCAN_TX_MB_RESERVED_OFF_FIFO);
- priv->tx_mb_idx = priv->mb_count - 1;
- priv->tx_mb = flexcan_get_mb(priv, priv->tx_mb_idx);
- priv->tx_mask = FLEXCAN_IFLAG_MB(priv->tx_mb_idx);
-
- priv->offload.mailbox_read = flexcan_mailbox_read;
-
- if (priv->devtype_data->quirks & FLEXCAN_QUIRK_USE_OFF_TIMESTAMP) {
- priv->offload.mb_first = FLEXCAN_RX_MB_OFF_TIMESTAMP_FIRST;
- priv->offload.mb_last = priv->mb_count - 2;
-
- priv->rx_mask = GENMASK_ULL(priv->offload.mb_last,
- priv->offload.mb_first);
- err = can_rx_offload_add_timestamp(dev, &priv->offload);
- } else {
- priv->rx_mask = FLEXCAN_IFLAG_RX_FIFO_OVERFLOW |
- FLEXCAN_IFLAG_RX_FIFO_AVAILABLE;
- err = can_rx_offload_add_fifo(dev, &priv->offload,
- FLEXCAN_NAPI_WEIGHT);
- }
-
- return err;
-}
-
-static void flexcan_chip_interrupts_enable(const struct net_device *dev)
-{
- const struct flexcan_priv *priv = netdev_priv(dev);
- struct flexcan_regs __iomem *regs = priv->regs;
- u64 reg_imask;
-
- disable_irq(dev->irq);
- priv->write(priv->reg_ctrl_default, ®s->ctrl);
- reg_imask = priv->rx_mask | priv->tx_mask;
- priv->write(upper_32_bits(reg_imask), ®s->imask2);
- priv->write(lower_32_bits(reg_imask), ®s->imask1);
- enable_irq(dev->irq);
-}
-
-static void flexcan_chip_interrupts_disable(const struct net_device *dev)
-{
- const struct flexcan_priv *priv = netdev_priv(dev);
- struct flexcan_regs __iomem *regs = priv->regs;
-
- priv->write(0, ®s->imask2);
- priv->write(0, ®s->imask1);
- priv->write(priv->reg_ctrl_default & ~FLEXCAN_CTRL_ERR_ALL,
- ®s->ctrl);
-}
-
-/* flexcan_chip_start
- *
- * this functions is entered with clocks enabled
- *
- */
-static int flexcan_chip_start(struct net_device *dev)
-{
- struct flexcan_priv *priv = netdev_priv(dev);
- struct flexcan_regs __iomem *regs = priv->regs;
- u32 reg_mcr, reg_ctrl, reg_ctrl2, reg_mecr;
- int err, i;
- struct flexcan_mb __iomem *mb;
-
- /* enable module */
- err = flexcan_chip_enable(priv);
- if (err)
- return err;
-
- /* soft reset */
- err = flexcan_chip_softreset(priv);
- if (err)
- goto out_chip_disable;
-
- if (priv->devtype_data->quirks & FLEXCAN_QUIRK_SUPPORT_ECC)
- flexcan_ram_init(dev);
-
- flexcan_set_bittiming(dev);
-
- /* set freeze, halt */
- err = flexcan_chip_freeze(priv);
- if (err)
- goto out_chip_disable;
-
- /* MCR
- *
- * only supervisor access
- * enable warning int
- * enable individual RX masking
- * choose format C
- * set max mailbox number
- */
- reg_mcr = priv->read(®s->mcr);
- reg_mcr &= ~FLEXCAN_MCR_MAXMB(0xff);
- reg_mcr |= FLEXCAN_MCR_SUPV | FLEXCAN_MCR_WRN_EN | FLEXCAN_MCR_IRMQ |
- FLEXCAN_MCR_IDAM_C | FLEXCAN_MCR_MAXMB(priv->tx_mb_idx);
-
- /* MCR
- *
- * FIFO:
- * - disable for timestamp mode
- * - enable for FIFO mode
- */
- if (priv->devtype_data->quirks & FLEXCAN_QUIRK_USE_OFF_TIMESTAMP)
- reg_mcr &= ~FLEXCAN_MCR_FEN;
- else
- reg_mcr |= FLEXCAN_MCR_FEN;
-
- /* MCR
- *
- * NOTE: In loopback mode, the CAN_MCR[SRXDIS] cannot be
- * asserted because this will impede the self reception
- * of a transmitted message. This is not documented in
- * earlier versions of flexcan block guide.
- *
- * Self Reception:
- * - enable Self Reception for loopback mode
- * (by clearing "Self Reception Disable" bit)
- * - disable for normal operation
- */
- if (priv->can.ctrlmode & CAN_CTRLMODE_LOOPBACK)
- reg_mcr &= ~FLEXCAN_MCR_SRX_DIS;
- else
- reg_mcr |= FLEXCAN_MCR_SRX_DIS;
-
- /* MCR - CAN-FD */
- if (priv->can.ctrlmode & CAN_CTRLMODE_FD)
- reg_mcr |= FLEXCAN_MCR_FDEN;
- else
- reg_mcr &= ~FLEXCAN_MCR_FDEN;
-
- netdev_dbg(dev, "%s: writing mcr=0x%08x", __func__, reg_mcr);
- priv->write(reg_mcr, ®s->mcr);
-
- /* CTRL
- *
- * disable timer sync feature
- *
- * disable auto busoff recovery
- * transmit lowest buffer first
- *
- * enable tx and rx warning interrupt
- * enable bus off interrupt
- * (== FLEXCAN_CTRL_ERR_STATE)
- */
- reg_ctrl = priv->read(®s->ctrl);
- reg_ctrl &= ~FLEXCAN_CTRL_TSYN;
- reg_ctrl |= FLEXCAN_CTRL_BOFF_REC | FLEXCAN_CTRL_LBUF |
- FLEXCAN_CTRL_ERR_STATE;
-
- /* enable the "error interrupt" (FLEXCAN_CTRL_ERR_MSK),
- * on most Flexcan cores, too. Otherwise we don't get
- * any error warning or passive interrupts.
- */
- if (priv->devtype_data->quirks & FLEXCAN_QUIRK_BROKEN_WERR_STATE ||
- priv->can.ctrlmode & CAN_CTRLMODE_BERR_REPORTING)
- reg_ctrl |= FLEXCAN_CTRL_ERR_MSK;
- else
- reg_ctrl &= ~FLEXCAN_CTRL_ERR_MSK;
-
- /* save for later use */
- priv->reg_ctrl_default = reg_ctrl;
- /* leave interrupts disabled for now */
- reg_ctrl &= ~FLEXCAN_CTRL_ERR_ALL;
- netdev_dbg(dev, "%s: writing ctrl=0x%08x", __func__, reg_ctrl);
- priv->write(reg_ctrl, ®s->ctrl);
-
- if ((priv->devtype_data->quirks & FLEXCAN_QUIRK_ENABLE_EACEN_RRS)) {
- reg_ctrl2 = priv->read(®s->ctrl2);
- reg_ctrl2 |= FLEXCAN_CTRL2_EACEN | FLEXCAN_CTRL2_RRS;
- priv->write(reg_ctrl2, ®s->ctrl2);
- }
-
- if (priv->can.ctrlmode_supported & CAN_CTRLMODE_FD) {
- u32 reg_fdctrl;
-
- reg_fdctrl = priv->read(®s->fdctrl);
- reg_fdctrl &= ~(FIELD_PREP(FLEXCAN_FDCTRL_MBDSR1, 0x3) |
- FIELD_PREP(FLEXCAN_FDCTRL_MBDSR0, 0x3));
-
- if (priv->can.ctrlmode & CAN_CTRLMODE_FD) {
- reg_fdctrl |=
- FIELD_PREP(FLEXCAN_FDCTRL_MBDSR1,
- FLEXCAN_FDCTRL_MBDSR_64) |
- FIELD_PREP(FLEXCAN_FDCTRL_MBDSR0,
- FLEXCAN_FDCTRL_MBDSR_64);
- } else {
- reg_fdctrl |=
- FIELD_PREP(FLEXCAN_FDCTRL_MBDSR1,
- FLEXCAN_FDCTRL_MBDSR_8) |
- FIELD_PREP(FLEXCAN_FDCTRL_MBDSR0,
- FLEXCAN_FDCTRL_MBDSR_8);
- }
-
- netdev_dbg(dev, "%s: writing fdctrl=0x%08x",
- __func__, reg_fdctrl);
- priv->write(reg_fdctrl, ®s->fdctrl);
- }
-
- if (priv->devtype_data->quirks & FLEXCAN_QUIRK_USE_OFF_TIMESTAMP) {
- for (i = priv->offload.mb_first; i <= priv->offload.mb_last; i++) {
- mb = flexcan_get_mb(priv, i);
- priv->write(FLEXCAN_MB_CODE_RX_EMPTY,
- &mb->can_ctrl);
- }
- } else {
- /* clear and invalidate unused mailboxes first */
- for (i = FLEXCAN_TX_MB_RESERVED_OFF_FIFO; i < priv->mb_count; i++) {
- mb = flexcan_get_mb(priv, i);
- priv->write(FLEXCAN_MB_CODE_RX_INACTIVE,
- &mb->can_ctrl);
- }
- }
-
- /* Errata ERR005829: mark first TX mailbox as INACTIVE */
- priv->write(FLEXCAN_MB_CODE_TX_INACTIVE,
- &priv->tx_mb_reserved->can_ctrl);
-
- /* mark TX mailbox as INACTIVE */
- priv->write(FLEXCAN_MB_CODE_TX_INACTIVE,
- &priv->tx_mb->can_ctrl);
-
- /* acceptance mask/acceptance code (accept everything) */
- priv->write(0x0, ®s->rxgmask);
- priv->write(0x0, ®s->rx14mask);
- priv->write(0x0, ®s->rx15mask);
-
- if (priv->devtype_data->quirks & FLEXCAN_QUIRK_DISABLE_RXFG)
- priv->write(0x0, ®s->rxfgmask);
-
- /* clear acceptance filters */
- for (i = 0; i < priv->mb_count; i++)
- priv->write(0, ®s->rximr[i]);
-
- /* On Vybrid, disable non-correctable errors interrupt and
- * freeze mode. It still can correct the correctable errors
- * when HW supports ECC.
- *
- * This also works around errata e5295 which generates false
- * positive memory errors and put the device in freeze mode.
- */
- if (priv->devtype_data->quirks & FLEXCAN_QUIRK_DISABLE_MECR) {
- /* Follow the protocol as described in "Detection
- * and Correction of Memory Errors" to write to
- * MECR register (step 1 - 5)
- *
- * 1. By default, CTRL2[ECRWRE] = 0, MECR[ECRWRDIS] = 1
- * 2. set CTRL2[ECRWRE]
- */
- reg_ctrl2 = priv->read(®s->ctrl2);
- reg_ctrl2 |= FLEXCAN_CTRL2_ECRWRE;
- priv->write(reg_ctrl2, ®s->ctrl2);
-
- /* 3. clear MECR[ECRWRDIS] */
- reg_mecr = priv->read(®s->mecr);
- reg_mecr &= ~FLEXCAN_MECR_ECRWRDIS;
- priv->write(reg_mecr, ®s->mecr);
-
- /* 4. all writes to MECR must keep MECR[ECRWRDIS] cleared */
- reg_mecr &= ~(FLEXCAN_MECR_NCEFAFRZ | FLEXCAN_MECR_HANCEI_MSK |
- FLEXCAN_MECR_FANCEI_MSK);
- priv->write(reg_mecr, ®s->mecr);
-
- /* 5. after configuration done, lock MECR by either
- * setting MECR[ECRWRDIS] or clearing CTRL2[ECRWRE]
- */
- reg_mecr |= FLEXCAN_MECR_ECRWRDIS;
- priv->write(reg_mecr, ®s->mecr);
-
- reg_ctrl2 &= ~FLEXCAN_CTRL2_ECRWRE;
- priv->write(reg_ctrl2, ®s->ctrl2);
- }
-
- /* synchronize with the can bus */
- err = flexcan_chip_unfreeze(priv);
- if (err)
- goto out_chip_disable;
-
- priv->can.state = CAN_STATE_ERROR_ACTIVE;
-
- /* print chip status */
- netdev_dbg(dev, "%s: reading mcr=0x%08x ctrl=0x%08x\n", __func__,
- priv->read(®s->mcr), priv->read(®s->ctrl));
-
- return 0;
-
- out_chip_disable:
- flexcan_chip_disable(priv);
- return err;
-}
-
-/* __flexcan_chip_stop
- *
- * this function is entered with clocks enabled
- */
-static int __flexcan_chip_stop(struct net_device *dev, bool disable_on_error)
-{
- struct flexcan_priv *priv = netdev_priv(dev);
- int err;
-
- /* freeze + disable module */
- err = flexcan_chip_freeze(priv);
- if (err && !disable_on_error)
- return err;
- err = flexcan_chip_disable(priv);
- if (err && !disable_on_error)
- goto out_chip_unfreeze;
-
- priv->can.state = CAN_STATE_STOPPED;
-
- return 0;
-
- out_chip_unfreeze:
- flexcan_chip_unfreeze(priv);
-
- return err;
-}
-
-static inline int flexcan_chip_stop_disable_on_error(struct net_device *dev)
-{
- return __flexcan_chip_stop(dev, true);
-}
-
-static inline int flexcan_chip_stop(struct net_device *dev)
-{
- return __flexcan_chip_stop(dev, false);
-}
-
-static int flexcan_open(struct net_device *dev)
-{
- struct flexcan_priv *priv = netdev_priv(dev);
- int err;
-
- if ((priv->can.ctrlmode & CAN_CTRLMODE_3_SAMPLES) &&
- (priv->can.ctrlmode & CAN_CTRLMODE_FD)) {
- netdev_err(dev, "Three Samples mode and CAN-FD mode can't be used together\n");
- return -EINVAL;
- }
-
- err = pm_runtime_get_sync(priv->dev);
- if (err < 0) {
- pm_runtime_put_noidle(priv->dev);
- return err;
- }
-
- err = open_candev(dev);
- if (err)
- goto out_runtime_put;
-
- err = flexcan_transceiver_enable(priv);
- if (err)
- goto out_close;
-
- err = flexcan_rx_offload_setup(dev);
- if (err)
- goto out_transceiver_disable;
-
- err = flexcan_chip_start(dev);
- if (err)
- goto out_can_rx_offload_del;
-
- can_rx_offload_enable(&priv->offload);
-
- err = request_irq(dev->irq, flexcan_irq, IRQF_SHARED, dev->name, dev);
- if (err)
- goto out_can_rx_offload_disable;
-
- if (priv->devtype_data->quirks & FLEXCAN_QUIRK_NR_IRQ_3) {
- err = request_irq(priv->irq_boff,
- flexcan_irq, IRQF_SHARED, dev->name, dev);
- if (err)
- goto out_free_irq;
-
- err = request_irq(priv->irq_err,
- flexcan_irq, IRQF_SHARED, dev->name, dev);
- if (err)
- goto out_free_irq_boff;
- }
-
- flexcan_chip_interrupts_enable(dev);
-
- can_led_event(dev, CAN_LED_EVENT_OPEN);
-
- netif_start_queue(dev);
-
- return 0;
-
- out_free_irq_boff:
- free_irq(priv->irq_boff, dev);
- out_free_irq:
- free_irq(dev->irq, dev);
- out_can_rx_offload_disable:
- can_rx_offload_disable(&priv->offload);
- flexcan_chip_stop(dev);
- out_can_rx_offload_del:
- can_rx_offload_del(&priv->offload);
- out_transceiver_disable:
- flexcan_transceiver_disable(priv);
- out_close:
- close_candev(dev);
- out_runtime_put:
- pm_runtime_put(priv->dev);
-
- return err;
-}
-
-static int flexcan_close(struct net_device *dev)
-{
- struct flexcan_priv *priv = netdev_priv(dev);
-
- netif_stop_queue(dev);
- flexcan_chip_interrupts_disable(dev);
-
- if (priv->devtype_data->quirks & FLEXCAN_QUIRK_NR_IRQ_3) {
- free_irq(priv->irq_err, dev);
- free_irq(priv->irq_boff, dev);
- }
-
- free_irq(dev->irq, dev);
- can_rx_offload_disable(&priv->offload);
- flexcan_chip_stop_disable_on_error(dev);
-
- can_rx_offload_del(&priv->offload);
- flexcan_transceiver_disable(priv);
- close_candev(dev);
-
- pm_runtime_put(priv->dev);
-
- can_led_event(dev, CAN_LED_EVENT_STOP);
-
- return 0;
-}
-
-static int flexcan_set_mode(struct net_device *dev, enum can_mode mode)
-{
- int err;
-
- switch (mode) {
- case CAN_MODE_START:
- err = flexcan_chip_start(dev);
- if (err)
- return err;
-
- flexcan_chip_interrupts_enable(dev);
-
- netif_wake_queue(dev);
- break;
-
- default:
- return -EOPNOTSUPP;
- }
-
- return 0;
-}
-
-static const struct net_device_ops flexcan_netdev_ops = {
- .ndo_open = flexcan_open,
- .ndo_stop = flexcan_close,
- .ndo_start_xmit = flexcan_start_xmit,
- .ndo_change_mtu = can_change_mtu,
-};
-
-static int register_flexcandev(struct net_device *dev)
-{
- struct flexcan_priv *priv = netdev_priv(dev);
- struct flexcan_regs __iomem *regs = priv->regs;
- u32 reg, err;
-
- err = flexcan_clks_enable(priv);
- if (err)
- return err;
-
- /* select "bus clock", chip must be disabled */
- err = flexcan_chip_disable(priv);
- if (err)
- goto out_clks_disable;
-
- reg = priv->read(®s->ctrl);
- if (priv->clk_src)
- reg |= FLEXCAN_CTRL_CLK_SRC;
- else
- reg &= ~FLEXCAN_CTRL_CLK_SRC;
- priv->write(reg, ®s->ctrl);
-
- err = flexcan_chip_enable(priv);
- if (err)
- goto out_chip_disable;
-
- /* set freeze, halt */
- err = flexcan_chip_freeze(priv);
- if (err)
- goto out_chip_disable;
-
- /* activate FIFO, restrict register access */
- reg = priv->read(®s->mcr);
- reg |= FLEXCAN_MCR_FEN | FLEXCAN_MCR_SUPV;
- priv->write(reg, ®s->mcr);
-
- /* Currently we only support newer versions of this core
- * featuring a RX hardware FIFO (although this driver doesn't
- * make use of it on some cores). Older cores, found on some
- * Coldfire derivates are not tested.
- */
- reg = priv->read(®s->mcr);
- if (!(reg & FLEXCAN_MCR_FEN)) {
- netdev_err(dev, "Could not enable RX FIFO, unsupported core\n");
- err = -ENODEV;
- goto out_chip_disable;
- }
-
- err = register_candev(dev);
- if (err)
- goto out_chip_disable;
-
- /* Disable core and let pm_runtime_put() disable the clocks.
- * If CONFIG_PM is not enabled, the clocks will stay powered.
- */
- flexcan_chip_disable(priv);
- pm_runtime_put(priv->dev);
-
- return 0;
-
- out_chip_disable:
- flexcan_chip_disable(priv);
- out_clks_disable:
- flexcan_clks_disable(priv);
- return err;
-}
-
-static void unregister_flexcandev(struct net_device *dev)
-{
- unregister_candev(dev);
-}
-
-static int flexcan_setup_stop_mode_gpr(struct platform_device *pdev)
-{
- struct net_device *dev = platform_get_drvdata(pdev);
- struct device_node *np = pdev->dev.of_node;
- struct device_node *gpr_np;
- struct flexcan_priv *priv;
- phandle phandle;
- u32 out_val[3];
- int ret;
-
- if (!np)
- return -EINVAL;
-
- /* stop mode property format is:
- * <&gpr req_gpr req_bit>.
- */
- ret = of_property_read_u32_array(np, "fsl,stop-mode", out_val,
- ARRAY_SIZE(out_val));
- if (ret) {
- dev_dbg(&pdev->dev, "no stop-mode property\n");
- return ret;
- }
- phandle = *out_val;
-
- gpr_np = of_find_node_by_phandle(phandle);
- if (!gpr_np) {
- dev_dbg(&pdev->dev, "could not find gpr node by phandle\n");
- return -ENODEV;
- }
-
- priv = netdev_priv(dev);
- priv->stm.gpr = syscon_node_to_regmap(gpr_np);
- if (IS_ERR(priv->stm.gpr)) {
- dev_dbg(&pdev->dev, "could not find gpr regmap\n");
- ret = PTR_ERR(priv->stm.gpr);
- goto out_put_node;
- }
-
- priv->stm.req_gpr = out_val[1];
- priv->stm.req_bit = out_val[2];
-
- dev_dbg(&pdev->dev,
- "gpr %s req_gpr=0x02%x req_bit=%u\n",
- gpr_np->full_name, priv->stm.req_gpr, priv->stm.req_bit);
-
- return 0;
-
-out_put_node:
- of_node_put(gpr_np);
- return ret;
-}
-
-static int flexcan_setup_stop_mode_scfw(struct platform_device *pdev)
-{
- struct net_device *dev = platform_get_drvdata(pdev);
- struct flexcan_priv *priv;
- u8 scu_idx;
- int ret;
-
- ret = of_property_read_u8(pdev->dev.of_node, "fsl,scu-index", &scu_idx);
- if (ret < 0) {
- dev_dbg(&pdev->dev, "failed to get scu index\n");
- return ret;
- }
-
- priv = netdev_priv(dev);
- priv->scu_idx = scu_idx;
-
- /* this function could be deferred probe, return -EPROBE_DEFER */
- return imx_scu_get_handle(&priv->sc_ipc_handle);
-}
-
-/* flexcan_setup_stop_mode - Setup stop mode for wakeup
- *
- * Return: = 0 setup stop mode successfully or doesn't support this feature
- * < 0 fail to setup stop mode (could be deferred probe)
- */
-static int flexcan_setup_stop_mode(struct platform_device *pdev)
-{
- struct net_device *dev = platform_get_drvdata(pdev);
- struct flexcan_priv *priv;
- int ret;
-
- priv = netdev_priv(dev);
-
- if (priv->devtype_data->quirks & FLEXCAN_QUIRK_SETUP_STOP_MODE_SCFW)
- ret = flexcan_setup_stop_mode_scfw(pdev);
- else if (priv->devtype_data->quirks & FLEXCAN_QUIRK_SETUP_STOP_MODE_GPR)
- ret = flexcan_setup_stop_mode_gpr(pdev);
- else
- /* return 0 directly if doesn't support stop mode feature */
- return 0;
-
- if (ret)
- return ret;
-
- device_set_wakeup_capable(&pdev->dev, true);
-
- if (of_property_read_bool(pdev->dev.of_node, "wakeup-source"))
- device_set_wakeup_enable(&pdev->dev, true);
-
- return 0;
-}
-
-static const struct of_device_id flexcan_of_match[] = {
- { .compatible = "fsl,imx8qm-flexcan", .data = &fsl_imx8qm_devtype_data, },
- { .compatible = "fsl,imx8mp-flexcan", .data = &fsl_imx8mp_devtype_data, },
- { .compatible = "fsl,imx6q-flexcan", .data = &fsl_imx6q_devtype_data, },
- { .compatible = "fsl,imx28-flexcan", .data = &fsl_imx28_devtype_data, },
- { .compatible = "fsl,imx53-flexcan", .data = &fsl_imx25_devtype_data, },
- { .compatible = "fsl,imx35-flexcan", .data = &fsl_imx25_devtype_data, },
- { .compatible = "fsl,imx25-flexcan", .data = &fsl_imx25_devtype_data, },
- { .compatible = "fsl,p1010-flexcan", .data = &fsl_p1010_devtype_data, },
- { .compatible = "fsl,vf610-flexcan", .data = &fsl_vf610_devtype_data, },
- { .compatible = "fsl,ls1021ar2-flexcan", .data = &fsl_ls1021a_r2_devtype_data, },
- { .compatible = "fsl,lx2160ar1-flexcan", .data = &fsl_lx2160a_r1_devtype_data, },
- { /* sentinel */ },
-};
-MODULE_DEVICE_TABLE(of, flexcan_of_match);
-
-static const struct platform_device_id flexcan_id_table[] = {
- {
- .name = "flexcan-mcf5441x",
- .driver_data = (kernel_ulong_t)&fsl_mcf5441x_devtype_data,
- }, {
- /* sentinel */
- },
-};
-MODULE_DEVICE_TABLE(platform, flexcan_id_table);
-
-static int flexcan_probe(struct platform_device *pdev)
-{
- const struct of_device_id *of_id;
- const struct flexcan_devtype_data *devtype_data;
- struct net_device *dev;
- struct flexcan_priv *priv;
- struct regulator *reg_xceiver;
- struct clk *clk_ipg = NULL, *clk_per = NULL;
- struct flexcan_regs __iomem *regs;
- struct flexcan_platform_data *pdata;
- int err, irq;
- u8 clk_src = 1;
- u32 clock_freq = 0;
-
- reg_xceiver = devm_regulator_get_optional(&pdev->dev, "xceiver");
- if (PTR_ERR(reg_xceiver) == -EPROBE_DEFER)
- return -EPROBE_DEFER;
- else if (PTR_ERR(reg_xceiver) == -ENODEV)
- reg_xceiver = NULL;
- else if (IS_ERR(reg_xceiver))
- return PTR_ERR(reg_xceiver);
-
- if (pdev->dev.of_node) {
- of_property_read_u32(pdev->dev.of_node,
- "clock-frequency", &clock_freq);
- of_property_read_u8(pdev->dev.of_node,
- "fsl,clk-source", &clk_src);
- } else {
- pdata = dev_get_platdata(&pdev->dev);
- if (pdata) {
- clock_freq = pdata->clock_frequency;
- clk_src = pdata->clk_src;
- }
- }
-
- if (!clock_freq) {
- clk_ipg = devm_clk_get(&pdev->dev, "ipg");
- if (IS_ERR(clk_ipg)) {
- dev_err(&pdev->dev, "no ipg clock defined\n");
- return PTR_ERR(clk_ipg);
- }
-
- clk_per = devm_clk_get(&pdev->dev, "per");
- if (IS_ERR(clk_per)) {
- dev_err(&pdev->dev, "no per clock defined\n");
- return PTR_ERR(clk_per);
- }
- clock_freq = clk_get_rate(clk_per);
- }
-
- irq = platform_get_irq(pdev, 0);
- if (irq <= 0)
- return -ENODEV;
-
- regs = devm_platform_ioremap_resource(pdev, 0);
- if (IS_ERR(regs))
- return PTR_ERR(regs);
-
- of_id = of_match_device(flexcan_of_match, &pdev->dev);
- if (of_id)
- devtype_data = of_id->data;
- else if (platform_get_device_id(pdev)->driver_data)
- devtype_data = (struct flexcan_devtype_data *)
- platform_get_device_id(pdev)->driver_data;
- else
- return -ENODEV;
-
- if ((devtype_data->quirks & FLEXCAN_QUIRK_SUPPORT_FD) &&
- !(devtype_data->quirks & FLEXCAN_QUIRK_USE_OFF_TIMESTAMP)) {
- dev_err(&pdev->dev, "CAN-FD mode doesn't work with FIFO mode!\n");
- return -EINVAL;
- }
-
- dev = alloc_candev(sizeof(struct flexcan_priv), 1);
- if (!dev)
- return -ENOMEM;
-
- platform_set_drvdata(pdev, dev);
- SET_NETDEV_DEV(dev, &pdev->dev);
-
- dev->netdev_ops = &flexcan_netdev_ops;
- dev->irq = irq;
- dev->flags |= IFF_ECHO;
-
- priv = netdev_priv(dev);
-
- if (of_property_read_bool(pdev->dev.of_node, "big-endian") ||
- devtype_data->quirks & FLEXCAN_QUIRK_DEFAULT_BIG_ENDIAN) {
- priv->read = flexcan_read_be;
- priv->write = flexcan_write_be;
- } else {
- priv->read = flexcan_read_le;
- priv->write = flexcan_write_le;
- }
-
- priv->dev = &pdev->dev;
- priv->can.clock.freq = clock_freq;
- priv->can.do_set_mode = flexcan_set_mode;
- priv->can.do_get_berr_counter = flexcan_get_berr_counter;
- priv->can.ctrlmode_supported = CAN_CTRLMODE_LOOPBACK |
- CAN_CTRLMODE_LISTENONLY | CAN_CTRLMODE_3_SAMPLES |
- CAN_CTRLMODE_BERR_REPORTING;
- priv->regs = regs;
- priv->clk_ipg = clk_ipg;
- priv->clk_per = clk_per;
- priv->clk_src = clk_src;
- priv->devtype_data = devtype_data;
- priv->reg_xceiver = reg_xceiver;
-
- if (devtype_data->quirks & FLEXCAN_QUIRK_NR_IRQ_3) {
- priv->irq_boff = platform_get_irq(pdev, 1);
- if (priv->irq_boff <= 0) {
- err = -ENODEV;
- goto failed_platform_get_irq;
- }
- priv->irq_err = platform_get_irq(pdev, 2);
- if (priv->irq_err <= 0) {
- err = -ENODEV;
- goto failed_platform_get_irq;
- }
- }
-
- if (priv->devtype_data->quirks & FLEXCAN_QUIRK_SUPPORT_FD) {
- priv->can.ctrlmode_supported |= CAN_CTRLMODE_FD |
- CAN_CTRLMODE_FD_NON_ISO;
- priv->can.bittiming_const = &flexcan_fd_bittiming_const;
- priv->can.data_bittiming_const =
- &flexcan_fd_data_bittiming_const;
- } else {
- priv->can.bittiming_const = &flexcan_bittiming_const;
- }
-
- pm_runtime_get_noresume(&pdev->dev);
- pm_runtime_set_active(&pdev->dev);
- pm_runtime_enable(&pdev->dev);
-
- err = register_flexcandev(dev);
- if (err) {
- dev_err(&pdev->dev, "registering netdev failed\n");
- goto failed_register;
- }
-
- err = flexcan_setup_stop_mode(pdev);
- if (err < 0) {
- if (err != -EPROBE_DEFER)
- dev_err(&pdev->dev, "setup stop mode failed\n");
- goto failed_setup_stop_mode;
- }
-
- of_can_transceiver(dev);
- devm_can_led_init(dev);
-
- return 0;
-
- failed_setup_stop_mode:
- unregister_flexcandev(dev);
- failed_register:
- pm_runtime_put_noidle(&pdev->dev);
- pm_runtime_disable(&pdev->dev);
- failed_platform_get_irq:
- free_candev(dev);
- return err;
-}
-
-static int flexcan_remove(struct platform_device *pdev)
-{
- struct net_device *dev = platform_get_drvdata(pdev);
-
- device_set_wakeup_enable(&pdev->dev, false);
- device_set_wakeup_capable(&pdev->dev, false);
- unregister_flexcandev(dev);
- pm_runtime_disable(&pdev->dev);
- free_candev(dev);
-
- return 0;
-}
-
-static int __maybe_unused flexcan_suspend(struct device *device)
-{
- struct net_device *dev = dev_get_drvdata(device);
- struct flexcan_priv *priv = netdev_priv(dev);
- int err;
-
- if (netif_running(dev)) {
- /* if wakeup is enabled, enter stop mode
- * else enter disabled mode.
- */
- if (device_may_wakeup(device)) {
- enable_irq_wake(dev->irq);
- err = flexcan_enter_stop_mode(priv);
- if (err)
- return err;
- } else {
- err = flexcan_chip_stop(dev);
- if (err)
- return err;
-
- flexcan_chip_interrupts_disable(dev);
-
- err = pinctrl_pm_select_sleep_state(device);
- if (err)
- return err;
- }
- netif_stop_queue(dev);
- netif_device_detach(dev);
- }
- priv->can.state = CAN_STATE_SLEEPING;
-
- return 0;
-}
-
-static int __maybe_unused flexcan_resume(struct device *device)
-{
- struct net_device *dev = dev_get_drvdata(device);
- struct flexcan_priv *priv = netdev_priv(dev);
- int err;
-
- priv->can.state = CAN_STATE_ERROR_ACTIVE;
- if (netif_running(dev)) {
- netif_device_attach(dev);
- netif_start_queue(dev);
- if (device_may_wakeup(device)) {
- disable_irq_wake(dev->irq);
- err = flexcan_exit_stop_mode(priv);
- if (err)
- return err;
- } else {
- err = pinctrl_pm_select_default_state(device);
- if (err)
- return err;
-
- err = flexcan_chip_start(dev);
- if (err)
- return err;
-
- flexcan_chip_interrupts_enable(dev);
- }
- }
-
- return 0;
-}
-
-static int __maybe_unused flexcan_runtime_suspend(struct device *device)
-{
- struct net_device *dev = dev_get_drvdata(device);
- struct flexcan_priv *priv = netdev_priv(dev);
-
- flexcan_clks_disable(priv);
-
- return 0;
-}
-
-static int __maybe_unused flexcan_runtime_resume(struct device *device)
-{
- struct net_device *dev = dev_get_drvdata(device);
- struct flexcan_priv *priv = netdev_priv(dev);
-
- return flexcan_clks_enable(priv);
-}
-
-static int __maybe_unused flexcan_noirq_suspend(struct device *device)
-{
- struct net_device *dev = dev_get_drvdata(device);
- struct flexcan_priv *priv = netdev_priv(dev);
-
- if (netif_running(dev)) {
- int err;
-
- if (device_may_wakeup(device))
- flexcan_enable_wakeup_irq(priv, true);
-
- err = pm_runtime_force_suspend(device);
- if (err)
- return err;
- }
-
- return 0;
-}
-
-static int __maybe_unused flexcan_noirq_resume(struct device *device)
-{
- struct net_device *dev = dev_get_drvdata(device);
- struct flexcan_priv *priv = netdev_priv(dev);
-
- if (netif_running(dev)) {
- int err;
-
- err = pm_runtime_force_resume(device);
- if (err)
- return err;
-
- if (device_may_wakeup(device))
- flexcan_enable_wakeup_irq(priv, false);
- }
-
- return 0;
-}
-
-static const struct dev_pm_ops flexcan_pm_ops = {
- SET_SYSTEM_SLEEP_PM_OPS(flexcan_suspend, flexcan_resume)
- SET_RUNTIME_PM_OPS(flexcan_runtime_suspend, flexcan_runtime_resume, NULL)
- SET_NOIRQ_SYSTEM_SLEEP_PM_OPS(flexcan_noirq_suspend, flexcan_noirq_resume)
-};
-
-static struct platform_driver flexcan_driver = {
- .driver = {
- .name = DRV_NAME,
- .pm = &flexcan_pm_ops,
- .of_match_table = flexcan_of_match,
- },
- .probe = flexcan_probe,
- .remove = flexcan_remove,
- .id_table = flexcan_id_table,
-};
-
-module_platform_driver(flexcan_driver);
-
-MODULE_AUTHOR("Sascha Hauer <kernel@pengutronix.de>, "
- "Marc Kleine-Budde <kernel@pengutronix.de>");
-MODULE_LICENSE("GPL v2");
-MODULE_DESCRIPTION("CAN port driver for flexcan based chip");
diff --git a/drivers/net/can/flexcan/Makefile b/drivers/net/can/flexcan/Makefile
new file mode 100644
index 0000000..89d5695
--- /dev/null
+++ b/drivers/net/can/flexcan/Makefile
@@ -0,0 +1,7 @@
+# SPDX-License-Identifier: GPL-2.0
+
+obj-$(CONFIG_CAN_FLEXCAN) += flexcan.o
+
+flexcan-objs :=
+flexcan-objs += flexcan-core.o
+flexcan-objs += flexcan-ethtool.o
diff --git a/drivers/net/can/flexcan/flexcan-core.c b/drivers/net/can/flexcan/flexcan-core.c
new file mode 100644
index 0000000..0bff188
--- /dev/null
+++ b/drivers/net/can/flexcan/flexcan-core.c
@@ -0,0 +1,2365 @@
+// SPDX-License-Identifier: GPL-2.0
+//
+// flexcan.c - FLEXCAN CAN controller driver
+//
+// Copyright (c) 2005-2006 Varma Electronics Oy
+// Copyright (c) 2009 Sascha Hauer, Pengutronix
+// Copyright (c) 2010-2017 Pengutronix, Marc Kleine-Budde <kernel@pengutronix.de>
+// Copyright (c) 2014 David Jander, Protonic Holland
+//
+// Based on code originally by Andrey Volkov <avolkov@varma-el.com>
+
+#include <dt-bindings/firmware/imx/rsrc.h>
+#include <linux/bitfield.h>
+#include <linux/can.h>
+#include <linux/can/dev.h>
+#include <linux/can/error.h>
+#include <linux/can/led.h>
+#include <linux/clk.h>
+#include <linux/delay.h>
+#include <linux/firmware/imx/sci.h>
+#include <linux/interrupt.h>
+#include <linux/io.h>
+#include <linux/mfd/syscon.h>
+#include <linux/module.h>
+#include <linux/netdevice.h>
+#include <linux/of.h>
+#include <linux/of_device.h>
+#include <linux/pinctrl/consumer.h>
+#include <linux/platform_device.h>
+#include <linux/can/platform/flexcan.h>
+#include <linux/pm_runtime.h>
+#include <linux/regmap.h>
+#include <linux/regulator/consumer.h>
+
+#include "flexcan.h"
+
+#define DRV_NAME "flexcan"
+
+/* 8 for RX fifo and 2 error handling */
+#define FLEXCAN_NAPI_WEIGHT (8 + 2)
+
+/* FLEXCAN module configuration register (CANMCR) bits */
+#define FLEXCAN_MCR_MDIS BIT(31)
+#define FLEXCAN_MCR_FRZ BIT(30)
+#define FLEXCAN_MCR_FEN BIT(29)
+#define FLEXCAN_MCR_HALT BIT(28)
+#define FLEXCAN_MCR_NOT_RDY BIT(27)
+#define FLEXCAN_MCR_WAK_MSK BIT(26)
+#define FLEXCAN_MCR_SOFTRST BIT(25)
+#define FLEXCAN_MCR_FRZ_ACK BIT(24)
+#define FLEXCAN_MCR_SUPV BIT(23)
+#define FLEXCAN_MCR_SLF_WAK BIT(22)
+#define FLEXCAN_MCR_WRN_EN BIT(21)
+#define FLEXCAN_MCR_LPM_ACK BIT(20)
+#define FLEXCAN_MCR_WAK_SRC BIT(19)
+#define FLEXCAN_MCR_DOZE BIT(18)
+#define FLEXCAN_MCR_SRX_DIS BIT(17)
+#define FLEXCAN_MCR_IRMQ BIT(16)
+#define FLEXCAN_MCR_LPRIO_EN BIT(13)
+#define FLEXCAN_MCR_AEN BIT(12)
+#define FLEXCAN_MCR_FDEN BIT(11)
+/* MCR_MAXMB: maximum used MBs is MAXMB + 1 */
+#define FLEXCAN_MCR_MAXMB(x) ((x) & 0x7f)
+#define FLEXCAN_MCR_IDAM_A (0x0 << 8)
+#define FLEXCAN_MCR_IDAM_B (0x1 << 8)
+#define FLEXCAN_MCR_IDAM_C (0x2 << 8)
+#define FLEXCAN_MCR_IDAM_D (0x3 << 8)
+
+/* FLEXCAN control register (CANCTRL) bits */
+#define FLEXCAN_CTRL_PRESDIV(x) (((x) & 0xff) << 24)
+#define FLEXCAN_CTRL_RJW(x) (((x) & 0x03) << 22)
+#define FLEXCAN_CTRL_PSEG1(x) (((x) & 0x07) << 19)
+#define FLEXCAN_CTRL_PSEG2(x) (((x) & 0x07) << 16)
+#define FLEXCAN_CTRL_BOFF_MSK BIT(15)
+#define FLEXCAN_CTRL_ERR_MSK BIT(14)
+#define FLEXCAN_CTRL_CLK_SRC BIT(13)
+#define FLEXCAN_CTRL_LPB BIT(12)
+#define FLEXCAN_CTRL_TWRN_MSK BIT(11)
+#define FLEXCAN_CTRL_RWRN_MSK BIT(10)
+#define FLEXCAN_CTRL_SMP BIT(7)
+#define FLEXCAN_CTRL_BOFF_REC BIT(6)
+#define FLEXCAN_CTRL_TSYN BIT(5)
+#define FLEXCAN_CTRL_LBUF BIT(4)
+#define FLEXCAN_CTRL_LOM BIT(3)
+#define FLEXCAN_CTRL_PROPSEG(x) ((x) & 0x07)
+#define FLEXCAN_CTRL_ERR_BUS (FLEXCAN_CTRL_ERR_MSK)
+#define FLEXCAN_CTRL_ERR_STATE \
+ (FLEXCAN_CTRL_TWRN_MSK | FLEXCAN_CTRL_RWRN_MSK | \
+ FLEXCAN_CTRL_BOFF_MSK)
+#define FLEXCAN_CTRL_ERR_ALL \
+ (FLEXCAN_CTRL_ERR_BUS | FLEXCAN_CTRL_ERR_STATE)
+
+/* FLEXCAN control register 2 (CTRL2) bits */
+#define FLEXCAN_CTRL2_ECRWRE BIT(29)
+#define FLEXCAN_CTRL2_WRMFRZ BIT(28)
+#define FLEXCAN_CTRL2_RFFN(x) (((x) & 0x0f) << 24)
+#define FLEXCAN_CTRL2_TASD(x) (((x) & 0x1f) << 19)
+#define FLEXCAN_CTRL2_MRP BIT(18)
+#define FLEXCAN_CTRL2_RRS BIT(17)
+#define FLEXCAN_CTRL2_EACEN BIT(16)
+#define FLEXCAN_CTRL2_ISOCANFDEN BIT(12)
+
+/* FLEXCAN memory error control register (MECR) bits */
+#define FLEXCAN_MECR_ECRWRDIS BIT(31)
+#define FLEXCAN_MECR_HANCEI_MSK BIT(19)
+#define FLEXCAN_MECR_FANCEI_MSK BIT(18)
+#define FLEXCAN_MECR_CEI_MSK BIT(16)
+#define FLEXCAN_MECR_HAERRIE BIT(15)
+#define FLEXCAN_MECR_FAERRIE BIT(14)
+#define FLEXCAN_MECR_EXTERRIE BIT(13)
+#define FLEXCAN_MECR_RERRDIS BIT(9)
+#define FLEXCAN_MECR_ECCDIS BIT(8)
+#define FLEXCAN_MECR_NCEFAFRZ BIT(7)
+
+/* FLEXCAN error and status register (ESR) bits */
+#define FLEXCAN_ESR_TWRN_INT BIT(17)
+#define FLEXCAN_ESR_RWRN_INT BIT(16)
+#define FLEXCAN_ESR_BIT1_ERR BIT(15)
+#define FLEXCAN_ESR_BIT0_ERR BIT(14)
+#define FLEXCAN_ESR_ACK_ERR BIT(13)
+#define FLEXCAN_ESR_CRC_ERR BIT(12)
+#define FLEXCAN_ESR_FRM_ERR BIT(11)
+#define FLEXCAN_ESR_STF_ERR BIT(10)
+#define FLEXCAN_ESR_TX_WRN BIT(9)
+#define FLEXCAN_ESR_RX_WRN BIT(8)
+#define FLEXCAN_ESR_IDLE BIT(7)
+#define FLEXCAN_ESR_TXRX BIT(6)
+#define FLEXCAN_EST_FLT_CONF_SHIFT (4)
+#define FLEXCAN_ESR_FLT_CONF_MASK (0x3 << FLEXCAN_EST_FLT_CONF_SHIFT)
+#define FLEXCAN_ESR_FLT_CONF_ACTIVE (0x0 << FLEXCAN_EST_FLT_CONF_SHIFT)
+#define FLEXCAN_ESR_FLT_CONF_PASSIVE (0x1 << FLEXCAN_EST_FLT_CONF_SHIFT)
+#define FLEXCAN_ESR_BOFF_INT BIT(2)
+#define FLEXCAN_ESR_ERR_INT BIT(1)
+#define FLEXCAN_ESR_WAK_INT BIT(0)
+#define FLEXCAN_ESR_ERR_BUS \
+ (FLEXCAN_ESR_BIT1_ERR | FLEXCAN_ESR_BIT0_ERR | \
+ FLEXCAN_ESR_ACK_ERR | FLEXCAN_ESR_CRC_ERR | \
+ FLEXCAN_ESR_FRM_ERR | FLEXCAN_ESR_STF_ERR)
+#define FLEXCAN_ESR_ERR_STATE \
+ (FLEXCAN_ESR_TWRN_INT | FLEXCAN_ESR_RWRN_INT | FLEXCAN_ESR_BOFF_INT)
+#define FLEXCAN_ESR_ERR_ALL \
+ (FLEXCAN_ESR_ERR_BUS | FLEXCAN_ESR_ERR_STATE)
+#define FLEXCAN_ESR_ALL_INT \
+ (FLEXCAN_ESR_TWRN_INT | FLEXCAN_ESR_RWRN_INT | \
+ FLEXCAN_ESR_BOFF_INT | FLEXCAN_ESR_ERR_INT)
+
+/* FLEXCAN Bit Timing register (CBT) bits */
+#define FLEXCAN_CBT_BTF BIT(31)
+#define FLEXCAN_CBT_EPRESDIV_MASK GENMASK(30, 21)
+#define FLEXCAN_CBT_ERJW_MASK GENMASK(20, 16)
+#define FLEXCAN_CBT_EPROPSEG_MASK GENMASK(15, 10)
+#define FLEXCAN_CBT_EPSEG1_MASK GENMASK(9, 5)
+#define FLEXCAN_CBT_EPSEG2_MASK GENMASK(4, 0)
+
+/* FLEXCAN FD control register (FDCTRL) bits */
+#define FLEXCAN_FDCTRL_FDRATE BIT(31)
+#define FLEXCAN_FDCTRL_MBDSR1 GENMASK(20, 19)
+#define FLEXCAN_FDCTRL_MBDSR0 GENMASK(17, 16)
+#define FLEXCAN_FDCTRL_MBDSR_8 0x0
+#define FLEXCAN_FDCTRL_MBDSR_12 0x1
+#define FLEXCAN_FDCTRL_MBDSR_32 0x2
+#define FLEXCAN_FDCTRL_MBDSR_64 0x3
+#define FLEXCAN_FDCTRL_TDCEN BIT(15)
+#define FLEXCAN_FDCTRL_TDCFAIL BIT(14)
+#define FLEXCAN_FDCTRL_TDCOFF GENMASK(12, 8)
+#define FLEXCAN_FDCTRL_TDCVAL GENMASK(5, 0)
+
+/* FLEXCAN FD Bit Timing register (FDCBT) bits */
+#define FLEXCAN_FDCBT_FPRESDIV_MASK GENMASK(29, 20)
+#define FLEXCAN_FDCBT_FRJW_MASK GENMASK(18, 16)
+#define FLEXCAN_FDCBT_FPROPSEG_MASK GENMASK(14, 10)
+#define FLEXCAN_FDCBT_FPSEG1_MASK GENMASK(7, 5)
+#define FLEXCAN_FDCBT_FPSEG2_MASK GENMASK(2, 0)
+
+/* FLEXCAN interrupt flag register (IFLAG) bits */
+/* Errata ERR005829 step7: Reserve first valid MB */
+#define FLEXCAN_TX_MB_RESERVED_RX_FIFO 8
+#define FLEXCAN_TX_MB_RESERVED_RX_MAILBOX 0
+#define FLEXCAN_RX_MB_RX_MAILBOX_FIRST (FLEXCAN_TX_MB_RESERVED_RX_MAILBOX + 1)
+#define FLEXCAN_IFLAG_MB(x) BIT_ULL(x)
+#define FLEXCAN_IFLAG_RX_FIFO_OVERFLOW BIT(7)
+#define FLEXCAN_IFLAG_RX_FIFO_WARN BIT(6)
+#define FLEXCAN_IFLAG_RX_FIFO_AVAILABLE BIT(5)
+
+/* FLEXCAN message buffers */
+#define FLEXCAN_MB_CODE_MASK (0xf << 24)
+#define FLEXCAN_MB_CODE_RX_BUSY_BIT (0x1 << 24)
+#define FLEXCAN_MB_CODE_RX_INACTIVE (0x0 << 24)
+#define FLEXCAN_MB_CODE_RX_EMPTY (0x4 << 24)
+#define FLEXCAN_MB_CODE_RX_FULL (0x2 << 24)
+#define FLEXCAN_MB_CODE_RX_OVERRUN (0x6 << 24)
+#define FLEXCAN_MB_CODE_RX_RANSWER (0xa << 24)
+
+#define FLEXCAN_MB_CODE_TX_INACTIVE (0x8 << 24)
+#define FLEXCAN_MB_CODE_TX_ABORT (0x9 << 24)
+#define FLEXCAN_MB_CODE_TX_DATA (0xc << 24)
+#define FLEXCAN_MB_CODE_TX_TANSWER (0xe << 24)
+
+#define FLEXCAN_MB_CNT_EDL BIT(31)
+#define FLEXCAN_MB_CNT_BRS BIT(30)
+#define FLEXCAN_MB_CNT_ESI BIT(29)
+#define FLEXCAN_MB_CNT_SRR BIT(22)
+#define FLEXCAN_MB_CNT_IDE BIT(21)
+#define FLEXCAN_MB_CNT_RTR BIT(20)
+#define FLEXCAN_MB_CNT_LENGTH(x) (((x) & 0xf) << 16)
+#define FLEXCAN_MB_CNT_TIMESTAMP(x) ((x) & 0xffff)
+
+#define FLEXCAN_TIMEOUT_US (250)
+
+/* Structure of the message buffer */
+struct flexcan_mb {
+ u32 can_ctrl;
+ u32 can_id;
+ u32 data[];
+};
+
+/* Structure of the hardware registers */
+struct flexcan_regs {
+ u32 mcr; /* 0x00 */
+ u32 ctrl; /* 0x04 - Not affected by Soft Reset */
+ u32 timer; /* 0x08 */
+ u32 tcr; /* 0x0c */
+ u32 rxgmask; /* 0x10 - Not affected by Soft Reset */
+ u32 rx14mask; /* 0x14 - Not affected by Soft Reset */
+ u32 rx15mask; /* 0x18 - Not affected by Soft Reset */
+ u32 ecr; /* 0x1c */
+ u32 esr; /* 0x20 */
+ u32 imask2; /* 0x24 */
+ u32 imask1; /* 0x28 */
+ u32 iflag2; /* 0x2c */
+ u32 iflag1; /* 0x30 */
+ union { /* 0x34 */
+ u32 gfwr_mx28; /* MX28, MX53 */
+ u32 ctrl2; /* MX6, VF610 - Not affected by Soft Reset */
+ };
+ u32 esr2; /* 0x38 */
+ u32 imeur; /* 0x3c */
+ u32 lrfr; /* 0x40 */
+ u32 crcr; /* 0x44 */
+ u32 rxfgmask; /* 0x48 */
+ u32 rxfir; /* 0x4c - Not affected by Soft Reset */
+ u32 cbt; /* 0x50 - Not affected by Soft Reset */
+ u32 _reserved2; /* 0x54 */
+ u32 dbg1; /* 0x58 */
+ u32 dbg2; /* 0x5c */
+ u32 _reserved3[8]; /* 0x60 */
+ struct_group(init,
+ u8 mb[2][512]; /* 0x80 - Not affected by Soft Reset */
+ /* FIFO-mode:
+ * MB
+ * 0x080...0x08f 0 RX message buffer
+ * 0x090...0x0df 1-5 reserved
+ * 0x0e0...0x0ff 6-7 8 entry ID table
+ * (mx25, mx28, mx35, mx53)
+ * 0x0e0...0x2df 6-7..37 8..128 entry ID table
+ * size conf'ed via ctrl2::RFFN
+ * (mx6, vf610)
+ */
+ u32 _reserved4[256]; /* 0x480 */
+ u32 rximr[64]; /* 0x880 - Not affected by Soft Reset */
+ u32 _reserved5[24]; /* 0x980 */
+ u32 gfwr_mx6; /* 0x9e0 - MX6 */
+ u32 _reserved6[39]; /* 0x9e4 */
+ u32 _rxfir[6]; /* 0xa80 */
+ u32 _reserved8[2]; /* 0xa98 */
+ u32 _rxmgmask; /* 0xaa0 */
+ u32 _rxfgmask; /* 0xaa4 */
+ u32 _rx14mask; /* 0xaa8 */
+ u32 _rx15mask; /* 0xaac */
+ u32 tx_smb[4]; /* 0xab0 */
+ u32 rx_smb0[4]; /* 0xac0 */
+ u32 rx_smb1[4]; /* 0xad0 */
+ );
+ u32 mecr; /* 0xae0 */
+ u32 erriar; /* 0xae4 */
+ u32 erridpr; /* 0xae8 */
+ u32 errippr; /* 0xaec */
+ u32 rerrar; /* 0xaf0 */
+ u32 rerrdr; /* 0xaf4 */
+ u32 rerrsynr; /* 0xaf8 */
+ u32 errsr; /* 0xafc */
+ u32 _reserved7[64]; /* 0xb00 */
+ u32 fdctrl; /* 0xc00 - Not affected by Soft Reset */
+ u32 fdcbt; /* 0xc04 - Not affected by Soft Reset */
+ u32 fdcrc; /* 0xc08 */
+ u32 _reserved9[199]; /* 0xc0c */
+ struct_group(init_fd,
+ u32 tx_smb_fd[18]; /* 0xf28 */
+ u32 rx_smb0_fd[18]; /* 0xf70 */
+ u32 rx_smb1_fd[18]; /* 0xfb8 */
+ );
+};
+
+static_assert(sizeof(struct flexcan_regs) == 0x4 * 18 + 0xfb8);
+
+static const struct flexcan_devtype_data fsl_mcf5441x_devtype_data = {
+ .quirks = FLEXCAN_QUIRK_BROKEN_PERR_STATE |
+ FLEXCAN_QUIRK_NR_IRQ_3 | FLEXCAN_QUIRK_NR_MB_16 |
+ FLEXCAN_QUIRK_SUPPPORT_RX_FIFO,
+};
+
+static const struct flexcan_devtype_data fsl_p1010_devtype_data = {
+ .quirks = FLEXCAN_QUIRK_BROKEN_WERR_STATE |
+ FLEXCAN_QUIRK_BROKEN_PERR_STATE |
+ FLEXCAN_QUIRK_DEFAULT_BIG_ENDIAN |
+ FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX |
+ FLEXCAN_QUIRK_SUPPPORT_RX_FIFO,
+};
+
+static const struct flexcan_devtype_data fsl_imx25_devtype_data = {
+ .quirks = FLEXCAN_QUIRK_BROKEN_WERR_STATE |
+ FLEXCAN_QUIRK_BROKEN_PERR_STATE |
+ FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX |
+ FLEXCAN_QUIRK_SUPPPORT_RX_FIFO,
+};
+
+static const struct flexcan_devtype_data fsl_imx28_devtype_data = {
+ .quirks = FLEXCAN_QUIRK_BROKEN_PERR_STATE |
+ FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX |
+ FLEXCAN_QUIRK_SUPPPORT_RX_FIFO,
+};
+
+static const struct flexcan_devtype_data fsl_imx6q_devtype_data = {
+ .quirks = FLEXCAN_QUIRK_DISABLE_RXFG | FLEXCAN_QUIRK_ENABLE_EACEN_RRS |
+ FLEXCAN_QUIRK_USE_RX_MAILBOX | FLEXCAN_QUIRK_BROKEN_PERR_STATE |
+ FLEXCAN_QUIRK_SETUP_STOP_MODE_GPR |
+ FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX |
+ FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX_RTR,
+};
+
+static const struct flexcan_devtype_data fsl_imx8qm_devtype_data = {
+ .quirks = FLEXCAN_QUIRK_DISABLE_RXFG | FLEXCAN_QUIRK_ENABLE_EACEN_RRS |
+ FLEXCAN_QUIRK_USE_RX_MAILBOX | FLEXCAN_QUIRK_BROKEN_PERR_STATE |
+ FLEXCAN_QUIRK_SUPPORT_FD | FLEXCAN_QUIRK_SETUP_STOP_MODE_SCFW |
+ FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX |
+ FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX_RTR,
+};
+
+static struct flexcan_devtype_data fsl_imx8mp_devtype_data = {
+ .quirks = FLEXCAN_QUIRK_DISABLE_RXFG | FLEXCAN_QUIRK_ENABLE_EACEN_RRS |
+ FLEXCAN_QUIRK_DISABLE_MECR | FLEXCAN_QUIRK_USE_RX_MAILBOX |
+ FLEXCAN_QUIRK_BROKEN_PERR_STATE | FLEXCAN_QUIRK_SETUP_STOP_MODE_GPR |
+ FLEXCAN_QUIRK_SUPPORT_FD | FLEXCAN_QUIRK_SUPPORT_ECC |
+ FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX |
+ FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX_RTR,
+};
+
+static const struct flexcan_devtype_data fsl_vf610_devtype_data = {
+ .quirks = FLEXCAN_QUIRK_DISABLE_RXFG | FLEXCAN_QUIRK_ENABLE_EACEN_RRS |
+ FLEXCAN_QUIRK_DISABLE_MECR | FLEXCAN_QUIRK_USE_RX_MAILBOX |
+ FLEXCAN_QUIRK_BROKEN_PERR_STATE | FLEXCAN_QUIRK_SUPPORT_ECC |
+ FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX |
+ FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX_RTR,
+};
+
+static const struct flexcan_devtype_data fsl_ls1021a_r2_devtype_data = {
+ .quirks = FLEXCAN_QUIRK_DISABLE_RXFG | FLEXCAN_QUIRK_ENABLE_EACEN_RRS |
+ FLEXCAN_QUIRK_BROKEN_PERR_STATE | FLEXCAN_QUIRK_USE_RX_MAILBOX |
+ FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX |
+ FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX_RTR,
+};
+
+static const struct flexcan_devtype_data fsl_lx2160a_r1_devtype_data = {
+ .quirks = FLEXCAN_QUIRK_DISABLE_RXFG | FLEXCAN_QUIRK_ENABLE_EACEN_RRS |
+ FLEXCAN_QUIRK_DISABLE_MECR | FLEXCAN_QUIRK_BROKEN_PERR_STATE |
+ FLEXCAN_QUIRK_USE_RX_MAILBOX | FLEXCAN_QUIRK_SUPPORT_FD |
+ FLEXCAN_QUIRK_SUPPORT_ECC |
+ FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX |
+ FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX_RTR,
+};
+
+static const struct can_bittiming_const flexcan_bittiming_const = {
+ .name = DRV_NAME,
+ .tseg1_min = 4,
+ .tseg1_max = 16,
+ .tseg2_min = 2,
+ .tseg2_max = 8,
+ .sjw_max = 4,
+ .brp_min = 1,
+ .brp_max = 256,
+ .brp_inc = 1,
+};
+
+static const struct can_bittiming_const flexcan_fd_bittiming_const = {
+ .name = DRV_NAME,
+ .tseg1_min = 2,
+ .tseg1_max = 96,
+ .tseg2_min = 2,
+ .tseg2_max = 32,
+ .sjw_max = 16,
+ .brp_min = 1,
+ .brp_max = 1024,
+ .brp_inc = 1,
+};
+
+static const struct can_bittiming_const flexcan_fd_data_bittiming_const = {
+ .name = DRV_NAME,
+ .tseg1_min = 2,
+ .tseg1_max = 39,
+ .tseg2_min = 2,
+ .tseg2_max = 8,
+ .sjw_max = 4,
+ .brp_min = 1,
+ .brp_max = 1024,
+ .brp_inc = 1,
+};
+
+/* FlexCAN module is essentially modelled as a little-endian IP in most
+ * SoCs, i.e the registers as well as the message buffer areas are
+ * implemented in a little-endian fashion.
+ *
+ * However there are some SoCs (e.g. LS1021A) which implement the FlexCAN
+ * module in a big-endian fashion (i.e the registers as well as the
+ * message buffer areas are implemented in a big-endian way).
+ *
+ * In addition, the FlexCAN module can be found on SoCs having ARM or
+ * PPC cores. So, we need to abstract off the register read/write
+ * functions, ensuring that these cater to all the combinations of module
+ * endianness and underlying CPU endianness.
+ */
+static inline u32 flexcan_read_be(void __iomem *addr)
+{
+ return ioread32be(addr);
+}
+
+static inline void flexcan_write_be(u32 val, void __iomem *addr)
+{
+ iowrite32be(val, addr);
+}
+
+static inline u32 flexcan_read_le(void __iomem *addr)
+{
+ return ioread32(addr);
+}
+
+static inline void flexcan_write_le(u32 val, void __iomem *addr)
+{
+ iowrite32(val, addr);
+}
+
+static struct flexcan_mb __iomem *flexcan_get_mb(const struct flexcan_priv *priv,
+ u8 mb_index)
+{
+ u8 bank_size;
+ bool bank;
+
+ if (WARN_ON(mb_index >= priv->mb_count))
+ return NULL;
+
+ bank_size = sizeof(priv->regs->mb[0]) / priv->mb_size;
+
+ bank = mb_index >= bank_size;
+ if (bank)
+ mb_index -= bank_size;
+
+ return (struct flexcan_mb __iomem *)
+ (&priv->regs->mb[bank][priv->mb_size * mb_index]);
+}
+
+static int flexcan_low_power_enter_ack(struct flexcan_priv *priv)
+{
+ struct flexcan_regs __iomem *regs = priv->regs;
+ unsigned int timeout = FLEXCAN_TIMEOUT_US / 10;
+
+ while (timeout-- && !(priv->read(®s->mcr) & FLEXCAN_MCR_LPM_ACK))
+ udelay(10);
+
+ if (!(priv->read(®s->mcr) & FLEXCAN_MCR_LPM_ACK))
+ return -ETIMEDOUT;
+
+ return 0;
+}
+
+static int flexcan_low_power_exit_ack(struct flexcan_priv *priv)
+{
+ struct flexcan_regs __iomem *regs = priv->regs;
+ unsigned int timeout = FLEXCAN_TIMEOUT_US / 10;
+
+ while (timeout-- && (priv->read(®s->mcr) & FLEXCAN_MCR_LPM_ACK))
+ udelay(10);
+
+ if (priv->read(®s->mcr) & FLEXCAN_MCR_LPM_ACK)
+ return -ETIMEDOUT;
+
+ return 0;
+}
+
+static void flexcan_enable_wakeup_irq(struct flexcan_priv *priv, bool enable)
+{
+ struct flexcan_regs __iomem *regs = priv->regs;
+ u32 reg_mcr;
+
+ reg_mcr = priv->read(®s->mcr);
+
+ if (enable)
+ reg_mcr |= FLEXCAN_MCR_WAK_MSK;
+ else
+ reg_mcr &= ~FLEXCAN_MCR_WAK_MSK;
+
+ priv->write(reg_mcr, ®s->mcr);
+}
+
+static int flexcan_stop_mode_enable_scfw(struct flexcan_priv *priv, bool enabled)
+{
+ u8 idx = priv->scu_idx;
+ u32 rsrc_id, val;
+
+ rsrc_id = IMX_SC_R_CAN(idx);
+
+ if (enabled)
+ val = 1;
+ else
+ val = 0;
+
+ /* stop mode request via scu firmware */
+ return imx_sc_misc_set_control(priv->sc_ipc_handle, rsrc_id,
+ IMX_SC_C_IPG_STOP, val);
+}
+
+static inline int flexcan_enter_stop_mode(struct flexcan_priv *priv)
+{
+ struct flexcan_regs __iomem *regs = priv->regs;
+ u32 reg_mcr;
+ int ret;
+
+ reg_mcr = priv->read(®s->mcr);
+ reg_mcr |= FLEXCAN_MCR_SLF_WAK;
+ priv->write(reg_mcr, ®s->mcr);
+
+ /* enable stop request */
+ if (priv->devtype_data.quirks & FLEXCAN_QUIRK_SETUP_STOP_MODE_SCFW) {
+ ret = flexcan_stop_mode_enable_scfw(priv, true);
+ if (ret < 0)
+ return ret;
+ } else {
+ regmap_update_bits(priv->stm.gpr, priv->stm.req_gpr,
+ 1 << priv->stm.req_bit, 1 << priv->stm.req_bit);
+ }
+
+ return flexcan_low_power_enter_ack(priv);
+}
+
+static inline int flexcan_exit_stop_mode(struct flexcan_priv *priv)
+{
+ struct flexcan_regs __iomem *regs = priv->regs;
+ u32 reg_mcr;
+ int ret;
+
+ /* remove stop request */
+ if (priv->devtype_data.quirks & FLEXCAN_QUIRK_SETUP_STOP_MODE_SCFW) {
+ ret = flexcan_stop_mode_enable_scfw(priv, false);
+ if (ret < 0)
+ return ret;
+ } else {
+ regmap_update_bits(priv->stm.gpr, priv->stm.req_gpr,
+ 1 << priv->stm.req_bit, 0);
+ }
+
+ reg_mcr = priv->read(®s->mcr);
+ reg_mcr &= ~FLEXCAN_MCR_SLF_WAK;
+ priv->write(reg_mcr, ®s->mcr);
+
+ return flexcan_low_power_exit_ack(priv);
+}
+
+static inline void flexcan_error_irq_enable(const struct flexcan_priv *priv)
+{
+ struct flexcan_regs __iomem *regs = priv->regs;
+ u32 reg_ctrl = (priv->reg_ctrl_default | FLEXCAN_CTRL_ERR_MSK);
+
+ priv->write(reg_ctrl, ®s->ctrl);
+}
+
+static inline void flexcan_error_irq_disable(const struct flexcan_priv *priv)
+{
+ struct flexcan_regs __iomem *regs = priv->regs;
+ u32 reg_ctrl = (priv->reg_ctrl_default & ~FLEXCAN_CTRL_ERR_MSK);
+
+ priv->write(reg_ctrl, ®s->ctrl);
+}
+
+static int flexcan_clks_enable(const struct flexcan_priv *priv)
+{
+ int err = 0;
+
+ if (priv->clk_ipg) {
+ err = clk_prepare_enable(priv->clk_ipg);
+ if (err)
+ return err;
+ }
+
+ if (priv->clk_per) {
+ err = clk_prepare_enable(priv->clk_per);
+ if (err)
+ clk_disable_unprepare(priv->clk_ipg);
+ }
+
+ return err;
+}
+
+static void flexcan_clks_disable(const struct flexcan_priv *priv)
+{
+ clk_disable_unprepare(priv->clk_per);
+ clk_disable_unprepare(priv->clk_ipg);
+}
+
+static inline int flexcan_transceiver_enable(const struct flexcan_priv *priv)
+{
+ if (!priv->reg_xceiver)
+ return 0;
+
+ return regulator_enable(priv->reg_xceiver);
+}
+
+static inline int flexcan_transceiver_disable(const struct flexcan_priv *priv)
+{
+ if (!priv->reg_xceiver)
+ return 0;
+
+ return regulator_disable(priv->reg_xceiver);
+}
+
+static int flexcan_chip_enable(struct flexcan_priv *priv)
+{
+ struct flexcan_regs __iomem *regs = priv->regs;
+ u32 reg;
+
+ reg = priv->read(®s->mcr);
+ reg &= ~FLEXCAN_MCR_MDIS;
+ priv->write(reg, ®s->mcr);
+
+ return flexcan_low_power_exit_ack(priv);
+}
+
+static int flexcan_chip_disable(struct flexcan_priv *priv)
+{
+ struct flexcan_regs __iomem *regs = priv->regs;
+ u32 reg;
+
+ reg = priv->read(®s->mcr);
+ reg |= FLEXCAN_MCR_MDIS;
+ priv->write(reg, ®s->mcr);
+
+ return flexcan_low_power_enter_ack(priv);
+}
+
+static int flexcan_chip_freeze(struct flexcan_priv *priv)
+{
+ struct flexcan_regs __iomem *regs = priv->regs;
+ unsigned int timeout;
+ u32 bitrate = priv->can.bittiming.bitrate;
+ u32 reg;
+
+ if (bitrate)
+ timeout = 1000 * 1000 * 10 / bitrate;
+ else
+ timeout = FLEXCAN_TIMEOUT_US / 10;
+
+ reg = priv->read(®s->mcr);
+ reg |= FLEXCAN_MCR_FRZ | FLEXCAN_MCR_HALT;
+ priv->write(reg, ®s->mcr);
+
+ while (timeout-- && !(priv->read(®s->mcr) & FLEXCAN_MCR_FRZ_ACK))
+ udelay(100);
+
+ if (!(priv->read(®s->mcr) & FLEXCAN_MCR_FRZ_ACK))
+ return -ETIMEDOUT;
+
+ return 0;
+}
+
+static int flexcan_chip_unfreeze(struct flexcan_priv *priv)
+{
+ struct flexcan_regs __iomem *regs = priv->regs;
+ unsigned int timeout = FLEXCAN_TIMEOUT_US / 10;
+ u32 reg;
+
+ reg = priv->read(®s->mcr);
+ reg &= ~FLEXCAN_MCR_HALT;
+ priv->write(reg, ®s->mcr);
+
+ while (timeout-- && (priv->read(®s->mcr) & FLEXCAN_MCR_FRZ_ACK))
+ udelay(10);
+
+ if (priv->read(®s->mcr) & FLEXCAN_MCR_FRZ_ACK)
+ return -ETIMEDOUT;
+
+ return 0;
+}
+
+static int flexcan_chip_softreset(struct flexcan_priv *priv)
+{
+ struct flexcan_regs __iomem *regs = priv->regs;
+ unsigned int timeout = FLEXCAN_TIMEOUT_US / 10;
+
+ priv->write(FLEXCAN_MCR_SOFTRST, ®s->mcr);
+ while (timeout-- && (priv->read(®s->mcr) & FLEXCAN_MCR_SOFTRST))
+ udelay(10);
+
+ if (priv->read(®s->mcr) & FLEXCAN_MCR_SOFTRST)
+ return -ETIMEDOUT;
+
+ return 0;
+}
+
+static int __flexcan_get_berr_counter(const struct net_device *dev,
+ struct can_berr_counter *bec)
+{
+ const struct flexcan_priv *priv = netdev_priv(dev);
+ struct flexcan_regs __iomem *regs = priv->regs;
+ u32 reg = priv->read(®s->ecr);
+
+ bec->txerr = (reg >> 0) & 0xff;
+ bec->rxerr = (reg >> 8) & 0xff;
+
+ return 0;
+}
+
+static int flexcan_get_berr_counter(const struct net_device *dev,
+ struct can_berr_counter *bec)
+{
+ const struct flexcan_priv *priv = netdev_priv(dev);
+ int err;
+
+ err = pm_runtime_get_sync(priv->dev);
+ if (err < 0) {
+ pm_runtime_put_noidle(priv->dev);
+ return err;
+ }
+
+ err = __flexcan_get_berr_counter(dev, bec);
+
+ pm_runtime_put(priv->dev);
+
+ return err;
+}
+
+static netdev_tx_t flexcan_start_xmit(struct sk_buff *skb, struct net_device *dev)
+{
+ const struct flexcan_priv *priv = netdev_priv(dev);
+ struct canfd_frame *cfd = (struct canfd_frame *)skb->data;
+ u32 can_id;
+ u32 data;
+ u32 ctrl = FLEXCAN_MB_CODE_TX_DATA | ((can_fd_len2dlc(cfd->len)) << 16);
+ int i;
+
+ if (can_dropped_invalid_skb(dev, skb))
+ return NETDEV_TX_OK;
+
+ netif_stop_queue(dev);
+
+ if (cfd->can_id & CAN_EFF_FLAG) {
+ can_id = cfd->can_id & CAN_EFF_MASK;
+ ctrl |= FLEXCAN_MB_CNT_IDE | FLEXCAN_MB_CNT_SRR;
+ } else {
+ can_id = (cfd->can_id & CAN_SFF_MASK) << 18;
+ }
+
+ if (cfd->can_id & CAN_RTR_FLAG)
+ ctrl |= FLEXCAN_MB_CNT_RTR;
+
+ if (can_is_canfd_skb(skb)) {
+ ctrl |= FLEXCAN_MB_CNT_EDL;
+
+ if (cfd->flags & CANFD_BRS)
+ ctrl |= FLEXCAN_MB_CNT_BRS;
+ }
+
+ for (i = 0; i < cfd->len; i += sizeof(u32)) {
+ data = be32_to_cpup((__be32 *)&cfd->data[i]);
+ priv->write(data, &priv->tx_mb->data[i / sizeof(u32)]);
+ }
+
+ can_put_echo_skb(skb, dev, 0, 0);
+
+ priv->write(can_id, &priv->tx_mb->can_id);
+ priv->write(ctrl, &priv->tx_mb->can_ctrl);
+
+ /* Errata ERR005829 step8:
+ * Write twice INACTIVE(0x8) code to first MB.
+ */
+ priv->write(FLEXCAN_MB_CODE_TX_INACTIVE,
+ &priv->tx_mb_reserved->can_ctrl);
+ priv->write(FLEXCAN_MB_CODE_TX_INACTIVE,
+ &priv->tx_mb_reserved->can_ctrl);
+
+ return NETDEV_TX_OK;
+}
+
+static void flexcan_irq_bus_err(struct net_device *dev, u32 reg_esr)
+{
+ struct flexcan_priv *priv = netdev_priv(dev);
+ struct flexcan_regs __iomem *regs = priv->regs;
+ struct sk_buff *skb;
+ struct can_frame *cf;
+ bool rx_errors = false, tx_errors = false;
+ u32 timestamp;
+ int err;
+
+ timestamp = priv->read(®s->timer) << 16;
+
+ skb = alloc_can_err_skb(dev, &cf);
+ if (unlikely(!skb))
+ return;
+
+ cf->can_id |= CAN_ERR_PROT | CAN_ERR_BUSERROR;
+
+ if (reg_esr & FLEXCAN_ESR_BIT1_ERR) {
+ netdev_dbg(dev, "BIT1_ERR irq\n");
+ cf->data[2] |= CAN_ERR_PROT_BIT1;
+ tx_errors = true;
+ }
+ if (reg_esr & FLEXCAN_ESR_BIT0_ERR) {
+ netdev_dbg(dev, "BIT0_ERR irq\n");
+ cf->data[2] |= CAN_ERR_PROT_BIT0;
+ tx_errors = true;
+ }
+ if (reg_esr & FLEXCAN_ESR_ACK_ERR) {
+ netdev_dbg(dev, "ACK_ERR irq\n");
+ cf->can_id |= CAN_ERR_ACK;
+ cf->data[3] = CAN_ERR_PROT_LOC_ACK;
+ tx_errors = true;
+ }
+ if (reg_esr & FLEXCAN_ESR_CRC_ERR) {
+ netdev_dbg(dev, "CRC_ERR irq\n");
+ cf->data[2] |= CAN_ERR_PROT_BIT;
+ cf->data[3] = CAN_ERR_PROT_LOC_CRC_SEQ;
+ rx_errors = true;
+ }
+ if (reg_esr & FLEXCAN_ESR_FRM_ERR) {
+ netdev_dbg(dev, "FRM_ERR irq\n");
+ cf->data[2] |= CAN_ERR_PROT_FORM;
+ rx_errors = true;
+ }
+ if (reg_esr & FLEXCAN_ESR_STF_ERR) {
+ netdev_dbg(dev, "STF_ERR irq\n");
+ cf->data[2] |= CAN_ERR_PROT_STUFF;
+ rx_errors = true;
+ }
+
+ priv->can.can_stats.bus_error++;
+ if (rx_errors)
+ dev->stats.rx_errors++;
+ if (tx_errors)
+ dev->stats.tx_errors++;
+
+ err = can_rx_offload_queue_sorted(&priv->offload, skb, timestamp);
+ if (err)
+ dev->stats.rx_fifo_errors++;
+}
+
+static void flexcan_irq_state(struct net_device *dev, u32 reg_esr)
+{
+ struct flexcan_priv *priv = netdev_priv(dev);
+ struct flexcan_regs __iomem *regs = priv->regs;
+ struct sk_buff *skb;
+ struct can_frame *cf;
+ enum can_state new_state, rx_state, tx_state;
+ int flt;
+ struct can_berr_counter bec;
+ u32 timestamp;
+ int err;
+
+ flt = reg_esr & FLEXCAN_ESR_FLT_CONF_MASK;
+ if (likely(flt == FLEXCAN_ESR_FLT_CONF_ACTIVE)) {
+ tx_state = unlikely(reg_esr & FLEXCAN_ESR_TX_WRN) ?
+ CAN_STATE_ERROR_WARNING : CAN_STATE_ERROR_ACTIVE;
+ rx_state = unlikely(reg_esr & FLEXCAN_ESR_RX_WRN) ?
+ CAN_STATE_ERROR_WARNING : CAN_STATE_ERROR_ACTIVE;
+ new_state = max(tx_state, rx_state);
+ } else {
+ __flexcan_get_berr_counter(dev, &bec);
+ new_state = flt == FLEXCAN_ESR_FLT_CONF_PASSIVE ?
+ CAN_STATE_ERROR_PASSIVE : CAN_STATE_BUS_OFF;
+ rx_state = bec.rxerr >= bec.txerr ? new_state : 0;
+ tx_state = bec.rxerr <= bec.txerr ? new_state : 0;
+ }
+
+ /* state hasn't changed */
+ if (likely(new_state == priv->can.state))
+ return;
+
+ timestamp = priv->read(®s->timer) << 16;
+
+ skb = alloc_can_err_skb(dev, &cf);
+ if (unlikely(!skb))
+ return;
+
+ can_change_state(dev, cf, tx_state, rx_state);
+
+ if (unlikely(new_state == CAN_STATE_BUS_OFF))
+ can_bus_off(dev);
+
+ err = can_rx_offload_queue_sorted(&priv->offload, skb, timestamp);
+ if (err)
+ dev->stats.rx_fifo_errors++;
+}
+
+static inline u64 flexcan_read64_mask(struct flexcan_priv *priv, void __iomem *addr, u64 mask)
+{
+ u64 reg = 0;
+
+ if (upper_32_bits(mask))
+ reg = (u64)priv->read(addr - 4) << 32;
+ if (lower_32_bits(mask))
+ reg |= priv->read(addr);
+
+ return reg & mask;
+}
+
+static inline void flexcan_write64(struct flexcan_priv *priv, u64 val, void __iomem *addr)
+{
+ if (upper_32_bits(val))
+ priv->write(upper_32_bits(val), addr - 4);
+ if (lower_32_bits(val))
+ priv->write(lower_32_bits(val), addr);
+}
+
+static inline u64 flexcan_read_reg_iflag_rx(struct flexcan_priv *priv)
+{
+ return flexcan_read64_mask(priv, &priv->regs->iflag1, priv->rx_mask);
+}
+
+static inline u64 flexcan_read_reg_iflag_tx(struct flexcan_priv *priv)
+{
+ return flexcan_read64_mask(priv, &priv->regs->iflag1, priv->tx_mask);
+}
+
+static inline struct flexcan_priv *rx_offload_to_priv(struct can_rx_offload *offload)
+{
+ return container_of(offload, struct flexcan_priv, offload);
+}
+
+static struct sk_buff *flexcan_mailbox_read(struct can_rx_offload *offload,
+ unsigned int n, u32 *timestamp,
+ bool drop)
+{
+ struct flexcan_priv *priv = rx_offload_to_priv(offload);
+ struct flexcan_regs __iomem *regs = priv->regs;
+ struct flexcan_mb __iomem *mb;
+ struct sk_buff *skb;
+ struct canfd_frame *cfd;
+ u32 reg_ctrl, reg_id, reg_iflag1;
+ int i;
+
+ if (unlikely(drop)) {
+ skb = ERR_PTR(-ENOBUFS);
+ goto mark_as_read;
+ }
+
+ mb = flexcan_get_mb(priv, n);
+
+ if (priv->devtype_data.quirks & FLEXCAN_QUIRK_USE_RX_MAILBOX) {
+ u32 code;
+
+ do {
+ reg_ctrl = priv->read(&mb->can_ctrl);
+ } while (reg_ctrl & FLEXCAN_MB_CODE_RX_BUSY_BIT);
+
+ /* is this MB empty? */
+ code = reg_ctrl & FLEXCAN_MB_CODE_MASK;
+ if ((code != FLEXCAN_MB_CODE_RX_FULL) &&
+ (code != FLEXCAN_MB_CODE_RX_OVERRUN))
+ return NULL;
+
+ if (code == FLEXCAN_MB_CODE_RX_OVERRUN) {
+ /* This MB was overrun, we lost data */
+ offload->dev->stats.rx_over_errors++;
+ offload->dev->stats.rx_errors++;
+ }
+ } else {
+ reg_iflag1 = priv->read(®s->iflag1);
+ if (!(reg_iflag1 & FLEXCAN_IFLAG_RX_FIFO_AVAILABLE))
+ return NULL;
+
+ reg_ctrl = priv->read(&mb->can_ctrl);
+ }
+
+ if (reg_ctrl & FLEXCAN_MB_CNT_EDL)
+ skb = alloc_canfd_skb(offload->dev, &cfd);
+ else
+ skb = alloc_can_skb(offload->dev, (struct can_frame **)&cfd);
+ if (unlikely(!skb)) {
+ skb = ERR_PTR(-ENOMEM);
+ goto mark_as_read;
+ }
+
+ /* increase timstamp to full 32 bit */
+ *timestamp = reg_ctrl << 16;
+
+ reg_id = priv->read(&mb->can_id);
+ if (reg_ctrl & FLEXCAN_MB_CNT_IDE)
+ cfd->can_id = ((reg_id >> 0) & CAN_EFF_MASK) | CAN_EFF_FLAG;
+ else
+ cfd->can_id = (reg_id >> 18) & CAN_SFF_MASK;
+
+ if (reg_ctrl & FLEXCAN_MB_CNT_EDL) {
+ cfd->len = can_fd_dlc2len((reg_ctrl >> 16) & 0xf);
+
+ if (reg_ctrl & FLEXCAN_MB_CNT_BRS)
+ cfd->flags |= CANFD_BRS;
+ } else {
+ cfd->len = can_cc_dlc2len((reg_ctrl >> 16) & 0xf);
+
+ if (reg_ctrl & FLEXCAN_MB_CNT_RTR)
+ cfd->can_id |= CAN_RTR_FLAG;
+ }
+
+ if (reg_ctrl & FLEXCAN_MB_CNT_ESI)
+ cfd->flags |= CANFD_ESI;
+
+ for (i = 0; i < cfd->len; i += sizeof(u32)) {
+ __be32 data = cpu_to_be32(priv->read(&mb->data[i / sizeof(u32)]));
+ *(__be32 *)(cfd->data + i) = data;
+ }
+
+ mark_as_read:
+ if (priv->devtype_data.quirks & FLEXCAN_QUIRK_USE_RX_MAILBOX)
+ flexcan_write64(priv, FLEXCAN_IFLAG_MB(n), ®s->iflag1);
+ else
+ priv->write(FLEXCAN_IFLAG_RX_FIFO_AVAILABLE, ®s->iflag1);
+
+ /* Read the Free Running Timer. It is optional but recommended
+ * to unlock Mailbox as soon as possible and make it available
+ * for reception.
+ */
+ priv->read(®s->timer);
+
+ return skb;
+}
+
+static irqreturn_t flexcan_irq(int irq, void *dev_id)
+{
+ struct net_device *dev = dev_id;
+ struct net_device_stats *stats = &dev->stats;
+ struct flexcan_priv *priv = netdev_priv(dev);
+ struct flexcan_regs __iomem *regs = priv->regs;
+ irqreturn_t handled = IRQ_NONE;
+ u64 reg_iflag_tx;
+ u32 reg_esr;
+ enum can_state last_state = priv->can.state;
+
+ /* reception interrupt */
+ if (priv->devtype_data.quirks & FLEXCAN_QUIRK_USE_RX_MAILBOX) {
+ u64 reg_iflag_rx;
+ int ret;
+
+ while ((reg_iflag_rx = flexcan_read_reg_iflag_rx(priv))) {
+ handled = IRQ_HANDLED;
+ ret = can_rx_offload_irq_offload_timestamp(&priv->offload,
+ reg_iflag_rx);
+ if (!ret)
+ break;
+ }
+ } else {
+ u32 reg_iflag1;
+
+ reg_iflag1 = priv->read(®s->iflag1);
+ if (reg_iflag1 & FLEXCAN_IFLAG_RX_FIFO_AVAILABLE) {
+ handled = IRQ_HANDLED;
+ can_rx_offload_irq_offload_fifo(&priv->offload);
+ }
+
+ /* FIFO overflow interrupt */
+ if (reg_iflag1 & FLEXCAN_IFLAG_RX_FIFO_OVERFLOW) {
+ handled = IRQ_HANDLED;
+ priv->write(FLEXCAN_IFLAG_RX_FIFO_OVERFLOW,
+ ®s->iflag1);
+ dev->stats.rx_over_errors++;
+ dev->stats.rx_errors++;
+ }
+ }
+
+ reg_iflag_tx = flexcan_read_reg_iflag_tx(priv);
+
+ /* transmission complete interrupt */
+ if (reg_iflag_tx & priv->tx_mask) {
+ u32 reg_ctrl = priv->read(&priv->tx_mb->can_ctrl);
+
+ handled = IRQ_HANDLED;
+ stats->tx_bytes +=
+ can_rx_offload_get_echo_skb(&priv->offload, 0,
+ reg_ctrl << 16, NULL);
+ stats->tx_packets++;
+ can_led_event(dev, CAN_LED_EVENT_TX);
+
+ /* after sending a RTR frame MB is in RX mode */
+ priv->write(FLEXCAN_MB_CODE_TX_INACTIVE,
+ &priv->tx_mb->can_ctrl);
+ flexcan_write64(priv, priv->tx_mask, ®s->iflag1);
+ netif_wake_queue(dev);
+ }
+
+ reg_esr = priv->read(®s->esr);
+
+ /* ACK all bus error, state change and wake IRQ sources */
+ if (reg_esr & (FLEXCAN_ESR_ALL_INT | FLEXCAN_ESR_WAK_INT)) {
+ handled = IRQ_HANDLED;
+ priv->write(reg_esr & (FLEXCAN_ESR_ALL_INT | FLEXCAN_ESR_WAK_INT), ®s->esr);
+ }
+
+ /* state change interrupt or broken error state quirk fix is enabled */
+ if ((reg_esr & FLEXCAN_ESR_ERR_STATE) ||
+ (priv->devtype_data.quirks & (FLEXCAN_QUIRK_BROKEN_WERR_STATE |
+ FLEXCAN_QUIRK_BROKEN_PERR_STATE)))
+ flexcan_irq_state(dev, reg_esr);
+
+ /* bus error IRQ - handle if bus error reporting is activated */
+ if ((reg_esr & FLEXCAN_ESR_ERR_BUS) &&
+ (priv->can.ctrlmode & CAN_CTRLMODE_BERR_REPORTING))
+ flexcan_irq_bus_err(dev, reg_esr);
+
+ /* availability of error interrupt among state transitions in case
+ * bus error reporting is de-activated and
+ * FLEXCAN_QUIRK_BROKEN_PERR_STATE is enabled:
+ * +--------------------------------------------------------------+
+ * | +----------------------------------------------+ [stopped / |
+ * | | | sleeping] -+
+ * +-+-> active <-> warning <-> passive -> bus off -+
+ * ___________^^^^^^^^^^^^_______________________________
+ * disabled(1) enabled disabled
+ *
+ * (1): enabled if FLEXCAN_QUIRK_BROKEN_WERR_STATE is enabled
+ */
+ if ((last_state != priv->can.state) &&
+ (priv->devtype_data.quirks & FLEXCAN_QUIRK_BROKEN_PERR_STATE) &&
+ !(priv->can.ctrlmode & CAN_CTRLMODE_BERR_REPORTING)) {
+ switch (priv->can.state) {
+ case CAN_STATE_ERROR_ACTIVE:
+ if (priv->devtype_data.quirks &
+ FLEXCAN_QUIRK_BROKEN_WERR_STATE)
+ flexcan_error_irq_enable(priv);
+ else
+ flexcan_error_irq_disable(priv);
+ break;
+
+ case CAN_STATE_ERROR_WARNING:
+ flexcan_error_irq_enable(priv);
+ break;
+
+ case CAN_STATE_ERROR_PASSIVE:
+ case CAN_STATE_BUS_OFF:
+ flexcan_error_irq_disable(priv);
+ break;
+
+ default:
+ break;
+ }
+ }
+
+ if (handled)
+ can_rx_offload_irq_finish(&priv->offload);
+
+ return handled;
+}
+
+static void flexcan_set_bittiming_ctrl(const struct net_device *dev)
+{
+ const struct flexcan_priv *priv = netdev_priv(dev);
+ const struct can_bittiming *bt = &priv->can.bittiming;
+ struct flexcan_regs __iomem *regs = priv->regs;
+ u32 reg;
+
+ reg = priv->read(®s->ctrl);
+ reg &= ~(FLEXCAN_CTRL_PRESDIV(0xff) |
+ FLEXCAN_CTRL_RJW(0x3) |
+ FLEXCAN_CTRL_PSEG1(0x7) |
+ FLEXCAN_CTRL_PSEG2(0x7) |
+ FLEXCAN_CTRL_PROPSEG(0x7));
+
+ reg |= FLEXCAN_CTRL_PRESDIV(bt->brp - 1) |
+ FLEXCAN_CTRL_PSEG1(bt->phase_seg1 - 1) |
+ FLEXCAN_CTRL_PSEG2(bt->phase_seg2 - 1) |
+ FLEXCAN_CTRL_RJW(bt->sjw - 1) |
+ FLEXCAN_CTRL_PROPSEG(bt->prop_seg - 1);
+
+ netdev_dbg(dev, "writing ctrl=0x%08x\n", reg);
+ priv->write(reg, ®s->ctrl);
+
+ /* print chip status */
+ netdev_dbg(dev, "%s: mcr=0x%08x ctrl=0x%08x\n", __func__,
+ priv->read(®s->mcr), priv->read(®s->ctrl));
+}
+
+static void flexcan_set_bittiming_cbt(const struct net_device *dev)
+{
+ struct flexcan_priv *priv = netdev_priv(dev);
+ struct can_bittiming *bt = &priv->can.bittiming;
+ struct can_bittiming *dbt = &priv->can.data_bittiming;
+ struct flexcan_regs __iomem *regs = priv->regs;
+ u32 reg_cbt, reg_fdctrl;
+
+ /* CBT */
+ /* CBT[EPSEG1] is 5 bit long and CBT[EPROPSEG] is 6 bit
+ * long. The can_calc_bittiming() tries to divide the tseg1
+ * equally between phase_seg1 and prop_seg, which may not fit
+ * in CBT register. Therefore, if phase_seg1 is more than
+ * possible value, increase prop_seg and decrease phase_seg1.
+ */
+ if (bt->phase_seg1 > 0x20) {
+ bt->prop_seg += (bt->phase_seg1 - 0x20);
+ bt->phase_seg1 = 0x20;
+ }
+
+ reg_cbt = FLEXCAN_CBT_BTF |
+ FIELD_PREP(FLEXCAN_CBT_EPRESDIV_MASK, bt->brp - 1) |
+ FIELD_PREP(FLEXCAN_CBT_ERJW_MASK, bt->sjw - 1) |
+ FIELD_PREP(FLEXCAN_CBT_EPROPSEG_MASK, bt->prop_seg - 1) |
+ FIELD_PREP(FLEXCAN_CBT_EPSEG1_MASK, bt->phase_seg1 - 1) |
+ FIELD_PREP(FLEXCAN_CBT_EPSEG2_MASK, bt->phase_seg2 - 1);
+
+ netdev_dbg(dev, "writing cbt=0x%08x\n", reg_cbt);
+ priv->write(reg_cbt, ®s->cbt);
+
+ if (priv->can.ctrlmode & CAN_CTRLMODE_FD) {
+ u32 reg_fdcbt, reg_ctrl2;
+
+ if (bt->brp != dbt->brp)
+ netdev_warn(dev, "Data brp=%d and brp=%d don't match, this may result in a phase error. Consider using different bitrate and/or data bitrate.\n",
+ dbt->brp, bt->brp);
+
+ /* FDCBT */
+ /* FDCBT[FPSEG1] is 3 bit long and FDCBT[FPROPSEG] is
+ * 5 bit long. The can_calc_bittiming tries to divide
+ * the tseg1 equally between phase_seg1 and prop_seg,
+ * which may not fit in FDCBT register. Therefore, if
+ * phase_seg1 is more than possible value, increase
+ * prop_seg and decrease phase_seg1
+ */
+ if (dbt->phase_seg1 > 0x8) {
+ dbt->prop_seg += (dbt->phase_seg1 - 0x8);
+ dbt->phase_seg1 = 0x8;
+ }
+
+ reg_fdcbt = priv->read(®s->fdcbt);
+ reg_fdcbt &= ~(FIELD_PREP(FLEXCAN_FDCBT_FPRESDIV_MASK, 0x3ff) |
+ FIELD_PREP(FLEXCAN_FDCBT_FRJW_MASK, 0x7) |
+ FIELD_PREP(FLEXCAN_FDCBT_FPROPSEG_MASK, 0x1f) |
+ FIELD_PREP(FLEXCAN_FDCBT_FPSEG1_MASK, 0x7) |
+ FIELD_PREP(FLEXCAN_FDCBT_FPSEG2_MASK, 0x7));
+
+ reg_fdcbt |= FIELD_PREP(FLEXCAN_FDCBT_FPRESDIV_MASK, dbt->brp - 1) |
+ FIELD_PREP(FLEXCAN_FDCBT_FRJW_MASK, dbt->sjw - 1) |
+ FIELD_PREP(FLEXCAN_FDCBT_FPROPSEG_MASK, dbt->prop_seg) |
+ FIELD_PREP(FLEXCAN_FDCBT_FPSEG1_MASK, dbt->phase_seg1 - 1) |
+ FIELD_PREP(FLEXCAN_FDCBT_FPSEG2_MASK, dbt->phase_seg2 - 1);
+
+ netdev_dbg(dev, "writing fdcbt=0x%08x\n", reg_fdcbt);
+ priv->write(reg_fdcbt, ®s->fdcbt);
+
+ /* CTRL2 */
+ reg_ctrl2 = priv->read(®s->ctrl2);
+ reg_ctrl2 &= ~FLEXCAN_CTRL2_ISOCANFDEN;
+ if (!(priv->can.ctrlmode & CAN_CTRLMODE_FD_NON_ISO))
+ reg_ctrl2 |= FLEXCAN_CTRL2_ISOCANFDEN;
+
+ netdev_dbg(dev, "writing ctrl2=0x%08x\n", reg_ctrl2);
+ priv->write(reg_ctrl2, ®s->ctrl2);
+ }
+
+ /* FDCTRL */
+ reg_fdctrl = priv->read(®s->fdctrl);
+ reg_fdctrl &= ~(FLEXCAN_FDCTRL_FDRATE |
+ FIELD_PREP(FLEXCAN_FDCTRL_TDCOFF, 0x1f));
+
+ if (priv->can.ctrlmode & CAN_CTRLMODE_FD) {
+ reg_fdctrl |= FLEXCAN_FDCTRL_FDRATE;
+
+ if (priv->can.ctrlmode & CAN_CTRLMODE_LOOPBACK) {
+ /* TDC must be disabled for Loop Back mode */
+ reg_fdctrl &= ~FLEXCAN_FDCTRL_TDCEN;
+ } else {
+ reg_fdctrl |= FLEXCAN_FDCTRL_TDCEN |
+ FIELD_PREP(FLEXCAN_FDCTRL_TDCOFF,
+ ((dbt->phase_seg1 - 1) +
+ dbt->prop_seg + 2) *
+ ((dbt->brp - 1 ) + 1));
+ }
+ }
+
+ netdev_dbg(dev, "writing fdctrl=0x%08x\n", reg_fdctrl);
+ priv->write(reg_fdctrl, ®s->fdctrl);
+
+ netdev_dbg(dev, "%s: mcr=0x%08x ctrl=0x%08x ctrl2=0x%08x fdctrl=0x%08x cbt=0x%08x fdcbt=0x%08x\n",
+ __func__,
+ priv->read(®s->mcr), priv->read(®s->ctrl),
+ priv->read(®s->ctrl2), priv->read(®s->fdctrl),
+ priv->read(®s->cbt), priv->read(®s->fdcbt));
+}
+
+static void flexcan_set_bittiming(struct net_device *dev)
+{
+ const struct flexcan_priv *priv = netdev_priv(dev);
+ struct flexcan_regs __iomem *regs = priv->regs;
+ u32 reg;
+
+ reg = priv->read(®s->ctrl);
+ reg &= ~(FLEXCAN_CTRL_LPB | FLEXCAN_CTRL_SMP |
+ FLEXCAN_CTRL_LOM);
+
+ if (priv->can.ctrlmode & CAN_CTRLMODE_LOOPBACK)
+ reg |= FLEXCAN_CTRL_LPB;
+ if (priv->can.ctrlmode & CAN_CTRLMODE_LISTENONLY)
+ reg |= FLEXCAN_CTRL_LOM;
+ if (priv->can.ctrlmode & CAN_CTRLMODE_3_SAMPLES)
+ reg |= FLEXCAN_CTRL_SMP;
+
+ netdev_dbg(dev, "writing ctrl=0x%08x\n", reg);
+ priv->write(reg, ®s->ctrl);
+
+ if (priv->can.ctrlmode_supported & CAN_CTRLMODE_FD)
+ return flexcan_set_bittiming_cbt(dev);
+ else
+ return flexcan_set_bittiming_ctrl(dev);
+}
+
+static void flexcan_ram_init(struct net_device *dev)
+{
+ struct flexcan_priv *priv = netdev_priv(dev);
+ struct flexcan_regs __iomem *regs = priv->regs;
+ u32 reg_ctrl2;
+
+ /* 11.8.3.13 Detection and correction of memory errors:
+ * CTRL2[WRMFRZ] grants write access to all memory positions
+ * that require initialization, ranging from 0x080 to 0xADF
+ * and from 0xF28 to 0xFFF when the CAN FD feature is enabled.
+ * The RXMGMASK, RX14MASK, RX15MASK, and RXFGMASK registers
+ * need to be initialized as well. MCR[RFEN] must not be set
+ * during memory initialization.
+ */
+ reg_ctrl2 = priv->read(®s->ctrl2);
+ reg_ctrl2 |= FLEXCAN_CTRL2_WRMFRZ;
+ priv->write(reg_ctrl2, ®s->ctrl2);
+
+ memset_io(®s->init, 0, sizeof(regs->init));
+
+ if (priv->can.ctrlmode & CAN_CTRLMODE_FD)
+ memset_io(®s->init_fd, 0, sizeof(regs->init_fd));
+
+ reg_ctrl2 &= ~FLEXCAN_CTRL2_WRMFRZ;
+ priv->write(reg_ctrl2, ®s->ctrl2);
+}
+
+static int flexcan_rx_offload_setup(struct net_device *dev)
+{
+ struct flexcan_priv *priv = netdev_priv(dev);
+ int err;
+
+ if (priv->can.ctrlmode & CAN_CTRLMODE_FD)
+ priv->mb_size = sizeof(struct flexcan_mb) + CANFD_MAX_DLEN;
+ else
+ priv->mb_size = sizeof(struct flexcan_mb) + CAN_MAX_DLEN;
+
+ if (priv->devtype_data.quirks & FLEXCAN_QUIRK_NR_MB_16)
+ priv->mb_count = 16;
+ else
+ priv->mb_count = (sizeof(priv->regs->mb[0]) / priv->mb_size) +
+ (sizeof(priv->regs->mb[1]) / priv->mb_size);
+
+ if (priv->devtype_data.quirks & FLEXCAN_QUIRK_USE_RX_MAILBOX)
+ priv->tx_mb_reserved =
+ flexcan_get_mb(priv, FLEXCAN_TX_MB_RESERVED_RX_MAILBOX);
+ else
+ priv->tx_mb_reserved =
+ flexcan_get_mb(priv, FLEXCAN_TX_MB_RESERVED_RX_FIFO);
+ priv->tx_mb_idx = priv->mb_count - 1;
+ priv->tx_mb = flexcan_get_mb(priv, priv->tx_mb_idx);
+ priv->tx_mask = FLEXCAN_IFLAG_MB(priv->tx_mb_idx);
+
+ priv->offload.mailbox_read = flexcan_mailbox_read;
+
+ if (priv->devtype_data.quirks & FLEXCAN_QUIRK_USE_RX_MAILBOX) {
+ priv->offload.mb_first = FLEXCAN_RX_MB_RX_MAILBOX_FIRST;
+ priv->offload.mb_last = priv->mb_count - 2;
+
+ priv->rx_mask = GENMASK_ULL(priv->offload.mb_last,
+ priv->offload.mb_first);
+ err = can_rx_offload_add_timestamp(dev, &priv->offload);
+ } else {
+ priv->rx_mask = FLEXCAN_IFLAG_RX_FIFO_OVERFLOW |
+ FLEXCAN_IFLAG_RX_FIFO_AVAILABLE;
+ err = can_rx_offload_add_fifo(dev, &priv->offload,
+ FLEXCAN_NAPI_WEIGHT);
+ }
+
+ return err;
+}
+
+static void flexcan_chip_interrupts_enable(const struct net_device *dev)
+{
+ const struct flexcan_priv *priv = netdev_priv(dev);
+ struct flexcan_regs __iomem *regs = priv->regs;
+ u64 reg_imask;
+
+ disable_irq(dev->irq);
+ priv->write(priv->reg_ctrl_default, ®s->ctrl);
+ reg_imask = priv->rx_mask | priv->tx_mask;
+ priv->write(upper_32_bits(reg_imask), ®s->imask2);
+ priv->write(lower_32_bits(reg_imask), ®s->imask1);
+ enable_irq(dev->irq);
+}
+
+static void flexcan_chip_interrupts_disable(const struct net_device *dev)
+{
+ const struct flexcan_priv *priv = netdev_priv(dev);
+ struct flexcan_regs __iomem *regs = priv->regs;
+
+ priv->write(0, ®s->imask2);
+ priv->write(0, ®s->imask1);
+ priv->write(priv->reg_ctrl_default & ~FLEXCAN_CTRL_ERR_ALL,
+ ®s->ctrl);
+}
+
+/* flexcan_chip_start
+ *
+ * this functions is entered with clocks enabled
+ *
+ */
+static int flexcan_chip_start(struct net_device *dev)
+{
+ struct flexcan_priv *priv = netdev_priv(dev);
+ struct flexcan_regs __iomem *regs = priv->regs;
+ u32 reg_mcr, reg_ctrl, reg_ctrl2, reg_mecr;
+ int err, i;
+ struct flexcan_mb __iomem *mb;
+
+ /* enable module */
+ err = flexcan_chip_enable(priv);
+ if (err)
+ return err;
+
+ /* soft reset */
+ err = flexcan_chip_softreset(priv);
+ if (err)
+ goto out_chip_disable;
+
+ if (priv->devtype_data.quirks & FLEXCAN_QUIRK_SUPPORT_ECC)
+ flexcan_ram_init(dev);
+
+ flexcan_set_bittiming(dev);
+
+ /* set freeze, halt */
+ err = flexcan_chip_freeze(priv);
+ if (err)
+ goto out_chip_disable;
+
+ /* MCR
+ *
+ * only supervisor access
+ * enable warning int
+ * enable individual RX masking
+ * choose format C
+ * set max mailbox number
+ */
+ reg_mcr = priv->read(®s->mcr);
+ reg_mcr &= ~FLEXCAN_MCR_MAXMB(0xff);
+ reg_mcr |= FLEXCAN_MCR_SUPV | FLEXCAN_MCR_WRN_EN | FLEXCAN_MCR_IRMQ |
+ FLEXCAN_MCR_IDAM_C | FLEXCAN_MCR_MAXMB(priv->tx_mb_idx);
+
+ /* MCR
+ *
+ * FIFO:
+ * - disable for mailbox mode
+ * - enable for FIFO mode
+ */
+ if (priv->devtype_data.quirks & FLEXCAN_QUIRK_USE_RX_MAILBOX)
+ reg_mcr &= ~FLEXCAN_MCR_FEN;
+ else
+ reg_mcr |= FLEXCAN_MCR_FEN;
+
+ /* MCR
+ *
+ * NOTE: In loopback mode, the CAN_MCR[SRXDIS] cannot be
+ * asserted because this will impede the self reception
+ * of a transmitted message. This is not documented in
+ * earlier versions of flexcan block guide.
+ *
+ * Self Reception:
+ * - enable Self Reception for loopback mode
+ * (by clearing "Self Reception Disable" bit)
+ * - disable for normal operation
+ */
+ if (priv->can.ctrlmode & CAN_CTRLMODE_LOOPBACK)
+ reg_mcr &= ~FLEXCAN_MCR_SRX_DIS;
+ else
+ reg_mcr |= FLEXCAN_MCR_SRX_DIS;
+
+ /* MCR - CAN-FD */
+ if (priv->can.ctrlmode & CAN_CTRLMODE_FD)
+ reg_mcr |= FLEXCAN_MCR_FDEN;
+ else
+ reg_mcr &= ~FLEXCAN_MCR_FDEN;
+
+ netdev_dbg(dev, "%s: writing mcr=0x%08x", __func__, reg_mcr);
+ priv->write(reg_mcr, ®s->mcr);
+
+ /* CTRL
+ *
+ * disable timer sync feature
+ *
+ * disable auto busoff recovery
+ * transmit lowest buffer first
+ *
+ * enable tx and rx warning interrupt
+ * enable bus off interrupt
+ * (== FLEXCAN_CTRL_ERR_STATE)
+ */
+ reg_ctrl = priv->read(®s->ctrl);
+ reg_ctrl &= ~FLEXCAN_CTRL_TSYN;
+ reg_ctrl |= FLEXCAN_CTRL_BOFF_REC | FLEXCAN_CTRL_LBUF |
+ FLEXCAN_CTRL_ERR_STATE;
+
+ /* enable the "error interrupt" (FLEXCAN_CTRL_ERR_MSK),
+ * on most Flexcan cores, too. Otherwise we don't get
+ * any error warning or passive interrupts.
+ */
+ if (priv->devtype_data.quirks & FLEXCAN_QUIRK_BROKEN_WERR_STATE ||
+ priv->can.ctrlmode & CAN_CTRLMODE_BERR_REPORTING)
+ reg_ctrl |= FLEXCAN_CTRL_ERR_MSK;
+ else
+ reg_ctrl &= ~FLEXCAN_CTRL_ERR_MSK;
+
+ /* save for later use */
+ priv->reg_ctrl_default = reg_ctrl;
+ /* leave interrupts disabled for now */
+ reg_ctrl &= ~FLEXCAN_CTRL_ERR_ALL;
+ netdev_dbg(dev, "%s: writing ctrl=0x%08x", __func__, reg_ctrl);
+ priv->write(reg_ctrl, ®s->ctrl);
+
+ if ((priv->devtype_data.quirks & FLEXCAN_QUIRK_ENABLE_EACEN_RRS)) {
+ reg_ctrl2 = priv->read(®s->ctrl2);
+ reg_ctrl2 |= FLEXCAN_CTRL2_EACEN | FLEXCAN_CTRL2_RRS;
+ priv->write(reg_ctrl2, ®s->ctrl2);
+ }
+
+ if (priv->can.ctrlmode_supported & CAN_CTRLMODE_FD) {
+ u32 reg_fdctrl;
+
+ reg_fdctrl = priv->read(®s->fdctrl);
+ reg_fdctrl &= ~(FIELD_PREP(FLEXCAN_FDCTRL_MBDSR1, 0x3) |
+ FIELD_PREP(FLEXCAN_FDCTRL_MBDSR0, 0x3));
+
+ if (priv->can.ctrlmode & CAN_CTRLMODE_FD) {
+ reg_fdctrl |=
+ FIELD_PREP(FLEXCAN_FDCTRL_MBDSR1,
+ FLEXCAN_FDCTRL_MBDSR_64) |
+ FIELD_PREP(FLEXCAN_FDCTRL_MBDSR0,
+ FLEXCAN_FDCTRL_MBDSR_64);
+ } else {
+ reg_fdctrl |=
+ FIELD_PREP(FLEXCAN_FDCTRL_MBDSR1,
+ FLEXCAN_FDCTRL_MBDSR_8) |
+ FIELD_PREP(FLEXCAN_FDCTRL_MBDSR0,
+ FLEXCAN_FDCTRL_MBDSR_8);
+ }
+
+ netdev_dbg(dev, "%s: writing fdctrl=0x%08x",
+ __func__, reg_fdctrl);
+ priv->write(reg_fdctrl, ®s->fdctrl);
+ }
+
+ if (priv->devtype_data.quirks & FLEXCAN_QUIRK_USE_RX_MAILBOX) {
+ for (i = priv->offload.mb_first; i <= priv->offload.mb_last; i++) {
+ mb = flexcan_get_mb(priv, i);
+ priv->write(FLEXCAN_MB_CODE_RX_EMPTY,
+ &mb->can_ctrl);
+ }
+ } else {
+ /* clear and invalidate unused mailboxes first */
+ for (i = FLEXCAN_TX_MB_RESERVED_RX_FIFO; i < priv->mb_count; i++) {
+ mb = flexcan_get_mb(priv, i);
+ priv->write(FLEXCAN_MB_CODE_RX_INACTIVE,
+ &mb->can_ctrl);
+ }
+ }
+
+ /* Errata ERR005829: mark first TX mailbox as INACTIVE */
+ priv->write(FLEXCAN_MB_CODE_TX_INACTIVE,
+ &priv->tx_mb_reserved->can_ctrl);
+
+ /* mark TX mailbox as INACTIVE */
+ priv->write(FLEXCAN_MB_CODE_TX_INACTIVE,
+ &priv->tx_mb->can_ctrl);
+
+ /* acceptance mask/acceptance code (accept everything) */
+ priv->write(0x0, ®s->rxgmask);
+ priv->write(0x0, ®s->rx14mask);
+ priv->write(0x0, ®s->rx15mask);
+
+ if (priv->devtype_data.quirks & FLEXCAN_QUIRK_DISABLE_RXFG)
+ priv->write(0x0, ®s->rxfgmask);
+
+ /* clear acceptance filters */
+ for (i = 0; i < priv->mb_count; i++)
+ priv->write(0, ®s->rximr[i]);
+
+ /* On Vybrid, disable non-correctable errors interrupt and
+ * freeze mode. It still can correct the correctable errors
+ * when HW supports ECC.
+ *
+ * This also works around errata e5295 which generates false
+ * positive memory errors and put the device in freeze mode.
+ */
+ if (priv->devtype_data.quirks & FLEXCAN_QUIRK_DISABLE_MECR) {
+ /* Follow the protocol as described in "Detection
+ * and Correction of Memory Errors" to write to
+ * MECR register (step 1 - 5)
+ *
+ * 1. By default, CTRL2[ECRWRE] = 0, MECR[ECRWRDIS] = 1
+ * 2. set CTRL2[ECRWRE]
+ */
+ reg_ctrl2 = priv->read(®s->ctrl2);
+ reg_ctrl2 |= FLEXCAN_CTRL2_ECRWRE;
+ priv->write(reg_ctrl2, ®s->ctrl2);
+
+ /* 3. clear MECR[ECRWRDIS] */
+ reg_mecr = priv->read(®s->mecr);
+ reg_mecr &= ~FLEXCAN_MECR_ECRWRDIS;
+ priv->write(reg_mecr, ®s->mecr);
+
+ /* 4. all writes to MECR must keep MECR[ECRWRDIS] cleared */
+ reg_mecr &= ~(FLEXCAN_MECR_NCEFAFRZ | FLEXCAN_MECR_HANCEI_MSK |
+ FLEXCAN_MECR_FANCEI_MSK);
+ priv->write(reg_mecr, ®s->mecr);
+
+ /* 5. after configuration done, lock MECR by either
+ * setting MECR[ECRWRDIS] or clearing CTRL2[ECRWRE]
+ */
+ reg_mecr |= FLEXCAN_MECR_ECRWRDIS;
+ priv->write(reg_mecr, ®s->mecr);
+
+ reg_ctrl2 &= ~FLEXCAN_CTRL2_ECRWRE;
+ priv->write(reg_ctrl2, ®s->ctrl2);
+ }
+
+ /* synchronize with the can bus */
+ err = flexcan_chip_unfreeze(priv);
+ if (err)
+ goto out_chip_disable;
+
+ priv->can.state = CAN_STATE_ERROR_ACTIVE;
+
+ /* print chip status */
+ netdev_dbg(dev, "%s: reading mcr=0x%08x ctrl=0x%08x\n", __func__,
+ priv->read(®s->mcr), priv->read(®s->ctrl));
+
+ return 0;
+
+ out_chip_disable:
+ flexcan_chip_disable(priv);
+ return err;
+}
+
+/* __flexcan_chip_stop
+ *
+ * this function is entered with clocks enabled
+ */
+static int __flexcan_chip_stop(struct net_device *dev, bool disable_on_error)
+{
+ struct flexcan_priv *priv = netdev_priv(dev);
+ int err;
+
+ /* freeze + disable module */
+ err = flexcan_chip_freeze(priv);
+ if (err && !disable_on_error)
+ return err;
+ err = flexcan_chip_disable(priv);
+ if (err && !disable_on_error)
+ goto out_chip_unfreeze;
+
+ priv->can.state = CAN_STATE_STOPPED;
+
+ return 0;
+
+ out_chip_unfreeze:
+ flexcan_chip_unfreeze(priv);
+
+ return err;
+}
+
+static inline int flexcan_chip_stop_disable_on_error(struct net_device *dev)
+{
+ return __flexcan_chip_stop(dev, true);
+}
+
+static inline int flexcan_chip_stop(struct net_device *dev)
+{
+ return __flexcan_chip_stop(dev, false);
+}
+
+static int flexcan_open(struct net_device *dev)
+{
+ struct flexcan_priv *priv = netdev_priv(dev);
+ int err;
+
+ if ((priv->can.ctrlmode & CAN_CTRLMODE_3_SAMPLES) &&
+ (priv->can.ctrlmode & CAN_CTRLMODE_FD)) {
+ netdev_err(dev, "Three Samples mode and CAN-FD mode can't be used together\n");
+ return -EINVAL;
+ }
+
+ err = pm_runtime_get_sync(priv->dev);
+ if (err < 0) {
+ pm_runtime_put_noidle(priv->dev);
+ return err;
+ }
+
+ err = open_candev(dev);
+ if (err)
+ goto out_runtime_put;
+
+ err = flexcan_transceiver_enable(priv);
+ if (err)
+ goto out_close;
+
+ err = flexcan_rx_offload_setup(dev);
+ if (err)
+ goto out_transceiver_disable;
+
+ err = flexcan_chip_start(dev);
+ if (err)
+ goto out_can_rx_offload_del;
+
+ can_rx_offload_enable(&priv->offload);
+
+ err = request_irq(dev->irq, flexcan_irq, IRQF_SHARED, dev->name, dev);
+ if (err)
+ goto out_can_rx_offload_disable;
+
+ if (priv->devtype_data.quirks & FLEXCAN_QUIRK_NR_IRQ_3) {
+ err = request_irq(priv->irq_boff,
+ flexcan_irq, IRQF_SHARED, dev->name, dev);
+ if (err)
+ goto out_free_irq;
+
+ err = request_irq(priv->irq_err,
+ flexcan_irq, IRQF_SHARED, dev->name, dev);
+ if (err)
+ goto out_free_irq_boff;
+ }
+
+ flexcan_chip_interrupts_enable(dev);
+
+ can_led_event(dev, CAN_LED_EVENT_OPEN);
+
+ netif_start_queue(dev);
+
+ return 0;
+
+ out_free_irq_boff:
+ free_irq(priv->irq_boff, dev);
+ out_free_irq:
+ free_irq(dev->irq, dev);
+ out_can_rx_offload_disable:
+ can_rx_offload_disable(&priv->offload);
+ flexcan_chip_stop(dev);
+ out_can_rx_offload_del:
+ can_rx_offload_del(&priv->offload);
+ out_transceiver_disable:
+ flexcan_transceiver_disable(priv);
+ out_close:
+ close_candev(dev);
+ out_runtime_put:
+ pm_runtime_put(priv->dev);
+
+ return err;
+}
+
+static int flexcan_close(struct net_device *dev)
+{
+ struct flexcan_priv *priv = netdev_priv(dev);
+
+ netif_stop_queue(dev);
+ flexcan_chip_interrupts_disable(dev);
+
+ if (priv->devtype_data.quirks & FLEXCAN_QUIRK_NR_IRQ_3) {
+ free_irq(priv->irq_err, dev);
+ free_irq(priv->irq_boff, dev);
+ }
+
+ free_irq(dev->irq, dev);
+ can_rx_offload_disable(&priv->offload);
+ flexcan_chip_stop_disable_on_error(dev);
+
+ can_rx_offload_del(&priv->offload);
+ flexcan_transceiver_disable(priv);
+ close_candev(dev);
+
+ pm_runtime_put(priv->dev);
+
+ can_led_event(dev, CAN_LED_EVENT_STOP);
+
+ return 0;
+}
+
+static int flexcan_set_mode(struct net_device *dev, enum can_mode mode)
+{
+ int err;
+
+ switch (mode) {
+ case CAN_MODE_START:
+ err = flexcan_chip_start(dev);
+ if (err)
+ return err;
+
+ flexcan_chip_interrupts_enable(dev);
+
+ netif_wake_queue(dev);
+ break;
+
+ default:
+ return -EOPNOTSUPP;
+ }
+
+ return 0;
+}
+
+static const struct net_device_ops flexcan_netdev_ops = {
+ .ndo_open = flexcan_open,
+ .ndo_stop = flexcan_close,
+ .ndo_start_xmit = flexcan_start_xmit,
+ .ndo_change_mtu = can_change_mtu,
+};
+
+static int register_flexcandev(struct net_device *dev)
+{
+ struct flexcan_priv *priv = netdev_priv(dev);
+ struct flexcan_regs __iomem *regs = priv->regs;
+ u32 reg, err;
+
+ err = flexcan_clks_enable(priv);
+ if (err)
+ return err;
+
+ /* select "bus clock", chip must be disabled */
+ err = flexcan_chip_disable(priv);
+ if (err)
+ goto out_clks_disable;
+
+ reg = priv->read(®s->ctrl);
+ if (priv->clk_src)
+ reg |= FLEXCAN_CTRL_CLK_SRC;
+ else
+ reg &= ~FLEXCAN_CTRL_CLK_SRC;
+ priv->write(reg, ®s->ctrl);
+
+ err = flexcan_chip_enable(priv);
+ if (err)
+ goto out_chip_disable;
+
+ /* set freeze, halt */
+ err = flexcan_chip_freeze(priv);
+ if (err)
+ goto out_chip_disable;
+
+ /* activate FIFO, restrict register access */
+ reg = priv->read(®s->mcr);
+ reg |= FLEXCAN_MCR_FEN | FLEXCAN_MCR_SUPV;
+ priv->write(reg, ®s->mcr);
+
+ /* Currently we only support newer versions of this core
+ * featuring a RX hardware FIFO (although this driver doesn't
+ * make use of it on some cores). Older cores, found on some
+ * Coldfire derivates are not tested.
+ */
+ reg = priv->read(®s->mcr);
+ if (!(reg & FLEXCAN_MCR_FEN)) {
+ netdev_err(dev, "Could not enable RX FIFO, unsupported core\n");
+ err = -ENODEV;
+ goto out_chip_disable;
+ }
+
+ err = register_candev(dev);
+ if (err)
+ goto out_chip_disable;
+
+ /* Disable core and let pm_runtime_put() disable the clocks.
+ * If CONFIG_PM is not enabled, the clocks will stay powered.
+ */
+ flexcan_chip_disable(priv);
+ pm_runtime_put(priv->dev);
+
+ return 0;
+
+ out_chip_disable:
+ flexcan_chip_disable(priv);
+ out_clks_disable:
+ flexcan_clks_disable(priv);
+ return err;
+}
+
+static void unregister_flexcandev(struct net_device *dev)
+{
+ unregister_candev(dev);
+}
+
+static int flexcan_setup_stop_mode_gpr(struct platform_device *pdev)
+{
+ struct net_device *dev = platform_get_drvdata(pdev);
+ struct device_node *np = pdev->dev.of_node;
+ struct device_node *gpr_np;
+ struct flexcan_priv *priv;
+ phandle phandle;
+ u32 out_val[3];
+ int ret;
+
+ if (!np)
+ return -EINVAL;
+
+ /* stop mode property format is:
+ * <&gpr req_gpr req_bit>.
+ */
+ ret = of_property_read_u32_array(np, "fsl,stop-mode", out_val,
+ ARRAY_SIZE(out_val));
+ if (ret) {
+ dev_dbg(&pdev->dev, "no stop-mode property\n");
+ return ret;
+ }
+ phandle = *out_val;
+
+ gpr_np = of_find_node_by_phandle(phandle);
+ if (!gpr_np) {
+ dev_dbg(&pdev->dev, "could not find gpr node by phandle\n");
+ return -ENODEV;
+ }
+
+ priv = netdev_priv(dev);
+ priv->stm.gpr = syscon_node_to_regmap(gpr_np);
+ if (IS_ERR(priv->stm.gpr)) {
+ dev_dbg(&pdev->dev, "could not find gpr regmap\n");
+ ret = PTR_ERR(priv->stm.gpr);
+ goto out_put_node;
+ }
+
+ priv->stm.req_gpr = out_val[1];
+ priv->stm.req_bit = out_val[2];
+
+ dev_dbg(&pdev->dev,
+ "gpr %s req_gpr=0x02%x req_bit=%u\n",
+ gpr_np->full_name, priv->stm.req_gpr, priv->stm.req_bit);
+
+ return 0;
+
+out_put_node:
+ of_node_put(gpr_np);
+ return ret;
+}
+
+static int flexcan_setup_stop_mode_scfw(struct platform_device *pdev)
+{
+ struct net_device *dev = platform_get_drvdata(pdev);
+ struct flexcan_priv *priv;
+ u8 scu_idx;
+ int ret;
+
+ ret = of_property_read_u8(pdev->dev.of_node, "fsl,scu-index", &scu_idx);
+ if (ret < 0) {
+ dev_dbg(&pdev->dev, "failed to get scu index\n");
+ return ret;
+ }
+
+ priv = netdev_priv(dev);
+ priv->scu_idx = scu_idx;
+
+ /* this function could be deferred probe, return -EPROBE_DEFER */
+ return imx_scu_get_handle(&priv->sc_ipc_handle);
+}
+
+/* flexcan_setup_stop_mode - Setup stop mode for wakeup
+ *
+ * Return: = 0 setup stop mode successfully or doesn't support this feature
+ * < 0 fail to setup stop mode (could be deferred probe)
+ */
+static int flexcan_setup_stop_mode(struct platform_device *pdev)
+{
+ struct net_device *dev = platform_get_drvdata(pdev);
+ struct flexcan_priv *priv;
+ int ret;
+
+ priv = netdev_priv(dev);
+
+ if (priv->devtype_data.quirks & FLEXCAN_QUIRK_SETUP_STOP_MODE_SCFW)
+ ret = flexcan_setup_stop_mode_scfw(pdev);
+ else if (priv->devtype_data.quirks & FLEXCAN_QUIRK_SETUP_STOP_MODE_GPR)
+ ret = flexcan_setup_stop_mode_gpr(pdev);
+ else
+ /* return 0 directly if doesn't support stop mode feature */
+ return 0;
+
+ if (ret)
+ return ret;
+
+ device_set_wakeup_capable(&pdev->dev, true);
+
+ if (of_property_read_bool(pdev->dev.of_node, "wakeup-source"))
+ device_set_wakeup_enable(&pdev->dev, true);
+
+ return 0;
+}
+
+static const struct of_device_id flexcan_of_match[] = {
+ { .compatible = "fsl,imx8qm-flexcan", .data = &fsl_imx8qm_devtype_data, },
+ { .compatible = "fsl,imx8mp-flexcan", .data = &fsl_imx8mp_devtype_data, },
+ { .compatible = "fsl,imx6q-flexcan", .data = &fsl_imx6q_devtype_data, },
+ { .compatible = "fsl,imx28-flexcan", .data = &fsl_imx28_devtype_data, },
+ { .compatible = "fsl,imx53-flexcan", .data = &fsl_imx25_devtype_data, },
+ { .compatible = "fsl,imx35-flexcan", .data = &fsl_imx25_devtype_data, },
+ { .compatible = "fsl,imx25-flexcan", .data = &fsl_imx25_devtype_data, },
+ { .compatible = "fsl,p1010-flexcan", .data = &fsl_p1010_devtype_data, },
+ { .compatible = "fsl,vf610-flexcan", .data = &fsl_vf610_devtype_data, },
+ { .compatible = "fsl,ls1021ar2-flexcan", .data = &fsl_ls1021a_r2_devtype_data, },
+ { .compatible = "fsl,lx2160ar1-flexcan", .data = &fsl_lx2160a_r1_devtype_data, },
+ { /* sentinel */ },
+};
+MODULE_DEVICE_TABLE(of, flexcan_of_match);
+
+static const struct platform_device_id flexcan_id_table[] = {
+ {
+ .name = "flexcan-mcf5441x",
+ .driver_data = (kernel_ulong_t)&fsl_mcf5441x_devtype_data,
+ }, {
+ /* sentinel */
+ },
+};
+MODULE_DEVICE_TABLE(platform, flexcan_id_table);
+
+static int flexcan_probe(struct platform_device *pdev)
+{
+ const struct of_device_id *of_id;
+ const struct flexcan_devtype_data *devtype_data;
+ struct net_device *dev;
+ struct flexcan_priv *priv;
+ struct regulator *reg_xceiver;
+ struct clk *clk_ipg = NULL, *clk_per = NULL;
+ struct flexcan_regs __iomem *regs;
+ struct flexcan_platform_data *pdata;
+ int err, irq;
+ u8 clk_src = 1;
+ u32 clock_freq = 0;
+
+ reg_xceiver = devm_regulator_get_optional(&pdev->dev, "xceiver");
+ if (PTR_ERR(reg_xceiver) == -EPROBE_DEFER)
+ return -EPROBE_DEFER;
+ else if (PTR_ERR(reg_xceiver) == -ENODEV)
+ reg_xceiver = NULL;
+ else if (IS_ERR(reg_xceiver))
+ return PTR_ERR(reg_xceiver);
+
+ if (pdev->dev.of_node) {
+ of_property_read_u32(pdev->dev.of_node,
+ "clock-frequency", &clock_freq);
+ of_property_read_u8(pdev->dev.of_node,
+ "fsl,clk-source", &clk_src);
+ } else {
+ pdata = dev_get_platdata(&pdev->dev);
+ if (pdata) {
+ clock_freq = pdata->clock_frequency;
+ clk_src = pdata->clk_src;
+ }
+ }
+
+ if (!clock_freq) {
+ clk_ipg = devm_clk_get(&pdev->dev, "ipg");
+ if (IS_ERR(clk_ipg)) {
+ dev_err(&pdev->dev, "no ipg clock defined\n");
+ return PTR_ERR(clk_ipg);
+ }
+
+ clk_per = devm_clk_get(&pdev->dev, "per");
+ if (IS_ERR(clk_per)) {
+ dev_err(&pdev->dev, "no per clock defined\n");
+ return PTR_ERR(clk_per);
+ }
+ clock_freq = clk_get_rate(clk_per);
+ }
+
+ irq = platform_get_irq(pdev, 0);
+ if (irq <= 0)
+ return -ENODEV;
+
+ regs = devm_platform_ioremap_resource(pdev, 0);
+ if (IS_ERR(regs))
+ return PTR_ERR(regs);
+
+ of_id = of_match_device(flexcan_of_match, &pdev->dev);
+ if (of_id)
+ devtype_data = of_id->data;
+ else if (platform_get_device_id(pdev)->driver_data)
+ devtype_data = (struct flexcan_devtype_data *)
+ platform_get_device_id(pdev)->driver_data;
+ else
+ return -ENODEV;
+
+ if ((devtype_data->quirks & FLEXCAN_QUIRK_SUPPORT_FD) &&
+ !((devtype_data->quirks &
+ (FLEXCAN_QUIRK_USE_RX_MAILBOX |
+ FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX |
+ FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX_RTR |
+ FLEXCAN_QUIRK_SUPPPORT_RX_FIFO)) ==
+ (FLEXCAN_QUIRK_USE_RX_MAILBOX |
+ FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX |
+ FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX_RTR))) {
+ dev_err(&pdev->dev, "CAN-FD mode doesn't work in RX-FIFO mode!\n");
+ return -EINVAL;
+ }
+
+ if ((devtype_data->quirks &
+ (FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX |
+ FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX_RTR)) ==
+ FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX_RTR) {
+ dev_err(&pdev->dev,
+ "Quirks (0x%08x) inconsistent: RX_MAILBOX_RX supported but not RX_MAILBOX\n",
+ devtype_data->quirks);
+ return -EINVAL;
+ }
+
+ dev = alloc_candev(sizeof(struct flexcan_priv), 1);
+ if (!dev)
+ return -ENOMEM;
+
+ platform_set_drvdata(pdev, dev);
+ SET_NETDEV_DEV(dev, &pdev->dev);
+
+ dev->netdev_ops = &flexcan_netdev_ops;
+ flexcan_set_ethtool_ops(dev);
+ dev->irq = irq;
+ dev->flags |= IFF_ECHO;
+
+ priv = netdev_priv(dev);
+ priv->devtype_data = *devtype_data;
+
+ if (of_property_read_bool(pdev->dev.of_node, "big-endian") ||
+ priv->devtype_data.quirks & FLEXCAN_QUIRK_DEFAULT_BIG_ENDIAN) {
+ priv->read = flexcan_read_be;
+ priv->write = flexcan_write_be;
+ } else {
+ priv->read = flexcan_read_le;
+ priv->write = flexcan_write_le;
+ }
+
+ priv->dev = &pdev->dev;
+ priv->can.clock.freq = clock_freq;
+ priv->can.do_set_mode = flexcan_set_mode;
+ priv->can.do_get_berr_counter = flexcan_get_berr_counter;
+ priv->can.ctrlmode_supported = CAN_CTRLMODE_LOOPBACK |
+ CAN_CTRLMODE_LISTENONLY | CAN_CTRLMODE_3_SAMPLES |
+ CAN_CTRLMODE_BERR_REPORTING;
+ priv->regs = regs;
+ priv->clk_ipg = clk_ipg;
+ priv->clk_per = clk_per;
+ priv->clk_src = clk_src;
+ priv->reg_xceiver = reg_xceiver;
+
+ if (priv->devtype_data.quirks & FLEXCAN_QUIRK_NR_IRQ_3) {
+ priv->irq_boff = platform_get_irq(pdev, 1);
+ if (priv->irq_boff <= 0) {
+ err = -ENODEV;
+ goto failed_platform_get_irq;
+ }
+ priv->irq_err = platform_get_irq(pdev, 2);
+ if (priv->irq_err <= 0) {
+ err = -ENODEV;
+ goto failed_platform_get_irq;
+ }
+ }
+
+ if (priv->devtype_data.quirks & FLEXCAN_QUIRK_SUPPORT_FD) {
+ priv->can.ctrlmode_supported |= CAN_CTRLMODE_FD |
+ CAN_CTRLMODE_FD_NON_ISO;
+ priv->can.bittiming_const = &flexcan_fd_bittiming_const;
+ priv->can.data_bittiming_const =
+ &flexcan_fd_data_bittiming_const;
+ } else {
+ priv->can.bittiming_const = &flexcan_bittiming_const;
+ }
+
+ pm_runtime_get_noresume(&pdev->dev);
+ pm_runtime_set_active(&pdev->dev);
+ pm_runtime_enable(&pdev->dev);
+
+ err = register_flexcandev(dev);
+ if (err) {
+ dev_err(&pdev->dev, "registering netdev failed\n");
+ goto failed_register;
+ }
+
+ err = flexcan_setup_stop_mode(pdev);
+ if (err < 0) {
+ if (err != -EPROBE_DEFER)
+ dev_err(&pdev->dev, "setup stop mode failed\n");
+ goto failed_setup_stop_mode;
+ }
+
+ of_can_transceiver(dev);
+ devm_can_led_init(dev);
+
+ return 0;
+
+ failed_setup_stop_mode:
+ unregister_flexcandev(dev);
+ failed_register:
+ pm_runtime_put_noidle(&pdev->dev);
+ pm_runtime_disable(&pdev->dev);
+ failed_platform_get_irq:
+ free_candev(dev);
+ return err;
+}
+
+static int flexcan_remove(struct platform_device *pdev)
+{
+ struct net_device *dev = platform_get_drvdata(pdev);
+
+ device_set_wakeup_enable(&pdev->dev, false);
+ device_set_wakeup_capable(&pdev->dev, false);
+ unregister_flexcandev(dev);
+ pm_runtime_disable(&pdev->dev);
+ free_candev(dev);
+
+ return 0;
+}
+
+static int __maybe_unused flexcan_suspend(struct device *device)
+{
+ struct net_device *dev = dev_get_drvdata(device);
+ struct flexcan_priv *priv = netdev_priv(dev);
+ int err;
+
+ if (netif_running(dev)) {
+ /* if wakeup is enabled, enter stop mode
+ * else enter disabled mode.
+ */
+ if (device_may_wakeup(device)) {
+ enable_irq_wake(dev->irq);
+ err = flexcan_enter_stop_mode(priv);
+ if (err)
+ return err;
+ } else {
+ err = flexcan_chip_stop(dev);
+ if (err)
+ return err;
+
+ flexcan_chip_interrupts_disable(dev);
+
+ err = pinctrl_pm_select_sleep_state(device);
+ if (err)
+ return err;
+ }
+ netif_stop_queue(dev);
+ netif_device_detach(dev);
+ }
+ priv->can.state = CAN_STATE_SLEEPING;
+
+ return 0;
+}
+
+static int __maybe_unused flexcan_resume(struct device *device)
+{
+ struct net_device *dev = dev_get_drvdata(device);
+ struct flexcan_priv *priv = netdev_priv(dev);
+ int err;
+
+ priv->can.state = CAN_STATE_ERROR_ACTIVE;
+ if (netif_running(dev)) {
+ netif_device_attach(dev);
+ netif_start_queue(dev);
+ if (device_may_wakeup(device)) {
+ disable_irq_wake(dev->irq);
+ err = flexcan_exit_stop_mode(priv);
+ if (err)
+ return err;
+ } else {
+ err = pinctrl_pm_select_default_state(device);
+ if (err)
+ return err;
+
+ err = flexcan_chip_start(dev);
+ if (err)
+ return err;
+
+ flexcan_chip_interrupts_enable(dev);
+ }
+ }
+
+ return 0;
+}
+
+static int __maybe_unused flexcan_runtime_suspend(struct device *device)
+{
+ struct net_device *dev = dev_get_drvdata(device);
+ struct flexcan_priv *priv = netdev_priv(dev);
+
+ flexcan_clks_disable(priv);
+
+ return 0;
+}
+
+static int __maybe_unused flexcan_runtime_resume(struct device *device)
+{
+ struct net_device *dev = dev_get_drvdata(device);
+ struct flexcan_priv *priv = netdev_priv(dev);
+
+ return flexcan_clks_enable(priv);
+}
+
+static int __maybe_unused flexcan_noirq_suspend(struct device *device)
+{
+ struct net_device *dev = dev_get_drvdata(device);
+ struct flexcan_priv *priv = netdev_priv(dev);
+
+ if (netif_running(dev)) {
+ int err;
+
+ if (device_may_wakeup(device))
+ flexcan_enable_wakeup_irq(priv, true);
+
+ err = pm_runtime_force_suspend(device);
+ if (err)
+ return err;
+ }
+
+ return 0;
+}
+
+static int __maybe_unused flexcan_noirq_resume(struct device *device)
+{
+ struct net_device *dev = dev_get_drvdata(device);
+ struct flexcan_priv *priv = netdev_priv(dev);
+
+ if (netif_running(dev)) {
+ int err;
+
+ err = pm_runtime_force_resume(device);
+ if (err)
+ return err;
+
+ if (device_may_wakeup(device))
+ flexcan_enable_wakeup_irq(priv, false);
+ }
+
+ return 0;
+}
+
+static const struct dev_pm_ops flexcan_pm_ops = {
+ SET_SYSTEM_SLEEP_PM_OPS(flexcan_suspend, flexcan_resume)
+ SET_RUNTIME_PM_OPS(flexcan_runtime_suspend, flexcan_runtime_resume, NULL)
+ SET_NOIRQ_SYSTEM_SLEEP_PM_OPS(flexcan_noirq_suspend, flexcan_noirq_resume)
+};
+
+static struct platform_driver flexcan_driver = {
+ .driver = {
+ .name = DRV_NAME,
+ .pm = &flexcan_pm_ops,
+ .of_match_table = flexcan_of_match,
+ },
+ .probe = flexcan_probe,
+ .remove = flexcan_remove,
+ .id_table = flexcan_id_table,
+};
+
+module_platform_driver(flexcan_driver);
+
+MODULE_AUTHOR("Sascha Hauer <kernel@pengutronix.de>, "
+ "Marc Kleine-Budde <kernel@pengutronix.de>");
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("CAN port driver for flexcan based chip");
diff --git a/drivers/net/can/flexcan/flexcan-ethtool.c b/drivers/net/can/flexcan/flexcan-ethtool.c
new file mode 100644
index 0000000..3ae5355
--- /dev/null
+++ b/drivers/net/can/flexcan/flexcan-ethtool.c
@@ -0,0 +1,114 @@
+// SPDX-License-Identifier: GPL-2.0+
+/* Copyright (c) 2022 Amarula Solutions, Dario Binacchi <dario.binacchi@amarulasolutions.com>
+ * Copyright (c) 2022 Pengutronix, Marc Kleine-Budde <kernel@pengutronix.de>
+ *
+ */
+
+#include <linux/can/dev.h>
+#include <linux/ethtool.h>
+#include <linux/kernel.h>
+#include <linux/netdevice.h>
+#include <linux/platform_device.h>
+
+#include "flexcan.h"
+
+static const char flexcan_priv_flags_strings[][ETH_GSTRING_LEN] = {
+#define FLEXCAN_PRIV_FLAGS_RX_RTR BIT(0)
+ "rx-rtr",
+};
+
+static void
+flexcan_get_ringparam(struct net_device *ndev, struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *ext_ack)
+{
+ const struct flexcan_priv *priv = netdev_priv(ndev);
+
+ ring->rx_max_pending = priv->mb_count;
+ ring->tx_max_pending = priv->mb_count;
+
+ if (priv->devtype_data.quirks & FLEXCAN_QUIRK_USE_RX_MAILBOX)
+ ring->rx_pending = priv->offload.mb_last -
+ priv->offload.mb_first + 1;
+ else
+ ring->rx_pending = 6; /* RX-FIFO depth is fixed */
+
+ /* the drive currently supports only on TX buffer */
+ ring->tx_pending = 1;
+}
+
+static void
+flexcan_get_strings(struct net_device *ndev, u32 stringset, u8 *data)
+{
+ switch (stringset) {
+ case ETH_SS_PRIV_FLAGS:
+ memcpy(data, flexcan_priv_flags_strings,
+ sizeof(flexcan_priv_flags_strings));
+ }
+}
+
+static u32 flexcan_get_priv_flags(struct net_device *ndev)
+{
+ const struct flexcan_priv *priv = netdev_priv(ndev);
+ u32 priv_flags = 0;
+
+ if (flexcan_active_rx_rtr(priv))
+ priv_flags |= FLEXCAN_PRIV_FLAGS_RX_RTR;
+
+ return priv_flags;
+}
+
+static int flexcan_set_priv_flags(struct net_device *ndev, u32 priv_flags)
+{
+ struct flexcan_priv *priv = netdev_priv(ndev);
+ u32 quirks = priv->devtype_data.quirks;
+
+ if (priv_flags & FLEXCAN_PRIV_FLAGS_RX_RTR) {
+ if (flexcan_supports_rx_mailbox_rtr(priv))
+ quirks |= FLEXCAN_QUIRK_USE_RX_MAILBOX;
+ else if (flexcan_supports_rx_fifo(priv))
+ quirks &= ~FLEXCAN_QUIRK_USE_RX_MAILBOX;
+ else
+ quirks |= FLEXCAN_QUIRK_USE_RX_MAILBOX;
+ } else {
+ if (flexcan_supports_rx_mailbox(priv))
+ quirks |= FLEXCAN_QUIRK_USE_RX_MAILBOX;
+ else
+ quirks &= ~FLEXCAN_QUIRK_USE_RX_MAILBOX;
+ }
+
+ if (quirks != priv->devtype_data.quirks && netif_running(ndev))
+ return -EBUSY;
+
+ priv->devtype_data.quirks = quirks;
+
+ if (!(priv_flags & FLEXCAN_PRIV_FLAGS_RX_RTR) &&
+ !flexcan_active_rx_rtr(priv))
+ netdev_info(ndev,
+ "Activating RX mailbox mode, cannot receive RTR frames.\n");
+
+ return 0;
+}
+
+static int flexcan_get_sset_count(struct net_device *netdev, int sset)
+{
+ switch (sset) {
+ case ETH_SS_PRIV_FLAGS:
+ return ARRAY_SIZE(flexcan_priv_flags_strings);
+ default:
+ return -EOPNOTSUPP;
+ }
+}
+
+static const struct ethtool_ops flexcan_ethtool_ops = {
+ .get_ringparam = flexcan_get_ringparam,
+ .get_strings = flexcan_get_strings,
+ .get_priv_flags = flexcan_get_priv_flags,
+ .set_priv_flags = flexcan_set_priv_flags,
+ .get_sset_count = flexcan_get_sset_count,
+};
+
+void flexcan_set_ethtool_ops(struct net_device *netdev)
+{
+ netdev->ethtool_ops = &flexcan_ethtool_ops;
+}
diff --git a/drivers/net/can/flexcan/flexcan.h b/drivers/net/can/flexcan/flexcan.h
new file mode 100644
index 0000000..fccdff8
--- /dev/null
+++ b/drivers/net/can/flexcan/flexcan.h
@@ -0,0 +1,163 @@
+/* SPDX-License-Identifier: GPL-2.0
+ * flexcan.c - FLEXCAN CAN controller driver
+ *
+ * Copyright (c) 2005-2006 Varma Electronics Oy
+ * Copyright (c) 2009 Sascha Hauer, Pengutronix
+ * Copyright (c) 2010-2017 Pengutronix, Marc Kleine-Budde <kernel@pengutronix.de>
+ * Copyright (c) 2014 David Jander, Protonic Holland
+ * Copyright (C) 2022 Amarula Solutions, Dario Binacchi <dario.binacchi@amarulasolutions.com>
+ *
+ * Based on code originally by Andrey Volkov <avolkov@varma-el.com>
+ *
+ */
+
+#ifndef _FLEXCAN_H
+#define _FLEXCAN_H
+
+#include <linux/can/rx-offload.h>
+
+/* FLEXCAN hardware feature flags
+ *
+ * Below is some version info we got:
+ * SOC Version IP-Version Glitch- [TR]WRN_INT IRQ Err Memory err RTR rece- FD Mode MB
+ * Filter? connected? Passive detection ption in MB Supported?
+ * MCF5441X FlexCAN2 ? no yes no no yes no 16
+ * MX25 FlexCAN2 03.00.00.00 no no no no no no 64
+ * MX28 FlexCAN2 03.00.04.00 yes yes no no no no 64
+ * MX35 FlexCAN2 03.00.00.00 no no no no no no 64
+ * MX53 FlexCAN2 03.00.00.00 yes no no no no no 64
+ * MX6s FlexCAN3 10.00.12.00 yes yes no no yes no 64
+ * MX8QM FlexCAN3 03.00.23.00 yes yes no no yes yes 64
+ * MX8MP FlexCAN3 03.00.17.01 yes yes no yes yes yes 64
+ * VF610 FlexCAN3 ? no yes no yes yes? no 64
+ * LS1021A FlexCAN2 03.00.04.00 no yes no no yes no 64
+ * LX2160A FlexCAN3 03.00.23.00 no yes no yes yes yes 64
+ *
+ * Some SOCs do not have the RX_WARN & TX_WARN interrupt line connected.
+ */
+
+/* [TR]WRN_INT not connected */
+#define FLEXCAN_QUIRK_BROKEN_WERR_STATE BIT(1)
+ /* Disable RX FIFO Global mask */
+#define FLEXCAN_QUIRK_DISABLE_RXFG BIT(2)
+/* Enable EACEN and RRS bit in ctrl2 */
+#define FLEXCAN_QUIRK_ENABLE_EACEN_RRS BIT(3)
+/* Disable non-correctable errors interrupt and freeze mode */
+#define FLEXCAN_QUIRK_DISABLE_MECR BIT(4)
+/* Use mailboxes (not FIFO) for RX path */
+#define FLEXCAN_QUIRK_USE_RX_MAILBOX BIT(5)
+/* No interrupt for error passive */
+#define FLEXCAN_QUIRK_BROKEN_PERR_STATE BIT(6)
+/* default to BE register access */
+#define FLEXCAN_QUIRK_DEFAULT_BIG_ENDIAN BIT(7)
+/* Setup stop mode with GPR to support wakeup */
+#define FLEXCAN_QUIRK_SETUP_STOP_MODE_GPR BIT(8)
+/* Support CAN-FD mode */
+#define FLEXCAN_QUIRK_SUPPORT_FD BIT(9)
+/* support memory detection and correction */
+#define FLEXCAN_QUIRK_SUPPORT_ECC BIT(10)
+/* Setup stop mode with SCU firmware to support wakeup */
+#define FLEXCAN_QUIRK_SETUP_STOP_MODE_SCFW BIT(11)
+/* Setup 3 separate interrupts, main, boff and err */
+#define FLEXCAN_QUIRK_NR_IRQ_3 BIT(12)
+/* Setup 16 mailboxes */
+#define FLEXCAN_QUIRK_NR_MB_16 BIT(13)
+/* Device supports RX via mailboxes */
+#define FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX BIT(14)
+/* Device supports RTR reception via mailboxes */
+#define FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX_RTR BIT(15)
+/* Device supports RX via FIFO */
+#define FLEXCAN_QUIRK_SUPPPORT_RX_FIFO BIT(16)
+
+struct flexcan_devtype_data {
+ u32 quirks; /* quirks needed for different IP cores */
+};
+
+struct flexcan_stop_mode {
+ struct regmap *gpr;
+ u8 req_gpr;
+ u8 req_bit;
+};
+
+struct flexcan_priv {
+ struct can_priv can;
+ struct can_rx_offload offload;
+ struct device *dev;
+
+ struct flexcan_regs __iomem *regs;
+ struct flexcan_mb __iomem *tx_mb;
+ struct flexcan_mb __iomem *tx_mb_reserved;
+ u8 tx_mb_idx;
+ u8 mb_count;
+ u8 mb_size;
+ u8 clk_src; /* clock source of CAN Protocol Engine */
+ u8 scu_idx;
+
+ u64 rx_mask;
+ u64 tx_mask;
+ u32 reg_ctrl_default;
+
+ struct clk *clk_ipg;
+ struct clk *clk_per;
+ struct flexcan_devtype_data devtype_data;
+ struct regulator *reg_xceiver;
+ struct flexcan_stop_mode stm;
+
+ int irq_boff;
+ int irq_err;
+
+ /* IPC handle when setup stop mode by System Controller firmware(scfw) */
+ struct imx_sc_ipc *sc_ipc_handle;
+
+ /* Read and Write APIs */
+ u32 (*read)(void __iomem *addr);
+ void (*write)(u32 val, void __iomem *addr);
+};
+
+void flexcan_set_ethtool_ops(struct net_device *dev);
+
+static inline bool
+flexcan_supports_rx_mailbox(const struct flexcan_priv *priv)
+{
+ const u32 quirks = priv->devtype_data.quirks;
+
+ return quirks & FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX;
+}
+
+static inline bool
+flexcan_supports_rx_mailbox_rtr(const struct flexcan_priv *priv)
+{
+ const u32 quirks = priv->devtype_data.quirks;
+
+ return (quirks & (FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX |
+ FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX_RTR)) ==
+ (FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX |
+ FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX_RTR);
+}
+
+static inline bool
+flexcan_supports_rx_fifo(const struct flexcan_priv *priv)
+{
+ const u32 quirks = priv->devtype_data.quirks;
+
+ return quirks & FLEXCAN_QUIRK_SUPPPORT_RX_FIFO;
+}
+
+static inline bool
+flexcan_active_rx_rtr(const struct flexcan_priv *priv)
+{
+ const u32 quirks = priv->devtype_data.quirks;
+
+ if (quirks & FLEXCAN_QUIRK_USE_RX_MAILBOX) {
+ if (quirks & FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX_RTR)
+ return true;
+ } else {
+ /* RX-FIFO is always RTR capable */
+ return true;
+ }
+
+ return false;
+}
+
+
+#endif /* _FLEXCAN_H */
diff --git a/drivers/net/can/grcan.c b/drivers/net/can/grcan.c
index 78e2794..d0c5a7a 100644
--- a/drivers/net/can/grcan.c
+++ b/drivers/net/can/grcan.c
@@ -255,7 +255,6 @@ struct grcan_priv {
struct grcan_dma dma;
struct sk_buff **echo_skb; /* We allocate this on our own */
- u8 *txdlc; /* Length of queued frames */
/* The echo skb pointer, pointing into echo_skb and indicating which
* frames can be echoed back. See the "Notes on the tx cyclic buffer
@@ -515,9 +514,7 @@ static int catch_up_echo_skb(struct net_device *dev, int budget, bool echo)
if (echo) {
/* Normal echo of messages */
stats->tx_packets++;
- stats->tx_bytes += priv->txdlc[i];
- priv->txdlc[i] = 0;
- can_get_echo_skb(dev, i, NULL);
+ stats->tx_bytes += can_get_echo_skb(dev, i, NULL);
} else {
/* For cleanup of untransmitted messages */
can_free_echo_skb(dev, i, NULL);
@@ -1062,16 +1059,10 @@ static int grcan_open(struct net_device *dev)
priv->can.echo_skb_max = dma->tx.size;
priv->can.echo_skb = priv->echo_skb;
- priv->txdlc = kcalloc(dma->tx.size, sizeof(*priv->txdlc), GFP_KERNEL);
- if (!priv->txdlc) {
- err = -ENOMEM;
- goto exit_free_echo_skb;
- }
-
/* Get can device up */
err = open_candev(dev);
if (err)
- goto exit_free_txdlc;
+ goto exit_free_echo_skb;
err = request_irq(dev->irq, grcan_interrupt, IRQF_SHARED,
dev->name, dev);
@@ -1093,8 +1084,6 @@ static int grcan_open(struct net_device *dev)
exit_close_candev:
close_candev(dev);
-exit_free_txdlc:
- kfree(priv->txdlc);
exit_free_echo_skb:
kfree(priv->echo_skb);
exit_free_dma_buffers:
@@ -1129,7 +1118,6 @@ static int grcan_close(struct net_device *dev)
priv->can.echo_skb_max = 0;
priv->can.echo_skb = NULL;
kfree(priv->echo_skb);
- kfree(priv->txdlc);
return 0;
}
@@ -1211,11 +1199,11 @@ static int grcan_receive(struct net_device *dev, int budget)
shift = GRCAN_MSG_DATA_SHIFT(i);
cf->data[i] = (u8)(slot[j] >> shift);
}
- }
- /* Update statistics and read pointer */
+ stats->rx_bytes += cf->len;
+ }
stats->rx_packets++;
- stats->rx_bytes += cf->len;
+
netif_receive_skb(skb);
rd = grcan_ring_add(rd, GRCAN_MSG_SIZE, dma->rx.size);
@@ -1447,7 +1435,6 @@ static netdev_tx_t grcan_start_xmit(struct sk_buff *skb,
* can_put_echo_skb would be an error unless other measures are
* taken.
*/
- priv->txdlc[slotindex] = cf->len; /* Store dlc for statistics */
can_put_echo_skb(skb, dev, slotindex, 0);
/* Make sure everything is written before allowing hardware to
diff --git a/drivers/net/can/ifi_canfd/ifi_canfd.c b/drivers/net/can/ifi_canfd/ifi_canfd.c
index 5bb957a..b0a3473 100644
--- a/drivers/net/can/ifi_canfd/ifi_canfd.c
+++ b/drivers/net/can/ifi_canfd/ifi_canfd.c
@@ -309,15 +309,15 @@ static void ifi_canfd_read_fifo(struct net_device *ndev)
*(u32 *)(cf->data + i) =
readl(priv->base + IFI_CANFD_RXFIFO_DATA + i);
}
+
+ stats->rx_bytes += cf->len;
}
+ stats->rx_packets++;
/* Remove the packet from FIFO */
writel(IFI_CANFD_RXSTCMD_REMOVE_MSG, priv->base + IFI_CANFD_RXSTCMD);
writel(rx_irq_mask, priv->base + IFI_CANFD_INTERRUPT);
- stats->rx_packets++;
- stats->rx_bytes += cf->len;
-
netif_receive_skb(skb);
}
@@ -430,8 +430,6 @@ static int ifi_canfd_handle_lec_err(struct net_device *ndev)
priv->base + IFI_CANFD_INTERRUPT);
writel(IFI_CANFD_ERROR_CTR_ER_ENABLE, priv->base + IFI_CANFD_ERROR_CTR);
- stats->rx_packets++;
- stats->rx_bytes += cf->len;
netif_receive_skb(skb);
return 1;
@@ -456,7 +454,6 @@ static int ifi_canfd_handle_state_change(struct net_device *ndev,
enum can_state new_state)
{
struct ifi_canfd_priv *priv = netdev_priv(ndev);
- struct net_device_stats *stats = &ndev->stats;
struct can_frame *cf;
struct sk_buff *skb;
struct can_berr_counter bec;
@@ -522,8 +519,6 @@ static int ifi_canfd_handle_state_change(struct net_device *ndev,
break;
}
- stats->rx_packets++;
- stats->rx_bytes += cf->len;
netif_receive_skb(skb);
return 1;
diff --git a/drivers/net/can/janz-ican3.c b/drivers/net/can/janz-ican3.c
index 32006db..808c105 100644
--- a/drivers/net/can/janz-ican3.c
+++ b/drivers/net/can/janz-ican3.c
@@ -1285,7 +1285,7 @@ static unsigned int ican3_get_echo_skb(struct ican3_dev *mod)
{
struct sk_buff *skb = skb_dequeue(&mod->echoq);
struct can_frame *cf;
- u8 dlc;
+ u8 dlc = 0;
/* this should never trigger unless there is a driver bug */
if (!skb) {
@@ -1294,7 +1294,8 @@ static unsigned int ican3_get_echo_skb(struct ican3_dev *mod)
}
cf = (struct can_frame *)skb->data;
- dlc = cf->len;
+ if (!(cf->can_id & CAN_RTR_FLAG))
+ dlc = cf->len;
/* check flag whether this packet has to be looped back */
if (skb->pkt_type != PACKET_LOOPBACK) {
@@ -1421,7 +1422,8 @@ static int ican3_recv_skb(struct ican3_dev *mod)
/* update statistics, receive the skb */
stats->rx_packets++;
- stats->rx_bytes += cf->len;
+ if (!(cf->can_id & CAN_RTR_FLAG))
+ stats->rx_bytes += cf->len;
netif_receive_skb(skb);
err_noalloc:
diff --git a/drivers/net/can/kvaser_pciefd.c b/drivers/net/can/kvaser_pciefd.c
index eb74cdf..017f2d3 100644
--- a/drivers/net/can/kvaser_pciefd.c
+++ b/drivers/net/can/kvaser_pciefd.c
@@ -1185,20 +1185,21 @@ static int kvaser_pciefd_handle_data_packet(struct kvaser_pciefd *pcie,
cf->len = can_fd_dlc2len(p->header[1] >> KVASER_PCIEFD_RPACKET_DLC_SHIFT);
- if (p->header[0] & KVASER_PCIEFD_RPACKET_RTR)
+ if (p->header[0] & KVASER_PCIEFD_RPACKET_RTR) {
cf->can_id |= CAN_RTR_FLAG;
- else
+ } else {
memcpy(cf->data, data, cf->len);
+ stats->rx_bytes += cf->len;
+ }
+ stats->rx_packets++;
+
shhwtstamps = skb_hwtstamps(skb);
shhwtstamps->hwtstamp =
ns_to_ktime(div_u64(p->timestamp * 1000,
pcie->freq_to_ticks_div));
- stats->rx_bytes += cf->len;
- stats->rx_packets++;
-
return netif_rx(skb);
}
@@ -1310,9 +1311,6 @@ static int kvaser_pciefd_rx_error_frame(struct kvaser_pciefd_can *can,
cf->data[6] = bec.txerr;
cf->data[7] = bec.rxerr;
- stats->rx_packets++;
- stats->rx_bytes += cf->len;
-
netif_rx(skb);
return 0;
}
@@ -1510,8 +1508,6 @@ static void kvaser_pciefd_handle_nack_packet(struct kvaser_pciefd_can *can,
if (skb) {
cf->can_id |= CAN_ERR_BUSERROR;
- stats->rx_bytes += cf->len;
- stats->rx_packets++;
netif_rx(skb);
} else {
stats->rx_dropped++;
diff --git a/drivers/net/can/m_can/m_can.c b/drivers/net/can/m_can/m_can.c
index c2a8421..5b47cd8 100644
--- a/drivers/net/can/m_can/m_can.c
+++ b/drivers/net/can/m_can/m_can.c
@@ -518,14 +518,14 @@ static int m_can_read_fifo(struct net_device *dev, u32 rxfs)
cf->data, DIV_ROUND_UP(cf->len, 4));
if (err)
goto out_free_skb;
+
+ stats->rx_bytes += cf->len;
}
+ stats->rx_packets++;
/* acknowledge rx fifo 0 */
m_can_write(cdev, M_CAN_RXF0A, fgi);
- stats->rx_packets++;
- stats->rx_bytes += cf->len;
-
timestamp = FIELD_GET(RX_BUF_RXTS_MASK, fifo_header.dlc);
m_can_receive_skb(cdev, skb, timestamp);
@@ -647,9 +647,6 @@ static int m_can_handle_lec_err(struct net_device *dev,
break;
}
- stats->rx_packets++;
- stats->rx_bytes += cf->len;
-
if (cdev->is_peripheral)
timestamp = m_can_get_timestamp(cdev);
@@ -706,7 +703,6 @@ static int m_can_handle_state_change(struct net_device *dev,
enum can_state new_state)
{
struct m_can_classdev *cdev = netdev_priv(dev);
- struct net_device_stats *stats = &dev->stats;
struct can_frame *cf;
struct sk_buff *skb;
struct can_berr_counter bec;
@@ -771,9 +767,6 @@ static int m_can_handle_state_change(struct net_device *dev,
break;
}
- stats->rx_packets++;
- stats->rx_bytes += cf->len;
-
if (cdev->is_peripheral)
timestamp = m_can_get_timestamp(cdev);
@@ -1463,7 +1456,7 @@ static bool m_can_niso_supported(struct m_can_classdev *cdev)
static int m_can_dev_setup(struct m_can_classdev *cdev)
{
struct net_device *dev = cdev->net;
- int m_can_version;
+ int m_can_version, err;
m_can_version = m_can_check_core_release(cdev);
/* return if unsupported version */
@@ -1493,7 +1486,9 @@ static int m_can_dev_setup(struct m_can_classdev *cdev)
switch (cdev->version) {
case 30:
/* CAN_CTRLMODE_FD_NON_ISO is fixed with M_CAN IP v3.0.x */
- can_set_static_ctrlmode(dev, CAN_CTRLMODE_FD_NON_ISO);
+ err = can_set_static_ctrlmode(dev, CAN_CTRLMODE_FD_NON_ISO);
+ if (err)
+ return err;
cdev->can.bittiming_const = cdev->bit_timing ?
cdev->bit_timing : &m_can_bittiming_const_30X;
@@ -1503,7 +1498,9 @@ static int m_can_dev_setup(struct m_can_classdev *cdev)
break;
case 31:
/* CAN_CTRLMODE_FD_NON_ISO is fixed with M_CAN IP v3.1.x */
- can_set_static_ctrlmode(dev, CAN_CTRLMODE_FD_NON_ISO);
+ err = can_set_static_ctrlmode(dev, CAN_CTRLMODE_FD_NON_ISO);
+ if (err)
+ return err;
cdev->can.bittiming_const = cdev->bit_timing ?
cdev->bit_timing : &m_can_bittiming_const_31X;
diff --git a/drivers/net/can/mscan/mscan.c b/drivers/net/can/mscan/mscan.c
index fa32e41..5b5802f 100644
--- a/drivers/net/can/mscan/mscan.c
+++ b/drivers/net/can/mscan/mscan.c
@@ -401,13 +401,15 @@ static int mscan_rx_poll(struct napi_struct *napi, int quota)
continue;
}
- if (canrflg & MSCAN_RXF)
+ if (canrflg & MSCAN_RXF) {
mscan_get_rx_frame(dev, frame);
- else if (canrflg & MSCAN_ERR_IF)
+ stats->rx_packets++;
+ if (!(frame->can_id & CAN_RTR_FLAG))
+ stats->rx_bytes += frame->len;
+ } else if (canrflg & MSCAN_ERR_IF) {
mscan_get_err_frame(dev, frame, canrflg);
+ }
- stats->rx_packets++;
- stats->rx_bytes += frame->len;
work_done++;
netif_receive_skb(skb);
}
@@ -446,9 +448,9 @@ static irqreturn_t mscan_isr(int irq, void *dev_id)
continue;
out_8(®s->cantbsel, mask);
- stats->tx_bytes += in_8(®s->tx.dlr);
+ stats->tx_bytes += can_get_echo_skb(dev, entry->id,
+ NULL);
stats->tx_packets++;
- can_get_echo_skb(dev, entry->id, NULL);
priv->tx_active &= ~mask;
list_del(pos);
}
diff --git a/drivers/net/can/pch_can.c b/drivers/net/can/pch_can.c
index 964c8a0..888bef0 100644
--- a/drivers/net/can/pch_can.c
+++ b/drivers/net/can/pch_can.c
@@ -561,9 +561,6 @@ static void pch_can_error(struct net_device *ndev, u32 status)
priv->can.state = state;
netif_receive_skb(skb);
-
- stats->rx_packets++;
- stats->rx_bytes += cf->len;
}
static irqreturn_t pch_can_interrupt(int irq, void *dev_id)
@@ -680,22 +677,23 @@ static int pch_can_rx_normal(struct net_device *ndev, u32 obj_num, int quota)
cf->can_id = id;
}
- if (id2 & PCH_ID2_DIR)
- cf->can_id |= CAN_RTR_FLAG;
-
cf->len = can_cc_dlc2len((ioread32(&priv->regs->
ifregs[0].mcont)) & 0xF);
- for (i = 0; i < cf->len; i += 2) {
- data_reg = ioread16(&priv->regs->ifregs[0].data[i / 2]);
- cf->data[i] = data_reg;
- cf->data[i + 1] = data_reg >> 8;
- }
+ if (id2 & PCH_ID2_DIR) {
+ cf->can_id |= CAN_RTR_FLAG;
+ } else {
+ for (i = 0; i < cf->len; i += 2) {
+ data_reg = ioread16(&priv->regs->ifregs[0].data[i / 2]);
+ cf->data[i] = data_reg;
+ cf->data[i + 1] = data_reg >> 8;
+ }
- rcv_pkts++;
+ stats->rx_bytes += cf->len;
+ }
stats->rx_packets++;
+ rcv_pkts++;
quota--;
- stats->rx_bytes += cf->len;
netif_receive_skb(skb);
pch_fifo_thresh(priv, obj_num);
@@ -709,16 +707,13 @@ static void pch_can_tx_complete(struct net_device *ndev, u32 int_stat)
{
struct pch_can_priv *priv = netdev_priv(ndev);
struct net_device_stats *stats = &(priv->ndev->stats);
- u32 dlc;
- can_get_echo_skb(ndev, int_stat - PCH_RX_OBJ_END - 1, NULL);
+ stats->tx_bytes += can_get_echo_skb(ndev, int_stat - PCH_RX_OBJ_END - 1,
+ NULL);
+ stats->tx_packets++;
iowrite32(PCH_CMASK_RX_TX_GET | PCH_CMASK_CLRINTPND,
&priv->regs->ifregs[1].cmask);
pch_can_rw_msg_obj(&priv->regs->ifregs[1].creq, int_stat);
- dlc = can_cc_dlc2len(ioread32(&priv->regs->ifregs[1].mcont) &
- PCH_IF_MCONT_DLC);
- stats->tx_bytes += dlc;
- stats->tx_packets++;
if (int_stat == PCH_TX_OBJ_END)
netif_wake_queue(ndev);
}
diff --git a/drivers/net/can/peak_canfd/peak_canfd.c b/drivers/net/can/peak_canfd/peak_canfd.c
index d08718e..b2dea36 100644
--- a/drivers/net/can/peak_canfd/peak_canfd.c
+++ b/drivers/net/can/peak_canfd/peak_canfd.c
@@ -266,10 +266,9 @@ static int pucan_handle_can_rx(struct peak_canfd_priv *priv,
unsigned long flags;
spin_lock_irqsave(&priv->echo_lock, flags);
- can_get_echo_skb(priv->ndev, msg->client, NULL);
/* count bytes of the echo instead of skb */
- stats->tx_bytes += cf_len;
+ stats->tx_bytes += can_get_echo_skb(priv->ndev, msg->client, NULL);
stats->tx_packets++;
/* restart tx queue (a slot is free) */
@@ -310,12 +309,13 @@ static int pucan_handle_can_rx(struct peak_canfd_priv *priv,
if (rx_msg_flags & PUCAN_MSG_EXT_ID)
cf->can_id |= CAN_EFF_FLAG;
- if (rx_msg_flags & PUCAN_MSG_RTR)
+ if (rx_msg_flags & PUCAN_MSG_RTR) {
cf->can_id |= CAN_RTR_FLAG;
- else
+ } else {
memcpy(cf->data, msg->d, cf->len);
- stats->rx_bytes += cf->len;
+ stats->rx_bytes += cf->len;
+ }
stats->rx_packets++;
pucan_netif_rx(skb, msg->ts_low, msg->ts_high);
@@ -409,8 +409,6 @@ static int pucan_handle_status(struct peak_canfd_priv *priv,
return -ENOMEM;
}
- stats->rx_packets++;
- stats->rx_bytes += cf->len;
pucan_netif_rx(skb, msg->ts_low, msg->ts_high);
return 0;
@@ -438,8 +436,6 @@ static int pucan_handle_cache_critical(struct peak_canfd_priv *priv)
cf->data[6] = priv->bec.txerr;
cf->data[7] = priv->bec.rxerr;
- stats->rx_bytes += cf->len;
- stats->rx_packets++;
netif_rx(skb);
return 0;
diff --git a/drivers/net/can/rcar/rcar_can.c b/drivers/net/can/rcar/rcar_can.c
index 8999ec9..33e3739 100644
--- a/drivers/net/can/rcar/rcar_can.c
+++ b/drivers/net/can/rcar/rcar_can.c
@@ -94,7 +94,6 @@ struct rcar_can_priv {
struct rcar_can_regs __iomem *regs;
struct clk *clk;
struct clk *can_clk;
- u8 tx_dlc[RCAR_CAN_FIFO_DEPTH];
u32 tx_head;
u32 tx_tail;
u8 clock_select;
@@ -223,7 +222,6 @@ static void tx_failure_cleanup(struct net_device *ndev)
static void rcar_can_error(struct net_device *ndev)
{
struct rcar_can_priv *priv = netdev_priv(ndev);
- struct net_device_stats *stats = &ndev->stats;
struct can_frame *cf;
struct sk_buff *skb;
u8 eifr, txerr = 0, rxerr = 0;
@@ -362,11 +360,8 @@ static void rcar_can_error(struct net_device *ndev)
}
}
- if (skb) {
- stats->rx_packets++;
- stats->rx_bytes += cf->len;
+ if (skb)
netif_rx(skb);
- }
}
static void rcar_can_tx_done(struct net_device *ndev)
@@ -383,10 +378,11 @@ static void rcar_can_tx_done(struct net_device *ndev)
if (priv->tx_head - priv->tx_tail <= unsent)
break;
stats->tx_packets++;
- stats->tx_bytes += priv->tx_dlc[priv->tx_tail %
- RCAR_CAN_FIFO_DEPTH];
- priv->tx_dlc[priv->tx_tail % RCAR_CAN_FIFO_DEPTH] = 0;
- can_get_echo_skb(ndev, priv->tx_tail % RCAR_CAN_FIFO_DEPTH, NULL);
+ stats->tx_bytes +=
+ can_get_echo_skb(ndev,
+ priv->tx_tail % RCAR_CAN_FIFO_DEPTH,
+ NULL);
+
priv->tx_tail++;
netif_wake_queue(ndev);
}
@@ -616,7 +612,6 @@ static netdev_tx_t rcar_can_start_xmit(struct sk_buff *skb,
writeb(cf->len, &priv->regs->mb[RCAR_CAN_TX_FIFO_MBX].dlc);
- priv->tx_dlc[priv->tx_head % RCAR_CAN_FIFO_DEPTH] = cf->len;
can_put_echo_skb(skb, ndev, priv->tx_head % RCAR_CAN_FIFO_DEPTH, 0);
priv->tx_head++;
/* Start Tx: write 0xff to the TFPCR register to increment
@@ -666,12 +661,13 @@ static void rcar_can_rx_pkt(struct rcar_can_priv *priv)
for (dlc = 0; dlc < cf->len; dlc++)
cf->data[dlc] =
readb(&priv->regs->mb[RCAR_CAN_RX_FIFO_MBX].data[dlc]);
+
+ stats->rx_bytes += cf->len;
}
+ stats->rx_packets++;
can_led_event(priv->ndev, CAN_LED_EVENT_RX);
- stats->rx_bytes += cf->len;
- stats->rx_packets++;
netif_receive_skb(skb);
}
diff --git a/drivers/net/can/rcar/rcar_canfd.c b/drivers/net/can/rcar/rcar_canfd.c
index ff9d0f5..b7dc1c3 100644
--- a/drivers/net/can/rcar/rcar_canfd.c
+++ b/drivers/net/can/rcar/rcar_canfd.c
@@ -502,7 +502,6 @@ struct rcar_canfd_channel {
struct rcar_canfd_global *gpriv; /* Controller reference */
void __iomem *base; /* Register base address */
struct napi_struct napi;
- u8 tx_len[RCANFD_FIFO_DEPTH]; /* For net stats */
u32 tx_head; /* Incremented on xmit */
u32 tx_tail; /* Incremented on xmit done */
u32 channel; /* Channel number */
@@ -1033,8 +1032,6 @@ static void rcar_canfd_error(struct net_device *ndev, u32 cerfl,
/* Clear channel error interrupts that are handled */
rcar_canfd_write(priv->base, RCANFD_CERFL(ch),
RCANFD_CERFL_ERR(~cerfl));
- stats->rx_packets++;
- stats->rx_bytes += cf->len;
netif_rx(skb);
}
@@ -1051,9 +1048,7 @@ static void rcar_canfd_tx_done(struct net_device *ndev)
sent = priv->tx_tail % RCANFD_FIFO_DEPTH;
stats->tx_packets++;
- stats->tx_bytes += priv->tx_len[sent];
- priv->tx_len[sent] = 0;
- can_get_echo_skb(ndev, sent, NULL);
+ stats->tx_bytes += can_get_echo_skb(ndev, sent, NULL);
spin_lock_irqsave(&priv->tx_lock, flags);
priv->tx_tail++;
@@ -1174,8 +1169,6 @@ static void rcar_canfd_state_change(struct net_device *ndev,
rx_state = txerr <= rxerr ? state : 0;
can_change_state(ndev, cf, tx_state, rx_state);
- stats->rx_packets++;
- stats->rx_bytes += cf->len;
netif_rx(skb);
}
}
@@ -1465,7 +1458,6 @@ static netdev_tx_t rcar_canfd_start_xmit(struct sk_buff *skb,
RCANFD_C_CFDF(ch, RCANFD_CFFIFO_IDX, 0));
}
- priv->tx_len[priv->tx_head % RCANFD_FIFO_DEPTH] = cf->len;
can_put_echo_skb(skb, ndev, priv->tx_head % RCANFD_FIFO_DEPTH, 0);
spin_lock_irqsave(&priv->tx_lock, flags);
@@ -1554,7 +1546,8 @@ static void rcar_canfd_rx_pkt(struct rcar_canfd_channel *priv)
can_led_event(priv->ndev, CAN_LED_EVENT_RX);
- stats->rx_bytes += cf->len;
+ if (!(cf->can_id & CAN_RTR_FLAG))
+ stats->rx_bytes += cf->len;
stats->rx_packets++;
netif_receive_skb(skb);
}
@@ -1640,8 +1633,7 @@ static int rcar_canfd_channel_probe(struct rcar_canfd_global *gpriv, u32 ch,
ndev = alloc_candev(sizeof(*priv), RCANFD_FIFO_DEPTH);
if (!ndev) {
dev_err(&pdev->dev, "alloc_candev() failed\n");
- err = -ENOMEM;
- goto fail;
+ return -ENOMEM;
}
priv = netdev_priv(ndev);
@@ -1706,7 +1698,9 @@ static int rcar_canfd_channel_probe(struct rcar_canfd_global *gpriv, u32 ch,
&rcar_canfd_data_bittiming_const;
/* Controller starts in CAN FD only mode */
- can_set_static_ctrlmode(ndev, CAN_CTRLMODE_FD);
+ err = can_set_static_ctrlmode(ndev, CAN_CTRLMODE_FD);
+ if (err)
+ goto fail;
priv->can.ctrlmode_supported = CAN_CTRLMODE_BERR_REPORTING;
} else {
/* Controller starts in Classical CAN only mode */
@@ -1735,8 +1729,8 @@ static int rcar_canfd_channel_probe(struct rcar_canfd_global *gpriv, u32 ch,
fail_candev:
netif_napi_del(&priv->napi);
- free_candev(ndev);
fail:
+ free_candev(ndev);
return err;
}
diff --git a/drivers/net/can/sja1000/sja1000.c b/drivers/net/can/sja1000/sja1000.c
index 3fad546..9663164 100644
--- a/drivers/net/can/sja1000/sja1000.c
+++ b/drivers/net/can/sja1000/sja1000.c
@@ -372,15 +372,16 @@ static void sja1000_rx(struct net_device *dev)
} else {
for (i = 0; i < cf->len; i++)
cf->data[i] = priv->read_reg(priv, dreg++);
+
+ stats->rx_bytes += cf->len;
}
+ stats->rx_packets++;
cf->can_id = id;
/* release receive buffer */
sja1000_write_cmdreg(priv, CMD_RRB);
- stats->rx_packets++;
- stats->rx_bytes += cf->len;
netif_rx(skb);
can_led_event(dev, CAN_LED_EVENT_RX);
@@ -487,8 +488,6 @@ static int sja1000_err(struct net_device *dev, uint8_t isrc, uint8_t status)
can_bus_off(dev);
}
- stats->rx_packets++;
- stats->rx_bytes += cf->len;
netif_rx(skb);
return 0;
@@ -528,10 +527,8 @@ irqreturn_t sja1000_interrupt(int irq, void *dev_id)
can_free_echo_skb(dev, 0, NULL);
} else {
/* transmission complete */
- stats->tx_bytes +=
- priv->read_reg(priv, SJA1000_FI) & 0xf;
+ stats->tx_bytes += can_get_echo_skb(dev, 0, NULL);
stats->tx_packets++;
- can_get_echo_skb(dev, 0, NULL);
}
netif_wake_queue(dev);
can_led_event(dev, CAN_LED_EVENT_TX);
diff --git a/drivers/net/can/sja1000/sja1000_platform.c b/drivers/net/can/sja1000/sja1000_platform.c
index d7c2ec5..f9ec7bd 100644
--- a/drivers/net/can/sja1000/sja1000_platform.c
+++ b/drivers/net/can/sja1000/sja1000_platform.c
@@ -17,7 +17,6 @@
#include <linux/io.h>
#include <linux/of.h>
#include <linux/of_device.h>
-#include <linux/of_irq.h>
#include "sja1000.h"
@@ -234,13 +233,15 @@ static int sp_probe(struct platform_device *pdev)
if (!addr)
return -ENOMEM;
- if (of)
- irq = irq_of_parse_and_map(of, 0);
- else
+ if (of) {
+ irq = platform_get_irq(pdev, 0);
+ if (irq < 0)
+ return irq;
+ } else {
res_irq = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
-
- if (!irq && !res_irq)
- return -ENODEV;
+ if (!res_irq)
+ return -ENODEV;
+ }
of_id = of_match_device(sp_of_table, &pdev->dev);
if (of_id && of_id->data) {
diff --git a/drivers/net/can/slcan.c b/drivers/net/can/slcan.c
index 9a4ebda..d4c7ce9 100644
--- a/drivers/net/can/slcan.c
+++ b/drivers/net/can/slcan.c
@@ -218,7 +218,9 @@ static void slc_bump(struct slcan *sl)
skb_put_data(skb, &cf, sizeof(struct can_frame));
sl->dev->stats.rx_packets++;
- sl->dev->stats.rx_bytes += cf.len;
+ if (!(cf.can_id & CAN_RTR_FLAG))
+ sl->dev->stats.rx_bytes += cf.len;
+
netif_rx_ni(skb);
}
@@ -288,6 +290,8 @@ static void slc_encaps(struct slcan *sl, struct can_frame *cf)
if (!(cf->can_id & CAN_RTR_FLAG)) {
for (i = 0; i < cf->len; i++)
pos = hex_byte_pack_upper(pos, cf->data[i]);
+
+ sl->dev->stats.tx_bytes += cf->len;
}
*pos++ = '\r';
@@ -304,7 +308,6 @@ static void slc_encaps(struct slcan *sl, struct can_frame *cf)
actual = sl->tty->ops->write(sl->tty, sl->xbuff, pos - sl->xbuff);
sl->xleft = (pos - sl->xbuff) - actual;
sl->xhead = sl->xbuff + actual;
- sl->dev->stats.tx_bytes += cf->len;
}
/* Write out any remaining transmit buffer. Scheduled when tty is writable */
diff --git a/drivers/net/can/softing/softing_cs.c b/drivers/net/can/softing/softing_cs.c
index 2e93ee7..e5c939b 100644
--- a/drivers/net/can/softing/softing_cs.c
+++ b/drivers/net/can/softing/softing_cs.c
@@ -293,7 +293,7 @@ static int softingcs_probe(struct pcmcia_device *pcmcia)
return 0;
platform_failed:
- kfree(dev);
+ platform_device_put(pdev);
mem_failed:
pcmcia_bad:
pcmcia_failed:
diff --git a/drivers/net/can/softing/softing_fw.c b/drivers/net/can/softing/softing_fw.c
index 7e15368..32286f8 100644
--- a/drivers/net/can/softing/softing_fw.c
+++ b/drivers/net/can/softing/softing_fw.c
@@ -565,18 +565,19 @@ int softing_startstop(struct net_device *dev, int up)
if (ret < 0)
goto failed;
}
- /* enable_error_frame */
- /*
+
+ /* enable_error_frame
+ *
* Error reporting is switched off at the moment since
* the receiving of them is not yet 100% verified
* This should be enabled sooner or later
- *
- if (error_reporting) {
+ */
+ if (0 && error_reporting) {
ret = softing_fct_cmd(card, 51, "enable_error_frame");
if (ret < 0)
goto failed;
}
- */
+
/* initialize interface */
iowrite16(1, &card->dpram[DPRAM_FCT_PARAM + 2]);
iowrite16(1, &card->dpram[DPRAM_FCT_PARAM + 4]);
diff --git a/drivers/net/can/softing/softing_main.c b/drivers/net/can/softing/softing_main.c
index cfc1325..d74e895 100644
--- a/drivers/net/can/softing/softing_main.c
+++ b/drivers/net/can/softing/softing_main.c
@@ -282,7 +282,10 @@ static int softing_handle_1(struct softing *card)
skb = priv->can.echo_skb[priv->tx.echo_get];
if (skb)
skb->tstamp = ktime;
- can_get_echo_skb(netdev, priv->tx.echo_get, NULL);
+ ++netdev->stats.tx_packets;
+ netdev->stats.tx_bytes +=
+ can_get_echo_skb(netdev, priv->tx.echo_get,
+ NULL);
++priv->tx.echo_get;
if (priv->tx.echo_get >= TX_ECHO_SKB_MAX)
priv->tx.echo_get = 0;
@@ -290,9 +293,6 @@ static int softing_handle_1(struct softing *card)
--priv->tx.pending;
if (card->tx.pending)
--card->tx.pending;
- ++netdev->stats.tx_packets;
- if (!(msg.can_id & CAN_RTR_FLAG))
- netdev->stats.tx_bytes += msg.len;
} else {
int ret;
diff --git a/drivers/net/can/spi/hi311x.c b/drivers/net/can/spi/hi311x.c
index 89d9c98..cfcc14f 100644
--- a/drivers/net/can/spi/hi311x.c
+++ b/drivers/net/can/spi/hi311x.c
@@ -25,11 +25,11 @@
#include <linux/interrupt.h>
#include <linux/io.h>
#include <linux/kernel.h>
+#include <linux/mod_devicetable.h>
#include <linux/module.h>
#include <linux/netdevice.h>
-#include <linux/of.h>
-#include <linux/of_device.h>
#include <linux/platform_device.h>
+#include <linux/property.h>
#include <linux/regulator/consumer.h>
#include <linux/slab.h>
#include <linux/spi/spi.h>
@@ -153,7 +153,6 @@ struct hi3110_priv {
u8 *spi_rx_buf;
struct sk_buff *tx_skb;
- int tx_len;
struct workqueue_struct *wq;
struct work_struct tx_work;
@@ -166,6 +165,8 @@ struct hi3110_priv {
#define HI3110_AFTER_SUSPEND_POWER 4
#define HI3110_AFTER_SUSPEND_RESTART 8
int restart_tx;
+ bool tx_busy;
+
struct regulator *power;
struct regulator *transceiver;
struct clk *clk;
@@ -175,13 +176,13 @@ static void hi3110_clean(struct net_device *net)
{
struct hi3110_priv *priv = netdev_priv(net);
- if (priv->tx_skb || priv->tx_len)
+ if (priv->tx_skb || priv->tx_busy)
net->stats.tx_errors++;
dev_kfree_skb(priv->tx_skb);
- if (priv->tx_len)
+ if (priv->tx_busy)
can_free_echo_skb(priv->net, 0, NULL);
priv->tx_skb = NULL;
- priv->tx_len = 0;
+ priv->tx_busy = false;
}
/* Note about handling of error return of hi3110_spi_trans: accessing
@@ -343,14 +344,15 @@ static void hi3110_hw_rx(struct spi_device *spi)
/* Data length */
frame->len = can_cc_dlc2len(buf[HI3110_FIFO_WOTIME_DLC_OFF] & 0x0F);
- if (buf[HI3110_FIFO_WOTIME_ID_OFF + 3] & HI3110_FIFO_WOTIME_ID_RTR)
+ if (buf[HI3110_FIFO_WOTIME_ID_OFF + 3] & HI3110_FIFO_WOTIME_ID_RTR) {
frame->can_id |= CAN_RTR_FLAG;
- else
+ } else {
memcpy(frame->data, buf + HI3110_FIFO_WOTIME_DAT_OFF,
frame->len);
+ priv->net->stats.rx_bytes += frame->len;
+ }
priv->net->stats.rx_packets++;
- priv->net->stats.rx_bytes += frame->len;
can_led_event(priv->net, CAN_LED_EVENT_RX);
@@ -368,7 +370,7 @@ static netdev_tx_t hi3110_hard_start_xmit(struct sk_buff *skb,
struct hi3110_priv *priv = netdev_priv(net);
struct spi_device *spi = priv->spi;
- if (priv->tx_skb || priv->tx_len) {
+ if (priv->tx_skb || priv->tx_busy) {
dev_err(&spi->dev, "hard_xmit called while tx busy\n");
return NETDEV_TX_BUSY;
}
@@ -585,7 +587,7 @@ static void hi3110_tx_work_handler(struct work_struct *ws)
} else {
frame = (struct can_frame *)priv->tx_skb->data;
hi3110_hw_tx(spi, frame);
- priv->tx_len = 1 + frame->len;
+ priv->tx_busy = true;
can_put_echo_skb(priv->tx_skb, net, 0, 0);
priv->tx_skb = NULL;
}
@@ -720,14 +722,11 @@ static irqreturn_t hi3110_can_ist(int irq, void *dev_id)
}
}
- if (priv->tx_len && statf & HI3110_STAT_TXMTY) {
+ if (priv->tx_busy && statf & HI3110_STAT_TXMTY) {
net->stats.tx_packets++;
- net->stats.tx_bytes += priv->tx_len - 1;
+ net->stats.tx_bytes += can_get_echo_skb(net, 0, NULL);
can_led_event(net, CAN_LED_EVENT_TX);
- if (priv->tx_len) {
- can_get_echo_skb(net, 0, NULL);
- priv->tx_len = 0;
- }
+ priv->tx_busy = false;
netif_wake_queue(net);
}
@@ -754,7 +753,7 @@ static int hi3110_open(struct net_device *net)
priv->force_quit = 0;
priv->tx_skb = NULL;
- priv->tx_len = 0;
+ priv->tx_busy = false;
ret = request_threaded_irq(spi->irq, NULL, hi3110_can_ist,
flags, DEVICE_NAME, priv);
@@ -828,19 +827,25 @@ MODULE_DEVICE_TABLE(spi, hi3110_id_table);
static int hi3110_can_probe(struct spi_device *spi)
{
- const struct of_device_id *of_id = of_match_device(hi3110_of_match,
- &spi->dev);
+ struct device *dev = &spi->dev;
struct net_device *net;
struct hi3110_priv *priv;
+ const void *match;
struct clk *clk;
- int freq, ret;
+ u32 freq;
+ int ret;
- clk = devm_clk_get(&spi->dev, NULL);
- if (IS_ERR(clk)) {
- dev_err(&spi->dev, "no CAN clock source defined\n");
- return PTR_ERR(clk);
+ clk = devm_clk_get_optional(&spi->dev, NULL);
+ if (IS_ERR(clk))
+ return dev_err_probe(dev, PTR_ERR(clk), "no CAN clock source defined\n");
+
+ if (clk) {
+ freq = clk_get_rate(clk);
+ } else {
+ ret = device_property_read_u32(dev, "clock-frequency", &freq);
+ if (ret)
+ return dev_err_probe(dev, ret, "Failed to get clock-frequency!\n");
}
- freq = clk_get_rate(clk);
/* Sanity check */
if (freq > 40000000)
@@ -851,11 +856,9 @@ static int hi3110_can_probe(struct spi_device *spi)
if (!net)
return -ENOMEM;
- if (!IS_ERR(clk)) {
- ret = clk_prepare_enable(clk);
- if (ret)
- goto out_free;
- }
+ ret = clk_prepare_enable(clk);
+ if (ret)
+ goto out_free;
net->netdev_ops = &hi3110_netdev_ops;
net->flags |= IFF_ECHO;
@@ -870,8 +873,9 @@ static int hi3110_can_probe(struct spi_device *spi)
CAN_CTRLMODE_LISTENONLY |
CAN_CTRLMODE_BERR_REPORTING;
- if (of_id)
- priv->model = (enum hi3110_model)(uintptr_t)of_id->data;
+ match = device_get_match_data(dev);
+ if (match)
+ priv->model = (enum hi3110_model)(uintptr_t)match;
else
priv->model = spi_get_device_id(spi)->driver_data;
priv->net = net;
@@ -918,9 +922,7 @@ static int hi3110_can_probe(struct spi_device *spi)
ret = hi3110_hw_probe(spi);
if (ret) {
- if (ret == -ENODEV)
- dev_err(&spi->dev, "Cannot initialize %x. Wrong wiring?\n",
- priv->model);
+ dev_err_probe(dev, ret, "Cannot initialize %x. Wrong wiring?\n", priv->model);
goto error_probe;
}
hi3110_hw_sleep(spi);
@@ -938,14 +940,12 @@ static int hi3110_can_probe(struct spi_device *spi)
hi3110_power_enable(priv->power, 0);
out_clk:
- if (!IS_ERR(clk))
- clk_disable_unprepare(clk);
+ clk_disable_unprepare(clk);
out_free:
free_candev(net);
- dev_err(&spi->dev, "Probe failed, err=%d\n", -ret);
- return ret;
+ return dev_err_probe(dev, ret, "Probe failed\n");
}
static int hi3110_can_remove(struct spi_device *spi)
@@ -957,8 +957,7 @@ static int hi3110_can_remove(struct spi_device *spi)
hi3110_power_enable(priv->power, 0);
- if (!IS_ERR(priv->clk))
- clk_disable_unprepare(priv->clk);
+ clk_disable_unprepare(priv->clk);
free_candev(net);
diff --git a/drivers/net/can/spi/mcp251x.c b/drivers/net/can/spi/mcp251x.c
index 0579ab7..025e07c 100644
--- a/drivers/net/can/spi/mcp251x.c
+++ b/drivers/net/can/spi/mcp251x.c
@@ -237,7 +237,6 @@ struct mcp251x_priv {
u8 *spi_rx_buf;
struct sk_buff *tx_skb;
- int tx_len;
struct workqueue_struct *wq;
struct work_struct tx_work;
@@ -250,6 +249,8 @@ struct mcp251x_priv {
#define AFTER_SUSPEND_POWER 4
#define AFTER_SUSPEND_RESTART 8
int restart_tx;
+ bool tx_busy;
+
struct regulator *power;
struct regulator *transceiver;
struct clk *clk;
@@ -272,13 +273,13 @@ static void mcp251x_clean(struct net_device *net)
{
struct mcp251x_priv *priv = netdev_priv(net);
- if (priv->tx_skb || priv->tx_len)
+ if (priv->tx_skb || priv->tx_busy)
net->stats.tx_errors++;
dev_kfree_skb(priv->tx_skb);
- if (priv->tx_len)
+ if (priv->tx_busy)
can_free_echo_skb(priv->net, 0, NULL);
priv->tx_skb = NULL;
- priv->tx_len = 0;
+ priv->tx_busy = false;
}
/* Note about handling of error return of mcp251x_spi_trans: accessing
@@ -600,9 +601,6 @@ static int mcp251x_gpio_setup(struct mcp251x_priv *priv)
gpio->ngpio = ARRAY_SIZE(mcp251x_gpio_names);
gpio->names = mcp251x_gpio_names;
gpio->can_sleep = true;
-#ifdef CONFIG_OF_GPIO
- gpio->of_node = priv->spi->dev.of_node;
-#endif
return devm_gpiochip_add_data(&priv->spi->dev, gpio, priv);
}
@@ -733,10 +731,12 @@ static void mcp251x_hw_rx(struct spi_device *spi, int buf_idx)
}
/* Data length */
frame->len = can_cc_dlc2len(buf[RXBDLC_OFF] & RXBDLC_LEN_MASK);
- memcpy(frame->data, buf + RXBDAT_OFF, frame->len);
+ if (!(frame->can_id & CAN_RTR_FLAG)) {
+ memcpy(frame->data, buf + RXBDAT_OFF, frame->len);
+ priv->net->stats.rx_bytes += frame->len;
+ }
priv->net->stats.rx_packets++;
- priv->net->stats.rx_bytes += frame->len;
can_led_event(priv->net, CAN_LED_EVENT_RX);
@@ -786,7 +786,7 @@ static netdev_tx_t mcp251x_hard_start_xmit(struct sk_buff *skb,
struct mcp251x_priv *priv = netdev_priv(net);
struct spi_device *spi = priv->spi;
- if (priv->tx_skb || priv->tx_len) {
+ if (priv->tx_skb || priv->tx_busy) {
dev_warn(&spi->dev, "hard_xmit called while tx busy\n");
return NETDEV_TX_BUSY;
}
@@ -1011,7 +1011,7 @@ static void mcp251x_tx_work_handler(struct work_struct *ws)
if (frame->len > CAN_FRAME_MAX_DATA_LEN)
frame->len = CAN_FRAME_MAX_DATA_LEN;
mcp251x_hw_tx(spi, frame, 0);
- priv->tx_len = 1 + frame->len;
+ priv->tx_busy = true;
can_put_echo_skb(priv->tx_skb, net, 0, 0);
priv->tx_skb = NULL;
}
@@ -1177,12 +1177,12 @@ static irqreturn_t mcp251x_can_ist(int irq, void *dev_id)
break;
if (intf & CANINTF_TX) {
- net->stats.tx_packets++;
- net->stats.tx_bytes += priv->tx_len - 1;
can_led_event(net, CAN_LED_EVENT_TX);
- if (priv->tx_len) {
- can_get_echo_skb(net, 0, NULL);
- priv->tx_len = 0;
+ if (priv->tx_busy) {
+ net->stats.tx_packets++;
+ net->stats.tx_bytes += can_get_echo_skb(net, 0,
+ NULL);
+ priv->tx_busy = false;
}
netif_wake_queue(net);
}
@@ -1209,7 +1209,7 @@ static int mcp251x_open(struct net_device *net)
priv->force_quit = 0;
priv->tx_skb = NULL;
- priv->tx_len = 0;
+ priv->tx_busy = false;
if (!dev_fwnode(&spi->dev))
flags = IRQF_TRIGGER_FALLING;
diff --git a/drivers/net/can/spi/mcp251xfd/Makefile b/drivers/net/can/spi/mcp251xfd/Makefile
index 3cba3b9..a83d685 100644
--- a/drivers/net/can/spi/mcp251xfd/Makefile
+++ b/drivers/net/can/spi/mcp251xfd/Makefile
@@ -3,9 +3,14 @@
obj-$(CONFIG_CAN_MCP251XFD) += mcp251xfd.o
mcp251xfd-objs :=
+mcp251xfd-objs += mcp251xfd-chip-fifo.o
mcp251xfd-objs += mcp251xfd-core.o
mcp251xfd-objs += mcp251xfd-crc16.o
mcp251xfd-objs += mcp251xfd-regmap.o
+mcp251xfd-objs += mcp251xfd-ring.o
+mcp251xfd-objs += mcp251xfd-rx.o
+mcp251xfd-objs += mcp251xfd-tef.o
mcp251xfd-objs += mcp251xfd-timestamp.o
+mcp251xfd-objs += mcp251xfd-tx.o
mcp251xfd-$(CONFIG_DEV_COREDUMP) += mcp251xfd-dump.o
diff --git a/drivers/net/can/spi/mcp251xfd/mcp251xfd-chip-fifo.c b/drivers/net/can/spi/mcp251xfd/mcp251xfd-chip-fifo.c
new file mode 100644
index 0000000..2f9a623
--- /dev/null
+++ b/drivers/net/can/spi/mcp251xfd/mcp251xfd-chip-fifo.c
@@ -0,0 +1,119 @@
+// SPDX-License-Identifier: GPL-2.0
+//
+// mcp251xfd - Microchip MCP251xFD Family CAN controller driver
+//
+// Copyright (c) 2019, 2020, 2021 Pengutronix,
+// Marc Kleine-Budde <kernel@pengutronix.de>
+//
+// Based on:
+//
+// CAN bus driver for Microchip 25XXFD CAN Controller with SPI Interface
+//
+// Copyright (c) 2019 Martin Sperl <kernel@martin.sperl.org>
+//
+
+#include <linux/bitfield.h>
+
+#include "mcp251xfd.h"
+
+static int
+mcp251xfd_chip_rx_fifo_init_one(const struct mcp251xfd_priv *priv,
+ const struct mcp251xfd_rx_ring *ring)
+{
+ u32 fifo_con;
+
+ /* Enable RXOVIE on _all_ RX FIFOs, not just the last one.
+ *
+ * FIFOs hit by a RX MAB overflow and RXOVIE enabled will
+ * generate a RXOVIF, use this to properly detect RX MAB
+ * overflows.
+ */
+ fifo_con = FIELD_PREP(MCP251XFD_REG_FIFOCON_FSIZE_MASK,
+ ring->obj_num - 1) |
+ MCP251XFD_REG_FIFOCON_RXTSEN |
+ MCP251XFD_REG_FIFOCON_RXOVIE |
+ MCP251XFD_REG_FIFOCON_TFNRFNIE;
+
+ if (mcp251xfd_is_fd_mode(priv))
+ fifo_con |= FIELD_PREP(MCP251XFD_REG_FIFOCON_PLSIZE_MASK,
+ MCP251XFD_REG_FIFOCON_PLSIZE_64);
+ else
+ fifo_con |= FIELD_PREP(MCP251XFD_REG_FIFOCON_PLSIZE_MASK,
+ MCP251XFD_REG_FIFOCON_PLSIZE_8);
+
+ return regmap_write(priv->map_reg,
+ MCP251XFD_REG_FIFOCON(ring->fifo_nr), fifo_con);
+}
+
+static int
+mcp251xfd_chip_rx_filter_init_one(const struct mcp251xfd_priv *priv,
+ const struct mcp251xfd_rx_ring *ring)
+{
+ u32 fltcon;
+
+ fltcon = MCP251XFD_REG_FLTCON_FLTEN(ring->nr) |
+ MCP251XFD_REG_FLTCON_FBP(ring->nr, ring->fifo_nr);
+
+ return regmap_update_bits(priv->map_reg,
+ MCP251XFD_REG_FLTCON(ring->nr >> 2),
+ MCP251XFD_REG_FLTCON_FLT_MASK(ring->nr),
+ fltcon);
+}
+
+int mcp251xfd_chip_fifo_init(const struct mcp251xfd_priv *priv)
+{
+ const struct mcp251xfd_tx_ring *tx_ring = priv->tx;
+ const struct mcp251xfd_rx_ring *rx_ring;
+ u32 val;
+ int err, n;
+
+ /* TEF */
+ val = FIELD_PREP(MCP251XFD_REG_TEFCON_FSIZE_MASK,
+ tx_ring->obj_num - 1) |
+ MCP251XFD_REG_TEFCON_TEFTSEN |
+ MCP251XFD_REG_TEFCON_TEFOVIE |
+ MCP251XFD_REG_TEFCON_TEFNEIE;
+
+ err = regmap_write(priv->map_reg, MCP251XFD_REG_TEFCON, val);
+ if (err)
+ return err;
+
+ /* FIFO 1 - TX */
+ val = FIELD_PREP(MCP251XFD_REG_FIFOCON_FSIZE_MASK,
+ tx_ring->obj_num - 1) |
+ MCP251XFD_REG_FIFOCON_TXEN |
+ MCP251XFD_REG_FIFOCON_TXATIE;
+
+ if (mcp251xfd_is_fd_mode(priv))
+ val |= FIELD_PREP(MCP251XFD_REG_FIFOCON_PLSIZE_MASK,
+ MCP251XFD_REG_FIFOCON_PLSIZE_64);
+ else
+ val |= FIELD_PREP(MCP251XFD_REG_FIFOCON_PLSIZE_MASK,
+ MCP251XFD_REG_FIFOCON_PLSIZE_8);
+
+ if (priv->can.ctrlmode & CAN_CTRLMODE_ONE_SHOT)
+ val |= FIELD_PREP(MCP251XFD_REG_FIFOCON_TXAT_MASK,
+ MCP251XFD_REG_FIFOCON_TXAT_ONE_SHOT);
+ else
+ val |= FIELD_PREP(MCP251XFD_REG_FIFOCON_TXAT_MASK,
+ MCP251XFD_REG_FIFOCON_TXAT_UNLIMITED);
+
+ err = regmap_write(priv->map_reg,
+ MCP251XFD_REG_FIFOCON(MCP251XFD_TX_FIFO),
+ val);
+ if (err)
+ return err;
+
+ /* RX FIFOs */
+ mcp251xfd_for_each_rx_ring(priv, rx_ring, n) {
+ err = mcp251xfd_chip_rx_fifo_init_one(priv, rx_ring);
+ if (err)
+ return err;
+
+ err = mcp251xfd_chip_rx_filter_init_one(priv, rx_ring);
+ if (err)
+ return err;
+ }
+
+ return 0;
+}
diff --git a/drivers/net/can/spi/mcp251xfd/mcp251xfd-core.c b/drivers/net/can/spi/mcp251xfd/mcp251xfd-core.c
index e16dc48..b5986df 100644
--- a/drivers/net/can/spi/mcp251xfd/mcp251xfd-core.c
+++ b/drivers/net/can/spi/mcp251xfd/mcp251xfd-core.c
@@ -20,8 +20,6 @@
#include <linux/pm_runtime.h>
#include <linux/property.h>
-#include <asm/unaligned.h>
-
#include "mcp251xfd.h"
#define DEVICE_NAME "mcp251xfd"
@@ -180,330 +178,6 @@ static int mcp251xfd_clks_and_vdd_disable(const struct mcp251xfd_priv *priv)
return 0;
}
-static inline u8
-mcp251xfd_cmd_prepare_write_reg(const struct mcp251xfd_priv *priv,
- union mcp251xfd_write_reg_buf *write_reg_buf,
- const u16 reg, const u32 mask, const u32 val)
-{
- u8 first_byte, last_byte, len;
- u8 *data;
- __le32 val_le32;
-
- first_byte = mcp251xfd_first_byte_set(mask);
- last_byte = mcp251xfd_last_byte_set(mask);
- len = last_byte - first_byte + 1;
-
- data = mcp251xfd_spi_cmd_write(priv, write_reg_buf, reg + first_byte);
- val_le32 = cpu_to_le32(val >> BITS_PER_BYTE * first_byte);
- memcpy(data, &val_le32, len);
-
- if (priv->devtype_data.quirks & MCP251XFD_QUIRK_CRC_REG) {
- u16 crc;
-
- mcp251xfd_spi_cmd_crc_set_len_in_reg(&write_reg_buf->crc.cmd,
- len);
- /* CRC */
- len += sizeof(write_reg_buf->crc.cmd);
- crc = mcp251xfd_crc16_compute(&write_reg_buf->crc, len);
- put_unaligned_be16(crc, (void *)write_reg_buf + len);
-
- /* Total length */
- len += sizeof(write_reg_buf->crc.crc);
- } else {
- len += sizeof(write_reg_buf->nocrc.cmd);
- }
-
- return len;
-}
-
-static inline int
-mcp251xfd_tef_tail_get_from_chip(const struct mcp251xfd_priv *priv,
- u8 *tef_tail)
-{
- u32 tef_ua;
- int err;
-
- err = regmap_read(priv->map_reg, MCP251XFD_REG_TEFUA, &tef_ua);
- if (err)
- return err;
-
- *tef_tail = tef_ua / sizeof(struct mcp251xfd_hw_tef_obj);
-
- return 0;
-}
-
-static inline int
-mcp251xfd_tx_tail_get_from_chip(const struct mcp251xfd_priv *priv,
- u8 *tx_tail)
-{
- u32 fifo_sta;
- int err;
-
- err = regmap_read(priv->map_reg,
- MCP251XFD_REG_FIFOSTA(MCP251XFD_TX_FIFO),
- &fifo_sta);
- if (err)
- return err;
-
- *tx_tail = FIELD_GET(MCP251XFD_REG_FIFOSTA_FIFOCI_MASK, fifo_sta);
-
- return 0;
-}
-
-static inline int
-mcp251xfd_rx_head_get_from_chip(const struct mcp251xfd_priv *priv,
- const struct mcp251xfd_rx_ring *ring,
- u8 *rx_head)
-{
- u32 fifo_sta;
- int err;
-
- err = regmap_read(priv->map_reg, MCP251XFD_REG_FIFOSTA(ring->fifo_nr),
- &fifo_sta);
- if (err)
- return err;
-
- *rx_head = FIELD_GET(MCP251XFD_REG_FIFOSTA_FIFOCI_MASK, fifo_sta);
-
- return 0;
-}
-
-static inline int
-mcp251xfd_rx_tail_get_from_chip(const struct mcp251xfd_priv *priv,
- const struct mcp251xfd_rx_ring *ring,
- u8 *rx_tail)
-{
- u32 fifo_ua;
- int err;
-
- err = regmap_read(priv->map_reg, MCP251XFD_REG_FIFOUA(ring->fifo_nr),
- &fifo_ua);
- if (err)
- return err;
-
- fifo_ua -= ring->base - MCP251XFD_RAM_START;
- *rx_tail = fifo_ua / ring->obj_size;
-
- return 0;
-}
-
-static void
-mcp251xfd_tx_ring_init_tx_obj(const struct mcp251xfd_priv *priv,
- const struct mcp251xfd_tx_ring *ring,
- struct mcp251xfd_tx_obj *tx_obj,
- const u8 rts_buf_len,
- const u8 n)
-{
- struct spi_transfer *xfer;
- u16 addr;
-
- /* FIFO load */
- addr = mcp251xfd_get_tx_obj_addr(ring, n);
- if (priv->devtype_data.quirks & MCP251XFD_QUIRK_CRC_TX)
- mcp251xfd_spi_cmd_write_crc_set_addr(&tx_obj->buf.crc.cmd,
- addr);
- else
- mcp251xfd_spi_cmd_write_nocrc(&tx_obj->buf.nocrc.cmd,
- addr);
-
- xfer = &tx_obj->xfer[0];
- xfer->tx_buf = &tx_obj->buf;
- xfer->len = 0; /* actual len is assigned on the fly */
- xfer->cs_change = 1;
- xfer->cs_change_delay.value = 0;
- xfer->cs_change_delay.unit = SPI_DELAY_UNIT_NSECS;
-
- /* FIFO request to send */
- xfer = &tx_obj->xfer[1];
- xfer->tx_buf = &ring->rts_buf;
- xfer->len = rts_buf_len;
-
- /* SPI message */
- spi_message_init_with_transfers(&tx_obj->msg, tx_obj->xfer,
- ARRAY_SIZE(tx_obj->xfer));
-}
-
-static void mcp251xfd_ring_init(struct mcp251xfd_priv *priv)
-{
- struct mcp251xfd_tef_ring *tef_ring;
- struct mcp251xfd_tx_ring *tx_ring;
- struct mcp251xfd_rx_ring *rx_ring, *prev_rx_ring = NULL;
- struct mcp251xfd_tx_obj *tx_obj;
- struct spi_transfer *xfer;
- u32 val;
- u16 addr;
- u8 len;
- int i, j;
-
- netdev_reset_queue(priv->ndev);
-
- /* TEF */
- tef_ring = priv->tef;
- tef_ring->head = 0;
- tef_ring->tail = 0;
-
- /* FIFO increment TEF tail pointer */
- addr = MCP251XFD_REG_TEFCON;
- val = MCP251XFD_REG_TEFCON_UINC;
- len = mcp251xfd_cmd_prepare_write_reg(priv, &tef_ring->uinc_buf,
- addr, val, val);
-
- for (j = 0; j < ARRAY_SIZE(tef_ring->uinc_xfer); j++) {
- xfer = &tef_ring->uinc_xfer[j];
- xfer->tx_buf = &tef_ring->uinc_buf;
- xfer->len = len;
- xfer->cs_change = 1;
- xfer->cs_change_delay.value = 0;
- xfer->cs_change_delay.unit = SPI_DELAY_UNIT_NSECS;
- }
-
- /* "cs_change == 1" on the last transfer results in an active
- * chip select after the complete SPI message. This causes the
- * controller to interpret the next register access as
- * data. Set "cs_change" of the last transfer to "0" to
- * properly deactivate the chip select at the end of the
- * message.
- */
- xfer->cs_change = 0;
-
- /* TX */
- tx_ring = priv->tx;
- tx_ring->head = 0;
- tx_ring->tail = 0;
- tx_ring->base = mcp251xfd_get_tef_obj_addr(tx_ring->obj_num);
-
- /* FIFO request to send */
- addr = MCP251XFD_REG_FIFOCON(MCP251XFD_TX_FIFO);
- val = MCP251XFD_REG_FIFOCON_TXREQ | MCP251XFD_REG_FIFOCON_UINC;
- len = mcp251xfd_cmd_prepare_write_reg(priv, &tx_ring->rts_buf,
- addr, val, val);
-
- mcp251xfd_for_each_tx_obj(tx_ring, tx_obj, i)
- mcp251xfd_tx_ring_init_tx_obj(priv, tx_ring, tx_obj, len, i);
-
- /* RX */
- mcp251xfd_for_each_rx_ring(priv, rx_ring, i) {
- rx_ring->head = 0;
- rx_ring->tail = 0;
- rx_ring->nr = i;
- rx_ring->fifo_nr = MCP251XFD_RX_FIFO(i);
-
- if (!prev_rx_ring)
- rx_ring->base =
- mcp251xfd_get_tx_obj_addr(tx_ring,
- tx_ring->obj_num);
- else
- rx_ring->base = prev_rx_ring->base +
- prev_rx_ring->obj_size *
- prev_rx_ring->obj_num;
-
- prev_rx_ring = rx_ring;
-
- /* FIFO increment RX tail pointer */
- addr = MCP251XFD_REG_FIFOCON(rx_ring->fifo_nr);
- val = MCP251XFD_REG_FIFOCON_UINC;
- len = mcp251xfd_cmd_prepare_write_reg(priv, &rx_ring->uinc_buf,
- addr, val, val);
-
- for (j = 0; j < ARRAY_SIZE(rx_ring->uinc_xfer); j++) {
- xfer = &rx_ring->uinc_xfer[j];
- xfer->tx_buf = &rx_ring->uinc_buf;
- xfer->len = len;
- xfer->cs_change = 1;
- xfer->cs_change_delay.value = 0;
- xfer->cs_change_delay.unit = SPI_DELAY_UNIT_NSECS;
- }
-
- /* "cs_change == 1" on the last transfer results in an
- * active chip select after the complete SPI
- * message. This causes the controller to interpret
- * the next register access as data. Set "cs_change"
- * of the last transfer to "0" to properly deactivate
- * the chip select at the end of the message.
- */
- xfer->cs_change = 0;
- }
-}
-
-static void mcp251xfd_ring_free(struct mcp251xfd_priv *priv)
-{
- int i;
-
- for (i = ARRAY_SIZE(priv->rx) - 1; i >= 0; i--) {
- kfree(priv->rx[i]);
- priv->rx[i] = NULL;
- }
-}
-
-static int mcp251xfd_ring_alloc(struct mcp251xfd_priv *priv)
-{
- struct mcp251xfd_tx_ring *tx_ring;
- struct mcp251xfd_rx_ring *rx_ring;
- int tef_obj_size, tx_obj_size, rx_obj_size;
- int tx_obj_num;
- int ram_free, i;
-
- tef_obj_size = sizeof(struct mcp251xfd_hw_tef_obj);
- /* listen-only mode works like FD mode */
- if (priv->can.ctrlmode & (CAN_CTRLMODE_LISTENONLY | CAN_CTRLMODE_FD)) {
- tx_obj_num = MCP251XFD_TX_OBJ_NUM_CANFD;
- tx_obj_size = sizeof(struct mcp251xfd_hw_tx_obj_canfd);
- rx_obj_size = sizeof(struct mcp251xfd_hw_rx_obj_canfd);
- } else {
- tx_obj_num = MCP251XFD_TX_OBJ_NUM_CAN;
- tx_obj_size = sizeof(struct mcp251xfd_hw_tx_obj_can);
- rx_obj_size = sizeof(struct mcp251xfd_hw_rx_obj_can);
- }
-
- tx_ring = priv->tx;
- tx_ring->obj_num = tx_obj_num;
- tx_ring->obj_size = tx_obj_size;
-
- ram_free = MCP251XFD_RAM_SIZE - tx_obj_num *
- (tef_obj_size + tx_obj_size);
-
- for (i = 0;
- i < ARRAY_SIZE(priv->rx) && ram_free >= rx_obj_size;
- i++) {
- int rx_obj_num;
-
- rx_obj_num = ram_free / rx_obj_size;
- rx_obj_num = min(1 << (fls(rx_obj_num) - 1),
- MCP251XFD_RX_OBJ_NUM_MAX);
-
- rx_ring = kzalloc(sizeof(*rx_ring) + rx_obj_size * rx_obj_num,
- GFP_KERNEL);
- if (!rx_ring) {
- mcp251xfd_ring_free(priv);
- return -ENOMEM;
- }
- rx_ring->obj_num = rx_obj_num;
- rx_ring->obj_size = rx_obj_size;
- priv->rx[i] = rx_ring;
-
- ram_free -= rx_ring->obj_num * rx_ring->obj_size;
- }
- priv->rx_ring_num = i;
-
- netdev_dbg(priv->ndev,
- "FIFO setup: TEF: %d*%d bytes = %d bytes, TX: %d*%d bytes = %d bytes\n",
- tx_obj_num, tef_obj_size, tef_obj_size * tx_obj_num,
- tx_obj_num, tx_obj_size, tx_obj_size * tx_obj_num);
-
- mcp251xfd_for_each_rx_ring(priv, rx_ring, i) {
- netdev_dbg(priv->ndev,
- "FIFO setup: RX-%d: %d*%d bytes = %d bytes\n",
- i, rx_ring->obj_num, rx_ring->obj_size,
- rx_ring->obj_size * rx_ring->obj_num);
- }
-
- netdev_dbg(priv->ndev,
- "FIFO setup: free: %d bytes\n",
- ram_free);
-
- return 0;
-}
-
static inline int
mcp251xfd_chip_get_mode(const struct mcp251xfd_priv *priv, u8 *mode)
{
@@ -838,108 +512,6 @@ static int mcp251xfd_chip_rx_int_disable(const struct mcp251xfd_priv *priv)
return regmap_write(priv->map_reg, MCP251XFD_REG_IOCON, val);
}
-static int
-mcp251xfd_chip_rx_fifo_init_one(const struct mcp251xfd_priv *priv,
- const struct mcp251xfd_rx_ring *ring)
-{
- u32 fifo_con;
-
- /* Enable RXOVIE on _all_ RX FIFOs, not just the last one.
- *
- * FIFOs hit by a RX MAB overflow and RXOVIE enabled will
- * generate a RXOVIF, use this to properly detect RX MAB
- * overflows.
- */
- fifo_con = FIELD_PREP(MCP251XFD_REG_FIFOCON_FSIZE_MASK,
- ring->obj_num - 1) |
- MCP251XFD_REG_FIFOCON_RXTSEN |
- MCP251XFD_REG_FIFOCON_RXOVIE |
- MCP251XFD_REG_FIFOCON_TFNRFNIE;
-
- if (priv->can.ctrlmode & (CAN_CTRLMODE_LISTENONLY | CAN_CTRLMODE_FD))
- fifo_con |= FIELD_PREP(MCP251XFD_REG_FIFOCON_PLSIZE_MASK,
- MCP251XFD_REG_FIFOCON_PLSIZE_64);
- else
- fifo_con |= FIELD_PREP(MCP251XFD_REG_FIFOCON_PLSIZE_MASK,
- MCP251XFD_REG_FIFOCON_PLSIZE_8);
-
- return regmap_write(priv->map_reg,
- MCP251XFD_REG_FIFOCON(ring->fifo_nr), fifo_con);
-}
-
-static int
-mcp251xfd_chip_rx_filter_init_one(const struct mcp251xfd_priv *priv,
- const struct mcp251xfd_rx_ring *ring)
-{
- u32 fltcon;
-
- fltcon = MCP251XFD_REG_FLTCON_FLTEN(ring->nr) |
- MCP251XFD_REG_FLTCON_FBP(ring->nr, ring->fifo_nr);
-
- return regmap_update_bits(priv->map_reg,
- MCP251XFD_REG_FLTCON(ring->nr >> 2),
- MCP251XFD_REG_FLTCON_FLT_MASK(ring->nr),
- fltcon);
-}
-
-static int mcp251xfd_chip_fifo_init(const struct mcp251xfd_priv *priv)
-{
- const struct mcp251xfd_tx_ring *tx_ring = priv->tx;
- const struct mcp251xfd_rx_ring *rx_ring;
- u32 val;
- int err, n;
-
- /* TEF */
- val = FIELD_PREP(MCP251XFD_REG_TEFCON_FSIZE_MASK,
- tx_ring->obj_num - 1) |
- MCP251XFD_REG_TEFCON_TEFTSEN |
- MCP251XFD_REG_TEFCON_TEFOVIE |
- MCP251XFD_REG_TEFCON_TEFNEIE;
-
- err = regmap_write(priv->map_reg, MCP251XFD_REG_TEFCON, val);
- if (err)
- return err;
-
- /* FIFO 1 - TX */
- val = FIELD_PREP(MCP251XFD_REG_FIFOCON_FSIZE_MASK,
- tx_ring->obj_num - 1) |
- MCP251XFD_REG_FIFOCON_TXEN |
- MCP251XFD_REG_FIFOCON_TXATIE;
-
- if (priv->can.ctrlmode & (CAN_CTRLMODE_LISTENONLY | CAN_CTRLMODE_FD))
- val |= FIELD_PREP(MCP251XFD_REG_FIFOCON_PLSIZE_MASK,
- MCP251XFD_REG_FIFOCON_PLSIZE_64);
- else
- val |= FIELD_PREP(MCP251XFD_REG_FIFOCON_PLSIZE_MASK,
- MCP251XFD_REG_FIFOCON_PLSIZE_8);
-
- if (priv->can.ctrlmode & CAN_CTRLMODE_ONE_SHOT)
- val |= FIELD_PREP(MCP251XFD_REG_FIFOCON_TXAT_MASK,
- MCP251XFD_REG_FIFOCON_TXAT_ONE_SHOT);
- else
- val |= FIELD_PREP(MCP251XFD_REG_FIFOCON_TXAT_MASK,
- MCP251XFD_REG_FIFOCON_TXAT_UNLIMITED);
-
- err = regmap_write(priv->map_reg,
- MCP251XFD_REG_FIFOCON(MCP251XFD_TX_FIFO),
- val);
- if (err)
- return err;
-
- /* RX FIFOs */
- mcp251xfd_for_each_rx_ring(priv, rx_ring, n) {
- err = mcp251xfd_chip_rx_fifo_init_one(priv, rx_ring);
- if (err)
- return err;
-
- err = mcp251xfd_chip_rx_filter_init_one(priv, rx_ring);
- if (err)
- return err;
- }
-
- return 0;
-}
-
static int mcp251xfd_chip_ecc_init(struct mcp251xfd_priv *priv)
{
struct mcp251xfd_ecc *ecc = &priv->ecc;
@@ -968,18 +540,10 @@ static int mcp251xfd_chip_ecc_init(struct mcp251xfd_priv *priv)
return err;
}
-static inline void mcp251xfd_ecc_tefif_successful(struct mcp251xfd_priv *priv)
-{
- struct mcp251xfd_ecc *ecc = &priv->ecc;
-
- ecc->ecc_stat = 0;
-}
-
static u8 mcp251xfd_get_normal_mode(const struct mcp251xfd_priv *priv)
{
u8 mode;
-
if (priv->can.ctrlmode & CAN_CTRLMODE_LOOPBACK)
mode = MCP251XFD_REG_CON_MODE_INT_LOOPBACK;
else if (priv->can.ctrlmode & CAN_CTRLMODE_LISTENONLY)
@@ -1186,433 +750,6 @@ static int mcp251xfd_get_berr_counter(const struct net_device *ndev,
return __mcp251xfd_get_berr_counter(ndev, bec);
}
-static int mcp251xfd_check_tef_tail(const struct mcp251xfd_priv *priv)
-{
- u8 tef_tail_chip, tef_tail;
- int err;
-
- if (!IS_ENABLED(CONFIG_CAN_MCP251XFD_SANITY))
- return 0;
-
- err = mcp251xfd_tef_tail_get_from_chip(priv, &tef_tail_chip);
- if (err)
- return err;
-
- tef_tail = mcp251xfd_get_tef_tail(priv);
- if (tef_tail_chip != tef_tail) {
- netdev_err(priv->ndev,
- "TEF tail of chip (0x%02x) and ours (0x%08x) inconsistent.\n",
- tef_tail_chip, tef_tail);
- return -EILSEQ;
- }
-
- return 0;
-}
-
-static int
-mcp251xfd_check_rx_tail(const struct mcp251xfd_priv *priv,
- const struct mcp251xfd_rx_ring *ring)
-{
- u8 rx_tail_chip, rx_tail;
- int err;
-
- if (!IS_ENABLED(CONFIG_CAN_MCP251XFD_SANITY))
- return 0;
-
- err = mcp251xfd_rx_tail_get_from_chip(priv, ring, &rx_tail_chip);
- if (err)
- return err;
-
- rx_tail = mcp251xfd_get_rx_tail(ring);
- if (rx_tail_chip != rx_tail) {
- netdev_err(priv->ndev,
- "RX tail of chip (%d) and ours (%d) inconsistent.\n",
- rx_tail_chip, rx_tail);
- return -EILSEQ;
- }
-
- return 0;
-}
-
-static int
-mcp251xfd_handle_tefif_recover(const struct mcp251xfd_priv *priv, const u32 seq)
-{
- const struct mcp251xfd_tx_ring *tx_ring = priv->tx;
- u32 tef_sta;
- int err;
-
- err = regmap_read(priv->map_reg, MCP251XFD_REG_TEFSTA, &tef_sta);
- if (err)
- return err;
-
- if (tef_sta & MCP251XFD_REG_TEFSTA_TEFOVIF) {
- netdev_err(priv->ndev,
- "Transmit Event FIFO buffer overflow.\n");
- return -ENOBUFS;
- }
-
- netdev_info(priv->ndev,
- "Transmit Event FIFO buffer %s. (seq=0x%08x, tef_tail=0x%08x, tef_head=0x%08x, tx_head=0x%08x).\n",
- tef_sta & MCP251XFD_REG_TEFSTA_TEFFIF ?
- "full" : tef_sta & MCP251XFD_REG_TEFSTA_TEFNEIF ?
- "not empty" : "empty",
- seq, priv->tef->tail, priv->tef->head, tx_ring->head);
-
- /* The Sequence Number in the TEF doesn't match our tef_tail. */
- return -EAGAIN;
-}
-
-static int
-mcp251xfd_handle_tefif_one(struct mcp251xfd_priv *priv,
- const struct mcp251xfd_hw_tef_obj *hw_tef_obj,
- unsigned int *frame_len_ptr)
-{
- struct net_device_stats *stats = &priv->ndev->stats;
- struct sk_buff *skb;
- u32 seq, seq_masked, tef_tail_masked, tef_tail;
-
- seq = FIELD_GET(MCP251XFD_OBJ_FLAGS_SEQ_MCP2518FD_MASK,
- hw_tef_obj->flags);
-
- /* Use the MCP2517FD mask on the MCP2518FD, too. We only
- * compare 7 bits, this should be enough to detect
- * net-yet-completed, i.e. old TEF objects.
- */
- seq_masked = seq &
- field_mask(MCP251XFD_OBJ_FLAGS_SEQ_MCP2517FD_MASK);
- tef_tail_masked = priv->tef->tail &
- field_mask(MCP251XFD_OBJ_FLAGS_SEQ_MCP2517FD_MASK);
- if (seq_masked != tef_tail_masked)
- return mcp251xfd_handle_tefif_recover(priv, seq);
-
- tef_tail = mcp251xfd_get_tef_tail(priv);
- skb = priv->can.echo_skb[tef_tail];
- if (skb)
- mcp251xfd_skb_set_timestamp(priv, skb, hw_tef_obj->ts);
- stats->tx_bytes +=
- can_rx_offload_get_echo_skb(&priv->offload,
- tef_tail, hw_tef_obj->ts,
- frame_len_ptr);
- stats->tx_packets++;
- priv->tef->tail++;
-
- return 0;
-}
-
-static int mcp251xfd_tef_ring_update(struct mcp251xfd_priv *priv)
-{
- const struct mcp251xfd_tx_ring *tx_ring = priv->tx;
- unsigned int new_head;
- u8 chip_tx_tail;
- int err;
-
- err = mcp251xfd_tx_tail_get_from_chip(priv, &chip_tx_tail);
- if (err)
- return err;
-
- /* chip_tx_tail, is the next TX-Object send by the HW.
- * The new TEF head must be >= the old head, ...
- */
- new_head = round_down(priv->tef->head, tx_ring->obj_num) + chip_tx_tail;
- if (new_head <= priv->tef->head)
- new_head += tx_ring->obj_num;
-
- /* ... but it cannot exceed the TX head. */
- priv->tef->head = min(new_head, tx_ring->head);
-
- return mcp251xfd_check_tef_tail(priv);
-}
-
-static inline int
-mcp251xfd_tef_obj_read(const struct mcp251xfd_priv *priv,
- struct mcp251xfd_hw_tef_obj *hw_tef_obj,
- const u8 offset, const u8 len)
-{
- const struct mcp251xfd_tx_ring *tx_ring = priv->tx;
- const int val_bytes = regmap_get_val_bytes(priv->map_rx);
-
- if (IS_ENABLED(CONFIG_CAN_MCP251XFD_SANITY) &&
- (offset > tx_ring->obj_num ||
- len > tx_ring->obj_num ||
- offset + len > tx_ring->obj_num)) {
- netdev_err(priv->ndev,
- "Trying to read to many TEF objects (max=%d, offset=%d, len=%d).\n",
- tx_ring->obj_num, offset, len);
- return -ERANGE;
- }
-
- return regmap_bulk_read(priv->map_rx,
- mcp251xfd_get_tef_obj_addr(offset),
- hw_tef_obj,
- sizeof(*hw_tef_obj) / val_bytes * len);
-}
-
-static int mcp251xfd_handle_tefif(struct mcp251xfd_priv *priv)
-{
- struct mcp251xfd_hw_tef_obj hw_tef_obj[MCP251XFD_TX_OBJ_NUM_MAX];
- unsigned int total_frame_len = 0;
- u8 tef_tail, len, l;
- int err, i;
-
- err = mcp251xfd_tef_ring_update(priv);
- if (err)
- return err;
-
- tef_tail = mcp251xfd_get_tef_tail(priv);
- len = mcp251xfd_get_tef_len(priv);
- l = mcp251xfd_get_tef_linear_len(priv);
- err = mcp251xfd_tef_obj_read(priv, hw_tef_obj, tef_tail, l);
- if (err)
- return err;
-
- if (l < len) {
- err = mcp251xfd_tef_obj_read(priv, &hw_tef_obj[l], 0, len - l);
- if (err)
- return err;
- }
-
- for (i = 0; i < len; i++) {
- unsigned int frame_len = 0;
-
- err = mcp251xfd_handle_tefif_one(priv, &hw_tef_obj[i], &frame_len);
- /* -EAGAIN means the Sequence Number in the TEF
- * doesn't match our tef_tail. This can happen if we
- * read the TEF objects too early. Leave loop let the
- * interrupt handler call us again.
- */
- if (err == -EAGAIN)
- goto out_netif_wake_queue;
- if (err)
- return err;
-
- total_frame_len += frame_len;
- }
-
- out_netif_wake_queue:
- len = i; /* number of handled goods TEFs */
- if (len) {
- struct mcp251xfd_tef_ring *ring = priv->tef;
- struct mcp251xfd_tx_ring *tx_ring = priv->tx;
- int offset;
-
- /* Increment the TEF FIFO tail pointer 'len' times in
- * a single SPI message.
- *
- * Note:
- * Calculate offset, so that the SPI transfer ends on
- * the last message of the uinc_xfer array, which has
- * "cs_change == 0", to properly deactivate the chip
- * select.
- */
- offset = ARRAY_SIZE(ring->uinc_xfer) - len;
- err = spi_sync_transfer(priv->spi,
- ring->uinc_xfer + offset, len);
- if (err)
- return err;
-
- tx_ring->tail += len;
- netdev_completed_queue(priv->ndev, len, total_frame_len);
-
- err = mcp251xfd_check_tef_tail(priv);
- if (err)
- return err;
- }
-
- mcp251xfd_ecc_tefif_successful(priv);
-
- if (mcp251xfd_get_tx_free(priv->tx)) {
- /* Make sure that anybody stopping the queue after
- * this sees the new tx_ring->tail.
- */
- smp_mb();
- netif_wake_queue(priv->ndev);
- }
-
- return 0;
-}
-
-static int
-mcp251xfd_rx_ring_update(const struct mcp251xfd_priv *priv,
- struct mcp251xfd_rx_ring *ring)
-{
- u32 new_head;
- u8 chip_rx_head;
- int err;
-
- err = mcp251xfd_rx_head_get_from_chip(priv, ring, &chip_rx_head);
- if (err)
- return err;
-
- /* chip_rx_head, is the next RX-Object filled by the HW.
- * The new RX head must be >= the old head.
- */
- new_head = round_down(ring->head, ring->obj_num) + chip_rx_head;
- if (new_head <= ring->head)
- new_head += ring->obj_num;
-
- ring->head = new_head;
-
- return mcp251xfd_check_rx_tail(priv, ring);
-}
-
-static void
-mcp251xfd_hw_rx_obj_to_skb(const struct mcp251xfd_priv *priv,
- const struct mcp251xfd_hw_rx_obj_canfd *hw_rx_obj,
- struct sk_buff *skb)
-{
- struct canfd_frame *cfd = (struct canfd_frame *)skb->data;
- u8 dlc;
-
- if (hw_rx_obj->flags & MCP251XFD_OBJ_FLAGS_IDE) {
- u32 sid, eid;
-
- eid = FIELD_GET(MCP251XFD_OBJ_ID_EID_MASK, hw_rx_obj->id);
- sid = FIELD_GET(MCP251XFD_OBJ_ID_SID_MASK, hw_rx_obj->id);
-
- cfd->can_id = CAN_EFF_FLAG |
- FIELD_PREP(MCP251XFD_REG_FRAME_EFF_EID_MASK, eid) |
- FIELD_PREP(MCP251XFD_REG_FRAME_EFF_SID_MASK, sid);
- } else {
- cfd->can_id = FIELD_GET(MCP251XFD_OBJ_ID_SID_MASK,
- hw_rx_obj->id);
- }
-
- dlc = FIELD_GET(MCP251XFD_OBJ_FLAGS_DLC_MASK, hw_rx_obj->flags);
-
- /* CANFD */
- if (hw_rx_obj->flags & MCP251XFD_OBJ_FLAGS_FDF) {
-
- if (hw_rx_obj->flags & MCP251XFD_OBJ_FLAGS_ESI)
- cfd->flags |= CANFD_ESI;
-
- if (hw_rx_obj->flags & MCP251XFD_OBJ_FLAGS_BRS)
- cfd->flags |= CANFD_BRS;
-
- cfd->len = can_fd_dlc2len(dlc);
- } else {
- if (hw_rx_obj->flags & MCP251XFD_OBJ_FLAGS_RTR)
- cfd->can_id |= CAN_RTR_FLAG;
-
- can_frame_set_cc_len((struct can_frame *)cfd, dlc,
- priv->can.ctrlmode);
- }
-
- if (!(hw_rx_obj->flags & MCP251XFD_OBJ_FLAGS_RTR))
- memcpy(cfd->data, hw_rx_obj->data, cfd->len);
-
- mcp251xfd_skb_set_timestamp(priv, skb, hw_rx_obj->ts);
-}
-
-static int
-mcp251xfd_handle_rxif_one(struct mcp251xfd_priv *priv,
- struct mcp251xfd_rx_ring *ring,
- const struct mcp251xfd_hw_rx_obj_canfd *hw_rx_obj)
-{
- struct net_device_stats *stats = &priv->ndev->stats;
- struct sk_buff *skb;
- struct canfd_frame *cfd;
- int err;
-
- if (hw_rx_obj->flags & MCP251XFD_OBJ_FLAGS_FDF)
- skb = alloc_canfd_skb(priv->ndev, &cfd);
- else
- skb = alloc_can_skb(priv->ndev, (struct can_frame **)&cfd);
-
- if (!skb) {
- stats->rx_dropped++;
- return 0;
- }
-
- mcp251xfd_hw_rx_obj_to_skb(priv, hw_rx_obj, skb);
- err = can_rx_offload_queue_sorted(&priv->offload, skb, hw_rx_obj->ts);
- if (err)
- stats->rx_fifo_errors++;
-
- return 0;
-}
-
-static inline int
-mcp251xfd_rx_obj_read(const struct mcp251xfd_priv *priv,
- const struct mcp251xfd_rx_ring *ring,
- struct mcp251xfd_hw_rx_obj_canfd *hw_rx_obj,
- const u8 offset, const u8 len)
-{
- const int val_bytes = regmap_get_val_bytes(priv->map_rx);
- int err;
-
- err = regmap_bulk_read(priv->map_rx,
- mcp251xfd_get_rx_obj_addr(ring, offset),
- hw_rx_obj,
- len * ring->obj_size / val_bytes);
-
- return err;
-}
-
-static int
-mcp251xfd_handle_rxif_ring(struct mcp251xfd_priv *priv,
- struct mcp251xfd_rx_ring *ring)
-{
- struct mcp251xfd_hw_rx_obj_canfd *hw_rx_obj = ring->obj;
- u8 rx_tail, len;
- int err, i;
-
- err = mcp251xfd_rx_ring_update(priv, ring);
- if (err)
- return err;
-
- while ((len = mcp251xfd_get_rx_linear_len(ring))) {
- int offset;
-
- rx_tail = mcp251xfd_get_rx_tail(ring);
-
- err = mcp251xfd_rx_obj_read(priv, ring, hw_rx_obj,
- rx_tail, len);
- if (err)
- return err;
-
- for (i = 0; i < len; i++) {
- err = mcp251xfd_handle_rxif_one(priv, ring,
- (void *)hw_rx_obj +
- i * ring->obj_size);
- if (err)
- return err;
- }
-
- /* Increment the RX FIFO tail pointer 'len' times in a
- * single SPI message.
- *
- * Note:
- * Calculate offset, so that the SPI transfer ends on
- * the last message of the uinc_xfer array, which has
- * "cs_change == 0", to properly deactivate the chip
- * select.
- */
- offset = ARRAY_SIZE(ring->uinc_xfer) - len;
- err = spi_sync_transfer(priv->spi,
- ring->uinc_xfer + offset, len);
- if (err)
- return err;
-
- ring->tail += len;
- }
-
- return 0;
-}
-
-static int mcp251xfd_handle_rxif(struct mcp251xfd_priv *priv)
-{
- struct mcp251xfd_rx_ring *ring;
- int err, n;
-
- mcp251xfd_for_each_rx_ring(priv, ring, n) {
- err = mcp251xfd_handle_rxif_ring(priv, ring);
- if (err)
- return err;
- }
-
- return 0;
-}
-
static struct sk_buff *
mcp251xfd_alloc_can_err_skb(struct mcp251xfd_priv *priv,
struct can_frame **cf, u32 *timestamp)
@@ -1653,12 +790,15 @@ static int mcp251xfd_handle_rxovif(struct mcp251xfd_priv *priv)
/* If SERRIF is active, there was a RX MAB overflow. */
if (priv->regs_status.intf & MCP251XFD_REG_INT_SERRIF) {
- netdev_info(priv->ndev,
- "RX-%d: MAB overflow detected.\n",
- ring->nr);
+ if (net_ratelimit())
+ netdev_dbg(priv->ndev,
+ "RX-%d: MAB overflow detected.\n",
+ ring->nr);
} else {
- netdev_info(priv->ndev,
- "RX-%d: FIFO overflow.\n", ring->nr);
+ if (net_ratelimit())
+ netdev_dbg(priv->ndev,
+ "RX-%d: FIFO overflow.\n",
+ ring->nr);
}
err = regmap_update_bits(priv->map_reg,
@@ -2311,212 +1451,23 @@ static irqreturn_t mcp251xfd_irq(int irq, void *dev_id)
return handled;
}
-static inline struct
-mcp251xfd_tx_obj *mcp251xfd_get_tx_obj_next(struct mcp251xfd_tx_ring *tx_ring)
-{
- u8 tx_head;
-
- tx_head = mcp251xfd_get_tx_head(tx_ring);
-
- return &tx_ring->obj[tx_head];
-}
-
-static void
-mcp251xfd_tx_obj_from_skb(const struct mcp251xfd_priv *priv,
- struct mcp251xfd_tx_obj *tx_obj,
- const struct sk_buff *skb,
- unsigned int seq)
-{
- const struct canfd_frame *cfd = (struct canfd_frame *)skb->data;
- struct mcp251xfd_hw_tx_obj_raw *hw_tx_obj;
- union mcp251xfd_tx_obj_load_buf *load_buf;
- u8 dlc;
- u32 id, flags;
- int len_sanitized = 0, len;
-
- if (cfd->can_id & CAN_EFF_FLAG) {
- u32 sid, eid;
-
- sid = FIELD_GET(MCP251XFD_REG_FRAME_EFF_SID_MASK, cfd->can_id);
- eid = FIELD_GET(MCP251XFD_REG_FRAME_EFF_EID_MASK, cfd->can_id);
-
- id = FIELD_PREP(MCP251XFD_OBJ_ID_EID_MASK, eid) |
- FIELD_PREP(MCP251XFD_OBJ_ID_SID_MASK, sid);
-
- flags = MCP251XFD_OBJ_FLAGS_IDE;
- } else {
- id = FIELD_PREP(MCP251XFD_OBJ_ID_SID_MASK, cfd->can_id);
- flags = 0;
- }
-
- /* Use the MCP2518FD mask even on the MCP2517FD. It doesn't
- * harm, only the lower 7 bits will be transferred into the
- * TEF object.
- */
- flags |= FIELD_PREP(MCP251XFD_OBJ_FLAGS_SEQ_MCP2518FD_MASK, seq);
-
- if (cfd->can_id & CAN_RTR_FLAG)
- flags |= MCP251XFD_OBJ_FLAGS_RTR;
- else
- len_sanitized = canfd_sanitize_len(cfd->len);
-
- /* CANFD */
- if (can_is_canfd_skb(skb)) {
- if (cfd->flags & CANFD_ESI)
- flags |= MCP251XFD_OBJ_FLAGS_ESI;
-
- flags |= MCP251XFD_OBJ_FLAGS_FDF;
-
- if (cfd->flags & CANFD_BRS)
- flags |= MCP251XFD_OBJ_FLAGS_BRS;
-
- dlc = can_fd_len2dlc(cfd->len);
- } else {
- dlc = can_get_cc_dlc((struct can_frame *)cfd,
- priv->can.ctrlmode);
- }
-
- flags |= FIELD_PREP(MCP251XFD_OBJ_FLAGS_DLC_MASK, dlc);
-
- load_buf = &tx_obj->buf;
- if (priv->devtype_data.quirks & MCP251XFD_QUIRK_CRC_TX)
- hw_tx_obj = &load_buf->crc.hw_tx_obj;
- else
- hw_tx_obj = &load_buf->nocrc.hw_tx_obj;
-
- put_unaligned_le32(id, &hw_tx_obj->id);
- put_unaligned_le32(flags, &hw_tx_obj->flags);
-
- /* Copy data */
- memcpy(hw_tx_obj->data, cfd->data, cfd->len);
-
- /* Clear unused data at end of CAN frame */
- if (MCP251XFD_SANITIZE_CAN && len_sanitized) {
- int pad_len;
-
- pad_len = len_sanitized - cfd->len;
- if (pad_len)
- memset(hw_tx_obj->data + cfd->len, 0x0, pad_len);
- }
-
- /* Number of bytes to be written into the RAM of the controller */
- len = sizeof(hw_tx_obj->id) + sizeof(hw_tx_obj->flags);
- if (MCP251XFD_SANITIZE_CAN)
- len += round_up(len_sanitized, sizeof(u32));
- else
- len += round_up(cfd->len, sizeof(u32));
-
- if (priv->devtype_data.quirks & MCP251XFD_QUIRK_CRC_TX) {
- u16 crc;
-
- mcp251xfd_spi_cmd_crc_set_len_in_ram(&load_buf->crc.cmd,
- len);
- /* CRC */
- len += sizeof(load_buf->crc.cmd);
- crc = mcp251xfd_crc16_compute(&load_buf->crc, len);
- put_unaligned_be16(crc, (void *)load_buf + len);
-
- /* Total length */
- len += sizeof(load_buf->crc.crc);
- } else {
- len += sizeof(load_buf->nocrc.cmd);
- }
-
- tx_obj->xfer[0].len = len;
-}
-
-static int mcp251xfd_tx_obj_write(const struct mcp251xfd_priv *priv,
- struct mcp251xfd_tx_obj *tx_obj)
-{
- return spi_async(priv->spi, &tx_obj->msg);
-}
-
-static bool mcp251xfd_tx_busy(const struct mcp251xfd_priv *priv,
- struct mcp251xfd_tx_ring *tx_ring)
-{
- if (mcp251xfd_get_tx_free(tx_ring) > 0)
- return false;
-
- netif_stop_queue(priv->ndev);
-
- /* Memory barrier before checking tx_free (head and tail) */
- smp_mb();
-
- if (mcp251xfd_get_tx_free(tx_ring) == 0) {
- netdev_dbg(priv->ndev,
- "Stopping tx-queue (tx_head=0x%08x, tx_tail=0x%08x, len=%d).\n",
- tx_ring->head, tx_ring->tail,
- tx_ring->head - tx_ring->tail);
-
- return true;
- }
-
- netif_start_queue(priv->ndev);
-
- return false;
-}
-
-static netdev_tx_t mcp251xfd_start_xmit(struct sk_buff *skb,
- struct net_device *ndev)
-{
- struct mcp251xfd_priv *priv = netdev_priv(ndev);
- struct mcp251xfd_tx_ring *tx_ring = priv->tx;
- struct mcp251xfd_tx_obj *tx_obj;
- unsigned int frame_len;
- u8 tx_head;
- int err;
-
- if (can_dropped_invalid_skb(ndev, skb))
- return NETDEV_TX_OK;
-
- if (mcp251xfd_tx_busy(priv, tx_ring))
- return NETDEV_TX_BUSY;
-
- tx_obj = mcp251xfd_get_tx_obj_next(tx_ring);
- mcp251xfd_tx_obj_from_skb(priv, tx_obj, skb, tx_ring->head);
-
- /* Stop queue if we occupy the complete TX FIFO */
- tx_head = mcp251xfd_get_tx_head(tx_ring);
- tx_ring->head++;
- if (mcp251xfd_get_tx_free(tx_ring) == 0)
- netif_stop_queue(ndev);
-
- frame_len = can_skb_get_frame_len(skb);
- err = can_put_echo_skb(skb, ndev, tx_head, frame_len);
- if (!err)
- netdev_sent_queue(priv->ndev, frame_len);
-
- err = mcp251xfd_tx_obj_write(priv, tx_obj);
- if (err)
- goto out_err;
-
- return NETDEV_TX_OK;
-
- out_err:
- netdev_err(priv->ndev, "ERROR in %s: %d\n", __func__, err);
-
- return NETDEV_TX_OK;
-}
-
static int mcp251xfd_open(struct net_device *ndev)
{
struct mcp251xfd_priv *priv = netdev_priv(ndev);
const struct spi_device *spi = priv->spi;
int err;
- err = pm_runtime_get_sync(ndev->dev.parent);
- if (err < 0) {
- pm_runtime_put_noidle(ndev->dev.parent);
- return err;
- }
-
err = open_candev(ndev);
if (err)
- goto out_pm_runtime_put;
+ return err;
+
+ err = pm_runtime_resume_and_get(ndev->dev.parent);
+ if (err)
+ goto out_close_candev;
err = mcp251xfd_ring_alloc(priv);
if (err)
- goto out_close_candev;
+ goto out_pm_runtime_put;
err = mcp251xfd_transceiver_enable(priv);
if (err)
@@ -2552,11 +1503,11 @@ static int mcp251xfd_open(struct net_device *ndev)
mcp251xfd_transceiver_disable(priv);
out_mcp251xfd_ring_free:
mcp251xfd_ring_free(priv);
- out_close_candev:
- close_candev(ndev);
out_pm_runtime_put:
mcp251xfd_chip_stop(priv, CAN_STATE_STOPPED);
pm_runtime_put(ndev->dev.parent);
+ out_close_candev:
+ close_candev(ndev);
return err;
}
@@ -2625,7 +1576,7 @@ static int mcp251xfd_register_chip_detect(struct mcp251xfd_priv *priv)
if (!mcp251xfd_is_251X(priv) &&
priv->devtype_data.model != devtype_data->model) {
netdev_info(ndev,
- "Detected %s, but firmware specifies a %s. Fixing up.",
+ "Detected %s, but firmware specifies a %s. Fixing up.\n",
__mcp251xfd_get_model_str(devtype_data->model),
mcp251xfd_get_model_str(priv));
}
@@ -2662,7 +1613,7 @@ static int mcp251xfd_register_check_rx_int(struct mcp251xfd_priv *priv)
return 0;
netdev_info(priv->ndev,
- "RX_INT active after softreset, disabling RX_INT support.");
+ "RX_INT active after softreset, disabling RX_INT support.\n");
devm_gpiod_put(&priv->spi->dev, priv->rx_int);
priv->rx_int = NULL;
diff --git a/drivers/net/can/spi/mcp251xfd/mcp251xfd-regmap.c b/drivers/net/can/spi/mcp251xfd/mcp251xfd-regmap.c
index 2974915..7b120c7 100644
--- a/drivers/net/can/spi/mcp251xfd/mcp251xfd-regmap.c
+++ b/drivers/net/can/spi/mcp251xfd/mcp251xfd-regmap.c
@@ -250,7 +250,6 @@ mcp251xfd_regmap_crc_read_check_crc(const struct mcp251xfd_map_buf_crc * const b
return 0;
}
-
static int
mcp251xfd_regmap_crc_read_one(struct mcp251xfd_priv *priv,
struct spi_message *msg, unsigned int data_len)
diff --git a/drivers/net/can/spi/mcp251xfd/mcp251xfd-ring.c b/drivers/net/can/spi/mcp251xfd/mcp251xfd-ring.c
new file mode 100644
index 0000000..92f9e9b
--- /dev/null
+++ b/drivers/net/can/spi/mcp251xfd/mcp251xfd-ring.c
@@ -0,0 +1,269 @@
+// SPDX-License-Identifier: GPL-2.0
+//
+// mcp251xfd - Microchip MCP251xFD Family CAN controller driver
+//
+// Copyright (c) 2019, 2020, 2021 Pengutronix,
+// Marc Kleine-Budde <kernel@pengutronix.de>
+//
+// Based on:
+//
+// CAN bus driver for Microchip 25XXFD CAN Controller with SPI Interface
+//
+// Copyright (c) 2019 Martin Sperl <kernel@martin.sperl.org>
+//
+
+#include <asm/unaligned.h>
+
+#include "mcp251xfd.h"
+
+static inline u8
+mcp251xfd_cmd_prepare_write_reg(const struct mcp251xfd_priv *priv,
+ union mcp251xfd_write_reg_buf *write_reg_buf,
+ const u16 reg, const u32 mask, const u32 val)
+{
+ u8 first_byte, last_byte, len;
+ u8 *data;
+ __le32 val_le32;
+
+ first_byte = mcp251xfd_first_byte_set(mask);
+ last_byte = mcp251xfd_last_byte_set(mask);
+ len = last_byte - first_byte + 1;
+
+ data = mcp251xfd_spi_cmd_write(priv, write_reg_buf, reg + first_byte);
+ val_le32 = cpu_to_le32(val >> BITS_PER_BYTE * first_byte);
+ memcpy(data, &val_le32, len);
+
+ if (priv->devtype_data.quirks & MCP251XFD_QUIRK_CRC_REG) {
+ u16 crc;
+
+ mcp251xfd_spi_cmd_crc_set_len_in_reg(&write_reg_buf->crc.cmd,
+ len);
+ /* CRC */
+ len += sizeof(write_reg_buf->crc.cmd);
+ crc = mcp251xfd_crc16_compute(&write_reg_buf->crc, len);
+ put_unaligned_be16(crc, (void *)write_reg_buf + len);
+
+ /* Total length */
+ len += sizeof(write_reg_buf->crc.crc);
+ } else {
+ len += sizeof(write_reg_buf->nocrc.cmd);
+ }
+
+ return len;
+}
+
+static void
+mcp251xfd_tx_ring_init_tx_obj(const struct mcp251xfd_priv *priv,
+ const struct mcp251xfd_tx_ring *ring,
+ struct mcp251xfd_tx_obj *tx_obj,
+ const u8 rts_buf_len,
+ const u8 n)
+{
+ struct spi_transfer *xfer;
+ u16 addr;
+
+ /* FIFO load */
+ addr = mcp251xfd_get_tx_obj_addr(ring, n);
+ if (priv->devtype_data.quirks & MCP251XFD_QUIRK_CRC_TX)
+ mcp251xfd_spi_cmd_write_crc_set_addr(&tx_obj->buf.crc.cmd,
+ addr);
+ else
+ mcp251xfd_spi_cmd_write_nocrc(&tx_obj->buf.nocrc.cmd,
+ addr);
+
+ xfer = &tx_obj->xfer[0];
+ xfer->tx_buf = &tx_obj->buf;
+ xfer->len = 0; /* actual len is assigned on the fly */
+ xfer->cs_change = 1;
+ xfer->cs_change_delay.value = 0;
+ xfer->cs_change_delay.unit = SPI_DELAY_UNIT_NSECS;
+
+ /* FIFO request to send */
+ xfer = &tx_obj->xfer[1];
+ xfer->tx_buf = &ring->rts_buf;
+ xfer->len = rts_buf_len;
+
+ /* SPI message */
+ spi_message_init_with_transfers(&tx_obj->msg, tx_obj->xfer,
+ ARRAY_SIZE(tx_obj->xfer));
+}
+
+void mcp251xfd_ring_init(struct mcp251xfd_priv *priv)
+{
+ struct mcp251xfd_tef_ring *tef_ring;
+ struct mcp251xfd_tx_ring *tx_ring;
+ struct mcp251xfd_rx_ring *rx_ring, *prev_rx_ring = NULL;
+ struct mcp251xfd_tx_obj *tx_obj;
+ struct spi_transfer *xfer;
+ u32 val;
+ u16 addr;
+ u8 len;
+ int i, j;
+
+ netdev_reset_queue(priv->ndev);
+
+ /* TEF */
+ tef_ring = priv->tef;
+ tef_ring->head = 0;
+ tef_ring->tail = 0;
+
+ /* FIFO increment TEF tail pointer */
+ addr = MCP251XFD_REG_TEFCON;
+ val = MCP251XFD_REG_TEFCON_UINC;
+ len = mcp251xfd_cmd_prepare_write_reg(priv, &tef_ring->uinc_buf,
+ addr, val, val);
+
+ for (j = 0; j < ARRAY_SIZE(tef_ring->uinc_xfer); j++) {
+ xfer = &tef_ring->uinc_xfer[j];
+ xfer->tx_buf = &tef_ring->uinc_buf;
+ xfer->len = len;
+ xfer->cs_change = 1;
+ xfer->cs_change_delay.value = 0;
+ xfer->cs_change_delay.unit = SPI_DELAY_UNIT_NSECS;
+ }
+
+ /* "cs_change == 1" on the last transfer results in an active
+ * chip select after the complete SPI message. This causes the
+ * controller to interpret the next register access as
+ * data. Set "cs_change" of the last transfer to "0" to
+ * properly deactivate the chip select at the end of the
+ * message.
+ */
+ xfer->cs_change = 0;
+
+ /* TX */
+ tx_ring = priv->tx;
+ tx_ring->head = 0;
+ tx_ring->tail = 0;
+ tx_ring->base = mcp251xfd_get_tef_obj_addr(tx_ring->obj_num);
+
+ /* FIFO request to send */
+ addr = MCP251XFD_REG_FIFOCON(MCP251XFD_TX_FIFO);
+ val = MCP251XFD_REG_FIFOCON_TXREQ | MCP251XFD_REG_FIFOCON_UINC;
+ len = mcp251xfd_cmd_prepare_write_reg(priv, &tx_ring->rts_buf,
+ addr, val, val);
+
+ mcp251xfd_for_each_tx_obj(tx_ring, tx_obj, i)
+ mcp251xfd_tx_ring_init_tx_obj(priv, tx_ring, tx_obj, len, i);
+
+ /* RX */
+ mcp251xfd_for_each_rx_ring(priv, rx_ring, i) {
+ rx_ring->head = 0;
+ rx_ring->tail = 0;
+ rx_ring->nr = i;
+ rx_ring->fifo_nr = MCP251XFD_RX_FIFO(i);
+
+ if (!prev_rx_ring)
+ rx_ring->base =
+ mcp251xfd_get_tx_obj_addr(tx_ring,
+ tx_ring->obj_num);
+ else
+ rx_ring->base = prev_rx_ring->base +
+ prev_rx_ring->obj_size *
+ prev_rx_ring->obj_num;
+
+ prev_rx_ring = rx_ring;
+
+ /* FIFO increment RX tail pointer */
+ addr = MCP251XFD_REG_FIFOCON(rx_ring->fifo_nr);
+ val = MCP251XFD_REG_FIFOCON_UINC;
+ len = mcp251xfd_cmd_prepare_write_reg(priv, &rx_ring->uinc_buf,
+ addr, val, val);
+
+ for (j = 0; j < ARRAY_SIZE(rx_ring->uinc_xfer); j++) {
+ xfer = &rx_ring->uinc_xfer[j];
+ xfer->tx_buf = &rx_ring->uinc_buf;
+ xfer->len = len;
+ xfer->cs_change = 1;
+ xfer->cs_change_delay.value = 0;
+ xfer->cs_change_delay.unit = SPI_DELAY_UNIT_NSECS;
+ }
+
+ /* "cs_change == 1" on the last transfer results in an
+ * active chip select after the complete SPI
+ * message. This causes the controller to interpret
+ * the next register access as data. Set "cs_change"
+ * of the last transfer to "0" to properly deactivate
+ * the chip select at the end of the message.
+ */
+ xfer->cs_change = 0;
+ }
+}
+
+void mcp251xfd_ring_free(struct mcp251xfd_priv *priv)
+{
+ int i;
+
+ for (i = ARRAY_SIZE(priv->rx) - 1; i >= 0; i--) {
+ kfree(priv->rx[i]);
+ priv->rx[i] = NULL;
+ }
+}
+
+int mcp251xfd_ring_alloc(struct mcp251xfd_priv *priv)
+{
+ struct mcp251xfd_tx_ring *tx_ring;
+ struct mcp251xfd_rx_ring *rx_ring;
+ int tef_obj_size, tx_obj_size, rx_obj_size;
+ int tx_obj_num;
+ int ram_free, i;
+
+ tef_obj_size = sizeof(struct mcp251xfd_hw_tef_obj);
+ if (mcp251xfd_is_fd_mode(priv)) {
+ tx_obj_num = MCP251XFD_TX_OBJ_NUM_CANFD;
+ tx_obj_size = sizeof(struct mcp251xfd_hw_tx_obj_canfd);
+ rx_obj_size = sizeof(struct mcp251xfd_hw_rx_obj_canfd);
+ } else {
+ tx_obj_num = MCP251XFD_TX_OBJ_NUM_CAN;
+ tx_obj_size = sizeof(struct mcp251xfd_hw_tx_obj_can);
+ rx_obj_size = sizeof(struct mcp251xfd_hw_rx_obj_can);
+ }
+
+ tx_ring = priv->tx;
+ tx_ring->obj_num = tx_obj_num;
+ tx_ring->obj_size = tx_obj_size;
+
+ ram_free = MCP251XFD_RAM_SIZE - tx_obj_num *
+ (tef_obj_size + tx_obj_size);
+
+ for (i = 0;
+ i < ARRAY_SIZE(priv->rx) && ram_free >= rx_obj_size;
+ i++) {
+ int rx_obj_num;
+
+ rx_obj_num = ram_free / rx_obj_size;
+ rx_obj_num = min(1 << (fls(rx_obj_num) - 1),
+ MCP251XFD_RX_OBJ_NUM_MAX);
+
+ rx_ring = kzalloc(sizeof(*rx_ring) + rx_obj_size * rx_obj_num,
+ GFP_KERNEL);
+ if (!rx_ring) {
+ mcp251xfd_ring_free(priv);
+ return -ENOMEM;
+ }
+ rx_ring->obj_num = rx_obj_num;
+ rx_ring->obj_size = rx_obj_size;
+ priv->rx[i] = rx_ring;
+
+ ram_free -= rx_ring->obj_num * rx_ring->obj_size;
+ }
+ priv->rx_ring_num = i;
+
+ netdev_dbg(priv->ndev,
+ "FIFO setup: TEF: %d*%d bytes = %d bytes, TX: %d*%d bytes = %d bytes\n",
+ tx_obj_num, tef_obj_size, tef_obj_size * tx_obj_num,
+ tx_obj_num, tx_obj_size, tx_obj_size * tx_obj_num);
+
+ mcp251xfd_for_each_rx_ring(priv, rx_ring, i) {
+ netdev_dbg(priv->ndev,
+ "FIFO setup: RX-%d: %d*%d bytes = %d bytes\n",
+ i, rx_ring->obj_num, rx_ring->obj_size,
+ rx_ring->obj_size * rx_ring->obj_num);
+ }
+
+ netdev_dbg(priv->ndev,
+ "FIFO setup: free: %d bytes\n",
+ ram_free);
+
+ return 0;
+}
diff --git a/drivers/net/can/spi/mcp251xfd/mcp251xfd-rx.c b/drivers/net/can/spi/mcp251xfd/mcp251xfd-rx.c
new file mode 100644
index 0000000..63f2526
--- /dev/null
+++ b/drivers/net/can/spi/mcp251xfd/mcp251xfd-rx.c
@@ -0,0 +1,260 @@
+// SPDX-License-Identifier: GPL-2.0
+//
+// mcp251xfd - Microchip MCP251xFD Family CAN controller driver
+//
+// Copyright (c) 2019, 2020, 2021 Pengutronix,
+// Marc Kleine-Budde <kernel@pengutronix.de>
+//
+// Based on:
+//
+// CAN bus driver for Microchip 25XXFD CAN Controller with SPI Interface
+//
+// Copyright (c) 2019 Martin Sperl <kernel@martin.sperl.org>
+//
+
+#include <linux/bitfield.h>
+
+#include "mcp251xfd.h"
+
+static inline int
+mcp251xfd_rx_head_get_from_chip(const struct mcp251xfd_priv *priv,
+ const struct mcp251xfd_rx_ring *ring,
+ u8 *rx_head)
+{
+ u32 fifo_sta;
+ int err;
+
+ err = regmap_read(priv->map_reg, MCP251XFD_REG_FIFOSTA(ring->fifo_nr),
+ &fifo_sta);
+ if (err)
+ return err;
+
+ *rx_head = FIELD_GET(MCP251XFD_REG_FIFOSTA_FIFOCI_MASK, fifo_sta);
+
+ return 0;
+}
+
+static inline int
+mcp251xfd_rx_tail_get_from_chip(const struct mcp251xfd_priv *priv,
+ const struct mcp251xfd_rx_ring *ring,
+ u8 *rx_tail)
+{
+ u32 fifo_ua;
+ int err;
+
+ err = regmap_read(priv->map_reg, MCP251XFD_REG_FIFOUA(ring->fifo_nr),
+ &fifo_ua);
+ if (err)
+ return err;
+
+ fifo_ua -= ring->base - MCP251XFD_RAM_START;
+ *rx_tail = fifo_ua / ring->obj_size;
+
+ return 0;
+}
+
+static int
+mcp251xfd_check_rx_tail(const struct mcp251xfd_priv *priv,
+ const struct mcp251xfd_rx_ring *ring)
+{
+ u8 rx_tail_chip, rx_tail;
+ int err;
+
+ if (!IS_ENABLED(CONFIG_CAN_MCP251XFD_SANITY))
+ return 0;
+
+ err = mcp251xfd_rx_tail_get_from_chip(priv, ring, &rx_tail_chip);
+ if (err)
+ return err;
+
+ rx_tail = mcp251xfd_get_rx_tail(ring);
+ if (rx_tail_chip != rx_tail) {
+ netdev_err(priv->ndev,
+ "RX tail of chip (%d) and ours (%d) inconsistent.\n",
+ rx_tail_chip, rx_tail);
+ return -EILSEQ;
+ }
+
+ return 0;
+}
+
+static int
+mcp251xfd_rx_ring_update(const struct mcp251xfd_priv *priv,
+ struct mcp251xfd_rx_ring *ring)
+{
+ u32 new_head;
+ u8 chip_rx_head;
+ int err;
+
+ err = mcp251xfd_rx_head_get_from_chip(priv, ring, &chip_rx_head);
+ if (err)
+ return err;
+
+ /* chip_rx_head, is the next RX-Object filled by the HW.
+ * The new RX head must be >= the old head.
+ */
+ new_head = round_down(ring->head, ring->obj_num) + chip_rx_head;
+ if (new_head <= ring->head)
+ new_head += ring->obj_num;
+
+ ring->head = new_head;
+
+ return mcp251xfd_check_rx_tail(priv, ring);
+}
+
+static void
+mcp251xfd_hw_rx_obj_to_skb(const struct mcp251xfd_priv *priv,
+ const struct mcp251xfd_hw_rx_obj_canfd *hw_rx_obj,
+ struct sk_buff *skb)
+{
+ struct canfd_frame *cfd = (struct canfd_frame *)skb->data;
+ u8 dlc;
+
+ if (hw_rx_obj->flags & MCP251XFD_OBJ_FLAGS_IDE) {
+ u32 sid, eid;
+
+ eid = FIELD_GET(MCP251XFD_OBJ_ID_EID_MASK, hw_rx_obj->id);
+ sid = FIELD_GET(MCP251XFD_OBJ_ID_SID_MASK, hw_rx_obj->id);
+
+ cfd->can_id = CAN_EFF_FLAG |
+ FIELD_PREP(MCP251XFD_REG_FRAME_EFF_EID_MASK, eid) |
+ FIELD_PREP(MCP251XFD_REG_FRAME_EFF_SID_MASK, sid);
+ } else {
+ cfd->can_id = FIELD_GET(MCP251XFD_OBJ_ID_SID_MASK,
+ hw_rx_obj->id);
+ }
+
+ dlc = FIELD_GET(MCP251XFD_OBJ_FLAGS_DLC_MASK, hw_rx_obj->flags);
+
+ /* CANFD */
+ if (hw_rx_obj->flags & MCP251XFD_OBJ_FLAGS_FDF) {
+ if (hw_rx_obj->flags & MCP251XFD_OBJ_FLAGS_ESI)
+ cfd->flags |= CANFD_ESI;
+
+ if (hw_rx_obj->flags & MCP251XFD_OBJ_FLAGS_BRS)
+ cfd->flags |= CANFD_BRS;
+
+ cfd->len = can_fd_dlc2len(dlc);
+ } else {
+ if (hw_rx_obj->flags & MCP251XFD_OBJ_FLAGS_RTR)
+ cfd->can_id |= CAN_RTR_FLAG;
+
+ can_frame_set_cc_len((struct can_frame *)cfd, dlc,
+ priv->can.ctrlmode);
+ }
+
+ if (!(hw_rx_obj->flags & MCP251XFD_OBJ_FLAGS_RTR))
+ memcpy(cfd->data, hw_rx_obj->data, cfd->len);
+
+ mcp251xfd_skb_set_timestamp(priv, skb, hw_rx_obj->ts);
+}
+
+static int
+mcp251xfd_handle_rxif_one(struct mcp251xfd_priv *priv,
+ struct mcp251xfd_rx_ring *ring,
+ const struct mcp251xfd_hw_rx_obj_canfd *hw_rx_obj)
+{
+ struct net_device_stats *stats = &priv->ndev->stats;
+ struct sk_buff *skb;
+ struct canfd_frame *cfd;
+ int err;
+
+ if (hw_rx_obj->flags & MCP251XFD_OBJ_FLAGS_FDF)
+ skb = alloc_canfd_skb(priv->ndev, &cfd);
+ else
+ skb = alloc_can_skb(priv->ndev, (struct can_frame **)&cfd);
+
+ if (!skb) {
+ stats->rx_dropped++;
+ return 0;
+ }
+
+ mcp251xfd_hw_rx_obj_to_skb(priv, hw_rx_obj, skb);
+ err = can_rx_offload_queue_sorted(&priv->offload, skb, hw_rx_obj->ts);
+ if (err)
+ stats->rx_fifo_errors++;
+
+ return 0;
+}
+
+static inline int
+mcp251xfd_rx_obj_read(const struct mcp251xfd_priv *priv,
+ const struct mcp251xfd_rx_ring *ring,
+ struct mcp251xfd_hw_rx_obj_canfd *hw_rx_obj,
+ const u8 offset, const u8 len)
+{
+ const int val_bytes = regmap_get_val_bytes(priv->map_rx);
+ int err;
+
+ err = regmap_bulk_read(priv->map_rx,
+ mcp251xfd_get_rx_obj_addr(ring, offset),
+ hw_rx_obj,
+ len * ring->obj_size / val_bytes);
+
+ return err;
+}
+
+static int
+mcp251xfd_handle_rxif_ring(struct mcp251xfd_priv *priv,
+ struct mcp251xfd_rx_ring *ring)
+{
+ struct mcp251xfd_hw_rx_obj_canfd *hw_rx_obj = ring->obj;
+ u8 rx_tail, len;
+ int err, i;
+
+ err = mcp251xfd_rx_ring_update(priv, ring);
+ if (err)
+ return err;
+
+ while ((len = mcp251xfd_get_rx_linear_len(ring))) {
+ int offset;
+
+ rx_tail = mcp251xfd_get_rx_tail(ring);
+
+ err = mcp251xfd_rx_obj_read(priv, ring, hw_rx_obj,
+ rx_tail, len);
+ if (err)
+ return err;
+
+ for (i = 0; i < len; i++) {
+ err = mcp251xfd_handle_rxif_one(priv, ring,
+ (void *)hw_rx_obj +
+ i * ring->obj_size);
+ if (err)
+ return err;
+ }
+
+ /* Increment the RX FIFO tail pointer 'len' times in a
+ * single SPI message.
+ *
+ * Note:
+ * Calculate offset, so that the SPI transfer ends on
+ * the last message of the uinc_xfer array, which has
+ * "cs_change == 0", to properly deactivate the chip
+ * select.
+ */
+ offset = ARRAY_SIZE(ring->uinc_xfer) - len;
+ err = spi_sync_transfer(priv->spi,
+ ring->uinc_xfer + offset, len);
+ if (err)
+ return err;
+
+ ring->tail += len;
+ }
+
+ return 0;
+}
+
+int mcp251xfd_handle_rxif(struct mcp251xfd_priv *priv)
+{
+ struct mcp251xfd_rx_ring *ring;
+ int err, n;
+
+ mcp251xfd_for_each_rx_ring(priv, ring, n) {
+ err = mcp251xfd_handle_rxif_ring(priv, ring);
+ if (err)
+ return err;
+ }
+
+ return 0;
+}
diff --git a/drivers/net/can/spi/mcp251xfd/mcp251xfd-tef.c b/drivers/net/can/spi/mcp251xfd/mcp251xfd-tef.c
new file mode 100644
index 0000000..4061660
--- /dev/null
+++ b/drivers/net/can/spi/mcp251xfd/mcp251xfd-tef.c
@@ -0,0 +1,260 @@
+// SPDX-License-Identifier: GPL-2.0
+//
+// mcp251xfd - Microchip MCP251xFD Family CAN controller driver
+//
+// Copyright (c) 2019, 2020, 2021 Pengutronix,
+// Marc Kleine-Budde <kernel@pengutronix.de>
+//
+// Based on:
+//
+// CAN bus driver for Microchip 25XXFD CAN Controller with SPI Interface
+//
+// Copyright (c) 2019 Martin Sperl <kernel@martin.sperl.org>
+//
+
+#include <linux/bitfield.h>
+
+#include "mcp251xfd.h"
+
+static inline int
+mcp251xfd_tef_tail_get_from_chip(const struct mcp251xfd_priv *priv,
+ u8 *tef_tail)
+{
+ u32 tef_ua;
+ int err;
+
+ err = regmap_read(priv->map_reg, MCP251XFD_REG_TEFUA, &tef_ua);
+ if (err)
+ return err;
+
+ *tef_tail = tef_ua / sizeof(struct mcp251xfd_hw_tef_obj);
+
+ return 0;
+}
+
+static int mcp251xfd_check_tef_tail(const struct mcp251xfd_priv *priv)
+{
+ u8 tef_tail_chip, tef_tail;
+ int err;
+
+ if (!IS_ENABLED(CONFIG_CAN_MCP251XFD_SANITY))
+ return 0;
+
+ err = mcp251xfd_tef_tail_get_from_chip(priv, &tef_tail_chip);
+ if (err)
+ return err;
+
+ tef_tail = mcp251xfd_get_tef_tail(priv);
+ if (tef_tail_chip != tef_tail) {
+ netdev_err(priv->ndev,
+ "TEF tail of chip (0x%02x) and ours (0x%08x) inconsistent.\n",
+ tef_tail_chip, tef_tail);
+ return -EILSEQ;
+ }
+
+ return 0;
+}
+
+static int
+mcp251xfd_handle_tefif_recover(const struct mcp251xfd_priv *priv, const u32 seq)
+{
+ const struct mcp251xfd_tx_ring *tx_ring = priv->tx;
+ u32 tef_sta;
+ int err;
+
+ err = regmap_read(priv->map_reg, MCP251XFD_REG_TEFSTA, &tef_sta);
+ if (err)
+ return err;
+
+ if (tef_sta & MCP251XFD_REG_TEFSTA_TEFOVIF) {
+ netdev_err(priv->ndev,
+ "Transmit Event FIFO buffer overflow.\n");
+ return -ENOBUFS;
+ }
+
+ netdev_info(priv->ndev,
+ "Transmit Event FIFO buffer %s. (seq=0x%08x, tef_tail=0x%08x, tef_head=0x%08x, tx_head=0x%08x).\n",
+ tef_sta & MCP251XFD_REG_TEFSTA_TEFFIF ?
+ "full" : tef_sta & MCP251XFD_REG_TEFSTA_TEFNEIF ?
+ "not empty" : "empty",
+ seq, priv->tef->tail, priv->tef->head, tx_ring->head);
+
+ /* The Sequence Number in the TEF doesn't match our tef_tail. */
+ return -EAGAIN;
+}
+
+static int
+mcp251xfd_handle_tefif_one(struct mcp251xfd_priv *priv,
+ const struct mcp251xfd_hw_tef_obj *hw_tef_obj,
+ unsigned int *frame_len_ptr)
+{
+ struct net_device_stats *stats = &priv->ndev->stats;
+ struct sk_buff *skb;
+ u32 seq, seq_masked, tef_tail_masked, tef_tail;
+
+ seq = FIELD_GET(MCP251XFD_OBJ_FLAGS_SEQ_MCP2518FD_MASK,
+ hw_tef_obj->flags);
+
+ /* Use the MCP2517FD mask on the MCP2518FD, too. We only
+ * compare 7 bits, this should be enough to detect
+ * net-yet-completed, i.e. old TEF objects.
+ */
+ seq_masked = seq &
+ field_mask(MCP251XFD_OBJ_FLAGS_SEQ_MCP2517FD_MASK);
+ tef_tail_masked = priv->tef->tail &
+ field_mask(MCP251XFD_OBJ_FLAGS_SEQ_MCP2517FD_MASK);
+ if (seq_masked != tef_tail_masked)
+ return mcp251xfd_handle_tefif_recover(priv, seq);
+
+ tef_tail = mcp251xfd_get_tef_tail(priv);
+ skb = priv->can.echo_skb[tef_tail];
+ if (skb)
+ mcp251xfd_skb_set_timestamp(priv, skb, hw_tef_obj->ts);
+ stats->tx_bytes +=
+ can_rx_offload_get_echo_skb(&priv->offload,
+ tef_tail, hw_tef_obj->ts,
+ frame_len_ptr);
+ stats->tx_packets++;
+ priv->tef->tail++;
+
+ return 0;
+}
+
+static int mcp251xfd_tef_ring_update(struct mcp251xfd_priv *priv)
+{
+ const struct mcp251xfd_tx_ring *tx_ring = priv->tx;
+ unsigned int new_head;
+ u8 chip_tx_tail;
+ int err;
+
+ err = mcp251xfd_tx_tail_get_from_chip(priv, &chip_tx_tail);
+ if (err)
+ return err;
+
+ /* chip_tx_tail, is the next TX-Object send by the HW.
+ * The new TEF head must be >= the old head, ...
+ */
+ new_head = round_down(priv->tef->head, tx_ring->obj_num) + chip_tx_tail;
+ if (new_head <= priv->tef->head)
+ new_head += tx_ring->obj_num;
+
+ /* ... but it cannot exceed the TX head. */
+ priv->tef->head = min(new_head, tx_ring->head);
+
+ return mcp251xfd_check_tef_tail(priv);
+}
+
+static inline int
+mcp251xfd_tef_obj_read(const struct mcp251xfd_priv *priv,
+ struct mcp251xfd_hw_tef_obj *hw_tef_obj,
+ const u8 offset, const u8 len)
+{
+ const struct mcp251xfd_tx_ring *tx_ring = priv->tx;
+ const int val_bytes = regmap_get_val_bytes(priv->map_rx);
+
+ if (IS_ENABLED(CONFIG_CAN_MCP251XFD_SANITY) &&
+ (offset > tx_ring->obj_num ||
+ len > tx_ring->obj_num ||
+ offset + len > tx_ring->obj_num)) {
+ netdev_err(priv->ndev,
+ "Trying to read too many TEF objects (max=%d, offset=%d, len=%d).\n",
+ tx_ring->obj_num, offset, len);
+ return -ERANGE;
+ }
+
+ return regmap_bulk_read(priv->map_rx,
+ mcp251xfd_get_tef_obj_addr(offset),
+ hw_tef_obj,
+ sizeof(*hw_tef_obj) / val_bytes * len);
+}
+
+static inline void mcp251xfd_ecc_tefif_successful(struct mcp251xfd_priv *priv)
+{
+ struct mcp251xfd_ecc *ecc = &priv->ecc;
+
+ ecc->ecc_stat = 0;
+}
+
+int mcp251xfd_handle_tefif(struct mcp251xfd_priv *priv)
+{
+ struct mcp251xfd_hw_tef_obj hw_tef_obj[MCP251XFD_TX_OBJ_NUM_MAX];
+ unsigned int total_frame_len = 0;
+ u8 tef_tail, len, l;
+ int err, i;
+
+ err = mcp251xfd_tef_ring_update(priv);
+ if (err)
+ return err;
+
+ tef_tail = mcp251xfd_get_tef_tail(priv);
+ len = mcp251xfd_get_tef_len(priv);
+ l = mcp251xfd_get_tef_linear_len(priv);
+ err = mcp251xfd_tef_obj_read(priv, hw_tef_obj, tef_tail, l);
+ if (err)
+ return err;
+
+ if (l < len) {
+ err = mcp251xfd_tef_obj_read(priv, &hw_tef_obj[l], 0, len - l);
+ if (err)
+ return err;
+ }
+
+ for (i = 0; i < len; i++) {
+ unsigned int frame_len = 0;
+
+ err = mcp251xfd_handle_tefif_one(priv, &hw_tef_obj[i], &frame_len);
+ /* -EAGAIN means the Sequence Number in the TEF
+ * doesn't match our tef_tail. This can happen if we
+ * read the TEF objects too early. Leave loop let the
+ * interrupt handler call us again.
+ */
+ if (err == -EAGAIN)
+ goto out_netif_wake_queue;
+ if (err)
+ return err;
+
+ total_frame_len += frame_len;
+ }
+
+ out_netif_wake_queue:
+ len = i; /* number of handled goods TEFs */
+ if (len) {
+ struct mcp251xfd_tef_ring *ring = priv->tef;
+ struct mcp251xfd_tx_ring *tx_ring = priv->tx;
+ int offset;
+
+ /* Increment the TEF FIFO tail pointer 'len' times in
+ * a single SPI message.
+ *
+ * Note:
+ * Calculate offset, so that the SPI transfer ends on
+ * the last message of the uinc_xfer array, which has
+ * "cs_change == 0", to properly deactivate the chip
+ * select.
+ */
+ offset = ARRAY_SIZE(ring->uinc_xfer) - len;
+ err = spi_sync_transfer(priv->spi,
+ ring->uinc_xfer + offset, len);
+ if (err)
+ return err;
+
+ tx_ring->tail += len;
+ netdev_completed_queue(priv->ndev, len, total_frame_len);
+
+ err = mcp251xfd_check_tef_tail(priv);
+ if (err)
+ return err;
+ }
+
+ mcp251xfd_ecc_tefif_successful(priv);
+
+ if (mcp251xfd_get_tx_free(priv->tx)) {
+ /* Make sure that anybody stopping the queue after
+ * this sees the new tx_ring->tail.
+ */
+ smp_mb();
+ netif_wake_queue(priv->ndev);
+ }
+
+ return 0;
+}
diff --git a/drivers/net/can/spi/mcp251xfd/mcp251xfd-tx.c b/drivers/net/can/spi/mcp251xfd/mcp251xfd-tx.c
new file mode 100644
index 0000000..ffb6c36
--- /dev/null
+++ b/drivers/net/can/spi/mcp251xfd/mcp251xfd-tx.c
@@ -0,0 +1,205 @@
+// SPDX-License-Identifier: GPL-2.0
+//
+// mcp251xfd - Microchip MCP251xFD Family CAN controller driver
+//
+// Copyright (c) 2019, 2020, 2021 Pengutronix,
+// Marc Kleine-Budde <kernel@pengutronix.de>
+//
+// Based on:
+//
+// CAN bus driver for Microchip 25XXFD CAN Controller with SPI Interface
+//
+// Copyright (c) 2019 Martin Sperl <kernel@martin.sperl.org>
+//
+
+#include <asm/unaligned.h>
+#include <linux/bitfield.h>
+
+#include "mcp251xfd.h"
+
+static inline struct
+mcp251xfd_tx_obj *mcp251xfd_get_tx_obj_next(struct mcp251xfd_tx_ring *tx_ring)
+{
+ u8 tx_head;
+
+ tx_head = mcp251xfd_get_tx_head(tx_ring);
+
+ return &tx_ring->obj[tx_head];
+}
+
+static void
+mcp251xfd_tx_obj_from_skb(const struct mcp251xfd_priv *priv,
+ struct mcp251xfd_tx_obj *tx_obj,
+ const struct sk_buff *skb,
+ unsigned int seq)
+{
+ const struct canfd_frame *cfd = (struct canfd_frame *)skb->data;
+ struct mcp251xfd_hw_tx_obj_raw *hw_tx_obj;
+ union mcp251xfd_tx_obj_load_buf *load_buf;
+ u8 dlc;
+ u32 id, flags;
+ int len_sanitized = 0, len;
+
+ if (cfd->can_id & CAN_EFF_FLAG) {
+ u32 sid, eid;
+
+ sid = FIELD_GET(MCP251XFD_REG_FRAME_EFF_SID_MASK, cfd->can_id);
+ eid = FIELD_GET(MCP251XFD_REG_FRAME_EFF_EID_MASK, cfd->can_id);
+
+ id = FIELD_PREP(MCP251XFD_OBJ_ID_EID_MASK, eid) |
+ FIELD_PREP(MCP251XFD_OBJ_ID_SID_MASK, sid);
+
+ flags = MCP251XFD_OBJ_FLAGS_IDE;
+ } else {
+ id = FIELD_PREP(MCP251XFD_OBJ_ID_SID_MASK, cfd->can_id);
+ flags = 0;
+ }
+
+ /* Use the MCP2518FD mask even on the MCP2517FD. It doesn't
+ * harm, only the lower 7 bits will be transferred into the
+ * TEF object.
+ */
+ flags |= FIELD_PREP(MCP251XFD_OBJ_FLAGS_SEQ_MCP2518FD_MASK, seq);
+
+ if (cfd->can_id & CAN_RTR_FLAG)
+ flags |= MCP251XFD_OBJ_FLAGS_RTR;
+ else
+ len_sanitized = canfd_sanitize_len(cfd->len);
+
+ /* CANFD */
+ if (can_is_canfd_skb(skb)) {
+ if (cfd->flags & CANFD_ESI)
+ flags |= MCP251XFD_OBJ_FLAGS_ESI;
+
+ flags |= MCP251XFD_OBJ_FLAGS_FDF;
+
+ if (cfd->flags & CANFD_BRS)
+ flags |= MCP251XFD_OBJ_FLAGS_BRS;
+
+ dlc = can_fd_len2dlc(cfd->len);
+ } else {
+ dlc = can_get_cc_dlc((struct can_frame *)cfd,
+ priv->can.ctrlmode);
+ }
+
+ flags |= FIELD_PREP(MCP251XFD_OBJ_FLAGS_DLC_MASK, dlc);
+
+ load_buf = &tx_obj->buf;
+ if (priv->devtype_data.quirks & MCP251XFD_QUIRK_CRC_TX)
+ hw_tx_obj = &load_buf->crc.hw_tx_obj;
+ else
+ hw_tx_obj = &load_buf->nocrc.hw_tx_obj;
+
+ put_unaligned_le32(id, &hw_tx_obj->id);
+ put_unaligned_le32(flags, &hw_tx_obj->flags);
+
+ /* Copy data */
+ memcpy(hw_tx_obj->data, cfd->data, cfd->len);
+
+ /* Clear unused data at end of CAN frame */
+ if (MCP251XFD_SANITIZE_CAN && len_sanitized) {
+ int pad_len;
+
+ pad_len = len_sanitized - cfd->len;
+ if (pad_len)
+ memset(hw_tx_obj->data + cfd->len, 0x0, pad_len);
+ }
+
+ /* Number of bytes to be written into the RAM of the controller */
+ len = sizeof(hw_tx_obj->id) + sizeof(hw_tx_obj->flags);
+ if (MCP251XFD_SANITIZE_CAN)
+ len += round_up(len_sanitized, sizeof(u32));
+ else
+ len += round_up(cfd->len, sizeof(u32));
+
+ if (priv->devtype_data.quirks & MCP251XFD_QUIRK_CRC_TX) {
+ u16 crc;
+
+ mcp251xfd_spi_cmd_crc_set_len_in_ram(&load_buf->crc.cmd,
+ len);
+ /* CRC */
+ len += sizeof(load_buf->crc.cmd);
+ crc = mcp251xfd_crc16_compute(&load_buf->crc, len);
+ put_unaligned_be16(crc, (void *)load_buf + len);
+
+ /* Total length */
+ len += sizeof(load_buf->crc.crc);
+ } else {
+ len += sizeof(load_buf->nocrc.cmd);
+ }
+
+ tx_obj->xfer[0].len = len;
+}
+
+static int mcp251xfd_tx_obj_write(const struct mcp251xfd_priv *priv,
+ struct mcp251xfd_tx_obj *tx_obj)
+{
+ return spi_async(priv->spi, &tx_obj->msg);
+}
+
+static bool mcp251xfd_tx_busy(const struct mcp251xfd_priv *priv,
+ struct mcp251xfd_tx_ring *tx_ring)
+{
+ if (mcp251xfd_get_tx_free(tx_ring) > 0)
+ return false;
+
+ netif_stop_queue(priv->ndev);
+
+ /* Memory barrier before checking tx_free (head and tail) */
+ smp_mb();
+
+ if (mcp251xfd_get_tx_free(tx_ring) == 0) {
+ netdev_dbg(priv->ndev,
+ "Stopping tx-queue (tx_head=0x%08x, tx_tail=0x%08x, len=%d).\n",
+ tx_ring->head, tx_ring->tail,
+ tx_ring->head - tx_ring->tail);
+
+ return true;
+ }
+
+ netif_start_queue(priv->ndev);
+
+ return false;
+}
+
+netdev_tx_t mcp251xfd_start_xmit(struct sk_buff *skb,
+ struct net_device *ndev)
+{
+ struct mcp251xfd_priv *priv = netdev_priv(ndev);
+ struct mcp251xfd_tx_ring *tx_ring = priv->tx;
+ struct mcp251xfd_tx_obj *tx_obj;
+ unsigned int frame_len;
+ u8 tx_head;
+ int err;
+
+ if (can_dropped_invalid_skb(ndev, skb))
+ return NETDEV_TX_OK;
+
+ if (mcp251xfd_tx_busy(priv, tx_ring))
+ return NETDEV_TX_BUSY;
+
+ tx_obj = mcp251xfd_get_tx_obj_next(tx_ring);
+ mcp251xfd_tx_obj_from_skb(priv, tx_obj, skb, tx_ring->head);
+
+ /* Stop queue if we occupy the complete TX FIFO */
+ tx_head = mcp251xfd_get_tx_head(tx_ring);
+ tx_ring->head++;
+ if (mcp251xfd_get_tx_free(tx_ring) == 0)
+ netif_stop_queue(ndev);
+
+ frame_len = can_skb_get_frame_len(skb);
+ err = can_put_echo_skb(skb, ndev, tx_head, frame_len);
+ if (!err)
+ netdev_sent_queue(priv->ndev, frame_len);
+
+ err = mcp251xfd_tx_obj_write(priv, tx_obj);
+ if (err)
+ goto out_err;
+
+ return NETDEV_TX_OK;
+
+ out_err:
+ netdev_err(priv->ndev, "ERROR in %s: %d\n", __func__, err);
+
+ return NETDEV_TX_OK;
+}
diff --git a/drivers/net/can/spi/mcp251xfd/mcp251xfd.h b/drivers/net/can/spi/mcp251xfd/mcp251xfd.h
index 0f322da..f551c90 100644
--- a/drivers/net/can/spi/mcp251xfd/mcp251xfd.h
+++ b/drivers/net/can/spi/mcp251xfd/mcp251xfd.h
@@ -10,6 +10,7 @@
#ifndef _MCP251XFD_H
#define _MCP251XFD_H
+#include <linux/bitfield.h>
#include <linux/can/core.h>
#include <linux/can/dev.h>
#include <linux/can/rx-offload.h>
@@ -625,6 +626,12 @@ MCP251XFD_IS(2517);
MCP251XFD_IS(2518);
MCP251XFD_IS(251X);
+static inline bool mcp251xfd_is_fd_mode(const struct mcp251xfd_priv *priv)
+{
+ /* listen-only mode works like FD mode */
+ return priv->can.ctrlmode & (CAN_CTRLMODE_LISTENONLY | CAN_CTRLMODE_FD);
+}
+
static inline u8 mcp251xfd_first_byte_set(u32 mask)
{
return (mask & 0x0000ffff) ?
@@ -761,6 +768,24 @@ mcp251xfd_get_rx_obj_addr(const struct mcp251xfd_rx_ring *ring, u8 n)
return ring->base + ring->obj_size * n;
}
+static inline int
+mcp251xfd_tx_tail_get_from_chip(const struct mcp251xfd_priv *priv,
+ u8 *tx_tail)
+{
+ u32 fifo_sta;
+ int err;
+
+ err = regmap_read(priv->map_reg,
+ MCP251XFD_REG_FIFOSTA(MCP251XFD_TX_FIFO),
+ &fifo_sta);
+ if (err)
+ return err;
+
+ *tx_tail = FIELD_GET(MCP251XFD_REG_FIFOSTA_FIFOCI_MASK, fifo_sta);
+
+ return 0;
+}
+
static inline u8 mcp251xfd_get_tef_head(const struct mcp251xfd_priv *priv)
{
return priv->tef->head & (priv->tx->obj_num - 1);
@@ -849,15 +874,24 @@ mcp251xfd_get_rx_linear_len(const struct mcp251xfd_rx_ring *ring)
(n) < (priv)->rx_ring_num; \
(n)++, (ring) = *((priv)->rx + (n)))
-int mcp251xfd_regmap_init(struct mcp251xfd_priv *priv);
+int mcp251xfd_chip_fifo_init(const struct mcp251xfd_priv *priv);
u16 mcp251xfd_crc16_compute2(const void *cmd, size_t cmd_size,
const void *data, size_t data_size);
u16 mcp251xfd_crc16_compute(const void *data, size_t data_size);
+int mcp251xfd_regmap_init(struct mcp251xfd_priv *priv);
+void mcp251xfd_ring_init(struct mcp251xfd_priv *priv);
+void mcp251xfd_ring_free(struct mcp251xfd_priv *priv);
+int mcp251xfd_ring_alloc(struct mcp251xfd_priv *priv);
+int mcp251xfd_handle_rxif(struct mcp251xfd_priv *priv);
+int mcp251xfd_handle_tefif(struct mcp251xfd_priv *priv);
void mcp251xfd_skb_set_timestamp(const struct mcp251xfd_priv *priv,
struct sk_buff *skb, u32 timestamp);
void mcp251xfd_timestamp_init(struct mcp251xfd_priv *priv);
void mcp251xfd_timestamp_stop(struct mcp251xfd_priv *priv);
+netdev_tx_t mcp251xfd_start_xmit(struct sk_buff *skb,
+ struct net_device *ndev);
+
#if IS_ENABLED(CONFIG_DEV_COREDUMP)
void mcp251xfd_dump(const struct mcp251xfd_priv *priv);
#else
diff --git a/drivers/net/can/sun4i_can.c b/drivers/net/can/sun4i_can.c
index 54aa7c2..25d6d81 100644
--- a/drivers/net/can/sun4i_can.c
+++ b/drivers/net/can/sun4i_can.c
@@ -61,6 +61,7 @@
#include <linux/of.h>
#include <linux/of_device.h>
#include <linux/platform_device.h>
+#include <linux/reset.h>
#define DRV_NAME "sun4i_can"
@@ -200,10 +201,20 @@
#define SUN4I_CAN_MAX_IRQ 20
#define SUN4I_MODE_MAX_RETRIES 100
+/**
+ * struct sun4ican_quirks - Differences between SoC variants.
+ *
+ * @has_reset: SoC needs reset deasserted.
+ */
+struct sun4ican_quirks {
+ bool has_reset;
+};
+
struct sun4ican_priv {
struct can_priv can;
void __iomem *base;
struct clk *clk;
+ struct reset_control *reset;
spinlock_t cmdreg_lock; /* lock for concurrent cmd register writes */
};
@@ -490,18 +501,20 @@ static void sun4i_can_rx(struct net_device *dev)
}
/* remote frame ? */
- if (fi & SUN4I_MSG_RTR_FLAG)
+ if (fi & SUN4I_MSG_RTR_FLAG) {
id |= CAN_RTR_FLAG;
- else
+ } else {
for (i = 0; i < cf->len; i++)
cf->data[i] = readl(priv->base + dreg + i * 4);
+ stats->rx_bytes += cf->len;
+ }
+ stats->rx_packets++;
+
cf->can_id = id;
sun4i_can_write_cmdreg(priv, SUN4I_CMD_RELEASE_RBUF);
- stats->rx_packets++;
- stats->rx_bytes += cf->len;
netif_rx(skb);
can_led_event(dev, CAN_LED_EVENT_RX);
@@ -622,13 +635,10 @@ static int sun4i_can_err(struct net_device *dev, u8 isrc, u8 status)
can_bus_off(dev);
}
- if (likely(skb)) {
- stats->rx_packets++;
- stats->rx_bytes += cf->len;
+ if (likely(skb))
netif_rx(skb);
- } else {
+ else
return -ENOMEM;
- }
return 0;
}
@@ -651,11 +661,8 @@ static irqreturn_t sun4i_can_interrupt(int irq, void *dev_id)
if (isrc & SUN4I_INT_TBUF_VLD) {
/* transmission complete interrupt */
- stats->tx_bytes +=
- readl(priv->base +
- SUN4I_REG_RBUF_RBACK_START_ADDR) & 0xf;
+ stats->tx_bytes += can_get_echo_skb(dev, 0, NULL);
stats->tx_packets++;
- can_get_echo_skb(dev, 0, NULL);
netif_wake_queue(dev);
can_led_event(dev, CAN_LED_EVENT_TX);
}
@@ -702,6 +709,13 @@ static int sun4ican_open(struct net_device *dev)
goto exit_irq;
}
+ /* software reset deassert */
+ err = reset_control_deassert(priv->reset);
+ if (err) {
+ netdev_err(dev, "could not deassert CAN reset\n");
+ goto exit_soft_reset;
+ }
+
/* turn on clocking for CAN peripheral block */
err = clk_prepare_enable(priv->clk);
if (err) {
@@ -723,6 +737,8 @@ static int sun4ican_open(struct net_device *dev)
exit_can_start:
clk_disable_unprepare(priv->clk);
exit_clock:
+ reset_control_assert(priv->reset);
+exit_soft_reset:
free_irq(dev->irq, dev);
exit_irq:
close_candev(dev);
@@ -736,6 +752,7 @@ static int sun4ican_close(struct net_device *dev)
netif_stop_queue(dev);
sun4i_can_stop(dev);
clk_disable_unprepare(priv->clk);
+ reset_control_assert(priv->reset);
free_irq(dev->irq, dev);
close_candev(dev);
@@ -750,9 +767,27 @@ static const struct net_device_ops sun4ican_netdev_ops = {
.ndo_start_xmit = sun4ican_start_xmit,
};
+static const struct sun4ican_quirks sun4ican_quirks_a10 = {
+ .has_reset = false,
+};
+
+static const struct sun4ican_quirks sun4ican_quirks_r40 = {
+ .has_reset = true,
+};
+
static const struct of_device_id sun4ican_of_match[] = {
- {.compatible = "allwinner,sun4i-a10-can"},
- {},
+ {
+ .compatible = "allwinner,sun4i-a10-can",
+ .data = &sun4ican_quirks_a10
+ }, {
+ .compatible = "allwinner,sun7i-a20-can",
+ .data = &sun4ican_quirks_a10
+ }, {
+ .compatible = "allwinner,sun8i-r40-can",
+ .data = &sun4ican_quirks_r40
+ }, {
+ /* sentinel */
+ },
};
MODULE_DEVICE_TABLE(of, sun4ican_of_match);
@@ -771,10 +806,28 @@ static int sun4ican_probe(struct platform_device *pdev)
{
struct device_node *np = pdev->dev.of_node;
struct clk *clk;
+ struct reset_control *reset = NULL;
void __iomem *addr;
int err, irq;
struct net_device *dev;
struct sun4ican_priv *priv;
+ const struct sun4ican_quirks *quirks;
+
+ quirks = of_device_get_match_data(&pdev->dev);
+ if (!quirks) {
+ dev_err(&pdev->dev, "failed to determine the quirks to use\n");
+ err = -ENODEV;
+ goto exit;
+ }
+
+ if (quirks->has_reset) {
+ reset = devm_reset_control_get_exclusive(&pdev->dev, NULL);
+ if (IS_ERR(reset)) {
+ dev_err(&pdev->dev, "unable to request reset\n");
+ err = PTR_ERR(reset);
+ goto exit;
+ }
+ }
clk = of_clk_get(np, 0);
if (IS_ERR(clk)) {
@@ -818,6 +871,7 @@ static int sun4ican_probe(struct platform_device *pdev)
CAN_CTRLMODE_3_SAMPLES;
priv->base = addr;
priv->clk = clk;
+ priv->reset = reset;
spin_lock_init(&priv->cmdreg_lock);
platform_set_drvdata(pdev, dev);
diff --git a/drivers/net/can/ti_hecc.c b/drivers/net/can/ti_hecc.c
index 353062e..ff31b993 100644
--- a/drivers/net/can/ti_hecc.c
+++ b/drivers/net/can/ti_hecc.c
@@ -859,7 +859,6 @@ static int ti_hecc_probe(struct platform_device *pdev)
struct net_device *ndev = (struct net_device *)0;
struct ti_hecc_priv *priv;
struct device_node *np = pdev->dev.of_node;
- struct resource *irq;
struct regulator *reg_xceiver;
int err = -ENODEV;
@@ -904,9 +903,9 @@ static int ti_hecc_probe(struct platform_device *pdev)
goto probe_exit_candev;
}
- irq = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
- if (!irq) {
- dev_err(&pdev->dev, "No irq resource\n");
+ ndev->irq = platform_get_irq(pdev, 0);
+ if (ndev->irq < 0) {
+ err = ndev->irq;
goto probe_exit_candev;
}
@@ -920,7 +919,6 @@ static int ti_hecc_probe(struct platform_device *pdev)
priv->can.ctrlmode_supported = CAN_CTRLMODE_3_SAMPLES;
spin_lock_init(&priv->mbx_lock);
- ndev->irq = irq->start;
ndev->flags |= IFF_ECHO;
platform_set_drvdata(pdev, ndev);
SET_NETDEV_DEV(ndev, &pdev->dev);
diff --git a/drivers/net/can/usb/ems_usb.c b/drivers/net/can/usb/ems_usb.c
index 2b5302e..7bedcef 100644
--- a/drivers/net/can/usb/ems_usb.c
+++ b/drivers/net/can/usb/ems_usb.c
@@ -230,7 +230,6 @@ struct ems_tx_urb_context {
struct ems_usb *dev;
u32 echo_index;
- u8 dlc;
};
struct ems_usb {
@@ -320,10 +319,11 @@ static void ems_usb_rx_can_msg(struct ems_usb *dev, struct ems_cpc_msg *msg)
} else {
for (i = 0; i < cf->len; i++)
cf->data[i] = msg->msg.can_msg.msg[i];
- }
+ stats->rx_bytes += cf->len;
+ }
stats->rx_packets++;
- stats->rx_bytes += cf->len;
+
netif_rx(skb);
}
@@ -397,8 +397,6 @@ static void ems_usb_rx_err(struct ems_usb *dev, struct ems_cpc_msg *msg)
stats->rx_errors++;
}
- stats->rx_packets++;
- stats->rx_bytes += cf->len;
netif_rx(skb);
}
@@ -518,9 +516,8 @@ static void ems_usb_write_bulk_callback(struct urb *urb)
/* transmission complete interrupt */
netdev->stats.tx_packets++;
- netdev->stats.tx_bytes += context->dlc;
-
- can_get_echo_skb(netdev, context->echo_index, NULL);
+ netdev->stats.tx_bytes += can_get_echo_skb(netdev, context->echo_index,
+ NULL);
/* Release context */
context->echo_index = MAX_TX_URBS;
@@ -806,7 +803,6 @@ static netdev_tx_t ems_usb_start_xmit(struct sk_buff *skb, struct net_device *ne
context->dev = dev;
context->echo_index = i;
- context->dlc = cf->len;
usb_fill_bulk_urb(urb, dev->udev, usb_sndbulkpipe(dev->udev, 2), buf,
size, ems_usb_write_bulk_callback, context);
diff --git a/drivers/net/can/usb/esd_usb2.c b/drivers/net/can/usb/esd_usb2.c
index c6068a2..286daaa 100644
--- a/drivers/net/can/usb/esd_usb2.c
+++ b/drivers/net/can/usb/esd_usb2.c
@@ -183,7 +183,6 @@ struct esd_usb2_net_priv;
struct esd_tx_urb_context {
struct esd_usb2_net_priv *priv;
u32 echo_index;
- int len; /* CAN payload length */
};
struct esd_usb2 {
@@ -293,8 +292,6 @@ static void esd_usb2_rx_event(struct esd_usb2_net_priv *priv,
priv->bec.txerr = txerr;
priv->bec.rxerr = rxerr;
- stats->rx_packets++;
- stats->rx_bytes += cf->len;
netif_rx(skb);
}
}
@@ -334,10 +331,11 @@ static void esd_usb2_rx_can_msg(struct esd_usb2_net_priv *priv,
} else {
for (i = 0; i < cf->len; i++)
cf->data[i] = msg->msg.rx.data[i];
- }
+ stats->rx_bytes += cf->len;
+ }
stats->rx_packets++;
- stats->rx_bytes += cf->len;
+
netif_rx(skb);
}
@@ -358,8 +356,8 @@ static void esd_usb2_tx_done_msg(struct esd_usb2_net_priv *priv,
if (!msg->msg.txdone.status) {
stats->tx_packets++;
- stats->tx_bytes += context->len;
- can_get_echo_skb(netdev, context->echo_index, NULL);
+ stats->tx_bytes += can_get_echo_skb(netdev, context->echo_index,
+ NULL);
} else {
stats->tx_errors++;
can_free_echo_skb(netdev, context->echo_index, NULL);
@@ -784,7 +782,6 @@ static netdev_tx_t esd_usb2_start_xmit(struct sk_buff *skb,
context->priv = priv;
context->echo_index = i;
- context->len = cf->len;
/* hnd must not be 0 - MSB is stripped in txdone handling */
msg->msg.tx.hnd = 0x80000000 | i; /* returned in TX done message */
diff --git a/drivers/net/can/usb/etas_es58x/es581_4.c b/drivers/net/can/usb/etas_es58x/es581_4.c
index 14e360c..1bcdcec 100644
--- a/drivers/net/can/usb/etas_es58x/es581_4.c
+++ b/drivers/net/can/usb/etas_es58x/es581_4.c
@@ -10,6 +10,7 @@
*/
#include <linux/kernel.h>
+#include <linux/units.h>
#include <asm/unaligned.h>
#include "es58x_core.h"
@@ -469,8 +470,8 @@ const struct es58x_parameters es581_4_param = {
.bittiming_const = &es581_4_bittiming_const,
.data_bittiming_const = NULL,
.tdc_const = NULL,
- .bitrate_max = 1 * CAN_MBPS,
- .clock = {.freq = 50 * CAN_MHZ},
+ .bitrate_max = 1 * MEGA /* BPS */,
+ .clock = {.freq = 50 * MEGA /* Hz */},
.ctrlmode_supported = CAN_CTRLMODE_CC_LEN8_DLC,
.tx_start_of_frame = 0xAFAF,
.rx_start_of_frame = 0xFAFA,
diff --git a/drivers/net/can/usb/etas_es58x/es58x_core.c b/drivers/net/can/usb/etas_es58x/es58x_core.c
index 24627ab..2ed2370 100644
--- a/drivers/net/can/usb/etas_es58x/es58x_core.c
+++ b/drivers/net/can/usb/etas_es58x/es58x_core.c
@@ -849,13 +849,6 @@ int es58x_rx_err_msg(struct net_device *netdev, enum es58x_err error,
break;
}
- /* driver/net/can/dev.c:can_restart() takes in account error
- * messages in the RX stats. Doing the same here for
- * consistency.
- */
- netdev->stats.rx_packets++;
- netdev->stats.rx_bytes += CAN_ERR_DLC;
-
if (cf) {
if (cf->data[1])
cf->can_id |= CAN_ERR_CRTL;
@@ -2094,6 +2087,7 @@ static int es58x_init_netdev(struct es58x_device *es58x_dev, int channel_idx)
netdev->netdev_ops = &es58x_netdev_ops;
netdev->flags |= IFF_ECHO; /* We support local echo */
+ netdev->dev_port = channel_idx;
ret = register_candev(netdev);
if (ret)
diff --git a/drivers/net/can/usb/etas_es58x/es58x_fd.c b/drivers/net/can/usb/etas_es58x/es58x_fd.c
index 4f0cae2..ec87126 100644
--- a/drivers/net/can/usb/etas_es58x/es58x_fd.c
+++ b/drivers/net/can/usb/etas_es58x/es58x_fd.c
@@ -12,6 +12,7 @@
*/
#include <linux/kernel.h>
+#include <linux/units.h>
#include <asm/unaligned.h>
#include "es58x_core.h"
@@ -522,8 +523,8 @@ const struct es58x_parameters es58x_fd_param = {
* Mbps work in an optimal environment but are not recommended
* for production environment.
*/
- .bitrate_max = 8 * CAN_MBPS,
- .clock = {.freq = 80 * CAN_MHZ},
+ .bitrate_max = 8 * MEGA /* BPS */,
+ .clock = {.freq = 80 * MEGA /* Hz */},
.ctrlmode_supported = CAN_CTRLMODE_LOOPBACK | CAN_CTRLMODE_LISTENONLY |
CAN_CTRLMODE_3_SAMPLES | CAN_CTRLMODE_FD | CAN_CTRLMODE_FD_NON_ISO |
CAN_CTRLMODE_CC_LEN8_DLC | CAN_CTRLMODE_TDC_AUTO,
diff --git a/drivers/net/can/usb/gs_usb.c b/drivers/net/can/usb/gs_usb.c
index 1b400de..b487e3f 100644
--- a/drivers/net/can/usb/gs_usb.c
+++ b/drivers/net/can/usb/gs_usb.c
@@ -321,7 +321,7 @@ static void gs_usb_receive_bulk_callback(struct urb *urb)
/* device reports out of range channel id */
if (hf->channel >= GS_MAX_INTF)
- goto resubmit_urb;
+ goto device_detach;
dev = usbcan->canch[hf->channel];
@@ -357,9 +357,6 @@ static void gs_usb_receive_bulk_callback(struct urb *urb)
goto resubmit_urb;
}
- netdev->stats.tx_packets++;
- netdev->stats.tx_bytes += hf->can_dlc;
-
txc = gs_get_tx_context(dev, hf->echo_id);
/* bad devices send bad echo_ids. */
@@ -370,7 +367,9 @@ static void gs_usb_receive_bulk_callback(struct urb *urb)
goto resubmit_urb;
}
- can_get_echo_skb(netdev, hf->echo_id, NULL);
+ netdev->stats.tx_packets++;
+ netdev->stats.tx_bytes += can_get_echo_skb(netdev, hf->echo_id,
+ NULL);
gs_free_tx_context(txc);
@@ -406,6 +405,7 @@ static void gs_usb_receive_bulk_callback(struct urb *urb)
/* USB failure take down all interfaces */
if (rc == -ENODEV) {
+ device_detach:
for (rc = 0; rc < GS_MAX_INTF; rc++) {
if (usbcan->canch[rc])
netif_device_detach(usbcan->canch[rc]->netdev);
@@ -507,6 +507,8 @@ static netdev_tx_t gs_can_start_xmit(struct sk_buff *skb,
hf->echo_id = idx;
hf->channel = dev->channel;
+ hf->flags = 0;
+ hf->reserved = 0;
cf = (struct can_frame *)skb->data;
diff --git a/drivers/net/can/usb/kvaser_usb/kvaser_usb.h b/drivers/net/can/usb/kvaser_usb/kvaser_usb.h
index 390b6bd..3a49257 100644
--- a/drivers/net/can/usb/kvaser_usb/kvaser_usb.h
+++ b/drivers/net/can/usb/kvaser_usb/kvaser_usb.h
@@ -77,7 +77,6 @@ struct kvaser_usb_dev_card_data {
struct kvaser_usb_tx_urb_context {
struct kvaser_usb_net_priv *priv;
u32 echo_index;
- int dlc;
};
struct kvaser_usb {
@@ -162,8 +161,8 @@ struct kvaser_usb_dev_ops {
void (*dev_read_bulk_callback)(struct kvaser_usb *dev, void *buf,
int len);
void *(*dev_frame_to_cmd)(const struct kvaser_usb_net_priv *priv,
- const struct sk_buff *skb, int *frame_len,
- int *cmd_len, u16 transid);
+ const struct sk_buff *skb, int *cmd_len,
+ u16 transid);
};
struct kvaser_usb_dev_cfg {
diff --git a/drivers/net/can/usb/kvaser_usb/kvaser_usb_core.c b/drivers/net/can/usb/kvaser_usb/kvaser_usb_core.c
index 0cc0fc8..c4b4d3d 100644
--- a/drivers/net/can/usb/kvaser_usb/kvaser_usb_core.c
+++ b/drivers/net/can/usb/kvaser_usb/kvaser_usb_core.c
@@ -279,8 +279,6 @@ int kvaser_usb_can_rx_over_error(struct net_device *netdev)
cf->can_id |= CAN_ERR_CRTL;
cf->data[1] = CAN_ERR_CRTL_RX_OVERFLOW;
- stats->rx_packets++;
- stats->rx_bytes += cf->len;
netif_rx(skb);
return 0;
@@ -567,7 +565,7 @@ static netdev_tx_t kvaser_usb_start_xmit(struct sk_buff *skb,
goto freeurb;
}
- buf = dev->ops->dev_frame_to_cmd(priv, skb, &context->dlc, &cmd_len,
+ buf = dev->ops->dev_frame_to_cmd(priv, skb, &cmd_len,
context->echo_index);
if (!buf) {
stats->tx_dropped++;
diff --git a/drivers/net/can/usb/kvaser_usb/kvaser_usb_hydra.c b/drivers/net/can/usb/kvaser_usb/kvaser_usb_hydra.c
index dcee8dc..a26823c 100644
--- a/drivers/net/can/usb/kvaser_usb/kvaser_usb_hydra.c
+++ b/drivers/net/can/usb/kvaser_usb/kvaser_usb_hydra.c
@@ -22,6 +22,7 @@
#include <linux/spinlock.h>
#include <linux/string.h>
#include <linux/types.h>
+#include <linux/units.h>
#include <linux/usb.h>
#include <linux/can.h>
@@ -295,6 +296,7 @@ struct kvaser_cmd {
#define KVASER_USB_HYDRA_CF_FLAG_OVERRUN BIT(1)
#define KVASER_USB_HYDRA_CF_FLAG_REMOTE_FRAME BIT(4)
#define KVASER_USB_HYDRA_CF_FLAG_EXTENDED_ID BIT(5)
+#define KVASER_USB_HYDRA_CF_FLAG_TX_ACK BIT(6)
/* CAN frame flags. Used in ext_rx_can and ext_tx_can */
#define KVASER_USB_HYDRA_CF_FLAG_OSM_NACK BIT(12)
#define KVASER_USB_HYDRA_CF_FLAG_ABL BIT(13)
@@ -869,7 +871,6 @@ static void kvaser_usb_hydra_update_state(struct kvaser_usb_net_priv *priv,
struct net_device *netdev = priv->netdev;
struct can_frame *cf;
struct sk_buff *skb;
- struct net_device_stats *stats;
enum can_state new_state, old_state;
old_state = priv->can.state;
@@ -919,9 +920,6 @@ static void kvaser_usb_hydra_update_state(struct kvaser_usb_net_priv *priv,
cf->data[6] = bec->txerr;
cf->data[7] = bec->rxerr;
- stats = &netdev->stats;
- stats->rx_packets++;
- stats->rx_bytes += cf->len;
netif_rx(skb);
}
@@ -1074,8 +1072,6 @@ kvaser_usb_hydra_error_frame(struct kvaser_usb_net_priv *priv,
cf->data[6] = bec.txerr;
cf->data[7] = bec.rxerr;
- stats->rx_packets++;
- stats->rx_bytes += cf->len;
netif_rx(skb);
priv->bec.txerr = bec.txerr;
@@ -1109,8 +1105,6 @@ static void kvaser_usb_hydra_one_shot_fail(struct kvaser_usb_net_priv *priv,
}
stats->tx_errors++;
- stats->rx_packets++;
- stats->rx_bytes += cf->len;
netif_rx(skb);
}
@@ -1120,7 +1114,9 @@ static void kvaser_usb_hydra_tx_acknowledge(const struct kvaser_usb *dev,
struct kvaser_usb_tx_urb_context *context;
struct kvaser_usb_net_priv *priv;
unsigned long irq_flags;
+ unsigned int len;
bool one_shot_fail = false;
+ bool is_err_frame = false;
u16 transid = kvaser_usb_hydra_get_cmd_transid(cmd);
priv = kvaser_usb_hydra_net_priv_from_cmd(dev, cmd);
@@ -1139,24 +1135,28 @@ static void kvaser_usb_hydra_tx_acknowledge(const struct kvaser_usb *dev,
kvaser_usb_hydra_one_shot_fail(priv, cmd_ext);
one_shot_fail = true;
}
+
+ is_err_frame = flags & KVASER_USB_HYDRA_CF_FLAG_TX_ACK &&
+ flags & KVASER_USB_HYDRA_CF_FLAG_ERROR_FRAME;
}
context = &priv->tx_contexts[transid % dev->max_tx_urbs];
- if (!one_shot_fail) {
- struct net_device_stats *stats = &priv->netdev->stats;
-
- stats->tx_packets++;
- stats->tx_bytes += can_fd_dlc2len(context->dlc);
- }
spin_lock_irqsave(&priv->tx_contexts_lock, irq_flags);
- can_get_echo_skb(priv->netdev, context->echo_index, NULL);
+ len = can_get_echo_skb(priv->netdev, context->echo_index, NULL);
context->echo_index = dev->max_tx_urbs;
--priv->active_tx_contexts;
netif_wake_queue(priv->netdev);
spin_unlock_irqrestore(&priv->tx_contexts_lock, irq_flags);
+
+ if (!one_shot_fail && !is_err_frame) {
+ struct net_device_stats *stats = &priv->netdev->stats;
+
+ stats->tx_packets++;
+ stats->tx_bytes += len;
+ }
}
static void kvaser_usb_hydra_rx_msg_std(const struct kvaser_usb *dev,
@@ -1208,13 +1208,15 @@ static void kvaser_usb_hydra_rx_msg_std(const struct kvaser_usb *dev,
cf->len = can_cc_dlc2len(cmd->rx_can.dlc);
- if (flags & KVASER_USB_HYDRA_CF_FLAG_REMOTE_FRAME)
+ if (flags & KVASER_USB_HYDRA_CF_FLAG_REMOTE_FRAME) {
cf->can_id |= CAN_RTR_FLAG;
- else
+ } else {
memcpy(cf->data, cmd->rx_can.data, cf->len);
+ stats->rx_bytes += cf->len;
+ }
stats->rx_packets++;
- stats->rx_bytes += cf->len;
+
netif_rx(skb);
}
@@ -1286,13 +1288,15 @@ static void kvaser_usb_hydra_rx_msg_ext(const struct kvaser_usb *dev,
cf->len = can_cc_dlc2len(dlc);
}
- if (flags & KVASER_USB_HYDRA_CF_FLAG_REMOTE_FRAME)
+ if (flags & KVASER_USB_HYDRA_CF_FLAG_REMOTE_FRAME) {
cf->can_id |= CAN_RTR_FLAG;
- else
+ } else {
memcpy(cf->data, cmd->rx_can.kcan_payload, cf->len);
+ stats->rx_bytes += cf->len;
+ }
stats->rx_packets++;
- stats->rx_bytes += cf->len;
+
netif_rx(skb);
}
@@ -1371,8 +1375,8 @@ static void kvaser_usb_hydra_handle_cmd(const struct kvaser_usb *dev,
static void *
kvaser_usb_hydra_frame_to_cmd_ext(const struct kvaser_usb_net_priv *priv,
- const struct sk_buff *skb, int *frame_len,
- int *cmd_len, u16 transid)
+ const struct sk_buff *skb, int *cmd_len,
+ u16 transid)
{
struct kvaser_usb *dev = priv->dev;
struct kvaser_cmd_ext *cmd;
@@ -1384,8 +1388,6 @@ kvaser_usb_hydra_frame_to_cmd_ext(const struct kvaser_usb_net_priv *priv,
u32 kcan_id;
u32 kcan_header;
- *frame_len = nbr_of_bytes;
-
cmd = kcalloc(1, sizeof(struct kvaser_cmd_ext), GFP_ATOMIC);
if (!cmd)
return NULL;
@@ -1451,8 +1453,8 @@ kvaser_usb_hydra_frame_to_cmd_ext(const struct kvaser_usb_net_priv *priv,
static void *
kvaser_usb_hydra_frame_to_cmd_std(const struct kvaser_usb_net_priv *priv,
- const struct sk_buff *skb, int *frame_len,
- int *cmd_len, u16 transid)
+ const struct sk_buff *skb, int *cmd_len,
+ u16 transid)
{
struct kvaser_usb *dev = priv->dev;
struct kvaser_cmd *cmd;
@@ -1460,8 +1462,6 @@ kvaser_usb_hydra_frame_to_cmd_std(const struct kvaser_usb_net_priv *priv,
u32 flags;
u32 id;
- *frame_len = cf->len;
-
cmd = kcalloc(1, sizeof(struct kvaser_cmd), GFP_ATOMIC);
if (!cmd)
return NULL;
@@ -1495,7 +1495,7 @@ kvaser_usb_hydra_frame_to_cmd_std(const struct kvaser_usb_net_priv *priv,
cmd->tx_can.id = cpu_to_le32(id);
cmd->tx_can.flags = flags;
- memcpy(cmd->tx_can.data, cf->data, *frame_len);
+ memcpy(cmd->tx_can.data, cf->data, cf->len);
return cmd;
}
@@ -2003,17 +2003,17 @@ static void kvaser_usb_hydra_read_bulk_callback(struct kvaser_usb *dev,
static void *
kvaser_usb_hydra_frame_to_cmd(const struct kvaser_usb_net_priv *priv,
- const struct sk_buff *skb, int *frame_len,
- int *cmd_len, u16 transid)
+ const struct sk_buff *skb, int *cmd_len,
+ u16 transid)
{
void *buf;
if (priv->dev->card_data.capabilities & KVASER_USB_HYDRA_CAP_EXT_CMD)
- buf = kvaser_usb_hydra_frame_to_cmd_ext(priv, skb, frame_len,
- cmd_len, transid);
+ buf = kvaser_usb_hydra_frame_to_cmd_ext(priv, skb, cmd_len,
+ transid);
else
- buf = kvaser_usb_hydra_frame_to_cmd_std(priv, skb, frame_len,
- cmd_len, transid);
+ buf = kvaser_usb_hydra_frame_to_cmd_std(priv, skb, cmd_len,
+ transid);
return buf;
}
@@ -2040,7 +2040,7 @@ const struct kvaser_usb_dev_ops kvaser_usb_hydra_dev_ops = {
static const struct kvaser_usb_dev_cfg kvaser_usb_hydra_dev_cfg_kcan = {
.clock = {
- .freq = 80000000,
+ .freq = 80 * MEGA /* Hz */,
},
.timestamp_freq = 80,
.bittiming_const = &kvaser_usb_hydra_kcan_bittiming_c,
@@ -2049,7 +2049,7 @@ static const struct kvaser_usb_dev_cfg kvaser_usb_hydra_dev_cfg_kcan = {
static const struct kvaser_usb_dev_cfg kvaser_usb_hydra_dev_cfg_flexc = {
.clock = {
- .freq = 24000000,
+ .freq = 24 * MEGA /* Hz */,
},
.timestamp_freq = 1,
.bittiming_const = &kvaser_usb_hydra_flexc_bittiming_c,
@@ -2057,7 +2057,7 @@ static const struct kvaser_usb_dev_cfg kvaser_usb_hydra_dev_cfg_flexc = {
static const struct kvaser_usb_dev_cfg kvaser_usb_hydra_dev_cfg_rt = {
.clock = {
- .freq = 80000000,
+ .freq = 80 * MEGA /* Hz */,
},
.timestamp_freq = 24,
.bittiming_const = &kvaser_usb_hydra_rt_bittiming_c,
diff --git a/drivers/net/can/usb/kvaser_usb/kvaser_usb_leaf.c b/drivers/net/can/usb/kvaser_usb/kvaser_usb_leaf.c
index f7af1bf..c805b99 100644
--- a/drivers/net/can/usb/kvaser_usb/kvaser_usb_leaf.c
+++ b/drivers/net/can/usb/kvaser_usb/kvaser_usb_leaf.c
@@ -19,6 +19,7 @@
#include <linux/spinlock.h>
#include <linux/string.h>
#include <linux/types.h>
+#include <linux/units.h>
#include <linux/usb.h>
#include <linux/can.h>
@@ -356,7 +357,7 @@ static const struct can_bittiming_const kvaser_usb_leaf_bittiming_const = {
static const struct kvaser_usb_dev_cfg kvaser_usb_leaf_dev_cfg_8mhz = {
.clock = {
- .freq = 8000000,
+ .freq = 8 * MEGA /* Hz */,
},
.timestamp_freq = 1,
.bittiming_const = &kvaser_usb_leaf_bittiming_const,
@@ -364,7 +365,7 @@ static const struct kvaser_usb_dev_cfg kvaser_usb_leaf_dev_cfg_8mhz = {
static const struct kvaser_usb_dev_cfg kvaser_usb_leaf_dev_cfg_16mhz = {
.clock = {
- .freq = 16000000,
+ .freq = 16 * MEGA /* Hz */,
},
.timestamp_freq = 1,
.bittiming_const = &kvaser_usb_leaf_bittiming_const,
@@ -372,7 +373,7 @@ static const struct kvaser_usb_dev_cfg kvaser_usb_leaf_dev_cfg_16mhz = {
static const struct kvaser_usb_dev_cfg kvaser_usb_leaf_dev_cfg_24mhz = {
.clock = {
- .freq = 24000000,
+ .freq = 24 * MEGA /* Hz */,
},
.timestamp_freq = 1,
.bittiming_const = &kvaser_usb_leaf_bittiming_const,
@@ -380,7 +381,7 @@ static const struct kvaser_usb_dev_cfg kvaser_usb_leaf_dev_cfg_24mhz = {
static const struct kvaser_usb_dev_cfg kvaser_usb_leaf_dev_cfg_32mhz = {
.clock = {
- .freq = 32000000,
+ .freq = 32 * MEGA /* Hz */,
},
.timestamp_freq = 1,
.bittiming_const = &kvaser_usb_leaf_bittiming_const,
@@ -388,16 +389,14 @@ static const struct kvaser_usb_dev_cfg kvaser_usb_leaf_dev_cfg_32mhz = {
static void *
kvaser_usb_leaf_frame_to_cmd(const struct kvaser_usb_net_priv *priv,
- const struct sk_buff *skb, int *frame_len,
- int *cmd_len, u16 transid)
+ const struct sk_buff *skb, int *cmd_len,
+ u16 transid)
{
struct kvaser_usb *dev = priv->dev;
struct kvaser_cmd *cmd;
u8 *cmd_tx_can_flags = NULL; /* GCC */
struct can_frame *cf = (struct can_frame *)skb->data;
- *frame_len = cf->len;
-
cmd = kmalloc(sizeof(*cmd), GFP_ATOMIC);
if (cmd) {
cmd->u.tx_can.tid = transid & 0xff;
@@ -641,8 +640,6 @@ static void kvaser_usb_leaf_tx_acknowledge(const struct kvaser_usb *dev,
if (skb) {
cf->can_id |= CAN_ERR_RESTARTED;
- stats->rx_packets++;
- stats->rx_bytes += cf->len;
netif_rx(skb);
} else {
netdev_err(priv->netdev,
@@ -655,12 +652,11 @@ static void kvaser_usb_leaf_tx_acknowledge(const struct kvaser_usb *dev,
priv->can.state = CAN_STATE_ERROR_ACTIVE;
}
- stats->tx_packets++;
- stats->tx_bytes += context->dlc;
-
spin_lock_irqsave(&priv->tx_contexts_lock, flags);
- can_get_echo_skb(priv->netdev, context->echo_index, NULL);
+ stats->tx_packets++;
+ stats->tx_bytes += can_get_echo_skb(priv->netdev,
+ context->echo_index, NULL);
context->echo_index = dev->max_tx_urbs;
--priv->active_tx_contexts;
netif_wake_queue(priv->netdev);
@@ -843,8 +839,6 @@ static void kvaser_usb_leaf_rx_error(const struct kvaser_usb *dev,
cf->data[6] = es->txerr;
cf->data[7] = es->rxerr;
- stats->rx_packets++;
- stats->rx_bytes += cf->len;
netif_rx(skb);
}
@@ -1071,7 +1065,8 @@ static void kvaser_usb_leaf_rx_can_msg(const struct kvaser_usb *dev,
}
stats->rx_packets++;
- stats->rx_bytes += cf->len;
+ if (!(cf->can_id & CAN_RTR_FLAG))
+ stats->rx_bytes += cf->len;
netif_rx(skb);
}
diff --git a/drivers/net/can/usb/mcba_usb.c b/drivers/net/can/usb/mcba_usb.c
index a1a154c..77bddff 100644
--- a/drivers/net/can/usb/mcba_usb.c
+++ b/drivers/net/can/usb/mcba_usb.c
@@ -64,7 +64,6 @@
struct mcba_usb_ctx {
struct mcba_priv *priv;
u32 ndx;
- u8 dlc;
bool can;
};
@@ -184,13 +183,10 @@ static inline struct mcba_usb_ctx *mcba_usb_get_free_ctx(struct mcba_priv *priv,
ctx = &priv->tx_context[i];
ctx->ndx = i;
- if (cf) {
+ if (cf)
ctx->can = true;
- ctx->dlc = cf->len;
- } else {
+ else
ctx->can = false;
- ctx->dlc = 0;
- }
atomic_dec(&priv->free_ctx_cnt);
break;
@@ -236,10 +232,10 @@ static void mcba_usb_write_bulk_callback(struct urb *urb)
return;
netdev->stats.tx_packets++;
- netdev->stats.tx_bytes += ctx->dlc;
+ netdev->stats.tx_bytes += can_get_echo_skb(netdev, ctx->ndx,
+ NULL);
can_led_event(netdev, CAN_LED_EVENT_TX);
- can_get_echo_skb(netdev, ctx->ndx, NULL);
}
if (urb->status)
@@ -450,15 +446,16 @@ static void mcba_usb_process_can(struct mcba_priv *priv,
cf->can_id = (sid & 0xffe0) >> 5;
}
- if (msg->dlc & MCBA_DLC_RTR_MASK)
- cf->can_id |= CAN_RTR_FLAG;
-
cf->len = can_cc_dlc2len(msg->dlc & MCBA_DLC_MASK);
- memcpy(cf->data, msg->data, cf->len);
+ if (msg->dlc & MCBA_DLC_RTR_MASK) {
+ cf->can_id |= CAN_RTR_FLAG;
+ } else {
+ memcpy(cf->data, msg->data, cf->len);
+ stats->rx_bytes += cf->len;
+ }
stats->rx_packets++;
- stats->rx_bytes += cf->len;
can_led_event(priv->netdev, CAN_LED_EVENT_RX);
netif_rx(skb);
diff --git a/drivers/net/can/usb/peak_usb/pcan_usb.c b/drivers/net/can/usb/peak_usb/pcan_usb.c
index 8762187..17dc178 100644
--- a/drivers/net/can/usb/peak_usb/pcan_usb.c
+++ b/drivers/net/can/usb/peak_usb/pcan_usb.c
@@ -8,6 +8,7 @@
*
* Many thanks to Klaus Hitschler <klaus.hitschler@gmx.de>
*/
+#include <asm/unaligned.h>
#include <linux/netdevice.h>
#include <linux/usb.h>
#include <linux/module.h>
@@ -520,8 +521,6 @@ static int pcan_usb_decode_error(struct pcan_usb_msg_context *mc, u8 n,
&hwts->hwtstamp);
}
- mc->netdev->stats.rx_packets++;
- mc->netdev->stats.rx_bytes += cf->len;
netif_rx(skb);
return 0;
@@ -678,15 +677,16 @@ static int pcan_usb_decode_data(struct pcan_usb_msg_context *mc, u8 status_len)
/* Ignore next byte (client private id) if SRR bit is set */
if (can_id_flags & PCAN_USB_TX_SRR)
mc->ptr++;
+
+ /* update statistics */
+ mc->netdev->stats.rx_bytes += cf->len;
}
+ mc->netdev->stats.rx_packets++;
/* convert timestamp into kernel time */
hwts = skb_hwtstamps(skb);
peak_usb_get_ts_time(&mc->pdev->time_ref, mc->ts16, &hwts->hwtstamp);
- /* update statistics */
- mc->netdev->stats.rx_packets++;
- mc->netdev->stats.rx_bytes += cf->len;
/* push the skb */
netif_rx(skb);
diff --git a/drivers/net/can/usb/peak_usb/pcan_usb_core.c b/drivers/net/can/usb/peak_usb/pcan_usb_core.c
index 6107fef..b850ff8 100644
--- a/drivers/net/can/usb/peak_usb/pcan_usb_core.c
+++ b/drivers/net/can/usb/peak_usb/pcan_usb_core.c
@@ -291,6 +291,7 @@ static void peak_usb_write_bulk_callback(struct urb *urb)
struct peak_tx_urb_context *context = urb->context;
struct peak_usb_device *dev;
struct net_device *netdev;
+ int tx_bytes;
BUG_ON(!context);
@@ -305,10 +306,6 @@ static void peak_usb_write_bulk_callback(struct urb *urb)
/* check tx status */
switch (urb->status) {
case 0:
- /* transmission complete */
- netdev->stats.tx_packets++;
- netdev->stats.tx_bytes += context->data_len;
-
/* prevent tx timeout */
netif_trans_update(netdev);
break;
@@ -327,12 +324,17 @@ static void peak_usb_write_bulk_callback(struct urb *urb)
}
/* should always release echo skb and corresponding context */
- can_get_echo_skb(netdev, context->echo_index, NULL);
+ tx_bytes = can_get_echo_skb(netdev, context->echo_index, NULL);
context->echo_index = PCAN_USB_MAX_TX_URBS;
- /* do wakeup tx queue in case of success only */
- if (!urb->status)
+ if (!urb->status) {
+ /* transmission complete */
+ netdev->stats.tx_packets++;
+ netdev->stats.tx_bytes += tx_bytes;
+
+ /* do wakeup tx queue in case of success only */
netif_wake_queue(netdev);
+ }
}
/*
@@ -344,7 +346,6 @@ static netdev_tx_t peak_usb_ndo_start_xmit(struct sk_buff *skb,
struct peak_usb_device *dev = netdev_priv(netdev);
struct peak_tx_urb_context *context = NULL;
struct net_device_stats *stats = &netdev->stats;
- struct canfd_frame *cfd = (struct canfd_frame *)skb->data;
struct urb *urb;
u8 *obuf;
int i, err;
@@ -378,9 +379,6 @@ static netdev_tx_t peak_usb_ndo_start_xmit(struct sk_buff *skb,
context->echo_index = i;
- /* Note: this works with CANFD frames too */
- context->data_len = cfd->len;
-
usb_anchor_urb(urb, &dev->tx_submitted);
can_put_echo_skb(skb, netdev, context->echo_index, 0);
diff --git a/drivers/net/can/usb/peak_usb/pcan_usb_core.h b/drivers/net/can/usb/peak_usb/pcan_usb_core.h
index daa19f5..f60af57 100644
--- a/drivers/net/can/usb/peak_usb/pcan_usb_core.h
+++ b/drivers/net/can/usb/peak_usb/pcan_usb_core.h
@@ -99,7 +99,6 @@ struct peak_time_ref {
struct peak_tx_urb_context {
struct peak_usb_device *dev;
u32 echo_index;
- u8 data_len;
struct urb *urb;
};
diff --git a/drivers/net/can/usb/peak_usb/pcan_usb_fd.c b/drivers/net/can/usb/peak_usb/pcan_usb_fd.c
index 6bd1254..65487ec 100644
--- a/drivers/net/can/usb/peak_usb/pcan_usb_fd.c
+++ b/drivers/net/can/usb/peak_usb/pcan_usb_fd.c
@@ -507,13 +507,13 @@ static int pcan_usb_fd_decode_canmsg(struct pcan_usb_fd_if *usb_if,
if (rx_msg_flags & PUCAN_MSG_EXT_ID)
cfd->can_id |= CAN_EFF_FLAG;
- if (rx_msg_flags & PUCAN_MSG_RTR)
+ if (rx_msg_flags & PUCAN_MSG_RTR) {
cfd->can_id |= CAN_RTR_FLAG;
- else
+ } else {
memcpy(cfd->data, rm->d, cfd->len);
-
+ netdev->stats.rx_bytes += cfd->len;
+ }
netdev->stats.rx_packets++;
- netdev->stats.rx_bytes += cfd->len;
peak_usb_netif_rx_64(skb, le32_to_cpu(rm->ts_low),
le32_to_cpu(rm->ts_high));
@@ -577,9 +577,6 @@ static int pcan_usb_fd_decode_status(struct pcan_usb_fd_if *usb_if,
if (!skb)
return -ENOMEM;
- netdev->stats.rx_packets++;
- netdev->stats.rx_bytes += cf->len;
-
peak_usb_netif_rx_64(skb, le32_to_cpu(sm->ts_low),
le32_to_cpu(sm->ts_high));
diff --git a/drivers/net/can/usb/peak_usb/pcan_usb_pro.c b/drivers/net/can/usb/peak_usb/pcan_usb_pro.c
index 858ab22..ebe087f 100644
--- a/drivers/net/can/usb/peak_usb/pcan_usb_pro.c
+++ b/drivers/net/can/usb/peak_usb/pcan_usb_pro.c
@@ -536,17 +536,19 @@ static int pcan_usb_pro_handle_canmsg(struct pcan_usb_pro_interface *usb_if,
if (rx->flags & PCAN_USBPRO_EXT)
can_frame->can_id |= CAN_EFF_FLAG;
- if (rx->flags & PCAN_USBPRO_RTR)
+ if (rx->flags & PCAN_USBPRO_RTR) {
can_frame->can_id |= CAN_RTR_FLAG;
- else
+ } else {
memcpy(can_frame->data, rx->data, can_frame->len);
+ netdev->stats.rx_bytes += can_frame->len;
+ }
+ netdev->stats.rx_packets++;
+
hwts = skb_hwtstamps(skb);
peak_usb_get_ts_time(&usb_if->time_ref, le32_to_cpu(rx->ts32),
&hwts->hwtstamp);
- netdev->stats.rx_packets++;
- netdev->stats.rx_bytes += can_frame->len;
netif_rx(skb);
return 0;
@@ -660,8 +662,6 @@ static int pcan_usb_pro_handle_error(struct pcan_usb_pro_interface *usb_if,
hwts = skb_hwtstamps(skb);
peak_usb_get_ts_time(&usb_if->time_ref, le32_to_cpu(er->ts32), &hwts->hwtstamp);
- netdev->stats.rx_packets++;
- netdev->stats.rx_bytes += can_frame->len;
netif_rx(skb);
return 0;
diff --git a/drivers/net/can/usb/ucan.c b/drivers/net/can/usb/ucan.c
index 1679cbe..c7c41d1 100644
--- a/drivers/net/can/usb/ucan.c
+++ b/drivers/net/can/usb/ucan.c
@@ -259,7 +259,6 @@ struct ucan_priv;
/* Context Information for transmission URBs */
struct ucan_urb_context {
struct ucan_priv *up;
- u8 dlc;
bool allocated;
};
@@ -621,8 +620,11 @@ static void ucan_rx_can_msg(struct ucan_priv *up, struct ucan_message_in *m)
memcpy(cf->data, m->msg.can_msg.data, cf->len);
/* don't count error frames as real packets */
- stats->rx_packets++;
- stats->rx_bytes += cf->len;
+ if (!(cf->can_id & CAN_ERR_FLAG)) {
+ stats->rx_packets++;
+ if (!(cf->can_id & CAN_RTR_FLAG))
+ stats->rx_bytes += cf->len;
+ }
/* pass it to Linux */
netif_rx(skb);
@@ -634,7 +636,7 @@ static void ucan_tx_complete_msg(struct ucan_priv *up,
{
unsigned long flags;
u16 count, i;
- u8 echo_index, dlc;
+ u8 echo_index;
u16 len = le16_to_cpu(m->len);
struct ucan_urb_context *context;
@@ -658,7 +660,6 @@ static void ucan_tx_complete_msg(struct ucan_priv *up,
/* gather information from the context */
context = &up->context_array[echo_index];
- dlc = READ_ONCE(context->dlc);
/* Release context and restart queue if necessary.
* Also check if the context was allocated
@@ -671,8 +672,8 @@ static void ucan_tx_complete_msg(struct ucan_priv *up,
UCAN_TX_COMPLETE_SUCCESS) {
/* update statistics */
up->netdev->stats.tx_packets++;
- up->netdev->stats.tx_bytes += dlc;
- can_get_echo_skb(up->netdev, echo_index, NULL);
+ up->netdev->stats.tx_bytes +=
+ can_get_echo_skb(up->netdev, echo_index, NULL);
} else {
up->netdev->stats.tx_dropped++;
can_free_echo_skb(up->netdev, echo_index, NULL);
@@ -1086,8 +1087,6 @@ static struct urb *ucan_prepare_tx_urb(struct ucan_priv *up,
}
m->len = cpu_to_le16(mlen);
- context->dlc = cf->len;
-
m->subtype = echo_index;
/* build the urb */
diff --git a/drivers/net/can/usb/usb_8dev.c b/drivers/net/can/usb/usb_8dev.c
index d1b83bd..431af1e 100644
--- a/drivers/net/can/usb/usb_8dev.c
+++ b/drivers/net/can/usb/usb_8dev.c
@@ -114,15 +114,12 @@ struct usb_8dev_tx_urb_context {
struct usb_8dev_priv *priv;
u32 echo_index;
- u8 dlc;
};
/* Structure to hold all of our device specific stuff */
struct usb_8dev_priv {
struct can_priv can; /* must be the first member */
- struct sk_buff *echo_skb[MAX_TX_URBS];
-
struct usb_device *udev;
struct net_device *netdev;
@@ -449,8 +446,6 @@ static void usb_8dev_rx_err_msg(struct usb_8dev_priv *priv,
priv->bec.txerr = txerr;
priv->bec.rxerr = rxerr;
- stats->rx_packets++;
- stats->rx_bytes += cf->len;
netif_rx(skb);
}
@@ -476,13 +471,14 @@ static void usb_8dev_rx_can_msg(struct usb_8dev_priv *priv,
if (msg->flags & USB_8DEV_EXTID)
cf->can_id |= CAN_EFF_FLAG;
- if (msg->flags & USB_8DEV_RTR)
+ if (msg->flags & USB_8DEV_RTR) {
cf->can_id |= CAN_RTR_FLAG;
- else
+ } else {
memcpy(cf->data, msg->data, cf->len);
-
+ stats->rx_bytes += cf->len;
+ }
stats->rx_packets++;
- stats->rx_bytes += cf->len;
+
netif_rx(skb);
can_led_event(priv->netdev, CAN_LED_EVENT_RX);
@@ -584,9 +580,7 @@ static void usb_8dev_write_bulk_callback(struct urb *urb)
urb->status);
netdev->stats.tx_packets++;
- netdev->stats.tx_bytes += context->dlc;
-
- can_get_echo_skb(netdev, context->echo_index, NULL);
+ netdev->stats.tx_bytes += can_get_echo_skb(netdev, context->echo_index, NULL);
can_led_event(netdev, CAN_LED_EVENT_TX);
@@ -657,7 +651,6 @@ static netdev_tx_t usb_8dev_start_xmit(struct sk_buff *skb,
context->priv = priv;
context->echo_index = i;
- context->dlc = cf->len;
usb_fill_bulk_urb(urb, priv->udev,
usb_sndbulkpipe(priv->udev, USB_8DEV_ENDP_DATA_TX),
diff --git a/drivers/net/can/vcan.c b/drivers/net/can/vcan.c
index 067705e..c42f188 100644
--- a/drivers/net/can/vcan.c
+++ b/drivers/net/can/vcan.c
@@ -87,13 +87,14 @@ static netdev_tx_t vcan_tx(struct sk_buff *skb, struct net_device *dev)
{
struct canfd_frame *cfd = (struct canfd_frame *)skb->data;
struct net_device_stats *stats = &dev->stats;
- int loop;
+ int loop, len;
if (can_dropped_invalid_skb(dev, skb))
return NETDEV_TX_OK;
+ len = cfd->can_id & CAN_RTR_FLAG ? 0 : cfd->len;
stats->tx_packets++;
- stats->tx_bytes += cfd->len;
+ stats->tx_bytes += len;
/* set flag whether this packet has to be looped back */
loop = skb->pkt_type == PACKET_LOOPBACK;
@@ -105,7 +106,7 @@ static netdev_tx_t vcan_tx(struct sk_buff *skb, struct net_device *dev)
* CAN core already did the echo for us
*/
stats->rx_packets++;
- stats->rx_bytes += cfd->len;
+ stats->rx_bytes += len;
}
consume_skb(skb);
return NETDEV_TX_OK;
diff --git a/drivers/net/can/vxcan.c b/drivers/net/can/vxcan.c
index 8861a7d..47ccc15 100644
--- a/drivers/net/can/vxcan.c
+++ b/drivers/net/can/vxcan.c
@@ -62,7 +62,7 @@ static netdev_tx_t vxcan_xmit(struct sk_buff *skb, struct net_device *dev)
skb->dev = peer;
skb->ip_summed = CHECKSUM_UNNECESSARY;
- len = cfd->len;
+ len = cfd->can_id & CAN_RTR_FLAG ? 0 : cfd->len;
if (netif_rx_ni(skb) == NET_RX_SUCCESS) {
srcstats->tx_packets++;
srcstats->tx_bytes += len;
diff --git a/drivers/net/can/xilinx_can.c b/drivers/net/can/xilinx_can.c
index e2b15d2..1674b56 100644
--- a/drivers/net/can/xilinx_can.c
+++ b/drivers/net/can/xilinx_can.c
@@ -787,10 +787,11 @@ static int xcan_rx(struct net_device *ndev, int frame_base)
*(__be32 *)(cf->data) = cpu_to_be32(data[0]);
if (cf->len > 4)
*(__be32 *)(cf->data + 4) = cpu_to_be32(data[1]);
- }
- stats->rx_bytes += cf->len;
+ stats->rx_bytes += cf->len;
+ }
stats->rx_packets++;
+
netif_receive_skb(skb);
return 1;
@@ -871,8 +872,11 @@ static int xcanfd_rx(struct net_device *ndev, int frame_base)
*(__be32 *)(cf->data + i) = cpu_to_be32(data[0]);
}
}
- stats->rx_bytes += cf->len;
+
+ if (!(cf->can_id & CAN_RTR_FLAG))
+ stats->rx_bytes += cf->len;
stats->rx_packets++;
+
netif_receive_skb(skb);
return 1;
@@ -965,13 +969,8 @@ static void xcan_update_error_state_after_rxtx(struct net_device *ndev)
xcan_set_error_state(ndev, new_state, skb ? cf : NULL);
- if (skb) {
- struct net_device_stats *stats = &ndev->stats;
-
- stats->rx_packets++;
- stats->rx_bytes += cf->len;
+ if (skb)
netif_rx(skb);
- }
}
}
@@ -1095,8 +1094,6 @@ static void xcan_err_interrupt(struct net_device *ndev, u32 isr)
if (skb) {
skb_cf->can_id |= cf.can_id;
memcpy(skb_cf->data, cf.data, CAN_ERR_DLC);
- stats->rx_packets++;
- stats->rx_bytes += CAN_ERR_DLC;
netif_rx(skb);
}
}
@@ -1761,7 +1758,12 @@ static int xcan_probe(struct platform_device *pdev)
spin_lock_init(&priv->tx_lock);
/* Get IRQ for the device */
- ndev->irq = platform_get_irq(pdev, 0);
+ ret = platform_get_irq(pdev, 0);
+ if (ret < 0)
+ goto err_free;
+
+ ndev->irq = ret;
+
ndev->flags |= IFF_ECHO; /* We support local echo */
platform_set_drvdata(pdev, ndev);
diff --git a/drivers/net/dsa/b53/b53_common.c b/drivers/net/dsa/b53/b53_common.c
index af47619..3867f3d 100644
--- a/drivers/net/dsa/b53/b53_common.c
+++ b/drivers/net/dsa/b53/b53_common.c
@@ -1860,7 +1860,8 @@ int b53_mdb_del(struct dsa_switch *ds, int port,
}
EXPORT_SYMBOL(b53_mdb_del);
-int b53_br_join(struct dsa_switch *ds, int port, struct net_device *br)
+int b53_br_join(struct dsa_switch *ds, int port, struct dsa_bridge bridge,
+ bool *tx_fwd_offload)
{
struct b53_device *dev = ds->priv;
s8 cpu_port = dsa_to_port(ds, port)->cpu_dp->index;
@@ -1887,7 +1888,7 @@ int b53_br_join(struct dsa_switch *ds, int port, struct net_device *br)
b53_read16(dev, B53_PVLAN_PAGE, B53_PVLAN_PORT_MASK(port), &pvlan);
b53_for_each_port(dev, i) {
- if (dsa_to_port(ds, i)->bridge_dev != br)
+ if (!dsa_port_offloads_bridge(dsa_to_port(ds, i), &bridge))
continue;
/* Add this local port to the remote port VLAN control
@@ -1911,7 +1912,7 @@ int b53_br_join(struct dsa_switch *ds, int port, struct net_device *br)
}
EXPORT_SYMBOL(b53_br_join);
-void b53_br_leave(struct dsa_switch *ds, int port, struct net_device *br)
+void b53_br_leave(struct dsa_switch *ds, int port, struct dsa_bridge bridge)
{
struct b53_device *dev = ds->priv;
struct b53_vlan *vl = &dev->vlans[0];
@@ -1923,7 +1924,7 @@ void b53_br_leave(struct dsa_switch *ds, int port, struct net_device *br)
b53_for_each_port(dev, i) {
/* Don't touch the remaining ports */
- if (dsa_to_port(ds, i)->bridge_dev != br)
+ if (!dsa_port_offloads_bridge(dsa_to_port(ds, i), &bridge))
continue;
b53_read16(dev, B53_PVLAN_PAGE, B53_PVLAN_PORT_MASK(i), ®);
diff --git a/drivers/net/dsa/b53/b53_priv.h b/drivers/net/dsa/b53/b53_priv.h
index 579da74..b41dc8a 100644
--- a/drivers/net/dsa/b53/b53_priv.h
+++ b/drivers/net/dsa/b53/b53_priv.h
@@ -324,8 +324,9 @@ void b53_get_strings(struct dsa_switch *ds, int port, u32 stringset,
void b53_get_ethtool_stats(struct dsa_switch *ds, int port, uint64_t *data);
int b53_get_sset_count(struct dsa_switch *ds, int port, int sset);
void b53_get_ethtool_phy_stats(struct dsa_switch *ds, int port, uint64_t *data);
-int b53_br_join(struct dsa_switch *ds, int port, struct net_device *bridge);
-void b53_br_leave(struct dsa_switch *ds, int port, struct net_device *bridge);
+int b53_br_join(struct dsa_switch *ds, int port, struct dsa_bridge bridge,
+ bool *tx_fwd_offload);
+void b53_br_leave(struct dsa_switch *ds, int port, struct dsa_bridge bridge);
void b53_br_set_stp_state(struct dsa_switch *ds, int port, u8 state);
void b53_br_fast_age(struct dsa_switch *ds, int port);
int b53_br_flags_pre(struct dsa_switch *ds, int port,
diff --git a/drivers/net/dsa/bcm_sf2.c b/drivers/net/dsa/bcm_sf2.c
index 13aa43b..33499fc 100644
--- a/drivers/net/dsa/bcm_sf2.c
+++ b/drivers/net/dsa/bcm_sf2.c
@@ -62,6 +62,38 @@ static u16 bcm_sf2_reg_rgmii_cntrl(struct bcm_sf2_priv *priv, int port)
return REG_SWITCH_STATUS;
}
+static u16 bcm_sf2_reg_led_base(struct bcm_sf2_priv *priv, int port)
+{
+ switch (port) {
+ case 0:
+ return REG_LED_0_CNTRL;
+ case 1:
+ return REG_LED_1_CNTRL;
+ case 2:
+ return REG_LED_2_CNTRL;
+ }
+
+ switch (priv->type) {
+ case BCM4908_DEVICE_ID:
+ switch (port) {
+ case 3:
+ return REG_LED_3_CNTRL;
+ case 7:
+ return REG_LED_4_CNTRL;
+ default:
+ break;
+ }
+ break;
+ default:
+ break;
+ }
+
+ WARN_ONCE(1, "Unsupported port %d\n", port);
+
+ /* RO fallback reg */
+ return REG_SWITCH_STATUS;
+}
+
/* Return the number of active ports, not counting the IMP (CPU) port */
static unsigned int bcm_sf2_num_active_ports(struct dsa_switch *ds)
{
@@ -187,9 +219,14 @@ static void bcm_sf2_gphy_enable_set(struct dsa_switch *ds, bool enable)
/* Use PHY-driven LED signaling */
if (!enable) {
- reg = reg_readl(priv, REG_LED_CNTRL(0));
- reg |= SPDLNK_SRC_SEL;
- reg_writel(priv, reg, REG_LED_CNTRL(0));
+ u16 led_ctrl = bcm_sf2_reg_led_base(priv, 0);
+
+ if (priv->type == BCM7278_DEVICE_ID ||
+ priv->type == BCM7445_DEVICE_ID) {
+ reg = reg_led_readl(priv, led_ctrl, 0);
+ reg |= LED_CNTRL_SPDLNK_SRC_SEL;
+ reg_led_writel(priv, reg, led_ctrl, 0);
+ }
}
}
@@ -1232,9 +1269,14 @@ static const u16 bcm_sf2_4908_reg_offsets[] = {
[REG_SPHY_CNTRL] = 0x24,
[REG_CROSSBAR] = 0xc8,
[REG_RGMII_11_CNTRL] = 0x014c,
- [REG_LED_0_CNTRL] = 0x40,
- [REG_LED_1_CNTRL] = 0x4c,
- [REG_LED_2_CNTRL] = 0x58,
+ [REG_LED_0_CNTRL] = 0x40,
+ [REG_LED_1_CNTRL] = 0x4c,
+ [REG_LED_2_CNTRL] = 0x58,
+ [REG_LED_3_CNTRL] = 0x64,
+ [REG_LED_4_CNTRL] = 0x88,
+ [REG_LED_5_CNTRL] = 0xa0,
+ [REG_LED_AGGREGATE_CTRL] = 0xb8,
+
};
static const struct bcm_sf2_of_data bcm_sf2_4908_data = {
diff --git a/drivers/net/dsa/bcm_sf2.h b/drivers/net/dsa/bcm_sf2.h
index 0d484020..00afc94 100644
--- a/drivers/net/dsa/bcm_sf2.h
+++ b/drivers/net/dsa/bcm_sf2.h
@@ -210,6 +210,16 @@ SF2_IO_MACRO(acb);
SWITCH_INTR_L2(0);
SWITCH_INTR_L2(1);
+static inline u32 reg_led_readl(struct bcm_sf2_priv *priv, u16 off, u16 reg)
+{
+ return readl_relaxed(priv->reg + priv->reg_offsets[off] + reg);
+}
+
+static inline void reg_led_writel(struct bcm_sf2_priv *priv, u32 val, u16 off, u16 reg)
+{
+ writel_relaxed(val, priv->reg + priv->reg_offsets[off] + reg);
+}
+
/* RXNFC */
int bcm_sf2_get_rxnfc(struct dsa_switch *ds, int port,
struct ethtool_rxnfc *nfc, u32 *rule_locs);
diff --git a/drivers/net/dsa/bcm_sf2_regs.h b/drivers/net/dsa/bcm_sf2_regs.h
index 7bffc80..da0dedb 100644
--- a/drivers/net/dsa/bcm_sf2_regs.h
+++ b/drivers/net/dsa/bcm_sf2_regs.h
@@ -25,6 +25,10 @@ enum bcm_sf2_reg_offs {
REG_LED_0_CNTRL,
REG_LED_1_CNTRL,
REG_LED_2_CNTRL,
+ REG_LED_3_CNTRL,
+ REG_LED_4_CNTRL,
+ REG_LED_5_CNTRL,
+ REG_LED_AGGREGATE_CTRL,
REG_SWITCH_REG_MAX,
};
@@ -56,6 +60,63 @@ enum bcm_sf2_reg_offs {
#define CROSSBAR_BCM4908_EXT_GPHY4 1
#define CROSSBAR_BCM4908_EXT_RGMII 2
+/* Relative to REG_LED_*_CNTRL (BCM7278, BCM7445) */
+#define LED_CNTRL_NO_LINK_ENCODE_SHIFT 0
+#define LED_CNTRL_M10_ENCODE_SHIFT 2
+#define LED_CNTRL_M100_ENCODE_SHIFT 4
+#define LED_CNTRL_M1000_ENCODE_SHIFT 6
+#define LED_CNTRL_SEL_NO_LINK_ENCODE_SHIFT 8
+#define LED_CNTRL_SEL_10M_ENCODE_SHIFT 10
+#define LED_CNTRL_SEL_100M_ENCODE_SHIFT 12
+#define LED_CNTRL_SEL_1000M_ENCODE_SHIFT 14
+#define LED_CNTRL_RX_DV_EN (1 << 16)
+#define LED_CNTRL_TX_EN_EN (1 << 17)
+#define LED_CNTRL_SPDLNK_LED0_ACT_SEL_SHIFT 18
+#define LED_CNTRL_SPDLNK_LED1_ACT_SEL_SHIFT 20
+#define LED_CNTRL_ACT_LED_ACT_SEL_SHIFT 22
+#define LED_CNTRL_SPDLNK_SRC_SEL (1 << 24)
+#define LED_CNTRL_SPDLNK_LED0_ACT_POL_SEL (1 << 25)
+#define LED_CNTRL_SPDLNK_LED1_ACT_POL_SEL (1 << 26)
+#define LED_CNTRL_ACT_LED_POL_SEL (1 << 27)
+#define LED_CNTRL_MASK 0x3
+
+/* Register relative to REG_LED_*_CNTRL (BCM4908) */
+#define REG_LED_CTRL 0x0
+#define LED_CTRL_RX_ACT_EN 0x00000001
+#define LED_CTRL_TX_ACT_EN 0x00000002
+#define LED_CTRL_SPDLNK_LED0_ACT_SEL 0x00000004
+#define LED_CTRL_SPDLNK_LED1_ACT_SEL 0x00000008
+#define LED_CTRL_SPDLNK_LED2_ACT_SEL 0x00000010
+#define LED_CTRL_ACT_LED_ACT_SEL 0x00000020
+#define LED_CTRL_SPDLNK_LED0_ACT_POL_SEL 0x00000040
+#define LED_CTRL_SPDLNK_LED1_ACT_POL_SEL 0x00000080
+#define LED_CTRL_SPDLNK_LED2_ACT_POL_SEL 0x00000100
+#define LED_CTRL_ACT_LED_POL_SEL 0x00000200
+#define LED_CTRL_LED_SPD_OVRD 0x00001c00
+#define LED_CTRL_LNK_STATUS_OVRD 0x00002000
+#define LED_CTRL_SPD_OVRD_EN 0x00004000
+#define LED_CTRL_LNK_OVRD_EN 0x00008000
+
+/* Register relative to REG_LED_*_CNTRL (BCM4908) */
+#define REG_LED_LINK_SPEED_ENC_SEL 0x4
+#define LED_LINK_SPEED_ENC_SEL_NO_LINK_SHIFT 0
+#define LED_LINK_SPEED_ENC_SEL_10M_SHIFT 3
+#define LED_LINK_SPEED_ENC_SEL_100M_SHIFT 6
+#define LED_LINK_SPEED_ENC_SEL_1000M_SHIFT 9
+#define LED_LINK_SPEED_ENC_SEL_2500M_SHIFT 12
+#define LED_LINK_SPEED_ENC_SEL_10G_SHIFT 15
+#define LED_LINK_SPEED_ENC_SEL_MASK 0x7
+
+/* Register relative to REG_LED_*_CNTRL (BCM4908) */
+#define REG_LED_LINK_SPEED_ENC 0x8
+#define LED_LINK_SPEED_ENC_NO_LINK_SHIFT 0
+#define LED_LINK_SPEED_ENC_M10_SHIFT 3
+#define LED_LINK_SPEED_ENC_M100_SHIFT 6
+#define LED_LINK_SPEED_ENC_M1000_SHIFT 9
+#define LED_LINK_SPEED_ENC_M2500_SHIFT 12
+#define LED_LINK_SPEED_ENC_M10G_SHIFT 15
+#define LED_LINK_SPEED_ENC_MASK 0x7
+
/* Relative to REG_RGMII_CNTRL */
#define RGMII_MODE_EN (1 << 0)
#define ID_MODE_DIS (1 << 1)
@@ -73,10 +134,6 @@ enum bcm_sf2_reg_offs {
#define LPI_COUNT_SHIFT 9
#define LPI_COUNT_MASK 0x3F
-#define REG_LED_CNTRL(x) (REG_LED_0_CNTRL + (x))
-
-#define SPDLNK_SRC_SEL (1 << 24)
-
/* Register set relative to 'INTRL2_0' and 'INTRL2_1' */
#define INTRL2_CPU_STATUS 0x00
#define INTRL2_CPU_SET 0x04
diff --git a/drivers/net/dsa/dsa_loop.c b/drivers/net/dsa/dsa_loop.c
index e638e3e..33daaf1 100644
--- a/drivers/net/dsa/dsa_loop.c
+++ b/drivers/net/dsa/dsa_loop.c
@@ -167,19 +167,20 @@ static int dsa_loop_phy_write(struct dsa_switch *ds, int port,
}
static int dsa_loop_port_bridge_join(struct dsa_switch *ds, int port,
- struct net_device *bridge)
+ struct dsa_bridge bridge,
+ bool *tx_fwd_offload)
{
dev_dbg(ds->dev, "%s: port: %d, bridge: %s\n",
- __func__, port, bridge->name);
+ __func__, port, bridge.dev->name);
return 0;
}
static void dsa_loop_port_bridge_leave(struct dsa_switch *ds, int port,
- struct net_device *bridge)
+ struct dsa_bridge bridge)
{
dev_dbg(ds->dev, "%s: port: %d, bridge: %s\n",
- __func__, port, bridge->name);
+ __func__, port, bridge.dev->name);
}
static void dsa_loop_port_stp_state_set(struct dsa_switch *ds, int port,
diff --git a/drivers/net/dsa/hirschmann/hellcreek.c b/drivers/net/dsa/hirschmann/hellcreek.c
index 4e0b53d9..726f267 100644
--- a/drivers/net/dsa/hirschmann/hellcreek.c
+++ b/drivers/net/dsa/hirschmann/hellcreek.c
@@ -674,7 +674,8 @@ static int hellcreek_bridge_flags(struct dsa_switch *ds, int port,
}
static int hellcreek_port_bridge_join(struct dsa_switch *ds, int port,
- struct net_device *br)
+ struct dsa_bridge bridge,
+ bool *tx_fwd_offload)
{
struct hellcreek *hellcreek = ds->priv;
@@ -691,7 +692,7 @@ static int hellcreek_port_bridge_join(struct dsa_switch *ds, int port,
}
static void hellcreek_port_bridge_leave(struct dsa_switch *ds, int port,
- struct net_device *br)
+ struct dsa_bridge bridge)
{
struct hellcreek *hellcreek = ds->priv;
@@ -710,8 +711,9 @@ static int __hellcreek_fdb_add(struct hellcreek *hellcreek,
u16 meta = 0;
dev_dbg(hellcreek->dev, "Add static FDB entry: MAC=%pM, MASK=0x%02x, "
- "OBT=%d, REPRIO_EN=%d, PRIO=%d\n", entry->mac, entry->portmask,
- entry->is_obt, entry->reprio_en, entry->reprio_tc);
+ "OBT=%d, PASS_BLOCKED=%d, REPRIO_EN=%d, PRIO=%d\n", entry->mac,
+ entry->portmask, entry->is_obt, entry->pass_blocked,
+ entry->reprio_en, entry->reprio_tc);
/* Add mac address */
hellcreek_write(hellcreek, entry->mac[1] | (entry->mac[0] << 8), HR_FDBWDH);
@@ -722,6 +724,8 @@ static int __hellcreek_fdb_add(struct hellcreek *hellcreek,
meta |= entry->portmask << HR_FDBWRM0_PORTMASK_SHIFT;
if (entry->is_obt)
meta |= HR_FDBWRM0_OBT;
+ if (entry->pass_blocked)
+ meta |= HR_FDBWRM0_PASS_BLOCKED;
if (entry->reprio_en) {
meta |= HR_FDBWRM0_REPRIO_EN;
meta |= entry->reprio_tc << HR_FDBWRM0_REPRIO_TC_SHIFT;
@@ -1049,7 +1053,7 @@ static void hellcreek_setup_tc_identity_mapping(struct hellcreek *hellcreek)
static int hellcreek_setup_fdb(struct hellcreek *hellcreek)
{
- static struct hellcreek_fdb_entry ptp = {
+ static struct hellcreek_fdb_entry l2_ptp = {
/* MAC: 01-1B-19-00-00-00 */
.mac = { 0x01, 0x1b, 0x19, 0x00, 0x00, 0x00 },
.portmask = 0x03, /* Management ports */
@@ -1060,24 +1064,94 @@ static int hellcreek_setup_fdb(struct hellcreek *hellcreek)
.reprio_tc = 6, /* TC: 6 as per IEEE 802.1AS */
.reprio_en = 1,
};
- static struct hellcreek_fdb_entry p2p = {
- /* MAC: 01-80-C2-00-00-0E */
- .mac = { 0x01, 0x80, 0xc2, 0x00, 0x00, 0x0e },
+ static struct hellcreek_fdb_entry udp4_ptp = {
+ /* MAC: 01-00-5E-00-01-81 */
+ .mac = { 0x01, 0x00, 0x5e, 0x00, 0x01, 0x81 },
.portmask = 0x03, /* Management ports */
.age = 0,
.is_obt = 0,
.pass_blocked = 0,
.is_static = 1,
+ .reprio_tc = 6,
+ .reprio_en = 1,
+ };
+ static struct hellcreek_fdb_entry udp6_ptp = {
+ /* MAC: 33-33-00-00-01-81 */
+ .mac = { 0x33, 0x33, 0x00, 0x00, 0x01, 0x81 },
+ .portmask = 0x03, /* Management ports */
+ .age = 0,
+ .is_obt = 0,
+ .pass_blocked = 0,
+ .is_static = 1,
+ .reprio_tc = 6,
+ .reprio_en = 1,
+ };
+ static struct hellcreek_fdb_entry l2_p2p = {
+ /* MAC: 01-80-C2-00-00-0E */
+ .mac = { 0x01, 0x80, 0xc2, 0x00, 0x00, 0x0e },
+ .portmask = 0x03, /* Management ports */
+ .age = 0,
+ .is_obt = 0,
+ .pass_blocked = 1,
+ .is_static = 1,
.reprio_tc = 6, /* TC: 6 as per IEEE 802.1AS */
.reprio_en = 1,
};
+ static struct hellcreek_fdb_entry udp4_p2p = {
+ /* MAC: 01-00-5E-00-00-6B */
+ .mac = { 0x01, 0x00, 0x5e, 0x00, 0x00, 0x6b },
+ .portmask = 0x03, /* Management ports */
+ .age = 0,
+ .is_obt = 0,
+ .pass_blocked = 1,
+ .is_static = 1,
+ .reprio_tc = 6,
+ .reprio_en = 1,
+ };
+ static struct hellcreek_fdb_entry udp6_p2p = {
+ /* MAC: 33-33-00-00-00-6B */
+ .mac = { 0x33, 0x33, 0x00, 0x00, 0x00, 0x6b },
+ .portmask = 0x03, /* Management ports */
+ .age = 0,
+ .is_obt = 0,
+ .pass_blocked = 1,
+ .is_static = 1,
+ .reprio_tc = 6,
+ .reprio_en = 1,
+ };
+ static struct hellcreek_fdb_entry stp = {
+ /* MAC: 01-80-C2-00-00-00 */
+ .mac = { 0x01, 0x80, 0xc2, 0x00, 0x00, 0x00 },
+ .portmask = 0x03, /* Management ports */
+ .age = 0,
+ .is_obt = 0,
+ .pass_blocked = 1,
+ .is_static = 1,
+ .reprio_tc = 6,
+ .reprio_en = 1,
+ };
int ret;
mutex_lock(&hellcreek->reg_lock);
- ret = __hellcreek_fdb_add(hellcreek, &ptp);
+ ret = __hellcreek_fdb_add(hellcreek, &l2_ptp);
if (ret)
goto out;
- ret = __hellcreek_fdb_add(hellcreek, &p2p);
+ ret = __hellcreek_fdb_add(hellcreek, &udp4_ptp);
+ if (ret)
+ goto out;
+ ret = __hellcreek_fdb_add(hellcreek, &udp6_ptp);
+ if (ret)
+ goto out;
+ ret = __hellcreek_fdb_add(hellcreek, &l2_p2p);
+ if (ret)
+ goto out;
+ ret = __hellcreek_fdb_add(hellcreek, &udp4_p2p);
+ if (ret)
+ goto out;
+ ret = __hellcreek_fdb_add(hellcreek, &udp6_p2p);
+ if (ret)
+ goto out;
+ ret = __hellcreek_fdb_add(hellcreek, &stp);
out:
mutex_unlock(&hellcreek->reg_lock);
@@ -1384,14 +1458,19 @@ static void hellcreek_teardown(struct dsa_switch *ds)
dsa_devlink_resources_unregister(ds);
}
-static void hellcreek_phylink_validate(struct dsa_switch *ds, int port,
- unsigned long *supported,
- struct phylink_link_state *state)
+static void hellcreek_phylink_get_caps(struct dsa_switch *ds, int port,
+ struct phylink_config *config)
{
- __ETHTOOL_DECLARE_LINK_MODE_MASK(mask) = { 0, };
struct hellcreek *hellcreek = ds->priv;
- dev_dbg(hellcreek->dev, "Phylink validate for port %d\n", port);
+ __set_bit(PHY_INTERFACE_MODE_MII, config->supported_interfaces);
+ __set_bit(PHY_INTERFACE_MODE_RGMII, config->supported_interfaces);
+
+ /* Include GMII - the hardware does not support this interface
+ * mode, but it's the default interface mode for phylib, so we
+ * need it for compatibility with existing DT.
+ */
+ __set_bit(PHY_INTERFACE_MODE_GMII, config->supported_interfaces);
/* The MAC settings are a hardware configuration option and cannot be
* changed at run time or by strapping. Therefore the attached PHYs
@@ -1399,12 +1478,9 @@ static void hellcreek_phylink_validate(struct dsa_switch *ds, int port,
* by the hardware.
*/
if (hellcreek->pdata->is_100_mbits)
- phylink_set(mask, 100baseT_Full);
+ config->mac_capabilities = MAC_100FD;
else
- phylink_set(mask, 1000baseT_Full);
-
- linkmode_and(supported, supported, mask);
- linkmode_and(state->advertising, state->advertising, mask);
+ config->mac_capabilities = MAC_1000FD;
}
static int
@@ -1755,7 +1831,7 @@ static const struct dsa_switch_ops hellcreek_ds_ops = {
.get_strings = hellcreek_get_strings,
.get_tag_protocol = hellcreek_get_tag_protocol,
.get_ts_info = hellcreek_get_ts_info,
- .phylink_validate = hellcreek_phylink_validate,
+ .phylink_get_caps = hellcreek_phylink_get_caps,
.port_bridge_flags = hellcreek_bridge_flags,
.port_bridge_join = hellcreek_port_bridge_join,
.port_bridge_leave = hellcreek_port_bridge_leave,
diff --git a/drivers/net/dsa/hirschmann/hellcreek_hwtstamp.c b/drivers/net/dsa/hirschmann/hellcreek_hwtstamp.c
index 40b41c7..b3bc948d 100644
--- a/drivers/net/dsa/hirschmann/hellcreek_hwtstamp.c
+++ b/drivers/net/dsa/hirschmann/hellcreek_hwtstamp.c
@@ -52,10 +52,6 @@ static int hellcreek_set_hwtstamp_config(struct hellcreek *hellcreek, int port,
*/
clear_bit_unlock(HELLCREEK_HWTSTAMP_ENABLED, &ps->state);
- /* Reserved for future extensions */
- if (config->flags)
- return -EINVAL;
-
switch (config->tx_type) {
case HWTSTAMP_TX_ON:
tx_tstamp_enable = true;
diff --git a/drivers/net/dsa/lan9303-core.c b/drivers/net/dsa/lan9303-core.c
index 89f9202..d55784d 100644
--- a/drivers/net/dsa/lan9303-core.c
+++ b/drivers/net/dsa/lan9303-core.c
@@ -1103,12 +1103,13 @@ static void lan9303_port_disable(struct dsa_switch *ds, int port)
}
static int lan9303_port_bridge_join(struct dsa_switch *ds, int port,
- struct net_device *br)
+ struct dsa_bridge bridge,
+ bool *tx_fwd_offload)
{
struct lan9303 *chip = ds->priv;
dev_dbg(chip->dev, "%s(port %d)\n", __func__, port);
- if (dsa_to_port(ds, 1)->bridge_dev == dsa_to_port(ds, 2)->bridge_dev) {
+ if (dsa_port_bridge_same(dsa_to_port(ds, 1), dsa_to_port(ds, 2))) {
lan9303_bridge_ports(chip);
chip->is_bridged = true; /* unleash stp_state_set() */
}
@@ -1117,7 +1118,7 @@ static int lan9303_port_bridge_join(struct dsa_switch *ds, int port,
}
static void lan9303_port_bridge_leave(struct dsa_switch *ds, int port,
- struct net_device *br)
+ struct dsa_bridge bridge)
{
struct lan9303 *chip = ds->priv;
diff --git a/drivers/net/dsa/lantiq_gswip.c b/drivers/net/dsa/lantiq_gswip.c
index 7056d98..46ed953 100644
--- a/drivers/net/dsa/lantiq_gswip.c
+++ b/drivers/net/dsa/lantiq_gswip.c
@@ -759,7 +759,7 @@ static int gswip_port_vlan_filtering(struct dsa_switch *ds, int port,
bool vlan_filtering,
struct netlink_ext_ack *extack)
{
- struct net_device *bridge = dsa_to_port(ds, port)->bridge_dev;
+ struct net_device *bridge = dsa_port_bridge_dev_get(dsa_to_port(ds, port));
struct gswip_priv *priv = ds->priv;
/* Do not allow changing the VLAN filtering options while in bridge */
@@ -1146,16 +1146,18 @@ static int gswip_vlan_remove(struct gswip_priv *priv,
}
static int gswip_port_bridge_join(struct dsa_switch *ds, int port,
- struct net_device *bridge)
+ struct dsa_bridge bridge,
+ bool *tx_fwd_offload)
{
+ struct net_device *br = bridge.dev;
struct gswip_priv *priv = ds->priv;
int err;
/* When the bridge uses VLAN filtering we have to configure VLAN
* specific bridges. No bridge is configured here.
*/
- if (!br_vlan_enabled(bridge)) {
- err = gswip_vlan_add_unaware(priv, bridge, port);
+ if (!br_vlan_enabled(br)) {
+ err = gswip_vlan_add_unaware(priv, br, port);
if (err)
return err;
priv->port_vlan_filter &= ~BIT(port);
@@ -1166,8 +1168,9 @@ static int gswip_port_bridge_join(struct dsa_switch *ds, int port,
}
static void gswip_port_bridge_leave(struct dsa_switch *ds, int port,
- struct net_device *bridge)
+ struct dsa_bridge bridge)
{
+ struct net_device *br = bridge.dev;
struct gswip_priv *priv = ds->priv;
gswip_add_single_port_br(priv, port, true);
@@ -1175,16 +1178,16 @@ static void gswip_port_bridge_leave(struct dsa_switch *ds, int port,
/* When the bridge uses VLAN filtering we have to configure VLAN
* specific bridges. No bridge is configured here.
*/
- if (!br_vlan_enabled(bridge))
- gswip_vlan_remove(priv, bridge, port, 0, true, false);
+ if (!br_vlan_enabled(br))
+ gswip_vlan_remove(priv, br, port, 0, true, false);
}
static int gswip_port_vlan_prepare(struct dsa_switch *ds, int port,
const struct switchdev_obj_port_vlan *vlan,
struct netlink_ext_ack *extack)
{
+ struct net_device *bridge = dsa_port_bridge_dev_get(dsa_to_port(ds, port));
struct gswip_priv *priv = ds->priv;
- struct net_device *bridge = dsa_to_port(ds, port)->bridge_dev;
unsigned int max_ports = priv->hw_info->max_ports;
int pos = max_ports;
int i, idx = -1;
@@ -1229,8 +1232,8 @@ static int gswip_port_vlan_add(struct dsa_switch *ds, int port,
const struct switchdev_obj_port_vlan *vlan,
struct netlink_ext_ack *extack)
{
+ struct net_device *bridge = dsa_port_bridge_dev_get(dsa_to_port(ds, port));
struct gswip_priv *priv = ds->priv;
- struct net_device *bridge = dsa_to_port(ds, port)->bridge_dev;
bool untagged = vlan->flags & BRIDGE_VLAN_INFO_UNTAGGED;
bool pvid = vlan->flags & BRIDGE_VLAN_INFO_PVID;
int err;
@@ -1254,8 +1257,8 @@ static int gswip_port_vlan_add(struct dsa_switch *ds, int port,
static int gswip_port_vlan_del(struct dsa_switch *ds, int port,
const struct switchdev_obj_port_vlan *vlan)
{
+ struct net_device *bridge = dsa_port_bridge_dev_get(dsa_to_port(ds, port));
struct gswip_priv *priv = ds->priv;
- struct net_device *bridge = dsa_to_port(ds, port)->bridge_dev;
bool pvid = vlan->flags & BRIDGE_VLAN_INFO_PVID;
/* We have to receive all packets on the CPU port and should not
@@ -1340,8 +1343,8 @@ static void gswip_port_stp_state_set(struct dsa_switch *ds, int port, u8 state)
static int gswip_port_fdb(struct dsa_switch *ds, int port,
const unsigned char *addr, u16 vid, bool add)
{
+ struct net_device *bridge = dsa_port_bridge_dev_get(dsa_to_port(ds, port));
struct gswip_priv *priv = ds->priv;
- struct net_device *bridge = dsa_to_port(ds, port)->bridge_dev;
struct gswip_pce_table_entry mac_bridge = {0,};
unsigned int cpu_port = priv->hw_info->cpu_port;
int fid = -1;
@@ -1438,114 +1441,70 @@ static int gswip_port_fdb_dump(struct dsa_switch *ds, int port,
return 0;
}
-static void gswip_phylink_set_capab(unsigned long *supported,
- struct phylink_link_state *state)
-{
- __ETHTOOL_DECLARE_LINK_MODE_MASK(mask) = { 0, };
-
- /* Allow all the expected bits */
- phylink_set(mask, Autoneg);
- phylink_set_port_modes(mask);
- phylink_set(mask, Pause);
- phylink_set(mask, Asym_Pause);
-
- /* With the exclusion of MII, Reverse MII and Reduced MII, we
- * support Gigabit, including Half duplex
- */
- if (state->interface != PHY_INTERFACE_MODE_MII &&
- state->interface != PHY_INTERFACE_MODE_REVMII &&
- state->interface != PHY_INTERFACE_MODE_RMII) {
- phylink_set(mask, 1000baseT_Full);
- phylink_set(mask, 1000baseT_Half);
- }
-
- phylink_set(mask, 10baseT_Half);
- phylink_set(mask, 10baseT_Full);
- phylink_set(mask, 100baseT_Half);
- phylink_set(mask, 100baseT_Full);
-
- linkmode_and(supported, supported, mask);
- linkmode_and(state->advertising, state->advertising, mask);
-}
-
-static void gswip_xrx200_phylink_validate(struct dsa_switch *ds, int port,
- unsigned long *supported,
- struct phylink_link_state *state)
+static void gswip_xrx200_phylink_get_caps(struct dsa_switch *ds, int port,
+ struct phylink_config *config)
{
switch (port) {
case 0:
case 1:
- if (!phy_interface_mode_is_rgmii(state->interface) &&
- state->interface != PHY_INTERFACE_MODE_MII &&
- state->interface != PHY_INTERFACE_MODE_REVMII &&
- state->interface != PHY_INTERFACE_MODE_RMII)
- goto unsupported;
+ phy_interface_set_rgmii(config->supported_interfaces);
+ __set_bit(PHY_INTERFACE_MODE_MII,
+ config->supported_interfaces);
+ __set_bit(PHY_INTERFACE_MODE_REVMII,
+ config->supported_interfaces);
+ __set_bit(PHY_INTERFACE_MODE_RMII,
+ config->supported_interfaces);
break;
+
case 2:
case 3:
case 4:
- if (state->interface != PHY_INTERFACE_MODE_INTERNAL)
- goto unsupported;
+ __set_bit(PHY_INTERFACE_MODE_INTERNAL,
+ config->supported_interfaces);
break;
+
case 5:
- if (!phy_interface_mode_is_rgmii(state->interface) &&
- state->interface != PHY_INTERFACE_MODE_INTERNAL)
- goto unsupported;
+ phy_interface_set_rgmii(config->supported_interfaces);
+ __set_bit(PHY_INTERFACE_MODE_INTERNAL,
+ config->supported_interfaces);
break;
- default:
- linkmode_zero(supported);
- dev_err(ds->dev, "Unsupported port: %i\n", port);
- return;
}
- gswip_phylink_set_capab(supported, state);
-
- return;
-
-unsupported:
- linkmode_zero(supported);
- dev_err(ds->dev, "Unsupported interface '%s' for port %d\n",
- phy_modes(state->interface), port);
+ config->mac_capabilities = MAC_ASYM_PAUSE | MAC_SYM_PAUSE |
+ MAC_10 | MAC_100 | MAC_1000;
}
-static void gswip_xrx300_phylink_validate(struct dsa_switch *ds, int port,
- unsigned long *supported,
- struct phylink_link_state *state)
+static void gswip_xrx300_phylink_get_caps(struct dsa_switch *ds, int port,
+ struct phylink_config *config)
{
switch (port) {
case 0:
- if (!phy_interface_mode_is_rgmii(state->interface) &&
- state->interface != PHY_INTERFACE_MODE_GMII &&
- state->interface != PHY_INTERFACE_MODE_RMII)
- goto unsupported;
+ phy_interface_set_rgmii(config->supported_interfaces);
+ __set_bit(PHY_INTERFACE_MODE_GMII,
+ config->supported_interfaces);
+ __set_bit(PHY_INTERFACE_MODE_RMII,
+ config->supported_interfaces);
break;
+
case 1:
case 2:
case 3:
case 4:
- if (state->interface != PHY_INTERFACE_MODE_INTERNAL)
- goto unsupported;
+ __set_bit(PHY_INTERFACE_MODE_INTERNAL,
+ config->supported_interfaces);
break;
+
case 5:
- if (!phy_interface_mode_is_rgmii(state->interface) &&
- state->interface != PHY_INTERFACE_MODE_INTERNAL &&
- state->interface != PHY_INTERFACE_MODE_RMII)
- goto unsupported;
+ phy_interface_set_rgmii(config->supported_interfaces);
+ __set_bit(PHY_INTERFACE_MODE_INTERNAL,
+ config->supported_interfaces);
+ __set_bit(PHY_INTERFACE_MODE_RMII,
+ config->supported_interfaces);
break;
- default:
- linkmode_zero(supported);
- dev_err(ds->dev, "Unsupported port: %i\n", port);
- return;
}
- gswip_phylink_set_capab(supported, state);
-
- return;
-
-unsupported:
- linkmode_zero(supported);
- dev_err(ds->dev, "Unsupported interface '%s' for port %d\n",
- phy_modes(state->interface), port);
+ config->mac_capabilities = MAC_ASYM_PAUSE | MAC_SYM_PAUSE |
+ MAC_10 | MAC_100 | MAC_1000;
}
static void gswip_port_set_link(struct gswip_priv *priv, int port, bool link)
@@ -1827,7 +1786,7 @@ static const struct dsa_switch_ops gswip_xrx200_switch_ops = {
.port_fdb_add = gswip_port_fdb_add,
.port_fdb_del = gswip_port_fdb_del,
.port_fdb_dump = gswip_port_fdb_dump,
- .phylink_validate = gswip_xrx200_phylink_validate,
+ .phylink_get_caps = gswip_xrx200_phylink_get_caps,
.phylink_mac_config = gswip_phylink_mac_config,
.phylink_mac_link_down = gswip_phylink_mac_link_down,
.phylink_mac_link_up = gswip_phylink_mac_link_up,
@@ -1851,7 +1810,7 @@ static const struct dsa_switch_ops gswip_xrx300_switch_ops = {
.port_fdb_add = gswip_port_fdb_add,
.port_fdb_del = gswip_port_fdb_del,
.port_fdb_dump = gswip_port_fdb_dump,
- .phylink_validate = gswip_xrx300_phylink_validate,
+ .phylink_get_caps = gswip_xrx300_phylink_get_caps,
.phylink_mac_config = gswip_phylink_mac_config,
.phylink_mac_link_down = gswip_phylink_mac_link_down,
.phylink_mac_link_up = gswip_phylink_mac_link_up,
diff --git a/drivers/net/dsa/microchip/ksz8795.c b/drivers/net/dsa/microchip/ksz8795.c
index 013e9c0..991b9c6 100644
--- a/drivers/net/dsa/microchip/ksz8795.c
+++ b/drivers/net/dsa/microchip/ksz8795.c
@@ -10,6 +10,7 @@
#include <linux/delay.h>
#include <linux/export.h>
#include <linux/gpio.h>
+#include <linux/if_vlan.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/platform_data/microchip-ksz.h>
diff --git a/drivers/net/dsa/microchip/ksz_common.c b/drivers/net/dsa/microchip/ksz_common.c
index 8a04302..55dbda0 100644
--- a/drivers/net/dsa/microchip/ksz_common.c
+++ b/drivers/net/dsa/microchip/ksz_common.c
@@ -43,7 +43,7 @@ void ksz_update_port_member(struct ksz_device *dev, int port)
continue;
if (port == i)
continue;
- if (!dp->bridge_dev || dp->bridge_dev != other_dp->bridge_dev)
+ if (!dsa_port_bridge_same(dp, other_dp))
continue;
if (other_p->stp_state == BR_STATE_FORWARDING &&
@@ -192,7 +192,8 @@ void ksz_get_ethtool_stats(struct dsa_switch *ds, int port, uint64_t *buf)
EXPORT_SYMBOL_GPL(ksz_get_ethtool_stats);
int ksz_port_bridge_join(struct dsa_switch *ds, int port,
- struct net_device *br)
+ struct dsa_bridge bridge,
+ bool *tx_fwd_offload)
{
/* port_stp_state_set() will be called after to put the port in
* appropriate state so there is no need to do anything.
@@ -203,7 +204,7 @@ int ksz_port_bridge_join(struct dsa_switch *ds, int port,
EXPORT_SYMBOL_GPL(ksz_port_bridge_join);
void ksz_port_bridge_leave(struct dsa_switch *ds, int port,
- struct net_device *br)
+ struct dsa_bridge bridge)
{
/* port_stp_state_set() will be called after to put the port in
* forwarding state so there is no need to do anything.
@@ -301,7 +302,6 @@ int ksz_port_mdb_del(struct dsa_switch *ds, int port,
struct ksz_device *dev = ds->priv;
struct alu_struct alu;
int index;
- int ret = 0;
for (index = 0; index < dev->num_statics; index++) {
if (!dev->dev_ops->r_sta_mac_table(dev, index, &alu)) {
@@ -323,7 +323,7 @@ int ksz_port_mdb_del(struct dsa_switch *ds, int port,
dev->dev_ops->w_sta_mac_table(dev, index, &alu);
exit:
- return ret;
+ return 0;
}
EXPORT_SYMBOL_GPL(ksz_port_mdb_del);
diff --git a/drivers/net/dsa/microchip/ksz_common.h b/drivers/net/dsa/microchip/ksz_common.h
index 54b456b..df8ae59 100644
--- a/drivers/net/dsa/microchip/ksz_common.h
+++ b/drivers/net/dsa/microchip/ksz_common.h
@@ -155,9 +155,9 @@ void ksz_mac_link_down(struct dsa_switch *ds, int port, unsigned int mode,
int ksz_sset_count(struct dsa_switch *ds, int port, int sset);
void ksz_get_ethtool_stats(struct dsa_switch *ds, int port, uint64_t *buf);
int ksz_port_bridge_join(struct dsa_switch *ds, int port,
- struct net_device *br);
+ struct dsa_bridge bridge, bool *tx_fwd_offload);
void ksz_port_bridge_leave(struct dsa_switch *ds, int port,
- struct net_device *br);
+ struct dsa_bridge bridge);
void ksz_port_fast_age(struct dsa_switch *ds, int port);
int ksz_port_fdb_dump(struct dsa_switch *ds, int port, dsa_fdb_dump_cb_t *cb,
void *data);
diff --git a/drivers/net/dsa/mt7530.c b/drivers/net/dsa/mt7530.c
index 9890672..b82512e 100644
--- a/drivers/net/dsa/mt7530.c
+++ b/drivers/net/dsa/mt7530.c
@@ -1186,29 +1186,33 @@ mt7530_port_bridge_flags(struct dsa_switch *ds, int port,
static int
mt7530_port_bridge_join(struct dsa_switch *ds, int port,
- struct net_device *bridge)
+ struct dsa_bridge bridge, bool *tx_fwd_offload)
{
- struct mt7530_priv *priv = ds->priv;
+ struct dsa_port *dp = dsa_to_port(ds, port), *other_dp;
u32 port_bitmap = BIT(MT7530_CPU_PORT);
- int i;
+ struct mt7530_priv *priv = ds->priv;
mutex_lock(&priv->reg_mutex);
- for (i = 0; i < MT7530_NUM_PORTS; i++) {
+ dsa_switch_for_each_user_port(other_dp, ds) {
+ int other_port = other_dp->index;
+
+ if (dp == other_dp)
+ continue;
+
/* Add this port to the port matrix of the other ports in the
* same bridge. If the port is disabled, port matrix is kept
* and not being setup until the port becomes enabled.
*/
- if (dsa_is_user_port(ds, i) && i != port) {
- if (dsa_to_port(ds, i)->bridge_dev != bridge)
- continue;
- if (priv->ports[i].enable)
- mt7530_set(priv, MT7530_PCR_P(i),
- PCR_MATRIX(BIT(port)));
- priv->ports[i].pm |= PCR_MATRIX(BIT(port));
+ if (!dsa_port_offloads_bridge(other_dp, &bridge))
+ continue;
- port_bitmap |= BIT(i);
- }
+ if (priv->ports[other_port].enable)
+ mt7530_set(priv, MT7530_PCR_P(other_port),
+ PCR_MATRIX(BIT(port)));
+ priv->ports[other_port].pm |= PCR_MATRIX(BIT(port));
+
+ port_bitmap |= BIT(other_port);
}
/* Add the all other ports to this port matrix. */
@@ -1236,7 +1240,7 @@ mt7530_port_set_vlan_unaware(struct dsa_switch *ds, int port)
/* This is called after .port_bridge_leave when leaving a VLAN-aware
* bridge. Don't set standalone ports to fallback mode.
*/
- if (dsa_to_port(ds, port)->bridge_dev)
+ if (dsa_port_bridge_dev_get(dsa_to_port(ds, port)))
mt7530_rmw(priv, MT7530_PCR_P(port), PCR_PORT_VLAN_MASK,
MT7530_PORT_FALLBACK_MODE);
@@ -1299,26 +1303,30 @@ mt7530_port_set_vlan_aware(struct dsa_switch *ds, int port)
static void
mt7530_port_bridge_leave(struct dsa_switch *ds, int port,
- struct net_device *bridge)
+ struct dsa_bridge bridge)
{
+ struct dsa_port *dp = dsa_to_port(ds, port), *other_dp;
struct mt7530_priv *priv = ds->priv;
- int i;
mutex_lock(&priv->reg_mutex);
- for (i = 0; i < MT7530_NUM_PORTS; i++) {
+ dsa_switch_for_each_user_port(other_dp, ds) {
+ int other_port = other_dp->index;
+
+ if (dp == other_dp)
+ continue;
+
/* Remove this port from the port matrix of the other ports
* in the same bridge. If the port is disabled, port matrix
* is kept and not being setup until the port becomes enabled.
*/
- if (dsa_is_user_port(ds, i) && i != port) {
- if (dsa_to_port(ds, i)->bridge_dev != bridge)
- continue;
- if (priv->ports[i].enable)
- mt7530_clear(priv, MT7530_PCR_P(i),
- PCR_MATRIX(BIT(port)));
- priv->ports[i].pm &= ~PCR_MATRIX(BIT(port));
- }
+ if (!dsa_port_offloads_bridge(other_dp, &bridge))
+ continue;
+
+ if (priv->ports[other_port].enable)
+ mt7530_clear(priv, MT7530_PCR_P(other_port),
+ PCR_MATRIX(BIT(port)));
+ priv->ports[other_port].pm &= ~PCR_MATRIX(BIT(port));
}
/* Set the cpu port to be the only one in the port matrix of
diff --git a/drivers/net/dsa/mv88e6xxx/chip.c b/drivers/net/dsa/mv88e6xxx/chip.c
index cd8462d..58ca684 100644
--- a/drivers/net/dsa/mv88e6xxx/chip.c
+++ b/drivers/net/dsa/mv88e6xxx/chip.c
@@ -1241,8 +1241,7 @@ static u16 mv88e6xxx_port_vlan(struct mv88e6xxx_chip *chip, int dev, int port)
{
struct dsa_switch *ds = chip->ds;
struct dsa_switch_tree *dst = ds->dst;
- struct net_device *br;
- struct dsa_port *dp;
+ struct dsa_port *dp, *other_dp;
bool found = false;
u16 pvlan;
@@ -1251,11 +1250,9 @@ static u16 mv88e6xxx_port_vlan(struct mv88e6xxx_chip *chip, int dev, int port)
list_for_each_entry(dp, &dst->ports, list) {
if (dp->ds->index == dev && dp->index == port) {
/* dp might be a DSA link or a user port, so it
- * might or might not have a bridge_dev
- * pointer. Use the "found" variable for both
- * cases.
+ * might or might not have a bridge.
+ * Use the "found" variable for both cases.
*/
- br = dp->bridge_dev;
found = true;
break;
}
@@ -1263,13 +1260,14 @@ static u16 mv88e6xxx_port_vlan(struct mv88e6xxx_chip *chip, int dev, int port)
/* dev is a virtual bridge */
} else {
list_for_each_entry(dp, &dst->ports, list) {
- if (dp->bridge_num < 0)
+ unsigned int bridge_num = dsa_port_bridge_num_get(dp);
+
+ if (!bridge_num)
continue;
- if (dp->bridge_num + 1 + dst->last_switch != dev)
+ if (bridge_num + dst->last_switch != dev)
continue;
- br = dp->bridge_dev;
found = true;
break;
}
@@ -1288,12 +1286,11 @@ static u16 mv88e6xxx_port_vlan(struct mv88e6xxx_chip *chip, int dev, int port)
/* Frames from user ports can egress any local DSA links and CPU ports,
* as well as any local member of their bridge group.
*/
- list_for_each_entry(dp, &dst->ports, list)
- if (dp->ds == ds &&
- (dp->type == DSA_PORT_TYPE_CPU ||
- dp->type == DSA_PORT_TYPE_DSA ||
- (br && dp->bridge_dev == br)))
- pvlan |= BIT(dp->index);
+ dsa_switch_for_each_port(other_dp, ds)
+ if (other_dp->type == DSA_PORT_TYPE_CPU ||
+ other_dp->type == DSA_PORT_TYPE_DSA ||
+ dsa_port_bridge_same(dp, other_dp))
+ pvlan |= BIT(other_dp->index);
return pvlan;
}
@@ -1660,12 +1657,13 @@ static int mv88e6xxx_atu_new(struct mv88e6xxx_chip *chip, u16 *fid)
static int mv88e6xxx_port_check_hw_vlan(struct dsa_switch *ds, int port,
u16 vid)
{
+ struct dsa_port *dp = dsa_to_port(ds, port), *other_dp;
struct mv88e6xxx_chip *chip = ds->priv;
struct mv88e6xxx_vtu_entry vlan;
- int i, err;
+ int err;
/* DSA and CPU ports have to be members of multiple vlans */
- if (dsa_is_dsa_port(ds, port) || dsa_is_cpu_port(ds, port))
+ if (dsa_port_is_dsa(dp) || dsa_port_is_cpu(dp))
return 0;
err = mv88e6xxx_vtu_get(chip, vid, &vlan);
@@ -1675,27 +1673,22 @@ static int mv88e6xxx_port_check_hw_vlan(struct dsa_switch *ds, int port,
if (!vlan.valid)
return 0;
- for (i = 0; i < mv88e6xxx_num_ports(chip); ++i) {
- if (dsa_is_dsa_port(ds, i) || dsa_is_cpu_port(ds, i))
- continue;
+ dsa_switch_for_each_user_port(other_dp, ds) {
+ struct net_device *other_br;
- if (!dsa_to_port(ds, i)->slave)
- continue;
-
- if (vlan.member[i] ==
+ if (vlan.member[other_dp->index] ==
MV88E6XXX_G1_VTU_DATA_MEMBER_TAG_NON_MEMBER)
continue;
- if (dsa_to_port(ds, i)->bridge_dev ==
- dsa_to_port(ds, port)->bridge_dev)
+ if (dsa_port_bridge_same(dp, other_dp))
break; /* same bridge, check next VLAN */
- if (!dsa_to_port(ds, i)->bridge_dev)
+ other_br = dsa_port_bridge_dev_get(other_dp);
+ if (!other_br)
continue;
dev_err(ds->dev, "p%d: hw VLAN %d already used by port %d in %s\n",
- port, vlan.vid, i,
- netdev_name(dsa_to_port(ds, i)->bridge_dev));
+ port, vlan.vid, other_dp->index, netdev_name(other_br));
return -EOPNOTSUPP;
}
@@ -1705,13 +1698,14 @@ static int mv88e6xxx_port_check_hw_vlan(struct dsa_switch *ds, int port,
static int mv88e6xxx_port_commit_pvid(struct mv88e6xxx_chip *chip, int port)
{
struct dsa_port *dp = dsa_to_port(chip->ds, port);
+ struct net_device *br = dsa_port_bridge_dev_get(dp);
struct mv88e6xxx_port *p = &chip->ports[port];
u16 pvid = MV88E6XXX_VID_STANDALONE;
bool drop_untagged = false;
int err;
- if (dp->bridge_dev) {
- if (br_vlan_enabled(dp->bridge_dev)) {
+ if (br) {
+ if (br_vlan_enabled(br)) {
pvid = p->bridge_pvid.vid;
drop_untagged = !p->bridge_pvid.valid;
} else {
@@ -2429,7 +2423,7 @@ static int mv88e6xxx_port_fdb_dump(struct dsa_switch *ds, int port,
}
static int mv88e6xxx_bridge_map(struct mv88e6xxx_chip *chip,
- struct net_device *br)
+ struct dsa_bridge bridge)
{
struct dsa_switch *ds = chip->ds;
struct dsa_switch_tree *dst = ds->dst;
@@ -2437,7 +2431,7 @@ static int mv88e6xxx_bridge_map(struct mv88e6xxx_chip *chip,
int err;
list_for_each_entry(dp, &dst->ports, list) {
- if (dp->bridge_dev == br) {
+ if (dsa_port_offloads_bridge(dp, &bridge)) {
if (dp->ds == ds) {
/* This is a local bridge group member,
* remap its Port VLAN Map.
@@ -2460,15 +2454,29 @@ static int mv88e6xxx_bridge_map(struct mv88e6xxx_chip *chip,
return 0;
}
+/* Treat the software bridge as a virtual single-port switch behind the
+ * CPU and map in the PVT. First dst->last_switch elements are taken by
+ * physical switches, so start from beyond that range.
+ */
+static int mv88e6xxx_map_virtual_bridge_to_pvt(struct dsa_switch *ds,
+ unsigned int bridge_num)
+{
+ u8 dev = bridge_num + ds->dst->last_switch;
+ struct mv88e6xxx_chip *chip = ds->priv;
+
+ return mv88e6xxx_pvt_map(chip, dev, 0);
+}
+
static int mv88e6xxx_port_bridge_join(struct dsa_switch *ds, int port,
- struct net_device *br)
+ struct dsa_bridge bridge,
+ bool *tx_fwd_offload)
{
struct mv88e6xxx_chip *chip = ds->priv;
int err;
mv88e6xxx_reg_lock(chip);
- err = mv88e6xxx_bridge_map(chip, br);
+ err = mv88e6xxx_bridge_map(chip, bridge);
if (err)
goto unlock;
@@ -2476,6 +2484,14 @@ static int mv88e6xxx_port_bridge_join(struct dsa_switch *ds, int port,
if (err)
goto unlock;
+ if (mv88e6xxx_has_pvt(chip)) {
+ err = mv88e6xxx_map_virtual_bridge_to_pvt(ds, bridge.num);
+ if (err)
+ goto unlock;
+
+ *tx_fwd_offload = true;
+ }
+
unlock:
mv88e6xxx_reg_unlock(chip);
@@ -2483,14 +2499,18 @@ static int mv88e6xxx_port_bridge_join(struct dsa_switch *ds, int port,
}
static void mv88e6xxx_port_bridge_leave(struct dsa_switch *ds, int port,
- struct net_device *br)
+ struct dsa_bridge bridge)
{
struct mv88e6xxx_chip *chip = ds->priv;
int err;
mv88e6xxx_reg_lock(chip);
- if (mv88e6xxx_bridge_map(chip, br) ||
+ if (bridge.tx_fwd_offload &&
+ mv88e6xxx_map_virtual_bridge_to_pvt(ds, bridge.num))
+ dev_err(ds->dev, "failed to remap cross-chip Port VLAN\n");
+
+ if (mv88e6xxx_bridge_map(chip, bridge) ||
mv88e6xxx_port_vlan_map(chip, port))
dev_err(ds->dev, "failed to remap in-chip Port VLAN\n");
@@ -2505,7 +2525,7 @@ static void mv88e6xxx_port_bridge_leave(struct dsa_switch *ds, int port,
static int mv88e6xxx_crosschip_bridge_join(struct dsa_switch *ds,
int tree_index, int sw_index,
- int port, struct net_device *br)
+ int port, struct dsa_bridge bridge)
{
struct mv88e6xxx_chip *chip = ds->priv;
int err;
@@ -2515,6 +2535,7 @@ static int mv88e6xxx_crosschip_bridge_join(struct dsa_switch *ds,
mv88e6xxx_reg_lock(chip);
err = mv88e6xxx_pvt_map(chip, sw_index, port);
+ err = err ? : mv88e6xxx_map_virtual_bridge_to_pvt(ds, bridge.num);
mv88e6xxx_reg_unlock(chip);
return err;
@@ -2522,7 +2543,7 @@ static int mv88e6xxx_crosschip_bridge_join(struct dsa_switch *ds,
static void mv88e6xxx_crosschip_bridge_leave(struct dsa_switch *ds,
int tree_index, int sw_index,
- int port, struct net_device *br)
+ int port, struct dsa_bridge bridge)
{
struct mv88e6xxx_chip *chip = ds->priv;
@@ -2530,49 +2551,12 @@ static void mv88e6xxx_crosschip_bridge_leave(struct dsa_switch *ds,
return;
mv88e6xxx_reg_lock(chip);
- if (mv88e6xxx_pvt_map(chip, sw_index, port))
+ if (mv88e6xxx_pvt_map(chip, sw_index, port) ||
+ mv88e6xxx_map_virtual_bridge_to_pvt(ds, bridge.num))
dev_err(ds->dev, "failed to remap cross-chip Port VLAN\n");
mv88e6xxx_reg_unlock(chip);
}
-/* Treat the software bridge as a virtual single-port switch behind the
- * CPU and map in the PVT. First dst->last_switch elements are taken by
- * physical switches, so start from beyond that range.
- */
-static int mv88e6xxx_map_virtual_bridge_to_pvt(struct dsa_switch *ds,
- int bridge_num)
-{
- u8 dev = bridge_num + ds->dst->last_switch + 1;
- struct mv88e6xxx_chip *chip = ds->priv;
- int err;
-
- mv88e6xxx_reg_lock(chip);
- err = mv88e6xxx_pvt_map(chip, dev, 0);
- mv88e6xxx_reg_unlock(chip);
-
- return err;
-}
-
-static int mv88e6xxx_bridge_tx_fwd_offload(struct dsa_switch *ds, int port,
- struct net_device *br,
- int bridge_num)
-{
- return mv88e6xxx_map_virtual_bridge_to_pvt(ds, bridge_num);
-}
-
-static void mv88e6xxx_bridge_tx_fwd_unoffload(struct dsa_switch *ds, int port,
- struct net_device *br,
- int bridge_num)
-{
- int err;
-
- err = mv88e6xxx_map_virtual_bridge_to_pvt(ds, bridge_num);
- if (err) {
- dev_err(ds->dev, "failed to remap cross-chip Port VLAN: %pe\n",
- ERR_PTR(err));
- }
-}
-
static int mv88e6xxx_software_reset(struct mv88e6xxx_chip *chip)
{
if (chip->info->ops->reset)
@@ -3199,8 +3183,8 @@ static int mv88e6xxx_setup(struct dsa_switch *ds)
* time.
*/
if (mv88e6xxx_has_pvt(chip))
- ds->num_fwd_offloading_bridges = MV88E6XXX_MAX_PVT_SWITCHES -
- ds->dst->last_switch - 1;
+ ds->max_num_bridges = MV88E6XXX_MAX_PVT_SWITCHES -
+ ds->dst->last_switch - 1;
mv88e6xxx_reg_lock(chip);
@@ -6292,8 +6276,6 @@ static const struct dsa_switch_ops mv88e6xxx_switch_ops = {
.crosschip_lag_change = mv88e6xxx_crosschip_lag_change,
.crosschip_lag_join = mv88e6xxx_crosschip_lag_join,
.crosschip_lag_leave = mv88e6xxx_crosschip_lag_leave,
- .port_bridge_tx_fwd_offload = mv88e6xxx_bridge_tx_fwd_offload,
- .port_bridge_tx_fwd_unoffload = mv88e6xxx_bridge_tx_fwd_unoffload,
};
static int mv88e6xxx_register_switch(struct mv88e6xxx_chip *chip)
diff --git a/drivers/net/dsa/mv88e6xxx/hwtstamp.c b/drivers/net/dsa/mv88e6xxx/hwtstamp.c
index 8f74ffc..389f8a6 100644
--- a/drivers/net/dsa/mv88e6xxx/hwtstamp.c
+++ b/drivers/net/dsa/mv88e6xxx/hwtstamp.c
@@ -100,10 +100,6 @@ static int mv88e6xxx_set_hwtstamp_config(struct mv88e6xxx_chip *chip, int port,
*/
clear_bit_unlock(MV88E6XXX_HWTSTAMP_ENABLED, &ps->state);
- /* reserved for future extensions */
- if (config->flags)
- return -EINVAL;
-
switch (config->tx_type) {
case HWTSTAMP_TX_OFF:
tstamp_enable = false;
diff --git a/drivers/net/dsa/ocelot/Kconfig b/drivers/net/dsa/ocelot/Kconfig
index 9948544..220b0b0 100644
--- a/drivers/net/dsa/ocelot/Kconfig
+++ b/drivers/net/dsa/ocelot/Kconfig
@@ -21,6 +21,7 @@
depends on NET_VENDOR_MICROSEMI
depends on HAS_IOMEM
depends on PTP_1588_CLOCK_OPTIONAL
+ select MDIO_MSCC_MIIM
select MSCC_OCELOT_SWITCH_LIB
select NET_DSA_TAG_OCELOT_8021Q
select NET_DSA_TAG_OCELOT
diff --git a/drivers/net/dsa/ocelot/felix.c b/drivers/net/dsa/ocelot/felix.c
index f1a05e7..9957772 100644
--- a/drivers/net/dsa/ocelot/felix.c
+++ b/drivers/net/dsa/ocelot/felix.c
@@ -21,7 +21,6 @@
#include <linux/of_net.h>
#include <linux/pci.h>
#include <linux/of.h>
-#include <linux/pcs-lynx.h>
#include <net/pkt_sched.h>
#include <net/dsa.h>
#include "felix.h"
@@ -240,24 +239,32 @@ static int felix_tag_8021q_vlan_del(struct dsa_switch *ds, int port, u16 vid)
*/
static void felix_8021q_cpu_port_init(struct ocelot *ocelot, int port)
{
+ mutex_lock(&ocelot->fwd_domain_lock);
+
ocelot->ports[port]->is_dsa_8021q_cpu = true;
ocelot->npi = -1;
/* Overwrite PGID_CPU with the non-tagging port */
ocelot_write_rix(ocelot, BIT(port), ANA_PGID_PGID, PGID_CPU);
- ocelot_apply_bridge_fwd_mask(ocelot);
+ ocelot_apply_bridge_fwd_mask(ocelot, true);
+
+ mutex_unlock(&ocelot->fwd_domain_lock);
}
static void felix_8021q_cpu_port_deinit(struct ocelot *ocelot, int port)
{
+ mutex_lock(&ocelot->fwd_domain_lock);
+
ocelot->ports[port]->is_dsa_8021q_cpu = false;
/* Restore PGID_CPU */
ocelot_write_rix(ocelot, BIT(ocelot->num_phys_ports), ANA_PGID_PGID,
PGID_CPU);
- ocelot_apply_bridge_fwd_mask(ocelot);
+ ocelot_apply_bridge_fwd_mask(ocelot, true);
+
+ mutex_unlock(&ocelot->fwd_domain_lock);
}
/* Set up a VCAP IS2 rule for delivering PTP frames to the CPU port module.
@@ -632,6 +639,17 @@ static int felix_set_ageing_time(struct dsa_switch *ds,
return 0;
}
+static void felix_port_fast_age(struct dsa_switch *ds, int port)
+{
+ struct ocelot *ocelot = ds->priv;
+ int err;
+
+ err = ocelot_mact_flush(ocelot, port);
+ if (err)
+ dev_err(ds->dev, "Flushing MAC table on port %d returned %pe\n",
+ port, ERR_PTR(err));
+}
+
static int felix_fdb_dump(struct dsa_switch *ds, int port,
dsa_fdb_dump_cb_t *cb, void *data)
{
@@ -701,21 +719,21 @@ static int felix_bridge_flags(struct dsa_switch *ds, int port,
}
static int felix_bridge_join(struct dsa_switch *ds, int port,
- struct net_device *br)
+ struct dsa_bridge bridge, bool *tx_fwd_offload)
{
struct ocelot *ocelot = ds->priv;
- ocelot_port_bridge_join(ocelot, port, br);
+ ocelot_port_bridge_join(ocelot, port, bridge.dev);
return 0;
}
static void felix_bridge_leave(struct dsa_switch *ds, int port,
- struct net_device *br)
+ struct dsa_bridge bridge)
{
struct ocelot *ocelot = ds->priv;
- ocelot_port_bridge_leave(ocelot, port, br);
+ ocelot_port_bridge_leave(ocelot, port, bridge.dev);
}
static int felix_lag_join(struct dsa_switch *ds, int port,
@@ -823,8 +841,8 @@ static void felix_phylink_mac_config(struct dsa_switch *ds, int port,
struct felix *felix = ocelot_to_felix(ocelot);
struct dsa_port *dp = dsa_to_port(ds, port);
- if (felix->pcs[port])
- phylink_set_pcs(dp->pl, &felix->pcs[port]->pcs);
+ if (felix->pcs && felix->pcs[port])
+ phylink_set_pcs(dp->pl, felix->pcs[port]);
}
static void felix_phylink_mac_link_down(struct dsa_switch *ds, int port,
@@ -992,6 +1010,10 @@ static int felix_init_structs(struct felix *felix, int num_phys_ports)
ocelot->num_stats = felix->info->num_stats;
ocelot->num_mact_rows = felix->info->num_mact_rows;
ocelot->vcap = felix->info->vcap;
+ ocelot->vcap_pol.base = felix->info->vcap_pol_base;
+ ocelot->vcap_pol.max = felix->info->vcap_pol_max;
+ ocelot->vcap_pol.base2 = felix->info->vcap_pol_base2;
+ ocelot->vcap_pol.max2 = felix->info->vcap_pol_max2;
ocelot->ops = felix->info->ops;
ocelot->npi_inj_prefix = OCELOT_TAG_PREFIX_SHORT;
ocelot->npi_xtr_prefix = OCELOT_TAG_PREFIX_SHORT;
@@ -1019,7 +1041,7 @@ static int felix_init_structs(struct felix *felix, int num_phys_ports)
res.start += felix->switch_base;
res.end += felix->switch_base;
- target = ocelot_regmap_init(ocelot, &res);
+ target = felix->info->init_regmap(ocelot, &res);
if (IS_ERR(target)) {
dev_err(ocelot->dev,
"Failed to map device memory space\n");
@@ -1056,7 +1078,7 @@ static int felix_init_structs(struct felix *felix, int num_phys_ports)
res.start += felix->switch_base;
res.end += felix->switch_base;
- target = ocelot_regmap_init(ocelot, &res);
+ target = felix->info->init_regmap(ocelot, &res);
if (IS_ERR(target)) {
dev_err(ocelot->dev,
"Failed to map memory space for port %d\n",
@@ -1143,38 +1165,22 @@ static void felix_port_deferred_xmit(struct kthread_work *work)
kfree(xmit_work);
}
-static int felix_port_setup_tagger_data(struct dsa_switch *ds, int port)
+static int felix_connect_tag_protocol(struct dsa_switch *ds,
+ enum dsa_tag_protocol proto)
{
- struct dsa_port *dp = dsa_to_port(ds, port);
- struct ocelot *ocelot = ds->priv;
- struct felix *felix = ocelot_to_felix(ocelot);
- struct felix_port *felix_port;
+ struct ocelot_8021q_tagger_data *tagger_data;
- if (!dsa_port_is_user(dp))
+ switch (proto) {
+ case DSA_TAG_PROTO_OCELOT_8021Q:
+ tagger_data = ocelot_8021q_tagger_data(ds);
+ tagger_data->xmit_work_fn = felix_port_deferred_xmit;
return 0;
-
- felix_port = kzalloc(sizeof(*felix_port), GFP_KERNEL);
- if (!felix_port)
- return -ENOMEM;
-
- felix_port->xmit_worker = felix->xmit_worker;
- felix_port->xmit_work_fn = felix_port_deferred_xmit;
-
- dp->priv = felix_port;
-
- return 0;
-}
-
-static void felix_port_teardown_tagger_data(struct dsa_switch *ds, int port)
-{
- struct dsa_port *dp = dsa_to_port(ds, port);
- struct felix_port *felix_port = dp->priv;
-
- if (!felix_port)
- return;
-
- dp->priv = NULL;
- kfree(felix_port);
+ case DSA_TAG_PROTO_OCELOT:
+ case DSA_TAG_PROTO_SEVILLE:
+ return 0;
+ default:
+ return -EPROTONOSUPPORT;
+ }
}
/* Hardware initialization done here so that we can allocate structures with
@@ -1205,12 +1211,6 @@ static int felix_setup(struct dsa_switch *ds)
}
}
- felix->xmit_worker = kthread_create_worker(0, "felix_xmit");
- if (IS_ERR(felix->xmit_worker)) {
- err = PTR_ERR(felix->xmit_worker);
- goto out_deinit_timestamp;
- }
-
for (port = 0; port < ds->num_ports; port++) {
if (dsa_is_unused_port(ds, port))
continue;
@@ -1221,14 +1221,6 @@ static int felix_setup(struct dsa_switch *ds)
* bits of vlan tag.
*/
felix_port_qos_map_init(ocelot, port);
-
- err = felix_port_setup_tagger_data(ds, port);
- if (err) {
- dev_err(ds->dev,
- "port %d failed to set up tagger data: %pe\n",
- port, ERR_PTR(err));
- goto out_deinit_ports;
- }
}
err = ocelot_devlink_sb_register(ocelot);
@@ -1256,13 +1248,9 @@ static int felix_setup(struct dsa_switch *ds)
if (dsa_is_unused_port(ds, port))
continue;
- felix_port_teardown_tagger_data(ds, port);
ocelot_deinit_port(ocelot, port);
}
- kthread_destroy_worker(felix->xmit_worker);
-
-out_deinit_timestamp:
ocelot_deinit_timestamp(ocelot);
ocelot_deinit(ocelot);
@@ -1291,12 +1279,9 @@ static void felix_teardown(struct dsa_switch *ds)
if (dsa_is_unused_port(ds, port))
continue;
- felix_port_teardown_tagger_data(ds, port);
ocelot_deinit_port(ocelot, port);
}
- kthread_destroy_worker(felix->xmit_worker);
-
ocelot_devlink_sb_unregister(ocelot);
ocelot_deinit_timestamp(ocelot);
ocelot_deinit(ocelot);
@@ -1636,6 +1621,7 @@ felix_mrp_del_ring_role(struct dsa_switch *ds, int port,
const struct dsa_switch_ops felix_switch_ops = {
.get_tag_protocol = felix_get_tag_protocol,
.change_tag_protocol = felix_change_tag_protocol,
+ .connect_tag_protocol = felix_connect_tag_protocol,
.setup = felix_setup,
.teardown = felix_teardown,
.set_ageing_time = felix_set_ageing_time,
@@ -1647,6 +1633,7 @@ const struct dsa_switch_ops felix_switch_ops = {
.phylink_mac_config = felix_phylink_mac_config,
.phylink_mac_link_down = felix_phylink_mac_link_down,
.phylink_mac_link_up = felix_phylink_mac_link_up,
+ .port_fast_age = felix_port_fast_age,
.port_fdb_dump = felix_fdb_dump,
.port_fdb_add = felix_fdb_add,
.port_fdb_del = felix_fdb_del,
diff --git a/drivers/net/dsa/ocelot/felix.h b/drivers/net/dsa/ocelot/felix.h
index be3e42e..9395ac1 100644
--- a/drivers/net/dsa/ocelot/felix.h
+++ b/drivers/net/dsa/ocelot/felix.h
@@ -21,8 +21,10 @@ struct felix_info {
int num_ports;
int num_tx_queues;
struct vcap_props *vcap;
- int switch_pci_bar;
- int imdio_pci_bar;
+ u16 vcap_pol_base;
+ u16 vcap_pol_max;
+ u16 vcap_pol_base2;
+ u16 vcap_pol_max2;
const struct ptp_clock_info *ptp_caps;
/* Some Ocelot switches are integrated into the SoC without the
@@ -48,6 +50,8 @@ struct felix_info {
enum tc_setup_type type, void *type_data);
void (*port_sched_speed_set)(struct ocelot *ocelot, int port,
u32 speed);
+ struct regmap *(*init_regmap)(struct ocelot *ocelot,
+ struct resource *res);
};
extern const struct dsa_switch_ops felix_switch_ops;
@@ -58,7 +62,7 @@ struct felix {
const struct felix_info *info;
struct ocelot ocelot;
struct mii_bus *imdio;
- struct lynx_pcs **pcs;
+ struct phylink_pcs **pcs;
resource_size_t switch_base;
resource_size_t imdio_base;
enum dsa_tag_protocol tag_proto;
diff --git a/drivers/net/dsa/ocelot/felix_vsc9959.c b/drivers/net/dsa/ocelot/felix_vsc9959.c
index 45c5ec7..bf8d382 100644
--- a/drivers/net/dsa/ocelot/felix_vsc9959.c
+++ b/drivers/net/dsa/ocelot/felix_vsc9959.c
@@ -5,8 +5,10 @@
#include <linux/fsl/enetc_mdio.h>
#include <soc/mscc/ocelot_qsys.h>
#include <soc/mscc/ocelot_vcap.h>
+#include <soc/mscc/ocelot_ana.h>
#include <soc/mscc/ocelot_ptp.h>
#include <soc/mscc/ocelot_sys.h>
+#include <net/tc_act/tc_gate.h>
#include <soc/mscc/ocelot.h>
#include <linux/dsa/ocelot.h>
#include <linux/pcs-lynx.h>
@@ -17,6 +19,10 @@
#include "felix.h"
#define VSC9959_TAS_GCL_ENTRY_MAX 63
+#define VSC9959_VCAP_POLICER_BASE 63
+#define VSC9959_VCAP_POLICER_MAX 383
+#define VSC9959_SWITCH_PCI_BAR 4
+#define VSC9959_IMDIO_PCI_BAR 0
static const u32 vsc9959_ana_regmap[] = {
REG(ANA_ADVLEARN, 0x0089a0),
@@ -292,7 +298,7 @@ static const u32 vsc9959_sys_regmap[] = {
REG_RESERVED(SYS_MMGT_FAST),
REG_RESERVED(SYS_EVENTS_DIF),
REG_RESERVED(SYS_EVENTS_CORE),
- REG_RESERVED(SYS_CNT),
+ REG(SYS_CNT, 0x000000),
REG(SYS_PTP_STATUS, 0x000f14),
REG(SYS_PTP_TXSTAMP, 0x000f18),
REG(SYS_PTP_NXT, 0x000f1c),
@@ -1020,15 +1026,6 @@ static void vsc9959_wm_stat(u32 val, u32 *inuse, u32 *maxuse)
*maxuse = val & GENMASK(11, 0);
}
-static const struct ocelot_ops vsc9959_ops = {
- .reset = vsc9959_reset,
- .wm_enc = vsc9959_wm_enc,
- .wm_dec = vsc9959_wm_dec,
- .wm_stat = vsc9959_wm_stat,
- .port_to_netdev = felix_port_to_netdev,
- .netdev_to_port = felix_netdev_to_port,
-};
-
static int vsc9959_mdio_bus_alloc(struct ocelot *ocelot)
{
struct felix *felix = ocelot_to_felix(ocelot);
@@ -1042,7 +1039,7 @@ static int vsc9959_mdio_bus_alloc(struct ocelot *ocelot)
int rc;
felix->pcs = devm_kcalloc(dev, felix->info->num_ports,
- sizeof(struct lynx_pcs *),
+ sizeof(struct phylink_pcs *),
GFP_KERNEL);
if (!felix->pcs) {
dev_err(dev, "failed to allocate array for PCS PHYs\n");
@@ -1091,8 +1088,8 @@ static int vsc9959_mdio_bus_alloc(struct ocelot *ocelot)
for (port = 0; port < felix->info->num_ports; port++) {
struct ocelot_port *ocelot_port = ocelot->ports[port];
- struct mdio_device *pcs;
- struct lynx_pcs *lynx;
+ struct phylink_pcs *phylink_pcs;
+ struct mdio_device *mdio_device;
if (dsa_is_unused_port(felix->ds, port))
continue;
@@ -1100,17 +1097,17 @@ static int vsc9959_mdio_bus_alloc(struct ocelot *ocelot)
if (ocelot_port->phy_mode == PHY_INTERFACE_MODE_INTERNAL)
continue;
- pcs = mdio_device_create(felix->imdio, port);
- if (IS_ERR(pcs))
+ mdio_device = mdio_device_create(felix->imdio, port);
+ if (IS_ERR(mdio_device))
continue;
- lynx = lynx_pcs_create(pcs);
- if (!lynx) {
- mdio_device_free(pcs);
+ phylink_pcs = lynx_pcs_create(mdio_device);
+ if (!phylink_pcs) {
+ mdio_device_free(mdio_device);
continue;
}
- felix->pcs[port] = lynx;
+ felix->pcs[port] = phylink_pcs;
dev_info(dev, "Found PCS at internal MDIO address %d\n", port);
}
@@ -1124,13 +1121,15 @@ static void vsc9959_mdio_bus_free(struct ocelot *ocelot)
int port;
for (port = 0; port < ocelot->num_phys_ports; port++) {
- struct lynx_pcs *pcs = felix->pcs[port];
+ struct phylink_pcs *phylink_pcs = felix->pcs[port];
+ struct mdio_device *mdio_device;
- if (!pcs)
+ if (!phylink_pcs)
continue;
- mdio_device_free(pcs->mdio);
- lynx_pcs_destroy(pcs);
+ mdio_device = lynx_get_mdio_device(phylink_pcs);
+ mdio_device_free(mdio_device);
+ lynx_pcs_destroy(phylink_pcs);
}
mdiobus_unregister(felix->imdio);
}
@@ -1344,6 +1343,877 @@ static int vsc9959_port_setup_tc(struct dsa_switch *ds, int port,
}
}
+#define VSC9959_PSFP_SFID_MAX 175
+#define VSC9959_PSFP_GATE_ID_MAX 183
+#define VSC9959_PSFP_POLICER_BASE 63
+#define VSC9959_PSFP_POLICER_MAX 383
+#define VSC9959_PSFP_GATE_LIST_NUM 4
+#define VSC9959_PSFP_GATE_CYCLETIME_MIN 5000
+
+struct felix_stream {
+ struct list_head list;
+ unsigned long id;
+ bool dummy;
+ int ports;
+ int port;
+ u8 dmac[ETH_ALEN];
+ u16 vid;
+ s8 prio;
+ u8 sfid_valid;
+ u8 ssid_valid;
+ u32 sfid;
+ u32 ssid;
+};
+
+struct felix_stream_filter {
+ struct list_head list;
+ refcount_t refcount;
+ u32 index;
+ u8 enable;
+ int portmask;
+ u8 sg_valid;
+ u32 sgid;
+ u8 fm_valid;
+ u32 fmid;
+ u8 prio_valid;
+ u8 prio;
+ u32 maxsdu;
+};
+
+struct felix_stream_filter_counters {
+ u32 match;
+ u32 not_pass_gate;
+ u32 not_pass_sdu;
+ u32 red;
+};
+
+struct felix_stream_gate {
+ u32 index;
+ u8 enable;
+ u8 ipv_valid;
+ u8 init_ipv;
+ u64 basetime;
+ u64 cycletime;
+ u64 cycletime_ext;
+ u32 num_entries;
+ struct action_gate_entry entries[];
+};
+
+struct felix_stream_gate_entry {
+ struct list_head list;
+ refcount_t refcount;
+ u32 index;
+};
+
+static int vsc9959_stream_identify(struct flow_cls_offload *f,
+ struct felix_stream *stream)
+{
+ struct flow_rule *rule = flow_cls_offload_flow_rule(f);
+ struct flow_dissector *dissector = rule->match.dissector;
+
+ if (dissector->used_keys &
+ ~(BIT(FLOW_DISSECTOR_KEY_CONTROL) |
+ BIT(FLOW_DISSECTOR_KEY_BASIC) |
+ BIT(FLOW_DISSECTOR_KEY_VLAN) |
+ BIT(FLOW_DISSECTOR_KEY_ETH_ADDRS)))
+ return -EOPNOTSUPP;
+
+ if (flow_rule_match_key(rule, FLOW_DISSECTOR_KEY_ETH_ADDRS)) {
+ struct flow_match_eth_addrs match;
+
+ flow_rule_match_eth_addrs(rule, &match);
+ ether_addr_copy(stream->dmac, match.key->dst);
+ if (!is_zero_ether_addr(match.mask->src))
+ return -EOPNOTSUPP;
+ } else {
+ return -EOPNOTSUPP;
+ }
+
+ if (flow_rule_match_key(rule, FLOW_DISSECTOR_KEY_VLAN)) {
+ struct flow_match_vlan match;
+
+ flow_rule_match_vlan(rule, &match);
+ if (match.mask->vlan_priority)
+ stream->prio = match.key->vlan_priority;
+ else
+ stream->prio = -1;
+
+ if (!match.mask->vlan_id)
+ return -EOPNOTSUPP;
+ stream->vid = match.key->vlan_id;
+ } else {
+ return -EOPNOTSUPP;
+ }
+
+ stream->id = f->cookie;
+
+ return 0;
+}
+
+static int vsc9959_mact_stream_set(struct ocelot *ocelot,
+ struct felix_stream *stream,
+ struct netlink_ext_ack *extack)
+{
+ enum macaccess_entry_type type;
+ int ret, sfid, ssid;
+ u32 vid, dst_idx;
+ u8 mac[ETH_ALEN];
+
+ ether_addr_copy(mac, stream->dmac);
+ vid = stream->vid;
+
+ /* Stream identification desn't support to add a stream with non
+ * existent MAC (The MAC entry has not been learned in MAC table).
+ */
+ ret = ocelot_mact_lookup(ocelot, &dst_idx, mac, vid, &type);
+ if (ret) {
+ if (extack)
+ NL_SET_ERR_MSG_MOD(extack, "Stream is not learned in MAC table");
+ return -EOPNOTSUPP;
+ }
+
+ if ((stream->sfid_valid || stream->ssid_valid) &&
+ type == ENTRYTYPE_NORMAL)
+ type = ENTRYTYPE_LOCKED;
+
+ sfid = stream->sfid_valid ? stream->sfid : -1;
+ ssid = stream->ssid_valid ? stream->ssid : -1;
+
+ ret = ocelot_mact_learn_streamdata(ocelot, dst_idx, mac, vid, type,
+ sfid, ssid);
+
+ return ret;
+}
+
+static struct felix_stream *
+vsc9959_stream_table_lookup(struct list_head *stream_list,
+ struct felix_stream *stream)
+{
+ struct felix_stream *tmp;
+
+ list_for_each_entry(tmp, stream_list, list)
+ if (ether_addr_equal(tmp->dmac, stream->dmac) &&
+ tmp->vid == stream->vid)
+ return tmp;
+
+ return NULL;
+}
+
+static int vsc9959_stream_table_add(struct ocelot *ocelot,
+ struct list_head *stream_list,
+ struct felix_stream *stream,
+ struct netlink_ext_ack *extack)
+{
+ struct felix_stream *stream_entry;
+ int ret;
+
+ stream_entry = kmemdup(stream, sizeof(*stream_entry), GFP_KERNEL);
+ if (!stream_entry)
+ return -ENOMEM;
+
+ if (!stream->dummy) {
+ ret = vsc9959_mact_stream_set(ocelot, stream_entry, extack);
+ if (ret) {
+ kfree(stream_entry);
+ return ret;
+ }
+ }
+
+ list_add_tail(&stream_entry->list, stream_list);
+
+ return 0;
+}
+
+static struct felix_stream *
+vsc9959_stream_table_get(struct list_head *stream_list, unsigned long id)
+{
+ struct felix_stream *tmp;
+
+ list_for_each_entry(tmp, stream_list, list)
+ if (tmp->id == id)
+ return tmp;
+
+ return NULL;
+}
+
+static void vsc9959_stream_table_del(struct ocelot *ocelot,
+ struct felix_stream *stream)
+{
+ if (!stream->dummy)
+ vsc9959_mact_stream_set(ocelot, stream, NULL);
+
+ list_del(&stream->list);
+ kfree(stream);
+}
+
+static u32 vsc9959_sfi_access_status(struct ocelot *ocelot)
+{
+ return ocelot_read(ocelot, ANA_TABLES_SFIDACCESS);
+}
+
+static int vsc9959_psfp_sfi_set(struct ocelot *ocelot,
+ struct felix_stream_filter *sfi)
+{
+ u32 val;
+
+ if (sfi->index > VSC9959_PSFP_SFID_MAX)
+ return -EINVAL;
+
+ if (!sfi->enable) {
+ ocelot_write(ocelot, ANA_TABLES_SFIDTIDX_SFID_INDEX(sfi->index),
+ ANA_TABLES_SFIDTIDX);
+
+ val = ANA_TABLES_SFIDACCESS_SFID_TBL_CMD(SFIDACCESS_CMD_WRITE);
+ ocelot_write(ocelot, val, ANA_TABLES_SFIDACCESS);
+
+ return readx_poll_timeout(vsc9959_sfi_access_status, ocelot, val,
+ (!ANA_TABLES_SFIDACCESS_SFID_TBL_CMD(val)),
+ 10, 100000);
+ }
+
+ if (sfi->sgid > VSC9959_PSFP_GATE_ID_MAX ||
+ sfi->fmid > VSC9959_PSFP_POLICER_MAX)
+ return -EINVAL;
+
+ ocelot_write(ocelot,
+ (sfi->sg_valid ? ANA_TABLES_SFIDTIDX_SGID_VALID : 0) |
+ ANA_TABLES_SFIDTIDX_SGID(sfi->sgid) |
+ (sfi->fm_valid ? ANA_TABLES_SFIDTIDX_POL_ENA : 0) |
+ ANA_TABLES_SFIDTIDX_POL_IDX(sfi->fmid) |
+ ANA_TABLES_SFIDTIDX_SFID_INDEX(sfi->index),
+ ANA_TABLES_SFIDTIDX);
+
+ ocelot_write(ocelot,
+ (sfi->prio_valid ? ANA_TABLES_SFIDACCESS_IGR_PRIO_MATCH_ENA : 0) |
+ ANA_TABLES_SFIDACCESS_IGR_PRIO(sfi->prio) |
+ ANA_TABLES_SFIDACCESS_MAX_SDU_LEN(sfi->maxsdu) |
+ ANA_TABLES_SFIDACCESS_SFID_TBL_CMD(SFIDACCESS_CMD_WRITE),
+ ANA_TABLES_SFIDACCESS);
+
+ return readx_poll_timeout(vsc9959_sfi_access_status, ocelot, val,
+ (!ANA_TABLES_SFIDACCESS_SFID_TBL_CMD(val)),
+ 10, 100000);
+}
+
+static int vsc9959_psfp_sfidmask_set(struct ocelot *ocelot, u32 sfid, int ports)
+{
+ u32 val;
+
+ ocelot_rmw(ocelot,
+ ANA_TABLES_SFIDTIDX_SFID_INDEX(sfid),
+ ANA_TABLES_SFIDTIDX_SFID_INDEX_M,
+ ANA_TABLES_SFIDTIDX);
+
+ ocelot_write(ocelot,
+ ANA_TABLES_SFID_MASK_IGR_PORT_MASK(ports) |
+ ANA_TABLES_SFID_MASK_IGR_SRCPORT_MATCH_ENA,
+ ANA_TABLES_SFID_MASK);
+
+ ocelot_rmw(ocelot,
+ ANA_TABLES_SFIDACCESS_SFID_TBL_CMD(SFIDACCESS_CMD_WRITE),
+ ANA_TABLES_SFIDACCESS_SFID_TBL_CMD_M,
+ ANA_TABLES_SFIDACCESS);
+
+ return readx_poll_timeout(vsc9959_sfi_access_status, ocelot, val,
+ (!ANA_TABLES_SFIDACCESS_SFID_TBL_CMD(val)),
+ 10, 100000);
+}
+
+static int vsc9959_psfp_sfi_list_add(struct ocelot *ocelot,
+ struct felix_stream_filter *sfi,
+ struct list_head *pos)
+{
+ struct felix_stream_filter *sfi_entry;
+ int ret;
+
+ sfi_entry = kmemdup(sfi, sizeof(*sfi_entry), GFP_KERNEL);
+ if (!sfi_entry)
+ return -ENOMEM;
+
+ refcount_set(&sfi_entry->refcount, 1);
+
+ ret = vsc9959_psfp_sfi_set(ocelot, sfi_entry);
+ if (ret) {
+ kfree(sfi_entry);
+ return ret;
+ }
+
+ vsc9959_psfp_sfidmask_set(ocelot, sfi->index, sfi->portmask);
+
+ list_add(&sfi_entry->list, pos);
+
+ return 0;
+}
+
+static int vsc9959_psfp_sfi_table_add(struct ocelot *ocelot,
+ struct felix_stream_filter *sfi)
+{
+ struct list_head *pos, *q, *last;
+ struct felix_stream_filter *tmp;
+ struct ocelot_psfp_list *psfp;
+ u32 insert = 0;
+
+ psfp = &ocelot->psfp;
+ last = &psfp->sfi_list;
+
+ list_for_each_safe(pos, q, &psfp->sfi_list) {
+ tmp = list_entry(pos, struct felix_stream_filter, list);
+ if (sfi->sg_valid == tmp->sg_valid &&
+ sfi->fm_valid == tmp->fm_valid &&
+ sfi->portmask == tmp->portmask &&
+ tmp->sgid == sfi->sgid &&
+ tmp->fmid == sfi->fmid) {
+ sfi->index = tmp->index;
+ refcount_inc(&tmp->refcount);
+ return 0;
+ }
+ /* Make sure that the index is increasing in order. */
+ if (tmp->index == insert) {
+ last = pos;
+ insert++;
+ }
+ }
+ sfi->index = insert;
+
+ return vsc9959_psfp_sfi_list_add(ocelot, sfi, last);
+}
+
+static int vsc9959_psfp_sfi_table_add2(struct ocelot *ocelot,
+ struct felix_stream_filter *sfi,
+ struct felix_stream_filter *sfi2)
+{
+ struct felix_stream_filter *tmp;
+ struct list_head *pos, *q, *last;
+ struct ocelot_psfp_list *psfp;
+ u32 insert = 0;
+ int ret;
+
+ psfp = &ocelot->psfp;
+ last = &psfp->sfi_list;
+
+ list_for_each_safe(pos, q, &psfp->sfi_list) {
+ tmp = list_entry(pos, struct felix_stream_filter, list);
+ /* Make sure that the index is increasing in order. */
+ if (tmp->index >= insert + 2)
+ break;
+
+ insert = tmp->index + 1;
+ last = pos;
+ }
+ sfi->index = insert;
+
+ ret = vsc9959_psfp_sfi_list_add(ocelot, sfi, last);
+ if (ret)
+ return ret;
+
+ sfi2->index = insert + 1;
+
+ return vsc9959_psfp_sfi_list_add(ocelot, sfi2, last->next);
+}
+
+static struct felix_stream_filter *
+vsc9959_psfp_sfi_table_get(struct list_head *sfi_list, u32 index)
+{
+ struct felix_stream_filter *tmp;
+
+ list_for_each_entry(tmp, sfi_list, list)
+ if (tmp->index == index)
+ return tmp;
+
+ return NULL;
+}
+
+static void vsc9959_psfp_sfi_table_del(struct ocelot *ocelot, u32 index)
+{
+ struct felix_stream_filter *tmp, *n;
+ struct ocelot_psfp_list *psfp;
+ u8 z;
+
+ psfp = &ocelot->psfp;
+
+ list_for_each_entry_safe(tmp, n, &psfp->sfi_list, list)
+ if (tmp->index == index) {
+ z = refcount_dec_and_test(&tmp->refcount);
+ if (z) {
+ tmp->enable = 0;
+ vsc9959_psfp_sfi_set(ocelot, tmp);
+ list_del(&tmp->list);
+ kfree(tmp);
+ }
+ break;
+ }
+}
+
+static void vsc9959_psfp_parse_gate(const struct flow_action_entry *entry,
+ struct felix_stream_gate *sgi)
+{
+ sgi->index = entry->hw_index;
+ sgi->ipv_valid = (entry->gate.prio < 0) ? 0 : 1;
+ sgi->init_ipv = (sgi->ipv_valid) ? entry->gate.prio : 0;
+ sgi->basetime = entry->gate.basetime;
+ sgi->cycletime = entry->gate.cycletime;
+ sgi->num_entries = entry->gate.num_entries;
+ sgi->enable = 1;
+
+ memcpy(sgi->entries, entry->gate.entries,
+ entry->gate.num_entries * sizeof(struct action_gate_entry));
+}
+
+static u32 vsc9959_sgi_cfg_status(struct ocelot *ocelot)
+{
+ return ocelot_read(ocelot, ANA_SG_ACCESS_CTRL);
+}
+
+static int vsc9959_psfp_sgi_set(struct ocelot *ocelot,
+ struct felix_stream_gate *sgi)
+{
+ struct action_gate_entry *e;
+ struct timespec64 base_ts;
+ u32 interval_sum = 0;
+ u32 val;
+ int i;
+
+ if (sgi->index > VSC9959_PSFP_GATE_ID_MAX)
+ return -EINVAL;
+
+ ocelot_write(ocelot, ANA_SG_ACCESS_CTRL_SGID(sgi->index),
+ ANA_SG_ACCESS_CTRL);
+
+ if (!sgi->enable) {
+ ocelot_rmw(ocelot, ANA_SG_CONFIG_REG_3_INIT_GATE_STATE,
+ ANA_SG_CONFIG_REG_3_INIT_GATE_STATE |
+ ANA_SG_CONFIG_REG_3_GATE_ENABLE,
+ ANA_SG_CONFIG_REG_3);
+
+ return 0;
+ }
+
+ if (sgi->cycletime < VSC9959_PSFP_GATE_CYCLETIME_MIN ||
+ sgi->cycletime > NSEC_PER_SEC)
+ return -EINVAL;
+
+ if (sgi->num_entries > VSC9959_PSFP_GATE_LIST_NUM)
+ return -EINVAL;
+
+ vsc9959_new_base_time(ocelot, sgi->basetime, sgi->cycletime, &base_ts);
+ ocelot_write(ocelot, base_ts.tv_nsec, ANA_SG_CONFIG_REG_1);
+ val = lower_32_bits(base_ts.tv_sec);
+ ocelot_write(ocelot, val, ANA_SG_CONFIG_REG_2);
+
+ val = upper_32_bits(base_ts.tv_sec);
+ ocelot_write(ocelot,
+ (sgi->ipv_valid ? ANA_SG_CONFIG_REG_3_IPV_VALID : 0) |
+ ANA_SG_CONFIG_REG_3_INIT_IPV(sgi->init_ipv) |
+ ANA_SG_CONFIG_REG_3_GATE_ENABLE |
+ ANA_SG_CONFIG_REG_3_LIST_LENGTH(sgi->num_entries) |
+ ANA_SG_CONFIG_REG_3_INIT_GATE_STATE |
+ ANA_SG_CONFIG_REG_3_BASE_TIME_SEC_MSB(val),
+ ANA_SG_CONFIG_REG_3);
+
+ ocelot_write(ocelot, sgi->cycletime, ANA_SG_CONFIG_REG_4);
+
+ e = sgi->entries;
+ for (i = 0; i < sgi->num_entries; i++) {
+ u32 ips = (e[i].ipv < 0) ? 0 : (e[i].ipv + 8);
+
+ ocelot_write_rix(ocelot, ANA_SG_GCL_GS_CONFIG_IPS(ips) |
+ (e[i].gate_state ?
+ ANA_SG_GCL_GS_CONFIG_GATE_STATE : 0),
+ ANA_SG_GCL_GS_CONFIG, i);
+
+ interval_sum += e[i].interval;
+ ocelot_write_rix(ocelot, interval_sum, ANA_SG_GCL_TI_CONFIG, i);
+ }
+
+ ocelot_rmw(ocelot, ANA_SG_ACCESS_CTRL_CONFIG_CHANGE,
+ ANA_SG_ACCESS_CTRL_CONFIG_CHANGE,
+ ANA_SG_ACCESS_CTRL);
+
+ return readx_poll_timeout(vsc9959_sgi_cfg_status, ocelot, val,
+ (!(ANA_SG_ACCESS_CTRL_CONFIG_CHANGE & val)),
+ 10, 100000);
+}
+
+static int vsc9959_psfp_sgi_table_add(struct ocelot *ocelot,
+ struct felix_stream_gate *sgi)
+{
+ struct felix_stream_gate_entry *tmp;
+ struct ocelot_psfp_list *psfp;
+ int ret;
+
+ psfp = &ocelot->psfp;
+
+ list_for_each_entry(tmp, &psfp->sgi_list, list)
+ if (tmp->index == sgi->index) {
+ refcount_inc(&tmp->refcount);
+ return 0;
+ }
+
+ tmp = kzalloc(sizeof(*tmp), GFP_KERNEL);
+ if (!tmp)
+ return -ENOMEM;
+
+ ret = vsc9959_psfp_sgi_set(ocelot, sgi);
+ if (ret) {
+ kfree(tmp);
+ return ret;
+ }
+
+ tmp->index = sgi->index;
+ refcount_set(&tmp->refcount, 1);
+ list_add_tail(&tmp->list, &psfp->sgi_list);
+
+ return 0;
+}
+
+static void vsc9959_psfp_sgi_table_del(struct ocelot *ocelot,
+ u32 index)
+{
+ struct felix_stream_gate_entry *tmp, *n;
+ struct felix_stream_gate sgi = {0};
+ struct ocelot_psfp_list *psfp;
+ u8 z;
+
+ psfp = &ocelot->psfp;
+
+ list_for_each_entry_safe(tmp, n, &psfp->sgi_list, list)
+ if (tmp->index == index) {
+ z = refcount_dec_and_test(&tmp->refcount);
+ if (z) {
+ sgi.index = index;
+ sgi.enable = 0;
+ vsc9959_psfp_sgi_set(ocelot, &sgi);
+ list_del(&tmp->list);
+ kfree(tmp);
+ }
+ break;
+ }
+}
+
+static void vsc9959_psfp_counters_get(struct ocelot *ocelot, u32 index,
+ struct felix_stream_filter_counters *counters)
+{
+ ocelot_rmw(ocelot, SYS_STAT_CFG_STAT_VIEW(index),
+ SYS_STAT_CFG_STAT_VIEW_M,
+ SYS_STAT_CFG);
+
+ counters->match = ocelot_read_gix(ocelot, SYS_CNT, 0x200);
+ counters->not_pass_gate = ocelot_read_gix(ocelot, SYS_CNT, 0x201);
+ counters->not_pass_sdu = ocelot_read_gix(ocelot, SYS_CNT, 0x202);
+ counters->red = ocelot_read_gix(ocelot, SYS_CNT, 0x203);
+
+ /* Clear the PSFP counter. */
+ ocelot_write(ocelot,
+ SYS_STAT_CFG_STAT_VIEW(index) |
+ SYS_STAT_CFG_STAT_CLEAR_SHOT(0x10),
+ SYS_STAT_CFG);
+}
+
+static int vsc9959_psfp_filter_add(struct ocelot *ocelot, int port,
+ struct flow_cls_offload *f)
+{
+ struct netlink_ext_ack *extack = f->common.extack;
+ struct felix_stream_filter old_sfi, *sfi_entry;
+ struct felix_stream_filter sfi = {0};
+ const struct flow_action_entry *a;
+ struct felix_stream *stream_entry;
+ struct felix_stream stream = {0};
+ struct felix_stream_gate *sgi;
+ struct ocelot_psfp_list *psfp;
+ struct ocelot_policer pol;
+ int ret, i, size;
+ u64 rate, burst;
+ u32 index;
+
+ psfp = &ocelot->psfp;
+
+ ret = vsc9959_stream_identify(f, &stream);
+ if (ret) {
+ NL_SET_ERR_MSG_MOD(extack, "Only can match on VID, PCP, and dest MAC");
+ return ret;
+ }
+
+ flow_action_for_each(i, a, &f->rule->action) {
+ switch (a->id) {
+ case FLOW_ACTION_GATE:
+ size = struct_size(sgi, entries, a->gate.num_entries);
+ sgi = kzalloc(size, GFP_KERNEL);
+ vsc9959_psfp_parse_gate(a, sgi);
+ ret = vsc9959_psfp_sgi_table_add(ocelot, sgi);
+ if (ret) {
+ kfree(sgi);
+ goto err;
+ }
+ sfi.sg_valid = 1;
+ sfi.sgid = sgi->index;
+ kfree(sgi);
+ break;
+ case FLOW_ACTION_POLICE:
+ index = a->hw_index + VSC9959_PSFP_POLICER_BASE;
+ if (index > VSC9959_PSFP_POLICER_MAX) {
+ ret = -EINVAL;
+ goto err;
+ }
+
+ rate = a->police.rate_bytes_ps;
+ burst = rate * PSCHED_NS2TICKS(a->police.burst);
+ pol = (struct ocelot_policer) {
+ .burst = div_u64(burst, PSCHED_TICKS_PER_SEC),
+ .rate = div_u64(rate, 1000) * 8,
+ };
+ ret = ocelot_vcap_policer_add(ocelot, index, &pol);
+ if (ret)
+ goto err;
+
+ sfi.fm_valid = 1;
+ sfi.fmid = index;
+ sfi.maxsdu = a->police.mtu;
+ break;
+ default:
+ return -EOPNOTSUPP;
+ }
+ }
+
+ stream.ports = BIT(port);
+ stream.port = port;
+
+ sfi.portmask = stream.ports;
+ sfi.prio_valid = (stream.prio < 0 ? 0 : 1);
+ sfi.prio = (sfi.prio_valid ? stream.prio : 0);
+ sfi.enable = 1;
+
+ /* Check if stream is set. */
+ stream_entry = vsc9959_stream_table_lookup(&psfp->stream_list, &stream);
+ if (stream_entry) {
+ if (stream_entry->ports & BIT(port)) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "The stream is added on this port");
+ ret = -EEXIST;
+ goto err;
+ }
+
+ if (stream_entry->ports != BIT(stream_entry->port)) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "The stream is added on two ports");
+ ret = -EEXIST;
+ goto err;
+ }
+
+ stream_entry->ports |= BIT(port);
+ stream.ports = stream_entry->ports;
+
+ sfi_entry = vsc9959_psfp_sfi_table_get(&psfp->sfi_list,
+ stream_entry->sfid);
+ memcpy(&old_sfi, sfi_entry, sizeof(old_sfi));
+
+ vsc9959_psfp_sfi_table_del(ocelot, stream_entry->sfid);
+
+ old_sfi.portmask = stream_entry->ports;
+ sfi.portmask = stream.ports;
+
+ if (stream_entry->port > port) {
+ ret = vsc9959_psfp_sfi_table_add2(ocelot, &sfi,
+ &old_sfi);
+ stream_entry->dummy = true;
+ } else {
+ ret = vsc9959_psfp_sfi_table_add2(ocelot, &old_sfi,
+ &sfi);
+ stream.dummy = true;
+ }
+ if (ret)
+ goto err;
+
+ stream_entry->sfid = old_sfi.index;
+ } else {
+ ret = vsc9959_psfp_sfi_table_add(ocelot, &sfi);
+ if (ret)
+ goto err;
+ }
+
+ stream.sfid = sfi.index;
+ stream.sfid_valid = 1;
+ ret = vsc9959_stream_table_add(ocelot, &psfp->stream_list,
+ &stream, extack);
+ if (ret) {
+ vsc9959_psfp_sfi_table_del(ocelot, stream.sfid);
+ goto err;
+ }
+
+ return 0;
+
+err:
+ if (sfi.sg_valid)
+ vsc9959_psfp_sgi_table_del(ocelot, sfi.sgid);
+
+ if (sfi.fm_valid)
+ ocelot_vcap_policer_del(ocelot, sfi.fmid);
+
+ return ret;
+}
+
+static int vsc9959_psfp_filter_del(struct ocelot *ocelot,
+ struct flow_cls_offload *f)
+{
+ struct felix_stream *stream, tmp, *stream_entry;
+ static struct felix_stream_filter *sfi;
+ struct ocelot_psfp_list *psfp;
+
+ psfp = &ocelot->psfp;
+
+ stream = vsc9959_stream_table_get(&psfp->stream_list, f->cookie);
+ if (!stream)
+ return -ENOMEM;
+
+ sfi = vsc9959_psfp_sfi_table_get(&psfp->sfi_list, stream->sfid);
+ if (!sfi)
+ return -ENOMEM;
+
+ if (sfi->sg_valid)
+ vsc9959_psfp_sgi_table_del(ocelot, sfi->sgid);
+
+ if (sfi->fm_valid)
+ ocelot_vcap_policer_del(ocelot, sfi->fmid);
+
+ vsc9959_psfp_sfi_table_del(ocelot, stream->sfid);
+
+ memcpy(&tmp, stream, sizeof(tmp));
+
+ stream->sfid_valid = 0;
+ vsc9959_stream_table_del(ocelot, stream);
+
+ stream_entry = vsc9959_stream_table_lookup(&psfp->stream_list, &tmp);
+ if (stream_entry) {
+ stream_entry->ports = BIT(stream_entry->port);
+ if (stream_entry->dummy) {
+ stream_entry->dummy = false;
+ vsc9959_mact_stream_set(ocelot, stream_entry, NULL);
+ }
+ vsc9959_psfp_sfidmask_set(ocelot, stream_entry->sfid,
+ stream_entry->ports);
+ }
+
+ return 0;
+}
+
+static int vsc9959_psfp_stats_get(struct ocelot *ocelot,
+ struct flow_cls_offload *f,
+ struct flow_stats *stats)
+{
+ struct felix_stream_filter_counters counters;
+ struct ocelot_psfp_list *psfp;
+ struct felix_stream *stream;
+
+ psfp = &ocelot->psfp;
+ stream = vsc9959_stream_table_get(&psfp->stream_list, f->cookie);
+ if (!stream)
+ return -ENOMEM;
+
+ vsc9959_psfp_counters_get(ocelot, stream->sfid, &counters);
+
+ stats->pkts = counters.match;
+ stats->drops = counters.not_pass_gate + counters.not_pass_sdu +
+ counters.red;
+
+ return 0;
+}
+
+static void vsc9959_psfp_init(struct ocelot *ocelot)
+{
+ struct ocelot_psfp_list *psfp = &ocelot->psfp;
+
+ INIT_LIST_HEAD(&psfp->stream_list);
+ INIT_LIST_HEAD(&psfp->sfi_list);
+ INIT_LIST_HEAD(&psfp->sgi_list);
+}
+
+/* When using cut-through forwarding and the egress port runs at a higher data
+ * rate than the ingress port, the packet currently under transmission would
+ * suffer an underrun since it would be transmitted faster than it is received.
+ * The Felix switch implementation of cut-through forwarding does not check in
+ * hardware whether this condition is satisfied or not, so we must restrict the
+ * list of ports that have cut-through forwarding enabled on egress to only be
+ * the ports operating at the lowest link speed within their respective
+ * forwarding domain.
+ */
+static void vsc9959_cut_through_fwd(struct ocelot *ocelot)
+{
+ struct felix *felix = ocelot_to_felix(ocelot);
+ struct dsa_switch *ds = felix->ds;
+ int port, other_port;
+
+ lockdep_assert_held(&ocelot->fwd_domain_lock);
+
+ for (port = 0; port < ocelot->num_phys_ports; port++) {
+ struct ocelot_port *ocelot_port = ocelot->ports[port];
+ int min_speed = ocelot_port->speed;
+ unsigned long mask = 0;
+ u32 tmp, val = 0;
+
+ /* Disable cut-through on ports that are down */
+ if (ocelot_port->speed <= 0)
+ goto set;
+
+ if (dsa_is_cpu_port(ds, port)) {
+ /* Ocelot switches forward from the NPI port towards
+ * any port, regardless of it being in the NPI port's
+ * forwarding domain or not.
+ */
+ mask = dsa_user_ports(ds);
+ } else {
+ mask = ocelot_get_bridge_fwd_mask(ocelot, port);
+ mask &= ~BIT(port);
+ if (ocelot->npi >= 0)
+ mask |= BIT(ocelot->npi);
+ else
+ mask |= ocelot_get_dsa_8021q_cpu_mask(ocelot);
+ }
+
+ /* Calculate the minimum link speed, among the ports that are
+ * up, of this source port's forwarding domain.
+ */
+ for_each_set_bit(other_port, &mask, ocelot->num_phys_ports) {
+ struct ocelot_port *other_ocelot_port;
+
+ other_ocelot_port = ocelot->ports[other_port];
+ if (other_ocelot_port->speed <= 0)
+ continue;
+
+ if (min_speed > other_ocelot_port->speed)
+ min_speed = other_ocelot_port->speed;
+ }
+
+ /* Enable cut-through forwarding for all traffic classes. */
+ if (ocelot_port->speed == min_speed)
+ val = GENMASK(7, 0);
+
+set:
+ tmp = ocelot_read_rix(ocelot, ANA_CUT_THRU_CFG, port);
+ if (tmp == val)
+ continue;
+
+ dev_dbg(ocelot->dev,
+ "port %d fwd mask 0x%lx speed %d min_speed %d, %s cut-through forwarding\n",
+ port, mask, ocelot_port->speed, min_speed,
+ val ? "enabling" : "disabling");
+
+ ocelot_write_rix(ocelot, val, ANA_CUT_THRU_CFG, port);
+ }
+}
+
+static const struct ocelot_ops vsc9959_ops = {
+ .reset = vsc9959_reset,
+ .wm_enc = vsc9959_wm_enc,
+ .wm_dec = vsc9959_wm_dec,
+ .wm_stat = vsc9959_wm_stat,
+ .port_to_netdev = felix_port_to_netdev,
+ .netdev_to_port = felix_netdev_to_port,
+ .psfp_init = vsc9959_psfp_init,
+ .psfp_filter_add = vsc9959_psfp_filter_add,
+ .psfp_filter_del = vsc9959_psfp_filter_del,
+ .psfp_stats_get = vsc9959_psfp_stats_get,
+ .cut_through_fwd = vsc9959_cut_through_fwd,
+};
+
static const struct felix_info felix_info_vsc9959 = {
.target_io_res = vsc9959_target_io_res,
.port_io_res = vsc9959_port_io_res,
@@ -1354,11 +2224,13 @@ static const struct felix_info felix_info_vsc9959 = {
.stats_layout = vsc9959_stats_layout,
.num_stats = ARRAY_SIZE(vsc9959_stats_layout),
.vcap = vsc9959_vcap_props,
+ .vcap_pol_base = VSC9959_VCAP_POLICER_BASE,
+ .vcap_pol_max = VSC9959_VCAP_POLICER_MAX,
+ .vcap_pol_base2 = 0,
+ .vcap_pol_max2 = 0,
.num_mact_rows = 2048,
.num_ports = 6,
.num_tx_queues = OCELOT_NUM_TC,
- .switch_pci_bar = 4,
- .imdio_pci_bar = 0,
.quirk_no_xtr_irq = true,
.ptp_caps = &vsc9959_ptp_caps,
.mdio_bus_alloc = vsc9959_mdio_bus_alloc,
@@ -1367,6 +2239,7 @@ static const struct felix_info felix_info_vsc9959 = {
.prevalidate_phy_mode = vsc9959_prevalidate_phy_mode,
.port_setup_tc = vsc9959_port_setup_tc,
.port_sched_speed_set = vsc9959_sched_speed_set,
+ .init_regmap = ocelot_regmap_init,
};
static irqreturn_t felix_irq_handler(int irq, void *data)
@@ -1417,10 +2290,8 @@ static int felix_pci_probe(struct pci_dev *pdev,
ocelot->dev = &pdev->dev;
ocelot->num_flooding_pgids = OCELOT_NUM_TC;
felix->info = &felix_info_vsc9959;
- felix->switch_base = pci_resource_start(pdev,
- felix->info->switch_pci_bar);
- felix->imdio_base = pci_resource_start(pdev,
- felix->info->imdio_pci_bar);
+ felix->switch_base = pci_resource_start(pdev, VSC9959_SWITCH_PCI_BAR);
+ felix->imdio_base = pci_resource_start(pdev, VSC9959_IMDIO_PCI_BAR);
pci_set_master(pdev);
diff --git a/drivers/net/dsa/ocelot/seville_vsc9953.c b/drivers/net/dsa/ocelot/seville_vsc9953.c
index 92eae63..8c1c9da 100644
--- a/drivers/net/dsa/ocelot/seville_vsc9953.c
+++ b/drivers/net/dsa/ocelot/seville_vsc9953.c
@@ -6,18 +6,18 @@
#include <soc/mscc/ocelot_vcap.h>
#include <soc/mscc/ocelot_sys.h>
#include <soc/mscc/ocelot.h>
+#include <linux/mdio/mdio-mscc-miim.h>
+#include <linux/of_mdio.h>
#include <linux/of_platform.h>
#include <linux/pcs-lynx.h>
#include <linux/dsa/ocelot.h>
#include <linux/iopoll.h>
#include "felix.h"
-#define MSCC_MIIM_CMD_OPR_WRITE BIT(1)
-#define MSCC_MIIM_CMD_OPR_READ BIT(2)
-#define MSCC_MIIM_CMD_WRDATA_SHIFT 4
-#define MSCC_MIIM_CMD_REGAD_SHIFT 20
-#define MSCC_MIIM_CMD_PHYAD_SHIFT 25
-#define MSCC_MIIM_CMD_VLD BIT(31)
+#define VSC9953_VCAP_POLICER_BASE 11
+#define VSC9953_VCAP_POLICER_MAX 31
+#define VSC9953_VCAP_POLICER_BASE2 120
+#define VSC9953_VCAP_POLICER_MAX2 161
static const u32 vsc9953_ana_regmap[] = {
REG(ANA_ADVLEARN, 0x00b500),
@@ -857,7 +857,6 @@ static struct vcap_props vsc9953_vcap_props[] = {
#define VSC9953_INIT_TIMEOUT 50000
#define VSC9953_GCB_RST_SLEEP 100
#define VSC9953_SYS_RAMINIT_SLEEP 80
-#define VCS9953_MII_TIMEOUT 10000
static int vsc9953_gcb_soft_rst_status(struct ocelot *ocelot)
{
@@ -877,82 +876,6 @@ static int vsc9953_sys_ram_init_status(struct ocelot *ocelot)
return val;
}
-static int vsc9953_gcb_miim_pending_status(struct ocelot *ocelot)
-{
- int val;
-
- ocelot_field_read(ocelot, GCB_MIIM_MII_STATUS_PENDING, &val);
-
- return val;
-}
-
-static int vsc9953_gcb_miim_busy_status(struct ocelot *ocelot)
-{
- int val;
-
- ocelot_field_read(ocelot, GCB_MIIM_MII_STATUS_BUSY, &val);
-
- return val;
-}
-
-static int vsc9953_mdio_write(struct mii_bus *bus, int phy_id, int regnum,
- u16 value)
-{
- struct ocelot *ocelot = bus->priv;
- int err, cmd, val;
-
- /* Wait while MIIM controller becomes idle */
- err = readx_poll_timeout(vsc9953_gcb_miim_pending_status, ocelot,
- val, !val, 10, VCS9953_MII_TIMEOUT);
- if (err) {
- dev_err(ocelot->dev, "MDIO write: pending timeout\n");
- goto out;
- }
-
- cmd = MSCC_MIIM_CMD_VLD | (phy_id << MSCC_MIIM_CMD_PHYAD_SHIFT) |
- (regnum << MSCC_MIIM_CMD_REGAD_SHIFT) |
- (value << MSCC_MIIM_CMD_WRDATA_SHIFT) |
- MSCC_MIIM_CMD_OPR_WRITE;
-
- ocelot_write(ocelot, cmd, GCB_MIIM_MII_CMD);
-
-out:
- return err;
-}
-
-static int vsc9953_mdio_read(struct mii_bus *bus, int phy_id, int regnum)
-{
- struct ocelot *ocelot = bus->priv;
- int err, cmd, val;
-
- /* Wait until MIIM controller becomes idle */
- err = readx_poll_timeout(vsc9953_gcb_miim_pending_status, ocelot,
- val, !val, 10, VCS9953_MII_TIMEOUT);
- if (err) {
- dev_err(ocelot->dev, "MDIO read: pending timeout\n");
- goto out;
- }
-
- /* Write the MIIM COMMAND register */
- cmd = MSCC_MIIM_CMD_VLD | (phy_id << MSCC_MIIM_CMD_PHYAD_SHIFT) |
- (regnum << MSCC_MIIM_CMD_REGAD_SHIFT) | MSCC_MIIM_CMD_OPR_READ;
-
- ocelot_write(ocelot, cmd, GCB_MIIM_MII_CMD);
-
- /* Wait while read operation via the MIIM controller is in progress */
- err = readx_poll_timeout(vsc9953_gcb_miim_busy_status, ocelot,
- val, !val, 10, VCS9953_MII_TIMEOUT);
- if (err) {
- dev_err(ocelot->dev, "MDIO read: busy timeout\n");
- goto out;
- }
-
- val = ocelot_read(ocelot, GCB_MIIM_MII_DATA);
-
- err = val & 0xFFFF;
-out:
- return err;
-}
/* CORE_ENA is in SYS:SYSTEM:RESET_CFG
* MEM_INIT is in SYS:SYSTEM:RESET_CFG
@@ -1089,26 +1012,24 @@ static int vsc9953_mdio_bus_alloc(struct ocelot *ocelot)
int rc;
felix->pcs = devm_kcalloc(dev, felix->info->num_ports,
- sizeof(struct phy_device *),
+ sizeof(struct phylink_pcs *),
GFP_KERNEL);
if (!felix->pcs) {
dev_err(dev, "failed to allocate array for PCS PHYs\n");
return -ENOMEM;
}
- bus = devm_mdiobus_alloc(dev);
- if (!bus)
- return -ENOMEM;
+ rc = mscc_miim_setup(dev, &bus, "VSC9953 internal MDIO bus",
+ ocelot->targets[GCB],
+ ocelot->map[GCB][GCB_MIIM_MII_STATUS & REG_MASK]);
- bus->name = "VSC9953 internal MDIO bus";
- bus->read = vsc9953_mdio_read;
- bus->write = vsc9953_mdio_write;
- bus->parent = dev;
- bus->priv = ocelot;
- snprintf(bus->id, MII_BUS_ID_SIZE, "%s-imdio", dev_name(dev));
+ if (rc) {
+ dev_err(dev, "failed to setup MDIO bus\n");
+ return rc;
+ }
/* Needed in order to initialize the bus mutex lock */
- rc = mdiobus_register(bus);
+ rc = of_mdiobus_register(bus, NULL);
if (rc < 0) {
dev_err(dev, "failed to register MDIO bus\n");
return rc;
@@ -1118,9 +1039,9 @@ static int vsc9953_mdio_bus_alloc(struct ocelot *ocelot)
for (port = 0; port < felix->info->num_ports; port++) {
struct ocelot_port *ocelot_port = ocelot->ports[port];
+ struct phylink_pcs *phylink_pcs;
+ struct mdio_device *mdio_device;
int addr = port + 4;
- struct mdio_device *pcs;
- struct lynx_pcs *lynx;
if (dsa_is_unused_port(felix->ds, port))
continue;
@@ -1128,17 +1049,17 @@ static int vsc9953_mdio_bus_alloc(struct ocelot *ocelot)
if (ocelot_port->phy_mode == PHY_INTERFACE_MODE_INTERNAL)
continue;
- pcs = mdio_device_create(felix->imdio, addr);
- if (IS_ERR(pcs))
+ mdio_device = mdio_device_create(felix->imdio, addr);
+ if (IS_ERR(mdio_device))
continue;
- lynx = lynx_pcs_create(pcs);
- if (!lynx) {
- mdio_device_free(pcs);
+ phylink_pcs = lynx_pcs_create(mdio_device);
+ if (!phylink_pcs) {
+ mdio_device_free(mdio_device);
continue;
}
- felix->pcs[port] = lynx;
+ felix->pcs[port] = phylink_pcs;
dev_info(dev, "Found PCS at internal MDIO address %d\n", addr);
}
@@ -1152,13 +1073,15 @@ static void vsc9953_mdio_bus_free(struct ocelot *ocelot)
int port;
for (port = 0; port < ocelot->num_phys_ports; port++) {
- struct lynx_pcs *pcs = felix->pcs[port];
+ struct phylink_pcs *phylink_pcs = felix->pcs[port];
+ struct mdio_device *mdio_device;
- if (!pcs)
+ if (!phylink_pcs)
continue;
- mdio_device_free(pcs->mdio);
- lynx_pcs_destroy(pcs);
+ mdio_device = lynx_get_mdio_device(phylink_pcs);
+ mdio_device_free(mdio_device);
+ lynx_pcs_destroy(phylink_pcs);
}
mdiobus_unregister(felix->imdio);
}
@@ -1172,6 +1095,10 @@ static const struct felix_info seville_info_vsc9953 = {
.stats_layout = vsc9953_stats_layout,
.num_stats = ARRAY_SIZE(vsc9953_stats_layout),
.vcap = vsc9953_vcap_props,
+ .vcap_pol_base = VSC9953_VCAP_POLICER_BASE,
+ .vcap_pol_max = VSC9953_VCAP_POLICER_MAX,
+ .vcap_pol_base2 = VSC9953_VCAP_POLICER_BASE2,
+ .vcap_pol_max2 = VSC9953_VCAP_POLICER_MAX2,
.num_mact_rows = 2048,
.num_ports = 10,
.num_tx_queues = OCELOT_NUM_TC,
@@ -1179,6 +1106,7 @@ static const struct felix_info seville_info_vsc9953 = {
.mdio_bus_free = vsc9953_mdio_bus_free,
.phylink_validate = vsc9953_phylink_validate,
.prevalidate_phy_mode = vsc9953_prevalidate_phy_mode,
+ .init_regmap = ocelot_regmap_init,
};
static int seville_probe(struct platform_device *pdev)
diff --git a/drivers/net/dsa/qca8k.c b/drivers/net/dsa/qca8k.c
index 147ca39..0396945 100644
--- a/drivers/net/dsa/qca8k.c
+++ b/drivers/net/dsa/qca8k.c
@@ -9,6 +9,8 @@
#include <linux/module.h>
#include <linux/phy.h>
#include <linux/netdevice.h>
+#include <linux/bitfield.h>
+#include <linux/regmap.h>
#include <net/dsa.h>
#include <linux/of_net.h>
#include <linux/of_mdio.h>
@@ -68,6 +70,8 @@ static const struct qca8k_mib_desc ar8327_mib[] = {
MIB_DESC(1, 0x9c, "TxExcDefer"),
MIB_DESC(1, 0xa0, "TxDefer"),
MIB_DESC(1, 0xa4, "TxLateCol"),
+ MIB_DESC(1, 0xa8, "RXUnicast"),
+ MIB_DESC(1, 0xac, "TXUnicast"),
};
/* The 32bit switch registers are accessed indirectly. To achieve this we need
@@ -151,6 +155,25 @@ qca8k_set_page(struct mii_bus *bus, u16 page)
static int
qca8k_read(struct qca8k_priv *priv, u32 reg, u32 *val)
{
+ return regmap_read(priv->regmap, reg, val);
+}
+
+static int
+qca8k_write(struct qca8k_priv *priv, u32 reg, u32 val)
+{
+ return regmap_write(priv->regmap, reg, val);
+}
+
+static int
+qca8k_rmw(struct qca8k_priv *priv, u32 reg, u32 mask, u32 write_val)
+{
+ return regmap_update_bits(priv->regmap, reg, mask, write_val);
+}
+
+static int
+qca8k_regmap_read(void *ctx, uint32_t reg, uint32_t *val)
+{
+ struct qca8k_priv *priv = (struct qca8k_priv *)ctx;
struct mii_bus *bus = priv->bus;
u16 r1, r2, page;
int ret;
@@ -171,8 +194,9 @@ qca8k_read(struct qca8k_priv *priv, u32 reg, u32 *val)
}
static int
-qca8k_write(struct qca8k_priv *priv, u32 reg, u32 val)
+qca8k_regmap_write(void *ctx, uint32_t reg, uint32_t val)
{
+ struct qca8k_priv *priv = (struct qca8k_priv *)ctx;
struct mii_bus *bus = priv->bus;
u16 r1, r2, page;
int ret;
@@ -193,8 +217,9 @@ qca8k_write(struct qca8k_priv *priv, u32 reg, u32 val)
}
static int
-qca8k_rmw(struct qca8k_priv *priv, u32 reg, u32 mask, u32 write_val)
+qca8k_regmap_update_bits(void *ctx, uint32_t reg, uint32_t mask, uint32_t write_val)
{
+ struct qca8k_priv *priv = (struct qca8k_priv *)ctx;
struct mii_bus *bus = priv->bus;
u16 r1, r2, page;
u32 val;
@@ -222,34 +247,6 @@ qca8k_rmw(struct qca8k_priv *priv, u32 reg, u32 mask, u32 write_val)
return ret;
}
-static int
-qca8k_reg_set(struct qca8k_priv *priv, u32 reg, u32 val)
-{
- return qca8k_rmw(priv, reg, 0, val);
-}
-
-static int
-qca8k_reg_clear(struct qca8k_priv *priv, u32 reg, u32 val)
-{
- return qca8k_rmw(priv, reg, val, 0);
-}
-
-static int
-qca8k_regmap_read(void *ctx, uint32_t reg, uint32_t *val)
-{
- struct qca8k_priv *priv = (struct qca8k_priv *)ctx;
-
- return qca8k_read(priv, reg, val);
-}
-
-static int
-qca8k_regmap_write(void *ctx, uint32_t reg, uint32_t val)
-{
- struct qca8k_priv *priv = (struct qca8k_priv *)ctx;
-
- return qca8k_write(priv, reg, val);
-}
-
static const struct regmap_range qca8k_readable_ranges[] = {
regmap_reg_range(0x0000, 0x00e4), /* Global control */
regmap_reg_range(0x0100, 0x0168), /* EEE control */
@@ -281,26 +278,19 @@ static struct regmap_config qca8k_regmap_config = {
.max_register = 0x16ac, /* end MIB - Port6 range */
.reg_read = qca8k_regmap_read,
.reg_write = qca8k_regmap_write,
+ .reg_update_bits = qca8k_regmap_update_bits,
.rd_table = &qca8k_readable_table,
+ .disable_locking = true, /* Locking is handled by qca8k read/write */
+ .cache_type = REGCACHE_NONE, /* Explicitly disable CACHE */
};
static int
qca8k_busy_wait(struct qca8k_priv *priv, u32 reg, u32 mask)
{
- int ret, ret1;
u32 val;
- ret = read_poll_timeout(qca8k_read, ret1, !(val & mask),
- 0, QCA8K_BUSY_WAIT_TIMEOUT * USEC_PER_MSEC, false,
- priv, reg, &val);
-
- /* Check if qca8k_read has failed for a different reason
- * before returning -ETIMEDOUT
- */
- if (ret < 0 && ret1 < 0)
- return ret1;
-
- return ret;
+ return regmap_read_poll_timeout(priv->regmap, reg, val, !(val & mask), 0,
+ QCA8K_BUSY_WAIT_TIMEOUT * USEC_PER_MSEC);
}
static int
@@ -319,18 +309,18 @@ qca8k_fdb_read(struct qca8k_priv *priv, struct qca8k_fdb *fdb)
}
/* vid - 83:72 */
- fdb->vid = (reg[2] >> QCA8K_ATU_VID_S) & QCA8K_ATU_VID_M;
+ fdb->vid = FIELD_GET(QCA8K_ATU_VID_MASK, reg[2]);
/* aging - 67:64 */
- fdb->aging = reg[2] & QCA8K_ATU_STATUS_M;
+ fdb->aging = FIELD_GET(QCA8K_ATU_STATUS_MASK, reg[2]);
/* portmask - 54:48 */
- fdb->port_mask = (reg[1] >> QCA8K_ATU_PORT_S) & QCA8K_ATU_PORT_M;
+ fdb->port_mask = FIELD_GET(QCA8K_ATU_PORT_MASK, reg[1]);
/* mac - 47:0 */
- fdb->mac[0] = (reg[1] >> QCA8K_ATU_ADDR0_S) & 0xff;
- fdb->mac[1] = reg[1] & 0xff;
- fdb->mac[2] = (reg[0] >> QCA8K_ATU_ADDR2_S) & 0xff;
- fdb->mac[3] = (reg[0] >> QCA8K_ATU_ADDR3_S) & 0xff;
- fdb->mac[4] = (reg[0] >> QCA8K_ATU_ADDR4_S) & 0xff;
- fdb->mac[5] = reg[0] & 0xff;
+ fdb->mac[0] = FIELD_GET(QCA8K_ATU_ADDR0_MASK, reg[1]);
+ fdb->mac[1] = FIELD_GET(QCA8K_ATU_ADDR1_MASK, reg[1]);
+ fdb->mac[2] = FIELD_GET(QCA8K_ATU_ADDR2_MASK, reg[0]);
+ fdb->mac[3] = FIELD_GET(QCA8K_ATU_ADDR3_MASK, reg[0]);
+ fdb->mac[4] = FIELD_GET(QCA8K_ATU_ADDR4_MASK, reg[0]);
+ fdb->mac[5] = FIELD_GET(QCA8K_ATU_ADDR5_MASK, reg[0]);
return 0;
}
@@ -343,18 +333,18 @@ qca8k_fdb_write(struct qca8k_priv *priv, u16 vid, u8 port_mask, const u8 *mac,
int i;
/* vid - 83:72 */
- reg[2] = (vid & QCA8K_ATU_VID_M) << QCA8K_ATU_VID_S;
+ reg[2] = FIELD_PREP(QCA8K_ATU_VID_MASK, vid);
/* aging - 67:64 */
- reg[2] |= aging & QCA8K_ATU_STATUS_M;
+ reg[2] |= FIELD_PREP(QCA8K_ATU_STATUS_MASK, aging);
/* portmask - 54:48 */
- reg[1] = (port_mask & QCA8K_ATU_PORT_M) << QCA8K_ATU_PORT_S;
+ reg[1] = FIELD_PREP(QCA8K_ATU_PORT_MASK, port_mask);
/* mac - 47:0 */
- reg[1] |= mac[0] << QCA8K_ATU_ADDR0_S;
- reg[1] |= mac[1];
- reg[0] |= mac[2] << QCA8K_ATU_ADDR2_S;
- reg[0] |= mac[3] << QCA8K_ATU_ADDR3_S;
- reg[0] |= mac[4] << QCA8K_ATU_ADDR4_S;
- reg[0] |= mac[5];
+ reg[1] |= FIELD_PREP(QCA8K_ATU_ADDR0_MASK, mac[0]);
+ reg[1] |= FIELD_PREP(QCA8K_ATU_ADDR1_MASK, mac[1]);
+ reg[0] |= FIELD_PREP(QCA8K_ATU_ADDR2_MASK, mac[2]);
+ reg[0] |= FIELD_PREP(QCA8K_ATU_ADDR3_MASK, mac[3]);
+ reg[0] |= FIELD_PREP(QCA8K_ATU_ADDR4_MASK, mac[4]);
+ reg[0] |= FIELD_PREP(QCA8K_ATU_ADDR5_MASK, mac[5]);
/* load the array into the ARL table */
for (i = 0; i < 3; i++)
@@ -372,7 +362,7 @@ qca8k_fdb_access(struct qca8k_priv *priv, enum qca8k_fdb_cmd cmd, int port)
reg |= cmd;
if (port >= 0) {
reg |= QCA8K_ATU_FUNC_PORT_EN;
- reg |= (port & QCA8K_ATU_FUNC_PORT_M) << QCA8K_ATU_FUNC_PORT_S;
+ reg |= FIELD_PREP(QCA8K_ATU_FUNC_PORT_MASK, port);
}
/* Write the function register triggering the table access */
@@ -446,6 +436,81 @@ qca8k_fdb_flush(struct qca8k_priv *priv)
}
static int
+qca8k_fdb_search_and_insert(struct qca8k_priv *priv, u8 port_mask,
+ const u8 *mac, u16 vid)
+{
+ struct qca8k_fdb fdb = { 0 };
+ int ret;
+
+ mutex_lock(&priv->reg_mutex);
+
+ qca8k_fdb_write(priv, vid, 0, mac, 0);
+ ret = qca8k_fdb_access(priv, QCA8K_FDB_SEARCH, -1);
+ if (ret < 0)
+ goto exit;
+
+ ret = qca8k_fdb_read(priv, &fdb);
+ if (ret < 0)
+ goto exit;
+
+ /* Rule exist. Delete first */
+ if (!fdb.aging) {
+ ret = qca8k_fdb_access(priv, QCA8K_FDB_PURGE, -1);
+ if (ret)
+ goto exit;
+ }
+
+ /* Add port to fdb portmask */
+ fdb.port_mask |= port_mask;
+
+ qca8k_fdb_write(priv, vid, fdb.port_mask, mac, fdb.aging);
+ ret = qca8k_fdb_access(priv, QCA8K_FDB_LOAD, -1);
+
+exit:
+ mutex_unlock(&priv->reg_mutex);
+ return ret;
+}
+
+static int
+qca8k_fdb_search_and_del(struct qca8k_priv *priv, u8 port_mask,
+ const u8 *mac, u16 vid)
+{
+ struct qca8k_fdb fdb = { 0 };
+ int ret;
+
+ mutex_lock(&priv->reg_mutex);
+
+ qca8k_fdb_write(priv, vid, 0, mac, 0);
+ ret = qca8k_fdb_access(priv, QCA8K_FDB_SEARCH, -1);
+ if (ret < 0)
+ goto exit;
+
+ /* Rule doesn't exist. Why delete? */
+ if (!fdb.aging) {
+ ret = -EINVAL;
+ goto exit;
+ }
+
+ ret = qca8k_fdb_access(priv, QCA8K_FDB_PURGE, -1);
+ if (ret)
+ goto exit;
+
+ /* Only port in the rule is this port. Don't re insert */
+ if (fdb.port_mask == port_mask)
+ goto exit;
+
+ /* Remove port from port mask */
+ fdb.port_mask &= ~port_mask;
+
+ qca8k_fdb_write(priv, vid, fdb.port_mask, mac, fdb.aging);
+ ret = qca8k_fdb_access(priv, QCA8K_FDB_LOAD, -1);
+
+exit:
+ mutex_unlock(&priv->reg_mutex);
+ return ret;
+}
+
+static int
qca8k_vlan_access(struct qca8k_priv *priv, enum qca8k_vlan_cmd cmd, u16 vid)
{
u32 reg;
@@ -454,7 +519,7 @@ qca8k_vlan_access(struct qca8k_priv *priv, enum qca8k_vlan_cmd cmd, u16 vid)
/* Set the command and VLAN index */
reg = QCA8K_VTU_FUNC1_BUSY;
reg |= cmd;
- reg |= vid << QCA8K_VTU_FUNC1_VID_S;
+ reg |= FIELD_PREP(QCA8K_VTU_FUNC1_VID_MASK, vid);
/* Write the function register triggering the table access */
ret = qca8k_write(priv, QCA8K_REG_VTU_FUNC1, reg);
@@ -500,13 +565,11 @@ qca8k_vlan_add(struct qca8k_priv *priv, u8 port, u16 vid, bool untagged)
if (ret < 0)
goto out;
reg |= QCA8K_VTU_FUNC0_VALID | QCA8K_VTU_FUNC0_IVL_EN;
- reg &= ~(QCA8K_VTU_FUNC0_EG_MODE_MASK << QCA8K_VTU_FUNC0_EG_MODE_S(port));
+ reg &= ~QCA8K_VTU_FUNC0_EG_MODE_PORT_MASK(port);
if (untagged)
- reg |= QCA8K_VTU_FUNC0_EG_MODE_UNTAG <<
- QCA8K_VTU_FUNC0_EG_MODE_S(port);
+ reg |= QCA8K_VTU_FUNC0_EG_MODE_PORT_UNTAG(port);
else
- reg |= QCA8K_VTU_FUNC0_EG_MODE_TAG <<
- QCA8K_VTU_FUNC0_EG_MODE_S(port);
+ reg |= QCA8K_VTU_FUNC0_EG_MODE_PORT_TAG(port);
ret = qca8k_write(priv, QCA8K_REG_VTU_FUNC0, reg);
if (ret)
@@ -534,15 +597,13 @@ qca8k_vlan_del(struct qca8k_priv *priv, u8 port, u16 vid)
ret = qca8k_read(priv, QCA8K_REG_VTU_FUNC0, ®);
if (ret < 0)
goto out;
- reg &= ~(3 << QCA8K_VTU_FUNC0_EG_MODE_S(port));
- reg |= QCA8K_VTU_FUNC0_EG_MODE_NOT <<
- QCA8K_VTU_FUNC0_EG_MODE_S(port);
+ reg &= ~QCA8K_VTU_FUNC0_EG_MODE_PORT_MASK(port);
+ reg |= QCA8K_VTU_FUNC0_EG_MODE_PORT_NOT(port);
/* Check if we're the last member to be removed */
del = true;
for (i = 0; i < QCA8K_NUM_PORTS; i++) {
- mask = QCA8K_VTU_FUNC0_EG_MODE_NOT;
- mask <<= QCA8K_VTU_FUNC0_EG_MODE_S(i);
+ mask = QCA8K_VTU_FUNC0_EG_MODE_PORT_NOT(i);
if ((reg & mask) != mask) {
del = false;
@@ -571,7 +632,7 @@ qca8k_mib_init(struct qca8k_priv *priv)
int ret;
mutex_lock(&priv->reg_mutex);
- ret = qca8k_reg_set(priv, QCA8K_REG_MIB, QCA8K_MIB_FLUSH | QCA8K_MIB_BUSY);
+ ret = regmap_set_bits(priv->regmap, QCA8K_REG_MIB, QCA8K_MIB_FLUSH | QCA8K_MIB_BUSY);
if (ret)
goto exit;
@@ -579,7 +640,7 @@ qca8k_mib_init(struct qca8k_priv *priv)
if (ret)
goto exit;
- ret = qca8k_reg_set(priv, QCA8K_REG_MIB, QCA8K_MIB_CPU_KEEP);
+ ret = regmap_set_bits(priv->regmap, QCA8K_REG_MIB, QCA8K_MIB_CPU_KEEP);
if (ret)
goto exit;
@@ -600,9 +661,9 @@ qca8k_port_set_status(struct qca8k_priv *priv, int port, int enable)
mask |= QCA8K_PORT_STATUS_LINK_AUTO;
if (enable)
- qca8k_reg_set(priv, QCA8K_REG_PORT_STATUS(port), mask);
+ regmap_set_bits(priv->regmap, QCA8K_REG_PORT_STATUS(port), mask);
else
- qca8k_reg_clear(priv, QCA8K_REG_PORT_STATUS(port), mask);
+ regmap_clear_bits(priv->regmap, QCA8K_REG_PORT_STATUS(port), mask);
}
static u32
@@ -864,8 +925,8 @@ qca8k_setup_mdio_bus(struct qca8k_priv *priv)
* a dt-overlay and driver reload changed the configuration
*/
- return qca8k_reg_clear(priv, QCA8K_MDIO_MASTER_CTRL,
- QCA8K_MDIO_MASTER_EN);
+ return regmap_clear_bits(priv->regmap, QCA8K_MDIO_MASTER_CTRL,
+ QCA8K_MDIO_MASTER_EN);
}
/* Check if the devicetree declare the port:phy mapping */
@@ -983,7 +1044,7 @@ qca8k_parse_port_config(struct qca8k_priv *priv)
u32 delay;
/* We have 2 CPU port. Check them */
- for (port = 0; port < QCA8K_NUM_PORTS && cpu_port_index < QCA8K_NUM_CPU_PORTS; port++) {
+ for (port = 0; port < QCA8K_NUM_PORTS; port++) {
/* Skip every other port */
if (port != 0 && port != 6)
continue;
@@ -1014,7 +1075,7 @@ qca8k_parse_port_config(struct qca8k_priv *priv)
mode == PHY_INTERFACE_MODE_RGMII_TXID)
delay = 1;
- if (delay > QCA8K_MAX_DELAY) {
+ if (!FIELD_FIT(QCA8K_PORT_PAD_RGMII_TX_DELAY_MASK, delay)) {
dev_err(priv->dev, "rgmii tx delay is limited to a max value of 3ns, setting to the max value");
delay = 3;
}
@@ -1030,7 +1091,7 @@ qca8k_parse_port_config(struct qca8k_priv *priv)
mode == PHY_INTERFACE_MODE_RGMII_RXID)
delay = 2;
- if (delay > QCA8K_MAX_DELAY) {
+ if (!FIELD_FIT(QCA8K_PORT_PAD_RGMII_RX_DELAY_MASK, delay)) {
dev_err(priv->dev, "rgmii rx delay is limited to a max value of 3ns, setting to the max value");
delay = 3;
}
@@ -1089,14 +1150,6 @@ qca8k_setup(struct dsa_switch *ds)
if (ret)
return ret;
- mutex_init(&priv->reg_mutex);
-
- /* Start by setting up the register mapping */
- priv->regmap = devm_regmap_init(ds->dev, NULL, priv,
- &qca8k_regmap_config);
- if (IS_ERR(priv->regmap))
- dev_warn(priv->dev, "regmap initialization failed");
-
ret = qca8k_setup_mdio_bus(priv);
if (ret)
return ret;
@@ -1110,16 +1163,16 @@ qca8k_setup(struct dsa_switch *ds)
return ret;
/* Make sure MAC06 is disabled */
- ret = qca8k_reg_clear(priv, QCA8K_REG_PORT0_PAD_CTRL,
- QCA8K_PORT0_PAD_MAC06_EXCHANGE_EN);
+ ret = regmap_clear_bits(priv->regmap, QCA8K_REG_PORT0_PAD_CTRL,
+ QCA8K_PORT0_PAD_MAC06_EXCHANGE_EN);
if (ret) {
dev_err(priv->dev, "failed disabling MAC06 exchange");
return ret;
}
/* Enable CPU Port */
- ret = qca8k_reg_set(priv, QCA8K_REG_GLOBAL_FW_CTRL0,
- QCA8K_GLOBAL_FW_CTRL0_CPU_PORT_EN);
+ ret = regmap_set_bits(priv->regmap, QCA8K_REG_GLOBAL_FW_CTRL0,
+ QCA8K_GLOBAL_FW_CTRL0_CPU_PORT_EN);
if (ret) {
dev_err(priv->dev, "failed enabling CPU port");
return ret;
@@ -1141,8 +1194,8 @@ qca8k_setup(struct dsa_switch *ds)
/* Enable QCA header mode on all cpu ports */
if (dsa_is_cpu_port(ds, i)) {
ret = qca8k_write(priv, QCA8K_REG_PORT_HDR_CTRL(i),
- QCA8K_PORT_HDR_CTRL_ALL << QCA8K_PORT_HDR_CTRL_TX_S |
- QCA8K_PORT_HDR_CTRL_ALL << QCA8K_PORT_HDR_CTRL_RX_S);
+ FIELD_PREP(QCA8K_PORT_HDR_CTRL_TX_MASK, QCA8K_PORT_HDR_CTRL_ALL) |
+ FIELD_PREP(QCA8K_PORT_HDR_CTRL_RX_MASK, QCA8K_PORT_HDR_CTRL_ALL));
if (ret) {
dev_err(priv->dev, "failed enabling QCA header mode");
return ret;
@@ -1159,10 +1212,10 @@ qca8k_setup(struct dsa_switch *ds)
* for igmp, unknown, multicast and broadcast packet
*/
ret = qca8k_write(priv, QCA8K_REG_GLOBAL_FW_CTRL1,
- BIT(cpu_port) << QCA8K_GLOBAL_FW_CTRL1_IGMP_DP_S |
- BIT(cpu_port) << QCA8K_GLOBAL_FW_CTRL1_BC_DP_S |
- BIT(cpu_port) << QCA8K_GLOBAL_FW_CTRL1_MC_DP_S |
- BIT(cpu_port) << QCA8K_GLOBAL_FW_CTRL1_UC_DP_S);
+ FIELD_PREP(QCA8K_GLOBAL_FW_CTRL1_IGMP_DP_MASK, BIT(cpu_port)) |
+ FIELD_PREP(QCA8K_GLOBAL_FW_CTRL1_BC_DP_MASK, BIT(cpu_port)) |
+ FIELD_PREP(QCA8K_GLOBAL_FW_CTRL1_MC_DP_MASK, BIT(cpu_port)) |
+ FIELD_PREP(QCA8K_GLOBAL_FW_CTRL1_UC_DP_MASK, BIT(cpu_port)));
if (ret)
return ret;
@@ -1180,8 +1233,6 @@ qca8k_setup(struct dsa_switch *ds)
/* Individual user ports get connected to CPU port only */
if (dsa_is_user_port(ds, i)) {
- int shift = 16 * (i % 2);
-
ret = qca8k_rmw(priv, QCA8K_PORT_LOOKUP_CTRL(i),
QCA8K_PORT_LOOKUP_MEMBER,
BIT(cpu_port));
@@ -1189,8 +1240,8 @@ qca8k_setup(struct dsa_switch *ds)
return ret;
/* Enable ARP Auto-learning by default */
- ret = qca8k_reg_set(priv, QCA8K_PORT_LOOKUP_CTRL(i),
- QCA8K_PORT_LOOKUP_LEARN);
+ ret = regmap_set_bits(priv->regmap, QCA8K_PORT_LOOKUP_CTRL(i),
+ QCA8K_PORT_LOOKUP_LEARN);
if (ret)
return ret;
@@ -1198,8 +1249,8 @@ qca8k_setup(struct dsa_switch *ds)
* default egress vid
*/
ret = qca8k_rmw(priv, QCA8K_EGRESS_VLAN(i),
- 0xfff << shift,
- QCA8K_PORT_VID_DEF << shift);
+ QCA8K_EGREES_VLAN_PORT_MASK(i),
+ QCA8K_EGREES_VLAN_PORT(i, QCA8K_PORT_VID_DEF));
if (ret)
return ret;
@@ -1246,7 +1297,7 @@ qca8k_setup(struct dsa_switch *ds)
QCA8K_PORT_HOL_CTRL1_EG_PORT_BUF_EN |
QCA8K_PORT_HOL_CTRL1_WRED_EN;
qca8k_rmw(priv, QCA8K_REG_PORT_HOL_CTRL1(i),
- QCA8K_PORT_HOL_CTRL1_ING_BUF |
+ QCA8K_PORT_HOL_CTRL1_ING_BUF_MASK |
QCA8K_PORT_HOL_CTRL1_EG_PRI_BUF_EN |
QCA8K_PORT_HOL_CTRL1_EG_PORT_BUF_EN |
QCA8K_PORT_HOL_CTRL1_WRED_EN,
@@ -1269,8 +1320,8 @@ qca8k_setup(struct dsa_switch *ds)
mask = QCA8K_GLOBAL_FC_GOL_XON_THRES(288) |
QCA8K_GLOBAL_FC_GOL_XOFF_THRES(496);
qca8k_rmw(priv, QCA8K_REG_GLOBAL_FC_THRESH,
- QCA8K_GLOBAL_FC_GOL_XON_THRES_S |
- QCA8K_GLOBAL_FC_GOL_XOFF_THRES_S,
+ QCA8K_GLOBAL_FC_GOL_XON_THRES_MASK |
+ QCA8K_GLOBAL_FC_GOL_XOFF_THRES_MASK,
mask);
}
@@ -1285,6 +1336,13 @@ qca8k_setup(struct dsa_switch *ds)
/* We don't have interrupts for link changes, so we need to poll */
ds->pcs_poll = true;
+ /* Set min a max ageing value supported */
+ ds->ageing_time_min = 7000;
+ ds->ageing_time_max = 458745000;
+
+ /* Set max number of LAGs supported */
+ ds->num_lag_ids = QCA8K_NUM_LAGS;
+
return 0;
}
@@ -1631,12 +1689,16 @@ qca8k_phylink_mac_link_up(struct dsa_switch *ds, int port, unsigned int mode,
static void
qca8k_get_strings(struct dsa_switch *ds, int port, u32 stringset, uint8_t *data)
{
+ const struct qca8k_match_data *match_data;
+ struct qca8k_priv *priv = ds->priv;
int i;
if (stringset != ETH_SS_STATS)
return;
- for (i = 0; i < ARRAY_SIZE(ar8327_mib); i++)
+ match_data = of_device_get_match_data(priv->dev);
+
+ for (i = 0; i < match_data->mib_count; i++)
strncpy(data + i * ETH_GSTRING_LEN, ar8327_mib[i].name,
ETH_GSTRING_LEN);
}
@@ -1646,12 +1708,15 @@ qca8k_get_ethtool_stats(struct dsa_switch *ds, int port,
uint64_t *data)
{
struct qca8k_priv *priv = (struct qca8k_priv *)ds->priv;
+ const struct qca8k_match_data *match_data;
const struct qca8k_mib_desc *mib;
u32 reg, i, val;
u32 hi = 0;
int ret;
- for (i = 0; i < ARRAY_SIZE(ar8327_mib); i++) {
+ match_data = of_device_get_match_data(priv->dev);
+
+ for (i = 0; i < match_data->mib_count; i++) {
mib = &ar8327_mib[i];
reg = QCA8K_PORT_MIB_COUNTER(port) + mib->offset;
@@ -1674,10 +1739,15 @@ qca8k_get_ethtool_stats(struct dsa_switch *ds, int port,
static int
qca8k_get_sset_count(struct dsa_switch *ds, int port, int sset)
{
+ const struct qca8k_match_data *match_data;
+ struct qca8k_priv *priv = ds->priv;
+
if (sset != ETH_SS_STATS)
return 0;
- return ARRAY_SIZE(ar8327_mib);
+ match_data = of_device_get_match_data(priv->dev);
+
+ return match_data->mib_count;
}
static int
@@ -1740,8 +1810,9 @@ qca8k_port_stp_state_set(struct dsa_switch *ds, int port, u8 state)
QCA8K_PORT_LOOKUP_STATE_MASK, stp_state);
}
-static int
-qca8k_port_bridge_join(struct dsa_switch *ds, int port, struct net_device *br)
+static int qca8k_port_bridge_join(struct dsa_switch *ds, int port,
+ struct dsa_bridge bridge,
+ bool *tx_fwd_offload)
{
struct qca8k_priv *priv = (struct qca8k_priv *)ds->priv;
int port_mask, cpu_port;
@@ -1753,14 +1824,14 @@ qca8k_port_bridge_join(struct dsa_switch *ds, int port, struct net_device *br)
for (i = 0; i < QCA8K_NUM_PORTS; i++) {
if (dsa_is_cpu_port(ds, i))
continue;
- if (dsa_to_port(ds, i)->bridge_dev != br)
+ if (!dsa_port_offloads_bridge(dsa_to_port(ds, i), &bridge))
continue;
/* Add this port to the portvlan mask of the other ports
* in the bridge
*/
- ret = qca8k_reg_set(priv,
- QCA8K_PORT_LOOKUP_CTRL(i),
- BIT(port));
+ ret = regmap_set_bits(priv->regmap,
+ QCA8K_PORT_LOOKUP_CTRL(i),
+ BIT(port));
if (ret)
return ret;
if (i != port)
@@ -1774,8 +1845,8 @@ qca8k_port_bridge_join(struct dsa_switch *ds, int port, struct net_device *br)
return ret;
}
-static void
-qca8k_port_bridge_leave(struct dsa_switch *ds, int port, struct net_device *br)
+static void qca8k_port_bridge_leave(struct dsa_switch *ds, int port,
+ struct dsa_bridge bridge)
{
struct qca8k_priv *priv = (struct qca8k_priv *)ds->priv;
int cpu_port, i;
@@ -1785,14 +1856,14 @@ qca8k_port_bridge_leave(struct dsa_switch *ds, int port, struct net_device *br)
for (i = 0; i < QCA8K_NUM_PORTS; i++) {
if (dsa_is_cpu_port(ds, i))
continue;
- if (dsa_to_port(ds, i)->bridge_dev != br)
+ if (!dsa_port_offloads_bridge(dsa_to_port(ds, i), &bridge))
continue;
/* Remove this port to the portvlan mask of the other ports
* in the bridge
*/
- qca8k_reg_clear(priv,
- QCA8K_PORT_LOOKUP_CTRL(i),
- BIT(port));
+ regmap_clear_bits(priv->regmap,
+ QCA8K_PORT_LOOKUP_CTRL(i),
+ BIT(port));
}
/* Set the cpu port to be the only one in the portvlan mask of
@@ -1802,6 +1873,36 @@ qca8k_port_bridge_leave(struct dsa_switch *ds, int port, struct net_device *br)
QCA8K_PORT_LOOKUP_MEMBER, BIT(cpu_port));
}
+static void
+qca8k_port_fast_age(struct dsa_switch *ds, int port)
+{
+ struct qca8k_priv *priv = ds->priv;
+
+ mutex_lock(&priv->reg_mutex);
+ qca8k_fdb_access(priv, QCA8K_FDB_FLUSH_PORT, port);
+ mutex_unlock(&priv->reg_mutex);
+}
+
+static int
+qca8k_set_ageing_time(struct dsa_switch *ds, unsigned int msecs)
+{
+ struct qca8k_priv *priv = ds->priv;
+ unsigned int secs = msecs / 1000;
+ u32 val;
+
+ /* AGE_TIME reg is set in 7s step */
+ val = secs / 7;
+
+ /* Handle case with 0 as val to NOT disable
+ * learning
+ */
+ if (!val)
+ val = 1;
+
+ return regmap_update_bits(priv->regmap, QCA8K_REG_ATU_CTRL, QCA8K_ATU_AGE_TIME_MASK,
+ QCA8K_ATU_AGE_TIME(val));
+}
+
static int
qca8k_port_enable(struct dsa_switch *ds, int port,
struct phy_device *phy)
@@ -1908,6 +2009,121 @@ qca8k_port_fdb_dump(struct dsa_switch *ds, int port,
}
static int
+qca8k_port_mdb_add(struct dsa_switch *ds, int port,
+ const struct switchdev_obj_port_mdb *mdb)
+{
+ struct qca8k_priv *priv = ds->priv;
+ const u8 *addr = mdb->addr;
+ u16 vid = mdb->vid;
+
+ return qca8k_fdb_search_and_insert(priv, BIT(port), addr, vid);
+}
+
+static int
+qca8k_port_mdb_del(struct dsa_switch *ds, int port,
+ const struct switchdev_obj_port_mdb *mdb)
+{
+ struct qca8k_priv *priv = ds->priv;
+ const u8 *addr = mdb->addr;
+ u16 vid = mdb->vid;
+
+ return qca8k_fdb_search_and_del(priv, BIT(port), addr, vid);
+}
+
+static int
+qca8k_port_mirror_add(struct dsa_switch *ds, int port,
+ struct dsa_mall_mirror_tc_entry *mirror,
+ bool ingress)
+{
+ struct qca8k_priv *priv = ds->priv;
+ int monitor_port, ret;
+ u32 reg, val;
+
+ /* Check for existent entry */
+ if ((ingress ? priv->mirror_rx : priv->mirror_tx) & BIT(port))
+ return -EEXIST;
+
+ ret = regmap_read(priv->regmap, QCA8K_REG_GLOBAL_FW_CTRL0, &val);
+ if (ret)
+ return ret;
+
+ /* QCA83xx can have only one port set to mirror mode.
+ * Check that the correct port is requested and return error otherwise.
+ * When no mirror port is set, the values is set to 0xF
+ */
+ monitor_port = FIELD_GET(QCA8K_GLOBAL_FW_CTRL0_MIRROR_PORT_NUM, val);
+ if (monitor_port != 0xF && monitor_port != mirror->to_local_port)
+ return -EEXIST;
+
+ /* Set the monitor port */
+ val = FIELD_PREP(QCA8K_GLOBAL_FW_CTRL0_MIRROR_PORT_NUM,
+ mirror->to_local_port);
+ ret = regmap_update_bits(priv->regmap, QCA8K_REG_GLOBAL_FW_CTRL0,
+ QCA8K_GLOBAL_FW_CTRL0_MIRROR_PORT_NUM, val);
+ if (ret)
+ return ret;
+
+ if (ingress) {
+ reg = QCA8K_PORT_LOOKUP_CTRL(port);
+ val = QCA8K_PORT_LOOKUP_ING_MIRROR_EN;
+ } else {
+ reg = QCA8K_REG_PORT_HOL_CTRL1(port);
+ val = QCA8K_PORT_HOL_CTRL1_EG_MIRROR_EN;
+ }
+
+ ret = regmap_update_bits(priv->regmap, reg, val, val);
+ if (ret)
+ return ret;
+
+ /* Track mirror port for tx and rx to decide when the
+ * mirror port has to be disabled.
+ */
+ if (ingress)
+ priv->mirror_rx |= BIT(port);
+ else
+ priv->mirror_tx |= BIT(port);
+
+ return 0;
+}
+
+static void
+qca8k_port_mirror_del(struct dsa_switch *ds, int port,
+ struct dsa_mall_mirror_tc_entry *mirror)
+{
+ struct qca8k_priv *priv = ds->priv;
+ u32 reg, val;
+ int ret;
+
+ if (mirror->ingress) {
+ reg = QCA8K_PORT_LOOKUP_CTRL(port);
+ val = QCA8K_PORT_LOOKUP_ING_MIRROR_EN;
+ } else {
+ reg = QCA8K_REG_PORT_HOL_CTRL1(port);
+ val = QCA8K_PORT_HOL_CTRL1_EG_MIRROR_EN;
+ }
+
+ ret = regmap_clear_bits(priv->regmap, reg, val);
+ if (ret)
+ goto err;
+
+ if (mirror->ingress)
+ priv->mirror_rx &= ~BIT(port);
+ else
+ priv->mirror_tx &= ~BIT(port);
+
+ /* No port set to send packet to mirror port. Disable mirror port */
+ if (!priv->mirror_rx && !priv->mirror_tx) {
+ val = FIELD_PREP(QCA8K_GLOBAL_FW_CTRL0_MIRROR_PORT_NUM, 0xF);
+ ret = regmap_update_bits(priv->regmap, QCA8K_REG_GLOBAL_FW_CTRL0,
+ QCA8K_GLOBAL_FW_CTRL0_MIRROR_PORT_NUM, val);
+ if (ret)
+ goto err;
+ }
+err:
+ dev_err(priv->dev, "Failed to del mirror port from %d", port);
+}
+
+static int
qca8k_port_vlan_filtering(struct dsa_switch *ds, int port, bool vlan_filtering,
struct netlink_ext_ack *extack)
{
@@ -1916,11 +2132,11 @@ qca8k_port_vlan_filtering(struct dsa_switch *ds, int port, bool vlan_filtering,
if (vlan_filtering) {
ret = qca8k_rmw(priv, QCA8K_PORT_LOOKUP_CTRL(port),
- QCA8K_PORT_LOOKUP_VLAN_MODE,
+ QCA8K_PORT_LOOKUP_VLAN_MODE_MASK,
QCA8K_PORT_LOOKUP_VLAN_MODE_SECURE);
} else {
ret = qca8k_rmw(priv, QCA8K_PORT_LOOKUP_CTRL(port),
- QCA8K_PORT_LOOKUP_VLAN_MODE,
+ QCA8K_PORT_LOOKUP_VLAN_MODE_MASK,
QCA8K_PORT_LOOKUP_VLAN_MODE_NONE);
}
@@ -1944,10 +2160,9 @@ qca8k_port_vlan_add(struct dsa_switch *ds, int port,
}
if (pvid) {
- int shift = 16 * (port % 2);
-
ret = qca8k_rmw(priv, QCA8K_EGRESS_VLAN(port),
- 0xfff << shift, vlan->vid << shift);
+ QCA8K_EGREES_VLAN_PORT_MASK(port),
+ QCA8K_EGREES_VLAN_PORT(port, vlan->vid));
if (ret)
return ret;
@@ -1996,12 +2211,185 @@ qca8k_get_tag_protocol(struct dsa_switch *ds, int port,
return DSA_TAG_PROTO_QCA;
}
+static bool
+qca8k_lag_can_offload(struct dsa_switch *ds,
+ struct net_device *lag,
+ struct netdev_lag_upper_info *info)
+{
+ struct dsa_port *dp;
+ int id, members = 0;
+
+ id = dsa_lag_id(ds->dst, lag);
+ if (id < 0 || id >= ds->num_lag_ids)
+ return false;
+
+ dsa_lag_foreach_port(dp, ds->dst, lag)
+ /* Includes the port joining the LAG */
+ members++;
+
+ if (members > QCA8K_NUM_PORTS_FOR_LAG)
+ return false;
+
+ if (info->tx_type != NETDEV_LAG_TX_TYPE_HASH)
+ return false;
+
+ if (info->hash_type != NETDEV_LAG_HASH_L2 &&
+ info->hash_type != NETDEV_LAG_HASH_L23)
+ return false;
+
+ return true;
+}
+
+static int
+qca8k_lag_setup_hash(struct dsa_switch *ds,
+ struct net_device *lag,
+ struct netdev_lag_upper_info *info)
+{
+ struct qca8k_priv *priv = ds->priv;
+ bool unique_lag = true;
+ u32 hash = 0;
+ int i, id;
+
+ id = dsa_lag_id(ds->dst, lag);
+
+ switch (info->hash_type) {
+ case NETDEV_LAG_HASH_L23:
+ hash |= QCA8K_TRUNK_HASH_SIP_EN;
+ hash |= QCA8K_TRUNK_HASH_DIP_EN;
+ fallthrough;
+ case NETDEV_LAG_HASH_L2:
+ hash |= QCA8K_TRUNK_HASH_SA_EN;
+ hash |= QCA8K_TRUNK_HASH_DA_EN;
+ break;
+ default: /* We should NEVER reach this */
+ return -EOPNOTSUPP;
+ }
+
+ /* Check if we are the unique configured LAG */
+ dsa_lags_foreach_id(i, ds->dst)
+ if (i != id && dsa_lag_dev(ds->dst, i)) {
+ unique_lag = false;
+ break;
+ }
+
+ /* Hash Mode is global. Make sure the same Hash Mode
+ * is set to all the 4 possible lag.
+ * If we are the unique LAG we can set whatever hash
+ * mode we want.
+ * To change hash mode it's needed to remove all LAG
+ * and change the mode with the latest.
+ */
+ if (unique_lag) {
+ priv->lag_hash_mode = hash;
+ } else if (priv->lag_hash_mode != hash) {
+ netdev_err(lag, "Error: Mismatched Hash Mode across different lag is not supported\n");
+ return -EOPNOTSUPP;
+ }
+
+ return regmap_update_bits(priv->regmap, QCA8K_TRUNK_HASH_EN_CTRL,
+ QCA8K_TRUNK_HASH_MASK, hash);
+}
+
+static int
+qca8k_lag_refresh_portmap(struct dsa_switch *ds, int port,
+ struct net_device *lag, bool delete)
+{
+ struct qca8k_priv *priv = ds->priv;
+ int ret, id, i;
+ u32 val;
+
+ id = dsa_lag_id(ds->dst, lag);
+
+ /* Read current port member */
+ ret = regmap_read(priv->regmap, QCA8K_REG_GOL_TRUNK_CTRL0, &val);
+ if (ret)
+ return ret;
+
+ /* Shift val to the correct trunk */
+ val >>= QCA8K_REG_GOL_TRUNK_SHIFT(id);
+ val &= QCA8K_REG_GOL_TRUNK_MEMBER_MASK;
+ if (delete)
+ val &= ~BIT(port);
+ else
+ val |= BIT(port);
+
+ /* Update port member. With empty portmap disable trunk */
+ ret = regmap_update_bits(priv->regmap, QCA8K_REG_GOL_TRUNK_CTRL0,
+ QCA8K_REG_GOL_TRUNK_MEMBER(id) |
+ QCA8K_REG_GOL_TRUNK_EN(id),
+ !val << QCA8K_REG_GOL_TRUNK_SHIFT(id) |
+ val << QCA8K_REG_GOL_TRUNK_SHIFT(id));
+
+ /* Search empty member if adding or port on deleting */
+ for (i = 0; i < QCA8K_NUM_PORTS_FOR_LAG; i++) {
+ ret = regmap_read(priv->regmap, QCA8K_REG_GOL_TRUNK_CTRL(id), &val);
+ if (ret)
+ return ret;
+
+ val >>= QCA8K_REG_GOL_TRUNK_ID_MEM_ID_SHIFT(id, i);
+ val &= QCA8K_REG_GOL_TRUNK_ID_MEM_ID_MASK;
+
+ if (delete) {
+ /* If port flagged to be disabled assume this member is
+ * empty
+ */
+ if (val != QCA8K_REG_GOL_TRUNK_ID_MEM_ID_EN_MASK)
+ continue;
+
+ val &= QCA8K_REG_GOL_TRUNK_ID_MEM_ID_PORT_MASK;
+ if (val != port)
+ continue;
+ } else {
+ /* If port flagged to be enabled assume this member is
+ * already set
+ */
+ if (val == QCA8K_REG_GOL_TRUNK_ID_MEM_ID_EN_MASK)
+ continue;
+ }
+
+ /* We have found the member to add/remove */
+ break;
+ }
+
+ /* Set port in the correct port mask or disable port if in delete mode */
+ return regmap_update_bits(priv->regmap, QCA8K_REG_GOL_TRUNK_CTRL(id),
+ QCA8K_REG_GOL_TRUNK_ID_MEM_ID_EN(id, i) |
+ QCA8K_REG_GOL_TRUNK_ID_MEM_ID_PORT(id, i),
+ !delete << QCA8K_REG_GOL_TRUNK_ID_MEM_ID_SHIFT(id, i) |
+ port << QCA8K_REG_GOL_TRUNK_ID_MEM_ID_SHIFT(id, i));
+}
+
+static int
+qca8k_port_lag_join(struct dsa_switch *ds, int port,
+ struct net_device *lag,
+ struct netdev_lag_upper_info *info)
+{
+ int ret;
+
+ if (!qca8k_lag_can_offload(ds, lag, info))
+ return -EOPNOTSUPP;
+
+ ret = qca8k_lag_setup_hash(ds, lag, info);
+ if (ret)
+ return ret;
+
+ return qca8k_lag_refresh_portmap(ds, port, lag, false);
+}
+
+static int
+qca8k_port_lag_leave(struct dsa_switch *ds, int port,
+ struct net_device *lag)
+{
+ return qca8k_lag_refresh_portmap(ds, port, lag, true);
+}
+
static const struct dsa_switch_ops qca8k_switch_ops = {
.get_tag_protocol = qca8k_get_tag_protocol,
.setup = qca8k_setup,
.get_strings = qca8k_get_strings,
.get_ethtool_stats = qca8k_get_ethtool_stats,
.get_sset_count = qca8k_get_sset_count,
+ .set_ageing_time = qca8k_set_ageing_time,
.get_mac_eee = qca8k_get_mac_eee,
.set_mac_eee = qca8k_set_mac_eee,
.port_enable = qca8k_port_enable,
@@ -2011,9 +2399,14 @@ static const struct dsa_switch_ops qca8k_switch_ops = {
.port_stp_state_set = qca8k_port_stp_state_set,
.port_bridge_join = qca8k_port_bridge_join,
.port_bridge_leave = qca8k_port_bridge_leave,
+ .port_fast_age = qca8k_port_fast_age,
.port_fdb_add = qca8k_port_fdb_add,
.port_fdb_del = qca8k_port_fdb_del,
.port_fdb_dump = qca8k_port_fdb_dump,
+ .port_mdb_add = qca8k_port_mdb_add,
+ .port_mdb_del = qca8k_port_mdb_del,
+ .port_mirror_add = qca8k_port_mirror_add,
+ .port_mirror_del = qca8k_port_mirror_del,
.port_vlan_filtering = qca8k_port_vlan_filtering,
.port_vlan_add = qca8k_port_vlan_add,
.port_vlan_del = qca8k_port_vlan_del,
@@ -2023,6 +2416,8 @@ static const struct dsa_switch_ops qca8k_switch_ops = {
.phylink_mac_link_down = qca8k_phylink_mac_link_down,
.phylink_mac_link_up = qca8k_phylink_mac_link_up,
.get_phy_flags = qca8k_get_phy_flags,
+ .port_lag_join = qca8k_port_lag_join,
+ .port_lag_leave = qca8k_port_lag_leave,
};
static int qca8k_read_switch_id(struct qca8k_priv *priv)
@@ -2041,7 +2436,7 @@ static int qca8k_read_switch_id(struct qca8k_priv *priv)
if (ret < 0)
return -ENODEV;
- id = QCA8K_MASK_CTRL_DEVICE_ID(val & QCA8K_MASK_CTRL_DEVICE_ID_MASK);
+ id = QCA8K_MASK_CTRL_DEVICE_ID(val);
if (id != data->id) {
dev_err(priv->dev, "Switch id detected %x but expected %x", id, data->id);
return -ENODEV;
@@ -2050,7 +2445,7 @@ static int qca8k_read_switch_id(struct qca8k_priv *priv)
priv->switch_id = id;
/* Save revision to communicate to the internal PHY driver */
- priv->switch_revision = (val & QCA8K_MASK_CTRL_REV_ID_MASK);
+ priv->switch_revision = QCA8K_MASK_CTRL_REV_ID(val);
return 0;
}
@@ -2085,6 +2480,14 @@ qca8k_sw_probe(struct mdio_device *mdiodev)
gpiod_set_value_cansleep(priv->reset_gpio, 0);
}
+ /* Start by setting up the register mapping */
+ priv->regmap = devm_regmap_init(&mdiodev->dev, NULL, priv,
+ &qca8k_regmap_config);
+ if (IS_ERR(priv->regmap)) {
+ dev_err(priv->dev, "regmap initialization failed");
+ return PTR_ERR(priv->regmap);
+ }
+
/* Check the detected switch id */
ret = qca8k_read_switch_id(priv);
if (ret)
@@ -2173,14 +2576,17 @@ static SIMPLE_DEV_PM_OPS(qca8k_pm_ops,
static const struct qca8k_match_data qca8327 = {
.id = QCA8K_ID_QCA8327,
.reduced_package = true,
+ .mib_count = QCA8K_QCA832X_MIB_COUNT,
};
static const struct qca8k_match_data qca8328 = {
.id = QCA8K_ID_QCA8327,
+ .mib_count = QCA8K_QCA832X_MIB_COUNT,
};
static const struct qca8k_match_data qca833x = {
.id = QCA8K_ID_QCA8337,
+ .mib_count = QCA8K_QCA833X_MIB_COUNT,
};
static const struct of_device_id qca8k_of_match[] = {
diff --git a/drivers/net/dsa/qca8k.h b/drivers/net/dsa/qca8k.h
index 128b8cf..ab4a417 100644
--- a/drivers/net/dsa/qca8k.h
+++ b/drivers/net/dsa/qca8k.h
@@ -15,12 +15,17 @@
#define QCA8K_NUM_PORTS 7
#define QCA8K_NUM_CPU_PORTS 2
#define QCA8K_MAX_MTU 9000
+#define QCA8K_NUM_LAGS 4
+#define QCA8K_NUM_PORTS_FOR_LAG 4
#define PHY_ID_QCA8327 0x004dd034
#define QCA8K_ID_QCA8327 0x12
#define PHY_ID_QCA8337 0x004dd036
#define QCA8K_ID_QCA8337 0x13
+#define QCA8K_QCA832X_MIB_COUNT 39
+#define QCA8K_QCA833X_MIB_COUNT 41
+
#define QCA8K_BUSY_WAIT_TIMEOUT 2000
#define QCA8K_NUM_FDB_RECORDS 2048
@@ -30,9 +35,9 @@
/* Global control registers */
#define QCA8K_REG_MASK_CTRL 0x000
#define QCA8K_MASK_CTRL_REV_ID_MASK GENMASK(7, 0)
-#define QCA8K_MASK_CTRL_REV_ID(x) ((x) >> 0)
+#define QCA8K_MASK_CTRL_REV_ID(x) FIELD_GET(QCA8K_MASK_CTRL_REV_ID_MASK, x)
#define QCA8K_MASK_CTRL_DEVICE_ID_MASK GENMASK(15, 8)
-#define QCA8K_MASK_CTRL_DEVICE_ID(x) ((x) >> 8)
+#define QCA8K_MASK_CTRL_DEVICE_ID(x) FIELD_GET(QCA8K_MASK_CTRL_DEVICE_ID_MASK, x)
#define QCA8K_REG_PORT0_PAD_CTRL 0x004
#define QCA8K_PORT0_PAD_MAC06_EXCHANGE_EN BIT(31)
#define QCA8K_PORT0_PAD_SGMII_RXCLK_FALLING_EDGE BIT(19)
@@ -41,12 +46,11 @@
#define QCA8K_REG_PORT6_PAD_CTRL 0x00c
#define QCA8K_PORT_PAD_RGMII_EN BIT(26)
#define QCA8K_PORT_PAD_RGMII_TX_DELAY_MASK GENMASK(23, 22)
-#define QCA8K_PORT_PAD_RGMII_TX_DELAY(x) ((x) << 22)
+#define QCA8K_PORT_PAD_RGMII_TX_DELAY(x) FIELD_PREP(QCA8K_PORT_PAD_RGMII_TX_DELAY_MASK, x)
#define QCA8K_PORT_PAD_RGMII_RX_DELAY_MASK GENMASK(21, 20)
-#define QCA8K_PORT_PAD_RGMII_RX_DELAY(x) ((x) << 20)
+#define QCA8K_PORT_PAD_RGMII_RX_DELAY(x) FIELD_PREP(QCA8K_PORT_PAD_RGMII_RX_DELAY_MASK, x)
#define QCA8K_PORT_PAD_RGMII_TX_DELAY_EN BIT(25)
#define QCA8K_PORT_PAD_RGMII_RX_DELAY_EN BIT(24)
-#define QCA8K_MAX_DELAY 3
#define QCA8K_PORT_PAD_SGMII_EN BIT(7)
#define QCA8K_REG_PWS 0x010
#define QCA8K_PWS_POWER_ON_SEL BIT(31)
@@ -68,10 +72,12 @@
#define QCA8K_MDIO_MASTER_READ BIT(27)
#define QCA8K_MDIO_MASTER_WRITE 0
#define QCA8K_MDIO_MASTER_SUP_PRE BIT(26)
-#define QCA8K_MDIO_MASTER_PHY_ADDR(x) ((x) << 21)
-#define QCA8K_MDIO_MASTER_REG_ADDR(x) ((x) << 16)
-#define QCA8K_MDIO_MASTER_DATA(x) (x)
+#define QCA8K_MDIO_MASTER_PHY_ADDR_MASK GENMASK(25, 21)
+#define QCA8K_MDIO_MASTER_PHY_ADDR(x) FIELD_PREP(QCA8K_MDIO_MASTER_PHY_ADDR_MASK, x)
+#define QCA8K_MDIO_MASTER_REG_ADDR_MASK GENMASK(20, 16)
+#define QCA8K_MDIO_MASTER_REG_ADDR(x) FIELD_PREP(QCA8K_MDIO_MASTER_REG_ADDR_MASK, x)
#define QCA8K_MDIO_MASTER_DATA_MASK GENMASK(15, 0)
+#define QCA8K_MDIO_MASTER_DATA(x) FIELD_PREP(QCA8K_MDIO_MASTER_DATA_MASK, x)
#define QCA8K_MDIO_MASTER_MAX_PORTS 5
#define QCA8K_MDIO_MASTER_MAX_REG 32
#define QCA8K_GOL_MAC_ADDR0 0x60
@@ -93,9 +99,7 @@
#define QCA8K_PORT_STATUS_FLOW_AUTO BIT(12)
#define QCA8K_REG_PORT_HDR_CTRL(_i) (0x9c + (_i * 4))
#define QCA8K_PORT_HDR_CTRL_RX_MASK GENMASK(3, 2)
-#define QCA8K_PORT_HDR_CTRL_RX_S 2
#define QCA8K_PORT_HDR_CTRL_TX_MASK GENMASK(1, 0)
-#define QCA8K_PORT_HDR_CTRL_TX_S 0
#define QCA8K_PORT_HDR_CTRL_ALL 2
#define QCA8K_PORT_HDR_CTRL_MGMT 1
#define QCA8K_PORT_HDR_CTRL_NONE 0
@@ -105,10 +109,11 @@
#define QCA8K_SGMII_EN_TX BIT(3)
#define QCA8K_SGMII_EN_SD BIT(4)
#define QCA8K_SGMII_CLK125M_DELAY BIT(7)
-#define QCA8K_SGMII_MODE_CTRL_MASK (BIT(22) | BIT(23))
-#define QCA8K_SGMII_MODE_CTRL_BASEX (0 << 22)
-#define QCA8K_SGMII_MODE_CTRL_PHY (1 << 22)
-#define QCA8K_SGMII_MODE_CTRL_MAC (2 << 22)
+#define QCA8K_SGMII_MODE_CTRL_MASK GENMASK(23, 22)
+#define QCA8K_SGMII_MODE_CTRL(x) FIELD_PREP(QCA8K_SGMII_MODE_CTRL_MASK, x)
+#define QCA8K_SGMII_MODE_CTRL_BASEX QCA8K_SGMII_MODE_CTRL(0x0)
+#define QCA8K_SGMII_MODE_CTRL_PHY QCA8K_SGMII_MODE_CTRL(0x1)
+#define QCA8K_SGMII_MODE_CTRL_MAC QCA8K_SGMII_MODE_CTRL(0x2)
/* MAC_PWR_SEL registers */
#define QCA8K_REG_MAC_PWR_SEL 0x0e4
@@ -119,102 +124,152 @@
#define QCA8K_REG_EEE_CTRL 0x100
#define QCA8K_REG_EEE_CTRL_LPI_EN(_i) ((_i + 1) * 2)
+/* TRUNK_HASH_EN registers */
+#define QCA8K_TRUNK_HASH_EN_CTRL 0x270
+#define QCA8K_TRUNK_HASH_SIP_EN BIT(3)
+#define QCA8K_TRUNK_HASH_DIP_EN BIT(2)
+#define QCA8K_TRUNK_HASH_SA_EN BIT(1)
+#define QCA8K_TRUNK_HASH_DA_EN BIT(0)
+#define QCA8K_TRUNK_HASH_MASK GENMASK(3, 0)
+
/* ACL registers */
#define QCA8K_REG_PORT_VLAN_CTRL0(_i) (0x420 + (_i * 8))
-#define QCA8K_PORT_VLAN_CVID(x) (x << 16)
-#define QCA8K_PORT_VLAN_SVID(x) x
+#define QCA8K_PORT_VLAN_CVID_MASK GENMASK(27, 16)
+#define QCA8K_PORT_VLAN_CVID(x) FIELD_PREP(QCA8K_PORT_VLAN_CVID_MASK, x)
+#define QCA8K_PORT_VLAN_SVID_MASK GENMASK(11, 0)
+#define QCA8K_PORT_VLAN_SVID(x) FIELD_PREP(QCA8K_PORT_VLAN_SVID_MASK, x)
#define QCA8K_REG_PORT_VLAN_CTRL1(_i) (0x424 + (_i * 8))
#define QCA8K_REG_IPV4_PRI_BASE_ADDR 0x470
#define QCA8K_REG_IPV4_PRI_ADDR_MASK 0x474
/* Lookup registers */
#define QCA8K_REG_ATU_DATA0 0x600
-#define QCA8K_ATU_ADDR2_S 24
-#define QCA8K_ATU_ADDR3_S 16
-#define QCA8K_ATU_ADDR4_S 8
+#define QCA8K_ATU_ADDR2_MASK GENMASK(31, 24)
+#define QCA8K_ATU_ADDR3_MASK GENMASK(23, 16)
+#define QCA8K_ATU_ADDR4_MASK GENMASK(15, 8)
+#define QCA8K_ATU_ADDR5_MASK GENMASK(7, 0)
#define QCA8K_REG_ATU_DATA1 0x604
-#define QCA8K_ATU_PORT_M 0x7f
-#define QCA8K_ATU_PORT_S 16
-#define QCA8K_ATU_ADDR0_S 8
+#define QCA8K_ATU_PORT_MASK GENMASK(22, 16)
+#define QCA8K_ATU_ADDR0_MASK GENMASK(15, 8)
+#define QCA8K_ATU_ADDR1_MASK GENMASK(7, 0)
#define QCA8K_REG_ATU_DATA2 0x608
-#define QCA8K_ATU_VID_M 0xfff
-#define QCA8K_ATU_VID_S 8
-#define QCA8K_ATU_STATUS_M 0xf
+#define QCA8K_ATU_VID_MASK GENMASK(19, 8)
+#define QCA8K_ATU_STATUS_MASK GENMASK(3, 0)
#define QCA8K_ATU_STATUS_STATIC 0xf
#define QCA8K_REG_ATU_FUNC 0x60c
#define QCA8K_ATU_FUNC_BUSY BIT(31)
#define QCA8K_ATU_FUNC_PORT_EN BIT(14)
#define QCA8K_ATU_FUNC_MULTI_EN BIT(13)
#define QCA8K_ATU_FUNC_FULL BIT(12)
-#define QCA8K_ATU_FUNC_PORT_M 0xf
-#define QCA8K_ATU_FUNC_PORT_S 8
+#define QCA8K_ATU_FUNC_PORT_MASK GENMASK(11, 8)
#define QCA8K_REG_VTU_FUNC0 0x610
#define QCA8K_VTU_FUNC0_VALID BIT(20)
#define QCA8K_VTU_FUNC0_IVL_EN BIT(19)
-#define QCA8K_VTU_FUNC0_EG_MODE_S(_i) (4 + (_i) * 2)
-#define QCA8K_VTU_FUNC0_EG_MODE_MASK 3
-#define QCA8K_VTU_FUNC0_EG_MODE_UNMOD 0
-#define QCA8K_VTU_FUNC0_EG_MODE_UNTAG 1
-#define QCA8K_VTU_FUNC0_EG_MODE_TAG 2
-#define QCA8K_VTU_FUNC0_EG_MODE_NOT 3
+/* QCA8K_VTU_FUNC0_EG_MODE_MASK GENMASK(17, 4)
+ * It does contain VLAN_MODE for each port [5:4] for port0,
+ * [7:6] for port1 ... [17:16] for port6. Use virtual port
+ * define to handle this.
+ */
+#define QCA8K_VTU_FUNC0_EG_MODE_PORT_SHIFT(_i) (4 + (_i) * 2)
+#define QCA8K_VTU_FUNC0_EG_MODE_MASK GENMASK(1, 0)
+#define QCA8K_VTU_FUNC0_EG_MODE_PORT_MASK(_i) (GENMASK(1, 0) << QCA8K_VTU_FUNC0_EG_MODE_PORT_SHIFT(_i))
+#define QCA8K_VTU_FUNC0_EG_MODE_UNMOD FIELD_PREP(QCA8K_VTU_FUNC0_EG_MODE_MASK, 0x0)
+#define QCA8K_VTU_FUNC0_EG_MODE_PORT_UNMOD(_i) (QCA8K_VTU_FUNC0_EG_MODE_UNMOD << QCA8K_VTU_FUNC0_EG_MODE_PORT_SHIFT(_i))
+#define QCA8K_VTU_FUNC0_EG_MODE_UNTAG FIELD_PREP(QCA8K_VTU_FUNC0_EG_MODE_MASK, 0x1)
+#define QCA8K_VTU_FUNC0_EG_MODE_PORT_UNTAG(_i) (QCA8K_VTU_FUNC0_EG_MODE_UNTAG << QCA8K_VTU_FUNC0_EG_MODE_PORT_SHIFT(_i))
+#define QCA8K_VTU_FUNC0_EG_MODE_TAG FIELD_PREP(QCA8K_VTU_FUNC0_EG_MODE_MASK, 0x2)
+#define QCA8K_VTU_FUNC0_EG_MODE_PORT_TAG(_i) (QCA8K_VTU_FUNC0_EG_MODE_TAG << QCA8K_VTU_FUNC0_EG_MODE_PORT_SHIFT(_i))
+#define QCA8K_VTU_FUNC0_EG_MODE_NOT FIELD_PREP(QCA8K_VTU_FUNC0_EG_MODE_MASK, 0x3)
+#define QCA8K_VTU_FUNC0_EG_MODE_PORT_NOT(_i) (QCA8K_VTU_FUNC0_EG_MODE_NOT << QCA8K_VTU_FUNC0_EG_MODE_PORT_SHIFT(_i))
#define QCA8K_REG_VTU_FUNC1 0x614
#define QCA8K_VTU_FUNC1_BUSY BIT(31)
-#define QCA8K_VTU_FUNC1_VID_S 16
+#define QCA8K_VTU_FUNC1_VID_MASK GENMASK(27, 16)
#define QCA8K_VTU_FUNC1_FULL BIT(4)
+#define QCA8K_REG_ATU_CTRL 0x618
+#define QCA8K_ATU_AGE_TIME_MASK GENMASK(15, 0)
+#define QCA8K_ATU_AGE_TIME(x) FIELD_PREP(QCA8K_ATU_AGE_TIME_MASK, (x))
#define QCA8K_REG_GLOBAL_FW_CTRL0 0x620
#define QCA8K_GLOBAL_FW_CTRL0_CPU_PORT_EN BIT(10)
+#define QCA8K_GLOBAL_FW_CTRL0_MIRROR_PORT_NUM GENMASK(7, 4)
#define QCA8K_REG_GLOBAL_FW_CTRL1 0x624
-#define QCA8K_GLOBAL_FW_CTRL1_IGMP_DP_S 24
-#define QCA8K_GLOBAL_FW_CTRL1_BC_DP_S 16
-#define QCA8K_GLOBAL_FW_CTRL1_MC_DP_S 8
-#define QCA8K_GLOBAL_FW_CTRL1_UC_DP_S 0
+#define QCA8K_GLOBAL_FW_CTRL1_IGMP_DP_MASK GENMASK(30, 24)
+#define QCA8K_GLOBAL_FW_CTRL1_BC_DP_MASK GENMASK(22, 16)
+#define QCA8K_GLOBAL_FW_CTRL1_MC_DP_MASK GENMASK(14, 8)
+#define QCA8K_GLOBAL_FW_CTRL1_UC_DP_MASK GENMASK(6, 0)
#define QCA8K_PORT_LOOKUP_CTRL(_i) (0x660 + (_i) * 0xc)
#define QCA8K_PORT_LOOKUP_MEMBER GENMASK(6, 0)
-#define QCA8K_PORT_LOOKUP_VLAN_MODE GENMASK(9, 8)
-#define QCA8K_PORT_LOOKUP_VLAN_MODE_NONE (0 << 8)
-#define QCA8K_PORT_LOOKUP_VLAN_MODE_FALLBACK (1 << 8)
-#define QCA8K_PORT_LOOKUP_VLAN_MODE_CHECK (2 << 8)
-#define QCA8K_PORT_LOOKUP_VLAN_MODE_SECURE (3 << 8)
+#define QCA8K_PORT_LOOKUP_VLAN_MODE_MASK GENMASK(9, 8)
+#define QCA8K_PORT_LOOKUP_VLAN_MODE(x) FIELD_PREP(QCA8K_PORT_LOOKUP_VLAN_MODE_MASK, x)
+#define QCA8K_PORT_LOOKUP_VLAN_MODE_NONE QCA8K_PORT_LOOKUP_VLAN_MODE(0x0)
+#define QCA8K_PORT_LOOKUP_VLAN_MODE_FALLBACK QCA8K_PORT_LOOKUP_VLAN_MODE(0x1)
+#define QCA8K_PORT_LOOKUP_VLAN_MODE_CHECK QCA8K_PORT_LOOKUP_VLAN_MODE(0x2)
+#define QCA8K_PORT_LOOKUP_VLAN_MODE_SECURE QCA8K_PORT_LOOKUP_VLAN_MODE(0x3)
#define QCA8K_PORT_LOOKUP_STATE_MASK GENMASK(18, 16)
-#define QCA8K_PORT_LOOKUP_STATE_DISABLED (0 << 16)
-#define QCA8K_PORT_LOOKUP_STATE_BLOCKING (1 << 16)
-#define QCA8K_PORT_LOOKUP_STATE_LISTENING (2 << 16)
-#define QCA8K_PORT_LOOKUP_STATE_LEARNING (3 << 16)
-#define QCA8K_PORT_LOOKUP_STATE_FORWARD (4 << 16)
-#define QCA8K_PORT_LOOKUP_STATE GENMASK(18, 16)
+#define QCA8K_PORT_LOOKUP_STATE(x) FIELD_PREP(QCA8K_PORT_LOOKUP_STATE_MASK, x)
+#define QCA8K_PORT_LOOKUP_STATE_DISABLED QCA8K_PORT_LOOKUP_STATE(0x0)
+#define QCA8K_PORT_LOOKUP_STATE_BLOCKING QCA8K_PORT_LOOKUP_STATE(0x1)
+#define QCA8K_PORT_LOOKUP_STATE_LISTENING QCA8K_PORT_LOOKUP_STATE(0x2)
+#define QCA8K_PORT_LOOKUP_STATE_LEARNING QCA8K_PORT_LOOKUP_STATE(0x3)
+#define QCA8K_PORT_LOOKUP_STATE_FORWARD QCA8K_PORT_LOOKUP_STATE(0x4)
#define QCA8K_PORT_LOOKUP_LEARN BIT(20)
+#define QCA8K_PORT_LOOKUP_ING_MIRROR_EN BIT(25)
+
+#define QCA8K_REG_GOL_TRUNK_CTRL0 0x700
+/* 4 max trunk first
+ * first 6 bit for member bitmap
+ * 7th bit is to enable trunk port
+ */
+#define QCA8K_REG_GOL_TRUNK_SHIFT(_i) ((_i) * 8)
+#define QCA8K_REG_GOL_TRUNK_EN_MASK BIT(7)
+#define QCA8K_REG_GOL_TRUNK_EN(_i) (QCA8K_REG_GOL_TRUNK_EN_MASK << QCA8K_REG_GOL_TRUNK_SHIFT(_i))
+#define QCA8K_REG_GOL_TRUNK_MEMBER_MASK GENMASK(6, 0)
+#define QCA8K_REG_GOL_TRUNK_MEMBER(_i) (QCA8K_REG_GOL_TRUNK_MEMBER_MASK << QCA8K_REG_GOL_TRUNK_SHIFT(_i))
+/* 0x704 for TRUNK 0-1 --- 0x708 for TRUNK 2-3 */
+#define QCA8K_REG_GOL_TRUNK_CTRL(_i) (0x704 + (((_i) / 2) * 4))
+#define QCA8K_REG_GOL_TRUNK_ID_MEM_ID_MASK GENMASK(3, 0)
+#define QCA8K_REG_GOL_TRUNK_ID_MEM_ID_EN_MASK BIT(3)
+#define QCA8K_REG_GOL_TRUNK_ID_MEM_ID_PORT_MASK GENMASK(2, 0)
+#define QCA8K_REG_GOL_TRUNK_ID_SHIFT(_i) (((_i) / 2) * 16)
+#define QCA8K_REG_GOL_MEM_ID_SHIFT(_i) ((_i) * 4)
+/* Complex shift: FIRST shift for port THEN shift for trunk */
+#define QCA8K_REG_GOL_TRUNK_ID_MEM_ID_SHIFT(_i, _j) (QCA8K_REG_GOL_MEM_ID_SHIFT(_j) + QCA8K_REG_GOL_TRUNK_ID_SHIFT(_i))
+#define QCA8K_REG_GOL_TRUNK_ID_MEM_ID_EN(_i, _j) (QCA8K_REG_GOL_TRUNK_ID_MEM_ID_EN_MASK << QCA8K_REG_GOL_TRUNK_ID_MEM_ID_SHIFT(_i, _j))
+#define QCA8K_REG_GOL_TRUNK_ID_MEM_ID_PORT(_i, _j) (QCA8K_REG_GOL_TRUNK_ID_MEM_ID_PORT_MASK << QCA8K_REG_GOL_TRUNK_ID_MEM_ID_SHIFT(_i, _j))
#define QCA8K_REG_GLOBAL_FC_THRESH 0x800
-#define QCA8K_GLOBAL_FC_GOL_XON_THRES(x) ((x) << 16)
-#define QCA8K_GLOBAL_FC_GOL_XON_THRES_S GENMASK(24, 16)
-#define QCA8K_GLOBAL_FC_GOL_XOFF_THRES(x) ((x) << 0)
-#define QCA8K_GLOBAL_FC_GOL_XOFF_THRES_S GENMASK(8, 0)
+#define QCA8K_GLOBAL_FC_GOL_XON_THRES_MASK GENMASK(24, 16)
+#define QCA8K_GLOBAL_FC_GOL_XON_THRES(x) FIELD_PREP(QCA8K_GLOBAL_FC_GOL_XON_THRES_MASK, x)
+#define QCA8K_GLOBAL_FC_GOL_XOFF_THRES_MASK GENMASK(8, 0)
+#define QCA8K_GLOBAL_FC_GOL_XOFF_THRES(x) FIELD_PREP(QCA8K_GLOBAL_FC_GOL_XOFF_THRES_MASK, x)
#define QCA8K_REG_PORT_HOL_CTRL0(_i) (0x970 + (_i) * 0x8)
-#define QCA8K_PORT_HOL_CTRL0_EG_PRI0_BUF GENMASK(3, 0)
-#define QCA8K_PORT_HOL_CTRL0_EG_PRI0(x) ((x) << 0)
-#define QCA8K_PORT_HOL_CTRL0_EG_PRI1_BUF GENMASK(7, 4)
-#define QCA8K_PORT_HOL_CTRL0_EG_PRI1(x) ((x) << 4)
-#define QCA8K_PORT_HOL_CTRL0_EG_PRI2_BUF GENMASK(11, 8)
-#define QCA8K_PORT_HOL_CTRL0_EG_PRI2(x) ((x) << 8)
-#define QCA8K_PORT_HOL_CTRL0_EG_PRI3_BUF GENMASK(15, 12)
-#define QCA8K_PORT_HOL_CTRL0_EG_PRI3(x) ((x) << 12)
-#define QCA8K_PORT_HOL_CTRL0_EG_PRI4_BUF GENMASK(19, 16)
-#define QCA8K_PORT_HOL_CTRL0_EG_PRI4(x) ((x) << 16)
-#define QCA8K_PORT_HOL_CTRL0_EG_PRI5_BUF GENMASK(23, 20)
-#define QCA8K_PORT_HOL_CTRL0_EG_PRI5(x) ((x) << 20)
-#define QCA8K_PORT_HOL_CTRL0_EG_PORT_BUF GENMASK(29, 24)
-#define QCA8K_PORT_HOL_CTRL0_EG_PORT(x) ((x) << 24)
+#define QCA8K_PORT_HOL_CTRL0_EG_PRI0_BUF_MASK GENMASK(3, 0)
+#define QCA8K_PORT_HOL_CTRL0_EG_PRI0(x) FIELD_PREP(QCA8K_PORT_HOL_CTRL0_EG_PRI0_BUF_MASK, x)
+#define QCA8K_PORT_HOL_CTRL0_EG_PRI1_BUF_MASK GENMASK(7, 4)
+#define QCA8K_PORT_HOL_CTRL0_EG_PRI1(x) FIELD_PREP(QCA8K_PORT_HOL_CTRL0_EG_PRI1_BUF_MASK, x)
+#define QCA8K_PORT_HOL_CTRL0_EG_PRI2_BUF_MASK GENMASK(11, 8)
+#define QCA8K_PORT_HOL_CTRL0_EG_PRI2(x) FIELD_PREP(QCA8K_PORT_HOL_CTRL0_EG_PRI2_BUF_MASK, x)
+#define QCA8K_PORT_HOL_CTRL0_EG_PRI3_BUF_MASK GENMASK(15, 12)
+#define QCA8K_PORT_HOL_CTRL0_EG_PRI3(x) FIELD_PREP(QCA8K_PORT_HOL_CTRL0_EG_PRI3_BUF_MASK, x)
+#define QCA8K_PORT_HOL_CTRL0_EG_PRI4_BUF_MASK GENMASK(19, 16)
+#define QCA8K_PORT_HOL_CTRL0_EG_PRI4(x) FIELD_PREP(QCA8K_PORT_HOL_CTRL0_EG_PRI4_BUF_MASK, x)
+#define QCA8K_PORT_HOL_CTRL0_EG_PRI5_BUF_MASK GENMASK(23, 20)
+#define QCA8K_PORT_HOL_CTRL0_EG_PRI5(x) FIELD_PREP(QCA8K_PORT_HOL_CTRL0_EG_PRI5_BUF_MASK, x)
+#define QCA8K_PORT_HOL_CTRL0_EG_PORT_BUF_MASK GENMASK(29, 24)
+#define QCA8K_PORT_HOL_CTRL0_EG_PORT(x) FIELD_PREP(QCA8K_PORT_HOL_CTRL0_EG_PORT_BUF_MASK, x)
#define QCA8K_REG_PORT_HOL_CTRL1(_i) (0x974 + (_i) * 0x8)
-#define QCA8K_PORT_HOL_CTRL1_ING_BUF GENMASK(3, 0)
-#define QCA8K_PORT_HOL_CTRL1_ING(x) ((x) << 0)
+#define QCA8K_PORT_HOL_CTRL1_ING_BUF_MASK GENMASK(3, 0)
+#define QCA8K_PORT_HOL_CTRL1_ING(x) FIELD_PREP(QCA8K_PORT_HOL_CTRL1_ING_BUF_MASK, x)
#define QCA8K_PORT_HOL_CTRL1_EG_PRI_BUF_EN BIT(6)
#define QCA8K_PORT_HOL_CTRL1_EG_PORT_BUF_EN BIT(7)
#define QCA8K_PORT_HOL_CTRL1_WRED_EN BIT(8)
#define QCA8K_PORT_HOL_CTRL1_EG_MIRROR_EN BIT(16)
/* Pkt edit registers */
+#define QCA8K_EGREES_VLAN_PORT_SHIFT(_i) (16 * ((_i) % 2))
+#define QCA8K_EGREES_VLAN_PORT_MASK(_i) (GENMASK(11, 0) << QCA8K_EGREES_VLAN_PORT_SHIFT(_i))
+#define QCA8K_EGREES_VLAN_PORT(_i, x) ((x) << QCA8K_EGREES_VLAN_PORT_SHIFT(_i))
#define QCA8K_EGRESS_VLAN(x) (0x0c70 + (4 * (x / 2)))
/* L3 registers */
@@ -244,6 +299,7 @@ enum qca8k_fdb_cmd {
QCA8K_FDB_FLUSH = 1,
QCA8K_FDB_LOAD = 2,
QCA8K_FDB_PURGE = 3,
+ QCA8K_FDB_FLUSH_PORT = 5,
QCA8K_FDB_NEXT = 6,
QCA8K_FDB_SEARCH = 7,
};
@@ -264,6 +320,7 @@ struct ar8xxx_port_status {
struct qca8k_match_data {
u8 id;
bool reduced_package;
+ u8 mib_count;
};
enum {
@@ -282,6 +339,9 @@ struct qca8k_ports_config {
struct qca8k_priv {
u8 switch_id;
u8 switch_revision;
+ u8 mirror_rx;
+ u8 mirror_tx;
+ u8 lag_hash_mode;
bool legacy_phy_port_mapping;
struct qca8k_ports_config ports_config;
struct regmap *regmap;
diff --git a/drivers/net/dsa/realtek-smi-core.c b/drivers/net/dsa/realtek-smi-core.c
index c66ebd0..aae46ad 100644
--- a/drivers/net/dsa/realtek-smi-core.c
+++ b/drivers/net/dsa/realtek-smi-core.c
@@ -456,7 +456,7 @@ static int realtek_smi_probe(struct platform_device *pdev)
smi->ds->ops = var->ds_ops;
ret = dsa_register_switch(smi->ds);
if (ret) {
- dev_err(dev, "unable to register switch ret = %d\n", ret);
+ dev_err_probe(dev, ret, "unable to register switch\n");
return ret;
}
return 0;
diff --git a/drivers/net/dsa/rtl8365mb.c b/drivers/net/dsa/rtl8365mb.c
index 078ca4c..3b72954 100644
--- a/drivers/net/dsa/rtl8365mb.c
+++ b/drivers/net/dsa/rtl8365mb.c
@@ -277,7 +277,7 @@
(RTL8365MB_PORT_ISOLATION_REG_BASE + (_physport))
#define RTL8365MB_PORT_ISOLATION_MASK 0x07FF
-/* MSTP port state registers - indexed by tree instancrSTI (tree ine */
+/* MSTP port state registers - indexed by tree instance */
#define RTL8365MB_MSTI_CTRL_BASE 0x0A00
#define RTL8365MB_MSTI_CTRL_REG(_msti, _physport) \
(RTL8365MB_MSTI_CTRL_BASE + ((_msti) << 1) + ((_physport) >> 3))
@@ -767,7 +767,8 @@ static int rtl8365mb_ext_config_rgmii(struct realtek_smi *smi, int port,
* 0 = no delay, 1 = 2 ns delay
* RX delay:
* 0 = no delay, 7 = maximum delay
- * No units are specified, but there are a total of 8 steps.
+ * Each step is approximately 0.3 ns, so the maximum delay is about
+ * 2.1 ns.
*
* The vendor driver also states that this must be configured *before*
* forcing the external interface into a particular mode, which is done
@@ -778,10 +779,6 @@ static int rtl8365mb_ext_config_rgmii(struct realtek_smi *smi, int port,
* specified. We ignore the detail of the RGMII interface mode
* (RGMII_{RXID, TXID, etc.}), as this is considered to be a PHY-only
* property.
- *
- * For the RX delay, we assume that a register value of 4 corresponds to
- * 2 ns. But this is just an educated guess, so ignore all other values
- * to avoid too much confusion.
*/
if (!of_property_read_u32(dn, "tx-internal-delay-ps", &val)) {
val = val / 1000; /* convert to ns */
@@ -794,13 +791,13 @@ static int rtl8365mb_ext_config_rgmii(struct realtek_smi *smi, int port,
}
if (!of_property_read_u32(dn, "rx-internal-delay-ps", &val)) {
- val = val / 1000; /* convert to ns */
+ val = DIV_ROUND_CLOSEST(val, 300); /* convert to 0.3 ns step */
- if (val == 0 || val == 2)
- rx_delay = val * 2;
+ if (val <= 7)
+ rx_delay = val;
else
dev_warn(smi->dev,
- "EXT port RX delay must be 0 to 2 ns\n");
+ "EXT port RX delay must be 0 to 2.1 ns\n");
}
ret = regmap_update_bits(
@@ -903,7 +900,8 @@ static bool rtl8365mb_phy_mode_supported(struct dsa_switch *ds, int port,
{
if (dsa_is_user_port(ds, port) &&
(interface == PHY_INTERFACE_MODE_NA ||
- interface == PHY_INTERFACE_MODE_INTERNAL))
+ interface == PHY_INTERFACE_MODE_INTERNAL ||
+ interface == PHY_INTERFACE_MODE_GMII))
/* Internal PHY */
return true;
else if (dsa_is_cpu_port(ds, port) &&
diff --git a/drivers/net/dsa/rtl8366rb.c b/drivers/net/dsa/rtl8366rb.c
index 03deacd..ecc19bd 100644
--- a/drivers/net/dsa/rtl8366rb.c
+++ b/drivers/net/dsa/rtl8366rb.c
@@ -1186,7 +1186,8 @@ rtl8366rb_port_disable(struct dsa_switch *ds, int port)
static int
rtl8366rb_port_bridge_join(struct dsa_switch *ds, int port,
- struct net_device *bridge)
+ struct dsa_bridge bridge,
+ bool *tx_fwd_offload)
{
struct realtek_smi *smi = ds->priv;
unsigned int port_bitmap = 0;
@@ -1198,7 +1199,7 @@ rtl8366rb_port_bridge_join(struct dsa_switch *ds, int port,
if (i == port)
continue;
/* Not on this bridge */
- if (dsa_to_port(ds, i)->bridge_dev != bridge)
+ if (!dsa_port_offloads_bridge(dsa_to_port(ds, i), &bridge))
continue;
/* Join this port to each other port on the bridge */
ret = regmap_update_bits(smi->map, RTL8366RB_PORT_ISO(i),
@@ -1218,7 +1219,7 @@ rtl8366rb_port_bridge_join(struct dsa_switch *ds, int port,
static void
rtl8366rb_port_bridge_leave(struct dsa_switch *ds, int port,
- struct net_device *bridge)
+ struct dsa_bridge bridge)
{
struct realtek_smi *smi = ds->priv;
unsigned int port_bitmap = 0;
@@ -1230,7 +1231,7 @@ rtl8366rb_port_bridge_leave(struct dsa_switch *ds, int port,
if (i == port)
continue;
/* Not on this bridge */
- if (dsa_to_port(ds, i)->bridge_dev != bridge)
+ if (!dsa_port_offloads_bridge(dsa_to_port(ds, i), &bridge))
continue;
/* Remove this port from any other port on the bridge */
ret = regmap_update_bits(smi->map, RTL8366RB_PORT_ISO(i),
diff --git a/drivers/net/dsa/sja1105/sja1105.h b/drivers/net/dsa/sja1105/sja1105.h
index 21dba16..9ba2ec2 100644
--- a/drivers/net/dsa/sja1105/sja1105.h
+++ b/drivers/net/dsa/sja1105/sja1105.h
@@ -249,6 +249,7 @@ struct sja1105_private {
bool fixed_link[SJA1105_MAX_NUM_PORTS];
unsigned long ucast_egress_floods;
unsigned long bcast_egress_floods;
+ unsigned long hwts_tx_en;
const struct sja1105_info *info;
size_t max_xfer_len;
struct spi_device *spidev;
@@ -256,11 +257,13 @@ struct sja1105_private {
u16 bridge_pvid[SJA1105_MAX_NUM_PORTS];
u16 tag_8021q_pvid[SJA1105_MAX_NUM_PORTS];
struct sja1105_flow_block flow_block;
- struct sja1105_port ports[SJA1105_MAX_NUM_PORTS];
/* Serializes transmission of management frames so that
* the switch doesn't confuse them with one another.
*/
struct mutex mgmt_lock;
+ /* PTP two-step TX timestamp ID, and its serialization lock */
+ spinlock_t ts_id_lock;
+ u8 ts_id;
/* Serializes access to the dynamic config interface */
struct mutex dynamic_config_lock;
struct devlink_region **regions;
@@ -269,7 +272,6 @@ struct sja1105_private {
struct mii_bus *mdio_base_tx;
struct mii_bus *mdio_pcs;
struct dw_xpcs *xpcs[SJA1105_MAX_NUM_PORTS];
- struct sja1105_tagger_data tagger_data;
struct sja1105_ptp_data ptp_data;
struct sja1105_tas_data tas_data;
};
diff --git a/drivers/net/dsa/sja1105/sja1105_flower.c b/drivers/net/dsa/sja1105/sja1105_flower.c
index 72b9b39..7dcdd78 100644
--- a/drivers/net/dsa/sja1105/sja1105_flower.c
+++ b/drivers/net/dsa/sja1105/sja1105_flower.c
@@ -379,7 +379,7 @@ int sja1105_cls_flower_add(struct dsa_switch *ds, int port,
vl_rule = true;
rc = sja1105_vl_gate(priv, port, extack, cookie,
- &key, act->gate.index,
+ &key, act->hw_index,
act->gate.prio,
act->gate.basetime,
act->gate.cycletime,
diff --git a/drivers/net/dsa/sja1105/sja1105_main.c b/drivers/net/dsa/sja1105/sja1105_main.c
index c343effe..b513713 100644
--- a/drivers/net/dsa/sja1105/sja1105_main.c
+++ b/drivers/net/dsa/sja1105/sja1105_main.c
@@ -118,13 +118,14 @@ static int sja1105_pvid_apply(struct sja1105_private *priv, int port, u16 pvid)
static int sja1105_commit_pvid(struct dsa_switch *ds, int port)
{
struct dsa_port *dp = dsa_to_port(ds, port);
+ struct net_device *br = dsa_port_bridge_dev_get(dp);
struct sja1105_private *priv = ds->priv;
struct sja1105_vlan_lookup_entry *vlan;
bool drop_untagged = false;
int match, rc;
u16 pvid;
- if (dp->bridge_dev && br_vlan_enabled(dp->bridge_dev))
+ if (br && br_vlan_enabled(br))
pvid = priv->bridge_pvid[port];
else
pvid = priv->tag_8021q_pvid[port];
@@ -1979,7 +1980,7 @@ static int sja1105_manage_flood_domains(struct sja1105_private *priv)
}
static int sja1105_bridge_member(struct dsa_switch *ds, int port,
- struct net_device *br, bool member)
+ struct dsa_bridge bridge, bool member)
{
struct sja1105_l2_forwarding_entry *l2_fwd;
struct sja1105_private *priv = ds->priv;
@@ -2004,7 +2005,7 @@ static int sja1105_bridge_member(struct dsa_switch *ds, int port,
*/
if (i == port)
continue;
- if (dsa_to_port(ds, i)->bridge_dev != br)
+ if (!dsa_port_offloads_bridge(dsa_to_port(ds, i), &bridge))
continue;
sja1105_port_allow_traffic(l2_fwd, i, port, member);
sja1105_port_allow_traffic(l2_fwd, port, i, member);
@@ -2073,15 +2074,31 @@ static void sja1105_bridge_stp_state_set(struct dsa_switch *ds, int port,
}
static int sja1105_bridge_join(struct dsa_switch *ds, int port,
- struct net_device *br)
+ struct dsa_bridge bridge,
+ bool *tx_fwd_offload)
{
- return sja1105_bridge_member(ds, port, br, true);
+ int rc;
+
+ rc = sja1105_bridge_member(ds, port, bridge, true);
+ if (rc)
+ return rc;
+
+ rc = dsa_tag_8021q_bridge_tx_fwd_offload(ds, port, bridge);
+ if (rc) {
+ sja1105_bridge_member(ds, port, bridge, false);
+ return rc;
+ }
+
+ *tx_fwd_offload = true;
+
+ return 0;
}
static void sja1105_bridge_leave(struct dsa_switch *ds, int port,
- struct net_device *br)
+ struct dsa_bridge bridge)
{
- sja1105_bridge_member(ds, port, br, false);
+ dsa_tag_8021q_bridge_tx_fwd_unoffload(ds, port, bridge);
+ sja1105_bridge_member(ds, port, bridge, false);
}
#define BYTES_PER_KBIT (1000LL / 8)
@@ -2587,8 +2604,9 @@ static int sja1105_prechangeupper(struct dsa_switch *ds, int port,
if (netif_is_bridge_master(upper)) {
list_for_each_entry(dp, &dst->ports, list) {
- if (dp->bridge_dev && dp->bridge_dev != upper &&
- br_vlan_enabled(dp->bridge_dev)) {
+ struct net_device *br = dsa_port_bridge_dev_get(dp);
+
+ if (br && br != upper && br_vlan_enabled(br)) {
NL_SET_ERR_MSG_MOD(extack,
"Only one VLAN-aware bridge is supported");
return -EBUSY;
@@ -2599,18 +2617,6 @@ static int sja1105_prechangeupper(struct dsa_switch *ds, int port,
return 0;
}
-static void sja1105_port_disable(struct dsa_switch *ds, int port)
-{
- struct sja1105_private *priv = ds->priv;
- struct sja1105_port *sp = &priv->ports[port];
-
- if (!dsa_is_user_port(ds, port))
- return;
-
- kthread_cancel_work_sync(&sp->xmit_work);
- skb_queue_purge(&sp->xmit_queue);
-}
-
static int sja1105_mgmt_xmit(struct dsa_switch *ds, int port, int slot,
struct sk_buff *skb, bool takets)
{
@@ -2669,10 +2675,8 @@ static int sja1105_mgmt_xmit(struct dsa_switch *ds, int port, int slot,
return NETDEV_TX_OK;
}
-#define work_to_port(work) \
- container_of((work), struct sja1105_port, xmit_work)
-#define tagger_to_sja1105(t) \
- container_of((t), struct sja1105_private, tagger_data)
+#define work_to_xmit_work(w) \
+ container_of((w), struct sja1105_deferred_xmit_work, work)
/* Deferred work is unfortunately necessary because setting up the management
* route cannot be done from atomit context (SPI transfer takes a sleepable
@@ -2680,25 +2684,41 @@ static int sja1105_mgmt_xmit(struct dsa_switch *ds, int port, int slot,
*/
static void sja1105_port_deferred_xmit(struct kthread_work *work)
{
- struct sja1105_port *sp = work_to_port(work);
- struct sja1105_tagger_data *tagger_data = sp->data;
- struct sja1105_private *priv = tagger_to_sja1105(tagger_data);
- int port = sp - priv->ports;
- struct sk_buff *skb;
+ struct sja1105_deferred_xmit_work *xmit_work = work_to_xmit_work(work);
+ struct sk_buff *clone, *skb = xmit_work->skb;
+ struct dsa_switch *ds = xmit_work->dp->ds;
+ struct sja1105_private *priv = ds->priv;
+ int port = xmit_work->dp->index;
- while ((skb = skb_dequeue(&sp->xmit_queue)) != NULL) {
- struct sk_buff *clone = SJA1105_SKB_CB(skb)->clone;
+ clone = SJA1105_SKB_CB(skb)->clone;
- mutex_lock(&priv->mgmt_lock);
+ mutex_lock(&priv->mgmt_lock);
- sja1105_mgmt_xmit(priv->ds, port, 0, skb, !!clone);
+ sja1105_mgmt_xmit(ds, port, 0, skb, !!clone);
- /* The clone, if there, was made by dsa_skb_tx_timestamp */
- if (clone)
- sja1105_ptp_txtstamp_skb(priv->ds, port, clone);
+ /* The clone, if there, was made by dsa_skb_tx_timestamp */
+ if (clone)
+ sja1105_ptp_txtstamp_skb(ds, port, clone);
- mutex_unlock(&priv->mgmt_lock);
- }
+ mutex_unlock(&priv->mgmt_lock);
+
+ kfree(xmit_work);
+}
+
+static int sja1105_connect_tag_protocol(struct dsa_switch *ds,
+ enum dsa_tag_protocol proto)
+{
+ struct sja1105_private *priv = ds->priv;
+ struct sja1105_tagger_data *tagger_data;
+
+ if (proto != priv->info->tag_proto)
+ return -EPROTONOSUPPORT;
+
+ tagger_data = sja1105_tagger_data(ds);
+ tagger_data->xmit_work_fn = sja1105_port_deferred_xmit;
+ tagger_data->meta_tstamp_handler = sja1110_process_meta_tstamp;
+
+ return 0;
}
/* The MAXAGE setting belongs to the L2 Forwarding Parameters table,
@@ -3001,58 +3021,6 @@ static int sja1105_port_bridge_flags(struct dsa_switch *ds, int port,
return 0;
}
-static void sja1105_teardown_ports(struct sja1105_private *priv)
-{
- struct dsa_switch *ds = priv->ds;
- int port;
-
- for (port = 0; port < ds->num_ports; port++) {
- struct sja1105_port *sp = &priv->ports[port];
-
- if (sp->xmit_worker)
- kthread_destroy_worker(sp->xmit_worker);
- }
-}
-
-static int sja1105_setup_ports(struct sja1105_private *priv)
-{
- struct sja1105_tagger_data *tagger_data = &priv->tagger_data;
- struct dsa_switch *ds = priv->ds;
- int port, rc;
-
- /* Connections between dsa_port and sja1105_port */
- for (port = 0; port < ds->num_ports; port++) {
- struct sja1105_port *sp = &priv->ports[port];
- struct dsa_port *dp = dsa_to_port(ds, port);
- struct kthread_worker *worker;
- struct net_device *slave;
-
- if (!dsa_port_is_user(dp))
- continue;
-
- dp->priv = sp;
- sp->data = tagger_data;
- slave = dp->slave;
- kthread_init_work(&sp->xmit_work, sja1105_port_deferred_xmit);
- worker = kthread_create_worker(0, "%s_xmit", slave->name);
- if (IS_ERR(worker)) {
- rc = PTR_ERR(worker);
- dev_err(ds->dev,
- "failed to create deferred xmit thread: %d\n",
- rc);
- goto out_destroy_workers;
- }
- sp->xmit_worker = worker;
- skb_queue_head_init(&sp->xmit_queue);
- }
-
- return 0;
-
-out_destroy_workers:
- sja1105_teardown_ports(priv);
- return rc;
-}
-
/* The programming model for the SJA1105 switch is "all-at-once" via static
* configuration tables. Some of these can be dynamically modified at runtime,
* but not the xMII mode parameters table.
@@ -3098,10 +3066,6 @@ static int sja1105_setup(struct dsa_switch *ds)
}
}
- rc = sja1105_setup_ports(priv);
- if (rc)
- goto out_static_config_free;
-
sja1105_tas_setup(ds);
sja1105_flower_setup(ds);
@@ -3139,7 +3103,7 @@ static int sja1105_setup(struct dsa_switch *ds)
ds->vlan_filtering_is_global = true;
ds->untag_bridge_pvid = true;
/* tag_8021q has 3 bits for the VBID, and the value 0 is reserved */
- ds->num_fwd_offloading_bridges = 7;
+ ds->max_num_bridges = 7;
/* Advertise the 8 egress queues */
ds->num_tx_queues = SJA1105_NUM_TC;
@@ -3158,7 +3122,6 @@ static int sja1105_setup(struct dsa_switch *ds)
out_flower_teardown:
sja1105_flower_teardown(ds);
sja1105_tas_teardown(ds);
- sja1105_teardown_ports(priv);
out_static_config_free:
sja1105_static_config_free(&priv->static_config);
@@ -3178,12 +3141,12 @@ static void sja1105_teardown(struct dsa_switch *ds)
sja1105_ptp_clock_unregister(ds);
sja1105_flower_teardown(ds);
sja1105_tas_teardown(ds);
- sja1105_teardown_ports(priv);
sja1105_static_config_free(&priv->static_config);
}
static const struct dsa_switch_ops sja1105_switch_ops = {
.get_tag_protocol = sja1105_get_tag_protocol,
+ .connect_tag_protocol = sja1105_connect_tag_protocol,
.setup = sja1105_setup,
.teardown = sja1105_teardown,
.set_ageing_time = sja1105_set_ageing_time,
@@ -3197,7 +3160,6 @@ static const struct dsa_switch_ops sja1105_switch_ops = {
.get_ethtool_stats = sja1105_get_ethtool_stats,
.get_sset_count = sja1105_get_sset_count,
.get_ts_info = sja1105_get_ts_info,
- .port_disable = sja1105_port_disable,
.port_fdb_dump = sja1105_fdb_dump,
.port_fdb_add = sja1105_fdb_add,
.port_fdb_del = sja1105_fdb_del,
@@ -3228,8 +3190,6 @@ static const struct dsa_switch_ops sja1105_switch_ops = {
.tag_8021q_vlan_add = sja1105_dsa_8021q_vlan_add,
.tag_8021q_vlan_del = sja1105_dsa_8021q_vlan_del,
.port_prechangeupper = sja1105_prechangeupper,
- .port_bridge_tx_fwd_offload = dsa_tag_8021q_bridge_tx_fwd_offload,
- .port_bridge_tx_fwd_unoffload = dsa_tag_8021q_bridge_tx_fwd_unoffload,
};
static const struct of_device_id sja1105_dt_ids[];
@@ -3367,6 +3327,7 @@ static int sja1105_probe(struct spi_device *spi)
mutex_init(&priv->ptp_data.lock);
mutex_init(&priv->dynamic_config_lock);
mutex_init(&priv->mgmt_lock);
+ spin_lock_init(&priv->ts_id_lock);
rc = sja1105_parse_dt(priv);
if (rc < 0) {
diff --git a/drivers/net/dsa/sja1105/sja1105_ptp.c b/drivers/net/dsa/sja1105/sja1105_ptp.c
index 5439699..be3068a 100644
--- a/drivers/net/dsa/sja1105/sja1105_ptp.c
+++ b/drivers/net/dsa/sja1105/sja1105_ptp.c
@@ -58,13 +58,12 @@ enum sja1105_ptp_clk_mode {
#define ptp_data_to_sja1105(d) \
container_of((d), struct sja1105_private, ptp_data)
-/* Must be called only with priv->tagger_data.state bit
- * SJA1105_HWTS_RX_EN cleared
+/* Must be called only while the RX timestamping state of the tagger
+ * is turned off
*/
static int sja1105_change_rxtstamping(struct sja1105_private *priv,
bool on)
{
- struct sja1105_tagger_data *tagger_data = &priv->tagger_data;
struct sja1105_ptp_data *ptp_data = &priv->ptp_data;
struct sja1105_general_params_entry *general_params;
struct sja1105_table *table;
@@ -74,13 +73,8 @@ static int sja1105_change_rxtstamping(struct sja1105_private *priv,
general_params->send_meta1 = on;
general_params->send_meta0 = on;
- /* Initialize the meta state machine to a known state */
- if (priv->tagger_data.stampable_skb) {
- kfree_skb(priv->tagger_data.stampable_skb);
- priv->tagger_data.stampable_skb = NULL;
- }
ptp_cancel_worker_sync(ptp_data->clock);
- skb_queue_purge(&tagger_data->skb_txtstamp_queue);
+ skb_queue_purge(&ptp_data->skb_txtstamp_queue);
skb_queue_purge(&ptp_data->skb_rxtstamp_queue);
return sja1105_static_config_reload(priv, SJA1105_RX_HWTSTAMPING);
@@ -88,6 +82,7 @@ static int sja1105_change_rxtstamping(struct sja1105_private *priv,
int sja1105_hwtstamp_set(struct dsa_switch *ds, int port, struct ifreq *ifr)
{
+ struct sja1105_tagger_data *tagger_data = sja1105_tagger_data(ds);
struct sja1105_private *priv = ds->priv;
struct hwtstamp_config config;
bool rx_on;
@@ -98,10 +93,10 @@ int sja1105_hwtstamp_set(struct dsa_switch *ds, int port, struct ifreq *ifr)
switch (config.tx_type) {
case HWTSTAMP_TX_OFF:
- priv->ports[port].hwts_tx_en = false;
+ priv->hwts_tx_en &= ~BIT(port);
break;
case HWTSTAMP_TX_ON:
- priv->ports[port].hwts_tx_en = true;
+ priv->hwts_tx_en |= BIT(port);
break;
default:
return -ERANGE;
@@ -116,8 +111,8 @@ int sja1105_hwtstamp_set(struct dsa_switch *ds, int port, struct ifreq *ifr)
break;
}
- if (rx_on != test_bit(SJA1105_HWTS_RX_EN, &priv->tagger_data.state)) {
- clear_bit(SJA1105_HWTS_RX_EN, &priv->tagger_data.state);
+ if (rx_on != tagger_data->rxtstamp_get_state(ds)) {
+ tagger_data->rxtstamp_set_state(ds, false);
rc = sja1105_change_rxtstamping(priv, rx_on);
if (rc < 0) {
@@ -126,7 +121,7 @@ int sja1105_hwtstamp_set(struct dsa_switch *ds, int port, struct ifreq *ifr)
return rc;
}
if (rx_on)
- set_bit(SJA1105_HWTS_RX_EN, &priv->tagger_data.state);
+ tagger_data->rxtstamp_set_state(ds, true);
}
if (copy_to_user(ifr->ifr_data, &config, sizeof(config)))
@@ -136,15 +131,16 @@ int sja1105_hwtstamp_set(struct dsa_switch *ds, int port, struct ifreq *ifr)
int sja1105_hwtstamp_get(struct dsa_switch *ds, int port, struct ifreq *ifr)
{
+ struct sja1105_tagger_data *tagger_data = sja1105_tagger_data(ds);
struct sja1105_private *priv = ds->priv;
struct hwtstamp_config config;
config.flags = 0;
- if (priv->ports[port].hwts_tx_en)
+ if (priv->hwts_tx_en & BIT(port))
config.tx_type = HWTSTAMP_TX_ON;
else
config.tx_type = HWTSTAMP_TX_OFF;
- if (test_bit(SJA1105_HWTS_RX_EN, &priv->tagger_data.state))
+ if (tagger_data->rxtstamp_get_state(ds))
config.rx_filter = HWTSTAMP_FILTER_PTP_V2_L2_EVENT;
else
config.rx_filter = HWTSTAMP_FILTER_NONE;
@@ -417,10 +413,11 @@ static long sja1105_rxtstamp_work(struct ptp_clock_info *ptp)
bool sja1105_rxtstamp(struct dsa_switch *ds, int port, struct sk_buff *skb)
{
+ struct sja1105_tagger_data *tagger_data = sja1105_tagger_data(ds);
struct sja1105_private *priv = ds->priv;
struct sja1105_ptp_data *ptp_data = &priv->ptp_data;
- if (!test_bit(SJA1105_HWTS_RX_EN, &priv->tagger_data.state))
+ if (!tagger_data->rxtstamp_get_state(ds))
return false;
/* We need to read the full PTP clock to reconstruct the Rx
@@ -453,6 +450,39 @@ bool sja1105_port_rxtstamp(struct dsa_switch *ds, int port,
return priv->info->rxtstamp(ds, port, skb);
}
+void sja1110_process_meta_tstamp(struct dsa_switch *ds, int port, u8 ts_id,
+ enum sja1110_meta_tstamp dir, u64 tstamp)
+{
+ struct sja1105_private *priv = ds->priv;
+ struct sja1105_ptp_data *ptp_data = &priv->ptp_data;
+ struct sk_buff *skb, *skb_tmp, *skb_match = NULL;
+ struct skb_shared_hwtstamps shwt = {0};
+
+ /* We don't care about RX timestamps on the CPU port */
+ if (dir == SJA1110_META_TSTAMP_RX)
+ return;
+
+ spin_lock(&ptp_data->skb_txtstamp_queue.lock);
+
+ skb_queue_walk_safe(&ptp_data->skb_txtstamp_queue, skb, skb_tmp) {
+ if (SJA1105_SKB_CB(skb)->ts_id != ts_id)
+ continue;
+
+ __skb_unlink(skb, &ptp_data->skb_txtstamp_queue);
+ skb_match = skb;
+
+ break;
+ }
+
+ spin_unlock(&ptp_data->skb_txtstamp_queue.lock);
+
+ if (WARN_ON(!skb_match))
+ return;
+
+ shwt.hwtstamp = ns_to_ktime(sja1105_ticks_to_ns(tstamp));
+ skb_complete_tx_timestamp(skb_match, &shwt);
+}
+
/* In addition to cloning the skb which is done by the common
* sja1105_port_txtstamp, we need to generate a timestamp ID and save the
* packet to the TX timestamping queue.
@@ -461,22 +491,22 @@ void sja1110_txtstamp(struct dsa_switch *ds, int port, struct sk_buff *skb)
{
struct sk_buff *clone = SJA1105_SKB_CB(skb)->clone;
struct sja1105_private *priv = ds->priv;
- struct sja1105_port *sp = &priv->ports[port];
+ struct sja1105_ptp_data *ptp_data = &priv->ptp_data;
u8 ts_id;
skb_shinfo(skb)->tx_flags |= SKBTX_IN_PROGRESS;
- spin_lock(&sp->data->meta_lock);
+ spin_lock(&priv->ts_id_lock);
- ts_id = sp->data->ts_id;
+ ts_id = priv->ts_id;
/* Deal automatically with 8-bit wraparound */
- sp->data->ts_id++;
+ priv->ts_id++;
SJA1105_SKB_CB(clone)->ts_id = ts_id;
- spin_unlock(&sp->data->meta_lock);
+ spin_unlock(&priv->ts_id_lock);
- skb_queue_tail(&sp->data->skb_txtstamp_queue, clone);
+ skb_queue_tail(&ptp_data->skb_txtstamp_queue, clone);
}
/* Called from dsa_skb_tx_timestamp. This callback is just to clone
@@ -486,10 +516,9 @@ void sja1110_txtstamp(struct dsa_switch *ds, int port, struct sk_buff *skb)
void sja1105_port_txtstamp(struct dsa_switch *ds, int port, struct sk_buff *skb)
{
struct sja1105_private *priv = ds->priv;
- struct sja1105_port *sp = &priv->ports[port];
struct sk_buff *clone;
- if (!sp->hwts_tx_en)
+ if (!(priv->hwts_tx_en & BIT(port)))
return;
clone = skb_clone_sk(skb);
@@ -896,7 +925,6 @@ static struct ptp_pin_desc sja1105_ptp_pin = {
int sja1105_ptp_clock_register(struct dsa_switch *ds)
{
struct sja1105_private *priv = ds->priv;
- struct sja1105_tagger_data *tagger_data = &priv->tagger_data;
struct sja1105_ptp_data *ptp_data = &priv->ptp_data;
ptp_data->caps = (struct ptp_clock_info) {
@@ -919,8 +947,7 @@ int sja1105_ptp_clock_register(struct dsa_switch *ds)
/* Only used on SJA1105 */
skb_queue_head_init(&ptp_data->skb_rxtstamp_queue);
/* Only used on SJA1110 */
- skb_queue_head_init(&tagger_data->skb_txtstamp_queue);
- spin_lock_init(&tagger_data->meta_lock);
+ skb_queue_head_init(&ptp_data->skb_txtstamp_queue);
ptp_data->clock = ptp_clock_register(&ptp_data->caps, ds->dev);
if (IS_ERR_OR_NULL(ptp_data->clock))
@@ -937,7 +964,6 @@ int sja1105_ptp_clock_register(struct dsa_switch *ds)
void sja1105_ptp_clock_unregister(struct dsa_switch *ds)
{
struct sja1105_private *priv = ds->priv;
- struct sja1105_tagger_data *tagger_data = &priv->tagger_data;
struct sja1105_ptp_data *ptp_data = &priv->ptp_data;
if (IS_ERR_OR_NULL(ptp_data->clock))
@@ -945,7 +971,7 @@ void sja1105_ptp_clock_unregister(struct dsa_switch *ds)
del_timer_sync(&ptp_data->extts_timer);
ptp_cancel_worker_sync(ptp_data->clock);
- skb_queue_purge(&tagger_data->skb_txtstamp_queue);
+ skb_queue_purge(&ptp_data->skb_txtstamp_queue);
skb_queue_purge(&ptp_data->skb_rxtstamp_queue);
ptp_clock_unregister(ptp_data->clock);
ptp_data->clock = NULL;
diff --git a/drivers/net/dsa/sja1105/sja1105_ptp.h b/drivers/net/dsa/sja1105/sja1105_ptp.h
index 3ae6b9f..416461e 100644
--- a/drivers/net/dsa/sja1105/sja1105_ptp.h
+++ b/drivers/net/dsa/sja1105/sja1105_ptp.h
@@ -8,6 +8,21 @@
#if IS_ENABLED(CONFIG_NET_DSA_SJA1105_PTP)
+/* Timestamps are in units of 8 ns clock ticks (equivalent to
+ * a fixed 125 MHz clock).
+ */
+#define SJA1105_TICK_NS 8
+
+static inline s64 ns_to_sja1105_ticks(s64 ns)
+{
+ return ns / SJA1105_TICK_NS;
+}
+
+static inline s64 sja1105_ticks_to_ns(s64 ticks)
+{
+ return ticks * SJA1105_TICK_NS;
+}
+
/* Calculate the first base_time in the future that satisfies this
* relationship:
*
@@ -62,6 +77,10 @@ struct sja1105_ptp_data {
struct timer_list extts_timer;
/* Used only on SJA1105 to reconstruct partial timestamps */
struct sk_buff_head skb_rxtstamp_queue;
+ /* Used on SJA1110 where meta frames are generated only for
+ * 2-step TX timestamps
+ */
+ struct sk_buff_head skb_txtstamp_queue;
struct ptp_clock_info caps;
struct ptp_clock *clock;
struct sja1105_ptp_cmd cmd;
@@ -112,6 +131,9 @@ bool sja1105_rxtstamp(struct dsa_switch *ds, int port, struct sk_buff *skb);
bool sja1110_rxtstamp(struct dsa_switch *ds, int port, struct sk_buff *skb);
void sja1110_txtstamp(struct dsa_switch *ds, int port, struct sk_buff *skb);
+void sja1110_process_meta_tstamp(struct dsa_switch *ds, int port, u8 ts_id,
+ enum sja1110_meta_tstamp dir, u64 tstamp);
+
#else
struct sja1105_ptp_cmd;
@@ -178,6 +200,8 @@ static inline int sja1105_ptp_commit(struct dsa_switch *ds,
#define sja1110_rxtstamp NULL
#define sja1110_txtstamp NULL
+#define sja1110_process_meta_tstamp NULL
+
#endif /* IS_ENABLED(CONFIG_NET_DSA_SJA1105_PTP) */
#endif /* _SJA1105_PTP_H */
diff --git a/drivers/net/dsa/vitesse-vsc73xx-core.c b/drivers/net/dsa/vitesse-vsc73xx-core.c
index a4b1447..ae55167 100644
--- a/drivers/net/dsa/vitesse-vsc73xx-core.c
+++ b/drivers/net/dsa/vitesse-vsc73xx-core.c
@@ -1122,9 +1122,6 @@ static int vsc73xx_gpio_probe(struct vsc73xx *vsc)
vsc->gc.ngpio = 4;
vsc->gc.owner = THIS_MODULE;
vsc->gc.parent = vsc->dev;
-#if IS_ENABLED(CONFIG_OF_GPIO)
- vsc->gc.of_node = vsc->dev->of_node;
-#endif
vsc->gc.base = -1;
vsc->gc.get = vsc73xx_gpio_get;
vsc->gc.set = vsc73xx_gpio_set;
@@ -1216,12 +1213,10 @@ int vsc73xx_probe(struct vsc73xx *vsc)
}
EXPORT_SYMBOL(vsc73xx_probe);
-int vsc73xx_remove(struct vsc73xx *vsc)
+void vsc73xx_remove(struct vsc73xx *vsc)
{
dsa_unregister_switch(vsc->ds);
gpiod_set_value(vsc->reset, 1);
-
- return 0;
}
EXPORT_SYMBOL(vsc73xx_remove);
diff --git a/drivers/net/dsa/vitesse-vsc73xx.h b/drivers/net/dsa/vitesse-vsc73xx.h
index 30b9515..30b1f0a 100644
--- a/drivers/net/dsa/vitesse-vsc73xx.h
+++ b/drivers/net/dsa/vitesse-vsc73xx.h
@@ -26,5 +26,5 @@ struct vsc73xx_ops {
int vsc73xx_is_addr_valid(u8 block, u8 subblock);
int vsc73xx_probe(struct vsc73xx *vsc);
-int vsc73xx_remove(struct vsc73xx *vsc);
+void vsc73xx_remove(struct vsc73xx *vsc);
void vsc73xx_shutdown(struct vsc73xx *vsc);
diff --git a/drivers/net/dsa/xrs700x/xrs700x.c b/drivers/net/dsa/xrs700x/xrs700x.c
index 910fcb3..0730352c 100644
--- a/drivers/net/dsa/xrs700x/xrs700x.c
+++ b/drivers/net/dsa/xrs700x/xrs700x.c
@@ -5,6 +5,7 @@
*/
#include <net/dsa.h>
+#include <linux/etherdevice.h>
#include <linux/if_bridge.h>
#include <linux/of_device.h>
#include <linux/netdev_features.h>
@@ -501,7 +502,7 @@ static void xrs700x_mac_link_up(struct dsa_switch *ds, int port,
}
static int xrs700x_bridge_common(struct dsa_switch *ds, int port,
- struct net_device *bridge, bool join)
+ struct dsa_bridge bridge, bool join)
{
unsigned int i, cpu_mask = 0, mask = 0;
struct xrs700x *priv = ds->priv;
@@ -513,14 +514,14 @@ static int xrs700x_bridge_common(struct dsa_switch *ds, int port,
cpu_mask |= BIT(i);
- if (dsa_to_port(ds, i)->bridge_dev == bridge)
+ if (dsa_port_offloads_bridge(dsa_to_port(ds, i), &bridge))
continue;
mask |= BIT(i);
}
for (i = 0; i < ds->num_ports; i++) {
- if (dsa_to_port(ds, i)->bridge_dev != bridge)
+ if (!dsa_port_offloads_bridge(dsa_to_port(ds, i), &bridge))
continue;
/* 1 = Disable forwarding to the port */
@@ -540,13 +541,13 @@ static int xrs700x_bridge_common(struct dsa_switch *ds, int port,
}
static int xrs700x_bridge_join(struct dsa_switch *ds, int port,
- struct net_device *bridge)
+ struct dsa_bridge bridge, bool *tx_fwd_offload)
{
return xrs700x_bridge_common(ds, port, bridge, true);
}
static void xrs700x_bridge_leave(struct dsa_switch *ds, int port,
- struct net_device *bridge)
+ struct dsa_bridge bridge)
{
xrs700x_bridge_common(ds, port, bridge, false);
}
diff --git a/drivers/net/eql.c b/drivers/net/eql.c
index 8ef3490..1111d1f 100644
--- a/drivers/net/eql.c
+++ b/drivers/net/eql.c
@@ -225,7 +225,7 @@ static void eql_kill_one_slave(slave_queue_t *queue, slave_t *slave)
list_del(&slave->list);
queue->num_slaves--;
slave->dev->flags &= ~IFF_SLAVE;
- dev_put(slave->dev);
+ dev_put_track(slave->dev, &slave->dev_tracker);
kfree(slave);
}
@@ -399,7 +399,7 @@ static int __eql_insert_slave(slave_queue_t *queue, slave_t *slave)
if (duplicate_slave)
eql_kill_one_slave(queue, duplicate_slave);
- dev_hold(slave->dev);
+ dev_hold_track(slave->dev, &slave->dev_tracker, GFP_ATOMIC);
list_add(&slave->list, &queue->all_slaves);
queue->num_slaves++;
slave->dev->flags |= IFF_SLAVE;
diff --git a/drivers/net/ethernet/3com/typhoon.c b/drivers/net/ethernet/3com/typhoon.c
index 05e15b6..481f1df 100644
--- a/drivers/net/ethernet/3com/typhoon.c
+++ b/drivers/net/ethernet/3com/typhoon.c
@@ -1138,7 +1138,9 @@ typhoon_set_wol(struct net_device *dev, struct ethtool_wolinfo *wol)
}
static void
-typhoon_get_ringparam(struct net_device *dev, struct ethtool_ringparam *ering)
+typhoon_get_ringparam(struct net_device *dev, struct ethtool_ringparam *ering,
+ struct kernel_ethtool_ringparam *kernel_ering,
+ struct netlink_ext_ack *extack)
{
ering->rx_max_pending = RXENT_ENTRIES;
ering->tx_max_pending = TXLO_ENTRIES - 1;
diff --git a/drivers/net/ethernet/8390/hydra.c b/drivers/net/ethernet/8390/hydra.c
index 941754e..1df7601 100644
--- a/drivers/net/ethernet/8390/hydra.c
+++ b/drivers/net/ethernet/8390/hydra.c
@@ -116,6 +116,7 @@ static int hydra_init(struct zorro_dev *z)
unsigned long ioaddr = board+HYDRA_NIC_BASE;
const char name[] = "NE2000";
int start_page, stop_page;
+ u8 macaddr[ETH_ALEN];
int j;
int err;
@@ -129,7 +130,8 @@ static int hydra_init(struct zorro_dev *z)
return -ENOMEM;
for (j = 0; j < ETH_ALEN; j++)
- dev->dev_addr[j] = *((u8 *)(board + HYDRA_ADDRPROM + 2*j));
+ macaddr[j] = *((u8 *)(board + HYDRA_ADDRPROM + 2*j));
+ eth_hw_addr_set(dev, macaddr);
/* We must set the 8390 for word mode. */
z_writeb(0x4b, ioaddr + NE_EN0_DCFG);
diff --git a/drivers/net/ethernet/8390/mac8390.c b/drivers/net/ethernet/8390/mac8390.c
index 91b04abf..7fb819b 100644
--- a/drivers/net/ethernet/8390/mac8390.c
+++ b/drivers/net/ethernet/8390/mac8390.c
@@ -292,6 +292,7 @@ static bool mac8390_rsrc_init(struct net_device *dev,
struct nubus_dirent ent;
int offset;
volatile unsigned short *i;
+ u8 addr[ETH_ALEN];
dev->irq = SLOT2IRQ(board->slot);
/* This is getting to be a habit */
@@ -314,7 +315,8 @@ static bool mac8390_rsrc_init(struct net_device *dev,
return false;
}
- nubus_get_rsrc_mem(dev->dev_addr, &ent, 6);
+ nubus_get_rsrc_mem(addr, &ent, 6);
+ eth_hw_addr_set(dev, addr);
if (useresources[cardtype] == 1) {
nubus_rewinddir(&dir);
diff --git a/drivers/net/ethernet/8390/smc-ultra.c b/drivers/net/ethernet/8390/smc-ultra.c
index 0890fa4..6e62c37 100644
--- a/drivers/net/ethernet/8390/smc-ultra.c
+++ b/drivers/net/ethernet/8390/smc-ultra.c
@@ -204,6 +204,7 @@ static int __init ultra_probe1(struct net_device *dev, int ioaddr)
{
int i, retval;
int checksum = 0;
+ u8 macaddr[ETH_ALEN];
const char *model_name;
unsigned char eeprom_irq = 0;
static unsigned version_printed;
@@ -239,7 +240,8 @@ static int __init ultra_probe1(struct net_device *dev, int ioaddr)
model_name = (idreg & 0xF0) == 0x20 ? "SMC Ultra" : "SMC EtherEZ";
for (i = 0; i < 6; i++)
- dev->dev_addr[i] = inb(ioaddr + 8 + i);
+ macaddr[i] = inb(ioaddr + 8 + i);
+ eth_hw_addr_set(dev, macaddr);
netdev_info(dev, "%s at %#3x, %pM", model_name,
ioaddr, dev->dev_addr);
diff --git a/drivers/net/ethernet/8390/wd.c b/drivers/net/ethernet/8390/wd.c
index 263a942..5b00c45 100644
--- a/drivers/net/ethernet/8390/wd.c
+++ b/drivers/net/ethernet/8390/wd.c
@@ -168,6 +168,7 @@ static int __init wd_probe1(struct net_device *dev, int ioaddr)
int checksum = 0;
int ancient = 0; /* An old card without config registers. */
int word16 = 0; /* 0 = 8 bit, 1 = 16 bit */
+ u8 addr[ETH_ALEN];
const char *model_name;
static unsigned version_printed;
struct ei_device *ei_local = netdev_priv(dev);
@@ -191,7 +192,8 @@ static int __init wd_probe1(struct net_device *dev, int ioaddr)
netdev_info(dev, version);
for (i = 0; i < 6; i++)
- dev->dev_addr[i] = inb(ioaddr + 8 + i);
+ addr[i] = inb(ioaddr + 8 + i);
+ eth_hw_addr_set(dev, addr);
netdev_info(dev, "WD80x3 at %#3x, %pM", ioaddr, dev->dev_addr);
diff --git a/drivers/net/ethernet/Kconfig b/drivers/net/ethernet/Kconfig
index 4601b38..db3ec47 100644
--- a/drivers/net/ethernet/Kconfig
+++ b/drivers/net/ethernet/Kconfig
@@ -73,6 +73,7 @@
source "drivers/net/ethernet/dec/Kconfig"
source "drivers/net/ethernet/dlink/Kconfig"
source "drivers/net/ethernet/emulex/Kconfig"
+source "drivers/net/ethernet/engleder/Kconfig"
source "drivers/net/ethernet/ezchip/Kconfig"
source "drivers/net/ethernet/faraday/Kconfig"
source "drivers/net/ethernet/freescale/Kconfig"
@@ -182,6 +183,7 @@
source "drivers/net/ethernet/ti/Kconfig"
source "drivers/net/ethernet/toshiba/Kconfig"
source "drivers/net/ethernet/tundra/Kconfig"
+source "drivers/net/ethernet/vertexcom/Kconfig"
source "drivers/net/ethernet/via/Kconfig"
source "drivers/net/ethernet/wiznet/Kconfig"
source "drivers/net/ethernet/xilinx/Kconfig"
diff --git a/drivers/net/ethernet/Makefile b/drivers/net/ethernet/Makefile
index fdd8c6c..8a87c10 100644
--- a/drivers/net/ethernet/Makefile
+++ b/drivers/net/ethernet/Makefile
@@ -36,6 +36,7 @@
obj-$(CONFIG_NET_VENDOR_DEC) += dec/
obj-$(CONFIG_NET_VENDOR_DLINK) += dlink/
obj-$(CONFIG_NET_VENDOR_EMULEX) += emulex/
+obj-$(CONFIG_NET_VENDOR_ENGLEDER) += engleder/
obj-$(CONFIG_NET_VENDOR_EZCHIP) += ezchip/
obj-$(CONFIG_NET_VENDOR_FARADAY) += faraday/
obj-$(CONFIG_NET_VENDOR_FREESCALE) += freescale/
@@ -92,6 +93,7 @@
obj-$(CONFIG_NET_VENDOR_TI) += ti/
obj-$(CONFIG_NET_VENDOR_TOSHIBA) += toshiba/
obj-$(CONFIG_NET_VENDOR_TUNDRA) += tundra/
+obj-$(CONFIG_NET_VENDOR_VERTEXCOM) += vertexcom/
obj-$(CONFIG_NET_VENDOR_VIA) += via/
obj-$(CONFIG_NET_VENDOR_WIZNET) += wiznet/
obj-$(CONFIG_NET_VENDOR_XILINX) += xilinx/
diff --git a/drivers/net/ethernet/agere/et131x.c b/drivers/net/ethernet/agere/et131x.c
index f4edc61..537e6a8 100644
--- a/drivers/net/ethernet/agere/et131x.c
+++ b/drivers/net/ethernet/agere/et131x.c
@@ -3914,10 +3914,9 @@ static int et131x_pci_setup(struct pci_dev *pdev,
pci_set_master(pdev);
/* Check the DMA addressing support of this device */
- if (dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)) &&
- dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32))) {
+ rc = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64));
+ if (rc) {
dev_err(&pdev->dev, "No usable DMA addressing method\n");
- rc = -EIO;
goto err_release_res;
}
diff --git a/drivers/net/ethernet/allwinner/sun4i-emac.c b/drivers/net/ethernet/allwinner/sun4i-emac.c
index 800ee02..849de45 100644
--- a/drivers/net/ethernet/allwinner/sun4i-emac.c
+++ b/drivers/net/ethernet/allwinner/sun4i-emac.c
@@ -29,6 +29,7 @@
#include <linux/platform_device.h>
#include <linux/phy.h>
#include <linux/soc/sunxi/sunxi_sram.h>
+#include <linux/dmaengine.h>
#include "sun4i-emac.h"
@@ -76,7 +77,6 @@ struct emac_board_info {
void __iomem *membase;
u32 msg_enable;
struct net_device *ndev;
- struct sk_buff *skb_last;
u16 tx_fifo_stat;
int emacrx_completed_flag;
@@ -87,6 +87,16 @@ struct emac_board_info {
unsigned int duplex;
phy_interface_t phy_interface;
+ struct dma_chan *rx_chan;
+ phys_addr_t emac_rx_fifo;
+};
+
+struct emac_dma_req {
+ struct emac_board_info *db;
+ struct dma_async_tx_descriptor *desc;
+ struct sk_buff *skb;
+ dma_addr_t rxbuf;
+ int count;
};
static void emac_update_speed(struct net_device *dev)
@@ -206,6 +216,117 @@ static void emac_inblk_32bit(void __iomem *reg, void *data, int count)
readsl(reg, data, round_up(count, 4) / 4);
}
+static struct emac_dma_req *
+emac_alloc_dma_req(struct emac_board_info *db,
+ struct dma_async_tx_descriptor *desc, struct sk_buff *skb,
+ dma_addr_t rxbuf, int count)
+{
+ struct emac_dma_req *req;
+
+ req = kzalloc(sizeof(struct emac_dma_req), GFP_ATOMIC);
+ if (!req)
+ return NULL;
+
+ req->db = db;
+ req->desc = desc;
+ req->skb = skb;
+ req->rxbuf = rxbuf;
+ req->count = count;
+ return req;
+}
+
+static void emac_free_dma_req(struct emac_dma_req *req)
+{
+ kfree(req);
+}
+
+static void emac_dma_done_callback(void *arg)
+{
+ struct emac_dma_req *req = arg;
+ struct emac_board_info *db = req->db;
+ struct sk_buff *skb = req->skb;
+ struct net_device *dev = db->ndev;
+ int rxlen = req->count;
+ u32 reg_val;
+
+ dma_unmap_single(db->dev, req->rxbuf, rxlen, DMA_FROM_DEVICE);
+
+ skb->protocol = eth_type_trans(skb, dev);
+ netif_rx(skb);
+ dev->stats.rx_bytes += rxlen;
+ /* Pass to upper layer */
+ dev->stats.rx_packets++;
+
+ /* re enable cpu receive */
+ reg_val = readl(db->membase + EMAC_RX_CTL_REG);
+ reg_val &= ~EMAC_RX_CTL_DMA_EN;
+ writel(reg_val, db->membase + EMAC_RX_CTL_REG);
+
+ /* re enable interrupt */
+ reg_val = readl(db->membase + EMAC_INT_CTL_REG);
+ reg_val |= (0x01 << 8);
+ writel(reg_val, db->membase + EMAC_INT_CTL_REG);
+
+ db->emacrx_completed_flag = 1;
+ emac_free_dma_req(req);
+}
+
+static int emac_dma_inblk_32bit(struct emac_board_info *db,
+ struct sk_buff *skb, void *rdptr, int count)
+{
+ struct dma_async_tx_descriptor *desc;
+ dma_cookie_t cookie;
+ dma_addr_t rxbuf;
+ struct emac_dma_req *req;
+ int ret = 0;
+
+ rxbuf = dma_map_single(db->dev, rdptr, count, DMA_FROM_DEVICE);
+ ret = dma_mapping_error(db->dev, rxbuf);
+ if (ret) {
+ dev_err(db->dev, "dma mapping error.\n");
+ return ret;
+ }
+
+ desc = dmaengine_prep_slave_single(db->rx_chan, rxbuf, count,
+ DMA_DEV_TO_MEM,
+ DMA_PREP_INTERRUPT | DMA_CTRL_ACK);
+ if (!desc) {
+ dev_err(db->dev, "prepare slave single failed\n");
+ ret = -ENOMEM;
+ goto prepare_err;
+ }
+
+ req = emac_alloc_dma_req(db, desc, skb, rxbuf, count);
+ if (!req) {
+ dev_err(db->dev, "alloc emac dma req error.\n");
+ ret = -ENOMEM;
+ goto alloc_req_err;
+ }
+
+ desc->callback_param = req;
+ desc->callback = emac_dma_done_callback;
+
+ cookie = dmaengine_submit(desc);
+ ret = dma_submit_error(cookie);
+ if (ret) {
+ dev_err(db->dev, "dma submit error.\n");
+ goto submit_err;
+ }
+
+ dma_async_issue_pending(db->rx_chan);
+ return ret;
+
+submit_err:
+ emac_free_dma_req(req);
+
+alloc_req_err:
+ dmaengine_desc_free(desc);
+
+prepare_err:
+ dma_unmap_single(db->dev, rxbuf, count, DMA_FROM_DEVICE);
+ return ret;
+}
+
/* ethtool ops */
static void emac_get_drvinfo(struct net_device *dev,
struct ethtool_drvinfo *info)
@@ -499,7 +620,6 @@ static void emac_rx(struct net_device *dev)
struct sk_buff *skb;
u8 *rdptr;
bool good_packet;
- static int rxlen_last;
unsigned int reg_val;
u32 rxhdr, rxstatus, rxcount, rxlen;
@@ -514,22 +634,6 @@ static void emac_rx(struct net_device *dev)
if (netif_msg_rx_status(db))
dev_dbg(db->dev, "RXCount: %x\n", rxcount);
- if ((db->skb_last != NULL) && (rxlen_last > 0)) {
- dev->stats.rx_bytes += rxlen_last;
-
- /* Pass to upper layer */
- db->skb_last->protocol = eth_type_trans(db->skb_last,
- dev);
- netif_rx(db->skb_last);
- dev->stats.rx_packets++;
- db->skb_last = NULL;
- rxlen_last = 0;
-
- reg_val = readl(db->membase + EMAC_RX_CTL_REG);
- reg_val &= ~EMAC_RX_CTL_DMA_EN;
- writel(reg_val, db->membase + EMAC_RX_CTL_REG);
- }
-
if (!rxcount) {
db->emacrx_completed_flag = 1;
reg_val = readl(db->membase + EMAC_INT_CTL_REG);
@@ -623,6 +727,19 @@ static void emac_rx(struct net_device *dev)
if (netif_msg_rx_status(db))
dev_dbg(db->dev, "RxLen %x\n", rxlen);
+ if (rxlen >= dev->mtu && db->rx_chan) {
+ reg_val = readl(db->membase + EMAC_RX_CTL_REG);
+ reg_val |= EMAC_RX_CTL_DMA_EN;
+ writel(reg_val, db->membase + EMAC_RX_CTL_REG);
+ if (!emac_dma_inblk_32bit(db, skb, rdptr, rxlen))
+ break;
+
+ /* re enable cpu receive. then try to receive by emac_inblk_32bit */
+ reg_val = readl(db->membase + EMAC_RX_CTL_REG);
+ reg_val &= ~EMAC_RX_CTL_DMA_EN;
+ writel(reg_val, db->membase + EMAC_RX_CTL_REG);
+ }
+
emac_inblk_32bit(db->membase + EMAC_RX_IO_DATA_REG,
rdptr, rxlen);
dev->stats.rx_bytes += rxlen;
@@ -677,7 +794,12 @@ static irqreturn_t emac_interrupt(int irq, void *dev_id)
reg_val = readl(db->membase + EMAC_INT_CTL_REG);
reg_val |= (0xf << 0) | (0x01 << 8);
writel(reg_val, db->membase + EMAC_INT_CTL_REG);
+ } else {
+ reg_val = readl(db->membase + EMAC_INT_CTL_REG);
+ reg_val |= (0xf << 0);
+ writel(reg_val, db->membase + EMAC_INT_CTL_REG);
}
+
spin_unlock(&db->lock);
return IRQ_HANDLED;
@@ -782,6 +904,58 @@ static const struct net_device_ops emac_netdev_ops = {
#endif
};
+static int emac_configure_dma(struct emac_board_info *db)
+{
+ struct platform_device *pdev = db->pdev;
+ struct net_device *ndev = db->ndev;
+ struct dma_slave_config conf = {};
+ struct resource *regs;
+ int err = 0;
+
+ regs = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ if (!regs) {
+ netdev_err(ndev, "get io resource from device failed.\n");
+ err = -ENOMEM;
+ goto out_clear_chan;
+ }
+
+ netdev_info(ndev, "get io resource from device: %pa, size = %u\n",
+ ®s->start, (unsigned int)resource_size(regs));
+ db->emac_rx_fifo = regs->start + EMAC_RX_IO_DATA_REG;
+
+ db->rx_chan = dma_request_chan(&pdev->dev, "rx");
+ if (IS_ERR(db->rx_chan)) {
+ netdev_err(ndev,
+ "failed to request dma channel. dma is disabled\n");
+ err = PTR_ERR(db->rx_chan);
+ goto out_clear_chan;
+ }
+
+ conf.direction = DMA_DEV_TO_MEM;
+ conf.dst_addr_width = DMA_SLAVE_BUSWIDTH_4_BYTES;
+ conf.src_addr_width = DMA_SLAVE_BUSWIDTH_4_BYTES;
+ conf.src_addr = db->emac_rx_fifo;
+ conf.dst_maxburst = 4;
+ conf.src_maxburst = 4;
+ conf.device_fc = false;
+
+ err = dmaengine_slave_config(db->rx_chan, &conf);
+ if (err) {
+ netdev_err(ndev, "config dma slave failed\n");
+ err = -EINVAL;
+ goto out_slave_configure_err;
+ }
+
+ return err;
+
+out_slave_configure_err:
+ dma_release_channel(db->rx_chan);
+
+out_clear_chan:
+ db->rx_chan = NULL;
+ return err;
+}
+
/* Search EMAC board, allocate space and register it
*/
static int emac_probe(struct platform_device *pdev)
@@ -824,6 +998,9 @@ static int emac_probe(struct platform_device *pdev)
goto out_iounmap;
}
+ if (emac_configure_dma(db))
+ netdev_info(ndev, "configure dma failed. disable dma.\n");
+
db->clk = devm_clk_get(&pdev->dev, NULL);
if (IS_ERR(db->clk)) {
ret = PTR_ERR(db->clk);
@@ -906,6 +1083,11 @@ static int emac_remove(struct platform_device *pdev)
struct net_device *ndev = platform_get_drvdata(pdev);
struct emac_board_info *db = netdev_priv(ndev);
+ if (db->rx_chan) {
+ dmaengine_terminate_all(db->rx_chan);
+ dma_release_channel(db->rx_chan);
+ }
+
unregister_netdev(ndev);
sunxi_sram_release(&pdev->dev);
clk_disable_unprepare(db->clk);
diff --git a/drivers/net/ethernet/alteon/acenic.c b/drivers/net/ethernet/alteon/acenic.c
index 732da15..22fe985 100644
--- a/drivers/net/ethernet/alteon/acenic.c
+++ b/drivers/net/ethernet/alteon/acenic.c
@@ -589,8 +589,7 @@ static int acenic_probe_one(struct pci_dev *pdev,
}
ap->name = dev->name;
- if (ap->pci_using_dac)
- dev->features |= NETIF_F_HIGHDMA;
+ dev->features |= NETIF_F_HIGHDMA;
pci_set_drvdata(pdev, dev);
@@ -1130,11 +1129,7 @@ static int ace_init(struct net_device *dev)
/*
* Configure DMA attributes.
*/
- if (!dma_set_mask(&pdev->dev, DMA_BIT_MASK(64))) {
- ap->pci_using_dac = 1;
- } else if (!dma_set_mask(&pdev->dev, DMA_BIT_MASK(32))) {
- ap->pci_using_dac = 0;
- } else {
+ if (dma_set_mask(&pdev->dev, DMA_BIT_MASK(64))) {
ecode = -ENODEV;
goto init_error;
}
diff --git a/drivers/net/ethernet/alteon/acenic.h b/drivers/net/ethernet/alteon/acenic.h
index 265fa60..ca5ce0c 100644
--- a/drivers/net/ethernet/alteon/acenic.h
+++ b/drivers/net/ethernet/alteon/acenic.h
@@ -692,7 +692,6 @@ struct ace_private
__attribute__ ((aligned (SMP_CACHE_BYTES)));
u32 last_tx, last_std_rx, last_mini_rx;
#endif
- int pci_using_dac;
u8 firmware_major;
u8 firmware_minor;
u8 firmware_fix;
diff --git a/drivers/net/ethernet/amazon/ena/ena_admin_defs.h b/drivers/net/ethernet/amazon/ena/ena_admin_defs.h
index f5ec35f..466ad947 100644
--- a/drivers/net/ethernet/amazon/ena/ena_admin_defs.h
+++ b/drivers/net/ethernet/amazon/ena/ena_admin_defs.h
@@ -48,6 +48,11 @@ enum ena_admin_aq_feature_id {
ENA_ADMIN_FEATURES_OPCODE_NUM = 32,
};
+/* device capabilities */
+enum ena_admin_aq_caps_id {
+ ENA_ADMIN_ENI_STATS = 0,
+};
+
enum ena_admin_placement_policy_type {
/* descriptors and headers are in host memory */
ENA_ADMIN_PLACEMENT_POLICY_HOST = 1,
@@ -455,7 +460,10 @@ struct ena_admin_device_attr_feature_desc {
*/
u32 supported_features;
- u32 reserved3;
+ /* bitmap of ena_admin_aq_caps_id, which represents device
+ * capabilities.
+ */
+ u32 capabilities;
/* Indicates how many bits are used physical address access. */
u32 phys_addr_width;
diff --git a/drivers/net/ethernet/amazon/ena/ena_com.c b/drivers/net/ethernet/amazon/ena/ena_com.c
index ab413fc..8c8b4c8 100644
--- a/drivers/net/ethernet/amazon/ena/ena_com.c
+++ b/drivers/net/ethernet/amazon/ena/ena_com.c
@@ -1971,6 +1971,7 @@ int ena_com_get_dev_attr_feat(struct ena_com_dev *ena_dev,
sizeof(get_resp.u.dev_attr));
ena_dev->supported_features = get_resp.u.dev_attr.supported_features;
+ ena_dev->capabilities = get_resp.u.dev_attr.capabilities;
if (ena_dev->supported_features & BIT(ENA_ADMIN_MAX_QUEUES_EXT)) {
rc = ena_com_get_feature(ena_dev, &get_resp,
@@ -2223,6 +2224,13 @@ int ena_com_get_eni_stats(struct ena_com_dev *ena_dev,
struct ena_com_stats_ctx ctx;
int ret;
+ if (!ena_com_get_cap(ena_dev, ENA_ADMIN_ENI_STATS)) {
+ netdev_err(ena_dev->net_device,
+ "Capability %d isn't supported\n",
+ ENA_ADMIN_ENI_STATS);
+ return -EOPNOTSUPP;
+ }
+
memset(&ctx, 0x0, sizeof(ctx));
ret = ena_get_dev_stats(ena_dev, &ctx, ENA_ADMIN_GET_STATS_TYPE_ENI);
if (likely(ret == 0))
diff --git a/drivers/net/ethernet/amazon/ena/ena_com.h b/drivers/net/ethernet/amazon/ena/ena_com.h
index 73b03ce..3c5081d 100644
--- a/drivers/net/ethernet/amazon/ena/ena_com.h
+++ b/drivers/net/ethernet/amazon/ena/ena_com.h
@@ -314,6 +314,7 @@ struct ena_com_dev {
struct ena_rss rss;
u32 supported_features;
+ u32 capabilities;
u32 dma_addr_bits;
struct ena_host_attribute host_attr;
@@ -967,6 +968,18 @@ static inline void ena_com_disable_adaptive_moderation(struct ena_com_dev *ena_d
ena_dev->adaptive_coalescing = false;
}
+/* ena_com_get_cap - query whether device supports a capability.
+ * @ena_dev: ENA communication layer struct
+ * @cap_id: enum value representing the capability
+ *
+ * @return - true if capability is supported or false otherwise
+ */
+static inline bool ena_com_get_cap(struct ena_com_dev *ena_dev,
+ enum ena_admin_aq_caps_id cap_id)
+{
+ return !!(ena_dev->capabilities & BIT(cap_id));
+}
+
/* ena_com_update_intr_reg - Prepare interrupt register
* @intr_reg: interrupt register to update.
* @rx_delay_interval: Rx interval in usecs
diff --git a/drivers/net/ethernet/amazon/ena/ena_ethtool.c b/drivers/net/ethernet/amazon/ena/ena_ethtool.c
index 13e745c..39242c5 100644
--- a/drivers/net/ethernet/amazon/ena/ena_ethtool.c
+++ b/drivers/net/ethernet/amazon/ena/ena_ethtool.c
@@ -82,7 +82,7 @@ static const struct ena_stats ena_stats_rx_strings[] = {
ENA_STAT_RX_ENTRY(rx_copybreak_pkt),
ENA_STAT_RX_ENTRY(csum_good),
ENA_STAT_RX_ENTRY(refil_partial),
- ENA_STAT_RX_ENTRY(bad_csum),
+ ENA_STAT_RX_ENTRY(csum_bad),
ENA_STAT_RX_ENTRY(page_alloc_fail),
ENA_STAT_RX_ENTRY(skb_alloc_fail),
ENA_STAT_RX_ENTRY(dma_mapping_err),
@@ -110,8 +110,7 @@ static const struct ena_stats ena_stats_ena_com_strings[] = {
#define ENA_STATS_ARRAY_TX ARRAY_SIZE(ena_stats_tx_strings)
#define ENA_STATS_ARRAY_RX ARRAY_SIZE(ena_stats_rx_strings)
#define ENA_STATS_ARRAY_ENA_COM ARRAY_SIZE(ena_stats_ena_com_strings)
-#define ENA_STATS_ARRAY_ENI(adapter) \
- (ARRAY_SIZE(ena_stats_eni_strings) * (adapter)->eni_stats_supported)
+#define ENA_STATS_ARRAY_ENI(adapter) ARRAY_SIZE(ena_stats_eni_strings)
static void ena_safe_update_stat(u64 *src, u64 *dst,
struct u64_stats_sync *syncp)
@@ -213,8 +212,9 @@ static void ena_get_ethtool_stats(struct net_device *netdev,
u64 *data)
{
struct ena_adapter *adapter = netdev_priv(netdev);
+ struct ena_com_dev *dev = adapter->ena_dev;
- ena_get_stats(adapter, data, adapter->eni_stats_supported);
+ ena_get_stats(adapter, data, ena_com_get_cap(dev, ENA_ADMIN_ENI_STATS));
}
static int ena_get_sw_stats_count(struct ena_adapter *adapter)
@@ -226,7 +226,9 @@ static int ena_get_sw_stats_count(struct ena_adapter *adapter)
static int ena_get_hw_stats_count(struct ena_adapter *adapter)
{
- return ENA_STATS_ARRAY_ENI(adapter);
+ bool supported = ena_com_get_cap(adapter->ena_dev, ENA_ADMIN_ENI_STATS);
+
+ return ENA_STATS_ARRAY_ENI(adapter) * supported;
}
int ena_get_sset_count(struct net_device *netdev, int sset)
@@ -316,10 +318,11 @@ static void ena_get_ethtool_strings(struct net_device *netdev,
u8 *data)
{
struct ena_adapter *adapter = netdev_priv(netdev);
+ struct ena_com_dev *dev = adapter->ena_dev;
switch (sset) {
case ETH_SS_STATS:
- ena_get_strings(adapter, data, adapter->eni_stats_supported);
+ ena_get_strings(adapter, data, ena_com_get_cap(dev, ENA_ADMIN_ENI_STATS));
break;
}
}
@@ -465,7 +468,9 @@ static void ena_get_drvinfo(struct net_device *dev,
}
static void ena_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct ena_adapter *adapter = netdev_priv(netdev);
@@ -476,7 +481,9 @@ static void ena_get_ringparam(struct net_device *netdev,
}
static int ena_set_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct ena_adapter *adapter = netdev_priv(netdev);
u32 new_tx_size, new_rx_size;
diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.c b/drivers/net/ethernet/amazon/ena/ena_netdev.c
index 7d5d885..53080fd 100644
--- a/drivers/net/ethernet/amazon/ena/ena_netdev.c
+++ b/drivers/net/ethernet/amazon/ena/ena_netdev.c
@@ -103,7 +103,7 @@ static void ena_tx_timeout(struct net_device *dev, unsigned int txqueue)
if (test_and_set_bit(ENA_FLAG_TRIGGER_RESET, &adapter->flags))
return;
- adapter->reset_reason = ENA_REGS_RESET_OS_NETDEV_WD;
+ ena_reset_device(adapter, ENA_REGS_RESET_OS_NETDEV_WD);
ena_increase_stat(&adapter->dev_stats.tx_timeout, 1, &adapter->syncp);
netif_err(adapter, tx_err, dev, "Transmit time out\n");
@@ -166,11 +166,9 @@ static int ena_xmit_common(struct net_device *dev,
"Failed to prepare tx bufs\n");
ena_increase_stat(&ring->tx_stats.prepare_ctx_err, 1,
&ring->syncp);
- if (rc != -ENOMEM) {
- adapter->reset_reason =
- ENA_REGS_RESET_DRIVER_INVALID_STATE;
- set_bit(ENA_FLAG_TRIGGER_RESET, &adapter->flags);
- }
+ if (rc != -ENOMEM)
+ ena_reset_device(adapter,
+ ENA_REGS_RESET_DRIVER_INVALID_STATE);
return rc;
}
@@ -434,7 +432,7 @@ static int ena_xdp_execute(struct ena_ring *rx_ring, struct xdp_buff *xdp)
xdp_stat = &rx_ring->rx_stats.xdp_pass;
break;
default:
- bpf_warn_invalid_xdp_action(verdict);
+ bpf_warn_invalid_xdp_action(rx_ring->netdev, xdp_prog, verdict);
xdp_stat = &rx_ring->rx_stats.xdp_invalid;
}
@@ -1269,45 +1267,39 @@ static int handle_invalid_req_id(struct ena_ring *ring, u16 req_id,
netif_err(ring->adapter,
tx_done,
ring->netdev,
- "tx_info doesn't have valid %s",
- is_xdp ? "xdp frame" : "skb");
+ "tx_info doesn't have valid %s. qid %u req_id %u",
+ is_xdp ? "xdp frame" : "skb", ring->qid, req_id);
else
netif_err(ring->adapter,
tx_done,
ring->netdev,
- "Invalid req_id: %hu\n",
- req_id);
+ "Invalid req_id %u in qid %u\n",
+ req_id, ring->qid);
ena_increase_stat(&ring->tx_stats.bad_req_id, 1, &ring->syncp);
+ ena_reset_device(ring->adapter, ENA_REGS_RESET_INV_TX_REQ_ID);
- /* Trigger device reset */
- ring->adapter->reset_reason = ENA_REGS_RESET_INV_TX_REQ_ID;
- set_bit(ENA_FLAG_TRIGGER_RESET, &ring->adapter->flags);
return -EFAULT;
}
static int validate_tx_req_id(struct ena_ring *tx_ring, u16 req_id)
{
- struct ena_tx_buffer *tx_info = NULL;
+ struct ena_tx_buffer *tx_info;
- if (likely(req_id < tx_ring->ring_size)) {
- tx_info = &tx_ring->tx_buffer_info[req_id];
- if (likely(tx_info->skb))
- return 0;
- }
+ tx_info = &tx_ring->tx_buffer_info[req_id];
+ if (likely(tx_info->skb))
+ return 0;
return handle_invalid_req_id(tx_ring, req_id, tx_info, false);
}
static int validate_xdp_req_id(struct ena_ring *xdp_ring, u16 req_id)
{
- struct ena_tx_buffer *tx_info = NULL;
+ struct ena_tx_buffer *tx_info;
- if (likely(req_id < xdp_ring->ring_size)) {
- tx_info = &xdp_ring->tx_buffer_info[req_id];
- if (likely(tx_info->xdpf))
- return 0;
- }
+ tx_info = &xdp_ring->tx_buffer_info[req_id];
+ if (likely(tx_info->xdpf))
+ return 0;
return handle_invalid_req_id(xdp_ring, req_id, tx_info, true);
}
@@ -1332,9 +1324,14 @@ static int ena_clean_tx_irq(struct ena_ring *tx_ring, u32 budget)
rc = ena_com_tx_comp_req_id_get(tx_ring->ena_com_io_cq,
&req_id);
- if (rc)
+ if (rc) {
+ if (unlikely(rc == -EINVAL))
+ handle_invalid_req_id(tx_ring, req_id, NULL,
+ false);
break;
+ }
+ /* validate that the request id points to a valid skb */
rc = validate_tx_req_id(tx_ring, req_id);
if (rc)
break;
@@ -1427,6 +1424,7 @@ static struct sk_buff *ena_rx_skb(struct ena_ring *rx_ring,
u16 *next_to_clean)
{
struct ena_rx_buffer *rx_info;
+ struct ena_adapter *adapter;
u16 len, req_id, buf = 0;
struct sk_buff *skb;
void *page_addr;
@@ -1439,8 +1437,11 @@ static struct sk_buff *ena_rx_skb(struct ena_ring *rx_ring,
rx_info = &rx_ring->rx_buffer_info[req_id];
if (unlikely(!rx_info->page)) {
- netif_err(rx_ring->adapter, rx_err, rx_ring->netdev,
- "Page is NULL\n");
+ adapter = rx_ring->adapter;
+ netif_err(adapter, rx_err, rx_ring->netdev,
+ "Page is NULL. qid %u req_id %u\n", rx_ring->qid, req_id);
+ ena_increase_stat(&rx_ring->rx_stats.bad_req_id, 1, &rx_ring->syncp);
+ ena_reset_device(adapter, ENA_REGS_RESET_INV_RX_REQ_ID);
return NULL;
}
@@ -1550,7 +1551,7 @@ static void ena_rx_checksum(struct ena_ring *rx_ring,
(ena_rx_ctx->l3_csum_err))) {
/* ipv4 checksum error */
skb->ip_summed = CHECKSUM_NONE;
- ena_increase_stat(&rx_ring->rx_stats.bad_csum, 1,
+ ena_increase_stat(&rx_ring->rx_stats.csum_bad, 1,
&rx_ring->syncp);
netif_dbg(rx_ring->adapter, rx_err, rx_ring->netdev,
"RX IPv4 header checksum error\n");
@@ -1562,7 +1563,7 @@ static void ena_rx_checksum(struct ena_ring *rx_ring,
(ena_rx_ctx->l4_proto == ENA_ETH_IO_L4_PROTO_UDP))) {
if (unlikely(ena_rx_ctx->l4_csum_err)) {
/* TCP/UDP checksum error */
- ena_increase_stat(&rx_ring->rx_stats.bad_csum, 1,
+ ena_increase_stat(&rx_ring->rx_stats.csum_bad, 1,
&rx_ring->syncp);
netif_dbg(rx_ring->adapter, rx_err, rx_ring->netdev,
"RX L4 checksum error\n");
@@ -1773,15 +1774,12 @@ static int ena_clean_rx_irq(struct ena_ring *rx_ring, struct napi_struct *napi,
if (rc == -ENOSPC) {
ena_increase_stat(&rx_ring->rx_stats.bad_desc_num, 1,
&rx_ring->syncp);
- adapter->reset_reason = ENA_REGS_RESET_TOO_MANY_RX_DESCS;
+ ena_reset_device(adapter, ENA_REGS_RESET_TOO_MANY_RX_DESCS);
} else {
ena_increase_stat(&rx_ring->rx_stats.bad_req_id, 1,
&rx_ring->syncp);
- adapter->reset_reason = ENA_REGS_RESET_INV_RX_REQ_ID;
+ ena_reset_device(adapter, ENA_REGS_RESET_INV_RX_REQ_ID);
}
-
- set_bit(ENA_FLAG_TRIGGER_RESET, &adapter->flags);
-
return 0;
}
@@ -1896,9 +1894,14 @@ static int ena_clean_xdp_irq(struct ena_ring *xdp_ring, u32 budget)
rc = ena_com_tx_comp_req_id_get(xdp_ring->ena_com_io_cq,
&req_id);
- if (rc)
+ if (rc) {
+ if (unlikely(rc == -EINVAL))
+ handle_invalid_req_id(xdp_ring, req_id, NULL,
+ true);
break;
+ }
+ /* validate that the request id points to a valid xdp_frame */
rc = validate_xdp_req_id(xdp_ring, req_id);
if (rc)
break;
@@ -3240,11 +3243,11 @@ static void ena_config_debug_area(struct ena_adapter *adapter)
int ena_update_hw_stats(struct ena_adapter *adapter)
{
- int rc = 0;
+ int rc;
rc = ena_com_get_eni_stats(adapter->ena_dev, &adapter->eni_stats);
if (rc) {
- dev_info_once(&adapter->pdev->dev, "Failed to get ENI stats\n");
+ netdev_err(adapter->netdev, "Failed to get ENI stats\n");
return rc;
}
@@ -3641,8 +3644,6 @@ static int ena_restore_device(struct ena_adapter *adapter)
mod_timer(&adapter->timer_service, round_jiffies(jiffies + HZ));
adapter->last_keep_alive_jiffies = jiffies;
- dev_err(&pdev->dev, "Device reset completed successfully\n");
-
return rc;
err_disable_msix:
ena_free_mgmnt_irq(adapter);
@@ -3672,6 +3673,8 @@ static void ena_fw_reset_device(struct work_struct *work)
if (likely(test_bit(ENA_FLAG_TRIGGER_RESET, &adapter->flags))) {
ena_destroy_device(adapter, false);
ena_restore_device(adapter);
+
+ dev_err(&adapter->pdev->dev, "Device reset completed successfully\n");
}
rtnl_unlock();
@@ -3694,9 +3697,8 @@ static int check_for_rx_interrupt_queue(struct ena_adapter *adapter,
netif_err(adapter, rx_err, adapter->netdev,
"Potential MSIX issue on Rx side Queue = %d. Reset the device\n",
rx_ring->qid);
- adapter->reset_reason = ENA_REGS_RESET_MISS_INTERRUPT;
- smp_mb__before_atomic();
- set_bit(ENA_FLAG_TRIGGER_RESET, &adapter->flags);
+
+ ena_reset_device(adapter, ENA_REGS_RESET_MISS_INTERRUPT);
return -EIO;
}
@@ -3733,9 +3735,7 @@ static int check_missing_comp_in_tx_queue(struct ena_adapter *adapter,
netif_err(adapter, tx_err, adapter->netdev,
"Potential MSIX issue on Tx side Queue = %d. Reset the device\n",
tx_ring->qid);
- adapter->reset_reason = ENA_REGS_RESET_MISS_INTERRUPT;
- smp_mb__before_atomic();
- set_bit(ENA_FLAG_TRIGGER_RESET, &adapter->flags);
+ ena_reset_device(adapter, ENA_REGS_RESET_MISS_INTERRUPT);
return -EIO;
}
@@ -3761,9 +3761,7 @@ static int check_missing_comp_in_tx_queue(struct ena_adapter *adapter,
"The number of lost tx completions is above the threshold (%d > %d). Reset the device\n",
missed_tx,
adapter->missing_tx_completion_threshold);
- adapter->reset_reason =
- ENA_REGS_RESET_MISS_TX_CMPL;
- set_bit(ENA_FLAG_TRIGGER_RESET, &adapter->flags);
+ ena_reset_device(adapter, ENA_REGS_RESET_MISS_TX_CMPL);
rc = -EIO;
}
@@ -3884,8 +3882,7 @@ static void check_for_missing_keep_alive(struct ena_adapter *adapter)
"Keep alive watchdog timeout.\n");
ena_increase_stat(&adapter->dev_stats.wd_expired, 1,
&adapter->syncp);
- adapter->reset_reason = ENA_REGS_RESET_KEEP_ALIVE_TO;
- set_bit(ENA_FLAG_TRIGGER_RESET, &adapter->flags);
+ ena_reset_device(adapter, ENA_REGS_RESET_KEEP_ALIVE_TO);
}
}
@@ -3896,8 +3893,7 @@ static void check_for_admin_com_state(struct ena_adapter *adapter)
"ENA admin queue is not in running state!\n");
ena_increase_stat(&adapter->dev_stats.admin_q_pause, 1,
&adapter->syncp);
- adapter->reset_reason = ENA_REGS_RESET_ADMIN_TO;
- set_bit(ENA_FLAG_TRIGGER_RESET, &adapter->flags);
+ ena_reset_device(adapter, ENA_REGS_RESET_ADMIN_TO);
}
}
@@ -4013,10 +4009,6 @@ static u32 ena_calc_max_io_queue_num(struct pci_dev *pdev,
max_num_io_queues = min_t(u32, max_num_io_queues, io_tx_cq_num);
/* 1 IRQ for mgmnt and 1 IRQs for each IO direction */
max_num_io_queues = min_t(u32, max_num_io_queues, pci_msix_vec_count(pdev) - 1);
- if (unlikely(!max_num_io_queues)) {
- dev_err(&pdev->dev, "The device doesn't have io queues\n");
- return -EFAULT;
- }
return max_num_io_queues;
}
@@ -4101,7 +4093,7 @@ static int ena_rss_init_default(struct ena_adapter *adapter)
val = ethtool_rxfh_indir_default(i, adapter->num_io_queues);
rc = ena_com_indirect_table_fill_entry(ena_dev, i,
ENA_IO_RXQ_IDX(val));
- if (unlikely(rc && (rc != -EOPNOTSUPP))) {
+ if (unlikely(rc)) {
dev_err(dev, "Cannot fill indirect table\n");
goto err_fill_indir;
}
@@ -4137,10 +4129,11 @@ static void ena_release_bars(struct ena_com_dev *ena_dev, struct pci_dev *pdev)
}
-static int ena_calc_io_queue_size(struct ena_calc_queue_size_ctx *ctx)
+static void ena_calc_io_queue_size(struct ena_adapter *adapter,
+ struct ena_com_dev_get_features_ctx *get_feat_ctx)
{
- struct ena_admin_feature_llq_desc *llq = &ctx->get_feat_ctx->llq;
- struct ena_com_dev *ena_dev = ctx->ena_dev;
+ struct ena_admin_feature_llq_desc *llq = &get_feat_ctx->llq;
+ struct ena_com_dev *ena_dev = adapter->ena_dev;
u32 tx_queue_size = ENA_DEFAULT_RING_SIZE;
u32 rx_queue_size = ENA_DEFAULT_RING_SIZE;
u32 max_tx_queue_size;
@@ -4148,7 +4141,7 @@ static int ena_calc_io_queue_size(struct ena_calc_queue_size_ctx *ctx)
if (ena_dev->supported_features & BIT(ENA_ADMIN_MAX_QUEUES_EXT)) {
struct ena_admin_queue_ext_feature_fields *max_queue_ext =
- &ctx->get_feat_ctx->max_queue_ext.max_queue_ext;
+ &get_feat_ctx->max_queue_ext.max_queue_ext;
max_rx_queue_size = min_t(u32, max_queue_ext->max_rx_cq_depth,
max_queue_ext->max_rx_sq_depth);
max_tx_queue_size = max_queue_ext->max_tx_cq_depth;
@@ -4160,13 +4153,13 @@ static int ena_calc_io_queue_size(struct ena_calc_queue_size_ctx *ctx)
max_tx_queue_size = min_t(u32, max_tx_queue_size,
max_queue_ext->max_tx_sq_depth);
- ctx->max_tx_sgl_size = min_t(u16, ENA_PKT_MAX_BUFS,
- max_queue_ext->max_per_packet_tx_descs);
- ctx->max_rx_sgl_size = min_t(u16, ENA_PKT_MAX_BUFS,
- max_queue_ext->max_per_packet_rx_descs);
+ adapter->max_tx_sgl_size = min_t(u16, ENA_PKT_MAX_BUFS,
+ max_queue_ext->max_per_packet_tx_descs);
+ adapter->max_rx_sgl_size = min_t(u16, ENA_PKT_MAX_BUFS,
+ max_queue_ext->max_per_packet_rx_descs);
} else {
struct ena_admin_queue_feature_desc *max_queues =
- &ctx->get_feat_ctx->max_queues;
+ &get_feat_ctx->max_queues;
max_rx_queue_size = min_t(u32, max_queues->max_cq_depth,
max_queues->max_sq_depth);
max_tx_queue_size = max_queues->max_cq_depth;
@@ -4178,10 +4171,10 @@ static int ena_calc_io_queue_size(struct ena_calc_queue_size_ctx *ctx)
max_tx_queue_size = min_t(u32, max_tx_queue_size,
max_queues->max_sq_depth);
- ctx->max_tx_sgl_size = min_t(u16, ENA_PKT_MAX_BUFS,
- max_queues->max_packet_tx_descs);
- ctx->max_rx_sgl_size = min_t(u16, ENA_PKT_MAX_BUFS,
- max_queues->max_packet_rx_descs);
+ adapter->max_tx_sgl_size = min_t(u16, ENA_PKT_MAX_BUFS,
+ max_queues->max_packet_tx_descs);
+ adapter->max_rx_sgl_size = min_t(u16, ENA_PKT_MAX_BUFS,
+ max_queues->max_packet_rx_descs);
}
max_tx_queue_size = rounddown_pow_of_two(max_tx_queue_size);
@@ -4195,12 +4188,10 @@ static int ena_calc_io_queue_size(struct ena_calc_queue_size_ctx *ctx)
tx_queue_size = rounddown_pow_of_two(tx_queue_size);
rx_queue_size = rounddown_pow_of_two(rx_queue_size);
- ctx->max_tx_queue_size = max_tx_queue_size;
- ctx->max_rx_queue_size = max_rx_queue_size;
- ctx->tx_queue_size = tx_queue_size;
- ctx->rx_queue_size = rx_queue_size;
-
- return 0;
+ adapter->max_tx_ring_size = max_tx_queue_size;
+ adapter->max_rx_ring_size = max_rx_queue_size;
+ adapter->requested_tx_ring_size = tx_queue_size;
+ adapter->requested_rx_ring_size = rx_queue_size;
}
/* ena_probe - Device Initialization Routine
@@ -4215,7 +4206,6 @@ static int ena_calc_io_queue_size(struct ena_calc_queue_size_ctx *ctx)
*/
static int ena_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
{
- struct ena_calc_queue_size_ctx calc_queue_ctx = {};
struct ena_com_dev_get_features_ctx get_feat_ctx;
struct ena_com_dev *ena_dev = NULL;
struct ena_adapter *adapter;
@@ -4300,10 +4290,6 @@ static int ena_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
goto err_device_destroy;
}
- calc_queue_ctx.ena_dev = ena_dev;
- calc_queue_ctx.get_feat_ctx = &get_feat_ctx;
- calc_queue_ctx.pdev = pdev;
-
/* Initial TX and RX interrupt delay. Assumes 1 usec granularity.
* Updated during device initialization with the real granularity
*/
@@ -4311,8 +4297,8 @@ static int ena_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
ena_dev->intr_moder_rx_interval = ENA_INTR_INITIAL_RX_INTERVAL_USECS;
ena_dev->intr_delay_resolution = ENA_DEFAULT_INTR_DELAY_RESOLUTION;
max_num_io_queues = ena_calc_max_io_queue_num(pdev, ena_dev, &get_feat_ctx);
- rc = ena_calc_io_queue_size(&calc_queue_ctx);
- if (rc || !max_num_io_queues) {
+ ena_calc_io_queue_size(adapter, &get_feat_ctx);
+ if (unlikely(!max_num_io_queues)) {
rc = -EFAULT;
goto err_device_destroy;
}
@@ -4321,13 +4307,6 @@ static int ena_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
adapter->reset_reason = ENA_REGS_RESET_NORMAL;
- adapter->requested_tx_ring_size = calc_queue_ctx.tx_queue_size;
- adapter->requested_rx_ring_size = calc_queue_ctx.rx_queue_size;
- adapter->max_tx_ring_size = calc_queue_ctx.max_tx_queue_size;
- adapter->max_rx_ring_size = calc_queue_ctx.max_rx_queue_size;
- adapter->max_tx_sgl_size = calc_queue_ctx.max_tx_sgl_size;
- adapter->max_rx_sgl_size = calc_queue_ctx.max_rx_sgl_size;
-
adapter->num_io_queues = max_num_io_queues;
adapter->max_num_io_queues = max_num_io_queues;
adapter->last_monitored_tx_qid = 0;
@@ -4378,11 +4357,6 @@ static int ena_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
ena_config_debug_area(adapter);
- if (!ena_update_hw_stats(adapter))
- adapter->eni_stats_supported = true;
- else
- adapter->eni_stats_supported = false;
-
memcpy(adapter->netdev->perm_addr, adapter->mac_addr, netdev->addr_len);
netif_carrier_off(netdev);
diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.h b/drivers/net/ethernet/amazon/ena/ena_netdev.h
index 0c39fc2..1bdce99 100644
--- a/drivers/net/ethernet/amazon/ena/ena_netdev.h
+++ b/drivers/net/ethernet/amazon/ena/ena_netdev.h
@@ -14,6 +14,7 @@
#include <linux/interrupt.h>
#include <linux/netdevice.h>
#include <linux/skbuff.h>
+#include <uapi/linux/bpf.h>
#include "ena_com.h"
#include "ena_eth_com.h"
@@ -139,18 +140,6 @@ struct ena_napi {
struct dim dim;
};
-struct ena_calc_queue_size_ctx {
- struct ena_com_dev_get_features_ctx *get_feat_ctx;
- struct ena_com_dev *ena_dev;
- struct pci_dev *pdev;
- u32 tx_queue_size;
- u32 rx_queue_size;
- u32 max_tx_queue_size;
- u32 max_rx_queue_size;
- u16 max_tx_sgl_size;
- u16 max_rx_sgl_size;
-};
-
struct ena_tx_buffer {
struct sk_buff *skb;
/* num of ena desc for this specific skb
@@ -215,7 +204,7 @@ struct ena_stats_rx {
u64 rx_copybreak_pkt;
u64 csum_good;
u64 refil_partial;
- u64 bad_csum;
+ u64 csum_bad;
u64 page_alloc_fail;
u64 skb_alloc_fail;
u64 dma_mapping_err;
@@ -378,7 +367,6 @@ struct ena_adapter {
struct u64_stats_sync syncp;
struct ena_stats_dev dev_stats;
struct ena_admin_eni_stats eni_stats;
- bool eni_stats_supported;
/* last queue index that was checked for uncompleted tx packets */
u32 last_monitored_tx_qid;
@@ -406,6 +394,15 @@ int ena_update_queue_count(struct ena_adapter *adapter, u32 new_channel_count);
int ena_get_sset_count(struct net_device *netdev, int sset);
+static inline void ena_reset_device(struct ena_adapter *adapter,
+ enum ena_regs_reset_reason_types reset_reason)
+{
+ adapter->reset_reason = reset_reason;
+ /* Make sure reset reason is set before triggering the reset */
+ smp_mb__before_atomic();
+ set_bit(ENA_FLAG_TRIGGER_RESET, &adapter->flags);
+}
+
enum ena_xdp_errors_t {
ENA_XDP_ALLOWED = 0,
ENA_XDP_CURRENT_MTU_TOO_LARGE,
diff --git a/drivers/net/ethernet/amd/a2065.c b/drivers/net/ethernet/amd/a2065.c
index 2f808dbc8..3a351d3 100644
--- a/drivers/net/ethernet/amd/a2065.c
+++ b/drivers/net/ethernet/amd/a2065.c
@@ -680,6 +680,7 @@ static int a2065_init_one(struct zorro_dev *z,
unsigned long base_addr = board + A2065_LANCE;
unsigned long mem_start = board + A2065_RAM;
struct resource *r1, *r2;
+ u8 addr[ETH_ALEN];
u32 serial;
int err;
@@ -706,17 +707,18 @@ static int a2065_init_one(struct zorro_dev *z,
r2->name = dev->name;
serial = be32_to_cpu(z->rom.er_SerialNumber);
- dev->dev_addr[0] = 0x00;
+ addr[0] = 0x00;
if (z->id != ZORRO_PROD_AMERISTAR_A2065) { /* Commodore */
- dev->dev_addr[1] = 0x80;
- dev->dev_addr[2] = 0x10;
+ addr[1] = 0x80;
+ addr[2] = 0x10;
} else { /* Ameristar */
- dev->dev_addr[1] = 0x00;
- dev->dev_addr[2] = 0x9f;
+ addr[1] = 0x00;
+ addr[2] = 0x9f;
}
- dev->dev_addr[3] = (serial >> 16) & 0xff;
- dev->dev_addr[4] = (serial >> 8) & 0xff;
- dev->dev_addr[5] = serial & 0xff;
+ addr[3] = (serial >> 16) & 0xff;
+ addr[4] = (serial >> 8) & 0xff;
+ addr[5] = serial & 0xff;
+ eth_hw_addr_set(dev, addr);
dev->base_addr = (unsigned long)ZTWO_VADDR(base_addr);
dev->mem_start = (unsigned long)ZTWO_VADDR(mem_start);
dev->mem_end = dev->mem_start + A2065_RAM_SIZE;
diff --git a/drivers/net/ethernet/amd/ariadne.c b/drivers/net/ethernet/amd/ariadne.c
index 5e0f645..4ea7b9f 100644
--- a/drivers/net/ethernet/amd/ariadne.c
+++ b/drivers/net/ethernet/amd/ariadne.c
@@ -441,11 +441,11 @@ static int ariadne_open(struct net_device *dev)
/* Set the Ethernet Hardware Address */
lance->RAP = CSR12; /* Physical Address Register, PADR[15:0] */
- lance->RDP = ((u_short *)&dev->dev_addr[0])[0];
+ lance->RDP = ((const u_short *)&dev->dev_addr[0])[0];
lance->RAP = CSR13; /* Physical Address Register, PADR[31:16] */
- lance->RDP = ((u_short *)&dev->dev_addr[0])[1];
+ lance->RDP = ((const u_short *)&dev->dev_addr[0])[1];
lance->RAP = CSR14; /* Physical Address Register, PADR[47:32] */
- lance->RDP = ((u_short *)&dev->dev_addr[0])[2];
+ lance->RDP = ((const u_short *)&dev->dev_addr[0])[2];
/* Set the Init Block Mode */
lance->RAP = CSR15; /* Mode Register */
@@ -717,6 +717,7 @@ static int ariadne_init_one(struct zorro_dev *z,
unsigned long mem_start = board + ARIADNE_RAM;
struct resource *r1, *r2;
struct net_device *dev;
+ u8 addr[ETH_ALEN];
u32 serial;
int err;
@@ -740,12 +741,13 @@ static int ariadne_init_one(struct zorro_dev *z,
r2->name = dev->name;
serial = be32_to_cpu(z->rom.er_SerialNumber);
- dev->dev_addr[0] = 0x00;
- dev->dev_addr[1] = 0x60;
- dev->dev_addr[2] = 0x30;
- dev->dev_addr[3] = (serial >> 16) & 0xff;
- dev->dev_addr[4] = (serial >> 8) & 0xff;
- dev->dev_addr[5] = serial & 0xff;
+ addr[0] = 0x00;
+ addr[1] = 0x60;
+ addr[2] = 0x30;
+ addr[3] = (serial >> 16) & 0xff;
+ addr[4] = (serial >> 8) & 0xff;
+ addr[5] = serial & 0xff;
+ eth_hw_addr_set(dev, addr);
dev->base_addr = (unsigned long)ZTWO_VADDR(base_addr);
dev->mem_start = (unsigned long)ZTWO_VADDR(mem_start);
dev->mem_end = dev->mem_start + ARIADNE_RAM_SIZE;
diff --git a/drivers/net/ethernet/amd/atarilance.c b/drivers/net/ethernet/amd/atarilance.c
index 9c7d969..2786916 100644
--- a/drivers/net/ethernet/amd/atarilance.c
+++ b/drivers/net/ethernet/amd/atarilance.c
@@ -471,6 +471,7 @@ static unsigned long __init lance_probe1( struct net_device *dev,
int i;
static int did_version;
unsigned short save1, save2;
+ u8 addr[ETH_ALEN];
PROBE_PRINT(( "Probing for Lance card at mem %#lx io %#lx\n",
(long)memaddr, (long)ioaddr ));
@@ -585,14 +586,16 @@ static unsigned long __init lance_probe1( struct net_device *dev,
eth_hw_addr_set(dev, OldRieblDefHwaddr);
break;
case NEW_RIEBL:
- lp->memcpy_f(dev->dev_addr, RIEBL_HWADDR_ADDR, ETH_ALEN);
+ lp->memcpy_f(addr, RIEBL_HWADDR_ADDR, ETH_ALEN);
+ eth_hw_addr_set(dev, addr);
break;
case PAM_CARD:
i = IO->eeprom;
for( i = 0; i < 6; ++i )
- dev->dev_addr[i] =
+ addr[i] =
((((unsigned short *)MEM)[i*2] & 0x0f) << 4) |
((((unsigned short *)MEM)[i*2+1] & 0x0f));
+ eth_hw_addr_set(dev, addr);
i = IO->mem;
break;
}
diff --git a/drivers/net/ethernet/amd/hplance.c b/drivers/net/ethernet/amd/hplance.c
index 6784f87..055fda1 100644
--- a/drivers/net/ethernet/amd/hplance.c
+++ b/drivers/net/ethernet/amd/hplance.c
@@ -129,6 +129,7 @@ static void hplance_init(struct net_device *dev, struct dio_dev *d)
{
unsigned long va = (d->resource.start + DIO_VIRADDRBASE);
struct hplance_private *lp;
+ u8 addr[ETH_ALEN];
int i;
/* reset the board */
@@ -144,9 +145,10 @@ static void hplance_init(struct net_device *dev, struct dio_dev *d)
/* The NVRAM holds our ethernet address, one nibble per byte,
* at bytes NVRAMOFF+1,3,5,7,9...
*/
- dev->dev_addr[i] = ((in_8(va + HPLANCE_NVRAMOFF + i*4 + 1) & 0xF) << 4)
+ addr[i] = ((in_8(va + HPLANCE_NVRAMOFF + i*4 + 1) & 0xF) << 4)
| (in_8(va + HPLANCE_NVRAMOFF + i*4 + 3) & 0xF);
}
+ eth_hw_addr_set(dev, addr);
lp = netdev_priv(dev);
lp->lance.name = d->name;
diff --git a/drivers/net/ethernet/amd/lance.c b/drivers/net/ethernet/amd/lance.c
index 945bf1d..4620166 100644
--- a/drivers/net/ethernet/amd/lance.c
+++ b/drivers/net/ethernet/amd/lance.c
@@ -480,6 +480,7 @@ static int __init lance_probe1(struct net_device *dev, int ioaddr, int irq, int
unsigned long flags;
int err = -ENOMEM;
void __iomem *bios;
+ u8 addr[ETH_ALEN];
/* First we look for special cases.
Check for HP's on-board ethernet by looking for 'HP' in the BIOS.
@@ -541,7 +542,8 @@ static int __init lance_probe1(struct net_device *dev, int ioaddr, int irq, int
/* There is a 16 byte station address PROM at the base address.
The first six bytes are the station address. */
for (i = 0; i < 6; i++)
- dev->dev_addr[i] = inb(ioaddr + i);
+ addr[i] = inb(ioaddr + i);
+ eth_hw_addr_set(dev, addr);
printk("%pM", dev->dev_addr);
dev->base_addr = ioaddr;
diff --git a/drivers/net/ethernet/amd/mvme147.c b/drivers/net/ethernet/amd/mvme147.c
index da97fcc..410c7b6 100644
--- a/drivers/net/ethernet/amd/mvme147.c
+++ b/drivers/net/ethernet/amd/mvme147.c
@@ -74,6 +74,7 @@ static struct net_device * __init mvme147lance_probe(void)
static int called;
static const char name[] = "MVME147 LANCE";
struct m147lance_private *lp;
+ u8 macaddr[ETH_ALEN];
u_long *addr;
u_long address;
int err;
@@ -93,15 +94,16 @@ static struct net_device * __init mvme147lance_probe(void)
addr = (u_long *)ETHERNET_ADDRESS;
address = *addr;
- dev->dev_addr[0] = 0x08;
- dev->dev_addr[1] = 0x00;
- dev->dev_addr[2] = 0x3e;
+ macaddr[0] = 0x08;
+ macaddr[1] = 0x00;
+ macaddr[2] = 0x3e;
address = address >> 8;
- dev->dev_addr[5] = address&0xff;
+ macaddr[5] = address&0xff;
address = address >> 8;
- dev->dev_addr[4] = address&0xff;
+ macaddr[4] = address&0xff;
address = address >> 8;
- dev->dev_addr[3] = address&0xff;
+ macaddr[3] = address&0xff;
+ eth_hw_addr_set(dev, macaddr);
printk("%s: MVME147 at 0x%08lx, irq %d, Hardware Address %pM\n",
dev->name, dev->base_addr, MVME147_LANCE_IRQ,
diff --git a/drivers/net/ethernet/amd/ni65.c b/drivers/net/ethernet/amd/ni65.c
index 032e892..8ba579b 100644
--- a/drivers/net/ethernet/amd/ni65.c
+++ b/drivers/net/ethernet/amd/ni65.c
@@ -251,7 +251,7 @@ static void ni65_recv_intr(struct net_device *dev,int);
static void ni65_xmit_intr(struct net_device *dev,int);
static int ni65_open(struct net_device *dev);
static int ni65_lance_reinit(struct net_device *dev);
-static void ni65_init_lance(struct priv *p,unsigned char*,int,int);
+static void ni65_init_lance(struct priv *p,const unsigned char*,int,int);
static netdev_tx_t ni65_send_packet(struct sk_buff *skb,
struct net_device *dev);
static void ni65_timeout(struct net_device *dev, unsigned int txqueue);
@@ -418,6 +418,7 @@ static int __init ni65_probe1(struct net_device *dev,int ioaddr)
{
int i,j;
struct priv *p;
+ u8 addr[ETH_ALEN];
unsigned long flags;
dev->irq = irq;
@@ -444,7 +445,8 @@ static int __init ni65_probe1(struct net_device *dev,int ioaddr)
return -ENODEV;
for(j=0;j<6;j++)
- dev->dev_addr[j] = inb(ioaddr+cards[i].addr_offset+j);
+ addr[j] = inb(ioaddr+cards[i].addr_offset+j);
+ eth_hw_addr_set(dev, addr);
if( (j=ni65_alloc_buffer(dev)) < 0) {
release_region(ioaddr, cards[i].total_size);
@@ -566,7 +568,7 @@ static int __init ni65_probe1(struct net_device *dev,int ioaddr)
/*
* set lance register and trigger init
*/
-static void ni65_init_lance(struct priv *p,unsigned char *daddr,int filter,int mode)
+static void ni65_init_lance(struct priv *p,const unsigned char *daddr,int filter,int mode)
{
int i;
u32 pib;
diff --git a/drivers/net/ethernet/amd/pcnet32.c b/drivers/net/ethernet/amd/pcnet32.c
index f5c50ff..c20c369 100644
--- a/drivers/net/ethernet/amd/pcnet32.c
+++ b/drivers/net/ethernet/amd/pcnet32.c
@@ -860,7 +860,9 @@ static int pcnet32_nway_reset(struct net_device *dev)
}
static void pcnet32_get_ringparam(struct net_device *dev,
- struct ethtool_ringparam *ering)
+ struct ethtool_ringparam *ering,
+ struct kernel_ethtool_ringparam *kernel_ering,
+ struct netlink_ext_ack *extack)
{
struct pcnet32_private *lp = netdev_priv(dev);
@@ -871,7 +873,9 @@ static void pcnet32_get_ringparam(struct net_device *dev,
}
static int pcnet32_set_ringparam(struct net_device *dev,
- struct ethtool_ringparam *ering)
+ struct ethtool_ringparam *ering,
+ struct kernel_ethtool_ringparam *kernel_ering,
+ struct netlink_ext_ack *extack)
{
struct pcnet32_private *lp = netdev_priv(dev);
unsigned long flags;
diff --git a/drivers/net/ethernet/amd/xgbe/xgbe-common.h b/drivers/net/ethernet/amd/xgbe/xgbe-common.h
index 533b8519..466273b 100644
--- a/drivers/net/ethernet/amd/xgbe/xgbe-common.h
+++ b/drivers/net/ethernet/amd/xgbe/xgbe-common.h
@@ -898,6 +898,8 @@
#define PCS_V2_WINDOW_SELECT 0x9064
#define PCS_V2_RV_WINDOW_DEF 0x1060
#define PCS_V2_RV_WINDOW_SELECT 0x1064
+#define PCS_V2_YC_WINDOW_DEF 0x18060
+#define PCS_V2_YC_WINDOW_SELECT 0x18064
/* PCS register entry bit positions and sizes */
#define PCS_V2_WINDOW_DEF_OFFSET_INDEX 6
@@ -1030,8 +1032,8 @@
#define XP_PROP_0_PORT_ID_WIDTH 8
#define XP_PROP_0_PORT_MODE_INDEX 8
#define XP_PROP_0_PORT_MODE_WIDTH 4
-#define XP_PROP_0_PORT_SPEEDS_INDEX 23
-#define XP_PROP_0_PORT_SPEEDS_WIDTH 4
+#define XP_PROP_0_PORT_SPEEDS_INDEX 22
+#define XP_PROP_0_PORT_SPEEDS_WIDTH 5
#define XP_PROP_1_MAX_RX_DMA_INDEX 24
#define XP_PROP_1_MAX_RX_DMA_WIDTH 5
#define XP_PROP_1_MAX_RX_QUEUES_INDEX 8
diff --git a/drivers/net/ethernet/amd/xgbe/xgbe-drv.c b/drivers/net/ethernet/amd/xgbe/xgbe-drv.c
index 30d24d1..492ac38 100644
--- a/drivers/net/ethernet/amd/xgbe/xgbe-drv.c
+++ b/drivers/net/ethernet/amd/xgbe/xgbe-drv.c
@@ -1508,9 +1508,6 @@ static int xgbe_set_hwtstamp_settings(struct xgbe_prv_data *pdata,
if (copy_from_user(&config, ifreq->ifr_data, sizeof(config)))
return -EFAULT;
- if (config.flags)
- return -EINVAL;
-
mac_tscr = 0;
switch (config.tx_type) {
diff --git a/drivers/net/ethernet/amd/xgbe/xgbe-ethtool.c b/drivers/net/ethernet/amd/xgbe/xgbe-ethtool.c
index 94879cf..6ceb1cd 100644
--- a/drivers/net/ethernet/amd/xgbe/xgbe-ethtool.c
+++ b/drivers/net/ethernet/amd/xgbe/xgbe-ethtool.c
@@ -619,8 +619,11 @@ static int xgbe_get_module_eeprom(struct net_device *netdev,
return pdata->phy_if.module_eeprom(pdata, eeprom, data);
}
-static void xgbe_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ringparam)
+static void
+xgbe_get_ringparam(struct net_device *netdev,
+ struct ethtool_ringparam *ringparam,
+ struct kernel_ethtool_ringparam *kernel_ringparam,
+ struct netlink_ext_ack *extack)
{
struct xgbe_prv_data *pdata = netdev_priv(netdev);
@@ -631,7 +634,9 @@ static void xgbe_get_ringparam(struct net_device *netdev,
}
static int xgbe_set_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ringparam)
+ struct ethtool_ringparam *ringparam,
+ struct kernel_ethtool_ringparam *kernel_ringparam,
+ struct netlink_ext_ack *extack)
{
struct xgbe_prv_data *pdata = netdev_priv(netdev);
unsigned int rx, tx;
diff --git a/drivers/net/ethernet/amd/xgbe/xgbe-pci.c b/drivers/net/ethernet/amd/xgbe/xgbe-pci.c
index 90cb55e..efdcf48 100644
--- a/drivers/net/ethernet/amd/xgbe/xgbe-pci.c
+++ b/drivers/net/ethernet/amd/xgbe/xgbe-pci.c
@@ -278,6 +278,13 @@ static int xgbe_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)
(rdev->vendor == PCI_VENDOR_ID_AMD) && (rdev->device == 0x15d0)) {
pdata->xpcs_window_def_reg = PCS_V2_RV_WINDOW_DEF;
pdata->xpcs_window_sel_reg = PCS_V2_RV_WINDOW_SELECT;
+ } else if (rdev && (rdev->vendor == PCI_VENDOR_ID_AMD) &&
+ (rdev->device == 0x14b5)) {
+ pdata->xpcs_window_def_reg = PCS_V2_YC_WINDOW_DEF;
+ pdata->xpcs_window_sel_reg = PCS_V2_YC_WINDOW_SELECT;
+
+ /* Yellow Carp devices do not need cdr workaround */
+ pdata->vdata->an_cdr_workaround = 0;
} else {
pdata->xpcs_window_def_reg = PCS_V2_WINDOW_DEF;
pdata->xpcs_window_sel_reg = PCS_V2_WINDOW_SELECT;
@@ -460,7 +467,7 @@ static int __maybe_unused xgbe_pci_resume(struct device *dev)
return ret;
}
-static const struct xgbe_version_data xgbe_v2a = {
+static struct xgbe_version_data xgbe_v2a = {
.init_function_ptrs_phy_impl = xgbe_init_function_ptrs_phy_v2,
.xpcs_access = XGBE_XPCS_ACCESS_V2,
.mmc_64bit = 1,
@@ -475,7 +482,7 @@ static const struct xgbe_version_data xgbe_v2a = {
.an_cdr_workaround = 1,
};
-static const struct xgbe_version_data xgbe_v2b = {
+static struct xgbe_version_data xgbe_v2b = {
.init_function_ptrs_phy_impl = xgbe_init_function_ptrs_phy_v2,
.xpcs_access = XGBE_XPCS_ACCESS_V2,
.mmc_64bit = 1,
diff --git a/drivers/net/ethernet/amd/xgbe/xgbe-phy-v2.c b/drivers/net/ethernet/amd/xgbe/xgbe-phy-v2.c
index 2137690..2156600 100644
--- a/drivers/net/ethernet/amd/xgbe/xgbe-phy-v2.c
+++ b/drivers/net/ethernet/amd/xgbe/xgbe-phy-v2.c
@@ -124,10 +124,10 @@
#include "xgbe.h"
#include "xgbe-common.h"
-#define XGBE_PHY_PORT_SPEED_100 BIT(0)
-#define XGBE_PHY_PORT_SPEED_1000 BIT(1)
-#define XGBE_PHY_PORT_SPEED_2500 BIT(2)
-#define XGBE_PHY_PORT_SPEED_10000 BIT(3)
+#define XGBE_PHY_PORT_SPEED_100 BIT(1)
+#define XGBE_PHY_PORT_SPEED_1000 BIT(2)
+#define XGBE_PHY_PORT_SPEED_2500 BIT(3)
+#define XGBE_PHY_PORT_SPEED_10000 BIT(4)
#define XGBE_MUTEX_RELEASE 0x80000000
diff --git a/drivers/net/ethernet/apm/xgene/xgene_enet_main.c b/drivers/net/ethernet/apm/xgene/xgene_enet_main.c
index 220dc42..ff2d099 100644
--- a/drivers/net/ethernet/apm/xgene/xgene_enet_main.c
+++ b/drivers/net/ethernet/apm/xgene/xgene_enet_main.c
@@ -869,7 +869,7 @@ static void xgene_enet_timeout(struct net_device *ndev, unsigned int txqueue)
for (i = 0; i < pdata->txq_cnt; i++) {
txq = netdev_get_tx_queue(ndev, i);
- txq->trans_start = jiffies;
+ txq_trans_cond_update(txq);
netif_tx_start_queue(txq);
}
}
diff --git a/drivers/net/ethernet/apple/macmace.c b/drivers/net/ethernet/apple/macmace.c
index 95d3061..8fcaf16 100644
--- a/drivers/net/ethernet/apple/macmace.c
+++ b/drivers/net/ethernet/apple/macmace.c
@@ -92,7 +92,7 @@ static void mace_reset(struct net_device *dev);
static irqreturn_t mace_interrupt(int irq, void *dev_id);
static irqreturn_t mace_dma_intr(int irq, void *dev_id);
static void mace_tx_timeout(struct net_device *dev, unsigned int txqueue);
-static void __mace_set_address(struct net_device *dev, void *addr);
+static void __mace_set_address(struct net_device *dev, const void *addr);
/*
* Load a receive DMA channel with a base address and ring length
@@ -197,6 +197,7 @@ static int mace_probe(struct platform_device *pdev)
unsigned char *addr;
struct net_device *dev;
unsigned char checksum = 0;
+ u8 macaddr[ETH_ALEN];
int err;
dev = alloc_etherdev(PRIV_BYTES);
@@ -229,8 +230,9 @@ static int mace_probe(struct platform_device *pdev)
for (j = 0; j < 6; ++j) {
u8 v = bitrev8(addr[j<<4]);
checksum ^= v;
- dev->dev_addr[j] = v;
+ macaddr[j] = v;
}
+ eth_hw_addr_set(dev, macaddr);
for (; j < 8; ++j) {
checksum ^= bitrev8(addr[j<<4]);
}
@@ -315,11 +317,12 @@ static void mace_reset(struct net_device *dev)
* Load the address on a mace controller.
*/
-static void __mace_set_address(struct net_device *dev, void *addr)
+static void __mace_set_address(struct net_device *dev, const void *addr)
{
struct mace_data *mp = netdev_priv(dev);
volatile struct mace *mb = mp->mace;
- unsigned char *p = addr;
+ const unsigned char *p = addr;
+ u8 macaddr[ETH_ALEN];
int i;
/* load up the hardware address */
@@ -331,7 +334,8 @@ static void __mace_set_address(struct net_device *dev, void *addr)
;
}
for (i = 0; i < 6; ++i)
- mb->padr = dev->dev_addr[i] = p[i];
+ mb->padr = macaddr[i] = p[i];
+ eth_hw_addr_set(dev, macaddr);
if (mp->chipid != BROKEN_ADDRCHG_REV)
mb->iac = 0;
}
diff --git a/drivers/net/ethernet/aquantia/atlantic/aq_ethtool.c b/drivers/net/ethernet/aquantia/atlantic/aq_ethtool.c
index a9ef054..a418238 100644
--- a/drivers/net/ethernet/aquantia/atlantic/aq_ethtool.c
+++ b/drivers/net/ethernet/aquantia/atlantic/aq_ethtool.c
@@ -812,7 +812,9 @@ static int aq_ethtool_set_pauseparam(struct net_device *ndev,
}
static void aq_get_ringparam(struct net_device *ndev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct aq_nic_s *aq_nic = netdev_priv(ndev);
struct aq_nic_cfg_s *cfg;
@@ -827,7 +829,9 @@ static void aq_get_ringparam(struct net_device *ndev,
}
static int aq_set_ringparam(struct net_device *ndev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct aq_nic_s *aq_nic = netdev_priv(ndev);
const struct aq_hw_caps_s *hw_caps;
diff --git a/drivers/net/ethernet/aquantia/atlantic/aq_main.c b/drivers/net/ethernet/aquantia/atlantic/aq_main.c
index e22935c..e65ce71 100644
--- a/drivers/net/ethernet/aquantia/atlantic/aq_main.c
+++ b/drivers/net/ethernet/aquantia/atlantic/aq_main.c
@@ -231,9 +231,6 @@ static void aq_ndev_set_multicast_settings(struct net_device *ndev)
static int aq_ndev_config_hwtstamp(struct aq_nic_s *aq_nic,
struct hwtstamp_config *config)
{
- if (config->flags)
- return -EINVAL;
-
switch (config->tx_type) {
case HWTSTAMP_TX_OFF:
case HWTSTAMP_TX_ON:
diff --git a/drivers/net/ethernet/aquantia/atlantic/aq_ring.c b/drivers/net/ethernet/aquantia/atlantic/aq_ring.c
index 81b3756..77e76c9 100644
--- a/drivers/net/ethernet/aquantia/atlantic/aq_ring.c
+++ b/drivers/net/ethernet/aquantia/atlantic/aq_ring.c
@@ -366,6 +366,10 @@ int aq_ring_rx_clean(struct aq_ring_s *self,
if (!buff->is_eop) {
buff_ = buff;
do {
+ if (buff_->next >= self->size) {
+ err = -EIO;
+ goto err_exit;
+ }
next_ = buff_->next,
buff_ = &self->buff_ring[next_];
is_rsc_completed =
@@ -389,6 +393,10 @@ int aq_ring_rx_clean(struct aq_ring_s *self,
(buff->is_lro && buff->is_cso_err)) {
buff_ = buff;
do {
+ if (buff_->next >= self->size) {
+ err = -EIO;
+ goto err_exit;
+ }
next_ = buff_->next,
buff_ = &self->buff_ring[next_];
diff --git a/drivers/net/ethernet/asix/ax88796c_main.c b/drivers/net/ethernet/asix/ax88796c_main.c
index e230d8d..e7a9f98 100644
--- a/drivers/net/ethernet/asix/ax88796c_main.c
+++ b/drivers/net/ethernet/asix/ax88796c_main.c
@@ -144,12 +144,13 @@ static void ax88796c_set_mac_addr(struct net_device *ndev)
static void ax88796c_load_mac_addr(struct net_device *ndev)
{
struct ax88796c_device *ax_local = to_ax88796c_device(ndev);
+ u8 addr[ETH_ALEN];
u16 temp;
lockdep_assert_held(&ax_local->spi_lock);
/* Try the device tree first */
- if (!eth_platform_get_mac_address(&ax_local->spi->dev, ndev->dev_addr) &&
+ if (!platform_get_ethdev_address(&ax_local->spi->dev, ndev) &&
is_valid_ether_addr(ndev->dev_addr)) {
if (netif_msg_probe(ax_local))
dev_info(&ax_local->spi->dev,
@@ -159,18 +160,19 @@ static void ax88796c_load_mac_addr(struct net_device *ndev)
/* Read the MAC address from AX88796C */
temp = AX_READ(&ax_local->ax_spi, P3_MACASR0);
- ndev->dev_addr[5] = (u8)temp;
- ndev->dev_addr[4] = (u8)(temp >> 8);
+ addr[5] = (u8)temp;
+ addr[4] = (u8)(temp >> 8);
temp = AX_READ(&ax_local->ax_spi, P3_MACASR1);
- ndev->dev_addr[3] = (u8)temp;
- ndev->dev_addr[2] = (u8)(temp >> 8);
+ addr[3] = (u8)temp;
+ addr[2] = (u8)(temp >> 8);
temp = AX_READ(&ax_local->ax_spi, P3_MACASR2);
- ndev->dev_addr[1] = (u8)temp;
- ndev->dev_addr[0] = (u8)(temp >> 8);
+ addr[1] = (u8)temp;
+ addr[0] = (u8)(temp >> 8);
- if (is_valid_ether_addr(ndev->dev_addr)) {
+ if (is_valid_ether_addr(addr)) {
+ eth_hw_addr_set(ndev, addr);
if (netif_msg_probe(ax_local))
dev_info(&ax_local->spi->dev,
"MAC address read from ASIX chip\n");
diff --git a/drivers/net/ethernet/atheros/ag71xx.c b/drivers/net/ethernet/atheros/ag71xx.c
index 88d2ab7..ec167af 100644
--- a/drivers/net/ethernet/atheros/ag71xx.c
+++ b/drivers/net/ethernet/atheros/ag71xx.c
@@ -766,7 +766,7 @@ static bool ag71xx_check_dma_stuck(struct ag71xx *ag)
unsigned long timestamp;
u32 rx_sm, tx_sm, rx_fd;
- timestamp = netdev_get_tx_queue(ag->ndev, 0)->trans_start;
+ timestamp = READ_ONCE(netdev_get_tx_queue(ag->ndev, 0)->trans_start);
if (likely(time_before(jiffies, timestamp + HZ / 10)))
return false;
@@ -1024,83 +1024,6 @@ static void ag71xx_mac_config(struct phylink_config *config, unsigned int mode,
ag71xx_wr(ag, AG71XX_REG_FIFO_CFG3, ag->fifodata[2]);
}
-static void ag71xx_mac_validate(struct phylink_config *config,
- unsigned long *supported,
- struct phylink_link_state *state)
-{
- struct ag71xx *ag = netdev_priv(to_net_dev(config->dev));
- __ETHTOOL_DECLARE_LINK_MODE_MASK(mask) = { 0, };
-
- switch (state->interface) {
- case PHY_INTERFACE_MODE_NA:
- break;
- case PHY_INTERFACE_MODE_MII:
- if ((ag71xx_is(ag, AR9330) && ag->mac_idx == 0) ||
- ag71xx_is(ag, AR9340) ||
- ag71xx_is(ag, QCA9530) ||
- (ag71xx_is(ag, QCA9550) && ag->mac_idx == 1))
- break;
- goto unsupported;
- case PHY_INTERFACE_MODE_GMII:
- if ((ag71xx_is(ag, AR9330) && ag->mac_idx == 1) ||
- (ag71xx_is(ag, AR9340) && ag->mac_idx == 1) ||
- (ag71xx_is(ag, QCA9530) && ag->mac_idx == 1))
- break;
- goto unsupported;
- case PHY_INTERFACE_MODE_SGMII:
- if (ag71xx_is(ag, QCA9550) && ag->mac_idx == 0)
- break;
- goto unsupported;
- case PHY_INTERFACE_MODE_RMII:
- if (ag71xx_is(ag, AR9340) && ag->mac_idx == 0)
- break;
- goto unsupported;
- case PHY_INTERFACE_MODE_RGMII:
- if ((ag71xx_is(ag, AR9340) && ag->mac_idx == 0) ||
- (ag71xx_is(ag, QCA9550) && ag->mac_idx == 1))
- break;
- goto unsupported;
- default:
- goto unsupported;
- }
-
- phylink_set(mask, MII);
-
- phylink_set(mask, Pause);
- phylink_set(mask, Asym_Pause);
- phylink_set(mask, Autoneg);
- phylink_set(mask, 10baseT_Half);
- phylink_set(mask, 10baseT_Full);
- phylink_set(mask, 100baseT_Half);
- phylink_set(mask, 100baseT_Full);
-
- if (state->interface == PHY_INTERFACE_MODE_NA ||
- state->interface == PHY_INTERFACE_MODE_SGMII ||
- state->interface == PHY_INTERFACE_MODE_RGMII ||
- state->interface == PHY_INTERFACE_MODE_GMII) {
- phylink_set(mask, 1000baseT_Full);
- phylink_set(mask, 1000baseX_Full);
- }
-
- linkmode_and(supported, supported, mask);
- linkmode_and(state->advertising, state->advertising, mask);
-
- return;
-unsupported:
- linkmode_zero(supported);
-}
-
-static void ag71xx_mac_pcs_get_state(struct phylink_config *config,
- struct phylink_link_state *state)
-{
- state->link = 0;
-}
-
-static void ag71xx_mac_an_restart(struct phylink_config *config)
-{
- /* Not Supported */
-}
-
static void ag71xx_mac_link_down(struct phylink_config *config,
unsigned int mode, phy_interface_t interface)
{
@@ -1163,9 +1086,7 @@ static void ag71xx_mac_link_up(struct phylink_config *config,
}
static const struct phylink_mac_ops ag71xx_phylink_mac_ops = {
- .validate = ag71xx_mac_validate,
- .mac_pcs_get_state = ag71xx_mac_pcs_get_state,
- .mac_an_restart = ag71xx_mac_an_restart,
+ .validate = phylink_generic_validate,
.mac_config = ag71xx_mac_config,
.mac_link_down = ag71xx_mac_link_down,
.mac_link_up = ag71xx_mac_link_up,
@@ -1177,6 +1098,34 @@ static int ag71xx_phylink_setup(struct ag71xx *ag)
ag->phylink_config.dev = &ag->ndev->dev;
ag->phylink_config.type = PHYLINK_NETDEV;
+ ag->phylink_config.mac_capabilities = MAC_SYM_PAUSE | MAC_ASYM_PAUSE |
+ MAC_10 | MAC_100 | MAC_1000FD;
+
+ if ((ag71xx_is(ag, AR9330) && ag->mac_idx == 0) ||
+ ag71xx_is(ag, AR9340) ||
+ ag71xx_is(ag, QCA9530) ||
+ (ag71xx_is(ag, QCA9550) && ag->mac_idx == 1))
+ __set_bit(PHY_INTERFACE_MODE_MII,
+ ag->phylink_config.supported_interfaces);
+
+ if ((ag71xx_is(ag, AR9330) && ag->mac_idx == 1) ||
+ (ag71xx_is(ag, AR9340) && ag->mac_idx == 1) ||
+ (ag71xx_is(ag, QCA9530) && ag->mac_idx == 1))
+ __set_bit(PHY_INTERFACE_MODE_GMII,
+ ag->phylink_config.supported_interfaces);
+
+ if (ag71xx_is(ag, QCA9550) && ag->mac_idx == 0)
+ __set_bit(PHY_INTERFACE_MODE_SGMII,
+ ag->phylink_config.supported_interfaces);
+
+ if (ag71xx_is(ag, AR9340) && ag->mac_idx == 0)
+ __set_bit(PHY_INTERFACE_MODE_RMII,
+ ag->phylink_config.supported_interfaces);
+
+ if ((ag71xx_is(ag, AR9340) && ag->mac_idx == 0) ||
+ (ag71xx_is(ag, QCA9550) && ag->mac_idx == 1))
+ __set_bit(PHY_INTERFACE_MODE_RGMII,
+ ag->phylink_config.supported_interfaces);
phylink = phylink_create(&ag->phylink_config, ag->pdev->dev.fwnode,
ag->phy_if_mode, &ag71xx_phylink_mac_ops);
@@ -1913,15 +1862,12 @@ static int ag71xx_probe(struct platform_device *pdev)
ag->mac_reset = devm_reset_control_get(&pdev->dev, "mac");
if (IS_ERR(ag->mac_reset)) {
netif_err(ag, probe, ndev, "missing mac reset\n");
- err = PTR_ERR(ag->mac_reset);
- goto err_free;
+ return PTR_ERR(ag->mac_reset);
}
ag->mac_base = devm_ioremap(&pdev->dev, res->start, resource_size(res));
- if (!ag->mac_base) {
- err = -ENOMEM;
- goto err_free;
- }
+ if (!ag->mac_base)
+ return -ENOMEM;
ndev->irq = platform_get_irq(pdev, 0);
err = devm_request_irq(&pdev->dev, ndev->irq, ag71xx_interrupt,
@@ -1929,7 +1875,7 @@ static int ag71xx_probe(struct platform_device *pdev)
if (err) {
netif_err(ag, probe, ndev, "unable to request IRQ %d\n",
ndev->irq);
- goto err_free;
+ return err;
}
ndev->netdev_ops = &ag71xx_netdev_ops;
@@ -1957,10 +1903,8 @@ static int ag71xx_probe(struct platform_device *pdev)
ag->stop_desc = dmam_alloc_coherent(&pdev->dev,
sizeof(struct ag71xx_desc),
&ag->stop_desc_dma, GFP_KERNEL);
- if (!ag->stop_desc) {
- err = -ENOMEM;
- goto err_free;
- }
+ if (!ag->stop_desc)
+ return -ENOMEM;
ag->stop_desc->data = 0;
ag->stop_desc->ctrl = 0;
@@ -1975,7 +1919,7 @@ static int ag71xx_probe(struct platform_device *pdev)
err = of_get_phy_mode(np, &ag->phy_if_mode);
if (err) {
netif_err(ag, probe, ndev, "missing phy-mode property in DT\n");
- goto err_free;
+ return err;
}
netif_napi_add(ndev, &ag->napi, ag71xx_poll, AG71XX_NAPI_WEIGHT);
@@ -1983,7 +1927,7 @@ static int ag71xx_probe(struct platform_device *pdev)
err = clk_prepare_enable(ag->clk_eth);
if (err) {
netif_err(ag, probe, ndev, "Failed to enable eth clk.\n");
- goto err_free;
+ return err;
}
ag71xx_wr(ag, AG71XX_REG_MAC_CFG1, 0);
@@ -2019,8 +1963,6 @@ static int ag71xx_probe(struct platform_device *pdev)
ag71xx_mdio_remove(ag);
err_put_clk:
clk_disable_unprepare(ag->clk_eth);
-err_free:
- free_netdev(ndev);
return err;
}
diff --git a/drivers/net/ethernet/atheros/atlx/atl1.c b/drivers/net/ethernet/atheros/atlx/atl1.c
index b4c9e80..6a96996 100644
--- a/drivers/net/ethernet/atheros/atlx/atl1.c
+++ b/drivers/net/ethernet/atheros/atlx/atl1.c
@@ -3438,7 +3438,9 @@ static void atl1_get_regs(struct net_device *netdev, struct ethtool_regs *regs,
}
static void atl1_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct atl1_adapter *adapter = netdev_priv(netdev);
struct atl1_tpd_ring *txdr = &adapter->tpd_ring;
@@ -3451,7 +3453,9 @@ static void atl1_get_ringparam(struct net_device *netdev,
}
static int atl1_set_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct atl1_adapter *adapter = netdev_priv(netdev);
struct atl1_tpd_ring *tpdr = &adapter->tpd_ring;
diff --git a/drivers/net/ethernet/broadcom/b44.c b/drivers/net/ethernet/broadcom/b44.c
index 969591b..e5857e8 100644
--- a/drivers/net/ethernet/broadcom/b44.c
+++ b/drivers/net/ethernet/broadcom/b44.c
@@ -1961,7 +1961,9 @@ static int b44_set_link_ksettings(struct net_device *dev,
}
static void b44_get_ringparam(struct net_device *dev,
- struct ethtool_ringparam *ering)
+ struct ethtool_ringparam *ering,
+ struct kernel_ethtool_ringparam *kernel_ering,
+ struct netlink_ext_ack *extack)
{
struct b44 *bp = netdev_priv(dev);
@@ -1972,7 +1974,9 @@ static void b44_get_ringparam(struct net_device *dev,
}
static int b44_set_ringparam(struct net_device *dev,
- struct ethtool_ringparam *ering)
+ struct ethtool_ringparam *ering,
+ struct kernel_ethtool_ringparam *kernel_ering,
+ struct netlink_ext_ack *extack)
{
struct b44 *bp = netdev_priv(dev);
diff --git a/drivers/net/ethernet/broadcom/bcm4908_enet.c b/drivers/net/ethernet/broadcom/bcm4908_enet.c
index b07cb9bc..4a2622b 100644
--- a/drivers/net/ethernet/broadcom/bcm4908_enet.c
+++ b/drivers/net/ethernet/broadcom/bcm4908_enet.c
@@ -635,7 +635,6 @@ static int bcm4908_enet_poll_tx(struct napi_struct *napi, int weight)
struct bcm4908_enet_dma_ring_bd *buf_desc;
struct bcm4908_enet_dma_ring_slot *slot;
struct device *dev = enet->dev;
- unsigned int bytes = 0;
int handled = 0;
while (handled < weight && tx_ring->read_idx != tx_ring->write_idx) {
@@ -646,7 +645,6 @@ static int bcm4908_enet_poll_tx(struct napi_struct *napi, int weight)
dma_unmap_single(dev, slot->dma_addr, slot->len, DMA_TO_DEVICE);
dev_kfree_skb(slot->skb);
- bytes += slot->len;
if (++tx_ring->read_idx == tx_ring->length)
tx_ring->read_idx = 0;
diff --git a/drivers/net/ethernet/broadcom/bcm63xx_enet.c b/drivers/net/ethernet/broadcom/bcm63xx_enet.c
index a568994..b04e423 100644
--- a/drivers/net/ethernet/broadcom/bcm63xx_enet.c
+++ b/drivers/net/ethernet/broadcom/bcm63xx_enet.c
@@ -1497,8 +1497,11 @@ static int bcm_enet_set_link_ksettings(struct net_device *dev,
}
}
-static void bcm_enet_get_ringparam(struct net_device *dev,
- struct ethtool_ringparam *ering)
+static void
+bcm_enet_get_ringparam(struct net_device *dev,
+ struct ethtool_ringparam *ering,
+ struct kernel_ethtool_ringparam *kernel_ering,
+ struct netlink_ext_ack *extack)
{
struct bcm_enet_priv *priv;
@@ -1512,7 +1515,9 @@ static void bcm_enet_get_ringparam(struct net_device *dev,
}
static int bcm_enet_set_ringparam(struct net_device *dev,
- struct ethtool_ringparam *ering)
+ struct ethtool_ringparam *ering,
+ struct kernel_ethtool_ringparam *kernel_ering,
+ struct netlink_ext_ack *extack)
{
struct bcm_enet_priv *priv;
int was_running;
@@ -2579,8 +2584,11 @@ static void bcm_enetsw_get_ethtool_stats(struct net_device *netdev,
}
}
-static void bcm_enetsw_get_ringparam(struct net_device *dev,
- struct ethtool_ringparam *ering)
+static void
+bcm_enetsw_get_ringparam(struct net_device *dev,
+ struct ethtool_ringparam *ering,
+ struct kernel_ethtool_ringparam *kernel_ering,
+ struct netlink_ext_ack *extack)
{
struct bcm_enet_priv *priv;
@@ -2595,8 +2603,11 @@ static void bcm_enetsw_get_ringparam(struct net_device *dev,
ering->tx_pending = priv->tx_ring_size;
}
-static int bcm_enetsw_set_ringparam(struct net_device *dev,
- struct ethtool_ringparam *ering)
+static int
+bcm_enetsw_set_ringparam(struct net_device *dev,
+ struct ethtool_ringparam *ering,
+ struct kernel_ethtool_ringparam *kernel_ering,
+ struct netlink_ext_ack *extack)
{
struct bcm_enet_priv *priv;
int was_running;
diff --git a/drivers/net/ethernet/broadcom/bnx2.c b/drivers/net/ethernet/broadcom/bnx2.c
index babc955..e20aafe 100644
--- a/drivers/net/ethernet/broadcom/bnx2.c
+++ b/drivers/net/ethernet/broadcom/bnx2.c
@@ -7318,7 +7318,9 @@ static int bnx2_set_coalesce(struct net_device *dev,
}
static void
-bnx2_get_ringparam(struct net_device *dev, struct ethtool_ringparam *ering)
+bnx2_get_ringparam(struct net_device *dev, struct ethtool_ringparam *ering,
+ struct kernel_ethtool_ringparam *kernel_ering,
+ struct netlink_ext_ack *extack)
{
struct bnx2 *bp = netdev_priv(dev);
@@ -7389,7 +7391,9 @@ bnx2_change_ring_size(struct bnx2 *bp, u32 rx, u32 tx, bool reset_irq)
}
static int
-bnx2_set_ringparam(struct net_device *dev, struct ethtool_ringparam *ering)
+bnx2_set_ringparam(struct net_device *dev, struct ethtool_ringparam *ering,
+ struct kernel_ethtool_ringparam *kernel_ering,
+ struct netlink_ext_ack *extack)
{
struct bnx2 *bp = netdev_priv(dev);
int rc;
diff --git a/drivers/net/ethernet/broadcom/bnx2x/bnx2x.h b/drivers/net/ethernet/broadcom/bnx2x/bnx2x.h
index 2b06d78b..a19dd67 100644
--- a/drivers/net/ethernet/broadcom/bnx2x/bnx2x.h
+++ b/drivers/net/ethernet/broadcom/bnx2x/bnx2x.h
@@ -1850,6 +1850,14 @@ struct bnx2x {
/* Vxlan/Geneve related information */
u16 udp_tunnel_ports[BNX2X_UDP_PORT_MAX];
+
+#define FW_CAP_INVALIDATE_VF_FP_HSI BIT(0)
+ u32 fw_cap;
+
+ u32 fw_major;
+ u32 fw_minor;
+ u32 fw_rev;
+ u32 fw_eng;
};
/* Tx queues may be less or equal to Rx queues */
@@ -2525,5 +2533,6 @@ void bnx2x_register_phc(struct bnx2x *bp);
* Meant for implicit re-load flows.
*/
int bnx2x_vlan_reconfigure_vid(struct bnx2x *bp);
-
+int bnx2x_init_firmware(struct bnx2x *bp);
+void bnx2x_release_firmware(struct bnx2x *bp);
#endif /* bnx2x.h */
diff --git a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_cmn.c b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_cmn.c
index e8e8c2d..8d36ebb 100644
--- a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_cmn.c
+++ b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_cmn.c
@@ -25,6 +25,7 @@
#include <linux/ip.h>
#include <linux/crash_dump.h>
#include <net/tcp.h>
+#include <net/gro.h>
#include <net/ipv6.h>
#include <net/ip6_checksum.h>
#include <linux/prefetch.h>
@@ -2364,10 +2365,8 @@ int bnx2x_compare_fw_ver(struct bnx2x *bp, u32 load_code, bool print_err)
if (load_code != FW_MSG_CODE_DRV_LOAD_COMMON_CHIP &&
load_code != FW_MSG_CODE_DRV_LOAD_COMMON) {
/* build my FW version dword */
- u32 my_fw = (BCM_5710_FW_MAJOR_VERSION) +
- (BCM_5710_FW_MINOR_VERSION << 8) +
- (BCM_5710_FW_REVISION_VERSION << 16) +
- (BCM_5710_FW_ENGINEERING_VERSION << 24);
+ u32 my_fw = (bp->fw_major) + (bp->fw_minor << 8) +
+ (bp->fw_rev << 16) + (bp->fw_eng << 24);
/* read loaded FW from chip */
u32 loaded_fw = REG_RD(bp, XSEM_REG_PRAM);
diff --git a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_ethtool.c b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_ethtool.c
index 472a3a4..0e319ac 100644
--- a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_ethtool.c
+++ b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_ethtool.c
@@ -1914,7 +1914,9 @@ static int bnx2x_set_coalesce(struct net_device *dev,
}
static void bnx2x_get_ringparam(struct net_device *dev,
- struct ethtool_ringparam *ering)
+ struct ethtool_ringparam *ering,
+ struct kernel_ethtool_ringparam *kernel_ering,
+ struct netlink_ext_ack *extack)
{
struct bnx2x *bp = netdev_priv(dev);
@@ -1938,7 +1940,9 @@ static void bnx2x_get_ringparam(struct net_device *dev,
}
static int bnx2x_set_ringparam(struct net_device *dev,
- struct ethtool_ringparam *ering)
+ struct ethtool_ringparam *ering,
+ struct kernel_ethtool_ringparam *kernel_ering,
+ struct netlink_ext_ack *extack)
{
struct bnx2x *bp = netdev_priv(dev);
diff --git a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_fw_defs.h b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_fw_defs.h
index 3f84352..a84d015 100644
--- a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_fw_defs.h
+++ b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_fw_defs.h
@@ -241,6 +241,8 @@
IRO[221].m2))
#define XSTORM_VF_TO_PF_OFFSET(funcId) \
(IRO[48].base + ((funcId) * IRO[48].m1))
+#define XSTORM_ETH_FUNCTION_INFO_FP_HSI_VALID_E2_OFFSET(fid) \
+ (IRO[386].base + ((fid) * IRO[386].m1))
#define COMMON_ASM_INVALID_ASSERT_OPCODE 0x0
/* eth hsi version */
diff --git a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_hsi.h b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_hsi.h
index 622fadc..611efee 100644
--- a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_hsi.h
+++ b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_hsi.h
@@ -3024,7 +3024,8 @@ struct afex_stats {
#define BCM_5710_FW_MAJOR_VERSION 7
#define BCM_5710_FW_MINOR_VERSION 13
-#define BCM_5710_FW_REVISION_VERSION 15
+#define BCM_5710_FW_REVISION_VERSION 21
+#define BCM_5710_FW_REVISION_VERSION_V15 15
#define BCM_5710_FW_ENGINEERING_VERSION 0
#define BCM_5710_FW_COMPILE_FLAGS 1
diff --git a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c
index aec666e..774c1f1a 100644
--- a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c
+++ b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c
@@ -74,9 +74,19 @@
__stringify(BCM_5710_FW_MINOR_VERSION) "." \
__stringify(BCM_5710_FW_REVISION_VERSION) "." \
__stringify(BCM_5710_FW_ENGINEERING_VERSION)
+
+#define FW_FILE_VERSION_V15 \
+ __stringify(BCM_5710_FW_MAJOR_VERSION) "." \
+ __stringify(BCM_5710_FW_MINOR_VERSION) "." \
+ __stringify(BCM_5710_FW_REVISION_VERSION_V15) "." \
+ __stringify(BCM_5710_FW_ENGINEERING_VERSION)
+
#define FW_FILE_NAME_E1 "bnx2x/bnx2x-e1-" FW_FILE_VERSION ".fw"
#define FW_FILE_NAME_E1H "bnx2x/bnx2x-e1h-" FW_FILE_VERSION ".fw"
#define FW_FILE_NAME_E2 "bnx2x/bnx2x-e2-" FW_FILE_VERSION ".fw"
+#define FW_FILE_NAME_E1_V15 "bnx2x/bnx2x-e1-" FW_FILE_VERSION_V15 ".fw"
+#define FW_FILE_NAME_E1H_V15 "bnx2x/bnx2x-e1h-" FW_FILE_VERSION_V15 ".fw"
+#define FW_FILE_NAME_E2_V15 "bnx2x/bnx2x-e2-" FW_FILE_VERSION_V15 ".fw"
/* Time in jiffies before concluding the transmitter is hung */
#define TX_TIMEOUT (5*HZ)
@@ -747,9 +757,7 @@ static int bnx2x_mc_assert(struct bnx2x *bp)
CHIP_IS_E1(bp) ? "everest1" :
CHIP_IS_E1H(bp) ? "everest1h" :
CHIP_IS_E2(bp) ? "everest2" : "everest3",
- BCM_5710_FW_MAJOR_VERSION,
- BCM_5710_FW_MINOR_VERSION,
- BCM_5710_FW_REVISION_VERSION);
+ bp->fw_major, bp->fw_minor, bp->fw_rev);
return rc;
}
@@ -12308,6 +12316,15 @@ static int bnx2x_init_bp(struct bnx2x *bp)
bnx2x_read_fwinfo(bp);
+ if (IS_PF(bp)) {
+ rc = bnx2x_init_firmware(bp);
+
+ if (rc) {
+ bnx2x_free_mem_bp(bp);
+ return rc;
+ }
+ }
+
func = BP_FUNC(bp);
/* need to reset chip if undi was active */
@@ -12320,6 +12337,7 @@ static int bnx2x_init_bp(struct bnx2x *bp)
rc = bnx2x_prev_unload(bp);
if (rc) {
+ bnx2x_release_firmware(bp);
bnx2x_free_mem_bp(bp);
return rc;
}
@@ -13026,19 +13044,6 @@ static const struct net_device_ops bnx2x_netdev_ops = {
.ndo_features_check = bnx2x_features_check,
};
-static int bnx2x_set_coherency_mask(struct bnx2x *bp)
-{
- struct device *dev = &bp->pdev->dev;
-
- if (dma_set_mask_and_coherent(dev, DMA_BIT_MASK(64)) != 0 &&
- dma_set_mask_and_coherent(dev, DMA_BIT_MASK(32)) != 0) {
- dev_err(dev, "System does not support DMA, aborting\n");
- return -EIO;
- }
-
- return 0;
-}
-
static void bnx2x_disable_pcie_error_reporting(struct bnx2x *bp)
{
if (bp->flags & AER_ENABLED) {
@@ -13116,9 +13121,11 @@ static int bnx2x_init_dev(struct bnx2x *bp, struct pci_dev *pdev,
goto err_out_release;
}
- rc = bnx2x_set_coherency_mask(bp);
- if (rc)
+ rc = dma_set_mask_and_coherent(&bp->pdev->dev, DMA_BIT_MASK(64));
+ if (rc) {
+ dev_err(&bp->pdev->dev, "System does not support DMA, aborting\n");
goto err_out_release;
+ }
dev->mem_start = pci_resource_start(pdev, 0);
dev->base_addr = dev->mem_start;
@@ -13317,16 +13324,11 @@ static int bnx2x_check_firmware(struct bnx2x *bp)
/* Check FW version */
offset = be32_to_cpu(fw_hdr->fw_version.offset);
fw_ver = firmware->data + offset;
- if ((fw_ver[0] != BCM_5710_FW_MAJOR_VERSION) ||
- (fw_ver[1] != BCM_5710_FW_MINOR_VERSION) ||
- (fw_ver[2] != BCM_5710_FW_REVISION_VERSION) ||
- (fw_ver[3] != BCM_5710_FW_ENGINEERING_VERSION)) {
+ if (fw_ver[0] != bp->fw_major || fw_ver[1] != bp->fw_minor ||
+ fw_ver[2] != bp->fw_rev || fw_ver[3] != bp->fw_eng) {
BNX2X_ERR("Bad FW version:%d.%d.%d.%d. Should be %d.%d.%d.%d\n",
- fw_ver[0], fw_ver[1], fw_ver[2], fw_ver[3],
- BCM_5710_FW_MAJOR_VERSION,
- BCM_5710_FW_MINOR_VERSION,
- BCM_5710_FW_REVISION_VERSION,
- BCM_5710_FW_ENGINEERING_VERSION);
+ fw_ver[0], fw_ver[1], fw_ver[2], fw_ver[3],
+ bp->fw_major, bp->fw_minor, bp->fw_rev, bp->fw_eng);
return -EINVAL;
}
@@ -13404,34 +13406,51 @@ do { \
(u8 *)bp->arr, len); \
} while (0)
-static int bnx2x_init_firmware(struct bnx2x *bp)
+int bnx2x_init_firmware(struct bnx2x *bp)
{
- const char *fw_file_name;
+ const char *fw_file_name, *fw_file_name_v15;
struct bnx2x_fw_file_hdr *fw_hdr;
int rc;
if (bp->firmware)
return 0;
- if (CHIP_IS_E1(bp))
+ if (CHIP_IS_E1(bp)) {
fw_file_name = FW_FILE_NAME_E1;
- else if (CHIP_IS_E1H(bp))
+ fw_file_name_v15 = FW_FILE_NAME_E1_V15;
+ } else if (CHIP_IS_E1H(bp)) {
fw_file_name = FW_FILE_NAME_E1H;
- else if (!CHIP_IS_E1x(bp))
+ fw_file_name_v15 = FW_FILE_NAME_E1H_V15;
+ } else if (!CHIP_IS_E1x(bp)) {
fw_file_name = FW_FILE_NAME_E2;
- else {
+ fw_file_name_v15 = FW_FILE_NAME_E2_V15;
+ } else {
BNX2X_ERR("Unsupported chip revision\n");
return -EINVAL;
}
+
BNX2X_DEV_INFO("Loading %s\n", fw_file_name);
rc = request_firmware(&bp->firmware, fw_file_name, &bp->pdev->dev);
if (rc) {
- BNX2X_ERR("Can't load firmware file %s\n",
- fw_file_name);
- goto request_firmware_exit;
+ BNX2X_DEV_INFO("Trying to load older fw %s\n", fw_file_name_v15);
+
+ /* try to load prev version */
+ rc = request_firmware(&bp->firmware, fw_file_name_v15, &bp->pdev->dev);
+
+ if (rc)
+ goto request_firmware_exit;
+
+ bp->fw_rev = BCM_5710_FW_REVISION_VERSION_V15;
+ } else {
+ bp->fw_cap |= FW_CAP_INVALIDATE_VF_FP_HSI;
+ bp->fw_rev = BCM_5710_FW_REVISION_VERSION;
}
+ bp->fw_major = BCM_5710_FW_MAJOR_VERSION;
+ bp->fw_minor = BCM_5710_FW_MINOR_VERSION;
+ bp->fw_eng = BCM_5710_FW_ENGINEERING_VERSION;
+
rc = bnx2x_check_firmware(bp);
if (rc) {
BNX2X_ERR("Corrupt firmware file %s\n", fw_file_name);
@@ -13487,7 +13506,7 @@ static int bnx2x_init_firmware(struct bnx2x *bp)
return rc;
}
-static void bnx2x_release_firmware(struct bnx2x *bp)
+void bnx2x_release_firmware(struct bnx2x *bp)
{
kfree(bp->init_ops_offsets);
kfree(bp->init_ops);
@@ -14004,6 +14023,7 @@ static int bnx2x_init_one(struct pci_dev *pdev,
return 0;
init_one_freemem:
+ bnx2x_release_firmware(bp);
bnx2x_free_mem_bp(bp);
init_one_exit:
@@ -15356,11 +15376,6 @@ static int bnx2x_hwtstamp_ioctl(struct bnx2x *bp, struct ifreq *ifr)
DP(BNX2X_MSG_PTP, "Requested tx_type: %d, requested rx_filters = %d\n",
config.tx_type, config.rx_filter);
- if (config.flags) {
- BNX2X_ERR("config.flags is reserved for future use\n");
- return -EINVAL;
- }
-
bp->hwtstamp_ioctl_called = true;
bp->tx_type = config.tx_type;
bp->rx_filter = config.rx_filter;
diff --git a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_sriov.c b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_sriov.c
index 74a8931..11d15cd 100644
--- a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_sriov.c
+++ b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_sriov.c
@@ -758,9 +758,18 @@ static void bnx2x_vf_igu_reset(struct bnx2x *bp, struct bnx2x_virtf *vf)
void bnx2x_vf_enable_access(struct bnx2x *bp, u8 abs_vfid)
{
+ u16 abs_fid;
+
+ abs_fid = FW_VF_HANDLE(abs_vfid);
+
/* set the VF-PF association in the FW */
- storm_memset_vf_to_pf(bp, FW_VF_HANDLE(abs_vfid), BP_FUNC(bp));
- storm_memset_func_en(bp, FW_VF_HANDLE(abs_vfid), 1);
+ storm_memset_vf_to_pf(bp, abs_fid, BP_FUNC(bp));
+ storm_memset_func_en(bp, abs_fid, 1);
+
+ /* Invalidate fp_hsi version for vfs */
+ if (bp->fw_cap & FW_CAP_INVALIDATE_VF_FP_HSI)
+ REG_WR8(bp, BAR_XSTRORM_INTMEM +
+ XSTORM_ETH_FUNCTION_INFO_FP_HSI_VALID_E2_OFFSET(abs_fid), 0);
/* clear vf errors*/
bnx2x_vf_semi_clear_err(bp, abs_vfid);
diff --git a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_sriov.h b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_sriov.h
index 8c2cf55..2dac704 100644
--- a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_sriov.h
+++ b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_sriov.h
@@ -586,7 +586,7 @@ static inline int bnx2x_vfpf_release(struct bnx2x *bp) {return 0; }
static inline int bnx2x_vfpf_init(struct bnx2x *bp) {return 0; }
static inline void bnx2x_vfpf_close_vf(struct bnx2x *bp) {}
static inline int bnx2x_vfpf_setup_q(struct bnx2x *bp, struct bnx2x_fastpath *fp, bool is_leading) {return 0; }
-static inline int bnx2x_vfpf_config_mac(struct bnx2x *bp, u8 *addr,
+static inline int bnx2x_vfpf_config_mac(struct bnx2x *bp, const u8 *addr,
u8 vf_qid, bool set) {return 0; }
static inline int bnx2x_vfpf_config_rss(struct bnx2x *bp,
struct bnx2x_config_rss_params *params) {return 0; }
diff --git a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_stats.c b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_stats.c
index 0b193ed..2bb133a 100644
--- a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_stats.c
+++ b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_stats.c
@@ -849,7 +849,8 @@ static int bnx2x_hw_stats_update(struct bnx2x *bp)
memcpy(old, new, sizeof(struct nig_stats));
- memcpy(&(estats->rx_stat_ifhcinbadoctets_hi), &(pstats->mac_stx[1]),
+ BUILD_BUG_ON(sizeof(estats->shared) != sizeof(pstats->mac_stx[1]));
+ memcpy(&(estats->shared), &(pstats->mac_stx[1]),
sizeof(struct mac_stx));
estats->brb_drop_hi = pstats->brb_drop_hi;
estats->brb_drop_lo = pstats->brb_drop_lo;
@@ -1634,9 +1635,9 @@ void bnx2x_stats_init(struct bnx2x *bp)
REG_RD(bp, NIG_REG_STAT0_BRB_TRUNCATE + port*0x38);
if (!CHIP_IS_E3(bp)) {
REG_RD_DMAE(bp, NIG_REG_STAT0_EGRESS_MAC_PKT0 + port*0x50,
- &(bp->port.old_nig_stats.egress_mac_pkt0_lo), 2);
+ &(bp->port.old_nig_stats.egress_mac_pkt0), 2);
REG_RD_DMAE(bp, NIG_REG_STAT0_EGRESS_MAC_PKT1 + port*0x50,
- &(bp->port.old_nig_stats.egress_mac_pkt1_lo), 2);
+ &(bp->port.old_nig_stats.egress_mac_pkt1), 2);
}
/* Prepare statistics ramrod data */
diff --git a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_stats.h b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_stats.h
index d55e636..ae93c07 100644
--- a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_stats.h
+++ b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_stats.h
@@ -36,10 +36,14 @@ struct nig_stats {
u32 pbf_octets;
u32 pbf_packet;
u32 safc_inp;
- u32 egress_mac_pkt0_lo;
- u32 egress_mac_pkt0_hi;
- u32 egress_mac_pkt1_lo;
- u32 egress_mac_pkt1_hi;
+ struct_group(egress_mac_pkt0,
+ u32 egress_mac_pkt0_lo;
+ u32 egress_mac_pkt0_hi;
+ );
+ struct_group(egress_mac_pkt1,
+ u32 egress_mac_pkt1_lo;
+ u32 egress_mac_pkt1_hi;
+ );
};
enum bnx2x_stats_event {
@@ -83,6 +87,7 @@ struct bnx2x_eth_stats {
u32 no_buff_discard_hi;
u32 no_buff_discard_lo;
+ struct_group(shared,
u32 rx_stat_ifhcinbadoctets_hi;
u32 rx_stat_ifhcinbadoctets_lo;
u32 tx_stat_ifhcoutbadoctets_hi;
@@ -159,6 +164,7 @@ struct bnx2x_eth_stats {
u32 tx_stat_dot3statsinternalmactransmiterrors_lo;
u32 tx_stat_bmac_ufl_hi;
u32 tx_stat_bmac_ufl_lo;
+ );
u32 pause_frames_received_hi;
u32 pause_frames_received_lo;
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index c04ea83..4f94136 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -37,6 +37,7 @@
#include <linux/if_bridge.h>
#include <linux/rtc.h>
#include <linux/bpf.h>
+#include <net/gro.h>
#include <net/ip.h>
#include <net/tcp.h>
#include <net/udp.h>
@@ -740,13 +741,16 @@ static struct page *__bnxt_alloc_rx_page(struct bnxt *bp, dma_addr_t *mapping,
return page;
}
-static inline u8 *__bnxt_alloc_rx_data(struct bnxt *bp, dma_addr_t *mapping,
+static inline u8 *__bnxt_alloc_rx_frag(struct bnxt *bp, dma_addr_t *mapping,
gfp_t gfp)
{
u8 *data;
struct pci_dev *pdev = bp->pdev;
- data = kmalloc(bp->rx_buf_size, gfp);
+ if (gfp == GFP_ATOMIC)
+ data = napi_alloc_frag(bp->rx_buf_size);
+ else
+ data = netdev_alloc_frag(bp->rx_buf_size);
if (!data)
return NULL;
@@ -755,7 +759,7 @@ static inline u8 *__bnxt_alloc_rx_data(struct bnxt *bp, dma_addr_t *mapping,
DMA_ATTR_WEAK_ORDERING);
if (dma_mapping_error(&pdev->dev, *mapping)) {
- kfree(data);
+ skb_free_frag(data);
data = NULL;
}
return data;
@@ -778,7 +782,7 @@ int bnxt_alloc_rx_data(struct bnxt *bp, struct bnxt_rx_ring_info *rxr,
rx_buf->data = page;
rx_buf->data_ptr = page_address(page) + bp->rx_offset;
} else {
- u8 *data = __bnxt_alloc_rx_data(bp, &mapping, gfp);
+ u8 *data = __bnxt_alloc_rx_frag(bp, &mapping, gfp);
if (!data)
return -ENOMEM;
@@ -1020,11 +1024,11 @@ static struct sk_buff *bnxt_rx_skb(struct bnxt *bp,
return NULL;
}
- skb = build_skb(data, 0);
+ skb = build_skb(data, bp->rx_buf_size);
dma_unmap_single_attrs(&bp->pdev->dev, dma_addr, bp->rx_buf_use_size,
bp->rx_dir, DMA_ATTR_WEAK_ORDERING);
if (!skb) {
- kfree(data);
+ skb_free_frag(data);
return NULL;
}
@@ -1612,7 +1616,7 @@ static inline struct sk_buff *bnxt_tpa_end(struct bnxt *bp,
u8 *new_data;
dma_addr_t new_mapping;
- new_data = __bnxt_alloc_rx_data(bp, &new_mapping, GFP_ATOMIC);
+ new_data = __bnxt_alloc_rx_frag(bp, &new_mapping, GFP_ATOMIC);
if (!new_data) {
bnxt_abort_tpa(cpr, idx, agg_bufs);
cpr->sw_stats.rx.rx_oom_discards += 1;
@@ -1623,13 +1627,13 @@ static inline struct sk_buff *bnxt_tpa_end(struct bnxt *bp,
tpa_info->data_ptr = new_data + bp->rx_offset;
tpa_info->mapping = new_mapping;
- skb = build_skb(data, 0);
+ skb = build_skb(data, bp->rx_buf_size);
dma_unmap_single_attrs(&bp->pdev->dev, mapping,
bp->rx_buf_use_size, bp->rx_dir,
DMA_ATTR_WEAK_ORDERING);
if (!skb) {
- kfree(data);
+ skb_free_frag(data);
bnxt_abort_tpa(cpr, idx, agg_bufs);
cpr->sw_stats.rx.rx_oom_discards += 1;
return NULL;
@@ -2043,13 +2047,22 @@ static u16 bnxt_agg_ring_id_to_grp_idx(struct bnxt *bp, u16 ring_id)
static void bnxt_event_error_report(struct bnxt *bp, u32 data1, u32 data2)
{
- switch (BNXT_EVENT_ERROR_REPORT_TYPE(data1)) {
+ u32 err_type = BNXT_EVENT_ERROR_REPORT_TYPE(data1);
+
+ switch (err_type) {
case ASYNC_EVENT_CMPL_ERROR_REPORT_BASE_EVENT_DATA1_ERROR_TYPE_INVALID_SIGNAL:
netdev_err(bp->dev, "1PPS: Received invalid signal on pin%lu from the external source. Please fix the signal and reconfigure the pin\n",
BNXT_EVENT_INVALID_SIGNAL_DATA(data2));
break;
+ case ASYNC_EVENT_CMPL_ERROR_REPORT_BASE_EVENT_DATA1_ERROR_TYPE_PAUSE_STORM:
+ netdev_warn(bp->dev, "Pause Storm detected!\n");
+ break;
+ case ASYNC_EVENT_CMPL_ERROR_REPORT_BASE_EVENT_DATA1_ERROR_TYPE_DOORBELL_DROP_THRESHOLD:
+ netdev_warn(bp->dev, "One or more MMIO doorbells dropped by the device!\n");
+ break;
default:
- netdev_err(bp->dev, "FW reported unknown error type\n");
+ netdev_err(bp->dev, "FW reported unknown error type %u\n",
+ err_type);
break;
}
}
@@ -2073,6 +2086,9 @@ static int bnxt_async_event_process(struct bnxt *bp,
u32 data1 = le32_to_cpu(cmpl->event_data1);
u32 data2 = le32_to_cpu(cmpl->event_data2);
+ netdev_dbg(bp->dev, "hwrm event 0x%x {0x%x, 0x%x}\n",
+ event_id, data1, data2);
+
/* TODO CHIMP_FW: Define event id's for link change, error etc */
switch (event_id) {
case ASYNC_EVENT_CMPL_EVENT_ID_LINK_SPEED_CFG_CHANGE: {
@@ -2419,7 +2435,7 @@ static int __bnxt_poll_work(struct bnxt *bp, struct bnxt_cp_ring_info *cpr,
}
if (event & BNXT_REDIRECT_EVENT)
- xdp_do_flush_map();
+ xdp_do_flush();
if (event & BNXT_TX_EVENT) {
struct bnxt_tx_ring_info *txr = bnapi->tx_ring;
@@ -2622,6 +2638,7 @@ static int bnxt_poll_p5(struct napi_struct *napi, int budget)
{
struct bnxt_napi *bnapi = container_of(napi, struct bnxt_napi, napi);
struct bnxt_cp_ring_info *cpr = &bnapi->cp_ring;
+ struct bnxt_cp_ring_info *cpr_rx;
u32 raw_cons = cpr->cp_raw_cons;
struct bnxt *bp = bnapi->bp;
struct nqe_cn *nqcmp;
@@ -2649,7 +2666,7 @@ static int bnxt_poll_p5(struct napi_struct *napi, int budget)
if (napi_complete_done(napi, work_done))
BNXT_DB_NQ_ARM_P5(&cpr->cp_db,
cpr->cp_raw_cons);
- return work_done;
+ goto poll_done;
}
/* The valid test of the entry must be done first before
@@ -2675,6 +2692,17 @@ static int bnxt_poll_p5(struct napi_struct *napi, int budget)
cpr->cp_raw_cons = raw_cons;
BNXT_DB_NQ_P5(&cpr->cp_db, raw_cons);
}
+poll_done:
+ cpr_rx = cpr->cp_ring_arr[BNXT_RX_HDL];
+ if (cpr_rx && (bp->flags & BNXT_FLAG_DIM)) {
+ struct dim_sample dim_sample = {};
+
+ dim_update_sample(cpr->event_ctr,
+ cpr_rx->rx_packets,
+ cpr_rx->rx_bytes,
+ &dim_sample);
+ net_dim(&cpr->dim, dim_sample);
+ }
return work_done;
}
@@ -2774,7 +2802,7 @@ static void bnxt_free_one_rx_ring_skbs(struct bnxt *bp, int ring_nr)
tpa_info->data = NULL;
- kfree(data);
+ skb_free_frag(data);
}
skip_rx_tpa_free:
@@ -2800,7 +2828,7 @@ static void bnxt_free_one_rx_ring_skbs(struct bnxt *bp, int ring_nr)
dma_unmap_single_attrs(&pdev->dev, mapping,
bp->rx_buf_use_size, bp->rx_dir,
DMA_ATTR_WEAK_ORDERING);
- kfree(data);
+ skb_free_frag(data);
}
}
@@ -3504,7 +3532,7 @@ static int bnxt_alloc_one_rx_ring(struct bnxt *bp, int ring_nr)
u8 *data;
for (i = 0; i < bp->max_tpa; i++) {
- data = __bnxt_alloc_rx_data(bp, &mapping, GFP_KERNEL);
+ data = __bnxt_alloc_rx_frag(bp, &mapping, GFP_KERNEL);
if (!data)
return -ENOMEM;
@@ -6474,8 +6502,8 @@ static void bnxt_hwrm_set_coal_params(struct bnxt *bp,
struct hwrm_ring_cmpl_ring_cfg_aggint_params_input *req)
{
struct bnxt_coal_cap *coal_cap = &bp->coal_cap;
+ u16 val, tmr, max, flags = hw_coal->flags;
u32 cmpl_params = coal_cap->cmpl_params;
- u16 val, tmr, max, flags = 0;
max = hw_coal->bufs_per_record * 128;
if (hw_coal->budget)
@@ -6518,8 +6546,6 @@ static void bnxt_hwrm_set_coal_params(struct bnxt *bp,
cpu_to_le16(BNXT_COAL_CMPL_AGGR_TMR_DURING_INT_ENABLE);
}
- if (cmpl_params & RING_AGGINT_QCAPS_RESP_CMPL_PARAMS_TIMER_RESET)
- flags |= RING_CMPL_RING_CFG_AGGINT_PARAMS_REQ_FLAGS_TIMER_RESET;
if ((cmpl_params & RING_AGGINT_QCAPS_RESP_CMPL_PARAMS_RING_IDLE) &&
hw_coal->idle_thresh && hw_coal->coal_ticks < hw_coal->idle_thresh)
flags |= RING_CMPL_RING_CFG_AGGINT_PARAMS_REQ_FLAGS_RING_IDLE;
@@ -7668,19 +7694,6 @@ static void __bnxt_map_fw_health_reg(struct bnxt *bp, u32 reg)
BNXT_FW_HEALTH_WIN_MAP_OFF);
}
-bool bnxt_is_fw_healthy(struct bnxt *bp)
-{
- if (bp->fw_health && bp->fw_health->status_reliable) {
- u32 fw_status;
-
- fw_status = bnxt_fw_health_readl(bp, BNXT_FW_HEALTH_REG);
- if (fw_status && !BNXT_FW_IS_HEALTHY(fw_status))
- return false;
- }
-
- return true;
-}
-
static void bnxt_inv_fw_health_reg(struct bnxt *bp)
{
struct bnxt_fw_health *fw_health = bp->fw_health;
@@ -8008,6 +8021,12 @@ static int bnxt_hwrm_ver_get(struct bnxt *bp)
bp->hwrm_cmd_timeout = le16_to_cpu(resp->def_req_timeout);
if (!bp->hwrm_cmd_timeout)
bp->hwrm_cmd_timeout = DFLT_HWRM_CMD_TIMEOUT;
+ bp->hwrm_cmd_max_timeout = le16_to_cpu(resp->max_req_timeout) * 1000;
+ if (!bp->hwrm_cmd_max_timeout)
+ bp->hwrm_cmd_max_timeout = HWRM_CMD_MAX_TIMEOUT;
+ else if (bp->hwrm_cmd_max_timeout > HWRM_CMD_MAX_TIMEOUT)
+ netdev_warn(bp->dev, "Device requests max timeout of %d seconds, may trigger hung task watchdog\n",
+ bp->hwrm_cmd_max_timeout / 1000);
if (resp->hwrm_intf_maj_8b >= 1) {
bp->hwrm_max_req_len = le16_to_cpu(resp->max_req_win_len);
@@ -8611,7 +8630,10 @@ static int bnxt_init_chip(struct bnxt *bp, bool irq_re_init)
/* Filter for default vnic 0 */
rc = bnxt_hwrm_set_vnic_filter(bp, 0, 0, bp->dev->dev_addr);
if (rc) {
- netdev_err(bp->dev, "HWRM vnic filter failure rc: %x\n", rc);
+ if (BNXT_VF(bp) && rc == -ENODEV)
+ netdev_err(bp->dev, "Cannot configure L2 filter while PF is unavailable\n");
+ else
+ netdev_err(bp->dev, "HWRM vnic filter failure rc: %x\n", rc);
goto err_out;
}
vnic->uc_filter_count = 1;
@@ -9404,6 +9426,10 @@ int bnxt_update_link(struct bnxt *bp, bool chng_link_state)
rc = hwrm_req_send(bp, req);
if (rc) {
hwrm_req_drop(bp, req);
+ if (BNXT_VF(bp) && rc == -ENODEV) {
+ netdev_warn(bp->dev, "Cannot obtain link state while PF unavailable.\n");
+ rc = 0;
+ }
return rc;
}
@@ -10802,12 +10828,21 @@ static int bnxt_cfg_rx_mode(struct bnxt *bp)
for (i = 1, off = 0; i < vnic->uc_filter_count; i++, off += ETH_ALEN) {
rc = bnxt_hwrm_set_vnic_filter(bp, 0, i, vnic->uc_list + off);
if (rc) {
- netdev_err(bp->dev, "HWRM vnic filter failure rc: %x\n",
- rc);
+ if (BNXT_VF(bp) && rc == -ENODEV) {
+ if (!test_and_set_bit(BNXT_STATE_L2_FILTER_RETRY, &bp->state))
+ netdev_warn(bp->dev, "Cannot configure L2 filters while PF is unavailable, will retry\n");
+ else
+ netdev_dbg(bp->dev, "PF still unavailable while configuring L2 filters.\n");
+ rc = 0;
+ } else {
+ netdev_err(bp->dev, "HWRM vnic filter failure rc: %x\n", rc);
+ }
vnic->uc_filter_count = i;
return rc;
}
}
+ if (test_and_clear_bit(BNXT_STATE_L2_FILTER_RETRY, &bp->state))
+ netdev_notice(bp->dev, "Retry of L2 filter configuration successful.\n");
skip_uc:
if ((vnic->rx_mask & CFA_L2_SET_RX_MASK_REQ_MASK_PROMISCUOUS) &&
@@ -11372,6 +11407,11 @@ static void bnxt_timer(struct timer_list *t)
}
}
+ if (test_bit(BNXT_STATE_L2_FILTER_RETRY, &bp->state)) {
+ set_bit(BNXT_RX_MASK_SP_EVENT, &bp->sp_event);
+ bnxt_queue_sp_work(bp);
+ }
+
if ((bp->flags & BNXT_FLAG_CHIP_P5) && !bp->chip_rev &&
netif_carrier_ok(dev)) {
set_bit(BNXT_RING_COAL_NOW_SP_EVENT, &bp->sp_event);
@@ -11875,7 +11915,13 @@ static void bnxt_cleanup_pci(struct bnxt *bp)
static void bnxt_init_dflt_coal(struct bnxt *bp)
{
+ struct bnxt_coal_cap *coal_cap = &bp->coal_cap;
struct bnxt_coal *coal;
+ u16 flags = 0;
+
+ if (coal_cap->cmpl_params &
+ RING_AGGINT_QCAPS_RESP_CMPL_PARAMS_TIMER_RESET)
+ flags |= RING_CMPL_RING_CFG_AGGINT_PARAMS_REQ_FLAGS_TIMER_RESET;
/* Tick values in micro seconds.
* 1 coal_buf x bufs_per_record = 1 completion record.
@@ -11888,6 +11934,7 @@ static void bnxt_init_dflt_coal(struct bnxt *bp)
coal->idle_thresh = 50;
coal->bufs_per_record = 2;
coal->budget = 64; /* NAPI budget */
+ coal->flags = flags;
coal = &bp->tx_coal;
coal->coal_ticks = 28;
@@ -11895,6 +11942,7 @@ static void bnxt_init_dflt_coal(struct bnxt *bp)
coal->coal_ticks_irq = 2;
coal->coal_bufs_irq = 2;
coal->bufs_per_record = 1;
+ coal->flags = flags;
bp->stats_coal_ticks = BNXT_DEF_STATS_COAL_TICKS;
}
@@ -12381,8 +12429,6 @@ static int bnxt_init_board(struct pci_dev *pdev, struct net_device *dev)
bp->rx_ring_size = BNXT_DEFAULT_RX_RING_SIZE;
bp->tx_ring_size = BNXT_DEFAULT_TX_RING_SIZE;
- bnxt_init_dflt_coal(bp);
-
timer_setup(&bp->timer, bnxt_timer, 0);
bp->current_interval = BNXT_TIMER_INTERVAL;
@@ -13072,7 +13118,7 @@ static int bnxt_set_dflt_rings(struct bnxt *bp, bool sh)
bp->tx_nr_rings = bp->tx_nr_rings_per_tc;
rc = __bnxt_reserve_rings(bp);
- if (rc)
+ if (rc && rc != -ENODEV)
netdev_warn(bp->dev, "Unable to reserve tx rings\n");
bp->tx_nr_rings_per_tc = bp->tx_nr_rings;
if (sh)
@@ -13081,7 +13127,7 @@ static int bnxt_set_dflt_rings(struct bnxt *bp, bool sh)
/* Rings may have been trimmed, re-reserve the trimmed rings. */
if (bnxt_need_reserve_rings(bp)) {
rc = __bnxt_reserve_rings(bp);
- if (rc)
+ if (rc && rc != -ENODEV)
netdev_warn(bp->dev, "2nd rings reservation failed.\n");
bp->tx_nr_rings_per_tc = bp->tx_nr_rings;
}
@@ -13107,7 +13153,10 @@ static int bnxt_init_dflt_ring_mode(struct bnxt *bp)
bnxt_clear_int_mode(bp);
rc = bnxt_set_dflt_rings(bp, true);
if (rc) {
- netdev_err(bp->dev, "Not enough rings available.\n");
+ if (BNXT_VF(bp) && rc == -ENODEV)
+ netdev_err(bp->dev, "Cannot configure VF rings while PF is unavailable.\n");
+ else
+ netdev_err(bp->dev, "Not enough rings available.\n");
goto init_dflt_ring_err;
}
rc = bnxt_init_int_mode(bp);
@@ -13395,13 +13444,19 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
bnxt_set_ring_params(bp);
rc = bnxt_set_dflt_rings(bp, true);
if (rc) {
- netdev_err(bp->dev, "Not enough rings available.\n");
- rc = -ENOMEM;
+ if (BNXT_VF(bp) && rc == -ENODEV) {
+ netdev_err(bp->dev, "Cannot configure VF rings while PF is unavailable.\n");
+ } else {
+ netdev_err(bp->dev, "Not enough rings available.\n");
+ rc = -ENOMEM;
+ }
goto init_err_pci_clean;
}
bnxt_fw_init_one_p3(bp);
+ bnxt_init_dflt_coal(bp);
+
if (dev->hw_features & BNXT_HW_FEATURE_VLAN_ALL_RX)
bp->flags |= BNXT_FLAG_STRIP_VLAN;
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
index 4c9507d..440dfeb 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
@@ -847,6 +847,7 @@ struct bnxt_coal {
u16 idle_thresh;
u8 bufs_per_record;
u8 budget;
+ u16 flags;
};
struct bnxt_tpa_info {
@@ -1915,6 +1916,7 @@ struct bnxt {
#define BNXT_STATE_DRV_REGISTERED 7
#define BNXT_STATE_PCI_CHANNEL_IO_FROZEN 8
#define BNXT_STATE_NAPI_DISABLED 9
+#define BNXT_STATE_L2_FILTER_RETRY 10
#define BNXT_STATE_FW_ACTIVATE 11
#define BNXT_STATE_RECOVER 12
#define BNXT_STATE_FW_NON_FATAL_COND 13
@@ -1985,7 +1987,8 @@ struct bnxt {
u16 hwrm_max_req_len;
u16 hwrm_max_ext_req_len;
- int hwrm_cmd_timeout;
+ unsigned int hwrm_cmd_timeout;
+ unsigned int hwrm_cmd_max_timeout;
struct mutex hwrm_cmd_lock; /* serialize hwrm messages */
struct hwrm_ver_get_output ver_resp;
#define FW_VER_STR_LEN 32
@@ -2302,7 +2305,6 @@ int bnxt_cancel_reservations(struct bnxt *bp, bool fw_reset);
int bnxt_hwrm_alloc_wol_fltr(struct bnxt *bp);
int bnxt_hwrm_free_wol_fltr(struct bnxt *bp);
int bnxt_hwrm_func_resc_qcaps(struct bnxt *bp, bool all);
-bool bnxt_is_fw_healthy(struct bnxt *bp);
int bnxt_hwrm_fw_set_time(struct bnxt *);
int bnxt_open_nic(struct bnxt *, bool, bool);
int bnxt_half_open_nic(struct bnxt *bp);
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_coredump.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_coredump.c
index d3cb2f2..c067898 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_coredump.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_coredump.c
@@ -32,7 +32,7 @@ static int bnxt_hwrm_dbg_dma_data(struct bnxt *bp, void *msg,
return -ENOMEM;
}
- hwrm_req_timeout(bp, msg, HWRM_COREDUMP_TIMEOUT);
+ hwrm_req_timeout(bp, msg, bp->hwrm_cmd_max_timeout);
cmn_resp = hwrm_req_hold(bp, msg);
resp = cmn_resp;
@@ -125,7 +125,7 @@ static int bnxt_hwrm_dbg_coredump_initiate(struct bnxt *bp, u16 component_id,
if (rc)
return rc;
- hwrm_req_timeout(bp, req, HWRM_COREDUMP_TIMEOUT);
+ hwrm_req_timeout(bp, req, bp->hwrm_cmd_max_timeout);
req->component_id = cpu_to_le16(component_id);
req->segment_id = cpu_to_le16(segment_id);
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_devlink.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_devlink.c
index 951c4c5..4da31b1 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_devlink.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_devlink.c
@@ -9,6 +9,7 @@
#include <linux/pci.h>
#include <linux/netdevice.h>
+#include <linux/vmalloc.h>
#include <net/devlink.h>
#include "bnxt_hsi.h"
#include "bnxt.h"
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
index 8188d55..003330e 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
@@ -31,9 +31,6 @@
#include "bnxt_nvm_defs.h" /* NVRAM content constant and structure defs */
#include "bnxt_fw_hdr.h" /* Firmware hdr constant and structure defs */
#include "bnxt_coredump.h"
-#define FLASH_NVRAM_TIMEOUT ((HWRM_CMD_TIMEOUT) * 100)
-#define FLASH_PACKAGE_TIMEOUT ((HWRM_CMD_TIMEOUT) * 200)
-#define INSTALL_PACKAGE_TIMEOUT ((HWRM_CMD_TIMEOUT) * 200)
static u32 bnxt_get_msglevel(struct net_device *dev)
{
@@ -68,6 +65,9 @@ static int bnxt_get_coalesce(struct net_device *dev,
coal->rx_max_coalesced_frames = hw_coal->coal_bufs / mult;
coal->rx_coalesce_usecs_irq = hw_coal->coal_ticks_irq;
coal->rx_max_coalesced_frames_irq = hw_coal->coal_bufs_irq / mult;
+ if (hw_coal->flags &
+ RING_CMPL_RING_CFG_AGGINT_PARAMS_REQ_FLAGS_TIMER_RESET)
+ kernel_coal->use_cqe_mode_rx = true;
hw_coal = &bp->tx_coal;
mult = hw_coal->bufs_per_record;
@@ -75,6 +75,9 @@ static int bnxt_get_coalesce(struct net_device *dev,
coal->tx_max_coalesced_frames = hw_coal->coal_bufs / mult;
coal->tx_coalesce_usecs_irq = hw_coal->coal_ticks_irq;
coal->tx_max_coalesced_frames_irq = hw_coal->coal_bufs_irq / mult;
+ if (hw_coal->flags &
+ RING_CMPL_RING_CFG_AGGINT_PARAMS_REQ_FLAGS_TIMER_RESET)
+ kernel_coal->use_cqe_mode_tx = true;
coal->stats_block_coalesce_usecs = bp->stats_coal_ticks;
@@ -101,12 +104,22 @@ static int bnxt_set_coalesce(struct net_device *dev,
}
}
+ if ((kernel_coal->use_cqe_mode_rx || kernel_coal->use_cqe_mode_tx) &&
+ !(bp->coal_cap.cmpl_params &
+ RING_AGGINT_QCAPS_RESP_CMPL_PARAMS_TIMER_RESET))
+ return -EOPNOTSUPP;
+
hw_coal = &bp->rx_coal;
mult = hw_coal->bufs_per_record;
hw_coal->coal_ticks = coal->rx_coalesce_usecs;
hw_coal->coal_bufs = coal->rx_max_coalesced_frames * mult;
hw_coal->coal_ticks_irq = coal->rx_coalesce_usecs_irq;
hw_coal->coal_bufs_irq = coal->rx_max_coalesced_frames_irq * mult;
+ hw_coal->flags &=
+ ~RING_CMPL_RING_CFG_AGGINT_PARAMS_REQ_FLAGS_TIMER_RESET;
+ if (kernel_coal->use_cqe_mode_rx)
+ hw_coal->flags |=
+ RING_CMPL_RING_CFG_AGGINT_PARAMS_REQ_FLAGS_TIMER_RESET;
hw_coal = &bp->tx_coal;
mult = hw_coal->bufs_per_record;
@@ -114,6 +127,11 @@ static int bnxt_set_coalesce(struct net_device *dev,
hw_coal->coal_bufs = coal->tx_max_coalesced_frames * mult;
hw_coal->coal_ticks_irq = coal->tx_coalesce_usecs_irq;
hw_coal->coal_bufs_irq = coal->tx_max_coalesced_frames_irq * mult;
+ hw_coal->flags &=
+ ~RING_CMPL_RING_CFG_AGGINT_PARAMS_REQ_FLAGS_TIMER_RESET;
+ if (kernel_coal->use_cqe_mode_tx)
+ hw_coal->flags |=
+ RING_CMPL_RING_CFG_AGGINT_PARAMS_REQ_FLAGS_TIMER_RESET;
if (bp->stats_coal_ticks != coal->stats_block_coalesce_usecs) {
u32 stats_ticks = coal->stats_block_coalesce_usecs;
@@ -775,7 +793,9 @@ static void bnxt_get_strings(struct net_device *dev, u32 stringset, u8 *buf)
}
static void bnxt_get_ringparam(struct net_device *dev,
- struct ethtool_ringparam *ering)
+ struct ethtool_ringparam *ering,
+ struct kernel_ethtool_ringparam *kernel_ering,
+ struct netlink_ext_ack *extack)
{
struct bnxt *bp = netdev_priv(dev);
@@ -794,7 +814,9 @@ static void bnxt_get_ringparam(struct net_device *dev,
}
static int bnxt_set_ringparam(struct net_device *dev,
- struct ethtool_ringparam *ering)
+ struct ethtool_ringparam *ering,
+ struct kernel_ethtool_ringparam *kernel_ering,
+ struct netlink_ext_ack *extack)
{
struct bnxt *bp = netdev_priv(dev);
@@ -2169,7 +2191,7 @@ static int bnxt_flash_nvram(struct net_device *dev, u16 dir_type,
req->host_src_addr = cpu_to_le64(dma_handle);
}
- hwrm_req_timeout(bp, req, FLASH_NVRAM_TIMEOUT);
+ hwrm_req_timeout(bp, req, bp->hwrm_cmd_max_timeout);
req->dir_type = cpu_to_le16(dir_type);
req->dir_ordinal = cpu_to_le16(dir_ordinal);
req->dir_ext = cpu_to_le16(dir_ext);
@@ -2515,8 +2537,8 @@ int bnxt_flash_package_from_fw_obj(struct net_device *dev, const struct firmware
return rc;
}
- hwrm_req_timeout(bp, modify, FLASH_PACKAGE_TIMEOUT);
- hwrm_req_timeout(bp, install, INSTALL_PACKAGE_TIMEOUT);
+ hwrm_req_timeout(bp, modify, bp->hwrm_cmd_max_timeout);
+ hwrm_req_timeout(bp, install, bp->hwrm_cmd_max_timeout);
hwrm_req_hold(bp, modify);
modify->host_src_addr = cpu_to_le64(dma_handle);
@@ -3917,7 +3939,8 @@ const struct ethtool_ops bnxt_ethtool_ops = {
ETHTOOL_COALESCE_USECS_IRQ |
ETHTOOL_COALESCE_MAX_FRAMES_IRQ |
ETHTOOL_COALESCE_STATS_BLOCK_USECS |
- ETHTOOL_COALESCE_USE_ADAPTIVE_RX,
+ ETHTOOL_COALESCE_USE_ADAPTIVE_RX |
+ ETHTOOL_COALESCE_USE_CQE,
.get_link_ksettings = bnxt_get_link_ksettings,
.set_link_ksettings = bnxt_set_link_ksettings,
.get_fec_stats = bnxt_get_fec_stats,
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_hwrm.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_hwrm.c
index bb7327b..566c948 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_hwrm.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_hwrm.c
@@ -359,6 +359,8 @@ static int __hwrm_to_stderr(u32 hwrm_err)
return -EAGAIN;
case HWRM_ERR_CODE_CMD_NOT_SUPPORTED:
return -EOPNOTSUPP;
+ case HWRM_ERR_CODE_PF_UNAVAILABLE:
+ return -ENODEV;
default:
return -EIO;
}
@@ -416,6 +418,44 @@ hwrm_update_token(struct bnxt *bp, u16 seq_id, enum bnxt_hwrm_wait_state state)
netdev_err(bp->dev, "Invalid hwrm seq id %d\n", seq_id);
}
+static void hwrm_req_dbg(struct bnxt *bp, struct input *req)
+{
+ u32 ring = le16_to_cpu(req->cmpl_ring);
+ u32 type = le16_to_cpu(req->req_type);
+ u32 tgt = le16_to_cpu(req->target_id);
+ u32 seq = le16_to_cpu(req->seq_id);
+ char opt[32] = "\n";
+
+ if (unlikely(ring != (u16)BNXT_HWRM_NO_CMPL_RING))
+ snprintf(opt, 16, " ring %d\n", ring);
+
+ if (unlikely(tgt != BNXT_HWRM_TARGET))
+ snprintf(opt + strlen(opt) - 1, 16, " tgt 0x%x\n", tgt);
+
+ netdev_dbg(bp->dev, "sent hwrm req_type 0x%x seq id 0x%x%s",
+ type, seq, opt);
+}
+
+#define hwrm_err(bp, ctx, fmt, ...) \
+ do { \
+ if ((ctx)->flags & BNXT_HWRM_CTX_SILENT) \
+ netdev_dbg((bp)->dev, fmt, __VA_ARGS__); \
+ else \
+ netdev_err((bp)->dev, fmt, __VA_ARGS__); \
+ } while (0)
+
+static bool hwrm_wait_must_abort(struct bnxt *bp, u32 req_type, u32 *fw_status)
+{
+ if (req_type == HWRM_VER_GET)
+ return false;
+
+ if (!bp->fw_health || !bp->fw_health->status_reliable)
+ return false;
+
+ *fw_status = bnxt_fw_health_readl(bp, BNXT_FW_HEALTH_REG);
+ return *fw_status && !BNXT_FW_IS_HEALTHY(*fw_status);
+}
+
static int __hwrm_send(struct bnxt *bp, struct bnxt_hwrm_ctx *ctx)
{
u32 doorbell_offset = BNXT_GRCPF_REG_CHIMP_COMM_TRIGGER;
@@ -427,8 +467,8 @@ static int __hwrm_send(struct bnxt *bp, struct bnxt_hwrm_ctx *ctx)
unsigned int i, timeout, tmo_count;
u32 *data = (u32 *)ctx->req;
u32 msg_len = ctx->req_len;
+ u32 req_type, sts;
int rc = -EBUSY;
- u32 req_type;
u16 len = 0;
u8 *valid;
@@ -436,8 +476,11 @@ static int __hwrm_send(struct bnxt *bp, struct bnxt_hwrm_ctx *ctx)
memset(ctx->resp, 0, PAGE_SIZE);
req_type = le16_to_cpu(ctx->req->req_type);
- if (BNXT_NO_FW_ACCESS(bp) && req_type != HWRM_FUNC_RESET)
+ if (BNXT_NO_FW_ACCESS(bp) && req_type != HWRM_FUNC_RESET) {
+ netdev_dbg(bp->dev, "hwrm req_type 0x%x skipped, FW channel down\n",
+ req_type);
goto exit;
+ }
if (msg_len > BNXT_HWRM_MAX_REQ_LEN &&
msg_len > bp->hwrm_max_ext_req_len) {
@@ -490,13 +533,15 @@ static int __hwrm_send(struct bnxt *bp, struct bnxt_hwrm_ctx *ctx)
/* Ring channel doorbell */
writel(1, bp->bar0 + doorbell_offset);
+ hwrm_req_dbg(bp, ctx->req);
+
if (!pci_is_enabled(bp->pdev)) {
rc = -ENODEV;
goto exit;
}
/* Limit timeout to an upper limit */
- timeout = min_t(uint, ctx->timeout, HWRM_CMD_MAX_TIMEOUT);
+ timeout = min(ctx->timeout, bp->hwrm_cmd_max_timeout ?: HWRM_CMD_MAX_TIMEOUT);
/* convert timeout to usec */
timeout *= 1000;
@@ -523,17 +568,19 @@ static int __hwrm_send(struct bnxt *bp, struct bnxt_hwrm_ctx *ctx)
usleep_range(HWRM_SHORT_MIN_TIMEOUT,
HWRM_SHORT_MAX_TIMEOUT);
} else {
- if (HWRM_WAIT_MUST_ABORT(bp, ctx))
- break;
+ if (hwrm_wait_must_abort(bp, req_type, &sts)) {
+ hwrm_err(bp, ctx, "Resp cmpl intr abandoning msg: 0x%x due to firmware status: 0x%x\n",
+ req_type, sts);
+ goto exit;
+ }
usleep_range(HWRM_MIN_TIMEOUT,
HWRM_MAX_TIMEOUT);
}
}
if (READ_ONCE(token->state) != BNXT_HWRM_COMPLETE) {
- if (!(ctx->flags & BNXT_HWRM_CTX_SILENT))
- netdev_err(bp->dev, "Resp cmpl intr err msg: 0x%x\n",
- le16_to_cpu(ctx->req->req_type));
+ hwrm_err(bp, ctx, "Resp cmpl intr err msg: 0x%x\n",
+ req_type);
goto exit;
}
len = le16_to_cpu(READ_ONCE(ctx->resp->resp_len));
@@ -565,7 +612,7 @@ static int __hwrm_send(struct bnxt *bp, struct bnxt_hwrm_ctx *ctx)
if (resp_seq != seen_out_of_seq) {
netdev_warn(bp->dev, "Discarding out of seq response: 0x%x for msg {0x%x 0x%x}\n",
le16_to_cpu(resp_seq),
- le16_to_cpu(ctx->req->req_type),
+ req_type,
le16_to_cpu(ctx->req->seq_id));
seen_out_of_seq = resp_seq;
}
@@ -576,20 +623,22 @@ static int __hwrm_send(struct bnxt *bp, struct bnxt_hwrm_ctx *ctx)
usleep_range(HWRM_SHORT_MIN_TIMEOUT,
HWRM_SHORT_MAX_TIMEOUT);
} else {
- if (HWRM_WAIT_MUST_ABORT(bp, ctx))
- goto timeout_abort;
+ if (hwrm_wait_must_abort(bp, req_type, &sts)) {
+ hwrm_err(bp, ctx, "Abandoning msg {0x%x 0x%x} len: %d due to firmware status: 0x%x\n",
+ req_type,
+ le16_to_cpu(ctx->req->seq_id),
+ len, sts);
+ goto exit;
+ }
usleep_range(HWRM_MIN_TIMEOUT,
HWRM_MAX_TIMEOUT);
}
}
if (i >= tmo_count) {
-timeout_abort:
- if (!(ctx->flags & BNXT_HWRM_CTX_SILENT))
- netdev_err(bp->dev, "Error (timeout: %u) msg {0x%x 0x%x} len:%d\n",
- hwrm_total_timeout(i),
- le16_to_cpu(ctx->req->req_type),
- le16_to_cpu(ctx->req->seq_id), len);
+ hwrm_err(bp, ctx, "Error (timeout: %u) msg {0x%x 0x%x} len:%d\n",
+ hwrm_total_timeout(i), req_type,
+ le16_to_cpu(ctx->req->seq_id), len);
goto exit;
}
@@ -604,12 +653,9 @@ static int __hwrm_send(struct bnxt *bp, struct bnxt_hwrm_ctx *ctx)
}
if (j >= HWRM_VALID_BIT_DELAY_USEC) {
- if (!(ctx->flags & BNXT_HWRM_CTX_SILENT))
- netdev_err(bp->dev, "Error (timeout: %u) msg {0x%x 0x%x} len:%d v:%d\n",
- hwrm_total_timeout(i),
- le16_to_cpu(ctx->req->req_type),
- le16_to_cpu(ctx->req->seq_id), len,
- *valid);
+ hwrm_err(bp, ctx, "Error (timeout: %u) msg {0x%x 0x%x} len:%d v:%d\n",
+ hwrm_total_timeout(i), req_type,
+ le16_to_cpu(ctx->req->seq_id), len, *valid);
goto exit;
}
}
@@ -620,11 +666,12 @@ static int __hwrm_send(struct bnxt *bp, struct bnxt_hwrm_ctx *ctx)
*/
*valid = 0;
rc = le16_to_cpu(ctx->resp->error_code);
- if (rc && !(ctx->flags & BNXT_HWRM_CTX_SILENT)) {
- netdev_err(bp->dev, "hwrm req_type 0x%x seq id 0x%x error 0x%x\n",
- le16_to_cpu(ctx->resp->req_type),
- le16_to_cpu(ctx->resp->seq_id), rc);
- }
+ if (rc == HWRM_ERR_CODE_BUSY && !(ctx->flags & BNXT_HWRM_CTX_SILENT))
+ netdev_warn(bp->dev, "FW returned busy, hwrm req_type 0x%x\n",
+ req_type);
+ else if (rc && rc != HWRM_ERR_CODE_PF_UNAVAILABLE)
+ hwrm_err(bp, ctx, "hwrm req_type 0x%x seq id 0x%x error 0x%x\n",
+ req_type, token->seq_id, rc);
rc = __hwrm_to_stderr(rc);
exit:
if (token)
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_hwrm.h b/drivers/net/ethernet/broadcom/bnxt/bnxt_hwrm.h
index 4d17f0d..d52bd2d 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_hwrm.h
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_hwrm.h
@@ -58,11 +58,10 @@ void hwrm_update_token(struct bnxt *bp, u16 seq, enum bnxt_hwrm_wait_state s);
#define BNXT_HWRM_MAX_REQ_LEN (bp->hwrm_max_req_len)
#define BNXT_HWRM_SHORT_REQ_LEN sizeof(struct hwrm_short_input)
-#define HWRM_CMD_MAX_TIMEOUT 40000
+#define HWRM_CMD_MAX_TIMEOUT 40000U
#define SHORT_HWRM_CMD_TIMEOUT 20
#define HWRM_CMD_TIMEOUT (bp->hwrm_cmd_timeout)
#define HWRM_RESET_TIMEOUT ((HWRM_CMD_TIMEOUT) * 4)
-#define HWRM_COREDUMP_TIMEOUT ((HWRM_CMD_TIMEOUT) * 12)
#define BNXT_HWRM_TARGET 0xffff
#define BNXT_HWRM_NO_CMPL_RING -1
#define BNXT_HWRM_REQ_MAX_SIZE 128
@@ -83,10 +82,6 @@ void hwrm_update_token(struct bnxt *bp, u16 seq, enum bnxt_hwrm_wait_state s);
#define HWRM_MIN_TIMEOUT 25
#define HWRM_MAX_TIMEOUT 40
-#define HWRM_WAIT_MUST_ABORT(bp, ctx) \
- (le16_to_cpu((ctx)->req->req_type) != HWRM_VER_GET && \
- !bnxt_is_fw_healthy(bp))
-
static inline unsigned int hwrm_total_timeout(unsigned int n)
{
return n <= HWRM_SHORT_TIMEOUT_COUNTER ? n * HWRM_SHORT_MIN_TIMEOUT :
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ptp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ptp.c
index 8388be1..4852096 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ptp.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ptp.c
@@ -417,9 +417,6 @@ int bnxt_hwtstamp_set(struct net_device *dev, struct ifreq *ifr)
if (copy_from_user(&stmpconf, ifr->ifr_data, sizeof(stmpconf)))
return -EFAULT;
- if (stmpconf.flags)
- return -EINVAL;
-
if (stmpconf.tx_type != HWTSTAMP_TX_ON &&
stmpconf.tx_type != HWTSTAMP_TX_OFF)
return -ERANGE;
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_tc.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_tc.c
index 1471b61..d8afcf8 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_tc.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_tc.c
@@ -1962,7 +1962,7 @@ static int bnxt_tc_setup_indr_cb(struct net_device *netdev, struct Qdisc *sch, v
void *data,
void (*cleanup)(struct flow_block_cb *block_cb))
{
- if (!bnxt_is_netdev_indr_offload(netdev))
+ if (!netdev || !bnxt_is_netdev_indr_offload(netdev))
return -EOPNOTSUPP;
switch (type) {
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
index c8083df..52fad0f 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
@@ -195,7 +195,7 @@ bool bnxt_rx_xdp(struct bnxt *bp, struct bnxt_rx_ring_info *rxr, u16 cons,
*event |= BNXT_REDIRECT_EVENT;
break;
default:
- bpf_warn_invalid_xdp_action(act);
+ bpf_warn_invalid_xdp_action(bp->dev, xdp_prog, act);
fallthrough;
case XDP_ABORTED:
trace_xdp_exception(bp->dev, xdp_prog, act);
diff --git a/drivers/net/ethernet/broadcom/tg3.c b/drivers/net/ethernet/broadcom/tg3.c
index 85ca390..c28f8cc 100644
--- a/drivers/net/ethernet/broadcom/tg3.c
+++ b/drivers/net/ethernet/broadcom/tg3.c
@@ -12390,7 +12390,10 @@ static int tg3_nway_reset(struct net_device *dev)
return r;
}
-static void tg3_get_ringparam(struct net_device *dev, struct ethtool_ringparam *ering)
+static void tg3_get_ringparam(struct net_device *dev,
+ struct ethtool_ringparam *ering,
+ struct kernel_ethtool_ringparam *kernel_ering,
+ struct netlink_ext_ack *extack)
{
struct tg3 *tp = netdev_priv(dev);
@@ -12411,7 +12414,10 @@ static void tg3_get_ringparam(struct net_device *dev, struct ethtool_ringparam *
ering->tx_pending = tp->napi[0].tx_pending;
}
-static int tg3_set_ringparam(struct net_device *dev, struct ethtool_ringparam *ering)
+static int tg3_set_ringparam(struct net_device *dev,
+ struct ethtool_ringparam *ering,
+ struct kernel_ethtool_ringparam *kernel_ering,
+ struct netlink_ext_ack *extack)
{
struct tg3 *tp = netdev_priv(dev);
int i, irq_sync = 0, err = 0;
@@ -13800,9 +13806,6 @@ static int tg3_hwtstamp_set(struct net_device *dev, struct ifreq *ifr)
if (copy_from_user(&stmpconf, ifr->ifr_data, sizeof(stmpconf)))
return -EFAULT;
- if (stmpconf.flags)
- return -EINVAL;
-
if (stmpconf.tx_type != HWTSTAMP_TX_ON &&
stmpconf.tx_type != HWTSTAMP_TX_OFF)
return -ERANGE;
diff --git a/drivers/net/ethernet/brocade/bna/bnad.c b/drivers/net/ethernet/brocade/bna/bnad.c
index bbdc829..f1d2c4c 100644
--- a/drivers/net/ethernet/brocade/bna/bnad.c
+++ b/drivers/net/ethernet/brocade/bna/bnad.c
@@ -3421,7 +3421,7 @@ static const struct net_device_ops bnad_netdev_ops = {
};
static void
-bnad_netdev_init(struct bnad *bnad, bool using_dac)
+bnad_netdev_init(struct bnad *bnad)
{
struct net_device *netdev = bnad->netdev;
@@ -3434,10 +3434,8 @@ bnad_netdev_init(struct bnad *bnad, bool using_dac)
NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM |
NETIF_F_TSO | NETIF_F_TSO6;
- netdev->features |= netdev->hw_features | NETIF_F_HW_VLAN_CTAG_FILTER;
-
- if (using_dac)
- netdev->features |= NETIF_F_HIGHDMA;
+ netdev->features |= netdev->hw_features | NETIF_F_HW_VLAN_CTAG_FILTER |
+ NETIF_F_HIGHDMA;
netdev->mem_start = bnad->mmio_start;
netdev->mem_end = bnad->mmio_start + bnad->mmio_len - 1;
@@ -3544,8 +3542,7 @@ bnad_lock_uninit(struct bnad *bnad)
/* PCI Initialization */
static int
-bnad_pci_init(struct bnad *bnad,
- struct pci_dev *pdev, bool *using_dac)
+bnad_pci_init(struct bnad *bnad, struct pci_dev *pdev)
{
int err;
@@ -3555,14 +3552,9 @@ bnad_pci_init(struct bnad *bnad,
err = pci_request_regions(pdev, BNAD_NAME);
if (err)
goto disable_device;
- if (!dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64))) {
- *using_dac = true;
- } else {
- err = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32));
- if (err)
- goto release_regions;
- *using_dac = false;
- }
+ err = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64));
+ if (err)
+ goto release_regions;
pci_set_master(pdev);
return 0;
@@ -3585,7 +3577,6 @@ static int
bnad_pci_probe(struct pci_dev *pdev,
const struct pci_device_id *pcidev_id)
{
- bool using_dac;
int err;
struct bnad *bnad;
struct bna *bna;
@@ -3615,13 +3606,8 @@ bnad_pci_probe(struct pci_dev *pdev,
bnad->id = atomic_inc_return(&bna_id) - 1;
mutex_lock(&bnad->conf_mutex);
- /*
- * PCI initialization
- * Output : using_dac = 1 for 64 bit DMA
- * = 0 for 32 bit DMA
- */
- using_dac = false;
- err = bnad_pci_init(bnad, pdev, &using_dac);
+ /* PCI initialization */
+ err = bnad_pci_init(bnad, pdev);
if (err)
goto unlock_mutex;
@@ -3634,7 +3620,7 @@ bnad_pci_probe(struct pci_dev *pdev,
goto pci_uninit;
/* Initialize netdev structure, set up ethtool ops */
- bnad_netdev_init(bnad, using_dac);
+ bnad_netdev_init(bnad);
/* Set link to down state */
netif_carrier_off(netdev);
diff --git a/drivers/net/ethernet/brocade/bna/bnad_ethtool.c b/drivers/net/ethernet/brocade/bna/bnad_ethtool.c
index 391b85f..8aca768 100644
--- a/drivers/net/ethernet/brocade/bna/bnad_ethtool.c
+++ b/drivers/net/ethernet/brocade/bna/bnad_ethtool.c
@@ -235,13 +235,18 @@ static int
bnad_get_link_ksettings(struct net_device *netdev,
struct ethtool_link_ksettings *cmd)
{
- u32 supported, advertising;
+ ethtool_link_ksettings_zero_link_mode(cmd, supported);
+ ethtool_link_ksettings_zero_link_mode(cmd, advertising);
- supported = SUPPORTED_10000baseT_Full;
- advertising = ADVERTISED_10000baseT_Full;
+ ethtool_link_ksettings_add_link_mode(cmd, supported, 10000baseCR_Full);
+ ethtool_link_ksettings_add_link_mode(cmd, supported, 10000baseSR_Full);
+ ethtool_link_ksettings_add_link_mode(cmd, supported, 10000baseLR_Full);
+ ethtool_link_ksettings_add_link_mode(cmd, advertising, 10000baseCR_Full);
+ ethtool_link_ksettings_add_link_mode(cmd, advertising, 10000baseSR_Full);
+ ethtool_link_ksettings_add_link_mode(cmd, advertising, 10000baseLR_Full);
cmd->base.autoneg = AUTONEG_DISABLE;
- supported |= SUPPORTED_FIBRE;
- advertising |= ADVERTISED_FIBRE;
+ ethtool_link_ksettings_add_link_mode(cmd, supported, FIBRE);
+ ethtool_link_ksettings_add_link_mode(cmd, advertising, FIBRE);
cmd->base.port = PORT_FIBRE;
cmd->base.phy_address = 0;
@@ -253,11 +258,6 @@ bnad_get_link_ksettings(struct net_device *netdev,
cmd->base.duplex = DUPLEX_UNKNOWN;
}
- ethtool_convert_legacy_u32_to_link_mode(cmd->link_modes.supported,
- supported);
- ethtool_convert_legacy_u32_to_link_mode(cmd->link_modes.advertising,
- advertising);
-
return 0;
}
@@ -405,7 +405,9 @@ static int bnad_set_coalesce(struct net_device *netdev,
static void
bnad_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ringparam)
+ struct ethtool_ringparam *ringparam,
+ struct kernel_ethtool_ringparam *kernel_ringparam,
+ struct netlink_ext_ack *extack)
{
struct bnad *bnad = netdev_priv(netdev);
@@ -418,7 +420,9 @@ bnad_get_ringparam(struct net_device *netdev,
static int
bnad_set_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ringparam)
+ struct ethtool_ringparam *ringparam,
+ struct kernel_ethtool_ringparam *kernel_ringparam,
+ struct netlink_ext_ack *extack)
{
int i, current_err, err = 0;
struct bnad *bnad = netdev_priv(netdev);
diff --git a/drivers/net/ethernet/cadence/macb.h b/drivers/net/ethernet/cadence/macb.h
index 5620b97..9ddbee7 100644
--- a/drivers/net/ethernet/cadence/macb.h
+++ b/drivers/net/ethernet/cadence/macb.h
@@ -1271,7 +1271,8 @@ struct macb {
struct mii_bus *mii_bus;
struct phylink *phylink;
struct phylink_config phylink_config;
- struct phylink_pcs phylink_pcs;
+ struct phylink_pcs phylink_usx_pcs;
+ struct phylink_pcs phylink_sgmii_pcs;
u32 caps;
unsigned int dma_burst_length;
diff --git a/drivers/net/ethernet/cadence/macb_main.c b/drivers/net/ethernet/cadence/macb_main.c
index ffce528..a363da9 100644
--- a/drivers/net/ethernet/cadence/macb_main.c
+++ b/drivers/net/ethernet/cadence/macb_main.c
@@ -506,79 +506,11 @@ static void macb_set_tx_clk(struct macb *bp, int speed)
netdev_err(bp->dev, "adjusting tx_clk failed.\n");
}
-static void macb_validate(struct phylink_config *config,
- unsigned long *supported,
- struct phylink_link_state *state)
-{
- struct net_device *ndev = to_net_dev(config->dev);
- __ETHTOOL_DECLARE_LINK_MODE_MASK(mask) = { 0, };
- struct macb *bp = netdev_priv(ndev);
-
- /* We only support MII, RMII, GMII, RGMII & SGMII. */
- if (state->interface != PHY_INTERFACE_MODE_NA &&
- state->interface != PHY_INTERFACE_MODE_MII &&
- state->interface != PHY_INTERFACE_MODE_RMII &&
- state->interface != PHY_INTERFACE_MODE_GMII &&
- state->interface != PHY_INTERFACE_MODE_SGMII &&
- state->interface != PHY_INTERFACE_MODE_10GBASER &&
- !phy_interface_mode_is_rgmii(state->interface)) {
- linkmode_zero(supported);
- return;
- }
-
- if (!macb_is_gem(bp) &&
- (state->interface == PHY_INTERFACE_MODE_GMII ||
- phy_interface_mode_is_rgmii(state->interface))) {
- linkmode_zero(supported);
- return;
- }
-
- if (state->interface == PHY_INTERFACE_MODE_10GBASER &&
- !(bp->caps & MACB_CAPS_HIGH_SPEED &&
- bp->caps & MACB_CAPS_PCS)) {
- linkmode_zero(supported);
- return;
- }
-
- phylink_set_port_modes(mask);
- phylink_set(mask, Autoneg);
- phylink_set(mask, Asym_Pause);
-
- if (bp->caps & MACB_CAPS_GIGABIT_MODE_AVAILABLE &&
- (state->interface == PHY_INTERFACE_MODE_NA ||
- state->interface == PHY_INTERFACE_MODE_10GBASER)) {
- phylink_set_10g_modes(mask);
- phylink_set(mask, 10000baseKR_Full);
- if (state->interface != PHY_INTERFACE_MODE_NA)
- goto out;
- }
-
- phylink_set(mask, 10baseT_Half);
- phylink_set(mask, 10baseT_Full);
- phylink_set(mask, 100baseT_Half);
- phylink_set(mask, 100baseT_Full);
-
- if (bp->caps & MACB_CAPS_GIGABIT_MODE_AVAILABLE &&
- (state->interface == PHY_INTERFACE_MODE_NA ||
- state->interface == PHY_INTERFACE_MODE_GMII ||
- state->interface == PHY_INTERFACE_MODE_SGMII ||
- phy_interface_mode_is_rgmii(state->interface))) {
- phylink_set(mask, 1000baseT_Full);
- phylink_set(mask, 1000baseX_Full);
-
- if (!(bp->caps & MACB_CAPS_NO_GIGABIT_HALF))
- phylink_set(mask, 1000baseT_Half);
- }
-out:
- linkmode_and(supported, supported, mask);
- linkmode_and(state->advertising, state->advertising, mask);
-}
-
static void macb_usx_pcs_link_up(struct phylink_pcs *pcs, unsigned int mode,
phy_interface_t interface, int speed,
int duplex)
{
- struct macb *bp = container_of(pcs, struct macb, phylink_pcs);
+ struct macb *bp = container_of(pcs, struct macb, phylink_usx_pcs);
u32 config;
config = gem_readl(bp, USX_CONTROL);
@@ -592,7 +524,7 @@ static void macb_usx_pcs_link_up(struct phylink_pcs *pcs, unsigned int mode,
static void macb_usx_pcs_get_state(struct phylink_pcs *pcs,
struct phylink_link_state *state)
{
- struct macb *bp = container_of(pcs, struct macb, phylink_pcs);
+ struct macb *bp = container_of(pcs, struct macb, phylink_usx_pcs);
u32 val;
state->speed = SPEED_10000;
@@ -612,7 +544,7 @@ static int macb_usx_pcs_config(struct phylink_pcs *pcs,
const unsigned long *advertising,
bool permit_pause_to_mac)
{
- struct macb *bp = container_of(pcs, struct macb, phylink_pcs);
+ struct macb *bp = container_of(pcs, struct macb, phylink_usx_pcs);
gem_writel(bp, USX_CONTROL, gem_readl(bp, USX_CONTROL) |
GEM_BIT(SIGNAL_OK));
@@ -795,28 +727,23 @@ static void macb_mac_link_up(struct phylink_config *config,
netif_tx_wake_all_queues(ndev);
}
-static int macb_mac_prepare(struct phylink_config *config, unsigned int mode,
- phy_interface_t interface)
+static struct phylink_pcs *macb_mac_select_pcs(struct phylink_config *config,
+ phy_interface_t interface)
{
struct net_device *ndev = to_net_dev(config->dev);
struct macb *bp = netdev_priv(ndev);
if (interface == PHY_INTERFACE_MODE_10GBASER)
- bp->phylink_pcs.ops = &macb_phylink_usx_pcs_ops;
+ return &bp->phylink_usx_pcs;
else if (interface == PHY_INTERFACE_MODE_SGMII)
- bp->phylink_pcs.ops = &macb_phylink_pcs_ops;
+ return &bp->phylink_sgmii_pcs;
else
- bp->phylink_pcs.ops = NULL;
-
- if (bp->phylink_pcs.ops)
- phylink_set_pcs(bp->phylink, &bp->phylink_pcs);
-
- return 0;
+ return NULL;
}
static const struct phylink_mac_ops macb_phylink_ops = {
- .validate = macb_validate,
- .mac_prepare = macb_mac_prepare,
+ .validate = phylink_generic_validate,
+ .mac_select_pcs = macb_mac_select_pcs,
.mac_config = macb_mac_config,
.mac_link_down = macb_mac_link_down,
.mac_link_up = macb_mac_link_up,
@@ -874,6 +801,9 @@ static int macb_mii_probe(struct net_device *dev)
{
struct macb *bp = netdev_priv(dev);
+ bp->phylink_sgmii_pcs.ops = &macb_phylink_pcs_ops;
+ bp->phylink_usx_pcs.ops = &macb_phylink_usx_pcs_ops;
+
bp->phylink_config.dev = &dev->dev;
bp->phylink_config.type = PHYLINK_NETDEV;
@@ -882,6 +812,35 @@ static int macb_mii_probe(struct net_device *dev)
bp->phylink_config.get_fixed_state = macb_get_pcs_fixed_state;
}
+ bp->phylink_config.mac_capabilities = MAC_ASYM_PAUSE |
+ MAC_10 | MAC_100;
+
+ __set_bit(PHY_INTERFACE_MODE_MII,
+ bp->phylink_config.supported_interfaces);
+ __set_bit(PHY_INTERFACE_MODE_RMII,
+ bp->phylink_config.supported_interfaces);
+
+ /* Determine what modes are supported */
+ if (macb_is_gem(bp) && (bp->caps & MACB_CAPS_GIGABIT_MODE_AVAILABLE)) {
+ bp->phylink_config.mac_capabilities |= MAC_1000FD;
+ if (!(bp->caps & MACB_CAPS_NO_GIGABIT_HALF))
+ bp->phylink_config.mac_capabilities |= MAC_1000HD;
+
+ __set_bit(PHY_INTERFACE_MODE_GMII,
+ bp->phylink_config.supported_interfaces);
+ phy_interface_set_rgmii(bp->phylink_config.supported_interfaces);
+
+ if (bp->caps & MACB_CAPS_PCS)
+ __set_bit(PHY_INTERFACE_MODE_SGMII,
+ bp->phylink_config.supported_interfaces);
+
+ if (bp->caps & MACB_CAPS_HIGH_SPEED) {
+ __set_bit(PHY_INTERFACE_MODE_10GBASER,
+ bp->phylink_config.supported_interfaces);
+ bp->phylink_config.mac_capabilities |= MAC_10000FD;
+ }
+ }
+
bp->phylink = phylink_create(&bp->phylink_config, bp->pdev->dev.fwnode,
bp->phy_interface, &macb_phylink_ops);
if (IS_ERR(bp->phylink)) {
@@ -3101,7 +3060,9 @@ static int macb_set_link_ksettings(struct net_device *netdev,
}
static void macb_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct macb *bp = netdev_priv(netdev);
@@ -3113,7 +3074,9 @@ static void macb_get_ringparam(struct net_device *netdev,
}
static int macb_set_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct macb *bp = netdev_priv(netdev);
u32 new_rx_size, new_tx_size;
diff --git a/drivers/net/ethernet/cadence/macb_ptp.c b/drivers/net/ethernet/cadence/macb_ptp.c
index 095c5a2..fb6b27f 100644
--- a/drivers/net/ethernet/cadence/macb_ptp.c
+++ b/drivers/net/ethernet/cadence/macb_ptp.c
@@ -464,10 +464,6 @@ int gem_set_hwtst(struct net_device *dev, struct ifreq *ifr, int cmd)
sizeof(*tstamp_config)))
return -EFAULT;
- /* reserved for future extensions */
- if (tstamp_config->flags)
- return -EINVAL;
-
switch (tstamp_config->tx_type) {
case HWTSTAMP_TX_OFF:
break;
diff --git a/drivers/net/ethernet/cavium/liquidio/lio_ethtool.c b/drivers/net/ethernet/cavium/liquidio/lio_ethtool.c
index 2b97478..2c10ae3 100644
--- a/drivers/net/ethernet/cavium/liquidio/lio_ethtool.c
+++ b/drivers/net/ethernet/cavium/liquidio/lio_ethtool.c
@@ -947,7 +947,9 @@ static int lio_set_phys_id(struct net_device *netdev,
static void
lio_ethtool_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ering)
+ struct ethtool_ringparam *ering,
+ struct kernel_ethtool_ringparam *kernel_ering,
+ struct netlink_ext_ack *extack)
{
struct lio *lio = GET_LIO(netdev);
struct octeon_device *oct = lio->oct_dev;
@@ -1252,8 +1254,11 @@ static int lio_reset_queues(struct net_device *netdev, uint32_t num_qs)
return 0;
}
-static int lio_ethtool_set_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ering)
+static int
+lio_ethtool_set_ringparam(struct net_device *netdev,
+ struct ethtool_ringparam *ering,
+ struct kernel_ethtool_ringparam *kernel_ering,
+ struct netlink_ext_ack *extack)
{
u32 rx_count, tx_count, rx_count_old, tx_count_old;
struct lio *lio = GET_LIO(netdev);
diff --git a/drivers/net/ethernet/cavium/liquidio/lio_main.c b/drivers/net/ethernet/cavium/liquidio/lio_main.c
index 12eee2b..ba28aa4 100644
--- a/drivers/net/ethernet/cavium/liquidio/lio_main.c
+++ b/drivers/net/ethernet/cavium/liquidio/lio_main.c
@@ -2114,9 +2114,6 @@ static int hwtstamp_ioctl(struct net_device *netdev, struct ifreq *ifr)
if (copy_from_user(&conf, ifr->ifr_data, sizeof(conf)))
return -EFAULT;
- if (conf.flags)
- return -EINVAL;
-
switch (conf.tx_type) {
case HWTSTAMP_TX_ON:
case HWTSTAMP_TX_OFF:
diff --git a/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c b/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c
index c607756..568f211 100644
--- a/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c
+++ b/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c
@@ -1254,9 +1254,6 @@ static int hwtstamp_ioctl(struct net_device *netdev, struct ifreq *ifr)
if (copy_from_user(&conf, ifr->ifr_data, sizeof(conf)))
return -EFAULT;
- if (conf.flags)
- return -EINVAL;
-
switch (conf.tx_type) {
case HWTSTAMP_TX_ON:
case HWTSTAMP_TX_OFF:
diff --git a/drivers/net/ethernet/cavium/octeon/octeon_mgmt.c b/drivers/net/ethernet/cavium/octeon/octeon_mgmt.c
index 4e39d71..103591d 100644
--- a/drivers/net/ethernet/cavium/octeon/octeon_mgmt.c
+++ b/drivers/net/ethernet/cavium/octeon/octeon_mgmt.c
@@ -548,7 +548,7 @@ struct octeon_mgmt_cam_state {
};
static void octeon_mgmt_cam_state_add(struct octeon_mgmt_cam_state *cs,
- unsigned char *addr)
+ const unsigned char *addr)
{
int i;
@@ -702,9 +702,6 @@ static int octeon_mgmt_ioctl_hwtstamp(struct net_device *netdev,
if (copy_from_user(&config, rq->ifr_data, sizeof(config)))
return -EFAULT;
- if (config.flags) /* reserved for future extensions */
- return -EINVAL;
-
/* Check the status of hardware for tiemstamps */
if (OCTEON_IS_MODEL(OCTEON_CN6XXX)) {
/* Get the current state of the PTP clock */
diff --git a/drivers/net/ethernet/cavium/thunder/nicvf_ethtool.c b/drivers/net/ethernet/cavium/thunder/nicvf_ethtool.c
index 7f28821..5a9fad6 100644
--- a/drivers/net/ethernet/cavium/thunder/nicvf_ethtool.c
+++ b/drivers/net/ethernet/cavium/thunder/nicvf_ethtool.c
@@ -467,7 +467,9 @@ static int nicvf_get_coalesce(struct net_device *netdev,
}
static void nicvf_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct nicvf *nic = netdev_priv(netdev);
struct queue_set *qs = nic->qs;
@@ -479,7 +481,9 @@ static void nicvf_get_ringparam(struct net_device *netdev,
}
static int nicvf_set_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct nicvf *nic = netdev_priv(netdev);
struct queue_set *qs = nic->qs;
diff --git a/drivers/net/ethernet/cavium/thunder/nicvf_main.c b/drivers/net/ethernet/cavium/thunder/nicvf_main.c
index bb45d5df..a04aa20 100644
--- a/drivers/net/ethernet/cavium/thunder/nicvf_main.c
+++ b/drivers/net/ethernet/cavium/thunder/nicvf_main.c
@@ -590,7 +590,7 @@ static inline bool nicvf_xdp_rx(struct nicvf *nic, struct bpf_prog *prog,
nicvf_xdp_sq_append_pkt(nic, sq, (u64)xdp.data, dma_addr, len);
return true;
default:
- bpf_warn_invalid_xdp_action(action);
+ bpf_warn_invalid_xdp_action(nic->netdev, prog, action);
fallthrough;
case XDP_ABORTED:
trace_xdp_exception(nic->netdev, prog, action);
@@ -1917,10 +1917,6 @@ static int nicvf_config_hwtstamp(struct net_device *netdev, struct ifreq *ifr)
if (copy_from_user(&config, ifr->ifr_data, sizeof(config)))
return -EFAULT;
- /* reserved for future extensions */
- if (config.flags)
- return -EINVAL;
-
switch (config.tx_type) {
case HWTSTAMP_TX_OFF:
case HWTSTAMP_TX_ON:
diff --git a/drivers/net/ethernet/cavium/thunder/nicvf_queues.c b/drivers/net/ethernet/cavium/thunder/nicvf_queues.c
index 50bbe79..4367edb 100644
--- a/drivers/net/ethernet/cavium/thunder/nicvf_queues.c
+++ b/drivers/net/ethernet/cavium/thunder/nicvf_queues.c
@@ -10,6 +10,7 @@
#include <linux/iommu.h>
#include <net/ip.h>
#include <net/tso.h>
+#include <uapi/linux/bpf.h>
#include "nic_reg.h"
#include "nic.h"
diff --git a/drivers/net/ethernet/chelsio/cxgb/cxgb2.c b/drivers/net/ethernet/chelsio/cxgb/cxgb2.c
index 609820e..f4054d25 100644
--- a/drivers/net/ethernet/chelsio/cxgb/cxgb2.c
+++ b/drivers/net/ethernet/chelsio/cxgb/cxgb2.c
@@ -710,7 +710,9 @@ static int set_pauseparam(struct net_device *dev,
return 0;
}
-static void get_sge_param(struct net_device *dev, struct ethtool_ringparam *e)
+static void get_sge_param(struct net_device *dev, struct ethtool_ringparam *e,
+ struct kernel_ethtool_ringparam *kernel_e,
+ struct netlink_ext_ack *extack)
{
struct adapter *adapter = dev->ml_priv;
int jumbo_fl = t1_is_T1B(adapter) ? 1 : 0;
@@ -724,7 +726,9 @@ static void get_sge_param(struct net_device *dev, struct ethtool_ringparam *e)
e->tx_pending = adapter->params.sge.cmdQ_size[0];
}
-static int set_sge_param(struct net_device *dev, struct ethtool_ringparam *e)
+static int set_sge_param(struct net_device *dev, struct ethtool_ringparam *e,
+ struct kernel_ethtool_ringparam *kernel_e,
+ struct netlink_ext_ack *extack)
{
struct adapter *adapter = dev->ml_priv;
int jumbo_fl = t1_is_T1B(adapter) ? 1 : 0;
@@ -940,11 +944,11 @@ static const struct net_device_ops cxgb_netdev_ops = {
static int init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
{
- int i, err, pci_using_dac = 0;
unsigned long mmio_start, mmio_len;
const struct board_info *bi;
struct adapter *adapter = NULL;
struct port_info *pi;
+ int i, err;
err = pci_enable_device(pdev);
if (err)
@@ -957,17 +961,8 @@ static int init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
goto out_disable_pdev;
}
- if (!dma_set_mask(&pdev->dev, DMA_BIT_MASK(64))) {
- pci_using_dac = 1;
-
- if (dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(64))) {
- pr_err("%s: unable to obtain 64-bit DMA for coherent allocations\n",
- pci_name(pdev));
- err = -ENODEV;
- goto out_disable_pdev;
- }
-
- } else if ((err = dma_set_mask(&pdev->dev, DMA_BIT_MASK(32))) != 0) {
+ err = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64));
+ if (err) {
pr_err("%s: no usable DMA configuration\n", pci_name(pdev));
goto out_disable_pdev;
}
@@ -1039,10 +1034,8 @@ static int init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
netdev->hw_features |= NETIF_F_SG | NETIF_F_IP_CSUM |
NETIF_F_RXCSUM;
netdev->features |= NETIF_F_SG | NETIF_F_IP_CSUM |
- NETIF_F_RXCSUM | NETIF_F_LLTX;
+ NETIF_F_RXCSUM | NETIF_F_LLTX | NETIF_F_HIGHDMA;
- if (pci_using_dac)
- netdev->features |= NETIF_F_HIGHDMA;
if (vlan_tso_capable(adapter)) {
netdev->features |=
NETIF_F_HW_VLAN_CTAG_TX |
diff --git a/drivers/net/ethernet/chelsio/cxgb/sge.c b/drivers/net/ethernet/chelsio/cxgb/sge.c
index cda01f2..12e76fd 100644
--- a/drivers/net/ethernet/chelsio/cxgb/sge.c
+++ b/drivers/net/ethernet/chelsio/cxgb/sge.c
@@ -1359,7 +1359,7 @@ static void restart_sched(struct tasklet_struct *t)
* @fl: the free list that contains the packet buffer
* @len: the packet length
*
- * Process an ingress ethernet pakcet and deliver it to the stack.
+ * Process an ingress ethernet packet and deliver it to the stack.
*/
static void sge_rx(struct sge *sge, struct freelQ *fl, unsigned int len)
{
diff --git a/drivers/net/ethernet/chelsio/cxgb3/cxgb3_main.c b/drivers/net/ethernet/chelsio/cxgb3/cxgb3_main.c
index bfffcae..6352131 100644
--- a/drivers/net/ethernet/chelsio/cxgb3/cxgb3_main.c
+++ b/drivers/net/ethernet/chelsio/cxgb3/cxgb3_main.c
@@ -1948,7 +1948,9 @@ static int set_pauseparam(struct net_device *dev,
return 0;
}
-static void get_sge_param(struct net_device *dev, struct ethtool_ringparam *e)
+static void get_sge_param(struct net_device *dev, struct ethtool_ringparam *e,
+ struct kernel_ethtool_ringparam *kernel_e,
+ struct netlink_ext_ack *extack)
{
struct port_info *pi = netdev_priv(dev);
struct adapter *adapter = pi->adapter;
@@ -1964,7 +1966,9 @@ static void get_sge_param(struct net_device *dev, struct ethtool_ringparam *e)
e->tx_pending = q->txq_size[0];
}
-static int set_sge_param(struct net_device *dev, struct ethtool_ringparam *e)
+static int set_sge_param(struct net_device *dev, struct ethtool_ringparam *e,
+ struct kernel_ethtool_ringparam *kernel_e,
+ struct netlink_ext_ack *extack)
{
struct port_info *pi = netdev_priv(dev);
struct adapter *adapter = pi->adapter;
@@ -3200,7 +3204,7 @@ static void cxgb3_init_iscsi_mac(struct net_device *dev)
NETIF_F_IPV6_CSUM | NETIF_F_HIGHDMA)
static int init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
{
- int i, err, pci_using_dac = 0;
+ int i, err;
resource_size_t mmio_start, mmio_len;
const struct adapter_info *ai;
struct adapter *adapter = NULL;
@@ -3227,9 +3231,8 @@ static int init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
goto out_disable_device;
}
- if (!dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64))) {
- pci_using_dac = 1;
- } else if ((err = dma_set_mask(&pdev->dev, DMA_BIT_MASK(32))) != 0) {
+ err = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64));
+ if (err) {
dev_err(&pdev->dev, "no usable DMA configuration\n");
goto out_release_regions;
}
@@ -3305,8 +3308,8 @@ static int init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
netdev->features |= netdev->hw_features |
NETIF_F_HW_VLAN_CTAG_TX;
netdev->vlan_features |= netdev->features & VLAN_FEAT;
- if (pci_using_dac)
- netdev->features |= NETIF_F_HIGHDMA;
+
+ netdev->features |= NETIF_F_HIGHDMA;
netdev->netdev_ops = &cxgb_netdev_ops;
netdev->ethtool_ops = &cxgb_ethtool_ops;
diff --git a/drivers/net/ethernet/chelsio/cxgb3/sge.c b/drivers/net/ethernet/chelsio/cxgb3/sge.c
index c3afec1..62dfbdd 100644
--- a/drivers/net/ethernet/chelsio/cxgb3/sge.c
+++ b/drivers/net/ethernet/chelsio/cxgb3/sge.c
@@ -126,8 +126,10 @@ struct rsp_desc { /* response queue descriptor */
struct rss_header rss_hdr;
__be32 flags;
__be32 len_cq;
- u8 imm_data[47];
- u8 intr_gen;
+ struct_group(immediate,
+ u8 imm_data[47];
+ u8 intr_gen;
+ );
};
/*
@@ -925,7 +927,8 @@ static inline struct sk_buff *get_imm_packet(const struct rsp_desc *resp)
if (skb) {
__skb_put(skb, IMMED_PKT_SIZE);
- skb_copy_to_linear_data(skb, resp->imm_data, IMMED_PKT_SIZE);
+ BUILD_BUG_ON(IMMED_PKT_SIZE != sizeof(resp->immediate));
+ skb_copy_to_linear_data(skb, &resp->immediate, IMMED_PKT_SIZE);
}
return skb;
}
@@ -1953,7 +1956,7 @@ static int ofld_poll(struct napi_struct *napi, int budget)
* @rx_gather: a gather list of packets if we are building a bundle
* @gather_idx: index of the next available slot in the bundle
*
- * Process an ingress offload pakcet and add it to the offload ingress
+ * Process an ingress offload packet and add it to the offload ingress
* queue. Returns the index of the next available slot in the bundle.
*/
static inline int rx_offload(struct t3cdev *tdev, struct sge_rspq *rq,
@@ -2079,7 +2082,7 @@ static void cxgb3_process_iscsi_prov_pack(struct port_info *pi,
* @pad: padding
* @lro: large receive offload
*
- * Process an ingress ethernet pakcet and deliver it to the stack.
+ * Process an ingress ethernet packet and deliver it to the stack.
* The padding is 2 if the packet was delivered in an Rx buffer and 0
* if it was immediate data in a response.
*/
diff --git a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_ethtool.c b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_ethtool.c
index 129352b..6c790af 100644
--- a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_ethtool.c
+++ b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_ethtool.c
@@ -890,7 +890,9 @@ static int set_pauseparam(struct net_device *dev,
return 0;
}
-static void get_sge_param(struct net_device *dev, struct ethtool_ringparam *e)
+static void get_sge_param(struct net_device *dev, struct ethtool_ringparam *e,
+ struct kernel_ethtool_ringparam *kernel_e,
+ struct netlink_ext_ack *extack)
{
const struct port_info *pi = netdev_priv(dev);
const struct sge *s = &pi->adapter->sge;
@@ -906,7 +908,9 @@ static void get_sge_param(struct net_device *dev, struct ethtool_ringparam *e)
e->tx_pending = s->ethtxq[pi->first_qset].q.size;
}
-static int set_sge_param(struct net_device *dev, struct ethtool_ringparam *e)
+static int set_sge_param(struct net_device *dev, struct ethtool_ringparam *e,
+ struct kernel_ethtool_ringparam *kernel_e,
+ struct netlink_ext_ack *extack)
{
int i;
const struct port_info *pi = netdev_priv(dev);
@@ -1989,6 +1993,15 @@ static int get_dump_data(struct net_device *dev, struct ethtool_dump *eth_dump,
return 0;
}
+static bool cxgb4_fw_mod_type_info_available(unsigned int fw_mod_type)
+{
+ /* Read port module EEPROM as long as it is plugged-in and
+ * safe to read.
+ */
+ return (fw_mod_type != FW_PORT_MOD_TYPE_NONE &&
+ fw_mod_type != FW_PORT_MOD_TYPE_ERROR);
+}
+
static int cxgb4_get_module_info(struct net_device *dev,
struct ethtool_modinfo *modinfo)
{
@@ -1997,7 +2010,7 @@ static int cxgb4_get_module_info(struct net_device *dev,
struct adapter *adapter = pi->adapter;
int ret;
- if (!t4_is_inserted_mod_type(pi->mod_type))
+ if (!cxgb4_fw_mod_type_info_available(pi->mod_type))
return -EINVAL;
switch (pi->port_type) {
diff --git a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c
index dde1cf5..0c78c0d 100644
--- a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c
+++ b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c
@@ -6608,7 +6608,6 @@ static int init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
static int adap_idx = 1;
int s_qpp, qpp, num_seg;
struct port_info *pi;
- bool highdma = false;
enum chip_type chip;
void __iomem *regs;
int func, chip_ver;
@@ -6687,14 +6686,10 @@ static int init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
return 0;
}
- if (!dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64))) {
- highdma = true;
- } else {
- err = dma_set_mask(&pdev->dev, DMA_BIT_MASK(32));
- if (err) {
- dev_err(&pdev->dev, "no usable DMA configuration\n");
- goto out_free_adapter;
- }
+ err = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64));
+ if (err) {
+ dev_err(&pdev->dev, "no usable DMA configuration\n");
+ goto out_free_adapter;
}
pci_enable_pcie_error_reporting(pdev);
@@ -6823,7 +6818,7 @@ static int init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM |
NETIF_F_RXCSUM | NETIF_F_RXHASH | NETIF_F_GRO |
NETIF_F_HW_VLAN_CTAG_TX | NETIF_F_HW_VLAN_CTAG_RX |
- NETIF_F_HW_TC | NETIF_F_NTUPLE;
+ NETIF_F_HW_TC | NETIF_F_NTUPLE | NETIF_F_HIGHDMA;
if (chip_ver > CHELSIO_T5) {
netdev->hw_enc_features |= NETIF_F_IP_CSUM |
@@ -6841,8 +6836,6 @@ static int init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
netdev->udp_tunnel_nic_info = &cxgb_udp_tunnels;
}
- if (highdma)
- netdev->hw_features |= NETIF_F_HIGHDMA;
netdev->features |= netdev->hw_features;
netdev->vlan_features = netdev->features & VLAN_FEAT;
#if IS_ENABLED(CONFIG_CHELSIO_TLS_DEVICE)
diff --git a/drivers/net/ethernet/chelsio/cxgb4/sge.c b/drivers/net/ethernet/chelsio/cxgb4/sge.c
index fa5b596..f889f40 100644
--- a/drivers/net/ethernet/chelsio/cxgb4/sge.c
+++ b/drivers/net/ethernet/chelsio/cxgb4/sge.c
@@ -1842,8 +1842,10 @@ static netdev_tx_t cxgb4_vf_eth_xmit(struct sk_buff *skb,
* (including the VLAN tag) into the header so we reject anything
* smaller than that ...
*/
- fw_hdr_copy_len = sizeof(wr->ethmacdst) + sizeof(wr->ethmacsrc) +
- sizeof(wr->ethtype) + sizeof(wr->vlantci);
+ BUILD_BUG_ON(sizeof(wr->firmware) !=
+ (sizeof(wr->ethmacdst) + sizeof(wr->ethmacsrc) +
+ sizeof(wr->ethtype) + sizeof(wr->vlantci)));
+ fw_hdr_copy_len = sizeof(wr->firmware);
ret = cxgb4_validate_skb(skb, dev, fw_hdr_copy_len);
if (ret)
goto out_free;
@@ -1924,7 +1926,7 @@ static netdev_tx_t cxgb4_vf_eth_xmit(struct sk_buff *skb,
wr->equiq_to_len16 = cpu_to_be32(wr_mid);
wr->r3[0] = cpu_to_be32(0);
wr->r3[1] = cpu_to_be32(0);
- skb_copy_from_linear_data(skb, (void *)wr->ethmacdst, fw_hdr_copy_len);
+ skb_copy_from_linear_data(skb, &wr->firmware, fw_hdr_copy_len);
end = (u64 *)wr + flits;
/* If this is a Large Send Offload packet we'll put in an LSO CPL
diff --git a/drivers/net/ethernet/chelsio/cxgb4/t4fw_api.h b/drivers/net/ethernet/chelsio/cxgb4/t4fw_api.h
index 0a326c0..2419459 100644
--- a/drivers/net/ethernet/chelsio/cxgb4/t4fw_api.h
+++ b/drivers/net/ethernet/chelsio/cxgb4/t4fw_api.h
@@ -794,10 +794,12 @@ struct fw_eth_tx_pkt_vm_wr {
__be32 op_immdlen;
__be32 equiq_to_len16;
__be32 r3[2];
- u8 ethmacdst[6];
- u8 ethmacsrc[6];
- __be16 ethtype;
- __be16 vlantci;
+ struct_group(firmware,
+ u8 ethmacdst[ETH_ALEN];
+ u8 ethmacsrc[ETH_ALEN];
+ __be16 ethtype;
+ __be16 vlantci;
+ );
};
#define FW_CMD_MAX_TIMEOUT 10000
diff --git a/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf_main.c b/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf_main.c
index ae9cca7..7de3800 100644
--- a/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf_main.c
+++ b/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf_main.c
@@ -1591,7 +1591,9 @@ static void cxgb4vf_set_msglevel(struct net_device *dev, u32 msglevel)
* first Queue Set.
*/
static void cxgb4vf_get_ringparam(struct net_device *dev,
- struct ethtool_ringparam *rp)
+ struct ethtool_ringparam *rp,
+ struct kernel_ethtool_ringparam *kernel_rp,
+ struct netlink_ext_ack *extack)
{
const struct port_info *pi = netdev_priv(dev);
const struct sge *s = &pi->adapter->sge;
@@ -1614,7 +1616,9 @@ static void cxgb4vf_get_ringparam(struct net_device *dev,
* device -- after vetting them of course!
*/
static int cxgb4vf_set_ringparam(struct net_device *dev,
- struct ethtool_ringparam *rp)
+ struct ethtool_ringparam *rp,
+ struct kernel_ethtool_ringparam *kernel_rp,
+ struct netlink_ext_ack *extack)
{
const struct port_info *pi = netdev_priv(dev);
struct adapter *adapter = pi->adapter;
@@ -2895,7 +2899,6 @@ static int cxgb4vf_pci_probe(struct pci_dev *pdev,
struct net_device *netdev;
struct port_info *pi;
unsigned int pmask;
- int pci_using_dac;
int err, pidx;
/*
@@ -2916,19 +2919,12 @@ static int cxgb4vf_pci_probe(struct pci_dev *pdev,
}
/*
- * Set up our DMA mask: try for 64-bit address masking first and
- * fall back to 32-bit if we can't get 64 bits ...
+ * Set up our DMA mask
*/
err = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64));
- if (err == 0) {
- pci_using_dac = 1;
- } else {
- err = dma_set_mask(&pdev->dev, DMA_BIT_MASK(32));
- if (err != 0) {
- dev_err(&pdev->dev, "no usable DMA configuration\n");
- goto err_release_regions;
- }
- pci_using_dac = 0;
+ if (err) {
+ dev_err(&pdev->dev, "no usable DMA configuration\n");
+ goto err_release_regions;
}
/*
@@ -3074,9 +3070,7 @@ static int cxgb4vf_pci_probe(struct pci_dev *pdev,
netdev->hw_features = NETIF_F_SG | TSO_FLAGS | NETIF_F_GRO |
NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM | NETIF_F_RXCSUM |
NETIF_F_HW_VLAN_CTAG_TX | NETIF_F_HW_VLAN_CTAG_RX;
- netdev->features = netdev->hw_features;
- if (pci_using_dac)
- netdev->features |= NETIF_F_HIGHDMA;
+ netdev->features = netdev->hw_features | NETIF_F_HIGHDMA;
netdev->vlan_features = netdev->features & VLAN_FEAT;
netdev->priv_flags |= IFF_UNICAST_FLT;
diff --git a/drivers/net/ethernet/chelsio/cxgb4vf/sge.c b/drivers/net/ethernet/chelsio/cxgb4vf/sge.c
index 0295b24..43b2ceb 100644
--- a/drivers/net/ethernet/chelsio/cxgb4vf/sge.c
+++ b/drivers/net/ethernet/chelsio/cxgb4vf/sge.c
@@ -1167,10 +1167,7 @@ netdev_tx_t t4vf_eth_xmit(struct sk_buff *skb, struct net_device *dev)
struct cpl_tx_pkt_core *cpl;
const struct skb_shared_info *ssi;
dma_addr_t addr[MAX_SKB_FRAGS + 1];
- const size_t fw_hdr_copy_len = (sizeof(wr->ethmacdst) +
- sizeof(wr->ethmacsrc) +
- sizeof(wr->ethtype) +
- sizeof(wr->vlantci));
+ const size_t fw_hdr_copy_len = sizeof(wr->firmware);
/*
* The chip minimum packet length is 10 octets but the firmware
@@ -1267,7 +1264,7 @@ netdev_tx_t t4vf_eth_xmit(struct sk_buff *skb, struct net_device *dev)
wr->equiq_to_len16 = cpu_to_be32(wr_mid);
wr->r3[0] = cpu_to_be32(0);
wr->r3[1] = cpu_to_be32(0);
- skb_copy_from_linear_data(skb, (void *)wr->ethmacdst, fw_hdr_copy_len);
+ skb_copy_from_linear_data(skb, &wr->firmware, fw_hdr_copy_len);
end = (u64 *)wr + flits;
/*
diff --git a/drivers/net/ethernet/cirrus/mac89x0.c b/drivers/net/ethernet/cirrus/mac89x0.c
index 84251b8..21a70b1 100644
--- a/drivers/net/ethernet/cirrus/mac89x0.c
+++ b/drivers/net/ethernet/cirrus/mac89x0.c
@@ -242,12 +242,15 @@ static int mac89x0_device_probe(struct platform_device *pdev)
pr_info("No EEPROM, giving up now.\n");
goto out1;
} else {
+ u8 addr[ETH_ALEN];
+
for (i = 0; i < ETH_ALEN; i += 2) {
/* Big-endian (why??!) */
unsigned short s = readreg(dev, PP_IA + i);
- dev->dev_addr[i] = s >> 8;
- dev->dev_addr[i+1] = s & 0xff;
+ addr[i] = s >> 8;
+ addr[i+1] = s & 0xff;
}
+ eth_hw_addr_set(dev, addr);
}
dev->irq = SLOT2IRQ(slot);
diff --git a/drivers/net/ethernet/cisco/enic/enic.h b/drivers/net/ethernet/cisco/enic/enic.h
index c67a16a..52aaf1b 100644
--- a/drivers/net/ethernet/cisco/enic/enic.h
+++ b/drivers/net/ethernet/cisco/enic/enic.h
@@ -304,7 +304,7 @@ static inline bool enic_is_notify_intr(struct enic *enic, int intr)
static inline int enic_dma_map_check(struct enic *enic, dma_addr_t dma_addr)
{
- if (unlikely(pci_dma_mapping_error(enic->pdev, dma_addr))) {
+ if (unlikely(dma_mapping_error(&enic->pdev->dev, dma_addr))) {
net_warn_ratelimited("%s: PCI dma mapping failed!\n",
enic->netdev->name);
enic->gen_stats.dma_map_error++;
diff --git a/drivers/net/ethernet/cisco/enic/enic_ethtool.c b/drivers/net/ethernet/cisco/enic/enic_ethtool.c
index 6ded4d9..6c11f9d 100644
--- a/drivers/net/ethernet/cisco/enic/enic_ethtool.c
+++ b/drivers/net/ethernet/cisco/enic/enic_ethtool.c
@@ -177,7 +177,9 @@ static void enic_get_strings(struct net_device *netdev, u32 stringset,
}
static void enic_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct enic *enic = netdev_priv(netdev);
struct vnic_enet_config *c = &enic->config;
@@ -189,7 +191,9 @@ static void enic_get_ringparam(struct net_device *netdev,
}
static int enic_set_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct enic *enic = netdev_priv(netdev);
struct vnic_enet_config *c = &enic->config;
diff --git a/drivers/net/ethernet/cisco/enic/enic_main.c b/drivers/net/ethernet/cisco/enic/enic_main.c
index aacf141..4db6889 100644
--- a/drivers/net/ethernet/cisco/enic/enic_main.c
+++ b/drivers/net/ethernet/cisco/enic/enic_main.c
@@ -2718,26 +2718,14 @@ static int enic_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
* fail to 32-bit.
*/
- err = dma_set_mask(&pdev->dev, DMA_BIT_MASK(47));
+ err = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(47));
if (err) {
- err = dma_set_mask(&pdev->dev, DMA_BIT_MASK(32));
+ err = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32));
if (err) {
dev_err(dev, "No usable DMA configuration, aborting\n");
goto err_out_release_regions;
}
- err = dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(32));
- if (err) {
- dev_err(dev, "Unable to obtain %u-bit DMA "
- "for consistent allocations, aborting\n", 32);
- goto err_out_release_regions;
- }
} else {
- err = dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(47));
- if (err) {
- dev_err(dev, "Unable to obtain %u-bit DMA "
- "for consistent allocations, aborting\n", 47);
- goto err_out_release_regions;
- }
using_dac = 1;
}
diff --git a/drivers/net/ethernet/cortina/gemini.c b/drivers/net/ethernet/cortina/gemini.c
index 941f175..c78b99a 100644
--- a/drivers/net/ethernet/cortina/gemini.c
+++ b/drivers/net/ethernet/cortina/gemini.c
@@ -305,21 +305,21 @@ static void gmac_speed_set(struct net_device *netdev)
switch (phydev->speed) {
case 1000:
status.bits.speed = GMAC_SPEED_1000;
- if (phydev->interface == PHY_INTERFACE_MODE_RGMII)
+ if (phy_interface_mode_is_rgmii(phydev->interface))
status.bits.mii_rmii = GMAC_PHY_RGMII_1000;
netdev_dbg(netdev, "connect %s to RGMII @ 1Gbit\n",
phydev_name(phydev));
break;
case 100:
status.bits.speed = GMAC_SPEED_100;
- if (phydev->interface == PHY_INTERFACE_MODE_RGMII)
+ if (phy_interface_mode_is_rgmii(phydev->interface))
status.bits.mii_rmii = GMAC_PHY_RGMII_100_10;
netdev_dbg(netdev, "connect %s to RGMII @ 100 Mbit\n",
phydev_name(phydev));
break;
case 10:
status.bits.speed = GMAC_SPEED_10;
- if (phydev->interface == PHY_INTERFACE_MODE_RGMII)
+ if (phy_interface_mode_is_rgmii(phydev->interface))
status.bits.mii_rmii = GMAC_PHY_RGMII_100_10;
netdev_dbg(netdev, "connect %s to RGMII @ 10 Mbit\n",
phydev_name(phydev));
@@ -389,6 +389,9 @@ static int gmac_setup_phy(struct net_device *netdev)
status.bits.mii_rmii = GMAC_PHY_GMII;
break;
case PHY_INTERFACE_MODE_RGMII:
+ case PHY_INTERFACE_MODE_RGMII_ID:
+ case PHY_INTERFACE_MODE_RGMII_TXID:
+ case PHY_INTERFACE_MODE_RGMII_RXID:
netdev_dbg(netdev,
"RGMII: set GMAC0 and GMAC1 to MII/RGMII mode\n");
status.bits.mii_rmii = GMAC_PHY_RGMII_100_10;
@@ -2105,7 +2108,9 @@ static void gmac_get_pauseparam(struct net_device *netdev,
}
static void gmac_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *rp)
+ struct ethtool_ringparam *rp,
+ struct kernel_ethtool_ringparam *kernel_rp,
+ struct netlink_ext_ack *extack)
{
struct gemini_ethernet_port *port = netdev_priv(netdev);
@@ -2123,7 +2128,9 @@ static void gmac_get_ringparam(struct net_device *netdev,
}
static int gmac_set_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *rp)
+ struct ethtool_ringparam *rp,
+ struct kernel_ethtool_ringparam *kernel_rp,
+ struct netlink_ext_ack *extack)
{
struct gemini_ethernet_port *port = netdev_priv(netdev);
int err = 0;
diff --git a/drivers/net/ethernet/emulex/benet/be_ethtool.c b/drivers/net/ethernet/emulex/benet/be_ethtool.c
index f995530..dfa7843 100644
--- a/drivers/net/ethernet/emulex/benet/be_ethtool.c
+++ b/drivers/net/ethernet/emulex/benet/be_ethtool.c
@@ -683,7 +683,9 @@ static int be_get_link_ksettings(struct net_device *netdev,
}
static void be_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct be_adapter *adapter = netdev_priv(netdev);
diff --git a/drivers/net/ethernet/emulex/benet/be_main.c b/drivers/net/ethernet/emulex/benet/be_main.c
index d51f24c..ad67b42 100644
--- a/drivers/net/ethernet/emulex/benet/be_main.c
+++ b/drivers/net/ethernet/emulex/benet/be_main.c
@@ -5194,7 +5194,8 @@ static void be_netdev_init(struct net_device *netdev)
netdev->hw_features |= NETIF_F_RXHASH;
netdev->features |= netdev->hw_features |
- NETIF_F_HW_VLAN_CTAG_RX | NETIF_F_HW_VLAN_CTAG_FILTER;
+ NETIF_F_HW_VLAN_CTAG_RX | NETIF_F_HW_VLAN_CTAG_FILTER |
+ NETIF_F_HIGHDMA;
netdev->vlan_features |= NETIF_F_SG | NETIF_F_TSO | NETIF_F_TSO6 |
NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM;
@@ -5840,14 +5841,9 @@ static int be_probe(struct pci_dev *pdev, const struct pci_device_id *pdev_id)
SET_NETDEV_DEV(netdev, &pdev->dev);
status = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64));
- if (!status) {
- netdev->features |= NETIF_F_HIGHDMA;
- } else {
- status = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32));
- if (status) {
- dev_err(&pdev->dev, "Could not set PCI DMA Mask\n");
- goto free_netdev;
- }
+ if (status) {
+ dev_err(&pdev->dev, "Could not set PCI DMA Mask\n");
+ goto free_netdev;
}
status = pci_enable_pcie_error_reporting(pdev);
diff --git a/drivers/net/ethernet/engleder/Kconfig b/drivers/net/ethernet/engleder/Kconfig
new file mode 100644
index 0000000..f4e2b11
--- /dev/null
+++ b/drivers/net/ethernet/engleder/Kconfig
@@ -0,0 +1,39 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Engleder network device configuration
+#
+
+config NET_VENDOR_ENGLEDER
+ bool "Engleder devices"
+ default y
+ help
+ If you have a network (Ethernet) card belonging to this class, say Y.
+
+ Note that the answer to this question doesn't directly affect the
+ kernel: saying N will just cause the configurator to skip all
+ the questions about Engleder devices. If you say Y, you will be asked
+ for your specific card in the following questions.
+
+if NET_VENDOR_ENGLEDER
+
+config TSNEP
+ tristate "TSN endpoint support"
+ depends on HAS_IOMEM && HAS_DMA
+ depends on PTP_1588_CLOCK_OPTIONAL
+ select PHYLIB
+ help
+ Support for the Engleder TSN endpoint Ethernet MAC IP Core.
+
+ To compile this driver as a module, choose M here. The module will be
+ called tsnep.
+
+config TSNEP_SELFTESTS
+ bool "TSN endpoint self test support"
+ default n
+ depends on TSNEP
+ help
+ This enables self test support within the TSN endpoint driver.
+
+ If unsure, say N.
+
+endif # NET_VENDOR_ENGLEDER
diff --git a/drivers/net/ethernet/engleder/Makefile b/drivers/net/ethernet/engleder/Makefile
new file mode 100644
index 0000000..cce2191
--- /dev/null
+++ b/drivers/net/ethernet/engleder/Makefile
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Makefile for the Engleder Ethernet drivers
+#
+
+obj-$(CONFIG_TSNEP) += tsnep.o
+
+tsnep-objs := tsnep_main.o tsnep_ethtool.o tsnep_ptp.o tsnep_tc.o \
+ $(tsnep-y)
+tsnep-$(CONFIG_TSNEP_SELFTESTS) += tsnep_selftests.o
diff --git a/drivers/net/ethernet/engleder/tsnep.h b/drivers/net/ethernet/engleder/tsnep.h
new file mode 100644
index 0000000..23bbece
--- /dev/null
+++ b/drivers/net/ethernet/engleder/tsnep.h
@@ -0,0 +1,189 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021 Gerhard Engleder <gerhard@engleder-embedded.com> */
+
+#ifndef _TSNEP_H
+#define _TSNEP_H
+
+#include "tsnep_hw.h"
+
+#include <linux/platform_device.h>
+#include <linux/dma-mapping.h>
+#include <linux/etherdevice.h>
+#include <linux/phy.h>
+#include <linux/ethtool.h>
+#include <linux/net_tstamp.h>
+#include <linux/ptp_clock_kernel.h>
+#include <linux/miscdevice.h>
+
+#define TSNEP "tsnep"
+
+#define TSNEP_RING_SIZE 256
+#define TSNEP_RING_ENTRIES_PER_PAGE (PAGE_SIZE / TSNEP_DESC_SIZE)
+#define TSNEP_RING_PAGE_COUNT (TSNEP_RING_SIZE / TSNEP_RING_ENTRIES_PER_PAGE)
+
+#define TSNEP_QUEUES 1
+
+struct tsnep_gcl {
+ void __iomem *addr;
+
+ u64 base_time;
+ u64 cycle_time;
+ u64 cycle_time_extension;
+
+ struct tsnep_gcl_operation operation[TSNEP_GCL_COUNT];
+ int count;
+
+ u64 change_limit;
+
+ u64 start_time;
+ bool change;
+};
+
+struct tsnep_tx_entry {
+ struct tsnep_tx_desc *desc;
+ struct tsnep_tx_desc_wb *desc_wb;
+ dma_addr_t desc_dma;
+ bool owner_user_flag;
+
+ u32 properties;
+
+ struct sk_buff *skb;
+ size_t len;
+ DEFINE_DMA_UNMAP_ADDR(dma);
+};
+
+struct tsnep_tx {
+ struct tsnep_adapter *adapter;
+ void __iomem *addr;
+
+ void *page[TSNEP_RING_PAGE_COUNT];
+ dma_addr_t page_dma[TSNEP_RING_PAGE_COUNT];
+
+ /* TX ring lock */
+ spinlock_t lock;
+ struct tsnep_tx_entry entry[TSNEP_RING_SIZE];
+ int write;
+ int read;
+ u32 owner_counter;
+ int increment_owner_counter;
+
+ u32 packets;
+ u32 bytes;
+ u32 dropped;
+};
+
+struct tsnep_rx_entry {
+ struct tsnep_rx_desc *desc;
+ struct tsnep_rx_desc_wb *desc_wb;
+ dma_addr_t desc_dma;
+
+ u32 properties;
+
+ struct sk_buff *skb;
+ size_t len;
+ DEFINE_DMA_UNMAP_ADDR(dma);
+};
+
+struct tsnep_rx {
+ struct tsnep_adapter *adapter;
+ void __iomem *addr;
+
+ void *page[TSNEP_RING_PAGE_COUNT];
+ dma_addr_t page_dma[TSNEP_RING_PAGE_COUNT];
+
+ struct tsnep_rx_entry entry[TSNEP_RING_SIZE];
+ int read;
+ u32 owner_counter;
+ int increment_owner_counter;
+
+ u32 packets;
+ u32 bytes;
+ u32 dropped;
+ u32 multicast;
+};
+
+struct tsnep_queue {
+ struct tsnep_adapter *adapter;
+
+ struct tsnep_tx *tx;
+ struct tsnep_rx *rx;
+
+ struct napi_struct napi;
+
+ u32 irq_mask;
+};
+
+struct tsnep_adapter {
+ struct net_device *netdev;
+ u8 mac_address[ETH_ALEN];
+ struct mii_bus *mdiobus;
+ bool suppress_preamble;
+ phy_interface_t phy_mode;
+ struct phy_device *phydev;
+ int msg_enable;
+
+ struct platform_device *pdev;
+ struct device *dmadev;
+ void __iomem *addr;
+ int irq;
+
+ bool gate_control;
+ /* gate control lock */
+ struct mutex gate_control_lock;
+ bool gate_control_active;
+ struct tsnep_gcl gcl[2];
+ int next_gcl;
+
+ struct hwtstamp_config hwtstamp_config;
+ struct ptp_clock *ptp_clock;
+ struct ptp_clock_info ptp_clock_info;
+ /* ptp clock lock */
+ spinlock_t ptp_lock;
+
+ int num_tx_queues;
+ struct tsnep_tx tx[TSNEP_MAX_QUEUES];
+ int num_rx_queues;
+ struct tsnep_rx rx[TSNEP_MAX_QUEUES];
+
+ int num_queues;
+ struct tsnep_queue queue[TSNEP_MAX_QUEUES];
+};
+
+extern const struct ethtool_ops tsnep_ethtool_ops;
+
+int tsnep_ptp_init(struct tsnep_adapter *adapter);
+void tsnep_ptp_cleanup(struct tsnep_adapter *adapter);
+int tsnep_ptp_ioctl(struct net_device *netdev, struct ifreq *ifr, int cmd);
+
+int tsnep_tc_init(struct tsnep_adapter *adapter);
+void tsnep_tc_cleanup(struct tsnep_adapter *adapter);
+int tsnep_tc_setup(struct net_device *netdev, enum tc_setup_type type,
+ void *type_data);
+
+#if IS_ENABLED(CONFIG_TSNEP_SELFTESTS)
+int tsnep_ethtool_get_test_count(void);
+void tsnep_ethtool_get_test_strings(u8 *data);
+void tsnep_ethtool_self_test(struct net_device *netdev,
+ struct ethtool_test *eth_test, u64 *data);
+#else
+static inline int tsnep_ethtool_get_test_count(void)
+{
+ return -EOPNOTSUPP;
+}
+
+static inline void tsnep_ethtool_get_test_strings(u8 *data)
+{
+ /* not enabled */
+}
+
+static inline void tsnep_ethtool_self_test(struct net_device *dev,
+ struct ethtool_test *eth_test,
+ u64 *data)
+{
+ /* not enabled */
+}
+#endif /* CONFIG_TSNEP_SELFTESTS */
+
+void tsnep_get_system_time(struct tsnep_adapter *adapter, u64 *time);
+
+#endif /* _TSNEP_H */
diff --git a/drivers/net/ethernet/engleder/tsnep_ethtool.c b/drivers/net/ethernet/engleder/tsnep_ethtool.c
new file mode 100644
index 0000000..e6760dc
--- /dev/null
+++ b/drivers/net/ethernet/engleder/tsnep_ethtool.c
@@ -0,0 +1,293 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2021 Gerhard Engleder <gerhard@engleder-embedded.com> */
+
+#include "tsnep.h"
+
+static const char tsnep_stats_strings[][ETH_GSTRING_LEN] = {
+ "rx_packets",
+ "rx_bytes",
+ "rx_dropped",
+ "rx_multicast",
+ "rx_phy_errors",
+ "rx_forwarded_phy_errors",
+ "rx_invalid_frame_errors",
+ "tx_packets",
+ "tx_bytes",
+ "tx_dropped",
+};
+
+struct tsnep_stats {
+ u64 rx_packets;
+ u64 rx_bytes;
+ u64 rx_dropped;
+ u64 rx_multicast;
+ u64 rx_phy_errors;
+ u64 rx_forwarded_phy_errors;
+ u64 rx_invalid_frame_errors;
+ u64 tx_packets;
+ u64 tx_bytes;
+ u64 tx_dropped;
+};
+
+#define TSNEP_STATS_COUNT (sizeof(struct tsnep_stats) / sizeof(u64))
+
+static const char tsnep_rx_queue_stats_strings[][ETH_GSTRING_LEN] = {
+ "rx_%d_packets",
+ "rx_%d_bytes",
+ "rx_%d_dropped",
+ "rx_%d_multicast",
+ "rx_%d_no_descriptor_errors",
+ "rx_%d_buffer_too_small_errors",
+ "rx_%d_fifo_overflow_errors",
+ "rx_%d_invalid_frame_errors",
+};
+
+struct tsnep_rx_queue_stats {
+ u64 rx_packets;
+ u64 rx_bytes;
+ u64 rx_dropped;
+ u64 rx_multicast;
+ u64 rx_no_descriptor_errors;
+ u64 rx_buffer_too_small_errors;
+ u64 rx_fifo_overflow_errors;
+ u64 rx_invalid_frame_errors;
+};
+
+#define TSNEP_RX_QUEUE_STATS_COUNT (sizeof(struct tsnep_rx_queue_stats) / \
+ sizeof(u64))
+
+static const char tsnep_tx_queue_stats_strings[][ETH_GSTRING_LEN] = {
+ "tx_%d_packets",
+ "tx_%d_bytes",
+ "tx_%d_dropped",
+};
+
+struct tsnep_tx_queue_stats {
+ u64 tx_packets;
+ u64 tx_bytes;
+ u64 tx_dropped;
+};
+
+#define TSNEP_TX_QUEUE_STATS_COUNT (sizeof(struct tsnep_tx_queue_stats) / \
+ sizeof(u64))
+
+static void tsnep_ethtool_get_drvinfo(struct net_device *netdev,
+ struct ethtool_drvinfo *drvinfo)
+{
+ struct tsnep_adapter *adapter = netdev_priv(netdev);
+
+ strscpy(drvinfo->driver, TSNEP, sizeof(drvinfo->driver));
+ strscpy(drvinfo->bus_info, dev_name(&adapter->pdev->dev),
+ sizeof(drvinfo->bus_info));
+}
+
+static int tsnep_ethtool_get_regs_len(struct net_device *netdev)
+{
+ struct tsnep_adapter *adapter = netdev_priv(netdev);
+ int len;
+ int num_additional_queues;
+
+ len = TSNEP_MAC_SIZE;
+
+ /* first queue pair is within TSNEP_MAC_SIZE, only queues additional to
+ * the first queue pair extend the register length by TSNEP_QUEUE_SIZE
+ */
+ num_additional_queues =
+ max(adapter->num_tx_queues, adapter->num_rx_queues) - 1;
+ len += TSNEP_QUEUE_SIZE * num_additional_queues;
+
+ return len;
+}
+
+static void tsnep_ethtool_get_regs(struct net_device *netdev,
+ struct ethtool_regs *regs,
+ void *p)
+{
+ struct tsnep_adapter *adapter = netdev_priv(netdev);
+
+ regs->version = 1;
+
+ memcpy_fromio(p, adapter->addr, regs->len);
+}
+
+static u32 tsnep_ethtool_get_msglevel(struct net_device *netdev)
+{
+ struct tsnep_adapter *adapter = netdev_priv(netdev);
+
+ return adapter->msg_enable;
+}
+
+static void tsnep_ethtool_set_msglevel(struct net_device *netdev, u32 data)
+{
+ struct tsnep_adapter *adapter = netdev_priv(netdev);
+
+ adapter->msg_enable = data;
+}
+
+static void tsnep_ethtool_get_strings(struct net_device *netdev, u32 stringset,
+ u8 *data)
+{
+ struct tsnep_adapter *adapter = netdev_priv(netdev);
+ int rx_count = adapter->num_rx_queues;
+ int tx_count = adapter->num_tx_queues;
+ int i, j;
+
+ switch (stringset) {
+ case ETH_SS_STATS:
+ memcpy(data, tsnep_stats_strings, sizeof(tsnep_stats_strings));
+ data += sizeof(tsnep_stats_strings);
+
+ for (i = 0; i < rx_count; i++) {
+ for (j = 0; j < TSNEP_RX_QUEUE_STATS_COUNT; j++) {
+ snprintf(data, ETH_GSTRING_LEN,
+ tsnep_rx_queue_stats_strings[j], i);
+ data += ETH_GSTRING_LEN;
+ }
+ }
+
+ for (i = 0; i < tx_count; i++) {
+ for (j = 0; j < TSNEP_TX_QUEUE_STATS_COUNT; j++) {
+ snprintf(data, ETH_GSTRING_LEN,
+ tsnep_tx_queue_stats_strings[j], i);
+ data += ETH_GSTRING_LEN;
+ }
+ }
+ break;
+ case ETH_SS_TEST:
+ tsnep_ethtool_get_test_strings(data);
+ break;
+ }
+}
+
+static void tsnep_ethtool_get_ethtool_stats(struct net_device *netdev,
+ struct ethtool_stats *stats,
+ u64 *data)
+{
+ struct tsnep_adapter *adapter = netdev_priv(netdev);
+ int rx_count = adapter->num_rx_queues;
+ int tx_count = adapter->num_tx_queues;
+ struct tsnep_stats tsnep_stats;
+ struct tsnep_rx_queue_stats tsnep_rx_queue_stats;
+ struct tsnep_tx_queue_stats tsnep_tx_queue_stats;
+ u32 reg;
+ int i;
+
+ memset(&tsnep_stats, 0, sizeof(tsnep_stats));
+ for (i = 0; i < adapter->num_rx_queues; i++) {
+ tsnep_stats.rx_packets += adapter->rx[i].packets;
+ tsnep_stats.rx_bytes += adapter->rx[i].bytes;
+ tsnep_stats.rx_dropped += adapter->rx[i].dropped;
+ tsnep_stats.rx_multicast += adapter->rx[i].multicast;
+ }
+ reg = ioread32(adapter->addr + ECM_STAT);
+ tsnep_stats.rx_phy_errors =
+ (reg & ECM_STAT_RX_ERR_MASK) >> ECM_STAT_RX_ERR_SHIFT;
+ tsnep_stats.rx_forwarded_phy_errors =
+ (reg & ECM_STAT_FWD_RX_ERR_MASK) >> ECM_STAT_FWD_RX_ERR_SHIFT;
+ tsnep_stats.rx_invalid_frame_errors =
+ (reg & ECM_STAT_INV_FRM_MASK) >> ECM_STAT_INV_FRM_SHIFT;
+ for (i = 0; i < adapter->num_tx_queues; i++) {
+ tsnep_stats.tx_packets += adapter->tx[i].packets;
+ tsnep_stats.tx_bytes += adapter->tx[i].bytes;
+ tsnep_stats.tx_dropped += adapter->tx[i].dropped;
+ }
+ memcpy(data, &tsnep_stats, sizeof(tsnep_stats));
+ data += TSNEP_STATS_COUNT;
+
+ for (i = 0; i < rx_count; i++) {
+ memset(&tsnep_rx_queue_stats, 0, sizeof(tsnep_rx_queue_stats));
+ tsnep_rx_queue_stats.rx_packets = adapter->rx[i].packets;
+ tsnep_rx_queue_stats.rx_bytes = adapter->rx[i].bytes;
+ tsnep_rx_queue_stats.rx_dropped = adapter->rx[i].dropped;
+ tsnep_rx_queue_stats.rx_multicast = adapter->rx[i].multicast;
+ reg = ioread32(adapter->addr + TSNEP_QUEUE(i) +
+ TSNEP_RX_STATISTIC);
+ tsnep_rx_queue_stats.rx_no_descriptor_errors =
+ (reg & TSNEP_RX_STATISTIC_NO_DESC_MASK) >>
+ TSNEP_RX_STATISTIC_NO_DESC_SHIFT;
+ tsnep_rx_queue_stats.rx_buffer_too_small_errors =
+ (reg & TSNEP_RX_STATISTIC_BUFFER_TOO_SMALL_MASK) >>
+ TSNEP_RX_STATISTIC_BUFFER_TOO_SMALL_SHIFT;
+ tsnep_rx_queue_stats.rx_fifo_overflow_errors =
+ (reg & TSNEP_RX_STATISTIC_FIFO_OVERFLOW_MASK) >>
+ TSNEP_RX_STATISTIC_FIFO_OVERFLOW_SHIFT;
+ tsnep_rx_queue_stats.rx_invalid_frame_errors =
+ (reg & TSNEP_RX_STATISTIC_INVALID_FRAME_MASK) >>
+ TSNEP_RX_STATISTIC_INVALID_FRAME_SHIFT;
+ memcpy(data, &tsnep_rx_queue_stats,
+ sizeof(tsnep_rx_queue_stats));
+ data += TSNEP_RX_QUEUE_STATS_COUNT;
+ }
+
+ for (i = 0; i < tx_count; i++) {
+ memset(&tsnep_tx_queue_stats, 0, sizeof(tsnep_tx_queue_stats));
+ tsnep_tx_queue_stats.tx_packets += adapter->tx[i].packets;
+ tsnep_tx_queue_stats.tx_bytes += adapter->tx[i].bytes;
+ tsnep_tx_queue_stats.tx_dropped += adapter->tx[i].dropped;
+ memcpy(data, &tsnep_tx_queue_stats,
+ sizeof(tsnep_tx_queue_stats));
+ data += TSNEP_TX_QUEUE_STATS_COUNT;
+ }
+}
+
+static int tsnep_ethtool_get_sset_count(struct net_device *netdev, int sset)
+{
+ struct tsnep_adapter *adapter = netdev_priv(netdev);
+ int rx_count;
+ int tx_count;
+
+ switch (sset) {
+ case ETH_SS_STATS:
+ rx_count = adapter->num_rx_queues;
+ tx_count = adapter->num_tx_queues;
+ return TSNEP_STATS_COUNT +
+ TSNEP_RX_QUEUE_STATS_COUNT * rx_count +
+ TSNEP_TX_QUEUE_STATS_COUNT * tx_count;
+ case ETH_SS_TEST:
+ return tsnep_ethtool_get_test_count();
+ default:
+ return -EOPNOTSUPP;
+ }
+}
+
+static int tsnep_ethtool_get_ts_info(struct net_device *dev,
+ struct ethtool_ts_info *info)
+{
+ struct tsnep_adapter *adapter = netdev_priv(dev);
+
+ info->so_timestamping = SOF_TIMESTAMPING_TX_SOFTWARE |
+ SOF_TIMESTAMPING_RX_SOFTWARE |
+ SOF_TIMESTAMPING_SOFTWARE |
+ SOF_TIMESTAMPING_TX_HARDWARE |
+ SOF_TIMESTAMPING_RX_HARDWARE |
+ SOF_TIMESTAMPING_RAW_HARDWARE;
+
+ if (adapter->ptp_clock)
+ info->phc_index = ptp_clock_index(adapter->ptp_clock);
+ else
+ info->phc_index = -1;
+
+ info->tx_types = BIT(HWTSTAMP_TX_OFF) |
+ BIT(HWTSTAMP_TX_ON);
+ info->rx_filters = BIT(HWTSTAMP_FILTER_NONE) |
+ BIT(HWTSTAMP_FILTER_ALL);
+
+ return 0;
+}
+
+const struct ethtool_ops tsnep_ethtool_ops = {
+ .get_drvinfo = tsnep_ethtool_get_drvinfo,
+ .get_regs_len = tsnep_ethtool_get_regs_len,
+ .get_regs = tsnep_ethtool_get_regs,
+ .get_msglevel = tsnep_ethtool_get_msglevel,
+ .set_msglevel = tsnep_ethtool_set_msglevel,
+ .nway_reset = phy_ethtool_nway_reset,
+ .get_link = ethtool_op_get_link,
+ .self_test = tsnep_ethtool_self_test,
+ .get_strings = tsnep_ethtool_get_strings,
+ .get_ethtool_stats = tsnep_ethtool_get_ethtool_stats,
+ .get_sset_count = tsnep_ethtool_get_sset_count,
+ .get_ts_info = tsnep_ethtool_get_ts_info,
+ .get_link_ksettings = phy_ethtool_get_link_ksettings,
+ .set_link_ksettings = phy_ethtool_set_link_ksettings,
+};
diff --git a/drivers/net/ethernet/engleder/tsnep_hw.h b/drivers/net/ethernet/engleder/tsnep_hw.h
new file mode 100644
index 0000000..71cc857
--- /dev/null
+++ b/drivers/net/ethernet/engleder/tsnep_hw.h
@@ -0,0 +1,230 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021 Gerhard Engleder <gerhard@engleder-embedded.com> */
+
+/* Hardware definition of TSNEP and EtherCAT MAC device */
+
+#ifndef _TSNEP_HW_H
+#define _TSNEP_HW_H
+
+#include <linux/types.h>
+
+/* type */
+#define ECM_TYPE 0x0000
+#define ECM_REVISION_MASK 0x000000FF
+#define ECM_REVISION_SHIFT 0
+#define ECM_VERSION_MASK 0x0000FF00
+#define ECM_VERSION_SHIFT 8
+#define ECM_QUEUE_COUNT_MASK 0x00070000
+#define ECM_QUEUE_COUNT_SHIFT 16
+#define ECM_GATE_CONTROL 0x02000000
+
+/* system time */
+#define ECM_SYSTEM_TIME_LOW 0x0008
+#define ECM_SYSTEM_TIME_HIGH 0x000C
+
+/* clock */
+#define ECM_CLOCK_RATE 0x0010
+#define ECM_CLOCK_RATE_OFFSET_MASK 0x7FFFFFFF
+#define ECM_CLOCK_RATE_OFFSET_SIGN 0x80000000
+
+/* interrupt */
+#define ECM_INT_ENABLE 0x0018
+#define ECM_INT_ACTIVE 0x001C
+#define ECM_INT_ACKNOWLEDGE 0x001C
+#define ECM_INT_LINK 0x00000020
+#define ECM_INT_TX_0 0x00000100
+#define ECM_INT_RX_0 0x00000200
+#define ECM_INT_ALL 0x7FFFFFFF
+#define ECM_INT_DISABLE 0x80000000
+
+/* reset */
+#define ECM_RESET 0x0020
+#define ECM_RESET_COMMON 0x00000001
+#define ECM_RESET_CHANNEL 0x00000100
+#define ECM_RESET_TXRX 0x00010000
+
+/* control and status */
+#define ECM_STATUS 0x0080
+#define ECM_LINK_MODE_OFF 0x01000000
+#define ECM_LINK_MODE_100 0x02000000
+#define ECM_LINK_MODE_1000 0x04000000
+#define ECM_NO_LINK 0x01000000
+#define ECM_LINK_MODE_MASK 0x06000000
+
+/* management data */
+#define ECM_MD_CONTROL 0x0084
+#define ECM_MD_STATUS 0x0084
+#define ECM_MD_PREAMBLE 0x00000001
+#define ECM_MD_READ 0x00000004
+#define ECM_MD_WRITE 0x00000002
+#define ECM_MD_ADDR_MASK 0x000000F8
+#define ECM_MD_ADDR_SHIFT 3
+#define ECM_MD_PHY_ADDR_MASK 0x00001F00
+#define ECM_MD_PHY_ADDR_SHIFT 8
+#define ECM_MD_BUSY 0x00000001
+#define ECM_MD_DATA_MASK 0xFFFF0000
+#define ECM_MD_DATA_SHIFT 16
+
+/* statistic */
+#define ECM_STAT 0x00B0
+#define ECM_STAT_RX_ERR_MASK 0x000000FF
+#define ECM_STAT_RX_ERR_SHIFT 0
+#define ECM_STAT_INV_FRM_MASK 0x0000FF00
+#define ECM_STAT_INV_FRM_SHIFT 8
+#define ECM_STAT_FWD_RX_ERR_MASK 0x00FF0000
+#define ECM_STAT_FWD_RX_ERR_SHIFT 16
+
+/* tsnep */
+#define TSNEP_MAC_SIZE 0x4000
+#define TSNEP_QUEUE_SIZE 0x1000
+#define TSNEP_QUEUE(n) ({ typeof(n) __n = (n); \
+ (__n) == 0 ? \
+ 0 : \
+ TSNEP_MAC_SIZE + TSNEP_QUEUE_SIZE * ((__n) - 1); })
+#define TSNEP_MAX_QUEUES 8
+#define TSNEP_MAX_FRAME_SIZE (2 * 1024) /* hardware supports actually 16k */
+#define TSNEP_DESC_SIZE 256
+#define TSNEP_DESC_OFFSET 128
+
+/* tsnep register */
+#define TSNEP_INFO 0x0100
+#define TSNEP_INFO_RX_ASSIGN 0x00010000
+#define TSNEP_INFO_TX_TIME 0x00020000
+#define TSNEP_CONTROL 0x0108
+#define TSNEP_CONTROL_TX_RESET 0x00000001
+#define TSNEP_CONTROL_TX_ENABLE 0x00000002
+#define TSNEP_CONTROL_TX_DMA_ERROR 0x00000010
+#define TSNEP_CONTROL_TX_DESC_ERROR 0x00000020
+#define TSNEP_CONTROL_RX_RESET 0x00000100
+#define TSNEP_CONTROL_RX_ENABLE 0x00000200
+#define TSNEP_CONTROL_RX_DISABLE 0x00000400
+#define TSNEP_CONTROL_RX_DMA_ERROR 0x00001000
+#define TSNEP_CONTROL_RX_DESC_ERROR 0x00002000
+#define TSNEP_TX_DESC_ADDR_LOW 0x0140
+#define TSNEP_TX_DESC_ADDR_HIGH 0x0144
+#define TSNEP_RX_DESC_ADDR_LOW 0x0180
+#define TSNEP_RX_DESC_ADDR_HIGH 0x0184
+#define TSNEP_RESET_OWNER_COUNTER 0x01
+#define TSNEP_RX_STATISTIC 0x0190
+#define TSNEP_RX_STATISTIC_NO_DESC_MASK 0x000000FF
+#define TSNEP_RX_STATISTIC_NO_DESC_SHIFT 0
+#define TSNEP_RX_STATISTIC_BUFFER_TOO_SMALL_MASK 0x0000FF00
+#define TSNEP_RX_STATISTIC_BUFFER_TOO_SMALL_SHIFT 8
+#define TSNEP_RX_STATISTIC_FIFO_OVERFLOW_MASK 0x00FF0000
+#define TSNEP_RX_STATISTIC_FIFO_OVERFLOW_SHIFT 16
+#define TSNEP_RX_STATISTIC_INVALID_FRAME_MASK 0xFF000000
+#define TSNEP_RX_STATISTIC_INVALID_FRAME_SHIFT 24
+#define TSNEP_RX_STATISTIC_NO_DESC 0x0190
+#define TSNEP_RX_STATISTIC_BUFFER_TOO_SMALL 0x0191
+#define TSNEP_RX_STATISTIC_FIFO_OVERFLOW 0x0192
+#define TSNEP_RX_STATISTIC_INVALID_FRAME 0x0193
+#define TSNEP_RX_ASSIGN 0x01A0
+#define TSNEP_RX_ASSIGN_ETHER_TYPE_ACTIVE 0x00000001
+#define TSNEP_RX_ASSIGN_ETHER_TYPE_MASK 0xFFFF0000
+#define TSNEP_RX_ASSIGN_ETHER_TYPE_SHIFT 16
+#define TSNEP_MAC_ADDRESS_LOW 0x0800
+#define TSNEP_MAC_ADDRESS_HIGH 0x0804
+#define TSNEP_RX_FILTER 0x0806
+#define TSNEP_RX_FILTER_ACCEPT_ALL_MULTICASTS 0x0001
+#define TSNEP_RX_FILTER_ACCEPT_ALL_UNICASTS 0x0002
+#define TSNEP_GC 0x0808
+#define TSNEP_GC_ENABLE_A 0x00000002
+#define TSNEP_GC_ENABLE_B 0x00000004
+#define TSNEP_GC_DISABLE 0x00000008
+#define TSNEP_GC_ENABLE_TIMEOUT 0x00000010
+#define TSNEP_GC_ACTIVE_A 0x00000002
+#define TSNEP_GC_ACTIVE_B 0x00000004
+#define TSNEP_GC_CHANGE_AB 0x00000008
+#define TSNEP_GC_TIMEOUT_ACTIVE 0x00000010
+#define TSNEP_GC_TIMEOUT_SIGNAL 0x00000020
+#define TSNEP_GC_LIST_ERROR 0x00000080
+#define TSNEP_GC_OPEN 0x00FF0000
+#define TSNEP_GC_OPEN_SHIFT 16
+#define TSNEP_GC_NEXT_OPEN 0xFF000000
+#define TSNEP_GC_NEXT_OPEN_SHIFT 24
+#define TSNEP_GC_TIMEOUT 131072
+#define TSNEP_GC_TIME 0x080C
+#define TSNEP_GC_CHANGE 0x0810
+#define TSNEP_GCL_A 0x2000
+#define TSNEP_GCL_B 0x2800
+#define TSNEP_GCL_SIZE SZ_2K
+
+/* tsnep gate control list operation */
+struct tsnep_gcl_operation {
+ u32 properties;
+ u32 interval;
+};
+
+#define TSNEP_GCL_COUNT (TSNEP_GCL_SIZE / sizeof(struct tsnep_gcl_operation))
+#define TSNEP_GCL_MASK 0x000000FF
+#define TSNEP_GCL_INSERT 0x20000000
+#define TSNEP_GCL_CHANGE 0x40000000
+#define TSNEP_GCL_LAST 0x80000000
+#define TSNEP_GCL_MIN_INTERVAL 32
+
+/* tsnep TX/RX descriptor */
+#define TSNEP_DESC_SIZE 256
+#define TSNEP_DESC_SIZE_DATA_AFTER 2048
+#define TSNEP_DESC_OFFSET 128
+#define TSNEP_DESC_OWNER_COUNTER_MASK 0xC0000000
+#define TSNEP_DESC_OWNER_COUNTER_SHIFT 30
+#define TSNEP_DESC_LENGTH_MASK 0x00003FFF
+#define TSNEP_DESC_INTERRUPT_FLAG 0x00040000
+#define TSNEP_DESC_EXTENDED_WRITEBACK_FLAG 0x00080000
+#define TSNEP_DESC_NO_LINK_FLAG 0x01000000
+
+/* tsnep TX descriptor */
+struct tsnep_tx_desc {
+ __le32 properties;
+ __le32 more_properties;
+ __le32 reserved[2];
+ __le64 next;
+ __le64 tx;
+};
+
+#define TSNEP_TX_DESC_OWNER_MASK 0xE0000000
+#define TSNEP_TX_DESC_OWNER_USER_FLAG 0x20000000
+#define TSNEP_TX_DESC_LAST_FRAGMENT_FLAG 0x00010000
+#define TSNEP_TX_DESC_DATA_AFTER_DESC_FLAG 0x00020000
+
+/* tsnep TX descriptor writeback */
+struct tsnep_tx_desc_wb {
+ __le32 properties;
+ __le32 reserved1[3];
+ __le64 timestamp;
+ __le32 dma_delay;
+ __le32 reserved2;
+};
+
+#define TSNEP_TX_DESC_UNDERRUN_ERROR_FLAG 0x00010000
+#define TSNEP_TX_DESC_DMA_DELAY_FIRST_DATA_MASK 0x0000FFFC
+#define TSNEP_TX_DESC_DMA_DELAY_FIRST_DATA_SHIFT 2
+#define TSNEP_TX_DESC_DMA_DELAY_LAST_DATA_MASK 0xFFFC0000
+#define TSNEP_TX_DESC_DMA_DELAY_LAST_DATA_SHIFT 18
+#define TSNEP_TX_DESC_DMA_DELAY_NS 64
+
+/* tsnep RX descriptor */
+struct tsnep_rx_desc {
+ __le32 properties;
+ __le32 reserved[3];
+ __le64 next;
+ __le64 rx;
+};
+
+#define TSNEP_RX_DESC_BUFFER_SIZE_MASK 0x00003FFC
+
+/* tsnep RX descriptor writeback */
+struct tsnep_rx_desc_wb {
+ __le32 properties;
+ __le32 reserved[7];
+};
+
+/* tsnep RX inline meta */
+struct tsnep_rx_inline {
+ __le64 reserved;
+ __le64 timestamp;
+};
+
+#define TSNEP_RX_INLINE_METADATA_SIZE (sizeof(struct tsnep_rx_inline))
+
+#endif /* _TSNEP_HW_H */
diff --git a/drivers/net/ethernet/engleder/tsnep_main.c b/drivers/net/ethernet/engleder/tsnep_main.c
new file mode 100644
index 0000000..904f330
--- /dev/null
+++ b/drivers/net/ethernet/engleder/tsnep_main.c
@@ -0,0 +1,1272 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2021 Gerhard Engleder <gerhard@engleder-embedded.com> */
+
+/* TSN endpoint Ethernet MAC driver
+ *
+ * The TSN endpoint Ethernet MAC is a FPGA based network device for real-time
+ * communication. It is designed for endpoints within TSN (Time Sensitive
+ * Networking) networks; e.g., for PLCs in the industrial automation case.
+ *
+ * It supports multiple TX/RX queue pairs. The first TX/RX queue pair is used
+ * by the driver.
+ *
+ * More information can be found here:
+ * - www.embedded-experts.at/tsn
+ * - www.engleder-embedded.com
+ */
+
+#include "tsnep.h"
+#include "tsnep_hw.h"
+
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/of_net.h>
+#include <linux/of_mdio.h>
+#include <linux/interrupt.h>
+#include <linux/etherdevice.h>
+#include <linux/phy.h>
+#include <linux/iopoll.h>
+
+#define RX_SKB_LENGTH (round_up(TSNEP_RX_INLINE_METADATA_SIZE + ETH_HLEN + \
+ TSNEP_MAX_FRAME_SIZE + ETH_FCS_LEN, 4))
+#define RX_SKB_RESERVE ((16 - TSNEP_RX_INLINE_METADATA_SIZE) + NET_IP_ALIGN)
+#define RX_SKB_ALLOC_LENGTH (RX_SKB_RESERVE + RX_SKB_LENGTH)
+
+#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
+#define DMA_ADDR_HIGH(dma_addr) ((u32)(((dma_addr) >> 32) & 0xFFFFFFFF))
+#else
+#define DMA_ADDR_HIGH(dma_addr) ((u32)(0))
+#endif
+#define DMA_ADDR_LOW(dma_addr) ((u32)((dma_addr) & 0xFFFFFFFF))
+
+static void tsnep_enable_irq(struct tsnep_adapter *adapter, u32 mask)
+{
+ iowrite32(mask, adapter->addr + ECM_INT_ENABLE);
+}
+
+static void tsnep_disable_irq(struct tsnep_adapter *adapter, u32 mask)
+{
+ mask |= ECM_INT_DISABLE;
+ iowrite32(mask, adapter->addr + ECM_INT_ENABLE);
+}
+
+static irqreturn_t tsnep_irq(int irq, void *arg)
+{
+ struct tsnep_adapter *adapter = arg;
+ u32 active = ioread32(adapter->addr + ECM_INT_ACTIVE);
+
+ /* acknowledge interrupt */
+ if (active != 0)
+ iowrite32(active, adapter->addr + ECM_INT_ACKNOWLEDGE);
+
+ /* handle link interrupt */
+ if ((active & ECM_INT_LINK) != 0) {
+ if (adapter->netdev->phydev)
+ phy_mac_interrupt(adapter->netdev->phydev);
+ }
+
+ /* handle TX/RX queue 0 interrupt */
+ if ((active & adapter->queue[0].irq_mask) != 0) {
+ if (adapter->netdev) {
+ tsnep_disable_irq(adapter, adapter->queue[0].irq_mask);
+ napi_schedule(&adapter->queue[0].napi);
+ }
+ }
+
+ return IRQ_HANDLED;
+}
+
+static int tsnep_mdiobus_read(struct mii_bus *bus, int addr, int regnum)
+{
+ struct tsnep_adapter *adapter = bus->priv;
+ u32 md;
+ int retval;
+
+ if (regnum & MII_ADDR_C45)
+ return -EOPNOTSUPP;
+
+ md = ECM_MD_READ;
+ if (!adapter->suppress_preamble)
+ md |= ECM_MD_PREAMBLE;
+ md |= (regnum << ECM_MD_ADDR_SHIFT) & ECM_MD_ADDR_MASK;
+ md |= (addr << ECM_MD_PHY_ADDR_SHIFT) & ECM_MD_PHY_ADDR_MASK;
+ iowrite32(md, adapter->addr + ECM_MD_CONTROL);
+ retval = readl_poll_timeout_atomic(adapter->addr + ECM_MD_STATUS, md,
+ !(md & ECM_MD_BUSY), 16, 1000);
+ if (retval != 0)
+ return retval;
+
+ return (md & ECM_MD_DATA_MASK) >> ECM_MD_DATA_SHIFT;
+}
+
+static int tsnep_mdiobus_write(struct mii_bus *bus, int addr, int regnum,
+ u16 val)
+{
+ struct tsnep_adapter *adapter = bus->priv;
+ u32 md;
+ int retval;
+
+ if (regnum & MII_ADDR_C45)
+ return -EOPNOTSUPP;
+
+ md = ECM_MD_WRITE;
+ if (!adapter->suppress_preamble)
+ md |= ECM_MD_PREAMBLE;
+ md |= (regnum << ECM_MD_ADDR_SHIFT) & ECM_MD_ADDR_MASK;
+ md |= (addr << ECM_MD_PHY_ADDR_SHIFT) & ECM_MD_PHY_ADDR_MASK;
+ md |= ((u32)val << ECM_MD_DATA_SHIFT) & ECM_MD_DATA_MASK;
+ iowrite32(md, adapter->addr + ECM_MD_CONTROL);
+ retval = readl_poll_timeout_atomic(adapter->addr + ECM_MD_STATUS, md,
+ !(md & ECM_MD_BUSY), 16, 1000);
+ if (retval != 0)
+ return retval;
+
+ return 0;
+}
+
+static void tsnep_phy_link_status_change(struct net_device *netdev)
+{
+ struct tsnep_adapter *adapter = netdev_priv(netdev);
+ struct phy_device *phydev = netdev->phydev;
+ u32 mode;
+
+ if (phydev->link) {
+ switch (phydev->speed) {
+ case SPEED_100:
+ mode = ECM_LINK_MODE_100;
+ break;
+ case SPEED_1000:
+ mode = ECM_LINK_MODE_1000;
+ break;
+ default:
+ mode = ECM_LINK_MODE_OFF;
+ break;
+ }
+ iowrite32(mode, adapter->addr + ECM_STATUS);
+ }
+
+ phy_print_status(netdev->phydev);
+}
+
+static int tsnep_phy_open(struct tsnep_adapter *adapter)
+{
+ struct phy_device *phydev;
+ struct ethtool_eee ethtool_eee;
+ int retval;
+
+ retval = phy_connect_direct(adapter->netdev, adapter->phydev,
+ tsnep_phy_link_status_change,
+ adapter->phy_mode);
+ if (retval)
+ return retval;
+ phydev = adapter->netdev->phydev;
+
+ /* MAC supports only 100Mbps|1000Mbps full duplex
+ * SPE (Single Pair Ethernet) is also an option but not implemented yet
+ */
+ phy_remove_link_mode(phydev, ETHTOOL_LINK_MODE_10baseT_Half_BIT);
+ phy_remove_link_mode(phydev, ETHTOOL_LINK_MODE_10baseT_Full_BIT);
+ phy_remove_link_mode(phydev, ETHTOOL_LINK_MODE_100baseT_Half_BIT);
+ phy_remove_link_mode(phydev, ETHTOOL_LINK_MODE_1000baseT_Half_BIT);
+
+ /* disable EEE autoneg, EEE not supported by TSNEP */
+ memset(ðtool_eee, 0, sizeof(ethtool_eee));
+ phy_ethtool_set_eee(adapter->phydev, ðtool_eee);
+
+ adapter->phydev->irq = PHY_MAC_INTERRUPT;
+ phy_start(adapter->phydev);
+
+ return 0;
+}
+
+static void tsnep_phy_close(struct tsnep_adapter *adapter)
+{
+ phy_stop(adapter->netdev->phydev);
+ phy_disconnect(adapter->netdev->phydev);
+ adapter->netdev->phydev = NULL;
+}
+
+static void tsnep_tx_ring_cleanup(struct tsnep_tx *tx)
+{
+ struct device *dmadev = tx->adapter->dmadev;
+ int i;
+
+ memset(tx->entry, 0, sizeof(tx->entry));
+
+ for (i = 0; i < TSNEP_RING_PAGE_COUNT; i++) {
+ if (tx->page[i]) {
+ dma_free_coherent(dmadev, PAGE_SIZE, tx->page[i],
+ tx->page_dma[i]);
+ tx->page[i] = NULL;
+ tx->page_dma[i] = 0;
+ }
+ }
+}
+
+static int tsnep_tx_ring_init(struct tsnep_tx *tx)
+{
+ struct device *dmadev = tx->adapter->dmadev;
+ struct tsnep_tx_entry *entry;
+ struct tsnep_tx_entry *next_entry;
+ int i, j;
+ int retval;
+
+ for (i = 0; i < TSNEP_RING_PAGE_COUNT; i++) {
+ tx->page[i] =
+ dma_alloc_coherent(dmadev, PAGE_SIZE, &tx->page_dma[i],
+ GFP_KERNEL);
+ if (!tx->page[i]) {
+ retval = -ENOMEM;
+ goto alloc_failed;
+ }
+ for (j = 0; j < TSNEP_RING_ENTRIES_PER_PAGE; j++) {
+ entry = &tx->entry[TSNEP_RING_ENTRIES_PER_PAGE * i + j];
+ entry->desc_wb = (struct tsnep_tx_desc_wb *)
+ (((u8 *)tx->page[i]) + TSNEP_DESC_SIZE * j);
+ entry->desc = (struct tsnep_tx_desc *)
+ (((u8 *)entry->desc_wb) + TSNEP_DESC_OFFSET);
+ entry->desc_dma = tx->page_dma[i] + TSNEP_DESC_SIZE * j;
+ }
+ }
+ for (i = 0; i < TSNEP_RING_SIZE; i++) {
+ entry = &tx->entry[i];
+ next_entry = &tx->entry[(i + 1) % TSNEP_RING_SIZE];
+ entry->desc->next = __cpu_to_le64(next_entry->desc_dma);
+ }
+
+ return 0;
+
+alloc_failed:
+ tsnep_tx_ring_cleanup(tx);
+ return retval;
+}
+
+static void tsnep_tx_activate(struct tsnep_tx *tx, int index, bool last)
+{
+ struct tsnep_tx_entry *entry = &tx->entry[index];
+
+ entry->properties = 0;
+ if (entry->skb) {
+ entry->properties =
+ skb_pagelen(entry->skb) & TSNEP_DESC_LENGTH_MASK;
+ entry->properties |= TSNEP_DESC_INTERRUPT_FLAG;
+ if (skb_shinfo(entry->skb)->tx_flags & SKBTX_IN_PROGRESS)
+ entry->properties |= TSNEP_DESC_EXTENDED_WRITEBACK_FLAG;
+
+ /* toggle user flag to prevent false acknowledge
+ *
+ * Only the first fragment is acknowledged. For all other
+ * fragments no acknowledge is done and the last written owner
+ * counter stays in the writeback descriptor. Therefore, it is
+ * possible that the last written owner counter is identical to
+ * the new incremented owner counter and a false acknowledge is
+ * detected before the real acknowledge has been done by
+ * hardware.
+ *
+ * The user flag is used to prevent this situation. The user
+ * flag is copied to the writeback descriptor by the hardware
+ * and is used as additional acknowledge data. By toggeling the
+ * user flag only for the first fragment (which is
+ * acknowledged), it is guaranteed that the last acknowledge
+ * done for this descriptor has used a different user flag and
+ * cannot be detected as false acknowledge.
+ */
+ entry->owner_user_flag = !entry->owner_user_flag;
+ }
+ if (last)
+ entry->properties |= TSNEP_TX_DESC_LAST_FRAGMENT_FLAG;
+ if (index == tx->increment_owner_counter) {
+ tx->owner_counter++;
+ if (tx->owner_counter == 4)
+ tx->owner_counter = 1;
+ tx->increment_owner_counter--;
+ if (tx->increment_owner_counter < 0)
+ tx->increment_owner_counter = TSNEP_RING_SIZE - 1;
+ }
+ entry->properties |=
+ (tx->owner_counter << TSNEP_DESC_OWNER_COUNTER_SHIFT) &
+ TSNEP_DESC_OWNER_COUNTER_MASK;
+ if (entry->owner_user_flag)
+ entry->properties |= TSNEP_TX_DESC_OWNER_USER_FLAG;
+ entry->desc->more_properties =
+ __cpu_to_le32(entry->len & TSNEP_DESC_LENGTH_MASK);
+
+ /* descriptor properties shall be written last, because valid data is
+ * signaled there
+ */
+ dma_wmb();
+
+ entry->desc->properties = __cpu_to_le32(entry->properties);
+}
+
+static int tsnep_tx_desc_available(struct tsnep_tx *tx)
+{
+ if (tx->read <= tx->write)
+ return TSNEP_RING_SIZE - tx->write + tx->read - 1;
+ else
+ return tx->read - tx->write - 1;
+}
+
+static int tsnep_tx_map(struct sk_buff *skb, struct tsnep_tx *tx, int count)
+{
+ struct device *dmadev = tx->adapter->dmadev;
+ struct tsnep_tx_entry *entry;
+ unsigned int len;
+ dma_addr_t dma;
+ int i;
+
+ for (i = 0; i < count; i++) {
+ entry = &tx->entry[(tx->write + i) % TSNEP_RING_SIZE];
+
+ if (i == 0) {
+ len = skb_headlen(skb);
+ dma = dma_map_single(dmadev, skb->data, len,
+ DMA_TO_DEVICE);
+ } else {
+ len = skb_frag_size(&skb_shinfo(skb)->frags[i - 1]);
+ dma = skb_frag_dma_map(dmadev,
+ &skb_shinfo(skb)->frags[i - 1],
+ 0, len, DMA_TO_DEVICE);
+ }
+ if (dma_mapping_error(dmadev, dma))
+ return -ENOMEM;
+
+ entry->len = len;
+ dma_unmap_addr_set(entry, dma, dma);
+
+ entry->desc->tx = __cpu_to_le64(dma);
+ }
+
+ return 0;
+}
+
+static void tsnep_tx_unmap(struct tsnep_tx *tx, int count)
+{
+ struct device *dmadev = tx->adapter->dmadev;
+ struct tsnep_tx_entry *entry;
+ int i;
+
+ for (i = 0; i < count; i++) {
+ entry = &tx->entry[(tx->read + i) % TSNEP_RING_SIZE];
+
+ if (entry->len) {
+ if (i == 0)
+ dma_unmap_single(dmadev,
+ dma_unmap_addr(entry, dma),
+ dma_unmap_len(entry, len),
+ DMA_TO_DEVICE);
+ else
+ dma_unmap_page(dmadev,
+ dma_unmap_addr(entry, dma),
+ dma_unmap_len(entry, len),
+ DMA_TO_DEVICE);
+ entry->len = 0;
+ }
+ }
+}
+
+static netdev_tx_t tsnep_xmit_frame_ring(struct sk_buff *skb,
+ struct tsnep_tx *tx)
+{
+ unsigned long flags;
+ int count = 1;
+ struct tsnep_tx_entry *entry;
+ int i;
+ int retval;
+
+ if (skb_shinfo(skb)->nr_frags > 0)
+ count += skb_shinfo(skb)->nr_frags;
+
+ spin_lock_irqsave(&tx->lock, flags);
+
+ if (tsnep_tx_desc_available(tx) < count) {
+ /* ring full, shall not happen because queue is stopped if full
+ * below
+ */
+ netif_stop_queue(tx->adapter->netdev);
+
+ spin_unlock_irqrestore(&tx->lock, flags);
+
+ return NETDEV_TX_BUSY;
+ }
+
+ entry = &tx->entry[tx->write];
+ entry->skb = skb;
+
+ retval = tsnep_tx_map(skb, tx, count);
+ if (retval != 0) {
+ tsnep_tx_unmap(tx, count);
+ dev_kfree_skb_any(entry->skb);
+ entry->skb = NULL;
+
+ tx->dropped++;
+
+ spin_unlock_irqrestore(&tx->lock, flags);
+
+ netdev_err(tx->adapter->netdev, "TX DMA map failed\n");
+
+ return NETDEV_TX_OK;
+ }
+
+ if (skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP)
+ skb_shinfo(skb)->tx_flags |= SKBTX_IN_PROGRESS;
+
+ for (i = 0; i < count; i++)
+ tsnep_tx_activate(tx, (tx->write + i) % TSNEP_RING_SIZE,
+ i == (count - 1));
+ tx->write = (tx->write + count) % TSNEP_RING_SIZE;
+
+ skb_tx_timestamp(skb);
+
+ /* descriptor properties shall be valid before hardware is notified */
+ dma_wmb();
+
+ iowrite32(TSNEP_CONTROL_TX_ENABLE, tx->addr + TSNEP_CONTROL);
+
+ if (tsnep_tx_desc_available(tx) < (MAX_SKB_FRAGS + 1)) {
+ /* ring can get full with next frame */
+ netif_stop_queue(tx->adapter->netdev);
+ }
+
+ tx->packets++;
+ tx->bytes += skb_pagelen(entry->skb) + ETH_FCS_LEN;
+
+ spin_unlock_irqrestore(&tx->lock, flags);
+
+ return NETDEV_TX_OK;
+}
+
+static bool tsnep_tx_poll(struct tsnep_tx *tx, int napi_budget)
+{
+ unsigned long flags;
+ int budget = 128;
+ struct tsnep_tx_entry *entry;
+ int count;
+
+ spin_lock_irqsave(&tx->lock, flags);
+
+ do {
+ if (tx->read == tx->write)
+ break;
+
+ entry = &tx->entry[tx->read];
+ if ((__le32_to_cpu(entry->desc_wb->properties) &
+ TSNEP_TX_DESC_OWNER_MASK) !=
+ (entry->properties & TSNEP_TX_DESC_OWNER_MASK))
+ break;
+
+ /* descriptor properties shall be read first, because valid data
+ * is signaled there
+ */
+ dma_rmb();
+
+ count = 1;
+ if (skb_shinfo(entry->skb)->nr_frags > 0)
+ count += skb_shinfo(entry->skb)->nr_frags;
+
+ tsnep_tx_unmap(tx, count);
+
+ if ((skb_shinfo(entry->skb)->tx_flags & SKBTX_IN_PROGRESS) &&
+ (__le32_to_cpu(entry->desc_wb->properties) &
+ TSNEP_DESC_EXTENDED_WRITEBACK_FLAG)) {
+ struct skb_shared_hwtstamps hwtstamps;
+ u64 timestamp =
+ __le64_to_cpu(entry->desc_wb->timestamp);
+
+ memset(&hwtstamps, 0, sizeof(hwtstamps));
+ hwtstamps.hwtstamp = ns_to_ktime(timestamp);
+
+ skb_tstamp_tx(entry->skb, &hwtstamps);
+ }
+
+ napi_consume_skb(entry->skb, budget);
+ entry->skb = NULL;
+
+ tx->read = (tx->read + count) % TSNEP_RING_SIZE;
+
+ budget--;
+ } while (likely(budget));
+
+ if ((tsnep_tx_desc_available(tx) >= ((MAX_SKB_FRAGS + 1) * 2)) &&
+ netif_queue_stopped(tx->adapter->netdev)) {
+ netif_wake_queue(tx->adapter->netdev);
+ }
+
+ spin_unlock_irqrestore(&tx->lock, flags);
+
+ return (budget != 0);
+}
+
+static int tsnep_tx_open(struct tsnep_adapter *adapter, void __iomem *addr,
+ struct tsnep_tx *tx)
+{
+ dma_addr_t dma;
+ int retval;
+
+ memset(tx, 0, sizeof(*tx));
+ tx->adapter = adapter;
+ tx->addr = addr;
+
+ retval = tsnep_tx_ring_init(tx);
+ if (retval)
+ return retval;
+
+ dma = tx->entry[0].desc_dma | TSNEP_RESET_OWNER_COUNTER;
+ iowrite32(DMA_ADDR_LOW(dma), tx->addr + TSNEP_TX_DESC_ADDR_LOW);
+ iowrite32(DMA_ADDR_HIGH(dma), tx->addr + TSNEP_TX_DESC_ADDR_HIGH);
+ tx->owner_counter = 1;
+ tx->increment_owner_counter = TSNEP_RING_SIZE - 1;
+
+ spin_lock_init(&tx->lock);
+
+ return 0;
+}
+
+static void tsnep_tx_close(struct tsnep_tx *tx)
+{
+ u32 val;
+
+ readx_poll_timeout(ioread32, tx->addr + TSNEP_CONTROL, val,
+ ((val & TSNEP_CONTROL_TX_ENABLE) == 0), 10000,
+ 1000000);
+
+ tsnep_tx_ring_cleanup(tx);
+}
+
+static void tsnep_rx_ring_cleanup(struct tsnep_rx *rx)
+{
+ struct device *dmadev = rx->adapter->dmadev;
+ struct tsnep_rx_entry *entry;
+ int i;
+
+ for (i = 0; i < TSNEP_RING_SIZE; i++) {
+ entry = &rx->entry[i];
+ if (dma_unmap_addr(entry, dma))
+ dma_unmap_single(dmadev, dma_unmap_addr(entry, dma),
+ dma_unmap_len(entry, len),
+ DMA_FROM_DEVICE);
+ if (entry->skb)
+ dev_kfree_skb(entry->skb);
+ }
+
+ memset(rx->entry, 0, sizeof(rx->entry));
+
+ for (i = 0; i < TSNEP_RING_PAGE_COUNT; i++) {
+ if (rx->page[i]) {
+ dma_free_coherent(dmadev, PAGE_SIZE, rx->page[i],
+ rx->page_dma[i]);
+ rx->page[i] = NULL;
+ rx->page_dma[i] = 0;
+ }
+ }
+}
+
+static int tsnep_rx_alloc_and_map_skb(struct tsnep_rx *rx,
+ struct tsnep_rx_entry *entry)
+{
+ struct device *dmadev = rx->adapter->dmadev;
+ struct sk_buff *skb;
+ dma_addr_t dma;
+
+ skb = __netdev_alloc_skb(rx->adapter->netdev, RX_SKB_ALLOC_LENGTH,
+ GFP_ATOMIC | GFP_DMA);
+ if (!skb)
+ return -ENOMEM;
+
+ skb_reserve(skb, RX_SKB_RESERVE);
+
+ dma = dma_map_single(dmadev, skb->data, RX_SKB_LENGTH,
+ DMA_FROM_DEVICE);
+ if (dma_mapping_error(dmadev, dma)) {
+ dev_kfree_skb(skb);
+ return -ENOMEM;
+ }
+
+ entry->skb = skb;
+ entry->len = RX_SKB_LENGTH;
+ dma_unmap_addr_set(entry, dma, dma);
+ entry->desc->rx = __cpu_to_le64(dma);
+
+ return 0;
+}
+
+static int tsnep_rx_ring_init(struct tsnep_rx *rx)
+{
+ struct device *dmadev = rx->adapter->dmadev;
+ struct tsnep_rx_entry *entry;
+ struct tsnep_rx_entry *next_entry;
+ int i, j;
+ int retval;
+
+ for (i = 0; i < TSNEP_RING_PAGE_COUNT; i++) {
+ rx->page[i] =
+ dma_alloc_coherent(dmadev, PAGE_SIZE, &rx->page_dma[i],
+ GFP_KERNEL);
+ if (!rx->page[i]) {
+ retval = -ENOMEM;
+ goto failed;
+ }
+ for (j = 0; j < TSNEP_RING_ENTRIES_PER_PAGE; j++) {
+ entry = &rx->entry[TSNEP_RING_ENTRIES_PER_PAGE * i + j];
+ entry->desc_wb = (struct tsnep_rx_desc_wb *)
+ (((u8 *)rx->page[i]) + TSNEP_DESC_SIZE * j);
+ entry->desc = (struct tsnep_rx_desc *)
+ (((u8 *)entry->desc_wb) + TSNEP_DESC_OFFSET);
+ entry->desc_dma = rx->page_dma[i] + TSNEP_DESC_SIZE * j;
+ }
+ }
+ for (i = 0; i < TSNEP_RING_SIZE; i++) {
+ entry = &rx->entry[i];
+ next_entry = &rx->entry[(i + 1) % TSNEP_RING_SIZE];
+ entry->desc->next = __cpu_to_le64(next_entry->desc_dma);
+
+ retval = tsnep_rx_alloc_and_map_skb(rx, entry);
+ if (retval)
+ goto failed;
+ }
+
+ return 0;
+
+failed:
+ tsnep_rx_ring_cleanup(rx);
+ return retval;
+}
+
+static void tsnep_rx_activate(struct tsnep_rx *rx, int index)
+{
+ struct tsnep_rx_entry *entry = &rx->entry[index];
+
+ /* RX_SKB_LENGTH is a multiple of 4 */
+ entry->properties = entry->len & TSNEP_DESC_LENGTH_MASK;
+ entry->properties |= TSNEP_DESC_INTERRUPT_FLAG;
+ if (index == rx->increment_owner_counter) {
+ rx->owner_counter++;
+ if (rx->owner_counter == 4)
+ rx->owner_counter = 1;
+ rx->increment_owner_counter--;
+ if (rx->increment_owner_counter < 0)
+ rx->increment_owner_counter = TSNEP_RING_SIZE - 1;
+ }
+ entry->properties |=
+ (rx->owner_counter << TSNEP_DESC_OWNER_COUNTER_SHIFT) &
+ TSNEP_DESC_OWNER_COUNTER_MASK;
+
+ /* descriptor properties shall be written last, because valid data is
+ * signaled there
+ */
+ dma_wmb();
+
+ entry->desc->properties = __cpu_to_le32(entry->properties);
+}
+
+static int tsnep_rx_poll(struct tsnep_rx *rx, struct napi_struct *napi,
+ int budget)
+{
+ struct device *dmadev = rx->adapter->dmadev;
+ int done = 0;
+ struct tsnep_rx_entry *entry;
+ struct sk_buff *skb;
+ size_t len;
+ dma_addr_t dma;
+ int length;
+ bool enable = false;
+ int retval;
+
+ while (likely(done < budget)) {
+ entry = &rx->entry[rx->read];
+ if ((__le32_to_cpu(entry->desc_wb->properties) &
+ TSNEP_DESC_OWNER_COUNTER_MASK) !=
+ (entry->properties & TSNEP_DESC_OWNER_COUNTER_MASK))
+ break;
+
+ /* descriptor properties shall be read first, because valid data
+ * is signaled there
+ */
+ dma_rmb();
+
+ skb = entry->skb;
+ len = dma_unmap_len(entry, len);
+ dma = dma_unmap_addr(entry, dma);
+
+ /* forward skb only if allocation is successful, otherwise
+ * skb is reused and frame dropped
+ */
+ retval = tsnep_rx_alloc_and_map_skb(rx, entry);
+ if (!retval) {
+ dma_unmap_single(dmadev, dma, len, DMA_FROM_DEVICE);
+
+ length = __le32_to_cpu(entry->desc_wb->properties) &
+ TSNEP_DESC_LENGTH_MASK;
+ skb_put(skb, length - ETH_FCS_LEN);
+ if (rx->adapter->hwtstamp_config.rx_filter ==
+ HWTSTAMP_FILTER_ALL) {
+ struct skb_shared_hwtstamps *hwtstamps =
+ skb_hwtstamps(skb);
+ struct tsnep_rx_inline *rx_inline =
+ (struct tsnep_rx_inline *)skb->data;
+ u64 timestamp =
+ __le64_to_cpu(rx_inline->timestamp);
+
+ memset(hwtstamps, 0, sizeof(*hwtstamps));
+ hwtstamps->hwtstamp = ns_to_ktime(timestamp);
+ }
+ skb_pull(skb, TSNEP_RX_INLINE_METADATA_SIZE);
+ skb->protocol = eth_type_trans(skb,
+ rx->adapter->netdev);
+
+ rx->packets++;
+ rx->bytes += length - TSNEP_RX_INLINE_METADATA_SIZE;
+ if (skb->pkt_type == PACKET_MULTICAST)
+ rx->multicast++;
+
+ napi_gro_receive(napi, skb);
+ done++;
+ } else {
+ rx->dropped++;
+ }
+
+ tsnep_rx_activate(rx, rx->read);
+
+ enable = true;
+
+ rx->read = (rx->read + 1) % TSNEP_RING_SIZE;
+ }
+
+ if (enable) {
+ /* descriptor properties shall be valid before hardware is
+ * notified
+ */
+ dma_wmb();
+
+ iowrite32(TSNEP_CONTROL_RX_ENABLE, rx->addr + TSNEP_CONTROL);
+ }
+
+ return done;
+}
+
+static int tsnep_rx_open(struct tsnep_adapter *adapter, void __iomem *addr,
+ struct tsnep_rx *rx)
+{
+ dma_addr_t dma;
+ int i;
+ int retval;
+
+ memset(rx, 0, sizeof(*rx));
+ rx->adapter = adapter;
+ rx->addr = addr;
+
+ retval = tsnep_rx_ring_init(rx);
+ if (retval)
+ return retval;
+
+ dma = rx->entry[0].desc_dma | TSNEP_RESET_OWNER_COUNTER;
+ iowrite32(DMA_ADDR_LOW(dma), rx->addr + TSNEP_RX_DESC_ADDR_LOW);
+ iowrite32(DMA_ADDR_HIGH(dma), rx->addr + TSNEP_RX_DESC_ADDR_HIGH);
+ rx->owner_counter = 1;
+ rx->increment_owner_counter = TSNEP_RING_SIZE - 1;
+
+ for (i = 0; i < TSNEP_RING_SIZE; i++)
+ tsnep_rx_activate(rx, i);
+
+ /* descriptor properties shall be valid before hardware is notified */
+ dma_wmb();
+
+ iowrite32(TSNEP_CONTROL_RX_ENABLE, rx->addr + TSNEP_CONTROL);
+
+ return 0;
+}
+
+static void tsnep_rx_close(struct tsnep_rx *rx)
+{
+ u32 val;
+
+ iowrite32(TSNEP_CONTROL_RX_DISABLE, rx->addr + TSNEP_CONTROL);
+ readx_poll_timeout(ioread32, rx->addr + TSNEP_CONTROL, val,
+ ((val & TSNEP_CONTROL_RX_ENABLE) == 0), 10000,
+ 1000000);
+
+ tsnep_rx_ring_cleanup(rx);
+}
+
+static int tsnep_poll(struct napi_struct *napi, int budget)
+{
+ struct tsnep_queue *queue = container_of(napi, struct tsnep_queue,
+ napi);
+ bool complete = true;
+ int done = 0;
+
+ if (queue->tx)
+ complete = tsnep_tx_poll(queue->tx, budget);
+
+ if (queue->rx) {
+ done = tsnep_rx_poll(queue->rx, napi, budget);
+ if (done >= budget)
+ complete = false;
+ }
+
+ /* if all work not completed, return budget and keep polling */
+ if (!complete)
+ return budget;
+
+ if (likely(napi_complete_done(napi, done)))
+ tsnep_enable_irq(queue->adapter, queue->irq_mask);
+
+ return min(done, budget - 1);
+}
+
+static int tsnep_netdev_open(struct net_device *netdev)
+{
+ struct tsnep_adapter *adapter = netdev_priv(netdev);
+ int i;
+ void __iomem *addr;
+ int tx_queue_index = 0;
+ int rx_queue_index = 0;
+ int retval;
+
+ retval = tsnep_phy_open(adapter);
+ if (retval)
+ return retval;
+
+ for (i = 0; i < adapter->num_queues; i++) {
+ adapter->queue[i].adapter = adapter;
+ if (adapter->queue[i].tx) {
+ addr = adapter->addr + TSNEP_QUEUE(tx_queue_index);
+ retval = tsnep_tx_open(adapter, addr,
+ adapter->queue[i].tx);
+ if (retval)
+ goto failed;
+ tx_queue_index++;
+ }
+ if (adapter->queue[i].rx) {
+ addr = adapter->addr + TSNEP_QUEUE(rx_queue_index);
+ retval = tsnep_rx_open(adapter, addr,
+ adapter->queue[i].rx);
+ if (retval)
+ goto failed;
+ rx_queue_index++;
+ }
+ }
+
+ retval = netif_set_real_num_tx_queues(adapter->netdev,
+ adapter->num_tx_queues);
+ if (retval)
+ goto failed;
+ retval = netif_set_real_num_rx_queues(adapter->netdev,
+ adapter->num_rx_queues);
+ if (retval)
+ goto failed;
+
+ for (i = 0; i < adapter->num_queues; i++) {
+ netif_napi_add(adapter->netdev, &adapter->queue[i].napi,
+ tsnep_poll, 64);
+ napi_enable(&adapter->queue[i].napi);
+
+ tsnep_enable_irq(adapter, adapter->queue[i].irq_mask);
+ }
+
+ return 0;
+
+failed:
+ for (i = 0; i < adapter->num_queues; i++) {
+ if (adapter->queue[i].rx)
+ tsnep_rx_close(adapter->queue[i].rx);
+ if (adapter->queue[i].tx)
+ tsnep_tx_close(adapter->queue[i].tx);
+ }
+ tsnep_phy_close(adapter);
+ return retval;
+}
+
+static int tsnep_netdev_close(struct net_device *netdev)
+{
+ struct tsnep_adapter *adapter = netdev_priv(netdev);
+ int i;
+
+ for (i = 0; i < adapter->num_queues; i++) {
+ tsnep_disable_irq(adapter, adapter->queue[i].irq_mask);
+
+ napi_disable(&adapter->queue[i].napi);
+ netif_napi_del(&adapter->queue[i].napi);
+
+ if (adapter->queue[i].rx)
+ tsnep_rx_close(adapter->queue[i].rx);
+ if (adapter->queue[i].tx)
+ tsnep_tx_close(adapter->queue[i].tx);
+ }
+
+ tsnep_phy_close(adapter);
+
+ return 0;
+}
+
+static netdev_tx_t tsnep_netdev_xmit_frame(struct sk_buff *skb,
+ struct net_device *netdev)
+{
+ struct tsnep_adapter *adapter = netdev_priv(netdev);
+ u16 queue_mapping = skb_get_queue_mapping(skb);
+
+ if (queue_mapping >= adapter->num_tx_queues)
+ queue_mapping = 0;
+
+ return tsnep_xmit_frame_ring(skb, &adapter->tx[queue_mapping]);
+}
+
+static int tsnep_netdev_ioctl(struct net_device *netdev, struct ifreq *ifr,
+ int cmd)
+{
+ if (!netif_running(netdev))
+ return -EINVAL;
+ if (cmd == SIOCSHWTSTAMP || cmd == SIOCGHWTSTAMP)
+ return tsnep_ptp_ioctl(netdev, ifr, cmd);
+ return phy_mii_ioctl(netdev->phydev, ifr, cmd);
+}
+
+static void tsnep_netdev_set_multicast(struct net_device *netdev)
+{
+ struct tsnep_adapter *adapter = netdev_priv(netdev);
+
+ u16 rx_filter = 0;
+
+ /* configured MAC address and broadcasts are never filtered */
+ if (netdev->flags & IFF_PROMISC) {
+ rx_filter |= TSNEP_RX_FILTER_ACCEPT_ALL_MULTICASTS;
+ rx_filter |= TSNEP_RX_FILTER_ACCEPT_ALL_UNICASTS;
+ } else if (!netdev_mc_empty(netdev) || (netdev->flags & IFF_ALLMULTI)) {
+ rx_filter |= TSNEP_RX_FILTER_ACCEPT_ALL_MULTICASTS;
+ }
+ iowrite16(rx_filter, adapter->addr + TSNEP_RX_FILTER);
+}
+
+static void tsnep_netdev_get_stats64(struct net_device *netdev,
+ struct rtnl_link_stats64 *stats)
+{
+ struct tsnep_adapter *adapter = netdev_priv(netdev);
+ u32 reg;
+ u32 val;
+ int i;
+
+ for (i = 0; i < adapter->num_tx_queues; i++) {
+ stats->tx_packets += adapter->tx[i].packets;
+ stats->tx_bytes += adapter->tx[i].bytes;
+ stats->tx_dropped += adapter->tx[i].dropped;
+ }
+ for (i = 0; i < adapter->num_rx_queues; i++) {
+ stats->rx_packets += adapter->rx[i].packets;
+ stats->rx_bytes += adapter->rx[i].bytes;
+ stats->rx_dropped += adapter->rx[i].dropped;
+ stats->multicast += adapter->rx[i].multicast;
+
+ reg = ioread32(adapter->addr + TSNEP_QUEUE(i) +
+ TSNEP_RX_STATISTIC);
+ val = (reg & TSNEP_RX_STATISTIC_NO_DESC_MASK) >>
+ TSNEP_RX_STATISTIC_NO_DESC_SHIFT;
+ stats->rx_dropped += val;
+ val = (reg & TSNEP_RX_STATISTIC_BUFFER_TOO_SMALL_MASK) >>
+ TSNEP_RX_STATISTIC_BUFFER_TOO_SMALL_SHIFT;
+ stats->rx_dropped += val;
+ val = (reg & TSNEP_RX_STATISTIC_FIFO_OVERFLOW_MASK) >>
+ TSNEP_RX_STATISTIC_FIFO_OVERFLOW_SHIFT;
+ stats->rx_errors += val;
+ stats->rx_fifo_errors += val;
+ val = (reg & TSNEP_RX_STATISTIC_INVALID_FRAME_MASK) >>
+ TSNEP_RX_STATISTIC_INVALID_FRAME_SHIFT;
+ stats->rx_errors += val;
+ stats->rx_frame_errors += val;
+ }
+
+ reg = ioread32(adapter->addr + ECM_STAT);
+ val = (reg & ECM_STAT_RX_ERR_MASK) >> ECM_STAT_RX_ERR_SHIFT;
+ stats->rx_errors += val;
+ val = (reg & ECM_STAT_INV_FRM_MASK) >> ECM_STAT_INV_FRM_SHIFT;
+ stats->rx_errors += val;
+ stats->rx_crc_errors += val;
+ val = (reg & ECM_STAT_FWD_RX_ERR_MASK) >> ECM_STAT_FWD_RX_ERR_SHIFT;
+ stats->rx_errors += val;
+}
+
+static void tsnep_mac_set_address(struct tsnep_adapter *adapter, u8 *addr)
+{
+ iowrite32(*(u32 *)addr, adapter->addr + TSNEP_MAC_ADDRESS_LOW);
+ iowrite16(*(u16 *)(addr + sizeof(u32)),
+ adapter->addr + TSNEP_MAC_ADDRESS_HIGH);
+
+ ether_addr_copy(adapter->mac_address, addr);
+ netif_info(adapter, drv, adapter->netdev, "MAC address set to %pM\n",
+ addr);
+}
+
+static int tsnep_netdev_set_mac_address(struct net_device *netdev, void *addr)
+{
+ struct tsnep_adapter *adapter = netdev_priv(netdev);
+ struct sockaddr *sock_addr = addr;
+ int retval;
+
+ retval = eth_prepare_mac_addr_change(netdev, sock_addr);
+ if (retval)
+ return retval;
+ eth_hw_addr_set(netdev, sock_addr->sa_data);
+ tsnep_mac_set_address(adapter, sock_addr->sa_data);
+
+ return 0;
+}
+
+static const struct net_device_ops tsnep_netdev_ops = {
+ .ndo_open = tsnep_netdev_open,
+ .ndo_stop = tsnep_netdev_close,
+ .ndo_start_xmit = tsnep_netdev_xmit_frame,
+ .ndo_eth_ioctl = tsnep_netdev_ioctl,
+ .ndo_set_rx_mode = tsnep_netdev_set_multicast,
+
+ .ndo_get_stats64 = tsnep_netdev_get_stats64,
+ .ndo_set_mac_address = tsnep_netdev_set_mac_address,
+ .ndo_setup_tc = tsnep_tc_setup,
+};
+
+static int tsnep_mac_init(struct tsnep_adapter *adapter)
+{
+ int retval;
+
+ /* initialize RX filtering, at least configured MAC address and
+ * broadcast are not filtered
+ */
+ iowrite16(0, adapter->addr + TSNEP_RX_FILTER);
+
+ /* try to get MAC address in the following order:
+ * - device tree
+ * - valid MAC address already set
+ * - MAC address register if valid
+ * - random MAC address
+ */
+ retval = of_get_mac_address(adapter->pdev->dev.of_node,
+ adapter->mac_address);
+ if (retval == -EPROBE_DEFER)
+ return retval;
+ if (retval && !is_valid_ether_addr(adapter->mac_address)) {
+ *(u32 *)adapter->mac_address =
+ ioread32(adapter->addr + TSNEP_MAC_ADDRESS_LOW);
+ *(u16 *)(adapter->mac_address + sizeof(u32)) =
+ ioread16(adapter->addr + TSNEP_MAC_ADDRESS_HIGH);
+ if (!is_valid_ether_addr(adapter->mac_address))
+ eth_random_addr(adapter->mac_address);
+ }
+
+ tsnep_mac_set_address(adapter, adapter->mac_address);
+ eth_hw_addr_set(adapter->netdev, adapter->mac_address);
+
+ return 0;
+}
+
+static int tsnep_mdio_init(struct tsnep_adapter *adapter)
+{
+ struct device_node *np = adapter->pdev->dev.of_node;
+ int retval;
+
+ if (np) {
+ np = of_get_child_by_name(np, "mdio");
+ if (!np)
+ return 0;
+
+ adapter->suppress_preamble =
+ of_property_read_bool(np, "suppress-preamble");
+ }
+
+ adapter->mdiobus = devm_mdiobus_alloc(&adapter->pdev->dev);
+ if (!adapter->mdiobus) {
+ retval = -ENOMEM;
+
+ goto out;
+ }
+
+ adapter->mdiobus->priv = (void *)adapter;
+ adapter->mdiobus->parent = &adapter->pdev->dev;
+ adapter->mdiobus->read = tsnep_mdiobus_read;
+ adapter->mdiobus->write = tsnep_mdiobus_write;
+ adapter->mdiobus->name = TSNEP "-mdiobus";
+ snprintf(adapter->mdiobus->id, MII_BUS_ID_SIZE, "%s",
+ adapter->pdev->name);
+
+ /* do not scan broadcast address */
+ adapter->mdiobus->phy_mask = 0x0000001;
+
+ retval = of_mdiobus_register(adapter->mdiobus, np);
+
+out:
+ if (np)
+ of_node_put(np);
+
+ return retval;
+}
+
+static int tsnep_phy_init(struct tsnep_adapter *adapter)
+{
+ struct device_node *phy_node;
+ int retval;
+
+ retval = of_get_phy_mode(adapter->pdev->dev.of_node,
+ &adapter->phy_mode);
+ if (retval)
+ adapter->phy_mode = PHY_INTERFACE_MODE_GMII;
+
+ phy_node = of_parse_phandle(adapter->pdev->dev.of_node, "phy-handle",
+ 0);
+ adapter->phydev = of_phy_find_device(phy_node);
+ of_node_put(phy_node);
+ if (!adapter->phydev && adapter->mdiobus)
+ adapter->phydev = phy_find_first(adapter->mdiobus);
+ if (!adapter->phydev)
+ return -EIO;
+
+ return 0;
+}
+
+static int tsnep_probe(struct platform_device *pdev)
+{
+ struct tsnep_adapter *adapter;
+ struct net_device *netdev;
+ struct resource *io;
+ u32 type;
+ int revision;
+ int version;
+ int retval;
+
+ netdev = devm_alloc_etherdev_mqs(&pdev->dev,
+ sizeof(struct tsnep_adapter),
+ TSNEP_MAX_QUEUES, TSNEP_MAX_QUEUES);
+ if (!netdev)
+ return -ENODEV;
+ SET_NETDEV_DEV(netdev, &pdev->dev);
+ adapter = netdev_priv(netdev);
+ platform_set_drvdata(pdev, adapter);
+ adapter->pdev = pdev;
+ adapter->dmadev = &pdev->dev;
+ adapter->netdev = netdev;
+ adapter->msg_enable = NETIF_MSG_DRV | NETIF_MSG_PROBE |
+ NETIF_MSG_LINK | NETIF_MSG_IFUP |
+ NETIF_MSG_IFDOWN | NETIF_MSG_TX_QUEUED;
+
+ netdev->min_mtu = ETH_MIN_MTU;
+ netdev->max_mtu = TSNEP_MAX_FRAME_SIZE;
+
+ mutex_init(&adapter->gate_control_lock);
+
+ io = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ adapter->addr = devm_ioremap_resource(&pdev->dev, io);
+ if (IS_ERR(adapter->addr))
+ return PTR_ERR(adapter->addr);
+ adapter->irq = platform_get_irq(pdev, 0);
+ netdev->mem_start = io->start;
+ netdev->mem_end = io->end;
+ netdev->irq = adapter->irq;
+
+ type = ioread32(adapter->addr + ECM_TYPE);
+ revision = (type & ECM_REVISION_MASK) >> ECM_REVISION_SHIFT;
+ version = (type & ECM_VERSION_MASK) >> ECM_VERSION_SHIFT;
+ adapter->gate_control = type & ECM_GATE_CONTROL;
+
+ adapter->num_tx_queues = TSNEP_QUEUES;
+ adapter->num_rx_queues = TSNEP_QUEUES;
+ adapter->num_queues = TSNEP_QUEUES;
+ adapter->queue[0].tx = &adapter->tx[0];
+ adapter->queue[0].rx = &adapter->rx[0];
+ adapter->queue[0].irq_mask = ECM_INT_TX_0 | ECM_INT_RX_0;
+
+ tsnep_disable_irq(adapter, ECM_INT_ALL);
+ retval = devm_request_irq(&adapter->pdev->dev, adapter->irq, tsnep_irq,
+ 0, TSNEP, adapter);
+ if (retval != 0) {
+ dev_err(&adapter->pdev->dev, "can't get assigned irq %d.\n",
+ adapter->irq);
+ return retval;
+ }
+ tsnep_enable_irq(adapter, ECM_INT_LINK);
+
+ retval = tsnep_mac_init(adapter);
+ if (retval)
+ goto mac_init_failed;
+
+ retval = tsnep_mdio_init(adapter);
+ if (retval)
+ goto mdio_init_failed;
+
+ retval = tsnep_phy_init(adapter);
+ if (retval)
+ goto phy_init_failed;
+
+ retval = tsnep_ptp_init(adapter);
+ if (retval)
+ goto ptp_init_failed;
+
+ retval = tsnep_tc_init(adapter);
+ if (retval)
+ goto tc_init_failed;
+
+ netdev->netdev_ops = &tsnep_netdev_ops;
+ netdev->ethtool_ops = &tsnep_ethtool_ops;
+ netdev->features = NETIF_F_SG;
+ netdev->hw_features = netdev->features;
+
+ /* carrier off reporting is important to ethtool even BEFORE open */
+ netif_carrier_off(netdev);
+
+ retval = register_netdev(netdev);
+ if (retval)
+ goto register_failed;
+
+ dev_info(&adapter->pdev->dev, "device version %d.%02d\n", version,
+ revision);
+ if (adapter->gate_control)
+ dev_info(&adapter->pdev->dev, "gate control detected\n");
+
+ return 0;
+
+register_failed:
+ tsnep_tc_cleanup(adapter);
+tc_init_failed:
+ tsnep_ptp_cleanup(adapter);
+ptp_init_failed:
+phy_init_failed:
+ if (adapter->mdiobus)
+ mdiobus_unregister(adapter->mdiobus);
+mdio_init_failed:
+mac_init_failed:
+ tsnep_disable_irq(adapter, ECM_INT_ALL);
+ return retval;
+}
+
+static int tsnep_remove(struct platform_device *pdev)
+{
+ struct tsnep_adapter *adapter = platform_get_drvdata(pdev);
+
+ unregister_netdev(adapter->netdev);
+
+ tsnep_tc_cleanup(adapter);
+
+ tsnep_ptp_cleanup(adapter);
+
+ if (adapter->mdiobus)
+ mdiobus_unregister(adapter->mdiobus);
+
+ tsnep_disable_irq(adapter, ECM_INT_ALL);
+
+ return 0;
+}
+
+static const struct of_device_id tsnep_of_match[] = {
+ { .compatible = "engleder,tsnep", },
+{ },
+};
+MODULE_DEVICE_TABLE(of, tsnep_of_match);
+
+static struct platform_driver tsnep_driver = {
+ .driver = {
+ .name = TSNEP,
+ .of_match_table = of_match_ptr(tsnep_of_match),
+ },
+ .probe = tsnep_probe,
+ .remove = tsnep_remove,
+};
+module_platform_driver(tsnep_driver);
+
+MODULE_AUTHOR("Gerhard Engleder <gerhard@engleder-embedded.com>");
+MODULE_DESCRIPTION("TSN endpoint Ethernet MAC driver");
+MODULE_LICENSE("GPL");
diff --git a/drivers/net/ethernet/engleder/tsnep_ptp.c b/drivers/net/ethernet/engleder/tsnep_ptp.c
new file mode 100644
index 0000000..eaad453
--- /dev/null
+++ b/drivers/net/ethernet/engleder/tsnep_ptp.c
@@ -0,0 +1,218 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2021 Gerhard Engleder <gerhard@engleder-embedded.com> */
+
+#include "tsnep.h"
+
+void tsnep_get_system_time(struct tsnep_adapter *adapter, u64 *time)
+{
+ u32 high_before;
+ u32 low;
+ u32 high;
+
+ /* read high dword twice to detect overrun */
+ high = ioread32(adapter->addr + ECM_SYSTEM_TIME_HIGH);
+ do {
+ low = ioread32(adapter->addr + ECM_SYSTEM_TIME_LOW);
+ high_before = high;
+ high = ioread32(adapter->addr + ECM_SYSTEM_TIME_HIGH);
+ } while (high != high_before);
+ *time = (((u64)high) << 32) | ((u64)low);
+}
+
+int tsnep_ptp_ioctl(struct net_device *netdev, struct ifreq *ifr, int cmd)
+{
+ struct tsnep_adapter *adapter = netdev_priv(netdev);
+ struct hwtstamp_config config;
+
+ if (!ifr)
+ return -EINVAL;
+
+ if (cmd == SIOCSHWTSTAMP) {
+ if (copy_from_user(&config, ifr->ifr_data, sizeof(config)))
+ return -EFAULT;
+
+ switch (config.tx_type) {
+ case HWTSTAMP_TX_OFF:
+ case HWTSTAMP_TX_ON:
+ break;
+ default:
+ return -ERANGE;
+ }
+
+ switch (config.rx_filter) {
+ case HWTSTAMP_FILTER_NONE:
+ break;
+ case HWTSTAMP_FILTER_ALL:
+ case HWTSTAMP_FILTER_PTP_V1_L4_EVENT:
+ case HWTSTAMP_FILTER_PTP_V1_L4_SYNC:
+ case HWTSTAMP_FILTER_PTP_V1_L4_DELAY_REQ:
+ case HWTSTAMP_FILTER_PTP_V2_L4_EVENT:
+ case HWTSTAMP_FILTER_PTP_V2_L4_SYNC:
+ case HWTSTAMP_FILTER_PTP_V2_L4_DELAY_REQ:
+ case HWTSTAMP_FILTER_PTP_V2_L2_EVENT:
+ case HWTSTAMP_FILTER_PTP_V2_L2_SYNC:
+ case HWTSTAMP_FILTER_PTP_V2_L2_DELAY_REQ:
+ case HWTSTAMP_FILTER_PTP_V2_EVENT:
+ case HWTSTAMP_FILTER_PTP_V2_SYNC:
+ case HWTSTAMP_FILTER_PTP_V2_DELAY_REQ:
+ case HWTSTAMP_FILTER_NTP_ALL:
+ config.rx_filter = HWTSTAMP_FILTER_ALL;
+ break;
+ default:
+ return -ERANGE;
+ }
+
+ memcpy(&adapter->hwtstamp_config, &config,
+ sizeof(adapter->hwtstamp_config));
+ }
+
+ if (copy_to_user(ifr->ifr_data, &adapter->hwtstamp_config,
+ sizeof(adapter->hwtstamp_config)))
+ return -EFAULT;
+
+ return 0;
+}
+
+static int tsnep_ptp_adjfine(struct ptp_clock_info *ptp, long scaled_ppm)
+{
+ struct tsnep_adapter *adapter = container_of(ptp, struct tsnep_adapter,
+ ptp_clock_info);
+ bool negative = false;
+ u64 rate_offset;
+
+ if (scaled_ppm < 0) {
+ scaled_ppm = -scaled_ppm;
+ negative = true;
+ }
+
+ /* convert from 16 bit to 32 bit binary fractional, divide by 1000000 to
+ * eliminate ppm, multiply with 8 to compensate 8ns clock cycle time,
+ * simplify calculation because 15625 * 8 = 1000000 / 8
+ */
+ rate_offset = scaled_ppm;
+ rate_offset <<= 16 - 3;
+ rate_offset = div_u64(rate_offset, 15625);
+
+ rate_offset &= ECM_CLOCK_RATE_OFFSET_MASK;
+ if (negative)
+ rate_offset |= ECM_CLOCK_RATE_OFFSET_SIGN;
+ iowrite32(rate_offset & 0xFFFFFFFF, adapter->addr + ECM_CLOCK_RATE);
+
+ return 0;
+}
+
+static int tsnep_ptp_adjtime(struct ptp_clock_info *ptp, s64 delta)
+{
+ struct tsnep_adapter *adapter = container_of(ptp, struct tsnep_adapter,
+ ptp_clock_info);
+ u64 system_time;
+ unsigned long flags;
+
+ spin_lock_irqsave(&adapter->ptp_lock, flags);
+
+ tsnep_get_system_time(adapter, &system_time);
+
+ system_time += delta;
+
+ /* high dword is buffered in hardware and synchronously written to
+ * system time when low dword is written
+ */
+ iowrite32(system_time >> 32, adapter->addr + ECM_SYSTEM_TIME_HIGH);
+ iowrite32(system_time & 0xFFFFFFFF,
+ adapter->addr + ECM_SYSTEM_TIME_LOW);
+
+ spin_unlock_irqrestore(&adapter->ptp_lock, flags);
+
+ return 0;
+}
+
+static int tsnep_ptp_gettimex64(struct ptp_clock_info *ptp,
+ struct timespec64 *ts,
+ struct ptp_system_timestamp *sts)
+{
+ struct tsnep_adapter *adapter = container_of(ptp, struct tsnep_adapter,
+ ptp_clock_info);
+ u32 high_before;
+ u32 low;
+ u32 high;
+ u64 system_time;
+
+ /* read high dword twice to detect overrun */
+ high = ioread32(adapter->addr + ECM_SYSTEM_TIME_HIGH);
+ do {
+ ptp_read_system_prets(sts);
+ low = ioread32(adapter->addr + ECM_SYSTEM_TIME_LOW);
+ ptp_read_system_postts(sts);
+ high_before = high;
+ high = ioread32(adapter->addr + ECM_SYSTEM_TIME_HIGH);
+ } while (high != high_before);
+ system_time = (((u64)high) << 32) | ((u64)low);
+
+ *ts = ns_to_timespec64(system_time);
+
+ return 0;
+}
+
+static int tsnep_ptp_settime64(struct ptp_clock_info *ptp,
+ const struct timespec64 *ts)
+{
+ struct tsnep_adapter *adapter = container_of(ptp, struct tsnep_adapter,
+ ptp_clock_info);
+ u64 system_time = timespec64_to_ns(ts);
+ unsigned long flags;
+
+ spin_lock_irqsave(&adapter->ptp_lock, flags);
+
+ /* high dword is buffered in hardware and synchronously written to
+ * system time when low dword is written
+ */
+ iowrite32(system_time >> 32, adapter->addr + ECM_SYSTEM_TIME_HIGH);
+ iowrite32(system_time & 0xFFFFFFFF,
+ adapter->addr + ECM_SYSTEM_TIME_LOW);
+
+ spin_unlock_irqrestore(&adapter->ptp_lock, flags);
+
+ return 0;
+}
+
+int tsnep_ptp_init(struct tsnep_adapter *adapter)
+{
+ int retval = 0;
+
+ adapter->hwtstamp_config.rx_filter = HWTSTAMP_FILTER_NONE;
+ adapter->hwtstamp_config.tx_type = HWTSTAMP_TX_OFF;
+
+ snprintf(adapter->ptp_clock_info.name, 16, "%s", TSNEP);
+ adapter->ptp_clock_info.owner = THIS_MODULE;
+ /* at most 2^-1ns adjustment every clock cycle for 8ns clock cycle time,
+ * stay slightly below because only bits below 2^-1ns are supported
+ */
+ adapter->ptp_clock_info.max_adj = (500000000 / 8 - 1);
+ adapter->ptp_clock_info.adjfine = tsnep_ptp_adjfine;
+ adapter->ptp_clock_info.adjtime = tsnep_ptp_adjtime;
+ adapter->ptp_clock_info.gettimex64 = tsnep_ptp_gettimex64;
+ adapter->ptp_clock_info.settime64 = tsnep_ptp_settime64;
+
+ spin_lock_init(&adapter->ptp_lock);
+
+ adapter->ptp_clock = ptp_clock_register(&adapter->ptp_clock_info,
+ &adapter->pdev->dev);
+ if (IS_ERR(adapter->ptp_clock)) {
+ netdev_err(adapter->netdev, "ptp_clock_register failed\n");
+
+ retval = PTR_ERR(adapter->ptp_clock);
+ adapter->ptp_clock = NULL;
+ } else if (adapter->ptp_clock) {
+ netdev_info(adapter->netdev, "PHC added\n");
+ }
+
+ return retval;
+}
+
+void tsnep_ptp_cleanup(struct tsnep_adapter *adapter)
+{
+ if (adapter->ptp_clock) {
+ ptp_clock_unregister(adapter->ptp_clock);
+ netdev_info(adapter->netdev, "PHC removed\n");
+ }
+}
diff --git a/drivers/net/ethernet/engleder/tsnep_selftests.c b/drivers/net/ethernet/engleder/tsnep_selftests.c
new file mode 100644
index 0000000..1581d6b
--- /dev/null
+++ b/drivers/net/ethernet/engleder/tsnep_selftests.c
@@ -0,0 +1,811 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2021 Gerhard Engleder <gerhard@engleder-embedded.com> */
+
+#include "tsnep.h"
+
+#include <net/pkt_sched.h>
+
+enum tsnep_test {
+ TSNEP_TEST_ENABLE = 0,
+ TSNEP_TEST_TAPRIO,
+ TSNEP_TEST_TAPRIO_CHANGE,
+ TSNEP_TEST_TAPRIO_EXTENSION,
+};
+
+static const char tsnep_test_strings[][ETH_GSTRING_LEN] = {
+ "Enable timeout (offline)",
+ "TAPRIO (offline)",
+ "TAPRIO change (offline)",
+ "TAPRIO extension (offline)",
+};
+
+#define TSNEP_TEST_COUNT (sizeof(tsnep_test_strings) / ETH_GSTRING_LEN)
+
+static bool enable_gc_timeout(struct tsnep_adapter *adapter)
+{
+ iowrite8(TSNEP_GC_ENABLE_TIMEOUT, adapter->addr + TSNEP_GC);
+ if (!(ioread32(adapter->addr + TSNEP_GC) & TSNEP_GC_TIMEOUT_ACTIVE))
+ return false;
+
+ return true;
+}
+
+static bool gc_timeout_signaled(struct tsnep_adapter *adapter)
+{
+ if (ioread32(adapter->addr + TSNEP_GC) & TSNEP_GC_TIMEOUT_SIGNAL)
+ return true;
+
+ return false;
+}
+
+static bool ack_gc_timeout(struct tsnep_adapter *adapter)
+{
+ iowrite8(TSNEP_GC_ENABLE_TIMEOUT, adapter->addr + TSNEP_GC);
+ if (ioread32(adapter->addr + TSNEP_GC) &
+ (TSNEP_GC_TIMEOUT_ACTIVE | TSNEP_GC_TIMEOUT_SIGNAL))
+ return false;
+ return true;
+}
+
+static bool enable_gc(struct tsnep_adapter *adapter, bool a)
+{
+ u8 enable;
+ u8 active;
+
+ if (a) {
+ enable = TSNEP_GC_ENABLE_A;
+ active = TSNEP_GC_ACTIVE_A;
+ } else {
+ enable = TSNEP_GC_ENABLE_B;
+ active = TSNEP_GC_ACTIVE_B;
+ }
+
+ iowrite8(enable, adapter->addr + TSNEP_GC);
+ if (!(ioread32(adapter->addr + TSNEP_GC) & active))
+ return false;
+
+ return true;
+}
+
+static bool disable_gc(struct tsnep_adapter *adapter)
+{
+ iowrite8(TSNEP_GC_DISABLE, adapter->addr + TSNEP_GC);
+ if (ioread32(adapter->addr + TSNEP_GC) &
+ (TSNEP_GC_ACTIVE_A | TSNEP_GC_ACTIVE_B))
+ return false;
+
+ return true;
+}
+
+static bool gc_delayed_enable(struct tsnep_adapter *adapter, bool a, int delay)
+{
+ u64 before, after;
+ u32 time;
+ bool enabled;
+
+ if (!disable_gc(adapter))
+ return false;
+
+ before = ktime_get_ns();
+
+ if (!enable_gc_timeout(adapter))
+ return false;
+
+ /* for start time after timeout, the timeout can guarantee, that enable
+ * is blocked if too late
+ */
+ time = ioread32(adapter->addr + ECM_SYSTEM_TIME_LOW);
+ time += TSNEP_GC_TIMEOUT;
+ iowrite32(time, adapter->addr + TSNEP_GC_TIME);
+
+ ndelay(delay);
+
+ enabled = enable_gc(adapter, a);
+ after = ktime_get_ns();
+
+ if (delay > TSNEP_GC_TIMEOUT) {
+ /* timeout must have blocked enable */
+ if (enabled)
+ return false;
+ } else if ((after - before) < TSNEP_GC_TIMEOUT * 14 / 16) {
+ /* timeout must not have blocked enable */
+ if (!enabled)
+ return false;
+ }
+
+ if (enabled) {
+ if (gc_timeout_signaled(adapter))
+ return false;
+ } else {
+ if (!gc_timeout_signaled(adapter))
+ return false;
+ if (!ack_gc_timeout(adapter))
+ return false;
+ }
+
+ if (!disable_gc(adapter))
+ return false;
+
+ return true;
+}
+
+static bool tsnep_test_gc_enable(struct tsnep_adapter *adapter)
+{
+ int i;
+
+ iowrite32(0x80000001, adapter->addr + TSNEP_GCL_A + 0);
+ iowrite32(100000, adapter->addr + TSNEP_GCL_A + 4);
+
+ for (i = 0; i < 200000; i += 100) {
+ if (!gc_delayed_enable(adapter, true, i))
+ return false;
+ }
+
+ iowrite32(0x80000001, adapter->addr + TSNEP_GCL_B + 0);
+ iowrite32(100000, adapter->addr + TSNEP_GCL_B + 4);
+
+ for (i = 0; i < 200000; i += 100) {
+ if (!gc_delayed_enable(adapter, false, i))
+ return false;
+ }
+
+ return true;
+}
+
+static void delay_base_time(struct tsnep_adapter *adapter,
+ struct tc_taprio_qopt_offload *qopt, s64 ms)
+{
+ u64 system_time;
+ u64 base_time = ktime_to_ns(qopt->base_time);
+ u64 n;
+
+ tsnep_get_system_time(adapter, &system_time);
+ system_time += ms * 1000000;
+ n = div64_u64(system_time - base_time, qopt->cycle_time);
+
+ qopt->base_time = ktime_add_ns(qopt->base_time,
+ (n + 1) * qopt->cycle_time);
+}
+
+static void get_gate_state(struct tsnep_adapter *adapter, u32 *gc, u32 *gc_time,
+ u64 *system_time)
+{
+ u32 time_high_before;
+ u32 time_low;
+ u32 time_high;
+ u32 gc_time_before;
+
+ time_high = ioread32(adapter->addr + ECM_SYSTEM_TIME_HIGH);
+ *gc_time = ioread32(adapter->addr + TSNEP_GC_TIME);
+ do {
+ time_low = ioread32(adapter->addr + ECM_SYSTEM_TIME_LOW);
+ *gc = ioread32(adapter->addr + TSNEP_GC);
+
+ gc_time_before = *gc_time;
+ *gc_time = ioread32(adapter->addr + TSNEP_GC_TIME);
+ time_high_before = time_high;
+ time_high = ioread32(adapter->addr + ECM_SYSTEM_TIME_HIGH);
+ } while ((time_high != time_high_before) ||
+ (*gc_time != gc_time_before));
+
+ *system_time = (((u64)time_high) << 32) | ((u64)time_low);
+}
+
+static int get_operation(struct tsnep_gcl *gcl, u64 system_time, u64 *next)
+{
+ u64 n = div64_u64(system_time - gcl->base_time, gcl->cycle_time);
+ u64 cycle_start = gcl->base_time + gcl->cycle_time * n;
+ int i;
+
+ *next = cycle_start;
+ for (i = 0; i < gcl->count; i++) {
+ *next += gcl->operation[i].interval;
+ if (*next > system_time)
+ break;
+ }
+
+ return i;
+}
+
+static bool check_gate(struct tsnep_adapter *adapter)
+{
+ u32 gc_time;
+ u32 gc;
+ u64 system_time;
+ struct tsnep_gcl *curr;
+ struct tsnep_gcl *prev;
+ u64 next_time;
+ u8 gate_open;
+ u8 next_gate_open;
+
+ get_gate_state(adapter, &gc, &gc_time, &system_time);
+
+ if (gc & TSNEP_GC_ACTIVE_A) {
+ curr = &adapter->gcl[0];
+ prev = &adapter->gcl[1];
+ } else if (gc & TSNEP_GC_ACTIVE_B) {
+ curr = &adapter->gcl[1];
+ prev = &adapter->gcl[0];
+ } else {
+ return false;
+ }
+ if (curr->start_time <= system_time) {
+ /* GCL is already active */
+ int index;
+
+ index = get_operation(curr, system_time, &next_time);
+ gate_open = curr->operation[index].properties & TSNEP_GCL_MASK;
+ if (index == curr->count - 1)
+ index = 0;
+ else
+ index++;
+ next_gate_open =
+ curr->operation[index].properties & TSNEP_GCL_MASK;
+ } else if (curr->change) {
+ /* operation of previous GCL is active */
+ int index;
+ u64 start_before;
+ u64 n;
+
+ index = get_operation(prev, system_time, &next_time);
+ next_time = curr->start_time;
+ start_before = prev->base_time;
+ n = div64_u64(curr->start_time - start_before,
+ prev->cycle_time);
+ start_before += n * prev->cycle_time;
+ if (curr->start_time == start_before)
+ start_before -= prev->cycle_time;
+ if (((start_before + prev->cycle_time_extension) >=
+ curr->start_time) &&
+ (curr->start_time - prev->cycle_time_extension <=
+ system_time)) {
+ /* extend */
+ index = prev->count - 1;
+ }
+ gate_open = prev->operation[index].properties & TSNEP_GCL_MASK;
+ next_gate_open =
+ curr->operation[0].properties & TSNEP_GCL_MASK;
+ } else {
+ /* GCL is waiting for start */
+ next_time = curr->start_time;
+ gate_open = 0xFF;
+ next_gate_open = curr->operation[0].properties & TSNEP_GCL_MASK;
+ }
+
+ if (gc_time != (next_time & 0xFFFFFFFF)) {
+ dev_err(&adapter->pdev->dev, "gate control time 0x%x!=0x%llx\n",
+ gc_time, next_time);
+ return false;
+ }
+ if (((gc & TSNEP_GC_OPEN) >> TSNEP_GC_OPEN_SHIFT) != gate_open) {
+ dev_err(&adapter->pdev->dev,
+ "gate control open 0x%02x!=0x%02x\n",
+ ((gc & TSNEP_GC_OPEN) >> TSNEP_GC_OPEN_SHIFT),
+ gate_open);
+ return false;
+ }
+ if (((gc & TSNEP_GC_NEXT_OPEN) >> TSNEP_GC_NEXT_OPEN_SHIFT) !=
+ next_gate_open) {
+ dev_err(&adapter->pdev->dev,
+ "gate control next open 0x%02x!=0x%02x\n",
+ ((gc & TSNEP_GC_NEXT_OPEN) >> TSNEP_GC_NEXT_OPEN_SHIFT),
+ next_gate_open);
+ return false;
+ }
+
+ return true;
+}
+
+static bool check_gate_duration(struct tsnep_adapter *adapter, s64 ms)
+{
+ ktime_t start = ktime_get();
+
+ do {
+ if (!check_gate(adapter))
+ return false;
+ } while (ktime_ms_delta(ktime_get(), start) < ms);
+
+ return true;
+}
+
+static bool enable_check_taprio(struct tsnep_adapter *adapter,
+ struct tc_taprio_qopt_offload *qopt, s64 ms)
+{
+ int retval;
+
+ retval = tsnep_tc_setup(adapter->netdev, TC_SETUP_QDISC_TAPRIO, qopt);
+ if (retval)
+ return false;
+
+ if (!check_gate_duration(adapter, ms))
+ return false;
+
+ return true;
+}
+
+static bool disable_taprio(struct tsnep_adapter *adapter)
+{
+ struct tc_taprio_qopt_offload qopt;
+ int retval;
+
+ memset(&qopt, 0, sizeof(qopt));
+ qopt.enable = 0;
+ retval = tsnep_tc_setup(adapter->netdev, TC_SETUP_QDISC_TAPRIO, &qopt);
+ if (retval)
+ return false;
+
+ return true;
+}
+
+static bool run_taprio(struct tsnep_adapter *adapter,
+ struct tc_taprio_qopt_offload *qopt, s64 ms)
+{
+ if (!enable_check_taprio(adapter, qopt, ms))
+ return false;
+
+ if (!disable_taprio(adapter))
+ return false;
+
+ return true;
+}
+
+static bool tsnep_test_taprio(struct tsnep_adapter *adapter)
+{
+ struct tc_taprio_qopt_offload *qopt;
+ int i;
+
+ qopt = kzalloc(struct_size(qopt, entries, 255), GFP_KERNEL);
+ if (!qopt)
+ return false;
+ for (i = 0; i < 255; i++)
+ qopt->entries[i].command = TC_TAPRIO_CMD_SET_GATES;
+
+ qopt->enable = 1;
+ qopt->base_time = ktime_set(0, 0);
+ qopt->cycle_time = 1500000;
+ qopt->cycle_time_extension = 0;
+ qopt->entries[0].gate_mask = 0x02;
+ qopt->entries[0].interval = 200000;
+ qopt->entries[1].gate_mask = 0x03;
+ qopt->entries[1].interval = 800000;
+ qopt->entries[2].gate_mask = 0x07;
+ qopt->entries[2].interval = 240000;
+ qopt->entries[3].gate_mask = 0x01;
+ qopt->entries[3].interval = 80000;
+ qopt->entries[4].gate_mask = 0x04;
+ qopt->entries[4].interval = 70000;
+ qopt->entries[5].gate_mask = 0x06;
+ qopt->entries[5].interval = 60000;
+ qopt->entries[6].gate_mask = 0x0F;
+ qopt->entries[6].interval = 50000;
+ qopt->num_entries = 7;
+ if (!run_taprio(adapter, qopt, 100))
+ goto failed;
+
+ qopt->enable = 1;
+ qopt->base_time = ktime_set(0, 0);
+ qopt->cycle_time = 411854;
+ qopt->cycle_time_extension = 0;
+ qopt->entries[0].gate_mask = 0x17;
+ qopt->entries[0].interval = 23842;
+ qopt->entries[1].gate_mask = 0x16;
+ qopt->entries[1].interval = 13482;
+ qopt->entries[2].gate_mask = 0x15;
+ qopt->entries[2].interval = 49428;
+ qopt->entries[3].gate_mask = 0x14;
+ qopt->entries[3].interval = 38189;
+ qopt->entries[4].gate_mask = 0x13;
+ qopt->entries[4].interval = 92321;
+ qopt->entries[5].gate_mask = 0x12;
+ qopt->entries[5].interval = 71239;
+ qopt->entries[6].gate_mask = 0x11;
+ qopt->entries[6].interval = 69932;
+ qopt->entries[7].gate_mask = 0x10;
+ qopt->entries[7].interval = 53421;
+ qopt->num_entries = 8;
+ if (!run_taprio(adapter, qopt, 100))
+ goto failed;
+
+ qopt->enable = 1;
+ qopt->base_time = ktime_set(0, 0);
+ delay_base_time(adapter, qopt, 12);
+ qopt->cycle_time = 125000;
+ qopt->cycle_time_extension = 0;
+ qopt->entries[0].gate_mask = 0x27;
+ qopt->entries[0].interval = 15000;
+ qopt->entries[1].gate_mask = 0x26;
+ qopt->entries[1].interval = 15000;
+ qopt->entries[2].gate_mask = 0x25;
+ qopt->entries[2].interval = 12500;
+ qopt->entries[3].gate_mask = 0x24;
+ qopt->entries[3].interval = 17500;
+ qopt->entries[4].gate_mask = 0x23;
+ qopt->entries[4].interval = 10000;
+ qopt->entries[5].gate_mask = 0x22;
+ qopt->entries[5].interval = 11000;
+ qopt->entries[6].gate_mask = 0x21;
+ qopt->entries[6].interval = 9000;
+ qopt->entries[7].gate_mask = 0x20;
+ qopt->entries[7].interval = 10000;
+ qopt->entries[8].gate_mask = 0x20;
+ qopt->entries[8].interval = 12500;
+ qopt->entries[9].gate_mask = 0x20;
+ qopt->entries[9].interval = 12500;
+ qopt->num_entries = 10;
+ if (!run_taprio(adapter, qopt, 100))
+ goto failed;
+
+ kfree(qopt);
+
+ return true;
+
+failed:
+ disable_taprio(adapter);
+ kfree(qopt);
+
+ return false;
+}
+
+static bool tsnep_test_taprio_change(struct tsnep_adapter *adapter)
+{
+ struct tc_taprio_qopt_offload *qopt;
+ int i;
+
+ qopt = kzalloc(struct_size(qopt, entries, 255), GFP_KERNEL);
+ if (!qopt)
+ return false;
+ for (i = 0; i < 255; i++)
+ qopt->entries[i].command = TC_TAPRIO_CMD_SET_GATES;
+
+ qopt->enable = 1;
+ qopt->base_time = ktime_set(0, 0);
+ qopt->cycle_time = 100000;
+ qopt->cycle_time_extension = 0;
+ qopt->entries[0].gate_mask = 0x30;
+ qopt->entries[0].interval = 20000;
+ qopt->entries[1].gate_mask = 0x31;
+ qopt->entries[1].interval = 80000;
+ qopt->num_entries = 2;
+ if (!enable_check_taprio(adapter, qopt, 100))
+ goto failed;
+
+ /* change to identical */
+ if (!enable_check_taprio(adapter, qopt, 100))
+ goto failed;
+ delay_base_time(adapter, qopt, 17);
+ if (!enable_check_taprio(adapter, qopt, 100))
+ goto failed;
+
+ /* change to same cycle time */
+ qopt->base_time = ktime_set(0, 0);
+ qopt->entries[0].gate_mask = 0x42;
+ qopt->entries[1].gate_mask = 0x43;
+ delay_base_time(adapter, qopt, 2);
+ if (!enable_check_taprio(adapter, qopt, 100))
+ goto failed;
+ qopt->base_time = ktime_set(0, 0);
+ qopt->entries[0].gate_mask = 0x54;
+ qopt->entries[0].interval = 33333;
+ qopt->entries[1].gate_mask = 0x55;
+ qopt->entries[1].interval = 66667;
+ delay_base_time(adapter, qopt, 23);
+ if (!enable_check_taprio(adapter, qopt, 100))
+ goto failed;
+ qopt->base_time = ktime_set(0, 0);
+ qopt->entries[0].gate_mask = 0x66;
+ qopt->entries[0].interval = 50000;
+ qopt->entries[1].gate_mask = 0x67;
+ qopt->entries[1].interval = 25000;
+ qopt->entries[2].gate_mask = 0x68;
+ qopt->entries[2].interval = 25000;
+ qopt->num_entries = 3;
+ delay_base_time(adapter, qopt, 11);
+ if (!enable_check_taprio(adapter, qopt, 100))
+ goto failed;
+
+ /* change to multiple of cycle time */
+ qopt->base_time = ktime_set(0, 0);
+ qopt->cycle_time = 200000;
+ qopt->entries[0].gate_mask = 0x79;
+ qopt->entries[0].interval = 50000;
+ qopt->entries[1].gate_mask = 0x7A;
+ qopt->entries[1].interval = 150000;
+ qopt->num_entries = 2;
+ delay_base_time(adapter, qopt, 11);
+ if (!enable_check_taprio(adapter, qopt, 100))
+ goto failed;
+ qopt->base_time = ktime_set(0, 0);
+ qopt->cycle_time = 1000000;
+ qopt->entries[0].gate_mask = 0x7B;
+ qopt->entries[0].interval = 125000;
+ qopt->entries[1].gate_mask = 0x7C;
+ qopt->entries[1].interval = 250000;
+ qopt->entries[2].gate_mask = 0x7D;
+ qopt->entries[2].interval = 375000;
+ qopt->entries[3].gate_mask = 0x7E;
+ qopt->entries[3].interval = 250000;
+ qopt->num_entries = 4;
+ delay_base_time(adapter, qopt, 3);
+ if (!enable_check_taprio(adapter, qopt, 100))
+ goto failed;
+
+ /* change to shorter cycle time */
+ qopt->base_time = ktime_set(0, 0);
+ qopt->cycle_time = 333333;
+ qopt->entries[0].gate_mask = 0x8F;
+ qopt->entries[0].interval = 166666;
+ qopt->entries[1].gate_mask = 0x80;
+ qopt->entries[1].interval = 166667;
+ qopt->num_entries = 2;
+ delay_base_time(adapter, qopt, 11);
+ if (!enable_check_taprio(adapter, qopt, 100))
+ goto failed;
+ qopt->base_time = ktime_set(0, 0);
+ qopt->cycle_time = 62500;
+ qopt->entries[0].gate_mask = 0x81;
+ qopt->entries[0].interval = 31250;
+ qopt->entries[1].gate_mask = 0x82;
+ qopt->entries[1].interval = 15625;
+ qopt->entries[2].gate_mask = 0x83;
+ qopt->entries[2].interval = 15625;
+ qopt->num_entries = 3;
+ delay_base_time(adapter, qopt, 1);
+ if (!enable_check_taprio(adapter, qopt, 100))
+ goto failed;
+
+ /* change to longer cycle time */
+ qopt->base_time = ktime_set(0, 0);
+ qopt->cycle_time = 400000;
+ qopt->entries[0].gate_mask = 0x84;
+ qopt->entries[0].interval = 100000;
+ qopt->entries[1].gate_mask = 0x85;
+ qopt->entries[1].interval = 100000;
+ qopt->entries[2].gate_mask = 0x86;
+ qopt->entries[2].interval = 100000;
+ qopt->entries[3].gate_mask = 0x87;
+ qopt->entries[3].interval = 100000;
+ qopt->num_entries = 4;
+ delay_base_time(adapter, qopt, 7);
+ if (!enable_check_taprio(adapter, qopt, 100))
+ goto failed;
+ qopt->base_time = ktime_set(0, 0);
+ qopt->cycle_time = 1700000;
+ qopt->entries[0].gate_mask = 0x88;
+ qopt->entries[0].interval = 200000;
+ qopt->entries[1].gate_mask = 0x89;
+ qopt->entries[1].interval = 300000;
+ qopt->entries[2].gate_mask = 0x8A;
+ qopt->entries[2].interval = 600000;
+ qopt->entries[3].gate_mask = 0x8B;
+ qopt->entries[3].interval = 100000;
+ qopt->entries[4].gate_mask = 0x8C;
+ qopt->entries[4].interval = 500000;
+ qopt->num_entries = 5;
+ delay_base_time(adapter, qopt, 6);
+ if (!enable_check_taprio(adapter, qopt, 100))
+ goto failed;
+
+ if (!disable_taprio(adapter))
+ goto failed;
+
+ kfree(qopt);
+
+ return true;
+
+failed:
+ disable_taprio(adapter);
+ kfree(qopt);
+
+ return false;
+}
+
+static bool tsnep_test_taprio_extension(struct tsnep_adapter *adapter)
+{
+ struct tc_taprio_qopt_offload *qopt;
+ int i;
+
+ qopt = kzalloc(struct_size(qopt, entries, 255), GFP_KERNEL);
+ if (!qopt)
+ return false;
+ for (i = 0; i < 255; i++)
+ qopt->entries[i].command = TC_TAPRIO_CMD_SET_GATES;
+
+ qopt->enable = 1;
+ qopt->base_time = ktime_set(0, 0);
+ qopt->cycle_time = 100000;
+ qopt->cycle_time_extension = 50000;
+ qopt->entries[0].gate_mask = 0x90;
+ qopt->entries[0].interval = 20000;
+ qopt->entries[1].gate_mask = 0x91;
+ qopt->entries[1].interval = 80000;
+ qopt->num_entries = 2;
+ if (!enable_check_taprio(adapter, qopt, 100))
+ goto failed;
+
+ /* change to different phase */
+ qopt->base_time = ktime_set(0, 50000);
+ qopt->entries[0].gate_mask = 0x92;
+ qopt->entries[0].interval = 33000;
+ qopt->entries[1].gate_mask = 0x93;
+ qopt->entries[1].interval = 67000;
+ qopt->num_entries = 2;
+ delay_base_time(adapter, qopt, 2);
+ if (!enable_check_taprio(adapter, qopt, 100))
+ goto failed;
+
+ /* change to different phase and longer cycle time */
+ qopt->base_time = ktime_set(0, 0);
+ qopt->cycle_time = 1000000;
+ qopt->cycle_time_extension = 700000;
+ qopt->entries[0].gate_mask = 0x94;
+ qopt->entries[0].interval = 400000;
+ qopt->entries[1].gate_mask = 0x95;
+ qopt->entries[1].interval = 600000;
+ qopt->num_entries = 2;
+ delay_base_time(adapter, qopt, 7);
+ if (!enable_check_taprio(adapter, qopt, 100))
+ goto failed;
+ qopt->base_time = ktime_set(0, 700000);
+ qopt->cycle_time = 2000000;
+ qopt->cycle_time_extension = 1900000;
+ qopt->entries[0].gate_mask = 0x96;
+ qopt->entries[0].interval = 400000;
+ qopt->entries[1].gate_mask = 0x97;
+ qopt->entries[1].interval = 1600000;
+ qopt->num_entries = 2;
+ delay_base_time(adapter, qopt, 9);
+ if (!enable_check_taprio(adapter, qopt, 100))
+ goto failed;
+
+ /* change to different phase and shorter cycle time */
+ qopt->base_time = ktime_set(0, 0);
+ qopt->cycle_time = 1500000;
+ qopt->cycle_time_extension = 700000;
+ qopt->entries[0].gate_mask = 0x98;
+ qopt->entries[0].interval = 400000;
+ qopt->entries[1].gate_mask = 0x99;
+ qopt->entries[1].interval = 600000;
+ qopt->entries[2].gate_mask = 0x9A;
+ qopt->entries[2].interval = 500000;
+ qopt->num_entries = 3;
+ delay_base_time(adapter, qopt, 3);
+ if (!enable_check_taprio(adapter, qopt, 100))
+ goto failed;
+ qopt->base_time = ktime_set(0, 100000);
+ qopt->cycle_time = 500000;
+ qopt->cycle_time_extension = 300000;
+ qopt->entries[0].gate_mask = 0x9B;
+ qopt->entries[0].interval = 150000;
+ qopt->entries[1].gate_mask = 0x9C;
+ qopt->entries[1].interval = 350000;
+ qopt->num_entries = 2;
+ delay_base_time(adapter, qopt, 9);
+ if (!enable_check_taprio(adapter, qopt, 100))
+ goto failed;
+
+ /* change to different cycle time */
+ qopt->base_time = ktime_set(0, 0);
+ qopt->cycle_time = 1000000;
+ qopt->cycle_time_extension = 700000;
+ qopt->entries[0].gate_mask = 0xAD;
+ qopt->entries[0].interval = 400000;
+ qopt->entries[1].gate_mask = 0xAE;
+ qopt->entries[1].interval = 300000;
+ qopt->entries[2].gate_mask = 0xAF;
+ qopt->entries[2].interval = 300000;
+ qopt->num_entries = 3;
+ if (!enable_check_taprio(adapter, qopt, 100))
+ goto failed;
+ qopt->base_time = ktime_set(0, 0);
+ qopt->cycle_time = 400000;
+ qopt->cycle_time_extension = 100000;
+ qopt->entries[0].gate_mask = 0xA0;
+ qopt->entries[0].interval = 200000;
+ qopt->entries[1].gate_mask = 0xA1;
+ qopt->entries[1].interval = 200000;
+ qopt->num_entries = 2;
+ delay_base_time(adapter, qopt, 19);
+ if (!enable_check_taprio(adapter, qopt, 100))
+ goto failed;
+ qopt->base_time = ktime_set(0, 0);
+ qopt->cycle_time = 500000;
+ qopt->cycle_time_extension = 499999;
+ qopt->entries[0].gate_mask = 0xB2;
+ qopt->entries[0].interval = 100000;
+ qopt->entries[1].gate_mask = 0xB3;
+ qopt->entries[1].interval = 100000;
+ qopt->entries[2].gate_mask = 0xB4;
+ qopt->entries[2].interval = 100000;
+ qopt->entries[3].gate_mask = 0xB5;
+ qopt->entries[3].interval = 200000;
+ qopt->num_entries = 4;
+ delay_base_time(adapter, qopt, 19);
+ if (!enable_check_taprio(adapter, qopt, 100))
+ goto failed;
+ qopt->base_time = ktime_set(0, 0);
+ qopt->cycle_time = 6000000;
+ qopt->cycle_time_extension = 5999999;
+ qopt->entries[0].gate_mask = 0xC6;
+ qopt->entries[0].interval = 1000000;
+ qopt->entries[1].gate_mask = 0xC7;
+ qopt->entries[1].interval = 1000000;
+ qopt->entries[2].gate_mask = 0xC8;
+ qopt->entries[2].interval = 1000000;
+ qopt->entries[3].gate_mask = 0xC9;
+ qopt->entries[3].interval = 1500000;
+ qopt->entries[4].gate_mask = 0xCA;
+ qopt->entries[4].interval = 1500000;
+ qopt->num_entries = 5;
+ delay_base_time(adapter, qopt, 1);
+ if (!enable_check_taprio(adapter, qopt, 100))
+ goto failed;
+
+ if (!disable_taprio(adapter))
+ goto failed;
+
+ kfree(qopt);
+
+ return true;
+
+failed:
+ disable_taprio(adapter);
+ kfree(qopt);
+
+ return false;
+}
+
+int tsnep_ethtool_get_test_count(void)
+{
+ return TSNEP_TEST_COUNT;
+}
+
+void tsnep_ethtool_get_test_strings(u8 *data)
+{
+ memcpy(data, tsnep_test_strings, sizeof(tsnep_test_strings));
+}
+
+void tsnep_ethtool_self_test(struct net_device *netdev,
+ struct ethtool_test *eth_test, u64 *data)
+{
+ struct tsnep_adapter *adapter = netdev_priv(netdev);
+
+ eth_test->len = TSNEP_TEST_COUNT;
+
+ if (eth_test->flags != ETH_TEST_FL_OFFLINE) {
+ /* no tests are done online */
+ data[TSNEP_TEST_ENABLE] = 0;
+ data[TSNEP_TEST_TAPRIO] = 0;
+ data[TSNEP_TEST_TAPRIO_CHANGE] = 0;
+ data[TSNEP_TEST_TAPRIO_EXTENSION] = 0;
+
+ return;
+ }
+
+ if (tsnep_test_gc_enable(adapter)) {
+ data[TSNEP_TEST_ENABLE] = 0;
+ } else {
+ eth_test->flags |= ETH_TEST_FL_FAILED;
+ data[TSNEP_TEST_ENABLE] = 1;
+ }
+
+ if (tsnep_test_taprio(adapter)) {
+ data[TSNEP_TEST_TAPRIO] = 0;
+ } else {
+ eth_test->flags |= ETH_TEST_FL_FAILED;
+ data[TSNEP_TEST_TAPRIO] = 1;
+ }
+
+ if (tsnep_test_taprio_change(adapter)) {
+ data[TSNEP_TEST_TAPRIO_CHANGE] = 0;
+ } else {
+ eth_test->flags |= ETH_TEST_FL_FAILED;
+ data[TSNEP_TEST_TAPRIO_CHANGE] = 1;
+ }
+
+ if (tsnep_test_taprio_extension(adapter)) {
+ data[TSNEP_TEST_TAPRIO_EXTENSION] = 0;
+ } else {
+ eth_test->flags |= ETH_TEST_FL_FAILED;
+ data[TSNEP_TEST_TAPRIO_EXTENSION] = 1;
+ }
+}
diff --git a/drivers/net/ethernet/engleder/tsnep_tc.c b/drivers/net/ethernet/engleder/tsnep_tc.c
new file mode 100644
index 0000000..c4c6e13
--- /dev/null
+++ b/drivers/net/ethernet/engleder/tsnep_tc.c
@@ -0,0 +1,443 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2021 Gerhard Engleder <gerhard@engleder-embedded.com> */
+
+#include "tsnep.h"
+
+#include <net/pkt_sched.h>
+
+/* save one operation at the end for additional operation at list change */
+#define TSNEP_MAX_GCL_NUM (TSNEP_GCL_COUNT - 1)
+
+static int tsnep_validate_gcl(struct tc_taprio_qopt_offload *qopt)
+{
+ int i;
+ u64 cycle_time;
+
+ if (!qopt->cycle_time)
+ return -ERANGE;
+ if (qopt->num_entries > TSNEP_MAX_GCL_NUM)
+ return -EINVAL;
+ cycle_time = 0;
+ for (i = 0; i < qopt->num_entries; i++) {
+ if (qopt->entries[i].command != TC_TAPRIO_CMD_SET_GATES)
+ return -EINVAL;
+ if (qopt->entries[i].gate_mask & ~TSNEP_GCL_MASK)
+ return -EINVAL;
+ if (qopt->entries[i].interval < TSNEP_GCL_MIN_INTERVAL)
+ return -EINVAL;
+ cycle_time += qopt->entries[i].interval;
+ }
+ if (qopt->cycle_time != cycle_time)
+ return -EINVAL;
+ if (qopt->cycle_time_extension >= qopt->cycle_time)
+ return -EINVAL;
+
+ return 0;
+}
+
+static void tsnep_write_gcl_operation(struct tsnep_gcl *gcl, int index,
+ u32 properties, u32 interval, bool flush)
+{
+ void __iomem *addr = gcl->addr +
+ sizeof(struct tsnep_gcl_operation) * index;
+
+ gcl->operation[index].properties = properties;
+ gcl->operation[index].interval = interval;
+
+ iowrite32(properties, addr);
+ iowrite32(interval, addr + sizeof(u32));
+
+ if (flush) {
+ /* flush write with read access */
+ ioread32(addr);
+ }
+}
+
+static u64 tsnep_change_duration(struct tsnep_gcl *gcl, int index)
+{
+ u64 duration;
+ int count;
+
+ /* change needs to be triggered one or two operations before start of
+ * new gate control list
+ * - change is triggered at start of operation (minimum one operation)
+ * - operation with adjusted interval is inserted on demand to exactly
+ * meet the start of the new gate control list (optional)
+ *
+ * additionally properties are read directly after start of previous
+ * operation
+ *
+ * therefore, three operations needs to be considered for the limit
+ */
+ duration = 0;
+ count = 3;
+ while (count) {
+ duration += gcl->operation[index].interval;
+
+ index--;
+ if (index < 0)
+ index = gcl->count - 1;
+
+ count--;
+ }
+
+ return duration;
+}
+
+static void tsnep_write_gcl(struct tsnep_gcl *gcl,
+ struct tc_taprio_qopt_offload *qopt)
+{
+ int i;
+ u32 properties;
+ u64 extend;
+ u64 cut;
+
+ gcl->base_time = ktime_to_ns(qopt->base_time);
+ gcl->cycle_time = qopt->cycle_time;
+ gcl->cycle_time_extension = qopt->cycle_time_extension;
+
+ for (i = 0; i < qopt->num_entries; i++) {
+ properties = qopt->entries[i].gate_mask;
+ if (i == (qopt->num_entries - 1))
+ properties |= TSNEP_GCL_LAST;
+
+ tsnep_write_gcl_operation(gcl, i, properties,
+ qopt->entries[i].interval, true);
+ }
+ gcl->count = qopt->num_entries;
+
+ /* calculate change limit; i.e., the time needed between enable and
+ * start of new gate control list
+ */
+
+ /* case 1: extend cycle time for change
+ * - change duration of last operation
+ * - cycle time extension
+ */
+ extend = tsnep_change_duration(gcl, gcl->count - 1);
+ extend += gcl->cycle_time_extension;
+
+ /* case 2: cut cycle time for change
+ * - maximum change duration
+ */
+ cut = 0;
+ for (i = 0; i < gcl->count; i++)
+ cut = max(cut, tsnep_change_duration(gcl, i));
+
+ /* use maximum, because the actual case (extend or cut) can be
+ * determined only after limit is known (chicken-and-egg problem)
+ */
+ gcl->change_limit = max(extend, cut);
+}
+
+static u64 tsnep_gcl_start_after(struct tsnep_gcl *gcl, u64 limit)
+{
+ u64 start = gcl->base_time;
+ u64 n;
+
+ if (start <= limit) {
+ n = div64_u64(limit - start, gcl->cycle_time);
+ start += (n + 1) * gcl->cycle_time;
+ }
+
+ return start;
+}
+
+static u64 tsnep_gcl_start_before(struct tsnep_gcl *gcl, u64 limit)
+{
+ u64 start = gcl->base_time;
+ u64 n;
+
+ n = div64_u64(limit - start, gcl->cycle_time);
+ start += n * gcl->cycle_time;
+ if (start == limit)
+ start -= gcl->cycle_time;
+
+ return start;
+}
+
+static u64 tsnep_set_gcl_change(struct tsnep_gcl *gcl, int index, u64 change,
+ bool insert)
+{
+ /* previous operation triggers change and properties are evaluated at
+ * start of operation
+ */
+ if (index == 0)
+ index = gcl->count - 1;
+ else
+ index = index - 1;
+ change -= gcl->operation[index].interval;
+
+ /* optionally change to new list with additional operation in between */
+ if (insert) {
+ void __iomem *addr = gcl->addr +
+ sizeof(struct tsnep_gcl_operation) * index;
+
+ gcl->operation[index].properties |= TSNEP_GCL_INSERT;
+ iowrite32(gcl->operation[index].properties, addr);
+ }
+
+ return change;
+}
+
+static void tsnep_clean_gcl(struct tsnep_gcl *gcl)
+{
+ int i;
+ u32 mask = TSNEP_GCL_LAST | TSNEP_GCL_MASK;
+ void __iomem *addr;
+
+ /* search for insert operation and reset properties */
+ for (i = 0; i < gcl->count; i++) {
+ if (gcl->operation[i].properties & ~mask) {
+ addr = gcl->addr +
+ sizeof(struct tsnep_gcl_operation) * i;
+
+ gcl->operation[i].properties &= mask;
+ iowrite32(gcl->operation[i].properties, addr);
+
+ break;
+ }
+ }
+}
+
+static u64 tsnep_insert_gcl_operation(struct tsnep_gcl *gcl, int ref,
+ u64 change, u32 interval)
+{
+ u32 properties;
+
+ properties = gcl->operation[ref].properties & TSNEP_GCL_MASK;
+ /* change to new list directly after inserted operation */
+ properties |= TSNEP_GCL_CHANGE;
+
+ /* last operation of list is reserved to insert operation */
+ tsnep_write_gcl_operation(gcl, TSNEP_GCL_COUNT - 1, properties,
+ interval, false);
+
+ return tsnep_set_gcl_change(gcl, ref, change, true);
+}
+
+static u64 tsnep_extend_gcl(struct tsnep_gcl *gcl, u64 start, u32 extension)
+{
+ int ref = gcl->count - 1;
+ u32 interval = gcl->operation[ref].interval + extension;
+
+ start -= gcl->operation[ref].interval;
+
+ return tsnep_insert_gcl_operation(gcl, ref, start, interval);
+}
+
+static u64 tsnep_cut_gcl(struct tsnep_gcl *gcl, u64 start, u64 cycle_time)
+{
+ u64 sum = 0;
+ int i;
+
+ /* find operation which shall be cutted */
+ for (i = 0; i < gcl->count; i++) {
+ u64 sum_tmp = sum + gcl->operation[i].interval;
+ u64 interval;
+
+ /* sum up operations as long as cycle time is not exceeded */
+ if (sum_tmp > cycle_time)
+ break;
+
+ /* remaining interval must be big enough for hardware */
+ interval = cycle_time - sum_tmp;
+ if (interval > 0 && interval < TSNEP_GCL_MIN_INTERVAL)
+ break;
+
+ sum = sum_tmp;
+ }
+ if (sum == cycle_time) {
+ /* no need to cut operation itself or whole cycle
+ * => change exactly at operation
+ */
+ return tsnep_set_gcl_change(gcl, i, start + sum, false);
+ }
+ return tsnep_insert_gcl_operation(gcl, i, start + sum,
+ cycle_time - sum);
+}
+
+static int tsnep_enable_gcl(struct tsnep_adapter *adapter,
+ struct tsnep_gcl *gcl, struct tsnep_gcl *curr)
+{
+ u64 system_time;
+ u64 timeout;
+ u64 limit;
+
+ /* estimate timeout limit after timeout enable, actually timeout limit
+ * in hardware will be earlier than estimate so we are on the safe side
+ */
+ tsnep_get_system_time(adapter, &system_time);
+ timeout = system_time + TSNEP_GC_TIMEOUT;
+
+ if (curr)
+ limit = timeout + curr->change_limit;
+ else
+ limit = timeout;
+
+ gcl->start_time = tsnep_gcl_start_after(gcl, limit);
+
+ /* gate control time register is only 32bit => time shall be in the near
+ * future (no driver support for far future implemented)
+ */
+ if ((gcl->start_time - system_time) >= U32_MAX)
+ return -EAGAIN;
+
+ if (curr) {
+ /* change gate control list */
+ u64 last;
+ u64 change;
+
+ last = tsnep_gcl_start_before(curr, gcl->start_time);
+ if ((last + curr->cycle_time) == gcl->start_time)
+ change = tsnep_cut_gcl(curr, last,
+ gcl->start_time - last);
+ else if (((gcl->start_time - last) <=
+ curr->cycle_time_extension) ||
+ ((gcl->start_time - last) <= TSNEP_GCL_MIN_INTERVAL))
+ change = tsnep_extend_gcl(curr, last,
+ gcl->start_time - last);
+ else
+ change = tsnep_cut_gcl(curr, last,
+ gcl->start_time - last);
+
+ WARN_ON(change <= timeout);
+ gcl->change = true;
+ iowrite32(change & 0xFFFFFFFF, adapter->addr + TSNEP_GC_CHANGE);
+ } else {
+ /* start gate control list */
+ WARN_ON(gcl->start_time <= timeout);
+ gcl->change = false;
+ iowrite32(gcl->start_time & 0xFFFFFFFF,
+ adapter->addr + TSNEP_GC_TIME);
+ }
+
+ return 0;
+}
+
+static int tsnep_taprio(struct tsnep_adapter *adapter,
+ struct tc_taprio_qopt_offload *qopt)
+{
+ struct tsnep_gcl *gcl;
+ struct tsnep_gcl *curr;
+ int retval;
+
+ if (!adapter->gate_control)
+ return -EOPNOTSUPP;
+
+ if (!qopt->enable) {
+ /* disable gate control if active */
+ mutex_lock(&adapter->gate_control_lock);
+
+ if (adapter->gate_control_active) {
+ iowrite8(TSNEP_GC_DISABLE, adapter->addr + TSNEP_GC);
+ adapter->gate_control_active = false;
+ }
+
+ mutex_unlock(&adapter->gate_control_lock);
+
+ return 0;
+ }
+
+ retval = tsnep_validate_gcl(qopt);
+ if (retval)
+ return retval;
+
+ mutex_lock(&adapter->gate_control_lock);
+
+ gcl = &adapter->gcl[adapter->next_gcl];
+ tsnep_write_gcl(gcl, qopt);
+
+ /* select current gate control list if active */
+ if (adapter->gate_control_active) {
+ if (adapter->next_gcl == 0)
+ curr = &adapter->gcl[1];
+ else
+ curr = &adapter->gcl[0];
+ } else {
+ curr = NULL;
+ }
+
+ for (;;) {
+ /* start timeout which discards late enable, this helps ensuring
+ * that start/change time are in the future at enable
+ */
+ iowrite8(TSNEP_GC_ENABLE_TIMEOUT, adapter->addr + TSNEP_GC);
+
+ retval = tsnep_enable_gcl(adapter, gcl, curr);
+ if (retval) {
+ mutex_unlock(&adapter->gate_control_lock);
+
+ return retval;
+ }
+
+ /* enable gate control list */
+ if (adapter->next_gcl == 0)
+ iowrite8(TSNEP_GC_ENABLE_A, adapter->addr + TSNEP_GC);
+ else
+ iowrite8(TSNEP_GC_ENABLE_B, adapter->addr + TSNEP_GC);
+
+ /* done if timeout did not happen */
+ if (!(ioread32(adapter->addr + TSNEP_GC) &
+ TSNEP_GC_TIMEOUT_SIGNAL))
+ break;
+
+ /* timeout is acknowledged with any enable */
+ iowrite8(TSNEP_GC_ENABLE_A, adapter->addr + TSNEP_GC);
+
+ if (curr)
+ tsnep_clean_gcl(curr);
+
+ /* retry because of timeout */
+ }
+
+ adapter->gate_control_active = true;
+
+ if (adapter->next_gcl == 0)
+ adapter->next_gcl = 1;
+ else
+ adapter->next_gcl = 0;
+
+ mutex_unlock(&adapter->gate_control_lock);
+
+ return 0;
+}
+
+int tsnep_tc_setup(struct net_device *netdev, enum tc_setup_type type,
+ void *type_data)
+{
+ struct tsnep_adapter *adapter = netdev_priv(netdev);
+
+ switch (type) {
+ case TC_SETUP_QDISC_TAPRIO:
+ return tsnep_taprio(adapter, type_data);
+ default:
+ return -EOPNOTSUPP;
+ }
+}
+
+int tsnep_tc_init(struct tsnep_adapter *adapter)
+{
+ if (!adapter->gate_control)
+ return 0;
+
+ /* open all gates */
+ iowrite8(TSNEP_GC_DISABLE, adapter->addr + TSNEP_GC);
+ iowrite32(TSNEP_GC_OPEN | TSNEP_GC_NEXT_OPEN, adapter->addr + TSNEP_GC);
+
+ adapter->gcl[0].addr = adapter->addr + TSNEP_GCL_A;
+ adapter->gcl[1].addr = adapter->addr + TSNEP_GCL_B;
+
+ return 0;
+}
+
+void tsnep_tc_cleanup(struct tsnep_adapter *adapter)
+{
+ if (!adapter->gate_control)
+ return;
+
+ if (adapter->gate_control_active) {
+ iowrite8(TSNEP_GC_DISABLE, adapter->addr + TSNEP_GC);
+ adapter->gate_control_active = false;
+ }
+}
diff --git a/drivers/net/ethernet/ethoc.c b/drivers/net/ethernet/ethoc.c
index b1c8ffe..437c5ac 100644
--- a/drivers/net/ethernet/ethoc.c
+++ b/drivers/net/ethernet/ethoc.c
@@ -945,7 +945,9 @@ static void ethoc_get_regs(struct net_device *dev, struct ethtool_regs *regs,
}
static void ethoc_get_ringparam(struct net_device *dev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct ethoc *priv = netdev_priv(dev);
@@ -961,7 +963,9 @@ static void ethoc_get_ringparam(struct net_device *dev,
}
static int ethoc_set_ringparam(struct net_device *dev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct ethoc *priv = netdev_priv(dev);
@@ -1074,14 +1078,11 @@ static int ethoc_probe(struct platform_device *pdev)
/* obtain device IRQ number */
- res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
- if (!res) {
- dev_err(&pdev->dev, "cannot obtain IRQ\n");
- ret = -ENXIO;
+ ret = platform_get_irq(pdev, 0);
+ if (ret < 0)
goto free;
- }
- netdev->irq = res->start;
+ netdev->irq = ret;
/* setup driver-private data */
priv = netdev_priv(netdev);
diff --git a/drivers/net/ethernet/faraday/ftgmac100.c b/drivers/net/ethernet/faraday/ftgmac100.c
index 97c5d70..691605c 100644
--- a/drivers/net/ethernet/faraday/ftgmac100.c
+++ b/drivers/net/ethernet/faraday/ftgmac100.c
@@ -1178,8 +1178,11 @@ static void ftgmac100_get_drvinfo(struct net_device *netdev,
strlcpy(info->bus_info, dev_name(&netdev->dev), sizeof(info->bus_info));
}
-static void ftgmac100_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ering)
+static void
+ftgmac100_get_ringparam(struct net_device *netdev,
+ struct ethtool_ringparam *ering,
+ struct kernel_ethtool_ringparam *kernel_ering,
+ struct netlink_ext_ack *extack)
{
struct ftgmac100 *priv = netdev_priv(netdev);
@@ -1190,8 +1193,11 @@ static void ftgmac100_get_ringparam(struct net_device *netdev,
ering->tx_pending = priv->tx_q_entries;
}
-static int ftgmac100_set_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ering)
+static int
+ftgmac100_set_ringparam(struct net_device *netdev,
+ struct ethtool_ringparam *ering,
+ struct kernel_ethtool_ringparam *kernel_ering,
+ struct netlink_ext_ack *extack)
{
struct ftgmac100 *priv = netdev_priv(netdev);
diff --git a/drivers/net/ethernet/freescale/dpaa/dpaa_eth.c b/drivers/net/ethernet/freescale/dpaa/dpaa_eth.c
index 6b2927d..c78883c 100644
--- a/drivers/net/ethernet/freescale/dpaa/dpaa_eth.c
+++ b/drivers/net/ethernet/freescale/dpaa/dpaa_eth.c
@@ -2325,7 +2325,7 @@ dpaa_start_xmit(struct sk_buff *skb, struct net_device *net_dev)
txq = netdev_get_tx_queue(net_dev, queue_mapping);
/* LLTX requires to do our own update of trans_start */
- txq->trans_start = jiffies;
+ txq_trans_cond_update(txq);
if (priv->tx_tstamp && skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP) {
fd.cmd |= cpu_to_be32(FM_FD_CMD_UPD);
@@ -2531,7 +2531,7 @@ static int dpaa_xdp_xmit_frame(struct net_device *net_dev,
/* Bump the trans_start */
txq = netdev_get_tx_queue(net_dev, smp_processor_id());
- txq->trans_start = jiffies;
+ txq_trans_cond_update(txq);
err = dpaa_xmit(priv, percpu_stats, smp_processor_id(), &fd);
if (err) {
@@ -2623,7 +2623,7 @@ static u32 dpaa_run_xdp(struct dpaa_priv *priv, struct qm_fd *fd, void *vaddr,
}
break;
default:
- bpf_warn_invalid_xdp_action(xdp_act);
+ bpf_warn_invalid_xdp_action(priv->net_dev, xdp_prog, xdp_act);
fallthrough;
case XDP_ABORTED:
trace_xdp_exception(priv->net_dev, xdp_prog, xdp_act);
diff --git a/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c b/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c
index 8e64356..d21ba70 100644
--- a/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c
+++ b/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c
@@ -374,7 +374,7 @@ static u32 dpaa2_eth_run_xdp(struct dpaa2_eth_priv *priv,
dpaa2_eth_xdp_enqueue(priv, ch, fd, vaddr, rx_fq->flowid);
break;
default:
- bpf_warn_invalid_xdp_action(xdp_act);
+ bpf_warn_invalid_xdp_action(priv->net_dev, xdp_prog, xdp_act);
fallthrough;
case XDP_ABORTED:
trace_xdp_exception(priv->net_dev, xdp_prog, xdp_act);
diff --git a/drivers/net/ethernet/freescale/dpaa2/dpaa2-mac.c b/drivers/net/ethernet/freescale/dpaa2/dpaa2-mac.c
index ef8f0a0..623d113 100644
--- a/drivers/net/ethernet/freescale/dpaa2/dpaa2-mac.c
+++ b/drivers/net/ethernet/freescale/dpaa2/dpaa2-mac.c
@@ -2,6 +2,7 @@
/* Copyright 2019 NXP */
#include <linux/acpi.h>
+#include <linux/pcs-lynx.h>
#include <linux/property.h>
#include "dpaa2-eth.h"
@@ -40,7 +41,7 @@ static int phy_mode(enum dpmac_eth_if eth_if, phy_interface_t *if_mode)
static struct fwnode_handle *dpaa2_mac_get_node(struct device *dev,
u16 dpmac_id)
{
- struct fwnode_handle *fwnode, *parent, *child = NULL;
+ struct fwnode_handle *fwnode, *parent = NULL, *child = NULL;
struct device_node *dpmacs = NULL;
int err;
u32 id;
@@ -53,8 +54,17 @@ static struct fwnode_handle *dpaa2_mac_get_node(struct device *dev,
parent = of_fwnode_handle(dpmacs);
} else if (is_acpi_node(fwnode)) {
parent = fwnode;
+ } else {
+ /* The root dprc device didn't yet get to finalize it's probe,
+ * thus the fwnode field is not yet set. Defer probe if we are
+ * facing this situation.
+ */
+ return ERR_PTR(-EPROBE_DEFER);
}
+ if (!parent)
+ return NULL;
+
fwnode_for_each_child_node(parent, child) {
err = -EINVAL;
if (is_acpi_device_node(child))
@@ -90,88 +100,6 @@ static int dpaa2_mac_get_if_mode(struct fwnode_handle *dpmac_node,
return err;
}
-static bool dpaa2_mac_phy_mode_mismatch(struct dpaa2_mac *mac,
- phy_interface_t interface)
-{
- switch (interface) {
- /* We can switch between SGMII and 1000BASE-X at runtime with
- * pcs-lynx
- */
- case PHY_INTERFACE_MODE_SGMII:
- case PHY_INTERFACE_MODE_1000BASEX:
- if (mac->pcs &&
- (mac->if_mode == PHY_INTERFACE_MODE_SGMII ||
- mac->if_mode == PHY_INTERFACE_MODE_1000BASEX))
- return false;
- return interface != mac->if_mode;
-
- case PHY_INTERFACE_MODE_10GBASER:
- case PHY_INTERFACE_MODE_USXGMII:
- case PHY_INTERFACE_MODE_QSGMII:
- case PHY_INTERFACE_MODE_RGMII:
- case PHY_INTERFACE_MODE_RGMII_ID:
- case PHY_INTERFACE_MODE_RGMII_RXID:
- case PHY_INTERFACE_MODE_RGMII_TXID:
- return (interface != mac->if_mode);
- default:
- return true;
- }
-}
-
-static void dpaa2_mac_validate(struct phylink_config *config,
- unsigned long *supported,
- struct phylink_link_state *state)
-{
- struct dpaa2_mac *mac = phylink_to_dpaa2_mac(config);
- __ETHTOOL_DECLARE_LINK_MODE_MASK(mask) = { 0, };
-
- if (state->interface != PHY_INTERFACE_MODE_NA &&
- dpaa2_mac_phy_mode_mismatch(mac, state->interface)) {
- goto empty_set;
- }
-
- phylink_set_port_modes(mask);
- phylink_set(mask, Autoneg);
- phylink_set(mask, Pause);
- phylink_set(mask, Asym_Pause);
-
- switch (state->interface) {
- case PHY_INTERFACE_MODE_NA:
- case PHY_INTERFACE_MODE_10GBASER:
- case PHY_INTERFACE_MODE_USXGMII:
- phylink_set_10g_modes(mask);
- if (state->interface == PHY_INTERFACE_MODE_10GBASER)
- break;
- phylink_set(mask, 5000baseT_Full);
- phylink_set(mask, 2500baseT_Full);
- fallthrough;
- case PHY_INTERFACE_MODE_SGMII:
- case PHY_INTERFACE_MODE_QSGMII:
- case PHY_INTERFACE_MODE_1000BASEX:
- case PHY_INTERFACE_MODE_RGMII:
- case PHY_INTERFACE_MODE_RGMII_ID:
- case PHY_INTERFACE_MODE_RGMII_RXID:
- case PHY_INTERFACE_MODE_RGMII_TXID:
- phylink_set(mask, 1000baseX_Full);
- phylink_set(mask, 1000baseT_Full);
- if (state->interface == PHY_INTERFACE_MODE_1000BASEX)
- break;
- phylink_set(mask, 100baseT_Full);
- phylink_set(mask, 10baseT_Full);
- break;
- default:
- goto empty_set;
- }
-
- linkmode_and(supported, supported, mask);
- linkmode_and(state->advertising, state->advertising, mask);
-
- return;
-
-empty_set:
- linkmode_zero(supported);
-}
-
static void dpaa2_mac_config(struct phylink_config *config, unsigned int mode,
const struct phylink_link_state *state)
{
@@ -243,7 +171,7 @@ static void dpaa2_mac_link_down(struct phylink_config *config,
}
static const struct phylink_mac_ops dpaa2_mac_phylink_ops = {
- .validate = dpaa2_mac_validate,
+ .validate = phylink_generic_validate,
.mac_config = dpaa2_mac_config,
.mac_link_up = dpaa2_mac_link_up,
.mac_link_down = dpaa2_mac_link_down,
@@ -286,11 +214,13 @@ static int dpaa2_pcs_create(struct dpaa2_mac *mac,
static void dpaa2_pcs_destroy(struct dpaa2_mac *mac)
{
- struct lynx_pcs *pcs = mac->pcs;
+ struct phylink_pcs *phylink_pcs = mac->pcs;
- if (pcs) {
- struct device *dev = &pcs->mdio->dev;
- lynx_pcs_destroy(pcs);
+ if (phylink_pcs) {
+ struct mdio_device *mdio = lynx_get_mdio_device(phylink_pcs);
+ struct device *dev = &mdio->dev;
+
+ lynx_pcs_destroy(phylink_pcs);
put_device(dev);
mac->pcs = NULL;
}
@@ -336,9 +266,34 @@ int dpaa2_mac_connect(struct dpaa2_mac *mac)
return err;
}
+ memset(&mac->phylink_config, 0, sizeof(mac->phylink_config));
mac->phylink_config.dev = &net_dev->dev;
mac->phylink_config.type = PHYLINK_NETDEV;
+ mac->phylink_config.mac_capabilities = MAC_SYM_PAUSE | MAC_ASYM_PAUSE |
+ MAC_10FD | MAC_100FD | MAC_1000FD | MAC_2500FD | MAC_5000FD |
+ MAC_10000FD;
+
+ /* We support the current interface mode, and if we have a PCS
+ * similar interface modes that do not require the PLLs to be
+ * reconfigured.
+ */
+ __set_bit(mac->if_mode, mac->phylink_config.supported_interfaces);
+ if (mac->pcs) {
+ switch (mac->if_mode) {
+ case PHY_INTERFACE_MODE_1000BASEX:
+ case PHY_INTERFACE_MODE_SGMII:
+ __set_bit(PHY_INTERFACE_MODE_1000BASEX,
+ mac->phylink_config.supported_interfaces);
+ __set_bit(PHY_INTERFACE_MODE_SGMII,
+ mac->phylink_config.supported_interfaces);
+ break;
+
+ default:
+ break;
+ }
+ }
+
phylink = phylink_create(&mac->phylink_config,
dpmac_node, mac->if_mode,
&dpaa2_mac_phylink_ops);
@@ -349,7 +304,7 @@ int dpaa2_mac_connect(struct dpaa2_mac *mac)
mac->phylink = phylink;
if (mac->pcs)
- phylink_set_pcs(mac->phylink, &mac->pcs->pcs);
+ phylink_set_pcs(mac->phylink, mac->pcs);
err = phylink_fwnode_phy_connect(mac->phylink, dpmac_node, 0);
if (err) {
@@ -381,6 +336,7 @@ int dpaa2_mac_open(struct dpaa2_mac *mac)
{
struct fsl_mc_device *dpmac_dev = mac->mc_dev;
struct net_device *net_dev = mac->net_dev;
+ struct fwnode_handle *fw_node;
int err;
err = dpmac_open(mac->mc_io, 0, dpmac_dev->obj_desc.id,
@@ -400,7 +356,13 @@ int dpaa2_mac_open(struct dpaa2_mac *mac)
/* Find the device node representing the MAC device and link the device
* behind the associated netdev to it.
*/
- mac->fw_node = dpaa2_mac_get_node(&mac->mc_dev->dev, mac->attr.id);
+ fw_node = dpaa2_mac_get_node(&mac->mc_dev->dev, mac->attr.id);
+ if (IS_ERR(fw_node)) {
+ err = PTR_ERR(fw_node);
+ goto err_close_dpmac;
+ }
+
+ mac->fw_node = fw_node;
net_dev->dev.of_node = to_of_node(mac->fw_node);
return 0;
diff --git a/drivers/net/ethernet/freescale/dpaa2/dpaa2-mac.h b/drivers/net/ethernet/freescale/dpaa2/dpaa2-mac.h
index 7842cbb..1331a84 100644
--- a/drivers/net/ethernet/freescale/dpaa2/dpaa2-mac.h
+++ b/drivers/net/ethernet/freescale/dpaa2/dpaa2-mac.h
@@ -7,7 +7,6 @@
#include <linux/of_mdio.h>
#include <linux/of_net.h>
#include <linux/phylink.h>
-#include <linux/pcs-lynx.h>
#include "dpmac.h"
#include "dpmac-cmd.h"
@@ -23,7 +22,7 @@ struct dpaa2_mac {
struct phylink *phylink;
phy_interface_t if_mode;
enum dpmac_link_type if_link_type;
- struct lynx_pcs *pcs;
+ struct phylink_pcs *pcs;
struct fwnode_handle *fw_node;
};
diff --git a/drivers/net/ethernet/freescale/dpaa2/dpaa2-switch.c b/drivers/net/ethernet/freescale/dpaa2/dpaa2-switch.c
index d039457..9b5512b 100644
--- a/drivers/net/ethernet/freescale/dpaa2/dpaa2-switch.c
+++ b/drivers/net/ethernet/freescale/dpaa2/dpaa2-switch.c
@@ -394,7 +394,8 @@ static int dpaa2_switch_dellink(struct ethsw_core *ethsw, u16 vid)
for (i = 0; i < ethsw->sw_attr.num_ifs; i++) {
ppriv_local = ethsw->ports[i];
- ppriv_local->vlans[vid] = 0;
+ if (ppriv_local)
+ ppriv_local->vlans[vid] = 0;
}
return 0;
@@ -1896,9 +1897,11 @@ static int dpaa2_switch_port_del_vlan(struct ethsw_port_priv *port_priv, u16 vid
/* Delete VLAN from switch if it is no longer configured on
* any port
*/
- for (i = 0; i < ethsw->sw_attr.num_ifs; i++)
- if (ethsw->ports[i]->vlans[vid] & ETHSW_VLAN_MEMBER)
+ for (i = 0; i < ethsw->sw_attr.num_ifs; i++) {
+ if (ethsw->ports[i] &&
+ ethsw->ports[i]->vlans[vid] & ETHSW_VLAN_MEMBER)
return 0; /* Found a port member in VID */
+ }
ethsw->vlans[vid] &= ~ETHSW_VLAN_GLOBAL;
diff --git a/drivers/net/ethernet/freescale/enetc/enetc.c b/drivers/net/ethernet/freescale/enetc/enetc.c
index 504e125..d6930a7 100644
--- a/drivers/net/ethernet/freescale/enetc/enetc.c
+++ b/drivers/net/ethernet/freescale/enetc/enetc.c
@@ -1547,7 +1547,7 @@ static int enetc_clean_rx_ring_xdp(struct enetc_bdr *rx_ring,
switch (xdp_act) {
default:
- bpf_warn_invalid_xdp_action(xdp_act);
+ bpf_warn_invalid_xdp_action(rx_ring->ndev, prog, xdp_act);
fallthrough;
case XDP_ABORTED:
trace_xdp_exception(rx_ring->ndev, prog, xdp_act);
@@ -2897,12 +2897,8 @@ int enetc_pci_probe(struct pci_dev *pdev, const char *name, int sizeof_priv)
/* set up for high or low dma */
err = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64));
if (err) {
- err = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32));
- if (err) {
- dev_err(&pdev->dev,
- "DMA configuration failed: 0x%x\n", err);
- goto err_dma;
- }
+ dev_err(&pdev->dev, "DMA configuration failed: 0x%x\n", err);
+ goto err_dma;
}
err = pci_request_mem_regions(pdev, name);
diff --git a/drivers/net/ethernet/freescale/enetc/enetc_ethtool.c b/drivers/net/ethernet/freescale/enetc/enetc_ethtool.c
index 910b9f7..fa5b4f8 100644
--- a/drivers/net/ethernet/freescale/enetc/enetc_ethtool.c
+++ b/drivers/net/ethernet/freescale/enetc/enetc_ethtool.c
@@ -562,7 +562,9 @@ static int enetc_set_rxfh(struct net_device *ndev, const u32 *indir,
}
static void enetc_get_ringparam(struct net_device *ndev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct enetc_ndev_priv *priv = netdev_priv(ndev);
diff --git a/drivers/net/ethernet/freescale/enetc/enetc_pf.c b/drivers/net/ethernet/freescale/enetc/enetc_pf.c
index 0e87c70..ed16a5a 100644
--- a/drivers/net/ethernet/freescale/enetc/enetc_pf.c
+++ b/drivers/net/ethernet/freescale/enetc/enetc_pf.c
@@ -8,6 +8,7 @@
#include <linux/of_platform.h>
#include <linux/of_mdio.h>
#include <linux/of_net.h>
+#include <linux/pcs-lynx.h>
#include "enetc_ierb.h"
#include "enetc_pf.h"
@@ -828,8 +829,8 @@ static int enetc_imdio_create(struct enetc_pf *pf)
{
struct device *dev = &pf->si->pdev->dev;
struct enetc_mdio_priv *mdio_priv;
- struct lynx_pcs *pcs_lynx;
- struct mdio_device *pcs;
+ struct phylink_pcs *phylink_pcs;
+ struct mdio_device *mdio_device;
struct mii_bus *bus;
int err;
@@ -853,23 +854,23 @@ static int enetc_imdio_create(struct enetc_pf *pf)
goto free_mdio_bus;
}
- pcs = mdio_device_create(bus, 0);
- if (IS_ERR(pcs)) {
- err = PTR_ERR(pcs);
- dev_err(dev, "cannot create pcs (%d)\n", err);
+ mdio_device = mdio_device_create(bus, 0);
+ if (IS_ERR(mdio_device)) {
+ err = PTR_ERR(mdio_device);
+ dev_err(dev, "cannot create mdio device (%d)\n", err);
goto unregister_mdiobus;
}
- pcs_lynx = lynx_pcs_create(pcs);
- if (!pcs_lynx) {
- mdio_device_free(pcs);
+ phylink_pcs = lynx_pcs_create(mdio_device);
+ if (!phylink_pcs) {
+ mdio_device_free(mdio_device);
err = -ENOMEM;
dev_err(dev, "cannot create lynx pcs (%d)\n", err);
goto unregister_mdiobus;
}
pf->imdio = bus;
- pf->pcs = pcs_lynx;
+ pf->pcs = phylink_pcs;
return 0;
@@ -882,8 +883,11 @@ static int enetc_imdio_create(struct enetc_pf *pf)
static void enetc_imdio_remove(struct enetc_pf *pf)
{
+ struct mdio_device *mdio_device;
+
if (pf->pcs) {
- mdio_device_free(pf->pcs->mdio);
+ mdio_device = lynx_get_mdio_device(pf->pcs);
+ mdio_device_free(mdio_device);
lynx_pcs_destroy(pf->pcs);
}
if (pf->imdio) {
@@ -930,45 +934,6 @@ static void enetc_mdiobus_destroy(struct enetc_pf *pf)
enetc_imdio_remove(pf);
}
-static void enetc_pl_mac_validate(struct phylink_config *config,
- unsigned long *supported,
- struct phylink_link_state *state)
-{
- __ETHTOOL_DECLARE_LINK_MODE_MASK(mask) = { 0, };
-
- if (state->interface != PHY_INTERFACE_MODE_NA &&
- state->interface != PHY_INTERFACE_MODE_INTERNAL &&
- state->interface != PHY_INTERFACE_MODE_SGMII &&
- state->interface != PHY_INTERFACE_MODE_2500BASEX &&
- state->interface != PHY_INTERFACE_MODE_USXGMII &&
- !phy_interface_mode_is_rgmii(state->interface)) {
- linkmode_zero(supported);
- return;
- }
-
- phylink_set_port_modes(mask);
- phylink_set(mask, Autoneg);
- phylink_set(mask, Pause);
- phylink_set(mask, Asym_Pause);
- phylink_set(mask, 10baseT_Half);
- phylink_set(mask, 10baseT_Full);
- phylink_set(mask, 100baseT_Half);
- phylink_set(mask, 100baseT_Full);
- phylink_set(mask, 100baseT_Half);
- phylink_set(mask, 1000baseT_Half);
- phylink_set(mask, 1000baseT_Full);
-
- if (state->interface == PHY_INTERFACE_MODE_INTERNAL ||
- state->interface == PHY_INTERFACE_MODE_2500BASEX ||
- state->interface == PHY_INTERFACE_MODE_USXGMII) {
- phylink_set(mask, 2500baseT_Full);
- phylink_set(mask, 2500baseX_Full);
- }
-
- linkmode_and(supported, supported, mask);
- linkmode_and(state->advertising, state->advertising, mask);
-}
-
static void enetc_pl_mac_config(struct phylink_config *config,
unsigned int mode,
const struct phylink_link_state *state)
@@ -980,7 +945,7 @@ static void enetc_pl_mac_config(struct phylink_config *config,
priv = netdev_priv(pf->si->ndev);
if (pf->pcs)
- phylink_set_pcs(priv->phylink, &pf->pcs->pcs);
+ phylink_set_pcs(priv->phylink, pf->pcs);
}
static void enetc_force_rgmii_mac(struct enetc_hw *hw, int speed, int duplex)
@@ -1096,7 +1061,7 @@ static void enetc_pl_mac_link_down(struct phylink_config *config,
}
static const struct phylink_mac_ops enetc_mac_phylink_ops = {
- .validate = enetc_pl_mac_validate,
+ .validate = phylink_generic_validate,
.mac_config = enetc_pl_mac_config,
.mac_link_up = enetc_pl_mac_link_up,
.mac_link_down = enetc_pl_mac_link_down,
@@ -1111,6 +1076,18 @@ static int enetc_phylink_create(struct enetc_ndev_priv *priv,
pf->phylink_config.dev = &priv->ndev->dev;
pf->phylink_config.type = PHYLINK_NETDEV;
+ pf->phylink_config.mac_capabilities = MAC_ASYM_PAUSE | MAC_SYM_PAUSE |
+ MAC_10 | MAC_100 | MAC_1000 | MAC_2500FD;
+
+ __set_bit(PHY_INTERFACE_MODE_INTERNAL,
+ pf->phylink_config.supported_interfaces);
+ __set_bit(PHY_INTERFACE_MODE_SGMII,
+ pf->phylink_config.supported_interfaces);
+ __set_bit(PHY_INTERFACE_MODE_2500BASEX,
+ pf->phylink_config.supported_interfaces);
+ __set_bit(PHY_INTERFACE_MODE_USXGMII,
+ pf->phylink_config.supported_interfaces);
+ phy_interface_set_rgmii(pf->phylink_config.supported_interfaces);
phylink = phylink_create(&pf->phylink_config, of_fwnode_handle(node),
pf->if_mode, &enetc_mac_phylink_ops);
diff --git a/drivers/net/ethernet/freescale/enetc/enetc_pf.h b/drivers/net/ethernet/freescale/enetc/enetc_pf.h
index 263946c..c26bd66 100644
--- a/drivers/net/ethernet/freescale/enetc/enetc_pf.h
+++ b/drivers/net/ethernet/freescale/enetc/enetc_pf.h
@@ -2,7 +2,7 @@
/* Copyright 2017-2019 NXP */
#include "enetc.h"
-#include <linux/pcs-lynx.h>
+#include <linux/phylink.h>
#define ENETC_PF_NUM_RINGS 8
@@ -46,7 +46,7 @@ struct enetc_pf {
struct mii_bus *mdio; /* saved for cleanup */
struct mii_bus *imdio;
- struct lynx_pcs *pcs;
+ struct phylink_pcs *pcs;
phy_interface_t if_mode;
struct phylink_config phylink_config;
diff --git a/drivers/net/ethernet/freescale/enetc/enetc_ptp.c b/drivers/net/ethernet/freescale/enetc/enetc_ptp.c
index 36b4f51..17c097c 100644
--- a/drivers/net/ethernet/freescale/enetc/enetc_ptp.c
+++ b/drivers/net/ethernet/freescale/enetc/enetc_ptp.c
@@ -42,15 +42,10 @@ static int enetc_ptp_probe(struct pci_dev *pdev,
if (err)
return dev_err_probe(&pdev->dev, err, "device enable failed\n");
- /* set up for high or low dma */
err = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64));
if (err) {
- err = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32));
- if (err) {
- dev_err(&pdev->dev,
- "DMA configuration failed: 0x%x\n", err);
- goto err_dma;
- }
+ dev_err(&pdev->dev, "DMA configuration failed: 0x%x\n", err);
+ goto err_dma;
}
err = pci_request_mem_regions(pdev, KBUILD_MODNAME);
diff --git a/drivers/net/ethernet/freescale/enetc/enetc_qos.c b/drivers/net/ethernet/freescale/enetc/enetc_qos.c
index 0536d2c..3555c12 100644
--- a/drivers/net/ethernet/freescale/enetc/enetc_qos.c
+++ b/drivers/net/ethernet/freescale/enetc/enetc_qos.c
@@ -1182,7 +1182,7 @@ static int enetc_psfp_parse_clsflower(struct enetc_ndev_priv *priv,
}
/* parsing gate action */
- if (entryg->gate.index >= priv->psfp_cap.max_psfp_gate) {
+ if (entryg->hw_index >= priv->psfp_cap.max_psfp_gate) {
NL_SET_ERR_MSG_MOD(extack, "No Stream Gate resource!");
err = -ENOSPC;
goto free_filter;
@@ -1202,7 +1202,7 @@ static int enetc_psfp_parse_clsflower(struct enetc_ndev_priv *priv,
}
refcount_set(&sgi->refcount, 1);
- sgi->index = entryg->gate.index;
+ sgi->index = entryg->hw_index;
sgi->init_ipv = entryg->gate.prio;
sgi->basetime = entryg->gate.basetime;
sgi->cycletime = entryg->gate.cycletime;
@@ -1244,7 +1244,7 @@ static int enetc_psfp_parse_clsflower(struct enetc_ndev_priv *priv,
refcount_set(&fmi->refcount, 1);
fmi->cir = entryp->police.rate_bytes_ps;
fmi->cbs = entryp->police.burst;
- fmi->index = entryp->police.index;
+ fmi->index = entryp->hw_index;
filter->flags |= ENETC_PSFP_FLAGS_FMI;
filter->fmi_index = fmi->index;
sfi->meter_id = fmi->index;
diff --git a/drivers/net/ethernet/freescale/fec_main.c b/drivers/net/ethernet/freescale/fec_main.c
index 1b1f7f2..796133d 100644
--- a/drivers/net/ethernet/freescale/fec_main.c
+++ b/drivers/net/ethernet/freescale/fec_main.c
@@ -1185,6 +1185,21 @@ static void fec_enet_stop_mode(struct fec_enet_private *fep, bool enabled)
}
}
+static void fec_irqs_disable(struct net_device *ndev)
+{
+ struct fec_enet_private *fep = netdev_priv(ndev);
+
+ writel(0, fep->hwp + FEC_IMASK);
+}
+
+static void fec_irqs_disable_except_wakeup(struct net_device *ndev)
+{
+ struct fec_enet_private *fep = netdev_priv(ndev);
+
+ writel(0, fep->hwp + FEC_IMASK);
+ writel(FEC_ENET_WAKEUP, fep->hwp + FEC_IMASK);
+}
+
static void
fec_stop(struct net_device *ndev)
{
@@ -1211,15 +1226,13 @@ fec_stop(struct net_device *ndev)
writel(1, fep->hwp + FEC_ECNTRL);
udelay(10);
}
- writel(FEC_DEFAULT_IMASK, fep->hwp + FEC_IMASK);
} else {
- writel(FEC_DEFAULT_IMASK | FEC_ENET_WAKEUP, fep->hwp + FEC_IMASK);
val = readl(fep->hwp + FEC_ECNTRL);
val |= (FEC_ECR_MAGICEN | FEC_ECR_SLEEP);
writel(val, fep->hwp + FEC_ECNTRL);
- fec_enet_stop_mode(fep, true);
}
writel(fep->phy_speed, fep->hwp + FEC_MII_SPEED);
+ writel(FEC_DEFAULT_IMASK, fep->hwp + FEC_IMASK);
/* We have to keep ENET enabled to have MII interrupt stay working */
if (fep->quirks & FEC_QUIRK_ENET_MAC &&
@@ -2877,15 +2890,10 @@ fec_enet_set_wol(struct net_device *ndev, struct ethtool_wolinfo *wol)
return -EINVAL;
device_set_wakeup_enable(&ndev->dev, wol->wolopts & WAKE_MAGIC);
- if (device_may_wakeup(&ndev->dev)) {
+ if (device_may_wakeup(&ndev->dev))
fep->wol_flag |= FEC_WOL_FLAG_ENABLE;
- if (fep->wake_irq > 0)
- enable_irq_wake(fep->wake_irq);
- } else {
+ else
fep->wol_flag &= (~FEC_WOL_FLAG_ENABLE);
- if (fep->wake_irq > 0)
- disable_irq_wake(fep->wake_irq);
- }
return 0;
}
@@ -3558,7 +3566,7 @@ static int fec_enet_init(struct net_device *ndev)
ndev->features |= NETIF_F_HW_VLAN_CTAG_RX;
if (fep->quirks & FEC_QUIRK_HAS_CSUM) {
- ndev->gso_max_segs = FEC_MAX_TSO_SEGS;
+ netif_set_gso_max_segs(ndev, FEC_MAX_TSO_SEGS);
/* enable hw accelerator */
ndev->features |= (NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM
@@ -4057,9 +4065,19 @@ static int __maybe_unused fec_suspend(struct device *dev)
netif_device_detach(ndev);
netif_tx_unlock_bh(ndev);
fec_stop(ndev);
- fec_enet_clk_enable(ndev, false);
- if (!(fep->wol_flag & FEC_WOL_FLAG_ENABLE))
+ if (!(fep->wol_flag & FEC_WOL_FLAG_ENABLE)) {
+ fec_irqs_disable(ndev);
pinctrl_pm_select_sleep_state(&fep->pdev->dev);
+ } else {
+ fec_irqs_disable_except_wakeup(ndev);
+ if (fep->wake_irq > 0) {
+ disable_irq(fep->wake_irq);
+ enable_irq_wake(fep->wake_irq);
+ }
+ fec_enet_stop_mode(fep, true);
+ }
+ /* It's safe to disable clocks since interrupts are masked */
+ fec_enet_clk_enable(ndev, false);
}
rtnl_unlock();
@@ -4097,6 +4115,10 @@ static int __maybe_unused fec_resume(struct device *dev)
}
if (fep->wol_flag & FEC_WOL_FLAG_ENABLE) {
fec_enet_stop_mode(fep, false);
+ if (fep->wake_irq) {
+ disable_irq_wake(fep->wake_irq);
+ enable_irq(fep->wake_irq);
+ }
val = readl(fep->hwp + FEC_ECNTRL);
val &= ~(FEC_ECR_MAGICEN | FEC_ECR_SLEEP);
diff --git a/drivers/net/ethernet/freescale/fec_ptp.c b/drivers/net/ethernet/freescale/fec_ptp.c
index d71eac7..af99017 100644
--- a/drivers/net/ethernet/freescale/fec_ptp.c
+++ b/drivers/net/ethernet/freescale/fec_ptp.c
@@ -473,10 +473,6 @@ int fec_ptp_set(struct net_device *ndev, struct ifreq *ifr)
if (copy_from_user(&config, ifr->ifr_data, sizeof(config)))
return -EFAULT;
- /* reserved for future extensions */
- if (config.flags)
- return -EINVAL;
-
switch (config.tx_type) {
case HWTSTAMP_TX_OFF:
fep->hwts_tx_en = 0;
diff --git a/drivers/net/ethernet/freescale/fman/fman.c b/drivers/net/ethernet/freescale/fman/fman.c
index ce0a121..8f0db61 100644
--- a/drivers/net/ethernet/freescale/fman/fman.c
+++ b/drivers/net/ethernet/freescale/fman/fman.c
@@ -2727,7 +2727,7 @@ static struct fman *read_dts_node(struct platform_device *of_dev)
fman = kzalloc(sizeof(*fman), GFP_KERNEL);
if (!fman)
- return NULL;
+ return ERR_PTR(-ENOMEM);
fm_node = of_node_get(of_dev->dev.of_node);
@@ -2740,26 +2740,21 @@ static struct fman *read_dts_node(struct platform_device *of_dev)
fman->dts_params.id = (u8)val;
/* Get the FM interrupt */
- res = platform_get_resource(of_dev, IORESOURCE_IRQ, 0);
- if (!res) {
- dev_err(&of_dev->dev, "%s: Can't get FMan IRQ resource\n",
- __func__);
+ err = platform_get_irq(of_dev, 0);
+ if (err < 0)
goto fman_node_put;
- }
- irq = res->start;
+ irq = err;
/* Get the FM error interrupt */
- res = platform_get_resource(of_dev, IORESOURCE_IRQ, 1);
- if (!res) {
- dev_err(&of_dev->dev, "%s: Can't get FMan Error IRQ resource\n",
- __func__);
+ err = platform_get_irq(of_dev, 1);
+ if (err < 0)
goto fman_node_put;
- }
- fman->dts_params.err_irq = res->start;
+ fman->dts_params.err_irq = err;
/* Get the FM address */
res = platform_get_resource(of_dev, IORESOURCE_MEM, 0);
if (!res) {
+ err = -EINVAL;
dev_err(&of_dev->dev, "%s: Can't get FMan memory resource\n",
__func__);
goto fman_node_put;
@@ -2770,6 +2765,7 @@ static struct fman *read_dts_node(struct platform_device *of_dev)
clk = of_clk_get(fm_node, 0);
if (IS_ERR(clk)) {
+ err = PTR_ERR(clk);
dev_err(&of_dev->dev, "%s: Failed to get FM%d clock structure\n",
__func__, fman->dts_params.id);
goto fman_node_put;
@@ -2777,6 +2773,7 @@ static struct fman *read_dts_node(struct platform_device *of_dev)
clk_rate = clk_get_rate(clk);
if (!clk_rate) {
+ err = -EINVAL;
dev_err(&of_dev->dev, "%s: Failed to determine FM%d clock rate\n",
__func__, fman->dts_params.id);
goto fman_node_put;
@@ -2797,6 +2794,7 @@ static struct fman *read_dts_node(struct platform_device *of_dev)
/* Get the MURAM base address and size */
muram_node = of_find_matching_node(fm_node, fman_muram_match);
if (!muram_node) {
+ err = -EINVAL;
dev_err(&of_dev->dev, "%s: could not find MURAM node\n",
__func__);
goto fman_free;
@@ -2836,6 +2834,7 @@ static struct fman *read_dts_node(struct platform_device *of_dev)
devm_request_mem_region(&of_dev->dev, phys_base_addr,
mem_size, "fman");
if (!fman->dts_params.res) {
+ err = -EBUSY;
dev_err(&of_dev->dev, "%s: request_mem_region() failed\n",
__func__);
goto fman_free;
@@ -2844,6 +2843,7 @@ static struct fman *read_dts_node(struct platform_device *of_dev)
fman->dts_params.base_addr =
devm_ioremap(&of_dev->dev, phys_base_addr, mem_size);
if (!fman->dts_params.base_addr) {
+ err = -ENOMEM;
dev_err(&of_dev->dev, "%s: devm_ioremap() failed\n", __func__);
goto fman_free;
}
@@ -2868,7 +2868,7 @@ static struct fman *read_dts_node(struct platform_device *of_dev)
of_node_put(fm_node);
fman_free:
kfree(fman);
- return NULL;
+ return ERR_PTR(err);
}
static int fman_probe(struct platform_device *of_dev)
@@ -2880,8 +2880,8 @@ static int fman_probe(struct platform_device *of_dev)
dev = &of_dev->dev;
fman = read_dts_node(of_dev);
- if (!fman)
- return -EIO;
+ if (IS_ERR(fman))
+ return PTR_ERR(fman);
err = fman_config(fman);
if (err) {
diff --git a/drivers/net/ethernet/freescale/fman/fman_port.c b/drivers/net/ethernet/freescale/fman/fman_port.c
index d9baac0..4c9d05c 100644
--- a/drivers/net/ethernet/freescale/fman/fman_port.c
+++ b/drivers/net/ethernet/freescale/fman/fman_port.c
@@ -1805,7 +1805,7 @@ static int fman_port_probe(struct platform_device *of_dev)
fman = dev_get_drvdata(&fm_pdev->dev);
if (!fman) {
err = -EINVAL;
- goto return_err;
+ goto put_device;
}
err = of_property_read_u32(port_node, "cell-index", &val);
@@ -1813,7 +1813,7 @@ static int fman_port_probe(struct platform_device *of_dev)
dev_err(port->dev, "%s: reading cell-index for %pOF failed\n",
__func__, port_node);
err = -EINVAL;
- goto return_err;
+ goto put_device;
}
port_id = (u8)val;
port->dts_params.id = port_id;
@@ -1847,7 +1847,7 @@ static int fman_port_probe(struct platform_device *of_dev)
} else {
dev_err(port->dev, "%s: Illegal port type\n", __func__);
err = -EINVAL;
- goto return_err;
+ goto put_device;
}
port->dts_params.type = port_type;
@@ -1861,7 +1861,7 @@ static int fman_port_probe(struct platform_device *of_dev)
dev_err(port->dev, "%s: incorrect qman-channel-id\n",
__func__);
err = -EINVAL;
- goto return_err;
+ goto put_device;
}
port->dts_params.qman_channel_id = qman_channel_id;
}
@@ -1871,7 +1871,7 @@ static int fman_port_probe(struct platform_device *of_dev)
dev_err(port->dev, "%s: of_address_to_resource() failed\n",
__func__);
err = -ENOMEM;
- goto return_err;
+ goto put_device;
}
port->dts_params.fman = fman;
@@ -1896,6 +1896,8 @@ static int fman_port_probe(struct platform_device *of_dev)
return 0;
+put_device:
+ put_device(&fm_pdev->dev);
return_err:
of_node_put(port_node);
free_port:
diff --git a/drivers/net/ethernet/freescale/fman/mac.c b/drivers/net/ethernet/freescale/fman/mac.c
index d9fc5c4..39ae965 100644
--- a/drivers/net/ethernet/freescale/fman/mac.c
+++ b/drivers/net/ethernet/freescale/fman/mac.c
@@ -94,14 +94,17 @@ static void mac_exception(void *handle, enum fman_mac_exceptions ex)
__func__, ex);
}
-static void set_fman_mac_params(struct mac_device *mac_dev,
- struct fman_mac_params *params)
+static int set_fman_mac_params(struct mac_device *mac_dev,
+ struct fman_mac_params *params)
{
struct mac_priv_s *priv = mac_dev->priv;
params->base_addr = (typeof(params->base_addr))
devm_ioremap(priv->dev, mac_dev->res->start,
resource_size(mac_dev->res));
+ if (!params->base_addr)
+ return -ENOMEM;
+
memcpy(¶ms->addr, mac_dev->addr, sizeof(mac_dev->addr));
params->max_speed = priv->max_speed;
params->phy_if = mac_dev->phy_if;
@@ -112,6 +115,8 @@ static void set_fman_mac_params(struct mac_device *mac_dev,
params->event_cb = mac_exception;
params->dev_id = mac_dev;
params->internal_phy_node = priv->internal_phy_node;
+
+ return 0;
}
static int tgec_initialization(struct mac_device *mac_dev)
@@ -123,7 +128,9 @@ static int tgec_initialization(struct mac_device *mac_dev)
priv = mac_dev->priv;
- set_fman_mac_params(mac_dev, ¶ms);
+ err = set_fman_mac_params(mac_dev, ¶ms);
+ if (err)
+ goto _return;
mac_dev->fman_mac = tgec_config(¶ms);
if (!mac_dev->fman_mac) {
@@ -169,7 +176,9 @@ static int dtsec_initialization(struct mac_device *mac_dev)
priv = mac_dev->priv;
- set_fman_mac_params(mac_dev, ¶ms);
+ err = set_fman_mac_params(mac_dev, ¶ms);
+ if (err)
+ goto _return;
mac_dev->fman_mac = dtsec_config(¶ms);
if (!mac_dev->fman_mac) {
@@ -218,7 +227,9 @@ static int memac_initialization(struct mac_device *mac_dev)
priv = mac_dev->priv;
- set_fman_mac_params(mac_dev, ¶ms);
+ err = set_fman_mac_params(mac_dev, ¶ms);
+ if (err)
+ goto _return;
if (priv->max_speed == SPEED_10000)
params.phy_if = PHY_INTERFACE_MODE_XGMII;
diff --git a/drivers/net/ethernet/freescale/gianfar.c b/drivers/net/ethernet/freescale/gianfar.c
index acab58f..206b7a3 100644
--- a/drivers/net/ethernet/freescale/gianfar.c
+++ b/drivers/net/ethernet/freescale/gianfar.c
@@ -2076,10 +2076,6 @@ static int gfar_hwtstamp_set(struct net_device *netdev, struct ifreq *ifr)
if (copy_from_user(&config, ifr->ifr_data, sizeof(config)))
return -EFAULT;
- /* reserved for future extensions */
- if (config.flags)
- return -EINVAL;
-
switch (config.tx_type) {
case HWTSTAMP_TX_OFF:
priv->hwts_tx_en = 0;
diff --git a/drivers/net/ethernet/freescale/gianfar_ethtool.c b/drivers/net/ethernet/freescale/gianfar_ethtool.c
index 7b32ed2..ff75626 100644
--- a/drivers/net/ethernet/freescale/gianfar_ethtool.c
+++ b/drivers/net/ethernet/freescale/gianfar_ethtool.c
@@ -372,7 +372,9 @@ static int gfar_scoalesce(struct net_device *dev,
* rx, rx_mini, and rx_jumbo rings are the same size, as mini and
* jumbo are ignored by the driver */
static void gfar_gringparam(struct net_device *dev,
- struct ethtool_ringparam *rvals)
+ struct ethtool_ringparam *rvals,
+ struct kernel_ethtool_ringparam *kernel_rvals,
+ struct netlink_ext_ack *extack)
{
struct gfar_private *priv = netdev_priv(dev);
struct gfar_priv_tx_q *tx_queue = NULL;
@@ -399,7 +401,9 @@ static void gfar_gringparam(struct net_device *dev,
* necessary so that we don't mess things up while we're in motion.
*/
static int gfar_sringparam(struct net_device *dev,
- struct ethtool_ringparam *rvals)
+ struct ethtool_ringparam *rvals,
+ struct kernel_ethtool_ringparam *kernel_rvals,
+ struct netlink_ext_ack *extack)
{
struct gfar_private *priv = netdev_priv(dev);
int err = 0, i;
diff --git a/drivers/net/ethernet/freescale/ucc_geth_ethtool.c b/drivers/net/ethernet/freescale/ucc_geth_ethtool.c
index 14c08a8..69b2b98 100644
--- a/drivers/net/ethernet/freescale/ucc_geth_ethtool.c
+++ b/drivers/net/ethernet/freescale/ucc_geth_ethtool.c
@@ -207,7 +207,9 @@ uec_get_regs(struct net_device *netdev,
static void
uec_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct ucc_geth_private *ugeth = netdev_priv(netdev);
struct ucc_geth_info *ug_info = ugeth->ug_info;
@@ -226,7 +228,9 @@ uec_get_ringparam(struct net_device *netdev,
static int
uec_set_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct ucc_geth_private *ugeth = netdev_priv(netdev);
struct ucc_geth_info *ug_info = ugeth->ug_info;
diff --git a/drivers/net/ethernet/freescale/xgmac_mdio.c b/drivers/net/ethernet/freescale/xgmac_mdio.c
index 0b68852..5b8b9bc 100644
--- a/drivers/net/ethernet/freescale/xgmac_mdio.c
+++ b/drivers/net/ethernet/freescale/xgmac_mdio.c
@@ -47,7 +47,6 @@ struct tgec_mdio_controller {
#define MDIO_CTL_READ BIT(15)
#define MDIO_DATA(x) (x & 0xffff)
-#define MDIO_DATA_BSY BIT(31)
struct mdio_fsl_priv {
struct tgec_mdio_controller __iomem *mdio_base;
diff --git a/drivers/net/ethernet/google/gve/gve.h b/drivers/net/ethernet/google/gve/gve.h
index b719f72..5f5d4f7 100644
--- a/drivers/net/ethernet/google/gve/gve.h
+++ b/drivers/net/ethernet/google/gve/gve.h
@@ -229,6 +229,7 @@ struct gve_rx_ring {
/* A TX desc ring entry */
union gve_tx_desc {
struct gve_tx_pkt_desc pkt; /* first desc for a packet */
+ struct gve_tx_mtd_desc mtd; /* optional metadata descriptor */
struct gve_tx_seg_desc seg; /* subsequent descs for a packet */
};
@@ -441,13 +442,13 @@ struct gve_tx_ring {
* associated with that irq.
*/
struct gve_notify_block {
- __be32 irq_db_index; /* idx into Bar2 - set by device, must be 1st */
+ __be32 *irq_db_index; /* pointer to idx into Bar2 */
char name[IFNAMSIZ + 16]; /* name registered with the kernel */
struct napi_struct napi; /* kernel napi struct for this block */
struct gve_priv *priv;
struct gve_tx_ring *tx; /* tx rings on this block */
struct gve_rx_ring *rx; /* rx rings on this block */
-} ____cacheline_aligned;
+};
/* Tracks allowed and current queue settings */
struct gve_queue_config {
@@ -466,6 +467,10 @@ struct gve_options_dqo_rda {
u16 rx_buff_ring_entries; /* number of rx_buff descriptors */
};
+struct gve_irq_db {
+ __be32 index;
+} ____cacheline_aligned;
+
struct gve_ptype {
u8 l3_type; /* `gve_l3_type` in gve_adminq.h */
u8 l4_type; /* `gve_l4_type` in gve_adminq.h */
@@ -492,7 +497,8 @@ struct gve_priv {
struct gve_rx_ring *rx; /* array of rx_cfg.num_queues */
struct gve_queue_page_list *qpls; /* array of num qpls */
struct gve_notify_block *ntfy_blocks; /* array of num_ntfy_blks */
- dma_addr_t ntfy_block_bus;
+ struct gve_irq_db *irq_db_indices; /* array of num_ntfy_blks */
+ dma_addr_t irq_db_indices_bus;
struct msix_entry *msix_vectors; /* array of num_ntfy_blks + 1 */
char mgmt_msix_name[IFNAMSIZ + 16];
u32 mgmt_msix_idx;
@@ -551,6 +557,8 @@ struct gve_priv {
u32 page_alloc_fail; /* count of page alloc fails */
u32 dma_mapping_error; /* count of dma mapping errors */
u32 stats_report_trigger_cnt; /* count of device-requested stats-reports since last reset */
+ u32 suspend_cnt; /* count of times suspended */
+ u32 resume_cnt; /* count of times resumed */
struct workqueue_struct *gve_wq;
struct work_struct service_task;
struct work_struct stats_report_task;
@@ -567,6 +575,7 @@ struct gve_priv {
/* Gvnic device link speed from hypervisor. */
u64 link_speed;
+ bool up_before_suspend; /* True if dev was up before suspend */
struct gve_options_dqo_rda options_dqo_rda;
struct gve_ptype_lut *ptype_lut_dqo;
@@ -575,6 +584,10 @@ struct gve_priv {
int data_buffer_size_dqo;
enum gve_queue_format queue_format;
+
+ /* Interrupt coalescing settings */
+ u32 tx_coalesce_usecs;
+ u32 rx_coalesce_usecs;
};
enum gve_service_task_flags_bit {
@@ -733,7 +746,7 @@ static inline void gve_clear_report_stats(struct gve_priv *priv)
static inline __be32 __iomem *gve_irq_doorbell(struct gve_priv *priv,
struct gve_notify_block *block)
{
- return &priv->db_bar2[be32_to_cpu(block->irq_db_index)];
+ return &priv->db_bar2[be32_to_cpu(*block->irq_db_index)];
}
/* Returns the index into ntfy_blocks of the given tx ring's block
diff --git a/drivers/net/ethernet/google/gve/gve_adminq.c b/drivers/net/ethernet/google/gve/gve_adminq.c
index 326b56b..2ad7f57 100644
--- a/drivers/net/ethernet/google/gve/gve_adminq.c
+++ b/drivers/net/ethernet/google/gve/gve_adminq.c
@@ -462,7 +462,7 @@ int gve_adminq_configure_device_resources(struct gve_priv *priv,
.num_counters = cpu_to_be32(num_counters),
.irq_db_addr = cpu_to_be64(db_array_bus_addr),
.num_irq_dbs = cpu_to_be32(num_ntfy_blks),
- .irq_db_stride = cpu_to_be32(sizeof(priv->ntfy_blocks[0])),
+ .irq_db_stride = cpu_to_be32(sizeof(*priv->irq_db_indices)),
.ntfy_blk_msix_base_idx =
cpu_to_be32(GVE_NTFY_BLK_BASE_MSIX_IDX),
.queue_format = priv->queue_format,
diff --git a/drivers/net/ethernet/google/gve/gve_desc.h b/drivers/net/ethernet/google/gve/gve_desc.h
index 4d225a1..f4ae9e1 100644
--- a/drivers/net/ethernet/google/gve/gve_desc.h
+++ b/drivers/net/ethernet/google/gve/gve_desc.h
@@ -33,6 +33,14 @@ struct gve_tx_pkt_desc {
__be64 seg_addr; /* Base address (see note) of this segment */
} __packed;
+struct gve_tx_mtd_desc {
+ u8 type_flags; /* type is lower 4 bits, subtype upper */
+ u8 path_state; /* state is lower 4 bits, hash type upper */
+ __be16 reserved0;
+ __be32 path_hash;
+ __be64 reserved1;
+} __packed;
+
struct gve_tx_seg_desc {
u8 type_flags; /* type is lower 4 bits, flags upper */
u8 l3_offset; /* TSO: 2 byte units to start of IPH */
@@ -46,6 +54,7 @@ struct gve_tx_seg_desc {
#define GVE_TXD_STD (0x0 << 4) /* Std with Host Address */
#define GVE_TXD_TSO (0x1 << 4) /* TSO with Host Address */
#define GVE_TXD_SEG (0x2 << 4) /* Seg with Host Address */
+#define GVE_TXD_MTD (0x3 << 4) /* Metadata */
/* GVE Transmit Descriptor Flags for Std Pkts */
#define GVE_TXF_L4CSUM BIT(0) /* Need csum offload */
@@ -54,6 +63,17 @@ struct gve_tx_seg_desc {
/* GVE Transmit Descriptor Flags for TSO Segs */
#define GVE_TXSF_IPV6 BIT(1) /* IPv6 TSO */
+/* GVE Transmit Descriptor Options for MTD Segs */
+#define GVE_MTD_SUBTYPE_PATH 0
+
+#define GVE_MTD_PATH_STATE_DEFAULT 0
+#define GVE_MTD_PATH_STATE_TIMEOUT 1
+#define GVE_MTD_PATH_STATE_CONGESTION 2
+#define GVE_MTD_PATH_STATE_RETRANSMIT 3
+
+#define GVE_MTD_PATH_HASH_NONE (0x0 << 4)
+#define GVE_MTD_PATH_HASH_L4 (0x1 << 4)
+
/* GVE Receive Packet Descriptor */
/* The start of an ethernet packet comes 2 bytes into the rx buffer.
* gVNIC adds this padding so that both the DMA and the L3/4 protocol header
diff --git a/drivers/net/ethernet/google/gve/gve_dqo.h b/drivers/net/ethernet/google/gve/gve_dqo.h
index 8360423..1eb4d5f 100644
--- a/drivers/net/ethernet/google/gve/gve_dqo.h
+++ b/drivers/net/ethernet/google/gve/gve_dqo.h
@@ -18,6 +18,7 @@
#define GVE_TX_IRQ_RATELIMIT_US_DQO 50
#define GVE_RX_IRQ_RATELIMIT_US_DQO 20
+#define GVE_MAX_ITR_INTERVAL_DQO (GVE_ITR_INTERVAL_DQO_MASK * 2)
/* Timeout in seconds to wait for a reinjection completion after receiving
* its corresponding miss completion.
@@ -54,17 +55,17 @@ gve_tx_put_doorbell_dqo(const struct gve_priv *priv,
}
/* Builds register value to write to DQO IRQ doorbell to enable with specified
- * ratelimit.
+ * ITR interval.
*/
-static inline u32 gve_set_itr_ratelimit_dqo(u32 ratelimit_us)
+static inline u32 gve_setup_itr_interval_dqo(u32 interval_us)
{
u32 result = GVE_ITR_ENABLE_BIT_DQO;
/* Interval has 2us granularity. */
- ratelimit_us >>= 1;
+ interval_us >>= 1;
- ratelimit_us &= GVE_ITR_INTERVAL_DQO_MASK;
- result |= (ratelimit_us << GVE_ITR_INTERVAL_DQO_SHIFT);
+ interval_us &= GVE_ITR_INTERVAL_DQO_MASK;
+ result |= (interval_us << GVE_ITR_INTERVAL_DQO_SHIFT);
return result;
}
@@ -73,9 +74,20 @@ static inline void
gve_write_irq_doorbell_dqo(const struct gve_priv *priv,
const struct gve_notify_block *block, u32 val)
{
- u32 index = be32_to_cpu(block->irq_db_index);
+ u32 index = be32_to_cpu(*block->irq_db_index);
iowrite32(val, &priv->db_bar2[index]);
}
+/* Sets interrupt throttling interval and enables interrupt
+ * by writing to IRQ doorbell.
+ */
+static inline void
+gve_set_itr_coalesce_usecs_dqo(struct gve_priv *priv,
+ struct gve_notify_block *block,
+ u32 usecs)
+{
+ gve_write_irq_doorbell_dqo(priv, block,
+ gve_setup_itr_interval_dqo(usecs));
+}
#endif /* _GVE_DQO_H_ */
diff --git a/drivers/net/ethernet/google/gve/gve_ethtool.c b/drivers/net/ethernet/google/gve/gve_ethtool.c
index c8df47a..50b3849 100644
--- a/drivers/net/ethernet/google/gve/gve_ethtool.c
+++ b/drivers/net/ethernet/google/gve/gve_ethtool.c
@@ -8,6 +8,7 @@
#include <linux/rtnetlink.h>
#include "gve.h"
#include "gve_adminq.h"
+#include "gve_dqo.h"
static void gve_get_drvinfo(struct net_device *netdev,
struct ethtool_drvinfo *info)
@@ -42,7 +43,7 @@ static const char gve_gstrings_main_stats[][ETH_GSTRING_LEN] = {
};
static const char gve_gstrings_rx_stats[][ETH_GSTRING_LEN] = {
- "rx_posted_desc[%u]", "rx_completed_desc[%u]", "rx_bytes[%u]",
+ "rx_posted_desc[%u]", "rx_completed_desc[%u]", "rx_consumed_desc[%u]", "rx_bytes[%u]",
"rx_cont_packet_cnt[%u]", "rx_frag_flip_cnt[%u]", "rx_frag_copy_cnt[%u]",
"rx_dropped_pkt[%u]", "rx_copybreak_pkt[%u]", "rx_copied_pkt[%u]",
"rx_queue_drop_cnt[%u]", "rx_no_buffers_posted[%u]",
@@ -50,7 +51,7 @@ static const char gve_gstrings_rx_stats[][ETH_GSTRING_LEN] = {
};
static const char gve_gstrings_tx_stats[][ETH_GSTRING_LEN] = {
- "tx_posted_desc[%u]", "tx_completed_desc[%u]", "tx_bytes[%u]",
+ "tx_posted_desc[%u]", "tx_completed_desc[%u]", "tx_consumed_desc[%u]", "tx_bytes[%u]",
"tx_wake[%u]", "tx_stop[%u]", "tx_event_counter[%u]",
"tx_dma_mapping_error[%u]",
};
@@ -139,10 +140,11 @@ static void
gve_get_ethtool_stats(struct net_device *netdev,
struct ethtool_stats *stats, u64 *data)
{
- u64 tmp_rx_pkts, tmp_rx_bytes, tmp_rx_skb_alloc_fail, tmp_rx_buf_alloc_fail,
- tmp_rx_desc_err_dropped_pkt, tmp_tx_pkts, tmp_tx_bytes;
+ u64 tmp_rx_pkts, tmp_rx_bytes, tmp_rx_skb_alloc_fail,
+ tmp_rx_buf_alloc_fail, tmp_rx_desc_err_dropped_pkt,
+ tmp_tx_pkts, tmp_tx_bytes;
u64 rx_buf_alloc_fail, rx_desc_err_dropped_pkt, rx_pkts,
- rx_skb_alloc_fail, rx_bytes, tx_pkts, tx_bytes;
+ rx_skb_alloc_fail, rx_bytes, tx_pkts, tx_bytes, tx_dropped;
int stats_idx, base_stats_idx, max_stats_idx;
struct stats *report_stats;
int *rx_qid_to_stats_idx;
@@ -191,7 +193,7 @@ gve_get_ethtool_stats(struct net_device *netdev,
rx_desc_err_dropped_pkt += tmp_rx_desc_err_dropped_pkt;
}
}
- for (tx_pkts = 0, tx_bytes = 0, ring = 0;
+ for (tx_pkts = 0, tx_bytes = 0, tx_dropped = 0, ring = 0;
ring < priv->tx_cfg.num_queues; ring++) {
if (priv->tx) {
do {
@@ -203,6 +205,7 @@ gve_get_ethtool_stats(struct net_device *netdev,
start));
tx_pkts += tmp_tx_pkts;
tx_bytes += tmp_tx_bytes;
+ tx_dropped += priv->tx[ring].dropped_pkt;
}
}
@@ -214,9 +217,7 @@ gve_get_ethtool_stats(struct net_device *netdev,
/* total rx dropped packets */
data[i++] = rx_skb_alloc_fail + rx_buf_alloc_fail +
rx_desc_err_dropped_pkt;
- /* Skip tx_dropped */
- i++;
-
+ data[i++] = tx_dropped;
data[i++] = priv->tx_timeo_cnt;
data[i++] = rx_skb_alloc_fail;
data[i++] = rx_buf_alloc_fail;
@@ -255,6 +256,7 @@ gve_get_ethtool_stats(struct net_device *netdev,
data[i++] = rx->fill_cnt;
data[i++] = rx->cnt;
+ data[i++] = rx->fill_cnt - rx->cnt;
do {
start =
u64_stats_fetch_begin(&priv->rx[ring].statss);
@@ -318,12 +320,14 @@ gve_get_ethtool_stats(struct net_device *netdev,
if (gve_is_gqi(priv)) {
data[i++] = tx->req;
data[i++] = tx->done;
+ data[i++] = tx->req - tx->done;
} else {
/* DQO doesn't currently support
* posted/completed descriptor counts;
*/
data[i++] = 0;
data[i++] = 0;
+ data[i++] = tx->dqo_tx.tail - tx->dqo_tx.head;
}
do {
start =
@@ -419,7 +423,9 @@ static int gve_set_channels(struct net_device *netdev,
}
static void gve_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *cmd)
+ struct ethtool_ringparam *cmd,
+ struct kernel_ethtool_ringparam *kernel_cmd,
+ struct netlink_ext_ack *extack)
{
struct gve_priv *priv = netdev_priv(netdev);
@@ -535,7 +541,65 @@ static int gve_get_link_ksettings(struct net_device *netdev,
return err;
}
+static int gve_get_coalesce(struct net_device *netdev,
+ struct ethtool_coalesce *ec,
+ struct kernel_ethtool_coalesce *kernel_ec,
+ struct netlink_ext_ack *extack)
+{
+ struct gve_priv *priv = netdev_priv(netdev);
+
+ if (gve_is_gqi(priv))
+ return -EOPNOTSUPP;
+ ec->tx_coalesce_usecs = priv->tx_coalesce_usecs;
+ ec->rx_coalesce_usecs = priv->rx_coalesce_usecs;
+
+ return 0;
+}
+
+static int gve_set_coalesce(struct net_device *netdev,
+ struct ethtool_coalesce *ec,
+ struct kernel_ethtool_coalesce *kernel_ec,
+ struct netlink_ext_ack *extack)
+{
+ struct gve_priv *priv = netdev_priv(netdev);
+ u32 tx_usecs_orig = priv->tx_coalesce_usecs;
+ u32 rx_usecs_orig = priv->rx_coalesce_usecs;
+ int idx;
+
+ if (gve_is_gqi(priv))
+ return -EOPNOTSUPP;
+
+ if (ec->tx_coalesce_usecs > GVE_MAX_ITR_INTERVAL_DQO ||
+ ec->rx_coalesce_usecs > GVE_MAX_ITR_INTERVAL_DQO)
+ return -EINVAL;
+ priv->tx_coalesce_usecs = ec->tx_coalesce_usecs;
+ priv->rx_coalesce_usecs = ec->rx_coalesce_usecs;
+
+ if (tx_usecs_orig != priv->tx_coalesce_usecs) {
+ for (idx = 0; idx < priv->tx_cfg.num_queues; idx++) {
+ int ntfy_idx = gve_tx_idx_to_ntfy(priv, idx);
+ struct gve_notify_block *block = &priv->ntfy_blocks[ntfy_idx];
+
+ gve_set_itr_coalesce_usecs_dqo(priv, block,
+ priv->tx_coalesce_usecs);
+ }
+ }
+
+ if (rx_usecs_orig != priv->rx_coalesce_usecs) {
+ for (idx = 0; idx < priv->rx_cfg.num_queues; idx++) {
+ int ntfy_idx = gve_rx_idx_to_ntfy(priv, idx);
+ struct gve_notify_block *block = &priv->ntfy_blocks[ntfy_idx];
+
+ gve_set_itr_coalesce_usecs_dqo(priv, block,
+ priv->rx_coalesce_usecs);
+ }
+ }
+
+ return 0;
+}
+
const struct ethtool_ops gve_ethtool_ops = {
+ .supported_coalesce_params = ETHTOOL_COALESCE_USECS,
.get_drvinfo = gve_get_drvinfo,
.get_strings = gve_get_strings,
.get_sset_count = gve_get_sset_count,
@@ -545,6 +609,8 @@ const struct ethtool_ops gve_ethtool_ops = {
.set_channels = gve_set_channels,
.get_channels = gve_get_channels,
.get_link = ethtool_op_get_link,
+ .get_coalesce = gve_get_coalesce,
+ .set_coalesce = gve_set_coalesce,
.get_ringparam = gve_get_ringparam,
.reset = gve_user_reset,
.get_tunable = gve_get_tunable,
diff --git a/drivers/net/ethernet/google/gve/gve_main.c b/drivers/net/ethernet/google/gve/gve_main.c
index 59b66f6..f7f65c4 100644
--- a/drivers/net/ethernet/google/gve/gve_main.c
+++ b/drivers/net/ethernet/google/gve/gve_main.c
@@ -334,15 +334,23 @@ static int gve_alloc_notify_blocks(struct gve_priv *priv)
dev_err(&priv->pdev->dev, "Did not receive management vector.\n");
goto abort_with_msix_enabled;
}
- priv->ntfy_blocks =
+ priv->irq_db_indices =
dma_alloc_coherent(&priv->pdev->dev,
priv->num_ntfy_blks *
- sizeof(*priv->ntfy_blocks),
- &priv->ntfy_block_bus, GFP_KERNEL);
- if (!priv->ntfy_blocks) {
+ sizeof(*priv->irq_db_indices),
+ &priv->irq_db_indices_bus, GFP_KERNEL);
+ if (!priv->irq_db_indices) {
err = -ENOMEM;
goto abort_with_mgmt_vector;
}
+
+ priv->ntfy_blocks = kvzalloc(priv->num_ntfy_blks *
+ sizeof(*priv->ntfy_blocks), GFP_KERNEL);
+ if (!priv->ntfy_blocks) {
+ err = -ENOMEM;
+ goto abort_with_irq_db_indices;
+ }
+
/* Setup the other blocks - the first n-1 vectors */
for (i = 0; i < priv->num_ntfy_blks; i++) {
struct gve_notify_block *block = &priv->ntfy_blocks[i];
@@ -361,6 +369,7 @@ static int gve_alloc_notify_blocks(struct gve_priv *priv)
}
irq_set_affinity_hint(priv->msix_vectors[msix_idx].vector,
get_cpu_mask(i % active_cpus));
+ block->irq_db_index = &priv->irq_db_indices[i].index;
}
return 0;
abort_with_some_ntfy_blocks:
@@ -372,10 +381,13 @@ static int gve_alloc_notify_blocks(struct gve_priv *priv)
NULL);
free_irq(priv->msix_vectors[msix_idx].vector, block);
}
- dma_free_coherent(&priv->pdev->dev, priv->num_ntfy_blks *
- sizeof(*priv->ntfy_blocks),
- priv->ntfy_blocks, priv->ntfy_block_bus);
+ kvfree(priv->ntfy_blocks);
priv->ntfy_blocks = NULL;
+abort_with_irq_db_indices:
+ dma_free_coherent(&priv->pdev->dev, priv->num_ntfy_blks *
+ sizeof(*priv->irq_db_indices),
+ priv->irq_db_indices, priv->irq_db_indices_bus);
+ priv->irq_db_indices = NULL;
abort_with_mgmt_vector:
free_irq(priv->msix_vectors[priv->mgmt_msix_idx].vector, priv);
abort_with_msix_enabled:
@@ -403,10 +415,12 @@ static void gve_free_notify_blocks(struct gve_priv *priv)
free_irq(priv->msix_vectors[msix_idx].vector, block);
}
free_irq(priv->msix_vectors[priv->mgmt_msix_idx].vector, priv);
- dma_free_coherent(&priv->pdev->dev,
- priv->num_ntfy_blks * sizeof(*priv->ntfy_blocks),
- priv->ntfy_blocks, priv->ntfy_block_bus);
+ kvfree(priv->ntfy_blocks);
priv->ntfy_blocks = NULL;
+ dma_free_coherent(&priv->pdev->dev, priv->num_ntfy_blks *
+ sizeof(*priv->irq_db_indices),
+ priv->irq_db_indices, priv->irq_db_indices_bus);
+ priv->irq_db_indices = NULL;
pci_disable_msix(priv->pdev);
kvfree(priv->msix_vectors);
priv->msix_vectors = NULL;
@@ -428,7 +442,7 @@ static int gve_setup_device_resources(struct gve_priv *priv)
err = gve_adminq_configure_device_resources(priv,
priv->counter_array_bus,
priv->num_event_counters,
- priv->ntfy_block_bus,
+ priv->irq_db_indices_bus,
priv->num_ntfy_blks);
if (unlikely(err)) {
dev_err(&priv->pdev->dev,
@@ -817,8 +831,7 @@ void gve_free_page(struct device *dev, struct page *page, dma_addr_t dma,
put_page(page);
}
-static void gve_free_queue_page_list(struct gve_priv *priv,
- int id)
+static void gve_free_queue_page_list(struct gve_priv *priv, u32 id)
{
struct gve_queue_page_list *qpl = &priv->qpls[id];
int i;
@@ -1100,9 +1113,8 @@ static void gve_turnup(struct gve_priv *priv)
if (gve_is_gqi(priv)) {
iowrite32be(0, gve_irq_doorbell(priv, block));
} else {
- u32 val = gve_set_itr_ratelimit_dqo(GVE_TX_IRQ_RATELIMIT_US_DQO);
-
- gve_write_irq_doorbell_dqo(priv, block, val);
+ gve_set_itr_coalesce_usecs_dqo(priv, block,
+ priv->tx_coalesce_usecs);
}
}
for (idx = 0; idx < priv->rx_cfg.num_queues; idx++) {
@@ -1113,9 +1125,8 @@ static void gve_turnup(struct gve_priv *priv)
if (gve_is_gqi(priv)) {
iowrite32be(0, gve_irq_doorbell(priv, block));
} else {
- u32 val = gve_set_itr_ratelimit_dqo(GVE_RX_IRQ_RATELIMIT_US_DQO);
-
- gve_write_irq_doorbell_dqo(priv, block, val);
+ gve_set_itr_coalesce_usecs_dqo(priv, block,
+ priv->rx_coalesce_usecs);
}
}
@@ -1412,6 +1423,11 @@ static int gve_init_priv(struct gve_priv *priv, bool skip_describe_device)
dev_info(&priv->pdev->dev, "Max TX queues %d, Max RX queues %d\n",
priv->tx_cfg.max_queues, priv->rx_cfg.max_queues);
+ if (!gve_is_gqi(priv)) {
+ priv->tx_coalesce_usecs = GVE_TX_IRQ_RATELIMIT_US_DQO;
+ priv->rx_coalesce_usecs = GVE_RX_IRQ_RATELIMIT_US_DQO;
+ }
+
setup_device:
err = gve_setup_device_resources(priv);
if (!err)
@@ -1663,6 +1679,58 @@ static void gve_remove(struct pci_dev *pdev)
pci_disable_device(pdev);
}
+static void gve_shutdown(struct pci_dev *pdev)
+{
+ struct net_device *netdev = pci_get_drvdata(pdev);
+ struct gve_priv *priv = netdev_priv(netdev);
+ bool was_up = netif_carrier_ok(priv->dev);
+
+ rtnl_lock();
+ if (was_up && gve_close(priv->dev)) {
+ /* If the dev was up, attempt to close, if close fails, reset */
+ gve_reset_and_teardown(priv, was_up);
+ } else {
+ /* If the dev wasn't up or close worked, finish tearing down */
+ gve_teardown_priv_resources(priv);
+ }
+ rtnl_unlock();
+}
+
+#ifdef CONFIG_PM
+static int gve_suspend(struct pci_dev *pdev, pm_message_t state)
+{
+ struct net_device *netdev = pci_get_drvdata(pdev);
+ struct gve_priv *priv = netdev_priv(netdev);
+ bool was_up = netif_carrier_ok(priv->dev);
+
+ priv->suspend_cnt++;
+ rtnl_lock();
+ if (was_up && gve_close(priv->dev)) {
+ /* If the dev was up, attempt to close, if close fails, reset */
+ gve_reset_and_teardown(priv, was_up);
+ } else {
+ /* If the dev wasn't up or close worked, finish tearing down */
+ gve_teardown_priv_resources(priv);
+ }
+ priv->up_before_suspend = was_up;
+ rtnl_unlock();
+ return 0;
+}
+
+static int gve_resume(struct pci_dev *pdev)
+{
+ struct net_device *netdev = pci_get_drvdata(pdev);
+ struct gve_priv *priv = netdev_priv(netdev);
+ int err;
+
+ priv->resume_cnt++;
+ rtnl_lock();
+ err = gve_reset_recovery(priv, priv->up_before_suspend);
+ rtnl_unlock();
+ return err;
+}
+#endif /* CONFIG_PM */
+
static const struct pci_device_id gve_id_table[] = {
{ PCI_DEVICE(PCI_VENDOR_ID_GOOGLE, PCI_DEV_ID_GVNIC) },
{ }
@@ -1673,6 +1741,11 @@ static struct pci_driver gvnic_driver = {
.id_table = gve_id_table,
.probe = gve_probe,
.remove = gve_remove,
+ .shutdown = gve_shutdown,
+#ifdef CONFIG_PM
+ .suspend = gve_suspend,
+ .resume = gve_resume,
+#endif
};
module_pci_driver(gvnic_driver);
diff --git a/drivers/net/ethernet/google/gve/gve_rx.c b/drivers/net/ethernet/google/gve/gve_rx.c
index 3d04b5a..9ddcc49 100644
--- a/drivers/net/ethernet/google/gve/gve_rx.c
+++ b/drivers/net/ethernet/google/gve/gve_rx.c
@@ -639,8 +639,6 @@ bool gve_rx_work_pending(struct gve_rx_ring *rx)
desc = rx->desc.desc_ring + next_idx;
flags_seq = desc->flags_seq;
- /* Make sure we have synchronized the seq no with the device */
- smp_rmb();
return (GVE_SEQNO(flags_seq) == rx->desc.seqno);
}
diff --git a/drivers/net/ethernet/google/gve/gve_tx.c b/drivers/net/ethernet/google/gve/gve_tx.c
index a9cb241..4888bf0 100644
--- a/drivers/net/ethernet/google/gve/gve_tx.c
+++ b/drivers/net/ethernet/google/gve/gve_tx.c
@@ -296,11 +296,14 @@ static inline int gve_skb_fifo_bytes_required(struct gve_tx_ring *tx,
return bytes;
}
-/* The most descriptors we could need is MAX_SKB_FRAGS + 3 : 1 for each skb frag,
- * +1 for the skb linear portion, +1 for when tcp hdr needs to be in separate descriptor,
- * and +1 if the payload wraps to the beginning of the FIFO.
+/* The most descriptors we could need is MAX_SKB_FRAGS + 4 :
+ * 1 for each skb frag
+ * 1 for the skb linear portion
+ * 1 for when tcp hdr needs to be in separate descriptor
+ * 1 if the payload wraps to the beginning of the FIFO
+ * 1 for metadata descriptor
*/
-#define MAX_TX_DESC_NEEDED (MAX_SKB_FRAGS + 3)
+#define MAX_TX_DESC_NEEDED (MAX_SKB_FRAGS + 4)
static void gve_tx_unmap_buf(struct device *dev, struct gve_tx_buffer_state *info)
{
if (info->skb) {
@@ -395,6 +398,19 @@ static void gve_tx_fill_pkt_desc(union gve_tx_desc *pkt_desc,
pkt_desc->pkt.seg_addr = cpu_to_be64(addr);
}
+static void gve_tx_fill_mtd_desc(union gve_tx_desc *mtd_desc,
+ struct sk_buff *skb)
+{
+ BUILD_BUG_ON(sizeof(mtd_desc->mtd) != sizeof(mtd_desc->pkt));
+
+ mtd_desc->mtd.type_flags = GVE_TXD_MTD | GVE_MTD_SUBTYPE_PATH;
+ mtd_desc->mtd.path_state = GVE_MTD_PATH_STATE_DEFAULT |
+ GVE_MTD_PATH_HASH_L4;
+ mtd_desc->mtd.path_hash = cpu_to_be32(skb->hash);
+ mtd_desc->mtd.reserved0 = 0;
+ mtd_desc->mtd.reserved1 = 0;
+}
+
static void gve_tx_fill_seg_desc(union gve_tx_desc *seg_desc,
struct sk_buff *skb, bool is_gso,
u16 len, u64 addr)
@@ -426,6 +442,7 @@ static int gve_tx_add_skb_copy(struct gve_priv *priv, struct gve_tx_ring *tx, st
int pad_bytes, hlen, hdr_nfrags, payload_nfrags, l4_hdr_offset;
union gve_tx_desc *pkt_desc, *seg_desc;
struct gve_tx_buffer_state *info;
+ int mtd_desc_nr = !!skb->l4_hash;
bool is_gso = skb_is_gso(skb);
u32 idx = tx->req & tx->mask;
int payload_iov = 2;
@@ -457,7 +474,7 @@ static int gve_tx_add_skb_copy(struct gve_priv *priv, struct gve_tx_ring *tx, st
&info->iov[payload_iov]);
gve_tx_fill_pkt_desc(pkt_desc, skb, is_gso, l4_hdr_offset,
- 1 + payload_nfrags, hlen,
+ 1 + mtd_desc_nr + payload_nfrags, hlen,
info->iov[hdr_nfrags - 1].iov_offset);
skb_copy_bits(skb, 0,
@@ -468,8 +485,13 @@ static int gve_tx_add_skb_copy(struct gve_priv *priv, struct gve_tx_ring *tx, st
info->iov[hdr_nfrags - 1].iov_len);
copy_offset = hlen;
+ if (mtd_desc_nr) {
+ next_idx = (tx->req + 1) & tx->mask;
+ gve_tx_fill_mtd_desc(&tx->desc[next_idx], skb);
+ }
+
for (i = payload_iov; i < payload_nfrags + payload_iov; i++) {
- next_idx = (tx->req + 1 + i - payload_iov) & tx->mask;
+ next_idx = (tx->req + 1 + mtd_desc_nr + i - payload_iov) & tx->mask;
seg_desc = &tx->desc[next_idx];
gve_tx_fill_seg_desc(seg_desc, skb, is_gso,
@@ -485,16 +507,17 @@ static int gve_tx_add_skb_copy(struct gve_priv *priv, struct gve_tx_ring *tx, st
copy_offset += info->iov[i].iov_len;
}
- return 1 + payload_nfrags;
+ return 1 + mtd_desc_nr + payload_nfrags;
}
static int gve_tx_add_skb_no_copy(struct gve_priv *priv, struct gve_tx_ring *tx,
struct sk_buff *skb)
{
const struct skb_shared_info *shinfo = skb_shinfo(skb);
- int hlen, payload_nfrags, l4_hdr_offset;
- union gve_tx_desc *pkt_desc, *seg_desc;
+ int hlen, num_descriptors, l4_hdr_offset;
+ union gve_tx_desc *pkt_desc, *mtd_desc, *seg_desc;
struct gve_tx_buffer_state *info;
+ int mtd_desc_nr = !!skb->l4_hash;
bool is_gso = skb_is_gso(skb);
u32 idx = tx->req & tx->mask;
u64 addr;
@@ -523,23 +546,30 @@ static int gve_tx_add_skb_no_copy(struct gve_priv *priv, struct gve_tx_ring *tx,
dma_unmap_len_set(info, len, len);
dma_unmap_addr_set(info, dma, addr);
- payload_nfrags = shinfo->nr_frags;
+ num_descriptors = 1 + shinfo->nr_frags;
+ if (hlen < len)
+ num_descriptors++;
+ if (mtd_desc_nr)
+ num_descriptors++;
+
+ gve_tx_fill_pkt_desc(pkt_desc, skb, is_gso, l4_hdr_offset,
+ num_descriptors, hlen, addr);
+
+ if (mtd_desc_nr) {
+ idx = (idx + 1) & tx->mask;
+ mtd_desc = &tx->desc[idx];
+ gve_tx_fill_mtd_desc(mtd_desc, skb);
+ }
+
if (hlen < len) {
/* For gso the rest of the linear portion of the skb needs to
* be in its own descriptor.
*/
- payload_nfrags++;
- gve_tx_fill_pkt_desc(pkt_desc, skb, is_gso, l4_hdr_offset,
- 1 + payload_nfrags, hlen, addr);
-
len -= hlen;
addr += hlen;
- idx = (tx->req + 1) & tx->mask;
+ idx = (idx + 1) & tx->mask;
seg_desc = &tx->desc[idx];
gve_tx_fill_seg_desc(seg_desc, skb, is_gso, len, addr);
- } else {
- gve_tx_fill_pkt_desc(pkt_desc, skb, is_gso, l4_hdr_offset,
- 1 + payload_nfrags, hlen, addr);
}
for (i = 0; i < shinfo->nr_frags; i++) {
@@ -560,11 +590,14 @@ static int gve_tx_add_skb_no_copy(struct gve_priv *priv, struct gve_tx_ring *tx,
gve_tx_fill_seg_desc(seg_desc, skb, is_gso, len, addr);
}
- return 1 + payload_nfrags;
+ return num_descriptors;
unmap_drop:
- i += (payload_nfrags == shinfo->nr_frags ? 1 : 2);
+ i += num_descriptors - shinfo->nr_frags;
while (i--) {
+ /* Skip metadata descriptor, if set */
+ if (i == 1 && mtd_desc_nr == 1)
+ continue;
idx--;
gve_tx_unmap_buf(tx->dev, &tx->info[idx & tx->mask]);
}
diff --git a/drivers/net/ethernet/hisilicon/hns/hns_ethtool.c b/drivers/net/ethernet/hisilicon/hns/hns_ethtool.c
index ab73902..d7a27c2 100644
--- a/drivers/net/ethernet/hisilicon/hns/hns_ethtool.c
+++ b/drivers/net/ethernet/hisilicon/hns/hns_ethtool.c
@@ -663,9 +663,13 @@ static void hns_nic_get_drvinfo(struct net_device *net_dev,
* hns_get_ringparam - get ring parameter
* @net_dev: net device
* @param: ethtool parameter
+ * @kernel_param: ethtool external parameter
+ * @extack: netlink extended ACK report struct
*/
static void hns_get_ringparam(struct net_device *net_dev,
- struct ethtool_ringparam *param)
+ struct ethtool_ringparam *param,
+ struct kernel_ethtool_ringparam *kernel_param,
+ struct netlink_ext_ack *extack)
{
struct hns_nic_priv *priv = netdev_priv(net_dev);
struct hnae_ae_ops *ops;
diff --git a/drivers/net/ethernet/hisilicon/hns3/Makefile b/drivers/net/ethernet/hisilicon/hns3/Makefile
index 7aa2fac..6efea46 100644
--- a/drivers/net/ethernet/hisilicon/hns3/Makefile
+++ b/drivers/net/ethernet/hisilicon/hns3/Makefile
@@ -4,9 +4,9 @@
#
ccflags-y += -I$(srctree)/$(src)
-
-obj-$(CONFIG_HNS3) += hns3pf/
-obj-$(CONFIG_HNS3) += hns3vf/
+ccflags-y += -I$(srctree)/drivers/net/ethernet/hisilicon/hns3/hns3pf
+ccflags-y += -I$(srctree)/drivers/net/ethernet/hisilicon/hns3/hns3vf
+ccflags-y += -I$(srctree)/drivers/net/ethernet/hisilicon/hns3/hns3_common
obj-$(CONFIG_HNS3) += hnae3.o
@@ -14,3 +14,16 @@
hns3-objs = hns3_enet.o hns3_ethtool.o hns3_debugfs.o
hns3-$(CONFIG_HNS3_DCB) += hns3_dcbnl.o
+
+obj-$(CONFIG_HNS3_HCLGEVF) += hclgevf.o
+
+hclgevf-objs = hns3vf/hclgevf_main.o hns3vf/hclgevf_mbx.o hns3vf/hclgevf_devlink.o \
+ hns3_common/hclge_comm_cmd.o hns3_common/hclge_comm_rss.o hns3_common/hclge_comm_tqp_stats.o
+
+obj-$(CONFIG_HNS3_HCLGE) += hclge.o
+hclge-objs = hns3pf/hclge_main.o hns3pf/hclge_mdio.o hns3pf/hclge_tm.o \
+ hns3pf/hclge_mbx.o hns3pf/hclge_err.o hns3pf/hclge_debugfs.o hns3pf/hclge_ptp.o hns3pf/hclge_devlink.o \
+ hns3_common/hclge_comm_cmd.o hns3_common/hclge_comm_rss.o hns3_common/hclge_comm_tqp_stats.o
+
+
+hclge-$(CONFIG_HNS3_DCB) += hns3pf/hclge_dcb.o
diff --git a/drivers/net/ethernet/hisilicon/hns3/hclge_mbx.h b/drivers/net/ethernet/hisilicon/hns3/hclge_mbx.h
index c2bd258..b668df6 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hclge_mbx.h
+++ b/drivers/net/ethernet/hisilicon/hns3/hclge_mbx.h
@@ -80,6 +80,9 @@ enum hclge_mbx_tbl_cfg_subcode {
#define HCLGE_MBX_MAX_RESP_DATA_SIZE 8U
#define HCLGE_MBX_MAX_RING_CHAIN_PARAM_NUM 4
+#define HCLGE_RESET_SCHED_TIMEOUT (3 * HZ)
+#define HCLGE_MBX_SCHED_TIMEOUT (HZ / 2)
+
struct hclge_ring_chain_param {
u8 ring_type;
u8 tqp_index;
diff --git a/drivers/net/ethernet/hisilicon/hns3/hnae3.h b/drivers/net/ethernet/hisilicon/hns3/hnae3.h
index 63f5abc..9298fbe 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hnae3.h
+++ b/drivers/net/ethernet/hisilicon/hns3/hnae3.h
@@ -861,6 +861,20 @@ struct hnae3_handle {
#define hnae3_get_bit(origin, shift) \
hnae3_get_field(origin, 0x1 << (shift), shift)
+#define HNAE3_FORMAT_MAC_ADDR_LEN 18
+#define HNAE3_FORMAT_MAC_ADDR_OFFSET_0 0
+#define HNAE3_FORMAT_MAC_ADDR_OFFSET_4 4
+#define HNAE3_FORMAT_MAC_ADDR_OFFSET_5 5
+
+static inline void hnae3_format_mac_addr(char *format_mac_addr,
+ const u8 *mac_addr)
+{
+ snprintf(format_mac_addr, HNAE3_FORMAT_MAC_ADDR_LEN, "%02x:**:**:**:%02x:%02x",
+ mac_addr[HNAE3_FORMAT_MAC_ADDR_OFFSET_0],
+ mac_addr[HNAE3_FORMAT_MAC_ADDR_OFFSET_4],
+ mac_addr[HNAE3_FORMAT_MAC_ADDR_OFFSET_5]);
+}
+
int hnae3_register_ae_dev(struct hnae3_ae_dev *ae_dev);
void hnae3_unregister_ae_dev(struct hnae3_ae_dev *ae_dev);
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_cmd.c b/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_cmd.c
new file mode 100644
index 0000000..c15ca71
--- /dev/null
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_cmd.c
@@ -0,0 +1,610 @@
+// SPDX-License-Identifier: GPL-2.0+
+// Copyright (c) 2021-2021 Hisilicon Limited.
+
+#include "hnae3.h"
+#include "hclge_comm_cmd.h"
+
+static void hclge_comm_cmd_config_regs(struct hclge_comm_hw *hw,
+ struct hclge_comm_cmq_ring *ring)
+{
+ dma_addr_t dma = ring->desc_dma_addr;
+ u32 reg_val;
+
+ if (ring->ring_type == HCLGE_COMM_TYPE_CSQ) {
+ hclge_comm_write_dev(hw, HCLGE_COMM_NIC_CSQ_BASEADDR_L_REG,
+ lower_32_bits(dma));
+ hclge_comm_write_dev(hw, HCLGE_COMM_NIC_CSQ_BASEADDR_H_REG,
+ upper_32_bits(dma));
+ reg_val = hclge_comm_read_dev(hw, HCLGE_COMM_NIC_CSQ_DEPTH_REG);
+ reg_val &= HCLGE_COMM_NIC_SW_RST_RDY;
+ reg_val |= ring->desc_num >> HCLGE_COMM_NIC_CMQ_DESC_NUM_S;
+ hclge_comm_write_dev(hw, HCLGE_COMM_NIC_CSQ_DEPTH_REG, reg_val);
+ hclge_comm_write_dev(hw, HCLGE_COMM_NIC_CSQ_HEAD_REG, 0);
+ hclge_comm_write_dev(hw, HCLGE_COMM_NIC_CSQ_TAIL_REG, 0);
+ } else {
+ hclge_comm_write_dev(hw, HCLGE_COMM_NIC_CRQ_BASEADDR_L_REG,
+ lower_32_bits(dma));
+ hclge_comm_write_dev(hw, HCLGE_COMM_NIC_CRQ_BASEADDR_H_REG,
+ upper_32_bits(dma));
+ reg_val = ring->desc_num >> HCLGE_COMM_NIC_CMQ_DESC_NUM_S;
+ hclge_comm_write_dev(hw, HCLGE_COMM_NIC_CRQ_DEPTH_REG, reg_val);
+ hclge_comm_write_dev(hw, HCLGE_COMM_NIC_CRQ_HEAD_REG, 0);
+ hclge_comm_write_dev(hw, HCLGE_COMM_NIC_CRQ_TAIL_REG, 0);
+ }
+}
+
+void hclge_comm_cmd_init_regs(struct hclge_comm_hw *hw)
+{
+ hclge_comm_cmd_config_regs(hw, &hw->cmq.csq);
+ hclge_comm_cmd_config_regs(hw, &hw->cmq.crq);
+}
+
+void hclge_comm_cmd_reuse_desc(struct hclge_desc *desc, bool is_read)
+{
+ desc->flag = cpu_to_le16(HCLGE_COMM_CMD_FLAG_NO_INTR |
+ HCLGE_COMM_CMD_FLAG_IN);
+ if (is_read)
+ desc->flag |= cpu_to_le16(HCLGE_COMM_CMD_FLAG_WR);
+ else
+ desc->flag &= cpu_to_le16(~HCLGE_COMM_CMD_FLAG_WR);
+}
+
+static void hclge_comm_set_default_capability(struct hnae3_ae_dev *ae_dev,
+ bool is_pf)
+{
+ set_bit(HNAE3_DEV_SUPPORT_FD_B, ae_dev->caps);
+ set_bit(HNAE3_DEV_SUPPORT_GRO_B, ae_dev->caps);
+ if (is_pf && ae_dev->dev_version == HNAE3_DEVICE_VERSION_V2) {
+ set_bit(HNAE3_DEV_SUPPORT_FEC_B, ae_dev->caps);
+ set_bit(HNAE3_DEV_SUPPORT_PAUSE_B, ae_dev->caps);
+ }
+}
+
+void hclge_comm_cmd_setup_basic_desc(struct hclge_desc *desc,
+ enum hclge_opcode_type opcode,
+ bool is_read)
+{
+ memset((void *)desc, 0, sizeof(struct hclge_desc));
+ desc->opcode = cpu_to_le16(opcode);
+ desc->flag = cpu_to_le16(HCLGE_COMM_CMD_FLAG_NO_INTR |
+ HCLGE_COMM_CMD_FLAG_IN);
+
+ if (is_read)
+ desc->flag |= cpu_to_le16(HCLGE_COMM_CMD_FLAG_WR);
+}
+
+int hclge_comm_firmware_compat_config(struct hnae3_ae_dev *ae_dev,
+ struct hclge_comm_hw *hw, bool en)
+{
+ struct hclge_comm_firmware_compat_cmd *req;
+ struct hclge_desc desc;
+ u32 compat = 0;
+
+ hclge_comm_cmd_setup_basic_desc(&desc, HCLGE_OPC_IMP_COMPAT_CFG, false);
+
+ if (en) {
+ req = (struct hclge_comm_firmware_compat_cmd *)desc.data;
+
+ hnae3_set_bit(compat, HCLGE_COMM_LINK_EVENT_REPORT_EN_B, 1);
+ hnae3_set_bit(compat, HCLGE_COMM_NCSI_ERROR_REPORT_EN_B, 1);
+ if (hclge_comm_dev_phy_imp_supported(ae_dev))
+ hnae3_set_bit(compat, HCLGE_COMM_PHY_IMP_EN_B, 1);
+ hnae3_set_bit(compat, HCLGE_COMM_MAC_STATS_EXT_EN_B, 1);
+ hnae3_set_bit(compat, HCLGE_COMM_SYNC_RX_RING_HEAD_EN_B, 1);
+
+ req->compat = cpu_to_le32(compat);
+ }
+
+ return hclge_comm_cmd_send(hw, &desc, 1);
+}
+
+void hclge_comm_free_cmd_desc(struct hclge_comm_cmq_ring *ring)
+{
+ int size = ring->desc_num * sizeof(struct hclge_desc);
+
+ if (!ring->desc)
+ return;
+
+ dma_free_coherent(&ring->pdev->dev, size,
+ ring->desc, ring->desc_dma_addr);
+ ring->desc = NULL;
+}
+
+static int hclge_comm_alloc_cmd_desc(struct hclge_comm_cmq_ring *ring)
+{
+ int size = ring->desc_num * sizeof(struct hclge_desc);
+
+ ring->desc = dma_alloc_coherent(&ring->pdev->dev,
+ size, &ring->desc_dma_addr, GFP_KERNEL);
+ if (!ring->desc)
+ return -ENOMEM;
+
+ return 0;
+}
+
+static __le32 hclge_comm_build_api_caps(void)
+{
+ u32 api_caps = 0;
+
+ hnae3_set_bit(api_caps, HCLGE_COMM_API_CAP_FLEX_RSS_TBL_B, 1);
+
+ return cpu_to_le32(api_caps);
+}
+
+static const struct hclge_comm_caps_bit_map hclge_pf_cmd_caps[] = {
+ {HCLGE_COMM_CAP_UDP_GSO_B, HNAE3_DEV_SUPPORT_UDP_GSO_B},
+ {HCLGE_COMM_CAP_PTP_B, HNAE3_DEV_SUPPORT_PTP_B},
+ {HCLGE_COMM_CAP_INT_QL_B, HNAE3_DEV_SUPPORT_INT_QL_B},
+ {HCLGE_COMM_CAP_TQP_TXRX_INDEP_B, HNAE3_DEV_SUPPORT_TQP_TXRX_INDEP_B},
+ {HCLGE_COMM_CAP_HW_TX_CSUM_B, HNAE3_DEV_SUPPORT_HW_TX_CSUM_B},
+ {HCLGE_COMM_CAP_UDP_TUNNEL_CSUM_B, HNAE3_DEV_SUPPORT_UDP_TUNNEL_CSUM_B},
+ {HCLGE_COMM_CAP_FD_FORWARD_TC_B, HNAE3_DEV_SUPPORT_FD_FORWARD_TC_B},
+ {HCLGE_COMM_CAP_FEC_B, HNAE3_DEV_SUPPORT_FEC_B},
+ {HCLGE_COMM_CAP_PAUSE_B, HNAE3_DEV_SUPPORT_PAUSE_B},
+ {HCLGE_COMM_CAP_PHY_IMP_B, HNAE3_DEV_SUPPORT_PHY_IMP_B},
+ {HCLGE_COMM_CAP_QB_B, HNAE3_DEV_SUPPORT_QB_B},
+ {HCLGE_COMM_CAP_TX_PUSH_B, HNAE3_DEV_SUPPORT_TX_PUSH_B},
+ {HCLGE_COMM_CAP_RAS_IMP_B, HNAE3_DEV_SUPPORT_RAS_IMP_B},
+ {HCLGE_COMM_CAP_RXD_ADV_LAYOUT_B, HNAE3_DEV_SUPPORT_RXD_ADV_LAYOUT_B},
+ {HCLGE_COMM_CAP_PORT_VLAN_BYPASS_B,
+ HNAE3_DEV_SUPPORT_PORT_VLAN_BYPASS_B},
+ {HCLGE_COMM_CAP_PORT_VLAN_BYPASS_B, HNAE3_DEV_SUPPORT_VLAN_FLTR_MDF_B},
+};
+
+static const struct hclge_comm_caps_bit_map hclge_vf_cmd_caps[] = {
+ {HCLGE_COMM_CAP_UDP_GSO_B, HNAE3_DEV_SUPPORT_UDP_GSO_B},
+ {HCLGE_COMM_CAP_INT_QL_B, HNAE3_DEV_SUPPORT_INT_QL_B},
+ {HCLGE_COMM_CAP_TQP_TXRX_INDEP_B, HNAE3_DEV_SUPPORT_TQP_TXRX_INDEP_B},
+ {HCLGE_COMM_CAP_HW_TX_CSUM_B, HNAE3_DEV_SUPPORT_HW_TX_CSUM_B},
+ {HCLGE_COMM_CAP_UDP_TUNNEL_CSUM_B, HNAE3_DEV_SUPPORT_UDP_TUNNEL_CSUM_B},
+ {HCLGE_COMM_CAP_QB_B, HNAE3_DEV_SUPPORT_QB_B},
+ {HCLGE_COMM_CAP_TX_PUSH_B, HNAE3_DEV_SUPPORT_TX_PUSH_B},
+ {HCLGE_COMM_CAP_RXD_ADV_LAYOUT_B, HNAE3_DEV_SUPPORT_RXD_ADV_LAYOUT_B},
+};
+
+static void
+hclge_comm_parse_capability(struct hnae3_ae_dev *ae_dev, bool is_pf,
+ struct hclge_comm_query_version_cmd *cmd)
+{
+ const struct hclge_comm_caps_bit_map *caps_map =
+ is_pf ? hclge_pf_cmd_caps : hclge_vf_cmd_caps;
+ u32 size = is_pf ? ARRAY_SIZE(hclge_pf_cmd_caps) :
+ ARRAY_SIZE(hclge_vf_cmd_caps);
+ u32 caps, i;
+
+ caps = __le32_to_cpu(cmd->caps[0]);
+ for (i = 0; i < size; i++)
+ if (hnae3_get_bit(caps, caps_map[i].imp_bit))
+ set_bit(caps_map[i].local_bit, ae_dev->caps);
+}
+
+int hclge_comm_alloc_cmd_queue(struct hclge_comm_hw *hw, int ring_type)
+{
+ struct hclge_comm_cmq_ring *ring =
+ (ring_type == HCLGE_COMM_TYPE_CSQ) ? &hw->cmq.csq :
+ &hw->cmq.crq;
+ int ret;
+
+ ring->ring_type = ring_type;
+
+ ret = hclge_comm_alloc_cmd_desc(ring);
+ if (ret)
+ dev_err(&ring->pdev->dev, "descriptor %s alloc error %d\n",
+ (ring_type == HCLGE_COMM_TYPE_CSQ) ? "CSQ" : "CRQ",
+ ret);
+
+ return ret;
+}
+
+int hclge_comm_cmd_query_version_and_capability(struct hnae3_ae_dev *ae_dev,
+ struct hclge_comm_hw *hw,
+ u32 *fw_version, bool is_pf)
+{
+ struct hclge_comm_query_version_cmd *resp;
+ struct hclge_desc desc;
+ int ret;
+
+ hclge_comm_cmd_setup_basic_desc(&desc, HCLGE_OPC_QUERY_FW_VER, 1);
+ resp = (struct hclge_comm_query_version_cmd *)desc.data;
+ resp->api_caps = hclge_comm_build_api_caps();
+
+ ret = hclge_comm_cmd_send(hw, &desc, 1);
+ if (ret)
+ return ret;
+
+ *fw_version = le32_to_cpu(resp->firmware);
+
+ ae_dev->dev_version = le32_to_cpu(resp->hardware) <<
+ HNAE3_PCI_REVISION_BIT_SIZE;
+ ae_dev->dev_version |= ae_dev->pdev->revision;
+
+ if (ae_dev->dev_version >= HNAE3_DEVICE_VERSION_V2)
+ hclge_comm_set_default_capability(ae_dev, is_pf);
+
+ hclge_comm_parse_capability(ae_dev, is_pf, resp);
+
+ return ret;
+}
+
+static const u16 spec_opcode[] = { HCLGE_OPC_STATS_64_BIT,
+ HCLGE_OPC_STATS_32_BIT,
+ HCLGE_OPC_STATS_MAC,
+ HCLGE_OPC_STATS_MAC_ALL,
+ HCLGE_OPC_QUERY_32_BIT_REG,
+ HCLGE_OPC_QUERY_64_BIT_REG,
+ HCLGE_QUERY_CLEAR_MPF_RAS_INT,
+ HCLGE_QUERY_CLEAR_PF_RAS_INT,
+ HCLGE_QUERY_CLEAR_ALL_MPF_MSIX_INT,
+ HCLGE_QUERY_CLEAR_ALL_PF_MSIX_INT,
+ HCLGE_QUERY_ALL_ERR_INFO };
+
+static bool hclge_comm_is_special_opcode(u16 opcode)
+{
+ /* these commands have several descriptors,
+ * and use the first one to save opcode and return value
+ */
+ u32 i;
+
+ for (i = 0; i < ARRAY_SIZE(spec_opcode); i++)
+ if (spec_opcode[i] == opcode)
+ return true;
+
+ return false;
+}
+
+static int hclge_comm_ring_space(struct hclge_comm_cmq_ring *ring)
+{
+ int ntc = ring->next_to_clean;
+ int ntu = ring->next_to_use;
+ int used = (ntu - ntc + ring->desc_num) % ring->desc_num;
+
+ return ring->desc_num - used - 1;
+}
+
+static void hclge_comm_cmd_copy_desc(struct hclge_comm_hw *hw,
+ struct hclge_desc *desc, int num)
+{
+ struct hclge_desc *desc_to_use;
+ int handle = 0;
+
+ while (handle < num) {
+ desc_to_use = &hw->cmq.csq.desc[hw->cmq.csq.next_to_use];
+ *desc_to_use = desc[handle];
+ (hw->cmq.csq.next_to_use)++;
+ if (hw->cmq.csq.next_to_use >= hw->cmq.csq.desc_num)
+ hw->cmq.csq.next_to_use = 0;
+ handle++;
+ }
+}
+
+static int hclge_comm_is_valid_csq_clean_head(struct hclge_comm_cmq_ring *ring,
+ int head)
+{
+ int ntc = ring->next_to_clean;
+ int ntu = ring->next_to_use;
+
+ if (ntu > ntc)
+ return head >= ntc && head <= ntu;
+
+ return head >= ntc || head <= ntu;
+}
+
+static int hclge_comm_cmd_csq_clean(struct hclge_comm_hw *hw)
+{
+ struct hclge_comm_cmq_ring *csq = &hw->cmq.csq;
+ int clean;
+ u32 head;
+
+ head = hclge_comm_read_dev(hw, HCLGE_COMM_NIC_CSQ_HEAD_REG);
+ rmb(); /* Make sure head is ready before touch any data */
+
+ if (!hclge_comm_is_valid_csq_clean_head(csq, head)) {
+ dev_warn(&hw->cmq.csq.pdev->dev, "wrong cmd head (%u, %d-%d)\n",
+ head, csq->next_to_use, csq->next_to_clean);
+ dev_warn(&hw->cmq.csq.pdev->dev,
+ "Disabling any further commands to IMP firmware\n");
+ set_bit(HCLGE_COMM_STATE_CMD_DISABLE, &hw->comm_state);
+ dev_warn(&hw->cmq.csq.pdev->dev,
+ "IMP firmware watchdog reset soon expected!\n");
+ return -EIO;
+ }
+
+ clean = (head - csq->next_to_clean + csq->desc_num) % csq->desc_num;
+ csq->next_to_clean = head;
+ return clean;
+}
+
+static int hclge_comm_cmd_csq_done(struct hclge_comm_hw *hw)
+{
+ u32 head = hclge_comm_read_dev(hw, HCLGE_COMM_NIC_CSQ_HEAD_REG);
+ return head == hw->cmq.csq.next_to_use;
+}
+
+static void hclge_comm_wait_for_resp(struct hclge_comm_hw *hw,
+ bool *is_completed)
+{
+ u32 timeout = 0;
+
+ do {
+ if (hclge_comm_cmd_csq_done(hw)) {
+ *is_completed = true;
+ break;
+ }
+ udelay(1);
+ timeout++;
+ } while (timeout < hw->cmq.tx_timeout);
+}
+
+static int hclge_comm_cmd_convert_err_code(u16 desc_ret)
+{
+ struct hclge_comm_errcode hclge_comm_cmd_errcode[] = {
+ { HCLGE_COMM_CMD_EXEC_SUCCESS, 0 },
+ { HCLGE_COMM_CMD_NO_AUTH, -EPERM },
+ { HCLGE_COMM_CMD_NOT_SUPPORTED, -EOPNOTSUPP },
+ { HCLGE_COMM_CMD_QUEUE_FULL, -EXFULL },
+ { HCLGE_COMM_CMD_NEXT_ERR, -ENOSR },
+ { HCLGE_COMM_CMD_UNEXE_ERR, -ENOTBLK },
+ { HCLGE_COMM_CMD_PARA_ERR, -EINVAL },
+ { HCLGE_COMM_CMD_RESULT_ERR, -ERANGE },
+ { HCLGE_COMM_CMD_TIMEOUT, -ETIME },
+ { HCLGE_COMM_CMD_HILINK_ERR, -ENOLINK },
+ { HCLGE_COMM_CMD_QUEUE_ILLEGAL, -ENXIO },
+ { HCLGE_COMM_CMD_INVALID, -EBADR },
+ };
+ u32 errcode_count = ARRAY_SIZE(hclge_comm_cmd_errcode);
+ u32 i;
+
+ for (i = 0; i < errcode_count; i++)
+ if (hclge_comm_cmd_errcode[i].imp_errcode == desc_ret)
+ return hclge_comm_cmd_errcode[i].common_errno;
+
+ return -EIO;
+}
+
+static int hclge_comm_cmd_check_retval(struct hclge_comm_hw *hw,
+ struct hclge_desc *desc, int num,
+ int ntc)
+{
+ u16 opcode, desc_ret;
+ int handle;
+
+ opcode = le16_to_cpu(desc[0].opcode);
+ for (handle = 0; handle < num; handle++) {
+ desc[handle] = hw->cmq.csq.desc[ntc];
+ ntc++;
+ if (ntc >= hw->cmq.csq.desc_num)
+ ntc = 0;
+ }
+ if (likely(!hclge_comm_is_special_opcode(opcode)))
+ desc_ret = le16_to_cpu(desc[num - 1].retval);
+ else
+ desc_ret = le16_to_cpu(desc[0].retval);
+
+ hw->cmq.last_status = desc_ret;
+
+ return hclge_comm_cmd_convert_err_code(desc_ret);
+}
+
+static int hclge_comm_cmd_check_result(struct hclge_comm_hw *hw,
+ struct hclge_desc *desc,
+ int num, int ntc)
+{
+ bool is_completed = false;
+ int handle, ret;
+
+ /* If the command is sync, wait for the firmware to write back,
+ * if multi descriptors to be sent, use the first one to check
+ */
+ if (HCLGE_COMM_SEND_SYNC(le16_to_cpu(desc->flag)))
+ hclge_comm_wait_for_resp(hw, &is_completed);
+
+ if (!is_completed)
+ ret = -EBADE;
+ else
+ ret = hclge_comm_cmd_check_retval(hw, desc, num, ntc);
+
+ /* Clean the command send queue */
+ handle = hclge_comm_cmd_csq_clean(hw);
+ if (handle < 0)
+ ret = handle;
+ else if (handle != num)
+ dev_warn(&hw->cmq.csq.pdev->dev,
+ "cleaned %d, need to clean %d\n", handle, num);
+ return ret;
+}
+
+/**
+ * hclge_comm_cmd_send - send command to command queue
+ * @hw: pointer to the hw struct
+ * @desc: prefilled descriptor for describing the command
+ * @num : the number of descriptors to be sent
+ *
+ * This is the main send command for command queue, it
+ * sends the queue, cleans the queue, etc
+ **/
+int hclge_comm_cmd_send(struct hclge_comm_hw *hw, struct hclge_desc *desc,
+ int num)
+{
+ struct hclge_comm_cmq_ring *csq = &hw->cmq.csq;
+ int ret;
+ int ntc;
+
+ spin_lock_bh(&hw->cmq.csq.lock);
+
+ if (test_bit(HCLGE_COMM_STATE_CMD_DISABLE, &hw->comm_state)) {
+ spin_unlock_bh(&hw->cmq.csq.lock);
+ return -EBUSY;
+ }
+
+ if (num > hclge_comm_ring_space(&hw->cmq.csq)) {
+ /* If CMDQ ring is full, SW HEAD and HW HEAD may be different,
+ * need update the SW HEAD pointer csq->next_to_clean
+ */
+ csq->next_to_clean =
+ hclge_comm_read_dev(hw, HCLGE_COMM_NIC_CSQ_HEAD_REG);
+ spin_unlock_bh(&hw->cmq.csq.lock);
+ return -EBUSY;
+ }
+
+ /**
+ * Record the location of desc in the ring for this time
+ * which will be use for hardware to write back
+ */
+ ntc = hw->cmq.csq.next_to_use;
+
+ hclge_comm_cmd_copy_desc(hw, desc, num);
+
+ /* Write to hardware */
+ hclge_comm_write_dev(hw, HCLGE_COMM_NIC_CSQ_TAIL_REG,
+ hw->cmq.csq.next_to_use);
+
+ ret = hclge_comm_cmd_check_result(hw, desc, num, ntc);
+
+ spin_unlock_bh(&hw->cmq.csq.lock);
+
+ return ret;
+}
+
+static void hclge_comm_cmd_uninit_regs(struct hclge_comm_hw *hw)
+{
+ hclge_comm_write_dev(hw, HCLGE_COMM_NIC_CSQ_BASEADDR_L_REG, 0);
+ hclge_comm_write_dev(hw, HCLGE_COMM_NIC_CSQ_BASEADDR_H_REG, 0);
+ hclge_comm_write_dev(hw, HCLGE_COMM_NIC_CSQ_DEPTH_REG, 0);
+ hclge_comm_write_dev(hw, HCLGE_COMM_NIC_CSQ_HEAD_REG, 0);
+ hclge_comm_write_dev(hw, HCLGE_COMM_NIC_CSQ_TAIL_REG, 0);
+ hclge_comm_write_dev(hw, HCLGE_COMM_NIC_CRQ_BASEADDR_L_REG, 0);
+ hclge_comm_write_dev(hw, HCLGE_COMM_NIC_CRQ_BASEADDR_H_REG, 0);
+ hclge_comm_write_dev(hw, HCLGE_COMM_NIC_CRQ_DEPTH_REG, 0);
+ hclge_comm_write_dev(hw, HCLGE_COMM_NIC_CRQ_HEAD_REG, 0);
+ hclge_comm_write_dev(hw, HCLGE_COMM_NIC_CRQ_TAIL_REG, 0);
+}
+
+void hclge_comm_cmd_uninit(struct hnae3_ae_dev *ae_dev,
+ struct hclge_comm_hw *hw)
+{
+ struct hclge_comm_cmq *cmdq = &hw->cmq;
+
+ hclge_comm_firmware_compat_config(ae_dev, hw, false);
+ set_bit(HCLGE_COMM_STATE_CMD_DISABLE, &hw->comm_state);
+
+ /* wait to ensure that the firmware completes the possible left
+ * over commands.
+ */
+ msleep(HCLGE_COMM_CMDQ_CLEAR_WAIT_TIME);
+ spin_lock_bh(&cmdq->csq.lock);
+ spin_lock(&cmdq->crq.lock);
+ hclge_comm_cmd_uninit_regs(hw);
+ spin_unlock(&cmdq->crq.lock);
+ spin_unlock_bh(&cmdq->csq.lock);
+
+ hclge_comm_free_cmd_desc(&cmdq->csq);
+ hclge_comm_free_cmd_desc(&cmdq->crq);
+}
+
+int hclge_comm_cmd_queue_init(struct pci_dev *pdev, struct hclge_comm_hw *hw)
+{
+ struct hclge_comm_cmq *cmdq = &hw->cmq;
+ int ret;
+
+ /* Setup the lock for command queue */
+ spin_lock_init(&cmdq->csq.lock);
+ spin_lock_init(&cmdq->crq.lock);
+
+ cmdq->csq.pdev = pdev;
+ cmdq->crq.pdev = pdev;
+
+ /* Setup the queue entries for use cmd queue */
+ cmdq->csq.desc_num = HCLGE_COMM_NIC_CMQ_DESC_NUM;
+ cmdq->crq.desc_num = HCLGE_COMM_NIC_CMQ_DESC_NUM;
+
+ /* Setup Tx write back timeout */
+ cmdq->tx_timeout = HCLGE_COMM_CMDQ_TX_TIMEOUT;
+
+ /* Setup queue rings */
+ ret = hclge_comm_alloc_cmd_queue(hw, HCLGE_COMM_TYPE_CSQ);
+ if (ret) {
+ dev_err(&pdev->dev, "CSQ ring setup error %d\n", ret);
+ return ret;
+ }
+
+ ret = hclge_comm_alloc_cmd_queue(hw, HCLGE_COMM_TYPE_CRQ);
+ if (ret) {
+ dev_err(&pdev->dev, "CRQ ring setup error %d\n", ret);
+ goto err_csq;
+ }
+
+ return 0;
+err_csq:
+ hclge_comm_free_cmd_desc(&hw->cmq.csq);
+ return ret;
+}
+
+int hclge_comm_cmd_init(struct hnae3_ae_dev *ae_dev, struct hclge_comm_hw *hw,
+ u32 *fw_version, bool is_pf,
+ unsigned long reset_pending)
+{
+ struct hclge_comm_cmq *cmdq = &hw->cmq;
+ int ret;
+
+ spin_lock_bh(&cmdq->csq.lock);
+ spin_lock(&cmdq->crq.lock);
+
+ cmdq->csq.next_to_clean = 0;
+ cmdq->csq.next_to_use = 0;
+ cmdq->crq.next_to_clean = 0;
+ cmdq->crq.next_to_use = 0;
+
+ hclge_comm_cmd_init_regs(hw);
+
+ spin_unlock(&cmdq->crq.lock);
+ spin_unlock_bh(&cmdq->csq.lock);
+
+ clear_bit(HCLGE_COMM_STATE_CMD_DISABLE, &hw->comm_state);
+
+ /* Check if there is new reset pending, because the higher level
+ * reset may happen when lower level reset is being processed.
+ */
+ if (reset_pending) {
+ ret = -EBUSY;
+ goto err_cmd_init;
+ }
+
+ /* get version and device capabilities */
+ ret = hclge_comm_cmd_query_version_and_capability(ae_dev, hw,
+ fw_version, is_pf);
+ if (ret) {
+ dev_err(&ae_dev->pdev->dev,
+ "failed to query version and capabilities, ret = %d\n",
+ ret);
+ goto err_cmd_init;
+ }
+
+ dev_info(&ae_dev->pdev->dev,
+ "The firmware version is %lu.%lu.%lu.%lu\n",
+ hnae3_get_field(*fw_version, HNAE3_FW_VERSION_BYTE3_MASK,
+ HNAE3_FW_VERSION_BYTE3_SHIFT),
+ hnae3_get_field(*fw_version, HNAE3_FW_VERSION_BYTE2_MASK,
+ HNAE3_FW_VERSION_BYTE2_SHIFT),
+ hnae3_get_field(*fw_version, HNAE3_FW_VERSION_BYTE1_MASK,
+ HNAE3_FW_VERSION_BYTE1_SHIFT),
+ hnae3_get_field(*fw_version, HNAE3_FW_VERSION_BYTE0_MASK,
+ HNAE3_FW_VERSION_BYTE0_SHIFT));
+
+ if (!is_pf && ae_dev->dev_version < HNAE3_DEVICE_VERSION_V3)
+ return 0;
+
+ /* ask the firmware to enable some features, driver can work without
+ * it.
+ */
+ ret = hclge_comm_firmware_compat_config(ae_dev, hw, true);
+ if (ret)
+ dev_warn(&ae_dev->pdev->dev,
+ "Firmware compatible features not enabled(%d).\n",
+ ret);
+ return 0;
+
+err_cmd_init:
+ set_bit(HCLGE_COMM_STATE_CMD_DISABLE, &hw->comm_state);
+
+ return ret;
+}
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_cmd.h b/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_cmd.h
new file mode 100644
index 0000000..876650ed
--- /dev/null
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_cmd.h
@@ -0,0 +1,458 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+// Copyright (c) 2021-2021 Hisilicon Limited.
+
+#ifndef __HCLGE_COMM_CMD_H
+#define __HCLGE_COMM_CMD_H
+#include <linux/types.h>
+
+#include "hnae3.h"
+
+#define HCLGE_COMM_CMD_FLAG_IN BIT(0)
+#define HCLGE_COMM_CMD_FLAG_NEXT BIT(2)
+#define HCLGE_COMM_CMD_FLAG_WR BIT(3)
+#define HCLGE_COMM_CMD_FLAG_NO_INTR BIT(4)
+
+#define HCLGE_COMM_SEND_SYNC(flag) \
+ ((flag) & HCLGE_COMM_CMD_FLAG_NO_INTR)
+
+#define HCLGE_COMM_LINK_EVENT_REPORT_EN_B 0
+#define HCLGE_COMM_NCSI_ERROR_REPORT_EN_B 1
+#define HCLGE_COMM_PHY_IMP_EN_B 2
+#define HCLGE_COMM_MAC_STATS_EXT_EN_B 3
+#define HCLGE_COMM_SYNC_RX_RING_HEAD_EN_B 4
+
+#define hclge_comm_dev_phy_imp_supported(ae_dev) \
+ test_bit(HNAE3_DEV_SUPPORT_PHY_IMP_B, (ae_dev)->caps)
+
+#define HCLGE_COMM_TYPE_CRQ 0
+#define HCLGE_COMM_TYPE_CSQ 1
+
+#define HCLGE_COMM_CMDQ_CLEAR_WAIT_TIME 200
+
+/* bar registers for cmdq */
+#define HCLGE_COMM_NIC_CSQ_BASEADDR_L_REG 0x27000
+#define HCLGE_COMM_NIC_CSQ_BASEADDR_H_REG 0x27004
+#define HCLGE_COMM_NIC_CSQ_DEPTH_REG 0x27008
+#define HCLGE_COMM_NIC_CSQ_TAIL_REG 0x27010
+#define HCLGE_COMM_NIC_CSQ_HEAD_REG 0x27014
+#define HCLGE_COMM_NIC_CRQ_BASEADDR_L_REG 0x27018
+#define HCLGE_COMM_NIC_CRQ_BASEADDR_H_REG 0x2701C
+#define HCLGE_COMM_NIC_CRQ_DEPTH_REG 0x27020
+#define HCLGE_COMM_NIC_CRQ_TAIL_REG 0x27024
+#define HCLGE_COMM_NIC_CRQ_HEAD_REG 0x27028
+/* Vector0 interrupt CMDQ event source register(RW) */
+#define HCLGE_COMM_VECTOR0_CMDQ_SRC_REG 0x27100
+/* Vector0 interrupt CMDQ event status register(RO) */
+#define HCLGE_COMM_VECTOR0_CMDQ_STATE_REG 0x27104
+#define HCLGE_COMM_CMDQ_INTR_EN_REG 0x27108
+#define HCLGE_COMM_CMDQ_INTR_GEN_REG 0x2710C
+#define HCLGE_COMM_CMDQ_INTR_STS_REG 0x27104
+
+/* this bit indicates that the driver is ready for hardware reset */
+#define HCLGE_COMM_NIC_SW_RST_RDY_B 16
+#define HCLGE_COMM_NIC_SW_RST_RDY BIT(HCLGE_COMM_NIC_SW_RST_RDY_B)
+#define HCLGE_COMM_NIC_CMQ_DESC_NUM_S 3
+#define HCLGE_COMM_NIC_CMQ_DESC_NUM 1024
+#define HCLGE_COMM_CMDQ_TX_TIMEOUT 30000
+
+enum hclge_opcode_type {
+ /* Generic commands */
+ HCLGE_OPC_QUERY_FW_VER = 0x0001,
+ HCLGE_OPC_CFG_RST_TRIGGER = 0x0020,
+ HCLGE_OPC_GBL_RST_STATUS = 0x0021,
+ HCLGE_OPC_QUERY_FUNC_STATUS = 0x0022,
+ HCLGE_OPC_QUERY_PF_RSRC = 0x0023,
+ HCLGE_OPC_QUERY_VF_RSRC = 0x0024,
+ HCLGE_OPC_GET_CFG_PARAM = 0x0025,
+ HCLGE_OPC_PF_RST_DONE = 0x0026,
+ HCLGE_OPC_QUERY_VF_RST_RDY = 0x0027,
+
+ HCLGE_OPC_STATS_64_BIT = 0x0030,
+ HCLGE_OPC_STATS_32_BIT = 0x0031,
+ HCLGE_OPC_STATS_MAC = 0x0032,
+ HCLGE_OPC_QUERY_MAC_REG_NUM = 0x0033,
+ HCLGE_OPC_STATS_MAC_ALL = 0x0034,
+
+ HCLGE_OPC_QUERY_REG_NUM = 0x0040,
+ HCLGE_OPC_QUERY_32_BIT_REG = 0x0041,
+ HCLGE_OPC_QUERY_64_BIT_REG = 0x0042,
+ HCLGE_OPC_DFX_BD_NUM = 0x0043,
+ HCLGE_OPC_DFX_BIOS_COMMON_REG = 0x0044,
+ HCLGE_OPC_DFX_SSU_REG_0 = 0x0045,
+ HCLGE_OPC_DFX_SSU_REG_1 = 0x0046,
+ HCLGE_OPC_DFX_IGU_EGU_REG = 0x0047,
+ HCLGE_OPC_DFX_RPU_REG_0 = 0x0048,
+ HCLGE_OPC_DFX_RPU_REG_1 = 0x0049,
+ HCLGE_OPC_DFX_NCSI_REG = 0x004A,
+ HCLGE_OPC_DFX_RTC_REG = 0x004B,
+ HCLGE_OPC_DFX_PPP_REG = 0x004C,
+ HCLGE_OPC_DFX_RCB_REG = 0x004D,
+ HCLGE_OPC_DFX_TQP_REG = 0x004E,
+ HCLGE_OPC_DFX_SSU_REG_2 = 0x004F,
+
+ HCLGE_OPC_QUERY_DEV_SPECS = 0x0050,
+
+ /* MAC command */
+ HCLGE_OPC_CONFIG_MAC_MODE = 0x0301,
+ HCLGE_OPC_CONFIG_AN_MODE = 0x0304,
+ HCLGE_OPC_QUERY_LINK_STATUS = 0x0307,
+ HCLGE_OPC_CONFIG_MAX_FRM_SIZE = 0x0308,
+ HCLGE_OPC_CONFIG_SPEED_DUP = 0x0309,
+ HCLGE_OPC_QUERY_MAC_TNL_INT = 0x0310,
+ HCLGE_OPC_MAC_TNL_INT_EN = 0x0311,
+ HCLGE_OPC_CLEAR_MAC_TNL_INT = 0x0312,
+ HCLGE_OPC_COMMON_LOOPBACK = 0x0315,
+ HCLGE_OPC_CONFIG_FEC_MODE = 0x031A,
+ HCLGE_OPC_QUERY_ROH_TYPE_INFO = 0x0389,
+
+ /* PTP commands */
+ HCLGE_OPC_PTP_INT_EN = 0x0501,
+ HCLGE_OPC_PTP_MODE_CFG = 0x0507,
+
+ /* PFC/Pause commands */
+ HCLGE_OPC_CFG_MAC_PAUSE_EN = 0x0701,
+ HCLGE_OPC_CFG_PFC_PAUSE_EN = 0x0702,
+ HCLGE_OPC_CFG_MAC_PARA = 0x0703,
+ HCLGE_OPC_CFG_PFC_PARA = 0x0704,
+ HCLGE_OPC_QUERY_MAC_TX_PKT_CNT = 0x0705,
+ HCLGE_OPC_QUERY_MAC_RX_PKT_CNT = 0x0706,
+ HCLGE_OPC_QUERY_PFC_TX_PKT_CNT = 0x0707,
+ HCLGE_OPC_QUERY_PFC_RX_PKT_CNT = 0x0708,
+ HCLGE_OPC_PRI_TO_TC_MAPPING = 0x0709,
+ HCLGE_OPC_QOS_MAP = 0x070A,
+
+ /* ETS/scheduler commands */
+ HCLGE_OPC_TM_PG_TO_PRI_LINK = 0x0804,
+ HCLGE_OPC_TM_QS_TO_PRI_LINK = 0x0805,
+ HCLGE_OPC_TM_NQ_TO_QS_LINK = 0x0806,
+ HCLGE_OPC_TM_RQ_TO_QS_LINK = 0x0807,
+ HCLGE_OPC_TM_PORT_WEIGHT = 0x0808,
+ HCLGE_OPC_TM_PG_WEIGHT = 0x0809,
+ HCLGE_OPC_TM_QS_WEIGHT = 0x080A,
+ HCLGE_OPC_TM_PRI_WEIGHT = 0x080B,
+ HCLGE_OPC_TM_PRI_C_SHAPPING = 0x080C,
+ HCLGE_OPC_TM_PRI_P_SHAPPING = 0x080D,
+ HCLGE_OPC_TM_PG_C_SHAPPING = 0x080E,
+ HCLGE_OPC_TM_PG_P_SHAPPING = 0x080F,
+ HCLGE_OPC_TM_PORT_SHAPPING = 0x0810,
+ HCLGE_OPC_TM_PG_SCH_MODE_CFG = 0x0812,
+ HCLGE_OPC_TM_PRI_SCH_MODE_CFG = 0x0813,
+ HCLGE_OPC_TM_QS_SCH_MODE_CFG = 0x0814,
+ HCLGE_OPC_TM_BP_TO_QSET_MAPPING = 0x0815,
+ HCLGE_OPC_TM_NODES = 0x0816,
+ HCLGE_OPC_ETS_TC_WEIGHT = 0x0843,
+ HCLGE_OPC_QSET_DFX_STS = 0x0844,
+ HCLGE_OPC_PRI_DFX_STS = 0x0845,
+ HCLGE_OPC_PG_DFX_STS = 0x0846,
+ HCLGE_OPC_PORT_DFX_STS = 0x0847,
+ HCLGE_OPC_SCH_NQ_CNT = 0x0848,
+ HCLGE_OPC_SCH_RQ_CNT = 0x0849,
+ HCLGE_OPC_TM_INTERNAL_STS = 0x0850,
+ HCLGE_OPC_TM_INTERNAL_CNT = 0x0851,
+ HCLGE_OPC_TM_INTERNAL_STS_1 = 0x0852,
+
+ /* Packet buffer allocate commands */
+ HCLGE_OPC_TX_BUFF_ALLOC = 0x0901,
+ HCLGE_OPC_RX_PRIV_BUFF_ALLOC = 0x0902,
+ HCLGE_OPC_RX_PRIV_WL_ALLOC = 0x0903,
+ HCLGE_OPC_RX_COM_THRD_ALLOC = 0x0904,
+ HCLGE_OPC_RX_COM_WL_ALLOC = 0x0905,
+ HCLGE_OPC_RX_GBL_PKT_CNT = 0x0906,
+
+ /* TQP management command */
+ HCLGE_OPC_SET_TQP_MAP = 0x0A01,
+
+ /* TQP commands */
+ HCLGE_OPC_CFG_TX_QUEUE = 0x0B01,
+ HCLGE_OPC_QUERY_TX_POINTER = 0x0B02,
+ HCLGE_OPC_QUERY_TX_STATS = 0x0B03,
+ HCLGE_OPC_TQP_TX_QUEUE_TC = 0x0B04,
+ HCLGE_OPC_CFG_RX_QUEUE = 0x0B11,
+ HCLGE_OPC_QUERY_RX_POINTER = 0x0B12,
+ HCLGE_OPC_QUERY_RX_STATS = 0x0B13,
+ HCLGE_OPC_STASH_RX_QUEUE_LRO = 0x0B16,
+ HCLGE_OPC_CFG_RX_QUEUE_LRO = 0x0B17,
+ HCLGE_OPC_CFG_COM_TQP_QUEUE = 0x0B20,
+ HCLGE_OPC_RESET_TQP_QUEUE = 0x0B22,
+
+ /* PPU commands */
+ HCLGE_OPC_PPU_PF_OTHER_INT_DFX = 0x0B4A,
+
+ /* TSO command */
+ HCLGE_OPC_TSO_GENERIC_CONFIG = 0x0C01,
+ HCLGE_OPC_GRO_GENERIC_CONFIG = 0x0C10,
+
+ /* RSS commands */
+ HCLGE_OPC_RSS_GENERIC_CONFIG = 0x0D01,
+ HCLGE_OPC_RSS_INDIR_TABLE = 0x0D07,
+ HCLGE_OPC_RSS_TC_MODE = 0x0D08,
+ HCLGE_OPC_RSS_INPUT_TUPLE = 0x0D02,
+
+ /* Promisuous mode command */
+ HCLGE_OPC_CFG_PROMISC_MODE = 0x0E01,
+
+ /* Vlan offload commands */
+ HCLGE_OPC_VLAN_PORT_TX_CFG = 0x0F01,
+ HCLGE_OPC_VLAN_PORT_RX_CFG = 0x0F02,
+
+ /* Interrupts commands */
+ HCLGE_OPC_ADD_RING_TO_VECTOR = 0x1503,
+ HCLGE_OPC_DEL_RING_TO_VECTOR = 0x1504,
+
+ /* MAC commands */
+ HCLGE_OPC_MAC_VLAN_ADD = 0x1000,
+ HCLGE_OPC_MAC_VLAN_REMOVE = 0x1001,
+ HCLGE_OPC_MAC_VLAN_TYPE_ID = 0x1002,
+ HCLGE_OPC_MAC_VLAN_INSERT = 0x1003,
+ HCLGE_OPC_MAC_VLAN_ALLOCATE = 0x1004,
+ HCLGE_OPC_MAC_ETHTYPE_ADD = 0x1010,
+ HCLGE_OPC_MAC_ETHTYPE_REMOVE = 0x1011,
+
+ /* MAC VLAN commands */
+ HCLGE_OPC_MAC_VLAN_SWITCH_PARAM = 0x1033,
+
+ /* VLAN commands */
+ HCLGE_OPC_VLAN_FILTER_CTRL = 0x1100,
+ HCLGE_OPC_VLAN_FILTER_PF_CFG = 0x1101,
+ HCLGE_OPC_VLAN_FILTER_VF_CFG = 0x1102,
+ HCLGE_OPC_PORT_VLAN_BYPASS = 0x1103,
+
+ /* Flow Director commands */
+ HCLGE_OPC_FD_MODE_CTRL = 0x1200,
+ HCLGE_OPC_FD_GET_ALLOCATION = 0x1201,
+ HCLGE_OPC_FD_KEY_CONFIG = 0x1202,
+ HCLGE_OPC_FD_TCAM_OP = 0x1203,
+ HCLGE_OPC_FD_AD_OP = 0x1204,
+ HCLGE_OPC_FD_CNT_OP = 0x1205,
+ HCLGE_OPC_FD_USER_DEF_OP = 0x1207,
+ HCLGE_OPC_FD_QB_CTRL = 0x1210,
+ HCLGE_OPC_FD_QB_AD_OP = 0x1211,
+
+ /* MDIO command */
+ HCLGE_OPC_MDIO_CONFIG = 0x1900,
+
+ /* QCN commands */
+ HCLGE_OPC_QCN_MOD_CFG = 0x1A01,
+ HCLGE_OPC_QCN_GRP_TMPLT_CFG = 0x1A02,
+ HCLGE_OPC_QCN_SHAPPING_CFG = 0x1A03,
+ HCLGE_OPC_QCN_SHAPPING_BS_CFG = 0x1A04,
+ HCLGE_OPC_QCN_QSET_LINK_CFG = 0x1A05,
+ HCLGE_OPC_QCN_RP_STATUS_GET = 0x1A06,
+ HCLGE_OPC_QCN_AJUST_INIT = 0x1A07,
+ HCLGE_OPC_QCN_DFX_CNT_STATUS = 0x1A08,
+
+ /* Mailbox command */
+ HCLGEVF_OPC_MBX_PF_TO_VF = 0x2000,
+ HCLGEVF_OPC_MBX_VF_TO_PF = 0x2001,
+
+ /* Led command */
+ HCLGE_OPC_LED_STATUS_CFG = 0xB000,
+
+ /* clear hardware resource command */
+ HCLGE_OPC_CLEAR_HW_RESOURCE = 0x700B,
+
+ /* NCL config command */
+ HCLGE_OPC_QUERY_NCL_CONFIG = 0x7011,
+
+ /* IMP stats command */
+ HCLGE_OPC_IMP_STATS_BD = 0x7012,
+ HCLGE_OPC_IMP_STATS_INFO = 0x7013,
+ HCLGE_OPC_IMP_COMPAT_CFG = 0x701A,
+
+ /* SFP command */
+ HCLGE_OPC_GET_SFP_EEPROM = 0x7100,
+ HCLGE_OPC_GET_SFP_EXIST = 0x7101,
+ HCLGE_OPC_GET_SFP_INFO = 0x7104,
+
+ /* Error INT commands */
+ HCLGE_MAC_COMMON_INT_EN = 0x030E,
+ HCLGE_TM_SCH_ECC_INT_EN = 0x0829,
+ HCLGE_SSU_ECC_INT_CMD = 0x0989,
+ HCLGE_SSU_COMMON_INT_CMD = 0x098C,
+ HCLGE_PPU_MPF_ECC_INT_CMD = 0x0B40,
+ HCLGE_PPU_MPF_OTHER_INT_CMD = 0x0B41,
+ HCLGE_PPU_PF_OTHER_INT_CMD = 0x0B42,
+ HCLGE_COMMON_ECC_INT_CFG = 0x1505,
+ HCLGE_QUERY_RAS_INT_STS_BD_NUM = 0x1510,
+ HCLGE_QUERY_CLEAR_MPF_RAS_INT = 0x1511,
+ HCLGE_QUERY_CLEAR_PF_RAS_INT = 0x1512,
+ HCLGE_QUERY_MSIX_INT_STS_BD_NUM = 0x1513,
+ HCLGE_QUERY_CLEAR_ALL_MPF_MSIX_INT = 0x1514,
+ HCLGE_QUERY_CLEAR_ALL_PF_MSIX_INT = 0x1515,
+ HCLGE_QUERY_ALL_ERR_BD_NUM = 0x1516,
+ HCLGE_QUERY_ALL_ERR_INFO = 0x1517,
+ HCLGE_CONFIG_ROCEE_RAS_INT_EN = 0x1580,
+ HCLGE_QUERY_CLEAR_ROCEE_RAS_INT = 0x1581,
+ HCLGE_ROCEE_PF_RAS_INT_CMD = 0x1584,
+ HCLGE_QUERY_ROCEE_ECC_RAS_INFO_CMD = 0x1585,
+ HCLGE_QUERY_ROCEE_AXI_RAS_INFO_CMD = 0x1586,
+ HCLGE_IGU_EGU_TNL_INT_EN = 0x1803,
+ HCLGE_IGU_COMMON_INT_EN = 0x1806,
+ HCLGE_TM_QCN_MEM_INT_CFG = 0x1A14,
+ HCLGE_PPP_CMD0_INT_CMD = 0x2100,
+ HCLGE_PPP_CMD1_INT_CMD = 0x2101,
+ HCLGE_MAC_ETHERTYPE_IDX_RD = 0x2105,
+ HCLGE_NCSI_INT_EN = 0x2401,
+
+ /* ROH MAC commands */
+ HCLGE_OPC_MAC_ADDR_CHECK = 0x9004,
+
+ /* PHY command */
+ HCLGE_OPC_PHY_LINK_KSETTING = 0x7025,
+ HCLGE_OPC_PHY_REG = 0x7026,
+
+ /* Query link diagnosis info command */
+ HCLGE_OPC_QUERY_LINK_DIAGNOSIS = 0x702A,
+};
+
+enum hclge_comm_cmd_return_status {
+ HCLGE_COMM_CMD_EXEC_SUCCESS = 0,
+ HCLGE_COMM_CMD_NO_AUTH = 1,
+ HCLGE_COMM_CMD_NOT_SUPPORTED = 2,
+ HCLGE_COMM_CMD_QUEUE_FULL = 3,
+ HCLGE_COMM_CMD_NEXT_ERR = 4,
+ HCLGE_COMM_CMD_UNEXE_ERR = 5,
+ HCLGE_COMM_CMD_PARA_ERR = 6,
+ HCLGE_COMM_CMD_RESULT_ERR = 7,
+ HCLGE_COMM_CMD_TIMEOUT = 8,
+ HCLGE_COMM_CMD_HILINK_ERR = 9,
+ HCLGE_COMM_CMD_QUEUE_ILLEGAL = 10,
+ HCLGE_COMM_CMD_INVALID = 11,
+};
+
+enum HCLGE_COMM_CAP_BITS {
+ HCLGE_COMM_CAP_UDP_GSO_B,
+ HCLGE_COMM_CAP_QB_B,
+ HCLGE_COMM_CAP_FD_FORWARD_TC_B,
+ HCLGE_COMM_CAP_PTP_B,
+ HCLGE_COMM_CAP_INT_QL_B,
+ HCLGE_COMM_CAP_HW_TX_CSUM_B,
+ HCLGE_COMM_CAP_TX_PUSH_B,
+ HCLGE_COMM_CAP_PHY_IMP_B,
+ HCLGE_COMM_CAP_TQP_TXRX_INDEP_B,
+ HCLGE_COMM_CAP_HW_PAD_B,
+ HCLGE_COMM_CAP_STASH_B,
+ HCLGE_COMM_CAP_UDP_TUNNEL_CSUM_B,
+ HCLGE_COMM_CAP_RAS_IMP_B = 12,
+ HCLGE_COMM_CAP_FEC_B = 13,
+ HCLGE_COMM_CAP_PAUSE_B = 14,
+ HCLGE_COMM_CAP_RXD_ADV_LAYOUT_B = 15,
+ HCLGE_COMM_CAP_PORT_VLAN_BYPASS_B = 17,
+};
+
+enum HCLGE_COMM_API_CAP_BITS {
+ HCLGE_COMM_API_CAP_FLEX_RSS_TBL_B,
+};
+
+/* capabilities bits map between imp firmware and local driver */
+struct hclge_comm_caps_bit_map {
+ u16 imp_bit;
+ u16 local_bit;
+};
+
+struct hclge_comm_firmware_compat_cmd {
+ __le32 compat;
+ u8 rsv[20];
+};
+
+enum hclge_comm_cmd_state {
+ HCLGE_COMM_STATE_CMD_DISABLE,
+};
+
+struct hclge_comm_errcode {
+ u32 imp_errcode;
+ int common_errno;
+};
+
+#define HCLGE_COMM_QUERY_CAP_LENGTH 3
+struct hclge_comm_query_version_cmd {
+ __le32 firmware;
+ __le32 hardware;
+ __le32 api_caps;
+ __le32 caps[HCLGE_COMM_QUERY_CAP_LENGTH]; /* capabilities of device */
+};
+
+#define HCLGE_DESC_DATA_LEN 6
+struct hclge_desc {
+ __le16 opcode;
+ __le16 flag;
+ __le16 retval;
+ __le16 rsv;
+ __le32 data[HCLGE_DESC_DATA_LEN];
+};
+
+struct hclge_comm_cmq_ring {
+ dma_addr_t desc_dma_addr;
+ struct hclge_desc *desc;
+ struct pci_dev *pdev;
+ u32 head;
+ u32 tail;
+
+ u16 buf_size;
+ u16 desc_num;
+ int next_to_use;
+ int next_to_clean;
+ u8 ring_type; /* cmq ring type */
+ spinlock_t lock; /* Command queue lock */
+};
+
+enum hclge_comm_cmd_status {
+ HCLGE_COMM_STATUS_SUCCESS = 0,
+ HCLGE_COMM_ERR_CSQ_FULL = -1,
+ HCLGE_COMM_ERR_CSQ_TIMEOUT = -2,
+ HCLGE_COMM_ERR_CSQ_ERROR = -3,
+};
+
+struct hclge_comm_cmq {
+ struct hclge_comm_cmq_ring csq;
+ struct hclge_comm_cmq_ring crq;
+ u16 tx_timeout;
+ enum hclge_comm_cmd_status last_status;
+};
+
+struct hclge_comm_hw {
+ void __iomem *io_base;
+ void __iomem *mem_base;
+ struct hclge_comm_cmq cmq;
+ unsigned long comm_state;
+};
+
+static inline void hclge_comm_write_reg(void __iomem *base, u32 reg, u32 value)
+{
+ writel(value, base + reg);
+}
+
+static inline u32 hclge_comm_read_reg(u8 __iomem *base, u32 reg)
+{
+ u8 __iomem *reg_addr = READ_ONCE(base);
+
+ return readl(reg_addr + reg);
+}
+
+#define hclge_comm_write_dev(a, reg, value) \
+ hclge_comm_write_reg((a)->io_base, reg, value)
+#define hclge_comm_read_dev(a, reg) \
+ hclge_comm_read_reg((a)->io_base, reg)
+
+void hclge_comm_cmd_init_regs(struct hclge_comm_hw *hw);
+int hclge_comm_cmd_query_version_and_capability(struct hnae3_ae_dev *ae_dev,
+ struct hclge_comm_hw *hw,
+ u32 *fw_version, bool is_pf);
+int hclge_comm_alloc_cmd_queue(struct hclge_comm_hw *hw, int ring_type);
+int hclge_comm_cmd_send(struct hclge_comm_hw *hw, struct hclge_desc *desc,
+ int num);
+void hclge_comm_cmd_reuse_desc(struct hclge_desc *desc, bool is_read);
+int hclge_comm_firmware_compat_config(struct hnae3_ae_dev *ae_dev,
+ struct hclge_comm_hw *hw, bool en);
+void hclge_comm_free_cmd_desc(struct hclge_comm_cmq_ring *ring);
+void hclge_comm_cmd_setup_basic_desc(struct hclge_desc *desc,
+ enum hclge_opcode_type opcode,
+ bool is_read);
+void hclge_comm_cmd_uninit(struct hnae3_ae_dev *ae_dev,
+ struct hclge_comm_hw *hw);
+int hclge_comm_cmd_queue_init(struct pci_dev *pdev, struct hclge_comm_hw *hw);
+int hclge_comm_cmd_init(struct hnae3_ae_dev *ae_dev, struct hclge_comm_hw *hw,
+ u32 *fw_version, bool is_pf,
+ unsigned long reset_pending);
+
+#endif
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_rss.c b/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_rss.c
new file mode 100644
index 0000000..e23729a
--- /dev/null
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_rss.c
@@ -0,0 +1,525 @@
+// SPDX-License-Identifier: GPL-2.0+
+// Copyright (c) 2021-2021 Hisilicon Limited.
+#include <linux/skbuff.h>
+
+#include "hnae3.h"
+#include "hclge_comm_cmd.h"
+#include "hclge_comm_rss.h"
+
+static const u8 hclge_comm_hash_key[] = {
+ 0x6D, 0x5A, 0x56, 0xDA, 0x25, 0x5B, 0x0E, 0xC2,
+ 0x41, 0x67, 0x25, 0x3D, 0x43, 0xA3, 0x8F, 0xB0,
+ 0xD0, 0xCA, 0x2B, 0xCB, 0xAE, 0x7B, 0x30, 0xB4,
+ 0x77, 0xCB, 0x2D, 0xA3, 0x80, 0x30, 0xF2, 0x0C,
+ 0x6A, 0x42, 0xB7, 0x3B, 0xBE, 0xAC, 0x01, 0xFA
+};
+
+static void
+hclge_comm_init_rss_tuple(struct hnae3_ae_dev *ae_dev,
+ struct hclge_comm_rss_tuple_cfg *rss_tuple_cfg)
+{
+ rss_tuple_cfg->ipv4_tcp_en = HCLGE_COMM_RSS_INPUT_TUPLE_OTHER;
+ rss_tuple_cfg->ipv4_udp_en = HCLGE_COMM_RSS_INPUT_TUPLE_OTHER;
+ rss_tuple_cfg->ipv4_sctp_en = HCLGE_COMM_RSS_INPUT_TUPLE_SCTP;
+ rss_tuple_cfg->ipv4_fragment_en = HCLGE_COMM_RSS_INPUT_TUPLE_OTHER;
+ rss_tuple_cfg->ipv6_tcp_en = HCLGE_COMM_RSS_INPUT_TUPLE_OTHER;
+ rss_tuple_cfg->ipv6_udp_en = HCLGE_COMM_RSS_INPUT_TUPLE_OTHER;
+ rss_tuple_cfg->ipv6_sctp_en =
+ ae_dev->dev_version <= HNAE3_DEVICE_VERSION_V2 ?
+ HCLGE_COMM_RSS_INPUT_TUPLE_SCTP_NO_PORT :
+ HCLGE_COMM_RSS_INPUT_TUPLE_SCTP;
+ rss_tuple_cfg->ipv6_fragment_en = HCLGE_COMM_RSS_INPUT_TUPLE_OTHER;
+}
+
+int hclge_comm_rss_init_cfg(struct hnae3_handle *nic,
+ struct hnae3_ae_dev *ae_dev,
+ struct hclge_comm_rss_cfg *rss_cfg)
+{
+ u16 rss_ind_tbl_size = ae_dev->dev_specs.rss_ind_tbl_size;
+ int rss_algo = HCLGE_COMM_RSS_HASH_ALGO_TOEPLITZ;
+ u16 *rss_ind_tbl;
+
+ if (nic->flags & HNAE3_SUPPORT_VF)
+ rss_cfg->rss_size = nic->kinfo.rss_size;
+
+ if (ae_dev->dev_version >= HNAE3_DEVICE_VERSION_V2)
+ rss_algo = HCLGE_COMM_RSS_HASH_ALGO_SIMPLE;
+
+ hclge_comm_init_rss_tuple(ae_dev, &rss_cfg->rss_tuple_sets);
+
+ rss_cfg->rss_algo = rss_algo;
+
+ rss_ind_tbl = devm_kcalloc(&ae_dev->pdev->dev, rss_ind_tbl_size,
+ sizeof(*rss_ind_tbl), GFP_KERNEL);
+ if (!rss_ind_tbl)
+ return -ENOMEM;
+
+ rss_cfg->rss_indirection_tbl = rss_ind_tbl;
+ memcpy(rss_cfg->rss_hash_key, hclge_comm_hash_key,
+ HCLGE_COMM_RSS_KEY_SIZE);
+
+ hclge_comm_rss_indir_init_cfg(ae_dev, rss_cfg);
+
+ return 0;
+}
+
+void hclge_comm_get_rss_tc_info(u16 rss_size, u8 hw_tc_map, u16 *tc_offset,
+ u16 *tc_valid, u16 *tc_size)
+{
+ u16 roundup_size;
+ u32 i;
+
+ roundup_size = roundup_pow_of_two(rss_size);
+ roundup_size = ilog2(roundup_size);
+
+ for (i = 0; i < HCLGE_COMM_MAX_TC_NUM; i++) {
+ tc_valid[i] = 1;
+ tc_size[i] = roundup_size;
+ tc_offset[i] = (hw_tc_map & BIT(i)) ? rss_size * i : 0;
+ }
+}
+
+int hclge_comm_set_rss_tc_mode(struct hclge_comm_hw *hw, u16 *tc_offset,
+ u16 *tc_valid, u16 *tc_size)
+{
+ struct hclge_comm_rss_tc_mode_cmd *req;
+ struct hclge_desc desc;
+ unsigned int i;
+ int ret;
+
+ req = (struct hclge_comm_rss_tc_mode_cmd *)desc.data;
+
+ hclge_comm_cmd_setup_basic_desc(&desc, HCLGE_OPC_RSS_TC_MODE, false);
+ for (i = 0; i < HCLGE_COMM_MAX_TC_NUM; i++) {
+ u16 mode = 0;
+
+ hnae3_set_bit(mode, HCLGE_COMM_RSS_TC_VALID_B,
+ (tc_valid[i] & 0x1));
+ hnae3_set_field(mode, HCLGE_COMM_RSS_TC_SIZE_M,
+ HCLGE_COMM_RSS_TC_SIZE_S, tc_size[i]);
+ hnae3_set_bit(mode, HCLGE_COMM_RSS_TC_SIZE_MSB_B,
+ tc_size[i] >> HCLGE_COMM_RSS_TC_SIZE_MSB_OFFSET &
+ 0x1);
+ hnae3_set_field(mode, HCLGE_COMM_RSS_TC_OFFSET_M,
+ HCLGE_COMM_RSS_TC_OFFSET_S, tc_offset[i]);
+
+ req->rss_tc_mode[i] = cpu_to_le16(mode);
+ }
+
+ ret = hclge_comm_cmd_send(hw, &desc, 1);
+ if (ret)
+ dev_err(&hw->cmq.csq.pdev->dev,
+ "failed to set rss tc mode, ret = %d.\n", ret);
+
+ return ret;
+}
+
+int hclge_comm_set_rss_hash_key(struct hclge_comm_rss_cfg *rss_cfg,
+ struct hclge_comm_hw *hw, const u8 *key,
+ const u8 hfunc)
+{
+ u8 hash_algo;
+ int ret;
+
+ ret = hclge_comm_parse_rss_hfunc(rss_cfg, hfunc, &hash_algo);
+ if (ret)
+ return ret;
+
+ /* Set the RSS Hash Key if specififed by the user */
+ if (key) {
+ ret = hclge_comm_set_rss_algo_key(hw, hash_algo, key);
+ if (ret)
+ return ret;
+
+ /* Update the shadow RSS key with user specified qids */
+ memcpy(rss_cfg->rss_hash_key, key, HCLGE_COMM_RSS_KEY_SIZE);
+ } else {
+ ret = hclge_comm_set_rss_algo_key(hw, hash_algo,
+ rss_cfg->rss_hash_key);
+ if (ret)
+ return ret;
+ }
+ rss_cfg->rss_algo = hash_algo;
+
+ return 0;
+}
+
+int hclge_comm_set_rss_tuple(struct hnae3_ae_dev *ae_dev,
+ struct hclge_comm_hw *hw,
+ struct hclge_comm_rss_cfg *rss_cfg,
+ struct ethtool_rxnfc *nfc)
+{
+ struct hclge_comm_rss_input_tuple_cmd *req;
+ struct hclge_desc desc;
+ int ret;
+
+ if (nfc->data &
+ ~(RXH_IP_SRC | RXH_IP_DST | RXH_L4_B_0_1 | RXH_L4_B_2_3))
+ return -EINVAL;
+
+ req = (struct hclge_comm_rss_input_tuple_cmd *)desc.data;
+ hclge_comm_cmd_setup_basic_desc(&desc, HCLGE_OPC_RSS_INPUT_TUPLE,
+ false);
+
+ ret = hclge_comm_init_rss_tuple_cmd(rss_cfg, nfc, ae_dev, req);
+ if (ret) {
+ dev_err(&hw->cmq.csq.pdev->dev,
+ "failed to init rss tuple cmd, ret = %d.\n", ret);
+ return ret;
+ }
+
+ ret = hclge_comm_cmd_send(hw, &desc, 1);
+ if (ret) {
+ dev_err(&hw->cmq.csq.pdev->dev,
+ "failed to set rss tuple, ret = %d.\n", ret);
+ return ret;
+ }
+
+ rss_cfg->rss_tuple_sets.ipv4_tcp_en = req->ipv4_tcp_en;
+ rss_cfg->rss_tuple_sets.ipv4_udp_en = req->ipv4_udp_en;
+ rss_cfg->rss_tuple_sets.ipv4_sctp_en = req->ipv4_sctp_en;
+ rss_cfg->rss_tuple_sets.ipv4_fragment_en = req->ipv4_fragment_en;
+ rss_cfg->rss_tuple_sets.ipv6_tcp_en = req->ipv6_tcp_en;
+ rss_cfg->rss_tuple_sets.ipv6_udp_en = req->ipv6_udp_en;
+ rss_cfg->rss_tuple_sets.ipv6_sctp_en = req->ipv6_sctp_en;
+ rss_cfg->rss_tuple_sets.ipv6_fragment_en = req->ipv6_fragment_en;
+ return 0;
+}
+
+u32 hclge_comm_get_rss_key_size(struct hnae3_handle *handle)
+{
+ return HCLGE_COMM_RSS_KEY_SIZE;
+}
+
+void hclge_comm_get_rss_type(struct hnae3_handle *nic,
+ struct hclge_comm_rss_tuple_cfg *rss_tuple_sets)
+{
+ if (rss_tuple_sets->ipv4_tcp_en ||
+ rss_tuple_sets->ipv4_udp_en ||
+ rss_tuple_sets->ipv4_sctp_en ||
+ rss_tuple_sets->ipv6_tcp_en ||
+ rss_tuple_sets->ipv6_udp_en ||
+ rss_tuple_sets->ipv6_sctp_en)
+ nic->kinfo.rss_type = PKT_HASH_TYPE_L4;
+ else if (rss_tuple_sets->ipv4_fragment_en ||
+ rss_tuple_sets->ipv6_fragment_en)
+ nic->kinfo.rss_type = PKT_HASH_TYPE_L3;
+ else
+ nic->kinfo.rss_type = PKT_HASH_TYPE_NONE;
+}
+
+int hclge_comm_parse_rss_hfunc(struct hclge_comm_rss_cfg *rss_cfg,
+ const u8 hfunc, u8 *hash_algo)
+{
+ switch (hfunc) {
+ case ETH_RSS_HASH_TOP:
+ *hash_algo = HCLGE_COMM_RSS_HASH_ALGO_TOEPLITZ;
+ return 0;
+ case ETH_RSS_HASH_XOR:
+ *hash_algo = HCLGE_COMM_RSS_HASH_ALGO_SIMPLE;
+ return 0;
+ case ETH_RSS_HASH_NO_CHANGE:
+ *hash_algo = rss_cfg->rss_algo;
+ return 0;
+ default:
+ return -EINVAL;
+ }
+}
+
+void hclge_comm_rss_indir_init_cfg(struct hnae3_ae_dev *ae_dev,
+ struct hclge_comm_rss_cfg *rss_cfg)
+{
+ u16 i;
+ /* Initialize RSS indirect table */
+ for (i = 0; i < ae_dev->dev_specs.rss_ind_tbl_size; i++)
+ rss_cfg->rss_indirection_tbl[i] = i % rss_cfg->rss_size;
+}
+
+int hclge_comm_get_rss_tuple(struct hclge_comm_rss_cfg *rss_cfg, int flow_type,
+ u8 *tuple_sets)
+{
+ switch (flow_type) {
+ case TCP_V4_FLOW:
+ *tuple_sets = rss_cfg->rss_tuple_sets.ipv4_tcp_en;
+ break;
+ case UDP_V4_FLOW:
+ *tuple_sets = rss_cfg->rss_tuple_sets.ipv4_udp_en;
+ break;
+ case TCP_V6_FLOW:
+ *tuple_sets = rss_cfg->rss_tuple_sets.ipv6_tcp_en;
+ break;
+ case UDP_V6_FLOW:
+ *tuple_sets = rss_cfg->rss_tuple_sets.ipv6_udp_en;
+ break;
+ case SCTP_V4_FLOW:
+ *tuple_sets = rss_cfg->rss_tuple_sets.ipv4_sctp_en;
+ break;
+ case SCTP_V6_FLOW:
+ *tuple_sets = rss_cfg->rss_tuple_sets.ipv6_sctp_en;
+ break;
+ case IPV4_FLOW:
+ case IPV6_FLOW:
+ *tuple_sets = HCLGE_COMM_S_IP_BIT | HCLGE_COMM_D_IP_BIT;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static void
+hclge_comm_append_rss_msb_info(struct hclge_comm_rss_ind_tbl_cmd *req,
+ u16 qid, u32 j)
+{
+ u8 rss_msb_oft;
+ u8 rss_msb_val;
+
+ rss_msb_oft =
+ j * HCLGE_COMM_RSS_CFG_TBL_BW_H / BITS_PER_BYTE;
+ rss_msb_val = (qid >> HCLGE_COMM_RSS_CFG_TBL_BW_L & 0x1) <<
+ (j * HCLGE_COMM_RSS_CFG_TBL_BW_H % BITS_PER_BYTE);
+ req->rss_qid_h[rss_msb_oft] |= rss_msb_val;
+}
+
+int hclge_comm_set_rss_indir_table(struct hnae3_ae_dev *ae_dev,
+ struct hclge_comm_hw *hw, const u16 *indir)
+{
+ struct hclge_comm_rss_ind_tbl_cmd *req;
+ struct hclge_desc desc;
+ u16 rss_cfg_tbl_num;
+ int ret;
+ u16 qid;
+ u16 i;
+ u32 j;
+
+ req = (struct hclge_comm_rss_ind_tbl_cmd *)desc.data;
+ rss_cfg_tbl_num = ae_dev->dev_specs.rss_ind_tbl_size /
+ HCLGE_COMM_RSS_CFG_TBL_SIZE;
+
+ for (i = 0; i < rss_cfg_tbl_num; i++) {
+ hclge_comm_cmd_setup_basic_desc(&desc,
+ HCLGE_OPC_RSS_INDIR_TABLE,
+ false);
+
+ req->start_table_index =
+ cpu_to_le16(i * HCLGE_COMM_RSS_CFG_TBL_SIZE);
+ req->rss_set_bitmap =
+ cpu_to_le16(HCLGE_COMM_RSS_SET_BITMAP_MSK);
+ for (j = 0; j < HCLGE_COMM_RSS_CFG_TBL_SIZE; j++) {
+ qid = indir[i * HCLGE_COMM_RSS_CFG_TBL_SIZE + j];
+ req->rss_qid_l[j] = qid & 0xff;
+ hclge_comm_append_rss_msb_info(req, qid, j);
+ }
+ ret = hclge_comm_cmd_send(hw, &desc, 1);
+ if (ret) {
+ dev_err(&hw->cmq.csq.pdev->dev,
+ "failed to configure rss table, ret = %d.\n",
+ ret);
+ return ret;
+ }
+ }
+ return 0;
+}
+
+int hclge_comm_set_rss_input_tuple(struct hnae3_handle *nic,
+ struct hclge_comm_hw *hw, bool is_pf,
+ struct hclge_comm_rss_cfg *rss_cfg)
+{
+ struct hclge_comm_rss_input_tuple_cmd *req;
+ struct hclge_desc desc;
+ int ret;
+
+ hclge_comm_cmd_setup_basic_desc(&desc, HCLGE_OPC_RSS_INPUT_TUPLE,
+ false);
+
+ req = (struct hclge_comm_rss_input_tuple_cmd *)desc.data;
+
+ req->ipv4_tcp_en = rss_cfg->rss_tuple_sets.ipv4_tcp_en;
+ req->ipv4_udp_en = rss_cfg->rss_tuple_sets.ipv4_udp_en;
+ req->ipv4_sctp_en = rss_cfg->rss_tuple_sets.ipv4_sctp_en;
+ req->ipv4_fragment_en = rss_cfg->rss_tuple_sets.ipv4_fragment_en;
+ req->ipv6_tcp_en = rss_cfg->rss_tuple_sets.ipv6_tcp_en;
+ req->ipv6_udp_en = rss_cfg->rss_tuple_sets.ipv6_udp_en;
+ req->ipv6_sctp_en = rss_cfg->rss_tuple_sets.ipv6_sctp_en;
+ req->ipv6_fragment_en = rss_cfg->rss_tuple_sets.ipv6_fragment_en;
+
+ if (is_pf)
+ hclge_comm_get_rss_type(nic, &rss_cfg->rss_tuple_sets);
+
+ ret = hclge_comm_cmd_send(hw, &desc, 1);
+ if (ret)
+ dev_err(&hw->cmq.csq.pdev->dev,
+ "failed to configure rss input, ret = %d.\n", ret);
+ return ret;
+}
+
+void hclge_comm_get_rss_hash_info(struct hclge_comm_rss_cfg *rss_cfg, u8 *key,
+ u8 *hfunc)
+{
+ /* Get hash algorithm */
+ if (hfunc) {
+ switch (rss_cfg->rss_algo) {
+ case HCLGE_COMM_RSS_HASH_ALGO_TOEPLITZ:
+ *hfunc = ETH_RSS_HASH_TOP;
+ break;
+ case HCLGE_COMM_RSS_HASH_ALGO_SIMPLE:
+ *hfunc = ETH_RSS_HASH_XOR;
+ break;
+ default:
+ *hfunc = ETH_RSS_HASH_UNKNOWN;
+ break;
+ }
+ }
+
+ /* Get the RSS Key required by the user */
+ if (key)
+ memcpy(key, rss_cfg->rss_hash_key, HCLGE_COMM_RSS_KEY_SIZE);
+}
+
+void hclge_comm_get_rss_indir_tbl(struct hclge_comm_rss_cfg *rss_cfg,
+ u32 *indir, u16 rss_ind_tbl_size)
+{
+ u16 i;
+
+ if (!indir)
+ return;
+
+ for (i = 0; i < rss_ind_tbl_size; i++)
+ indir[i] = rss_cfg->rss_indirection_tbl[i];
+}
+
+int hclge_comm_set_rss_algo_key(struct hclge_comm_hw *hw, const u8 hfunc,
+ const u8 *key)
+{
+ struct hclge_comm_rss_config_cmd *req;
+ unsigned int key_offset = 0;
+ struct hclge_desc desc;
+ int key_counts;
+ int key_size;
+ int ret;
+
+ key_counts = HCLGE_COMM_RSS_KEY_SIZE;
+ req = (struct hclge_comm_rss_config_cmd *)desc.data;
+
+ while (key_counts) {
+ hclge_comm_cmd_setup_basic_desc(&desc,
+ HCLGE_OPC_RSS_GENERIC_CONFIG,
+ false);
+
+ req->hash_config |= (hfunc & HCLGE_COMM_RSS_HASH_ALGO_MASK);
+ req->hash_config |=
+ (key_offset << HCLGE_COMM_RSS_HASH_KEY_OFFSET_B);
+
+ key_size = min(HCLGE_COMM_RSS_HASH_KEY_NUM, key_counts);
+ memcpy(req->hash_key,
+ key + key_offset * HCLGE_COMM_RSS_HASH_KEY_NUM,
+ key_size);
+
+ key_counts -= key_size;
+ key_offset++;
+ ret = hclge_comm_cmd_send(hw, &desc, 1);
+ if (ret) {
+ dev_err(&hw->cmq.csq.pdev->dev,
+ "failed to configure RSS key, ret = %d.\n",
+ ret);
+ return ret;
+ }
+ }
+
+ return 0;
+}
+
+static u8 hclge_comm_get_rss_hash_bits(struct ethtool_rxnfc *nfc)
+{
+ u8 hash_sets = nfc->data & RXH_L4_B_0_1 ? HCLGE_COMM_S_PORT_BIT : 0;
+
+ if (nfc->data & RXH_L4_B_2_3)
+ hash_sets |= HCLGE_COMM_D_PORT_BIT;
+ else
+ hash_sets &= ~HCLGE_COMM_D_PORT_BIT;
+
+ if (nfc->data & RXH_IP_SRC)
+ hash_sets |= HCLGE_COMM_S_IP_BIT;
+ else
+ hash_sets &= ~HCLGE_COMM_S_IP_BIT;
+
+ if (nfc->data & RXH_IP_DST)
+ hash_sets |= HCLGE_COMM_D_IP_BIT;
+ else
+ hash_sets &= ~HCLGE_COMM_D_IP_BIT;
+
+ if (nfc->flow_type == SCTP_V4_FLOW || nfc->flow_type == SCTP_V6_FLOW)
+ hash_sets |= HCLGE_COMM_V_TAG_BIT;
+
+ return hash_sets;
+}
+
+int hclge_comm_init_rss_tuple_cmd(struct hclge_comm_rss_cfg *rss_cfg,
+ struct ethtool_rxnfc *nfc,
+ struct hnae3_ae_dev *ae_dev,
+ struct hclge_comm_rss_input_tuple_cmd *req)
+{
+ u8 tuple_sets;
+
+ req->ipv4_tcp_en = rss_cfg->rss_tuple_sets.ipv4_tcp_en;
+ req->ipv4_udp_en = rss_cfg->rss_tuple_sets.ipv4_udp_en;
+ req->ipv4_sctp_en = rss_cfg->rss_tuple_sets.ipv4_sctp_en;
+ req->ipv4_fragment_en = rss_cfg->rss_tuple_sets.ipv4_fragment_en;
+ req->ipv6_tcp_en = rss_cfg->rss_tuple_sets.ipv6_tcp_en;
+ req->ipv6_udp_en = rss_cfg->rss_tuple_sets.ipv6_udp_en;
+ req->ipv6_sctp_en = rss_cfg->rss_tuple_sets.ipv6_sctp_en;
+ req->ipv6_fragment_en = rss_cfg->rss_tuple_sets.ipv6_fragment_en;
+
+ tuple_sets = hclge_comm_get_rss_hash_bits(nfc);
+ switch (nfc->flow_type) {
+ case TCP_V4_FLOW:
+ req->ipv4_tcp_en = tuple_sets;
+ break;
+ case TCP_V6_FLOW:
+ req->ipv6_tcp_en = tuple_sets;
+ break;
+ case UDP_V4_FLOW:
+ req->ipv4_udp_en = tuple_sets;
+ break;
+ case UDP_V6_FLOW:
+ req->ipv6_udp_en = tuple_sets;
+ break;
+ case SCTP_V4_FLOW:
+ req->ipv4_sctp_en = tuple_sets;
+ break;
+ case SCTP_V6_FLOW:
+ if (ae_dev->dev_version <= HNAE3_DEVICE_VERSION_V2 &&
+ (nfc->data & (RXH_L4_B_0_1 | RXH_L4_B_2_3)))
+ return -EINVAL;
+
+ req->ipv6_sctp_en = tuple_sets;
+ break;
+ case IPV4_FLOW:
+ req->ipv4_fragment_en = HCLGE_COMM_RSS_INPUT_TUPLE_OTHER;
+ break;
+ case IPV6_FLOW:
+ req->ipv6_fragment_en = HCLGE_COMM_RSS_INPUT_TUPLE_OTHER;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+u64 hclge_comm_convert_rss_tuple(u8 tuple_sets)
+{
+ u64 tuple_data = 0;
+
+ if (tuple_sets & HCLGE_COMM_D_PORT_BIT)
+ tuple_data |= RXH_L4_B_2_3;
+ if (tuple_sets & HCLGE_COMM_S_PORT_BIT)
+ tuple_data |= RXH_L4_B_0_1;
+ if (tuple_sets & HCLGE_COMM_D_IP_BIT)
+ tuple_data |= RXH_IP_DST;
+ if (tuple_sets & HCLGE_COMM_S_IP_BIT)
+ tuple_data |= RXH_IP_SRC;
+
+ return tuple_data;
+}
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_rss.h b/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_rss.h
new file mode 100644
index 0000000..aa1d7a6
--- /dev/null
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_rss.h
@@ -0,0 +1,136 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+// Copyright (c) 2021-2021 Hisilicon Limited.
+
+#ifndef __HCLGE_COMM_RSS_H
+#define __HCLGE_COMM_RSS_H
+#include <linux/types.h>
+
+#include "hnae3.h"
+#include "hclge_comm_cmd.h"
+
+#define HCLGE_COMM_RSS_HASH_ALGO_TOEPLITZ 0
+#define HCLGE_COMM_RSS_HASH_ALGO_SIMPLE 1
+#define HCLGE_COMM_RSS_HASH_ALGO_SYMMETRIC 2
+
+#define HCLGE_COMM_RSS_INPUT_TUPLE_OTHER GENMASK(3, 0)
+#define HCLGE_COMM_RSS_INPUT_TUPLE_SCTP GENMASK(4, 0)
+
+#define HCLGE_COMM_D_PORT_BIT BIT(0)
+#define HCLGE_COMM_S_PORT_BIT BIT(1)
+#define HCLGE_COMM_D_IP_BIT BIT(2)
+#define HCLGE_COMM_S_IP_BIT BIT(3)
+#define HCLGE_COMM_V_TAG_BIT BIT(4)
+#define HCLGE_COMM_RSS_INPUT_TUPLE_SCTP_NO_PORT \
+ (HCLGE_COMM_D_IP_BIT | HCLGE_COMM_S_IP_BIT | HCLGE_COMM_V_TAG_BIT)
+#define HCLGE_COMM_MAX_TC_NUM 8
+
+#define HCLGE_COMM_RSS_TC_OFFSET_S 0
+#define HCLGE_COMM_RSS_TC_OFFSET_M GENMASK(10, 0)
+#define HCLGE_COMM_RSS_TC_SIZE_MSB_B 11
+#define HCLGE_COMM_RSS_TC_SIZE_S 12
+#define HCLGE_COMM_RSS_TC_SIZE_M GENMASK(14, 12)
+#define HCLGE_COMM_RSS_TC_VALID_B 15
+#define HCLGE_COMM_RSS_TC_SIZE_MSB_OFFSET 3
+
+struct hclge_comm_rss_tuple_cfg {
+ u8 ipv4_tcp_en;
+ u8 ipv4_udp_en;
+ u8 ipv4_sctp_en;
+ u8 ipv4_fragment_en;
+ u8 ipv6_tcp_en;
+ u8 ipv6_udp_en;
+ u8 ipv6_sctp_en;
+ u8 ipv6_fragment_en;
+};
+
+#define HCLGE_COMM_RSS_KEY_SIZE 40
+#define HCLGE_COMM_RSS_CFG_TBL_SIZE 16
+#define HCLGE_COMM_RSS_CFG_TBL_BW_H 2U
+#define HCLGE_COMM_RSS_CFG_TBL_BW_L 8U
+#define HCLGE_COMM_RSS_CFG_TBL_SIZE_H 4
+#define HCLGE_COMM_RSS_SET_BITMAP_MSK GENMASK(15, 0)
+#define HCLGE_COMM_RSS_HASH_ALGO_MASK GENMASK(3, 0)
+#define HCLGE_COMM_RSS_HASH_KEY_OFFSET_B 4
+
+#define HCLGE_COMM_RSS_HASH_KEY_NUM 16
+struct hclge_comm_rss_config_cmd {
+ u8 hash_config;
+ u8 rsv[7];
+ u8 hash_key[HCLGE_COMM_RSS_HASH_KEY_NUM];
+};
+
+struct hclge_comm_rss_cfg {
+ u8 rss_hash_key[HCLGE_COMM_RSS_KEY_SIZE]; /* user configured hash keys */
+
+ /* shadow table */
+ u16 *rss_indirection_tbl;
+ u32 rss_algo;
+
+ struct hclge_comm_rss_tuple_cfg rss_tuple_sets;
+ u32 rss_size;
+};
+
+struct hclge_comm_rss_input_tuple_cmd {
+ u8 ipv4_tcp_en;
+ u8 ipv4_udp_en;
+ u8 ipv4_sctp_en;
+ u8 ipv4_fragment_en;
+ u8 ipv6_tcp_en;
+ u8 ipv6_udp_en;
+ u8 ipv6_sctp_en;
+ u8 ipv6_fragment_en;
+ u8 rsv[16];
+};
+
+struct hclge_comm_rss_ind_tbl_cmd {
+ __le16 start_table_index;
+ __le16 rss_set_bitmap;
+ u8 rss_qid_h[HCLGE_COMM_RSS_CFG_TBL_SIZE_H];
+ u8 rss_qid_l[HCLGE_COMM_RSS_CFG_TBL_SIZE];
+};
+
+struct hclge_comm_rss_tc_mode_cmd {
+ __le16 rss_tc_mode[HCLGE_COMM_MAX_TC_NUM];
+ u8 rsv[8];
+};
+
+u32 hclge_comm_get_rss_key_size(struct hnae3_handle *handle);
+void hclge_comm_get_rss_type(struct hnae3_handle *nic,
+ struct hclge_comm_rss_tuple_cfg *rss_tuple_sets);
+void hclge_comm_rss_indir_init_cfg(struct hnae3_ae_dev *ae_dev,
+ struct hclge_comm_rss_cfg *rss_cfg);
+int hclge_comm_get_rss_tuple(struct hclge_comm_rss_cfg *rss_cfg, int flow_type,
+ u8 *tuple_sets);
+int hclge_comm_parse_rss_hfunc(struct hclge_comm_rss_cfg *rss_cfg,
+ const u8 hfunc, u8 *hash_algo);
+void hclge_comm_get_rss_hash_info(struct hclge_comm_rss_cfg *rss_cfg, u8 *key,
+ u8 *hfunc);
+void hclge_comm_get_rss_indir_tbl(struct hclge_comm_rss_cfg *rss_cfg,
+ u32 *indir, __le16 rss_ind_tbl_size);
+int hclge_comm_set_rss_algo_key(struct hclge_comm_hw *hw, const u8 hfunc,
+ const u8 *key);
+int hclge_comm_init_rss_tuple_cmd(struct hclge_comm_rss_cfg *rss_cfg,
+ struct ethtool_rxnfc *nfc,
+ struct hnae3_ae_dev *ae_dev,
+ struct hclge_comm_rss_input_tuple_cmd *req);
+u64 hclge_comm_convert_rss_tuple(u8 tuple_sets);
+int hclge_comm_set_rss_input_tuple(struct hnae3_handle *nic,
+ struct hclge_comm_hw *hw, bool is_pf,
+ struct hclge_comm_rss_cfg *rss_cfg);
+int hclge_comm_set_rss_indir_table(struct hnae3_ae_dev *ae_dev,
+ struct hclge_comm_hw *hw, const u16 *indir);
+int hclge_comm_rss_init_cfg(struct hnae3_handle *nic,
+ struct hnae3_ae_dev *ae_dev,
+ struct hclge_comm_rss_cfg *rss_cfg);
+void hclge_comm_get_rss_tc_info(u16 rss_size, u8 hw_tc_map, u16 *tc_offset,
+ u16 *tc_valid, u16 *tc_size);
+int hclge_comm_set_rss_tc_mode(struct hclge_comm_hw *hw, u16 *tc_offset,
+ u16 *tc_valid, u16 *tc_size);
+int hclge_comm_set_rss_hash_key(struct hclge_comm_rss_cfg *rss_cfg,
+ struct hclge_comm_hw *hw, const u8 *key,
+ const u8 hfunc);
+int hclge_comm_set_rss_tuple(struct hnae3_ae_dev *ae_dev,
+ struct hclge_comm_hw *hw,
+ struct hclge_comm_rss_cfg *rss_cfg,
+ struct ethtool_rxnfc *nfc);
+#endif
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_tqp_stats.c b/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_tqp_stats.c
new file mode 100644
index 0000000..0c60f41
--- /dev/null
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_tqp_stats.c
@@ -0,0 +1,115 @@
+// SPDX-License-Identifier: GPL-2.0+
+// Copyright (c) 2021-2021 Hisilicon Limited.
+
+#include <linux/err.h>
+
+#include "hnae3.h"
+#include "hclge_comm_cmd.h"
+#include "hclge_comm_tqp_stats.h"
+
+u64 *hclge_comm_tqps_get_stats(struct hnae3_handle *handle, u64 *data)
+{
+ struct hnae3_knic_private_info *kinfo = &handle->kinfo;
+ struct hclge_comm_tqp *tqp;
+ u64 *buff = data;
+ u16 i;
+
+ for (i = 0; i < kinfo->num_tqps; i++) {
+ tqp = container_of(kinfo->tqp[i], struct hclge_comm_tqp, q);
+ *buff++ = tqp->tqp_stats.rcb_tx_ring_pktnum_rcd;
+ }
+
+ for (i = 0; i < kinfo->num_tqps; i++) {
+ tqp = container_of(kinfo->tqp[i], struct hclge_comm_tqp, q);
+ *buff++ = tqp->tqp_stats.rcb_rx_ring_pktnum_rcd;
+ }
+
+ return buff;
+}
+
+int hclge_comm_tqps_get_sset_count(struct hnae3_handle *handle)
+{
+ struct hnae3_knic_private_info *kinfo = &handle->kinfo;
+
+ return kinfo->num_tqps * HCLGE_COMM_QUEUE_PAIR_SIZE;
+}
+
+u8 *hclge_comm_tqps_get_strings(struct hnae3_handle *handle, u8 *data)
+{
+ struct hnae3_knic_private_info *kinfo = &handle->kinfo;
+ u8 *buff = data;
+ u16 i;
+
+ for (i = 0; i < kinfo->num_tqps; i++) {
+ struct hclge_comm_tqp *tqp =
+ container_of(kinfo->tqp[i], struct hclge_comm_tqp, q);
+ snprintf(buff, ETH_GSTRING_LEN, "txq%u_pktnum_rcd", tqp->index);
+ buff += ETH_GSTRING_LEN;
+ }
+
+ for (i = 0; i < kinfo->num_tqps; i++) {
+ struct hclge_comm_tqp *tqp =
+ container_of(kinfo->tqp[i], struct hclge_comm_tqp, q);
+ snprintf(buff, ETH_GSTRING_LEN, "rxq%u_pktnum_rcd", tqp->index);
+ buff += ETH_GSTRING_LEN;
+ }
+
+ return buff;
+}
+
+int hclge_comm_tqps_update_stats(struct hnae3_handle *handle,
+ struct hclge_comm_hw *hw)
+{
+ struct hnae3_knic_private_info *kinfo = &handle->kinfo;
+ struct hclge_comm_tqp *tqp;
+ struct hclge_desc desc;
+ int ret;
+ u16 i;
+
+ for (i = 0; i < kinfo->num_tqps; i++) {
+ tqp = container_of(kinfo->tqp[i], struct hclge_comm_tqp, q);
+ hclge_comm_cmd_setup_basic_desc(&desc, HCLGE_OPC_QUERY_RX_STATS,
+ true);
+
+ desc.data[0] = cpu_to_le32(tqp->index);
+ ret = hclge_comm_cmd_send(hw, &desc, 1);
+ if (ret) {
+ dev_err(&hw->cmq.csq.pdev->dev,
+ "failed to get tqp stat, ret = %d, tx = %u.\n",
+ ret, i);
+ return ret;
+ }
+ tqp->tqp_stats.rcb_rx_ring_pktnum_rcd +=
+ le32_to_cpu(desc.data[1]);
+
+ hclge_comm_cmd_setup_basic_desc(&desc, HCLGE_OPC_QUERY_TX_STATS,
+ true);
+
+ desc.data[0] = cpu_to_le32(tqp->index & 0x1ff);
+ ret = hclge_comm_cmd_send(hw, &desc, 1);
+ if (ret) {
+ dev_err(&hw->cmq.csq.pdev->dev,
+ "failed to get tqp stat, ret = %d, rx = %u.\n",
+ ret, i);
+ return ret;
+ }
+ tqp->tqp_stats.rcb_tx_ring_pktnum_rcd +=
+ le32_to_cpu(desc.data[1]);
+ }
+
+ return 0;
+}
+
+void hclge_comm_reset_tqp_stats(struct hnae3_handle *handle)
+{
+ struct hnae3_knic_private_info *kinfo = &handle->kinfo;
+ struct hclge_comm_tqp *tqp;
+ struct hnae3_queue *queue;
+ u16 i;
+
+ for (i = 0; i < kinfo->num_tqps; i++) {
+ queue = kinfo->tqp[i];
+ tqp = container_of(queue, struct hclge_comm_tqp, q);
+ memset(&tqp->tqp_stats, 0, sizeof(tqp->tqp_stats));
+ }
+}
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_tqp_stats.h b/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_tqp_stats.h
new file mode 100644
index 0000000..a463501
--- /dev/null
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_tqp_stats.h
@@ -0,0 +1,39 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+// Copyright (c) 2021-2021 Hisilicon Limited.
+
+#ifndef __HCLGE_COMM_TQP_STATS_H
+#define __HCLGE_COMM_TQP_STATS_H
+#include <linux/types.h>
+#include <linux/etherdevice.h>
+#include "hnae3.h"
+
+/* each tqp has TX & RX two queues */
+#define HCLGE_COMM_QUEUE_PAIR_SIZE 2
+
+/* TQP stats */
+struct hclge_comm_tqp_stats {
+ /* query_tqp_tx_queue_statistics ,opcode id: 0x0B03 */
+ u64 rcb_tx_ring_pktnum_rcd; /* 32bit */
+ /* query_tqp_rx_queue_statistics ,opcode id: 0x0B13 */
+ u64 rcb_rx_ring_pktnum_rcd; /* 32bit */
+};
+
+struct hclge_comm_tqp {
+ /* copy of device pointer from pci_dev,
+ * used when perform DMA mapping
+ */
+ struct device *dev;
+ struct hnae3_queue q;
+ struct hclge_comm_tqp_stats tqp_stats;
+ u16 index; /* Global index in a NIC controller */
+
+ bool alloced;
+};
+
+u64 *hclge_comm_tqps_get_stats(struct hnae3_handle *handle, u64 *data);
+int hclge_comm_tqps_get_sset_count(struct hnae3_handle *handle);
+u8 *hclge_comm_tqps_get_strings(struct hnae3_handle *handle, u8 *data);
+void hclge_comm_reset_tqp_stats(struct hnae3_handle *handle);
+int hclge_comm_tqps_update_stats(struct hnae3_handle *handle,
+ struct hclge_comm_hw *hw);
+#endif
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3_debugfs.c b/drivers/net/ethernet/hisilicon/hns3/hns3_debugfs.c
index c381f8a..f726a5b 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3_debugfs.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3_debugfs.c
@@ -1083,7 +1083,7 @@ static void hns3_dump_page_pool_info(struct hns3_enet_ring *ring,
sprintf(result[j++], "%u", index);
sprintf(result[j++], "%u",
READ_ONCE(ring->page_pool->pages_state_hold_cnt));
- sprintf(result[j++], "%u",
+ sprintf(result[j++], "%d",
atomic_read(&ring->page_pool->pages_state_release_cnt));
sprintf(result[j++], "%u", ring->page_pool->p.pool_size);
sprintf(result[j++], "%u", ring->page_pool->p.order);
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3_debugfs.h b/drivers/net/ethernet/hisilicon/hns3/hns3_debugfs.h
index bd88010..83aa145 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3_debugfs.h
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3_debugfs.h
@@ -4,6 +4,8 @@
#ifndef __HNS3_DEBUGFS_H
#define __HNS3_DEBUGFS_H
+#include "hnae3.h"
+
#define HNS3_DBG_READ_LEN 65536
#define HNS3_DBG_READ_LEN_128KB 0x20000
#define HNS3_DBG_READ_LEN_1MB 0x100000
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c b/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c
index 9ccebba..babc5d7 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c
@@ -17,6 +17,7 @@
#include <linux/skbuff.h>
#include <linux/sctp.h>
#include <net/gre.h>
+#include <net/gro.h>
#include <net/ip6_checksum.h>
#include <net/pkt_cls.h>
#include <net/tcp.h>
@@ -53,10 +54,6 @@ static int debug = -1;
module_param(debug, int, 0);
MODULE_PARM_DESC(debug, " Network interface message level setting");
-static unsigned int tx_spare_buf_size;
-module_param(tx_spare_buf_size, uint, 0400);
-MODULE_PARM_DESC(tx_spare_buf_size, "Size used to allocate tx spare buffer");
-
static unsigned int tx_sgl = 1;
module_param(tx_sgl, uint, 0600);
MODULE_PARM_DESC(tx_sgl, "Minimum number of frags when using dma_map_sg() to optimize the IOMMU mapping");
@@ -1005,9 +1002,7 @@ static bool hns3_can_use_tx_bounce(struct hns3_enet_ring *ring,
return false;
if (ALIGN(len, dma_get_cache_alignment()) > space) {
- u64_stats_update_begin(&ring->syncp);
- ring->stats.tx_spare_full++;
- u64_stats_update_end(&ring->syncp);
+ hns3_ring_stats_update(ring, tx_spare_full);
return false;
}
@@ -1024,9 +1019,7 @@ static bool hns3_can_use_tx_sgl(struct hns3_enet_ring *ring,
return false;
if (space < HNS3_MAX_SGL_SIZE) {
- u64_stats_update_begin(&ring->syncp);
- ring->stats.tx_spare_full++;
- u64_stats_update_end(&ring->syncp);
+ hns3_ring_stats_update(ring, tx_spare_full);
return false;
}
@@ -1041,8 +1034,7 @@ static void hns3_init_tx_spare_buffer(struct hns3_enet_ring *ring)
dma_addr_t dma;
int order;
- alloc_size = tx_spare_buf_size ? tx_spare_buf_size :
- ring->tqp->handle->kinfo.tx_spare_buf_size;
+ alloc_size = ring->tqp->handle->kinfo.tx_spare_buf_size;
if (!alloc_size)
return;
@@ -1306,7 +1298,7 @@ static bool hns3_tunnel_csum_bug(struct sk_buff *skb)
if (!(!skb->encapsulation &&
(l4.udp->dest == htons(IANA_VXLAN_UDP_PORT) ||
l4.udp->dest == htons(GENEVE_UDP_PORT) ||
- l4.udp->dest == htons(4790))))
+ l4.udp->dest == htons(IANA_VXLAN_GPE_UDP_PORT))))
return false;
return true;
@@ -1359,44 +1351,9 @@ static void hns3_set_outer_l2l3l4(struct sk_buff *skb, u8 ol4_proto,
HNS3_TUN_NVGRE);
}
-static int hns3_set_l2l3l4(struct sk_buff *skb, u8 ol4_proto,
- u8 il4_proto, u32 *type_cs_vlan_tso,
- u32 *ol_type_vlan_len_msec)
+static void hns3_set_l3_type(struct sk_buff *skb, union l3_hdr_info l3,
+ u32 *type_cs_vlan_tso)
{
- unsigned char *l2_hdr = skb->data;
- u32 l4_proto = ol4_proto;
- union l4_hdr_info l4;
- union l3_hdr_info l3;
- u32 l2_len, l3_len;
-
- l4.hdr = skb_transport_header(skb);
- l3.hdr = skb_network_header(skb);
-
- /* handle encapsulation skb */
- if (skb->encapsulation) {
- /* If this is a not UDP/GRE encapsulation skb */
- if (!(ol4_proto == IPPROTO_UDP || ol4_proto == IPPROTO_GRE)) {
- /* drop the skb tunnel packet if hardware don't support,
- * because hardware can't calculate csum when TSO.
- */
- if (skb_is_gso(skb))
- return -EDOM;
-
- /* the stack computes the IP header already,
- * driver calculate l4 checksum when not TSO.
- */
- return skb_checksum_help(skb);
- }
-
- hns3_set_outer_l2l3l4(skb, ol4_proto, ol_type_vlan_len_msec);
-
- /* switch to inner header */
- l2_hdr = skb_inner_mac_header(skb);
- l3.hdr = skb_inner_network_header(skb);
- l4.hdr = skb_inner_transport_header(skb);
- l4_proto = il4_proto;
- }
-
if (l3.v4->version == 4) {
hns3_set_field(*type_cs_vlan_tso, HNS3_TXD_L3T_S,
HNS3_L3T_IPV4);
@@ -1410,15 +1367,11 @@ static int hns3_set_l2l3l4(struct sk_buff *skb, u8 ol4_proto,
hns3_set_field(*type_cs_vlan_tso, HNS3_TXD_L3T_S,
HNS3_L3T_IPV6);
}
+}
- /* compute inner(/normal) L2 header size, defined in 2 Bytes */
- l2_len = l3.hdr - l2_hdr;
- hns3_set_field(*type_cs_vlan_tso, HNS3_TXD_L2LEN_S, l2_len >> 1);
-
- /* compute inner(/normal) L3 header size, defined in 4 Bytes */
- l3_len = l4.hdr - l3.hdr;
- hns3_set_field(*type_cs_vlan_tso, HNS3_TXD_L3LEN_S, l3_len >> 2);
-
+static int hns3_set_l4_csum_length(struct sk_buff *skb, union l4_hdr_info l4,
+ u32 l4_proto, u32 *type_cs_vlan_tso)
+{
/* compute inner(/normal) L4 header size, defined in 4 Bytes */
switch (l4_proto) {
case IPPROTO_TCP:
@@ -1464,6 +1417,57 @@ static int hns3_set_l2l3l4(struct sk_buff *skb, u8 ol4_proto,
return 0;
}
+static int hns3_set_l2l3l4(struct sk_buff *skb, u8 ol4_proto,
+ u8 il4_proto, u32 *type_cs_vlan_tso,
+ u32 *ol_type_vlan_len_msec)
+{
+ unsigned char *l2_hdr = skb->data;
+ u32 l4_proto = ol4_proto;
+ union l4_hdr_info l4;
+ union l3_hdr_info l3;
+ u32 l2_len, l3_len;
+
+ l4.hdr = skb_transport_header(skb);
+ l3.hdr = skb_network_header(skb);
+
+ /* handle encapsulation skb */
+ if (skb->encapsulation) {
+ /* If this is a not UDP/GRE encapsulation skb */
+ if (!(ol4_proto == IPPROTO_UDP || ol4_proto == IPPROTO_GRE)) {
+ /* drop the skb tunnel packet if hardware don't support,
+ * because hardware can't calculate csum when TSO.
+ */
+ if (skb_is_gso(skb))
+ return -EDOM;
+
+ /* the stack computes the IP header already,
+ * driver calculate l4 checksum when not TSO.
+ */
+ return skb_checksum_help(skb);
+ }
+
+ hns3_set_outer_l2l3l4(skb, ol4_proto, ol_type_vlan_len_msec);
+
+ /* switch to inner header */
+ l2_hdr = skb_inner_mac_header(skb);
+ l3.hdr = skb_inner_network_header(skb);
+ l4.hdr = skb_inner_transport_header(skb);
+ l4_proto = il4_proto;
+ }
+
+ hns3_set_l3_type(skb, l3, type_cs_vlan_tso);
+
+ /* compute inner(/normal) L2 header size, defined in 2 Bytes */
+ l2_len = l3.hdr - l2_hdr;
+ hns3_set_field(*type_cs_vlan_tso, HNS3_TXD_L2LEN_S, l2_len >> 1);
+
+ /* compute inner(/normal) L3 header size, defined in 4 Bytes */
+ l3_len = l4.hdr - l3.hdr;
+ hns3_set_field(*type_cs_vlan_tso, HNS3_TXD_L3LEN_S, l3_len >> 2);
+
+ return hns3_set_l4_csum_length(skb, l4, l4_proto, type_cs_vlan_tso);
+}
+
static int hns3_handle_vtags(struct hns3_enet_ring *tx_ring,
struct sk_buff *skb)
{
@@ -1540,92 +1544,122 @@ static bool hns3_check_hw_tx_csum(struct sk_buff *skb)
return true;
}
-static int hns3_fill_skb_desc(struct hns3_enet_ring *ring,
- struct sk_buff *skb, struct hns3_desc *desc,
- struct hns3_desc_cb *desc_cb)
+struct hns3_desc_param {
+ u32 paylen_ol4cs;
+ u32 ol_type_vlan_len_msec;
+ u32 type_cs_vlan_tso;
+ u16 mss_hw_csum;
+ u16 inner_vtag;
+ u16 out_vtag;
+};
+
+static void hns3_init_desc_data(struct sk_buff *skb, struct hns3_desc_param *pa)
{
- u32 ol_type_vlan_len_msec = 0;
- u32 paylen_ol4cs = skb->len;
- u32 type_cs_vlan_tso = 0;
- u16 mss_hw_csum = 0;
- u16 inner_vtag = 0;
- u16 out_vtag = 0;
+ pa->paylen_ol4cs = skb->len;
+ pa->ol_type_vlan_len_msec = 0;
+ pa->type_cs_vlan_tso = 0;
+ pa->mss_hw_csum = 0;
+ pa->inner_vtag = 0;
+ pa->out_vtag = 0;
+}
+
+static int hns3_handle_vlan_info(struct hns3_enet_ring *ring,
+ struct sk_buff *skb,
+ struct hns3_desc_param *param)
+{
int ret;
ret = hns3_handle_vtags(ring, skb);
if (unlikely(ret < 0)) {
- u64_stats_update_begin(&ring->syncp);
- ring->stats.tx_vlan_err++;
- u64_stats_update_end(&ring->syncp);
+ hns3_ring_stats_update(ring, tx_vlan_err);
return ret;
} else if (ret == HNS3_INNER_VLAN_TAG) {
- inner_vtag = skb_vlan_tag_get(skb);
- inner_vtag |= (skb->priority << VLAN_PRIO_SHIFT) &
+ param->inner_vtag = skb_vlan_tag_get(skb);
+ param->inner_vtag |= (skb->priority << VLAN_PRIO_SHIFT) &
VLAN_PRIO_MASK;
- hns3_set_field(type_cs_vlan_tso, HNS3_TXD_VLAN_B, 1);
+ hns3_set_field(param->type_cs_vlan_tso, HNS3_TXD_VLAN_B, 1);
} else if (ret == HNS3_OUTER_VLAN_TAG) {
- out_vtag = skb_vlan_tag_get(skb);
- out_vtag |= (skb->priority << VLAN_PRIO_SHIFT) &
+ param->out_vtag = skb_vlan_tag_get(skb);
+ param->out_vtag |= (skb->priority << VLAN_PRIO_SHIFT) &
VLAN_PRIO_MASK;
- hns3_set_field(ol_type_vlan_len_msec, HNS3_TXD_OVLAN_B,
+ hns3_set_field(param->ol_type_vlan_len_msec, HNS3_TXD_OVLAN_B,
1);
}
+ return 0;
+}
+
+static int hns3_handle_csum_partial(struct hns3_enet_ring *ring,
+ struct sk_buff *skb,
+ struct hns3_desc_cb *desc_cb,
+ struct hns3_desc_param *param)
+{
+ u8 ol4_proto, il4_proto;
+ int ret;
+
+ if (hns3_check_hw_tx_csum(skb)) {
+ /* set checksum start and offset, defined in 2 Bytes */
+ hns3_set_field(param->type_cs_vlan_tso, HNS3_TXD_CSUM_START_S,
+ skb_checksum_start_offset(skb) >> 1);
+ hns3_set_field(param->ol_type_vlan_len_msec,
+ HNS3_TXD_CSUM_OFFSET_S,
+ skb->csum_offset >> 1);
+ param->mss_hw_csum |= BIT(HNS3_TXD_HW_CS_B);
+ return 0;
+ }
+
+ skb_reset_mac_len(skb);
+
+ ret = hns3_get_l4_protocol(skb, &ol4_proto, &il4_proto);
+ if (unlikely(ret < 0)) {
+ hns3_ring_stats_update(ring, tx_l4_proto_err);
+ return ret;
+ }
+
+ ret = hns3_set_l2l3l4(skb, ol4_proto, il4_proto,
+ ¶m->type_cs_vlan_tso,
+ ¶m->ol_type_vlan_len_msec);
+ if (unlikely(ret < 0)) {
+ hns3_ring_stats_update(ring, tx_l2l3l4_err);
+ return ret;
+ }
+
+ ret = hns3_set_tso(skb, ¶m->paylen_ol4cs, ¶m->mss_hw_csum,
+ ¶m->type_cs_vlan_tso, &desc_cb->send_bytes);
+ if (unlikely(ret < 0)) {
+ hns3_ring_stats_update(ring, tx_tso_err);
+ return ret;
+ }
+ return 0;
+}
+
+static int hns3_fill_skb_desc(struct hns3_enet_ring *ring,
+ struct sk_buff *skb, struct hns3_desc *desc,
+ struct hns3_desc_cb *desc_cb)
+{
+ struct hns3_desc_param param;
+ int ret;
+
+ hns3_init_desc_data(skb, ¶m);
+ ret = hns3_handle_vlan_info(ring, skb, ¶m);
+ if (unlikely(ret < 0))
+ return ret;
desc_cb->send_bytes = skb->len;
if (skb->ip_summed == CHECKSUM_PARTIAL) {
- u8 ol4_proto, il4_proto;
-
- if (hns3_check_hw_tx_csum(skb)) {
- /* set checksum start and offset, defined in 2 Bytes */
- hns3_set_field(type_cs_vlan_tso, HNS3_TXD_CSUM_START_S,
- skb_checksum_start_offset(skb) >> 1);
- hns3_set_field(ol_type_vlan_len_msec,
- HNS3_TXD_CSUM_OFFSET_S,
- skb->csum_offset >> 1);
- mss_hw_csum |= BIT(HNS3_TXD_HW_CS_B);
- goto out_hw_tx_csum;
- }
-
- skb_reset_mac_len(skb);
-
- ret = hns3_get_l4_protocol(skb, &ol4_proto, &il4_proto);
- if (unlikely(ret < 0)) {
- u64_stats_update_begin(&ring->syncp);
- ring->stats.tx_l4_proto_err++;
- u64_stats_update_end(&ring->syncp);
+ ret = hns3_handle_csum_partial(ring, skb, desc_cb, ¶m);
+ if (ret)
return ret;
- }
-
- ret = hns3_set_l2l3l4(skb, ol4_proto, il4_proto,
- &type_cs_vlan_tso,
- &ol_type_vlan_len_msec);
- if (unlikely(ret < 0)) {
- u64_stats_update_begin(&ring->syncp);
- ring->stats.tx_l2l3l4_err++;
- u64_stats_update_end(&ring->syncp);
- return ret;
- }
-
- ret = hns3_set_tso(skb, &paylen_ol4cs, &mss_hw_csum,
- &type_cs_vlan_tso, &desc_cb->send_bytes);
- if (unlikely(ret < 0)) {
- u64_stats_update_begin(&ring->syncp);
- ring->stats.tx_tso_err++;
- u64_stats_update_end(&ring->syncp);
- return ret;
- }
}
-out_hw_tx_csum:
/* Set txbd */
desc->tx.ol_type_vlan_len_msec =
- cpu_to_le32(ol_type_vlan_len_msec);
- desc->tx.type_cs_vlan_tso_len = cpu_to_le32(type_cs_vlan_tso);
- desc->tx.paylen_ol4cs = cpu_to_le32(paylen_ol4cs);
- desc->tx.mss_hw_csum = cpu_to_le16(mss_hw_csum);
- desc->tx.vlan_tag = cpu_to_le16(inner_vtag);
- desc->tx.outer_vlan_tag = cpu_to_le16(out_vtag);
+ cpu_to_le32(param.ol_type_vlan_len_msec);
+ desc->tx.type_cs_vlan_tso_len = cpu_to_le32(param.type_cs_vlan_tso);
+ desc->tx.paylen_ol4cs = cpu_to_le32(param.paylen_ol4cs);
+ desc->tx.mss_hw_csum = cpu_to_le16(param.mss_hw_csum);
+ desc->tx.vlan_tag = cpu_to_le16(param.inner_vtag);
+ desc->tx.outer_vlan_tag = cpu_to_le16(param.out_vtag);
return 0;
}
@@ -1705,9 +1739,7 @@ static int hns3_map_and_fill_desc(struct hns3_enet_ring *ring, void *priv,
}
if (unlikely(dma_mapping_error(dev, dma))) {
- u64_stats_update_begin(&ring->syncp);
- ring->stats.sw_err_cnt++;
- u64_stats_update_end(&ring->syncp);
+ hns3_ring_stats_update(ring, sw_err_cnt);
return -ENOMEM;
}
@@ -1853,9 +1885,7 @@ static int hns3_skb_linearize(struct hns3_enet_ring *ring,
* recursion level of over HNS3_MAX_RECURSION_LEVEL.
*/
if (bd_num == UINT_MAX) {
- u64_stats_update_begin(&ring->syncp);
- ring->stats.over_max_recursion++;
- u64_stats_update_end(&ring->syncp);
+ hns3_ring_stats_update(ring, over_max_recursion);
return -ENOMEM;
}
@@ -1864,16 +1894,12 @@ static int hns3_skb_linearize(struct hns3_enet_ring *ring,
*/
if (skb->len > HNS3_MAX_TSO_SIZE ||
(!skb_is_gso(skb) && skb->len > HNS3_MAX_NON_TSO_SIZE)) {
- u64_stats_update_begin(&ring->syncp);
- ring->stats.hw_limitation++;
- u64_stats_update_end(&ring->syncp);
+ hns3_ring_stats_update(ring, hw_limitation);
return -ENOMEM;
}
if (__skb_linearize(skb)) {
- u64_stats_update_begin(&ring->syncp);
- ring->stats.sw_err_cnt++;
- u64_stats_update_end(&ring->syncp);
+ hns3_ring_stats_update(ring, sw_err_cnt);
return -ENOMEM;
}
@@ -1903,9 +1929,7 @@ static int hns3_nic_maybe_stop_tx(struct hns3_enet_ring *ring,
bd_num = hns3_tx_bd_count(skb->len);
- u64_stats_update_begin(&ring->syncp);
- ring->stats.tx_copy++;
- u64_stats_update_end(&ring->syncp);
+ hns3_ring_stats_update(ring, tx_copy);
}
out:
@@ -1925,9 +1949,7 @@ static int hns3_nic_maybe_stop_tx(struct hns3_enet_ring *ring,
return bd_num;
}
- u64_stats_update_begin(&ring->syncp);
- ring->stats.tx_busy++;
- u64_stats_update_end(&ring->syncp);
+ hns3_ring_stats_update(ring, tx_busy);
return -EBUSY;
}
@@ -2012,9 +2034,7 @@ static void hns3_tx_doorbell(struct hns3_enet_ring *ring, int num,
ring->pending_buf += num;
if (!doorbell) {
- u64_stats_update_begin(&ring->syncp);
- ring->stats.tx_more++;
- u64_stats_update_end(&ring->syncp);
+ hns3_ring_stats_update(ring, tx_more);
return;
}
@@ -2064,9 +2084,7 @@ static int hns3_handle_tx_bounce(struct hns3_enet_ring *ring,
ret = skb_copy_bits(skb, 0, buf, size);
if (unlikely(ret < 0)) {
hns3_tx_spare_rollback(ring, cb_len);
- u64_stats_update_begin(&ring->syncp);
- ring->stats.copy_bits_err++;
- u64_stats_update_end(&ring->syncp);
+ hns3_ring_stats_update(ring, copy_bits_err);
return ret;
}
@@ -2089,9 +2107,8 @@ static int hns3_handle_tx_bounce(struct hns3_enet_ring *ring,
dma_sync_single_for_device(ring_to_dev(ring), dma, size,
DMA_TO_DEVICE);
- u64_stats_update_begin(&ring->syncp);
- ring->stats.tx_bounce++;
- u64_stats_update_end(&ring->syncp);
+ hns3_ring_stats_update(ring, tx_bounce);
+
return bd_num;
}
@@ -2121,9 +2138,7 @@ static int hns3_handle_tx_sgl(struct hns3_enet_ring *ring,
nents = skb_to_sgvec(skb, sgt->sgl, 0, skb->len);
if (unlikely(nents < 0)) {
hns3_tx_spare_rollback(ring, cb_len);
- u64_stats_update_begin(&ring->syncp);
- ring->stats.skb2sgl_err++;
- u64_stats_update_end(&ring->syncp);
+ hns3_ring_stats_update(ring, skb2sgl_err);
return -ENOMEM;
}
@@ -2132,9 +2147,7 @@ static int hns3_handle_tx_sgl(struct hns3_enet_ring *ring,
DMA_TO_DEVICE);
if (unlikely(!sgt->nents)) {
hns3_tx_spare_rollback(ring, cb_len);
- u64_stats_update_begin(&ring->syncp);
- ring->stats.map_sg_err++;
- u64_stats_update_end(&ring->syncp);
+ hns3_ring_stats_update(ring, map_sg_err);
return -ENOMEM;
}
@@ -2146,10 +2159,7 @@ static int hns3_handle_tx_sgl(struct hns3_enet_ring *ring,
for (i = 0; i < sgt->nents; i++)
bd_num += hns3_fill_desc(ring, sg_dma_address(sgt->sgl + i),
sg_dma_len(sgt->sgl + i));
-
- u64_stats_update_begin(&ring->syncp);
- ring->stats.tx_sgl++;
- u64_stats_update_end(&ring->syncp);
+ hns3_ring_stats_update(ring, tx_sgl);
return bd_num;
}
@@ -2174,23 +2184,45 @@ static int hns3_handle_desc_filling(struct hns3_enet_ring *ring,
return hns3_fill_skb_to_desc(ring, skb, DESC_TYPE_SKB);
}
+static int hns3_handle_skb_desc(struct hns3_enet_ring *ring,
+ struct sk_buff *skb,
+ struct hns3_desc_cb *desc_cb,
+ int next_to_use_head)
+{
+ int ret;
+
+ ret = hns3_fill_skb_desc(ring, skb, &ring->desc[ring->next_to_use],
+ desc_cb);
+ if (unlikely(ret < 0))
+ goto fill_err;
+
+ /* 'ret < 0' means filling error, 'ret == 0' means skb->len is
+ * zero, which is unlikely, and 'ret > 0' means how many tx desc
+ * need to be notified to the hw.
+ */
+ ret = hns3_handle_desc_filling(ring, skb);
+ if (likely(ret > 0))
+ return ret;
+
+fill_err:
+ hns3_clear_desc(ring, next_to_use_head);
+ return ret;
+}
+
netdev_tx_t hns3_nic_net_xmit(struct sk_buff *skb, struct net_device *netdev)
{
struct hns3_nic_priv *priv = netdev_priv(netdev);
struct hns3_enet_ring *ring = &priv->ring[skb->queue_mapping];
struct hns3_desc_cb *desc_cb = &ring->desc_cb[ring->next_to_use];
struct netdev_queue *dev_queue;
- int pre_ntu, next_to_use_head;
+ int pre_ntu, ret;
bool doorbell;
- int ret;
/* Hardware can only handle short frames above 32 bytes */
if (skb_put_padto(skb, HNS3_MIN_TX_LEN)) {
hns3_tx_doorbell(ring, 0, !netdev_xmit_more());
- u64_stats_update_begin(&ring->syncp);
- ring->stats.sw_err_cnt++;
- u64_stats_update_end(&ring->syncp);
+ hns3_ring_stats_update(ring, sw_err_cnt);
return NETDEV_TX_OK;
}
@@ -2209,20 +2241,9 @@ netdev_tx_t hns3_nic_net_xmit(struct sk_buff *skb, struct net_device *netdev)
goto out_err_tx_ok;
}
- next_to_use_head = ring->next_to_use;
-
- ret = hns3_fill_skb_desc(ring, skb, &ring->desc[ring->next_to_use],
- desc_cb);
- if (unlikely(ret < 0))
- goto fill_err;
-
- /* 'ret < 0' means filling error, 'ret == 0' means skb->len is
- * zero, which is unlikely, and 'ret > 0' means how many tx desc
- * need to be notified to the hw.
- */
- ret = hns3_handle_desc_filling(ring, skb);
+ ret = hns3_handle_skb_desc(ring, skb, desc_cb, ring->next_to_use);
if (unlikely(ret <= 0))
- goto fill_err;
+ goto out_err_tx_ok;
pre_ntu = ring->next_to_use ? (ring->next_to_use - 1) :
(ring->desc_num - 1);
@@ -2244,9 +2265,6 @@ netdev_tx_t hns3_nic_net_xmit(struct sk_buff *skb, struct net_device *netdev)
return NETDEV_TX_OK;
-fill_err:
- hns3_clear_desc(ring, next_to_use_head);
-
out_err_tx_ok:
dev_kfree_skb_any(skb);
hns3_tx_doorbell(ring, 0, !netdev_xmit_more());
@@ -2255,6 +2273,8 @@ netdev_tx_t hns3_nic_net_xmit(struct sk_buff *skb, struct net_device *netdev)
static int hns3_nic_net_set_mac_address(struct net_device *netdev, void *p)
{
+ char format_mac_addr_perm[HNAE3_FORMAT_MAC_ADDR_LEN];
+ char format_mac_addr_sa[HNAE3_FORMAT_MAC_ADDR_LEN];
struct hnae3_handle *h = hns3_get_handle(netdev);
struct sockaddr *mac_addr = p;
int ret;
@@ -2263,8 +2283,9 @@ static int hns3_nic_net_set_mac_address(struct net_device *netdev, void *p)
return -EADDRNOTAVAIL;
if (ether_addr_equal(netdev->dev_addr, mac_addr->sa_data)) {
- netdev_info(netdev, "already using mac address %pM\n",
- mac_addr->sa_data);
+ hnae3_format_mac_addr(format_mac_addr_sa, mac_addr->sa_data);
+ netdev_info(netdev, "already using mac address %s\n",
+ format_mac_addr_sa);
return 0;
}
@@ -2273,8 +2294,10 @@ static int hns3_nic_net_set_mac_address(struct net_device *netdev, void *p)
*/
if (!hns3_is_phys_func(h->pdev) &&
!is_zero_ether_addr(netdev->perm_addr)) {
- netdev_err(netdev, "has permanent MAC %pM, user MAC %pM not allow\n",
- netdev->perm_addr, mac_addr->sa_data);
+ hnae3_format_mac_addr(format_mac_addr_perm, netdev->perm_addr);
+ hnae3_format_mac_addr(format_mac_addr_sa, mac_addr->sa_data);
+ netdev_err(netdev, "has permanent MAC %s, user MAC %s not allow\n",
+ format_mac_addr_perm, format_mac_addr_sa);
return -EPERM;
}
@@ -2382,90 +2405,89 @@ static netdev_features_t hns3_features_check(struct sk_buff *skb,
return features;
}
+static void hns3_fetch_stats(struct rtnl_link_stats64 *stats,
+ struct hns3_enet_ring *ring, bool is_tx)
+{
+ unsigned int start;
+
+ do {
+ start = u64_stats_fetch_begin_irq(&ring->syncp);
+ if (is_tx) {
+ stats->tx_bytes += ring->stats.tx_bytes;
+ stats->tx_packets += ring->stats.tx_pkts;
+ stats->tx_dropped += ring->stats.sw_err_cnt;
+ stats->tx_dropped += ring->stats.tx_vlan_err;
+ stats->tx_dropped += ring->stats.tx_l4_proto_err;
+ stats->tx_dropped += ring->stats.tx_l2l3l4_err;
+ stats->tx_dropped += ring->stats.tx_tso_err;
+ stats->tx_dropped += ring->stats.over_max_recursion;
+ stats->tx_dropped += ring->stats.hw_limitation;
+ stats->tx_dropped += ring->stats.copy_bits_err;
+ stats->tx_dropped += ring->stats.skb2sgl_err;
+ stats->tx_dropped += ring->stats.map_sg_err;
+ stats->tx_errors += ring->stats.sw_err_cnt;
+ stats->tx_errors += ring->stats.tx_vlan_err;
+ stats->tx_errors += ring->stats.tx_l4_proto_err;
+ stats->tx_errors += ring->stats.tx_l2l3l4_err;
+ stats->tx_errors += ring->stats.tx_tso_err;
+ stats->tx_errors += ring->stats.over_max_recursion;
+ stats->tx_errors += ring->stats.hw_limitation;
+ stats->tx_errors += ring->stats.copy_bits_err;
+ stats->tx_errors += ring->stats.skb2sgl_err;
+ stats->tx_errors += ring->stats.map_sg_err;
+ } else {
+ stats->rx_bytes += ring->stats.rx_bytes;
+ stats->rx_packets += ring->stats.rx_pkts;
+ stats->rx_dropped += ring->stats.l2_err;
+ stats->rx_errors += ring->stats.l2_err;
+ stats->rx_errors += ring->stats.l3l4_csum_err;
+ stats->rx_crc_errors += ring->stats.l2_err;
+ stats->multicast += ring->stats.rx_multicast;
+ stats->rx_length_errors += ring->stats.err_pkt_len;
+ }
+ } while (u64_stats_fetch_retry_irq(&ring->syncp, start));
+}
+
static void hns3_nic_get_stats64(struct net_device *netdev,
struct rtnl_link_stats64 *stats)
{
struct hns3_nic_priv *priv = netdev_priv(netdev);
int queue_num = priv->ae_handle->kinfo.num_tqps;
struct hnae3_handle *handle = priv->ae_handle;
+ struct rtnl_link_stats64 ring_total_stats;
struct hns3_enet_ring *ring;
- u64 rx_length_errors = 0;
- u64 rx_crc_errors = 0;
- u64 rx_multicast = 0;
- unsigned int start;
- u64 tx_errors = 0;
- u64 rx_errors = 0;
unsigned int idx;
- u64 tx_bytes = 0;
- u64 rx_bytes = 0;
- u64 tx_pkts = 0;
- u64 rx_pkts = 0;
- u64 tx_drop = 0;
- u64 rx_drop = 0;
if (test_bit(HNS3_NIC_STATE_DOWN, &priv->state))
return;
handle->ae_algo->ops->update_stats(handle, &netdev->stats);
+ memset(&ring_total_stats, 0, sizeof(ring_total_stats));
for (idx = 0; idx < queue_num; idx++) {
/* fetch the tx stats */
ring = &priv->ring[idx];
- do {
- start = u64_stats_fetch_begin_irq(&ring->syncp);
- tx_bytes += ring->stats.tx_bytes;
- tx_pkts += ring->stats.tx_pkts;
- tx_drop += ring->stats.sw_err_cnt;
- tx_drop += ring->stats.tx_vlan_err;
- tx_drop += ring->stats.tx_l4_proto_err;
- tx_drop += ring->stats.tx_l2l3l4_err;
- tx_drop += ring->stats.tx_tso_err;
- tx_drop += ring->stats.over_max_recursion;
- tx_drop += ring->stats.hw_limitation;
- tx_drop += ring->stats.copy_bits_err;
- tx_drop += ring->stats.skb2sgl_err;
- tx_drop += ring->stats.map_sg_err;
- tx_errors += ring->stats.sw_err_cnt;
- tx_errors += ring->stats.tx_vlan_err;
- tx_errors += ring->stats.tx_l4_proto_err;
- tx_errors += ring->stats.tx_l2l3l4_err;
- tx_errors += ring->stats.tx_tso_err;
- tx_errors += ring->stats.over_max_recursion;
- tx_errors += ring->stats.hw_limitation;
- tx_errors += ring->stats.copy_bits_err;
- tx_errors += ring->stats.skb2sgl_err;
- tx_errors += ring->stats.map_sg_err;
- } while (u64_stats_fetch_retry_irq(&ring->syncp, start));
+ hns3_fetch_stats(&ring_total_stats, ring, true);
/* fetch the rx stats */
ring = &priv->ring[idx + queue_num];
- do {
- start = u64_stats_fetch_begin_irq(&ring->syncp);
- rx_bytes += ring->stats.rx_bytes;
- rx_pkts += ring->stats.rx_pkts;
- rx_drop += ring->stats.l2_err;
- rx_errors += ring->stats.l2_err;
- rx_errors += ring->stats.l3l4_csum_err;
- rx_crc_errors += ring->stats.l2_err;
- rx_multicast += ring->stats.rx_multicast;
- rx_length_errors += ring->stats.err_pkt_len;
- } while (u64_stats_fetch_retry_irq(&ring->syncp, start));
+ hns3_fetch_stats(&ring_total_stats, ring, false);
}
- stats->tx_bytes = tx_bytes;
- stats->tx_packets = tx_pkts;
- stats->rx_bytes = rx_bytes;
- stats->rx_packets = rx_pkts;
+ stats->tx_bytes = ring_total_stats.tx_bytes;
+ stats->tx_packets = ring_total_stats.tx_packets;
+ stats->rx_bytes = ring_total_stats.rx_bytes;
+ stats->rx_packets = ring_total_stats.rx_packets;
- stats->rx_errors = rx_errors;
- stats->multicast = rx_multicast;
- stats->rx_length_errors = rx_length_errors;
- stats->rx_crc_errors = rx_crc_errors;
+ stats->rx_errors = ring_total_stats.rx_errors;
+ stats->multicast = ring_total_stats.multicast;
+ stats->rx_length_errors = ring_total_stats.rx_length_errors;
+ stats->rx_crc_errors = ring_total_stats.rx_crc_errors;
stats->rx_missed_errors = netdev->stats.rx_missed_errors;
- stats->tx_errors = tx_errors;
- stats->rx_dropped = rx_drop;
- stats->tx_dropped = tx_drop;
+ stats->tx_errors = ring_total_stats.tx_errors;
+ stats->rx_dropped = ring_total_stats.rx_dropped;
+ stats->tx_dropped = ring_total_stats.tx_dropped;
stats->collisions = netdev->stats.collisions;
stats->rx_over_errors = netdev->stats.rx_over_errors;
stats->rx_frame_errors = netdev->stats.rx_frame_errors;
@@ -2658,18 +2680,8 @@ static int hns3_nic_change_mtu(struct net_device *netdev, int new_mtu)
return ret;
}
-static bool hns3_get_tx_timeo_queue_info(struct net_device *ndev)
+static int hns3_get_timeout_queue(struct net_device *ndev)
{
- struct hns3_nic_priv *priv = netdev_priv(ndev);
- struct hnae3_handle *h = hns3_get_handle(ndev);
- struct hns3_enet_ring *tx_ring;
- struct napi_struct *napi;
- int timeout_queue = 0;
- int hw_head, hw_tail;
- int fbd_num, fbd_oft;
- int ebd_num, ebd_oft;
- int bd_num, bd_err;
- int ring_en, tc;
int i;
/* Find the stopped queue the same way the stack does */
@@ -2678,11 +2690,17 @@ static bool hns3_get_tx_timeo_queue_info(struct net_device *ndev)
unsigned long trans_start;
q = netdev_get_tx_queue(ndev, i);
- trans_start = q->trans_start;
+ trans_start = READ_ONCE(q->trans_start);
if (netif_xmit_stopped(q) &&
time_after(jiffies,
(trans_start + ndev->watchdog_timeo))) {
- timeout_queue = i;
+#ifdef CONFIG_BQL
+ struct dql *dql = &q->dql;
+
+ netdev_info(ndev, "DQL info last_cnt: %u, queued: %u, adj_limit: %u, completed: %u\n",
+ dql->last_obj_cnt, dql->num_queued,
+ dql->adj_limit, dql->num_completed);
+#endif
netdev_info(ndev, "queue state: 0x%lx, delta msecs: %u\n",
q->state,
jiffies_to_msecs(jiffies - trans_start));
@@ -2690,17 +2708,15 @@ static bool hns3_get_tx_timeo_queue_info(struct net_device *ndev)
}
}
- if (i == ndev->num_tx_queues) {
- netdev_info(ndev,
- "no netdev TX timeout queue found, timeout count: %llu\n",
- priv->tx_timeout_count);
- return false;
- }
+ return i;
+}
- priv->tx_timeout_count++;
-
- tx_ring = &priv->ring[timeout_queue];
- napi = &tx_ring->tqp_vector->napi;
+static void hns3_dump_queue_stats(struct net_device *ndev,
+ struct hns3_enet_ring *tx_ring,
+ int timeout_queue)
+{
+ struct napi_struct *napi = &tx_ring->tqp_vector->napi;
+ struct hns3_nic_priv *priv = netdev_priv(ndev);
netdev_info(ndev,
"tx_timeout count: %llu, queue id: %d, SW_NTU: 0x%x, SW_NTC: 0x%x, napi state: %lu\n",
@@ -2716,6 +2732,48 @@ static bool hns3_get_tx_timeo_queue_info(struct net_device *ndev)
"seg_pkt_cnt: %llu, tx_more: %llu, restart_queue: %llu, tx_busy: %llu\n",
tx_ring->stats.seg_pkt_cnt, tx_ring->stats.tx_more,
tx_ring->stats.restart_queue, tx_ring->stats.tx_busy);
+}
+
+static void hns3_dump_queue_reg(struct net_device *ndev,
+ struct hns3_enet_ring *tx_ring)
+{
+ netdev_info(ndev,
+ "BD_NUM: 0x%x HW_HEAD: 0x%x, HW_TAIL: 0x%x, BD_ERR: 0x%x, INT: 0x%x\n",
+ hns3_tqp_read_reg(tx_ring, HNS3_RING_TX_RING_BD_NUM_REG),
+ hns3_tqp_read_reg(tx_ring, HNS3_RING_TX_RING_HEAD_REG),
+ hns3_tqp_read_reg(tx_ring, HNS3_RING_TX_RING_TAIL_REG),
+ hns3_tqp_read_reg(tx_ring, HNS3_RING_TX_RING_BD_ERR_REG),
+ readl(tx_ring->tqp_vector->mask_addr));
+ netdev_info(ndev,
+ "RING_EN: 0x%x, TC: 0x%x, FBD_NUM: 0x%x FBD_OFT: 0x%x, EBD_NUM: 0x%x, EBD_OFT: 0x%x\n",
+ hns3_tqp_read_reg(tx_ring, HNS3_RING_EN_REG),
+ hns3_tqp_read_reg(tx_ring, HNS3_RING_TX_RING_TC_REG),
+ hns3_tqp_read_reg(tx_ring, HNS3_RING_TX_RING_FBDNUM_REG),
+ hns3_tqp_read_reg(tx_ring, HNS3_RING_TX_RING_OFFSET_REG),
+ hns3_tqp_read_reg(tx_ring, HNS3_RING_TX_RING_EBDNUM_REG),
+ hns3_tqp_read_reg(tx_ring,
+ HNS3_RING_TX_RING_EBD_OFFSET_REG));
+}
+
+static bool hns3_get_tx_timeo_queue_info(struct net_device *ndev)
+{
+ struct hns3_nic_priv *priv = netdev_priv(ndev);
+ struct hnae3_handle *h = hns3_get_handle(ndev);
+ struct hns3_enet_ring *tx_ring;
+ int timeout_queue;
+
+ timeout_queue = hns3_get_timeout_queue(ndev);
+ if (timeout_queue >= ndev->num_tx_queues) {
+ netdev_info(ndev,
+ "no netdev TX timeout queue found, timeout count: %llu\n",
+ priv->tx_timeout_count);
+ return false;
+ }
+
+ priv->tx_timeout_count++;
+
+ tx_ring = &priv->ring[timeout_queue];
+ hns3_dump_queue_stats(ndev, tx_ring, timeout_queue);
/* When mac received many pause frames continuous, it's unable to send
* packets, which may cause tx timeout
@@ -2728,32 +2786,7 @@ static bool hns3_get_tx_timeo_queue_info(struct net_device *ndev)
mac_stats.tx_pause_cnt, mac_stats.rx_pause_cnt);
}
- hw_head = readl_relaxed(tx_ring->tqp->io_base +
- HNS3_RING_TX_RING_HEAD_REG);
- hw_tail = readl_relaxed(tx_ring->tqp->io_base +
- HNS3_RING_TX_RING_TAIL_REG);
- fbd_num = readl_relaxed(tx_ring->tqp->io_base +
- HNS3_RING_TX_RING_FBDNUM_REG);
- fbd_oft = readl_relaxed(tx_ring->tqp->io_base +
- HNS3_RING_TX_RING_OFFSET_REG);
- ebd_num = readl_relaxed(tx_ring->tqp->io_base +
- HNS3_RING_TX_RING_EBDNUM_REG);
- ebd_oft = readl_relaxed(tx_ring->tqp->io_base +
- HNS3_RING_TX_RING_EBD_OFFSET_REG);
- bd_num = readl_relaxed(tx_ring->tqp->io_base +
- HNS3_RING_TX_RING_BD_NUM_REG);
- bd_err = readl_relaxed(tx_ring->tqp->io_base +
- HNS3_RING_TX_RING_BD_ERR_REG);
- ring_en = readl_relaxed(tx_ring->tqp->io_base + HNS3_RING_EN_REG);
- tc = readl_relaxed(tx_ring->tqp->io_base + HNS3_RING_TX_RING_TC_REG);
-
- netdev_info(ndev,
- "BD_NUM: 0x%x HW_HEAD: 0x%x, HW_TAIL: 0x%x, BD_ERR: 0x%x, INT: 0x%x\n",
- bd_num, hw_head, hw_tail, bd_err,
- readl(tx_ring->tqp_vector->mask_addr));
- netdev_info(ndev,
- "RING_EN: 0x%x, TC: 0x%x, FBD_NUM: 0x%x FBD_OFT: 0x%x, EBD_NUM: 0x%x, EBD_OFT: 0x%x\n",
- ring_en, tc, fbd_num, fbd_oft, ebd_num, ebd_oft);
+ hns3_dump_queue_reg(ndev, tx_ring);
return true;
}
@@ -2836,14 +2869,16 @@ static int hns3_nic_set_vf_rate(struct net_device *ndev, int vf,
static int hns3_nic_set_vf_mac(struct net_device *netdev, int vf_id, u8 *mac)
{
struct hnae3_handle *h = hns3_get_handle(netdev);
+ char format_mac_addr[HNAE3_FORMAT_MAC_ADDR_LEN];
if (!h->ae_algo->ops->set_vf_mac)
return -EOPNOTSUPP;
if (is_multicast_ether_addr(mac)) {
+ hnae3_format_mac_addr(format_mac_addr, mac);
netdev_err(netdev,
- "Invalid MAC:%pM specified. Could not set MAC\n",
- mac);
+ "Invalid MAC:%s specified. Could not set MAC\n",
+ format_mac_addr);
return -EINVAL;
}
@@ -3497,17 +3532,13 @@ static bool hns3_nic_alloc_rx_buffers(struct hns3_enet_ring *ring,
for (i = 0; i < cleand_count; i++) {
desc_cb = &ring->desc_cb[ring->next_to_use];
if (desc_cb->reuse_flag) {
- u64_stats_update_begin(&ring->syncp);
- ring->stats.reuse_pg_cnt++;
- u64_stats_update_end(&ring->syncp);
+ hns3_ring_stats_update(ring, reuse_pg_cnt);
hns3_reuse_buffer(ring, ring->next_to_use);
} else {
ret = hns3_alloc_and_map_buffer(ring, &res_cbs);
if (ret) {
- u64_stats_update_begin(&ring->syncp);
- ring->stats.sw_err_cnt++;
- u64_stats_update_end(&ring->syncp);
+ hns3_ring_stats_update(ring, sw_err_cnt);
hns3_rl_err(ring_to_netdev(ring),
"alloc rx buffer failed: %d\n",
@@ -3519,9 +3550,7 @@ static bool hns3_nic_alloc_rx_buffers(struct hns3_enet_ring *ring,
}
hns3_replace_buffer(ring, ring->next_to_use, &res_cbs);
- u64_stats_update_begin(&ring->syncp);
- ring->stats.non_reuse_pg++;
- u64_stats_update_end(&ring->syncp);
+ hns3_ring_stats_update(ring, non_reuse_pg);
}
ring_ptr_move_fw(ring, next_to_use);
@@ -3536,6 +3565,34 @@ static bool hns3_can_reuse_page(struct hns3_desc_cb *cb)
return page_count(cb->priv) == cb->pagecnt_bias;
}
+static int hns3_handle_rx_copybreak(struct sk_buff *skb, int i,
+ struct hns3_enet_ring *ring,
+ int pull_len,
+ struct hns3_desc_cb *desc_cb)
+{
+ struct hns3_desc *desc = &ring->desc[ring->next_to_clean];
+ u32 frag_offset = desc_cb->page_offset + pull_len;
+ int size = le16_to_cpu(desc->rx.size);
+ u32 frag_size = size - pull_len;
+ void *frag = napi_alloc_frag(frag_size);
+
+ if (unlikely(!frag)) {
+ hns3_ring_stats_update(ring, frag_alloc_err);
+
+ hns3_rl_err(ring_to_netdev(ring),
+ "failed to allocate rx frag\n");
+ return -ENOMEM;
+ }
+
+ desc_cb->reuse_flag = 1;
+ memcpy(frag, desc_cb->buf + frag_offset, frag_size);
+ skb_add_rx_frag(skb, i, virt_to_page(frag),
+ offset_in_page(frag), frag_size, frag_size);
+
+ hns3_ring_stats_update(ring, frag_alloc);
+ return 0;
+}
+
static void hns3_nic_reuse_page(struct sk_buff *skb, int i,
struct hns3_enet_ring *ring, int pull_len,
struct hns3_desc_cb *desc_cb)
@@ -3545,6 +3602,7 @@ static void hns3_nic_reuse_page(struct sk_buff *skb, int i,
int size = le16_to_cpu(desc->rx.size);
u32 truesize = hns3_buf_size(ring);
u32 frag_size = size - pull_len;
+ int ret = 0;
bool reused;
if (ring->page_pool) {
@@ -3579,27 +3637,9 @@ static void hns3_nic_reuse_page(struct sk_buff *skb, int i,
desc_cb->page_offset = 0;
desc_cb->reuse_flag = 1;
} else if (frag_size <= ring->rx_copybreak) {
- void *frag = napi_alloc_frag(frag_size);
-
- if (unlikely(!frag)) {
- u64_stats_update_begin(&ring->syncp);
- ring->stats.frag_alloc_err++;
- u64_stats_update_end(&ring->syncp);
-
- hns3_rl_err(ring_to_netdev(ring),
- "failed to allocate rx frag\n");
+ ret = hns3_handle_rx_copybreak(skb, i, ring, pull_len, desc_cb);
+ if (ret)
goto out;
- }
-
- desc_cb->reuse_flag = 1;
- memcpy(frag, desc_cb->buf + frag_offset, frag_size);
- skb_add_rx_frag(skb, i, virt_to_page(frag),
- offset_in_page(frag), frag_size, frag_size);
-
- u64_stats_update_begin(&ring->syncp);
- ring->stats.frag_alloc++;
- u64_stats_update_end(&ring->syncp);
- return;
}
out:
@@ -3682,9 +3722,7 @@ static bool hns3_checksum_complete(struct hns3_enet_ring *ring,
hns3_rx_ptype_tbl[ptype].ip_summed != CHECKSUM_COMPLETE)
return false;
- u64_stats_update_begin(&ring->syncp);
- ring->stats.csum_complete++;
- u64_stats_update_end(&ring->syncp);
+ hns3_ring_stats_update(ring, csum_complete);
skb->ip_summed = CHECKSUM_COMPLETE;
skb->csum = csum_unfold((__force __sum16)csum);
@@ -3758,9 +3796,7 @@ static void hns3_rx_checksum(struct hns3_enet_ring *ring, struct sk_buff *skb,
if (unlikely(l234info & (BIT(HNS3_RXD_L3E_B) | BIT(HNS3_RXD_L4E_B) |
BIT(HNS3_RXD_OL3E_B) |
BIT(HNS3_RXD_OL4E_B)))) {
- u64_stats_update_begin(&ring->syncp);
- ring->stats.l3l4_csum_err++;
- u64_stats_update_end(&ring->syncp);
+ hns3_ring_stats_update(ring, l3l4_csum_err);
return;
}
@@ -3851,10 +3887,7 @@ static int hns3_alloc_skb(struct hns3_enet_ring *ring, unsigned int length,
skb = ring->skb;
if (unlikely(!skb)) {
hns3_rl_err(netdev, "alloc rx skb fail\n");
-
- u64_stats_update_begin(&ring->syncp);
- ring->stats.sw_err_cnt++;
- u64_stats_update_end(&ring->syncp);
+ hns3_ring_stats_update(ring, sw_err_cnt);
return -ENOMEM;
}
@@ -3885,9 +3918,7 @@ static int hns3_alloc_skb(struct hns3_enet_ring *ring, unsigned int length,
if (ring->page_pool)
skb_mark_for_recycle(skb);
- u64_stats_update_begin(&ring->syncp);
- ring->stats.seg_pkt_cnt++;
- u64_stats_update_end(&ring->syncp);
+ hns3_ring_stats_update(ring, seg_pkt_cnt);
ring->pull_len = eth_get_headlen(netdev, va, HNS3_RX_HEAD_SIZE);
__skb_put(skb, ring->pull_len);
@@ -4015,6 +4046,39 @@ static void hns3_set_rx_skb_rss_type(struct hns3_enet_ring *ring,
skb_set_hash(skb, rss_hash, rss_type);
}
+static void hns3_handle_rx_ts_info(struct net_device *netdev,
+ struct hns3_desc *desc, struct sk_buff *skb,
+ u32 bd_base_info)
+{
+ if (unlikely(bd_base_info & BIT(HNS3_RXD_TS_VLD_B))) {
+ struct hnae3_handle *h = hns3_get_handle(netdev);
+ u32 nsec = le32_to_cpu(desc->ts_nsec);
+ u32 sec = le32_to_cpu(desc->ts_sec);
+
+ if (h->ae_algo->ops->get_rx_hwts)
+ h->ae_algo->ops->get_rx_hwts(h, skb, nsec, sec);
+ }
+}
+
+static void hns3_handle_rx_vlan_tag(struct hns3_enet_ring *ring,
+ struct hns3_desc *desc, struct sk_buff *skb,
+ u32 l234info)
+{
+ struct net_device *netdev = ring_to_netdev(ring);
+
+ /* Based on hw strategy, the tag offloaded will be stored at
+ * ot_vlan_tag in two layer tag case, and stored at vlan_tag
+ * in one layer tag case.
+ */
+ if (netdev->features & NETIF_F_HW_VLAN_CTAG_RX) {
+ u16 vlan_tag;
+
+ if (hns3_parse_vlan_tag(ring, desc, l234info, &vlan_tag))
+ __vlan_hwaccel_put_tag(skb, htons(ETH_P_8021Q),
+ vlan_tag);
+ }
+}
+
static int hns3_handle_bdinfo(struct hns3_enet_ring *ring, struct sk_buff *skb)
{
struct net_device *netdev = ring_to_netdev(ring);
@@ -4037,26 +4101,9 @@ static int hns3_handle_bdinfo(struct hns3_enet_ring *ring, struct sk_buff *skb)
ol_info = le32_to_cpu(desc->rx.ol_info);
csum = le16_to_cpu(desc->csum);
- if (unlikely(bd_base_info & BIT(HNS3_RXD_TS_VLD_B))) {
- struct hnae3_handle *h = hns3_get_handle(netdev);
- u32 nsec = le32_to_cpu(desc->ts_nsec);
- u32 sec = le32_to_cpu(desc->ts_sec);
+ hns3_handle_rx_ts_info(netdev, desc, skb, bd_base_info);
- if (h->ae_algo->ops->get_rx_hwts)
- h->ae_algo->ops->get_rx_hwts(h, skb, nsec, sec);
- }
-
- /* Based on hw strategy, the tag offloaded will be stored at
- * ot_vlan_tag in two layer tag case, and stored at vlan_tag
- * in one layer tag case.
- */
- if (netdev->features & NETIF_F_HW_VLAN_CTAG_RX) {
- u16 vlan_tag;
-
- if (hns3_parse_vlan_tag(ring, desc, l234info, &vlan_tag))
- __vlan_hwaccel_put_tag(skb, htons(ETH_P_8021Q),
- vlan_tag);
- }
+ hns3_handle_rx_vlan_tag(ring, desc, skb, l234info);
if (unlikely(!desc->rx.pkt_len || (l234info & (BIT(HNS3_RXD_TRUNCAT_B) |
BIT(HNS3_RXD_L2E_B))))) {
@@ -4079,9 +4126,7 @@ static int hns3_handle_bdinfo(struct hns3_enet_ring *ring, struct sk_buff *skb)
ret = hns3_set_gro_and_checksum(ring, skb, l234info,
bd_base_info, ol_info, csum);
if (unlikely(ret)) {
- u64_stats_update_begin(&ring->syncp);
- ring->stats.rx_err_cnt++;
- u64_stats_update_end(&ring->syncp);
+ hns3_ring_stats_update(ring, rx_err_cnt);
return ret;
}
@@ -4297,87 +4342,70 @@ static int hns3_nic_common_poll(struct napi_struct *napi, int budget)
return rx_pkt_total;
}
-static int hns3_get_vector_ring_chain(struct hns3_enet_tqp_vector *tqp_vector,
- struct hnae3_ring_chain_node *head)
+static int hns3_create_ring_chain(struct hns3_enet_tqp_vector *tqp_vector,
+ struct hnae3_ring_chain_node **head,
+ bool is_tx)
{
+ u32 bit_value = is_tx ? HNAE3_RING_TYPE_TX : HNAE3_RING_TYPE_RX;
+ u32 field_value = is_tx ? HNAE3_RING_GL_TX : HNAE3_RING_GL_RX;
+ struct hnae3_ring_chain_node *cur_chain = *head;
struct pci_dev *pdev = tqp_vector->handle->pdev;
- struct hnae3_ring_chain_node *cur_chain = head;
struct hnae3_ring_chain_node *chain;
- struct hns3_enet_ring *tx_ring;
- struct hns3_enet_ring *rx_ring;
+ struct hns3_enet_ring *ring;
- tx_ring = tqp_vector->tx_group.ring;
- if (tx_ring) {
- cur_chain->tqp_index = tx_ring->tqp->tqp_index;
- hnae3_set_bit(cur_chain->flag, HNAE3_RING_TYPE_B,
- HNAE3_RING_TYPE_TX);
- hnae3_set_field(cur_chain->int_gl_idx, HNAE3_RING_GL_IDX_M,
- HNAE3_RING_GL_IDX_S, HNAE3_RING_GL_TX);
+ ring = is_tx ? tqp_vector->tx_group.ring : tqp_vector->rx_group.ring;
- cur_chain->next = NULL;
-
- while (tx_ring->next) {
- tx_ring = tx_ring->next;
-
- chain = devm_kzalloc(&pdev->dev, sizeof(*chain),
- GFP_KERNEL);
- if (!chain)
- goto err_free_chain;
-
- cur_chain->next = chain;
- chain->tqp_index = tx_ring->tqp->tqp_index;
- hnae3_set_bit(chain->flag, HNAE3_RING_TYPE_B,
- HNAE3_RING_TYPE_TX);
- hnae3_set_field(chain->int_gl_idx,
- HNAE3_RING_GL_IDX_M,
- HNAE3_RING_GL_IDX_S,
- HNAE3_RING_GL_TX);
-
- cur_chain = chain;
- }
+ if (cur_chain) {
+ while (cur_chain->next)
+ cur_chain = cur_chain->next;
}
- rx_ring = tqp_vector->rx_group.ring;
- if (!tx_ring && rx_ring) {
- cur_chain->next = NULL;
- cur_chain->tqp_index = rx_ring->tqp->tqp_index;
- hnae3_set_bit(cur_chain->flag, HNAE3_RING_TYPE_B,
- HNAE3_RING_TYPE_RX);
- hnae3_set_field(cur_chain->int_gl_idx, HNAE3_RING_GL_IDX_M,
- HNAE3_RING_GL_IDX_S, HNAE3_RING_GL_RX);
-
- rx_ring = rx_ring->next;
- }
-
- while (rx_ring) {
+ while (ring) {
chain = devm_kzalloc(&pdev->dev, sizeof(*chain), GFP_KERNEL);
if (!chain)
- goto err_free_chain;
-
- cur_chain->next = chain;
- chain->tqp_index = rx_ring->tqp->tqp_index;
+ return -ENOMEM;
+ if (cur_chain)
+ cur_chain->next = chain;
+ else
+ *head = chain;
+ chain->tqp_index = ring->tqp->tqp_index;
hnae3_set_bit(chain->flag, HNAE3_RING_TYPE_B,
- HNAE3_RING_TYPE_RX);
- hnae3_set_field(chain->int_gl_idx, HNAE3_RING_GL_IDX_M,
- HNAE3_RING_GL_IDX_S, HNAE3_RING_GL_RX);
+ bit_value);
+ hnae3_set_field(chain->int_gl_idx,
+ HNAE3_RING_GL_IDX_M,
+ HNAE3_RING_GL_IDX_S, field_value);
cur_chain = chain;
- rx_ring = rx_ring->next;
+ ring = ring->next;
}
return 0;
+}
+
+static struct hnae3_ring_chain_node *
+hns3_get_vector_ring_chain(struct hns3_enet_tqp_vector *tqp_vector)
+{
+ struct pci_dev *pdev = tqp_vector->handle->pdev;
+ struct hnae3_ring_chain_node *cur_chain = NULL;
+ struct hnae3_ring_chain_node *chain;
+
+ if (hns3_create_ring_chain(tqp_vector, &cur_chain, true))
+ goto err_free_chain;
+
+ if (hns3_create_ring_chain(tqp_vector, &cur_chain, false))
+ goto err_free_chain;
+
+ return cur_chain;
err_free_chain:
- cur_chain = head->next;
while (cur_chain) {
chain = cur_chain->next;
devm_kfree(&pdev->dev, cur_chain);
cur_chain = chain;
}
- head->next = NULL;
- return -ENOMEM;
+ return NULL;
}
static void hns3_free_vector_ring_chain(struct hns3_enet_tqp_vector *tqp_vector,
@@ -4386,7 +4414,7 @@ static void hns3_free_vector_ring_chain(struct hns3_enet_tqp_vector *tqp_vector,
struct pci_dev *pdev = tqp_vector->handle->pdev;
struct hnae3_ring_chain_node *chain_tmp, *chain;
- chain = head->next;
+ chain = head;
while (chain) {
chain_tmp = chain->next;
@@ -4501,7 +4529,7 @@ static int hns3_nic_init_vector_data(struct hns3_nic_priv *priv)
}
for (i = 0; i < priv->vector_num; i++) {
- struct hnae3_ring_chain_node vector_ring_chain;
+ struct hnae3_ring_chain_node *vector_ring_chain;
tqp_vector = &priv->tqp_vector[i];
@@ -4511,15 +4539,16 @@ static int hns3_nic_init_vector_data(struct hns3_nic_priv *priv)
tqp_vector->tx_group.total_packets = 0;
tqp_vector->handle = h;
- ret = hns3_get_vector_ring_chain(tqp_vector,
- &vector_ring_chain);
- if (ret)
+ vector_ring_chain = hns3_get_vector_ring_chain(tqp_vector);
+ if (!vector_ring_chain) {
+ ret = -ENOMEM;
goto map_ring_fail;
+ }
ret = h->ae_algo->ops->map_ring_to_vector(h,
- tqp_vector->vector_irq, &vector_ring_chain);
+ tqp_vector->vector_irq, vector_ring_chain);
- hns3_free_vector_ring_chain(tqp_vector, &vector_ring_chain);
+ hns3_free_vector_ring_chain(tqp_vector, vector_ring_chain);
if (ret)
goto map_ring_fail;
@@ -4618,7 +4647,7 @@ static void hns3_clear_ring_group(struct hns3_enet_ring_group *group)
static void hns3_nic_uninit_vector_data(struct hns3_nic_priv *priv)
{
- struct hnae3_ring_chain_node vector_ring_chain;
+ struct hnae3_ring_chain_node *vector_ring_chain;
struct hnae3_handle *h = priv->ae_handle;
struct hns3_enet_tqp_vector *tqp_vector;
int i;
@@ -4633,13 +4662,14 @@ static void hns3_nic_uninit_vector_data(struct hns3_nic_priv *priv)
* chain between vector and ring, we should go on to deal with
* the remaining options.
*/
- if (hns3_get_vector_ring_chain(tqp_vector, &vector_ring_chain))
+ vector_ring_chain = hns3_get_vector_ring_chain(tqp_vector);
+ if (!vector_ring_chain)
dev_warn(priv->dev, "failed to get ring chain\n");
h->ae_algo->ops->unmap_ring_from_vector(h,
- tqp_vector->vector_irq, &vector_ring_chain);
+ tqp_vector->vector_irq, vector_ring_chain);
- hns3_free_vector_ring_chain(tqp_vector, &vector_ring_chain);
+ hns3_free_vector_ring_chain(tqp_vector, vector_ring_chain);
hns3_clear_ring_group(&tqp_vector->rx_group);
hns3_clear_ring_group(&tqp_vector->tx_group);
@@ -4934,6 +4964,7 @@ static void hns3_uninit_all_ring(struct hns3_nic_priv *priv)
static int hns3_init_mac_addr(struct net_device *netdev)
{
struct hns3_nic_priv *priv = netdev_priv(netdev);
+ char format_mac_addr[HNAE3_FORMAT_MAC_ADDR_LEN];
struct hnae3_handle *h = priv->ae_handle;
u8 mac_addr_temp[ETH_ALEN];
int ret = 0;
@@ -4944,8 +4975,9 @@ static int hns3_init_mac_addr(struct net_device *netdev)
/* Check if the MAC address is valid, if not get a random one */
if (!is_valid_ether_addr(mac_addr_temp)) {
eth_hw_addr_random(netdev);
- dev_warn(priv->dev, "using random MAC address %pM\n",
- netdev->dev_addr);
+ hnae3_format_mac_addr(format_mac_addr, netdev->dev_addr);
+ dev_warn(priv->dev, "using random MAC address %s\n",
+ format_mac_addr);
} else if (!ether_addr_equal(netdev->dev_addr, mac_addr_temp)) {
eth_hw_addr_set(netdev, mac_addr_temp);
ether_addr_copy(netdev->perm_addr, mac_addr_temp);
@@ -4997,8 +5029,10 @@ static void hns3_client_stop(struct hnae3_handle *handle)
static void hns3_info_show(struct hns3_nic_priv *priv)
{
struct hnae3_knic_private_info *kinfo = &priv->ae_handle->kinfo;
+ char format_mac_addr[HNAE3_FORMAT_MAC_ADDR_LEN];
- dev_info(priv->dev, "MAC address: %pM\n", priv->netdev->dev_addr);
+ hnae3_format_mac_addr(format_mac_addr, priv->netdev->dev_addr);
+ dev_info(priv->dev, "MAC address: %s\n", format_mac_addr);
dev_info(priv->dev, "Task queue pairs numbers: %u\n", kinfo->num_tqps);
dev_info(priv->dev, "RSS size: %u\n", kinfo->rss_size);
dev_info(priv->dev, "Allocated RSS size: %u\n", kinfo->req_rss_size);
@@ -5287,9 +5321,7 @@ static int hns3_clear_rx_ring(struct hns3_enet_ring *ring)
if (!ring->desc_cb[ring->next_to_use].reuse_flag) {
ret = hns3_alloc_and_map_buffer(ring, &res_cbs);
if (ret) {
- u64_stats_update_begin(&ring->syncp);
- ring->stats.sw_err_cnt++;
- u64_stats_update_end(&ring->syncp);
+ hns3_ring_stats_update(ring, sw_err_cnt);
/* if alloc new buffer fail, exit directly
* and reclear in up flow.
*/
@@ -5531,8 +5563,8 @@ static int hns3_reset_notify_uninit_enet(struct hnae3_handle *handle)
return 0;
}
-static int hns3_reset_notify(struct hnae3_handle *handle,
- enum hnae3_reset_notify_type type)
+int hns3_reset_notify(struct hnae3_handle *handle,
+ enum hnae3_reset_notify_type type)
{
int ret = 0;
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3_enet.h b/drivers/net/ethernet/hisilicon/hns3/hns3_enet.h
index 1715c98..a05a0c7 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3_enet.h
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3_enet.h
@@ -10,6 +10,9 @@
#include "hnae3.h"
+struct iphdr;
+struct ipv6hdr;
+
enum hns3_nic_state {
HNS3_NIC_STATE_TESTING,
HNS3_NIC_STATE_RESETTING,
@@ -621,6 +624,11 @@ static inline int ring_space(struct hns3_enet_ring *ring)
(begin - end)) - 1;
}
+static inline u32 hns3_tqp_read_reg(struct hns3_enet_ring *ring, u32 reg)
+{
+ return readl_relaxed(ring->tqp->io_base + reg);
+}
+
static inline u32 hns3_read_reg(void __iomem *base, u32 reg)
{
return readl(base + reg);
@@ -655,6 +663,13 @@ static inline bool hns3_nic_resetting(struct net_device *netdev)
#define hns3_buf_size(_ring) ((_ring)->buf_size)
+#define hns3_ring_stats_update(ring, cnt) do { \
+ typeof(ring) (tmp) = (ring); \
+ u64_stats_update_begin(&(tmp)->syncp); \
+ ((tmp)->stats.cnt)++; \
+ u64_stats_update_end(&(tmp)->syncp); \
+} while (0) \
+
static inline unsigned int hns3_page_order(struct hns3_enet_ring *ring)
{
#if (PAGE_SIZE < 8192)
@@ -705,6 +720,8 @@ void hns3_set_vector_coalesce_tx_ql(struct hns3_enet_tqp_vector *tqp_vector,
u32 ql_value);
void hns3_request_update_promisc_mode(struct hnae3_handle *handle);
+int hns3_reset_notify(struct hnae3_handle *handle,
+ enum hnae3_reset_notify_type type);
#ifdef CONFIG_HNS3_DCB
void hns3_dcbnl_setup(struct hnae3_handle *handle);
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3_ethtool.c b/drivers/net/ethernet/hisilicon/hns3/hns3_ethtool.c
index c9b4568..c06c39e 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3_ethtool.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3_ethtool.c
@@ -643,11 +643,13 @@ static u32 hns3_get_link(struct net_device *netdev)
}
static void hns3_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *param)
+ struct ethtool_ringparam *param,
+ struct kernel_ethtool_ringparam *kernel_param,
+ struct netlink_ext_ack *extack)
{
struct hns3_nic_priv *priv = netdev_priv(netdev);
struct hnae3_handle *h = priv->ae_handle;
- int queue_num = h->kinfo.num_tqps;
+ int rx_queue_index = h->kinfo.num_tqps;
if (hns3_nic_resetting(netdev)) {
netdev_err(netdev, "dev resetting!");
@@ -658,7 +660,8 @@ static void hns3_get_ringparam(struct net_device *netdev,
param->rx_max_pending = HNS3_RING_MAX_PENDING;
param->tx_pending = priv->ring[0].desc_num;
- param->rx_pending = priv->ring[queue_num].desc_num;
+ param->rx_pending = priv->ring[rx_queue_index].desc_num;
+ kernel_param->rx_buf_len = priv->ring[rx_queue_index].buf_size;
}
static void hns3_get_pauseparam(struct net_device *netdev,
@@ -1064,14 +1067,23 @@ static struct hns3_enet_ring *hns3_backup_ringparam(struct hns3_nic_priv *priv)
}
static int hns3_check_ringparam(struct net_device *ndev,
- struct ethtool_ringparam *param)
+ struct ethtool_ringparam *param,
+ struct kernel_ethtool_ringparam *kernel_param)
{
+#define RX_BUF_LEN_2K 2048
+#define RX_BUF_LEN_4K 4096
if (hns3_nic_resetting(ndev))
return -EBUSY;
if (param->rx_mini_pending || param->rx_jumbo_pending)
return -EINVAL;
+ if (kernel_param->rx_buf_len != RX_BUF_LEN_2K &&
+ kernel_param->rx_buf_len != RX_BUF_LEN_4K) {
+ netdev_err(ndev, "Rx buf len only support 2048 and 4096\n");
+ return -EINVAL;
+ }
+
if (param->tx_pending > HNS3_RING_MAX_PENDING ||
param->tx_pending < HNS3_RING_MIN_PENDING ||
param->rx_pending > HNS3_RING_MAX_PENDING ||
@@ -1084,8 +1096,26 @@ static int hns3_check_ringparam(struct net_device *ndev,
return 0;
}
+static int hns3_change_rx_buf_len(struct net_device *ndev, u32 rx_buf_len)
+{
+ struct hns3_nic_priv *priv = netdev_priv(ndev);
+ struct hnae3_handle *h = priv->ae_handle;
+ int i;
+
+ h->kinfo.rx_buf_len = rx_buf_len;
+
+ for (i = 0; i < h->kinfo.num_tqps; i++) {
+ h->kinfo.tqp[i]->buf_size = rx_buf_len;
+ priv->ring[i + h->kinfo.num_tqps].buf_size = rx_buf_len;
+ }
+
+ return 0;
+}
+
static int hns3_set_ringparam(struct net_device *ndev,
- struct ethtool_ringparam *param)
+ struct ethtool_ringparam *param,
+ struct kernel_ethtool_ringparam *kernel_param,
+ struct netlink_ext_ack *extack)
{
struct hns3_nic_priv *priv = netdev_priv(ndev);
struct hnae3_handle *h = priv->ae_handle;
@@ -1094,9 +1124,10 @@ static int hns3_set_ringparam(struct net_device *ndev,
u32 old_tx_desc_num, new_tx_desc_num;
u32 old_rx_desc_num, new_rx_desc_num;
u16 queue_num = h->kinfo.num_tqps;
+ u32 old_rx_buf_len;
int ret, i;
- ret = hns3_check_ringparam(ndev, param);
+ ret = hns3_check_ringparam(ndev, param, kernel_param);
if (ret)
return ret;
@@ -1105,8 +1136,10 @@ static int hns3_set_ringparam(struct net_device *ndev,
new_rx_desc_num = ALIGN(param->rx_pending, HNS3_RING_BD_MULTIPLE);
old_tx_desc_num = priv->ring[0].desc_num;
old_rx_desc_num = priv->ring[queue_num].desc_num;
+ old_rx_buf_len = priv->ring[queue_num].buf_size;
if (old_tx_desc_num == new_tx_desc_num &&
- old_rx_desc_num == new_rx_desc_num)
+ old_rx_desc_num == new_rx_desc_num &&
+ kernel_param->rx_buf_len == old_rx_buf_len)
return 0;
tmp_rings = hns3_backup_ringparam(priv);
@@ -1117,19 +1150,22 @@ static int hns3_set_ringparam(struct net_device *ndev,
}
netdev_info(ndev,
- "Changing Tx/Rx ring depth from %u/%u to %u/%u\n",
+ "Changing Tx/Rx ring depth from %u/%u to %u/%u, Changing rx buffer len from %d to %d\n",
old_tx_desc_num, old_rx_desc_num,
- new_tx_desc_num, new_rx_desc_num);
+ new_tx_desc_num, new_rx_desc_num,
+ old_rx_buf_len, kernel_param->rx_buf_len);
if (if_running)
ndev->netdev_ops->ndo_stop(ndev);
hns3_change_all_ring_bd_num(priv, new_tx_desc_num, new_rx_desc_num);
+ hns3_change_rx_buf_len(ndev, kernel_param->rx_buf_len);
ret = hns3_init_all_ring(priv);
if (ret) {
- netdev_err(ndev, "Change bd num fail, revert to old value(%d)\n",
+ netdev_err(ndev, "set ringparam fail, revert to old value(%d)\n",
ret);
+ hns3_change_rx_buf_len(ndev, old_rx_buf_len);
hns3_change_all_ring_bd_num(priv, old_tx_desc_num,
old_rx_desc_num);
for (i = 0; i < h->kinfo.num_tqps * 2; i++)
@@ -1699,6 +1735,7 @@ static int hns3_get_tunable(struct net_device *netdev,
void *data)
{
struct hns3_nic_priv *priv = netdev_priv(netdev);
+ struct hnae3_handle *h = priv->ae_handle;
int ret = 0;
switch (tuna->id) {
@@ -1709,6 +1746,9 @@ static int hns3_get_tunable(struct net_device *netdev,
case ETHTOOL_RX_COPYBREAK:
*(u32 *)data = priv->rx_copybreak;
break;
+ case ETHTOOL_TX_COPYBREAK_BUF_SIZE:
+ *(u32 *)data = h->kinfo.tx_spare_buf_size;
+ break;
default:
ret = -EOPNOTSUPP;
break;
@@ -1717,11 +1757,43 @@ static int hns3_get_tunable(struct net_device *netdev,
return ret;
}
+static int hns3_set_tx_spare_buf_size(struct net_device *netdev,
+ u32 data)
+{
+ struct hns3_nic_priv *priv = netdev_priv(netdev);
+ struct hnae3_handle *h = priv->ae_handle;
+ int ret;
+
+ if (hns3_nic_resetting(netdev))
+ return -EBUSY;
+
+ h->kinfo.tx_spare_buf_size = data;
+
+ ret = hns3_reset_notify(h, HNAE3_DOWN_CLIENT);
+ if (ret)
+ return ret;
+
+ ret = hns3_reset_notify(h, HNAE3_UNINIT_CLIENT);
+ if (ret)
+ return ret;
+
+ ret = hns3_reset_notify(h, HNAE3_INIT_CLIENT);
+ if (ret)
+ return ret;
+
+ ret = hns3_reset_notify(h, HNAE3_UP_CLIENT);
+ if (ret)
+ hns3_reset_notify(h, HNAE3_UNINIT_CLIENT);
+
+ return ret;
+}
+
static int hns3_set_tunable(struct net_device *netdev,
const struct ethtool_tunable *tuna,
const void *data)
{
struct hns3_nic_priv *priv = netdev_priv(netdev);
+ u32 old_tx_spare_buf_size, new_tx_spare_buf_size;
struct hnae3_handle *h = priv->ae_handle;
int i, ret = 0;
@@ -1740,6 +1812,26 @@ static int hns3_set_tunable(struct net_device *netdev,
priv->ring[i].rx_copybreak = priv->rx_copybreak;
break;
+ case ETHTOOL_TX_COPYBREAK_BUF_SIZE:
+ old_tx_spare_buf_size = h->kinfo.tx_spare_buf_size;
+ new_tx_spare_buf_size = *(u32 *)data;
+ ret = hns3_set_tx_spare_buf_size(netdev, new_tx_spare_buf_size);
+ if (ret) {
+ int ret1;
+
+ netdev_warn(netdev,
+ "change tx spare buf size fail, revert to old value\n");
+ ret1 = hns3_set_tx_spare_buf_size(netdev,
+ old_tx_spare_buf_size);
+ if (ret1) {
+ netdev_err(netdev,
+ "revert to old tx spare buf size fail\n");
+ return ret1;
+ }
+
+ return ret;
+ }
+ break;
default:
ret = -EOPNOTSUPP;
break;
@@ -1755,6 +1847,8 @@ static int hns3_set_tunable(struct net_device *netdev,
ETHTOOL_COALESCE_MAX_FRAMES | \
ETHTOOL_COALESCE_USE_CQE)
+#define HNS3_ETHTOOL_RING ETHTOOL_RING_USE_RX_BUF_LEN
+
static int hns3_get_ts_info(struct net_device *netdev,
struct ethtool_ts_info *info)
{
@@ -1833,6 +1927,7 @@ static int hns3_get_link_ext_state(struct net_device *netdev,
static const struct ethtool_ops hns3vf_ethtool_ops = {
.supported_coalesce_params = HNS3_ETHTOOL_COALESCE,
+ .supported_ring_params = HNS3_ETHTOOL_RING,
.get_drvinfo = hns3_get_drvinfo,
.get_ringparam = hns3_get_ringparam,
.set_ringparam = hns3_set_ringparam,
@@ -1864,6 +1959,7 @@ static const struct ethtool_ops hns3vf_ethtool_ops = {
static const struct ethtool_ops hns3_ethtool_ops = {
.supported_coalesce_params = HNS3_ETHTOOL_COALESCE,
+ .supported_ring_params = HNS3_ETHTOOL_RING,
.self_test = hns3_self_test,
.get_drvinfo = hns3_get_drvinfo,
.get_link = hns3_get_link,
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/Makefile b/drivers/net/ethernet/hisilicon/hns3/hns3pf/Makefile
deleted file mode 100644
index d1bf5c4..0000000
--- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/Makefile
+++ /dev/null
@@ -1,12 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0+
-#
-# Makefile for the HISILICON network device drivers.
-#
-
-ccflags-y := -I $(srctree)/drivers/net/ethernet/hisilicon/hns3
-ccflags-y += -I $(srctree)/$(src)
-
-obj-$(CONFIG_HNS3_HCLGE) += hclge.o
-hclge-objs = hclge_main.o hclge_cmd.o hclge_mdio.o hclge_tm.o hclge_mbx.o hclge_err.o hclge_debugfs.o hclge_ptp.o hclge_devlink.o
-
-hclge-$(CONFIG_HNS3_DCB) += hclge_dcb.o
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_cmd.c b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_cmd.c
deleted file mode 100644
index c5d5466..0000000
--- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_cmd.c
+++ /dev/null
@@ -1,591 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0+
-// Copyright (c) 2016-2017 Hisilicon Limited.
-
-#include <linux/dma-mapping.h>
-#include <linux/slab.h>
-#include <linux/pci.h>
-#include <linux/device.h>
-#include <linux/err.h>
-#include <linux/dma-direction.h>
-#include "hclge_cmd.h"
-#include "hnae3.h"
-#include "hclge_main.h"
-
-#define cmq_ring_to_dev(ring) (&(ring)->dev->pdev->dev)
-
-static int hclge_ring_space(struct hclge_cmq_ring *ring)
-{
- int ntu = ring->next_to_use;
- int ntc = ring->next_to_clean;
- int used = (ntu - ntc + ring->desc_num) % ring->desc_num;
-
- return ring->desc_num - used - 1;
-}
-
-static int is_valid_csq_clean_head(struct hclge_cmq_ring *ring, int head)
-{
- int ntu = ring->next_to_use;
- int ntc = ring->next_to_clean;
-
- if (ntu > ntc)
- return head >= ntc && head <= ntu;
-
- return head >= ntc || head <= ntu;
-}
-
-static int hclge_alloc_cmd_desc(struct hclge_cmq_ring *ring)
-{
- int size = ring->desc_num * sizeof(struct hclge_desc);
-
- ring->desc = dma_alloc_coherent(cmq_ring_to_dev(ring), size,
- &ring->desc_dma_addr, GFP_KERNEL);
- if (!ring->desc)
- return -ENOMEM;
-
- return 0;
-}
-
-static void hclge_free_cmd_desc(struct hclge_cmq_ring *ring)
-{
- int size = ring->desc_num * sizeof(struct hclge_desc);
-
- if (ring->desc) {
- dma_free_coherent(cmq_ring_to_dev(ring), size,
- ring->desc, ring->desc_dma_addr);
- ring->desc = NULL;
- }
-}
-
-static int hclge_alloc_cmd_queue(struct hclge_dev *hdev, int ring_type)
-{
- struct hclge_hw *hw = &hdev->hw;
- struct hclge_cmq_ring *ring =
- (ring_type == HCLGE_TYPE_CSQ) ? &hw->cmq.csq : &hw->cmq.crq;
- int ret;
-
- ring->ring_type = ring_type;
- ring->dev = hdev;
-
- ret = hclge_alloc_cmd_desc(ring);
- if (ret) {
- dev_err(&hdev->pdev->dev, "descriptor %s alloc error %d\n",
- (ring_type == HCLGE_TYPE_CSQ) ? "CSQ" : "CRQ", ret);
- return ret;
- }
-
- return 0;
-}
-
-void hclge_cmd_reuse_desc(struct hclge_desc *desc, bool is_read)
-{
- desc->flag = cpu_to_le16(HCLGE_CMD_FLAG_NO_INTR | HCLGE_CMD_FLAG_IN);
- if (is_read)
- desc->flag |= cpu_to_le16(HCLGE_CMD_FLAG_WR);
- else
- desc->flag &= cpu_to_le16(~HCLGE_CMD_FLAG_WR);
-}
-
-void hclge_cmd_setup_basic_desc(struct hclge_desc *desc,
- enum hclge_opcode_type opcode, bool is_read)
-{
- memset((void *)desc, 0, sizeof(struct hclge_desc));
- desc->opcode = cpu_to_le16(opcode);
- desc->flag = cpu_to_le16(HCLGE_CMD_FLAG_NO_INTR | HCLGE_CMD_FLAG_IN);
-
- if (is_read)
- desc->flag |= cpu_to_le16(HCLGE_CMD_FLAG_WR);
-}
-
-static void hclge_cmd_config_regs(struct hclge_cmq_ring *ring)
-{
- dma_addr_t dma = ring->desc_dma_addr;
- struct hclge_dev *hdev = ring->dev;
- struct hclge_hw *hw = &hdev->hw;
- u32 reg_val;
-
- if (ring->ring_type == HCLGE_TYPE_CSQ) {
- hclge_write_dev(hw, HCLGE_NIC_CSQ_BASEADDR_L_REG,
- lower_32_bits(dma));
- hclge_write_dev(hw, HCLGE_NIC_CSQ_BASEADDR_H_REG,
- upper_32_bits(dma));
- reg_val = hclge_read_dev(hw, HCLGE_NIC_CSQ_DEPTH_REG);
- reg_val &= HCLGE_NIC_SW_RST_RDY;
- reg_val |= ring->desc_num >> HCLGE_NIC_CMQ_DESC_NUM_S;
- hclge_write_dev(hw, HCLGE_NIC_CSQ_DEPTH_REG, reg_val);
- hclge_write_dev(hw, HCLGE_NIC_CSQ_HEAD_REG, 0);
- hclge_write_dev(hw, HCLGE_NIC_CSQ_TAIL_REG, 0);
- } else {
- hclge_write_dev(hw, HCLGE_NIC_CRQ_BASEADDR_L_REG,
- lower_32_bits(dma));
- hclge_write_dev(hw, HCLGE_NIC_CRQ_BASEADDR_H_REG,
- upper_32_bits(dma));
- hclge_write_dev(hw, HCLGE_NIC_CRQ_DEPTH_REG,
- ring->desc_num >> HCLGE_NIC_CMQ_DESC_NUM_S);
- hclge_write_dev(hw, HCLGE_NIC_CRQ_HEAD_REG, 0);
- hclge_write_dev(hw, HCLGE_NIC_CRQ_TAIL_REG, 0);
- }
-}
-
-static void hclge_cmd_init_regs(struct hclge_hw *hw)
-{
- hclge_cmd_config_regs(&hw->cmq.csq);
- hclge_cmd_config_regs(&hw->cmq.crq);
-}
-
-static int hclge_cmd_csq_clean(struct hclge_hw *hw)
-{
- struct hclge_dev *hdev = container_of(hw, struct hclge_dev, hw);
- struct hclge_cmq_ring *csq = &hw->cmq.csq;
- u32 head;
- int clean;
-
- head = hclge_read_dev(hw, HCLGE_NIC_CSQ_HEAD_REG);
- rmb(); /* Make sure head is ready before touch any data */
-
- if (!is_valid_csq_clean_head(csq, head)) {
- dev_warn(&hdev->pdev->dev, "wrong cmd head (%u, %d-%d)\n", head,
- csq->next_to_use, csq->next_to_clean);
- dev_warn(&hdev->pdev->dev,
- "Disabling any further commands to IMP firmware\n");
- set_bit(HCLGE_STATE_CMD_DISABLE, &hdev->state);
- dev_warn(&hdev->pdev->dev,
- "IMP firmware watchdog reset soon expected!\n");
- return -EIO;
- }
-
- clean = (head - csq->next_to_clean + csq->desc_num) % csq->desc_num;
- csq->next_to_clean = head;
- return clean;
-}
-
-static int hclge_cmd_csq_done(struct hclge_hw *hw)
-{
- u32 head = hclge_read_dev(hw, HCLGE_NIC_CSQ_HEAD_REG);
- return head == hw->cmq.csq.next_to_use;
-}
-
-static bool hclge_is_special_opcode(u16 opcode)
-{
- /* these commands have several descriptors,
- * and use the first one to save opcode and return value
- */
- static const u16 spec_opcode[] = {
- HCLGE_OPC_STATS_64_BIT,
- HCLGE_OPC_STATS_32_BIT,
- HCLGE_OPC_STATS_MAC,
- HCLGE_OPC_STATS_MAC_ALL,
- HCLGE_OPC_QUERY_32_BIT_REG,
- HCLGE_OPC_QUERY_64_BIT_REG,
- HCLGE_QUERY_CLEAR_MPF_RAS_INT,
- HCLGE_QUERY_CLEAR_PF_RAS_INT,
- HCLGE_QUERY_CLEAR_ALL_MPF_MSIX_INT,
- HCLGE_QUERY_CLEAR_ALL_PF_MSIX_INT,
- HCLGE_QUERY_ALL_ERR_INFO
- };
- int i;
-
- for (i = 0; i < ARRAY_SIZE(spec_opcode); i++) {
- if (spec_opcode[i] == opcode)
- return true;
- }
-
- return false;
-}
-
-struct errcode {
- u32 imp_errcode;
- int common_errno;
-};
-
-static void hclge_cmd_copy_desc(struct hclge_hw *hw, struct hclge_desc *desc,
- int num)
-{
- struct hclge_desc *desc_to_use;
- int handle = 0;
-
- while (handle < num) {
- desc_to_use = &hw->cmq.csq.desc[hw->cmq.csq.next_to_use];
- *desc_to_use = desc[handle];
- (hw->cmq.csq.next_to_use)++;
- if (hw->cmq.csq.next_to_use >= hw->cmq.csq.desc_num)
- hw->cmq.csq.next_to_use = 0;
- handle++;
- }
-}
-
-static int hclge_cmd_convert_err_code(u16 desc_ret)
-{
- struct errcode hclge_cmd_errcode[] = {
- {HCLGE_CMD_EXEC_SUCCESS, 0},
- {HCLGE_CMD_NO_AUTH, -EPERM},
- {HCLGE_CMD_NOT_SUPPORTED, -EOPNOTSUPP},
- {HCLGE_CMD_QUEUE_FULL, -EXFULL},
- {HCLGE_CMD_NEXT_ERR, -ENOSR},
- {HCLGE_CMD_UNEXE_ERR, -ENOTBLK},
- {HCLGE_CMD_PARA_ERR, -EINVAL},
- {HCLGE_CMD_RESULT_ERR, -ERANGE},
- {HCLGE_CMD_TIMEOUT, -ETIME},
- {HCLGE_CMD_HILINK_ERR, -ENOLINK},
- {HCLGE_CMD_QUEUE_ILLEGAL, -ENXIO},
- {HCLGE_CMD_INVALID, -EBADR},
- };
- u32 errcode_count = ARRAY_SIZE(hclge_cmd_errcode);
- u32 i;
-
- for (i = 0; i < errcode_count; i++)
- if (hclge_cmd_errcode[i].imp_errcode == desc_ret)
- return hclge_cmd_errcode[i].common_errno;
-
- return -EIO;
-}
-
-static int hclge_cmd_check_retval(struct hclge_hw *hw, struct hclge_desc *desc,
- int num, int ntc)
-{
- u16 opcode, desc_ret;
- int handle;
-
- opcode = le16_to_cpu(desc[0].opcode);
- for (handle = 0; handle < num; handle++) {
- desc[handle] = hw->cmq.csq.desc[ntc];
- ntc++;
- if (ntc >= hw->cmq.csq.desc_num)
- ntc = 0;
- }
- if (likely(!hclge_is_special_opcode(opcode)))
- desc_ret = le16_to_cpu(desc[num - 1].retval);
- else
- desc_ret = le16_to_cpu(desc[0].retval);
-
- hw->cmq.last_status = desc_ret;
-
- return hclge_cmd_convert_err_code(desc_ret);
-}
-
-static int hclge_cmd_check_result(struct hclge_hw *hw, struct hclge_desc *desc,
- int num, int ntc)
-{
- struct hclge_dev *hdev = container_of(hw, struct hclge_dev, hw);
- bool is_completed = false;
- u32 timeout = 0;
- int handle, ret;
-
- /**
- * If the command is sync, wait for the firmware to write back,
- * if multi descriptors to be sent, use the first one to check
- */
- if (HCLGE_SEND_SYNC(le16_to_cpu(desc->flag))) {
- do {
- if (hclge_cmd_csq_done(hw)) {
- is_completed = true;
- break;
- }
- udelay(1);
- timeout++;
- } while (timeout < hw->cmq.tx_timeout);
- }
-
- if (!is_completed)
- ret = -EBADE;
- else
- ret = hclge_cmd_check_retval(hw, desc, num, ntc);
-
- /* Clean the command send queue */
- handle = hclge_cmd_csq_clean(hw);
- if (handle < 0)
- ret = handle;
- else if (handle != num)
- dev_warn(&hdev->pdev->dev,
- "cleaned %d, need to clean %d\n", handle, num);
- return ret;
-}
-
-/**
- * hclge_cmd_send - send command to command queue
- * @hw: pointer to the hw struct
- * @desc: prefilled descriptor for describing the command
- * @num : the number of descriptors to be sent
- *
- * This is the main send command for command queue, it
- * sends the queue, cleans the queue, etc
- **/
-int hclge_cmd_send(struct hclge_hw *hw, struct hclge_desc *desc, int num)
-{
- struct hclge_dev *hdev = container_of(hw, struct hclge_dev, hw);
- struct hclge_cmq_ring *csq = &hw->cmq.csq;
- int ret;
- int ntc;
-
- spin_lock_bh(&hw->cmq.csq.lock);
-
- if (test_bit(HCLGE_STATE_CMD_DISABLE, &hdev->state)) {
- spin_unlock_bh(&hw->cmq.csq.lock);
- return -EBUSY;
- }
-
- if (num > hclge_ring_space(&hw->cmq.csq)) {
- /* If CMDQ ring is full, SW HEAD and HW HEAD may be different,
- * need update the SW HEAD pointer csq->next_to_clean
- */
- csq->next_to_clean = hclge_read_dev(hw, HCLGE_NIC_CSQ_HEAD_REG);
- spin_unlock_bh(&hw->cmq.csq.lock);
- return -EBUSY;
- }
-
- /**
- * Record the location of desc in the ring for this time
- * which will be use for hardware to write back
- */
- ntc = hw->cmq.csq.next_to_use;
-
- hclge_cmd_copy_desc(hw, desc, num);
-
- /* Write to hardware */
- hclge_write_dev(hw, HCLGE_NIC_CSQ_TAIL_REG, hw->cmq.csq.next_to_use);
-
- ret = hclge_cmd_check_result(hw, desc, num, ntc);
-
- spin_unlock_bh(&hw->cmq.csq.lock);
-
- return ret;
-}
-
-static void hclge_set_default_capability(struct hclge_dev *hdev)
-{
- struct hnae3_ae_dev *ae_dev = pci_get_drvdata(hdev->pdev);
-
- set_bit(HNAE3_DEV_SUPPORT_FD_B, ae_dev->caps);
- set_bit(HNAE3_DEV_SUPPORT_GRO_B, ae_dev->caps);
- if (hdev->ae_dev->dev_version == HNAE3_DEVICE_VERSION_V2) {
- set_bit(HNAE3_DEV_SUPPORT_FEC_B, ae_dev->caps);
- set_bit(HNAE3_DEV_SUPPORT_PAUSE_B, ae_dev->caps);
- }
-}
-
-static const struct hclge_caps_bit_map hclge_cmd_caps_bit_map0[] = {
- {HCLGE_CAP_UDP_GSO_B, HNAE3_DEV_SUPPORT_UDP_GSO_B},
- {HCLGE_CAP_PTP_B, HNAE3_DEV_SUPPORT_PTP_B},
- {HCLGE_CAP_INT_QL_B, HNAE3_DEV_SUPPORT_INT_QL_B},
- {HCLGE_CAP_TQP_TXRX_INDEP_B, HNAE3_DEV_SUPPORT_TQP_TXRX_INDEP_B},
- {HCLGE_CAP_HW_TX_CSUM_B, HNAE3_DEV_SUPPORT_HW_TX_CSUM_B},
- {HCLGE_CAP_UDP_TUNNEL_CSUM_B, HNAE3_DEV_SUPPORT_UDP_TUNNEL_CSUM_B},
- {HCLGE_CAP_FD_FORWARD_TC_B, HNAE3_DEV_SUPPORT_FD_FORWARD_TC_B},
- {HCLGE_CAP_FEC_B, HNAE3_DEV_SUPPORT_FEC_B},
- {HCLGE_CAP_PAUSE_B, HNAE3_DEV_SUPPORT_PAUSE_B},
- {HCLGE_CAP_PHY_IMP_B, HNAE3_DEV_SUPPORT_PHY_IMP_B},
- {HCLGE_CAP_RAS_IMP_B, HNAE3_DEV_SUPPORT_RAS_IMP_B},
- {HCLGE_CAP_RXD_ADV_LAYOUT_B, HNAE3_DEV_SUPPORT_RXD_ADV_LAYOUT_B},
- {HCLGE_CAP_PORT_VLAN_BYPASS_B, HNAE3_DEV_SUPPORT_PORT_VLAN_BYPASS_B},
- {HCLGE_CAP_PORT_VLAN_BYPASS_B, HNAE3_DEV_SUPPORT_VLAN_FLTR_MDF_B},
-};
-
-static void hclge_parse_capability(struct hclge_dev *hdev,
- struct hclge_query_version_cmd *cmd)
-{
- struct hnae3_ae_dev *ae_dev = pci_get_drvdata(hdev->pdev);
- u32 caps, i;
-
- caps = __le32_to_cpu(cmd->caps[0]);
- for (i = 0; i < ARRAY_SIZE(hclge_cmd_caps_bit_map0); i++)
- if (hnae3_get_bit(caps, hclge_cmd_caps_bit_map0[i].imp_bit))
- set_bit(hclge_cmd_caps_bit_map0[i].local_bit,
- ae_dev->caps);
-}
-
-static __le32 hclge_build_api_caps(void)
-{
- u32 api_caps = 0;
-
- hnae3_set_bit(api_caps, HCLGE_API_CAP_FLEX_RSS_TBL_B, 1);
-
- return cpu_to_le32(api_caps);
-}
-
-static enum hclge_cmd_status
-hclge_cmd_query_version_and_capability(struct hclge_dev *hdev)
-{
- struct hnae3_ae_dev *ae_dev = pci_get_drvdata(hdev->pdev);
- struct hclge_query_version_cmd *resp;
- struct hclge_desc desc;
- int ret;
-
- hclge_cmd_setup_basic_desc(&desc, HCLGE_OPC_QUERY_FW_VER, 1);
- resp = (struct hclge_query_version_cmd *)desc.data;
- resp->api_caps = hclge_build_api_caps();
-
- ret = hclge_cmd_send(&hdev->hw, &desc, 1);
- if (ret)
- return ret;
-
- hdev->fw_version = le32_to_cpu(resp->firmware);
-
- ae_dev->dev_version = le32_to_cpu(resp->hardware) <<
- HNAE3_PCI_REVISION_BIT_SIZE;
- ae_dev->dev_version |= hdev->pdev->revision;
-
- if (ae_dev->dev_version >= HNAE3_DEVICE_VERSION_V2)
- hclge_set_default_capability(hdev);
-
- hclge_parse_capability(hdev, resp);
-
- return ret;
-}
-
-int hclge_cmd_queue_init(struct hclge_dev *hdev)
-{
- int ret;
-
- /* Setup the lock for command queue */
- spin_lock_init(&hdev->hw.cmq.csq.lock);
- spin_lock_init(&hdev->hw.cmq.crq.lock);
-
- /* Setup the queue entries for use cmd queue */
- hdev->hw.cmq.csq.desc_num = HCLGE_NIC_CMQ_DESC_NUM;
- hdev->hw.cmq.crq.desc_num = HCLGE_NIC_CMQ_DESC_NUM;
-
- /* Setup Tx write back timeout */
- hdev->hw.cmq.tx_timeout = HCLGE_CMDQ_TX_TIMEOUT;
-
- /* Setup queue rings */
- ret = hclge_alloc_cmd_queue(hdev, HCLGE_TYPE_CSQ);
- if (ret) {
- dev_err(&hdev->pdev->dev,
- "CSQ ring setup error %d\n", ret);
- return ret;
- }
-
- ret = hclge_alloc_cmd_queue(hdev, HCLGE_TYPE_CRQ);
- if (ret) {
- dev_err(&hdev->pdev->dev,
- "CRQ ring setup error %d\n", ret);
- goto err_csq;
- }
-
- return 0;
-err_csq:
- hclge_free_cmd_desc(&hdev->hw.cmq.csq);
- return ret;
-}
-
-static int hclge_firmware_compat_config(struct hclge_dev *hdev, bool en)
-{
- struct hclge_firmware_compat_cmd *req;
- struct hclge_desc desc;
- u32 compat = 0;
-
- hclge_cmd_setup_basic_desc(&desc, HCLGE_OPC_IMP_COMPAT_CFG, false);
-
- if (en) {
- req = (struct hclge_firmware_compat_cmd *)desc.data;
-
- hnae3_set_bit(compat, HCLGE_LINK_EVENT_REPORT_EN_B, 1);
- hnae3_set_bit(compat, HCLGE_NCSI_ERROR_REPORT_EN_B, 1);
- if (hnae3_dev_phy_imp_supported(hdev))
- hnae3_set_bit(compat, HCLGE_PHY_IMP_EN_B, 1);
- hnae3_set_bit(compat, HCLGE_MAC_STATS_EXT_EN_B, 1);
- hnae3_set_bit(compat, HCLGE_SYNC_RX_RING_HEAD_EN_B, 1);
-
- req->compat = cpu_to_le32(compat);
- }
-
- return hclge_cmd_send(&hdev->hw, &desc, 1);
-}
-
-int hclge_cmd_init(struct hclge_dev *hdev)
-{
- int ret;
-
- spin_lock_bh(&hdev->hw.cmq.csq.lock);
- spin_lock(&hdev->hw.cmq.crq.lock);
-
- hdev->hw.cmq.csq.next_to_clean = 0;
- hdev->hw.cmq.csq.next_to_use = 0;
- hdev->hw.cmq.crq.next_to_clean = 0;
- hdev->hw.cmq.crq.next_to_use = 0;
-
- hclge_cmd_init_regs(&hdev->hw);
-
- spin_unlock(&hdev->hw.cmq.crq.lock);
- spin_unlock_bh(&hdev->hw.cmq.csq.lock);
-
- clear_bit(HCLGE_STATE_CMD_DISABLE, &hdev->state);
-
- /* Check if there is new reset pending, because the higher level
- * reset may happen when lower level reset is being processed.
- */
- if ((hclge_is_reset_pending(hdev))) {
- dev_err(&hdev->pdev->dev,
- "failed to init cmd since reset %#lx pending\n",
- hdev->reset_pending);
- ret = -EBUSY;
- goto err_cmd_init;
- }
-
- /* get version and device capabilities */
- ret = hclge_cmd_query_version_and_capability(hdev);
- if (ret) {
- dev_err(&hdev->pdev->dev,
- "failed to query version and capabilities, ret = %d\n",
- ret);
- goto err_cmd_init;
- }
-
- dev_info(&hdev->pdev->dev, "The firmware version is %lu.%lu.%lu.%lu\n",
- hnae3_get_field(hdev->fw_version, HNAE3_FW_VERSION_BYTE3_MASK,
- HNAE3_FW_VERSION_BYTE3_SHIFT),
- hnae3_get_field(hdev->fw_version, HNAE3_FW_VERSION_BYTE2_MASK,
- HNAE3_FW_VERSION_BYTE2_SHIFT),
- hnae3_get_field(hdev->fw_version, HNAE3_FW_VERSION_BYTE1_MASK,
- HNAE3_FW_VERSION_BYTE1_SHIFT),
- hnae3_get_field(hdev->fw_version, HNAE3_FW_VERSION_BYTE0_MASK,
- HNAE3_FW_VERSION_BYTE0_SHIFT));
-
- /* ask the firmware to enable some features, driver can work without
- * it.
- */
- ret = hclge_firmware_compat_config(hdev, true);
- if (ret)
- dev_warn(&hdev->pdev->dev,
- "Firmware compatible features not enabled(%d).\n",
- ret);
-
- return 0;
-
-err_cmd_init:
- set_bit(HCLGE_STATE_CMD_DISABLE, &hdev->state);
-
- return ret;
-}
-
-static void hclge_cmd_uninit_regs(struct hclge_hw *hw)
-{
- hclge_write_dev(hw, HCLGE_NIC_CSQ_BASEADDR_L_REG, 0);
- hclge_write_dev(hw, HCLGE_NIC_CSQ_BASEADDR_H_REG, 0);
- hclge_write_dev(hw, HCLGE_NIC_CSQ_DEPTH_REG, 0);
- hclge_write_dev(hw, HCLGE_NIC_CSQ_HEAD_REG, 0);
- hclge_write_dev(hw, HCLGE_NIC_CSQ_TAIL_REG, 0);
- hclge_write_dev(hw, HCLGE_NIC_CRQ_BASEADDR_L_REG, 0);
- hclge_write_dev(hw, HCLGE_NIC_CRQ_BASEADDR_H_REG, 0);
- hclge_write_dev(hw, HCLGE_NIC_CRQ_DEPTH_REG, 0);
- hclge_write_dev(hw, HCLGE_NIC_CRQ_HEAD_REG, 0);
- hclge_write_dev(hw, HCLGE_NIC_CRQ_TAIL_REG, 0);
-}
-
-void hclge_cmd_uninit(struct hclge_dev *hdev)
-{
- hclge_firmware_compat_config(hdev, false);
-
- set_bit(HCLGE_STATE_CMD_DISABLE, &hdev->state);
- /* wait to ensure that the firmware completes the possible left
- * over commands.
- */
- msleep(HCLGE_CMDQ_CLEAR_WAIT_TIME);
- spin_lock_bh(&hdev->hw.cmq.csq.lock);
- spin_lock(&hdev->hw.cmq.crq.lock);
- hclge_cmd_uninit_regs(&hdev->hw);
- spin_unlock(&hdev->hw.cmq.crq.lock);
- spin_unlock_bh(&hdev->hw.cmq.csq.lock);
-
- hclge_free_cmd_desc(&hdev->hw.cmq.csq);
- hclge_free_cmd_desc(&hdev->hw.cmq.crq);
-}
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_cmd.h b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_cmd.h
index d24e590..f9d8951 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_cmd.h
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_cmd.h
@@ -7,323 +7,21 @@
#include <linux/io.h>
#include <linux/etherdevice.h>
#include "hnae3.h"
-
-#define HCLGE_CMDQ_TX_TIMEOUT 30000
-#define HCLGE_CMDQ_CLEAR_WAIT_TIME 200
-#define HCLGE_DESC_DATA_LEN 6
+#include "hclge_comm_cmd.h"
struct hclge_dev;
-struct hclge_desc {
- __le16 opcode;
#define HCLGE_CMDQ_RX_INVLD_B 0
#define HCLGE_CMDQ_RX_OUTVLD_B 1
- __le16 flag;
- __le16 retval;
- __le16 rsv;
- __le32 data[HCLGE_DESC_DATA_LEN];
-};
-
-struct hclge_cmq_ring {
- dma_addr_t desc_dma_addr;
- struct hclge_desc *desc;
- struct hclge_dev *dev;
- u32 head;
- u32 tail;
-
- u16 buf_size;
- u16 desc_num;
- int next_to_use;
- int next_to_clean;
- u8 ring_type; /* cmq ring type */
- spinlock_t lock; /* Command queue lock */
-};
-
-enum hclge_cmd_return_status {
- HCLGE_CMD_EXEC_SUCCESS = 0,
- HCLGE_CMD_NO_AUTH = 1,
- HCLGE_CMD_NOT_SUPPORTED = 2,
- HCLGE_CMD_QUEUE_FULL = 3,
- HCLGE_CMD_NEXT_ERR = 4,
- HCLGE_CMD_UNEXE_ERR = 5,
- HCLGE_CMD_PARA_ERR = 6,
- HCLGE_CMD_RESULT_ERR = 7,
- HCLGE_CMD_TIMEOUT = 8,
- HCLGE_CMD_HILINK_ERR = 9,
- HCLGE_CMD_QUEUE_ILLEGAL = 10,
- HCLGE_CMD_INVALID = 11,
-};
-
-enum hclge_cmd_status {
- HCLGE_STATUS_SUCCESS = 0,
- HCLGE_ERR_CSQ_FULL = -1,
- HCLGE_ERR_CSQ_TIMEOUT = -2,
- HCLGE_ERR_CSQ_ERROR = -3,
-};
-
struct hclge_misc_vector {
u8 __iomem *addr;
int vector_irq;
char name[HNAE3_INT_NAME_LEN];
};
-struct hclge_cmq {
- struct hclge_cmq_ring csq;
- struct hclge_cmq_ring crq;
- u16 tx_timeout;
- enum hclge_cmd_status last_status;
-};
-
-#define HCLGE_CMD_FLAG_IN BIT(0)
-#define HCLGE_CMD_FLAG_OUT BIT(1)
-#define HCLGE_CMD_FLAG_NEXT BIT(2)
-#define HCLGE_CMD_FLAG_WR BIT(3)
-#define HCLGE_CMD_FLAG_NO_INTR BIT(4)
-#define HCLGE_CMD_FLAG_ERR_INTR BIT(5)
-
-enum hclge_opcode_type {
- /* Generic commands */
- HCLGE_OPC_QUERY_FW_VER = 0x0001,
- HCLGE_OPC_CFG_RST_TRIGGER = 0x0020,
- HCLGE_OPC_GBL_RST_STATUS = 0x0021,
- HCLGE_OPC_QUERY_FUNC_STATUS = 0x0022,
- HCLGE_OPC_QUERY_PF_RSRC = 0x0023,
- HCLGE_OPC_QUERY_VF_RSRC = 0x0024,
- HCLGE_OPC_GET_CFG_PARAM = 0x0025,
- HCLGE_OPC_PF_RST_DONE = 0x0026,
- HCLGE_OPC_QUERY_VF_RST_RDY = 0x0027,
-
- HCLGE_OPC_STATS_64_BIT = 0x0030,
- HCLGE_OPC_STATS_32_BIT = 0x0031,
- HCLGE_OPC_STATS_MAC = 0x0032,
- HCLGE_OPC_QUERY_MAC_REG_NUM = 0x0033,
- HCLGE_OPC_STATS_MAC_ALL = 0x0034,
-
- HCLGE_OPC_QUERY_REG_NUM = 0x0040,
- HCLGE_OPC_QUERY_32_BIT_REG = 0x0041,
- HCLGE_OPC_QUERY_64_BIT_REG = 0x0042,
- HCLGE_OPC_DFX_BD_NUM = 0x0043,
- HCLGE_OPC_DFX_BIOS_COMMON_REG = 0x0044,
- HCLGE_OPC_DFX_SSU_REG_0 = 0x0045,
- HCLGE_OPC_DFX_SSU_REG_1 = 0x0046,
- HCLGE_OPC_DFX_IGU_EGU_REG = 0x0047,
- HCLGE_OPC_DFX_RPU_REG_0 = 0x0048,
- HCLGE_OPC_DFX_RPU_REG_1 = 0x0049,
- HCLGE_OPC_DFX_NCSI_REG = 0x004A,
- HCLGE_OPC_DFX_RTC_REG = 0x004B,
- HCLGE_OPC_DFX_PPP_REG = 0x004C,
- HCLGE_OPC_DFX_RCB_REG = 0x004D,
- HCLGE_OPC_DFX_TQP_REG = 0x004E,
- HCLGE_OPC_DFX_SSU_REG_2 = 0x004F,
-
- HCLGE_OPC_QUERY_DEV_SPECS = 0x0050,
-
- /* MAC command */
- HCLGE_OPC_CONFIG_MAC_MODE = 0x0301,
- HCLGE_OPC_CONFIG_AN_MODE = 0x0304,
- HCLGE_OPC_QUERY_LINK_STATUS = 0x0307,
- HCLGE_OPC_CONFIG_MAX_FRM_SIZE = 0x0308,
- HCLGE_OPC_CONFIG_SPEED_DUP = 0x0309,
- HCLGE_OPC_QUERY_MAC_TNL_INT = 0x0310,
- HCLGE_OPC_MAC_TNL_INT_EN = 0x0311,
- HCLGE_OPC_CLEAR_MAC_TNL_INT = 0x0312,
- HCLGE_OPC_COMMON_LOOPBACK = 0x0315,
- HCLGE_OPC_CONFIG_FEC_MODE = 0x031A,
-
- /* PTP commands */
- HCLGE_OPC_PTP_INT_EN = 0x0501,
- HCLGE_OPC_PTP_MODE_CFG = 0x0507,
-
- /* PFC/Pause commands */
- HCLGE_OPC_CFG_MAC_PAUSE_EN = 0x0701,
- HCLGE_OPC_CFG_PFC_PAUSE_EN = 0x0702,
- HCLGE_OPC_CFG_MAC_PARA = 0x0703,
- HCLGE_OPC_CFG_PFC_PARA = 0x0704,
- HCLGE_OPC_QUERY_MAC_TX_PKT_CNT = 0x0705,
- HCLGE_OPC_QUERY_MAC_RX_PKT_CNT = 0x0706,
- HCLGE_OPC_QUERY_PFC_TX_PKT_CNT = 0x0707,
- HCLGE_OPC_QUERY_PFC_RX_PKT_CNT = 0x0708,
- HCLGE_OPC_PRI_TO_TC_MAPPING = 0x0709,
- HCLGE_OPC_QOS_MAP = 0x070A,
-
- /* ETS/scheduler commands */
- HCLGE_OPC_TM_PG_TO_PRI_LINK = 0x0804,
- HCLGE_OPC_TM_QS_TO_PRI_LINK = 0x0805,
- HCLGE_OPC_TM_NQ_TO_QS_LINK = 0x0806,
- HCLGE_OPC_TM_RQ_TO_QS_LINK = 0x0807,
- HCLGE_OPC_TM_PORT_WEIGHT = 0x0808,
- HCLGE_OPC_TM_PG_WEIGHT = 0x0809,
- HCLGE_OPC_TM_QS_WEIGHT = 0x080A,
- HCLGE_OPC_TM_PRI_WEIGHT = 0x080B,
- HCLGE_OPC_TM_PRI_C_SHAPPING = 0x080C,
- HCLGE_OPC_TM_PRI_P_SHAPPING = 0x080D,
- HCLGE_OPC_TM_PG_C_SHAPPING = 0x080E,
- HCLGE_OPC_TM_PG_P_SHAPPING = 0x080F,
- HCLGE_OPC_TM_PORT_SHAPPING = 0x0810,
- HCLGE_OPC_TM_PG_SCH_MODE_CFG = 0x0812,
- HCLGE_OPC_TM_PRI_SCH_MODE_CFG = 0x0813,
- HCLGE_OPC_TM_QS_SCH_MODE_CFG = 0x0814,
- HCLGE_OPC_TM_BP_TO_QSET_MAPPING = 0x0815,
- HCLGE_OPC_TM_NODES = 0x0816,
- HCLGE_OPC_ETS_TC_WEIGHT = 0x0843,
- HCLGE_OPC_QSET_DFX_STS = 0x0844,
- HCLGE_OPC_PRI_DFX_STS = 0x0845,
- HCLGE_OPC_PG_DFX_STS = 0x0846,
- HCLGE_OPC_PORT_DFX_STS = 0x0847,
- HCLGE_OPC_SCH_NQ_CNT = 0x0848,
- HCLGE_OPC_SCH_RQ_CNT = 0x0849,
- HCLGE_OPC_TM_INTERNAL_STS = 0x0850,
- HCLGE_OPC_TM_INTERNAL_CNT = 0x0851,
- HCLGE_OPC_TM_INTERNAL_STS_1 = 0x0852,
-
- /* Packet buffer allocate commands */
- HCLGE_OPC_TX_BUFF_ALLOC = 0x0901,
- HCLGE_OPC_RX_PRIV_BUFF_ALLOC = 0x0902,
- HCLGE_OPC_RX_PRIV_WL_ALLOC = 0x0903,
- HCLGE_OPC_RX_COM_THRD_ALLOC = 0x0904,
- HCLGE_OPC_RX_COM_WL_ALLOC = 0x0905,
- HCLGE_OPC_RX_GBL_PKT_CNT = 0x0906,
-
- /* TQP management command */
- HCLGE_OPC_SET_TQP_MAP = 0x0A01,
-
- /* TQP commands */
- HCLGE_OPC_CFG_TX_QUEUE = 0x0B01,
- HCLGE_OPC_QUERY_TX_POINTER = 0x0B02,
- HCLGE_OPC_QUERY_TX_STATS = 0x0B03,
- HCLGE_OPC_TQP_TX_QUEUE_TC = 0x0B04,
- HCLGE_OPC_CFG_RX_QUEUE = 0x0B11,
- HCLGE_OPC_QUERY_RX_POINTER = 0x0B12,
- HCLGE_OPC_QUERY_RX_STATS = 0x0B13,
- HCLGE_OPC_STASH_RX_QUEUE_LRO = 0x0B16,
- HCLGE_OPC_CFG_RX_QUEUE_LRO = 0x0B17,
- HCLGE_OPC_CFG_COM_TQP_QUEUE = 0x0B20,
- HCLGE_OPC_RESET_TQP_QUEUE = 0x0B22,
-
- /* PPU commands */
- HCLGE_OPC_PPU_PF_OTHER_INT_DFX = 0x0B4A,
-
- /* TSO command */
- HCLGE_OPC_TSO_GENERIC_CONFIG = 0x0C01,
- HCLGE_OPC_GRO_GENERIC_CONFIG = 0x0C10,
-
- /* RSS commands */
- HCLGE_OPC_RSS_GENERIC_CONFIG = 0x0D01,
- HCLGE_OPC_RSS_INDIR_TABLE = 0x0D07,
- HCLGE_OPC_RSS_TC_MODE = 0x0D08,
- HCLGE_OPC_RSS_INPUT_TUPLE = 0x0D02,
-
- /* Promisuous mode command */
- HCLGE_OPC_CFG_PROMISC_MODE = 0x0E01,
-
- /* Vlan offload commands */
- HCLGE_OPC_VLAN_PORT_TX_CFG = 0x0F01,
- HCLGE_OPC_VLAN_PORT_RX_CFG = 0x0F02,
-
- /* Interrupts commands */
- HCLGE_OPC_ADD_RING_TO_VECTOR = 0x1503,
- HCLGE_OPC_DEL_RING_TO_VECTOR = 0x1504,
-
- /* MAC commands */
- HCLGE_OPC_MAC_VLAN_ADD = 0x1000,
- HCLGE_OPC_MAC_VLAN_REMOVE = 0x1001,
- HCLGE_OPC_MAC_VLAN_TYPE_ID = 0x1002,
- HCLGE_OPC_MAC_VLAN_INSERT = 0x1003,
- HCLGE_OPC_MAC_VLAN_ALLOCATE = 0x1004,
- HCLGE_OPC_MAC_ETHTYPE_ADD = 0x1010,
- HCLGE_OPC_MAC_ETHTYPE_REMOVE = 0x1011,
-
- /* MAC VLAN commands */
- HCLGE_OPC_MAC_VLAN_SWITCH_PARAM = 0x1033,
-
- /* VLAN commands */
- HCLGE_OPC_VLAN_FILTER_CTRL = 0x1100,
- HCLGE_OPC_VLAN_FILTER_PF_CFG = 0x1101,
- HCLGE_OPC_VLAN_FILTER_VF_CFG = 0x1102,
- HCLGE_OPC_PORT_VLAN_BYPASS = 0x1103,
-
- /* Flow Director commands */
- HCLGE_OPC_FD_MODE_CTRL = 0x1200,
- HCLGE_OPC_FD_GET_ALLOCATION = 0x1201,
- HCLGE_OPC_FD_KEY_CONFIG = 0x1202,
- HCLGE_OPC_FD_TCAM_OP = 0x1203,
- HCLGE_OPC_FD_AD_OP = 0x1204,
- HCLGE_OPC_FD_CNT_OP = 0x1205,
- HCLGE_OPC_FD_USER_DEF_OP = 0x1207,
-
- /* MDIO command */
- HCLGE_OPC_MDIO_CONFIG = 0x1900,
-
- /* QCN commands */
- HCLGE_OPC_QCN_MOD_CFG = 0x1A01,
- HCLGE_OPC_QCN_GRP_TMPLT_CFG = 0x1A02,
- HCLGE_OPC_QCN_SHAPPING_CFG = 0x1A03,
- HCLGE_OPC_QCN_SHAPPING_BS_CFG = 0x1A04,
- HCLGE_OPC_QCN_QSET_LINK_CFG = 0x1A05,
- HCLGE_OPC_QCN_RP_STATUS_GET = 0x1A06,
- HCLGE_OPC_QCN_AJUST_INIT = 0x1A07,
- HCLGE_OPC_QCN_DFX_CNT_STATUS = 0x1A08,
-
- /* Mailbox command */
- HCLGEVF_OPC_MBX_PF_TO_VF = 0x2000,
-
- /* Led command */
- HCLGE_OPC_LED_STATUS_CFG = 0xB000,
-
- /* clear hardware resource command */
- HCLGE_OPC_CLEAR_HW_RESOURCE = 0x700B,
-
- /* NCL config command */
- HCLGE_OPC_QUERY_NCL_CONFIG = 0x7011,
-
- /* IMP stats command */
- HCLGE_OPC_IMP_STATS_BD = 0x7012,
- HCLGE_OPC_IMP_STATS_INFO = 0x7013,
- HCLGE_OPC_IMP_COMPAT_CFG = 0x701A,
-
- /* SFP command */
- HCLGE_OPC_GET_SFP_EEPROM = 0x7100,
- HCLGE_OPC_GET_SFP_EXIST = 0x7101,
- HCLGE_OPC_GET_SFP_INFO = 0x7104,
-
- /* Error INT commands */
- HCLGE_MAC_COMMON_INT_EN = 0x030E,
- HCLGE_TM_SCH_ECC_INT_EN = 0x0829,
- HCLGE_SSU_ECC_INT_CMD = 0x0989,
- HCLGE_SSU_COMMON_INT_CMD = 0x098C,
- HCLGE_PPU_MPF_ECC_INT_CMD = 0x0B40,
- HCLGE_PPU_MPF_OTHER_INT_CMD = 0x0B41,
- HCLGE_PPU_PF_OTHER_INT_CMD = 0x0B42,
- HCLGE_COMMON_ECC_INT_CFG = 0x1505,
- HCLGE_QUERY_RAS_INT_STS_BD_NUM = 0x1510,
- HCLGE_QUERY_CLEAR_MPF_RAS_INT = 0x1511,
- HCLGE_QUERY_CLEAR_PF_RAS_INT = 0x1512,
- HCLGE_QUERY_MSIX_INT_STS_BD_NUM = 0x1513,
- HCLGE_QUERY_CLEAR_ALL_MPF_MSIX_INT = 0x1514,
- HCLGE_QUERY_CLEAR_ALL_PF_MSIX_INT = 0x1515,
- HCLGE_QUERY_ALL_ERR_BD_NUM = 0x1516,
- HCLGE_QUERY_ALL_ERR_INFO = 0x1517,
- HCLGE_CONFIG_ROCEE_RAS_INT_EN = 0x1580,
- HCLGE_QUERY_CLEAR_ROCEE_RAS_INT = 0x1581,
- HCLGE_ROCEE_PF_RAS_INT_CMD = 0x1584,
- HCLGE_QUERY_ROCEE_ECC_RAS_INFO_CMD = 0x1585,
- HCLGE_QUERY_ROCEE_AXI_RAS_INFO_CMD = 0x1586,
- HCLGE_IGU_EGU_TNL_INT_EN = 0x1803,
- HCLGE_IGU_COMMON_INT_EN = 0x1806,
- HCLGE_TM_QCN_MEM_INT_CFG = 0x1A14,
- HCLGE_PPP_CMD0_INT_CMD = 0x2100,
- HCLGE_PPP_CMD1_INT_CMD = 0x2101,
- HCLGE_MAC_ETHERTYPE_IDX_RD = 0x2105,
- HCLGE_NCSI_INT_EN = 0x2401,
-
- /* PHY command */
- HCLGE_OPC_PHY_LINK_KSETTING = 0x7025,
- HCLGE_OPC_PHY_REG = 0x7026,
-
- /* Query link diagnosis info command */
- HCLGE_OPC_QUERY_LINK_DIAGNOSIS = 0x702A,
-};
+#define hclge_cmd_setup_basic_desc(desc, opcode, is_read) \
+ hclge_comm_cmd_setup_basic_desc(desc, opcode, is_read)
#define HCLGE_TQP_REG_OFFSET 0x80000
#define HCLGE_TQP_REG_SIZE 0x200
@@ -391,38 +89,6 @@ struct hclge_rx_priv_buff_cmd {
u8 rsv[6];
};
-enum HCLGE_CAP_BITS {
- HCLGE_CAP_UDP_GSO_B,
- HCLGE_CAP_QB_B,
- HCLGE_CAP_FD_FORWARD_TC_B,
- HCLGE_CAP_PTP_B,
- HCLGE_CAP_INT_QL_B,
- HCLGE_CAP_HW_TX_CSUM_B,
- HCLGE_CAP_TX_PUSH_B,
- HCLGE_CAP_PHY_IMP_B,
- HCLGE_CAP_TQP_TXRX_INDEP_B,
- HCLGE_CAP_HW_PAD_B,
- HCLGE_CAP_STASH_B,
- HCLGE_CAP_UDP_TUNNEL_CSUM_B,
- HCLGE_CAP_RAS_IMP_B = 12,
- HCLGE_CAP_FEC_B = 13,
- HCLGE_CAP_PAUSE_B = 14,
- HCLGE_CAP_RXD_ADV_LAYOUT_B = 15,
- HCLGE_CAP_PORT_VLAN_BYPASS_B = 17,
-};
-
-enum HCLGE_API_CAP_BITS {
- HCLGE_API_CAP_FLEX_RSS_TBL_B,
-};
-
-#define HCLGE_QUERY_CAP_LENGTH 3
-struct hclge_query_version_cmd {
- __le32 firmware;
- __le32 hardware;
- __le32 api_caps;
- __le32 caps[HCLGE_QUERY_CAP_LENGTH]; /* capabilities of device */
-};
-
#define HCLGE_RX_PRIV_EN_B 15
#define HCLGE_TC_NUM_ONE_DESC 4
struct hclge_priv_wl {
@@ -571,38 +237,10 @@ struct hclge_vf_num_cmd {
};
#define HCLGE_RSS_DEFAULT_OUTPORT_B 4
-#define HCLGE_RSS_HASH_KEY_OFFSET_B 4
-#define HCLGE_RSS_HASH_KEY_NUM 16
-struct hclge_rss_config_cmd {
- u8 hash_config;
- u8 rsv[7];
- u8 hash_key[HCLGE_RSS_HASH_KEY_NUM];
-};
-struct hclge_rss_input_tuple_cmd {
- u8 ipv4_tcp_en;
- u8 ipv4_udp_en;
- u8 ipv4_sctp_en;
- u8 ipv4_fragment_en;
- u8 ipv6_tcp_en;
- u8 ipv6_udp_en;
- u8 ipv6_sctp_en;
- u8 ipv6_fragment_en;
- u8 rsv[16];
-};
-
-#define HCLGE_RSS_CFG_TBL_SIZE 16
#define HCLGE_RSS_CFG_TBL_SIZE_H 4
-#define HCLGE_RSS_CFG_TBL_BW_H 2U
#define HCLGE_RSS_CFG_TBL_BW_L 8U
-struct hclge_rss_indirection_table_cmd {
- __le16 start_table_index;
- __le16 rss_set_bitmap;
- u8 rss_qid_h[HCLGE_RSS_CFG_TBL_SIZE_H];
- u8 rss_qid_l[HCLGE_RSS_CFG_TBL_SIZE];
-};
-
#define HCLGE_RSS_TC_OFFSET_S 0
#define HCLGE_RSS_TC_OFFSET_M GENMASK(10, 0)
#define HCLGE_RSS_TC_SIZE_MSB_B 11
@@ -610,10 +248,6 @@ struct hclge_rss_indirection_table_cmd {
#define HCLGE_RSS_TC_SIZE_M GENMASK(14, 12)
#define HCLGE_RSS_TC_SIZE_MSB_OFFSET 3
#define HCLGE_RSS_TC_VALID_B 15
-struct hclge_rss_tc_mode_cmd {
- __le16 rss_tc_mode[HCLGE_MAX_TC_NUM];
- u8 rsv[8];
-};
#define HCLGE_LINK_STATUS_UP_B 0
#define HCLGE_LINK_STATUS_UP_M BIT(HCLGE_LINK_STATUS_UP_B)
@@ -1015,16 +649,6 @@ struct hclge_common_lb_cmd {
#define HCLGE_DEFAULT_NON_DCB_DV 0x7800 /* 30K byte */
#define HCLGE_NON_DCB_ADDITIONAL_BUF 0x1400 /* 5120 byte */
-#define HCLGE_TYPE_CRQ 0
-#define HCLGE_TYPE_CSQ 1
-
-/* this bit indicates that the driver is ready for hardware reset */
-#define HCLGE_NIC_SW_RST_RDY_B 16
-#define HCLGE_NIC_SW_RST_RDY BIT(HCLGE_NIC_SW_RST_RDY_B)
-
-#define HCLGE_NIC_CMQ_DESC_NUM 1024
-#define HCLGE_NIC_CMQ_DESC_NUM_S 3
-
#define HCLGE_LED_LOCATE_STATE_S 0
#define HCLGE_LED_LOCATE_STATE_M GENMASK(1, 0)
@@ -1147,16 +771,6 @@ struct hclge_query_ppu_pf_other_int_dfx_cmd {
u8 rsv[4];
};
-#define HCLGE_LINK_EVENT_REPORT_EN_B 0
-#define HCLGE_NCSI_ERROR_REPORT_EN_B 1
-#define HCLGE_PHY_IMP_EN_B 2
-#define HCLGE_MAC_STATS_EXT_EN_B 3
-#define HCLGE_SYNC_RX_RING_HEAD_EN_B 4
-struct hclge_firmware_compat_cmd {
- __le32 compat;
- u8 rsv[20];
-};
-
#define HCLGE_SFP_INFO_CMD_NUM 6
#define HCLGE_SFP_INFO_BD0_LEN 20
#define HCLGE_SFP_INFO_BDX_LEN 24
@@ -1239,44 +853,10 @@ struct hclge_phy_reg_cmd {
u8 rsv1[18];
};
-/* capabilities bits map between imp firmware and local driver */
-struct hclge_caps_bit_map {
- u16 imp_bit;
- u16 local_bit;
-};
-
-int hclge_cmd_init(struct hclge_dev *hdev);
-static inline void hclge_write_reg(void __iomem *base, u32 reg, u32 value)
-{
- writel(value, base + reg);
-}
-
-#define hclge_write_dev(a, reg, value) \
- hclge_write_reg((a)->io_base, reg, value)
-#define hclge_read_dev(a, reg) \
- hclge_read_reg((a)->io_base, reg)
-
-static inline u32 hclge_read_reg(u8 __iomem *base, u32 reg)
-{
- u8 __iomem *reg_addr = READ_ONCE(base);
-
- return readl(reg_addr + reg);
-}
-
-#define HCLGE_SEND_SYNC(flag) \
- ((flag) & HCLGE_CMD_FLAG_NO_INTR)
-
struct hclge_hw;
int hclge_cmd_send(struct hclge_hw *hw, struct hclge_desc *desc, int num);
-void hclge_cmd_setup_basic_desc(struct hclge_desc *desc,
- enum hclge_opcode_type opcode, bool is_read);
-void hclge_cmd_reuse_desc(struct hclge_desc *desc, bool is_read);
-
-enum hclge_cmd_status hclge_cmd_mdio_write(struct hclge_hw *hw,
- struct hclge_desc *desc);
-enum hclge_cmd_status hclge_cmd_mdio_read(struct hclge_hw *hw,
- struct hclge_desc *desc);
-
-void hclge_cmd_uninit(struct hclge_dev *hdev);
-int hclge_cmd_queue_init(struct hclge_dev *hdev);
+enum hclge_comm_cmd_status hclge_cmd_mdio_write(struct hclge_hw *hw,
+ struct hclge_desc *desc);
+enum hclge_comm_cmd_status hclge_cmd_mdio_read(struct hclge_hw *hw,
+ struct hclge_desc *desc);
#endif
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_dcb.c b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_dcb.c
index 375ebf1..69b8673 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_dcb.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_dcb.c
@@ -203,7 +203,7 @@ static int hclge_map_update(struct hclge_dev *hdev)
if (ret)
return ret;
- hclge_rss_indir_init_cfg(hdev);
+ hclge_comm_rss_indir_init_cfg(hdev->ae_dev, &hdev->rss_cfg);
return hclge_rss_init_hw(hdev);
}
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_debugfs.c b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_debugfs.c
index 4e0a8c2..9b870e7 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_debugfs.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_debugfs.c
@@ -77,6 +77,10 @@ static const struct hclge_dbg_reg_type_info hclge_dbg_reg_info[] = {
.cmd = HCLGE_OPC_DFX_TQP_REG } },
};
+/* make sure: len(name) + interval >= maxlen(item data) + 2,
+ * for example, name = "pkt_num"(len: 7), the prototype of item data is u32,
+ * and print as "%u"(maxlen: 10), so the interval should be at least 5.
+ */
static void hclge_dbg_fill_content(char *content, u16 len,
const struct hclge_dbg_item *items,
const char **result, u16 size)
@@ -99,7 +103,7 @@ static void hclge_dbg_fill_content(char *content, u16 len,
static char *hclge_dbg_get_func_id_str(char *buf, u8 id)
{
if (id)
- sprintf(buf, "vf%u", id - 1);
+ sprintf(buf, "vf%u", id - 1U);
else
sprintf(buf, "pf");
@@ -146,7 +150,7 @@ static int hclge_dbg_cmd_send(struct hclge_dev *hdev,
desc->data[0] = cpu_to_le32(index);
for (i = 1; i < bd_num; i++) {
- desc->flag |= cpu_to_le16(HCLGE_CMD_FLAG_NEXT);
+ desc->flag |= cpu_to_le16(HCLGE_COMM_CMD_FLAG_NEXT);
desc++;
hclge_cmd_setup_basic_desc(desc, cmd, true);
}
@@ -258,12 +262,29 @@ hclge_dbg_dump_reg_common(struct hclge_dev *hdev,
return 0;
}
+static const struct hclge_dbg_status_dfx_info hclge_dbg_mac_en_status[] = {
+ {HCLGE_MAC_TX_EN_B, "mac_trans_en"},
+ {HCLGE_MAC_RX_EN_B, "mac_rcv_en"},
+ {HCLGE_MAC_PAD_TX_B, "pad_trans_en"},
+ {HCLGE_MAC_PAD_RX_B, "pad_rcv_en"},
+ {HCLGE_MAC_1588_TX_B, "1588_trans_en"},
+ {HCLGE_MAC_1588_RX_B, "1588_rcv_en"},
+ {HCLGE_MAC_APP_LP_B, "mac_app_loop_en"},
+ {HCLGE_MAC_LINE_LP_B, "mac_line_loop_en"},
+ {HCLGE_MAC_FCS_TX_B, "mac_fcs_tx_en"},
+ {HCLGE_MAC_RX_OVERSIZE_TRUNCATE_B, "mac_rx_oversize_truncate_en"},
+ {HCLGE_MAC_RX_FCS_STRIP_B, "mac_rx_fcs_strip_en"},
+ {HCLGE_MAC_RX_FCS_B, "mac_rx_fcs_en"},
+ {HCLGE_MAC_TX_UNDER_MIN_ERR_B, "mac_tx_under_min_err_en"},
+ {HCLGE_MAC_TX_OVERSIZE_TRUNCATE_B, "mac_tx_oversize_truncate_en"}
+};
+
static int hclge_dbg_dump_mac_enable_status(struct hclge_dev *hdev, char *buf,
int len, int *pos)
{
struct hclge_config_mac_mode_cmd *req;
struct hclge_desc desc;
- u32 loop_en;
+ u32 loop_en, i, offset;
int ret;
hclge_cmd_setup_basic_desc(&desc, HCLGE_OPC_CONFIG_MAC_MODE, true);
@@ -278,39 +299,12 @@ static int hclge_dbg_dump_mac_enable_status(struct hclge_dev *hdev, char *buf,
req = (struct hclge_config_mac_mode_cmd *)desc.data;
loop_en = le32_to_cpu(req->txrx_pad_fcs_loop_en);
- *pos += scnprintf(buf + *pos, len - *pos, "mac_trans_en: %#x\n",
- hnae3_get_bit(loop_en, HCLGE_MAC_TX_EN_B));
- *pos += scnprintf(buf + *pos, len - *pos, "mac_rcv_en: %#x\n",
- hnae3_get_bit(loop_en, HCLGE_MAC_RX_EN_B));
- *pos += scnprintf(buf + *pos, len - *pos, "pad_trans_en: %#x\n",
- hnae3_get_bit(loop_en, HCLGE_MAC_PAD_TX_B));
- *pos += scnprintf(buf + *pos, len - *pos, "pad_rcv_en: %#x\n",
- hnae3_get_bit(loop_en, HCLGE_MAC_PAD_RX_B));
- *pos += scnprintf(buf + *pos, len - *pos, "1588_trans_en: %#x\n",
- hnae3_get_bit(loop_en, HCLGE_MAC_1588_TX_B));
- *pos += scnprintf(buf + *pos, len - *pos, "1588_rcv_en: %#x\n",
- hnae3_get_bit(loop_en, HCLGE_MAC_1588_RX_B));
- *pos += scnprintf(buf + *pos, len - *pos, "mac_app_loop_en: %#x\n",
- hnae3_get_bit(loop_en, HCLGE_MAC_APP_LP_B));
- *pos += scnprintf(buf + *pos, len - *pos, "mac_line_loop_en: %#x\n",
- hnae3_get_bit(loop_en, HCLGE_MAC_LINE_LP_B));
- *pos += scnprintf(buf + *pos, len - *pos, "mac_fcs_tx_en: %#x\n",
- hnae3_get_bit(loop_en, HCLGE_MAC_FCS_TX_B));
- *pos += scnprintf(buf + *pos, len - *pos,
- "mac_rx_oversize_truncate_en: %#x\n",
- hnae3_get_bit(loop_en,
- HCLGE_MAC_RX_OVERSIZE_TRUNCATE_B));
- *pos += scnprintf(buf + *pos, len - *pos, "mac_rx_fcs_strip_en: %#x\n",
- hnae3_get_bit(loop_en, HCLGE_MAC_RX_FCS_STRIP_B));
- *pos += scnprintf(buf + *pos, len - *pos, "mac_rx_fcs_en: %#x\n",
- hnae3_get_bit(loop_en, HCLGE_MAC_RX_FCS_B));
- *pos += scnprintf(buf + *pos, len - *pos,
- "mac_tx_under_min_err_en: %#x\n",
- hnae3_get_bit(loop_en, HCLGE_MAC_TX_UNDER_MIN_ERR_B));
- *pos += scnprintf(buf + *pos, len - *pos,
- "mac_tx_oversize_truncate_en: %#x\n",
- hnae3_get_bit(loop_en,
- HCLGE_MAC_TX_OVERSIZE_TRUNCATE_B));
+ for (i = 0; i < ARRAY_SIZE(hclge_dbg_mac_en_status); i++) {
+ offset = hclge_dbg_mac_en_status[i].offset;
+ *pos += scnprintf(buf + *pos, len - *pos, "%s: %#x\n",
+ hclge_dbg_mac_en_status[i].message,
+ hnae3_get_bit(loop_en, offset));
+ }
return 0;
}
@@ -788,7 +782,6 @@ static int hclge_dbg_dump_tm_pg(struct hclge_dev *hdev, char *buf, int len)
data_str = kcalloc(ARRAY_SIZE(tm_pg_items),
HCLGE_DBG_DATA_STR_LEN, GFP_KERNEL);
-
if (!data_str)
return -ENOMEM;
@@ -1273,7 +1266,7 @@ static int hclge_dbg_dump_rx_priv_wl_buf_cfg(struct hclge_dev *hdev, char *buf,
int i, ret;
hclge_cmd_setup_basic_desc(&desc[0], HCLGE_OPC_RX_PRIV_WL_ALLOC, true);
- desc[0].flag |= cpu_to_le16(HCLGE_CMD_FLAG_NEXT);
+ desc[0].flag |= cpu_to_le16(HCLGE_COMM_CMD_FLAG_NEXT);
hclge_cmd_setup_basic_desc(&desc[1], HCLGE_OPC_RX_PRIV_WL_ALLOC, true);
ret = hclge_cmd_send(&hdev->hw, desc, 2);
if (ret) {
@@ -1309,7 +1302,7 @@ static int hclge_dbg_dump_rx_common_threshold_cfg(struct hclge_dev *hdev,
int i, ret;
hclge_cmd_setup_basic_desc(&desc[0], HCLGE_OPC_RX_COM_THRD_ALLOC, true);
- desc[0].flag |= cpu_to_le16(HCLGE_CMD_FLAG_NEXT);
+ desc[0].flag |= cpu_to_le16(HCLGE_COMM_CMD_FLAG_NEXT);
hclge_cmd_setup_basic_desc(&desc[1], HCLGE_OPC_RX_COM_THRD_ALLOC, true);
ret = hclge_cmd_send(&hdev->hw, desc, 2);
if (ret) {
@@ -1454,9 +1447,9 @@ static int hclge_dbg_fd_tcam_read(struct hclge_dev *hdev, bool sel_x,
u32 *req;
hclge_cmd_setup_basic_desc(&desc[0], HCLGE_OPC_FD_TCAM_OP, true);
- desc[0].flag |= cpu_to_le16(HCLGE_CMD_FLAG_NEXT);
+ desc[0].flag |= cpu_to_le16(HCLGE_COMM_CMD_FLAG_NEXT);
hclge_cmd_setup_basic_desc(&desc[1], HCLGE_OPC_FD_TCAM_OP, true);
- desc[1].flag |= cpu_to_le16(HCLGE_CMD_FLAG_NEXT);
+ desc[1].flag |= cpu_to_le16(HCLGE_COMM_CMD_FLAG_NEXT);
hclge_cmd_setup_basic_desc(&desc[2], HCLGE_OPC_FD_TCAM_OP, true);
req1 = (struct hclge_fd_tcam_config_1_cmd *)desc[0].data;
@@ -1614,8 +1607,19 @@ static int hclge_dbg_dump_fd_counter(struct hclge_dev *hdev, char *buf, int len)
return 0;
}
+static const struct hclge_dbg_status_dfx_info hclge_dbg_rst_info[] = {
+ {HCLGE_MISC_VECTOR_REG_BASE, "vector0 interrupt enable status"},
+ {HCLGE_MISC_RESET_STS_REG, "reset interrupt source"},
+ {HCLGE_MISC_VECTOR_INT_STS, "reset interrupt status"},
+ {HCLGE_RAS_PF_OTHER_INT_STS_REG, "RAS interrupt status"},
+ {HCLGE_GLOBAL_RESET_REG, "hardware reset status"},
+ {HCLGE_NIC_CSQ_DEPTH_REG, "handshake status"},
+ {HCLGE_FUN_RST_ING, "function reset status"}
+};
+
int hclge_dbg_dump_rst_info(struct hclge_dev *hdev, char *buf, int len)
{
+ u32 i, offset;
int pos = 0;
pos += scnprintf(buf + pos, len - pos, "PF reset count: %u\n",
@@ -1634,22 +1638,14 @@ int hclge_dbg_dump_rst_info(struct hclge_dev *hdev, char *buf, int len)
hdev->rst_stats.reset_cnt);
pos += scnprintf(buf + pos, len - pos, "reset fail count: %u\n",
hdev->rst_stats.reset_fail_cnt);
- pos += scnprintf(buf + pos, len - pos,
- "vector0 interrupt enable status: 0x%x\n",
- hclge_read_dev(&hdev->hw, HCLGE_MISC_VECTOR_REG_BASE));
- pos += scnprintf(buf + pos, len - pos, "reset interrupt source: 0x%x\n",
- hclge_read_dev(&hdev->hw, HCLGE_MISC_RESET_STS_REG));
- pos += scnprintf(buf + pos, len - pos, "reset interrupt status: 0x%x\n",
- hclge_read_dev(&hdev->hw, HCLGE_MISC_VECTOR_INT_STS));
- pos += scnprintf(buf + pos, len - pos, "RAS interrupt status: 0x%x\n",
- hclge_read_dev(&hdev->hw,
- HCLGE_RAS_PF_OTHER_INT_STS_REG));
- pos += scnprintf(buf + pos, len - pos, "hardware reset status: 0x%x\n",
- hclge_read_dev(&hdev->hw, HCLGE_GLOBAL_RESET_REG));
- pos += scnprintf(buf + pos, len - pos, "handshake status: 0x%x\n",
- hclge_read_dev(&hdev->hw, HCLGE_NIC_CSQ_DEPTH_REG));
- pos += scnprintf(buf + pos, len - pos, "function reset status: 0x%x\n",
- hclge_read_dev(&hdev->hw, HCLGE_FUN_RST_ING));
+
+ for (i = 0; i < ARRAY_SIZE(hclge_dbg_rst_info); i++) {
+ offset = hclge_dbg_rst_info[i].offset;
+ pos += scnprintf(buf + pos, len - pos, "%s: 0x%x\n",
+ hclge_dbg_rst_info[i].message,
+ hclge_read_dev(&hdev->hw, offset));
+ }
+
pos += scnprintf(buf + pos, len - pos, "hdev state: 0x%lx\n",
hdev->state);
@@ -1771,7 +1767,7 @@ hclge_dbg_get_imp_stats_info(struct hclge_dev *hdev, char *buf, int len)
#define HCLGE_MAX_NCL_CONFIG_LENGTH 16384
static void hclge_ncl_config_data_print(struct hclge_desc *desc, int *index,
- char *buf, int *len, int *pos)
+ char *buf, int len, int *pos)
{
#define HCLGE_CMD_DATA_NUM 6
@@ -1783,7 +1779,7 @@ static void hclge_ncl_config_data_print(struct hclge_desc *desc, int *index,
if (i == 0 && j == 0)
continue;
- *pos += scnprintf(buf + *pos, *len - *pos,
+ *pos += scnprintf(buf + *pos, len - *pos,
"0x%04x | 0x%08x\n", offset,
le32_to_cpu(desc[i].data[j]));
@@ -1821,7 +1817,7 @@ hclge_dbg_dump_ncl_config(struct hclge_dev *hdev, char *buf, int len)
if (ret)
return ret;
- hclge_ncl_config_data_print(desc, &index, buf, &len, &pos);
+ hclge_ncl_config_data_print(desc, &index, buf, len, &pos);
}
return 0;
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_debugfs.h b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_debugfs.h
index c526591..7240529 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_debugfs.h
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_debugfs.h
@@ -94,6 +94,11 @@ struct hclge_dbg_func {
char *buf, int len);
};
+struct hclge_dbg_status_dfx_info {
+ u32 offset;
+ char message[HCLGE_DBG_MAX_DFX_MSG_LEN];
+};
+
static const struct hclge_dbg_dfx_message hclge_dbg_bios_common_reg[] = {
{false, "Reserved"},
{true, "BP_CPU_STATE"},
@@ -321,10 +326,10 @@ static const struct hclge_dbg_dfx_message hclge_dbg_igu_egu_reg[] = {
{true, "IGU_RX_OUT_UDP0_PKT"},
{true, "IGU_RX_IN_UDP0_PKT"},
- {false, "Reserved"},
- {false, "Reserved"},
- {false, "Reserved"},
- {false, "Reserved"},
+ {true, "IGU_MC_CAR_DROP_PKT_L"},
+ {true, "IGU_MC_CAR_DROP_PKT_H"},
+ {true, "IGU_BC_CAR_DROP_PKT_L"},
+ {true, "IGU_BC_CAR_DROP_PKT_H"},
{false, "Reserved"},
{true, "IGU_RX_OVERSIZE_PKT_L"},
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_err.c b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_err.c
index 20e628c2..42a9e73 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_err.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_err.c
@@ -1399,7 +1399,7 @@ static int hclge_config_common_hw_err_int(struct hclge_dev *hdev, bool en)
/* configure common error interrupts */
hclge_cmd_setup_basic_desc(&desc[0], HCLGE_COMMON_ECC_INT_CFG, false);
- desc[0].flag |= cpu_to_le16(HCLGE_CMD_FLAG_NEXT);
+ desc[0].flag |= cpu_to_le16(HCLGE_COMM_CMD_FLAG_NEXT);
hclge_cmd_setup_basic_desc(&desc[1], HCLGE_COMMON_ECC_INT_CFG, false);
if (en) {
@@ -1498,7 +1498,7 @@ static int hclge_config_ppp_error_interrupt(struct hclge_dev *hdev, u32 cmd,
/* configure PPP error interrupts */
hclge_cmd_setup_basic_desc(&desc[0], cmd, false);
- desc[0].flag |= cpu_to_le16(HCLGE_CMD_FLAG_NEXT);
+ desc[0].flag |= cpu_to_le16(HCLGE_COMM_CMD_FLAG_NEXT);
hclge_cmd_setup_basic_desc(&desc[1], cmd, false);
if (cmd == HCLGE_PPP_CMD0_INT_CMD) {
@@ -1633,7 +1633,7 @@ static int hclge_config_ppu_error_interrupts(struct hclge_dev *hdev, u32 cmd,
/* configure PPU error interrupts */
if (cmd == HCLGE_PPU_MPF_ECC_INT_CMD) {
hclge_cmd_setup_basic_desc(&desc[0], cmd, false);
- desc[0].flag |= cpu_to_le16(HCLGE_CMD_FLAG_NEXT);
+ desc[0].flag |= cpu_to_le16(HCLGE_COMM_CMD_FLAG_NEXT);
hclge_cmd_setup_basic_desc(&desc[1], cmd, false);
if (en) {
desc[0].data[0] =
@@ -1718,7 +1718,7 @@ static int hclge_config_ssu_hw_err_int(struct hclge_dev *hdev, bool en)
/* configure SSU ecc error interrupts */
hclge_cmd_setup_basic_desc(&desc[0], HCLGE_SSU_ECC_INT_CMD, false);
- desc[0].flag |= cpu_to_le16(HCLGE_CMD_FLAG_NEXT);
+ desc[0].flag |= cpu_to_le16(HCLGE_COMM_CMD_FLAG_NEXT);
hclge_cmd_setup_basic_desc(&desc[1], HCLGE_SSU_ECC_INT_CMD, false);
if (en) {
desc[0].data[0] = cpu_to_le32(HCLGE_SSU_1BIT_ECC_ERR_INT_EN);
@@ -1740,7 +1740,7 @@ static int hclge_config_ssu_hw_err_int(struct hclge_dev *hdev, bool en)
/* configure SSU common error interrupts */
hclge_cmd_setup_basic_desc(&desc[0], HCLGE_SSU_COMMON_INT_CMD, false);
- desc[0].flag |= cpu_to_le16(HCLGE_CMD_FLAG_NEXT);
+ desc[0].flag |= cpu_to_le16(HCLGE_COMM_CMD_FLAG_NEXT);
hclge_cmd_setup_basic_desc(&desc[1], HCLGE_SSU_COMMON_INT_CMD, false);
if (en) {
@@ -1963,7 +1963,7 @@ static int hclge_handle_mpf_ras_error(struct hclge_dev *hdev,
&ae_dev->hw_err_reset_req);
/* clear all main PF RAS errors */
- hclge_cmd_reuse_desc(&desc[0], false);
+ hclge_comm_cmd_reuse_desc(&desc[0], false);
ret = hclge_cmd_send(&hdev->hw, &desc[0], num);
if (ret)
dev_err(dev, "clear all mpf ras int cmd failed (%d)\n", ret);
@@ -2036,7 +2036,7 @@ static int hclge_handle_pf_ras_error(struct hclge_dev *hdev,
}
/* clear all PF RAS errors */
- hclge_cmd_reuse_desc(&desc[0], false);
+ hclge_comm_cmd_reuse_desc(&desc[0], false);
ret = hclge_cmd_send(&hdev->hw, &desc[0], num);
if (ret)
dev_err(dev, "clear all pf ras int cmd failed (%d)\n", ret);
@@ -2087,8 +2087,8 @@ static int hclge_log_rocee_axi_error(struct hclge_dev *hdev)
true);
hclge_cmd_setup_basic_desc(&desc[2], HCLGE_QUERY_ROCEE_AXI_RAS_INFO_CMD,
true);
- desc[0].flag |= cpu_to_le16(HCLGE_CMD_FLAG_NEXT);
- desc[1].flag |= cpu_to_le16(HCLGE_CMD_FLAG_NEXT);
+ desc[0].flag |= cpu_to_le16(HCLGE_COMM_CMD_FLAG_NEXT);
+ desc[1].flag |= cpu_to_le16(HCLGE_COMM_CMD_FLAG_NEXT);
ret = hclge_cmd_send(&hdev->hw, &desc[0], 3);
if (ret) {
@@ -2119,7 +2119,7 @@ static int hclge_log_rocee_ecc_error(struct hclge_dev *hdev)
ret = hclge_cmd_query_error(hdev, &desc[0],
HCLGE_QUERY_ROCEE_ECC_RAS_INFO_CMD,
- HCLGE_CMD_FLAG_NEXT);
+ HCLGE_COMM_CMD_FLAG_NEXT);
if (ret) {
dev_err(dev, "failed(%d) to query ROCEE ECC error sts\n", ret);
return ret;
@@ -2235,7 +2235,7 @@ hclge_log_and_clear_rocee_ras_error(struct hclge_dev *hdev)
}
/* clear error status */
- hclge_cmd_reuse_desc(&desc[0], false);
+ hclge_comm_cmd_reuse_desc(&desc[0], false);
ret = hclge_cmd_send(&hdev->hw, &desc[0], 1);
if (ret) {
dev_err(dev, "failed(%d) to clear ROCEE RAS error\n", ret);
@@ -2405,7 +2405,8 @@ static int hclge_clear_hw_msix_error(struct hclge_dev *hdev,
else
desc[0].opcode = cpu_to_le16(HCLGE_QUERY_CLEAR_ALL_PF_MSIX_INT);
- desc[0].flag = cpu_to_le16(HCLGE_CMD_FLAG_NO_INTR | HCLGE_CMD_FLAG_IN);
+ desc[0].flag = cpu_to_le16(HCLGE_COMM_CMD_FLAG_NO_INTR |
+ HCLGE_COMM_CMD_FLAG_IN);
return hclge_cmd_send(&hdev->hw, &desc[0], bd_num);
}
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.c b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.c
index c2a5810..24f7afa 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.c
@@ -24,6 +24,7 @@
#include "hclge_err.h"
#include "hnae3.h"
#include "hclge_devlink.h"
+#include "hclge_comm_cmd.h"
#define HCLGE_NAME "hclge"
@@ -90,20 +91,20 @@ static const struct pci_device_id ae_algo_pci_tbl[] = {
MODULE_DEVICE_TABLE(pci, ae_algo_pci_tbl);
-static const u32 cmdq_reg_addr_list[] = {HCLGE_NIC_CSQ_BASEADDR_L_REG,
- HCLGE_NIC_CSQ_BASEADDR_H_REG,
- HCLGE_NIC_CSQ_DEPTH_REG,
- HCLGE_NIC_CSQ_TAIL_REG,
- HCLGE_NIC_CSQ_HEAD_REG,
- HCLGE_NIC_CRQ_BASEADDR_L_REG,
- HCLGE_NIC_CRQ_BASEADDR_H_REG,
- HCLGE_NIC_CRQ_DEPTH_REG,
- HCLGE_NIC_CRQ_TAIL_REG,
- HCLGE_NIC_CRQ_HEAD_REG,
- HCLGE_VECTOR0_CMDQ_SRC_REG,
- HCLGE_CMDQ_INTR_STS_REG,
- HCLGE_CMDQ_INTR_EN_REG,
- HCLGE_CMDQ_INTR_GEN_REG};
+static const u32 cmdq_reg_addr_list[] = {HCLGE_COMM_NIC_CSQ_BASEADDR_L_REG,
+ HCLGE_COMM_NIC_CSQ_BASEADDR_H_REG,
+ HCLGE_COMM_NIC_CSQ_DEPTH_REG,
+ HCLGE_COMM_NIC_CSQ_TAIL_REG,
+ HCLGE_COMM_NIC_CSQ_HEAD_REG,
+ HCLGE_COMM_NIC_CRQ_BASEADDR_L_REG,
+ HCLGE_COMM_NIC_CRQ_BASEADDR_H_REG,
+ HCLGE_COMM_NIC_CRQ_DEPTH_REG,
+ HCLGE_COMM_NIC_CRQ_TAIL_REG,
+ HCLGE_COMM_NIC_CRQ_HEAD_REG,
+ HCLGE_COMM_VECTOR0_CMDQ_SRC_REG,
+ HCLGE_COMM_CMDQ_INTR_STS_REG,
+ HCLGE_COMM_CMDQ_INTR_EN_REG,
+ HCLGE_COMM_CMDQ_INTR_GEN_REG};
static const u32 common_reg_addr_list[] = {HCLGE_MISC_VECTOR_REG_BASE,
HCLGE_PF_OTHER_INT_REG,
@@ -370,14 +371,6 @@ static const struct hclge_mac_mgr_tbl_entry_cmd hclge_mgr_table[] = {
},
};
-static const u8 hclge_hash_key[] = {
- 0x6D, 0x5A, 0x56, 0xDA, 0x25, 0x5B, 0x0E, 0xC2,
- 0x41, 0x67, 0x25, 0x3D, 0x43, 0xA3, 0x8F, 0xB0,
- 0xD0, 0xCA, 0x2B, 0xCB, 0xAE, 0x7B, 0x30, 0xB4,
- 0x77, 0xCB, 0x2D, 0xA3, 0x80, 0x30, 0xF2, 0x0C,
- 0x6A, 0x42, 0xB7, 0x3B, 0xBE, 0xAC, 0x01, 0xFA
-};
-
static const u32 hclge_dfx_bd_offset_list[] = {
HCLGE_DFX_BIOS_BD_OFFSET,
HCLGE_DFX_SSU_0_BD_OFFSET,
@@ -478,6 +471,20 @@ static const struct key_info tuple_key_info[] = {
offsetof(struct hclge_fd_rule, tuples_mask.l4_user_def) },
};
+/**
+ * hclge_cmd_send - send command to command queue
+ * @hw: pointer to the hw struct
+ * @desc: prefilled descriptor for describing the command
+ * @num : the number of descriptors to be sent
+ *
+ * This is the main send command for command queue, it
+ * sends the queue, cleans the queue, etc
+ **/
+int hclge_cmd_send(struct hclge_hw *hw, struct hclge_desc *desc, int num)
+{
+ return hclge_comm_cmd_send(&hw->hw, desc, num);
+}
+
static int hclge_mac_update_stats_defective(struct hclge_dev *hdev)
{
#define HCLGE_MAC_CMD_NUM 21
@@ -604,111 +611,6 @@ int hclge_mac_update_stats(struct hclge_dev *hdev)
return hclge_mac_update_stats_defective(hdev);
}
-static int hclge_tqps_update_stats(struct hnae3_handle *handle)
-{
- struct hnae3_knic_private_info *kinfo = &handle->kinfo;
- struct hclge_vport *vport = hclge_get_vport(handle);
- struct hclge_dev *hdev = vport->back;
- struct hnae3_queue *queue;
- struct hclge_desc desc[1];
- struct hclge_tqp *tqp;
- int ret, i;
-
- for (i = 0; i < kinfo->num_tqps; i++) {
- queue = handle->kinfo.tqp[i];
- tqp = container_of(queue, struct hclge_tqp, q);
- /* command : HCLGE_OPC_QUERY_IGU_STAT */
- hclge_cmd_setup_basic_desc(&desc[0], HCLGE_OPC_QUERY_RX_STATS,
- true);
-
- desc[0].data[0] = cpu_to_le32(tqp->index);
- ret = hclge_cmd_send(&hdev->hw, desc, 1);
- if (ret) {
- dev_err(&hdev->pdev->dev,
- "Query tqp stat fail, status = %d,queue = %d\n",
- ret, i);
- return ret;
- }
- tqp->tqp_stats.rcb_rx_ring_pktnum_rcd +=
- le32_to_cpu(desc[0].data[1]);
- }
-
- for (i = 0; i < kinfo->num_tqps; i++) {
- queue = handle->kinfo.tqp[i];
- tqp = container_of(queue, struct hclge_tqp, q);
- /* command : HCLGE_OPC_QUERY_IGU_STAT */
- hclge_cmd_setup_basic_desc(&desc[0],
- HCLGE_OPC_QUERY_TX_STATS,
- true);
-
- desc[0].data[0] = cpu_to_le32(tqp->index);
- ret = hclge_cmd_send(&hdev->hw, desc, 1);
- if (ret) {
- dev_err(&hdev->pdev->dev,
- "Query tqp stat fail, status = %d,queue = %d\n",
- ret, i);
- return ret;
- }
- tqp->tqp_stats.rcb_tx_ring_pktnum_rcd +=
- le32_to_cpu(desc[0].data[1]);
- }
-
- return 0;
-}
-
-static u64 *hclge_tqps_get_stats(struct hnae3_handle *handle, u64 *data)
-{
- struct hnae3_knic_private_info *kinfo = &handle->kinfo;
- struct hclge_tqp *tqp;
- u64 *buff = data;
- int i;
-
- for (i = 0; i < kinfo->num_tqps; i++) {
- tqp = container_of(kinfo->tqp[i], struct hclge_tqp, q);
- *buff++ = tqp->tqp_stats.rcb_tx_ring_pktnum_rcd;
- }
-
- for (i = 0; i < kinfo->num_tqps; i++) {
- tqp = container_of(kinfo->tqp[i], struct hclge_tqp, q);
- *buff++ = tqp->tqp_stats.rcb_rx_ring_pktnum_rcd;
- }
-
- return buff;
-}
-
-static int hclge_tqps_get_sset_count(struct hnae3_handle *handle, int stringset)
-{
- struct hnae3_knic_private_info *kinfo = &handle->kinfo;
-
- /* each tqp has TX & RX two queues */
- return kinfo->num_tqps * (2);
-}
-
-static u8 *hclge_tqps_get_strings(struct hnae3_handle *handle, u8 *data)
-{
- struct hnae3_knic_private_info *kinfo = &handle->kinfo;
- u8 *buff = data;
- int i;
-
- for (i = 0; i < kinfo->num_tqps; i++) {
- struct hclge_tqp *tqp = container_of(handle->kinfo.tqp[i],
- struct hclge_tqp, q);
- snprintf(buff, ETH_GSTRING_LEN, "txq%u_pktnum_rcd",
- tqp->index);
- buff = buff + ETH_GSTRING_LEN;
- }
-
- for (i = 0; i < kinfo->num_tqps; i++) {
- struct hclge_tqp *tqp = container_of(kinfo->tqp[i],
- struct hclge_tqp, q);
- snprintf(buff, ETH_GSTRING_LEN, "rxq%u_pktnum_rcd",
- tqp->index);
- buff = buff + ETH_GSTRING_LEN;
- }
-
- return buff;
-}
-
static int hclge_comm_get_count(struct hclge_dev *hdev,
const struct hclge_comm_stats_str strs[],
u32 size)
@@ -769,7 +671,7 @@ static void hclge_update_stats_for_all(struct hclge_dev *hdev)
handle = &hdev->vport[0].nic;
if (handle->client) {
- status = hclge_tqps_update_stats(handle);
+ status = hclge_comm_tqps_update_stats(handle, &hdev->hw.hw);
if (status) {
dev_err(&hdev->pdev->dev,
"Update TQPS stats fail, status = %d.\n",
@@ -799,7 +701,7 @@ static void hclge_update_stats(struct hnae3_handle *handle,
"Update MAC stats fail, status = %d.\n",
status);
- status = hclge_tqps_update_stats(handle);
+ status = hclge_comm_tqps_update_stats(handle, &hdev->hw.hw);
if (status)
dev_err(&hdev->pdev->dev,
"Update TQPS stats fail, status = %d.\n",
@@ -848,7 +750,7 @@ static int hclge_get_sset_count(struct hnae3_handle *handle, int stringset)
} else if (stringset == ETH_SS_STATS) {
count = hclge_comm_get_count(hdev, g_mac_stats_string,
ARRAY_SIZE(g_mac_stats_string)) +
- hclge_tqps_get_sset_count(handle, stringset);
+ hclge_comm_tqps_get_sset_count(handle);
}
return count;
@@ -866,7 +768,7 @@ static void hclge_get_strings(struct hnae3_handle *handle, u32 stringset,
size = ARRAY_SIZE(g_mac_stats_string);
p = hclge_comm_get_strings(hdev, stringset, g_mac_stats_string,
size, p);
- p = hclge_tqps_get_strings(handle, p);
+ p = hclge_comm_tqps_get_strings(handle, p);
} else if (stringset == ETH_SS_TEST) {
if (handle->flags & HNAE3_SUPPORT_APP_LOOPBACK) {
memcpy(p, hns3_nic_test_strs[HNAE3_LOOP_APP],
@@ -900,7 +802,7 @@ static void hclge_get_stats(struct hnae3_handle *handle, u64 *data)
p = hclge_comm_get_stats(hdev, g_mac_stats_string,
ARRAY_SIZE(g_mac_stats_string), data);
- p = hclge_tqps_get_stats(handle, p);
+ p = hclge_comm_tqps_get_stats(handle, p);
}
static void hclge_get_mac_stat(struct hnae3_handle *handle,
@@ -1480,7 +1382,7 @@ static void hclge_set_default_dev_specs(struct hclge_dev *hdev)
ae_dev->dev_specs.max_non_tso_bd_num = HCLGE_MAX_NON_TSO_BD_NUM;
ae_dev->dev_specs.rss_ind_tbl_size = HCLGE_RSS_IND_TBL_SIZE;
- ae_dev->dev_specs.rss_key_size = HCLGE_RSS_KEY_SIZE;
+ ae_dev->dev_specs.rss_key_size = HCLGE_COMM_RSS_KEY_SIZE;
ae_dev->dev_specs.max_tm_rate = HCLGE_ETHER_MAX_RATE;
ae_dev->dev_specs.max_int_gl = HCLGE_DEF_MAX_INT_GL;
ae_dev->dev_specs.max_frm_size = HCLGE_MAC_MAX_FRAME;
@@ -1520,7 +1422,7 @@ static void hclge_check_dev_specs(struct hclge_dev *hdev)
if (!dev_specs->rss_ind_tbl_size)
dev_specs->rss_ind_tbl_size = HCLGE_RSS_IND_TBL_SIZE;
if (!dev_specs->rss_key_size)
- dev_specs->rss_key_size = HCLGE_RSS_KEY_SIZE;
+ dev_specs->rss_key_size = HCLGE_COMM_RSS_KEY_SIZE;
if (!dev_specs->max_tm_rate)
dev_specs->max_tm_rate = HCLGE_ETHER_MAX_RATE;
if (!dev_specs->max_qset_num)
@@ -1567,7 +1469,7 @@ static int hclge_query_dev_specs(struct hclge_dev *hdev)
for (i = 0; i < HCLGE_QUERY_DEV_SPECS_BD_NUM - 1; i++) {
hclge_cmd_setup_basic_desc(&desc[i], HCLGE_OPC_QUERY_DEV_SPECS,
true);
- desc[i].flag |= cpu_to_le16(HCLGE_CMD_FLAG_NEXT);
+ desc[i].flag |= cpu_to_le16(HCLGE_COMM_CMD_FLAG_NEXT);
}
hclge_cmd_setup_basic_desc(&desc[i], HCLGE_OPC_QUERY_DEV_SPECS, true);
@@ -1613,12 +1515,39 @@ static void hclge_init_kdump_kernel_config(struct hclge_dev *hdev)
hdev->num_rx_desc = HCLGE_MIN_RX_DESC;
}
+static void hclge_init_tc_config(struct hclge_dev *hdev)
+{
+ unsigned int i;
+
+ if (hdev->tc_max > HNAE3_MAX_TC ||
+ hdev->tc_max < 1) {
+ dev_warn(&hdev->pdev->dev, "TC num = %u.\n",
+ hdev->tc_max);
+ hdev->tc_max = 1;
+ }
+
+ /* Dev does not support DCB */
+ if (!hnae3_dev_dcb_supported(hdev)) {
+ hdev->tc_max = 1;
+ hdev->pfc_max = 0;
+ } else {
+ hdev->pfc_max = hdev->tc_max;
+ }
+
+ hdev->tm_info.num_tc = 1;
+
+ /* Currently not support uncontiuous tc */
+ for (i = 0; i < hdev->tm_info.num_tc; i++)
+ hnae3_set_bit(hdev->hw_tc_map, i, 1);
+
+ hdev->tx_sch_mode = HCLGE_FLAG_TC_BASE_SCH_MODE;
+}
+
static int hclge_configure(struct hclge_dev *hdev)
{
struct hnae3_ae_dev *ae_dev = pci_get_drvdata(hdev->pdev);
const struct cpumask *cpumask = cpu_online_mask;
struct hclge_cfg cfg;
- unsigned int i;
int node, ret;
ret = hclge_get_cfg(hdev, &cfg);
@@ -1662,29 +1591,7 @@ static int hclge_configure(struct hclge_dev *hdev)
hdev->hw.mac.max_speed = hclge_get_max_speed(cfg.speed_ability);
- if ((hdev->tc_max > HNAE3_MAX_TC) ||
- (hdev->tc_max < 1)) {
- dev_warn(&hdev->pdev->dev, "TC num = %u.\n",
- hdev->tc_max);
- hdev->tc_max = 1;
- }
-
- /* Dev does not support DCB */
- if (!hnae3_dev_dcb_supported(hdev)) {
- hdev->tc_max = 1;
- hdev->pfc_max = 0;
- } else {
- hdev->pfc_max = hdev->tc_max;
- }
-
- hdev->tm_info.num_tc = 1;
-
- /* Currently not support uncontiuous tc */
- for (i = 0; i < hdev->tm_info.num_tc; i++)
- hnae3_set_bit(hdev->hw_tc_map, i, 1);
-
- hdev->tx_sch_mode = HCLGE_FLAG_TC_BASE_SCH_MODE;
-
+ hclge_init_tc_config(hdev);
hclge_init_kdump_kernel_config(hdev);
/* Set the affinity based on numa node */
@@ -1736,11 +1643,11 @@ static int hclge_config_gro(struct hclge_dev *hdev)
static int hclge_alloc_tqps(struct hclge_dev *hdev)
{
- struct hclge_tqp *tqp;
+ struct hclge_comm_tqp *tqp;
int i;
hdev->htqp = devm_kcalloc(&hdev->pdev->dev, hdev->num_tqps,
- sizeof(struct hclge_tqp), GFP_KERNEL);
+ sizeof(struct hclge_comm_tqp), GFP_KERNEL);
if (!hdev->htqp)
return -ENOMEM;
@@ -1759,11 +1666,11 @@ static int hclge_alloc_tqps(struct hclge_dev *hdev)
* HCLGE_TQP_MAX_SIZE_DEV_V2
*/
if (i < HCLGE_TQP_MAX_SIZE_DEV_V2)
- tqp->q.io_base = hdev->hw.io_base +
+ tqp->q.io_base = hdev->hw.hw.io_base +
HCLGE_TQP_REG_OFFSET +
i * HCLGE_TQP_REG_SIZE;
else
- tqp->q.io_base = hdev->hw.io_base +
+ tqp->q.io_base = hdev->hw.hw.io_base +
HCLGE_TQP_REG_OFFSET +
HCLGE_TQP_EXT_REG_OFFSET +
(i - HCLGE_TQP_MAX_SIZE_DEV_V2) *
@@ -1864,8 +1771,8 @@ static int hclge_map_tqp_to_vport(struct hclge_dev *hdev,
kinfo = &nic->kinfo;
for (i = 0; i < vport->alloc_tqps; i++) {
- struct hclge_tqp *q =
- container_of(kinfo->tqp[i], struct hclge_tqp, q);
+ struct hclge_comm_tqp *q =
+ container_of(kinfo->tqp[i], struct hclge_comm_tqp, q);
bool is_pf;
int ret;
@@ -1885,7 +1792,7 @@ static int hclge_map_tqp(struct hclge_dev *hdev)
u16 i, num_vport;
num_vport = hdev->num_req_vfs + 1;
- for (i = 0; i < num_vport; i++) {
+ for (i = 0; i < num_vport; i++) {
int ret;
ret = hclge_map_tqp_to_vport(hdev, vport);
@@ -1907,7 +1814,7 @@ static int hclge_vport_setup(struct hclge_vport *vport, u16 num_tqps)
nic->pdev = hdev->pdev;
nic->ae_algo = &ae_algo;
nic->numa_node_mask = hdev->numa_node_mask;
- nic->kinfo.io_base = hdev->hw.io_base;
+ nic->kinfo.io_base = hdev->hw.hw.io_base;
ret = hclge_knic_setup(vport, num_tqps,
hdev->num_tx_desc, hdev->num_rx_desc);
@@ -2416,9 +2323,9 @@ static int hclge_rx_priv_wl_config(struct hclge_dev *hdev,
/* The first descriptor set the NEXT bit to 1 */
if (i == 0)
- desc[i].flag |= cpu_to_le16(HCLGE_CMD_FLAG_NEXT);
+ desc[i].flag |= cpu_to_le16(HCLGE_COMM_CMD_FLAG_NEXT);
else
- desc[i].flag &= ~cpu_to_le16(HCLGE_CMD_FLAG_NEXT);
+ desc[i].flag &= ~cpu_to_le16(HCLGE_COMM_CMD_FLAG_NEXT);
for (j = 0; j < HCLGE_TC_NUM_ONE_DESC; j++) {
u32 idx = i * HCLGE_TC_NUM_ONE_DESC + j;
@@ -2461,9 +2368,9 @@ static int hclge_common_thrd_config(struct hclge_dev *hdev,
/* The first descriptor set the NEXT bit to 1 */
if (i == 0)
- desc[i].flag |= cpu_to_le16(HCLGE_CMD_FLAG_NEXT);
+ desc[i].flag |= cpu_to_le16(HCLGE_COMM_CMD_FLAG_NEXT);
else
- desc[i].flag &= ~cpu_to_le16(HCLGE_CMD_FLAG_NEXT);
+ desc[i].flag &= ~cpu_to_le16(HCLGE_COMM_CMD_FLAG_NEXT);
for (j = 0; j < HCLGE_TC_NUM_ONE_DESC; j++) {
tc = &s_buf->tc_thrd[i * HCLGE_TC_NUM_ONE_DESC + j];
@@ -2592,8 +2499,8 @@ static int hclge_init_roce_base_info(struct hclge_vport *vport)
roce->rinfo.base_vector = hdev->num_nic_msi;
roce->rinfo.netdev = nic->kinfo.netdev;
- roce->rinfo.roce_io_base = hdev->hw.io_base;
- roce->rinfo.roce_mem_base = hdev->hw.mem_base;
+ roce->rinfo.roce_io_base = hdev->hw.hw.io_base;
+ roce->rinfo.roce_mem_base = hdev->hw.hw.mem_base;
roce->pdev = nic->pdev;
roce->ae_algo = nic->ae_algo;
@@ -2653,11 +2560,38 @@ static u8 hclge_check_speed_dup(u8 duplex, int speed)
return duplex;
}
+static struct hclge_mac_speed_map hclge_mac_speed_map_to_fw[] = {
+ {HCLGE_MAC_SPEED_10M, HCLGE_FW_MAC_SPEED_10M},
+ {HCLGE_MAC_SPEED_100M, HCLGE_FW_MAC_SPEED_100M},
+ {HCLGE_MAC_SPEED_1G, HCLGE_FW_MAC_SPEED_1G},
+ {HCLGE_MAC_SPEED_10G, HCLGE_FW_MAC_SPEED_10G},
+ {HCLGE_MAC_SPEED_25G, HCLGE_FW_MAC_SPEED_25G},
+ {HCLGE_MAC_SPEED_40G, HCLGE_FW_MAC_SPEED_40G},
+ {HCLGE_MAC_SPEED_50G, HCLGE_FW_MAC_SPEED_50G},
+ {HCLGE_MAC_SPEED_100G, HCLGE_FW_MAC_SPEED_100G},
+ {HCLGE_MAC_SPEED_200G, HCLGE_FW_MAC_SPEED_200G},
+};
+
+static int hclge_convert_to_fw_speed(u32 speed_drv, u32 *speed_fw)
+{
+ u16 i;
+
+ for (i = 0; i < ARRAY_SIZE(hclge_mac_speed_map_to_fw); i++) {
+ if (hclge_mac_speed_map_to_fw[i].speed_drv == speed_drv) {
+ *speed_fw = hclge_mac_speed_map_to_fw[i].speed_fw;
+ return 0;
+ }
+ }
+
+ return -EINVAL;
+}
+
static int hclge_cfg_mac_speed_dup_hw(struct hclge_dev *hdev, int speed,
u8 duplex)
{
struct hclge_config_mac_speed_dup_cmd *req;
struct hclge_desc desc;
+ u32 speed_fw;
int ret;
req = (struct hclge_config_mac_speed_dup_cmd *)desc.data;
@@ -2667,48 +2601,14 @@ static int hclge_cfg_mac_speed_dup_hw(struct hclge_dev *hdev, int speed,
if (duplex)
hnae3_set_bit(req->speed_dup, HCLGE_CFG_DUPLEX_B, 1);
- switch (speed) {
- case HCLGE_MAC_SPEED_10M:
- hnae3_set_field(req->speed_dup, HCLGE_CFG_SPEED_M,
- HCLGE_CFG_SPEED_S, HCLGE_FW_MAC_SPEED_10M);
- break;
- case HCLGE_MAC_SPEED_100M:
- hnae3_set_field(req->speed_dup, HCLGE_CFG_SPEED_M,
- HCLGE_CFG_SPEED_S, HCLGE_FW_MAC_SPEED_100M);
- break;
- case HCLGE_MAC_SPEED_1G:
- hnae3_set_field(req->speed_dup, HCLGE_CFG_SPEED_M,
- HCLGE_CFG_SPEED_S, HCLGE_FW_MAC_SPEED_1G);
- break;
- case HCLGE_MAC_SPEED_10G:
- hnae3_set_field(req->speed_dup, HCLGE_CFG_SPEED_M,
- HCLGE_CFG_SPEED_S, HCLGE_FW_MAC_SPEED_10G);
- break;
- case HCLGE_MAC_SPEED_25G:
- hnae3_set_field(req->speed_dup, HCLGE_CFG_SPEED_M,
- HCLGE_CFG_SPEED_S, HCLGE_FW_MAC_SPEED_25G);
- break;
- case HCLGE_MAC_SPEED_40G:
- hnae3_set_field(req->speed_dup, HCLGE_CFG_SPEED_M,
- HCLGE_CFG_SPEED_S, HCLGE_FW_MAC_SPEED_40G);
- break;
- case HCLGE_MAC_SPEED_50G:
- hnae3_set_field(req->speed_dup, HCLGE_CFG_SPEED_M,
- HCLGE_CFG_SPEED_S, HCLGE_FW_MAC_SPEED_50G);
- break;
- case HCLGE_MAC_SPEED_100G:
- hnae3_set_field(req->speed_dup, HCLGE_CFG_SPEED_M,
- HCLGE_CFG_SPEED_S, HCLGE_FW_MAC_SPEED_100G);
- break;
- case HCLGE_MAC_SPEED_200G:
- hnae3_set_field(req->speed_dup, HCLGE_CFG_SPEED_M,
- HCLGE_CFG_SPEED_S, HCLGE_FW_MAC_SPEED_200G);
- break;
- default:
+ ret = hclge_convert_to_fw_speed(speed, &speed_fw);
+ if (ret) {
dev_err(&hdev->pdev->dev, "invalid speed (%d)\n", speed);
- return -EINVAL;
+ return ret;
}
+ hnae3_set_field(req->speed_dup, HCLGE_CFG_SPEED_M, HCLGE_CFG_SPEED_S,
+ speed_fw);
hnae3_set_bit(req->mac_change_fec_en, HCLGE_CFG_MAC_SPEED_CHANGE_EN_B,
1);
@@ -2933,16 +2833,20 @@ static int hclge_mac_init(struct hclge_dev *hdev)
static void hclge_mbx_task_schedule(struct hclge_dev *hdev)
{
if (!test_bit(HCLGE_STATE_REMOVING, &hdev->state) &&
- !test_and_set_bit(HCLGE_STATE_MBX_SERVICE_SCHED, &hdev->state))
+ !test_and_set_bit(HCLGE_STATE_MBX_SERVICE_SCHED, &hdev->state)) {
+ hdev->last_mbx_scheduled = jiffies;
mod_delayed_work(hclge_wq, &hdev->service_task, 0);
+ }
}
static void hclge_reset_task_schedule(struct hclge_dev *hdev)
{
if (!test_bit(HCLGE_STATE_REMOVING, &hdev->state) &&
test_bit(HCLGE_STATE_SERVICE_INITED, &hdev->state) &&
- !test_and_set_bit(HCLGE_STATE_RST_SERVICE_SCHED, &hdev->state))
+ !test_and_set_bit(HCLGE_STATE_RST_SERVICE_SCHED, &hdev->state)) {
+ hdev->last_rst_scheduled = jiffies;
mod_delayed_work(hclge_wq, &hdev->service_task, 0);
+ }
}
static void hclge_errhand_task_schedule(struct hclge_dev *hdev)
@@ -3237,7 +3141,7 @@ static int hclge_get_phy_link_ksettings(struct hnae3_handle *handle,
hclge_cmd_setup_basic_desc(&desc[0], HCLGE_OPC_PHY_LINK_KSETTING,
true);
- desc[0].flag |= cpu_to_le16(HCLGE_CMD_FLAG_NEXT);
+ desc[0].flag |= cpu_to_le16(HCLGE_COMM_CMD_FLAG_NEXT);
hclge_cmd_setup_basic_desc(&desc[1], HCLGE_OPC_PHY_LINK_KSETTING,
true);
@@ -3294,7 +3198,7 @@ hclge_set_phy_link_ksettings(struct hnae3_handle *handle,
hclge_cmd_setup_basic_desc(&desc[0], HCLGE_OPC_PHY_LINK_KSETTING,
false);
- desc[0].flag |= cpu_to_le16(HCLGE_CMD_FLAG_NEXT);
+ desc[0].flag |= cpu_to_le16(HCLGE_COMM_CMD_FLAG_NEXT);
hclge_cmd_setup_basic_desc(&desc[1], HCLGE_OPC_PHY_LINK_KSETTING,
false);
@@ -3501,7 +3405,7 @@ static u32 hclge_check_event_cause(struct hclge_dev *hdev, u32 *clearval)
if (BIT(HCLGE_VECTOR0_IMPRESET_INT_B) & msix_src_reg) {
dev_info(&hdev->pdev->dev, "IMP reset interrupt\n");
set_bit(HNAE3_IMP_RESET, &hdev->reset_pending);
- set_bit(HCLGE_STATE_CMD_DISABLE, &hdev->state);
+ set_bit(HCLGE_COMM_STATE_CMD_DISABLE, &hdev->hw.hw.comm_state);
*clearval = BIT(HCLGE_VECTOR0_IMPRESET_INT_B);
hdev->rst_stats.imp_rst_cnt++;
return HCLGE_VECTOR0_EVENT_RST;
@@ -3509,7 +3413,7 @@ static u32 hclge_check_event_cause(struct hclge_dev *hdev, u32 *clearval)
if (BIT(HCLGE_VECTOR0_GLOBALRESET_INT_B) & msix_src_reg) {
dev_info(&hdev->pdev->dev, "global reset interrupt\n");
- set_bit(HCLGE_STATE_CMD_DISABLE, &hdev->state);
+ set_bit(HCLGE_COMM_STATE_CMD_DISABLE, &hdev->hw.hw.comm_state);
set_bit(HNAE3_GLOBAL_RESET, &hdev->reset_pending);
*clearval = BIT(HCLGE_VECTOR0_GLOBALRESET_INT_B);
hdev->rst_stats.global_rst_cnt++;
@@ -3643,7 +3547,7 @@ static void hclge_get_misc_vector(struct hclge_dev *hdev)
vector->vector_irq = pci_irq_vector(hdev->pdev, 0);
- vector->addr = hdev->hw.io_base + HCLGE_MISC_VECTOR_REG_BASE;
+ vector->addr = hdev->hw.hw.io_base + HCLGE_MISC_VECTOR_REG_BASE;
hdev->vector_status[0] = 0;
hdev->num_msi_left -= 1;
@@ -3827,10 +3731,17 @@ static int hclge_set_all_vf_rst(struct hclge_dev *hdev, bool reset)
static void hclge_mailbox_service_task(struct hclge_dev *hdev)
{
if (!test_and_clear_bit(HCLGE_STATE_MBX_SERVICE_SCHED, &hdev->state) ||
- test_bit(HCLGE_STATE_CMD_DISABLE, &hdev->state) ||
+ test_bit(HCLGE_COMM_STATE_CMD_DISABLE, &hdev->hw.hw.comm_state) ||
test_and_set_bit(HCLGE_STATE_MBX_HANDLING, &hdev->state))
return;
+ if (time_is_before_jiffies(hdev->last_mbx_scheduled +
+ HCLGE_MBX_SCHED_TIMEOUT))
+ dev_warn(&hdev->pdev->dev,
+ "mbx service task is scheduled after %ums on cpu%u!\n",
+ jiffies_to_msecs(jiffies - hdev->last_mbx_scheduled),
+ smp_processor_id());
+
hclge_mbx_handler(hdev);
clear_bit(HCLGE_STATE_MBX_HANDLING, &hdev->state);
@@ -3865,7 +3776,7 @@ static void hclge_func_reset_sync_vf(struct hclge_dev *hdev)
return;
}
msleep(HCLGE_PF_RESET_SYNC_TIME);
- hclge_cmd_reuse_desc(&desc, true);
+ hclge_comm_cmd_reuse_desc(&desc, true);
} while (cnt++ < HCLGE_PF_RESET_SYNC_CNT);
dev_warn(&hdev->pdev->dev, "sync with VF timeout!\n");
@@ -4022,13 +3933,13 @@ static void hclge_reset_handshake(struct hclge_dev *hdev, bool enable)
{
u32 reg_val;
- reg_val = hclge_read_dev(&hdev->hw, HCLGE_NIC_CSQ_DEPTH_REG);
+ reg_val = hclge_read_dev(&hdev->hw, HCLGE_COMM_NIC_CSQ_DEPTH_REG);
if (enable)
- reg_val |= HCLGE_NIC_SW_RST_RDY;
+ reg_val |= HCLGE_COMM_NIC_SW_RST_RDY;
else
- reg_val &= ~HCLGE_NIC_SW_RST_RDY;
+ reg_val &= ~HCLGE_COMM_NIC_SW_RST_RDY;
- hclge_write_dev(&hdev->hw, HCLGE_NIC_CSQ_DEPTH_REG, reg_val);
+ hclge_write_dev(&hdev->hw, HCLGE_COMM_NIC_CSQ_DEPTH_REG, reg_val);
}
static int hclge_func_reset_notify_vf(struct hclge_dev *hdev)
@@ -4065,9 +3976,9 @@ static int hclge_reset_prepare_wait(struct hclge_dev *hdev)
/* After performaning pf reset, it is not necessary to do the
* mailbox handling or send any command to firmware, because
* any mailbox handling or command to firmware is only valid
- * after hclge_cmd_init is called.
+ * after hclge_comm_cmd_init is called.
*/
- set_bit(HCLGE_STATE_CMD_DISABLE, &hdev->state);
+ set_bit(HCLGE_COMM_STATE_CMD_DISABLE, &hdev->hw.hw.comm_state);
hdev->rst_stats.pf_rst_cnt++;
break;
case HNAE3_FLR_RESET:
@@ -4480,6 +4391,13 @@ static void hclge_reset_service_task(struct hclge_dev *hdev)
if (!test_and_clear_bit(HCLGE_STATE_RST_SERVICE_SCHED, &hdev->state))
return;
+ if (time_is_before_jiffies(hdev->last_rst_scheduled +
+ HCLGE_RESET_SCHED_TIMEOUT))
+ dev_warn(&hdev->pdev->dev,
+ "reset service task is scheduled after %ums on cpu%u!\n",
+ jiffies_to_msecs(jiffies - hdev->last_rst_scheduled),
+ smp_processor_id());
+
down(&hdev->reset_sem);
set_bit(HCLGE_STATE_RST_HANDLING, &hdev->state);
@@ -4614,11 +4532,11 @@ static void hclge_get_vector_info(struct hclge_dev *hdev, u16 idx,
/* need an extend offset to config vector >= 64 */
if (idx - 1 < HCLGE_PF_MAX_VECTOR_NUM_DEV_V2)
- vector_info->io_addr = hdev->hw.io_base +
+ vector_info->io_addr = hdev->hw.hw.io_base +
HCLGE_VECTOR_REG_BASE +
(idx - 1) * HCLGE_VECTOR_REG_OFFSET;
else
- vector_info->io_addr = hdev->hw.io_base +
+ vector_info->io_addr = hdev->hw.hw.io_base +
HCLGE_VECTOR_EXT_REG_BASE +
(idx - 1) / HCLGE_PF_MAX_VECTOR_NUM_DEV_V2 *
HCLGE_VECTOR_REG_OFFSET_H +
@@ -4688,334 +4606,43 @@ static int hclge_put_vector(struct hnae3_handle *handle, int vector)
return 0;
}
-static u32 hclge_get_rss_key_size(struct hnae3_handle *handle)
-{
- return HCLGE_RSS_KEY_SIZE;
-}
-
-static int hclge_set_rss_algo_key(struct hclge_dev *hdev,
- const u8 hfunc, const u8 *key)
-{
- struct hclge_rss_config_cmd *req;
- unsigned int key_offset = 0;
- struct hclge_desc desc;
- int key_counts;
- int key_size;
- int ret;
-
- key_counts = HCLGE_RSS_KEY_SIZE;
- req = (struct hclge_rss_config_cmd *)desc.data;
-
- while (key_counts) {
- hclge_cmd_setup_basic_desc(&desc, HCLGE_OPC_RSS_GENERIC_CONFIG,
- false);
-
- req->hash_config |= (hfunc & HCLGE_RSS_HASH_ALGO_MASK);
- req->hash_config |= (key_offset << HCLGE_RSS_HASH_KEY_OFFSET_B);
-
- key_size = min(HCLGE_RSS_HASH_KEY_NUM, key_counts);
- memcpy(req->hash_key,
- key + key_offset * HCLGE_RSS_HASH_KEY_NUM, key_size);
-
- key_counts -= key_size;
- key_offset++;
- ret = hclge_cmd_send(&hdev->hw, &desc, 1);
- if (ret) {
- dev_err(&hdev->pdev->dev,
- "Configure RSS config fail, status = %d\n",
- ret);
- return ret;
- }
- }
- return 0;
-}
-
-static int hclge_set_rss_indir_table(struct hclge_dev *hdev, const u16 *indir)
-{
- struct hclge_rss_indirection_table_cmd *req;
- struct hclge_desc desc;
- int rss_cfg_tbl_num;
- u8 rss_msb_oft;
- u8 rss_msb_val;
- int ret;
- u16 qid;
- int i;
- u32 j;
-
- req = (struct hclge_rss_indirection_table_cmd *)desc.data;
- rss_cfg_tbl_num = hdev->ae_dev->dev_specs.rss_ind_tbl_size /
- HCLGE_RSS_CFG_TBL_SIZE;
-
- for (i = 0; i < rss_cfg_tbl_num; i++) {
- hclge_cmd_setup_basic_desc
- (&desc, HCLGE_OPC_RSS_INDIR_TABLE, false);
-
- req->start_table_index =
- cpu_to_le16(i * HCLGE_RSS_CFG_TBL_SIZE);
- req->rss_set_bitmap = cpu_to_le16(HCLGE_RSS_SET_BITMAP_MSK);
- for (j = 0; j < HCLGE_RSS_CFG_TBL_SIZE; j++) {
- qid = indir[i * HCLGE_RSS_CFG_TBL_SIZE + j];
- req->rss_qid_l[j] = qid & 0xff;
- rss_msb_oft =
- j * HCLGE_RSS_CFG_TBL_BW_H / BITS_PER_BYTE;
- rss_msb_val = (qid >> HCLGE_RSS_CFG_TBL_BW_L & 0x1) <<
- (j * HCLGE_RSS_CFG_TBL_BW_H % BITS_PER_BYTE);
- req->rss_qid_h[rss_msb_oft] |= rss_msb_val;
- }
- ret = hclge_cmd_send(&hdev->hw, &desc, 1);
- if (ret) {
- dev_err(&hdev->pdev->dev,
- "Configure rss indir table fail,status = %d\n",
- ret);
- return ret;
- }
- }
- return 0;
-}
-
-static int hclge_set_rss_tc_mode(struct hclge_dev *hdev, u16 *tc_valid,
- u16 *tc_size, u16 *tc_offset)
-{
- struct hclge_rss_tc_mode_cmd *req;
- struct hclge_desc desc;
- int ret;
- int i;
-
- hclge_cmd_setup_basic_desc(&desc, HCLGE_OPC_RSS_TC_MODE, false);
- req = (struct hclge_rss_tc_mode_cmd *)desc.data;
-
- for (i = 0; i < HCLGE_MAX_TC_NUM; i++) {
- u16 mode = 0;
-
- hnae3_set_bit(mode, HCLGE_RSS_TC_VALID_B, (tc_valid[i] & 0x1));
- hnae3_set_field(mode, HCLGE_RSS_TC_SIZE_M,
- HCLGE_RSS_TC_SIZE_S, tc_size[i]);
- hnae3_set_bit(mode, HCLGE_RSS_TC_SIZE_MSB_B,
- tc_size[i] >> HCLGE_RSS_TC_SIZE_MSB_OFFSET & 0x1);
- hnae3_set_field(mode, HCLGE_RSS_TC_OFFSET_M,
- HCLGE_RSS_TC_OFFSET_S, tc_offset[i]);
-
- req->rss_tc_mode[i] = cpu_to_le16(mode);
- }
-
- ret = hclge_cmd_send(&hdev->hw, &desc, 1);
- if (ret)
- dev_err(&hdev->pdev->dev,
- "Configure rss tc mode fail, status = %d\n", ret);
-
- return ret;
-}
-
-static void hclge_get_rss_type(struct hclge_vport *vport)
-{
- if (vport->rss_tuple_sets.ipv4_tcp_en ||
- vport->rss_tuple_sets.ipv4_udp_en ||
- vport->rss_tuple_sets.ipv4_sctp_en ||
- vport->rss_tuple_sets.ipv6_tcp_en ||
- vport->rss_tuple_sets.ipv6_udp_en ||
- vport->rss_tuple_sets.ipv6_sctp_en)
- vport->nic.kinfo.rss_type = PKT_HASH_TYPE_L4;
- else if (vport->rss_tuple_sets.ipv4_fragment_en ||
- vport->rss_tuple_sets.ipv6_fragment_en)
- vport->nic.kinfo.rss_type = PKT_HASH_TYPE_L3;
- else
- vport->nic.kinfo.rss_type = PKT_HASH_TYPE_NONE;
-}
-
-static int hclge_set_rss_input_tuple(struct hclge_dev *hdev)
-{
- struct hclge_rss_input_tuple_cmd *req;
- struct hclge_desc desc;
- int ret;
-
- hclge_cmd_setup_basic_desc(&desc, HCLGE_OPC_RSS_INPUT_TUPLE, false);
-
- req = (struct hclge_rss_input_tuple_cmd *)desc.data;
-
- /* Get the tuple cfg from pf */
- req->ipv4_tcp_en = hdev->vport[0].rss_tuple_sets.ipv4_tcp_en;
- req->ipv4_udp_en = hdev->vport[0].rss_tuple_sets.ipv4_udp_en;
- req->ipv4_sctp_en = hdev->vport[0].rss_tuple_sets.ipv4_sctp_en;
- req->ipv4_fragment_en = hdev->vport[0].rss_tuple_sets.ipv4_fragment_en;
- req->ipv6_tcp_en = hdev->vport[0].rss_tuple_sets.ipv6_tcp_en;
- req->ipv6_udp_en = hdev->vport[0].rss_tuple_sets.ipv6_udp_en;
- req->ipv6_sctp_en = hdev->vport[0].rss_tuple_sets.ipv6_sctp_en;
- req->ipv6_fragment_en = hdev->vport[0].rss_tuple_sets.ipv6_fragment_en;
- hclge_get_rss_type(&hdev->vport[0]);
- ret = hclge_cmd_send(&hdev->hw, &desc, 1);
- if (ret)
- dev_err(&hdev->pdev->dev,
- "Configure rss input fail, status = %d\n", ret);
- return ret;
-}
-
static int hclge_get_rss(struct hnae3_handle *handle, u32 *indir,
u8 *key, u8 *hfunc)
{
struct hnae3_ae_dev *ae_dev = pci_get_drvdata(handle->pdev);
struct hclge_vport *vport = hclge_get_vport(handle);
- int i;
+ struct hclge_comm_rss_cfg *rss_cfg = &vport->back->rss_cfg;
- /* Get hash algorithm */
- if (hfunc) {
- switch (vport->rss_algo) {
- case HCLGE_RSS_HASH_ALGO_TOEPLITZ:
- *hfunc = ETH_RSS_HASH_TOP;
- break;
- case HCLGE_RSS_HASH_ALGO_SIMPLE:
- *hfunc = ETH_RSS_HASH_XOR;
- break;
- default:
- *hfunc = ETH_RSS_HASH_UNKNOWN;
- break;
- }
- }
+ hclge_comm_get_rss_hash_info(rss_cfg, key, hfunc);
- /* Get the RSS Key required by the user */
- if (key)
- memcpy(key, vport->rss_hash_key, HCLGE_RSS_KEY_SIZE);
-
- /* Get indirect table */
- if (indir)
- for (i = 0; i < ae_dev->dev_specs.rss_ind_tbl_size; i++)
- indir[i] = vport->rss_indirection_tbl[i];
+ hclge_comm_get_rss_indir_tbl(rss_cfg, indir,
+ ae_dev->dev_specs.rss_ind_tbl_size);
return 0;
}
-static int hclge_parse_rss_hfunc(struct hclge_vport *vport, const u8 hfunc,
- u8 *hash_algo)
-{
- switch (hfunc) {
- case ETH_RSS_HASH_TOP:
- *hash_algo = HCLGE_RSS_HASH_ALGO_TOEPLITZ;
- return 0;
- case ETH_RSS_HASH_XOR:
- *hash_algo = HCLGE_RSS_HASH_ALGO_SIMPLE;
- return 0;
- case ETH_RSS_HASH_NO_CHANGE:
- *hash_algo = vport->rss_algo;
- return 0;
- default:
- return -EINVAL;
- }
-}
-
static int hclge_set_rss(struct hnae3_handle *handle, const u32 *indir,
const u8 *key, const u8 hfunc)
{
struct hnae3_ae_dev *ae_dev = pci_get_drvdata(handle->pdev);
struct hclge_vport *vport = hclge_get_vport(handle);
struct hclge_dev *hdev = vport->back;
- u8 hash_algo;
+ struct hclge_comm_rss_cfg *rss_cfg = &hdev->rss_cfg;
int ret, i;
- ret = hclge_parse_rss_hfunc(vport, hfunc, &hash_algo);
+ ret = hclge_comm_set_rss_hash_key(rss_cfg, &hdev->hw.hw, key, hfunc);
if (ret) {
dev_err(&hdev->pdev->dev, "invalid hfunc type %u\n", hfunc);
return ret;
}
- /* Set the RSS Hash Key if specififed by the user */
- if (key) {
- ret = hclge_set_rss_algo_key(hdev, hash_algo, key);
- if (ret)
- return ret;
-
- /* Update the shadow RSS key with user specified qids */
- memcpy(vport->rss_hash_key, key, HCLGE_RSS_KEY_SIZE);
- } else {
- ret = hclge_set_rss_algo_key(hdev, hash_algo,
- vport->rss_hash_key);
- if (ret)
- return ret;
- }
- vport->rss_algo = hash_algo;
-
/* Update the shadow RSS table with user specified qids */
for (i = 0; i < ae_dev->dev_specs.rss_ind_tbl_size; i++)
- vport->rss_indirection_tbl[i] = indir[i];
+ rss_cfg->rss_indirection_tbl[i] = indir[i];
/* Update the hardware */
- return hclge_set_rss_indir_table(hdev, vport->rss_indirection_tbl);
-}
-
-static u8 hclge_get_rss_hash_bits(struct ethtool_rxnfc *nfc)
-{
- u8 hash_sets = nfc->data & RXH_L4_B_0_1 ? HCLGE_S_PORT_BIT : 0;
-
- if (nfc->data & RXH_L4_B_2_3)
- hash_sets |= HCLGE_D_PORT_BIT;
- else
- hash_sets &= ~HCLGE_D_PORT_BIT;
-
- if (nfc->data & RXH_IP_SRC)
- hash_sets |= HCLGE_S_IP_BIT;
- else
- hash_sets &= ~HCLGE_S_IP_BIT;
-
- if (nfc->data & RXH_IP_DST)
- hash_sets |= HCLGE_D_IP_BIT;
- else
- hash_sets &= ~HCLGE_D_IP_BIT;
-
- if (nfc->flow_type == SCTP_V4_FLOW || nfc->flow_type == SCTP_V6_FLOW)
- hash_sets |= HCLGE_V_TAG_BIT;
-
- return hash_sets;
-}
-
-static int hclge_init_rss_tuple_cmd(struct hclge_vport *vport,
- struct ethtool_rxnfc *nfc,
- struct hclge_rss_input_tuple_cmd *req)
-{
- struct hclge_dev *hdev = vport->back;
- u8 tuple_sets;
-
- req->ipv4_tcp_en = vport->rss_tuple_sets.ipv4_tcp_en;
- req->ipv4_udp_en = vport->rss_tuple_sets.ipv4_udp_en;
- req->ipv4_sctp_en = vport->rss_tuple_sets.ipv4_sctp_en;
- req->ipv4_fragment_en = vport->rss_tuple_sets.ipv4_fragment_en;
- req->ipv6_tcp_en = vport->rss_tuple_sets.ipv6_tcp_en;
- req->ipv6_udp_en = vport->rss_tuple_sets.ipv6_udp_en;
- req->ipv6_sctp_en = vport->rss_tuple_sets.ipv6_sctp_en;
- req->ipv6_fragment_en = vport->rss_tuple_sets.ipv6_fragment_en;
-
- tuple_sets = hclge_get_rss_hash_bits(nfc);
- switch (nfc->flow_type) {
- case TCP_V4_FLOW:
- req->ipv4_tcp_en = tuple_sets;
- break;
- case TCP_V6_FLOW:
- req->ipv6_tcp_en = tuple_sets;
- break;
- case UDP_V4_FLOW:
- req->ipv4_udp_en = tuple_sets;
- break;
- case UDP_V6_FLOW:
- req->ipv6_udp_en = tuple_sets;
- break;
- case SCTP_V4_FLOW:
- req->ipv4_sctp_en = tuple_sets;
- break;
- case SCTP_V6_FLOW:
- if (hdev->ae_dev->dev_version <= HNAE3_DEVICE_VERSION_V2 &&
- (nfc->data & (RXH_L4_B_0_1 | RXH_L4_B_2_3)))
- return -EINVAL;
-
- req->ipv6_sctp_en = tuple_sets;
- break;
- case IPV4_FLOW:
- req->ipv4_fragment_en = HCLGE_RSS_INPUT_TUPLE_OTHER;
- break;
- case IPV6_FLOW:
- req->ipv6_fragment_en = HCLGE_RSS_INPUT_TUPLE_OTHER;
- break;
- default:
- return -EINVAL;
- }
-
- return 0;
+ return hclge_comm_set_rss_indir_table(ae_dev, &hdev->hw.hw,
+ rss_cfg->rss_indirection_tbl);
}
static int hclge_set_rss_tuple(struct hnae3_handle *handle,
@@ -5023,92 +4650,20 @@ static int hclge_set_rss_tuple(struct hnae3_handle *handle,
{
struct hclge_vport *vport = hclge_get_vport(handle);
struct hclge_dev *hdev = vport->back;
- struct hclge_rss_input_tuple_cmd *req;
- struct hclge_desc desc;
int ret;
- if (nfc->data & ~(RXH_IP_SRC | RXH_IP_DST |
- RXH_L4_B_0_1 | RXH_L4_B_2_3))
- return -EINVAL;
-
- req = (struct hclge_rss_input_tuple_cmd *)desc.data;
- hclge_cmd_setup_basic_desc(&desc, HCLGE_OPC_RSS_INPUT_TUPLE, false);
-
- ret = hclge_init_rss_tuple_cmd(vport, nfc, req);
+ ret = hclge_comm_set_rss_tuple(hdev->ae_dev, &hdev->hw.hw,
+ &hdev->rss_cfg, nfc);
if (ret) {
dev_err(&hdev->pdev->dev,
- "failed to init rss tuple cmd, ret = %d\n", ret);
+ "failed to set rss tuple, ret = %d.\n", ret);
return ret;
}
- ret = hclge_cmd_send(&hdev->hw, &desc, 1);
- if (ret) {
- dev_err(&hdev->pdev->dev,
- "Set rss tuple fail, status = %d\n", ret);
- return ret;
- }
-
- vport->rss_tuple_sets.ipv4_tcp_en = req->ipv4_tcp_en;
- vport->rss_tuple_sets.ipv4_udp_en = req->ipv4_udp_en;
- vport->rss_tuple_sets.ipv4_sctp_en = req->ipv4_sctp_en;
- vport->rss_tuple_sets.ipv4_fragment_en = req->ipv4_fragment_en;
- vport->rss_tuple_sets.ipv6_tcp_en = req->ipv6_tcp_en;
- vport->rss_tuple_sets.ipv6_udp_en = req->ipv6_udp_en;
- vport->rss_tuple_sets.ipv6_sctp_en = req->ipv6_sctp_en;
- vport->rss_tuple_sets.ipv6_fragment_en = req->ipv6_fragment_en;
- hclge_get_rss_type(vport);
+ hclge_comm_get_rss_type(&vport->nic, &hdev->rss_cfg.rss_tuple_sets);
return 0;
}
-static int hclge_get_vport_rss_tuple(struct hclge_vport *vport, int flow_type,
- u8 *tuple_sets)
-{
- switch (flow_type) {
- case TCP_V4_FLOW:
- *tuple_sets = vport->rss_tuple_sets.ipv4_tcp_en;
- break;
- case UDP_V4_FLOW:
- *tuple_sets = vport->rss_tuple_sets.ipv4_udp_en;
- break;
- case TCP_V6_FLOW:
- *tuple_sets = vport->rss_tuple_sets.ipv6_tcp_en;
- break;
- case UDP_V6_FLOW:
- *tuple_sets = vport->rss_tuple_sets.ipv6_udp_en;
- break;
- case SCTP_V4_FLOW:
- *tuple_sets = vport->rss_tuple_sets.ipv4_sctp_en;
- break;
- case SCTP_V6_FLOW:
- *tuple_sets = vport->rss_tuple_sets.ipv6_sctp_en;
- break;
- case IPV4_FLOW:
- case IPV6_FLOW:
- *tuple_sets = HCLGE_S_IP_BIT | HCLGE_D_IP_BIT;
- break;
- default:
- return -EINVAL;
- }
-
- return 0;
-}
-
-static u64 hclge_convert_rss_tuple(u8 tuple_sets)
-{
- u64 tuple_data = 0;
-
- if (tuple_sets & HCLGE_D_PORT_BIT)
- tuple_data |= RXH_L4_B_2_3;
- if (tuple_sets & HCLGE_S_PORT_BIT)
- tuple_data |= RXH_L4_B_0_1;
- if (tuple_sets & HCLGE_D_IP_BIT)
- tuple_data |= RXH_IP_DST;
- if (tuple_sets & HCLGE_S_IP_BIT)
- tuple_data |= RXH_IP_SRC;
-
- return tuple_data;
-}
-
static int hclge_get_rss_tuple(struct hnae3_handle *handle,
struct ethtool_rxnfc *nfc)
{
@@ -5118,11 +4673,12 @@ static int hclge_get_rss_tuple(struct hnae3_handle *handle,
nfc->data = 0;
- ret = hclge_get_vport_rss_tuple(vport, nfc->flow_type, &tuple_sets);
+ ret = hclge_comm_get_rss_tuple(&vport->back->rss_cfg, nfc->flow_type,
+ &tuple_sets);
if (ret || !tuple_sets)
return ret;
- nfc->data = hclge_convert_rss_tuple(tuple_sets);
+ nfc->data = hclge_comm_convert_rss_tuple(tuple_sets);
return 0;
}
@@ -5175,78 +4731,35 @@ static int hclge_init_rss_tc_mode(struct hclge_dev *hdev)
tc_offset[i] = tc_info->tqp_offset[i];
}
- return hclge_set_rss_tc_mode(hdev, tc_valid, tc_size, tc_offset);
+ return hclge_comm_set_rss_tc_mode(&hdev->hw.hw, tc_offset, tc_valid,
+ tc_size);
}
int hclge_rss_init_hw(struct hclge_dev *hdev)
{
- struct hclge_vport *vport = hdev->vport;
- u16 *rss_indir = vport[0].rss_indirection_tbl;
- u8 *key = vport[0].rss_hash_key;
- u8 hfunc = vport[0].rss_algo;
+ u16 *rss_indir = hdev->rss_cfg.rss_indirection_tbl;
+ u8 *key = hdev->rss_cfg.rss_hash_key;
+ u8 hfunc = hdev->rss_cfg.rss_algo;
int ret;
- ret = hclge_set_rss_indir_table(hdev, rss_indir);
+ ret = hclge_comm_set_rss_indir_table(hdev->ae_dev, &hdev->hw.hw,
+ rss_indir);
if (ret)
return ret;
- ret = hclge_set_rss_algo_key(hdev, hfunc, key);
+ ret = hclge_comm_set_rss_algo_key(&hdev->hw.hw, hfunc, key);
if (ret)
return ret;
- ret = hclge_set_rss_input_tuple(hdev);
+ ret = hclge_comm_set_rss_input_tuple(&hdev->vport[0].nic,
+ &hdev->hw.hw, true,
+ &hdev->rss_cfg);
if (ret)
return ret;
return hclge_init_rss_tc_mode(hdev);
}
-void hclge_rss_indir_init_cfg(struct hclge_dev *hdev)
-{
- struct hclge_vport *vport = &hdev->vport[0];
- int i;
-
- for (i = 0; i < hdev->ae_dev->dev_specs.rss_ind_tbl_size; i++)
- vport->rss_indirection_tbl[i] = i % vport->alloc_rss_size;
-}
-
-static int hclge_rss_init_cfg(struct hclge_dev *hdev)
-{
- u16 rss_ind_tbl_size = hdev->ae_dev->dev_specs.rss_ind_tbl_size;
- int rss_algo = HCLGE_RSS_HASH_ALGO_TOEPLITZ;
- struct hclge_vport *vport = &hdev->vport[0];
- u16 *rss_ind_tbl;
-
- if (hdev->ae_dev->dev_version >= HNAE3_DEVICE_VERSION_V2)
- rss_algo = HCLGE_RSS_HASH_ALGO_SIMPLE;
-
- vport->rss_tuple_sets.ipv4_tcp_en = HCLGE_RSS_INPUT_TUPLE_OTHER;
- vport->rss_tuple_sets.ipv4_udp_en = HCLGE_RSS_INPUT_TUPLE_OTHER;
- vport->rss_tuple_sets.ipv4_sctp_en = HCLGE_RSS_INPUT_TUPLE_SCTP;
- vport->rss_tuple_sets.ipv4_fragment_en = HCLGE_RSS_INPUT_TUPLE_OTHER;
- vport->rss_tuple_sets.ipv6_tcp_en = HCLGE_RSS_INPUT_TUPLE_OTHER;
- vport->rss_tuple_sets.ipv6_udp_en = HCLGE_RSS_INPUT_TUPLE_OTHER;
- vport->rss_tuple_sets.ipv6_sctp_en =
- hdev->ae_dev->dev_version <= HNAE3_DEVICE_VERSION_V2 ?
- HCLGE_RSS_INPUT_TUPLE_SCTP_NO_PORT :
- HCLGE_RSS_INPUT_TUPLE_SCTP;
- vport->rss_tuple_sets.ipv6_fragment_en = HCLGE_RSS_INPUT_TUPLE_OTHER;
-
- vport->rss_algo = rss_algo;
-
- rss_ind_tbl = devm_kcalloc(&hdev->pdev->dev, rss_ind_tbl_size,
- sizeof(*rss_ind_tbl), GFP_KERNEL);
- if (!rss_ind_tbl)
- return -ENOMEM;
-
- vport->rss_indirection_tbl = rss_ind_tbl;
- memcpy(vport->rss_hash_key, hclge_hash_key, HCLGE_RSS_KEY_SIZE);
-
- hclge_rss_indir_init_cfg(hdev);
-
- return 0;
-}
-
int hclge_bind_ring_with_vector(struct hclge_vport *vport,
int vector_id, bool en,
struct hnae3_ring_chain_node *ring_chain)
@@ -5256,7 +4769,7 @@ int hclge_bind_ring_with_vector(struct hclge_vport *vport,
struct hclge_desc desc;
struct hclge_ctrl_vector_chain_cmd *req =
(struct hclge_ctrl_vector_chain_cmd *)desc.data;
- enum hclge_cmd_status status;
+ enum hclge_comm_cmd_status status;
enum hclge_opcode_type op;
u16 tqp_type_and_id;
int i;
@@ -5886,9 +5399,9 @@ static int hclge_fd_tcam_config(struct hclge_dev *hdev, u8 stage, bool sel_x,
int ret;
hclge_cmd_setup_basic_desc(&desc[0], HCLGE_OPC_FD_TCAM_OP, false);
- desc[0].flag |= cpu_to_le16(HCLGE_CMD_FLAG_NEXT);
+ desc[0].flag |= cpu_to_le16(HCLGE_COMM_CMD_FLAG_NEXT);
hclge_cmd_setup_basic_desc(&desc[1], HCLGE_OPC_FD_TCAM_OP, false);
- desc[1].flag |= cpu_to_le16(HCLGE_CMD_FLAG_NEXT);
+ desc[1].flag |= cpu_to_le16(HCLGE_COMM_CMD_FLAG_NEXT);
hclge_cmd_setup_basic_desc(&desc[2], HCLGE_OPC_FD_TCAM_OP, false);
req1 = (struct hclge_fd_tcam_config_1_cmd *)desc[0].data;
@@ -6790,7 +6303,7 @@ static int hclge_fd_parse_ring_cookie(struct hclge_dev *hdev, u64 ring_cookie,
if (vf > hdev->num_req_vfs) {
dev_err(&hdev->pdev->dev,
"Error: vf id (%u) should be less than %u\n",
- vf - 1, hdev->num_req_vfs);
+ vf - 1U, hdev->num_req_vfs);
return -EINVAL;
}
@@ -6800,7 +6313,7 @@ static int hclge_fd_parse_ring_cookie(struct hclge_dev *hdev, u64 ring_cookie,
if (ring >= tqps) {
dev_err(&hdev->pdev->dev,
"Error: queue id (%u) > max tqp num (%u)\n",
- ring, tqps - 1);
+ ring, tqps - 1U);
return -EINVAL;
}
@@ -7161,6 +6674,37 @@ static void hclge_fd_get_ext_info(struct ethtool_rx_flow_spec *fs,
}
}
+static struct hclge_fd_rule *hclge_get_fd_rule(struct hclge_dev *hdev,
+ u16 location)
+{
+ struct hclge_fd_rule *rule = NULL;
+ struct hlist_node *node2;
+
+ hlist_for_each_entry_safe(rule, node2, &hdev->fd_rule_list, rule_node) {
+ if (rule->location == location)
+ return rule;
+ else if (rule->location > location)
+ return NULL;
+ }
+
+ return NULL;
+}
+
+static void hclge_fd_get_ring_cookie(struct ethtool_rx_flow_spec *fs,
+ struct hclge_fd_rule *rule)
+{
+ if (rule->action == HCLGE_FD_ACTION_DROP_PACKET) {
+ fs->ring_cookie = RX_CLS_FLOW_DISC;
+ } else {
+ u64 vf_id;
+
+ fs->ring_cookie = rule->queue_id;
+ vf_id = rule->vf_id;
+ vf_id <<= ETHTOOL_RX_FLOW_SPEC_RING_VF_OFF;
+ fs->ring_cookie |= vf_id;
+ }
+}
+
static int hclge_get_fd_rule_info(struct hnae3_handle *handle,
struct ethtool_rxnfc *cmd)
{
@@ -7168,7 +6712,6 @@ static int hclge_get_fd_rule_info(struct hnae3_handle *handle,
struct hclge_fd_rule *rule = NULL;
struct hclge_dev *hdev = vport->back;
struct ethtool_rx_flow_spec *fs;
- struct hlist_node *node2;
if (!hnae3_dev_fd_supported(hdev))
return -EOPNOTSUPP;
@@ -7177,14 +6720,9 @@ static int hclge_get_fd_rule_info(struct hnae3_handle *handle,
spin_lock_bh(&hdev->fd_rule_lock);
- hlist_for_each_entry_safe(rule, node2, &hdev->fd_rule_list, rule_node) {
- if (rule->location >= fs->location)
- break;
- }
-
- if (!rule || fs->location != rule->location) {
+ rule = hclge_get_fd_rule(hdev, fs->location);
+ if (!rule) {
spin_unlock_bh(&hdev->fd_rule_lock);
-
return -ENOENT;
}
@@ -7222,16 +6760,7 @@ static int hclge_get_fd_rule_info(struct hnae3_handle *handle,
hclge_fd_get_ext_info(fs, rule);
- if (rule->action == HCLGE_FD_ACTION_DROP_PACKET) {
- fs->ring_cookie = RX_CLS_FLOW_DISC;
- } else {
- u64 vf_id;
-
- fs->ring_cookie = rule->queue_id;
- vf_id = rule->vf_id;
- vf_id <<= ETHTOOL_RX_FLOW_SPEC_RING_VF_OFF;
- fs->ring_cookie |= vf_id;
- }
+ hclge_fd_get_ring_cookie(fs, rule);
spin_unlock_bh(&hdev->fd_rule_lock);
@@ -7776,7 +7305,7 @@ static bool hclge_get_cmdq_stat(struct hnae3_handle *handle)
struct hclge_vport *vport = hclge_get_vport(handle);
struct hclge_dev *hdev = vport->back;
- return test_bit(HCLGE_STATE_CMD_DISABLE, &hdev->state);
+ return test_bit(HCLGE_COMM_STATE_CMD_DISABLE, &hdev->hw.hw.comm_state);
}
static bool hclge_ae_dev_resetting(struct hnae3_handle *handle)
@@ -7866,7 +7395,7 @@ static int hclge_config_switch_param(struct hclge_dev *hdev, int vfid,
}
/* modify and write new config parameter */
- hclge_cmd_reuse_desc(&desc, false);
+ hclge_comm_cmd_reuse_desc(&desc, false);
req->switch_param = (req->switch_param & param_mask) | switch_param;
req->param_mask = param_mask;
@@ -7960,7 +7489,7 @@ static int hclge_set_app_loopback(struct hclge_dev *hdev, bool en)
/* 3 Config mac work mode with loopback flag
* and its original configure parameters
*/
- hclge_cmd_reuse_desc(&desc, false);
+ hclge_comm_cmd_reuse_desc(&desc, false);
ret = hclge_cmd_send(&hdev->hw, &desc, 1);
if (ret)
dev_err(&hdev->pdev->dev,
@@ -7968,16 +7497,13 @@ static int hclge_set_app_loopback(struct hclge_dev *hdev, bool en)
return ret;
}
-static int hclge_cfg_common_loopback(struct hclge_dev *hdev, bool en,
- enum hnae3_loop loop_mode)
+static int hclge_cfg_common_loopback_cmd_send(struct hclge_dev *hdev, bool en,
+ enum hnae3_loop loop_mode)
{
-#define HCLGE_COMMON_LB_RETRY_MS 10
-#define HCLGE_COMMON_LB_RETRY_NUM 100
-
struct hclge_common_lb_cmd *req;
struct hclge_desc desc;
- int ret, i = 0;
u8 loop_mode_b;
+ int ret;
req = (struct hclge_common_lb_cmd *)desc.data;
hclge_cmd_setup_basic_desc(&desc, HCLGE_OPC_COMMON_LOOPBACK, false);
@@ -7994,23 +7520,34 @@ static int hclge_cfg_common_loopback(struct hclge_dev *hdev, bool en,
break;
default:
dev_err(&hdev->pdev->dev,
- "unsupported common loopback mode %d\n", loop_mode);
+ "unsupported loopback mode %d\n", loop_mode);
return -ENOTSUPP;
}
- if (en) {
+ req->mask = loop_mode_b;
+ if (en)
req->enable = loop_mode_b;
- req->mask = loop_mode_b;
- } else {
- req->mask = loop_mode_b;
- }
ret = hclge_cmd_send(&hdev->hw, &desc, 1);
- if (ret) {
+ if (ret)
dev_err(&hdev->pdev->dev,
- "common loopback set fail, ret = %d\n", ret);
- return ret;
- }
+ "failed to send loopback cmd, loop_mode = %d, ret = %d\n",
+ loop_mode, ret);
+
+ return ret;
+}
+
+static int hclge_cfg_common_loopback_wait(struct hclge_dev *hdev)
+{
+#define HCLGE_COMMON_LB_RETRY_MS 10
+#define HCLGE_COMMON_LB_RETRY_NUM 100
+
+ struct hclge_common_lb_cmd *req;
+ struct hclge_desc desc;
+ u32 i = 0;
+ int ret;
+
+ req = (struct hclge_common_lb_cmd *)desc.data;
do {
msleep(HCLGE_COMMON_LB_RETRY_MS);
@@ -8019,20 +7556,34 @@ static int hclge_cfg_common_loopback(struct hclge_dev *hdev, bool en,
ret = hclge_cmd_send(&hdev->hw, &desc, 1);
if (ret) {
dev_err(&hdev->pdev->dev,
- "common loopback get, ret = %d\n", ret);
+ "failed to get loopback done status, ret = %d\n",
+ ret);
return ret;
}
} while (++i < HCLGE_COMMON_LB_RETRY_NUM &&
!(req->result & HCLGE_CMD_COMMON_LB_DONE_B));
if (!(req->result & HCLGE_CMD_COMMON_LB_DONE_B)) {
- dev_err(&hdev->pdev->dev, "common loopback set timeout\n");
+ dev_err(&hdev->pdev->dev, "wait loopback timeout\n");
return -EBUSY;
} else if (!(req->result & HCLGE_CMD_COMMON_LB_SUCCESS_B)) {
- dev_err(&hdev->pdev->dev, "common loopback set failed in fw\n");
+ dev_err(&hdev->pdev->dev, "failed to do loopback test\n");
return -EIO;
}
- return ret;
+
+ return 0;
+}
+
+static int hclge_cfg_common_loopback(struct hclge_dev *hdev, bool en,
+ enum hnae3_loop loop_mode)
+{
+ int ret;
+
+ ret = hclge_cfg_common_loopback_cmd_send(hdev, en, loop_mode);
+ if (ret)
+ return ret;
+
+ return hclge_cfg_common_loopback_wait(hdev);
}
static int hclge_set_common_loopback(struct hclge_dev *hdev, bool en,
@@ -8213,22 +7764,6 @@ static int hclge_set_default_loopback(struct hclge_dev *hdev)
HNAE3_LOOP_PARALLEL_SERDES);
}
-static void hclge_reset_tqp_stats(struct hnae3_handle *handle)
-{
- struct hclge_vport *vport = hclge_get_vport(handle);
- struct hnae3_knic_private_info *kinfo;
- struct hnae3_queue *queue;
- struct hclge_tqp *tqp;
- int i;
-
- kinfo = &vport->nic.kinfo;
- for (i = 0; i < kinfo->num_tqps; i++) {
- queue = handle->kinfo.tqp[i];
- tqp = container_of(queue, struct hclge_tqp, q);
- memset(&tqp->tqp_stats, 0, sizeof(tqp->tqp_stats));
- }
-}
-
static void hclge_flush_link_update(struct hclge_dev *hdev)
{
#define HCLGE_FLUSH_LINK_TIMEOUT 100000
@@ -8270,7 +7805,7 @@ static int hclge_ae_start(struct hnae3_handle *handle)
hdev->hw.mac.link = 0;
/* reset tqp stats */
- hclge_reset_tqp_stats(handle);
+ hclge_comm_reset_tqp_stats(handle);
hclge_mac_start_phy(hdev);
@@ -8308,7 +7843,7 @@ static void hclge_ae_stop(struct hnae3_handle *handle)
hclge_mac_stop_phy(hdev);
/* reset tqp stats */
- hclge_reset_tqp_stats(handle);
+ hclge_comm_reset_tqp_stats(handle);
hclge_update_link_status(hdev);
}
@@ -8511,14 +8046,14 @@ static int hclge_lookup_mac_vlan_tbl(struct hclge_vport *vport,
hclge_cmd_setup_basic_desc(&desc[0], HCLGE_OPC_MAC_VLAN_ADD, true);
if (is_mc) {
- desc[0].flag |= cpu_to_le16(HCLGE_CMD_FLAG_NEXT);
+ desc[0].flag |= cpu_to_le16(HCLGE_COMM_CMD_FLAG_NEXT);
memcpy(desc[0].data,
req,
sizeof(struct hclge_mac_vlan_tbl_entry_cmd));
hclge_cmd_setup_basic_desc(&desc[1],
HCLGE_OPC_MAC_VLAN_ADD,
true);
- desc[1].flag |= cpu_to_le16(HCLGE_CMD_FLAG_NEXT);
+ desc[1].flag |= cpu_to_le16(HCLGE_COMM_CMD_FLAG_NEXT);
hclge_cmd_setup_basic_desc(&desc[2],
HCLGE_OPC_MAC_VLAN_ADD,
true);
@@ -8568,12 +8103,12 @@ static int hclge_add_mac_vlan_tbl(struct hclge_vport *vport,
resp_code,
HCLGE_MAC_VLAN_ADD);
} else {
- hclge_cmd_reuse_desc(&mc_desc[0], false);
- mc_desc[0].flag |= cpu_to_le16(HCLGE_CMD_FLAG_NEXT);
- hclge_cmd_reuse_desc(&mc_desc[1], false);
- mc_desc[1].flag |= cpu_to_le16(HCLGE_CMD_FLAG_NEXT);
- hclge_cmd_reuse_desc(&mc_desc[2], false);
- mc_desc[2].flag &= cpu_to_le16(~HCLGE_CMD_FLAG_NEXT);
+ hclge_comm_cmd_reuse_desc(&mc_desc[0], false);
+ mc_desc[0].flag |= cpu_to_le16(HCLGE_COMM_CMD_FLAG_NEXT);
+ hclge_comm_cmd_reuse_desc(&mc_desc[1], false);
+ mc_desc[1].flag |= cpu_to_le16(HCLGE_COMM_CMD_FLAG_NEXT);
+ hclge_comm_cmd_reuse_desc(&mc_desc[2], false);
+ mc_desc[2].flag &= cpu_to_le16(~HCLGE_COMM_CMD_FLAG_NEXT);
memcpy(mc_desc[0].data, req,
sizeof(struct hclge_mac_vlan_tbl_entry_cmd));
ret = hclge_cmd_send(&hdev->hw, mc_desc, 3);
@@ -8743,6 +8278,7 @@ int hclge_update_mac_list(struct hclge_vport *vport,
enum HCLGE_MAC_ADDR_TYPE mac_type,
const unsigned char *addr)
{
+ char format_mac_addr[HNAE3_FORMAT_MAC_ADDR_LEN];
struct hclge_dev *hdev = vport->back;
struct hclge_mac_node *mac_node;
struct list_head *list;
@@ -8767,9 +8303,10 @@ int hclge_update_mac_list(struct hclge_vport *vport,
/* if this address is never added, unnecessary to delete */
if (state == HCLGE_MAC_TO_DEL) {
spin_unlock_bh(&vport->mac_list_lock);
+ hnae3_format_mac_addr(format_mac_addr, addr);
dev_err(&hdev->pdev->dev,
- "failed to delete address %pM from mac list\n",
- addr);
+ "failed to delete address %s from mac list\n",
+ format_mac_addr);
return -ENOENT;
}
@@ -8802,6 +8339,7 @@ static int hclge_add_uc_addr(struct hnae3_handle *handle,
int hclge_add_uc_addr_common(struct hclge_vport *vport,
const unsigned char *addr)
{
+ char format_mac_addr[HNAE3_FORMAT_MAC_ADDR_LEN];
struct hclge_dev *hdev = vport->back;
struct hclge_mac_vlan_tbl_entry_cmd req;
struct hclge_desc desc;
@@ -8812,9 +8350,10 @@ int hclge_add_uc_addr_common(struct hclge_vport *vport,
if (is_zero_ether_addr(addr) ||
is_broadcast_ether_addr(addr) ||
is_multicast_ether_addr(addr)) {
+ hnae3_format_mac_addr(format_mac_addr, addr);
dev_err(&hdev->pdev->dev,
- "Set_uc mac err! invalid mac:%pM. is_zero:%d,is_br=%d,is_mul=%d\n",
- addr, is_zero_ether_addr(addr),
+ "Set_uc mac err! invalid mac:%s. is_zero:%d,is_br=%d,is_mul=%d\n",
+ format_mac_addr, is_zero_ether_addr(addr),
is_broadcast_ether_addr(addr),
is_multicast_ether_addr(addr));
return -EINVAL;
@@ -8871,6 +8410,7 @@ static int hclge_rm_uc_addr(struct hnae3_handle *handle,
int hclge_rm_uc_addr_common(struct hclge_vport *vport,
const unsigned char *addr)
{
+ char format_mac_addr[HNAE3_FORMAT_MAC_ADDR_LEN];
struct hclge_dev *hdev = vport->back;
struct hclge_mac_vlan_tbl_entry_cmd req;
int ret;
@@ -8879,8 +8419,9 @@ int hclge_rm_uc_addr_common(struct hclge_vport *vport,
if (is_zero_ether_addr(addr) ||
is_broadcast_ether_addr(addr) ||
is_multicast_ether_addr(addr)) {
- dev_dbg(&hdev->pdev->dev, "Remove mac err! invalid mac:%pM.\n",
- addr);
+ hnae3_format_mac_addr(format_mac_addr, addr);
+ dev_dbg(&hdev->pdev->dev, "Remove mac err! invalid mac:%s.\n",
+ format_mac_addr);
return -EINVAL;
}
@@ -8911,6 +8452,7 @@ static int hclge_add_mc_addr(struct hnae3_handle *handle,
int hclge_add_mc_addr_common(struct hclge_vport *vport,
const unsigned char *addr)
{
+ char format_mac_addr[HNAE3_FORMAT_MAC_ADDR_LEN];
struct hclge_dev *hdev = vport->back;
struct hclge_mac_vlan_tbl_entry_cmd req;
struct hclge_desc desc[3];
@@ -8919,9 +8461,10 @@ int hclge_add_mc_addr_common(struct hclge_vport *vport,
/* mac addr check */
if (!is_multicast_ether_addr(addr)) {
+ hnae3_format_mac_addr(format_mac_addr, addr);
dev_err(&hdev->pdev->dev,
- "Add mc mac err! invalid mac:%pM.\n",
- addr);
+ "Add mc mac err! invalid mac:%s.\n",
+ format_mac_addr);
return -EINVAL;
}
memset(&req, 0, sizeof(req));
@@ -8973,16 +8516,18 @@ static int hclge_rm_mc_addr(struct hnae3_handle *handle,
int hclge_rm_mc_addr_common(struct hclge_vport *vport,
const unsigned char *addr)
{
+ char format_mac_addr[HNAE3_FORMAT_MAC_ADDR_LEN];
struct hclge_dev *hdev = vport->back;
struct hclge_mac_vlan_tbl_entry_cmd req;
- enum hclge_cmd_status status;
+ enum hclge_comm_cmd_status status;
struct hclge_desc desc[3];
/* mac addr check */
if (!is_multicast_ether_addr(addr)) {
+ hnae3_format_mac_addr(format_mac_addr, addr);
dev_dbg(&hdev->pdev->dev,
- "Remove mc mac err! invalid mac:%pM.\n",
- addr);
+ "Remove mc mac err! invalid mac:%s.\n",
+ format_mac_addr);
return -EINVAL;
}
@@ -9422,16 +8967,18 @@ static int hclge_set_vf_mac(struct hnae3_handle *handle, int vf,
u8 *mac_addr)
{
struct hclge_vport *vport = hclge_get_vport(handle);
+ char format_mac_addr[HNAE3_FORMAT_MAC_ADDR_LEN];
struct hclge_dev *hdev = vport->back;
vport = hclge_get_vf_vport(hdev, vf);
if (!vport)
return -EINVAL;
+ hnae3_format_mac_addr(format_mac_addr, mac_addr);
if (ether_addr_equal(mac_addr, vport->vf_info.mac)) {
dev_info(&hdev->pdev->dev,
- "Specified MAC(=%pM) is same as before, no change committed!\n",
- mac_addr);
+ "Specified MAC(=%s) is same as before, no change committed!\n",
+ format_mac_addr);
return 0;
}
@@ -9439,13 +8986,13 @@ static int hclge_set_vf_mac(struct hnae3_handle *handle, int vf,
if (test_bit(HCLGE_VPORT_STATE_ALIVE, &vport->state)) {
dev_info(&hdev->pdev->dev,
- "MAC of VF %d has been set to %pM, and it will be reinitialized!\n",
- vf, mac_addr);
+ "MAC of VF %d has been set to %s, and it will be reinitialized!\n",
+ vf, format_mac_addr);
return hclge_inform_reset_assert_to_vf(vport);
}
- dev_info(&hdev->pdev->dev, "MAC of VF %d has been set to %pM\n",
- vf, mac_addr);
+ dev_info(&hdev->pdev->dev, "MAC of VF %d has been set to %s\n",
+ vf, format_mac_addr);
return 0;
}
@@ -9549,6 +9096,7 @@ static int hclge_set_mac_addr(struct hnae3_handle *handle, const void *p,
{
const unsigned char *new_addr = (const unsigned char *)p;
struct hclge_vport *vport = hclge_get_vport(handle);
+ char format_mac_addr[HNAE3_FORMAT_MAC_ADDR_LEN];
struct hclge_dev *hdev = vport->back;
unsigned char *old_addr = NULL;
int ret;
@@ -9557,9 +9105,10 @@ static int hclge_set_mac_addr(struct hnae3_handle *handle, const void *p,
if (is_zero_ether_addr(new_addr) ||
is_broadcast_ether_addr(new_addr) ||
is_multicast_ether_addr(new_addr)) {
+ hnae3_format_mac_addr(format_mac_addr, new_addr);
dev_err(&hdev->pdev->dev,
- "change uc mac err! invalid mac: %pM.\n",
- new_addr);
+ "change uc mac err! invalid mac: %s.\n",
+ format_mac_addr);
return -EINVAL;
}
@@ -9577,9 +9126,10 @@ static int hclge_set_mac_addr(struct hnae3_handle *handle, const void *p,
spin_lock_bh(&vport->mac_list_lock);
ret = hclge_update_mac_node_for_dev_addr(vport, old_addr, new_addr);
if (ret) {
+ hnae3_format_mac_addr(format_mac_addr, new_addr);
dev_err(&hdev->pdev->dev,
- "failed to change the mac addr:%pM, ret = %d\n",
- new_addr, ret);
+ "failed to change the mac addr:%s, ret = %d\n",
+ format_mac_addr, ret);
spin_unlock_bh(&vport->mac_list_lock);
if (!is_first)
@@ -9677,20 +9227,20 @@ static int hclge_set_vlan_filter_ctrl(struct hclge_dev *hdev, u8 vlan_type,
ret = hclge_cmd_send(&hdev->hw, &desc, 1);
if (ret) {
- dev_err(&hdev->pdev->dev,
- "failed to get vlan filter config, ret = %d.\n", ret);
+ dev_err(&hdev->pdev->dev, "failed to get vport%u vlan filter config, ret = %d.\n",
+ vf_id, ret);
return ret;
}
/* modify and write new config parameter */
- hclge_cmd_reuse_desc(&desc, false);
+ hclge_comm_cmd_reuse_desc(&desc, false);
req->vlan_fe = filter_en ?
(req->vlan_fe | fe_type) : (req->vlan_fe & ~fe_type);
ret = hclge_cmd_send(&hdev->hw, &desc, 1);
if (ret)
- dev_err(&hdev->pdev->dev, "failed to set vlan filter, ret = %d.\n",
- ret);
+ dev_err(&hdev->pdev->dev, "failed to set vport%u vlan filter, ret = %d.\n",
+ vf_id, ret);
return ret;
}
@@ -9809,7 +9359,7 @@ static int hclge_set_vf_vlan_filter_cmd(struct hclge_dev *hdev, u16 vfid,
hclge_cmd_setup_basic_desc(&desc[1],
HCLGE_OPC_VLAN_FILTER_VF_CFG, false);
- desc[0].flag |= cpu_to_le16(HCLGE_CMD_FLAG_NEXT);
+ desc[0].flag |= cpu_to_le16(HCLGE_COMM_CMD_FLAG_NEXT);
vf_byte_off = vfid / 8;
vf_byte_val = 1 << (vfid % 8);
@@ -9936,6 +9486,32 @@ static int hclge_set_port_vlan_filter(struct hclge_dev *hdev, __be16 proto,
return ret;
}
+static bool hclge_need_update_port_vlan(struct hclge_dev *hdev, u16 vport_id,
+ u16 vlan_id, bool is_kill)
+{
+ /* vlan 0 may be added twice when 8021q module is enabled */
+ if (!is_kill && !vlan_id &&
+ test_bit(vport_id, hdev->vlan_table[vlan_id]))
+ return false;
+
+ if (!is_kill && test_and_set_bit(vport_id, hdev->vlan_table[vlan_id])) {
+ dev_warn(&hdev->pdev->dev,
+ "Add port vlan failed, vport %u is already in vlan %u\n",
+ vport_id, vlan_id);
+ return false;
+ }
+
+ if (is_kill &&
+ !test_and_clear_bit(vport_id, hdev->vlan_table[vlan_id])) {
+ dev_warn(&hdev->pdev->dev,
+ "Delete port vlan failed, vport %u is not in vlan %u\n",
+ vport_id, vlan_id);
+ return false;
+ }
+
+ return true;
+}
+
static int hclge_set_vlan_filter_hw(struct hclge_dev *hdev, __be16 proto,
u16 vport_id, u16 vlan_id,
bool is_kill)
@@ -9957,26 +9533,9 @@ static int hclge_set_vlan_filter_hw(struct hclge_dev *hdev, __be16 proto,
return ret;
}
- /* vlan 0 may be added twice when 8021q module is enabled */
- if (!is_kill && !vlan_id &&
- test_bit(vport_id, hdev->vlan_table[vlan_id]))
+ if (!hclge_need_update_port_vlan(hdev, vport_id, vlan_id, is_kill))
return 0;
- if (!is_kill && test_and_set_bit(vport_id, hdev->vlan_table[vlan_id])) {
- dev_err(&hdev->pdev->dev,
- "Add port vlan failed, vport %u is already in vlan %u\n",
- vport_id, vlan_id);
- return -EINVAL;
- }
-
- if (is_kill &&
- !test_and_clear_bit(vport_id, hdev->vlan_table[vlan_id])) {
- dev_err(&hdev->pdev->dev,
- "Delete port vlan failed, vport %u is not in vlan %u\n",
- vport_id, vlan_id);
- return -EINVAL;
- }
-
for_each_set_bit(vport_idx, hdev->vlan_table[vlan_id], HCLGE_VPORT_NUM)
vport_num++;
@@ -10168,67 +9727,80 @@ static int hclge_set_vlan_protocol_type(struct hclge_dev *hdev)
return status;
}
-static int hclge_init_vlan_config(struct hclge_dev *hdev)
+static int hclge_init_vlan_filter(struct hclge_dev *hdev)
{
-#define HCLGE_DEF_VLAN_TYPE 0x8100
-
- struct hnae3_handle *handle = &hdev->vport[0].nic;
struct hclge_vport *vport;
int ret;
int i;
- if (hdev->ae_dev->dev_version >= HNAE3_DEVICE_VERSION_V2) {
- /* for revision 0x21, vf vlan filter is per function */
- for (i = 0; i < hdev->num_alloc_vport; i++) {
- vport = &hdev->vport[i];
- ret = hclge_set_vlan_filter_ctrl(hdev,
- HCLGE_FILTER_TYPE_VF,
- HCLGE_FILTER_FE_EGRESS,
- true,
- vport->vport_id);
- if (ret)
- return ret;
- vport->cur_vlan_fltr_en = true;
- }
+ if (hdev->ae_dev->dev_version < HNAE3_DEVICE_VERSION_V2)
+ return hclge_set_vlan_filter_ctrl(hdev, HCLGE_FILTER_TYPE_VF,
+ HCLGE_FILTER_FE_EGRESS_V1_B,
+ true, 0);
- ret = hclge_set_vlan_filter_ctrl(hdev, HCLGE_FILTER_TYPE_PORT,
- HCLGE_FILTER_FE_INGRESS, true,
- 0);
+ /* for revision 0x21, vf vlan filter is per function */
+ for (i = 0; i < hdev->num_alloc_vport; i++) {
+ vport = &hdev->vport[i];
+ ret = hclge_set_vlan_filter_ctrl(hdev, HCLGE_FILTER_TYPE_VF,
+ HCLGE_FILTER_FE_EGRESS, true,
+ vport->vport_id);
if (ret)
return ret;
- } else {
- ret = hclge_set_vlan_filter_ctrl(hdev, HCLGE_FILTER_TYPE_VF,
- HCLGE_FILTER_FE_EGRESS_V1_B,
- true, 0);
+ vport->cur_vlan_fltr_en = true;
+ }
+
+ return hclge_set_vlan_filter_ctrl(hdev, HCLGE_FILTER_TYPE_PORT,
+ HCLGE_FILTER_FE_INGRESS, true, 0);
+}
+
+static int hclge_init_vlan_type(struct hclge_dev *hdev)
+{
+ hdev->vlan_type_cfg.rx_in_fst_vlan_type = ETH_P_8021Q;
+ hdev->vlan_type_cfg.rx_in_sec_vlan_type = ETH_P_8021Q;
+ hdev->vlan_type_cfg.rx_ot_fst_vlan_type = ETH_P_8021Q;
+ hdev->vlan_type_cfg.rx_ot_sec_vlan_type = ETH_P_8021Q;
+ hdev->vlan_type_cfg.tx_ot_vlan_type = ETH_P_8021Q;
+ hdev->vlan_type_cfg.tx_in_vlan_type = ETH_P_8021Q;
+
+ return hclge_set_vlan_protocol_type(hdev);
+}
+
+static int hclge_init_vport_vlan_offload(struct hclge_dev *hdev)
+{
+ struct hclge_port_base_vlan_config *cfg;
+ struct hclge_vport *vport;
+ int ret;
+ int i;
+
+ for (i = 0; i < hdev->num_alloc_vport; i++) {
+ vport = &hdev->vport[i];
+ cfg = &vport->port_base_vlan_cfg;
+
+ ret = hclge_vlan_offload_cfg(vport, cfg->state,
+ cfg->vlan_info.vlan_tag,
+ cfg->vlan_info.qos);
if (ret)
return ret;
}
+ return 0;
+}
- hdev->vlan_type_cfg.rx_in_fst_vlan_type = HCLGE_DEF_VLAN_TYPE;
- hdev->vlan_type_cfg.rx_in_sec_vlan_type = HCLGE_DEF_VLAN_TYPE;
- hdev->vlan_type_cfg.rx_ot_fst_vlan_type = HCLGE_DEF_VLAN_TYPE;
- hdev->vlan_type_cfg.rx_ot_sec_vlan_type = HCLGE_DEF_VLAN_TYPE;
- hdev->vlan_type_cfg.tx_ot_vlan_type = HCLGE_DEF_VLAN_TYPE;
- hdev->vlan_type_cfg.tx_in_vlan_type = HCLGE_DEF_VLAN_TYPE;
+static int hclge_init_vlan_config(struct hclge_dev *hdev)
+{
+ struct hnae3_handle *handle = &hdev->vport[0].nic;
+ int ret;
- ret = hclge_set_vlan_protocol_type(hdev);
+ ret = hclge_init_vlan_filter(hdev);
if (ret)
return ret;
- for (i = 0; i < hdev->num_alloc_vport; i++) {
- u16 vlan_tag;
- u8 qos;
+ ret = hclge_init_vlan_type(hdev);
+ if (ret)
+ return ret;
- vport = &hdev->vport[i];
- vlan_tag = vport->port_base_vlan_cfg.vlan_info.vlan_tag;
- qos = vport->port_base_vlan_cfg.vlan_info.qos;
-
- ret = hclge_vlan_offload_cfg(vport,
- vport->port_base_vlan_cfg.state,
- vlan_tag, qos);
- if (ret)
- return ret;
- }
+ ret = hclge_init_vport_vlan_offload(hdev);
+ if (ret)
+ return ret;
return hclge_set_vlan_filter(handle, htons(ETH_P_8021Q), 0, false);
}
@@ -10485,12 +10057,41 @@ static bool hclge_need_update_vlan_filter(const struct hclge_vlan_info *new_cfg,
return false;
}
+static int hclge_modify_port_base_vlan_tag(struct hclge_vport *vport,
+ struct hclge_vlan_info *new_info,
+ struct hclge_vlan_info *old_info)
+{
+ struct hclge_dev *hdev = vport->back;
+ int ret;
+
+ /* add new VLAN tag */
+ ret = hclge_set_vlan_filter_hw(hdev, htons(new_info->vlan_proto),
+ vport->vport_id, new_info->vlan_tag,
+ false);
+ if (ret)
+ return ret;
+
+ /* remove old VLAN tag */
+ if (old_info->vlan_tag == 0)
+ ret = hclge_set_vf_vlan_common(hdev, vport->vport_id,
+ true, 0);
+ else
+ ret = hclge_set_vlan_filter_hw(hdev, htons(ETH_P_8021Q),
+ vport->vport_id,
+ old_info->vlan_tag, true);
+ if (ret)
+ dev_err(&hdev->pdev->dev,
+ "failed to clear vport%u port base vlan %u, ret = %d.\n",
+ vport->vport_id, old_info->vlan_tag, ret);
+
+ return ret;
+}
+
int hclge_update_port_base_vlan_cfg(struct hclge_vport *vport, u16 state,
struct hclge_vlan_info *vlan_info)
{
struct hnae3_handle *nic = &vport->nic;
struct hclge_vlan_info *old_vlan_info;
- struct hclge_dev *hdev = vport->back;
int ret;
old_vlan_info = &vport->port_base_vlan_cfg.vlan_info;
@@ -10503,38 +10104,12 @@ int hclge_update_port_base_vlan_cfg(struct hclge_vport *vport, u16 state,
if (!hclge_need_update_vlan_filter(vlan_info, old_vlan_info))
goto out;
- if (state == HNAE3_PORT_BASE_VLAN_MODIFY) {
- /* add new VLAN tag */
- ret = hclge_set_vlan_filter_hw(hdev,
- htons(vlan_info->vlan_proto),
- vport->vport_id,
- vlan_info->vlan_tag,
- false);
- if (ret)
- return ret;
-
- /* remove old VLAN tag */
- if (old_vlan_info->vlan_tag == 0)
- ret = hclge_set_vf_vlan_common(hdev, vport->vport_id,
- true, 0);
- else
- ret = hclge_set_vlan_filter_hw(hdev,
- htons(ETH_P_8021Q),
- vport->vport_id,
- old_vlan_info->vlan_tag,
- true);
- if (ret) {
- dev_err(&hdev->pdev->dev,
- "failed to clear vport%u port base vlan %u, ret = %d.\n",
- vport->vport_id, old_vlan_info->vlan_tag, ret);
- return ret;
- }
-
- goto out;
- }
-
- ret = hclge_update_vlan_filter_entries(vport, state, vlan_info,
- old_vlan_info);
+ if (state == HNAE3_PORT_BASE_VLAN_MODIFY)
+ ret = hclge_modify_port_base_vlan_tag(vport, vlan_info,
+ old_vlan_info);
+ else
+ ret = hclge_update_vlan_filter_entries(vport, state, vlan_info,
+ old_vlan_info);
if (ret)
return ret;
@@ -10881,11 +10456,11 @@ static int hclge_get_reset_status(struct hclge_dev *hdev, u16 queue_id,
u16 hclge_covert_handle_qid_global(struct hnae3_handle *handle, u16 queue_id)
{
+ struct hclge_comm_tqp *tqp;
struct hnae3_queue *queue;
- struct hclge_tqp *tqp;
queue = handle->kinfo.tqp[queue_id];
- tqp = container_of(queue, struct hclge_tqp, q);
+ tqp = container_of(queue, struct hclge_comm_tqp, q);
return tqp->index;
}
@@ -11442,10 +11017,11 @@ static int hclge_dev_mem_map(struct hclge_dev *hdev)
if (!(pci_select_bars(pdev, IORESOURCE_MEM) & BIT(HCLGE_MEM_BAR)))
return 0;
- hw->mem_base = devm_ioremap_wc(&pdev->dev,
- pci_resource_start(pdev, HCLGE_MEM_BAR),
- pci_resource_len(pdev, HCLGE_MEM_BAR));
- if (!hw->mem_base) {
+ hw->hw.mem_base =
+ devm_ioremap_wc(&pdev->dev,
+ pci_resource_start(pdev, HCLGE_MEM_BAR),
+ pci_resource_len(pdev, HCLGE_MEM_BAR));
+ if (!hw->hw.mem_base) {
dev_err(&pdev->dev, "failed to map device memory\n");
return -EFAULT;
}
@@ -11484,8 +11060,8 @@ static int hclge_pci_init(struct hclge_dev *hdev)
pci_set_master(pdev);
hw = &hdev->hw;
- hw->io_base = pcim_iomap(pdev, 2, 0);
- if (!hw->io_base) {
+ hw->hw.io_base = pcim_iomap(pdev, 2, 0);
+ if (!hw->hw.io_base) {
dev_err(&pdev->dev, "Can't map configuration register space\n");
ret = -ENOMEM;
goto err_clr_master;
@@ -11500,7 +11076,7 @@ static int hclge_pci_init(struct hclge_dev *hdev)
return 0;
err_unmap_io_base:
- pcim_iounmap(pdev, hdev->hw.io_base);
+ pcim_iounmap(pdev, hdev->hw.hw.io_base);
err_clr_master:
pci_clear_master(pdev);
pci_release_regions(pdev);
@@ -11514,10 +11090,10 @@ static void hclge_pci_uninit(struct hclge_dev *hdev)
{
struct pci_dev *pdev = hdev->pdev;
- if (hdev->hw.mem_base)
- devm_iounmap(&pdev->dev, hdev->hw.mem_base);
+ if (hdev->hw.hw.mem_base)
+ devm_iounmap(&pdev->dev, hdev->hw.hw.mem_base);
- pcim_iounmap(pdev, hdev->hw.io_base);
+ pcim_iounmap(pdev, hdev->hw.hw.io_base);
pci_free_irq_vectors(pdev);
pci_clear_master(pdev);
pci_release_mem_regions(pdev);
@@ -11556,29 +11132,25 @@ static void hclge_reset_prepare_general(struct hnae3_ae_dev *ae_dev,
int retry_cnt = 0;
int ret;
-retry:
- down(&hdev->reset_sem);
- set_bit(HCLGE_STATE_RST_HANDLING, &hdev->state);
- hdev->reset_type = rst_type;
- ret = hclge_reset_prepare(hdev);
- if (ret || hdev->reset_pending) {
- dev_err(&hdev->pdev->dev, "fail to prepare to reset, ret=%d\n",
- ret);
- if (hdev->reset_pending ||
- retry_cnt++ < HCLGE_RESET_RETRY_CNT) {
- dev_err(&hdev->pdev->dev,
- "reset_pending:0x%lx, retry_cnt:%d\n",
- hdev->reset_pending, retry_cnt);
- clear_bit(HCLGE_STATE_RST_HANDLING, &hdev->state);
- up(&hdev->reset_sem);
- msleep(HCLGE_RESET_RETRY_WAIT_MS);
- goto retry;
- }
+ while (retry_cnt++ < HCLGE_RESET_RETRY_CNT) {
+ down(&hdev->reset_sem);
+ set_bit(HCLGE_STATE_RST_HANDLING, &hdev->state);
+ hdev->reset_type = rst_type;
+ ret = hclge_reset_prepare(hdev);
+ if (!ret && !hdev->reset_pending)
+ break;
+
+ dev_err(&hdev->pdev->dev,
+ "failed to prepare to reset, ret=%d, reset_pending:0x%lx, retry_cnt:%d\n",
+ ret, hdev->reset_pending, retry_cnt);
+ clear_bit(HCLGE_STATE_RST_HANDLING, &hdev->state);
+ up(&hdev->reset_sem);
+ msleep(HCLGE_RESET_RETRY_WAIT_MS);
}
/* disable misc vector before reset done */
hclge_enable_vector(&hdev->misc_vector, false);
- set_bit(HCLGE_STATE_CMD_DISABLE, &hdev->state);
+ set_bit(HCLGE_COMM_STATE_CMD_DISABLE, &hdev->hw.hw.comm_state);
if (hdev->reset_type == HNAE3_FLR_RESET)
hdev->rst_stats.flr_rst_cnt++;
@@ -11683,12 +11255,13 @@ static int hclge_init_ae_dev(struct hnae3_ae_dev *ae_dev)
goto err_pci_uninit;
/* Firmware command queue initialize */
- ret = hclge_cmd_queue_init(hdev);
+ ret = hclge_comm_cmd_queue_init(hdev->pdev, &hdev->hw.hw);
if (ret)
goto err_devlink_uninit;
/* Firmware command initialize */
- ret = hclge_cmd_init(hdev);
+ ret = hclge_comm_cmd_init(hdev->ae_dev, &hdev->hw.hw, &hdev->fw_version,
+ true, hdev->reset_pending);
if (ret)
goto err_cmd_uninit;
@@ -11776,7 +11349,8 @@ static int hclge_init_ae_dev(struct hnae3_ae_dev *ae_dev)
goto err_mdiobus_unreg;
}
- ret = hclge_rss_init_cfg(hdev);
+ ret = hclge_comm_rss_init_cfg(&hdev->vport->nic, hdev->ae_dev,
+ &hdev->rss_cfg);
if (ret) {
dev_err(&pdev->dev, "failed to init rss cfg, ret = %d\n", ret);
goto err_mdiobus_unreg;
@@ -11861,11 +11435,11 @@ static int hclge_init_ae_dev(struct hnae3_ae_dev *ae_dev)
err_msi_uninit:
pci_free_irq_vectors(pdev);
err_cmd_uninit:
- hclge_cmd_uninit(hdev);
+ hclge_comm_cmd_uninit(hdev->ae_dev, &hdev->hw.hw);
err_devlink_uninit:
hclge_devlink_uninit(hdev);
err_pci_uninit:
- pcim_iounmap(pdev, hdev->hw.io_base);
+ pcim_iounmap(pdev, hdev->hw.hw.io_base);
pci_clear_master(pdev);
pci_release_regions(pdev);
pci_disable_device(pdev);
@@ -12112,7 +11686,8 @@ static int hclge_reset_ae_dev(struct hnae3_ae_dev *ae_dev)
hclge_reset_umv_space(hdev);
}
- ret = hclge_cmd_init(hdev);
+ ret = hclge_comm_cmd_init(hdev->ae_dev, &hdev->hw.hw, &hdev->fw_version,
+ true, hdev->reset_pending);
if (ret) {
dev_err(&pdev->dev, "Cmd queue init failed\n");
return ret;
@@ -12252,7 +11827,7 @@ static void hclge_uninit_ae_dev(struct hnae3_ae_dev *ae_dev)
hclge_config_nic_hw_error(hdev, false);
hclge_config_rocee_ras_interrupt(hdev, false);
- hclge_cmd_uninit(hdev);
+ hclge_comm_cmd_uninit(hdev->ae_dev, &hdev->hw.hw);
hclge_misc_irq_uninit(hdev);
hclge_devlink_uninit(hdev);
hclge_pci_uninit(hdev);
@@ -12288,19 +11863,43 @@ static void hclge_get_tqps_and_rss_info(struct hnae3_handle *handle,
*max_rss_size = hdev->pf_rss_size_max;
}
+static int hclge_set_rss_tc_mode_cfg(struct hnae3_handle *handle)
+{
+ struct hclge_vport *vport = hclge_get_vport(handle);
+ u16 tc_offset[HCLGE_MAX_TC_NUM] = {0};
+ struct hclge_dev *hdev = vport->back;
+ u16 tc_size[HCLGE_MAX_TC_NUM] = {0};
+ u16 tc_valid[HCLGE_MAX_TC_NUM];
+ u16 roundup_size;
+ unsigned int i;
+
+ roundup_size = roundup_pow_of_two(vport->nic.kinfo.rss_size);
+ roundup_size = ilog2(roundup_size);
+ /* Set the RSS TC mode according to the new RSS size */
+ for (i = 0; i < HCLGE_MAX_TC_NUM; i++) {
+ tc_valid[i] = 0;
+
+ if (!(hdev->hw_tc_map & BIT(i)))
+ continue;
+
+ tc_valid[i] = 1;
+ tc_size[i] = roundup_size;
+ tc_offset[i] = vport->nic.kinfo.rss_size * i;
+ }
+
+ return hclge_comm_set_rss_tc_mode(&hdev->hw.hw, tc_offset, tc_valid,
+ tc_size);
+}
+
static int hclge_set_channels(struct hnae3_handle *handle, u32 new_tqps_num,
bool rxfh_configured)
{
struct hnae3_ae_dev *ae_dev = pci_get_drvdata(handle->pdev);
struct hclge_vport *vport = hclge_get_vport(handle);
struct hnae3_knic_private_info *kinfo = &vport->nic.kinfo;
- u16 tc_offset[HCLGE_MAX_TC_NUM] = {0};
struct hclge_dev *hdev = vport->back;
- u16 tc_size[HCLGE_MAX_TC_NUM] = {0};
u16 cur_rss_size = kinfo->rss_size;
u16 cur_tqps = kinfo->num_tqps;
- u16 tc_valid[HCLGE_MAX_TC_NUM];
- u16 roundup_size;
u32 *rss_indir;
unsigned int i;
int ret;
@@ -12313,20 +11912,7 @@ static int hclge_set_channels(struct hnae3_handle *handle, u32 new_tqps_num,
return ret;
}
- roundup_size = roundup_pow_of_two(kinfo->rss_size);
- roundup_size = ilog2(roundup_size);
- /* Set the RSS TC mode according to the new RSS size */
- for (i = 0; i < HCLGE_MAX_TC_NUM; i++) {
- tc_valid[i] = 0;
-
- if (!(hdev->hw_tc_map & BIT(i)))
- continue;
-
- tc_valid[i] = 1;
- tc_size[i] = roundup_size;
- tc_offset[i] = kinfo->rss_size * i;
- }
- ret = hclge_set_rss_tc_mode(hdev, tc_valid, tc_size, tc_offset);
+ ret = hclge_set_rss_tc_mode_cfg(handle);
if (ret)
return ret;
@@ -12508,7 +12094,7 @@ int hclge_query_bd_num_cmd_send(struct hclge_dev *hdev, struct hclge_desc *desc)
for (i = 0; i < HCLGE_GET_DFX_REG_TYPE_CNT - 1; i++) {
hclge_cmd_setup_basic_desc(&desc[i], HCLGE_OPC_DFX_BD_NUM,
true);
- desc[i].flag |= cpu_to_le16(HCLGE_CMD_FLAG_NEXT);
+ desc[i].flag |= cpu_to_le16(HCLGE_COMM_CMD_FLAG_NEXT);
}
/* initialize the last command BD */
@@ -12552,7 +12138,7 @@ static int hclge_dfx_reg_cmd_send(struct hclge_dev *hdev,
hclge_cmd_setup_basic_desc(desc, cmd, true);
for (i = 0; i < bd_num - 1; i++) {
- desc->flag |= cpu_to_le16(HCLGE_CMD_FLAG_NEXT);
+ desc->flag |= cpu_to_le16(HCLGE_COMM_CMD_FLAG_NEXT);
desc++;
hclge_cmd_setup_basic_desc(desc, cmd, true);
}
@@ -12985,7 +12571,7 @@ static u16 hclge_get_sfp_eeprom_info(struct hclge_dev *hdev, u32 offset,
/* bd0~bd4 need next flag */
if (i < HCLGE_SFP_INFO_CMD_NUM - 1)
- desc[i].flag |= cpu_to_le16(HCLGE_CMD_FLAG_NEXT);
+ desc[i].flag |= cpu_to_le16(HCLGE_COMM_CMD_FLAG_NEXT);
}
/* setup bd0, this bd contains offset and read length. */
@@ -13095,7 +12681,7 @@ static const struct hnae3_ae_ops hclge_ops = {
.check_port_speed = hclge_check_port_speed,
.get_fec = hclge_get_fec,
.set_fec = hclge_set_fec,
- .get_rss_key_size = hclge_get_rss_key_size,
+ .get_rss_key_size = hclge_comm_get_rss_key_size,
.get_rss = hclge_get_rss,
.set_rss = hclge_set_rss,
.set_rss_tuple = hclge_set_rss_tuple,
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.h b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.h
index ebba603..adfb26e 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.h
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.h
@@ -13,6 +13,8 @@
#include "hclge_cmd.h"
#include "hclge_ptp.h"
#include "hnae3.h"
+#include "hclge_comm_rss.h"
+#include "hclge_comm_tqp_stats.h"
#define HCLGE_MOD_VERSION "1.0"
#define HCLGE_DRIVER_NAME "hclge"
@@ -38,20 +40,7 @@
#define HCLGE_VECTOR_REG_OFFSET_H 0x1000
#define HCLGE_VECTOR_VF_OFFSET 0x100000
-#define HCLGE_NIC_CSQ_BASEADDR_L_REG 0x27000
-#define HCLGE_NIC_CSQ_BASEADDR_H_REG 0x27004
#define HCLGE_NIC_CSQ_DEPTH_REG 0x27008
-#define HCLGE_NIC_CSQ_TAIL_REG 0x27010
-#define HCLGE_NIC_CSQ_HEAD_REG 0x27014
-#define HCLGE_NIC_CRQ_BASEADDR_L_REG 0x27018
-#define HCLGE_NIC_CRQ_BASEADDR_H_REG 0x2701C
-#define HCLGE_NIC_CRQ_DEPTH_REG 0x27020
-#define HCLGE_NIC_CRQ_TAIL_REG 0x27024
-#define HCLGE_NIC_CRQ_HEAD_REG 0x27028
-
-#define HCLGE_CMDQ_INTR_STS_REG 0x27104
-#define HCLGE_CMDQ_INTR_EN_REG 0x27108
-#define HCLGE_CMDQ_INTR_GEN_REG 0x2710C
/* bar registers for common func */
#define HCLGE_GRO_EN_REG 0x28000
@@ -93,22 +82,6 @@
#define HCLGE_TQP_INTR_RL_REG 0x20900
#define HCLGE_RSS_IND_TBL_SIZE 512
-#define HCLGE_RSS_SET_BITMAP_MSK GENMASK(15, 0)
-#define HCLGE_RSS_KEY_SIZE 40
-#define HCLGE_RSS_HASH_ALGO_TOEPLITZ 0
-#define HCLGE_RSS_HASH_ALGO_SIMPLE 1
-#define HCLGE_RSS_HASH_ALGO_SYMMETRIC 2
-#define HCLGE_RSS_HASH_ALGO_MASK GENMASK(3, 0)
-
-#define HCLGE_RSS_INPUT_TUPLE_OTHER GENMASK(3, 0)
-#define HCLGE_RSS_INPUT_TUPLE_SCTP GENMASK(4, 0)
-#define HCLGE_D_PORT_BIT BIT(0)
-#define HCLGE_S_PORT_BIT BIT(1)
-#define HCLGE_D_IP_BIT BIT(2)
-#define HCLGE_S_IP_BIT BIT(3)
-#define HCLGE_V_TAG_BIT BIT(4)
-#define HCLGE_RSS_INPUT_TUPLE_SCTP_NO_PORT \
- (HCLGE_D_IP_BIT | HCLGE_S_IP_BIT | HCLGE_V_TAG_BIT)
#define HCLGE_RSS_TC_SIZE_0 1
#define HCLGE_RSS_TC_SIZE_1 2
@@ -228,7 +201,6 @@ enum HCLGE_DEV_STATE {
HCLGE_STATE_MBX_HANDLING,
HCLGE_STATE_ERR_SERVICE_SCHED,
HCLGE_STATE_STATISTICS_UPDATING,
- HCLGE_STATE_CMD_DISABLE,
HCLGE_STATE_LINK_UPDATING,
HCLGE_STATE_RST_FAIL,
HCLGE_STATE_FD_TBL_CHANGED,
@@ -294,31 +266,9 @@ struct hclge_mac {
};
struct hclge_hw {
- void __iomem *io_base;
- void __iomem *mem_base;
+ struct hclge_comm_hw hw;
struct hclge_mac mac;
int num_vec;
- struct hclge_cmq cmq;
-};
-
-/* TQP stats */
-struct hlcge_tqp_stats {
- /* query_tqp_tx_queue_statistics ,opcode id: 0x0B03 */
- u64 rcb_tx_ring_pktnum_rcd; /* 32bit */
- /* query_tqp_rx_queue_statistics ,opcode id: 0x0B13 */
- u64 rcb_rx_ring_pktnum_rcd; /* 32bit */
-};
-
-struct hclge_tqp {
- /* copy of device pointer from pci_dev,
- * used when perform DMA mapping
- */
- struct device *dev;
- struct hnae3_queue q;
- struct hlcge_tqp_stats tqp_stats;
- u16 index; /* Global index in a NIC controller */
-
- bool alloced;
};
enum hclge_fc_mode {
@@ -641,6 +591,11 @@ struct key_info {
#define MAX_FD_FILTER_NUM 4096
#define HCLGE_ARFS_EXPIRE_INTERVAL 5UL
+#define hclge_read_dev(a, reg) \
+ hclge_comm_read_reg((a)->hw.io_base, reg)
+#define hclge_write_dev(a, reg, value) \
+ hclge_comm_write_reg((a)->hw.io_base, reg, value)
+
enum HCLGE_FD_ACTIVE_RULE_TYPE {
HCLGE_FD_RULE_NONE,
HCLGE_FD_ARFS_ACTIVE,
@@ -920,7 +875,7 @@ struct hclge_dev {
bool cur_promisc;
int num_alloc_vfs; /* Actual number of VFs allocated */
- struct hclge_tqp *htqp;
+ struct hclge_comm_tqp *htqp;
struct hclge_vport *vport;
struct dentry *hclge_dbgfs;
@@ -955,6 +910,8 @@ struct hclge_dev {
u16 hclge_fd_rule_num;
unsigned long serv_processed_cnt;
unsigned long last_serv_processed;
+ unsigned long last_rst_scheduled;
+ unsigned long last_mbx_scheduled;
unsigned long fd_bmap[BITS_TO_LONGS(MAX_FD_FILTER_NUM)];
enum HCLGE_FD_ACTIVE_RULE_TYPE fd_active_type;
u8 fd_en;
@@ -977,6 +934,7 @@ struct hclge_dev {
cpumask_t affinity_mask;
struct hclge_ptp *ptp;
struct devlink *devlink;
+ struct hclge_comm_rss_cfg rss_cfg;
};
/* VPort level vlan tag configuration for TX direction */
@@ -1003,17 +961,6 @@ struct hclge_rx_vtag_cfg {
bool strip_tag2_discard_en; /* Outer vlan tag discard for BD enable */
};
-struct hclge_rss_tuple_cfg {
- u8 ipv4_tcp_en;
- u8 ipv4_udp_en;
- u8 ipv4_sctp_en;
- u8 ipv4_fragment_en;
- u8 ipv6_tcp_en;
- u8 ipv6_udp_en;
- u8 ipv6_sctp_en;
- u8 ipv6_fragment_en;
-};
-
enum HCLGE_VPORT_STATE {
HCLGE_VPORT_STATE_ALIVE,
HCLGE_VPORT_STATE_MAC_TBL_CHANGE,
@@ -1047,15 +994,6 @@ struct hclge_vf_info {
struct hclge_vport {
u16 alloc_tqps; /* Allocated Tx/Rx queues */
- u8 rss_hash_key[HCLGE_RSS_KEY_SIZE]; /* User configured hash keys */
- /* User configured lookup table entries */
- u16 *rss_indirection_tbl;
- int rss_algo; /* User configured hash algorithm */
- /* User configured rss tuple sets */
- struct hclge_rss_tuple_cfg rss_tuple_sets;
-
- u16 alloc_rss_size;
-
u16 qs_offset;
u32 bw_limit; /* VSI BW Limit (0 = disabled) */
u8 dwrr;
@@ -1093,6 +1031,11 @@ struct hclge_speed_bit_map {
u32 speed_bit;
};
+struct hclge_mac_speed_map {
+ u32 speed_drv; /* speed defined in driver */
+ u32 speed_fw; /* speed defined in firmware */
+};
+
int hclge_set_vport_promisc_mode(struct hclge_vport *vport, bool en_uc_pmc,
bool en_mc_pmc, bool en_bc_pmc);
int hclge_add_uc_addr_common(struct hclge_vport *vport,
@@ -1111,7 +1054,8 @@ int hclge_bind_ring_with_vector(struct hclge_vport *vport,
static inline int hclge_get_queue_id(struct hnae3_queue *queue)
{
- struct hclge_tqp *tqp = container_of(queue, struct hclge_tqp, q);
+ struct hclge_comm_tqp *tqp =
+ container_of(queue, struct hclge_comm_tqp, q);
return tqp->index;
}
@@ -1129,7 +1073,6 @@ int hclge_en_hw_strip_rxvtag(struct hnae3_handle *handle, bool enable);
int hclge_buffer_alloc(struct hclge_dev *hdev);
int hclge_rss_init_hw(struct hclge_dev *hdev);
-void hclge_rss_indir_init_cfg(struct hclge_dev *hdev);
void hclge_mbx_handler(struct hclge_dev *hdev);
int hclge_reset_tqp(struct hnae3_handle *handle);
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_mbx.c b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_mbx.c
index 65d78ee..6799d16 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_mbx.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_mbx.c
@@ -4,6 +4,7 @@
#include "hclge_main.h"
#include "hclge_mbx.h"
#include "hnae3.h"
+#include "hclge_comm_rss.h"
#define CREATE_TRACE_POINTS
#include "hclge_trace.h"
@@ -33,7 +34,7 @@ static int hclge_gen_resp_to_vf(struct hclge_vport *vport,
{
struct hclge_mbx_pf_to_vf_cmd *resp_pf_to_vf;
struct hclge_dev *hdev = vport->back;
- enum hclge_cmd_status status;
+ enum hclge_comm_cmd_status status;
struct hclge_desc desc;
u16 resp;
@@ -90,7 +91,7 @@ static int hclge_send_mbx_msg(struct hclge_vport *vport, u8 *msg, u16 msg_len,
{
struct hclge_mbx_pf_to_vf_cmd *resp_pf_to_vf;
struct hclge_dev *hdev = vport->back;
- enum hclge_cmd_status status;
+ enum hclge_comm_cmd_status status;
struct hclge_desc desc;
resp_pf_to_vf = (struct hclge_mbx_pf_to_vf_cmd *)desc.data;
@@ -181,7 +182,7 @@ static int hclge_get_ring_chain_from_mbx(
if (req->msg.param[i].tqp_index >= vport->nic.kinfo.rss_size) {
dev_err(&hdev->pdev->dev, "tqp index(%u) is out of range(0-%u)\n",
req->msg.param[i].tqp_index,
- vport->nic.kinfo.rss_size - 1);
+ vport->nic.kinfo.rss_size - 1U);
return -EINVAL;
}
}
@@ -612,15 +613,17 @@ static void hclge_get_rss_key(struct hclge_vport *vport,
{
#define HCLGE_RSS_MBX_RESP_LEN 8
struct hclge_dev *hdev = vport->back;
+ struct hclge_comm_rss_cfg *rss_cfg;
u8 index;
index = mbx_req->msg.data[0];
+ rss_cfg = &hdev->rss_cfg;
/* Check the query index of rss_hash_key from VF, make sure no
* more than the size of rss_hash_key.
*/
if (((index + 1) * HCLGE_RSS_MBX_RESP_LEN) >
- sizeof(vport[0].rss_hash_key)) {
+ sizeof(rss_cfg->rss_hash_key)) {
dev_warn(&hdev->pdev->dev,
"failed to get the rss hash key, the index(%u) invalid !\n",
index);
@@ -628,7 +631,7 @@ static void hclge_get_rss_key(struct hclge_vport *vport,
}
memcpy(resp_msg->data,
- &hdev->vport[0].rss_hash_key[index * HCLGE_RSS_MBX_RESP_LEN],
+ &rss_cfg->rss_hash_key[index * HCLGE_RSS_MBX_RESP_LEN],
HCLGE_RSS_MBX_RESP_LEN);
resp_msg->len = HCLGE_RSS_MBX_RESP_LEN;
}
@@ -661,9 +664,9 @@ static void hclge_handle_link_change_event(struct hclge_dev *hdev,
static bool hclge_cmd_crq_empty(struct hclge_hw *hw)
{
- u32 tail = hclge_read_dev(hw, HCLGE_NIC_CRQ_TAIL_REG);
+ u32 tail = hclge_read_dev(hw, HCLGE_COMM_NIC_CRQ_TAIL_REG);
- return tail == hw->cmq.crq.next_to_use;
+ return tail == hw->hw.cmq.crq.next_to_use;
}
static void hclge_handle_ncsi_error(struct hclge_dev *hdev)
@@ -694,7 +697,7 @@ static void hclge_handle_vf_tbl(struct hclge_vport *vport,
void hclge_mbx_handler(struct hclge_dev *hdev)
{
- struct hclge_cmq_ring *crq = &hdev->hw.cmq.crq;
+ struct hclge_comm_cmq_ring *crq = &hdev->hw.hw.cmq.crq;
struct hclge_respond_to_vf_msg resp_msg;
struct hclge_mbx_vf_to_pf_cmd *req;
struct hclge_vport *vport;
@@ -705,7 +708,8 @@ void hclge_mbx_handler(struct hclge_dev *hdev)
/* handle all the mailbox requests in the queue */
while (!hclge_cmd_crq_empty(&hdev->hw)) {
- if (test_bit(HCLGE_STATE_CMD_DISABLE, &hdev->state)) {
+ if (test_bit(HCLGE_COMM_STATE_CMD_DISABLE,
+ &hdev->hw.hw.comm_state)) {
dev_warn(&hdev->pdev->dev,
"command queue needs re-initializing\n");
return;
@@ -848,6 +852,14 @@ void hclge_mbx_handler(struct hclge_dev *hdev)
if (hnae3_get_bit(req->mbx_need_resp, HCLGE_MBX_NEED_RESP_B) &&
req->msg.code < HCLGE_MBX_GET_VF_FLR_STATUS) {
resp_msg.status = ret;
+ if (time_is_before_jiffies(hdev->last_mbx_scheduled +
+ HCLGE_MBX_SCHED_TIMEOUT))
+ dev_warn(&hdev->pdev->dev,
+ "resp vport%u mbx(%u,%u) late\n",
+ req->mbx_src_vfid,
+ req->msg.code,
+ req->msg.subcode);
+
hclge_gen_resp_to_vf(vport, req, &resp_msg);
}
@@ -859,5 +871,6 @@ void hclge_mbx_handler(struct hclge_dev *hdev)
}
/* Write back CMDQ_RQ header pointer, M7 need this pointer */
- hclge_write_dev(&hdev->hw, HCLGE_NIC_CRQ_HEAD_REG, crq->next_to_use);
+ hclge_write_dev(&hdev->hw, HCLGE_COMM_NIC_CRQ_HEAD_REG,
+ crq->next_to_use);
}
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_mdio.c b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_mdio.c
index 1231c34..63d2be4 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_mdio.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_mdio.c
@@ -47,7 +47,7 @@ static int hclge_mdio_write(struct mii_bus *bus, int phyid, int regnum,
struct hclge_desc desc;
int ret;
- if (test_bit(HCLGE_STATE_CMD_DISABLE, &hdev->state))
+ if (test_bit(HCLGE_COMM_STATE_CMD_DISABLE, &hdev->hw.hw.comm_state))
return 0;
hclge_cmd_setup_basic_desc(&desc, HCLGE_OPC_MDIO_CONFIG, false);
@@ -85,7 +85,7 @@ static int hclge_mdio_read(struct mii_bus *bus, int phyid, int regnum)
struct hclge_desc desc;
int ret;
- if (test_bit(HCLGE_STATE_CMD_DISABLE, &hdev->state))
+ if (test_bit(HCLGE_COMM_STATE_CMD_DISABLE, &hdev->hw.hw.comm_state))
return 0;
hclge_cmd_setup_basic_desc(&desc, HCLGE_OPC_MDIO_CONFIG, true);
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_mdio.h b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_mdio.h
index fd0e201..4200d0b 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_mdio.h
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_mdio.h
@@ -4,6 +4,10 @@
#ifndef __HCLGE_MDIO_H
#define __HCLGE_MDIO_H
+#include "hnae3.h"
+
+struct hclge_dev;
+
int hclge_mac_mdio_config(struct hclge_dev *hdev);
int hclge_mac_connect_phy(struct hnae3_handle *handle);
void hclge_mac_disconnect_phy(struct hnae3_handle *handle);
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_ptp.c b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_ptp.c
index befa9bc..a40b158 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_ptp.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_ptp.c
@@ -464,7 +464,7 @@ static int hclge_ptp_create_clock(struct hclge_dev *hdev)
}
spin_lock_init(&ptp->lock);
- ptp->io_base = hdev->hw.io_base + HCLGE_PTP_REG_OFFSET;
+ ptp->io_base = hdev->hw.hw.io_base + HCLGE_PTP_REG_OFFSET;
ptp->ts_cfg.rx_filter = HWTSTAMP_FILTER_NONE;
ptp->ts_cfg.tx_type = HWTSTAMP_TX_OFF;
hdev->ptp = ptp;
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_ptp.h b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_ptp.h
index 7a9b77d..bbee74c 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_ptp.h
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_ptp.h
@@ -8,6 +8,9 @@
#include <linux/net_tstamp.h>
#include <linux/types.h>
+struct hclge_dev;
+struct ifreq;
+
#define HCLGE_PTP_REG_OFFSET 0x29000
#define HCLGE_PTP_TX_TS_SEQID_REG 0x0
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_tm.c b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_tm.c
index 429652a..089f444 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_tm.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_tm.c
@@ -678,8 +678,8 @@ static void hclge_tm_vport_tc_info_update(struct hclge_vport *vport)
hclge_tm_update_kinfo_rss_size(vport);
kinfo->num_tqps = hclge_vport_get_tqp_num(vport);
vport->dwrr = 100; /* 100 percent as init */
- vport->alloc_rss_size = kinfo->rss_size;
vport->bw_limit = hdev->tm_info.pg_info[0].bw_limit;
+ hdev->rss_cfg.rss_size = kinfo->rss_size;
/* when enable mqprio, the tc_info has been updated. */
if (kinfo->tc_info.mqprio_active)
@@ -916,37 +916,62 @@ static int hclge_vport_q_to_qs_map(struct hclge_dev *hdev,
return 0;
}
+static int hclge_tm_pri_q_qs_cfg_tc_base(struct hclge_dev *hdev)
+{
+ struct hclge_vport *vport = hdev->vport;
+ u16 i, k;
+ int ret;
+
+ /* Cfg qs -> pri mapping, one by one mapping */
+ for (k = 0; k < hdev->num_alloc_vport; k++) {
+ struct hnae3_knic_private_info *kinfo = &vport[k].nic.kinfo;
+
+ for (i = 0; i < kinfo->tc_info.num_tc; i++) {
+ ret = hclge_tm_qs_to_pri_map_cfg(hdev,
+ vport[k].qs_offset + i,
+ i);
+ if (ret)
+ return ret;
+ }
+ }
+
+ return 0;
+}
+
+static int hclge_tm_pri_q_qs_cfg_vnet_base(struct hclge_dev *hdev)
+{
+ struct hclge_vport *vport = hdev->vport;
+ u16 i, k;
+ int ret;
+
+ /* Cfg qs -> pri mapping, qs = tc, pri = vf, 8 qs -> 1 pri */
+ for (k = 0; k < hdev->num_alloc_vport; k++)
+ for (i = 0; i < HNAE3_MAX_TC; i++) {
+ ret = hclge_tm_qs_to_pri_map_cfg(hdev,
+ vport[k].qs_offset + i,
+ k);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
static int hclge_tm_pri_q_qs_cfg(struct hclge_dev *hdev)
{
struct hclge_vport *vport = hdev->vport;
int ret;
- u32 i, k;
+ u32 i;
- if (hdev->tx_sch_mode == HCLGE_FLAG_TC_BASE_SCH_MODE) {
- /* Cfg qs -> pri mapping, one by one mapping */
- for (k = 0; k < hdev->num_alloc_vport; k++) {
- struct hnae3_knic_private_info *kinfo =
- &vport[k].nic.kinfo;
-
- for (i = 0; i < kinfo->tc_info.num_tc; i++) {
- ret = hclge_tm_qs_to_pri_map_cfg(
- hdev, vport[k].qs_offset + i, i);
- if (ret)
- return ret;
- }
- }
- } else if (hdev->tx_sch_mode == HCLGE_FLAG_VNET_BASE_SCH_MODE) {
- /* Cfg qs -> pri mapping, qs = tc, pri = vf, 8 qs -> 1 pri */
- for (k = 0; k < hdev->num_alloc_vport; k++)
- for (i = 0; i < HNAE3_MAX_TC; i++) {
- ret = hclge_tm_qs_to_pri_map_cfg(
- hdev, vport[k].qs_offset + i, k);
- if (ret)
- return ret;
- }
- } else {
+ if (hdev->tx_sch_mode == HCLGE_FLAG_TC_BASE_SCH_MODE)
+ ret = hclge_tm_pri_q_qs_cfg_tc_base(hdev);
+ else if (hdev->tx_sch_mode == HCLGE_FLAG_VNET_BASE_SCH_MODE)
+ ret = hclge_tm_pri_q_qs_cfg_vnet_base(hdev);
+ else
return -EINVAL;
- }
+
+ if (ret)
+ return ret;
/* Cfg q -> qs mapping */
for (i = 0; i < hdev->num_alloc_vport; i++) {
@@ -1274,6 +1299,27 @@ static int hclge_tm_lvl2_schd_mode_cfg(struct hclge_dev *hdev)
return 0;
}
+static int hclge_tm_schd_mode_tc_base_cfg(struct hclge_dev *hdev, u8 pri_id)
+{
+ struct hclge_vport *vport = hdev->vport;
+ int ret;
+ u16 i;
+
+ ret = hclge_tm_pri_schd_mode_cfg(hdev, pri_id);
+ if (ret)
+ return ret;
+
+ for (i = 0; i < hdev->num_alloc_vport; i++) {
+ ret = hclge_tm_qs_schd_mode_cfg(hdev,
+ vport[i].qs_offset + pri_id,
+ HCLGE_SCH_MODE_DWRR);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
static int hclge_tm_schd_mode_vnet_base_cfg(struct hclge_vport *vport)
{
struct hnae3_knic_private_info *kinfo = &vport->nic.kinfo;
@@ -1304,21 +1350,13 @@ static int hclge_tm_lvl34_schd_mode_cfg(struct hclge_dev *hdev)
{
struct hclge_vport *vport = hdev->vport;
int ret;
- u8 i, k;
+ u8 i;
if (hdev->tx_sch_mode == HCLGE_FLAG_TC_BASE_SCH_MODE) {
for (i = 0; i < hdev->tm_info.num_tc; i++) {
- ret = hclge_tm_pri_schd_mode_cfg(hdev, i);
+ ret = hclge_tm_schd_mode_tc_base_cfg(hdev, i);
if (ret)
return ret;
-
- for (k = 0; k < hdev->num_alloc_vport; k++) {
- ret = hclge_tm_qs_schd_mode_cfg(
- hdev, vport[k].qs_offset + i,
- HCLGE_SCH_MODE_DWRR);
- if (ret)
- return ret;
- }
}
} else {
for (i = 0; i < hdev->num_alloc_vport; i++) {
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_tm.h b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_tm.h
index 1db7f40..619cc30 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_tm.h
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_tm.h
@@ -6,6 +6,12 @@
#include <linux/types.h>
+#include "hnae3.h"
+
+struct hclge_dev;
+struct hclge_vport;
+enum hclge_opcode_type;
+
/* MAC Pause */
#define HCLGE_TX_MAC_PAUSE_EN_MSK BIT(0)
#define HCLGE_RX_MAC_PAUSE_EN_MSK BIT(1)
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3vf/Makefile b/drivers/net/ethernet/hisilicon/hns3/hns3vf/Makefile
deleted file mode 100644
index 51ff7d8..0000000
--- a/drivers/net/ethernet/hisilicon/hns3/hns3vf/Makefile
+++ /dev/null
@@ -1,10 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0+
-#
-# Makefile for the HISILICON network device drivers.
-#
-
-ccflags-y := -I $(srctree)/drivers/net/ethernet/hisilicon/hns3
-ccflags-y += -I $(srctree)/$(src)
-
-obj-$(CONFIG_HNS3_HCLGEVF) += hclgevf.o
-hclgevf-objs = hclgevf_main.o hclgevf_cmd.o hclgevf_mbx.o hclgevf_devlink.o
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_cmd.c b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_cmd.c
deleted file mode 100644
index e605c2c..0000000
--- a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_cmd.c
+++ /dev/null
@@ -1,556 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0+
-// Copyright (c) 2016-2017 Hisilicon Limited.
-
-#include <linux/device.h>
-#include <linux/dma-direction.h>
-#include <linux/dma-mapping.h>
-#include <linux/err.h>
-#include <linux/pci.h>
-#include <linux/slab.h>
-#include "hclgevf_cmd.h"
-#include "hclgevf_main.h"
-#include "hnae3.h"
-
-#define cmq_ring_to_dev(ring) (&(ring)->dev->pdev->dev)
-
-static int hclgevf_ring_space(struct hclgevf_cmq_ring *ring)
-{
- int ntc = ring->next_to_clean;
- int ntu = ring->next_to_use;
- int used;
-
- used = (ntu - ntc + ring->desc_num) % ring->desc_num;
-
- return ring->desc_num - used - 1;
-}
-
-static int hclgevf_is_valid_csq_clean_head(struct hclgevf_cmq_ring *ring,
- int head)
-{
- int ntu = ring->next_to_use;
- int ntc = ring->next_to_clean;
-
- if (ntu > ntc)
- return head >= ntc && head <= ntu;
-
- return head >= ntc || head <= ntu;
-}
-
-static int hclgevf_cmd_csq_clean(struct hclgevf_hw *hw)
-{
- struct hclgevf_dev *hdev = container_of(hw, struct hclgevf_dev, hw);
- struct hclgevf_cmq_ring *csq = &hw->cmq.csq;
- int clean;
- u32 head;
-
- head = hclgevf_read_dev(hw, HCLGEVF_NIC_CSQ_HEAD_REG);
- rmb(); /* Make sure head is ready before touch any data */
-
- if (!hclgevf_is_valid_csq_clean_head(csq, head)) {
- dev_warn(&hdev->pdev->dev, "wrong cmd head (%u, %d-%d)\n", head,
- csq->next_to_use, csq->next_to_clean);
- dev_warn(&hdev->pdev->dev,
- "Disabling any further commands to IMP firmware\n");
- set_bit(HCLGEVF_STATE_CMD_DISABLE, &hdev->state);
- return -EIO;
- }
-
- clean = (head - csq->next_to_clean + csq->desc_num) % csq->desc_num;
- csq->next_to_clean = head;
- return clean;
-}
-
-static bool hclgevf_cmd_csq_done(struct hclgevf_hw *hw)
-{
- u32 head;
-
- head = hclgevf_read_dev(hw, HCLGEVF_NIC_CSQ_HEAD_REG);
-
- return head == hw->cmq.csq.next_to_use;
-}
-
-static bool hclgevf_is_special_opcode(u16 opcode)
-{
- const u16 spec_opcode[] = {0x30, 0x31, 0x32};
- int i;
-
- for (i = 0; i < ARRAY_SIZE(spec_opcode); i++) {
- if (spec_opcode[i] == opcode)
- return true;
- }
-
- return false;
-}
-
-static void hclgevf_cmd_config_regs(struct hclgevf_cmq_ring *ring)
-{
- struct hclgevf_dev *hdev = ring->dev;
- struct hclgevf_hw *hw = &hdev->hw;
- u32 reg_val;
-
- if (ring->flag == HCLGEVF_TYPE_CSQ) {
- reg_val = lower_32_bits(ring->desc_dma_addr);
- hclgevf_write_dev(hw, HCLGEVF_NIC_CSQ_BASEADDR_L_REG, reg_val);
- reg_val = upper_32_bits(ring->desc_dma_addr);
- hclgevf_write_dev(hw, HCLGEVF_NIC_CSQ_BASEADDR_H_REG, reg_val);
-
- reg_val = hclgevf_read_dev(hw, HCLGEVF_NIC_CSQ_DEPTH_REG);
- reg_val &= HCLGEVF_NIC_SW_RST_RDY;
- reg_val |= (ring->desc_num >> HCLGEVF_NIC_CMQ_DESC_NUM_S);
- hclgevf_write_dev(hw, HCLGEVF_NIC_CSQ_DEPTH_REG, reg_val);
-
- hclgevf_write_dev(hw, HCLGEVF_NIC_CSQ_HEAD_REG, 0);
- hclgevf_write_dev(hw, HCLGEVF_NIC_CSQ_TAIL_REG, 0);
- } else {
- reg_val = lower_32_bits(ring->desc_dma_addr);
- hclgevf_write_dev(hw, HCLGEVF_NIC_CRQ_BASEADDR_L_REG, reg_val);
- reg_val = upper_32_bits(ring->desc_dma_addr);
- hclgevf_write_dev(hw, HCLGEVF_NIC_CRQ_BASEADDR_H_REG, reg_val);
-
- reg_val = (ring->desc_num >> HCLGEVF_NIC_CMQ_DESC_NUM_S);
- hclgevf_write_dev(hw, HCLGEVF_NIC_CRQ_DEPTH_REG, reg_val);
-
- hclgevf_write_dev(hw, HCLGEVF_NIC_CRQ_HEAD_REG, 0);
- hclgevf_write_dev(hw, HCLGEVF_NIC_CRQ_TAIL_REG, 0);
- }
-}
-
-static void hclgevf_cmd_init_regs(struct hclgevf_hw *hw)
-{
- hclgevf_cmd_config_regs(&hw->cmq.csq);
- hclgevf_cmd_config_regs(&hw->cmq.crq);
-}
-
-static int hclgevf_alloc_cmd_desc(struct hclgevf_cmq_ring *ring)
-{
- int size = ring->desc_num * sizeof(struct hclgevf_desc);
-
- ring->desc = dma_alloc_coherent(cmq_ring_to_dev(ring), size,
- &ring->desc_dma_addr, GFP_KERNEL);
- if (!ring->desc)
- return -ENOMEM;
-
- return 0;
-}
-
-static void hclgevf_free_cmd_desc(struct hclgevf_cmq_ring *ring)
-{
- int size = ring->desc_num * sizeof(struct hclgevf_desc);
-
- if (ring->desc) {
- dma_free_coherent(cmq_ring_to_dev(ring), size,
- ring->desc, ring->desc_dma_addr);
- ring->desc = NULL;
- }
-}
-
-static int hclgevf_alloc_cmd_queue(struct hclgevf_dev *hdev, int ring_type)
-{
- struct hclgevf_hw *hw = &hdev->hw;
- struct hclgevf_cmq_ring *ring =
- (ring_type == HCLGEVF_TYPE_CSQ) ? &hw->cmq.csq : &hw->cmq.crq;
- int ret;
-
- ring->dev = hdev;
- ring->flag = ring_type;
-
- /* allocate CSQ/CRQ descriptor */
- ret = hclgevf_alloc_cmd_desc(ring);
- if (ret)
- dev_err(&hdev->pdev->dev, "failed(%d) to alloc %s desc\n", ret,
- (ring_type == HCLGEVF_TYPE_CSQ) ? "CSQ" : "CRQ");
-
- return ret;
-}
-
-void hclgevf_cmd_setup_basic_desc(struct hclgevf_desc *desc,
- enum hclgevf_opcode_type opcode, bool is_read)
-{
- memset(desc, 0, sizeof(struct hclgevf_desc));
- desc->opcode = cpu_to_le16(opcode);
- desc->flag = cpu_to_le16(HCLGEVF_CMD_FLAG_NO_INTR |
- HCLGEVF_CMD_FLAG_IN);
- if (is_read)
- desc->flag |= cpu_to_le16(HCLGEVF_CMD_FLAG_WR);
- else
- desc->flag &= cpu_to_le16(~HCLGEVF_CMD_FLAG_WR);
-}
-
-struct vf_errcode {
- u32 imp_errcode;
- int common_errno;
-};
-
-static void hclgevf_cmd_copy_desc(struct hclgevf_hw *hw,
- struct hclgevf_desc *desc, int num)
-{
- struct hclgevf_desc *desc_to_use;
- int handle = 0;
-
- while (handle < num) {
- desc_to_use = &hw->cmq.csq.desc[hw->cmq.csq.next_to_use];
- *desc_to_use = desc[handle];
- (hw->cmq.csq.next_to_use)++;
- if (hw->cmq.csq.next_to_use == hw->cmq.csq.desc_num)
- hw->cmq.csq.next_to_use = 0;
- handle++;
- }
-}
-
-static int hclgevf_cmd_convert_err_code(u16 desc_ret)
-{
- struct vf_errcode hclgevf_cmd_errcode[] = {
- {HCLGEVF_CMD_EXEC_SUCCESS, 0},
- {HCLGEVF_CMD_NO_AUTH, -EPERM},
- {HCLGEVF_CMD_NOT_SUPPORTED, -EOPNOTSUPP},
- {HCLGEVF_CMD_QUEUE_FULL, -EXFULL},
- {HCLGEVF_CMD_NEXT_ERR, -ENOSR},
- {HCLGEVF_CMD_UNEXE_ERR, -ENOTBLK},
- {HCLGEVF_CMD_PARA_ERR, -EINVAL},
- {HCLGEVF_CMD_RESULT_ERR, -ERANGE},
- {HCLGEVF_CMD_TIMEOUT, -ETIME},
- {HCLGEVF_CMD_HILINK_ERR, -ENOLINK},
- {HCLGEVF_CMD_QUEUE_ILLEGAL, -ENXIO},
- {HCLGEVF_CMD_INVALID, -EBADR},
- };
- u32 errcode_count = ARRAY_SIZE(hclgevf_cmd_errcode);
- u32 i;
-
- for (i = 0; i < errcode_count; i++)
- if (hclgevf_cmd_errcode[i].imp_errcode == desc_ret)
- return hclgevf_cmd_errcode[i].common_errno;
-
- return -EIO;
-}
-
-static int hclgevf_cmd_check_retval(struct hclgevf_hw *hw,
- struct hclgevf_desc *desc, int num, int ntc)
-{
- u16 opcode, desc_ret;
- int handle;
-
- opcode = le16_to_cpu(desc[0].opcode);
- for (handle = 0; handle < num; handle++) {
- /* Get the result of hardware write back */
- desc[handle] = hw->cmq.csq.desc[ntc];
- ntc++;
- if (ntc == hw->cmq.csq.desc_num)
- ntc = 0;
- }
- if (likely(!hclgevf_is_special_opcode(opcode)))
- desc_ret = le16_to_cpu(desc[num - 1].retval);
- else
- desc_ret = le16_to_cpu(desc[0].retval);
- hw->cmq.last_status = desc_ret;
-
- return hclgevf_cmd_convert_err_code(desc_ret);
-}
-
-static int hclgevf_cmd_check_result(struct hclgevf_hw *hw,
- struct hclgevf_desc *desc, int num, int ntc)
-{
- struct hclgevf_dev *hdev = (struct hclgevf_dev *)hw->hdev;
- bool is_completed = false;
- u32 timeout = 0;
- int handle, ret;
-
- /* If the command is sync, wait for the firmware to write back,
- * if multi descriptors to be sent, use the first one to check
- */
- if (HCLGEVF_SEND_SYNC(le16_to_cpu(desc->flag))) {
- do {
- if (hclgevf_cmd_csq_done(hw)) {
- is_completed = true;
- break;
- }
- udelay(1);
- timeout++;
- } while (timeout < hw->cmq.tx_timeout);
- }
-
- if (!is_completed)
- ret = -EBADE;
- else
- ret = hclgevf_cmd_check_retval(hw, desc, num, ntc);
-
- /* Clean the command send queue */
- handle = hclgevf_cmd_csq_clean(hw);
- if (handle < 0)
- ret = handle;
- else if (handle != num)
- dev_warn(&hdev->pdev->dev,
- "cleaned %d, need to clean %d\n", handle, num);
- return ret;
-}
-
-/* hclgevf_cmd_send - send command to command queue
- * @hw: pointer to the hw struct
- * @desc: prefilled descriptor for describing the command
- * @num : the number of descriptors to be sent
- *
- * This is the main send command for command queue, it
- * sends the queue, cleans the queue, etc
- */
-int hclgevf_cmd_send(struct hclgevf_hw *hw, struct hclgevf_desc *desc, int num)
-{
- struct hclgevf_dev *hdev = (struct hclgevf_dev *)hw->hdev;
- struct hclgevf_cmq_ring *csq = &hw->cmq.csq;
- int ret;
- int ntc;
-
- spin_lock_bh(&hw->cmq.csq.lock);
-
- if (test_bit(HCLGEVF_STATE_CMD_DISABLE, &hdev->state)) {
- spin_unlock_bh(&hw->cmq.csq.lock);
- return -EBUSY;
- }
-
- if (num > hclgevf_ring_space(&hw->cmq.csq)) {
- /* If CMDQ ring is full, SW HEAD and HW HEAD may be different,
- * need update the SW HEAD pointer csq->next_to_clean
- */
- csq->next_to_clean = hclgevf_read_dev(hw,
- HCLGEVF_NIC_CSQ_HEAD_REG);
- spin_unlock_bh(&hw->cmq.csq.lock);
- return -EBUSY;
- }
-
- /* Record the location of desc in the ring for this time
- * which will be use for hardware to write back
- */
- ntc = hw->cmq.csq.next_to_use;
-
- hclgevf_cmd_copy_desc(hw, desc, num);
-
- /* Write to hardware */
- hclgevf_write_dev(hw, HCLGEVF_NIC_CSQ_TAIL_REG,
- hw->cmq.csq.next_to_use);
-
- ret = hclgevf_cmd_check_result(hw, desc, num, ntc);
-
- spin_unlock_bh(&hw->cmq.csq.lock);
-
- return ret;
-}
-
-static void hclgevf_set_default_capability(struct hclgevf_dev *hdev)
-{
- struct hnae3_ae_dev *ae_dev = pci_get_drvdata(hdev->pdev);
-
- set_bit(HNAE3_DEV_SUPPORT_FD_B, ae_dev->caps);
- set_bit(HNAE3_DEV_SUPPORT_GRO_B, ae_dev->caps);
- set_bit(HNAE3_DEV_SUPPORT_FEC_B, ae_dev->caps);
-}
-
-static const struct hclgevf_caps_bit_map hclgevf_cmd_caps_bit_map0[] = {
- {HCLGEVF_CAP_UDP_GSO_B, HNAE3_DEV_SUPPORT_UDP_GSO_B},
- {HCLGEVF_CAP_INT_QL_B, HNAE3_DEV_SUPPORT_INT_QL_B},
- {HCLGEVF_CAP_TQP_TXRX_INDEP_B, HNAE3_DEV_SUPPORT_TQP_TXRX_INDEP_B},
- {HCLGEVF_CAP_HW_TX_CSUM_B, HNAE3_DEV_SUPPORT_HW_TX_CSUM_B},
- {HCLGEVF_CAP_UDP_TUNNEL_CSUM_B, HNAE3_DEV_SUPPORT_UDP_TUNNEL_CSUM_B},
- {HCLGEVF_CAP_RXD_ADV_LAYOUT_B, HNAE3_DEV_SUPPORT_RXD_ADV_LAYOUT_B},
-};
-
-static void hclgevf_parse_capability(struct hclgevf_dev *hdev,
- struct hclgevf_query_version_cmd *cmd)
-{
- struct hnae3_ae_dev *ae_dev = pci_get_drvdata(hdev->pdev);
- u32 caps, i;
-
- caps = __le32_to_cpu(cmd->caps[0]);
- for (i = 0; i < ARRAY_SIZE(hclgevf_cmd_caps_bit_map0); i++)
- if (hnae3_get_bit(caps, hclgevf_cmd_caps_bit_map0[i].imp_bit))
- set_bit(hclgevf_cmd_caps_bit_map0[i].local_bit,
- ae_dev->caps);
-}
-
-static __le32 hclgevf_build_api_caps(void)
-{
- u32 api_caps = 0;
-
- hnae3_set_bit(api_caps, HCLGEVF_API_CAP_FLEX_RSS_TBL_B, 1);
-
- return cpu_to_le32(api_caps);
-}
-
-static int hclgevf_cmd_query_version_and_capability(struct hclgevf_dev *hdev)
-{
- struct hnae3_ae_dev *ae_dev = pci_get_drvdata(hdev->pdev);
- struct hclgevf_query_version_cmd *resp;
- struct hclgevf_desc desc;
- int status;
-
- resp = (struct hclgevf_query_version_cmd *)desc.data;
-
- hclgevf_cmd_setup_basic_desc(&desc, HCLGEVF_OPC_QUERY_FW_VER, 1);
- resp->api_caps = hclgevf_build_api_caps();
- status = hclgevf_cmd_send(&hdev->hw, &desc, 1);
- if (status)
- return status;
-
- hdev->fw_version = le32_to_cpu(resp->firmware);
-
- ae_dev->dev_version = le32_to_cpu(resp->hardware) <<
- HNAE3_PCI_REVISION_BIT_SIZE;
- ae_dev->dev_version |= hdev->pdev->revision;
-
- if (ae_dev->dev_version >= HNAE3_DEVICE_VERSION_V2)
- hclgevf_set_default_capability(hdev);
-
- hclgevf_parse_capability(hdev, resp);
-
- return status;
-}
-
-int hclgevf_cmd_queue_init(struct hclgevf_dev *hdev)
-{
- int ret;
-
- /* Setup the lock for command queue */
- spin_lock_init(&hdev->hw.cmq.csq.lock);
- spin_lock_init(&hdev->hw.cmq.crq.lock);
-
- hdev->hw.cmq.tx_timeout = HCLGEVF_CMDQ_TX_TIMEOUT;
- hdev->hw.cmq.csq.desc_num = HCLGEVF_NIC_CMQ_DESC_NUM;
- hdev->hw.cmq.crq.desc_num = HCLGEVF_NIC_CMQ_DESC_NUM;
-
- ret = hclgevf_alloc_cmd_queue(hdev, HCLGEVF_TYPE_CSQ);
- if (ret) {
- dev_err(&hdev->pdev->dev,
- "CSQ ring setup error %d\n", ret);
- return ret;
- }
-
- ret = hclgevf_alloc_cmd_queue(hdev, HCLGEVF_TYPE_CRQ);
- if (ret) {
- dev_err(&hdev->pdev->dev,
- "CRQ ring setup error %d\n", ret);
- goto err_csq;
- }
-
- return 0;
-err_csq:
- hclgevf_free_cmd_desc(&hdev->hw.cmq.csq);
- return ret;
-}
-
-static int hclgevf_firmware_compat_config(struct hclgevf_dev *hdev, bool en)
-{
- struct hclgevf_firmware_compat_cmd *req;
- struct hclgevf_desc desc;
- u32 compat = 0;
-
- hclgevf_cmd_setup_basic_desc(&desc, HCLGEVF_OPC_IMP_COMPAT_CFG, false);
-
- if (en) {
- req = (struct hclgevf_firmware_compat_cmd *)desc.data;
-
- hnae3_set_bit(compat, HCLGEVF_SYNC_RX_RING_HEAD_EN_B, 1);
-
- req->compat = cpu_to_le32(compat);
- }
-
- return hclgevf_cmd_send(&hdev->hw, &desc, 1);
-}
-
-int hclgevf_cmd_init(struct hclgevf_dev *hdev)
-{
- struct hnae3_ae_dev *ae_dev = pci_get_drvdata(hdev->pdev);
- int ret;
-
- spin_lock_bh(&hdev->hw.cmq.csq.lock);
- spin_lock(&hdev->hw.cmq.crq.lock);
-
- /* initialize the pointers of async rx queue of mailbox */
- hdev->arq.hdev = hdev;
- hdev->arq.head = 0;
- hdev->arq.tail = 0;
- atomic_set(&hdev->arq.count, 0);
- hdev->hw.cmq.csq.next_to_clean = 0;
- hdev->hw.cmq.csq.next_to_use = 0;
- hdev->hw.cmq.crq.next_to_clean = 0;
- hdev->hw.cmq.crq.next_to_use = 0;
-
- hclgevf_cmd_init_regs(&hdev->hw);
-
- spin_unlock(&hdev->hw.cmq.crq.lock);
- spin_unlock_bh(&hdev->hw.cmq.csq.lock);
-
- clear_bit(HCLGEVF_STATE_CMD_DISABLE, &hdev->state);
-
- /* Check if there is new reset pending, because the higher level
- * reset may happen when lower level reset is being processed.
- */
- if (hclgevf_is_reset_pending(hdev)) {
- ret = -EBUSY;
- goto err_cmd_init;
- }
-
- /* get version and device capabilities */
- ret = hclgevf_cmd_query_version_and_capability(hdev);
- if (ret) {
- dev_err(&hdev->pdev->dev,
- "failed to query version and capabilities, ret = %d\n", ret);
- goto err_cmd_init;
- }
-
- dev_info(&hdev->pdev->dev, "The firmware version is %lu.%lu.%lu.%lu\n",
- hnae3_get_field(hdev->fw_version, HNAE3_FW_VERSION_BYTE3_MASK,
- HNAE3_FW_VERSION_BYTE3_SHIFT),
- hnae3_get_field(hdev->fw_version, HNAE3_FW_VERSION_BYTE2_MASK,
- HNAE3_FW_VERSION_BYTE2_SHIFT),
- hnae3_get_field(hdev->fw_version, HNAE3_FW_VERSION_BYTE1_MASK,
- HNAE3_FW_VERSION_BYTE1_SHIFT),
- hnae3_get_field(hdev->fw_version, HNAE3_FW_VERSION_BYTE0_MASK,
- HNAE3_FW_VERSION_BYTE0_SHIFT));
-
- if (ae_dev->dev_version >= HNAE3_DEVICE_VERSION_V3) {
- /* ask the firmware to enable some features, driver can work
- * without it.
- */
- ret = hclgevf_firmware_compat_config(hdev, true);
- if (ret)
- dev_warn(&hdev->pdev->dev,
- "Firmware compatible features not enabled(%d).\n",
- ret);
- }
-
- return 0;
-
-err_cmd_init:
- set_bit(HCLGEVF_STATE_CMD_DISABLE, &hdev->state);
-
- return ret;
-}
-
-static void hclgevf_cmd_uninit_regs(struct hclgevf_hw *hw)
-{
- hclgevf_write_dev(hw, HCLGEVF_NIC_CSQ_BASEADDR_L_REG, 0);
- hclgevf_write_dev(hw, HCLGEVF_NIC_CSQ_BASEADDR_H_REG, 0);
- hclgevf_write_dev(hw, HCLGEVF_NIC_CSQ_DEPTH_REG, 0);
- hclgevf_write_dev(hw, HCLGEVF_NIC_CSQ_HEAD_REG, 0);
- hclgevf_write_dev(hw, HCLGEVF_NIC_CSQ_TAIL_REG, 0);
- hclgevf_write_dev(hw, HCLGEVF_NIC_CRQ_BASEADDR_L_REG, 0);
- hclgevf_write_dev(hw, HCLGEVF_NIC_CRQ_BASEADDR_H_REG, 0);
- hclgevf_write_dev(hw, HCLGEVF_NIC_CRQ_DEPTH_REG, 0);
- hclgevf_write_dev(hw, HCLGEVF_NIC_CRQ_HEAD_REG, 0);
- hclgevf_write_dev(hw, HCLGEVF_NIC_CRQ_TAIL_REG, 0);
-}
-
-void hclgevf_cmd_uninit(struct hclgevf_dev *hdev)
-{
- hclgevf_firmware_compat_config(hdev, false);
- set_bit(HCLGEVF_STATE_CMD_DISABLE, &hdev->state);
- /* wait to ensure that the firmware completes the possible left
- * over commands.
- */
- msleep(HCLGEVF_CMDQ_CLEAR_WAIT_TIME);
- spin_lock_bh(&hdev->hw.cmq.csq.lock);
- spin_lock(&hdev->hw.cmq.crq.lock);
- hclgevf_cmd_uninit_regs(&hdev->hw);
- spin_unlock(&hdev->hw.cmq.crq.lock);
- spin_unlock_bh(&hdev->hw.cmq.csq.lock);
-
- hclgevf_free_cmd_desc(&hdev->hw.cmq.csq);
- hclgevf_free_cmd_desc(&hdev->hw.cmq.crq);
-}
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_cmd.h b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_cmd.h
index edc9e15..537b887 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_cmd.h
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_cmd.h
@@ -6,9 +6,8 @@
#include <linux/io.h>
#include <linux/types.h>
#include "hnae3.h"
+#include "hclge_comm_cmd.h"
-#define HCLGEVF_CMDQ_TX_TIMEOUT 30000
-#define HCLGEVF_CMDQ_CLEAR_WAIT_TIME 200
#define HCLGEVF_CMDQ_RX_INVLD_B 0
#define HCLGEVF_CMDQ_RX_OUTVLD_B 1
@@ -16,107 +15,6 @@ struct hclgevf_hw;
struct hclgevf_dev;
#define HCLGEVF_SYNC_RX_RING_HEAD_EN_B 4
-struct hclgevf_firmware_compat_cmd {
- __le32 compat;
- u8 rsv[20];
-};
-
-struct hclgevf_desc {
- __le16 opcode;
- __le16 flag;
- __le16 retval;
- __le16 rsv;
- __le32 data[6];
-};
-
-struct hclgevf_desc_cb {
- dma_addr_t dma;
- void *va;
- u32 length;
-};
-
-struct hclgevf_cmq_ring {
- dma_addr_t desc_dma_addr;
- struct hclgevf_desc *desc;
- struct hclgevf_desc_cb *desc_cb;
- struct hclgevf_dev *dev;
- u32 head;
- u32 tail;
-
- u16 buf_size;
- u16 desc_num;
- int next_to_use;
- int next_to_clean;
- u8 flag;
- spinlock_t lock; /* Command queue lock */
-};
-
-enum hclgevf_cmd_return_status {
- HCLGEVF_CMD_EXEC_SUCCESS = 0,
- HCLGEVF_CMD_NO_AUTH = 1,
- HCLGEVF_CMD_NOT_SUPPORTED = 2,
- HCLGEVF_CMD_QUEUE_FULL = 3,
- HCLGEVF_CMD_NEXT_ERR = 4,
- HCLGEVF_CMD_UNEXE_ERR = 5,
- HCLGEVF_CMD_PARA_ERR = 6,
- HCLGEVF_CMD_RESULT_ERR = 7,
- HCLGEVF_CMD_TIMEOUT = 8,
- HCLGEVF_CMD_HILINK_ERR = 9,
- HCLGEVF_CMD_QUEUE_ILLEGAL = 10,
- HCLGEVF_CMD_INVALID = 11,
-};
-
-enum hclgevf_cmd_status {
- HCLGEVF_STATUS_SUCCESS = 0,
- HCLGEVF_ERR_CSQ_FULL = -1,
- HCLGEVF_ERR_CSQ_TIMEOUT = -2,
- HCLGEVF_ERR_CSQ_ERROR = -3
-};
-
-struct hclgevf_cmq {
- struct hclgevf_cmq_ring csq;
- struct hclgevf_cmq_ring crq;
- u16 tx_timeout; /* Tx timeout */
- enum hclgevf_cmd_status last_status;
-};
-
-#define HCLGEVF_CMD_FLAG_IN_VALID_SHIFT 0
-#define HCLGEVF_CMD_FLAG_OUT_VALID_SHIFT 1
-#define HCLGEVF_CMD_FLAG_NEXT_SHIFT 2
-#define HCLGEVF_CMD_FLAG_WR_OR_RD_SHIFT 3
-#define HCLGEVF_CMD_FLAG_NO_INTR_SHIFT 4
-#define HCLGEVF_CMD_FLAG_ERR_INTR_SHIFT 5
-
-#define HCLGEVF_CMD_FLAG_IN BIT(HCLGEVF_CMD_FLAG_IN_VALID_SHIFT)
-#define HCLGEVF_CMD_FLAG_OUT BIT(HCLGEVF_CMD_FLAG_OUT_VALID_SHIFT)
-#define HCLGEVF_CMD_FLAG_NEXT BIT(HCLGEVF_CMD_FLAG_NEXT_SHIFT)
-#define HCLGEVF_CMD_FLAG_WR BIT(HCLGEVF_CMD_FLAG_WR_OR_RD_SHIFT)
-#define HCLGEVF_CMD_FLAG_NO_INTR BIT(HCLGEVF_CMD_FLAG_NO_INTR_SHIFT)
-#define HCLGEVF_CMD_FLAG_ERR_INTR BIT(HCLGEVF_CMD_FLAG_ERR_INTR_SHIFT)
-
-enum hclgevf_opcode_type {
- /* Generic command */
- HCLGEVF_OPC_QUERY_FW_VER = 0x0001,
- HCLGEVF_OPC_QUERY_VF_RSRC = 0x0024,
- HCLGEVF_OPC_QUERY_DEV_SPECS = 0x0050,
-
- /* TQP command */
- HCLGEVF_OPC_QUERY_TX_STATUS = 0x0B03,
- HCLGEVF_OPC_QUERY_RX_STATUS = 0x0B13,
- HCLGEVF_OPC_CFG_COM_TQP_QUEUE = 0x0B20,
- /* GRO command */
- HCLGEVF_OPC_GRO_GENERIC_CONFIG = 0x0C10,
- /* RSS cmd */
- HCLGEVF_OPC_RSS_GENERIC_CONFIG = 0x0D01,
- HCLGEVF_OPC_RSS_INPUT_TUPLE = 0x0D02,
- HCLGEVF_OPC_RSS_INDIR_TABLE = 0x0D07,
- HCLGEVF_OPC_RSS_TC_MODE = 0x0D08,
- /* Mailbox cmd */
- HCLGEVF_OPC_MBX_VF_TO_PF = 0x2001,
-
- /* IMP stats command */
- HCLGEVF_OPC_IMP_COMPAT_CFG = 0x701A,
-};
#define HCLGEVF_TQP_REG_OFFSET 0x80000
#define HCLGEVF_TQP_REG_SIZE 0x200
@@ -156,34 +54,6 @@ struct hclgevf_ctrl_vector_chain {
u8 resv;
};
-enum HCLGEVF_CAP_BITS {
- HCLGEVF_CAP_UDP_GSO_B,
- HCLGEVF_CAP_QB_B,
- HCLGEVF_CAP_FD_FORWARD_TC_B,
- HCLGEVF_CAP_PTP_B,
- HCLGEVF_CAP_INT_QL_B,
- HCLGEVF_CAP_HW_TX_CSUM_B,
- HCLGEVF_CAP_TX_PUSH_B,
- HCLGEVF_CAP_PHY_IMP_B,
- HCLGEVF_CAP_TQP_TXRX_INDEP_B,
- HCLGEVF_CAP_HW_PAD_B,
- HCLGEVF_CAP_STASH_B,
- HCLGEVF_CAP_UDP_TUNNEL_CSUM_B,
- HCLGEVF_CAP_RXD_ADV_LAYOUT_B = 15,
-};
-
-enum HCLGEVF_API_CAP_BITS {
- HCLGEVF_API_CAP_FLEX_RSS_TBL_B,
-};
-
-#define HCLGEVF_QUERY_CAP_LENGTH 3
-struct hclgevf_query_version_cmd {
- __le32 firmware;
- __le32 hardware;
- __le32 api_caps;
- __le32 caps[HCLGEVF_QUERY_CAP_LENGTH]; /* capabilities of device */
-};
-
#define HCLGEVF_MSIX_OFT_ROCEE_S 0
#define HCLGEVF_MSIX_OFT_ROCEE_M (0xffff << HCLGEVF_MSIX_OFT_ROCEE_S)
#define HCLGEVF_VEC_NUM_S 0
@@ -203,50 +73,6 @@ struct hclgevf_cfg_gro_status_cmd {
u8 rsv[23];
};
-#define HCLGEVF_RSS_DEFAULT_OUTPORT_B 4
-#define HCLGEVF_RSS_HASH_KEY_OFFSET_B 4
-#define HCLGEVF_RSS_HASH_KEY_NUM 16
-struct hclgevf_rss_config_cmd {
- u8 hash_config;
- u8 rsv[7];
- u8 hash_key[HCLGEVF_RSS_HASH_KEY_NUM];
-};
-
-struct hclgevf_rss_input_tuple_cmd {
- u8 ipv4_tcp_en;
- u8 ipv4_udp_en;
- u8 ipv4_sctp_en;
- u8 ipv4_fragment_en;
- u8 ipv6_tcp_en;
- u8 ipv6_udp_en;
- u8 ipv6_sctp_en;
- u8 ipv6_fragment_en;
- u8 rsv[16];
-};
-
-#define HCLGEVF_RSS_CFG_TBL_SIZE 16
-
-struct hclgevf_rss_indirection_table_cmd {
- __le16 start_table_index;
- __le16 rss_set_bitmap;
- u8 rsv[4];
- u8 rss_result[HCLGEVF_RSS_CFG_TBL_SIZE];
-};
-
-#define HCLGEVF_RSS_TC_OFFSET_S 0
-#define HCLGEVF_RSS_TC_OFFSET_M GENMASK(10, 0)
-#define HCLGEVF_RSS_TC_SIZE_MSB_B 11
-#define HCLGEVF_RSS_TC_SIZE_S 12
-#define HCLGEVF_RSS_TC_SIZE_M GENMASK(14, 12)
-#define HCLGEVF_RSS_TC_VALID_B 15
-#define HCLGEVF_MAX_TC_NUM 8
-#define HCLGEVF_RSS_TC_SIZE_MSB_OFFSET 3
-
-struct hclgevf_rss_tc_mode_cmd {
- __le16 rss_tc_mode[HCLGEVF_MAX_TC_NUM];
- u8 rsv[8];
-};
-
#define HCLGEVF_LINK_STS_B 0
#define HCLGEVF_LINK_STATUS BIT(HCLGEVF_LINK_STS_B)
struct hclgevf_link_status_cmd {
@@ -273,9 +99,6 @@ struct hclgevf_cfg_tx_queue_pointer_cmd {
u8 rsv[14];
};
-#define HCLGEVF_TYPE_CRQ 0
-#define HCLGEVF_TYPE_CSQ 1
-
/* this bit indicates that the driver is ready for hardware reset */
#define HCLGEVF_NIC_SW_RST_RDY_B 16
#define HCLGEVF_NIC_SW_RST_RDY BIT(HCLGEVF_NIC_SW_RST_RDY_B)
@@ -285,6 +108,9 @@ struct hclgevf_cfg_tx_queue_pointer_cmd {
#define HCLGEVF_QUERY_DEV_SPECS_BD_NUM 4
+#define hclgevf_cmd_setup_basic_desc(desc, opcode, is_read) \
+ hclge_comm_cmd_setup_basic_desc(desc, opcode, is_read)
+
struct hclgevf_dev_specs_0_cmd {
__le32 rsv0;
__le32 mac_entry_num;
@@ -305,38 +131,6 @@ struct hclgevf_dev_specs_1_cmd {
u8 rsv1[18];
};
-/* capabilities bits map between imp firmware and local driver */
-struct hclgevf_caps_bit_map {
- u16 imp_bit;
- u16 local_bit;
-};
-
-static inline void hclgevf_write_reg(void __iomem *base, u32 reg, u32 value)
-{
- writel(value, base + reg);
-}
-
-static inline u32 hclgevf_read_reg(u8 __iomem *base, u32 reg)
-{
- u8 __iomem *reg_addr = READ_ONCE(base);
-
- return readl(reg_addr + reg);
-}
-
-#define hclgevf_write_dev(a, reg, value) \
- hclgevf_write_reg((a)->io_base, reg, value)
-#define hclgevf_read_dev(a, reg) \
- hclgevf_read_reg((a)->io_base, reg)
-
-#define HCLGEVF_SEND_SYNC(flag) \
- ((flag) & HCLGEVF_CMD_FLAG_NO_INTR)
-
-int hclgevf_cmd_init(struct hclgevf_dev *hdev);
-void hclgevf_cmd_uninit(struct hclgevf_dev *hdev);
-int hclgevf_cmd_queue_init(struct hclgevf_dev *hdev);
-
-int hclgevf_cmd_send(struct hclgevf_hw *hw, struct hclgevf_desc *desc, int num);
-void hclgevf_cmd_setup_basic_desc(struct hclgevf_desc *desc,
- enum hclgevf_opcode_type opcode,
- bool is_read);
+int hclgevf_cmd_send(struct hclgevf_hw *hw, struct hclge_desc *desc, int num);
+void hclgevf_arq_init(struct hclgevf_dev *hdev);
#endif
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.c b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.c
index 41afaee..7df8761 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.c
@@ -9,6 +9,7 @@
#include "hclge_mbx.h"
#include "hnae3.h"
#include "hclgevf_devlink.h"
+#include "hclge_comm_rss.h"
#define HCLGEVF_NAME "hclgevf"
@@ -30,30 +31,22 @@ static const struct pci_device_id ae_algovf_pci_tbl[] = {
{0, }
};
-static const u8 hclgevf_hash_key[] = {
- 0x6D, 0x5A, 0x56, 0xDA, 0x25, 0x5B, 0x0E, 0xC2,
- 0x41, 0x67, 0x25, 0x3D, 0x43, 0xA3, 0x8F, 0xB0,
- 0xD0, 0xCA, 0x2B, 0xCB, 0xAE, 0x7B, 0x30, 0xB4,
- 0x77, 0xCB, 0x2D, 0xA3, 0x80, 0x30, 0xF2, 0x0C,
- 0x6A, 0x42, 0xB7, 0x3B, 0xBE, 0xAC, 0x01, 0xFA
-};
-
MODULE_DEVICE_TABLE(pci, ae_algovf_pci_tbl);
-static const u32 cmdq_reg_addr_list[] = {HCLGEVF_NIC_CSQ_BASEADDR_L_REG,
- HCLGEVF_NIC_CSQ_BASEADDR_H_REG,
- HCLGEVF_NIC_CSQ_DEPTH_REG,
- HCLGEVF_NIC_CSQ_TAIL_REG,
- HCLGEVF_NIC_CSQ_HEAD_REG,
- HCLGEVF_NIC_CRQ_BASEADDR_L_REG,
- HCLGEVF_NIC_CRQ_BASEADDR_H_REG,
- HCLGEVF_NIC_CRQ_DEPTH_REG,
- HCLGEVF_NIC_CRQ_TAIL_REG,
- HCLGEVF_NIC_CRQ_HEAD_REG,
- HCLGEVF_VECTOR0_CMDQ_SRC_REG,
- HCLGEVF_VECTOR0_CMDQ_STATE_REG,
- HCLGEVF_CMDQ_INTR_EN_REG,
- HCLGEVF_CMDQ_INTR_GEN_REG};
+static const u32 cmdq_reg_addr_list[] = {HCLGE_COMM_NIC_CSQ_BASEADDR_L_REG,
+ HCLGE_COMM_NIC_CSQ_BASEADDR_H_REG,
+ HCLGE_COMM_NIC_CSQ_DEPTH_REG,
+ HCLGE_COMM_NIC_CSQ_TAIL_REG,
+ HCLGE_COMM_NIC_CSQ_HEAD_REG,
+ HCLGE_COMM_NIC_CRQ_BASEADDR_L_REG,
+ HCLGE_COMM_NIC_CRQ_BASEADDR_H_REG,
+ HCLGE_COMM_NIC_CRQ_DEPTH_REG,
+ HCLGE_COMM_NIC_CRQ_TAIL_REG,
+ HCLGE_COMM_NIC_CRQ_HEAD_REG,
+ HCLGE_COMM_VECTOR0_CMDQ_SRC_REG,
+ HCLGE_COMM_VECTOR0_CMDQ_STATE_REG,
+ HCLGE_COMM_CMDQ_INTR_EN_REG,
+ HCLGE_COMM_CMDQ_INTR_GEN_REG};
static const u32 common_reg_addr_list[] = {HCLGEVF_MISC_VECTOR_REG_BASE,
HCLGEVF_RST_ING,
@@ -92,6 +85,32 @@ static const u32 tqp_intr_reg_addr_list[] = {HCLGEVF_TQP_INTR_CTRL_REG,
HCLGEVF_TQP_INTR_GL2_REG,
HCLGEVF_TQP_INTR_RL_REG};
+/* hclgevf_cmd_send - send command to command queue
+ * @hw: pointer to the hw struct
+ * @desc: prefilled descriptor for describing the command
+ * @num : the number of descriptors to be sent
+ *
+ * This is the main send command for command queue, it
+ * sends the queue, cleans the queue, etc
+ */
+int hclgevf_cmd_send(struct hclgevf_hw *hw, struct hclge_desc *desc, int num)
+{
+ return hclge_comm_cmd_send(&hw->hw, desc, num);
+}
+
+void hclgevf_arq_init(struct hclgevf_dev *hdev)
+{
+ struct hclge_comm_cmq *cmdq = &hdev->hw.hw.cmq;
+
+ spin_lock(&cmdq->crq.lock);
+ /* initialize the pointers of async rx queue of mailbox */
+ hdev->arq.hdev = hdev;
+ hdev->arq.head = 0;
+ hdev->arq.tail = 0;
+ atomic_set(&hdev->arq.count, 0);
+ spin_unlock(&cmdq->crq.lock);
+}
+
static struct hclgevf_dev *hclgevf_ae_get_hdev(struct hnae3_handle *handle)
{
if (!handle->client)
@@ -102,108 +121,13 @@ static struct hclgevf_dev *hclgevf_ae_get_hdev(struct hnae3_handle *handle)
return container_of(handle, struct hclgevf_dev, nic);
}
-static int hclgevf_tqps_update_stats(struct hnae3_handle *handle)
-{
- struct hnae3_knic_private_info *kinfo = &handle->kinfo;
- struct hclgevf_dev *hdev = hclgevf_ae_get_hdev(handle);
- struct hclgevf_desc desc;
- struct hclgevf_tqp *tqp;
- int status;
- int i;
-
- for (i = 0; i < kinfo->num_tqps; i++) {
- tqp = container_of(kinfo->tqp[i], struct hclgevf_tqp, q);
- hclgevf_cmd_setup_basic_desc(&desc,
- HCLGEVF_OPC_QUERY_RX_STATUS,
- true);
-
- desc.data[0] = cpu_to_le32(tqp->index & 0x1ff);
- status = hclgevf_cmd_send(&hdev->hw, &desc, 1);
- if (status) {
- dev_err(&hdev->pdev->dev,
- "Query tqp stat fail, status = %d,queue = %d\n",
- status, i);
- return status;
- }
- tqp->tqp_stats.rcb_rx_ring_pktnum_rcd +=
- le32_to_cpu(desc.data[1]);
-
- hclgevf_cmd_setup_basic_desc(&desc, HCLGEVF_OPC_QUERY_TX_STATUS,
- true);
-
- desc.data[0] = cpu_to_le32(tqp->index & 0x1ff);
- status = hclgevf_cmd_send(&hdev->hw, &desc, 1);
- if (status) {
- dev_err(&hdev->pdev->dev,
- "Query tqp stat fail, status = %d,queue = %d\n",
- status, i);
- return status;
- }
- tqp->tqp_stats.rcb_tx_ring_pktnum_rcd +=
- le32_to_cpu(desc.data[1]);
- }
-
- return 0;
-}
-
-static u64 *hclgevf_tqps_get_stats(struct hnae3_handle *handle, u64 *data)
-{
- struct hnae3_knic_private_info *kinfo = &handle->kinfo;
- struct hclgevf_tqp *tqp;
- u64 *buff = data;
- int i;
-
- for (i = 0; i < kinfo->num_tqps; i++) {
- tqp = container_of(kinfo->tqp[i], struct hclgevf_tqp, q);
- *buff++ = tqp->tqp_stats.rcb_tx_ring_pktnum_rcd;
- }
- for (i = 0; i < kinfo->num_tqps; i++) {
- tqp = container_of(kinfo->tqp[i], struct hclgevf_tqp, q);
- *buff++ = tqp->tqp_stats.rcb_rx_ring_pktnum_rcd;
- }
-
- return buff;
-}
-
-static int hclgevf_tqps_get_sset_count(struct hnae3_handle *handle, int strset)
-{
- struct hnae3_knic_private_info *kinfo = &handle->kinfo;
-
- return kinfo->num_tqps * 2;
-}
-
-static u8 *hclgevf_tqps_get_strings(struct hnae3_handle *handle, u8 *data)
-{
- struct hnae3_knic_private_info *kinfo = &handle->kinfo;
- u8 *buff = data;
- int i;
-
- for (i = 0; i < kinfo->num_tqps; i++) {
- struct hclgevf_tqp *tqp = container_of(kinfo->tqp[i],
- struct hclgevf_tqp, q);
- snprintf(buff, ETH_GSTRING_LEN, "txq%u_pktnum_rcd",
- tqp->index);
- buff += ETH_GSTRING_LEN;
- }
-
- for (i = 0; i < kinfo->num_tqps; i++) {
- struct hclgevf_tqp *tqp = container_of(kinfo->tqp[i],
- struct hclgevf_tqp, q);
- snprintf(buff, ETH_GSTRING_LEN, "rxq%u_pktnum_rcd",
- tqp->index);
- buff += ETH_GSTRING_LEN;
- }
-
- return buff;
-}
-
static void hclgevf_update_stats(struct hnae3_handle *handle,
struct net_device_stats *net_stats)
{
struct hclgevf_dev *hdev = hclgevf_ae_get_hdev(handle);
int status;
- status = hclgevf_tqps_update_stats(handle);
+ status = hclge_comm_tqps_update_stats(handle, &hdev->hw.hw);
if (status)
dev_err(&hdev->pdev->dev,
"VF update of TQPS stats fail, status = %d.\n",
@@ -215,7 +139,7 @@ static int hclgevf_get_sset_count(struct hnae3_handle *handle, int strset)
if (strset == ETH_SS_TEST)
return -EOPNOTSUPP;
else if (strset == ETH_SS_STATS)
- return hclgevf_tqps_get_sset_count(handle, strset);
+ return hclge_comm_tqps_get_sset_count(handle);
return 0;
}
@@ -226,12 +150,12 @@ static void hclgevf_get_strings(struct hnae3_handle *handle, u32 strset,
u8 *p = (char *)data;
if (strset == ETH_SS_STATS)
- p = hclgevf_tqps_get_strings(handle, p);
+ p = hclge_comm_tqps_get_strings(handle, p);
}
static void hclgevf_get_stats(struct hnae3_handle *handle, u64 *data)
{
- hclgevf_tqps_get_stats(handle, data);
+ hclge_comm_tqps_get_stats(handle, data);
}
static void hclgevf_build_send_msg(struct hclge_vf_to_pf_msg *msg, u8 code,
@@ -397,11 +321,11 @@ static int hclgevf_get_pf_media_type(struct hclgevf_dev *hdev)
static int hclgevf_alloc_tqps(struct hclgevf_dev *hdev)
{
- struct hclgevf_tqp *tqp;
+ struct hclge_comm_tqp *tqp;
int i;
hdev->htqp = devm_kcalloc(&hdev->pdev->dev, hdev->num_tqps,
- sizeof(struct hclgevf_tqp), GFP_KERNEL);
+ sizeof(struct hclge_comm_tqp), GFP_KERNEL);
if (!hdev->htqp)
return -ENOMEM;
@@ -420,11 +344,11 @@ static int hclgevf_alloc_tqps(struct hclgevf_dev *hdev)
* HCLGEVF_TQP_MAX_SIZE_DEV_V2.
*/
if (i < HCLGEVF_TQP_MAX_SIZE_DEV_V2)
- tqp->q.io_base = hdev->hw.io_base +
+ tqp->q.io_base = hdev->hw.hw.io_base +
HCLGEVF_TQP_REG_OFFSET +
i * HCLGEVF_TQP_REG_SIZE;
else
- tqp->q.io_base = hdev->hw.io_base +
+ tqp->q.io_base = hdev->hw.hw.io_base +
HCLGEVF_TQP_REG_OFFSET +
HCLGEVF_TQP_EXT_REG_OFFSET +
(i - HCLGEVF_TQP_MAX_SIZE_DEV_V2) *
@@ -448,7 +372,7 @@ static int hclgevf_knic_setup(struct hclgevf_dev *hdev)
kinfo->num_tx_desc = hdev->num_tx_desc;
kinfo->num_rx_desc = hdev->num_rx_desc;
kinfo->rx_buf_len = hdev->rx_buf_len;
- for (i = 0; i < HCLGEVF_MAX_TC_NUM; i++)
+ for (i = 0; i < HCLGE_COMM_MAX_TC_NUM; i++)
if (hdev->hw_tc_map & BIT(i))
num_tc++;
@@ -539,7 +463,7 @@ static int hclgevf_set_handle_info(struct hclgevf_dev *hdev)
nic->pdev = hdev->pdev;
nic->numa_node_mask = hdev->numa_node_mask;
nic->flags |= HNAE3_SUPPORT_VF;
- nic->kinfo.io_base = hdev->hw.io_base;
+ nic->kinfo.io_base = hdev->hw.hw.io_base;
ret = hclgevf_knic_setup(hdev);
if (ret)
@@ -576,7 +500,7 @@ static int hclgevf_get_vector(struct hnae3_handle *handle, u16 vector_num,
for (i = HCLGEVF_MISC_VECTOR_NUM + 1; i < hdev->num_msi; i++) {
if (hdev->vector_status[i] == HCLGEVF_INVALID_VPORT) {
vector->vector = pci_irq_vector(hdev->pdev, i);
- vector->io_addr = hdev->hw.io_base +
+ vector->io_addr = hdev->hw.hw.io_base +
HCLGEVF_VECTOR_REG_BASE +
(i - 1) * HCLGEVF_VECTOR_REG_OFFSET;
hdev->vector_status[i] = 0;
@@ -606,137 +530,11 @@ static int hclgevf_get_vector_index(struct hclgevf_dev *hdev, int vector)
return -EINVAL;
}
-static int hclgevf_set_rss_algo_key(struct hclgevf_dev *hdev,
- const u8 hfunc, const u8 *key)
-{
- struct hclgevf_rss_config_cmd *req;
- unsigned int key_offset = 0;
- struct hclgevf_desc desc;
- int key_counts;
- int key_size;
- int ret;
-
- key_counts = HCLGEVF_RSS_KEY_SIZE;
- req = (struct hclgevf_rss_config_cmd *)desc.data;
-
- while (key_counts) {
- hclgevf_cmd_setup_basic_desc(&desc,
- HCLGEVF_OPC_RSS_GENERIC_CONFIG,
- false);
-
- req->hash_config |= (hfunc & HCLGEVF_RSS_HASH_ALGO_MASK);
- req->hash_config |=
- (key_offset << HCLGEVF_RSS_HASH_KEY_OFFSET_B);
-
- key_size = min(HCLGEVF_RSS_HASH_KEY_NUM, key_counts);
- memcpy(req->hash_key,
- key + key_offset * HCLGEVF_RSS_HASH_KEY_NUM, key_size);
-
- key_counts -= key_size;
- key_offset++;
- ret = hclgevf_cmd_send(&hdev->hw, &desc, 1);
- if (ret) {
- dev_err(&hdev->pdev->dev,
- "Configure RSS config fail, status = %d\n",
- ret);
- return ret;
- }
- }
-
- return 0;
-}
-
-static u32 hclgevf_get_rss_key_size(struct hnae3_handle *handle)
-{
- return HCLGEVF_RSS_KEY_SIZE;
-}
-
-static int hclgevf_set_rss_indir_table(struct hclgevf_dev *hdev)
-{
- const u8 *indir = hdev->rss_cfg.rss_indirection_tbl;
- struct hclgevf_rss_indirection_table_cmd *req;
- struct hclgevf_desc desc;
- int rss_cfg_tbl_num;
- int status;
- int i, j;
-
- req = (struct hclgevf_rss_indirection_table_cmd *)desc.data;
- rss_cfg_tbl_num = hdev->ae_dev->dev_specs.rss_ind_tbl_size /
- HCLGEVF_RSS_CFG_TBL_SIZE;
-
- for (i = 0; i < rss_cfg_tbl_num; i++) {
- hclgevf_cmd_setup_basic_desc(&desc, HCLGEVF_OPC_RSS_INDIR_TABLE,
- false);
- req->start_table_index =
- cpu_to_le16(i * HCLGEVF_RSS_CFG_TBL_SIZE);
- req->rss_set_bitmap = cpu_to_le16(HCLGEVF_RSS_SET_BITMAP_MSK);
- for (j = 0; j < HCLGEVF_RSS_CFG_TBL_SIZE; j++)
- req->rss_result[j] =
- indir[i * HCLGEVF_RSS_CFG_TBL_SIZE + j];
-
- status = hclgevf_cmd_send(&hdev->hw, &desc, 1);
- if (status) {
- dev_err(&hdev->pdev->dev,
- "VF failed(=%d) to set RSS indirection table\n",
- status);
- return status;
- }
- }
-
- return 0;
-}
-
-static int hclgevf_set_rss_tc_mode(struct hclgevf_dev *hdev, u16 rss_size)
-{
- struct hclgevf_rss_tc_mode_cmd *req;
- u16 tc_offset[HCLGEVF_MAX_TC_NUM];
- u16 tc_valid[HCLGEVF_MAX_TC_NUM];
- u16 tc_size[HCLGEVF_MAX_TC_NUM];
- struct hclgevf_desc desc;
- u16 roundup_size;
- unsigned int i;
- int status;
-
- req = (struct hclgevf_rss_tc_mode_cmd *)desc.data;
-
- roundup_size = roundup_pow_of_two(rss_size);
- roundup_size = ilog2(roundup_size);
-
- for (i = 0; i < HCLGEVF_MAX_TC_NUM; i++) {
- tc_valid[i] = 1;
- tc_size[i] = roundup_size;
- tc_offset[i] = (hdev->hw_tc_map & BIT(i)) ? rss_size * i : 0;
- }
-
- hclgevf_cmd_setup_basic_desc(&desc, HCLGEVF_OPC_RSS_TC_MODE, false);
- for (i = 0; i < HCLGEVF_MAX_TC_NUM; i++) {
- u16 mode = 0;
-
- hnae3_set_bit(mode, HCLGEVF_RSS_TC_VALID_B,
- (tc_valid[i] & 0x1));
- hnae3_set_field(mode, HCLGEVF_RSS_TC_SIZE_M,
- HCLGEVF_RSS_TC_SIZE_S, tc_size[i]);
- hnae3_set_bit(mode, HCLGEVF_RSS_TC_SIZE_MSB_B,
- tc_size[i] >> HCLGEVF_RSS_TC_SIZE_MSB_OFFSET &
- 0x1);
- hnae3_set_field(mode, HCLGEVF_RSS_TC_OFFSET_M,
- HCLGEVF_RSS_TC_OFFSET_S, tc_offset[i]);
-
- req->rss_tc_mode[i] = cpu_to_le16(mode);
- }
- status = hclgevf_cmd_send(&hdev->hw, &desc, 1);
- if (status)
- dev_err(&hdev->pdev->dev,
- "VF failed(=%d) to set rss tc mode\n", status);
-
- return status;
-}
-
/* for revision 0x20, vf shared the same rss config with pf */
static int hclgevf_get_rss_hash_key(struct hclgevf_dev *hdev)
{
#define HCLGEVF_RSS_MBX_RESP_LEN 8
- struct hclgevf_rss_cfg *rss_cfg = &hdev->rss_cfg;
+ struct hclge_comm_rss_cfg *rss_cfg = &hdev->rss_cfg;
u8 resp_msg[HCLGEVF_RSS_MBX_RESP_LEN];
struct hclge_vf_to_pf_msg send_msg;
u16 msg_num, hash_key_index;
@@ -744,7 +542,7 @@ static int hclgevf_get_rss_hash_key(struct hclgevf_dev *hdev)
int ret;
hclgevf_build_send_msg(&send_msg, HCLGE_MBX_GET_RSS_KEY, 0);
- msg_num = (HCLGEVF_RSS_KEY_SIZE + HCLGEVF_RSS_MBX_RESP_LEN - 1) /
+ msg_num = (HCLGE_COMM_RSS_KEY_SIZE + HCLGEVF_RSS_MBX_RESP_LEN - 1) /
HCLGEVF_RSS_MBX_RESP_LEN;
for (index = 0; index < msg_num; index++) {
send_msg.data[0] = index;
@@ -761,7 +559,7 @@ static int hclgevf_get_rss_hash_key(struct hclgevf_dev *hdev)
if (index == msg_num - 1)
memcpy(&rss_cfg->rss_hash_key[hash_key_index],
&resp_msg[0],
- HCLGEVF_RSS_KEY_SIZE - hash_key_index);
+ HCLGE_COMM_RSS_KEY_SIZE - hash_key_index);
else
memcpy(&rss_cfg->rss_hash_key[hash_key_index],
&resp_msg[0], HCLGEVF_RSS_MBX_RESP_LEN);
@@ -774,29 +572,11 @@ static int hclgevf_get_rss(struct hnae3_handle *handle, u32 *indir, u8 *key,
u8 *hfunc)
{
struct hclgevf_dev *hdev = hclgevf_ae_get_hdev(handle);
- struct hclgevf_rss_cfg *rss_cfg = &hdev->rss_cfg;
- int i, ret;
+ struct hclge_comm_rss_cfg *rss_cfg = &hdev->rss_cfg;
+ int ret;
if (hdev->ae_dev->dev_version >= HNAE3_DEVICE_VERSION_V2) {
- /* Get hash algorithm */
- if (hfunc) {
- switch (rss_cfg->hash_algo) {
- case HCLGEVF_RSS_HASH_ALGO_TOEPLITZ:
- *hfunc = ETH_RSS_HASH_TOP;
- break;
- case HCLGEVF_RSS_HASH_ALGO_SIMPLE:
- *hfunc = ETH_RSS_HASH_XOR;
- break;
- default:
- *hfunc = ETH_RSS_HASH_UNKNOWN;
- break;
- }
- }
-
- /* Get the RSS Key required by the user */
- if (key)
- memcpy(key, rss_cfg->rss_hash_key,
- HCLGEVF_RSS_KEY_SIZE);
+ hclge_comm_get_rss_hash_info(rss_cfg, key, hfunc);
} else {
if (hfunc)
*hfunc = ETH_RSS_HASH_TOP;
@@ -805,67 +585,28 @@ static int hclgevf_get_rss(struct hnae3_handle *handle, u32 *indir, u8 *key,
if (ret)
return ret;
memcpy(key, rss_cfg->rss_hash_key,
- HCLGEVF_RSS_KEY_SIZE);
+ HCLGE_COMM_RSS_KEY_SIZE);
}
}
- if (indir)
- for (i = 0; i < hdev->ae_dev->dev_specs.rss_ind_tbl_size; i++)
- indir[i] = rss_cfg->rss_indirection_tbl[i];
+ hclge_comm_get_rss_indir_tbl(rss_cfg, indir,
+ hdev->ae_dev->dev_specs.rss_ind_tbl_size);
return 0;
}
-static int hclgevf_parse_rss_hfunc(struct hclgevf_dev *hdev, const u8 hfunc,
- u8 *hash_algo)
-{
- switch (hfunc) {
- case ETH_RSS_HASH_TOP:
- *hash_algo = HCLGEVF_RSS_HASH_ALGO_TOEPLITZ;
- return 0;
- case ETH_RSS_HASH_XOR:
- *hash_algo = HCLGEVF_RSS_HASH_ALGO_SIMPLE;
- return 0;
- case ETH_RSS_HASH_NO_CHANGE:
- *hash_algo = hdev->rss_cfg.hash_algo;
- return 0;
- default:
- return -EINVAL;
- }
-}
-
static int hclgevf_set_rss(struct hnae3_handle *handle, const u32 *indir,
const u8 *key, const u8 hfunc)
{
struct hclgevf_dev *hdev = hclgevf_ae_get_hdev(handle);
- struct hclgevf_rss_cfg *rss_cfg = &hdev->rss_cfg;
- u8 hash_algo;
+ struct hclge_comm_rss_cfg *rss_cfg = &hdev->rss_cfg;
int ret, i;
if (hdev->ae_dev->dev_version >= HNAE3_DEVICE_VERSION_V2) {
- ret = hclgevf_parse_rss_hfunc(hdev, hfunc, &hash_algo);
+ ret = hclge_comm_set_rss_hash_key(rss_cfg, &hdev->hw.hw, key,
+ hfunc);
if (ret)
return ret;
-
- /* Set the RSS Hash Key if specififed by the user */
- if (key) {
- ret = hclgevf_set_rss_algo_key(hdev, hash_algo, key);
- if (ret) {
- dev_err(&hdev->pdev->dev,
- "invalid hfunc type %u\n", hfunc);
- return ret;
- }
-
- /* Update the shadow RSS key with user specified qids */
- memcpy(rss_cfg->rss_hash_key, key,
- HCLGEVF_RSS_KEY_SIZE);
- } else {
- ret = hclgevf_set_rss_algo_key(hdev, hash_algo,
- rss_cfg->rss_hash_key);
- if (ret)
- return ret;
- }
- rss_cfg->hash_algo = hash_algo;
}
/* update the shadow RSS table with user specified qids */
@@ -873,179 +614,26 @@ static int hclgevf_set_rss(struct hnae3_handle *handle, const u32 *indir,
rss_cfg->rss_indirection_tbl[i] = indir[i];
/* update the hardware */
- return hclgevf_set_rss_indir_table(hdev);
-}
-
-static u8 hclgevf_get_rss_hash_bits(struct ethtool_rxnfc *nfc)
-{
- u8 hash_sets = nfc->data & RXH_L4_B_0_1 ? HCLGEVF_S_PORT_BIT : 0;
-
- if (nfc->data & RXH_L4_B_2_3)
- hash_sets |= HCLGEVF_D_PORT_BIT;
- else
- hash_sets &= ~HCLGEVF_D_PORT_BIT;
-
- if (nfc->data & RXH_IP_SRC)
- hash_sets |= HCLGEVF_S_IP_BIT;
- else
- hash_sets &= ~HCLGEVF_S_IP_BIT;
-
- if (nfc->data & RXH_IP_DST)
- hash_sets |= HCLGEVF_D_IP_BIT;
- else
- hash_sets &= ~HCLGEVF_D_IP_BIT;
-
- if (nfc->flow_type == SCTP_V4_FLOW || nfc->flow_type == SCTP_V6_FLOW)
- hash_sets |= HCLGEVF_V_TAG_BIT;
-
- return hash_sets;
-}
-
-static int hclgevf_init_rss_tuple_cmd(struct hnae3_handle *handle,
- struct ethtool_rxnfc *nfc,
- struct hclgevf_rss_input_tuple_cmd *req)
-{
- struct hclgevf_dev *hdev = hclgevf_ae_get_hdev(handle);
- struct hclgevf_rss_cfg *rss_cfg = &hdev->rss_cfg;
- u8 tuple_sets;
-
- req->ipv4_tcp_en = rss_cfg->rss_tuple_sets.ipv4_tcp_en;
- req->ipv4_udp_en = rss_cfg->rss_tuple_sets.ipv4_udp_en;
- req->ipv4_sctp_en = rss_cfg->rss_tuple_sets.ipv4_sctp_en;
- req->ipv4_fragment_en = rss_cfg->rss_tuple_sets.ipv4_fragment_en;
- req->ipv6_tcp_en = rss_cfg->rss_tuple_sets.ipv6_tcp_en;
- req->ipv6_udp_en = rss_cfg->rss_tuple_sets.ipv6_udp_en;
- req->ipv6_sctp_en = rss_cfg->rss_tuple_sets.ipv6_sctp_en;
- req->ipv6_fragment_en = rss_cfg->rss_tuple_sets.ipv6_fragment_en;
-
- tuple_sets = hclgevf_get_rss_hash_bits(nfc);
- switch (nfc->flow_type) {
- case TCP_V4_FLOW:
- req->ipv4_tcp_en = tuple_sets;
- break;
- case TCP_V6_FLOW:
- req->ipv6_tcp_en = tuple_sets;
- break;
- case UDP_V4_FLOW:
- req->ipv4_udp_en = tuple_sets;
- break;
- case UDP_V6_FLOW:
- req->ipv6_udp_en = tuple_sets;
- break;
- case SCTP_V4_FLOW:
- req->ipv4_sctp_en = tuple_sets;
- break;
- case SCTP_V6_FLOW:
- if (hdev->ae_dev->dev_version <= HNAE3_DEVICE_VERSION_V2 &&
- (nfc->data & (RXH_L4_B_0_1 | RXH_L4_B_2_3)))
- return -EINVAL;
-
- req->ipv6_sctp_en = tuple_sets;
- break;
- case IPV4_FLOW:
- req->ipv4_fragment_en = HCLGEVF_RSS_INPUT_TUPLE_OTHER;
- break;
- case IPV6_FLOW:
- req->ipv6_fragment_en = HCLGEVF_RSS_INPUT_TUPLE_OTHER;
- break;
- default:
- return -EINVAL;
- }
-
- return 0;
+ return hclge_comm_set_rss_indir_table(hdev->ae_dev, &hdev->hw.hw,
+ rss_cfg->rss_indirection_tbl);
}
static int hclgevf_set_rss_tuple(struct hnae3_handle *handle,
struct ethtool_rxnfc *nfc)
{
struct hclgevf_dev *hdev = hclgevf_ae_get_hdev(handle);
- struct hclgevf_rss_cfg *rss_cfg = &hdev->rss_cfg;
- struct hclgevf_rss_input_tuple_cmd *req;
- struct hclgevf_desc desc;
int ret;
if (hdev->ae_dev->dev_version < HNAE3_DEVICE_VERSION_V2)
return -EOPNOTSUPP;
- if (nfc->data &
- ~(RXH_IP_SRC | RXH_IP_DST | RXH_L4_B_0_1 | RXH_L4_B_2_3))
- return -EINVAL;
-
- req = (struct hclgevf_rss_input_tuple_cmd *)desc.data;
- hclgevf_cmd_setup_basic_desc(&desc, HCLGEVF_OPC_RSS_INPUT_TUPLE, false);
-
- ret = hclgevf_init_rss_tuple_cmd(handle, nfc, req);
- if (ret) {
+ ret = hclge_comm_set_rss_tuple(hdev->ae_dev, &hdev->hw.hw,
+ &hdev->rss_cfg, nfc);
+ if (ret)
dev_err(&hdev->pdev->dev,
- "failed to init rss tuple cmd, ret = %d\n", ret);
- return ret;
- }
+ "failed to set rss tuple, ret = %d.\n", ret);
- ret = hclgevf_cmd_send(&hdev->hw, &desc, 1);
- if (ret) {
- dev_err(&hdev->pdev->dev,
- "Set rss tuple fail, status = %d\n", ret);
- return ret;
- }
-
- rss_cfg->rss_tuple_sets.ipv4_tcp_en = req->ipv4_tcp_en;
- rss_cfg->rss_tuple_sets.ipv4_udp_en = req->ipv4_udp_en;
- rss_cfg->rss_tuple_sets.ipv4_sctp_en = req->ipv4_sctp_en;
- rss_cfg->rss_tuple_sets.ipv4_fragment_en = req->ipv4_fragment_en;
- rss_cfg->rss_tuple_sets.ipv6_tcp_en = req->ipv6_tcp_en;
- rss_cfg->rss_tuple_sets.ipv6_udp_en = req->ipv6_udp_en;
- rss_cfg->rss_tuple_sets.ipv6_sctp_en = req->ipv6_sctp_en;
- rss_cfg->rss_tuple_sets.ipv6_fragment_en = req->ipv6_fragment_en;
- return 0;
-}
-
-static int hclgevf_get_rss_tuple_by_flow_type(struct hclgevf_dev *hdev,
- int flow_type, u8 *tuple_sets)
-{
- switch (flow_type) {
- case TCP_V4_FLOW:
- *tuple_sets = hdev->rss_cfg.rss_tuple_sets.ipv4_tcp_en;
- break;
- case UDP_V4_FLOW:
- *tuple_sets = hdev->rss_cfg.rss_tuple_sets.ipv4_udp_en;
- break;
- case TCP_V6_FLOW:
- *tuple_sets = hdev->rss_cfg.rss_tuple_sets.ipv6_tcp_en;
- break;
- case UDP_V6_FLOW:
- *tuple_sets = hdev->rss_cfg.rss_tuple_sets.ipv6_udp_en;
- break;
- case SCTP_V4_FLOW:
- *tuple_sets = hdev->rss_cfg.rss_tuple_sets.ipv4_sctp_en;
- break;
- case SCTP_V6_FLOW:
- *tuple_sets = hdev->rss_cfg.rss_tuple_sets.ipv6_sctp_en;
- break;
- case IPV4_FLOW:
- case IPV6_FLOW:
- *tuple_sets = HCLGEVF_S_IP_BIT | HCLGEVF_D_IP_BIT;
- break;
- default:
- return -EINVAL;
- }
-
- return 0;
-}
-
-static u64 hclgevf_convert_rss_tuple(u8 tuple_sets)
-{
- u64 tuple_data = 0;
-
- if (tuple_sets & HCLGEVF_D_PORT_BIT)
- tuple_data |= RXH_L4_B_2_3;
- if (tuple_sets & HCLGEVF_S_PORT_BIT)
- tuple_data |= RXH_L4_B_0_1;
- if (tuple_sets & HCLGEVF_D_IP_BIT)
- tuple_data |= RXH_IP_DST;
- if (tuple_sets & HCLGEVF_S_IP_BIT)
- tuple_data |= RXH_IP_SRC;
-
- return tuple_data;
+ return ret;
}
static int hclgevf_get_rss_tuple(struct hnae3_handle *handle,
@@ -1060,47 +648,20 @@ static int hclgevf_get_rss_tuple(struct hnae3_handle *handle,
nfc->data = 0;
- ret = hclgevf_get_rss_tuple_by_flow_type(hdev, nfc->flow_type,
- &tuple_sets);
+ ret = hclge_comm_get_rss_tuple(&hdev->rss_cfg, nfc->flow_type,
+ &tuple_sets);
if (ret || !tuple_sets)
return ret;
- nfc->data = hclgevf_convert_rss_tuple(tuple_sets);
+ nfc->data = hclge_comm_convert_rss_tuple(tuple_sets);
return 0;
}
-static int hclgevf_set_rss_input_tuple(struct hclgevf_dev *hdev,
- struct hclgevf_rss_cfg *rss_cfg)
-{
- struct hclgevf_rss_input_tuple_cmd *req;
- struct hclgevf_desc desc;
- int ret;
-
- hclgevf_cmd_setup_basic_desc(&desc, HCLGEVF_OPC_RSS_INPUT_TUPLE, false);
-
- req = (struct hclgevf_rss_input_tuple_cmd *)desc.data;
-
- req->ipv4_tcp_en = rss_cfg->rss_tuple_sets.ipv4_tcp_en;
- req->ipv4_udp_en = rss_cfg->rss_tuple_sets.ipv4_udp_en;
- req->ipv4_sctp_en = rss_cfg->rss_tuple_sets.ipv4_sctp_en;
- req->ipv4_fragment_en = rss_cfg->rss_tuple_sets.ipv4_fragment_en;
- req->ipv6_tcp_en = rss_cfg->rss_tuple_sets.ipv6_tcp_en;
- req->ipv6_udp_en = rss_cfg->rss_tuple_sets.ipv6_udp_en;
- req->ipv6_sctp_en = rss_cfg->rss_tuple_sets.ipv6_sctp_en;
- req->ipv6_fragment_en = rss_cfg->rss_tuple_sets.ipv6_fragment_en;
-
- ret = hclgevf_cmd_send(&hdev->hw, &desc, 1);
- if (ret)
- dev_err(&hdev->pdev->dev,
- "Configure rss input fail, status = %d\n", ret);
- return ret;
-}
-
static int hclgevf_get_tc_size(struct hnae3_handle *handle)
{
struct hclgevf_dev *hdev = hclgevf_ae_get_hdev(handle);
- struct hclgevf_rss_cfg *rss_cfg = &hdev->rss_cfg;
+ struct hclge_comm_rss_cfg *rss_cfg = &hdev->rss_cfg;
return rss_cfg->rss_size;
}
@@ -1273,12 +834,11 @@ static int hclgevf_tqp_enable_cmd_send(struct hclgevf_dev *hdev, u16 tqp_id,
u16 stream_id, bool enable)
{
struct hclgevf_cfg_com_tqp_queue_cmd *req;
- struct hclgevf_desc desc;
+ struct hclge_desc desc;
req = (struct hclgevf_cfg_com_tqp_queue_cmd *)desc.data;
- hclgevf_cmd_setup_basic_desc(&desc, HCLGEVF_OPC_CFG_COM_TQP_QUEUE,
- false);
+ hclgevf_cmd_setup_basic_desc(&desc, HCLGE_OPC_CFG_COM_TQP_QUEUE, false);
req->tqp_id = cpu_to_le16(tqp_id & HCLGEVF_RING_ID_MASK);
req->stream_id = cpu_to_le16(stream_id);
if (enable)
@@ -1302,18 +862,6 @@ static int hclgevf_tqp_enable(struct hnae3_handle *handle, bool enable)
return 0;
}
-static void hclgevf_reset_tqp_stats(struct hnae3_handle *handle)
-{
- struct hnae3_knic_private_info *kinfo = &handle->kinfo;
- struct hclgevf_tqp *tqp;
- int i;
-
- for (i = 0; i < kinfo->num_tqps; i++) {
- tqp = container_of(kinfo->tqp[i], struct hclgevf_tqp, q);
- memset(&tqp->tqp_stats, 0, sizeof(tqp->tqp_stats));
- }
-}
-
static int hclgevf_get_host_mac_addr(struct hclgevf_dev *hdev, u8 *p)
{
struct hclge_vf_to_pf_msg send_msg;
@@ -1514,15 +1062,18 @@ static void hclgevf_config_mac_list(struct hclgevf_dev *hdev,
struct list_head *list,
enum HCLGEVF_MAC_ADDR_TYPE mac_type)
{
+ char format_mac_addr[HNAE3_FORMAT_MAC_ADDR_LEN];
struct hclgevf_mac_addr_node *mac_node, *tmp;
int ret;
list_for_each_entry_safe(mac_node, tmp, list, node) {
ret = hclgevf_add_del_mac_addr(hdev, mac_node, mac_type);
if (ret) {
+ hnae3_format_mac_addr(format_mac_addr,
+ mac_node->mac_addr);
dev_err(&hdev->pdev->dev,
- "failed to configure mac %pM, state = %d, ret = %d\n",
- mac_node->mac_addr, mac_node->state, ret);
+ "failed to configure mac %s, state = %d, ret = %d\n",
+ format_mac_addr, mac_node->state, ret);
return;
}
if (mac_node->state == HCLGEVF_MAC_TO_ADD) {
@@ -1859,13 +1410,13 @@ static int hclgevf_reset_wait(struct hclgevf_dev *hdev)
int ret;
if (hdev->reset_type == HNAE3_VF_RESET)
- ret = readl_poll_timeout(hdev->hw.io_base +
+ ret = readl_poll_timeout(hdev->hw.hw.io_base +
HCLGEVF_VF_RST_ING, val,
!(val & HCLGEVF_VF_RST_ING_BIT),
HCLGEVF_RESET_WAIT_US,
HCLGEVF_RESET_WAIT_TIMEOUT_US);
else
- ret = readl_poll_timeout(hdev->hw.io_base +
+ ret = readl_poll_timeout(hdev->hw.hw.io_base +
HCLGEVF_RST_ING, val,
!(val & HCLGEVF_RST_ING_BITS),
HCLGEVF_RESET_WAIT_US,
@@ -1891,13 +1442,13 @@ static void hclgevf_reset_handshake(struct hclgevf_dev *hdev, bool enable)
{
u32 reg_val;
- reg_val = hclgevf_read_dev(&hdev->hw, HCLGEVF_NIC_CSQ_DEPTH_REG);
+ reg_val = hclgevf_read_dev(&hdev->hw, HCLGE_COMM_NIC_CSQ_DEPTH_REG);
if (enable)
reg_val |= HCLGEVF_NIC_SW_RST_RDY;
else
reg_val &= ~HCLGEVF_NIC_SW_RST_RDY;
- hclgevf_write_dev(&hdev->hw, HCLGEVF_NIC_CSQ_DEPTH_REG,
+ hclgevf_write_dev(&hdev->hw, HCLGE_COMM_NIC_CSQ_DEPTH_REG,
reg_val);
}
@@ -1948,7 +1499,7 @@ static int hclgevf_reset_prepare_wait(struct hclgevf_dev *hdev)
hdev->rst_stats.vf_func_rst_cnt++;
}
- set_bit(HCLGEVF_STATE_CMD_DISABLE, &hdev->state);
+ set_bit(HCLGE_COMM_STATE_CMD_DISABLE, &hdev->hw.hw.comm_state);
/* inform hardware that preparatory work is done */
msleep(HCLGEVF_RESET_SYNC_TIME);
hclgevf_reset_handshake(hdev, true);
@@ -1977,9 +1528,9 @@ static void hclgevf_dump_rst_info(struct hclgevf_dev *hdev)
dev_info(&hdev->pdev->dev, "vector0 interrupt enable status: 0x%x\n",
hclgevf_read_dev(&hdev->hw, HCLGEVF_MISC_VECTOR_REG_BASE));
dev_info(&hdev->pdev->dev, "vector0 interrupt status: 0x%x\n",
- hclgevf_read_dev(&hdev->hw, HCLGEVF_VECTOR0_CMDQ_STATE_REG));
+ hclgevf_read_dev(&hdev->hw, HCLGE_COMM_VECTOR0_CMDQ_STATE_REG));
dev_info(&hdev->pdev->dev, "handshake status: 0x%x\n",
- hclgevf_read_dev(&hdev->hw, HCLGEVF_NIC_CSQ_DEPTH_REG));
+ hclgevf_read_dev(&hdev->hw, HCLGE_COMM_NIC_CSQ_DEPTH_REG));
dev_info(&hdev->pdev->dev, "function reset status: 0x%x\n",
hclgevf_read_dev(&hdev->hw, HCLGEVF_RST_ING));
dev_info(&hdev->pdev->dev, "hdev state: 0x%lx\n", hdev->state);
@@ -2163,24 +1714,20 @@ static void hclgevf_reset_prepare_general(struct hnae3_ae_dev *ae_dev,
int retry_cnt = 0;
int ret;
-retry:
- down(&hdev->reset_sem);
- set_bit(HCLGEVF_STATE_RST_HANDLING, &hdev->state);
- hdev->reset_type = rst_type;
- ret = hclgevf_reset_prepare(hdev);
- if (ret) {
- dev_err(&hdev->pdev->dev, "fail to prepare to reset, ret=%d\n",
- ret);
- if (hdev->reset_pending ||
- retry_cnt++ < HCLGEVF_RESET_RETRY_CNT) {
- dev_err(&hdev->pdev->dev,
- "reset_pending:0x%lx, retry_cnt:%d\n",
- hdev->reset_pending, retry_cnt);
- clear_bit(HCLGEVF_STATE_RST_HANDLING, &hdev->state);
- up(&hdev->reset_sem);
- msleep(HCLGEVF_RESET_RETRY_WAIT_MS);
- goto retry;
- }
+ while (retry_cnt++ < HCLGEVF_RESET_RETRY_CNT) {
+ down(&hdev->reset_sem);
+ set_bit(HCLGEVF_STATE_RST_HANDLING, &hdev->state);
+ hdev->reset_type = rst_type;
+ ret = hclgevf_reset_prepare(hdev);
+ if (!ret && !hdev->reset_pending)
+ break;
+
+ dev_err(&hdev->pdev->dev,
+ "failed to prepare to reset, ret=%d, reset_pending:0x%lx, retry_cnt:%d\n",
+ ret, hdev->reset_pending, retry_cnt);
+ clear_bit(HCLGEVF_STATE_RST_HANDLING, &hdev->state);
+ up(&hdev->reset_sem);
+ msleep(HCLGEVF_RESET_RETRY_WAIT_MS);
}
/* disable misc vector before reset done */
@@ -2220,7 +1767,7 @@ static void hclgevf_get_misc_vector(struct hclgevf_dev *hdev)
vector->vector_irq = pci_irq_vector(hdev->pdev,
HCLGEVF_MISC_VECTOR_NUM);
- vector->addr = hdev->hw.io_base + HCLGEVF_MISC_VECTOR_REG_BASE;
+ vector->addr = hdev->hw.hw.io_base + HCLGEVF_MISC_VECTOR_REG_BASE;
/* vector status always valid for Vector 0 */
hdev->vector_status[HCLGEVF_MISC_VECTOR_NUM] = 0;
hdev->vector_irq[HCLGEVF_MISC_VECTOR_NUM] = vector->vector_irq;
@@ -2341,7 +1888,7 @@ static void hclgevf_keep_alive(struct hclgevf_dev *hdev)
struct hclge_vf_to_pf_msg send_msg;
int ret;
- if (test_bit(HCLGEVF_STATE_CMD_DISABLE, &hdev->state))
+ if (test_bit(HCLGE_COMM_STATE_CMD_DISABLE, &hdev->hw.hw.comm_state))
return;
hclgevf_build_send_msg(&send_msg, HCLGE_MBX_KEEP_ALIVE, 0);
@@ -2378,7 +1925,7 @@ static void hclgevf_periodic_service_task(struct hclgevf_dev *hdev)
}
if (!(hdev->serv_processed_cnt % HCLGEVF_STATS_TIMER_INTERVAL))
- hclgevf_tqps_update_stats(handle);
+ hclge_comm_tqps_update_stats(handle, &hdev->hw.hw);
/* VF does not need to request link status when this bit is set, because
* PF will push its link status to VFs when link status changed.
@@ -2419,7 +1966,7 @@ static void hclgevf_service_task(struct work_struct *work)
static void hclgevf_clear_event_cause(struct hclgevf_dev *hdev, u32 regclr)
{
- hclgevf_write_dev(&hdev->hw, HCLGEVF_VECTOR0_CMDQ_SRC_REG, regclr);
+ hclgevf_write_dev(&hdev->hw, HCLGE_COMM_VECTOR0_CMDQ_SRC_REG, regclr);
}
static enum hclgevf_evt_cause hclgevf_check_evt_cause(struct hclgevf_dev *hdev,
@@ -2429,14 +1976,14 @@ static enum hclgevf_evt_cause hclgevf_check_evt_cause(struct hclgevf_dev *hdev,
/* fetch the events from their corresponding regs */
cmdq_stat_reg = hclgevf_read_dev(&hdev->hw,
- HCLGEVF_VECTOR0_CMDQ_STATE_REG);
+ HCLGE_COMM_VECTOR0_CMDQ_STATE_REG);
if (BIT(HCLGEVF_VECTOR0_RST_INT_B) & cmdq_stat_reg) {
rst_ing_reg = hclgevf_read_dev(&hdev->hw, HCLGEVF_RST_ING);
dev_info(&hdev->pdev->dev,
"receive reset interrupt 0x%x!\n", rst_ing_reg);
set_bit(HNAE3_VF_RESET, &hdev->reset_pending);
set_bit(HCLGEVF_RESET_PENDING, &hdev->reset_state);
- set_bit(HCLGEVF_STATE_CMD_DISABLE, &hdev->state);
+ set_bit(HCLGE_COMM_STATE_CMD_DISABLE, &hdev->hw.hw.comm_state);
*clearval = ~(1U << HCLGEVF_VECTOR0_RST_INT_B);
hdev->rst_stats.vf_rst_cnt++;
/* set up VF hardware reset status, its PF will clear
@@ -2560,8 +2107,8 @@ static int hclgevf_init_roce_base_info(struct hclgevf_dev *hdev)
roce->rinfo.base_vector = hdev->roce_base_msix_offset;
roce->rinfo.netdev = nic->kinfo.netdev;
- roce->rinfo.roce_io_base = hdev->hw.io_base;
- roce->rinfo.roce_mem_base = hdev->hw.mem_base;
+ roce->rinfo.roce_io_base = hdev->hw.hw.io_base;
+ roce->rinfo.roce_mem_base = hdev->hw.hw.mem_base;
roce->pdev = nic->pdev;
roce->ae_algo = nic->ae_algo;
@@ -2573,13 +2120,13 @@ static int hclgevf_init_roce_base_info(struct hclgevf_dev *hdev)
static int hclgevf_config_gro(struct hclgevf_dev *hdev)
{
struct hclgevf_cfg_gro_status_cmd *req;
- struct hclgevf_desc desc;
+ struct hclge_desc desc;
int ret;
if (!hnae3_dev_gro_supported(hdev))
return 0;
- hclgevf_cmd_setup_basic_desc(&desc, HCLGEVF_OPC_GRO_GENERIC_CONFIG,
+ hclgevf_cmd_setup_basic_desc(&desc, HCLGE_OPC_GRO_GENERIC_CONFIG,
false);
req = (struct hclgevf_cfg_gro_status_cmd *)desc.data;
@@ -2593,71 +2140,37 @@ static int hclgevf_config_gro(struct hclgevf_dev *hdev)
return ret;
}
-static int hclgevf_rss_init_cfg(struct hclgevf_dev *hdev)
-{
- u16 rss_ind_tbl_size = hdev->ae_dev->dev_specs.rss_ind_tbl_size;
- struct hclgevf_rss_cfg *rss_cfg = &hdev->rss_cfg;
- struct hclgevf_rss_tuple_cfg *tuple_sets;
- u32 i;
-
- rss_cfg->hash_algo = HCLGEVF_RSS_HASH_ALGO_TOEPLITZ;
- rss_cfg->rss_size = hdev->nic.kinfo.rss_size;
- tuple_sets = &rss_cfg->rss_tuple_sets;
- if (hdev->ae_dev->dev_version >= HNAE3_DEVICE_VERSION_V2) {
- u8 *rss_ind_tbl;
-
- rss_cfg->hash_algo = HCLGEVF_RSS_HASH_ALGO_SIMPLE;
-
- rss_ind_tbl = devm_kcalloc(&hdev->pdev->dev, rss_ind_tbl_size,
- sizeof(*rss_ind_tbl), GFP_KERNEL);
- if (!rss_ind_tbl)
- return -ENOMEM;
-
- rss_cfg->rss_indirection_tbl = rss_ind_tbl;
- memcpy(rss_cfg->rss_hash_key, hclgevf_hash_key,
- HCLGEVF_RSS_KEY_SIZE);
-
- tuple_sets->ipv4_tcp_en = HCLGEVF_RSS_INPUT_TUPLE_OTHER;
- tuple_sets->ipv4_udp_en = HCLGEVF_RSS_INPUT_TUPLE_OTHER;
- tuple_sets->ipv4_sctp_en = HCLGEVF_RSS_INPUT_TUPLE_SCTP;
- tuple_sets->ipv4_fragment_en = HCLGEVF_RSS_INPUT_TUPLE_OTHER;
- tuple_sets->ipv6_tcp_en = HCLGEVF_RSS_INPUT_TUPLE_OTHER;
- tuple_sets->ipv6_udp_en = HCLGEVF_RSS_INPUT_TUPLE_OTHER;
- tuple_sets->ipv6_sctp_en =
- hdev->ae_dev->dev_version <= HNAE3_DEVICE_VERSION_V2 ?
- HCLGEVF_RSS_INPUT_TUPLE_SCTP_NO_PORT :
- HCLGEVF_RSS_INPUT_TUPLE_SCTP;
- tuple_sets->ipv6_fragment_en = HCLGEVF_RSS_INPUT_TUPLE_OTHER;
- }
-
- /* Initialize RSS indirect table */
- for (i = 0; i < rss_ind_tbl_size; i++)
- rss_cfg->rss_indirection_tbl[i] = i % rss_cfg->rss_size;
-
- return 0;
-}
-
static int hclgevf_rss_init_hw(struct hclgevf_dev *hdev)
{
- struct hclgevf_rss_cfg *rss_cfg = &hdev->rss_cfg;
+ struct hclge_comm_rss_cfg *rss_cfg = &hdev->rss_cfg;
+ u16 tc_offset[HCLGE_COMM_MAX_TC_NUM];
+ u16 tc_valid[HCLGE_COMM_MAX_TC_NUM];
+ u16 tc_size[HCLGE_COMM_MAX_TC_NUM];
int ret;
if (hdev->ae_dev->dev_version >= HNAE3_DEVICE_VERSION_V2) {
- ret = hclgevf_set_rss_algo_key(hdev, rss_cfg->hash_algo,
- rss_cfg->rss_hash_key);
+ ret = hclge_comm_set_rss_algo_key(&hdev->hw.hw,
+ rss_cfg->rss_algo,
+ rss_cfg->rss_hash_key);
if (ret)
return ret;
- ret = hclgevf_set_rss_input_tuple(hdev, rss_cfg);
+ ret = hclge_comm_set_rss_input_tuple(&hdev->nic, &hdev->hw.hw,
+ false, rss_cfg);
if (ret)
return ret;
}
- ret = hclgevf_set_rss_indir_table(hdev);
+ ret = hclge_comm_set_rss_indir_table(hdev->ae_dev, &hdev->hw.hw,
+ rss_cfg->rss_indirection_tbl);
if (ret)
return ret;
- return hclgevf_set_rss_tc_mode(hdev, rss_cfg->rss_size);
+ hclge_comm_get_rss_tc_info(rss_cfg->rss_size, hdev->hw_tc_map,
+ tc_offset, tc_valid, tc_size);
+
+ return hclge_comm_set_rss_tc_mode(&hdev->hw.hw, tc_offset,
+ tc_valid, tc_size);
}
static int hclgevf_init_vlan_config(struct hclgevf_dev *hdev)
@@ -2711,7 +2224,7 @@ static int hclgevf_ae_start(struct hnae3_handle *handle)
clear_bit(HCLGEVF_STATE_DOWN, &hdev->state);
clear_bit(HCLGEVF_STATE_PF_PUSH_LINK_STATUS, &hdev->state);
- hclgevf_reset_tqp_stats(handle);
+ hclge_comm_reset_tqp_stats(handle);
hclgevf_request_link_info(hdev);
@@ -2729,7 +2242,7 @@ static void hclgevf_ae_stop(struct hnae3_handle *handle)
if (hdev->reset_type != HNAE3_VF_RESET)
hclgevf_reset_tqp(handle);
- hclgevf_reset_tqp_stats(handle);
+ hclge_comm_reset_tqp_stats(handle);
hclgevf_update_link_status(hdev, 0);
}
@@ -3043,11 +2556,11 @@ static int hclgevf_dev_mem_map(struct hclgevf_dev *hdev)
if (!(pci_select_bars(pdev, IORESOURCE_MEM) & BIT(HCLGEVF_MEM_BAR)))
return 0;
- hw->mem_base = devm_ioremap_wc(&pdev->dev,
- pci_resource_start(pdev,
- HCLGEVF_MEM_BAR),
- pci_resource_len(pdev, HCLGEVF_MEM_BAR));
- if (!hw->mem_base) {
+ hw->hw.mem_base =
+ devm_ioremap_wc(&pdev->dev,
+ pci_resource_start(pdev, HCLGEVF_MEM_BAR),
+ pci_resource_len(pdev, HCLGEVF_MEM_BAR));
+ if (!hw->hw.mem_base) {
dev_err(&pdev->dev, "failed to map device memory\n");
return -EFAULT;
}
@@ -3081,9 +2594,8 @@ static int hclgevf_pci_init(struct hclgevf_dev *hdev)
pci_set_master(pdev);
hw = &hdev->hw;
- hw->hdev = hdev;
- hw->io_base = pci_iomap(pdev, 2, 0);
- if (!hw->io_base) {
+ hw->hw.io_base = pci_iomap(pdev, 2, 0);
+ if (!hw->hw.io_base) {
dev_err(&pdev->dev, "can't map configuration register space\n");
ret = -ENOMEM;
goto err_clr_master;
@@ -3096,7 +2608,7 @@ static int hclgevf_pci_init(struct hclgevf_dev *hdev)
return 0;
err_unmap_io_base:
- pci_iounmap(pdev, hdev->hw.io_base);
+ pci_iounmap(pdev, hdev->hw.hw.io_base);
err_clr_master:
pci_clear_master(pdev);
pci_release_regions(pdev);
@@ -3110,10 +2622,10 @@ static void hclgevf_pci_uninit(struct hclgevf_dev *hdev)
{
struct pci_dev *pdev = hdev->pdev;
- if (hdev->hw.mem_base)
- devm_iounmap(&pdev->dev, hdev->hw.mem_base);
+ if (hdev->hw.hw.mem_base)
+ devm_iounmap(&pdev->dev, hdev->hw.hw.mem_base);
- pci_iounmap(pdev, hdev->hw.io_base);
+ pci_iounmap(pdev, hdev->hw.hw.io_base);
pci_clear_master(pdev);
pci_release_regions(pdev);
pci_disable_device(pdev);
@@ -3122,10 +2634,10 @@ static void hclgevf_pci_uninit(struct hclgevf_dev *hdev)
static int hclgevf_query_vf_resource(struct hclgevf_dev *hdev)
{
struct hclgevf_query_res_cmd *req;
- struct hclgevf_desc desc;
+ struct hclge_desc desc;
int ret;
- hclgevf_cmd_setup_basic_desc(&desc, HCLGEVF_OPC_QUERY_VF_RSRC, true);
+ hclgevf_cmd_setup_basic_desc(&desc, HCLGE_OPC_QUERY_VF_RSRC, true);
ret = hclgevf_cmd_send(&hdev->hw, &desc, 1);
if (ret) {
dev_err(&hdev->pdev->dev,
@@ -3179,13 +2691,13 @@ static void hclgevf_set_default_dev_specs(struct hclgevf_dev *hdev)
ae_dev->dev_specs.max_non_tso_bd_num =
HCLGEVF_MAX_NON_TSO_BD_NUM;
ae_dev->dev_specs.rss_ind_tbl_size = HCLGEVF_RSS_IND_TBL_SIZE;
- ae_dev->dev_specs.rss_key_size = HCLGEVF_RSS_KEY_SIZE;
+ ae_dev->dev_specs.rss_key_size = HCLGE_COMM_RSS_KEY_SIZE;
ae_dev->dev_specs.max_int_gl = HCLGEVF_DEF_MAX_INT_GL;
ae_dev->dev_specs.max_frm_size = HCLGEVF_MAC_MAX_FRAME;
}
static void hclgevf_parse_dev_specs(struct hclgevf_dev *hdev,
- struct hclgevf_desc *desc)
+ struct hclge_desc *desc)
{
struct hnae3_ae_dev *ae_dev = pci_get_drvdata(hdev->pdev);
struct hclgevf_dev_specs_0_cmd *req0;
@@ -3212,7 +2724,7 @@ static void hclgevf_check_dev_specs(struct hclgevf_dev *hdev)
if (!dev_specs->rss_ind_tbl_size)
dev_specs->rss_ind_tbl_size = HCLGEVF_RSS_IND_TBL_SIZE;
if (!dev_specs->rss_key_size)
- dev_specs->rss_key_size = HCLGEVF_RSS_KEY_SIZE;
+ dev_specs->rss_key_size = HCLGE_COMM_RSS_KEY_SIZE;
if (!dev_specs->max_int_gl)
dev_specs->max_int_gl = HCLGEVF_DEF_MAX_INT_GL;
if (!dev_specs->max_frm_size)
@@ -3221,7 +2733,7 @@ static void hclgevf_check_dev_specs(struct hclgevf_dev *hdev)
static int hclgevf_query_dev_specs(struct hclgevf_dev *hdev)
{
- struct hclgevf_desc desc[HCLGEVF_QUERY_DEV_SPECS_BD_NUM];
+ struct hclge_desc desc[HCLGEVF_QUERY_DEV_SPECS_BD_NUM];
int ret;
int i;
@@ -3235,11 +2747,10 @@ static int hclgevf_query_dev_specs(struct hclgevf_dev *hdev)
for (i = 0; i < HCLGEVF_QUERY_DEV_SPECS_BD_NUM - 1; i++) {
hclgevf_cmd_setup_basic_desc(&desc[i],
- HCLGEVF_OPC_QUERY_DEV_SPECS, true);
- desc[i].flag |= cpu_to_le16(HCLGEVF_CMD_FLAG_NEXT);
+ HCLGE_OPC_QUERY_DEV_SPECS, true);
+ desc[i].flag |= cpu_to_le16(HCLGE_COMM_CMD_FLAG_NEXT);
}
- hclgevf_cmd_setup_basic_desc(&desc[i], HCLGEVF_OPC_QUERY_DEV_SPECS,
- true);
+ hclgevf_cmd_setup_basic_desc(&desc[i], HCLGE_OPC_QUERY_DEV_SPECS, true);
ret = hclgevf_cmd_send(&hdev->hw, desc, HCLGEVF_QUERY_DEV_SPECS_BD_NUM);
if (ret)
@@ -3318,7 +2829,10 @@ static int hclgevf_reset_hdev(struct hclgevf_dev *hdev)
return ret;
}
- ret = hclgevf_cmd_init(hdev);
+ hclgevf_arq_init(hdev);
+ ret = hclge_comm_cmd_init(hdev->ae_dev, &hdev->hw.hw,
+ &hdev->fw_version, false,
+ hdev->reset_pending);
if (ret) {
dev_err(&pdev->dev, "cmd failed %d\n", ret);
return ret;
@@ -3364,11 +2878,14 @@ static int hclgevf_init_hdev(struct hclgevf_dev *hdev)
if (ret)
goto err_devlink_init;
- ret = hclgevf_cmd_queue_init(hdev);
+ ret = hclge_comm_cmd_queue_init(hdev->pdev, &hdev->hw.hw);
if (ret)
goto err_cmd_queue_init;
- ret = hclgevf_cmd_init(hdev);
+ hclgevf_arq_init(hdev);
+ ret = hclge_comm_cmd_init(hdev->ae_dev, &hdev->hw.hw,
+ &hdev->fw_version, false,
+ hdev->reset_pending);
if (ret)
goto err_cmd_init;
@@ -3421,7 +2938,8 @@ static int hclgevf_init_hdev(struct hclgevf_dev *hdev)
goto err_config;
/* Initialize RSS for this VF */
- ret = hclgevf_rss_init_cfg(hdev);
+ ret = hclge_comm_rss_init_cfg(&hdev->nic, hdev->ae_dev,
+ &hdev->rss_cfg);
if (ret) {
dev_err(&pdev->dev, "failed to init rss cfg, ret = %d\n", ret);
goto err_config;
@@ -3468,7 +2986,7 @@ static int hclgevf_init_hdev(struct hclgevf_dev *hdev)
hclgevf_state_uninit(hdev);
hclgevf_uninit_msi(hdev);
err_cmd_init:
- hclgevf_cmd_uninit(hdev);
+ hclge_comm_cmd_uninit(hdev->ae_dev, &hdev->hw.hw);
err_cmd_queue_init:
hclgevf_devlink_uninit(hdev);
err_devlink_init:
@@ -3492,7 +3010,7 @@ static void hclgevf_uninit_hdev(struct hclgevf_dev *hdev)
hclgevf_uninit_msi(hdev);
}
- hclgevf_cmd_uninit(hdev);
+ hclge_comm_cmd_uninit(hdev->ae_dev, &hdev->hw.hw);
hclgevf_devlink_uninit(hdev);
hclgevf_pci_uninit(hdev);
hclgevf_uninit_mac_list(hdev);
@@ -3595,6 +3113,9 @@ static int hclgevf_set_channels(struct hnae3_handle *handle, u32 new_tqps_num,
{
struct hclgevf_dev *hdev = hclgevf_ae_get_hdev(handle);
struct hnae3_knic_private_info *kinfo = &handle->kinfo;
+ u16 tc_offset[HCLGE_COMM_MAX_TC_NUM];
+ u16 tc_valid[HCLGE_COMM_MAX_TC_NUM];
+ u16 tc_size[HCLGE_COMM_MAX_TC_NUM];
u16 cur_rss_size = kinfo->rss_size;
u16 cur_tqps = kinfo->num_tqps;
u32 *rss_indir;
@@ -3603,7 +3124,10 @@ static int hclgevf_set_channels(struct hnae3_handle *handle, u32 new_tqps_num,
hclgevf_update_rss_size(handle, new_tqps_num);
- ret = hclgevf_set_rss_tc_mode(hdev, kinfo->rss_size);
+ hclge_comm_get_rss_tc_info(cur_rss_size, hdev->hw_tc_map,
+ tc_offset, tc_valid, tc_size);
+ ret = hclge_comm_set_rss_tc_mode(&hdev->hw.hw, tc_offset,
+ tc_valid, tc_size);
if (ret)
return ret;
@@ -3704,7 +3228,7 @@ static bool hclgevf_get_cmdq_stat(struct hnae3_handle *handle)
{
struct hclgevf_dev *hdev = hclgevf_ae_get_hdev(handle);
- return test_bit(HCLGEVF_STATE_CMD_DISABLE, &hdev->state);
+ return test_bit(HCLGE_COMM_STATE_CMD_DISABLE, &hdev->hw.hw.comm_state);
}
static bool hclgevf_ae_dev_resetting(struct hnae3_handle *handle)
@@ -3862,7 +3386,7 @@ static const struct hnae3_ae_ops hclgevf_ops = {
.update_stats = hclgevf_update_stats,
.get_strings = hclgevf_get_strings,
.get_sset_count = hclgevf_get_sset_count,
- .get_rss_key_size = hclgevf_get_rss_key_size,
+ .get_rss_key_size = hclge_comm_get_rss_key_size,
.get_rss = hclgevf_get_rss,
.set_rss = hclgevf_set_rss,
.get_rss_tuple = hclgevf_get_rss_tuple,
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.h b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.h
index f6f736c..502ca1c 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.h
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.h
@@ -10,6 +10,8 @@
#include "hclge_mbx.h"
#include "hclgevf_cmd.h"
#include "hnae3.h"
+#include "hclge_comm_rss.h"
+#include "hclge_comm_tqp_stats.h"
#define HCLGEVF_MOD_VERSION "1.0"
#define HCLGEVF_DRIVER_NAME "hclgevf"
@@ -32,21 +34,6 @@
#define HCLGEVF_VECTOR_REG_OFFSET 0x4
#define HCLGEVF_VECTOR_VF_OFFSET 0x100000
-/* bar registers for cmdq */
-#define HCLGEVF_NIC_CSQ_BASEADDR_L_REG 0x27000
-#define HCLGEVF_NIC_CSQ_BASEADDR_H_REG 0x27004
-#define HCLGEVF_NIC_CSQ_DEPTH_REG 0x27008
-#define HCLGEVF_NIC_CSQ_TAIL_REG 0x27010
-#define HCLGEVF_NIC_CSQ_HEAD_REG 0x27014
-#define HCLGEVF_NIC_CRQ_BASEADDR_L_REG 0x27018
-#define HCLGEVF_NIC_CRQ_BASEADDR_H_REG 0x2701C
-#define HCLGEVF_NIC_CRQ_DEPTH_REG 0x27020
-#define HCLGEVF_NIC_CRQ_TAIL_REG 0x27024
-#define HCLGEVF_NIC_CRQ_HEAD_REG 0x27028
-
-#define HCLGEVF_CMDQ_INTR_EN_REG 0x27108
-#define HCLGEVF_CMDQ_INTR_GEN_REG 0x2710C
-
/* bar registers for common func */
#define HCLGEVF_GRO_EN_REG 0x28000
#define HCLGEVF_RXD_ADV_LAYOUT_EN_REG 0x28008
@@ -86,10 +73,6 @@
#define HCLGEVF_TQP_INTR_GL2_REG 0x20300
#define HCLGEVF_TQP_INTR_RL_REG 0x20900
-/* Vector0 interrupt CMDQ event source register(RW) */
-#define HCLGEVF_VECTOR0_CMDQ_SRC_REG 0x27100
-/* Vector0 interrupt CMDQ event status register(RO) */
-#define HCLGEVF_VECTOR0_CMDQ_STATE_REG 0x27104
/* CMDQ register bits for RX event(=MBX event) */
#define HCLGEVF_VECTOR0_RX_CMDQ_INT_B 1
/* RST register bits for RESET event */
@@ -112,27 +95,16 @@
#define HCLGEVF_WAIT_RESET_DONE 100
#define HCLGEVF_RSS_IND_TBL_SIZE 512
-#define HCLGEVF_RSS_SET_BITMAP_MSK 0xffff
-#define HCLGEVF_RSS_KEY_SIZE 40
-#define HCLGEVF_RSS_HASH_ALGO_TOEPLITZ 0
-#define HCLGEVF_RSS_HASH_ALGO_SIMPLE 1
-#define HCLGEVF_RSS_HASH_ALGO_SYMMETRIC 2
-#define HCLGEVF_RSS_HASH_ALGO_MASK 0xf
-
-#define HCLGEVF_RSS_INPUT_TUPLE_OTHER GENMASK(3, 0)
-#define HCLGEVF_RSS_INPUT_TUPLE_SCTP GENMASK(4, 0)
-#define HCLGEVF_D_PORT_BIT BIT(0)
-#define HCLGEVF_S_PORT_BIT BIT(1)
-#define HCLGEVF_D_IP_BIT BIT(2)
-#define HCLGEVF_S_IP_BIT BIT(3)
-#define HCLGEVF_V_TAG_BIT BIT(4)
-#define HCLGEVF_RSS_INPUT_TUPLE_SCTP_NO_PORT \
- (HCLGEVF_D_IP_BIT | HCLGEVF_S_IP_BIT | HCLGEVF_V_TAG_BIT)
#define HCLGEVF_MAC_MAX_FRAME 9728
#define HCLGEVF_STATS_TIMER_INTERVAL 36U
+#define hclgevf_read_dev(a, reg) \
+ hclge_comm_read_reg((a)->hw.io_base, reg)
+#define hclgevf_write_dev(a, reg, value) \
+ hclge_comm_write_reg((a)->hw.io_base, reg, value)
+
enum hclgevf_evt_cause {
HCLGEVF_VECTOR0_EVENT_RST,
HCLGEVF_VECTOR0_EVENT_MBX,
@@ -154,7 +126,6 @@ enum hclgevf_states {
HCLGEVF_STATE_RST_HANDLING,
HCLGEVF_STATE_MBX_SERVICE_SCHED,
HCLGEVF_STATE_MBX_HANDLING,
- HCLGEVF_STATE_CMD_DISABLE,
HCLGEVF_STATE_LINK_UPDATING,
HCLGEVF_STATE_PROMISC_CHANGED,
HCLGEVF_STATE_RST_FAIL,
@@ -173,29 +144,9 @@ struct hclgevf_mac {
};
struct hclgevf_hw {
- void __iomem *io_base;
- void __iomem *mem_base;
+ struct hclge_comm_hw hw;
int num_vec;
- struct hclgevf_cmq cmq;
struct hclgevf_mac mac;
- void *hdev; /* hchgevf device it is part of */
-};
-
-/* TQP stats */
-struct hlcgevf_tqp_stats {
- /* query_tqp_tx_queue_statistics, opcode id: 0x0B03 */
- u64 rcb_tx_ring_pktnum_rcd; /* 32bit */
- /* query_tqp_rx_queue_statistics, opcode id: 0x0B13 */
- u64 rcb_rx_ring_pktnum_rcd; /* 32bit */
-};
-
-struct hclgevf_tqp {
- struct device *dev; /* device for DMA mapping */
- struct hnae3_queue q;
- struct hlcgevf_tqp_stats tqp_stats;
- u16 index; /* global index in a NIC controller */
-
- bool alloced;
};
struct hclgevf_cfg {
@@ -208,27 +159,6 @@ struct hclgevf_cfg {
u32 numa_node_map;
};
-struct hclgevf_rss_tuple_cfg {
- u8 ipv4_tcp_en;
- u8 ipv4_udp_en;
- u8 ipv4_sctp_en;
- u8 ipv4_fragment_en;
- u8 ipv6_tcp_en;
- u8 ipv6_udp_en;
- u8 ipv6_sctp_en;
- u8 ipv6_fragment_en;
-};
-
-struct hclgevf_rss_cfg {
- u8 rss_hash_key[HCLGEVF_RSS_KEY_SIZE]; /* user configured hash keys */
- u32 hash_algo;
- u32 rss_size;
- u8 hw_tc_map;
- /* shadow table */
- u8 *rss_indirection_tbl;
- struct hclgevf_rss_tuple_cfg rss_tuple_sets;
-};
-
struct hclgevf_misc_vector {
u8 __iomem *addr;
int vector_irq;
@@ -273,7 +203,7 @@ struct hclgevf_dev {
struct hnae3_ae_dev *ae_dev;
struct hclgevf_hw hw;
struct hclgevf_misc_vector misc_vector;
- struct hclgevf_rss_cfg rss_cfg;
+ struct hclge_comm_rss_cfg rss_cfg;
unsigned long state;
unsigned long flr_state;
unsigned long default_reset_request;
@@ -324,7 +254,7 @@ struct hclgevf_dev {
struct delayed_work service_task;
- struct hclgevf_tqp *htqp;
+ struct hclge_comm_tqp *htqp;
struct hnae3_handle nic;
struct hnae3_handle roce;
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_mbx.c b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_mbx.c
index c5ac6ec..d5e0a3f 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_mbx.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_mbx.c
@@ -53,7 +53,8 @@ static int hclgevf_get_mbx_resp(struct hclgevf_dev *hdev, u16 code0, u16 code1,
}
while ((!hdev->mbx_resp.received_resp) && (i < HCLGEVF_MAX_TRY_TIMES)) {
- if (test_bit(HCLGEVF_STATE_CMD_DISABLE, &hdev->state))
+ if (test_bit(HCLGE_COMM_STATE_CMD_DISABLE,
+ &hdev->hw.hw.comm_state))
return -EIO;
usleep_range(HCLGEVF_SLEEP_USECOND, HCLGEVF_SLEEP_USECOND * 2);
@@ -97,7 +98,7 @@ int hclgevf_send_mbx_msg(struct hclgevf_dev *hdev,
u8 *resp_data, u16 resp_len)
{
struct hclge_mbx_vf_to_pf_cmd *req;
- struct hclgevf_desc desc;
+ struct hclge_desc desc;
int status;
req = (struct hclge_mbx_vf_to_pf_cmd *)desc.data;
@@ -151,9 +152,9 @@ int hclgevf_send_mbx_msg(struct hclgevf_dev *hdev,
static bool hclgevf_cmd_crq_empty(struct hclgevf_hw *hw)
{
- u32 tail = hclgevf_read_dev(hw, HCLGEVF_NIC_CRQ_TAIL_REG);
+ u32 tail = hclgevf_read_dev(hw, HCLGE_COMM_NIC_CRQ_TAIL_REG);
- return tail == hw->cmq.crq.next_to_use;
+ return tail == hw->hw.cmq.crq.next_to_use;
}
static void hclgevf_handle_mbx_response(struct hclgevf_dev *hdev,
@@ -212,14 +213,15 @@ static void hclgevf_handle_mbx_msg(struct hclgevf_dev *hdev,
void hclgevf_mbx_handler(struct hclgevf_dev *hdev)
{
struct hclge_mbx_pf_to_vf_cmd *req;
- struct hclgevf_cmq_ring *crq;
- struct hclgevf_desc *desc;
+ struct hclge_comm_cmq_ring *crq;
+ struct hclge_desc *desc;
u16 flag;
- crq = &hdev->hw.cmq.crq;
+ crq = &hdev->hw.hw.cmq.crq;
while (!hclgevf_cmd_crq_empty(&hdev->hw)) {
- if (test_bit(HCLGEVF_STATE_CMD_DISABLE, &hdev->state)) {
+ if (test_bit(HCLGE_COMM_STATE_CMD_DISABLE,
+ &hdev->hw.hw.comm_state)) {
dev_info(&hdev->pdev->dev, "vf crq need init\n");
return;
}
@@ -269,7 +271,7 @@ void hclgevf_mbx_handler(struct hclgevf_dev *hdev)
}
/* Write back CMDQ_RQ header pointer, M7 need this pointer */
- hclgevf_write_dev(&hdev->hw, HCLGEVF_NIC_CRQ_HEAD_REG,
+ hclgevf_write_dev(&hdev->hw, HCLGE_COMM_NIC_CRQ_HEAD_REG,
crq->next_to_use);
}
@@ -296,7 +298,8 @@ void hclgevf_mbx_async_handler(struct hclgevf_dev *hdev)
/* process all the async queue messages */
while (tail != hdev->arq.head) {
- if (test_bit(HCLGEVF_STATE_CMD_DISABLE, &hdev->state)) {
+ if (test_bit(HCLGE_COMM_STATE_CMD_DISABLE,
+ &hdev->hw.hw.comm_state)) {
dev_info(&hdev->pdev->dev,
"vf crq need init in async\n");
return;
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_ethtool.c b/drivers/net/ethernet/huawei/hinic/hinic_ethtool.c
index a856670..93192f5 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_ethtool.c
+++ b/drivers/net/ethernet/huawei/hinic/hinic_ethtool.c
@@ -547,7 +547,9 @@ static void hinic_get_drvinfo(struct net_device *netdev,
}
static void hinic_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct hinic_dev *nic_dev = netdev_priv(netdev);
@@ -580,7 +582,9 @@ static int check_ringparam_valid(struct hinic_dev *nic_dev,
}
static int hinic_set_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct hinic_dev *nic_dev = netdev_priv(netdev);
u16 new_sq_depth, new_rq_depth;
@@ -1205,8 +1209,6 @@ static u32 hinic_get_rxfh_indir_size(struct net_device *netdev)
return HINIC_RSS_INDIR_SIZE;
}
-#define ARRAY_LEN(arr) ((int)((int)sizeof(arr) / (int)sizeof(arr[0])))
-
#define HINIC_FUNC_STAT(_stat_item) { \
.name = #_stat_item, \
.size = sizeof_field(struct hinic_vport_stats, _stat_item), \
@@ -1374,7 +1376,7 @@ static void get_drv_queue_stats(struct hinic_dev *nic_dev, u64 *data)
break;
hinic_txq_get_stats(&nic_dev->txqs[qid], &txq_stats);
- for (j = 0; j < ARRAY_LEN(hinic_tx_queue_stats); j++, i++) {
+ for (j = 0; j < ARRAY_SIZE(hinic_tx_queue_stats); j++, i++) {
p = (char *)&txq_stats +
hinic_tx_queue_stats[j].offset;
data[i] = (hinic_tx_queue_stats[j].size ==
@@ -1387,7 +1389,7 @@ static void get_drv_queue_stats(struct hinic_dev *nic_dev, u64 *data)
break;
hinic_rxq_get_stats(&nic_dev->rxqs[qid], &rxq_stats);
- for (j = 0; j < ARRAY_LEN(hinic_rx_queue_stats); j++, i++) {
+ for (j = 0; j < ARRAY_SIZE(hinic_rx_queue_stats); j++, i++) {
p = (char *)&rxq_stats +
hinic_rx_queue_stats[j].offset;
data[i] = (hinic_rx_queue_stats[j].size ==
@@ -1411,7 +1413,7 @@ static void hinic_get_ethtool_stats(struct net_device *netdev,
netif_err(nic_dev, drv, netdev,
"Failed to get vport stats from firmware\n");
- for (j = 0; j < ARRAY_LEN(hinic_function_stats); j++, i++) {
+ for (j = 0; j < ARRAY_SIZE(hinic_function_stats); j++, i++) {
p = (char *)&vport_stats + hinic_function_stats[j].offset;
data[i] = (hinic_function_stats[j].size ==
sizeof(u64)) ? *(u64 *)p : *(u32 *)p;
@@ -1420,8 +1422,8 @@ static void hinic_get_ethtool_stats(struct net_device *netdev,
port_stats = kzalloc(sizeof(*port_stats), GFP_KERNEL);
if (!port_stats) {
memset(&data[i], 0,
- ARRAY_LEN(hinic_port_stats) * sizeof(*data));
- i += ARRAY_LEN(hinic_port_stats);
+ ARRAY_SIZE(hinic_port_stats) * sizeof(*data));
+ i += ARRAY_SIZE(hinic_port_stats);
goto get_drv_stats;
}
@@ -1430,7 +1432,7 @@ static void hinic_get_ethtool_stats(struct net_device *netdev,
netif_err(nic_dev, drv, netdev,
"Failed to get port stats from firmware\n");
- for (j = 0; j < ARRAY_LEN(hinic_port_stats); j++, i++) {
+ for (j = 0; j < ARRAY_SIZE(hinic_port_stats); j++, i++) {
p = (char *)port_stats + hinic_port_stats[j].offset;
data[i] = (hinic_port_stats[j].size ==
sizeof(u64)) ? *(u64 *)p : *(u32 *)p;
@@ -1449,14 +1451,14 @@ static int hinic_get_sset_count(struct net_device *netdev, int sset)
switch (sset) {
case ETH_SS_TEST:
- return ARRAY_LEN(hinic_test_strings);
+ return ARRAY_SIZE(hinic_test_strings);
case ETH_SS_STATS:
q_num = nic_dev->num_qps;
- count = ARRAY_LEN(hinic_function_stats) +
- (ARRAY_LEN(hinic_tx_queue_stats) +
- ARRAY_LEN(hinic_rx_queue_stats)) * q_num;
+ count = ARRAY_SIZE(hinic_function_stats) +
+ (ARRAY_SIZE(hinic_tx_queue_stats) +
+ ARRAY_SIZE(hinic_rx_queue_stats)) * q_num;
- count += ARRAY_LEN(hinic_port_stats);
+ count += ARRAY_SIZE(hinic_port_stats);
return count;
default:
@@ -1476,27 +1478,27 @@ static void hinic_get_strings(struct net_device *netdev,
memcpy(data, *hinic_test_strings, sizeof(hinic_test_strings));
return;
case ETH_SS_STATS:
- for (i = 0; i < ARRAY_LEN(hinic_function_stats); i++) {
+ for (i = 0; i < ARRAY_SIZE(hinic_function_stats); i++) {
memcpy(p, hinic_function_stats[i].name,
ETH_GSTRING_LEN);
p += ETH_GSTRING_LEN;
}
- for (i = 0; i < ARRAY_LEN(hinic_port_stats); i++) {
+ for (i = 0; i < ARRAY_SIZE(hinic_port_stats); i++) {
memcpy(p, hinic_port_stats[i].name,
ETH_GSTRING_LEN);
p += ETH_GSTRING_LEN;
}
for (i = 0; i < nic_dev->num_qps; i++) {
- for (j = 0; j < ARRAY_LEN(hinic_tx_queue_stats); j++) {
+ for (j = 0; j < ARRAY_SIZE(hinic_tx_queue_stats); j++) {
sprintf(p, hinic_tx_queue_stats[j].name, i);
p += ETH_GSTRING_LEN;
}
}
for (i = 0; i < nic_dev->num_qps; i++) {
- for (j = 0; j < ARRAY_LEN(hinic_rx_queue_stats); j++) {
+ for (j = 0; j < ARRAY_SIZE(hinic_rx_queue_stats); j++) {
sprintf(p, hinic_rx_queue_stats[j].name, i);
p += ETH_GSTRING_LEN;
}
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_api_cmd.c b/drivers/net/ethernet/huawei/hinic/hinic_hw_api_cmd.c
index 0658617..998717f 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_hw_api_cmd.c
+++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_api_cmd.c
@@ -814,7 +814,6 @@ static int api_chain_init(struct hinic_api_cmd_chain *chain,
{
struct hinic_hwif *hwif = attr->hwif;
struct pci_dev *pdev = hwif->pdev;
- size_t cell_ctxt_size;
chain->hwif = hwif;
chain->chain_type = attr->chain_type;
@@ -826,8 +825,8 @@ static int api_chain_init(struct hinic_api_cmd_chain *chain,
sema_init(&chain->sem, 1);
- cell_ctxt_size = chain->num_cells * sizeof(*chain->cell_ctxt);
- chain->cell_ctxt = devm_kzalloc(&pdev->dev, cell_ctxt_size, GFP_KERNEL);
+ chain->cell_ctxt = devm_kcalloc(&pdev->dev, chain->num_cells,
+ sizeof(*chain->cell_ctxt), GFP_KERNEL);
if (!chain->cell_ctxt)
return -ENOMEM;
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_cmdq.c b/drivers/net/ethernet/huawei/hinic/hinic_hw_cmdq.c
index 307a6d4..a627237 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_hw_cmdq.c
+++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_cmdq.c
@@ -796,11 +796,10 @@ static int init_cmdqs_ctxt(struct hinic_hwdev *hwdev,
struct hinic_cmdq_ctxt *cmdq_ctxts;
struct pci_dev *pdev = hwif->pdev;
struct hinic_pfhwdev *pfhwdev;
- size_t cmdq_ctxts_size;
int err;
- cmdq_ctxts_size = HINIC_MAX_CMDQ_TYPES * sizeof(*cmdq_ctxts);
- cmdq_ctxts = devm_kzalloc(&pdev->dev, cmdq_ctxts_size, GFP_KERNEL);
+ cmdq_ctxts = devm_kcalloc(&pdev->dev, HINIC_MAX_CMDQ_TYPES,
+ sizeof(*cmdq_ctxts), GFP_KERNEL);
if (!cmdq_ctxts)
return -ENOMEM;
@@ -884,7 +883,6 @@ int hinic_init_cmdqs(struct hinic_cmdqs *cmdqs, struct hinic_hwif *hwif,
struct hinic_func_to_io *func_to_io = cmdqs_to_func_to_io(cmdqs);
struct pci_dev *pdev = hwif->pdev;
struct hinic_hwdev *hwdev;
- size_t saved_wqs_size;
u16 max_wqe_size;
int err;
@@ -895,8 +893,8 @@ int hinic_init_cmdqs(struct hinic_cmdqs *cmdqs, struct hinic_hwif *hwif,
if (!cmdqs->cmdq_buf_pool)
return -ENOMEM;
- saved_wqs_size = HINIC_MAX_CMDQ_TYPES * sizeof(struct hinic_wq);
- cmdqs->saved_wqs = devm_kzalloc(&pdev->dev, saved_wqs_size, GFP_KERNEL);
+ cmdqs->saved_wqs = devm_kcalloc(&pdev->dev, HINIC_MAX_CMDQ_TYPES,
+ sizeof(*cmdqs->saved_wqs), GFP_KERNEL);
if (!cmdqs->saved_wqs) {
err = -ENOMEM;
goto err_saved_wqs;
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_dev.c b/drivers/net/ethernet/huawei/hinic/hinic_hw_dev.c
index 657a154..2127a48 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_hw_dev.c
+++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_dev.c
@@ -162,7 +162,6 @@ static int init_msix(struct hinic_hwdev *hwdev)
struct hinic_hwif *hwif = hwdev->hwif;
struct pci_dev *pdev = hwif->pdev;
int nr_irqs, num_aeqs, num_ceqs;
- size_t msix_entries_size;
int i, err;
num_aeqs = HINIC_HWIF_NUM_AEQS(hwif);
@@ -171,8 +170,8 @@ static int init_msix(struct hinic_hwdev *hwdev)
if (nr_irqs > HINIC_HWIF_NUM_IRQS(hwif))
nr_irqs = HINIC_HWIF_NUM_IRQS(hwif);
- msix_entries_size = nr_irqs * sizeof(*hwdev->msix_entries);
- hwdev->msix_entries = devm_kzalloc(&pdev->dev, msix_entries_size,
+ hwdev->msix_entries = devm_kcalloc(&pdev->dev, nr_irqs,
+ sizeof(*hwdev->msix_entries),
GFP_KERNEL);
if (!hwdev->msix_entries)
return -ENOMEM;
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_eqs.c b/drivers/net/ethernet/huawei/hinic/hinic_hw_eqs.c
index d3fc05a..045c477 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_hw_eqs.c
+++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_eqs.c
@@ -631,16 +631,15 @@ static int alloc_eq_pages(struct hinic_eq *eq)
struct hinic_hwif *hwif = eq->hwif;
struct pci_dev *pdev = hwif->pdev;
u32 init_val, addr, val;
- size_t addr_size;
int err, pg;
- addr_size = eq->num_pages * sizeof(*eq->dma_addr);
- eq->dma_addr = devm_kzalloc(&pdev->dev, addr_size, GFP_KERNEL);
+ eq->dma_addr = devm_kcalloc(&pdev->dev, eq->num_pages,
+ sizeof(*eq->dma_addr), GFP_KERNEL);
if (!eq->dma_addr)
return -ENOMEM;
- addr_size = eq->num_pages * sizeof(*eq->virt_addr);
- eq->virt_addr = devm_kzalloc(&pdev->dev, addr_size, GFP_KERNEL);
+ eq->virt_addr = devm_kcalloc(&pdev->dev, eq->num_pages,
+ sizeof(*eq->virt_addr), GFP_KERNEL);
if (!eq->virt_addr) {
err = -ENOMEM;
goto err_virt_addr_alloc;
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_io.c b/drivers/net/ethernet/huawei/hinic/hinic_hw_io.c
index a6e43d6..c4a0ba6 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_hw_io.c
+++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_io.c
@@ -375,31 +375,30 @@ int hinic_io_create_qps(struct hinic_func_to_io *func_to_io,
{
struct hinic_hwif *hwif = func_to_io->hwif;
struct pci_dev *pdev = hwif->pdev;
- size_t qps_size, wq_size, db_size;
void *ci_addr_base;
int i, j, err;
- qps_size = num_qps * sizeof(*func_to_io->qps);
- func_to_io->qps = devm_kzalloc(&pdev->dev, qps_size, GFP_KERNEL);
+ func_to_io->qps = devm_kcalloc(&pdev->dev, num_qps,
+ sizeof(*func_to_io->qps), GFP_KERNEL);
if (!func_to_io->qps)
return -ENOMEM;
- wq_size = num_qps * sizeof(*func_to_io->sq_wq);
- func_to_io->sq_wq = devm_kzalloc(&pdev->dev, wq_size, GFP_KERNEL);
+ func_to_io->sq_wq = devm_kcalloc(&pdev->dev, num_qps,
+ sizeof(*func_to_io->sq_wq), GFP_KERNEL);
if (!func_to_io->sq_wq) {
err = -ENOMEM;
goto err_sq_wq;
}
- wq_size = num_qps * sizeof(*func_to_io->rq_wq);
- func_to_io->rq_wq = devm_kzalloc(&pdev->dev, wq_size, GFP_KERNEL);
+ func_to_io->rq_wq = devm_kcalloc(&pdev->dev, num_qps,
+ sizeof(*func_to_io->rq_wq), GFP_KERNEL);
if (!func_to_io->rq_wq) {
err = -ENOMEM;
goto err_rq_wq;
}
- db_size = num_qps * sizeof(*func_to_io->sq_db);
- func_to_io->sq_db = devm_kzalloc(&pdev->dev, db_size, GFP_KERNEL);
+ func_to_io->sq_db = devm_kcalloc(&pdev->dev, num_qps,
+ sizeof(*func_to_io->sq_db), GFP_KERNEL);
if (!func_to_io->sq_db) {
err = -ENOMEM;
goto err_sq_db;
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_wq.c b/drivers/net/ethernet/huawei/hinic/hinic_hw_wq.c
index 7f0f1aa..2d9b06d 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_hw_wq.c
+++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_wq.c
@@ -193,20 +193,20 @@ static int alloc_page_arrays(struct hinic_wqs *wqs)
{
struct hinic_hwif *hwif = wqs->hwif;
struct pci_dev *pdev = hwif->pdev;
- size_t size;
- size = wqs->num_pages * sizeof(*wqs->page_paddr);
- wqs->page_paddr = devm_kzalloc(&pdev->dev, size, GFP_KERNEL);
+ wqs->page_paddr = devm_kcalloc(&pdev->dev, wqs->num_pages,
+ sizeof(*wqs->page_paddr), GFP_KERNEL);
if (!wqs->page_paddr)
return -ENOMEM;
- size = wqs->num_pages * sizeof(*wqs->page_vaddr);
- wqs->page_vaddr = devm_kzalloc(&pdev->dev, size, GFP_KERNEL);
+ wqs->page_vaddr = devm_kcalloc(&pdev->dev, wqs->num_pages,
+ sizeof(*wqs->page_vaddr), GFP_KERNEL);
if (!wqs->page_vaddr)
goto err_page_vaddr;
- size = wqs->num_pages * sizeof(*wqs->shadow_page_vaddr);
- wqs->shadow_page_vaddr = devm_kzalloc(&pdev->dev, size, GFP_KERNEL);
+ wqs->shadow_page_vaddr = devm_kcalloc(&pdev->dev, wqs->num_pages,
+ sizeof(*wqs->shadow_page_vaddr),
+ GFP_KERNEL);
if (!wqs->shadow_page_vaddr)
goto err_page_shadow_vaddr;
@@ -379,15 +379,14 @@ static int alloc_wqes_shadow(struct hinic_wq *wq)
{
struct hinic_hwif *hwif = wq->hwif;
struct pci_dev *pdev = hwif->pdev;
- size_t size;
- size = wq->num_q_pages * wq->max_wqe_size;
- wq->shadow_wqe = devm_kzalloc(&pdev->dev, size, GFP_KERNEL);
+ wq->shadow_wqe = devm_kcalloc(&pdev->dev, wq->num_q_pages,
+ wq->max_wqe_size, GFP_KERNEL);
if (!wq->shadow_wqe)
return -ENOMEM;
- size = wq->num_q_pages * sizeof(wq->prod_idx);
- wq->shadow_idx = devm_kzalloc(&pdev->dev, size, GFP_KERNEL);
+ wq->shadow_idx = devm_kcalloc(&pdev->dev, wq->num_q_pages,
+ sizeof(wq->prod_idx), GFP_KERNEL);
if (!wq->shadow_idx)
goto err_shadow_idx;
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_main.c b/drivers/net/ethernet/huawei/hinic/hinic_main.c
index f9a766b..0532929 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_main.c
+++ b/drivers/net/ethernet/huawei/hinic/hinic_main.c
@@ -144,13 +144,12 @@ static int create_txqs(struct hinic_dev *nic_dev)
{
int err, i, j, num_txqs = hinic_hwdev_num_qps(nic_dev->hwdev);
struct net_device *netdev = nic_dev->netdev;
- size_t txq_size;
if (nic_dev->txqs)
return -EINVAL;
- txq_size = num_txqs * sizeof(*nic_dev->txqs);
- nic_dev->txqs = devm_kzalloc(&netdev->dev, txq_size, GFP_KERNEL);
+ nic_dev->txqs = devm_kcalloc(&netdev->dev, num_txqs,
+ sizeof(*nic_dev->txqs), GFP_KERNEL);
if (!nic_dev->txqs)
return -ENOMEM;
@@ -241,13 +240,12 @@ static int create_rxqs(struct hinic_dev *nic_dev)
{
int err, i, j, num_rxqs = hinic_hwdev_num_qps(nic_dev->hwdev);
struct net_device *netdev = nic_dev->netdev;
- size_t rxq_size;
if (nic_dev->rxqs)
return -EINVAL;
- rxq_size = num_rxqs * sizeof(*nic_dev->rxqs);
- nic_dev->rxqs = devm_kzalloc(&netdev->dev, rxq_size, GFP_KERNEL);
+ nic_dev->rxqs = devm_kcalloc(&netdev->dev, num_rxqs,
+ sizeof(*nic_dev->rxqs), GFP_KERNEL);
if (!nic_dev->rxqs)
return -ENOMEM;
@@ -1394,12 +1392,8 @@ static int hinic_probe(struct pci_dev *pdev,
err = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64));
if (err) {
- dev_warn(&pdev->dev, "Couldn't set 64-bit DMA mask\n");
- err = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32));
- if (err) {
- dev_err(&pdev->dev, "Failed to set DMA mask\n");
- goto err_dma_mask;
- }
+ dev_err(&pdev->dev, "Failed to set DMA mask\n");
+ goto err_dma_mask;
}
err = nic_dev_init(pdev);
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_tx.c b/drivers/net/ethernet/huawei/hinic/hinic_tx.c
index c5bdb0d..8d59bab 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_tx.c
+++ b/drivers/net/ethernet/huawei/hinic/hinic_tx.c
@@ -4,6 +4,7 @@
* Copyright(c) 2017 Huawei Technologies Co., Ltd
*/
+#include <linux/if_vlan.h>
#include <linux/kernel.h>
#include <linux/netdevice.h>
#include <linux/u64_stats_sync.h>
@@ -862,7 +863,6 @@ int hinic_init_txq(struct hinic_txq *txq, struct hinic_sq *sq,
struct hinic_dev *nic_dev = netdev_priv(netdev);
struct hinic_hwdev *hwdev = nic_dev->hwdev;
int err, irqname_len;
- size_t sges_size;
txq->netdev = netdev;
txq->sq = sq;
@@ -871,13 +871,13 @@ int hinic_init_txq(struct hinic_txq *txq, struct hinic_sq *sq,
txq->max_sges = HINIC_MAX_SQ_BUFDESCS;
- sges_size = txq->max_sges * sizeof(*txq->sges);
- txq->sges = devm_kzalloc(&netdev->dev, sges_size, GFP_KERNEL);
+ txq->sges = devm_kcalloc(&netdev->dev, txq->max_sges,
+ sizeof(*txq->sges), GFP_KERNEL);
if (!txq->sges)
return -ENOMEM;
- sges_size = txq->max_sges * sizeof(*txq->free_sges);
- txq->free_sges = devm_kzalloc(&netdev->dev, sges_size, GFP_KERNEL);
+ txq->free_sges = devm_kcalloc(&netdev->dev, txq->max_sges,
+ sizeof(*txq->free_sges), GFP_KERNEL);
if (!txq->free_sges) {
err = -ENOMEM;
goto err_alloc_free_sges;
diff --git a/drivers/net/ethernet/i825xx/82596.c b/drivers/net/ethernet/i825xx/82596.c
index b482f6f..3ee89ae 100644
--- a/drivers/net/ethernet/i825xx/82596.c
+++ b/drivers/net/ethernet/i825xx/82596.c
@@ -1178,7 +1178,8 @@ static struct net_device * __init i82596_probe(void)
DEB(DEB_PROBE,printk(KERN_INFO "%s: 82596 at %#3lx,", dev->name, dev->base_addr));
for (i = 0; i < 6; i++)
- DEB(DEB_PROBE,printk(" %2.2X", dev->dev_addr[i] = eth_addr[i]));
+ DEB(DEB_PROBE,printk(" %2.2X", eth_addr[i]));
+ eth_hw_addr_set(dev, eth_addr);
DEB(DEB_PROBE,printk(" IRQ %d.\n", dev->irq));
diff --git a/drivers/net/ethernet/i825xx/lasi_82596.c b/drivers/net/ethernet/i825xx/lasi_82596.c
index 48e0018..0af7009 100644
--- a/drivers/net/ethernet/i825xx/lasi_82596.c
+++ b/drivers/net/ethernet/i825xx/lasi_82596.c
@@ -147,6 +147,7 @@ lan_init_chip(struct parisc_device *dev)
struct net_device *netdevice;
struct i596_private *lp;
int retval = -ENOMEM;
+ u8 addr[ETH_ALEN];
int i;
if (!dev->irq) {
@@ -167,13 +168,14 @@ lan_init_chip(struct parisc_device *dev)
netdevice->base_addr = dev->hpa.start;
netdevice->irq = dev->irq;
- if (pdc_lan_station_id(netdevice->dev_addr, netdevice->base_addr)) {
+ if (pdc_lan_station_id(addr, netdevice->base_addr)) {
for (i = 0; i < 6; i++) {
- netdevice->dev_addr[i] = gsc_readb(LAN_PROM_ADDR + i);
+ addr[i] = gsc_readb(LAN_PROM_ADDR + i);
}
printk(KERN_INFO
"%s: MAC of HP700 LAN read from EEPROM\n", __FILE__);
}
+ eth_hw_addr_set(netdevice, addr);
lp = netdev_priv(netdevice);
lp->options = dev->id.sversion == 0x72 ? OPT_SWAP_PORT : 0;
diff --git a/drivers/net/ethernet/ibm/emac/core.c b/drivers/net/ethernet/ibm/emac/core.c
index 6b3fc88..fbea9f7 100644
--- a/drivers/net/ethernet/ibm/emac/core.c
+++ b/drivers/net/ethernet/ibm/emac/core.c
@@ -2137,8 +2137,11 @@ emac_ethtool_set_link_ksettings(struct net_device *ndev,
return 0;
}
-static void emac_ethtool_get_ringparam(struct net_device *ndev,
- struct ethtool_ringparam *rp)
+static void
+emac_ethtool_get_ringparam(struct net_device *ndev,
+ struct ethtool_ringparam *rp,
+ struct kernel_ethtool_ringparam *kernel_rp,
+ struct netlink_ext_ack *extack)
{
rp->rx_max_pending = rp->rx_pending = NUM_RX_BUFF;
rp->tx_max_pending = rp->tx_pending = NUM_TX_BUFF;
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index 45ba40c..22fb0d1 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -1890,6 +1890,7 @@ static struct attribute *veth_pool_attrs[] = {
&veth_size_attr,
NULL,
};
+ATTRIBUTE_GROUPS(veth_pool);
static const struct sysfs_ops veth_pool_ops = {
.show = veth_pool_show,
@@ -1899,7 +1900,7 @@ static const struct sysfs_ops veth_pool_ops = {
static struct kobj_type ktype_veth_pool = {
.release = NULL,
.sysfs_ops = &veth_pool_ops,
- .default_attrs = veth_pool_attrs,
+ .default_groups = veth_pool_groups,
};
static int ibmveth_resume(struct device *dev)
diff --git a/drivers/net/ethernet/ibm/ibmvnic.c b/drivers/net/ethernet/ibm/ibmvnic.c
index 0bb3911..59536bd 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.c
+++ b/drivers/net/ethernet/ibm/ibmvnic.c
@@ -308,7 +308,7 @@ static int alloc_long_term_buff(struct ibmvnic_adapter *adapter,
if (adapter->fw_done_rc) {
dev_err(dev, "Couldn't map LTB, rc = %d\n",
adapter->fw_done_rc);
- rc = -1;
+ rc = -EIO;
goto out;
}
rc = 0;
@@ -540,13 +540,15 @@ static int init_stats_token(struct ibmvnic_adapter *adapter)
{
struct device *dev = &adapter->vdev->dev;
dma_addr_t stok;
+ int rc;
stok = dma_map_single(dev, &adapter->stats,
sizeof(struct ibmvnic_statistics),
DMA_FROM_DEVICE);
- if (dma_mapping_error(dev, stok)) {
- dev_err(dev, "Couldn't map stats buffer\n");
- return -1;
+ rc = dma_mapping_error(dev, stok);
+ if (rc) {
+ dev_err(dev, "Couldn't map stats buffer, rc = %d\n", rc);
+ return rc;
}
adapter->stats_token = stok;
@@ -655,7 +657,7 @@ static int init_rx_pools(struct net_device *netdev)
u64 num_pools;
u64 pool_size; /* # of buffers in one pool */
u64 buff_size;
- int i, j;
+ int i, j, rc;
pool_size = adapter->req_rx_add_entries_per_subcrq;
num_pools = adapter->req_rx_queues;
@@ -674,7 +676,7 @@ static int init_rx_pools(struct net_device *netdev)
GFP_KERNEL);
if (!adapter->rx_pool) {
dev_err(dev, "Failed to allocate rx pools\n");
- return -1;
+ return -ENOMEM;
}
/* Set num_active_rx_pools early. If we fail below after partial
@@ -697,6 +699,7 @@ static int init_rx_pools(struct net_device *netdev)
GFP_KERNEL);
if (!rx_pool->free_map) {
dev_err(dev, "Couldn't alloc free_map %d\n", i);
+ rc = -ENOMEM;
goto out_release;
}
@@ -705,6 +708,7 @@ static int init_rx_pools(struct net_device *netdev)
GFP_KERNEL);
if (!rx_pool->rx_buff) {
dev_err(dev, "Couldn't alloc rx buffers\n");
+ rc = -ENOMEM;
goto out_release;
}
}
@@ -718,8 +722,9 @@ static int init_rx_pools(struct net_device *netdev)
dev_dbg(dev, "Updating LTB for rx pool %d [%d, %d]\n",
i, rx_pool->size, rx_pool->buff_size);
- if (alloc_long_term_buff(adapter, &rx_pool->long_term_buff,
- rx_pool->size * rx_pool->buff_size))
+ rc = alloc_long_term_buff(adapter, &rx_pool->long_term_buff,
+ rx_pool->size * rx_pool->buff_size);
+ if (rc)
goto out;
for (j = 0; j < rx_pool->size; ++j) {
@@ -756,7 +761,7 @@ static int init_rx_pools(struct net_device *netdev)
/* We failed to allocate one or more LTBs or map them on the VIOS.
* Hold onto the pools and any LTBs that we did allocate/map.
*/
- return -1;
+ return rc;
}
static void release_vpd_data(struct ibmvnic_adapter *adapter)
@@ -817,13 +822,13 @@ static int init_one_tx_pool(struct net_device *netdev,
sizeof(struct ibmvnic_tx_buff),
GFP_KERNEL);
if (!tx_pool->tx_buff)
- return -1;
+ return -ENOMEM;
tx_pool->free_map = kcalloc(pool_size, sizeof(int), GFP_KERNEL);
if (!tx_pool->free_map) {
kfree(tx_pool->tx_buff);
tx_pool->tx_buff = NULL;
- return -1;
+ return -ENOMEM;
}
for (i = 0; i < pool_size; i++)
@@ -914,7 +919,7 @@ static int init_tx_pools(struct net_device *netdev)
adapter->tx_pool = kcalloc(num_pools,
sizeof(struct ibmvnic_tx_pool), GFP_KERNEL);
if (!adapter->tx_pool)
- return -1;
+ return -ENOMEM;
adapter->tso_pool = kcalloc(num_pools,
sizeof(struct ibmvnic_tx_pool), GFP_KERNEL);
@@ -924,7 +929,7 @@ static int init_tx_pools(struct net_device *netdev)
if (!adapter->tso_pool) {
kfree(adapter->tx_pool);
adapter->tx_pool = NULL;
- return -1;
+ return -ENOMEM;
}
/* Set num_active_tx_pools early. If we fail below after partial
@@ -1113,7 +1118,7 @@ static int ibmvnic_login(struct net_device *netdev)
retry = false;
if (retry_count > retries) {
netdev_warn(netdev, "Login attempts exceeded\n");
- return -1;
+ return -EACCES;
}
adapter->init_done_rc = 0;
@@ -1154,25 +1159,26 @@ static int ibmvnic_login(struct net_device *netdev)
timeout)) {
netdev_warn(netdev,
"Capabilities query timed out\n");
- return -1;
+ return -ETIMEDOUT;
}
rc = init_sub_crqs(adapter);
if (rc) {
netdev_warn(netdev,
"SCRQ initialization failed\n");
- return -1;
+ return rc;
}
rc = init_sub_crq_irqs(adapter);
if (rc) {
netdev_warn(netdev,
"SCRQ irq initialization failed\n");
- return -1;
+ return rc;
}
} else if (adapter->init_done_rc) {
- netdev_warn(netdev, "Adapter login failed\n");
- return -1;
+ netdev_warn(netdev, "Adapter login failed, init_done_rc = %d\n",
+ adapter->init_done_rc);
+ return -EIO;
}
} while (retry);
@@ -1231,7 +1237,7 @@ static int set_link_state(struct ibmvnic_adapter *adapter, u8 link_state)
if (!wait_for_completion_timeout(&adapter->init_done,
timeout)) {
netdev_err(netdev, "timeout setting link state\n");
- return -1;
+ return -ETIMEDOUT;
}
if (adapter->init_done_rc == PARTIALSUCCESS) {
@@ -2042,7 +2048,7 @@ static netdev_tx_t ibmvnic_xmit(struct sk_buff *skb, struct net_device *netdev)
tx_packets++;
tx_bytes += skb->len;
- txq->trans_start = jiffies;
+ txq_trans_cond_update(txq);
ret = NETDEV_TX_OK;
goto out;
@@ -2288,7 +2294,7 @@ static int do_reset(struct ibmvnic_adapter *adapter,
/* If someone else changed the adapter state
* when we dropped the rtnl, fail the reset
*/
- rc = -1;
+ rc = -EAGAIN;
goto out;
}
adapter->state = VNIC_CLOSED;
@@ -2330,10 +2336,8 @@ static int do_reset(struct ibmvnic_adapter *adapter,
}
rc = ibmvnic_reset_init(adapter, true);
- if (rc) {
- rc = IBMVNIC_INIT_FAILED;
+ if (rc)
goto out;
- }
/* If the adapter was in PROBE or DOWN state prior to the reset,
* exit here.
@@ -3072,7 +3076,9 @@ static u32 ibmvnic_get_link(struct net_device *netdev)
}
static void ibmvnic_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct ibmvnic_adapter *adapter = netdev_priv(netdev);
@@ -3092,7 +3098,9 @@ static void ibmvnic_get_ringparam(struct net_device *netdev,
}
static int ibmvnic_set_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct ibmvnic_adapter *adapter = netdev_priv(netdev);
int ret;
@@ -3759,7 +3767,7 @@ static int init_sub_crqs(struct ibmvnic_adapter *adapter)
allqueues = kcalloc(total_queues, sizeof(*allqueues), GFP_KERNEL);
if (!allqueues)
- return -1;
+ return -ENOMEM;
for (i = 0; i < total_queues; i++) {
allqueues[i] = init_sub_crq_queue(adapter);
@@ -3828,7 +3836,7 @@ static int init_sub_crqs(struct ibmvnic_adapter *adapter)
for (i = 0; i < registered_queues; i++)
release_sub_crq_queue(adapter, allqueues[i], 1);
kfree(allqueues);
- return -1;
+ return -ENOMEM;
}
static void send_request_cap(struct ibmvnic_adapter *adapter, int retry)
@@ -4187,7 +4195,7 @@ static int send_login(struct ibmvnic_adapter *adapter)
if (!adapter->tx_scrq || !adapter->rx_scrq) {
netdev_err(adapter->netdev,
"RX or TX queues are not allocated, device login failed\n");
- return -1;
+ return -ENOMEM;
}
release_login_buffer(adapter);
@@ -4307,7 +4315,7 @@ static int send_login(struct ibmvnic_adapter *adapter)
kfree(login_buffer);
adapter->login_buf = NULL;
buf_alloc_failed:
- return -1;
+ return -ENOMEM;
}
static int send_request_map(struct ibmvnic_adapter *adapter, dma_addr_t addr,
@@ -5628,7 +5636,7 @@ static int ibmvnic_reset_init(struct ibmvnic_adapter *adapter, bool reset)
if (!wait_for_completion_timeout(&adapter->init_done, timeout)) {
dev_err(dev, "Initialization sequence timed out\n");
- return -1;
+ return -ETIMEDOUT;
}
if (adapter->init_done_rc) {
@@ -5639,7 +5647,7 @@ static int ibmvnic_reset_init(struct ibmvnic_adapter *adapter, bool reset)
if (adapter->from_passive_init) {
adapter->state = VNIC_OPEN;
adapter->from_passive_init = false;
- return -1;
+ return -EINVAL;
}
if (reset &&
diff --git a/drivers/net/ethernet/ibm/ibmvnic.h b/drivers/net/ethernet/ibm/ibmvnic.h
index b8e42f6..4a8f36e 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.h
+++ b/drivers/net/ethernet/ibm/ibmvnic.h
@@ -18,8 +18,6 @@
#define IBMVNIC_NAME "ibmvnic"
#define IBMVNIC_DRIVER_VERSION "1.0.1"
#define IBMVNIC_INVALID_MAP -1
-#define IBMVNIC_STATS_TIMEOUT 1
-#define IBMVNIC_INIT_FAILED 2
#define IBMVNIC_OPEN_FAILED 3
/* basic structures plus 100 2k buffers */
diff --git a/drivers/net/ethernet/intel/Kconfig b/drivers/net/ethernet/intel/Kconfig
index 0b274d8..3facb55 100644
--- a/drivers/net/ethernet/intel/Kconfig
+++ b/drivers/net/ethernet/intel/Kconfig
@@ -327,6 +327,16 @@
If unsure, say N.
+config ICE_HWTS
+ bool "Support HW cross-timestamp on platforms with PTM support"
+ default y
+ depends on ICE && X86
+ help
+ Say Y to enable hardware supported cross-timestamping on platforms
+ with PCIe PTM support. The cross-timestamp is available through
+ the PTP clock driver precise cross-timestamp ioctl
+ (PTP_SYS_OFFSET_PRECISE).
+
config FM10K
tristate "Intel(R) FM10000 Ethernet Switch Host Interface Support"
default n
diff --git a/drivers/net/ethernet/intel/e100.c b/drivers/net/ethernet/intel/e100.c
index 0bf3d47..4a8013f 100644
--- a/drivers/net/ethernet/intel/e100.c
+++ b/drivers/net/ethernet/intel/e100.c
@@ -2557,7 +2557,9 @@ static int e100_set_eeprom(struct net_device *netdev,
}
static void e100_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct nic *nic = netdev_priv(netdev);
struct param_range *rfds = &nic->params.rfds;
@@ -2570,7 +2572,9 @@ static void e100_get_ringparam(struct net_device *netdev,
}
static int e100_set_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct nic *nic = netdev_priv(netdev);
struct param_range *rfds = &nic->params.rfds;
diff --git a/drivers/net/ethernet/intel/e1000/e1000_ethtool.c b/drivers/net/ethernet/intel/e1000/e1000_ethtool.c
index 0a57172..32803b0 100644
--- a/drivers/net/ethernet/intel/e1000/e1000_ethtool.c
+++ b/drivers/net/ethernet/intel/e1000/e1000_ethtool.c
@@ -539,7 +539,9 @@ static void e1000_get_drvinfo(struct net_device *netdev,
}
static void e1000_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct e1000_adapter *adapter = netdev_priv(netdev);
struct e1000_hw *hw = &adapter->hw;
@@ -556,7 +558,9 @@ static void e1000_get_ringparam(struct net_device *netdev,
}
static int e1000_set_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct e1000_adapter *adapter = netdev_priv(netdev);
struct e1000_hw *hw = &adapter->hw;
diff --git a/drivers/net/ethernet/intel/e1000/e1000_main.c b/drivers/net/ethernet/intel/e1000/e1000_main.c
index 669060a..3f5feb5 100644
--- a/drivers/net/ethernet/intel/e1000/e1000_main.c
+++ b/drivers/net/ethernet/intel/e1000/e1000_main.c
@@ -1953,7 +1953,8 @@ void e1000_free_all_tx_resources(struct e1000_adapter *adapter)
static void
e1000_unmap_and_free_tx_resource(struct e1000_adapter *adapter,
- struct e1000_tx_buffer *buffer_info)
+ struct e1000_tx_buffer *buffer_info,
+ int budget)
{
if (buffer_info->dma) {
if (buffer_info->mapped_as_page)
@@ -1966,7 +1967,7 @@ e1000_unmap_and_free_tx_resource(struct e1000_adapter *adapter,
buffer_info->dma = 0;
}
if (buffer_info->skb) {
- dev_kfree_skb_any(buffer_info->skb);
+ napi_consume_skb(buffer_info->skb, budget);
buffer_info->skb = NULL;
}
buffer_info->time_stamp = 0;
@@ -1990,7 +1991,7 @@ static void e1000_clean_tx_ring(struct e1000_adapter *adapter,
for (i = 0; i < tx_ring->count; i++) {
buffer_info = &tx_ring->buffer_info[i];
- e1000_unmap_and_free_tx_resource(adapter, buffer_info);
+ e1000_unmap_and_free_tx_resource(adapter, buffer_info, 0);
}
netdev_reset_queue(adapter->netdev);
@@ -2958,7 +2959,7 @@ static int e1000_tx_map(struct e1000_adapter *adapter,
i += tx_ring->count;
i--;
buffer_info = &tx_ring->buffer_info[i];
- e1000_unmap_and_free_tx_resource(adapter, buffer_info);
+ e1000_unmap_and_free_tx_resource(adapter, buffer_info, 0);
}
return 0;
@@ -3856,7 +3857,8 @@ static bool e1000_clean_tx_irq(struct e1000_adapter *adapter,
}
}
- e1000_unmap_and_free_tx_resource(adapter, buffer_info);
+ e1000_unmap_and_free_tx_resource(adapter, buffer_info,
+ 64);
tx_desc->upper.data = 0;
if (unlikely(++i == tx_ring->count))
@@ -4382,7 +4384,7 @@ static bool e1000_clean_rx_irq(struct e1000_adapter *adapter,
if (!skb) {
unsigned int frag_len = e1000_frag_len(adapter);
- skb = build_skb(data - E1000_HEADROOM, frag_len);
+ skb = napi_build_skb(data - E1000_HEADROOM, frag_len);
if (!skb) {
adapter->alloc_rx_buff_failed++;
break;
diff --git a/drivers/net/ethernet/intel/e1000e/ethtool.c b/drivers/net/ethernet/intel/e1000e/ethtool.c
index 8515e00..b80ae9a 100644
--- a/drivers/net/ethernet/intel/e1000e/ethtool.c
+++ b/drivers/net/ethernet/intel/e1000e/ethtool.c
@@ -655,7 +655,9 @@ static void e1000_get_drvinfo(struct net_device *netdev,
}
static void e1000_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct e1000_adapter *adapter = netdev_priv(netdev);
@@ -666,7 +668,9 @@ static void e1000_get_ringparam(struct net_device *netdev,
}
static int e1000_set_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct e1000_adapter *adapter = netdev_priv(netdev);
struct e1000_ring *temp_tx = NULL, *temp_rx = NULL;
diff --git a/drivers/net/ethernet/intel/e1000e/netdev.c b/drivers/net/ethernet/intel/e1000e/netdev.c
index 44e2dc8..635a959 100644
--- a/drivers/net/ethernet/intel/e1000e/netdev.c
+++ b/drivers/net/ethernet/intel/e1000e/netdev.c
@@ -3614,10 +3614,6 @@ static int e1000e_config_hwtstamp(struct e1000_adapter *adapter,
if (!(adapter->flags & FLAG_HAS_HW_TIMESTAMP))
return -EINVAL;
- /* flags reserved for future extensions - must be zero */
- if (config->flags)
- return -EINVAL;
-
switch (config->tx_type) {
case HWTSTAMP_TX_OFF:
tsync_tx_ctl = 0;
diff --git a/drivers/net/ethernet/intel/fm10k/fm10k_ethtool.c b/drivers/net/ethernet/intel/fm10k/fm10k_ethtool.c
index 0d37f01..d53369e 100644
--- a/drivers/net/ethernet/intel/fm10k/fm10k_ethtool.c
+++ b/drivers/net/ethernet/intel/fm10k/fm10k_ethtool.c
@@ -502,7 +502,9 @@ static void fm10k_set_msglevel(struct net_device *netdev, u32 data)
}
static void fm10k_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct fm10k_intfc *interface = netdev_priv(netdev);
@@ -517,7 +519,9 @@ static void fm10k_get_ringparam(struct net_device *netdev,
}
static int fm10k_set_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct fm10k_intfc *interface = netdev_priv(netdev);
struct fm10k_ring *temp_ring;
diff --git a/drivers/net/ethernet/intel/fm10k/fm10k_tlv.c b/drivers/net/ethernet/intel/fm10k/fm10k_tlv.c
index 21eff08..f6d5686 100644
--- a/drivers/net/ethernet/intel/fm10k/fm10k_tlv.c
+++ b/drivers/net/ethernet/intel/fm10k/fm10k_tlv.c
@@ -143,7 +143,7 @@ s32 fm10k_tlv_attr_put_mac_vlan(u32 *msg, u16 attr_id,
* @vlan: location of buffer to store VLAN
*
* This function pulls the MAC address back out of the attribute and will
- * place it in the array pointed by by mac_addr. It will return success
+ * place it in the array pointed by mac_addr. It will return success
* if provided with a valid pointers.
**/
s32 fm10k_tlv_attr_get_mac_vlan(u32 *attr, u8 *mac_addr, u16 *vlan)
diff --git a/drivers/net/ethernet/intel/i40e/i40e_adminq.c b/drivers/net/ethernet/intel/i40e/i40e_adminq.c
index 593912b..7abef88 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_adminq.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_adminq.c
@@ -769,21 +769,22 @@ static bool i40e_asq_done(struct i40e_hw *hw)
}
/**
- * i40e_asq_send_command - send command to Admin Queue
+ * i40e_asq_send_command_atomic - send command to Admin Queue
* @hw: pointer to the hw struct
* @desc: prefilled descriptor describing the command (non DMA mem)
* @buff: buffer to use for indirect commands
* @buff_size: size of buffer for indirect commands
* @cmd_details: pointer to command details structure
+ * @is_atomic_context: is the function called in an atomic context?
*
* This is the main send command driver routine for the Admin Queue send
* queue. It runs the queue, cleans the queue, etc
**/
-i40e_status i40e_asq_send_command(struct i40e_hw *hw,
- struct i40e_aq_desc *desc,
- void *buff, /* can be NULL */
- u16 buff_size,
- struct i40e_asq_cmd_details *cmd_details)
+i40e_status
+i40e_asq_send_command_atomic(struct i40e_hw *hw, struct i40e_aq_desc *desc,
+ void *buff, /* can be NULL */ u16 buff_size,
+ struct i40e_asq_cmd_details *cmd_details,
+ bool is_atomic_context)
{
i40e_status status = 0;
struct i40e_dma_mem *dma_buff = NULL;
@@ -910,7 +911,12 @@ i40e_status i40e_asq_send_command(struct i40e_hw *hw,
*/
if (i40e_asq_done(hw))
break;
- udelay(50);
+
+ if (is_atomic_context)
+ udelay(50);
+ else
+ usleep_range(40, 60);
+
total_delay += 50;
} while (total_delay < hw->aq.asq_cmd_timeout);
}
@@ -967,6 +973,15 @@ i40e_status i40e_asq_send_command(struct i40e_hw *hw,
return status;
}
+i40e_status
+i40e_asq_send_command(struct i40e_hw *hw, struct i40e_aq_desc *desc,
+ void *buff, /* can be NULL */ u16 buff_size,
+ struct i40e_asq_cmd_details *cmd_details)
+{
+ return i40e_asq_send_command_atomic(hw, desc, buff, buff_size,
+ cmd_details, false);
+}
+
/**
* i40e_fill_default_direct_cmd_desc - AQ descriptor helper function
* @desc: pointer to the temp descriptor (non DMA mem)
diff --git a/drivers/net/ethernet/intel/i40e/i40e_adminq_cmd.h b/drivers/net/ethernet/intel/i40e/i40e_adminq_cmd.h
index 140b677..60f9e0a 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_adminq_cmd.h
+++ b/drivers/net/ethernet/intel/i40e/i40e_adminq_cmd.h
@@ -11,8 +11,8 @@
*/
#define I40E_FW_API_VERSION_MAJOR 0x0001
-#define I40E_FW_API_VERSION_MINOR_X722 0x0009
-#define I40E_FW_API_VERSION_MINOR_X710 0x0009
+#define I40E_FW_API_VERSION_MINOR_X722 0x000C
+#define I40E_FW_API_VERSION_MINOR_X710 0x000F
#define I40E_FW_MINOR_VERSION(_h) ((_h)->mac.type == I40E_MAC_XL710 ? \
I40E_FW_API_VERSION_MINOR_X710 : \
diff --git a/drivers/net/ethernet/intel/i40e/i40e_common.c b/drivers/net/ethernet/intel/i40e/i40e_common.c
index b4d3fed..9ddeb01 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_common.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_common.c
@@ -154,8 +154,8 @@ const char *i40e_stat_str(struct i40e_hw *hw, i40e_status stat_err)
return "I40E_ERR_INVALID_MAC_ADDR";
case I40E_ERR_DEVICE_NOT_SUPPORTED:
return "I40E_ERR_DEVICE_NOT_SUPPORTED";
- case I40E_ERR_MASTER_REQUESTS_PENDING:
- return "I40E_ERR_MASTER_REQUESTS_PENDING";
+ case I40E_ERR_PRIMARY_REQUESTS_PENDING:
+ return "I40E_ERR_PRIMARY_REQUESTS_PENDING";
case I40E_ERR_INVALID_LINK_SETTINGS:
return "I40E_ERR_INVALID_LINK_SETTINGS";
case I40E_ERR_AUTONEG_NOT_COMPLETE:
@@ -2073,7 +2073,8 @@ enum i40e_status_code i40e_aq_set_vsi_mc_promisc_on_vlan(struct i40e_hw *hw,
cmd->seid = cpu_to_le16(seid);
cmd->vlan_tag = cpu_to_le16(vid | I40E_AQC_SET_VSI_VLAN_VALID);
- status = i40e_asq_send_command(hw, &desc, NULL, 0, cmd_details);
+ status = i40e_asq_send_command_atomic(hw, &desc, NULL, 0,
+ cmd_details, true);
return status;
}
@@ -2114,7 +2115,8 @@ enum i40e_status_code i40e_aq_set_vsi_uc_promisc_on_vlan(struct i40e_hw *hw,
cmd->seid = cpu_to_le16(seid);
cmd->vlan_tag = cpu_to_le16(vid | I40E_AQC_SET_VSI_VLAN_VALID);
- status = i40e_asq_send_command(hw, &desc, NULL, 0, cmd_details);
+ status = i40e_asq_send_command_atomic(hw, &desc, NULL, 0,
+ cmd_details, true);
return status;
}
@@ -4136,7 +4138,6 @@ static i40e_status i40e_validate_filter_settings(struct i40e_hw *hw,
struct i40e_filter_control_settings *settings)
{
u32 fcoe_cntx_size, fcoe_filt_size;
- u32 pe_cntx_size, pe_filt_size;
u32 fcoe_fmax;
u32 val;
@@ -4180,8 +4181,6 @@ static i40e_status i40e_validate_filter_settings(struct i40e_hw *hw,
case I40E_HASH_FILTER_SIZE_256K:
case I40E_HASH_FILTER_SIZE_512K:
case I40E_HASH_FILTER_SIZE_1M:
- pe_filt_size = I40E_HASH_FILTER_BASE_SIZE;
- pe_filt_size <<= (u32)settings->pe_filt_num;
break;
default:
return I40E_ERR_PARAM;
@@ -4198,8 +4197,6 @@ static i40e_status i40e_validate_filter_settings(struct i40e_hw *hw,
case I40E_DMA_CNTX_SIZE_64K:
case I40E_DMA_CNTX_SIZE_128K:
case I40E_DMA_CNTX_SIZE_256K:
- pe_cntx_size = I40E_DMA_CNTX_BASE_SIZE;
- pe_cntx_size <<= (u32)settings->pe_cntx_num;
break;
default:
return I40E_ERR_PARAM;
diff --git a/drivers/net/ethernet/intel/i40e/i40e_ethtool.c b/drivers/net/ethernet/intel/i40e/i40e_ethtool.c
index 513ba69..091f36a 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_ethtool.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_ethtool.c
@@ -1916,7 +1916,9 @@ static void i40e_get_drvinfo(struct net_device *netdev,
}
static void i40e_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct i40e_netdev_priv *np = netdev_priv(netdev);
struct i40e_pf *pf = np->vsi->back;
@@ -1944,7 +1946,9 @@ static bool i40e_active_tx_ring_index(struct i40e_vsi *vsi, u16 index)
}
static int i40e_set_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct i40e_ring *tx_rings = NULL, *rx_rings = NULL;
struct i40e_netdev_priv *np = netdev_priv(netdev);
diff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c b/drivers/net/ethernet/intel/i40e/i40e_main.c
index e118cf9..61afc22 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_main.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_main.c
@@ -99,6 +99,24 @@ MODULE_LICENSE("GPL v2");
static struct workqueue_struct *i40e_wq;
+static void netdev_hw_addr_refcnt(struct i40e_mac_filter *f,
+ struct net_device *netdev, int delta)
+{
+ struct netdev_hw_addr *ha;
+
+ if (!f || !netdev)
+ return;
+
+ netdev_for_each_mc_addr(ha, netdev) {
+ if (ether_addr_equal(ha->addr, f->macaddr)) {
+ ha->refcount += delta;
+ if (ha->refcount <= 0)
+ ha->refcount = 1;
+ break;
+ }
+ }
+}
+
/**
* i40e_allocate_dma_mem_d - OS specific memory alloc for shared code
* @hw: pointer to the HW structure
@@ -2036,6 +2054,7 @@ static void i40e_undo_add_filter_entries(struct i40e_vsi *vsi,
hlist_for_each_entry_safe(new, h, from, hlist) {
/* We can simply free the wrapper structure */
hlist_del(&new->hlist);
+ netdev_hw_addr_refcnt(new->f, vsi->netdev, -1);
kfree(new);
}
}
@@ -2383,6 +2402,10 @@ int i40e_sync_vsi_filters(struct i40e_vsi *vsi)
&tmp_add_list,
&tmp_del_list,
vlan_filters);
+
+ hlist_for_each_entry(new, &tmp_add_list, hlist)
+ netdev_hw_addr_refcnt(new->f, vsi->netdev, 1);
+
if (retval)
goto err_no_memory_locked;
@@ -2515,6 +2538,7 @@ int i40e_sync_vsi_filters(struct i40e_vsi *vsi)
if (new->f->state == I40E_FILTER_NEW)
new->f->state = new->state;
hlist_del(&new->hlist);
+ netdev_hw_addr_refcnt(new->f, vsi->netdev, -1);
kfree(new);
}
spin_unlock_bh(&vsi->mac_filter_hash_lock);
@@ -8717,6 +8741,27 @@ int i40e_open(struct net_device *netdev)
}
/**
+ * i40e_netif_set_realnum_tx_rx_queues - Update number of tx/rx queues
+ * @vsi: vsi structure
+ *
+ * This updates netdev's number of tx/rx queues
+ *
+ * Returns status of setting tx/rx queues
+ **/
+static int i40e_netif_set_realnum_tx_rx_queues(struct i40e_vsi *vsi)
+{
+ int ret;
+
+ ret = netif_set_real_num_rx_queues(vsi->netdev,
+ vsi->num_queue_pairs);
+ if (ret)
+ return ret;
+
+ return netif_set_real_num_tx_queues(vsi->netdev,
+ vsi->num_queue_pairs);
+}
+
+/**
* i40e_vsi_open -
* @vsi: the VSI to open
*
@@ -8752,13 +8797,7 @@ int i40e_vsi_open(struct i40e_vsi *vsi)
goto err_setup_rx;
/* Notify the stack of the actual queue counts. */
- err = netif_set_real_num_tx_queues(vsi->netdev,
- vsi->num_queue_pairs);
- if (err)
- goto err_set_queues;
-
- err = netif_set_real_num_rx_queues(vsi->netdev,
- vsi->num_queue_pairs);
+ err = i40e_netif_set_realnum_tx_rx_queues(vsi);
if (err)
goto err_set_queues;
@@ -14151,6 +14190,9 @@ struct i40e_vsi *i40e_vsi_setup(struct i40e_pf *pf, u8 type,
ret = i40e_config_netdev(vsi);
if (ret)
goto err_netdev;
+ ret = i40e_netif_set_realnum_tx_rx_queues(vsi);
+ if (ret)
+ goto err_netdev;
ret = register_netdev(vsi->netdev);
if (ret)
goto err_netdev;
@@ -15451,8 +15493,8 @@ static int i40e_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
if (hw->aq.api_maj_ver == I40E_FW_API_VERSION_MAJOR &&
hw->aq.api_min_ver > I40E_FW_MINOR_VERSION(hw))
- dev_info(&pdev->dev,
- "The driver for the device detected a newer version of the NVM image v%u.%u than expected v%u.%u. Please install the most recent version of the network driver.\n",
+ dev_dbg(&pdev->dev,
+ "The driver for the device detected a newer version of the NVM image v%u.%u than v%u.%u.\n",
hw->aq.api_maj_ver,
hw->aq.api_min_ver,
I40E_FW_API_VERSION_MAJOR,
diff --git a/drivers/net/ethernet/intel/i40e/i40e_prototype.h b/drivers/net/ethernet/intel/i40e/i40e_prototype.h
index aaea297..9241b60 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_prototype.h
+++ b/drivers/net/ethernet/intel/i40e/i40e_prototype.h
@@ -22,11 +22,15 @@ void i40e_adminq_init_ring_data(struct i40e_hw *hw);
i40e_status i40e_clean_arq_element(struct i40e_hw *hw,
struct i40e_arq_event_info *e,
u16 *events_pending);
-i40e_status i40e_asq_send_command(struct i40e_hw *hw,
- struct i40e_aq_desc *desc,
- void *buff, /* can be NULL */
- u16 buff_size,
- struct i40e_asq_cmd_details *cmd_details);
+i40e_status
+i40e_asq_send_command(struct i40e_hw *hw, struct i40e_aq_desc *desc,
+ void *buff, /* can be NULL */ u16 buff_size,
+ struct i40e_asq_cmd_details *cmd_details);
+i40e_status
+i40e_asq_send_command_atomic(struct i40e_hw *hw, struct i40e_aq_desc *desc,
+ void *buff, /* can be NULL */ u16 buff_size,
+ struct i40e_asq_cmd_details *cmd_details,
+ bool is_atomic_context);
/* debug function for adminq */
void i40e_debug_aq(struct i40e_hw *hw, enum i40e_debug_mask mask,
diff --git a/drivers/net/ethernet/intel/i40e/i40e_ptp.c b/drivers/net/ethernet/intel/i40e/i40e_ptp.c
index 09b1d5a..61e5789 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_ptp.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_ptp.c
@@ -1205,10 +1205,6 @@ static int i40e_ptp_set_timestamp_mode(struct i40e_pf *pf,
INIT_WORK(&pf->ptp_extts0_work, i40e_ptp_extts0_work);
- /* Reserved for future extensions. */
- if (config->flags)
- return -EINVAL;
-
switch (config->tx_type) {
case HWTSTAMP_TX_OFF:
pf->ptp_tx = false;
diff --git a/drivers/net/ethernet/intel/i40e/i40e_status.h b/drivers/net/ethernet/intel/i40e/i40e_status.h
index 77be070..db3714a 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_status.h
+++ b/drivers/net/ethernet/intel/i40e/i40e_status.h
@@ -18,7 +18,7 @@ enum i40e_status_code {
I40E_ERR_ADAPTER_STOPPED = -9,
I40E_ERR_INVALID_MAC_ADDR = -10,
I40E_ERR_DEVICE_NOT_SUPPORTED = -11,
- I40E_ERR_MASTER_REQUESTS_PENDING = -12,
+ I40E_ERR_PRIMARY_REQUESTS_PENDING = -12,
I40E_ERR_INVALID_LINK_SETTINGS = -13,
I40E_ERR_AUTONEG_NOT_COMPLETE = -14,
I40E_ERR_RESET_FAILED = -15,
diff --git a/drivers/net/ethernet/intel/i40e/i40e_txrx.c b/drivers/net/ethernet/intel/i40e/i40e_txrx.c
index 10a83e5..66cc795 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_txrx.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_txrx.c
@@ -2204,7 +2204,7 @@ static struct sk_buff *i40e_build_skb(struct i40e_ring *rx_ring,
net_prefetch(xdp->data_meta);
/* build an skb around the page buffer */
- skb = build_skb(xdp->data_hard_start, truesize);
+ skb = napi_build_skb(xdp->data_hard_start, truesize);
if (unlikely(!skb))
return NULL;
@@ -2322,7 +2322,7 @@ static int i40e_run_xdp(struct i40e_ring *rx_ring, struct xdp_buff *xdp)
result = I40E_XDP_REDIR;
break;
default:
- bpf_warn_invalid_xdp_action(act);
+ bpf_warn_invalid_xdp_action(rx_ring->netdev, xdp_prog, act);
fallthrough;
case XDP_ABORTED:
out_failure:
diff --git a/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c b/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c
index 2ea4deb..b785d09 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c
@@ -1877,17 +1877,19 @@ int i40e_pci_sriov_configure(struct pci_dev *pdev, int num_vfs)
/***********************virtual channel routines******************/
/**
- * i40e_vc_send_msg_to_vf
+ * i40e_vc_send_msg_to_vf_ex
* @vf: pointer to the VF info
* @v_opcode: virtual channel opcode
* @v_retval: virtual channel return value
* @msg: pointer to the msg buffer
* @msglen: msg length
+ * @is_quiet: true for not printing unsuccessful return values, false otherwise
*
* send msg to VF
**/
-static int i40e_vc_send_msg_to_vf(struct i40e_vf *vf, u32 v_opcode,
- u32 v_retval, u8 *msg, u16 msglen)
+static int i40e_vc_send_msg_to_vf_ex(struct i40e_vf *vf, u32 v_opcode,
+ u32 v_retval, u8 *msg, u16 msglen,
+ bool is_quiet)
{
struct i40e_pf *pf;
struct i40e_hw *hw;
@@ -1903,7 +1905,7 @@ static int i40e_vc_send_msg_to_vf(struct i40e_vf *vf, u32 v_opcode,
abs_vf_id = vf->vf_id + hw->func_caps.vf_base_id;
/* single place to detect unsuccessful return values */
- if (v_retval) {
+ if (v_retval && !is_quiet) {
vf->num_invalid_msgs++;
dev_info(&pf->pdev->dev, "VF %d failed opcode %d, retval: %d\n",
vf->vf_id, v_opcode, v_retval);
@@ -1934,6 +1936,23 @@ static int i40e_vc_send_msg_to_vf(struct i40e_vf *vf, u32 v_opcode,
}
/**
+ * i40e_vc_send_msg_to_vf
+ * @vf: pointer to the VF info
+ * @v_opcode: virtual channel opcode
+ * @v_retval: virtual channel return value
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ *
+ * send msg to VF
+ **/
+static int i40e_vc_send_msg_to_vf(struct i40e_vf *vf, u32 v_opcode,
+ u32 v_retval, u8 *msg, u16 msglen)
+{
+ return i40e_vc_send_msg_to_vf_ex(vf, v_opcode, v_retval,
+ msg, msglen, false);
+}
+
+/**
* i40e_vc_send_resp_to_vf
* @vf: pointer to the VF info
* @opcode: operation code
@@ -2685,16 +2704,26 @@ static int i40e_vc_get_stats_msg(struct i40e_vf *vf, u8 *msg)
(u8 *)&stats, sizeof(stats));
}
+#define I40E_MAX_MACVLAN_PER_HW 3072
+#define I40E_MAX_MACVLAN_PER_PF(num_ports) (I40E_MAX_MACVLAN_PER_HW / \
+ (num_ports))
/* If the VF is not trusted restrict the number of MAC/VLAN it can program
* MAC filters: 16 for multicast, 1 for MAC, 1 for broadcast
*/
#define I40E_VC_MAX_MAC_ADDR_PER_VF (16 + 1 + 1)
#define I40E_VC_MAX_VLAN_PER_VF 16
+#define I40E_VC_MAX_MACVLAN_PER_TRUSTED_VF(vf_num, num_ports) \
+({ typeof(vf_num) vf_num_ = (vf_num); \
+ typeof(num_ports) num_ports_ = (num_ports); \
+ ((I40E_MAX_MACVLAN_PER_PF(num_ports_) - vf_num_ * \
+ I40E_VC_MAX_MAC_ADDR_PER_VF) / vf_num_) + \
+ I40E_VC_MAX_MAC_ADDR_PER_VF; })
/**
* i40e_check_vf_permission
* @vf: pointer to the VF info
* @al: MAC address list from virtchnl
+ * @is_quiet: set true for printing msg without opcode info, false otherwise
*
* Check that the given list of MAC addresses is allowed. Will return -EPERM
* if any address in the list is not valid. Checks the following conditions:
@@ -2709,13 +2738,16 @@ static int i40e_vc_get_stats_msg(struct i40e_vf *vf, u8 *msg)
* addresses might not be accurate.
**/
static inline int i40e_check_vf_permission(struct i40e_vf *vf,
- struct virtchnl_ether_addr_list *al)
+ struct virtchnl_ether_addr_list *al,
+ bool *is_quiet)
{
struct i40e_pf *pf = vf->pf;
struct i40e_vsi *vsi = pf->vsi[vf->lan_vsi_idx];
+ struct i40e_hw *hw = &pf->hw;
int mac2add_cnt = 0;
int i;
+ *is_quiet = false;
for (i = 0; i < al->num_elements; i++) {
struct i40e_mac_filter *f;
u8 *addr = al->list[i].addr;
@@ -2739,6 +2771,7 @@ static inline int i40e_check_vf_permission(struct i40e_vf *vf,
!ether_addr_equal(addr, vf->default_lan_addr.addr)) {
dev_err(&pf->pdev->dev,
"VF attempting to override administratively set MAC address, bring down and up the VF interface to resume normal operation\n");
+ *is_quiet = true;
return -EPERM;
}
@@ -2752,12 +2785,26 @@ static inline int i40e_check_vf_permission(struct i40e_vf *vf,
* number of addresses. Check to make sure that the additions do not
* push us over the limit.
*/
- if (!test_bit(I40E_VIRTCHNL_VF_CAP_PRIVILEGE, &vf->vf_caps) &&
- (i40e_count_filters(vsi) + mac2add_cnt) >
+ if (!test_bit(I40E_VIRTCHNL_VF_CAP_PRIVILEGE, &vf->vf_caps)) {
+ if ((i40e_count_filters(vsi) + mac2add_cnt) >
I40E_VC_MAX_MAC_ADDR_PER_VF) {
- dev_err(&pf->pdev->dev,
- "Cannot add more MAC addresses, VF is not trusted, switch the VF to trusted to add more functionality\n");
- return -EPERM;
+ dev_err(&pf->pdev->dev,
+ "Cannot add more MAC addresses, VF is not trusted, switch the VF to trusted to add more functionality\n");
+ return -EPERM;
+ }
+ /* If this VF is trusted, it can use more resources than untrusted.
+ * However to ensure that every trusted VF has appropriate number of
+ * resources, divide whole pool of resources per port and then across
+ * all VFs.
+ */
+ } else {
+ if ((i40e_count_filters(vsi) + mac2add_cnt) >
+ I40E_VC_MAX_MACVLAN_PER_TRUSTED_VF(pf->num_alloc_vfs,
+ hw->num_ports)) {
+ dev_err(&pf->pdev->dev,
+ "Cannot add more MAC addresses, trusted VF exhausted it's resources\n");
+ return -EPERM;
+ }
}
return 0;
}
@@ -2775,6 +2822,7 @@ static int i40e_vc_add_mac_addr_msg(struct i40e_vf *vf, u8 *msg)
(struct virtchnl_ether_addr_list *)msg;
struct i40e_pf *pf = vf->pf;
struct i40e_vsi *vsi = NULL;
+ bool is_quiet = false;
i40e_status ret = 0;
int i;
@@ -2791,7 +2839,7 @@ static int i40e_vc_add_mac_addr_msg(struct i40e_vf *vf, u8 *msg)
*/
spin_lock_bh(&vsi->mac_filter_hash_lock);
- ret = i40e_check_vf_permission(vf, al);
+ ret = i40e_check_vf_permission(vf, al, &is_quiet);
if (ret) {
spin_unlock_bh(&vsi->mac_filter_hash_lock);
goto error_param;
@@ -2829,8 +2877,8 @@ static int i40e_vc_add_mac_addr_msg(struct i40e_vf *vf, u8 *msg)
error_param:
/* send the response to the VF */
- return i40e_vc_send_resp_to_vf(vf, VIRTCHNL_OP_ADD_ETH_ADDR,
- ret);
+ return i40e_vc_send_msg_to_vf_ex(vf, VIRTCHNL_OP_ADD_ETH_ADDR,
+ ret, NULL, 0, is_quiet);
}
/**
diff --git a/drivers/net/ethernet/intel/i40e/i40e_xsk.c b/drivers/net/ethernet/intel/i40e/i40e_xsk.c
index ea06e95..945b1bb 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_xsk.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_xsk.c
@@ -176,7 +176,7 @@ static int i40e_run_xdp_zc(struct i40e_ring *rx_ring, struct xdp_buff *xdp)
goto out_failure;
break;
default:
- bpf_warn_invalid_xdp_action(act);
+ bpf_warn_invalid_xdp_action(rx_ring->netdev, xdp_prog, act);
fallthrough;
case XDP_ABORTED:
out_failure:
diff --git a/drivers/net/ethernet/intel/i40e/i40e_xsk.h b/drivers/net/ethernet/intel/i40e/i40e_xsk.h
index ea88f45..bb96298 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_xsk.h
+++ b/drivers/net/ethernet/intel/i40e/i40e_xsk.h
@@ -22,7 +22,6 @@
struct i40e_vsi;
struct xsk_buff_pool;
-struct zero_copy_allocator;
int i40e_queue_pair_disable(struct i40e_vsi *vsi, int queue_pair);
int i40e_queue_pair_enable(struct i40e_vsi *vsi, int queue_pair);
diff --git a/drivers/net/ethernet/intel/iavf/iavf.h b/drivers/net/ethernet/intel/iavf/iavf.h
index 3789269..59806d1 100644
--- a/drivers/net/ethernet/intel/iavf/iavf.h
+++ b/drivers/net/ethernet/intel/iavf/iavf.h
@@ -55,7 +55,8 @@ enum iavf_vsi_state_t {
struct iavf_vsi {
struct iavf_adapter *back;
struct net_device *netdev;
- unsigned long active_vlans[BITS_TO_LONGS(VLAN_N_VID)];
+ unsigned long active_cvlans[BITS_TO_LONGS(VLAN_N_VID)];
+ unsigned long active_svlans[BITS_TO_LONGS(VLAN_N_VID)];
u16 seid;
u16 id;
DECLARE_BITMAP(state, __IAVF_VSI_STATE_SIZE__);
@@ -137,14 +138,24 @@ struct iavf_q_vector {
struct iavf_mac_filter {
struct list_head list;
u8 macaddr[ETH_ALEN];
- bool is_new_mac; /* filter is new, wait for PF decision */
- bool remove; /* filter needs to be removed */
- bool add; /* filter needs to be added */
+ struct {
+ u8 is_new_mac:1; /* filter is new, wait for PF decision */
+ u8 remove:1; /* filter needs to be removed */
+ u8 add:1; /* filter needs to be added */
+ u8 is_primary:1; /* filter is a default VF MAC */
+ u8 padding:4;
+ };
+};
+
+#define IAVF_VLAN(vid, tpid) ((struct iavf_vlan){ vid, tpid })
+struct iavf_vlan {
+ u16 vid;
+ u16 tpid;
};
struct iavf_vlan_filter {
struct list_head list;
- u16 vlan;
+ struct iavf_vlan vlan;
bool remove; /* filter needs to be removed */
bool add; /* filter needs to be added */
};
@@ -177,6 +188,8 @@ enum iavf_state_t {
__IAVF_REMOVE, /* driver is being unloaded */
__IAVF_INIT_VERSION_CHECK, /* aq msg sent, awaiting reply */
__IAVF_INIT_GET_RESOURCES, /* aq msg sent, awaiting reply */
+ __IAVF_INIT_GET_OFFLOAD_VLAN_V2_CAPS,
+ __IAVF_INIT_CONFIG_ADAPTER,
__IAVF_INIT_SW, /* got resources, setting up structs */
__IAVF_INIT_FAILED, /* init failed, restarting procedure */
__IAVF_RESETTING, /* in reset */
@@ -274,38 +287,47 @@ struct iavf_adapter {
/* duplicates for common code */
#define IAVF_FLAG_DCB_ENABLED 0
/* flags for admin queue service task */
- u32 aq_required;
-#define IAVF_FLAG_AQ_ENABLE_QUEUES BIT(0)
-#define IAVF_FLAG_AQ_DISABLE_QUEUES BIT(1)
-#define IAVF_FLAG_AQ_ADD_MAC_FILTER BIT(2)
-#define IAVF_FLAG_AQ_ADD_VLAN_FILTER BIT(3)
-#define IAVF_FLAG_AQ_DEL_MAC_FILTER BIT(4)
-#define IAVF_FLAG_AQ_DEL_VLAN_FILTER BIT(5)
-#define IAVF_FLAG_AQ_CONFIGURE_QUEUES BIT(6)
-#define IAVF_FLAG_AQ_MAP_VECTORS BIT(7)
-#define IAVF_FLAG_AQ_HANDLE_RESET BIT(8)
-#define IAVF_FLAG_AQ_CONFIGURE_RSS BIT(9) /* direct AQ config */
-#define IAVF_FLAG_AQ_GET_CONFIG BIT(10)
+ u64 aq_required;
+#define IAVF_FLAG_AQ_ENABLE_QUEUES BIT_ULL(0)
+#define IAVF_FLAG_AQ_DISABLE_QUEUES BIT_ULL(1)
+#define IAVF_FLAG_AQ_ADD_MAC_FILTER BIT_ULL(2)
+#define IAVF_FLAG_AQ_ADD_VLAN_FILTER BIT_ULL(3)
+#define IAVF_FLAG_AQ_DEL_MAC_FILTER BIT_ULL(4)
+#define IAVF_FLAG_AQ_DEL_VLAN_FILTER BIT_ULL(5)
+#define IAVF_FLAG_AQ_CONFIGURE_QUEUES BIT_ULL(6)
+#define IAVF_FLAG_AQ_MAP_VECTORS BIT_ULL(7)
+#define IAVF_FLAG_AQ_HANDLE_RESET BIT_ULL(8)
+#define IAVF_FLAG_AQ_CONFIGURE_RSS BIT_ULL(9) /* direct AQ config */
+#define IAVF_FLAG_AQ_GET_CONFIG BIT_ULL(10)
/* Newer style, RSS done by the PF so we can ignore hardware vagaries. */
-#define IAVF_FLAG_AQ_GET_HENA BIT(11)
-#define IAVF_FLAG_AQ_SET_HENA BIT(12)
-#define IAVF_FLAG_AQ_SET_RSS_KEY BIT(13)
-#define IAVF_FLAG_AQ_SET_RSS_LUT BIT(14)
-#define IAVF_FLAG_AQ_REQUEST_PROMISC BIT(15)
-#define IAVF_FLAG_AQ_RELEASE_PROMISC BIT(16)
-#define IAVF_FLAG_AQ_REQUEST_ALLMULTI BIT(17)
-#define IAVF_FLAG_AQ_RELEASE_ALLMULTI BIT(18)
-#define IAVF_FLAG_AQ_ENABLE_VLAN_STRIPPING BIT(19)
-#define IAVF_FLAG_AQ_DISABLE_VLAN_STRIPPING BIT(20)
-#define IAVF_FLAG_AQ_ENABLE_CHANNELS BIT(21)
-#define IAVF_FLAG_AQ_DISABLE_CHANNELS BIT(22)
-#define IAVF_FLAG_AQ_ADD_CLOUD_FILTER BIT(23)
-#define IAVF_FLAG_AQ_DEL_CLOUD_FILTER BIT(24)
-#define IAVF_FLAG_AQ_ADD_FDIR_FILTER BIT(25)
-#define IAVF_FLAG_AQ_DEL_FDIR_FILTER BIT(26)
-#define IAVF_FLAG_AQ_ADD_ADV_RSS_CFG BIT(27)
-#define IAVF_FLAG_AQ_DEL_ADV_RSS_CFG BIT(28)
-#define IAVF_FLAG_AQ_REQUEST_STATS BIT(29)
+#define IAVF_FLAG_AQ_GET_HENA BIT_ULL(11)
+#define IAVF_FLAG_AQ_SET_HENA BIT_ULL(12)
+#define IAVF_FLAG_AQ_SET_RSS_KEY BIT_ULL(13)
+#define IAVF_FLAG_AQ_SET_RSS_LUT BIT_ULL(14)
+#define IAVF_FLAG_AQ_REQUEST_PROMISC BIT_ULL(15)
+#define IAVF_FLAG_AQ_RELEASE_PROMISC BIT_ULL(16)
+#define IAVF_FLAG_AQ_REQUEST_ALLMULTI BIT_ULL(17)
+#define IAVF_FLAG_AQ_RELEASE_ALLMULTI BIT_ULL(18)
+#define IAVF_FLAG_AQ_ENABLE_VLAN_STRIPPING BIT_ULL(19)
+#define IAVF_FLAG_AQ_DISABLE_VLAN_STRIPPING BIT_ULL(20)
+#define IAVF_FLAG_AQ_ENABLE_CHANNELS BIT_ULL(21)
+#define IAVF_FLAG_AQ_DISABLE_CHANNELS BIT_ULL(22)
+#define IAVF_FLAG_AQ_ADD_CLOUD_FILTER BIT_ULL(23)
+#define IAVF_FLAG_AQ_DEL_CLOUD_FILTER BIT_ULL(24)
+#define IAVF_FLAG_AQ_ADD_FDIR_FILTER BIT_ULL(25)
+#define IAVF_FLAG_AQ_DEL_FDIR_FILTER BIT_ULL(26)
+#define IAVF_FLAG_AQ_ADD_ADV_RSS_CFG BIT_ULL(27)
+#define IAVF_FLAG_AQ_DEL_ADV_RSS_CFG BIT_ULL(28)
+#define IAVF_FLAG_AQ_REQUEST_STATS BIT_ULL(29)
+#define IAVF_FLAG_AQ_GET_OFFLOAD_VLAN_V2_CAPS BIT_ULL(30)
+#define IAVF_FLAG_AQ_ENABLE_CTAG_VLAN_STRIPPING BIT_ULL(31)
+#define IAVF_FLAG_AQ_DISABLE_CTAG_VLAN_STRIPPING BIT_ULL(32)
+#define IAVF_FLAG_AQ_ENABLE_STAG_VLAN_STRIPPING BIT_ULL(33)
+#define IAVF_FLAG_AQ_DISABLE_STAG_VLAN_STRIPPING BIT_ULL(34)
+#define IAVF_FLAG_AQ_ENABLE_CTAG_VLAN_INSERTION BIT_ULL(35)
+#define IAVF_FLAG_AQ_DISABLE_CTAG_VLAN_INSERTION BIT_ULL(36)
+#define IAVF_FLAG_AQ_ENABLE_STAG_VLAN_INSERTION BIT_ULL(37)
+#define IAVF_FLAG_AQ_DISABLE_STAG_VLAN_INSERTION BIT_ULL(38)
/* OS defined structs */
struct net_device *netdev;
@@ -345,6 +367,14 @@ struct iavf_adapter {
VIRTCHNL_VF_OFFLOAD_RSS_PF)))
#define VLAN_ALLOWED(_a) ((_a)->vf_res->vf_cap_flags & \
VIRTCHNL_VF_OFFLOAD_VLAN)
+#define VLAN_V2_ALLOWED(_a) ((_a)->vf_res->vf_cap_flags & \
+ VIRTCHNL_VF_OFFLOAD_VLAN_V2)
+#define VLAN_V2_FILTERING_ALLOWED(_a) \
+ (VLAN_V2_ALLOWED((_a)) && \
+ ((_a)->vlan_v2_caps.filtering.filtering_support.outer || \
+ (_a)->vlan_v2_caps.filtering.filtering_support.inner))
+#define VLAN_FILTERING_ALLOWED(_a) \
+ (VLAN_ALLOWED((_a)) || VLAN_V2_FILTERING_ALLOWED((_a)))
#define ADV_LINK_SUPPORT(_a) ((_a)->vf_res->vf_cap_flags & \
VIRTCHNL_VF_CAP_ADV_LINK_SPEED)
#define FDIR_FLTR_SUPPORT(_a) ((_a)->vf_res->vf_cap_flags & \
@@ -356,6 +386,7 @@ struct iavf_adapter {
struct virtchnl_version_info pf_version;
#define PF_IS_V11(_a) (((_a)->pf_version.major == 1) && \
((_a)->pf_version.minor == 1))
+ struct virtchnl_vlan_caps vlan_v2_caps;
u16 msg_enable;
struct iavf_eth_stats current_stats;
struct iavf_vsi vsi;
@@ -444,6 +475,7 @@ static inline void iavf_change_state(struct iavf_adapter *adapter,
int iavf_up(struct iavf_adapter *adapter);
void iavf_down(struct iavf_adapter *adapter);
int iavf_process_config(struct iavf_adapter *adapter);
+int iavf_parse_vf_resource_msg(struct iavf_adapter *adapter);
void iavf_schedule_reset(struct iavf_adapter *adapter);
void iavf_schedule_request_stats(struct iavf_adapter *adapter);
void iavf_reset(struct iavf_adapter *adapter);
@@ -462,6 +494,9 @@ int iavf_send_api_ver(struct iavf_adapter *adapter);
int iavf_verify_api_ver(struct iavf_adapter *adapter);
int iavf_send_vf_config_msg(struct iavf_adapter *adapter);
int iavf_get_vf_config(struct iavf_adapter *adapter);
+int iavf_get_vf_vlan_v2_caps(struct iavf_adapter *adapter);
+int iavf_send_vf_offload_vlan_v2_msg(struct iavf_adapter *adapter);
+void iavf_set_queue_vlan_tag_loc(struct iavf_adapter *adapter);
void iavf_irq_enable(struct iavf_adapter *adapter, bool flush);
void iavf_configure_queues(struct iavf_adapter *adapter);
void iavf_deconfigure_queues(struct iavf_adapter *adapter);
@@ -497,6 +532,14 @@ void iavf_enable_channels(struct iavf_adapter *adapter);
void iavf_disable_channels(struct iavf_adapter *adapter);
void iavf_add_cloud_filter(struct iavf_adapter *adapter);
void iavf_del_cloud_filter(struct iavf_adapter *adapter);
+void iavf_enable_vlan_stripping_v2(struct iavf_adapter *adapter, u16 tpid);
+void iavf_disable_vlan_stripping_v2(struct iavf_adapter *adapter, u16 tpid);
+void iavf_enable_vlan_insertion_v2(struct iavf_adapter *adapter, u16 tpid);
+void iavf_disable_vlan_insertion_v2(struct iavf_adapter *adapter, u16 tpid);
+void
+iavf_set_vlan_offload_features(struct iavf_adapter *adapter,
+ netdev_features_t prev_features,
+ netdev_features_t features);
void iavf_add_fdir_filter(struct iavf_adapter *adapter);
void iavf_del_fdir_filter(struct iavf_adapter *adapter);
void iavf_add_adv_rss_cfg(struct iavf_adapter *adapter);
diff --git a/drivers/net/ethernet/intel/iavf/iavf_adminq.c b/drivers/net/ethernet/intel/iavf/iavf_adminq.c
index 9fa3fa99..cd4e6a2 100644
--- a/drivers/net/ethernet/intel/iavf/iavf_adminq.c
+++ b/drivers/net/ethernet/intel/iavf/iavf_adminq.c
@@ -551,15 +551,13 @@ enum iavf_status iavf_init_adminq(struct iavf_hw *hw)
**/
enum iavf_status iavf_shutdown_adminq(struct iavf_hw *hw)
{
- enum iavf_status ret_code = 0;
-
if (iavf_check_asq_alive(hw))
iavf_aq_queue_shutdown(hw, true);
iavf_shutdown_asq(hw);
iavf_shutdown_arq(hw);
- return ret_code;
+ return 0;
}
/**
diff --git a/drivers/net/ethernet/intel/iavf/iavf_ethtool.c b/drivers/net/ethernet/intel/iavf/iavf_ethtool.c
index 461f523..3bb5671 100644
--- a/drivers/net/ethernet/intel/iavf/iavf_ethtool.c
+++ b/drivers/net/ethernet/intel/iavf/iavf_ethtool.c
@@ -331,9 +331,16 @@ static int iavf_get_link_ksettings(struct net_device *netdev,
**/
static int iavf_get_sset_count(struct net_device *netdev, int sset)
{
+ /* Report the maximum number queues, even if not every queue is
+ * currently configured. Since allocation of queues is in pairs,
+ * use netdev->real_num_tx_queues * 2. The real_num_tx_queues is set
+ * at device creation and never changes.
+ */
+
if (sset == ETH_SS_STATS)
return IAVF_STATS_LEN +
- (IAVF_QUEUE_STATS_LEN * 2 * IAVF_MAX_REQ_QUEUES);
+ (IAVF_QUEUE_STATS_LEN * 2 *
+ netdev->real_num_tx_queues);
else if (sset == ETH_SS_PRIV_FLAGS)
return IAVF_PRIV_FLAGS_STR_LEN;
else
@@ -360,17 +367,18 @@ static void iavf_get_ethtool_stats(struct net_device *netdev,
iavf_add_ethtool_stats(&data, adapter, iavf_gstrings_stats);
rcu_read_lock();
- for (i = 0; i < IAVF_MAX_REQ_QUEUES; i++) {
+ /* As num_active_queues describe both tx and rx queues, we can use
+ * it to iterate over rings' stats.
+ */
+ for (i = 0; i < adapter->num_active_queues; i++) {
struct iavf_ring *ring;
- /* Avoid accessing un-allocated queues */
- ring = (i < adapter->num_active_queues ?
- &adapter->tx_rings[i] : NULL);
+ /* Tx rings stats */
+ ring = &adapter->tx_rings[i];
iavf_add_queue_stats(&data, ring);
- /* Avoid accessing un-allocated queues */
- ring = (i < adapter->num_active_queues ?
- &adapter->rx_rings[i] : NULL);
+ /* Rx rings stats */
+ ring = &adapter->rx_rings[i];
iavf_add_queue_stats(&data, ring);
}
rcu_read_unlock();
@@ -407,10 +415,10 @@ static void iavf_get_stat_strings(struct net_device *netdev, u8 *data)
iavf_add_stat_strings(&data, iavf_gstrings_stats);
- /* Queues are always allocated in pairs, so we just use num_tx_queues
- * for both Tx and Rx queues.
+ /* Queues are always allocated in pairs, so we just use
+ * real_num_tx_queues for both Tx and Rx queues.
*/
- for (i = 0; i < netdev->num_tx_queues; i++) {
+ for (i = 0; i < netdev->real_num_tx_queues; i++) {
iavf_add_stat_strings(&data, iavf_gstrings_queue_stats,
"tx", i);
iavf_add_stat_strings(&data, iavf_gstrings_queue_stats,
@@ -583,12 +591,16 @@ static void iavf_get_drvinfo(struct net_device *netdev,
* iavf_get_ringparam - Get ring parameters
* @netdev: network interface device structure
* @ring: ethtool ringparam structure
+ * @kernel_ring: ethtool extenal ringparam structure
+ * @extack: netlink extended ACK report struct
*
* Returns current ring parameters. TX and RX rings are reported separately,
* but the number of rings is not reported.
**/
static void iavf_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct iavf_adapter *adapter = netdev_priv(netdev);
@@ -602,12 +614,16 @@ static void iavf_get_ringparam(struct net_device *netdev,
* iavf_set_ringparam - Set ring parameters
* @netdev: network interface device structure
* @ring: ethtool ringparam structure
+ * @kernel_ring: ethtool external ringparam structure
+ * @extack: netlink extended ACK report struct
*
* Sets ring parameters. TX and RX rings are controlled separately, but the
* number of rings is not specified, so all rings get the same settings.
**/
static int iavf_set_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct iavf_adapter *adapter = netdev_priv(netdev);
u32 new_rx_count, new_tx_count;
@@ -1923,7 +1939,7 @@ static int iavf_get_rxfh(struct net_device *netdev, u32 *indir, u8 *key,
* @key: hash key
* @hfunc: hash function to use
*
- * Returns -EINVAL if the table specifies an inavlid queue id, otherwise
+ * Returns -EINVAL if the table specifies an invalid queue id, otherwise
* returns 0 after programming the table.
**/
static int iavf_set_rxfh(struct net_device *netdev, const u32 *indir,
@@ -1932,19 +1948,21 @@ static int iavf_set_rxfh(struct net_device *netdev, const u32 *indir,
struct iavf_adapter *adapter = netdev_priv(netdev);
u16 i;
- /* We do not allow change in unsupported parameters */
- if (key ||
- (hfunc != ETH_RSS_HASH_NO_CHANGE && hfunc != ETH_RSS_HASH_TOP))
+ /* Only support toeplitz hash function */
+ if (hfunc != ETH_RSS_HASH_NO_CHANGE && hfunc != ETH_RSS_HASH_TOP)
return -EOPNOTSUPP;
- if (!indir)
+
+ if (!key && !indir)
return 0;
if (key)
memcpy(adapter->rss_key, key, adapter->rss_key_size);
- /* Each 32 bits pointed by 'indir' is stored with a lut entry */
- for (i = 0; i < adapter->rss_lut_size; i++)
- adapter->rss_lut[i] = (u8)(indir[i]);
+ if (indir) {
+ /* Each 32 bits pointed by 'indir' is stored with a lut entry */
+ for (i = 0; i < adapter->rss_lut_size; i++)
+ adapter->rss_lut[i] = (u8)(indir[i]);
+ }
return iavf_config_rss(adapter);
}
diff --git a/drivers/net/ethernet/intel/iavf/iavf_main.c b/drivers/net/ethernet/intel/iavf/iavf_main.c
index 4e7c040..95116ef 100644
--- a/drivers/net/ethernet/intel/iavf/iavf_main.c
+++ b/drivers/net/ethernet/intel/iavf/iavf_main.c
@@ -463,14 +463,14 @@ iavf_request_traffic_irqs(struct iavf_adapter *adapter, char *basename)
if (q_vector->tx.ring && q_vector->rx.ring) {
snprintf(q_vector->name, sizeof(q_vector->name),
- "iavf-%s-TxRx-%d", basename, rx_int_idx++);
+ "iavf-%s-TxRx-%u", basename, rx_int_idx++);
tx_int_idx++;
} else if (q_vector->rx.ring) {
snprintf(q_vector->name, sizeof(q_vector->name),
- "iavf-%s-rx-%d", basename, rx_int_idx++);
+ "iavf-%s-rx-%u", basename, rx_int_idx++);
} else if (q_vector->tx.ring) {
snprintf(q_vector->name, sizeof(q_vector->name),
- "iavf-%s-tx-%d", basename, tx_int_idx++);
+ "iavf-%s-tx-%u", basename, tx_int_idx++);
} else {
/* skip this unused q_vector */
continue;
@@ -646,14 +646,17 @@ static void iavf_configure_rx(struct iavf_adapter *adapter)
* mac_vlan_list_lock.
**/
static struct
-iavf_vlan_filter *iavf_find_vlan(struct iavf_adapter *adapter, u16 vlan)
+iavf_vlan_filter *iavf_find_vlan(struct iavf_adapter *adapter,
+ struct iavf_vlan vlan)
{
struct iavf_vlan_filter *f;
list_for_each_entry(f, &adapter->vlan_filter_list, list) {
- if (vlan == f->vlan)
+ if (f->vlan.vid == vlan.vid &&
+ f->vlan.tpid == vlan.tpid)
return f;
}
+
return NULL;
}
@@ -665,7 +668,8 @@ iavf_vlan_filter *iavf_find_vlan(struct iavf_adapter *adapter, u16 vlan)
* Returns ptr to the filter object or NULL when no memory available.
**/
static struct
-iavf_vlan_filter *iavf_add_vlan(struct iavf_adapter *adapter, u16 vlan)
+iavf_vlan_filter *iavf_add_vlan(struct iavf_adapter *adapter,
+ struct iavf_vlan vlan)
{
struct iavf_vlan_filter *f = NULL;
@@ -694,7 +698,7 @@ iavf_vlan_filter *iavf_add_vlan(struct iavf_adapter *adapter, u16 vlan)
* @adapter: board private structure
* @vlan: VLAN tag
**/
-static void iavf_del_vlan(struct iavf_adapter *adapter, u16 vlan)
+static void iavf_del_vlan(struct iavf_adapter *adapter, struct iavf_vlan vlan)
{
struct iavf_vlan_filter *f;
@@ -720,8 +724,55 @@ static void iavf_restore_filters(struct iavf_adapter *adapter)
u16 vid;
/* re-add all VLAN filters */
- for_each_set_bit(vid, adapter->vsi.active_vlans, VLAN_N_VID)
- iavf_add_vlan(adapter, vid);
+ for_each_set_bit(vid, adapter->vsi.active_cvlans, VLAN_N_VID)
+ iavf_add_vlan(adapter, IAVF_VLAN(vid, ETH_P_8021Q));
+
+ for_each_set_bit(vid, adapter->vsi.active_svlans, VLAN_N_VID)
+ iavf_add_vlan(adapter, IAVF_VLAN(vid, ETH_P_8021AD));
+}
+
+/**
+ * iavf_get_num_vlans_added - get number of VLANs added
+ * @adapter: board private structure
+ */
+static u16 iavf_get_num_vlans_added(struct iavf_adapter *adapter)
+{
+ return bitmap_weight(adapter->vsi.active_cvlans, VLAN_N_VID) +
+ bitmap_weight(adapter->vsi.active_svlans, VLAN_N_VID);
+}
+
+/**
+ * iavf_get_max_vlans_allowed - get maximum VLANs allowed for this VF
+ * @adapter: board private structure
+ *
+ * This depends on the negotiated VLAN capability. For VIRTCHNL_VF_OFFLOAD_VLAN,
+ * do not impose a limit as that maintains current behavior and for
+ * VIRTCHNL_VF_OFFLOAD_VLAN_V2, use the maximum allowed sent from the PF.
+ **/
+static u16 iavf_get_max_vlans_allowed(struct iavf_adapter *adapter)
+{
+ /* don't impose any limit for VIRTCHNL_VF_OFFLOAD_VLAN since there has
+ * never been a limit on the VF driver side
+ */
+ if (VLAN_ALLOWED(adapter))
+ return VLAN_N_VID;
+ else if (VLAN_V2_ALLOWED(adapter))
+ return adapter->vlan_v2_caps.filtering.max_filters;
+
+ return 0;
+}
+
+/**
+ * iavf_max_vlans_added - check if maximum VLANs allowed already exist
+ * @adapter: board private structure
+ **/
+static bool iavf_max_vlans_added(struct iavf_adapter *adapter)
+{
+ if (iavf_get_num_vlans_added(adapter) <
+ iavf_get_max_vlans_allowed(adapter))
+ return false;
+
+ return true;
}
/**
@@ -735,13 +786,23 @@ static int iavf_vlan_rx_add_vid(struct net_device *netdev,
{
struct iavf_adapter *adapter = netdev_priv(netdev);
- if (!VLAN_ALLOWED(adapter))
+ if (!VLAN_FILTERING_ALLOWED(adapter))
return -EIO;
- if (iavf_add_vlan(adapter, vid) == NULL)
+ if (iavf_max_vlans_added(adapter)) {
+ netdev_err(netdev, "Max allowed VLAN filters %u. Remove existing VLANs or disable filtering via Ethtool if supported.\n",
+ iavf_get_max_vlans_allowed(adapter));
+ return -EIO;
+ }
+
+ if (!iavf_add_vlan(adapter, IAVF_VLAN(vid, be16_to_cpu(proto))))
return -ENOMEM;
- set_bit(vid, adapter->vsi.active_vlans);
+ if (proto == cpu_to_be16(ETH_P_8021Q))
+ set_bit(vid, adapter->vsi.active_cvlans);
+ else
+ set_bit(vid, adapter->vsi.active_svlans);
+
return 0;
}
@@ -756,8 +817,11 @@ static int iavf_vlan_rx_kill_vid(struct net_device *netdev,
{
struct iavf_adapter *adapter = netdev_priv(netdev);
- iavf_del_vlan(adapter, vid);
- clear_bit(vid, adapter->vsi.active_vlans);
+ iavf_del_vlan(adapter, IAVF_VLAN(vid, be16_to_cpu(proto)));
+ if (proto == cpu_to_be16(ETH_P_8021Q))
+ clear_bit(vid, adapter->vsi.active_cvlans);
+ else
+ clear_bit(vid, adapter->vsi.active_svlans);
return 0;
}
@@ -1152,6 +1216,86 @@ static void iavf_free_queues(struct iavf_adapter *adapter)
}
/**
+ * iavf_set_queue_vlan_tag_loc - set location for VLAN tag offload
+ * @adapter: board private structure
+ *
+ * Based on negotiated capabilities, the VLAN tag needs to be inserted and/or
+ * stripped in certain descriptor fields. Instead of checking the offload
+ * capability bits in the hot path, cache the location the ring specific
+ * flags.
+ */
+void iavf_set_queue_vlan_tag_loc(struct iavf_adapter *adapter)
+{
+ int i;
+
+ for (i = 0; i < adapter->num_active_queues; i++) {
+ struct iavf_ring *tx_ring = &adapter->tx_rings[i];
+ struct iavf_ring *rx_ring = &adapter->rx_rings[i];
+
+ /* prevent multiple L2TAG bits being set after VFR */
+ tx_ring->flags &=
+ ~(IAVF_TXRX_FLAGS_VLAN_TAG_LOC_L2TAG1 |
+ IAVF_TXR_FLAGS_VLAN_TAG_LOC_L2TAG2);
+ rx_ring->flags &=
+ ~(IAVF_TXRX_FLAGS_VLAN_TAG_LOC_L2TAG1 |
+ IAVF_RXR_FLAGS_VLAN_TAG_LOC_L2TAG2_2);
+
+ if (VLAN_ALLOWED(adapter)) {
+ tx_ring->flags |= IAVF_TXRX_FLAGS_VLAN_TAG_LOC_L2TAG1;
+ rx_ring->flags |= IAVF_TXRX_FLAGS_VLAN_TAG_LOC_L2TAG1;
+ } else if (VLAN_V2_ALLOWED(adapter)) {
+ struct virtchnl_vlan_supported_caps *stripping_support;
+ struct virtchnl_vlan_supported_caps *insertion_support;
+
+ stripping_support =
+ &adapter->vlan_v2_caps.offloads.stripping_support;
+ insertion_support =
+ &adapter->vlan_v2_caps.offloads.insertion_support;
+
+ if (stripping_support->outer) {
+ if (stripping_support->outer &
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG1)
+ rx_ring->flags |=
+ IAVF_TXRX_FLAGS_VLAN_TAG_LOC_L2TAG1;
+ else if (stripping_support->outer &
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2_2)
+ rx_ring->flags |=
+ IAVF_RXR_FLAGS_VLAN_TAG_LOC_L2TAG2_2;
+ } else if (stripping_support->inner) {
+ if (stripping_support->inner &
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG1)
+ rx_ring->flags |=
+ IAVF_TXRX_FLAGS_VLAN_TAG_LOC_L2TAG1;
+ else if (stripping_support->inner &
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2_2)
+ rx_ring->flags |=
+ IAVF_RXR_FLAGS_VLAN_TAG_LOC_L2TAG2_2;
+ }
+
+ if (insertion_support->outer) {
+ if (insertion_support->outer &
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG1)
+ tx_ring->flags |=
+ IAVF_TXRX_FLAGS_VLAN_TAG_LOC_L2TAG1;
+ else if (insertion_support->outer &
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2)
+ tx_ring->flags |=
+ IAVF_TXR_FLAGS_VLAN_TAG_LOC_L2TAG2;
+ } else if (insertion_support->inner) {
+ if (insertion_support->inner &
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG1)
+ tx_ring->flags |=
+ IAVF_TXRX_FLAGS_VLAN_TAG_LOC_L2TAG1;
+ else if (insertion_support->inner &
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2)
+ tx_ring->flags |=
+ IAVF_TXR_FLAGS_VLAN_TAG_LOC_L2TAG2;
+ }
+ }
+ }
+}
+
+/**
* iavf_alloc_queues - Allocate memory for all rings
* @adapter: board private structure to initialize
*
@@ -1212,6 +1356,8 @@ static int iavf_alloc_queues(struct iavf_adapter *adapter)
adapter->num_active_queues = num_active_queues;
+ iavf_set_queue_vlan_tag_loc(adapter);
+
return 0;
err_out:
@@ -1584,6 +1730,8 @@ static int iavf_process_aq_command(struct iavf_adapter *adapter)
{
if (adapter->aq_required & IAVF_FLAG_AQ_GET_CONFIG)
return iavf_send_vf_config_msg(adapter);
+ if (adapter->aq_required & IAVF_FLAG_AQ_GET_OFFLOAD_VLAN_V2_CAPS)
+ return iavf_send_vf_offload_vlan_v2_msg(adapter);
if (adapter->aq_required & IAVF_FLAG_AQ_DISABLE_QUEUES) {
iavf_disable_queues(adapter);
return 0;
@@ -1717,6 +1865,39 @@ static int iavf_process_aq_command(struct iavf_adapter *adapter)
iavf_del_adv_rss_cfg(adapter);
return 0;
}
+ if (adapter->aq_required & IAVF_FLAG_AQ_DISABLE_CTAG_VLAN_STRIPPING) {
+ iavf_disable_vlan_stripping_v2(adapter, ETH_P_8021Q);
+ return 0;
+ }
+ if (adapter->aq_required & IAVF_FLAG_AQ_DISABLE_STAG_VLAN_STRIPPING) {
+ iavf_disable_vlan_stripping_v2(adapter, ETH_P_8021AD);
+ return 0;
+ }
+ if (adapter->aq_required & IAVF_FLAG_AQ_ENABLE_CTAG_VLAN_STRIPPING) {
+ iavf_enable_vlan_stripping_v2(adapter, ETH_P_8021Q);
+ return 0;
+ }
+ if (adapter->aq_required & IAVF_FLAG_AQ_ENABLE_STAG_VLAN_STRIPPING) {
+ iavf_enable_vlan_stripping_v2(adapter, ETH_P_8021AD);
+ return 0;
+ }
+ if (adapter->aq_required & IAVF_FLAG_AQ_DISABLE_CTAG_VLAN_INSERTION) {
+ iavf_disable_vlan_insertion_v2(adapter, ETH_P_8021Q);
+ return 0;
+ }
+ if (adapter->aq_required & IAVF_FLAG_AQ_DISABLE_STAG_VLAN_INSERTION) {
+ iavf_disable_vlan_insertion_v2(adapter, ETH_P_8021AD);
+ return 0;
+ }
+ if (adapter->aq_required & IAVF_FLAG_AQ_ENABLE_CTAG_VLAN_INSERTION) {
+ iavf_enable_vlan_insertion_v2(adapter, ETH_P_8021Q);
+ return 0;
+ }
+ if (adapter->aq_required & IAVF_FLAG_AQ_ENABLE_STAG_VLAN_INSERTION) {
+ iavf_enable_vlan_insertion_v2(adapter, ETH_P_8021AD);
+ return 0;
+ }
+
if (adapter->aq_required & IAVF_FLAG_AQ_REQUEST_STATS) {
iavf_request_stats(adapter);
return 0;
@@ -1726,6 +1907,91 @@ static int iavf_process_aq_command(struct iavf_adapter *adapter)
}
/**
+ * iavf_set_vlan_offload_features - set VLAN offload configuration
+ * @adapter: board private structure
+ * @prev_features: previous features used for comparison
+ * @features: updated features used for configuration
+ *
+ * Set the aq_required bit(s) based on the requested features passed in to
+ * configure VLAN stripping and/or VLAN insertion if supported. Also, schedule
+ * the watchdog if any changes are requested to expedite the request via
+ * virtchnl.
+ **/
+void
+iavf_set_vlan_offload_features(struct iavf_adapter *adapter,
+ netdev_features_t prev_features,
+ netdev_features_t features)
+{
+ bool enable_stripping = true, enable_insertion = true;
+ u16 vlan_ethertype = 0;
+ u64 aq_required = 0;
+
+ /* keep cases separate because one ethertype for offloads can be
+ * disabled at the same time as another is disabled, so check for an
+ * enabled ethertype first, then check for disabled. Default to
+ * ETH_P_8021Q so an ethertype is specified if disabling insertion and
+ * stripping.
+ */
+ if (features & (NETIF_F_HW_VLAN_STAG_RX | NETIF_F_HW_VLAN_STAG_TX))
+ vlan_ethertype = ETH_P_8021AD;
+ else if (features & (NETIF_F_HW_VLAN_CTAG_RX | NETIF_F_HW_VLAN_CTAG_TX))
+ vlan_ethertype = ETH_P_8021Q;
+ else if (prev_features & (NETIF_F_HW_VLAN_STAG_RX | NETIF_F_HW_VLAN_STAG_TX))
+ vlan_ethertype = ETH_P_8021AD;
+ else if (prev_features & (NETIF_F_HW_VLAN_CTAG_RX | NETIF_F_HW_VLAN_CTAG_TX))
+ vlan_ethertype = ETH_P_8021Q;
+ else
+ vlan_ethertype = ETH_P_8021Q;
+
+ if (!(features & (NETIF_F_HW_VLAN_STAG_RX | NETIF_F_HW_VLAN_CTAG_RX)))
+ enable_stripping = false;
+ if (!(features & (NETIF_F_HW_VLAN_STAG_TX | NETIF_F_HW_VLAN_CTAG_TX)))
+ enable_insertion = false;
+
+ if (VLAN_ALLOWED(adapter)) {
+ /* VIRTCHNL_VF_OFFLOAD_VLAN only has support for toggling VLAN
+ * stripping via virtchnl. VLAN insertion can be toggled on the
+ * netdev, but it doesn't require a virtchnl message
+ */
+ if (enable_stripping)
+ aq_required |= IAVF_FLAG_AQ_ENABLE_VLAN_STRIPPING;
+ else
+ aq_required |= IAVF_FLAG_AQ_DISABLE_VLAN_STRIPPING;
+
+ } else if (VLAN_V2_ALLOWED(adapter)) {
+ switch (vlan_ethertype) {
+ case ETH_P_8021Q:
+ if (enable_stripping)
+ aq_required |= IAVF_FLAG_AQ_ENABLE_CTAG_VLAN_STRIPPING;
+ else
+ aq_required |= IAVF_FLAG_AQ_DISABLE_CTAG_VLAN_STRIPPING;
+
+ if (enable_insertion)
+ aq_required |= IAVF_FLAG_AQ_ENABLE_CTAG_VLAN_INSERTION;
+ else
+ aq_required |= IAVF_FLAG_AQ_DISABLE_CTAG_VLAN_INSERTION;
+ break;
+ case ETH_P_8021AD:
+ if (enable_stripping)
+ aq_required |= IAVF_FLAG_AQ_ENABLE_STAG_VLAN_STRIPPING;
+ else
+ aq_required |= IAVF_FLAG_AQ_DISABLE_STAG_VLAN_STRIPPING;
+
+ if (enable_insertion)
+ aq_required |= IAVF_FLAG_AQ_ENABLE_STAG_VLAN_INSERTION;
+ else
+ aq_required |= IAVF_FLAG_AQ_DISABLE_STAG_VLAN_INSERTION;
+ break;
+ }
+ }
+
+ if (aq_required) {
+ adapter->aq_required |= aq_required;
+ mod_delayed_work(iavf_wq, &adapter->watchdog_task, 0);
+ }
+}
+
+/**
* iavf_startup - first step of driver startup
* @adapter: board private structure
*
@@ -1827,6 +2093,59 @@ static void iavf_init_version_check(struct iavf_adapter *adapter)
}
/**
+ * iavf_parse_vf_resource_msg - parse response from VIRTCHNL_OP_GET_VF_RESOURCES
+ * @adapter: board private structure
+ */
+int iavf_parse_vf_resource_msg(struct iavf_adapter *adapter)
+{
+ int i, num_req_queues = adapter->num_req_queues;
+ struct iavf_vsi *vsi = &adapter->vsi;
+
+ for (i = 0; i < adapter->vf_res->num_vsis; i++) {
+ if (adapter->vf_res->vsi_res[i].vsi_type == VIRTCHNL_VSI_SRIOV)
+ adapter->vsi_res = &adapter->vf_res->vsi_res[i];
+ }
+ if (!adapter->vsi_res) {
+ dev_err(&adapter->pdev->dev, "No LAN VSI found\n");
+ return -ENODEV;
+ }
+
+ if (num_req_queues &&
+ num_req_queues > adapter->vsi_res->num_queue_pairs) {
+ /* Problem. The PF gave us fewer queues than what we had
+ * negotiated in our request. Need a reset to see if we can't
+ * get back to a working state.
+ */
+ dev_err(&adapter->pdev->dev,
+ "Requested %d queues, but PF only gave us %d.\n",
+ num_req_queues,
+ adapter->vsi_res->num_queue_pairs);
+ adapter->flags |= IAVF_FLAG_REINIT_ITR_NEEDED;
+ adapter->num_req_queues = adapter->vsi_res->num_queue_pairs;
+ iavf_schedule_reset(adapter);
+
+ return -EAGAIN;
+ }
+ adapter->num_req_queues = 0;
+ adapter->vsi.id = adapter->vsi_res->vsi_id;
+
+ adapter->vsi.back = adapter;
+ adapter->vsi.base_vector = 1;
+ adapter->vsi.work_limit = IAVF_DEFAULT_IRQ_WORK;
+ vsi->netdev = adapter->netdev;
+ vsi->qs_handle = adapter->vsi_res->qset_handle;
+ if (adapter->vf_res->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_RSS_PF) {
+ adapter->rss_key_size = adapter->vf_res->rss_key_size;
+ adapter->rss_lut_size = adapter->vf_res->rss_lut_size;
+ } else {
+ adapter->rss_key_size = IAVF_HKEY_ARRAY_SIZE;
+ adapter->rss_lut_size = IAVF_HLUT_ARRAY_SIZE;
+ }
+
+ return 0;
+}
+
+/**
* iavf_init_get_resources - third step of driver startup
* @adapter: board private structure
*
@@ -1837,7 +2156,6 @@ static void iavf_init_version_check(struct iavf_adapter *adapter)
**/
static void iavf_init_get_resources(struct iavf_adapter *adapter)
{
- struct net_device *netdev = adapter->netdev;
struct pci_dev *pdev = adapter->pdev;
struct iavf_hw *hw = &adapter->hw;
int err;
@@ -1855,7 +2173,7 @@ static void iavf_init_get_resources(struct iavf_adapter *adapter)
err = iavf_get_vf_config(adapter);
if (err == IAVF_ERR_ADMIN_QUEUE_NO_WORK) {
err = iavf_send_vf_config_msg(adapter);
- goto err;
+ goto err_alloc;
} else if (err == IAVF_ERR_PARAM) {
/* We only get ERR_PARAM if the device is in a very bad
* state or if we've been disabled for previous bad
@@ -1870,9 +2188,83 @@ static void iavf_init_get_resources(struct iavf_adapter *adapter)
goto err_alloc;
}
- err = iavf_process_config(adapter);
+ err = iavf_parse_vf_resource_msg(adapter);
if (err)
goto err_alloc;
+
+ err = iavf_send_vf_offload_vlan_v2_msg(adapter);
+ if (err == -EOPNOTSUPP) {
+ /* underlying PF doesn't support VIRTCHNL_VF_OFFLOAD_VLAN_V2, so
+ * go directly to finishing initialization
+ */
+ iavf_change_state(adapter, __IAVF_INIT_CONFIG_ADAPTER);
+ return;
+ } else if (err) {
+ dev_err(&pdev->dev, "Unable to send offload vlan v2 request (%d)\n",
+ err);
+ goto err_alloc;
+ }
+
+ /* underlying PF supports VIRTCHNL_VF_OFFLOAD_VLAN_V2, so update the
+ * state accordingly
+ */
+ iavf_change_state(adapter, __IAVF_INIT_GET_OFFLOAD_VLAN_V2_CAPS);
+ return;
+
+err_alloc:
+ kfree(adapter->vf_res);
+ adapter->vf_res = NULL;
+err:
+ iavf_change_state(adapter, __IAVF_INIT_FAILED);
+}
+
+/**
+ * iavf_init_get_offload_vlan_v2_caps - part of driver startup
+ * @adapter: board private structure
+ *
+ * Function processes __IAVF_INIT_GET_OFFLOAD_VLAN_V2_CAPS driver state if the
+ * VF negotiates VIRTCHNL_VF_OFFLOAD_VLAN_V2. If VIRTCHNL_VF_OFFLOAD_VLAN_V2 is
+ * not negotiated, then this state will never be entered.
+ **/
+static void iavf_init_get_offload_vlan_v2_caps(struct iavf_adapter *adapter)
+{
+ int ret;
+
+ WARN_ON(adapter->state != __IAVF_INIT_GET_OFFLOAD_VLAN_V2_CAPS);
+
+ memset(&adapter->vlan_v2_caps, 0, sizeof(adapter->vlan_v2_caps));
+
+ ret = iavf_get_vf_vlan_v2_caps(adapter);
+ if (ret) {
+ if (ret == IAVF_ERR_ADMIN_QUEUE_NO_WORK)
+ iavf_send_vf_offload_vlan_v2_msg(adapter);
+ goto err;
+ }
+
+ iavf_change_state(adapter, __IAVF_INIT_CONFIG_ADAPTER);
+ return;
+err:
+ iavf_change_state(adapter, __IAVF_INIT_FAILED);
+}
+
+/**
+ * iavf_init_config_adapter - last part of driver startup
+ * @adapter: board private structure
+ *
+ * After all the supported capabilities are negotiated, then the
+ * __IAVF_INIT_CONFIG_ADAPTER state will finish driver initialization.
+ */
+static void iavf_init_config_adapter(struct iavf_adapter *adapter)
+{
+ struct net_device *netdev = adapter->netdev;
+ struct pci_dev *pdev = adapter->pdev;
+ int err;
+
+ WARN_ON(adapter->state != __IAVF_INIT_CONFIG_ADAPTER);
+
+ if (iavf_process_config(adapter))
+ goto err;
+
adapter->current_op = VIRTCHNL_OP_UNKNOWN;
adapter->flags |= IAVF_FLAG_RX_CSUM_ENABLED;
@@ -1955,6 +2347,10 @@ static void iavf_init_get_resources(struct iavf_adapter *adapter)
else
iavf_init_rss(adapter);
+ if (VLAN_V2_ALLOWED(adapter))
+ /* request initial VLAN offload settings */
+ iavf_set_vlan_offload_features(adapter, 0, netdev->features);
+
return;
err_mem:
iavf_free_rss(adapter);
@@ -1962,9 +2358,6 @@ static void iavf_init_get_resources(struct iavf_adapter *adapter)
iavf_free_misc_irq(adapter);
err_sw_init:
iavf_reset_interrupt_capability(adapter);
-err_alloc:
- kfree(adapter->vf_res);
- adapter->vf_res = NULL;
err:
iavf_change_state(adapter, __IAVF_INIT_FAILED);
}
@@ -2013,6 +2406,18 @@ static void iavf_watchdog_task(struct work_struct *work)
queue_delayed_work(iavf_wq, &adapter->watchdog_task,
msecs_to_jiffies(1));
return;
+ case __IAVF_INIT_GET_OFFLOAD_VLAN_V2_CAPS:
+ iavf_init_get_offload_vlan_v2_caps(adapter);
+ mutex_unlock(&adapter->crit_lock);
+ queue_delayed_work(iavf_wq, &adapter->watchdog_task,
+ msecs_to_jiffies(1));
+ return;
+ case __IAVF_INIT_CONFIG_ADAPTER:
+ iavf_init_config_adapter(adapter);
+ mutex_unlock(&adapter->crit_lock);
+ queue_delayed_work(iavf_wq, &adapter->watchdog_task,
+ msecs_to_jiffies(1));
+ return;
case __IAVF_INIT_FAILED:
if (++adapter->aq_wait_count > IAVF_AQ_MAX_ERR) {
dev_err(&adapter->pdev->dev,
@@ -2066,10 +2471,13 @@ static void iavf_watchdog_task(struct work_struct *work)
iavf_send_api_ver(adapter);
}
} else {
+ int ret = iavf_process_aq_command(adapter);
+
/* An error will be returned if no commands were
* processed; use this opportunity to update stats
+ * if the error isn't -ENOTSUPP
*/
- if (iavf_process_aq_command(adapter) &&
+ if (ret && ret != -EOPNOTSUPP &&
adapter->state == __IAVF_RUNNING)
iavf_request_stats(adapter);
}
@@ -2308,6 +2716,13 @@ static void iavf_reset_task(struct work_struct *work)
}
adapter->aq_required |= IAVF_FLAG_AQ_GET_CONFIG;
+ /* always set since VIRTCHNL_OP_GET_VF_RESOURCES has not been
+ * sent/received yet, so VLAN_V2_ALLOWED() cannot is not reliable here,
+ * however the VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS won't be sent until
+ * VIRTCHNL_OP_GET_VF_RESOURCES and VIRTCHNL_VF_OFFLOAD_VLAN_V2 have
+ * been successfully sent and negotiated
+ */
+ adapter->aq_required |= IAVF_FLAG_AQ_GET_OFFLOAD_VLAN_V2_CAPS;
adapter->aq_required |= IAVF_FLAG_AQ_MAP_VECTORS;
spin_lock_bh(&adapter->mac_vlan_list_lock);
@@ -2708,8 +3123,11 @@ static int iavf_validate_ch_config(struct iavf_adapter *adapter,
total_max_rate += tx_rate;
num_qps += mqprio_qopt->qopt.count[i];
}
- if (num_qps > IAVF_MAX_REQ_QUEUES)
+ if (num_qps > adapter->num_active_queues) {
+ dev_err(&adapter->pdev->dev,
+ "Cannot support requested number of queues\n");
return -EINVAL;
+ }
ret = iavf_validate_tx_bandwidth(adapter, total_max_rate);
return ret;
@@ -2910,7 +3328,7 @@ static int iavf_parse_cls_flower(struct iavf_adapter *adapter,
} else {
dev_err(&adapter->pdev->dev, "Bad ether dest mask %pM\n",
match.mask->dst);
- return IAVF_ERR_CONFIG;
+ return -EINVAL;
}
}
@@ -2920,7 +3338,7 @@ static int iavf_parse_cls_flower(struct iavf_adapter *adapter,
} else {
dev_err(&adapter->pdev->dev, "Bad ether src mask %pM\n",
match.mask->src);
- return IAVF_ERR_CONFIG;
+ return -EINVAL;
}
}
@@ -2955,7 +3373,7 @@ static int iavf_parse_cls_flower(struct iavf_adapter *adapter,
} else {
dev_err(&adapter->pdev->dev, "Bad vlan mask %u\n",
match.mask->vlan_id);
- return IAVF_ERR_CONFIG;
+ return -EINVAL;
}
}
vf->mask.tcp_spec.vlan_id |= cpu_to_be16(0xffff);
@@ -2979,7 +3397,7 @@ static int iavf_parse_cls_flower(struct iavf_adapter *adapter,
} else {
dev_err(&adapter->pdev->dev, "Bad ip dst mask 0x%08x\n",
be32_to_cpu(match.mask->dst));
- return IAVF_ERR_CONFIG;
+ return -EINVAL;
}
}
@@ -2989,13 +3407,13 @@ static int iavf_parse_cls_flower(struct iavf_adapter *adapter,
} else {
dev_err(&adapter->pdev->dev, "Bad ip src mask 0x%08x\n",
be32_to_cpu(match.mask->dst));
- return IAVF_ERR_CONFIG;
+ return -EINVAL;
}
}
if (field_flags & IAVF_CLOUD_FIELD_TEN_ID) {
dev_info(&adapter->pdev->dev, "Tenant id not allowed for ip filter\n");
- return IAVF_ERR_CONFIG;
+ return -EINVAL;
}
if (match.key->dst) {
vf->mask.tcp_spec.dst_ip[0] |= cpu_to_be32(0xffffffff);
@@ -3016,7 +3434,7 @@ static int iavf_parse_cls_flower(struct iavf_adapter *adapter,
if (ipv6_addr_any(&match.mask->dst)) {
dev_err(&adapter->pdev->dev, "Bad ipv6 dst mask 0x%02x\n",
IPV6_ADDR_ANY);
- return IAVF_ERR_CONFIG;
+ return -EINVAL;
}
/* src and dest IPv6 address should not be LOOPBACK
@@ -3026,7 +3444,7 @@ static int iavf_parse_cls_flower(struct iavf_adapter *adapter,
ipv6_addr_loopback(&match.key->src)) {
dev_err(&adapter->pdev->dev,
"ipv6 addr should not be loopback\n");
- return IAVF_ERR_CONFIG;
+ return -EINVAL;
}
if (!ipv6_addr_any(&match.mask->dst) ||
!ipv6_addr_any(&match.mask->src))
@@ -3051,7 +3469,7 @@ static int iavf_parse_cls_flower(struct iavf_adapter *adapter,
} else {
dev_err(&adapter->pdev->dev, "Bad src port mask %u\n",
be16_to_cpu(match.mask->src));
- return IAVF_ERR_CONFIG;
+ return -EINVAL;
}
}
@@ -3061,7 +3479,7 @@ static int iavf_parse_cls_flower(struct iavf_adapter *adapter,
} else {
dev_err(&adapter->pdev->dev, "Bad dst port mask %u\n",
be16_to_cpu(match.mask->dst));
- return IAVF_ERR_CONFIG;
+ return -EINVAL;
}
}
if (match.key->dst) {
@@ -3428,6 +3846,8 @@ static int iavf_change_mtu(struct net_device *netdev, int new_mtu)
{
struct iavf_adapter *adapter = netdev_priv(netdev);
+ netdev_dbg(netdev, "changing MTU from %d to %d\n",
+ netdev->mtu, new_mtu);
netdev->mtu = new_mtu;
if (CLIENT_ENABLED(adapter)) {
iavf_notify_client_l2_params(&adapter->vsi);
@@ -3439,6 +3859,11 @@ static int iavf_change_mtu(struct net_device *netdev, int new_mtu)
return 0;
}
+#define NETIF_VLAN_OFFLOAD_FEATURES (NETIF_F_HW_VLAN_CTAG_RX | \
+ NETIF_F_HW_VLAN_CTAG_TX | \
+ NETIF_F_HW_VLAN_STAG_RX | \
+ NETIF_F_HW_VLAN_STAG_TX)
+
/**
* iavf_set_features - set the netdev feature flags
* @netdev: ptr to the netdev being adjusted
@@ -3450,25 +3875,11 @@ static int iavf_set_features(struct net_device *netdev,
{
struct iavf_adapter *adapter = netdev_priv(netdev);
- /* Don't allow enabling VLAN features when adapter is not capable
- * of VLAN offload/filtering
- */
- if (!VLAN_ALLOWED(adapter)) {
- netdev->hw_features &= ~(NETIF_F_HW_VLAN_CTAG_RX |
- NETIF_F_HW_VLAN_CTAG_TX |
- NETIF_F_HW_VLAN_CTAG_FILTER);
- if (features & (NETIF_F_HW_VLAN_CTAG_RX |
- NETIF_F_HW_VLAN_CTAG_TX |
- NETIF_F_HW_VLAN_CTAG_FILTER))
- return -EINVAL;
- } else if ((netdev->features ^ features) & NETIF_F_HW_VLAN_CTAG_RX) {
- if (features & NETIF_F_HW_VLAN_CTAG_RX)
- adapter->aq_required |=
- IAVF_FLAG_AQ_ENABLE_VLAN_STRIPPING;
- else
- adapter->aq_required |=
- IAVF_FLAG_AQ_DISABLE_VLAN_STRIPPING;
- }
+ /* trigger update on any VLAN feature change */
+ if ((netdev->features & NETIF_VLAN_OFFLOAD_FEATURES) ^
+ (features & NETIF_VLAN_OFFLOAD_FEATURES))
+ iavf_set_vlan_offload_features(adapter, netdev->features,
+ features);
return 0;
}
@@ -3532,6 +3943,228 @@ static netdev_features_t iavf_features_check(struct sk_buff *skb,
}
/**
+ * iavf_get_netdev_vlan_hw_features - get NETDEV VLAN features that can toggle on/off
+ * @adapter: board private structure
+ *
+ * Depending on whether VIRTHCNL_VF_OFFLOAD_VLAN or VIRTCHNL_VF_OFFLOAD_VLAN_V2
+ * were negotiated determine the VLAN features that can be toggled on and off.
+ **/
+static netdev_features_t
+iavf_get_netdev_vlan_hw_features(struct iavf_adapter *adapter)
+{
+ netdev_features_t hw_features = 0;
+
+ if (!adapter->vf_res || !adapter->vf_res->vf_cap_flags)
+ return hw_features;
+
+ /* Enable VLAN features if supported */
+ if (VLAN_ALLOWED(adapter)) {
+ hw_features |= (NETIF_F_HW_VLAN_CTAG_TX |
+ NETIF_F_HW_VLAN_CTAG_RX);
+ } else if (VLAN_V2_ALLOWED(adapter)) {
+ struct virtchnl_vlan_caps *vlan_v2_caps =
+ &adapter->vlan_v2_caps;
+ struct virtchnl_vlan_supported_caps *stripping_support =
+ &vlan_v2_caps->offloads.stripping_support;
+ struct virtchnl_vlan_supported_caps *insertion_support =
+ &vlan_v2_caps->offloads.insertion_support;
+
+ if (stripping_support->outer != VIRTCHNL_VLAN_UNSUPPORTED &&
+ stripping_support->outer & VIRTCHNL_VLAN_TOGGLE) {
+ if (stripping_support->outer &
+ VIRTCHNL_VLAN_ETHERTYPE_8100)
+ hw_features |= NETIF_F_HW_VLAN_CTAG_RX;
+ if (stripping_support->outer &
+ VIRTCHNL_VLAN_ETHERTYPE_88A8)
+ hw_features |= NETIF_F_HW_VLAN_STAG_RX;
+ } else if (stripping_support->inner !=
+ VIRTCHNL_VLAN_UNSUPPORTED &&
+ stripping_support->inner & VIRTCHNL_VLAN_TOGGLE) {
+ if (stripping_support->inner &
+ VIRTCHNL_VLAN_ETHERTYPE_8100)
+ hw_features |= NETIF_F_HW_VLAN_CTAG_RX;
+ }
+
+ if (insertion_support->outer != VIRTCHNL_VLAN_UNSUPPORTED &&
+ insertion_support->outer & VIRTCHNL_VLAN_TOGGLE) {
+ if (insertion_support->outer &
+ VIRTCHNL_VLAN_ETHERTYPE_8100)
+ hw_features |= NETIF_F_HW_VLAN_CTAG_TX;
+ if (insertion_support->outer &
+ VIRTCHNL_VLAN_ETHERTYPE_88A8)
+ hw_features |= NETIF_F_HW_VLAN_STAG_TX;
+ } else if (insertion_support->inner &&
+ insertion_support->inner & VIRTCHNL_VLAN_TOGGLE) {
+ if (insertion_support->inner &
+ VIRTCHNL_VLAN_ETHERTYPE_8100)
+ hw_features |= NETIF_F_HW_VLAN_CTAG_TX;
+ }
+ }
+
+ return hw_features;
+}
+
+/**
+ * iavf_get_netdev_vlan_features - get the enabled NETDEV VLAN fetures
+ * @adapter: board private structure
+ *
+ * Depending on whether VIRTHCNL_VF_OFFLOAD_VLAN or VIRTCHNL_VF_OFFLOAD_VLAN_V2
+ * were negotiated determine the VLAN features that are enabled by default.
+ **/
+static netdev_features_t
+iavf_get_netdev_vlan_features(struct iavf_adapter *adapter)
+{
+ netdev_features_t features = 0;
+
+ if (!adapter->vf_res || !adapter->vf_res->vf_cap_flags)
+ return features;
+
+ if (VLAN_ALLOWED(adapter)) {
+ features |= NETIF_F_HW_VLAN_CTAG_FILTER |
+ NETIF_F_HW_VLAN_CTAG_RX | NETIF_F_HW_VLAN_CTAG_TX;
+ } else if (VLAN_V2_ALLOWED(adapter)) {
+ struct virtchnl_vlan_caps *vlan_v2_caps =
+ &adapter->vlan_v2_caps;
+ struct virtchnl_vlan_supported_caps *filtering_support =
+ &vlan_v2_caps->filtering.filtering_support;
+ struct virtchnl_vlan_supported_caps *stripping_support =
+ &vlan_v2_caps->offloads.stripping_support;
+ struct virtchnl_vlan_supported_caps *insertion_support =
+ &vlan_v2_caps->offloads.insertion_support;
+ u32 ethertype_init;
+
+ /* give priority to outer stripping and don't support both outer
+ * and inner stripping
+ */
+ ethertype_init = vlan_v2_caps->offloads.ethertype_init;
+ if (stripping_support->outer != VIRTCHNL_VLAN_UNSUPPORTED) {
+ if (stripping_support->outer &
+ VIRTCHNL_VLAN_ETHERTYPE_8100 &&
+ ethertype_init & VIRTCHNL_VLAN_ETHERTYPE_8100)
+ features |= NETIF_F_HW_VLAN_CTAG_RX;
+ else if (stripping_support->outer &
+ VIRTCHNL_VLAN_ETHERTYPE_88A8 &&
+ ethertype_init & VIRTCHNL_VLAN_ETHERTYPE_88A8)
+ features |= NETIF_F_HW_VLAN_STAG_RX;
+ } else if (stripping_support->inner !=
+ VIRTCHNL_VLAN_UNSUPPORTED) {
+ if (stripping_support->inner &
+ VIRTCHNL_VLAN_ETHERTYPE_8100 &&
+ ethertype_init & VIRTCHNL_VLAN_ETHERTYPE_8100)
+ features |= NETIF_F_HW_VLAN_CTAG_RX;
+ }
+
+ /* give priority to outer insertion and don't support both outer
+ * and inner insertion
+ */
+ if (insertion_support->outer != VIRTCHNL_VLAN_UNSUPPORTED) {
+ if (insertion_support->outer &
+ VIRTCHNL_VLAN_ETHERTYPE_8100 &&
+ ethertype_init & VIRTCHNL_VLAN_ETHERTYPE_8100)
+ features |= NETIF_F_HW_VLAN_CTAG_TX;
+ else if (insertion_support->outer &
+ VIRTCHNL_VLAN_ETHERTYPE_88A8 &&
+ ethertype_init & VIRTCHNL_VLAN_ETHERTYPE_88A8)
+ features |= NETIF_F_HW_VLAN_STAG_TX;
+ } else if (insertion_support->inner !=
+ VIRTCHNL_VLAN_UNSUPPORTED) {
+ if (insertion_support->inner &
+ VIRTCHNL_VLAN_ETHERTYPE_8100 &&
+ ethertype_init & VIRTCHNL_VLAN_ETHERTYPE_8100)
+ features |= NETIF_F_HW_VLAN_CTAG_TX;
+ }
+
+ /* give priority to outer filtering and don't bother if both
+ * outer and inner filtering are enabled
+ */
+ ethertype_init = vlan_v2_caps->filtering.ethertype_init;
+ if (filtering_support->outer != VIRTCHNL_VLAN_UNSUPPORTED) {
+ if (filtering_support->outer &
+ VIRTCHNL_VLAN_ETHERTYPE_8100 &&
+ ethertype_init & VIRTCHNL_VLAN_ETHERTYPE_8100)
+ features |= NETIF_F_HW_VLAN_CTAG_FILTER;
+ if (filtering_support->outer &
+ VIRTCHNL_VLAN_ETHERTYPE_88A8 &&
+ ethertype_init & VIRTCHNL_VLAN_ETHERTYPE_88A8)
+ features |= NETIF_F_HW_VLAN_STAG_FILTER;
+ } else if (filtering_support->inner !=
+ VIRTCHNL_VLAN_UNSUPPORTED) {
+ if (filtering_support->inner &
+ VIRTCHNL_VLAN_ETHERTYPE_8100 &&
+ ethertype_init & VIRTCHNL_VLAN_ETHERTYPE_8100)
+ features |= NETIF_F_HW_VLAN_CTAG_FILTER;
+ if (filtering_support->inner &
+ VIRTCHNL_VLAN_ETHERTYPE_88A8 &&
+ ethertype_init & VIRTCHNL_VLAN_ETHERTYPE_88A8)
+ features |= NETIF_F_HW_VLAN_STAG_FILTER;
+ }
+ }
+
+ return features;
+}
+
+#define IAVF_NETDEV_VLAN_FEATURE_ALLOWED(requested, allowed, feature_bit) \
+ (!(((requested) & (feature_bit)) && \
+ !((allowed) & (feature_bit))))
+
+/**
+ * iavf_fix_netdev_vlan_features - fix NETDEV VLAN features based on support
+ * @adapter: board private structure
+ * @requested_features: stack requested NETDEV features
+ **/
+static netdev_features_t
+iavf_fix_netdev_vlan_features(struct iavf_adapter *adapter,
+ netdev_features_t requested_features)
+{
+ netdev_features_t allowed_features;
+
+ allowed_features = iavf_get_netdev_vlan_hw_features(adapter) |
+ iavf_get_netdev_vlan_features(adapter);
+
+ if (!IAVF_NETDEV_VLAN_FEATURE_ALLOWED(requested_features,
+ allowed_features,
+ NETIF_F_HW_VLAN_CTAG_TX))
+ requested_features &= ~NETIF_F_HW_VLAN_CTAG_TX;
+
+ if (!IAVF_NETDEV_VLAN_FEATURE_ALLOWED(requested_features,
+ allowed_features,
+ NETIF_F_HW_VLAN_CTAG_RX))
+ requested_features &= ~NETIF_F_HW_VLAN_CTAG_RX;
+
+ if (!IAVF_NETDEV_VLAN_FEATURE_ALLOWED(requested_features,
+ allowed_features,
+ NETIF_F_HW_VLAN_STAG_TX))
+ requested_features &= ~NETIF_F_HW_VLAN_STAG_TX;
+ if (!IAVF_NETDEV_VLAN_FEATURE_ALLOWED(requested_features,
+ allowed_features,
+ NETIF_F_HW_VLAN_STAG_RX))
+ requested_features &= ~NETIF_F_HW_VLAN_STAG_RX;
+
+ if (!IAVF_NETDEV_VLAN_FEATURE_ALLOWED(requested_features,
+ allowed_features,
+ NETIF_F_HW_VLAN_CTAG_FILTER))
+ requested_features &= ~NETIF_F_HW_VLAN_CTAG_FILTER;
+
+ if (!IAVF_NETDEV_VLAN_FEATURE_ALLOWED(requested_features,
+ allowed_features,
+ NETIF_F_HW_VLAN_STAG_FILTER))
+ requested_features &= ~NETIF_F_HW_VLAN_STAG_FILTER;
+
+ if ((requested_features &
+ (NETIF_F_HW_VLAN_CTAG_RX | NETIF_F_HW_VLAN_CTAG_TX)) &&
+ (requested_features &
+ (NETIF_F_HW_VLAN_STAG_RX | NETIF_F_HW_VLAN_STAG_TX)) &&
+ adapter->vlan_v2_caps.offloads.ethertype_match ==
+ VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION) {
+ netdev_warn(adapter->netdev, "cannot support CTAG and STAG VLAN stripping and/or insertion simultaneously since CTAG and STAG offloads are mutually exclusive, clearing STAG offload settings\n");
+ requested_features &= ~(NETIF_F_HW_VLAN_STAG_RX |
+ NETIF_F_HW_VLAN_STAG_TX);
+ }
+
+ return requested_features;
+}
+
+/**
* iavf_fix_features - fix up the netdev feature bits
* @netdev: our net device
* @features: desired feature bits
@@ -3543,13 +4176,7 @@ static netdev_features_t iavf_fix_features(struct net_device *netdev,
{
struct iavf_adapter *adapter = netdev_priv(netdev);
- if (adapter->vf_res &&
- !(adapter->vf_res->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_VLAN))
- features &= ~(NETIF_F_HW_VLAN_CTAG_TX |
- NETIF_F_HW_VLAN_CTAG_RX |
- NETIF_F_HW_VLAN_CTAG_FILTER);
-
- return features;
+ return iavf_fix_netdev_vlan_features(adapter, features);
}
static const struct net_device_ops iavf_netdev_ops = {
@@ -3601,39 +4228,11 @@ static int iavf_check_reset_complete(struct iavf_hw *hw)
int iavf_process_config(struct iavf_adapter *adapter)
{
struct virtchnl_vf_resource *vfres = adapter->vf_res;
- int i, num_req_queues = adapter->num_req_queues;
+ netdev_features_t hw_vlan_features, vlan_features;
struct net_device *netdev = adapter->netdev;
- struct iavf_vsi *vsi = &adapter->vsi;
netdev_features_t hw_enc_features;
netdev_features_t hw_features;
- /* got VF config message back from PF, now we can parse it */
- for (i = 0; i < vfres->num_vsis; i++) {
- if (vfres->vsi_res[i].vsi_type == VIRTCHNL_VSI_SRIOV)
- adapter->vsi_res = &vfres->vsi_res[i];
- }
- if (!adapter->vsi_res) {
- dev_err(&adapter->pdev->dev, "No LAN VSI found\n");
- return -ENODEV;
- }
-
- if (num_req_queues &&
- num_req_queues > adapter->vsi_res->num_queue_pairs) {
- /* Problem. The PF gave us fewer queues than what we had
- * negotiated in our request. Need a reset to see if we can't
- * get back to a working state.
- */
- dev_err(&adapter->pdev->dev,
- "Requested %d queues, but PF only gave us %d.\n",
- num_req_queues,
- adapter->vsi_res->num_queue_pairs);
- adapter->flags |= IAVF_FLAG_REINIT_ITR_NEEDED;
- adapter->num_req_queues = adapter->vsi_res->num_queue_pairs;
- iavf_schedule_reset(adapter);
- return -ENODEV;
- }
- adapter->num_req_queues = 0;
-
hw_enc_features = NETIF_F_SG |
NETIF_F_IP_CSUM |
NETIF_F_IPV6_CSUM |
@@ -3677,19 +4276,19 @@ int iavf_process_config(struct iavf_adapter *adapter)
*/
hw_features = hw_enc_features;
- /* Enable VLAN features if supported */
- if (vfres->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_VLAN)
- hw_features |= (NETIF_F_HW_VLAN_CTAG_TX |
- NETIF_F_HW_VLAN_CTAG_RX);
+ /* get HW VLAN features that can be toggled */
+ hw_vlan_features = iavf_get_netdev_vlan_hw_features(adapter);
+
/* Enable cloud filter if ADQ is supported */
if (vfres->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_ADQ)
hw_features |= NETIF_F_HW_TC;
if (vfres->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_USO)
hw_features |= NETIF_F_GSO_UDP_L4;
- netdev->hw_features |= hw_features;
+ netdev->hw_features |= hw_features | hw_vlan_features;
+ vlan_features = iavf_get_netdev_vlan_features(adapter);
- netdev->features |= hw_features;
+ netdev->features |= hw_features | vlan_features;
if (vfres->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_VLAN)
netdev->features |= NETIF_F_HW_VLAN_CTAG_FILTER;
@@ -3714,21 +4313,6 @@ int iavf_process_config(struct iavf_adapter *adapter)
netdev->features &= ~NETIF_F_GSO;
}
- adapter->vsi.id = adapter->vsi_res->vsi_id;
-
- adapter->vsi.back = adapter;
- adapter->vsi.base_vector = 1;
- adapter->vsi.work_limit = IAVF_DEFAULT_IRQ_WORK;
- vsi->netdev = adapter->netdev;
- vsi->qs_handle = adapter->vsi_res->qset_handle;
- if (vfres->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_RSS_PF) {
- adapter->rss_key_size = vfres->rss_key_size;
- adapter->rss_lut_size = vfres->rss_lut_size;
- } else {
- adapter->rss_key_size = IAVF_HKEY_ARRAY_SIZE;
- adapter->rss_lut_size = IAVF_HLUT_ARRAY_SIZE;
- }
-
return 0;
}
@@ -3998,6 +4582,7 @@ static void iavf_remove(struct pci_dev *pdev)
if (iavf_lock_timeout(&adapter->crit_lock, 5000))
dev_warn(&adapter->pdev->dev, "failed to acquire crit_lock in %s\n", __FUNCTION__);
+ dev_info(&adapter->pdev->dev, "Removing device\n");
/* Shut down all the garbage mashers on the detention level */
iavf_change_state(adapter, __IAVF_REMOVE);
adapter->aq_required = 0;
diff --git a/drivers/net/ethernet/intel/iavf/iavf_txrx.c b/drivers/net/ethernet/intel/iavf/iavf_txrx.c
index 3525eab..8cbe7ad 100644
--- a/drivers/net/ethernet/intel/iavf/iavf_txrx.c
+++ b/drivers/net/ethernet/intel/iavf/iavf_txrx.c
@@ -865,6 +865,9 @@ static void iavf_receive_skb(struct iavf_ring *rx_ring,
if ((rx_ring->netdev->features & NETIF_F_HW_VLAN_CTAG_RX) &&
(vlan_tag & VLAN_VID_MASK))
__vlan_hwaccel_put_tag(skb, htons(ETH_P_8021Q), vlan_tag);
+ else if ((rx_ring->netdev->features & NETIF_F_HW_VLAN_STAG_RX) &&
+ vlan_tag & VLAN_VID_MASK)
+ __vlan_hwaccel_put_tag(skb, htons(ETH_P_8021AD), vlan_tag);
napi_gro_receive(&q_vector->napi, skb);
}
@@ -1363,7 +1366,7 @@ static struct sk_buff *iavf_build_skb(struct iavf_ring *rx_ring,
net_prefetch(va);
/* build an skb around the page buffer */
- skb = build_skb(va - IAVF_SKB_PAD, truesize);
+ skb = napi_build_skb(va - IAVF_SKB_PAD, truesize);
if (unlikely(!skb))
return NULL;
@@ -1468,7 +1471,7 @@ static int iavf_clean_rx_irq(struct iavf_ring *rx_ring, int budget)
struct iavf_rx_buffer *rx_buffer;
union iavf_rx_desc *rx_desc;
unsigned int size;
- u16 vlan_tag;
+ u16 vlan_tag = 0;
u8 rx_ptype;
u64 qword;
@@ -1551,9 +1554,13 @@ static int iavf_clean_rx_irq(struct iavf_ring *rx_ring, int budget)
/* populate checksum, VLAN, and protocol */
iavf_process_skb_fields(rx_ring, rx_desc, skb, rx_ptype);
-
- vlan_tag = (qword & BIT(IAVF_RX_DESC_STATUS_L2TAG1P_SHIFT)) ?
- le16_to_cpu(rx_desc->wb.qword0.lo_dword.l2tag1) : 0;
+ if (qword & BIT(IAVF_RX_DESC_STATUS_L2TAG1P_SHIFT) &&
+ rx_ring->flags & IAVF_TXRX_FLAGS_VLAN_TAG_LOC_L2TAG1)
+ vlan_tag = le16_to_cpu(rx_desc->wb.qword0.lo_dword.l2tag1);
+ if (rx_desc->wb.qword2.ext_status &
+ cpu_to_le16(BIT(IAVF_RX_DESC_EXT_STATUS_L2TAG2P_SHIFT)) &&
+ rx_ring->flags & IAVF_RXR_FLAGS_VLAN_TAG_LOC_L2TAG2_2)
+ vlan_tag = le16_to_cpu(rx_desc->wb.qword2.l2tag2_2);
iavf_trace(clean_rx_irq_rx, rx_ring, rx_desc, skb);
iavf_receive_skb(rx_ring, skb, vlan_tag);
@@ -1766,7 +1773,7 @@ int iavf_napi_poll(struct napi_struct *napi, int budget)
if (likely(napi_complete_done(napi, work_done)))
iavf_update_enable_itr(vsi, q_vector);
- return min(work_done, budget - 1);
+ return min_t(int, work_done, budget - 1);
}
/**
@@ -1781,46 +1788,29 @@ int iavf_napi_poll(struct napi_struct *napi, int budget)
* Returns error code indicate the frame should be dropped upon error and the
* otherwise returns 0 to indicate the flags has been set properly.
**/
-static inline int iavf_tx_prepare_vlan_flags(struct sk_buff *skb,
- struct iavf_ring *tx_ring,
- u32 *flags)
+static void iavf_tx_prepare_vlan_flags(struct sk_buff *skb,
+ struct iavf_ring *tx_ring, u32 *flags)
{
- __be16 protocol = skb->protocol;
u32 tx_flags = 0;
- if (protocol == htons(ETH_P_8021Q) &&
- !(tx_ring->netdev->features & NETIF_F_HW_VLAN_CTAG_TX)) {
- /* When HW VLAN acceleration is turned off by the user the
- * stack sets the protocol to 8021q so that the driver
- * can take any steps required to support the SW only
- * VLAN handling. In our case the driver doesn't need
- * to take any further steps so just set the protocol
- * to the encapsulated ethertype.
- */
- skb->protocol = vlan_get_protocol(skb);
- goto out;
- }
- /* if we have a HW VLAN tag being added, default to the HW one */
- if (skb_vlan_tag_present(skb)) {
- tx_flags |= skb_vlan_tag_get(skb) << IAVF_TX_FLAGS_VLAN_SHIFT;
+ /* stack will only request hardware VLAN insertion offload for protocols
+ * that the driver supports and has enabled
+ */
+ if (!skb_vlan_tag_present(skb))
+ return;
+
+ tx_flags |= skb_vlan_tag_get(skb) << IAVF_TX_FLAGS_VLAN_SHIFT;
+ if (tx_ring->flags & IAVF_TXR_FLAGS_VLAN_TAG_LOC_L2TAG2) {
+ tx_flags |= IAVF_TX_FLAGS_HW_OUTER_SINGLE_VLAN;
+ } else if (tx_ring->flags & IAVF_TXRX_FLAGS_VLAN_TAG_LOC_L2TAG1) {
tx_flags |= IAVF_TX_FLAGS_HW_VLAN;
- /* else if it is a SW VLAN, check the next protocol and store the tag */
- } else if (protocol == htons(ETH_P_8021Q)) {
- struct vlan_hdr *vhdr, _vhdr;
-
- vhdr = skb_header_pointer(skb, ETH_HLEN, sizeof(_vhdr), &_vhdr);
- if (!vhdr)
- return -EINVAL;
-
- protocol = vhdr->h_vlan_encapsulated_proto;
- tx_flags |= ntohs(vhdr->h_vlan_TCI) << IAVF_TX_FLAGS_VLAN_SHIFT;
- tx_flags |= IAVF_TX_FLAGS_SW_VLAN;
+ } else {
+ dev_dbg(tx_ring->dev, "Unsupported Tx VLAN tag location requested\n");
+ return;
}
-out:
*flags = tx_flags;
- return 0;
}
/**
@@ -2440,8 +2430,13 @@ static netdev_tx_t iavf_xmit_frame_ring(struct sk_buff *skb,
first->gso_segs = 1;
/* prepare the xmit flags */
- if (iavf_tx_prepare_vlan_flags(skb, tx_ring, &tx_flags))
- goto out_drop;
+ iavf_tx_prepare_vlan_flags(skb, tx_ring, &tx_flags);
+ if (tx_flags & IAVF_TX_FLAGS_HW_OUTER_SINGLE_VLAN) {
+ cd_type_cmd_tso_mss |= IAVF_TX_CTX_DESC_IL2TAG2 <<
+ IAVF_TXD_CTX_QW1_CMD_SHIFT;
+ cd_l2tag2 = (tx_flags & IAVF_TX_FLAGS_VLAN_MASK) >>
+ IAVF_TX_FLAGS_VLAN_SHIFT;
+ }
/* obtain protocol of skb */
protocol = vlan_get_protocol(skb);
diff --git a/drivers/net/ethernet/intel/iavf/iavf_txrx.h b/drivers/net/ethernet/intel/iavf/iavf_txrx.h
index e5b9ba4..2624bf6 100644
--- a/drivers/net/ethernet/intel/iavf/iavf_txrx.h
+++ b/drivers/net/ethernet/intel/iavf/iavf_txrx.h
@@ -243,19 +243,20 @@ static inline unsigned int iavf_txd_use_count(unsigned int size)
#define DESC_NEEDED (MAX_SKB_FRAGS + 6)
#define IAVF_MIN_DESC_PENDING 4
-#define IAVF_TX_FLAGS_HW_VLAN BIT(1)
-#define IAVF_TX_FLAGS_SW_VLAN BIT(2)
-#define IAVF_TX_FLAGS_TSO BIT(3)
-#define IAVF_TX_FLAGS_IPV4 BIT(4)
-#define IAVF_TX_FLAGS_IPV6 BIT(5)
-#define IAVF_TX_FLAGS_FCCRC BIT(6)
-#define IAVF_TX_FLAGS_FSO BIT(7)
-#define IAVF_TX_FLAGS_FD_SB BIT(9)
-#define IAVF_TX_FLAGS_VXLAN_TUNNEL BIT(10)
-#define IAVF_TX_FLAGS_VLAN_MASK 0xffff0000
-#define IAVF_TX_FLAGS_VLAN_PRIO_MASK 0xe0000000
-#define IAVF_TX_FLAGS_VLAN_PRIO_SHIFT 29
-#define IAVF_TX_FLAGS_VLAN_SHIFT 16
+#define IAVF_TX_FLAGS_HW_VLAN BIT(1)
+#define IAVF_TX_FLAGS_SW_VLAN BIT(2)
+#define IAVF_TX_FLAGS_TSO BIT(3)
+#define IAVF_TX_FLAGS_IPV4 BIT(4)
+#define IAVF_TX_FLAGS_IPV6 BIT(5)
+#define IAVF_TX_FLAGS_FCCRC BIT(6)
+#define IAVF_TX_FLAGS_FSO BIT(7)
+#define IAVF_TX_FLAGS_FD_SB BIT(9)
+#define IAVF_TX_FLAGS_VXLAN_TUNNEL BIT(10)
+#define IAVF_TX_FLAGS_HW_OUTER_SINGLE_VLAN BIT(11)
+#define IAVF_TX_FLAGS_VLAN_MASK 0xffff0000
+#define IAVF_TX_FLAGS_VLAN_PRIO_MASK 0xe0000000
+#define IAVF_TX_FLAGS_VLAN_PRIO_SHIFT 29
+#define IAVF_TX_FLAGS_VLAN_SHIFT 16
struct iavf_tx_buffer {
struct iavf_tx_desc *next_to_watch;
@@ -362,6 +363,9 @@ struct iavf_ring {
u16 flags;
#define IAVF_TXR_FLAGS_WB_ON_ITR BIT(0)
#define IAVF_RXR_FLAGS_BUILD_SKB_ENABLED BIT(1)
+#define IAVF_TXRX_FLAGS_VLAN_TAG_LOC_L2TAG1 BIT(3)
+#define IAVF_TXR_FLAGS_VLAN_TAG_LOC_L2TAG2 BIT(4)
+#define IAVF_RXR_FLAGS_VLAN_TAG_LOC_L2TAG2_2 BIT(5)
/* stats structs */
struct iavf_queue_stats stats;
diff --git a/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c b/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c
index d60bf7c..5ee1d11 100644
--- a/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c
+++ b/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c
@@ -137,6 +137,7 @@ int iavf_send_vf_config_msg(struct iavf_adapter *adapter)
VIRTCHNL_VF_OFFLOAD_WB_ON_ITR |
VIRTCHNL_VF_OFFLOAD_RSS_PCTYPE_V2 |
VIRTCHNL_VF_OFFLOAD_ENCAP |
+ VIRTCHNL_VF_OFFLOAD_VLAN_V2 |
VIRTCHNL_VF_OFFLOAD_ENCAP_CSUM |
VIRTCHNL_VF_OFFLOAD_REQ_QUEUES |
VIRTCHNL_VF_OFFLOAD_ADQ |
@@ -155,6 +156,19 @@ int iavf_send_vf_config_msg(struct iavf_adapter *adapter)
NULL, 0);
}
+int iavf_send_vf_offload_vlan_v2_msg(struct iavf_adapter *adapter)
+{
+ adapter->aq_required &= ~IAVF_FLAG_AQ_GET_OFFLOAD_VLAN_V2_CAPS;
+
+ if (!VLAN_V2_ALLOWED(adapter))
+ return -EOPNOTSUPP;
+
+ adapter->current_op = VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS;
+
+ return iavf_send_pf_msg(adapter, VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS,
+ NULL, 0);
+}
+
/**
* iavf_validate_num_queues
* @adapter: adapter structure
@@ -235,6 +249,45 @@ int iavf_get_vf_config(struct iavf_adapter *adapter)
return err;
}
+int iavf_get_vf_vlan_v2_caps(struct iavf_adapter *adapter)
+{
+ struct iavf_hw *hw = &adapter->hw;
+ struct iavf_arq_event_info event;
+ enum virtchnl_ops op;
+ enum iavf_status err;
+ u16 len;
+
+ len = sizeof(struct virtchnl_vlan_caps);
+ event.buf_len = len;
+ event.msg_buf = kzalloc(event.buf_len, GFP_KERNEL);
+ if (!event.msg_buf) {
+ err = -ENOMEM;
+ goto out;
+ }
+
+ while (1) {
+ /* When the AQ is empty, iavf_clean_arq_element will return
+ * nonzero and this loop will terminate.
+ */
+ err = iavf_clean_arq_element(hw, &event, NULL);
+ if (err)
+ goto out_alloc;
+ op = (enum virtchnl_ops)le32_to_cpu(event.desc.cookie_high);
+ if (op == VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS)
+ break;
+ }
+
+ err = (enum iavf_status)le32_to_cpu(event.desc.cookie_low);
+ if (err)
+ goto out_alloc;
+
+ memcpy(&adapter->vlan_v2_caps, event.msg_buf, min(event.msg_len, len));
+out_alloc:
+ kfree(event.msg_buf);
+out:
+ return err;
+}
+
/**
* iavf_configure_queues
* @adapter: adapter structure
@@ -589,7 +642,6 @@ static void iavf_mac_add_reject(struct iavf_adapter *adapter)
**/
void iavf_add_vlans(struct iavf_adapter *adapter)
{
- struct virtchnl_vlan_filter_list *vvfl;
int len, i = 0, count = 0;
struct iavf_vlan_filter *f;
bool more = false;
@@ -607,48 +659,105 @@ void iavf_add_vlans(struct iavf_adapter *adapter)
if (f->add)
count++;
}
- if (!count || !VLAN_ALLOWED(adapter)) {
+ if (!count || !VLAN_FILTERING_ALLOWED(adapter)) {
adapter->aq_required &= ~IAVF_FLAG_AQ_ADD_VLAN_FILTER;
spin_unlock_bh(&adapter->mac_vlan_list_lock);
return;
}
- adapter->current_op = VIRTCHNL_OP_ADD_VLAN;
- len = sizeof(struct virtchnl_vlan_filter_list) +
- (count * sizeof(u16));
- if (len > IAVF_MAX_AQ_BUF_SIZE) {
- dev_warn(&adapter->pdev->dev, "Too many add VLAN changes in one request\n");
- count = (IAVF_MAX_AQ_BUF_SIZE -
- sizeof(struct virtchnl_vlan_filter_list)) /
- sizeof(u16);
- len = sizeof(struct virtchnl_vlan_filter_list) +
- (count * sizeof(u16));
- more = true;
- }
- vvfl = kzalloc(len, GFP_ATOMIC);
- if (!vvfl) {
- spin_unlock_bh(&adapter->mac_vlan_list_lock);
- return;
- }
+ if (VLAN_ALLOWED(adapter)) {
+ struct virtchnl_vlan_filter_list *vvfl;
- vvfl->vsi_id = adapter->vsi_res->vsi_id;
- vvfl->num_elements = count;
- list_for_each_entry(f, &adapter->vlan_filter_list, list) {
- if (f->add) {
- vvfl->vlan_id[i] = f->vlan;
- i++;
- f->add = false;
- if (i == count)
- break;
+ adapter->current_op = VIRTCHNL_OP_ADD_VLAN;
+
+ len = sizeof(*vvfl) + (count * sizeof(u16));
+ if (len > IAVF_MAX_AQ_BUF_SIZE) {
+ dev_warn(&adapter->pdev->dev, "Too many add VLAN changes in one request\n");
+ count = (IAVF_MAX_AQ_BUF_SIZE - sizeof(*vvfl)) /
+ sizeof(u16);
+ len = sizeof(*vvfl) + (count * sizeof(u16));
+ more = true;
}
+ vvfl = kzalloc(len, GFP_ATOMIC);
+ if (!vvfl) {
+ spin_unlock_bh(&adapter->mac_vlan_list_lock);
+ return;
+ }
+
+ vvfl->vsi_id = adapter->vsi_res->vsi_id;
+ vvfl->num_elements = count;
+ list_for_each_entry(f, &adapter->vlan_filter_list, list) {
+ if (f->add) {
+ vvfl->vlan_id[i] = f->vlan.vid;
+ i++;
+ f->add = false;
+ if (i == count)
+ break;
+ }
+ }
+ if (!more)
+ adapter->aq_required &= ~IAVF_FLAG_AQ_ADD_VLAN_FILTER;
+
+ spin_unlock_bh(&adapter->mac_vlan_list_lock);
+
+ iavf_send_pf_msg(adapter, VIRTCHNL_OP_ADD_VLAN, (u8 *)vvfl, len);
+ kfree(vvfl);
+ } else {
+ struct virtchnl_vlan_filter_list_v2 *vvfl_v2;
+
+ adapter->current_op = VIRTCHNL_OP_ADD_VLAN_V2;
+
+ len = sizeof(*vvfl_v2) + ((count - 1) *
+ sizeof(struct virtchnl_vlan_filter));
+ if (len > IAVF_MAX_AQ_BUF_SIZE) {
+ dev_warn(&adapter->pdev->dev, "Too many add VLAN changes in one request\n");
+ count = (IAVF_MAX_AQ_BUF_SIZE - sizeof(*vvfl_v2)) /
+ sizeof(struct virtchnl_vlan_filter);
+ len = sizeof(*vvfl_v2) +
+ ((count - 1) *
+ sizeof(struct virtchnl_vlan_filter));
+ more = true;
+ }
+
+ vvfl_v2 = kzalloc(len, GFP_ATOMIC);
+ if (!vvfl_v2) {
+ spin_unlock_bh(&adapter->mac_vlan_list_lock);
+ return;
+ }
+
+ vvfl_v2->vport_id = adapter->vsi_res->vsi_id;
+ vvfl_v2->num_elements = count;
+ list_for_each_entry(f, &adapter->vlan_filter_list, list) {
+ if (f->add) {
+ struct virtchnl_vlan_supported_caps *filtering_support =
+ &adapter->vlan_v2_caps.filtering.filtering_support;
+ struct virtchnl_vlan *vlan;
+
+ /* give priority over outer if it's enabled */
+ if (filtering_support->outer)
+ vlan = &vvfl_v2->filters[i].outer;
+ else
+ vlan = &vvfl_v2->filters[i].inner;
+
+ vlan->tci = f->vlan.vid;
+ vlan->tpid = f->vlan.tpid;
+
+ i++;
+ f->add = false;
+ if (i == count)
+ break;
+ }
+ }
+
+ if (!more)
+ adapter->aq_required &= ~IAVF_FLAG_AQ_ADD_VLAN_FILTER;
+
+ spin_unlock_bh(&adapter->mac_vlan_list_lock);
+
+ iavf_send_pf_msg(adapter, VIRTCHNL_OP_ADD_VLAN_V2,
+ (u8 *)vvfl_v2, len);
+ kfree(vvfl_v2);
}
- if (!more)
- adapter->aq_required &= ~IAVF_FLAG_AQ_ADD_VLAN_FILTER;
-
- spin_unlock_bh(&adapter->mac_vlan_list_lock);
-
- iavf_send_pf_msg(adapter, VIRTCHNL_OP_ADD_VLAN, (u8 *)vvfl, len);
- kfree(vvfl);
}
/**
@@ -659,7 +768,6 @@ void iavf_add_vlans(struct iavf_adapter *adapter)
**/
void iavf_del_vlans(struct iavf_adapter *adapter)
{
- struct virtchnl_vlan_filter_list *vvfl;
struct iavf_vlan_filter *f, *ftmp;
int len, i = 0, count = 0;
bool more = false;
@@ -680,56 +788,116 @@ void iavf_del_vlans(struct iavf_adapter *adapter)
* filters marked for removal to enable bailing out before
* sending a virtchnl message
*/
- if (f->remove && !VLAN_ALLOWED(adapter)) {
+ if (f->remove && !VLAN_FILTERING_ALLOWED(adapter)) {
list_del(&f->list);
kfree(f);
} else if (f->remove) {
count++;
}
}
- if (!count) {
+ if (!count || !VLAN_FILTERING_ALLOWED(adapter)) {
adapter->aq_required &= ~IAVF_FLAG_AQ_DEL_VLAN_FILTER;
spin_unlock_bh(&adapter->mac_vlan_list_lock);
return;
}
- adapter->current_op = VIRTCHNL_OP_DEL_VLAN;
- len = sizeof(struct virtchnl_vlan_filter_list) +
- (count * sizeof(u16));
- if (len > IAVF_MAX_AQ_BUF_SIZE) {
- dev_warn(&adapter->pdev->dev, "Too many delete VLAN changes in one request\n");
- count = (IAVF_MAX_AQ_BUF_SIZE -
- sizeof(struct virtchnl_vlan_filter_list)) /
- sizeof(u16);
- len = sizeof(struct virtchnl_vlan_filter_list) +
- (count * sizeof(u16));
- more = true;
- }
- vvfl = kzalloc(len, GFP_ATOMIC);
- if (!vvfl) {
- spin_unlock_bh(&adapter->mac_vlan_list_lock);
- return;
- }
+ if (VLAN_ALLOWED(adapter)) {
+ struct virtchnl_vlan_filter_list *vvfl;
- vvfl->vsi_id = adapter->vsi_res->vsi_id;
- vvfl->num_elements = count;
- list_for_each_entry_safe(f, ftmp, &adapter->vlan_filter_list, list) {
- if (f->remove) {
- vvfl->vlan_id[i] = f->vlan;
- i++;
- list_del(&f->list);
- kfree(f);
- if (i == count)
- break;
+ adapter->current_op = VIRTCHNL_OP_DEL_VLAN;
+
+ len = sizeof(*vvfl) + (count * sizeof(u16));
+ if (len > IAVF_MAX_AQ_BUF_SIZE) {
+ dev_warn(&adapter->pdev->dev, "Too many delete VLAN changes in one request\n");
+ count = (IAVF_MAX_AQ_BUF_SIZE - sizeof(*vvfl)) /
+ sizeof(u16);
+ len = sizeof(*vvfl) + (count * sizeof(u16));
+ more = true;
}
+ vvfl = kzalloc(len, GFP_ATOMIC);
+ if (!vvfl) {
+ spin_unlock_bh(&adapter->mac_vlan_list_lock);
+ return;
+ }
+
+ vvfl->vsi_id = adapter->vsi_res->vsi_id;
+ vvfl->num_elements = count;
+ list_for_each_entry_safe(f, ftmp, &adapter->vlan_filter_list, list) {
+ if (f->remove) {
+ vvfl->vlan_id[i] = f->vlan.vid;
+ i++;
+ list_del(&f->list);
+ kfree(f);
+ if (i == count)
+ break;
+ }
+ }
+
+ if (!more)
+ adapter->aq_required &= ~IAVF_FLAG_AQ_DEL_VLAN_FILTER;
+
+ spin_unlock_bh(&adapter->mac_vlan_list_lock);
+
+ iavf_send_pf_msg(adapter, VIRTCHNL_OP_DEL_VLAN, (u8 *)vvfl, len);
+ kfree(vvfl);
+ } else {
+ struct virtchnl_vlan_filter_list_v2 *vvfl_v2;
+
+ adapter->current_op = VIRTCHNL_OP_DEL_VLAN_V2;
+
+ len = sizeof(*vvfl_v2) +
+ ((count - 1) * sizeof(struct virtchnl_vlan_filter));
+ if (len > IAVF_MAX_AQ_BUF_SIZE) {
+ dev_warn(&adapter->pdev->dev, "Too many add VLAN changes in one request\n");
+ count = (IAVF_MAX_AQ_BUF_SIZE -
+ sizeof(*vvfl_v2)) /
+ sizeof(struct virtchnl_vlan_filter);
+ len = sizeof(*vvfl_v2) +
+ ((count - 1) *
+ sizeof(struct virtchnl_vlan_filter));
+ more = true;
+ }
+
+ vvfl_v2 = kzalloc(len, GFP_ATOMIC);
+ if (!vvfl_v2) {
+ spin_unlock_bh(&adapter->mac_vlan_list_lock);
+ return;
+ }
+
+ vvfl_v2->vport_id = adapter->vsi_res->vsi_id;
+ vvfl_v2->num_elements = count;
+ list_for_each_entry_safe(f, ftmp, &adapter->vlan_filter_list, list) {
+ if (f->remove) {
+ struct virtchnl_vlan_supported_caps *filtering_support =
+ &adapter->vlan_v2_caps.filtering.filtering_support;
+ struct virtchnl_vlan *vlan;
+
+ /* give priority over outer if it's enabled */
+ if (filtering_support->outer)
+ vlan = &vvfl_v2->filters[i].outer;
+ else
+ vlan = &vvfl_v2->filters[i].inner;
+
+ vlan->tci = f->vlan.vid;
+ vlan->tpid = f->vlan.tpid;
+
+ list_del(&f->list);
+ kfree(f);
+ i++;
+ if (i == count)
+ break;
+ }
+ }
+
+ if (!more)
+ adapter->aq_required &= ~IAVF_FLAG_AQ_DEL_VLAN_FILTER;
+
+ spin_unlock_bh(&adapter->mac_vlan_list_lock);
+
+ iavf_send_pf_msg(adapter, VIRTCHNL_OP_DEL_VLAN_V2,
+ (u8 *)vvfl_v2, len);
+ kfree(vvfl_v2);
}
- if (!more)
- adapter->aq_required &= ~IAVF_FLAG_AQ_DEL_VLAN_FILTER;
-
- spin_unlock_bh(&adapter->mac_vlan_list_lock);
-
- iavf_send_pf_msg(adapter, VIRTCHNL_OP_DEL_VLAN, (u8 *)vvfl, len);
- kfree(vvfl);
}
/**
@@ -762,15 +930,23 @@ void iavf_set_promiscuous(struct iavf_adapter *adapter, int flags)
if (flags & FLAG_VF_MULTICAST_PROMISC) {
adapter->flags |= IAVF_FLAG_ALLMULTI_ON;
adapter->aq_required &= ~IAVF_FLAG_AQ_REQUEST_ALLMULTI;
- dev_info(&adapter->pdev->dev, "Entering multicast promiscuous mode\n");
+ dev_info(&adapter->pdev->dev, "%s is entering multicast promiscuous mode\n",
+ adapter->netdev->name);
}
if (!flags) {
- adapter->flags &= ~(IAVF_FLAG_PROMISC_ON |
- IAVF_FLAG_ALLMULTI_ON);
- adapter->aq_required &= ~(IAVF_FLAG_AQ_RELEASE_PROMISC |
- IAVF_FLAG_AQ_RELEASE_ALLMULTI);
- dev_info(&adapter->pdev->dev, "Leaving promiscuous mode\n");
+ if (adapter->flags & IAVF_FLAG_PROMISC_ON) {
+ adapter->flags &= ~IAVF_FLAG_PROMISC_ON;
+ adapter->aq_required &= ~IAVF_FLAG_AQ_RELEASE_PROMISC;
+ dev_info(&adapter->pdev->dev, "Leaving promiscuous mode\n");
+ }
+
+ if (adapter->flags & IAVF_FLAG_ALLMULTI_ON) {
+ adapter->flags &= ~IAVF_FLAG_ALLMULTI_ON;
+ adapter->aq_required &= ~IAVF_FLAG_AQ_RELEASE_ALLMULTI;
+ dev_info(&adapter->pdev->dev, "%s is leaving multicast promiscuous mode\n",
+ adapter->netdev->name);
+ }
}
adapter->current_op = VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE;
@@ -948,6 +1124,204 @@ void iavf_disable_vlan_stripping(struct iavf_adapter *adapter)
iavf_send_pf_msg(adapter, VIRTCHNL_OP_DISABLE_VLAN_STRIPPING, NULL, 0);
}
+/**
+ * iavf_tpid_to_vc_ethertype - transform from VLAN TPID to virtchnl ethertype
+ * @tpid: VLAN TPID (i.e. 0x8100, 0x88a8, etc.)
+ */
+static u32 iavf_tpid_to_vc_ethertype(u16 tpid)
+{
+ switch (tpid) {
+ case ETH_P_8021Q:
+ return VIRTCHNL_VLAN_ETHERTYPE_8100;
+ case ETH_P_8021AD:
+ return VIRTCHNL_VLAN_ETHERTYPE_88A8;
+ }
+
+ return 0;
+}
+
+/**
+ * iavf_set_vc_offload_ethertype - set virtchnl ethertype for offload message
+ * @adapter: adapter structure
+ * @msg: message structure used for updating offloads over virtchnl to update
+ * @tpid: VLAN TPID (i.e. 0x8100, 0x88a8, etc.)
+ * @offload_op: opcode used to determine which support structure to check
+ */
+static int
+iavf_set_vc_offload_ethertype(struct iavf_adapter *adapter,
+ struct virtchnl_vlan_setting *msg, u16 tpid,
+ enum virtchnl_ops offload_op)
+{
+ struct virtchnl_vlan_supported_caps *offload_support;
+ u16 vc_ethertype = iavf_tpid_to_vc_ethertype(tpid);
+
+ /* reference the correct offload support structure */
+ switch (offload_op) {
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2:
+ offload_support =
+ &adapter->vlan_v2_caps.offloads.stripping_support;
+ break;
+ case VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2:
+ offload_support =
+ &adapter->vlan_v2_caps.offloads.insertion_support;
+ break;
+ default:
+ dev_err(&adapter->pdev->dev, "Invalid opcode %d for setting virtchnl ethertype to enable/disable VLAN offloads\n",
+ offload_op);
+ return -EINVAL;
+ }
+
+ /* make sure ethertype is supported */
+ if (offload_support->outer & vc_ethertype &&
+ offload_support->outer & VIRTCHNL_VLAN_TOGGLE) {
+ msg->outer_ethertype_setting = vc_ethertype;
+ } else if (offload_support->inner & vc_ethertype &&
+ offload_support->inner & VIRTCHNL_VLAN_TOGGLE) {
+ msg->inner_ethertype_setting = vc_ethertype;
+ } else {
+ dev_dbg(&adapter->pdev->dev, "opcode %d unsupported for VLAN TPID 0x%04x\n",
+ offload_op, tpid);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+/**
+ * iavf_clear_offload_v2_aq_required - clear AQ required bit for offload request
+ * @adapter: adapter structure
+ * @tpid: VLAN TPID
+ * @offload_op: opcode used to determine which AQ required bit to clear
+ */
+static void
+iavf_clear_offload_v2_aq_required(struct iavf_adapter *adapter, u16 tpid,
+ enum virtchnl_ops offload_op)
+{
+ switch (offload_op) {
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2:
+ if (tpid == ETH_P_8021Q)
+ adapter->aq_required &=
+ ~IAVF_FLAG_AQ_ENABLE_CTAG_VLAN_STRIPPING;
+ else if (tpid == ETH_P_8021AD)
+ adapter->aq_required &=
+ ~IAVF_FLAG_AQ_ENABLE_STAG_VLAN_STRIPPING;
+ break;
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2:
+ if (tpid == ETH_P_8021Q)
+ adapter->aq_required &=
+ ~IAVF_FLAG_AQ_DISABLE_CTAG_VLAN_STRIPPING;
+ else if (tpid == ETH_P_8021AD)
+ adapter->aq_required &=
+ ~IAVF_FLAG_AQ_DISABLE_STAG_VLAN_STRIPPING;
+ break;
+ case VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2:
+ if (tpid == ETH_P_8021Q)
+ adapter->aq_required &=
+ ~IAVF_FLAG_AQ_ENABLE_CTAG_VLAN_INSERTION;
+ else if (tpid == ETH_P_8021AD)
+ adapter->aq_required &=
+ ~IAVF_FLAG_AQ_ENABLE_STAG_VLAN_INSERTION;
+ break;
+ case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2:
+ if (tpid == ETH_P_8021Q)
+ adapter->aq_required &=
+ ~IAVF_FLAG_AQ_DISABLE_CTAG_VLAN_INSERTION;
+ else if (tpid == ETH_P_8021AD)
+ adapter->aq_required &=
+ ~IAVF_FLAG_AQ_DISABLE_STAG_VLAN_INSERTION;
+ break;
+ default:
+ dev_err(&adapter->pdev->dev, "Unsupported opcode %d specified for clearing aq_required bits for VIRTCHNL_VF_OFFLOAD_VLAN_V2 offload request\n",
+ offload_op);
+ }
+}
+
+/**
+ * iavf_send_vlan_offload_v2 - send offload enable/disable over virtchnl
+ * @adapter: adapter structure
+ * @tpid: VLAN TPID used for the command (i.e. 0x8100 or 0x88a8)
+ * @offload_op: offload_op used to make the request over virtchnl
+ */
+static void
+iavf_send_vlan_offload_v2(struct iavf_adapter *adapter, u16 tpid,
+ enum virtchnl_ops offload_op)
+{
+ struct virtchnl_vlan_setting *msg;
+ int len = sizeof(*msg);
+
+ if (adapter->current_op != VIRTCHNL_OP_UNKNOWN) {
+ /* bail because we already have a command pending */
+ dev_err(&adapter->pdev->dev, "Cannot send %d, command %d pending\n",
+ offload_op, adapter->current_op);
+ return;
+ }
+
+ adapter->current_op = offload_op;
+
+ msg = kzalloc(len, GFP_KERNEL);
+ if (!msg)
+ return;
+
+ msg->vport_id = adapter->vsi_res->vsi_id;
+
+ /* always clear to prevent unsupported and endless requests */
+ iavf_clear_offload_v2_aq_required(adapter, tpid, offload_op);
+
+ /* only send valid offload requests */
+ if (!iavf_set_vc_offload_ethertype(adapter, msg, tpid, offload_op))
+ iavf_send_pf_msg(adapter, offload_op, (u8 *)msg, len);
+ else
+ adapter->current_op = VIRTCHNL_OP_UNKNOWN;
+
+ kfree(msg);
+}
+
+/**
+ * iavf_enable_vlan_stripping_v2 - enable VLAN stripping
+ * @adapter: adapter structure
+ * @tpid: VLAN TPID used to enable VLAN stripping
+ */
+void iavf_enable_vlan_stripping_v2(struct iavf_adapter *adapter, u16 tpid)
+{
+ iavf_send_vlan_offload_v2(adapter, tpid,
+ VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2);
+}
+
+/**
+ * iavf_disable_vlan_stripping_v2 - disable VLAN stripping
+ * @adapter: adapter structure
+ * @tpid: VLAN TPID used to disable VLAN stripping
+ */
+void iavf_disable_vlan_stripping_v2(struct iavf_adapter *adapter, u16 tpid)
+{
+ iavf_send_vlan_offload_v2(adapter, tpid,
+ VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2);
+}
+
+/**
+ * iavf_enable_vlan_insertion_v2 - enable VLAN insertion
+ * @adapter: adapter structure
+ * @tpid: VLAN TPID used to enable VLAN insertion
+ */
+void iavf_enable_vlan_insertion_v2(struct iavf_adapter *adapter, u16 tpid)
+{
+ iavf_send_vlan_offload_v2(adapter, tpid,
+ VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2);
+}
+
+/**
+ * iavf_disable_vlan_insertion_v2 - disable VLAN insertion
+ * @adapter: adapter structure
+ * @tpid: VLAN TPID used to disable VLAN insertion
+ */
+void iavf_disable_vlan_insertion_v2(struct iavf_adapter *adapter, u16 tpid)
+{
+ iavf_send_vlan_offload_v2(adapter, tpid,
+ VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2);
+}
+
#define IAVF_MAX_SPEED_STRLEN 13
/**
@@ -1017,7 +1391,7 @@ static void iavf_print_link_message(struct iavf_adapter *adapter)
} else if (link_speed_mbps == SPEED_UNKNOWN) {
snprintf(speed, IAVF_MAX_SPEED_STRLEN, "%s", "Unknown Mbps");
} else {
- snprintf(speed, IAVF_MAX_SPEED_STRLEN, "%u %s",
+ snprintf(speed, IAVF_MAX_SPEED_STRLEN, "%d %s",
link_speed_mbps, "Mbps");
}
@@ -1522,7 +1896,7 @@ void iavf_virtchnl_completion(struct iavf_adapter *adapter,
iavf_print_link_message(adapter);
break;
case VIRTCHNL_EVENT_RESET_IMPENDING:
- dev_info(&adapter->pdev->dev, "Reset warning received from the PF\n");
+ dev_info(&adapter->pdev->dev, "Reset indication received from the PF\n");
if (!(adapter->flags & IAVF_FLAG_RESET_PENDING)) {
adapter->flags |= IAVF_FLAG_RESET_PENDING;
dev_info(&adapter->pdev->dev, "Scheduling reset task\n");
@@ -1751,6 +2125,26 @@ void iavf_virtchnl_completion(struct iavf_adapter *adapter,
}
spin_unlock_bh(&adapter->mac_vlan_list_lock);
+
+ iavf_parse_vf_resource_msg(adapter);
+
+ /* negotiated VIRTCHNL_VF_OFFLOAD_VLAN_V2, so wait for the
+ * response to VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS to finish
+ * configuration
+ */
+ if (VLAN_V2_ALLOWED(adapter))
+ break;
+ /* fallthrough and finish config if VIRTCHNL_VF_OFFLOAD_VLAN_V2
+ * wasn't successfully negotiated with the PF
+ */
+ }
+ fallthrough;
+ case VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS: {
+ if (v_opcode == VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS)
+ memcpy(&adapter->vlan_v2_caps, msg,
+ min_t(u16, msglen,
+ sizeof(adapter->vlan_v2_caps)));
+
iavf_process_config(adapter);
/* unlock crit_lock before acquiring rtnl_lock as other
@@ -1758,13 +2152,23 @@ void iavf_virtchnl_completion(struct iavf_adapter *adapter,
* crit_lock
*/
mutex_unlock(&adapter->crit_lock);
+ /* VLAN capabilities can change during VFR, so make sure to
+ * update the netdev features with the new capabilities
+ */
rtnl_lock();
- netdev_update_features(adapter->netdev);
+ netdev_update_features(netdev);
rtnl_unlock();
if (iavf_lock_timeout(&adapter->crit_lock, 10000))
dev_warn(&adapter->pdev->dev, "failed to acquire crit_lock in %s\n",
__FUNCTION__);
+ /* Request VLAN offload settings */
+ if (VLAN_V2_ALLOWED(adapter))
+ iavf_set_vlan_offload_features(adapter, 0,
+ netdev->features);
+
+ iavf_set_queue_vlan_tag_loc(adapter);
+
}
break;
case VIRTCHNL_OP_ENABLE_QUEUES:
diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h
index b2db39e..4e16d18 100644
--- a/drivers/net/ethernet/intel/ice/ice.h
+++ b/drivers/net/ethernet/intel/ice/ice.h
@@ -112,7 +112,6 @@
#define ICE_MAX_RXQS_PER_TC 256 /* Used when setting VSI context per TC Rx queues */
#define ICE_CHNL_START_TC 1
-#define ICE_CHNL_MAX_TC 16
#define ICE_MAX_RESET_WAIT 20
@@ -201,6 +200,7 @@ struct ice_channel {
struct ice_aqc_vsi_props info;
u64 max_tx_rate;
u64 min_tx_rate;
+ atomic_t num_sb_fltr;
struct ice_vsi *ch_vsi;
};
@@ -503,6 +503,7 @@ struct ice_pf {
struct pci_dev *pdev;
struct devlink_region *nvm_region;
+ struct devlink_region *sram_region;
struct devlink_region *devcaps_region;
/* devlink port data */
@@ -552,6 +553,7 @@ struct ice_pf {
spinlock_t aq_wait_lock;
struct hlist_head aq_wait_list;
wait_queue_head_t aq_wait_queue;
+ bool fw_emp_reset_disabled;
wait_queue_head_t reset_wait_queue;
@@ -576,6 +578,7 @@ struct ice_pf {
struct ice_hw_port_stats stats_prev;
struct ice_hw hw;
u8 stat_prev_loaded:1; /* has previous stats been loaded */
+ u8 rdma_mode;
u16 dcbx_cap;
u32 tx_timeout_count;
unsigned long tx_timeout_last_recovery;
@@ -789,6 +792,9 @@ static inline void ice_clear_sriov_cap(struct ice_pf *pf)
#define ICE_FD_STAT_PF_IDX(base_idx) \
((base_idx) * ICE_FD_STAT_CTR_BLOCK_COUNT)
#define ICE_FD_SB_STAT_IDX(base_idx) ICE_FD_STAT_PF_IDX(base_idx)
+#define ICE_FD_STAT_CH 1
+#define ICE_FD_CH_STAT_IDX(base_idx) \
+ (ICE_FD_STAT_PF_IDX(base_idx) + ICE_FD_STAT_CH)
/**
* ice_is_adq_active - any active ADQs
@@ -847,9 +853,9 @@ void ice_print_link_msg(struct ice_vsi *vsi, bool isup);
int ice_plug_aux_dev(struct ice_pf *pf);
void ice_unplug_aux_dev(struct ice_pf *pf);
int ice_init_rdma(struct ice_pf *pf);
-const char *ice_stat_str(enum ice_status stat_err);
const char *ice_aq_str(enum ice_aq_err aq_err);
bool ice_is_wol_supported(struct ice_hw *hw);
+void ice_fdir_del_all_fltrs(struct ice_vsi *vsi);
int
ice_fdir_write_fltr(struct ice_pf *pf, struct ice_fdir_fltr *input, bool add,
bool is_tun);
@@ -860,6 +866,7 @@ int ice_get_ethtool_fdir_entry(struct ice_hw *hw, struct ethtool_rxnfc *cmd);
int
ice_get_fdir_fltr_ids(struct ice_hw *hw, struct ethtool_rxnfc *cmd,
u32 *rule_locs);
+void ice_fdir_rem_adq_chnl(struct ice_hw *hw, u16 vsi_idx);
void ice_fdir_release_flows(struct ice_hw *hw);
void ice_fdir_replay_flows(struct ice_hw *hw);
void ice_fdir_replay_fltrs(struct ice_pf *pf);
diff --git a/drivers/net/ethernet/intel/ice/ice_adminq_cmd.h b/drivers/net/ethernet/intel/ice/ice_adminq_cmd.h
index 4eef348..ad1dcfa 100644
--- a/drivers/net/ethernet/intel/ice/ice_adminq_cmd.h
+++ b/drivers/net/ethernet/intel/ice/ice_adminq_cmd.h
@@ -117,6 +117,8 @@ struct ice_aqc_list_caps_elem {
#define ICE_AQC_CAPS_NET_VER 0x004C
#define ICE_AQC_CAPS_PENDING_NET_VER 0x004D
#define ICE_AQC_CAPS_RDMA 0x0051
+#define ICE_AQC_CAPS_PCIE_RESET_AVOIDANCE 0x0076
+#define ICE_AQC_CAPS_POST_UPDATE_RESET_RESTRICT 0x0077
#define ICE_AQC_CAPS_NVM_MGMT 0x0080
u8 major_ver;
@@ -1408,6 +1410,11 @@ struct ice_aqc_nvm {
#define ICE_AQC_NVM_REVERT_LAST_ACTIV BIT(6) /* Write Activate only */
#define ICE_AQC_NVM_ACTIV_SEL_MASK ICE_M(0x7, 3)
#define ICE_AQC_NVM_FLASH_ONLY BIT(7)
+#define ICE_AQC_NVM_RESET_LVL_M ICE_M(0x3, 0) /* Write reply only */
+#define ICE_AQC_NVM_POR_FLAG 0
+#define ICE_AQC_NVM_PERST_FLAG 1
+#define ICE_AQC_NVM_EMPR_FLAG 2
+#define ICE_AQC_NVM_EMPR_ENA BIT(0) /* Write Activate reply only */
__le16 module_typeid;
__le16 length;
#define ICE_AQC_NVM_ERASE_LEN 0xFFFF
diff --git a/drivers/net/ethernet/intel/ice/ice_base.c b/drivers/net/ethernet/intel/ice/ice_base.c
index fafe020..1a5ece3 100644
--- a/drivers/net/ethernet/intel/ice/ice_base.c
+++ b/drivers/net/ethernet/intel/ice/ice_base.c
@@ -776,7 +776,7 @@ ice_vsi_cfg_txq(struct ice_vsi *vsi, struct ice_tx_ring *ring,
struct ice_channel *ch = ring->ch;
struct ice_pf *pf = vsi->back;
struct ice_hw *hw = &pf->hw;
- enum ice_status status;
+ int status;
u16 pf_q;
u8 tc;
@@ -821,9 +821,9 @@ ice_vsi_cfg_txq(struct ice_vsi *vsi, struct ice_tx_ring *ring,
ring->q_handle, 1, qg_buf, buf_len,
NULL);
if (status) {
- dev_err(ice_pf_to_dev(pf), "Failed to set LAN Tx queue context, error: %s\n",
- ice_stat_str(status));
- return -ENODEV;
+ dev_err(ice_pf_to_dev(pf), "Failed to set LAN Tx queue context, error: %d\n",
+ status);
+ return status;
}
/* Add Tx Queue TEID into the VSI Tx ring from the
@@ -946,7 +946,7 @@ ice_vsi_stop_tx_ring(struct ice_vsi *vsi, enum ice_disq_rst_src rst_src,
struct ice_pf *pf = vsi->back;
struct ice_q_vector *q_vector;
struct ice_hw *hw = &pf->hw;
- enum ice_status status;
+ int status;
u32 val;
/* clear cause_ena bit for disabled queues */
@@ -970,18 +970,18 @@ ice_vsi_stop_tx_ring(struct ice_vsi *vsi, enum ice_disq_rst_src rst_src,
rel_vmvf_num, NULL);
/* if the disable queue command was exercised during an
- * active reset flow, ICE_ERR_RESET_ONGOING is returned.
+ * active reset flow, -EBUSY is returned.
* This is not an error as the reset operation disables
* queues at the hardware level anyway.
*/
- if (status == ICE_ERR_RESET_ONGOING) {
+ if (status == -EBUSY) {
dev_dbg(ice_pf_to_dev(vsi->back), "Reset in progress. LAN Tx queues already disabled\n");
- } else if (status == ICE_ERR_DOES_NOT_EXIST) {
+ } else if (status == -ENOENT) {
dev_dbg(ice_pf_to_dev(vsi->back), "LAN Tx queues do not exist, nothing to disable\n");
} else if (status) {
- dev_dbg(ice_pf_to_dev(vsi->back), "Failed to disable LAN Tx queues, error: %s\n",
- ice_stat_str(status));
- return -ENODEV;
+ dev_dbg(ice_pf_to_dev(vsi->back), "Failed to disable LAN Tx queues, error: %d\n",
+ status);
+ return status;
}
return 0;
diff --git a/drivers/net/ethernet/intel/ice/ice_cgu_regs.h b/drivers/net/ethernet/intel/ice/ice_cgu_regs.h
new file mode 100644
index 0000000..57abd52
--- /dev/null
+++ b/drivers/net/ethernet/intel/ice/ice_cgu_regs.h
@@ -0,0 +1,116 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2018-2021, Intel Corporation. */
+
+#ifndef _ICE_CGU_REGS_H_
+#define _ICE_CGU_REGS_H_
+
+#define NAC_CGU_DWORD9 0x24
+union nac_cgu_dword9 {
+ struct {
+ u32 time_ref_freq_sel : 3;
+ u32 clk_eref1_en : 1;
+ u32 clk_eref0_en : 1;
+ u32 time_ref_en : 1;
+ u32 time_sync_en : 1;
+ u32 one_pps_out_en : 1;
+ u32 clk_ref_synce_en : 1;
+ u32 clk_synce1_en : 1;
+ u32 clk_synce0_en : 1;
+ u32 net_clk_ref1_en : 1;
+ u32 net_clk_ref0_en : 1;
+ u32 clk_synce1_amp : 2;
+ u32 misc6 : 1;
+ u32 clk_synce0_amp : 2;
+ u32 one_pps_out_amp : 2;
+ u32 misc24 : 12;
+ } field;
+ u32 val;
+};
+
+#define NAC_CGU_DWORD19 0x4c
+union nac_cgu_dword19 {
+ struct {
+ u32 tspll_fbdiv_intgr : 8;
+ u32 fdpll_ulck_thr : 5;
+ u32 misc15 : 3;
+ u32 tspll_ndivratio : 4;
+ u32 tspll_iref_ndivratio : 3;
+ u32 misc19 : 1;
+ u32 japll_ndivratio : 4;
+ u32 japll_iref_ndivratio : 3;
+ u32 misc27 : 1;
+ } field;
+ u32 val;
+};
+
+#define NAC_CGU_DWORD22 0x58
+union nac_cgu_dword22 {
+ struct {
+ u32 fdpll_frac_div_out_nc : 2;
+ u32 fdpll_lock_int_for : 1;
+ u32 synce_hdov_int_for : 1;
+ u32 synce_lock_int_for : 1;
+ u32 fdpll_phlead_slip_nc : 1;
+ u32 fdpll_acc1_ovfl_nc : 1;
+ u32 fdpll_acc2_ovfl_nc : 1;
+ u32 synce_status_nc : 6;
+ u32 fdpll_acc1f_ovfl : 1;
+ u32 misc18 : 1;
+ u32 fdpllclk_div : 4;
+ u32 time1588clk_div : 4;
+ u32 synceclk_div : 4;
+ u32 synceclk_sel_div2 : 1;
+ u32 fdpllclk_sel_div2 : 1;
+ u32 time1588clk_sel_div2 : 1;
+ u32 misc3 : 1;
+ } field;
+ u32 val;
+};
+
+#define NAC_CGU_DWORD24 0x60
+union nac_cgu_dword24 {
+ struct {
+ u32 tspll_fbdiv_frac : 22;
+ u32 misc20 : 2;
+ u32 ts_pll_enable : 1;
+ u32 time_sync_tspll_align_sel : 1;
+ u32 ext_synce_sel : 1;
+ u32 ref1588_ck_div : 4;
+ u32 time_ref_sel : 1;
+ } field;
+ u32 val;
+};
+
+#define TSPLL_CNTR_BIST_SETTINGS 0x344
+union tspll_cntr_bist_settings {
+ struct {
+ u32 i_irefgen_settling_time_cntr_7_0 : 8;
+ u32 i_irefgen_settling_time_ro_standby_1_0 : 2;
+ u32 reserved195 : 5;
+ u32 i_plllock_sel_0 : 1;
+ u32 i_plllock_sel_1 : 1;
+ u32 i_plllock_cnt_6_0 : 7;
+ u32 i_plllock_cnt_10_7 : 4;
+ u32 reserved200 : 4;
+ } field;
+ u32 val;
+};
+
+#define TSPLL_RO_BWM_LF 0x370
+union tspll_ro_bwm_lf {
+ struct {
+ u32 bw_freqov_high_cri_7_0 : 8;
+ u32 bw_freqov_high_cri_9_8 : 2;
+ u32 biascaldone_cri : 1;
+ u32 plllock_gain_tran_cri : 1;
+ u32 plllock_true_lock_cri : 1;
+ u32 pllunlock_flag_cri : 1;
+ u32 afcerr_cri : 1;
+ u32 afcdone_cri : 1;
+ u32 feedfwrdgain_cal_cri_7_0 : 8;
+ u32 m2fbdivmod_cri_7_0 : 8;
+ } field;
+ u32 val;
+};
+
+#endif /* _ICE_CGU_REGS_H_ */
diff --git a/drivers/net/ethernet/intel/ice/ice_common.c b/drivers/net/ethernet/intel/ice/ice_common.c
index b3066d0..408d15a 100644
--- a/drivers/net/ethernet/intel/ice/ice_common.c
+++ b/drivers/net/ethernet/intel/ice/ice_common.c
@@ -2,7 +2,6 @@
/* Copyright (c) 2018, Intel Corporation. */
#include "ice_common.h"
-#include "ice_lib.h"
#include "ice_sched.h"
#include "ice_adminq_cmd.h"
#include "ice_flow.h"
@@ -16,10 +15,10 @@
* This function sets the MAC type of the adapter based on the
* vendor ID and device ID stored in the HW structure.
*/
-static enum ice_status ice_set_mac_type(struct ice_hw *hw)
+static int ice_set_mac_type(struct ice_hw *hw)
{
if (hw->vendor_id != PCI_VENDOR_ID_INTEL)
- return ICE_ERR_DEVICE_NOT_SUPPORTED;
+ return -ENODEV;
switch (hw->device_id) {
case ICE_DEV_ID_E810C_BACKPLANE:
@@ -99,7 +98,7 @@ bool ice_is_e810t(struct ice_hw *hw)
* Clears any existing PF configuration (VSIs, VSI lists, switch rules, port
* configuration, flow director filters, etc.).
*/
-enum ice_status ice_clear_pf_cfg(struct ice_hw *hw)
+int ice_clear_pf_cfg(struct ice_hw *hw)
{
struct ice_aq_desc desc;
@@ -123,21 +122,21 @@ enum ice_status ice_clear_pf_cfg(struct ice_hw *hw)
* ice_discover_dev_caps is expected to be called before this function is
* called.
*/
-static enum ice_status
+static int
ice_aq_manage_mac_read(struct ice_hw *hw, void *buf, u16 buf_size,
struct ice_sq_cd *cd)
{
struct ice_aqc_manage_mac_read_resp *resp;
struct ice_aqc_manage_mac_read *cmd;
struct ice_aq_desc desc;
- enum ice_status status;
+ int status;
u16 flags;
u8 i;
cmd = &desc.params.mac_read;
if (buf_size < sizeof(*resp))
- return ICE_ERR_BUF_TOO_SHORT;
+ return -EINVAL;
ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_manage_mac_read);
@@ -150,7 +149,7 @@ ice_aq_manage_mac_read(struct ice_hw *hw, void *buf, u16 buf_size,
if (!(flags & ICE_AQC_MAN_MAC_LAN_ADDR_VALID)) {
ice_debug(hw, ICE_DBG_LAN, "got invalid MAC address\n");
- return ICE_ERR_CFG;
+ return -EIO;
}
/* A single port can report up to two (LAN and WoL) addresses */
@@ -176,7 +175,7 @@ ice_aq_manage_mac_read(struct ice_hw *hw, void *buf, u16 buf_size,
*
* Returns the various PHY capabilities supported on the Port (0x0600)
*/
-enum ice_status
+int
ice_aq_get_phy_caps(struct ice_port_info *pi, bool qual_mods, u8 report_mode,
struct ice_aqc_get_phy_caps_data *pcaps,
struct ice_sq_cd *cd)
@@ -184,18 +183,18 @@ ice_aq_get_phy_caps(struct ice_port_info *pi, bool qual_mods, u8 report_mode,
struct ice_aqc_get_phy_caps *cmd;
u16 pcaps_size = sizeof(*pcaps);
struct ice_aq_desc desc;
- enum ice_status status;
struct ice_hw *hw;
+ int status;
cmd = &desc.params.get_phy;
if (!pcaps || (report_mode & ~ICE_AQC_REPORT_MODE_M) || !pi)
- return ICE_ERR_PARAM;
+ return -EINVAL;
hw = pi->hw;
if (report_mode == ICE_AQC_REPORT_DFLT_CFG &&
!ice_fw_supports_report_dflt_cfg(hw))
- return ICE_ERR_PARAM;
+ return -EINVAL;
ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_get_phy_caps);
@@ -252,7 +251,7 @@ ice_aq_get_phy_caps(struct ice_port_info *pi, bool qual_mods, u8 report_mode,
* returns error (ENOENT), then no cage present. If no cage present, then
* connection type is backplane or BASE-T.
*/
-static enum ice_status
+static int
ice_aq_get_link_topo_handle(struct ice_port_info *pi, u8 node_type,
struct ice_sq_cd *cd)
{
@@ -418,7 +417,7 @@ static enum ice_media_type ice_get_media_type(struct ice_port_info *pi)
*
* Get Link Status (0x607). Returns the link status of the adapter.
*/
-enum ice_status
+int
ice_aq_get_link_info(struct ice_port_info *pi, bool ena_lse,
struct ice_link_status *link, struct ice_sq_cd *cd)
{
@@ -429,12 +428,12 @@ ice_aq_get_link_info(struct ice_port_info *pi, bool ena_lse,
struct ice_fc_info *hw_fc_info;
bool tx_pause, rx_pause;
struct ice_aq_desc desc;
- enum ice_status status;
struct ice_hw *hw;
u16 cmd_flags;
+ int status;
if (!pi)
- return ICE_ERR_PARAM;
+ return -EINVAL;
hw = pi->hw;
li_old = &pi->phy.link_info_old;
hw_media_type = &pi->phy.media_type;
@@ -556,7 +555,7 @@ ice_fill_tx_timer_and_fc_thresh(struct ice_hw *hw,
*
* Set MAC configuration (0x0603)
*/
-enum ice_status
+int
ice_aq_set_mac_cfg(struct ice_hw *hw, u16 max_frame_size, struct ice_sq_cd *cd)
{
struct ice_aqc_set_mac_cfg *cmd;
@@ -565,7 +564,7 @@ ice_aq_set_mac_cfg(struct ice_hw *hw, u16 max_frame_size, struct ice_sq_cd *cd)
cmd = &desc.params.set_mac_cfg;
if (max_frame_size == 0)
- return ICE_ERR_PARAM;
+ return -EINVAL;
ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_set_mac_cfg);
@@ -580,17 +579,17 @@ ice_aq_set_mac_cfg(struct ice_hw *hw, u16 max_frame_size, struct ice_sq_cd *cd)
* ice_init_fltr_mgmt_struct - initializes filter management list and locks
* @hw: pointer to the HW struct
*/
-static enum ice_status ice_init_fltr_mgmt_struct(struct ice_hw *hw)
+static int ice_init_fltr_mgmt_struct(struct ice_hw *hw)
{
struct ice_switch_info *sw;
- enum ice_status status;
+ int status;
hw->switch_info = devm_kzalloc(ice_hw_to_dev(hw),
sizeof(*hw->switch_info), GFP_KERNEL);
sw = hw->switch_info;
if (!sw)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
INIT_LIST_HEAD(&sw->vsi_list_map_head);
sw->prof_res_bm_init = 0;
@@ -666,17 +665,17 @@ static void ice_cleanup_fltr_mgmt_struct(struct ice_hw *hw)
* ice_get_fw_log_cfg - get FW logging configuration
* @hw: pointer to the HW struct
*/
-static enum ice_status ice_get_fw_log_cfg(struct ice_hw *hw)
+static int ice_get_fw_log_cfg(struct ice_hw *hw)
{
struct ice_aq_desc desc;
- enum ice_status status;
__le16 *config;
+ int status;
u16 size;
size = sizeof(*config) * ICE_AQC_FW_LOG_ID_MAX;
config = devm_kzalloc(ice_hw_to_dev(hw), size, GFP_KERNEL);
if (!config)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_fw_logging_info);
@@ -738,15 +737,15 @@ static enum ice_status ice_get_fw_log_cfg(struct ice_hw *hw)
* messages from FW to SW. Interrupts are typically disabled during the device's
* initialization phase.
*/
-static enum ice_status ice_cfg_fw_log(struct ice_hw *hw, bool enable)
+static int ice_cfg_fw_log(struct ice_hw *hw, bool enable)
{
struct ice_aqc_fw_logging *cmd;
- enum ice_status status = 0;
u16 i, chgs = 0, len = 0;
struct ice_aq_desc desc;
__le16 *data = NULL;
u8 actv_evnts = 0;
void *buf = NULL;
+ int status = 0;
if (!hw->fw_log.cq_en && !hw->fw_log.uart_en)
return 0;
@@ -790,7 +789,7 @@ static enum ice_status ice_cfg_fw_log(struct ice_hw *hw, bool enable)
sizeof(*data),
GFP_KERNEL);
if (!data)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
}
val = i << ICE_AQC_FW_LOG_ID_S;
@@ -904,12 +903,12 @@ static void ice_get_itr_intrl_gran(struct ice_hw *hw)
* ice_init_hw - main hardware initialization routine
* @hw: pointer to the hardware structure
*/
-enum ice_status ice_init_hw(struct ice_hw *hw)
+int ice_init_hw(struct ice_hw *hw)
{
struct ice_aqc_get_phy_caps_data *pcaps;
- enum ice_status status;
u16 mac_buf_len;
void *mac_buf;
+ int status;
/* Set MAC type based on DeviceID */
status = ice_set_mac_type(hw);
@@ -956,7 +955,7 @@ enum ice_status ice_init_hw(struct ice_hw *hw)
hw->port_info = devm_kzalloc(ice_hw_to_dev(hw),
sizeof(*hw->port_info), GFP_KERNEL);
if (!hw->port_info) {
- status = ICE_ERR_NO_MEMORY;
+ status = -ENOMEM;
goto err_unroll_cqinit;
}
@@ -985,7 +984,7 @@ enum ice_status ice_init_hw(struct ice_hw *hw)
pcaps = devm_kzalloc(ice_hw_to_dev(hw), sizeof(*pcaps), GFP_KERNEL);
if (!pcaps) {
- status = ICE_ERR_NO_MEMORY;
+ status = -ENOMEM;
goto err_unroll_sched;
}
@@ -1006,7 +1005,7 @@ enum ice_status ice_init_hw(struct ice_hw *hw)
/* need a valid SW entry point to build a Tx tree */
if (!hw->sw_entry_point_layer) {
ice_debug(hw, ICE_DBG_SCHED, "invalid sw entry point\n");
- status = ICE_ERR_CFG;
+ status = -EIO;
goto err_unroll_sched;
}
INIT_LIST_HEAD(&hw->agg_list);
@@ -1026,7 +1025,7 @@ enum ice_status ice_init_hw(struct ice_hw *hw)
mac_buf_len = 2 * sizeof(struct ice_aqc_manage_mac_read_resp);
if (!mac_buf) {
- status = ICE_ERR_NO_MEMORY;
+ status = -ENOMEM;
goto err_unroll_fltr_mgmt_struct;
}
@@ -1096,7 +1095,7 @@ void ice_deinit_hw(struct ice_hw *hw)
* ice_check_reset - Check to see if a global reset is complete
* @hw: pointer to the hardware structure
*/
-enum ice_status ice_check_reset(struct ice_hw *hw)
+int ice_check_reset(struct ice_hw *hw)
{
u32 cnt, reg = 0, grst_timeout, uld_mask;
@@ -1116,7 +1115,7 @@ enum ice_status ice_check_reset(struct ice_hw *hw)
if (cnt == grst_timeout) {
ice_debug(hw, ICE_DBG_INIT, "Global reset polling failed to complete.\n");
- return ICE_ERR_RESET_FAILED;
+ return -EIO;
}
#define ICE_RESET_DONE_MASK (GLNVM_ULD_PCIER_DONE_M |\
@@ -1143,7 +1142,7 @@ enum ice_status ice_check_reset(struct ice_hw *hw)
if (cnt == ICE_PF_RESET_WAIT_COUNT) {
ice_debug(hw, ICE_DBG_INIT, "Wait for Reset Done timed out. GLNVM_ULD = 0x%x\n",
reg);
- return ICE_ERR_RESET_FAILED;
+ return -EIO;
}
return 0;
@@ -1156,7 +1155,7 @@ enum ice_status ice_check_reset(struct ice_hw *hw)
* If a global reset has been triggered, this function checks
* for its completion and then issues the PF reset
*/
-static enum ice_status ice_pf_reset(struct ice_hw *hw)
+static int ice_pf_reset(struct ice_hw *hw)
{
u32 cnt, reg;
@@ -1169,7 +1168,7 @@ static enum ice_status ice_pf_reset(struct ice_hw *hw)
(rd32(hw, GLNVM_ULD) & ICE_RESET_DONE_MASK) ^ ICE_RESET_DONE_MASK) {
/* poll on global reset currently in progress until done */
if (ice_check_reset(hw))
- return ICE_ERR_RESET_FAILED;
+ return -EIO;
return 0;
}
@@ -1194,7 +1193,7 @@ static enum ice_status ice_pf_reset(struct ice_hw *hw)
if (cnt == ICE_PF_RESET_WAIT_COUNT) {
ice_debug(hw, ICE_DBG_INIT, "PF reset polling failed to complete.\n");
- return ICE_ERR_RESET_FAILED;
+ return -EIO;
}
return 0;
@@ -1212,7 +1211,7 @@ static enum ice_status ice_pf_reset(struct ice_hw *hw)
* This has to be cleared using ice_clear_pxe_mode again, once the AQ
* interface has been restored in the rebuild flow.
*/
-enum ice_status ice_reset(struct ice_hw *hw, enum ice_reset_req req)
+int ice_reset(struct ice_hw *hw, enum ice_reset_req req)
{
u32 val = 0;
@@ -1228,7 +1227,7 @@ enum ice_status ice_reset(struct ice_hw *hw, enum ice_reset_req req)
val = GLGEN_RTRIG_GLOBR_M;
break;
default:
- return ICE_ERR_PARAM;
+ return -EINVAL;
}
val |= rd32(hw, GLGEN_RTRIG);
@@ -1247,16 +1246,16 @@ enum ice_status ice_reset(struct ice_hw *hw, enum ice_reset_req req)
*
* Copies rxq context from dense structure to HW register space
*/
-static enum ice_status
+static int
ice_copy_rxq_ctx_to_hw(struct ice_hw *hw, u8 *ice_rxq_ctx, u32 rxq_index)
{
u8 i;
if (!ice_rxq_ctx)
- return ICE_ERR_BAD_PTR;
+ return -EINVAL;
if (rxq_index > QRX_CTRL_MAX_INDEX)
- return ICE_ERR_PARAM;
+ return -EINVAL;
/* Copy each dword separately to HW */
for (i = 0; i < ICE_RXQ_CTX_SIZE_DWORDS; i++) {
@@ -1306,14 +1305,14 @@ static const struct ice_ctx_ele ice_rlan_ctx_info[] = {
* it to HW register space and enables the hardware to prefetch descriptors
* instead of only fetching them on demand
*/
-enum ice_status
+int
ice_write_rxq_ctx(struct ice_hw *hw, struct ice_rlan_ctx *rlan_ctx,
u32 rxq_index)
{
u8 ctx_buf[ICE_RXQ_CTX_SZ] = { 0 };
if (!rlan_ctx)
- return ICE_ERR_BAD_PTR;
+ return -EINVAL;
rlan_ctx->prefena = 1;
@@ -1369,9 +1368,8 @@ static int
ice_sbq_send_cmd(struct ice_hw *hw, struct ice_sbq_cmd_desc *desc,
void *buf, u16 buf_size, struct ice_sq_cd *cd)
{
- return ice_status_to_errno(ice_sq_send_cmd(hw, ice_get_sbq(hw),
- (struct ice_aq_desc *)desc,
- buf, buf_size, cd));
+ return ice_sq_send_cmd(hw, ice_get_sbq(hw),
+ (struct ice_aq_desc *)desc, buf, buf_size, cd);
}
/**
@@ -1453,17 +1451,17 @@ static bool ice_should_retry_sq_send_cmd(u16 opcode)
* Retry sending the FW Admin Queue command, multiple times, to the FW Admin
* Queue if the EBUSY AQ error is returned.
*/
-static enum ice_status
+static int
ice_sq_send_cmd_retry(struct ice_hw *hw, struct ice_ctl_q_info *cq,
struct ice_aq_desc *desc, void *buf, u16 buf_size,
struct ice_sq_cd *cd)
{
struct ice_aq_desc desc_cpy;
- enum ice_status status;
bool is_cmd_for_retry;
u8 *buf_cpy = NULL;
u8 idx = 0;
u16 opcode;
+ int status;
opcode = le16_to_cpu(desc->opcode);
is_cmd_for_retry = ice_should_retry_sq_send_cmd(opcode);
@@ -1473,7 +1471,7 @@ ice_sq_send_cmd_retry(struct ice_hw *hw, struct ice_ctl_q_info *cq,
if (buf) {
buf_cpy = kzalloc(buf_size, GFP_KERNEL);
if (!buf_cpy)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
}
memcpy(&desc_cpy, desc, sizeof(desc_cpy));
@@ -1510,13 +1508,13 @@ ice_sq_send_cmd_retry(struct ice_hw *hw, struct ice_ctl_q_info *cq,
*
* Helper function to send FW Admin Queue commands to the FW Admin Queue.
*/
-enum ice_status
+int
ice_aq_send_cmd(struct ice_hw *hw, struct ice_aq_desc *desc, void *buf,
u16 buf_size, struct ice_sq_cd *cd)
{
struct ice_aqc_req_res *cmd = &desc->params.res_owner;
bool lock_acquired = false;
- enum ice_status status;
+ int status;
/* When a package download is in process (i.e. when the firmware's
* Global Configuration Lock resource is held), only the Download
@@ -1555,11 +1553,11 @@ ice_aq_send_cmd(struct ice_hw *hw, struct ice_aq_desc *desc, void *buf,
*
* Get the firmware version (0x0001) from the admin queue commands
*/
-enum ice_status ice_aq_get_fw_ver(struct ice_hw *hw, struct ice_sq_cd *cd)
+int ice_aq_get_fw_ver(struct ice_hw *hw, struct ice_sq_cd *cd)
{
struct ice_aqc_get_ver *resp;
struct ice_aq_desc desc;
- enum ice_status status;
+ int status;
resp = &desc.params.get_ver;
@@ -1590,7 +1588,7 @@ enum ice_status ice_aq_get_fw_ver(struct ice_hw *hw, struct ice_sq_cd *cd)
*
* Send the driver version (0x0002) to the firmware
*/
-enum ice_status
+int
ice_aq_send_driver_ver(struct ice_hw *hw, struct ice_driver_ver *dv,
struct ice_sq_cd *cd)
{
@@ -1601,7 +1599,7 @@ ice_aq_send_driver_ver(struct ice_hw *hw, struct ice_driver_ver *dv,
cmd = &desc.params.driver_ver;
if (!dv)
- return ICE_ERR_PARAM;
+ return -EINVAL;
ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_driver_ver);
@@ -1627,7 +1625,7 @@ ice_aq_send_driver_ver(struct ice_hw *hw, struct ice_driver_ver *dv,
* Tell the Firmware that we're shutting down the AdminQ and whether
* or not the driver is unloading as well (0x0003).
*/
-enum ice_status ice_aq_q_shutdown(struct ice_hw *hw, bool unloading)
+int ice_aq_q_shutdown(struct ice_hw *hw, bool unloading)
{
struct ice_aqc_q_shutdown *cmd;
struct ice_aq_desc desc;
@@ -1654,12 +1652,12 @@ enum ice_status ice_aq_q_shutdown(struct ice_hw *hw, bool unloading)
* Requests common resource using the admin queue commands (0x0008).
* When attempting to acquire the Global Config Lock, the driver can
* learn of three states:
- * 1) ICE_SUCCESS - acquired lock, and can perform download package
- * 2) ICE_ERR_AQ_ERROR - did not get lock, driver should fail to load
- * 3) ICE_ERR_AQ_NO_WORK - did not get lock, but another driver has
- * successfully downloaded the package; the driver does
- * not have to download the package and can continue
- * loading
+ * 1) 0 - acquired lock, and can perform download package
+ * 2) -EIO - did not get lock, driver should fail to load
+ * 3) -EALREADY - did not get lock, but another driver has
+ * successfully downloaded the package; the driver does
+ * not have to download the package and can continue
+ * loading
*
* Note that if the caller is in an acquire lock, perform action, release lock
* phase of operation, it is possible that the FW may detect a timeout and issue
@@ -1668,14 +1666,14 @@ enum ice_status ice_aq_q_shutdown(struct ice_hw *hw, bool unloading)
* will likely get an error propagated back to it indicating the Download
* Package, Update Package or the Release Resource AQ commands timed out.
*/
-static enum ice_status
+static int
ice_aq_req_res(struct ice_hw *hw, enum ice_aq_res_ids res,
enum ice_aq_res_access_type access, u8 sdp_number, u32 *timeout,
struct ice_sq_cd *cd)
{
struct ice_aqc_req_res *cmd_resp;
struct ice_aq_desc desc;
- enum ice_status status;
+ int status;
cmd_resp = &desc.params.res_owner;
@@ -1707,15 +1705,15 @@ ice_aq_req_res(struct ice_hw *hw, enum ice_aq_res_ids res,
} else if (le16_to_cpu(cmd_resp->status) ==
ICE_AQ_RES_GLBL_IN_PROG) {
*timeout = le32_to_cpu(cmd_resp->timeout);
- return ICE_ERR_AQ_ERROR;
+ return -EIO;
} else if (le16_to_cpu(cmd_resp->status) ==
ICE_AQ_RES_GLBL_DONE) {
- return ICE_ERR_AQ_NO_WORK;
+ return -EALREADY;
}
/* invalid FW response, force a timeout immediately */
*timeout = 0;
- return ICE_ERR_AQ_ERROR;
+ return -EIO;
}
/* If the resource is held by some other driver, the command completes
@@ -1737,7 +1735,7 @@ ice_aq_req_res(struct ice_hw *hw, enum ice_aq_res_ids res,
*
* release common resource using the admin queue commands (0x0009)
*/
-static enum ice_status
+static int
ice_aq_release_res(struct ice_hw *hw, enum ice_aq_res_ids res, u8 sdp_number,
struct ice_sq_cd *cd)
{
@@ -1763,23 +1761,23 @@ ice_aq_release_res(struct ice_hw *hw, enum ice_aq_res_ids res, u8 sdp_number,
*
* This function will attempt to acquire the ownership of a resource.
*/
-enum ice_status
+int
ice_acquire_res(struct ice_hw *hw, enum ice_aq_res_ids res,
enum ice_aq_res_access_type access, u32 timeout)
{
#define ICE_RES_POLLING_DELAY_MS 10
u32 delay = ICE_RES_POLLING_DELAY_MS;
u32 time_left = timeout;
- enum ice_status status;
+ int status;
status = ice_aq_req_res(hw, res, access, 0, &time_left, NULL);
- /* A return code of ICE_ERR_AQ_NO_WORK means that another driver has
+ /* A return code of -EALREADY means that another driver has
* previously acquired the resource and performed any necessary updates;
* in this case the caller does not obtain the resource and has no
* further work to do.
*/
- if (status == ICE_ERR_AQ_NO_WORK)
+ if (status == -EALREADY)
goto ice_acquire_res_exit;
if (status)
@@ -1792,7 +1790,7 @@ ice_acquire_res(struct ice_hw *hw, enum ice_aq_res_ids res,
timeout = (timeout > delay) ? timeout - delay : 0;
status = ice_aq_req_res(hw, res, access, 0, &time_left, NULL);
- if (status == ICE_ERR_AQ_NO_WORK)
+ if (status == -EALREADY)
/* lock free, but no work to do */
break;
@@ -1800,15 +1798,15 @@ ice_acquire_res(struct ice_hw *hw, enum ice_aq_res_ids res,
/* lock acquired */
break;
}
- if (status && status != ICE_ERR_AQ_NO_WORK)
+ if (status && status != -EALREADY)
ice_debug(hw, ICE_DBG_RES, "resource acquire timed out.\n");
ice_acquire_res_exit:
- if (status == ICE_ERR_AQ_NO_WORK) {
+ if (status == -EALREADY) {
if (access == ICE_RES_WRITE)
ice_debug(hw, ICE_DBG_RES, "resource indicates no work to do.\n");
else
- ice_debug(hw, ICE_DBG_RES, "Warning: ICE_ERR_AQ_NO_WORK not expected\n");
+ ice_debug(hw, ICE_DBG_RES, "Warning: -EALREADY not expected\n");
}
return status;
}
@@ -1822,16 +1820,15 @@ ice_acquire_res(struct ice_hw *hw, enum ice_aq_res_ids res,
*/
void ice_release_res(struct ice_hw *hw, enum ice_aq_res_ids res)
{
- enum ice_status status;
u32 total_delay = 0;
+ int status;
status = ice_aq_release_res(hw, res, 0, NULL);
/* there are some rare cases when trying to release the resource
* results in an admin queue timeout, so handle them correctly
*/
- while ((status == ICE_ERR_AQ_TIMEOUT) &&
- (total_delay < hw->adminq.sq_cmd_timeout)) {
+ while ((status == -EIO) && (total_delay < hw->adminq.sq_cmd_timeout)) {
mdelay(1);
status = ice_aq_release_res(hw, res, 0, NULL);
total_delay++;
@@ -1849,7 +1846,7 @@ void ice_release_res(struct ice_hw *hw, enum ice_aq_res_ids res)
*
* Helper function to allocate/free resources using the admin queue commands
*/
-enum ice_status
+int
ice_aq_alloc_free_res(struct ice_hw *hw, u16 num_entries,
struct ice_aqc_alloc_free_res_elem *buf, u16 buf_size,
enum ice_adminq_opc opc, struct ice_sq_cd *cd)
@@ -1860,10 +1857,10 @@ ice_aq_alloc_free_res(struct ice_hw *hw, u16 num_entries,
cmd = &desc.params.sw_res_ctrl;
if (!buf)
- return ICE_ERR_PARAM;
+ return -EINVAL;
if (buf_size < flex_array_size(buf, elem, num_entries))
- return ICE_ERR_PARAM;
+ return -EINVAL;
ice_fill_dflt_direct_cmd_desc(&desc, opc);
@@ -1882,17 +1879,17 @@ ice_aq_alloc_free_res(struct ice_hw *hw, u16 num_entries,
* @btm: allocate from bottom
* @res: pointer to array that will receive the resources
*/
-enum ice_status
+int
ice_alloc_hw_res(struct ice_hw *hw, u16 type, u16 num, bool btm, u16 *res)
{
struct ice_aqc_alloc_free_res_elem *buf;
- enum ice_status status;
u16 buf_len;
+ int status;
buf_len = struct_size(buf, elem, num);
buf = kzalloc(buf_len, GFP_KERNEL);
if (!buf)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
/* Prepare buffer to allocate resource. */
buf->num_elems = cpu_to_le16(num);
@@ -1920,16 +1917,16 @@ ice_alloc_hw_res(struct ice_hw *hw, u16 type, u16 num, bool btm, u16 *res)
* @num: number of resources
* @res: pointer to array that contains the resources to free
*/
-enum ice_status ice_free_hw_res(struct ice_hw *hw, u16 type, u16 num, u16 *res)
+int ice_free_hw_res(struct ice_hw *hw, u16 type, u16 num, u16 *res)
{
struct ice_aqc_alloc_free_res_elem *buf;
- enum ice_status status;
u16 buf_len;
+ int status;
buf_len = struct_size(buf, elem, num);
buf = kzalloc(buf_len, GFP_KERNEL);
if (!buf)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
/* Prepare buffer to free resource. */
buf->num_elems = cpu_to_le16(num);
@@ -2071,6 +2068,18 @@ ice_parse_common_caps(struct ice_hw *hw, struct ice_hw_common_caps *caps,
ice_debug(hw, ICE_DBG_INIT, "%s: max_mtu = %d\n",
prefix, caps->max_mtu);
break;
+ case ICE_AQC_CAPS_PCIE_RESET_AVOIDANCE:
+ caps->pcie_reset_avoidance = (number > 0);
+ ice_debug(hw, ICE_DBG_INIT,
+ "%s: pcie_reset_avoidance = %d\n", prefix,
+ caps->pcie_reset_avoidance);
+ break;
+ case ICE_AQC_CAPS_POST_UPDATE_RESET_RESTRICT:
+ caps->reset_restrict_support = (number == 1);
+ ice_debug(hw, ICE_DBG_INIT,
+ "%s: reset_restrict_support = %d\n", prefix,
+ caps->reset_restrict_support);
+ break;
default:
/* Not one of the recognized common capabilities */
found = false;
@@ -2180,6 +2189,18 @@ ice_parse_1588_func_caps(struct ice_hw *hw, struct ice_hw_func_caps *func_p,
info->clk_freq = (number & ICE_TS_CLK_FREQ_M) >> ICE_TS_CLK_FREQ_S;
info->clk_src = ((number & ICE_TS_CLK_SRC_M) != 0);
+ if (info->clk_freq < NUM_ICE_TIME_REF_FREQ) {
+ info->time_ref = (enum ice_time_ref_freq)info->clk_freq;
+ } else {
+ /* Unknown clock frequency, so assume a (probably incorrect)
+ * default to avoid out-of-bounds look ups of frequency
+ * related information.
+ */
+ ice_debug(hw, ICE_DBG_INIT, "1588 func caps: unknown clock frequency %u\n",
+ info->clk_freq);
+ info->time_ref = ICE_TIME_REF_FREQ_25_000;
+ }
+
ice_debug(hw, ICE_DBG_INIT, "func caps: ieee_1588 = %u\n",
func_p->common_cap.ieee_1588);
ice_debug(hw, ICE_DBG_INIT, "func caps: src_tmr_owned = %u\n",
@@ -2486,19 +2507,19 @@ ice_parse_dev_caps(struct ice_hw *hw, struct ice_hw_dev_caps *dev_p,
* buffer size be set to ICE_AQ_MAX_BUF_LEN (the largest possible buffer that
* firmware could return) to avoid this.
*/
-enum ice_status
+int
ice_aq_list_caps(struct ice_hw *hw, void *buf, u16 buf_size, u32 *cap_count,
enum ice_adminq_opc opc, struct ice_sq_cd *cd)
{
struct ice_aqc_list_caps *cmd;
struct ice_aq_desc desc;
- enum ice_status status;
+ int status;
cmd = &desc.params.get_cap;
if (opc != ice_aqc_opc_list_func_caps &&
opc != ice_aqc_opc_list_dev_caps)
- return ICE_ERR_PARAM;
+ return -EINVAL;
ice_fill_dflt_direct_cmd_desc(&desc, opc);
status = ice_aq_send_cmd(hw, &desc, buf, buf_size, cd);
@@ -2517,16 +2538,16 @@ ice_aq_list_caps(struct ice_hw *hw, void *buf, u16 buf_size, u32 *cap_count,
* Read the device capabilities and extract them into the dev_caps structure
* for later use.
*/
-enum ice_status
+int
ice_discover_dev_caps(struct ice_hw *hw, struct ice_hw_dev_caps *dev_caps)
{
- enum ice_status status;
u32 cap_count = 0;
void *cbuf;
+ int status;
cbuf = kzalloc(ICE_AQ_MAX_BUF_LEN, GFP_KERNEL);
if (!cbuf)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
/* Although the driver doesn't know the number of capabilities the
* device will return, we can simply send a 4KB buffer, the maximum
@@ -2551,16 +2572,16 @@ ice_discover_dev_caps(struct ice_hw *hw, struct ice_hw_dev_caps *dev_caps)
* Read the function capabilities and extract them into the func_caps structure
* for later use.
*/
-static enum ice_status
+static int
ice_discover_func_caps(struct ice_hw *hw, struct ice_hw_func_caps *func_caps)
{
- enum ice_status status;
u32 cap_count = 0;
void *cbuf;
+ int status;
cbuf = kzalloc(ICE_AQ_MAX_BUF_LEN, GFP_KERNEL);
if (!cbuf)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
/* Although the driver doesn't know the number of capabilities the
* device will return, we can simply send a 4KB buffer, the maximum
@@ -2650,9 +2671,9 @@ void ice_set_safe_mode_caps(struct ice_hw *hw)
* ice_get_caps - get info about the HW
* @hw: pointer to the hardware structure
*/
-enum ice_status ice_get_caps(struct ice_hw *hw)
+int ice_get_caps(struct ice_hw *hw)
{
- enum ice_status status;
+ int status;
status = ice_discover_dev_caps(hw, &hw->dev_caps);
if (status)
@@ -2670,7 +2691,7 @@ enum ice_status ice_get_caps(struct ice_hw *hw)
*
* This function is used to write MAC address to the NVM (0x0108).
*/
-enum ice_status
+int
ice_aq_manage_mac_write(struct ice_hw *hw, const u8 *mac_addr, u8 flags,
struct ice_sq_cd *cd)
{
@@ -2692,7 +2713,7 @@ ice_aq_manage_mac_write(struct ice_hw *hw, const u8 *mac_addr, u8 flags,
*
* Tell the firmware that the driver is taking over from PXE (0x0110).
*/
-static enum ice_status ice_aq_clear_pxe_mode(struct ice_hw *hw)
+static int ice_aq_clear_pxe_mode(struct ice_hw *hw)
{
struct ice_aq_desc desc;
@@ -2903,15 +2924,15 @@ ice_update_phy_type(u64 *phy_type_low, u64 *phy_type_high,
* mode as the PF may not have the privilege to set some of the PHY Config
* parameters. This status will be indicated by the command response (0x0601).
*/
-enum ice_status
+int
ice_aq_set_phy_cfg(struct ice_hw *hw, struct ice_port_info *pi,
struct ice_aqc_set_phy_cfg_data *cfg, struct ice_sq_cd *cd)
{
struct ice_aq_desc desc;
- enum ice_status status;
+ int status;
if (!cfg)
- return ICE_ERR_PARAM;
+ return -EINVAL;
/* Ensure that only valid bits of cfg->caps can be turned on. */
if (cfg->caps & ~ICE_AQ_PHY_ENA_VALID_MASK) {
@@ -2952,13 +2973,13 @@ ice_aq_set_phy_cfg(struct ice_hw *hw, struct ice_port_info *pi,
* ice_update_link_info - update status of the HW network link
* @pi: port info structure of the interested logical port
*/
-enum ice_status ice_update_link_info(struct ice_port_info *pi)
+int ice_update_link_info(struct ice_port_info *pi)
{
struct ice_link_status *li;
- enum ice_status status;
+ int status;
if (!pi)
- return ICE_ERR_PARAM;
+ return -EINVAL;
li = &pi->phy.link_info;
@@ -2974,7 +2995,7 @@ enum ice_status ice_update_link_info(struct ice_port_info *pi)
pcaps = devm_kzalloc(ice_hw_to_dev(hw), sizeof(*pcaps),
GFP_KERNEL);
if (!pcaps)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
status = ice_aq_get_phy_caps(pi, false, ICE_AQC_REPORT_TOPO_CAP_MEDIA,
pcaps, NULL);
@@ -3070,7 +3091,7 @@ enum ice_fec_mode ice_caps_to_fec_mode(u8 caps, u8 fec_options)
* @cfg: PHY configuration data to set FC mode
* @req_mode: FC mode to configure
*/
-enum ice_status
+int
ice_cfg_phy_fc(struct ice_port_info *pi, struct ice_aqc_set_phy_cfg_data *cfg,
enum ice_fc_mode req_mode)
{
@@ -3078,7 +3099,7 @@ ice_cfg_phy_fc(struct ice_port_info *pi, struct ice_aqc_set_phy_cfg_data *cfg,
u8 pause_mask = 0x0;
if (!pi || !cfg)
- return ICE_ERR_BAD_PTR;
+ return -EINVAL;
switch (req_mode) {
case ICE_FC_FULL:
@@ -3117,23 +3138,23 @@ ice_cfg_phy_fc(struct ice_port_info *pi, struct ice_aqc_set_phy_cfg_data *cfg,
*
* Set the requested flow control mode.
*/
-enum ice_status
+int
ice_set_fc(struct ice_port_info *pi, u8 *aq_failures, bool ena_auto_link_update)
{
struct ice_aqc_set_phy_cfg_data cfg = { 0 };
struct ice_aqc_get_phy_caps_data *pcaps;
- enum ice_status status;
struct ice_hw *hw;
+ int status;
if (!pi || !aq_failures)
- return ICE_ERR_BAD_PTR;
+ return -EINVAL;
*aq_failures = 0;
hw = pi->hw;
pcaps = devm_kzalloc(ice_hw_to_dev(hw), sizeof(*pcaps), GFP_KERNEL);
if (!pcaps)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
/* Get the current PHY config */
status = ice_aq_get_phy_caps(pi, false, ICE_AQC_REPORT_ACTIVE_CFG,
@@ -3258,22 +3279,22 @@ ice_copy_phy_caps_to_cfg(struct ice_port_info *pi,
* @cfg: PHY configuration data to set FEC mode
* @fec: FEC mode to configure
*/
-enum ice_status
+int
ice_cfg_phy_fec(struct ice_port_info *pi, struct ice_aqc_set_phy_cfg_data *cfg,
enum ice_fec_mode fec)
{
struct ice_aqc_get_phy_caps_data *pcaps;
- enum ice_status status;
struct ice_hw *hw;
+ int status;
if (!pi || !cfg)
- return ICE_ERR_BAD_PTR;
+ return -EINVAL;
hw = pi->hw;
pcaps = kzalloc(sizeof(*pcaps), GFP_KERNEL);
if (!pcaps)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
status = ice_aq_get_phy_caps(pi, false,
(ice_fw_supports_report_dflt_cfg(hw) ?
@@ -3313,7 +3334,7 @@ ice_cfg_phy_fec(struct ice_port_info *pi, struct ice_aqc_set_phy_cfg_data *cfg,
cfg->link_fec_opt |= pcaps->link_fec_options;
break;
default:
- status = ICE_ERR_PARAM;
+ status = -EINVAL;
break;
}
@@ -3344,13 +3365,13 @@ ice_cfg_phy_fec(struct ice_port_info *pi, struct ice_aqc_set_phy_cfg_data *cfg,
* The variable link_up is invalid if status is non zero. As a
* result of this call, link status reporting becomes enabled
*/
-enum ice_status ice_get_link_status(struct ice_port_info *pi, bool *link_up)
+int ice_get_link_status(struct ice_port_info *pi, bool *link_up)
{
struct ice_phy_info *phy_info;
- enum ice_status status = 0;
+ int status = 0;
if (!pi || !link_up)
- return ICE_ERR_PARAM;
+ return -EINVAL;
phy_info = &pi->phy;
@@ -3375,7 +3396,7 @@ enum ice_status ice_get_link_status(struct ice_port_info *pi, bool *link_up)
*
* Sets up the link and restarts the Auto-Negotiation over the link.
*/
-enum ice_status
+int
ice_aq_set_link_restart_an(struct ice_port_info *pi, bool ena_link,
struct ice_sq_cd *cd)
{
@@ -3405,7 +3426,7 @@ ice_aq_set_link_restart_an(struct ice_port_info *pi, bool ena_link,
*
* Set event mask (0x0613)
*/
-enum ice_status
+int
ice_aq_set_event_mask(struct ice_hw *hw, u8 port_num, u16 mask,
struct ice_sq_cd *cd)
{
@@ -3430,7 +3451,7 @@ ice_aq_set_event_mask(struct ice_hw *hw, u8 port_num, u16 mask,
*
* Enable/disable loopback on a given port
*/
-enum ice_status
+int
ice_aq_set_mac_loopback(struct ice_hw *hw, bool ena_lpbk, struct ice_sq_cd *cd)
{
struct ice_aqc_set_mac_lb *cmd;
@@ -3453,7 +3474,7 @@ ice_aq_set_mac_loopback(struct ice_hw *hw, bool ena_lpbk, struct ice_sq_cd *cd)
*
* Set LED value for the given port (0x06e9)
*/
-enum ice_status
+int
ice_aq_set_port_id_led(struct ice_port_info *pi, bool is_orig_mode,
struct ice_sq_cd *cd)
{
@@ -3488,17 +3509,17 @@ ice_aq_set_port_id_led(struct ice_port_info *pi, bool is_orig_mode,
*
* Read/Write SFF EEPROM (0x06EE)
*/
-enum ice_status
+int
ice_aq_sff_eeprom(struct ice_hw *hw, u16 lport, u8 bus_addr,
u16 mem_addr, u8 page, u8 set_page, u8 *data, u8 length,
bool write, struct ice_sq_cd *cd)
{
struct ice_aqc_sff_eeprom *cmd;
struct ice_aq_desc desc;
- enum ice_status status;
+ int status;
if (!data || (mem_addr & 0xff00))
- return ICE_ERR_PARAM;
+ return -EINVAL;
ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_sff_eeprom);
cmd = &desc.params.read_write_sff_param;
@@ -3527,23 +3548,23 @@ ice_aq_sff_eeprom(struct ice_hw *hw, u16 lport, u8 bus_addr,
*
* Internal function to get (0x0B05) or set (0x0B03) RSS look up table
*/
-static enum ice_status
+static int
__ice_aq_get_set_rss_lut(struct ice_hw *hw, struct ice_aq_get_set_rss_lut_params *params, bool set)
{
u16 flags = 0, vsi_id, lut_type, lut_size, glob_lut_idx, vsi_handle;
struct ice_aqc_get_set_rss_lut *cmd_resp;
struct ice_aq_desc desc;
- enum ice_status status;
+ int status;
u8 *lut;
if (!params)
- return ICE_ERR_PARAM;
+ return -EINVAL;
vsi_handle = params->vsi_handle;
lut = params->lut;
if (!ice_is_vsi_valid(hw, vsi_handle) || !lut)
- return ICE_ERR_PARAM;
+ return -EINVAL;
lut_size = params->lut_size;
lut_type = params->lut_type;
@@ -3572,7 +3593,7 @@ __ice_aq_get_set_rss_lut(struct ice_hw *hw, struct ice_aq_get_set_rss_lut_params
ICE_AQC_GSET_RSS_LUT_TABLE_TYPE_M);
break;
default:
- status = ICE_ERR_PARAM;
+ status = -EINVAL;
goto ice_aq_get_set_rss_lut_exit;
}
@@ -3607,7 +3628,7 @@ __ice_aq_get_set_rss_lut(struct ice_hw *hw, struct ice_aq_get_set_rss_lut_params
}
fallthrough;
default:
- status = ICE_ERR_PARAM;
+ status = -EINVAL;
goto ice_aq_get_set_rss_lut_exit;
}
@@ -3626,7 +3647,7 @@ __ice_aq_get_set_rss_lut(struct ice_hw *hw, struct ice_aq_get_set_rss_lut_params
*
* get the RSS lookup table, PF or VSI type
*/
-enum ice_status
+int
ice_aq_get_rss_lut(struct ice_hw *hw, struct ice_aq_get_set_rss_lut_params *get_params)
{
return __ice_aq_get_set_rss_lut(hw, get_params, false);
@@ -3639,7 +3660,7 @@ ice_aq_get_rss_lut(struct ice_hw *hw, struct ice_aq_get_set_rss_lut_params *get_
*
* set the RSS lookup table, PF or VSI type
*/
-enum ice_status
+int
ice_aq_set_rss_lut(struct ice_hw *hw, struct ice_aq_get_set_rss_lut_params *set_params)
{
return __ice_aq_get_set_rss_lut(hw, set_params, true);
@@ -3654,10 +3675,9 @@ ice_aq_set_rss_lut(struct ice_hw *hw, struct ice_aq_get_set_rss_lut_params *set_
*
* get (0x0B04) or set (0x0B02) the RSS key per VSI
*/
-static enum
-ice_status __ice_aq_get_set_rss_key(struct ice_hw *hw, u16 vsi_id,
- struct ice_aqc_get_set_rss_keys *key,
- bool set)
+static int
+__ice_aq_get_set_rss_key(struct ice_hw *hw, u16 vsi_id,
+ struct ice_aqc_get_set_rss_keys *key, bool set)
{
struct ice_aqc_get_set_rss_key *cmd_resp;
u16 key_size = sizeof(*key);
@@ -3688,12 +3708,12 @@ ice_status __ice_aq_get_set_rss_key(struct ice_hw *hw, u16 vsi_id,
*
* get the RSS key per VSI
*/
-enum ice_status
+int
ice_aq_get_rss_key(struct ice_hw *hw, u16 vsi_handle,
struct ice_aqc_get_set_rss_keys *key)
{
if (!ice_is_vsi_valid(hw, vsi_handle) || !key)
- return ICE_ERR_PARAM;
+ return -EINVAL;
return __ice_aq_get_set_rss_key(hw, ice_get_hw_vsi_num(hw, vsi_handle),
key, false);
@@ -3707,12 +3727,12 @@ ice_aq_get_rss_key(struct ice_hw *hw, u16 vsi_handle,
*
* set the RSS key per VSI
*/
-enum ice_status
+int
ice_aq_set_rss_key(struct ice_hw *hw, u16 vsi_handle,
struct ice_aqc_get_set_rss_keys *keys)
{
if (!ice_is_vsi_valid(hw, vsi_handle) || !keys)
- return ICE_ERR_PARAM;
+ return -EINVAL;
return __ice_aq_get_set_rss_key(hw, ice_get_hw_vsi_num(hw, vsi_handle),
keys, true);
@@ -3739,7 +3759,7 @@ ice_aq_set_rss_key(struct ice_hw *hw, u16 vsi_handle,
* Association of Tx queue to Doorbell queue is not part of Add LAN Tx queue
* flow.
*/
-static enum ice_status
+static int
ice_aq_add_lan_txq(struct ice_hw *hw, u8 num_qgrps,
struct ice_aqc_add_tx_qgrp *qg_list, u16 buf_size,
struct ice_sq_cd *cd)
@@ -3754,10 +3774,10 @@ ice_aq_add_lan_txq(struct ice_hw *hw, u8 num_qgrps,
ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_add_txqs);
if (!qg_list)
- return ICE_ERR_PARAM;
+ return -EINVAL;
if (num_qgrps > ICE_LAN_TXQ_MAX_QGRPS)
- return ICE_ERR_PARAM;
+ return -EINVAL;
for (i = 0, list = qg_list; i < num_qgrps; i++) {
sum_size += struct_size(list, txqs, list->num_txqs);
@@ -3766,7 +3786,7 @@ ice_aq_add_lan_txq(struct ice_hw *hw, u8 num_qgrps,
}
if (buf_size != sum_size)
- return ICE_ERR_PARAM;
+ return -EINVAL;
desc.flags |= cpu_to_le16(ICE_AQ_FLAG_RD);
@@ -3787,7 +3807,7 @@ ice_aq_add_lan_txq(struct ice_hw *hw, u8 num_qgrps,
*
* Disable LAN Tx queue (0x0C31)
*/
-static enum ice_status
+static int
ice_aq_dis_lan_txq(struct ice_hw *hw, u8 num_qgrps,
struct ice_aqc_dis_txq_item *qg_list, u16 buf_size,
enum ice_disq_rst_src rst_src, u16 vmvf_num,
@@ -3796,18 +3816,18 @@ ice_aq_dis_lan_txq(struct ice_hw *hw, u8 num_qgrps,
struct ice_aqc_dis_txq_item *item;
struct ice_aqc_dis_txqs *cmd;
struct ice_aq_desc desc;
- enum ice_status status;
u16 i, sz = 0;
+ int status;
cmd = &desc.params.dis_txqs;
ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_dis_txqs);
/* qg_list can be NULL only in VM/VF reset flow */
if (!qg_list && !rst_src)
- return ICE_ERR_PARAM;
+ return -EINVAL;
if (num_qgrps > ICE_LAN_TXQ_MAX_QGRPS)
- return ICE_ERR_PARAM;
+ return -EINVAL;
cmd->num_entries = num_qgrps;
@@ -3856,7 +3876,7 @@ ice_aq_dis_lan_txq(struct ice_hw *hw, u8 num_qgrps,
}
if (buf_size != sz)
- return ICE_ERR_PARAM;
+ return -EINVAL;
do_aq:
status = ice_aq_send_cmd(hw, &desc, qg_list, buf_size, cd);
@@ -3914,8 +3934,7 @@ ice_aq_add_rdma_qsets(struct ice_hw *hw, u8 num_qset_grps,
cmd->num_qset_grps = num_qset_grps;
- return ice_status_to_errno(ice_aq_send_cmd(hw, &desc, qset_list,
- buf_size, cd));
+ return ice_aq_send_cmd(hw, &desc, qset_list, buf_size, cd);
}
/* End of FW Admin Queue command wrappers */
@@ -4111,7 +4130,7 @@ ice_write_qword(u8 *src_ctx, u8 *dest_ctx, const struct ice_ctx_ele *ce_info)
* @dest_ctx: pointer to memory for the packed structure
* @ce_info: a description of the structure to be transformed
*/
-enum ice_status
+int
ice_set_ctx(struct ice_hw *hw, u8 *src_ctx, u8 *dest_ctx,
const struct ice_ctx_ele *ce_info)
{
@@ -4141,7 +4160,7 @@ ice_set_ctx(struct ice_hw *hw, u8 *src_ctx, u8 *dest_ctx,
ice_write_qword(src_ctx, dest_ctx, &ce_info[f]);
break;
default:
- return ICE_ERR_INVAL_SIZE;
+ return -EINVAL;
}
}
@@ -4185,7 +4204,7 @@ ice_get_lan_q_ctx(struct ice_hw *hw, u16 vsi_handle, u8 tc, u16 q_handle)
*
* This function adds one LAN queue
*/
-enum ice_status
+int
ice_ena_vsi_txq(struct ice_port_info *pi, u16 vsi_handle, u8 tc, u16 q_handle,
u8 num_qgrps, struct ice_aqc_add_tx_qgrp *buf, u16 buf_size,
struct ice_sq_cd *cd)
@@ -4193,19 +4212,19 @@ ice_ena_vsi_txq(struct ice_port_info *pi, u16 vsi_handle, u8 tc, u16 q_handle,
struct ice_aqc_txsched_elem_data node = { 0 };
struct ice_sched_node *parent;
struct ice_q_ctx *q_ctx;
- enum ice_status status;
struct ice_hw *hw;
+ int status;
if (!pi || pi->port_state != ICE_SCHED_PORT_STATE_READY)
- return ICE_ERR_CFG;
+ return -EIO;
if (num_qgrps > 1 || buf->num_txqs > 1)
- return ICE_ERR_MAX_LIMIT;
+ return -ENOSPC;
hw = pi->hw;
if (!ice_is_vsi_valid(hw, vsi_handle))
- return ICE_ERR_PARAM;
+ return -EINVAL;
mutex_lock(&pi->sched_lock);
@@ -4213,7 +4232,7 @@ ice_ena_vsi_txq(struct ice_port_info *pi, u16 vsi_handle, u8 tc, u16 q_handle,
if (!q_ctx) {
ice_debug(hw, ICE_DBG_SCHED, "Enaq: invalid queue handle %d\n",
q_handle);
- status = ICE_ERR_PARAM;
+ status = -EINVAL;
goto ena_txq_exit;
}
@@ -4221,7 +4240,7 @@ ice_ena_vsi_txq(struct ice_port_info *pi, u16 vsi_handle, u8 tc, u16 q_handle,
parent = ice_sched_get_free_qparent(pi, vsi_handle, tc,
ICE_SCHED_NODE_OWNER_LAN);
if (!parent) {
- status = ICE_ERR_PARAM;
+ status = -EINVAL;
goto ena_txq_exit;
}
@@ -4290,20 +4309,20 @@ ice_ena_vsi_txq(struct ice_port_info *pi, u16 vsi_handle, u8 tc, u16 q_handle,
*
* This function removes queues and their corresponding nodes in SW DB
*/
-enum ice_status
+int
ice_dis_vsi_txq(struct ice_port_info *pi, u16 vsi_handle, u8 tc, u8 num_queues,
u16 *q_handles, u16 *q_ids, u32 *q_teids,
enum ice_disq_rst_src rst_src, u16 vmvf_num,
struct ice_sq_cd *cd)
{
- enum ice_status status = ICE_ERR_DOES_NOT_EXIST;
struct ice_aqc_dis_txq_item *qg_list;
struct ice_q_ctx *q_ctx;
+ int status = -ENOENT;
struct ice_hw *hw;
u16 i, buf_size;
if (!pi || pi->port_state != ICE_SCHED_PORT_STATE_READY)
- return ICE_ERR_CFG;
+ return -EIO;
hw = pi->hw;
@@ -4315,13 +4334,13 @@ ice_dis_vsi_txq(struct ice_port_info *pi, u16 vsi_handle, u8 tc, u8 num_queues,
if (rst_src)
return ice_aq_dis_lan_txq(hw, 0, NULL, 0, rst_src,
vmvf_num, NULL);
- return ICE_ERR_CFG;
+ return -EIO;
}
buf_size = struct_size(qg_list, q_id, 1);
qg_list = kzalloc(buf_size, GFP_KERNEL);
if (!qg_list)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
mutex_lock(&pi->sched_lock);
@@ -4368,18 +4387,18 @@ ice_dis_vsi_txq(struct ice_port_info *pi, u16 vsi_handle, u8 tc, u8 num_queues,
*
* This function adds/updates the VSI queues per TC.
*/
-static enum ice_status
+static int
ice_cfg_vsi_qs(struct ice_port_info *pi, u16 vsi_handle, u8 tc_bitmap,
u16 *maxqs, u8 owner)
{
- enum ice_status status = 0;
+ int status = 0;
u8 i;
if (!pi || pi->port_state != ICE_SCHED_PORT_STATE_READY)
- return ICE_ERR_CFG;
+ return -EIO;
if (!ice_is_vsi_valid(pi->hw, vsi_handle))
- return ICE_ERR_PARAM;
+ return -EINVAL;
mutex_lock(&pi->sched_lock);
@@ -4407,7 +4426,7 @@ ice_cfg_vsi_qs(struct ice_port_info *pi, u16 vsi_handle, u8 tc_bitmap,
*
* This function adds/updates the VSI LAN queues per TC.
*/
-enum ice_status
+int
ice_cfg_vsi_lan(struct ice_port_info *pi, u16 vsi_handle, u8 tc_bitmap,
u16 *max_lanqs)
{
@@ -4428,9 +4447,8 @@ int
ice_cfg_vsi_rdma(struct ice_port_info *pi, u16 vsi_handle, u16 tc_bitmap,
u16 *max_rdmaqs)
{
- return ice_status_to_errno(ice_cfg_vsi_qs(pi, vsi_handle, tc_bitmap,
- max_rdmaqs,
- ICE_SCHED_NODE_OWNER_RDMA));
+ return ice_cfg_vsi_qs(pi, vsi_handle, tc_bitmap, max_rdmaqs,
+ ICE_SCHED_NODE_OWNER_RDMA);
}
/**
@@ -4451,7 +4469,6 @@ ice_ena_vsi_rdma_qset(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
struct ice_aqc_txsched_elem_data node = { 0 };
struct ice_aqc_add_rdma_qset_data *buf;
struct ice_sched_node *parent;
- enum ice_status status;
struct ice_hw *hw;
u16 i, buf_size;
int ret;
@@ -4502,12 +4519,10 @@ ice_ena_vsi_rdma_qset(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
node.data.elem_type = ICE_AQC_ELEM_TYPE_LEAF;
for (i = 0; i < num_qsets; i++) {
node.node_teid = buf->rdma_qsets[i].qset_teid;
- status = ice_sched_add_node(pi, hw->num_tx_sched_layers - 1,
- &node);
- if (status) {
- ret = ice_status_to_errno(status);
+ ret = ice_sched_add_node(pi, hw->num_tx_sched_layers - 1,
+ &node);
+ if (ret)
break;
- }
qset_teid[i] = le32_to_cpu(node.node_teid);
}
rdma_error_exit:
@@ -4528,8 +4543,8 @@ ice_dis_vsi_rdma_qset(struct ice_port_info *pi, u16 count, u32 *qset_teid,
u16 *q_id)
{
struct ice_aqc_dis_txq_item *qg_list;
- enum ice_status status = 0;
struct ice_hw *hw;
+ int status = 0;
u16 qg_size;
int i;
@@ -4568,7 +4583,7 @@ ice_dis_vsi_rdma_qset(struct ice_port_info *pi, u16 count, u32 *qset_teid,
mutex_unlock(&pi->sched_lock);
kfree(qg_list);
- return ice_status_to_errno(status);
+ return status;
}
/**
@@ -4577,7 +4592,7 @@ ice_dis_vsi_rdma_qset(struct ice_port_info *pi, u16 count, u32 *qset_teid,
*
* Initializes required config data for VSI, FD, ACL, and RSS before replay.
*/
-static enum ice_status ice_replay_pre_init(struct ice_hw *hw)
+static int ice_replay_pre_init(struct ice_hw *hw)
{
struct ice_switch_info *sw = hw->switch_info;
u8 i;
@@ -4588,7 +4603,7 @@ static enum ice_status ice_replay_pre_init(struct ice_hw *hw)
* will allow adding rules entries back to filt_rules list,
* which is operational list.
*/
- for (i = 0; i < ICE_SW_LKUP_LAST; i++)
+ for (i = 0; i < ICE_MAX_NUM_RECIPES; i++)
list_replace_init(&sw->recp_list[i].filt_rules,
&sw->recp_list[i].filt_replay_rules);
ice_sched_replay_agg_vsi_preinit(hw);
@@ -4604,12 +4619,12 @@ static enum ice_status ice_replay_pre_init(struct ice_hw *hw)
* Restore all VSI configuration after reset. It is required to call this
* function with main VSI first.
*/
-enum ice_status ice_replay_vsi(struct ice_hw *hw, u16 vsi_handle)
+int ice_replay_vsi(struct ice_hw *hw, u16 vsi_handle)
{
- enum ice_status status;
+ int status;
if (!ice_is_vsi_valid(hw, vsi_handle))
- return ICE_ERR_PARAM;
+ return -EINVAL;
/* Replay pre-initialization if there is any */
if (vsi_handle == ICE_MAIN_VSI_HANDLE) {
@@ -4725,12 +4740,12 @@ ice_stat_update32(struct ice_hw *hw, u32 reg, bool prev_stat_loaded,
*
* This function queries HW element information
*/
-enum ice_status
+int
ice_sched_query_elem(struct ice_hw *hw, u32 node_teid,
struct ice_aqc_txsched_elem_data *buf)
{
u16 buf_size, num_elem_ret = 0;
- enum ice_status status;
+ int status;
buf_size = sizeof(*buf);
memset(buf, 0, buf_size);
@@ -4775,7 +4790,7 @@ ice_aq_set_driver_param(struct ice_hw *hw, enum ice_aqc_driver_params idx,
cmd->param_indx = idx;
cmd->param_val = cpu_to_le32(value);
- return ice_status_to_errno(ice_aq_send_cmd(hw, &desc, NULL, 0, cd));
+ return ice_aq_send_cmd(hw, &desc, NULL, 0, cd);
}
/**
@@ -4796,7 +4811,7 @@ ice_aq_get_driver_param(struct ice_hw *hw, enum ice_aqc_driver_params idx,
{
struct ice_aqc_driver_shared_params *cmd;
struct ice_aq_desc desc;
- enum ice_status status;
+ int status;
if (idx >= ICE_AQC_DRIVER_PARAM_MAX)
return -EIO;
@@ -4810,7 +4825,7 @@ ice_aq_get_driver_param(struct ice_hw *hw, enum ice_aqc_driver_params idx,
status = ice_aq_send_cmd(hw, &desc, NULL, 0, cd);
if (status)
- return ice_status_to_errno(status);
+ return status;
*value = le32_to_cpu(cmd->param_val);
@@ -4840,7 +4855,7 @@ ice_aq_set_gpio(struct ice_hw *hw, u16 gpio_ctrl_handle, u8 pin_idx, bool value,
cmd->gpio_num = pin_idx;
cmd->gpio_val = value ? 1 : 0;
- return ice_status_to_errno(ice_aq_send_cmd(hw, &desc, NULL, 0, cd));
+ return ice_aq_send_cmd(hw, &desc, NULL, 0, cd);
}
/**
@@ -4860,7 +4875,7 @@ ice_aq_get_gpio(struct ice_hw *hw, u16 gpio_ctrl_handle, u8 pin_idx,
{
struct ice_aqc_gpio *cmd;
struct ice_aq_desc desc;
- enum ice_status status;
+ int status;
ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_get_gpio);
cmd = &desc.params.read_write_gpio;
@@ -4869,7 +4884,7 @@ ice_aq_get_gpio(struct ice_hw *hw, u16 gpio_ctrl_handle, u8 pin_idx,
status = ice_aq_send_cmd(hw, &desc, NULL, 0, cd);
if (status)
- return ice_status_to_errno(status);
+ return status;
*value = !!cmd->gpio_val;
return 0;
@@ -4903,13 +4918,13 @@ bool ice_fw_supports_link_override(struct ice_hw *hw)
*
* Gets the link default override for a port
*/
-enum ice_status
+int
ice_get_link_default_override(struct ice_link_default_override_tlv *ldo,
struct ice_port_info *pi)
{
u16 i, tlv, tlv_len, tlv_start, buf, offset;
struct ice_hw *hw = pi->hw;
- enum ice_status status;
+ int status;
status = ice_get_pfa_module_tlv(hw, &tlv, &tlv_len,
ICE_SR_LINK_DEFAULT_OVERRIDE_PTR);
@@ -4994,7 +5009,7 @@ bool ice_is_phy_caps_an_enabled(struct ice_aqc_get_phy_caps_data *caps)
*
* Set the LLDP MIB. (0x0A08)
*/
-enum ice_status
+int
ice_aq_set_lldp_mib(struct ice_hw *hw, u8 mib_type, void *buf, u16 buf_size,
struct ice_sq_cd *cd)
{
@@ -5004,7 +5019,7 @@ ice_aq_set_lldp_mib(struct ice_hw *hw, u8 mib_type, void *buf, u16 buf_size,
cmd = &desc.params.lldp_set_mib;
if (buf_size == 0 || !buf)
- return ICE_ERR_PARAM;
+ return -EINVAL;
ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_lldp_set_local_mib);
@@ -5044,7 +5059,7 @@ bool ice_fw_supports_lldp_fltr_ctrl(struct ice_hw *hw)
* @vsi_num: absolute HW index for VSI
* @add: boolean for if adding or removing a filter
*/
-enum ice_status
+int
ice_lldp_fltr_add_remove(struct ice_hw *hw, u16 vsi_num, bool add)
{
struct ice_aqc_lldp_filter_ctrl *cmd;
diff --git a/drivers/net/ethernet/intel/ice/ice_common.h b/drivers/net/ethernet/intel/ice/ice_common.h
index 65c1b32..1c57097 100644
--- a/drivers/net/ethernet/intel/ice/ice_common.h
+++ b/drivers/net/ethernet/intel/ice/ice_common.h
@@ -14,108 +14,108 @@
#define ICE_SQ_SEND_DELAY_TIME_MS 10
#define ICE_SQ_SEND_MAX_EXECUTE 3
-enum ice_status ice_init_hw(struct ice_hw *hw);
+int ice_init_hw(struct ice_hw *hw);
void ice_deinit_hw(struct ice_hw *hw);
-enum ice_status ice_check_reset(struct ice_hw *hw);
-enum ice_status ice_reset(struct ice_hw *hw, enum ice_reset_req req);
-enum ice_status ice_create_all_ctrlq(struct ice_hw *hw);
-enum ice_status ice_init_all_ctrlq(struct ice_hw *hw);
+int ice_check_reset(struct ice_hw *hw);
+int ice_reset(struct ice_hw *hw, enum ice_reset_req req);
+int ice_create_all_ctrlq(struct ice_hw *hw);
+int ice_init_all_ctrlq(struct ice_hw *hw);
void ice_shutdown_all_ctrlq(struct ice_hw *hw);
void ice_destroy_all_ctrlq(struct ice_hw *hw);
-enum ice_status
+int
ice_clean_rq_elem(struct ice_hw *hw, struct ice_ctl_q_info *cq,
struct ice_rq_event_info *e, u16 *pending);
-enum ice_status
+int
ice_get_link_status(struct ice_port_info *pi, bool *link_up);
-enum ice_status ice_update_link_info(struct ice_port_info *pi);
-enum ice_status
+int ice_update_link_info(struct ice_port_info *pi);
+int
ice_acquire_res(struct ice_hw *hw, enum ice_aq_res_ids res,
enum ice_aq_res_access_type access, u32 timeout);
void ice_release_res(struct ice_hw *hw, enum ice_aq_res_ids res);
-enum ice_status
+int
ice_alloc_hw_res(struct ice_hw *hw, u16 type, u16 num, bool btm, u16 *res);
-enum ice_status
+int
ice_free_hw_res(struct ice_hw *hw, u16 type, u16 num, u16 *res);
-enum ice_status
+int
ice_aq_alloc_free_res(struct ice_hw *hw, u16 num_entries,
struct ice_aqc_alloc_free_res_elem *buf, u16 buf_size,
enum ice_adminq_opc opc, struct ice_sq_cd *cd);
bool ice_is_sbq_supported(struct ice_hw *hw);
struct ice_ctl_q_info *ice_get_sbq(struct ice_hw *hw);
-enum ice_status
+int
ice_sq_send_cmd(struct ice_hw *hw, struct ice_ctl_q_info *cq,
struct ice_aq_desc *desc, void *buf, u16 buf_size,
struct ice_sq_cd *cd);
void ice_clear_pxe_mode(struct ice_hw *hw);
-enum ice_status ice_get_caps(struct ice_hw *hw);
+int ice_get_caps(struct ice_hw *hw);
void ice_set_safe_mode_caps(struct ice_hw *hw);
-enum ice_status
+int
ice_write_rxq_ctx(struct ice_hw *hw, struct ice_rlan_ctx *rlan_ctx,
u32 rxq_index);
-enum ice_status
+int
ice_aq_get_rss_lut(struct ice_hw *hw, struct ice_aq_get_set_rss_lut_params *get_params);
-enum ice_status
+int
ice_aq_set_rss_lut(struct ice_hw *hw, struct ice_aq_get_set_rss_lut_params *set_params);
-enum ice_status
+int
ice_aq_get_rss_key(struct ice_hw *hw, u16 vsi_handle,
struct ice_aqc_get_set_rss_keys *keys);
-enum ice_status
+int
ice_aq_set_rss_key(struct ice_hw *hw, u16 vsi_handle,
struct ice_aqc_get_set_rss_keys *keys);
bool ice_check_sq_alive(struct ice_hw *hw, struct ice_ctl_q_info *cq);
-enum ice_status ice_aq_q_shutdown(struct ice_hw *hw, bool unloading);
+int ice_aq_q_shutdown(struct ice_hw *hw, bool unloading);
void ice_fill_dflt_direct_cmd_desc(struct ice_aq_desc *desc, u16 opcode);
extern const struct ice_ctx_ele ice_tlan_ctx_info[];
-enum ice_status
+int
ice_set_ctx(struct ice_hw *hw, u8 *src_ctx, u8 *dest_ctx,
const struct ice_ctx_ele *ce_info);
extern struct mutex ice_global_cfg_lock_sw;
-enum ice_status
+int
ice_aq_send_cmd(struct ice_hw *hw, struct ice_aq_desc *desc,
void *buf, u16 buf_size, struct ice_sq_cd *cd);
-enum ice_status ice_aq_get_fw_ver(struct ice_hw *hw, struct ice_sq_cd *cd);
+int ice_aq_get_fw_ver(struct ice_hw *hw, struct ice_sq_cd *cd);
-enum ice_status
+int
ice_aq_send_driver_ver(struct ice_hw *hw, struct ice_driver_ver *dv,
struct ice_sq_cd *cd);
-enum ice_status
+int
ice_aq_get_phy_caps(struct ice_port_info *pi, bool qual_mods, u8 report_mode,
struct ice_aqc_get_phy_caps_data *caps,
struct ice_sq_cd *cd);
-enum ice_status
+int
ice_aq_list_caps(struct ice_hw *hw, void *buf, u16 buf_size, u32 *cap_count,
enum ice_adminq_opc opc, struct ice_sq_cd *cd);
-enum ice_status
+int
ice_discover_dev_caps(struct ice_hw *hw, struct ice_hw_dev_caps *dev_caps);
void
ice_update_phy_type(u64 *phy_type_low, u64 *phy_type_high,
u16 link_speeds_bitmap);
-enum ice_status
+int
ice_aq_manage_mac_write(struct ice_hw *hw, const u8 *mac_addr, u8 flags,
struct ice_sq_cd *cd);
bool ice_is_e810(struct ice_hw *hw);
-enum ice_status ice_clear_pf_cfg(struct ice_hw *hw);
-enum ice_status
+int ice_clear_pf_cfg(struct ice_hw *hw);
+int
ice_aq_set_phy_cfg(struct ice_hw *hw, struct ice_port_info *pi,
struct ice_aqc_set_phy_cfg_data *cfg, struct ice_sq_cd *cd);
bool ice_fw_supports_link_override(struct ice_hw *hw);
-enum ice_status
+int
ice_get_link_default_override(struct ice_link_default_override_tlv *ldo,
struct ice_port_info *pi);
bool ice_is_phy_caps_an_enabled(struct ice_aqc_get_phy_caps_data *caps);
enum ice_fc_mode ice_caps_to_fc_mode(u8 caps);
enum ice_fec_mode ice_caps_to_fec_mode(u8 caps, u8 fec_options);
-enum ice_status
+int
ice_set_fc(struct ice_port_info *pi, u8 *aq_failures,
bool ena_auto_link_update);
-enum ice_status
+int
ice_cfg_phy_fc(struct ice_port_info *pi, struct ice_aqc_set_phy_cfg_data *cfg,
enum ice_fc_mode fc);
bool
@@ -125,27 +125,27 @@ void
ice_copy_phy_caps_to_cfg(struct ice_port_info *pi,
struct ice_aqc_get_phy_caps_data *caps,
struct ice_aqc_set_phy_cfg_data *cfg);
-enum ice_status
+int
ice_cfg_phy_fec(struct ice_port_info *pi, struct ice_aqc_set_phy_cfg_data *cfg,
enum ice_fec_mode fec);
-enum ice_status
+int
ice_aq_set_link_restart_an(struct ice_port_info *pi, bool ena_link,
struct ice_sq_cd *cd);
-enum ice_status
+int
ice_aq_set_mac_cfg(struct ice_hw *hw, u16 max_frame_size, struct ice_sq_cd *cd);
-enum ice_status
+int
ice_aq_get_link_info(struct ice_port_info *pi, bool ena_lse,
struct ice_link_status *link, struct ice_sq_cd *cd);
-enum ice_status
+int
ice_aq_set_event_mask(struct ice_hw *hw, u8 port_num, u16 mask,
struct ice_sq_cd *cd);
-enum ice_status
+int
ice_aq_set_mac_loopback(struct ice_hw *hw, bool ena_lpbk, struct ice_sq_cd *cd);
-enum ice_status
+int
ice_aq_set_port_id_led(struct ice_port_info *pi, bool is_orig_mode,
struct ice_sq_cd *cd);
-enum ice_status
+int
ice_aq_sff_eeprom(struct ice_hw *hw, u16 lport, u8 bus_addr,
u16 mem_addr, u8 page, u8 set_page, u8 *data, u8 length,
bool write, struct ice_sq_cd *cd);
@@ -159,19 +159,19 @@ ice_ena_vsi_rdma_qset(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
int
ice_dis_vsi_rdma_qset(struct ice_port_info *pi, u16 count, u32 *qset_teid,
u16 *q_id);
-enum ice_status
+int
ice_dis_vsi_txq(struct ice_port_info *pi, u16 vsi_handle, u8 tc, u8 num_queues,
u16 *q_handle, u16 *q_ids, u32 *q_teids,
enum ice_disq_rst_src rst_src, u16 vmvf_num,
struct ice_sq_cd *cd);
-enum ice_status
+int
ice_cfg_vsi_lan(struct ice_port_info *pi, u16 vsi_handle, u8 tc_bitmap,
u16 *max_lanqs);
-enum ice_status
+int
ice_ena_vsi_txq(struct ice_port_info *pi, u16 vsi_handle, u8 tc, u16 q_handle,
u8 num_qgrps, struct ice_aqc_add_tx_qgrp *buf, u16 buf_size,
struct ice_sq_cd *cd);
-enum ice_status ice_replay_vsi(struct ice_hw *hw, u16 vsi_handle);
+int ice_replay_vsi(struct ice_hw *hw, u16 vsi_handle);
void ice_replay_post(struct ice_hw *hw);
void ice_output_fw_log(struct ice_hw *hw, struct ice_aq_desc *desc, void *buf);
struct ice_q_ctx *
@@ -184,7 +184,7 @@ void
ice_stat_update32(struct ice_hw *hw, u32 reg, bool prev_stat_loaded,
u64 *prev_stat, u64 *cur_stat);
bool ice_is_e810t(struct ice_hw *hw);
-enum ice_status
+int
ice_sched_query_elem(struct ice_hw *hw, u32 node_teid,
struct ice_aqc_txsched_elem_data *buf);
int
@@ -199,11 +199,11 @@ ice_aq_set_gpio(struct ice_hw *hw, u16 gpio_ctrl_handle, u8 pin_idx, bool value,
int
ice_aq_get_gpio(struct ice_hw *hw, u16 gpio_ctrl_handle, u8 pin_idx,
bool *value, struct ice_sq_cd *cd);
-enum ice_status
+int
ice_aq_set_lldp_mib(struct ice_hw *hw, u8 mib_type, void *buf, u16 buf_size,
struct ice_sq_cd *cd);
bool ice_fw_supports_lldp_fltr_ctrl(struct ice_hw *hw);
-enum ice_status
+int
ice_lldp_fltr_add_remove(struct ice_hw *hw, u16 vsi_num, bool add);
bool ice_fw_supports_report_dflt_cfg(struct ice_hw *hw);
#endif /* _ICE_COMMON_H_ */
diff --git a/drivers/net/ethernet/intel/ice/ice_controlq.c b/drivers/net/ethernet/intel/ice/ice_controlq.c
index 03bdb12..6bcfee2 100644
--- a/drivers/net/ethernet/intel/ice/ice_controlq.c
+++ b/drivers/net/ethernet/intel/ice/ice_controlq.c
@@ -87,7 +87,7 @@ bool ice_check_sq_alive(struct ice_hw *hw, struct ice_ctl_q_info *cq)
* @hw: pointer to the hardware structure
* @cq: pointer to the specific Control queue
*/
-static enum ice_status
+static int
ice_alloc_ctrlq_sq_ring(struct ice_hw *hw, struct ice_ctl_q_info *cq)
{
size_t size = cq->num_sq_entries * sizeof(struct ice_aq_desc);
@@ -96,7 +96,7 @@ ice_alloc_ctrlq_sq_ring(struct ice_hw *hw, struct ice_ctl_q_info *cq)
&cq->sq.desc_buf.pa,
GFP_KERNEL | __GFP_ZERO);
if (!cq->sq.desc_buf.va)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
cq->sq.desc_buf.size = size;
cq->sq.cmd_buf = devm_kcalloc(ice_hw_to_dev(hw), cq->num_sq_entries,
@@ -107,7 +107,7 @@ ice_alloc_ctrlq_sq_ring(struct ice_hw *hw, struct ice_ctl_q_info *cq)
cq->sq.desc_buf.va = NULL;
cq->sq.desc_buf.pa = 0;
cq->sq.desc_buf.size = 0;
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
}
return 0;
@@ -118,7 +118,7 @@ ice_alloc_ctrlq_sq_ring(struct ice_hw *hw, struct ice_ctl_q_info *cq)
* @hw: pointer to the hardware structure
* @cq: pointer to the specific Control queue
*/
-static enum ice_status
+static int
ice_alloc_ctrlq_rq_ring(struct ice_hw *hw, struct ice_ctl_q_info *cq)
{
size_t size = cq->num_rq_entries * sizeof(struct ice_aq_desc);
@@ -127,7 +127,7 @@ ice_alloc_ctrlq_rq_ring(struct ice_hw *hw, struct ice_ctl_q_info *cq)
&cq->rq.desc_buf.pa,
GFP_KERNEL | __GFP_ZERO);
if (!cq->rq.desc_buf.va)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
cq->rq.desc_buf.size = size;
return 0;
}
@@ -154,7 +154,7 @@ static void ice_free_cq_ring(struct ice_hw *hw, struct ice_ctl_q_ring *ring)
* @hw: pointer to the hardware structure
* @cq: pointer to the specific Control queue
*/
-static enum ice_status
+static int
ice_alloc_rq_bufs(struct ice_hw *hw, struct ice_ctl_q_info *cq)
{
int i;
@@ -165,7 +165,7 @@ ice_alloc_rq_bufs(struct ice_hw *hw, struct ice_ctl_q_info *cq)
cq->rq.dma_head = devm_kcalloc(ice_hw_to_dev(hw), cq->num_rq_entries,
sizeof(cq->rq.desc_buf), GFP_KERNEL);
if (!cq->rq.dma_head)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
cq->rq.r.rq_bi = (struct ice_dma_mem *)cq->rq.dma_head;
/* allocate the mapped buffers */
@@ -218,7 +218,7 @@ ice_alloc_rq_bufs(struct ice_hw *hw, struct ice_ctl_q_info *cq)
devm_kfree(ice_hw_to_dev(hw), cq->rq.dma_head);
cq->rq.dma_head = NULL;
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
}
/**
@@ -226,7 +226,7 @@ ice_alloc_rq_bufs(struct ice_hw *hw, struct ice_ctl_q_info *cq)
* @hw: pointer to the hardware structure
* @cq: pointer to the specific Control queue
*/
-static enum ice_status
+static int
ice_alloc_sq_bufs(struct ice_hw *hw, struct ice_ctl_q_info *cq)
{
int i;
@@ -235,7 +235,7 @@ ice_alloc_sq_bufs(struct ice_hw *hw, struct ice_ctl_q_info *cq)
cq->sq.dma_head = devm_kcalloc(ice_hw_to_dev(hw), cq->num_sq_entries,
sizeof(cq->sq.desc_buf), GFP_KERNEL);
if (!cq->sq.dma_head)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
cq->sq.r.sq_bi = (struct ice_dma_mem *)cq->sq.dma_head;
/* allocate the mapped buffers */
@@ -266,10 +266,10 @@ ice_alloc_sq_bufs(struct ice_hw *hw, struct ice_ctl_q_info *cq)
devm_kfree(ice_hw_to_dev(hw), cq->sq.dma_head);
cq->sq.dma_head = NULL;
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
}
-static enum ice_status
+static int
ice_cfg_cq_regs(struct ice_hw *hw, struct ice_ctl_q_ring *ring, u16 num_entries)
{
/* Clear Head and Tail */
@@ -283,7 +283,7 @@ ice_cfg_cq_regs(struct ice_hw *hw, struct ice_ctl_q_ring *ring, u16 num_entries)
/* Check one register to verify that config was applied */
if (rd32(hw, ring->bal) != lower_32_bits(ring->desc_buf.pa))
- return ICE_ERR_AQ_ERROR;
+ return -EIO;
return 0;
}
@@ -295,8 +295,7 @@ ice_cfg_cq_regs(struct ice_hw *hw, struct ice_ctl_q_ring *ring, u16 num_entries)
*
* Configure base address and length registers for the transmit queue
*/
-static enum ice_status
-ice_cfg_sq_regs(struct ice_hw *hw, struct ice_ctl_q_info *cq)
+static int ice_cfg_sq_regs(struct ice_hw *hw, struct ice_ctl_q_info *cq)
{
return ice_cfg_cq_regs(hw, &cq->sq, cq->num_sq_entries);
}
@@ -308,10 +307,9 @@ ice_cfg_sq_regs(struct ice_hw *hw, struct ice_ctl_q_info *cq)
*
* Configure base address and length registers for the receive (event queue)
*/
-static enum ice_status
-ice_cfg_rq_regs(struct ice_hw *hw, struct ice_ctl_q_info *cq)
+static int ice_cfg_rq_regs(struct ice_hw *hw, struct ice_ctl_q_info *cq)
{
- enum ice_status status;
+ int status;
status = ice_cfg_cq_regs(hw, &cq->rq, cq->num_rq_entries);
if (status)
@@ -361,19 +359,19 @@ do { \
* Do *NOT* hold the lock when calling this as the memory allocation routines
* called are not going to be atomic context safe
*/
-static enum ice_status ice_init_sq(struct ice_hw *hw, struct ice_ctl_q_info *cq)
+static int ice_init_sq(struct ice_hw *hw, struct ice_ctl_q_info *cq)
{
- enum ice_status ret_code;
+ int ret_code;
if (cq->sq.count > 0) {
/* queue already initialized */
- ret_code = ICE_ERR_NOT_READY;
+ ret_code = -EBUSY;
goto init_ctrlq_exit;
}
/* verify input for valid configuration */
if (!cq->num_sq_entries || !cq->sq_buf_size) {
- ret_code = ICE_ERR_CFG;
+ ret_code = -EIO;
goto init_ctrlq_exit;
}
@@ -421,19 +419,19 @@ static enum ice_status ice_init_sq(struct ice_hw *hw, struct ice_ctl_q_info *cq)
* Do *NOT* hold the lock when calling this as the memory allocation routines
* called are not going to be atomic context safe
*/
-static enum ice_status ice_init_rq(struct ice_hw *hw, struct ice_ctl_q_info *cq)
+static int ice_init_rq(struct ice_hw *hw, struct ice_ctl_q_info *cq)
{
- enum ice_status ret_code;
+ int ret_code;
if (cq->rq.count > 0) {
/* queue already initialized */
- ret_code = ICE_ERR_NOT_READY;
+ ret_code = -EBUSY;
goto init_ctrlq_exit;
}
/* verify input for valid configuration */
if (!cq->num_rq_entries || !cq->rq_buf_size) {
- ret_code = ICE_ERR_CFG;
+ ret_code = -EIO;
goto init_ctrlq_exit;
}
@@ -474,15 +472,14 @@ static enum ice_status ice_init_rq(struct ice_hw *hw, struct ice_ctl_q_info *cq)
*
* The main shutdown routine for the Control Transmit Queue
*/
-static enum ice_status
-ice_shutdown_sq(struct ice_hw *hw, struct ice_ctl_q_info *cq)
+static int ice_shutdown_sq(struct ice_hw *hw, struct ice_ctl_q_info *cq)
{
- enum ice_status ret_code = 0;
+ int ret_code = 0;
mutex_lock(&cq->sq_lock);
if (!cq->sq.count) {
- ret_code = ICE_ERR_NOT_READY;
+ ret_code = -EBUSY;
goto shutdown_sq_out;
}
@@ -541,15 +538,14 @@ static bool ice_aq_ver_check(struct ice_hw *hw)
*
* The main shutdown routine for the Control Receive Queue
*/
-static enum ice_status
-ice_shutdown_rq(struct ice_hw *hw, struct ice_ctl_q_info *cq)
+static int ice_shutdown_rq(struct ice_hw *hw, struct ice_ctl_q_info *cq)
{
- enum ice_status ret_code = 0;
+ int ret_code = 0;
mutex_lock(&cq->rq_lock);
if (!cq->rq.count) {
- ret_code = ICE_ERR_NOT_READY;
+ ret_code = -EBUSY;
goto shutdown_rq_out;
}
@@ -576,17 +572,17 @@ ice_shutdown_rq(struct ice_hw *hw, struct ice_ctl_q_info *cq)
* ice_init_check_adminq - Check version for Admin Queue to know if its alive
* @hw: pointer to the hardware structure
*/
-static enum ice_status ice_init_check_adminq(struct ice_hw *hw)
+static int ice_init_check_adminq(struct ice_hw *hw)
{
struct ice_ctl_q_info *cq = &hw->adminq;
- enum ice_status status;
+ int status;
status = ice_aq_get_fw_ver(hw, NULL);
if (status)
goto init_ctrlq_free_rq;
if (!ice_aq_ver_check(hw)) {
- status = ICE_ERR_FW_API_VER;
+ status = -EIO;
goto init_ctrlq_free_rq;
}
@@ -612,10 +608,10 @@ static enum ice_status ice_init_check_adminq(struct ice_hw *hw)
*
* NOTE: this function does not initialize the controlq locks
*/
-static enum ice_status ice_init_ctrlq(struct ice_hw *hw, enum ice_ctl_q q_type)
+static int ice_init_ctrlq(struct ice_hw *hw, enum ice_ctl_q q_type)
{
struct ice_ctl_q_info *cq;
- enum ice_status ret_code;
+ int ret_code;
switch (q_type) {
case ICE_CTL_Q_ADMIN:
@@ -631,14 +627,14 @@ static enum ice_status ice_init_ctrlq(struct ice_hw *hw, enum ice_ctl_q q_type)
cq = &hw->mailboxq;
break;
default:
- return ICE_ERR_PARAM;
+ return -EINVAL;
}
cq->qtype = q_type;
/* verify input for valid configuration */
if (!cq->num_rq_entries || !cq->num_sq_entries ||
!cq->rq_buf_size || !cq->sq_buf_size) {
- return ICE_ERR_CFG;
+ return -EIO;
}
/* setup SQ command write back timeout */
@@ -751,10 +747,10 @@ void ice_shutdown_all_ctrlq(struct ice_hw *hw)
*
* NOTE: this function does not initialize the controlq locks.
*/
-enum ice_status ice_init_all_ctrlq(struct ice_hw *hw)
+int ice_init_all_ctrlq(struct ice_hw *hw)
{
- enum ice_status status;
u32 retry = 0;
+ int status;
/* Init FW admin queue */
do {
@@ -763,7 +759,7 @@ enum ice_status ice_init_all_ctrlq(struct ice_hw *hw)
return status;
status = ice_init_check_adminq(hw);
- if (status != ICE_ERR_AQ_FW_CRITICAL)
+ if (status != -EIO)
break;
ice_debug(hw, ICE_DBG_AQ_MSG, "Retry Admin Queue init due to FW critical error\n");
@@ -814,7 +810,7 @@ static void ice_init_ctrlq_locks(struct ice_ctl_q_info *cq)
* driver needs to re-initialize control queues at run time it should call
* ice_init_all_ctrlq instead.
*/
-enum ice_status ice_create_all_ctrlq(struct ice_hw *hw)
+int ice_create_all_ctrlq(struct ice_hw *hw)
{
ice_init_ctrlq_locks(&hw->adminq);
if (ice_is_sbq_supported(hw))
@@ -962,7 +958,7 @@ static bool ice_sq_done(struct ice_hw *hw, struct ice_ctl_q_info *cq)
* This is the main send command routine for the ATQ. It runs the queue,
* cleans the queue, etc.
*/
-enum ice_status
+int
ice_sq_send_cmd(struct ice_hw *hw, struct ice_ctl_q_info *cq,
struct ice_aq_desc *desc, void *buf, u16 buf_size,
struct ice_sq_cd *cd)
@@ -970,27 +966,27 @@ ice_sq_send_cmd(struct ice_hw *hw, struct ice_ctl_q_info *cq,
struct ice_dma_mem *dma_buf = NULL;
struct ice_aq_desc *desc_on_ring;
bool cmd_completed = false;
- enum ice_status status = 0;
struct ice_sq_cd *details;
u32 total_delay = 0;
+ int status = 0;
u16 retval = 0;
u32 val = 0;
/* if reset is in progress return a soft error */
if (hw->reset_ongoing)
- return ICE_ERR_RESET_ONGOING;
+ return -EBUSY;
mutex_lock(&cq->sq_lock);
cq->sq_last_status = ICE_AQ_RC_OK;
if (!cq->sq.count) {
ice_debug(hw, ICE_DBG_AQ_MSG, "Control Send queue not initialized.\n");
- status = ICE_ERR_AQ_EMPTY;
+ status = -EIO;
goto sq_send_command_error;
}
if ((buf && !buf_size) || (!buf && buf_size)) {
- status = ICE_ERR_PARAM;
+ status = -EINVAL;
goto sq_send_command_error;
}
@@ -998,7 +994,7 @@ ice_sq_send_cmd(struct ice_hw *hw, struct ice_ctl_q_info *cq,
if (buf_size > cq->sq_buf_size) {
ice_debug(hw, ICE_DBG_AQ_MSG, "Invalid buffer size for Control Send queue: %d.\n",
buf_size);
- status = ICE_ERR_INVAL_SIZE;
+ status = -EINVAL;
goto sq_send_command_error;
}
@@ -1011,7 +1007,7 @@ ice_sq_send_cmd(struct ice_hw *hw, struct ice_ctl_q_info *cq,
if (val >= cq->num_sq_entries) {
ice_debug(hw, ICE_DBG_AQ_MSG, "head overrun at %d in the Control Send Queue ring\n",
val);
- status = ICE_ERR_AQ_EMPTY;
+ status = -EIO;
goto sq_send_command_error;
}
@@ -1028,7 +1024,7 @@ ice_sq_send_cmd(struct ice_hw *hw, struct ice_ctl_q_info *cq,
*/
if (ice_clean_sq(hw, cq) == 0) {
ice_debug(hw, ICE_DBG_AQ_MSG, "Error: Control Send Queue is full.\n");
- status = ICE_ERR_AQ_FULL;
+ status = -ENOSPC;
goto sq_send_command_error;
}
@@ -1082,7 +1078,7 @@ ice_sq_send_cmd(struct ice_hw *hw, struct ice_ctl_q_info *cq,
if (copy_size > buf_size) {
ice_debug(hw, ICE_DBG_AQ_MSG, "Return len %d > than buf len %d\n",
copy_size, buf_size);
- status = ICE_ERR_AQ_ERROR;
+ status = -EIO;
} else {
memcpy(buf, dma_buf->va, copy_size);
}
@@ -1098,7 +1094,7 @@ ice_sq_send_cmd(struct ice_hw *hw, struct ice_ctl_q_info *cq,
}
cmd_completed = true;
if (!status && retval != ICE_AQ_RC_OK)
- status = ICE_ERR_AQ_ERROR;
+ status = -EIO;
cq->sq_last_status = (enum ice_aq_err)retval;
}
@@ -1116,10 +1112,10 @@ ice_sq_send_cmd(struct ice_hw *hw, struct ice_ctl_q_info *cq,
if (rd32(hw, cq->rq.len) & cq->rq.len_crit_mask ||
rd32(hw, cq->sq.len) & cq->sq.len_crit_mask) {
ice_debug(hw, ICE_DBG_AQ_MSG, "Critical FW error.\n");
- status = ICE_ERR_AQ_FW_CRITICAL;
+ status = -EIO;
} else {
ice_debug(hw, ICE_DBG_AQ_MSG, "Control Send Queue Writeback timeout.\n");
- status = ICE_ERR_AQ_TIMEOUT;
+ status = -EIO;
}
}
@@ -1154,15 +1150,15 @@ void ice_fill_dflt_direct_cmd_desc(struct ice_aq_desc *desc, u16 opcode)
* the contents through e. It can also return how many events are
* left to process through 'pending'.
*/
-enum ice_status
+int
ice_clean_rq_elem(struct ice_hw *hw, struct ice_ctl_q_info *cq,
struct ice_rq_event_info *e, u16 *pending)
{
u16 ntc = cq->rq.next_to_clean;
enum ice_aq_err rq_last_status;
- enum ice_status ret_code = 0;
struct ice_aq_desc *desc;
struct ice_dma_mem *bi;
+ int ret_code = 0;
u16 desc_idx;
u16 datalen;
u16 flags;
@@ -1176,7 +1172,7 @@ ice_clean_rq_elem(struct ice_hw *hw, struct ice_ctl_q_info *cq,
if (!cq->rq.count) {
ice_debug(hw, ICE_DBG_AQ_MSG, "Control Receive queue not initialized.\n");
- ret_code = ICE_ERR_AQ_EMPTY;
+ ret_code = -EIO;
goto clean_rq_elem_err;
}
@@ -1185,7 +1181,7 @@ ice_clean_rq_elem(struct ice_hw *hw, struct ice_ctl_q_info *cq,
if (ntu == ntc) {
/* nothing to do - shouldn't need to update ring's values */
- ret_code = ICE_ERR_AQ_NO_WORK;
+ ret_code = -EALREADY;
goto clean_rq_elem_out;
}
@@ -1196,7 +1192,7 @@ ice_clean_rq_elem(struct ice_hw *hw, struct ice_ctl_q_info *cq,
rq_last_status = (enum ice_aq_err)le16_to_cpu(desc->retval);
flags = le16_to_cpu(desc->flags);
if (flags & ICE_AQ_FLAG_ERR) {
- ret_code = ICE_ERR_AQ_ERROR;
+ ret_code = -EIO;
ice_debug(hw, ICE_DBG_AQ_MSG, "Control Receive Queue Event 0x%04X received with error 0x%X\n",
le16_to_cpu(desc->opcode), rq_last_status);
}
diff --git a/drivers/net/ethernet/intel/ice/ice_dcb.c b/drivers/net/ethernet/intel/ice/ice_dcb.c
index 241427c..0b146a0 100644
--- a/drivers/net/ethernet/intel/ice/ice_dcb.c
+++ b/drivers/net/ethernet/intel/ice/ice_dcb.c
@@ -2,7 +2,6 @@
/* Copyright (c) 2019, Intel Corporation. */
#include "ice_common.h"
-#include "ice_lib.h"
#include "ice_sched.h"
#include "ice_dcb.h"
@@ -19,19 +18,19 @@
*
* Requests the complete LLDP MIB (entire packet). (0x0A00)
*/
-static enum ice_status
+static int
ice_aq_get_lldp_mib(struct ice_hw *hw, u8 bridge_type, u8 mib_type, void *buf,
u16 buf_size, u16 *local_len, u16 *remote_len,
struct ice_sq_cd *cd)
{
struct ice_aqc_lldp_get_mib *cmd;
struct ice_aq_desc desc;
- enum ice_status status;
+ int status;
cmd = &desc.params.lldp_get_mib;
if (buf_size == 0 || !buf)
- return ICE_ERR_PARAM;
+ return -EINVAL;
ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_lldp_get_mib);
@@ -61,7 +60,7 @@ ice_aq_get_lldp_mib(struct ice_hw *hw, u8 bridge_type, u8 mib_type, void *buf,
* Enable or Disable posting of an event on ARQ when LLDP MIB
* associated with the interface changes (0x0A01)
*/
-static enum ice_status
+static int
ice_aq_cfg_lldp_mib_change(struct ice_hw *hw, bool ena_update,
struct ice_sq_cd *cd)
{
@@ -89,7 +88,7 @@ ice_aq_cfg_lldp_mib_change(struct ice_hw *hw, bool ena_update,
*
* Stop or Shutdown the embedded LLDP Agent (0x0A05)
*/
-enum ice_status
+int
ice_aq_stop_lldp(struct ice_hw *hw, bool shutdown_lldp_agent, bool persist,
struct ice_sq_cd *cd)
{
@@ -117,8 +116,7 @@ ice_aq_stop_lldp(struct ice_hw *hw, bool shutdown_lldp_agent, bool persist,
*
* Start the embedded LLDP Agent on all ports. (0x0A06)
*/
-enum ice_status
-ice_aq_start_lldp(struct ice_hw *hw, bool persist, struct ice_sq_cd *cd)
+int ice_aq_start_lldp(struct ice_hw *hw, bool persist, struct ice_sq_cd *cd)
{
struct ice_aqc_lldp_start *cmd;
struct ice_aq_desc desc;
@@ -598,18 +596,17 @@ ice_parse_org_tlv(struct ice_lldp_org_tlv *tlv, struct ice_dcbx_cfg *dcbcfg)
*
* Parse DCB configuration from the LLDPDU
*/
-static enum ice_status
-ice_lldp_to_dcb_cfg(u8 *lldpmib, struct ice_dcbx_cfg *dcbcfg)
+static int ice_lldp_to_dcb_cfg(u8 *lldpmib, struct ice_dcbx_cfg *dcbcfg)
{
struct ice_lldp_org_tlv *tlv;
- enum ice_status ret = 0;
u16 offset = 0;
+ int ret = 0;
u16 typelen;
u16 type;
u16 len;
if (!lldpmib || !dcbcfg)
- return ICE_ERR_PARAM;
+ return -EINVAL;
/* set to the start of LLDPDU */
lldpmib += ETH_HLEN;
@@ -649,17 +646,17 @@ ice_lldp_to_dcb_cfg(u8 *lldpmib, struct ice_dcbx_cfg *dcbcfg)
*
* Query DCB configuration from the firmware
*/
-enum ice_status
+int
ice_aq_get_dcb_cfg(struct ice_hw *hw, u8 mib_type, u8 bridgetype,
struct ice_dcbx_cfg *dcbcfg)
{
- enum ice_status ret;
u8 *lldpmib;
+ int ret;
/* Allocate the LLDPDU */
lldpmib = devm_kzalloc(ice_hw_to_dev(hw), ICE_LLDPDU_SIZE, GFP_KERNEL);
if (!lldpmib)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
ret = ice_aq_get_lldp_mib(hw, bridgetype, mib_type, (void *)lldpmib,
ICE_LLDPDU_SIZE, NULL, NULL, NULL);
@@ -684,17 +681,17 @@ ice_aq_get_dcb_cfg(struct ice_hw *hw, u8 mib_type, u8 bridgetype,
* @cd: pointer to command details structure or NULL
*
* Start/Stop the embedded dcbx Agent. In case that this wrapper function
- * returns ICE_SUCCESS, caller will need to check if FW returns back the same
+ * returns 0, caller will need to check if FW returns back the same
* value as stated in dcbx_agent_status, and react accordingly. (0x0A09)
*/
-enum ice_status
+int
ice_aq_start_stop_dcbx(struct ice_hw *hw, bool start_dcbx_agent,
bool *dcbx_agent_status, struct ice_sq_cd *cd)
{
struct ice_aqc_lldp_stop_start_specific_agent *cmd;
- enum ice_status status;
struct ice_aq_desc desc;
u16 opcode;
+ int status;
cmd = &desc.params.lldp_agent_ctrl;
@@ -724,7 +721,7 @@ ice_aq_start_stop_dcbx(struct ice_hw *hw, bool start_dcbx_agent,
*
* Get CEE DCBX mode operational configuration from firmware (0x0A07)
*/
-static enum ice_status
+static int
ice_aq_get_cee_dcb_cfg(struct ice_hw *hw,
struct ice_aqc_get_cee_dcb_cfg_resp *buff,
struct ice_sq_cd *cd)
@@ -749,7 +746,7 @@ int ice_aq_set_pfc_mode(struct ice_hw *hw, u8 pfc_mode, struct ice_sq_cd *cd)
{
struct ice_aqc_set_query_pfc_mode *cmd;
struct ice_aq_desc desc;
- enum ice_status status;
+ int status;
if (pfc_mode > ICE_AQC_PFC_DSCP_BASED_PFC)
return -EINVAL;
@@ -762,7 +759,7 @@ int ice_aq_set_pfc_mode(struct ice_hw *hw, u8 pfc_mode, struct ice_sq_cd *cd)
status = ice_aq_send_cmd(hw, &desc, NULL, 0, cd);
if (status)
- return ice_status_to_errno(status);
+ return status;
/* FW will write the PFC mode set back into cmd->pfc_mode, but if DCB is
* disabled, FW will write back 0 to cmd->pfc_mode. After the AQ has
@@ -903,14 +900,13 @@ ice_cee_to_dcb_cfg(struct ice_aqc_get_cee_dcb_cfg_resp *cee_cfg,
*
* Get IEEE or CEE mode DCB configuration from the Firmware
*/
-static enum ice_status
-ice_get_ieee_or_cee_dcb_cfg(struct ice_port_info *pi, u8 dcbx_mode)
+static int ice_get_ieee_or_cee_dcb_cfg(struct ice_port_info *pi, u8 dcbx_mode)
{
struct ice_dcbx_cfg *dcbx_cfg = NULL;
- enum ice_status ret;
+ int ret;
if (!pi)
- return ICE_ERR_PARAM;
+ return -EINVAL;
if (dcbx_mode == ICE_DCBX_MODE_IEEE)
dcbx_cfg = &pi->qos_cfg.local_dcbx_cfg;
@@ -943,14 +939,14 @@ ice_get_ieee_or_cee_dcb_cfg(struct ice_port_info *pi, u8 dcbx_mode)
*
* Get DCB configuration from the Firmware
*/
-enum ice_status ice_get_dcb_cfg(struct ice_port_info *pi)
+int ice_get_dcb_cfg(struct ice_port_info *pi)
{
struct ice_aqc_get_cee_dcb_cfg_resp cee_cfg;
struct ice_dcbx_cfg *dcbx_cfg;
- enum ice_status ret;
+ int ret;
if (!pi)
- return ICE_ERR_PARAM;
+ return -EINVAL;
ret = ice_aq_get_cee_dcb_cfg(pi->hw, &cee_cfg, NULL);
if (!ret) {
@@ -974,13 +970,13 @@ enum ice_status ice_get_dcb_cfg(struct ice_port_info *pi)
*
* Update DCB configuration from the Firmware
*/
-enum ice_status ice_init_dcb(struct ice_hw *hw, bool enable_mib_change)
+int ice_init_dcb(struct ice_hw *hw, bool enable_mib_change)
{
struct ice_qos_cfg *qos_cfg = &hw->port_info->qos_cfg;
- enum ice_status ret = 0;
+ int ret = 0;
if (!hw->func_caps.common_cap.dcb)
- return ICE_ERR_NOT_SUPPORTED;
+ return -EOPNOTSUPP;
qos_cfg->is_sw_lldp = true;
@@ -996,7 +992,7 @@ enum ice_status ice_init_dcb(struct ice_hw *hw, bool enable_mib_change)
return ret;
qos_cfg->is_sw_lldp = false;
} else if (qos_cfg->dcbx_status == ICE_DCBX_STATUS_DIS) {
- return ICE_ERR_NOT_READY;
+ return -EBUSY;
}
/* Configure the LLDP MIB change event */
@@ -1016,19 +1012,19 @@ enum ice_status ice_init_dcb(struct ice_hw *hw, bool enable_mib_change)
*
* Configure (disable/enable) MIB
*/
-enum ice_status ice_cfg_lldp_mib_change(struct ice_hw *hw, bool ena_mib)
+int ice_cfg_lldp_mib_change(struct ice_hw *hw, bool ena_mib)
{
struct ice_qos_cfg *qos_cfg = &hw->port_info->qos_cfg;
- enum ice_status ret;
+ int ret;
if (!hw->func_caps.common_cap.dcb)
- return ICE_ERR_NOT_SUPPORTED;
+ return -EOPNOTSUPP;
/* Get DCBX status */
qos_cfg->dcbx_status = ice_get_dcbx_status(hw);
if (qos_cfg->dcbx_status == ICE_DCBX_STATUS_DIS)
- return ICE_ERR_NOT_READY;
+ return -EBUSY;
ret = ice_aq_cfg_lldp_mib_change(hw, ena_mib, NULL);
if (!ret)
@@ -1469,16 +1465,16 @@ ice_dcb_cfg_to_lldp(u8 *lldpmib, u16 *miblen, struct ice_dcbx_cfg *dcbcfg)
*
* Set DCB configuration to the Firmware
*/
-enum ice_status ice_set_dcb_cfg(struct ice_port_info *pi)
+int ice_set_dcb_cfg(struct ice_port_info *pi)
{
u8 mib_type, *lldpmib = NULL;
struct ice_dcbx_cfg *dcbcfg;
- enum ice_status ret;
struct ice_hw *hw;
u16 miblen;
+ int ret;
if (!pi)
- return ICE_ERR_PARAM;
+ return -EINVAL;
hw = pi->hw;
@@ -1487,7 +1483,7 @@ enum ice_status ice_set_dcb_cfg(struct ice_port_info *pi)
/* Allocate the LLDPDU */
lldpmib = devm_kzalloc(ice_hw_to_dev(hw), ICE_LLDPDU_SIZE, GFP_KERNEL);
if (!lldpmib)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
mib_type = SET_LOCAL_MIB_TYPE_LOCAL_MIB;
if (dcbcfg->app_mode == ICE_DCBX_APPS_NON_WILLING)
@@ -1511,17 +1507,17 @@ enum ice_status ice_set_dcb_cfg(struct ice_port_info *pi)
*
* query current port ETS configuration
*/
-static enum ice_status
+static int
ice_aq_query_port_ets(struct ice_port_info *pi,
struct ice_aqc_port_ets_elem *buf, u16 buf_size,
struct ice_sq_cd *cd)
{
struct ice_aqc_query_port_ets *cmd;
struct ice_aq_desc desc;
- enum ice_status status;
+ int status;
if (!pi)
- return ICE_ERR_PARAM;
+ return -EINVAL;
cmd = &desc.params.port_ets;
ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_query_port_ets);
cmd->port_teid = pi->root->info.node_teid;
@@ -1537,18 +1533,18 @@ ice_aq_query_port_ets(struct ice_port_info *pi,
*
* update the SW DB with the new TC changes
*/
-static enum ice_status
+static int
ice_update_port_tc_tree_cfg(struct ice_port_info *pi,
struct ice_aqc_port_ets_elem *buf)
{
struct ice_sched_node *node, *tc_node;
struct ice_aqc_txsched_elem_data elem;
- enum ice_status status = 0;
u32 teid1, teid2;
+ int status = 0;
u8 i, j;
if (!pi)
- return ICE_ERR_PARAM;
+ return -EINVAL;
/* suspend the missing TC nodes */
for (i = 0; i < pi->root->num_children; i++) {
teid1 = le32_to_cpu(pi->root->children[i]->info.node_teid);
@@ -1605,12 +1601,12 @@ ice_update_port_tc_tree_cfg(struct ice_port_info *pi,
* query current port ETS configuration and update the
* SW DB with the TC changes
*/
-enum ice_status
+int
ice_query_port_ets(struct ice_port_info *pi,
struct ice_aqc_port_ets_elem *buf, u16 buf_size,
struct ice_sq_cd *cd)
{
- enum ice_status status;
+ int status;
mutex_lock(&pi->sched_lock);
status = ice_aq_query_port_ets(pi, buf, buf_size, cd);
diff --git a/drivers/net/ethernet/intel/ice/ice_dcb.h b/drivers/net/ethernet/intel/ice/ice_dcb.h
index 9b6f87a..d73348f 100644
--- a/drivers/net/ethernet/intel/ice/ice_dcb.h
+++ b/drivers/net/ethernet/intel/ice/ice_dcb.h
@@ -138,28 +138,27 @@ struct ice_cee_app_prio {
} __packed;
int ice_aq_set_pfc_mode(struct ice_hw *hw, u8 pfc_mode, struct ice_sq_cd *cd);
-enum ice_status
+int
ice_aq_get_dcb_cfg(struct ice_hw *hw, u8 mib_type, u8 bridgetype,
struct ice_dcbx_cfg *dcbcfg);
-enum ice_status ice_get_dcb_cfg(struct ice_port_info *pi);
-enum ice_status ice_set_dcb_cfg(struct ice_port_info *pi);
-enum ice_status ice_init_dcb(struct ice_hw *hw, bool enable_mib_change);
-enum ice_status
+int ice_get_dcb_cfg(struct ice_port_info *pi);
+int ice_set_dcb_cfg(struct ice_port_info *pi);
+int ice_init_dcb(struct ice_hw *hw, bool enable_mib_change);
+int
ice_query_port_ets(struct ice_port_info *pi,
struct ice_aqc_port_ets_elem *buf, u16 buf_size,
struct ice_sq_cd *cmd_details);
#ifdef CONFIG_DCB
-enum ice_status
+int
ice_aq_stop_lldp(struct ice_hw *hw, bool shutdown_lldp_agent, bool persist,
struct ice_sq_cd *cd);
-enum ice_status
-ice_aq_start_lldp(struct ice_hw *hw, bool persist, struct ice_sq_cd *cd);
-enum ice_status
+int ice_aq_start_lldp(struct ice_hw *hw, bool persist, struct ice_sq_cd *cd);
+int
ice_aq_start_stop_dcbx(struct ice_hw *hw, bool start_dcbx_agent,
bool *dcbx_agent_status, struct ice_sq_cd *cd);
-enum ice_status ice_cfg_lldp_mib_change(struct ice_hw *hw, bool ena_mib);
+int ice_cfg_lldp_mib_change(struct ice_hw *hw, bool ena_mib);
#else /* CONFIG_DCB */
-static inline enum ice_status
+static inline int
ice_aq_stop_lldp(struct ice_hw __always_unused *hw,
bool __always_unused shutdown_lldp_agent,
bool __always_unused persist,
@@ -168,7 +167,7 @@ ice_aq_stop_lldp(struct ice_hw __always_unused *hw,
return 0;
}
-static inline enum ice_status
+static inline int
ice_aq_start_lldp(struct ice_hw __always_unused *hw,
bool __always_unused persist,
struct ice_sq_cd __always_unused *cd)
@@ -176,7 +175,7 @@ ice_aq_start_lldp(struct ice_hw __always_unused *hw,
return 0;
}
-static inline enum ice_status
+static inline int
ice_aq_start_stop_dcbx(struct ice_hw __always_unused *hw,
bool __always_unused start_dcbx_agent,
bool *dcbx_agent_status,
@@ -187,7 +186,7 @@ ice_aq_start_stop_dcbx(struct ice_hw __always_unused *hw,
return 0;
}
-static inline enum ice_status
+static inline int
ice_cfg_lldp_mib_change(struct ice_hw __always_unused *hw,
bool __always_unused ena_mib)
{
diff --git a/drivers/net/ethernet/intel/ice/ice_dcb_lib.c b/drivers/net/ethernet/intel/ice/ice_dcb_lib.c
index a72e183..b94d8da 100644
--- a/drivers/net/ethernet/intel/ice/ice_dcb_lib.c
+++ b/drivers/net/ethernet/intel/ice/ice_dcb_lib.c
@@ -528,7 +528,7 @@ void ice_dcb_rebuild(struct ice_pf *pf)
struct ice_aqc_port_ets_elem buf = { 0 };
struct device *dev = ice_pf_to_dev(pf);
struct ice_dcbx_cfg *err_cfg;
- enum ice_status ret;
+ int ret;
ret = ice_query_port_ets(pf->hw.port_info, &buf, sizeof(buf), NULL);
if (ret) {
diff --git a/drivers/net/ethernet/intel/ice/ice_devlink.c b/drivers/net/ethernet/intel/ice/ice_devlink.c
index b9bd9f9..a230edb 100644
--- a/drivers/net/ethernet/intel/ice/ice_devlink.c
+++ b/drivers/net/ethernet/intel/ice/ice_devlink.c
@@ -1,6 +1,8 @@
// SPDX-License-Identifier: GPL-2.0
/* Copyright (c) 2020, Intel Corporation. */
+#include <linux/vmalloc.h>
+
#include "ice.h"
#include "ice_lib.h"
#include "ice_devlink.h"
@@ -39,13 +41,13 @@ static void ice_info_get_dsn(struct ice_pf *pf, struct ice_info_ctx *ctx)
static void ice_info_pba(struct ice_pf *pf, struct ice_info_ctx *ctx)
{
struct ice_hw *hw = &pf->hw;
- enum ice_status status;
+ int status;
status = ice_read_pba_string(hw, (u8 *)ctx->buf, sizeof(ctx->buf));
if (status)
/* We failed to locate the PBA, so just skip this entry */
- dev_dbg(ice_pf_to_dev(pf), "Failed to read Product Board Assembly string, status %s\n",
- ice_stat_str(status));
+ dev_dbg(ice_pf_to_dev(pf), "Failed to read Product Board Assembly string, status %d\n",
+ status);
}
static void ice_info_fw_mgmt(struct ice_pf *pf, struct ice_info_ctx *ctx)
@@ -251,7 +253,6 @@ static int ice_devlink_info_get(struct devlink *devlink,
struct device *dev = ice_pf_to_dev(pf);
struct ice_hw *hw = &pf->hw;
struct ice_info_ctx *ctx;
- enum ice_status status;
size_t i;
int err;
@@ -266,20 +267,19 @@ static int ice_devlink_info_get(struct devlink *devlink,
return -ENOMEM;
/* discover capabilities first */
- status = ice_discover_dev_caps(hw, &ctx->dev_caps);
- if (status) {
- dev_dbg(dev, "Failed to discover device capabilities, status %s aq_err %s\n",
- ice_stat_str(status), ice_aq_str(hw->adminq.sq_last_status));
+ err = ice_discover_dev_caps(hw, &ctx->dev_caps);
+ if (err) {
+ dev_dbg(dev, "Failed to discover device capabilities, status %d aq_err %s\n",
+ err, ice_aq_str(hw->adminq.sq_last_status));
NL_SET_ERR_MSG_MOD(extack, "Unable to discover device capabilities");
- err = -EIO;
goto out_free_ctx;
}
if (ctx->dev_caps.common_cap.nvm_update_pending_orom) {
- status = ice_get_inactive_orom_ver(hw, &ctx->pending_orom);
- if (status) {
- dev_dbg(dev, "Unable to read inactive Option ROM version data, status %s aq_err %s\n",
- ice_stat_str(status), ice_aq_str(hw->adminq.sq_last_status));
+ err = ice_get_inactive_orom_ver(hw, &ctx->pending_orom);
+ if (err) {
+ dev_dbg(dev, "Unable to read inactive Option ROM version data, status %d aq_err %s\n",
+ err, ice_aq_str(hw->adminq.sq_last_status));
/* disable display of pending Option ROM */
ctx->dev_caps.common_cap.nvm_update_pending_orom = false;
@@ -287,10 +287,10 @@ static int ice_devlink_info_get(struct devlink *devlink,
}
if (ctx->dev_caps.common_cap.nvm_update_pending_nvm) {
- status = ice_get_inactive_nvm_ver(hw, &ctx->pending_nvm);
- if (status) {
- dev_dbg(dev, "Unable to read inactive NVM version data, status %s aq_err %s\n",
- ice_stat_str(status), ice_aq_str(hw->adminq.sq_last_status));
+ err = ice_get_inactive_nvm_ver(hw, &ctx->pending_nvm);
+ if (err) {
+ dev_dbg(dev, "Unable to read inactive NVM version data, status %d aq_err %s\n",
+ err, ice_aq_str(hw->adminq.sq_last_status));
/* disable display of pending Option ROM */
ctx->dev_caps.common_cap.nvm_update_pending_nvm = false;
@@ -298,10 +298,10 @@ static int ice_devlink_info_get(struct devlink *devlink,
}
if (ctx->dev_caps.common_cap.nvm_update_pending_netlist) {
- status = ice_get_inactive_netlist_ver(hw, &ctx->pending_netlist);
- if (status) {
- dev_dbg(dev, "Unable to read inactive Netlist version data, status %s aq_err %s\n",
- ice_stat_str(status), ice_aq_str(hw->adminq.sq_last_status));
+ err = ice_get_inactive_netlist_ver(hw, &ctx->pending_netlist);
+ if (err) {
+ dev_dbg(dev, "Unable to read inactive Netlist version data, status %d aq_err %s\n",
+ err, ice_aq_str(hw->adminq.sq_last_status));
/* disable display of pending Option ROM */
ctx->dev_caps.common_cap.nvm_update_pending_netlist = false;
@@ -373,63 +373,225 @@ static int ice_devlink_info_get(struct devlink *devlink,
}
/**
- * ice_devlink_flash_update - Update firmware stored in flash on the device
- * @devlink: pointer to devlink associated with device to update
- * @params: flash update parameters
+ * ice_devlink_reload_empr_start - Start EMP reset to activate new firmware
+ * @devlink: pointer to the devlink instance to reload
+ * @netns_change: if true, the network namespace is changing
+ * @action: the action to perform. Must be DEVLINK_RELOAD_ACTION_FW_ACTIVATE
+ * @limit: limits on what reload should do, such as not resetting
* @extack: netlink extended ACK structure
*
- * Perform a device flash update. The bulk of the update logic is contained
- * within the ice_flash_pldm_image function.
+ * Allow user to activate new Embedded Management Processor firmware by
+ * issuing device specific EMP reset. Called in response to
+ * a DEVLINK_CMD_RELOAD with the DEVLINK_RELOAD_ACTION_FW_ACTIVATE.
*
- * Returns: zero on success, or an error code on failure.
+ * Note that teardown and rebuild of the driver state happens automatically as
+ * part of an interrupt and watchdog task. This is because all physical
+ * functions on the device must be able to reset when an EMP reset occurs from
+ * any source.
*/
static int
-ice_devlink_flash_update(struct devlink *devlink,
- struct devlink_flash_update_params *params,
- struct netlink_ext_ack *extack)
+ice_devlink_reload_empr_start(struct devlink *devlink, bool netns_change,
+ enum devlink_reload_action action,
+ enum devlink_reload_limit limit,
+ struct netlink_ext_ack *extack)
{
struct ice_pf *pf = devlink_priv(devlink);
+ struct device *dev = ice_pf_to_dev(pf);
struct ice_hw *hw = &pf->hw;
- u8 preservation;
+ u8 pending;
int err;
- if (!params->overwrite_mask) {
- /* preserve all settings and identifiers */
- preservation = ICE_AQC_NVM_PRESERVE_ALL;
- } else if (params->overwrite_mask == DEVLINK_FLASH_OVERWRITE_SETTINGS) {
- /* overwrite settings, but preserve the vital device identifiers */
- preservation = ICE_AQC_NVM_PRESERVE_SELECTED;
- } else if (params->overwrite_mask == (DEVLINK_FLASH_OVERWRITE_SETTINGS |
- DEVLINK_FLASH_OVERWRITE_IDENTIFIERS)) {
- /* overwrite both settings and identifiers, preserve nothing */
- preservation = ICE_AQC_NVM_NO_PRESERVATION;
- } else {
- NL_SET_ERR_MSG_MOD(extack, "Requested overwrite mask is not supported");
- return -EOPNOTSUPP;
- }
-
- if (!hw->dev_caps.common_cap.nvm_unified_update) {
- NL_SET_ERR_MSG_MOD(extack, "Current firmware does not support unified update");
- return -EOPNOTSUPP;
- }
-
- err = ice_check_for_pending_update(pf, NULL, extack);
+ err = ice_get_pending_updates(pf, &pending, extack);
if (err)
return err;
- devlink_flash_update_status_notify(devlink, "Preparing to flash", NULL, 0, 0);
+ /* pending is a bitmask of which flash banks have a pending update,
+ * including the main NVM bank, the Option ROM bank, and the netlist
+ * bank. If any of these bits are set, then there is a pending update
+ * waiting to be activated.
+ */
+ if (!pending) {
+ NL_SET_ERR_MSG_MOD(extack, "No pending firmware update");
+ return -ECANCELED;
+ }
- return ice_flash_pldm_image(pf, params->fw, preservation, extack);
+ if (pf->fw_emp_reset_disabled) {
+ NL_SET_ERR_MSG_MOD(extack, "EMP reset is not available. To activate firmware, a reboot or power cycle is needed");
+ return -ECANCELED;
+ }
+
+ dev_dbg(dev, "Issuing device EMP reset to activate firmware\n");
+
+ err = ice_aq_nvm_update_empr(hw);
+ if (err) {
+ dev_err(dev, "Failed to trigger EMP device reset to reload firmware, err %d aq_err %s\n",
+ err, ice_aq_str(hw->adminq.sq_last_status));
+ NL_SET_ERR_MSG_MOD(extack, "Failed to trigger EMP device reset to reload firmware");
+ return err;
+ }
+
+ return 0;
+}
+
+/**
+ * ice_devlink_reload_empr_finish - Wait for EMP reset to finish
+ * @devlink: pointer to the devlink instance reloading
+ * @action: the action requested
+ * @limit: limits imposed by userspace, such as not resetting
+ * @actions_performed: on return, indicate what actions actually performed
+ * @extack: netlink extended ACK structure
+ *
+ * Wait for driver to finish rebuilding after EMP reset is completed. This
+ * includes time to wait for both the actual device reset as well as the time
+ * for the driver's rebuild to complete.
+ */
+static int
+ice_devlink_reload_empr_finish(struct devlink *devlink,
+ enum devlink_reload_action action,
+ enum devlink_reload_limit limit,
+ u32 *actions_performed,
+ struct netlink_ext_ack *extack)
+{
+ struct ice_pf *pf = devlink_priv(devlink);
+ int err;
+
+ *actions_performed = BIT(DEVLINK_RELOAD_ACTION_FW_ACTIVATE);
+
+ err = ice_wait_for_reset(pf, 60 * HZ);
+ if (err) {
+ NL_SET_ERR_MSG_MOD(extack, "Device still resetting after 1 minute");
+ return err;
+ }
+
+ return 0;
}
static const struct devlink_ops ice_devlink_ops = {
.supported_flash_update_params = DEVLINK_SUPPORT_FLASH_UPDATE_OVERWRITE_MASK,
+ .reload_actions = BIT(DEVLINK_RELOAD_ACTION_FW_ACTIVATE),
+ /* The ice driver currently does not support driver reinit */
+ .reload_down = ice_devlink_reload_empr_start,
+ .reload_up = ice_devlink_reload_empr_finish,
.eswitch_mode_get = ice_eswitch_mode_get,
.eswitch_mode_set = ice_eswitch_mode_set,
.info_get = ice_devlink_info_get,
.flash_update = ice_devlink_flash_update,
};
+static int
+ice_devlink_enable_roce_get(struct devlink *devlink, u32 id,
+ struct devlink_param_gset_ctx *ctx)
+{
+ struct ice_pf *pf = devlink_priv(devlink);
+
+ ctx->val.vbool = pf->rdma_mode & IIDC_RDMA_PROTOCOL_ROCEV2 ? true : false;
+
+ return 0;
+}
+
+static int
+ice_devlink_enable_roce_set(struct devlink *devlink, u32 id,
+ struct devlink_param_gset_ctx *ctx)
+{
+ struct ice_pf *pf = devlink_priv(devlink);
+ bool roce_ena = ctx->val.vbool;
+ int ret;
+
+ if (!roce_ena) {
+ ice_unplug_aux_dev(pf);
+ pf->rdma_mode &= ~IIDC_RDMA_PROTOCOL_ROCEV2;
+ return 0;
+ }
+
+ pf->rdma_mode |= IIDC_RDMA_PROTOCOL_ROCEV2;
+ ret = ice_plug_aux_dev(pf);
+ if (ret)
+ pf->rdma_mode &= ~IIDC_RDMA_PROTOCOL_ROCEV2;
+
+ return ret;
+}
+
+static int
+ice_devlink_enable_roce_validate(struct devlink *devlink, u32 id,
+ union devlink_param_value val,
+ struct netlink_ext_ack *extack)
+{
+ struct ice_pf *pf = devlink_priv(devlink);
+
+ if (!test_bit(ICE_FLAG_RDMA_ENA, pf->flags))
+ return -EOPNOTSUPP;
+
+ if (pf->rdma_mode & IIDC_RDMA_PROTOCOL_IWARP) {
+ NL_SET_ERR_MSG_MOD(extack, "iWARP is currently enabled. This device cannot enable iWARP and RoCEv2 simultaneously");
+ return -EOPNOTSUPP;
+ }
+
+ return 0;
+}
+
+static int
+ice_devlink_enable_iw_get(struct devlink *devlink, u32 id,
+ struct devlink_param_gset_ctx *ctx)
+{
+ struct ice_pf *pf = devlink_priv(devlink);
+
+ ctx->val.vbool = pf->rdma_mode & IIDC_RDMA_PROTOCOL_IWARP;
+
+ return 0;
+}
+
+static int
+ice_devlink_enable_iw_set(struct devlink *devlink, u32 id,
+ struct devlink_param_gset_ctx *ctx)
+{
+ struct ice_pf *pf = devlink_priv(devlink);
+ bool iw_ena = ctx->val.vbool;
+ int ret;
+
+ if (!iw_ena) {
+ ice_unplug_aux_dev(pf);
+ pf->rdma_mode &= ~IIDC_RDMA_PROTOCOL_IWARP;
+ return 0;
+ }
+
+ pf->rdma_mode |= IIDC_RDMA_PROTOCOL_IWARP;
+ ret = ice_plug_aux_dev(pf);
+ if (ret)
+ pf->rdma_mode &= ~IIDC_RDMA_PROTOCOL_IWARP;
+
+ return ret;
+}
+
+static int
+ice_devlink_enable_iw_validate(struct devlink *devlink, u32 id,
+ union devlink_param_value val,
+ struct netlink_ext_ack *extack)
+{
+ struct ice_pf *pf = devlink_priv(devlink);
+
+ if (!test_bit(ICE_FLAG_RDMA_ENA, pf->flags))
+ return -EOPNOTSUPP;
+
+ if (pf->rdma_mode & IIDC_RDMA_PROTOCOL_ROCEV2) {
+ NL_SET_ERR_MSG_MOD(extack, "RoCEv2 is currently enabled. This device cannot enable iWARP and RoCEv2 simultaneously");
+ return -EOPNOTSUPP;
+ }
+
+ return 0;
+}
+
+static const struct devlink_param ice_devlink_params[] = {
+ DEVLINK_PARAM_GENERIC(ENABLE_ROCE, BIT(DEVLINK_PARAM_CMODE_RUNTIME),
+ ice_devlink_enable_roce_get,
+ ice_devlink_enable_roce_set,
+ ice_devlink_enable_roce_validate),
+ DEVLINK_PARAM_GENERIC(ENABLE_IWARP, BIT(DEVLINK_PARAM_CMODE_RUNTIME),
+ ice_devlink_enable_iw_get,
+ ice_devlink_enable_iw_set,
+ ice_devlink_enable_iw_validate),
+
+};
+
static void ice_devlink_free(void *devlink_ptr)
{
devlink_free((struct devlink *)devlink_ptr);
@@ -470,6 +632,7 @@ void ice_devlink_register(struct ice_pf *pf)
{
struct devlink *devlink = priv_to_devlink(pf);
+ devlink_set_features(devlink, DEVLINK_F_RELOAD);
devlink_register(devlink);
}
@@ -484,6 +647,36 @@ void ice_devlink_unregister(struct ice_pf *pf)
devlink_unregister(priv_to_devlink(pf));
}
+int ice_devlink_register_params(struct ice_pf *pf)
+{
+ struct devlink *devlink = priv_to_devlink(pf);
+ union devlink_param_value value;
+ int err;
+
+ err = devlink_params_register(devlink, ice_devlink_params,
+ ARRAY_SIZE(ice_devlink_params));
+ if (err)
+ return err;
+
+ value.vbool = false;
+ devlink_param_driverinit_value_set(devlink,
+ DEVLINK_PARAM_GENERIC_ID_ENABLE_IWARP,
+ value);
+
+ value.vbool = test_bit(ICE_FLAG_RDMA_ENA, pf->flags) ? true : false;
+ devlink_param_driverinit_value_set(devlink,
+ DEVLINK_PARAM_GENERIC_ID_ENABLE_ROCE,
+ value);
+
+ return 0;
+}
+
+void ice_devlink_unregister_params(struct ice_pf *pf)
+{
+ devlink_params_unregister(priv_to_devlink(pf), ice_devlink_params,
+ ARRAY_SIZE(ice_devlink_params));
+}
+
/**
* ice_devlink_create_pf_port - Create a devlink port for this PF
* @pf: the PF to create a devlink port for
@@ -597,7 +790,68 @@ void ice_devlink_destroy_vf_port(struct ice_vf *vf)
}
/**
- * ice_devlink_nvm_snapshot - Capture a snapshot of the Shadow RAM contents
+ * ice_devlink_nvm_snapshot - Capture a snapshot of the NVM flash contents
+ * @devlink: the devlink instance
+ * @ops: the devlink region being snapshotted
+ * @extack: extended ACK response structure
+ * @data: on exit points to snapshot data buffer
+ *
+ * This function is called in response to the DEVLINK_CMD_REGION_TRIGGER for
+ * the nvm-flash devlink region. It captures a snapshot of the full NVM flash
+ * contents, including both banks of flash. This snapshot can later be viewed
+ * via the devlink-region interface.
+ *
+ * It captures the flash using the FLASH_ONLY bit set when reading via
+ * firmware, so it does not read the current Shadow RAM contents. For that,
+ * use the shadow-ram region.
+ *
+ * @returns zero on success, and updates the data pointer. Returns a non-zero
+ * error code on failure.
+ */
+static int ice_devlink_nvm_snapshot(struct devlink *devlink,
+ const struct devlink_region_ops *ops,
+ struct netlink_ext_ack *extack, u8 **data)
+{
+ struct ice_pf *pf = devlink_priv(devlink);
+ struct device *dev = ice_pf_to_dev(pf);
+ struct ice_hw *hw = &pf->hw;
+ void *nvm_data;
+ u32 nvm_size;
+ int status;
+
+ nvm_size = hw->flash.flash_size;
+ nvm_data = vzalloc(nvm_size);
+ if (!nvm_data)
+ return -ENOMEM;
+
+ status = ice_acquire_nvm(hw, ICE_RES_READ);
+ if (status) {
+ dev_dbg(dev, "ice_acquire_nvm failed, err %d aq_err %d\n",
+ status, hw->adminq.sq_last_status);
+ NL_SET_ERR_MSG_MOD(extack, "Failed to acquire NVM semaphore");
+ vfree(nvm_data);
+ return status;
+ }
+
+ status = ice_read_flat_nvm(hw, 0, &nvm_size, nvm_data, false);
+ if (status) {
+ dev_dbg(dev, "ice_read_flat_nvm failed after reading %u bytes, err %d aq_err %d\n",
+ nvm_size, status, hw->adminq.sq_last_status);
+ NL_SET_ERR_MSG_MOD(extack, "Failed to read NVM contents");
+ ice_release_nvm(hw);
+ vfree(nvm_data);
+ return status;
+ }
+
+ ice_release_nvm(hw);
+
+ *data = nvm_data;
+
+ return 0;
+}
+
+/**
+ * ice_devlink_sram_snapshot - Capture a snapshot of the Shadow RAM contents
* @devlink: the devlink instance
* @ops: the devlink region being snapshotted
* @extack: extended ACK response structure
@@ -611,44 +865,47 @@ void ice_devlink_destroy_vf_port(struct ice_vf *vf)
* @returns zero on success, and updates the data pointer. Returns a non-zero
* error code on failure.
*/
-static int ice_devlink_nvm_snapshot(struct devlink *devlink,
- const struct devlink_region_ops *ops,
- struct netlink_ext_ack *extack, u8 **data)
+static int
+ice_devlink_sram_snapshot(struct devlink *devlink,
+ const struct devlink_region_ops __always_unused *ops,
+ struct netlink_ext_ack *extack, u8 **data)
{
struct ice_pf *pf = devlink_priv(devlink);
struct device *dev = ice_pf_to_dev(pf);
struct ice_hw *hw = &pf->hw;
- enum ice_status status;
- void *nvm_data;
- u32 nvm_size;
+ u8 *sram_data;
+ u32 sram_size;
+ int err;
- nvm_size = hw->flash.flash_size;
- nvm_data = vzalloc(nvm_size);
- if (!nvm_data)
+ sram_size = hw->flash.sr_words * 2u;
+ sram_data = vzalloc(sram_size);
+ if (!sram_data)
return -ENOMEM;
- status = ice_acquire_nvm(hw, ICE_RES_READ);
- if (status) {
+ err = ice_acquire_nvm(hw, ICE_RES_READ);
+ if (err) {
dev_dbg(dev, "ice_acquire_nvm failed, err %d aq_err %d\n",
- status, hw->adminq.sq_last_status);
+ err, hw->adminq.sq_last_status);
NL_SET_ERR_MSG_MOD(extack, "Failed to acquire NVM semaphore");
- vfree(nvm_data);
- return -EIO;
+ vfree(sram_data);
+ return err;
}
- status = ice_read_flat_nvm(hw, 0, &nvm_size, nvm_data, false);
- if (status) {
+ /* Read from the Shadow RAM, rather than directly from NVM */
+ err = ice_read_flat_nvm(hw, 0, &sram_size, sram_data, true);
+ if (err) {
dev_dbg(dev, "ice_read_flat_nvm failed after reading %u bytes, err %d aq_err %d\n",
- nvm_size, status, hw->adminq.sq_last_status);
- NL_SET_ERR_MSG_MOD(extack, "Failed to read NVM contents");
+ sram_size, err, hw->adminq.sq_last_status);
+ NL_SET_ERR_MSG_MOD(extack,
+ "Failed to read Shadow RAM contents");
ice_release_nvm(hw);
- vfree(nvm_data);
- return -EIO;
+ vfree(sram_data);
+ return err;
}
ice_release_nvm(hw);
- *data = nvm_data;
+ *data = sram_data;
return 0;
}
@@ -675,8 +932,8 @@ ice_devlink_devcaps_snapshot(struct devlink *devlink,
struct ice_pf *pf = devlink_priv(devlink);
struct device *dev = ice_pf_to_dev(pf);
struct ice_hw *hw = &pf->hw;
- enum ice_status status;
void *devcaps;
+ int status;
devcaps = vzalloc(ICE_AQ_MAX_BUF_LEN);
if (!devcaps)
@@ -689,7 +946,7 @@ ice_devlink_devcaps_snapshot(struct devlink *devlink,
status, hw->adminq.sq_last_status);
NL_SET_ERR_MSG_MOD(extack, "Failed to read device capabilities");
vfree(devcaps);
- return -EIO;
+ return status;
}
*data = (u8 *)devcaps;
@@ -703,6 +960,12 @@ static const struct devlink_region_ops ice_nvm_region_ops = {
.snapshot = ice_devlink_nvm_snapshot,
};
+static const struct devlink_region_ops ice_sram_region_ops = {
+ .name = "shadow-ram",
+ .destructor = vfree,
+ .snapshot = ice_devlink_sram_snapshot,
+};
+
static const struct devlink_region_ops ice_devcaps_region_ops = {
.name = "device-caps",
.destructor = vfree,
@@ -720,7 +983,7 @@ void ice_devlink_init_regions(struct ice_pf *pf)
{
struct devlink *devlink = priv_to_devlink(pf);
struct device *dev = ice_pf_to_dev(pf);
- u64 nvm_size;
+ u64 nvm_size, sram_size;
nvm_size = pf->hw.flash.flash_size;
pf->nvm_region = devlink_region_create(devlink, &ice_nvm_region_ops, 1,
@@ -731,6 +994,15 @@ void ice_devlink_init_regions(struct ice_pf *pf)
pf->nvm_region = NULL;
}
+ sram_size = pf->hw.flash.sr_words * 2u;
+ pf->sram_region = devlink_region_create(devlink, &ice_sram_region_ops,
+ 1, sram_size);
+ if (IS_ERR(pf->sram_region)) {
+ dev_err(dev, "failed to create shadow-ram devlink region, err %ld\n",
+ PTR_ERR(pf->sram_region));
+ pf->sram_region = NULL;
+ }
+
pf->devcaps_region = devlink_region_create(devlink,
&ice_devcaps_region_ops, 10,
ICE_AQ_MAX_BUF_LEN);
@@ -751,6 +1023,10 @@ void ice_devlink_destroy_regions(struct ice_pf *pf)
{
if (pf->nvm_region)
devlink_region_destroy(pf->nvm_region);
+
+ if (pf->sram_region)
+ devlink_region_destroy(pf->sram_region);
+
if (pf->devcaps_region)
devlink_region_destroy(pf->devcaps_region);
}
diff --git a/drivers/net/ethernet/intel/ice/ice_devlink.h b/drivers/net/ethernet/intel/ice/ice_devlink.h
index b7f9551e..fe006d9 100644
--- a/drivers/net/ethernet/intel/ice/ice_devlink.h
+++ b/drivers/net/ethernet/intel/ice/ice_devlink.h
@@ -8,6 +8,8 @@ struct ice_pf *ice_allocate_pf(struct device *dev);
void ice_devlink_register(struct ice_pf *pf);
void ice_devlink_unregister(struct ice_pf *pf);
+int ice_devlink_register_params(struct ice_pf *pf);
+void ice_devlink_unregister_params(struct ice_pf *pf);
int ice_devlink_create_pf_port(struct ice_pf *pf);
void ice_devlink_destroy_pf_port(struct ice_pf *pf);
int ice_devlink_create_vf_port(struct ice_vf *vf);
diff --git a/drivers/net/ethernet/intel/ice/ice_eswitch.c b/drivers/net/ethernet/intel/ice/ice_eswitch.c
index d1d7389..864692b 100644
--- a/drivers/net/ethernet/intel/ice/ice_eswitch.c
+++ b/drivers/net/ethernet/intel/ice/ice_eswitch.c
@@ -10,6 +10,100 @@
#include "ice_tc_lib.h"
/**
+ * ice_eswitch_add_vf_mac_rule - add adv rule with VF's MAC
+ * @pf: pointer to PF struct
+ * @vf: pointer to VF struct
+ * @mac: VF's MAC address
+ *
+ * This function adds advanced rule that forwards packets with
+ * VF's MAC address (src MAC) to the corresponding switchdev ctrl VSI queue.
+ */
+int
+ice_eswitch_add_vf_mac_rule(struct ice_pf *pf, struct ice_vf *vf, const u8 *mac)
+{
+ struct ice_vsi *ctrl_vsi = pf->switchdev.control_vsi;
+ struct ice_adv_rule_info rule_info = { 0 };
+ struct ice_adv_lkup_elem *list;
+ struct ice_hw *hw = &pf->hw;
+ const u16 lkups_cnt = 1;
+ int err;
+
+ list = kcalloc(lkups_cnt, sizeof(*list), GFP_ATOMIC);
+ if (!list)
+ return -ENOMEM;
+
+ list[0].type = ICE_MAC_OFOS;
+ ether_addr_copy(list[0].h_u.eth_hdr.src_addr, mac);
+ eth_broadcast_addr(list[0].m_u.eth_hdr.src_addr);
+
+ rule_info.sw_act.flag |= ICE_FLTR_TX;
+ rule_info.sw_act.vsi_handle = ctrl_vsi->idx;
+ rule_info.sw_act.fltr_act = ICE_FWD_TO_Q;
+ rule_info.rx = false;
+ rule_info.sw_act.fwd_id.q_id = hw->func_caps.common_cap.rxq_first_id +
+ ctrl_vsi->rxq_map[vf->vf_id];
+ rule_info.flags_info.act |= ICE_SINGLE_ACT_LB_ENABLE;
+ rule_info.flags_info.act_valid = true;
+
+ err = ice_add_adv_rule(hw, list, lkups_cnt, &rule_info,
+ vf->repr->mac_rule);
+ if (err)
+ dev_err(ice_pf_to_dev(pf), "Unable to add VF mac rule in switchdev mode for VF %d",
+ vf->vf_id);
+ else
+ vf->repr->rule_added = true;
+
+ kfree(list);
+ return err;
+}
+
+/**
+ * ice_eswitch_replay_vf_mac_rule - replay adv rule with VF's MAC
+ * @vf: pointer to vF struct
+ *
+ * This function replays VF's MAC rule after reset.
+ */
+void ice_eswitch_replay_vf_mac_rule(struct ice_vf *vf)
+{
+ int err;
+
+ if (!ice_is_switchdev_running(vf->pf))
+ return;
+
+ if (is_valid_ether_addr(vf->hw_lan_addr.addr)) {
+ err = ice_eswitch_add_vf_mac_rule(vf->pf, vf,
+ vf->hw_lan_addr.addr);
+ if (err) {
+ dev_err(ice_pf_to_dev(vf->pf), "Failed to add MAC %pM for VF %d\n, error %d\n",
+ vf->hw_lan_addr.addr, vf->vf_id, err);
+ return;
+ }
+ vf->num_mac++;
+
+ ether_addr_copy(vf->dev_lan_addr.addr, vf->hw_lan_addr.addr);
+ }
+}
+
+/**
+ * ice_eswitch_del_vf_mac_rule - delete adv rule with VF's MAC
+ * @vf: pointer to the VF struct
+ *
+ * Delete the advanced rule that was used to forward packets with the VF's MAC
+ * address (src MAC) to the corresponding switchdev ctrl VSI queue.
+ */
+void ice_eswitch_del_vf_mac_rule(struct ice_vf *vf)
+{
+ if (!ice_is_switchdev_running(vf->pf))
+ return;
+
+ if (!vf->repr->rule_added)
+ return;
+
+ ice_rem_adv_rule_by_id(&vf->pf->hw, vf->repr->mac_rule);
+ vf->repr->rule_added = false;
+}
+
+/**
* ice_eswitch_setup_env - configure switchdev HW filters
* @pf: pointer to PF struct
*
@@ -21,7 +115,6 @@ static int ice_eswitch_setup_env(struct ice_pf *pf)
struct ice_vsi *uplink_vsi = pf->switchdev.uplink_vsi;
struct net_device *uplink_netdev = uplink_vsi->netdev;
struct ice_vsi *ctrl_vsi = pf->switchdev.control_vsi;
- struct ice_port_info *pi = pf->hw.port_info;
bool rule_added = false;
ice_vsi_manage_vlan_stripping(ctrl_vsi, false);
@@ -42,29 +135,17 @@ static int ice_eswitch_setup_env(struct ice_pf *pf)
rule_added = true;
}
- if (ice_cfg_dflt_vsi(pi->hw, ctrl_vsi->idx, true, ICE_FLTR_TX))
- goto err_def_tx;
-
if (ice_vsi_update_security(uplink_vsi, ice_vsi_ctx_set_allow_override))
goto err_override_uplink;
if (ice_vsi_update_security(ctrl_vsi, ice_vsi_ctx_set_allow_override))
goto err_override_control;
- if (ice_fltr_update_flags_dflt_rule(ctrl_vsi, pi->dflt_tx_vsi_rule_id,
- ICE_FLTR_TX,
- ICE_SINGLE_ACT_LB_ENABLE))
- goto err_update_action;
-
return 0;
-err_update_action:
- ice_vsi_update_security(ctrl_vsi, ice_vsi_ctx_clear_allow_override);
err_override_control:
ice_vsi_update_security(uplink_vsi, ice_vsi_ctx_clear_allow_override);
err_override_uplink:
- ice_cfg_dflt_vsi(pi->hw, ctrl_vsi->idx, false, ICE_FLTR_TX);
-err_def_tx:
if (rule_added)
ice_clear_dflt_vsi(uplink_vsi->vsw);
err_def_rx:
@@ -167,21 +248,11 @@ static int ice_eswitch_setup_reprs(struct ice_pf *pf)
netif_keep_dst(vf->repr->netdev);
}
- kfree(ctrl_vsi->target_netdevs);
-
- ctrl_vsi->target_netdevs = kcalloc(max_vsi_num + 1,
- sizeof(*ctrl_vsi->target_netdevs),
- GFP_KERNEL);
- if (!ctrl_vsi->target_netdevs)
- goto err;
-
ice_for_each_vf(pf, i) {
struct ice_repr *repr = pf->vf[i].repr;
struct ice_vsi *vsi = repr->src_vsi;
struct metadata_dst *dst;
- ctrl_vsi->target_netdevs[vsi->vsi_num] = repr->netdev;
-
dst = repr->dst;
dst->u.port_info.port_id = vsi->vsi_num;
dst->u.port_info.lower_dev = repr->netdev;
@@ -214,7 +285,6 @@ ice_eswitch_release_reprs(struct ice_pf *pf, struct ice_vsi *ctrl_vsi)
{
int i;
- kfree(ctrl_vsi->target_netdevs);
ice_for_each_vf(pf, i) {
struct ice_vsi *vsi = pf->vf[i].repr->src_vsi;
struct ice_vf *vf = &pf->vf[i];
@@ -320,7 +390,6 @@ static void ice_eswitch_release_env(struct ice_pf *pf)
ice_vsi_update_security(ctrl_vsi, ice_vsi_ctx_clear_allow_override);
ice_vsi_update_security(uplink_vsi, ice_vsi_ctx_clear_allow_override);
- ice_cfg_dflt_vsi(&pf->hw, ctrl_vsi->idx, false, ICE_FLTR_TX);
ice_clear_dflt_vsi(uplink_vsi->vsw);
ice_fltr_add_mac_and_broadcast(uplink_vsi,
uplink_vsi->port_info->mac.perm_addr,
@@ -375,24 +444,6 @@ static void ice_eswitch_napi_disable(struct ice_pf *pf)
}
/**
- * ice_eswitch_set_rxdid - configure rxdid on all Rx queues from VSI
- * @vsi: VSI to setup rxdid on
- * @rxdid: flex descriptor id
- */
-static void ice_eswitch_set_rxdid(struct ice_vsi *vsi, u32 rxdid)
-{
- struct ice_hw *hw = &vsi->back->hw;
- int i;
-
- ice_for_each_rxq(vsi, i) {
- struct ice_rx_ring *ring = vsi->rx_rings[i];
- u16 pf_q = vsi->rxq_map[ring->q_index];
-
- ice_write_qrxflxp_cntxt(hw, pf_q, rxdid, 0x3, true);
- }
-}
-
-/**
* ice_eswitch_enable_switchdev - configure eswitch in switchdev mode
* @pf: pointer to PF structure
*/
@@ -425,8 +476,6 @@ static int ice_eswitch_enable_switchdev(struct ice_pf *pf)
ice_eswitch_napi_enable(pf);
- ice_eswitch_set_rxdid(ctrl_vsi, ICE_RXDID_FLEX_NIC_2);
-
return 0;
err_setup_reprs:
@@ -448,6 +497,7 @@ static void ice_eswitch_disable_switchdev(struct ice_pf *pf)
ice_eswitch_napi_disable(pf);
ice_eswitch_release_env(pf);
+ ice_rem_adv_rule_for_vsi(&pf->hw, ctrl_vsi->idx);
ice_eswitch_release_reprs(pf, ctrl_vsi);
ice_vsi_release(ctrl_vsi);
ice_repr_rem_from_all_vfs(pf);
@@ -497,34 +547,6 @@ ice_eswitch_mode_set(struct devlink *devlink, u16 mode,
}
/**
- * ice_eswitch_get_target_netdev - return port representor netdev
- * @rx_ring: pointer to Rx ring
- * @rx_desc: pointer to Rx descriptor
- *
- * When working in switchdev mode context (when control VSI is used), this
- * function returns netdev of appropriate port representor. For non-switchdev
- * context, regular netdev associated with Rx ring is returned.
- */
-struct net_device *
-ice_eswitch_get_target_netdev(struct ice_rx_ring *rx_ring,
- union ice_32b_rx_flex_desc *rx_desc)
-{
- struct ice_32b_rx_flex_desc_nic_2 *desc;
- struct ice_vsi *vsi = rx_ring->vsi;
- struct ice_vsi *control_vsi;
- u16 target_vsi_id;
-
- control_vsi = vsi->back->switchdev.control_vsi;
- if (vsi != control_vsi)
- return rx_ring->netdev;
-
- desc = (struct ice_32b_rx_flex_desc_nic_2 *)rx_desc;
- target_vsi_id = le16_to_cpu(desc->src_vsi);
-
- return vsi->target_netdevs[target_vsi_id];
-}
-
-/**
* ice_eswitch_mode_get - get current eswitch mode
* @devlink: pointer to devlink structure
* @mode: output parameter for current eswitch mode
@@ -648,7 +670,6 @@ int ice_eswitch_rebuild(struct ice_pf *pf)
return status;
ice_eswitch_napi_enable(pf);
- ice_eswitch_set_rxdid(ctrl_vsi, ICE_RXDID_FLEX_NIC_2);
ice_eswitch_start_all_tx_queues(pf);
return 0;
diff --git a/drivers/net/ethernet/intel/ice/ice_eswitch.h b/drivers/net/ethernet/intel/ice/ice_eswitch.h
index 364cd2a..bd58d9d 100644
--- a/drivers/net/ethernet/intel/ice/ice_eswitch.h
+++ b/drivers/net/ethernet/intel/ice/ice_eswitch.h
@@ -20,10 +20,11 @@ bool ice_is_eswitch_mode_switchdev(struct ice_pf *pf);
void ice_eswitch_update_repr(struct ice_vsi *vsi);
void ice_eswitch_stop_all_tx_queues(struct ice_pf *pf);
-
-struct net_device *
-ice_eswitch_get_target_netdev(struct ice_rx_ring *rx_ring,
- union ice_32b_rx_flex_desc *rx_desc);
+int
+ice_eswitch_add_vf_mac_rule(struct ice_pf *pf, struct ice_vf *vf,
+ const u8 *mac);
+void ice_eswitch_replay_vf_mac_rule(struct ice_vf *vf);
+void ice_eswitch_del_vf_mac_rule(struct ice_vf *vf);
void ice_eswitch_set_target_vsi(struct sk_buff *skb,
struct ice_tx_offload_params *off);
@@ -33,6 +34,15 @@ ice_eswitch_port_start_xmit(struct sk_buff *skb, struct net_device *netdev);
static inline void ice_eswitch_release(struct ice_pf *pf) { }
static inline void ice_eswitch_stop_all_tx_queues(struct ice_pf *pf) { }
+static inline void ice_eswitch_replay_vf_mac_rule(struct ice_vf *vf) { }
+static inline void ice_eswitch_del_vf_mac_rule(struct ice_vf *vf) { }
+
+static inline int
+ice_eswitch_add_vf_mac_rule(struct ice_pf *pf, struct ice_vf *vf,
+ const u8 *mac)
+{
+ return -EOPNOTSUPP;
+}
static inline void
ice_eswitch_set_target_vsi(struct sk_buff *skb,
@@ -67,13 +77,6 @@ static inline bool ice_is_eswitch_mode_switchdev(struct ice_pf *pf)
return false;
}
-static inline struct net_device *
-ice_eswitch_get_target_netdev(struct ice_rx_ring *rx_ring,
- union ice_32b_rx_flex_desc *rx_desc)
-{
- return rx_ring->netdev;
-}
-
static inline netdev_tx_t
ice_eswitch_port_start_xmit(struct sk_buff *skb, struct net_device *netdev)
{
diff --git a/drivers/net/ethernet/intel/ice/ice_ethtool.c b/drivers/net/ethernet/intel/ice/ice_ethtool.c
index 572519e..e2e3ef7 100644
--- a/drivers/net/ethernet/intel/ice/ice_ethtool.c
+++ b/drivers/net/ethernet/intel/ice/ice_ethtool.c
@@ -270,9 +270,8 @@ ice_get_eeprom(struct net_device *netdev, struct ethtool_eeprom *eeprom,
struct ice_vsi *vsi = np->vsi;
struct ice_pf *pf = vsi->back;
struct ice_hw *hw = &pf->hw;
- enum ice_status status;
struct device *dev;
- int ret = 0;
+ int ret;
u8 *buf;
dev = ice_pf_to_dev(pf);
@@ -285,22 +284,18 @@ ice_get_eeprom(struct net_device *netdev, struct ethtool_eeprom *eeprom,
if (!buf)
return -ENOMEM;
- status = ice_acquire_nvm(hw, ICE_RES_READ);
- if (status) {
- dev_err(dev, "ice_acquire_nvm failed, err %s aq_err %s\n",
- ice_stat_str(status),
- ice_aq_str(hw->adminq.sq_last_status));
- ret = -EIO;
+ ret = ice_acquire_nvm(hw, ICE_RES_READ);
+ if (ret) {
+ dev_err(dev, "ice_acquire_nvm failed, err %d aq_err %s\n",
+ ret, ice_aq_str(hw->adminq.sq_last_status));
goto out;
}
- status = ice_read_flat_nvm(hw, eeprom->offset, &eeprom->len, buf,
- false);
- if (status) {
- dev_err(dev, "ice_read_flat_nvm failed, err %s aq_err %s\n",
- ice_stat_str(status),
- ice_aq_str(hw->adminq.sq_last_status));
- ret = -EIO;
+ ret = ice_read_flat_nvm(hw, eeprom->offset, &eeprom->len, buf,
+ false);
+ if (ret) {
+ dev_err(dev, "ice_read_flat_nvm failed, err %d aq_err %s\n",
+ ret, ice_aq_str(hw->adminq.sq_last_status));
goto release;
}
@@ -342,14 +337,14 @@ static bool ice_active_vfs(struct ice_pf *pf)
static u64 ice_link_test(struct net_device *netdev)
{
struct ice_netdev_priv *np = netdev_priv(netdev);
- enum ice_status status;
bool link_up = false;
+ int status;
netdev_info(netdev, "link test\n");
status = ice_get_link_status(np->vsi->port_info, &link_up);
if (status) {
- netdev_err(netdev, "link query error, status = %s\n",
- ice_stat_str(status));
+ netdev_err(netdev, "link query error, status = %d\n",
+ status);
return 1;
}
@@ -1052,8 +1047,7 @@ ice_get_fecparam(struct net_device *netdev, struct ethtool_fecparam *fecparam)
struct ice_link_status *link_info;
struct ice_vsi *vsi = np->vsi;
struct ice_port_info *pi;
- enum ice_status status;
- int err = 0;
+ int err;
pi = vsi->port_info;
@@ -1079,12 +1073,10 @@ ice_get_fecparam(struct net_device *netdev, struct ethtool_fecparam *fecparam)
if (!caps)
return -ENOMEM;
- status = ice_aq_get_phy_caps(pi, false, ICE_AQC_REPORT_TOPO_CAP_MEDIA,
- caps, NULL);
- if (status) {
- err = -EAGAIN;
+ err = ice_aq_get_phy_caps(pi, false, ICE_AQC_REPORT_TOPO_CAP_MEDIA,
+ caps, NULL);
+ if (err)
goto done;
- }
/* Set supported/configured FEC modes based on PHY capability */
if (caps->caps & ICE_AQC_PHY_EN_AUTO_FEC)
@@ -1203,7 +1195,7 @@ static int ice_set_priv_flags(struct net_device *netdev, u32 flags)
if (test_bit(ICE_FLAG_FW_LLDP_AGENT, change_flags)) {
if (!test_bit(ICE_FLAG_FW_LLDP_AGENT, pf->flags)) {
- enum ice_status status;
+ int status;
/* Disable FW LLDP engine */
status = ice_cfg_lldp_mib_change(&pf->hw, false);
@@ -1232,8 +1224,8 @@ static int ice_set_priv_flags(struct net_device *netdev, u32 flags)
pf->dcbx_cap &= ~DCB_CAP_DCBX_LLD_MANAGED;
pf->dcbx_cap |= DCB_CAP_DCBX_HOST;
} else {
- enum ice_status status;
bool dcbx_agent_status;
+ int status;
if (ice_get_pfc_mode(pf) == ICE_QOS_MODE_DSCP) {
clear_bit(ICE_FLAG_FW_LLDP_AGENT, pf->flags);
@@ -1288,8 +1280,10 @@ static int ice_set_priv_flags(struct net_device *netdev, u32 flags)
}
if (test_bit(ICE_FLAG_LEGACY_RX, change_flags)) {
/* down and up VSI so that changes of Rx cfg are reflected. */
- ice_down(vsi);
- ice_up(vsi);
+ if (!test_and_set_bit(ICE_VSI_DOWN, vsi->state)) {
+ ice_down(vsi);
+ ice_up(vsi);
+ }
}
/* don't allow modification of this flag when a single VF is in
* promiscuous mode because it's not supported
@@ -1938,8 +1932,7 @@ ice_get_link_ksettings(struct net_device *netdev,
struct ice_aqc_get_phy_caps_data *caps;
struct ice_link_status *hw_link_info;
struct ice_vsi *vsi = np->vsi;
- enum ice_status status;
- int err = 0;
+ int err;
ethtool_link_ksettings_zero_link_mode(ks, supported);
ethtool_link_ksettings_zero_link_mode(ks, advertising);
@@ -1990,12 +1983,10 @@ ice_get_link_ksettings(struct net_device *netdev,
if (!caps)
return -ENOMEM;
- status = ice_aq_get_phy_caps(vsi->port_info, false,
- ICE_AQC_REPORT_ACTIVE_CFG, caps, NULL);
- if (status) {
- err = -EIO;
+ err = ice_aq_get_phy_caps(vsi->port_info, false,
+ ICE_AQC_REPORT_ACTIVE_CFG, caps, NULL);
+ if (err)
goto done;
- }
/* Set the advertised flow control based on the PHY capability */
if ((caps->caps & ICE_AQC_PHY_EN_TX_LINK_PAUSE) &&
@@ -2027,12 +2018,10 @@ ice_get_link_ksettings(struct net_device *netdev,
caps->link_fec_options & ICE_AQC_PHY_FEC_25G_RS_544_REQ)
ethtool_link_ksettings_add_link_mode(ks, advertising, FEC_RS);
- status = ice_aq_get_phy_caps(vsi->port_info, false,
- ICE_AQC_REPORT_TOPO_CAP_MEDIA, caps, NULL);
- if (status) {
- err = -EIO;
+ err = ice_aq_get_phy_caps(vsi->port_info, false,
+ ICE_AQC_REPORT_TOPO_CAP_MEDIA, caps, NULL);
+ if (err)
goto done;
- }
/* Set supported FEC modes based on PHY capability */
ethtool_link_ksettings_add_link_mode(ks, supported, FEC_NONE);
@@ -2210,11 +2199,10 @@ ice_set_link_ksettings(struct net_device *netdev,
struct ice_pf *pf = np->vsi->back;
struct ice_port_info *pi;
u8 autoneg_changed = 0;
- enum ice_status status;
u64 phy_type_high = 0;
u64 phy_type_low = 0;
- int err = 0;
bool linkup;
+ int err;
pi = np->vsi->port_info;
@@ -2234,15 +2222,13 @@ ice_set_link_ksettings(struct net_device *netdev,
/* Get the PHY capabilities based on media */
if (ice_fw_supports_report_dflt_cfg(pi->hw))
- status = ice_aq_get_phy_caps(pi, false, ICE_AQC_REPORT_DFLT_CFG,
- phy_caps, NULL);
+ err = ice_aq_get_phy_caps(pi, false, ICE_AQC_REPORT_DFLT_CFG,
+ phy_caps, NULL);
else
- status = ice_aq_get_phy_caps(pi, false, ICE_AQC_REPORT_TOPO_CAP_MEDIA,
- phy_caps, NULL);
- if (status) {
- err = -EIO;
+ err = ice_aq_get_phy_caps(pi, false, ICE_AQC_REPORT_TOPO_CAP_MEDIA,
+ phy_caps, NULL);
+ if (err)
goto done;
- }
/* save autoneg out of ksettings */
autoneg = copy_ks.base.autoneg;
@@ -2308,11 +2294,9 @@ ice_set_link_ksettings(struct net_device *netdev,
/* Call to get the current link speed */
pi->phy.get_link_info = true;
- status = ice_get_link_status(pi, &linkup);
- if (status) {
- err = -EIO;
+ err = ice_get_link_status(pi, &linkup);
+ if (err)
goto done;
- }
curr_link_speed = pi->phy.link_info.link_speed;
adv_link_speed = ice_ksettings_find_adv_link_speed(ks);
@@ -2381,10 +2365,9 @@ ice_set_link_ksettings(struct net_device *netdev,
}
/* make the aq call */
- status = ice_aq_set_phy_cfg(&pf->hw, pi, &config, NULL);
- if (status) {
+ err = ice_aq_set_phy_cfg(&pf->hw, pi, &config, NULL);
+ if (err) {
netdev_info(netdev, "Set phy config failed,\n");
- err = -EIO;
goto done;
}
@@ -2522,9 +2505,9 @@ static int
ice_set_rss_hash_opt(struct ice_vsi *vsi, struct ethtool_rxnfc *nfc)
{
struct ice_pf *pf = vsi->back;
- enum ice_status status;
struct device *dev;
u64 hashed_flds;
+ int status;
u32 hdrs;
dev = ice_pf_to_dev(pf);
@@ -2550,9 +2533,9 @@ ice_set_rss_hash_opt(struct ice_vsi *vsi, struct ethtool_rxnfc *nfc)
status = ice_add_rss_cfg(&pf->hw, vsi->idx, hashed_flds, hdrs);
if (status) {
- dev_dbg(dev, "ice_add_rss_cfg failed, vsi num = %d, error = %s\n",
- vsi->vsi_num, ice_stat_str(status));
- return -EINVAL;
+ dev_dbg(dev, "ice_add_rss_cfg failed, vsi num = %d, error = %d\n",
+ vsi->vsi_num, status);
+ return status;
}
return 0;
@@ -2686,7 +2669,9 @@ ice_get_rxnfc(struct net_device *netdev, struct ethtool_rxnfc *cmd,
}
static void
-ice_get_ringparam(struct net_device *netdev, struct ethtool_ringparam *ring)
+ice_get_ringparam(struct net_device *netdev, struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct ice_netdev_priv *np = netdev_priv(netdev);
struct ice_vsi *vsi = np->vsi;
@@ -2704,7 +2689,9 @@ ice_get_ringparam(struct net_device *netdev, struct ethtool_ringparam *ring)
}
static int
-ice_set_ringparam(struct net_device *netdev, struct ethtool_ringparam *ring)
+ice_set_ringparam(struct net_device *netdev, struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct ice_netdev_priv *np = netdev_priv(netdev);
struct ice_tx_ring *xdp_rings = NULL;
@@ -2949,7 +2936,7 @@ ice_get_pauseparam(struct net_device *netdev, struct ethtool_pauseparam *pause)
struct ice_port_info *pi = np->vsi->port_info;
struct ice_aqc_get_phy_caps_data *pcaps;
struct ice_dcbx_cfg *dcbx_cfg;
- enum ice_status status;
+ int status;
/* Initialize pause params */
pause->rx_pause = 0;
@@ -2999,11 +2986,10 @@ ice_set_pauseparam(struct net_device *netdev, struct ethtool_pauseparam *pause)
struct ice_vsi *vsi = np->vsi;
struct ice_hw *hw = &pf->hw;
struct ice_port_info *pi;
- enum ice_status status;
u8 aq_failures;
bool link_up;
- int err = 0;
u32 is_an;
+ int err;
pi = vsi->port_info;
hw_link_info = &pi->phy.link_info;
@@ -3029,11 +3015,11 @@ ice_set_pauseparam(struct net_device *netdev, struct ethtool_pauseparam *pause)
return -ENOMEM;
/* Get current PHY config */
- status = ice_aq_get_phy_caps(pi, false, ICE_AQC_REPORT_ACTIVE_CFG, pcaps,
- NULL);
- if (status) {
+ err = ice_aq_get_phy_caps(pi, false, ICE_AQC_REPORT_ACTIVE_CFG, pcaps,
+ NULL);
+ if (err) {
kfree(pcaps);
- return -EIO;
+ return err;
}
is_an = ice_is_phy_caps_an_enabled(pcaps) ? AUTONEG_ENABLE :
@@ -3069,22 +3055,19 @@ ice_set_pauseparam(struct net_device *netdev, struct ethtool_pauseparam *pause)
return -EINVAL;
/* Set the FC mode and only restart AN if link is up */
- status = ice_set_fc(pi, &aq_failures, link_up);
+ err = ice_set_fc(pi, &aq_failures, link_up);
if (aq_failures & ICE_SET_FC_AQ_FAIL_GET) {
- netdev_info(netdev, "Set fc failed on the get_phy_capabilities call with err %s aq_err %s\n",
- ice_stat_str(status),
- ice_aq_str(hw->adminq.sq_last_status));
+ netdev_info(netdev, "Set fc failed on the get_phy_capabilities call with err %d aq_err %s\n",
+ err, ice_aq_str(hw->adminq.sq_last_status));
err = -EAGAIN;
} else if (aq_failures & ICE_SET_FC_AQ_FAIL_SET) {
- netdev_info(netdev, "Set fc failed on the set_phy_config call with err %s aq_err %s\n",
- ice_stat_str(status),
- ice_aq_str(hw->adminq.sq_last_status));
+ netdev_info(netdev, "Set fc failed on the set_phy_config call with err %d aq_err %s\n",
+ err, ice_aq_str(hw->adminq.sq_last_status));
err = -EAGAIN;
} else if (aq_failures & ICE_SET_FC_AQ_FAIL_UPDATE) {
- netdev_info(netdev, "Set fc failed on the get_link_info call with err %s aq_err %s\n",
- ice_stat_str(status),
- ice_aq_str(hw->adminq.sq_last_status));
+ netdev_info(netdev, "Set fc failed on the get_link_info call with err %d aq_err %s\n",
+ err, ice_aq_str(hw->adminq.sq_last_status));
err = -EAGAIN;
}
@@ -3924,16 +3907,16 @@ ice_get_module_info(struct net_device *netdev,
struct ice_vsi *vsi = np->vsi;
struct ice_pf *pf = vsi->back;
struct ice_hw *hw = &pf->hw;
- enum ice_status status;
u8 sff8472_comp = 0;
u8 sff8472_swap = 0;
u8 sff8636_rev = 0;
u8 value = 0;
+ int status;
status = ice_aq_sff_eeprom(hw, 0, ICE_I2C_EEPROM_DEV_ADDR, 0x00, 0x00,
0, &value, 1, 0, NULL);
if (status)
- return -EIO;
+ return status;
switch (value) {
case ICE_MODULE_TYPE_SFP:
@@ -3941,12 +3924,12 @@ ice_get_module_info(struct net_device *netdev,
ICE_MODULE_SFF_8472_COMP, 0x00, 0,
&sff8472_comp, 1, 0, NULL);
if (status)
- return -EIO;
+ return status;
status = ice_aq_sff_eeprom(hw, 0, ICE_I2C_EEPROM_DEV_ADDR,
ICE_MODULE_SFF_8472_SWAP, 0x00, 0,
&sff8472_swap, 1, 0, NULL);
if (status)
- return -EIO;
+ return status;
if (sff8472_swap & ICE_MODULE_SFF_ADDR_MODE) {
modinfo->type = ETH_MODULE_SFF_8079;
@@ -3966,7 +3949,7 @@ ice_get_module_info(struct net_device *netdev,
ICE_MODULE_REVISION_ADDR, 0x00, 0,
&sff8636_rev, 1, 0, NULL);
if (status)
- return -EIO;
+ return status;
/* Check revision compliance */
if (sff8636_rev > 0x02) {
/* Module is SFF-8636 compliant */
@@ -4001,11 +3984,11 @@ ice_get_module_eeprom(struct net_device *netdev,
struct ice_vsi *vsi = np->vsi;
struct ice_pf *pf = vsi->back;
struct ice_hw *hw = &pf->hw;
- enum ice_status status;
bool is_sfp = false;
unsigned int i, j;
u16 offset = 0;
u8 page = 0;
+ int status;
if (!ee || !ee->len || !data)
return -EINVAL;
@@ -4013,7 +3996,7 @@ ice_get_module_eeprom(struct net_device *netdev,
status = ice_aq_sff_eeprom(hw, 0, addr, offset, page, 0, value, 1, 0,
NULL);
if (status)
- return -EIO;
+ return status;
if (value[0] == ICE_MODULE_TYPE_SFP)
is_sfp = true;
diff --git a/drivers/net/ethernet/intel/ice/ice_ethtool_fdir.c b/drivers/net/ethernet/intel/ice/ice_ethtool_fdir.c
index b6e7f47..5d10c4f 100644
--- a/drivers/net/ethernet/intel/ice/ice_ethtool_fdir.c
+++ b/drivers/net/ethernet/intel/ice/ice_ethtool_fdir.c
@@ -5,6 +5,7 @@
#include "ice.h"
#include "ice_lib.h"
+#include "ice_fdir.h"
#include "ice_flow.h"
static struct in6_addr full_ipv6_addr_mask = {
@@ -205,7 +206,7 @@ int ice_get_ethtool_fdir_entry(struct ice_hw *hw, struct ethtool_rxnfc *cmd)
if (rule->dest_ctl == ICE_FLTR_PRGM_DESC_DEST_DROP_PKT)
fsp->ring_cookie = RX_CLS_FLOW_DISC;
else
- fsp->ring_cookie = rule->q_index;
+ fsp->ring_cookie = rule->orig_q_index;
idx = ice_ethtool_flow_to_fltr(fsp->flow_type);
if (idx == ICE_FLTR_PTYPE_NONF_NONE) {
@@ -257,6 +258,80 @@ ice_get_fdir_fltr_ids(struct ice_hw *hw, struct ethtool_rxnfc *cmd,
}
/**
+ * ice_fdir_remap_entries - update the FDir entries in profile
+ * @prof: FDir structure pointer
+ * @tun: tunneled or non-tunneled packet
+ * @idx: FDir entry index
+ */
+static void
+ice_fdir_remap_entries(struct ice_fd_hw_prof *prof, int tun, int idx)
+{
+ if (idx != prof->cnt && tun < ICE_FD_HW_SEG_MAX) {
+ int i;
+
+ for (i = idx; i < (prof->cnt - 1); i++) {
+ u64 old_entry_h;
+
+ old_entry_h = prof->entry_h[i + 1][tun];
+ prof->entry_h[i][tun] = old_entry_h;
+ prof->vsi_h[i] = prof->vsi_h[i + 1];
+ }
+
+ prof->entry_h[i][tun] = 0;
+ prof->vsi_h[i] = 0;
+ }
+}
+
+/**
+ * ice_fdir_rem_adq_chnl - remove an ADQ channel from HW filter rules
+ * @hw: hardware structure containing filter list
+ * @vsi_idx: VSI handle
+ */
+void ice_fdir_rem_adq_chnl(struct ice_hw *hw, u16 vsi_idx)
+{
+ int status, flow;
+
+ if (!hw->fdir_prof)
+ return;
+
+ for (flow = 0; flow < ICE_FLTR_PTYPE_MAX; flow++) {
+ struct ice_fd_hw_prof *prof = hw->fdir_prof[flow];
+ int tun, i;
+
+ if (!prof || !prof->cnt)
+ continue;
+
+ for (tun = 0; tun < ICE_FD_HW_SEG_MAX; tun++) {
+ u64 prof_id;
+
+ prof_id = flow + tun * ICE_FLTR_PTYPE_MAX;
+
+ for (i = 0; i < prof->cnt; i++) {
+ if (prof->vsi_h[i] != vsi_idx)
+ continue;
+
+ prof->entry_h[i][tun] = 0;
+ prof->vsi_h[i] = 0;
+ break;
+ }
+
+ /* after clearing FDir entries update the remaining */
+ ice_fdir_remap_entries(prof, tun, i);
+
+ /* find flow profile corresponding to prof_id and clear
+ * vsi_idx from bitmap.
+ */
+ status = ice_flow_rem_vsi_prof(hw, vsi_idx, prof_id);
+ if (status) {
+ dev_err(ice_hw_to_dev(hw), "ice_flow_rem_vsi_prof() failed status=%d\n",
+ status);
+ }
+ }
+ prof->cnt--;
+ }
+}
+
+/**
* ice_fdir_get_hw_prof - return the ice_fd_hw_proc associated with a flow
* @hw: hardware structure containing the filter list
* @blk: hardware block
@@ -514,6 +589,28 @@ ice_fdir_alloc_flow_prof(struct ice_hw *hw, enum ice_fltr_ptype flow)
}
/**
+ * ice_fdir_prof_vsi_idx - find or insert a vsi_idx in structure
+ * @prof: pointer to flow director HW profile
+ * @vsi_idx: vsi_idx to locate
+ *
+ * return the index of the vsi_idx. if vsi_idx is not found insert it
+ * into the vsi_h table.
+ */
+static u16
+ice_fdir_prof_vsi_idx(struct ice_fd_hw_prof *prof, int vsi_idx)
+{
+ u16 idx = 0;
+
+ for (idx = 0; idx < prof->cnt; idx++)
+ if (prof->vsi_h[idx] == vsi_idx)
+ return idx;
+
+ if (idx == prof->cnt)
+ prof->vsi_h[prof->cnt++] = vsi_idx;
+ return idx;
+}
+
+/**
* ice_fdir_set_hw_fltr_rule - Configure HW tables to generate a FDir rule
* @pf: pointer to the PF structure
* @seg: protocol header description pointer
@@ -530,11 +627,12 @@ ice_fdir_set_hw_fltr_rule(struct ice_pf *pf, struct ice_flow_seg_info *seg,
struct ice_flow_prof *prof = NULL;
struct ice_fd_hw_prof *hw_prof;
struct ice_hw *hw = &pf->hw;
- enum ice_status status;
u64 entry1_h = 0;
u64 entry2_h = 0;
+ bool del_last;
u64 prof_id;
int err;
+ int idx;
main_vsi = ice_get_main_vsi(pf);
if (!main_vsi)
@@ -581,24 +679,20 @@ ice_fdir_set_hw_fltr_rule(struct ice_pf *pf, struct ice_flow_seg_info *seg,
* actions (NULL) and zero actions 0.
*/
prof_id = flow + tun * ICE_FLTR_PTYPE_MAX;
- status = ice_flow_add_prof(hw, ICE_BLK_FD, ICE_FLOW_RX, prof_id, seg,
- TNL_SEG_CNT(tun), &prof);
- if (status)
- return ice_status_to_errno(status);
- status = ice_flow_add_entry(hw, ICE_BLK_FD, prof_id, main_vsi->idx,
- main_vsi->idx, ICE_FLOW_PRIO_NORMAL,
- seg, &entry1_h);
- if (status) {
- err = ice_status_to_errno(status);
+ err = ice_flow_add_prof(hw, ICE_BLK_FD, ICE_FLOW_RX, prof_id, seg,
+ TNL_SEG_CNT(tun), &prof);
+ if (err)
+ return err;
+ err = ice_flow_add_entry(hw, ICE_BLK_FD, prof_id, main_vsi->idx,
+ main_vsi->idx, ICE_FLOW_PRIO_NORMAL,
+ seg, &entry1_h);
+ if (err)
goto err_prof;
- }
- status = ice_flow_add_entry(hw, ICE_BLK_FD, prof_id, main_vsi->idx,
- ctrl_vsi->idx, ICE_FLOW_PRIO_NORMAL,
- seg, &entry2_h);
- if (status) {
- err = ice_status_to_errno(status);
+ err = ice_flow_add_entry(hw, ICE_BLK_FD, prof_id, main_vsi->idx,
+ ctrl_vsi->idx, ICE_FLOW_PRIO_NORMAL,
+ seg, &entry2_h);
+ if (err)
goto err_entry;
- }
hw_prof->fdir_seg[tun] = seg;
hw_prof->entry_h[0][tun] = entry1_h;
@@ -608,8 +702,60 @@ ice_fdir_set_hw_fltr_rule(struct ice_pf *pf, struct ice_flow_seg_info *seg,
if (!hw_prof->cnt)
hw_prof->cnt = 2;
+ for (idx = 1; idx < ICE_CHNL_MAX_TC; idx++) {
+ u16 vsi_idx;
+ u16 vsi_h;
+
+ if (!ice_is_adq_active(pf) || !main_vsi->tc_map_vsi[idx])
+ continue;
+
+ entry1_h = 0;
+ vsi_h = main_vsi->tc_map_vsi[idx]->idx;
+ err = ice_flow_add_entry(hw, ICE_BLK_FD, prof_id,
+ main_vsi->idx, vsi_h,
+ ICE_FLOW_PRIO_NORMAL, seg,
+ &entry1_h);
+ if (err) {
+ dev_err(dev, "Could not add Channel VSI %d to flow group\n",
+ idx);
+ goto err_unroll;
+ }
+
+ vsi_idx = ice_fdir_prof_vsi_idx(hw_prof,
+ main_vsi->tc_map_vsi[idx]->idx);
+ hw_prof->entry_h[vsi_idx][tun] = entry1_h;
+ }
+
return 0;
+err_unroll:
+ entry1_h = 0;
+ hw_prof->fdir_seg[tun] = NULL;
+
+ /* The variable del_last will be used to determine when to clean up
+ * the VSI group data. The VSI data is not needed if there are no
+ * segments.
+ */
+ del_last = true;
+ for (idx = 0; idx < ICE_FD_HW_SEG_MAX; idx++)
+ if (hw_prof->fdir_seg[idx]) {
+ del_last = false;
+ break;
+ }
+
+ for (idx = 0; idx < hw_prof->cnt; idx++) {
+ u16 vsi_num = ice_get_hw_vsi_num(hw, hw_prof->vsi_h[idx]);
+
+ if (!hw_prof->entry_h[idx][tun])
+ continue;
+ ice_rem_prof_id_flow(hw, ICE_BLK_FD, vsi_num, prof_id);
+ ice_flow_rem_entry(hw, ICE_BLK_FD, hw_prof->entry_h[idx][tun]);
+ hw_prof->entry_h[idx][tun] = 0;
+ if (del_last)
+ hw_prof->vsi_h[idx] = 0;
+ }
+ if (del_last)
+ hw_prof->cnt = 0;
err_entry:
ice_rem_prof_id_flow(hw, ICE_BLK_FD,
ice_get_hw_vsi_num(hw, main_vsi->idx), prof_id);
@@ -1174,6 +1320,31 @@ ice_cfg_fdir_xtrct_seq(struct ice_pf *pf, struct ethtool_rx_flow_spec *fsp,
}
/**
+ * ice_update_per_q_fltr
+ * @vsi: ptr to VSI
+ * @q_index: queue index
+ * @inc: true to increment or false to decrement per queue filter count
+ *
+ * This function is used to keep track of per queue sideband filters
+ */
+static void ice_update_per_q_fltr(struct ice_vsi *vsi, u32 q_index, bool inc)
+{
+ struct ice_rx_ring *rx_ring;
+
+ if (!vsi->num_rxq || q_index >= vsi->num_rxq)
+ return;
+
+ rx_ring = vsi->rx_rings[q_index];
+ if (!rx_ring || !rx_ring->ch)
+ return;
+
+ if (inc)
+ atomic_inc(&rx_ring->ch->num_sb_fltr);
+ else
+ atomic_dec_if_positive(&rx_ring->ch->num_sb_fltr);
+}
+
+/**
* ice_fdir_write_fltr - send a flow director filter to the hardware
* @pf: PF data structure
* @input: filter structure
@@ -1190,7 +1361,6 @@ ice_fdir_write_fltr(struct ice_pf *pf, struct ice_fdir_fltr *input, bool add,
struct ice_hw *hw = &pf->hw;
struct ice_fltr_desc desc;
struct ice_vsi *ctrl_vsi;
- enum ice_status status;
u8 *pkt, *frag_pkt;
bool has_frag;
int err;
@@ -1209,11 +1379,9 @@ ice_fdir_write_fltr(struct ice_pf *pf, struct ice_fdir_fltr *input, bool add,
}
ice_fdir_get_prgm_desc(hw, input, &desc, add);
- status = ice_fdir_get_gen_prgm_pkt(hw, input, pkt, false, is_tun);
- if (status) {
- err = ice_status_to_errno(status);
+ err = ice_fdir_get_gen_prgm_pkt(hw, input, pkt, false, is_tun);
+ if (err)
goto err_free_all;
- }
err = ice_prgm_fdir_fltr(ctrl_vsi, &desc, pkt);
if (err)
goto err_free_all;
@@ -1223,12 +1391,10 @@ ice_fdir_write_fltr(struct ice_pf *pf, struct ice_fdir_fltr *input, bool add,
if (has_frag) {
/* does not return error */
ice_fdir_get_prgm_desc(hw, input, &desc, add);
- status = ice_fdir_get_gen_prgm_pkt(hw, input, frag_pkt, true,
- is_tun);
- if (status) {
- err = ice_status_to_errno(status);
+ err = ice_fdir_get_gen_prgm_pkt(hw, input, frag_pkt, true,
+ is_tun);
+ if (err)
goto err_frag;
- }
err = ice_prgm_fdir_fltr(ctrl_vsi, &desc, frag_pkt);
if (err)
goto err_frag;
@@ -1324,13 +1490,32 @@ int ice_fdir_create_dflt_rules(struct ice_pf *pf)
}
/**
+ * ice_fdir_del_all_fltrs - Delete all flow director filters
+ * @vsi: the VSI being changed
+ *
+ * This function needs to be called while holding hw->fdir_fltr_lock
+ */
+void ice_fdir_del_all_fltrs(struct ice_vsi *vsi)
+{
+ struct ice_fdir_fltr *f_rule, *tmp;
+ struct ice_pf *pf = vsi->back;
+ struct ice_hw *hw = &pf->hw;
+
+ list_for_each_entry_safe(f_rule, tmp, &hw->fdir_list_head, fltr_node) {
+ ice_fdir_write_all_fltr(pf, f_rule, false);
+ ice_fdir_update_cntrs(hw, f_rule->flow_type, false);
+ list_del(&f_rule->fltr_node);
+ devm_kfree(ice_pf_to_dev(pf), f_rule);
+ }
+}
+
+/**
* ice_vsi_manage_fdir - turn on/off flow director
* @vsi: the VSI being changed
* @ena: boolean value indicating if this is an enable or disable request
*/
void ice_vsi_manage_fdir(struct ice_vsi *vsi, bool ena)
{
- struct ice_fdir_fltr *f_rule, *tmp;
struct ice_pf *pf = vsi->back;
struct ice_hw *hw = &pf->hw;
enum ice_fltr_ptype flow;
@@ -1344,13 +1529,8 @@ void ice_vsi_manage_fdir(struct ice_vsi *vsi, bool ena)
mutex_lock(&hw->fdir_fltr_lock);
if (!test_and_clear_bit(ICE_FLAG_FD_ENA, pf->flags))
goto release_lock;
- list_for_each_entry_safe(f_rule, tmp, &hw->fdir_list_head, fltr_node) {
- /* ignore return value */
- ice_fdir_write_all_fltr(pf, f_rule, false);
- ice_fdir_update_cntrs(hw, f_rule->flow_type, false);
- list_del(&f_rule->fltr_node);
- devm_kfree(ice_hw_to_dev(hw), f_rule);
- }
+
+ ice_fdir_del_all_fltrs(vsi);
if (hw->fdir_prof)
for (flow = ICE_FLTR_PTYPE_NONF_NONE; flow < ICE_FLTR_PTYPE_MAX;
@@ -1401,18 +1581,25 @@ ice_fdir_update_list_entry(struct ice_pf *pf, struct ice_fdir_fltr *input,
{
struct ice_fdir_fltr *old_fltr;
struct ice_hw *hw = &pf->hw;
+ struct ice_vsi *vsi;
int err = -ENOENT;
/* Do not update filters during reset */
if (ice_is_reset_in_progress(pf->state))
return -EBUSY;
+ vsi = ice_get_main_vsi(pf);
+ if (!vsi)
+ return -EINVAL;
+
old_fltr = ice_fdir_find_fltr_by_idx(hw, fltr_idx);
if (old_fltr) {
err = ice_fdir_write_all_fltr(pf, old_fltr, false);
if (err)
return err;
ice_fdir_update_cntrs(hw, old_fltr->flow_type, false);
+ /* update sb-filters count, specific to ring->channel */
+ ice_update_per_q_fltr(vsi, old_fltr->orig_q_index, false);
if (!input && !hw->fdir_fltr_cnt[old_fltr->flow_type])
/* we just deleted the last filter of flow_type so we
* should also delete the HW filter info.
@@ -1424,6 +1611,8 @@ ice_fdir_update_list_entry(struct ice_pf *pf, struct ice_fdir_fltr *input,
if (!input)
return err;
ice_fdir_list_add_fltr(hw, input);
+ /* update sb-filters count, specific to ring->channel */
+ ice_update_per_q_fltr(vsi, input->orig_q_index, true);
ice_fdir_update_cntrs(hw, input->flow_type, true);
return 0;
}
@@ -1463,6 +1652,39 @@ int ice_del_fdir_ethtool(struct ice_vsi *vsi, struct ethtool_rxnfc *cmd)
}
/**
+ * ice_update_ring_dest_vsi - update dest ring and dest VSI
+ * @vsi: pointer to target VSI
+ * @dest_vsi: ptr to dest VSI index
+ * @ring: ptr to dest ring
+ *
+ * This function updates destination VSI and queue if user specifies
+ * target queue which falls in channel's (aka ADQ) queue region
+ */
+static void
+ice_update_ring_dest_vsi(struct ice_vsi *vsi, u16 *dest_vsi, u32 *ring)
+{
+ struct ice_channel *ch;
+
+ list_for_each_entry(ch, &vsi->ch_list, list) {
+ if (!ch->ch_vsi)
+ continue;
+
+ /* make sure to locate corresponding channel based on "queue"
+ * specified
+ */
+ if ((*ring < ch->base_q) ||
+ (*ring >= (ch->base_q + ch->num_rxq)))
+ continue;
+
+ /* update the dest_vsi based on channel */
+ *dest_vsi = ch->ch_vsi->idx;
+
+ /* update the "ring" to be correct based on channel */
+ *ring -= ch->base_q;
+ }
+}
+
+/**
* ice_set_fdir_input_set - Set the input set for Flow Director
* @vsi: pointer to target VSI
* @fsp: pointer to ethtool Rx flow specification
@@ -1473,6 +1695,7 @@ ice_set_fdir_input_set(struct ice_vsi *vsi, struct ethtool_rx_flow_spec *fsp,
struct ice_fdir_fltr *input)
{
u16 dest_vsi, q_index = 0;
+ u16 orig_q_index = 0;
struct ice_pf *pf;
struct ice_hw *hw;
int flow_type;
@@ -1499,6 +1722,8 @@ ice_set_fdir_input_set(struct ice_vsi *vsi, struct ethtool_rx_flow_spec *fsp,
if (ring >= vsi->num_rxq)
return -EINVAL;
+ orig_q_index = ring;
+ ice_update_ring_dest_vsi(vsi, &dest_vsi, &ring);
dest_ctl = ICE_FLTR_PRGM_DESC_DEST_DIRECT_PKT_QINDEX;
q_index = ring;
}
@@ -1507,6 +1732,11 @@ ice_set_fdir_input_set(struct ice_vsi *vsi, struct ethtool_rx_flow_spec *fsp,
input->q_index = q_index;
flow_type = fsp->flow_type & ~FLOW_EXT;
+ /* Record the original queue index as specified by user.
+ * with channel configuration 'q_index' becomes relative
+ * to TC (channel).
+ */
+ input->orig_q_index = orig_q_index;
input->dest_vsi = dest_vsi;
input->dest_ctl = dest_ctl;
input->fltr_status = ICE_FLTR_PRGM_DESC_FD_STATUS_FD_ID;
@@ -1694,6 +1924,8 @@ int ice_add_fdir_ethtool(struct ice_vsi *vsi, struct ethtool_rxnfc *cmd)
remove_sw_rule:
ice_fdir_update_cntrs(hw, input->flow_type, false);
+ /* update sb-filters count, specific to ring->channel */
+ ice_update_per_q_fltr(vsi, input->orig_q_index, false);
list_del(&input->fltr_node);
release_lock:
mutex_unlock(&hw->fdir_fltr_lock);
diff --git a/drivers/net/ethernet/intel/ice/ice_fdir.c b/drivers/net/ethernet/intel/ice/ice_fdir.c
index 4dca009..ae089d3 100644
--- a/drivers/net/ethernet/intel/ice/ice_fdir.c
+++ b/drivers/net/ethernet/intel/ice/ice_fdir.c
@@ -712,7 +712,7 @@ ice_fdir_get_prgm_desc(struct ice_hw *hw, struct ice_fdir_fltr *input,
* @hw: pointer to the hardware structure
* @cntr_id: returns counter index
*/
-enum ice_status ice_alloc_fd_res_cntr(struct ice_hw *hw, u16 *cntr_id)
+int ice_alloc_fd_res_cntr(struct ice_hw *hw, u16 *cntr_id)
{
return ice_alloc_res_cntr(hw, ICE_AQC_RES_TYPE_FDIR_COUNTER_BLOCK,
ICE_AQC_RES_TYPE_FLAG_DEDICATED, 1, cntr_id);
@@ -723,7 +723,7 @@ enum ice_status ice_alloc_fd_res_cntr(struct ice_hw *hw, u16 *cntr_id)
* @hw: pointer to the hardware structure
* @cntr_id: counter index to be freed
*/
-enum ice_status ice_free_fd_res_cntr(struct ice_hw *hw, u16 cntr_id)
+int ice_free_fd_res_cntr(struct ice_hw *hw, u16 cntr_id)
{
return ice_free_res_cntr(hw, ICE_AQC_RES_TYPE_FDIR_COUNTER_BLOCK,
ICE_AQC_RES_TYPE_FLAG_DEDICATED, 1, cntr_id);
@@ -735,8 +735,7 @@ enum ice_status ice_free_fd_res_cntr(struct ice_hw *hw, u16 cntr_id)
* @cntr_id: returns counter index
* @num_fltr: number of filter entries to be allocated
*/
-enum ice_status
-ice_alloc_fd_guar_item(struct ice_hw *hw, u16 *cntr_id, u16 num_fltr)
+int ice_alloc_fd_guar_item(struct ice_hw *hw, u16 *cntr_id, u16 num_fltr)
{
return ice_alloc_res_cntr(hw, ICE_AQC_RES_TYPE_FDIR_GUARANTEED_ENTRIES,
ICE_AQC_RES_TYPE_FLAG_DEDICATED, num_fltr,
@@ -749,8 +748,7 @@ ice_alloc_fd_guar_item(struct ice_hw *hw, u16 *cntr_id, u16 num_fltr)
* @cntr_id: returns counter index
* @num_fltr: number of filter entries to be allocated
*/
-enum ice_status
-ice_alloc_fd_shrd_item(struct ice_hw *hw, u16 *cntr_id, u16 num_fltr)
+int ice_alloc_fd_shrd_item(struct ice_hw *hw, u16 *cntr_id, u16 num_fltr)
{
return ice_alloc_res_cntr(hw, ICE_AQC_RES_TYPE_FDIR_SHARED_ENTRIES,
ICE_AQC_RES_TYPE_FLAG_DEDICATED, num_fltr,
@@ -872,7 +870,7 @@ static void ice_pkt_insert_mac_addr(u8 *pkt, u8 *addr)
* @frag: generate a fragment packet
* @tun: true implies generate a tunnel packet
*/
-enum ice_status
+int
ice_fdir_get_gen_prgm_pkt(struct ice_hw *hw, struct ice_fdir_fltr *input,
u8 *pkt, bool frag, bool tun)
{
@@ -919,15 +917,15 @@ ice_fdir_get_gen_prgm_pkt(struct ice_hw *hw, struct ice_fdir_fltr *input,
if (ice_fdir_pkt[idx].flow == flow)
break;
if (idx == ICE_FDIR_NUM_PKT)
- return ICE_ERR_PARAM;
+ return -EINVAL;
if (!tun) {
memcpy(pkt, ice_fdir_pkt[idx].pkt, ice_fdir_pkt[idx].pkt_len);
loc = pkt;
} else {
if (!ice_get_open_tunnel_port(hw, &tnl_port, TNL_ALL))
- return ICE_ERR_DOES_NOT_EXIST;
+ return -ENOENT;
if (!ice_fdir_pkt[idx].tun_pkt)
- return ICE_ERR_PARAM;
+ return -EINVAL;
memcpy(pkt, ice_fdir_pkt[idx].tun_pkt,
ice_fdir_pkt[idx].tun_pkt_len);
ice_pkt_insert_u16(pkt, ICE_IPV4_UDP_DST_PORT_OFFSET,
@@ -1111,7 +1109,7 @@ ice_fdir_get_gen_prgm_pkt(struct ice_hw *hw, struct ice_fdir_fltr *input,
ice_pkt_insert_mac_addr(loc, input->ext_data.dst_mac);
break;
default:
- return ICE_ERR_PARAM;
+ return -EINVAL;
}
if (input->flex_fltr)
diff --git a/drivers/net/ethernet/intel/ice/ice_fdir.h b/drivers/net/ethernet/intel/ice/ice_fdir.h
index da41638..1b9b844 100644
--- a/drivers/net/ethernet/intel/ice/ice_fdir.h
+++ b/drivers/net/ethernet/intel/ice/ice_fdir.h
@@ -182,6 +182,7 @@ struct ice_fdir_fltr {
/* filter control */
u16 q_index;
+ u16 orig_q_index;
u16 dest_vsi;
u8 dest_ctl;
u8 cnt_ena;
@@ -201,16 +202,14 @@ struct ice_fdir_base_pkt {
const u8 *tun_pkt;
};
-enum ice_status ice_alloc_fd_res_cntr(struct ice_hw *hw, u16 *cntr_id);
-enum ice_status ice_free_fd_res_cntr(struct ice_hw *hw, u16 cntr_id);
-enum ice_status
-ice_alloc_fd_guar_item(struct ice_hw *hw, u16 *cntr_id, u16 num_fltr);
-enum ice_status
-ice_alloc_fd_shrd_item(struct ice_hw *hw, u16 *cntr_id, u16 num_fltr);
+int ice_alloc_fd_res_cntr(struct ice_hw *hw, u16 *cntr_id);
+int ice_free_fd_res_cntr(struct ice_hw *hw, u16 cntr_id);
+int ice_alloc_fd_guar_item(struct ice_hw *hw, u16 *cntr_id, u16 num_fltr);
+int ice_alloc_fd_shrd_item(struct ice_hw *hw, u16 *cntr_id, u16 num_fltr);
void
ice_fdir_get_prgm_desc(struct ice_hw *hw, struct ice_fdir_fltr *input,
struct ice_fltr_desc *fdesc, bool add);
-enum ice_status
+int
ice_fdir_get_gen_prgm_pkt(struct ice_hw *hw, struct ice_fdir_fltr *input,
u8 *pkt, bool frag, bool tun);
int ice_get_fdir_cnt_all(struct ice_hw *hw);
diff --git a/drivers/net/ethernet/intel/ice/ice_flex_pipe.c b/drivers/net/ethernet/intel/ice/ice_flex_pipe.c
index 6ad1c25..4deb2c9 100644
--- a/drivers/net/ethernet/intel/ice/ice_flex_pipe.c
+++ b/drivers/net/ethernet/intel/ice/ice_flex_pipe.c
@@ -314,6 +314,78 @@ ice_pkg_enum_entry(struct ice_seg *ice_seg, struct ice_pkg_enum *state,
}
/**
+ * ice_hw_ptype_ena - check if the PTYPE is enabled or not
+ * @hw: pointer to the HW structure
+ * @ptype: the hardware PTYPE
+ */
+bool ice_hw_ptype_ena(struct ice_hw *hw, u16 ptype)
+{
+ return ptype < ICE_FLOW_PTYPE_MAX &&
+ test_bit(ptype, hw->hw_ptype);
+}
+
+/**
+ * ice_marker_ptype_tcam_handler
+ * @sect_type: section type
+ * @section: pointer to section
+ * @index: index of the Marker PType TCAM entry to be returned
+ * @offset: pointer to receive absolute offset, always 0 for ptype TCAM sections
+ *
+ * This is a callback function that can be passed to ice_pkg_enum_entry.
+ * Handles enumeration of individual Marker PType TCAM entries.
+ */
+static void *
+ice_marker_ptype_tcam_handler(u32 sect_type, void *section, u32 index,
+ u32 *offset)
+{
+ struct ice_marker_ptype_tcam_section *marker_ptype;
+
+ if (sect_type != ICE_SID_RXPARSER_MARKER_PTYPE)
+ return NULL;
+
+ if (index > ICE_MAX_MARKER_PTYPE_TCAMS_IN_BUF)
+ return NULL;
+
+ if (offset)
+ *offset = 0;
+
+ marker_ptype = section;
+ if (index >= le16_to_cpu(marker_ptype->count))
+ return NULL;
+
+ return marker_ptype->tcam + index;
+}
+
+/**
+ * ice_fill_hw_ptype - fill the enabled PTYPE bit information
+ * @hw: pointer to the HW structure
+ */
+static void ice_fill_hw_ptype(struct ice_hw *hw)
+{
+ struct ice_marker_ptype_tcam_entry *tcam;
+ struct ice_seg *seg = hw->seg;
+ struct ice_pkg_enum state;
+
+ bitmap_zero(hw->hw_ptype, ICE_FLOW_PTYPE_MAX);
+ if (!seg)
+ return;
+
+ memset(&state, 0, sizeof(state));
+
+ do {
+ tcam = ice_pkg_enum_entry(seg, &state,
+ ICE_SID_RXPARSER_MARKER_PTYPE, NULL,
+ ice_marker_ptype_tcam_handler);
+ if (tcam &&
+ le16_to_cpu(tcam->addr) < ICE_MARKER_PTYPE_TCAM_ADDR_MAX &&
+ le16_to_cpu(tcam->ptype) < ICE_FLOW_PTYPE_MAX)
+ set_bit(le16_to_cpu(tcam->ptype), hw->hw_ptype);
+
+ seg = NULL;
+ } while (tcam);
+}
+
+/**
* ice_boost_tcam_handler
* @sect_type: section type
* @section: pointer to section
@@ -358,7 +430,7 @@ ice_boost_tcam_handler(u32 sect_type, void *section, u32 index, u32 *offset)
* if it is found. The ice_seg parameter must not be NULL since the first call
* to ice_pkg_enum_entry requires a pointer to an actual ice_segment structure.
*/
-static enum ice_status
+static int
ice_find_boost_entry(struct ice_seg *ice_seg, u16 addr,
struct ice_boost_tcam_entry **entry)
{
@@ -368,7 +440,7 @@ ice_find_boost_entry(struct ice_seg *ice_seg, u16 addr,
memset(&state, 0, sizeof(state));
if (!ice_seg)
- return ICE_ERR_PARAM;
+ return -EINVAL;
do {
tcam = ice_pkg_enum_entry(ice_seg, &state,
@@ -383,7 +455,7 @@ ice_find_boost_entry(struct ice_seg *ice_seg, u16 addr,
} while (tcam);
*entry = NULL;
- return ICE_ERR_CFG;
+ return -EIO;
}
/**
@@ -549,7 +621,7 @@ static void ice_init_pkg_hints(struct ice_hw *hw, struct ice_seg *ice_seg)
* ------------------------------
* Result: key: b01 10 11 11 00 00
*/
-static enum ice_status
+static int
ice_gen_key_word(u8 val, u8 valid, u8 dont_care, u8 nvr_mtch, u8 *key,
u8 *key_inv)
{
@@ -558,7 +630,7 @@ ice_gen_key_word(u8 val, u8 valid, u8 dont_care, u8 nvr_mtch, u8 *key,
/* 'dont_care' and 'nvr_mtch' masks cannot overlap */
if ((dont_care ^ nvr_mtch) != (dont_care | nvr_mtch))
- return ICE_ERR_CFG;
+ return -EIO;
*key = 0;
*key_inv = 0;
@@ -651,7 +723,7 @@ static bool ice_bits_max_set(const u8 *mask, u16 size, u16 max)
* dc == NULL --> dc mask is all 0's (no don't care bits)
* nm == NULL --> nm mask is all 0's (no never match bits)
*/
-static enum ice_status
+static int
ice_set_key(u8 *key, u16 size, u8 *val, u8 *upd, u8 *dc, u8 *nm, u16 off,
u16 len)
{
@@ -660,11 +732,11 @@ ice_set_key(u8 *key, u16 size, u8 *val, u8 *upd, u8 *dc, u8 *nm, u16 off,
/* size must be a multiple of 2 bytes. */
if (size % 2)
- return ICE_ERR_CFG;
+ return -EIO;
half_size = size / 2;
if (off + len > half_size)
- return ICE_ERR_CFG;
+ return -EIO;
/* Make sure at most one bit is set in the never match mask. Having more
* than one never match mask bit set will cause HW to consume excessive
@@ -672,13 +744,13 @@ ice_set_key(u8 *key, u16 size, u8 *val, u8 *upd, u8 *dc, u8 *nm, u16 off,
*/
#define ICE_NVR_MTCH_BITS_MAX 1
if (nm && !ice_bits_max_set(nm, len, ICE_NVR_MTCH_BITS_MAX))
- return ICE_ERR_CFG;
+ return -EIO;
for (i = 0; i < len; i++)
if (ice_gen_key_word(val[i], upd ? upd[i] : 0xff,
dc ? dc[i] : 0, nm ? nm[i] : 0,
key + off + i, key + half_size + off + i))
- return ICE_ERR_CFG;
+ return -EIO;
return 0;
}
@@ -692,25 +764,25 @@ ice_set_key(u8 *key, u16 size, u8 *val, u8 *upd, u8 *dc, u8 *nm, u16 off,
* or writing of the package. When attempting to obtain write access, the
* caller must check for the following two return values:
*
- * ICE_SUCCESS - Means the caller has acquired the global config lock
- * and can perform writing of the package.
- * ICE_ERR_AQ_NO_WORK - Indicates another driver has already written the
- * package or has found that no update was necessary; in
- * this case, the caller can just skip performing any
- * update of the package.
+ * 0 - Means the caller has acquired the global config lock
+ * and can perform writing of the package.
+ * -EALREADY - Indicates another driver has already written the
+ * package or has found that no update was necessary; in
+ * this case, the caller can just skip performing any
+ * update of the package.
*/
-static enum ice_status
+static int
ice_acquire_global_cfg_lock(struct ice_hw *hw,
enum ice_aq_res_access_type access)
{
- enum ice_status status;
+ int status;
status = ice_acquire_res(hw, ICE_GLOBAL_CFG_LOCK_RES_ID, access,
ICE_GLOBAL_CFG_LOCK_TIMEOUT);
if (!status)
mutex_lock(&ice_global_cfg_lock_sw);
- else if (status == ICE_ERR_AQ_NO_WORK)
+ else if (status == -EALREADY)
ice_debug(hw, ICE_DBG_PKG, "Global config lock: No work to do\n");
return status;
@@ -735,7 +807,7 @@ static void ice_release_global_cfg_lock(struct ice_hw *hw)
*
* This function will request ownership of the change lock.
*/
-enum ice_status
+int
ice_acquire_change_lock(struct ice_hw *hw, enum ice_aq_res_access_type access)
{
return ice_acquire_res(hw, ICE_CHANGE_LOCK_RES_ID, access,
@@ -765,14 +837,14 @@ void ice_release_change_lock(struct ice_hw *hw)
*
* Download Package (0x0C40)
*/
-static enum ice_status
+static int
ice_aq_download_pkg(struct ice_hw *hw, struct ice_buf_hdr *pkg_buf,
u16 buf_size, bool last_buf, u32 *error_offset,
u32 *error_info, struct ice_sq_cd *cd)
{
struct ice_aqc_download_pkg *cmd;
struct ice_aq_desc desc;
- enum ice_status status;
+ int status;
if (error_offset)
*error_offset = 0;
@@ -787,7 +859,7 @@ ice_aq_download_pkg(struct ice_hw *hw, struct ice_buf_hdr *pkg_buf,
cmd->flags |= ICE_AQC_DOWNLOAD_PKG_LAST_BUF;
status = ice_aq_send_cmd(hw, &desc, pkg_buf, buf_size, cd);
- if (status == ICE_ERR_AQ_ERROR) {
+ if (status == -EIO) {
/* Read error from buffer only when the FW returned an error */
struct ice_aqc_download_pkg_resp *resp;
@@ -813,14 +885,14 @@ ice_aq_download_pkg(struct ice_hw *hw, struct ice_buf_hdr *pkg_buf,
*
* Update Package (0x0C42)
*/
-static enum ice_status
+static int
ice_aq_update_pkg(struct ice_hw *hw, struct ice_buf_hdr *pkg_buf, u16 buf_size,
bool last_buf, u32 *error_offset, u32 *error_info,
struct ice_sq_cd *cd)
{
struct ice_aqc_download_pkg *cmd;
struct ice_aq_desc desc;
- enum ice_status status;
+ int status;
if (error_offset)
*error_offset = 0;
@@ -835,7 +907,7 @@ ice_aq_update_pkg(struct ice_hw *hw, struct ice_buf_hdr *pkg_buf, u16 buf_size,
cmd->flags |= ICE_AQC_DOWNLOAD_PKG_LAST_BUF;
status = ice_aq_send_cmd(hw, &desc, pkg_buf, buf_size, cd);
- if (status == ICE_ERR_AQ_ERROR) {
+ if (status == -EIO) {
/* Read error from buffer only when the FW returned an error */
struct ice_aqc_download_pkg_resp *resp;
@@ -892,11 +964,10 @@ ice_find_seg_in_pkg(struct ice_hw *hw, u32 seg_type,
*
* Obtains change lock and updates package.
*/
-static enum ice_status
-ice_update_pkg(struct ice_hw *hw, struct ice_buf *bufs, u32 count)
+static int ice_update_pkg(struct ice_hw *hw, struct ice_buf *bufs, u32 count)
{
- enum ice_status status;
u32 offset, info, i;
+ int status;
status = ice_acquire_change_lock(hw, ICE_RES_WRITE);
if (status)
@@ -921,6 +992,22 @@ ice_update_pkg(struct ice_hw *hw, struct ice_buf *bufs, u32 count)
return status;
}
+static enum ice_ddp_state ice_map_aq_err_to_ddp_state(enum ice_aq_err aq_err)
+{
+ switch (aq_err) {
+ case ICE_AQ_RC_ENOSEC:
+ case ICE_AQ_RC_EBADSIG:
+ return ICE_DDP_PKG_FILE_SIGNATURE_INVALID;
+ case ICE_AQ_RC_ESVN:
+ return ICE_DDP_PKG_FILE_REVISION_TOO_LOW;
+ case ICE_AQ_RC_EBADMAN:
+ case ICE_AQ_RC_EBADBUF:
+ return ICE_DDP_PKG_LOAD_ERROR;
+ default:
+ return ICE_DDP_PKG_ERR;
+ }
+}
+
/**
* ice_dwnld_cfg_bufs
* @hw: pointer to the hardware structure
@@ -931,15 +1018,17 @@ ice_update_pkg(struct ice_hw *hw, struct ice_buf *bufs, u32 count)
* to the firmware. Metadata buffers are skipped, and the first metadata buffer
* found indicates that the rest of the buffers are all metadata buffers.
*/
-static enum ice_status
+static enum ice_ddp_state
ice_dwnld_cfg_bufs(struct ice_hw *hw, struct ice_buf *bufs, u32 count)
{
- enum ice_status status;
+ enum ice_ddp_state state = ICE_DDP_PKG_SUCCESS;
struct ice_buf_hdr *bh;
+ enum ice_aq_err err;
u32 offset, info, i;
+ int status;
if (!bufs || !count)
- return ICE_ERR_PARAM;
+ return ICE_DDP_PKG_ERR;
/* If the first buffer's first section has its metadata bit set
* then there are no buffers to be downloaded, and the operation is
@@ -947,20 +1036,13 @@ ice_dwnld_cfg_bufs(struct ice_hw *hw, struct ice_buf *bufs, u32 count)
*/
bh = (struct ice_buf_hdr *)bufs;
if (le32_to_cpu(bh->section_entry[0].type) & ICE_METADATA_BUF)
- return 0;
-
- /* reset pkg_dwnld_status in case this function is called in the
- * reset/rebuild flow
- */
- hw->pkg_dwnld_status = ICE_AQ_RC_OK;
+ return ICE_DDP_PKG_SUCCESS;
status = ice_acquire_global_cfg_lock(hw, ICE_RES_WRITE);
if (status) {
- if (status == ICE_ERR_AQ_NO_WORK)
- hw->pkg_dwnld_status = ICE_AQ_RC_EEXIST;
- else
- hw->pkg_dwnld_status = hw->adminq.sq_last_status;
- return status;
+ if (status == -EALREADY)
+ return ICE_DDP_PKG_ALREADY_LOADED;
+ return ice_map_aq_err_to_ddp_state(hw->adminq.sq_last_status);
}
for (i = 0; i < count; i++) {
@@ -986,11 +1068,11 @@ ice_dwnld_cfg_bufs(struct ice_hw *hw, struct ice_buf *bufs, u32 count)
&offset, &info, NULL);
/* Save AQ status from download package */
- hw->pkg_dwnld_status = hw->adminq.sq_last_status;
if (status) {
ice_debug(hw, ICE_DBG_PKG, "Pkg download failed: err %d off %d inf %d\n",
status, offset, info);
-
+ err = hw->adminq.sq_last_status;
+ state = ice_map_aq_err_to_ddp_state(err);
break;
}
@@ -1000,7 +1082,7 @@ ice_dwnld_cfg_bufs(struct ice_hw *hw, struct ice_buf *bufs, u32 count)
ice_release_global_cfg_lock(hw);
- return status;
+ return state;
}
/**
@@ -1012,7 +1094,7 @@ ice_dwnld_cfg_bufs(struct ice_hw *hw, struct ice_buf *bufs, u32 count)
*
* Get Package Info List (0x0C43)
*/
-static enum ice_status
+static int
ice_aq_get_pkg_info_list(struct ice_hw *hw,
struct ice_aqc_get_pkg_info_resp *pkg_info,
u16 buf_size, struct ice_sq_cd *cd)
@@ -1031,7 +1113,7 @@ ice_aq_get_pkg_info_list(struct ice_hw *hw,
*
* Handles the download of a complete package.
*/
-static enum ice_status
+static enum ice_ddp_state
ice_download_pkg(struct ice_hw *hw, struct ice_seg *ice_seg)
{
struct ice_buf_table *ice_buf_tbl;
@@ -1062,13 +1144,13 @@ ice_download_pkg(struct ice_hw *hw, struct ice_seg *ice_seg)
*
* Saves off the package details into the HW structure.
*/
-static enum ice_status
+static enum ice_ddp_state
ice_init_pkg_info(struct ice_hw *hw, struct ice_pkg_hdr *pkg_hdr)
{
struct ice_generic_seg_hdr *seg_hdr;
if (!pkg_hdr)
- return ICE_ERR_PARAM;
+ return ICE_DDP_PKG_ERR;
seg_hdr = ice_find_seg_in_pkg(hw, SEGMENT_TYPE_ICE, pkg_hdr);
if (seg_hdr) {
@@ -1082,7 +1164,7 @@ ice_init_pkg_info(struct ice_hw *hw, struct ice_pkg_hdr *pkg_hdr)
ICE_SID_METADATA);
if (!meta) {
ice_debug(hw, ICE_DBG_INIT, "Did not find ice metadata section in package\n");
- return ICE_ERR_CFG;
+ return ICE_DDP_PKG_INVALID_FILE;
}
hw->pkg_ver = meta->ver;
@@ -1104,10 +1186,10 @@ ice_init_pkg_info(struct ice_hw *hw, struct ice_pkg_hdr *pkg_hdr)
seg_hdr->seg_id);
} else {
ice_debug(hw, ICE_DBG_INIT, "Did not find ice segment in driver package\n");
- return ICE_ERR_CFG;
+ return ICE_DDP_PKG_INVALID_FILE;
}
- return 0;
+ return ICE_DDP_PKG_SUCCESS;
}
/**
@@ -1116,21 +1198,22 @@ ice_init_pkg_info(struct ice_hw *hw, struct ice_pkg_hdr *pkg_hdr)
*
* Store details of the package currently loaded in HW into the HW structure.
*/
-static enum ice_status ice_get_pkg_info(struct ice_hw *hw)
+static enum ice_ddp_state ice_get_pkg_info(struct ice_hw *hw)
{
+ enum ice_ddp_state state = ICE_DDP_PKG_SUCCESS;
struct ice_aqc_get_pkg_info_resp *pkg_info;
- enum ice_status status;
u16 size;
u32 i;
size = struct_size(pkg_info, pkg_info, ICE_PKG_CNT);
pkg_info = kzalloc(size, GFP_KERNEL);
if (!pkg_info)
- return ICE_ERR_NO_MEMORY;
+ return ICE_DDP_PKG_ERR;
- status = ice_aq_get_pkg_info_list(hw, pkg_info, size, NULL);
- if (status)
+ if (ice_aq_get_pkg_info_list(hw, pkg_info, size, NULL)) {
+ state = ICE_DDP_PKG_ERR;
goto init_pkg_free_alloc;
+ }
for (i = 0; i < le32_to_cpu(pkg_info->count); i++) {
#define ICE_PKG_FLAG_COUNT 4
@@ -1165,7 +1248,7 @@ static enum ice_status ice_get_pkg_info(struct ice_hw *hw)
init_pkg_free_alloc:
kfree(pkg_info);
- return status;
+ return state;
}
/**
@@ -1176,28 +1259,28 @@ static enum ice_status ice_get_pkg_info(struct ice_hw *hw)
* Verifies various attributes of the package file, including length, format
* version, and the requirement of at least one segment.
*/
-static enum ice_status ice_verify_pkg(struct ice_pkg_hdr *pkg, u32 len)
+static enum ice_ddp_state ice_verify_pkg(struct ice_pkg_hdr *pkg, u32 len)
{
u32 seg_count;
u32 i;
if (len < struct_size(pkg, seg_offset, 1))
- return ICE_ERR_BUF_TOO_SHORT;
+ return ICE_DDP_PKG_INVALID_FILE;
if (pkg->pkg_format_ver.major != ICE_PKG_FMT_VER_MAJ ||
pkg->pkg_format_ver.minor != ICE_PKG_FMT_VER_MNR ||
pkg->pkg_format_ver.update != ICE_PKG_FMT_VER_UPD ||
pkg->pkg_format_ver.draft != ICE_PKG_FMT_VER_DFT)
- return ICE_ERR_CFG;
+ return ICE_DDP_PKG_INVALID_FILE;
/* pkg must have at least one segment */
seg_count = le32_to_cpu(pkg->seg_count);
if (seg_count < 1)
- return ICE_ERR_CFG;
+ return ICE_DDP_PKG_INVALID_FILE;
/* make sure segment array fits in package length */
if (len < struct_size(pkg, seg_offset, seg_count))
- return ICE_ERR_BUF_TOO_SHORT;
+ return ICE_DDP_PKG_INVALID_FILE;
/* all segments must fit within length */
for (i = 0; i < seg_count; i++) {
@@ -1206,16 +1289,16 @@ static enum ice_status ice_verify_pkg(struct ice_pkg_hdr *pkg, u32 len)
/* segment header must fit */
if (len < off + sizeof(*seg))
- return ICE_ERR_BUF_TOO_SHORT;
+ return ICE_DDP_PKG_INVALID_FILE;
seg = (struct ice_generic_seg_hdr *)((u8 *)pkg + off);
/* segment body must fit */
if (len < off + le32_to_cpu(seg->seg_size))
- return ICE_ERR_BUF_TOO_SHORT;
+ return ICE_DDP_PKG_INVALID_FILE;
}
- return 0;
+ return ICE_DDP_PKG_SUCCESS;
}
/**
@@ -1259,13 +1342,18 @@ static void ice_init_pkg_regs(struct ice_hw *hw)
* version must match our ICE_PKG_SUPP_VER_MAJ and ICE_PKG_SUPP_VER_MNR
* definitions.
*/
-static enum ice_status ice_chk_pkg_version(struct ice_pkg_ver *pkg_ver)
+static enum ice_ddp_state ice_chk_pkg_version(struct ice_pkg_ver *pkg_ver)
{
- if (pkg_ver->major != ICE_PKG_SUPP_VER_MAJ ||
- pkg_ver->minor != ICE_PKG_SUPP_VER_MNR)
- return ICE_ERR_NOT_SUPPORTED;
+ if (pkg_ver->major > ICE_PKG_SUPP_VER_MAJ ||
+ (pkg_ver->major == ICE_PKG_SUPP_VER_MAJ &&
+ pkg_ver->minor > ICE_PKG_SUPP_VER_MNR))
+ return ICE_DDP_PKG_FILE_VERSION_TOO_HIGH;
+ else if (pkg_ver->major < ICE_PKG_SUPP_VER_MAJ ||
+ (pkg_ver->major == ICE_PKG_SUPP_VER_MAJ &&
+ pkg_ver->minor < ICE_PKG_SUPP_VER_MNR))
+ return ICE_DDP_PKG_FILE_VERSION_TOO_LOW;
- return 0;
+ return ICE_DDP_PKG_SUCCESS;
}
/**
@@ -1276,20 +1364,20 @@ static enum ice_status ice_chk_pkg_version(struct ice_pkg_ver *pkg_ver)
*
* This function checks the package version compatibility with driver and NVM
*/
-static enum ice_status
+static enum ice_ddp_state
ice_chk_pkg_compat(struct ice_hw *hw, struct ice_pkg_hdr *ospkg,
struct ice_seg **seg)
{
struct ice_aqc_get_pkg_info_resp *pkg;
- enum ice_status status;
+ enum ice_ddp_state state;
u16 size;
u32 i;
/* Check package version compatibility */
- status = ice_chk_pkg_version(&hw->pkg_ver);
- if (status) {
+ state = ice_chk_pkg_version(&hw->pkg_ver);
+ if (state) {
ice_debug(hw, ICE_DBG_INIT, "Package version check failed.\n");
- return status;
+ return state;
}
/* find ICE segment in given package */
@@ -1297,18 +1385,19 @@ ice_chk_pkg_compat(struct ice_hw *hw, struct ice_pkg_hdr *ospkg,
ospkg);
if (!*seg) {
ice_debug(hw, ICE_DBG_INIT, "no ice segment in package.\n");
- return ICE_ERR_CFG;
+ return ICE_DDP_PKG_INVALID_FILE;
}
/* Check if FW is compatible with the OS package */
size = struct_size(pkg, pkg_info, ICE_PKG_CNT);
pkg = kzalloc(size, GFP_KERNEL);
if (!pkg)
- return ICE_ERR_NO_MEMORY;
+ return ICE_DDP_PKG_ERR;
- status = ice_aq_get_pkg_info_list(hw, pkg, size, NULL);
- if (status)
+ if (ice_aq_get_pkg_info_list(hw, pkg, size, NULL)) {
+ state = ICE_DDP_PKG_LOAD_ERROR;
goto fw_ddp_compat_free_alloc;
+ }
for (i = 0; i < le32_to_cpu(pkg->count); i++) {
/* loop till we find the NVM package */
@@ -1318,7 +1407,7 @@ ice_chk_pkg_compat(struct ice_hw *hw, struct ice_pkg_hdr *ospkg,
pkg->pkg_info[i].ver.major ||
(*seg)->hdr.seg_format_ver.minor >
pkg->pkg_info[i].ver.minor) {
- status = ICE_ERR_FW_DDP_MISMATCH;
+ state = ICE_DDP_PKG_FW_MISMATCH;
ice_debug(hw, ICE_DBG_INIT, "OS package is not compatible with NVM.\n");
}
/* done processing NVM package so break */
@@ -1326,7 +1415,7 @@ ice_chk_pkg_compat(struct ice_hw *hw, struct ice_pkg_hdr *ospkg,
}
fw_ddp_compat_free_alloc:
kfree(pkg);
- return status;
+ return state;
}
/**
@@ -1367,7 +1456,7 @@ ice_sw_fv_handler(u32 sect_type, void *section, u32 index, u32 *offset)
* and store the index number in struct ice_switch_info *switch_info
* in HW for following use.
*/
-static enum ice_status ice_get_prof_index_max(struct ice_hw *hw)
+static int ice_get_prof_index_max(struct ice_hw *hw)
{
u16 prof_index = 0, j, max_prof_index = 0;
struct ice_pkg_enum state;
@@ -1379,7 +1468,7 @@ static enum ice_status ice_get_prof_index_max(struct ice_hw *hw)
memset(&state, 0, sizeof(state));
if (!hw->seg)
- return ICE_ERR_PARAM;
+ return -EINVAL;
ice_seg = hw->seg;
@@ -1410,6 +1499,34 @@ static enum ice_status ice_get_prof_index_max(struct ice_hw *hw)
}
/**
+ * ice_get_ddp_pkg_state - get DDP pkg state after download
+ * @hw: pointer to the HW struct
+ * @already_loaded: indicates if pkg was already loaded onto the device
+ */
+static enum ice_ddp_state
+ice_get_ddp_pkg_state(struct ice_hw *hw, bool already_loaded)
+{
+ if (hw->pkg_ver.major == hw->active_pkg_ver.major &&
+ hw->pkg_ver.minor == hw->active_pkg_ver.minor &&
+ hw->pkg_ver.update == hw->active_pkg_ver.update &&
+ hw->pkg_ver.draft == hw->active_pkg_ver.draft &&
+ !memcmp(hw->pkg_name, hw->active_pkg_name, sizeof(hw->pkg_name))) {
+ if (already_loaded)
+ return ICE_DDP_PKG_SAME_VERSION_ALREADY_LOADED;
+ else
+ return ICE_DDP_PKG_SUCCESS;
+ } else if (hw->active_pkg_ver.major != ICE_PKG_SUPP_VER_MAJ ||
+ hw->active_pkg_ver.minor != ICE_PKG_SUPP_VER_MNR) {
+ return ICE_DDP_PKG_ALREADY_LOADED_NOT_SUPPORTED;
+ } else if (hw->active_pkg_ver.major == ICE_PKG_SUPP_VER_MAJ &&
+ hw->active_pkg_ver.minor == ICE_PKG_SUPP_VER_MNR) {
+ return ICE_DDP_PKG_COMPATIBLE_ALREADY_LOADED;
+ } else {
+ return ICE_DDP_PKG_ERR;
+ }
+}
+
+/**
* ice_init_pkg - initialize/download package
* @hw: pointer to the hardware structure
* @buf: pointer to the package buffer
@@ -1434,53 +1551,54 @@ static enum ice_status ice_get_prof_index_max(struct ice_hw *hw)
* ice_copy_and_init_pkg() instead of directly calling ice_init_pkg() in this
* case.
*/
-enum ice_status ice_init_pkg(struct ice_hw *hw, u8 *buf, u32 len)
+enum ice_ddp_state ice_init_pkg(struct ice_hw *hw, u8 *buf, u32 len)
{
+ bool already_loaded = false;
+ enum ice_ddp_state state;
struct ice_pkg_hdr *pkg;
- enum ice_status status;
struct ice_seg *seg;
if (!buf || !len)
- return ICE_ERR_PARAM;
+ return ICE_DDP_PKG_ERR;
pkg = (struct ice_pkg_hdr *)buf;
- status = ice_verify_pkg(pkg, len);
- if (status) {
+ state = ice_verify_pkg(pkg, len);
+ if (state) {
ice_debug(hw, ICE_DBG_INIT, "failed to verify pkg (err: %d)\n",
- status);
- return status;
+ state);
+ return state;
}
/* initialize package info */
- status = ice_init_pkg_info(hw, pkg);
- if (status)
- return status;
+ state = ice_init_pkg_info(hw, pkg);
+ if (state)
+ return state;
/* before downloading the package, check package version for
* compatibility with driver
*/
- status = ice_chk_pkg_compat(hw, pkg, &seg);
- if (status)
- return status;
+ state = ice_chk_pkg_compat(hw, pkg, &seg);
+ if (state)
+ return state;
/* initialize package hints and then download package */
ice_init_pkg_hints(hw, seg);
- status = ice_download_pkg(hw, seg);
- if (status == ICE_ERR_AQ_NO_WORK) {
+ state = ice_download_pkg(hw, seg);
+ if (state == ICE_DDP_PKG_ALREADY_LOADED) {
ice_debug(hw, ICE_DBG_INIT, "package previously loaded - no work.\n");
- status = 0;
+ already_loaded = true;
}
/* Get information on the package currently loaded in HW, then make sure
* the driver is compatible with this version.
*/
- if (!status) {
- status = ice_get_pkg_info(hw);
- if (!status)
- status = ice_chk_pkg_version(&hw->active_pkg_ver);
+ if (!state || state == ICE_DDP_PKG_ALREADY_LOADED) {
+ state = ice_get_pkg_info(hw);
+ if (!state)
+ state = ice_get_ddp_pkg_state(hw, already_loaded);
}
- if (!status) {
+ if (ice_is_init_pkg_successful(state)) {
hw->seg = seg;
/* on successful package download update other required
* registers to support the package and fill HW tables
@@ -1488,13 +1606,14 @@ enum ice_status ice_init_pkg(struct ice_hw *hw, u8 *buf, u32 len)
*/
ice_init_pkg_regs(hw);
ice_fill_blk_tbls(hw);
+ ice_fill_hw_ptype(hw);
ice_get_prof_index_max(hw);
} else {
ice_debug(hw, ICE_DBG_INIT, "package load failed, %d\n",
- status);
+ state);
}
- return status;
+ return state;
}
/**
@@ -1520,18 +1639,19 @@ enum ice_status ice_init_pkg(struct ice_hw *hw, u8 *buf, u32 len)
* package buffer, as the new copy will be managed by this function and
* related routines.
*/
-enum ice_status ice_copy_and_init_pkg(struct ice_hw *hw, const u8 *buf, u32 len)
+enum ice_ddp_state
+ice_copy_and_init_pkg(struct ice_hw *hw, const u8 *buf, u32 len)
{
- enum ice_status status;
+ enum ice_ddp_state state;
u8 *buf_copy;
if (!buf || !len)
- return ICE_ERR_PARAM;
+ return ICE_DDP_PKG_ERR;
buf_copy = devm_kmemdup(ice_hw_to_dev(hw), buf, len, GFP_KERNEL);
- status = ice_init_pkg(hw, buf_copy, len);
- if (status) {
+ state = ice_init_pkg(hw, buf_copy, len);
+ if (!ice_is_init_pkg_successful(state)) {
/* Free the copy, since we failed to initialize the package */
devm_kfree(ice_hw_to_dev(hw), buf_copy);
} else {
@@ -1540,7 +1660,23 @@ enum ice_status ice_copy_and_init_pkg(struct ice_hw *hw, const u8 *buf, u32 len)
hw->pkg_size = len;
}
- return status;
+ return state;
+}
+
+/**
+ * ice_is_init_pkg_successful - check if DDP init was successful
+ * @state: state of the DDP pkg after download
+ */
+bool ice_is_init_pkg_successful(enum ice_ddp_state state)
+{
+ switch (state) {
+ case ICE_DDP_PKG_SUCCESS:
+ case ICE_DDP_PKG_SAME_VERSION_ALREADY_LOADED:
+ case ICE_DDP_PKG_COMPATIBLE_ALREADY_LOADED:
+ return true;
+ default:
+ return false;
+ }
}
/**
@@ -1644,7 +1780,7 @@ ice_get_sw_fv_bitmap(struct ice_hw *hw, enum ice_prof_type req_profs,
* NOTE: The caller of the function is responsible for freeing the memory
* allocated for every list entry.
*/
-enum ice_status
+int
ice_get_sw_fv_list(struct ice_hw *hw, u8 *prot_ids, u16 ids_cnt,
unsigned long *bm, struct list_head *fv_list)
{
@@ -1658,7 +1794,7 @@ ice_get_sw_fv_list(struct ice_hw *hw, u8 *prot_ids, u16 ids_cnt,
memset(&state, 0, sizeof(state));
if (!ids_cnt || !hw->seg)
- return ICE_ERR_PARAM;
+ return -EINVAL;
ice_seg = hw->seg;
do {
@@ -1702,7 +1838,7 @@ ice_get_sw_fv_list(struct ice_hw *hw, u8 *prot_ids, u16 ids_cnt,
}
} while (fv);
if (list_empty(fv_list))
- return ICE_ERR_CFG;
+ return -EIO;
return 0;
err:
@@ -1711,7 +1847,7 @@ ice_get_sw_fv_list(struct ice_hw *hw, u8 *prot_ids, u16 ids_cnt,
devm_kfree(ice_hw_to_dev(hw), fvl);
}
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
}
/**
@@ -1779,7 +1915,7 @@ static void ice_pkg_buf_free(struct ice_hw *hw, struct ice_buf_build *bld)
* result in some wasted space in the buffer.
* Note: all package contents must be in Little Endian form.
*/
-static enum ice_status
+static int
ice_pkg_buf_reserve_section(struct ice_buf_build *bld, u16 count)
{
struct ice_buf_hdr *buf;
@@ -1787,17 +1923,17 @@ ice_pkg_buf_reserve_section(struct ice_buf_build *bld, u16 count)
u16 data_end;
if (!bld)
- return ICE_ERR_PARAM;
+ return -EINVAL;
buf = (struct ice_buf_hdr *)&bld->buf;
/* already an active section, can't increase table size */
section_count = le16_to_cpu(buf->section_count);
if (section_count > 0)
- return ICE_ERR_CFG;
+ return -EIO;
if (bld->reserved_section_table_entries + count > ICE_MAX_S_COUNT)
- return ICE_ERR_CFG;
+ return -EIO;
bld->reserved_section_table_entries += count;
data_end = le16_to_cpu(buf->data_end) +
@@ -1959,19 +2095,19 @@ static u16 ice_tunnel_idx_to_entry(struct ice_hw *hw, enum ice_tunnel_type type,
* creating a package buffer with the tunnel info and issuing an update package
* command.
*/
-static enum ice_status
+static int
ice_create_tunnel(struct ice_hw *hw, u16 index,
enum ice_tunnel_type type, u16 port)
{
struct ice_boost_tcam_section *sect_rx, *sect_tx;
- enum ice_status status = ICE_ERR_MAX_LIMIT;
struct ice_buf_build *bld;
+ int status = -ENOSPC;
mutex_lock(&hw->tnl_lock);
bld = ice_pkg_buf_alloc(hw);
if (!bld) {
- status = ICE_ERR_NO_MEMORY;
+ status = -ENOMEM;
goto ice_create_tunnel_end;
}
@@ -2030,26 +2166,26 @@ ice_create_tunnel(struct ice_hw *hw, u16 index,
* targeting the specific updates requested and then performing an update
* package.
*/
-static enum ice_status
+static int
ice_destroy_tunnel(struct ice_hw *hw, u16 index, enum ice_tunnel_type type,
u16 port)
{
struct ice_boost_tcam_section *sect_rx, *sect_tx;
- enum ice_status status = ICE_ERR_MAX_LIMIT;
struct ice_buf_build *bld;
+ int status = -ENOSPC;
mutex_lock(&hw->tnl_lock);
if (WARN_ON(!hw->tnl.tbl[index].valid ||
hw->tnl.tbl[index].type != type ||
hw->tnl.tbl[index].port != port)) {
- status = ICE_ERR_OUT_OF_RANGE;
+ status = -EIO;
goto ice_destroy_tunnel_end;
}
bld = ice_pkg_buf_alloc(hw);
if (!bld) {
- status = ICE_ERR_NO_MEMORY;
+ status = -ENOMEM;
goto ice_destroy_tunnel_end;
}
@@ -2097,7 +2233,7 @@ int ice_udp_tunnel_set_port(struct net_device *netdev, unsigned int table,
struct ice_vsi *vsi = np->vsi;
struct ice_pf *pf = vsi->back;
enum ice_tunnel_type tnl_type;
- enum ice_status status;
+ int status;
u16 index;
tnl_type = ti->type == UDP_TUNNEL_TYPE_VXLAN ? TNL_VXLAN : TNL_GENEVE;
@@ -2105,8 +2241,8 @@ int ice_udp_tunnel_set_port(struct net_device *netdev, unsigned int table,
status = ice_create_tunnel(&pf->hw, index, tnl_type, ntohs(ti->port));
if (status) {
- netdev_err(netdev, "Error adding UDP tunnel - %s\n",
- ice_stat_str(status));
+ netdev_err(netdev, "Error adding UDP tunnel - %d\n",
+ status);
return -EIO;
}
@@ -2121,15 +2257,15 @@ int ice_udp_tunnel_unset_port(struct net_device *netdev, unsigned int table,
struct ice_vsi *vsi = np->vsi;
struct ice_pf *pf = vsi->back;
enum ice_tunnel_type tnl_type;
- enum ice_status status;
+ int status;
tnl_type = ti->type == UDP_TUNNEL_TYPE_VXLAN ? TNL_VXLAN : TNL_GENEVE;
status = ice_destroy_tunnel(&pf->hw, ti->hw_priv, tnl_type,
ntohs(ti->port));
if (status) {
- netdev_err(netdev, "Error removing UDP tunnel - %s\n",
- ice_stat_str(status));
+ netdev_err(netdev, "Error removing UDP tunnel - %d\n",
+ status);
return -EIO;
}
@@ -2145,17 +2281,17 @@ int ice_udp_tunnel_unset_port(struct net_device *netdev, unsigned int table,
* @prot: variable to receive the protocol ID
* @off: variable to receive the protocol offset
*/
-enum ice_status
+int
ice_find_prot_off(struct ice_hw *hw, enum ice_block blk, u8 prof, u16 fv_idx,
u8 *prot, u16 *off)
{
struct ice_fv_word *fv_ext;
if (prof >= hw->blk[blk].es.count)
- return ICE_ERR_PARAM;
+ return -EINVAL;
if (fv_idx >= hw->blk[blk].es.fvw)
- return ICE_ERR_PARAM;
+ return -EINVAL;
fv_ext = hw->blk[blk].es.t + (prof * hw->blk[blk].es.fvw);
@@ -2178,11 +2314,11 @@ ice_find_prot_off(struct ice_hw *hw, enum ice_block blk, u8 prof, u16 fv_idx,
* PTG ID that contains it through the PTG parameter, with the value of
* ICE_DEFAULT_PTG (0) meaning it is part the default PTG.
*/
-static enum ice_status
+static int
ice_ptg_find_ptype(struct ice_hw *hw, enum ice_block blk, u16 ptype, u8 *ptg)
{
if (ptype >= ICE_XLT1_CNT || !ptg)
- return ICE_ERR_PARAM;
+ return -EINVAL;
*ptg = hw->blk[blk].xlt1.ptypes[ptype].ptg;
return 0;
@@ -2212,21 +2348,21 @@ static void ice_ptg_alloc_val(struct ice_hw *hw, enum ice_block blk, u8 ptg)
* This function will remove the ptype from the specific PTG, and move it to
* the default PTG (ICE_DEFAULT_PTG).
*/
-static enum ice_status
+static int
ice_ptg_remove_ptype(struct ice_hw *hw, enum ice_block blk, u16 ptype, u8 ptg)
{
struct ice_ptg_ptype **ch;
struct ice_ptg_ptype *p;
if (ptype > ICE_XLT1_CNT - 1)
- return ICE_ERR_PARAM;
+ return -EINVAL;
if (!hw->blk[blk].xlt1.ptg_tbl[ptg].in_use)
- return ICE_ERR_DOES_NOT_EXIST;
+ return -ENOENT;
/* Should not happen if .in_use is set, bad config */
if (!hw->blk[blk].xlt1.ptg_tbl[ptg].first_ptype)
- return ICE_ERR_CFG;
+ return -EIO;
/* find the ptype within this PTG, and bypass the link over it */
p = hw->blk[blk].xlt1.ptg_tbl[ptg].first_ptype;
@@ -2259,17 +2395,17 @@ ice_ptg_remove_ptype(struct ice_hw *hw, enum ice_block blk, u16 ptype, u8 ptg)
* a destination PTG ID of ICE_DEFAULT_PTG (0) will move the ptype to the
* default PTG.
*/
-static enum ice_status
+static int
ice_ptg_add_mv_ptype(struct ice_hw *hw, enum ice_block blk, u16 ptype, u8 ptg)
{
- enum ice_status status;
u8 original_ptg;
+ int status;
if (ptype > ICE_XLT1_CNT - 1)
- return ICE_ERR_PARAM;
+ return -EINVAL;
if (!hw->blk[blk].xlt1.ptg_tbl[ptg].in_use && ptg != ICE_DEFAULT_PTG)
- return ICE_ERR_DOES_NOT_EXIST;
+ return -ENOENT;
status = ice_ptg_find_ptype(hw, blk, ptype, &original_ptg);
if (status)
@@ -2404,11 +2540,11 @@ ice_match_prop_lst(struct list_head *list1, struct list_head *list2)
* This function will lookup the VSI entry in the XLT2 list and return
* the VSI group its associated with.
*/
-static enum ice_status
+static int
ice_vsig_find_vsi(struct ice_hw *hw, enum ice_block blk, u16 vsi, u16 *vsig)
{
if (!vsig || vsi >= ICE_MAX_VSI)
- return ICE_ERR_PARAM;
+ return -EINVAL;
/* As long as there's a default or valid VSIG associated with the input
* VSI, the functions returns a success. Any handling of VSIG will be
@@ -2473,7 +2609,7 @@ static u16 ice_vsig_alloc(struct ice_hw *hw, enum ice_block blk)
* for, the list must match exactly, including the order in which the
* characteristics are listed.
*/
-static enum ice_status
+static int
ice_find_dup_props_vsig(struct ice_hw *hw, enum ice_block blk,
struct list_head *chs, u16 *vsig)
{
@@ -2487,7 +2623,7 @@ ice_find_dup_props_vsig(struct ice_hw *hw, enum ice_block blk,
return 0;
}
- return ICE_ERR_DOES_NOT_EXIST;
+ return -ENOENT;
}
/**
@@ -2499,8 +2635,7 @@ ice_find_dup_props_vsig(struct ice_hw *hw, enum ice_block blk,
* The function will remove all VSIs associated with the input VSIG and move
* them to the DEFAULT_VSIG and mark the VSIG available.
*/
-static enum ice_status
-ice_vsig_free(struct ice_hw *hw, enum ice_block blk, u16 vsig)
+static int ice_vsig_free(struct ice_hw *hw, enum ice_block blk, u16 vsig)
{
struct ice_vsig_prof *dtmp, *del;
struct ice_vsig_vsi *vsi_cur;
@@ -2508,10 +2643,10 @@ ice_vsig_free(struct ice_hw *hw, enum ice_block blk, u16 vsig)
idx = vsig & ICE_VSIG_IDX_M;
if (idx >= ICE_MAX_VSIGS)
- return ICE_ERR_PARAM;
+ return -EINVAL;
if (!hw->blk[blk].xlt2.vsig_tbl[idx].in_use)
- return ICE_ERR_DOES_NOT_EXIST;
+ return -ENOENT;
hw->blk[blk].xlt2.vsig_tbl[idx].in_use = false;
@@ -2560,7 +2695,7 @@ ice_vsig_free(struct ice_hw *hw, enum ice_block blk, u16 vsig)
* The function will remove the input VSI from its VSI group and move it
* to the DEFAULT_VSIG.
*/
-static enum ice_status
+static int
ice_vsig_remove_vsi(struct ice_hw *hw, enum ice_block blk, u16 vsi, u16 vsig)
{
struct ice_vsig_vsi **vsi_head, *vsi_cur, *vsi_tgt;
@@ -2569,10 +2704,10 @@ ice_vsig_remove_vsi(struct ice_hw *hw, enum ice_block blk, u16 vsi, u16 vsig)
idx = vsig & ICE_VSIG_IDX_M;
if (vsi >= ICE_MAX_VSI || idx >= ICE_MAX_VSIGS)
- return ICE_ERR_PARAM;
+ return -EINVAL;
if (!hw->blk[blk].xlt2.vsig_tbl[idx].in_use)
- return ICE_ERR_DOES_NOT_EXIST;
+ return -ENOENT;
/* entry already in default VSIG, don't have to remove */
if (idx == ICE_DEFAULT_VSIG)
@@ -2580,7 +2715,7 @@ ice_vsig_remove_vsi(struct ice_hw *hw, enum ice_block blk, u16 vsi, u16 vsig)
vsi_head = &hw->blk[blk].xlt2.vsig_tbl[idx].first_vsi;
if (!(*vsi_head))
- return ICE_ERR_CFG;
+ return -EIO;
vsi_tgt = &hw->blk[blk].xlt2.vsis[vsi];
vsi_cur = (*vsi_head);
@@ -2597,7 +2732,7 @@ ice_vsig_remove_vsi(struct ice_hw *hw, enum ice_block blk, u16 vsi, u16 vsig)
/* verify if VSI was removed from group list */
if (!vsi_cur)
- return ICE_ERR_DOES_NOT_EXIST;
+ return -ENOENT;
vsi_cur->vsig = ICE_DEFAULT_VSIG;
vsi_cur->changed = 1;
@@ -2618,24 +2753,24 @@ ice_vsig_remove_vsi(struct ice_hw *hw, enum ice_block blk, u16 vsi, u16 vsig)
* move the entry to the DEFAULT_VSIG, update the original VSIG and
* then move entry to the new VSIG.
*/
-static enum ice_status
+static int
ice_vsig_add_mv_vsi(struct ice_hw *hw, enum ice_block blk, u16 vsi, u16 vsig)
{
struct ice_vsig_vsi *tmp;
- enum ice_status status;
u16 orig_vsig, idx;
+ int status;
idx = vsig & ICE_VSIG_IDX_M;
if (vsi >= ICE_MAX_VSI || idx >= ICE_MAX_VSIGS)
- return ICE_ERR_PARAM;
+ return -EINVAL;
/* if VSIG not in use and VSIG is not default type this VSIG
* doesn't exist.
*/
if (!hw->blk[blk].xlt2.vsig_tbl[idx].in_use &&
vsig != ICE_DEFAULT_VSIG)
- return ICE_ERR_DOES_NOT_EXIST;
+ return -ENOENT;
status = ice_vsig_find_vsi(hw, blk, vsi, &orig_vsig);
if (status)
@@ -2741,7 +2876,7 @@ ice_prof_has_mask(struct ice_hw *hw, enum ice_block blk, u8 prof, u16 *masks)
* @masks: masks for FV
* @prof_id: receives the profile ID
*/
-static enum ice_status
+static int
ice_find_prof_id_with_mask(struct ice_hw *hw, enum ice_block blk,
struct ice_fv_word *fv, u16 *masks, u8 *prof_id)
{
@@ -2752,7 +2887,7 @@ ice_find_prof_id_with_mask(struct ice_hw *hw, enum ice_block blk,
* field vector and mask. This will cause rule interference.
*/
if (blk == ICE_BLK_FD)
- return ICE_ERR_DOES_NOT_EXIST;
+ return -ENOENT;
for (i = 0; i < (u8)es->count; i++) {
u16 off = i * es->fvw;
@@ -2768,7 +2903,7 @@ ice_find_prof_id_with_mask(struct ice_hw *hw, enum ice_block blk,
return 0;
}
- return ICE_ERR_DOES_NOT_EXIST;
+ return -ENOENT;
}
/**
@@ -2821,14 +2956,14 @@ static bool ice_tcam_ent_rsrc_type(enum ice_block blk, u16 *rsrc_type)
* This function allocates a new entry in a Profile ID TCAM for a specific
* block.
*/
-static enum ice_status
+static int
ice_alloc_tcam_ent(struct ice_hw *hw, enum ice_block blk, bool btm,
u16 *tcam_idx)
{
u16 res_type;
if (!ice_tcam_ent_rsrc_type(blk, &res_type))
- return ICE_ERR_PARAM;
+ return -EINVAL;
return ice_alloc_hw_res(hw, res_type, 1, btm, tcam_idx);
}
@@ -2841,13 +2976,13 @@ ice_alloc_tcam_ent(struct ice_hw *hw, enum ice_block blk, bool btm,
*
* This function frees an entry in a Profile ID TCAM for a specific block.
*/
-static enum ice_status
+static int
ice_free_tcam_ent(struct ice_hw *hw, enum ice_block blk, u16 tcam_idx)
{
u16 res_type;
if (!ice_tcam_ent_rsrc_type(blk, &res_type))
- return ICE_ERR_PARAM;
+ return -EINVAL;
return ice_free_hw_res(hw, res_type, 1, &tcam_idx);
}
@@ -2861,15 +2996,14 @@ ice_free_tcam_ent(struct ice_hw *hw, enum ice_block blk, u16 tcam_idx)
* This function allocates a new profile ID, which also corresponds to a Field
* Vector (Extraction Sequence) entry.
*/
-static enum ice_status
-ice_alloc_prof_id(struct ice_hw *hw, enum ice_block blk, u8 *prof_id)
+static int ice_alloc_prof_id(struct ice_hw *hw, enum ice_block blk, u8 *prof_id)
{
- enum ice_status status;
u16 res_type;
u16 get_prof;
+ int status;
if (!ice_prof_id_rsrc_type(blk, &res_type))
- return ICE_ERR_PARAM;
+ return -EINVAL;
status = ice_alloc_hw_res(hw, res_type, 1, false, &get_prof);
if (!status)
@@ -2886,14 +3020,13 @@ ice_alloc_prof_id(struct ice_hw *hw, enum ice_block blk, u8 *prof_id)
*
* This function frees a profile ID, which also corresponds to a Field Vector.
*/
-static enum ice_status
-ice_free_prof_id(struct ice_hw *hw, enum ice_block blk, u8 prof_id)
+static int ice_free_prof_id(struct ice_hw *hw, enum ice_block blk, u8 prof_id)
{
u16 tmp_prof_id = (u16)prof_id;
u16 res_type;
if (!ice_prof_id_rsrc_type(blk, &res_type))
- return ICE_ERR_PARAM;
+ return -EINVAL;
return ice_free_hw_res(hw, res_type, 1, &tmp_prof_id);
}
@@ -2904,11 +3037,10 @@ ice_free_prof_id(struct ice_hw *hw, enum ice_block blk, u8 prof_id)
* @blk: the block from which to free the profile ID
* @prof_id: the profile ID for which to increment the reference count
*/
-static enum ice_status
-ice_prof_inc_ref(struct ice_hw *hw, enum ice_block blk, u8 prof_id)
+static int ice_prof_inc_ref(struct ice_hw *hw, enum ice_block blk, u8 prof_id)
{
if (prof_id > hw->blk[blk].es.count)
- return ICE_ERR_PARAM;
+ return -EINVAL;
hw->blk[blk].es.ref_count[prof_id]++;
@@ -3025,17 +3157,17 @@ static void ice_init_all_prof_masks(struct ice_hw *hw)
* @mask: the 16-bit mask
* @mask_idx: variable to receive the mask index
*/
-static enum ice_status
+static int
ice_alloc_prof_mask(struct ice_hw *hw, enum ice_block blk, u16 idx, u16 mask,
u16 *mask_idx)
{
bool found_unused = false, found_copy = false;
- enum ice_status status = ICE_ERR_MAX_LIMIT;
u16 unused_idx = 0, copy_idx = 0;
+ int status = -ENOSPC;
u16 i;
if (blk != ICE_BLK_RSS && blk != ICE_BLK_FD)
- return ICE_ERR_PARAM;
+ return -EINVAL;
mutex_lock(&hw->blk[blk].masks.lock);
@@ -3093,15 +3225,15 @@ ice_alloc_prof_mask(struct ice_hw *hw, enum ice_block blk, u16 idx, u16 mask,
* @blk: hardware block
* @mask_idx: index of mask
*/
-static enum ice_status
+static int
ice_free_prof_mask(struct ice_hw *hw, enum ice_block blk, u16 mask_idx)
{
if (blk != ICE_BLK_RSS && blk != ICE_BLK_FD)
- return ICE_ERR_PARAM;
+ return -EINVAL;
if (!(mask_idx >= hw->blk[blk].masks.first &&
mask_idx < hw->blk[blk].masks.first + hw->blk[blk].masks.count))
- return ICE_ERR_DOES_NOT_EXIST;
+ return -ENOENT;
mutex_lock(&hw->blk[blk].masks.lock);
@@ -3135,14 +3267,14 @@ ice_free_prof_mask(struct ice_hw *hw, enum ice_block blk, u16 mask_idx)
* @blk: hardware block
* @prof_id: profile ID
*/
-static enum ice_status
+static int
ice_free_prof_masks(struct ice_hw *hw, enum ice_block blk, u16 prof_id)
{
u32 mask_bm;
u16 i;
if (blk != ICE_BLK_RSS && blk != ICE_BLK_FD)
- return ICE_ERR_PARAM;
+ return -EINVAL;
mask_bm = hw->blk[blk].es.mask_ena[prof_id];
for (i = 0; i < BITS_PER_BYTE * sizeof(mask_bm); i++)
@@ -3197,7 +3329,7 @@ static void ice_shutdown_all_prof_masks(struct ice_hw *hw)
* @prof_id: profile ID
* @masks: masks
*/
-static enum ice_status
+static int
ice_update_prof_masking(struct ice_hw *hw, enum ice_block blk, u16 prof_id,
u16 *masks)
{
@@ -3227,7 +3359,7 @@ ice_update_prof_masking(struct ice_hw *hw, enum ice_block blk, u16 prof_id,
if (ena_mask & BIT(i))
ice_free_prof_mask(hw, blk, i);
- return ICE_ERR_OUT_OF_RANGE;
+ return -EIO;
}
/* enable the masks for this profile */
@@ -3269,11 +3401,11 @@ ice_write_es(struct ice_hw *hw, enum ice_block blk, u8 prof_id,
* @blk: the block from which to free the profile ID
* @prof_id: the profile ID for which to decrement the reference count
*/
-static enum ice_status
+static int
ice_prof_dec_ref(struct ice_hw *hw, enum ice_block blk, u8 prof_id)
{
if (prof_id > hw->blk[blk].es.count)
- return ICE_ERR_PARAM;
+ return -EINVAL;
if (hw->blk[blk].es.ref_count[prof_id] > 0) {
if (!--hw->blk[blk].es.ref_count[prof_id]) {
@@ -3711,7 +3843,7 @@ void ice_clear_hw_tbls(struct ice_hw *hw)
* ice_init_hw_tbls - init hardware table memory
* @hw: pointer to the hardware structure
*/
-enum ice_status ice_init_hw_tbls(struct ice_hw *hw)
+int ice_init_hw_tbls(struct ice_hw *hw)
{
u8 i;
@@ -3830,7 +3962,7 @@ enum ice_status ice_init_hw_tbls(struct ice_hw *hw)
err:
ice_free_hw_tbls(hw);
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
}
/**
@@ -3846,7 +3978,7 @@ enum ice_status ice_init_hw_tbls(struct ice_hw *hw)
* @nm_msk: never match mask
* @key: output of profile ID key
*/
-static enum ice_status
+static int
ice_prof_gen_key(struct ice_hw *hw, enum ice_block blk, u8 ptg, u16 vsig,
u8 cdid, u16 flags, u8 vl_msk[ICE_TCAM_KEY_VAL_SZ],
u8 dc_msk[ICE_TCAM_KEY_VAL_SZ], u8 nm_msk[ICE_TCAM_KEY_VAL_SZ],
@@ -3902,7 +4034,7 @@ ice_prof_gen_key(struct ice_hw *hw, enum ice_block blk, u8 ptg, u16 vsig,
* @dc_msk: don't care mask
* @nm_msk: never match mask
*/
-static enum ice_status
+static int
ice_tcam_write_entry(struct ice_hw *hw, enum ice_block blk, u16 idx,
u8 prof_id, u8 ptg, u16 vsig, u8 cdid, u16 flags,
u8 vl_msk[ICE_TCAM_KEY_VAL_SZ],
@@ -3910,7 +4042,7 @@ ice_tcam_write_entry(struct ice_hw *hw, enum ice_block blk, u16 idx,
u8 nm_msk[ICE_TCAM_KEY_VAL_SZ])
{
struct ice_prof_tcam_entry;
- enum ice_status status;
+ int status;
status = ice_prof_gen_key(hw, blk, ptg, vsig, cdid, flags, vl_msk,
dc_msk, nm_msk, hw->blk[blk].prof.t[idx].key);
@@ -3929,7 +4061,7 @@ ice_tcam_write_entry(struct ice_hw *hw, enum ice_block blk, u16 idx,
* @vsig: VSIG to query
* @refs: pointer to variable to receive the reference count
*/
-static enum ice_status
+static int
ice_vsig_get_ref(struct ice_hw *hw, enum ice_block blk, u16 vsig, u16 *refs)
{
u16 idx = vsig & ICE_VSIG_IDX_M;
@@ -3938,7 +4070,7 @@ ice_vsig_get_ref(struct ice_hw *hw, enum ice_block blk, u16 vsig, u16 *refs)
*refs = 0;
if (!hw->blk[blk].xlt2.vsig_tbl[idx].in_use)
- return ICE_ERR_DOES_NOT_EXIST;
+ return -ENOENT;
ptr = hw->blk[blk].xlt2.vsig_tbl[idx].first_vsi;
while (ptr) {
@@ -3979,7 +4111,7 @@ ice_has_prof_vsig(struct ice_hw *hw, enum ice_block blk, u16 vsig, u64 hdl)
* @bld: the update package buffer build to add to
* @chgs: the list of changes to make in hardware
*/
-static enum ice_status
+static int
ice_prof_bld_es(struct ice_hw *hw, enum ice_block blk,
struct ice_buf_build *bld, struct list_head *chgs)
{
@@ -3999,7 +4131,7 @@ ice_prof_bld_es(struct ice_hw *hw, enum ice_block blk,
sizeof(p->es[0]));
if (!p)
- return ICE_ERR_MAX_LIMIT;
+ return -ENOSPC;
p->count = cpu_to_le16(1);
p->offset = cpu_to_le16(tmp->prof_id);
@@ -4017,7 +4149,7 @@ ice_prof_bld_es(struct ice_hw *hw, enum ice_block blk,
* @bld: the update package buffer build to add to
* @chgs: the list of changes to make in hardware
*/
-static enum ice_status
+static int
ice_prof_bld_tcam(struct ice_hw *hw, enum ice_block blk,
struct ice_buf_build *bld, struct list_head *chgs)
{
@@ -4033,7 +4165,7 @@ ice_prof_bld_tcam(struct ice_hw *hw, enum ice_block blk,
struct_size(p, entry, 1));
if (!p)
- return ICE_ERR_MAX_LIMIT;
+ return -ENOSPC;
p->count = cpu_to_le16(1);
p->entry[0].addr = cpu_to_le16(tmp->tcam_idx);
@@ -4053,7 +4185,7 @@ ice_prof_bld_tcam(struct ice_hw *hw, enum ice_block blk,
* @bld: the update package buffer build to add to
* @chgs: the list of changes to make in hardware
*/
-static enum ice_status
+static int
ice_prof_bld_xlt1(enum ice_block blk, struct ice_buf_build *bld,
struct list_head *chgs)
{
@@ -4069,7 +4201,7 @@ ice_prof_bld_xlt1(enum ice_block blk, struct ice_buf_build *bld,
struct_size(p, value, 1));
if (!p)
- return ICE_ERR_MAX_LIMIT;
+ return -ENOSPC;
p->count = cpu_to_le16(1);
p->offset = cpu_to_le16(tmp->ptype);
@@ -4085,7 +4217,7 @@ ice_prof_bld_xlt1(enum ice_block blk, struct ice_buf_build *bld,
* @bld: the update package buffer build to add to
* @chgs: the list of changes to make in hardware
*/
-static enum ice_status
+static int
ice_prof_bld_xlt2(enum ice_block blk, struct ice_buf_build *bld,
struct list_head *chgs)
{
@@ -4104,7 +4236,7 @@ ice_prof_bld_xlt2(enum ice_block blk, struct ice_buf_build *bld,
struct_size(p, value, 1));
if (!p)
- return ICE_ERR_MAX_LIMIT;
+ return -ENOSPC;
p->count = cpu_to_le16(1);
p->offset = cpu_to_le16(tmp->vsi);
@@ -4124,18 +4256,18 @@ ice_prof_bld_xlt2(enum ice_block blk, struct ice_buf_build *bld,
* @blk: hardware block
* @chgs: the list of changes to make in hardware
*/
-static enum ice_status
+static int
ice_upd_prof_hw(struct ice_hw *hw, enum ice_block blk,
struct list_head *chgs)
{
struct ice_buf_build *b;
struct ice_chs_chg *tmp;
- enum ice_status status;
u16 pkg_sects;
u16 xlt1 = 0;
u16 xlt2 = 0;
u16 tcam = 0;
u16 es = 0;
+ int status;
u16 sects;
/* count number of sections we need */
@@ -4167,7 +4299,7 @@ ice_upd_prof_hw(struct ice_hw *hw, enum ice_block blk,
/* Build update package buffer */
b = ice_pkg_buf_alloc(hw);
if (!b)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
status = ice_pkg_buf_reserve_section(b, sects);
if (status)
@@ -4204,13 +4336,13 @@ ice_upd_prof_hw(struct ice_hw *hw, enum ice_block blk,
*/
pkg_sects = ice_pkg_buf_get_active_sections(b);
if (!pkg_sects || pkg_sects != sects) {
- status = ICE_ERR_INVAL_SIZE;
+ status = -EINVAL;
goto error_tmp;
}
/* update package */
status = ice_update_pkg(hw, ice_pkg_buf(b), 1);
- if (status == ICE_ERR_AQ_ERROR)
+ if (status == -EIO)
ice_debug(hw, ICE_DBG_INIT, "Unable to update HW profile\n");
error_tmp:
@@ -4276,7 +4408,7 @@ static const struct ice_fd_src_dst_pair ice_fd_pairs[] = {
* @prof_id: profile ID
* @es: extraction sequence (length of array is determined by the block)
*/
-static enum ice_status
+static int
ice_update_fd_swap(struct ice_hw *hw, u16 prof_id, struct ice_fv_word *es)
{
DECLARE_BITMAP(pair_list, ICE_FD_SRC_DST_PAIR_COUNT);
@@ -4308,7 +4440,7 @@ ice_update_fd_swap(struct ice_hw *hw, u16 prof_id, struct ice_fv_word *es)
for (j = 0; j < ICE_FD_SRC_DST_PAIR_COUNT; j++)
if (es[i].prot_id == ice_fd_pairs[j].prot_id &&
es[i].off == ice_fd_pairs[j].off) {
- set_bit(j, pair_list);
+ __set_bit(j, pair_list);
pair_start[j] = i;
}
}
@@ -4331,7 +4463,7 @@ ice_update_fd_swap(struct ice_hw *hw, u16 prof_id, struct ice_fv_word *es)
/* check for room */
if (first_free + 1 < (s8)ice_fd_pairs[index].count)
- return ICE_ERR_MAX_LIMIT;
+ return -ENOSPC;
/* place in extraction sequence */
for (k = 0; k < ice_fd_pairs[index].count; k++) {
@@ -4341,7 +4473,7 @@ ice_update_fd_swap(struct ice_hw *hw, u16 prof_id, struct ice_fv_word *es)
ice_fd_pairs[index].off + (k * 2);
if (k > first_free)
- return ICE_ERR_OUT_OF_RANGE;
+ return -EIO;
/* keep track of non-relevant fields */
mask_sel |= BIT(first_free - k);
@@ -4452,7 +4584,7 @@ ice_get_ptype_attrib_info(enum ice_ptype_attrib_type type,
* @attr: array of attributes that will be considered
* @attr_cnt: number of elements in the attribute array
*/
-static enum ice_status
+static int
ice_add_prof_attrib(struct ice_prof_map *prof, u8 ptg, u16 ptype,
const struct ice_ptype_attributes *attr, u16 attr_cnt)
{
@@ -4468,11 +4600,11 @@ ice_add_prof_attrib(struct ice_prof_map *prof, u8 ptg, u16 ptype,
&prof->attr[prof->ptg_cnt]);
if (++prof->ptg_cnt >= ICE_MAX_PTG_PER_PROFILE)
- return ICE_ERR_MAX_LIMIT;
+ return -ENOSPC;
}
if (!found)
- return ICE_ERR_DOES_NOT_EXIST;
+ return -ENOENT;
return 0;
}
@@ -4493,7 +4625,7 @@ ice_add_prof_attrib(struct ice_prof_map *prof, u8 ptg, u16 ptype,
* it will not be written until the first call to ice_add_flow that specifies
* the ID value used here.
*/
-enum ice_status
+int
ice_add_prof(struct ice_hw *hw, enum ice_block blk, u64 id, u8 ptypes[],
const struct ice_ptype_attributes *attr, u16 attr_cnt,
struct ice_fv_word *es, u16 *masks)
@@ -4501,9 +4633,9 @@ ice_add_prof(struct ice_hw *hw, enum ice_block blk, u64 id, u8 ptypes[],
u32 bytes = DIV_ROUND_UP(ICE_FLOW_PTYPE_MAX, BITS_PER_BYTE);
DECLARE_BITMAP(ptgs_used, ICE_XLT1_CNT);
struct ice_prof_map *prof;
- enum ice_status status;
u8 byte = 0;
u8 prof_id;
+ int status;
bitmap_zero(ptgs_used, ICE_XLT1_CNT);
@@ -4541,7 +4673,7 @@ ice_add_prof(struct ice_hw *hw, enum ice_block blk, u64 id, u8 ptypes[],
/* add profile info */
prof = devm_kzalloc(ice_hw_to_dev(hw), sizeof(*prof), GFP_KERNEL);
if (!prof) {
- status = ICE_ERR_NO_MEMORY;
+ status = -ENOMEM;
goto err_ice_add_prof;
}
@@ -4578,13 +4710,13 @@ ice_add_prof(struct ice_hw *hw, enum ice_block blk, u64 id, u8 ptypes[],
if (test_bit(ptg, ptgs_used))
continue;
- set_bit(ptg, ptgs_used);
+ __set_bit(ptg, ptgs_used);
/* Check to see there are any attributes for
* this PTYPE, and add them if found.
*/
status = ice_add_prof_attrib(prof, ptg, ptype,
attr, attr_cnt);
- if (status == ICE_ERR_MAX_LIMIT)
+ if (status == -ENOSPC)
break;
if (status) {
/* This is simple a PTYPE/PTG with no
@@ -4661,14 +4793,13 @@ ice_vsig_prof_id_count(struct ice_hw *hw, enum ice_block blk, u16 vsig)
* @blk: hardware block
* @idx: the index to release
*/
-static enum ice_status
-ice_rel_tcam_idx(struct ice_hw *hw, enum ice_block blk, u16 idx)
+static int ice_rel_tcam_idx(struct ice_hw *hw, enum ice_block blk, u16 idx)
{
/* Masks to invoke a never match entry */
u8 vl_msk[ICE_TCAM_KEY_VAL_SZ] = { 0xFF, 0xFF, 0xFF, 0xFF, 0xFF };
u8 dc_msk[ICE_TCAM_KEY_VAL_SZ] = { 0xFE, 0xFF, 0xFF, 0xFF, 0xFF };
u8 nm_msk[ICE_TCAM_KEY_VAL_SZ] = { 0x01, 0x00, 0x00, 0x00, 0x00 };
- enum ice_status status;
+ int status;
/* write the TCAM entry */
status = ice_tcam_write_entry(hw, blk, idx, 0, 0, 0, 0, 0, vl_msk,
@@ -4688,11 +4819,11 @@ ice_rel_tcam_idx(struct ice_hw *hw, enum ice_block blk, u16 idx)
* @blk: hardware block
* @prof: pointer to profile structure to remove
*/
-static enum ice_status
+static int
ice_rem_prof_id(struct ice_hw *hw, enum ice_block blk,
struct ice_vsig_prof *prof)
{
- enum ice_status status;
+ int status;
u16 i;
for (i = 0; i < prof->tcam_count; i++)
@@ -4701,7 +4832,7 @@ ice_rem_prof_id(struct ice_hw *hw, enum ice_block blk,
status = ice_rel_tcam_idx(hw, blk,
prof->tcam[i].tcam_idx);
if (status)
- return ICE_ERR_HW_TABLE;
+ return -EIO;
}
return 0;
@@ -4714,14 +4845,14 @@ ice_rem_prof_id(struct ice_hw *hw, enum ice_block blk,
* @vsig: the VSIG to remove
* @chg: the change list
*/
-static enum ice_status
+static int
ice_rem_vsig(struct ice_hw *hw, enum ice_block blk, u16 vsig,
struct list_head *chg)
{
u16 idx = vsig & ICE_VSIG_IDX_M;
struct ice_vsig_vsi *vsi_cur;
struct ice_vsig_prof *d, *t;
- enum ice_status status;
+ int status;
/* remove TCAM entries */
list_for_each_entry_safe(d, t,
@@ -4748,7 +4879,7 @@ ice_rem_vsig(struct ice_hw *hw, enum ice_block blk, u16 vsig,
p = devm_kzalloc(ice_hw_to_dev(hw), sizeof(*p),
GFP_KERNEL);
if (!p)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
p->type = ICE_VSIG_REM;
p->orig_vsig = vsig;
@@ -4771,13 +4902,13 @@ ice_rem_vsig(struct ice_hw *hw, enum ice_block blk, u16 vsig,
* @hdl: profile handle indicating which profile to remove
* @chg: list to receive a record of changes
*/
-static enum ice_status
+static int
ice_rem_prof_id_vsig(struct ice_hw *hw, enum ice_block blk, u16 vsig, u64 hdl,
struct list_head *chg)
{
u16 idx = vsig & ICE_VSIG_IDX_M;
struct ice_vsig_prof *p, *t;
- enum ice_status status;
+ int status;
list_for_each_entry_safe(p, t,
&hw->blk[blk].xlt2.vsig_tbl[idx].prop_lst,
@@ -4795,7 +4926,7 @@ ice_rem_prof_id_vsig(struct ice_hw *hw, enum ice_block blk, u16 vsig, u64 hdl,
return status;
}
- return ICE_ERR_DOES_NOT_EXIST;
+ return -ENOENT;
}
/**
@@ -4804,12 +4935,11 @@ ice_rem_prof_id_vsig(struct ice_hw *hw, enum ice_block blk, u16 vsig, u64 hdl,
* @blk: hardware block
* @id: profile tracking ID
*/
-static enum ice_status
-ice_rem_flow_all(struct ice_hw *hw, enum ice_block blk, u64 id)
+static int ice_rem_flow_all(struct ice_hw *hw, enum ice_block blk, u64 id)
{
struct ice_chs_chg *del, *tmp;
- enum ice_status status;
struct list_head chg;
+ int status;
u16 i;
INIT_LIST_HEAD(&chg);
@@ -4845,16 +4975,16 @@ ice_rem_flow_all(struct ice_hw *hw, enum ice_block blk, u64 id)
* previously created through ice_add_prof. If any existing entries
* are associated with this profile, they will be removed as well.
*/
-enum ice_status ice_rem_prof(struct ice_hw *hw, enum ice_block blk, u64 id)
+int ice_rem_prof(struct ice_hw *hw, enum ice_block blk, u64 id)
{
struct ice_prof_map *pmap;
- enum ice_status status;
+ int status;
mutex_lock(&hw->blk[blk].es.prof_map_lock);
pmap = ice_search_prof_id(hw, blk, id);
if (!pmap) {
- status = ICE_ERR_DOES_NOT_EXIST;
+ status = -ENOENT;
goto err_ice_rem_prof;
}
@@ -4881,20 +5011,20 @@ enum ice_status ice_rem_prof(struct ice_hw *hw, enum ice_block blk, u64 id)
* @hdl: profile handle
* @chg: change list
*/
-static enum ice_status
+static int
ice_get_prof(struct ice_hw *hw, enum ice_block blk, u64 hdl,
struct list_head *chg)
{
- enum ice_status status = 0;
struct ice_prof_map *map;
struct ice_chs_chg *p;
+ int status = 0;
u16 i;
mutex_lock(&hw->blk[blk].es.prof_map_lock);
/* Get the details on the profile specified by the handle ID */
map = ice_search_prof_id(hw, blk, hdl);
if (!map) {
- status = ICE_ERR_DOES_NOT_EXIST;
+ status = -ENOENT;
goto err_ice_get_prof;
}
@@ -4904,7 +5034,7 @@ ice_get_prof(struct ice_hw *hw, enum ice_block blk, u64 hdl,
p = devm_kzalloc(ice_hw_to_dev(hw), sizeof(*p),
GFP_KERNEL);
if (!p) {
- status = ICE_ERR_NO_MEMORY;
+ status = -ENOMEM;
goto err_ice_get_prof;
}
@@ -4936,7 +5066,7 @@ ice_get_prof(struct ice_hw *hw, enum ice_block blk, u64 hdl,
*
* This routine makes a copy of the list of profiles in the specified VSIG.
*/
-static enum ice_status
+static int
ice_get_profs_vsig(struct ice_hw *hw, enum ice_block blk, u16 vsig,
struct list_head *lst)
{
@@ -4964,7 +5094,7 @@ ice_get_profs_vsig(struct ice_hw *hw, enum ice_block blk, u16 vsig,
devm_kfree(ice_hw_to_dev(hw), ent1);
}
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
}
/**
@@ -4974,25 +5104,25 @@ ice_get_profs_vsig(struct ice_hw *hw, enum ice_block blk, u16 vsig,
* @lst: the list to be added to
* @hdl: profile handle of entry to add
*/
-static enum ice_status
+static int
ice_add_prof_to_lst(struct ice_hw *hw, enum ice_block blk,
struct list_head *lst, u64 hdl)
{
- enum ice_status status = 0;
struct ice_prof_map *map;
struct ice_vsig_prof *p;
+ int status = 0;
u16 i;
mutex_lock(&hw->blk[blk].es.prof_map_lock);
map = ice_search_prof_id(hw, blk, hdl);
if (!map) {
- status = ICE_ERR_DOES_NOT_EXIST;
+ status = -ENOENT;
goto err_ice_add_prof_to_lst;
}
p = devm_kzalloc(ice_hw_to_dev(hw), sizeof(*p), GFP_KERNEL);
if (!p) {
- status = ICE_ERR_NO_MEMORY;
+ status = -ENOMEM;
goto err_ice_add_prof_to_lst;
}
@@ -5021,17 +5151,17 @@ ice_add_prof_to_lst(struct ice_hw *hw, enum ice_block blk,
* @vsig: the VSIG to move the VSI to
* @chg: the change list
*/
-static enum ice_status
+static int
ice_move_vsi(struct ice_hw *hw, enum ice_block blk, u16 vsi, u16 vsig,
struct list_head *chg)
{
- enum ice_status status;
struct ice_chs_chg *p;
u16 orig_vsig;
+ int status;
p = devm_kzalloc(ice_hw_to_dev(hw), sizeof(*p), GFP_KERNEL);
if (!p)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
status = ice_vsig_find_vsi(hw, blk, vsi, &orig_vsig);
if (!status)
@@ -5081,13 +5211,13 @@ ice_rem_chg_tcam_ent(struct ice_hw *hw, u16 idx, struct list_head *chg)
*
* This function appends an enable or disable TCAM entry in the change log
*/
-static enum ice_status
+static int
ice_prof_tcam_ena_dis(struct ice_hw *hw, enum ice_block blk, bool enable,
u16 vsig, struct ice_tcam_inf *tcam,
struct list_head *chg)
{
- enum ice_status status;
struct ice_chs_chg *p;
+ int status;
u8 vl_msk[ICE_TCAM_KEY_VAL_SZ] = { 0xFF, 0xFF, 0xFF, 0xFF, 0xFF };
u8 dc_msk[ICE_TCAM_KEY_VAL_SZ] = { 0xFF, 0xFF, 0x00, 0x00, 0x00 };
@@ -5120,7 +5250,7 @@ ice_prof_tcam_ena_dis(struct ice_hw *hw, enum ice_block blk, bool enable,
/* add TCAM to change list */
p = devm_kzalloc(ice_hw_to_dev(hw), sizeof(*p), GFP_KERNEL);
if (!p)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
status = ice_tcam_write_entry(hw, blk, tcam->tcam_idx, tcam->prof_id,
tcam->ptg, vsig, 0, tcam->attr.flags,
@@ -5154,13 +5284,13 @@ ice_prof_tcam_ena_dis(struct ice_hw *hw, enum ice_block blk, bool enable,
* @vsig: the VSIG for which to adjust profile priorities
* @chg: the change list
*/
-static enum ice_status
+static int
ice_adj_prof_priorities(struct ice_hw *hw, enum ice_block blk, u16 vsig,
struct list_head *chg)
{
DECLARE_BITMAP(ptgs_used, ICE_XLT1_CNT);
struct ice_vsig_prof *t;
- enum ice_status status;
+ int status;
u16 idx;
bitmap_zero(ptgs_used, ICE_XLT1_CNT);
@@ -5209,7 +5339,7 @@ ice_adj_prof_priorities(struct ice_hw *hw, enum ice_block blk, u16 vsig,
}
/* keep track of used ptgs */
- set_bit(t->tcam[i].ptg, ptgs_used);
+ __set_bit(t->tcam[i].ptg, ptgs_used);
}
}
@@ -5225,7 +5355,7 @@ ice_adj_prof_priorities(struct ice_hw *hw, enum ice_block blk, u16 vsig,
* @rev: true to add entries to the end of the list
* @chg: the change list
*/
-static enum ice_status
+static int
ice_add_prof_id_vsig(struct ice_hw *hw, enum ice_block blk, u16 vsig, u64 hdl,
bool rev, struct list_head *chg)
{
@@ -5233,26 +5363,26 @@ ice_add_prof_id_vsig(struct ice_hw *hw, enum ice_block blk, u16 vsig, u64 hdl,
u8 vl_msk[ICE_TCAM_KEY_VAL_SZ] = { 0xFF, 0xFF, 0xFF, 0xFF, 0xFF };
u8 dc_msk[ICE_TCAM_KEY_VAL_SZ] = { 0xFF, 0xFF, 0x00, 0x00, 0x00 };
u8 nm_msk[ICE_TCAM_KEY_VAL_SZ] = { 0x00, 0x00, 0x00, 0x00, 0x00 };
- enum ice_status status = 0;
struct ice_prof_map *map;
struct ice_vsig_prof *t;
struct ice_chs_chg *p;
u16 vsig_idx, i;
+ int status = 0;
/* Error, if this VSIG already has this profile */
if (ice_has_prof_vsig(hw, blk, vsig, hdl))
- return ICE_ERR_ALREADY_EXISTS;
+ return -EEXIST;
/* new VSIG profile structure */
t = devm_kzalloc(ice_hw_to_dev(hw), sizeof(*t), GFP_KERNEL);
if (!t)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
mutex_lock(&hw->blk[blk].es.prof_map_lock);
/* Get the details on the profile specified by the handle ID */
map = ice_search_prof_id(hw, blk, hdl);
if (!map) {
- status = ICE_ERR_DOES_NOT_EXIST;
+ status = -ENOENT;
goto err_ice_add_prof_id_vsig;
}
@@ -5267,7 +5397,7 @@ ice_add_prof_id_vsig(struct ice_hw *hw, enum ice_block blk, u16 vsig, u64 hdl,
/* add TCAM to change list */
p = devm_kzalloc(ice_hw_to_dev(hw), sizeof(*p), GFP_KERNEL);
if (!p) {
- status = ICE_ERR_NO_MEMORY;
+ status = -ENOMEM;
goto err_ice_add_prof_id_vsig;
}
@@ -5337,21 +5467,21 @@ ice_add_prof_id_vsig(struct ice_hw *hw, enum ice_block blk, u16 vsig, u64 hdl,
* @hdl: the profile handle of the profile that will be added to the VSIG
* @chg: the change list
*/
-static enum ice_status
+static int
ice_create_prof_id_vsig(struct ice_hw *hw, enum ice_block blk, u16 vsi, u64 hdl,
struct list_head *chg)
{
- enum ice_status status;
struct ice_chs_chg *p;
u16 new_vsig;
+ int status;
p = devm_kzalloc(ice_hw_to_dev(hw), sizeof(*p), GFP_KERNEL);
if (!p)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
new_vsig = ice_vsig_alloc(hw, blk);
if (!new_vsig) {
- status = ICE_ERR_HW_TABLE;
+ status = -EIO;
goto err_ice_create_prof_id_vsig;
}
@@ -5387,18 +5517,18 @@ ice_create_prof_id_vsig(struct ice_hw *hw, enum ice_block blk, u16 vsi, u64 hdl,
* @new_vsig: return of new VSIG
* @chg: the change list
*/
-static enum ice_status
+static int
ice_create_vsig_from_lst(struct ice_hw *hw, enum ice_block blk, u16 vsi,
struct list_head *lst, u16 *new_vsig,
struct list_head *chg)
{
struct ice_vsig_prof *t;
- enum ice_status status;
+ int status;
u16 vsig;
vsig = ice_vsig_alloc(hw, blk);
if (!vsig)
- return ICE_ERR_HW_TABLE;
+ return -EIO;
status = ice_move_vsi(hw, blk, vsi, vsig, chg);
if (status)
@@ -5428,8 +5558,8 @@ static bool
ice_find_prof_vsig(struct ice_hw *hw, enum ice_block blk, u64 hdl, u16 *vsig)
{
struct ice_vsig_prof *t;
- enum ice_status status;
struct list_head lst;
+ int status;
INIT_LIST_HEAD(&lst);
@@ -5459,14 +5589,14 @@ ice_find_prof_vsig(struct ice_hw *hw, enum ice_block blk, u64 hdl, u16 *vsig)
* profile indicated by the ID parameter for the VSIs specified in the VSI
* array. Once successfully called, the flow will be enabled.
*/
-enum ice_status
+int
ice_add_prof_id_flow(struct ice_hw *hw, enum ice_block blk, u16 vsi, u64 hdl)
{
struct ice_vsig_prof *tmp1, *del1;
struct ice_chs_chg *tmp, *del;
struct list_head union_lst;
- enum ice_status status;
struct list_head chg;
+ int status;
u16 vsig;
INIT_LIST_HEAD(&union_lst);
@@ -5492,7 +5622,7 @@ ice_add_prof_id_flow(struct ice_hw *hw, enum ice_block blk, u16 vsi, u64 hdl)
* scenario
*/
if (ice_has_prof_vsig(hw, blk, vsig, hdl)) {
- status = ICE_ERR_ALREADY_EXISTS;
+ status = -EEXIST;
goto err_ice_add_prof_id_flow;
}
@@ -5600,7 +5730,7 @@ ice_add_prof_id_flow(struct ice_hw *hw, enum ice_block blk, u16 vsi, u64 hdl)
* @lst: list to remove the profile from
* @hdl: the profile handle indicating the profile to remove
*/
-static enum ice_status
+static int
ice_rem_prof_from_list(struct ice_hw *hw, struct list_head *lst, u64 hdl)
{
struct ice_vsig_prof *ent, *tmp;
@@ -5612,7 +5742,7 @@ ice_rem_prof_from_list(struct ice_hw *hw, struct list_head *lst, u64 hdl)
return 0;
}
- return ICE_ERR_DOES_NOT_EXIST;
+ return -ENOENT;
}
/**
@@ -5626,13 +5756,13 @@ ice_rem_prof_from_list(struct ice_hw *hw, struct list_head *lst, u64 hdl)
* profile indicated by the ID parameter for the VSIs specified in the VSI
* array. Once successfully called, the flow will be disabled.
*/
-enum ice_status
+int
ice_rem_prof_id_flow(struct ice_hw *hw, enum ice_block blk, u16 vsi, u64 hdl)
{
struct ice_vsig_prof *tmp1, *del1;
struct ice_chs_chg *tmp, *del;
struct list_head chg, copy;
- enum ice_status status;
+ int status;
u16 vsig;
INIT_LIST_HEAD(©);
@@ -5727,7 +5857,7 @@ ice_rem_prof_id_flow(struct ice_hw *hw, enum ice_block blk, u16 vsi, u64 hdl)
}
}
} else {
- status = ICE_ERR_DOES_NOT_EXIST;
+ status = -ENOENT;
}
/* update hardware tables */
diff --git a/drivers/net/ethernet/intel/ice/ice_flex_pipe.h b/drivers/net/ethernet/intel/ice/ice_flex_pipe.h
index a2863f3..6cbc29b 100644
--- a/drivers/net/ethernet/intel/ice/ice_flex_pipe.h
+++ b/drivers/net/ethernet/intel/ice/ice_flex_pipe.h
@@ -18,10 +18,67 @@
#define ICE_PKG_CNT 4
-enum ice_status
+enum ice_ddp_state {
+ /* Indicates that this call to ice_init_pkg
+ * successfully loaded the requested DDP package
+ */
+ ICE_DDP_PKG_SUCCESS = 0,
+
+ /* Generic error for already loaded errors, it is mapped later to
+ * the more specific one (one of the next 3)
+ */
+ ICE_DDP_PKG_ALREADY_LOADED = -1,
+
+ /* Indicates that a DDP package of the same version has already been
+ * loaded onto the device by a previous call or by another PF
+ */
+ ICE_DDP_PKG_SAME_VERSION_ALREADY_LOADED = -2,
+
+ /* The device has a DDP package that is not supported by the driver */
+ ICE_DDP_PKG_ALREADY_LOADED_NOT_SUPPORTED = -3,
+
+ /* The device has a compatible package
+ * (but different from the request) already loaded
+ */
+ ICE_DDP_PKG_COMPATIBLE_ALREADY_LOADED = -4,
+
+ /* The firmware loaded on the device is not compatible with
+ * the DDP package loaded
+ */
+ ICE_DDP_PKG_FW_MISMATCH = -5,
+
+ /* The DDP package file is invalid */
+ ICE_DDP_PKG_INVALID_FILE = -6,
+
+ /* The version of the DDP package provided is higher than
+ * the driver supports
+ */
+ ICE_DDP_PKG_FILE_VERSION_TOO_HIGH = -7,
+
+ /* The version of the DDP package provided is lower than the
+ * driver supports
+ */
+ ICE_DDP_PKG_FILE_VERSION_TOO_LOW = -8,
+
+ /* The signature of the DDP package file provided is invalid */
+ ICE_DDP_PKG_FILE_SIGNATURE_INVALID = -9,
+
+ /* The DDP package file security revision is too low and not
+ * supported by firmware
+ */
+ ICE_DDP_PKG_FILE_REVISION_TOO_LOW = -10,
+
+ /* An error occurred in firmware while loading the DDP package */
+ ICE_DDP_PKG_LOAD_ERROR = -11,
+
+ /* Other errors */
+ ICE_DDP_PKG_ERR = -12
+};
+
+int
ice_acquire_change_lock(struct ice_hw *hw, enum ice_aq_res_access_type access);
void ice_release_change_lock(struct ice_hw *hw);
-enum ice_status
+int
ice_find_prot_off(struct ice_hw *hw, enum ice_block blk, u8 prof, u16 fv_idx,
u8 *prot, u16 *off);
void
@@ -29,7 +86,7 @@ ice_get_sw_fv_bitmap(struct ice_hw *hw, enum ice_prof_type type,
unsigned long *bm);
void
ice_init_prof_result_bm(struct ice_hw *hw);
-enum ice_status
+int
ice_get_sw_fv_list(struct ice_hw *hw, u8 *prot_ids, u16 ids_cnt,
unsigned long *bm, struct list_head *fv_list);
bool
@@ -40,22 +97,26 @@ int ice_udp_tunnel_set_port(struct net_device *netdev, unsigned int table,
int ice_udp_tunnel_unset_port(struct net_device *netdev, unsigned int table,
unsigned int idx, struct udp_tunnel_info *ti);
-enum ice_status
+/* Rx parser PTYPE functions */
+bool ice_hw_ptype_ena(struct ice_hw *hw, u16 ptype);
+
+/* XLT2/VSI group functions */
+int
ice_add_prof(struct ice_hw *hw, enum ice_block blk, u64 id, u8 ptypes[],
const struct ice_ptype_attributes *attr, u16 attr_cnt,
struct ice_fv_word *es, u16 *masks);
-enum ice_status
+int
ice_add_prof_id_flow(struct ice_hw *hw, enum ice_block blk, u16 vsi, u64 hdl);
-enum ice_status
+int
ice_rem_prof_id_flow(struct ice_hw *hw, enum ice_block blk, u16 vsi, u64 hdl);
-enum ice_status ice_init_pkg(struct ice_hw *hw, u8 *buff, u32 len);
-enum ice_status
+enum ice_ddp_state ice_init_pkg(struct ice_hw *hw, u8 *buff, u32 len);
+enum ice_ddp_state
ice_copy_and_init_pkg(struct ice_hw *hw, const u8 *buf, u32 len);
-enum ice_status ice_init_hw_tbls(struct ice_hw *hw);
+bool ice_is_init_pkg_successful(enum ice_ddp_state state);
+int ice_init_hw_tbls(struct ice_hw *hw);
void ice_free_seg(struct ice_hw *hw);
void ice_fill_blk_tbls(struct ice_hw *hw);
void ice_clear_hw_tbls(struct ice_hw *hw);
void ice_free_hw_tbls(struct ice_hw *hw);
-enum ice_status
-ice_rem_prof(struct ice_hw *hw, enum ice_block blk, u64 id);
+int ice_rem_prof(struct ice_hw *hw, enum ice_block blk, u64 id);
#endif /* _ICE_FLEX_PIPE_H_ */
diff --git a/drivers/net/ethernet/intel/ice/ice_flex_type.h b/drivers/net/ethernet/intel/ice/ice_flex_type.h
index 0f572a3..fc087e0 100644
--- a/drivers/net/ethernet/intel/ice/ice_flex_type.h
+++ b/drivers/net/ethernet/intel/ice/ice_flex_type.h
@@ -160,6 +160,7 @@ struct ice_meta_sect {
#define ICE_SID_CDID_KEY_BUILDER_RSS 47
#define ICE_SID_CDID_REDIR_RSS 48
+#define ICE_SID_RXPARSER_MARKER_PTYPE 55
#define ICE_SID_RXPARSER_BOOST_TCAM 56
#define ICE_SID_TXPARSER_BOOST_TCAM 66
@@ -201,6 +202,24 @@ enum ice_sect {
ICE_SECT_COUNT
};
+/* Packet Type (PTYPE) values */
+#define ICE_PTYPE_MAC_PAY 1
+#define ICE_PTYPE_IPV4_PAY 23
+#define ICE_PTYPE_IPV4_UDP_PAY 24
+#define ICE_PTYPE_IPV4_TCP_PAY 26
+#define ICE_PTYPE_IPV4_SCTP_PAY 27
+#define ICE_PTYPE_IPV6_PAY 89
+#define ICE_PTYPE_IPV6_UDP_PAY 90
+#define ICE_PTYPE_IPV6_TCP_PAY 92
+#define ICE_PTYPE_IPV6_SCTP_PAY 93
+#define ICE_MAC_IPV4_ESP 160
+#define ICE_MAC_IPV6_ESP 161
+#define ICE_MAC_IPV4_AH 162
+#define ICE_MAC_IPV6_AH 163
+#define ICE_MAC_IPV4_NAT_T_ESP 164
+#define ICE_MAC_IPV6_NAT_T_ESP 165
+#define ICE_MAC_IPV4_GTPU 329
+#define ICE_MAC_IPV6_GTPU 330
#define ICE_MAC_IPV4_GTPU_IPV4_FRAG 331
#define ICE_MAC_IPV4_GTPU_IPV4_PAY 332
#define ICE_MAC_IPV4_GTPU_IPV4_UDP_PAY 333
@@ -221,6 +240,10 @@ enum ice_sect {
#define ICE_MAC_IPV6_GTPU_IPV6_UDP_PAY 348
#define ICE_MAC_IPV6_GTPU_IPV6_TCP 349
#define ICE_MAC_IPV6_GTPU_IPV6_ICMPV6 350
+#define ICE_MAC_IPV4_PFCP_SESSION 352
+#define ICE_MAC_IPV6_PFCP_SESSION 354
+#define ICE_MAC_IPV4_L2TPV3 360
+#define ICE_MAC_IPV6_L2TPV3 361
/* Attributes that can modify PTYPE definitions.
*
@@ -329,6 +352,25 @@ struct ice_boost_tcam_section {
sizeof(struct ice_boost_tcam_entry), \
sizeof(struct ice_boost_tcam_entry))
+/* package Marker Ptype TCAM entry */
+struct ice_marker_ptype_tcam_entry {
+#define ICE_MARKER_PTYPE_TCAM_ADDR_MAX 1024
+ __le16 addr;
+ __le16 ptype;
+ u8 keys[20];
+};
+
+struct ice_marker_ptype_tcam_section {
+ __le16 count;
+ __le16 reserved;
+ struct ice_marker_ptype_tcam_entry tcam[];
+};
+
+#define ICE_MAX_MARKER_PTYPE_TCAMS_IN_BUF \
+ ICE_MAX_ENTRIES_IN_BUF(struct_size((struct ice_marker_ptype_tcam_section *)0, tcam, 1) - \
+ sizeof(struct ice_marker_ptype_tcam_entry), \
+ sizeof(struct ice_marker_ptype_tcam_entry))
+
struct ice_xlt1_section {
__le16 count;
__le16 offset;
diff --git a/drivers/net/ethernet/intel/ice/ice_flow.c b/drivers/net/ethernet/intel/ice/ice_flow.c
index f160672..beed483 100644
--- a/drivers/net/ethernet/intel/ice/ice_flow.c
+++ b/drivers/net/ethernet/intel/ice/ice_flow.c
@@ -609,8 +609,6 @@ struct ice_flow_prof_params {
ICE_FLOW_SEG_HDR_ESP | ICE_FLOW_SEG_HDR_AH | \
ICE_FLOW_SEG_HDR_NAT_T_ESP)
-#define ICE_FLOW_SEG_HDRS_L2_MASK \
- (ICE_FLOW_SEG_HDR_ETH | ICE_FLOW_SEG_HDR_VLAN)
#define ICE_FLOW_SEG_HDRS_L3_MASK \
(ICE_FLOW_SEG_HDR_IPV4 | ICE_FLOW_SEG_HDR_IPV6 | ICE_FLOW_SEG_HDR_ARP)
#define ICE_FLOW_SEG_HDRS_L4_MASK \
@@ -625,8 +623,7 @@ struct ice_flow_prof_params {
* @segs: array of one or more packet segments that describe the flow
* @segs_cnt: number of packet segments provided
*/
-static enum ice_status
-ice_flow_val_hdrs(struct ice_flow_seg_info *segs, u8 segs_cnt)
+static int ice_flow_val_hdrs(struct ice_flow_seg_info *segs, u8 segs_cnt)
{
u8 i;
@@ -634,12 +631,12 @@ ice_flow_val_hdrs(struct ice_flow_seg_info *segs, u8 segs_cnt)
/* Multiple L3 headers */
if (segs[i].hdrs & ICE_FLOW_SEG_HDRS_L3_MASK &&
!is_power_of_2(segs[i].hdrs & ICE_FLOW_SEG_HDRS_L3_MASK))
- return ICE_ERR_PARAM;
+ return -EINVAL;
/* Multiple L4 headers */
if (segs[i].hdrs & ICE_FLOW_SEG_HDRS_L4_MASK &&
!is_power_of_2(segs[i].hdrs & ICE_FLOW_SEG_HDRS_L4_MASK))
- return ICE_ERR_PARAM;
+ return -EINVAL;
}
return 0;
@@ -700,8 +697,7 @@ static u16 ice_flow_calc_seg_sz(struct ice_flow_prof_params *params, u8 seg)
* This function identifies the packet types associated with the protocol
* headers being present in packet segments of the specified flow profile.
*/
-static enum ice_status
-ice_flow_proc_seg_hdrs(struct ice_flow_prof_params *params)
+static int ice_flow_proc_seg_hdrs(struct ice_flow_prof_params *params)
{
struct ice_flow_prof *prof;
u8 i;
@@ -898,7 +894,7 @@ ice_flow_proc_seg_hdrs(struct ice_flow_prof_params *params)
* field. It then allocates one or more extraction sequence entries for the
* given field, and fill the entries with protocol ID and offset information.
*/
-static enum ice_status
+static int
ice_flow_xtract_fld(struct ice_hw *hw, struct ice_flow_prof_params *params,
u8 seg, enum ice_flow_field fld, u64 match)
{
@@ -1035,7 +1031,7 @@ ice_flow_xtract_fld(struct ice_hw *hw, struct ice_flow_prof_params *params,
prot_id = ICE_PROT_GRE_OF;
break;
default:
- return ICE_ERR_NOT_IMPL;
+ return -EOPNOTSUPP;
}
/* Each extraction sequence entry is a word in size, and extracts a
@@ -1073,7 +1069,7 @@ ice_flow_xtract_fld(struct ice_hw *hw, struct ice_flow_prof_params *params,
* does not exceed the block's capability
*/
if (params->es_cnt >= fv_words)
- return ICE_ERR_MAX_LIMIT;
+ return -ENOSPC;
/* some blocks require a reversed field vector layout */
if (hw->blk[params->blk].es.reverse)
@@ -1099,7 +1095,7 @@ ice_flow_xtract_fld(struct ice_hw *hw, struct ice_flow_prof_params *params,
* @params: information about the flow to be processed
* @seg: index of packet segment whose raw fields are to be extracted
*/
-static enum ice_status
+static int
ice_flow_xtract_raws(struct ice_hw *hw, struct ice_flow_prof_params *params,
u8 seg)
{
@@ -1112,12 +1108,12 @@ ice_flow_xtract_raws(struct ice_hw *hw, struct ice_flow_prof_params *params,
if (params->prof->segs[seg].raws_cnt >
ARRAY_SIZE(params->prof->segs[seg].raws))
- return ICE_ERR_MAX_LIMIT;
+ return -ENOSPC;
/* Offsets within the segment headers are not supported */
hdrs_sz = ice_flow_calc_seg_sz(params, seg);
if (!hdrs_sz)
- return ICE_ERR_PARAM;
+ return -EINVAL;
fv_words = hw->blk[params->blk].es.fvw;
@@ -1150,7 +1146,7 @@ ice_flow_xtract_raws(struct ice_hw *hw, struct ice_flow_prof_params *params,
*/
if (params->es_cnt >= hw->blk[params->blk].es.count ||
params->es_cnt >= ICE_MAX_FV_WORDS)
- return ICE_ERR_MAX_LIMIT;
+ return -ENOSPC;
/* some blocks require a reversed field vector layout */
if (hw->blk[params->blk].es.reverse)
@@ -1176,12 +1172,12 @@ ice_flow_xtract_raws(struct ice_hw *hw, struct ice_flow_prof_params *params,
* This function iterates through all matched fields in the given segments, and
* creates an extraction sequence for the fields.
*/
-static enum ice_status
+static int
ice_flow_create_xtrct_seq(struct ice_hw *hw,
struct ice_flow_prof_params *params)
{
struct ice_flow_prof *prof = params->prof;
- enum ice_status status = 0;
+ int status = 0;
u8 i;
for (i = 0; i < prof->segs_cnt; i++) {
@@ -1210,10 +1206,10 @@ ice_flow_create_xtrct_seq(struct ice_hw *hw,
* @hw: pointer to the HW struct
* @params: information about the flow to be processed
*/
-static enum ice_status
+static int
ice_flow_proc_segs(struct ice_hw *hw, struct ice_flow_prof_params *params)
{
- enum ice_status status;
+ int status;
status = ice_flow_proc_seg_hdrs(params);
if (status)
@@ -1229,7 +1225,7 @@ ice_flow_proc_segs(struct ice_hw *hw, struct ice_flow_prof_params *params)
status = 0;
break;
default:
- return ICE_ERR_NOT_IMPL;
+ return -EOPNOTSUPP;
}
return status;
@@ -1329,12 +1325,12 @@ ice_dealloc_flow_entry(struct ice_hw *hw, struct ice_flow_entry *entry)
* @blk: classification stage
* @entry: flow entry to be removed
*/
-static enum ice_status
+static int
ice_flow_rem_entry_sync(struct ice_hw *hw, enum ice_block __always_unused blk,
struct ice_flow_entry *entry)
{
if (!entry)
- return ICE_ERR_BAD_PTR;
+ return -EINVAL;
list_del(&entry->l_entry);
@@ -1355,27 +1351,27 @@ ice_flow_rem_entry_sync(struct ice_hw *hw, enum ice_block __always_unused blk,
*
* Assumption: the caller has acquired the lock to the profile list
*/
-static enum ice_status
+static int
ice_flow_add_prof_sync(struct ice_hw *hw, enum ice_block blk,
enum ice_flow_dir dir, u64 prof_id,
struct ice_flow_seg_info *segs, u8 segs_cnt,
struct ice_flow_prof **prof)
{
struct ice_flow_prof_params *params;
- enum ice_status status;
+ int status;
u8 i;
if (!prof)
- return ICE_ERR_BAD_PTR;
+ return -EINVAL;
params = kzalloc(sizeof(*params), GFP_KERNEL);
if (!params)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
params->prof = devm_kzalloc(ice_hw_to_dev(hw), sizeof(*params->prof),
GFP_KERNEL);
if (!params->prof) {
- status = ICE_ERR_NO_MEMORY;
+ status = -ENOMEM;
goto free_params;
}
@@ -1432,11 +1428,11 @@ ice_flow_add_prof_sync(struct ice_hw *hw, enum ice_block blk,
*
* Assumption: the caller has acquired the lock to the profile list
*/
-static enum ice_status
+static int
ice_flow_rem_prof_sync(struct ice_hw *hw, enum ice_block blk,
struct ice_flow_prof *prof)
{
- enum ice_status status;
+ int status;
/* Remove all remaining flow entries before removing the flow profile */
if (!list_empty(&prof->entries)) {
@@ -1474,11 +1470,11 @@ ice_flow_rem_prof_sync(struct ice_hw *hw, enum ice_block blk,
* Assumption: the caller has acquired the lock to the profile list
* and the software VSI handle has been validated
*/
-static enum ice_status
+static int
ice_flow_assoc_prof(struct ice_hw *hw, enum ice_block blk,
struct ice_flow_prof *prof, u16 vsi_handle)
{
- enum ice_status status = 0;
+ int status = 0;
if (!test_bit(vsi_handle, prof->vsis)) {
status = ice_add_prof_id_flow(hw, blk,
@@ -1505,11 +1501,11 @@ ice_flow_assoc_prof(struct ice_hw *hw, enum ice_block blk,
* Assumption: the caller has acquired the lock to the profile list
* and the software VSI handle has been validated
*/
-static enum ice_status
+static int
ice_flow_disassoc_prof(struct ice_hw *hw, enum ice_block blk,
struct ice_flow_prof *prof, u16 vsi_handle)
{
- enum ice_status status = 0;
+ int status = 0;
if (test_bit(vsi_handle, prof->vsis)) {
status = ice_rem_prof_id_flow(hw, blk,
@@ -1536,21 +1532,21 @@ ice_flow_disassoc_prof(struct ice_hw *hw, enum ice_block blk,
* @segs_cnt: number of packet segments provided
* @prof: stores the returned flow profile added
*/
-enum ice_status
+int
ice_flow_add_prof(struct ice_hw *hw, enum ice_block blk, enum ice_flow_dir dir,
u64 prof_id, struct ice_flow_seg_info *segs, u8 segs_cnt,
struct ice_flow_prof **prof)
{
- enum ice_status status;
+ int status;
if (segs_cnt > ICE_FLOW_SEG_MAX)
- return ICE_ERR_MAX_LIMIT;
+ return -ENOSPC;
if (!segs_cnt)
- return ICE_ERR_PARAM;
+ return -EINVAL;
if (!segs)
- return ICE_ERR_BAD_PTR;
+ return -EINVAL;
status = ice_flow_val_hdrs(segs, segs_cnt);
if (status)
@@ -1574,17 +1570,16 @@ ice_flow_add_prof(struct ice_hw *hw, enum ice_block blk, enum ice_flow_dir dir,
* @blk: the block for which the flow profile is to be removed
* @prof_id: unique ID of the flow profile to be removed
*/
-enum ice_status
-ice_flow_rem_prof(struct ice_hw *hw, enum ice_block blk, u64 prof_id)
+int ice_flow_rem_prof(struct ice_hw *hw, enum ice_block blk, u64 prof_id)
{
struct ice_flow_prof *prof;
- enum ice_status status;
+ int status;
mutex_lock(&hw->fl_profs_locks[blk]);
prof = ice_flow_find_prof_id(hw, blk, prof_id);
if (!prof) {
- status = ICE_ERR_DOES_NOT_EXIST;
+ status = -ENOENT;
goto out;
}
@@ -1608,34 +1603,34 @@ ice_flow_rem_prof(struct ice_hw *hw, enum ice_block blk, u64 prof_id)
* @data: pointer to a data buffer containing flow entry's match values/masks
* @entry_h: pointer to buffer that receives the new flow entry's handle
*/
-enum ice_status
+int
ice_flow_add_entry(struct ice_hw *hw, enum ice_block blk, u64 prof_id,
u64 entry_id, u16 vsi_handle, enum ice_flow_priority prio,
void *data, u64 *entry_h)
{
struct ice_flow_entry *e = NULL;
struct ice_flow_prof *prof;
- enum ice_status status;
+ int status;
/* No flow entry data is expected for RSS */
if (!entry_h || (!data && blk != ICE_BLK_RSS))
- return ICE_ERR_BAD_PTR;
+ return -EINVAL;
if (!ice_is_vsi_valid(hw, vsi_handle))
- return ICE_ERR_PARAM;
+ return -EINVAL;
mutex_lock(&hw->fl_profs_locks[blk]);
prof = ice_flow_find_prof_id(hw, blk, prof_id);
if (!prof) {
- status = ICE_ERR_DOES_NOT_EXIST;
+ status = -ENOENT;
} else {
/* Allocate memory for the entry being added and associate
* the VSI to the found flow profile
*/
e = devm_kzalloc(ice_hw_to_dev(hw), sizeof(*e), GFP_KERNEL);
if (!e)
- status = ICE_ERR_NO_MEMORY;
+ status = -ENOMEM;
else
status = ice_flow_assoc_prof(hw, blk, prof, vsi_handle);
}
@@ -1654,7 +1649,7 @@ ice_flow_add_entry(struct ice_hw *hw, enum ice_block blk, u64 prof_id,
case ICE_BLK_RSS:
break;
default:
- status = ICE_ERR_NOT_IMPL;
+ status = -EOPNOTSUPP;
goto out;
}
@@ -1680,15 +1675,14 @@ ice_flow_add_entry(struct ice_hw *hw, enum ice_block blk, u64 prof_id,
* @blk: classification stage
* @entry_h: handle to the flow entry to be removed
*/
-enum ice_status ice_flow_rem_entry(struct ice_hw *hw, enum ice_block blk,
- u64 entry_h)
+int ice_flow_rem_entry(struct ice_hw *hw, enum ice_block blk, u64 entry_h)
{
struct ice_flow_entry *entry;
struct ice_flow_prof *prof;
- enum ice_status status = 0;
+ int status = 0;
if (entry_h == ICE_FLOW_ENTRY_HANDLE_INVAL)
- return ICE_ERR_PARAM;
+ return -EINVAL;
entry = ICE_FLOW_ENTRY_PTR(entry_h);
@@ -1812,6 +1806,57 @@ ice_flow_add_fld_raw(struct ice_flow_seg_info *seg, u16 off, u8 len,
seg->raws_cnt++;
}
+/**
+ * ice_flow_rem_vsi_prof - remove VSI from flow profile
+ * @hw: pointer to the hardware structure
+ * @vsi_handle: software VSI handle
+ * @prof_id: unique ID to identify this flow profile
+ *
+ * This function removes the flow entries associated to the input
+ * VSI handle and disassociate the VSI from the flow profile.
+ */
+int ice_flow_rem_vsi_prof(struct ice_hw *hw, u16 vsi_handle, u64 prof_id)
+{
+ struct ice_flow_prof *prof;
+ int status = 0;
+
+ if (!ice_is_vsi_valid(hw, vsi_handle))
+ return -EINVAL;
+
+ /* find flow profile pointer with input package block and profile ID */
+ prof = ice_flow_find_prof_id(hw, ICE_BLK_FD, prof_id);
+ if (!prof) {
+ ice_debug(hw, ICE_DBG_PKG, "Cannot find flow profile id=%llu\n",
+ prof_id);
+ return -ENOENT;
+ }
+
+ /* Remove all remaining flow entries before removing the flow profile */
+ if (!list_empty(&prof->entries)) {
+ struct ice_flow_entry *e, *t;
+
+ mutex_lock(&prof->entries_lock);
+ list_for_each_entry_safe(e, t, &prof->entries, l_entry) {
+ if (e->vsi_handle != vsi_handle)
+ continue;
+
+ status = ice_flow_rem_entry_sync(hw, ICE_BLK_FD, e);
+ if (status)
+ break;
+ }
+ mutex_unlock(&prof->entries_lock);
+ }
+ if (status)
+ return status;
+
+ /* disassociate the flow profile from sw VSI handle */
+ status = ice_flow_disassoc_prof(hw, ICE_BLK_FD, prof, vsi_handle);
+ if (status)
+ ice_debug(hw, ICE_DBG_PKG, "ice_flow_disassoc_prof() failed with status=%d\n",
+ status);
+ return status;
+}
+
#define ICE_FLOW_RSS_SEG_HDR_L2_MASKS \
(ICE_FLOW_SEG_HDR_ETH | ICE_FLOW_SEG_HDR_VLAN)
@@ -1836,7 +1881,7 @@ ice_flow_add_fld_raw(struct ice_flow_seg_info *seg, u16 off, u8 len,
* header value to set flow field segment for further use in flow
* profile entry or removal.
*/
-static enum ice_status
+static int
ice_flow_set_rss_seg_info(struct ice_flow_seg_info *segs, u64 hash_fields,
u32 flow_hdr)
{
@@ -1853,15 +1898,15 @@ ice_flow_set_rss_seg_info(struct ice_flow_seg_info *segs, u64 hash_fields,
if (segs->hdrs & ~ICE_FLOW_RSS_SEG_HDR_VAL_MASKS &
~ICE_FLOW_RSS_HDRS_INNER_MASK & ~ICE_FLOW_SEG_HDR_IPV_OTHER)
- return ICE_ERR_PARAM;
+ return -EINVAL;
val = (u64)(segs->hdrs & ICE_FLOW_RSS_SEG_HDR_L3_MASKS);
if (val && !is_power_of_2(val))
- return ICE_ERR_CFG;
+ return -EIO;
val = (u64)(segs->hdrs & ICE_FLOW_RSS_SEG_HDR_L4_MASKS);
if (val && !is_power_of_2(val))
- return ICE_ERR_CFG;
+ return -EIO;
return 0;
}
@@ -1899,14 +1944,14 @@ void ice_rem_vsi_rss_list(struct ice_hw *hw, u16 vsi_handle)
* the VSI from that profile. If the flow profile has no VSIs it will
* be removed.
*/
-enum ice_status ice_rem_vsi_rss_cfg(struct ice_hw *hw, u16 vsi_handle)
+int ice_rem_vsi_rss_cfg(struct ice_hw *hw, u16 vsi_handle)
{
const enum ice_block blk = ICE_BLK_RSS;
struct ice_flow_prof *p, *t;
- enum ice_status status = 0;
+ int status = 0;
if (!ice_is_vsi_valid(hw, vsi_handle))
- return ICE_ERR_PARAM;
+ return -EINVAL;
if (list_empty(&hw->fl_profs[blk]))
return 0;
@@ -1966,7 +2011,7 @@ ice_rem_rss_list(struct ice_hw *hw, u16 vsi_handle, struct ice_flow_prof *prof)
*
* Assumption: lock has already been acquired for RSS list
*/
-static enum ice_status
+static int
ice_add_rss_list(struct ice_hw *hw, u16 vsi_handle, struct ice_flow_prof *prof)
{
struct ice_rss_cfg *r, *rss_cfg;
@@ -1981,7 +2026,7 @@ ice_add_rss_list(struct ice_hw *hw, u16 vsi_handle, struct ice_flow_prof *prof)
rss_cfg = devm_kzalloc(ice_hw_to_dev(hw), sizeof(*rss_cfg),
GFP_KERNEL);
if (!rss_cfg)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
rss_cfg->hashed_flds = prof->segs[prof->segs_cnt - 1].match;
rss_cfg->packet_hdr = prof->segs[prof->segs_cnt - 1].hdrs;
@@ -2022,21 +2067,21 @@ ice_add_rss_list(struct ice_hw *hw, u16 vsi_handle, struct ice_flow_prof *prof)
*
* Assumption: lock has already been acquired for RSS list
*/
-static enum ice_status
+static int
ice_add_rss_cfg_sync(struct ice_hw *hw, u16 vsi_handle, u64 hashed_flds,
u32 addl_hdrs, u8 segs_cnt)
{
const enum ice_block blk = ICE_BLK_RSS;
struct ice_flow_prof *prof = NULL;
struct ice_flow_seg_info *segs;
- enum ice_status status;
+ int status;
if (!segs_cnt || segs_cnt > ICE_FLOW_SEG_MAX)
- return ICE_ERR_PARAM;
+ return -EINVAL;
segs = kcalloc(segs_cnt, sizeof(*segs), GFP_KERNEL);
if (!segs)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
/* Construct the packet segment info from the hashed fields */
status = ice_flow_set_rss_seg_info(&segs[segs_cnt - 1], hashed_flds,
@@ -2128,15 +2173,15 @@ ice_add_rss_cfg_sync(struct ice_hw *hw, u16 vsi_handle, u64 hashed_flds,
* the input fields to hash on, the flow type and use the VSI number to add
* a flow entry to the profile.
*/
-enum ice_status
+int
ice_add_rss_cfg(struct ice_hw *hw, u16 vsi_handle, u64 hashed_flds,
u32 addl_hdrs)
{
- enum ice_status status;
+ int status;
if (hashed_flds == ICE_HASH_INVALID ||
!ice_is_vsi_valid(hw, vsi_handle))
- return ICE_ERR_PARAM;
+ return -EINVAL;
mutex_lock(&hw->rss_locks);
status = ice_add_rss_cfg_sync(hw, vsi_handle, hashed_flds, addl_hdrs,
@@ -2159,18 +2204,18 @@ ice_add_rss_cfg(struct ice_hw *hw, u16 vsi_handle, u64 hashed_flds,
*
* Assumption: lock has already been acquired for RSS list
*/
-static enum ice_status
+static int
ice_rem_rss_cfg_sync(struct ice_hw *hw, u16 vsi_handle, u64 hashed_flds,
u32 addl_hdrs, u8 segs_cnt)
{
const enum ice_block blk = ICE_BLK_RSS;
struct ice_flow_seg_info *segs;
struct ice_flow_prof *prof;
- enum ice_status status;
+ int status;
segs = kcalloc(segs_cnt, sizeof(*segs), GFP_KERNEL);
if (!segs)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
/* Construct the packet segment info from the hashed fields */
status = ice_flow_set_rss_seg_info(&segs[segs_cnt - 1], hashed_flds,
@@ -2182,7 +2227,7 @@ ice_rem_rss_cfg_sync(struct ice_hw *hw, u16 vsi_handle, u64 hashed_flds,
vsi_handle,
ICE_FLOW_FIND_PROF_CHK_FLDS);
if (!prof) {
- status = ICE_ERR_DOES_NOT_EXIST;
+ status = -ENOENT;
goto out;
}
@@ -2216,15 +2261,15 @@ ice_rem_rss_cfg_sync(struct ice_hw *hw, u16 vsi_handle, u64 hashed_flds,
* removed. Calls are made to underlying flow s which will APIs
* turn build or update buffers for RSS XLT1 section.
*/
-enum ice_status __maybe_unused
+int __maybe_unused
ice_rem_rss_cfg(struct ice_hw *hw, u16 vsi_handle, u64 hashed_flds,
u32 addl_hdrs)
{
- enum ice_status status;
+ int status;
if (hashed_flds == ICE_HASH_INVALID ||
!ice_is_vsi_valid(hw, vsi_handle))
- return ICE_ERR_PARAM;
+ return -EINVAL;
mutex_lock(&hw->rss_locks);
status = ice_rem_rss_cfg_sync(hw, vsi_handle, hashed_flds, addl_hdrs,
@@ -2279,20 +2324,19 @@ ice_rem_rss_cfg(struct ice_hw *hw, u16 vsi_handle, u64 hashed_flds,
* message, convert it to ICE-compatible values, and configure RSS flow
* profiles.
*/
-enum ice_status
-ice_add_avf_rss_cfg(struct ice_hw *hw, u16 vsi_handle, u64 avf_hash)
+int ice_add_avf_rss_cfg(struct ice_hw *hw, u16 vsi_handle, u64 avf_hash)
{
- enum ice_status status = 0;
+ int status = 0;
u64 hash_flds;
if (avf_hash == ICE_AVF_FLOW_FIELD_INVALID ||
!ice_is_vsi_valid(hw, vsi_handle))
- return ICE_ERR_PARAM;
+ return -EINVAL;
/* Make sure no unsupported bits are specified */
if (avf_hash & ~(ICE_FLOW_AVF_RSS_ALL_IPV4_MASKS |
ICE_FLOW_AVF_RSS_ALL_IPV6_MASKS))
- return ICE_ERR_CFG;
+ return -EIO;
hash_flds = avf_hash;
@@ -2352,7 +2396,7 @@ ice_add_avf_rss_cfg(struct ice_hw *hw, u16 vsi_handle, u64 avf_hash)
}
if (rss_hash == ICE_HASH_INVALID)
- return ICE_ERR_OUT_OF_RANGE;
+ return -EIO;
status = ice_add_rss_cfg(hw, vsi_handle, rss_hash,
ICE_FLOW_SEG_HDR_NONE);
@@ -2368,13 +2412,13 @@ ice_add_avf_rss_cfg(struct ice_hw *hw, u16 vsi_handle, u64 avf_hash)
* @hw: pointer to the hardware structure
* @vsi_handle: software VSI handle
*/
-enum ice_status ice_replay_rss_cfg(struct ice_hw *hw, u16 vsi_handle)
+int ice_replay_rss_cfg(struct ice_hw *hw, u16 vsi_handle)
{
- enum ice_status status = 0;
struct ice_rss_cfg *r;
+ int status = 0;
if (!ice_is_vsi_valid(hw, vsi_handle))
- return ICE_ERR_PARAM;
+ return -EINVAL;
mutex_lock(&hw->rss_locks);
list_for_each_entry(r, &hw->rss_list_head, l_entry) {
diff --git a/drivers/net/ethernet/intel/ice/ice_flow.h b/drivers/net/ethernet/intel/ice/ice_flow.h
index 2a2d8c1..84b6e44 100644
--- a/drivers/net/ethernet/intel/ice/ice_flow.h
+++ b/drivers/net/ethernet/intel/ice/ice_flow.h
@@ -383,33 +383,31 @@ struct ice_rss_cfg {
u32 packet_hdr;
};
-enum ice_status
+int
ice_flow_add_prof(struct ice_hw *hw, enum ice_block blk, enum ice_flow_dir dir,
u64 prof_id, struct ice_flow_seg_info *segs, u8 segs_cnt,
struct ice_flow_prof **prof);
-enum ice_status
-ice_flow_rem_prof(struct ice_hw *hw, enum ice_block blk, u64 prof_id);
-enum ice_status
+int ice_flow_rem_prof(struct ice_hw *hw, enum ice_block blk, u64 prof_id);
+int
ice_flow_add_entry(struct ice_hw *hw, enum ice_block blk, u64 prof_id,
u64 entry_id, u16 vsi, enum ice_flow_priority prio,
void *data, u64 *entry_h);
-enum ice_status
-ice_flow_rem_entry(struct ice_hw *hw, enum ice_block blk, u64 entry_h);
+int ice_flow_rem_entry(struct ice_hw *hw, enum ice_block blk, u64 entry_h);
void
ice_flow_set_fld(struct ice_flow_seg_info *seg, enum ice_flow_field fld,
u16 val_loc, u16 mask_loc, u16 last_loc, bool range);
void
ice_flow_add_fld_raw(struct ice_flow_seg_info *seg, u16 off, u8 len,
u16 val_loc, u16 mask_loc);
+int ice_flow_rem_vsi_prof(struct ice_hw *hw, u16 vsi_handle, u64 prof_id);
void ice_rem_vsi_rss_list(struct ice_hw *hw, u16 vsi_handle);
-enum ice_status ice_replay_rss_cfg(struct ice_hw *hw, u16 vsi_handle);
-enum ice_status
-ice_add_avf_rss_cfg(struct ice_hw *hw, u16 vsi_handle, u64 hashed_flds);
-enum ice_status ice_rem_vsi_rss_cfg(struct ice_hw *hw, u16 vsi_handle);
-enum ice_status
+int ice_replay_rss_cfg(struct ice_hw *hw, u16 vsi_handle);
+int ice_add_avf_rss_cfg(struct ice_hw *hw, u16 vsi_handle, u64 hashed_flds);
+int ice_rem_vsi_rss_cfg(struct ice_hw *hw, u16 vsi_handle);
+int
ice_add_rss_cfg(struct ice_hw *hw, u16 vsi_handle, u64 hashed_flds,
u32 addl_hdrs);
-enum ice_status
+int
ice_rem_rss_cfg(struct ice_hw *hw, u16 vsi_handle, u64 hashed_flds,
u32 addl_hdrs);
u64 ice_get_rss_cfg(struct ice_hw *hw, u16 vsi_handle, u32 hdrs);
diff --git a/drivers/net/ethernet/intel/ice/ice_fltr.c b/drivers/net/ethernet/intel/ice/ice_fltr.c
index c2e78ea..c29177c 100644
--- a/drivers/net/ethernet/intel/ice/ice_fltr.c
+++ b/drivers/net/ethernet/intel/ice/ice_fltr.c
@@ -47,12 +47,69 @@ ice_fltr_add_entry_to_list(struct device *dev, struct ice_fltr_info *info,
}
/**
+ * ice_fltr_set_vlan_vsi_promisc
+ * @hw: pointer to the hardware structure
+ * @vsi: the VSI being configured
+ * @promisc_mask: mask of promiscuous config bits
+ *
+ * Set VSI with all associated VLANs to given promiscuous mode(s)
+ */
+int
+ice_fltr_set_vlan_vsi_promisc(struct ice_hw *hw, struct ice_vsi *vsi,
+ u8 promisc_mask)
+{
+ return ice_set_vlan_vsi_promisc(hw, vsi->idx, promisc_mask, false);
+}
+
+/**
+ * ice_fltr_clear_vlan_vsi_promisc
+ * @hw: pointer to the hardware structure
+ * @vsi: the VSI being configured
+ * @promisc_mask: mask of promiscuous config bits
+ *
+ * Clear VSI with all associated VLANs to given promiscuous mode(s)
+ */
+int
+ice_fltr_clear_vlan_vsi_promisc(struct ice_hw *hw, struct ice_vsi *vsi,
+ u8 promisc_mask)
+{
+ return ice_set_vlan_vsi_promisc(hw, vsi->idx, promisc_mask, true);
+}
+
+/**
+ * ice_fltr_clear_vsi_promisc - clear specified promiscuous mode(s)
+ * @hw: pointer to the hardware structure
+ * @vsi_handle: VSI handle to clear mode
+ * @promisc_mask: mask of promiscuous config bits to clear
+ * @vid: VLAN ID to clear VLAN promiscuous
+ */
+int
+ice_fltr_clear_vsi_promisc(struct ice_hw *hw, u16 vsi_handle, u8 promisc_mask,
+ u16 vid)
+{
+ return ice_clear_vsi_promisc(hw, vsi_handle, promisc_mask, vid);
+}
+
+/**
+ * ice_fltr_set_vsi_promisc - set given VSI to given promiscuous mode(s)
+ * @hw: pointer to the hardware structure
+ * @vsi_handle: VSI handle to configure
+ * @promisc_mask: mask of promiscuous config bits
+ * @vid: VLAN ID to set VLAN promiscuous
+ */
+int
+ice_fltr_set_vsi_promisc(struct ice_hw *hw, u16 vsi_handle, u8 promisc_mask,
+ u16 vid)
+{
+ return ice_set_vsi_promisc(hw, vsi_handle, promisc_mask, vid);
+}
+
+/**
* ice_fltr_add_mac_list - add list of MAC filters
* @vsi: pointer to VSI struct
* @list: list of filters
*/
-enum ice_status
-ice_fltr_add_mac_list(struct ice_vsi *vsi, struct list_head *list)
+int ice_fltr_add_mac_list(struct ice_vsi *vsi, struct list_head *list)
{
return ice_add_mac(&vsi->back->hw, list);
}
@@ -62,8 +119,7 @@ ice_fltr_add_mac_list(struct ice_vsi *vsi, struct list_head *list)
* @vsi: pointer to VSI struct
* @list: list of filters
*/
-enum ice_status
-ice_fltr_remove_mac_list(struct ice_vsi *vsi, struct list_head *list)
+int ice_fltr_remove_mac_list(struct ice_vsi *vsi, struct list_head *list)
{
return ice_remove_mac(&vsi->back->hw, list);
}
@@ -73,8 +129,7 @@ ice_fltr_remove_mac_list(struct ice_vsi *vsi, struct list_head *list)
* @vsi: pointer to VSI struct
* @list: list of filters
*/
-static enum ice_status
-ice_fltr_add_vlan_list(struct ice_vsi *vsi, struct list_head *list)
+static int ice_fltr_add_vlan_list(struct ice_vsi *vsi, struct list_head *list)
{
return ice_add_vlan(&vsi->back->hw, list);
}
@@ -84,7 +139,7 @@ ice_fltr_add_vlan_list(struct ice_vsi *vsi, struct list_head *list)
* @vsi: pointer to VSI struct
* @list: list of filters
*/
-static enum ice_status
+static int
ice_fltr_remove_vlan_list(struct ice_vsi *vsi, struct list_head *list)
{
return ice_remove_vlan(&vsi->back->hw, list);
@@ -95,8 +150,7 @@ ice_fltr_remove_vlan_list(struct ice_vsi *vsi, struct list_head *list)
* @vsi: pointer to VSI struct
* @list: list of filters
*/
-static enum ice_status
-ice_fltr_add_eth_list(struct ice_vsi *vsi, struct list_head *list)
+static int ice_fltr_add_eth_list(struct ice_vsi *vsi, struct list_head *list)
{
return ice_add_eth_mac(&vsi->back->hw, list);
}
@@ -106,8 +160,7 @@ ice_fltr_add_eth_list(struct ice_vsi *vsi, struct list_head *list)
* @vsi: pointer to VSI struct
* @list: list of filters
*/
-static enum ice_status
-ice_fltr_remove_eth_list(struct ice_vsi *vsi, struct list_head *list)
+static int ice_fltr_remove_eth_list(struct ice_vsi *vsi, struct list_head *list)
{
return ice_remove_eth_mac(&vsi->back->hw, list);
}
@@ -207,18 +260,17 @@ ice_fltr_add_eth_to_list(struct ice_vsi *vsi, struct list_head *list,
* @action: action to be performed on filter match
* @mac_action: pointer to add or remove MAC function
*/
-static enum ice_status
+static int
ice_fltr_prepare_mac(struct ice_vsi *vsi, const u8 *mac,
enum ice_sw_fwd_act_type action,
- enum ice_status (*mac_action)(struct ice_vsi *,
- struct list_head *))
+ int (*mac_action)(struct ice_vsi *, struct list_head *))
{
- enum ice_status result;
LIST_HEAD(tmp_list);
+ int result;
if (ice_fltr_add_mac_to_list(vsi, &tmp_list, mac, action)) {
ice_fltr_free_list(ice_pf_to_dev(vsi->back), &tmp_list);
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
}
result = mac_action(vsi, &tmp_list);
@@ -233,21 +285,21 @@ ice_fltr_prepare_mac(struct ice_vsi *vsi, const u8 *mac,
* @action: action to be performed on filter match
* @mac_action: pointer to add or remove MAC function
*/
-static enum ice_status
+static int
ice_fltr_prepare_mac_and_broadcast(struct ice_vsi *vsi, const u8 *mac,
enum ice_sw_fwd_act_type action,
- enum ice_status(*mac_action)
+ int(*mac_action)
(struct ice_vsi *, struct list_head *))
{
u8 broadcast[ETH_ALEN];
- enum ice_status result;
LIST_HEAD(tmp_list);
+ int result;
eth_broadcast_addr(broadcast);
if (ice_fltr_add_mac_to_list(vsi, &tmp_list, mac, action) ||
ice_fltr_add_mac_to_list(vsi, &tmp_list, broadcast, action)) {
ice_fltr_free_list(ice_pf_to_dev(vsi->back), &tmp_list);
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
}
result = mac_action(vsi, &tmp_list);
@@ -262,17 +314,16 @@ ice_fltr_prepare_mac_and_broadcast(struct ice_vsi *vsi, const u8 *mac,
* @action: action to be performed on filter match
* @vlan_action: pointer to add or remove VLAN function
*/
-static enum ice_status
+static int
ice_fltr_prepare_vlan(struct ice_vsi *vsi, u16 vlan_id,
enum ice_sw_fwd_act_type action,
- enum ice_status (*vlan_action)(struct ice_vsi *,
- struct list_head *))
+ int (*vlan_action)(struct ice_vsi *, struct list_head *))
{
- enum ice_status result;
LIST_HEAD(tmp_list);
+ int result;
if (ice_fltr_add_vlan_to_list(vsi, &tmp_list, vlan_id, action))
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
result = vlan_action(vsi, &tmp_list);
ice_fltr_free_list(ice_pf_to_dev(vsi->back), &tmp_list);
@@ -287,17 +338,16 @@ ice_fltr_prepare_vlan(struct ice_vsi *vsi, u16 vlan_id,
* @action: action to be performed on filter match
* @eth_action: pointer to add or remove ethertype function
*/
-static enum ice_status
+static int
ice_fltr_prepare_eth(struct ice_vsi *vsi, u16 ethertype, u16 flag,
enum ice_sw_fwd_act_type action,
- enum ice_status (*eth_action)(struct ice_vsi *,
- struct list_head *))
+ int (*eth_action)(struct ice_vsi *, struct list_head *))
{
- enum ice_status result;
LIST_HEAD(tmp_list);
+ int result;
if (ice_fltr_add_eth_to_list(vsi, &tmp_list, ethertype, flag, action))
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
result = eth_action(vsi, &tmp_list);
ice_fltr_free_list(ice_pf_to_dev(vsi->back), &tmp_list);
@@ -310,8 +360,8 @@ ice_fltr_prepare_eth(struct ice_vsi *vsi, u16 ethertype, u16 flag,
* @mac: MAC to add
* @action: action to be performed on filter match
*/
-enum ice_status ice_fltr_add_mac(struct ice_vsi *vsi, const u8 *mac,
- enum ice_sw_fwd_act_type action)
+int ice_fltr_add_mac(struct ice_vsi *vsi, const u8 *mac,
+ enum ice_sw_fwd_act_type action)
{
return ice_fltr_prepare_mac(vsi, mac, action, ice_fltr_add_mac_list);
}
@@ -322,7 +372,7 @@ enum ice_status ice_fltr_add_mac(struct ice_vsi *vsi, const u8 *mac,
* @mac: MAC to add
* @action: action to be performed on filter match
*/
-enum ice_status
+int
ice_fltr_add_mac_and_broadcast(struct ice_vsi *vsi, const u8 *mac,
enum ice_sw_fwd_act_type action)
{
@@ -336,8 +386,8 @@ ice_fltr_add_mac_and_broadcast(struct ice_vsi *vsi, const u8 *mac,
* @mac: filter MAC to remove
* @action: action to remove
*/
-enum ice_status ice_fltr_remove_mac(struct ice_vsi *vsi, const u8 *mac,
- enum ice_sw_fwd_act_type action)
+int ice_fltr_remove_mac(struct ice_vsi *vsi, const u8 *mac,
+ enum ice_sw_fwd_act_type action)
{
return ice_fltr_prepare_mac(vsi, mac, action, ice_fltr_remove_mac_list);
}
@@ -348,8 +398,8 @@ enum ice_status ice_fltr_remove_mac(struct ice_vsi *vsi, const u8 *mac,
* @vlan_id: VLAN ID to add
* @action: action to be performed on filter match
*/
-enum ice_status ice_fltr_add_vlan(struct ice_vsi *vsi, u16 vlan_id,
- enum ice_sw_fwd_act_type action)
+int ice_fltr_add_vlan(struct ice_vsi *vsi, u16 vlan_id,
+ enum ice_sw_fwd_act_type action)
{
return ice_fltr_prepare_vlan(vsi, vlan_id, action,
ice_fltr_add_vlan_list);
@@ -361,8 +411,8 @@ enum ice_status ice_fltr_add_vlan(struct ice_vsi *vsi, u16 vlan_id,
* @vlan_id: filter VLAN to remove
* @action: action to remove
*/
-enum ice_status ice_fltr_remove_vlan(struct ice_vsi *vsi, u16 vlan_id,
- enum ice_sw_fwd_act_type action)
+int ice_fltr_remove_vlan(struct ice_vsi *vsi, u16 vlan_id,
+ enum ice_sw_fwd_act_type action)
{
return ice_fltr_prepare_vlan(vsi, vlan_id, action,
ice_fltr_remove_vlan_list);
@@ -375,8 +425,8 @@ enum ice_status ice_fltr_remove_vlan(struct ice_vsi *vsi, u16 vlan_id,
* @flag: direction of packet to be filtered, Tx or Rx
* @action: action to be performed on filter match
*/
-enum ice_status ice_fltr_add_eth(struct ice_vsi *vsi, u16 ethertype, u16 flag,
- enum ice_sw_fwd_act_type action)
+int ice_fltr_add_eth(struct ice_vsi *vsi, u16 ethertype, u16 flag,
+ enum ice_sw_fwd_act_type action)
{
return ice_fltr_prepare_eth(vsi, ethertype, flag, action,
ice_fltr_add_eth_list);
@@ -389,89 +439,9 @@ enum ice_status ice_fltr_add_eth(struct ice_vsi *vsi, u16 ethertype, u16 flag,
* @flag: direction of filter
* @action: action to remove
*/
-enum ice_status ice_fltr_remove_eth(struct ice_vsi *vsi, u16 ethertype,
- u16 flag, enum ice_sw_fwd_act_type action)
+int ice_fltr_remove_eth(struct ice_vsi *vsi, u16 ethertype, u16 flag,
+ enum ice_sw_fwd_act_type action)
{
return ice_fltr_prepare_eth(vsi, ethertype, flag, action,
ice_fltr_remove_eth_list);
}
-
-/**
- * ice_fltr_update_rule_flags - update lan_en/lb_en flags
- * @hw: pointer to hw
- * @rule_id: id of rule being updated
- * @recipe_id: recipe id of rule
- * @act: current action field
- * @type: Rx or Tx
- * @src: source VSI
- * @new_flags: combinations of lb_en and lan_en
- */
-static enum ice_status
-ice_fltr_update_rule_flags(struct ice_hw *hw, u16 rule_id, u16 recipe_id,
- u32 act, u16 type, u16 src, u32 new_flags)
-{
- struct ice_aqc_sw_rules_elem *s_rule;
- enum ice_status err;
- u32 flags_mask;
-
- s_rule = kzalloc(ICE_SW_RULE_RX_TX_NO_HDR_SIZE, GFP_KERNEL);
- if (!s_rule)
- return ICE_ERR_NO_MEMORY;
-
- flags_mask = ICE_SINGLE_ACT_LB_ENABLE | ICE_SINGLE_ACT_LAN_ENABLE;
- act &= ~flags_mask;
- act |= (flags_mask & new_flags);
-
- s_rule->pdata.lkup_tx_rx.recipe_id = cpu_to_le16(recipe_id);
- s_rule->pdata.lkup_tx_rx.index = cpu_to_le16(rule_id);
- s_rule->pdata.lkup_tx_rx.act = cpu_to_le32(act);
-
- if (type & ICE_FLTR_RX) {
- s_rule->pdata.lkup_tx_rx.src =
- cpu_to_le16(hw->port_info->lport);
- s_rule->type = cpu_to_le16(ICE_AQC_SW_RULES_T_LKUP_RX);
-
- } else {
- s_rule->pdata.lkup_tx_rx.src = cpu_to_le16(src);
- s_rule->type = cpu_to_le16(ICE_AQC_SW_RULES_T_LKUP_TX);
- }
-
- err = ice_aq_sw_rules(hw, s_rule, ICE_SW_RULE_RX_TX_NO_HDR_SIZE, 1,
- ice_aqc_opc_update_sw_rules, NULL);
-
- kfree(s_rule);
- return err;
-}
-
-/**
- * ice_fltr_build_action - build action for rule
- * @vsi_id: id of VSI which is use to build action
- */
-static u32 ice_fltr_build_action(u16 vsi_id)
-{
- return ((vsi_id << ICE_SINGLE_ACT_VSI_ID_S) & ICE_SINGLE_ACT_VSI_ID_M) |
- ICE_SINGLE_ACT_VSI_FORWARDING | ICE_SINGLE_ACT_VALID_BIT;
-}
-
-/**
- * ice_fltr_update_flags_dflt_rule - update flags on default rule
- * @vsi: pointer to VSI
- * @rule_id: id of rule
- * @direction: Tx or Rx
- * @new_flags: flags to update
- *
- * Function updates flags on default rule with ICE_SW_LKUP_DFLT.
- *
- * Flags should be a combination of ICE_SINGLE_ACT_LB_ENABLE and
- * ICE_SINGLE_ACT_LAN_ENABLE.
- */
-enum ice_status
-ice_fltr_update_flags_dflt_rule(struct ice_vsi *vsi, u16 rule_id, u8 direction,
- u32 new_flags)
-{
- u32 action = ice_fltr_build_action(vsi->vsi_num);
- struct ice_hw *hw = &vsi->back->hw;
-
- return ice_fltr_update_rule_flags(hw, rule_id, ICE_SW_LKUP_DFLT, action,
- direction, vsi->vsi_num, new_flags);
-}
diff --git a/drivers/net/ethernet/intel/ice/ice_fltr.h b/drivers/net/ethernet/intel/ice/ice_fltr.h
index 8eec4fe..3eb4247 100644
--- a/drivers/net/ethernet/intel/ice/ice_fltr.h
+++ b/drivers/net/ethernet/intel/ice/ice_fltr.h
@@ -5,38 +5,49 @@
#define _ICE_FLTR_H_
void ice_fltr_free_list(struct device *dev, struct list_head *h);
-enum ice_status
+int
+ice_fltr_set_vlan_vsi_promisc(struct ice_hw *hw, struct ice_vsi *vsi,
+ u8 promisc_mask);
+int
+ice_fltr_clear_vlan_vsi_promisc(struct ice_hw *hw, struct ice_vsi *vsi,
+ u8 promisc_mask);
+int
+ice_fltr_clear_vsi_promisc(struct ice_hw *hw, u16 vsi_handle, u8 promisc_mask,
+ u16 vid);
+int
+ice_fltr_set_vsi_promisc(struct ice_hw *hw, u16 vsi_handle, u8 promisc_mask,
+ u16 vid);
+int
ice_fltr_add_mac_to_list(struct ice_vsi *vsi, struct list_head *list,
const u8 *mac, enum ice_sw_fwd_act_type action);
-enum ice_status
+int
ice_fltr_add_mac(struct ice_vsi *vsi, const u8 *mac,
enum ice_sw_fwd_act_type action);
-enum ice_status
+int
ice_fltr_add_mac_and_broadcast(struct ice_vsi *vsi, const u8 *mac,
enum ice_sw_fwd_act_type action);
-enum ice_status
-ice_fltr_add_mac_list(struct ice_vsi *vsi, struct list_head *list);
-enum ice_status
+int ice_fltr_add_mac_list(struct ice_vsi *vsi, struct list_head *list);
+int
ice_fltr_remove_mac(struct ice_vsi *vsi, const u8 *mac,
enum ice_sw_fwd_act_type action);
-enum ice_status
-ice_fltr_remove_mac_list(struct ice_vsi *vsi, struct list_head *list);
+int ice_fltr_remove_mac_list(struct ice_vsi *vsi, struct list_head *list);
-enum ice_status
+int
ice_fltr_add_vlan(struct ice_vsi *vsi, u16 vid,
enum ice_sw_fwd_act_type action);
-enum ice_status
+int
ice_fltr_remove_vlan(struct ice_vsi *vsi, u16 vid,
enum ice_sw_fwd_act_type action);
-enum ice_status
+int
ice_fltr_add_eth(struct ice_vsi *vsi, u16 ethertype, u16 flag,
enum ice_sw_fwd_act_type action);
-enum ice_status
+int
ice_fltr_remove_eth(struct ice_vsi *vsi, u16 ethertype, u16 flag,
enum ice_sw_fwd_act_type action);
void ice_fltr_remove_all(struct ice_vsi *vsi);
-enum ice_status
-ice_fltr_update_flags_dflt_rule(struct ice_vsi *vsi, u16 rule_id, u8 direction,
- u32 new_flags);
+
+int
+ice_fltr_update_flags(struct ice_vsi *vsi, u16 rule_id, u16 recipe_id,
+ u32 new_flags);
#endif
diff --git a/drivers/net/ethernet/intel/ice/ice_fw_update.c b/drivers/net/ethernet/intel/ice/ice_fw_update.c
index f8601d5..665a344 100644
--- a/drivers/net/ethernet/intel/ice/ice_fw_update.c
+++ b/drivers/net/ethernet/intel/ice/ice_fw_update.c
@@ -16,6 +16,18 @@ struct ice_fwu_priv {
/* Track which NVM banks to activate at the end of the update */
u8 activate_flags;
+
+ /* Track the firmware response of the required reset to complete the
+ * flash update.
+ *
+ * 0 - ICE_AQC_NVM_POR_FLAG - A full power on is required
+ * 1 - ICE_AQC_NVM_PERST_FLAG - A cold PCIe reset is required
+ * 2 - ICE_AQC_NVM_EMPR_FLAG - An EMP reset is required
+ */
+ u8 reset_level;
+
+ /* Track if EMP reset is available */
+ u8 emp_reset_available;
};
/**
@@ -40,8 +52,8 @@ ice_send_package_data(struct pldmfw *context, const u8 *data, u16 length)
struct device *dev = context->dev;
struct ice_pf *pf = priv->pf;
struct ice_hw *hw = &pf->hw;
- enum ice_status status;
u8 *package_data;
+ int status;
dev_dbg(dev, "Sending PLDM record package data to firmware\n");
@@ -54,9 +66,8 @@ ice_send_package_data(struct pldmfw *context, const u8 *data, u16 length)
kfree(package_data);
if (status) {
- dev_err(dev, "Failed to send record package data to firmware, err %s aq_err %s\n",
- ice_stat_str(status),
- ice_aq_str(hw->adminq.sq_last_status));
+ dev_err(dev, "Failed to send record package data to firmware, err %d aq_err %s\n",
+ status, ice_aq_str(hw->adminq.sq_last_status));
NL_SET_ERR_MSG_MOD(extack, "Failed to record package data to firmware");
return -EIO;
}
@@ -203,8 +214,8 @@ ice_send_component_table(struct pldmfw *context, struct pldmfw_component *compon
struct device *dev = context->dev;
struct ice_pf *pf = priv->pf;
struct ice_hw *hw = &pf->hw;
- enum ice_status status;
size_t length;
+ int status;
switch (component->identifier) {
case NVM_COMP_ID_OROM:
@@ -240,9 +251,8 @@ ice_send_component_table(struct pldmfw *context, struct pldmfw_component *compon
kfree(comp_tbl);
if (status) {
- dev_err(dev, "Failed to transfer component table to firmware, err %s aq_err %s\n",
- ice_stat_str(status),
- ice_aq_str(hw->adminq.sq_last_status));
+ dev_err(dev, "Failed to transfer component table to firmware, err %d aq_err %s\n",
+ status, ice_aq_str(hw->adminq.sq_last_status));
NL_SET_ERR_MSG_MOD(extack, "Failed to transfer component table to firmware");
return -EIO;
}
@@ -259,6 +269,7 @@ ice_send_component_table(struct pldmfw *context, struct pldmfw_component *compon
* @block_size: size of the block to write, up to 4k
* @block: pointer to block of data to write
* @last_cmd: whether this is the last command
+ * @reset_level: storage for reset level required
* @extack: netlink extended ACK structure
*
* Write a block of data to a flash module, and await for the completion
@@ -266,18 +277,24 @@ ice_send_component_table(struct pldmfw *context, struct pldmfw_component *compon
*
* Note this function assumes the caller has acquired the NVM resource.
*
+ * On successful return, reset level indicates the device reset required to
+ * complete the update.
+ *
+ * 0 - ICE_AQC_NVM_POR_FLAG - A full power on is required
+ * 1 - ICE_AQC_NVM_PERST_FLAG - A cold PCIe reset is required
+ * 2 - ICE_AQC_NVM_EMPR_FLAG - An EMP reset is required
+ *
* Returns: zero on success, or a negative error code on failure.
*/
static int
ice_write_one_nvm_block(struct ice_pf *pf, u16 module, u32 offset,
u16 block_size, u8 *block, bool last_cmd,
- struct netlink_ext_ack *extack)
+ u8 *reset_level, struct netlink_ext_ack *extack)
{
u16 completion_module, completion_retval;
struct device *dev = ice_pf_to_dev(pf);
struct ice_rq_event_info event;
struct ice_hw *hw = &pf->hw;
- enum ice_status status;
u32 completion_offset;
int err;
@@ -286,11 +303,11 @@ ice_write_one_nvm_block(struct ice_pf *pf, u16 module, u32 offset,
dev_dbg(dev, "Writing block of %u bytes for module 0x%02x at offset %u\n",
block_size, module, offset);
- status = ice_aq_update_nvm(hw, module, offset, block_size, block,
- last_cmd, 0, NULL);
- if (status) {
- dev_err(dev, "Failed to flash module 0x%02x with block of size %u at offset %u, err %s aq_err %s\n",
- module, block_size, offset, ice_stat_str(status),
+ err = ice_aq_update_nvm(hw, module, offset, block_size, block,
+ last_cmd, 0, NULL);
+ if (err) {
+ dev_err(dev, "Failed to flash module 0x%02x with block of size %u at offset %u, err %d aq_err %s\n",
+ module, block_size, offset, err,
ice_aq_str(hw->adminq.sq_last_status));
NL_SET_ERR_MSG_MOD(extack, "Failed to program flash module");
return -EIO;
@@ -338,6 +355,24 @@ ice_write_one_nvm_block(struct ice_pf *pf, u16 module, u32 offset,
return -EIO;
}
+ /* For the last command to write the NVM bank, newer versions of
+ * firmware indicate the required level of reset to complete
+ * activation of firmware. If the firmware supports this, cache the
+ * response for indicating to the user later. Otherwise, assume that
+ * a full power cycle is required.
+ */
+ if (reset_level && last_cmd && module == ICE_SR_1ST_NVM_BANK_PTR) {
+ if (hw->dev_caps.common_cap.pcie_reset_avoidance) {
+ *reset_level = (event.desc.params.nvm.cmd_flags &
+ ICE_AQC_NVM_RESET_LVL_M);
+ dev_dbg(dev, "Firmware reported required reset level as %u\n",
+ *reset_level);
+ } else {
+ *reset_level = ICE_AQC_NVM_POR_FLAG;
+ dev_dbg(dev, "Firmware doesn't support indicating required reset level. Assuming a power cycle is required\n");
+ }
+ }
+
return 0;
}
@@ -348,6 +383,7 @@ ice_write_one_nvm_block(struct ice_pf *pf, u16 module, u32 offset,
* @component: the name of the component being updated
* @image: buffer of image data to write to the NVM
* @length: length of the buffer
+ * @reset_level: storage for reset level required
* @extack: netlink extended ACK structure
*
* Loop over the data for a given NVM module and program it in 4 Kb
@@ -360,7 +396,7 @@ ice_write_one_nvm_block(struct ice_pf *pf, u16 module, u32 offset,
*/
static int
ice_write_nvm_module(struct ice_pf *pf, u16 module, const char *component,
- const u8 *image, u32 length,
+ const u8 *image, u32 length, u8 *reset_level,
struct netlink_ext_ack *extack)
{
struct device *dev = ice_pf_to_dev(pf);
@@ -394,7 +430,8 @@ ice_write_nvm_module(struct ice_pf *pf, u16 module, const char *component,
memcpy(block, image + offset, block_size);
err = ice_write_one_nvm_block(pf, module, offset, block_size,
- block, last_cmd, extack);
+ block, last_cmd, reset_level,
+ extack);
if (err)
break;
@@ -445,7 +482,6 @@ ice_erase_nvm_module(struct ice_pf *pf, u16 module, const char *component,
struct ice_rq_event_info event;
struct ice_hw *hw = &pf->hw;
struct devlink *devlink;
- enum ice_status status;
int err;
dev_dbg(dev, "Beginning erase of flash component '%s', module 0x%02x\n", component, module);
@@ -456,10 +492,10 @@ ice_erase_nvm_module(struct ice_pf *pf, u16 module, const char *component,
devlink_flash_update_timeout_notify(devlink, "Erasing", component, ICE_FW_ERASE_TIMEOUT);
- status = ice_aq_erase_nvm(hw, module, NULL);
- if (status) {
- dev_err(dev, "Failed to erase %s (module 0x%02x), err %s aq_err %s\n",
- component, module, ice_stat_str(status),
+ err = ice_aq_erase_nvm(hw, module, NULL);
+ if (err) {
+ dev_err(dev, "Failed to erase %s (module 0x%02x), err %d aq_err %s\n",
+ component, module, err,
ice_aq_str(hw->adminq.sq_last_status));
NL_SET_ERR_MSG_MOD(extack, "Failed to erase flash module");
err = -EIO;
@@ -511,6 +547,7 @@ ice_erase_nvm_module(struct ice_pf *pf, u16 module, const char *component,
* ice_switch_flash_banks - Tell firmware to switch NVM banks
* @pf: Pointer to the PF data structure
* @activate_flags: flags used for the activation command
+ * @emp_reset_available: on return, indicates if EMP reset is available
* @extack: netlink extended ACK structure
*
* Notify firmware to activate the newly written flash banks, and wait for the
@@ -518,27 +555,43 @@ ice_erase_nvm_module(struct ice_pf *pf, u16 module, const char *component,
*
* Returns: zero on success or an error code on failure.
*/
-static int ice_switch_flash_banks(struct ice_pf *pf, u8 activate_flags,
- struct netlink_ext_ack *extack)
+static int
+ice_switch_flash_banks(struct ice_pf *pf, u8 activate_flags,
+ u8 *emp_reset_available, struct netlink_ext_ack *extack)
{
struct device *dev = ice_pf_to_dev(pf);
struct ice_rq_event_info event;
struct ice_hw *hw = &pf->hw;
- enum ice_status status;
u16 completion_retval;
+ u8 response_flags;
int err;
memset(&event, 0, sizeof(event));
- status = ice_nvm_write_activate(hw, activate_flags);
- if (status) {
- dev_err(dev, "Failed to switch active flash banks, err %s aq_err %s\n",
- ice_stat_str(status),
- ice_aq_str(hw->adminq.sq_last_status));
+ err = ice_nvm_write_activate(hw, activate_flags, &response_flags);
+ if (err) {
+ dev_err(dev, "Failed to switch active flash banks, err %d aq_err %s\n",
+ err, ice_aq_str(hw->adminq.sq_last_status));
NL_SET_ERR_MSG_MOD(extack, "Failed to switch active flash banks");
return -EIO;
}
+ /* Newer versions of firmware have support to indicate whether an EMP
+ * reset to reload firmware is available. For older firmware, EMP
+ * reset is always available.
+ */
+ if (emp_reset_available) {
+ if (hw->dev_caps.common_cap.reset_restrict_support) {
+ *emp_reset_available = response_flags & ICE_AQC_NVM_EMPR_ENA;
+ dev_dbg(dev, "Firmware indicated that EMP reset is %s\n",
+ *emp_reset_available ?
+ "available" : "not available");
+ } else {
+ *emp_reset_available = ICE_AQC_NVM_EMPR_ENA;
+ dev_dbg(dev, "Firmware does not support restricting EMP reset availability\n");
+ }
+ }
+
err = ice_aq_wait_for_event(pf, ice_aqc_opc_nvm_write_activate, 30 * HZ,
&event);
if (err) {
@@ -579,6 +632,7 @@ ice_flash_component(struct pldmfw *context, struct pldmfw_component *component)
struct netlink_ext_ack *extack = priv->extack;
struct ice_pf *pf = priv->pf;
const char *name;
+ u8 *reset_level;
u16 module;
u8 flag;
int err;
@@ -587,16 +641,19 @@ ice_flash_component(struct pldmfw *context, struct pldmfw_component *component)
case NVM_COMP_ID_OROM:
module = ICE_SR_1ST_OROM_BANK_PTR;
flag = ICE_AQC_NVM_ACTIV_SEL_OROM;
+ reset_level = NULL;
name = "fw.undi";
break;
case NVM_COMP_ID_NVM:
module = ICE_SR_1ST_NVM_BANK_PTR;
flag = ICE_AQC_NVM_ACTIV_SEL_NVM;
+ reset_level = &priv->reset_level;
name = "fw.mgmt";
break;
case NVM_COMP_ID_NETLIST:
module = ICE_SR_NETLIST_BANK_PTR;
flag = ICE_AQC_NVM_ACTIV_SEL_NETLIST;
+ reset_level = NULL;
name = "fw.netlist";
break;
default:
@@ -616,7 +673,8 @@ ice_flash_component(struct pldmfw *context, struct pldmfw_component *component)
return err;
return ice_write_nvm_module(pf, module, name, component->component_data,
- component->component_size, extack);
+ component->component_size, reset_level,
+ extack);
}
/**
@@ -634,9 +692,48 @@ static int ice_finalize_update(struct pldmfw *context)
struct ice_fwu_priv *priv = container_of(context, struct ice_fwu_priv, context);
struct netlink_ext_ack *extack = priv->extack;
struct ice_pf *pf = priv->pf;
+ struct devlink *devlink;
+ int err;
/* Finally, notify firmware to activate the written NVM banks */
- return ice_switch_flash_banks(pf, priv->activate_flags, extack);
+ err = ice_switch_flash_banks(pf, priv->activate_flags,
+ &priv->emp_reset_available, extack);
+ if (err)
+ return err;
+
+ devlink = priv_to_devlink(pf);
+
+ /* If the required reset is EMPR, but EMPR is disabled, report that
+ * a reboot is required instead.
+ */
+ if (priv->reset_level == ICE_AQC_NVM_EMPR_FLAG &&
+ !priv->emp_reset_available) {
+ dev_dbg(ice_pf_to_dev(pf), "Firmware indicated EMP reset as sufficient, but EMP reset is disabled\n");
+ priv->reset_level = ICE_AQC_NVM_PERST_FLAG;
+ }
+
+ switch (priv->reset_level) {
+ case ICE_AQC_NVM_EMPR_FLAG:
+ devlink_flash_update_status_notify(devlink,
+ "Activate new firmware by devlink reload",
+ NULL, 0, 0);
+ break;
+ case ICE_AQC_NVM_PERST_FLAG:
+ devlink_flash_update_status_notify(devlink,
+ "Activate new firmware by rebooting the system",
+ NULL, 0, 0);
+ break;
+ case ICE_AQC_NVM_POR_FLAG:
+ default:
+ devlink_flash_update_status_notify(devlink,
+ "Activate new firmware by power cycling the system",
+ NULL, 0, 0);
+ break;
+ }
+
+ pf->fw_emp_reset_disabled = !priv->emp_reset_available;
+
+ return 0;
}
static const struct pldmfw_ops ice_fwu_ops = {
@@ -648,96 +745,22 @@ static const struct pldmfw_ops ice_fwu_ops = {
};
/**
- * ice_flash_pldm_image - Write a PLDM-formatted firmware image to the device
- * @pf: private device driver structure
- * @fw: firmware object pointing to the relevant firmware file
- * @preservation: preservation level to request from firmware
- * @extack: netlink extended ACK structure
- *
- * Parse the data for a given firmware file, verifying that it is a valid PLDM
- * formatted image that matches this device.
- *
- * Extract the device record Package Data and Component Tables and send them
- * to the firmware. Extract and write the flash data for each of the three
- * main flash components, "fw.mgmt", "fw.undi", and "fw.netlist". Notify
- * firmware once the data is written to the inactive banks.
- *
- * Returns: zero on success or a negative error code on failure.
- */
-int ice_flash_pldm_image(struct ice_pf *pf, const struct firmware *fw,
- u8 preservation, struct netlink_ext_ack *extack)
-{
- struct device *dev = ice_pf_to_dev(pf);
- struct ice_hw *hw = &pf->hw;
- struct ice_fwu_priv priv;
- enum ice_status status;
- int err;
-
- switch (preservation) {
- case ICE_AQC_NVM_PRESERVE_ALL:
- case ICE_AQC_NVM_PRESERVE_SELECTED:
- case ICE_AQC_NVM_NO_PRESERVATION:
- case ICE_AQC_NVM_FACTORY_DEFAULT:
- break;
- default:
- WARN(1, "Unexpected preservation level request %u", preservation);
- return -EINVAL;
- }
-
- memset(&priv, 0, sizeof(priv));
-
- priv.context.ops = &ice_fwu_ops;
- priv.context.dev = dev;
- priv.extack = extack;
- priv.pf = pf;
- priv.activate_flags = preservation;
-
- status = ice_acquire_nvm(hw, ICE_RES_WRITE);
- if (status) {
- dev_err(dev, "Failed to acquire device flash lock, err %s aq_err %s\n",
- ice_stat_str(status),
- ice_aq_str(hw->adminq.sq_last_status));
- NL_SET_ERR_MSG_MOD(extack, "Failed to acquire device flash lock");
- return -EIO;
- }
-
- err = pldmfw_flash_image(&priv.context, fw);
- if (err == -ENOENT) {
- dev_err(dev, "Firmware image has no record matching this device\n");
- NL_SET_ERR_MSG_MOD(extack, "Firmware image has no record matching this device");
- } else if (err) {
- /* Do not set a generic extended ACK message here. A more
- * specific message may already have been set by one of our
- * ops.
- */
- dev_err(dev, "Failed to flash PLDM image, err %d", err);
- }
-
- ice_release_nvm(hw);
-
- return err;
-}
-
-/**
- * ice_check_for_pending_update - Check for a pending flash update
+ * ice_get_pending_updates - Check if the component has a pending update
* @pf: the PF driver structure
- * @component: if not NULL, the name of the component being updated
- * @extack: Netlink extended ACK structure
+ * @pending: on return, bitmap of updates pending
+ * @extack: Netlink extended ACK
*
- * Check whether the device already has a pending flash update. If such an
- * update is found, cancel it so that the requested update may proceed.
+ * Check if the device has any pending updates on any flash components.
*
- * Returns: zero on success, or a negative error code on failure.
+ * Returns: zero on success, or a negative error code on failure. Updates
+ * pending with the bitmap of pending updates.
*/
-int ice_check_for_pending_update(struct ice_pf *pf, const char *component,
- struct netlink_ext_ack *extack)
+int ice_get_pending_updates(struct ice_pf *pf, u8 *pending,
+ struct netlink_ext_ack *extack)
{
- struct devlink *devlink = priv_to_devlink(pf);
struct device *dev = ice_pf_to_dev(pf);
struct ice_hw_dev_caps *dev_caps;
struct ice_hw *hw = &pf->hw;
- enum ice_status status;
- u8 pending = 0;
int err;
dev_caps = kzalloc(sizeof(*dev_caps), GFP_KERNEL);
@@ -749,30 +772,60 @@ int ice_check_for_pending_update(struct ice_pf *pf, const char *component,
* may have changed, e.g. if an update was previously completed and
* the system has not yet rebooted.
*/
- status = ice_discover_dev_caps(hw, dev_caps);
- if (status) {
+ err = ice_discover_dev_caps(hw, dev_caps);
+ if (err) {
NL_SET_ERR_MSG_MOD(extack, "Unable to read device capabilities");
kfree(dev_caps);
- return -EIO;
+ return err;
}
+ *pending = 0;
+
if (dev_caps->common_cap.nvm_update_pending_nvm) {
dev_info(dev, "The fw.mgmt flash component has a pending update\n");
- pending |= ICE_AQC_NVM_ACTIV_SEL_NVM;
+ *pending |= ICE_AQC_NVM_ACTIV_SEL_NVM;
}
if (dev_caps->common_cap.nvm_update_pending_orom) {
dev_info(dev, "The fw.undi flash component has a pending update\n");
- pending |= ICE_AQC_NVM_ACTIV_SEL_OROM;
+ *pending |= ICE_AQC_NVM_ACTIV_SEL_OROM;
}
if (dev_caps->common_cap.nvm_update_pending_netlist) {
dev_info(dev, "The fw.netlist flash component has a pending update\n");
- pending |= ICE_AQC_NVM_ACTIV_SEL_NETLIST;
+ *pending |= ICE_AQC_NVM_ACTIV_SEL_NETLIST;
}
kfree(dev_caps);
+ return 0;
+}
+
+/**
+ * ice_cancel_pending_update - Cancel any pending update for a component
+ * @pf: the PF driver structure
+ * @component: if not NULL, the name of the component being updated
+ * @extack: Netlink extended ACK structure
+ *
+ * Cancel any pending update for the specified component. If component is
+ * NULL, all device updates will be canceled.
+ *
+ * Returns: zero on success, or a negative error code on failure.
+ */
+static int
+ice_cancel_pending_update(struct ice_pf *pf, const char *component,
+ struct netlink_ext_ack *extack)
+{
+ struct devlink *devlink = priv_to_devlink(pf);
+ struct device *dev = ice_pf_to_dev(pf);
+ struct ice_hw *hw = &pf->hw;
+ u8 pending;
+ int err;
+
+ err = ice_get_pending_updates(pf, &pending, extack);
+ if (err)
+ return err;
+
/* If the flash_update request is for a specific component, ignore all
* of the other components.
*/
@@ -798,17 +851,107 @@ int ice_check_for_pending_update(struct ice_pf *pf, const char *component,
"Canceling previous pending update",
component, 0, 0);
- status = ice_acquire_nvm(hw, ICE_RES_WRITE);
- if (status) {
- dev_err(dev, "Failed to acquire device flash lock, err %s aq_err %s\n",
- ice_stat_str(status),
- ice_aq_str(hw->adminq.sq_last_status));
+ err = ice_acquire_nvm(hw, ICE_RES_WRITE);
+ if (err) {
+ dev_err(dev, "Failed to acquire device flash lock, err %d aq_err %s\n",
+ err, ice_aq_str(hw->adminq.sq_last_status));
NL_SET_ERR_MSG_MOD(extack, "Failed to acquire device flash lock");
- return -EIO;
+ return err;
}
pending |= ICE_AQC_NVM_REVERT_LAST_ACTIV;
- err = ice_switch_flash_banks(pf, pending, extack);
+ err = ice_switch_flash_banks(pf, pending, NULL, extack);
+
+ ice_release_nvm(hw);
+
+ /* Since we've canceled the pending update, we no longer know if EMP
+ * reset is restricted.
+ */
+ pf->fw_emp_reset_disabled = false;
+
+ return err;
+}
+
+/**
+ * ice_devlink_flash_update - Write a firmware image to the device
+ * @devlink: pointer to devlink associated with the device to update
+ * @params: devlink flash update parameters
+ * @extack: netlink extended ACK structure
+ *
+ * Parse the data for a given firmware file, verifying that it is a valid PLDM
+ * formatted image that matches this device.
+ *
+ * Extract the device record Package Data and Component Tables and send them
+ * to the firmware. Extract and write the flash data for each of the three
+ * main flash components, "fw.mgmt", "fw.undi", and "fw.netlist". Notify
+ * firmware once the data is written to the inactive banks.
+ *
+ * Returns: zero on success or a negative error code on failure.
+ */
+int ice_devlink_flash_update(struct devlink *devlink,
+ struct devlink_flash_update_params *params,
+ struct netlink_ext_ack *extack)
+{
+ struct ice_pf *pf = devlink_priv(devlink);
+ struct device *dev = ice_pf_to_dev(pf);
+ struct ice_hw *hw = &pf->hw;
+ struct ice_fwu_priv priv;
+ u8 preservation;
+ int err;
+
+ if (!params->overwrite_mask) {
+ /* preserve all settings and identifiers */
+ preservation = ICE_AQC_NVM_PRESERVE_ALL;
+ } else if (params->overwrite_mask == DEVLINK_FLASH_OVERWRITE_SETTINGS) {
+ /* overwrite settings, but preserve the vital device identifiers */
+ preservation = ICE_AQC_NVM_PRESERVE_SELECTED;
+ } else if (params->overwrite_mask == (DEVLINK_FLASH_OVERWRITE_SETTINGS |
+ DEVLINK_FLASH_OVERWRITE_IDENTIFIERS)) {
+ /* overwrite both settings and identifiers, preserve nothing */
+ preservation = ICE_AQC_NVM_NO_PRESERVATION;
+ } else {
+ NL_SET_ERR_MSG_MOD(extack, "Requested overwrite mask is not supported");
+ return -EOPNOTSUPP;
+ }
+
+ if (!hw->dev_caps.common_cap.nvm_unified_update) {
+ NL_SET_ERR_MSG_MOD(extack, "Current firmware does not support unified update");
+ return -EOPNOTSUPP;
+ }
+
+ memset(&priv, 0, sizeof(priv));
+
+ priv.context.ops = &ice_fwu_ops;
+ priv.context.dev = dev;
+ priv.extack = extack;
+ priv.pf = pf;
+ priv.activate_flags = preservation;
+
+ devlink_flash_update_status_notify(devlink, "Preparing to flash", NULL, 0, 0);
+
+ err = ice_cancel_pending_update(pf, NULL, extack);
+ if (err)
+ return err;
+
+ err = ice_acquire_nvm(hw, ICE_RES_WRITE);
+ if (err) {
+ dev_err(dev, "Failed to acquire device flash lock, err %d aq_err %s\n",
+ err, ice_aq_str(hw->adminq.sq_last_status));
+ NL_SET_ERR_MSG_MOD(extack, "Failed to acquire device flash lock");
+ return err;
+ }
+
+ err = pldmfw_flash_image(&priv.context, params->fw);
+ if (err == -ENOENT) {
+ dev_err(dev, "Firmware image has no record matching this device\n");
+ NL_SET_ERR_MSG_MOD(extack, "Firmware image has no record matching this device");
+ } else if (err) {
+ /* Do not set a generic extended ACK message here. A more
+ * specific message may already have been set by one of our
+ * ops.
+ */
+ dev_err(dev, "Failed to flash PLDM image, err %d", err);
+ }
ice_release_nvm(hw);
diff --git a/drivers/net/ethernet/intel/ice/ice_fw_update.h b/drivers/net/ethernet/intel/ice/ice_fw_update.h
index c6390f6..7505748 100644
--- a/drivers/net/ethernet/intel/ice/ice_fw_update.h
+++ b/drivers/net/ethernet/intel/ice/ice_fw_update.h
@@ -4,9 +4,10 @@
#ifndef _ICE_FW_UPDATE_H_
#define _ICE_FW_UPDATE_H_
-int ice_flash_pldm_image(struct ice_pf *pf, const struct firmware *fw,
- u8 preservation, struct netlink_ext_ack *extack);
-int ice_check_for_pending_update(struct ice_pf *pf, const char *component,
- struct netlink_ext_ack *extack);
+int ice_devlink_flash_update(struct devlink *devlink,
+ struct devlink_flash_update_params *params,
+ struct netlink_ext_ack *extack);
+int ice_get_pending_updates(struct ice_pf *pf, u8 *pending,
+ struct netlink_ext_ack *extack);
#endif
diff --git a/drivers/net/ethernet/intel/ice/ice_hw_autogen.h b/drivers/net/ethernet/intel/ice/ice_hw_autogen.h
index a490824..d16738a 100644
--- a/drivers/net/ethernet/intel/ice/ice_hw_autogen.h
+++ b/drivers/net/ethernet/intel/ice/ice_hw_autogen.h
@@ -100,6 +100,7 @@
#define PF_SB_ATQT 0x0022FE00
#define PF_SB_ATQT_ATQT_S 0
#define PF_SB_ATQT_ATQT_M ICE_M(0x3FF, 0)
+#define PF_SB_REM_DEV_CTL 0x002300F0
#define PRTDCB_GENC 0x00083000
#define PRTDCB_GENC_PFCLDA_S 16
#define PRTDCB_GENC_PFCLDA_M ICE_M(0xFFFF, 16)
@@ -440,6 +441,10 @@
#define GLV_UPRCL(_i) (0x003B2000 + ((_i) * 8))
#define GLV_UPTCL(_i) (0x0030A000 + ((_i) * 8))
#define PRTRPB_RDPC 0x000AC260
+#define GLHH_ART_CTL 0x000A41D4
+#define GLHH_ART_CTL_ACTIVE_M BIT(0)
+#define GLHH_ART_TIME_H 0x000A41D8
+#define GLHH_ART_TIME_L 0x000A41DC
#define GLTSYN_AUX_IN_0(_i) (0x000889D8 + ((_i) * 4))
#define GLTSYN_AUX_IN_0_INT_ENA_M BIT(4)
#define GLTSYN_AUX_OUT_0(_i) (0x00088998 + ((_i) * 4))
@@ -452,6 +457,8 @@
#define GLTSYN_ENA_TSYN_ENA_M BIT(0)
#define GLTSYN_EVNT_H_0(_i) (0x00088970 + ((_i) * 4))
#define GLTSYN_EVNT_L_0(_i) (0x00088968 + ((_i) * 4))
+#define GLTSYN_HHTIME_H(_i) (0x00088900 + ((_i) * 4))
+#define GLTSYN_HHTIME_L(_i) (0x000888F8 + ((_i) * 4))
#define GLTSYN_INCVAL_H(_i) (0x00088920 + ((_i) * 4))
#define GLTSYN_INCVAL_L(_i) (0x00088918 + ((_i) * 4))
#define GLTSYN_SHADJ_H(_i) (0x00088910 + ((_i) * 4))
@@ -468,6 +475,8 @@
#define GLTSYN_TGT_L_0(_i) (0x00088928 + ((_i) * 4))
#define GLTSYN_TIME_H(_i) (0x000888D8 + ((_i) * 4))
#define GLTSYN_TIME_L(_i) (0x000888D0 + ((_i) * 4))
+#define PFHH_SEM 0x000A4200 /* Reset Source: PFR */
+#define PFHH_SEM_BUSY_M BIT(0)
#define PFTSYN_SEM 0x00088880
#define PFTSYN_SEM_BUSY_M BIT(0)
#define VSIQF_FD_CNT(_VSI) (0x00464000 + ((_VSI) * 4))
diff --git a/drivers/net/ethernet/intel/ice/ice_idc.c b/drivers/net/ethernet/intel/ice/ice_idc.c
index adcc9a2..fc35801 100644
--- a/drivers/net/ethernet/intel/ice/ice_idc.c
+++ b/drivers/net/ethernet/intel/ice/ice_idc.c
@@ -288,7 +288,7 @@ int ice_plug_aux_dev(struct ice_pf *pf)
adev->id = pf->aux_idx;
adev->dev.release = ice_adev_release;
adev->dev.parent = &pf->pdev->dev;
- adev->name = IIDC_RDMA_ROCE_NAME;
+ adev->name = pf->rdma_mode & IIDC_RDMA_PROTOCOL_ROCEV2 ? "roce" : "iwarp";
ret = auxiliary_device_init(adev);
if (ret) {
@@ -335,6 +335,6 @@ int ice_init_rdma(struct ice_pf *pf)
dev_err(dev, "failed to reserve vectors for RDMA\n");
return ret;
}
-
+ pf->rdma_mode |= IIDC_RDMA_PROTOCOL_ROCEV2;
return ice_plug_aux_dev(pf);
}
diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c
index 09a3297..0c187cf 100644
--- a/drivers/net/ethernet/intel/ice/ice_lib.c
+++ b/drivers/net/ethernet/intel/ice/ice_lib.c
@@ -291,7 +291,7 @@ void ice_vsi_delete(struct ice_vsi *vsi)
{
struct ice_pf *pf = vsi->back;
struct ice_vsi_ctx *ctxt;
- enum ice_status status;
+ int status;
ctxt = kzalloc(sizeof(*ctxt), GFP_KERNEL);
if (!ctxt)
@@ -305,8 +305,8 @@ void ice_vsi_delete(struct ice_vsi *vsi)
status = ice_free_vsi(&pf->hw, vsi->idx, ctxt, false, NULL);
if (status)
- dev_err(ice_pf_to_dev(pf), "Failed to delete VSI %i in FW - error: %s\n",
- vsi->vsi_num, ice_stat_str(status));
+ dev_err(ice_pf_to_dev(pf), "Failed to delete VSI %i in FW - error: %d\n",
+ vsi->vsi_num, status);
kfree(ctxt);
}
@@ -570,10 +570,16 @@ static int ice_alloc_fd_res(struct ice_vsi *vsi)
struct ice_pf *pf = vsi->back;
u32 g_val, b_val;
- /* Flow Director filters are only allocated/assigned to the PF VSI which
- * passes the traffic. The CTRL VSI is only used to add/delete filters
- * so we don't allocate resources to it
+ /* Flow Director filters are only allocated/assigned to the PF VSI or
+ * CHNL VSI which passes the traffic. The CTRL VSI is only used to
+ * add/delete filters so resources are not allocated to it
*/
+ if (!test_bit(ICE_FLAG_FD_ENA, pf->flags))
+ return -EPERM;
+
+ if (!(vsi->type == ICE_VSI_PF || vsi->type == ICE_VSI_VF ||
+ vsi->type == ICE_VSI_CHNL))
+ return -EPERM;
/* FD filters from guaranteed pool per VSI */
g_val = pf->hw.func_caps.fd_fltr_guar;
@@ -585,22 +591,59 @@ static int ice_alloc_fd_res(struct ice_vsi *vsi)
if (!b_val)
return -EPERM;
- if (!(vsi->type == ICE_VSI_PF || vsi->type == ICE_VSI_VF))
- return -EPERM;
+ /* PF main VSI gets only 64 FD resources from guaranteed pool
+ * when ADQ is configured.
+ */
+#define ICE_PF_VSI_GFLTR 64
- if (!test_bit(ICE_FLAG_FD_ENA, pf->flags))
- return -EPERM;
+ /* determine FD filter resources per VSI from shared(best effort) and
+ * dedicated pool
+ */
+ if (vsi->type == ICE_VSI_PF) {
+ vsi->num_gfltr = g_val;
+ /* if MQPRIO is configured, main VSI doesn't get all FD
+ * resources from guaranteed pool. PF VSI gets 64 FD resources
+ */
+ if (test_bit(ICE_FLAG_TC_MQPRIO, pf->flags)) {
+ if (g_val < ICE_PF_VSI_GFLTR)
+ return -EPERM;
+ /* allow bare minimum entries for PF VSI */
+ vsi->num_gfltr = ICE_PF_VSI_GFLTR;
+ }
- vsi->num_gfltr = g_val / pf->num_alloc_vsi;
-
- /* each VSI gets same "best_effort" quota */
- vsi->num_bfltr = b_val;
-
- if (vsi->type == ICE_VSI_VF) {
+ /* each VSI gets same "best_effort" quota */
+ vsi->num_bfltr = b_val;
+ } else if (vsi->type == ICE_VSI_VF) {
vsi->num_gfltr = 0;
/* each VSI gets same "best_effort" quota */
vsi->num_bfltr = b_val;
+ } else {
+ struct ice_vsi *main_vsi;
+ int numtc;
+
+ main_vsi = ice_get_main_vsi(pf);
+ if (!main_vsi)
+ return -EPERM;
+
+ if (!main_vsi->all_numtc)
+ return -EINVAL;
+
+ /* figure out ADQ numtc */
+ numtc = main_vsi->all_numtc - ICE_CHNL_START_TC;
+
+ /* only one TC but still asking resources for channels,
+ * invalid config
+ */
+ if (numtc < ICE_CHNL_START_TC)
+ return -EPERM;
+
+ g_val -= ICE_PF_VSI_GFLTR;
+ /* channel VSIs gets equal share from guaranteed pool */
+ vsi->num_gfltr = g_val / numtc;
+
+ /* each VSI gets same "best_effort" quota */
+ vsi->num_bfltr = b_val;
}
return 0;
@@ -709,15 +752,15 @@ bool ice_is_aux_ena(struct ice_pf *pf)
static void ice_vsi_clean_rss_flow_fld(struct ice_vsi *vsi)
{
struct ice_pf *pf = vsi->back;
- enum ice_status status;
+ int status;
if (ice_is_safe_mode(pf))
return;
status = ice_rem_vsi_rss_cfg(&pf->hw, vsi->idx);
if (status)
- dev_dbg(ice_pf_to_dev(pf), "ice_rem_vsi_rss_cfg failed for vsi = %d, error = %s\n",
- vsi->vsi_num, ice_stat_str(status));
+ dev_dbg(ice_pf_to_dev(pf), "ice_rem_vsi_rss_cfg failed for vsi = %d, error = %d\n",
+ vsi->vsi_num, status);
}
/**
@@ -942,7 +985,7 @@ static void ice_set_fd_vsi_ctx(struct ice_vsi_ctx *ctxt, struct ice_vsi *vsi)
u16 dflt_q, report_q, val;
if (vsi->type != ICE_VSI_PF && vsi->type != ICE_VSI_CTRL &&
- vsi->type != ICE_VSI_VF)
+ vsi->type != ICE_VSI_VF && vsi->type != ICE_VSI_CHNL)
return;
val = ICE_AQ_VSI_PROP_FLOW_DIR_VALID;
@@ -1545,8 +1588,8 @@ int ice_vsi_cfg_rss_lut_key(struct ice_vsi *vsi)
static void ice_vsi_set_vf_rss_flow_fld(struct ice_vsi *vsi)
{
struct ice_pf *pf = vsi->back;
- enum ice_status status;
struct device *dev;
+ int status;
dev = ice_pf_to_dev(pf);
if (ice_is_safe_mode(pf)) {
@@ -1557,8 +1600,8 @@ static void ice_vsi_set_vf_rss_flow_fld(struct ice_vsi *vsi)
status = ice_add_avf_rss_cfg(&pf->hw, vsi->idx, ICE_DEFAULT_RSS_HENA);
if (status)
- dev_dbg(dev, "ice_add_avf_rss_cfg failed for vsi = %d, error = %s\n",
- vsi->vsi_num, ice_stat_str(status));
+ dev_dbg(dev, "ice_add_avf_rss_cfg failed for vsi = %d, error = %d\n",
+ vsi->vsi_num, status);
}
/**
@@ -1577,8 +1620,8 @@ static void ice_vsi_set_rss_flow_fld(struct ice_vsi *vsi)
u16 vsi_handle = vsi->idx, vsi_num = vsi->vsi_num;
struct ice_pf *pf = vsi->back;
struct ice_hw *hw = &pf->hw;
- enum ice_status status;
struct device *dev;
+ int status;
dev = ice_pf_to_dev(pf);
if (ice_is_safe_mode(pf)) {
@@ -1590,57 +1633,57 @@ static void ice_vsi_set_rss_flow_fld(struct ice_vsi *vsi)
status = ice_add_rss_cfg(hw, vsi_handle, ICE_FLOW_HASH_IPV4,
ICE_FLOW_SEG_HDR_IPV4);
if (status)
- dev_dbg(dev, "ice_add_rss_cfg failed for ipv4 flow, vsi = %d, error = %s\n",
- vsi_num, ice_stat_str(status));
+ dev_dbg(dev, "ice_add_rss_cfg failed for ipv4 flow, vsi = %d, error = %d\n",
+ vsi_num, status);
/* configure RSS for IPv6 with input set IPv6 src/dst */
status = ice_add_rss_cfg(hw, vsi_handle, ICE_FLOW_HASH_IPV6,
ICE_FLOW_SEG_HDR_IPV6);
if (status)
- dev_dbg(dev, "ice_add_rss_cfg failed for ipv6 flow, vsi = %d, error = %s\n",
- vsi_num, ice_stat_str(status));
+ dev_dbg(dev, "ice_add_rss_cfg failed for ipv6 flow, vsi = %d, error = %d\n",
+ vsi_num, status);
/* configure RSS for tcp4 with input set IP src/dst, TCP src/dst */
status = ice_add_rss_cfg(hw, vsi_handle, ICE_HASH_TCP_IPV4,
ICE_FLOW_SEG_HDR_TCP | ICE_FLOW_SEG_HDR_IPV4);
if (status)
- dev_dbg(dev, "ice_add_rss_cfg failed for tcp4 flow, vsi = %d, error = %s\n",
- vsi_num, ice_stat_str(status));
+ dev_dbg(dev, "ice_add_rss_cfg failed for tcp4 flow, vsi = %d, error = %d\n",
+ vsi_num, status);
/* configure RSS for udp4 with input set IP src/dst, UDP src/dst */
status = ice_add_rss_cfg(hw, vsi_handle, ICE_HASH_UDP_IPV4,
ICE_FLOW_SEG_HDR_UDP | ICE_FLOW_SEG_HDR_IPV4);
if (status)
- dev_dbg(dev, "ice_add_rss_cfg failed for udp4 flow, vsi = %d, error = %s\n",
- vsi_num, ice_stat_str(status));
+ dev_dbg(dev, "ice_add_rss_cfg failed for udp4 flow, vsi = %d, error = %d\n",
+ vsi_num, status);
/* configure RSS for sctp4 with input set IP src/dst */
status = ice_add_rss_cfg(hw, vsi_handle, ICE_FLOW_HASH_IPV4,
ICE_FLOW_SEG_HDR_SCTP | ICE_FLOW_SEG_HDR_IPV4);
if (status)
- dev_dbg(dev, "ice_add_rss_cfg failed for sctp4 flow, vsi = %d, error = %s\n",
- vsi_num, ice_stat_str(status));
+ dev_dbg(dev, "ice_add_rss_cfg failed for sctp4 flow, vsi = %d, error = %d\n",
+ vsi_num, status);
/* configure RSS for tcp6 with input set IPv6 src/dst, TCP src/dst */
status = ice_add_rss_cfg(hw, vsi_handle, ICE_HASH_TCP_IPV6,
ICE_FLOW_SEG_HDR_TCP | ICE_FLOW_SEG_HDR_IPV6);
if (status)
- dev_dbg(dev, "ice_add_rss_cfg failed for tcp6 flow, vsi = %d, error = %s\n",
- vsi_num, ice_stat_str(status));
+ dev_dbg(dev, "ice_add_rss_cfg failed for tcp6 flow, vsi = %d, error = %d\n",
+ vsi_num, status);
/* configure RSS for udp6 with input set IPv6 src/dst, UDP src/dst */
status = ice_add_rss_cfg(hw, vsi_handle, ICE_HASH_UDP_IPV6,
ICE_FLOW_SEG_HDR_UDP | ICE_FLOW_SEG_HDR_IPV6);
if (status)
- dev_dbg(dev, "ice_add_rss_cfg failed for udp6 flow, vsi = %d, error = %s\n",
- vsi_num, ice_stat_str(status));
+ dev_dbg(dev, "ice_add_rss_cfg failed for udp6 flow, vsi = %d, error = %d\n",
+ vsi_num, status);
/* configure RSS for sctp6 with input set IPv6 src/dst */
status = ice_add_rss_cfg(hw, vsi_handle, ICE_FLOW_HASH_IPV6,
ICE_FLOW_SEG_HDR_SCTP | ICE_FLOW_SEG_HDR_IPV6);
if (status)
- dev_dbg(dev, "ice_add_rss_cfg failed for sctp6 flow, vsi = %d, error = %s\n",
- vsi_num, ice_stat_str(status));
+ dev_dbg(dev, "ice_add_rss_cfg failed for sctp6 flow, vsi = %d, error = %d\n",
+ vsi_num, status);
}
/**
@@ -1749,22 +1792,21 @@ ice_vsi_add_vlan(struct ice_vsi *vsi, u16 vid, enum ice_sw_fwd_act_type action)
int ice_vsi_kill_vlan(struct ice_vsi *vsi, u16 vid)
{
struct ice_pf *pf = vsi->back;
- enum ice_status status;
struct device *dev;
- int err = 0;
+ int err;
dev = ice_pf_to_dev(pf);
- status = ice_fltr_remove_vlan(vsi, vid, ICE_FWD_TO_VSI);
- if (!status) {
+ err = ice_fltr_remove_vlan(vsi, vid, ICE_FWD_TO_VSI);
+ if (!err) {
vsi->num_vlan--;
- } else if (status == ICE_ERR_DOES_NOT_EXIST) {
- dev_dbg(dev, "Failed to remove VLAN %d on VSI %i, it does not exist, status: %s\n",
- vid, vsi->vsi_num, ice_stat_str(status));
+ } else if (err == -ENOENT) {
+ dev_dbg(dev, "Failed to remove VLAN %d on VSI %i, it does not exist, error: %d\n",
+ vid, vsi->vsi_num, err);
+ err = 0;
} else {
- dev_err(dev, "Error removing VLAN %d on vsi %i error: %s\n",
- vid, vsi->vsi_num, ice_stat_str(status));
- err = -EIO;
+ dev_err(dev, "Error removing VLAN %d on vsi %i error: %d\n",
+ vid, vsi->vsi_num, err);
}
return err;
@@ -2105,8 +2147,7 @@ int ice_vsi_manage_vlan_insertion(struct ice_vsi *vsi)
{
struct ice_hw *hw = &vsi->back->hw;
struct ice_vsi_ctx *ctxt;
- enum ice_status status;
- int ret = 0;
+ int ret;
ctxt = kzalloc(sizeof(*ctxt), GFP_KERNEL);
if (!ctxt)
@@ -2124,12 +2165,10 @@ int ice_vsi_manage_vlan_insertion(struct ice_vsi *vsi)
ctxt->info.valid_sections = cpu_to_le16(ICE_AQ_VSI_PROP_VLAN_VALID);
- status = ice_update_vsi(hw, vsi->idx, ctxt, NULL);
- if (status) {
- dev_err(ice_pf_to_dev(vsi->back), "update VSI for VLAN insert failed, err %s aq_err %s\n",
- ice_stat_str(status),
- ice_aq_str(hw->adminq.sq_last_status));
- ret = -EIO;
+ ret = ice_update_vsi(hw, vsi->idx, ctxt, NULL);
+ if (ret) {
+ dev_err(ice_pf_to_dev(vsi->back), "update VSI for VLAN insert failed, err %d aq_err %s\n",
+ ret, ice_aq_str(hw->adminq.sq_last_status));
goto out;
}
@@ -2148,8 +2187,7 @@ int ice_vsi_manage_vlan_stripping(struct ice_vsi *vsi, bool ena)
{
struct ice_hw *hw = &vsi->back->hw;
struct ice_vsi_ctx *ctxt;
- enum ice_status status;
- int ret = 0;
+ int ret;
/* do not allow modifying VLAN stripping when a port VLAN is configured
* on this VSI
@@ -2177,12 +2215,10 @@ int ice_vsi_manage_vlan_stripping(struct ice_vsi *vsi, bool ena)
ctxt->info.valid_sections = cpu_to_le16(ICE_AQ_VSI_PROP_VLAN_VALID);
- status = ice_update_vsi(hw, vsi->idx, ctxt, NULL);
- if (status) {
- dev_err(ice_pf_to_dev(vsi->back), "update VSI for VLAN strip failed, ena = %d err %s aq_err %s\n",
- ena, ice_stat_str(status),
- ice_aq_str(hw->adminq.sq_last_status));
- ret = -EIO;
+ ret = ice_update_vsi(hw, vsi->idx, ctxt, NULL);
+ if (ret) {
+ dev_err(ice_pf_to_dev(vsi->back), "update VSI for VLAN strip failed, ena = %d err %d aq_err %s\n",
+ ena, ret, ice_aq_str(hw->adminq.sq_last_status));
goto out;
}
@@ -2324,10 +2360,9 @@ int ice_cfg_vlan_pruning(struct ice_vsi *vsi, bool ena)
status = ice_update_vsi(&pf->hw, vsi->idx, ctxt, NULL);
if (status) {
- netdev_err(vsi->netdev, "%sabling VLAN pruning on VSI handle: %d, VSI HW ID: %d failed, err = %s, aq_err = %s\n",
+ netdev_err(vsi->netdev, "%sabling VLAN pruning on VSI handle: %d, VSI HW ID: %d failed, err = %d, aq_err = %s\n",
ena ? "En" : "Dis", vsi->idx, vsi->vsi_num,
- ice_stat_str(status),
- ice_aq_str(pf->hw.adminq.sq_last_status));
+ status, ice_aq_str(pf->hw.adminq.sq_last_status));
goto err_out;
}
@@ -2405,11 +2440,11 @@ ice_vsi_set_q_vectors_reg_idx(struct ice_vsi *vsi)
*/
void ice_cfg_sw_lldp(struct ice_vsi *vsi, bool tx, bool create)
{
- enum ice_status (*eth_fltr)(struct ice_vsi *v, u16 type, u16 flag,
- enum ice_sw_fwd_act_type act);
+ int (*eth_fltr)(struct ice_vsi *v, u16 type, u16 flag,
+ enum ice_sw_fwd_act_type act);
struct ice_pf *pf = vsi->back;
- enum ice_status status;
struct device *dev;
+ int status;
dev = ice_pf_to_dev(pf);
eth_fltr = create ? ice_fltr_add_eth : ice_fltr_remove_eth;
@@ -2428,9 +2463,9 @@ void ice_cfg_sw_lldp(struct ice_vsi *vsi, bool tx, bool create)
}
if (status)
- dev_dbg(dev, "Fail %s %s LLDP rule on VSI %i error: %s\n",
+ dev_dbg(dev, "Fail %s %s LLDP rule on VSI %i error: %d\n",
create ? "adding" : "removing", tx ? "TX" : "RX",
- vsi->vsi_num, ice_stat_str(status));
+ vsi->vsi_num, status);
}
/**
@@ -2450,7 +2485,7 @@ static void ice_set_agg_vsi(struct ice_vsi *vsi)
struct ice_port_info *port_info;
struct ice_pf *pf = vsi->back;
u32 agg_node_id_start = 0;
- enum ice_status status;
+ int status;
/* create (as needed) scheduler aggregator node and move VSI into
* corresponding aggregator node
@@ -2576,7 +2611,6 @@ ice_vsi_setup(struct ice_pf *pf, struct ice_port_info *pi,
{
u16 max_txqs[ICE_MAX_TRAFFIC_CLASS] = { 0 };
struct device *dev = ice_pf_to_dev(pf);
- enum ice_status status;
struct ice_vsi *vsi;
int ret, i;
@@ -2725,11 +2759,11 @@ ice_vsi_setup(struct ice_pf *pf, struct ice_port_info *pi,
}
dev_dbg(dev, "vsi->tc_cfg.ena_tc = %d\n", vsi->tc_cfg.ena_tc);
- status = ice_cfg_vsi_lan(vsi->port_info, vsi->idx, vsi->tc_cfg.ena_tc,
- max_txqs);
- if (status) {
- dev_err(dev, "VSI %d failed lan queue config, error %s\n",
- vsi->vsi_num, ice_stat_str(status));
+ ret = ice_cfg_vsi_lan(vsi->port_info, vsi->idx, vsi->tc_cfg.ena_tc,
+ max_txqs);
+ if (ret) {
+ dev_err(dev, "VSI %d failed lan queue config, error %d\n",
+ vsi->vsi_num, ret);
goto unroll_clear_rings;
}
@@ -3036,8 +3070,8 @@ void ice_napi_del(struct ice_vsi *vsi)
*/
int ice_vsi_release(struct ice_vsi *vsi)
{
- enum ice_status err;
struct ice_pf *pf;
+ int err;
if (!vsi->back)
return -ENODEV;
@@ -3273,7 +3307,6 @@ int ice_vsi_rebuild(struct ice_vsi *vsi, bool init_vsi)
int prev_num_q_vectors = 0;
struct ice_vf *vf = NULL;
enum ice_vsi_type vtype;
- enum ice_status status;
struct ice_pf *pf;
int ret, i;
@@ -3421,14 +3454,14 @@ int ice_vsi_rebuild(struct ice_vsi *vsi, bool init_vsi)
/* If MQPRIO is set, means channel code path, hence for main
* VSI's, use TC as 1
*/
- status = ice_cfg_vsi_lan(vsi->port_info, vsi->idx, 1, max_txqs);
+ ret = ice_cfg_vsi_lan(vsi->port_info, vsi->idx, 1, max_txqs);
else
- status = ice_cfg_vsi_lan(vsi->port_info, vsi->idx,
- vsi->tc_cfg.ena_tc, max_txqs);
+ ret = ice_cfg_vsi_lan(vsi->port_info, vsi->idx,
+ vsi->tc_cfg.ena_tc, max_txqs);
- if (status) {
- dev_err(ice_pf_to_dev(pf), "VSI %d failed lan queue config, error %s\n",
- vsi->vsi_num, ice_stat_str(status));
+ if (ret) {
+ dev_err(ice_pf_to_dev(pf), "VSI %d failed lan queue config, error %d\n",
+ vsi->vsi_num, ret);
if (init_vsi) {
ret = -EIO;
goto err_vectors;
@@ -3663,7 +3696,6 @@ int ice_vsi_cfg_tc(struct ice_vsi *vsi, u8 ena_tc)
u16 max_txqs[ICE_MAX_TRAFFIC_CLASS] = { 0 };
struct ice_pf *pf = vsi->back;
struct ice_vsi_ctx *ctx;
- enum ice_status status;
struct device *dev;
int i, ret = 0;
u8 num_tc = 0;
@@ -3705,25 +3737,22 @@ int ice_vsi_cfg_tc(struct ice_vsi *vsi, u8 ena_tc)
/* must to indicate which section of VSI context are being modified */
ctx->info.valid_sections = cpu_to_le16(ICE_AQ_VSI_PROP_RXQ_MAP_VALID);
- status = ice_update_vsi(&pf->hw, vsi->idx, ctx, NULL);
- if (status) {
+ ret = ice_update_vsi(&pf->hw, vsi->idx, ctx, NULL);
+ if (ret) {
dev_info(dev, "Failed VSI Update\n");
- ret = -EIO;
goto out;
}
if (vsi->type == ICE_VSI_PF &&
test_bit(ICE_FLAG_TC_MQPRIO, pf->flags))
- status = ice_cfg_vsi_lan(vsi->port_info, vsi->idx, 1,
- max_txqs);
+ ret = ice_cfg_vsi_lan(vsi->port_info, vsi->idx, 1, max_txqs);
else
- status = ice_cfg_vsi_lan(vsi->port_info, vsi->idx,
- vsi->tc_cfg.ena_tc, max_txqs);
+ ret = ice_cfg_vsi_lan(vsi->port_info, vsi->idx,
+ vsi->tc_cfg.ena_tc, max_txqs);
- if (status) {
- dev_err(dev, "VSI %d failed TC config, error %s\n",
- vsi->vsi_num, ice_stat_str(status));
- ret = -EIO;
+ if (ret) {
+ dev_err(dev, "VSI %d failed TC config, error %d\n",
+ vsi->vsi_num, ret);
goto out;
}
ice_vsi_update_q_map(vsi, ctx);
@@ -3776,39 +3805,6 @@ void ice_update_rx_ring_stats(struct ice_rx_ring *rx_ring, u64 pkts, u64 bytes)
}
/**
- * ice_status_to_errno - convert from enum ice_status to Linux errno
- * @err: ice_status value to convert
- */
-int ice_status_to_errno(enum ice_status err)
-{
- switch (err) {
- case ICE_SUCCESS:
- return 0;
- case ICE_ERR_DOES_NOT_EXIST:
- return -ENOENT;
- case ICE_ERR_OUT_OF_RANGE:
- case ICE_ERR_AQ_ERROR:
- case ICE_ERR_AQ_TIMEOUT:
- case ICE_ERR_AQ_EMPTY:
- case ICE_ERR_AQ_FW_CRITICAL:
- return -EIO;
- case ICE_ERR_PARAM:
- case ICE_ERR_INVAL_SIZE:
- return -EINVAL;
- case ICE_ERR_NO_MEMORY:
- return -ENOMEM;
- case ICE_ERR_MAX_LIMIT:
- return -EAGAIN;
- case ICE_ERR_RESET_ONGOING:
- return -EBUSY;
- case ICE_ERR_AQ_FULL:
- return -ENOSPC;
- default:
- return -EINVAL;
- }
-}
-
-/**
* ice_is_dflt_vsi_in_use - check if the default forwarding VSI is being used
* @sw: switch to check if its default forwarding VSI is free
*
@@ -3849,8 +3845,8 @@ bool ice_is_vsi_dflt_vsi(struct ice_sw *sw, struct ice_vsi *vsi)
*/
int ice_set_dflt_vsi(struct ice_sw *sw, struct ice_vsi *vsi)
{
- enum ice_status status;
struct device *dev;
+ int status;
if (!sw || !vsi)
return -EINVAL;
@@ -3873,9 +3869,9 @@ int ice_set_dflt_vsi(struct ice_sw *sw, struct ice_vsi *vsi)
status = ice_cfg_dflt_vsi(&vsi->back->hw, vsi->idx, true, ICE_FLTR_RX);
if (status) {
- dev_err(dev, "Failed to set VSI %d as the default forwarding VSI, error %s\n",
- vsi->vsi_num, ice_stat_str(status));
- return -EIO;
+ dev_err(dev, "Failed to set VSI %d as the default forwarding VSI, error %d\n",
+ vsi->vsi_num, status);
+ return status;
}
sw->dflt_vsi = vsi;
@@ -3895,8 +3891,8 @@ int ice_set_dflt_vsi(struct ice_sw *sw, struct ice_vsi *vsi)
int ice_clear_dflt_vsi(struct ice_sw *sw)
{
struct ice_vsi *dflt_vsi;
- enum ice_status status;
struct device *dev;
+ int status;
if (!sw)
return -EINVAL;
@@ -3912,8 +3908,8 @@ int ice_clear_dflt_vsi(struct ice_sw *sw)
status = ice_cfg_dflt_vsi(&dflt_vsi->back->hw, dflt_vsi->idx, false,
ICE_FLTR_RX);
if (status) {
- dev_err(dev, "Failed to clear the default forwarding VSI %d, error %s\n",
- dflt_vsi->vsi_num, ice_stat_str(status));
+ dev_err(dev, "Failed to clear the default forwarding VSI %d, error %d\n",
+ dflt_vsi->vsi_num, status);
return -EIO;
}
@@ -3987,8 +3983,8 @@ int ice_get_link_speed_kbps(struct ice_vsi *vsi)
int ice_set_min_bw_limit(struct ice_vsi *vsi, u64 min_tx_rate)
{
struct ice_pf *pf = vsi->back;
- enum ice_status status;
struct device *dev;
+ int status;
int speed;
dev = ice_pf_to_dev(pf);
@@ -4014,7 +4010,7 @@ int ice_set_min_bw_limit(struct ice_vsi *vsi, u64 min_tx_rate)
dev_err(dev, "failed to set min Tx rate(%llu Kbps) for %s %d\n",
min_tx_rate, ice_vsi_type_str(vsi->type),
vsi->idx);
- return -EIO;
+ return status;
}
dev_dbg(dev, "set min Tx rate(%llu Kbps) for %s\n",
@@ -4026,7 +4022,7 @@ int ice_set_min_bw_limit(struct ice_vsi *vsi, u64 min_tx_rate)
if (status) {
dev_err(dev, "failed to clear min Tx rate configuration for %s %d\n",
ice_vsi_type_str(vsi->type), vsi->idx);
- return -EIO;
+ return status;
}
dev_dbg(dev, "cleared min Tx rate configuration for %s %d\n",
@@ -4048,8 +4044,8 @@ int ice_set_min_bw_limit(struct ice_vsi *vsi, u64 min_tx_rate)
int ice_set_max_bw_limit(struct ice_vsi *vsi, u64 max_tx_rate)
{
struct ice_pf *pf = vsi->back;
- enum ice_status status;
struct device *dev;
+ int status;
int speed;
dev = ice_pf_to_dev(pf);
@@ -4075,7 +4071,7 @@ int ice_set_max_bw_limit(struct ice_vsi *vsi, u64 max_tx_rate)
dev_err(dev, "failed setting max Tx rate(%llu Kbps) for %s %d\n",
max_tx_rate, ice_vsi_type_str(vsi->type),
vsi->idx);
- return -EIO;
+ return status;
}
dev_dbg(dev, "set max Tx rate(%llu Kbps) for %s %d\n",
@@ -4087,7 +4083,7 @@ int ice_set_max_bw_limit(struct ice_vsi *vsi, u64 max_tx_rate)
if (status) {
dev_err(dev, "failed clearing max Tx rate configuration for %s %d\n",
ice_vsi_type_str(vsi->type), vsi->idx);
- return -EIO;
+ return status;
}
dev_dbg(dev, "cleared max Tx rate configuration for %s %d\n",
@@ -4107,7 +4103,7 @@ int ice_set_link(struct ice_vsi *vsi, bool ena)
struct device *dev = ice_pf_to_dev(vsi->back);
struct ice_port_info *pi = vsi->port_info;
struct ice_hw *hw = pi->hw;
- enum ice_status status;
+ int status;
if (vsi->type != ICE_VSI_PF)
return -EINVAL;
@@ -4119,16 +4115,16 @@ int ice_set_link(struct ice_vsi *vsi, bool ena)
* a success code. Return an error if FW returns an error code other
* than ICE_AQ_RC_EMODE
*/
- if (status == ICE_ERR_AQ_ERROR) {
+ if (status == -EIO) {
if (hw->adminq.sq_last_status == ICE_AQ_RC_EMODE)
- dev_warn(dev, "can't set link to %s, err %s aq_err %s. not fatal, continuing\n",
- (ena ? "ON" : "OFF"), ice_stat_str(status),
+ dev_warn(dev, "can't set link to %s, err %d aq_err %s. not fatal, continuing\n",
+ (ena ? "ON" : "OFF"), status,
ice_aq_str(hw->adminq.sq_last_status));
} else if (status) {
- dev_err(dev, "can't set link to %s, err %s aq_err %s\n",
- (ena ? "ON" : "OFF"), ice_stat_str(status),
+ dev_err(dev, "can't set link to %s, err %d aq_err %s\n",
+ (ena ? "ON" : "OFF"), status,
ice_aq_str(hw->adminq.sq_last_status));
- return -EIO;
+ return status;
}
return 0;
diff --git a/drivers/net/ethernet/intel/ice/ice_lib.h b/drivers/net/ethernet/intel/ice/ice_lib.h
index 6c80340..b2ed189 100644
--- a/drivers/net/ethernet/intel/ice/ice_lib.h
+++ b/drivers/net/ethernet/intel/ice/ice_lib.h
@@ -103,15 +103,11 @@ void ice_update_tx_ring_stats(struct ice_tx_ring *ring, u64 pkts, u64 bytes);
void ice_update_rx_ring_stats(struct ice_rx_ring *ring, u64 pkts, u64 bytes);
void ice_vsi_cfg_frame_size(struct ice_vsi *vsi);
-
-int ice_status_to_errno(enum ice_status err);
-
void ice_write_intrl(struct ice_q_vector *q_vector, u8 intrl);
void ice_write_itr(struct ice_ring_container *rc, u16 itr);
void ice_set_q_vector_intrl(struct ice_q_vector *q_vector);
-enum ice_status
-ice_vsi_cfg_mac_fltr(struct ice_vsi *vsi, const u8 *macaddr, bool set);
+int ice_vsi_cfg_mac_fltr(struct ice_vsi *vsi, const u8 *macaddr, bool set);
bool ice_is_safe_mode(struct ice_pf *pf);
bool ice_is_aux_ena(struct ice_pf *pf);
diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c
index 73c61cd..3081443 100644
--- a/drivers/net/ethernet/intel/ice/ice_main.c
+++ b/drivers/net/ethernet/intel/ice/ice_main.c
@@ -155,7 +155,6 @@ static void ice_check_for_hang_subtask(struct ice_pf *pf)
*/
static int ice_init_mac_fltr(struct ice_pf *pf)
{
- enum ice_status status;
struct ice_vsi *vsi;
u8 *perm_addr;
@@ -164,11 +163,7 @@ static int ice_init_mac_fltr(struct ice_pf *pf)
return -EINVAL;
perm_addr = vsi->port_info->mac.perm_addr;
- status = ice_fltr_add_mac_and_broadcast(vsi, perm_addr, ICE_FWD_TO_VSI);
- if (status)
- return -EIO;
-
- return 0;
+ return ice_fltr_add_mac_and_broadcast(vsi, perm_addr, ICE_FWD_TO_VSI);
}
/**
@@ -237,36 +232,43 @@ static bool ice_vsi_fltr_changed(struct ice_vsi *vsi)
}
/**
- * ice_cfg_promisc - Enable or disable promiscuous mode for a given PF
+ * ice_set_promisc - Enable promiscuous mode for a given PF
* @vsi: the VSI being configured
* @promisc_m: mask of promiscuous config bits
- * @set_promisc: enable or disable promisc flag request
*
*/
-static int ice_cfg_promisc(struct ice_vsi *vsi, u8 promisc_m, bool set_promisc)
+static int ice_set_promisc(struct ice_vsi *vsi, u8 promisc_m)
{
- struct ice_hw *hw = &vsi->back->hw;
- enum ice_status status = 0;
+ int status;
if (vsi->type != ICE_VSI_PF)
return 0;
- if (vsi->num_vlan > 1) {
- status = ice_set_vlan_vsi_promisc(hw, vsi->idx, promisc_m,
- set_promisc);
- } else {
- if (set_promisc)
- status = ice_set_vsi_promisc(hw, vsi->idx, promisc_m,
- 0);
- else
- status = ice_clear_vsi_promisc(hw, vsi->idx, promisc_m,
- 0);
- }
+ if (vsi->num_vlan > 1)
+ status = ice_fltr_set_vlan_vsi_promisc(&vsi->back->hw, vsi, promisc_m);
+ else
+ status = ice_fltr_set_vsi_promisc(&vsi->back->hw, vsi->idx, promisc_m, 0);
+ return status;
+}
- if (status)
- return -EIO;
+/**
+ * ice_clear_promisc - Disable promiscuous mode for a given PF
+ * @vsi: the VSI being configured
+ * @promisc_m: mask of promiscuous config bits
+ *
+ */
+static int ice_clear_promisc(struct ice_vsi *vsi, u8 promisc_m)
+{
+ int status;
- return 0;
+ if (vsi->type != ICE_VSI_PF)
+ return 0;
+
+ if (vsi->num_vlan > 1)
+ status = ice_fltr_clear_vlan_vsi_promisc(&vsi->back->hw, vsi, promisc_m);
+ else
+ status = ice_fltr_clear_vsi_promisc(&vsi->back->hw, vsi->idx, promisc_m, 0);
+ return status;
}
/**
@@ -282,10 +284,9 @@ static int ice_vsi_sync_fltr(struct ice_vsi *vsi)
bool promisc_forced_on = false;
struct ice_pf *pf = vsi->back;
struct ice_hw *hw = &pf->hw;
- enum ice_status status = 0;
u32 changed_flags = 0;
u8 promisc_m;
- int err = 0;
+ int err;
if (!vsi->netdev)
return -EINVAL;
@@ -315,25 +316,23 @@ static int ice_vsi_sync_fltr(struct ice_vsi *vsi)
}
/* Remove MAC addresses in the unsync list */
- status = ice_fltr_remove_mac_list(vsi, &vsi->tmp_unsync_list);
+ err = ice_fltr_remove_mac_list(vsi, &vsi->tmp_unsync_list);
ice_fltr_free_list(dev, &vsi->tmp_unsync_list);
- if (status) {
+ if (err) {
netdev_err(netdev, "Failed to delete MAC filters\n");
/* if we failed because of alloc failures, just bail */
- if (status == ICE_ERR_NO_MEMORY) {
- err = -ENOMEM;
+ if (err == -ENOMEM)
goto out;
- }
}
/* Add MAC addresses in the sync list */
- status = ice_fltr_add_mac_list(vsi, &vsi->tmp_sync_list);
+ err = ice_fltr_add_mac_list(vsi, &vsi->tmp_sync_list);
ice_fltr_free_list(dev, &vsi->tmp_sync_list);
/* If filter is added successfully or already exists, do not go into
* 'if' condition and report it as error. Instead continue processing
* rest of the function.
*/
- if (status && status != ICE_ERR_ALREADY_EXISTS) {
+ if (err && err != -EEXIST) {
netdev_err(netdev, "Failed to add MAC filters\n");
/* If there is no more space for new umac filters, VSI
* should go into promiscuous mode. There should be some
@@ -346,10 +345,10 @@ static int ice_vsi_sync_fltr(struct ice_vsi *vsi)
netdev_warn(netdev, "Reached MAC filter limit, forcing promisc mode on VSI %d\n",
vsi->vsi_num);
} else {
- err = -EIO;
goto out;
}
}
+ err = 0;
/* check for changes in promiscuous modes */
if (changed_flags & IFF_ALLMULTI) {
if (vsi->current_netdev_flags & IFF_ALLMULTI) {
@@ -358,7 +357,7 @@ static int ice_vsi_sync_fltr(struct ice_vsi *vsi)
else
promisc_m = ICE_MCAST_PROMISC_BITS;
- err = ice_cfg_promisc(vsi, promisc_m, true);
+ err = ice_set_promisc(vsi, promisc_m);
if (err) {
netdev_err(netdev, "Error setting Multicast promiscuous mode on VSI %i\n",
vsi->vsi_num);
@@ -372,7 +371,7 @@ static int ice_vsi_sync_fltr(struct ice_vsi *vsi)
else
promisc_m = ICE_MCAST_PROMISC_BITS;
- err = ice_cfg_promisc(vsi, promisc_m, false);
+ err = ice_clear_promisc(vsi, promisc_m);
if (err) {
netdev_err(netdev, "Error clearing Multicast promiscuous mode on VSI %i\n",
vsi->vsi_num);
@@ -396,6 +395,7 @@ static int ice_vsi_sync_fltr(struct ice_vsi *vsi)
~IFF_PROMISC;
goto out_promisc;
}
+ err = 0;
ice_cfg_vlan_pruning(vsi, false);
}
} else {
@@ -472,6 +472,25 @@ static void ice_pf_dis_all_vsi(struct ice_pf *pf, bool locked)
}
/**
+ * ice_clear_sw_switch_recipes - clear switch recipes
+ * @pf: board private structure
+ *
+ * Mark switch recipes as not created in sw structures. There are cases where
+ * rules (especially advanced rules) need to be restored, either re-read from
+ * hardware or added again. For example after the reset. 'recp_created' flag
+ * prevents from doing that and need to be cleared upfront.
+ */
+static void ice_clear_sw_switch_recipes(struct ice_pf *pf)
+{
+ struct ice_sw_recipe *recp;
+ u8 i;
+
+ recp = pf->hw.switch_info->recp_list;
+ for (i = 0; i < ICE_MAX_NUM_RECIPES; i++)
+ recp[i].recp_created = false;
+}
+
+/**
* ice_prepare_for_reset - prep for reset
* @pf: board private structure
* @reset_type: reset type requested
@@ -501,6 +520,11 @@ ice_prepare_for_reset(struct ice_pf *pf, enum ice_reset_req reset_type)
ice_for_each_vf(pf, i)
ice_set_vf_state_qs_dis(&pf->vf[i]);
+ if (ice_is_eswitch_mode_switchdev(pf)) {
+ if (reset_type != ICE_RESET_PFR)
+ ice_clear_sw_switch_recipes(pf);
+ }
+
/* release ADQ specific HW and SW resources */
vsi = ice_get_main_vsi(pf);
if (!vsi)
@@ -539,7 +563,7 @@ ice_prepare_for_reset(struct ice_pf *pf, enum ice_reset_req reset_type)
ice_pf_dis_all_vsi(pf, false);
if (test_bit(ICE_FLAG_PTP_SUPPORTED, pf->flags))
- ice_ptp_release(pf);
+ ice_ptp_prepare_for_reset(pf);
if (hw->port_info)
ice_sched_clear_port(hw->port_info);
@@ -695,12 +719,12 @@ void ice_print_link_msg(struct ice_vsi *vsi, bool isup)
{
struct ice_aqc_get_phy_caps_data *caps;
const char *an_advertised;
- enum ice_status status;
const char *fec_req;
const char *speed;
const char *fec;
const char *fc;
const char *an;
+ int status;
if (!vsi)
return;
@@ -1020,10 +1044,10 @@ ice_link_event(struct ice_pf *pf, struct ice_port_info *pi, bool link_up,
{
struct device *dev = ice_pf_to_dev(pf);
struct ice_phy_info *phy_info;
- enum ice_status status;
struct ice_vsi *vsi;
u16 old_link_speed;
bool old_link;
+ int status;
phy_info = &pi->phy;
phy_info->link_info_old = phy_info->link_info;
@@ -1036,8 +1060,8 @@ ice_link_event(struct ice_pf *pf, struct ice_port_info *pi, bool link_up,
*/
status = ice_update_link_info(pi);
if (status)
- dev_dbg(dev, "Failed to update link status on port %d, err %s aq_err %s\n",
- pi->lport, ice_stat_str(status),
+ dev_dbg(dev, "Failed to update link status on port %d, err %d aq_err %s\n",
+ pi->lport, status,
ice_aq_str(pi->hw->adminq.sq_last_status));
ice_check_link_cfg_err(pf, pi->phy.link_info.link_cfg_err);
@@ -1063,6 +1087,9 @@ ice_link_event(struct ice_pf *pf, struct ice_port_info *pi, bool link_up,
if (link_up == old_link && link_speed == old_link_speed)
return 0;
+ if (!ice_is_e810(&pf->hw))
+ ice_ptp_link_change(pf, pf->hw.pf_id, link_up);
+
if (ice_is_dcb_active(pf)) {
if (test_bit(ICE_FLAG_DCB_ENA, pf->flags))
ice_dcb_rebuild(pf);
@@ -1407,15 +1434,15 @@ static int __ice_clean_ctrlq(struct ice_pf *pf, enum ice_ctl_q q_type)
return 0;
do {
- enum ice_status ret;
u16 opcode;
+ int ret;
ret = ice_clean_rq_elem(hw, cq, &event, &pending);
- if (ret == ICE_ERR_AQ_NO_WORK)
+ if (ret == -EALREADY)
break;
if (ret) {
- dev_err(dev, "%s Receive Queue event error %s\n", qtype,
- ice_stat_str(ret));
+ dev_err(dev, "%s Receive Queue event error %d\n", qtype,
+ ret);
break;
}
@@ -1866,19 +1893,17 @@ static int ice_init_nvm_phy_type(struct ice_port_info *pi)
{
struct ice_aqc_get_phy_caps_data *pcaps;
struct ice_pf *pf = pi->hw->back;
- enum ice_status status;
- int err = 0;
+ int err;
pcaps = kzalloc(sizeof(*pcaps), GFP_KERNEL);
if (!pcaps)
return -ENOMEM;
- status = ice_aq_get_phy_caps(pi, false, ICE_AQC_REPORT_TOPO_CAP_NO_MEDIA, pcaps,
- NULL);
+ err = ice_aq_get_phy_caps(pi, false, ICE_AQC_REPORT_TOPO_CAP_NO_MEDIA,
+ pcaps, NULL);
- if (status) {
+ if (err) {
dev_err(ice_pf_to_dev(pf), "Get PHY capability failed.\n");
- err = -EIO;
goto out;
}
@@ -1977,8 +2002,7 @@ static int ice_init_phy_user_cfg(struct ice_port_info *pi)
struct ice_aqc_get_phy_caps_data *pcaps;
struct ice_phy_info *phy = &pi->phy;
struct ice_pf *pf = pi->hw->back;
- enum ice_status status;
- int err = 0;
+ int err;
if (!(phy->link_info.link_info & ICE_AQ_MEDIA_AVAILABLE))
return -EIO;
@@ -1988,14 +2012,13 @@ static int ice_init_phy_user_cfg(struct ice_port_info *pi)
return -ENOMEM;
if (ice_fw_supports_report_dflt_cfg(pi->hw))
- status = ice_aq_get_phy_caps(pi, false, ICE_AQC_REPORT_DFLT_CFG,
- pcaps, NULL);
+ err = ice_aq_get_phy_caps(pi, false, ICE_AQC_REPORT_DFLT_CFG,
+ pcaps, NULL);
else
- status = ice_aq_get_phy_caps(pi, false, ICE_AQC_REPORT_TOPO_CAP_MEDIA,
- pcaps, NULL);
- if (status) {
+ err = ice_aq_get_phy_caps(pi, false, ICE_AQC_REPORT_TOPO_CAP_MEDIA,
+ pcaps, NULL);
+ if (err) {
dev_err(ice_pf_to_dev(pf), "Get PHY capability failed.\n");
- err = -EIO;
goto err_out;
}
@@ -2049,8 +2072,7 @@ static int ice_configure_phy(struct ice_vsi *vsi)
struct ice_aqc_set_phy_cfg_data *cfg;
struct ice_phy_info *phy = &pi->phy;
struct ice_pf *pf = vsi->back;
- enum ice_status status;
- int err = 0;
+ int err;
/* Ensure we have media as we cannot configure a medialess port */
if (!(phy->link_info.link_info & ICE_AQ_MEDIA_AVAILABLE))
@@ -2070,12 +2092,11 @@ static int ice_configure_phy(struct ice_vsi *vsi)
return -ENOMEM;
/* Get current PHY config */
- status = ice_aq_get_phy_caps(pi, false, ICE_AQC_REPORT_ACTIVE_CFG, pcaps,
- NULL);
- if (status) {
- dev_err(dev, "Failed to get PHY configuration, VSI %d error %s\n",
- vsi->vsi_num, ice_stat_str(status));
- err = -EIO;
+ err = ice_aq_get_phy_caps(pi, false, ICE_AQC_REPORT_ACTIVE_CFG, pcaps,
+ NULL);
+ if (err) {
+ dev_err(dev, "Failed to get PHY configuration, VSI %d error %d\n",
+ vsi->vsi_num, err);
goto done;
}
@@ -2089,15 +2110,14 @@ static int ice_configure_phy(struct ice_vsi *vsi)
/* Use PHY topology as baseline for configuration */
memset(pcaps, 0, sizeof(*pcaps));
if (ice_fw_supports_report_dflt_cfg(pi->hw))
- status = ice_aq_get_phy_caps(pi, false, ICE_AQC_REPORT_DFLT_CFG,
- pcaps, NULL);
+ err = ice_aq_get_phy_caps(pi, false, ICE_AQC_REPORT_DFLT_CFG,
+ pcaps, NULL);
else
- status = ice_aq_get_phy_caps(pi, false, ICE_AQC_REPORT_TOPO_CAP_MEDIA,
- pcaps, NULL);
- if (status) {
- dev_err(dev, "Failed to get PHY caps, VSI %d error %s\n",
- vsi->vsi_num, ice_stat_str(status));
- err = -EIO;
+ err = ice_aq_get_phy_caps(pi, false, ICE_AQC_REPORT_TOPO_CAP_MEDIA,
+ pcaps, NULL);
+ if (err) {
+ dev_err(dev, "Failed to get PHY caps, VSI %d error %d\n",
+ vsi->vsi_num, err);
goto done;
}
@@ -2150,12 +2170,10 @@ static int ice_configure_phy(struct ice_vsi *vsi)
/* Enable link and link update */
cfg->caps |= ICE_AQ_PHY_ENA_AUTO_LINK_UPDT | ICE_AQ_PHY_ENA_LINK;
- status = ice_aq_set_phy_cfg(&pf->hw, pi, cfg, NULL);
- if (status) {
- dev_err(dev, "Failed to set phy config, VSI %d error %s\n",
- vsi->vsi_num, ice_stat_str(status));
- err = -EIO;
- }
+ err = ice_aq_set_phy_cfg(&pf->hw, pi, cfg, NULL);
+ if (err)
+ dev_err(dev, "Failed to set phy config, VSI %d error %d\n",
+ vsi->vsi_num, err);
kfree(cfg);
done:
@@ -2549,9 +2567,9 @@ int ice_prepare_xdp_rings(struct ice_vsi *vsi, struct bpf_prog *prog)
.vsi_map_offset = vsi->alloc_txq,
.mapping_mode = ICE_VSI_MAP_CONTIG
};
- enum ice_status status;
struct device *dev;
int i, v_idx;
+ int status;
dev = ice_pf_to_dev(pf);
vsi->xdp_rings = devm_kcalloc(dev, vsi->num_xdp_txq,
@@ -2605,8 +2623,8 @@ int ice_prepare_xdp_rings(struct ice_vsi *vsi, struct bpf_prog *prog)
status = ice_cfg_vsi_lan(vsi->port_info, vsi->idx, vsi->tc_cfg.ena_tc,
max_txqs);
if (status) {
- dev_err(dev, "Failed VSI LAN queue config for XDP, error: %s\n",
- ice_stat_str(status));
+ dev_err(dev, "Failed VSI LAN queue config for XDP, error: %d\n",
+ status);
goto clear_xdp_rings;
}
@@ -3520,7 +3538,7 @@ static int ice_setup_pf_sw(struct ice_pf *pf)
{
struct device *dev = ice_pf_to_dev(pf);
struct ice_vsi *vsi;
- int status = 0;
+ int status;
if (ice_is_reset_in_progress(pf->state))
return -EBUSY;
@@ -3533,10 +3551,8 @@ static int ice_setup_pf_sw(struct ice_pf *pf)
INIT_LIST_HEAD(&vsi->ch_list);
status = ice_cfg_netdev(vsi);
- if (status) {
- status = -ENODEV;
+ if (status)
goto unroll_vsi_setup;
- }
/* netdev has to be configured before setting frame size */
ice_vsi_cfg_frame_size(vsi);
@@ -3560,14 +3576,13 @@ static int ice_setup_pf_sw(struct ice_pf *pf)
if (status) {
dev_err(dev, "Failed to set CPU Rx map VSI %d error %d\n",
vsi->vsi_num, status);
- status = -EINVAL;
goto unroll_napi_add;
}
status = ice_init_mac_fltr(pf);
if (status)
goto free_cpu_rx_map;
- return status;
+ return 0;
free_cpu_rx_map:
ice_free_cpu_rx_rmap(vsi);
@@ -4023,8 +4038,8 @@ static void ice_set_safe_mode_vlan_cfg(struct ice_pf *pf)
{
struct ice_vsi *vsi = ice_get_main_vsi(pf);
struct ice_vsi_ctx *ctxt;
- enum ice_status status;
struct ice_hw *hw;
+ int status;
if (!vsi)
return;
@@ -4054,9 +4069,8 @@ static void ice_set_safe_mode_vlan_cfg(struct ice_pf *pf)
status = ice_update_vsi(hw, vsi->idx, ctxt, NULL);
if (status) {
- dev_err(ice_pf_to_dev(vsi->back), "Failed to update VSI for safe mode VLANs, err %s aq_err %s\n",
- ice_stat_str(status),
- ice_aq_str(hw->adminq.sq_last_status));
+ dev_err(ice_pf_to_dev(vsi->back), "Failed to update VSI for safe mode VLANs, err %d aq_err %s\n",
+ status, ice_aq_str(hw->adminq.sq_last_status));
} else {
vsi->info.sec_flags = ctxt->info.sec_flags;
vsi->info.sw_flags2 = ctxt->info.sw_flags2;
@@ -4069,108 +4083,80 @@ static void ice_set_safe_mode_vlan_cfg(struct ice_pf *pf)
/**
* ice_log_pkg_init - log result of DDP package load
* @hw: pointer to hardware info
- * @status: status of package load
+ * @state: state of package load
*/
-static void
-ice_log_pkg_init(struct ice_hw *hw, enum ice_status *status)
+static void ice_log_pkg_init(struct ice_hw *hw, enum ice_ddp_state state)
{
- struct ice_pf *pf = (struct ice_pf *)hw->back;
- struct device *dev = ice_pf_to_dev(pf);
+ struct ice_pf *pf = hw->back;
+ struct device *dev;
- switch (*status) {
- case ICE_SUCCESS:
- /* The package download AdminQ command returned success because
- * this download succeeded or ICE_ERR_AQ_NO_WORK since there is
- * already a package loaded on the device.
- */
- if (hw->pkg_ver.major == hw->active_pkg_ver.major &&
- hw->pkg_ver.minor == hw->active_pkg_ver.minor &&
- hw->pkg_ver.update == hw->active_pkg_ver.update &&
- hw->pkg_ver.draft == hw->active_pkg_ver.draft &&
- !memcmp(hw->pkg_name, hw->active_pkg_name,
- sizeof(hw->pkg_name))) {
- if (hw->pkg_dwnld_status == ICE_AQ_RC_EEXIST)
- dev_info(dev, "DDP package already present on device: %s version %d.%d.%d.%d\n",
- hw->active_pkg_name,
- hw->active_pkg_ver.major,
- hw->active_pkg_ver.minor,
- hw->active_pkg_ver.update,
- hw->active_pkg_ver.draft);
- else
- dev_info(dev, "The DDP package was successfully loaded: %s version %d.%d.%d.%d\n",
- hw->active_pkg_name,
- hw->active_pkg_ver.major,
- hw->active_pkg_ver.minor,
- hw->active_pkg_ver.update,
- hw->active_pkg_ver.draft);
- } else if (hw->active_pkg_ver.major != ICE_PKG_SUPP_VER_MAJ ||
- hw->active_pkg_ver.minor != ICE_PKG_SUPP_VER_MNR) {
- dev_err(dev, "The device has a DDP package that is not supported by the driver. The device has package '%s' version %d.%d.x.x. The driver requires version %d.%d.x.x. Entering Safe Mode.\n",
- hw->active_pkg_name,
- hw->active_pkg_ver.major,
- hw->active_pkg_ver.minor,
- ICE_PKG_SUPP_VER_MAJ, ICE_PKG_SUPP_VER_MNR);
- *status = ICE_ERR_NOT_SUPPORTED;
- } else if (hw->active_pkg_ver.major == ICE_PKG_SUPP_VER_MAJ &&
- hw->active_pkg_ver.minor == ICE_PKG_SUPP_VER_MNR) {
- dev_info(dev, "The driver could not load the DDP package file because a compatible DDP package is already present on the device. The device has package '%s' version %d.%d.%d.%d. The package file found by the driver: '%s' version %d.%d.%d.%d.\n",
- hw->active_pkg_name,
- hw->active_pkg_ver.major,
- hw->active_pkg_ver.minor,
- hw->active_pkg_ver.update,
- hw->active_pkg_ver.draft,
- hw->pkg_name,
- hw->pkg_ver.major,
- hw->pkg_ver.minor,
- hw->pkg_ver.update,
- hw->pkg_ver.draft);
- } else {
- dev_err(dev, "An unknown error occurred when loading the DDP package, please reboot the system. If the problem persists, update the NVM. Entering Safe Mode.\n");
- *status = ICE_ERR_NOT_SUPPORTED;
- }
+ dev = ice_pf_to_dev(pf);
+
+ switch (state) {
+ case ICE_DDP_PKG_SUCCESS:
+ dev_info(dev, "The DDP package was successfully loaded: %s version %d.%d.%d.%d\n",
+ hw->active_pkg_name,
+ hw->active_pkg_ver.major,
+ hw->active_pkg_ver.minor,
+ hw->active_pkg_ver.update,
+ hw->active_pkg_ver.draft);
break;
- case ICE_ERR_FW_DDP_MISMATCH:
+ case ICE_DDP_PKG_SAME_VERSION_ALREADY_LOADED:
+ dev_info(dev, "DDP package already present on device: %s version %d.%d.%d.%d\n",
+ hw->active_pkg_name,
+ hw->active_pkg_ver.major,
+ hw->active_pkg_ver.minor,
+ hw->active_pkg_ver.update,
+ hw->active_pkg_ver.draft);
+ break;
+ case ICE_DDP_PKG_ALREADY_LOADED_NOT_SUPPORTED:
+ dev_err(dev, "The device has a DDP package that is not supported by the driver. The device has package '%s' version %d.%d.x.x. The driver requires version %d.%d.x.x. Entering Safe Mode.\n",
+ hw->active_pkg_name,
+ hw->active_pkg_ver.major,
+ hw->active_pkg_ver.minor,
+ ICE_PKG_SUPP_VER_MAJ, ICE_PKG_SUPP_VER_MNR);
+ break;
+ case ICE_DDP_PKG_COMPATIBLE_ALREADY_LOADED:
+ dev_info(dev, "The driver could not load the DDP package file because a compatible DDP package is already present on the device. The device has package '%s' version %d.%d.%d.%d. The package file found by the driver: '%s' version %d.%d.%d.%d.\n",
+ hw->active_pkg_name,
+ hw->active_pkg_ver.major,
+ hw->active_pkg_ver.minor,
+ hw->active_pkg_ver.update,
+ hw->active_pkg_ver.draft,
+ hw->pkg_name,
+ hw->pkg_ver.major,
+ hw->pkg_ver.minor,
+ hw->pkg_ver.update,
+ hw->pkg_ver.draft);
+ break;
+ case ICE_DDP_PKG_FW_MISMATCH:
dev_err(dev, "The firmware loaded on the device is not compatible with the DDP package. Please update the device's NVM. Entering safe mode.\n");
break;
- case ICE_ERR_BUF_TOO_SHORT:
- case ICE_ERR_CFG:
+ case ICE_DDP_PKG_INVALID_FILE:
dev_err(dev, "The DDP package file is invalid. Entering Safe Mode.\n");
break;
- case ICE_ERR_NOT_SUPPORTED:
- /* Package File version not supported */
- if (hw->pkg_ver.major > ICE_PKG_SUPP_VER_MAJ ||
- (hw->pkg_ver.major == ICE_PKG_SUPP_VER_MAJ &&
- hw->pkg_ver.minor > ICE_PKG_SUPP_VER_MNR))
- dev_err(dev, "The DDP package file version is higher than the driver supports. Please use an updated driver. Entering Safe Mode.\n");
- else if (hw->pkg_ver.major < ICE_PKG_SUPP_VER_MAJ ||
- (hw->pkg_ver.major == ICE_PKG_SUPP_VER_MAJ &&
- hw->pkg_ver.minor < ICE_PKG_SUPP_VER_MNR))
- dev_err(dev, "The DDP package file version is lower than the driver supports. The driver requires version %d.%d.x.x. Please use an updated DDP Package file. Entering Safe Mode.\n",
- ICE_PKG_SUPP_VER_MAJ, ICE_PKG_SUPP_VER_MNR);
+ case ICE_DDP_PKG_FILE_VERSION_TOO_HIGH:
+ dev_err(dev, "The DDP package file version is higher than the driver supports. Please use an updated driver. Entering Safe Mode.\n");
break;
- case ICE_ERR_AQ_ERROR:
- switch (hw->pkg_dwnld_status) {
- case ICE_AQ_RC_ENOSEC:
- case ICE_AQ_RC_EBADSIG:
- dev_err(dev, "The DDP package could not be loaded because its signature is not valid. Please use a valid DDP Package. Entering Safe Mode.\n");
- return;
- case ICE_AQ_RC_ESVN:
- dev_err(dev, "The DDP Package could not be loaded because its security revision is too low. Please use an updated DDP Package. Entering Safe Mode.\n");
- return;
- case ICE_AQ_RC_EBADMAN:
- case ICE_AQ_RC_EBADBUF:
- dev_err(dev, "An error occurred on the device while loading the DDP package. The device will be reset.\n");
- /* poll for reset to complete */
- if (ice_check_reset(hw))
- dev_err(dev, "Error resetting device. Please reload the driver\n");
- return;
- default:
- break;
- }
- fallthrough;
+ case ICE_DDP_PKG_FILE_VERSION_TOO_LOW:
+ dev_err(dev, "The DDP package file version is lower than the driver supports. The driver requires version %d.%d.x.x. Please use an updated DDP Package file. Entering Safe Mode.\n",
+ ICE_PKG_SUPP_VER_MAJ, ICE_PKG_SUPP_VER_MNR);
+ break;
+ case ICE_DDP_PKG_FILE_SIGNATURE_INVALID:
+ dev_err(dev, "The DDP package could not be loaded because its signature is not valid. Please use a valid DDP Package. Entering Safe Mode.\n");
+ break;
+ case ICE_DDP_PKG_FILE_REVISION_TOO_LOW:
+ dev_err(dev, "The DDP Package could not be loaded because its security revision is too low. Please use an updated DDP Package. Entering Safe Mode.\n");
+ break;
+ case ICE_DDP_PKG_LOAD_ERROR:
+ dev_err(dev, "An error occurred on the device while loading the DDP package. The device will be reset.\n");
+ /* poll for reset to complete */
+ if (ice_check_reset(hw))
+ dev_err(dev, "Error resetting device. Please reload the driver\n");
+ break;
+ case ICE_DDP_PKG_ERR:
default:
- dev_err(dev, "An unknown error (%d) occurred when loading the DDP package. Entering Safe Mode.\n",
- *status);
+ dev_err(dev, "An unknown error occurred when loading the DDP package. Entering Safe Mode.\n");
break;
}
}
@@ -4186,24 +4172,24 @@ ice_log_pkg_init(struct ice_hw *hw, enum ice_status *status)
static void
ice_load_pkg(const struct firmware *firmware, struct ice_pf *pf)
{
- enum ice_status status = ICE_ERR_PARAM;
+ enum ice_ddp_state state = ICE_DDP_PKG_ERR;
struct device *dev = ice_pf_to_dev(pf);
struct ice_hw *hw = &pf->hw;
/* Load DDP Package */
if (firmware && !hw->pkg_copy) {
- status = ice_copy_and_init_pkg(hw, firmware->data,
- firmware->size);
- ice_log_pkg_init(hw, &status);
+ state = ice_copy_and_init_pkg(hw, firmware->data,
+ firmware->size);
+ ice_log_pkg_init(hw, state);
} else if (!firmware && hw->pkg_copy) {
/* Reload package during rebuild after CORER/GLOBR reset */
- status = ice_init_pkg(hw, hw->pkg_copy, hw->pkg_size);
- ice_log_pkg_init(hw, &status);
+ state = ice_init_pkg(hw, hw->pkg_copy, hw->pkg_size);
+ ice_log_pkg_init(hw, state);
} else {
dev_err(dev, "The DDP package file failed to load. Entering Safe Mode.\n");
}
- if (status) {
+ if (!ice_is_init_pkg_successful(state)) {
/* Safe Mode */
clear_bit(ICE_FLAG_ADV_FEATURES, pf->flags);
return;
@@ -4234,9 +4220,9 @@ static void ice_verify_cacheline_size(struct ice_pf *pf)
* ice_send_version - update firmware with driver version
* @pf: PF struct
*
- * Returns ICE_SUCCESS on success, else error code
+ * Returns 0 on success, else error code
*/
-static enum ice_status ice_send_version(struct ice_pf *pf)
+static int ice_send_version(struct ice_pf *pf)
{
struct ice_driver_ver dv;
@@ -4526,7 +4512,6 @@ ice_probe(struct pci_dev *pdev, const struct pci_device_id __always_unused *ent)
* true
*/
if (ice_is_safe_mode(pf)) {
- dev_err(dev, "Package download failed. Advanced features disabled - Device now in Safe Mode\n");
/* we already got function/device capabilities but these don't
* reflect what the driver needs to do in safe mode. Instead of
* adding conditional logic everywhere to ignore these
@@ -4721,6 +4706,10 @@ ice_probe(struct pci_dev *pdev, const struct pci_device_id __always_unused *ent)
if (err)
goto err_netdev_reg;
+ err = ice_devlink_register_params(pf);
+ if (err)
+ goto err_netdev_reg;
+
/* ready to go, so clear down state bit */
clear_bit(ICE_DOWN, pf->state);
if (ice_is_aux_ena(pf)) {
@@ -4728,7 +4717,7 @@ ice_probe(struct pci_dev *pdev, const struct pci_device_id __always_unused *ent)
if (pf->aux_idx < 0) {
dev_err(dev, "Failed to allocate device ID for AUX driver\n");
err = -ENOMEM;
- goto err_netdev_reg;
+ goto err_devlink_reg_param;
}
err = ice_init_rdma(pf);
@@ -4747,6 +4736,8 @@ ice_probe(struct pci_dev *pdev, const struct pci_device_id __always_unused *ent)
err_init_aux_unroll:
pf->adev = NULL;
ida_free(&ice_aux_ida, pf->aux_idx);
+err_devlink_reg_param:
+ ice_devlink_unregister_params(pf);
err_netdev_reg:
err_send_version_unroll:
ice_vsi_release_all(pf);
@@ -4803,9 +4794,9 @@ static void ice_setup_mc_magic_wake(struct ice_pf *pf)
{
struct device *dev = ice_pf_to_dev(pf);
struct ice_hw *hw = &pf->hw;
- enum ice_status status;
u8 mac_addr[ETH_ALEN];
struct ice_vsi *vsi;
+ int status;
u8 flags;
if (!pf->wol_ena)
@@ -4827,9 +4818,8 @@ static void ice_setup_mc_magic_wake(struct ice_pf *pf)
status = ice_aq_manage_mac_write(hw, mac_addr, flags, NULL);
if (status)
- dev_err(dev, "Failed to enable Multicast Magic Packet wake, err %s aq_err %s\n",
- ice_stat_str(status),
- ice_aq_str(hw->adminq.sq_last_status));
+ dev_err(dev, "Failed to enable Multicast Magic Packet wake, err %d aq_err %s\n",
+ status, ice_aq_str(hw->adminq.sq_last_status));
}
/**
@@ -4861,6 +4851,7 @@ static void ice_remove(struct pci_dev *pdev)
ice_unplug_aux_dev(pf);
if (pf->aux_idx >= 0)
ida_free(&ice_aux_ida, pf->aux_idx);
+ ice_devlink_unregister_params(pf);
set_bit(ICE_DOWN, pf->state);
mutex_destroy(&(&pf->hw)->fdir_fltr_lock);
@@ -5367,11 +5358,10 @@ static int ice_set_mac_address(struct net_device *netdev, void *pi)
struct ice_pf *pf = vsi->back;
struct ice_hw *hw = &pf->hw;
struct sockaddr *addr = pi;
- enum ice_status status;
u8 old_mac[ETH_ALEN];
u8 flags = 0;
- int err = 0;
u8 *mac;
+ int err;
mac = (u8 *)addr->sa_data;
@@ -5403,22 +5393,22 @@ static int ice_set_mac_address(struct net_device *netdev, void *pi)
netif_addr_unlock_bh(netdev);
/* Clean up old MAC filter. Not an error if old filter doesn't exist */
- status = ice_fltr_remove_mac(vsi, old_mac, ICE_FWD_TO_VSI);
- if (status && status != ICE_ERR_DOES_NOT_EXIST) {
+ err = ice_fltr_remove_mac(vsi, old_mac, ICE_FWD_TO_VSI);
+ if (err && err != -ENOENT) {
err = -EADDRNOTAVAIL;
goto err_update_filters;
}
/* Add filter for new MAC. If filter exists, return success */
- status = ice_fltr_add_mac(vsi, mac, ICE_FWD_TO_VSI);
- if (status == ICE_ERR_ALREADY_EXISTS)
+ err = ice_fltr_add_mac(vsi, mac, ICE_FWD_TO_VSI);
+ if (err == -EEXIST)
/* Although this MAC filter is already present in hardware it's
* possible in some cases (e.g. bonding) that dev_addr was
* modified outside of the driver and needs to be restored back
* to this value.
*/
netdev_dbg(netdev, "filter for MAC %pM already exists\n", mac);
- else if (status)
+ else if (err)
/* error if the new filter addition failed */
err = -EADDRNOTAVAIL;
@@ -5437,10 +5427,10 @@ static int ice_set_mac_address(struct net_device *netdev, void *pi)
/* write new MAC address to the firmware */
flags = ICE_AQC_MAN_MAC_UPDATE_LAA_WOL;
- status = ice_aq_manage_mac_write(hw, mac, flags, NULL);
- if (status) {
- netdev_err(netdev, "can't set MAC %pM. write to firmware failed error %s\n",
- mac, ice_stat_str(status));
+ err = ice_aq_manage_mac_write(hw, mac, flags, NULL);
+ if (err) {
+ netdev_err(netdev, "can't set MAC %pM. write to firmware failed error %d\n",
+ mac, err);
}
return 0;
}
@@ -5482,8 +5472,8 @@ ice_set_tx_maxrate(struct net_device *netdev, int queue_index, u32 maxrate)
{
struct ice_netdev_priv *np = netdev_priv(netdev);
struct ice_vsi *vsi = np->vsi;
- enum ice_status status;
u16 q_handle;
+ int status;
u8 tc;
/* Validate maxrate requested is within permitted range */
@@ -5503,13 +5493,11 @@ ice_set_tx_maxrate(struct net_device *netdev, int queue_index, u32 maxrate)
else
status = ice_cfg_q_bw_lmt(vsi->port_info, vsi->idx, tc,
q_handle, ICE_MAX_BW, maxrate * 1000);
- if (status) {
- netdev_err(netdev, "Unable to set Tx max rate, error %s\n",
- ice_stat_str(status));
- return -EIO;
- }
+ if (status)
+ netdev_err(netdev, "Unable to set Tx max rate, error %d\n",
+ status);
- return 0;
+ return status;
}
/**
@@ -5879,6 +5867,8 @@ static int ice_up_complete(struct ice_vsi *vsi)
ice_print_link_msg(vsi, true);
netif_tx_start_all_queues(vsi->netdev);
netif_carrier_on(vsi->netdev);
+ if (!ice_is_e810(&pf->hw))
+ ice_ptp_link_change(pf, pf->hw.pf_id, true);
}
/* clear this now, and the first stats read will be used as baseline */
@@ -6269,15 +6259,18 @@ static void ice_napi_disable_all(struct ice_vsi *vsi)
/**
* ice_down - Shutdown the connection
* @vsi: The VSI being stopped
+ *
+ * Caller of this function is expected to set the vsi->state ICE_DOWN bit
*/
int ice_down(struct ice_vsi *vsi)
{
int i, tx_err, rx_err, link_err = 0;
- /* Caller of this function is expected to set the
- * vsi->state ICE_DOWN bit
- */
+ WARN_ON(!test_bit(ICE_VSI_DOWN, vsi->state));
+
if (vsi->netdev && vsi->type == ICE_VSI_PF) {
+ if (!ice_is_e810(&vsi->back->hw))
+ ice_ptp_link_change(vsi->back, vsi->back->hw.pf_id, false);
netif_carrier_off(vsi->netdev);
netif_tx_disable(vsi->netdev);
} else if (vsi->type == ICE_VSI_SWITCHDEV_CTRL) {
@@ -6543,7 +6536,6 @@ static void ice_vsi_release_all(struct ice_pf *pf)
static int ice_vsi_rebuild_by_type(struct ice_pf *pf, enum ice_vsi_type type)
{
struct device *dev = ice_pf_to_dev(pf);
- enum ice_status status;
int i, err;
ice_for_each_vsi(pf, i) {
@@ -6561,12 +6553,11 @@ static int ice_vsi_rebuild_by_type(struct ice_pf *pf, enum ice_vsi_type type)
}
/* replay filters for the VSI */
- status = ice_replay_vsi(&pf->hw, vsi->idx);
- if (status) {
- dev_err(dev, "replay VSI failed, status %s, VSI index %d, type %s\n",
- ice_stat_str(status), vsi->idx,
- ice_vsi_type_str(type));
- return -EIO;
+ err = ice_replay_vsi(&pf->hw, vsi->idx);
+ if (err) {
+ dev_err(dev, "replay VSI failed, error %d, VSI index %d, type %s\n",
+ err, vsi->idx, ice_vsi_type_str(type));
+ return err;
}
/* Re-map HW VSI number, using VSI handle that has been
@@ -6629,7 +6620,6 @@ static void ice_rebuild(struct ice_pf *pf, enum ice_reset_req reset_type)
{
struct device *dev = ice_pf_to_dev(pf);
struct ice_hw *hw = &pf->hw;
- enum ice_status ret;
int err;
if (test_bit(ICE_DOWN, pf->state))
@@ -6637,10 +6627,17 @@ static void ice_rebuild(struct ice_pf *pf, enum ice_reset_req reset_type)
dev_dbg(dev, "rebuilding PF after reset_type=%d\n", reset_type);
- ret = ice_init_all_ctrlq(hw);
- if (ret) {
- dev_err(dev, "control queues init failed %s\n",
- ice_stat_str(ret));
+ if (reset_type == ICE_RESET_EMPR) {
+ /* If an EMP reset has occurred, any previously pending flash
+ * update will have completed. We no longer know whether or
+ * not the NVM update EMP reset is restricted.
+ */
+ pf->fw_emp_reset_disabled = false;
+ }
+
+ err = ice_init_all_ctrlq(hw);
+ if (err) {
+ dev_err(dev, "control queues init failed %d\n", err);
goto err_init_ctrlq;
}
@@ -6654,10 +6651,9 @@ static void ice_rebuild(struct ice_pf *pf, enum ice_reset_req reset_type)
ice_load_pkg(NULL, pf);
}
- ret = ice_clear_pf_cfg(hw);
- if (ret) {
- dev_err(dev, "clear PF configuration failed %s\n",
- ice_stat_str(ret));
+ err = ice_clear_pf_cfg(hw);
+ if (err) {
+ dev_err(dev, "clear PF configuration failed %d\n", err);
goto err_init_ctrlq;
}
@@ -6669,21 +6665,21 @@ static void ice_rebuild(struct ice_pf *pf, enum ice_reset_req reset_type)
ice_clear_pxe_mode(hw);
- ret = ice_init_nvm(hw);
- if (ret) {
- dev_err(dev, "ice_init_nvm failed %s\n", ice_stat_str(ret));
+ err = ice_init_nvm(hw);
+ if (err) {
+ dev_err(dev, "ice_init_nvm failed %d\n", err);
goto err_init_ctrlq;
}
- ret = ice_get_caps(hw);
- if (ret) {
- dev_err(dev, "ice_get_caps failed %s\n", ice_stat_str(ret));
+ err = ice_get_caps(hw);
+ if (err) {
+ dev_err(dev, "ice_get_caps failed %d\n", err);
goto err_init_ctrlq;
}
- ret = ice_aq_set_mac_cfg(hw, ICE_AQ_SET_MAC_FRAME_SIZE_MAX, NULL);
- if (ret) {
- dev_err(dev, "set_mac_cfg failed %s\n", ice_stat_str(ret));
+ err = ice_aq_set_mac_cfg(hw, ICE_AQ_SET_MAC_FRAME_SIZE_MAX, NULL);
+ if (err) {
+ dev_err(dev, "set_mac_cfg failed %d\n", err);
goto err_init_ctrlq;
}
@@ -6721,7 +6717,7 @@ static void ice_rebuild(struct ice_pf *pf, enum ice_reset_req reset_type)
* fail.
*/
if (test_bit(ICE_FLAG_PTP_SUPPORTED, pf->flags))
- ice_ptp_init(pf);
+ ice_ptp_reset(pf);
/* rebuild PF VSI */
err = ice_vsi_rebuild_by_type(pf, ICE_VSI_PF);
@@ -6730,6 +6726,10 @@ static void ice_rebuild(struct ice_pf *pf, enum ice_reset_req reset_type)
goto err_vsi_rebuild;
}
+ /* configure PTP timestamping after VSI rebuild */
+ if (test_bit(ICE_FLAG_PTP_SUPPORTED, pf->flags))
+ ice_ptp_cfg_timestamp(pf, false);
+
err = ice_vsi_rebuild_by_type(pf, ICE_VSI_SWITCHDEV_CTRL);
if (err) {
dev_err(dev, "Switchdev CTRL VSI rebuild failed: %d\n", err);
@@ -6766,10 +6766,10 @@ static void ice_rebuild(struct ice_pf *pf, enum ice_reset_req reset_type)
ice_update_pf_netdev_link(pf);
/* tell the firmware we are up */
- ret = ice_send_version(pf);
- if (ret) {
- dev_err(dev, "Rebuild failed due to error sending driver version: %s\n",
- ice_stat_str(ret));
+ err = ice_send_version(pf);
+ if (err) {
+ dev_err(dev, "Rebuild failed due to error sending driver version: %d\n",
+ err);
goto err_vsi_rebuild;
}
@@ -6950,78 +6950,6 @@ const char *ice_aq_str(enum ice_aq_err aq_err)
}
/**
- * ice_stat_str - convert status err code to a string
- * @stat_err: the status error code to convert
- */
-const char *ice_stat_str(enum ice_status stat_err)
-{
- switch (stat_err) {
- case ICE_SUCCESS:
- return "OK";
- case ICE_ERR_PARAM:
- return "ICE_ERR_PARAM";
- case ICE_ERR_NOT_IMPL:
- return "ICE_ERR_NOT_IMPL";
- case ICE_ERR_NOT_READY:
- return "ICE_ERR_NOT_READY";
- case ICE_ERR_NOT_SUPPORTED:
- return "ICE_ERR_NOT_SUPPORTED";
- case ICE_ERR_BAD_PTR:
- return "ICE_ERR_BAD_PTR";
- case ICE_ERR_INVAL_SIZE:
- return "ICE_ERR_INVAL_SIZE";
- case ICE_ERR_DEVICE_NOT_SUPPORTED:
- return "ICE_ERR_DEVICE_NOT_SUPPORTED";
- case ICE_ERR_RESET_FAILED:
- return "ICE_ERR_RESET_FAILED";
- case ICE_ERR_FW_API_VER:
- return "ICE_ERR_FW_API_VER";
- case ICE_ERR_NO_MEMORY:
- return "ICE_ERR_NO_MEMORY";
- case ICE_ERR_CFG:
- return "ICE_ERR_CFG";
- case ICE_ERR_OUT_OF_RANGE:
- return "ICE_ERR_OUT_OF_RANGE";
- case ICE_ERR_ALREADY_EXISTS:
- return "ICE_ERR_ALREADY_EXISTS";
- case ICE_ERR_NVM:
- return "ICE_ERR_NVM";
- case ICE_ERR_NVM_CHECKSUM:
- return "ICE_ERR_NVM_CHECKSUM";
- case ICE_ERR_BUF_TOO_SHORT:
- return "ICE_ERR_BUF_TOO_SHORT";
- case ICE_ERR_NVM_BLANK_MODE:
- return "ICE_ERR_NVM_BLANK_MODE";
- case ICE_ERR_IN_USE:
- return "ICE_ERR_IN_USE";
- case ICE_ERR_MAX_LIMIT:
- return "ICE_ERR_MAX_LIMIT";
- case ICE_ERR_RESET_ONGOING:
- return "ICE_ERR_RESET_ONGOING";
- case ICE_ERR_HW_TABLE:
- return "ICE_ERR_HW_TABLE";
- case ICE_ERR_DOES_NOT_EXIST:
- return "ICE_ERR_DOES_NOT_EXIST";
- case ICE_ERR_FW_DDP_MISMATCH:
- return "ICE_ERR_FW_DDP_MISMATCH";
- case ICE_ERR_AQ_ERROR:
- return "ICE_ERR_AQ_ERROR";
- case ICE_ERR_AQ_TIMEOUT:
- return "ICE_ERR_AQ_TIMEOUT";
- case ICE_ERR_AQ_FULL:
- return "ICE_ERR_AQ_FULL";
- case ICE_ERR_AQ_NO_WORK:
- return "ICE_ERR_AQ_NO_WORK";
- case ICE_ERR_AQ_EMPTY:
- return "ICE_ERR_AQ_EMPTY";
- case ICE_ERR_AQ_FW_CRITICAL:
- return "ICE_ERR_AQ_FW_CRITICAL";
- }
-
- return "ICE_ERR_UNKNOWN";
-}
-
-/**
* ice_set_rss_lut - Set RSS LUT
* @vsi: Pointer to VSI structure
* @lut: Lookup table
@@ -7033,7 +6961,7 @@ int ice_set_rss_lut(struct ice_vsi *vsi, u8 *lut, u16 lut_size)
{
struct ice_aq_get_set_rss_lut_params params = {};
struct ice_hw *hw = &vsi->back->hw;
- enum ice_status status;
+ int status;
if (!lut)
return -EINVAL;
@@ -7044,14 +6972,11 @@ int ice_set_rss_lut(struct ice_vsi *vsi, u8 *lut, u16 lut_size)
params.lut = lut;
status = ice_aq_set_rss_lut(hw, ¶ms);
- if (status) {
- dev_err(ice_pf_to_dev(vsi->back), "Cannot set RSS lut, err %s aq_err %s\n",
- ice_stat_str(status),
- ice_aq_str(hw->adminq.sq_last_status));
- return -EIO;
- }
+ if (status)
+ dev_err(ice_pf_to_dev(vsi->back), "Cannot set RSS lut, err %d aq_err %s\n",
+ status, ice_aq_str(hw->adminq.sq_last_status));
- return 0;
+ return status;
}
/**
@@ -7064,20 +6989,17 @@ int ice_set_rss_lut(struct ice_vsi *vsi, u8 *lut, u16 lut_size)
int ice_set_rss_key(struct ice_vsi *vsi, u8 *seed)
{
struct ice_hw *hw = &vsi->back->hw;
- enum ice_status status;
+ int status;
if (!seed)
return -EINVAL;
status = ice_aq_set_rss_key(hw, vsi->idx, (struct ice_aqc_get_set_rss_keys *)seed);
- if (status) {
- dev_err(ice_pf_to_dev(vsi->back), "Cannot set RSS key, err %s aq_err %s\n",
- ice_stat_str(status),
- ice_aq_str(hw->adminq.sq_last_status));
- return -EIO;
- }
+ if (status)
+ dev_err(ice_pf_to_dev(vsi->back), "Cannot set RSS key, err %d aq_err %s\n",
+ status, ice_aq_str(hw->adminq.sq_last_status));
- return 0;
+ return status;
}
/**
@@ -7092,7 +7014,7 @@ int ice_get_rss_lut(struct ice_vsi *vsi, u8 *lut, u16 lut_size)
{
struct ice_aq_get_set_rss_lut_params params = {};
struct ice_hw *hw = &vsi->back->hw;
- enum ice_status status;
+ int status;
if (!lut)
return -EINVAL;
@@ -7103,14 +7025,11 @@ int ice_get_rss_lut(struct ice_vsi *vsi, u8 *lut, u16 lut_size)
params.lut = lut;
status = ice_aq_get_rss_lut(hw, ¶ms);
- if (status) {
- dev_err(ice_pf_to_dev(vsi->back), "Cannot get RSS lut, err %s aq_err %s\n",
- ice_stat_str(status),
- ice_aq_str(hw->adminq.sq_last_status));
- return -EIO;
- }
+ if (status)
+ dev_err(ice_pf_to_dev(vsi->back), "Cannot get RSS lut, err %d aq_err %s\n",
+ status, ice_aq_str(hw->adminq.sq_last_status));
- return 0;
+ return status;
}
/**
@@ -7123,20 +7042,17 @@ int ice_get_rss_lut(struct ice_vsi *vsi, u8 *lut, u16 lut_size)
int ice_get_rss_key(struct ice_vsi *vsi, u8 *seed)
{
struct ice_hw *hw = &vsi->back->hw;
- enum ice_status status;
+ int status;
if (!seed)
return -EINVAL;
status = ice_aq_get_rss_key(hw, vsi->idx, (struct ice_aqc_get_set_rss_keys *)seed);
- if (status) {
- dev_err(ice_pf_to_dev(vsi->back), "Cannot get RSS key, err %s aq_err %s\n",
- ice_stat_str(status),
- ice_aq_str(hw->adminq.sq_last_status));
- return -EIO;
- }
+ if (status)
+ dev_err(ice_pf_to_dev(vsi->back), "Cannot get RSS key, err %d aq_err %s\n",
+ status, ice_aq_str(hw->adminq.sq_last_status));
- return 0;
+ return status;
}
/**
@@ -7177,8 +7093,7 @@ static int ice_vsi_update_bridge_mode(struct ice_vsi *vsi, u16 bmode)
struct ice_aqc_vsi_props *vsi_props;
struct ice_hw *hw = &vsi->back->hw;
struct ice_vsi_ctx *ctxt;
- enum ice_status status;
- int ret = 0;
+ int ret;
vsi_props = &vsi->info;
@@ -7196,12 +7111,10 @@ static int ice_vsi_update_bridge_mode(struct ice_vsi *vsi, u16 bmode)
ctxt->info.sw_flags &= ~ICE_AQ_VSI_SW_FLAG_ALLOW_LB;
ctxt->info.valid_sections = cpu_to_le16(ICE_AQ_VSI_PROP_SW_VALID);
- status = ice_update_vsi(hw, vsi->idx, ctxt, NULL);
- if (status) {
- dev_err(ice_pf_to_dev(vsi->back), "update VSI for bridge mode failed, bmode = %d err %s aq_err %s\n",
- bmode, ice_stat_str(status),
- ice_aq_str(hw->adminq.sq_last_status));
- ret = -EIO;
+ ret = ice_update_vsi(hw, vsi->idx, ctxt, NULL);
+ if (ret) {
+ dev_err(ice_pf_to_dev(vsi->back), "update VSI for bridge mode failed, bmode = %d err %d aq_err %s\n",
+ bmode, ret, ice_aq_str(hw->adminq.sq_last_status));
goto out;
}
/* Update sw flags for book keeping */
@@ -7233,7 +7146,6 @@ ice_bridge_setlink(struct net_device *dev, struct nlmsghdr *nlh,
struct ice_pf *pf = np->vsi->back;
struct nlattr *attr, *br_spec;
struct ice_hw *hw = &pf->hw;
- enum ice_status status;
struct ice_sw *pf_sw;
int rem, v, err = 0;
@@ -7267,14 +7179,14 @@ ice_bridge_setlink(struct net_device *dev, struct nlmsghdr *nlh,
/* Update the unicast switch filter rules for the corresponding
* switch of the netdev
*/
- status = ice_update_sw_rule_bridge_mode(hw);
- if (status) {
- netdev_err(dev, "switch rule update failed, mode = %d err %s aq_err %s\n",
- mode, ice_stat_str(status),
+ err = ice_update_sw_rule_bridge_mode(hw);
+ if (err) {
+ netdev_err(dev, "switch rule update failed, mode = %d err %d aq_err %s\n",
+ mode, err,
ice_aq_str(hw->adminq.sq_last_status));
/* revert hw->evb_veb */
hw->evb_veb = (pf_sw->bridge_mode == BRIDGE_MODE_VEB);
- return -EIO;
+ return err;
}
pf_sw->bridge_mode = mode;
@@ -7552,6 +7464,67 @@ ice_validate_mqprio_qopt(struct ice_vsi *vsi,
}
/**
+ * ice_add_vsi_to_fdir - add a VSI to the flow director group for PF
+ * @pf: ptr to PF device
+ * @vsi: ptr to VSI
+ */
+static int ice_add_vsi_to_fdir(struct ice_pf *pf, struct ice_vsi *vsi)
+{
+ struct device *dev = ice_pf_to_dev(pf);
+ bool added = false;
+ struct ice_hw *hw;
+ int flow;
+
+ if (!(vsi->num_gfltr || vsi->num_bfltr))
+ return -EINVAL;
+
+ hw = &pf->hw;
+ for (flow = 0; flow < ICE_FLTR_PTYPE_MAX; flow++) {
+ struct ice_fd_hw_prof *prof;
+ int tun, status;
+ u64 entry_h;
+
+ if (!(hw->fdir_prof && hw->fdir_prof[flow] &&
+ hw->fdir_prof[flow]->cnt))
+ continue;
+
+ for (tun = 0; tun < ICE_FD_HW_SEG_MAX; tun++) {
+ enum ice_flow_priority prio;
+ u64 prof_id;
+
+ /* add this VSI to FDir profile for this flow */
+ prio = ICE_FLOW_PRIO_NORMAL;
+ prof = hw->fdir_prof[flow];
+ prof_id = flow + tun * ICE_FLTR_PTYPE_MAX;
+ status = ice_flow_add_entry(hw, ICE_BLK_FD, prof_id,
+ prof->vsi_h[0], vsi->idx,
+ prio, prof->fdir_seg[tun],
+ &entry_h);
+ if (status) {
+ dev_err(dev, "channel VSI idx %d, not able to add to group %d\n",
+ vsi->idx, flow);
+ continue;
+ }
+
+ prof->entry_h[prof->cnt][tun] = entry_h;
+ }
+
+ /* store VSI for filter replay and delete */
+ prof->vsi_h[prof->cnt] = vsi->idx;
+ prof->cnt++;
+
+ added = true;
+ dev_dbg(dev, "VSI idx %d added to fdir group %d\n", vsi->idx,
+ flow);
+ }
+
+ if (!added)
+ dev_dbg(dev, "VSI idx %d not added to fdir groups\n", vsi->idx);
+
+ return 0;
+}
+
+/**
* ice_add_channel - add a channel by adding VSI
* @pf: ptr to PF device
* @sw_id: underlying HW switching element ID
@@ -7575,6 +7548,8 @@ static int ice_add_channel(struct ice_pf *pf, u16 sw_id, struct ice_channel *ch)
return -EINVAL;
}
+ ice_add_vsi_to_fdir(pf, vsi);
+
ch->sw_id = sw_id;
ch->vsi_num = vsi->vsi_num;
ch->info.mapping_flags = vsi->info.mapping_flags;
@@ -7875,6 +7850,15 @@ static void ice_remove_q_channels(struct ice_vsi *vsi, bool rem_fltr)
if (rem_fltr)
ice_rem_all_chnl_fltrs(pf);
+ /* remove ntuple filters since queue configuration is being changed */
+ if (vsi->netdev->features & NETIF_F_NTUPLE) {
+ struct ice_hw *hw = &pf->hw;
+
+ mutex_lock(&hw->fdir_fltr_lock);
+ ice_fdir_del_all_fltrs(vsi);
+ mutex_unlock(&hw->fdir_fltr_lock);
+ }
+
/* perform cleanup for channels if they exist */
list_for_each_entry_safe(ch, ch_tmp, &vsi->ch_list, list) {
struct ice_vsi *ch_vsi;
@@ -7905,6 +7889,9 @@ static void ice_remove_q_channels(struct ice_vsi *vsi, bool rem_fltr)
}
}
+ /* Release FD resources for the channel VSI */
+ ice_fdir_rem_adq_chnl(&pf->hw, ch->ch_vsi->idx);
+
/* clear the VSI from scheduler tree */
ice_rm_vsi_lan_cfg(ch->ch_vsi->port_info, ch->ch_vsi->idx);
@@ -8449,7 +8436,6 @@ int ice_open_internal(struct net_device *netdev)
struct ice_vsi *vsi = np->vsi;
struct ice_pf *pf = vsi->back;
struct ice_port_info *pi;
- enum ice_status status;
int err;
if (test_bit(ICE_NEEDS_RESTART, pf->state)) {
@@ -8460,11 +8446,10 @@ int ice_open_internal(struct net_device *netdev)
netif_carrier_off(netdev);
pi = vsi->port_info;
- status = ice_update_link_info(pi);
- if (status) {
- netdev_err(netdev, "Failed to get link info, error %s\n",
- ice_stat_str(status));
- return -EIO;
+ err = ice_update_link_info(pi);
+ if (err) {
+ netdev_err(netdev, "Failed to get link info, error %d\n", err);
+ return err;
}
ice_check_link_cfg_err(pf, pi->phy.link_info.link_cfg_err);
diff --git a/drivers/net/ethernet/intel/ice/ice_nvm.c b/drivers/net/ethernet/intel/ice/ice_nvm.c
index fee37a5..4eb0599 100644
--- a/drivers/net/ethernet/intel/ice/ice_nvm.c
+++ b/drivers/net/ethernet/intel/ice/ice_nvm.c
@@ -1,6 +1,8 @@
// SPDX-License-Identifier: GPL-2.0
/* Copyright (c) 2018, Intel Corporation. */
+#include <linux/vmalloc.h>
+
#include "ice_common.h"
/**
@@ -16,7 +18,7 @@
*
* Read the NVM using the admin queue commands (0x0701)
*/
-static enum ice_status
+static int
ice_aq_read_nvm(struct ice_hw *hw, u16 module_typeid, u32 offset, u16 length,
void *data, bool last_command, bool read_shadow_ram,
struct ice_sq_cd *cd)
@@ -27,7 +29,7 @@ ice_aq_read_nvm(struct ice_hw *hw, u16 module_typeid, u32 offset, u16 length,
cmd = &desc.params.nvm;
if (offset > ICE_AQC_NVM_MAX_OFFSET)
- return ICE_ERR_PARAM;
+ return -EINVAL;
ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_nvm_read);
@@ -60,21 +62,21 @@ ice_aq_read_nvm(struct ice_hw *hw, u16 module_typeid, u32 offset, u16 length,
* Returns a status code on failure. Note that the data pointer may be
* partially updated if some reads succeed before a failure.
*/
-enum ice_status
+int
ice_read_flat_nvm(struct ice_hw *hw, u32 offset, u32 *length, u8 *data,
bool read_shadow_ram)
{
- enum ice_status status;
u32 inlen = *length;
u32 bytes_read = 0;
bool last_cmd;
+ int status;
*length = 0;
/* Verify the length of the read if this is for the Shadow RAM */
if (read_shadow_ram && ((offset + inlen) > (hw->flash.sr_words * 2u))) {
ice_debug(hw, ICE_DBG_NVM, "NVM error: requested offset is beyond Shadow RAM limit\n");
- return ICE_ERR_PARAM;
+ return -EINVAL;
}
do {
@@ -119,7 +121,7 @@ ice_read_flat_nvm(struct ice_hw *hw, u32 offset, u32 *length, u8 *data,
*
* Update the NVM using the admin queue commands (0x0703)
*/
-enum ice_status
+int
ice_aq_update_nvm(struct ice_hw *hw, u16 module_typeid, u32 offset,
u16 length, void *data, bool last_command, u8 command_flags,
struct ice_sq_cd *cd)
@@ -131,7 +133,7 @@ ice_aq_update_nvm(struct ice_hw *hw, u16 module_typeid, u32 offset,
/* In offset the highest byte must be zeroed. */
if (offset & 0xFF000000)
- return ICE_ERR_PARAM;
+ return -EINVAL;
ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_nvm_write);
@@ -158,8 +160,7 @@ ice_aq_update_nvm(struct ice_hw *hw, u16 module_typeid, u32 offset,
*
* Erase the NVM sector using the admin queue commands (0x0702)
*/
-enum ice_status
-ice_aq_erase_nvm(struct ice_hw *hw, u16 module_typeid, struct ice_sq_cd *cd)
+int ice_aq_erase_nvm(struct ice_hw *hw, u16 module_typeid, struct ice_sq_cd *cd)
{
struct ice_aq_desc desc;
struct ice_aqc_nvm *cmd;
@@ -184,12 +185,11 @@ ice_aq_erase_nvm(struct ice_hw *hw, u16 module_typeid, struct ice_sq_cd *cd)
*
* Reads one 16 bit word from the Shadow RAM using ice_read_flat_nvm.
*/
-static enum ice_status
-ice_read_sr_word_aq(struct ice_hw *hw, u16 offset, u16 *data)
+static int ice_read_sr_word_aq(struct ice_hw *hw, u16 offset, u16 *data)
{
u32 bytes = sizeof(u16);
- enum ice_status status;
__le16 data_local;
+ int status;
/* Note that ice_read_flat_nvm takes into account the 4Kb AdminQ and
* Shadow RAM sector restrictions necessary when reading from the NVM.
@@ -210,8 +210,7 @@ ice_read_sr_word_aq(struct ice_hw *hw, u16 offset, u16 *data)
*
* This function will request NVM ownership.
*/
-enum ice_status
-ice_acquire_nvm(struct ice_hw *hw, enum ice_aq_res_access_type access)
+int ice_acquire_nvm(struct ice_hw *hw, enum ice_aq_res_access_type access)
{
if (hw->flash.blank_nvm_mode)
return 0;
@@ -318,18 +317,18 @@ static u32 ice_get_flash_bank_offset(struct ice_hw *hw, enum ice_bank_select ban
* hw->flash.banks data being setup by ice_determine_active_flash_banks()
* during initialization.
*/
-static enum ice_status
+static int
ice_read_flash_module(struct ice_hw *hw, enum ice_bank_select bank, u16 module,
u32 offset, u8 *data, u32 length)
{
- enum ice_status status;
+ int status;
u32 start;
start = ice_get_flash_bank_offset(hw, bank, module);
if (!start) {
ice_debug(hw, ICE_DBG_NVM, "Unable to calculate flash bank offset for module 0x%04x\n",
module);
- return ICE_ERR_PARAM;
+ return -EINVAL;
}
status = ice_acquire_nvm(hw, ICE_RES_READ);
@@ -353,11 +352,11 @@ ice_read_flash_module(struct ice_hw *hw, enum ice_bank_select bank, u16 module,
* Read the specified word from the active NVM module. This includes the CSS
* header at the start of the NVM module.
*/
-static enum ice_status
+static int
ice_read_nvm_module(struct ice_hw *hw, enum ice_bank_select bank, u32 offset, u16 *data)
{
- enum ice_status status;
__le16 data_local;
+ int status;
status = ice_read_flash_module(hw, bank, ICE_SR_1ST_NVM_BANK_PTR, offset * sizeof(u16),
(__force u8 *)&data_local, sizeof(u16));
@@ -377,7 +376,7 @@ ice_read_nvm_module(struct ice_hw *hw, enum ice_bank_select bank, u32 offset, u1
* Read the specified word from the copy of the Shadow RAM found in the
* specified NVM module.
*/
-static enum ice_status
+static int
ice_read_nvm_sr_copy(struct ice_hw *hw, enum ice_bank_select bank, u32 offset, u16 *data)
{
return ice_read_nvm_module(hw, bank, ICE_NVM_SR_COPY_WORD_OFFSET + offset, data);
@@ -392,11 +391,11 @@ ice_read_nvm_sr_copy(struct ice_hw *hw, enum ice_bank_select bank, u32 offset, u
*
* Read a word from the specified netlist bank.
*/
-static enum ice_status
+static int
ice_read_netlist_module(struct ice_hw *hw, enum ice_bank_select bank, u32 offset, u16 *data)
{
- enum ice_status status;
__le16 data_local;
+ int status;
status = ice_read_flash_module(hw, bank, ICE_SR_NETLIST_BANK_PTR, offset * sizeof(u16),
(__force u8 *)&data_local, sizeof(u16));
@@ -414,9 +413,9 @@ ice_read_netlist_module(struct ice_hw *hw, enum ice_bank_select bank, u32 offset
*
* Reads one 16 bit word from the Shadow RAM using the ice_read_sr_word_aq.
*/
-enum ice_status ice_read_sr_word(struct ice_hw *hw, u16 offset, u16 *data)
+int ice_read_sr_word(struct ice_hw *hw, u16 offset, u16 *data)
{
- enum ice_status status;
+ int status;
status = ice_acquire_nvm(hw, ICE_RES_READ);
if (!status) {
@@ -438,13 +437,13 @@ enum ice_status ice_read_sr_word(struct ice_hw *hw, u16 offset, u16 *data)
* Area (PFA) and returns the TLV pointer and length. The caller can
* use these to read the variable length TLV value.
*/
-enum ice_status
+int
ice_get_pfa_module_tlv(struct ice_hw *hw, u16 *module_tlv, u16 *module_tlv_len,
u16 module_type)
{
- enum ice_status status;
u16 pfa_len, pfa_ptr;
u16 next_tlv;
+ int status;
status = ice_read_sr_word(hw, ICE_SR_PFA_PTR, &pfa_ptr);
if (status) {
@@ -482,7 +481,7 @@ ice_get_pfa_module_tlv(struct ice_hw *hw, u16 *module_tlv, u16 *module_tlv_len,
*module_tlv_len = tlv_len;
return 0;
}
- return ICE_ERR_INVAL_SIZE;
+ return -EINVAL;
}
/* Check next TLV, i.e. current TLV pointer + length + 2 words
* (for current TLV's type and length)
@@ -490,7 +489,7 @@ ice_get_pfa_module_tlv(struct ice_hw *hw, u16 *module_tlv, u16 *module_tlv_len,
next_tlv = next_tlv + tlv_len + 2;
}
/* Module does not exist */
- return ICE_ERR_DOES_NOT_EXIST;
+ return -ENOENT;
}
/**
@@ -501,12 +500,11 @@ ice_get_pfa_module_tlv(struct ice_hw *hw, u16 *module_tlv, u16 *module_tlv_len,
*
* Reads the part number string from the NVM.
*/
-enum ice_status
-ice_read_pba_string(struct ice_hw *hw, u8 *pba_num, u32 pba_num_size)
+int ice_read_pba_string(struct ice_hw *hw, u8 *pba_num, u32 pba_num_size)
{
u16 pba_tlv, pba_tlv_len;
- enum ice_status status;
u16 pba_word, pba_size;
+ int status;
u16 i;
status = ice_get_pfa_module_tlv(hw, &pba_tlv, &pba_tlv_len,
@@ -525,7 +523,7 @@ ice_read_pba_string(struct ice_hw *hw, u8 *pba_num, u32 pba_num_size)
if (pba_tlv_len < pba_size) {
ice_debug(hw, ICE_DBG_INIT, "Invalid PBA Block TLV size.\n");
- return ICE_ERR_INVAL_SIZE;
+ return -EINVAL;
}
/* Subtract one to get PBA word count (PBA Size word is included in
@@ -534,7 +532,7 @@ ice_read_pba_string(struct ice_hw *hw, u8 *pba_num, u32 pba_num_size)
pba_size--;
if (pba_num_size < (((u32)pba_size * 2) + 1)) {
ice_debug(hw, ICE_DBG_INIT, "Buffer too small for PBA data.\n");
- return ICE_ERR_PARAM;
+ return -EINVAL;
}
for (i = 0; i < pba_size; i++) {
@@ -561,11 +559,11 @@ ice_read_pba_string(struct ice_hw *hw, u8 *pba_num, u32 pba_num_size)
* Read the NVM EETRACK ID and map version of the main NVM image bank, filling
* in the NVM info structure.
*/
-static enum ice_status
+static int
ice_get_nvm_ver_info(struct ice_hw *hw, enum ice_bank_select bank, struct ice_nvm_info *nvm)
{
u16 eetrack_lo, eetrack_hi, ver;
- enum ice_status status;
+ int status;
status = ice_read_nvm_sr_copy(hw, bank, ICE_SR_NVM_DEV_STARTER_VER, &ver);
if (status) {
@@ -601,7 +599,7 @@ ice_get_nvm_ver_info(struct ice_hw *hw, enum ice_bank_select bank, struct ice_nv
* inactive NVM bank. Used to access version data for a pending update that
* has not yet been activated.
*/
-enum ice_status ice_get_inactive_nvm_ver(struct ice_hw *hw, struct ice_nvm_info *nvm)
+int ice_get_inactive_nvm_ver(struct ice_hw *hw, struct ice_nvm_info *nvm)
{
return ice_get_nvm_ver_info(hw, ICE_INACTIVE_FLASH_BANK, nvm);
}
@@ -615,49 +613,73 @@ enum ice_status ice_get_inactive_nvm_ver(struct ice_hw *hw, struct ice_nvm_info
* Searches through the Option ROM flash contents to locate the CIVD data for
* the image.
*/
-static enum ice_status
+static int
ice_get_orom_civd_data(struct ice_hw *hw, enum ice_bank_select bank,
struct ice_orom_civd_info *civd)
{
- struct ice_orom_civd_info tmp;
- enum ice_status status;
+ u8 *orom_data;
+ int status;
u32 offset;
/* The CIVD section is located in the Option ROM aligned to 512 bytes.
* The first 4 bytes must contain the ASCII characters "$CIV".
* A simple modulo 256 sum of all of the bytes of the structure must
* equal 0.
+ *
+ * The exact location is unknown and varies between images but is
+ * usually somewhere in the middle of the bank. We need to scan the
+ * Option ROM bank to locate it.
+ *
+ * It's significantly faster to read the entire Option ROM up front
+ * using the maximum page size, than to read each possible location
+ * with a separate firmware command.
*/
+ orom_data = vzalloc(hw->flash.banks.orom_size);
+ if (!orom_data)
+ return -ENOMEM;
+
+ status = ice_read_flash_module(hw, bank, ICE_SR_1ST_OROM_BANK_PTR, 0,
+ orom_data, hw->flash.banks.orom_size);
+ if (status) {
+ ice_debug(hw, ICE_DBG_NVM, "Unable to read Option ROM data\n");
+ return status;
+ }
+
+ /* Scan the memory buffer to locate the CIVD data section */
for (offset = 0; (offset + 512) <= hw->flash.banks.orom_size; offset += 512) {
+ struct ice_orom_civd_info *tmp;
u8 sum = 0, i;
- status = ice_read_flash_module(hw, bank, ICE_SR_1ST_OROM_BANK_PTR,
- offset, (u8 *)&tmp, sizeof(tmp));
- if (status) {
- ice_debug(hw, ICE_DBG_NVM, "Unable to read Option ROM CIVD data\n");
- return status;
- }
+ tmp = (struct ice_orom_civd_info *)&orom_data[offset];
/* Skip forward until we find a matching signature */
- if (memcmp("$CIV", tmp.signature, sizeof(tmp.signature)) != 0)
+ if (memcmp("$CIV", tmp->signature, sizeof(tmp->signature)) != 0)
continue;
+ ice_debug(hw, ICE_DBG_NVM, "Found CIVD section at offset %u\n",
+ offset);
+
/* Verify that the simple checksum is zero */
- for (i = 0; i < sizeof(tmp); i++)
+ for (i = 0; i < sizeof(*tmp); i++)
/* cppcheck-suppress objectIndex */
- sum += ((u8 *)&tmp)[i];
+ sum += ((u8 *)tmp)[i];
if (sum) {
ice_debug(hw, ICE_DBG_NVM, "Found CIVD data with invalid checksum of %u\n",
sum);
- return ICE_ERR_NVM;
+ goto err_invalid_checksum;
}
- *civd = tmp;
+ *civd = *tmp;
+ vfree(orom_data);
return 0;
}
- return ICE_ERR_NVM;
+ ice_debug(hw, ICE_DBG_NVM, "Unable to locate CIVD data within the Option ROM\n");
+
+err_invalid_checksum:
+ vfree(orom_data);
+ return -EIO;
}
/**
@@ -669,12 +691,12 @@ ice_get_orom_civd_data(struct ice_hw *hw, enum ice_bank_select bank,
* Read Option ROM version and security revision from the Option ROM flash
* section.
*/
-static enum ice_status
+static int
ice_get_orom_ver_info(struct ice_hw *hw, enum ice_bank_select bank, struct ice_orom_info *orom)
{
struct ice_orom_civd_info civd;
- enum ice_status status;
u32 combo_ver;
+ int status;
status = ice_get_orom_civd_data(hw, bank, &civd);
if (status) {
@@ -700,7 +722,7 @@ ice_get_orom_ver_info(struct ice_hw *hw, enum ice_bank_select bank, struct ice_o
* section of flash. Used to access version data for a pending update that has
* not yet been activated.
*/
-enum ice_status ice_get_inactive_orom_ver(struct ice_hw *hw, struct ice_orom_info *orom)
+int ice_get_inactive_orom_ver(struct ice_hw *hw, struct ice_orom_info *orom)
{
return ice_get_orom_ver_info(hw, ICE_INACTIVE_FLASH_BANK, orom);
}
@@ -715,13 +737,13 @@ enum ice_status ice_get_inactive_orom_ver(struct ice_hw *hw, struct ice_orom_inf
* Topology section to find the Netlist ID block and extract the relevant
* information into the netlist version structure.
*/
-static enum ice_status
+static int
ice_get_netlist_info(struct ice_hw *hw, enum ice_bank_select bank,
struct ice_netlist_info *netlist)
{
u16 module_id, length, node_count, i;
- enum ice_status status;
u16 *id_blk;
+ int status;
status = ice_read_netlist_module(hw, bank, ICE_NETLIST_TYPE_OFFSET, &module_id);
if (status)
@@ -730,7 +752,7 @@ ice_get_netlist_info(struct ice_hw *hw, enum ice_bank_select bank,
if (module_id != ICE_NETLIST_LINK_TOPO_MOD_ID) {
ice_debug(hw, ICE_DBG_NVM, "Expected netlist module_id ID of 0x%04x, but got 0x%04x\n",
ICE_NETLIST_LINK_TOPO_MOD_ID, module_id);
- return ICE_ERR_NVM;
+ return -EIO;
}
status = ice_read_netlist_module(hw, bank, ICE_LINK_TOPO_MODULE_LEN, &length);
@@ -741,7 +763,7 @@ ice_get_netlist_info(struct ice_hw *hw, enum ice_bank_select bank,
if (length < ICE_NETLIST_ID_BLK_SIZE) {
ice_debug(hw, ICE_DBG_NVM, "Netlist Link Topology module too small. Expected at least %u words, but got %u words.\n",
ICE_NETLIST_ID_BLK_SIZE, length);
- return ICE_ERR_NVM;
+ return -EIO;
}
status = ice_read_netlist_module(hw, bank, ICE_LINK_TOPO_NODE_COUNT, &node_count);
@@ -751,7 +773,7 @@ ice_get_netlist_info(struct ice_hw *hw, enum ice_bank_select bank,
id_blk = kcalloc(ICE_NETLIST_ID_BLK_SIZE, sizeof(*id_blk), GFP_KERNEL);
if (!id_blk)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
/* Read out the entire Netlist ID Block at once. */
status = ice_read_flash_module(hw, bank, ICE_SR_NETLIST_BANK_PTR,
@@ -791,7 +813,7 @@ ice_get_netlist_info(struct ice_hw *hw, enum ice_bank_select bank,
* extract version data of a pending flash update in order to display the
* version data.
*/
-enum ice_status ice_get_inactive_netlist_ver(struct ice_hw *hw, struct ice_netlist_info *netlist)
+int ice_get_inactive_netlist_ver(struct ice_hw *hw, struct ice_netlist_info *netlist)
{
return ice_get_netlist_info(hw, ICE_INACTIVE_FLASH_BANK, netlist);
}
@@ -804,10 +826,10 @@ enum ice_status ice_get_inactive_netlist_ver(struct ice_hw *hw, struct ice_netli
* the actual size is smaller. Use bisection to determine the accessible size
* of flash memory.
*/
-static enum ice_status ice_discover_flash_size(struct ice_hw *hw)
+static int ice_discover_flash_size(struct ice_hw *hw)
{
u32 min_size = 0, max_size = ICE_AQC_NVM_MAX_OFFSET + 1;
- enum ice_status status;
+ int status;
status = ice_acquire_nvm(hw, ICE_RES_READ);
if (status)
@@ -819,7 +841,7 @@ static enum ice_status ice_discover_flash_size(struct ice_hw *hw)
u8 data;
status = ice_read_flat_nvm(hw, offset, &len, &data, false);
- if (status == ICE_ERR_AQ_ERROR &&
+ if (status == -EIO &&
hw->adminq.sq_last_status == ICE_AQ_RC_EINVAL) {
ice_debug(hw, ICE_DBG_NVM, "%s: New upper bound of %u bytes\n",
__func__, offset);
@@ -859,10 +881,9 @@ static enum ice_status ice_discover_flash_size(struct ice_hw *hw)
* sector size by using the highest bit. The reported pointer value will be in
* bytes, intended for flat NVM reads.
*/
-static enum ice_status
-ice_read_sr_pointer(struct ice_hw *hw, u16 offset, u32 *pointer)
+static int ice_read_sr_pointer(struct ice_hw *hw, u16 offset, u32 *pointer)
{
- enum ice_status status;
+ int status;
u16 value;
status = ice_read_sr_word(hw, offset, &value);
@@ -891,10 +912,9 @@ ice_read_sr_pointer(struct ice_hw *hw, u16 offset, u32 *pointer)
* Each area size word is specified in 4KB sector units. This function reports
* the size in bytes, intended for flat NVM reads.
*/
-static enum ice_status
-ice_read_sr_area_size(struct ice_hw *hw, u16 offset, u32 *size)
+static int ice_read_sr_area_size(struct ice_hw *hw, u16 offset, u32 *size)
{
- enum ice_status status;
+ int status;
u16 value;
status = ice_read_sr_word(hw, offset, &value);
@@ -917,12 +937,11 @@ ice_read_sr_area_size(struct ice_hw *hw, u16 offset, u32 *size)
* structure for later use in order to calculate the correct offset to read
* from the active module.
*/
-static enum ice_status
-ice_determine_active_flash_banks(struct ice_hw *hw)
+static int ice_determine_active_flash_banks(struct ice_hw *hw)
{
struct ice_bank_info *banks = &hw->flash.banks;
- enum ice_status status;
u16 ctrl_word;
+ int status;
status = ice_read_sr_word(hw, ICE_SR_NVM_CTRL_WORD, &ctrl_word);
if (status) {
@@ -933,7 +952,7 @@ ice_determine_active_flash_banks(struct ice_hw *hw)
/* Check that the control word indicates validity */
if ((ctrl_word & ICE_SR_CTRL_WORD_1_M) >> ICE_SR_CTRL_WORD_1_S != ICE_SR_CTRL_WORD_VALID) {
ice_debug(hw, ICE_DBG_NVM, "Shadow RAM control word is invalid\n");
- return ICE_ERR_CFG;
+ return -EIO;
}
if (!(ctrl_word & ICE_SR_CTRL_WORD_NVM_BANK))
@@ -997,12 +1016,12 @@ ice_determine_active_flash_banks(struct ice_hw *hw)
* This function reads and populates NVM settings such as Shadow RAM size,
* max_timeout, and blank_nvm_mode
*/
-enum ice_status ice_init_nvm(struct ice_hw *hw)
+int ice_init_nvm(struct ice_hw *hw)
{
struct ice_flash_info *flash = &hw->flash;
- enum ice_status status;
u32 fla, gens_stat;
u8 sr_size;
+ int status;
/* The SR size is stored regardless of the NVM programming mode
* as the blank mode may be used in the factory line.
@@ -1021,7 +1040,7 @@ enum ice_status ice_init_nvm(struct ice_hw *hw)
/* Blank programming mode */
flash->blank_nvm_mode = true;
ice_debug(hw, ICE_DBG_NVM, "NVM init error: unsupported blank mode.\n");
- return ICE_ERR_NVM_BLANK_MODE;
+ return -EIO;
}
status = ice_discover_flash_size(hw);
@@ -1060,11 +1079,11 @@ enum ice_status ice_init_nvm(struct ice_hw *hw)
*
* Verify NVM PFA checksum validity (0x0706)
*/
-enum ice_status ice_nvm_validate_checksum(struct ice_hw *hw)
+int ice_nvm_validate_checksum(struct ice_hw *hw)
{
struct ice_aqc_nvm_checksum *cmd;
struct ice_aq_desc desc;
- enum ice_status status;
+ int status;
status = ice_acquire_nvm(hw, ICE_RES_READ);
if (status)
@@ -1080,7 +1099,7 @@ enum ice_status ice_nvm_validate_checksum(struct ice_hw *hw)
if (!status)
if (le16_to_cpu(cmd->checksum) != ICE_AQC_NVM_CHECKSUM_CORRECT)
- status = ICE_ERR_NVM_CHECKSUM;
+ status = -EIO;
return status;
}
@@ -1088,22 +1107,35 @@ enum ice_status ice_nvm_validate_checksum(struct ice_hw *hw)
/**
* ice_nvm_write_activate
* @hw: pointer to the HW struct
- * @cmd_flags: NVM activate admin command bits (banks to be validated)
+ * @cmd_flags: flags for write activate command
+ * @response_flags: response indicators from firmware
*
* Update the control word with the required banks' validity bits
* and dumps the Shadow RAM to flash (0x0707)
+ *
+ * cmd_flags controls which banks to activate, and the preservation level to
+ * use when activating the NVM bank.
+ *
+ * On successful return of the firmware command, the response_flags variable
+ * is updated with the flags reported by firmware indicating certain status,
+ * such as whether EMP reset is enabled.
*/
-enum ice_status ice_nvm_write_activate(struct ice_hw *hw, u8 cmd_flags)
+int ice_nvm_write_activate(struct ice_hw *hw, u8 cmd_flags, u8 *response_flags)
{
struct ice_aqc_nvm *cmd;
struct ice_aq_desc desc;
+ int err;
cmd = &desc.params.nvm;
ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_nvm_write_activate);
cmd->cmd_flags = cmd_flags;
- return ice_aq_send_cmd(hw, &desc, NULL, 0, NULL);
+ err = ice_aq_send_cmd(hw, &desc, NULL, 0, NULL);
+ if (!err && response_flags)
+ *response_flags = cmd->cmd_flags;
+
+ return err;
}
/**
@@ -1113,7 +1145,7 @@ enum ice_status ice_nvm_write_activate(struct ice_hw *hw, u8 cmd_flags)
* Update empr (0x0709). This command allows SW to
* request an EMPR to activate new FW.
*/
-enum ice_status ice_aq_nvm_update_empr(struct ice_hw *hw)
+int ice_aq_nvm_update_empr(struct ice_hw *hw)
{
struct ice_aq_desc desc;
@@ -1136,7 +1168,7 @@ enum ice_status ice_aq_nvm_update_empr(struct ice_hw *hw)
* as part of the NVM update as the first cmd in the flow.
*/
-enum ice_status
+int
ice_nvm_set_pkg_data(struct ice_hw *hw, bool del_pkg_data_flag, u8 *data,
u16 length, struct ice_sq_cd *cd)
{
@@ -1144,7 +1176,7 @@ ice_nvm_set_pkg_data(struct ice_hw *hw, bool del_pkg_data_flag, u8 *data,
struct ice_aq_desc desc;
if (length != 0 && !data)
- return ICE_ERR_PARAM;
+ return -EINVAL;
cmd = &desc.params.pkg_data;
@@ -1173,17 +1205,17 @@ ice_nvm_set_pkg_data(struct ice_hw *hw, bool del_pkg_data_flag, u8 *data,
* the TransferFlag is set to End or StartAndEnd.
*/
-enum ice_status
+int
ice_nvm_pass_component_tbl(struct ice_hw *hw, u8 *data, u16 length,
u8 transfer_flag, u8 *comp_response,
u8 *comp_response_code, struct ice_sq_cd *cd)
{
struct ice_aqc_nvm_pass_comp_tbl *cmd;
struct ice_aq_desc desc;
- enum ice_status status;
+ int status;
if (!data || !comp_response || !comp_response_code)
- return ICE_ERR_PARAM;
+ return -EINVAL;
cmd = &desc.params.pass_comp_tbl;
diff --git a/drivers/net/ethernet/intel/ice/ice_nvm.h b/drivers/net/ethernet/intel/ice/ice_nvm.h
index c6f05f4..856d1ad 100644
--- a/drivers/net/ethernet/intel/ice/ice_nvm.h
+++ b/drivers/net/ethernet/intel/ice/ice_nvm.h
@@ -12,38 +12,34 @@ struct ice_orom_civd_info {
__le16 combo_name[32]; /* Unicode string representing the Combo Image version */
} __packed;
-enum ice_status
-ice_acquire_nvm(struct ice_hw *hw, enum ice_aq_res_access_type access);
+int ice_acquire_nvm(struct ice_hw *hw, enum ice_aq_res_access_type access);
void ice_release_nvm(struct ice_hw *hw);
-enum ice_status
+int
ice_read_flat_nvm(struct ice_hw *hw, u32 offset, u32 *length, u8 *data,
bool read_shadow_ram);
-enum ice_status
+int
ice_get_pfa_module_tlv(struct ice_hw *hw, u16 *module_tlv, u16 *module_tlv_len,
u16 module_type);
-enum ice_status
-ice_get_inactive_orom_ver(struct ice_hw *hw, struct ice_orom_info *orom);
-enum ice_status
-ice_get_inactive_nvm_ver(struct ice_hw *hw, struct ice_nvm_info *nvm);
-enum ice_status
+int ice_get_inactive_orom_ver(struct ice_hw *hw, struct ice_orom_info *orom);
+int ice_get_inactive_nvm_ver(struct ice_hw *hw, struct ice_nvm_info *nvm);
+int
ice_get_inactive_netlist_ver(struct ice_hw *hw, struct ice_netlist_info *netlist);
-enum ice_status
-ice_read_pba_string(struct ice_hw *hw, u8 *pba_num, u32 pba_num_size);
-enum ice_status ice_init_nvm(struct ice_hw *hw);
-enum ice_status ice_read_sr_word(struct ice_hw *hw, u16 offset, u16 *data);
-enum ice_status
+int ice_read_pba_string(struct ice_hw *hw, u8 *pba_num, u32 pba_num_size);
+int ice_init_nvm(struct ice_hw *hw);
+int ice_read_sr_word(struct ice_hw *hw, u16 offset, u16 *data);
+int
ice_aq_update_nvm(struct ice_hw *hw, u16 module_typeid, u32 offset,
u16 length, void *data, bool last_command, u8 command_flags,
struct ice_sq_cd *cd);
-enum ice_status
+int
ice_aq_erase_nvm(struct ice_hw *hw, u16 module_typeid, struct ice_sq_cd *cd);
-enum ice_status ice_nvm_validate_checksum(struct ice_hw *hw);
-enum ice_status ice_nvm_write_activate(struct ice_hw *hw, u8 cmd_flags);
-enum ice_status ice_aq_nvm_update_empr(struct ice_hw *hw);
-enum ice_status
+int ice_nvm_validate_checksum(struct ice_hw *hw);
+int ice_nvm_write_activate(struct ice_hw *hw, u8 cmd_flags, u8 *response_flags);
+int ice_aq_nvm_update_empr(struct ice_hw *hw);
+int
ice_nvm_set_pkg_data(struct ice_hw *hw, bool del_pkg_data_flag, u8 *data,
u16 length, struct ice_sq_cd *cd);
-enum ice_status
+int
ice_nvm_pass_component_tbl(struct ice_hw *hw, u8 *data, u16 length,
u8 transfer_flag, u8 *comp_response,
u8 *comp_response_code, struct ice_sq_cd *cd);
diff --git a/drivers/net/ethernet/intel/ice/ice_ptp.c b/drivers/net/ethernet/intel/ice/ice_ptp.c
index 442b031..ae291d4 100644
--- a/drivers/net/ethernet/intel/ice/ice_ptp.c
+++ b/drivers/net/ethernet/intel/ice/ice_ptp.c
@@ -6,6 +6,8 @@
#define E810_OUT_PROP_DELAY_NS 1
+#define UNKNOWN_INCVAL_E822 0x100000000ULL
+
static const struct ptp_pin_desc ice_pin_desc_e810t[] = {
/* name idx func chan */
{ "GNSS", GNSS, PTP_PF_EXTTS, 0, { 0, } },
@@ -281,6 +283,8 @@ static void ice_set_tx_tstamp(struct ice_pf *pf, bool on)
else
val &= ~PFINT_OICR_TSYN_TX_M;
wr32(&pf->hw, PFINT_OICR_ENA, val);
+
+ pf->ptp.tstamp_config.tx_type = on ? HWTSTAMP_TX_ON : HWTSTAMP_TX_OFF;
}
/**
@@ -303,6 +307,9 @@ static void ice_set_rx_tstamp(struct ice_pf *pf, bool on)
continue;
vsi->rx_rings[i]->ptp_rx = on;
}
+
+ pf->ptp.tstamp_config.rx_filter = on ? HWTSTAMP_FILTER_ALL :
+ HWTSTAMP_FILTER_NONE;
}
/**
@@ -313,18 +320,10 @@ static void ice_set_rx_tstamp(struct ice_pf *pf, bool on)
* This function will configure timestamping during PTP initialization
* and deinitialization
*/
-static void ice_ptp_cfg_timestamp(struct ice_pf *pf, bool ena)
+void ice_ptp_cfg_timestamp(struct ice_pf *pf, bool ena)
{
ice_set_tx_tstamp(pf, ena);
ice_set_rx_tstamp(pf, ena);
-
- if (ena) {
- pf->ptp.tstamp_config.rx_filter = HWTSTAMP_FILTER_ALL;
- pf->ptp.tstamp_config.tx_type = HWTSTAMP_TX_ON;
- } else {
- pf->ptp.tstamp_config.rx_filter = HWTSTAMP_FILTER_NONE;
- pf->ptp.tstamp_config.tx_type = HWTSTAMP_TX_OFF;
- }
}
/**
@@ -682,6 +681,406 @@ static int ice_ptp_write_adj(struct ice_pf *pf, s32 adj)
}
/**
+ * ice_base_incval - Get base timer increment value
+ * @pf: Board private structure
+ *
+ * Look up the base timer increment value for this device. The base increment
+ * value is used to define the nominal clock tick rate. This increment value
+ * is programmed during device initialization. It is also used as the basis
+ * for calculating adjustments using scaled_ppm.
+ */
+static u64 ice_base_incval(struct ice_pf *pf)
+{
+ struct ice_hw *hw = &pf->hw;
+ u64 incval;
+
+ if (ice_is_e810(hw))
+ incval = ICE_PTP_NOMINAL_INCVAL_E810;
+ else if (ice_e822_time_ref(hw) < NUM_ICE_TIME_REF_FREQ)
+ incval = ice_e822_nominal_incval(ice_e822_time_ref(hw));
+ else
+ incval = UNKNOWN_INCVAL_E822;
+
+ dev_dbg(ice_pf_to_dev(pf), "PTP: using base increment value of 0x%016llx\n",
+ incval);
+
+ return incval;
+}
+
+/**
+ * ice_ptp_reset_ts_memory_quad - Reset timestamp memory for one quad
+ * @pf: The PF private data structure
+ * @quad: The quad (0-4)
+ */
+static void ice_ptp_reset_ts_memory_quad(struct ice_pf *pf, int quad)
+{
+ struct ice_hw *hw = &pf->hw;
+
+ ice_write_quad_reg_e822(hw, quad, Q_REG_TS_CTRL, Q_REG_TS_CTRL_M);
+ ice_write_quad_reg_e822(hw, quad, Q_REG_TS_CTRL, ~(u32)Q_REG_TS_CTRL_M);
+}
+
+/**
+ * ice_ptp_check_tx_fifo - Check whether Tx FIFO is in an OK state
+ * @port: PTP port for which Tx FIFO is checked
+ */
+static int ice_ptp_check_tx_fifo(struct ice_ptp_port *port)
+{
+ int quad = port->port_num / ICE_PORTS_PER_QUAD;
+ int offs = port->port_num % ICE_PORTS_PER_QUAD;
+ struct ice_pf *pf;
+ struct ice_hw *hw;
+ u32 val, phy_sts;
+ int err;
+
+ pf = ptp_port_to_pf(port);
+ hw = &pf->hw;
+
+ if (port->tx_fifo_busy_cnt == FIFO_OK)
+ return 0;
+
+ /* need to read FIFO state */
+ if (offs == 0 || offs == 1)
+ err = ice_read_quad_reg_e822(hw, quad, Q_REG_FIFO01_STATUS,
+ &val);
+ else
+ err = ice_read_quad_reg_e822(hw, quad, Q_REG_FIFO23_STATUS,
+ &val);
+
+ if (err) {
+ dev_err(ice_pf_to_dev(pf), "PTP failed to check port %d Tx FIFO, err %d\n",
+ port->port_num, err);
+ return err;
+ }
+
+ if (offs & 0x1)
+ phy_sts = (val & Q_REG_FIFO13_M) >> Q_REG_FIFO13_S;
+ else
+ phy_sts = (val & Q_REG_FIFO02_M) >> Q_REG_FIFO02_S;
+
+ if (phy_sts & FIFO_EMPTY) {
+ port->tx_fifo_busy_cnt = FIFO_OK;
+ return 0;
+ }
+
+ port->tx_fifo_busy_cnt++;
+
+ dev_dbg(ice_pf_to_dev(pf), "Try %d, port %d FIFO not empty\n",
+ port->tx_fifo_busy_cnt, port->port_num);
+
+ if (port->tx_fifo_busy_cnt == ICE_PTP_FIFO_NUM_CHECKS) {
+ dev_dbg(ice_pf_to_dev(pf),
+ "Port %d Tx FIFO still not empty; resetting quad %d\n",
+ port->port_num, quad);
+ ice_ptp_reset_ts_memory_quad(pf, quad);
+ port->tx_fifo_busy_cnt = FIFO_OK;
+ return 0;
+ }
+
+ return -EAGAIN;
+}
+
+/**
+ * ice_ptp_check_tx_offset_valid - Check if the Tx PHY offset is valid
+ * @port: the PTP port to check
+ *
+ * Checks whether the Tx offset for the PHY associated with this port is
+ * valid. Returns 0 if the offset is valid, and a non-zero error code if it is
+ * not.
+ */
+static int ice_ptp_check_tx_offset_valid(struct ice_ptp_port *port)
+{
+ struct ice_pf *pf = ptp_port_to_pf(port);
+ struct device *dev = ice_pf_to_dev(pf);
+ struct ice_hw *hw = &pf->hw;
+ u32 val;
+ int err;
+
+ err = ice_ptp_check_tx_fifo(port);
+ if (err)
+ return err;
+
+ err = ice_read_phy_reg_e822(hw, port->port_num, P_REG_TX_OV_STATUS,
+ &val);
+ if (err) {
+ dev_err(dev, "Failed to read TX_OV_STATUS for port %d, err %d\n",
+ port->port_num, err);
+ return -EAGAIN;
+ }
+
+ if (!(val & P_REG_TX_OV_STATUS_OV_M))
+ return -EAGAIN;
+
+ return 0;
+}
+
+/**
+ * ice_ptp_check_rx_offset_valid - Check if the Rx PHY offset is valid
+ * @port: the PTP port to check
+ *
+ * Checks whether the Rx offset for the PHY associated with this port is
+ * valid. Returns 0 if the offset is valid, and a non-zero error code if it is
+ * not.
+ */
+static int ice_ptp_check_rx_offset_valid(struct ice_ptp_port *port)
+{
+ struct ice_pf *pf = ptp_port_to_pf(port);
+ struct device *dev = ice_pf_to_dev(pf);
+ struct ice_hw *hw = &pf->hw;
+ int err;
+ u32 val;
+
+ err = ice_read_phy_reg_e822(hw, port->port_num, P_REG_RX_OV_STATUS,
+ &val);
+ if (err) {
+ dev_err(dev, "Failed to read RX_OV_STATUS for port %d, err %d\n",
+ port->port_num, err);
+ return err;
+ }
+
+ if (!(val & P_REG_RX_OV_STATUS_OV_M))
+ return -EAGAIN;
+
+ return 0;
+}
+
+/**
+ * ice_ptp_check_offset_valid - Check port offset valid bit
+ * @port: Port for which offset valid bit is checked
+ *
+ * Returns 0 if both Tx and Rx offset are valid, and -EAGAIN if one of the
+ * offset is not ready.
+ */
+static int ice_ptp_check_offset_valid(struct ice_ptp_port *port)
+{
+ int tx_err, rx_err;
+
+ /* always check both Tx and Rx offset validity */
+ tx_err = ice_ptp_check_tx_offset_valid(port);
+ rx_err = ice_ptp_check_rx_offset_valid(port);
+
+ if (tx_err || rx_err)
+ return -EAGAIN;
+
+ return 0;
+}
+
+/**
+ * ice_ptp_wait_for_offset_valid - Check for valid Tx and Rx offsets
+ * @work: Pointer to the kthread_work structure for this task
+ *
+ * Check whether both the Tx and Rx offsets are valid for enabling the vernier
+ * calibration.
+ *
+ * Once we have valid offsets from hardware, update the total Tx and Rx
+ * offsets, and exit bypass mode. This enables more precise timestamps using
+ * the extra data measured during the vernier calibration process.
+ */
+static void ice_ptp_wait_for_offset_valid(struct kthread_work *work)
+{
+ struct ice_ptp_port *port;
+ int err;
+ struct device *dev;
+ struct ice_pf *pf;
+ struct ice_hw *hw;
+
+ port = container_of(work, struct ice_ptp_port, ov_work.work);
+ pf = ptp_port_to_pf(port);
+ hw = &pf->hw;
+ dev = ice_pf_to_dev(pf);
+
+ if (ice_ptp_check_offset_valid(port)) {
+ /* Offsets not ready yet, try again later */
+ kthread_queue_delayed_work(pf->ptp.kworker,
+ &port->ov_work,
+ msecs_to_jiffies(100));
+ return;
+ }
+
+ /* Offsets are valid, so it is safe to exit bypass mode */
+ err = ice_phy_exit_bypass_e822(hw, port->port_num);
+ if (err) {
+ dev_warn(dev, "Failed to exit bypass mode for PHY port %u, err %d\n",
+ port->port_num, err);
+ return;
+ }
+}
+
+/**
+ * ice_ptp_port_phy_stop - Stop timestamping for a PHY port
+ * @ptp_port: PTP port to stop
+ */
+static int
+ice_ptp_port_phy_stop(struct ice_ptp_port *ptp_port)
+{
+ struct ice_pf *pf = ptp_port_to_pf(ptp_port);
+ u8 port = ptp_port->port_num;
+ struct ice_hw *hw = &pf->hw;
+ int err;
+
+ if (ice_is_e810(hw))
+ return 0;
+
+ mutex_lock(&ptp_port->ps_lock);
+
+ kthread_cancel_delayed_work_sync(&ptp_port->ov_work);
+
+ err = ice_stop_phy_timer_e822(hw, port, true);
+ if (err)
+ dev_err(ice_pf_to_dev(pf), "PTP failed to set PHY port %d down, err %d\n",
+ port, err);
+
+ mutex_unlock(&ptp_port->ps_lock);
+
+ return err;
+}
+
+/**
+ * ice_ptp_port_phy_restart - (Re)start and calibrate PHY timestamping
+ * @ptp_port: PTP port for which the PHY start is set
+ *
+ * Start the PHY timestamping block, and initiate Vernier timestamping
+ * calibration. If timestamping cannot be calibrated (such as if link is down)
+ * then disable the timestamping block instead.
+ */
+static int
+ice_ptp_port_phy_restart(struct ice_ptp_port *ptp_port)
+{
+ struct ice_pf *pf = ptp_port_to_pf(ptp_port);
+ u8 port = ptp_port->port_num;
+ struct ice_hw *hw = &pf->hw;
+ int err;
+
+ if (ice_is_e810(hw))
+ return 0;
+
+ if (!ptp_port->link_up)
+ return ice_ptp_port_phy_stop(ptp_port);
+
+ mutex_lock(&ptp_port->ps_lock);
+
+ kthread_cancel_delayed_work_sync(&ptp_port->ov_work);
+
+ /* temporarily disable Tx timestamps while calibrating PHY offset */
+ ptp_port->tx.calibrating = true;
+ ptp_port->tx_fifo_busy_cnt = 0;
+
+ /* Start the PHY timer in bypass mode */
+ err = ice_start_phy_timer_e822(hw, port, true);
+ if (err)
+ goto out_unlock;
+
+ /* Enable Tx timestamps right away */
+ ptp_port->tx.calibrating = false;
+
+ kthread_queue_delayed_work(pf->ptp.kworker, &ptp_port->ov_work, 0);
+
+out_unlock:
+ if (err)
+ dev_err(ice_pf_to_dev(pf), "PTP failed to set PHY port %d up, err %d\n",
+ port, err);
+
+ mutex_unlock(&ptp_port->ps_lock);
+
+ return err;
+}
+
+/**
+ * ice_ptp_link_change - Set or clear port registers for timestamping
+ * @pf: Board private structure
+ * @port: Port for which the PHY start is set
+ * @linkup: Link is up or down
+ */
+int ice_ptp_link_change(struct ice_pf *pf, u8 port, bool linkup)
+{
+ struct ice_ptp_port *ptp_port;
+
+ if (!test_bit(ICE_FLAG_PTP_SUPPORTED, pf->flags))
+ return 0;
+
+ if (port >= ICE_NUM_EXTERNAL_PORTS)
+ return -EINVAL;
+
+ ptp_port = &pf->ptp.port;
+ if (ptp_port->port_num != port)
+ return -EINVAL;
+
+ /* Update cached link err for this port immediately */
+ ptp_port->link_up = linkup;
+
+ if (!test_bit(ICE_FLAG_PTP, pf->flags))
+ /* PTP is not setup */
+ return -EAGAIN;
+
+ return ice_ptp_port_phy_restart(ptp_port);
+}
+
+/**
+ * ice_ptp_reset_ts_memory - Reset timestamp memory for all quads
+ * @pf: The PF private data structure
+ */
+static void ice_ptp_reset_ts_memory(struct ice_pf *pf)
+{
+ int quad;
+
+ quad = pf->hw.port_info->lport / ICE_PORTS_PER_QUAD;
+ ice_ptp_reset_ts_memory_quad(pf, quad);
+}
+
+/**
+ * ice_ptp_tx_ena_intr - Enable or disable the Tx timestamp interrupt
+ * @pf: PF private structure
+ * @ena: bool value to enable or disable interrupt
+ * @threshold: Minimum number of packets at which intr is triggered
+ *
+ * Utility function to enable or disable Tx timestamp interrupt and threshold
+ */
+static int ice_ptp_tx_ena_intr(struct ice_pf *pf, bool ena, u32 threshold)
+{
+ struct ice_hw *hw = &pf->hw;
+ int err = 0;
+ int quad;
+ u32 val;
+
+ ice_ptp_reset_ts_memory(pf);
+
+ for (quad = 0; quad < ICE_MAX_QUAD; quad++) {
+ err = ice_read_quad_reg_e822(hw, quad, Q_REG_TX_MEM_GBL_CFG,
+ &val);
+ if (err)
+ break;
+
+ if (ena) {
+ val |= Q_REG_TX_MEM_GBL_CFG_INTR_ENA_M;
+ val &= ~Q_REG_TX_MEM_GBL_CFG_INTR_THR_M;
+ val |= ((threshold << Q_REG_TX_MEM_GBL_CFG_INTR_THR_S) &
+ Q_REG_TX_MEM_GBL_CFG_INTR_THR_M);
+ } else {
+ val &= ~Q_REG_TX_MEM_GBL_CFG_INTR_ENA_M;
+ }
+
+ err = ice_write_quad_reg_e822(hw, quad, Q_REG_TX_MEM_GBL_CFG,
+ val);
+ if (err)
+ break;
+ }
+
+ if (err)
+ dev_err(ice_pf_to_dev(pf), "PTP failed in intr ena, err %d\n",
+ err);
+ return err;
+}
+
+/**
+ * ice_ptp_reset_phy_timestamping - Reset PHY timestamping block
+ * @pf: Board private structure
+ */
+static void ice_ptp_reset_phy_timestamping(struct ice_pf *pf)
+{
+ ice_ptp_port_phy_restart(&pf->ptp.port);
+}
+
+/**
* ice_ptp_adjfine - Adjust clock increment rate
* @info: the driver's PTP info structure
* @scaled_ppm: Parts per million with 16-bit fractional field
@@ -698,7 +1097,7 @@ static int ice_ptp_adjfine(struct ptp_clock_info *info, long scaled_ppm)
int neg_adj = 0;
int err;
- incval = ICE_PTP_NOMINAL_INCVAL_E810;
+ incval = ice_base_incval(pf);
if (scaled_ppm < 0) {
neg_adj = 1;
@@ -905,7 +1304,10 @@ static int ice_ptp_cfg_clkout(struct ice_pf *pf, unsigned int chan,
start_time = div64_u64(current_time + NSEC_PER_SEC - 1,
NSEC_PER_SEC) * NSEC_PER_SEC + phase;
- start_time -= E810_OUT_PROP_DELAY_NS;
+ if (ice_is_e810(hw))
+ start_time -= E810_OUT_PROP_DELAY_NS;
+ else
+ start_time -= ice_e822_pps_delay(ice_e822_time_ref(hw));
/* 2. Write TARGET time */
wr32(hw, GLTSYN_TGT_L(chan, tmr_idx), lower_32_bits(start_time));
@@ -1088,6 +1490,12 @@ ice_ptp_settime64(struct ptp_clock_info *info, const struct timespec64 *ts)
struct ice_hw *hw = &pf->hw;
int err;
+ /* For Vernier mode, we need to recalibrate after new settime
+ * Start with disabling timestamp block
+ */
+ if (pf->ptp.port.link_up)
+ ice_ptp_port_phy_stop(&pf->ptp.port);
+
if (!ice_ptp_lock(hw)) {
err = -EBUSY;
goto exit;
@@ -1104,6 +1512,10 @@ ice_ptp_settime64(struct ptp_clock_info *info, const struct timespec64 *ts)
/* Reenable periodic outputs */
ice_ptp_enable_all_clkout(pf);
+
+ /* Recalibrate and re-enable timestamp block */
+ if (pf->ptp.port.link_up)
+ ice_ptp_port_phy_restart(&pf->ptp.port);
exit:
if (err) {
dev_err(ice_pf_to_dev(pf), "PTP failed to set time %d\n", err);
@@ -1177,6 +1589,101 @@ static int ice_ptp_adjtime(struct ptp_clock_info *info, s64 delta)
return 0;
}
+#ifdef CONFIG_ICE_HWTS
+/**
+ * ice_ptp_get_syncdevicetime - Get the cross time stamp info
+ * @device: Current device time
+ * @system: System counter value read synchronously with device time
+ * @ctx: Context provided by timekeeping code
+ *
+ * Read device and system (ART) clock simultaneously and return the corrected
+ * clock values in ns.
+ */
+static int
+ice_ptp_get_syncdevicetime(ktime_t *device,
+ struct system_counterval_t *system,
+ void *ctx)
+{
+ struct ice_pf *pf = (struct ice_pf *)ctx;
+ struct ice_hw *hw = &pf->hw;
+ u32 hh_lock, hh_art_ctl;
+ int i;
+
+ /* Get the HW lock */
+ hh_lock = rd32(hw, PFHH_SEM + (PFTSYN_SEM_BYTES * hw->pf_id));
+ if (hh_lock & PFHH_SEM_BUSY_M) {
+ dev_err(ice_pf_to_dev(pf), "PTP failed to get hh lock\n");
+ return -EFAULT;
+ }
+
+ /* Start the ART and device clock sync sequence */
+ hh_art_ctl = rd32(hw, GLHH_ART_CTL);
+ hh_art_ctl = hh_art_ctl | GLHH_ART_CTL_ACTIVE_M;
+ wr32(hw, GLHH_ART_CTL, hh_art_ctl);
+
+#define MAX_HH_LOCK_TRIES 100
+
+ for (i = 0; i < MAX_HH_LOCK_TRIES; i++) {
+ /* Wait for sync to complete */
+ hh_art_ctl = rd32(hw, GLHH_ART_CTL);
+ if (hh_art_ctl & GLHH_ART_CTL_ACTIVE_M) {
+ udelay(1);
+ continue;
+ } else {
+ u32 hh_ts_lo, hh_ts_hi, tmr_idx;
+ u64 hh_ts;
+
+ tmr_idx = hw->func_caps.ts_func_info.tmr_index_assoc;
+ /* Read ART time */
+ hh_ts_lo = rd32(hw, GLHH_ART_TIME_L);
+ hh_ts_hi = rd32(hw, GLHH_ART_TIME_H);
+ hh_ts = ((u64)hh_ts_hi << 32) | hh_ts_lo;
+ *system = convert_art_ns_to_tsc(hh_ts);
+ /* Read Device source clock time */
+ hh_ts_lo = rd32(hw, GLTSYN_HHTIME_L(tmr_idx));
+ hh_ts_hi = rd32(hw, GLTSYN_HHTIME_H(tmr_idx));
+ hh_ts = ((u64)hh_ts_hi << 32) | hh_ts_lo;
+ *device = ns_to_ktime(hh_ts);
+ break;
+ }
+ }
+ /* Release HW lock */
+ hh_lock = rd32(hw, PFHH_SEM + (PFTSYN_SEM_BYTES * hw->pf_id));
+ hh_lock = hh_lock & ~PFHH_SEM_BUSY_M;
+ wr32(hw, PFHH_SEM + (PFTSYN_SEM_BYTES * hw->pf_id), hh_lock);
+
+ if (i == MAX_HH_LOCK_TRIES)
+ return -ETIMEDOUT;
+
+ return 0;
+}
+
+/**
+ * ice_ptp_getcrosststamp_e822 - Capture a device cross timestamp
+ * @info: the driver's PTP info structure
+ * @cts: The memory to fill the cross timestamp info
+ *
+ * Capture a cross timestamp between the ART and the device PTP hardware
+ * clock. Fill the cross timestamp information and report it back to the
+ * caller.
+ *
+ * This is only valid for E822 devices which have support for generating the
+ * cross timestamp via PCIe PTM.
+ *
+ * In order to correctly correlate the ART timestamp back to the TSC time, the
+ * CPU must have X86_FEATURE_TSC_KNOWN_FREQ.
+ */
+static int
+ice_ptp_getcrosststamp_e822(struct ptp_clock_info *info,
+ struct system_device_crosststamp *cts)
+{
+ struct ice_pf *pf = ptp_info_to_pf(info);
+
+ return get_device_system_crosststamp(ice_ptp_get_syncdevicetime,
+ pf, NULL, cts);
+}
+#endif /* CONFIG_ICE_HWTS */
+
/**
* ice_ptp_get_ts_config - ioctl interface to read the timestamping config
* @pf: Board private structure
@@ -1205,10 +1712,6 @@ int ice_ptp_get_ts_config(struct ice_pf *pf, struct ifreq *ifr)
static int
ice_ptp_set_timestamp_mode(struct ice_pf *pf, struct hwtstamp_config *config)
{
- /* Reserved for future extensions. */
- if (config->flags)
- return -EINVAL;
-
switch (config->tx_type) {
case HWTSTAMP_TX_OFF:
ice_set_tx_tstamp(pf, false);
@@ -1238,7 +1741,6 @@ ice_ptp_set_timestamp_mode(struct ice_pf *pf, struct hwtstamp_config *config)
case HWTSTAMP_FILTER_PTP_V2_L4_DELAY_REQ:
case HWTSTAMP_FILTER_NTP_ALL:
case HWTSTAMP_FILTER_ALL:
- config->rx_filter = HWTSTAMP_FILTER_ALL;
ice_set_rx_tstamp(pf, true);
break;
default:
@@ -1270,8 +1772,8 @@ int ice_ptp_set_ts_config(struct ice_pf *pf, struct ifreq *ifr)
if (err)
return err;
- /* Save these settings for future reference */
- pf->ptp.tstamp_config = config;
+ /* Return the actual configuration set */
+ config = pf->ptp.tstamp_config;
return copy_to_user(ifr->ifr_data, &config, sizeof(config)) ?
-EFAULT : 0;
@@ -1403,6 +1905,26 @@ static void ice_ptp_setup_pins_e810(struct ptp_clock_info *info)
}
/**
+ * ice_ptp_set_funcs_e822 - Set specialized functions for E822 support
+ * @pf: Board private structure
+ * @info: PTP info to fill
+ *
+ * Assign functions to the PTP capabiltiies structure for E822 devices.
+ * Functions which operate across all device families should be set directly
+ * in ice_ptp_set_caps. Only add functions here which are distinct for E822
+ * devices.
+ */
+static void
+ice_ptp_set_funcs_e822(struct ice_pf *pf, struct ptp_clock_info *info)
+{
+#ifdef CONFIG_ICE_HWTS
+ if (boot_cpu_has(X86_FEATURE_ART) &&
+ boot_cpu_has(X86_FEATURE_TSC_KNOWN_FREQ))
+ info->getcrosststamp = ice_ptp_getcrosststamp_e822;
+#endif /* CONFIG_ICE_HWTS */
+}
+
+/**
* ice_ptp_set_funcs_e810 - Set specialized functions for E810 support
* @pf: Board private structure
* @info: PTP info to fill
@@ -1441,7 +1963,10 @@ static void ice_ptp_set_caps(struct ice_pf *pf)
info->gettimex64 = ice_ptp_gettimex64;
info->settime64 = ice_ptp_settime64;
- ice_ptp_set_funcs_e810(pf, info);
+ if (ice_is_e810(&pf->hw))
+ ice_ptp_set_funcs_e810(pf, info);
+ else
+ ice_ptp_set_funcs_e822(pf, info);
}
/**
@@ -1588,7 +2113,7 @@ s8 ice_ptp_request_ts(struct ice_ptp_tx *tx, struct sk_buff *skb)
u8 idx;
/* Check if this tracker is initialized */
- if (!tx->init)
+ if (!tx->init || tx->calibrating)
return -1;
spin_lock(&tx->lock);
@@ -1704,13 +2229,34 @@ ice_ptp_release_tx_tracker(struct ice_pf *pf, struct ice_ptp_tx *tx)
kfree(tx->tstamps);
tx->tstamps = NULL;
- kfree(tx->in_use);
+ bitmap_free(tx->in_use);
tx->in_use = NULL;
tx->len = 0;
}
/**
+ * ice_ptp_init_tx_e822 - Initialize tracking for Tx timestamps
+ * @pf: Board private structure
+ * @tx: the Tx tracking structure to initialize
+ * @port: the port this structure tracks
+ *
+ * Initialize the Tx timestamp tracker for this port. For generic MAC devices,
+ * the timestamp block is shared for all ports in the same quad. To avoid
+ * ports using the same timestamp index, logically break the block of
+ * registers into chunks based on the port number.
+ */
+static int
+ice_ptp_init_tx_e822(struct ice_pf *pf, struct ice_ptp_tx *tx, u8 port)
+{
+ tx->quad = port / ICE_PORTS_PER_QUAD;
+ tx->quad_offset = tx->quad * INDEX_PER_PORT;
+ tx->len = INDEX_PER_PORT;
+
+ return ice_ptp_alloc_tx_tracker(tx);
+}
+
+/**
* ice_ptp_init_tx_e810 - Initialize tracking for Tx timestamps
* @pf: Board private structure
* @tx: the Tx tracking structure to initialize
@@ -1781,6 +2327,130 @@ static void ice_ptp_periodic_work(struct kthread_work *work)
}
/**
+ * ice_ptp_reset - Initialize PTP hardware clock support after reset
+ * @pf: Board private structure
+ */
+void ice_ptp_reset(struct ice_pf *pf)
+{
+ struct ice_ptp *ptp = &pf->ptp;
+ struct ice_hw *hw = &pf->hw;
+ struct timespec64 ts;
+ int err, itr = 1;
+ u64 time_diff;
+
+ if (test_bit(ICE_PFR_REQ, pf->state))
+ goto pfr;
+
+ if (!hw->func_caps.ts_func_info.src_tmr_owned)
+ goto reset_ts;
+
+ err = ice_ptp_init_phc(hw);
+ if (err)
+ goto err;
+
+ /* Acquire the global hardware lock */
+ if (!ice_ptp_lock(hw)) {
+ err = -EBUSY;
+ goto err;
+ }
+
+ /* Write the increment time value to PHY and LAN */
+ err = ice_ptp_write_incval(hw, ice_base_incval(pf));
+ if (err) {
+ ice_ptp_unlock(hw);
+ goto err;
+ }
+
+ /* Write the initial Time value to PHY and LAN using the cached PHC
+ * time before the reset and time difference between stopping and
+ * starting the clock.
+ */
+ if (ptp->cached_phc_time) {
+ time_diff = ktime_get_real_ns() - ptp->reset_time;
+ ts = ns_to_timespec64(ptp->cached_phc_time + time_diff);
+ } else {
+ ts = ktime_to_timespec64(ktime_get_real());
+ }
+ err = ice_ptp_write_init(pf, &ts);
+ if (err) {
+ ice_ptp_unlock(hw);
+ goto err;
+ }
+
+ /* Release the global hardware lock */
+ ice_ptp_unlock(hw);
+
+ if (!ice_is_e810(hw)) {
+ /* Enable quad interrupts */
+ err = ice_ptp_tx_ena_intr(pf, true, itr);
+ if (err)
+ goto err;
+ }
+
+reset_ts:
+ /* Restart the PHY timestamping block */
+ ice_ptp_reset_phy_timestamping(pf);
+
+pfr:
+ /* Init Tx structures */
+ if (ice_is_e810(&pf->hw)) {
+ err = ice_ptp_init_tx_e810(pf, &ptp->port.tx);
+ } else {
+ kthread_init_delayed_work(&ptp->port.ov_work,
+ ice_ptp_wait_for_offset_valid);
+ err = ice_ptp_init_tx_e822(pf, &ptp->port.tx,
+ ptp->port.port_num);
+ }
+ if (err)
+ goto err;
+
+ set_bit(ICE_FLAG_PTP, pf->flags);
+
+ /* Start periodic work going */
+ kthread_queue_delayed_work(ptp->kworker, &ptp->work, 0);
+
+ dev_info(ice_pf_to_dev(pf), "PTP reset successful\n");
+ return;
+
+err:
+ dev_err(ice_pf_to_dev(pf), "PTP reset failed %d\n", err);
+}
+
+/**
+ * ice_ptp_prepare_for_reset - Prepare PTP for reset
+ * @pf: Board private structure
+ */
+void ice_ptp_prepare_for_reset(struct ice_pf *pf)
+{
+ struct ice_ptp *ptp = &pf->ptp;
+ u8 src_tmr;
+
+ clear_bit(ICE_FLAG_PTP, pf->flags);
+
+ /* Disable timestamping for both Tx and Rx */
+ ice_ptp_cfg_timestamp(pf, false);
+
+ kthread_cancel_delayed_work_sync(&ptp->work);
+ kthread_cancel_work_sync(&ptp->extts_work);
+
+ if (test_bit(ICE_PFR_REQ, pf->state))
+ return;
+
+ ice_ptp_release_tx_tracker(pf, &pf->ptp.port.tx);
+
+ /* Disable periodic outputs */
+ ice_ptp_disable_all_clkout(pf);
+
+ src_tmr = ice_get_ptp_src_clock_index(&pf->hw);
+
+ /* Disable source clock */
+ wr32(&pf->hw, GLTSYN_ENA(src_tmr), (u32)~GLTSYN_ENA_TSYN_ENA_M);
+
+ /* Acquire PHC and system timer to restore after reset */
+ ptp->reset_time = ktime_get_real_ns();
+}
+
+/**
* ice_ptp_init_owner - Initialize PTP_1588_CLOCK device
* @pf: Board private structure
*
@@ -1790,27 +2460,16 @@ static void ice_ptp_periodic_work(struct kthread_work *work)
*/
static int ice_ptp_init_owner(struct ice_pf *pf)
{
- struct device *dev = ice_pf_to_dev(pf);
struct ice_hw *hw = &pf->hw;
struct timespec64 ts;
- u8 src_idx;
- int err;
+ int err, itr = 1;
- wr32(hw, GLTSYN_SYNC_DLAY, 0);
-
- /* Clear some HW residue and enable source clock */
- src_idx = hw->func_caps.ts_func_info.tmr_index_owned;
-
- /* Enable source clocks */
- wr32(hw, GLTSYN_ENA(src_idx), GLTSYN_ENA_TSYN_ENA_M);
-
- /* Enable PHY time sync */
- err = ice_ptp_init_phy_e810(hw);
- if (err)
- goto err_exit;
-
- /* Clear event status indications for auxiliary pins */
- (void)rd32(hw, GLTSYN_STAT(src_idx));
+ err = ice_ptp_init_phc(hw);
+ if (err) {
+ dev_err(ice_pf_to_dev(pf), "Failed to initialize PHC, err %d\n",
+ err);
+ return err;
+ }
/* Acquire the global hardware lock */
if (!ice_ptp_lock(hw)) {
@@ -1819,7 +2478,7 @@ static int ice_ptp_init_owner(struct ice_pf *pf)
}
/* Write the increment time value to PHY and LAN */
- err = ice_ptp_write_incval(hw, ICE_PTP_NOMINAL_INCVAL_E810);
+ err = ice_ptp_write_incval(hw, ice_base_incval(pf));
if (err) {
ice_ptp_unlock(hw);
goto err_exit;
@@ -1836,6 +2495,13 @@ static int ice_ptp_init_owner(struct ice_pf *pf)
/* Release the global hardware lock */
ice_ptp_unlock(hw);
+ if (!ice_is_e810(hw)) {
+ /* Enable quad interrupts */
+ err = ice_ptp_tx_ena_intr(pf, true, itr);
+ if (err)
+ goto err_exit;
+ }
+
/* Ensure we have a clock device */
err = ice_ptp_create_clock(pf);
if (err)
@@ -1849,72 +2515,106 @@ static int ice_ptp_init_owner(struct ice_pf *pf)
err_clk:
pf->ptp.clock = NULL;
err_exit:
- dev_err(dev, "PTP failed to register clock, err %d\n", err);
-
return err;
}
/**
- * ice_ptp_init - Initialize the PTP support after device probe or reset
+ * ice_ptp_init_work - Initialize PTP work threads
* @pf: Board private structure
- *
- * This function sets device up for PTP support. The first time it is run, it
- * will create a clock device. It does not create a clock device if one
- * already exists. It also reconfigures the device after a reset.
+ * @ptp: PF PTP structure
*/
-void ice_ptp_init(struct ice_pf *pf)
+static int ice_ptp_init_work(struct ice_pf *pf, struct ice_ptp *ptp)
{
- struct device *dev = ice_pf_to_dev(pf);
struct kthread_worker *kworker;
- struct ice_hw *hw = &pf->hw;
- int err;
-
- /* PTP is currently only supported on E810 devices */
- if (!ice_is_e810(hw))
- return;
-
- /* Check if this PF owns the source timer */
- if (hw->func_caps.ts_func_info.src_tmr_owned) {
- err = ice_ptp_init_owner(pf);
- if (err)
- return;
- }
-
- /* Disable timestamping for both Tx and Rx */
- ice_ptp_cfg_timestamp(pf, false);
-
- /* Initialize the PTP port Tx timestamp tracker */
- ice_ptp_init_tx_e810(pf, &pf->ptp.port.tx);
/* Initialize work functions */
- kthread_init_delayed_work(&pf->ptp.work, ice_ptp_periodic_work);
- kthread_init_work(&pf->ptp.extts_work, ice_ptp_extts_work);
+ kthread_init_delayed_work(&ptp->work, ice_ptp_periodic_work);
+ kthread_init_work(&ptp->extts_work, ice_ptp_extts_work);
/* Allocate a kworker for handling work required for the ports
* connected to the PTP hardware clock.
*/
- kworker = kthread_create_worker(0, "ice-ptp-%s", dev_name(dev));
- if (IS_ERR(kworker)) {
- err = PTR_ERR(kworker);
- goto err_kworker;
- }
- pf->ptp.kworker = kworker;
+ kworker = kthread_create_worker(0, "ice-ptp-%s",
+ dev_name(ice_pf_to_dev(pf)));
+ if (IS_ERR(kworker))
+ return PTR_ERR(kworker);
- set_bit(ICE_FLAG_PTP, pf->flags);
+ ptp->kworker = kworker;
/* Start periodic work going */
- kthread_queue_delayed_work(pf->ptp.kworker, &pf->ptp.work, 0);
+ kthread_queue_delayed_work(ptp->kworker, &ptp->work, 0);
- dev_info(dev, "PTP init successful\n");
+ return 0;
+}
+
+/**
+ * ice_ptp_init_port - Initialize PTP port structure
+ * @pf: Board private structure
+ * @ptp_port: PTP port structure
+ */
+static int ice_ptp_init_port(struct ice_pf *pf, struct ice_ptp_port *ptp_port)
+{
+ mutex_init(&ptp_port->ps_lock);
+
+ if (ice_is_e810(&pf->hw))
+ return ice_ptp_init_tx_e810(pf, &ptp_port->tx);
+
+ kthread_init_delayed_work(&ptp_port->ov_work,
+ ice_ptp_wait_for_offset_valid);
+ return ice_ptp_init_tx_e822(pf, &ptp_port->tx, ptp_port->port_num);
+}
+
+/**
+ * ice_ptp_init - Initialize PTP hardware clock support
+ * @pf: Board private structure
+ *
+ * Set up the device for interacting with the PTP hardware clock for all
+ * functions, both the function that owns the clock hardware, and the
+ * functions connected to the clock hardware.
+ *
+ * The clock owner will allocate and register a ptp_clock with the
+ * PTP_1588_CLOCK infrastructure. All functions allocate a kthread and work
+ * items used for asynchronous work such as Tx timestamps and periodic work.
+ */
+void ice_ptp_init(struct ice_pf *pf)
+{
+ struct ice_ptp *ptp = &pf->ptp;
+ struct ice_hw *hw = &pf->hw;
+ int err;
+
+ /* If this function owns the clock hardware, it must allocate and
+ * configure the PTP clock device to represent it.
+ */
+ if (hw->func_caps.ts_func_info.src_tmr_owned) {
+ err = ice_ptp_init_owner(pf);
+ if (err)
+ goto err;
+ }
+
+ ptp->port.port_num = hw->pf_id;
+ err = ice_ptp_init_port(pf, &ptp->port);
+ if (err)
+ goto err;
+
+ /* Start the PHY timestamping block */
+ ice_ptp_reset_phy_timestamping(pf);
+
+ set_bit(ICE_FLAG_PTP, pf->flags);
+ err = ice_ptp_init_work(pf, ptp);
+ if (err)
+ goto err;
+
+ dev_info(ice_pf_to_dev(pf), "PTP init successful\n");
return;
-err_kworker:
+err:
/* If we registered a PTP clock, release it */
if (pf->ptp.clock) {
- ptp_clock_unregister(pf->ptp.clock);
+ ptp_clock_unregister(ptp->clock);
pf->ptp.clock = NULL;
}
- dev_err(dev, "PTP failed %d\n", err);
+ clear_bit(ICE_FLAG_PTP, pf->flags);
+ dev_err(ice_pf_to_dev(pf), "PTP failed %d\n", err);
}
/**
@@ -1938,6 +2638,8 @@ void ice_ptp_release(struct ice_pf *pf)
kthread_cancel_delayed_work_sync(&pf->ptp.work);
+ ice_ptp_port_phy_stop(&pf->ptp.port);
+ mutex_destroy(&pf->ptp.port.ps_lock);
if (pf->ptp.kworker) {
kthread_destroy_worker(pf->ptp.kworker);
pf->ptp.kworker = NULL;
diff --git a/drivers/net/ethernet/intel/ice/ice_ptp.h b/drivers/net/ethernet/intel/ice/ice_ptp.h
index 53c15fc..afd048d 100644
--- a/drivers/net/ethernet/intel/ice/ice_ptp.h
+++ b/drivers/net/ethernet/intel/ice/ice_ptp.h
@@ -82,6 +82,8 @@ struct ice_tx_tstamp {
* @quad_offset: offset into timestamp block of the quad to get the real index
* @len: length of the tstamps and in_use fields.
* @init: if true, the tracker is initialized;
+ * @calibrating: if true, the PHY is calibrating the Tx offset. During this
+ * window, timestamps are temporarily disabled.
*/
struct ice_ptp_tx {
struct kthread_work work;
@@ -92,6 +94,7 @@ struct ice_ptp_tx {
u8 quad_offset;
u8 len;
u8 init;
+ u8 calibrating;
};
/* Quad and port information for initializing timestamp blocks */
@@ -101,15 +104,24 @@ struct ice_ptp_tx {
/**
* struct ice_ptp_port - data used to initialize an external port for PTP
*
- * This structure contains PTP data related to the external ports. Currently
- * it is used for tracking the Tx timestamps of a port. In the future this
- * structure will also hold information for the E822 port initialization
- * logic.
+ * This structure contains data indicating whether a single external port is
+ * ready for PTP functionality. It is used to track the port initialization
+ * and determine when the port's PHY offset is valid.
*
* @tx: Tx timestamp tracking for this port
+ * @ov_work: delayed work task for tracking when PHY offset is valid
+ * @ps_lock: mutex used to protect the overall PTP PHY start procedure
+ * @link_up: indicates whether the link is up
+ * @tx_fifo_busy_cnt: number of times the Tx FIFO was busy
+ * @port_num: the port number this structure represents
*/
struct ice_ptp_port {
struct ice_ptp_tx tx;
+ struct kthread_delayed_work ov_work;
+ struct mutex ps_lock; /* protects overall PTP PHY start procedure */
+ bool link_up;
+ u8 tx_fifo_busy_cnt;
+ u8 port_num;
};
#define GLTSYN_TGT_H_IDX_MAX 4
@@ -127,6 +139,7 @@ struct ice_ptp_port {
* @info: structure defining PTP hardware capabilities
* @clock: pointer to registered PTP clock device
* @tstamp_config: hardware timestamping configuration
+ * @reset_time: kernel time after clock stop on reset
*/
struct ice_ptp {
struct ice_ptp_port port;
@@ -140,6 +153,7 @@ struct ice_ptp {
struct ptp_clock_info info;
struct ptp_clock *clock;
struct hwtstamp_config tstamp_config;
+ u64 reset_time;
};
#define __ptp_port_to_ptp(p) \
@@ -152,9 +166,15 @@ struct ice_ptp {
#define ptp_info_to_pf(i) \
container_of(__ptp_info_to_ptp((i)), struct ice_pf, ptp)
+#define PFTSYN_SEM_BYTES 4
#define PTP_SHARED_CLK_IDX_VALID BIT(31)
+#define TS_CMD_MASK 0xF
+#define SYNC_EXEC_CMD 0x3
#define ICE_PTP_TS_VALID BIT(0)
+#define FIFO_EMPTY BIT(2)
+#define FIFO_OK 0xFF
+#define ICE_PTP_FIFO_NUM_CHECKS 5
/* Per-channel register definitions */
#define GLTSYN_AUX_OUT(_chan, _idx) (GLTSYN_AUX_OUT_0(_idx) + ((_chan) * 8))
#define GLTSYN_AUX_IN(_chan, _idx) (GLTSYN_AUX_IN_0(_idx) + ((_chan) * 8))
@@ -175,11 +195,13 @@ struct ice_ptp {
#define N_PER_OUT_E810T 3
#define N_PER_OUT_E810T_NO_SMA 2
#define N_EXT_TS_E810_NO_SMA 2
+#define ETH_GLTSYN_ENA(_i) (0x03000348 + ((_i) * 4))
#if IS_ENABLED(CONFIG_PTP_1588_CLOCK)
struct ice_pf;
int ice_ptp_set_ts_config(struct ice_pf *pf, struct ifreq *ifr);
int ice_ptp_get_ts_config(struct ice_pf *pf, struct ifreq *ifr);
+void ice_ptp_cfg_timestamp(struct ice_pf *pf, bool ena);
int ice_get_ptp_clock_index(struct ice_pf *pf);
s8 ice_ptp_request_ts(struct ice_ptp_tx *tx, struct sk_buff *skb);
@@ -188,8 +210,11 @@ void ice_ptp_process_ts(struct ice_pf *pf);
void
ice_ptp_rx_hwtstamp(struct ice_rx_ring *rx_ring,
union ice_32b_rx_flex_desc *rx_desc, struct sk_buff *skb);
+void ice_ptp_reset(struct ice_pf *pf);
+void ice_ptp_prepare_for_reset(struct ice_pf *pf);
void ice_ptp_init(struct ice_pf *pf);
void ice_ptp_release(struct ice_pf *pf);
+int ice_ptp_link_change(struct ice_pf *pf, u8 port, bool linkup);
#else /* IS_ENABLED(CONFIG_PTP_1588_CLOCK) */
static inline int ice_ptp_set_ts_config(struct ice_pf *pf, struct ifreq *ifr)
{
@@ -201,6 +226,7 @@ static inline int ice_ptp_get_ts_config(struct ice_pf *pf, struct ifreq *ifr)
return -EOPNOTSUPP;
}
+static inline void ice_ptp_cfg_timestamp(struct ice_pf *pf, bool ena) { }
static inline int ice_get_ptp_clock_index(struct ice_pf *pf)
{
return -1;
@@ -216,7 +242,11 @@ static inline void ice_ptp_process_ts(struct ice_pf *pf) { }
static inline void
ice_ptp_rx_hwtstamp(struct ice_rx_ring *rx_ring,
union ice_32b_rx_flex_desc *rx_desc, struct sk_buff *skb) { }
+static inline void ice_ptp_reset(struct ice_pf *pf) { }
+static inline void ice_ptp_prepare_for_reset(struct ice_pf *pf) { }
static inline void ice_ptp_init(struct ice_pf *pf) { }
static inline void ice_ptp_release(struct ice_pf *pf) { }
+static inline int ice_ptp_link_change(struct ice_pf *pf, u8 port, bool linkup)
+{ return 0; }
#endif /* IS_ENABLED(CONFIG_PTP_1588_CLOCK) */
#endif /* _ICE_PTP_H_ */
diff --git a/drivers/net/ethernet/intel/ice/ice_ptp_consts.h b/drivers/net/ethernet/intel/ice/ice_ptp_consts.h
new file mode 100644
index 0000000..4109aa3
--- /dev/null
+++ b/drivers/net/ethernet/intel/ice/ice_ptp_consts.h
@@ -0,0 +1,374 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2018-2021, Intel Corporation. */
+
+#ifndef _ICE_PTP_CONSTS_H_
+#define _ICE_PTP_CONSTS_H_
+
+/* Constant definitions related to the hardware clock used for PTP 1588
+ * features and functionality.
+ */
+/* Constants defined for the PTP 1588 clock hardware. */
+
+/* struct ice_time_ref_info_e822
+ *
+ * E822 hardware can use different sources as the reference for the PTP
+ * hardware clock. Each clock has different characteristics such as a slightly
+ * different frequency, etc.
+ *
+ * This lookup table defines several constants that depend on the current time
+ * reference. See the struct ice_time_ref_info_e822 for information about the
+ * meaning of each constant.
+ */
+const struct ice_time_ref_info_e822 e822_time_ref[NUM_ICE_TIME_REF_FREQ] = {
+ /* ICE_TIME_REF_FREQ_25_000 -> 25 MHz */
+ {
+ /* pll_freq */
+ 823437500, /* 823.4375 MHz PLL */
+ /* nominal_incval */
+ 0x136e44fabULL,
+ /* pps_delay */
+ 11,
+ },
+
+ /* ICE_TIME_REF_FREQ_122_880 -> 122.88 MHz */
+ {
+ /* pll_freq */
+ 783360000, /* 783.36 MHz */
+ /* nominal_incval */
+ 0x146cc2177ULL,
+ /* pps_delay */
+ 12,
+ },
+
+ /* ICE_TIME_REF_FREQ_125_000 -> 125 MHz */
+ {
+ /* pll_freq */
+ 796875000, /* 796.875 MHz */
+ /* nominal_incval */
+ 0x141414141ULL,
+ /* pps_delay */
+ 12,
+ },
+
+ /* ICE_TIME_REF_FREQ_153_600 -> 153.6 MHz */
+ {
+ /* pll_freq */
+ 816000000, /* 816 MHz */
+ /* nominal_incval */
+ 0x139b9b9baULL,
+ /* pps_delay */
+ 12,
+ },
+
+ /* ICE_TIME_REF_FREQ_156_250 -> 156.25 MHz */
+ {
+ /* pll_freq */
+ 830078125, /* 830.78125 MHz */
+ /* nominal_incval */
+ 0x134679aceULL,
+ /* pps_delay */
+ 11,
+ },
+
+ /* ICE_TIME_REF_FREQ_245_760 -> 245.76 MHz */
+ {
+ /* pll_freq */
+ 783360000, /* 783.36 MHz */
+ /* nominal_incval */
+ 0x146cc2177ULL,
+ /* pps_delay */
+ 12,
+ },
+};
+
+const struct ice_cgu_pll_params_e822 e822_cgu_params[NUM_ICE_TIME_REF_FREQ] = {
+ /* ICE_TIME_REF_FREQ_25_000 -> 25 MHz */
+ {
+ /* refclk_pre_div */
+ 1,
+ /* feedback_div */
+ 197,
+ /* frac_n_div */
+ 2621440,
+ /* post_pll_div */
+ 6,
+ },
+
+ /* ICE_TIME_REF_FREQ_122_880 -> 122.88 MHz */
+ {
+ /* refclk_pre_div */
+ 5,
+ /* feedback_div */
+ 223,
+ /* frac_n_div */
+ 524288,
+ /* post_pll_div */
+ 7,
+ },
+
+ /* ICE_TIME_REF_FREQ_125_000 -> 125 MHz */
+ {
+ /* refclk_pre_div */
+ 5,
+ /* feedback_div */
+ 223,
+ /* frac_n_div */
+ 524288,
+ /* post_pll_div */
+ 7,
+ },
+
+ /* ICE_TIME_REF_FREQ_153_600 -> 153.6 MHz */
+ {
+ /* refclk_pre_div */
+ 5,
+ /* feedback_div */
+ 159,
+ /* frac_n_div */
+ 1572864,
+ /* post_pll_div */
+ 6,
+ },
+
+ /* ICE_TIME_REF_FREQ_156_250 -> 156.25 MHz */
+ {
+ /* refclk_pre_div */
+ 5,
+ /* feedback_div */
+ 159,
+ /* frac_n_div */
+ 1572864,
+ /* post_pll_div */
+ 6,
+ },
+
+ /* ICE_TIME_REF_FREQ_245_760 -> 245.76 MHz */
+ {
+ /* refclk_pre_div */
+ 10,
+ /* feedback_div */
+ 223,
+ /* frac_n_div */
+ 524288,
+ /* post_pll_div */
+ 7,
+ },
+};
+
+/* struct ice_vernier_info_e822
+ *
+ * E822 hardware calibrates the delay of the timestamp indication from the
+ * actual packet transmission or reception during the initialization of the
+ * PHY. To do this, the hardware mechanism uses some conversions between the
+ * various clocks within the PHY block. This table defines constants used to
+ * calculate the correct conversion ratios in the PHY registers.
+ *
+ * Many of the values relate to the PAR/PCS clock conversion registers. For
+ * these registers, a value of 0 means that the associated register is not
+ * used by this link speed, and that the register should be cleared by writing
+ * 0. Other values specify the clock frequency in Hz.
+ */
+const struct ice_vernier_info_e822 e822_vernier[NUM_ICE_PTP_LNK_SPD] = {
+ /* ICE_PTP_LNK_SPD_1G */
+ {
+ /* tx_par_clk */
+ 31250000, /* 31.25 MHz */
+ /* rx_par_clk */
+ 31250000, /* 31.25 MHz */
+ /* tx_pcs_clk */
+ 125000000, /* 125 MHz */
+ /* rx_pcs_clk */
+ 125000000, /* 125 MHz */
+ /* tx_desk_rsgb_par */
+ 0, /* unused */
+ /* rx_desk_rsgb_par */
+ 0, /* unused */
+ /* tx_desk_rsgb_pcs */
+ 0, /* unused */
+ /* rx_desk_rsgb_pcs */
+ 0, /* unused */
+ /* tx_fixed_delay */
+ 25140,
+ /* pmd_adj_divisor */
+ 10000000,
+ /* rx_fixed_delay */
+ 17372,
+ },
+ /* ICE_PTP_LNK_SPD_10G */
+ {
+ /* tx_par_clk */
+ 257812500, /* 257.8125 MHz */
+ /* rx_par_clk */
+ 257812500, /* 257.8125 MHz */
+ /* tx_pcs_clk */
+ 156250000, /* 156.25 MHz */
+ /* rx_pcs_clk */
+ 156250000, /* 156.25 MHz */
+ /* tx_desk_rsgb_par */
+ 0, /* unused */
+ /* rx_desk_rsgb_par */
+ 0, /* unused */
+ /* tx_desk_rsgb_pcs */
+ 0, /* unused */
+ /* rx_desk_rsgb_pcs */
+ 0, /* unused */
+ /* tx_fixed_delay */
+ 6938,
+ /* pmd_adj_divisor */
+ 82500000,
+ /* rx_fixed_delay */
+ 6212,
+ },
+ /* ICE_PTP_LNK_SPD_25G */
+ {
+ /* tx_par_clk */
+ 644531250, /* 644.53125 MHZ */
+ /* rx_par_clk */
+ 644531250, /* 644.53125 MHz */
+ /* tx_pcs_clk */
+ 390625000, /* 390.625 MHz */
+ /* rx_pcs_clk */
+ 390625000, /* 390.625 MHz */
+ /* tx_desk_rsgb_par */
+ 0, /* unused */
+ /* rx_desk_rsgb_par */
+ 0, /* unused */
+ /* tx_desk_rsgb_pcs */
+ 0, /* unused */
+ /* rx_desk_rsgb_pcs */
+ 0, /* unused */
+ /* tx_fixed_delay */
+ 2778,
+ /* pmd_adj_divisor */
+ 206250000,
+ /* rx_fixed_delay */
+ 2491,
+ },
+ /* ICE_PTP_LNK_SPD_25G_RS */
+ {
+ /* tx_par_clk */
+ 0, /* unused */
+ /* rx_par_clk */
+ 0, /* unused */
+ /* tx_pcs_clk */
+ 0, /* unused */
+ /* rx_pcs_clk */
+ 0, /* unused */
+ /* tx_desk_rsgb_par */
+ 161132812, /* 162.1328125 MHz Reed Solomon gearbox */
+ /* rx_desk_rsgb_par */
+ 161132812, /* 162.1328125 MHz Reed Solomon gearbox */
+ /* tx_desk_rsgb_pcs */
+ 97656250, /* 97.62625 MHz Reed Solomon gearbox */
+ /* rx_desk_rsgb_pcs */
+ 97656250, /* 97.62625 MHz Reed Solomon gearbox */
+ /* tx_fixed_delay */
+ 3928,
+ /* pmd_adj_divisor */
+ 206250000,
+ /* rx_fixed_delay */
+ 29535,
+ },
+ /* ICE_PTP_LNK_SPD_40G */
+ {
+ /* tx_par_clk */
+ 257812500,
+ /* rx_par_clk */
+ 257812500,
+ /* tx_pcs_clk */
+ 156250000, /* 156.25 MHz */
+ /* rx_pcs_clk */
+ 156250000, /* 156.25 MHz */
+ /* tx_desk_rsgb_par */
+ 0, /* unused */
+ /* rx_desk_rsgb_par */
+ 156250000, /* 156.25 MHz deskew clock */
+ /* tx_desk_rsgb_pcs */
+ 0, /* unused */
+ /* rx_desk_rsgb_pcs */
+ 156250000, /* 156.25 MHz deskew clock */
+ /* tx_fixed_delay */
+ 5666,
+ /* pmd_adj_divisor */
+ 82500000,
+ /* rx_fixed_delay */
+ 4244,
+ },
+ /* ICE_PTP_LNK_SPD_50G */
+ {
+ /* tx_par_clk */
+ 644531250, /* 644.53125 MHZ */
+ /* rx_par_clk */
+ 644531250, /* 644.53125 MHZ */
+ /* tx_pcs_clk */
+ 390625000, /* 390.625 MHz */
+ /* rx_pcs_clk */
+ 390625000, /* 390.625 MHz */
+ /* tx_desk_rsgb_par */
+ 0, /* unused */
+ /* rx_desk_rsgb_par */
+ 195312500, /* 193.3125 MHz deskew clock */
+ /* tx_desk_rsgb_pcs */
+ 0, /* unused */
+ /* rx_desk_rsgb_pcs */
+ 195312500, /* 193.3125 MHz deskew clock */
+ /* tx_fixed_delay */
+ 2778,
+ /* pmd_adj_divisor */
+ 206250000,
+ /* rx_fixed_delay */
+ 2868,
+ },
+ /* ICE_PTP_LNK_SPD_50G_RS */
+ {
+ /* tx_par_clk */
+ 0, /* unused */
+ /* rx_par_clk */
+ 644531250, /* 644.53125 MHz */
+ /* tx_pcs_clk */
+ 0, /* unused */
+ /* rx_pcs_clk */
+ 644531250, /* 644.53125 MHz */
+ /* tx_desk_rsgb_par */
+ 322265625, /* 322.265625 MHz Reed Solomon gearbox */
+ /* rx_desk_rsgb_par */
+ 322265625, /* 322.265625 MHz Reed Solomon gearbox */
+ /* tx_desk_rsgb_pcs */
+ 644531250, /* 644.53125 MHz Reed Solomon gearbox */
+ /* rx_desk_rsgb_pcs */
+ 644531250, /* 644.53125 MHz Reed Solomon gearbox */
+ /* tx_fixed_delay */
+ 2095,
+ /* pmd_adj_divisor */
+ 206250000,
+ /* rx_fixed_delay */
+ 14524,
+ },
+ /* ICE_PTP_LNK_SPD_100G_RS */
+ {
+ /* tx_par_clk */
+ 0, /* unused */
+ /* rx_par_clk */
+ 644531250, /* 644.53125 MHz */
+ /* tx_pcs_clk */
+ 0, /* unused */
+ /* rx_pcs_clk */
+ 644531250, /* 644.53125 MHz */
+ /* tx_desk_rsgb_par */
+ 644531250, /* 644.53125 MHz Reed Solomon gearbox */
+ /* rx_desk_rsgb_par */
+ 644531250, /* 644.53125 MHz Reed Solomon gearbox */
+ /* tx_desk_rsgb_pcs */
+ 644531250, /* 644.53125 MHz Reed Solomon gearbox */
+ /* rx_desk_rsgb_pcs */
+ 644531250, /* 644.53125 MHz Reed Solomon gearbox */
+ /* tx_fixed_delay */
+ 1620,
+ /* pmd_adj_divisor */
+ 206250000,
+ /* rx_fixed_delay */
+ 7775,
+ },
+};
+
+#endif /* _ICE_PTP_CONSTS_H_ */
diff --git a/drivers/net/ethernet/intel/ice/ice_ptp_hw.c b/drivers/net/ethernet/intel/ice/ice_ptp_hw.c
index 29f947c..ec8450f 100644
--- a/drivers/net/ethernet/intel/ice/ice_ptp_hw.c
+++ b/drivers/net/ethernet/intel/ice/ice_ptp_hw.c
@@ -3,6 +3,8 @@
#include "ice_common.h"
#include "ice_ptp_hw.h"
+#include "ice_ptp_consts.h"
+#include "ice_cgu_regs.h"
/* Low level functions for interacting with and managing the device clock used
* for the Precision Time Protocol.
@@ -29,6 +31,15 @@
*
* For E810 devices, the increment frequency is 812.5 MHz
*
+ * For E822 devices the clock can be derived from different sources, and the
+ * increment has an effective frequency of one of the following:
+ * - 823.4375 MHz
+ * - 783.36 MHz
+ * - 796.875 MHz
+ * - 816 MHz
+ * - 830.078125 MHz
+ * - 783.36 MHz
+ *
* The hardware captures timestamps in the PHY for incoming packets, and for
* outgoing packets on request. To support this, the PHY maintains a timer
* that matches the lower 64 bits of the global source timer.
@@ -37,6 +48,24 @@
* shadow registers are used to prepare the desired initial values. A special
* sync command is issued to trigger copying from the shadow registers into
* the appropriate source and PHY registers simultaneously.
+ *
+ * The driver supports devices which have different PHYs with subtly different
+ * mechanisms to program and control the timers. We divide the devices into
+ * families named after the first major device, E810 and similar devices, and
+ * E822 and similar devices.
+ *
+ * - E822 based devices have additional support for fine grained Vernier
+ * calibration which requires significant setup
+ * - The layout of timestamp data in the PHY register blocks is different
+ * - The way timer synchronization commands are issued is different.
+ *
+ * To support this, very low level functions have an e810 or e822 suffix
+ * indicating what type of device they work on. Higher level abstractions for
+ * tasks that can be done on both devices do not have the suffix and will
+ * correctly look up the appropriate low level function when running.
+ *
+ * Functions which only make sense on a single device family may not have
+ * a suitable generic implementation
*/
/**
@@ -51,6 +80,2447 @@ u8 ice_get_ptp_src_clock_index(struct ice_hw *hw)
return hw->func_caps.ts_func_info.tmr_index_assoc;
}
+/**
+ * ice_ptp_read_src_incval - Read source timer increment value
+ * @hw: pointer to HW struct
+ *
+ * Read the increment value of the source timer and return it.
+ */
+static u64 ice_ptp_read_src_incval(struct ice_hw *hw)
+{
+ u32 lo, hi;
+ u8 tmr_idx;
+
+ tmr_idx = ice_get_ptp_src_clock_index(hw);
+
+ lo = rd32(hw, GLTSYN_INCVAL_L(tmr_idx));
+ hi = rd32(hw, GLTSYN_INCVAL_H(tmr_idx));
+
+ return ((u64)(hi & INCVAL_HIGH_M) << 32) | lo;
+}
+
+/**
+ * ice_ptp_src_cmd - Prepare source timer for a timer command
+ * @hw: pointer to HW structure
+ * @cmd: Timer command
+ *
+ * Prepare the source timer for an upcoming timer sync command.
+ */
+static void ice_ptp_src_cmd(struct ice_hw *hw, enum ice_ptp_tmr_cmd cmd)
+{
+ u32 cmd_val;
+ u8 tmr_idx;
+
+ tmr_idx = ice_get_ptp_src_clock_index(hw);
+ cmd_val = tmr_idx << SEL_CPK_SRC;
+
+ switch (cmd) {
+ case INIT_TIME:
+ cmd_val |= GLTSYN_CMD_INIT_TIME;
+ break;
+ case INIT_INCVAL:
+ cmd_val |= GLTSYN_CMD_INIT_INCVAL;
+ break;
+ case ADJ_TIME:
+ cmd_val |= GLTSYN_CMD_ADJ_TIME;
+ break;
+ case ADJ_TIME_AT_TIME:
+ cmd_val |= GLTSYN_CMD_ADJ_INIT_TIME;
+ break;
+ case READ_TIME:
+ cmd_val |= GLTSYN_CMD_READ_TIME;
+ break;
+ }
+
+ wr32(hw, GLTSYN_CMD, cmd_val);
+}
+
+/**
+ * ice_ptp_exec_tmr_cmd - Execute all prepared timer commands
+ * @hw: pointer to HW struct
+ *
+ * Write the SYNC_EXEC_CMD bit to the GLTSYN_CMD_SYNC register, and flush the
+ * write immediately. This triggers the hardware to begin executing all of the
+ * source and PHY timer commands synchronously.
+ */
+static void ice_ptp_exec_tmr_cmd(struct ice_hw *hw)
+{
+ wr32(hw, GLTSYN_CMD_SYNC, SYNC_EXEC_CMD);
+ ice_flush(hw);
+}
+
+/* E822 family functions
+ *
+ * The following functions operate on the E822 family of devices.
+ */
+
+/**
+ * ice_fill_phy_msg_e822 - Fill message data for a PHY register access
+ * @msg: the PHY message buffer to fill in
+ * @port: the port to access
+ * @offset: the register offset
+ */
+static void
+ice_fill_phy_msg_e822(struct ice_sbq_msg_input *msg, u8 port, u16 offset)
+{
+ int phy_port, phy, quadtype;
+
+ phy_port = port % ICE_PORTS_PER_PHY;
+ phy = port / ICE_PORTS_PER_PHY;
+ quadtype = (port / ICE_PORTS_PER_QUAD) % ICE_NUM_QUAD_TYPE;
+
+ if (quadtype == 0) {
+ msg->msg_addr_low = P_Q0_L(P_0_BASE + offset, phy_port);
+ msg->msg_addr_high = P_Q0_H(P_0_BASE + offset, phy_port);
+ } else {
+ msg->msg_addr_low = P_Q1_L(P_4_BASE + offset, phy_port);
+ msg->msg_addr_high = P_Q1_H(P_4_BASE + offset, phy_port);
+ }
+
+ if (phy == 0)
+ msg->dest_dev = rmn_0;
+ else if (phy == 1)
+ msg->dest_dev = rmn_1;
+ else
+ msg->dest_dev = rmn_2;
+}
+
+/**
+ * ice_is_64b_phy_reg_e822 - Check if this is a 64bit PHY register
+ * @low_addr: the low address to check
+ * @high_addr: on return, contains the high address of the 64bit register
+ *
+ * Checks if the provided low address is one of the known 64bit PHY values
+ * represented as two 32bit registers. If it is, return the appropriate high
+ * register offset to use.
+ */
+static bool ice_is_64b_phy_reg_e822(u16 low_addr, u16 *high_addr)
+{
+ switch (low_addr) {
+ case P_REG_PAR_PCS_TX_OFFSET_L:
+ *high_addr = P_REG_PAR_PCS_TX_OFFSET_U;
+ return true;
+ case P_REG_PAR_PCS_RX_OFFSET_L:
+ *high_addr = P_REG_PAR_PCS_RX_OFFSET_U;
+ return true;
+ case P_REG_PAR_TX_TIME_L:
+ *high_addr = P_REG_PAR_TX_TIME_U;
+ return true;
+ case P_REG_PAR_RX_TIME_L:
+ *high_addr = P_REG_PAR_RX_TIME_U;
+ return true;
+ case P_REG_TOTAL_TX_OFFSET_L:
+ *high_addr = P_REG_TOTAL_TX_OFFSET_U;
+ return true;
+ case P_REG_TOTAL_RX_OFFSET_L:
+ *high_addr = P_REG_TOTAL_RX_OFFSET_U;
+ return true;
+ case P_REG_UIX66_10G_40G_L:
+ *high_addr = P_REG_UIX66_10G_40G_U;
+ return true;
+ case P_REG_UIX66_25G_100G_L:
+ *high_addr = P_REG_UIX66_25G_100G_U;
+ return true;
+ case P_REG_TX_CAPTURE_L:
+ *high_addr = P_REG_TX_CAPTURE_U;
+ return true;
+ case P_REG_RX_CAPTURE_L:
+ *high_addr = P_REG_RX_CAPTURE_U;
+ return true;
+ case P_REG_TX_TIMER_INC_PRE_L:
+ *high_addr = P_REG_TX_TIMER_INC_PRE_U;
+ return true;
+ case P_REG_RX_TIMER_INC_PRE_L:
+ *high_addr = P_REG_RX_TIMER_INC_PRE_U;
+ return true;
+ default:
+ return false;
+ }
+}
+
+/**
+ * ice_is_40b_phy_reg_e822 - Check if this is a 40bit PHY register
+ * @low_addr: the low address to check
+ * @high_addr: on return, contains the high address of the 40bit value
+ *
+ * Checks if the provided low address is one of the known 40bit PHY values
+ * split into two registers with the lower 8 bits in the low register and the
+ * upper 32 bits in the high register. If it is, return the appropriate high
+ * register offset to use.
+ */
+static bool ice_is_40b_phy_reg_e822(u16 low_addr, u16 *high_addr)
+{
+ switch (low_addr) {
+ case P_REG_TIMETUS_L:
+ *high_addr = P_REG_TIMETUS_U;
+ return true;
+ case P_REG_PAR_RX_TUS_L:
+ *high_addr = P_REG_PAR_RX_TUS_U;
+ return true;
+ case P_REG_PAR_TX_TUS_L:
+ *high_addr = P_REG_PAR_TX_TUS_U;
+ return true;
+ case P_REG_PCS_RX_TUS_L:
+ *high_addr = P_REG_PCS_RX_TUS_U;
+ return true;
+ case P_REG_PCS_TX_TUS_L:
+ *high_addr = P_REG_PCS_TX_TUS_U;
+ return true;
+ case P_REG_DESK_PAR_RX_TUS_L:
+ *high_addr = P_REG_DESK_PAR_RX_TUS_U;
+ return true;
+ case P_REG_DESK_PAR_TX_TUS_L:
+ *high_addr = P_REG_DESK_PAR_TX_TUS_U;
+ return true;
+ case P_REG_DESK_PCS_RX_TUS_L:
+ *high_addr = P_REG_DESK_PCS_RX_TUS_U;
+ return true;
+ case P_REG_DESK_PCS_TX_TUS_L:
+ *high_addr = P_REG_DESK_PCS_TX_TUS_U;
+ return true;
+ default:
+ return false;
+ }
+}
+
+/**
+ * ice_read_phy_reg_e822 - Read a PHY register
+ * @hw: pointer to the HW struct
+ * @port: PHY port to read from
+ * @offset: PHY register offset to read
+ * @val: on return, the contents read from the PHY
+ *
+ * Read a PHY register for the given port over the device sideband queue.
+ */
+int
+ice_read_phy_reg_e822(struct ice_hw *hw, u8 port, u16 offset, u32 *val)
+{
+ struct ice_sbq_msg_input msg = {0};
+ int err;
+
+ ice_fill_phy_msg_e822(&msg, port, offset);
+ msg.opcode = ice_sbq_msg_rd;
+
+ err = ice_sbq_rw_reg(hw, &msg);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to send message to PHY, err %d\n",
+ err);
+ return err;
+ }
+
+ *val = msg.data;
+
+ return 0;
+}
+
+/**
+ * ice_read_64b_phy_reg_e822 - Read a 64bit value from PHY registers
+ * @hw: pointer to the HW struct
+ * @port: PHY port to read from
+ * @low_addr: offset of the lower register to read from
+ * @val: on return, the contents of the 64bit value from the PHY registers
+ *
+ * Reads the two registers associated with a 64bit value and returns it in the
+ * val pointer. The offset always specifies the lower register offset to use.
+ * The high offset is looked up. This function only operates on registers
+ * known to be two parts of a 64bit value.
+ */
+static int
+ice_read_64b_phy_reg_e822(struct ice_hw *hw, u8 port, u16 low_addr, u64 *val)
+{
+ u32 low, high;
+ u16 high_addr;
+ int err;
+
+ /* Only operate on registers known to be split into two 32bit
+ * registers.
+ */
+ if (!ice_is_64b_phy_reg_e822(low_addr, &high_addr)) {
+ ice_debug(hw, ICE_DBG_PTP, "Invalid 64b register addr 0x%08x\n",
+ low_addr);
+ return -EINVAL;
+ }
+
+ err = ice_read_phy_reg_e822(hw, port, low_addr, &low);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to read from low register 0x%08x\n, err %d",
+ low_addr, err);
+ return err;
+ }
+
+ err = ice_read_phy_reg_e822(hw, port, high_addr, &high);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to read from high register 0x%08x\n, err %d",
+ high_addr, err);
+ return err;
+ }
+
+ *val = (u64)high << 32 | low;
+
+ return 0;
+}
+
+/**
+ * ice_write_phy_reg_e822 - Write a PHY register
+ * @hw: pointer to the HW struct
+ * @port: PHY port to write to
+ * @offset: PHY register offset to write
+ * @val: The value to write to the register
+ *
+ * Write a PHY register for the given port over the device sideband queue.
+ */
+int
+ice_write_phy_reg_e822(struct ice_hw *hw, u8 port, u16 offset, u32 val)
+{
+ struct ice_sbq_msg_input msg = {0};
+ int err;
+
+ ice_fill_phy_msg_e822(&msg, port, offset);
+ msg.opcode = ice_sbq_msg_wr;
+ msg.data = val;
+
+ err = ice_sbq_rw_reg(hw, &msg);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to send message to PHY, err %d\n",
+ err);
+ return err;
+ }
+
+ return 0;
+}
+
+/**
+ * ice_write_40b_phy_reg_e822 - Write a 40b value to the PHY
+ * @hw: pointer to the HW struct
+ * @port: port to write to
+ * @low_addr: offset of the low register
+ * @val: 40b value to write
+ *
+ * Write the provided 40b value to the two associated registers by splitting
+ * it up into two chunks, the lower 8 bits and the upper 32 bits.
+ */
+static int
+ice_write_40b_phy_reg_e822(struct ice_hw *hw, u8 port, u16 low_addr, u64 val)
+{
+ u32 low, high;
+ u16 high_addr;
+ int err;
+
+ /* Only operate on registers known to be split into a lower 8 bit
+ * register and an upper 32 bit register.
+ */
+ if (!ice_is_40b_phy_reg_e822(low_addr, &high_addr)) {
+ ice_debug(hw, ICE_DBG_PTP, "Invalid 40b register addr 0x%08x\n",
+ low_addr);
+ return -EINVAL;
+ }
+
+ low = (u32)(val & P_REG_40B_LOW_M);
+ high = (u32)(val >> P_REG_40B_HIGH_S);
+
+ err = ice_write_phy_reg_e822(hw, port, low_addr, low);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to write to low register 0x%08x\n, err %d",
+ low_addr, err);
+ return err;
+ }
+
+ err = ice_write_phy_reg_e822(hw, port, high_addr, high);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to write to high register 0x%08x\n, err %d",
+ high_addr, err);
+ return err;
+ }
+
+ return 0;
+}
+
+/**
+ * ice_write_64b_phy_reg_e822 - Write a 64bit value to PHY registers
+ * @hw: pointer to the HW struct
+ * @port: PHY port to read from
+ * @low_addr: offset of the lower register to read from
+ * @val: the contents of the 64bit value to write to PHY
+ *
+ * Write the 64bit value to the two associated 32bit PHY registers. The offset
+ * is always specified as the lower register, and the high address is looked
+ * up. This function only operates on registers known to be two parts of
+ * a 64bit value.
+ */
+static int
+ice_write_64b_phy_reg_e822(struct ice_hw *hw, u8 port, u16 low_addr, u64 val)
+{
+ u32 low, high;
+ u16 high_addr;
+ int err;
+
+ /* Only operate on registers known to be split into two 32bit
+ * registers.
+ */
+ if (!ice_is_64b_phy_reg_e822(low_addr, &high_addr)) {
+ ice_debug(hw, ICE_DBG_PTP, "Invalid 64b register addr 0x%08x\n",
+ low_addr);
+ return -EINVAL;
+ }
+
+ low = lower_32_bits(val);
+ high = upper_32_bits(val);
+
+ err = ice_write_phy_reg_e822(hw, port, low_addr, low);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to write to low register 0x%08x\n, err %d",
+ low_addr, err);
+ return err;
+ }
+
+ err = ice_write_phy_reg_e822(hw, port, high_addr, high);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to write to high register 0x%08x\n, err %d",
+ high_addr, err);
+ return err;
+ }
+
+ return 0;
+}
+
+/**
+ * ice_fill_quad_msg_e822 - Fill message data for quad register access
+ * @msg: the PHY message buffer to fill in
+ * @quad: the quad to access
+ * @offset: the register offset
+ *
+ * Fill a message buffer for accessing a register in a quad shared between
+ * multiple PHYs.
+ */
+static void
+ice_fill_quad_msg_e822(struct ice_sbq_msg_input *msg, u8 quad, u16 offset)
+{
+ u32 addr;
+
+ msg->dest_dev = rmn_0;
+
+ if ((quad % ICE_NUM_QUAD_TYPE) == 0)
+ addr = Q_0_BASE + offset;
+ else
+ addr = Q_1_BASE + offset;
+
+ msg->msg_addr_low = lower_16_bits(addr);
+ msg->msg_addr_high = upper_16_bits(addr);
+}
+
+/**
+ * ice_read_quad_reg_e822 - Read a PHY quad register
+ * @hw: pointer to the HW struct
+ * @quad: quad to read from
+ * @offset: quad register offset to read
+ * @val: on return, the contents read from the quad
+ *
+ * Read a quad register over the device sideband queue. Quad registers are
+ * shared between multiple PHYs.
+ */
+int
+ice_read_quad_reg_e822(struct ice_hw *hw, u8 quad, u16 offset, u32 *val)
+{
+ struct ice_sbq_msg_input msg = {0};
+ int err;
+
+ if (quad >= ICE_MAX_QUAD)
+ return -EINVAL;
+
+ ice_fill_quad_msg_e822(&msg, quad, offset);
+ msg.opcode = ice_sbq_msg_rd;
+
+ err = ice_sbq_rw_reg(hw, &msg);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to send message to PHY, err %d\n",
+ err);
+ return err;
+ }
+
+ *val = msg.data;
+
+ return 0;
+}
+
+/**
+ * ice_write_quad_reg_e822 - Write a PHY quad register
+ * @hw: pointer to the HW struct
+ * @quad: quad to write to
+ * @offset: quad register offset to write
+ * @val: The value to write to the register
+ *
+ * Write a quad register over the device sideband queue. Quad registers are
+ * shared between multiple PHYs.
+ */
+int
+ice_write_quad_reg_e822(struct ice_hw *hw, u8 quad, u16 offset, u32 val)
+{
+ struct ice_sbq_msg_input msg = {0};
+ int err;
+
+ if (quad >= ICE_MAX_QUAD)
+ return -EINVAL;
+
+ ice_fill_quad_msg_e822(&msg, quad, offset);
+ msg.opcode = ice_sbq_msg_wr;
+ msg.data = val;
+
+ err = ice_sbq_rw_reg(hw, &msg);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to send message to PHY, err %d\n",
+ err);
+ return err;
+ }
+
+ return 0;
+}
+
+/**
+ * ice_read_phy_tstamp_e822 - Read a PHY timestamp out of the quad block
+ * @hw: pointer to the HW struct
+ * @quad: the quad to read from
+ * @idx: the timestamp index to read
+ * @tstamp: on return, the 40bit timestamp value
+ *
+ * Read a 40bit timestamp value out of the two associated registers in the
+ * quad memory block that is shared between the internal PHYs of the E822
+ * family of devices.
+ */
+static int
+ice_read_phy_tstamp_e822(struct ice_hw *hw, u8 quad, u8 idx, u64 *tstamp)
+{
+ u16 lo_addr, hi_addr;
+ u32 lo, hi;
+ int err;
+
+ lo_addr = (u16)TS_L(Q_REG_TX_MEMORY_BANK_START, idx);
+ hi_addr = (u16)TS_H(Q_REG_TX_MEMORY_BANK_START, idx);
+
+ err = ice_read_quad_reg_e822(hw, quad, lo_addr, &lo);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to read low PTP timestamp register, err %d\n",
+ err);
+ return err;
+ }
+
+ err = ice_read_quad_reg_e822(hw, quad, hi_addr, &hi);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to read high PTP timestamp register, err %d\n",
+ err);
+ return err;
+ }
+
+ /* For E822 based internal PHYs, the timestamp is reported with the
+ * lower 8 bits in the low register, and the upper 32 bits in the high
+ * register.
+ */
+ *tstamp = ((u64)hi) << TS_PHY_HIGH_S | ((u64)lo & TS_PHY_LOW_M);
+
+ return 0;
+}
+
+/**
+ * ice_clear_phy_tstamp_e822 - Clear a timestamp from the quad block
+ * @hw: pointer to the HW struct
+ * @quad: the quad to read from
+ * @idx: the timestamp index to reset
+ *
+ * Clear a timestamp, resetting its valid bit, from the PHY quad block that is
+ * shared between the internal PHYs on the E822 devices.
+ */
+static int
+ice_clear_phy_tstamp_e822(struct ice_hw *hw, u8 quad, u8 idx)
+{
+ u16 lo_addr, hi_addr;
+ int err;
+
+ lo_addr = (u16)TS_L(Q_REG_TX_MEMORY_BANK_START, idx);
+ hi_addr = (u16)TS_H(Q_REG_TX_MEMORY_BANK_START, idx);
+
+ err = ice_write_quad_reg_e822(hw, quad, lo_addr, 0);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to clear low PTP timestamp register, err %d\n",
+ err);
+ return err;
+ }
+
+ err = ice_write_quad_reg_e822(hw, quad, hi_addr, 0);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to clear high PTP timestamp register, err %d\n",
+ err);
+ return err;
+ }
+
+ return 0;
+}
+
+/**
+ * ice_read_cgu_reg_e822 - Read a CGU register
+ * @hw: pointer to the HW struct
+ * @addr: Register address to read
+ * @val: storage for register value read
+ *
+ * Read the contents of a register of the Clock Generation Unit. Only
+ * applicable to E822 devices.
+ */
+static int
+ice_read_cgu_reg_e822(struct ice_hw *hw, u32 addr, u32 *val)
+{
+ struct ice_sbq_msg_input cgu_msg;
+ int err;
+
+ cgu_msg.opcode = ice_sbq_msg_rd;
+ cgu_msg.dest_dev = cgu;
+ cgu_msg.msg_addr_low = addr;
+ cgu_msg.msg_addr_high = 0x0;
+
+ err = ice_sbq_rw_reg(hw, &cgu_msg);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to read CGU register 0x%04x, err %d\n",
+ addr, err);
+ return err;
+ }
+
+ *val = cgu_msg.data;
+
+ return err;
+}
+
+/**
+ * ice_write_cgu_reg_e822 - Write a CGU register
+ * @hw: pointer to the HW struct
+ * @addr: Register address to write
+ * @val: value to write into the register
+ *
+ * Write the specified value to a register of the Clock Generation Unit. Only
+ * applicable to E822 devices.
+ */
+static int
+ice_write_cgu_reg_e822(struct ice_hw *hw, u32 addr, u32 val)
+{
+ struct ice_sbq_msg_input cgu_msg;
+ int err;
+
+ cgu_msg.opcode = ice_sbq_msg_wr;
+ cgu_msg.dest_dev = cgu;
+ cgu_msg.msg_addr_low = addr;
+ cgu_msg.msg_addr_high = 0x0;
+ cgu_msg.data = val;
+
+ err = ice_sbq_rw_reg(hw, &cgu_msg);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to write CGU register 0x%04x, err %d\n",
+ addr, err);
+ return err;
+ }
+
+ return err;
+}
+
+/**
+ * ice_clk_freq_str - Convert time_ref_freq to string
+ * @clk_freq: Clock frequency
+ *
+ * Convert the specified TIME_REF clock frequency to a string.
+ */
+static const char *ice_clk_freq_str(u8 clk_freq)
+{
+ switch ((enum ice_time_ref_freq)clk_freq) {
+ case ICE_TIME_REF_FREQ_25_000:
+ return "25 MHz";
+ case ICE_TIME_REF_FREQ_122_880:
+ return "122.88 MHz";
+ case ICE_TIME_REF_FREQ_125_000:
+ return "125 MHz";
+ case ICE_TIME_REF_FREQ_153_600:
+ return "153.6 MHz";
+ case ICE_TIME_REF_FREQ_156_250:
+ return "156.25 MHz";
+ case ICE_TIME_REF_FREQ_245_760:
+ return "245.76 MHz";
+ default:
+ return "Unknown";
+ }
+}
+
+/**
+ * ice_clk_src_str - Convert time_ref_src to string
+ * @clk_src: Clock source
+ *
+ * Convert the specified clock source to its string name.
+ */
+static const char *ice_clk_src_str(u8 clk_src)
+{
+ switch ((enum ice_clk_src)clk_src) {
+ case ICE_CLK_SRC_TCX0:
+ return "TCX0";
+ case ICE_CLK_SRC_TIME_REF:
+ return "TIME_REF";
+ default:
+ return "Unknown";
+ }
+}
+
+/**
+ * ice_cfg_cgu_pll_e822 - Configure the Clock Generation Unit
+ * @hw: pointer to the HW struct
+ * @clk_freq: Clock frequency to program
+ * @clk_src: Clock source to select (TIME_REF, or TCX0)
+ *
+ * Configure the Clock Generation Unit with the desired clock frequency and
+ * time reference, enabling the PLL which drives the PTP hardware clock.
+ */
+static int
+ice_cfg_cgu_pll_e822(struct ice_hw *hw, enum ice_time_ref_freq clk_freq,
+ enum ice_clk_src clk_src)
+{
+ union tspll_ro_bwm_lf bwm_lf;
+ union nac_cgu_dword19 dw19;
+ union nac_cgu_dword22 dw22;
+ union nac_cgu_dword24 dw24;
+ union nac_cgu_dword9 dw9;
+ int err;
+
+ if (clk_freq >= NUM_ICE_TIME_REF_FREQ) {
+ dev_warn(ice_hw_to_dev(hw), "Invalid TIME_REF frequency %u\n",
+ clk_freq);
+ return -EINVAL;
+ }
+
+ if (clk_src >= NUM_ICE_CLK_SRC) {
+ dev_warn(ice_hw_to_dev(hw), "Invalid clock source %u\n",
+ clk_src);
+ return -EINVAL;
+ }
+
+ if (clk_src == ICE_CLK_SRC_TCX0 &&
+ clk_freq != ICE_TIME_REF_FREQ_25_000) {
+ dev_warn(ice_hw_to_dev(hw),
+ "TCX0 only supports 25 MHz frequency\n");
+ return -EINVAL;
+ }
+
+ err = ice_read_cgu_reg_e822(hw, NAC_CGU_DWORD9, &dw9.val);
+ if (err)
+ return err;
+
+ err = ice_read_cgu_reg_e822(hw, NAC_CGU_DWORD24, &dw24.val);
+ if (err)
+ return err;
+
+ err = ice_read_cgu_reg_e822(hw, TSPLL_RO_BWM_LF, &bwm_lf.val);
+ if (err)
+ return err;
+
+ /* Log the current clock configuration */
+ ice_debug(hw, ICE_DBG_PTP, "Current CGU configuration -- %s, clk_src %s, clk_freq %s, PLL %s\n",
+ dw24.field.ts_pll_enable ? "enabled" : "disabled",
+ ice_clk_src_str(dw24.field.time_ref_sel),
+ ice_clk_freq_str(dw9.field.time_ref_freq_sel),
+ bwm_lf.field.plllock_true_lock_cri ? "locked" : "unlocked");
+
+ /* Disable the PLL before changing the clock source or frequency */
+ if (dw24.field.ts_pll_enable) {
+ dw24.field.ts_pll_enable = 0;
+
+ err = ice_write_cgu_reg_e822(hw, NAC_CGU_DWORD24, dw24.val);
+ if (err)
+ return err;
+ }
+
+ /* Set the frequency */
+ dw9.field.time_ref_freq_sel = clk_freq;
+ err = ice_write_cgu_reg_e822(hw, NAC_CGU_DWORD9, dw9.val);
+ if (err)
+ return err;
+
+ /* Configure the TS PLL feedback divisor */
+ err = ice_read_cgu_reg_e822(hw, NAC_CGU_DWORD19, &dw19.val);
+ if (err)
+ return err;
+
+ dw19.field.tspll_fbdiv_intgr = e822_cgu_params[clk_freq].feedback_div;
+ dw19.field.tspll_ndivratio = 1;
+
+ err = ice_write_cgu_reg_e822(hw, NAC_CGU_DWORD19, dw19.val);
+ if (err)
+ return err;
+
+ /* Configure the TS PLL post divisor */
+ err = ice_read_cgu_reg_e822(hw, NAC_CGU_DWORD22, &dw22.val);
+ if (err)
+ return err;
+
+ dw22.field.time1588clk_div = e822_cgu_params[clk_freq].post_pll_div;
+ dw22.field.time1588clk_sel_div2 = 0;
+
+ err = ice_write_cgu_reg_e822(hw, NAC_CGU_DWORD22, dw22.val);
+ if (err)
+ return err;
+
+ /* Configure the TS PLL pre divisor and clock source */
+ err = ice_read_cgu_reg_e822(hw, NAC_CGU_DWORD24, &dw24.val);
+ if (err)
+ return err;
+
+ dw24.field.ref1588_ck_div = e822_cgu_params[clk_freq].refclk_pre_div;
+ dw24.field.tspll_fbdiv_frac = e822_cgu_params[clk_freq].frac_n_div;
+ dw24.field.time_ref_sel = clk_src;
+
+ err = ice_write_cgu_reg_e822(hw, NAC_CGU_DWORD24, dw24.val);
+ if (err)
+ return err;
+
+ /* Finally, enable the PLL */
+ dw24.field.ts_pll_enable = 1;
+
+ err = ice_write_cgu_reg_e822(hw, NAC_CGU_DWORD24, dw24.val);
+ if (err)
+ return err;
+
+ /* Wait to verify if the PLL locks */
+ usleep_range(1000, 5000);
+
+ err = ice_read_cgu_reg_e822(hw, TSPLL_RO_BWM_LF, &bwm_lf.val);
+ if (err)
+ return err;
+
+ if (!bwm_lf.field.plllock_true_lock_cri) {
+ dev_warn(ice_hw_to_dev(hw), "CGU PLL failed to lock\n");
+ return -EBUSY;
+ }
+
+ /* Log the current clock configuration */
+ ice_debug(hw, ICE_DBG_PTP, "New CGU configuration -- %s, clk_src %s, clk_freq %s, PLL %s\n",
+ dw24.field.ts_pll_enable ? "enabled" : "disabled",
+ ice_clk_src_str(dw24.field.time_ref_sel),
+ ice_clk_freq_str(dw9.field.time_ref_freq_sel),
+ bwm_lf.field.plllock_true_lock_cri ? "locked" : "unlocked");
+
+ return 0;
+}
+
+/**
+ * ice_init_cgu_e822 - Initialize CGU with settings from firmware
+ * @hw: pointer to the HW structure
+ *
+ * Initialize the Clock Generation Unit of the E822 device.
+ */
+static int ice_init_cgu_e822(struct ice_hw *hw)
+{
+ struct ice_ts_func_info *ts_info = &hw->func_caps.ts_func_info;
+ union tspll_cntr_bist_settings cntr_bist;
+ int err;
+
+ err = ice_read_cgu_reg_e822(hw, TSPLL_CNTR_BIST_SETTINGS,
+ &cntr_bist.val);
+ if (err)
+ return err;
+
+ /* Disable sticky lock detection so lock err reported is accurate */
+ cntr_bist.field.i_plllock_sel_0 = 0;
+ cntr_bist.field.i_plllock_sel_1 = 0;
+
+ err = ice_write_cgu_reg_e822(hw, TSPLL_CNTR_BIST_SETTINGS,
+ cntr_bist.val);
+ if (err)
+ return err;
+
+ /* Configure the CGU PLL using the parameters from the function
+ * capabilities.
+ */
+ err = ice_cfg_cgu_pll_e822(hw, ts_info->time_ref,
+ (enum ice_clk_src)ts_info->clk_src);
+ if (err)
+ return err;
+
+ return 0;
+}
+
+/**
+ * ice_ptp_set_vernier_wl - Set the window length for vernier calibration
+ * @hw: pointer to the HW struct
+ *
+ * Set the window length used for the vernier port calibration process.
+ */
+static int ice_ptp_set_vernier_wl(struct ice_hw *hw)
+{
+ u8 port;
+
+ for (port = 0; port < ICE_NUM_EXTERNAL_PORTS; port++) {
+ int err;
+
+ err = ice_write_phy_reg_e822(hw, port, P_REG_WL,
+ PTP_VERNIER_WL);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to set vernier window length for port %u, err %d\n",
+ port, err);
+ return err;
+ }
+ }
+
+ return 0;
+}
+
+/**
+ * ice_ptp_init_phc_e822 - Perform E822 specific PHC initialization
+ * @hw: pointer to HW struct
+ *
+ * Perform PHC initialization steps specific to E822 devices.
+ */
+static int ice_ptp_init_phc_e822(struct ice_hw *hw)
+{
+ int err;
+ u32 regval;
+
+ /* Enable reading switch and PHY registers over the sideband queue */
+#define PF_SB_REM_DEV_CTL_SWITCH_READ BIT(1)
+#define PF_SB_REM_DEV_CTL_PHY0 BIT(2)
+ regval = rd32(hw, PF_SB_REM_DEV_CTL);
+ regval |= (PF_SB_REM_DEV_CTL_SWITCH_READ |
+ PF_SB_REM_DEV_CTL_PHY0);
+ wr32(hw, PF_SB_REM_DEV_CTL, regval);
+
+ /* Initialize the Clock Generation Unit */
+ err = ice_init_cgu_e822(hw);
+ if (err)
+ return err;
+
+ /* Set window length for all the ports */
+ return ice_ptp_set_vernier_wl(hw);
+}
+
+/**
+ * ice_ptp_prep_phy_time_e822 - Prepare PHY port with initial time
+ * @hw: pointer to the HW struct
+ * @time: Time to initialize the PHY port clocks to
+ *
+ * Program the PHY port registers with a new initial time value. The port
+ * clock will be initialized once the driver issues an INIT_TIME sync
+ * command. The time value is the upper 32 bits of the PHY timer, usually in
+ * units of nominal nanoseconds.
+ */
+static int
+ice_ptp_prep_phy_time_e822(struct ice_hw *hw, u32 time)
+{
+ u64 phy_time;
+ u8 port;
+ int err;
+
+ /* The time represents the upper 32 bits of the PHY timer, so we need
+ * to shift to account for this when programming.
+ */
+ phy_time = (u64)time << 32;
+
+ for (port = 0; port < ICE_NUM_EXTERNAL_PORTS; port++) {
+ /* Tx case */
+ err = ice_write_64b_phy_reg_e822(hw, port,
+ P_REG_TX_TIMER_INC_PRE_L,
+ phy_time);
+ if (err)
+ goto exit_err;
+
+ /* Rx case */
+ err = ice_write_64b_phy_reg_e822(hw, port,
+ P_REG_RX_TIMER_INC_PRE_L,
+ phy_time);
+ if (err)
+ goto exit_err;
+ }
+
+ return 0;
+
+exit_err:
+ ice_debug(hw, ICE_DBG_PTP, "Failed to write init time for port %u, err %d\n",
+ port, err);
+
+ return err;
+}
+
+/**
+ * ice_ptp_prep_port_adj_e822 - Prepare a single port for time adjust
+ * @hw: pointer to HW struct
+ * @port: Port number to be programmed
+ * @time: time in cycles to adjust the port Tx and Rx clocks
+ *
+ * Program the port for an atomic adjustment by writing the Tx and Rx timer
+ * registers. The atomic adjustment won't be completed until the driver issues
+ * an ADJ_TIME command.
+ *
+ * Note that time is not in units of nanoseconds. It is in clock time
+ * including the lower sub-nanosecond portion of the port timer.
+ *
+ * Negative adjustments are supported using 2s complement arithmetic.
+ */
+int
+ice_ptp_prep_port_adj_e822(struct ice_hw *hw, u8 port, s64 time)
+{
+ u32 l_time, u_time;
+ int err;
+
+ l_time = lower_32_bits(time);
+ u_time = upper_32_bits(time);
+
+ /* Tx case */
+ err = ice_write_phy_reg_e822(hw, port, P_REG_TX_TIMER_INC_PRE_L,
+ l_time);
+ if (err)
+ goto exit_err;
+
+ err = ice_write_phy_reg_e822(hw, port, P_REG_TX_TIMER_INC_PRE_U,
+ u_time);
+ if (err)
+ goto exit_err;
+
+ /* Rx case */
+ err = ice_write_phy_reg_e822(hw, port, P_REG_RX_TIMER_INC_PRE_L,
+ l_time);
+ if (err)
+ goto exit_err;
+
+ err = ice_write_phy_reg_e822(hw, port, P_REG_RX_TIMER_INC_PRE_U,
+ u_time);
+ if (err)
+ goto exit_err;
+
+ return 0;
+
+exit_err:
+ ice_debug(hw, ICE_DBG_PTP, "Failed to write time adjust for port %u, err %d\n",
+ port, err);
+ return err;
+}
+
+/**
+ * ice_ptp_prep_phy_adj_e822 - Prep PHY ports for a time adjustment
+ * @hw: pointer to HW struct
+ * @adj: adjustment in nanoseconds
+ *
+ * Prepare the PHY ports for an atomic time adjustment by programming the PHY
+ * Tx and Rx port registers. The actual adjustment is completed by issuing an
+ * ADJ_TIME or ADJ_TIME_AT_TIME sync command.
+ */
+static int
+ice_ptp_prep_phy_adj_e822(struct ice_hw *hw, s32 adj)
+{
+ s64 cycles;
+ u8 port;
+
+ /* The port clock supports adjustment of the sub-nanosecond portion of
+ * the clock. We shift the provided adjustment in nanoseconds to
+ * calculate the appropriate adjustment to program into the PHY ports.
+ */
+ if (adj > 0)
+ cycles = (s64)adj << 32;
+ else
+ cycles = -(((s64)-adj) << 32);
+
+ for (port = 0; port < ICE_NUM_EXTERNAL_PORTS; port++) {
+ int err;
+
+ err = ice_ptp_prep_port_adj_e822(hw, port, cycles);
+ if (err)
+ return err;
+ }
+
+ return 0;
+}
+
+/**
+ * ice_ptp_prep_phy_incval_e822 - Prepare PHY ports for time adjustment
+ * @hw: pointer to HW struct
+ * @incval: new increment value to prepare
+ *
+ * Prepare each of the PHY ports for a new increment value by programming the
+ * port's TIMETUS registers. The new increment value will be updated after
+ * issuing an INIT_INCVAL command.
+ */
+static int
+ice_ptp_prep_phy_incval_e822(struct ice_hw *hw, u64 incval)
+{
+ int err;
+ u8 port;
+
+ for (port = 0; port < ICE_NUM_EXTERNAL_PORTS; port++) {
+ err = ice_write_40b_phy_reg_e822(hw, port, P_REG_TIMETUS_L,
+ incval);
+ if (err)
+ goto exit_err;
+ }
+
+ return 0;
+
+exit_err:
+ ice_debug(hw, ICE_DBG_PTP, "Failed to write incval for port %u, err %d\n",
+ port, err);
+
+ return err;
+}
+
+/**
+ * ice_ptp_read_port_capture - Read a port's local time capture
+ * @hw: pointer to HW struct
+ * @port: Port number to read
+ * @tx_ts: on return, the Tx port time capture
+ * @rx_ts: on return, the Rx port time capture
+ *
+ * Read the port's Tx and Rx local time capture values.
+ *
+ * Note this has no equivalent for the E810 devices.
+ */
+static int
+ice_ptp_read_port_capture(struct ice_hw *hw, u8 port, u64 *tx_ts, u64 *rx_ts)
+{
+ int err;
+
+ /* Tx case */
+ err = ice_read_64b_phy_reg_e822(hw, port, P_REG_TX_CAPTURE_L, tx_ts);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to read REG_TX_CAPTURE, err %d\n",
+ err);
+ return err;
+ }
+
+ ice_debug(hw, ICE_DBG_PTP, "tx_init = 0x%016llx\n",
+ (unsigned long long)*tx_ts);
+
+ /* Rx case */
+ err = ice_read_64b_phy_reg_e822(hw, port, P_REG_RX_CAPTURE_L, rx_ts);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to read RX_CAPTURE, err %d\n",
+ err);
+ return err;
+ }
+
+ ice_debug(hw, ICE_DBG_PTP, "rx_init = 0x%016llx\n",
+ (unsigned long long)*rx_ts);
+
+ return 0;
+}
+
+/**
+ * ice_ptp_one_port_cmd - Prepare a single PHY port for a timer command
+ * @hw: pointer to HW struct
+ * @port: Port to which cmd has to be sent
+ * @cmd: Command to be sent to the port
+ *
+ * Prepare the requested port for an upcoming timer sync command.
+ *
+ * Note there is no equivalent of this operation on E810, as that device
+ * always handles all external PHYs internally.
+ */
+static int
+ice_ptp_one_port_cmd(struct ice_hw *hw, u8 port, enum ice_ptp_tmr_cmd cmd)
+{
+ u32 cmd_val, val;
+ u8 tmr_idx;
+ int err;
+
+ tmr_idx = ice_get_ptp_src_clock_index(hw);
+ cmd_val = tmr_idx << SEL_PHY_SRC;
+ switch (cmd) {
+ case INIT_TIME:
+ cmd_val |= PHY_CMD_INIT_TIME;
+ break;
+ case INIT_INCVAL:
+ cmd_val |= PHY_CMD_INIT_INCVAL;
+ break;
+ case ADJ_TIME:
+ cmd_val |= PHY_CMD_ADJ_TIME;
+ break;
+ case READ_TIME:
+ cmd_val |= PHY_CMD_READ_TIME;
+ break;
+ case ADJ_TIME_AT_TIME:
+ cmd_val |= PHY_CMD_ADJ_TIME_AT_TIME;
+ break;
+ }
+
+ /* Tx case */
+ /* Read, modify, write */
+ err = ice_read_phy_reg_e822(hw, port, P_REG_TX_TMR_CMD, &val);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to read TX_TMR_CMD, err %d\n",
+ err);
+ return err;
+ }
+
+ /* Modify necessary bits only and perform write */
+ val &= ~TS_CMD_MASK;
+ val |= cmd_val;
+
+ err = ice_write_phy_reg_e822(hw, port, P_REG_TX_TMR_CMD, val);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to write back TX_TMR_CMD, err %d\n",
+ err);
+ return err;
+ }
+
+ /* Rx case */
+ /* Read, modify, write */
+ err = ice_read_phy_reg_e822(hw, port, P_REG_RX_TMR_CMD, &val);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to read RX_TMR_CMD, err %d\n",
+ err);
+ return err;
+ }
+
+ /* Modify necessary bits only and perform write */
+ val &= ~TS_CMD_MASK;
+ val |= cmd_val;
+
+ err = ice_write_phy_reg_e822(hw, port, P_REG_RX_TMR_CMD, val);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to write back RX_TMR_CMD, err %d\n",
+ err);
+ return err;
+ }
+
+ return 0;
+}
+
+/**
+ * ice_ptp_port_cmd_e822 - Prepare all ports for a timer command
+ * @hw: pointer to the HW struct
+ * @cmd: timer command to prepare
+ *
+ * Prepare all ports connected to this device for an upcoming timer sync
+ * command.
+ */
+static int
+ice_ptp_port_cmd_e822(struct ice_hw *hw, enum ice_ptp_tmr_cmd cmd)
+{
+ u8 port;
+
+ for (port = 0; port < ICE_NUM_EXTERNAL_PORTS; port++) {
+ int err;
+
+ err = ice_ptp_one_port_cmd(hw, port, cmd);
+ if (err)
+ return err;
+ }
+
+ return 0;
+}
+
+/* E822 Vernier calibration functions
+ *
+ * The following functions are used as part of the vernier calibration of
+ * a port. This calibration increases the precision of the timestamps on the
+ * port.
+ */
+
+/**
+ * ice_phy_get_speed_and_fec_e822 - Get link speed and FEC based on serdes mode
+ * @hw: pointer to HW struct
+ * @port: the port to read from
+ * @link_out: if non-NULL, holds link speed on success
+ * @fec_out: if non-NULL, holds FEC algorithm on success
+ *
+ * Read the serdes data for the PHY port and extract the link speed and FEC
+ * algorithm.
+ */
+static int
+ice_phy_get_speed_and_fec_e822(struct ice_hw *hw, u8 port,
+ enum ice_ptp_link_spd *link_out,
+ enum ice_ptp_fec_mode *fec_out)
+{
+ enum ice_ptp_link_spd link;
+ enum ice_ptp_fec_mode fec;
+ u32 serdes;
+ int err;
+
+ err = ice_read_phy_reg_e822(hw, port, P_REG_LINK_SPEED, &serdes);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to read serdes info\n");
+ return err;
+ }
+
+ /* Determine the FEC algorithm */
+ fec = (enum ice_ptp_fec_mode)P_REG_LINK_SPEED_FEC_MODE(serdes);
+
+ serdes &= P_REG_LINK_SPEED_SERDES_M;
+
+ /* Determine the link speed */
+ if (fec == ICE_PTP_FEC_MODE_RS_FEC) {
+ switch (serdes) {
+ case ICE_PTP_SERDES_25G:
+ link = ICE_PTP_LNK_SPD_25G_RS;
+ break;
+ case ICE_PTP_SERDES_50G:
+ link = ICE_PTP_LNK_SPD_50G_RS;
+ break;
+ case ICE_PTP_SERDES_100G:
+ link = ICE_PTP_LNK_SPD_100G_RS;
+ break;
+ default:
+ return -EIO;
+ }
+ } else {
+ switch (serdes) {
+ case ICE_PTP_SERDES_1G:
+ link = ICE_PTP_LNK_SPD_1G;
+ break;
+ case ICE_PTP_SERDES_10G:
+ link = ICE_PTP_LNK_SPD_10G;
+ break;
+ case ICE_PTP_SERDES_25G:
+ link = ICE_PTP_LNK_SPD_25G;
+ break;
+ case ICE_PTP_SERDES_40G:
+ link = ICE_PTP_LNK_SPD_40G;
+ break;
+ case ICE_PTP_SERDES_50G:
+ link = ICE_PTP_LNK_SPD_50G;
+ break;
+ default:
+ return -EIO;
+ }
+ }
+
+ if (link_out)
+ *link_out = link;
+ if (fec_out)
+ *fec_out = fec;
+
+ return 0;
+}
+
+/**
+ * ice_phy_cfg_lane_e822 - Configure PHY quad for single/multi-lane timestamp
+ * @hw: pointer to HW struct
+ * @port: to configure the quad for
+ */
+static void ice_phy_cfg_lane_e822(struct ice_hw *hw, u8 port)
+{
+ enum ice_ptp_link_spd link_spd;
+ int err;
+ u32 val;
+ u8 quad;
+
+ err = ice_phy_get_speed_and_fec_e822(hw, port, &link_spd, NULL);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to get PHY link speed, err %d\n",
+ err);
+ return;
+ }
+
+ quad = port / ICE_PORTS_PER_QUAD;
+
+ err = ice_read_quad_reg_e822(hw, quad, Q_REG_TX_MEM_GBL_CFG, &val);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to read TX_MEM_GLB_CFG, err %d\n",
+ err);
+ return;
+ }
+
+ if (link_spd >= ICE_PTP_LNK_SPD_40G)
+ val &= ~Q_REG_TX_MEM_GBL_CFG_LANE_TYPE_M;
+ else
+ val |= Q_REG_TX_MEM_GBL_CFG_LANE_TYPE_M;
+
+ err = ice_write_quad_reg_e822(hw, quad, Q_REG_TX_MEM_GBL_CFG, val);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to write back TX_MEM_GBL_CFG, err %d\n",
+ err);
+ return;
+ }
+}
+
+/**
+ * ice_phy_cfg_uix_e822 - Configure Serdes UI to TU conversion for E822
+ * @hw: pointer to the HW structure
+ * @port: the port to configure
+ *
+ * Program the conversion ration of Serdes clock "unit intervals" (UIs) to PHC
+ * hardware clock time units (TUs). That is, determine the number of TUs per
+ * serdes unit interval, and program the UIX registers with this conversion.
+ *
+ * This conversion is used as part of the calibration process when determining
+ * the additional error of a timestamp vs the real time of transmission or
+ * receipt of the packet.
+ *
+ * Hardware uses the number of TUs per 66 UIs, written to the UIX registers
+ * for the two main serdes clock rates, 10G/40G and 25G/100G serdes clocks.
+ *
+ * To calculate the conversion ratio, we use the following facts:
+ *
+ * a) the clock frequency in Hz (cycles per second)
+ * b) the number of TUs per cycle (the increment value of the clock)
+ * c) 1 second per 1 billion nanoseconds
+ * d) the duration of 66 UIs in nanoseconds
+ *
+ * Given these facts, we can use the following table to work out what ratios
+ * to multiply in order to get the number of TUs per 66 UIs:
+ *
+ * cycles | 1 second | incval (TUs) | nanoseconds
+ * -------+--------------+--------------+-------------
+ * second | 1 billion ns | cycle | 66 UIs
+ *
+ * To perform the multiplication using integers without too much loss of
+ * precision, we can take use the following equation:
+ *
+ * (freq * incval * 6600 LINE_UI ) / ( 100 * 1 billion)
+ *
+ * We scale up to using 6600 UI instead of 66 in order to avoid fractional
+ * nanosecond UIs (66 UI at 10G/40G is 6.4 ns)
+ *
+ * The increment value has a maximum expected range of about 34 bits, while
+ * the frequency value is about 29 bits. Multiplying these values shouldn't
+ * overflow the 64 bits. However, we must then further multiply them again by
+ * the Serdes unit interval duration. To avoid overflow here, we split the
+ * overall divide by 1e11 into a divide by 256 (shift down by 8 bits) and
+ * a divide by 390,625,000. This does lose some precision, but avoids
+ * miscalculation due to arithmetic overflow.
+ */
+static int ice_phy_cfg_uix_e822(struct ice_hw *hw, u8 port)
+{
+ u64 cur_freq, clk_incval, tu_per_sec, uix;
+ int err;
+
+ cur_freq = ice_e822_pll_freq(ice_e822_time_ref(hw));
+ clk_incval = ice_ptp_read_src_incval(hw);
+
+ /* Calculate TUs per second divided by 256 */
+ tu_per_sec = (cur_freq * clk_incval) >> 8;
+
+#define LINE_UI_10G_40G 640 /* 6600 UIs is 640 nanoseconds at 10Gb/40Gb */
+#define LINE_UI_25G_100G 256 /* 6600 UIs is 256 nanoseconds at 25Gb/100Gb */
+
+ /* Program the 10Gb/40Gb conversion ratio */
+ uix = div_u64(tu_per_sec * LINE_UI_10G_40G, 390625000);
+
+ err = ice_write_64b_phy_reg_e822(hw, port, P_REG_UIX66_10G_40G_L,
+ uix);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to write UIX66_10G_40G, err %d\n",
+ err);
+ return err;
+ }
+
+ /* Program the 25Gb/100Gb conversion ratio */
+ uix = div_u64(tu_per_sec * LINE_UI_25G_100G, 390625000);
+
+ err = ice_write_64b_phy_reg_e822(hw, port, P_REG_UIX66_25G_100G_L,
+ uix);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to write UIX66_25G_100G, err %d\n",
+ err);
+ return err;
+ }
+
+ return 0;
+}
+
+/**
+ * ice_phy_cfg_parpcs_e822 - Configure TUs per PAR/PCS clock cycle
+ * @hw: pointer to the HW struct
+ * @port: port to configure
+ *
+ * Configure the number of TUs for the PAR and PCS clocks used as part of the
+ * timestamp calibration process. This depends on the link speed, as the PHY
+ * uses different markers depending on the speed.
+ *
+ * 1Gb/10Gb/25Gb:
+ * - Tx/Rx PAR/PCS markers
+ *
+ * 25Gb RS:
+ * - Tx/Rx Reed Solomon gearbox PAR/PCS markers
+ *
+ * 40Gb/50Gb:
+ * - Tx/Rx PAR/PCS markers
+ * - Rx Deskew PAR/PCS markers
+ *
+ * 50G RS and 100GB RS:
+ * - Tx/Rx Reed Solomon gearbox PAR/PCS markers
+ * - Rx Deskew PAR/PCS markers
+ * - Tx PAR/PCS markers
+ *
+ * To calculate the conversion, we use the PHC clock frequency (cycles per
+ * second), the increment value (TUs per cycle), and the related PHY clock
+ * frequency to calculate the TUs per unit of the PHY link clock. The
+ * following table shows how the units convert:
+ *
+ * cycles | TUs | second
+ * -------+-------+--------
+ * second | cycle | cycles
+ *
+ * For each conversion register, look up the appropriate frequency from the
+ * e822 PAR/PCS table and calculate the TUs per unit of that clock. Program
+ * this to the appropriate register, preparing hardware to perform timestamp
+ * calibration to calculate the total Tx or Rx offset to adjust the timestamp
+ * in order to calibrate for the internal PHY delays.
+ *
+ * Note that the increment value ranges up to ~34 bits, and the clock
+ * frequency is ~29 bits, so multiplying them together should fit within the
+ * 64 bit arithmetic.
+ */
+static int ice_phy_cfg_parpcs_e822(struct ice_hw *hw, u8 port)
+{
+ u64 cur_freq, clk_incval, tu_per_sec, phy_tus;
+ enum ice_ptp_link_spd link_spd;
+ enum ice_ptp_fec_mode fec_mode;
+ int err;
+
+ err = ice_phy_get_speed_and_fec_e822(hw, port, &link_spd, &fec_mode);
+ if (err)
+ return err;
+
+ cur_freq = ice_e822_pll_freq(ice_e822_time_ref(hw));
+ clk_incval = ice_ptp_read_src_incval(hw);
+
+ /* Calculate TUs per cycle of the PHC clock */
+ tu_per_sec = cur_freq * clk_incval;
+
+ /* For each PHY conversion register, look up the appropriate link
+ * speed frequency and determine the TUs per that clock's cycle time.
+ * Split this into a high and low value and then program the
+ * appropriate register. If that link speed does not use the
+ * associated register, write zeros to clear it instead.
+ */
+
+ /* P_REG_PAR_TX_TUS */
+ if (e822_vernier[link_spd].tx_par_clk)
+ phy_tus = div_u64(tu_per_sec,
+ e822_vernier[link_spd].tx_par_clk);
+ else
+ phy_tus = 0;
+
+ err = ice_write_40b_phy_reg_e822(hw, port, P_REG_PAR_TX_TUS_L,
+ phy_tus);
+ if (err)
+ return err;
+
+ /* P_REG_PAR_RX_TUS */
+ if (e822_vernier[link_spd].rx_par_clk)
+ phy_tus = div_u64(tu_per_sec,
+ e822_vernier[link_spd].rx_par_clk);
+ else
+ phy_tus = 0;
+
+ err = ice_write_40b_phy_reg_e822(hw, port, P_REG_PAR_RX_TUS_L,
+ phy_tus);
+ if (err)
+ return err;
+
+ /* P_REG_PCS_TX_TUS */
+ if (e822_vernier[link_spd].tx_pcs_clk)
+ phy_tus = div_u64(tu_per_sec,
+ e822_vernier[link_spd].tx_pcs_clk);
+ else
+ phy_tus = 0;
+
+ err = ice_write_40b_phy_reg_e822(hw, port, P_REG_PCS_TX_TUS_L,
+ phy_tus);
+ if (err)
+ return err;
+
+ /* P_REG_PCS_RX_TUS */
+ if (e822_vernier[link_spd].rx_pcs_clk)
+ phy_tus = div_u64(tu_per_sec,
+ e822_vernier[link_spd].rx_pcs_clk);
+ else
+ phy_tus = 0;
+
+ err = ice_write_40b_phy_reg_e822(hw, port, P_REG_PCS_RX_TUS_L,
+ phy_tus);
+ if (err)
+ return err;
+
+ /* P_REG_DESK_PAR_TX_TUS */
+ if (e822_vernier[link_spd].tx_desk_rsgb_par)
+ phy_tus = div_u64(tu_per_sec,
+ e822_vernier[link_spd].tx_desk_rsgb_par);
+ else
+ phy_tus = 0;
+
+ err = ice_write_40b_phy_reg_e822(hw, port, P_REG_DESK_PAR_TX_TUS_L,
+ phy_tus);
+ if (err)
+ return err;
+
+ /* P_REG_DESK_PAR_RX_TUS */
+ if (e822_vernier[link_spd].rx_desk_rsgb_par)
+ phy_tus = div_u64(tu_per_sec,
+ e822_vernier[link_spd].rx_desk_rsgb_par);
+ else
+ phy_tus = 0;
+
+ err = ice_write_40b_phy_reg_e822(hw, port, P_REG_DESK_PAR_RX_TUS_L,
+ phy_tus);
+ if (err)
+ return err;
+
+ /* P_REG_DESK_PCS_TX_TUS */
+ if (e822_vernier[link_spd].tx_desk_rsgb_pcs)
+ phy_tus = div_u64(tu_per_sec,
+ e822_vernier[link_spd].tx_desk_rsgb_pcs);
+ else
+ phy_tus = 0;
+
+ err = ice_write_40b_phy_reg_e822(hw, port, P_REG_DESK_PCS_TX_TUS_L,
+ phy_tus);
+ if (err)
+ return err;
+
+ /* P_REG_DESK_PCS_RX_TUS */
+ if (e822_vernier[link_spd].rx_desk_rsgb_pcs)
+ phy_tus = div_u64(tu_per_sec,
+ e822_vernier[link_spd].rx_desk_rsgb_pcs);
+ else
+ phy_tus = 0;
+
+ return ice_write_40b_phy_reg_e822(hw, port, P_REG_DESK_PCS_RX_TUS_L,
+ phy_tus);
+}
+
+/**
+ * ice_calc_fixed_tx_offset_e822 - Calculated Fixed Tx offset for a port
+ * @hw: pointer to the HW struct
+ * @link_spd: the Link speed to calculate for
+ *
+ * Calculate the fixed offset due to known static latency data.
+ */
+static u64
+ice_calc_fixed_tx_offset_e822(struct ice_hw *hw, enum ice_ptp_link_spd link_spd)
+{
+ u64 cur_freq, clk_incval, tu_per_sec, fixed_offset;
+
+ cur_freq = ice_e822_pll_freq(ice_e822_time_ref(hw));
+ clk_incval = ice_ptp_read_src_incval(hw);
+
+ /* Calculate TUs per second */
+ tu_per_sec = cur_freq * clk_incval;
+
+ /* Calculate number of TUs to add for the fixed Tx latency. Since the
+ * latency measurement is in 1/100th of a nanosecond, we need to
+ * multiply by tu_per_sec and then divide by 1e11. This calculation
+ * overflows 64 bit integer arithmetic, so break it up into two
+ * divisions by 1e4 first then by 1e7.
+ */
+ fixed_offset = div_u64(tu_per_sec, 10000);
+ fixed_offset *= e822_vernier[link_spd].tx_fixed_delay;
+ fixed_offset = div_u64(fixed_offset, 10000000);
+
+ return fixed_offset;
+}
+
+/**
+ * ice_phy_cfg_tx_offset_e822 - Configure total Tx timestamp offset
+ * @hw: pointer to the HW struct
+ * @port: the PHY port to configure
+ *
+ * Program the P_REG_TOTAL_TX_OFFSET register with the total number of TUs to
+ * adjust Tx timestamps by. This is calculated by combining some known static
+ * latency along with the Vernier offset computations done by hardware.
+ *
+ * This function must be called only after the offset registers are valid,
+ * i.e. after the Vernier calibration wait has passed, to ensure that the PHY
+ * has measured the offset.
+ *
+ * To avoid overflow, when calculating the offset based on the known static
+ * latency values, we use measurements in 1/100th of a nanosecond, and divide
+ * the TUs per second up front. This avoids overflow while allowing
+ * calculation of the adjustment using integer arithmetic.
+ */
+static int ice_phy_cfg_tx_offset_e822(struct ice_hw *hw, u8 port)
+{
+ enum ice_ptp_link_spd link_spd;
+ enum ice_ptp_fec_mode fec_mode;
+ u64 total_offset, val;
+ int err;
+
+ err = ice_phy_get_speed_and_fec_e822(hw, port, &link_spd, &fec_mode);
+ if (err)
+ return err;
+
+ total_offset = ice_calc_fixed_tx_offset_e822(hw, link_spd);
+
+ /* Read the first Vernier offset from the PHY register and add it to
+ * the total offset.
+ */
+ if (link_spd == ICE_PTP_LNK_SPD_1G ||
+ link_spd == ICE_PTP_LNK_SPD_10G ||
+ link_spd == ICE_PTP_LNK_SPD_25G ||
+ link_spd == ICE_PTP_LNK_SPD_25G_RS ||
+ link_spd == ICE_PTP_LNK_SPD_40G ||
+ link_spd == ICE_PTP_LNK_SPD_50G) {
+ err = ice_read_64b_phy_reg_e822(hw, port,
+ P_REG_PAR_PCS_TX_OFFSET_L,
+ &val);
+ if (err)
+ return err;
+
+ total_offset += val;
+ }
+
+ /* For Tx, we only need to use the second Vernier offset for
+ * multi-lane link speeds with RS-FEC. The lanes will always be
+ * aligned.
+ */
+ if (link_spd == ICE_PTP_LNK_SPD_50G_RS ||
+ link_spd == ICE_PTP_LNK_SPD_100G_RS) {
+ err = ice_read_64b_phy_reg_e822(hw, port,
+ P_REG_PAR_TX_TIME_L,
+ &val);
+ if (err)
+ return err;
+
+ total_offset += val;
+ }
+
+ /* Now that the total offset has been calculated, program it to the
+ * PHY and indicate that the Tx offset is ready. After this,
+ * timestamps will be enabled.
+ */
+ err = ice_write_64b_phy_reg_e822(hw, port, P_REG_TOTAL_TX_OFFSET_L,
+ total_offset);
+ if (err)
+ return err;
+
+ err = ice_write_phy_reg_e822(hw, port, P_REG_TX_OR, 1);
+ if (err)
+ return err;
+
+ return 0;
+}
+
+/**
+ * ice_phy_cfg_fixed_tx_offset_e822 - Configure Tx offset for bypass mode
+ * @hw: pointer to the HW struct
+ * @port: the PHY port to configure
+ *
+ * Calculate and program the fixed Tx offset, and indicate that the offset is
+ * ready. This can be used when operating in bypass mode.
+ */
+static int
+ice_phy_cfg_fixed_tx_offset_e822(struct ice_hw *hw, u8 port)
+{
+ enum ice_ptp_link_spd link_spd;
+ enum ice_ptp_fec_mode fec_mode;
+ u64 total_offset;
+ int err;
+
+ err = ice_phy_get_speed_and_fec_e822(hw, port, &link_spd, &fec_mode);
+ if (err)
+ return err;
+
+ total_offset = ice_calc_fixed_tx_offset_e822(hw, link_spd);
+
+ /* Program the fixed Tx offset into the P_REG_TOTAL_TX_OFFSET_L
+ * register, then indicate that the Tx offset is ready. After this,
+ * timestamps will be enabled.
+ *
+ * Note that this skips including the more precise offsets generated
+ * by the Vernier calibration.
+ */
+ err = ice_write_64b_phy_reg_e822(hw, port, P_REG_TOTAL_TX_OFFSET_L,
+ total_offset);
+ if (err)
+ return err;
+
+ err = ice_write_phy_reg_e822(hw, port, P_REG_TX_OR, 1);
+ if (err)
+ return err;
+
+ return 0;
+}
+
+/**
+ * ice_phy_calc_pmd_adj_e822 - Calculate PMD adjustment for Rx
+ * @hw: pointer to the HW struct
+ * @port: the PHY port to adjust for
+ * @link_spd: the current link speed of the PHY
+ * @fec_mode: the current FEC mode of the PHY
+ * @pmd_adj: on return, the amount to adjust the Rx total offset by
+ *
+ * Calculates the adjustment to Rx timestamps due to PMD alignment in the PHY.
+ * This varies by link speed and FEC mode. The value calculated accounts for
+ * various delays caused when receiving a packet.
+ */
+static int
+ice_phy_calc_pmd_adj_e822(struct ice_hw *hw, u8 port,
+ enum ice_ptp_link_spd link_spd,
+ enum ice_ptp_fec_mode fec_mode, u64 *pmd_adj)
+{
+ u64 cur_freq, clk_incval, tu_per_sec, mult, adj;
+ u8 pmd_align;
+ u32 val;
+ int err;
+
+ err = ice_read_phy_reg_e822(hw, port, P_REG_PMD_ALIGNMENT, &val);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to read PMD alignment, err %d\n",
+ err);
+ return err;
+ }
+
+ pmd_align = (u8)val;
+
+ cur_freq = ice_e822_pll_freq(ice_e822_time_ref(hw));
+ clk_incval = ice_ptp_read_src_incval(hw);
+
+ /* Calculate TUs per second */
+ tu_per_sec = cur_freq * clk_incval;
+
+ /* The PMD alignment adjustment measurement depends on the link speed,
+ * and whether FEC is enabled. For each link speed, the alignment
+ * adjustment is calculated by dividing a value by the length of
+ * a Time Unit in nanoseconds.
+ *
+ * 1G: align == 4 ? 10 * 0.8 : (align + 6 % 10) * 0.8
+ * 10G: align == 65 ? 0 : (align * 0.1 * 32/33)
+ * 10G w/FEC: align * 0.1 * 32/33
+ * 25G: align == 65 ? 0 : (align * 0.4 * 32/33)
+ * 25G w/FEC: align * 0.4 * 32/33
+ * 40G: align == 65 ? 0 : (align * 0.1 * 32/33)
+ * 40G w/FEC: align * 0.1 * 32/33
+ * 50G: align == 65 ? 0 : (align * 0.4 * 32/33)
+ * 50G w/FEC: align * 0.8 * 32/33
+ *
+ * For RS-FEC, if align is < 17 then we must also add 1.6 * 32/33.
+ *
+ * To allow for calculating this value using integer arithmetic, we
+ * instead start with the number of TUs per second, (inverse of the
+ * length of a Time Unit in nanoseconds), multiply by a value based
+ * on the PMD alignment register, and then divide by the right value
+ * calculated based on the table above. To avoid integer overflow this
+ * division is broken up into a step of dividing by 125 first.
+ */
+ if (link_spd == ICE_PTP_LNK_SPD_1G) {
+ if (pmd_align == 4)
+ mult = 10;
+ else
+ mult = (pmd_align + 6) % 10;
+ } else if (link_spd == ICE_PTP_LNK_SPD_10G ||
+ link_spd == ICE_PTP_LNK_SPD_25G ||
+ link_spd == ICE_PTP_LNK_SPD_40G ||
+ link_spd == ICE_PTP_LNK_SPD_50G) {
+ /* If Clause 74 FEC, always calculate PMD adjust */
+ if (pmd_align != 65 || fec_mode == ICE_PTP_FEC_MODE_CLAUSE74)
+ mult = pmd_align;
+ else
+ mult = 0;
+ } else if (link_spd == ICE_PTP_LNK_SPD_25G_RS ||
+ link_spd == ICE_PTP_LNK_SPD_50G_RS ||
+ link_spd == ICE_PTP_LNK_SPD_100G_RS) {
+ if (pmd_align < 17)
+ mult = pmd_align + 40;
+ else
+ mult = pmd_align;
+ } else {
+ ice_debug(hw, ICE_DBG_PTP, "Unknown link speed %d, skipping PMD adjustment\n",
+ link_spd);
+ mult = 0;
+ }
+
+ /* In some cases, there's no need to adjust for the PMD alignment */
+ if (!mult) {
+ *pmd_adj = 0;
+ return 0;
+ }
+
+ /* Calculate the adjustment by multiplying TUs per second by the
+ * appropriate multiplier and divisor. To avoid overflow, we first
+ * divide by 125, and then handle remaining divisor based on the link
+ * speed pmd_adj_divisor value.
+ */
+ adj = div_u64(tu_per_sec, 125);
+ adj *= mult;
+ adj = div_u64(adj, e822_vernier[link_spd].pmd_adj_divisor);
+
+ /* Finally, for 25G-RS and 50G-RS, a further adjustment for the Rx
+ * cycle count is necessary.
+ */
+ if (link_spd == ICE_PTP_LNK_SPD_25G_RS) {
+ u64 cycle_adj;
+ u8 rx_cycle;
+
+ err = ice_read_phy_reg_e822(hw, port, P_REG_RX_40_TO_160_CNT,
+ &val);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to read 25G-RS Rx cycle count, err %d\n",
+ err);
+ return err;
+ }
+
+ rx_cycle = val & P_REG_RX_40_TO_160_CNT_RXCYC_M;
+ if (rx_cycle) {
+ mult = (4 - rx_cycle) * 40;
+
+ cycle_adj = div_u64(tu_per_sec, 125);
+ cycle_adj *= mult;
+ cycle_adj = div_u64(cycle_adj, e822_vernier[link_spd].pmd_adj_divisor);
+
+ adj += cycle_adj;
+ }
+ } else if (link_spd == ICE_PTP_LNK_SPD_50G_RS) {
+ u64 cycle_adj;
+ u8 rx_cycle;
+
+ err = ice_read_phy_reg_e822(hw, port, P_REG_RX_80_TO_160_CNT,
+ &val);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to read 50G-RS Rx cycle count, err %d\n",
+ err);
+ return err;
+ }
+
+ rx_cycle = val & P_REG_RX_80_TO_160_CNT_RXCYC_M;
+ if (rx_cycle) {
+ mult = rx_cycle * 40;
+
+ cycle_adj = div_u64(tu_per_sec, 125);
+ cycle_adj *= mult;
+ cycle_adj = div_u64(cycle_adj, e822_vernier[link_spd].pmd_adj_divisor);
+
+ adj += cycle_adj;
+ }
+ }
+
+ /* Return the calculated adjustment */
+ *pmd_adj = adj;
+
+ return 0;
+}
+
+/**
+ * ice_calc_fixed_rx_offset_e822 - Calculated the fixed Rx offset for a port
+ * @hw: pointer to HW struct
+ * @link_spd: The Link speed to calculate for
+ *
+ * Determine the fixed Rx latency for a given link speed.
+ */
+static u64
+ice_calc_fixed_rx_offset_e822(struct ice_hw *hw, enum ice_ptp_link_spd link_spd)
+{
+ u64 cur_freq, clk_incval, tu_per_sec, fixed_offset;
+
+ cur_freq = ice_e822_pll_freq(ice_e822_time_ref(hw));
+ clk_incval = ice_ptp_read_src_incval(hw);
+
+ /* Calculate TUs per second */
+ tu_per_sec = cur_freq * clk_incval;
+
+ /* Calculate number of TUs to add for the fixed Rx latency. Since the
+ * latency measurement is in 1/100th of a nanosecond, we need to
+ * multiply by tu_per_sec and then divide by 1e11. This calculation
+ * overflows 64 bit integer arithmetic, so break it up into two
+ * divisions by 1e4 first then by 1e7.
+ */
+ fixed_offset = div_u64(tu_per_sec, 10000);
+ fixed_offset *= e822_vernier[link_spd].rx_fixed_delay;
+ fixed_offset = div_u64(fixed_offset, 10000000);
+
+ return fixed_offset;
+}
+
+/**
+ * ice_phy_cfg_rx_offset_e822 - Configure total Rx timestamp offset
+ * @hw: pointer to the HW struct
+ * @port: the PHY port to configure
+ *
+ * Program the P_REG_TOTAL_RX_OFFSET register with the number of Time Units to
+ * adjust Rx timestamps by. This combines calculations from the Vernier offset
+ * measurements taken in hardware with some data about known fixed delay as
+ * well as adjusting for multi-lane alignment delay.
+ *
+ * This function must be called only after the offset registers are valid,
+ * i.e. after the Vernier calibration wait has passed, to ensure that the PHY
+ * has measured the offset.
+ *
+ * To avoid overflow, when calculating the offset based on the known static
+ * latency values, we use measurements in 1/100th of a nanosecond, and divide
+ * the TUs per second up front. This avoids overflow while allowing
+ * calculation of the adjustment using integer arithmetic.
+ */
+static int ice_phy_cfg_rx_offset_e822(struct ice_hw *hw, u8 port)
+{
+ enum ice_ptp_link_spd link_spd;
+ enum ice_ptp_fec_mode fec_mode;
+ u64 total_offset, pmd, val;
+ int err;
+
+ err = ice_phy_get_speed_and_fec_e822(hw, port, &link_spd, &fec_mode);
+ if (err)
+ return err;
+
+ total_offset = ice_calc_fixed_rx_offset_e822(hw, link_spd);
+
+ /* Read the first Vernier offset from the PHY register and add it to
+ * the total offset.
+ */
+ err = ice_read_64b_phy_reg_e822(hw, port,
+ P_REG_PAR_PCS_RX_OFFSET_L,
+ &val);
+ if (err)
+ return err;
+
+ total_offset += val;
+
+ /* For Rx, all multi-lane link speeds include a second Vernier
+ * calibration, because the lanes might not be aligned.
+ */
+ if (link_spd == ICE_PTP_LNK_SPD_40G ||
+ link_spd == ICE_PTP_LNK_SPD_50G ||
+ link_spd == ICE_PTP_LNK_SPD_50G_RS ||
+ link_spd == ICE_PTP_LNK_SPD_100G_RS) {
+ err = ice_read_64b_phy_reg_e822(hw, port,
+ P_REG_PAR_RX_TIME_L,
+ &val);
+ if (err)
+ return err;
+
+ total_offset += val;
+ }
+
+ /* In addition, Rx must account for the PMD alignment */
+ err = ice_phy_calc_pmd_adj_e822(hw, port, link_spd, fec_mode, &pmd);
+ if (err)
+ return err;
+
+ /* For RS-FEC, this adjustment adds delay, but for other modes, it
+ * subtracts delay.
+ */
+ if (fec_mode == ICE_PTP_FEC_MODE_RS_FEC)
+ total_offset += pmd;
+ else
+ total_offset -= pmd;
+
+ /* Now that the total offset has been calculated, program it to the
+ * PHY and indicate that the Rx offset is ready. After this,
+ * timestamps will be enabled.
+ */
+ err = ice_write_64b_phy_reg_e822(hw, port, P_REG_TOTAL_RX_OFFSET_L,
+ total_offset);
+ if (err)
+ return err;
+
+ err = ice_write_phy_reg_e822(hw, port, P_REG_RX_OR, 1);
+ if (err)
+ return err;
+
+ return 0;
+}
+
+/**
+ * ice_phy_cfg_fixed_rx_offset_e822 - Configure fixed Rx offset for bypass mode
+ * @hw: pointer to the HW struct
+ * @port: the PHY port to configure
+ *
+ * Calculate and program the fixed Rx offset, and indicate that the offset is
+ * ready. This can be used when operating in bypass mode.
+ */
+static int
+ice_phy_cfg_fixed_rx_offset_e822(struct ice_hw *hw, u8 port)
+{
+ enum ice_ptp_link_spd link_spd;
+ enum ice_ptp_fec_mode fec_mode;
+ u64 total_offset;
+ int err;
+
+ err = ice_phy_get_speed_and_fec_e822(hw, port, &link_spd, &fec_mode);
+ if (err)
+ return err;
+
+ total_offset = ice_calc_fixed_rx_offset_e822(hw, link_spd);
+
+ /* Program the fixed Rx offset into the P_REG_TOTAL_RX_OFFSET_L
+ * register, then indicate that the Rx offset is ready. After this,
+ * timestamps will be enabled.
+ *
+ * Note that this skips including the more precise offsets generated
+ * by Vernier calibration.
+ */
+ err = ice_write_64b_phy_reg_e822(hw, port, P_REG_TOTAL_RX_OFFSET_L,
+ total_offset);
+ if (err)
+ return err;
+
+ err = ice_write_phy_reg_e822(hw, port, P_REG_RX_OR, 1);
+ if (err)
+ return err;
+
+ return 0;
+}
+
+/**
+ * ice_read_phy_and_phc_time_e822 - Simultaneously capture PHC and PHY time
+ * @hw: pointer to the HW struct
+ * @port: the PHY port to read
+ * @phy_time: on return, the 64bit PHY timer value
+ * @phc_time: on return, the lower 64bits of PHC time
+ *
+ * Issue a READ_TIME timer command to simultaneously capture the PHY and PHC
+ * timer values.
+ */
+static int
+ice_read_phy_and_phc_time_e822(struct ice_hw *hw, u8 port, u64 *phy_time,
+ u64 *phc_time)
+{
+ u64 tx_time, rx_time;
+ u32 zo, lo;
+ u8 tmr_idx;
+ int err;
+
+ tmr_idx = ice_get_ptp_src_clock_index(hw);
+
+ /* Prepare the PHC timer for a READ_TIME capture command */
+ ice_ptp_src_cmd(hw, READ_TIME);
+
+ /* Prepare the PHY timer for a READ_TIME capture command */
+ err = ice_ptp_one_port_cmd(hw, port, READ_TIME);
+ if (err)
+ return err;
+
+ /* Issue the sync to start the READ_TIME capture */
+ ice_ptp_exec_tmr_cmd(hw);
+
+ /* Read the captured PHC time from the shadow time registers */
+ zo = rd32(hw, GLTSYN_SHTIME_0(tmr_idx));
+ lo = rd32(hw, GLTSYN_SHTIME_L(tmr_idx));
+ *phc_time = (u64)lo << 32 | zo;
+
+ /* Read the captured PHY time from the PHY shadow registers */
+ err = ice_ptp_read_port_capture(hw, port, &tx_time, &rx_time);
+ if (err)
+ return err;
+
+ /* If the PHY Tx and Rx timers don't match, log a warning message.
+ * Note that this should not happen in normal circumstances since the
+ * driver always programs them together.
+ */
+ if (tx_time != rx_time)
+ dev_warn(ice_hw_to_dev(hw),
+ "PHY port %u Tx and Rx timers do not match, tx_time 0x%016llX, rx_time 0x%016llX\n",
+ port, (unsigned long long)tx_time,
+ (unsigned long long)rx_time);
+
+ *phy_time = tx_time;
+
+ return 0;
+}
+
+/**
+ * ice_sync_phy_timer_e822 - Synchronize the PHY timer with PHC timer
+ * @hw: pointer to the HW struct
+ * @port: the PHY port to synchronize
+ *
+ * Perform an adjustment to ensure that the PHY and PHC timers are in sync.
+ * This is done by issuing a READ_TIME command which triggers a simultaneous
+ * read of the PHY timer and PHC timer. Then we use the difference to
+ * calculate an appropriate 2s complement addition to add to the PHY timer in
+ * order to ensure it reads the same value as the primary PHC timer.
+ */
+static int ice_sync_phy_timer_e822(struct ice_hw *hw, u8 port)
+{
+ u64 phc_time, phy_time, difference;
+ int err;
+
+ if (!ice_ptp_lock(hw)) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to acquire PTP semaphore\n");
+ return -EBUSY;
+ }
+
+ err = ice_read_phy_and_phc_time_e822(hw, port, &phy_time, &phc_time);
+ if (err)
+ goto err_unlock;
+
+ /* Calculate the amount required to add to the port time in order for
+ * it to match the PHC time.
+ *
+ * Note that the port adjustment is done using 2s complement
+ * arithmetic. This is convenient since it means that we can simply
+ * calculate the difference between the PHC time and the port time,
+ * and it will be interpreted correctly.
+ */
+ difference = phc_time - phy_time;
+
+ err = ice_ptp_prep_port_adj_e822(hw, port, (s64)difference);
+ if (err)
+ goto err_unlock;
+
+ err = ice_ptp_one_port_cmd(hw, port, ADJ_TIME);
+ if (err)
+ goto err_unlock;
+
+ /* Issue the sync to activate the time adjustment */
+ ice_ptp_exec_tmr_cmd(hw);
+
+ /* Re-capture the timer values to flush the command registers and
+ * verify that the time was properly adjusted.
+ */
+ err = ice_read_phy_and_phc_time_e822(hw, port, &phy_time, &phc_time);
+ if (err)
+ goto err_unlock;
+
+ dev_info(ice_hw_to_dev(hw),
+ "Port %u PHY time synced to PHC: 0x%016llX, 0x%016llX\n",
+ port, (unsigned long long)phy_time,
+ (unsigned long long)phc_time);
+
+ ice_ptp_unlock(hw);
+
+ return 0;
+
+err_unlock:
+ ice_ptp_unlock(hw);
+ return err;
+}
+
+/**
+ * ice_stop_phy_timer_e822 - Stop the PHY clock timer
+ * @hw: pointer to the HW struct
+ * @port: the PHY port to stop
+ * @soft_reset: if true, hold the SOFT_RESET bit of P_REG_PS
+ *
+ * Stop the clock of a PHY port. This must be done as part of the flow to
+ * re-calibrate Tx and Rx timestamping offsets whenever the clock time is
+ * initialized or when link speed changes.
+ */
+int
+ice_stop_phy_timer_e822(struct ice_hw *hw, u8 port, bool soft_reset)
+{
+ int err;
+ u32 val;
+
+ err = ice_write_phy_reg_e822(hw, port, P_REG_TX_OR, 0);
+ if (err)
+ return err;
+
+ err = ice_write_phy_reg_e822(hw, port, P_REG_RX_OR, 0);
+ if (err)
+ return err;
+
+ err = ice_read_phy_reg_e822(hw, port, P_REG_PS, &val);
+ if (err)
+ return err;
+
+ val &= ~P_REG_PS_START_M;
+ err = ice_write_phy_reg_e822(hw, port, P_REG_PS, val);
+ if (err)
+ return err;
+
+ val &= ~P_REG_PS_ENA_CLK_M;
+ err = ice_write_phy_reg_e822(hw, port, P_REG_PS, val);
+ if (err)
+ return err;
+
+ if (soft_reset) {
+ val |= P_REG_PS_SFT_RESET_M;
+ err = ice_write_phy_reg_e822(hw, port, P_REG_PS, val);
+ if (err)
+ return err;
+ }
+
+ ice_debug(hw, ICE_DBG_PTP, "Disabled clock on PHY port %u\n", port);
+
+ return 0;
+}
+
+/**
+ * ice_start_phy_timer_e822 - Start the PHY clock timer
+ * @hw: pointer to the HW struct
+ * @port: the PHY port to start
+ * @bypass: if true, start the PHY in bypass mode
+ *
+ * Start the clock of a PHY port. This must be done as part of the flow to
+ * re-calibrate Tx and Rx timestamping offsets whenever the clock time is
+ * initialized or when link speed changes.
+ *
+ * Bypass mode enables timestamps immediately without waiting for Vernier
+ * calibration to complete. Hardware will still continue taking Vernier
+ * measurements on Tx or Rx of packets, but they will not be applied to
+ * timestamps. Use ice_phy_exit_bypass_e822 to exit bypass mode once hardware
+ * has completed offset calculation.
+ */
+int
+ice_start_phy_timer_e822(struct ice_hw *hw, u8 port, bool bypass)
+{
+ u32 lo, hi, val;
+ u64 incval;
+ u8 tmr_idx;
+ int err;
+
+ tmr_idx = ice_get_ptp_src_clock_index(hw);
+
+ err = ice_stop_phy_timer_e822(hw, port, false);
+ if (err)
+ return err;
+
+ ice_phy_cfg_lane_e822(hw, port);
+
+ err = ice_phy_cfg_uix_e822(hw, port);
+ if (err)
+ return err;
+
+ err = ice_phy_cfg_parpcs_e822(hw, port);
+ if (err)
+ return err;
+
+ lo = rd32(hw, GLTSYN_INCVAL_L(tmr_idx));
+ hi = rd32(hw, GLTSYN_INCVAL_H(tmr_idx));
+ incval = (u64)hi << 32 | lo;
+
+ err = ice_write_40b_phy_reg_e822(hw, port, P_REG_TIMETUS_L, incval);
+ if (err)
+ return err;
+
+ err = ice_ptp_one_port_cmd(hw, port, INIT_INCVAL);
+ if (err)
+ return err;
+
+ ice_ptp_exec_tmr_cmd(hw);
+
+ err = ice_read_phy_reg_e822(hw, port, P_REG_PS, &val);
+ if (err)
+ return err;
+
+ val |= P_REG_PS_SFT_RESET_M;
+ err = ice_write_phy_reg_e822(hw, port, P_REG_PS, val);
+ if (err)
+ return err;
+
+ val |= P_REG_PS_START_M;
+ err = ice_write_phy_reg_e822(hw, port, P_REG_PS, val);
+ if (err)
+ return err;
+
+ val &= ~P_REG_PS_SFT_RESET_M;
+ err = ice_write_phy_reg_e822(hw, port, P_REG_PS, val);
+ if (err)
+ return err;
+
+ err = ice_ptp_one_port_cmd(hw, port, INIT_INCVAL);
+ if (err)
+ return err;
+
+ ice_ptp_exec_tmr_cmd(hw);
+
+ val |= P_REG_PS_ENA_CLK_M;
+ err = ice_write_phy_reg_e822(hw, port, P_REG_PS, val);
+ if (err)
+ return err;
+
+ val |= P_REG_PS_LOAD_OFFSET_M;
+ err = ice_write_phy_reg_e822(hw, port, P_REG_PS, val);
+ if (err)
+ return err;
+
+ ice_ptp_exec_tmr_cmd(hw);
+
+ err = ice_sync_phy_timer_e822(hw, port);
+ if (err)
+ return err;
+
+ if (bypass) {
+ val |= P_REG_PS_BYPASS_MODE_M;
+ /* Enter BYPASS mode, enabling timestamps immediately. */
+ err = ice_write_phy_reg_e822(hw, port, P_REG_PS, val);
+ if (err)
+ return err;
+
+ /* Program the fixed Tx offset */
+ err = ice_phy_cfg_fixed_tx_offset_e822(hw, port);
+ if (err)
+ return err;
+
+ /* Program the fixed Rx offset */
+ err = ice_phy_cfg_fixed_rx_offset_e822(hw, port);
+ if (err)
+ return err;
+ }
+
+ ice_debug(hw, ICE_DBG_PTP, "Enabled clock on PHY port %u\n", port);
+
+ return 0;
+}
+
+/**
+ * ice_phy_exit_bypass_e822 - Exit bypass mode, after vernier calculations
+ * @hw: pointer to the HW struct
+ * @port: the PHY port to configure
+ *
+ * After hardware finishes vernier calculations for the Tx and Rx offset, this
+ * function can be used to exit bypass mode by updating the total Tx and Rx
+ * offsets, and then disabling bypass. This will enable hardware to include
+ * the more precise offset calibrations, increasing precision of the generated
+ * timestamps.
+ *
+ * This cannot be done until hardware has measured the offsets, which requires
+ * waiting until at least one packet has been sent and received by the device.
+ */
+int ice_phy_exit_bypass_e822(struct ice_hw *hw, u8 port)
+{
+ int err;
+ u32 val;
+
+ err = ice_read_phy_reg_e822(hw, port, P_REG_TX_OV_STATUS, &val);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to read TX_OV_STATUS for port %u, err %d\n",
+ port, err);
+ return err;
+ }
+
+ if (!(val & P_REG_TX_OV_STATUS_OV_M)) {
+ ice_debug(hw, ICE_DBG_PTP, "Tx offset is not yet valid for port %u\n",
+ port);
+ return -EBUSY;
+ }
+
+ err = ice_read_phy_reg_e822(hw, port, P_REG_RX_OV_STATUS, &val);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to read RX_OV_STATUS for port %u, err %d\n",
+ port, err);
+ return err;
+ }
+
+ if (!(val & P_REG_TX_OV_STATUS_OV_M)) {
+ ice_debug(hw, ICE_DBG_PTP, "Rx offset is not yet valid for port %u\n",
+ port);
+ return -EBUSY;
+ }
+
+ err = ice_phy_cfg_tx_offset_e822(hw, port);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to program total Tx offset for port %u, err %d\n",
+ port, err);
+ return err;
+ }
+
+ err = ice_phy_cfg_rx_offset_e822(hw, port);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to program total Rx offset for port %u, err %d\n",
+ port, err);
+ return err;
+ }
+
+ /* Exit bypass mode now that the offset has been updated */
+ err = ice_read_phy_reg_e822(hw, port, P_REG_PS, &val);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to read P_REG_PS for port %u, err %d\n",
+ port, err);
+ return err;
+ }
+
+ if (!(val & P_REG_PS_BYPASS_MODE_M))
+ ice_debug(hw, ICE_DBG_PTP, "Port %u not in bypass mode\n",
+ port);
+
+ val &= ~P_REG_PS_BYPASS_MODE_M;
+ err = ice_write_phy_reg_e822(hw, port, P_REG_PS, val);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to disable bypass for port %u, err %d\n",
+ port, err);
+ return err;
+ }
+
+ dev_info(ice_hw_to_dev(hw), "Exiting bypass mode on PHY port %u\n",
+ port);
+
+ return 0;
+}
+
/* E810 functions
*
* The following functions operate on the E810 series devices which use
@@ -68,18 +2538,18 @@ u8 ice_get_ptp_src_clock_index(struct ice_hw *hw)
static int ice_read_phy_reg_e810(struct ice_hw *hw, u32 addr, u32 *val)
{
struct ice_sbq_msg_input msg = {0};
- int status;
+ int err;
msg.msg_addr_low = lower_16_bits(addr);
msg.msg_addr_high = upper_16_bits(addr);
msg.opcode = ice_sbq_msg_rd;
msg.dest_dev = rmn_0;
- status = ice_sbq_rw_reg(hw, &msg);
- if (status) {
- ice_debug(hw, ICE_DBG_PTP, "Failed to send message to PHY, status %d\n",
- status);
- return status;
+ err = ice_sbq_rw_reg(hw, &msg);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to send message to PHY, err %d\n",
+ err);
+ return err;
}
*val = msg.data;
@@ -98,7 +2568,7 @@ static int ice_read_phy_reg_e810(struct ice_hw *hw, u32 addr, u32 *val)
static int ice_write_phy_reg_e810(struct ice_hw *hw, u32 addr, u32 val)
{
struct ice_sbq_msg_input msg = {0};
- int status;
+ int err;
msg.msg_addr_low = lower_16_bits(addr);
msg.msg_addr_high = upper_16_bits(addr);
@@ -106,11 +2576,11 @@ static int ice_write_phy_reg_e810(struct ice_hw *hw, u32 addr, u32 val)
msg.dest_dev = rmn_0;
msg.data = val;
- status = ice_sbq_rw_reg(hw, &msg);
- if (status) {
- ice_debug(hw, ICE_DBG_PTP, "Failed to send message to PHY, status %d\n",
- status);
- return status;
+ err = ice_sbq_rw_reg(hw, &msg);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to send message to PHY, err %d\n",
+ err);
+ return err;
}
return 0;
@@ -130,23 +2600,23 @@ static int
ice_read_phy_tstamp_e810(struct ice_hw *hw, u8 lport, u8 idx, u64 *tstamp)
{
u32 lo_addr, hi_addr, lo, hi;
- int status;
+ int err;
lo_addr = TS_EXT(LOW_TX_MEMORY_BANK_START, lport, idx);
hi_addr = TS_EXT(HIGH_TX_MEMORY_BANK_START, lport, idx);
- status = ice_read_phy_reg_e810(hw, lo_addr, &lo);
- if (status) {
- ice_debug(hw, ICE_DBG_PTP, "Failed to read low PTP timestamp register, status %d\n",
- status);
- return status;
+ err = ice_read_phy_reg_e810(hw, lo_addr, &lo);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to read low PTP timestamp register, err %d\n",
+ err);
+ return err;
}
- status = ice_read_phy_reg_e810(hw, hi_addr, &hi);
- if (status) {
- ice_debug(hw, ICE_DBG_PTP, "Failed to read high PTP timestamp register, status %d\n",
- status);
- return status;
+ err = ice_read_phy_reg_e810(hw, hi_addr, &hi);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to read high PTP timestamp register, err %d\n",
+ err);
+ return err;
}
/* For E810 devices, the timestamp is reported with the lower 32 bits
@@ -169,23 +2639,23 @@ ice_read_phy_tstamp_e810(struct ice_hw *hw, u8 lport, u8 idx, u64 *tstamp)
static int ice_clear_phy_tstamp_e810(struct ice_hw *hw, u8 lport, u8 idx)
{
u32 lo_addr, hi_addr;
- int status;
+ int err;
lo_addr = TS_EXT(LOW_TX_MEMORY_BANK_START, lport, idx);
hi_addr = TS_EXT(HIGH_TX_MEMORY_BANK_START, lport, idx);
- status = ice_write_phy_reg_e810(hw, lo_addr, 0);
- if (status) {
- ice_debug(hw, ICE_DBG_PTP, "Failed to clear low PTP timestamp register, status %d\n",
- status);
- return status;
+ err = ice_write_phy_reg_e810(hw, lo_addr, 0);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to clear low PTP timestamp register, err %d\n",
+ err);
+ return err;
}
- status = ice_write_phy_reg_e810(hw, hi_addr, 0);
- if (status) {
- ice_debug(hw, ICE_DBG_PTP, "Failed to clear high PTP timestamp register, status %d\n",
- status);
- return status;
+ err = ice_write_phy_reg_e810(hw, hi_addr, 0);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to clear high PTP timestamp register, err %d\n",
+ err);
+ return err;
}
return 0;
@@ -200,17 +2670,32 @@ static int ice_clear_phy_tstamp_e810(struct ice_hw *hw, u8 lport, u8 idx)
*/
int ice_ptp_init_phy_e810(struct ice_hw *hw)
{
- int status;
u8 tmr_idx;
+ int err;
tmr_idx = hw->func_caps.ts_func_info.tmr_index_owned;
- status = ice_write_phy_reg_e810(hw, ETH_GLTSYN_ENA(tmr_idx),
- GLTSYN_ENA_TSYN_ENA_M);
- if (status)
+ err = ice_write_phy_reg_e810(hw, ETH_GLTSYN_ENA(tmr_idx),
+ GLTSYN_ENA_TSYN_ENA_M);
+ if (err)
ice_debug(hw, ICE_DBG_PTP, "PTP failed in ena_phy_time_syn %d\n",
- status);
+ err);
- return status;
+ return err;
+}
+
+/**
+ * ice_ptp_init_phc_e810 - Perform E810 specific PHC initialization
+ * @hw: pointer to HW struct
+ *
+ * Perform E810-specific PTP hardware clock initialization steps.
+ */
+static int ice_ptp_init_phc_e810(struct ice_hw *hw)
+{
+ /* Ensure synchronization delay is zero */
+ wr32(hw, GLTSYN_SYNC_DLAY, 0);
+
+ /* Initialize the PHY */
+ return ice_ptp_init_phy_e810(hw);
}
/**
@@ -227,22 +2712,22 @@ int ice_ptp_init_phy_e810(struct ice_hw *hw)
*/
static int ice_ptp_prep_phy_time_e810(struct ice_hw *hw, u32 time)
{
- int status;
u8 tmr_idx;
+ int err;
tmr_idx = hw->func_caps.ts_func_info.tmr_index_owned;
- status = ice_write_phy_reg_e810(hw, ETH_GLTSYN_SHTIME_0(tmr_idx), 0);
- if (status) {
- ice_debug(hw, ICE_DBG_PTP, "Failed to write SHTIME_0, status %d\n",
- status);
- return status;
+ err = ice_write_phy_reg_e810(hw, ETH_GLTSYN_SHTIME_0(tmr_idx), 0);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to write SHTIME_0, err %d\n",
+ err);
+ return err;
}
- status = ice_write_phy_reg_e810(hw, ETH_GLTSYN_SHTIME_L(tmr_idx), time);
- if (status) {
- ice_debug(hw, ICE_DBG_PTP, "Failed to write SHTIME_L, status %d\n",
- status);
- return status;
+ err = ice_write_phy_reg_e810(hw, ETH_GLTSYN_SHTIME_L(tmr_idx), time);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to write SHTIME_L, err %d\n",
+ err);
+ return err;
}
return 0;
@@ -263,26 +2748,26 @@ static int ice_ptp_prep_phy_time_e810(struct ice_hw *hw, u32 time)
*/
static int ice_ptp_prep_phy_adj_e810(struct ice_hw *hw, s32 adj)
{
- int status;
u8 tmr_idx;
+ int err;
tmr_idx = hw->func_caps.ts_func_info.tmr_index_owned;
/* Adjustments are represented as signed 2's complement values in
* nanoseconds. Sub-nanosecond adjustment is not supported.
*/
- status = ice_write_phy_reg_e810(hw, ETH_GLTSYN_SHADJ_L(tmr_idx), 0);
- if (status) {
- ice_debug(hw, ICE_DBG_PTP, "Failed to write adj to PHY SHADJ_L, status %d\n",
- status);
- return status;
+ err = ice_write_phy_reg_e810(hw, ETH_GLTSYN_SHADJ_L(tmr_idx), 0);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to write adj to PHY SHADJ_L, err %d\n",
+ err);
+ return err;
}
- status = ice_write_phy_reg_e810(hw, ETH_GLTSYN_SHADJ_H(tmr_idx), adj);
- if (status) {
- ice_debug(hw, ICE_DBG_PTP, "Failed to write adj to PHY SHADJ_H, status %d\n",
- status);
- return status;
+ err = ice_write_phy_reg_e810(hw, ETH_GLTSYN_SHADJ_H(tmr_idx), adj);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to write adj to PHY SHADJ_H, err %d\n",
+ err);
+ return err;
}
return 0;
@@ -300,25 +2785,25 @@ static int ice_ptp_prep_phy_adj_e810(struct ice_hw *hw, s32 adj)
static int ice_ptp_prep_phy_incval_e810(struct ice_hw *hw, u64 incval)
{
u32 high, low;
- int status;
u8 tmr_idx;
+ int err;
tmr_idx = hw->func_caps.ts_func_info.tmr_index_owned;
low = lower_32_bits(incval);
high = upper_32_bits(incval);
- status = ice_write_phy_reg_e810(hw, ETH_GLTSYN_SHADJ_L(tmr_idx), low);
- if (status) {
- ice_debug(hw, ICE_DBG_PTP, "Failed to write incval to PHY SHADJ_L, status %d\n",
- status);
- return status;
+ err = ice_write_phy_reg_e810(hw, ETH_GLTSYN_SHADJ_L(tmr_idx), low);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to write incval to PHY SHADJ_L, err %d\n",
+ err);
+ return err;
}
- status = ice_write_phy_reg_e810(hw, ETH_GLTSYN_SHADJ_H(tmr_idx), high);
- if (status) {
- ice_debug(hw, ICE_DBG_PTP, "Failed to write incval PHY SHADJ_H, status %d\n",
- status);
- return status;
+ err = ice_write_phy_reg_e810(hw, ETH_GLTSYN_SHADJ_H(tmr_idx), high);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to write incval PHY SHADJ_H, err %d\n",
+ err);
+ return err;
}
return 0;
@@ -335,7 +2820,7 @@ static int ice_ptp_prep_phy_incval_e810(struct ice_hw *hw, u64 incval)
static int ice_ptp_port_cmd_e810(struct ice_hw *hw, enum ice_ptp_tmr_cmd cmd)
{
u32 cmd_val, val;
- int status;
+ int err;
switch (cmd) {
case INIT_TIME:
@@ -356,20 +2841,20 @@ static int ice_ptp_port_cmd_e810(struct ice_hw *hw, enum ice_ptp_tmr_cmd cmd)
}
/* Read, modify, write */
- status = ice_read_phy_reg_e810(hw, ETH_GLTSYN_CMD, &val);
- if (status) {
- ice_debug(hw, ICE_DBG_PTP, "Failed to read GLTSYN_CMD, status %d\n", status);
- return status;
+ err = ice_read_phy_reg_e810(hw, ETH_GLTSYN_CMD, &val);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to read GLTSYN_CMD, err %d\n", err);
+ return err;
}
/* Modify necessary bits only and perform write */
val &= ~TS_CMD_MASK_E810;
val |= cmd_val;
- status = ice_write_phy_reg_e810(hw, ETH_GLTSYN_CMD, val);
- if (status) {
- ice_debug(hw, ICE_DBG_PTP, "Failed to write back GLTSYN_CMD, status %d\n", status);
- return status;
+ err = ice_write_phy_reg_e810(hw, ETH_GLTSYN_CMD, val);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to write back GLTSYN_CMD, err %d\n", err);
+ return err;
}
return 0;
@@ -377,12 +2862,9 @@ static int ice_ptp_port_cmd_e810(struct ice_hw *hw, enum ice_ptp_tmr_cmd cmd)
/* Device agnostic functions
*
- * The following functions implement useful behavior to hide the differences
- * between E810 and other devices. They call the device-specific
- * implementations where necessary.
- *
- * Currently, the driver only supports E810, but future work will enable
- * support for E822-based devices.
+ * The following functions implement shared behavior common to both E822 and
+ * E810 devices, possibly calling a device specific implementation where
+ * necessary.
*/
/**
@@ -433,42 +2915,6 @@ void ice_ptp_unlock(struct ice_hw *hw)
}
/**
- * ice_ptp_src_cmd - Prepare source timer for a timer command
- * @hw: pointer to HW structure
- * @cmd: Timer command
- *
- * Prepare the source timer for an upcoming timer sync command.
- */
-static void ice_ptp_src_cmd(struct ice_hw *hw, enum ice_ptp_tmr_cmd cmd)
-{
- u32 cmd_val;
- u8 tmr_idx;
-
- tmr_idx = ice_get_ptp_src_clock_index(hw);
- cmd_val = tmr_idx << SEL_CPK_SRC;
-
- switch (cmd) {
- case INIT_TIME:
- cmd_val |= GLTSYN_CMD_INIT_TIME;
- break;
- case INIT_INCVAL:
- cmd_val |= GLTSYN_CMD_INIT_INCVAL;
- break;
- case ADJ_TIME:
- cmd_val |= GLTSYN_CMD_ADJ_TIME;
- break;
- case ADJ_TIME_AT_TIME:
- cmd_val |= GLTSYN_CMD_ADJ_INIT_TIME;
- break;
- case READ_TIME:
- cmd_val |= GLTSYN_CMD_READ_TIME;
- break;
- }
-
- wr32(hw, GLTSYN_CMD, cmd_val);
-}
-
-/**
* ice_ptp_tmr_cmd - Prepare and trigger a timer sync command
* @hw: pointer to HW struct
* @cmd: the command to issue
@@ -480,23 +2926,26 @@ static void ice_ptp_src_cmd(struct ice_hw *hw, enum ice_ptp_tmr_cmd cmd)
*/
static int ice_ptp_tmr_cmd(struct ice_hw *hw, enum ice_ptp_tmr_cmd cmd)
{
- int status;
+ int err;
/* First, prepare the source timer */
ice_ptp_src_cmd(hw, cmd);
/* Next, prepare the ports */
- status = ice_ptp_port_cmd_e810(hw, cmd);
- if (status) {
- ice_debug(hw, ICE_DBG_PTP, "Failed to prepare PHY ports for timer command %u, status %d\n",
- cmd, status);
- return status;
+ if (ice_is_e810(hw))
+ err = ice_ptp_port_cmd_e810(hw, cmd);
+ else
+ err = ice_ptp_port_cmd_e822(hw, cmd);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP, "Failed to prepare PHY ports for timer command %u, err %d\n",
+ cmd, err);
+ return err;
}
- /* Write the sync command register to drive both source and PHY timer commands
- * synchronously
+ /* Write the sync command register to drive both source and PHY timer
+ * commands synchronously
*/
- wr32(hw, GLTSYN_CMD_SYNC, SYNC_EXEC_CMD);
+ ice_ptp_exec_tmr_cmd(hw);
return 0;
}
@@ -516,8 +2965,8 @@ static int ice_ptp_tmr_cmd(struct ice_hw *hw, enum ice_ptp_tmr_cmd cmd)
*/
int ice_ptp_init_time(struct ice_hw *hw, u64 time)
{
- int status;
u8 tmr_idx;
+ int err;
tmr_idx = hw->func_caps.ts_func_info.tmr_index_owned;
@@ -528,9 +2977,12 @@ int ice_ptp_init_time(struct ice_hw *hw, u64 time)
/* PHY timers */
/* Fill Rx and Tx ports and send msg to PHY */
- status = ice_ptp_prep_phy_time_e810(hw, time & 0xFFFFFFFF);
- if (status)
- return status;
+ if (ice_is_e810(hw))
+ err = ice_ptp_prep_phy_time_e810(hw, time & 0xFFFFFFFF);
+ else
+ err = ice_ptp_prep_phy_time_e822(hw, time & 0xFFFFFFFF);
+ if (err)
+ return err;
return ice_ptp_tmr_cmd(hw, INIT_TIME);
}
@@ -551,8 +3003,8 @@ int ice_ptp_init_time(struct ice_hw *hw, u64 time)
*/
int ice_ptp_write_incval(struct ice_hw *hw, u64 incval)
{
- int status;
u8 tmr_idx;
+ int err;
tmr_idx = hw->func_caps.ts_func_info.tmr_index_owned;
@@ -560,9 +3012,12 @@ int ice_ptp_write_incval(struct ice_hw *hw, u64 incval)
wr32(hw, GLTSYN_SHADJ_L(tmr_idx), lower_32_bits(incval));
wr32(hw, GLTSYN_SHADJ_H(tmr_idx), upper_32_bits(incval));
- status = ice_ptp_prep_phy_incval_e810(hw, incval);
- if (status)
- return status;
+ if (ice_is_e810(hw))
+ err = ice_ptp_prep_phy_incval_e810(hw, incval);
+ else
+ err = ice_ptp_prep_phy_incval_e822(hw, incval);
+ if (err)
+ return err;
return ice_ptp_tmr_cmd(hw, INIT_INCVAL);
}
@@ -576,16 +3031,16 @@ int ice_ptp_write_incval(struct ice_hw *hw, u64 incval)
*/
int ice_ptp_write_incval_locked(struct ice_hw *hw, u64 incval)
{
- int status;
+ int err;
if (!ice_ptp_lock(hw))
return -EBUSY;
- status = ice_ptp_write_incval(hw, incval);
+ err = ice_ptp_write_incval(hw, incval);
ice_ptp_unlock(hw);
- return status;
+ return err;
}
/**
@@ -603,8 +3058,8 @@ int ice_ptp_write_incval_locked(struct ice_hw *hw, u64 incval)
*/
int ice_ptp_adj_clock(struct ice_hw *hw, s32 adj)
{
- int status;
u8 tmr_idx;
+ int err;
tmr_idx = hw->func_caps.ts_func_info.tmr_index_owned;
@@ -616,9 +3071,12 @@ int ice_ptp_adj_clock(struct ice_hw *hw, s32 adj)
wr32(hw, GLTSYN_SHADJ_L(tmr_idx), 0);
wr32(hw, GLTSYN_SHADJ_H(tmr_idx), adj);
- status = ice_ptp_prep_phy_adj_e810(hw, adj);
- if (status)
- return status;
+ if (ice_is_e810(hw))
+ err = ice_ptp_prep_phy_adj_e810(hw, adj);
+ else
+ err = ice_ptp_prep_phy_adj_e822(hw, adj);
+ if (err)
+ return err;
return ice_ptp_tmr_cmd(hw, ADJ_TIME);
}
@@ -630,11 +3088,16 @@ int ice_ptp_adj_clock(struct ice_hw *hw, s32 adj)
* @idx: the timestamp index to read
* @tstamp: on return, the 40bit timestamp value
*
- * Read a 40bit timestamp value out of the timestamp block.
+ * Read a 40bit timestamp value out of the timestamp block. For E822 devices,
+ * the block is the quad to read from. For E810 devices, the block is the
+ * logical port to read from.
*/
int ice_read_phy_tstamp(struct ice_hw *hw, u8 block, u8 idx, u64 *tstamp)
{
- return ice_read_phy_tstamp_e810(hw, block, idx, tstamp);
+ if (ice_is_e810(hw))
+ return ice_read_phy_tstamp_e810(hw, block, idx, tstamp);
+ else
+ return ice_read_phy_tstamp_e822(hw, block, idx, tstamp);
}
/**
@@ -643,11 +3106,16 @@ int ice_read_phy_tstamp(struct ice_hw *hw, u8 block, u8 idx, u64 *tstamp)
* @block: the block to read from
* @idx: the timestamp index to reset
*
- * Clear a timestamp, resetting its valid bit, from the timestamp block.
+ * Clear a timestamp, resetting its valid bit, from the timestamp block. For
+ * E822 devices, the block is the quad to clear from. For E810 devices, the
+ * block is the logical port to clear from.
*/
int ice_clear_phy_tstamp(struct ice_hw *hw, u8 block, u8 idx)
{
- return ice_clear_phy_tstamp_e810(hw, block, idx);
+ if (ice_is_e810(hw))
+ return ice_clear_phy_tstamp_e810(hw, block, idx);
+ else
+ return ice_clear_phy_tstamp_e822(hw, block, idx);
}
/* E810T SMA functions
@@ -800,3 +3268,25 @@ bool ice_is_pca9575_present(struct ice_hw *hw)
return !status && handle;
}
+
+/**
+ * ice_ptp_init_phc - Initialize PTP hardware clock
+ * @hw: pointer to the HW struct
+ *
+ * Perform the steps required to initialize the PTP hardware clock.
+ */
+int ice_ptp_init_phc(struct ice_hw *hw)
+{
+ u8 src_idx = hw->func_caps.ts_func_info.tmr_index_owned;
+
+ /* Enable source clocks */
+ wr32(hw, GLTSYN_ENA(src_idx), GLTSYN_ENA_TSYN_ENA_M);
+
+ /* Clear event err indications for auxiliary pins */
+ (void)rd32(hw, GLTSYN_STAT(src_idx));
+
+ if (ice_is_e810(hw))
+ return ice_ptp_init_phc_e810(hw);
+ else
+ return ice_ptp_init_phc_e822(hw);
+}
diff --git a/drivers/net/ethernet/intel/ice/ice_ptp_hw.h b/drivers/net/ethernet/intel/ice/ice_ptp_hw.h
index b2984b5..519e754 100644
--- a/drivers/net/ethernet/intel/ice/ice_ptp_hw.h
+++ b/drivers/net/ethernet/intel/ice/ice_ptp_hw.h
@@ -12,6 +12,112 @@ enum ice_ptp_tmr_cmd {
READ_TIME
};
+enum ice_ptp_serdes {
+ ICE_PTP_SERDES_1G,
+ ICE_PTP_SERDES_10G,
+ ICE_PTP_SERDES_25G,
+ ICE_PTP_SERDES_40G,
+ ICE_PTP_SERDES_50G,
+ ICE_PTP_SERDES_100G
+};
+
+enum ice_ptp_link_spd {
+ ICE_PTP_LNK_SPD_1G,
+ ICE_PTP_LNK_SPD_10G,
+ ICE_PTP_LNK_SPD_25G,
+ ICE_PTP_LNK_SPD_25G_RS,
+ ICE_PTP_LNK_SPD_40G,
+ ICE_PTP_LNK_SPD_50G,
+ ICE_PTP_LNK_SPD_50G_RS,
+ ICE_PTP_LNK_SPD_100G_RS,
+ NUM_ICE_PTP_LNK_SPD /* Must be last */
+};
+
+enum ice_ptp_fec_mode {
+ ICE_PTP_FEC_MODE_NONE,
+ ICE_PTP_FEC_MODE_CLAUSE74,
+ ICE_PTP_FEC_MODE_RS_FEC
+};
+
+/**
+ * struct ice_time_ref_info_e822
+ * @pll_freq: Frequency of PLL that drives timer ticks in Hz
+ * @nominal_incval: increment to generate nanoseconds in GLTSYN_TIME_L
+ * @pps_delay: propagation delay of the PPS output signal
+ *
+ * Characteristic information for the various TIME_REF sources possible in the
+ * E822 devices
+ */
+struct ice_time_ref_info_e822 {
+ u64 pll_freq;
+ u64 nominal_incval;
+ u8 pps_delay;
+};
+
+/**
+ * struct ice_vernier_info_e822
+ * @tx_par_clk: Frequency used to calculate P_REG_PAR_TX_TUS
+ * @rx_par_clk: Frequency used to calculate P_REG_PAR_RX_TUS
+ * @tx_pcs_clk: Frequency used to calculate P_REG_PCS_TX_TUS
+ * @rx_pcs_clk: Frequency used to calculate P_REG_PCS_RX_TUS
+ * @tx_desk_rsgb_par: Frequency used to calculate P_REG_DESK_PAR_TX_TUS
+ * @rx_desk_rsgb_par: Frequency used to calculate P_REG_DESK_PAR_RX_TUS
+ * @tx_desk_rsgb_pcs: Frequency used to calculate P_REG_DESK_PCS_TX_TUS
+ * @rx_desk_rsgb_pcs: Frequency used to calculate P_REG_DESK_PCS_RX_TUS
+ * @tx_fixed_delay: Fixed Tx latency measured in 1/100th nanoseconds
+ * @pmd_adj_divisor: Divisor used to calculate PDM alignment adjustment
+ * @rx_fixed_delay: Fixed Rx latency measured in 1/100th nanoseconds
+ *
+ * Table of constants used during as part of the Vernier calibration of the Tx
+ * and Rx timestamps. This includes frequency values used to compute TUs per
+ * PAR/PCS clock cycle, and static delay values measured during hardware
+ * design.
+ *
+ * Note that some values are not used for all link speeds, and the
+ * P_REG_DESK_PAR* registers may represent different clock markers at
+ * different link speeds, either the deskew marker for multi-lane link speeds
+ * or the Reed Solomon gearbox marker for RS-FEC.
+ */
+struct ice_vernier_info_e822 {
+ u32 tx_par_clk;
+ u32 rx_par_clk;
+ u32 tx_pcs_clk;
+ u32 rx_pcs_clk;
+ u32 tx_desk_rsgb_par;
+ u32 rx_desk_rsgb_par;
+ u32 tx_desk_rsgb_pcs;
+ u32 rx_desk_rsgb_pcs;
+ u32 tx_fixed_delay;
+ u32 pmd_adj_divisor;
+ u32 rx_fixed_delay;
+};
+
+/**
+ * struct ice_cgu_pll_params_e822
+ * @refclk_pre_div: Reference clock pre-divisor
+ * @feedback_div: Feedback divisor
+ * @frac_n_div: Fractional divisor
+ * @post_pll_div: Post PLL divisor
+ *
+ * Clock Generation Unit parameters used to program the PLL based on the
+ * selected TIME_REF frequency.
+ */
+struct ice_cgu_pll_params_e822 {
+ u32 refclk_pre_div;
+ u32 feedback_div;
+ u32 frac_n_div;
+ u32 post_pll_div;
+};
+
+extern const struct
+ice_cgu_pll_params_e822 e822_cgu_params[NUM_ICE_TIME_REF_FREQ];
+
+/* Table of constants related to possible TIME_REF sources */
+extern const struct ice_time_ref_info_e822 e822_time_ref[NUM_ICE_TIME_REF_FREQ];
+
+/* Table of constants for Vernier calibration on E822 */
+extern const struct ice_vernier_info_e822 e822_vernier[NUM_ICE_PTP_LNK_SPD];
+
/* Increment value to generate nanoseconds in the GLTSYN_TIME_L register for
* the E810 devices. Based off of a PLL with an 812.5 MHz frequency.
*/
@@ -27,6 +133,59 @@ int ice_ptp_write_incval_locked(struct ice_hw *hw, u64 incval);
int ice_ptp_adj_clock(struct ice_hw *hw, s32 adj);
int ice_read_phy_tstamp(struct ice_hw *hw, u8 block, u8 idx, u64 *tstamp);
int ice_clear_phy_tstamp(struct ice_hw *hw, u8 block, u8 idx);
+int ice_ptp_init_phc(struct ice_hw *hw);
+
+/* E822 family functions */
+int ice_read_phy_reg_e822(struct ice_hw *hw, u8 port, u16 offset, u32 *val);
+int ice_write_phy_reg_e822(struct ice_hw *hw, u8 port, u16 offset, u32 val);
+int ice_read_quad_reg_e822(struct ice_hw *hw, u8 quad, u16 offset, u32 *val);
+int ice_write_quad_reg_e822(struct ice_hw *hw, u8 quad, u16 offset, u32 val);
+int ice_ptp_prep_port_adj_e822(struct ice_hw *hw, u8 port, s64 time);
+
+/**
+ * ice_e822_time_ref - Get the current TIME_REF from capabilities
+ * @hw: pointer to the HW structure
+ *
+ * Returns the current TIME_REF from the capabilities structure.
+ */
+static inline enum ice_time_ref_freq ice_e822_time_ref(struct ice_hw *hw)
+{
+ return hw->func_caps.ts_func_info.time_ref;
+}
+
+/**
+ * ice_set_e822_time_ref - Set new TIME_REF
+ * @hw: pointer to the HW structure
+ * @time_ref: new TIME_REF to set
+ *
+ * Update the TIME_REF in the capabilities structure in response to some
+ * change, such as an update to the CGU registers.
+ */
+static inline void
+ice_set_e822_time_ref(struct ice_hw *hw, enum ice_time_ref_freq time_ref)
+{
+ hw->func_caps.ts_func_info.time_ref = time_ref;
+}
+
+static inline u64 ice_e822_pll_freq(enum ice_time_ref_freq time_ref)
+{
+ return e822_time_ref[time_ref].pll_freq;
+}
+
+static inline u64 ice_e822_nominal_incval(enum ice_time_ref_freq time_ref)
+{
+ return e822_time_ref[time_ref].nominal_incval;
+}
+
+static inline u64 ice_e822_pps_delay(enum ice_time_ref_freq time_ref)
+{
+ return e822_time_ref[time_ref].pps_delay;
+}
+
+/* E822 Vernier calibration functions */
+int ice_stop_phy_timer_e822(struct ice_hw *hw, u8 port, bool soft_reset);
+int ice_start_phy_timer_e822(struct ice_hw *hw, u8 port, bool bypass);
+int ice_phy_exit_bypass_e822(struct ice_hw *hw, u8 port);
/* E810 family functions */
int ice_ptp_init_phy_e810(struct ice_hw *hw);
@@ -36,19 +195,194 @@ bool ice_is_pca9575_present(struct ice_hw *hw);
#define PFTSYN_SEM_BYTES 4
+#define ICE_PTP_CLOCK_INDEX_0 0x00
+#define ICE_PTP_CLOCK_INDEX_1 0x01
+
/* PHY timer commands */
#define SEL_CPK_SRC 8
+#define SEL_PHY_SRC 3
/* Time Sync command Definitions */
#define GLTSYN_CMD_INIT_TIME BIT(0)
#define GLTSYN_CMD_INIT_INCVAL BIT(1)
+#define GLTSYN_CMD_INIT_TIME_INCVAL (BIT(0) | BIT(1))
#define GLTSYN_CMD_ADJ_TIME BIT(2)
#define GLTSYN_CMD_ADJ_INIT_TIME (BIT(2) | BIT(3))
#define GLTSYN_CMD_READ_TIME BIT(7)
+/* PHY port Time Sync command definitions */
+#define PHY_CMD_INIT_TIME BIT(0)
+#define PHY_CMD_INIT_INCVAL BIT(1)
+#define PHY_CMD_ADJ_TIME (BIT(0) | BIT(1))
+#define PHY_CMD_ADJ_TIME_AT_TIME (BIT(0) | BIT(2))
+#define PHY_CMD_READ_TIME (BIT(0) | BIT(1) | BIT(2))
+
#define TS_CMD_MASK_E810 0xFF
+#define TS_CMD_MASK 0xF
#define SYNC_EXEC_CMD 0x3
+/* Macros to derive port low and high addresses on both quads */
+#define P_Q0_L(a, p) ((((a) + (0x2000 * (p)))) & 0xFFFF)
+#define P_Q0_H(a, p) ((((a) + (0x2000 * (p)))) >> 16)
+#define P_Q1_L(a, p) ((((a) - (0x2000 * ((p) - ICE_PORTS_PER_QUAD)))) & 0xFFFF)
+#define P_Q1_H(a, p) ((((a) - (0x2000 * ((p) - ICE_PORTS_PER_QUAD)))) >> 16)
+
+/* PHY QUAD register base addresses */
+#define Q_0_BASE 0x94000
+#define Q_1_BASE 0x114000
+
+/* Timestamp memory reset registers */
+#define Q_REG_TS_CTRL 0x618
+#define Q_REG_TS_CTRL_S 0
+#define Q_REG_TS_CTRL_M BIT(0)
+
+/* Timestamp availability status registers */
+#define Q_REG_TX_MEMORY_STATUS_L 0xCF0
+#define Q_REG_TX_MEMORY_STATUS_U 0xCF4
+
+/* Tx FIFO status registers */
+#define Q_REG_FIFO23_STATUS 0xCF8
+#define Q_REG_FIFO01_STATUS 0xCFC
+#define Q_REG_FIFO02_S 0
+#define Q_REG_FIFO02_M ICE_M(0x3FF, 0)
+#define Q_REG_FIFO13_S 10
+#define Q_REG_FIFO13_M ICE_M(0x3FF, 10)
+
+/* Interrupt control Config registers */
+#define Q_REG_TX_MEM_GBL_CFG 0xC08
+#define Q_REG_TX_MEM_GBL_CFG_LANE_TYPE_S 0
+#define Q_REG_TX_MEM_GBL_CFG_LANE_TYPE_M BIT(0)
+#define Q_REG_TX_MEM_GBL_CFG_TX_TYPE_S 1
+#define Q_REG_TX_MEM_GBL_CFG_TX_TYPE_M ICE_M(0xFF, 1)
+#define Q_REG_TX_MEM_GBL_CFG_INTR_THR_S 9
+#define Q_REG_TX_MEM_GBL_CFG_INTR_THR_M ICE_M(0x3F, 9)
+#define Q_REG_TX_MEM_GBL_CFG_INTR_ENA_S 15
+#define Q_REG_TX_MEM_GBL_CFG_INTR_ENA_M BIT(15)
+
+/* Tx Timestamp data registers */
+#define Q_REG_TX_MEMORY_BANK_START 0xA00
+
+/* PHY port register base addresses */
+#define P_0_BASE 0x80000
+#define P_4_BASE 0x106000
+
+/* Timestamp init registers */
+#define P_REG_RX_TIMER_INC_PRE_L 0x46C
+#define P_REG_RX_TIMER_INC_PRE_U 0x470
+#define P_REG_TX_TIMER_INC_PRE_L 0x44C
+#define P_REG_TX_TIMER_INC_PRE_U 0x450
+
+/* Timestamp match and adjust target registers */
+#define P_REG_RX_TIMER_CNT_ADJ_L 0x474
+#define P_REG_RX_TIMER_CNT_ADJ_U 0x478
+#define P_REG_TX_TIMER_CNT_ADJ_L 0x454
+#define P_REG_TX_TIMER_CNT_ADJ_U 0x458
+
+/* Timestamp capture registers */
+#define P_REG_RX_CAPTURE_L 0x4D8
+#define P_REG_RX_CAPTURE_U 0x4DC
+#define P_REG_TX_CAPTURE_L 0x4B4
+#define P_REG_TX_CAPTURE_U 0x4B8
+
+/* Timestamp PHY incval registers */
+#define P_REG_TIMETUS_L 0x410
+#define P_REG_TIMETUS_U 0x414
+
+#define P_REG_40B_LOW_M 0xFF
+#define P_REG_40B_HIGH_S 8
+
+/* PHY window length registers */
+#define P_REG_WL 0x40C
+
+#define PTP_VERNIER_WL 0x111ed
+
+/* PHY start registers */
+#define P_REG_PS 0x408
+#define P_REG_PS_START_S 0
+#define P_REG_PS_START_M BIT(0)
+#define P_REG_PS_BYPASS_MODE_S 1
+#define P_REG_PS_BYPASS_MODE_M BIT(1)
+#define P_REG_PS_ENA_CLK_S 2
+#define P_REG_PS_ENA_CLK_M BIT(2)
+#define P_REG_PS_LOAD_OFFSET_S 3
+#define P_REG_PS_LOAD_OFFSET_M BIT(3)
+#define P_REG_PS_SFT_RESET_S 11
+#define P_REG_PS_SFT_RESET_M BIT(11)
+
+/* PHY offset valid registers */
+#define P_REG_TX_OV_STATUS 0x4D4
+#define P_REG_TX_OV_STATUS_OV_S 0
+#define P_REG_TX_OV_STATUS_OV_M BIT(0)
+#define P_REG_RX_OV_STATUS 0x4F8
+#define P_REG_RX_OV_STATUS_OV_S 0
+#define P_REG_RX_OV_STATUS_OV_M BIT(0)
+
+/* PHY offset ready registers */
+#define P_REG_TX_OR 0x45C
+#define P_REG_RX_OR 0x47C
+
+/* PHY total offset registers */
+#define P_REG_TOTAL_RX_OFFSET_L 0x460
+#define P_REG_TOTAL_RX_OFFSET_U 0x464
+#define P_REG_TOTAL_TX_OFFSET_L 0x440
+#define P_REG_TOTAL_TX_OFFSET_U 0x444
+
+/* Timestamp PAR/PCS registers */
+#define P_REG_UIX66_10G_40G_L 0x480
+#define P_REG_UIX66_10G_40G_U 0x484
+#define P_REG_UIX66_25G_100G_L 0x488
+#define P_REG_UIX66_25G_100G_U 0x48C
+#define P_REG_DESK_PAR_RX_TUS_L 0x490
+#define P_REG_DESK_PAR_RX_TUS_U 0x494
+#define P_REG_DESK_PAR_TX_TUS_L 0x498
+#define P_REG_DESK_PAR_TX_TUS_U 0x49C
+#define P_REG_DESK_PCS_RX_TUS_L 0x4A0
+#define P_REG_DESK_PCS_RX_TUS_U 0x4A4
+#define P_REG_DESK_PCS_TX_TUS_L 0x4A8
+#define P_REG_DESK_PCS_TX_TUS_U 0x4AC
+#define P_REG_PAR_RX_TUS_L 0x420
+#define P_REG_PAR_RX_TUS_U 0x424
+#define P_REG_PAR_TX_TUS_L 0x428
+#define P_REG_PAR_TX_TUS_U 0x42C
+#define P_REG_PCS_RX_TUS_L 0x430
+#define P_REG_PCS_RX_TUS_U 0x434
+#define P_REG_PCS_TX_TUS_L 0x438
+#define P_REG_PCS_TX_TUS_U 0x43C
+#define P_REG_PAR_RX_TIME_L 0x4F0
+#define P_REG_PAR_RX_TIME_U 0x4F4
+#define P_REG_PAR_TX_TIME_L 0x4CC
+#define P_REG_PAR_TX_TIME_U 0x4D0
+#define P_REG_PAR_PCS_RX_OFFSET_L 0x4E8
+#define P_REG_PAR_PCS_RX_OFFSET_U 0x4EC
+#define P_REG_PAR_PCS_TX_OFFSET_L 0x4C4
+#define P_REG_PAR_PCS_TX_OFFSET_U 0x4C8
+#define P_REG_LINK_SPEED 0x4FC
+#define P_REG_LINK_SPEED_SERDES_S 0
+#define P_REG_LINK_SPEED_SERDES_M ICE_M(0x7, 0)
+#define P_REG_LINK_SPEED_FEC_MODE_S 3
+#define P_REG_LINK_SPEED_FEC_MODE_M ICE_M(0x3, 3)
+#define P_REG_LINK_SPEED_FEC_MODE(reg) \
+ (((reg) & P_REG_LINK_SPEED_FEC_MODE_M) >> \
+ P_REG_LINK_SPEED_FEC_MODE_S)
+
+/* PHY timestamp related registers */
+#define P_REG_PMD_ALIGNMENT 0x0FC
+#define P_REG_RX_80_TO_160_CNT 0x6FC
+#define P_REG_RX_80_TO_160_CNT_RXCYC_S 0
+#define P_REG_RX_80_TO_160_CNT_RXCYC_M BIT(0)
+#define P_REG_RX_40_TO_160_CNT 0x8FC
+#define P_REG_RX_40_TO_160_CNT_RXCYC_S 0
+#define P_REG_RX_40_TO_160_CNT_RXCYC_M ICE_M(0x3, 0)
+
+/* Rx FIFO status registers */
+#define P_REG_RX_OV_FS 0x4F8
+#define P_REG_RX_OV_FS_FIFO_STATUS_S 2
+#define P_REG_RX_OV_FS_FIFO_STATUS_M ICE_M(0x3FF, 2)
+
+/* Timestamp command registers */
+#define P_REG_TX_TMR_CMD 0x448
+#define P_REG_RX_TMR_CMD 0x468
+
/* E810 timesync enable register */
#define ETH_GLTSYN_ENA(_i) (0x03000348 + ((_i) * 4))
@@ -68,9 +402,20 @@ bool ice_is_pca9575_present(struct ice_hw *hw);
/* Timestamp block macros */
#define TS_LOW_M 0xFFFFFFFF
+#define TS_HIGH_M 0xFF
#define TS_HIGH_S 32
+#define TS_PHY_LOW_M 0xFF
+#define TS_PHY_HIGH_M 0xFFFFFFFF
+#define TS_PHY_HIGH_S 8
+
#define BYTES_PER_IDX_ADDR_L_U 8
+#define BYTES_PER_IDX_ADDR_L 4
+
+/* Internal PHY timestamp address */
+#define TS_L(a, idx) ((a) + ((idx) * BYTES_PER_IDX_ADDR_L_U))
+#define TS_H(a, idx) ((a) + ((idx) * BYTES_PER_IDX_ADDR_L_U + \
+ BYTES_PER_IDX_ADDR_L))
/* External PHY timestamp address */
#define TS_EXT(a, port, idx) ((a) + (0x1000 * (port)) + \
diff --git a/drivers/net/ethernet/intel/ice/ice_repr.c b/drivers/net/ethernet/intel/ice/ice_repr.c
index af8e6ef..dcc310e 100644
--- a/drivers/net/ethernet/intel/ice/ice_repr.c
+++ b/drivers/net/ethernet/intel/ice/ice_repr.c
@@ -244,6 +244,14 @@ static int ice_repr_add(struct ice_vf *vf)
if (!repr)
return -ENOMEM;
+#ifdef CONFIG_ICE_SWITCHDEV
+ repr->mac_rule = kzalloc(sizeof(*repr->mac_rule), GFP_KERNEL);
+ if (!repr->mac_rule) {
+ err = -ENOMEM;
+ goto err_alloc_rule;
+ }
+#endif
+
repr->netdev = alloc_etherdev(sizeof(struct ice_netdev_priv));
if (!repr->netdev) {
err = -ENOMEM;
@@ -287,6 +295,11 @@ static int ice_repr_add(struct ice_vf *vf)
free_netdev(repr->netdev);
repr->netdev = NULL;
err_alloc:
+#ifdef CONFIG_ICE_SWITCHDEV
+ kfree(repr->mac_rule);
+ repr->mac_rule = NULL;
+err_alloc_rule:
+#endif
kfree(repr);
vf->repr = NULL;
return err;
@@ -304,6 +317,10 @@ static void ice_repr_rem(struct ice_vf *vf)
unregister_netdev(vf->repr->netdev);
free_netdev(vf->repr->netdev);
vf->repr->netdev = NULL;
+#ifdef CONFIG_ICE_SWITCHDEV
+ kfree(vf->repr->mac_rule);
+ vf->repr->mac_rule = NULL;
+#endif
kfree(vf->repr);
vf->repr = NULL;
}
diff --git a/drivers/net/ethernet/intel/ice/ice_repr.h b/drivers/net/ethernet/intel/ice/ice_repr.h
index 806de22..0c77ff0 100644
--- a/drivers/net/ethernet/intel/ice/ice_repr.h
+++ b/drivers/net/ethernet/intel/ice/ice_repr.h
@@ -13,6 +13,11 @@ struct ice_repr {
struct ice_q_vector *q_vector;
struct net_device *netdev;
struct metadata_dst *dst;
+#ifdef CONFIG_ICE_SWITCHDEV
+ /* info about slow path MAC rule */
+ struct ice_rule_query_data *mac_rule;
+ u8 rule_added;
+#endif
};
int ice_repr_add_for_all_vfs(struct ice_pf *pf);
diff --git a/drivers/net/ethernet/intel/ice/ice_sched.c b/drivers/net/ethernet/intel/ice/ice_sched.c
index ce3c7bd..7947223 100644
--- a/drivers/net/ethernet/intel/ice/ice_sched.c
+++ b/drivers/net/ethernet/intel/ice/ice_sched.c
@@ -11,7 +11,7 @@
* This function inserts the root node of the scheduling tree topology
* to the SW DB.
*/
-static enum ice_status
+static int
ice_sched_add_root_node(struct ice_port_info *pi,
struct ice_aqc_txsched_elem_data *info)
{
@@ -19,20 +19,20 @@ ice_sched_add_root_node(struct ice_port_info *pi,
struct ice_hw *hw;
if (!pi)
- return ICE_ERR_PARAM;
+ return -EINVAL;
hw = pi->hw;
root = devm_kzalloc(ice_hw_to_dev(hw), sizeof(*root), GFP_KERNEL);
if (!root)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
/* coverity[suspicious_sizeof] */
root->children = devm_kcalloc(ice_hw_to_dev(hw), hw->max_children[0],
sizeof(*root), GFP_KERNEL);
if (!root->children) {
devm_kfree(ice_hw_to_dev(hw), root);
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
}
memcpy(&root->info, info, sizeof(*info));
@@ -96,14 +96,14 @@ ice_sched_find_node_by_teid(struct ice_sched_node *start_node, u32 teid)
*
* This function sends a scheduling elements cmd (cmd_opc)
*/
-static enum ice_status
+static int
ice_aqc_send_sched_elem_cmd(struct ice_hw *hw, enum ice_adminq_opc cmd_opc,
u16 elems_req, void *buf, u16 buf_size,
u16 *elems_resp, struct ice_sq_cd *cd)
{
struct ice_aqc_sched_elem_cmd *cmd;
struct ice_aq_desc desc;
- enum ice_status status;
+ int status;
cmd = &desc.params.sched_elem_cmd;
ice_fill_dflt_direct_cmd_desc(&desc, cmd_opc);
@@ -127,7 +127,7 @@ ice_aqc_send_sched_elem_cmd(struct ice_hw *hw, enum ice_adminq_opc cmd_opc,
*
* Query scheduling elements (0x0404)
*/
-enum ice_status
+int
ice_aq_query_sched_elems(struct ice_hw *hw, u16 elems_req,
struct ice_aqc_txsched_elem_data *buf, u16 buf_size,
u16 *elems_ret, struct ice_sq_cd *cd)
@@ -145,18 +145,18 @@ ice_aq_query_sched_elems(struct ice_hw *hw, u16 elems_req,
*
* This function inserts a scheduler node to the SW DB.
*/
-enum ice_status
+int
ice_sched_add_node(struct ice_port_info *pi, u8 layer,
struct ice_aqc_txsched_elem_data *info)
{
struct ice_aqc_txsched_elem_data elem;
struct ice_sched_node *parent;
struct ice_sched_node *node;
- enum ice_status status;
struct ice_hw *hw;
+ int status;
if (!pi)
- return ICE_ERR_PARAM;
+ return -EINVAL;
hw = pi->hw;
@@ -166,7 +166,7 @@ ice_sched_add_node(struct ice_port_info *pi, u8 layer,
if (!parent) {
ice_debug(hw, ICE_DBG_SCHED, "Parent Node not found for parent_teid=0x%x\n",
le32_to_cpu(info->parent_teid));
- return ICE_ERR_PARAM;
+ return -EINVAL;
}
/* query the current node information from FW before adding it
@@ -178,7 +178,7 @@ ice_sched_add_node(struct ice_port_info *pi, u8 layer,
node = devm_kzalloc(ice_hw_to_dev(hw), sizeof(*node), GFP_KERNEL);
if (!node)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
if (hw->max_children[layer]) {
/* coverity[suspicious_sizeof] */
node->children = devm_kcalloc(ice_hw_to_dev(hw),
@@ -186,7 +186,7 @@ ice_sched_add_node(struct ice_port_info *pi, u8 layer,
sizeof(*node), GFP_KERNEL);
if (!node->children) {
devm_kfree(ice_hw_to_dev(hw), node);
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
}
}
@@ -209,7 +209,7 @@ ice_sched_add_node(struct ice_port_info *pi, u8 layer,
*
* Delete scheduling elements (0x040F)
*/
-static enum ice_status
+static int
ice_aq_delete_sched_elems(struct ice_hw *hw, u16 grps_req,
struct ice_aqc_delete_elem *buf, u16 buf_size,
u16 *grps_del, struct ice_sq_cd *cd)
@@ -228,19 +228,19 @@ ice_aq_delete_sched_elems(struct ice_hw *hw, u16 grps_req,
*
* This function remove nodes from HW
*/
-static enum ice_status
+static int
ice_sched_remove_elems(struct ice_hw *hw, struct ice_sched_node *parent,
u16 num_nodes, u32 *node_teids)
{
struct ice_aqc_delete_elem *buf;
u16 i, num_groups_removed = 0;
- enum ice_status status;
u16 buf_size;
+ int status;
buf_size = struct_size(buf, teid, num_nodes);
buf = devm_kzalloc(ice_hw_to_dev(hw), buf_size, GFP_KERNEL);
if (!buf)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
buf->hdr.parent_teid = parent->info.node_teid;
buf->hdr.num_elems = cpu_to_le16(num_nodes);
@@ -369,14 +369,14 @@ void ice_free_sched_node(struct ice_port_info *pi, struct ice_sched_node *node)
*
* Get default scheduler topology (0x400)
*/
-static enum ice_status
+static int
ice_aq_get_dflt_topo(struct ice_hw *hw, u8 lport,
struct ice_aqc_get_topo_elem *buf, u16 buf_size,
u8 *num_branches, struct ice_sq_cd *cd)
{
struct ice_aqc_get_topo *cmd;
struct ice_aq_desc desc;
- enum ice_status status;
+ int status;
cmd = &desc.params.get_topo;
ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_get_dflt_topo);
@@ -399,7 +399,7 @@ ice_aq_get_dflt_topo(struct ice_hw *hw, u8 lport,
*
* Add scheduling elements (0x0401)
*/
-static enum ice_status
+static int
ice_aq_add_sched_elems(struct ice_hw *hw, u16 grps_req,
struct ice_aqc_add_elem *buf, u16 buf_size,
u16 *grps_added, struct ice_sq_cd *cd)
@@ -420,7 +420,7 @@ ice_aq_add_sched_elems(struct ice_hw *hw, u16 grps_req,
*
* Configure scheduling elements (0x0403)
*/
-static enum ice_status
+static int
ice_aq_cfg_sched_elems(struct ice_hw *hw, u16 elems_req,
struct ice_aqc_txsched_elem_data *buf, u16 buf_size,
u16 *elems_cfgd, struct ice_sq_cd *cd)
@@ -441,7 +441,7 @@ ice_aq_cfg_sched_elems(struct ice_hw *hw, u16 elems_req,
*
* Move scheduling elements (0x0408)
*/
-static enum ice_status
+static int
ice_aq_move_sched_elems(struct ice_hw *hw, u16 grps_req,
struct ice_aqc_move_elem *buf, u16 buf_size,
u16 *grps_movd, struct ice_sq_cd *cd)
@@ -462,7 +462,7 @@ ice_aq_move_sched_elems(struct ice_hw *hw, u16 grps_req,
*
* Suspend scheduling elements (0x0409)
*/
-static enum ice_status
+static int
ice_aq_suspend_sched_elems(struct ice_hw *hw, u16 elems_req, __le32 *buf,
u16 buf_size, u16 *elems_ret, struct ice_sq_cd *cd)
{
@@ -482,7 +482,7 @@ ice_aq_suspend_sched_elems(struct ice_hw *hw, u16 elems_req, __le32 *buf,
*
* resume scheduling elements (0x040A)
*/
-static enum ice_status
+static int
ice_aq_resume_sched_elems(struct ice_hw *hw, u16 elems_req, __le32 *buf,
u16 buf_size, u16 *elems_ret, struct ice_sq_cd *cd)
{
@@ -500,7 +500,7 @@ ice_aq_resume_sched_elems(struct ice_hw *hw, u16 elems_req, __le32 *buf,
*
* Query scheduler resource allocation (0x0412)
*/
-static enum ice_status
+static int
ice_aq_query_sched_res(struct ice_hw *hw, u16 buf_size,
struct ice_aqc_query_txsched_res_resp *buf,
struct ice_sq_cd *cd)
@@ -520,18 +520,18 @@ ice_aq_query_sched_res(struct ice_hw *hw, u16 buf_size,
*
* This function suspends or resumes HW nodes
*/
-static enum ice_status
+static int
ice_sched_suspend_resume_elems(struct ice_hw *hw, u8 num_nodes, u32 *node_teids,
bool suspend)
{
u16 i, buf_size, num_elem_ret = 0;
- enum ice_status status;
__le32 *buf;
+ int status;
buf_size = sizeof(*buf) * num_nodes;
buf = devm_kzalloc(ice_hw_to_dev(hw), buf_size, GFP_KERNEL);
if (!buf)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
for (i = 0; i < num_nodes; i++)
buf[i] = cpu_to_le32(node_teids[i]);
@@ -558,7 +558,7 @@ ice_sched_suspend_resume_elems(struct ice_hw *hw, u8 num_nodes, u32 *node_teids,
* @tc: TC number
* @new_numqs: number of queues
*/
-static enum ice_status
+static int
ice_alloc_lan_q_ctx(struct ice_hw *hw, u16 vsi_handle, u8 tc, u16 new_numqs)
{
struct ice_vsi_ctx *vsi_ctx;
@@ -566,7 +566,7 @@ ice_alloc_lan_q_ctx(struct ice_hw *hw, u16 vsi_handle, u8 tc, u16 new_numqs)
vsi_ctx = ice_get_vsi_ctx(hw, vsi_handle);
if (!vsi_ctx)
- return ICE_ERR_PARAM;
+ return -EINVAL;
/* allocate LAN queue contexts */
if (!vsi_ctx->lan_q_ctx[tc]) {
vsi_ctx->lan_q_ctx[tc] = devm_kcalloc(ice_hw_to_dev(hw),
@@ -574,7 +574,7 @@ ice_alloc_lan_q_ctx(struct ice_hw *hw, u16 vsi_handle, u8 tc, u16 new_numqs)
sizeof(*q_ctx),
GFP_KERNEL);
if (!vsi_ctx->lan_q_ctx[tc])
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
vsi_ctx->num_lan_q_entries[tc] = new_numqs;
return 0;
}
@@ -585,7 +585,7 @@ ice_alloc_lan_q_ctx(struct ice_hw *hw, u16 vsi_handle, u8 tc, u16 new_numqs)
q_ctx = devm_kcalloc(ice_hw_to_dev(hw), new_numqs,
sizeof(*q_ctx), GFP_KERNEL);
if (!q_ctx)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
memcpy(q_ctx, vsi_ctx->lan_q_ctx[tc],
prev_num * sizeof(*q_ctx));
devm_kfree(ice_hw_to_dev(hw), vsi_ctx->lan_q_ctx[tc]);
@@ -602,7 +602,7 @@ ice_alloc_lan_q_ctx(struct ice_hw *hw, u16 vsi_handle, u8 tc, u16 new_numqs)
* @tc: TC number
* @new_numqs: number of queues
*/
-static enum ice_status
+static int
ice_alloc_rdma_q_ctx(struct ice_hw *hw, u16 vsi_handle, u8 tc, u16 new_numqs)
{
struct ice_vsi_ctx *vsi_ctx;
@@ -610,7 +610,7 @@ ice_alloc_rdma_q_ctx(struct ice_hw *hw, u16 vsi_handle, u8 tc, u16 new_numqs)
vsi_ctx = ice_get_vsi_ctx(hw, vsi_handle);
if (!vsi_ctx)
- return ICE_ERR_PARAM;
+ return -EINVAL;
/* allocate RDMA queue contexts */
if (!vsi_ctx->rdma_q_ctx[tc]) {
vsi_ctx->rdma_q_ctx[tc] = devm_kcalloc(ice_hw_to_dev(hw),
@@ -618,7 +618,7 @@ ice_alloc_rdma_q_ctx(struct ice_hw *hw, u16 vsi_handle, u8 tc, u16 new_numqs)
sizeof(*q_ctx),
GFP_KERNEL);
if (!vsi_ctx->rdma_q_ctx[tc])
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
vsi_ctx->num_rdma_q_entries[tc] = new_numqs;
return 0;
}
@@ -629,7 +629,7 @@ ice_alloc_rdma_q_ctx(struct ice_hw *hw, u16 vsi_handle, u8 tc, u16 new_numqs)
q_ctx = devm_kcalloc(ice_hw_to_dev(hw), new_numqs,
sizeof(*q_ctx), GFP_KERNEL);
if (!q_ctx)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
memcpy(q_ctx, vsi_ctx->rdma_q_ctx[tc],
prev_num * sizeof(*q_ctx));
devm_kfree(ice_hw_to_dev(hw), vsi_ctx->rdma_q_ctx[tc]);
@@ -651,14 +651,14 @@ ice_alloc_rdma_q_ctx(struct ice_hw *hw, u16 vsi_handle, u8 tc, u16 new_numqs)
*
* RL profile function to add, query, or remove profile(s)
*/
-static enum ice_status
+static int
ice_aq_rl_profile(struct ice_hw *hw, enum ice_adminq_opc opcode,
u16 num_profiles, struct ice_aqc_rl_profile_elem *buf,
u16 buf_size, u16 *num_processed, struct ice_sq_cd *cd)
{
struct ice_aqc_rl_profile *cmd;
struct ice_aq_desc desc;
- enum ice_status status;
+ int status;
cmd = &desc.params.rl_profile;
@@ -682,7 +682,7 @@ ice_aq_rl_profile(struct ice_hw *hw, enum ice_adminq_opc opcode,
*
* Add RL profile (0x0410)
*/
-static enum ice_status
+static int
ice_aq_add_rl_profile(struct ice_hw *hw, u16 num_profiles,
struct ice_aqc_rl_profile_elem *buf, u16 buf_size,
u16 *num_profiles_added, struct ice_sq_cd *cd)
@@ -702,7 +702,7 @@ ice_aq_add_rl_profile(struct ice_hw *hw, u16 num_profiles,
*
* Remove RL profile (0x0415)
*/
-static enum ice_status
+static int
ice_aq_remove_rl_profile(struct ice_hw *hw, u16 num_profiles,
struct ice_aqc_rl_profile_elem *buf, u16 buf_size,
u16 *num_profiles_removed, struct ice_sq_cd *cd)
@@ -721,24 +721,24 @@ ice_aq_remove_rl_profile(struct ice_hw *hw, u16 num_profiles,
* its associated parameters from HW DB,and locally. The caller needs to
* hold scheduler lock.
*/
-static enum ice_status
+static int
ice_sched_del_rl_profile(struct ice_hw *hw,
struct ice_aqc_rl_profile_info *rl_info)
{
struct ice_aqc_rl_profile_elem *buf;
u16 num_profiles_removed;
- enum ice_status status;
u16 num_profiles = 1;
+ int status;
if (rl_info->prof_id_ref != 0)
- return ICE_ERR_IN_USE;
+ return -EBUSY;
/* Safe to remove profile ID */
buf = &rl_info->profile;
status = ice_aq_remove_rl_profile(hw, num_profiles, buf, sizeof(*buf),
&num_profiles_removed, NULL);
if (status || num_profiles_removed != num_profiles)
- return ICE_ERR_CFG;
+ return -EIO;
/* Delete stale entry now */
list_del(&rl_info->list_entry);
@@ -763,7 +763,7 @@ static void ice_sched_clear_rl_prof(struct ice_port_info *pi)
list_for_each_entry_safe(rl_prof_elem, rl_prof_tmp,
&pi->rl_prof_list[ln], list_entry) {
struct ice_hw *hw = pi->hw;
- enum ice_status status;
+ int status;
rl_prof_elem->prof_id_ref = 0;
status = ice_sched_del_rl_profile(hw, rl_prof_elem);
@@ -875,7 +875,7 @@ void ice_sched_cleanup_all(struct ice_hw *hw)
*
* This function add nodes to HW as well as to SW DB for a given layer
*/
-static enum ice_status
+static int
ice_sched_add_elems(struct ice_port_info *pi, struct ice_sched_node *tc_node,
struct ice_sched_node *parent, u8 layer, u16 num_nodes,
u16 *num_nodes_added, u32 *first_node_teid)
@@ -883,15 +883,15 @@ ice_sched_add_elems(struct ice_port_info *pi, struct ice_sched_node *tc_node,
struct ice_sched_node *prev, *new_node;
struct ice_aqc_add_elem *buf;
u16 i, num_groups_added = 0;
- enum ice_status status = 0;
struct ice_hw *hw = pi->hw;
size_t buf_size;
+ int status = 0;
u32 teid;
buf_size = struct_size(buf, generic, num_nodes);
buf = devm_kzalloc(ice_hw_to_dev(hw), buf_size, GFP_KERNEL);
if (!buf)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
buf->hdr.parent_teid = parent->info.node_teid;
buf->hdr.num_elems = cpu_to_le16(num_nodes);
@@ -918,7 +918,7 @@ ice_sched_add_elems(struct ice_port_info *pi, struct ice_sched_node *tc_node,
ice_debug(hw, ICE_DBG_SCHED, "add node failed FW Error %d\n",
hw->adminq.sq_last_status);
devm_kfree(ice_hw_to_dev(hw), buf);
- return ICE_ERR_CFG;
+ return -EIO;
}
*num_nodes_added = num_nodes;
@@ -974,7 +974,7 @@ ice_sched_add_elems(struct ice_port_info *pi, struct ice_sched_node *tc_node,
*
* Add nodes into specific HW layer.
*/
-static enum ice_status
+static int
ice_sched_add_nodes_to_hw_layer(struct ice_port_info *pi,
struct ice_sched_node *tc_node,
struct ice_sched_node *parent, u8 layer,
@@ -989,7 +989,7 @@ ice_sched_add_nodes_to_hw_layer(struct ice_port_info *pi,
return 0;
if (!parent || layer < pi->hw->sw_entry_point_layer)
- return ICE_ERR_PARAM;
+ return -EINVAL;
/* max children per node per layer */
max_child_nodes = pi->hw->max_children[parent->tx_sched_layer];
@@ -998,8 +998,8 @@ ice_sched_add_nodes_to_hw_layer(struct ice_port_info *pi,
if ((parent->num_children + num_nodes) > max_child_nodes) {
/* Fail if the parent is a TC node */
if (parent == tc_node)
- return ICE_ERR_CFG;
- return ICE_ERR_MAX_LIMIT;
+ return -EIO;
+ return -ENOSPC;
}
return ice_sched_add_elems(pi, tc_node, parent, layer, num_nodes,
@@ -1018,7 +1018,7 @@ ice_sched_add_nodes_to_hw_layer(struct ice_port_info *pi,
*
* This function add nodes to a given layer.
*/
-static enum ice_status
+static int
ice_sched_add_nodes_to_layer(struct ice_port_info *pi,
struct ice_sched_node *tc_node,
struct ice_sched_node *parent, u8 layer,
@@ -1027,7 +1027,7 @@ ice_sched_add_nodes_to_layer(struct ice_port_info *pi,
{
u32 *first_teid_ptr = first_node_teid;
u16 new_num_nodes = num_nodes;
- enum ice_status status = 0;
+ int status = 0;
*num_nodes_added = 0;
while (*num_nodes_added < num_nodes) {
@@ -1045,14 +1045,14 @@ ice_sched_add_nodes_to_layer(struct ice_port_info *pi,
if (*num_nodes_added > num_nodes) {
ice_debug(pi->hw, ICE_DBG_SCHED, "added extra nodes %d %d\n", num_nodes,
*num_nodes_added);
- status = ICE_ERR_CFG;
+ status = -EIO;
break;
}
/* break if all the nodes are added successfully */
if (!status && (*num_nodes_added == num_nodes))
break;
/* break if the error is not max limit */
- if (status && status != ICE_ERR_MAX_LIMIT)
+ if (status && status != -ENOSPC)
break;
/* Exceeded the max children */
max_child_nodes = pi->hw->max_children[parent->tx_sched_layer];
@@ -1152,7 +1152,7 @@ static void ice_rm_dflt_leaf_node(struct ice_port_info *pi)
}
if (node && node->info.data.elem_type == ICE_AQC_ELEM_TYPE_LEAF) {
u32 teid = le32_to_cpu(node->info.node_teid);
- enum ice_status status;
+ int status;
/* remove the default leaf node */
status = ice_sched_remove_elems(pi->hw, node->parent, 1, &teid);
@@ -1198,23 +1198,23 @@ static void ice_sched_rm_dflt_nodes(struct ice_port_info *pi)
* resources, default topology created by firmware and storing the information
* in SW DB.
*/
-enum ice_status ice_sched_init_port(struct ice_port_info *pi)
+int ice_sched_init_port(struct ice_port_info *pi)
{
struct ice_aqc_get_topo_elem *buf;
- enum ice_status status;
struct ice_hw *hw;
u8 num_branches;
u16 num_elems;
+ int status;
u8 i, j;
if (!pi)
- return ICE_ERR_PARAM;
+ return -EINVAL;
hw = pi->hw;
/* Query the Default Topology from FW */
buf = devm_kzalloc(ice_hw_to_dev(hw), ICE_AQ_MAX_BUF_LEN, GFP_KERNEL);
if (!buf)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
/* Query default scheduling tree topology */
status = ice_aq_get_dflt_topo(hw, pi->lport, buf, ICE_AQ_MAX_BUF_LEN,
@@ -1226,7 +1226,7 @@ enum ice_status ice_sched_init_port(struct ice_port_info *pi)
if (num_branches < 1 || num_branches > ICE_TXSCHED_MAX_BRANCHES) {
ice_debug(hw, ICE_DBG_SCHED, "num_branches unexpected %d\n",
num_branches);
- status = ICE_ERR_PARAM;
+ status = -EINVAL;
goto err_init_port;
}
@@ -1237,7 +1237,7 @@ enum ice_status ice_sched_init_port(struct ice_port_info *pi)
if (num_elems < 1 || num_elems > ICE_AQC_TOPO_MAX_LEVEL_NUM) {
ice_debug(hw, ICE_DBG_SCHED, "num_elems unexpected %d\n",
num_elems);
- status = ICE_ERR_PARAM;
+ status = -EINVAL;
goto err_init_port;
}
@@ -1300,11 +1300,11 @@ enum ice_status ice_sched_init_port(struct ice_port_info *pi)
*
* query FW for allocated scheduler resources and store in HW struct
*/
-enum ice_status ice_sched_query_res_alloc(struct ice_hw *hw)
+int ice_sched_query_res_alloc(struct ice_hw *hw)
{
struct ice_aqc_query_txsched_res_resp *buf;
- enum ice_status status = 0;
__le16 max_sibl;
+ int status = 0;
u16 i;
if (hw->layer_info)
@@ -1312,7 +1312,7 @@ enum ice_status ice_sched_query_res_alloc(struct ice_hw *hw)
buf = devm_kzalloc(ice_hw_to_dev(hw), sizeof(*buf), GFP_KERNEL);
if (!buf)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
status = ice_aq_query_sched_res(hw, sizeof(*buf), buf, NULL);
if (status)
@@ -1341,7 +1341,7 @@ enum ice_status ice_sched_query_res_alloc(struct ice_hw *hw)
sizeof(*hw->layer_info)),
GFP_KERNEL);
if (!hw->layer_info) {
- status = ICE_ERR_NO_MEMORY;
+ status = -ENOMEM;
goto sched_query_out;
}
@@ -1614,31 +1614,31 @@ ice_sched_calc_vsi_child_nodes(struct ice_hw *hw, u16 num_qs, u16 *num_nodes)
* This function adds the VSI child nodes to tree. It gets called for
* LAN and RDMA separately.
*/
-static enum ice_status
+static int
ice_sched_add_vsi_child_nodes(struct ice_port_info *pi, u16 vsi_handle,
struct ice_sched_node *tc_node, u16 *num_nodes,
u8 owner)
{
struct ice_sched_node *parent, *node;
struct ice_hw *hw = pi->hw;
- enum ice_status status;
u32 first_node_teid;
u16 num_added = 0;
u8 i, qgl, vsil;
+ int status;
qgl = ice_sched_get_qgrp_layer(hw);
vsil = ice_sched_get_vsi_layer(hw);
parent = ice_sched_get_vsi_node(pi, tc_node, vsi_handle);
for (i = vsil + 1; i <= qgl; i++) {
if (!parent)
- return ICE_ERR_CFG;
+ return -EIO;
status = ice_sched_add_nodes_to_layer(pi, tc_node, parent, i,
num_nodes[i],
&first_node_teid,
&num_added);
if (status || num_nodes[i] != num_added)
- return ICE_ERR_CFG;
+ return -EIO;
/* The newly added node can be a new parent for the next
* layer nodes
@@ -1717,18 +1717,18 @@ ice_sched_calc_vsi_support_nodes(struct ice_port_info *pi,
* This function adds the VSI supported nodes into Tx tree including the
* VSI, its parent and intermediate nodes in below layers
*/
-static enum ice_status
+static int
ice_sched_add_vsi_support_nodes(struct ice_port_info *pi, u16 vsi_handle,
struct ice_sched_node *tc_node, u16 *num_nodes)
{
struct ice_sched_node *parent = tc_node;
- enum ice_status status;
u32 first_node_teid;
u16 num_added = 0;
u8 i, vsil;
+ int status;
if (!pi)
- return ICE_ERR_PARAM;
+ return -EINVAL;
vsil = ice_sched_get_vsi_layer(pi->hw);
for (i = pi->hw->sw_entry_point_layer; i <= vsil; i++) {
@@ -1737,7 +1737,7 @@ ice_sched_add_vsi_support_nodes(struct ice_port_info *pi, u16 vsi_handle,
&first_node_teid,
&num_added);
if (status || num_nodes[i] != num_added)
- return ICE_ERR_CFG;
+ return -EIO;
/* The newly added node can be a new parent for the next
* layer nodes
@@ -1749,7 +1749,7 @@ ice_sched_add_vsi_support_nodes(struct ice_port_info *pi, u16 vsi_handle,
parent = parent->children[0];
if (!parent)
- return ICE_ERR_CFG;
+ return -EIO;
if (i == vsil)
parent->vsi_handle = vsi_handle;
@@ -1766,7 +1766,7 @@ ice_sched_add_vsi_support_nodes(struct ice_port_info *pi, u16 vsi_handle,
*
* This function adds a new VSI into scheduler tree
*/
-static enum ice_status
+static int
ice_sched_add_vsi_to_topo(struct ice_port_info *pi, u16 vsi_handle, u8 tc)
{
u16 num_nodes[ICE_AQC_TOPO_MAX_LEVEL_NUM] = { 0 };
@@ -1774,7 +1774,7 @@ ice_sched_add_vsi_to_topo(struct ice_port_info *pi, u16 vsi_handle, u8 tc)
tc_node = ice_sched_get_tc_node(pi, tc);
if (!tc_node)
- return ICE_ERR_PARAM;
+ return -EINVAL;
/* calculate number of supported nodes needed for this VSI */
ice_sched_calc_vsi_support_nodes(pi, tc_node, num_nodes);
@@ -1794,7 +1794,7 @@ ice_sched_add_vsi_to_topo(struct ice_port_info *pi, u16 vsi_handle, u8 tc)
*
* This function updates the VSI child nodes based on the number of queues
*/
-static enum ice_status
+static int
ice_sched_update_vsi_child_nodes(struct ice_port_info *pi, u16 vsi_handle,
u8 tc, u16 new_numqs, u8 owner)
{
@@ -1802,21 +1802,21 @@ ice_sched_update_vsi_child_nodes(struct ice_port_info *pi, u16 vsi_handle,
struct ice_sched_node *vsi_node;
struct ice_sched_node *tc_node;
struct ice_vsi_ctx *vsi_ctx;
- enum ice_status status = 0;
struct ice_hw *hw = pi->hw;
u16 prev_numqs;
+ int status = 0;
tc_node = ice_sched_get_tc_node(pi, tc);
if (!tc_node)
- return ICE_ERR_CFG;
+ return -EIO;
vsi_node = ice_sched_get_vsi_node(pi, tc_node, vsi_handle);
if (!vsi_node)
- return ICE_ERR_CFG;
+ return -EIO;
vsi_ctx = ice_get_vsi_ctx(hw, vsi_handle);
if (!vsi_ctx)
- return ICE_ERR_PARAM;
+ return -EINVAL;
if (owner == ICE_SCHED_NODE_OWNER_LAN)
prev_numqs = vsi_ctx->sched.max_lanq[tc];
@@ -1869,22 +1869,22 @@ ice_sched_update_vsi_child_nodes(struct ice_port_info *pi, u16 vsi_handle,
* enabled and VSI is in suspended state then resume the VSI back. If TC is
* disabled then suspend the VSI if it is not already.
*/
-enum ice_status
+int
ice_sched_cfg_vsi(struct ice_port_info *pi, u16 vsi_handle, u8 tc, u16 maxqs,
u8 owner, bool enable)
{
struct ice_sched_node *vsi_node, *tc_node;
struct ice_vsi_ctx *vsi_ctx;
- enum ice_status status = 0;
struct ice_hw *hw = pi->hw;
+ int status = 0;
ice_debug(pi->hw, ICE_DBG_SCHED, "add/config VSI %d\n", vsi_handle);
tc_node = ice_sched_get_tc_node(pi, tc);
if (!tc_node)
- return ICE_ERR_PARAM;
+ return -EINVAL;
vsi_ctx = ice_get_vsi_ctx(hw, vsi_handle);
if (!vsi_ctx)
- return ICE_ERR_PARAM;
+ return -EINVAL;
vsi_node = ice_sched_get_vsi_node(pi, tc_node, vsi_handle);
/* suspend the VSI if TC is not enabled */
@@ -1908,7 +1908,7 @@ ice_sched_cfg_vsi(struct ice_port_info *pi, u16 vsi_handle, u8 tc, u16 maxqs,
vsi_node = ice_sched_get_vsi_node(pi, tc_node, vsi_handle);
if (!vsi_node)
- return ICE_ERR_CFG;
+ return -EIO;
vsi_ctx->sched.vsi_node[tc] = vsi_node;
vsi_node->in_use = true;
@@ -1993,11 +1993,11 @@ static bool ice_sched_is_leaf_node_present(struct ice_sched_node *node)
* This function removes the VSI and its LAN or RDMA children nodes from the
* scheduler tree.
*/
-static enum ice_status
+static int
ice_sched_rm_vsi_cfg(struct ice_port_info *pi, u16 vsi_handle, u8 owner)
{
- enum ice_status status = ICE_ERR_PARAM;
struct ice_vsi_ctx *vsi_ctx;
+ int status = -EINVAL;
u8 i;
ice_debug(pi->hw, ICE_DBG_SCHED, "removing VSI %d\n", vsi_handle);
@@ -2022,7 +2022,7 @@ ice_sched_rm_vsi_cfg(struct ice_port_info *pi, u16 vsi_handle, u8 owner)
if (ice_sched_is_leaf_node_present(vsi_node)) {
ice_debug(pi->hw, ICE_DBG_SCHED, "VSI has leaf nodes in TC %d\n", i);
- status = ICE_ERR_IN_USE;
+ status = -EBUSY;
goto exit_sched_rm_vsi_cfg;
}
while (j < vsi_node->num_children) {
@@ -2065,7 +2065,7 @@ ice_sched_rm_vsi_cfg(struct ice_port_info *pi, u16 vsi_handle, u8 owner)
* This function clears the VSI and its LAN children nodes from scheduler tree
* for all TCs.
*/
-enum ice_status ice_rm_vsi_lan_cfg(struct ice_port_info *pi, u16 vsi_handle)
+int ice_rm_vsi_lan_cfg(struct ice_port_info *pi, u16 vsi_handle)
{
return ice_sched_rm_vsi_cfg(pi, vsi_handle, ICE_SCHED_NODE_OWNER_LAN);
}
@@ -2078,7 +2078,7 @@ enum ice_status ice_rm_vsi_lan_cfg(struct ice_port_info *pi, u16 vsi_handle)
* This function clears the VSI and its RDMA children nodes from scheduler tree
* for all TCs.
*/
-enum ice_status ice_rm_vsi_rdma_cfg(struct ice_port_info *pi, u16 vsi_handle)
+int ice_rm_vsi_rdma_cfg(struct ice_port_info *pi, u16 vsi_handle)
{
return ice_sched_rm_vsi_cfg(pi, vsi_handle, ICE_SCHED_NODE_OWNER_RDMA);
}
@@ -2188,36 +2188,36 @@ ice_sched_update_parent(struct ice_sched_node *new_parent,
*
* This function move the child nodes to a given parent.
*/
-static enum ice_status
+static int
ice_sched_move_nodes(struct ice_port_info *pi, struct ice_sched_node *parent,
u16 num_items, u32 *list)
{
struct ice_aqc_move_elem *buf;
struct ice_sched_node *node;
- enum ice_status status = 0;
u16 i, grps_movd = 0;
struct ice_hw *hw;
+ int status = 0;
u16 buf_len;
hw = pi->hw;
if (!parent || !num_items)
- return ICE_ERR_PARAM;
+ return -EINVAL;
/* Does parent have enough space */
if (parent->num_children + num_items >
hw->max_children[parent->tx_sched_layer])
- return ICE_ERR_AQ_FULL;
+ return -ENOSPC;
buf_len = struct_size(buf, teid, 1);
buf = kzalloc(buf_len, GFP_KERNEL);
if (!buf)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
for (i = 0; i < num_items; i++) {
node = ice_sched_find_node_by_teid(pi->root, list[i]);
if (!node) {
- status = ICE_ERR_PARAM;
+ status = -EINVAL;
goto move_err_exit;
}
@@ -2228,7 +2228,7 @@ ice_sched_move_nodes(struct ice_port_info *pi, struct ice_sched_node *parent,
status = ice_aq_move_sched_elems(hw, 1, buf, buf_len,
&grps_movd, NULL);
if (status && grps_movd != 1) {
- status = ICE_ERR_CFG;
+ status = -EIO;
goto move_err_exit;
}
@@ -2251,28 +2251,28 @@ ice_sched_move_nodes(struct ice_port_info *pi, struct ice_sched_node *parent,
* This function moves a VSI to an aggregator node or its subtree.
* Intermediate nodes may be created if required.
*/
-static enum ice_status
+static int
ice_sched_move_vsi_to_agg(struct ice_port_info *pi, u16 vsi_handle, u32 agg_id,
u8 tc)
{
struct ice_sched_node *vsi_node, *agg_node, *tc_node, *parent;
u16 num_nodes[ICE_AQC_TOPO_MAX_LEVEL_NUM] = { 0 };
u32 first_node_teid, vsi_teid;
- enum ice_status status;
u16 num_nodes_added;
u8 aggl, vsil, i;
+ int status;
tc_node = ice_sched_get_tc_node(pi, tc);
if (!tc_node)
- return ICE_ERR_CFG;
+ return -EIO;
agg_node = ice_sched_get_agg_node(pi, tc_node, agg_id);
if (!agg_node)
- return ICE_ERR_DOES_NOT_EXIST;
+ return -ENOENT;
vsi_node = ice_sched_get_vsi_node(pi, tc_node, vsi_handle);
if (!vsi_node)
- return ICE_ERR_DOES_NOT_EXIST;
+ return -ENOENT;
/* Is this VSI already part of given aggregator? */
if (ice_sched_find_node_in_subtree(pi->hw, agg_node, vsi_node))
@@ -2302,7 +2302,7 @@ ice_sched_move_vsi_to_agg(struct ice_port_info *pi, u16 vsi_handle, u32 agg_id,
&first_node_teid,
&num_nodes_added);
if (status || num_nodes[i] != num_nodes_added)
- return ICE_ERR_CFG;
+ return -EIO;
/* The newly added node can be a new parent for the next
* layer nodes
@@ -2314,7 +2314,7 @@ ice_sched_move_vsi_to_agg(struct ice_port_info *pi, u16 vsi_handle, u32 agg_id,
parent = parent->children[0];
if (!parent)
- return ICE_ERR_CFG;
+ return -EIO;
}
move_nodes:
@@ -2333,14 +2333,14 @@ ice_sched_move_vsi_to_agg(struct ice_port_info *pi, u16 vsi_handle, u32 agg_id,
* aggregator VSI info based on passed in boolean parameter rm_vsi_info. The
* caller holds the scheduler lock.
*/
-static enum ice_status
+static int
ice_move_all_vsi_to_dflt_agg(struct ice_port_info *pi,
struct ice_sched_agg_info *agg_info, u8 tc,
bool rm_vsi_info)
{
struct ice_sched_agg_vsi_info *agg_vsi_info;
struct ice_sched_agg_vsi_info *tmp;
- enum ice_status status = 0;
+ int status = 0;
list_for_each_entry_safe(agg_vsi_info, tmp, &agg_info->agg_vsi_list,
list_entry) {
@@ -2397,7 +2397,7 @@ ice_sched_is_agg_inuse(struct ice_port_info *pi, struct ice_sched_node *node)
* This function removes the aggregator node and intermediate nodes if any
* from the given TC
*/
-static enum ice_status
+static int
ice_sched_rm_agg_cfg(struct ice_port_info *pi, u32 agg_id, u8 tc)
{
struct ice_sched_node *tc_node, *agg_node;
@@ -2405,15 +2405,15 @@ ice_sched_rm_agg_cfg(struct ice_port_info *pi, u32 agg_id, u8 tc)
tc_node = ice_sched_get_tc_node(pi, tc);
if (!tc_node)
- return ICE_ERR_CFG;
+ return -EIO;
agg_node = ice_sched_get_agg_node(pi, tc_node, agg_id);
if (!agg_node)
- return ICE_ERR_DOES_NOT_EXIST;
+ return -ENOENT;
/* Can't remove the aggregator node if it has children */
if (ice_sched_is_agg_inuse(pi, agg_node))
- return ICE_ERR_IN_USE;
+ return -EBUSY;
/* need to remove the whole subtree if aggregator node is the
* only child.
@@ -2422,7 +2422,7 @@ ice_sched_rm_agg_cfg(struct ice_port_info *pi, u32 agg_id, u8 tc)
struct ice_sched_node *parent = agg_node->parent;
if (!parent)
- return ICE_ERR_CFG;
+ return -EIO;
if (parent->num_children > 1)
break;
@@ -2445,11 +2445,11 @@ ice_sched_rm_agg_cfg(struct ice_port_info *pi, u32 agg_id, u8 tc)
* the aggregator configuration completely for requested TC. The caller needs
* to hold the scheduler lock.
*/
-static enum ice_status
+static int
ice_rm_agg_cfg_tc(struct ice_port_info *pi, struct ice_sched_agg_info *agg_info,
u8 tc, bool rm_vsi_info)
{
- enum ice_status status = 0;
+ int status = 0;
/* If nothing to remove - return success */
if (!ice_is_tc_ena(agg_info->tc_bitmap[0], tc))
@@ -2478,7 +2478,7 @@ ice_rm_agg_cfg_tc(struct ice_port_info *pi, struct ice_sched_agg_info *agg_info,
* Save aggregator TC bitmap. This function needs to be called with scheduler
* lock held.
*/
-static enum ice_status
+static int
ice_save_agg_tc_bitmap(struct ice_port_info *pi, u32 agg_id,
unsigned long *tc_bitmap)
{
@@ -2486,7 +2486,7 @@ ice_save_agg_tc_bitmap(struct ice_port_info *pi, u32 agg_id,
agg_info = ice_get_agg_info(pi->hw, agg_id);
if (!agg_info)
- return ICE_ERR_PARAM;
+ return -EINVAL;
bitmap_copy(agg_info->replay_tc_bitmap, tc_bitmap,
ICE_MAX_TRAFFIC_CLASS);
return 0;
@@ -2501,20 +2501,20 @@ ice_save_agg_tc_bitmap(struct ice_port_info *pi, u32 agg_id,
* This function creates an aggregator node and intermediate nodes if required
* for the given TC
*/
-static enum ice_status
+static int
ice_sched_add_agg_cfg(struct ice_port_info *pi, u32 agg_id, u8 tc)
{
struct ice_sched_node *parent, *agg_node, *tc_node;
u16 num_nodes[ICE_AQC_TOPO_MAX_LEVEL_NUM] = { 0 };
- enum ice_status status = 0;
struct ice_hw *hw = pi->hw;
u32 first_node_teid;
u16 num_nodes_added;
+ int status = 0;
u8 i, aggl;
tc_node = ice_sched_get_tc_node(pi, tc);
if (!tc_node)
- return ICE_ERR_CFG;
+ return -EIO;
agg_node = ice_sched_get_agg_node(pi, tc_node, agg_id);
/* Does Agg node already exist ? */
@@ -2549,14 +2549,14 @@ ice_sched_add_agg_cfg(struct ice_port_info *pi, u32 agg_id, u8 tc)
parent = tc_node;
for (i = hw->sw_entry_point_layer; i <= aggl; i++) {
if (!parent)
- return ICE_ERR_CFG;
+ return -EIO;
status = ice_sched_add_nodes_to_layer(pi, tc_node, parent, i,
num_nodes[i],
&first_node_teid,
&num_nodes_added);
if (status || num_nodes[i] != num_nodes_added)
- return ICE_ERR_CFG;
+ return -EIO;
/* The newly added node can be a new parent for the next
* layer nodes
@@ -2591,13 +2591,13 @@ ice_sched_add_agg_cfg(struct ice_port_info *pi, u32 agg_id, u8 tc)
* resources and remove aggregator ID.
* This function needs to be called with scheduler lock held.
*/
-static enum ice_status
+static int
ice_sched_cfg_agg(struct ice_port_info *pi, u32 agg_id,
enum ice_agg_type agg_type, unsigned long *tc_bitmap)
{
struct ice_sched_agg_info *agg_info;
- enum ice_status status = 0;
struct ice_hw *hw = pi->hw;
+ int status = 0;
u8 tc;
agg_info = ice_get_agg_info(hw, agg_id);
@@ -2606,7 +2606,7 @@ ice_sched_cfg_agg(struct ice_port_info *pi, u32 agg_id,
agg_info = devm_kzalloc(ice_hw_to_dev(hw), sizeof(*agg_info),
GFP_KERNEL);
if (!agg_info)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
agg_info->agg_id = agg_id;
agg_info->agg_type = agg_type;
@@ -2653,19 +2653,17 @@ ice_sched_cfg_agg(struct ice_port_info *pi, u32 agg_id,
*
* This function configures aggregator node(s).
*/
-enum ice_status
+int
ice_cfg_agg(struct ice_port_info *pi, u32 agg_id, enum ice_agg_type agg_type,
u8 tc_bitmap)
{
unsigned long bitmap = tc_bitmap;
- enum ice_status status;
+ int status;
mutex_lock(&pi->sched_lock);
- status = ice_sched_cfg_agg(pi, agg_id, agg_type,
- (unsigned long *)&bitmap);
+ status = ice_sched_cfg_agg(pi, agg_id, agg_type, &bitmap);
if (!status)
- status = ice_save_agg_tc_bitmap(pi, agg_id,
- (unsigned long *)&bitmap);
+ status = ice_save_agg_tc_bitmap(pi, agg_id, &bitmap);
mutex_unlock(&pi->sched_lock);
return status;
}
@@ -2724,7 +2722,7 @@ ice_get_vsi_agg_info(struct ice_hw *hw, u16 vsi_handle)
* Save VSI to aggregator TC bitmap. This function needs to call with scheduler
* lock held.
*/
-static enum ice_status
+static int
ice_save_agg_vsi_tc_bitmap(struct ice_port_info *pi, u32 agg_id, u16 vsi_handle,
unsigned long *tc_bitmap)
{
@@ -2733,11 +2731,11 @@ ice_save_agg_vsi_tc_bitmap(struct ice_port_info *pi, u32 agg_id, u16 vsi_handle,
agg_info = ice_get_agg_info(pi->hw, agg_id);
if (!agg_info)
- return ICE_ERR_PARAM;
+ return -EINVAL;
/* check if entry already exist */
agg_vsi_info = ice_get_agg_vsi_info(agg_info, vsi_handle);
if (!agg_vsi_info)
- return ICE_ERR_PARAM;
+ return -EINVAL;
bitmap_copy(agg_vsi_info->replay_tc_bitmap, tc_bitmap,
ICE_MAX_TRAFFIC_CLASS);
return 0;
@@ -2754,21 +2752,21 @@ ice_save_agg_vsi_tc_bitmap(struct ice_port_info *pi, u32 agg_id, u16 vsi_handle,
* already associated to the aggregator node then no operation is performed on
* the tree. This function needs to be called with scheduler lock held.
*/
-static enum ice_status
+static int
ice_sched_assoc_vsi_to_agg(struct ice_port_info *pi, u32 agg_id,
u16 vsi_handle, unsigned long *tc_bitmap)
{
struct ice_sched_agg_vsi_info *agg_vsi_info, *old_agg_vsi_info = NULL;
struct ice_sched_agg_info *agg_info, *old_agg_info;
- enum ice_status status = 0;
struct ice_hw *hw = pi->hw;
+ int status = 0;
u8 tc;
if (!ice_is_vsi_valid(pi->hw, vsi_handle))
- return ICE_ERR_PARAM;
+ return -EINVAL;
agg_info = ice_get_agg_info(hw, agg_id);
if (!agg_info)
- return ICE_ERR_PARAM;
+ return -EINVAL;
/* If the VSI is already part of another aggregator then update
* its VSI info list
*/
@@ -2790,7 +2788,7 @@ ice_sched_assoc_vsi_to_agg(struct ice_port_info *pi, u32 agg_id,
agg_vsi_info = devm_kzalloc(ice_hw_to_dev(hw),
sizeof(*agg_vsi_info), GFP_KERNEL);
if (!agg_vsi_info)
- return ICE_ERR_PARAM;
+ return -EINVAL;
/* add VSI ID into the aggregator list */
agg_vsi_info->vsi_handle = vsi_handle;
@@ -2851,14 +2849,14 @@ static void ice_sched_rm_unused_rl_prof(struct ice_port_info *pi)
* returns success or error on config sched element failure. The caller
* needs to hold scheduler lock.
*/
-static enum ice_status
+static int
ice_sched_update_elem(struct ice_hw *hw, struct ice_sched_node *node,
struct ice_aqc_txsched_elem_data *info)
{
struct ice_aqc_txsched_elem_data buf;
- enum ice_status status;
u16 elem_cfgd = 0;
u16 num_elems = 1;
+ int status;
buf = *info;
/* Parent TEID is reserved field in this aq call */
@@ -2874,7 +2872,7 @@ ice_sched_update_elem(struct ice_hw *hw, struct ice_sched_node *node,
&elem_cfgd, NULL);
if (status || elem_cfgd != num_elems) {
ice_debug(hw, ICE_DBG_SCHED, "Config sched elem error\n");
- return ICE_ERR_CFG;
+ return -EIO;
}
/* Config success case */
@@ -2893,7 +2891,7 @@ ice_sched_update_elem(struct ice_hw *hw, struct ice_sched_node *node,
*
* This function configures node element's BW allocation.
*/
-static enum ice_status
+static int
ice_sched_cfg_node_bw_alloc(struct ice_hw *hw, struct ice_sched_node *node,
enum ice_rl_type rl_type, u16 bw_alloc)
{
@@ -2909,7 +2907,7 @@ ice_sched_cfg_node_bw_alloc(struct ice_hw *hw, struct ice_sched_node *node,
data->valid_sections |= ICE_AQC_ELEM_VALID_EIR;
data->eir_bw.bw_alloc = cpu_to_le16(bw_alloc);
} else {
- return ICE_ERR_PARAM;
+ return -EINVAL;
}
/* Configure element */
@@ -2925,12 +2923,12 @@ ice_sched_cfg_node_bw_alloc(struct ice_hw *hw, struct ice_sched_node *node,
*
* Move or associate VSI to a new or default aggregator node.
*/
-enum ice_status
+int
ice_move_vsi_to_agg(struct ice_port_info *pi, u32 agg_id, u16 vsi_handle,
u8 tc_bitmap)
{
unsigned long bitmap = tc_bitmap;
- enum ice_status status;
+ int status;
mutex_lock(&pi->sched_lock);
status = ice_sched_assoc_vsi_to_agg(pi, agg_id, vsi_handle,
@@ -3098,12 +3096,12 @@ static u16 ice_sched_calc_wakeup(struct ice_hw *hw, s32 bw)
*
* This function converts the BW to profile structure format.
*/
-static enum ice_status
+static int
ice_sched_bw_to_rl_profile(struct ice_hw *hw, u32 bw,
struct ice_aqc_rl_profile_elem *profile)
{
- enum ice_status status = ICE_ERR_PARAM;
s64 bytes_per_sec, ts_rate, mv_tmp;
+ int status = -EINVAL;
bool found = false;
s32 encode = 0;
s64 mv = 0;
@@ -3150,7 +3148,7 @@ ice_sched_bw_to_rl_profile(struct ice_hw *hw, u32 bw,
profile->rl_encode = cpu_to_le16(encode);
status = 0;
} else {
- status = ICE_ERR_DOES_NOT_EXIST;
+ status = -ENOENT;
}
return status;
@@ -3176,9 +3174,9 @@ ice_sched_add_rl_profile(struct ice_port_info *pi,
struct ice_aqc_rl_profile_info *rl_prof_elem;
u16 profiles_added = 0, num_profiles = 1;
struct ice_aqc_rl_profile_elem *buf;
- enum ice_status status;
struct ice_hw *hw;
u8 profile_type;
+ int status;
if (layer_num >= ICE_AQC_TOPO_MAX_LEVEL_NUM)
return NULL;
@@ -3249,7 +3247,7 @@ ice_sched_add_rl_profile(struct ice_port_info *pi,
*
* This function configures node element's BW limit.
*/
-static enum ice_status
+static int
ice_sched_cfg_node_bw_lmt(struct ice_hw *hw, struct ice_sched_node *node,
enum ice_rl_type rl_type, u16 rl_prof_id)
{
@@ -3268,7 +3266,7 @@ ice_sched_cfg_node_bw_lmt(struct ice_hw *hw, struct ice_sched_node *node,
* hence only one of them may be set for any given element
*/
if (data->valid_sections & ICE_AQC_ELEM_VALID_SHARED)
- return ICE_ERR_CFG;
+ return -EIO;
data->valid_sections |= ICE_AQC_ELEM_VALID_EIR;
data->eir_bw.bw_profile_idx = cpu_to_le16(rl_prof_id);
break;
@@ -3291,7 +3289,7 @@ ice_sched_cfg_node_bw_lmt(struct ice_hw *hw, struct ice_sched_node *node,
if ((data->valid_sections & ICE_AQC_ELEM_VALID_EIR) &&
(le16_to_cpu(data->eir_bw.bw_profile_idx) !=
ICE_SCHED_DFLT_RL_PROF_ID))
- return ICE_ERR_CFG;
+ return -EIO;
/* EIR BW is set to default, disable it */
data->valid_sections &= ~ICE_AQC_ELEM_VALID_EIR;
/* Okay to enable shared BW now */
@@ -3300,7 +3298,7 @@ ice_sched_cfg_node_bw_lmt(struct ice_hw *hw, struct ice_sched_node *node,
break;
default:
/* Unknown rate limit type */
- return ICE_ERR_PARAM;
+ return -EINVAL;
}
/* Configure element */
@@ -3420,15 +3418,15 @@ ice_sched_get_srl_node(struct ice_sched_node *node, u8 srl_layer)
* 'profile_type' and profile ID as 'profile_id'. The caller needs to hold
* scheduler lock.
*/
-static enum ice_status
+static int
ice_sched_rm_rl_profile(struct ice_port_info *pi, u8 layer_num, u8 profile_type,
u16 profile_id)
{
struct ice_aqc_rl_profile_info *rl_prof_elem;
- enum ice_status status = 0;
+ int status = 0;
if (layer_num >= ICE_AQC_TOPO_MAX_LEVEL_NUM)
- return ICE_ERR_PARAM;
+ return -EINVAL;
/* Check the existing list for RL profile */
list_for_each_entry(rl_prof_elem, &pi->rl_prof_list[layer_num],
list_entry)
@@ -3441,11 +3439,11 @@ ice_sched_rm_rl_profile(struct ice_port_info *pi, u8 layer_num, u8 profile_type,
/* Remove old profile ID from database */
status = ice_sched_del_rl_profile(pi->hw, rl_prof_elem);
- if (status && status != ICE_ERR_IN_USE)
+ if (status && status != -EBUSY)
ice_debug(pi->hw, ICE_DBG_SCHED, "Remove rl profile failed\n");
break;
}
- if (status == ICE_ERR_IN_USE)
+ if (status == -EBUSY)
status = 0;
return status;
}
@@ -3461,16 +3459,16 @@ ice_sched_rm_rl_profile(struct ice_port_info *pi, u8 layer_num, u8 profile_type,
* type CIR, EIR, or SRL to default. This function needs to be called
* with the scheduler lock held.
*/
-static enum ice_status
+static int
ice_sched_set_node_bw_dflt(struct ice_port_info *pi,
struct ice_sched_node *node,
enum ice_rl_type rl_type, u8 layer_num)
{
- enum ice_status status;
struct ice_hw *hw;
u8 profile_type;
u16 rl_prof_id;
u16 old_id;
+ int status;
hw = pi->hw;
switch (rl_type) {
@@ -3488,7 +3486,7 @@ ice_sched_set_node_bw_dflt(struct ice_port_info *pi,
rl_prof_id = ICE_SCHED_NO_SHARED_RL_PROF_ID;
break;
default:
- return ICE_ERR_PARAM;
+ return -EINVAL;
}
/* Save existing RL prof ID for later clean up */
old_id = ice_sched_get_node_rl_prof_id(node, rl_type);
@@ -3518,7 +3516,7 @@ ice_sched_set_node_bw_dflt(struct ice_port_info *pi,
* them may be set for any given element. This function needs to be called
* with the scheduler lock held.
*/
-static enum ice_status
+static int
ice_sched_set_eir_srl_excl(struct ice_port_info *pi,
struct ice_sched_node *node,
u8 layer_num, enum ice_rl_type rl_type, u32 bw)
@@ -3562,14 +3560,14 @@ ice_sched_set_eir_srl_excl(struct ice_port_info *pi,
* node's RL profile ID of type CIR, EIR, or SRL, and removes old profile
* ID from local database. The caller needs to hold scheduler lock.
*/
-static enum ice_status
+static int
ice_sched_set_node_bw(struct ice_port_info *pi, struct ice_sched_node *node,
enum ice_rl_type rl_type, u32 bw, u8 layer_num)
{
struct ice_aqc_rl_profile_info *rl_prof_info;
- enum ice_status status = ICE_ERR_PARAM;
struct ice_hw *hw = pi->hw;
u16 old_id, rl_prof_id;
+ int status = -EINVAL;
rl_prof_info = ice_sched_add_rl_profile(pi, rl_type, bw, layer_num);
if (!rl_prof_info)
@@ -3608,31 +3606,31 @@ ice_sched_set_node_bw(struct ice_port_info *pi, struct ice_sched_node *node,
* It updates node's BW limit parameters like BW RL profile ID of type CIR,
* EIR, or SRL. The caller needs to hold scheduler lock.
*/
-static enum ice_status
+static int
ice_sched_set_node_bw_lmt(struct ice_port_info *pi, struct ice_sched_node *node,
enum ice_rl_type rl_type, u32 bw)
{
struct ice_sched_node *cfg_node = node;
- enum ice_status status;
+ int status;
struct ice_hw *hw;
u8 layer_num;
if (!pi)
- return ICE_ERR_PARAM;
+ return -EINVAL;
hw = pi->hw;
/* Remove unused RL profile IDs from HW and SW DB */
ice_sched_rm_unused_rl_prof(pi);
layer_num = ice_sched_get_rl_prof_layer(pi, rl_type,
node->tx_sched_layer);
if (layer_num >= hw->num_tx_sched_layers)
- return ICE_ERR_PARAM;
+ return -EINVAL;
if (rl_type == ICE_SHARED_BW) {
/* SRL node may be different */
cfg_node = ice_sched_get_srl_node(node, layer_num);
if (!cfg_node)
- return ICE_ERR_CFG;
+ return -EIO;
}
/* EIR BW and Shared BW profiles are mutually exclusive and
* hence only one of them may be set for any given element
@@ -3657,7 +3655,7 @@ ice_sched_set_node_bw_lmt(struct ice_port_info *pi, struct ice_sched_node *node,
* type CIR, EIR, or SRL to default. This function needs to be called
* with the scheduler lock held.
*/
-static enum ice_status
+static int
ice_sched_set_node_bw_dflt_lmt(struct ice_port_info *pi,
struct ice_sched_node *node,
enum ice_rl_type rl_type)
@@ -3675,7 +3673,7 @@ ice_sched_set_node_bw_dflt_lmt(struct ice_port_info *pi,
* behalf of the requested node (first argument). This function needs to be
* called with scheduler lock held.
*/
-static enum ice_status
+static int
ice_sched_validate_srl_node(struct ice_sched_node *node, u8 sel_layer)
{
/* SRL profiles are not available on all layers. Check if the
@@ -3690,7 +3688,7 @@ ice_sched_validate_srl_node(struct ice_sched_node *node, u8 sel_layer)
(node->parent && node->parent->num_children == 1)))
return 0;
- return ICE_ERR_CFG;
+ return -EIO;
}
/**
@@ -3701,7 +3699,7 @@ ice_sched_validate_srl_node(struct ice_sched_node *node, u8 sel_layer)
*
* Save BW information of queue type node for post replay use.
*/
-static enum ice_status
+static int
ice_sched_save_q_bw(struct ice_q_ctx *q_ctx, enum ice_rl_type rl_type, u32 bw)
{
switch (rl_type) {
@@ -3715,7 +3713,7 @@ ice_sched_save_q_bw(struct ice_q_ctx *q_ctx, enum ice_rl_type rl_type, u32 bw)
ice_set_clear_shared_bw(&q_ctx->bw_t_info, bw);
break;
default:
- return ICE_ERR_PARAM;
+ return -EINVAL;
}
return 0;
}
@@ -3731,16 +3729,16 @@ ice_sched_save_q_bw(struct ice_q_ctx *q_ctx, enum ice_rl_type rl_type, u32 bw)
*
* This function sets BW limit of queue scheduling node.
*/
-static enum ice_status
+static int
ice_sched_set_q_bw_lmt(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
u16 q_handle, enum ice_rl_type rl_type, u32 bw)
{
- enum ice_status status = ICE_ERR_PARAM;
struct ice_sched_node *node;
struct ice_q_ctx *q_ctx;
+ int status = -EINVAL;
if (!ice_is_vsi_valid(pi->hw, vsi_handle))
- return ICE_ERR_PARAM;
+ return -EINVAL;
mutex_lock(&pi->sched_lock);
q_ctx = ice_get_lan_q_ctx(pi->hw, vsi_handle, tc, q_handle);
if (!q_ctx)
@@ -3762,7 +3760,7 @@ ice_sched_set_q_bw_lmt(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
sel_layer = ice_sched_get_rl_prof_layer(pi, rl_type,
node->tx_sched_layer);
if (sel_layer >= pi->hw->num_tx_sched_layers) {
- status = ICE_ERR_PARAM;
+ status = -EINVAL;
goto exit_q_bw_lmt;
}
status = ice_sched_validate_srl_node(node, sel_layer);
@@ -3794,7 +3792,7 @@ ice_sched_set_q_bw_lmt(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
*
* This function configures BW limit of queue scheduling node.
*/
-enum ice_status
+int
ice_cfg_q_bw_lmt(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
u16 q_handle, enum ice_rl_type rl_type, u32 bw)
{
@@ -3812,7 +3810,7 @@ ice_cfg_q_bw_lmt(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
*
* This function configures BW default limit of queue scheduling node.
*/
-enum ice_status
+int
ice_cfg_q_bw_dflt_lmt(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
u16 q_handle, enum ice_rl_type rl_type)
{
@@ -3880,13 +3878,13 @@ ice_sched_get_node_by_id_type(struct ice_port_info *pi, u32 id,
* This function sets BW limit of VSI or Aggregator scheduling node
* based on TC information from passed in argument BW.
*/
-static enum ice_status
+int
ice_sched_set_node_bw_lmt_per_tc(struct ice_port_info *pi, u32 id,
enum ice_agg_type agg_type, u8 tc,
enum ice_rl_type rl_type, u32 bw)
{
- enum ice_status status = ICE_ERR_PARAM;
struct ice_sched_node *node;
+ int status = -EINVAL;
if (!pi)
return status;
@@ -3921,7 +3919,7 @@ ice_sched_set_node_bw_lmt_per_tc(struct ice_port_info *pi, u32 id,
* This function configures BW limit of VSI scheduling node based on TC
* information.
*/
-enum ice_status
+int
ice_cfg_vsi_bw_lmt_per_tc(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
enum ice_rl_type rl_type, u32 bw)
{
@@ -3948,7 +3946,7 @@ ice_cfg_vsi_bw_lmt_per_tc(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
* This function configures default BW limit of VSI scheduling node based on TC
* information.
*/
-enum ice_status
+int
ice_cfg_vsi_bw_dflt_lmt_per_tc(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
enum ice_rl_type rl_type)
{
@@ -3976,13 +3974,13 @@ ice_cfg_vsi_bw_dflt_lmt_per_tc(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
* burst size value is used for future rate limit calls. It doesn't change the
* existing or previously created RL profiles.
*/
-enum ice_status ice_cfg_rl_burst_size(struct ice_hw *hw, u32 bytes)
+int ice_cfg_rl_burst_size(struct ice_hw *hw, u32 bytes)
{
u16 burst_size_to_prog;
if (bytes < ICE_MIN_BURST_SIZE_ALLOWED ||
bytes > ICE_MAX_BURST_SIZE_ALLOWED)
- return ICE_ERR_PARAM;
+ return -EINVAL;
if (ice_round_to_num(bytes, 64) <=
ICE_MAX_BURST_SIZE_64_BYTE_GRANULARITY) {
/* 64 byte granularity case */
@@ -4017,13 +4015,13 @@ enum ice_status ice_cfg_rl_burst_size(struct ice_hw *hw, u32 bytes)
* This function configures node element's priority value. It
* needs to be called with scheduler lock held.
*/
-static enum ice_status
+static int
ice_sched_replay_node_prio(struct ice_hw *hw, struct ice_sched_node *node,
u8 priority)
{
struct ice_aqc_txsched_elem_data buf;
struct ice_aqc_txsched_elem *data;
- enum ice_status status;
+ int status;
buf = node->info;
data = &buf.data;
@@ -4044,12 +4042,12 @@ ice_sched_replay_node_prio(struct ice_hw *hw, struct ice_sched_node *node,
* This function restores node's BW from bw_t_info. The caller needs
* to hold the scheduler lock.
*/
-static enum ice_status
+static int
ice_sched_replay_node_bw(struct ice_hw *hw, struct ice_sched_node *node,
struct ice_bw_type_info *bw_t_info)
{
struct ice_port_info *pi = hw->port_info;
- enum ice_status status = ICE_ERR_PARAM;
+ int status = -EINVAL;
u16 bw_alloc;
if (!node)
@@ -4137,7 +4135,7 @@ void ice_sched_replay_agg(struct ice_hw *hw)
if (!bitmap_equal(agg_info->tc_bitmap, agg_info->replay_tc_bitmap,
ICE_MAX_TRAFFIC_CLASS)) {
DECLARE_BITMAP(replay_bitmap, ICE_MAX_TRAFFIC_CLASS);
- enum ice_status status;
+ int status;
bitmap_zero(replay_bitmap, ICE_MAX_TRAFFIC_CLASS);
ice_sched_get_ena_tc_bitmap(pi,
@@ -4191,18 +4189,17 @@ void ice_sched_replay_agg_vsi_preinit(struct ice_hw *hw)
* their node bandwidth information. This function needs to be called with
* scheduler lock held.
*/
-static enum ice_status
-ice_sched_replay_vsi_agg(struct ice_hw *hw, u16 vsi_handle)
+static int ice_sched_replay_vsi_agg(struct ice_hw *hw, u16 vsi_handle)
{
DECLARE_BITMAP(replay_bitmap, ICE_MAX_TRAFFIC_CLASS);
struct ice_sched_agg_vsi_info *agg_vsi_info;
struct ice_port_info *pi = hw->port_info;
struct ice_sched_agg_info *agg_info;
- enum ice_status status;
+ int status;
bitmap_zero(replay_bitmap, ICE_MAX_TRAFFIC_CLASS);
if (!ice_is_vsi_valid(hw, vsi_handle))
- return ICE_ERR_PARAM;
+ return -EINVAL;
agg_info = ice_get_vsi_agg_info(hw, vsi_handle);
if (!agg_info)
return 0; /* Not present in list - default Agg case */
@@ -4233,10 +4230,10 @@ ice_sched_replay_vsi_agg(struct ice_hw *hw, u16 vsi_handle)
* This function replays association of VSI to aggregator type nodes, and
* node bandwidth information.
*/
-enum ice_status ice_replay_vsi_agg(struct ice_hw *hw, u16 vsi_handle)
+int ice_replay_vsi_agg(struct ice_hw *hw, u16 vsi_handle)
{
struct ice_port_info *pi = hw->port_info;
- enum ice_status status;
+ int status;
mutex_lock(&pi->sched_lock);
status = ice_sched_replay_vsi_agg(hw, vsi_handle);
@@ -4252,14 +4249,13 @@ enum ice_status ice_replay_vsi_agg(struct ice_hw *hw, u16 vsi_handle)
* This function replays queue type node bandwidth. This function needs to be
* called with scheduler lock held.
*/
-enum ice_status
-ice_sched_replay_q_bw(struct ice_port_info *pi, struct ice_q_ctx *q_ctx)
+int ice_sched_replay_q_bw(struct ice_port_info *pi, struct ice_q_ctx *q_ctx)
{
struct ice_sched_node *q_node;
/* Following also checks the presence of node in tree */
q_node = ice_sched_find_node_by_teid(pi->root, q_ctx->q_teid);
if (!q_node)
- return ICE_ERR_PARAM;
+ return -EINVAL;
return ice_sched_replay_node_bw(pi->hw, q_node, &q_ctx->bw_t_info);
}
diff --git a/drivers/net/ethernet/intel/ice/ice_sched.h b/drivers/net/ethernet/intel/ice/ice_sched.h
index 6bddcbe..4f91577 100644
--- a/drivers/net/ethernet/intel/ice/ice_sched.h
+++ b/drivers/net/ethernet/intel/ice/ice_sched.h
@@ -65,12 +65,12 @@ struct ice_sched_agg_info {
};
/* FW AQ command calls */
-enum ice_status
+int
ice_aq_query_sched_elems(struct ice_hw *hw, u16 elems_req,
struct ice_aqc_txsched_elem_data *buf, u16 buf_size,
u16 *elems_ret, struct ice_sq_cd *cd);
-enum ice_status ice_sched_init_port(struct ice_port_info *pi);
-enum ice_status ice_sched_query_res_alloc(struct ice_hw *hw);
+int ice_sched_init_port(struct ice_port_info *pi);
+int ice_sched_query_res_alloc(struct ice_hw *hw);
void ice_sched_get_psm_clk_freq(struct ice_hw *hw);
void ice_sched_clear_port(struct ice_port_info *pi);
@@ -79,7 +79,7 @@ void ice_sched_clear_agg(struct ice_hw *hw);
struct ice_sched_node *
ice_sched_find_node_by_teid(struct ice_sched_node *start_node, u32 teid);
-enum ice_status
+int
ice_sched_add_node(struct ice_port_info *pi, u8 layer,
struct ice_aqc_txsched_elem_data *info);
void ice_free_sched_node(struct ice_port_info *pi, struct ice_sched_node *node);
@@ -87,35 +87,38 @@ struct ice_sched_node *ice_sched_get_tc_node(struct ice_port_info *pi, u8 tc);
struct ice_sched_node *
ice_sched_get_free_qparent(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
u8 owner);
-enum ice_status
+int
ice_sched_cfg_vsi(struct ice_port_info *pi, u16 vsi_handle, u8 tc, u16 maxqs,
u8 owner, bool enable);
-enum ice_status ice_rm_vsi_lan_cfg(struct ice_port_info *pi, u16 vsi_handle);
-enum ice_status ice_rm_vsi_rdma_cfg(struct ice_port_info *pi, u16 vsi_handle);
+int ice_rm_vsi_lan_cfg(struct ice_port_info *pi, u16 vsi_handle);
+int ice_rm_vsi_rdma_cfg(struct ice_port_info *pi, u16 vsi_handle);
/* Tx scheduler rate limiter functions */
-enum ice_status
+int
ice_cfg_agg(struct ice_port_info *pi, u32 agg_id,
enum ice_agg_type agg_type, u8 tc_bitmap);
-enum ice_status
+int
ice_move_vsi_to_agg(struct ice_port_info *pi, u32 agg_id, u16 vsi_handle,
u8 tc_bitmap);
-enum ice_status
+int
ice_cfg_q_bw_lmt(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
u16 q_handle, enum ice_rl_type rl_type, u32 bw);
-enum ice_status
+int
ice_cfg_q_bw_dflt_lmt(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
u16 q_handle, enum ice_rl_type rl_type);
-enum ice_status
+int
ice_cfg_vsi_bw_lmt_per_tc(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
enum ice_rl_type rl_type, u32 bw);
-enum ice_status
+int
ice_cfg_vsi_bw_dflt_lmt_per_tc(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
enum ice_rl_type rl_type);
-enum ice_status ice_cfg_rl_burst_size(struct ice_hw *hw, u32 bytes);
+int
+ice_sched_set_node_bw_lmt_per_tc(struct ice_port_info *pi, u32 id,
+ enum ice_agg_type agg_type, u8 tc,
+ enum ice_rl_type rl_type, u32 bw);
+int ice_cfg_rl_burst_size(struct ice_hw *hw, u32 bytes);
void ice_sched_replay_agg_vsi_preinit(struct ice_hw *hw);
void ice_sched_replay_agg(struct ice_hw *hw);
-enum ice_status ice_replay_vsi_agg(struct ice_hw *hw, u16 vsi_handle);
-enum ice_status
-ice_sched_replay_q_bw(struct ice_port_info *pi, struct ice_q_ctx *q_ctx);
+int ice_replay_vsi_agg(struct ice_hw *hw, u16 vsi_handle);
+int ice_sched_replay_q_bw(struct ice_port_info *pi, struct ice_q_ctx *q_ctx);
#endif /* _ICE_SCHED_H_ */
diff --git a/drivers/net/ethernet/intel/ice/ice_sriov.c b/drivers/net/ethernet/intel/ice/ice_sriov.c
index aa11d07..52c6bac 100644
--- a/drivers/net/ethernet/intel/ice/ice_sriov.c
+++ b/drivers/net/ethernet/intel/ice/ice_sriov.c
@@ -18,7 +18,7 @@
* queue and asynchronously sending message via
* ice_sq_send_cmd() function
*/
-enum ice_status
+int
ice_aq_send_msg_to_vf(struct ice_hw *hw, u16 vfid, u32 v_opcode, u32 v_retval,
u8 *msg, u16 msglen, struct ice_sq_cd *cd)
{
@@ -228,7 +228,7 @@ ice_mbx_traverse(struct ice_hw *hw,
* sent per VF and marks the VF as malicious if it exceeds
* the permissible number of messages to send.
*/
-static enum ice_status
+static int
ice_mbx_detect_malvf(struct ice_hw *hw, u16 vf_id,
enum ice_mbx_snapshot_state *new_state,
bool *is_malvf)
@@ -236,7 +236,7 @@ ice_mbx_detect_malvf(struct ice_hw *hw, u16 vf_id,
struct ice_mbx_snapshot *snap = &hw->mbx_snapshot;
if (vf_id >= snap->mbx_vf.vfcntr_len)
- return ICE_ERR_OUT_OF_RANGE;
+ return -EIO;
/* increment the message count in the VF array */
snap->mbx_vf.vf_cntr[vf_id]++;
@@ -297,7 +297,7 @@ static void ice_mbx_reset_snapshot(struct ice_mbx_snapshot *snap)
* Detect: If pending message count exceeds watermark traverse
* the static snapshot and look for a malicious VF.
*/
-enum ice_status
+int
ice_mbx_vf_state_handler(struct ice_hw *hw,
struct ice_mbx_data *mbx_data, u16 vf_id,
bool *is_malvf)
@@ -306,10 +306,10 @@ ice_mbx_vf_state_handler(struct ice_hw *hw,
struct ice_mbx_snap_buffer_data *snap_buf;
struct ice_ctl_q_info *cq = &hw->mailboxq;
enum ice_mbx_snapshot_state new_state;
- enum ice_status status = 0;
+ int status = 0;
if (!is_malvf || !mbx_data)
- return ICE_ERR_BAD_PTR;
+ return -EINVAL;
/* When entering the mailbox state machine assume that the VF
* is not malicious until detected.
@@ -320,7 +320,7 @@ ice_mbx_vf_state_handler(struct ice_hw *hw,
* interrupt is not less than the defined AVF message threshold.
*/
if (mbx_data->max_num_msgs_mbx <= ICE_ASYNC_VF_MSG_THRESHOLD)
- return ICE_ERR_INVAL_SIZE;
+ return -EINVAL;
/* The watermark value should not be lesser than the threshold limit
* set for the number of asynchronous messages a VF can send to mailbox
@@ -329,7 +329,7 @@ ice_mbx_vf_state_handler(struct ice_hw *hw,
*/
if (mbx_data->async_watermark_val < ICE_ASYNC_VF_MSG_THRESHOLD ||
mbx_data->async_watermark_val > mbx_data->max_num_msgs_mbx)
- return ICE_ERR_PARAM;
+ return -EINVAL;
new_state = ICE_MAL_VF_DETECT_STATE_INVALID;
snap_buf = &snap->mbx_buf;
@@ -383,7 +383,7 @@ ice_mbx_vf_state_handler(struct ice_hw *hw,
default:
new_state = ICE_MAL_VF_DETECT_STATE_INVALID;
- status = ICE_ERR_CFG;
+ status = -EIO;
}
snap_buf->state = new_state;
@@ -405,20 +405,20 @@ ice_mbx_vf_state_handler(struct ice_hw *hw,
* the input vf_id against the bitmap to verify if the VF has been
* detected in any previous mailbox iterations.
*/
-enum ice_status
+int
ice_mbx_report_malvf(struct ice_hw *hw, unsigned long *all_malvfs,
u16 bitmap_len, u16 vf_id, bool *report_malvf)
{
if (!all_malvfs || !report_malvf)
- return ICE_ERR_PARAM;
+ return -EINVAL;
*report_malvf = false;
if (bitmap_len < hw->mbx_snapshot.mbx_vf.vfcntr_len)
- return ICE_ERR_INVAL_SIZE;
+ return -EINVAL;
if (vf_id >= bitmap_len)
- return ICE_ERR_OUT_OF_RANGE;
+ return -EIO;
/* If the vf_id is found in the bitmap set bit and boolean to true */
if (!test_and_set_bit(vf_id, all_malvfs))
@@ -441,19 +441,19 @@ ice_mbx_report_malvf(struct ice_hw *hw, unsigned long *all_malvfs,
* that the new VF loaded is not considered malicious before going
* through the overflow detection algorithm.
*/
-enum ice_status
+int
ice_mbx_clear_malvf(struct ice_mbx_snapshot *snap, unsigned long *all_malvfs,
u16 bitmap_len, u16 vf_id)
{
if (!snap || !all_malvfs)
- return ICE_ERR_PARAM;
+ return -EINVAL;
if (bitmap_len < snap->mbx_vf.vfcntr_len)
- return ICE_ERR_INVAL_SIZE;
+ return -EINVAL;
/* Ensure VF ID value is not larger than bitmap or VF counter length */
if (vf_id >= bitmap_len || vf_id >= snap->mbx_vf.vfcntr_len)
- return ICE_ERR_OUT_OF_RANGE;
+ return -EIO;
/* Clear VF ID bit in the bitmap tracking malicious VFs attached to PF */
clear_bit(vf_id, all_malvfs);
@@ -482,7 +482,7 @@ ice_mbx_clear_malvf(struct ice_mbx_snapshot *snap, unsigned long *all_malvfs,
* called to ensure that the vf_count can be compared against the number
* of VFs supported as defined in the functional capabilities of the device.
*/
-enum ice_status ice_mbx_init_snapshot(struct ice_hw *hw, u16 vf_count)
+int ice_mbx_init_snapshot(struct ice_hw *hw, u16 vf_count)
{
struct ice_mbx_snapshot *snap = &hw->mbx_snapshot;
@@ -491,13 +491,13 @@ enum ice_status ice_mbx_init_snapshot(struct ice_hw *hw, u16 vf_count)
* the functional capabilities of the PF.
*/
if (!vf_count || vf_count > hw->func_caps.num_allocd_vfs)
- return ICE_ERR_INVAL_SIZE;
+ return -EINVAL;
snap->mbx_vf.vf_cntr = devm_kcalloc(ice_hw_to_dev(hw), vf_count,
sizeof(*snap->mbx_vf.vf_cntr),
GFP_KERNEL);
if (!snap->mbx_vf.vf_cntr)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
/* Setting the VF counter length to the number of allocated
* VFs for given PF's functional capabilities.
diff --git a/drivers/net/ethernet/intel/ice/ice_sriov.h b/drivers/net/ethernet/intel/ice/ice_sriov.h
index 161dc55..68686a3 100644
--- a/drivers/net/ethernet/intel/ice/ice_sriov.h
+++ b/drivers/net/ethernet/intel/ice/ice_sriov.h
@@ -14,24 +14,24 @@
#define ICE_ASYNC_VF_MSG_THRESHOLD 63
#ifdef CONFIG_PCI_IOV
-enum ice_status
+int
ice_aq_send_msg_to_vf(struct ice_hw *hw, u16 vfid, u32 v_opcode, u32 v_retval,
u8 *msg, u16 msglen, struct ice_sq_cd *cd);
u32 ice_conv_link_speed_to_virtchnl(bool adv_link_support, u16 link_speed);
-enum ice_status
+int
ice_mbx_vf_state_handler(struct ice_hw *hw, struct ice_mbx_data *mbx_data,
u16 vf_id, bool *is_mal_vf);
-enum ice_status
+int
ice_mbx_clear_malvf(struct ice_mbx_snapshot *snap, unsigned long *all_malvfs,
u16 bitmap_len, u16 vf_id);
-enum ice_status ice_mbx_init_snapshot(struct ice_hw *hw, u16 vf_count);
+int ice_mbx_init_snapshot(struct ice_hw *hw, u16 vf_count);
void ice_mbx_deinit_snapshot(struct ice_hw *hw);
-enum ice_status
+int
ice_mbx_report_malvf(struct ice_hw *hw, unsigned long *all_malvfs,
u16 bitmap_len, u16 vf_id, bool *report_malvf);
#else /* CONFIG_PCI_IOV */
-static inline enum ice_status
+static inline int
ice_aq_send_msg_to_vf(struct ice_hw __always_unused *hw,
u16 __always_unused vfid, u32 __always_unused v_opcode,
u32 __always_unused v_retval, u8 __always_unused *msg,
diff --git a/drivers/net/ethernet/intel/ice/ice_status.h b/drivers/net/ethernet/intel/ice/ice_status.h
deleted file mode 100644
index dbf6605..0000000
--- a/drivers/net/ethernet/intel/ice/ice_status.h
+++ /dev/null
@@ -1,44 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-/* Copyright (c) 2018, Intel Corporation. */
-
-#ifndef _ICE_STATUS_H_
-#define _ICE_STATUS_H_
-
-/* Error Codes */
-enum ice_status {
- ICE_SUCCESS = 0,
-
- /* Generic codes : Range -1..-49 */
- ICE_ERR_PARAM = -1,
- ICE_ERR_NOT_IMPL = -2,
- ICE_ERR_NOT_READY = -3,
- ICE_ERR_NOT_SUPPORTED = -4,
- ICE_ERR_BAD_PTR = -5,
- ICE_ERR_INVAL_SIZE = -6,
- ICE_ERR_DEVICE_NOT_SUPPORTED = -8,
- ICE_ERR_RESET_FAILED = -9,
- ICE_ERR_FW_API_VER = -10,
- ICE_ERR_NO_MEMORY = -11,
- ICE_ERR_CFG = -12,
- ICE_ERR_OUT_OF_RANGE = -13,
- ICE_ERR_ALREADY_EXISTS = -14,
- ICE_ERR_DOES_NOT_EXIST = -15,
- ICE_ERR_IN_USE = -16,
- ICE_ERR_MAX_LIMIT = -17,
- ICE_ERR_RESET_ONGOING = -18,
- ICE_ERR_HW_TABLE = -19,
- ICE_ERR_FW_DDP_MISMATCH = -20,
-
- ICE_ERR_NVM = -50,
- ICE_ERR_NVM_CHECKSUM = -51,
- ICE_ERR_BUF_TOO_SHORT = -52,
- ICE_ERR_NVM_BLANK_MODE = -53,
- ICE_ERR_AQ_ERROR = -100,
- ICE_ERR_AQ_TIMEOUT = -101,
- ICE_ERR_AQ_FULL = -102,
- ICE_ERR_AQ_NO_WORK = -103,
- ICE_ERR_AQ_EMPTY = -104,
- ICE_ERR_AQ_FW_CRITICAL = -105,
-};
-
-#endif /* _ICE_STATUS_H_ */
diff --git a/drivers/net/ethernet/intel/ice/ice_switch.c b/drivers/net/ethernet/intel/ice/ice_switch.c
index 183d930..11ae0be 100644
--- a/drivers/net/ethernet/intel/ice/ice_switch.c
+++ b/drivers/net/ethernet/intel/ice/ice_switch.c
@@ -528,7 +528,7 @@ static DECLARE_BITMAP(profile_to_recipe[ICE_MAX_NUM_PROFILES],
* Allocate memory for the entire recipe table and initialize the structures/
* entries corresponding to basic recipes.
*/
-enum ice_status ice_init_def_sw_recp(struct ice_hw *hw)
+int ice_init_def_sw_recp(struct ice_hw *hw)
{
struct ice_sw_recipe *recps;
u8 i;
@@ -536,7 +536,7 @@ enum ice_status ice_init_def_sw_recp(struct ice_hw *hw)
recps = devm_kcalloc(ice_hw_to_dev(hw), ICE_MAX_NUM_RECIPES,
sizeof(*recps), GFP_KERNEL);
if (!recps)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
for (i = 0; i < ICE_MAX_NUM_RECIPES; i++) {
recps[i].root_rid = i;
@@ -576,14 +576,14 @@ enum ice_status ice_init_def_sw_recp(struct ice_hw *hw)
* in response buffer. The caller of this function to use *num_elems while
* parsing the response buffer.
*/
-static enum ice_status
+static int
ice_aq_get_sw_cfg(struct ice_hw *hw, struct ice_aqc_get_sw_cfg_resp_elem *buf,
u16 buf_size, u16 *req_desc, u16 *num_elems,
struct ice_sq_cd *cd)
{
struct ice_aqc_get_sw_cfg *cmd;
struct ice_aq_desc desc;
- enum ice_status status;
+ int status;
ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_get_sw_cfg);
cmd = &desc.params.get_sw_conf;
@@ -606,14 +606,14 @@ ice_aq_get_sw_cfg(struct ice_hw *hw, struct ice_aqc_get_sw_cfg_resp_elem *buf,
*
* Add a VSI context to the hardware (0x0210)
*/
-static enum ice_status
+static int
ice_aq_add_vsi(struct ice_hw *hw, struct ice_vsi_ctx *vsi_ctx,
struct ice_sq_cd *cd)
{
struct ice_aqc_add_update_free_vsi_resp *res;
struct ice_aqc_add_get_update_free_vsi *cmd;
struct ice_aq_desc desc;
- enum ice_status status;
+ int status;
cmd = &desc.params.vsi_cmd;
res = &desc.params.add_update_free_vsi_res;
@@ -650,14 +650,14 @@ ice_aq_add_vsi(struct ice_hw *hw, struct ice_vsi_ctx *vsi_ctx,
*
* Free VSI context info from hardware (0x0213)
*/
-static enum ice_status
+static int
ice_aq_free_vsi(struct ice_hw *hw, struct ice_vsi_ctx *vsi_ctx,
bool keep_vsi_alloc, struct ice_sq_cd *cd)
{
struct ice_aqc_add_update_free_vsi_resp *resp;
struct ice_aqc_add_get_update_free_vsi *cmd;
struct ice_aq_desc desc;
- enum ice_status status;
+ int status;
cmd = &desc.params.vsi_cmd;
resp = &desc.params.add_update_free_vsi_res;
@@ -685,14 +685,14 @@ ice_aq_free_vsi(struct ice_hw *hw, struct ice_vsi_ctx *vsi_ctx,
*
* Update VSI context in the hardware (0x0211)
*/
-static enum ice_status
+static int
ice_aq_update_vsi(struct ice_hw *hw, struct ice_vsi_ctx *vsi_ctx,
struct ice_sq_cd *cd)
{
struct ice_aqc_add_update_free_vsi_resp *resp;
struct ice_aqc_add_get_update_free_vsi *cmd;
struct ice_aq_desc desc;
- enum ice_status status;
+ int status;
cmd = &desc.params.vsi_cmd;
resp = &desc.params.add_update_free_vsi_res;
@@ -832,15 +832,15 @@ void ice_clear_all_vsi_ctx(struct ice_hw *hw)
* If this function gets called after reset for existing VSIs then update
* with the new HW VSI number in the corresponding VSI handle list entry.
*/
-enum ice_status
+int
ice_add_vsi(struct ice_hw *hw, u16 vsi_handle, struct ice_vsi_ctx *vsi_ctx,
struct ice_sq_cd *cd)
{
struct ice_vsi_ctx *tmp_vsi_ctx;
- enum ice_status status;
+ int status;
if (vsi_handle >= ICE_MAX_VSI)
- return ICE_ERR_PARAM;
+ return -EINVAL;
status = ice_aq_add_vsi(hw, vsi_ctx, cd);
if (status)
return status;
@@ -851,7 +851,7 @@ ice_add_vsi(struct ice_hw *hw, u16 vsi_handle, struct ice_vsi_ctx *vsi_ctx,
sizeof(*tmp_vsi_ctx), GFP_KERNEL);
if (!tmp_vsi_ctx) {
ice_aq_free_vsi(hw, vsi_ctx, false, cd);
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
}
*tmp_vsi_ctx = *vsi_ctx;
ice_save_vsi_ctx(hw, vsi_handle, tmp_vsi_ctx);
@@ -873,14 +873,14 @@ ice_add_vsi(struct ice_hw *hw, u16 vsi_handle, struct ice_vsi_ctx *vsi_ctx,
*
* Free VSI context info from hardware as well as from VSI handle list
*/
-enum ice_status
+int
ice_free_vsi(struct ice_hw *hw, u16 vsi_handle, struct ice_vsi_ctx *vsi_ctx,
bool keep_vsi_alloc, struct ice_sq_cd *cd)
{
- enum ice_status status;
+ int status;
if (!ice_is_vsi_valid(hw, vsi_handle))
- return ICE_ERR_PARAM;
+ return -EINVAL;
vsi_ctx->vsi_num = ice_get_hw_vsi_num(hw, vsi_handle);
status = ice_aq_free_vsi(hw, vsi_ctx, keep_vsi_alloc, cd);
if (!status)
@@ -897,12 +897,12 @@ ice_free_vsi(struct ice_hw *hw, u16 vsi_handle, struct ice_vsi_ctx *vsi_ctx,
*
* Update VSI context in the hardware
*/
-enum ice_status
+int
ice_update_vsi(struct ice_hw *hw, u16 vsi_handle, struct ice_vsi_ctx *vsi_ctx,
struct ice_sq_cd *cd)
{
if (!ice_is_vsi_valid(hw, vsi_handle))
- return ICE_ERR_PARAM;
+ return -EINVAL;
vsi_ctx->vsi_num = ice_get_hw_vsi_num(hw, vsi_handle);
return ice_aq_update_vsi(hw, vsi_ctx, cd);
}
@@ -927,7 +927,7 @@ ice_cfg_rdma_fltr(struct ice_hw *hw, u16 vsi_handle, bool enable)
else
ctx->info.q_opt_flags &= ~ICE_AQ_VSI_Q_OPT_PE_FLTR_EN;
- return ice_status_to_errno(ice_update_vsi(hw, vsi_handle, ctx, NULL));
+ return ice_update_vsi(hw, vsi_handle, ctx, NULL);
}
/**
@@ -939,20 +939,20 @@ ice_cfg_rdma_fltr(struct ice_hw *hw, u16 vsi_handle, bool enable)
*
* allocates or free a VSI list resource
*/
-static enum ice_status
+static int
ice_aq_alloc_free_vsi_list(struct ice_hw *hw, u16 *vsi_list_id,
enum ice_sw_lkup_type lkup_type,
enum ice_adminq_opc opc)
{
struct ice_aqc_alloc_free_res_elem *sw_buf;
struct ice_aqc_res_elem *vsi_ele;
- enum ice_status status;
u16 buf_len;
+ int status;
buf_len = struct_size(sw_buf, elem, 1);
sw_buf = devm_kzalloc(ice_hw_to_dev(hw), buf_len, GFP_KERNEL);
if (!sw_buf)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
sw_buf->num_elems = cpu_to_le16(1);
if (lkup_type == ICE_SW_LKUP_MAC ||
@@ -966,7 +966,7 @@ ice_aq_alloc_free_vsi_list(struct ice_hw *hw, u16 *vsi_list_id,
sw_buf->res_type =
cpu_to_le16(ICE_AQC_RES_TYPE_VSI_LIST_PRUNE);
} else {
- status = ICE_ERR_PARAM;
+ status = -EINVAL;
goto ice_aq_alloc_free_vsi_list_exit;
}
@@ -998,17 +998,17 @@ ice_aq_alloc_free_vsi_list(struct ice_hw *hw, u16 *vsi_list_id,
*
* Add(0x02a0)/Update(0x02a1)/Remove(0x02a2) switch rules commands to firmware
*/
-enum ice_status
+int
ice_aq_sw_rules(struct ice_hw *hw, void *rule_list, u16 rule_list_sz,
u8 num_rules, enum ice_adminq_opc opc, struct ice_sq_cd *cd)
{
struct ice_aq_desc desc;
- enum ice_status status;
+ int status;
if (opc != ice_aqc_opc_add_sw_rules &&
opc != ice_aqc_opc_update_sw_rules &&
opc != ice_aqc_opc_remove_sw_rules)
- return ICE_ERR_PARAM;
+ return -EINVAL;
ice_fill_dflt_direct_cmd_desc(&desc, opc);
@@ -1018,7 +1018,7 @@ ice_aq_sw_rules(struct ice_hw *hw, void *rule_list, u16 rule_list_sz,
status = ice_aq_send_cmd(hw, &desc, rule_list, rule_list_sz, cd);
if (opc != ice_aqc_opc_add_sw_rules &&
hw->adminq.sq_last_status == ICE_AQ_RC_ENOENT)
- status = ICE_ERR_DOES_NOT_EXIST;
+ status = -ENOENT;
return status;
}
@@ -1032,7 +1032,7 @@ ice_aq_sw_rules(struct ice_hw *hw, void *rule_list, u16 rule_list_sz,
*
* Add(0x0290)
*/
-static enum ice_status
+static int
ice_aq_add_recipe(struct ice_hw *hw,
struct ice_aqc_recipe_data_elem *s_recipe_list,
u16 num_recipes, struct ice_sq_cd *cd)
@@ -1069,18 +1069,18 @@ ice_aq_add_recipe(struct ice_hw *hw,
* The caller must supply enough space in s_recipe_list to hold all possible
* recipes and *num_recipes must equal ICE_MAX_NUM_RECIPES.
*/
-static enum ice_status
+static int
ice_aq_get_recipe(struct ice_hw *hw,
struct ice_aqc_recipe_data_elem *s_recipe_list,
u16 *num_recipes, u16 recipe_root, struct ice_sq_cd *cd)
{
struct ice_aqc_add_get_recipe *cmd;
struct ice_aq_desc desc;
- enum ice_status status;
u16 buf_size;
+ int status;
if (*num_recipes != ICE_MAX_NUM_RECIPES)
- return ICE_ERR_PARAM;
+ return -EINVAL;
cmd = &desc.params.add_get_recipe;
ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_get_recipe);
@@ -1104,7 +1104,7 @@ ice_aq_get_recipe(struct ice_hw *hw,
* @cd: pointer to command details structure or NULL
* Recipe to profile association (0x0291)
*/
-static enum ice_status
+static int
ice_aq_map_recipe_to_profile(struct ice_hw *hw, u32 profile_id, u8 *r_bitmap,
struct ice_sq_cd *cd)
{
@@ -1130,13 +1130,13 @@ ice_aq_map_recipe_to_profile(struct ice_hw *hw, u32 profile_id, u8 *r_bitmap,
* @cd: pointer to command details structure or NULL
* Associate profile ID with given recipe (0x0293)
*/
-static enum ice_status
+static int
ice_aq_get_recipe_to_profile(struct ice_hw *hw, u32 profile_id, u8 *r_bitmap,
struct ice_sq_cd *cd)
{
struct ice_aqc_recipe_to_profile *cmd;
struct ice_aq_desc desc;
- enum ice_status status;
+ int status;
cmd = &desc.params.recipe_to_profile;
ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_get_recipe_to_profile);
@@ -1154,16 +1154,16 @@ ice_aq_get_recipe_to_profile(struct ice_hw *hw, u32 profile_id, u8 *r_bitmap,
* @hw: pointer to the hardware structure
* @rid: recipe ID returned as response to AQ call
*/
-static enum ice_status ice_alloc_recipe(struct ice_hw *hw, u16 *rid)
+static int ice_alloc_recipe(struct ice_hw *hw, u16 *rid)
{
struct ice_aqc_alloc_free_res_elem *sw_buf;
- enum ice_status status;
u16 buf_len;
+ int status;
buf_len = struct_size(sw_buf, elem, 1);
sw_buf = kzalloc(buf_len, GFP_KERNEL);
if (!sw_buf)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
sw_buf->num_elems = cpu_to_le16(1);
sw_buf->res_type = cpu_to_le16((ICE_AQC_RES_TYPE_RECIPE <<
@@ -1230,7 +1230,7 @@ ice_collect_result_idx(struct ice_aqc_recipe_data_elem *buf,
* bookkeeping so that we have a current list of all the recipes that are
* programmed in the firmware.
*/
-static enum ice_status
+static int
ice_get_recp_frm_fw(struct ice_hw *hw, struct ice_sw_recipe *recps, u8 rid,
bool *refresh_required)
{
@@ -1238,16 +1238,16 @@ ice_get_recp_frm_fw(struct ice_hw *hw, struct ice_sw_recipe *recps, u8 rid,
struct ice_aqc_recipe_data_elem *tmp;
u16 num_recps = ICE_MAX_NUM_RECIPES;
struct ice_prot_lkup_ext *lkup_exts;
- enum ice_status status;
u8 fv_word_idx = 0;
u16 sub_recps;
+ int status;
bitmap_zero(result_bm, ICE_MAX_FV_WORDS);
/* we need a buffer big enough to accommodate all the recipes */
tmp = kcalloc(ICE_MAX_NUM_RECIPES, sizeof(*tmp), GFP_KERNEL);
if (!tmp)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
tmp[0].recipe_indx = rid;
status = ice_aq_get_recipe(hw, tmp, &num_recps, rid, NULL);
@@ -1284,7 +1284,7 @@ ice_get_recp_frm_fw(struct ice_hw *hw, struct ice_sw_recipe *recps, u8 rid,
rg_entry = devm_kzalloc(ice_hw_to_dev(hw), sizeof(*rg_entry),
GFP_KERNEL);
if (!rg_entry) {
- status = ICE_ERR_NO_MEMORY;
+ status = -ENOMEM;
goto err_unroll;
}
@@ -1364,7 +1364,7 @@ ice_get_recp_frm_fw(struct ice_hw *hw, struct ice_sw_recipe *recps, u8 rid,
recps[rid].n_grp_count * sizeof(*recps[rid].root_buf),
GFP_KERNEL);
if (!recps[rid].root_buf) {
- status = ICE_ERR_NO_MEMORY;
+ status = -ENOMEM;
goto err_unroll;
}
@@ -1407,19 +1407,19 @@ ice_init_port_info(struct ice_port_info *pi, u16 vsi_port_num, u8 type,
/* ice_get_initial_sw_cfg - Get initial port and default VSI data
* @hw: pointer to the hardware structure
*/
-enum ice_status ice_get_initial_sw_cfg(struct ice_hw *hw)
+int ice_get_initial_sw_cfg(struct ice_hw *hw)
{
struct ice_aqc_get_sw_cfg_resp_elem *rbuf;
- enum ice_status status;
u16 req_desc = 0;
u16 num_elems;
+ int status;
u16 i;
rbuf = devm_kzalloc(ice_hw_to_dev(hw), ICE_SW_CFG_MAX_BUF_LEN,
GFP_KERNEL);
if (!rbuf)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
/* Multiple calls to ice_aq_get_sw_cfg may be required
* to get all the switch configuration information. The need
@@ -1670,7 +1670,7 @@ ice_fill_sw_rule(struct ice_hw *hw, struct ice_fltr_info *f_info,
* Create a large action to hold software marker and update the switch rule
* entry pointed by m_ent with newly created large action
*/
-static enum ice_status
+static int
ice_add_marker_act(struct ice_hw *hw, struct ice_fltr_mgmt_list_entry *m_ent,
u16 sw_marker, u16 l_id)
{
@@ -1681,14 +1681,14 @@ ice_add_marker_act(struct ice_hw *hw, struct ice_fltr_mgmt_list_entry *m_ent,
* 3. GENERIC VALUE action to hold the software marker ID
*/
const u16 num_lg_acts = 3;
- enum ice_status status;
u16 lg_act_size;
u16 rules_size;
+ int status;
u32 act;
u16 id;
if (m_ent->fltr_info.lkup_type != ICE_SW_LKUP_MAC)
- return ICE_ERR_PARAM;
+ return -EINVAL;
/* Create two back-to-back switch rules and submit them to the HW using
* one memory buffer:
@@ -1699,7 +1699,7 @@ ice_add_marker_act(struct ice_hw *hw, struct ice_fltr_mgmt_list_entry *m_ent,
rules_size = lg_act_size + ICE_SW_RULE_RX_TX_ETH_HDR_SIZE;
lg_act = devm_kzalloc(ice_hw_to_dev(hw), rules_size, GFP_KERNEL);
if (!lg_act)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
rx_tx = (struct ice_aqc_sw_rules_elem *)((u8 *)lg_act + lg_act_size);
@@ -1808,19 +1808,19 @@ ice_create_vsi_list_map(struct ice_hw *hw, u16 *vsi_handle_arr, u16 num_vsi,
* Call AQ command to add a new switch rule or update existing switch rule
* using the given VSI list ID
*/
-static enum ice_status
+static int
ice_update_vsi_list_rule(struct ice_hw *hw, u16 *vsi_handle_arr, u16 num_vsi,
u16 vsi_list_id, bool remove, enum ice_adminq_opc opc,
enum ice_sw_lkup_type lkup_type)
{
struct ice_aqc_sw_rules_elem *s_rule;
- enum ice_status status;
u16 s_rule_size;
u16 rule_type;
+ int status;
int i;
if (!num_vsi)
- return ICE_ERR_PARAM;
+ return -EINVAL;
if (lkup_type == ICE_SW_LKUP_MAC ||
lkup_type == ICE_SW_LKUP_MAC_VLAN ||
@@ -1834,15 +1834,15 @@ ice_update_vsi_list_rule(struct ice_hw *hw, u16 *vsi_handle_arr, u16 num_vsi,
rule_type = remove ? ICE_AQC_SW_RULES_T_PRUNE_LIST_CLEAR :
ICE_AQC_SW_RULES_T_PRUNE_LIST_SET;
else
- return ICE_ERR_PARAM;
+ return -EINVAL;
s_rule_size = (u16)ICE_SW_RULE_VSI_LIST_SIZE(num_vsi);
s_rule = devm_kzalloc(ice_hw_to_dev(hw), s_rule_size, GFP_KERNEL);
if (!s_rule)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
for (i = 0; i < num_vsi; i++) {
if (!ice_is_vsi_valid(hw, vsi_handle_arr[i])) {
- status = ICE_ERR_PARAM;
+ status = -EINVAL;
goto exit;
}
/* AQ call requires hw_vsi_id(s) */
@@ -1869,11 +1869,11 @@ ice_update_vsi_list_rule(struct ice_hw *hw, u16 *vsi_handle_arr, u16 num_vsi,
* @vsi_list_id: stores the ID of the VSI list to be created
* @lkup_type: switch rule filter's lookup type
*/
-static enum ice_status
+static int
ice_create_vsi_list_rule(struct ice_hw *hw, u16 *vsi_handle_arr, u16 num_vsi,
u16 *vsi_list_id, enum ice_sw_lkup_type lkup_type)
{
- enum ice_status status;
+ int status;
status = ice_aq_alloc_free_vsi_list(hw, vsi_list_id, lkup_type,
ice_aqc_opc_alloc_res);
@@ -1895,7 +1895,7 @@ ice_create_vsi_list_rule(struct ice_hw *hw, u16 *vsi_handle_arr, u16 num_vsi,
* to the corresponding filter management list to track this switch rule
* and VSI mapping
*/
-static enum ice_status
+static int
ice_create_pkt_fwd_rule(struct ice_hw *hw,
struct ice_fltr_list_entry *f_entry)
{
@@ -1903,16 +1903,16 @@ ice_create_pkt_fwd_rule(struct ice_hw *hw,
struct ice_aqc_sw_rules_elem *s_rule;
enum ice_sw_lkup_type l_type;
struct ice_sw_recipe *recp;
- enum ice_status status;
+ int status;
s_rule = devm_kzalloc(ice_hw_to_dev(hw),
ICE_SW_RULE_RX_TX_ETH_HDR_SIZE, GFP_KERNEL);
if (!s_rule)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
fm_entry = devm_kzalloc(ice_hw_to_dev(hw), sizeof(*fm_entry),
GFP_KERNEL);
if (!fm_entry) {
- status = ICE_ERR_NO_MEMORY;
+ status = -ENOMEM;
goto ice_create_pkt_fwd_rule_exit;
}
@@ -1959,16 +1959,16 @@ ice_create_pkt_fwd_rule(struct ice_hw *hw,
* Call AQ command to update a previously created switch rule with a
* VSI list ID
*/
-static enum ice_status
+static int
ice_update_pkt_fwd_rule(struct ice_hw *hw, struct ice_fltr_info *f_info)
{
struct ice_aqc_sw_rules_elem *s_rule;
- enum ice_status status;
+ int status;
s_rule = devm_kzalloc(ice_hw_to_dev(hw),
ICE_SW_RULE_RX_TX_ETH_HDR_SIZE, GFP_KERNEL);
if (!s_rule)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
ice_fill_sw_rule(hw, f_info, s_rule, ice_aqc_opc_update_sw_rules);
@@ -1988,13 +1988,13 @@ ice_update_pkt_fwd_rule(struct ice_hw *hw, struct ice_fltr_info *f_info)
*
* Updates unicast switch filter rules based on VEB/VEPA mode
*/
-enum ice_status ice_update_sw_rule_bridge_mode(struct ice_hw *hw)
+int ice_update_sw_rule_bridge_mode(struct ice_hw *hw)
{
struct ice_switch_info *sw = hw->switch_info;
struct ice_fltr_mgmt_list_entry *fm_entry;
- enum ice_status status = 0;
struct list_head *rule_head;
struct mutex *rule_lock; /* Lock to protect filter rule list */
+ int status = 0;
rule_lock = &sw->recp_list[ICE_SW_LKUP_MAC].filt_rule_lock;
rule_head = &sw->recp_list[ICE_SW_LKUP_MAC].filt_rules;
@@ -2044,24 +2044,24 @@ enum ice_status ice_update_sw_rule_bridge_mode(struct ice_hw *hw)
* Add the new VSI to the previously created VSI list set
* using the update switch rule command
*/
-static enum ice_status
+static int
ice_add_update_vsi_list(struct ice_hw *hw,
struct ice_fltr_mgmt_list_entry *m_entry,
struct ice_fltr_info *cur_fltr,
struct ice_fltr_info *new_fltr)
{
- enum ice_status status = 0;
u16 vsi_list_id = 0;
+ int status = 0;
if ((cur_fltr->fltr_act == ICE_FWD_TO_Q ||
cur_fltr->fltr_act == ICE_FWD_TO_QGRP))
- return ICE_ERR_NOT_IMPL;
+ return -EOPNOTSUPP;
if ((new_fltr->fltr_act == ICE_FWD_TO_Q ||
new_fltr->fltr_act == ICE_FWD_TO_QGRP) &&
(cur_fltr->fltr_act == ICE_FWD_TO_VSI ||
cur_fltr->fltr_act == ICE_FWD_TO_VSI_LIST))
- return ICE_ERR_NOT_IMPL;
+ return -EOPNOTSUPP;
if (m_entry->vsi_count < 2 && !m_entry->vsi_list_info) {
/* Only one entry existed in the mapping and it was not already
@@ -2073,7 +2073,7 @@ ice_add_update_vsi_list(struct ice_hw *hw,
/* A rule already exists with the new VSI being added */
if (cur_fltr->fwd_id.hw_vsi_id == new_fltr->fwd_id.hw_vsi_id)
- return ICE_ERR_ALREADY_EXISTS;
+ return -EEXIST;
vsi_handle_arr[0] = cur_fltr->vsi_handle;
vsi_handle_arr[1] = new_fltr->vsi_handle;
@@ -2101,7 +2101,7 @@ ice_add_update_vsi_list(struct ice_hw *hw,
vsi_list_id);
if (!m_entry->vsi_list_info)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
/* If this entry was large action then the large action needs
* to be updated to point to FWD to VSI list
@@ -2116,7 +2116,7 @@ ice_add_update_vsi_list(struct ice_hw *hw,
enum ice_adminq_opc opcode;
if (!m_entry->vsi_list_info)
- return ICE_ERR_CFG;
+ return -EIO;
/* A rule already exists with the new VSI being added */
if (test_bit(vsi_handle, m_entry->vsi_list_info->vsi_map))
@@ -2209,7 +2209,7 @@ ice_find_vsi_list_entry(struct ice_hw *hw, u8 recp_id, u16 vsi_handle,
*
* Adds or updates the rule lists for a given recipe
*/
-static enum ice_status
+static int
ice_add_rule_internal(struct ice_hw *hw, u8 recp_id,
struct ice_fltr_list_entry *f_entry)
{
@@ -2217,10 +2217,10 @@ ice_add_rule_internal(struct ice_hw *hw, u8 recp_id,
struct ice_fltr_info *new_fltr, *cur_fltr;
struct ice_fltr_mgmt_list_entry *m_entry;
struct mutex *rule_lock; /* Lock to protect filter rule list */
- enum ice_status status = 0;
+ int status = 0;
if (!ice_is_vsi_valid(hw, f_entry->fltr_info.vsi_handle))
- return ICE_ERR_PARAM;
+ return -EINVAL;
f_entry->fltr_info.fwd_id.hw_vsi_id =
ice_get_hw_vsi_num(hw, f_entry->fltr_info.vsi_handle);
@@ -2255,18 +2255,18 @@ ice_add_rule_internal(struct ice_hw *hw, u8 recp_id,
* The VSI list should be emptied before this function is called to remove the
* VSI list.
*/
-static enum ice_status
+static int
ice_remove_vsi_list_rule(struct ice_hw *hw, u16 vsi_list_id,
enum ice_sw_lkup_type lkup_type)
{
struct ice_aqc_sw_rules_elem *s_rule;
- enum ice_status status;
u16 s_rule_size;
+ int status;
s_rule_size = (u16)ICE_SW_RULE_VSI_LIST_SIZE(0);
s_rule = devm_kzalloc(ice_hw_to_dev(hw), s_rule_size, GFP_KERNEL);
if (!s_rule)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
s_rule->type = cpu_to_le16(ICE_AQC_SW_RULES_T_VSI_LIST_CLEAR);
s_rule->pdata.vsi_list.index = cpu_to_le16(vsi_list_id);
@@ -2288,21 +2288,21 @@ ice_remove_vsi_list_rule(struct ice_hw *hw, u16 vsi_list_id,
* @fm_list: filter management entry for which the VSI list management needs to
* be done
*/
-static enum ice_status
+static int
ice_rem_update_vsi_list(struct ice_hw *hw, u16 vsi_handle,
struct ice_fltr_mgmt_list_entry *fm_list)
{
enum ice_sw_lkup_type lkup_type;
- enum ice_status status = 0;
u16 vsi_list_id;
+ int status = 0;
if (fm_list->fltr_info.fltr_act != ICE_FWD_TO_VSI_LIST ||
fm_list->vsi_count == 0)
- return ICE_ERR_PARAM;
+ return -EINVAL;
/* A rule with the VSI being removed does not exist */
if (!test_bit(vsi_handle, fm_list->vsi_list_info->vsi_map))
- return ICE_ERR_DOES_NOT_EXIST;
+ return -ENOENT;
lkup_type = fm_list->fltr_info.lkup_type;
vsi_list_id = fm_list->fltr_info.fwd_id.vsi_list_id;
@@ -2324,7 +2324,7 @@ ice_rem_update_vsi_list(struct ice_hw *hw, u16 vsi_handle,
rem_vsi_handle = find_first_bit(vsi_list_info->vsi_map,
ICE_MAX_VSI);
if (!ice_is_vsi_valid(hw, rem_vsi_handle))
- return ICE_ERR_OUT_OF_RANGE;
+ return -EIO;
/* Make sure VSI list is empty before removing it below */
status = ice_update_vsi_list_rule(hw, &rem_vsi_handle, 1,
@@ -2375,19 +2375,19 @@ ice_rem_update_vsi_list(struct ice_hw *hw, u16 vsi_handle,
* @recp_id: recipe ID for which the rule needs to removed
* @f_entry: rule entry containing filter information
*/
-static enum ice_status
+static int
ice_remove_rule_internal(struct ice_hw *hw, u8 recp_id,
struct ice_fltr_list_entry *f_entry)
{
struct ice_switch_info *sw = hw->switch_info;
struct ice_fltr_mgmt_list_entry *list_elem;
struct mutex *rule_lock; /* Lock to protect filter rule list */
- enum ice_status status = 0;
bool remove_rule = false;
u16 vsi_handle;
+ int status = 0;
if (!ice_is_vsi_valid(hw, f_entry->fltr_info.vsi_handle))
- return ICE_ERR_PARAM;
+ return -EINVAL;
f_entry->fltr_info.fwd_id.hw_vsi_id =
ice_get_hw_vsi_num(hw, f_entry->fltr_info.vsi_handle);
@@ -2395,14 +2395,14 @@ ice_remove_rule_internal(struct ice_hw *hw, u8 recp_id,
mutex_lock(rule_lock);
list_elem = ice_find_rule_entry(hw, recp_id, &f_entry->fltr_info);
if (!list_elem) {
- status = ICE_ERR_DOES_NOT_EXIST;
+ status = -ENOENT;
goto exit;
}
if (list_elem->fltr_info.fltr_act != ICE_FWD_TO_VSI_LIST) {
remove_rule = true;
} else if (!list_elem->vsi_list_info) {
- status = ICE_ERR_DOES_NOT_EXIST;
+ status = -ENOENT;
goto exit;
} else if (list_elem->vsi_list_info->ref_cnt > 1) {
/* a ref_cnt > 1 indicates that the vsi_list is being
@@ -2435,7 +2435,7 @@ ice_remove_rule_internal(struct ice_hw *hw, u8 recp_id,
ICE_SW_RULE_RX_TX_NO_HDR_SIZE,
GFP_KERNEL);
if (!s_rule) {
- status = ICE_ERR_NO_MEMORY;
+ status = -ENOMEM;
goto exit;
}
@@ -2590,7 +2590,7 @@ bool ice_vlan_fltr_exist(struct ice_hw *hw, u16 vlan_id, u16 vsi_handle)
* check for duplicates in this case, removing duplicates from a given
* list should be taken care of in the caller of this function.
*/
-enum ice_status ice_add_mac(struct ice_hw *hw, struct list_head *m_list)
+int ice_add_mac(struct ice_hw *hw, struct list_head *m_list)
{
struct ice_aqc_sw_rules_elem *s_rule, *r_iter;
struct ice_fltr_list_entry *m_list_itr;
@@ -2598,12 +2598,12 @@ enum ice_status ice_add_mac(struct ice_hw *hw, struct list_head *m_list)
u16 total_elem_left, s_rule_size;
struct ice_switch_info *sw;
struct mutex *rule_lock; /* Lock to protect filter rule list */
- enum ice_status status = 0;
u16 num_unicast = 0;
+ int status = 0;
u8 elem_sent;
if (!m_list || !hw)
- return ICE_ERR_PARAM;
+ return -EINVAL;
s_rule = NULL;
sw = hw->switch_info;
@@ -2616,23 +2616,23 @@ enum ice_status ice_add_mac(struct ice_hw *hw, struct list_head *m_list)
m_list_itr->fltr_info.flag = ICE_FLTR_TX;
vsi_handle = m_list_itr->fltr_info.vsi_handle;
if (!ice_is_vsi_valid(hw, vsi_handle))
- return ICE_ERR_PARAM;
+ return -EINVAL;
hw_vsi_id = ice_get_hw_vsi_num(hw, vsi_handle);
m_list_itr->fltr_info.fwd_id.hw_vsi_id = hw_vsi_id;
/* update the src in case it is VSI num */
if (m_list_itr->fltr_info.src_id != ICE_SRC_ID_VSI)
- return ICE_ERR_PARAM;
+ return -EINVAL;
m_list_itr->fltr_info.src = hw_vsi_id;
if (m_list_itr->fltr_info.lkup_type != ICE_SW_LKUP_MAC ||
is_zero_ether_addr(add))
- return ICE_ERR_PARAM;
+ return -EINVAL;
if (is_unicast_ether_addr(add) && !hw->ucast_shared) {
/* Don't overwrite the unicast address */
mutex_lock(rule_lock);
if (ice_find_rule_entry(hw, ICE_SW_LKUP_MAC,
&m_list_itr->fltr_info)) {
mutex_unlock(rule_lock);
- return ICE_ERR_ALREADY_EXISTS;
+ return -EEXIST;
}
mutex_unlock(rule_lock);
num_unicast++;
@@ -2660,7 +2660,7 @@ enum ice_status ice_add_mac(struct ice_hw *hw, struct list_head *m_list)
s_rule = devm_kcalloc(ice_hw_to_dev(hw), num_unicast, s_rule_size,
GFP_KERNEL);
if (!s_rule) {
- status = ICE_ERR_NO_MEMORY;
+ status = -ENOMEM;
goto ice_add_mac_exit;
}
@@ -2710,7 +2710,7 @@ enum ice_status ice_add_mac(struct ice_hw *hw, struct list_head *m_list)
fm_entry = devm_kzalloc(ice_hw_to_dev(hw),
sizeof(*fm_entry), GFP_KERNEL);
if (!fm_entry) {
- status = ICE_ERR_NO_MEMORY;
+ status = -ENOMEM;
goto ice_add_mac_exit;
}
fm_entry->fltr_info = *f_info;
@@ -2737,7 +2737,7 @@ enum ice_status ice_add_mac(struct ice_hw *hw, struct list_head *m_list)
* @hw: pointer to the hardware structure
* @f_entry: filter entry containing one VLAN information
*/
-static enum ice_status
+static int
ice_add_vlan_internal(struct ice_hw *hw, struct ice_fltr_list_entry *f_entry)
{
struct ice_switch_info *sw = hw->switch_info;
@@ -2746,10 +2746,10 @@ ice_add_vlan_internal(struct ice_hw *hw, struct ice_fltr_list_entry *f_entry)
enum ice_sw_lkup_type lkup_type;
u16 vsi_list_id = 0, vsi_handle;
struct mutex *rule_lock; /* Lock to protect filter rule list */
- enum ice_status status = 0;
+ int status = 0;
if (!ice_is_vsi_valid(hw, f_entry->fltr_info.vsi_handle))
- return ICE_ERR_PARAM;
+ return -EINVAL;
f_entry->fltr_info.fwd_id.hw_vsi_id =
ice_get_hw_vsi_num(hw, f_entry->fltr_info.vsi_handle);
@@ -2757,10 +2757,10 @@ ice_add_vlan_internal(struct ice_hw *hw, struct ice_fltr_list_entry *f_entry)
/* VLAN ID should only be 12 bits */
if (new_fltr->l_data.vlan.vlan_id > ICE_MAX_VLAN_ID)
- return ICE_ERR_PARAM;
+ return -EINVAL;
if (new_fltr->src_id != ICE_SRC_ID_VSI)
- return ICE_ERR_PARAM;
+ return -EINVAL;
new_fltr->src = new_fltr->fwd_id.hw_vsi_id;
lkup_type = new_fltr->lkup_type;
@@ -2799,7 +2799,7 @@ ice_add_vlan_internal(struct ice_hw *hw, struct ice_fltr_list_entry *f_entry)
v_list_itr = ice_find_rule_entry(hw, ICE_SW_LKUP_VLAN,
new_fltr);
if (!v_list_itr) {
- status = ICE_ERR_DOES_NOT_EXIST;
+ status = -ENOENT;
goto exit;
}
/* reuse VSI list for new rule and increment ref_cnt */
@@ -2835,7 +2835,7 @@ ice_add_vlan_internal(struct ice_hw *hw, struct ice_fltr_list_entry *f_entry)
if (v_list_itr->vsi_count > 1 &&
v_list_itr->vsi_list_info->ref_cnt > 1) {
ice_debug(hw, ICE_DBG_SW, "Invalid configuration: Optimization to reuse VSI list with more than one VSI is not being done yet\n");
- status = ICE_ERR_CFG;
+ status = -EIO;
goto exit;
}
@@ -2845,7 +2845,7 @@ ice_add_vlan_internal(struct ice_hw *hw, struct ice_fltr_list_entry *f_entry)
/* A rule already exists with the new VSI being added */
if (cur_handle == vsi_handle) {
- status = ICE_ERR_ALREADY_EXISTS;
+ status = -EEXIST;
goto exit;
}
@@ -2890,16 +2890,16 @@ ice_add_vlan_internal(struct ice_hw *hw, struct ice_fltr_list_entry *f_entry)
* @hw: pointer to the hardware structure
* @v_list: list of VLAN entries and forwarding information
*/
-enum ice_status ice_add_vlan(struct ice_hw *hw, struct list_head *v_list)
+int ice_add_vlan(struct ice_hw *hw, struct list_head *v_list)
{
struct ice_fltr_list_entry *v_list_itr;
if (!v_list || !hw)
- return ICE_ERR_PARAM;
+ return -EINVAL;
list_for_each_entry(v_list_itr, v_list, list_entry) {
if (v_list_itr->fltr_info.lkup_type != ICE_SW_LKUP_VLAN)
- return ICE_ERR_PARAM;
+ return -EINVAL;
v_list_itr->fltr_info.flag = ICE_FLTR_TX;
v_list_itr->status = ice_add_vlan_internal(hw, v_list_itr);
if (v_list_itr->status)
@@ -2917,13 +2917,12 @@ enum ice_status ice_add_vlan(struct ice_hw *hw, struct list_head *v_list)
* the filter list with the necessary fields (including flags to
* indicate Tx or Rx rules).
*/
-enum ice_status
-ice_add_eth_mac(struct ice_hw *hw, struct list_head *em_list)
+int ice_add_eth_mac(struct ice_hw *hw, struct list_head *em_list)
{
struct ice_fltr_list_entry *em_list_itr;
if (!em_list || !hw)
- return ICE_ERR_PARAM;
+ return -EINVAL;
list_for_each_entry(em_list_itr, em_list, list_entry) {
enum ice_sw_lkup_type l_type =
@@ -2931,7 +2930,7 @@ ice_add_eth_mac(struct ice_hw *hw, struct list_head *em_list)
if (l_type != ICE_SW_LKUP_ETHERTYPE_MAC &&
l_type != ICE_SW_LKUP_ETHERTYPE)
- return ICE_ERR_PARAM;
+ return -EINVAL;
em_list_itr->status = ice_add_rule_internal(hw, l_type,
em_list_itr);
@@ -2946,13 +2945,12 @@ ice_add_eth_mac(struct ice_hw *hw, struct list_head *em_list)
* @hw: pointer to the hardware structure
* @em_list: list of ethertype or ethertype MAC entries
*/
-enum ice_status
-ice_remove_eth_mac(struct ice_hw *hw, struct list_head *em_list)
+int ice_remove_eth_mac(struct ice_hw *hw, struct list_head *em_list)
{
struct ice_fltr_list_entry *em_list_itr, *tmp;
if (!em_list || !hw)
- return ICE_ERR_PARAM;
+ return -EINVAL;
list_for_each_entry_safe(em_list_itr, tmp, em_list, list_entry) {
enum ice_sw_lkup_type l_type =
@@ -2960,7 +2958,7 @@ ice_remove_eth_mac(struct ice_hw *hw, struct list_head *em_list)
if (l_type != ICE_SW_LKUP_ETHERTYPE_MAC &&
l_type != ICE_SW_LKUP_ETHERTYPE)
- return ICE_ERR_PARAM;
+ return -EINVAL;
em_list_itr->status = ice_remove_rule_internal(hw, l_type,
em_list_itr);
@@ -3020,18 +3018,17 @@ ice_rem_adv_rule_info(struct ice_hw *hw, struct list_head *rule_head)
* add filter rule to set/unset given VSI as default VSI for the switch
* (represented by swid)
*/
-enum ice_status
-ice_cfg_dflt_vsi(struct ice_hw *hw, u16 vsi_handle, bool set, u8 direction)
+int ice_cfg_dflt_vsi(struct ice_hw *hw, u16 vsi_handle, bool set, u8 direction)
{
struct ice_aqc_sw_rules_elem *s_rule;
struct ice_fltr_info f_info;
enum ice_adminq_opc opcode;
- enum ice_status status;
u16 s_rule_size;
u16 hw_vsi_id;
+ int status;
if (!ice_is_vsi_valid(hw, vsi_handle))
- return ICE_ERR_PARAM;
+ return -EINVAL;
hw_vsi_id = ice_get_hw_vsi_num(hw, vsi_handle);
s_rule_size = set ? ICE_SW_RULE_RX_TX_ETH_HDR_SIZE :
@@ -3039,7 +3036,7 @@ ice_cfg_dflt_vsi(struct ice_hw *hw, u16 vsi_handle, bool set, u8 direction)
s_rule = devm_kzalloc(ice_hw_to_dev(hw), s_rule_size, GFP_KERNEL);
if (!s_rule)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
memset(&f_info, 0, sizeof(f_info));
@@ -3137,18 +3134,18 @@ ice_find_ucast_rule_entry(struct ice_hw *hw, u8 recp_id,
* This function removes either a MAC filter rule or a specific VSI from a
* VSI list for a multicast MAC address.
*
- * Returns ICE_ERR_DOES_NOT_EXIST if a given entry was not added by
- * ice_add_mac. Caller should be aware that this call will only work if all
- * the entries passed into m_list were added previously. It will not attempt to
- * do a partial remove of entries that were found.
+ * Returns -ENOENT if a given entry was not added by ice_add_mac. Caller should
+ * be aware that this call will only work if all the entries passed into m_list
+ * were added previously. It will not attempt to do a partial remove of entries
+ * that were found.
*/
-enum ice_status ice_remove_mac(struct ice_hw *hw, struct list_head *m_list)
+int ice_remove_mac(struct ice_hw *hw, struct list_head *m_list)
{
struct ice_fltr_list_entry *list_itr, *tmp;
struct mutex *rule_lock; /* Lock to protect filter rule list */
if (!m_list)
- return ICE_ERR_PARAM;
+ return -EINVAL;
rule_lock = &hw->switch_info->recp_list[ICE_SW_LKUP_MAC].filt_rule_lock;
list_for_each_entry_safe(list_itr, tmp, m_list, list_entry) {
@@ -3157,11 +3154,11 @@ enum ice_status ice_remove_mac(struct ice_hw *hw, struct list_head *m_list)
u16 vsi_handle;
if (l_type != ICE_SW_LKUP_MAC)
- return ICE_ERR_PARAM;
+ return -EINVAL;
vsi_handle = list_itr->fltr_info.vsi_handle;
if (!ice_is_vsi_valid(hw, vsi_handle))
- return ICE_ERR_PARAM;
+ return -EINVAL;
list_itr->fltr_info.fwd_id.hw_vsi_id =
ice_get_hw_vsi_num(hw, vsi_handle);
@@ -3174,7 +3171,7 @@ enum ice_status ice_remove_mac(struct ice_hw *hw, struct list_head *m_list)
if (!ice_find_ucast_rule_entry(hw, ICE_SW_LKUP_MAC,
&list_itr->fltr_info)) {
mutex_unlock(rule_lock);
- return ICE_ERR_DOES_NOT_EXIST;
+ return -ENOENT;
}
mutex_unlock(rule_lock);
}
@@ -3192,19 +3189,18 @@ enum ice_status ice_remove_mac(struct ice_hw *hw, struct list_head *m_list)
* @hw: pointer to the hardware structure
* @v_list: list of VLAN entries and forwarding information
*/
-enum ice_status
-ice_remove_vlan(struct ice_hw *hw, struct list_head *v_list)
+int ice_remove_vlan(struct ice_hw *hw, struct list_head *v_list)
{
struct ice_fltr_list_entry *v_list_itr, *tmp;
if (!v_list || !hw)
- return ICE_ERR_PARAM;
+ return -EINVAL;
list_for_each_entry_safe(v_list_itr, tmp, v_list, list_entry) {
enum ice_sw_lkup_type l_type = v_list_itr->fltr_info.lkup_type;
if (l_type != ICE_SW_LKUP_VLAN)
- return ICE_ERR_PARAM;
+ return -EINVAL;
v_list_itr->status = ice_remove_rule_internal(hw,
ICE_SW_LKUP_VLAN,
v_list_itr);
@@ -3242,7 +3238,7 @@ ice_vsi_uses_fltr(struct ice_fltr_mgmt_list_entry *fm_entry, u16 vsi_handle)
* fltr_info.fwd_id fields. These are set such that later logic can
* extract which VSI to remove the fltr from, and pass on that information.
*/
-static enum ice_status
+static int
ice_add_entry_to_vsi_fltr_list(struct ice_hw *hw, u16 vsi_handle,
struct list_head *vsi_list_head,
struct ice_fltr_info *fi)
@@ -3254,7 +3250,7 @@ ice_add_entry_to_vsi_fltr_list(struct ice_hw *hw, u16 vsi_handle,
*/
tmp = devm_kzalloc(ice_hw_to_dev(hw), sizeof(*tmp), GFP_KERNEL);
if (!tmp)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
tmp->fltr_info = *fi;
@@ -3285,17 +3281,17 @@ ice_add_entry_to_vsi_fltr_list(struct ice_hw *hw, u16 vsi_handle,
* Note that this means all entries in vsi_list_head must be explicitly
* deallocated by the caller when done with list.
*/
-static enum ice_status
+static int
ice_add_to_vsi_fltr_list(struct ice_hw *hw, u16 vsi_handle,
struct list_head *lkup_list_head,
struct list_head *vsi_list_head)
{
struct ice_fltr_mgmt_list_entry *fm_entry;
- enum ice_status status = 0;
+ int status = 0;
/* check to make sure VSI ID is valid and within boundary */
if (!ice_is_vsi_valid(hw, vsi_handle))
- return ICE_ERR_PARAM;
+ return -EINVAL;
list_for_each_entry(fm_entry, lkup_list_head, list_entry) {
if (!ice_vsi_uses_fltr(fm_entry, vsi_handle))
@@ -3349,9 +3345,8 @@ static u8 ice_determine_promisc_mask(struct ice_fltr_info *fi)
* @recp_id: recipe ID for which the rule needs to removed
* @v_list: list of promisc entries
*/
-static enum ice_status
-ice_remove_promisc(struct ice_hw *hw, u8 recp_id,
- struct list_head *v_list)
+static int
+ice_remove_promisc(struct ice_hw *hw, u8 recp_id, struct list_head *v_list)
{
struct ice_fltr_list_entry *v_list_itr, *tmp;
@@ -3371,7 +3366,7 @@ ice_remove_promisc(struct ice_hw *hw, u8 recp_id,
* @promisc_mask: mask of promiscuous config bits to clear
* @vid: VLAN ID to clear VLAN promiscuous
*/
-enum ice_status
+int
ice_clear_vsi_promisc(struct ice_hw *hw, u16 vsi_handle, u8 promisc_mask,
u16 vid)
{
@@ -3381,11 +3376,11 @@ ice_clear_vsi_promisc(struct ice_hw *hw, u16 vsi_handle, u8 promisc_mask,
struct ice_fltr_mgmt_list_entry *itr;
struct list_head *rule_head;
struct mutex *rule_lock; /* Lock to protect filter rule list */
- enum ice_status status = 0;
+ int status = 0;
u8 recipe_id;
if (!ice_is_vsi_valid(hw, vsi_handle))
- return ICE_ERR_PARAM;
+ return -EINVAL;
if (promisc_mask & (ICE_PROMISC_VLAN_RX | ICE_PROMISC_VLAN_TX))
recipe_id = ICE_SW_LKUP_PROMISC_VLAN;
@@ -3444,20 +3439,20 @@ ice_clear_vsi_promisc(struct ice_hw *hw, u16 vsi_handle, u8 promisc_mask,
* @promisc_mask: mask of promiscuous config bits
* @vid: VLAN ID to set VLAN promiscuous
*/
-enum ice_status
+int
ice_set_vsi_promisc(struct ice_hw *hw, u16 vsi_handle, u8 promisc_mask, u16 vid)
{
enum { UCAST_FLTR = 1, MCAST_FLTR, BCAST_FLTR };
struct ice_fltr_list_entry f_list_entry;
struct ice_fltr_info new_fltr;
- enum ice_status status = 0;
bool is_tx_fltr;
+ int status = 0;
u16 hw_vsi_id;
int pkt_type;
u8 recipe_id;
if (!ice_is_vsi_valid(hw, vsi_handle))
- return ICE_ERR_PARAM;
+ return -EINVAL;
hw_vsi_id = ice_get_hw_vsi_num(hw, vsi_handle);
memset(&new_fltr, 0, sizeof(new_fltr));
@@ -3558,7 +3553,7 @@ ice_set_vsi_promisc(struct ice_hw *hw, u16 vsi_handle, u8 promisc_mask, u16 vid)
*
* Configure VSI with all associated VLANs to given promiscuous mode(s)
*/
-enum ice_status
+int
ice_set_vlan_vsi_promisc(struct ice_hw *hw, u16 vsi_handle, u8 promisc_mask,
bool rm_vlan_promisc)
{
@@ -3567,8 +3562,8 @@ ice_set_vlan_vsi_promisc(struct ice_hw *hw, u16 vsi_handle, u8 promisc_mask,
struct list_head vsi_list_head;
struct list_head *vlan_head;
struct mutex *vlan_lock; /* Lock to protect filter rule list */
- enum ice_status status;
u16 vlan_id;
+ int status;
INIT_LIST_HEAD(&vsi_list_head);
vlan_lock = &sw->recp_list[ICE_SW_LKUP_VLAN].filt_rule_lock;
@@ -3616,7 +3611,7 @@ ice_remove_vsi_lkup_fltr(struct ice_hw *hw, u16 vsi_handle,
struct list_head *rule_head;
struct ice_fltr_list_entry *tmp;
struct mutex *rule_lock; /* Lock to protect filter rule list */
- enum ice_status status;
+ int status;
INIT_LIST_HEAD(&remove_list_head);
rule_lock = &sw->recp_list[lkup].filt_rule_lock;
@@ -3681,19 +3676,19 @@ void ice_remove_vsi_fltr(struct ice_hw *hw, u16 vsi_handle)
* @num_items: number of entries requested for FD resource type
* @counter_id: counter index returned by AQ call
*/
-enum ice_status
+int
ice_alloc_res_cntr(struct ice_hw *hw, u8 type, u8 alloc_shared, u16 num_items,
u16 *counter_id)
{
struct ice_aqc_alloc_free_res_elem *buf;
- enum ice_status status;
u16 buf_len;
+ int status;
/* Allocate resource */
buf_len = struct_size(buf, elem, 1);
buf = kzalloc(buf_len, GFP_KERNEL);
if (!buf)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
buf->num_elems = cpu_to_le16(num_items);
buf->res_type = cpu_to_le16(((type << ICE_AQC_RES_TYPE_S) &
@@ -3719,19 +3714,19 @@ ice_alloc_res_cntr(struct ice_hw *hw, u8 type, u8 alloc_shared, u16 num_items,
* @num_items: number of entries to be freed for FD resource type
* @counter_id: counter ID resource which needs to be freed
*/
-enum ice_status
+int
ice_free_res_cntr(struct ice_hw *hw, u8 type, u8 alloc_shared, u16 num_items,
u16 counter_id)
{
struct ice_aqc_alloc_free_res_elem *buf;
- enum ice_status status;
u16 buf_len;
+ int status;
/* Free resource */
buf_len = struct_size(buf, elem, 1);
buf = kzalloc(buf_len, GFP_KERNEL);
if (!buf)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
buf->num_elems = cpu_to_le16(num_items);
buf->res_type = cpu_to_le16(((type << ICE_AQC_RES_TYPE_S) &
@@ -3941,7 +3936,7 @@ ice_fill_valid_words(struct ice_adv_lkup_elem *rule,
* and start grouping them in 4-word groups. Each group makes up one
* recipe.
*/
-static enum ice_status
+static int
ice_create_first_fit_recp_def(struct ice_hw *hw,
struct ice_prot_lkup_ext *lkup_exts,
struct list_head *rg_list,
@@ -3965,7 +3960,7 @@ ice_create_first_fit_recp_def(struct ice_hw *hw,
sizeof(*entry),
GFP_KERNEL);
if (!entry)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
list_add(&entry->l_entry, rg_list);
grp = &entry->r_group;
(*recp_cnt)++;
@@ -3991,7 +3986,7 @@ ice_create_first_fit_recp_def(struct ice_hw *hw,
* Helper function to fill in the field vector indices for protocol-offset
* pairs. These indexes are then ultimately programmed into a recipe.
*/
-static enum ice_status
+static int
ice_fill_fv_word_index(struct ice_hw *hw, struct list_head *fv_list,
struct list_head *rg_list)
{
@@ -4033,7 +4028,7 @@ ice_fill_fv_word_index(struct ice_hw *hw, struct list_head *fv_list,
* invalid pair
*/
if (!found)
- return ICE_ERR_PARAM;
+ return -EINVAL;
}
}
@@ -4075,10 +4070,8 @@ ice_find_free_recp_res_idx(struct ice_hw *hw, const unsigned long *profiles,
DECLARE_BITMAP(used_idx, ICE_MAX_FV_WORDS);
u16 bit;
- bitmap_zero(possible_idx, ICE_MAX_FV_WORDS);
bitmap_zero(recipes, ICE_MAX_NUM_RECIPES);
bitmap_zero(used_idx, ICE_MAX_FV_WORDS);
- bitmap_zero(free_idx, ICE_MAX_FV_WORDS);
bitmap_set(possible_idx, 0, ICE_MAX_FV_WORDS);
@@ -4115,7 +4108,7 @@ ice_find_free_recp_res_idx(struct ice_hw *hw, const unsigned long *profiles,
* @rm: recipe management list entry
* @profiles: bitmap of profiles that will be associated.
*/
-static enum ice_status
+static int
ice_add_sw_recipe(struct ice_hw *hw, struct ice_sw_recipe *rm,
unsigned long *profiles)
{
@@ -4123,11 +4116,11 @@ ice_add_sw_recipe(struct ice_hw *hw, struct ice_sw_recipe *rm,
struct ice_aqc_recipe_data_elem *tmp;
struct ice_aqc_recipe_data_elem *buf;
struct ice_recp_grp_entry *entry;
- enum ice_status status;
u16 free_res_idx;
u16 recipe_count;
u8 chain_idx;
u8 recps = 0;
+ int status;
/* When more than one recipe are required, another recipe is needed to
* chain them together. Matching a tunnel metadata ID takes up one of
@@ -4143,22 +4136,22 @@ ice_add_sw_recipe(struct ice_hw *hw, struct ice_sw_recipe *rm,
if (rm->n_grp_count > 1) {
if (rm->n_grp_count > free_res_idx)
- return ICE_ERR_MAX_LIMIT;
+ return -ENOSPC;
rm->n_grp_count++;
}
if (rm->n_grp_count > ICE_MAX_CHAIN_RECIPE)
- return ICE_ERR_MAX_LIMIT;
+ return -ENOSPC;
tmp = kcalloc(ICE_MAX_NUM_RECIPES, sizeof(*tmp), GFP_KERNEL);
if (!tmp)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
buf = devm_kcalloc(ice_hw_to_dev(hw), rm->n_grp_count, sizeof(*buf),
GFP_KERNEL);
if (!buf) {
- status = ICE_ERR_NO_MEMORY;
+ status = -ENOMEM;
goto err_mem;
}
@@ -4218,7 +4211,7 @@ ice_add_sw_recipe(struct ice_hw *hw, struct ice_sw_recipe *rm,
*/
if (chain_idx >= ICE_MAX_FV_WORDS) {
ice_debug(hw, ICE_DBG_SW, "No chain index available\n");
- status = ICE_ERR_MAX_LIMIT;
+ status = -ENOSPC;
goto err_unroll;
}
@@ -4249,7 +4242,7 @@ ice_add_sw_recipe(struct ice_hw *hw, struct ice_sw_recipe *rm,
memcpy(buf[0].recipe_bitmap, rm->r_bitmap,
sizeof(buf[0].recipe_bitmap));
} else {
- status = ICE_ERR_BAD_PTR;
+ status = -EINVAL;
goto err_unroll;
}
/* Applicable only for ROOT_RECIPE, set the fwd_priority for
@@ -4285,7 +4278,7 @@ ice_add_sw_recipe(struct ice_hw *hw, struct ice_sw_recipe *rm,
sizeof(*last_chain_entry),
GFP_KERNEL);
if (!last_chain_entry) {
- status = ICE_ERR_NO_MEMORY;
+ status = -ENOMEM;
goto err_unroll;
}
last_chain_entry->rid = rid;
@@ -4320,7 +4313,7 @@ ice_add_sw_recipe(struct ice_hw *hw, struct ice_sw_recipe *rm,
memcpy(buf[recps].recipe_bitmap, rm->r_bitmap,
sizeof(buf[recps].recipe_bitmap));
} else {
- status = ICE_ERR_BAD_PTR;
+ status = -EINVAL;
goto err_unroll;
}
buf[recps].content.act_ctrl_fwd_priority = rm->priority;
@@ -4354,7 +4347,7 @@ ice_add_sw_recipe(struct ice_hw *hw, struct ice_sw_recipe *rm,
}
if (!idx_found) {
- status = ICE_ERR_OUT_OF_RANGE;
+ status = -EIO;
goto err_unroll;
}
@@ -4407,12 +4400,12 @@ ice_add_sw_recipe(struct ice_hw *hw, struct ice_sw_recipe *rm,
* @rm: recipe management list entry
* @lkup_exts: lookup elements
*/
-static enum ice_status
+static int
ice_create_recipe_group(struct ice_hw *hw, struct ice_sw_recipe *rm,
struct ice_prot_lkup_ext *lkup_exts)
{
- enum ice_status status;
u8 recp_count = 0;
+ int status;
rm->n_grp_count = 0;
@@ -4442,21 +4435,21 @@ ice_create_recipe_group(struct ice_hw *hw, struct ice_sw_recipe *rm,
* @bm: bitmap of field vectors to consider
* @fv_list: pointer to a list that holds the returned field vectors
*/
-static enum ice_status
+static int
ice_get_fv(struct ice_hw *hw, struct ice_adv_lkup_elem *lkups, u16 lkups_cnt,
unsigned long *bm, struct list_head *fv_list)
{
- enum ice_status status;
u8 *prot_ids;
+ int status;
u16 i;
prot_ids = kcalloc(lkups_cnt, sizeof(*prot_ids), GFP_KERNEL);
if (!prot_ids)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
for (i = 0; i < lkups_cnt; i++)
if (!ice_prot_type_to_id(lkups[i].type, &prot_ids[i])) {
- status = ICE_ERR_CFG;
+ status = -EIO;
goto free_mem;
}
@@ -4493,7 +4486,7 @@ static bool ice_tun_type_match_word(enum ice_sw_tunnel_type tun_type, u16 *mask)
* @rinfo: other information regarding the rule e.g. priority and action info
* @lkup_exts: lookup word structure
*/
-static enum ice_status
+static int
ice_add_special_words(struct ice_adv_rule_info *rinfo,
struct ice_prot_lkup_ext *lkup_exts)
{
@@ -4510,7 +4503,7 @@ ice_add_special_words(struct ice_adv_rule_info *rinfo,
lkup_exts->fv_words[word].off = ICE_TUN_FLAG_MDID_OFF;
lkup_exts->field_mask[word] = mask;
} else {
- return ICE_ERR_MAX_LIMIT;
+ return -ENOSPC;
}
}
@@ -4561,7 +4554,7 @@ ice_get_compat_fv_bitmap(struct ice_hw *hw, struct ice_adv_rule_info *rinfo,
* @rinfo: other information regarding the rule e.g. priority and action info
* @rid: return the recipe ID of the recipe created
*/
-static enum ice_status
+static int
ice_add_adv_recipe(struct ice_hw *hw, struct ice_adv_lkup_elem *lkups,
u16 lkups_cnt, struct ice_adv_rule_info *rinfo, u16 *rid)
{
@@ -4572,16 +4565,16 @@ ice_add_adv_recipe(struct ice_hw *hw, struct ice_adv_lkup_elem *lkups,
struct ice_sw_fv_list_entry *fvit;
struct ice_recp_grp_entry *r_tmp;
struct ice_sw_fv_list_entry *tmp;
- enum ice_status status = 0;
struct ice_sw_recipe *rm;
+ int status = 0;
u8 i;
if (!lkups_cnt)
- return ICE_ERR_PARAM;
+ return -EINVAL;
lkup_exts = kzalloc(sizeof(*lkup_exts), GFP_KERNEL);
if (!lkup_exts)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
/* Determine the number of words to be matched and if it exceeds a
* recipe's restrictions
@@ -4590,20 +4583,20 @@ ice_add_adv_recipe(struct ice_hw *hw, struct ice_adv_lkup_elem *lkups,
u16 count;
if (lkups[i].type >= ICE_PROTOCOL_LAST) {
- status = ICE_ERR_CFG;
+ status = -EIO;
goto err_free_lkup_exts;
}
count = ice_fill_valid_words(&lkups[i], lkup_exts);
if (!count) {
- status = ICE_ERR_CFG;
+ status = -EIO;
goto err_free_lkup_exts;
}
}
rm = kzalloc(sizeof(*rm), GFP_KERNEL);
if (!rm) {
- status = ICE_ERR_NO_MEMORY;
+ status = -ENOMEM;
goto err_free_lkup_exts;
}
@@ -4849,7 +4842,7 @@ ice_find_dummy_packet(struct ice_adv_lkup_elem *lkups, u16 lkups_cnt,
* @pkt_len: packet length of dummy packet
* @offsets: offset info for the dummy packet
*/
-static enum ice_status
+static int
ice_fill_adv_dummy_packet(struct ice_adv_lkup_elem *lkups, u16 lkups_cnt,
struct ice_aqc_sw_rules_elem *s_rule,
const u8 *dummy_pkt, u16 pkt_len,
@@ -4883,7 +4876,7 @@ ice_fill_adv_dummy_packet(struct ice_adv_lkup_elem *lkups, u16 lkups_cnt,
}
/* this should never happen in a correct calling sequence */
if (!found)
- return ICE_ERR_PARAM;
+ return -EINVAL;
switch (lkups[i].type) {
case ICE_MAC_OFOS:
@@ -4920,12 +4913,12 @@ ice_fill_adv_dummy_packet(struct ice_adv_lkup_elem *lkups, u16 lkups_cnt,
len = sizeof(struct ice_udp_tnl_hdr);
break;
default:
- return ICE_ERR_PARAM;
+ return -EINVAL;
}
/* the length should be a word multiple */
if (len % ICE_BYTES_PER_WORD)
- return ICE_ERR_CFG;
+ return -EIO;
/* We have the offset to the header start, the length, the
* caller's header values and mask. Use this information to
@@ -4955,7 +4948,7 @@ ice_fill_adv_dummy_packet(struct ice_adv_lkup_elem *lkups, u16 lkups_cnt,
* @pkt: dummy packet to fill in
* @offsets: offset info for the dummy packet
*/
-static enum ice_status
+static int
ice_fill_adv_packet_tun(struct ice_hw *hw, enum ice_sw_tunnel_type tun_type,
u8 *pkt, const struct ice_dummy_pkt_offsets *offsets)
{
@@ -4964,11 +4957,11 @@ ice_fill_adv_packet_tun(struct ice_hw *hw, enum ice_sw_tunnel_type tun_type,
switch (tun_type) {
case ICE_SW_TUN_VXLAN:
if (!ice_get_open_tunnel_port(hw, &open_port, TNL_VXLAN))
- return ICE_ERR_CFG;
+ return -EIO;
break;
case ICE_SW_TUN_GENEVE:
if (!ice_get_open_tunnel_port(hw, &open_port, TNL_GENEVE))
- return ICE_ERR_CFG;
+ return -EIO;
break;
default:
/* Nothing needs to be done for this tunnel type */
@@ -4989,7 +4982,7 @@ ice_fill_adv_packet_tun(struct ice_hw *hw, enum ice_sw_tunnel_type tun_type,
}
}
- return ICE_ERR_CFG;
+ return -EIO;
}
/**
@@ -5054,25 +5047,25 @@ ice_find_adv_rule_entry(struct ice_hw *hw, struct ice_adv_lkup_elem *lkups,
* Add the new VSI to the previously created VSI list set
* using the update switch rule command
*/
-static enum ice_status
+static int
ice_adv_add_update_vsi_list(struct ice_hw *hw,
struct ice_adv_fltr_mgmt_list_entry *m_entry,
struct ice_adv_rule_info *cur_fltr,
struct ice_adv_rule_info *new_fltr)
{
- enum ice_status status;
u16 vsi_list_id = 0;
+ int status;
if (cur_fltr->sw_act.fltr_act == ICE_FWD_TO_Q ||
cur_fltr->sw_act.fltr_act == ICE_FWD_TO_QGRP ||
cur_fltr->sw_act.fltr_act == ICE_DROP_PACKET)
- return ICE_ERR_NOT_IMPL;
+ return -EOPNOTSUPP;
if ((new_fltr->sw_act.fltr_act == ICE_FWD_TO_Q ||
new_fltr->sw_act.fltr_act == ICE_FWD_TO_QGRP) &&
(cur_fltr->sw_act.fltr_act == ICE_FWD_TO_VSI ||
cur_fltr->sw_act.fltr_act == ICE_FWD_TO_VSI_LIST))
- return ICE_ERR_NOT_IMPL;
+ return -EOPNOTSUPP;
if (m_entry->vsi_count < 2 && !m_entry->vsi_list_info) {
/* Only one entry existed in the mapping and it was not already
@@ -5085,7 +5078,7 @@ ice_adv_add_update_vsi_list(struct ice_hw *hw,
/* A rule already exists with the new VSI being added */
if (cur_fltr->sw_act.fwd_id.hw_vsi_id ==
new_fltr->sw_act.fwd_id.hw_vsi_id)
- return ICE_ERR_ALREADY_EXISTS;
+ return -EEXIST;
vsi_handle_arr[0] = cur_fltr->sw_act.vsi_handle;
vsi_handle_arr[1] = new_fltr->sw_act.vsi_handle;
@@ -5118,7 +5111,7 @@ ice_adv_add_update_vsi_list(struct ice_hw *hw,
u16 vsi_handle = new_fltr->sw_act.vsi_handle;
if (!m_entry->vsi_list_info)
- return ICE_ERR_CFG;
+ return -EIO;
/* A rule already exists with the new VSI being added */
if (test_bit(vsi_handle, m_entry->vsi_list_info->vsi_map))
@@ -5160,7 +5153,7 @@ ice_adv_add_update_vsi_list(struct ice_hw *hw,
* rinfo describes other information related to this rule such as forwarding
* IDs, priority of this rule, etc.
*/
-enum ice_status
+int
ice_add_adv_rule(struct ice_hw *hw, struct ice_adv_lkup_elem *lkups,
u16 lkups_cnt, struct ice_adv_rule_info *rinfo,
struct ice_rule_query_data *added_entry)
@@ -5171,10 +5164,10 @@ ice_add_adv_rule(struct ice_hw *hw, struct ice_adv_lkup_elem *lkups,
struct ice_aqc_sw_rules_elem *s_rule = NULL;
struct list_head *rule_head;
struct ice_switch_info *sw;
- enum ice_status status;
const u8 *pkt = NULL;
u16 word_cnt;
u32 act = 0;
+ int status;
u8 q_rgn;
/* Initialize profile to result index bitmap */
@@ -5184,7 +5177,7 @@ ice_add_adv_rule(struct ice_hw *hw, struct ice_adv_lkup_elem *lkups,
}
if (!lkups_cnt)
- return ICE_ERR_PARAM;
+ return -EINVAL;
/* get # of words we need to match */
word_cnt = 0;
@@ -5198,13 +5191,13 @@ ice_add_adv_rule(struct ice_hw *hw, struct ice_adv_lkup_elem *lkups,
}
if (!word_cnt || word_cnt > ICE_MAX_CHAIN_WORDS)
- return ICE_ERR_PARAM;
+ return -EINVAL;
/* make sure that we can locate a dummy packet */
ice_find_dummy_packet(lkups, lkups_cnt, rinfo->tun_type, &pkt, &pkt_len,
&pkt_offsets);
if (!pkt) {
- status = ICE_ERR_PARAM;
+ status = -EINVAL;
goto err_ice_add_adv_rule;
}
@@ -5212,11 +5205,11 @@ ice_add_adv_rule(struct ice_hw *hw, struct ice_adv_lkup_elem *lkups,
rinfo->sw_act.fltr_act == ICE_FWD_TO_Q ||
rinfo->sw_act.fltr_act == ICE_FWD_TO_QGRP ||
rinfo->sw_act.fltr_act == ICE_DROP_PACKET))
- return ICE_ERR_CFG;
+ return -EIO;
vsi_handle = rinfo->sw_act.vsi_handle;
if (!ice_is_vsi_valid(hw, vsi_handle))
- return ICE_ERR_PARAM;
+ return -EINVAL;
if (rinfo->sw_act.fltr_act == ICE_FWD_TO_VSI)
rinfo->sw_act.fwd_id.hw_vsi_id =
@@ -5250,7 +5243,7 @@ ice_add_adv_rule(struct ice_hw *hw, struct ice_adv_lkup_elem *lkups,
rule_buf_sz = ICE_SW_RULE_RX_TX_NO_HDR_SIZE + pkt_len;
s_rule = kzalloc(rule_buf_sz, GFP_KERNEL);
if (!s_rule)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
if (!rinfo->flags_info.act_valid) {
act |= ICE_SINGLE_ACT_LAN_ENABLE;
act |= ICE_SINGLE_ACT_LB_ENABLE;
@@ -5284,7 +5277,7 @@ ice_add_adv_rule(struct ice_hw *hw, struct ice_adv_lkup_elem *lkups,
ICE_SINGLE_ACT_VALID_BIT;
break;
default:
- status = ICE_ERR_CFG;
+ status = -EIO;
goto err_ice_add_adv_rule;
}
@@ -5329,14 +5322,14 @@ ice_add_adv_rule(struct ice_hw *hw, struct ice_adv_lkup_elem *lkups,
sizeof(struct ice_adv_fltr_mgmt_list_entry),
GFP_KERNEL);
if (!adv_fltr) {
- status = ICE_ERR_NO_MEMORY;
+ status = -ENOMEM;
goto err_ice_add_adv_rule;
}
adv_fltr->lkups = devm_kmemdup(ice_hw_to_dev(hw), lkups,
lkups_cnt * sizeof(*lkups), GFP_KERNEL);
if (!adv_fltr->lkups) {
- status = ICE_ERR_NO_MEMORY;
+ status = -ENOMEM;
goto err_ice_add_adv_rule;
}
@@ -5379,12 +5372,12 @@ ice_add_adv_rule(struct ice_hw *hw, struct ice_adv_lkup_elem *lkups,
* Replays the filter of recipe recp_id for a VSI represented via vsi_handle.
* It is required to pass valid VSI handle.
*/
-static enum ice_status
+static int
ice_replay_vsi_fltr(struct ice_hw *hw, u16 vsi_handle, u8 recp_id,
struct list_head *list_head)
{
struct ice_fltr_mgmt_list_entry *itr;
- enum ice_status status = 0;
+ int status = 0;
u16 hw_vsi_id;
if (list_empty(list_head))
@@ -5433,22 +5426,22 @@ ice_replay_vsi_fltr(struct ice_hw *hw, u16 vsi_handle, u8 recp_id,
* @fm_list: filter management entry for which the VSI list management needs to
* be done
*/
-static enum ice_status
+static int
ice_adv_rem_update_vsi_list(struct ice_hw *hw, u16 vsi_handle,
struct ice_adv_fltr_mgmt_list_entry *fm_list)
{
struct ice_vsi_list_map_info *vsi_list_info;
enum ice_sw_lkup_type lkup_type;
- enum ice_status status;
u16 vsi_list_id;
+ int status;
if (fm_list->rule_info.sw_act.fltr_act != ICE_FWD_TO_VSI_LIST ||
fm_list->vsi_count == 0)
- return ICE_ERR_PARAM;
+ return -EINVAL;
/* A rule with the VSI being removed does not exist */
if (!test_bit(vsi_handle, fm_list->vsi_list_info->vsi_map))
- return ICE_ERR_DOES_NOT_EXIST;
+ return -ENOENT;
lkup_type = ICE_SW_LKUP_LAST;
vsi_list_id = fm_list->rule_info.sw_act.fwd_id.vsi_list_id;
@@ -5468,7 +5461,7 @@ ice_adv_rem_update_vsi_list(struct ice_hw *hw, u16 vsi_handle,
rem_vsi_handle = find_first_bit(vsi_list_info->vsi_map,
ICE_MAX_VSI);
if (!ice_is_vsi_valid(hw, rem_vsi_handle))
- return ICE_ERR_OUT_OF_RANGE;
+ return -EIO;
/* Make sure VSI list is empty before removing it below */
status = ice_update_vsi_list_rule(hw, &rem_vsi_handle, 1,
@@ -5532,27 +5525,27 @@ ice_adv_rem_update_vsi_list(struct ice_hw *hw, u16 vsi_handle,
* header. rinfo describes other information related to this rule such as
* forwarding IDs, priority of this rule, etc.
*/
-static enum ice_status
+static int
ice_rem_adv_rule(struct ice_hw *hw, struct ice_adv_lkup_elem *lkups,
u16 lkups_cnt, struct ice_adv_rule_info *rinfo)
{
struct ice_adv_fltr_mgmt_list_entry *list_elem;
struct ice_prot_lkup_ext lkup_exts;
- enum ice_status status = 0;
bool remove_rule = false;
struct mutex *rule_lock; /* Lock to protect filter rule list */
u16 i, rid, vsi_handle;
+ int status = 0;
memset(&lkup_exts, 0, sizeof(lkup_exts));
for (i = 0; i < lkups_cnt; i++) {
u16 count;
if (lkups[i].type >= ICE_PROTOCOL_LAST)
- return ICE_ERR_CFG;
+ return -EIO;
count = ice_fill_valid_words(&lkups[i], &lkup_exts);
if (!count)
- return ICE_ERR_CFG;
+ return -EIO;
}
/* Create any special protocol/offset pairs, such as looking at tunnel
@@ -5565,7 +5558,7 @@ ice_rem_adv_rule(struct ice_hw *hw, struct ice_adv_lkup_elem *lkups,
rid = ice_find_recp(hw, &lkup_exts, rinfo->tun_type);
/* If did not find a recipe that match the existing criteria */
if (rid == ICE_MAX_NUM_RECIPES)
- return ICE_ERR_PARAM;
+ return -EINVAL;
rule_lock = &hw->switch_info->recp_list[rid].filt_rule_lock;
list_elem = ice_find_adv_rule_entry(hw, lkups, lkups_cnt, rid, rinfo);
@@ -5597,7 +5590,7 @@ ice_rem_adv_rule(struct ice_hw *hw, struct ice_adv_lkup_elem *lkups,
rule_buf_sz = ICE_SW_RULE_RX_TX_NO_HDR_SIZE;
s_rule = kzalloc(rule_buf_sz, GFP_KERNEL);
if (!s_rule)
- return ICE_ERR_NO_MEMORY;
+ return -ENOMEM;
s_rule->pdata.lkup_tx_rx.act = 0;
s_rule->pdata.lkup_tx_rx.index =
cpu_to_le16(list_elem->rule_info.fltr_rule_id);
@@ -5605,7 +5598,7 @@ ice_rem_adv_rule(struct ice_hw *hw, struct ice_adv_lkup_elem *lkups,
status = ice_aq_sw_rules(hw, (struct ice_aqc_sw_rules *)s_rule,
rule_buf_sz, 1,
ice_aqc_opc_remove_sw_rules, NULL);
- if (!status || status == ICE_ERR_DOES_NOT_EXIST) {
+ if (!status || status == -ENOENT) {
struct ice_switch_info *sw = hw->switch_info;
mutex_lock(rule_lock);
@@ -5630,7 +5623,7 @@ ice_rem_adv_rule(struct ice_hw *hw, struct ice_adv_lkup_elem *lkups,
* the remove_entry parameter. This function will remove rule for a given
* vsi_handle with a given rule_id which is passed as parameter in remove_entry
*/
-enum ice_status
+int
ice_rem_adv_rule_by_id(struct ice_hw *hw,
struct ice_rule_query_data *remove_entry)
{
@@ -5641,7 +5634,7 @@ ice_rem_adv_rule_by_id(struct ice_hw *hw,
sw = hw->switch_info;
if (!sw->recp_list[remove_entry->rid].recp_created)
- return ICE_ERR_PARAM;
+ return -EINVAL;
list_head = &sw->recp_list[remove_entry->rid].filt_rules;
list_for_each_entry(list_itr, list_head, list_entry) {
if (list_itr->rule_info.fltr_rule_id ==
@@ -5653,7 +5646,92 @@ ice_rem_adv_rule_by_id(struct ice_hw *hw,
}
}
/* either list is empty or unable to find rule */
- return ICE_ERR_DOES_NOT_EXIST;
+ return -ENOENT;
+}
+
+/**
+ * ice_rem_adv_rule_for_vsi - removes existing advanced switch rules for a
+ * given VSI handle
+ * @hw: pointer to the hardware structure
+ * @vsi_handle: VSI handle for which we are supposed to remove all the rules.
+ *
+ * This function is used to remove all the rules for a given VSI and as soon
+ * as removing a rule fails, it will return immediately with the error code,
+ * else it will return success.
+ */
+int ice_rem_adv_rule_for_vsi(struct ice_hw *hw, u16 vsi_handle)
+{
+ struct ice_adv_fltr_mgmt_list_entry *list_itr, *tmp_entry;
+ struct ice_vsi_list_map_info *map_info;
+ struct ice_adv_rule_info rinfo;
+ struct list_head *list_head;
+ struct ice_switch_info *sw;
+ int status;
+ u8 rid;
+
+ sw = hw->switch_info;
+ for (rid = 0; rid < ICE_MAX_NUM_RECIPES; rid++) {
+ if (!sw->recp_list[rid].recp_created)
+ continue;
+ if (!sw->recp_list[rid].adv_rule)
+ continue;
+
+ list_head = &sw->recp_list[rid].filt_rules;
+ list_for_each_entry_safe(list_itr, tmp_entry, list_head,
+ list_entry) {
+ rinfo = list_itr->rule_info;
+
+ if (rinfo.sw_act.fltr_act == ICE_FWD_TO_VSI_LIST) {
+ map_info = list_itr->vsi_list_info;
+ if (!map_info)
+ continue;
+
+ if (!test_bit(vsi_handle, map_info->vsi_map))
+ continue;
+ } else if (rinfo.sw_act.vsi_handle != vsi_handle) {
+ continue;
+ }
+
+ rinfo.sw_act.vsi_handle = vsi_handle;
+ status = ice_rem_adv_rule(hw, list_itr->lkups,
+ list_itr->lkups_cnt, &rinfo);
+ if (status)
+ return status;
+ }
+ }
+ return 0;
+}
+
+/**
+ * ice_replay_vsi_adv_rule - Replay advanced rule for requested VSI
+ * @hw: pointer to the hardware structure
+ * @vsi_handle: driver VSI handle
+ * @list_head: list for which filters need to be replayed
+ *
+ * Replay the advanced rule for the given VSI.
+ */
+static int
+ice_replay_vsi_adv_rule(struct ice_hw *hw, u16 vsi_handle,
+ struct list_head *list_head)
+{
+ struct ice_rule_query_data added_entry = { 0 };
+ struct ice_adv_fltr_mgmt_list_entry *adv_fltr;
+ int status = 0;
+
+ if (list_empty(list_head))
+ return status;
+ list_for_each_entry(adv_fltr, list_head, list_entry) {
+ struct ice_adv_rule_info *rinfo = &adv_fltr->rule_info;
+ u16 lk_cnt = adv_fltr->lkups_cnt;
+
+ if (vsi_handle != rinfo->sw_act.vsi_handle)
+ continue;
+ status = ice_add_adv_rule(hw, adv_fltr->lkups, lk_cnt, rinfo,
+ &added_entry);
+ if (status)
+ break;
+ }
+ return status;
}
/**
@@ -5663,17 +5741,20 @@ ice_rem_adv_rule_by_id(struct ice_hw *hw,
*
* Replays filters for requested VSI via vsi_handle.
*/
-enum ice_status ice_replay_vsi_all_fltr(struct ice_hw *hw, u16 vsi_handle)
+int ice_replay_vsi_all_fltr(struct ice_hw *hw, u16 vsi_handle)
{
struct ice_switch_info *sw = hw->switch_info;
- enum ice_status status = 0;
+ int status;
u8 i;
- for (i = 0; i < ICE_SW_LKUP_LAST; i++) {
+ for (i = 0; i < ICE_MAX_NUM_RECIPES; i++) {
struct list_head *head;
head = &sw->recp_list[i].filt_replay_rules;
- status = ice_replay_vsi_fltr(hw, vsi_handle, i, head);
+ if (!sw->recp_list[i].adv_rule)
+ status = ice_replay_vsi_fltr(hw, vsi_handle, i, head);
+ else
+ status = ice_replay_vsi_adv_rule(hw, vsi_handle, head);
if (status)
return status;
}
diff --git a/drivers/net/ethernet/intel/ice/ice_switch.h b/drivers/net/ethernet/intel/ice/ice_switch.h
index d8a3890..d8334be 100644
--- a/drivers/net/ethernet/intel/ice/ice_switch.h
+++ b/drivers/net/ethernet/intel/ice/ice_switch.h
@@ -256,7 +256,7 @@ struct ice_vsi_list_map_info {
struct ice_fltr_list_entry {
struct list_head list_entry;
- enum ice_status status;
+ int status;
struct ice_fltr_info fltr_info;
};
@@ -302,75 +302,69 @@ enum ice_promisc_flags {
};
/* VSI related commands */
-enum ice_status
+int
ice_add_vsi(struct ice_hw *hw, u16 vsi_handle, struct ice_vsi_ctx *vsi_ctx,
struct ice_sq_cd *cd);
-enum ice_status
+int
ice_free_vsi(struct ice_hw *hw, u16 vsi_handle, struct ice_vsi_ctx *vsi_ctx,
bool keep_vsi_alloc, struct ice_sq_cd *cd);
-enum ice_status
+int
ice_update_vsi(struct ice_hw *hw, u16 vsi_handle, struct ice_vsi_ctx *vsi_ctx,
struct ice_sq_cd *cd);
bool ice_is_vsi_valid(struct ice_hw *hw, u16 vsi_handle);
struct ice_vsi_ctx *ice_get_vsi_ctx(struct ice_hw *hw, u16 vsi_handle);
void ice_clear_all_vsi_ctx(struct ice_hw *hw);
/* Switch config */
-enum ice_status ice_get_initial_sw_cfg(struct ice_hw *hw);
+int ice_get_initial_sw_cfg(struct ice_hw *hw);
-enum ice_status
+int
ice_alloc_res_cntr(struct ice_hw *hw, u8 type, u8 alloc_shared, u16 num_items,
u16 *counter_id);
-enum ice_status
+int
ice_free_res_cntr(struct ice_hw *hw, u8 type, u8 alloc_shared, u16 num_items,
u16 counter_id);
/* Switch/bridge related commands */
-enum ice_status
+int
ice_add_adv_rule(struct ice_hw *hw, struct ice_adv_lkup_elem *lkups,
u16 lkups_cnt, struct ice_adv_rule_info *rinfo,
struct ice_rule_query_data *added_entry);
-enum ice_status ice_update_sw_rule_bridge_mode(struct ice_hw *hw);
-enum ice_status ice_add_mac(struct ice_hw *hw, struct list_head *m_lst);
-enum ice_status ice_remove_mac(struct ice_hw *hw, struct list_head *m_lst);
-enum ice_status
-ice_add_eth_mac(struct ice_hw *hw, struct list_head *em_list);
-enum ice_status
-ice_remove_eth_mac(struct ice_hw *hw, struct list_head *em_list);
-int
-ice_cfg_rdma_fltr(struct ice_hw *hw, u16 vsi_handle, bool enable);
+int ice_update_sw_rule_bridge_mode(struct ice_hw *hw);
+int ice_add_vlan(struct ice_hw *hw, struct list_head *m_list);
+int ice_remove_vlan(struct ice_hw *hw, struct list_head *v_list);
+int ice_add_mac(struct ice_hw *hw, struct list_head *m_lst);
+int ice_remove_mac(struct ice_hw *hw, struct list_head *m_lst);
bool ice_mac_fltr_exist(struct ice_hw *hw, u8 *mac, u16 vsi_handle);
bool ice_vlan_fltr_exist(struct ice_hw *hw, u16 vlan_id, u16 vsi_handle);
+int ice_add_eth_mac(struct ice_hw *hw, struct list_head *em_list);
+int ice_remove_eth_mac(struct ice_hw *hw, struct list_head *em_list);
+int ice_cfg_rdma_fltr(struct ice_hw *hw, u16 vsi_handle, bool enable);
void ice_remove_vsi_fltr(struct ice_hw *hw, u16 vsi_handle);
-enum ice_status
-ice_add_vlan(struct ice_hw *hw, struct list_head *m_list);
-enum ice_status ice_remove_vlan(struct ice_hw *hw, struct list_head *v_list);
/* Promisc/defport setup for VSIs */
-enum ice_status
-ice_cfg_dflt_vsi(struct ice_hw *hw, u16 vsi_handle, bool set, u8 direction);
-enum ice_status
+int ice_cfg_dflt_vsi(struct ice_hw *hw, u16 vsi_handle, bool set, u8 direction);
+int
ice_set_vsi_promisc(struct ice_hw *hw, u16 vsi_handle, u8 promisc_mask,
u16 vid);
-enum ice_status
+int
ice_clear_vsi_promisc(struct ice_hw *hw, u16 vsi_handle, u8 promisc_mask,
u16 vid);
-enum ice_status
+int
ice_set_vlan_vsi_promisc(struct ice_hw *hw, u16 vsi_handle, u8 promisc_mask,
bool rm_vlan_promisc);
-enum ice_status
-ice_rem_adv_rule_for_vsi(struct ice_hw *hw, u16 vsi_handle);
-enum ice_status
+int ice_rem_adv_rule_for_vsi(struct ice_hw *hw, u16 vsi_handle);
+int
ice_rem_adv_rule_by_id(struct ice_hw *hw,
struct ice_rule_query_data *remove_entry);
-enum ice_status ice_init_def_sw_recp(struct ice_hw *hw);
+int ice_init_def_sw_recp(struct ice_hw *hw);
u16 ice_get_hw_vsi_num(struct ice_hw *hw, u16 vsi_handle);
-enum ice_status ice_replay_vsi_all_fltr(struct ice_hw *hw, u16 vsi_handle);
+int ice_replay_vsi_all_fltr(struct ice_hw *hw, u16 vsi_handle);
void ice_rm_all_sw_replay_rule_info(struct ice_hw *hw);
-enum ice_status
+int
ice_aq_sw_rules(struct ice_hw *hw, void *rule_list, u16 rule_list_sz,
u8 num_rules, enum ice_adminq_opc opc, struct ice_sq_cd *cd);
#endif /* _ICE_SWITCH_H_ */
diff --git a/drivers/net/ethernet/intel/ice/ice_tc_lib.c b/drivers/net/ethernet/intel/ice/ice_tc_lib.c
index 25cca5c..e8aab66 100644
--- a/drivers/net/ethernet/intel/ice/ice_tc_lib.c
+++ b/drivers/net/ethernet/intel/ice/ice_tc_lib.c
@@ -398,9 +398,8 @@ ice_eswitch_add_tc_fltr(struct ice_vsi *vsi, struct ice_tc_flower_fltr *fltr)
struct ice_hw *hw = &vsi->back->hw;
struct ice_adv_lkup_elem *list;
u32 flags = fltr->flags;
- enum ice_status status;
int lkups_cnt;
- int ret = 0;
+ int ret;
int i;
if (!flags || (flags & ICE_TC_FLWR_FIELD_ENC_SRC_L4_PORT)) {
@@ -449,14 +448,13 @@ ice_eswitch_add_tc_fltr(struct ice_vsi *vsi, struct ice_tc_flower_fltr *fltr)
/* specify the cookie as filter_rule_id */
rule_info.fltr_rule_id = fltr->cookie;
- status = ice_add_adv_rule(hw, list, lkups_cnt, &rule_info, &rule_added);
- if (status == ICE_ERR_ALREADY_EXISTS) {
+ ret = ice_add_adv_rule(hw, list, lkups_cnt, &rule_info, &rule_added);
+ if (ret == -EEXIST) {
NL_SET_ERR_MSG_MOD(fltr->extack, "Unable to add filter because it already exist");
ret = -EINVAL;
goto exit;
- } else if (status) {
+ } else if (ret) {
NL_SET_ERR_MSG_MOD(fltr->extack, "Unable to add filter due to error");
- ret = -EIO;
goto exit;
}
@@ -1162,7 +1160,7 @@ static int ice_del_tc_fltr(struct ice_vsi *vsi, struct ice_tc_flower_fltr *fltr)
rule_rem.vsi_handle = fltr->dest_id;
err = ice_rem_adv_rule_by_id(&pf->hw, &rule_rem);
if (err) {
- if (err == ICE_ERR_DOES_NOT_EXIST) {
+ if (err == -ENOENT) {
NL_SET_ERR_MSG_MOD(fltr->extack, "Filter does not exist");
return -ENOENT;
}
diff --git a/drivers/net/ethernet/intel/ice/ice_txrx.c b/drivers/net/ethernet/intel/ice/ice_txrx.c
index dccf09e..3e38695 100644
--- a/drivers/net/ethernet/intel/ice/ice_txrx.c
+++ b/drivers/net/ethernet/intel/ice/ice_txrx.c
@@ -3,8 +3,9 @@
/* The driver transmit and receive code */
-#include <linux/prefetch.h>
#include <linux/mm.h>
+#include <linux/netdevice.h>
+#include <linux/prefetch.h>
#include <linux/bpf_trace.h>
#include <net/dsfield.h>
#include <net/xdp.h>
@@ -219,6 +220,10 @@ static bool ice_clean_tx_irq(struct ice_tx_ring *tx_ring, int napi_budget)
struct ice_tx_desc *tx_desc;
struct ice_tx_buf *tx_buf;
+ /* get the bql data ready */
+ if (!ice_ring_is_xdp(tx_ring))
+ netdev_txq_bql_complete_prefetchw(txring_txq(tx_ring));
+
tx_buf = &tx_ring->tx_buf[i];
tx_desc = ICE_TX_DESC(tx_ring, i);
i -= tx_ring->count;
@@ -232,6 +237,9 @@ static bool ice_clean_tx_irq(struct ice_tx_ring *tx_ring, int napi_budget)
if (!eop_desc)
break;
+ /* follow the guidelines of other drivers */
+ prefetchw(&tx_buf->skb->users);
+
smp_rmb(); /* prevent any other reads prior to eop_desc */
ice_trace(clean_tx_irq, tx_ring, tx_desc, tx_buf);
@@ -304,8 +312,10 @@ static bool ice_clean_tx_irq(struct ice_tx_ring *tx_ring, int napi_budget)
ice_update_tx_ring_stats(tx_ring, total_pkts, total_bytes);
- netdev_tx_completed_queue(txring_txq(tx_ring), total_pkts,
- total_bytes);
+ if (ice_ring_is_xdp(tx_ring))
+ return !!budget;
+
+ netdev_tx_completed_queue(txring_txq(tx_ring), total_pkts, total_bytes);
#define TX_WAKE_THRESHOLD ((s16)(DESC_NEEDED * 2))
if (unlikely(total_pkts && netif_carrier_ok(tx_ring->netdev) &&
@@ -314,11 +324,9 @@ static bool ice_clean_tx_irq(struct ice_tx_ring *tx_ring, int napi_budget)
* sees the new next_to_clean.
*/
smp_mb();
- if (__netif_subqueue_stopped(tx_ring->netdev,
- tx_ring->q_index) &&
+ if (netif_tx_queue_stopped(txring_txq(tx_ring)) &&
!test_bit(ICE_VSI_DOWN, vsi->state)) {
- netif_wake_subqueue(tx_ring->netdev,
- tx_ring->q_index);
+ netif_tx_wake_queue(txring_txq(tx_ring));
++tx_ring->tx_stats.restart_q;
}
}
@@ -568,7 +576,7 @@ ice_run_xdp(struct ice_rx_ring *rx_ring, struct xdp_buff *xdp,
goto out_failure;
return ICE_XDP_REDIR;
default:
- bpf_warn_invalid_xdp_action(act);
+ bpf_warn_invalid_xdp_action(rx_ring->netdev, xdp_prog, act);
fallthrough;
case XDP_ABORTED:
out_failure:
@@ -940,7 +948,7 @@ ice_build_skb(struct ice_rx_ring *rx_ring, struct ice_rx_buf *rx_buf,
*/
net_prefetch(xdp->data_meta);
/* build an skb around the page buffer */
- skb = build_skb(xdp->data_hard_start, truesize);
+ skb = napi_build_skb(xdp->data_hard_start, truesize);
if (unlikely(!skb))
return NULL;
@@ -1524,7 +1532,7 @@ int ice_napi_poll(struct napi_struct *napi, int budget)
*/
static int __ice_maybe_stop_tx(struct ice_tx_ring *tx_ring, unsigned int size)
{
- netif_stop_subqueue(tx_ring->netdev, tx_ring->q_index);
+ netif_tx_stop_queue(txring_txq(tx_ring));
/* Memory barrier before checking head and tail */
smp_mb();
@@ -1532,8 +1540,8 @@ static int __ice_maybe_stop_tx(struct ice_tx_ring *tx_ring, unsigned int size)
if (likely(ICE_DESC_UNUSED(tx_ring) < size))
return -EBUSY;
- /* A reprieve! - use start_subqueue because it doesn't call schedule */
- netif_start_subqueue(tx_ring->netdev, tx_ring->q_index);
+ /* A reprieve! - use start_queue because it doesn't call schedule */
+ netif_tx_start_queue(txring_txq(tx_ring));
++tx_ring->tx_stats.restart_q;
return 0;
}
@@ -1575,6 +1583,7 @@ ice_tx_map(struct ice_tx_ring *tx_ring, struct ice_tx_buf *first,
struct sk_buff *skb;
skb_frag_t *frag;
dma_addr_t dma;
+ bool kick;
td_tag = off->td_l2tag1;
td_cmd = off->td_cmd;
@@ -1656,9 +1665,6 @@ ice_tx_map(struct ice_tx_ring *tx_ring, struct ice_tx_buf *first,
tx_buf = &tx_ring->tx_buf[i];
}
- /* record bytecount for BQL */
- netdev_tx_sent_queue(txring_txq(tx_ring), first->bytecount);
-
/* record SW timestamp if HW timestamp is not available */
skb_tx_timestamp(first->skb);
@@ -1687,7 +1693,10 @@ ice_tx_map(struct ice_tx_ring *tx_ring, struct ice_tx_buf *first,
ice_maybe_stop_tx(tx_ring, DESC_NEEDED);
/* notify HW of packet */
- if (netif_xmit_stopped(txring_txq(tx_ring)) || !netdev_xmit_more())
+ kick = __netdev_tx_sent_queue(txring_txq(tx_ring), first->bytecount,
+ netdev_xmit_more());
+ if (kick)
+ /* notify HW of packet */
writel(i, tx_ring->tail);
return;
@@ -2272,6 +2281,9 @@ ice_xmit_frame_ring(struct sk_buff *skb, struct ice_tx_ring *tx_ring)
return NETDEV_TX_BUSY;
}
+ /* prefetch for bql data which is infrequently used */
+ netdev_txq_bql_enqueue_prefetchw(txring_txq(tx_ring));
+
offload.tx_ring = tx_ring;
/* record the location of the first descriptor for this packet */
diff --git a/drivers/net/ethernet/intel/ice/ice_txrx_lib.c b/drivers/net/ethernet/intel/ice/ice_txrx_lib.c
index 1dd7e84..0e87b98 100644
--- a/drivers/net/ethernet/intel/ice/ice_txrx_lib.c
+++ b/drivers/net/ethernet/intel/ice/ice_txrx_lib.c
@@ -1,6 +1,8 @@
// SPDX-License-Identifier: GPL-2.0
/* Copyright (c) 2019, Intel Corporation. */
+#include <linux/filter.h>
+
#include "ice_txrx_lib.h"
#include "ice_eswitch.h"
#include "ice_lib.h"
@@ -187,8 +189,7 @@ ice_process_skb_fields(struct ice_rx_ring *rx_ring,
ice_rx_hash(rx_ring, rx_desc, skb, ptype);
/* modifies the skb - consumes the enet header */
- skb->protocol = eth_type_trans(skb, ice_eswitch_get_target_netdev
- (rx_ring, rx_desc));
+ skb->protocol = eth_type_trans(skb, rx_ring->netdev);
ice_rx_csum(rx_ring, skb, rx_desc, ptype);
diff --git a/drivers/net/ethernet/intel/ice/ice_type.h b/drivers/net/ethernet/intel/ice/ice_type.h
index 9e0c292..546145d 100644
--- a/drivers/net/ethernet/intel/ice/ice_type.h
+++ b/drivers/net/ethernet/intel/ice/ice_type.h
@@ -6,8 +6,8 @@
#define ICE_BYTES_PER_WORD 2
#define ICE_BYTES_PER_DWORD 4
+#define ICE_CHNL_MAX_TC 16
-#include "ice_status.h"
#include "ice_hw_autogen.h"
#include "ice_osdep.h"
#include "ice_controlq.h"
@@ -230,8 +230,8 @@ enum ice_fd_hw_seg {
ICE_FD_HW_SEG_MAX,
};
-/* 2 VSI = 1 ICE_VSI_PF + 1 ICE_VSI_CTRL */
-#define ICE_MAX_FDIR_VSI_PER_FILTER 2
+/* 1 ICE_VSI_PF + 1 ICE_VSI_CTRL + ICE_CHNL_MAX_TC */
+#define ICE_MAX_FDIR_VSI_PER_FILTER (2 + ICE_CHNL_MAX_TC)
struct ice_fd_hw_prof {
struct ice_flow_seg_info *fdir_seg[ICE_FD_HW_SEG_MAX];
@@ -279,6 +279,10 @@ struct ice_hw_common_caps {
#define ICE_NVM_PENDING_NETLIST BIT(2)
bool nvm_unified_update;
#define ICE_NVM_MGMT_UNIFIED_UPD_SUPPORT BIT(3)
+ /* PCIe reset avoidance */
+ bool pcie_reset_avoidance;
+ /* Post update reset restriction */
+ bool reset_restrict_support;
};
/* IEEE 1588 TIME_SYNC specific info */
@@ -295,9 +299,30 @@ struct ice_hw_common_caps {
#define ICE_TS_TMR_IDX_ASSOC_S 24
#define ICE_TS_TMR_IDX_ASSOC_M BIT(24)
+/* TIME_REF clock rate specification */
+enum ice_time_ref_freq {
+ ICE_TIME_REF_FREQ_25_000 = 0,
+ ICE_TIME_REF_FREQ_122_880 = 1,
+ ICE_TIME_REF_FREQ_125_000 = 2,
+ ICE_TIME_REF_FREQ_153_600 = 3,
+ ICE_TIME_REF_FREQ_156_250 = 4,
+ ICE_TIME_REF_FREQ_245_760 = 5,
+
+ NUM_ICE_TIME_REF_FREQ
+};
+
+/* Clock source specification */
+enum ice_clk_src {
+ ICE_CLK_SRC_TCX0 = 0, /* Temperature compensated oscillator */
+ ICE_CLK_SRC_TIME_REF = 1, /* Use TIME_REF reference clock */
+
+ NUM_ICE_CLK_SRC
+};
+
struct ice_ts_func_info {
/* Function specific info */
- u32 clk_freq;
+ enum ice_time_ref_freq time_ref;
+ u8 clk_freq;
u8 clk_src;
u8 tmr_index_assoc;
u8 ena;
@@ -873,8 +898,6 @@ struct ice_hw {
u8 active_pkg_name[ICE_PKG_NAME_SIZE];
u8 active_pkg_in_nvm;
- enum ice_aq_err pkg_dwnld_status;
-
/* Driver's package ver - (from the Ice Metadata section) */
struct ice_pkg_ver pkg_ver;
u8 pkg_name[ICE_PKG_NAME_SIZE];
@@ -919,6 +942,7 @@ struct ice_hw {
struct mutex rss_locks; /* protect RSS configuration */
struct list_head rss_list_head;
struct ice_mbx_snapshot mbx_snapshot;
+ DECLARE_BITMAP(hw_ptype, ICE_FLOW_PTYPE_MAX);
u16 io_expander_handle;
};
diff --git a/drivers/net/ethernet/intel/ice/ice_virtchnl_fdir.c b/drivers/net/ethernet/intel/ice/ice_virtchnl_fdir.c
index eee180d..d64df81 100644
--- a/drivers/net/ethernet/intel/ice/ice_virtchnl_fdir.c
+++ b/drivers/net/ethernet/intel/ice/ice_virtchnl_fdir.c
@@ -47,197 +47,6 @@ struct virtchnl_fdir_fltr_conf {
u32 flow_id;
};
-static enum virtchnl_proto_hdr_type vc_pattern_ether[] = {
- VIRTCHNL_PROTO_HDR_ETH,
- VIRTCHNL_PROTO_HDR_NONE,
-};
-
-static enum virtchnl_proto_hdr_type vc_pattern_ipv4[] = {
- VIRTCHNL_PROTO_HDR_ETH,
- VIRTCHNL_PROTO_HDR_IPV4,
- VIRTCHNL_PROTO_HDR_NONE,
-};
-
-static enum virtchnl_proto_hdr_type vc_pattern_ipv4_tcp[] = {
- VIRTCHNL_PROTO_HDR_ETH,
- VIRTCHNL_PROTO_HDR_IPV4,
- VIRTCHNL_PROTO_HDR_TCP,
- VIRTCHNL_PROTO_HDR_NONE,
-};
-
-static enum virtchnl_proto_hdr_type vc_pattern_ipv4_udp[] = {
- VIRTCHNL_PROTO_HDR_ETH,
- VIRTCHNL_PROTO_HDR_IPV4,
- VIRTCHNL_PROTO_HDR_UDP,
- VIRTCHNL_PROTO_HDR_NONE,
-};
-
-static enum virtchnl_proto_hdr_type vc_pattern_ipv4_sctp[] = {
- VIRTCHNL_PROTO_HDR_ETH,
- VIRTCHNL_PROTO_HDR_IPV4,
- VIRTCHNL_PROTO_HDR_SCTP,
- VIRTCHNL_PROTO_HDR_NONE,
-};
-
-static enum virtchnl_proto_hdr_type vc_pattern_ipv6[] = {
- VIRTCHNL_PROTO_HDR_ETH,
- VIRTCHNL_PROTO_HDR_IPV6,
- VIRTCHNL_PROTO_HDR_NONE,
-};
-
-static enum virtchnl_proto_hdr_type vc_pattern_ipv6_tcp[] = {
- VIRTCHNL_PROTO_HDR_ETH,
- VIRTCHNL_PROTO_HDR_IPV6,
- VIRTCHNL_PROTO_HDR_TCP,
- VIRTCHNL_PROTO_HDR_NONE,
-};
-
-static enum virtchnl_proto_hdr_type vc_pattern_ipv6_udp[] = {
- VIRTCHNL_PROTO_HDR_ETH,
- VIRTCHNL_PROTO_HDR_IPV6,
- VIRTCHNL_PROTO_HDR_UDP,
- VIRTCHNL_PROTO_HDR_NONE,
-};
-
-static enum virtchnl_proto_hdr_type vc_pattern_ipv6_sctp[] = {
- VIRTCHNL_PROTO_HDR_ETH,
- VIRTCHNL_PROTO_HDR_IPV6,
- VIRTCHNL_PROTO_HDR_SCTP,
- VIRTCHNL_PROTO_HDR_NONE,
-};
-
-static enum virtchnl_proto_hdr_type vc_pattern_ipv4_gtpu[] = {
- VIRTCHNL_PROTO_HDR_ETH,
- VIRTCHNL_PROTO_HDR_IPV4,
- VIRTCHNL_PROTO_HDR_UDP,
- VIRTCHNL_PROTO_HDR_GTPU_IP,
- VIRTCHNL_PROTO_HDR_NONE,
-};
-
-static enum virtchnl_proto_hdr_type vc_pattern_ipv4_gtpu_eh[] = {
- VIRTCHNL_PROTO_HDR_ETH,
- VIRTCHNL_PROTO_HDR_IPV4,
- VIRTCHNL_PROTO_HDR_UDP,
- VIRTCHNL_PROTO_HDR_GTPU_IP,
- VIRTCHNL_PROTO_HDR_GTPU_EH,
- VIRTCHNL_PROTO_HDR_NONE,
-};
-
-static enum virtchnl_proto_hdr_type vc_pattern_ipv4_l2tpv3[] = {
- VIRTCHNL_PROTO_HDR_ETH,
- VIRTCHNL_PROTO_HDR_IPV4,
- VIRTCHNL_PROTO_HDR_L2TPV3,
- VIRTCHNL_PROTO_HDR_NONE,
-};
-
-static enum virtchnl_proto_hdr_type vc_pattern_ipv6_l2tpv3[] = {
- VIRTCHNL_PROTO_HDR_ETH,
- VIRTCHNL_PROTO_HDR_IPV6,
- VIRTCHNL_PROTO_HDR_L2TPV3,
- VIRTCHNL_PROTO_HDR_NONE,
-};
-
-static enum virtchnl_proto_hdr_type vc_pattern_ipv4_esp[] = {
- VIRTCHNL_PROTO_HDR_ETH,
- VIRTCHNL_PROTO_HDR_IPV4,
- VIRTCHNL_PROTO_HDR_ESP,
- VIRTCHNL_PROTO_HDR_NONE,
-};
-
-static enum virtchnl_proto_hdr_type vc_pattern_ipv6_esp[] = {
- VIRTCHNL_PROTO_HDR_ETH,
- VIRTCHNL_PROTO_HDR_IPV6,
- VIRTCHNL_PROTO_HDR_ESP,
- VIRTCHNL_PROTO_HDR_NONE,
-};
-
-static enum virtchnl_proto_hdr_type vc_pattern_ipv4_ah[] = {
- VIRTCHNL_PROTO_HDR_ETH,
- VIRTCHNL_PROTO_HDR_IPV4,
- VIRTCHNL_PROTO_HDR_AH,
- VIRTCHNL_PROTO_HDR_NONE,
-};
-
-static enum virtchnl_proto_hdr_type vc_pattern_ipv6_ah[] = {
- VIRTCHNL_PROTO_HDR_ETH,
- VIRTCHNL_PROTO_HDR_IPV6,
- VIRTCHNL_PROTO_HDR_AH,
- VIRTCHNL_PROTO_HDR_NONE,
-};
-
-static enum virtchnl_proto_hdr_type vc_pattern_ipv4_nat_t_esp[] = {
- VIRTCHNL_PROTO_HDR_ETH,
- VIRTCHNL_PROTO_HDR_IPV4,
- VIRTCHNL_PROTO_HDR_UDP,
- VIRTCHNL_PROTO_HDR_ESP,
- VIRTCHNL_PROTO_HDR_NONE,
-};
-
-static enum virtchnl_proto_hdr_type vc_pattern_ipv6_nat_t_esp[] = {
- VIRTCHNL_PROTO_HDR_ETH,
- VIRTCHNL_PROTO_HDR_IPV6,
- VIRTCHNL_PROTO_HDR_UDP,
- VIRTCHNL_PROTO_HDR_ESP,
- VIRTCHNL_PROTO_HDR_NONE,
-};
-
-static enum virtchnl_proto_hdr_type vc_pattern_ipv4_pfcp[] = {
- VIRTCHNL_PROTO_HDR_ETH,
- VIRTCHNL_PROTO_HDR_IPV4,
- VIRTCHNL_PROTO_HDR_UDP,
- VIRTCHNL_PROTO_HDR_PFCP,
- VIRTCHNL_PROTO_HDR_NONE,
-};
-
-static enum virtchnl_proto_hdr_type vc_pattern_ipv6_pfcp[] = {
- VIRTCHNL_PROTO_HDR_ETH,
- VIRTCHNL_PROTO_HDR_IPV6,
- VIRTCHNL_PROTO_HDR_UDP,
- VIRTCHNL_PROTO_HDR_PFCP,
- VIRTCHNL_PROTO_HDR_NONE,
-};
-
-struct virtchnl_fdir_pattern_match_item {
- enum virtchnl_proto_hdr_type *list;
- u64 input_set;
- u64 *meta;
-};
-
-static const struct virtchnl_fdir_pattern_match_item vc_fdir_pattern_os[] = {
- {vc_pattern_ipv4, 0, NULL},
- {vc_pattern_ipv4_tcp, 0, NULL},
- {vc_pattern_ipv4_udp, 0, NULL},
- {vc_pattern_ipv4_sctp, 0, NULL},
- {vc_pattern_ipv6, 0, NULL},
- {vc_pattern_ipv6_tcp, 0, NULL},
- {vc_pattern_ipv6_udp, 0, NULL},
- {vc_pattern_ipv6_sctp, 0, NULL},
-};
-
-static const struct virtchnl_fdir_pattern_match_item vc_fdir_pattern_comms[] = {
- {vc_pattern_ipv4, 0, NULL},
- {vc_pattern_ipv4_tcp, 0, NULL},
- {vc_pattern_ipv4_udp, 0, NULL},
- {vc_pattern_ipv4_sctp, 0, NULL},
- {vc_pattern_ipv6, 0, NULL},
- {vc_pattern_ipv6_tcp, 0, NULL},
- {vc_pattern_ipv6_udp, 0, NULL},
- {vc_pattern_ipv6_sctp, 0, NULL},
- {vc_pattern_ether, 0, NULL},
- {vc_pattern_ipv4_gtpu, 0, NULL},
- {vc_pattern_ipv4_gtpu_eh, 0, NULL},
- {vc_pattern_ipv4_l2tpv3, 0, NULL},
- {vc_pattern_ipv6_l2tpv3, 0, NULL},
- {vc_pattern_ipv4_esp, 0, NULL},
- {vc_pattern_ipv6_esp, 0, NULL},
- {vc_pattern_ipv4_ah, 0, NULL},
- {vc_pattern_ipv6_ah, 0, NULL},
- {vc_pattern_ipv4_nat_t_esp, 0, NULL},
- {vc_pattern_ipv6_nat_t_esp, 0, NULL},
- {vc_pattern_ipv4_pfcp, 0, NULL},
- {vc_pattern_ipv6_pfcp, 0, NULL},
-};
-
struct virtchnl_fdir_inset_map {
enum virtchnl_proto_hdr_field field;
enum ice_flow_field fld;
@@ -751,7 +560,6 @@ ice_vc_fdir_write_flow_prof(struct ice_vf *vf, enum ice_fltr_ptype flow,
struct ice_flow_seg_info *old_seg;
struct ice_flow_prof *prof = NULL;
struct ice_fd_hw_prof *vf_prof;
- enum ice_status status;
struct device *dev;
struct ice_pf *pf;
struct ice_hw *hw;
@@ -794,29 +602,26 @@ ice_vc_fdir_write_flow_prof(struct ice_vf *vf, enum ice_fltr_ptype flow,
prof_id = ICE_FLOW_PROF_FD(vf_vsi->vsi_num, flow,
tun ? ICE_FLTR_PTYPE_MAX : 0);
- status = ice_flow_add_prof(hw, ICE_BLK_FD, ICE_FLOW_RX, prof_id, seg,
- tun + 1, &prof);
- ret = ice_status_to_errno(status);
+ ret = ice_flow_add_prof(hw, ICE_BLK_FD, ICE_FLOW_RX, prof_id, seg,
+ tun + 1, &prof);
if (ret) {
dev_dbg(dev, "Could not add VSI flow 0x%x for VF %d\n",
flow, vf->vf_id);
goto err_exit;
}
- status = ice_flow_add_entry(hw, ICE_BLK_FD, prof_id, vf_vsi->idx,
- vf_vsi->idx, ICE_FLOW_PRIO_NORMAL,
- seg, &entry1_h);
- ret = ice_status_to_errno(status);
+ ret = ice_flow_add_entry(hw, ICE_BLK_FD, prof_id, vf_vsi->idx,
+ vf_vsi->idx, ICE_FLOW_PRIO_NORMAL,
+ seg, &entry1_h);
if (ret) {
dev_dbg(dev, "Could not add flow 0x%x VSI entry for VF %d\n",
flow, vf->vf_id);
goto err_prof;
}
- status = ice_flow_add_entry(hw, ICE_BLK_FD, prof_id, vf_vsi->idx,
- ctrl_vsi->idx, ICE_FLOW_PRIO_NORMAL,
- seg, &entry2_h);
- ret = ice_status_to_errno(status);
+ ret = ice_flow_add_entry(hw, ICE_BLK_FD, prof_id, vf_vsi->idx,
+ ctrl_vsi->idx, ICE_FLOW_PRIO_NORMAL,
+ seg, &entry2_h);
if (ret) {
dev_dbg(dev,
"Could not add flow 0x%x Ctrl VSI entry for VF %d\n",
@@ -911,83 +716,6 @@ ice_vc_fdir_config_input_set(struct ice_vf *vf, struct virtchnl_fdir_add *fltr,
}
/**
- * ice_vc_fdir_match_pattern
- * @fltr: virtual channel add cmd buffer
- * @type: virtual channel protocol filter header type
- *
- * Matching the header type by comparing fltr and type's value.
- *
- * Return: true on success, and false on error.
- */
-static bool
-ice_vc_fdir_match_pattern(struct virtchnl_fdir_add *fltr,
- enum virtchnl_proto_hdr_type *type)
-{
- struct virtchnl_proto_hdrs *proto = &fltr->rule_cfg.proto_hdrs;
- int i = 0;
-
- while ((i < proto->count) &&
- (*type == proto->proto_hdr[i].type) &&
- (*type != VIRTCHNL_PROTO_HDR_NONE)) {
- type++;
- i++;
- }
-
- return ((i == proto->count) && (*type == VIRTCHNL_PROTO_HDR_NONE));
-}
-
-/**
- * ice_vc_fdir_get_pattern - get while list pattern
- * @vf: pointer to the VF info
- * @len: filter list length
- *
- * Return: pointer to allowed filter list
- */
-static const struct virtchnl_fdir_pattern_match_item *
-ice_vc_fdir_get_pattern(struct ice_vf *vf, int *len)
-{
- const struct virtchnl_fdir_pattern_match_item *item;
- struct ice_pf *pf = vf->pf;
- struct ice_hw *hw;
-
- hw = &pf->hw;
- if (!strncmp(hw->active_pkg_name, "ICE COMMS Package",
- sizeof(hw->active_pkg_name))) {
- item = vc_fdir_pattern_comms;
- *len = ARRAY_SIZE(vc_fdir_pattern_comms);
- } else {
- item = vc_fdir_pattern_os;
- *len = ARRAY_SIZE(vc_fdir_pattern_os);
- }
-
- return item;
-}
-
-/**
- * ice_vc_fdir_search_pattern
- * @vf: pointer to the VF info
- * @fltr: virtual channel add cmd buffer
- *
- * Search for matched pattern from supported pattern list
- *
- * Return: 0 on success, and other on error.
- */
-static int
-ice_vc_fdir_search_pattern(struct ice_vf *vf, struct virtchnl_fdir_add *fltr)
-{
- const struct virtchnl_fdir_pattern_match_item *pattern;
- int len, i;
-
- pattern = ice_vc_fdir_get_pattern(vf, &len);
-
- for (i = 0; i < len; i++)
- if (ice_vc_fdir_match_pattern(fltr, pattern[i].list))
- return 0;
-
- return -EINVAL;
-}
-
-/**
* ice_vc_fdir_parse_pattern
* @vf: pointer to the VF info
* @fltr: virtual channel add cmd buffer
@@ -1299,11 +1027,11 @@ static int
ice_vc_validate_fdir_fltr(struct ice_vf *vf, struct virtchnl_fdir_add *fltr,
struct virtchnl_fdir_fltr_conf *conf)
{
+ struct virtchnl_proto_hdrs *proto = &fltr->rule_cfg.proto_hdrs;
int ret;
- ret = ice_vc_fdir_search_pattern(vf, fltr);
- if (ret)
- return ret;
+ if (!ice_vc_validate_pattern(vf, proto))
+ return -EINVAL;
ret = ice_vc_fdir_parse_pattern(vf, fltr, conf);
if (ret)
@@ -1467,7 +1195,6 @@ static int ice_vc_fdir_write_fltr(struct ice_vf *vf,
struct ice_fdir_fltr *input = &conf->input;
struct ice_vsi *vsi, *ctrl_vsi;
struct ice_fltr_desc desc;
- enum ice_status status;
struct device *dev;
struct ice_pf *pf;
struct ice_hw *hw;
@@ -1497,8 +1224,7 @@ static int ice_vc_fdir_write_fltr(struct ice_vf *vf,
return -ENOMEM;
ice_fdir_get_prgm_desc(hw, input, &desc, add);
- status = ice_fdir_get_gen_prgm_pkt(hw, input, pkt, false, is_tun);
- ret = ice_status_to_errno(status);
+ ret = ice_fdir_get_gen_prgm_pkt(hw, input, pkt, false, is_tun);
if (ret) {
dev_dbg(dev, "Gen training pkt for VF %d ptype %d failed\n",
vf->vf_id, input->flow_type);
diff --git a/drivers/net/ethernet/intel/ice/ice_virtchnl_pf.c b/drivers/net/ethernet/intel/ice/ice_virtchnl_pf.c
index 6427e7e..39b8012 100644
--- a/drivers/net/ethernet/intel/ice/ice_virtchnl_pf.c
+++ b/drivers/net/ethernet/intel/ice/ice_virtchnl_pf.c
@@ -9,6 +9,7 @@
#include "ice_flow.h"
#include "ice_eswitch.h"
#include "ice_virtchnl_allowlist.h"
+#include "ice_flex_pipe.h"
#define FIELD_SELECTOR(proto_hdr_field) \
BIT((proto_hdr_field) & PROTO_HDR_FIELD_MASK)
@@ -18,18 +19,7 @@ struct ice_vc_hdr_match_type {
u32 ice_hdr; /* ice headers (ICE_FLOW_SEG_HDR_XXX) */
};
-static const struct ice_vc_hdr_match_type ice_vc_hdr_list_os[] = {
- {VIRTCHNL_PROTO_HDR_NONE, ICE_FLOW_SEG_HDR_NONE},
- {VIRTCHNL_PROTO_HDR_IPV4, ICE_FLOW_SEG_HDR_IPV4 |
- ICE_FLOW_SEG_HDR_IPV_OTHER},
- {VIRTCHNL_PROTO_HDR_IPV6, ICE_FLOW_SEG_HDR_IPV6 |
- ICE_FLOW_SEG_HDR_IPV_OTHER},
- {VIRTCHNL_PROTO_HDR_TCP, ICE_FLOW_SEG_HDR_TCP},
- {VIRTCHNL_PROTO_HDR_UDP, ICE_FLOW_SEG_HDR_UDP},
- {VIRTCHNL_PROTO_HDR_SCTP, ICE_FLOW_SEG_HDR_SCTP},
-};
-
-static const struct ice_vc_hdr_match_type ice_vc_hdr_list_comms[] = {
+static const struct ice_vc_hdr_match_type ice_vc_hdr_list[] = {
{VIRTCHNL_PROTO_HDR_NONE, ICE_FLOW_SEG_HDR_NONE},
{VIRTCHNL_PROTO_HDR_ETH, ICE_FLOW_SEG_HDR_ETH},
{VIRTCHNL_PROTO_HDR_S_VLAN, ICE_FLOW_SEG_HDR_VLAN},
@@ -67,83 +57,7 @@ struct ice_vc_hash_field_match_type {
};
static const struct
-ice_vc_hash_field_match_type ice_vc_hash_field_list_os[] = {
- {VIRTCHNL_PROTO_HDR_IPV4, FIELD_SELECTOR(VIRTCHNL_PROTO_HDR_IPV4_SRC),
- BIT_ULL(ICE_FLOW_FIELD_IDX_IPV4_SA)},
- {VIRTCHNL_PROTO_HDR_IPV4, FIELD_SELECTOR(VIRTCHNL_PROTO_HDR_IPV4_DST),
- BIT_ULL(ICE_FLOW_FIELD_IDX_IPV4_DA)},
- {VIRTCHNL_PROTO_HDR_IPV4, FIELD_SELECTOR(VIRTCHNL_PROTO_HDR_IPV4_SRC) |
- FIELD_SELECTOR(VIRTCHNL_PROTO_HDR_IPV4_DST),
- ICE_FLOW_HASH_IPV4},
- {VIRTCHNL_PROTO_HDR_IPV4, FIELD_SELECTOR(VIRTCHNL_PROTO_HDR_IPV4_SRC) |
- FIELD_SELECTOR(VIRTCHNL_PROTO_HDR_IPV4_PROT),
- BIT_ULL(ICE_FLOW_FIELD_IDX_IPV4_SA) |
- BIT_ULL(ICE_FLOW_FIELD_IDX_IPV4_PROT)},
- {VIRTCHNL_PROTO_HDR_IPV4, FIELD_SELECTOR(VIRTCHNL_PROTO_HDR_IPV4_DST) |
- FIELD_SELECTOR(VIRTCHNL_PROTO_HDR_IPV4_PROT),
- BIT_ULL(ICE_FLOW_FIELD_IDX_IPV4_DA) |
- BIT_ULL(ICE_FLOW_FIELD_IDX_IPV4_PROT)},
- {VIRTCHNL_PROTO_HDR_IPV4, FIELD_SELECTOR(VIRTCHNL_PROTO_HDR_IPV4_SRC) |
- FIELD_SELECTOR(VIRTCHNL_PROTO_HDR_IPV4_DST) |
- FIELD_SELECTOR(VIRTCHNL_PROTO_HDR_IPV4_PROT),
- ICE_FLOW_HASH_IPV4 | BIT_ULL(ICE_FLOW_FIELD_IDX_IPV4_PROT)},
- {VIRTCHNL_PROTO_HDR_IPV4, FIELD_SELECTOR(VIRTCHNL_PROTO_HDR_IPV4_PROT),
- BIT_ULL(ICE_FLOW_FIELD_IDX_IPV4_PROT)},
- {VIRTCHNL_PROTO_HDR_IPV6, FIELD_SELECTOR(VIRTCHNL_PROTO_HDR_IPV6_SRC),
- BIT_ULL(ICE_FLOW_FIELD_IDX_IPV6_SA)},
- {VIRTCHNL_PROTO_HDR_IPV6, FIELD_SELECTOR(VIRTCHNL_PROTO_HDR_IPV6_DST),
- BIT_ULL(ICE_FLOW_FIELD_IDX_IPV6_DA)},
- {VIRTCHNL_PROTO_HDR_IPV6, FIELD_SELECTOR(VIRTCHNL_PROTO_HDR_IPV6_SRC) |
- FIELD_SELECTOR(VIRTCHNL_PROTO_HDR_IPV6_DST),
- ICE_FLOW_HASH_IPV6},
- {VIRTCHNL_PROTO_HDR_IPV6, FIELD_SELECTOR(VIRTCHNL_PROTO_HDR_IPV6_SRC) |
- FIELD_SELECTOR(VIRTCHNL_PROTO_HDR_IPV6_PROT),
- BIT_ULL(ICE_FLOW_FIELD_IDX_IPV6_SA) |
- BIT_ULL(ICE_FLOW_FIELD_IDX_IPV6_PROT)},
- {VIRTCHNL_PROTO_HDR_IPV6, FIELD_SELECTOR(VIRTCHNL_PROTO_HDR_IPV6_DST) |
- FIELD_SELECTOR(VIRTCHNL_PROTO_HDR_IPV6_PROT),
- BIT_ULL(ICE_FLOW_FIELD_IDX_IPV6_DA) |
- BIT_ULL(ICE_FLOW_FIELD_IDX_IPV6_PROT)},
- {VIRTCHNL_PROTO_HDR_IPV6, FIELD_SELECTOR(VIRTCHNL_PROTO_HDR_IPV6_SRC) |
- FIELD_SELECTOR(VIRTCHNL_PROTO_HDR_IPV6_DST) |
- FIELD_SELECTOR(VIRTCHNL_PROTO_HDR_IPV6_PROT),
- ICE_FLOW_HASH_IPV6 | BIT_ULL(ICE_FLOW_FIELD_IDX_IPV6_PROT)},
- {VIRTCHNL_PROTO_HDR_IPV6, FIELD_SELECTOR(VIRTCHNL_PROTO_HDR_IPV6_PROT),
- BIT_ULL(ICE_FLOW_FIELD_IDX_IPV6_PROT)},
- {VIRTCHNL_PROTO_HDR_TCP,
- FIELD_SELECTOR(VIRTCHNL_PROTO_HDR_TCP_SRC_PORT),
- BIT_ULL(ICE_FLOW_FIELD_IDX_TCP_SRC_PORT)},
- {VIRTCHNL_PROTO_HDR_TCP,
- FIELD_SELECTOR(VIRTCHNL_PROTO_HDR_TCP_DST_PORT),
- BIT_ULL(ICE_FLOW_FIELD_IDX_TCP_DST_PORT)},
- {VIRTCHNL_PROTO_HDR_TCP,
- FIELD_SELECTOR(VIRTCHNL_PROTO_HDR_TCP_SRC_PORT) |
- FIELD_SELECTOR(VIRTCHNL_PROTO_HDR_TCP_DST_PORT),
- ICE_FLOW_HASH_TCP_PORT},
- {VIRTCHNL_PROTO_HDR_UDP,
- FIELD_SELECTOR(VIRTCHNL_PROTO_HDR_UDP_SRC_PORT),
- BIT_ULL(ICE_FLOW_FIELD_IDX_UDP_SRC_PORT)},
- {VIRTCHNL_PROTO_HDR_UDP,
- FIELD_SELECTOR(VIRTCHNL_PROTO_HDR_UDP_DST_PORT),
- BIT_ULL(ICE_FLOW_FIELD_IDX_UDP_DST_PORT)},
- {VIRTCHNL_PROTO_HDR_UDP,
- FIELD_SELECTOR(VIRTCHNL_PROTO_HDR_UDP_SRC_PORT) |
- FIELD_SELECTOR(VIRTCHNL_PROTO_HDR_UDP_DST_PORT),
- ICE_FLOW_HASH_UDP_PORT},
- {VIRTCHNL_PROTO_HDR_SCTP,
- FIELD_SELECTOR(VIRTCHNL_PROTO_HDR_SCTP_SRC_PORT),
- BIT_ULL(ICE_FLOW_FIELD_IDX_SCTP_SRC_PORT)},
- {VIRTCHNL_PROTO_HDR_SCTP,
- FIELD_SELECTOR(VIRTCHNL_PROTO_HDR_SCTP_DST_PORT),
- BIT_ULL(ICE_FLOW_FIELD_IDX_SCTP_DST_PORT)},
- {VIRTCHNL_PROTO_HDR_SCTP,
- FIELD_SELECTOR(VIRTCHNL_PROTO_HDR_SCTP_SRC_PORT) |
- FIELD_SELECTOR(VIRTCHNL_PROTO_HDR_SCTP_DST_PORT),
- ICE_FLOW_HASH_SCTP_PORT},
-};
-
-static const struct
-ice_vc_hash_field_match_type ice_vc_hash_field_list_comms[] = {
+ice_vc_hash_field_match_type ice_vc_hash_field_list[] = {
{VIRTCHNL_PROTO_HDR_ETH, FIELD_SELECTOR(VIRTCHNL_PROTO_HDR_ETH_SRC),
BIT_ULL(ICE_FLOW_FIELD_IDX_ETH_SA)},
{VIRTCHNL_PROTO_HDR_ETH, FIELD_SELECTOR(VIRTCHNL_PROTO_HDR_ETH_DST),
@@ -289,37 +203,6 @@ static int ice_check_vf_init(struct ice_pf *pf, struct ice_vf *vf)
}
/**
- * ice_err_to_virt_err - translate errors for VF return code
- * @ice_err: error return code
- */
-static enum virtchnl_status_code ice_err_to_virt_err(enum ice_status ice_err)
-{
- switch (ice_err) {
- case ICE_SUCCESS:
- return VIRTCHNL_STATUS_SUCCESS;
- case ICE_ERR_BAD_PTR:
- case ICE_ERR_INVAL_SIZE:
- case ICE_ERR_DEVICE_NOT_SUPPORTED:
- case ICE_ERR_PARAM:
- case ICE_ERR_CFG:
- return VIRTCHNL_STATUS_ERR_PARAM;
- case ICE_ERR_NO_MEMORY:
- return VIRTCHNL_STATUS_ERR_NO_MEMORY;
- case ICE_ERR_NOT_READY:
- case ICE_ERR_RESET_FAILED:
- case ICE_ERR_FW_API_VER:
- case ICE_ERR_AQ_ERROR:
- case ICE_ERR_AQ_TIMEOUT:
- case ICE_ERR_AQ_FULL:
- case ICE_ERR_AQ_NO_WORK:
- case ICE_ERR_AQ_EMPTY:
- return VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR;
- default:
- return VIRTCHNL_STATUS_ERR_NOT_SUPPORTED;
- }
-}
-
-/**
* ice_vc_vf_broadcast - Broadcast a message to all VFs on PF
* @pf: pointer to the PF structure
* @v_opcode: operation code
@@ -770,8 +653,7 @@ static int ice_vsi_manage_pvid(struct ice_vsi *vsi, u16 pvid_info, bool enable)
struct ice_hw *hw = &vsi->back->hw;
struct ice_aqc_vsi_props *info;
struct ice_vsi_ctx *ctxt;
- enum ice_status status;
- int ret = 0;
+ int ret;
ctxt = kzalloc(sizeof(*ctxt), GFP_KERNEL);
if (!ctxt)
@@ -794,12 +676,10 @@ static int ice_vsi_manage_pvid(struct ice_vsi *vsi, u16 pvid_info, bool enable)
info->valid_sections = cpu_to_le16(ICE_AQ_VSI_PROP_VLAN_VALID |
ICE_AQ_VSI_PROP_SW_VALID);
- status = ice_update_vsi(hw, vsi->idx, ctxt, NULL);
- if (status) {
- dev_info(ice_hw_to_dev(hw), "update VSI for port VLAN failed, err %s aq_err %s\n",
- ice_stat_str(status),
- ice_aq_str(hw->adminq.sq_last_status));
- ret = -EIO;
+ ret = ice_update_vsi(hw, vsi->idx, ctxt, NULL);
+ if (ret) {
+ dev_info(ice_hw_to_dev(hw), "update VSI for port VLAN failed, err %d aq_err %s\n",
+ ret, ice_aq_str(hw->adminq.sq_last_status));
goto out;
}
@@ -968,8 +848,8 @@ static int ice_vf_rebuild_host_mac_cfg(struct ice_vf *vf)
{
struct device *dev = ice_pf_to_dev(vf->pf);
struct ice_vsi *vsi = ice_get_vf_vsi(vf);
- enum ice_status status;
u8 broadcast[ETH_ALEN];
+ int status;
if (ice_is_eswitch_mode_switchdev(vf->pf))
return 0;
@@ -977,9 +857,9 @@ static int ice_vf_rebuild_host_mac_cfg(struct ice_vf *vf)
eth_broadcast_addr(broadcast);
status = ice_fltr_add_mac(vsi, broadcast, ICE_FWD_TO_VSI);
if (status) {
- dev_err(dev, "failed to add broadcast MAC filter for VF %u, error %s\n",
- vf->vf_id, ice_stat_str(status));
- return ice_status_to_errno(status);
+ dev_err(dev, "failed to add broadcast MAC filter for VF %u, error %d\n",
+ vf->vf_id, status);
+ return status;
}
vf->num_mac++;
@@ -988,10 +868,10 @@ static int ice_vf_rebuild_host_mac_cfg(struct ice_vf *vf)
status = ice_fltr_add_mac(vsi, vf->hw_lan_addr.addr,
ICE_FWD_TO_VSI);
if (status) {
- dev_err(dev, "failed to add default unicast MAC filter %pM for VF %u, error %s\n",
+ dev_err(dev, "failed to add default unicast MAC filter %pM for VF %u, error %d\n",
&vf->hw_lan_addr.addr[0], vf->vf_id,
- ice_stat_str(status));
- return ice_status_to_errno(status);
+ status);
+ return status;
}
vf->num_mac++;
@@ -1341,45 +1221,50 @@ static void ice_clear_vf_reset_trigger(struct ice_vf *vf)
ice_flush(hw);
}
-/**
- * ice_vf_set_vsi_promisc - set given VF VSI to given promiscuous mode(s)
- * @vf: pointer to the VF info
- * @vsi: the VSI being configured
- * @promisc_m: mask of promiscuous config bits
- * @rm_promisc: promisc flag request from the VF to remove or add filter
- *
- * This function configures VF VSI promiscuous mode, based on the VF requests,
- * for Unicast, Multicast and VLAN
- */
-static enum ice_status
-ice_vf_set_vsi_promisc(struct ice_vf *vf, struct ice_vsi *vsi, u8 promisc_m,
- bool rm_promisc)
+static int
+ice_vf_set_vsi_promisc(struct ice_vf *vf, struct ice_vsi *vsi, u8 promisc_m)
{
- struct ice_pf *pf = vf->pf;
- enum ice_status status = 0;
- struct ice_hw *hw;
+ struct ice_hw *hw = &vsi->back->hw;
+ int status;
- hw = &pf->hw;
- if (vsi->num_vlan) {
- status = ice_set_vlan_vsi_promisc(hw, vsi->idx, promisc_m,
- rm_promisc);
- } else if (vf->port_vlan_info) {
- if (rm_promisc)
- status = ice_clear_vsi_promisc(hw, vsi->idx, promisc_m,
- vf->port_vlan_info);
- else
- status = ice_set_vsi_promisc(hw, vsi->idx, promisc_m,
- vf->port_vlan_info);
- } else {
- if (rm_promisc)
- status = ice_clear_vsi_promisc(hw, vsi->idx, promisc_m,
- 0);
- else
- status = ice_set_vsi_promisc(hw, vsi->idx, promisc_m,
- 0);
+ if (vf->port_vlan_info)
+ status = ice_fltr_set_vsi_promisc(hw, vsi->idx, promisc_m,
+ vf->port_vlan_info & VLAN_VID_MASK);
+ else if (vsi->num_vlan > 1)
+ status = ice_fltr_set_vlan_vsi_promisc(hw, vsi, promisc_m);
+ else
+ status = ice_fltr_set_vsi_promisc(hw, vsi->idx, promisc_m, 0);
+
+ if (status && status != -EEXIST) {
+ dev_err(ice_pf_to_dev(vsi->back), "enable Tx/Rx filter promiscuous mode on VF-%u failed, error: %d\n",
+ vf->vf_id, status);
+ return status;
}
- return status;
+ return 0;
+}
+
+static int
+ice_vf_clear_vsi_promisc(struct ice_vf *vf, struct ice_vsi *vsi, u8 promisc_m)
+{
+ struct ice_hw *hw = &vsi->back->hw;
+ int status;
+
+ if (vf->port_vlan_info)
+ status = ice_fltr_clear_vsi_promisc(hw, vsi->idx, promisc_m,
+ vf->port_vlan_info & VLAN_VID_MASK);
+ else if (vsi->num_vlan > 1)
+ status = ice_fltr_clear_vlan_vsi_promisc(hw, vsi, promisc_m);
+ else
+ status = ice_fltr_clear_vsi_promisc(hw, vsi->idx, promisc_m, 0);
+
+ if (status && status != -ENOENT) {
+ dev_err(ice_pf_to_dev(vsi->back), "disable Tx/Rx filter promiscuous mode on VF-%u failed, error: %d\n",
+ vf->vf_id, status);
+ return status;
+ }
+
+ return 0;
}
static void ice_vf_clear_counters(struct ice_vf *vf)
@@ -1415,8 +1300,8 @@ static void ice_vf_pre_vsi_rebuild(struct ice_vf *vf)
static void ice_vf_rebuild_aggregator_node_cfg(struct ice_vsi *vsi)
{
struct ice_pf *pf = vsi->back;
- enum ice_status status;
struct device *dev;
+ int status;
if (!vsi->agg_node)
return;
@@ -1743,10 +1628,12 @@ bool ice_reset_vf(struct ice_vf *vf, bool is_vflr)
else
promisc_m = ICE_UCAST_PROMISC_BITS;
- if (ice_vf_set_vsi_promisc(vf, vsi, promisc_m, true))
+ if (ice_vf_clear_vsi_promisc(vf, vsi, promisc_m))
dev_err(dev, "disabling promiscuous mode failed\n");
}
+ ice_eswitch_del_vf_mac_rule(vf);
+
ice_vf_fdir_exit(vf);
ice_vf_fdir_init(vf);
/* clean VF control VSI when resetting VF since it should be setup
@@ -1765,6 +1652,7 @@ bool ice_reset_vf(struct ice_vf *vf, bool is_vflr)
ice_vf_post_vsi_rebuild(vf);
vsi = ice_get_vf_vsi(vf);
ice_eswitch_update_repr(vsi);
+ ice_eswitch_replay_vf_mac_rule(vf);
/* if the VF has been reset allow it to come up again */
if (ice_mbx_clear_malvf(&hw->mbx_snapshot, pf->malvfs, ICE_MAX_VF_COUNT, vf->vf_id))
@@ -1846,7 +1734,6 @@ static int ice_init_vf_vsi_res(struct ice_vf *vf)
{
struct ice_pf *pf = vf->pf;
u8 broadcast[ETH_ALEN];
- enum ice_status status;
struct ice_vsi *vsi;
struct device *dev;
int err;
@@ -1866,11 +1753,10 @@ static int ice_init_vf_vsi_res(struct ice_vf *vf)
}
eth_broadcast_addr(broadcast);
- status = ice_fltr_add_mac(vsi, broadcast, ICE_FWD_TO_VSI);
- if (status) {
- dev_err(dev, "Failed to add broadcast MAC filter for VF %d, status %s\n",
- vf->vf_id, ice_stat_str(status));
- err = ice_status_to_errno(status);
+ err = ice_fltr_add_mac(vsi, broadcast, ICE_FWD_TO_VSI);
+ if (err) {
+ dev_err(dev, "Failed to add broadcast MAC filter for VF %d, error %d\n",
+ vf->vf_id, err);
goto release_vsi;
}
@@ -2116,7 +2002,6 @@ int ice_sriov_configure(struct pci_dev *pdev, int num_vfs)
{
struct ice_pf *pf = pci_get_drvdata(pdev);
struct device *dev = ice_pf_to_dev(pf);
- enum ice_status status;
int err;
err = ice_check_sriov_allowed(pf);
@@ -2136,9 +2021,9 @@ int ice_sriov_configure(struct pci_dev *pdev, int num_vfs)
return -EBUSY;
}
- status = ice_mbx_init_snapshot(&pf->hw, num_vfs);
- if (status)
- return ice_status_to_errno(status);
+ err = ice_mbx_init_snapshot(&pf->hw, num_vfs);
+ if (err)
+ return err;
err = ice_pci_sriov_ena(pf, num_vfs);
if (err) {
@@ -2272,9 +2157,9 @@ int
ice_vc_send_msg_to_vf(struct ice_vf *vf, u32 v_opcode,
enum virtchnl_status_code v_retval, u8 *msg, u16 msglen)
{
- enum ice_status aq_ret;
struct device *dev;
struct ice_pf *pf;
+ int aq_ret;
if (!vf)
return -EINVAL;
@@ -2306,8 +2191,8 @@ ice_vc_send_msg_to_vf(struct ice_vf *vf, u32 v_opcode,
aq_ret = ice_aq_send_msg_to_vf(&pf->hw, vf->vf_id, v_opcode, v_retval,
msg, msglen, NULL);
if (aq_ret && pf->hw.mailboxq.sq_last_status != ICE_AQ_RC_ENOSYS) {
- dev_info(dev, "Unable to send the message to VF %d ret %s aq_err %s\n",
- vf->vf_id, ice_stat_str(aq_ret),
+ dev_info(dev, "Unable to send the message to VF %d ret %d aq_err %s\n",
+ vf->vf_id, aq_ret,
ice_aq_str(pf->hw.mailboxq.sq_last_status));
return -EIO;
}
@@ -2556,6 +2441,100 @@ static bool ice_vc_isvalid_ring_len(u16 ring_len)
}
/**
+ * ice_vc_validate_pattern
+ * @vf: pointer to the VF info
+ * @proto: virtchnl protocol headers
+ *
+ * validate the pattern is supported or not.
+ *
+ * Return: true on success, false on error.
+ */
+bool
+ice_vc_validate_pattern(struct ice_vf *vf, struct virtchnl_proto_hdrs *proto)
+{
+ bool is_ipv4 = false;
+ bool is_ipv6 = false;
+ bool is_udp = false;
+ u16 ptype = -1;
+ int i = 0;
+
+ while (i < proto->count &&
+ proto->proto_hdr[i].type != VIRTCHNL_PROTO_HDR_NONE) {
+ switch (proto->proto_hdr[i].type) {
+ case VIRTCHNL_PROTO_HDR_ETH:
+ ptype = ICE_PTYPE_MAC_PAY;
+ break;
+ case VIRTCHNL_PROTO_HDR_IPV4:
+ ptype = ICE_PTYPE_IPV4_PAY;
+ is_ipv4 = true;
+ break;
+ case VIRTCHNL_PROTO_HDR_IPV6:
+ ptype = ICE_PTYPE_IPV6_PAY;
+ is_ipv6 = true;
+ break;
+ case VIRTCHNL_PROTO_HDR_UDP:
+ if (is_ipv4)
+ ptype = ICE_PTYPE_IPV4_UDP_PAY;
+ else if (is_ipv6)
+ ptype = ICE_PTYPE_IPV6_UDP_PAY;
+ is_udp = true;
+ break;
+ case VIRTCHNL_PROTO_HDR_TCP:
+ if (is_ipv4)
+ ptype = ICE_PTYPE_IPV4_TCP_PAY;
+ else if (is_ipv6)
+ ptype = ICE_PTYPE_IPV6_TCP_PAY;
+ break;
+ case VIRTCHNL_PROTO_HDR_SCTP:
+ if (is_ipv4)
+ ptype = ICE_PTYPE_IPV4_SCTP_PAY;
+ else if (is_ipv6)
+ ptype = ICE_PTYPE_IPV6_SCTP_PAY;
+ break;
+ case VIRTCHNL_PROTO_HDR_GTPU_IP:
+ case VIRTCHNL_PROTO_HDR_GTPU_EH:
+ if (is_ipv4)
+ ptype = ICE_MAC_IPV4_GTPU;
+ else if (is_ipv6)
+ ptype = ICE_MAC_IPV6_GTPU;
+ goto out;
+ case VIRTCHNL_PROTO_HDR_L2TPV3:
+ if (is_ipv4)
+ ptype = ICE_MAC_IPV4_L2TPV3;
+ else if (is_ipv6)
+ ptype = ICE_MAC_IPV6_L2TPV3;
+ goto out;
+ case VIRTCHNL_PROTO_HDR_ESP:
+ if (is_ipv4)
+ ptype = is_udp ? ICE_MAC_IPV4_NAT_T_ESP :
+ ICE_MAC_IPV4_ESP;
+ else if (is_ipv6)
+ ptype = is_udp ? ICE_MAC_IPV6_NAT_T_ESP :
+ ICE_MAC_IPV6_ESP;
+ goto out;
+ case VIRTCHNL_PROTO_HDR_AH:
+ if (is_ipv4)
+ ptype = ICE_MAC_IPV4_AH;
+ else if (is_ipv6)
+ ptype = ICE_MAC_IPV6_AH;
+ goto out;
+ case VIRTCHNL_PROTO_HDR_PFCP:
+ if (is_ipv4)
+ ptype = ICE_MAC_IPV4_PFCP_SESSION;
+ else if (is_ipv6)
+ ptype = ICE_MAC_IPV6_PFCP_SESSION;
+ goto out;
+ default:
+ break;
+ }
+ i++;
+ }
+
+out:
+ return ice_hw_ptype_ena(&vf->pf->hw, ptype);
+}
+
+/**
* ice_vc_parse_rss_cfg - parses hash fields and headers from
* a specific virtchnl RSS cfg
* @hw: pointer to the hardware
@@ -2578,18 +2557,10 @@ ice_vc_parse_rss_cfg(struct ice_hw *hw, struct virtchnl_rss_cfg *rss_cfg,
const struct ice_vc_hdr_match_type *hdr_list;
int i, hf_list_len, hdr_list_len;
- if (!strncmp(hw->active_pkg_name, "ICE COMMS Package",
- sizeof(hw->active_pkg_name))) {
- hf_list = ice_vc_hash_field_list_comms;
- hf_list_len = ARRAY_SIZE(ice_vc_hash_field_list_comms);
- hdr_list = ice_vc_hdr_list_comms;
- hdr_list_len = ARRAY_SIZE(ice_vc_hdr_list_comms);
- } else {
- hf_list = ice_vc_hash_field_list_os;
- hf_list_len = ARRAY_SIZE(ice_vc_hash_field_list_os);
- hdr_list = ice_vc_hdr_list_os;
- hdr_list_len = ARRAY_SIZE(ice_vc_hdr_list_os);
- }
+ hf_list = ice_vc_hash_field_list;
+ hf_list_len = ARRAY_SIZE(ice_vc_hash_field_list);
+ hdr_list = ice_vc_hdr_list;
+ hdr_list_len = ARRAY_SIZE(ice_vc_hdr_list);
for (i = 0; i < rss_cfg->proto_hdrs.count; i++) {
struct virtchnl_proto_hdr *proto_hdr =
@@ -2691,10 +2662,15 @@ static int ice_vc_handle_rss_cfg(struct ice_vf *vf, u8 *msg, bool add)
goto error_param;
}
+ if (!ice_vc_validate_pattern(vf, &rss_cfg->proto_hdrs)) {
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ goto error_param;
+ }
+
if (rss_cfg->rss_algorithm == VIRTCHNL_RSS_ALG_R_ASYMMETRIC) {
struct ice_vsi_ctx *ctx;
- enum ice_status status;
u8 lut_type, hash_type;
+ int status;
lut_type = ICE_AQ_VSI_Q_OPT_RSS_LUT_VSI;
hash_type = add ? ICE_AQ_VSI_Q_OPT_RSS_XOR :
@@ -2723,9 +2699,8 @@ static int ice_vc_handle_rss_cfg(struct ice_vf *vf, u8 *msg, bool add)
status = ice_update_vsi(hw, vsi->idx, ctx, NULL);
if (status) {
- dev_err(dev, "update VSI for RSS failed, err %s aq_err %s\n",
- ice_stat_str(status),
- ice_aq_str(hw->adminq.sq_last_status));
+ dev_err(dev, "update VSI for RSS failed, err %d aq_err %s\n",
+ status, ice_aq_str(hw->adminq.sq_last_status));
v_ret = VIRTCHNL_STATUS_ERR_PARAM;
} else {
vsi->info.q_opt_rss = ctx->info.q_opt_rss;
@@ -2750,19 +2725,18 @@ static int ice_vc_handle_rss_cfg(struct ice_vf *vf, u8 *msg, bool add)
vsi->vsi_num, v_ret);
}
} else {
- enum ice_status status;
+ int status;
status = ice_rem_rss_cfg(hw, vsi->idx, hash_flds,
addl_hdrs);
- /* We just ignore ICE_ERR_DOES_NOT_EXIST, because
- * if two configurations share the same profile remove
- * one of them actually removes both, since the
- * profile is deleted.
+ /* We just ignore -ENOENT, because if two configurations
+ * share the same profile remove one of them actually
+ * removes both, since the profile is deleted.
*/
- if (status && status != ICE_ERR_DOES_NOT_EXIST) {
+ if (status && status != -ENOENT) {
v_ret = VIRTCHNL_STATUS_ERR_PARAM;
- dev_err(dev, "ice_rem_rss_cfg failed for VF ID:%d, error:%s\n",
- vf->vf_id, ice_stat_str(status));
+ dev_err(dev, "ice_rem_rss_cfg failed for VF ID:%d, error:%d\n",
+ vf->vf_id, status);
}
}
}
@@ -2920,7 +2894,6 @@ int ice_set_vf_spoofchk(struct net_device *netdev, int vf_id, bool ena)
struct ice_pf *pf = np->vsi->back;
struct ice_vsi_ctx *ctx;
struct ice_vsi *vf_vsi;
- enum ice_status status;
struct device *dev;
struct ice_vf *vf;
int ret;
@@ -2970,12 +2943,10 @@ int ice_set_vf_spoofchk(struct net_device *netdev, int vf_id, bool ena)
ICE_AQ_VSI_SEC_TX_PRUNE_ENA_S));
}
- status = ice_update_vsi(&pf->hw, vf_vsi->idx, ctx, NULL);
- if (status) {
- dev_err(dev, "Failed to %sable spoofchk on VF %d VSI %d\n error %s\n",
- ena ? "en" : "dis", vf->vf_id, vf_vsi->vsi_num,
- ice_stat_str(status));
- ret = -EIO;
+ ret = ice_update_vsi(&pf->hw, vf_vsi->idx, ctx, NULL);
+ if (ret) {
+ dev_err(dev, "Failed to %sable spoofchk on VF %d VSI %d\n error %d\n",
+ ena ? "en" : "dis", vf->vf_id, vf_vsi->vsi_num, ret);
goto out;
}
@@ -3021,10 +2992,10 @@ bool ice_is_any_vf_in_promisc(struct ice_pf *pf)
static int ice_vc_cfg_promiscuous_mode_msg(struct ice_vf *vf, u8 *msg)
{
enum virtchnl_status_code v_ret = VIRTCHNL_STATUS_SUCCESS;
- enum ice_status mcast_status = 0, ucast_status = 0;
bool rm_promisc, alluni = false, allmulti = false;
struct virtchnl_promisc_info *info =
(struct virtchnl_promisc_info *)msg;
+ int mcast_err = 0, ucast_err = 0;
struct ice_pf *pf = vf->pf;
struct ice_vsi *vsi;
struct device *dev;
@@ -3106,24 +3077,21 @@ static int ice_vc_cfg_promiscuous_mode_msg(struct ice_vf *vf, u8 *msg)
ucast_m = ICE_UCAST_PROMISC_BITS;
}
- ucast_status = ice_vf_set_vsi_promisc(vf, vsi, ucast_m,
- !alluni);
- if (ucast_status) {
- dev_err(dev, "%sable Tx/Rx filter promiscuous mode on VF-%d failed\n",
- alluni ? "en" : "dis", vf->vf_id);
- v_ret = ice_err_to_virt_err(ucast_status);
- }
+ if (alluni)
+ ucast_err = ice_vf_set_vsi_promisc(vf, vsi, ucast_m);
+ else
+ ucast_err = ice_vf_clear_vsi_promisc(vf, vsi, ucast_m);
- mcast_status = ice_vf_set_vsi_promisc(vf, vsi, mcast_m,
- !allmulti);
- if (mcast_status) {
- dev_err(dev, "%sable Tx/Rx filter promiscuous mode on VF-%d failed\n",
- allmulti ? "en" : "dis", vf->vf_id);
- v_ret = ice_err_to_virt_err(mcast_status);
- }
+ if (allmulti)
+ mcast_err = ice_vf_set_vsi_promisc(vf, vsi, mcast_m);
+ else
+ mcast_err = ice_vf_clear_vsi_promisc(vf, vsi, mcast_m);
+
+ if (ucast_err || mcast_err)
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
}
- if (!mcast_status) {
+ if (!mcast_err) {
if (allmulti &&
!test_and_set_bit(ICE_VF_STATE_MC_PROMISC, vf->vf_states))
dev_info(dev, "VF %u successfully set multicast promiscuous mode\n",
@@ -3133,7 +3101,7 @@ static int ice_vc_cfg_promiscuous_mode_msg(struct ice_vf *vf, u8 *msg)
vf->vf_id);
}
- if (!ucast_status) {
+ if (!ucast_err) {
if (alluni && !test_and_set_bit(ICE_VF_STATE_UC_PROMISC, vf->vf_states))
dev_info(dev, "VF %u successfully set unicast promiscuous mode\n",
vf->vf_id);
@@ -3813,8 +3781,7 @@ ice_vc_add_mac_addr(struct ice_vf *vf, struct ice_vsi *vsi,
{
struct device *dev = ice_pf_to_dev(vf->pf);
u8 *mac_addr = vc_ether_addr->addr;
- enum ice_status status;
- int ret = 0;
+ int ret;
/* device MAC already added */
if (ether_addr_equal(mac_addr, vf->dev_lan_addr.addr))
@@ -3825,18 +3792,17 @@ ice_vc_add_mac_addr(struct ice_vf *vf, struct ice_vsi *vsi,
return -EPERM;
}
- status = ice_fltr_add_mac(vsi, mac_addr, ICE_FWD_TO_VSI);
- if (status == ICE_ERR_ALREADY_EXISTS) {
+ ret = ice_fltr_add_mac(vsi, mac_addr, ICE_FWD_TO_VSI);
+ if (ret == -EEXIST) {
dev_dbg(dev, "MAC %pM already exists for VF %d\n", mac_addr,
vf->vf_id);
/* don't return since we might need to update
* the primary MAC in ice_vfhw_mac_add() below
*/
- ret = -EEXIST;
- } else if (status) {
- dev_err(dev, "Failed to add MAC %pM for VF %d\n, error %s\n",
- mac_addr, vf->vf_id, ice_stat_str(status));
- return -EIO;
+ } else if (ret) {
+ dev_err(dev, "Failed to add MAC %pM for VF %d\n, error %d\n",
+ mac_addr, vf->vf_id, ret);
+ return ret;
} else {
vf->num_mac++;
}
@@ -3913,20 +3879,20 @@ ice_vc_del_mac_addr(struct ice_vf *vf, struct ice_vsi *vsi,
{
struct device *dev = ice_pf_to_dev(vf->pf);
u8 *mac_addr = vc_ether_addr->addr;
- enum ice_status status;
+ int status;
if (!ice_can_vf_change_mac(vf) &&
ether_addr_equal(vf->dev_lan_addr.addr, mac_addr))
return 0;
status = ice_fltr_remove_mac(vsi, mac_addr, ICE_FWD_TO_VSI);
- if (status == ICE_ERR_DOES_NOT_EXIST) {
+ if (status == -ENOENT) {
dev_err(dev, "MAC %pM does not exist for VF %d\n", mac_addr,
vf->vf_id);
return -ENOENT;
} else if (status) {
- dev_err(dev, "Failed to delete MAC %pM for VF %d, error %s\n",
- mac_addr, vf->vf_id, ice_stat_str(status));
+ dev_err(dev, "Failed to delete MAC %pM for VF %d, error %d\n",
+ mac_addr, vf->vf_id, status);
return -EIO;
}
@@ -4533,6 +4499,7 @@ static int ice_vc_repr_add_mac(struct ice_vf *vf, u8 *msg)
for (i = 0; i < al->num_elements; i++) {
u8 *mac_addr = al->list[i].addr;
+ int result;
if (!is_unicast_ether_addr(mac_addr) ||
ether_addr_equal(mac_addr, vf->hw_lan_addr.addr))
@@ -4544,6 +4511,13 @@ static int ice_vc_repr_add_mac(struct ice_vf *vf, u8 *msg)
goto handle_mac_exit;
}
+ result = ice_eswitch_add_vf_mac_rule(pf, vf, mac_addr);
+ if (result) {
+ dev_err(ice_pf_to_dev(pf), "Failed to add MAC %pM for VF %d\n, error %d\n",
+ mac_addr, vf->vf_id, result);
+ goto handle_mac_exit;
+ }
+
ice_vfhw_mac_add(vf, &al->list[i]);
vf->num_mac++;
break;
@@ -5289,9 +5263,9 @@ ice_is_malicious_vf(struct ice_pf *pf, struct ice_rq_event_info *event,
s16 vf_id = le16_to_cpu(event->desc.retval);
struct device *dev = ice_pf_to_dev(pf);
struct ice_mbx_data mbxdata;
- enum ice_status status;
bool malvf = false;
struct ice_vf *vf;
+ int status;
if (ice_validate_vf_id(pf, vf_id))
return false;
diff --git a/drivers/net/ethernet/intel/ice/ice_virtchnl_pf.h b/drivers/net/ethernet/intel/ice/ice_virtchnl_pf.h
index 7e28ecb..752487a 100644
--- a/drivers/net/ethernet/intel/ice/ice_virtchnl_pf.h
+++ b/drivers/net/ethernet/intel/ice/ice_virtchnl_pf.h
@@ -203,6 +203,8 @@ void
ice_vf_lan_overflow_event(struct ice_pf *pf, struct ice_rq_event_info *event);
void ice_print_vfs_mdd_events(struct ice_pf *pf);
void ice_print_vf_rx_mdd_event(struct ice_vf *vf);
+bool
+ice_vc_validate_pattern(struct ice_vf *vf, struct virtchnl_proto_hdrs *proto);
struct ice_vsi *ice_vf_ctrl_vsi_setup(struct ice_vf *vf);
int
ice_vc_send_msg_to_vf(struct ice_vf *vf, u32 v_opcode,
diff --git a/drivers/net/ethernet/intel/ice/ice_xsk.c b/drivers/net/ethernet/intel/ice/ice_xsk.c
index c895351..2388837 100644
--- a/drivers/net/ethernet/intel/ice/ice_xsk.c
+++ b/drivers/net/ethernet/intel/ice/ice_xsk.c
@@ -481,7 +481,7 @@ ice_run_xdp_zc(struct ice_rx_ring *rx_ring, struct xdp_buff *xdp,
goto out_failure;
break;
default:
- bpf_warn_invalid_xdp_action(act);
+ bpf_warn_invalid_xdp_action(rx_ring->netdev, xdp_prog, act);
fallthrough;
case XDP_ABORTED:
out_failure:
diff --git a/drivers/net/ethernet/intel/igb/e1000_i210.c b/drivers/net/ethernet/intel/igb/e1000_i210.c
index 9265901..b9b9d35 100644
--- a/drivers/net/ethernet/intel/igb/e1000_i210.c
+++ b/drivers/net/ethernet/intel/igb/e1000_i210.c
@@ -792,7 +792,6 @@ s32 igb_write_xmdio_reg(struct e1000_hw *hw, u16 addr, u8 dev_addr, u16 data)
**/
s32 igb_init_nvm_params_i210(struct e1000_hw *hw)
{
- s32 ret_val = 0;
struct e1000_nvm_info *nvm = &hw->nvm;
nvm->ops.acquire = igb_acquire_nvm_i210;
@@ -813,7 +812,7 @@ s32 igb_init_nvm_params_i210(struct e1000_hw *hw)
nvm->ops.validate = NULL;
nvm->ops.update = NULL;
}
- return ret_val;
+ return 0;
}
/**
diff --git a/drivers/net/ethernet/intel/igb/igb_ethtool.c b/drivers/net/ethernet/intel/igb/igb_ethtool.c
index fb10293..51a2dca 100644
--- a/drivers/net/ethernet/intel/igb/igb_ethtool.c
+++ b/drivers/net/ethernet/intel/igb/igb_ethtool.c
@@ -864,7 +864,9 @@ static void igb_get_drvinfo(struct net_device *netdev,
}
static void igb_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct igb_adapter *adapter = netdev_priv(netdev);
@@ -875,7 +877,9 @@ static void igb_get_ringparam(struct net_device *netdev,
}
static int igb_set_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct igb_adapter *adapter = netdev_priv(netdev);
struct igb_ring *temp_ring;
diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
index 446894d..38ba920 100644
--- a/drivers/net/ethernet/intel/igb/igb_main.c
+++ b/drivers/net/ethernet/intel/igb/igb_main.c
@@ -2927,7 +2927,7 @@ static int igb_xdp_xmit_back(struct igb_adapter *adapter, struct xdp_buff *xdp)
nq = txring_txq(tx_ring);
__netif_tx_lock(nq, cpu);
/* Avoid transmit queue timeout since we share it with the slow path */
- nq->trans_start = jiffies;
+ txq_trans_cond_update(nq);
ret = igb_xmit_xdp_ring(adapter, tx_ring, xdpf);
__netif_tx_unlock(nq);
@@ -2961,7 +2961,7 @@ static int igb_xdp_xmit(struct net_device *dev, int n,
__netif_tx_lock(nq, cpu);
/* Avoid transmit queue timeout since we share it with the slow path */
- nq->trans_start = jiffies;
+ txq_trans_cond_update(nq);
for (i = 0; i < n; i++) {
struct xdp_frame *xdpf = frames[i];
@@ -6739,12 +6739,119 @@ void igb_update_stats(struct igb_adapter *adapter)
}
}
-static void igb_tsync_interrupt(struct igb_adapter *adapter)
+static void igb_perout(struct igb_adapter *adapter, int tsintr_tt)
{
+ int pin = ptp_find_pin(adapter->ptp_clock, PTP_PF_PEROUT, tsintr_tt);
+ struct e1000_hw *hw = &adapter->hw;
+ struct timespec64 ts;
+ u32 tsauxc;
+
+ if (pin < 0 || pin >= IGB_N_PEROUT)
+ return;
+
+ spin_lock(&adapter->tmreg_lock);
+
+ if (hw->mac.type == e1000_82580 ||
+ hw->mac.type == e1000_i354 ||
+ hw->mac.type == e1000_i350) {
+ s64 ns = timespec64_to_ns(&adapter->perout[pin].period);
+ u32 systiml, systimh, level_mask, level, rem;
+ u64 systim, now;
+
+ /* read systim registers in sequence */
+ rd32(E1000_SYSTIMR);
+ systiml = rd32(E1000_SYSTIML);
+ systimh = rd32(E1000_SYSTIMH);
+ systim = (((u64)(systimh & 0xFF)) << 32) | ((u64)systiml);
+ now = timecounter_cyc2time(&adapter->tc, systim);
+
+ if (pin < 2) {
+ level_mask = (tsintr_tt == 1) ? 0x80000 : 0x40000;
+ level = (rd32(E1000_CTRL) & level_mask) ? 1 : 0;
+ } else {
+ level_mask = (tsintr_tt == 1) ? 0x80 : 0x40;
+ level = (rd32(E1000_CTRL_EXT) & level_mask) ? 1 : 0;
+ }
+
+ div_u64_rem(now, ns, &rem);
+ systim = systim + (ns - rem);
+
+ /* synchronize pin level with rising/falling edges */
+ div_u64_rem(now, ns << 1, &rem);
+ if (rem < ns) {
+ /* first half of period */
+ if (level == 0) {
+ /* output is already low, skip this period */
+ systim += ns;
+ pr_notice("igb: periodic output on %s missed falling edge\n",
+ adapter->sdp_config[pin].name);
+ }
+ } else {
+ /* second half of period */
+ if (level == 1) {
+ /* output is already high, skip this period */
+ systim += ns;
+ pr_notice("igb: periodic output on %s missed rising edge\n",
+ adapter->sdp_config[pin].name);
+ }
+ }
+
+ /* for this chip family tv_sec is the upper part of the binary value,
+ * so not seconds
+ */
+ ts.tv_nsec = (u32)systim;
+ ts.tv_sec = ((u32)(systim >> 32)) & 0xFF;
+ } else {
+ ts = timespec64_add(adapter->perout[pin].start,
+ adapter->perout[pin].period);
+ }
+
+ /* u32 conversion of tv_sec is safe until y2106 */
+ wr32((tsintr_tt == 1) ? E1000_TRGTTIML1 : E1000_TRGTTIML0, ts.tv_nsec);
+ wr32((tsintr_tt == 1) ? E1000_TRGTTIMH1 : E1000_TRGTTIMH0, (u32)ts.tv_sec);
+ tsauxc = rd32(E1000_TSAUXC);
+ tsauxc |= TSAUXC_EN_TT0;
+ wr32(E1000_TSAUXC, tsauxc);
+ adapter->perout[pin].start = ts;
+
+ spin_unlock(&adapter->tmreg_lock);
+}
+
+static void igb_extts(struct igb_adapter *adapter, int tsintr_tt)
+{
+ int pin = ptp_find_pin(adapter->ptp_clock, PTP_PF_EXTTS, tsintr_tt);
+ int auxstmpl = (tsintr_tt == 1) ? E1000_AUXSTMPL1 : E1000_AUXSTMPL0;
+ int auxstmph = (tsintr_tt == 1) ? E1000_AUXSTMPH1 : E1000_AUXSTMPH0;
struct e1000_hw *hw = &adapter->hw;
struct ptp_clock_event event;
struct timespec64 ts;
- u32 ack = 0, tsauxc, sec, nsec, tsicr = rd32(E1000_TSICR);
+
+ if (pin < 0 || pin >= IGB_N_EXTTS)
+ return;
+
+ if (hw->mac.type == e1000_82580 ||
+ hw->mac.type == e1000_i354 ||
+ hw->mac.type == e1000_i350) {
+ s64 ns = rd32(auxstmpl);
+
+ ns += ((s64)(rd32(auxstmph) & 0xFF)) << 32;
+ ts = ns_to_timespec64(ns);
+ } else {
+ ts.tv_nsec = rd32(auxstmpl);
+ ts.tv_sec = rd32(auxstmph);
+ }
+
+ event.type = PTP_CLOCK_EXTTS;
+ event.index = tsintr_tt;
+ event.timestamp = ts.tv_sec * 1000000000ULL + ts.tv_nsec;
+ ptp_clock_event(adapter->ptp_clock, &event);
+}
+
+static void igb_tsync_interrupt(struct igb_adapter *adapter)
+{
+ struct e1000_hw *hw = &adapter->hw;
+ u32 ack = 0, tsicr = rd32(E1000_TSICR);
+ struct ptp_clock_event event;
if (tsicr & TSINTR_SYS_WRAP) {
event.type = PTP_CLOCK_PPS;
@@ -6760,51 +6867,22 @@ static void igb_tsync_interrupt(struct igb_adapter *adapter)
}
if (tsicr & TSINTR_TT0) {
- spin_lock(&adapter->tmreg_lock);
- ts = timespec64_add(adapter->perout[0].start,
- adapter->perout[0].period);
- /* u32 conversion of tv_sec is safe until y2106 */
- wr32(E1000_TRGTTIML0, ts.tv_nsec);
- wr32(E1000_TRGTTIMH0, (u32)ts.tv_sec);
- tsauxc = rd32(E1000_TSAUXC);
- tsauxc |= TSAUXC_EN_TT0;
- wr32(E1000_TSAUXC, tsauxc);
- adapter->perout[0].start = ts;
- spin_unlock(&adapter->tmreg_lock);
+ igb_perout(adapter, 0);
ack |= TSINTR_TT0;
}
if (tsicr & TSINTR_TT1) {
- spin_lock(&adapter->tmreg_lock);
- ts = timespec64_add(adapter->perout[1].start,
- adapter->perout[1].period);
- wr32(E1000_TRGTTIML1, ts.tv_nsec);
- wr32(E1000_TRGTTIMH1, (u32)ts.tv_sec);
- tsauxc = rd32(E1000_TSAUXC);
- tsauxc |= TSAUXC_EN_TT1;
- wr32(E1000_TSAUXC, tsauxc);
- adapter->perout[1].start = ts;
- spin_unlock(&adapter->tmreg_lock);
+ igb_perout(adapter, 1);
ack |= TSINTR_TT1;
}
if (tsicr & TSINTR_AUTT0) {
- nsec = rd32(E1000_AUXSTMPL0);
- sec = rd32(E1000_AUXSTMPH0);
- event.type = PTP_CLOCK_EXTTS;
- event.index = 0;
- event.timestamp = sec * 1000000000ULL + nsec;
- ptp_clock_event(adapter->ptp_clock, &event);
+ igb_extts(adapter, 0);
ack |= TSINTR_AUTT0;
}
if (tsicr & TSINTR_AUTT1) {
- nsec = rd32(E1000_AUXSTMPL1);
- sec = rd32(E1000_AUXSTMPH1);
- event.type = PTP_CLOCK_EXTTS;
- event.index = 1;
- event.timestamp = sec * 1000000000ULL + nsec;
- ptp_clock_event(adapter->ptp_clock, &event);
+ igb_extts(adapter, 1);
ack |= TSINTR_AUTT1;
}
@@ -8367,7 +8445,7 @@ static struct sk_buff *igb_build_skb(struct igb_ring *rx_ring,
net_prefetch(xdp->data_meta);
/* build an skb around the page buffer */
- skb = build_skb(xdp->data_hard_start, truesize);
+ skb = napi_build_skb(xdp->data_hard_start, truesize);
if (unlikely(!skb))
return NULL;
@@ -8422,7 +8500,7 @@ static struct sk_buff *igb_run_xdp(struct igb_adapter *adapter,
result = IGB_XDP_REDIR;
break;
default:
- bpf_warn_invalid_xdp_action(act);
+ bpf_warn_invalid_xdp_action(adapter->netdev, xdp_prog, act);
fallthrough;
case XDP_ABORTED:
out_failure:
diff --git a/drivers/net/ethernet/intel/igb/igb_ptp.c b/drivers/net/ethernet/intel/igb/igb_ptp.c
index 0011b15..6580fcd 100644
--- a/drivers/net/ethernet/intel/igb/igb_ptp.c
+++ b/drivers/net/ethernet/intel/igb/igb_ptp.c
@@ -69,6 +69,7 @@
#define IGB_NBITS_82580 40
static void igb_ptp_tx_hwtstamp(struct igb_adapter *adapter);
+static void igb_ptp_sdp_init(struct igb_adapter *adapter);
/* SYSTIM read access for the 82576 */
static u64 igb_ptp_read_82576(const struct cyclecounter *cc)
@@ -507,6 +508,158 @@ static void igb_pin_perout(struct igb_adapter *igb, int chan, int pin, int freq)
wr32(E1000_CTRL_EXT, ctrl_ext);
}
+static int igb_ptp_feature_enable_82580(struct ptp_clock_info *ptp,
+ struct ptp_clock_request *rq, int on)
+{
+ struct igb_adapter *igb =
+ container_of(ptp, struct igb_adapter, ptp_caps);
+ u32 tsauxc, tsim, tsauxc_mask, tsim_mask, trgttiml, trgttimh, systiml,
+ systimh, level_mask, level, rem;
+ struct e1000_hw *hw = &igb->hw;
+ struct timespec64 ts, start;
+ unsigned long flags;
+ u64 systim, now;
+ int pin = -1;
+ s64 ns;
+
+ switch (rq->type) {
+ case PTP_CLK_REQ_EXTTS:
+ /* Reject requests with unsupported flags */
+ if (rq->extts.flags & ~(PTP_ENABLE_FEATURE |
+ PTP_RISING_EDGE |
+ PTP_FALLING_EDGE |
+ PTP_STRICT_FLAGS))
+ return -EOPNOTSUPP;
+
+ if (on) {
+ pin = ptp_find_pin(igb->ptp_clock, PTP_PF_EXTTS,
+ rq->extts.index);
+ if (pin < 0)
+ return -EBUSY;
+ }
+ if (rq->extts.index == 1) {
+ tsauxc_mask = TSAUXC_EN_TS1;
+ tsim_mask = TSINTR_AUTT1;
+ } else {
+ tsauxc_mask = TSAUXC_EN_TS0;
+ tsim_mask = TSINTR_AUTT0;
+ }
+ spin_lock_irqsave(&igb->tmreg_lock, flags);
+ tsauxc = rd32(E1000_TSAUXC);
+ tsim = rd32(E1000_TSIM);
+ if (on) {
+ igb_pin_extts(igb, rq->extts.index, pin);
+ tsauxc |= tsauxc_mask;
+ tsim |= tsim_mask;
+ } else {
+ tsauxc &= ~tsauxc_mask;
+ tsim &= ~tsim_mask;
+ }
+ wr32(E1000_TSAUXC, tsauxc);
+ wr32(E1000_TSIM, tsim);
+ spin_unlock_irqrestore(&igb->tmreg_lock, flags);
+ return 0;
+
+ case PTP_CLK_REQ_PEROUT:
+ /* Reject requests with unsupported flags */
+ if (rq->perout.flags)
+ return -EOPNOTSUPP;
+
+ if (on) {
+ pin = ptp_find_pin(igb->ptp_clock, PTP_PF_PEROUT,
+ rq->perout.index);
+ if (pin < 0)
+ return -EBUSY;
+ }
+ ts.tv_sec = rq->perout.period.sec;
+ ts.tv_nsec = rq->perout.period.nsec;
+ ns = timespec64_to_ns(&ts);
+ ns = ns >> 1;
+ if (on && ns < 8LL)
+ return -EINVAL;
+ ts = ns_to_timespec64(ns);
+ if (rq->perout.index == 1) {
+ tsauxc_mask = TSAUXC_EN_TT1;
+ tsim_mask = TSINTR_TT1;
+ trgttiml = E1000_TRGTTIML1;
+ trgttimh = E1000_TRGTTIMH1;
+ } else {
+ tsauxc_mask = TSAUXC_EN_TT0;
+ tsim_mask = TSINTR_TT0;
+ trgttiml = E1000_TRGTTIML0;
+ trgttimh = E1000_TRGTTIMH0;
+ }
+ spin_lock_irqsave(&igb->tmreg_lock, flags);
+ tsauxc = rd32(E1000_TSAUXC);
+ tsim = rd32(E1000_TSIM);
+ if (rq->perout.index == 1) {
+ tsauxc &= ~(TSAUXC_EN_TT1 | TSAUXC_EN_CLK1 | TSAUXC_ST1);
+ tsim &= ~TSINTR_TT1;
+ } else {
+ tsauxc &= ~(TSAUXC_EN_TT0 | TSAUXC_EN_CLK0 | TSAUXC_ST0);
+ tsim &= ~TSINTR_TT0;
+ }
+ if (on) {
+ int i = rq->perout.index;
+
+ /* read systim registers in sequence */
+ rd32(E1000_SYSTIMR);
+ systiml = rd32(E1000_SYSTIML);
+ systimh = rd32(E1000_SYSTIMH);
+ systim = (((u64)(systimh & 0xFF)) << 32) | ((u64)systiml);
+ now = timecounter_cyc2time(&igb->tc, systim);
+
+ if (pin < 2) {
+ level_mask = (i == 1) ? 0x80000 : 0x40000;
+ level = (rd32(E1000_CTRL) & level_mask) ? 1 : 0;
+ } else {
+ level_mask = (i == 1) ? 0x80 : 0x40;
+ level = (rd32(E1000_CTRL_EXT) & level_mask) ? 1 : 0;
+ }
+
+ div_u64_rem(now, ns, &rem);
+ systim = systim + (ns - rem);
+
+ /* synchronize pin level with rising/falling edges */
+ div_u64_rem(now, ns << 1, &rem);
+ if (rem < ns) {
+ /* first half of period */
+ if (level == 0) {
+ /* output is already low, skip this period */
+ systim += ns;
+ }
+ } else {
+ /* second half of period */
+ if (level == 1) {
+ /* output is already high, skip this period */
+ systim += ns;
+ }
+ }
+
+ start = ns_to_timespec64(systim + (ns - rem));
+ igb_pin_perout(igb, i, pin, 0);
+ igb->perout[i].start.tv_sec = start.tv_sec;
+ igb->perout[i].start.tv_nsec = start.tv_nsec;
+ igb->perout[i].period.tv_sec = ts.tv_sec;
+ igb->perout[i].period.tv_nsec = ts.tv_nsec;
+
+ wr32(trgttiml, (u32)systim);
+ wr32(trgttimh, ((u32)(systim >> 32)) & 0xFF);
+ tsauxc |= tsauxc_mask;
+ tsim |= tsim_mask;
+ }
+ wr32(E1000_TSAUXC, tsauxc);
+ wr32(E1000_TSIM, tsim);
+ spin_unlock_irqrestore(&igb->tmreg_lock, flags);
+ return 0;
+
+ case PTP_CLK_REQ_PPS:
+ return -EOPNOTSUPP;
+ }
+
+ return -EOPNOTSUPP;
+}
+
static int igb_ptp_feature_enable_i210(struct ptp_clock_info *ptp,
struct ptp_clock_request *rq, int on)
{
@@ -1015,10 +1168,6 @@ static int igb_ptp_set_timestamp_mode(struct igb_adapter *adapter,
bool is_l2 = false;
u32 regval;
- /* reserved for future extensions */
- if (config->flags)
- return -EINVAL;
-
switch (config->tx_type) {
case HWTSTAMP_TX_OFF:
tsync_tx_ctl = 0;
@@ -1192,7 +1341,6 @@ void igb_ptp_init(struct igb_adapter *adapter)
{
struct e1000_hw *hw = &adapter->hw;
struct net_device *netdev = adapter->netdev;
- int i;
switch (hw->mac.type) {
case e1000_82576:
@@ -1215,16 +1363,21 @@ void igb_ptp_init(struct igb_adapter *adapter)
case e1000_82580:
case e1000_i354:
case e1000_i350:
+ igb_ptp_sdp_init(adapter);
snprintf(adapter->ptp_caps.name, 16, "%pm", netdev->dev_addr);
adapter->ptp_caps.owner = THIS_MODULE;
adapter->ptp_caps.max_adj = 62499999;
- adapter->ptp_caps.n_ext_ts = 0;
+ adapter->ptp_caps.n_ext_ts = IGB_N_EXTTS;
+ adapter->ptp_caps.n_per_out = IGB_N_PEROUT;
+ adapter->ptp_caps.n_pins = IGB_N_SDP;
adapter->ptp_caps.pps = 0;
+ adapter->ptp_caps.pin_config = adapter->sdp_config;
adapter->ptp_caps.adjfine = igb_ptp_adjfine_82580;
adapter->ptp_caps.adjtime = igb_ptp_adjtime_82576;
adapter->ptp_caps.gettimex64 = igb_ptp_gettimex_82580;
adapter->ptp_caps.settime64 = igb_ptp_settime_82576;
- adapter->ptp_caps.enable = igb_ptp_feature_enable;
+ adapter->ptp_caps.enable = igb_ptp_feature_enable_82580;
+ adapter->ptp_caps.verify = igb_ptp_verify_pin;
adapter->cc.read = igb_ptp_read_82580;
adapter->cc.mask = CYCLECOUNTER_MASK(IGB_NBITS_82580);
adapter->cc.mult = 1;
@@ -1233,13 +1386,7 @@ void igb_ptp_init(struct igb_adapter *adapter)
break;
case e1000_i210:
case e1000_i211:
- for (i = 0; i < IGB_N_SDP; i++) {
- struct ptp_pin_desc *ppd = &adapter->sdp_config[i];
-
- snprintf(ppd->name, sizeof(ppd->name), "SDP%d", i);
- ppd->index = i;
- ppd->func = PTP_PF_NONE;
- }
+ igb_ptp_sdp_init(adapter);
snprintf(adapter->ptp_caps.name, 16, "%pm", netdev->dev_addr);
adapter->ptp_caps.owner = THIS_MODULE;
adapter->ptp_caps.max_adj = 62499999;
@@ -1285,6 +1432,23 @@ void igb_ptp_init(struct igb_adapter *adapter)
}
/**
+ * igb_ptp_sdp_init - utility function which inits the SDP config structs
+ * @adapter: Board private structure.
+ **/
+void igb_ptp_sdp_init(struct igb_adapter *adapter)
+{
+ int i;
+
+ for (i = 0; i < IGB_N_SDP; i++) {
+ struct ptp_pin_desc *ppd = &adapter->sdp_config[i];
+
+ snprintf(ppd->name, sizeof(ppd->name), "SDP%d", i);
+ ppd->index = i;
+ ppd->func = PTP_PF_NONE;
+ }
+}
+
+/**
* igb_ptp_suspend - Disable PTP work items and prepare for suspend
* @adapter: Board private structure
*
diff --git a/drivers/net/ethernet/intel/igbvf/ethtool.c b/drivers/net/ethernet/intel/igbvf/ethtool.c
index 06e5bd6..9d4322b 100644
--- a/drivers/net/ethernet/intel/igbvf/ethtool.c
+++ b/drivers/net/ethernet/intel/igbvf/ethtool.c
@@ -175,7 +175,9 @@ static void igbvf_get_drvinfo(struct net_device *netdev,
}
static void igbvf_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct igbvf_adapter *adapter = netdev_priv(netdev);
struct igbvf_ring *tx_ring = adapter->tx_ring;
@@ -188,7 +190,9 @@ static void igbvf_get_ringparam(struct net_device *netdev,
}
static int igbvf_set_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct igbvf_adapter *adapter = netdev_priv(netdev);
struct igbvf_ring *temp_ring;
diff --git a/drivers/net/ethernet/intel/igbvf/netdev.c b/drivers/net/ethernet/intel/igbvf/netdev.c
index 4d988da..b784072 100644
--- a/drivers/net/ethernet/intel/igbvf/netdev.c
+++ b/drivers/net/ethernet/intel/igbvf/netdev.c
@@ -1520,7 +1520,7 @@ static void igbvf_reset(struct igbvf_adapter *adapter)
/* Allow time for pending master requests to run */
if (mac->ops.reset_hw(hw))
- dev_warn(&adapter->pdev->dev, "PF still resetting\n");
+ dev_info(&adapter->pdev->dev, "PF still resetting\n");
mac->ops.init_hw(hw);
diff --git a/drivers/net/ethernet/intel/igc/igc_defines.h b/drivers/net/ethernet/intel/igc/igc_defines.h
index c7fe615..5c66b97 100644
--- a/drivers/net/ethernet/intel/igc/igc_defines.h
+++ b/drivers/net/ethernet/intel/igc/igc_defines.h
@@ -85,9 +85,6 @@
#define IGC_WUFC_EXT_FILTER_MASK GENMASK(31, 8)
-/* Physical Func Reset Done Indication */
-#define IGC_CTRL_EXT_LINK_MODE_MASK 0x00C00000
-
/* Loop limit on how long we wait for auto-negotiation to complete */
#define COPPER_LINK_UP_LIMIT 10
#define PHY_AUTO_NEG_LIMIT 45
@@ -584,7 +581,6 @@
#define IGC_GEN_POLL_TIMEOUT 1920
/* PHY Control Register */
-#define MII_CR_FULL_DUPLEX 0x0100 /* FDX =1, half duplex =0 */
#define MII_CR_RESTART_AUTO_NEG 0x0200 /* Restart auto negotiation */
#define MII_CR_POWER_DOWN 0x0800 /* Power down */
#define MII_CR_AUTO_NEG_EN 0x1000 /* Auto Neg Enable */
@@ -605,9 +601,6 @@
#define PHY_1000T_CTRL 0x09 /* 1000Base-T Control Reg */
#define PHY_1000T_STATUS 0x0A /* 1000Base-T Status Reg */
-/* Bit definitions for valid PHY IDs. I = Integrated E = External */
-#define I225_I_PHY_ID 0x67C9DC00
-
/* MDI Control */
#define IGC_MDIC_DATA_MASK 0x0000FFFF
#define IGC_MDIC_REG_MASK 0x001F0000
diff --git a/drivers/net/ethernet/intel/igc/igc_ethtool.c b/drivers/net/ethernet/intel/igc/igc_ethtool.c
index e0a76ac..8cc077b 100644
--- a/drivers/net/ethernet/intel/igc/igc_ethtool.c
+++ b/drivers/net/ethernet/intel/igc/igc_ethtool.c
@@ -567,8 +567,11 @@ static int igc_ethtool_set_eeprom(struct net_device *netdev,
return ret_val;
}
-static void igc_ethtool_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+static void
+igc_ethtool_get_ringparam(struct net_device *netdev,
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ering,
+ struct netlink_ext_ack *extack)
{
struct igc_adapter *adapter = netdev_priv(netdev);
@@ -578,8 +581,11 @@ static void igc_ethtool_get_ringparam(struct net_device *netdev,
ring->tx_pending = adapter->tx_ring_count;
}
-static int igc_ethtool_set_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+static int
+igc_ethtool_set_ringparam(struct net_device *netdev,
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ering,
+ struct netlink_ext_ack *extack)
{
struct igc_adapter *adapter = netdev_priv(netdev);
struct igc_ring *temp_ring;
diff --git a/drivers/net/ethernet/intel/igc/igc_hw.h b/drivers/net/ethernet/intel/igc/igc_hw.h
index 587db74..b1e72ec 100644
--- a/drivers/net/ethernet/intel/igc/igc_hw.h
+++ b/drivers/net/ethernet/intel/igc/igc_hw.h
@@ -55,7 +55,6 @@ enum igc_mac_type {
enum igc_phy_type {
igc_phy_unknown = 0,
- igc_phy_none,
igc_phy_i225,
};
@@ -68,8 +67,6 @@ enum igc_media_type {
enum igc_nvm_type {
igc_nvm_unknown = 0,
igc_nvm_eeprom_spi,
- igc_nvm_flash_hw,
- igc_nvm_invm,
};
struct igc_info {
diff --git a/drivers/net/ethernet/intel/igc/igc_i225.c b/drivers/net/ethernet/intel/igc/igc_i225.c
index b6807e1..66ea566 100644
--- a/drivers/net/ethernet/intel/igc/igc_i225.c
+++ b/drivers/net/ethernet/intel/igc/igc_i225.c
@@ -473,13 +473,11 @@ s32 igc_init_nvm_params_i225(struct igc_hw *hw)
/* NVM Function Pointers */
if (igc_get_flash_presence_i225(hw)) {
- hw->nvm.type = igc_nvm_flash_hw;
nvm->ops.read = igc_read_nvm_srrd_i225;
nvm->ops.write = igc_write_nvm_srwr_i225;
nvm->ops.validate = igc_validate_nvm_checksum_i225;
nvm->ops.update = igc_update_nvm_checksum_i225;
} else {
- hw->nvm.type = igc_nvm_invm;
nvm->ops.read = igc_read_nvm_eerd;
nvm->ops.write = NULL;
nvm->ops.validate = NULL;
diff --git a/drivers/net/ethernet/intel/igc/igc_main.c b/drivers/net/ethernet/intel/igc/igc_main.c
index 8e44828..2f17f36 100644
--- a/drivers/net/ethernet/intel/igc/igc_main.c
+++ b/drivers/net/ethernet/intel/igc/igc_main.c
@@ -1718,24 +1718,26 @@ static void igc_add_rx_frag(struct igc_ring *rx_ring,
static struct sk_buff *igc_build_skb(struct igc_ring *rx_ring,
struct igc_rx_buffer *rx_buffer,
- union igc_adv_rx_desc *rx_desc,
- unsigned int size)
+ struct xdp_buff *xdp)
{
- void *va = page_address(rx_buffer->page) + rx_buffer->page_offset;
+ unsigned int size = xdp->data_end - xdp->data;
unsigned int truesize = igc_get_rx_frame_truesize(rx_ring, size);
+ unsigned int metasize = xdp->data - xdp->data_meta;
struct sk_buff *skb;
/* prefetch first cache line of first page */
- net_prefetch(va);
+ net_prefetch(xdp->data_meta);
/* build an skb around the page buffer */
- skb = build_skb(va - IGC_SKB_PAD, truesize);
+ skb = napi_build_skb(xdp->data_hard_start, truesize);
if (unlikely(!skb))
return NULL;
/* update pointers within the skb to store the data */
- skb_reserve(skb, IGC_SKB_PAD);
+ skb_reserve(skb, xdp->data - xdp->data_hard_start);
__skb_put(skb, size);
+ if (metasize)
+ skb_metadata_set(skb, metasize);
igc_rx_buffer_flip(rx_buffer, truesize);
return skb;
@@ -1746,6 +1748,7 @@ static struct sk_buff *igc_construct_skb(struct igc_ring *rx_ring,
struct xdp_buff *xdp,
ktime_t timestamp)
{
+ unsigned int metasize = xdp->data - xdp->data_meta;
unsigned int size = xdp->data_end - xdp->data;
unsigned int truesize = igc_get_rx_frame_truesize(rx_ring, size);
void *va = xdp->data;
@@ -1753,10 +1756,11 @@ static struct sk_buff *igc_construct_skb(struct igc_ring *rx_ring,
struct sk_buff *skb;
/* prefetch first cache line of first page */
- net_prefetch(va);
+ net_prefetch(xdp->data_meta);
/* allocate a skb to store the frags */
- skb = napi_alloc_skb(&rx_ring->q_vector->napi, IGC_RX_HDR_LEN);
+ skb = napi_alloc_skb(&rx_ring->q_vector->napi,
+ IGC_RX_HDR_LEN + metasize);
if (unlikely(!skb))
return NULL;
@@ -1769,7 +1773,13 @@ static struct sk_buff *igc_construct_skb(struct igc_ring *rx_ring,
headlen = eth_get_headlen(skb->dev, va, IGC_RX_HDR_LEN);
/* align pull length to size of long to optimize memcpy performance */
- memcpy(__skb_put(skb, headlen), va, ALIGN(headlen, sizeof(long)));
+ memcpy(__skb_put(skb, headlen + metasize), xdp->data_meta,
+ ALIGN(headlen + metasize, sizeof(long)));
+
+ if (metasize) {
+ skb_metadata_set(skb, metasize);
+ __skb_pull(skb, metasize);
+ }
/* update all of the pointers */
size -= headlen;
@@ -2231,7 +2241,7 @@ static int __igc_xdp_run_prog(struct igc_adapter *adapter,
return IGC_XDP_REDIRECT;
break;
default:
- bpf_warn_invalid_xdp_action(act);
+ bpf_warn_invalid_xdp_action(adapter->netdev, prog, act);
fallthrough;
case XDP_ABORTED:
out_failure:
@@ -2354,7 +2364,8 @@ static int igc_clean_rx_irq(struct igc_q_vector *q_vector, const int budget)
if (!skb) {
xdp_init_buff(&xdp, truesize, &rx_ring->xdp_rxq);
xdp_prepare_buff(&xdp, pktbuf - igc_rx_offset(rx_ring),
- igc_rx_offset(rx_ring) + pkt_offset, size, false);
+ igc_rx_offset(rx_ring) + pkt_offset,
+ size, true);
skb = igc_xdp_run_prog(adapter, &xdp);
}
@@ -2378,7 +2389,7 @@ static int igc_clean_rx_irq(struct igc_q_vector *q_vector, const int budget)
} else if (skb)
igc_add_rx_frag(rx_ring, rx_buffer, skb, size);
else if (ring_uses_build_skb(rx_ring))
- skb = igc_build_skb(rx_ring, rx_buffer, rx_desc, size);
+ skb = igc_build_skb(rx_ring, rx_buffer, &xdp);
else
skb = igc_construct_skb(rx_ring, rx_buffer, &xdp,
timestamp);
@@ -2448,8 +2459,10 @@ static struct sk_buff *igc_construct_skb_zc(struct igc_ring *ring,
skb_reserve(skb, xdp->data_meta - xdp->data_hard_start);
memcpy(__skb_put(skb, totalsize), xdp->data_meta, totalsize);
- if (metasize)
+ if (metasize) {
skb_metadata_set(skb, metasize);
+ __skb_pull(skb, metasize);
+ }
return skb;
}
@@ -5467,6 +5480,9 @@ static irqreturn_t igc_intr_msi(int irq, void *data)
mod_timer(&adapter->watchdog_timer, jiffies + 1);
}
+ if (icr & IGC_ICR_TS)
+ igc_tsync_interrupt(adapter);
+
napi_schedule(&q_vector->napi);
return IRQ_HANDLED;
@@ -5510,6 +5526,9 @@ static irqreturn_t igc_intr(int irq, void *data)
mod_timer(&adapter->watchdog_timer, jiffies + 1);
}
+ if (icr & IGC_ICR_TS)
+ igc_tsync_interrupt(adapter);
+
napi_schedule(&q_vector->napi);
return IRQ_HANDLED;
diff --git a/drivers/net/ethernet/intel/igc/igc_ptp.c b/drivers/net/ethernet/intel/igc/igc_ptp.c
index 30568e3..0d6e321 100644
--- a/drivers/net/ethernet/intel/igc/igc_ptp.c
+++ b/drivers/net/ethernet/intel/igc/igc_ptp.c
@@ -560,10 +560,6 @@ static void igc_ptp_enable_tx_timestamp(struct igc_adapter *adapter)
static int igc_ptp_set_timestamp_mode(struct igc_adapter *adapter,
struct hwtstamp_config *config)
{
- /* reserved for future extensions */
- if (config->flags)
- return -EINVAL;
-
switch (config->tx_type) {
case HWTSTAMP_TX_OFF:
igc_ptp_disable_tx_timestamp(adapter);
@@ -768,7 +764,20 @@ int igc_ptp_get_ts_config(struct net_device *netdev, struct ifreq *ifr)
*/
static bool igc_is_crosststamp_supported(struct igc_adapter *adapter)
{
- return IS_ENABLED(CONFIG_X86_TSC) ? pcie_ptm_enabled(adapter->pdev) : false;
+ if (!IS_ENABLED(CONFIG_X86_TSC))
+ return false;
+
+ /* FIXME: it was noticed that enabling support for PCIe PTM in
+ * some i225-V models could cause lockups when bringing the
+ * interface up/down. There should be no downsides to
+ * disabling crosstimestamping support for i225-V, as it
+ * doesn't have any PTP support. That way we gain some time
+ * while root causing the issue.
+ */
+ if (adapter->pdev->device == IGC_DEV_ID_I225_V)
+ return false;
+
+ return pcie_ptm_enabled(adapter->pdev);
}
static struct system_counterval_t igc_device_tstamp_to_system(u64 tstamp)
diff --git a/drivers/net/ethernet/intel/igc/igc_xdp.c b/drivers/net/ethernet/intel/igc/igc_xdp.c
index a8cf537..aeeb34e 100644
--- a/drivers/net/ethernet/intel/igc/igc_xdp.c
+++ b/drivers/net/ethernet/intel/igc/igc_xdp.c
@@ -1,6 +1,7 @@
// SPDX-License-Identifier: GPL-2.0
/* Copyright (c) 2020, Intel Corporation. */
+#include <linux/if_vlan.h>
#include <net/xdp_sock_drv.h>
#include "igc.h"
diff --git a/drivers/net/ethernet/intel/ixgb/ixgb_ethtool.c b/drivers/net/ethernet/intel/ixgb/ixgb_ethtool.c
index 582099a..46efcfa 100644
--- a/drivers/net/ethernet/intel/ixgb/ixgb_ethtool.c
+++ b/drivers/net/ethernet/intel/ixgb/ixgb_ethtool.c
@@ -464,7 +464,9 @@ ixgb_get_drvinfo(struct net_device *netdev,
static void
ixgb_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct ixgb_adapter *adapter = netdev_priv(netdev);
struct ixgb_desc_ring *txdr = &adapter->tx_ring;
@@ -478,7 +480,9 @@ ixgb_get_ringparam(struct net_device *netdev,
static int
ixgb_set_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct ixgb_adapter *adapter = netdev_priv(netdev);
struct ixgb_desc_ring *txdr = &adapter->tx_ring;
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c
index 8362822..f70967c 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c
@@ -1118,7 +1118,9 @@ static void ixgbe_get_drvinfo(struct net_device *netdev,
}
static void ixgbe_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct ixgbe_adapter *adapter = netdev_priv(netdev);
struct ixgbe_ring *tx_ring = adapter->tx_ring[0];
@@ -1131,7 +1133,9 @@ static void ixgbe_get_ringparam(struct net_device *netdev,
}
static int ixgbe_set_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct ixgbe_adapter *adapter = netdev_priv(netdev);
struct ixgbe_ring *temp_ring;
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
index 45e2ec4..c6ff656 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
@@ -2170,7 +2170,7 @@ static struct sk_buff *ixgbe_build_skb(struct ixgbe_ring *rx_ring,
net_prefetch(xdp->data_meta);
/* build an skb to around the page buffer */
- skb = build_skb(xdp->data_hard_start, truesize);
+ skb = napi_build_skb(xdp->data_hard_start, truesize);
if (unlikely(!skb))
return NULL;
@@ -2235,7 +2235,7 @@ static struct sk_buff *ixgbe_run_xdp(struct ixgbe_adapter *adapter,
result = IXGBE_XDP_REDIR;
break;
default:
- bpf_warn_invalid_xdp_action(act);
+ bpf_warn_invalid_xdp_action(rx_ring->netdev, xdp_prog, act);
fallthrough;
case XDP_ABORTED:
out_failure:
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_ptp.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_ptp.c
index 23ddfd7..336426a 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_ptp.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_ptp.c
@@ -992,10 +992,6 @@ static int ixgbe_ptp_set_timestamp_mode(struct ixgbe_adapter *adapter,
bool is_l2 = false;
u32 regval;
- /* reserved for future extensions */
- if (config->flags)
- return -EINVAL;
-
switch (config->tx_type) {
case HWTSTAMP_TX_OFF:
tsync_tx_ctl = 0;
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_txrx_common.h b/drivers/net/ethernet/intel/ixgbe/ixgbe_txrx_common.h
index a82533f..bba3fea 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_txrx_common.h
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_txrx_common.h
@@ -35,8 +35,6 @@ int ixgbe_xsk_pool_setup(struct ixgbe_adapter *adapter,
struct xsk_buff_pool *pool,
u16 qid);
-void ixgbe_zca_free(struct zero_copy_allocator *alloc, unsigned long handle);
-
bool ixgbe_alloc_rx_buffers_zc(struct ixgbe_ring *rx_ring, u16 cleaned_count);
int ixgbe_clean_rx_irq_zc(struct ixgbe_q_vector *q_vector,
struct ixgbe_ring *rx_ring,
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c
index db2bc58..b3fd8e5 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c
@@ -131,7 +131,7 @@ static int ixgbe_run_xdp_zc(struct ixgbe_adapter *adapter,
goto out_failure;
break;
default:
- bpf_warn_invalid_xdp_action(act);
+ bpf_warn_invalid_xdp_action(rx_ring->netdev, xdp_prog, act);
fallthrough;
case XDP_ABORTED:
out_failure:
diff --git a/drivers/net/ethernet/intel/ixgbevf/defines.h b/drivers/net/ethernet/intel/ixgbevf/defines.h
index 6bace74..5f08779 100644
--- a/drivers/net/ethernet/intel/ixgbevf/defines.h
+++ b/drivers/net/ethernet/intel/ixgbevf/defines.h
@@ -281,6 +281,10 @@ struct ixgbe_adv_tx_context_desc {
#define IXGBE_ERR_INVALID_MAC_ADDR -1
#define IXGBE_ERR_RESET_FAILED -2
#define IXGBE_ERR_INVALID_ARGUMENT -3
+#define IXGBE_ERR_CONFIG -4
+#define IXGBE_ERR_MBX -5
+#define IXGBE_ERR_TIMEOUT -6
+#define IXGBE_ERR_PARAM -7
/* Transmit Config masks */
#define IXGBE_TXDCTL_ENABLE 0x02000000 /* Ena specific Tx Queue */
diff --git a/drivers/net/ethernet/intel/ixgbevf/ethtool.c b/drivers/net/ethernet/intel/ixgbevf/ethtool.c
index 8380f90..3b41f83 100644
--- a/drivers/net/ethernet/intel/ixgbevf/ethtool.c
+++ b/drivers/net/ethernet/intel/ixgbevf/ethtool.c
@@ -225,7 +225,9 @@ static void ixgbevf_get_drvinfo(struct net_device *netdev,
}
static void ixgbevf_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct ixgbevf_adapter *adapter = netdev_priv(netdev);
@@ -236,7 +238,9 @@ static void ixgbevf_get_ringparam(struct net_device *netdev,
}
static int ixgbevf_set_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct ixgbevf_adapter *adapter = netdev_priv(netdev);
struct ixgbevf_ring *tx_ring = NULL, *rx_ring = NULL;
diff --git a/drivers/net/ethernet/intel/ixgbevf/ipsec.c b/drivers/net/ethernet/intel/ixgbevf/ipsec.c
index e3e4676..e763cee 100644
--- a/drivers/net/ethernet/intel/ixgbevf/ipsec.c
+++ b/drivers/net/ethernet/intel/ixgbevf/ipsec.c
@@ -40,16 +40,16 @@ static int ixgbevf_ipsec_set_pf_sa(struct ixgbevf_adapter *adapter,
spin_lock_bh(&adapter->mbx_lock);
- ret = hw->mbx.ops.write_posted(hw, msgbuf, IXGBE_VFMAILBOX_SIZE);
+ ret = ixgbevf_write_mbx(hw, msgbuf, IXGBE_VFMAILBOX_SIZE);
if (ret)
goto out;
- ret = hw->mbx.ops.read_posted(hw, msgbuf, 2);
+ ret = ixgbevf_poll_mbx(hw, msgbuf, 2);
if (ret)
goto out;
ret = (int)msgbuf[1];
- if (msgbuf[0] & IXGBE_VT_MSGTYPE_NACK && ret >= 0)
+ if (msgbuf[0] & IXGBE_VT_MSGTYPE_FAILURE && ret >= 0)
ret = -1;
out:
@@ -77,11 +77,11 @@ static int ixgbevf_ipsec_del_pf_sa(struct ixgbevf_adapter *adapter, int pfsa)
spin_lock_bh(&adapter->mbx_lock);
- err = hw->mbx.ops.write_posted(hw, msgbuf, 2);
+ err = ixgbevf_write_mbx(hw, msgbuf, 2);
if (err)
goto out;
- err = hw->mbx.ops.read_posted(hw, msgbuf, 2);
+ err = ixgbevf_poll_mbx(hw, msgbuf, 2);
if (err)
goto out;
@@ -623,6 +623,7 @@ void ixgbevf_init_ipsec_offload(struct ixgbevf_adapter *adapter)
switch (adapter->hw.api_version) {
case ixgbe_mbox_api_14:
+ case ixgbe_mbox_api_15:
break;
default:
return;
diff --git a/drivers/net/ethernet/intel/ixgbevf/ixgbevf.h b/drivers/net/ethernet/intel/ixgbevf/ixgbevf.h
index a0e3257..e257390 100644
--- a/drivers/net/ethernet/intel/ixgbevf/ixgbevf.h
+++ b/drivers/net/ethernet/intel/ixgbevf/ixgbevf.h
@@ -430,6 +430,7 @@ extern const struct ixgbevf_info ixgbevf_X540_vf_info;
extern const struct ixgbevf_info ixgbevf_X550_vf_info;
extern const struct ixgbevf_info ixgbevf_X550EM_x_vf_info;
extern const struct ixgbe_mbx_operations ixgbevf_mbx_ops;
+extern const struct ixgbe_mbx_operations ixgbevf_mbx_ops_legacy;
extern const struct ixgbevf_info ixgbevf_x550em_a_vf_info;
extern const struct ixgbevf_info ixgbevf_82599_vf_hv_info;
@@ -491,4 +492,8 @@ void ixgbe_napi_del_all(struct ixgbevf_adapter *adapter);
#define hw_dbg(hw, format, arg...) \
netdev_dbg(ixgbevf_hw_to_netdev(hw), format, ## arg)
+
+s32 ixgbevf_poll_mbx(struct ixgbe_hw *hw, u32 *msg, u16 size);
+s32 ixgbevf_write_mbx(struct ixgbe_hw *hw, u32 *msg, u16 size);
+
#endif /* _IXGBEVF_H_ */
diff --git a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c
index d81811a..0015fcf 100644
--- a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c
+++ b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c
@@ -944,7 +944,7 @@ static struct sk_buff *ixgbevf_build_skb(struct ixgbevf_ring *rx_ring,
net_prefetch(xdp->data_meta);
/* build an skb around the page buffer */
- skb = build_skb(xdp->data_hard_start, truesize);
+ skb = napi_build_skb(xdp->data_hard_start, truesize);
if (unlikely(!skb))
return NULL;
@@ -1070,7 +1070,7 @@ static struct sk_buff *ixgbevf_run_xdp(struct ixgbevf_adapter *adapter,
goto out_failure;
break;
default:
- bpf_warn_invalid_xdp_action(act);
+ bpf_warn_invalid_xdp_action(rx_ring->netdev, xdp_prog, act);
fallthrough;
case XDP_ABORTED:
out_failure:
@@ -2266,6 +2266,7 @@ static void ixgbevf_negotiate_api(struct ixgbevf_adapter *adapter)
{
struct ixgbe_hw *hw = &adapter->hw;
static const int api[] = {
+ ixgbe_mbox_api_15,
ixgbe_mbox_api_14,
ixgbe_mbox_api_13,
ixgbe_mbox_api_12,
@@ -2284,6 +2285,12 @@ static void ixgbevf_negotiate_api(struct ixgbevf_adapter *adapter)
idx++;
}
+ if (hw->api_version >= ixgbe_mbox_api_15) {
+ hw->mbx.ops.init_params(hw);
+ memcpy(&hw->mbx.ops, &ixgbevf_mbx_ops,
+ sizeof(struct ixgbe_mbx_operations));
+ }
+
spin_unlock_bh(&adapter->mbx_lock);
}
@@ -2627,6 +2634,7 @@ static void ixgbevf_set_num_queues(struct ixgbevf_adapter *adapter)
case ixgbe_mbox_api_12:
case ixgbe_mbox_api_13:
case ixgbe_mbox_api_14:
+ case ixgbe_mbox_api_15:
if (adapter->xdp_prog &&
hw->mac.max_tx_queues == rss)
rss = rss > 3 ? 2 : 1;
@@ -4565,7 +4573,7 @@ static int ixgbevf_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
memcpy(&hw->mac.ops, ii->mac_ops, sizeof(hw->mac.ops));
hw->mac.type = ii->mac;
- memcpy(&hw->mbx.ops, &ixgbevf_mbx_ops,
+ memcpy(&hw->mbx.ops, &ixgbevf_mbx_ops_legacy,
sizeof(struct ixgbe_mbx_operations));
/* setup the private structure */
@@ -4625,6 +4633,7 @@ static int ixgbevf_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
case ixgbe_mbox_api_12:
case ixgbe_mbox_api_13:
case ixgbe_mbox_api_14:
+ case ixgbe_mbox_api_15:
netdev->max_mtu = IXGBE_MAX_JUMBO_FRAME_SIZE -
(ETH_HLEN + ETH_FCS_LEN);
break;
diff --git a/drivers/net/ethernet/intel/ixgbevf/mbx.c b/drivers/net/ethernet/intel/ixgbevf/mbx.c
index 6bc1953..a55dd97 100644
--- a/drivers/net/ethernet/intel/ixgbevf/mbx.c
+++ b/drivers/net/ethernet/intel/ixgbevf/mbx.c
@@ -15,16 +15,15 @@ static s32 ixgbevf_poll_for_msg(struct ixgbe_hw *hw)
struct ixgbe_mbx_info *mbx = &hw->mbx;
int countdown = mbx->timeout;
+ if (!countdown || !mbx->ops.check_for_msg)
+ return IXGBE_ERR_CONFIG;
+
while (countdown && mbx->ops.check_for_msg(hw)) {
countdown--;
udelay(mbx->udelay);
}
- /* if we failed, all future posted messages fail until reset */
- if (!countdown)
- mbx->timeout = 0;
-
- return countdown ? 0 : IXGBE_ERR_MBX;
+ return countdown ? 0 : IXGBE_ERR_TIMEOUT;
}
/**
@@ -38,87 +37,82 @@ static s32 ixgbevf_poll_for_ack(struct ixgbe_hw *hw)
struct ixgbe_mbx_info *mbx = &hw->mbx;
int countdown = mbx->timeout;
+ if (!countdown || !mbx->ops.check_for_ack)
+ return IXGBE_ERR_CONFIG;
+
while (countdown && mbx->ops.check_for_ack(hw)) {
countdown--;
udelay(mbx->udelay);
}
- /* if we failed, all future posted messages fail until reset */
- if (!countdown)
- mbx->timeout = 0;
-
- return countdown ? 0 : IXGBE_ERR_MBX;
+ return countdown ? 0 : IXGBE_ERR_TIMEOUT;
}
/**
- * ixgbevf_read_posted_mbx - Wait for message notification and receive message
- * @hw: pointer to the HW structure
- * @msg: The message buffer
- * @size: Length of buffer
+ * ixgbevf_read_mailbox_vf - read VF's mailbox register
+ * @hw: pointer to the HW structure
*
- * returns 0 if it successfully received a message notification and
- * copied it into the receive buffer.
+ * This function is used to read the mailbox register dedicated for VF without
+ * losing the read to clear status bits.
**/
-static s32 ixgbevf_read_posted_mbx(struct ixgbe_hw *hw, u32 *msg, u16 size)
+static u32 ixgbevf_read_mailbox_vf(struct ixgbe_hw *hw)
{
- struct ixgbe_mbx_info *mbx = &hw->mbx;
- s32 ret_val = IXGBE_ERR_MBX;
+ u32 vf_mailbox = IXGBE_READ_REG(hw, IXGBE_VFMAILBOX);
- if (!mbx->ops.read)
- goto out;
+ vf_mailbox |= hw->mbx.vf_mailbox;
+ hw->mbx.vf_mailbox |= vf_mailbox & IXGBE_VFMAILBOX_R2C_BITS;
- ret_val = ixgbevf_poll_for_msg(hw);
-
- /* if ack received read message, otherwise we timed out */
- if (!ret_val)
- ret_val = mbx->ops.read(hw, msg, size);
-out:
- return ret_val;
+ return vf_mailbox;
}
/**
- * ixgbevf_write_posted_mbx - Write a message to the mailbox, wait for ack
- * @hw: pointer to the HW structure
- * @msg: The message buffer
- * @size: Length of buffer
+ * ixgbevf_clear_msg_vf - clear PF status bit
+ * @hw: pointer to the HW structure
*
- * returns 0 if it successfully copied message into the buffer and
- * received an ack to that message within delay * timeout period
+ * This function is used to clear PFSTS bit in the VFMAILBOX register
**/
-static s32 ixgbevf_write_posted_mbx(struct ixgbe_hw *hw, u32 *msg, u16 size)
+static void ixgbevf_clear_msg_vf(struct ixgbe_hw *hw)
{
- struct ixgbe_mbx_info *mbx = &hw->mbx;
- s32 ret_val = IXGBE_ERR_MBX;
+ u32 vf_mailbox = ixgbevf_read_mailbox_vf(hw);
- /* exit if either we can't write or there isn't a defined timeout */
- if (!mbx->ops.write || !mbx->timeout)
- goto out;
-
- /* send msg */
- ret_val = mbx->ops.write(hw, msg, size);
-
- /* if msg sent wait until we receive an ack */
- if (!ret_val)
- ret_val = ixgbevf_poll_for_ack(hw);
-out:
- return ret_val;
+ if (vf_mailbox & IXGBE_VFMAILBOX_PFSTS) {
+ hw->mbx.stats.reqs++;
+ hw->mbx.vf_mailbox &= ~IXGBE_VFMAILBOX_PFSTS;
+ }
}
/**
- * ixgbevf_read_v2p_mailbox - read v2p mailbox
- * @hw: pointer to the HW structure
+ * ixgbevf_clear_ack_vf - clear PF ACK bit
+ * @hw: pointer to the HW structure
*
- * This function is used to read the v2p mailbox without losing the read to
- * clear status bits.
+ * This function is used to clear PFACK bit in the VFMAILBOX register
**/
-static u32 ixgbevf_read_v2p_mailbox(struct ixgbe_hw *hw)
+static void ixgbevf_clear_ack_vf(struct ixgbe_hw *hw)
{
- u32 v2p_mailbox = IXGBE_READ_REG(hw, IXGBE_VFMAILBOX);
+ u32 vf_mailbox = ixgbevf_read_mailbox_vf(hw);
- v2p_mailbox |= hw->mbx.v2p_mailbox;
- hw->mbx.v2p_mailbox |= v2p_mailbox & IXGBE_VFMAILBOX_R2C_BITS;
+ if (vf_mailbox & IXGBE_VFMAILBOX_PFACK) {
+ hw->mbx.stats.acks++;
+ hw->mbx.vf_mailbox &= ~IXGBE_VFMAILBOX_PFACK;
+ }
+}
- return v2p_mailbox;
+/**
+ * ixgbevf_clear_rst_vf - clear PF reset bit
+ * @hw: pointer to the HW structure
+ *
+ * This function is used to clear reset indication and reset done bit in
+ * VFMAILBOX register after reset the shared resources and the reset sequence.
+ **/
+static void ixgbevf_clear_rst_vf(struct ixgbe_hw *hw)
+{
+ u32 vf_mailbox = ixgbevf_read_mailbox_vf(hw);
+
+ if (vf_mailbox & (IXGBE_VFMAILBOX_RSTI | IXGBE_VFMAILBOX_RSTD)) {
+ hw->mbx.stats.rsts++;
+ hw->mbx.vf_mailbox &= ~(IXGBE_VFMAILBOX_RSTI |
+ IXGBE_VFMAILBOX_RSTD);
+ }
}
/**
@@ -131,14 +125,12 @@ static u32 ixgbevf_read_v2p_mailbox(struct ixgbe_hw *hw)
**/
static s32 ixgbevf_check_for_bit_vf(struct ixgbe_hw *hw, u32 mask)
{
- u32 v2p_mailbox = ixgbevf_read_v2p_mailbox(hw);
+ u32 vf_mailbox = ixgbevf_read_mailbox_vf(hw);
s32 ret_val = IXGBE_ERR_MBX;
- if (v2p_mailbox & mask)
+ if (vf_mailbox & mask)
ret_val = 0;
- hw->mbx.v2p_mailbox &= ~mask;
-
return ret_val;
}
@@ -172,6 +164,7 @@ static s32 ixgbevf_check_for_ack_vf(struct ixgbe_hw *hw)
if (!ixgbevf_check_for_bit_vf(hw, IXGBE_VFMAILBOX_PFACK)) {
ret_val = 0;
+ ixgbevf_clear_ack_vf(hw);
hw->mbx.stats.acks++;
}
@@ -191,6 +184,7 @@ static s32 ixgbevf_check_for_rst_vf(struct ixgbe_hw *hw)
if (!ixgbevf_check_for_bit_vf(hw, (IXGBE_VFMAILBOX_RSTD |
IXGBE_VFMAILBOX_RSTI))) {
ret_val = 0;
+ ixgbevf_clear_rst_vf(hw);
hw->mbx.stats.rsts++;
}
@@ -205,19 +199,59 @@ static s32 ixgbevf_check_for_rst_vf(struct ixgbe_hw *hw)
**/
static s32 ixgbevf_obtain_mbx_lock_vf(struct ixgbe_hw *hw)
{
- s32 ret_val = IXGBE_ERR_MBX;
+ struct ixgbe_mbx_info *mbx = &hw->mbx;
+ s32 ret_val = IXGBE_ERR_CONFIG;
+ int countdown = mbx->timeout;
+ u32 vf_mailbox;
- /* Take ownership of the buffer */
- IXGBE_WRITE_REG(hw, IXGBE_VFMAILBOX, IXGBE_VFMAILBOX_VFU);
+ if (!mbx->timeout)
+ return ret_val;
- /* reserve mailbox for VF use */
- if (ixgbevf_read_v2p_mailbox(hw) & IXGBE_VFMAILBOX_VFU)
- ret_val = 0;
+ while (countdown--) {
+ /* Reserve mailbox for VF use */
+ vf_mailbox = ixgbevf_read_mailbox_vf(hw);
+ vf_mailbox |= IXGBE_VFMAILBOX_VFU;
+ IXGBE_WRITE_REG(hw, IXGBE_VFMAILBOX, vf_mailbox);
+
+ /* Verify that VF is the owner of the lock */
+ if (ixgbevf_read_mailbox_vf(hw) & IXGBE_VFMAILBOX_VFU) {
+ ret_val = 0;
+ break;
+ }
+
+ /* Wait a bit before trying again */
+ udelay(mbx->udelay);
+ }
+
+ if (ret_val)
+ ret_val = IXGBE_ERR_TIMEOUT;
return ret_val;
}
/**
+ * ixgbevf_release_mbx_lock_vf - release mailbox lock
+ * @hw: pointer to the HW structure
+ **/
+static void ixgbevf_release_mbx_lock_vf(struct ixgbe_hw *hw)
+{
+ u32 vf_mailbox;
+
+ /* Return ownership of the buffer */
+ vf_mailbox = ixgbevf_read_mailbox_vf(hw);
+ vf_mailbox &= ~IXGBE_VFMAILBOX_VFU;
+ IXGBE_WRITE_REG(hw, IXGBE_VFMAILBOX, vf_mailbox);
+}
+
+/**
+ * ixgbevf_release_mbx_lock_vf_legacy - release mailbox lock
+ * @hw: pointer to the HW structure
+ **/
+static void ixgbevf_release_mbx_lock_vf_legacy(struct ixgbe_hw *__always_unused hw)
+{
+}
+
+/**
* ixgbevf_write_mbx_vf - Write a message to the mailbox
* @hw: pointer to the HW structure
* @msg: The message buffer
@@ -227,6 +261,50 @@ static s32 ixgbevf_obtain_mbx_lock_vf(struct ixgbe_hw *hw)
**/
static s32 ixgbevf_write_mbx_vf(struct ixgbe_hw *hw, u32 *msg, u16 size)
{
+ u32 vf_mailbox;
+ s32 ret_val;
+ u16 i;
+
+ /* lock the mailbox to prevent PF/VF race condition */
+ ret_val = ixgbevf_obtain_mbx_lock_vf(hw);
+ if (ret_val)
+ goto out_no_write;
+
+ /* flush msg and acks as we are overwriting the message buffer */
+ ixgbevf_clear_msg_vf(hw);
+ ixgbevf_clear_ack_vf(hw);
+
+ /* copy the caller specified message to the mailbox memory buffer */
+ for (i = 0; i < size; i++)
+ IXGBE_WRITE_REG_ARRAY(hw, IXGBE_VFMBMEM, i, msg[i]);
+
+ /* update stats */
+ hw->mbx.stats.msgs_tx++;
+
+ /* interrupt the PF to tell it a message has been sent */
+ vf_mailbox = ixgbevf_read_mailbox_vf(hw);
+ vf_mailbox |= IXGBE_VFMAILBOX_REQ;
+ IXGBE_WRITE_REG(hw, IXGBE_VFMAILBOX, vf_mailbox);
+
+ /* if msg sent wait until we receive an ack */
+ ret_val = ixgbevf_poll_for_ack(hw);
+
+out_no_write:
+ hw->mbx.ops.release(hw);
+
+ return ret_val;
+}
+
+/**
+ * ixgbevf_write_mbx_vf_legacy - Write a message to the mailbox
+ * @hw: pointer to the HW structure
+ * @msg: The message buffer
+ * @size: Length of buffer
+ *
+ * returns 0 if it successfully copied message into the buffer
+ **/
+static s32 ixgbevf_write_mbx_vf_legacy(struct ixgbe_hw *hw, u32 *msg, u16 size)
+{
s32 ret_val;
u16 i;
@@ -237,7 +315,9 @@ static s32 ixgbevf_write_mbx_vf(struct ixgbe_hw *hw, u32 *msg, u16 size)
/* flush msg and acks as we are overwriting the message buffer */
ixgbevf_check_for_msg_vf(hw);
+ ixgbevf_clear_msg_vf(hw);
ixgbevf_check_for_ack_vf(hw);
+ ixgbevf_clear_ack_vf(hw);
/* copy the caller specified message to the mailbox memory buffer */
for (i = 0; i < size; i++)
@@ -263,6 +343,42 @@ static s32 ixgbevf_write_mbx_vf(struct ixgbe_hw *hw, u32 *msg, u16 size)
**/
static s32 ixgbevf_read_mbx_vf(struct ixgbe_hw *hw, u32 *msg, u16 size)
{
+ u32 vf_mailbox;
+ s32 ret_val;
+ u16 i;
+
+ /* check if there is a message from PF */
+ ret_val = ixgbevf_check_for_msg_vf(hw);
+ if (ret_val)
+ return ret_val;
+
+ ixgbevf_clear_msg_vf(hw);
+
+ /* copy the message from the mailbox memory buffer */
+ for (i = 0; i < size; i++)
+ msg[i] = IXGBE_READ_REG_ARRAY(hw, IXGBE_VFMBMEM, i);
+
+ /* Acknowledge receipt */
+ vf_mailbox = ixgbevf_read_mailbox_vf(hw);
+ vf_mailbox |= IXGBE_VFMAILBOX_ACK;
+ IXGBE_WRITE_REG(hw, IXGBE_VFMAILBOX, vf_mailbox);
+
+ /* update stats */
+ hw->mbx.stats.msgs_rx++;
+
+ return ret_val;
+}
+
+/**
+ * ixgbevf_read_mbx_vf_legacy - Reads a message from the inbox intended for VF
+ * @hw: pointer to the HW structure
+ * @msg: The message buffer
+ * @size: Length of buffer
+ *
+ * returns 0 if it successfully read message from buffer
+ **/
+static s32 ixgbevf_read_mbx_vf_legacy(struct ixgbe_hw *hw, u32 *msg, u16 size)
+{
s32 ret_val = 0;
u16 i;
@@ -298,7 +414,7 @@ static s32 ixgbevf_init_mbx_params_vf(struct ixgbe_hw *hw)
/* start mailbox as timed out and let the reset_hw call set the timeout
* value to begin communications
*/
- mbx->timeout = 0;
+ mbx->timeout = IXGBE_VF_MBX_INIT_TIMEOUT;
mbx->udelay = IXGBE_VF_MBX_INIT_DELAY;
mbx->size = IXGBE_VFMAILBOX_SIZE;
@@ -312,12 +428,79 @@ static s32 ixgbevf_init_mbx_params_vf(struct ixgbe_hw *hw)
return 0;
}
+/**
+ * ixgbevf_poll_mbx - Wait for message and read it from the mailbox
+ * @hw: pointer to the HW structure
+ * @msg: The message buffer
+ * @size: Length of buffer
+ *
+ * returns 0 if it successfully read message from buffer
+ **/
+s32 ixgbevf_poll_mbx(struct ixgbe_hw *hw, u32 *msg, u16 size)
+{
+ struct ixgbe_mbx_info *mbx = &hw->mbx;
+ s32 ret_val = IXGBE_ERR_CONFIG;
+
+ if (!mbx->ops.read || !mbx->ops.check_for_msg || !mbx->timeout)
+ return ret_val;
+
+ /* limit read to size of mailbox */
+ if (size > mbx->size)
+ size = mbx->size;
+
+ ret_val = ixgbevf_poll_for_msg(hw);
+ /* if ack received read message, otherwise we timed out */
+ if (!ret_val)
+ ret_val = mbx->ops.read(hw, msg, size);
+
+ return ret_val;
+}
+
+/**
+ * ixgbevf_write_mbx - Write a message to the mailbox and wait for ACK
+ * @hw: pointer to the HW structure
+ * @msg: The message buffer
+ * @size: Length of buffer
+ *
+ * returns 0 if it successfully copied message into the buffer and
+ * received an ACK to that message within specified period
+ **/
+s32 ixgbevf_write_mbx(struct ixgbe_hw *hw, u32 *msg, u16 size)
+{
+ struct ixgbe_mbx_info *mbx = &hw->mbx;
+ s32 ret_val = IXGBE_ERR_CONFIG;
+
+ /**
+ * exit if either we can't write, release
+ * or there is no timeout defined
+ */
+ if (!mbx->ops.write || !mbx->ops.check_for_ack || !mbx->ops.release ||
+ !mbx->timeout)
+ return ret_val;
+
+ if (size > mbx->size)
+ ret_val = IXGBE_ERR_PARAM;
+ else
+ ret_val = mbx->ops.write(hw, msg, size);
+
+ return ret_val;
+}
+
const struct ixgbe_mbx_operations ixgbevf_mbx_ops = {
.init_params = ixgbevf_init_mbx_params_vf,
+ .release = ixgbevf_release_mbx_lock_vf,
.read = ixgbevf_read_mbx_vf,
.write = ixgbevf_write_mbx_vf,
- .read_posted = ixgbevf_read_posted_mbx,
- .write_posted = ixgbevf_write_posted_mbx,
+ .check_for_msg = ixgbevf_check_for_msg_vf,
+ .check_for_ack = ixgbevf_check_for_ack_vf,
+ .check_for_rst = ixgbevf_check_for_rst_vf,
+};
+
+const struct ixgbe_mbx_operations ixgbevf_mbx_ops_legacy = {
+ .init_params = ixgbevf_init_mbx_params_vf,
+ .release = ixgbevf_release_mbx_lock_vf_legacy,
+ .read = ixgbevf_read_mbx_vf_legacy,
+ .write = ixgbevf_write_mbx_vf_legacy,
.check_for_msg = ixgbevf_check_for_msg_vf,
.check_for_ack = ixgbevf_check_for_ack_vf,
.check_for_rst = ixgbevf_check_for_rst_vf,
diff --git a/drivers/net/ethernet/intel/ixgbevf/mbx.h b/drivers/net/ethernet/intel/ixgbevf/mbx.h
index 853796c..7346ccf 100644
--- a/drivers/net/ethernet/intel/ixgbevf/mbx.h
+++ b/drivers/net/ethernet/intel/ixgbevf/mbx.h
@@ -7,7 +7,6 @@
#include "vf.h"
#define IXGBE_VFMAILBOX_SIZE 16 /* 16 32 bit words - 64 bytes */
-#define IXGBE_ERR_MBX -100
#define IXGBE_VFMAILBOX 0x002FC
#define IXGBE_VFMBMEM 0x00200
@@ -39,14 +38,17 @@
/* If it's a IXGBE_VF_* msg then it originates in the VF and is sent to the
* PF. The reverse is true if it is IXGBE_PF_*.
- * Message ACK's are the value or'd with 0xF0000000
+ * Message results are the value or'd with 0xF0000000
*/
-/* Messages below or'd with this are the ACK */
-#define IXGBE_VT_MSGTYPE_ACK 0x80000000
-/* Messages below or'd with this are the NACK */
-#define IXGBE_VT_MSGTYPE_NACK 0x40000000
-/* Indicates that VF is still clear to send requests */
-#define IXGBE_VT_MSGTYPE_CTS 0x20000000
+#define IXGBE_VT_MSGTYPE_SUCCESS 0x80000000 /* Messages or'd with this
+ * have succeeded
+ */
+#define IXGBE_VT_MSGTYPE_FAILURE 0x40000000 /* Messages or'd with this
+ * have failed
+ */
+#define IXGBE_VT_MSGTYPE_CTS 0x20000000 /* Indicates that VF is still
+ * clear to send requests
+ */
#define IXGBE_VT_MSGINFO_SHIFT 16
/* bits 23:16 are used for exra info for certain messages */
#define IXGBE_VT_MSGINFO_MASK (0xFF << IXGBE_VT_MSGINFO_SHIFT)
@@ -63,6 +65,7 @@ enum ixgbe_pfvf_api_rev {
ixgbe_mbox_api_12, /* API version 1.2, linux/freebsd VF driver */
ixgbe_mbox_api_13, /* API version 1.3, linux/freebsd VF driver */
ixgbe_mbox_api_14, /* API version 1.4, linux/freebsd VF driver */
+ ixgbe_mbox_api_15, /* API version 1.5, linux/freebsd VF driver */
/* This value should always be last */
ixgbe_mbox_api_unknown, /* indicates that API version is not known */
};
diff --git a/drivers/net/ethernet/intel/ixgbevf/vf.c b/drivers/net/ethernet/intel/ixgbevf/vf.c
index d459f5c..61d8970 100644
--- a/drivers/net/ethernet/intel/ixgbevf/vf.c
+++ b/drivers/net/ethernet/intel/ixgbevf/vf.c
@@ -13,13 +13,12 @@
static inline s32 ixgbevf_write_msg_read_ack(struct ixgbe_hw *hw, u32 *msg,
u32 *retmsg, u16 size)
{
- struct ixgbe_mbx_info *mbx = &hw->mbx;
- s32 retval = mbx->ops.write_posted(hw, msg, size);
+ s32 retval = ixgbevf_write_mbx(hw, msg, size);
if (retval)
return retval;
- return mbx->ops.read_posted(hw, retmsg, size);
+ return ixgbevf_poll_mbx(hw, retmsg, size);
}
/**
@@ -75,6 +74,9 @@ static s32 ixgbevf_reset_hw_vf(struct ixgbe_hw *hw)
/* reset the api version */
hw->api_version = ixgbe_mbox_api_10;
+ hw->mbx.ops.init_params(hw);
+ memcpy(&hw->mbx.ops, &ixgbevf_mbx_ops_legacy,
+ sizeof(struct ixgbe_mbx_operations));
IXGBE_WRITE_REG(hw, IXGBE_VFCTRL, IXGBE_CTRL_RST);
IXGBE_WRITE_FLUSH(hw);
@@ -92,7 +94,7 @@ static s32 ixgbevf_reset_hw_vf(struct ixgbe_hw *hw)
mbx->timeout = IXGBE_VF_MBX_INIT_TIMEOUT;
msgbuf[0] = IXGBE_VF_RESET;
- mbx->ops.write_posted(hw, msgbuf, 1);
+ ixgbevf_write_mbx(hw, msgbuf, 1);
mdelay(10);
@@ -100,7 +102,7 @@ static s32 ixgbevf_reset_hw_vf(struct ixgbe_hw *hw)
* also set up the mc_filter_type which is piggy backed
* on the mac address in word 3
*/
- ret_val = mbx->ops.read_posted(hw, msgbuf, IXGBE_VF_PERMADDR_MSG_LEN);
+ ret_val = ixgbevf_poll_mbx(hw, msgbuf, IXGBE_VF_PERMADDR_MSG_LEN);
if (ret_val)
return ret_val;
@@ -108,11 +110,11 @@ static s32 ixgbevf_reset_hw_vf(struct ixgbe_hw *hw)
* to indicate that no MAC address has yet been assigned for
* the VF.
*/
- if (msgbuf[0] != (IXGBE_VF_RESET | IXGBE_VT_MSGTYPE_ACK) &&
- msgbuf[0] != (IXGBE_VF_RESET | IXGBE_VT_MSGTYPE_NACK))
+ if (msgbuf[0] != (IXGBE_VF_RESET | IXGBE_VT_MSGTYPE_SUCCESS) &&
+ msgbuf[0] != (IXGBE_VF_RESET | IXGBE_VT_MSGTYPE_FAILURE))
return IXGBE_ERR_INVALID_MAC_ADDR;
- if (msgbuf[0] == (IXGBE_VF_RESET | IXGBE_VT_MSGTYPE_ACK))
+ if (msgbuf[0] == (IXGBE_VF_RESET | IXGBE_VT_MSGTYPE_SUCCESS))
ether_addr_copy(hw->mac.perm_addr, addr);
hw->mac.mc_filter_type = msgbuf[IXGBE_VF_MC_TYPE_WORD];
@@ -269,7 +271,7 @@ static s32 ixgbevf_set_uc_addr_vf(struct ixgbe_hw *hw, u32 index, u8 *addr)
if (!ret_val) {
msgbuf[0] &= ~IXGBE_VT_MSGTYPE_CTS;
- if (msgbuf[0] == (msgbuf_chk | IXGBE_VT_MSGTYPE_NACK))
+ if (msgbuf[0] == (msgbuf_chk | IXGBE_VT_MSGTYPE_FAILURE))
return -ENOMEM;
}
@@ -311,6 +313,7 @@ int ixgbevf_get_reta_locked(struct ixgbe_hw *hw, u32 *reta, int num_rx_queues)
* is not supported for this device type.
*/
switch (hw->api_version) {
+ case ixgbe_mbox_api_15:
case ixgbe_mbox_api_14:
case ixgbe_mbox_api_13:
case ixgbe_mbox_api_12:
@@ -323,12 +326,12 @@ int ixgbevf_get_reta_locked(struct ixgbe_hw *hw, u32 *reta, int num_rx_queues)
msgbuf[0] = IXGBE_VF_GET_RETA;
- err = hw->mbx.ops.write_posted(hw, msgbuf, 1);
+ err = ixgbevf_write_mbx(hw, msgbuf, 1);
if (err)
return err;
- err = hw->mbx.ops.read_posted(hw, msgbuf, dwords + 1);
+ err = ixgbevf_poll_mbx(hw, msgbuf, dwords + 1);
if (err)
return err;
@@ -336,14 +339,14 @@ int ixgbevf_get_reta_locked(struct ixgbe_hw *hw, u32 *reta, int num_rx_queues)
msgbuf[0] &= ~IXGBE_VT_MSGTYPE_CTS;
/* If the operation has been refused by a PF return -EPERM */
- if (msgbuf[0] == (IXGBE_VF_GET_RETA | IXGBE_VT_MSGTYPE_NACK))
+ if (msgbuf[0] == (IXGBE_VF_GET_RETA | IXGBE_VT_MSGTYPE_FAILURE))
return -EPERM;
/* If we didn't get an ACK there must have been
* some sort of mailbox error so we should treat it
* as such.
*/
- if (msgbuf[0] != (IXGBE_VF_GET_RETA | IXGBE_VT_MSGTYPE_ACK))
+ if (msgbuf[0] != (IXGBE_VF_GET_RETA | IXGBE_VT_MSGTYPE_SUCCESS))
return IXGBE_ERR_MBX;
/* ixgbevf doesn't support more than 2 queues at the moment */
@@ -379,6 +382,7 @@ int ixgbevf_get_rss_key_locked(struct ixgbe_hw *hw, u8 *rss_key)
* or if the operation is not supported for this device type.
*/
switch (hw->api_version) {
+ case ixgbe_mbox_api_15:
case ixgbe_mbox_api_14:
case ixgbe_mbox_api_13:
case ixgbe_mbox_api_12:
@@ -390,12 +394,12 @@ int ixgbevf_get_rss_key_locked(struct ixgbe_hw *hw, u8 *rss_key)
}
msgbuf[0] = IXGBE_VF_GET_RSS_KEY;
- err = hw->mbx.ops.write_posted(hw, msgbuf, 1);
+ err = ixgbevf_write_mbx(hw, msgbuf, 1);
if (err)
return err;
- err = hw->mbx.ops.read_posted(hw, msgbuf, 11);
+ err = ixgbevf_poll_mbx(hw, msgbuf, 11);
if (err)
return err;
@@ -403,14 +407,14 @@ int ixgbevf_get_rss_key_locked(struct ixgbe_hw *hw, u8 *rss_key)
msgbuf[0] &= ~IXGBE_VT_MSGTYPE_CTS;
/* If the operation has been refused by a PF return -EPERM */
- if (msgbuf[0] == (IXGBE_VF_GET_RSS_KEY | IXGBE_VT_MSGTYPE_NACK))
+ if (msgbuf[0] == (IXGBE_VF_GET_RSS_KEY | IXGBE_VT_MSGTYPE_FAILURE))
return -EPERM;
/* If we didn't get an ACK there must have been
* some sort of mailbox error so we should treat it
* as such.
*/
- if (msgbuf[0] != (IXGBE_VF_GET_RSS_KEY | IXGBE_VT_MSGTYPE_ACK))
+ if (msgbuf[0] != (IXGBE_VF_GET_RSS_KEY | IXGBE_VT_MSGTYPE_SUCCESS))
return IXGBE_ERR_MBX;
memcpy(rss_key, msgbuf + 1, IXGBEVF_RSS_HASH_KEY_SIZE);
@@ -442,7 +446,7 @@ static s32 ixgbevf_set_rar_vf(struct ixgbe_hw *hw, u32 index, u8 *addr,
/* if nacked the address was rejected, use "perm_addr" */
if (!ret_val &&
- (msgbuf[0] == (IXGBE_VF_SET_MAC_ADDR | IXGBE_VT_MSGTYPE_NACK))) {
+ (msgbuf[0] == (IXGBE_VF_SET_MAC_ADDR | IXGBE_VT_MSGTYPE_FAILURE))) {
ixgbevf_get_mac_addr_vf(hw, hw->mac.addr);
return IXGBE_ERR_MBX;
}
@@ -545,8 +549,9 @@ static s32 ixgbevf_update_xcast_mode(struct ixgbe_hw *hw, int xcast_mode)
if (xcast_mode == IXGBEVF_XCAST_MODE_PROMISC)
return -EOPNOTSUPP;
fallthrough;
- case ixgbe_mbox_api_14:
case ixgbe_mbox_api_13:
+ case ixgbe_mbox_api_14:
+ case ixgbe_mbox_api_15:
break;
default:
return -EOPNOTSUPP;
@@ -561,7 +566,7 @@ static s32 ixgbevf_update_xcast_mode(struct ixgbe_hw *hw, int xcast_mode)
return err;
msgbuf[0] &= ~IXGBE_VT_MSGTYPE_CTS;
- if (msgbuf[0] == (IXGBE_VF_UPDATE_XCAST_MODE | IXGBE_VT_MSGTYPE_NACK))
+ if (msgbuf[0] == (IXGBE_VF_UPDATE_XCAST_MODE | IXGBE_VT_MSGTYPE_FAILURE))
return -EPERM;
return 0;
@@ -606,7 +611,7 @@ static s32 ixgbevf_set_vfta_vf(struct ixgbe_hw *hw, u32 vlan, u32 vind,
msgbuf[0] &= ~IXGBE_VT_MSGTYPE_CTS;
msgbuf[0] &= ~(0xFF << IXGBE_VT_MSGINFO_SHIFT);
- if (msgbuf[0] != (IXGBE_VF_SET_VLAN | IXGBE_VT_MSGTYPE_ACK))
+ if (msgbuf[0] != (IXGBE_VF_SET_VLAN | IXGBE_VT_MSGTYPE_SUCCESS))
err = IXGBE_ERR_INVALID_ARGUMENT;
mbx_err:
@@ -705,12 +710,15 @@ static s32 ixgbevf_check_mac_link_vf(struct ixgbe_hw *hw,
/* if the read failed it could just be a mailbox collision, best wait
* until we are called again and don't report an error
*/
- if (mbx->ops.read(hw, &in_msg, 1))
+ if (mbx->ops.read(hw, &in_msg, 1)) {
+ if (hw->api_version >= ixgbe_mbox_api_15)
+ mac->get_link_status = false;
goto out;
+ }
if (!(in_msg & IXGBE_VT_MSGTYPE_CTS)) {
/* msg is not CTS and is NACK we must have lost CTS status */
- if (in_msg & IXGBE_VT_MSGTYPE_NACK)
+ if (in_msg & IXGBE_VT_MSGTYPE_FAILURE)
ret_val = -1;
goto out;
}
@@ -816,7 +824,7 @@ static s32 ixgbevf_set_rlpml_vf(struct ixgbe_hw *hw, u16 max_size)
if (ret_val)
return ret_val;
if ((msgbuf[0] & IXGBE_VF_SET_LPE) &&
- (msgbuf[0] & IXGBE_VT_MSGTYPE_NACK))
+ (msgbuf[0] & IXGBE_VT_MSGTYPE_FAILURE))
return IXGBE_ERR_MBX;
return 0;
@@ -863,7 +871,8 @@ static int ixgbevf_negotiate_api_version_vf(struct ixgbe_hw *hw, int api)
msg[0] &= ~IXGBE_VT_MSGTYPE_CTS;
/* Store value and return 0 on success */
- if (msg[0] == (IXGBE_VF_API_NEGOTIATE | IXGBE_VT_MSGTYPE_ACK)) {
+ if (msg[0] == (IXGBE_VF_API_NEGOTIATE |
+ IXGBE_VT_MSGTYPE_SUCCESS)) {
hw->api_version = api;
return 0;
}
@@ -901,6 +910,7 @@ int ixgbevf_get_queues(struct ixgbe_hw *hw, unsigned int *num_tcs,
case ixgbe_mbox_api_12:
case ixgbe_mbox_api_13:
case ixgbe_mbox_api_14:
+ case ixgbe_mbox_api_15:
break;
default:
return 0;
@@ -918,7 +928,7 @@ int ixgbevf_get_queues(struct ixgbe_hw *hw, unsigned int *num_tcs,
* some sort of mailbox error so we should treat it
* as such
*/
- if (msg[0] != (IXGBE_VF_GET_QUEUE | IXGBE_VT_MSGTYPE_ACK))
+ if (msg[0] != (IXGBE_VF_GET_QUEUE | IXGBE_VT_MSGTYPE_SUCCESS))
return IXGBE_ERR_MBX;
/* record and validate values from message */
diff --git a/drivers/net/ethernet/intel/ixgbevf/vf.h b/drivers/net/ethernet/intel/ixgbevf/vf.h
index 1d8209d..54158da 100644
--- a/drivers/net/ethernet/intel/ixgbevf/vf.h
+++ b/drivers/net/ethernet/intel/ixgbevf/vf.h
@@ -73,10 +73,9 @@ struct ixgbe_mac_info {
struct ixgbe_mbx_operations {
s32 (*init_params)(struct ixgbe_hw *hw);
+ void (*release)(struct ixgbe_hw *hw);
s32 (*read)(struct ixgbe_hw *, u32 *, u16);
s32 (*write)(struct ixgbe_hw *, u32 *, u16);
- s32 (*read_posted)(struct ixgbe_hw *, u32 *, u16);
- s32 (*write_posted)(struct ixgbe_hw *, u32 *, u16);
s32 (*check_for_msg)(struct ixgbe_hw *);
s32 (*check_for_ack)(struct ixgbe_hw *);
s32 (*check_for_rst)(struct ixgbe_hw *);
@@ -96,7 +95,7 @@ struct ixgbe_mbx_info {
struct ixgbe_mbx_stats stats;
u32 timeout;
u32 udelay;
- u32 v2p_mailbox;
+ u32 vf_mailbox;
u16 size;
};
diff --git a/drivers/net/ethernet/lantiq_etop.c b/drivers/net/ethernet/lantiq_etop.c
index 072391c..9b6fa27 100644
--- a/drivers/net/ethernet/lantiq_etop.c
+++ b/drivers/net/ethernet/lantiq_etop.c
@@ -65,8 +65,8 @@
/* use 2 static channels for TX/RX */
#define LTQ_ETOP_TX_CHANNEL 1
#define LTQ_ETOP_RX_CHANNEL 6
-#define IS_TX(x) (x == LTQ_ETOP_TX_CHANNEL)
-#define IS_RX(x) (x == LTQ_ETOP_RX_CHANNEL)
+#define IS_TX(x) ((x) == LTQ_ETOP_TX_CHANNEL)
+#define IS_RX(x) ((x) == LTQ_ETOP_RX_CHANNEL)
#define ltq_etop_r32(x) ltq_r32(ltq_etop_membase + (x))
#define ltq_etop_w32(x, y) ltq_w32(x, ltq_etop_membase + (y))
@@ -111,9 +111,9 @@ ltq_etop_alloc_skb(struct ltq_etop_chan *ch)
ch->skb[ch->dma.desc] = netdev_alloc_skb(ch->netdev, MAX_DMA_DATA_LEN);
if (!ch->skb[ch->dma.desc])
return -ENOMEM;
- ch->dma.desc_base[ch->dma.desc].addr = dma_map_single(&priv->pdev->dev,
- ch->skb[ch->dma.desc]->data, MAX_DMA_DATA_LEN,
- DMA_FROM_DEVICE);
+ ch->dma.desc_base[ch->dma.desc].addr =
+ dma_map_single(&priv->pdev->dev, ch->skb[ch->dma.desc]->data,
+ MAX_DMA_DATA_LEN, DMA_FROM_DEVICE);
ch->dma.desc_base[ch->dma.desc].addr =
CPHYSADDR(ch->skb[ch->dma.desc]->data);
ch->dma.desc_base[ch->dma.desc].ctl =
@@ -135,7 +135,7 @@ ltq_etop_hw_receive(struct ltq_etop_chan *ch)
spin_lock_irqsave(&priv->lock, flags);
if (ltq_etop_alloc_skb(ch)) {
netdev_err(ch->netdev,
- "failed to allocate new rx buffer, stopping DMA\n");
+ "failed to allocate new rx buffer, stopping DMA\n");
ltq_dma_close(&ch->dma);
}
ch->dma.desc++;
@@ -185,7 +185,7 @@ ltq_etop_poll_tx(struct napi_struct *napi, int budget)
dev_kfree_skb_any(ch->skb[ch->tx_free]);
ch->skb[ch->tx_free] = NULL;
memset(&ch->dma.desc_base[ch->tx_free], 0,
- sizeof(struct ltq_dma_desc));
+ sizeof(struct ltq_dma_desc));
ch->tx_free++;
ch->tx_free %= LTQ_DESC_NUM;
}
@@ -218,6 +218,7 @@ ltq_etop_free_channel(struct net_device *dev, struct ltq_etop_chan *ch)
free_irq(ch->dma.irq, priv);
if (IS_RX(ch->idx)) {
int desc;
+
for (desc = 0; desc < LTQ_DESC_NUM; desc++)
dev_kfree_skb_any(ch->skb[ch->dma.desc]);
}
@@ -246,18 +247,18 @@ ltq_etop_hw_init(struct net_device *dev)
switch (priv->pldata->mii_mode) {
case PHY_INTERFACE_MODE_RMII:
- ltq_etop_w32_mask(ETOP_MII_MASK,
- ETOP_MII_REVERSE, LTQ_ETOP_CFG);
+ ltq_etop_w32_mask(ETOP_MII_MASK, ETOP_MII_REVERSE,
+ LTQ_ETOP_CFG);
break;
case PHY_INTERFACE_MODE_MII:
- ltq_etop_w32_mask(ETOP_MII_MASK,
- ETOP_MII_NORMAL, LTQ_ETOP_CFG);
+ ltq_etop_w32_mask(ETOP_MII_MASK, ETOP_MII_NORMAL,
+ LTQ_ETOP_CFG);
break;
default:
netdev_err(dev, "unknown mii mode %d\n",
- priv->pldata->mii_mode);
+ priv->pldata->mii_mode);
return -ENOTSUPP;
}
@@ -270,7 +271,8 @@ ltq_etop_hw_init(struct net_device *dev)
int irq = LTQ_DMA_CH0_INT + i;
struct ltq_etop_chan *ch = &priv->ch[i];
- ch->idx = ch->dma.nr = i;
+ ch->dma.nr = i;
+ ch->idx = ch->dma.nr;
ch->dma.dev = &priv->pdev->dev;
if (IS_TX(i)) {
@@ -305,9 +307,9 @@ ltq_etop_hw_init(struct net_device *dev)
static void
ltq_etop_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info)
{
- strlcpy(info->driver, "Lantiq ETOP", sizeof(info->driver));
- strlcpy(info->bus_info, "internal", sizeof(info->bus_info));
- strlcpy(info->version, DRV_VERSION, sizeof(info->version));
+ strscpy(info->driver, "Lantiq ETOP", sizeof(info->driver));
+ strscpy(info->bus_info, "internal", sizeof(info->bus_info));
+ strscpy(info->version, DRV_VERSION, sizeof(info->version));
}
static const struct ethtool_ops ltq_etop_ethtool_ops = {
@@ -399,7 +401,7 @@ ltq_etop_mdio_init(struct net_device *dev)
priv->mii_bus->write = ltq_etop_mdio_wr;
priv->mii_bus->name = "ltq_mii";
snprintf(priv->mii_bus->id, MII_BUS_ID_SIZE, "%s-%x",
- priv->pdev->name, priv->pdev->id);
+ priv->pdev->name, priv->pdev->id);
if (mdiobus_register(priv->mii_bus)) {
err = -ENXIO;
goto err_out_free_mdiobus;
@@ -496,8 +498,9 @@ ltq_etop_tx(struct sk_buff *skb, struct net_device *dev)
netif_trans_update(dev);
spin_lock_irqsave(&priv->lock, flags);
- desc->addr = ((unsigned int) dma_map_single(&priv->pdev->dev, skb->data, len,
+ desc->addr = ((unsigned int)dma_map_single(&priv->pdev->dev, skb->data, len,
DMA_TO_DEVICE)) - byte_offset;
+ /* Make sure the address is written before we give it to HW */
wmb();
desc->ctl = LTQ_DMA_OWN | LTQ_DMA_SOP | LTQ_DMA_EOP |
LTQ_DMA_TX_OFFSET(byte_offset) | (len & LTQ_DMA_SIZE_MASK);
@@ -539,7 +542,7 @@ ltq_etop_set_mac_address(struct net_device *dev, void *p)
spin_lock_irqsave(&priv->lock, flags);
ltq_etop_w32(*((u32 *)dev->dev_addr), LTQ_ETOP_MAC_DA0);
ltq_etop_w32(*((u16 *)&dev->dev_addr[4]) << 16,
- LTQ_ETOP_MAC_DA1);
+ LTQ_ETOP_MAC_DA1);
spin_unlock_irqrestore(&priv->lock, flags);
}
return ret;
@@ -652,15 +655,15 @@ ltq_etop_probe(struct platform_device *pdev)
}
res = devm_request_mem_region(&pdev->dev, res->start,
- resource_size(res), dev_name(&pdev->dev));
+ resource_size(res), dev_name(&pdev->dev));
if (!res) {
dev_err(&pdev->dev, "failed to request etop resource\n");
err = -EBUSY;
goto err_out;
}
- ltq_etop_membase = devm_ioremap(&pdev->dev,
- res->start, resource_size(res));
+ ltq_etop_membase = devm_ioremap(&pdev->dev, res->start,
+ resource_size(res));
if (!ltq_etop_membase) {
dev_err(&pdev->dev, "failed to remap etop engine %d\n",
pdev->id);
@@ -687,22 +690,22 @@ ltq_etop_probe(struct platform_device *pdev)
err = device_property_read_u32(&pdev->dev, "lantiq,tx-burst-length", &priv->tx_burst_len);
if (err < 0) {
dev_err(&pdev->dev, "unable to read tx-burst-length property\n");
- return err;
+ goto err_free;
}
err = device_property_read_u32(&pdev->dev, "lantiq,rx-burst-length", &priv->rx_burst_len);
if (err < 0) {
dev_err(&pdev->dev, "unable to read rx-burst-length property\n");
- return err;
+ goto err_free;
}
for (i = 0; i < MAX_DMA_CHAN; i++) {
if (IS_TX(i))
netif_napi_add(dev, &priv->ch[i].napi,
- ltq_etop_poll_tx, 8);
+ ltq_etop_poll_tx, 8);
else if (IS_RX(i))
netif_napi_add(dev, &priv->ch[i].napi,
- ltq_etop_poll_rx, 32);
+ ltq_etop_poll_rx, 32);
priv->ch[i].netdev = dev;
}
diff --git a/drivers/net/ethernet/lantiq_xrx200.c b/drivers/net/ethernet/lantiq_xrx200.c
index 96bd6f2..41d1113 100644
--- a/drivers/net/ethernet/lantiq_xrx200.c
+++ b/drivers/net/ethernet/lantiq_xrx200.c
@@ -27,6 +27,9 @@
#define XRX200_DMA_TX 1
#define XRX200_DMA_BURST_LEN 8
+#define XRX200_DMA_PACKET_COMPLETE 0
+#define XRX200_DMA_PACKET_IN_PROGRESS 1
+
/* cpu port mac */
#define PMAC_RX_IPG 0x0024
#define PMAC_RX_IPG_MASK 0xf
@@ -60,7 +63,14 @@ struct xrx200_chan {
struct napi_struct napi;
struct ltq_dma_channel dma;
- struct sk_buff *skb[LTQ_DESC_NUM];
+
+ union {
+ struct sk_buff *skb[LTQ_DESC_NUM];
+ void *rx_buff[LTQ_DESC_NUM];
+ };
+
+ struct sk_buff *skb_head;
+ struct sk_buff *skb_tail;
struct xrx200_priv *priv;
};
@@ -72,6 +82,7 @@ struct xrx200_priv {
struct xrx200_chan chan_rx;
u16 rx_buf_size;
+ u16 rx_skb_size;
struct net_device *net_dev;
struct device *dev;
@@ -109,6 +120,12 @@ static int xrx200_buffer_size(int mtu)
return round_up(xrx200_max_frame_len(mtu), 4 * XRX200_DMA_BURST_LEN);
}
+static int xrx200_skb_size(u16 buf_size)
+{
+ return SKB_DATA_ALIGN(buf_size + NET_SKB_PAD + NET_IP_ALIGN) +
+ SKB_DATA_ALIGN(sizeof(struct skb_shared_info));
+}
+
/* drop all the packets from the DMA ring */
static void xrx200_flush_dma(struct xrx200_chan *ch)
{
@@ -167,30 +184,29 @@ static int xrx200_close(struct net_device *net_dev)
return 0;
}
-static int xrx200_alloc_skb(struct xrx200_chan *ch)
+static int xrx200_alloc_buf(struct xrx200_chan *ch, void *(*alloc)(unsigned int size))
{
- struct sk_buff *skb = ch->skb[ch->dma.desc];
+ void *buf = ch->rx_buff[ch->dma.desc];
struct xrx200_priv *priv = ch->priv;
dma_addr_t mapping;
int ret = 0;
- ch->skb[ch->dma.desc] = netdev_alloc_skb_ip_align(priv->net_dev,
- priv->rx_buf_size);
- if (!ch->skb[ch->dma.desc]) {
+ ch->rx_buff[ch->dma.desc] = alloc(priv->rx_skb_size);
+ if (!ch->rx_buff[ch->dma.desc]) {
ret = -ENOMEM;
goto skip;
}
- mapping = dma_map_single(priv->dev, ch->skb[ch->dma.desc]->data,
+ mapping = dma_map_single(priv->dev, ch->rx_buff[ch->dma.desc],
priv->rx_buf_size, DMA_FROM_DEVICE);
if (unlikely(dma_mapping_error(priv->dev, mapping))) {
- dev_kfree_skb_any(ch->skb[ch->dma.desc]);
- ch->skb[ch->dma.desc] = skb;
+ skb_free_frag(ch->rx_buff[ch->dma.desc]);
+ ch->rx_buff[ch->dma.desc] = buf;
ret = -ENOMEM;
goto skip;
}
- ch->dma.desc_base[ch->dma.desc].addr = mapping;
+ ch->dma.desc_base[ch->dma.desc].addr = mapping + NET_SKB_PAD + NET_IP_ALIGN;
/* Make sure the address is written before we give it to HW */
wmb();
skip:
@@ -204,12 +220,14 @@ static int xrx200_hw_receive(struct xrx200_chan *ch)
{
struct xrx200_priv *priv = ch->priv;
struct ltq_dma_desc *desc = &ch->dma.desc_base[ch->dma.desc];
- struct sk_buff *skb = ch->skb[ch->dma.desc];
- int len = (desc->ctl & LTQ_DMA_SIZE_MASK);
+ void *buf = ch->rx_buff[ch->dma.desc];
+ u32 ctl = desc->ctl;
+ int len = (ctl & LTQ_DMA_SIZE_MASK);
struct net_device *net_dev = priv->net_dev;
+ struct sk_buff *skb;
int ret;
- ret = xrx200_alloc_skb(ch);
+ ret = xrx200_alloc_buf(ch, napi_alloc_frag);
ch->dma.desc++;
ch->dma.desc %= LTQ_DESC_NUM;
@@ -220,13 +238,39 @@ static int xrx200_hw_receive(struct xrx200_chan *ch)
return ret;
}
+ skb = build_skb(buf, priv->rx_skb_size);
+ skb_reserve(skb, NET_SKB_PAD);
skb_put(skb, len);
- skb->protocol = eth_type_trans(skb, net_dev);
- netif_receive_skb(skb);
- net_dev->stats.rx_packets++;
- net_dev->stats.rx_bytes += len - ETH_FCS_LEN;
- return 0;
+ /* add buffers to skb via skb->frag_list */
+ if (ctl & LTQ_DMA_SOP) {
+ ch->skb_head = skb;
+ ch->skb_tail = skb;
+ skb_reserve(skb, NET_IP_ALIGN);
+ } else if (ch->skb_head) {
+ if (ch->skb_head == ch->skb_tail)
+ skb_shinfo(ch->skb_tail)->frag_list = skb;
+ else
+ ch->skb_tail->next = skb;
+ ch->skb_tail = skb;
+ ch->skb_head->len += skb->len;
+ ch->skb_head->data_len += skb->len;
+ ch->skb_head->truesize += skb->truesize;
+ }
+
+ if (ctl & LTQ_DMA_EOP) {
+ ch->skb_head->protocol = eth_type_trans(ch->skb_head, net_dev);
+ netif_receive_skb(ch->skb_head);
+ net_dev->stats.rx_packets++;
+ net_dev->stats.rx_bytes += ch->skb_head->len;
+ ch->skb_head = NULL;
+ ch->skb_tail = NULL;
+ ret = XRX200_DMA_PACKET_COMPLETE;
+ } else {
+ ret = XRX200_DMA_PACKET_IN_PROGRESS;
+ }
+
+ return ret;
}
static int xrx200_poll_rx(struct napi_struct *napi, int budget)
@@ -241,7 +285,9 @@ static int xrx200_poll_rx(struct napi_struct *napi, int budget)
if ((desc->ctl & (LTQ_DMA_OWN | LTQ_DMA_C)) == LTQ_DMA_C) {
ret = xrx200_hw_receive(ch);
- if (ret)
+ if (ret == XRX200_DMA_PACKET_IN_PROGRESS)
+ continue;
+ if (ret != XRX200_DMA_PACKET_COMPLETE)
return ret;
rx++;
} else {
@@ -362,12 +408,13 @@ xrx200_change_mtu(struct net_device *net_dev, int new_mtu)
struct xrx200_chan *ch_rx = &priv->chan_rx;
int old_mtu = net_dev->mtu;
bool running = false;
- struct sk_buff *skb;
+ void *buff;
int curr_desc;
int ret = 0;
net_dev->mtu = new_mtu;
priv->rx_buf_size = xrx200_buffer_size(new_mtu);
+ priv->rx_skb_size = xrx200_skb_size(priv->rx_buf_size);
if (new_mtu <= old_mtu)
return ret;
@@ -383,14 +430,15 @@ xrx200_change_mtu(struct net_device *net_dev, int new_mtu)
for (ch_rx->dma.desc = 0; ch_rx->dma.desc < LTQ_DESC_NUM;
ch_rx->dma.desc++) {
- skb = ch_rx->skb[ch_rx->dma.desc];
- ret = xrx200_alloc_skb(ch_rx);
+ buff = ch_rx->rx_buff[ch_rx->dma.desc];
+ ret = xrx200_alloc_buf(ch_rx, netdev_alloc_frag);
if (ret) {
net_dev->mtu = old_mtu;
priv->rx_buf_size = xrx200_buffer_size(old_mtu);
+ priv->rx_skb_size = xrx200_skb_size(priv->rx_buf_size);
break;
}
- dev_kfree_skb_any(skb);
+ skb_free_frag(buff);
}
ch_rx->dma.desc = curr_desc;
@@ -443,7 +491,7 @@ static int xrx200_dma_init(struct xrx200_priv *priv)
ltq_dma_alloc_rx(&ch_rx->dma);
for (ch_rx->dma.desc = 0; ch_rx->dma.desc < LTQ_DESC_NUM;
ch_rx->dma.desc++) {
- ret = xrx200_alloc_skb(ch_rx);
+ ret = xrx200_alloc_buf(ch_rx, netdev_alloc_frag);
if (ret)
goto rx_free;
}
@@ -478,7 +526,7 @@ static int xrx200_dma_init(struct xrx200_priv *priv)
/* free the allocated RX ring */
for (i = 0; i < LTQ_DESC_NUM; i++) {
if (priv->chan_rx.skb[i])
- dev_kfree_skb_any(priv->chan_rx.skb[i]);
+ skb_free_frag(priv->chan_rx.rx_buff[i]);
}
rx_free:
@@ -495,7 +543,7 @@ static void xrx200_hw_cleanup(struct xrx200_priv *priv)
/* free the allocated RX ring */
for (i = 0; i < LTQ_DESC_NUM; i++)
- dev_kfree_skb_any(priv->chan_rx.skb[i]);
+ skb_free_frag(priv->chan_rx.rx_buff[i]);
}
static int xrx200_probe(struct platform_device *pdev)
@@ -520,6 +568,7 @@ static int xrx200_probe(struct platform_device *pdev)
net_dev->min_mtu = ETH_ZLEN;
net_dev->max_mtu = XRX200_DMA_DATA_LEN - xrx200_max_frame_len(0);
priv->rx_buf_size = xrx200_buffer_size(ETH_DATA_LEN);
+ priv->rx_skb_size = xrx200_skb_size(priv->rx_buf_size);
/* load the memory ranges */
priv->pmac_reg = devm_platform_get_and_ioremap_resource(pdev, 0, NULL);
@@ -564,8 +613,10 @@ static int xrx200_probe(struct platform_device *pdev)
PMAC_HD_CTL);
/* setup NAPI */
- netif_napi_add(net_dev, &priv->chan_rx.napi, xrx200_poll_rx, 32);
- netif_tx_napi_add(net_dev, &priv->chan_tx.napi, xrx200_tx_housekeeping, 32);
+ netif_napi_add(net_dev, &priv->chan_rx.napi, xrx200_poll_rx,
+ NAPI_POLL_WEIGHT);
+ netif_tx_napi_add(net_dev, &priv->chan_tx.napi, xrx200_tx_housekeeping,
+ NAPI_POLL_WEIGHT);
platform_set_drvdata(pdev, priv);
diff --git a/drivers/net/ethernet/marvell/mv643xx_eth.c b/drivers/net/ethernet/marvell/mv643xx_eth.c
index bb14fa2..1052475 100644
--- a/drivers/net/ethernet/marvell/mv643xx_eth.c
+++ b/drivers/net/ethernet/marvell/mv643xx_eth.c
@@ -1638,7 +1638,9 @@ static int mv643xx_eth_set_coalesce(struct net_device *dev,
}
static void
-mv643xx_eth_get_ringparam(struct net_device *dev, struct ethtool_ringparam *er)
+mv643xx_eth_get_ringparam(struct net_device *dev, struct ethtool_ringparam *er,
+ struct kernel_ethtool_ringparam *kernel_er,
+ struct netlink_ext_ack *extack)
{
struct mv643xx_eth_private *mp = netdev_priv(dev);
@@ -1650,7 +1652,9 @@ mv643xx_eth_get_ringparam(struct net_device *dev, struct ethtool_ringparam *er)
}
static int
-mv643xx_eth_set_ringparam(struct net_device *dev, struct ethtool_ringparam *er)
+mv643xx_eth_set_ringparam(struct net_device *dev, struct ethtool_ringparam *er,
+ struct kernel_ethtool_ringparam *kernel_er,
+ struct netlink_ext_ack *extack)
{
struct mv643xx_eth_private *mp = netdev_priv(dev);
@@ -3197,7 +3201,7 @@ static int mv643xx_eth_probe(struct platform_device *pdev)
dev->hw_features = dev->features;
dev->priv_flags |= IFF_UNICAST_FLT;
- dev->gso_max_segs = MV643XX_MAX_TSO_SEGS;
+ netif_set_gso_max_segs(dev, MV643XX_MAX_TSO_SEGS);
/* MTU range: 64 - 9500 */
dev->min_mtu = 64;
diff --git a/drivers/net/ethernet/marvell/mvneta.c b/drivers/net/ethernet/marvell/mvneta.c
index 5a7bdca..83c8908 100644
--- a/drivers/net/ethernet/marvell/mvneta.c
+++ b/drivers/net/ethernet/marvell/mvneta.c
@@ -38,6 +38,7 @@
#include <net/ipv6.h>
#include <net/tso.h>
#include <net/page_pool.h>
+#include <net/pkt_cls.h>
#include <linux/bpf_trace.h>
/* Registers */
@@ -247,12 +248,39 @@
#define MVNETA_TXQ_SENT_DESC_MASK 0x3fff0000
#define MVNETA_PORT_TX_RESET 0x3cf0
#define MVNETA_PORT_TX_DMA_RESET BIT(0)
+#define MVNETA_TXQ_CMD1_REG 0x3e00
+#define MVNETA_TXQ_CMD1_BW_LIM_SEL_V1 BIT(3)
+#define MVNETA_TXQ_CMD1_BW_LIM_EN BIT(0)
+#define MVNETA_REFILL_NUM_CLK_REG 0x3e08
+#define MVNETA_REFILL_MAX_NUM_CLK 0x0000ffff
#define MVNETA_TX_MTU 0x3e0c
#define MVNETA_TX_TOKEN_SIZE 0x3e14
#define MVNETA_TX_TOKEN_SIZE_MAX 0xffffffff
+#define MVNETA_TXQ_BUCKET_REFILL_REG(q) (0x3e20 + ((q) << 2))
+#define MVNETA_TXQ_BUCKET_REFILL_PERIOD_MASK 0x3ff00000
+#define MVNETA_TXQ_BUCKET_REFILL_PERIOD_SHIFT 20
+#define MVNETA_TXQ_BUCKET_REFILL_VALUE_MAX 0x0007ffff
#define MVNETA_TXQ_TOKEN_SIZE_REG(q) (0x3e40 + ((q) << 2))
#define MVNETA_TXQ_TOKEN_SIZE_MAX 0x7fffffff
+/* The values of the bucket refill base period and refill period are taken from
+ * the reference manual, and adds up to a base resolution of 10Kbps. This allows
+ * to cover all rate-limit values from 10Kbps up to 5Gbps
+ */
+
+/* Base period for the rate limit algorithm */
+#define MVNETA_TXQ_BUCKET_REFILL_BASE_PERIOD_NS 100
+
+/* Number of Base Period to wait between each bucket refill */
+#define MVNETA_TXQ_BUCKET_REFILL_PERIOD 1000
+
+/* The base resolution for rate limiting, in bps. Any max_rate value should be
+ * a multiple of that value.
+ */
+#define MVNETA_TXQ_RATE_LIMIT_RESOLUTION (NSEC_PER_SEC / \
+ (MVNETA_TXQ_BUCKET_REFILL_BASE_PERIOD_NS * \
+ MVNETA_TXQ_BUCKET_REFILL_PERIOD))
+
#define MVNETA_LPI_CTRL_0 0x2cc0
#define MVNETA_LPI_CTRL_1 0x2cc4
#define MVNETA_LPI_REQUEST_ENABLE BIT(0)
@@ -492,13 +520,13 @@ struct mvneta_port {
u8 mcast_count[256];
u16 tx_ring_size;
u16 rx_ring_size;
- u8 prio_tc_map[8];
phy_interface_t phy_interface;
struct device_node *dn;
unsigned int tx_csum_limit;
struct phylink *phylink;
struct phylink_config phylink_config;
+ struct phylink_pcs phylink_pcs;
struct phy *comphy;
struct mvneta_bm *bm_priv;
@@ -2212,7 +2240,7 @@ mvneta_run_xdp(struct mvneta_port *pp, struct mvneta_rx_queue *rxq,
mvneta_xdp_put_buff(pp, rxq, xdp, sinfo, sync);
break;
default:
- bpf_warn_invalid_xdp_action(act);
+ bpf_warn_invalid_xdp_action(pp->dev, prog, act);
fallthrough;
case XDP_ABORTED:
trace_xdp_exception(pp->dev, prog, act);
@@ -3819,58 +3847,31 @@ static int mvneta_set_mac_addr(struct net_device *dev, void *addr)
return 0;
}
-static void mvneta_validate(struct phylink_config *config,
- unsigned long *supported,
- struct phylink_link_state *state)
+static struct mvneta_port *mvneta_pcs_to_port(struct phylink_pcs *pcs)
{
- __ETHTOOL_DECLARE_LINK_MODE_MASK(mask) = { 0, };
+ return container_of(pcs, struct mvneta_port, phylink_pcs);
+}
+static int mvneta_pcs_validate(struct phylink_pcs *pcs,
+ unsigned long *supported,
+ const struct phylink_link_state *state)
+{
/* We only support QSGMII, SGMII, 802.3z and RGMII modes.
* When in 802.3z mode, we must have AN enabled:
* "Bit 2 Field InBandAnEn In-band Auto-Negotiation enable. ...
* When <PortType> = 1 (1000BASE-X) this field must be set to 1."
*/
if (phy_interface_mode_is_8023z(state->interface) &&
- !phylink_test(state->advertising, Autoneg)) {
- linkmode_zero(supported);
- return;
- }
+ !phylink_test(state->advertising, Autoneg))
+ return -EINVAL;
- /* Allow all the expected bits */
- phylink_set(mask, Autoneg);
- phylink_set_port_modes(mask);
-
- /* Asymmetric pause is unsupported */
- phylink_set(mask, Pause);
-
- /* Half-duplex at speeds higher than 100Mbit is unsupported */
- if (state->interface != PHY_INTERFACE_MODE_2500BASEX) {
- phylink_set(mask, 1000baseT_Full);
- phylink_set(mask, 1000baseX_Full);
- }
-
- if (state->interface == PHY_INTERFACE_MODE_2500BASEX) {
- phylink_set(mask, 2500baseT_Full);
- phylink_set(mask, 2500baseX_Full);
- }
-
- if (!phy_interface_mode_is_8023z(state->interface)) {
- /* 10M and 100M are only supported in non-802.3z mode */
- phylink_set(mask, 10baseT_Half);
- phylink_set(mask, 10baseT_Full);
- phylink_set(mask, 100baseT_Half);
- phylink_set(mask, 100baseT_Full);
- }
-
- linkmode_and(supported, supported, mask);
- linkmode_and(state->advertising, state->advertising, mask);
+ return 0;
}
-static void mvneta_mac_pcs_get_state(struct phylink_config *config,
- struct phylink_link_state *state)
+static void mvneta_pcs_get_state(struct phylink_pcs *pcs,
+ struct phylink_link_state *state)
{
- struct net_device *ndev = to_net_dev(config->dev);
- struct mvneta_port *pp = netdev_priv(ndev);
+ struct mvneta_port *pp = mvneta_pcs_to_port(pcs);
u32 gmac_stat;
gmac_stat = mvreg_read(pp, MVNETA_GMAC_STATUS);
@@ -3888,17 +3889,71 @@ static void mvneta_mac_pcs_get_state(struct phylink_config *config,
state->link = !!(gmac_stat & MVNETA_GMAC_LINK_UP);
state->duplex = !!(gmac_stat & MVNETA_GMAC_FULL_DUPLEX);
- state->pause = 0;
if (gmac_stat & MVNETA_GMAC_RX_FLOW_CTRL_ENABLE)
state->pause |= MLO_PAUSE_RX;
if (gmac_stat & MVNETA_GMAC_TX_FLOW_CTRL_ENABLE)
state->pause |= MLO_PAUSE_TX;
}
-static void mvneta_mac_an_restart(struct phylink_config *config)
+static int mvneta_pcs_config(struct phylink_pcs *pcs,
+ unsigned int mode, phy_interface_t interface,
+ const unsigned long *advertising,
+ bool permit_pause_to_mac)
{
- struct net_device *ndev = to_net_dev(config->dev);
- struct mvneta_port *pp = netdev_priv(ndev);
+ struct mvneta_port *pp = mvneta_pcs_to_port(pcs);
+ u32 mask, val, an, old_an, changed;
+
+ mask = MVNETA_GMAC_INBAND_AN_ENABLE |
+ MVNETA_GMAC_INBAND_RESTART_AN |
+ MVNETA_GMAC_AN_SPEED_EN |
+ MVNETA_GMAC_AN_FLOW_CTRL_EN |
+ MVNETA_GMAC_AN_DUPLEX_EN;
+
+ if (phylink_autoneg_inband(mode)) {
+ mask |= MVNETA_GMAC_CONFIG_MII_SPEED |
+ MVNETA_GMAC_CONFIG_GMII_SPEED |
+ MVNETA_GMAC_CONFIG_FULL_DUPLEX;
+ val = MVNETA_GMAC_INBAND_AN_ENABLE;
+
+ if (interface == PHY_INTERFACE_MODE_SGMII) {
+ /* SGMII mode receives the speed and duplex from PHY */
+ val |= MVNETA_GMAC_AN_SPEED_EN |
+ MVNETA_GMAC_AN_DUPLEX_EN;
+ } else {
+ /* 802.3z mode has fixed speed and duplex */
+ val |= MVNETA_GMAC_CONFIG_GMII_SPEED |
+ MVNETA_GMAC_CONFIG_FULL_DUPLEX;
+
+ /* The FLOW_CTRL_EN bit selects either the hardware
+ * automatically or the CONFIG_FLOW_CTRL manually
+ * controls the GMAC pause mode.
+ */
+ if (permit_pause_to_mac)
+ val |= MVNETA_GMAC_AN_FLOW_CTRL_EN;
+
+ /* Update the advertisement bits */
+ mask |= MVNETA_GMAC_ADVERT_SYM_FLOW_CTRL;
+ if (phylink_test(advertising, Pause))
+ val |= MVNETA_GMAC_ADVERT_SYM_FLOW_CTRL;
+ }
+ } else {
+ /* Phy or fixed speed - disable in-band AN modes */
+ val = 0;
+ }
+
+ old_an = an = mvreg_read(pp, MVNETA_GMAC_AUTONEG_CONFIG);
+ an = (an & ~mask) | val;
+ changed = old_an ^ an;
+ if (changed)
+ mvreg_write(pp, MVNETA_GMAC_AUTONEG_CONFIG, an);
+
+ /* We are only interested in the advertisement bits changing */
+ return !!(changed & MVNETA_GMAC_ADVERT_SYM_FLOW_CTRL);
+}
+
+static void mvneta_pcs_an_restart(struct phylink_pcs *pcs)
+{
+ struct mvneta_port *pp = mvneta_pcs_to_port(pcs);
u32 gmac_an = mvreg_read(pp, MVNETA_GMAC_AUTONEG_CONFIG);
mvreg_write(pp, MVNETA_GMAC_AUTONEG_CONFIG,
@@ -3907,6 +3962,47 @@ static void mvneta_mac_an_restart(struct phylink_config *config)
gmac_an & ~MVNETA_GMAC_INBAND_RESTART_AN);
}
+static const struct phylink_pcs_ops mvneta_phylink_pcs_ops = {
+ .pcs_validate = mvneta_pcs_validate,
+ .pcs_get_state = mvneta_pcs_get_state,
+ .pcs_config = mvneta_pcs_config,
+ .pcs_an_restart = mvneta_pcs_an_restart,
+};
+
+static int mvneta_mac_prepare(struct phylink_config *config, unsigned int mode,
+ phy_interface_t interface)
+{
+ struct net_device *ndev = to_net_dev(config->dev);
+ struct mvneta_port *pp = netdev_priv(ndev);
+ u32 val;
+
+ if (pp->phy_interface != interface ||
+ phylink_autoneg_inband(mode)) {
+ /* Force the link down when changing the interface or if in
+ * in-band mode. According to Armada 370 documentation, we
+ * can only change the port mode and in-band enable when the
+ * link is down.
+ */
+ val = mvreg_read(pp, MVNETA_GMAC_AUTONEG_CONFIG);
+ val &= ~MVNETA_GMAC_FORCE_LINK_PASS;
+ val |= MVNETA_GMAC_FORCE_LINK_DOWN;
+ mvreg_write(pp, MVNETA_GMAC_AUTONEG_CONFIG, val);
+ }
+
+ if (pp->phy_interface != interface)
+ WARN_ON(phy_power_off(pp->comphy));
+
+ /* Enable the 1ms clock */
+ if (phylink_autoneg_inband(mode)) {
+ unsigned long rate = clk_get_rate(pp->clk);
+
+ mvreg_write(pp, MVNETA_GMAC_CLOCK_DIVIDER,
+ MVNETA_GMAC_1MS_CLOCK_ENABLE | (rate / 1000));
+ }
+
+ return 0;
+}
+
static void mvneta_mac_config(struct phylink_config *config, unsigned int mode,
const struct phylink_link_state *state)
{
@@ -3915,20 +4011,11 @@ static void mvneta_mac_config(struct phylink_config *config, unsigned int mode,
u32 new_ctrl0, gmac_ctrl0 = mvreg_read(pp, MVNETA_GMAC_CTRL_0);
u32 new_ctrl2, gmac_ctrl2 = mvreg_read(pp, MVNETA_GMAC_CTRL_2);
u32 new_ctrl4, gmac_ctrl4 = mvreg_read(pp, MVNETA_GMAC_CTRL_4);
- u32 new_clk, gmac_clk = mvreg_read(pp, MVNETA_GMAC_CLOCK_DIVIDER);
- u32 new_an, gmac_an = mvreg_read(pp, MVNETA_GMAC_AUTONEG_CONFIG);
new_ctrl0 = gmac_ctrl0 & ~MVNETA_GMAC0_PORT_1000BASE_X;
new_ctrl2 = gmac_ctrl2 & ~(MVNETA_GMAC2_INBAND_AN_ENABLE |
MVNETA_GMAC2_PORT_RESET);
new_ctrl4 = gmac_ctrl4 & ~(MVNETA_GMAC4_SHORT_PREAMBLE_ENABLE);
- new_clk = gmac_clk & ~MVNETA_GMAC_1MS_CLOCK_ENABLE;
- new_an = gmac_an & ~(MVNETA_GMAC_INBAND_AN_ENABLE |
- MVNETA_GMAC_INBAND_RESTART_AN |
- MVNETA_GMAC_AN_SPEED_EN |
- MVNETA_GMAC_ADVERT_SYM_FLOW_CTRL |
- MVNETA_GMAC_AN_FLOW_CTRL_EN |
- MVNETA_GMAC_AN_DUPLEX_EN);
/* Even though it might look weird, when we're configured in
* SGMII or QSGMII mode, the RGMII bit needs to be set.
@@ -3940,9 +4027,6 @@ static void mvneta_mac_config(struct phylink_config *config, unsigned int mode,
phy_interface_mode_is_8023z(state->interface))
new_ctrl2 |= MVNETA_GMAC2_PCS_ENABLE;
- if (phylink_test(state->advertising, Pause))
- new_an |= MVNETA_GMAC_ADVERT_SYM_FLOW_CTRL;
-
if (!phylink_autoneg_inband(mode)) {
/* Phy or fixed speed - nothing to do, leave the
* configured speed, duplex and flow control as-is.
@@ -3950,66 +4034,23 @@ static void mvneta_mac_config(struct phylink_config *config, unsigned int mode,
} else if (state->interface == PHY_INTERFACE_MODE_SGMII) {
/* SGMII mode receives the state from the PHY */
new_ctrl2 |= MVNETA_GMAC2_INBAND_AN_ENABLE;
- new_clk |= MVNETA_GMAC_1MS_CLOCK_ENABLE;
- new_an = (new_an & ~(MVNETA_GMAC_FORCE_LINK_DOWN |
- MVNETA_GMAC_FORCE_LINK_PASS |
- MVNETA_GMAC_CONFIG_MII_SPEED |
- MVNETA_GMAC_CONFIG_GMII_SPEED |
- MVNETA_GMAC_CONFIG_FULL_DUPLEX)) |
- MVNETA_GMAC_INBAND_AN_ENABLE |
- MVNETA_GMAC_AN_SPEED_EN |
- MVNETA_GMAC_AN_DUPLEX_EN;
} else {
/* 802.3z negotiation - only 1000base-X */
new_ctrl0 |= MVNETA_GMAC0_PORT_1000BASE_X;
- new_clk |= MVNETA_GMAC_1MS_CLOCK_ENABLE;
- new_an = (new_an & ~(MVNETA_GMAC_FORCE_LINK_DOWN |
- MVNETA_GMAC_FORCE_LINK_PASS |
- MVNETA_GMAC_CONFIG_MII_SPEED)) |
- MVNETA_GMAC_INBAND_AN_ENABLE |
- MVNETA_GMAC_CONFIG_GMII_SPEED |
- /* The MAC only supports FD mode */
- MVNETA_GMAC_CONFIG_FULL_DUPLEX;
-
- if (state->pause & MLO_PAUSE_AN && state->an_enabled)
- new_an |= MVNETA_GMAC_AN_FLOW_CTRL_EN;
}
- /* Armada 370 documentation says we can only change the port mode
- * and in-band enable when the link is down, so force it down
- * while making these changes. We also do this for GMAC_CTRL2
- */
- if ((new_ctrl0 ^ gmac_ctrl0) & MVNETA_GMAC0_PORT_1000BASE_X ||
- (new_ctrl2 ^ gmac_ctrl2) & MVNETA_GMAC2_INBAND_AN_ENABLE ||
- (new_an ^ gmac_an) & MVNETA_GMAC_INBAND_AN_ENABLE) {
- mvreg_write(pp, MVNETA_GMAC_AUTONEG_CONFIG,
- (gmac_an & ~MVNETA_GMAC_FORCE_LINK_PASS) |
- MVNETA_GMAC_FORCE_LINK_DOWN);
- }
-
-
/* When at 2.5G, the link partner can send frames with shortened
* preambles.
*/
if (state->interface == PHY_INTERFACE_MODE_2500BASEX)
new_ctrl4 |= MVNETA_GMAC4_SHORT_PREAMBLE_ENABLE;
- if (pp->phy_interface != state->interface) {
- if (pp->comphy)
- WARN_ON(phy_power_off(pp->comphy));
- WARN_ON(mvneta_config_interface(pp, state->interface));
- }
-
if (new_ctrl0 != gmac_ctrl0)
mvreg_write(pp, MVNETA_GMAC_CTRL_0, new_ctrl0);
if (new_ctrl2 != gmac_ctrl2)
mvreg_write(pp, MVNETA_GMAC_CTRL_2, new_ctrl2);
if (new_ctrl4 != gmac_ctrl4)
mvreg_write(pp, MVNETA_GMAC_CTRL_4, new_ctrl4);
- if (new_clk != gmac_clk)
- mvreg_write(pp, MVNETA_GMAC_CLOCK_DIVIDER, new_clk);
- if (new_an != gmac_an)
- mvreg_write(pp, MVNETA_GMAC_AUTONEG_CONFIG, new_an);
if (gmac_ctrl2 & MVNETA_GMAC2_PORT_RESET) {
while ((mvreg_read(pp, MVNETA_GMAC_CTRL_2) &
@@ -4018,6 +4059,36 @@ static void mvneta_mac_config(struct phylink_config *config, unsigned int mode,
}
}
+static int mvneta_mac_finish(struct phylink_config *config, unsigned int mode,
+ phy_interface_t interface)
+{
+ struct net_device *ndev = to_net_dev(config->dev);
+ struct mvneta_port *pp = netdev_priv(ndev);
+ u32 val, clk;
+
+ /* Disable 1ms clock if not in in-band mode */
+ if (!phylink_autoneg_inband(mode)) {
+ clk = mvreg_read(pp, MVNETA_GMAC_CLOCK_DIVIDER);
+ clk &= ~MVNETA_GMAC_1MS_CLOCK_ENABLE;
+ mvreg_write(pp, MVNETA_GMAC_CLOCK_DIVIDER, clk);
+ }
+
+ if (pp->phy_interface != interface)
+ /* Enable the Serdes PHY */
+ WARN_ON(mvneta_config_interface(pp, interface));
+
+ /* Allow the link to come up if in in-band mode, otherwise the
+ * link is forced via mac_link_down()/mac_link_up()
+ */
+ if (phylink_autoneg_inband(mode)) {
+ val = mvreg_read(pp, MVNETA_GMAC_AUTONEG_CONFIG);
+ val &= ~MVNETA_GMAC_FORCE_LINK_DOWN;
+ mvreg_write(pp, MVNETA_GMAC_AUTONEG_CONFIG, val);
+ }
+
+ return 0;
+}
+
static void mvneta_set_eee(struct mvneta_port *pp, bool enable)
{
u32 lpi_ctl1;
@@ -4104,10 +4175,10 @@ static void mvneta_mac_link_up(struct phylink_config *config,
}
static const struct phylink_mac_ops mvneta_phylink_ops = {
- .validate = mvneta_validate,
- .mac_pcs_get_state = mvneta_mac_pcs_get_state,
- .mac_an_restart = mvneta_mac_an_restart,
+ .validate = phylink_generic_validate,
+ .mac_prepare = mvneta_mac_prepare,
.mac_config = mvneta_mac_config,
+ .mac_finish = mvneta_mac_finish,
.mac_link_down = mvneta_mac_link_down,
.mac_link_up = mvneta_mac_link_up,
};
@@ -4539,8 +4610,11 @@ static void mvneta_ethtool_get_drvinfo(struct net_device *dev,
}
-static void mvneta_ethtool_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+static void
+mvneta_ethtool_get_ringparam(struct net_device *netdev,
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct mvneta_port *pp = netdev_priv(netdev);
@@ -4550,8 +4624,11 @@ static void mvneta_ethtool_get_ringparam(struct net_device *netdev,
ring->tx_pending = pp->tx_ring_size;
}
-static int mvneta_ethtool_set_ringparam(struct net_device *dev,
- struct ethtool_ringparam *ring)
+static int
+mvneta_ethtool_set_ringparam(struct net_device *dev,
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct mvneta_port *pp = netdev_priv(dev);
@@ -4919,43 +4996,144 @@ static void mvneta_clear_rx_prio_map(struct mvneta_port *pp)
mvreg_write(pp, MVNETA_VLAN_PRIO_TO_RXQ, 0);
}
-static void mvneta_setup_rx_prio_map(struct mvneta_port *pp)
+static void mvneta_map_vlan_prio_to_rxq(struct mvneta_port *pp, u8 pri, u8 rxq)
{
- u32 val = 0;
- int i;
+ u32 val = mvreg_read(pp, MVNETA_VLAN_PRIO_TO_RXQ);
- for (i = 0; i < rxq_number; i++)
- val |= MVNETA_VLAN_PRIO_RXQ_MAP(i, pp->prio_tc_map[i]);
+ val &= ~MVNETA_VLAN_PRIO_RXQ_MAP(pri, 0x7);
+ val |= MVNETA_VLAN_PRIO_RXQ_MAP(pri, rxq);
mvreg_write(pp, MVNETA_VLAN_PRIO_TO_RXQ, val);
}
+static int mvneta_enable_per_queue_rate_limit(struct mvneta_port *pp)
+{
+ unsigned long core_clk_rate;
+ u32 refill_cycles;
+ u32 val;
+
+ core_clk_rate = clk_get_rate(pp->clk);
+ if (!core_clk_rate)
+ return -EINVAL;
+
+ refill_cycles = MVNETA_TXQ_BUCKET_REFILL_BASE_PERIOD_NS /
+ (NSEC_PER_SEC / core_clk_rate);
+
+ if (refill_cycles > MVNETA_REFILL_MAX_NUM_CLK)
+ return -EINVAL;
+
+ /* Enable bw limit algorithm version 3 */
+ val = mvreg_read(pp, MVNETA_TXQ_CMD1_REG);
+ val &= ~(MVNETA_TXQ_CMD1_BW_LIM_SEL_V1 | MVNETA_TXQ_CMD1_BW_LIM_EN);
+ mvreg_write(pp, MVNETA_TXQ_CMD1_REG, val);
+
+ /* Set the base refill rate */
+ mvreg_write(pp, MVNETA_REFILL_NUM_CLK_REG, refill_cycles);
+
+ return 0;
+}
+
+static void mvneta_disable_per_queue_rate_limit(struct mvneta_port *pp)
+{
+ u32 val = mvreg_read(pp, MVNETA_TXQ_CMD1_REG);
+
+ val |= (MVNETA_TXQ_CMD1_BW_LIM_SEL_V1 | MVNETA_TXQ_CMD1_BW_LIM_EN);
+ mvreg_write(pp, MVNETA_TXQ_CMD1_REG, val);
+}
+
+static int mvneta_setup_queue_rates(struct mvneta_port *pp, int queue,
+ u64 min_rate, u64 max_rate)
+{
+ u32 refill_val, rem;
+ u32 val = 0;
+
+ /* Convert to from Bps to bps */
+ max_rate *= 8;
+
+ if (min_rate)
+ return -EINVAL;
+
+ refill_val = div_u64_rem(max_rate, MVNETA_TXQ_RATE_LIMIT_RESOLUTION,
+ &rem);
+
+ if (rem || !refill_val ||
+ refill_val > MVNETA_TXQ_BUCKET_REFILL_VALUE_MAX)
+ return -EINVAL;
+
+ val = refill_val;
+ val |= (MVNETA_TXQ_BUCKET_REFILL_PERIOD <<
+ MVNETA_TXQ_BUCKET_REFILL_PERIOD_SHIFT);
+
+ mvreg_write(pp, MVNETA_TXQ_BUCKET_REFILL_REG(queue), val);
+
+ return 0;
+}
+
static int mvneta_setup_mqprio(struct net_device *dev,
- struct tc_mqprio_qopt *qopt)
+ struct tc_mqprio_qopt_offload *mqprio)
{
struct mvneta_port *pp = netdev_priv(dev);
+ int rxq, txq, tc, ret;
u8 num_tc;
- int i;
- qopt->hw = TC_MQPRIO_HW_OFFLOAD_TCS;
- num_tc = qopt->num_tc;
+ if (mqprio->qopt.hw != TC_MQPRIO_HW_OFFLOAD_TCS)
+ return 0;
+
+ num_tc = mqprio->qopt.num_tc;
if (num_tc > rxq_number)
return -EINVAL;
+ mvneta_clear_rx_prio_map(pp);
+
if (!num_tc) {
- mvneta_clear_rx_prio_map(pp);
+ mvneta_disable_per_queue_rate_limit(pp);
netdev_reset_tc(dev);
return 0;
}
- memcpy(pp->prio_tc_map, qopt->prio_tc_map, sizeof(pp->prio_tc_map));
+ netdev_set_num_tc(dev, mqprio->qopt.num_tc);
- mvneta_setup_rx_prio_map(pp);
+ for (tc = 0; tc < mqprio->qopt.num_tc; tc++) {
+ netdev_set_tc_queue(dev, tc, mqprio->qopt.count[tc],
+ mqprio->qopt.offset[tc]);
- netdev_set_num_tc(dev, qopt->num_tc);
- for (i = 0; i < qopt->num_tc; i++)
- netdev_set_tc_queue(dev, i, qopt->count[i], qopt->offset[i]);
+ for (rxq = mqprio->qopt.offset[tc];
+ rxq < mqprio->qopt.count[tc] + mqprio->qopt.offset[tc];
+ rxq++) {
+ if (rxq >= rxq_number)
+ return -EINVAL;
+
+ mvneta_map_vlan_prio_to_rxq(pp, tc, rxq);
+ }
+ }
+
+ if (mqprio->shaper != TC_MQPRIO_SHAPER_BW_RATE) {
+ mvneta_disable_per_queue_rate_limit(pp);
+ return 0;
+ }
+
+ if (mqprio->qopt.num_tc > txq_number)
+ return -EINVAL;
+
+ ret = mvneta_enable_per_queue_rate_limit(pp);
+ if (ret)
+ return ret;
+
+ for (tc = 0; tc < mqprio->qopt.num_tc; tc++) {
+ for (txq = mqprio->qopt.offset[tc];
+ txq < mqprio->qopt.count[tc] + mqprio->qopt.offset[tc];
+ txq++) {
+ if (txq >= txq_number)
+ return -EINVAL;
+
+ ret = mvneta_setup_queue_rates(pp, txq,
+ mqprio->min_rate[tc],
+ mqprio->max_rate[tc]);
+ if (ret)
+ return ret;
+ }
+ }
return 0;
}
@@ -5166,6 +5344,9 @@ static int mvneta_probe(struct platform_device *pdev)
pp->phylink_config.dev = &dev->dev;
pp->phylink_config.type = PHYLINK_NETDEV;
+ pp->phylink_config.mac_capabilities = MAC_SYM_PAUSE | MAC_10 |
+ MAC_100 | MAC_1000FD | MAC_2500FD;
+
phy_interface_set_rgmii(pp->phylink_config.supported_interfaces);
__set_bit(PHY_INTERFACE_MODE_QSGMII,
pp->phylink_config.supported_interfaces);
@@ -5237,6 +5418,9 @@ static int mvneta_probe(struct platform_device *pdev)
goto err_clk;
}
+ pp->phylink_pcs.ops = &mvneta_phylink_pcs_ops;
+ phylink_set_pcs(phylink, &pp->phylink_pcs);
+
/* Alloc per-cpu port structure */
pp->ports = alloc_percpu(struct mvneta_pcpu_port);
if (!pp->ports) {
@@ -5355,7 +5539,7 @@ static int mvneta_probe(struct platform_device *pdev)
dev->hw_features |= dev->features;
dev->vlan_features |= dev->features;
dev->priv_flags |= IFF_LIVE_ADDR_CHANGE;
- dev->gso_max_segs = MVNETA_MAX_TSO_SEGS;
+ netif_set_gso_max_segs(dev, MVNETA_MAX_TSO_SEGS);
/* MTU range: 68 - 9676 */
dev->min_mtu = ETH_MIN_MTU;
diff --git a/drivers/net/ethernet/marvell/mvpp2/mvpp2.h b/drivers/net/ethernet/marvell/mvpp2/mvpp2.h
index cf8acab..ad73a48 100644
--- a/drivers/net/ethernet/marvell/mvpp2/mvpp2.h
+++ b/drivers/net/ethernet/marvell/mvpp2/mvpp2.h
@@ -1239,7 +1239,8 @@ struct mvpp2_port {
phy_interface_t phy_interface;
struct phylink *phylink;
struct phylink_config phylink_config;
- struct phylink_pcs phylink_pcs;
+ struct phylink_pcs pcs_gmac;
+ struct phylink_pcs pcs_xlg;
struct phy *comphy;
struct mvpp2_bm_pool *pool_long;
diff --git a/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c b/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c
index 6da8a59..7cdbf8b 100644
--- a/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c
+++ b/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c
@@ -1488,6 +1488,7 @@ static bool mvpp2_port_supports_rgmii(struct mvpp2_port *port)
static bool mvpp2_is_xlg(phy_interface_t interface)
{
return interface == PHY_INTERFACE_MODE_10GBASER ||
+ interface == PHY_INTERFACE_MODE_5GBASER ||
interface == PHY_INTERFACE_MODE_XAUI;
}
@@ -1627,6 +1628,7 @@ static int mvpp22_gop_init(struct mvpp2_port *port, phy_interface_t interface)
case PHY_INTERFACE_MODE_2500BASEX:
mvpp22_gop_init_sgmii(port);
break;
+ case PHY_INTERFACE_MODE_5GBASER:
case PHY_INTERFACE_MODE_10GBASER:
if (!mvpp2_port_supports_xlg(port))
goto invalid_conf;
@@ -2186,6 +2188,7 @@ static void mvpp22_pcs_reset_deassert(struct mvpp2_port *port,
xpcs = priv->iface_base + MVPP22_XPCS_BASE(port->gop_id);
switch (interface) {
+ case PHY_INTERFACE_MODE_5GBASER:
case PHY_INTERFACE_MODE_10GBASER:
val = readl(mpcs + MVPP22_MPCS_CLK_RESET);
val |= MAC_CLK_RESET_MAC | MAC_CLK_RESET_SD_RX |
@@ -3820,7 +3823,7 @@ mvpp2_run_xdp(struct mvpp2_port *port, struct bpf_prog *prog,
}
break;
default:
- bpf_warn_invalid_xdp_action(act);
+ bpf_warn_invalid_xdp_action(port->dev, prog, act);
fallthrough;
case XDP_ABORTED:
trace_xdp_exception(port->dev, prog, act);
@@ -5139,9 +5142,6 @@ static int mvpp2_set_ts_config(struct mvpp2_port *port, struct ifreq *ifr)
if (copy_from_user(&config, ifr->ifr_data, sizeof(config)))
return -EFAULT;
- if (config.flags)
- return -EINVAL;
-
if (config.tx_type != HWTSTAMP_TX_OFF &&
config.tx_type != HWTSTAMP_TX_ON)
return -ERANGE;
@@ -5433,8 +5433,11 @@ static void mvpp2_ethtool_get_drvinfo(struct net_device *dev,
sizeof(drvinfo->bus_info));
}
-static void mvpp2_ethtool_get_ringparam(struct net_device *dev,
- struct ethtool_ringparam *ring)
+static void
+mvpp2_ethtool_get_ringparam(struct net_device *dev,
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct mvpp2_port *port = netdev_priv(dev);
@@ -5444,8 +5447,11 @@ static void mvpp2_ethtool_get_ringparam(struct net_device *dev,
ring->tx_pending = port->tx_ring_size;
}
-static int mvpp2_ethtool_set_ringparam(struct net_device *dev,
- struct ethtool_ringparam *ring)
+static int
+mvpp2_ethtool_set_ringparam(struct net_device *dev,
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct mvpp2_port *port = netdev_priv(dev);
u16 prev_rx_ring_size = port->rx_ring_size;
@@ -6109,18 +6115,26 @@ static struct mvpp2_port *mvpp2_phylink_to_port(struct phylink_config *config)
return container_of(config, struct mvpp2_port, phylink_config);
}
-static struct mvpp2_port *mvpp2_pcs_to_port(struct phylink_pcs *pcs)
+static struct mvpp2_port *mvpp2_pcs_xlg_to_port(struct phylink_pcs *pcs)
{
- return container_of(pcs, struct mvpp2_port, phylink_pcs);
+ return container_of(pcs, struct mvpp2_port, pcs_xlg);
+}
+
+static struct mvpp2_port *mvpp2_pcs_gmac_to_port(struct phylink_pcs *pcs)
+{
+ return container_of(pcs, struct mvpp2_port, pcs_gmac);
}
static void mvpp2_xlg_pcs_get_state(struct phylink_pcs *pcs,
struct phylink_link_state *state)
{
- struct mvpp2_port *port = mvpp2_pcs_to_port(pcs);
+ struct mvpp2_port *port = mvpp2_pcs_xlg_to_port(pcs);
u32 val;
- state->speed = SPEED_10000;
+ if (port->phy_interface == PHY_INTERFACE_MODE_5GBASER)
+ state->speed = SPEED_5000;
+ else
+ state->speed = SPEED_10000;
state->duplex = 1;
state->an_complete = 1;
@@ -6149,10 +6163,25 @@ static const struct phylink_pcs_ops mvpp2_phylink_xlg_pcs_ops = {
.pcs_config = mvpp2_xlg_pcs_config,
};
+static int mvpp2_gmac_pcs_validate(struct phylink_pcs *pcs,
+ unsigned long *supported,
+ const struct phylink_link_state *state)
+{
+ /* When in 802.3z mode, we must have AN enabled:
+ * Bit 2 Field InBandAnEn In-band Auto-Negotiation enable. ...
+ * When <PortType> = 1 (1000BASE-X) this field must be set to 1.
+ */
+ if (phy_interface_mode_is_8023z(state->interface) &&
+ !phylink_test(state->advertising, Autoneg))
+ return -EINVAL;
+
+ return 0;
+}
+
static void mvpp2_gmac_pcs_get_state(struct phylink_pcs *pcs,
struct phylink_link_state *state)
{
- struct mvpp2_port *port = mvpp2_pcs_to_port(pcs);
+ struct mvpp2_port *port = mvpp2_pcs_gmac_to_port(pcs);
u32 val;
val = readl(port->base + MVPP2_GMAC_STATUS0);
@@ -6189,7 +6218,7 @@ static int mvpp2_gmac_pcs_config(struct phylink_pcs *pcs, unsigned int mode,
const unsigned long *advertising,
bool permit_pause_to_mac)
{
- struct mvpp2_port *port = mvpp2_pcs_to_port(pcs);
+ struct mvpp2_port *port = mvpp2_pcs_gmac_to_port(pcs);
u32 mask, val, an, old_an, changed;
mask = MVPP2_GMAC_IN_BAND_AUTONEG_BYPASS |
@@ -6243,7 +6272,7 @@ static int mvpp2_gmac_pcs_config(struct phylink_pcs *pcs, unsigned int mode,
static void mvpp2_gmac_pcs_an_restart(struct phylink_pcs *pcs)
{
- struct mvpp2_port *port = mvpp2_pcs_to_port(pcs);
+ struct mvpp2_port *port = mvpp2_pcs_gmac_to_port(pcs);
u32 val = readl(port->base + MVPP2_GMAC_AUTONEG_CONFIG);
writel(val | MVPP2_GMAC_IN_BAND_RESTART_AN,
@@ -6253,78 +6282,12 @@ static void mvpp2_gmac_pcs_an_restart(struct phylink_pcs *pcs)
}
static const struct phylink_pcs_ops mvpp2_phylink_gmac_pcs_ops = {
+ .pcs_validate = mvpp2_gmac_pcs_validate,
.pcs_get_state = mvpp2_gmac_pcs_get_state,
.pcs_config = mvpp2_gmac_pcs_config,
.pcs_an_restart = mvpp2_gmac_pcs_an_restart,
};
-static void mvpp2_phylink_validate(struct phylink_config *config,
- unsigned long *supported,
- struct phylink_link_state *state)
-{
- struct mvpp2_port *port = mvpp2_phylink_to_port(config);
- __ETHTOOL_DECLARE_LINK_MODE_MASK(mask) = { 0, };
-
- /* When in 802.3z mode, we must have AN enabled:
- * Bit 2 Field InBandAnEn In-band Auto-Negotiation enable. ...
- * When <PortType> = 1 (1000BASE-X) this field must be set to 1.
- */
- if (phy_interface_mode_is_8023z(state->interface) &&
- !phylink_test(state->advertising, Autoneg))
- goto empty_set;
-
- phylink_set(mask, Autoneg);
- phylink_set_port_modes(mask);
-
- if (port->priv->global_tx_fc) {
- phylink_set(mask, Pause);
- phylink_set(mask, Asym_Pause);
- }
-
- switch (state->interface) {
- case PHY_INTERFACE_MODE_10GBASER:
- case PHY_INTERFACE_MODE_XAUI:
- if (mvpp2_port_supports_xlg(port)) {
- phylink_set_10g_modes(mask);
- phylink_set(mask, 10000baseKR_Full);
- }
- break;
-
- case PHY_INTERFACE_MODE_RGMII:
- case PHY_INTERFACE_MODE_RGMII_ID:
- case PHY_INTERFACE_MODE_RGMII_RXID:
- case PHY_INTERFACE_MODE_RGMII_TXID:
- case PHY_INTERFACE_MODE_SGMII:
- phylink_set(mask, 10baseT_Half);
- phylink_set(mask, 10baseT_Full);
- phylink_set(mask, 100baseT_Half);
- phylink_set(mask, 100baseT_Full);
- phylink_set(mask, 1000baseT_Full);
- phylink_set(mask, 1000baseX_Full);
- break;
-
- case PHY_INTERFACE_MODE_1000BASEX:
- phylink_set(mask, 1000baseT_Full);
- phylink_set(mask, 1000baseX_Full);
- break;
-
- case PHY_INTERFACE_MODE_2500BASEX:
- phylink_set(mask, 2500baseT_Full);
- phylink_set(mask, 2500baseX_Full);
- break;
-
- default:
- goto empty_set;
- }
-
- linkmode_and(supported, supported, mask);
- linkmode_and(state->advertising, state->advertising, mask);
- return;
-
-empty_set:
- linkmode_zero(supported);
-}
-
static void mvpp2_xlg_config(struct mvpp2_port *port, unsigned int mode,
const struct phylink_link_state *state)
{
@@ -6404,8 +6367,23 @@ static void mvpp2_gmac_config(struct mvpp2_port *port, unsigned int mode,
writel(ctrl4, port->base + MVPP22_GMAC_CTRL_4_REG);
}
-static int mvpp2__mac_prepare(struct phylink_config *config, unsigned int mode,
- phy_interface_t interface)
+static struct phylink_pcs *mvpp2_select_pcs(struct phylink_config *config,
+ phy_interface_t interface)
+{
+ struct mvpp2_port *port = mvpp2_phylink_to_port(config);
+
+ /* Select the appropriate PCS operations depending on the
+ * configured interface mode. We will only switch to a mode
+ * that the validate() checks have already passed.
+ */
+ if (mvpp2_is_xlg(interface))
+ return &port->pcs_xlg;
+ else
+ return &port->pcs_gmac;
+}
+
+static int mvpp2_mac_prepare(struct phylink_config *config, unsigned int mode,
+ phy_interface_t interface)
{
struct mvpp2_port *port = mvpp2_phylink_to_port(config);
@@ -6454,31 +6432,9 @@ static int mvpp2__mac_prepare(struct phylink_config *config, unsigned int mode,
}
}
- /* Select the appropriate PCS operations depending on the
- * configured interface mode. We will only switch to a mode
- * that the validate() checks have already passed.
- */
- if (mvpp2_is_xlg(interface))
- port->phylink_pcs.ops = &mvpp2_phylink_xlg_pcs_ops;
- else
- port->phylink_pcs.ops = &mvpp2_phylink_gmac_pcs_ops;
-
return 0;
}
-static int mvpp2_mac_prepare(struct phylink_config *config, unsigned int mode,
- phy_interface_t interface)
-{
- struct mvpp2_port *port = mvpp2_phylink_to_port(config);
- int ret;
-
- ret = mvpp2__mac_prepare(config, mode, interface);
- if (ret == 0)
- phylink_set_pcs(port->phylink, &port->phylink_pcs);
-
- return ret;
-}
-
static void mvpp2_mac_config(struct phylink_config *config, unsigned int mode,
const struct phylink_link_state *state)
{
@@ -6649,7 +6605,8 @@ static void mvpp2_mac_link_down(struct phylink_config *config,
}
static const struct phylink_mac_ops mvpp2_phylink_ops = {
- .validate = mvpp2_phylink_validate,
+ .validate = phylink_generic_validate,
+ .mac_select_pcs = mvpp2_select_pcs,
.mac_prepare = mvpp2_mac_prepare,
.mac_config = mvpp2_mac_config,
.mac_finish = mvpp2_mac_finish,
@@ -6667,12 +6624,15 @@ static void mvpp2_acpi_start(struct mvpp2_port *port)
struct phylink_link_state state = {
.interface = port->phy_interface,
};
- mvpp2__mac_prepare(&port->phylink_config, MLO_AN_INBAND,
- port->phy_interface);
+ struct phylink_pcs *pcs;
+
+ pcs = mvpp2_select_pcs(&port->phylink_config, port->phy_interface);
+
+ mvpp2_mac_prepare(&port->phylink_config, MLO_AN_INBAND,
+ port->phy_interface);
mvpp2_mac_config(&port->phylink_config, MLO_AN_INBAND, &state);
- port->phylink_pcs.ops->pcs_config(&port->phylink_pcs, MLO_AN_INBAND,
- port->phy_interface,
- state.advertising, false);
+ pcs->ops->pcs_config(pcs, MLO_AN_INBAND, port->phy_interface,
+ state.advertising, false);
mvpp2_mac_finish(&port->phylink_config, MLO_AN_INBAND,
port->phy_interface);
mvpp2_mac_link_up(&port->phylink_config, NULL,
@@ -6901,7 +6861,7 @@ static int mvpp2_port_probe(struct platform_device *pdev,
mvpp2_set_hw_csum(port, port->pool_long->id);
dev->vlan_features |= features;
- dev->gso_max_segs = MVPP2_MAX_TSO_SEGS;
+ netif_set_gso_max_segs(dev, MVPP2_MAX_TSO_SEGS);
dev->priv_flags |= IFF_UNICAST_FLT;
/* MTU range: 68 - 9704 */
@@ -6913,12 +6873,44 @@ static int mvpp2_port_probe(struct platform_device *pdev,
if (!mvpp2_use_acpi_compat_mode(port_fwnode)) {
port->phylink_config.dev = &dev->dev;
port->phylink_config.type = PHYLINK_NETDEV;
+ port->phylink_config.mac_capabilities =
+ MAC_2500FD | MAC_1000FD | MAC_100 | MAC_10;
+
+ if (port->priv->global_tx_fc)
+ port->phylink_config.mac_capabilities |=
+ MAC_SYM_PAUSE | MAC_ASYM_PAUSE;
if (mvpp2_port_supports_xlg(port)) {
- __set_bit(PHY_INTERFACE_MODE_10GBASER,
- port->phylink_config.supported_interfaces);
- __set_bit(PHY_INTERFACE_MODE_XAUI,
- port->phylink_config.supported_interfaces);
+ /* If a COMPHY is present, we can support any of
+ * the serdes modes and switch between them.
+ */
+ if (comphy) {
+ __set_bit(PHY_INTERFACE_MODE_5GBASER,
+ port->phylink_config.supported_interfaces);
+ __set_bit(PHY_INTERFACE_MODE_10GBASER,
+ port->phylink_config.supported_interfaces);
+ __set_bit(PHY_INTERFACE_MODE_XAUI,
+ port->phylink_config.supported_interfaces);
+ } else if (phy_mode == PHY_INTERFACE_MODE_5GBASER) {
+ __set_bit(PHY_INTERFACE_MODE_5GBASER,
+ port->phylink_config.supported_interfaces);
+ } else if (phy_mode == PHY_INTERFACE_MODE_10GBASER) {
+ __set_bit(PHY_INTERFACE_MODE_10GBASER,
+ port->phylink_config.supported_interfaces);
+ } else if (phy_mode == PHY_INTERFACE_MODE_XAUI) {
+ __set_bit(PHY_INTERFACE_MODE_XAUI,
+ port->phylink_config.supported_interfaces);
+ }
+
+ if (comphy)
+ port->phylink_config.mac_capabilities |=
+ MAC_10000FD | MAC_5000FD;
+ else if (phy_mode == PHY_INTERFACE_MODE_5GBASER)
+ port->phylink_config.mac_capabilities |=
+ MAC_5000FD;
+ else
+ port->phylink_config.mac_capabilities |=
+ MAC_10000FD;
}
if (mvpp2_port_supports_rgmii(port))
@@ -6948,6 +6940,9 @@ static int mvpp2_port_probe(struct platform_device *pdev,
port->phylink_config.supported_interfaces);
}
+ port->pcs_gmac.ops = &mvpp2_phylink_gmac_pcs_ops;
+ port->pcs_xlg.ops = &mvpp2_phylink_xlg_pcs_ops;
+
phylink = phylink_create(&port->phylink_config, port_fwnode,
phy_mode, &mvpp2_phylink_ops);
if (IS_ERR(phylink)) {
diff --git a/drivers/net/ethernet/marvell/octeontx2/af/ptp.c b/drivers/net/ethernet/marvell/octeontx2/af/ptp.c
index d6321de..e682b7b 100644
--- a/drivers/net/ethernet/marvell/octeontx2/af/ptp.c
+++ b/drivers/net/ethernet/marvell/octeontx2/af/ptp.c
@@ -60,6 +60,8 @@ struct ptp *ptp_get(void)
/* Check driver is bound to PTP block */
if (!ptp)
ptp = ERR_PTR(-EPROBE_DEFER);
+ else
+ pci_dev_get(ptp->pdev);
return ptp;
}
diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_cpt.c b/drivers/net/ethernet/marvell/octeontx2/af/rvu_cpt.c
index 45357de..a73a801 100644
--- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_cpt.c
+++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_cpt.c
@@ -172,14 +172,13 @@ static int cpt_10k_register_interrupts(struct rvu_block *block, int off)
{
struct rvu *rvu = block->rvu;
int blkaddr = block->addr;
- char irq_name[16];
int i, ret;
for (i = CPT_10K_AF_INT_VEC_FLT0; i < CPT_10K_AF_INT_VEC_RVU; i++) {
- snprintf(irq_name, sizeof(irq_name), "CPTAF FLT%d", i);
+ sprintf(&rvu->irq_name[(off + i) * NAME_SIZE], "CPTAF FLT%d", i);
ret = rvu_cpt_do_register_interrupt(block, off + i,
rvu_cpt_af_flt_intr_handler,
- irq_name);
+ &rvu->irq_name[(off + i) * NAME_SIZE]);
if (ret)
goto err;
rvu_write64(rvu, blkaddr, CPT_AF_FLTX_INT_ENA_W1S(i), 0x1);
diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_devlink.c b/drivers/net/ethernet/marvell/octeontx2/af/rvu_devlink.c
index 70bacd3..d0ab8f2 100644
--- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_devlink.c
+++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_devlink.c
@@ -41,7 +41,7 @@ static bool rvu_common_request_irq(struct rvu *rvu, int offset,
struct rvu_devlink *rvu_dl = rvu->rvu_dl;
int rc;
- sprintf(&rvu->irq_name[offset * NAME_SIZE], name);
+ sprintf(&rvu->irq_name[offset * NAME_SIZE], "%s", name);
rc = request_irq(pci_irq_vector(rvu->pdev, offset), fn, 0,
&rvu->irq_name[offset * NAME_SIZE], rvu_dl);
if (rc)
diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_ethtool.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_ethtool.c
index 80d4ce6..d85db90 100644
--- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_ethtool.c
+++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_ethtool.c
@@ -360,7 +360,9 @@ static int otx2_set_pauseparam(struct net_device *netdev,
}
static void otx2_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct otx2_nic *pfvf = netdev_priv(netdev);
struct otx2_qset *qs = &pfvf->qset;
@@ -372,7 +374,9 @@ static void otx2_get_ringparam(struct net_device *netdev,
}
static int otx2_set_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct otx2_nic *pfvf = netdev_priv(netdev);
bool if_up = netif_running(netdev);
diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c
index 1e0d0c9c..6080ebd 100644
--- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c
+++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c
@@ -2002,10 +2002,6 @@ int otx2_config_hwtstamp(struct net_device *netdev, struct ifreq *ifr)
if (copy_from_user(&config, ifr->ifr_data, sizeof(config)))
return -EFAULT;
- /* reserved for future extensions */
- if (config.flags)
- return -EINVAL;
-
switch (config.tx_type) {
case HWTSTAMP_TX_OFF:
otx2_config_hw_tx_tstamp(pfvf, false);
@@ -2741,7 +2737,7 @@ static int otx2_probe(struct pci_dev *pdev, const struct pci_device_id *id)
netdev->hw_features |= NETIF_F_LOOPBACK | NETIF_F_RXALL;
- netdev->gso_max_segs = OTX2_MAX_GSO_SEGS;
+ netif_set_gso_max_segs(netdev, OTX2_MAX_GSO_SEGS);
netdev->watchdog_timeo = OTX2_TX_TIMEOUT;
netdev->netdev_ops = &otx2_netdev_ops;
diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c
index 0cc6353..7c4068c 100644
--- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c
+++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c
@@ -1198,7 +1198,7 @@ static bool otx2_xdp_rcv_pkt_handler(struct otx2_nic *pfvf,
put_page(page);
break;
default:
- bpf_warn_invalid_xdp_action(act);
+ bpf_warn_invalid_xdp_action(pfvf->netdev, prog, act);
break;
case XDP_ABORTED:
trace_xdp_exception(pfvf->netdev, prog, act);
diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c
index 78944ad..925b74e 100644
--- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c
+++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c
@@ -663,7 +663,7 @@ static int otx2vf_probe(struct pci_dev *pdev, const struct pci_device_id *id)
netdev->hw_features |= NETIF_F_NTUPLE;
netdev->hw_features |= NETIF_F_RXALL;
- netdev->gso_max_segs = OTX2_MAX_GSO_SEGS;
+ netif_set_gso_max_segs(netdev, OTX2_MAX_GSO_SEGS);
netdev->watchdog_timeo = OTX2_TX_TIMEOUT;
netdev->netdev_ops = &otx2vf_netdev_ops;
@@ -684,7 +684,7 @@ static int otx2vf_probe(struct pci_dev *pdev, const struct pci_device_id *id)
err = register_netdev(netdev);
if (err) {
dev_err(dev, "Failed to register netdevice\n");
- goto err_detach_rsrc;
+ goto err_ptp_destroy;
}
err = otx2_wq_init(vf);
@@ -709,6 +709,8 @@ static int otx2vf_probe(struct pci_dev *pdev, const struct pci_device_id *id)
err_unreg_netdev:
unregister_netdev(netdev);
+err_ptp_destroy:
+ otx2_ptp_destroy(vf);
err_detach_rsrc:
if (test_bit(CN10K_LMTST, &vf->hw.cap_flag))
qmem_free(vf->dev, vf->dync_lmt);
@@ -742,6 +744,7 @@ static void otx2vf_remove(struct pci_dev *pdev)
unregister_netdev(netdev);
if (vf->otx2_wq)
destroy_workqueue(vf->otx2_wq);
+ otx2_ptp_destroy(vf);
otx2vf_disable_mbox_intr(vf);
otx2_detach_resources(&vf->mbox);
if (test_bit(CN10K_LMTST, &vf->hw.cap_flag))
diff --git a/drivers/net/ethernet/marvell/prestera/Makefile b/drivers/net/ethernet/marvell/prestera/Makefile
index 0609df8..d395f41 100644
--- a/drivers/net/ethernet/marvell/prestera/Makefile
+++ b/drivers/net/ethernet/marvell/prestera/Makefile
@@ -3,6 +3,7 @@
prestera-objs := prestera_main.o prestera_hw.o prestera_dsa.o \
prestera_rxtx.o prestera_devlink.o prestera_ethtool.o \
prestera_switchdev.o prestera_acl.o prestera_flow.o \
- prestera_flower.o prestera_span.o
+ prestera_flower.o prestera_span.o prestera_counter.o \
+ prestera_router.o prestera_router_hw.o
obj-$(CONFIG_PRESTERA_PCI) += prestera_pci.o
diff --git a/drivers/net/ethernet/marvell/prestera/prestera.h b/drivers/net/ethernet/marvell/prestera/prestera.h
index 2a4c14c..a0a5a8e 100644
--- a/drivers/net/ethernet/marvell/prestera/prestera.h
+++ b/drivers/net/ethernet/marvell/prestera/prestera.h
@@ -225,6 +225,29 @@ struct prestera_event {
};
};
+enum prestera_if_type {
+ /* the interface is of port type (dev,port) */
+ PRESTERA_IF_PORT_E = 0,
+
+ /* the interface is of lag type (lag-id) */
+ PRESTERA_IF_LAG_E = 1,
+
+ /* the interface is of Vid type (vlan-id) */
+ PRESTERA_IF_VID_E = 3,
+};
+
+struct prestera_iface {
+ enum prestera_if_type type;
+ struct {
+ u32 hw_dev_num;
+ u32 port_num;
+ } dev_port;
+ u32 hw_dev_num;
+ u16 vr_id;
+ u16 lag_id;
+ u16 vlan_id;
+};
+
struct prestera_switchdev;
struct prestera_span;
struct prestera_rxtx;
@@ -247,11 +270,22 @@ struct prestera_switch {
u32 mtu_min;
u32 mtu_max;
u8 id;
+ struct prestera_router *router;
struct prestera_lag *lags;
+ struct prestera_counter *counter;
u8 lag_member_max;
u8 lag_max;
};
+struct prestera_router {
+ struct prestera_switch *sw;
+ struct list_head vr_list;
+ struct list_head rif_entry_list;
+ struct notifier_block inetaddr_nb;
+ struct notifier_block inetaddr_valid_nb;
+ bool aborted;
+};
+
struct prestera_rxtx_params {
bool use_sdma;
u32 map_addr;
@@ -279,6 +313,9 @@ struct prestera_port *prestera_port_find_by_hwid(struct prestera_switch *sw,
int prestera_port_autoneg_set(struct prestera_port *port, u64 link_modes);
+int prestera_router_init(struct prestera_switch *sw);
+void prestera_router_fini(struct prestera_switch *sw);
+
struct prestera_port *prestera_find_port(struct prestera_switch *sw, u32 id);
int prestera_port_cfg_mac_read(struct prestera_port *port,
@@ -293,6 +330,8 @@ int prestera_port_pvid_set(struct prestera_port *port, u16 vid);
bool prestera_netdev_check(const struct net_device *dev);
+int prestera_is_valid_mac_addr(struct prestera_port *port, const u8 *addr);
+
bool prestera_port_is_lag_member(const struct prestera_port *port);
struct prestera_lag *prestera_lag_by_id(struct prestera_switch *sw, u16 id);
diff --git a/drivers/net/ethernet/marvell/prestera/prestera_acl.c b/drivers/net/ethernet/marvell/prestera/prestera_acl.c
index 83c75ff..f0d9f59 100644
--- a/drivers/net/ethernet/marvell/prestera/prestera_acl.c
+++ b/drivers/net/ethernet/marvell/prestera/prestera_acl.c
@@ -1,35 +1,72 @@
// SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0
-/* Copyright (c) 2020 Marvell International Ltd. All rights reserved */
+/* Copyright (c) 2020-2021 Marvell International Ltd. All rights reserved */
#include <linux/rhashtable.h>
-#include "prestera.h"
-#include "prestera_hw.h"
#include "prestera_acl.h"
-#include "prestera_span.h"
+#include "prestera_flow.h"
+#include "prestera_hw.h"
+#include "prestera.h"
+
+#define ACL_KEYMASK_SIZE \
+ (sizeof(__be32) * __PRESTERA_ACL_RULE_MATCH_TYPE_MAX)
struct prestera_acl {
struct prestera_switch *sw;
+ struct list_head vtcam_list;
struct list_head rules;
+ struct rhashtable ruleset_ht;
+ struct rhashtable acl_rule_entry_ht;
+ struct idr uid;
+};
+
+struct prestera_acl_ruleset_ht_key {
+ struct prestera_flow_block *block;
+};
+
+struct prestera_acl_rule_entry {
+ struct rhash_head ht_node;
+ struct prestera_acl_rule_entry_key key;
+ u32 hw_id;
+ u32 vtcam_id;
+ struct {
+ struct {
+ u8 valid:1;
+ } accept, drop, trap;
+ struct {
+ u32 id;
+ struct prestera_counter_block *block;
+ } counter;
+ };
};
struct prestera_acl_ruleset {
+ struct rhash_head ht_node; /* Member of acl HT */
+ struct prestera_acl_ruleset_ht_key ht_key;
struct rhashtable rule_ht;
- struct prestera_switch *sw;
- u16 id;
+ struct prestera_acl *acl;
+ unsigned long rule_count;
+ refcount_t refcount;
+ void *keymask;
+ u32 vtcam_id;
+ u16 pcl_id;
+ bool offload;
};
-struct prestera_acl_rule {
- struct rhash_head ht_node;
+struct prestera_acl_vtcam {
struct list_head list;
- struct list_head match_list;
- struct list_head action_list;
- struct prestera_flow_block *block;
- unsigned long cookie;
- u32 priority;
- u8 n_actions;
- u8 n_matches;
+ __be32 keymask[__PRESTERA_ACL_RULE_MATCH_TYPE_MAX];
+ refcount_t refcount;
u32 id;
+ bool is_keymask_set;
+ u8 lookup;
+};
+
+static const struct rhashtable_params prestera_acl_ruleset_ht_params = {
+ .key_len = sizeof(struct prestera_acl_ruleset_ht_key),
+ .key_offset = offsetof(struct prestera_acl_ruleset, ht_key),
+ .head_offset = offsetof(struct prestera_acl_ruleset, ht_node),
+ .automatic_shrinking = true,
};
static const struct rhashtable_params prestera_acl_rule_ht_params = {
@@ -39,28 +76,48 @@ static const struct rhashtable_params prestera_acl_rule_ht_params = {
.automatic_shrinking = true,
};
+static const struct rhashtable_params __prestera_acl_rule_entry_ht_params = {
+ .key_offset = offsetof(struct prestera_acl_rule_entry, key),
+ .head_offset = offsetof(struct prestera_acl_rule_entry, ht_node),
+ .key_len = sizeof(struct prestera_acl_rule_entry_key),
+ .automatic_shrinking = true,
+};
+
static struct prestera_acl_ruleset *
-prestera_acl_ruleset_create(struct prestera_switch *sw)
+prestera_acl_ruleset_create(struct prestera_acl *acl,
+ struct prestera_flow_block *block)
{
struct prestera_acl_ruleset *ruleset;
+ u32 uid = 0;
int err;
ruleset = kzalloc(sizeof(*ruleset), GFP_KERNEL);
if (!ruleset)
return ERR_PTR(-ENOMEM);
+ ruleset->acl = acl;
+ ruleset->ht_key.block = block;
+ refcount_set(&ruleset->refcount, 1);
+
err = rhashtable_init(&ruleset->rule_ht, &prestera_acl_rule_ht_params);
if (err)
goto err_rhashtable_init;
- err = prestera_hw_acl_ruleset_create(sw, &ruleset->id);
+ err = idr_alloc_u32(&acl->uid, NULL, &uid, U8_MAX, GFP_KERNEL);
if (err)
goto err_ruleset_create;
- ruleset->sw = sw;
+ /* make pcl-id based on uid */
+ ruleset->pcl_id = (u8)uid;
+ err = rhashtable_insert_fast(&acl->ruleset_ht, &ruleset->ht_node,
+ prestera_acl_ruleset_ht_params);
+ if (err)
+ goto err_ruleset_ht_insert;
return ruleset;
+err_ruleset_ht_insert:
+ idr_remove(&acl->uid, uid);
err_ruleset_create:
rhashtable_destroy(&ruleset->rule_ht);
err_rhashtable_init:
@@ -68,117 +125,164 @@ prestera_acl_ruleset_create(struct prestera_switch *sw)
return ERR_PTR(err);
}
+void prestera_acl_ruleset_keymask_set(struct prestera_acl_ruleset *ruleset,
+ void *keymask)
+{
+ ruleset->keymask = kmemdup(keymask, ACL_KEYMASK_SIZE, GFP_KERNEL);
+}
+
+int prestera_acl_ruleset_offload(struct prestera_acl_ruleset *ruleset)
+{
+ u32 vtcam_id;
+ int err;
+
+ if (ruleset->offload)
+ return -EEXIST;
+
+ err = prestera_acl_vtcam_id_get(ruleset->acl, 0,
+ ruleset->keymask, &vtcam_id);
+ if (err)
+ return err;
+
+ ruleset->vtcam_id = vtcam_id;
+ ruleset->offload = true;
+ return 0;
+}
+
static void prestera_acl_ruleset_destroy(struct prestera_acl_ruleset *ruleset)
{
- prestera_hw_acl_ruleset_del(ruleset->sw, ruleset->id);
+ struct prestera_acl *acl = ruleset->acl;
+ u8 uid = ruleset->pcl_id & PRESTERA_ACL_KEYMASK_PCL_ID_USER;
+
+ rhashtable_remove_fast(&acl->ruleset_ht, &ruleset->ht_node,
+ prestera_acl_ruleset_ht_params);
+
+ if (ruleset->offload)
+ WARN_ON(prestera_acl_vtcam_id_put(acl, ruleset->vtcam_id));
+
+ idr_remove(&acl->uid, uid);
+
rhashtable_destroy(&ruleset->rule_ht);
+ kfree(ruleset->keymask);
kfree(ruleset);
}
-struct prestera_flow_block *
-prestera_acl_block_create(struct prestera_switch *sw, struct net *net)
+static struct prestera_acl_ruleset *
+__prestera_acl_ruleset_lookup(struct prestera_acl *acl,
+ struct prestera_flow_block *block)
{
- struct prestera_flow_block *block;
+ struct prestera_acl_ruleset_ht_key ht_key;
- block = kzalloc(sizeof(*block), GFP_KERNEL);
- if (!block)
- return NULL;
- INIT_LIST_HEAD(&block->binding_list);
- block->net = net;
- block->sw = sw;
+ memset(&ht_key, 0, sizeof(ht_key));
+ ht_key.block = block;
+ return rhashtable_lookup_fast(&acl->ruleset_ht, &ht_key,
+ prestera_acl_ruleset_ht_params);
+}
- block->ruleset = prestera_acl_ruleset_create(sw);
- if (IS_ERR(block->ruleset)) {
- kfree(block);
- return NULL;
+struct prestera_acl_ruleset *
+prestera_acl_ruleset_lookup(struct prestera_acl *acl,
+ struct prestera_flow_block *block)
+{
+ struct prestera_acl_ruleset *ruleset;
+
+ ruleset = __prestera_acl_ruleset_lookup(acl, block);
+ if (!ruleset)
+ return ERR_PTR(-ENOENT);
+
+ refcount_inc(&ruleset->refcount);
+ return ruleset;
+}
+
+struct prestera_acl_ruleset *
+prestera_acl_ruleset_get(struct prestera_acl *acl,
+ struct prestera_flow_block *block)
+{
+ struct prestera_acl_ruleset *ruleset;
+
+ ruleset = __prestera_acl_ruleset_lookup(acl, block);
+ if (ruleset) {
+ refcount_inc(&ruleset->refcount);
+ return ruleset;
}
- return block;
+ return prestera_acl_ruleset_create(acl, block);
}
-void prestera_acl_block_destroy(struct prestera_flow_block *block)
+void prestera_acl_ruleset_put(struct prestera_acl_ruleset *ruleset)
{
- prestera_acl_ruleset_destroy(block->ruleset);
- WARN_ON(!list_empty(&block->binding_list));
- kfree(block);
+ if (!refcount_dec_and_test(&ruleset->refcount))
+ return;
+
+ prestera_acl_ruleset_destroy(ruleset);
}
-static struct prestera_flow_block_binding *
-prestera_acl_block_lookup(struct prestera_flow_block *block,
- struct prestera_port *port)
+int prestera_acl_ruleset_bind(struct prestera_acl_ruleset *ruleset,
+ struct prestera_port *port)
{
- struct prestera_flow_block_binding *binding;
+ struct prestera_acl_iface iface = {
+ .type = PRESTERA_ACL_IFACE_TYPE_PORT,
+ .port = port
+ };
- list_for_each_entry(binding, &block->binding_list, list)
- if (binding->port == port)
- return binding;
-
- return NULL;
+ return prestera_hw_vtcam_iface_bind(port->sw, &iface, ruleset->vtcam_id,
+ ruleset->pcl_id);
}
-int prestera_acl_block_bind(struct prestera_flow_block *block,
- struct prestera_port *port)
+int prestera_acl_ruleset_unbind(struct prestera_acl_ruleset *ruleset,
+ struct prestera_port *port)
+{
+ struct prestera_acl_iface iface = {
+ .type = PRESTERA_ACL_IFACE_TYPE_PORT,
+ .port = port
+ };
+
+ return prestera_hw_vtcam_iface_unbind(port->sw, &iface,
+ ruleset->vtcam_id);
+}
+
+static int prestera_acl_ruleset_block_bind(struct prestera_acl_ruleset *ruleset,
+ struct prestera_flow_block *block)
{
struct prestera_flow_block_binding *binding;
int err;
- if (WARN_ON(prestera_acl_block_lookup(block, port)))
- return -EEXIST;
-
- binding = kzalloc(sizeof(*binding), GFP_KERNEL);
- if (!binding)
- return -ENOMEM;
- binding->span_id = PRESTERA_SPAN_INVALID_ID;
- binding->port = port;
-
- err = prestera_hw_acl_port_bind(port, block->ruleset->id);
- if (err)
- goto err_rules_bind;
-
- list_add(&binding->list, &block->binding_list);
+ block->ruleset_zero = ruleset;
+ list_for_each_entry(binding, &block->binding_list, list) {
+ err = prestera_acl_ruleset_bind(ruleset, binding->port);
+ if (err)
+ goto rollback;
+ }
return 0;
-err_rules_bind:
- kfree(binding);
+rollback:
+ list_for_each_entry_continue_reverse(binding, &block->binding_list,
+ list)
+ err = prestera_acl_ruleset_unbind(ruleset, binding->port);
+ block->ruleset_zero = NULL;
+
return err;
}
-int prestera_acl_block_unbind(struct prestera_flow_block *block,
- struct prestera_port *port)
+static void
+prestera_acl_ruleset_block_unbind(struct prestera_acl_ruleset *ruleset,
+ struct prestera_flow_block *block)
{
struct prestera_flow_block_binding *binding;
- binding = prestera_acl_block_lookup(block, port);
- if (!binding)
- return -ENOENT;
-
- list_del(&binding->list);
-
- prestera_hw_acl_port_unbind(port, block->ruleset->id);
-
- kfree(binding);
- return 0;
+ list_for_each_entry(binding, &block->binding_list, list)
+ prestera_acl_ruleset_unbind(ruleset, binding->port);
+ block->ruleset_zero = NULL;
}
-struct prestera_acl_ruleset *
-prestera_acl_block_ruleset_get(struct prestera_flow_block *block)
+void
+prestera_acl_rule_keymask_pcl_id_set(struct prestera_acl_rule *rule, u16 pcl_id)
{
- return block->ruleset;
-}
+ struct prestera_acl_match *r_match = &rule->re_key.match;
+ __be16 pcl_id_mask = htons(PRESTERA_ACL_KEYMASK_PCL_ID);
+ __be16 pcl_id_key = htons(pcl_id);
-u16 prestera_acl_rule_ruleset_id_get(const struct prestera_acl_rule *rule)
-{
- return rule->block->ruleset->id;
-}
-
-struct net *prestera_acl_block_net(struct prestera_flow_block *block)
-{
- return block->net;
-}
-
-struct prestera_switch *prestera_acl_block_sw(struct prestera_flow_block *block)
-{
- return block->sw;
+ rule_match_set(r_match->key, PCL_ID, pcl_id_key);
+ rule_match_set(r_match->mask, PCL_ID, pcl_id_mask);
}
struct prestera_acl_rule *
@@ -189,8 +293,13 @@ prestera_acl_rule_lookup(struct prestera_acl_ruleset *ruleset,
prestera_acl_rule_ht_params);
}
+bool prestera_acl_ruleset_is_offload(struct prestera_acl_ruleset *ruleset)
+{
+ return ruleset->offload;
+}
+
struct prestera_acl_rule *
-prestera_acl_rule_create(struct prestera_flow_block *block,
+prestera_acl_rule_create(struct prestera_acl_ruleset *ruleset,
unsigned long cookie)
{
struct prestera_acl_rule *rule;
@@ -199,140 +308,98 @@ prestera_acl_rule_create(struct prestera_flow_block *block,
if (!rule)
return ERR_PTR(-ENOMEM);
- INIT_LIST_HEAD(&rule->match_list);
- INIT_LIST_HEAD(&rule->action_list);
+ rule->ruleset = ruleset;
rule->cookie = cookie;
- rule->block = block;
+
+ refcount_inc(&ruleset->refcount);
return rule;
}
-struct list_head *
-prestera_acl_rule_match_list_get(struct prestera_acl_rule *rule)
-{
- return &rule->match_list;
-}
-
-struct list_head *
-prestera_acl_rule_action_list_get(struct prestera_acl_rule *rule)
-{
- return &rule->action_list;
-}
-
-int prestera_acl_rule_action_add(struct prestera_acl_rule *rule,
- struct prestera_acl_rule_action_entry *entry)
-{
- struct prestera_acl_rule_action_entry *a_entry;
-
- a_entry = kmalloc(sizeof(*a_entry), GFP_KERNEL);
- if (!a_entry)
- return -ENOMEM;
-
- memcpy(a_entry, entry, sizeof(*entry));
- list_add(&a_entry->list, &rule->action_list);
-
- rule->n_actions++;
- return 0;
-}
-
-u8 prestera_acl_rule_action_len(struct prestera_acl_rule *rule)
-{
- return rule->n_actions;
-}
-
-u32 prestera_acl_rule_priority_get(struct prestera_acl_rule *rule)
-{
- return rule->priority;
-}
-
void prestera_acl_rule_priority_set(struct prestera_acl_rule *rule,
u32 priority)
{
rule->priority = priority;
}
-int prestera_acl_rule_match_add(struct prestera_acl_rule *rule,
- struct prestera_acl_rule_match_entry *entry)
-{
- struct prestera_acl_rule_match_entry *m_entry;
-
- m_entry = kmalloc(sizeof(*m_entry), GFP_KERNEL);
- if (!m_entry)
- return -ENOMEM;
-
- memcpy(m_entry, entry, sizeof(*entry));
- list_add(&m_entry->list, &rule->match_list);
-
- rule->n_matches++;
- return 0;
-}
-
-u8 prestera_acl_rule_match_len(struct prestera_acl_rule *rule)
-{
- return rule->n_matches;
-}
-
void prestera_acl_rule_destroy(struct prestera_acl_rule *rule)
{
- struct prestera_acl_rule_action_entry *a_entry;
- struct prestera_acl_rule_match_entry *m_entry;
- struct list_head *pos, *n;
-
- list_for_each_safe(pos, n, &rule->match_list) {
- m_entry = list_entry(pos, typeof(*m_entry), list);
- list_del(pos);
- kfree(m_entry);
- }
-
- list_for_each_safe(pos, n, &rule->action_list) {
- a_entry = list_entry(pos, typeof(*a_entry), list);
- list_del(pos);
- kfree(a_entry);
- }
-
+ prestera_acl_ruleset_put(rule->ruleset);
kfree(rule);
}
int prestera_acl_rule_add(struct prestera_switch *sw,
struct prestera_acl_rule *rule)
{
- u32 rule_id;
int err;
+ struct prestera_acl_ruleset *ruleset = rule->ruleset;
+ struct prestera_flow_block *block = ruleset->ht_key.block;
/* try to add rule to hash table first */
- err = rhashtable_insert_fast(&rule->block->ruleset->rule_ht,
- &rule->ht_node,
+ err = rhashtable_insert_fast(&ruleset->rule_ht, &rule->ht_node,
prestera_acl_rule_ht_params);
if (err)
- return err;
+ goto err_ht_insert;
- /* add rule to hw */
- err = prestera_hw_acl_rule_add(sw, rule, &rule_id);
+ prestera_acl_rule_keymask_pcl_id_set(rule, ruleset->pcl_id);
+ rule->re_arg.vtcam_id = ruleset->vtcam_id;
+ rule->re_key.prio = rule->priority;
+
+ /* setup counter */
+ rule->re_arg.count.valid = true;
+ rule->re_arg.count.client = PRESTERA_HW_COUNTER_CLIENT_LOOKUP_0;
+
+ rule->re = prestera_acl_rule_entry_find(sw->acl, &rule->re_key);
+ err = WARN_ON(rule->re) ? -EEXIST : 0;
if (err)
goto err_rule_add;
- rule->id = rule_id;
+ rule->re = prestera_acl_rule_entry_create(sw->acl, &rule->re_key,
+ &rule->re_arg);
+ err = !rule->re ? -EINVAL : 0;
+ if (err)
+ goto err_rule_add;
+
+ /* bind the block (all ports) to chain index 0 */
+ if (!ruleset->rule_count) {
+ err = prestera_acl_ruleset_block_bind(ruleset, block);
+ if (err)
+ goto err_acl_block_bind;
+ }
list_add_tail(&rule->list, &sw->acl->rules);
-
+ ruleset->rule_count++;
return 0;
+err_acl_block_bind:
+ prestera_acl_rule_entry_destroy(sw->acl, rule->re);
err_rule_add:
- rhashtable_remove_fast(&rule->block->ruleset->rule_ht, &rule->ht_node,
+ rule->re = NULL;
+ rhashtable_remove_fast(&ruleset->rule_ht, &rule->ht_node,
prestera_acl_rule_ht_params);
+err_ht_insert:
return err;
}
void prestera_acl_rule_del(struct prestera_switch *sw,
struct prestera_acl_rule *rule)
{
- rhashtable_remove_fast(&rule->block->ruleset->rule_ht, &rule->ht_node,
+ struct prestera_acl_ruleset *ruleset = rule->ruleset;
+ struct prestera_flow_block *block = ruleset->ht_key.block;
+
+ rhashtable_remove_fast(&ruleset->rule_ht, &rule->ht_node,
prestera_acl_rule_ht_params);
+ ruleset->rule_count--;
list_del(&rule->list);
- prestera_hw_acl_rule_del(sw, rule->id);
+
+ prestera_acl_rule_entry_destroy(sw->acl, rule->re);
+
+ /* unbind block (all ports) */
+ if (!ruleset->rule_count)
+ prestera_acl_ruleset_block_unbind(ruleset, block);
}
-int prestera_acl_rule_get_stats(struct prestera_switch *sw,
+int prestera_acl_rule_get_stats(struct prestera_acl *acl,
struct prestera_acl_rule *rule,
u64 *packets, u64 *bytes, u64 *last_use)
{
@@ -340,8 +407,10 @@ int prestera_acl_rule_get_stats(struct prestera_switch *sw,
u64 current_bytes;
int err;
- err = prestera_hw_acl_rule_stats_get(sw, rule->id, ¤t_packets,
- ¤t_bytes);
+ err = prestera_counter_stats_get(acl->sw->counter,
+ rule->re->counter.block,
+ rule->re->counter.id,
+ ¤t_packets, ¤t_bytes);
if (err)
return err;
@@ -352,25 +421,323 @@ int prestera_acl_rule_get_stats(struct prestera_switch *sw,
return 0;
}
+struct prestera_acl_rule_entry *
+prestera_acl_rule_entry_find(struct prestera_acl *acl,
+ struct prestera_acl_rule_entry_key *key)
+{
+ return rhashtable_lookup_fast(&acl->acl_rule_entry_ht, key,
+ __prestera_acl_rule_entry_ht_params);
+}
+
+static int __prestera_acl_rule_entry2hw_del(struct prestera_switch *sw,
+ struct prestera_acl_rule_entry *e)
+{
+ return prestera_hw_vtcam_rule_del(sw, e->vtcam_id, e->hw_id);
+}
+
+static int __prestera_acl_rule_entry2hw_add(struct prestera_switch *sw,
+ struct prestera_acl_rule_entry *e)
+{
+ struct prestera_acl_hw_action_info act_hw[PRESTERA_ACL_RULE_ACTION_MAX];
+ int act_num;
+
+ memset(&act_hw, 0, sizeof(act_hw));
+ act_num = 0;
+
+ /* accept */
+ if (e->accept.valid) {
+ act_hw[act_num].id = PRESTERA_ACL_RULE_ACTION_ACCEPT;
+ act_num++;
+ }
+ /* drop */
+ if (e->drop.valid) {
+ act_hw[act_num].id = PRESTERA_ACL_RULE_ACTION_DROP;
+ act_num++;
+ }
+ /* trap */
+ if (e->trap.valid) {
+ act_hw[act_num].id = PRESTERA_ACL_RULE_ACTION_TRAP;
+ act_num++;
+ }
+ /* counter */
+ if (e->counter.block) {
+ act_hw[act_num].id = PRESTERA_ACL_RULE_ACTION_COUNT;
+ act_hw[act_num].count.id = e->counter.id;
+ act_num++;
+ }
+
+ return prestera_hw_vtcam_rule_add(sw, e->vtcam_id, e->key.prio,
+ e->key.match.key, e->key.match.mask,
+ act_hw, act_num, &e->hw_id);
+}
+
+static void
+__prestera_acl_rule_entry_act_destruct(struct prestera_switch *sw,
+ struct prestera_acl_rule_entry *e)
+{
+ /* counter */
+ prestera_counter_put(sw->counter, e->counter.block, e->counter.id);
+}
+
+void prestera_acl_rule_entry_destroy(struct prestera_acl *acl,
+ struct prestera_acl_rule_entry *e)
+{
+ int ret;
+
+ rhashtable_remove_fast(&acl->acl_rule_entry_ht, &e->ht_node,
+ __prestera_acl_rule_entry_ht_params);
+
+ ret = __prestera_acl_rule_entry2hw_del(acl->sw, e);
+ WARN_ON(ret && ret != -ENODEV);
+
+ __prestera_acl_rule_entry_act_destruct(acl->sw, e);
+ kfree(e);
+}
+
+static int
+__prestera_acl_rule_entry_act_construct(struct prestera_switch *sw,
+ struct prestera_acl_rule_entry *e,
+ struct prestera_acl_rule_entry_arg *arg)
+{
+ /* accept */
+ e->accept.valid = arg->accept.valid;
+ /* drop */
+ e->drop.valid = arg->drop.valid;
+ /* trap */
+ e->trap.valid = arg->trap.valid;
+ /* counter */
+ if (arg->count.valid) {
+ int err;
+
+ err = prestera_counter_get(sw->counter, arg->count.client,
+ &e->counter.block,
+ &e->counter.id);
+ if (err)
+ goto err_out;
+ }
+
+ return 0;
+
+err_out:
+ __prestera_acl_rule_entry_act_destruct(sw, e);
+ return -EINVAL;
+}
+
+struct prestera_acl_rule_entry *
+prestera_acl_rule_entry_create(struct prestera_acl *acl,
+ struct prestera_acl_rule_entry_key *key,
+ struct prestera_acl_rule_entry_arg *arg)
+{
+ struct prestera_acl_rule_entry *e;
+ int err;
+
+ e = kzalloc(sizeof(*e), GFP_KERNEL);
+ if (!e)
+ goto err_kzalloc;
+
+ memcpy(&e->key, key, sizeof(*key));
+ e->vtcam_id = arg->vtcam_id;
+ err = __prestera_acl_rule_entry_act_construct(acl->sw, e, arg);
+ if (err)
+ goto err_act_construct;
+
+ err = __prestera_acl_rule_entry2hw_add(acl->sw, e);
+ if (err)
+ goto err_hw_add;
+
+ err = rhashtable_insert_fast(&acl->acl_rule_entry_ht, &e->ht_node,
+ __prestera_acl_rule_entry_ht_params);
+ if (err)
+ goto err_ht_insert;
+
+ return e;
+
+err_ht_insert:
+ WARN_ON(__prestera_acl_rule_entry2hw_del(acl->sw, e));
+err_hw_add:
+ __prestera_acl_rule_entry_act_destruct(acl->sw, e);
+err_act_construct:
+ kfree(e);
+err_kzalloc:
+ return NULL;
+}
+
+static int __prestera_acl_vtcam_id_try_fit(struct prestera_acl *acl, u8 lookup,
+ void *keymask, u32 *vtcam_id)
+{
+ struct prestera_acl_vtcam *vtcam;
+ int i;
+
+ list_for_each_entry(vtcam, &acl->vtcam_list, list) {
+ if (lookup != vtcam->lookup)
+ continue;
+
+ if (!keymask && !vtcam->is_keymask_set)
+ goto vtcam_found;
+
+ if (!(keymask && vtcam->is_keymask_set))
+ continue;
+
+ /* try to fit with vtcam keymask */
+ for (i = 0; i < __PRESTERA_ACL_RULE_MATCH_TYPE_MAX; i++) {
+ __be32 __keymask = ((__be32 *)keymask)[i];
+
+ if (!__keymask)
+ /* vtcam keymask in not interested */
+ continue;
+
+ if (__keymask & ~vtcam->keymask[i])
+ /* keymask does not fit the vtcam keymask */
+ break;
+ }
+
+ if (i == __PRESTERA_ACL_RULE_MATCH_TYPE_MAX)
+ /* keymask fits vtcam keymask, return it */
+ goto vtcam_found;
+ }
+
+ /* nothing is found */
+ return -ENOENT;
+
+vtcam_found:
+ refcount_inc(&vtcam->refcount);
+ *vtcam_id = vtcam->id;
+ return 0;
+}
+
+int prestera_acl_vtcam_id_get(struct prestera_acl *acl, u8 lookup,
+ void *keymask, u32 *vtcam_id)
+{
+ struct prestera_acl_vtcam *vtcam;
+ u32 new_vtcam_id;
+ int err;
+
+ /* find the vtcam that suits keymask. We do not expect to have
+ * a big number of vtcams, so, the list type for vtcam list is
+ * fine for now
+ */
+ list_for_each_entry(vtcam, &acl->vtcam_list, list) {
+ if (lookup != vtcam->lookup)
+ continue;
+
+ if (!keymask && !vtcam->is_keymask_set) {
+ refcount_inc(&vtcam->refcount);
+ goto vtcam_found;
+ }
+
+ if (keymask && vtcam->is_keymask_set &&
+ !memcmp(keymask, vtcam->keymask, sizeof(vtcam->keymask))) {
+ refcount_inc(&vtcam->refcount);
+ goto vtcam_found;
+ }
+ }
+
+ /* vtcam not found, try to create new one */
+ vtcam = kzalloc(sizeof(*vtcam), GFP_KERNEL);
+ if (!vtcam)
+ return -ENOMEM;
+
+ err = prestera_hw_vtcam_create(acl->sw, lookup, keymask, &new_vtcam_id,
+ PRESTERA_HW_VTCAM_DIR_INGRESS);
+ if (err) {
+ kfree(vtcam);
+
+ /* cannot create new, try to fit into existing vtcam */
+ if (__prestera_acl_vtcam_id_try_fit(acl, lookup,
+ keymask, &new_vtcam_id))
+ return err;
+
+ *vtcam_id = new_vtcam_id;
+ return 0;
+ }
+
+ vtcam->id = new_vtcam_id;
+ vtcam->lookup = lookup;
+ if (keymask) {
+ memcpy(vtcam->keymask, keymask, sizeof(vtcam->keymask));
+ vtcam->is_keymask_set = true;
+ }
+ refcount_set(&vtcam->refcount, 1);
+ list_add_rcu(&vtcam->list, &acl->vtcam_list);
+
+vtcam_found:
+ *vtcam_id = vtcam->id;
+ return 0;
+}
+
+int prestera_acl_vtcam_id_put(struct prestera_acl *acl, u32 vtcam_id)
+{
+ struct prestera_acl_vtcam *vtcam;
+ int err;
+
+ list_for_each_entry(vtcam, &acl->vtcam_list, list) {
+ if (vtcam_id != vtcam->id)
+ continue;
+
+ if (!refcount_dec_and_test(&vtcam->refcount))
+ return 0;
+
+ err = prestera_hw_vtcam_destroy(acl->sw, vtcam->id);
+ if (err && err != -ENODEV) {
+ refcount_set(&vtcam->refcount, 1);
+ return err;
+ }
+
+ list_del(&vtcam->list);
+ kfree(vtcam);
+ return 0;
+ }
+
+ return -ENOENT;
+}
+
int prestera_acl_init(struct prestera_switch *sw)
{
struct prestera_acl *acl;
+ int err;
acl = kzalloc(sizeof(*acl), GFP_KERNEL);
if (!acl)
return -ENOMEM;
- INIT_LIST_HEAD(&acl->rules);
- sw->acl = acl;
acl->sw = sw;
+ INIT_LIST_HEAD(&acl->rules);
+ INIT_LIST_HEAD(&acl->vtcam_list);
+ idr_init(&acl->uid);
+
+ err = rhashtable_init(&acl->acl_rule_entry_ht,
+ &__prestera_acl_rule_entry_ht_params);
+ if (err)
+ goto err_acl_rule_entry_ht_init;
+
+ err = rhashtable_init(&acl->ruleset_ht,
+ &prestera_acl_ruleset_ht_params);
+ if (err)
+ goto err_ruleset_ht_init;
+
+ sw->acl = acl;
return 0;
+
+err_ruleset_ht_init:
+ rhashtable_destroy(&acl->acl_rule_entry_ht);
+err_acl_rule_entry_ht_init:
+ kfree(acl);
+ return err;
}
void prestera_acl_fini(struct prestera_switch *sw)
{
struct prestera_acl *acl = sw->acl;
+ WARN_ON(!idr_is_empty(&acl->uid));
+ idr_destroy(&acl->uid);
+
+ WARN_ON(!list_empty(&acl->vtcam_list));
WARN_ON(!list_empty(&acl->rules));
+
+ rhashtable_destroy(&acl->ruleset_ht);
+ rhashtable_destroy(&acl->acl_rule_entry_ht);
+
kfree(acl);
}
diff --git a/drivers/net/ethernet/marvell/prestera/prestera_acl.h b/drivers/net/ethernet/marvell/prestera/prestera_acl.h
index 39b7869b..40f6c1d 100644
--- a/drivers/net/ethernet/marvell/prestera/prestera_acl.h
+++ b/drivers/net/ethernet/marvell/prestera/prestera_acl.h
@@ -1,114 +1,130 @@
/* SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0 */
-/* Copyright (c) 2020 Marvell International Ltd. All rights reserved. */
+/* Copyright (c) 2020-2021 Marvell International Ltd. All rights reserved. */
#ifndef _PRESTERA_ACL_H_
#define _PRESTERA_ACL_H_
-enum prestera_acl_rule_match_entry_type {
- PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_ETH_TYPE = 1,
- PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_ETH_DMAC,
- PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_ETH_SMAC,
- PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_IP_PROTO,
- PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_PORT,
- PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_IP_SRC,
- PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_IP_DST,
- PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_L4_PORT_SRC,
- PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_L4_PORT_DST,
- PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_L4_PORT_RANGE_SRC,
- PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_L4_PORT_RANGE_DST,
- PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_VLAN_ID,
- PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_VLAN_TPID,
- PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_ICMP_TYPE,
- PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_ICMP_CODE
+#include <linux/types.h>
+#include "prestera_counter.h"
+
+#define PRESTERA_ACL_KEYMASK_PCL_ID 0x3FF
+#define PRESTERA_ACL_KEYMASK_PCL_ID_USER \
+ (PRESTERA_ACL_KEYMASK_PCL_ID & 0x00FF)
+
+#define rule_match_set_n(match_p, type, val_p, size) \
+ memcpy(&(match_p)[PRESTERA_ACL_RULE_MATCH_TYPE_##type], \
+ val_p, size)
+#define rule_match_set(match_p, type, val) \
+ memcpy(&(match_p)[PRESTERA_ACL_RULE_MATCH_TYPE_##type], \
+ &(val), sizeof(val))
+
+enum prestera_acl_match_type {
+ PRESTERA_ACL_RULE_MATCH_TYPE_PCL_ID,
+ PRESTERA_ACL_RULE_MATCH_TYPE_ETH_TYPE,
+ PRESTERA_ACL_RULE_MATCH_TYPE_ETH_DMAC_0,
+ PRESTERA_ACL_RULE_MATCH_TYPE_ETH_DMAC_1,
+ PRESTERA_ACL_RULE_MATCH_TYPE_ETH_SMAC_0,
+ PRESTERA_ACL_RULE_MATCH_TYPE_ETH_SMAC_1,
+ PRESTERA_ACL_RULE_MATCH_TYPE_IP_PROTO,
+ PRESTERA_ACL_RULE_MATCH_TYPE_SYS_PORT,
+ PRESTERA_ACL_RULE_MATCH_TYPE_SYS_DEV,
+ PRESTERA_ACL_RULE_MATCH_TYPE_IP_SRC,
+ PRESTERA_ACL_RULE_MATCH_TYPE_IP_DST,
+ PRESTERA_ACL_RULE_MATCH_TYPE_L4_PORT_SRC,
+ PRESTERA_ACL_RULE_MATCH_TYPE_L4_PORT_DST,
+ PRESTERA_ACL_RULE_MATCH_TYPE_L4_PORT_RANGE_SRC,
+ PRESTERA_ACL_RULE_MATCH_TYPE_L4_PORT_RANGE_DST,
+ PRESTERA_ACL_RULE_MATCH_TYPE_VLAN_ID,
+ PRESTERA_ACL_RULE_MATCH_TYPE_VLAN_TPID,
+ PRESTERA_ACL_RULE_MATCH_TYPE_ICMP_TYPE,
+ PRESTERA_ACL_RULE_MATCH_TYPE_ICMP_CODE,
+
+ __PRESTERA_ACL_RULE_MATCH_TYPE_MAX
};
enum prestera_acl_rule_action {
- PRESTERA_ACL_RULE_ACTION_ACCEPT,
- PRESTERA_ACL_RULE_ACTION_DROP,
- PRESTERA_ACL_RULE_ACTION_TRAP
+ PRESTERA_ACL_RULE_ACTION_ACCEPT = 0,
+ PRESTERA_ACL_RULE_ACTION_DROP = 1,
+ PRESTERA_ACL_RULE_ACTION_TRAP = 2,
+ PRESTERA_ACL_RULE_ACTION_COUNT = 7,
+
+ PRESTERA_ACL_RULE_ACTION_MAX
};
-struct prestera_switch;
-struct prestera_port;
-struct prestera_acl_rule;
-struct prestera_acl_ruleset;
-
-struct prestera_flow_block_binding {
- struct list_head list;
- struct prestera_port *port;
- int span_id;
+enum {
+ PRESTERA_ACL_IFACE_TYPE_PORT,
+ PRESTERA_ACL_IFACE_TYPE_INDEX
};
-struct prestera_flow_block {
- struct list_head binding_list;
- struct prestera_switch *sw;
- struct net *net;
- struct prestera_acl_ruleset *ruleset;
- struct flow_block_cb *block_cb;
+struct prestera_acl_match {
+ __be32 key[__PRESTERA_ACL_RULE_MATCH_TYPE_MAX];
+ __be32 mask[__PRESTERA_ACL_RULE_MATCH_TYPE_MAX];
};
-struct prestera_acl_rule_action_entry {
- struct list_head list;
+struct prestera_acl_action_count {
+ u32 id;
+};
+
+struct prestera_acl_rule_entry_key {
+ u32 prio;
+ struct prestera_acl_match match;
+};
+
+struct prestera_acl_hw_action_info {
enum prestera_acl_rule_action id;
+ union {
+ struct prestera_acl_action_count count;
+ };
};
-struct prestera_acl_rule_match_entry {
- struct list_head list;
- enum prestera_acl_rule_match_entry_type type;
- union {
+/* This struct (arg) used only to be passed as parameter for
+ * acl_rule_entry_create. Must be flat. Can contain object keys, which will be
+ * resolved to object links, before saving to acl_rule_entry struct
+ */
+struct prestera_acl_rule_entry_arg {
+ u32 vtcam_id;
+ struct {
struct {
- u8 key;
- u8 mask;
- } u8;
+ u8 valid:1;
+ } accept, drop, trap;
struct {
- u16 key;
- u16 mask;
- } u16;
- struct {
- u32 key;
- u32 mask;
- } u32;
- struct {
- u64 key;
- u64 mask;
- } u64;
- struct {
- u8 key[ETH_ALEN];
- u8 mask[ETH_ALEN];
- } mac;
- } keymask;
+ u8 valid:1;
+ u32 client;
+ } count;
+ };
};
+struct prestera_acl_rule {
+ struct rhash_head ht_node; /* Member of acl HT */
+ struct list_head list;
+ struct prestera_acl_ruleset *ruleset;
+ unsigned long cookie;
+ u32 priority;
+ struct prestera_acl_rule_entry_key re_key;
+ struct prestera_acl_rule_entry_arg re_arg;
+ struct prestera_acl_rule_entry *re;
+};
+
+struct prestera_acl_iface {
+ union {
+ struct prestera_port *port;
+ u32 index;
+ };
+ u8 type;
+};
+
+struct prestera_acl;
+struct prestera_switch;
+struct prestera_flow_block;
+
int prestera_acl_init(struct prestera_switch *sw);
void prestera_acl_fini(struct prestera_switch *sw);
-struct prestera_flow_block *
-prestera_acl_block_create(struct prestera_switch *sw, struct net *net);
-void prestera_acl_block_destroy(struct prestera_flow_block *block);
-struct net *prestera_acl_block_net(struct prestera_flow_block *block);
-struct prestera_switch *prestera_acl_block_sw(struct prestera_flow_block *block);
-int prestera_acl_block_bind(struct prestera_flow_block *block,
- struct prestera_port *port);
-int prestera_acl_block_unbind(struct prestera_flow_block *block,
- struct prestera_port *port);
-struct prestera_acl_ruleset *
-prestera_acl_block_ruleset_get(struct prestera_flow_block *block);
+
struct prestera_acl_rule *
-prestera_acl_rule_create(struct prestera_flow_block *block,
+prestera_acl_rule_create(struct prestera_acl_ruleset *ruleset,
unsigned long cookie);
-u32 prestera_acl_rule_priority_get(struct prestera_acl_rule *rule);
void prestera_acl_rule_priority_set(struct prestera_acl_rule *rule,
u32 priority);
-u16 prestera_acl_rule_ruleset_id_get(const struct prestera_acl_rule *rule);
-struct list_head *
-prestera_acl_rule_action_list_get(struct prestera_acl_rule *rule);
-u8 prestera_acl_rule_action_len(struct prestera_acl_rule *rule);
-u8 prestera_acl_rule_match_len(struct prestera_acl_rule *rule);
-int prestera_acl_rule_action_add(struct prestera_acl_rule *rule,
- struct prestera_acl_rule_action_entry *entry);
-struct list_head *
-prestera_acl_rule_match_list_get(struct prestera_acl_rule *rule);
-int prestera_acl_rule_match_add(struct prestera_acl_rule *rule,
- struct prestera_acl_rule_match_entry *entry);
void prestera_acl_rule_destroy(struct prestera_acl_rule *rule);
struct prestera_acl_rule *
prestera_acl_rule_lookup(struct prestera_acl_ruleset *ruleset,
@@ -117,8 +133,39 @@ int prestera_acl_rule_add(struct prestera_switch *sw,
struct prestera_acl_rule *rule);
void prestera_acl_rule_del(struct prestera_switch *sw,
struct prestera_acl_rule *rule);
-int prestera_acl_rule_get_stats(struct prestera_switch *sw,
+int prestera_acl_rule_get_stats(struct prestera_acl *acl,
struct prestera_acl_rule *rule,
u64 *packets, u64 *bytes, u64 *last_use);
+struct prestera_acl_rule_entry *
+prestera_acl_rule_entry_find(struct prestera_acl *acl,
+ struct prestera_acl_rule_entry_key *key);
+void prestera_acl_rule_entry_destroy(struct prestera_acl *acl,
+ struct prestera_acl_rule_entry *e);
+struct prestera_acl_rule_entry *
+prestera_acl_rule_entry_create(struct prestera_acl *acl,
+ struct prestera_acl_rule_entry_key *key,
+ struct prestera_acl_rule_entry_arg *arg);
+struct prestera_acl_ruleset *
+prestera_acl_ruleset_get(struct prestera_acl *acl,
+ struct prestera_flow_block *block);
+struct prestera_acl_ruleset *
+prestera_acl_ruleset_lookup(struct prestera_acl *acl,
+ struct prestera_flow_block *block);
+void prestera_acl_ruleset_keymask_set(struct prestera_acl_ruleset *ruleset,
+ void *keymask);
+bool prestera_acl_ruleset_is_offload(struct prestera_acl_ruleset *ruleset);
+int prestera_acl_ruleset_offload(struct prestera_acl_ruleset *ruleset);
+void prestera_acl_ruleset_put(struct prestera_acl_ruleset *ruleset);
+int prestera_acl_ruleset_bind(struct prestera_acl_ruleset *ruleset,
+ struct prestera_port *port);
+int prestera_acl_ruleset_unbind(struct prestera_acl_ruleset *ruleset,
+ struct prestera_port *port);
+void
+prestera_acl_rule_keymask_pcl_id_set(struct prestera_acl_rule *rule,
+ u16 pcl_id);
+
+int prestera_acl_vtcam_id_get(struct prestera_acl *acl, u8 lookup,
+ void *keymask, u32 *vtcam_id);
+int prestera_acl_vtcam_id_put(struct prestera_acl *acl, u32 vtcam_id);
#endif /* _PRESTERA_ACL_H_ */
diff --git a/drivers/net/ethernet/marvell/prestera/prestera_counter.c b/drivers/net/ethernet/marvell/prestera/prestera_counter.c
new file mode 100644
index 0000000..4cd53a2
--- /dev/null
+++ b/drivers/net/ethernet/marvell/prestera/prestera_counter.c
@@ -0,0 +1,475 @@
+// SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0
+/* Copyright (c) 2021 Marvell International Ltd. All rights reserved */
+
+#include "prestera.h"
+#include "prestera_hw.h"
+#include "prestera_acl.h"
+#include "prestera_counter.h"
+
+#define COUNTER_POLL_TIME (msecs_to_jiffies(1000))
+#define COUNTER_RESCHED_TIME (msecs_to_jiffies(50))
+#define COUNTER_BULK_SIZE (256)
+
+struct prestera_counter {
+ struct prestera_switch *sw;
+ struct delayed_work stats_dw;
+ struct mutex mtx; /* protect block_list */
+ struct prestera_counter_block **block_list;
+ u32 total_read;
+ u32 block_list_len;
+ u32 curr_idx;
+ bool is_fetching;
+};
+
+struct prestera_counter_block {
+ struct list_head list;
+ u32 id;
+ u32 offset;
+ u32 num_counters;
+ u32 client;
+ struct idr counter_idr;
+ refcount_t refcnt;
+ struct mutex mtx; /* protect stats and counter_idr */
+ struct prestera_counter_stats *stats;
+ u8 *counter_flag;
+ bool is_updating;
+ bool full;
+};
+
+enum {
+ COUNTER_FLAG_READY = 0,
+ COUNTER_FLAG_INVALID = 1
+};
+
+static bool
+prestera_counter_is_ready(struct prestera_counter_block *block, u32 id)
+{
+ return block->counter_flag[id - block->offset] == COUNTER_FLAG_READY;
+}
+
+static void prestera_counter_lock(struct prestera_counter *counter)
+{
+ mutex_lock(&counter->mtx);
+}
+
+static void prestera_counter_unlock(struct prestera_counter *counter)
+{
+ mutex_unlock(&counter->mtx);
+}
+
+static void prestera_counter_block_lock(struct prestera_counter_block *block)
+{
+ mutex_lock(&block->mtx);
+}
+
+static void prestera_counter_block_unlock(struct prestera_counter_block *block)
+{
+ mutex_unlock(&block->mtx);
+}
+
+static bool prestera_counter_block_incref(struct prestera_counter_block *block)
+{
+ return refcount_inc_not_zero(&block->refcnt);
+}
+
+static bool prestera_counter_block_decref(struct prestera_counter_block *block)
+{
+ return refcount_dec_and_test(&block->refcnt);
+}
+
+/* must be called with prestera_counter_block_lock() */
+static void prestera_counter_stats_clear(struct prestera_counter_block *block,
+ u32 counter_id)
+{
+ memset(&block->stats[counter_id - block->offset], 0,
+ sizeof(*block->stats));
+}
+
+static struct prestera_counter_block *
+prestera_counter_block_lookup_not_full(struct prestera_counter *counter,
+ u32 client)
+{
+ u32 i;
+
+ prestera_counter_lock(counter);
+ for (i = 0; i < counter->block_list_len; i++) {
+ if (counter->block_list[i] &&
+ counter->block_list[i]->client == client &&
+ !counter->block_list[i]->full &&
+ prestera_counter_block_incref(counter->block_list[i])) {
+ prestera_counter_unlock(counter);
+ return counter->block_list[i];
+ }
+ }
+ prestera_counter_unlock(counter);
+
+ return NULL;
+}
+
+static int prestera_counter_block_list_add(struct prestera_counter *counter,
+ struct prestera_counter_block *block)
+{
+ struct prestera_counter_block **arr;
+ u32 i;
+
+ prestera_counter_lock(counter);
+
+ for (i = 0; i < counter->block_list_len; i++) {
+ if (counter->block_list[i])
+ continue;
+
+ counter->block_list[i] = block;
+ prestera_counter_unlock(counter);
+ return 0;
+ }
+
+ arr = krealloc(counter->block_list, (counter->block_list_len + 1) *
+ sizeof(*counter->block_list), GFP_KERNEL);
+ if (!arr) {
+ prestera_counter_unlock(counter);
+ return -ENOMEM;
+ }
+
+ counter->block_list = arr;
+ counter->block_list[counter->block_list_len] = block;
+ counter->block_list_len++;
+ prestera_counter_unlock(counter);
+ return 0;
+}
+
+static struct prestera_counter_block *
+prestera_counter_block_get(struct prestera_counter *counter, u32 client)
+{
+ struct prestera_counter_block *block;
+ int err;
+
+ block = prestera_counter_block_lookup_not_full(counter, client);
+ if (block)
+ return block;
+
+ block = kzalloc(sizeof(*block), GFP_KERNEL);
+ if (!block)
+ return ERR_PTR(-ENOMEM);
+
+ err = prestera_hw_counter_block_get(counter->sw, client,
+ &block->id, &block->offset,
+ &block->num_counters);
+ if (err)
+ goto err_block;
+
+ block->stats = kcalloc(block->num_counters,
+ sizeof(*block->stats), GFP_KERNEL);
+ if (!block->stats) {
+ err = -ENOMEM;
+ goto err_stats;
+ }
+
+ block->counter_flag = kcalloc(block->num_counters,
+ sizeof(*block->counter_flag),
+ GFP_KERNEL);
+ if (!block->counter_flag) {
+ err = -ENOMEM;
+ goto err_flag;
+ }
+
+ block->client = client;
+ mutex_init(&block->mtx);
+ refcount_set(&block->refcnt, 1);
+ idr_init_base(&block->counter_idr, block->offset);
+
+ err = prestera_counter_block_list_add(counter, block);
+ if (err)
+ goto err_list_add;
+
+ return block;
+
+err_list_add:
+ idr_destroy(&block->counter_idr);
+ mutex_destroy(&block->mtx);
+ kfree(block->counter_flag);
+err_flag:
+ kfree(block->stats);
+err_stats:
+ prestera_hw_counter_block_release(counter->sw, block->id);
+err_block:
+ kfree(block);
+ return ERR_PTR(err);
+}
+
+static void prestera_counter_block_put(struct prestera_counter *counter,
+ struct prestera_counter_block *block)
+{
+ u32 i;
+
+ if (!prestera_counter_block_decref(block))
+ return;
+
+ prestera_counter_lock(counter);
+ for (i = 0; i < counter->block_list_len; i++) {
+ if (counter->block_list[i] &&
+ counter->block_list[i]->id == block->id) {
+ counter->block_list[i] = NULL;
+ break;
+ }
+ }
+ prestera_counter_unlock(counter);
+
+ WARN_ON(!idr_is_empty(&block->counter_idr));
+
+ prestera_hw_counter_block_release(counter->sw, block->id);
+ idr_destroy(&block->counter_idr);
+ mutex_destroy(&block->mtx);
+ kfree(block->stats);
+ kfree(block);
+}
+
+static int prestera_counter_get_vacant(struct prestera_counter_block *block,
+ u32 *id)
+{
+ int free_id;
+
+ if (block->full)
+ return -ENOSPC;
+
+ prestera_counter_block_lock(block);
+ free_id = idr_alloc_cyclic(&block->counter_idr, NULL, block->offset,
+ block->offset + block->num_counters,
+ GFP_KERNEL);
+ if (free_id < 0) {
+ if (free_id == -ENOSPC)
+ block->full = true;
+
+ prestera_counter_block_unlock(block);
+ return free_id;
+ }
+ *id = free_id;
+ prestera_counter_block_unlock(block);
+
+ return 0;
+}
+
+int prestera_counter_get(struct prestera_counter *counter, u32 client,
+ struct prestera_counter_block **bl, u32 *counter_id)
+{
+ struct prestera_counter_block *block;
+ int err;
+ u32 id;
+
+get_next_block:
+ block = prestera_counter_block_get(counter, client);
+ if (IS_ERR(block))
+ return PTR_ERR(block);
+
+ err = prestera_counter_get_vacant(block, &id);
+ if (err) {
+ prestera_counter_block_put(counter, block);
+
+ if (err == -ENOSPC)
+ goto get_next_block;
+
+ return err;
+ }
+
+ prestera_counter_block_lock(block);
+ if (block->is_updating)
+ block->counter_flag[id - block->offset] = COUNTER_FLAG_INVALID;
+ prestera_counter_block_unlock(block);
+
+ *counter_id = id;
+ *bl = block;
+
+ return 0;
+}
+
+void prestera_counter_put(struct prestera_counter *counter,
+ struct prestera_counter_block *block, u32 counter_id)
+{
+ if (!block)
+ return;
+
+ prestera_counter_block_lock(block);
+ idr_remove(&block->counter_idr, counter_id);
+ block->full = false;
+ prestera_counter_stats_clear(block, counter_id);
+ prestera_counter_block_unlock(block);
+
+ prestera_hw_counter_clear(counter->sw, block->id, counter_id);
+ prestera_counter_block_put(counter, block);
+}
+
+static u32 prestera_counter_block_idx_next(struct prestera_counter *counter,
+ u32 curr_idx)
+{
+ u32 idx, i, start = curr_idx + 1;
+
+ prestera_counter_lock(counter);
+ for (i = 0; i < counter->block_list_len; i++) {
+ idx = (start + i) % counter->block_list_len;
+ if (!counter->block_list[idx])
+ continue;
+
+ prestera_counter_unlock(counter);
+ return idx;
+ }
+ prestera_counter_unlock(counter);
+
+ return 0;
+}
+
+static struct prestera_counter_block *
+prestera_counter_block_get_by_idx(struct prestera_counter *counter, u32 idx)
+{
+ if (idx >= counter->block_list_len)
+ return NULL;
+
+ prestera_counter_lock(counter);
+
+ if (!counter->block_list[idx] ||
+ !prestera_counter_block_incref(counter->block_list[idx])) {
+ prestera_counter_unlock(counter);
+ return NULL;
+ }
+
+ prestera_counter_unlock(counter);
+ return counter->block_list[idx];
+}
+
+static void prestera_counter_stats_work(struct work_struct *work)
+{
+ struct delayed_work *dl_work =
+ container_of(work, struct delayed_work, work);
+ struct prestera_counter *counter =
+ container_of(dl_work, struct prestera_counter, stats_dw);
+ struct prestera_counter_block *block;
+ u32 resched_time = COUNTER_POLL_TIME;
+ u32 count = COUNTER_BULK_SIZE;
+ bool done = false;
+ int err;
+ u32 i;
+
+ block = prestera_counter_block_get_by_idx(counter, counter->curr_idx);
+ if (!block) {
+ if (counter->is_fetching)
+ goto abort;
+
+ goto next;
+ }
+
+ if (!counter->is_fetching) {
+ err = prestera_hw_counter_trigger(counter->sw, block->id);
+ if (err)
+ goto abort;
+
+ prestera_counter_block_lock(block);
+ block->is_updating = true;
+ prestera_counter_block_unlock(block);
+
+ counter->is_fetching = true;
+ counter->total_read = 0;
+ resched_time = COUNTER_RESCHED_TIME;
+ goto resched;
+ }
+
+ prestera_counter_block_lock(block);
+ err = prestera_hw_counters_get(counter->sw, counter->total_read,
+ &count, &done,
+ &block->stats[counter->total_read]);
+ prestera_counter_block_unlock(block);
+ if (err)
+ goto abort;
+
+ counter->total_read += count;
+ if (!done || counter->total_read < block->num_counters) {
+ resched_time = COUNTER_RESCHED_TIME;
+ goto resched;
+ }
+
+ for (i = 0; i < block->num_counters; i++) {
+ if (block->counter_flag[i] == COUNTER_FLAG_INVALID) {
+ prestera_counter_block_lock(block);
+ block->counter_flag[i] = COUNTER_FLAG_READY;
+ memset(&block->stats[i], 0, sizeof(*block->stats));
+ prestera_counter_block_unlock(block);
+ }
+ }
+
+ prestera_counter_block_lock(block);
+ block->is_updating = false;
+ prestera_counter_block_unlock(block);
+
+ goto next;
+abort:
+ prestera_hw_counter_abort(counter->sw);
+next:
+ counter->is_fetching = false;
+ counter->curr_idx =
+ prestera_counter_block_idx_next(counter, counter->curr_idx);
+resched:
+ if (block)
+ prestera_counter_block_put(counter, block);
+
+ schedule_delayed_work(&counter->stats_dw, resched_time);
+}
+
+/* Can be executed without rtnl_lock().
+ * So pay attention when something changing.
+ */
+int prestera_counter_stats_get(struct prestera_counter *counter,
+ struct prestera_counter_block *block,
+ u32 counter_id, u64 *packets, u64 *bytes)
+{
+ if (!block || !prestera_counter_is_ready(block, counter_id)) {
+ *packets = 0;
+ *bytes = 0;
+ return 0;
+ }
+
+ prestera_counter_block_lock(block);
+ *packets = block->stats[counter_id - block->offset].packets;
+ *bytes = block->stats[counter_id - block->offset].bytes;
+
+ prestera_counter_stats_clear(block, counter_id);
+ prestera_counter_block_unlock(block);
+
+ return 0;
+}
+
+int prestera_counter_init(struct prestera_switch *sw)
+{
+ struct prestera_counter *counter;
+
+ counter = kzalloc(sizeof(*counter), GFP_KERNEL);
+ if (!counter)
+ return -ENOMEM;
+
+ counter->block_list = kzalloc(sizeof(*counter->block_list), GFP_KERNEL);
+ if (!counter->block_list) {
+ kfree(counter);
+ return -ENOMEM;
+ }
+
+ mutex_init(&counter->mtx);
+ counter->block_list_len = 1;
+ counter->sw = sw;
+ sw->counter = counter;
+
+ INIT_DELAYED_WORK(&counter->stats_dw, prestera_counter_stats_work);
+ schedule_delayed_work(&counter->stats_dw, COUNTER_POLL_TIME);
+
+ return 0;
+}
+
+void prestera_counter_fini(struct prestera_switch *sw)
+{
+ struct prestera_counter *counter = sw->counter;
+ u32 i;
+
+ cancel_delayed_work_sync(&counter->stats_dw);
+
+ for (i = 0; i < counter->block_list_len; i++)
+ WARN_ON(counter->block_list[i]);
+
+ mutex_destroy(&counter->mtx);
+ kfree(counter->block_list);
+ kfree(counter);
+}
diff --git a/drivers/net/ethernet/marvell/prestera/prestera_counter.h b/drivers/net/ethernet/marvell/prestera/prestera_counter.h
new file mode 100644
index 0000000..ad6b739
--- /dev/null
+++ b/drivers/net/ethernet/marvell/prestera/prestera_counter.h
@@ -0,0 +1,30 @@
+/* SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0 */
+/* Copyright (c) 2021 Marvell International Ltd. All rights reserved. */
+
+#ifndef _PRESTERA_COUNTER_H_
+#define _PRESTERA_COUNTER_H_
+
+#include <linux/types.h>
+
+struct prestera_counter_stats {
+ u64 packets;
+ u64 bytes;
+};
+
+struct prestera_switch;
+struct prestera_counter;
+struct prestera_counter_block;
+
+int prestera_counter_init(struct prestera_switch *sw);
+void prestera_counter_fini(struct prestera_switch *sw);
+
+int prestera_counter_get(struct prestera_counter *counter, u32 client,
+ struct prestera_counter_block **block,
+ u32 *counter_id);
+void prestera_counter_put(struct prestera_counter *counter,
+ struct prestera_counter_block *block, u32 counter_id);
+int prestera_counter_stats_get(struct prestera_counter *counter,
+ struct prestera_counter_block *block,
+ u32 counter_id, u64 *packets, u64 *bytes);
+
+#endif /* _PRESTERA_COUNTER_H_ */
diff --git a/drivers/net/ethernet/marvell/prestera/prestera_flow.c b/drivers/net/ethernet/marvell/prestera/prestera_flow.c
index c9891e9..d849f04 100644
--- a/drivers/net/ethernet/marvell/prestera/prestera_flow.c
+++ b/drivers/net/ethernet/marvell/prestera/prestera_flow.c
@@ -40,6 +40,11 @@ static int prestera_flow_block_flower_cb(struct prestera_flow_block *block,
return 0;
case FLOW_CLS_STATS:
return prestera_flower_stats(block, f);
+ case FLOW_CLS_TMPLT_CREATE:
+ return prestera_flower_tmplt_create(block, f);
+ case FLOW_CLS_TMPLT_DESTROY:
+ prestera_flower_tmplt_destroy(block, f);
+ return 0;
default:
return -EOPNOTSUPP;
}
@@ -60,11 +65,102 @@ static int prestera_flow_block_cb(enum tc_setup_type type,
}
}
+static void prestera_flow_block_destroy(void *cb_priv)
+{
+ struct prestera_flow_block *block = cb_priv;
+
+ prestera_flower_template_cleanup(block);
+
+ WARN_ON(!list_empty(&block->binding_list));
+
+ kfree(block);
+}
+
+static struct prestera_flow_block *
+prestera_flow_block_create(struct prestera_switch *sw, struct net *net)
+{
+ struct prestera_flow_block *block;
+
+ block = kzalloc(sizeof(*block), GFP_KERNEL);
+ if (!block)
+ return NULL;
+
+ INIT_LIST_HEAD(&block->binding_list);
+ block->net = net;
+ block->sw = sw;
+
+ return block;
+}
+
static void prestera_flow_block_release(void *cb_priv)
{
struct prestera_flow_block *block = cb_priv;
- prestera_acl_block_destroy(block);
+ prestera_flow_block_destroy(block);
+}
+
+static bool
+prestera_flow_block_is_bound(const struct prestera_flow_block *block)
+{
+ return block->ruleset_zero;
+}
+
+static struct prestera_flow_block_binding *
+prestera_flow_block_lookup(struct prestera_flow_block *block,
+ struct prestera_port *port)
+{
+ struct prestera_flow_block_binding *binding;
+
+ list_for_each_entry(binding, &block->binding_list, list)
+ if (binding->port == port)
+ return binding;
+
+ return NULL;
+}
+
+static int prestera_flow_block_bind(struct prestera_flow_block *block,
+ struct prestera_port *port)
+{
+ struct prestera_flow_block_binding *binding;
+ int err;
+
+ binding = kzalloc(sizeof(*binding), GFP_KERNEL);
+ if (!binding)
+ return -ENOMEM;
+
+ binding->span_id = PRESTERA_SPAN_INVALID_ID;
+ binding->port = port;
+
+ if (prestera_flow_block_is_bound(block)) {
+ err = prestera_acl_ruleset_bind(block->ruleset_zero, port);
+ if (err)
+ goto err_ruleset_bind;
+ }
+
+ list_add(&binding->list, &block->binding_list);
+ return 0;
+
+err_ruleset_bind:
+ kfree(binding);
+ return err;
+}
+
+static int prestera_flow_block_unbind(struct prestera_flow_block *block,
+ struct prestera_port *port)
+{
+ struct prestera_flow_block_binding *binding;
+
+ binding = prestera_flow_block_lookup(block, port);
+ if (!binding)
+ return -ENOENT;
+
+ list_del(&binding->list);
+
+ if (prestera_flow_block_is_bound(block))
+ prestera_acl_ruleset_unbind(block->ruleset_zero, port);
+
+ kfree(binding);
+ return 0;
}
static struct prestera_flow_block *
@@ -78,7 +174,7 @@ prestera_flow_block_get(struct prestera_switch *sw,
block_cb = flow_block_cb_lookup(f->block,
prestera_flow_block_cb, sw);
if (!block_cb) {
- block = prestera_acl_block_create(sw, f->net);
+ block = prestera_flow_block_create(sw, f->net);
if (!block)
return ERR_PTR(-ENOMEM);
@@ -86,7 +182,7 @@ prestera_flow_block_get(struct prestera_switch *sw,
sw, block,
prestera_flow_block_release);
if (IS_ERR(block_cb)) {
- prestera_acl_block_destroy(block);
+ prestera_flow_block_destroy(block);
return ERR_CAST(block_cb);
}
@@ -110,7 +206,7 @@ static void prestera_flow_block_put(struct prestera_flow_block *block)
return;
flow_block_cb_free(block_cb);
- prestera_acl_block_destroy(block);
+ prestera_flow_block_destroy(block);
}
static int prestera_setup_flow_block_bind(struct prestera_port *port,
@@ -128,7 +224,7 @@ static int prestera_setup_flow_block_bind(struct prestera_port *port,
block_cb = block->block_cb;
- err = prestera_acl_block_bind(block, port);
+ err = prestera_flow_block_bind(block, port);
if (err)
goto err_block_bind;
@@ -162,7 +258,7 @@ static void prestera_setup_flow_block_unbind(struct prestera_port *port,
prestera_span_destroy(block);
- err = prestera_acl_block_unbind(block, port);
+ err = prestera_flow_block_unbind(block, port);
if (err)
goto error;
diff --git a/drivers/net/ethernet/marvell/prestera/prestera_flow.h b/drivers/net/ethernet/marvell/prestera/prestera_flow.h
index 467c703..1ea5b74 100644
--- a/drivers/net/ethernet/marvell/prestera/prestera_flow.h
+++ b/drivers/net/ethernet/marvell/prestera/prestera_flow.h
@@ -7,6 +7,24 @@
#include <net/flow_offload.h>
struct prestera_port;
+struct prestera_switch;
+struct prestera_flower_template;
+
+struct prestera_flow_block_binding {
+ struct list_head list;
+ struct prestera_port *port;
+ int span_id;
+};
+
+struct prestera_flow_block {
+ struct list_head binding_list;
+ struct prestera_switch *sw;
+ struct net *net;
+ struct prestera_acl_ruleset *ruleset_zero;
+ struct flow_block_cb *block_cb;
+ struct prestera_flower_template *tmplt;
+ unsigned int rule_count;
+};
int prestera_flow_block_setup(struct prestera_port *port,
struct flow_block_offload *f);
diff --git a/drivers/net/ethernet/marvell/prestera/prestera_flower.c b/drivers/net/ethernet/marvell/prestera/prestera_flower.c
index e571ba0..19c1417 100644
--- a/drivers/net/ethernet/marvell/prestera/prestera_flower.c
+++ b/drivers/net/ethernet/marvell/prestera/prestera_flower.c
@@ -3,42 +3,61 @@
#include "prestera.h"
#include "prestera_acl.h"
+#include "prestera_flow.h"
#include "prestera_flower.h"
+struct prestera_flower_template {
+ struct prestera_acl_ruleset *ruleset;
+};
+
+void prestera_flower_template_cleanup(struct prestera_flow_block *block)
+{
+ if (block->tmplt) {
+ /* put the reference to the ruleset kept in create */
+ prestera_acl_ruleset_put(block->tmplt->ruleset);
+ kfree(block->tmplt);
+ block->tmplt = NULL;
+ return;
+ }
+}
+
static int prestera_flower_parse_actions(struct prestera_flow_block *block,
struct prestera_acl_rule *rule,
struct flow_action *flow_action,
struct netlink_ext_ack *extack)
{
- struct prestera_acl_rule_action_entry a_entry;
const struct flow_action_entry *act;
- int err, i;
+ int i;
+ /* whole struct (rule->re_arg) must be initialized with 0 */
if (!flow_action_has_entries(flow_action))
return 0;
flow_action_for_each(i, act, flow_action) {
- memset(&a_entry, 0, sizeof(a_entry));
-
switch (act->id) {
case FLOW_ACTION_ACCEPT:
- a_entry.id = PRESTERA_ACL_RULE_ACTION_ACCEPT;
+ if (rule->re_arg.accept.valid)
+ return -EEXIST;
+
+ rule->re_arg.accept.valid = 1;
break;
case FLOW_ACTION_DROP:
- a_entry.id = PRESTERA_ACL_RULE_ACTION_DROP;
+ if (rule->re_arg.drop.valid)
+ return -EEXIST;
+
+ rule->re_arg.drop.valid = 1;
break;
case FLOW_ACTION_TRAP:
- a_entry.id = PRESTERA_ACL_RULE_ACTION_TRAP;
+ if (rule->re_arg.trap.valid)
+ return -EEXIST;
+
+ rule->re_arg.trap.valid = 1;
break;
default:
NL_SET_ERR_MSG_MOD(extack, "Unsupported action");
pr_err("Unsupported action\n");
return -EOPNOTSUPP;
}
-
- err = prestera_acl_rule_action_add(rule, &a_entry);
- if (err)
- return err;
}
return 0;
@@ -47,12 +66,12 @@ static int prestera_flower_parse_actions(struct prestera_flow_block *block,
static int prestera_flower_parse_meta(struct prestera_acl_rule *rule,
struct flow_cls_offload *f,
struct prestera_flow_block *block)
-{
- struct flow_rule *f_rule = flow_cls_offload_flow_rule(f);
- struct prestera_acl_rule_match_entry m_entry = {0};
+{ struct flow_rule *f_rule = flow_cls_offload_flow_rule(f);
+ struct prestera_acl_match *r_match = &rule->re_key.match;
+ struct prestera_port *port;
struct net_device *ingress_dev;
struct flow_match_meta match;
- struct prestera_port *port;
+ __be16 key, mask;
flow_rule_match_meta(f_rule, &match);
if (match.mask->ingress_ifindex != 0xFFFFFFFF) {
@@ -61,7 +80,7 @@ static int prestera_flower_parse_meta(struct prestera_acl_rule *rule,
return -EINVAL;
}
- ingress_dev = __dev_get_by_index(prestera_acl_block_net(block),
+ ingress_dev = __dev_get_by_index(block->net,
match.key->ingress_ifindex);
if (!ingress_dev) {
NL_SET_ERR_MSG_MOD(f->common.extack,
@@ -76,22 +95,28 @@ static int prestera_flower_parse_meta(struct prestera_acl_rule *rule,
}
port = netdev_priv(ingress_dev);
- m_entry.type = PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_PORT;
- m_entry.keymask.u64.key = port->hw_id | ((u64)port->dev_id << 32);
- m_entry.keymask.u64.mask = ~(u64)0;
+ mask = htons(0x1FFF);
+ key = htons(port->hw_id);
+ rule_match_set(r_match->key, SYS_PORT, key);
+ rule_match_set(r_match->mask, SYS_PORT, mask);
- return prestera_acl_rule_match_add(rule, &m_entry);
+ mask = htons(0x1FF);
+ key = htons(port->dev_id);
+ rule_match_set(r_match->key, SYS_DEV, key);
+ rule_match_set(r_match->mask, SYS_DEV, mask);
+
+ return 0;
+
}
static int prestera_flower_parse(struct prestera_flow_block *block,
struct prestera_acl_rule *rule,
struct flow_cls_offload *f)
-{
- struct flow_rule *f_rule = flow_cls_offload_flow_rule(f);
+{ struct flow_rule *f_rule = flow_cls_offload_flow_rule(f);
struct flow_dissector *dissector = f_rule->match.dissector;
- struct prestera_acl_rule_match_entry m_entry;
- u16 n_proto_mask = 0;
- u16 n_proto_key = 0;
+ struct prestera_acl_match *r_match = &rule->re_key.match;
+ __be16 n_proto_mask = 0;
+ __be16 n_proto_key = 0;
u16 addr_type = 0;
u8 ip_proto = 0;
int err;
@@ -129,32 +154,19 @@ static int prestera_flower_parse(struct prestera_flow_block *block,
struct flow_match_basic match;
flow_rule_match_basic(f_rule, &match);
- n_proto_key = ntohs(match.key->n_proto);
- n_proto_mask = ntohs(match.mask->n_proto);
+ n_proto_key = match.key->n_proto;
+ n_proto_mask = match.mask->n_proto;
- if (n_proto_key == ETH_P_ALL) {
+ if (ntohs(match.key->n_proto) == ETH_P_ALL) {
n_proto_key = 0;
n_proto_mask = 0;
}
- /* add eth type key,mask */
- memset(&m_entry, 0, sizeof(m_entry));
- m_entry.type = PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_ETH_TYPE;
- m_entry.keymask.u16.key = n_proto_key;
- m_entry.keymask.u16.mask = n_proto_mask;
- err = prestera_acl_rule_match_add(rule, &m_entry);
- if (err)
- return err;
+ rule_match_set(r_match->key, ETH_TYPE, n_proto_key);
+ rule_match_set(r_match->mask, ETH_TYPE, n_proto_mask);
- /* add ip proto key,mask */
- memset(&m_entry, 0, sizeof(m_entry));
- m_entry.type = PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_IP_PROTO;
- m_entry.keymask.u8.key = match.key->ip_proto;
- m_entry.keymask.u8.mask = match.mask->ip_proto;
- err = prestera_acl_rule_match_add(rule, &m_entry);
- if (err)
- return err;
-
+ rule_match_set(r_match->key, IP_PROTO, match.key->ip_proto);
+ rule_match_set(r_match->mask, IP_PROTO, match.mask->ip_proto);
ip_proto = match.key->ip_proto;
}
@@ -163,27 +175,27 @@ static int prestera_flower_parse(struct prestera_flow_block *block,
flow_rule_match_eth_addrs(f_rule, &match);
- /* add ethernet dst key,mask */
- memset(&m_entry, 0, sizeof(m_entry));
- m_entry.type = PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_ETH_DMAC;
- memcpy(&m_entry.keymask.mac.key,
- &match.key->dst, sizeof(match.key->dst));
- memcpy(&m_entry.keymask.mac.mask,
- &match.mask->dst, sizeof(match.mask->dst));
- err = prestera_acl_rule_match_add(rule, &m_entry);
- if (err)
- return err;
+ /* DA key, mask */
+ rule_match_set_n(r_match->key,
+ ETH_DMAC_0, &match.key->dst[0], 4);
+ rule_match_set_n(r_match->key,
+ ETH_DMAC_1, &match.key->dst[4], 2);
- /* add ethernet src key,mask */
- memset(&m_entry, 0, sizeof(m_entry));
- m_entry.type = PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_ETH_SMAC;
- memcpy(&m_entry.keymask.mac.key,
- &match.key->src, sizeof(match.key->src));
- memcpy(&m_entry.keymask.mac.mask,
- &match.mask->src, sizeof(match.mask->src));
- err = prestera_acl_rule_match_add(rule, &m_entry);
- if (err)
- return err;
+ rule_match_set_n(r_match->mask,
+ ETH_DMAC_0, &match.mask->dst[0], 4);
+ rule_match_set_n(r_match->mask,
+ ETH_DMAC_1, &match.mask->dst[4], 2);
+
+ /* SA key, mask */
+ rule_match_set_n(r_match->key,
+ ETH_SMAC_0, &match.key->src[0], 4);
+ rule_match_set_n(r_match->key,
+ ETH_SMAC_1, &match.key->src[4], 2);
+
+ rule_match_set_n(r_match->mask,
+ ETH_SMAC_0, &match.mask->src[0], 4);
+ rule_match_set_n(r_match->mask,
+ ETH_SMAC_1, &match.mask->src[4], 2);
}
if (addr_type == FLOW_DISSECTOR_KEY_IPV4_ADDRS) {
@@ -191,25 +203,11 @@ static int prestera_flower_parse(struct prestera_flow_block *block,
flow_rule_match_ipv4_addrs(f_rule, &match);
- memset(&m_entry, 0, sizeof(m_entry));
- m_entry.type = PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_IP_SRC;
- memcpy(&m_entry.keymask.u32.key,
- &match.key->src, sizeof(match.key->src));
- memcpy(&m_entry.keymask.u32.mask,
- &match.mask->src, sizeof(match.mask->src));
- err = prestera_acl_rule_match_add(rule, &m_entry);
- if (err)
- return err;
+ rule_match_set(r_match->key, IP_SRC, match.key->src);
+ rule_match_set(r_match->mask, IP_SRC, match.mask->src);
- memset(&m_entry, 0, sizeof(m_entry));
- m_entry.type = PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_IP_DST;
- memcpy(&m_entry.keymask.u32.key,
- &match.key->dst, sizeof(match.key->dst));
- memcpy(&m_entry.keymask.u32.mask,
- &match.mask->dst, sizeof(match.mask->dst));
- err = prestera_acl_rule_match_add(rule, &m_entry);
- if (err)
- return err;
+ rule_match_set(r_match->key, IP_DST, match.key->dst);
+ rule_match_set(r_match->mask, IP_DST, match.mask->dst);
}
if (flow_rule_match_key(f_rule, FLOW_DISSECTOR_KEY_PORTS)) {
@@ -224,21 +222,11 @@ static int prestera_flower_parse(struct prestera_flow_block *block,
flow_rule_match_ports(f_rule, &match);
- memset(&m_entry, 0, sizeof(m_entry));
- m_entry.type = PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_L4_PORT_SRC;
- m_entry.keymask.u16.key = ntohs(match.key->src);
- m_entry.keymask.u16.mask = ntohs(match.mask->src);
- err = prestera_acl_rule_match_add(rule, &m_entry);
- if (err)
- return err;
+ rule_match_set(r_match->key, L4_PORT_SRC, match.key->src);
+ rule_match_set(r_match->mask, L4_PORT_SRC, match.mask->src);
- memset(&m_entry, 0, sizeof(m_entry));
- m_entry.type = PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_L4_PORT_DST;
- m_entry.keymask.u16.key = ntohs(match.key->dst);
- m_entry.keymask.u16.mask = ntohs(match.mask->dst);
- err = prestera_acl_rule_match_add(rule, &m_entry);
- if (err)
- return err;
+ rule_match_set(r_match->key, L4_PORT_DST, match.key->dst);
+ rule_match_set(r_match->mask, L4_PORT_DST, match.mask->dst);
}
if (flow_rule_match_key(f_rule, FLOW_DISSECTOR_KEY_VLAN)) {
@@ -247,22 +235,15 @@ static int prestera_flower_parse(struct prestera_flow_block *block,
flow_rule_match_vlan(f_rule, &match);
if (match.mask->vlan_id != 0) {
- memset(&m_entry, 0, sizeof(m_entry));
- m_entry.type = PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_VLAN_ID;
- m_entry.keymask.u16.key = match.key->vlan_id;
- m_entry.keymask.u16.mask = match.mask->vlan_id;
- err = prestera_acl_rule_match_add(rule, &m_entry);
- if (err)
- return err;
+ __be16 key = cpu_to_be16(match.key->vlan_id);
+ __be16 mask = cpu_to_be16(match.mask->vlan_id);
+
+ rule_match_set(r_match->key, VLAN_ID, key);
+ rule_match_set(r_match->mask, VLAN_ID, mask);
}
- memset(&m_entry, 0, sizeof(m_entry));
- m_entry.type = PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_VLAN_TPID;
- m_entry.keymask.u16.key = ntohs(match.key->vlan_tpid);
- m_entry.keymask.u16.mask = ntohs(match.mask->vlan_tpid);
- err = prestera_acl_rule_match_add(rule, &m_entry);
- if (err)
- return err;
+ rule_match_set(r_match->key, VLAN_TPID, match.key->vlan_tpid);
+ rule_match_set(r_match->mask, VLAN_TPID, match.mask->vlan_tpid);
}
if (flow_rule_match_key(f_rule, FLOW_DISSECTOR_KEY_ICMP)) {
@@ -270,90 +251,164 @@ static int prestera_flower_parse(struct prestera_flow_block *block,
flow_rule_match_icmp(f_rule, &match);
- memset(&m_entry, 0, sizeof(m_entry));
- m_entry.type = PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_ICMP_TYPE;
- m_entry.keymask.u8.key = match.key->type;
- m_entry.keymask.u8.mask = match.mask->type;
- err = prestera_acl_rule_match_add(rule, &m_entry);
- if (err)
- return err;
+ rule_match_set(r_match->key, ICMP_TYPE, match.key->type);
+ rule_match_set(r_match->mask, ICMP_TYPE, match.mask->type);
- memset(&m_entry, 0, sizeof(m_entry));
- m_entry.type = PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_ICMP_CODE;
- m_entry.keymask.u8.key = match.key->code;
- m_entry.keymask.u8.mask = match.mask->code;
- err = prestera_acl_rule_match_add(rule, &m_entry);
- if (err)
- return err;
+ rule_match_set(r_match->key, ICMP_CODE, match.key->code);
+ rule_match_set(r_match->mask, ICMP_CODE, match.mask->code);
}
- return prestera_flower_parse_actions(block, rule,
- &f->rule->action,
+ return prestera_flower_parse_actions(block, rule, &f->rule->action,
f->common.extack);
}
int prestera_flower_replace(struct prestera_flow_block *block,
struct flow_cls_offload *f)
{
- struct prestera_switch *sw = prestera_acl_block_sw(block);
+ struct prestera_acl_ruleset *ruleset;
+ struct prestera_acl *acl = block->sw->acl;
struct prestera_acl_rule *rule;
int err;
- rule = prestera_acl_rule_create(block, f->cookie);
- if (IS_ERR(rule))
- return PTR_ERR(rule);
+ ruleset = prestera_acl_ruleset_get(acl, block);
+ if (IS_ERR(ruleset))
+ return PTR_ERR(ruleset);
+
+ /* increments the ruleset reference */
+ rule = prestera_acl_rule_create(ruleset, f->cookie);
+ if (IS_ERR(rule)) {
+ err = PTR_ERR(rule);
+ goto err_rule_create;
+ }
err = prestera_flower_parse(block, rule, f);
if (err)
- goto err_flower_parse;
+ goto err_rule_add;
- err = prestera_acl_rule_add(sw, rule);
+ if (!prestera_acl_ruleset_is_offload(ruleset)) {
+ err = prestera_acl_ruleset_offload(ruleset);
+ if (err)
+ goto err_ruleset_offload;
+ }
+
+ err = prestera_acl_rule_add(block->sw, rule);
if (err)
goto err_rule_add;
+ prestera_acl_ruleset_put(ruleset);
return 0;
+err_ruleset_offload:
err_rule_add:
-err_flower_parse:
prestera_acl_rule_destroy(rule);
+err_rule_create:
+ prestera_acl_ruleset_put(ruleset);
return err;
}
void prestera_flower_destroy(struct prestera_flow_block *block,
struct flow_cls_offload *f)
{
+ struct prestera_acl_ruleset *ruleset;
struct prestera_acl_rule *rule;
- struct prestera_switch *sw;
- rule = prestera_acl_rule_lookup(prestera_acl_block_ruleset_get(block),
- f->cookie);
+ ruleset = prestera_acl_ruleset_lookup(block->sw->acl, block);
+ if (IS_ERR(ruleset))
+ return;
+
+ rule = prestera_acl_rule_lookup(ruleset, f->cookie);
if (rule) {
- sw = prestera_acl_block_sw(block);
- prestera_acl_rule_del(sw, rule);
+ prestera_acl_rule_del(block->sw, rule);
prestera_acl_rule_destroy(rule);
}
+ prestera_acl_ruleset_put(ruleset);
+
+}
+
+int prestera_flower_tmplt_create(struct prestera_flow_block *block,
+ struct flow_cls_offload *f)
+{
+ struct prestera_flower_template *template;
+ struct prestera_acl_ruleset *ruleset;
+ struct prestera_acl_rule rule;
+ int err;
+
+ memset(&rule, 0, sizeof(rule));
+ err = prestera_flower_parse(block, &rule, f);
+ if (err)
+ return err;
+
+ template = kmalloc(sizeof(*template), GFP_KERNEL);
+ if (!template) {
+ err = -ENOMEM;
+ goto err_malloc;
+ }
+
+ prestera_acl_rule_keymask_pcl_id_set(&rule, 0);
+ ruleset = prestera_acl_ruleset_get(block->sw->acl, block);
+ if (IS_ERR_OR_NULL(ruleset)) {
+ err = -EINVAL;
+ goto err_ruleset_get;
+ }
+
+ /* preserve keymask/template to this ruleset */
+ prestera_acl_ruleset_keymask_set(ruleset, rule.re_key.match.mask);
+
+ /* skip error, as it is not possible to reject template operation,
+ * so, keep the reference to the ruleset for rules to be added
+ * to that ruleset later. In case of offload fail, the ruleset
+ * will be offloaded again during adding a new rule. Also,
+ * unlikly possble that ruleset is already offloaded at this staage.
+ */
+ prestera_acl_ruleset_offload(ruleset);
+
+ /* keep the reference to the ruleset */
+ template->ruleset = ruleset;
+ block->tmplt = template;
+ return 0;
+
+err_ruleset_get:
+ kfree(template);
+err_malloc:
+ NL_SET_ERR_MSG_MOD(f->common.extack, "Create chain template failed");
+ return err;
+}
+
+void prestera_flower_tmplt_destroy(struct prestera_flow_block *block,
+ struct flow_cls_offload *f)
+{
+ prestera_flower_template_cleanup(block);
}
int prestera_flower_stats(struct prestera_flow_block *block,
struct flow_cls_offload *f)
{
- struct prestera_switch *sw = prestera_acl_block_sw(block);
+ struct prestera_acl_ruleset *ruleset;
struct prestera_acl_rule *rule;
u64 packets;
u64 lastuse;
u64 bytes;
int err;
- rule = prestera_acl_rule_lookup(prestera_acl_block_ruleset_get(block),
- f->cookie);
- if (!rule)
- return -EINVAL;
+ ruleset = prestera_acl_ruleset_lookup(block->sw->acl, block);
+ if (IS_ERR(ruleset))
+ return PTR_ERR(ruleset);
- err = prestera_acl_rule_get_stats(sw, rule, &packets, &bytes, &lastuse);
+ rule = prestera_acl_rule_lookup(ruleset, f->cookie);
+ if (!rule) {
+ err = -EINVAL;
+ goto err_rule_get_stats;
+ }
+
+ err = prestera_acl_rule_get_stats(block->sw->acl, rule, &packets,
+ &bytes, &lastuse);
if (err)
- return err;
+ goto err_rule_get_stats;
flow_stats_update(&f->stats, bytes, packets, 0, lastuse,
- FLOW_ACTION_HW_STATS_IMMEDIATE);
- return 0;
+ FLOW_ACTION_HW_STATS_DELAYED);
+
+err_rule_get_stats:
+ prestera_acl_ruleset_put(ruleset);
+ return err;
}
diff --git a/drivers/net/ethernet/marvell/prestera/prestera_flower.h b/drivers/net/ethernet/marvell/prestera/prestera_flower.h
index 91e045e..dc3aa42 100644
--- a/drivers/net/ethernet/marvell/prestera/prestera_flower.h
+++ b/drivers/net/ethernet/marvell/prestera/prestera_flower.h
@@ -1,11 +1,12 @@
/* SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0 */
-/* Copyright (c) 2020 Marvell International Ltd. All rights reserved. */
+/* Copyright (c) 2020-2021 Marvell International Ltd. All rights reserved. */
#ifndef _PRESTERA_FLOWER_H_
#define _PRESTERA_FLOWER_H_
#include <net/pkt_cls.h>
+struct prestera_switch;
struct prestera_flow_block;
int prestera_flower_replace(struct prestera_flow_block *block,
@@ -14,5 +15,10 @@ void prestera_flower_destroy(struct prestera_flow_block *block,
struct flow_cls_offload *f);
int prestera_flower_stats(struct prestera_flow_block *block,
struct flow_cls_offload *f);
+int prestera_flower_tmplt_create(struct prestera_flow_block *block,
+ struct flow_cls_offload *f);
+void prestera_flower_tmplt_destroy(struct prestera_flow_block *block,
+ struct flow_cls_offload *f);
+void prestera_flower_template_cleanup(struct prestera_flow_block *block);
#endif /* _PRESTERA_FLOWER_H_ */
diff --git a/drivers/net/ethernet/marvell/prestera/prestera_hw.c b/drivers/net/ethernet/marvell/prestera/prestera_hw.c
index 9b8b1ed..51fc841 100644
--- a/drivers/net/ethernet/marvell/prestera/prestera_hw.c
+++ b/drivers/net/ethernet/marvell/prestera/prestera_hw.c
@@ -9,6 +9,7 @@
#include "prestera.h"
#include "prestera_hw.h"
#include "prestera_acl.h"
+#include "prestera_counter.h"
#define PRESTERA_SWITCH_INIT_TIMEOUT_MS (30 * 1000)
@@ -38,13 +39,24 @@ enum prestera_cmd_type_t {
PRESTERA_CMD_TYPE_BRIDGE_PORT_ADD = 0x402,
PRESTERA_CMD_TYPE_BRIDGE_PORT_DELETE = 0x403,
- PRESTERA_CMD_TYPE_ACL_RULE_ADD = 0x500,
- PRESTERA_CMD_TYPE_ACL_RULE_DELETE = 0x501,
- PRESTERA_CMD_TYPE_ACL_RULE_STATS_GET = 0x510,
- PRESTERA_CMD_TYPE_ACL_RULESET_CREATE = 0x520,
- PRESTERA_CMD_TYPE_ACL_RULESET_DELETE = 0x521,
- PRESTERA_CMD_TYPE_ACL_PORT_BIND = 0x530,
- PRESTERA_CMD_TYPE_ACL_PORT_UNBIND = 0x531,
+ PRESTERA_CMD_TYPE_COUNTER_GET = 0x510,
+ PRESTERA_CMD_TYPE_COUNTER_ABORT = 0x511,
+ PRESTERA_CMD_TYPE_COUNTER_TRIGGER = 0x512,
+ PRESTERA_CMD_TYPE_COUNTER_BLOCK_GET = 0x513,
+ PRESTERA_CMD_TYPE_COUNTER_BLOCK_RELEASE = 0x514,
+ PRESTERA_CMD_TYPE_COUNTER_CLEAR = 0x515,
+
+ PRESTERA_CMD_TYPE_VTCAM_CREATE = 0x540,
+ PRESTERA_CMD_TYPE_VTCAM_DESTROY = 0x541,
+ PRESTERA_CMD_TYPE_VTCAM_RULE_ADD = 0x550,
+ PRESTERA_CMD_TYPE_VTCAM_RULE_DELETE = 0x551,
+ PRESTERA_CMD_TYPE_VTCAM_IFACE_BIND = 0x560,
+ PRESTERA_CMD_TYPE_VTCAM_IFACE_UNBIND = 0x561,
+
+ PRESTERA_CMD_TYPE_ROUTER_RIF_CREATE = 0x600,
+ PRESTERA_CMD_TYPE_ROUTER_RIF_DELETE = 0x601,
+ PRESTERA_CMD_TYPE_ROUTER_VR_CREATE = 0x630,
+ PRESTERA_CMD_TYPE_ROUTER_VR_DELETE = 0x631,
PRESTERA_CMD_TYPE_RXTX_INIT = 0x800,
@@ -359,76 +371,84 @@ struct prestera_msg_bridge_resp {
u8 pad[2];
};
-struct prestera_msg_acl_action {
- __le32 id;
- __le32 reserved[5];
+struct prestera_msg_vtcam_create_req {
+ struct prestera_msg_cmd cmd;
+ __le32 keymask[__PRESTERA_ACL_RULE_MATCH_TYPE_MAX];
+ u8 direction;
+ u8 lookup;
+ u8 pad[2];
};
-struct prestera_msg_acl_match {
- __le32 type;
+struct prestera_msg_vtcam_destroy_req {
+ struct prestera_msg_cmd cmd;
+ __le32 vtcam_id;
+};
+
+struct prestera_msg_vtcam_rule_add_req {
+ struct prestera_msg_cmd cmd;
+ __le32 key[__PRESTERA_ACL_RULE_MATCH_TYPE_MAX];
+ __le32 keymask[__PRESTERA_ACL_RULE_MATCH_TYPE_MAX];
+ __le32 vtcam_id;
+ __le32 prio;
+ __le32 n_act;
+};
+
+struct prestera_msg_vtcam_rule_del_req {
+ struct prestera_msg_cmd cmd;
+ __le32 vtcam_id;
+ __le32 id;
+};
+
+struct prestera_msg_vtcam_bind_req {
+ struct prestera_msg_cmd cmd;
+ union {
+ struct {
+ __le32 hw_id;
+ __le32 dev_id;
+ } port;
+ __le32 index;
+ };
+ __le32 vtcam_id;
+ __le16 pcl_id;
+ __le16 type;
+};
+
+struct prestera_msg_vtcam_resp {
+ struct prestera_msg_ret ret;
+ __le32 vtcam_id;
+ __le32 rule_id;
+};
+
+struct prestera_msg_acl_action {
+ __le32 id;
__le32 __reserved;
union {
struct {
- u8 key;
- u8 mask;
- } u8;
- struct {
- __le16 key;
- __le16 mask;
- } u16;
- struct {
- __le32 key;
- __le32 mask;
- } u32;
- struct {
- __le64 key;
- __le64 mask;
- } u64;
- struct {
- u8 key[ETH_ALEN];
- u8 mask[ETH_ALEN];
- } mac;
- } keymask;
+ __le32 id;
+ } count;
+ __le32 reserved[6];
+ };
};
-struct prestera_msg_acl_rule_req {
+struct prestera_msg_counter_req {
struct prestera_msg_cmd cmd;
- __le32 id;
- __le32 priority;
- __le16 ruleset_id;
- u8 n_actions;
- u8 n_matches;
+ __le32 client;
+ __le32 block_id;
+ __le32 num_counters;
};
-struct prestera_msg_acl_rule_resp {
- struct prestera_msg_ret ret;
- __le32 id;
-};
-
-struct prestera_msg_acl_rule_stats_resp {
- struct prestera_msg_ret ret;
+struct prestera_msg_counter_stats {
__le64 packets;
__le64 bytes;
};
-struct prestera_msg_acl_ruleset_bind_req {
- struct prestera_msg_cmd cmd;
- __le32 port;
- __le32 dev;
- __le16 ruleset_id;
- u8 pad[2];
-};
-
-struct prestera_msg_acl_ruleset_req {
- struct prestera_msg_cmd cmd;
- __le16 id;
- u8 pad[2];
-};
-
-struct prestera_msg_acl_ruleset_resp {
+struct prestera_msg_counter_resp {
struct prestera_msg_ret ret;
- __le16 id;
- u8 pad[2];
+ __le32 block_id;
+ __le32 offset;
+ __le32 num_counters;
+ __le32 done;
+ struct prestera_msg_counter_stats stats[];
};
struct prestera_msg_span_req {
@@ -465,6 +485,48 @@ struct prestera_msg_rxtx_resp {
__le32 map_addr;
};
+struct prestera_msg_iface {
+ union {
+ struct {
+ __le32 dev;
+ __le32 port;
+ };
+ __le16 lag_id;
+ };
+ __le16 vr_id;
+ __le16 vid;
+ u8 type;
+ u8 __pad[3];
+};
+
+struct prestera_msg_rif_req {
+ struct prestera_msg_cmd cmd;
+ struct prestera_msg_iface iif;
+ __le32 mtu;
+ __le16 rif_id;
+ __le16 __reserved;
+ u8 mac[ETH_ALEN];
+ u8 __pad[2];
+};
+
+struct prestera_msg_rif_resp {
+ struct prestera_msg_ret ret;
+ __le16 rif_id;
+ u8 __pad[2];
+};
+
+struct prestera_msg_vr_req {
+ struct prestera_msg_cmd cmd;
+ __le16 vr_id;
+ u8 __pad[2];
+};
+
+struct prestera_msg_vr_resp {
+ struct prestera_msg_ret ret;
+ __le16 vr_id;
+ u8 __pad[2];
+};
+
struct prestera_msg_lag_req {
struct prestera_msg_cmd cmd;
__le32 port;
@@ -521,14 +583,24 @@ static void prestera_hw_build_tests(void)
BUILD_BUG_ON(sizeof(struct prestera_msg_vlan_req) != 16);
BUILD_BUG_ON(sizeof(struct prestera_msg_fdb_req) != 28);
BUILD_BUG_ON(sizeof(struct prestera_msg_bridge_req) != 16);
- BUILD_BUG_ON(sizeof(struct prestera_msg_acl_rule_req) != 16);
- BUILD_BUG_ON(sizeof(struct prestera_msg_acl_ruleset_bind_req) != 16);
- BUILD_BUG_ON(sizeof(struct prestera_msg_acl_ruleset_req) != 8);
BUILD_BUG_ON(sizeof(struct prestera_msg_span_req) != 16);
BUILD_BUG_ON(sizeof(struct prestera_msg_stp_req) != 16);
BUILD_BUG_ON(sizeof(struct prestera_msg_rxtx_req) != 8);
BUILD_BUG_ON(sizeof(struct prestera_msg_lag_req) != 16);
BUILD_BUG_ON(sizeof(struct prestera_msg_cpu_code_counter_req) != 8);
+ BUILD_BUG_ON(sizeof(struct prestera_msg_vtcam_create_req) != 84);
+ BUILD_BUG_ON(sizeof(struct prestera_msg_vtcam_destroy_req) != 8);
+ BUILD_BUG_ON(sizeof(struct prestera_msg_vtcam_rule_add_req) != 168);
+ BUILD_BUG_ON(sizeof(struct prestera_msg_vtcam_rule_del_req) != 12);
+ BUILD_BUG_ON(sizeof(struct prestera_msg_vtcam_bind_req) != 20);
+ BUILD_BUG_ON(sizeof(struct prestera_msg_acl_action) != 32);
+ BUILD_BUG_ON(sizeof(struct prestera_msg_counter_req) != 16);
+ BUILD_BUG_ON(sizeof(struct prestera_msg_counter_stats) != 16);
+ BUILD_BUG_ON(sizeof(struct prestera_msg_rif_req) != 36);
+ BUILD_BUG_ON(sizeof(struct prestera_msg_vr_req) != 8);
+
+ /* structure that are part of req/resp fw messages */
+ BUILD_BUG_ON(sizeof(struct prestera_msg_iface) != 16);
/* check responses */
BUILD_BUG_ON(sizeof(struct prestera_msg_common_resp) != 8);
@@ -537,11 +609,12 @@ static void prestera_hw_build_tests(void)
BUILD_BUG_ON(sizeof(struct prestera_msg_port_stats_resp) != 248);
BUILD_BUG_ON(sizeof(struct prestera_msg_port_info_resp) != 20);
BUILD_BUG_ON(sizeof(struct prestera_msg_bridge_resp) != 12);
- BUILD_BUG_ON(sizeof(struct prestera_msg_acl_rule_resp) != 12);
- BUILD_BUG_ON(sizeof(struct prestera_msg_acl_rule_stats_resp) != 24);
- BUILD_BUG_ON(sizeof(struct prestera_msg_acl_ruleset_resp) != 12);
BUILD_BUG_ON(sizeof(struct prestera_msg_span_resp) != 12);
BUILD_BUG_ON(sizeof(struct prestera_msg_rxtx_resp) != 12);
+ BUILD_BUG_ON(sizeof(struct prestera_msg_vtcam_resp) != 16);
+ BUILD_BUG_ON(sizeof(struct prestera_msg_counter_resp) != 24);
+ BUILD_BUG_ON(sizeof(struct prestera_msg_rif_resp) != 12);
+ BUILD_BUG_ON(sizeof(struct prestera_msg_vr_resp) != 12);
/* check events */
BUILD_BUG_ON(sizeof(struct prestera_msg_event_port) != 20);
@@ -1044,225 +1117,162 @@ static void prestera_hw_remote_fc_to_eth(u8 fc, bool *pause, bool *asym_pause)
}
}
-int prestera_hw_acl_ruleset_create(struct prestera_switch *sw, u16 *ruleset_id)
+int prestera_hw_vtcam_create(struct prestera_switch *sw,
+ u8 lookup, const u32 *keymask, u32 *vtcam_id,
+ enum prestera_hw_vtcam_direction_t dir)
{
- struct prestera_msg_acl_ruleset_resp resp;
- struct prestera_msg_acl_ruleset_req req;
int err;
+ struct prestera_msg_vtcam_resp resp;
+ struct prestera_msg_vtcam_create_req req = {
+ .lookup = lookup,
+ .direction = dir,
+ };
- err = prestera_cmd_ret(sw, PRESTERA_CMD_TYPE_ACL_RULESET_CREATE,
+ if (keymask)
+ memcpy(req.keymask, keymask, sizeof(req.keymask));
+ else
+ memset(req.keymask, 0, sizeof(req.keymask));
+
+ err = prestera_cmd_ret(sw, PRESTERA_CMD_TYPE_VTCAM_CREATE,
&req.cmd, sizeof(req), &resp.ret, sizeof(resp));
if (err)
return err;
- *ruleset_id = __le16_to_cpu(resp.id);
-
+ *vtcam_id = __le32_to_cpu(resp.vtcam_id);
return 0;
}
-int prestera_hw_acl_ruleset_del(struct prestera_switch *sw, u16 ruleset_id)
+int prestera_hw_vtcam_destroy(struct prestera_switch *sw, u32 vtcam_id)
{
- struct prestera_msg_acl_ruleset_req req = {
- .id = __cpu_to_le16(ruleset_id),
+ struct prestera_msg_vtcam_destroy_req req = {
+ .vtcam_id = __cpu_to_le32(vtcam_id),
};
- return prestera_cmd(sw, PRESTERA_CMD_TYPE_ACL_RULESET_DELETE,
+ return prestera_cmd(sw, PRESTERA_CMD_TYPE_VTCAM_DESTROY,
&req.cmd, sizeof(req));
}
-static int prestera_hw_acl_actions_put(struct prestera_msg_acl_action *action,
- struct prestera_acl_rule *rule)
+static int
+prestera_acl_rule_add_put_action(struct prestera_msg_acl_action *action,
+ struct prestera_acl_hw_action_info *info)
{
- struct list_head *a_list = prestera_acl_rule_action_list_get(rule);
- struct prestera_acl_rule_action_entry *a_entry;
- int i = 0;
+ action->id = __cpu_to_le32(info->id);
- list_for_each_entry(a_entry, a_list, list) {
- action[i].id = __cpu_to_le32(a_entry->id);
-
- switch (a_entry->id) {
- case PRESTERA_ACL_RULE_ACTION_ACCEPT:
- case PRESTERA_ACL_RULE_ACTION_DROP:
- case PRESTERA_ACL_RULE_ACTION_TRAP:
- /* just rule action id, no specific data */
- break;
- default:
- return -EINVAL;
- }
-
- i++;
+ switch (info->id) {
+ case PRESTERA_ACL_RULE_ACTION_ACCEPT:
+ case PRESTERA_ACL_RULE_ACTION_DROP:
+ case PRESTERA_ACL_RULE_ACTION_TRAP:
+ /* just rule action id, no specific data */
+ break;
+ case PRESTERA_ACL_RULE_ACTION_COUNT:
+ action->count.id = __cpu_to_le32(info->count.id);
+ break;
+ default:
+ return -EINVAL;
}
return 0;
}
-static int prestera_hw_acl_matches_put(struct prestera_msg_acl_match *match,
- struct prestera_acl_rule *rule)
+int prestera_hw_vtcam_rule_add(struct prestera_switch *sw,
+ u32 vtcam_id, u32 prio, void *key, void *keymask,
+ struct prestera_acl_hw_action_info *act,
+ u8 n_act, u32 *rule_id)
{
- struct list_head *m_list = prestera_acl_rule_match_list_get(rule);
- struct prestera_acl_rule_match_entry *m_entry;
- int i = 0;
-
- list_for_each_entry(m_entry, m_list, list) {
- match[i].type = __cpu_to_le32(m_entry->type);
-
- switch (m_entry->type) {
- case PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_ETH_TYPE:
- case PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_L4_PORT_SRC:
- case PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_L4_PORT_DST:
- case PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_VLAN_ID:
- case PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_VLAN_TPID:
- match[i].keymask.u16.key =
- __cpu_to_le16(m_entry->keymask.u16.key);
- match[i].keymask.u16.mask =
- __cpu_to_le16(m_entry->keymask.u16.mask);
- break;
- case PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_ICMP_TYPE:
- case PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_ICMP_CODE:
- case PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_IP_PROTO:
- match[i].keymask.u8.key = m_entry->keymask.u8.key;
- match[i].keymask.u8.mask = m_entry->keymask.u8.mask;
- break;
- case PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_ETH_SMAC:
- case PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_ETH_DMAC:
- memcpy(match[i].keymask.mac.key,
- m_entry->keymask.mac.key,
- sizeof(match[i].keymask.mac.key));
- memcpy(match[i].keymask.mac.mask,
- m_entry->keymask.mac.mask,
- sizeof(match[i].keymask.mac.mask));
- break;
- case PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_IP_SRC:
- case PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_IP_DST:
- case PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_L4_PORT_RANGE_SRC:
- case PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_L4_PORT_RANGE_DST:
- match[i].keymask.u32.key =
- __cpu_to_le32(m_entry->keymask.u32.key);
- match[i].keymask.u32.mask =
- __cpu_to_le32(m_entry->keymask.u32.mask);
- break;
- case PRESTERA_ACL_RULE_MATCH_ENTRY_TYPE_PORT:
- match[i].keymask.u64.key =
- __cpu_to_le64(m_entry->keymask.u64.key);
- match[i].keymask.u64.mask =
- __cpu_to_le64(m_entry->keymask.u64.mask);
- break;
- default:
- return -EINVAL;
- }
-
- i++;
- }
-
- return 0;
-}
-
-int prestera_hw_acl_rule_add(struct prestera_switch *sw,
- struct prestera_acl_rule *rule,
- u32 *rule_id)
-{
- struct prestera_msg_acl_action *actions;
- struct prestera_msg_acl_match *matches;
- struct prestera_msg_acl_rule_resp resp;
- struct prestera_msg_acl_rule_req *req;
- u8 n_actions;
- u8 n_matches;
+ struct prestera_msg_acl_action *actions_msg;
+ struct prestera_msg_vtcam_rule_add_req *req;
+ struct prestera_msg_vtcam_resp resp;
void *buff;
u32 size;
int err;
+ u8 i;
- n_actions = prestera_acl_rule_action_len(rule);
- n_matches = prestera_acl_rule_match_len(rule);
-
- size = sizeof(*req) + sizeof(*actions) * n_actions +
- sizeof(*matches) * n_matches;
+ size = sizeof(*req) + sizeof(*actions_msg) * n_act;
buff = kzalloc(size, GFP_KERNEL);
if (!buff)
return -ENOMEM;
req = buff;
- actions = buff + sizeof(*req);
- matches = buff + sizeof(*req) + sizeof(*actions) * n_actions;
-
- /* put acl actions into the message */
- err = prestera_hw_acl_actions_put(actions, rule);
- if (err)
- goto free_buff;
+ req->n_act = __cpu_to_le32(n_act);
+ actions_msg = buff + sizeof(*req);
/* put acl matches into the message */
- err = prestera_hw_acl_matches_put(matches, rule);
- if (err)
- goto free_buff;
+ memcpy(req->key, key, sizeof(req->key));
+ memcpy(req->keymask, keymask, sizeof(req->keymask));
- req->ruleset_id = __cpu_to_le16(prestera_acl_rule_ruleset_id_get(rule));
- req->priority = __cpu_to_le32(prestera_acl_rule_priority_get(rule));
- req->n_actions = prestera_acl_rule_action_len(rule);
- req->n_matches = prestera_acl_rule_match_len(rule);
+ /* put acl actions into the message */
+ for (i = 0; i < n_act; i++) {
+ err = prestera_acl_rule_add_put_action(&actions_msg[i],
+ &act[i]);
+ if (err)
+ goto free_buff;
+ }
- err = prestera_cmd_ret(sw, PRESTERA_CMD_TYPE_ACL_RULE_ADD,
+ req->vtcam_id = __cpu_to_le32(vtcam_id);
+ req->prio = __cpu_to_le32(prio);
+
+ err = prestera_cmd_ret(sw, PRESTERA_CMD_TYPE_VTCAM_RULE_ADD,
&req->cmd, size, &resp.ret, sizeof(resp));
if (err)
goto free_buff;
- *rule_id = __le32_to_cpu(resp.id);
+ *rule_id = __le32_to_cpu(resp.rule_id);
free_buff:
kfree(buff);
return err;
}
-int prestera_hw_acl_rule_del(struct prestera_switch *sw, u32 rule_id)
+int prestera_hw_vtcam_rule_del(struct prestera_switch *sw,
+ u32 vtcam_id, u32 rule_id)
{
- struct prestera_msg_acl_rule_req req = {
+ struct prestera_msg_vtcam_rule_del_req req = {
+ .vtcam_id = __cpu_to_le32(vtcam_id),
.id = __cpu_to_le32(rule_id)
};
- return prestera_cmd(sw, PRESTERA_CMD_TYPE_ACL_RULE_DELETE,
+ return prestera_cmd(sw, PRESTERA_CMD_TYPE_VTCAM_RULE_DELETE,
&req.cmd, sizeof(req));
}
-int prestera_hw_acl_rule_stats_get(struct prestera_switch *sw, u32 rule_id,
- u64 *packets, u64 *bytes)
+int prestera_hw_vtcam_iface_bind(struct prestera_switch *sw,
+ struct prestera_acl_iface *iface,
+ u32 vtcam_id, u16 pcl_id)
{
- struct prestera_msg_acl_rule_stats_resp resp;
- struct prestera_msg_acl_rule_req req = {
- .id = __cpu_to_le32(rule_id)
- };
- int err;
-
- err = prestera_cmd_ret(sw, PRESTERA_CMD_TYPE_ACL_RULE_STATS_GET,
- &req.cmd, sizeof(req), &resp.ret, sizeof(resp));
- if (err)
- return err;
-
- *packets = __le64_to_cpu(resp.packets);
- *bytes = __le64_to_cpu(resp.bytes);
-
- return 0;
-}
-
-int prestera_hw_acl_port_bind(const struct prestera_port *port, u16 ruleset_id)
-{
- struct prestera_msg_acl_ruleset_bind_req req = {
- .port = __cpu_to_le32(port->hw_id),
- .dev = __cpu_to_le32(port->dev_id),
- .ruleset_id = __cpu_to_le16(ruleset_id),
+ struct prestera_msg_vtcam_bind_req req = {
+ .vtcam_id = __cpu_to_le32(vtcam_id),
+ .type = __cpu_to_le16(iface->type),
+ .pcl_id = __cpu_to_le16(pcl_id)
};
- return prestera_cmd(port->sw, PRESTERA_CMD_TYPE_ACL_PORT_BIND,
+ if (iface->type == PRESTERA_ACL_IFACE_TYPE_PORT) {
+ req.port.dev_id = __cpu_to_le32(iface->port->dev_id);
+ req.port.hw_id = __cpu_to_le32(iface->port->hw_id);
+ } else {
+ req.index = __cpu_to_le32(iface->index);
+ }
+
+ return prestera_cmd(sw, PRESTERA_CMD_TYPE_VTCAM_IFACE_BIND,
&req.cmd, sizeof(req));
}
-int prestera_hw_acl_port_unbind(const struct prestera_port *port,
- u16 ruleset_id)
+int prestera_hw_vtcam_iface_unbind(struct prestera_switch *sw,
+ struct prestera_acl_iface *iface,
+ u32 vtcam_id)
{
- struct prestera_msg_acl_ruleset_bind_req req = {
- .port = __cpu_to_le32(port->hw_id),
- .dev = __cpu_to_le32(port->dev_id),
- .ruleset_id = __cpu_to_le16(ruleset_id),
+ struct prestera_msg_vtcam_bind_req req = {
+ .vtcam_id = __cpu_to_le32(vtcam_id),
+ .type = __cpu_to_le16(iface->type)
};
- return prestera_cmd(port->sw, PRESTERA_CMD_TYPE_ACL_PORT_UNBIND,
+ if (iface->type == PRESTERA_ACL_IFACE_TYPE_PORT) {
+ req.port.dev_id = __cpu_to_le32(iface->port->dev_id);
+ req.port.hw_id = __cpu_to_le32(iface->port->hw_id);
+ } else {
+ req.index = __cpu_to_le32(iface->index);
+ }
+
+ return prestera_cmd(sw, PRESTERA_CMD_TYPE_VTCAM_IFACE_UNBIND,
&req.cmd, sizeof(req));
}
@@ -1796,6 +1806,91 @@ int prestera_hw_bridge_port_delete(struct prestera_port *port, u16 bridge_id)
&req.cmd, sizeof(req));
}
+static int prestera_iface_to_msg(struct prestera_iface *iface,
+ struct prestera_msg_iface *msg_if)
+{
+ switch (iface->type) {
+ case PRESTERA_IF_PORT_E:
+ case PRESTERA_IF_VID_E:
+ msg_if->port = __cpu_to_le32(iface->dev_port.port_num);
+ msg_if->dev = __cpu_to_le32(iface->dev_port.hw_dev_num);
+ break;
+ case PRESTERA_IF_LAG_E:
+ msg_if->lag_id = __cpu_to_le16(iface->lag_id);
+ break;
+ default:
+ return -EOPNOTSUPP;
+ }
+
+ msg_if->vr_id = __cpu_to_le16(iface->vr_id);
+ msg_if->vid = __cpu_to_le16(iface->vlan_id);
+ msg_if->type = iface->type;
+ return 0;
+}
+
+int prestera_hw_rif_create(struct prestera_switch *sw,
+ struct prestera_iface *iif, u8 *mac, u16 *rif_id)
+{
+ struct prestera_msg_rif_req req;
+ struct prestera_msg_rif_resp resp;
+ int err;
+
+ memcpy(req.mac, mac, ETH_ALEN);
+
+ err = prestera_iface_to_msg(iif, &req.iif);
+ if (err)
+ return err;
+
+ err = prestera_cmd_ret(sw, PRESTERA_CMD_TYPE_ROUTER_RIF_CREATE,
+ &req.cmd, sizeof(req), &resp.ret, sizeof(resp));
+ if (err)
+ return err;
+
+ *rif_id = __le16_to_cpu(resp.rif_id);
+ return err;
+}
+
+int prestera_hw_rif_delete(struct prestera_switch *sw, u16 rif_id,
+ struct prestera_iface *iif)
+{
+ struct prestera_msg_rif_req req = {
+ .rif_id = __cpu_to_le16(rif_id),
+ };
+ int err;
+
+ err = prestera_iface_to_msg(iif, &req.iif);
+ if (err)
+ return err;
+
+ return prestera_cmd(sw, PRESTERA_CMD_TYPE_ROUTER_RIF_DELETE, &req.cmd,
+ sizeof(req));
+}
+
+int prestera_hw_vr_create(struct prestera_switch *sw, u16 *vr_id)
+{
+ int err;
+ struct prestera_msg_vr_resp resp;
+ struct prestera_msg_vr_req req;
+
+ err = prestera_cmd_ret(sw, PRESTERA_CMD_TYPE_ROUTER_VR_CREATE,
+ &req.cmd, sizeof(req), &resp.ret, sizeof(resp));
+ if (err)
+ return err;
+
+ *vr_id = __le16_to_cpu(resp.vr_id);
+ return err;
+}
+
+int prestera_hw_vr_delete(struct prestera_switch *sw, u16 vr_id)
+{
+ struct prestera_msg_vr_req req = {
+ .vr_id = __cpu_to_le16(vr_id),
+ };
+
+ return prestera_cmd(sw, PRESTERA_CMD_TYPE_ROUTER_VR_DELETE, &req.cmd,
+ sizeof(req));
+}
+
int prestera_hw_rxtx_init(struct prestera_switch *sw,
struct prestera_rxtx_params *params)
{
@@ -1916,3 +2011,100 @@ void prestera_hw_event_handler_unregister(struct prestera_switch *sw,
list_del_rcu(&eh->list);
kfree_rcu(eh, rcu);
}
+
+int prestera_hw_counter_trigger(struct prestera_switch *sw, u32 block_id)
+{
+ struct prestera_msg_counter_req req = {
+ .block_id = __cpu_to_le32(block_id)
+ };
+
+ return prestera_cmd(sw, PRESTERA_CMD_TYPE_COUNTER_TRIGGER,
+ &req.cmd, sizeof(req));
+}
+
+int prestera_hw_counter_abort(struct prestera_switch *sw)
+{
+ struct prestera_msg_counter_req req;
+
+ return prestera_cmd(sw, PRESTERA_CMD_TYPE_COUNTER_ABORT,
+ &req.cmd, sizeof(req));
+}
+
+int prestera_hw_counters_get(struct prestera_switch *sw, u32 idx,
+ u32 *len, bool *done,
+ struct prestera_counter_stats *stats)
+{
+ struct prestera_msg_counter_resp *resp;
+ struct prestera_msg_counter_req req = {
+ .block_id = __cpu_to_le32(idx),
+ .num_counters = __cpu_to_le32(*len),
+ };
+ size_t size = struct_size(resp, stats, *len);
+ int err, i;
+
+ resp = kmalloc(size, GFP_KERNEL);
+ if (!resp)
+ return -ENOMEM;
+
+ err = prestera_cmd_ret(sw, PRESTERA_CMD_TYPE_COUNTER_GET,
+ &req.cmd, sizeof(req), &resp->ret, size);
+ if (err)
+ goto free_buff;
+
+ for (i = 0; i < __le32_to_cpu(resp->num_counters); i++) {
+ stats[i].packets += __le64_to_cpu(resp->stats[i].packets);
+ stats[i].bytes += __le64_to_cpu(resp->stats[i].bytes);
+ }
+
+ *len = __le32_to_cpu(resp->num_counters);
+ *done = __le32_to_cpu(resp->done);
+
+free_buff:
+ kfree(resp);
+ return err;
+}
+
+int prestera_hw_counter_block_get(struct prestera_switch *sw,
+ u32 client, u32 *block_id, u32 *offset,
+ u32 *num_counters)
+{
+ struct prestera_msg_counter_resp resp;
+ struct prestera_msg_counter_req req = {
+ .client = __cpu_to_le32(client)
+ };
+ int err;
+
+ err = prestera_cmd_ret(sw, PRESTERA_CMD_TYPE_COUNTER_BLOCK_GET,
+ &req.cmd, sizeof(req), &resp.ret, sizeof(resp));
+ if (err)
+ return err;
+
+ *block_id = __le32_to_cpu(resp.block_id);
+ *offset = __le32_to_cpu(resp.offset);
+ *num_counters = __le32_to_cpu(resp.num_counters);
+
+ return 0;
+}
+
+int prestera_hw_counter_block_release(struct prestera_switch *sw,
+ u32 block_id)
+{
+ struct prestera_msg_counter_req req = {
+ .block_id = __cpu_to_le32(block_id)
+ };
+
+ return prestera_cmd(sw, PRESTERA_CMD_TYPE_COUNTER_BLOCK_RELEASE,
+ &req.cmd, sizeof(req));
+}
+
+int prestera_hw_counter_clear(struct prestera_switch *sw, u32 block_id,
+ u32 counter_id)
+{
+ struct prestera_msg_counter_req req = {
+ .block_id = __cpu_to_le32(block_id),
+ .num_counters = __cpu_to_le32(counter_id)
+ };
+
+ return prestera_cmd(sw, PRESTERA_CMD_TYPE_COUNTER_CLEAR,
+ &req.cmd, sizeof(req));
+}
diff --git a/drivers/net/ethernet/marvell/prestera/prestera_hw.h b/drivers/net/ethernet/marvell/prestera/prestera_hw.h
index 57a3c2e..3ff12ba 100644
--- a/drivers/net/ethernet/marvell/prestera/prestera_hw.h
+++ b/drivers/net/ethernet/marvell/prestera/prestera_hw.h
@@ -5,6 +5,7 @@
#define _PRESTERA_HW_H_
#include <linux/types.h>
+#include "prestera_acl.h"
enum prestera_accept_frm_type {
PRESTERA_ACCEPT_FRAME_TYPE_TAGGED,
@@ -111,18 +112,32 @@ enum prestera_hw_cpu_code_cnt_t {
PRESTERA_HW_CPU_CODE_CNT_TYPE_TRAP = 1,
};
+enum prestera_hw_vtcam_direction_t {
+ PRESTERA_HW_VTCAM_DIR_INGRESS = 0,
+ PRESTERA_HW_VTCAM_DIR_EGRESS = 1,
+};
+
+enum {
+ PRESTERA_HW_COUNTER_CLIENT_LOOKUP_0 = 0,
+ PRESTERA_HW_COUNTER_CLIENT_LOOKUP_1 = 1,
+ PRESTERA_HW_COUNTER_CLIENT_LOOKUP_2 = 2,
+};
+
struct prestera_switch;
struct prestera_port;
struct prestera_port_stats;
struct prestera_port_caps;
enum prestera_event_type;
struct prestera_event;
-struct prestera_acl_rule;
typedef void (*prestera_event_cb_t)
(struct prestera_switch *sw, struct prestera_event *evt, void *arg);
struct prestera_rxtx_params;
+struct prestera_acl_hw_action_info;
+struct prestera_acl_iface;
+struct prestera_counter_stats;
+struct prestera_iface;
/* Switch API */
int prestera_hw_switch_init(struct prestera_switch *sw);
@@ -186,21 +201,37 @@ int prestera_hw_bridge_delete(struct prestera_switch *sw, u16 bridge_id);
int prestera_hw_bridge_port_add(struct prestera_port *port, u16 bridge_id);
int prestera_hw_bridge_port_delete(struct prestera_port *port, u16 bridge_id);
-/* ACL API */
-int prestera_hw_acl_ruleset_create(struct prestera_switch *sw,
- u16 *ruleset_id);
-int prestera_hw_acl_ruleset_del(struct prestera_switch *sw,
- u16 ruleset_id);
-int prestera_hw_acl_rule_add(struct prestera_switch *sw,
- struct prestera_acl_rule *rule,
- u32 *rule_id);
-int prestera_hw_acl_rule_del(struct prestera_switch *sw, u32 rule_id);
-int prestera_hw_acl_rule_stats_get(struct prestera_switch *sw,
- u32 rule_id, u64 *packets, u64 *bytes);
-int prestera_hw_acl_port_bind(const struct prestera_port *port,
- u16 ruleset_id);
-int prestera_hw_acl_port_unbind(const struct prestera_port *port,
- u16 ruleset_id);
+/* vTCAM API */
+int prestera_hw_vtcam_create(struct prestera_switch *sw,
+ u8 lookup, const u32 *keymask, u32 *vtcam_id,
+ enum prestera_hw_vtcam_direction_t direction);
+int prestera_hw_vtcam_rule_add(struct prestera_switch *sw, u32 vtcam_id,
+ u32 prio, void *key, void *keymask,
+ struct prestera_acl_hw_action_info *act,
+ u8 n_act, u32 *rule_id);
+int prestera_hw_vtcam_rule_del(struct prestera_switch *sw,
+ u32 vtcam_id, u32 rule_id);
+int prestera_hw_vtcam_destroy(struct prestera_switch *sw, u32 vtcam_id);
+int prestera_hw_vtcam_iface_bind(struct prestera_switch *sw,
+ struct prestera_acl_iface *iface,
+ u32 vtcam_id, u16 pcl_id);
+int prestera_hw_vtcam_iface_unbind(struct prestera_switch *sw,
+ struct prestera_acl_iface *iface,
+ u32 vtcam_id);
+
+/* Counter API */
+int prestera_hw_counter_trigger(struct prestera_switch *sw, u32 block_id);
+int prestera_hw_counter_abort(struct prestera_switch *sw);
+int prestera_hw_counters_get(struct prestera_switch *sw, u32 idx,
+ u32 *len, bool *done,
+ struct prestera_counter_stats *stats);
+int prestera_hw_counter_block_get(struct prestera_switch *sw,
+ u32 client, u32 *block_id, u32 *offset,
+ u32 *num_counters);
+int prestera_hw_counter_block_release(struct prestera_switch *sw,
+ u32 block_id);
+int prestera_hw_counter_clear(struct prestera_switch *sw, u32 block_id,
+ u32 counter_id);
/* SPAN API */
int prestera_hw_span_get(const struct prestera_port *port, u8 *span_id);
@@ -208,6 +239,16 @@ int prestera_hw_span_bind(const struct prestera_port *port, u8 span_id);
int prestera_hw_span_unbind(const struct prestera_port *port);
int prestera_hw_span_release(struct prestera_switch *sw, u8 span_id);
+/* Router API */
+int prestera_hw_rif_create(struct prestera_switch *sw,
+ struct prestera_iface *iif, u8 *mac, u16 *rif_id);
+int prestera_hw_rif_delete(struct prestera_switch *sw, u16 rif_id,
+ struct prestera_iface *iif);
+
+/* Virtual Router API */
+int prestera_hw_vr_create(struct prestera_switch *sw, u16 *vr_id);
+int prestera_hw_vr_delete(struct prestera_switch *sw, u16 vr_id);
+
/* Event handlers */
int prestera_hw_event_handler_register(struct prestera_switch *sw,
enum prestera_event_type type,
diff --git a/drivers/net/ethernet/marvell/prestera/prestera_main.c b/drivers/net/ethernet/marvell/prestera/prestera_main.c
index c687dc9..08fdd1e 100644
--- a/drivers/net/ethernet/marvell/prestera/prestera_main.c
+++ b/drivers/net/ethernet/marvell/prestera/prestera_main.c
@@ -18,6 +18,7 @@
#include "prestera_rxtx.h"
#include "prestera_devlink.h"
#include "prestera_ethtool.h"
+#include "prestera_counter.h"
#include "prestera_switchdev.h"
#define PRESTERA_MTU_DEFAULT 1536
@@ -162,7 +163,7 @@ static netdev_tx_t prestera_port_xmit(struct sk_buff *skb,
return prestera_rxtx_xmit(netdev_priv(dev), skb);
}
-static int prestera_is_valid_mac_addr(struct prestera_port *port, u8 *addr)
+int prestera_is_valid_mac_addr(struct prestera_port *port, const u8 *addr)
{
if (!is_valid_ether_addr(addr))
return -EADDRNOTAVAIL;
@@ -901,6 +902,10 @@ static int prestera_switch_init(struct prestera_switch *sw)
if (err)
return err;
+ err = prestera_router_init(sw);
+ if (err)
+ goto err_router_init;
+
err = prestera_switchdev_init(sw);
if (err)
goto err_swdev_register;
@@ -913,6 +918,10 @@ static int prestera_switch_init(struct prestera_switch *sw)
if (err)
goto err_handlers_register;
+ err = prestera_counter_init(sw);
+ if (err)
+ goto err_counter_init;
+
err = prestera_acl_init(sw);
if (err)
goto err_acl_init;
@@ -945,12 +954,16 @@ static int prestera_switch_init(struct prestera_switch *sw)
err_span_init:
prestera_acl_fini(sw);
err_acl_init:
+ prestera_counter_fini(sw);
+err_counter_init:
prestera_event_handlers_unregister(sw);
err_handlers_register:
prestera_rxtx_switch_fini(sw);
err_rxtx_register:
prestera_switchdev_fini(sw);
err_swdev_register:
+ prestera_router_fini(sw);
+err_router_init:
prestera_netdev_event_handler_unregister(sw);
prestera_hw_switch_fini(sw);
@@ -965,6 +978,7 @@ static void prestera_switch_fini(struct prestera_switch *sw)
prestera_devlink_traps_unregister(sw);
prestera_span_fini(sw);
prestera_acl_fini(sw);
+ prestera_counter_fini(sw);
prestera_event_handlers_unregister(sw);
prestera_rxtx_switch_fini(sw);
prestera_switchdev_fini(sw);
diff --git a/drivers/net/ethernet/marvell/prestera/prestera_router.c b/drivers/net/ethernet/marvell/prestera/prestera_router.c
new file mode 100644
index 0000000..8a3b7b6
--- /dev/null
+++ b/drivers/net/ethernet/marvell/prestera/prestera_router.c
@@ -0,0 +1,182 @@
+// SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0
+/* Copyright (c) 2019-2021 Marvell International Ltd. All rights reserved */
+
+#include <linux/kernel.h>
+#include <linux/types.h>
+#include <linux/inetdevice.h>
+#include <net/switchdev.h>
+
+#include "prestera.h"
+#include "prestera_router_hw.h"
+
+/* This util to be used, to convert kernel rules for default vr in hw_vr */
+static u32 prestera_fix_tb_id(u32 tb_id)
+{
+ if (tb_id == RT_TABLE_UNSPEC ||
+ tb_id == RT_TABLE_LOCAL ||
+ tb_id == RT_TABLE_DEFAULT)
+ tb_id = RT_TABLE_MAIN;
+
+ return tb_id;
+}
+
+static int __prestera_inetaddr_port_event(struct net_device *port_dev,
+ unsigned long event,
+ struct netlink_ext_ack *extack)
+{
+ struct prestera_port *port = netdev_priv(port_dev);
+ int err;
+ struct prestera_rif_entry *re;
+ struct prestera_rif_entry_key re_key = {};
+ u32 kern_tb_id;
+
+ err = prestera_is_valid_mac_addr(port, port_dev->dev_addr);
+ if (err) {
+ NL_SET_ERR_MSG_MOD(extack, "RIF MAC must have the same prefix");
+ return err;
+ }
+
+ kern_tb_id = l3mdev_fib_table(port_dev);
+ re_key.iface.type = PRESTERA_IF_PORT_E;
+ re_key.iface.dev_port.hw_dev_num = port->dev_id;
+ re_key.iface.dev_port.port_num = port->hw_id;
+ re = prestera_rif_entry_find(port->sw, &re_key);
+
+ switch (event) {
+ case NETDEV_UP:
+ if (re) {
+ NL_SET_ERR_MSG_MOD(extack, "rif_entry already exist");
+ return -EEXIST;
+ }
+ re = prestera_rif_entry_create(port->sw, &re_key,
+ prestera_fix_tb_id(kern_tb_id),
+ port_dev->dev_addr);
+ if (!re) {
+ NL_SET_ERR_MSG_MOD(extack, "Can't create rif_entry");
+ return -EINVAL;
+ }
+ dev_hold(port_dev);
+ break;
+ case NETDEV_DOWN:
+ if (!re) {
+ NL_SET_ERR_MSG_MOD(extack, "rif_entry not exist");
+ return -EEXIST;
+ }
+ prestera_rif_entry_destroy(port->sw, re);
+ dev_put(port_dev);
+ break;
+ }
+
+ return 0;
+}
+
+static int __prestera_inetaddr_event(struct prestera_switch *sw,
+ struct net_device *dev,
+ unsigned long event,
+ struct netlink_ext_ack *extack)
+{
+ if (prestera_netdev_check(dev) && !netif_is_bridge_port(dev) &&
+ !netif_is_lag_port(dev) && !netif_is_ovs_port(dev))
+ return __prestera_inetaddr_port_event(dev, event, extack);
+
+ return 0;
+}
+
+static int __prestera_inetaddr_cb(struct notifier_block *nb,
+ unsigned long event, void *ptr)
+{
+ struct in_ifaddr *ifa = (struct in_ifaddr *)ptr;
+ struct net_device *dev = ifa->ifa_dev->dev;
+ struct prestera_router *router = container_of(nb,
+ struct prestera_router,
+ inetaddr_nb);
+ struct in_device *idev;
+ int err = 0;
+
+ if (event != NETDEV_DOWN)
+ goto out;
+
+ /* Ignore if this is not latest address */
+ idev = __in_dev_get_rtnl(dev);
+ if (idev && idev->ifa_list)
+ goto out;
+
+ err = __prestera_inetaddr_event(router->sw, dev, event, NULL);
+out:
+ return notifier_from_errno(err);
+}
+
+static int __prestera_inetaddr_valid_cb(struct notifier_block *nb,
+ unsigned long event, void *ptr)
+{
+ struct in_validator_info *ivi = (struct in_validator_info *)ptr;
+ struct net_device *dev = ivi->ivi_dev->dev;
+ struct prestera_router *router = container_of(nb,
+ struct prestera_router,
+ inetaddr_valid_nb);
+ struct in_device *idev;
+ int err = 0;
+
+ if (event != NETDEV_UP)
+ goto out;
+
+ /* Ignore if this is not first address */
+ idev = __in_dev_get_rtnl(dev);
+ if (idev && idev->ifa_list)
+ goto out;
+
+ if (ipv4_is_multicast(ivi->ivi_addr)) {
+ err = -EINVAL;
+ goto out;
+ }
+
+ err = __prestera_inetaddr_event(router->sw, dev, event, ivi->extack);
+out:
+ return notifier_from_errno(err);
+}
+
+int prestera_router_init(struct prestera_switch *sw)
+{
+ struct prestera_router *router;
+ int err;
+
+ router = kzalloc(sizeof(*sw->router), GFP_KERNEL);
+ if (!router)
+ return -ENOMEM;
+
+ sw->router = router;
+ router->sw = sw;
+
+ err = prestera_router_hw_init(sw);
+ if (err)
+ goto err_router_lib_init;
+
+ router->inetaddr_valid_nb.notifier_call = __prestera_inetaddr_valid_cb;
+ err = register_inetaddr_validator_notifier(&router->inetaddr_valid_nb);
+ if (err)
+ goto err_register_inetaddr_validator_notifier;
+
+ router->inetaddr_nb.notifier_call = __prestera_inetaddr_cb;
+ err = register_inetaddr_notifier(&router->inetaddr_nb);
+ if (err)
+ goto err_register_inetaddr_notifier;
+
+ return 0;
+
+err_register_inetaddr_notifier:
+ unregister_inetaddr_validator_notifier(&router->inetaddr_valid_nb);
+err_register_inetaddr_validator_notifier:
+ /* prestera_router_hw_fini */
+err_router_lib_init:
+ kfree(sw->router);
+ return err;
+}
+
+void prestera_router_fini(struct prestera_switch *sw)
+{
+ unregister_inetaddr_notifier(&sw->router->inetaddr_nb);
+ unregister_inetaddr_validator_notifier(&sw->router->inetaddr_valid_nb);
+ /* router_hw_fini */
+ kfree(sw->router);
+ sw->router = NULL;
+}
diff --git a/drivers/net/ethernet/marvell/prestera/prestera_router_hw.c b/drivers/net/ethernet/marvell/prestera/prestera_router_hw.c
new file mode 100644
index 0000000..5866a4b
--- /dev/null
+++ b/drivers/net/ethernet/marvell/prestera/prestera_router_hw.c
@@ -0,0 +1,208 @@
+// SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0
+/* Copyright (c) 2019-2021 Marvell International Ltd. All rights reserved */
+
+#include <linux/rhashtable.h>
+
+#include "prestera.h"
+#include "prestera_hw.h"
+#include "prestera_router_hw.h"
+#include "prestera_acl.h"
+
+/* +--+
+ * +------->|vr|
+ * | +--+
+ * |
+ * +-+-------+
+ * |rif_entry|
+ * +---------+
+ * Rif is
+ * used as
+ * entry point
+ * for vr in hw
+ */
+
+int prestera_router_hw_init(struct prestera_switch *sw)
+{
+ INIT_LIST_HEAD(&sw->router->vr_list);
+ INIT_LIST_HEAD(&sw->router->rif_entry_list);
+
+ return 0;
+}
+
+static struct prestera_vr *__prestera_vr_find(struct prestera_switch *sw,
+ u32 tb_id)
+{
+ struct prestera_vr *vr;
+
+ list_for_each_entry(vr, &sw->router->vr_list, router_node) {
+ if (vr->tb_id == tb_id)
+ return vr;
+ }
+
+ return NULL;
+}
+
+static struct prestera_vr *__prestera_vr_create(struct prestera_switch *sw,
+ u32 tb_id,
+ struct netlink_ext_ack *extack)
+{
+ struct prestera_vr *vr;
+ u16 hw_vr_id;
+ int err;
+
+ err = prestera_hw_vr_create(sw, &hw_vr_id);
+ if (err)
+ return ERR_PTR(-ENOMEM);
+
+ vr = kzalloc(sizeof(*vr), GFP_KERNEL);
+ if (!vr) {
+ err = -ENOMEM;
+ goto err_alloc_vr;
+ }
+
+ vr->tb_id = tb_id;
+ vr->hw_vr_id = hw_vr_id;
+
+ list_add(&vr->router_node, &sw->router->vr_list);
+
+ return vr;
+
+err_alloc_vr:
+ prestera_hw_vr_delete(sw, hw_vr_id);
+ kfree(vr);
+ return ERR_PTR(err);
+}
+
+static void __prestera_vr_destroy(struct prestera_switch *sw,
+ struct prestera_vr *vr)
+{
+ prestera_hw_vr_delete(sw, vr->hw_vr_id);
+ list_del(&vr->router_node);
+ kfree(vr);
+}
+
+static struct prestera_vr *prestera_vr_get(struct prestera_switch *sw, u32 tb_id,
+ struct netlink_ext_ack *extack)
+{
+ struct prestera_vr *vr;
+
+ vr = __prestera_vr_find(sw, tb_id);
+ if (!vr)
+ vr = __prestera_vr_create(sw, tb_id, extack);
+ if (IS_ERR(vr))
+ return ERR_CAST(vr);
+
+ return vr;
+}
+
+static void prestera_vr_put(struct prestera_switch *sw, struct prestera_vr *vr)
+{
+ if (!vr->ref_cnt)
+ __prestera_vr_destroy(sw, vr);
+}
+
+/* iface is overhead struct. vr_id also can be removed. */
+static int
+__prestera_rif_entry_key_copy(const struct prestera_rif_entry_key *in,
+ struct prestera_rif_entry_key *out)
+{
+ memset(out, 0, sizeof(*out));
+
+ switch (in->iface.type) {
+ case PRESTERA_IF_PORT_E:
+ out->iface.dev_port.hw_dev_num = in->iface.dev_port.hw_dev_num;
+ out->iface.dev_port.port_num = in->iface.dev_port.port_num;
+ break;
+ case PRESTERA_IF_LAG_E:
+ out->iface.lag_id = in->iface.lag_id;
+ break;
+ case PRESTERA_IF_VID_E:
+ out->iface.vlan_id = in->iface.vlan_id;
+ break;
+ default:
+ pr_err("Unsupported iface type");
+ return -EINVAL;
+ }
+
+ out->iface.type = in->iface.type;
+ return 0;
+}
+
+struct prestera_rif_entry *
+prestera_rif_entry_find(const struct prestera_switch *sw,
+ const struct prestera_rif_entry_key *k)
+{
+ struct prestera_rif_entry *rif_entry;
+ struct prestera_rif_entry_key lk; /* lookup key */
+
+ if (__prestera_rif_entry_key_copy(k, &lk))
+ return NULL;
+
+ list_for_each_entry(rif_entry, &sw->router->rif_entry_list,
+ router_node) {
+ if (!memcmp(k, &rif_entry->key, sizeof(*k)))
+ return rif_entry;
+ }
+
+ return NULL;
+}
+
+void prestera_rif_entry_destroy(struct prestera_switch *sw,
+ struct prestera_rif_entry *e)
+{
+ struct prestera_iface iface;
+
+ list_del(&e->router_node);
+
+ memcpy(&iface, &e->key.iface, sizeof(iface));
+ iface.vr_id = e->vr->hw_vr_id;
+ prestera_hw_rif_delete(sw, e->hw_id, &iface);
+
+ e->vr->ref_cnt--;
+ prestera_vr_put(sw, e->vr);
+ kfree(e);
+}
+
+struct prestera_rif_entry *
+prestera_rif_entry_create(struct prestera_switch *sw,
+ struct prestera_rif_entry_key *k,
+ u32 tb_id, const unsigned char *addr)
+{
+ int err;
+ struct prestera_rif_entry *e;
+ struct prestera_iface iface;
+
+ e = kzalloc(sizeof(*e), GFP_KERNEL);
+ if (!e)
+ goto err_kzalloc;
+
+ if (__prestera_rif_entry_key_copy(k, &e->key))
+ goto err_key_copy;
+
+ e->vr = prestera_vr_get(sw, tb_id, NULL);
+ if (IS_ERR(e->vr))
+ goto err_vr_get;
+
+ e->vr->ref_cnt++;
+ memcpy(&e->addr, addr, sizeof(e->addr));
+
+ /* HW */
+ memcpy(&iface, &e->key.iface, sizeof(iface));
+ iface.vr_id = e->vr->hw_vr_id;
+ err = prestera_hw_rif_create(sw, &iface, e->addr, &e->hw_id);
+ if (err)
+ goto err_hw_create;
+
+ list_add(&e->router_node, &sw->router->rif_entry_list);
+
+ return e;
+
+err_hw_create:
+ e->vr->ref_cnt--;
+ prestera_vr_put(sw, e->vr);
+err_vr_get:
+err_key_copy:
+ kfree(e);
+err_kzalloc:
+ return NULL;
+}
diff --git a/drivers/net/ethernet/marvell/prestera/prestera_router_hw.h b/drivers/net/ethernet/marvell/prestera/prestera_router_hw.h
new file mode 100644
index 0000000..fed5359
--- /dev/null
+++ b/drivers/net/ethernet/marvell/prestera/prestera_router_hw.h
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0 */
+/* Copyright (c) 2019-2021 Marvell International Ltd. All rights reserved. */
+
+#ifndef _PRESTERA_ROUTER_HW_H_
+#define _PRESTERA_ROUTER_HW_H_
+
+struct prestera_vr {
+ struct list_head router_node;
+ unsigned int ref_cnt;
+ u32 tb_id; /* key (kernel fib table id) */
+ u16 hw_vr_id; /* virtual router ID */
+ u8 __pad[2];
+};
+
+struct prestera_rif_entry {
+ struct prestera_rif_entry_key {
+ struct prestera_iface iface;
+ } key;
+ struct prestera_vr *vr;
+ unsigned char addr[ETH_ALEN];
+ u16 hw_id; /* rif_id */
+ struct list_head router_node; /* ht */
+};
+
+struct prestera_rif_entry *
+prestera_rif_entry_find(const struct prestera_switch *sw,
+ const struct prestera_rif_entry_key *k);
+void prestera_rif_entry_destroy(struct prestera_switch *sw,
+ struct prestera_rif_entry *e);
+struct prestera_rif_entry *
+prestera_rif_entry_create(struct prestera_switch *sw,
+ struct prestera_rif_entry_key *k,
+ u32 tb_id, const unsigned char *addr);
+int prestera_router_hw_init(struct prestera_switch *sw);
+
+#endif /* _PRESTERA_ROUTER_HW_H_ */
diff --git a/drivers/net/ethernet/marvell/prestera/prestera_span.c b/drivers/net/ethernet/marvell/prestera/prestera_span.c
index 3cafca8..845e9d8 100644
--- a/drivers/net/ethernet/marvell/prestera/prestera_span.c
+++ b/drivers/net/ethernet/marvell/prestera/prestera_span.c
@@ -7,6 +7,7 @@
#include "prestera.h"
#include "prestera_hw.h"
#include "prestera_acl.h"
+#include "prestera_flow.h"
#include "prestera_span.h"
struct prestera_span_entry {
diff --git a/drivers/net/ethernet/marvell/pxa168_eth.c b/drivers/net/ethernet/marvell/pxa168_eth.c
index 1d607bc..52bef50 100644
--- a/drivers/net/ethernet/marvell/pxa168_eth.c
+++ b/drivers/net/ethernet/marvell/pxa168_eth.c
@@ -1388,7 +1388,6 @@ static int pxa168_eth_probe(struct platform_device *pdev)
{
struct pxa168_eth_private *pep = NULL;
struct net_device *dev = NULL;
- struct resource *res;
struct clk *clk;
struct device_node *np;
int err;
@@ -1419,9 +1418,11 @@ static int pxa168_eth_probe(struct platform_device *pdev)
goto err_netdev;
}
- res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
- BUG_ON(!res);
- dev->irq = res->start;
+ err = platform_get_irq(pdev, 0);
+ if (err == -EPROBE_DEFER)
+ goto err_netdev;
+ BUG_ON(dev->irq < 0);
+ dev->irq = err;
dev->netdev_ops = &pxa168_eth_netdev_ops;
dev->watchdog_timeo = 2 * HZ;
dev->base_addr = 0;
diff --git a/drivers/net/ethernet/marvell/skge.c b/drivers/net/ethernet/marvell/skge.c
index 0c864e5..cf03c67 100644
--- a/drivers/net/ethernet/marvell/skge.c
+++ b/drivers/net/ethernet/marvell/skge.c
@@ -492,7 +492,9 @@ static void skge_get_strings(struct net_device *dev, u32 stringset, u8 *data)
}
static void skge_get_ring_param(struct net_device *dev,
- struct ethtool_ringparam *p)
+ struct ethtool_ringparam *p,
+ struct kernel_ethtool_ringparam *kernel_p,
+ struct netlink_ext_ack *extack)
{
struct skge_port *skge = netdev_priv(dev);
@@ -504,7 +506,9 @@ static void skge_get_ring_param(struct net_device *dev,
}
static int skge_set_ring_param(struct net_device *dev,
- struct ethtool_ringparam *p)
+ struct ethtool_ringparam *p,
+ struct kernel_ethtool_ringparam *kernel_p,
+ struct netlink_ext_ack *extack)
{
struct skge_port *skge = netdev_priv(dev);
int err = 0;
diff --git a/drivers/net/ethernet/marvell/sky2.c b/drivers/net/ethernet/marvell/sky2.c
index 28b5b934..ea16b1d 100644
--- a/drivers/net/ethernet/marvell/sky2.c
+++ b/drivers/net/ethernet/marvell/sky2.c
@@ -4149,7 +4149,9 @@ static unsigned long roundup_ring_size(unsigned long pending)
}
static void sky2_get_ringparam(struct net_device *dev,
- struct ethtool_ringparam *ering)
+ struct ethtool_ringparam *ering,
+ struct kernel_ethtool_ringparam *kernel_ering,
+ struct netlink_ext_ack *extack)
{
struct sky2_port *sky2 = netdev_priv(dev);
@@ -4161,7 +4163,9 @@ static void sky2_get_ringparam(struct net_device *dev,
}
static int sky2_set_ringparam(struct net_device *dev,
- struct ethtool_ringparam *ering)
+ struct ethtool_ringparam *ering,
+ struct kernel_ethtool_ringparam *kernel_ering,
+ struct netlink_ext_ack *extack)
{
struct sky2_port *sky2 = netdev_priv(dev);
@@ -4266,96 +4270,36 @@ static int sky2_get_eeprom_len(struct net_device *dev)
return 1 << ( ((reg2 & PCI_VPD_ROM_SZ) >> 14) + 8);
}
-static int sky2_vpd_wait(const struct sky2_hw *hw, int cap, u16 busy)
-{
- unsigned long start = jiffies;
-
- while ( (sky2_pci_read16(hw, cap + PCI_VPD_ADDR) & PCI_VPD_ADDR_F) == busy) {
- /* Can take up to 10.6 ms for write */
- if (time_after(jiffies, start + HZ/4)) {
- dev_err(&hw->pdev->dev, "VPD cycle timed out\n");
- return -ETIMEDOUT;
- }
- msleep(1);
- }
-
- return 0;
-}
-
-static int sky2_vpd_read(struct sky2_hw *hw, int cap, void *data,
- u16 offset, size_t length)
-{
- int rc = 0;
-
- while (length > 0) {
- u32 val;
-
- sky2_pci_write16(hw, cap + PCI_VPD_ADDR, offset);
- rc = sky2_vpd_wait(hw, cap, 0);
- if (rc)
- break;
-
- val = sky2_pci_read32(hw, cap + PCI_VPD_DATA);
-
- memcpy(data, &val, min(sizeof(val), length));
- offset += sizeof(u32);
- data += sizeof(u32);
- length -= sizeof(u32);
- }
-
- return rc;
-}
-
-static int sky2_vpd_write(struct sky2_hw *hw, int cap, const void *data,
- u16 offset, unsigned int length)
-{
- unsigned int i;
- int rc = 0;
-
- for (i = 0; i < length; i += sizeof(u32)) {
- u32 val = *(u32 *)(data + i);
-
- sky2_pci_write32(hw, cap + PCI_VPD_DATA, val);
- sky2_pci_write32(hw, cap + PCI_VPD_ADDR, offset | PCI_VPD_ADDR_F);
-
- rc = sky2_vpd_wait(hw, cap, PCI_VPD_ADDR_F);
- if (rc)
- break;
- }
- return rc;
-}
-
static int sky2_get_eeprom(struct net_device *dev, struct ethtool_eeprom *eeprom,
u8 *data)
{
struct sky2_port *sky2 = netdev_priv(dev);
- int cap = pci_find_capability(sky2->hw->pdev, PCI_CAP_ID_VPD);
-
- if (!cap)
- return -EINVAL;
+ int rc;
eeprom->magic = SKY2_EEPROM_MAGIC;
+ rc = pci_read_vpd_any(sky2->hw->pdev, eeprom->offset, eeprom->len,
+ data);
+ if (rc < 0)
+ return rc;
- return sky2_vpd_read(sky2->hw, cap, data, eeprom->offset, eeprom->len);
+ eeprom->len = rc;
+
+ return 0;
}
static int sky2_set_eeprom(struct net_device *dev, struct ethtool_eeprom *eeprom,
u8 *data)
{
struct sky2_port *sky2 = netdev_priv(dev);
- int cap = pci_find_capability(sky2->hw->pdev, PCI_CAP_ID_VPD);
-
- if (!cap)
- return -EINVAL;
+ int rc;
if (eeprom->magic != SKY2_EEPROM_MAGIC)
return -EINVAL;
- /* Partial writes not supported */
- if ((eeprom->offset & 3) || (eeprom->len & 3))
- return -EINVAL;
+ rc = pci_write_vpd_any(sky2->hw->pdev, eeprom->offset, eeprom->len,
+ data);
- return sky2_vpd_write(sky2->hw, cap, data, eeprom->offset, eeprom->len);
+ return rc < 0 ? rc : 0;
}
static netdev_features_t sky2_fix_features(struct net_device *dev,
diff --git a/drivers/net/ethernet/mediatek/Kconfig b/drivers/net/ethernet/mediatek/Kconfig
index c357c19..86d356b 100644
--- a/drivers/net/ethernet/mediatek/Kconfig
+++ b/drivers/net/ethernet/mediatek/Kconfig
@@ -1,7 +1,7 @@
# SPDX-License-Identifier: GPL-2.0-only
config NET_VENDOR_MEDIATEK
bool "MediaTek devices"
- depends on ARCH_MEDIATEK || SOC_MT7621 || SOC_MT7620
+ depends on ARCH_MEDIATEK || SOC_MT7621 || SOC_MT7620 || COMPILE_TEST
help
If you have a Mediatek SoC with ethernet, say Y.
@@ -10,6 +10,7 @@
config NET_MEDIATEK_SOC
tristate "MediaTek SoC Gigabit Ethernet support"
depends on NET_DSA || !NET_DSA
+ select PINCTRL
select PHYLINK
select DIMLIB
help
diff --git a/drivers/net/ethernet/mediatek/mtk_eth_soc.c b/drivers/net/ethernet/mediatek/mtk_eth_soc.c
index 75d67d1..b67b432 100644
--- a/drivers/net/ethernet/mediatek/mtk_eth_soc.c
+++ b/drivers/net/ethernet/mediatek/mtk_eth_soc.c
@@ -91,46 +91,96 @@ static int mtk_mdio_busy_wait(struct mtk_eth *eth)
}
dev_err(eth->dev, "mdio: MDIO timeout\n");
- return -1;
+ return -ETIMEDOUT;
}
-static u32 _mtk_mdio_write(struct mtk_eth *eth, u32 phy_addr,
- u32 phy_register, u32 write_data)
+static int _mtk_mdio_write(struct mtk_eth *eth, u32 phy_addr, u32 phy_reg,
+ u32 write_data)
{
- if (mtk_mdio_busy_wait(eth))
- return -1;
+ int ret;
- write_data &= 0xffff;
+ ret = mtk_mdio_busy_wait(eth);
+ if (ret < 0)
+ return ret;
- mtk_w32(eth, PHY_IAC_ACCESS | PHY_IAC_START | PHY_IAC_WRITE |
- (phy_register << PHY_IAC_REG_SHIFT) |
- (phy_addr << PHY_IAC_ADDR_SHIFT) | write_data,
- MTK_PHY_IAC);
+ if (phy_reg & MII_ADDR_C45) {
+ mtk_w32(eth, PHY_IAC_ACCESS |
+ PHY_IAC_START_C45 |
+ PHY_IAC_CMD_C45_ADDR |
+ PHY_IAC_REG(mdiobus_c45_devad(phy_reg)) |
+ PHY_IAC_ADDR(phy_addr) |
+ PHY_IAC_DATA(mdiobus_c45_regad(phy_reg)),
+ MTK_PHY_IAC);
- if (mtk_mdio_busy_wait(eth))
- return -1;
+ ret = mtk_mdio_busy_wait(eth);
+ if (ret < 0)
+ return ret;
+
+ mtk_w32(eth, PHY_IAC_ACCESS |
+ PHY_IAC_START_C45 |
+ PHY_IAC_CMD_WRITE |
+ PHY_IAC_REG(mdiobus_c45_devad(phy_reg)) |
+ PHY_IAC_ADDR(phy_addr) |
+ PHY_IAC_DATA(write_data),
+ MTK_PHY_IAC);
+ } else {
+ mtk_w32(eth, PHY_IAC_ACCESS |
+ PHY_IAC_START_C22 |
+ PHY_IAC_CMD_WRITE |
+ PHY_IAC_REG(phy_reg) |
+ PHY_IAC_ADDR(phy_addr) |
+ PHY_IAC_DATA(write_data),
+ MTK_PHY_IAC);
+ }
+
+ ret = mtk_mdio_busy_wait(eth);
+ if (ret < 0)
+ return ret;
return 0;
}
-static u32 _mtk_mdio_read(struct mtk_eth *eth, int phy_addr, int phy_reg)
+static int _mtk_mdio_read(struct mtk_eth *eth, u32 phy_addr, u32 phy_reg)
{
- u32 d;
+ int ret;
- if (mtk_mdio_busy_wait(eth))
- return 0xffff;
+ ret = mtk_mdio_busy_wait(eth);
+ if (ret < 0)
+ return ret;
- mtk_w32(eth, PHY_IAC_ACCESS | PHY_IAC_START | PHY_IAC_READ |
- (phy_reg << PHY_IAC_REG_SHIFT) |
- (phy_addr << PHY_IAC_ADDR_SHIFT),
- MTK_PHY_IAC);
+ if (phy_reg & MII_ADDR_C45) {
+ mtk_w32(eth, PHY_IAC_ACCESS |
+ PHY_IAC_START_C45 |
+ PHY_IAC_CMD_C45_ADDR |
+ PHY_IAC_REG(mdiobus_c45_devad(phy_reg)) |
+ PHY_IAC_ADDR(phy_addr) |
+ PHY_IAC_DATA(mdiobus_c45_regad(phy_reg)),
+ MTK_PHY_IAC);
- if (mtk_mdio_busy_wait(eth))
- return 0xffff;
+ ret = mtk_mdio_busy_wait(eth);
+ if (ret < 0)
+ return ret;
- d = mtk_r32(eth, MTK_PHY_IAC) & 0xffff;
+ mtk_w32(eth, PHY_IAC_ACCESS |
+ PHY_IAC_START_C45 |
+ PHY_IAC_CMD_C45_READ |
+ PHY_IAC_REG(mdiobus_c45_devad(phy_reg)) |
+ PHY_IAC_ADDR(phy_addr),
+ MTK_PHY_IAC);
+ } else {
+ mtk_w32(eth, PHY_IAC_ACCESS |
+ PHY_IAC_START_C22 |
+ PHY_IAC_CMD_C22_READ |
+ PHY_IAC_REG(phy_reg) |
+ PHY_IAC_ADDR(phy_addr),
+ MTK_PHY_IAC);
+ }
- return d;
+ ret = mtk_mdio_busy_wait(eth);
+ if (ret < 0)
+ return ret;
+
+ return mtk_r32(eth, MTK_PHY_IAC) & PHY_IAC_DATA_MASK;
}
static int mtk_mdio_write(struct mii_bus *bus, int phy_addr,
@@ -463,94 +513,8 @@ static void mtk_mac_link_up(struct phylink_config *config,
mtk_w32(mac->hw, mcr, MTK_MAC_MCR(mac->id));
}
-static void mtk_validate(struct phylink_config *config,
- unsigned long *supported,
- struct phylink_link_state *state)
-{
- struct mtk_mac *mac = container_of(config, struct mtk_mac,
- phylink_config);
- __ETHTOOL_DECLARE_LINK_MODE_MASK(mask) = { 0, };
-
- if (state->interface != PHY_INTERFACE_MODE_NA &&
- state->interface != PHY_INTERFACE_MODE_MII &&
- state->interface != PHY_INTERFACE_MODE_GMII &&
- !(MTK_HAS_CAPS(mac->hw->soc->caps, MTK_RGMII) &&
- phy_interface_mode_is_rgmii(state->interface)) &&
- !(MTK_HAS_CAPS(mac->hw->soc->caps, MTK_TRGMII) &&
- !mac->id && state->interface == PHY_INTERFACE_MODE_TRGMII) &&
- !(MTK_HAS_CAPS(mac->hw->soc->caps, MTK_SGMII) &&
- (state->interface == PHY_INTERFACE_MODE_SGMII ||
- phy_interface_mode_is_8023z(state->interface)))) {
- linkmode_zero(supported);
- return;
- }
-
- phylink_set_port_modes(mask);
- phylink_set(mask, Autoneg);
-
- switch (state->interface) {
- case PHY_INTERFACE_MODE_TRGMII:
- phylink_set(mask, 1000baseT_Full);
- break;
- case PHY_INTERFACE_MODE_1000BASEX:
- case PHY_INTERFACE_MODE_2500BASEX:
- phylink_set(mask, 1000baseX_Full);
- phylink_set(mask, 2500baseX_Full);
- break;
- case PHY_INTERFACE_MODE_GMII:
- case PHY_INTERFACE_MODE_RGMII:
- case PHY_INTERFACE_MODE_RGMII_ID:
- case PHY_INTERFACE_MODE_RGMII_RXID:
- case PHY_INTERFACE_MODE_RGMII_TXID:
- phylink_set(mask, 1000baseT_Half);
- fallthrough;
- case PHY_INTERFACE_MODE_SGMII:
- phylink_set(mask, 1000baseT_Full);
- phylink_set(mask, 1000baseX_Full);
- fallthrough;
- case PHY_INTERFACE_MODE_MII:
- case PHY_INTERFACE_MODE_RMII:
- case PHY_INTERFACE_MODE_REVMII:
- case PHY_INTERFACE_MODE_NA:
- default:
- phylink_set(mask, 10baseT_Half);
- phylink_set(mask, 10baseT_Full);
- phylink_set(mask, 100baseT_Half);
- phylink_set(mask, 100baseT_Full);
- break;
- }
-
- if (state->interface == PHY_INTERFACE_MODE_NA) {
- if (MTK_HAS_CAPS(mac->hw->soc->caps, MTK_SGMII)) {
- phylink_set(mask, 1000baseT_Full);
- phylink_set(mask, 1000baseX_Full);
- phylink_set(mask, 2500baseX_Full);
- }
- if (MTK_HAS_CAPS(mac->hw->soc->caps, MTK_RGMII)) {
- phylink_set(mask, 1000baseT_Full);
- phylink_set(mask, 1000baseT_Half);
- phylink_set(mask, 1000baseX_Full);
- }
- if (MTK_HAS_CAPS(mac->hw->soc->caps, MTK_GEPHY)) {
- phylink_set(mask, 1000baseT_Full);
- phylink_set(mask, 1000baseT_Half);
- }
- }
-
- phylink_set(mask, Pause);
- phylink_set(mask, Asym_Pause);
-
- linkmode_and(supported, supported, mask);
- linkmode_and(state->advertising, state->advertising, mask);
-
- /* We can only operate at 2500BaseX or 1000BaseX. If requested
- * to advertise both, only report advertising at 2500BaseX.
- */
- phylink_helper_basex_speed(state);
-}
-
static const struct phylink_mac_ops mtk_phylink_ops = {
- .validate = mtk_validate,
+ .validate = phylink_generic_validate,
.mac_pcs_get_state = mtk_mac_pcs_get_state,
.mac_an_restart = mtk_mac_an_restart,
.mac_config = mtk_mac_config,
@@ -583,6 +547,7 @@ static int mtk_mdio_init(struct mtk_eth *eth)
eth->mii_bus->name = "mdio";
eth->mii_bus->read = mtk_mdio_read;
eth->mii_bus->write = mtk_mdio_write;
+ eth->mii_bus->probe_capabilities = MDIOBUS_C22_C45;
eth->mii_bus->priv = eth;
eth->mii_bus->parent = eth->dev;
@@ -2297,7 +2262,6 @@ static int mtk_open(struct net_device *dev)
/* we run 2 netdevs on the same dma ring so we only bring it up once */
if (!refcount_read(ð->dma_refcnt)) {
u32 gdm_config = MTK_GDMA_TO_PDMA;
- int err;
err = mtk_start_dma(eth);
if (err)
@@ -3009,6 +2973,33 @@ static int mtk_add_mac(struct mtk_eth *eth, struct device_node *np)
mac->phylink_config.dev = ð->netdev[id]->dev;
mac->phylink_config.type = PHYLINK_NETDEV;
+ /* This driver makes use of state->speed/state->duplex in
+ * mac_config
+ */
+ mac->phylink_config.legacy_pre_march2020 = true;
+ mac->phylink_config.mac_capabilities = MAC_ASYM_PAUSE | MAC_SYM_PAUSE |
+ MAC_10 | MAC_100 | MAC_1000 | MAC_2500FD;
+
+ __set_bit(PHY_INTERFACE_MODE_MII,
+ mac->phylink_config.supported_interfaces);
+ __set_bit(PHY_INTERFACE_MODE_GMII,
+ mac->phylink_config.supported_interfaces);
+
+ if (MTK_HAS_CAPS(mac->hw->soc->caps, MTK_RGMII))
+ phy_interface_set_rgmii(mac->phylink_config.supported_interfaces);
+
+ if (MTK_HAS_CAPS(mac->hw->soc->caps, MTK_TRGMII) && !mac->id)
+ __set_bit(PHY_INTERFACE_MODE_TRGMII,
+ mac->phylink_config.supported_interfaces);
+
+ if (MTK_HAS_CAPS(mac->hw->soc->caps, MTK_SGMII)) {
+ __set_bit(PHY_INTERFACE_MODE_SGMII,
+ mac->phylink_config.supported_interfaces);
+ __set_bit(PHY_INTERFACE_MODE_1000BASEX,
+ mac->phylink_config.supported_interfaces);
+ __set_bit(PHY_INTERFACE_MODE_2500BASEX,
+ mac->phylink_config.supported_interfaces);
+ }
phylink = phylink_create(&mac->phylink_config,
of_fwnode_handle(mac->of_node),
diff --git a/drivers/net/ethernet/mediatek/mtk_eth_soc.h b/drivers/net/ethernet/mediatek/mtk_eth_soc.h
index 5ef70dd..c9d42be 100644
--- a/drivers/net/ethernet/mediatek/mtk_eth_soc.h
+++ b/drivers/net/ethernet/mediatek/mtk_eth_soc.h
@@ -341,11 +341,20 @@
/* PHY Indirect Access Control registers */
#define MTK_PHY_IAC 0x10004
#define PHY_IAC_ACCESS BIT(31)
-#define PHY_IAC_READ BIT(19)
-#define PHY_IAC_WRITE BIT(18)
-#define PHY_IAC_START BIT(16)
-#define PHY_IAC_ADDR_SHIFT 20
-#define PHY_IAC_REG_SHIFT 25
+#define PHY_IAC_REG_MASK GENMASK(29, 25)
+#define PHY_IAC_REG(x) FIELD_PREP(PHY_IAC_REG_MASK, (x))
+#define PHY_IAC_ADDR_MASK GENMASK(24, 20)
+#define PHY_IAC_ADDR(x) FIELD_PREP(PHY_IAC_ADDR_MASK, (x))
+#define PHY_IAC_CMD_MASK GENMASK(19, 18)
+#define PHY_IAC_CMD_C45_ADDR FIELD_PREP(PHY_IAC_CMD_MASK, 0)
+#define PHY_IAC_CMD_WRITE FIELD_PREP(PHY_IAC_CMD_MASK, 1)
+#define PHY_IAC_CMD_C22_READ FIELD_PREP(PHY_IAC_CMD_MASK, 2)
+#define PHY_IAC_CMD_C45_READ FIELD_PREP(PHY_IAC_CMD_MASK, 3)
+#define PHY_IAC_START_MASK GENMASK(17, 16)
+#define PHY_IAC_START_C45 FIELD_PREP(PHY_IAC_START_MASK, 0)
+#define PHY_IAC_START_C22 FIELD_PREP(PHY_IAC_START_MASK, 1)
+#define PHY_IAC_DATA_MASK GENMASK(15, 0)
+#define PHY_IAC_DATA(x) FIELD_PREP(PHY_IAC_DATA_MASK, (x))
#define PHY_IAC_TIMEOUT HZ
#define MTK_MAC_MISC 0x1000c
diff --git a/drivers/net/ethernet/mediatek/mtk_ppe_debugfs.c b/drivers/net/ethernet/mediatek/mtk_ppe_debugfs.c
index 98b1d35..d4b4823 100644
--- a/drivers/net/ethernet/mediatek/mtk_ppe_debugfs.c
+++ b/drivers/net/ethernet/mediatek/mtk_ppe_debugfs.c
@@ -207,9 +207,6 @@ int mtk_ppe_debugfs_init(struct mtk_ppe *ppe)
struct dentry *root;
root = debugfs_create_dir("mtk_ppe", NULL);
- if (!root)
- return -ENOMEM;
-
debugfs_create_file("entries", S_IRUGO, root, ppe, &fops_all);
debugfs_create_file("bind", S_IRUGO, root, ppe, &fops_bind);
diff --git a/drivers/net/ethernet/mellanox/mlx4/en_ethtool.c b/drivers/net/ethernet/mellanox/mlx4/en_ethtool.c
index 10238be..ed5038d 100644
--- a/drivers/net/ethernet/mellanox/mlx4/en_ethtool.c
+++ b/drivers/net/ethernet/mellanox/mlx4/en_ethtool.c
@@ -1141,7 +1141,9 @@ static void mlx4_en_get_pauseparam(struct net_device *dev,
}
static int mlx4_en_set_ringparam(struct net_device *dev,
- struct ethtool_ringparam *param)
+ struct ethtool_ringparam *param,
+ struct kernel_ethtool_ringparam *kernel_param,
+ struct netlink_ext_ack *extack)
{
struct mlx4_en_priv *priv = netdev_priv(dev);
struct mlx4_en_dev *mdev = priv->mdev;
@@ -1208,7 +1210,9 @@ static int mlx4_en_set_ringparam(struct net_device *dev,
}
static void mlx4_en_get_ringparam(struct net_device *dev,
- struct ethtool_ringparam *param)
+ struct ethtool_ringparam *param,
+ struct kernel_ethtool_ringparam *kernel_param,
+ struct netlink_ext_ack *extack)
{
struct mlx4_en_priv *priv = netdev_priv(dev);
diff --git a/drivers/net/ethernet/mellanox/mlx4/en_netdev.c b/drivers/net/ethernet/mellanox/mlx4/en_netdev.c
index f1c10f2..c61dc7a 100644
--- a/drivers/net/ethernet/mellanox/mlx4/en_netdev.c
+++ b/drivers/net/ethernet/mellanox/mlx4/en_netdev.c
@@ -33,6 +33,7 @@
#include <linux/bpf.h>
#include <linux/etherdevice.h>
+#include <linux/filter.h>
#include <linux/tcp.h>
#include <linux/if_vlan.h>
#include <linux/delay.h>
@@ -2427,10 +2428,6 @@ static int mlx4_en_hwtstamp_set(struct net_device *dev, struct ifreq *ifr)
if (copy_from_user(&config, ifr->ifr_data, sizeof(config)))
return -EFAULT;
- /* reserved for future extensions */
- if (config.flags)
- return -EINVAL;
-
/* device doesn't support time stamping */
if (!(mdev->dev->caps.flags2 & MLX4_DEV_CAP_FLAG2_TS))
return -EINVAL;
diff --git a/drivers/net/ethernet/mellanox/mlx4/en_rx.c b/drivers/net/ethernet/mellanox/mlx4/en_rx.c
index 650e6a1..8cfc649 100644
--- a/drivers/net/ethernet/mellanox/mlx4/en_rx.c
+++ b/drivers/net/ethernet/mellanox/mlx4/en_rx.c
@@ -812,7 +812,7 @@ int mlx4_en_process_rx_cq(struct net_device *dev, struct mlx4_en_cq *cq, int bud
trace_xdp_exception(dev, xdp_prog, act);
goto xdp_drop_no_cnt; /* Drop on xmit failure */
default:
- bpf_warn_invalid_xdp_action(act);
+ bpf_warn_invalid_xdp_action(dev, xdp_prog, act);
fallthrough;
case XDP_ABORTED:
trace_xdp_exception(dev, xdp_prog, act);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/Kconfig b/drivers/net/ethernet/mellanox/mlx5/core/Kconfig
index 9205645..4ba1a78 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/Kconfig
+++ b/drivers/net/ethernet/mellanox/mlx5/core/Kconfig
@@ -115,6 +115,7 @@
config MLX5_TC_SAMPLE
bool "MLX5 TC sample offload support"
depends on MLX5_CLS_ACT
+ depends on PSAMPLE=y || PSAMPLE=n || MLX5_CORE=m
default y
help
Say Y here if you want to support offloading sample rules via tc
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/Makefile b/drivers/net/ethernet/mellanox/mlx5/core/Makefile
index e63bb9c..fcfd38f 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/Makefile
+++ b/drivers/net/ethernet/mellanox/mlx5/core/Makefile
@@ -46,6 +46,15 @@
en/tc_tun_vxlan.o en/tc_tun_gre.o en/tc_tun_geneve.o \
en/tc_tun_mplsoudp.o diag/en_tc_tracepoint.o \
en/tc/post_act.o en/tc/int_port.o
+
+mlx5_core-$(CONFIG_MLX5_CLS_ACT) += en/tc/act/act.o en/tc/act/drop.o en/tc/act/trap.o \
+ en/tc/act/accept.o en/tc/act/mark.o en/tc/act/goto.o \
+ en/tc/act/tun.o en/tc/act/csum.o en/tc/act/pedit.o \
+ en/tc/act/vlan.o en/tc/act/vlan_mangle.o en/tc/act/mpls.o \
+ en/tc/act/mirred.o en/tc/act/mirred_nic.o \
+ en/tc/act/ct.o en/tc/act/sample.o en/tc/act/ptype.o \
+ en/tc/act/redirect_ingress.o
+
mlx5_core-$(CONFIG_MLX5_TC_CT) += en/tc_ct.o
mlx5_core-$(CONFIG_MLX5_TC_SAMPLE) += en/tc/sample.o
@@ -95,11 +104,12 @@
steering/dr_ste.o steering/dr_send.o \
steering/dr_ste_v0.o steering/dr_ste_v1.o \
steering/dr_cmd.o steering/dr_fw.o \
- steering/dr_action.o steering/fs_dr.o
+ steering/dr_action.o steering/fs_dr.o \
+ steering/dr_dbg.o
#
# SF device
#
-mlx5_core-$(CONFIG_MLX5_SF) += sf/vhca_event.o sf/dev/dev.o sf/dev/driver.o
+mlx5_core-$(CONFIG_MLX5_SF) += sf/vhca_event.o sf/dev/dev.o sf/dev/driver.o irq_affinity.o
#
# SF manager
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/cmd.c b/drivers/net/ethernet/mellanox/mlx5/core/cmd.c
index a46284c..17fe058 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/cmd.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/cmd.c
@@ -148,8 +148,12 @@ static void cmd_ent_put(struct mlx5_cmd_work_ent *ent)
if (!refcount_dec_and_test(&ent->refcnt))
return;
- if (ent->idx >= 0)
- cmd_free_index(ent->cmd, ent->idx);
+ if (ent->idx >= 0) {
+ struct mlx5_cmd *cmd = ent->cmd;
+
+ cmd_free_index(cmd, ent->idx);
+ up(ent->page_queue ? &cmd->pages_sem : &cmd->sem);
+ }
cmd_free_ent(ent);
}
@@ -900,25 +904,6 @@ static bool opcode_allowed(struct mlx5_cmd *cmd, u16 opcode)
return cmd->allowed_opcode == opcode;
}
-static int cmd_alloc_index_retry(struct mlx5_cmd *cmd)
-{
- unsigned long alloc_end = jiffies + msecs_to_jiffies(1000);
- int idx;
-
-retry:
- idx = cmd_alloc_index(cmd);
- if (idx < 0 && time_before(jiffies, alloc_end)) {
- /* Index allocation can fail on heavy load of commands. This is a temporary
- * situation as the current command already holds the semaphore, meaning that
- * another command completion is being handled and it is expected to release
- * the entry index soon.
- */
- cpu_relax();
- goto retry;
- }
- return idx;
-}
-
bool mlx5_cmd_is_down(struct mlx5_core_dev *dev)
{
return pci_channel_offline(dev->pdev) ||
@@ -946,7 +931,7 @@ static void cmd_work_handler(struct work_struct *work)
sem = ent->page_queue ? &cmd->pages_sem : &cmd->sem;
down(sem);
if (!ent->page_queue) {
- alloc_ret = cmd_alloc_index_retry(cmd);
+ alloc_ret = cmd_alloc_index(cmd);
if (alloc_ret < 0) {
mlx5_core_err_rl(dev, "failed to allocate command entry\n");
if (ent->callback) {
@@ -1602,8 +1587,6 @@ static void mlx5_cmd_comp_handler(struct mlx5_core_dev *dev, u64 vec, bool force
vector = vec & 0xffffffff;
for (i = 0; i < (1 << cmd->log_sz); i++) {
if (test_bit(i, &vector)) {
- struct semaphore *sem;
-
ent = cmd->ent_arr[i];
/* if we already completed the command, ignore it */
@@ -1626,10 +1609,6 @@ static void mlx5_cmd_comp_handler(struct mlx5_core_dev *dev, u64 vec, bool force
dev->state == MLX5_DEVICE_STATE_INTERNAL_ERROR)
cmd_ent_put(ent);
- if (ent->page_queue)
- sem = &cmd->pages_sem;
- else
- sem = &cmd->sem;
ent->ts2 = ktime_get_ns();
memcpy(ent->out->first.data, ent->lay->out, sizeof(ent->lay->out));
dump_command(dev, ent, 0);
@@ -1683,7 +1662,6 @@ static void mlx5_cmd_comp_handler(struct mlx5_core_dev *dev, u64 vec, bool force
*/
complete(&ent->done);
}
- up(sem);
}
}
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/dev.c b/drivers/net/ethernet/mellanox/mlx5/core/dev.c
index a8b84d5..ba6dad9 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/dev.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/dev.c
@@ -538,7 +538,7 @@ int mlx5_rescan_drivers_locked(struct mlx5_core_dev *dev)
return add_drivers(dev);
}
-static bool mlx5_same_hw_devs(struct mlx5_core_dev *dev, struct mlx5_core_dev *peer_dev)
+bool mlx5_same_hw_devs(struct mlx5_core_dev *dev, struct mlx5_core_dev *peer_dev)
{
u64 fsystem_guid, psystem_guid;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/devlink.c b/drivers/net/ethernet/mellanox/mlx5/core/devlink.c
index 1c98652..d1093bb 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/devlink.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/devlink.c
@@ -546,6 +546,13 @@ static int mlx5_devlink_enable_remote_dev_reset_get(struct devlink *devlink, u32
return 0;
}
+static int mlx5_devlink_eq_depth_validate(struct devlink *devlink, u32 id,
+ union devlink_param_value val,
+ struct netlink_ext_ack *extack)
+{
+ return (val.vu16 >= 64 && val.vu16 <= 4096) ? 0 : -EINVAL;
+}
+
static const struct devlink_param mlx5_devlink_params[] = {
DEVLINK_PARAM_DRIVER(MLX5_DEVLINK_PARAM_ID_FLOW_STEERING_MODE,
"flow_steering_mode", DEVLINK_PARAM_TYPE_STRING,
@@ -570,6 +577,10 @@ static const struct devlink_param mlx5_devlink_params[] = {
DEVLINK_PARAM_GENERIC(ENABLE_REMOTE_DEV_RESET, BIT(DEVLINK_PARAM_CMODE_RUNTIME),
mlx5_devlink_enable_remote_dev_reset_get,
mlx5_devlink_enable_remote_dev_reset_set, NULL),
+ DEVLINK_PARAM_GENERIC(IO_EQ_SIZE, BIT(DEVLINK_PARAM_CMODE_DRIVERINIT),
+ NULL, NULL, mlx5_devlink_eq_depth_validate),
+ DEVLINK_PARAM_GENERIC(EVENT_EQ_SIZE, BIT(DEVLINK_PARAM_CMODE_DRIVERINIT),
+ NULL, NULL, mlx5_devlink_eq_depth_validate),
};
static void mlx5_devlink_set_params_init_values(struct devlink *devlink)
@@ -608,6 +619,16 @@ static void mlx5_devlink_set_params_init_values(struct devlink *devlink)
value);
}
#endif
+
+ value.vu32 = MLX5_COMP_EQ_SIZE;
+ devlink_param_driverinit_value_set(devlink,
+ DEVLINK_PARAM_GENERIC_ID_IO_EQ_SIZE,
+ value);
+
+ value.vu32 = MLX5_NUM_ASYNC_EQE;
+ devlink_param_driverinit_value_set(devlink,
+ DEVLINK_PARAM_GENERIC_ID_EVENT_EQ_SIZE,
+ value);
}
static const struct devlink_param enable_eth_param =
@@ -752,6 +773,66 @@ static void mlx5_devlink_auxdev_params_unregister(struct devlink *devlink)
mlx5_devlink_eth_param_unregister(devlink);
}
+static int mlx5_devlink_max_uc_list_validate(struct devlink *devlink, u32 id,
+ union devlink_param_value val,
+ struct netlink_ext_ack *extack)
+{
+ struct mlx5_core_dev *dev = devlink_priv(devlink);
+
+ if (val.vu32 == 0) {
+ NL_SET_ERR_MSG_MOD(extack, "max_macs value must be greater than 0");
+ return -EINVAL;
+ }
+
+ if (!is_power_of_2(val.vu32)) {
+ NL_SET_ERR_MSG_MOD(extack, "Only power of 2 values are supported for max_macs");
+ return -EINVAL;
+ }
+
+ if (ilog2(val.vu32) >
+ MLX5_CAP_GEN_MAX(dev, log_max_current_uc_list)) {
+ NL_SET_ERR_MSG_MOD(extack, "max_macs value is out of the supported range");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static const struct devlink_param max_uc_list_param =
+ DEVLINK_PARAM_GENERIC(MAX_MACS, BIT(DEVLINK_PARAM_CMODE_DRIVERINIT),
+ NULL, NULL, mlx5_devlink_max_uc_list_validate);
+
+static int mlx5_devlink_max_uc_list_param_register(struct devlink *devlink)
+{
+ struct mlx5_core_dev *dev = devlink_priv(devlink);
+ union devlink_param_value value;
+ int err;
+
+ if (!MLX5_CAP_GEN_MAX(dev, log_max_current_uc_list_wr_supported))
+ return 0;
+
+ err = devlink_param_register(devlink, &max_uc_list_param);
+ if (err)
+ return err;
+
+ value.vu32 = 1 << MLX5_CAP_GEN(dev, log_max_current_uc_list);
+ devlink_param_driverinit_value_set(devlink,
+ DEVLINK_PARAM_GENERIC_ID_MAX_MACS,
+ value);
+ return 0;
+}
+
+static void
+mlx5_devlink_max_uc_list_param_unregister(struct devlink *devlink)
+{
+ struct mlx5_core_dev *dev = devlink_priv(devlink);
+
+ if (!MLX5_CAP_GEN_MAX(dev, log_max_current_uc_list_wr_supported))
+ return;
+
+ devlink_param_unregister(devlink, &max_uc_list_param);
+}
+
#define MLX5_TRAP_DROP(_id, _group_id) \
DEVLINK_TRAP_GENERIC(DROP, DROP, _id, \
DEVLINK_TRAP_GROUP_GENERIC_ID_##_group_id, \
@@ -811,6 +892,10 @@ int mlx5_devlink_register(struct devlink *devlink)
if (err)
goto auxdev_reg_err;
+ err = mlx5_devlink_max_uc_list_param_register(devlink);
+ if (err)
+ goto max_uc_list_err;
+
err = mlx5_devlink_traps_register(devlink);
if (err)
goto traps_reg_err;
@@ -821,6 +906,8 @@ int mlx5_devlink_register(struct devlink *devlink)
return 0;
traps_reg_err:
+ mlx5_devlink_max_uc_list_param_unregister(devlink);
+max_uc_list_err:
mlx5_devlink_auxdev_params_unregister(devlink);
auxdev_reg_err:
devlink_params_unregister(devlink, mlx5_devlink_params,
@@ -831,6 +918,7 @@ int mlx5_devlink_register(struct devlink *devlink)
void mlx5_devlink_unregister(struct devlink *devlink)
{
mlx5_devlink_traps_unregister(devlink);
+ mlx5_devlink_max_uc_list_param_unregister(devlink);
mlx5_devlink_auxdev_params_unregister(devlink);
devlink_params_unregister(devlink, mlx5_devlink_params,
ARRAY_SIZE(mlx5_devlink_params));
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h
index f0ac6b0..812e681 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h
@@ -145,7 +145,6 @@ struct page_pool;
#define MLX5E_MIN_NUM_CHANNELS 0x1
#define MLX5E_MAX_NUM_CHANNELS (MLX5E_INDIR_RQT_SIZE / 2)
-#define MLX5E_MAX_NUM_SQS (MLX5E_MAX_NUM_CHANNELS * MLX5E_MAX_NUM_TC)
#define MLX5E_TX_CQ_POLL_BUDGET 128
#define MLX5E_TX_XSK_POLL_BUDGET 64
#define MLX5E_SQ_RECOVER_MIN_INTERVAL 500 /* msecs */
@@ -173,7 +172,7 @@ struct page_pool;
#define MLX5E_KLM_ENTRIES_PER_WQE(wqe_size)\
ALIGN_DOWN(MLX5E_KLM_MAX_ENTRIES_PER_WQE(wqe_size), MLX5_UMR_KLM_ALIGNMENT)
-#define MLX5E_MAX_KLM_PER_WQE(mdev) \
+#define MLX5E_MAX_KLM_PER_WQE \
MLX5E_KLM_ENTRIES_PER_WQE(MLX5E_TX_MPW_MAX_NUM_DS << MLX5_MKEY_BSF_OCTO_SIZE)
#define MLX5E_MSG_LEVEL NETIF_MSG_LINK
@@ -783,6 +782,8 @@ struct mlx5e_channel {
DECLARE_BITMAP(state, MLX5E_CHANNEL_NUM_STATES);
int ix;
int cpu;
+ /* Sync between icosq recovery and XSK enable/disable. */
+ struct mutex icosq_recovery_lock;
};
struct mlx5e_ptp;
@@ -875,10 +876,8 @@ struct mlx5e_trap;
struct mlx5e_priv {
/* priv data path fields - start */
- /* +1 for port ptp ts */
- struct mlx5e_txqsq *txq2sq[(MLX5E_MAX_NUM_CHANNELS + 1) * MLX5E_MAX_NUM_TC +
- MLX5E_QOS_MAX_LEAF_NODES];
- int channel_tc2realtxq[MLX5E_MAX_NUM_CHANNELS][MLX5E_MAX_NUM_TC];
+ struct mlx5e_txqsq **txq2sq;
+ int **channel_tc2realtxq;
int port_ptp_tc2realtxq[MLX5E_MAX_NUM_TC];
#ifdef CONFIG_MLX5_CORE_EN_DCB
struct mlx5e_dcbx_dp dcbx_dp;
@@ -893,7 +892,7 @@ struct mlx5e_priv {
struct mlx5e_channels channels;
u32 tisn[MLX5_MAX_PORTS][MLX5E_MAX_NUM_TC];
struct mlx5e_rx_res *rx_res;
- u32 tx_rates[MLX5E_MAX_NUM_SQS];
+ u32 *tx_rates;
struct mlx5e_flow_steering fs;
@@ -909,7 +908,7 @@ struct mlx5e_priv {
struct net_device *netdev;
struct mlx5e_trap *en_trap;
struct mlx5e_stats stats;
- struct mlx5e_channel_stats channel_stats[MLX5E_MAX_NUM_CHANNELS];
+ struct mlx5e_channel_stats **channel_stats;
struct mlx5e_channel_stats trap_stats;
struct mlx5e_ptp_stats ptp_stats;
u16 stats_nch;
@@ -956,6 +955,12 @@ struct mlx5e_rx_handlers {
extern const struct mlx5e_rx_handlers mlx5e_rx_handlers_nic;
+enum mlx5e_profile_feature {
+ MLX5E_PROFILE_FEATURE_PTP_RX,
+ MLX5E_PROFILE_FEATURE_PTP_TX,
+ MLX5E_PROFILE_FEATURE_QOS_HTB,
+};
+
struct mlx5e_profile {
int (*init)(struct mlx5_core_dev *mdev,
struct net_device *netdev);
@@ -969,14 +974,18 @@ struct mlx5e_profile {
int (*update_rx)(struct mlx5e_priv *priv);
void (*update_stats)(struct mlx5e_priv *priv);
void (*update_carrier)(struct mlx5e_priv *priv);
+ int (*max_nch_limit)(struct mlx5_core_dev *mdev);
unsigned int (*stats_grps_num)(struct mlx5e_priv *priv);
mlx5e_stats_grp_t *stats_grps;
const struct mlx5e_rx_handlers *rx_handlers;
int max_tc;
u8 rq_groups;
- bool rx_ptp_support;
+ u32 features;
};
+#define mlx5e_profile_feature_cap(profile, feature) \
+ ((profile)->features & BIT(MLX5E_PROFILE_FEATURE_##feature))
+
void mlx5e_build_ptys2ethtool_map(void);
bool mlx5e_check_fragmented_striding_rq_cap(struct mlx5_core_dev *mdev);
@@ -1014,9 +1023,6 @@ int mlx5e_create_rq(struct mlx5e_rq *rq, struct mlx5e_rq_param *param);
void mlx5e_destroy_rq(struct mlx5e_rq *rq);
struct mlx5e_sq_param;
-int mlx5e_open_icosq(struct mlx5e_channel *c, struct mlx5e_params *params,
- struct mlx5e_sq_param *param, struct mlx5e_icosq *sq);
-void mlx5e_close_icosq(struct mlx5e_icosq *sq);
int mlx5e_open_xdpsq(struct mlx5e_channel *c, struct mlx5e_params *params,
struct mlx5e_sq_param *param, struct xsk_buff_pool *xsk_pool,
struct mlx5e_xdpsq *sq, bool is_redirect);
@@ -1057,7 +1063,6 @@ int mlx5e_safe_switch_params(struct mlx5e_priv *priv,
mlx5e_fp_preactivate preactivate,
void *context, bool reset);
int mlx5e_update_tx_netdev_queues(struct mlx5e_priv *priv);
-int mlx5e_num_channels_changed(struct mlx5e_priv *priv);
int mlx5e_num_channels_changed_ctx(struct mlx5e_priv *priv, void *context);
void mlx5e_activate_priv_channels(struct mlx5e_priv *priv);
void mlx5e_deactivate_priv_channels(struct mlx5e_priv *priv);
@@ -1148,9 +1153,12 @@ void mlx5e_ethtool_get_channels(struct mlx5e_priv *priv,
int mlx5e_ethtool_set_channels(struct mlx5e_priv *priv,
struct ethtool_channels *ch);
int mlx5e_ethtool_get_coalesce(struct mlx5e_priv *priv,
- struct ethtool_coalesce *coal);
+ struct ethtool_coalesce *coal,
+ struct kernel_ethtool_coalesce *kernel_coal);
int mlx5e_ethtool_set_coalesce(struct mlx5e_priv *priv,
- struct ethtool_coalesce *coal);
+ struct ethtool_coalesce *coal,
+ struct kernel_ethtool_coalesce *kernel_coal,
+ struct netlink_ext_ack *extack);
int mlx5e_ethtool_get_link_ksettings(struct mlx5e_priv *priv,
struct ethtool_link_ksettings *link_ksettings);
int mlx5e_ethtool_set_link_ksettings(struct mlx5e_priv *priv,
@@ -1186,8 +1194,7 @@ int mlx5e_priv_init(struct mlx5e_priv *priv,
struct mlx5_core_dev *mdev);
void mlx5e_priv_cleanup(struct mlx5e_priv *priv);
struct net_device *
-mlx5e_create_netdev(struct mlx5_core_dev *mdev, const struct mlx5e_profile *profile,
- unsigned int txqs, unsigned int rxqs);
+mlx5e_create_netdev(struct mlx5_core_dev *mdev, const struct mlx5e_profile *profile);
int mlx5e_attach_netdev(struct mlx5e_priv *priv);
void mlx5e_detach_netdev(struct mlx5e_priv *priv);
void mlx5e_destroy_netdev(struct mlx5e_priv *priv);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/health.h b/drivers/net/ethernet/mellanox/mlx5/core/en/health.h
index d5b7110..0107e4e 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/health.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/health.h
@@ -30,6 +30,8 @@ void mlx5e_reporter_rx_destroy(struct mlx5e_priv *priv);
void mlx5e_reporter_icosq_cqe_err(struct mlx5e_icosq *icosq);
void mlx5e_reporter_rq_cqe_err(struct mlx5e_rq *rq);
void mlx5e_reporter_rx_timeout(struct mlx5e_rq *rq);
+void mlx5e_reporter_icosq_suspend_recovery(struct mlx5e_channel *c);
+void mlx5e_reporter_icosq_resume_recovery(struct mlx5e_channel *c);
#define MLX5E_REPORTER_PER_Q_MAX_LEN 256
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/hv_vhca_stats.c b/drivers/net/ethernet/mellanox/mlx5/core/en/hv_vhca_stats.c
index d290d72..b4f3bd7 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/hv_vhca_stats.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/hv_vhca_stats.c
@@ -20,7 +20,7 @@ mlx5e_hv_vhca_fill_ring_stats(struct mlx5e_priv *priv, int ch,
struct mlx5e_channel_stats *stats;
int tc;
- stats = &priv->channel_stats[ch];
+ stats = priv->channel_stats[ch];
data->rx_packets = stats->rq.packets;
data->rx_bytes = stats->rq.bytes;
@@ -120,14 +120,14 @@ static void mlx5e_hv_vhca_stats_cleanup(struct mlx5_hv_vhca_agent *agent)
cancel_delayed_work_sync(&priv->stats_agent.work);
}
-int mlx5e_hv_vhca_stats_create(struct mlx5e_priv *priv)
+void mlx5e_hv_vhca_stats_create(struct mlx5e_priv *priv)
{
int buf_len = mlx5e_hv_vhca_stats_buf_size(priv);
struct mlx5_hv_vhca_agent *agent;
priv->stats_agent.buf = kvzalloc(buf_len, GFP_KERNEL);
if (!priv->stats_agent.buf)
- return -ENOMEM;
+ return;
agent = mlx5_hv_vhca_agent_create(priv->mdev->hv_vhca,
MLX5_HV_VHCA_AGENT_STATS,
@@ -142,13 +142,11 @@ int mlx5e_hv_vhca_stats_create(struct mlx5e_priv *priv)
PTR_ERR(agent));
kvfree(priv->stats_agent.buf);
- return IS_ERR_OR_NULL(agent);
+ return;
}
priv->stats_agent.agent = agent;
INIT_DELAYED_WORK(&priv->stats_agent.work, mlx5e_hv_vhca_stats_work);
-
- return 0;
}
void mlx5e_hv_vhca_stats_destroy(struct mlx5e_priv *priv)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/hv_vhca_stats.h b/drivers/net/ethernet/mellanox/mlx5/core/en/hv_vhca_stats.h
index 664463f..29c8c6d 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/hv_vhca_stats.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/hv_vhca_stats.h
@@ -7,19 +7,12 @@
#if IS_ENABLED(CONFIG_PCI_HYPERV_INTERFACE)
-int mlx5e_hv_vhca_stats_create(struct mlx5e_priv *priv);
+void mlx5e_hv_vhca_stats_create(struct mlx5e_priv *priv);
void mlx5e_hv_vhca_stats_destroy(struct mlx5e_priv *priv);
#else
-
-static inline int mlx5e_hv_vhca_stats_create(struct mlx5e_priv *priv)
-{
- return 0;
-}
-
-static inline void mlx5e_hv_vhca_stats_destroy(struct mlx5e_priv *priv)
-{
-}
+static inline void mlx5e_hv_vhca_stats_create(struct mlx5e_priv *priv) {}
+static inline void mlx5e_hv_vhca_stats_destroy(struct mlx5e_priv *priv) {}
#endif
#endif /* __MLX5_EN_STATS_VHCA_H__ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/mod_hdr.c b/drivers/net/ethernet/mellanox/mlx5/core/en/mod_hdr.c
index 7edde4d..17325c5 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/mod_hdr.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/mod_hdr.c
@@ -155,3 +155,61 @@ struct mlx5_modify_hdr *mlx5e_mod_hdr_get(struct mlx5e_mod_hdr_handle *mh)
return mh->modify_hdr;
}
+char *
+mlx5e_mod_hdr_alloc(struct mlx5_core_dev *mdev, int namespace,
+ struct mlx5e_tc_mod_hdr_acts *mod_hdr_acts)
+{
+ int new_num_actions, max_hw_actions;
+ size_t new_sz, old_sz;
+ void *ret;
+
+ if (mod_hdr_acts->num_actions < mod_hdr_acts->max_actions)
+ goto out;
+
+ max_hw_actions = mlx5e_mod_hdr_max_actions(mdev, namespace);
+ new_num_actions = min(max_hw_actions,
+ mod_hdr_acts->actions ?
+ mod_hdr_acts->max_actions * 2 : 1);
+ if (mod_hdr_acts->max_actions == new_num_actions)
+ return ERR_PTR(-ENOSPC);
+
+ new_sz = MLX5_MH_ACT_SZ * new_num_actions;
+ old_sz = mod_hdr_acts->max_actions * MLX5_MH_ACT_SZ;
+
+ if (mod_hdr_acts->is_static) {
+ ret = kzalloc(new_sz, GFP_KERNEL);
+ if (ret) {
+ memcpy(ret, mod_hdr_acts->actions, old_sz);
+ mod_hdr_acts->is_static = false;
+ }
+ } else {
+ ret = krealloc(mod_hdr_acts->actions, new_sz, GFP_KERNEL);
+ if (ret)
+ memset(ret + old_sz, 0, new_sz - old_sz);
+ }
+ if (!ret)
+ return ERR_PTR(-ENOMEM);
+
+ mod_hdr_acts->actions = ret;
+ mod_hdr_acts->max_actions = new_num_actions;
+
+out:
+ return mod_hdr_acts->actions + (mod_hdr_acts->num_actions * MLX5_MH_ACT_SZ);
+}
+
+void
+mlx5e_mod_hdr_dealloc(struct mlx5e_tc_mod_hdr_acts *mod_hdr_acts)
+{
+ if (!mod_hdr_acts->is_static)
+ kfree(mod_hdr_acts->actions);
+
+ mod_hdr_acts->actions = NULL;
+ mod_hdr_acts->num_actions = 0;
+ mod_hdr_acts->max_actions = 0;
+}
+
+char *
+mlx5e_mod_hdr_get_item(struct mlx5e_tc_mod_hdr_acts *mod_hdr_acts, int pos)
+{
+ return mod_hdr_acts->actions + (pos * MLX5_MH_ACT_SZ);
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/mod_hdr.h b/drivers/net/ethernet/mellanox/mlx5/core/en/mod_hdr.h
index 33b23d8..b8dac41 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/mod_hdr.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/mod_hdr.h
@@ -7,14 +7,32 @@
#include <linux/hashtable.h>
#include <linux/mlx5/fs.h>
+#define MLX5_MH_ACT_SZ MLX5_UN_SZ_BYTES(set_add_copy_action_in_auto)
+
struct mlx5e_mod_hdr_handle;
struct mlx5e_tc_mod_hdr_acts {
int num_actions;
int max_actions;
+ bool is_static;
void *actions;
};
+#define DECLARE_MOD_HDR_ACTS_ACTIONS(name, len) \
+ u8 name[len][MLX5_MH_ACT_SZ] = {}
+
+#define DECLARE_MOD_HDR_ACTS(name, acts_arr) \
+ struct mlx5e_tc_mod_hdr_acts name = { \
+ .max_actions = ARRAY_SIZE(acts_arr), \
+ .is_static = true, \
+ .actions = acts_arr, \
+ }
+
+char *mlx5e_mod_hdr_alloc(struct mlx5_core_dev *mdev, int namespace,
+ struct mlx5e_tc_mod_hdr_acts *mod_hdr_acts);
+void mlx5e_mod_hdr_dealloc(struct mlx5e_tc_mod_hdr_acts *mod_hdr_acts);
+char *mlx5e_mod_hdr_get_item(struct mlx5e_tc_mod_hdr_acts *mod_hdr_acts, int pos);
+
struct mlx5e_mod_hdr_handle *
mlx5e_mod_hdr_attach(struct mlx5_core_dev *mdev,
struct mod_hdr_tbl *tbl,
@@ -28,4 +46,12 @@ struct mlx5_modify_hdr *mlx5e_mod_hdr_get(struct mlx5e_mod_hdr_handle *mh);
void mlx5e_mod_hdr_tbl_init(struct mod_hdr_tbl *tbl);
void mlx5e_mod_hdr_tbl_destroy(struct mod_hdr_tbl *tbl);
+static inline int mlx5e_mod_hdr_max_actions(struct mlx5_core_dev *mdev, int namespace)
+{
+ if (namespace == MLX5_FLOW_NAMESPACE_FDB) /* FDB offloading */
+ return MLX5_CAP_ESW_FLOWTABLE_FDB(mdev, max_modify_header_actions);
+ else /* namespace is MLX5_FLOW_NAMESPACE_KERNEL - NIC offloading */
+ return MLX5_CAP_FLOWTABLE_NIC_RX(mdev, max_modify_header_actions);
+}
+
#endif /* __MLX5E_EN_MOD_HDR_H__ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/params.c b/drivers/net/ethernet/mellanox/mlx5/core/en/params.c
index f8c2902..66180ff 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/params.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/params.c
@@ -717,7 +717,7 @@ static u32 mlx5e_shampo_icosq_sz(struct mlx5_core_dev *mdev,
int wq_size = BIT(MLX5_GET(wq, wqc, log_wq_sz));
u32 wqebbs;
- max_klm_per_umr = MLX5E_MAX_KLM_PER_WQE(mdev);
+ max_klm_per_umr = MLX5E_MAX_KLM_PER_WQE;
max_hd_per_wqe = mlx5e_shampo_hd_per_wqe(mdev, params, rq_param);
max_num_of_umr_per_wqe = max_hd_per_wqe / max_klm_per_umr;
rest = max_hd_per_wqe % max_klm_per_umr;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c b/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c
index 18d542b..82baafd 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c
@@ -768,7 +768,7 @@ int mlx5e_ptp_alloc_rx_fs(struct mlx5e_priv *priv)
{
struct mlx5e_ptp_fs *ptp_fs;
- if (!priv->profile->rx_ptp_support)
+ if (!mlx5e_profile_feature_cap(priv->profile, PTP_RX))
return 0;
ptp_fs = kzalloc(sizeof(*ptp_fs), GFP_KERNEL);
@@ -783,7 +783,7 @@ void mlx5e_ptp_free_rx_fs(struct mlx5e_priv *priv)
{
struct mlx5e_ptp_fs *ptp_fs = priv->fs.ptp_fs;
- if (!priv->profile->rx_ptp_support)
+ if (!mlx5e_profile_feature_cap(priv->profile, PTP_RX))
return;
mlx5e_ptp_rx_unset_fs(priv);
@@ -794,7 +794,7 @@ int mlx5e_ptp_rx_manage_fs(struct mlx5e_priv *priv, bool set)
{
struct mlx5e_ptp *c = priv->channels.ptp;
- if (!priv->profile->rx_ptp_support)
+ if (!mlx5e_profile_feature_cap(priv->profile, PTP_RX))
return 0;
if (!test_bit(MLX5E_STATE_OPENED, &priv->state))
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/qos.c b/drivers/net/ethernet/mellanox/mlx5/core/en/qos.c
index 50977f0..00449df 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/qos.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/qos.c
@@ -1,5 +1,6 @@
// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
/* Copyright (c) 2020, Mellanox Technologies inc. All rights reserved. */
+#include <net/sch_generic.h>
#include "en.h"
#include "params.h"
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/rep/tc.c b/drivers/net/ethernet/mellanox/mlx5/core/en/rep/tc.c
index fcb0892..0991345 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/rep/tc.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/rep/tc.c
@@ -517,6 +517,9 @@ int mlx5e_rep_indr_setup_cb(struct net_device *netdev, struct Qdisc *sch, void *
void *data,
void (*cleanup)(struct flow_block_cb *block_cb))
{
+ if (!netdev)
+ return -EOPNOTSUPP;
+
switch (type) {
case TC_SETUP_BLOCK:
return mlx5e_rep_indr_setup_block(netdev, sch, cb_priv, type_data,
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/rep/tc.h b/drivers/net/ethernet/mellanox/mlx5/core/en/rep/tc.h
index d6c7c81..7c9dd3a 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/rep/tc.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/rep/tc.h
@@ -66,7 +66,7 @@ mlx5e_rep_setup_tc(struct net_device *dev, enum tc_setup_type type,
static inline void
mlx5e_rep_tc_receive(struct mlx5_cqe64 *cqe, struct mlx5e_rq *rq,
- struct sk_buff *skb) {}
+ struct sk_buff *skb) { napi_gro_receive(rq->cq.napi, skb); }
#endif /* CONFIG_MLX5_CLS_ACT */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/reporter_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en/reporter_rx.c
index 74086eb55..2684e9d 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/reporter_rx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/reporter_rx.c
@@ -62,6 +62,7 @@ static void mlx5e_reset_icosq_cc_pc(struct mlx5e_icosq *icosq)
static int mlx5e_rx_reporter_err_icosq_cqe_recover(void *ctx)
{
+ struct mlx5e_rq *xskrq = NULL;
struct mlx5_core_dev *mdev;
struct mlx5e_icosq *icosq;
struct net_device *dev;
@@ -70,7 +71,13 @@ static int mlx5e_rx_reporter_err_icosq_cqe_recover(void *ctx)
int err;
icosq = ctx;
+
+ mutex_lock(&icosq->channel->icosq_recovery_lock);
+
+ /* mlx5e_close_rq cancels this work before RQ and ICOSQ are killed. */
rq = &icosq->channel->rq;
+ if (test_bit(MLX5E_RQ_STATE_ENABLED, &icosq->channel->xskrq.state))
+ xskrq = &icosq->channel->xskrq;
mdev = icosq->channel->mdev;
dev = icosq->channel->netdev;
err = mlx5_core_query_sq_state(mdev, icosq->sqn, &state);
@@ -84,6 +91,9 @@ static int mlx5e_rx_reporter_err_icosq_cqe_recover(void *ctx)
goto out;
mlx5e_deactivate_rq(rq);
+ if (xskrq)
+ mlx5e_deactivate_rq(xskrq);
+
err = mlx5e_wait_for_icosq_flush(icosq);
if (err)
goto out;
@@ -97,15 +107,28 @@ static int mlx5e_rx_reporter_err_icosq_cqe_recover(void *ctx)
goto out;
mlx5e_reset_icosq_cc_pc(icosq);
+
mlx5e_free_rx_in_progress_descs(rq);
+ if (xskrq)
+ mlx5e_free_rx_in_progress_descs(xskrq);
+
clear_bit(MLX5E_SQ_STATE_RECOVERING, &icosq->state);
mlx5e_activate_icosq(icosq);
- mlx5e_activate_rq(rq);
+ mlx5e_activate_rq(rq);
rq->stats->recover++;
+
+ if (xskrq) {
+ mlx5e_activate_rq(xskrq);
+ xskrq->stats->recover++;
+ }
+
+ mutex_unlock(&icosq->channel->icosq_recovery_lock);
+
return 0;
out:
clear_bit(MLX5E_SQ_STATE_RECOVERING, &icosq->state);
+ mutex_unlock(&icosq->channel->icosq_recovery_lock);
return err;
}
@@ -706,6 +729,16 @@ void mlx5e_reporter_icosq_cqe_err(struct mlx5e_icosq *icosq)
mlx5e_health_report(priv, priv->rx_reporter, err_str, &err_ctx);
}
+void mlx5e_reporter_icosq_suspend_recovery(struct mlx5e_channel *c)
+{
+ mutex_lock(&c->icosq_recovery_lock);
+}
+
+void mlx5e_reporter_icosq_resume_recovery(struct mlx5e_channel *c)
+{
+ mutex_unlock(&c->icosq_recovery_lock);
+}
+
static const struct devlink_health_reporter_ops mlx5_rx_reporter_ops = {
.name = "rx",
.recover = mlx5e_rx_reporter_recover,
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/reporter_tx.c b/drivers/net/ethernet/mellanox/mlx5/core/en/reporter_tx.c
index 4f4bc87..60bc5b5 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/reporter_tx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/reporter_tx.c
@@ -466,6 +466,14 @@ static int mlx5e_tx_reporter_dump_sq(struct mlx5e_priv *priv, struct devlink_fms
return mlx5e_health_fmsg_named_obj_nest_end(fmsg);
}
+static int mlx5e_tx_reporter_timeout_dump(struct mlx5e_priv *priv, struct devlink_fmsg *fmsg,
+ void *ctx)
+{
+ struct mlx5e_tx_timeout_ctx *to_ctx = ctx;
+
+ return mlx5e_tx_reporter_dump_sq(priv, fmsg, to_ctx->sq);
+}
+
static int mlx5e_tx_reporter_dump_all_sqs(struct mlx5e_priv *priv,
struct devlink_fmsg *fmsg)
{
@@ -561,11 +569,11 @@ int mlx5e_reporter_tx_timeout(struct mlx5e_txqsq *sq)
to_ctx.sq = sq;
err_ctx.ctx = &to_ctx;
err_ctx.recover = mlx5e_tx_reporter_timeout_recover;
- err_ctx.dump = mlx5e_tx_reporter_dump_sq;
+ err_ctx.dump = mlx5e_tx_reporter_timeout_dump;
snprintf(err_str, sizeof(err_str),
"TX timeout on queue: %d, SQ: 0x%x, CQ: 0x%x, SQ Cons: 0x%x SQ Prod: 0x%x, usecs since last trans: %u",
sq->ch_ix, sq->sqn, sq->cq.mcq.cqn, sq->cc, sq->pc,
- jiffies_to_usecs(jiffies - sq->txq->trans_start));
+ jiffies_to_usecs(jiffies - READ_ONCE(sq->txq->trans_start)));
mlx5e_health_report(priv, priv->tx_reporter, err_str, &err_ctx);
return to_ctx.status;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/rx_res.c b/drivers/net/ethernet/mellanox/mlx5/core/en/rx_res.c
index 0015a81..24c32f7 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/rx_res.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/rx_res.c
@@ -37,7 +37,6 @@ struct mlx5e_rx_res {
/* API for rx_res_rss_* */
static int mlx5e_rx_res_rss_init_def(struct mlx5e_rx_res *res,
- const struct mlx5e_packet_merge_param *init_pkt_merge_param,
unsigned int init_nch)
{
bool inner_ft_support = res->features & MLX5E_RX_RES_FEATURE_INNER_FT;
@@ -52,7 +51,7 @@ static int mlx5e_rx_res_rss_init_def(struct mlx5e_rx_res *res,
return -ENOMEM;
err = mlx5e_rss_init(rss, res->mdev, inner_ft_support, res->drop_rqn,
- init_pkt_merge_param);
+ &res->pkt_merge_param);
if (err)
goto err_rss_free;
@@ -277,8 +276,7 @@ struct mlx5e_rx_res *mlx5e_rx_res_alloc(void)
return kvzalloc(sizeof(struct mlx5e_rx_res), GFP_KERNEL);
}
-static int mlx5e_rx_res_channels_init(struct mlx5e_rx_res *res,
- const struct mlx5e_packet_merge_param *init_pkt_merge_param)
+static int mlx5e_rx_res_channels_init(struct mlx5e_rx_res *res)
{
bool inner_ft_support = res->features & MLX5E_RX_RES_FEATURE_INNER_FT;
struct mlx5e_tir_builder *builder;
@@ -309,7 +307,7 @@ static int mlx5e_rx_res_channels_init(struct mlx5e_rx_res *res,
mlx5e_tir_builder_build_rqt(builder, res->mdev->mlx5e_res.hw_objs.td.tdn,
mlx5e_rqt_get_rqtn(&res->channels[ix].direct_rqt),
inner_ft_support);
- mlx5e_tir_builder_build_packet_merge(builder, init_pkt_merge_param);
+ mlx5e_tir_builder_build_packet_merge(builder, &res->pkt_merge_param);
mlx5e_tir_builder_build_direct(builder);
err = mlx5e_tir_init(&res->channels[ix].direct_tir, builder, res->mdev, true);
@@ -339,7 +337,7 @@ static int mlx5e_rx_res_channels_init(struct mlx5e_rx_res *res,
mlx5e_tir_builder_build_rqt(builder, res->mdev->mlx5e_res.hw_objs.td.tdn,
mlx5e_rqt_get_rqtn(&res->channels[ix].xsk_rqt),
inner_ft_support);
- mlx5e_tir_builder_build_packet_merge(builder, init_pkt_merge_param);
+ mlx5e_tir_builder_build_packet_merge(builder, &res->pkt_merge_param);
mlx5e_tir_builder_build_direct(builder);
err = mlx5e_tir_init(&res->channels[ix].xsk_tir, builder, res->mdev, true);
@@ -454,11 +452,11 @@ int mlx5e_rx_res_init(struct mlx5e_rx_res *res, struct mlx5_core_dev *mdev,
res->pkt_merge_param = *init_pkt_merge_param;
init_rwsem(&res->pkt_merge_param_sem);
- err = mlx5e_rx_res_rss_init_def(res, init_pkt_merge_param, init_nch);
+ err = mlx5e_rx_res_rss_init_def(res, init_nch);
if (err)
goto err_out;
- err = mlx5e_rx_res_channels_init(res, init_pkt_merge_param);
+ err = mlx5e_rx_res_channels_init(res);
if (err)
goto err_rss_destroy;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/accept.c b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/accept.c
new file mode 100644
index 0000000..b0de6b9
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/accept.c
@@ -0,0 +1,31 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+// Copyright (c) 2021, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#include "act.h"
+#include "en/tc_priv.h"
+
+static bool
+tc_act_can_offload_accept(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ int act_index)
+{
+ return true;
+}
+
+static int
+tc_act_parse_accept(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ struct mlx5e_priv *priv,
+ struct mlx5_flow_attr *attr)
+{
+ attr->action |= MLX5_FLOW_CONTEXT_ACTION_FWD_DEST |
+ MLX5_FLOW_CONTEXT_ACTION_COUNT;
+ attr->flags |= MLX5_ESW_ATTR_FLAG_ACCEPT;
+
+ return 0;
+}
+
+struct mlx5e_tc_act mlx5e_tc_act_accept = {
+ .can_offload = tc_act_can_offload_accept,
+ .parse_action = tc_act_parse_accept,
+};
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/act.c b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/act.c
new file mode 100644
index 0000000..e600924
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/act.c
@@ -0,0 +1,103 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+// Copyright (c) 2021, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#include "act.h"
+#include "en/tc_priv.h"
+#include "mlx5_core.h"
+
+/* Must be aligned with enum flow_action_id. */
+static struct mlx5e_tc_act *tc_acts_fdb[NUM_FLOW_ACTIONS] = {
+ &mlx5e_tc_act_accept,
+ &mlx5e_tc_act_drop,
+ &mlx5e_tc_act_trap,
+ &mlx5e_tc_act_goto,
+ &mlx5e_tc_act_mirred,
+ &mlx5e_tc_act_mirred,
+ &mlx5e_tc_act_redirect_ingress,
+ NULL, /* FLOW_ACTION_MIRRED_INGRESS, */
+ &mlx5e_tc_act_vlan,
+ &mlx5e_tc_act_vlan,
+ &mlx5e_tc_act_vlan_mangle,
+ &mlx5e_tc_act_tun_encap,
+ &mlx5e_tc_act_tun_decap,
+ &mlx5e_tc_act_pedit,
+ &mlx5e_tc_act_pedit,
+ &mlx5e_tc_act_csum,
+ NULL, /* FLOW_ACTION_MARK, */
+ &mlx5e_tc_act_ptype,
+ NULL, /* FLOW_ACTION_PRIORITY, */
+ NULL, /* FLOW_ACTION_WAKE, */
+ NULL, /* FLOW_ACTION_QUEUE, */
+ &mlx5e_tc_act_sample,
+ NULL, /* FLOW_ACTION_POLICE, */
+ &mlx5e_tc_act_ct,
+ NULL, /* FLOW_ACTION_CT_METADATA, */
+ &mlx5e_tc_act_mpls_push,
+ &mlx5e_tc_act_mpls_pop,
+};
+
+/* Must be aligned with enum flow_action_id. */
+static struct mlx5e_tc_act *tc_acts_nic[NUM_FLOW_ACTIONS] = {
+ &mlx5e_tc_act_accept,
+ &mlx5e_tc_act_drop,
+ NULL, /* FLOW_ACTION_TRAP, */
+ &mlx5e_tc_act_goto,
+ &mlx5e_tc_act_mirred_nic,
+ NULL, /* FLOW_ACTION_MIRRED, */
+ NULL, /* FLOW_ACTION_REDIRECT_INGRESS, */
+ NULL, /* FLOW_ACTION_MIRRED_INGRESS, */
+ NULL, /* FLOW_ACTION_VLAN_PUSH, */
+ NULL, /* FLOW_ACTION_VLAN_POP, */
+ NULL, /* FLOW_ACTION_VLAN_MANGLE, */
+ NULL, /* FLOW_ACTION_TUNNEL_ENCAP, */
+ NULL, /* FLOW_ACTION_TUNNEL_DECAP, */
+ &mlx5e_tc_act_pedit,
+ &mlx5e_tc_act_pedit,
+ &mlx5e_tc_act_csum,
+ &mlx5e_tc_act_mark,
+ NULL, /* FLOW_ACTION_PTYPE, */
+ NULL, /* FLOW_ACTION_PRIORITY, */
+ NULL, /* FLOW_ACTION_WAKE, */
+ NULL, /* FLOW_ACTION_QUEUE, */
+ NULL, /* FLOW_ACTION_SAMPLE, */
+ NULL, /* FLOW_ACTION_POLICE, */
+ &mlx5e_tc_act_ct,
+};
+
+/**
+ * mlx5e_tc_act_get() - Get an action parser for an action id.
+ * @act_id: Flow action id.
+ * @ns_type: flow namespace type.
+ */
+struct mlx5e_tc_act *
+mlx5e_tc_act_get(enum flow_action_id act_id,
+ enum mlx5_flow_namespace_type ns_type)
+{
+ struct mlx5e_tc_act **tc_acts;
+
+ tc_acts = ns_type == MLX5_FLOW_NAMESPACE_FDB ? tc_acts_fdb : tc_acts_nic;
+
+ return tc_acts[act_id];
+}
+
+/**
+ * mlx5e_tc_act_init_parse_state() - Init a new parse_state.
+ * @parse_state: Parsing state.
+ * @flow: mlx5e tc flow being handled.
+ * @flow_action: flow action to parse.
+ * @extack: to set an error msg.
+ *
+ * The same parse_state should be passed to action parsers
+ * for tracking the current parsing state.
+ */
+void
+mlx5e_tc_act_init_parse_state(struct mlx5e_tc_act_parse_state *parse_state,
+ struct mlx5e_tc_flow *flow,
+ struct flow_action *flow_action,
+ struct netlink_ext_ack *extack)
+{
+ memset(parse_state, 0, sizeof(*parse_state));
+ parse_state->flow = flow;
+ parse_state->num_actions = flow_action->num_entries;
+ parse_state->extack = extack;
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/act.h b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/act.h
new file mode 100644
index 0000000..26efa33
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/act.h
@@ -0,0 +1,75 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+/* Copyright (c) 2021, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */
+
+#ifndef __MLX5_EN_TC_ACT_H__
+#define __MLX5_EN_TC_ACT_H__
+
+#include <net/tc_act/tc_pedit.h>
+#include <net/flow_offload.h>
+#include <linux/netlink.h>
+#include "eswitch.h"
+#include "pedit.h"
+
+struct mlx5_flow_attr;
+
+struct mlx5e_tc_act_parse_state {
+ unsigned int num_actions;
+ struct mlx5e_tc_flow *flow;
+ struct netlink_ext_ack *extack;
+ bool encap;
+ bool decap;
+ bool mpls_push;
+ bool ptype_host;
+ const struct ip_tunnel_info *tun_info;
+ struct pedit_headers_action hdrs[__PEDIT_CMD_MAX];
+ int ifindexes[MLX5_MAX_FLOW_FWD_VPORTS];
+ int if_count;
+ struct mlx5_tc_ct_priv *ct_priv;
+};
+
+struct mlx5e_tc_act {
+ bool (*can_offload)(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ int act_index);
+
+ int (*parse_action)(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ struct mlx5e_priv *priv,
+ struct mlx5_flow_attr *attr);
+
+ int (*post_parse)(struct mlx5e_tc_act_parse_state *parse_state,
+ struct mlx5e_priv *priv,
+ struct mlx5_flow_attr *attr);
+};
+
+extern struct mlx5e_tc_act mlx5e_tc_act_drop;
+extern struct mlx5e_tc_act mlx5e_tc_act_trap;
+extern struct mlx5e_tc_act mlx5e_tc_act_accept;
+extern struct mlx5e_tc_act mlx5e_tc_act_mark;
+extern struct mlx5e_tc_act mlx5e_tc_act_goto;
+extern struct mlx5e_tc_act mlx5e_tc_act_tun_encap;
+extern struct mlx5e_tc_act mlx5e_tc_act_tun_decap;
+extern struct mlx5e_tc_act mlx5e_tc_act_csum;
+extern struct mlx5e_tc_act mlx5e_tc_act_pedit;
+extern struct mlx5e_tc_act mlx5e_tc_act_vlan;
+extern struct mlx5e_tc_act mlx5e_tc_act_vlan_mangle;
+extern struct mlx5e_tc_act mlx5e_tc_act_mpls_push;
+extern struct mlx5e_tc_act mlx5e_tc_act_mpls_pop;
+extern struct mlx5e_tc_act mlx5e_tc_act_mirred;
+extern struct mlx5e_tc_act mlx5e_tc_act_mirred_nic;
+extern struct mlx5e_tc_act mlx5e_tc_act_ct;
+extern struct mlx5e_tc_act mlx5e_tc_act_sample;
+extern struct mlx5e_tc_act mlx5e_tc_act_ptype;
+extern struct mlx5e_tc_act mlx5e_tc_act_redirect_ingress;
+
+struct mlx5e_tc_act *
+mlx5e_tc_act_get(enum flow_action_id act_id,
+ enum mlx5_flow_namespace_type ns_type);
+
+void
+mlx5e_tc_act_init_parse_state(struct mlx5e_tc_act_parse_state *parse_state,
+ struct mlx5e_tc_flow *flow,
+ struct flow_action *flow_action,
+ struct netlink_ext_ack *extack);
+
+#endif /* __MLX5_EN_TC_ACT_H__ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/csum.c b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/csum.c
new file mode 100644
index 0000000..29920ef
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/csum.c
@@ -0,0 +1,61 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+// Copyright (c) 2021, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#include <linux/tc_act/tc_csum.h>
+#include "act.h"
+#include "en/tc_priv.h"
+
+static bool
+csum_offload_supported(struct mlx5e_priv *priv,
+ u32 action,
+ u32 update_flags,
+ struct netlink_ext_ack *extack)
+{
+ u32 prot_flags = TCA_CSUM_UPDATE_FLAG_IPV4HDR | TCA_CSUM_UPDATE_FLAG_TCP |
+ TCA_CSUM_UPDATE_FLAG_UDP;
+
+ /* The HW recalcs checksums only if re-writing headers */
+ if (!(action & MLX5_FLOW_CONTEXT_ACTION_MOD_HDR)) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "TC csum action is only offloaded with pedit");
+ netdev_warn(priv->netdev,
+ "TC csum action is only offloaded with pedit\n");
+ return false;
+ }
+
+ if (update_flags & ~prot_flags) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "can't offload TC csum action for some header/s");
+ netdev_warn(priv->netdev,
+ "can't offload TC csum action for some header/s - flags %#x\n",
+ update_flags);
+ return false;
+ }
+
+ return true;
+}
+
+static bool
+tc_act_can_offload_csum(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ int act_index)
+{
+ struct mlx5e_tc_flow *flow = parse_state->flow;
+
+ return csum_offload_supported(flow->priv, flow->attr->action,
+ act->csum_flags, parse_state->extack);
+}
+
+static int
+tc_act_parse_csum(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ struct mlx5e_priv *priv,
+ struct mlx5_flow_attr *attr)
+{
+ return 0;
+}
+
+struct mlx5e_tc_act mlx5e_tc_act_csum = {
+ .can_offload = tc_act_can_offload_csum,
+ .parse_action = tc_act_parse_csum,
+};
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/ct.c b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/ct.c
new file mode 100644
index 0000000..06ec30c
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/ct.c
@@ -0,0 +1,50 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+// Copyright (c) 2021, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#include "act.h"
+#include "en/tc_priv.h"
+#include "en/tc_ct.h"
+
+static bool
+tc_act_can_offload_ct(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ int act_index)
+{
+ struct netlink_ext_ack *extack = parse_state->extack;
+
+ if (flow_flag_test(parse_state->flow, SAMPLE)) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "Sample action with connection tracking is not supported");
+ return false;
+ }
+
+ return true;
+}
+
+static int
+tc_act_parse_ct(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ struct mlx5e_priv *priv,
+ struct mlx5_flow_attr *attr)
+{
+ int err;
+
+ err = mlx5_tc_ct_parse_action(parse_state->ct_priv, attr,
+ &attr->parse_attr->mod_hdr_acts,
+ act, parse_state->extack);
+ if (err)
+ return err;
+
+ flow_flag_set(parse_state->flow, CT);
+
+ if (mlx5e_is_eswitch_flow(parse_state->flow))
+ attr->esw_attr->split_count = attr->esw_attr->out_count;
+
+ return 0;
+}
+
+struct mlx5e_tc_act mlx5e_tc_act_ct = {
+ .can_offload = tc_act_can_offload_ct,
+ .parse_action = tc_act_parse_ct,
+};
+
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/drop.c b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/drop.c
new file mode 100644
index 0000000..2e29a23
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/drop.c
@@ -0,0 +1,30 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+// Copyright (c) 2021, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#include "act.h"
+#include "en/tc_priv.h"
+
+static bool
+tc_act_can_offload_drop(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ int act_index)
+{
+ return true;
+}
+
+static int
+tc_act_parse_drop(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ struct mlx5e_priv *priv,
+ struct mlx5_flow_attr *attr)
+{
+ attr->action |= MLX5_FLOW_CONTEXT_ACTION_DROP |
+ MLX5_FLOW_CONTEXT_ACTION_COUNT;
+
+ return 0;
+}
+
+struct mlx5e_tc_act mlx5e_tc_act_drop = {
+ .can_offload = tc_act_can_offload_drop,
+ .parse_action = tc_act_parse_drop,
+};
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/goto.c b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/goto.c
new file mode 100644
index 0000000..f445150
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/goto.c
@@ -0,0 +1,122 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+// Copyright (c) 2021, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#include "act.h"
+#include "en/tc_priv.h"
+#include "eswitch.h"
+
+static int
+validate_goto_chain(struct mlx5e_priv *priv,
+ struct mlx5e_tc_flow *flow,
+ const struct flow_action_entry *act,
+ struct netlink_ext_ack *extack)
+{
+ bool is_esw = mlx5e_is_eswitch_flow(flow);
+ bool ft_flow = mlx5e_is_ft_flow(flow);
+ u32 dest_chain = act->chain_index;
+ struct mlx5_fs_chains *chains;
+ struct mlx5_eswitch *esw;
+ u32 reformat_and_fwd;
+ u32 max_chain;
+
+ esw = priv->mdev->priv.eswitch;
+ chains = is_esw ? esw_chains(esw) : mlx5e_nic_chains(priv);
+ max_chain = mlx5_chains_get_chain_range(chains);
+ reformat_and_fwd = is_esw ?
+ MLX5_CAP_ESW_FLOWTABLE_FDB(priv->mdev, reformat_and_fwd_to_table) :
+ MLX5_CAP_FLOWTABLE_NIC_RX(priv->mdev, reformat_and_fwd_to_table);
+
+ if (ft_flow) {
+ NL_SET_ERR_MSG_MOD(extack, "Goto action is not supported");
+ return -EOPNOTSUPP;
+ }
+
+ if (!mlx5_chains_backwards_supported(chains) &&
+ dest_chain <= flow->attr->chain) {
+ NL_SET_ERR_MSG_MOD(extack, "Goto lower numbered chain isn't supported");
+ return -EOPNOTSUPP;
+ }
+
+ if (dest_chain > max_chain) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "Requested destination chain is out of supported range");
+ return -EOPNOTSUPP;
+ }
+
+ if (flow->attr->action & (MLX5_FLOW_CONTEXT_ACTION_PACKET_REFORMAT |
+ MLX5_FLOW_CONTEXT_ACTION_DECAP) &&
+ !reformat_and_fwd) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "Goto chain is not allowed if action has reformat or decap");
+ return -EOPNOTSUPP;
+ }
+
+ return 0;
+}
+
+static bool
+tc_act_can_offload_goto(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ int act_index)
+{
+ struct netlink_ext_ack *extack = parse_state->extack;
+ struct mlx5e_tc_flow *flow = parse_state->flow;
+
+ if (validate_goto_chain(flow->priv, flow, act, extack))
+ return false;
+
+ return true;
+}
+
+static int
+tc_act_parse_goto(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ struct mlx5e_priv *priv,
+ struct mlx5_flow_attr *attr)
+{
+ attr->action |= MLX5_FLOW_CONTEXT_ACTION_FWD_DEST |
+ MLX5_FLOW_CONTEXT_ACTION_COUNT;
+ attr->dest_chain = act->chain_index;
+
+ return 0;
+}
+
+static int
+tc_act_post_parse_goto(struct mlx5e_tc_act_parse_state *parse_state,
+ struct mlx5e_priv *priv,
+ struct mlx5_flow_attr *attr)
+{
+ struct mlx5e_tc_flow_parse_attr *parse_attr = attr->parse_attr;
+ struct netlink_ext_ack *extack = parse_state->extack;
+ struct mlx5e_tc_flow *flow = parse_state->flow;
+
+ if (!attr->dest_chain)
+ return 0;
+
+ if (parse_state->decap) {
+ /* It can be supported if we'll create a mapping for
+ * the tunnel device only (without tunnel), and set
+ * this tunnel id with this decap flow.
+ *
+ * On restore (miss), we'll just set this saved tunnel
+ * device.
+ */
+
+ NL_SET_ERR_MSG_MOD(extack, "Decap with goto isn't supported");
+ netdev_warn(priv->netdev, "Decap with goto isn't supported");
+ return -EOPNOTSUPP;
+ }
+
+ if (!mlx5e_is_eswitch_flow(flow) && parse_attr->mirred_ifindex[0]) {
+ NL_SET_ERR_MSG_MOD(extack, "Mirroring goto chain rules isn't supported");
+ return -EOPNOTSUPP;
+ }
+
+ return 0;
+}
+
+struct mlx5e_tc_act mlx5e_tc_act_goto = {
+ .can_offload = tc_act_can_offload_goto,
+ .parse_action = tc_act_parse_goto,
+ .post_parse = tc_act_post_parse_goto,
+};
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/mark.c b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/mark.c
new file mode 100644
index 0000000..d775c3d9
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/mark.c
@@ -0,0 +1,35 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+// Copyright (c) 2021, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#include "act.h"
+#include "en_tc.h"
+
+static bool
+tc_act_can_offload_mark(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ int act_index)
+{
+ if (act->mark & ~MLX5E_TC_FLOW_ID_MASK) {
+ NL_SET_ERR_MSG_MOD(parse_state->extack, "Bad flow mark, only 16 bit supported");
+ return false;
+ }
+
+ return true;
+}
+
+static int
+tc_act_parse_mark(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ struct mlx5e_priv *priv,
+ struct mlx5_flow_attr *attr)
+{
+ attr->nic_attr->flow_tag = act->mark;
+ attr->action |= MLX5_FLOW_CONTEXT_ACTION_FWD_DEST;
+
+ return 0;
+}
+
+struct mlx5e_tc_act mlx5e_tc_act_mark = {
+ .can_offload = tc_act_can_offload_mark,
+ .parse_action = tc_act_parse_mark,
+};
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/mirred.c b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/mirred.c
new file mode 100644
index 0000000..c614fc7
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/mirred.c
@@ -0,0 +1,307 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+// Copyright (c) 2021, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#include <linux/if_macvlan.h>
+#include <linux/if_vlan.h>
+#include <net/bareudp.h>
+#include <net/bonding.h>
+#include "act.h"
+#include "vlan.h"
+#include "en/tc_tun_encap.h"
+#include "en/tc_priv.h"
+#include "en_rep.h"
+
+static bool
+same_vf_reps(struct mlx5e_priv *priv, struct net_device *out_dev)
+{
+ return mlx5e_eswitch_vf_rep(priv->netdev) &&
+ priv->netdev == out_dev;
+}
+
+static int
+verify_uplink_forwarding(struct mlx5e_priv *priv,
+ struct mlx5_flow_attr *attr,
+ struct net_device *out_dev,
+ struct netlink_ext_ack *extack)
+{
+ struct mlx5_eswitch *esw = priv->mdev->priv.eswitch;
+ struct mlx5e_rep_priv *rep_priv;
+
+ /* Forwarding non encapsulated traffic between
+ * uplink ports is allowed only if
+ * termination_table_raw_traffic cap is set.
+ *
+ * Input vport was stored attr->in_rep.
+ * In LAG case, *priv* is the private data of
+ * uplink which may be not the input vport.
+ */
+ rep_priv = mlx5e_rep_to_rep_priv(attr->esw_attr->in_rep);
+
+ if (!(mlx5e_eswitch_uplink_rep(rep_priv->netdev) &&
+ mlx5e_eswitch_uplink_rep(out_dev)))
+ return 0;
+
+ if (!MLX5_CAP_ESW_FLOWTABLE_FDB(esw->dev,
+ termination_table_raw_traffic)) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "devices are both uplink, can't offload forwarding");
+ return -EOPNOTSUPP;
+ } else if (out_dev != rep_priv->netdev) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "devices are not the same uplink, can't offload forwarding");
+ return -EOPNOTSUPP;
+ }
+ return 0;
+}
+
+static bool
+is_duplicated_output_device(struct net_device *dev,
+ struct net_device *out_dev,
+ int *ifindexes, int if_count,
+ struct netlink_ext_ack *extack)
+{
+ int i;
+
+ for (i = 0; i < if_count; i++) {
+ if (ifindexes[i] == out_dev->ifindex) {
+ NL_SET_ERR_MSG_MOD(extack, "can't duplicate output to same device");
+ netdev_err(dev, "can't duplicate output to same device: %s\n",
+ out_dev->name);
+ return true;
+ }
+ }
+
+ return false;
+}
+
+static struct net_device *
+get_fdb_out_dev(struct net_device *uplink_dev, struct net_device *out_dev)
+{
+ struct net_device *fdb_out_dev = out_dev;
+ struct net_device *uplink_upper;
+
+ rcu_read_lock();
+ uplink_upper = netdev_master_upper_dev_get_rcu(uplink_dev);
+ if (uplink_upper && netif_is_lag_master(uplink_upper) &&
+ uplink_upper == out_dev) {
+ fdb_out_dev = uplink_dev;
+ } else if (netif_is_lag_master(out_dev)) {
+ fdb_out_dev = bond_option_active_slave_get_rcu(netdev_priv(out_dev));
+ if (fdb_out_dev &&
+ (!mlx5e_eswitch_rep(fdb_out_dev) ||
+ !netdev_port_same_parent_id(fdb_out_dev, uplink_dev)))
+ fdb_out_dev = NULL;
+ }
+ rcu_read_unlock();
+ return fdb_out_dev;
+}
+
+static bool
+tc_act_can_offload_mirred(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ int act_index)
+{
+ struct netlink_ext_ack *extack = parse_state->extack;
+ struct mlx5e_tc_flow *flow = parse_state->flow;
+ struct mlx5e_tc_flow_parse_attr *parse_attr;
+ struct net_device *out_dev = act->dev;
+ struct mlx5e_priv *priv = flow->priv;
+ struct mlx5_esw_flow_attr *esw_attr;
+
+ parse_attr = flow->attr->parse_attr;
+ esw_attr = flow->attr->esw_attr;
+
+ if (!out_dev) {
+ /* out_dev is NULL when filters with
+ * non-existing mirred device are replayed to
+ * the driver.
+ */
+ return false;
+ }
+
+ if (parse_state->mpls_push && !netif_is_bareudp(out_dev)) {
+ NL_SET_ERR_MSG_MOD(extack, "mpls is supported only through a bareudp device");
+ return false;
+ }
+
+ if (mlx5e_is_ft_flow(flow) && out_dev == priv->netdev) {
+ /* Ignore forward to self rules generated
+ * by adding both mlx5 devs to the flow table
+ * block on a normal nft offload setup.
+ */
+ return false;
+ }
+
+ if (esw_attr->out_count >= MLX5_MAX_FLOW_FWD_VPORTS) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "can't support more output ports, can't offload forwarding");
+ netdev_warn(priv->netdev,
+ "can't support more than %d output ports, can't offload forwarding\n",
+ esw_attr->out_count);
+ return false;
+ }
+
+ if (parse_state->encap ||
+ netdev_port_same_parent_id(priv->netdev, out_dev) ||
+ netif_is_ovs_master(out_dev))
+ return true;
+
+ if (parse_attr->filter_dev != priv->netdev) {
+ /* All mlx5 devices are called to configure
+ * high level device filters. Therefore, the
+ * *attempt* to install a filter on invalid
+ * eswitch should not trigger an explicit error
+ */
+ return false;
+ }
+
+ NL_SET_ERR_MSG_MOD(extack, "devices are not on same switch HW, can't offload forwarding");
+
+ return false;
+}
+
+static int
+parse_mirred_encap(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ struct mlx5_flow_attr *attr)
+{
+ struct mlx5e_tc_flow_parse_attr *parse_attr = attr->parse_attr;
+ struct mlx5_esw_flow_attr *esw_attr = attr->esw_attr;
+ struct net_device *out_dev = act->dev;
+
+ parse_attr->mirred_ifindex[esw_attr->out_count] = out_dev->ifindex;
+ parse_attr->tun_info[esw_attr->out_count] =
+ mlx5e_dup_tun_info(parse_state->tun_info);
+
+ if (!parse_attr->tun_info[esw_attr->out_count])
+ return -ENOMEM;
+
+ parse_state->encap = false;
+ esw_attr->dests[esw_attr->out_count].flags |= MLX5_ESW_DEST_ENCAP;
+ esw_attr->out_count++;
+ /* attr->dests[].rep is resolved when we handle encap */
+
+ return 0;
+}
+
+static int
+parse_mirred(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ struct mlx5e_priv *priv,
+ struct mlx5_flow_attr *attr)
+{
+ struct mlx5e_tc_flow_parse_attr *parse_attr = attr->parse_attr;
+ struct mlx5_esw_flow_attr *esw_attr = attr->esw_attr;
+ struct netlink_ext_ack *extack = parse_state->extack;
+ struct mlx5e_rep_priv *rpriv = priv->ppriv;
+ struct net_device *out_dev = act->dev;
+ struct net_device *uplink_dev;
+ struct mlx5e_priv *out_priv;
+ struct mlx5_eswitch *esw;
+ int *ifindexes;
+ int if_count;
+ int err;
+
+ esw = priv->mdev->priv.eswitch;
+ uplink_dev = mlx5_eswitch_uplink_get_proto_dev(esw, REP_ETH);
+ ifindexes = parse_state->ifindexes;
+ if_count = parse_state->if_count;
+
+ if (is_duplicated_output_device(priv->netdev, out_dev, ifindexes, if_count, extack))
+ return -EOPNOTSUPP;
+
+ parse_state->ifindexes[if_count] = out_dev->ifindex;
+ parse_state->if_count++;
+
+ out_dev = get_fdb_out_dev(uplink_dev, out_dev);
+ if (!out_dev)
+ return -ENODEV;
+
+ if (is_vlan_dev(out_dev)) {
+ err = mlx5e_tc_act_vlan_add_push_action(priv, attr, &out_dev, extack);
+ if (err)
+ return err;
+ }
+
+ if (is_vlan_dev(parse_attr->filter_dev)) {
+ err = mlx5e_tc_act_vlan_add_pop_action(priv, attr, extack);
+ if (err)
+ return err;
+ }
+
+ if (netif_is_macvlan(out_dev))
+ out_dev = macvlan_dev_real_dev(out_dev);
+
+ err = verify_uplink_forwarding(priv, attr, out_dev, extack);
+ if (err)
+ return err;
+
+ if (!mlx5e_is_valid_eswitch_fwd_dev(priv, out_dev)) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "devices are not on same switch HW, can't offload forwarding");
+ return -EOPNOTSUPP;
+ }
+
+ if (same_vf_reps(priv, out_dev)) {
+ NL_SET_ERR_MSG_MOD(extack, "can't forward from a VF to itself");
+ return -EOPNOTSUPP;
+ }
+
+ out_priv = netdev_priv(out_dev);
+ rpriv = out_priv->ppriv;
+ esw_attr->dests[esw_attr->out_count].rep = rpriv->rep;
+ esw_attr->dests[esw_attr->out_count].mdev = out_priv->mdev;
+ esw_attr->out_count++;
+
+ return 0;
+}
+
+static int
+parse_mirred_ovs_master(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ struct mlx5e_priv *priv,
+ struct mlx5_flow_attr *attr)
+{
+ struct mlx5_esw_flow_attr *esw_attr = attr->esw_attr;
+ struct net_device *out_dev = act->dev;
+ int err;
+
+ err = mlx5e_set_fwd_to_int_port_actions(priv, attr, out_dev->ifindex,
+ MLX5E_TC_INT_PORT_EGRESS,
+ &attr->action, esw_attr->out_count);
+ if (err)
+ return err;
+
+ esw_attr->out_count++;
+ return 0;
+}
+
+static int
+tc_act_parse_mirred(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ struct mlx5e_priv *priv,
+ struct mlx5_flow_attr *attr)
+{
+ struct net_device *out_dev = act->dev;
+ int err = -EOPNOTSUPP;
+
+ if (parse_state->encap)
+ err = parse_mirred_encap(parse_state, act, attr);
+ else if (netdev_port_same_parent_id(priv->netdev, out_dev))
+ err = parse_mirred(parse_state, act, priv, attr);
+ else if (netif_is_ovs_master(out_dev))
+ err = parse_mirred_ovs_master(parse_state, act, priv, attr);
+
+ if (err)
+ return err;
+
+ attr->action |= MLX5_FLOW_CONTEXT_ACTION_FWD_DEST |
+ MLX5_FLOW_CONTEXT_ACTION_COUNT;
+
+ return 0;
+}
+
+struct mlx5e_tc_act mlx5e_tc_act_mirred = {
+ .can_offload = tc_act_can_offload_mirred,
+ .parse_action = tc_act_parse_mirred,
+};
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/mirred_nic.c b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/mirred_nic.c
new file mode 100644
index 0000000..2c74567
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/mirred_nic.c
@@ -0,0 +1,51 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+// Copyright (c) 2021, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#include "act.h"
+#include "en/tc_priv.h"
+
+static bool
+tc_act_can_offload_mirred_nic(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ int act_index)
+{
+ struct netlink_ext_ack *extack = parse_state->extack;
+ struct mlx5e_tc_flow *flow = parse_state->flow;
+ struct net_device *out_dev = act->dev;
+ struct mlx5e_priv *priv = flow->priv;
+
+ if (act->id != FLOW_ACTION_REDIRECT)
+ return false;
+
+ if (priv->netdev->netdev_ops != out_dev->netdev_ops ||
+ !mlx5e_same_hw_devs(priv, netdev_priv(out_dev))) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "devices are not on same switch HW, can't offload forwarding");
+ netdev_warn(priv->netdev,
+ "devices %s %s not on same switch HW, can't offload forwarding\n",
+ netdev_name(priv->netdev),
+ out_dev->name);
+ return false;
+ }
+
+ return true;
+}
+
+static int
+tc_act_parse_mirred_nic(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ struct mlx5e_priv *priv,
+ struct mlx5_flow_attr *attr)
+{
+ attr->parse_attr->mirred_ifindex[0] = act->dev->ifindex;
+ flow_flag_set(parse_state->flow, HAIRPIN);
+ attr->action |= MLX5_FLOW_CONTEXT_ACTION_FWD_DEST |
+ MLX5_FLOW_CONTEXT_ACTION_COUNT;
+
+ return 0;
+}
+
+struct mlx5e_tc_act mlx5e_tc_act_mirred_nic = {
+ .can_offload = tc_act_can_offload_mirred_nic,
+ .parse_action = tc_act_parse_mirred_nic,
+};
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/mpls.c b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/mpls.c
new file mode 100644
index 0000000..784fc4f
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/mpls.c
@@ -0,0 +1,86 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+// Copyright (c) 2021, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#include <net/bareudp.h>
+#include "act.h"
+#include "en/tc_priv.h"
+
+static bool
+tc_act_can_offload_mpls_push(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ int act_index)
+{
+ struct netlink_ext_ack *extack = parse_state->extack;
+ struct mlx5e_priv *priv = parse_state->flow->priv;
+
+ if (!MLX5_CAP_ESW_FLOWTABLE_FDB(priv->mdev, reformat_l2_to_l3_tunnel) ||
+ act->mpls_push.proto != htons(ETH_P_MPLS_UC)) {
+ NL_SET_ERR_MSG_MOD(extack, "mpls push is supported only for mpls_uc protocol");
+ return false;
+ }
+
+ return true;
+}
+
+static int
+tc_act_parse_mpls_push(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ struct mlx5e_priv *priv,
+ struct mlx5_flow_attr *attr)
+{
+ parse_state->mpls_push = true;
+
+ return 0;
+}
+
+static bool
+tc_act_can_offload_mpls_pop(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ int act_index)
+{
+ struct netlink_ext_ack *extack = parse_state->extack;
+ struct mlx5e_tc_flow *flow = parse_state->flow;
+ struct net_device *filter_dev;
+
+ filter_dev = flow->attr->parse_attr->filter_dev;
+
+ /* we only support mpls pop if it is the first action
+ * and the filter net device is bareudp. Subsequent
+ * actions can be pedit and the last can be mirred
+ * egress redirect.
+ */
+ if (act_index) {
+ NL_SET_ERR_MSG_MOD(extack, "mpls pop supported only as first action");
+ return false;
+ }
+
+ if (!netif_is_bareudp(filter_dev)) {
+ NL_SET_ERR_MSG_MOD(extack, "mpls pop supported only on bareudp devices");
+ return false;
+ }
+
+ return true;
+}
+
+static int
+tc_act_parse_mpls_pop(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ struct mlx5e_priv *priv,
+ struct mlx5_flow_attr *attr)
+{
+ attr->parse_attr->eth.h_proto = act->mpls_pop.proto;
+ attr->action |= MLX5_FLOW_CONTEXT_ACTION_PACKET_REFORMAT;
+ flow_flag_set(parse_state->flow, L3_TO_L2_DECAP);
+
+ return 0;
+}
+
+struct mlx5e_tc_act mlx5e_tc_act_mpls_push = {
+ .can_offload = tc_act_can_offload_mpls_push,
+ .parse_action = tc_act_parse_mpls_push,
+};
+
+struct mlx5e_tc_act mlx5e_tc_act_mpls_pop = {
+ .can_offload = tc_act_can_offload_mpls_pop,
+ .parse_action = tc_act_parse_mpls_pop,
+};
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/pedit.c b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/pedit.c
new file mode 100644
index 0000000..79addbb
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/pedit.c
@@ -0,0 +1,165 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+// Copyright (c) 2021, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#include <linux/if_vlan.h>
+#include "act.h"
+#include "pedit.h"
+#include "en/tc_priv.h"
+#include "en/mod_hdr.h"
+
+static int pedit_header_offsets[] = {
+ [FLOW_ACT_MANGLE_HDR_TYPE_ETH] = offsetof(struct pedit_headers, eth),
+ [FLOW_ACT_MANGLE_HDR_TYPE_IP4] = offsetof(struct pedit_headers, ip4),
+ [FLOW_ACT_MANGLE_HDR_TYPE_IP6] = offsetof(struct pedit_headers, ip6),
+ [FLOW_ACT_MANGLE_HDR_TYPE_TCP] = offsetof(struct pedit_headers, tcp),
+ [FLOW_ACT_MANGLE_HDR_TYPE_UDP] = offsetof(struct pedit_headers, udp),
+};
+
+#define pedit_header(_ph, _htype) ((void *)(_ph) + pedit_header_offsets[_htype])
+
+static int
+set_pedit_val(u8 hdr_type, u32 mask, u32 val, u32 offset,
+ struct pedit_headers_action *hdrs,
+ struct netlink_ext_ack *extack)
+{
+ u32 *curr_pmask, *curr_pval;
+
+ curr_pmask = (u32 *)(pedit_header(&hdrs->masks, hdr_type) + offset);
+ curr_pval = (u32 *)(pedit_header(&hdrs->vals, hdr_type) + offset);
+
+ if (*curr_pmask & mask) { /* disallow acting twice on the same location */
+ NL_SET_ERR_MSG_MOD(extack,
+ "curr_pmask and new mask same. Acting twice on same location");
+ goto out_err;
+ }
+
+ *curr_pmask |= mask;
+ *curr_pval |= (val & mask);
+
+ return 0;
+
+out_err:
+ return -EOPNOTSUPP;
+}
+
+static int
+parse_pedit_to_modify_hdr(struct mlx5e_priv *priv,
+ const struct flow_action_entry *act, int namespace,
+ struct mlx5e_tc_flow_parse_attr *parse_attr,
+ struct pedit_headers_action *hdrs,
+ struct netlink_ext_ack *extack)
+{
+ u8 cmd = (act->id == FLOW_ACTION_MANGLE) ? 0 : 1;
+ u8 htype = act->mangle.htype;
+ int err = -EOPNOTSUPP;
+ u32 mask, val, offset;
+
+ if (htype == FLOW_ACT_MANGLE_UNSPEC) {
+ NL_SET_ERR_MSG_MOD(extack, "legacy pedit isn't offloaded");
+ goto out_err;
+ }
+
+ if (!mlx5e_mod_hdr_max_actions(priv->mdev, namespace)) {
+ NL_SET_ERR_MSG_MOD(extack, "The pedit offload action is not supported");
+ goto out_err;
+ }
+
+ mask = act->mangle.mask;
+ val = act->mangle.val;
+ offset = act->mangle.offset;
+
+ err = set_pedit_val(htype, ~mask, val, offset, &hdrs[cmd], extack);
+ if (err)
+ goto out_err;
+
+ hdrs[cmd].pedits++;
+
+ return 0;
+out_err:
+ return err;
+}
+
+static int
+parse_pedit_to_reformat(const struct flow_action_entry *act,
+ struct mlx5e_tc_flow_parse_attr *parse_attr,
+ struct netlink_ext_ack *extack)
+{
+ u32 mask, val, offset;
+ u32 *p;
+
+ if (act->id != FLOW_ACTION_MANGLE) {
+ NL_SET_ERR_MSG_MOD(extack, "Unsupported action id");
+ return -EOPNOTSUPP;
+ }
+
+ if (act->mangle.htype != FLOW_ACT_MANGLE_HDR_TYPE_ETH) {
+ NL_SET_ERR_MSG_MOD(extack, "Only Ethernet modification is supported");
+ return -EOPNOTSUPP;
+ }
+
+ mask = ~act->mangle.mask;
+ val = act->mangle.val;
+ offset = act->mangle.offset;
+ p = (u32 *)&parse_attr->eth;
+ *(p + (offset >> 2)) |= (val & mask);
+
+ return 0;
+}
+
+int
+mlx5e_tc_act_pedit_parse_action(struct mlx5e_priv *priv,
+ const struct flow_action_entry *act, int namespace,
+ struct mlx5e_tc_flow_parse_attr *parse_attr,
+ struct pedit_headers_action *hdrs,
+ struct mlx5e_tc_flow *flow,
+ struct netlink_ext_ack *extack)
+{
+ if (flow && flow_flag_test(flow, L3_TO_L2_DECAP))
+ return parse_pedit_to_reformat(act, parse_attr, extack);
+
+ return parse_pedit_to_modify_hdr(priv, act, namespace, parse_attr, hdrs, extack);
+}
+
+static bool
+tc_act_can_offload_pedit(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ int act_index)
+{
+ return true;
+}
+
+static int
+tc_act_parse_pedit(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ struct mlx5e_priv *priv,
+ struct mlx5_flow_attr *attr)
+{
+ struct mlx5_esw_flow_attr *esw_attr = attr->esw_attr;
+ struct mlx5e_tc_flow *flow = parse_state->flow;
+ enum mlx5_flow_namespace_type ns_type;
+ int err;
+
+ ns_type = mlx5e_get_flow_namespace(flow);
+
+ err = mlx5e_tc_act_pedit_parse_action(flow->priv, act, ns_type,
+ attr->parse_attr, parse_state->hdrs,
+ flow, parse_state->extack);
+ if (err)
+ return err;
+
+ if (flow_flag_test(flow, L3_TO_L2_DECAP))
+ goto out;
+
+ attr->action |= MLX5_FLOW_CONTEXT_ACTION_MOD_HDR;
+
+ if (ns_type == MLX5_FLOW_NAMESPACE_FDB)
+ esw_attr->split_count = esw_attr->out_count;
+
+out:
+ return 0;
+}
+
+struct mlx5e_tc_act mlx5e_tc_act_pedit = {
+ .can_offload = tc_act_can_offload_pedit,
+ .parse_action = tc_act_parse_pedit,
+};
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/pedit.h b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/pedit.h
new file mode 100644
index 0000000..da8ab03
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/pedit.h
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+/* Copyright (c) 2021, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */
+
+#ifndef __MLX5_EN_TC_ACT_PEDIT_H__
+#define __MLX5_EN_TC_ACT_PEDIT_H__
+
+#include "en_tc.h"
+
+struct pedit_headers {
+ struct ethhdr eth;
+ struct vlan_hdr vlan;
+ struct iphdr ip4;
+ struct ipv6hdr ip6;
+ struct tcphdr tcp;
+ struct udphdr udp;
+};
+
+struct pedit_headers_action {
+ struct pedit_headers vals;
+ struct pedit_headers masks;
+ u32 pedits;
+};
+
+int
+mlx5e_tc_act_pedit_parse_action(struct mlx5e_priv *priv,
+ const struct flow_action_entry *act, int namespace,
+ struct mlx5e_tc_flow_parse_attr *parse_attr,
+ struct pedit_headers_action *hdrs,
+ struct mlx5e_tc_flow *flow,
+ struct netlink_ext_ack *extack);
+
+#endif /* __MLX5_EN_TC_ACT_PEDIT_H__ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/ptype.c b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/ptype.c
new file mode 100644
index 0000000..0819110
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/ptype.c
@@ -0,0 +1,35 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+// Copyright (c) 2021, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#include "act.h"
+#include "en/tc_priv.h"
+
+static bool
+tc_act_can_offload_ptype(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ int act_index)
+{
+ return true;
+}
+
+static int
+tc_act_parse_ptype(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ struct mlx5e_priv *priv,
+ struct mlx5_flow_attr *attr)
+{
+ struct netlink_ext_ack *extack = parse_state->extack;
+
+ if (act->ptype != PACKET_HOST) {
+ NL_SET_ERR_MSG_MOD(extack, "skbedit ptype is only supported with type host");
+ return -EOPNOTSUPP;
+ }
+
+ parse_state->ptype_host = true;
+ return 0;
+}
+
+struct mlx5e_tc_act mlx5e_tc_act_ptype = {
+ .can_offload = tc_act_can_offload_ptype,
+ .parse_action = tc_act_parse_ptype,
+};
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/redirect_ingress.c b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/redirect_ingress.c
new file mode 100644
index 0000000..1c32e24
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/redirect_ingress.c
@@ -0,0 +1,79 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+// Copyright (c) 2021, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#include "act.h"
+#include "en/tc_priv.h"
+
+static bool
+tc_act_can_offload_redirect_ingress(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ int act_index)
+{
+ struct netlink_ext_ack *extack = parse_state->extack;
+ struct mlx5e_tc_flow *flow = parse_state->flow;
+ struct mlx5e_tc_flow_parse_attr *parse_attr;
+ struct net_device *out_dev = act->dev;
+ struct mlx5_esw_flow_attr *esw_attr;
+
+ parse_attr = flow->attr->parse_attr;
+ esw_attr = flow->attr->esw_attr;
+
+ if (!out_dev)
+ return false;
+
+ if (!netif_is_ovs_master(out_dev)) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "redirect to ingress is supported only for OVS internal ports");
+ return false;
+ }
+
+ if (netif_is_ovs_master(parse_attr->filter_dev)) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "redirect to ingress is not supported from internal port");
+ return false;
+ }
+
+ if (!parse_state->ptype_host) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "redirect to int port ingress requires ptype=host action");
+ return false;
+ }
+
+ if (esw_attr->out_count) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "redirect to int port ingress is supported only as single destination");
+ return false;
+ }
+
+ return true;
+}
+
+static int
+tc_act_parse_redirect_ingress(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ struct mlx5e_priv *priv,
+ struct mlx5_flow_attr *attr)
+{
+ struct mlx5_esw_flow_attr *esw_attr = attr->esw_attr;
+ struct net_device *out_dev = act->dev;
+ int err;
+
+ attr->action |= MLX5_FLOW_CONTEXT_ACTION_FWD_DEST |
+ MLX5_FLOW_CONTEXT_ACTION_COUNT;
+
+ err = mlx5e_set_fwd_to_int_port_actions(priv, attr, out_dev->ifindex,
+ MLX5E_TC_INT_PORT_INGRESS,
+ &attr->action, esw_attr->out_count);
+ if (err)
+ return err;
+
+ esw_attr->out_count++;
+
+ return 0;
+}
+
+struct mlx5e_tc_act mlx5e_tc_act_redirect_ingress = {
+ .can_offload = tc_act_can_offload_redirect_ingress,
+ .parse_action = tc_act_parse_redirect_ingress,
+};
+
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/sample.c b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/sample.c
new file mode 100644
index 0000000..6699bdf
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/sample.c
@@ -0,0 +1,51 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+// Copyright (c) 2021, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#include <net/psample.h>
+#include "act.h"
+#include "en/tc_priv.h"
+
+static bool
+tc_act_can_offload_sample(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ int act_index)
+{
+ struct netlink_ext_ack *extack = parse_state->extack;
+
+ if (flow_flag_test(parse_state->flow, CT)) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "Sample action with connection tracking is not supported");
+ return false;
+ }
+
+ return true;
+}
+
+static int
+tc_act_parse_sample(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ struct mlx5e_priv *priv,
+ struct mlx5_flow_attr *attr)
+{
+ struct mlx5e_sample_attr *sample_attr;
+
+ sample_attr = kzalloc(sizeof(*attr->sample_attr), GFP_KERNEL);
+ if (!sample_attr)
+ return -ENOMEM;
+
+ sample_attr->rate = act->sample.rate;
+ sample_attr->group_num = act->sample.psample_group->group_num;
+
+ if (act->sample.truncate)
+ sample_attr->trunc_size = act->sample.trunc_size;
+
+ attr->sample_attr = sample_attr;
+ flow_flag_set(parse_state->flow, SAMPLE);
+
+ return 0;
+}
+
+struct mlx5e_tc_act mlx5e_tc_act_sample = {
+ .can_offload = tc_act_can_offload_sample,
+ .parse_action = tc_act_parse_sample,
+};
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/trap.c b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/trap.c
new file mode 100644
index 0000000..046b64c
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/trap.c
@@ -0,0 +1,38 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+// Copyright (c) 2021, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#include "act.h"
+#include "en/tc_priv.h"
+
+static bool
+tc_act_can_offload_trap(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ int act_index)
+{
+ struct netlink_ext_ack *extack = parse_state->extack;
+
+ if (parse_state->num_actions != 1) {
+ NL_SET_ERR_MSG_MOD(extack, "action trap is supported as a sole action only");
+ return false;
+ }
+
+ return true;
+}
+
+static int
+tc_act_parse_trap(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ struct mlx5e_priv *priv,
+ struct mlx5_flow_attr *attr)
+{
+ attr->action |= MLX5_FLOW_CONTEXT_ACTION_FWD_DEST |
+ MLX5_FLOW_CONTEXT_ACTION_COUNT;
+ attr->flags |= MLX5_ESW_ATTR_FLAG_SLOW_PATH;
+
+ return 0;
+}
+
+struct mlx5e_tc_act mlx5e_tc_act_trap = {
+ .can_offload = tc_act_can_offload_trap,
+ .parse_action = tc_act_parse_trap,
+};
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/tun.c b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/tun.c
new file mode 100644
index 0000000..6f4a2cf
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/tun.c
@@ -0,0 +1,61 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+// Copyright (c) 2021, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#include "act.h"
+#include "en/tc_tun_encap.h"
+#include "en/tc_priv.h"
+
+static bool
+tc_act_can_offload_tun_encap(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ int act_index)
+{
+ if (!act->tunnel) {
+ NL_SET_ERR_MSG_MOD(parse_state->extack,
+ "Zero tunnel attributes is not supported");
+ return false;
+ }
+
+ return true;
+}
+
+static int
+tc_act_parse_tun_encap(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ struct mlx5e_priv *priv,
+ struct mlx5_flow_attr *attr)
+{
+ parse_state->tun_info = act->tunnel;
+ parse_state->encap = true;
+
+ return 0;
+}
+
+static bool
+tc_act_can_offload_tun_decap(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ int act_index)
+{
+ return true;
+}
+
+static int
+tc_act_parse_tun_decap(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ struct mlx5e_priv *priv,
+ struct mlx5_flow_attr *attr)
+{
+ parse_state->decap = true;
+
+ return 0;
+}
+
+struct mlx5e_tc_act mlx5e_tc_act_tun_encap = {
+ .can_offload = tc_act_can_offload_tun_encap,
+ .parse_action = tc_act_parse_tun_encap,
+};
+
+struct mlx5e_tc_act mlx5e_tc_act_tun_decap = {
+ .can_offload = tc_act_can_offload_tun_decap,
+ .parse_action = tc_act_parse_tun_decap,
+};
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/vlan.c b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/vlan.c
new file mode 100644
index 0000000..70fc0c2
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/vlan.c
@@ -0,0 +1,218 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+// Copyright (c) 2021, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#include <linux/if_vlan.h>
+#include "act.h"
+#include "vlan.h"
+#include "en/tc_priv.h"
+
+static int
+add_vlan_prio_tag_rewrite_action(struct mlx5e_priv *priv,
+ struct mlx5e_tc_flow_parse_attr *parse_attr,
+ struct pedit_headers_action *hdrs,
+ u32 *action, struct netlink_ext_ack *extack)
+{
+ const struct flow_action_entry prio_tag_act = {
+ .vlan.vid = 0,
+ .vlan.prio =
+ MLX5_GET(fte_match_set_lyr_2_4,
+ mlx5e_get_match_headers_value(*action,
+ &parse_attr->spec),
+ first_prio) &
+ MLX5_GET(fte_match_set_lyr_2_4,
+ mlx5e_get_match_headers_criteria(*action,
+ &parse_attr->spec),
+ first_prio),
+ };
+
+ return mlx5e_tc_act_vlan_add_rewrite_action(priv, MLX5_FLOW_NAMESPACE_FDB,
+ &prio_tag_act, parse_attr, hdrs, action,
+ extack);
+}
+
+static int
+parse_tc_vlan_action(struct mlx5e_priv *priv,
+ const struct flow_action_entry *act,
+ struct mlx5_esw_flow_attr *attr,
+ u32 *action,
+ struct netlink_ext_ack *extack)
+{
+ u8 vlan_idx = attr->total_vlan;
+
+ if (vlan_idx >= MLX5_FS_VLAN_DEPTH) {
+ NL_SET_ERR_MSG_MOD(extack, "Total vlans used is greater than supported");
+ return -EOPNOTSUPP;
+ }
+
+ switch (act->id) {
+ case FLOW_ACTION_VLAN_POP:
+ if (vlan_idx) {
+ if (!mlx5_eswitch_vlan_actions_supported(priv->mdev,
+ MLX5_FS_VLAN_DEPTH)) {
+ NL_SET_ERR_MSG_MOD(extack, "vlan pop action is not supported");
+ return -EOPNOTSUPP;
+ }
+
+ *action |= MLX5_FLOW_CONTEXT_ACTION_VLAN_POP_2;
+ } else {
+ *action |= MLX5_FLOW_CONTEXT_ACTION_VLAN_POP;
+ }
+ break;
+ case FLOW_ACTION_VLAN_PUSH:
+ attr->vlan_vid[vlan_idx] = act->vlan.vid;
+ attr->vlan_prio[vlan_idx] = act->vlan.prio;
+ attr->vlan_proto[vlan_idx] = act->vlan.proto;
+ if (!attr->vlan_proto[vlan_idx])
+ attr->vlan_proto[vlan_idx] = htons(ETH_P_8021Q);
+
+ if (vlan_idx) {
+ if (!mlx5_eswitch_vlan_actions_supported(priv->mdev,
+ MLX5_FS_VLAN_DEPTH)) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "vlan push action is not supported for vlan depth > 1");
+ return -EOPNOTSUPP;
+ }
+
+ *action |= MLX5_FLOW_CONTEXT_ACTION_VLAN_PUSH_2;
+ } else {
+ if (!mlx5_eswitch_vlan_actions_supported(priv->mdev, 1) &&
+ (act->vlan.proto != htons(ETH_P_8021Q) ||
+ act->vlan.prio)) {
+ NL_SET_ERR_MSG_MOD(extack, "vlan push action is not supported");
+ return -EOPNOTSUPP;
+ }
+
+ *action |= MLX5_FLOW_CONTEXT_ACTION_VLAN_PUSH;
+ }
+ break;
+ default:
+ NL_SET_ERR_MSG_MOD(extack, "Unexpected action id for VLAN");
+ return -EINVAL;
+ }
+
+ attr->total_vlan = vlan_idx + 1;
+
+ return 0;
+}
+
+int
+mlx5e_tc_act_vlan_add_push_action(struct mlx5e_priv *priv,
+ struct mlx5_flow_attr *attr,
+ struct net_device **out_dev,
+ struct netlink_ext_ack *extack)
+{
+ struct net_device *vlan_dev = *out_dev;
+ struct flow_action_entry vlan_act = {
+ .id = FLOW_ACTION_VLAN_PUSH,
+ .vlan.vid = vlan_dev_vlan_id(vlan_dev),
+ .vlan.proto = vlan_dev_vlan_proto(vlan_dev),
+ .vlan.prio = 0,
+ };
+ int err;
+
+ err = parse_tc_vlan_action(priv, &vlan_act, attr->esw_attr, &attr->action, extack);
+ if (err)
+ return err;
+
+ rcu_read_lock();
+ *out_dev = dev_get_by_index_rcu(dev_net(vlan_dev), dev_get_iflink(vlan_dev));
+ rcu_read_unlock();
+ if (!*out_dev)
+ return -ENODEV;
+
+ if (is_vlan_dev(*out_dev))
+ err = mlx5e_tc_act_vlan_add_push_action(priv, attr, out_dev, extack);
+
+ return err;
+}
+
+int
+mlx5e_tc_act_vlan_add_pop_action(struct mlx5e_priv *priv,
+ struct mlx5_flow_attr *attr,
+ struct netlink_ext_ack *extack)
+{
+ struct flow_action_entry vlan_act = {
+ .id = FLOW_ACTION_VLAN_POP,
+ };
+ int nest_level, err = 0;
+
+ nest_level = attr->parse_attr->filter_dev->lower_level -
+ priv->netdev->lower_level;
+ while (nest_level--) {
+ err = parse_tc_vlan_action(priv, &vlan_act, attr->esw_attr, &attr->action,
+ extack);
+ if (err)
+ return err;
+ }
+
+ return err;
+}
+
+static bool
+tc_act_can_offload_vlan(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ int act_index)
+{
+ return true;
+}
+
+static int
+tc_act_parse_vlan(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ struct mlx5e_priv *priv,
+ struct mlx5_flow_attr *attr)
+{
+ struct mlx5_esw_flow_attr *esw_attr = attr->esw_attr;
+ int err;
+
+ if (act->id == FLOW_ACTION_VLAN_PUSH &&
+ (attr->action & MLX5_FLOW_CONTEXT_ACTION_VLAN_POP)) {
+ /* Replace vlan pop+push with vlan modify */
+ attr->action &= ~MLX5_FLOW_CONTEXT_ACTION_VLAN_POP;
+ err = mlx5e_tc_act_vlan_add_rewrite_action(priv, MLX5_FLOW_NAMESPACE_FDB, act,
+ attr->parse_attr, parse_state->hdrs,
+ &attr->action, parse_state->extack);
+ } else {
+ err = parse_tc_vlan_action(priv, act, esw_attr, &attr->action,
+ parse_state->extack);
+ }
+
+ if (err)
+ return err;
+
+ esw_attr->split_count = esw_attr->out_count;
+
+ return 0;
+}
+
+static int
+tc_act_post_parse_vlan(struct mlx5e_tc_act_parse_state *parse_state,
+ struct mlx5e_priv *priv,
+ struct mlx5_flow_attr *attr)
+{
+ struct mlx5e_tc_flow_parse_attr *parse_attr = attr->parse_attr;
+ struct pedit_headers_action *hdrs = parse_state->hdrs;
+ struct netlink_ext_ack *extack = parse_state->extack;
+ struct mlx5_eswitch *esw = priv->mdev->priv.eswitch;
+ int err;
+
+ if (MLX5_CAP_GEN(esw->dev, prio_tag_required) &&
+ attr->action & MLX5_FLOW_CONTEXT_ACTION_VLAN_POP) {
+ /* For prio tag mode, replace vlan pop with rewrite vlan prio
+ * tag rewrite.
+ */
+ attr->action &= ~MLX5_FLOW_CONTEXT_ACTION_VLAN_POP;
+ err = add_vlan_prio_tag_rewrite_action(priv, parse_attr, hdrs,
+ &attr->action, extack);
+ if (err)
+ return err;
+ }
+
+ return 0;
+}
+
+struct mlx5e_tc_act mlx5e_tc_act_vlan = {
+ .can_offload = tc_act_can_offload_vlan,
+ .parse_action = tc_act_parse_vlan,
+ .post_parse = tc_act_post_parse_vlan,
+};
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/vlan.h b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/vlan.h
new file mode 100644
index 0000000..3d62f13
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/vlan.h
@@ -0,0 +1,30 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+/* Copyright (c) 2021, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */
+
+#ifndef __MLX5_EN_TC_ACT_VLAN_H__
+#define __MLX5_EN_TC_ACT_VLAN_H__
+
+#include <net/flow_offload.h>
+#include "en/tc_priv.h"
+
+struct pedit_headers_action;
+
+int
+mlx5e_tc_act_vlan_add_push_action(struct mlx5e_priv *priv,
+ struct mlx5_flow_attr *attr,
+ struct net_device **out_dev,
+ struct netlink_ext_ack *extack);
+
+int
+mlx5e_tc_act_vlan_add_pop_action(struct mlx5e_priv *priv,
+ struct mlx5_flow_attr *attr,
+ struct netlink_ext_ack *extack);
+
+int
+mlx5e_tc_act_vlan_add_rewrite_action(struct mlx5e_priv *priv, int namespace,
+ const struct flow_action_entry *act,
+ struct mlx5e_tc_flow_parse_attr *parse_attr,
+ struct pedit_headers_action *hdrs,
+ u32 *action, struct netlink_ext_ack *extack);
+
+#endif /* __MLX5_EN_TC_ACT_VLAN_H__ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/vlan_mangle.c b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/vlan_mangle.c
new file mode 100644
index 0000000..63e36e7
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/vlan_mangle.c
@@ -0,0 +1,87 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+// Copyright (c) 2021, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#include <linux/if_vlan.h>
+#include "act.h"
+#include "vlan.h"
+#include "en/tc_priv.h"
+
+struct pedit_headers_action;
+
+int
+mlx5e_tc_act_vlan_add_rewrite_action(struct mlx5e_priv *priv, int namespace,
+ const struct flow_action_entry *act,
+ struct mlx5e_tc_flow_parse_attr *parse_attr,
+ struct pedit_headers_action *hdrs,
+ u32 *action, struct netlink_ext_ack *extack)
+{
+ u16 mask16 = VLAN_VID_MASK;
+ u16 val16 = act->vlan.vid & VLAN_VID_MASK;
+ const struct flow_action_entry pedit_act = {
+ .id = FLOW_ACTION_MANGLE,
+ .mangle.htype = FLOW_ACT_MANGLE_HDR_TYPE_ETH,
+ .mangle.offset = offsetof(struct vlan_ethhdr, h_vlan_TCI),
+ .mangle.mask = ~(u32)be16_to_cpu(*(__be16 *)&mask16),
+ .mangle.val = (u32)be16_to_cpu(*(__be16 *)&val16),
+ };
+ u8 match_prio_mask, match_prio_val;
+ void *headers_c, *headers_v;
+ int err;
+
+ headers_c = mlx5e_get_match_headers_criteria(*action, &parse_attr->spec);
+ headers_v = mlx5e_get_match_headers_value(*action, &parse_attr->spec);
+
+ if (!(MLX5_GET(fte_match_set_lyr_2_4, headers_c, cvlan_tag) &&
+ MLX5_GET(fte_match_set_lyr_2_4, headers_v, cvlan_tag))) {
+ NL_SET_ERR_MSG_MOD(extack, "VLAN rewrite action must have VLAN protocol match");
+ return -EOPNOTSUPP;
+ }
+
+ match_prio_mask = MLX5_GET(fte_match_set_lyr_2_4, headers_c, first_prio);
+ match_prio_val = MLX5_GET(fte_match_set_lyr_2_4, headers_v, first_prio);
+ if (act->vlan.prio != (match_prio_val & match_prio_mask)) {
+ NL_SET_ERR_MSG_MOD(extack, "Changing VLAN prio is not supported");
+ return -EOPNOTSUPP;
+ }
+
+ err = mlx5e_tc_act_pedit_parse_action(priv, &pedit_act, namespace, parse_attr, hdrs,
+ NULL, extack);
+ *action |= MLX5_FLOW_CONTEXT_ACTION_MOD_HDR;
+
+ return err;
+}
+
+static bool
+tc_act_can_offload_vlan_mangle(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ int act_index)
+{
+ return true;
+}
+
+static int
+tc_act_parse_vlan_mangle(struct mlx5e_tc_act_parse_state *parse_state,
+ const struct flow_action_entry *act,
+ struct mlx5e_priv *priv,
+ struct mlx5_flow_attr *attr)
+{
+ enum mlx5_flow_namespace_type ns_type;
+ int err;
+
+ ns_type = mlx5e_get_flow_namespace(parse_state->flow);
+ err = mlx5e_tc_act_vlan_add_rewrite_action(priv, ns_type, act,
+ attr->parse_attr, parse_state->hdrs,
+ &attr->action, parse_state->extack);
+ if (err)
+ return err;
+
+ if (ns_type == MLX5_FLOW_NAMESPACE_FDB)
+ attr->esw_attr->split_count = attr->esw_attr->out_count;
+
+ return 0;
+}
+
+struct mlx5e_tc_act mlx5e_tc_act_vlan_mangle = {
+ .can_offload = tc_act_can_offload_vlan_mangle,
+ .parse_action = tc_act_parse_vlan_mangle,
+};
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/tc/sample.c b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/sample.c
index df6888c..ff4b4f8 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/tc/sample.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/sample.c
@@ -5,6 +5,7 @@
#include <net/psample.h>
#include "en/mapping.h"
#include "en/tc/post_act.h"
+#include "en/mod_hdr.h"
#include "sample.h"
#include "eswitch.h"
#include "en_tc.h"
@@ -255,12 +256,12 @@ sample_modify_hdr_get(struct mlx5_core_dev *mdev, u32 obj_id,
goto err_modify_hdr;
}
- dealloc_mod_hdr_actions(&mod_acts);
+ mlx5e_mod_hdr_dealloc(&mod_acts);
return modify_hdr;
err_modify_hdr:
err_post_act:
- dealloc_mod_hdr_actions(&mod_acts);
+ mlx5e_mod_hdr_dealloc(&mod_acts);
err_set_regc0:
return ERR_PTR(err);
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/tc_ct.c b/drivers/net/ethernet/mellanox/mlx5/core/en/tc_ct.c
index 2445e2a..4a0d38d 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/tc_ct.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/tc_ct.c
@@ -14,6 +14,7 @@
#include <linux/workqueue.h>
#include <linux/refcount.h>
#include <linux/xarray.h>
+#include <linux/if_macvlan.h>
#include "lib/fs_chains.h"
#include "en/tc_ct.h"
@@ -36,6 +37,12 @@
#define MLX5_CT_LABELS_BITS (mlx5e_tc_attr_to_reg_mappings[LABELS_TO_REG].mlen)
#define MLX5_CT_LABELS_MASK GENMASK(MLX5_CT_LABELS_BITS - 1, 0)
+/* Statically allocate modify actions for
+ * ipv6 and port nat (5) + tuple fields (4) + nic mode zone restore (1) = 10.
+ * This will be increased dynamically if needed (for the ipv6 snat + dnat).
+ */
+#define MLX5_CT_MIN_MOD_ACTS 10
+
#define ct_dbg(fmt, args...)\
netdev_dbg(ct_priv->netdev, "ct_debug: " fmt "\n", ##args)
@@ -320,7 +327,33 @@ mlx5_tc_ct_rule_to_tuple_nat(struct mlx5_ct_tuple *tuple,
}
static int
-mlx5_tc_ct_set_tuple_match(struct mlx5e_priv *priv, struct mlx5_flow_spec *spec,
+mlx5_tc_ct_get_flow_source_match(struct mlx5_tc_ct_priv *ct_priv,
+ struct net_device *ndev)
+{
+ struct mlx5e_priv *other_priv = netdev_priv(ndev);
+ struct mlx5_core_dev *mdev = ct_priv->dev;
+ bool vf_rep, uplink_rep;
+
+ vf_rep = mlx5e_eswitch_vf_rep(ndev) && mlx5_same_hw_devs(mdev, other_priv->mdev);
+ uplink_rep = mlx5e_eswitch_uplink_rep(ndev) && mlx5_same_hw_devs(mdev, other_priv->mdev);
+
+ if (vf_rep)
+ return MLX5_FLOW_CONTEXT_FLOW_SOURCE_LOCAL_VPORT;
+ if (uplink_rep)
+ return MLX5_FLOW_CONTEXT_FLOW_SOURCE_UPLINK;
+ if (is_vlan_dev(ndev))
+ return mlx5_tc_ct_get_flow_source_match(ct_priv, vlan_dev_real_dev(ndev));
+ if (netif_is_macvlan(ndev))
+ return mlx5_tc_ct_get_flow_source_match(ct_priv, macvlan_dev_real_dev(ndev));
+ if (mlx5e_get_tc_tun(ndev) || netif_is_lag_master(ndev))
+ return MLX5_FLOW_CONTEXT_FLOW_SOURCE_UPLINK;
+
+ return MLX5_FLOW_CONTEXT_FLOW_SOURCE_ANY_VPORT;
+}
+
+static int
+mlx5_tc_ct_set_tuple_match(struct mlx5_tc_ct_priv *ct_priv,
+ struct mlx5_flow_spec *spec,
struct flow_rule *rule)
{
void *headers_c = MLX5_ADDR_OF(fte_match_param, spec->match_criteria,
@@ -335,8 +368,7 @@ mlx5_tc_ct_set_tuple_match(struct mlx5e_priv *priv, struct mlx5_flow_spec *spec,
flow_rule_match_basic(rule, &match);
- mlx5e_tc_set_ethertype(priv->mdev, &match, true, headers_c,
- headers_v);
+ mlx5e_tc_set_ethertype(ct_priv->dev, &match, true, headers_c, headers_v);
MLX5_SET(fte_match_set_lyr_2_4, headers_c, ip_protocol,
match.mask->ip_proto);
MLX5_SET(fte_match_set_lyr_2_4, headers_v, ip_protocol,
@@ -432,6 +464,23 @@ mlx5_tc_ct_set_tuple_match(struct mlx5e_priv *priv, struct mlx5_flow_spec *spec,
ntohs(match.key->flags));
}
+ if (flow_rule_match_key(rule, FLOW_DISSECTOR_KEY_META)) {
+ struct flow_match_meta match;
+
+ flow_rule_match_meta(rule, &match);
+
+ if (match.key->ingress_ifindex & match.mask->ingress_ifindex) {
+ struct net_device *dev;
+
+ dev = dev_get_by_index(&init_net, match.key->ingress_ifindex);
+ if (dev && MLX5_CAP_ESW_FLOWTABLE(ct_priv->dev, flow_source))
+ spec->flow_context.flow_source =
+ mlx5_tc_ct_get_flow_source_match(ct_priv, dev);
+
+ dev_put(dev);
+ }
+ }
+
return 0;
}
@@ -609,22 +658,15 @@ mlx5_tc_ct_entry_create_nat(struct mlx5_tc_ct_priv *ct_priv,
struct flow_action *flow_action = &flow_rule->action;
struct mlx5_core_dev *mdev = ct_priv->dev;
struct flow_action_entry *act;
- size_t action_size;
char *modact;
int err, i;
- action_size = MLX5_UN_SZ_BYTES(set_add_copy_action_in_auto);
-
flow_action_for_each(i, act, flow_action) {
switch (act->id) {
case FLOW_ACTION_MANGLE: {
- err = alloc_mod_hdr_actions(mdev, ct_priv->ns_type,
- mod_acts);
- if (err)
- return err;
-
- modact = mod_acts->actions +
- mod_acts->num_actions * action_size;
+ modact = mlx5e_mod_hdr_alloc(mdev, ct_priv->ns_type, mod_acts);
+ if (IS_ERR(modact))
+ return PTR_ERR(modact);
err = mlx5_tc_ct_parse_mangle_to_mod_act(act, modact);
if (err)
@@ -652,7 +694,8 @@ mlx5_tc_ct_entry_create_mod_hdr(struct mlx5_tc_ct_priv *ct_priv,
struct mlx5e_mod_hdr_handle **mh,
u8 zone_restore_id, bool nat)
{
- struct mlx5e_tc_mod_hdr_acts mod_acts = {};
+ DECLARE_MOD_HDR_ACTS_ACTIONS(actions_arr, MLX5_CT_MIN_MOD_ACTS);
+ DECLARE_MOD_HDR_ACTS(mod_acts, actions_arr);
struct flow_action_entry *meta;
u16 ct_state = 0;
int err;
@@ -706,11 +749,11 @@ mlx5_tc_ct_entry_create_mod_hdr(struct mlx5_tc_ct_priv *ct_priv,
attr->modify_hdr = mlx5e_mod_hdr_get(*mh);
}
- dealloc_mod_hdr_actions(&mod_acts);
+ mlx5e_mod_hdr_dealloc(&mod_acts);
return 0;
err_mapping:
- dealloc_mod_hdr_actions(&mod_acts);
+ mlx5e_mod_hdr_dealloc(&mod_acts);
mlx5_put_label_mapping(ct_priv, attr->ct_attr.ct_labels_id);
return err;
}
@@ -770,7 +813,7 @@ mlx5_tc_ct_entry_add_rule(struct mlx5_tc_ct_priv *ct_priv,
if (ct_priv->ns_type == MLX5_FLOW_NAMESPACE_FDB)
attr->esw_attr->in_mdev = priv->mdev;
- mlx5_tc_ct_set_tuple_match(netdev_priv(ct_priv->netdev), spec, flow_rule);
+ mlx5_tc_ct_set_tuple_match(ct_priv, spec, flow_rule);
mlx5e_tc_match_to_reg_match(spec, ZONE_TO_REG, entry->tuple.zone, MLX5_CT_ZONE_MASK);
zone_rule->rule = mlx5_tc_rule_insert(priv, spec, attr);
@@ -907,12 +950,9 @@ mlx5_tc_ct_shared_counter_get(struct mlx5_tc_ct_priv *ct_priv,
struct mlx5_ct_tuple rev_tuple = entry->tuple;
struct mlx5_ct_counter *shared_counter;
struct mlx5_ct_entry *rev_entry;
- __be16 tmp_port;
/* get the reversed tuple */
- tmp_port = rev_tuple.port.src;
- rev_tuple.port.src = rev_tuple.port.dst;
- rev_tuple.port.dst = tmp_port;
+ swap(rev_tuple.port.src, rev_tuple.port.dst);
if (rev_tuple.addr_type == FLOW_DISSECTOR_KEY_IPV4_ADDRS) {
__be32 tmp_addr = rev_tuple.ip.src_v4;
@@ -1460,7 +1500,7 @@ static int tc_ct_pre_ct_add_rules(struct mlx5_ct_ft *ct_ft,
}
pre_ct->miss_rule = rule;
- dealloc_mod_hdr_actions(&pre_mod_acts);
+ mlx5e_mod_hdr_dealloc(&pre_mod_acts);
kvfree(spec);
return 0;
@@ -1469,7 +1509,7 @@ static int tc_ct_pre_ct_add_rules(struct mlx5_ct_ft *ct_ft,
err_flow_rule:
mlx5_modify_header_dealloc(dev, pre_ct->modify_hdr);
err_mapping:
- dealloc_mod_hdr_actions(&pre_mod_acts);
+ mlx5e_mod_hdr_dealloc(&pre_mod_acts);
kvfree(spec);
return err;
}
@@ -1865,14 +1905,14 @@ __mlx5_tc_ct_flow_offload(struct mlx5_tc_ct_priv *ct_priv,
}
attr->ct_attr.ct_flow = ct_flow;
- dealloc_mod_hdr_actions(&pre_mod_acts);
+ mlx5e_mod_hdr_dealloc(&pre_mod_acts);
return ct_flow->pre_ct_rule;
err_insert_orig:
mlx5_modify_header_dealloc(priv->mdev, pre_ct_attr->modify_hdr);
err_mapping:
- dealloc_mod_hdr_actions(&pre_mod_acts);
+ mlx5e_mod_hdr_dealloc(&pre_mod_acts);
mlx5_chains_put_chain_mapping(ct_priv->chains, ct_flow->chain_mapping);
err_get_chain:
kfree(ct_flow->pre_ct_attr);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/tc_priv.h b/drivers/net/ethernet/mellanox/mlx5/core/en/tc_priv.h
index b689701..f832c26 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/tc_priv.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/tc_priv.h
@@ -5,11 +5,14 @@
#define __MLX5_EN_TC_PRIV_H__
#include "en_tc.h"
+#include "en/tc/act/act.h"
#define MLX5E_TC_FLOW_BASE (MLX5E_TC_FLAG_LAST_EXPORTED_BIT + 1)
#define MLX5E_TC_MAX_SPLITS 1
+#define mlx5e_nic_chains(priv) ((priv)->fs.tc.chains)
+
enum {
MLX5E_TC_FLOW_FLAG_INGRESS = MLX5E_TC_FLAG_INGRESS_BIT,
MLX5E_TC_FLOW_FLAG_EGRESS = MLX5E_TC_FLAG_EGRESS_BIT,
@@ -37,6 +40,7 @@ struct mlx5e_tc_flow_parse_attr {
struct mlx5e_tc_mod_hdr_acts mod_hdr_acts;
int mirred_ifindex[MLX5_MAX_FLOW_FWD_VPORTS];
struct ethhdr eth;
+ struct mlx5e_tc_act_parse_state parse_state;
};
/* Helper struct for accessing a struct containing list_head array.
@@ -115,7 +119,11 @@ mlx5e_tc_offload_fdb_rules(struct mlx5_eswitch *esw,
struct mlx5_flow_spec *spec,
struct mlx5_flow_attr *attr);
+bool mlx5e_is_eswitch_flow(struct mlx5e_tc_flow *flow);
+bool mlx5e_is_ft_flow(struct mlx5e_tc_flow *flow);
bool mlx5e_is_offloaded_flow(struct mlx5e_tc_flow *flow);
+int mlx5e_get_flow_namespace(struct mlx5e_tc_flow *flow);
+bool mlx5e_same_hw_devs(struct mlx5e_priv *priv, struct mlx5e_priv *peer_priv);
static inline void __flow_flag_set(struct mlx5e_tc_flow *flow, unsigned long flag)
{
@@ -176,4 +184,8 @@ struct mlx5_fc *mlx5e_tc_get_counter(struct mlx5e_tc_flow *flow);
struct mlx5e_tc_int_port_priv *
mlx5e_get_int_port_priv(struct mlx5e_priv *priv);
+
+void *mlx5e_get_match_headers_value(u32 flags, struct mlx5_flow_spec *spec);
+void *mlx5e_get_match_headers_criteria(u32 flags, struct mlx5_flow_spec *spec);
+
#endif /* __MLX5_EN_TC_PRIV_H__ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/tc_tun.c b/drivers/net/ethernet/mellanox/mlx5/core/en/tc_tun.c
index a5e4509..3381524 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/tc_tun.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/tc_tun.c
@@ -103,7 +103,7 @@ static int get_route_and_out_devs(struct mlx5e_priv *priv,
}
static int mlx5e_route_lookup_ipv4_get(struct mlx5e_priv *priv,
- struct net_device *mirred_dev,
+ struct net_device *dev,
struct mlx5e_tc_tun_route_attr *attr)
{
struct net_device *route_dev;
@@ -122,13 +122,13 @@ static int mlx5e_route_lookup_ipv4_get(struct mlx5e_priv *priv,
uplink_dev = mlx5_eswitch_uplink_get_proto_dev(esw, REP_ETH);
attr->fl.fl4.flowi4_oif = uplink_dev->ifindex;
} else {
- struct mlx5e_tc_tunnel *tunnel = mlx5e_get_tc_tun(mirred_dev);
+ struct mlx5e_tc_tunnel *tunnel = mlx5e_get_tc_tun(dev);
if (tunnel && tunnel->get_remote_ifindex)
- attr->fl.fl4.flowi4_oif = tunnel->get_remote_ifindex(mirred_dev);
+ attr->fl.fl4.flowi4_oif = tunnel->get_remote_ifindex(dev);
}
- rt = ip_route_output_key(dev_net(mirred_dev), &attr->fl.fl4);
+ rt = ip_route_output_key(dev_net(dev), &attr->fl.fl4);
if (IS_ERR(rt))
return PTR_ERR(rt);
@@ -440,10 +440,10 @@ int mlx5e_tc_tun_update_header_ipv4(struct mlx5e_priv *priv,
#if IS_ENABLED(CONFIG_INET) && IS_ENABLED(CONFIG_IPV6)
static int mlx5e_route_lookup_ipv6_get(struct mlx5e_priv *priv,
- struct net_device *mirred_dev,
+ struct net_device *dev,
struct mlx5e_tc_tun_route_attr *attr)
{
- struct mlx5e_tc_tunnel *tunnel = mlx5e_get_tc_tun(mirred_dev);
+ struct mlx5e_tc_tunnel *tunnel = mlx5e_get_tc_tun(dev);
struct net_device *route_dev;
struct net_device *out_dev;
struct dst_entry *dst;
@@ -451,8 +451,8 @@ static int mlx5e_route_lookup_ipv6_get(struct mlx5e_priv *priv,
int ret;
if (tunnel && tunnel->get_remote_ifindex)
- attr->fl.fl6.flowi6_oif = tunnel->get_remote_ifindex(mirred_dev);
- dst = ipv6_stub->ipv6_dst_lookup_flow(dev_net(mirred_dev), NULL, &attr->fl.fl6,
+ attr->fl.fl6.flowi6_oif = tunnel->get_remote_ifindex(dev);
+ dst = ipv6_stub->ipv6_dst_lookup_flow(dev_net(dev), NULL, &attr->fl.fl6,
NULL);
if (IS_ERR(dst))
return PTR_ERR(dst);
@@ -708,7 +708,8 @@ int mlx5e_tc_tun_update_header_ipv6(struct mlx5e_priv *priv,
int mlx5e_tc_tun_route_lookup(struct mlx5e_priv *priv,
struct mlx5_flow_spec *spec,
- struct mlx5_flow_attr *flow_attr)
+ struct mlx5_flow_attr *flow_attr,
+ struct net_device *filter_dev)
{
struct mlx5_esw_flow_attr *esw_attr = flow_attr->esw_attr;
struct mlx5e_tc_int_port *int_port;
@@ -720,14 +721,14 @@ int mlx5e_tc_tun_route_lookup(struct mlx5e_priv *priv,
/* Addresses are swapped for decap */
attr.fl.fl4.saddr = esw_attr->rx_tun_attr->dst_ip.v4;
attr.fl.fl4.daddr = esw_attr->rx_tun_attr->src_ip.v4;
- err = mlx5e_route_lookup_ipv4_get(priv, priv->netdev, &attr);
+ err = mlx5e_route_lookup_ipv4_get(priv, filter_dev, &attr);
}
#if IS_ENABLED(CONFIG_INET) && IS_ENABLED(CONFIG_IPV6)
else if (flow_attr->tun_ip_version == 6) {
/* Addresses are swapped for decap */
attr.fl.fl6.saddr = esw_attr->rx_tun_attr->dst_ip.v6;
attr.fl.fl6.daddr = esw_attr->rx_tun_attr->src_ip.v6;
- err = mlx5e_route_lookup_ipv6_get(priv, priv->netdev, &attr);
+ err = mlx5e_route_lookup_ipv6_get(priv, filter_dev, &attr);
}
#endif
else
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/tc_tun.h b/drivers/net/ethernet/mellanox/mlx5/core/en/tc_tun.h
index aa092ea..b38f693 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/tc_tun.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/tc_tun.h
@@ -94,7 +94,8 @@ mlx5e_tc_tun_update_header_ipv6(struct mlx5e_priv *priv,
#endif
int mlx5e_tc_tun_route_lookup(struct mlx5e_priv *priv,
struct mlx5_flow_spec *spec,
- struct mlx5_flow_attr *attr);
+ struct mlx5_flow_attr *attr,
+ struct net_device *filter_dev);
bool mlx5e_tc_tun_device_to_offload(struct mlx5e_priv *priv,
struct net_device *netdev);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/tc_tun_encap.c b/drivers/net/ethernet/mellanox/mlx5/core/en/tc_tun_encap.c
index 042b1ab..9918ed8 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/tc_tun_encap.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/tc_tun_encap.c
@@ -1159,7 +1159,7 @@ int mlx5e_attach_decap_route(struct mlx5e_priv *priv,
tbl_time_before = mlx5e_route_tbl_get_last_update(priv);
tbl_time_after = tbl_time_before;
- err = mlx5e_tc_tun_route_lookup(priv, &parse_attr->spec, attr);
+ err = mlx5e_tc_tun_route_lookup(priv, &parse_attr->spec, attr, parse_attr->filter_dev);
if (err || !esw_attr->rx_tun_attr->decap_vport)
goto out;
@@ -1480,7 +1480,7 @@ static void mlx5e_reoffload_decap(struct mlx5e_priv *priv,
parse_attr = attr->parse_attr;
spec = &parse_attr->spec;
- err = mlx5e_tc_tun_route_lookup(priv, spec, attr);
+ err = mlx5e_tc_tun_route_lookup(priv, spec, attr, parse_attr->filter_dev);
if (err) {
mlx5_core_warn(priv->mdev, "Failed to lookup route for flow, %d\n",
err);
@@ -1579,6 +1579,8 @@ mlx5e_init_fib_work_ipv4(struct mlx5e_priv *priv,
struct net_device *fib_dev;
fen_info = container_of(info, struct fib_entry_notifier_info, info);
+ if (fen_info->fi->nh)
+ return NULL;
fib_dev = fib_info_nh(fen_info->fi, 0)->fib_nh_dev;
if (!fib_dev || fib_dev->netdev_ops != &mlx5e_netdev_ops ||
fen_info->dst_len != 32)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c b/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c
index 2f0df5c..338d65e 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c
@@ -151,7 +151,7 @@ bool mlx5e_xdp_handle(struct mlx5e_rq *rq, struct mlx5e_dma_info *di,
rq->stats->xdp_redirect++;
return true;
default:
- bpf_warn_invalid_xdp_action(act);
+ bpf_warn_invalid_xdp_action(rq->netdev, prog, act);
fallthrough;
case XDP_ABORTED:
xdp_abort:
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/pool.c b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/pool.c
index 7b562d2..279cd8f 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/pool.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/pool.c
@@ -11,13 +11,13 @@ static int mlx5e_xsk_map_pool(struct mlx5e_priv *priv,
{
struct device *dev = mlx5_core_dma_dev(priv->mdev);
- return xsk_pool_dma_map(pool, dev, 0);
+ return xsk_pool_dma_map(pool, dev, DMA_ATTR_SKIP_CPU_SYNC);
}
static void mlx5e_xsk_unmap_pool(struct mlx5e_priv *priv,
struct xsk_buff_pool *pool)
{
- return xsk_pool_dma_unmap(pool, 0);
+ return xsk_pool_dma_unmap(pool, DMA_ATTR_SKIP_CPU_SYNC);
}
static int mlx5e_xsk_get_pools(struct mlx5e_xsk *xsk)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/setup.c b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/setup.c
index 538bc24..25eac9e 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/setup.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/setup.c
@@ -4,6 +4,7 @@
#include "setup.h"
#include "en/params.h"
#include "en/txrx.h"
+#include "en/health.h"
/* It matches XDP_UMEM_MIN_CHUNK_SIZE, but as this constant is private and may
* change unexpectedly, and mlx5e has a minimum valid stride size for striding
@@ -67,7 +68,7 @@ static int mlx5e_init_xsk_rq(struct mlx5e_channel *c,
rq->hw_mtu = MLX5E_SW2HW_MTU(params, params->sw_mtu);
rq->xdpsq = &c->rq_xdpsq;
rq->xsk_pool = pool;
- rq->stats = &c->priv->channel_stats[c->ix].xskrq;
+ rq->stats = &c->priv->channel_stats[c->ix]->xskrq;
rq->ptp_cyc2time = mlx5_rq_ts_translator(mdev);
rq_xdp_ix = c->ix + params->num_channels * MLX5E_RQ_GROUP_XSK;
err = mlx5e_rq_set_handlers(rq, params, xsk);
@@ -170,7 +171,13 @@ void mlx5e_close_xsk(struct mlx5e_channel *c)
void mlx5e_activate_xsk(struct mlx5e_channel *c)
{
+ /* ICOSQ recovery deactivates RQs. Suspend the recovery to avoid
+ * activating XSKRQ in the middle of recovery.
+ */
+ mlx5e_reporter_icosq_suspend_recovery(c);
set_bit(MLX5E_RQ_STATE_ENABLED, &c->xskrq.state);
+ mlx5e_reporter_icosq_resume_recovery(c);
+
/* TX queue is created active. */
spin_lock_bh(&c->async_icosq_lock);
@@ -180,6 +187,13 @@ void mlx5e_activate_xsk(struct mlx5e_channel *c)
void mlx5e_deactivate_xsk(struct mlx5e_channel *c)
{
- mlx5e_deactivate_rq(&c->xskrq);
+ /* ICOSQ recovery may reactivate XSKRQ if clear_bit is called in the
+ * middle of recovery. Suspend the recovery to avoid it.
+ */
+ mlx5e_reporter_icosq_suspend_recovery(c);
+ clear_bit(MLX5E_RQ_STATE_ENABLED, &c->xskrq.state);
+ mlx5e_reporter_icosq_resume_recovery(c);
+ synchronize_net(); /* Sync with NAPI to prevent mlx5e_post_rx_wqes. */
+
/* TX queue is disabled on close. */
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_rx.c
index 1571181..96064a2 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_rx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_rx.c
@@ -611,7 +611,7 @@ int mlx5e_ktls_add_rx(struct net_device *netdev, struct sock *sk,
priv_rx->rxq = rxq;
priv_rx->sk = sk;
- priv_rx->rq_stats = &priv->channel_stats[rxq].rq;
+ priv_rx->rq_stats = &priv->channel_stats[rxq]->rq;
priv_rx->sw_stats = &priv->tls->sw_stats;
mlx5e_set_ktls_rx_priv_ctx(tls_ctx, priv_rx);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_arfs.c b/drivers/net/ethernet/mellanox/mlx5/core/en_arfs.c
index fe5d82f..49cca6b 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_arfs.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_arfs.c
@@ -556,7 +556,7 @@ static struct mlx5_flow_handle *arfs_add_rule(struct mlx5e_priv *priv,
rule = mlx5_add_flow_rules(ft, spec, &flow_act, &dest, 1);
if (IS_ERR(rule)) {
err = PTR_ERR(rule);
- priv->channel_stats[arfs_rule->rxq].rq.arfs_err++;
+ priv->channel_stats[arfs_rule->rxq]->rq.arfs_err++;
mlx5e_dbg(HW, priv,
"%s: add rule(filter id=%d, rq idx=%d, ip proto=0x%x) failed,err=%d\n",
__func__, arfs_rule->filter_id, arfs_rule->rxq,
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
index c2ea5fa..57d755d 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
@@ -314,7 +314,9 @@ void mlx5e_ethtool_get_ringparam(struct mlx5e_priv *priv,
}
static void mlx5e_get_ringparam(struct net_device *dev,
- struct ethtool_ringparam *param)
+ struct ethtool_ringparam *param,
+ struct kernel_ethtool_ringparam *kernel_param,
+ struct netlink_ext_ack *extack)
{
struct mlx5e_priv *priv = netdev_priv(dev);
@@ -380,7 +382,9 @@ int mlx5e_ethtool_set_ringparam(struct mlx5e_priv *priv,
}
static int mlx5e_set_ringparam(struct net_device *dev,
- struct ethtool_ringparam *param)
+ struct ethtool_ringparam *param,
+ struct kernel_ethtool_ringparam *kernel_param,
+ struct netlink_ext_ack *extack)
{
struct mlx5e_priv *priv = netdev_priv(dev);
@@ -511,7 +515,8 @@ static int mlx5e_set_channels(struct net_device *dev,
}
int mlx5e_ethtool_get_coalesce(struct mlx5e_priv *priv,
- struct ethtool_coalesce *coal)
+ struct ethtool_coalesce *coal,
+ struct kernel_ethtool_coalesce *kernel_coal)
{
struct dim_cq_moder *rx_moder, *tx_moder;
@@ -528,6 +533,11 @@ int mlx5e_ethtool_get_coalesce(struct mlx5e_priv *priv,
coal->tx_max_coalesced_frames = tx_moder->pkts;
coal->use_adaptive_tx_coalesce = priv->channels.params.tx_dim_enabled;
+ kernel_coal->use_cqe_mode_rx =
+ MLX5E_GET_PFLAG(&priv->channels.params, MLX5E_PFLAG_RX_CQE_BASED_MODER);
+ kernel_coal->use_cqe_mode_tx =
+ MLX5E_GET_PFLAG(&priv->channels.params, MLX5E_PFLAG_TX_CQE_BASED_MODER);
+
return 0;
}
@@ -538,7 +548,7 @@ static int mlx5e_get_coalesce(struct net_device *netdev,
{
struct mlx5e_priv *priv = netdev_priv(netdev);
- return mlx5e_ethtool_get_coalesce(priv, coal);
+ return mlx5e_ethtool_get_coalesce(priv, coal, kernel_coal);
}
#define MLX5E_MAX_COAL_TIME MLX5_MAX_CQ_PERIOD
@@ -578,14 +588,26 @@ mlx5e_set_priv_channels_rx_coalesce(struct mlx5e_priv *priv, struct ethtool_coal
}
}
+/* convert a boolean value of cq_mode to mlx5 period mode
+ * true : MLX5_CQ_PERIOD_MODE_START_FROM_CQE
+ * false : MLX5_CQ_PERIOD_MODE_START_FROM_EQE
+ */
+static int cqe_mode_to_period_mode(bool val)
+{
+ return val ? MLX5_CQ_PERIOD_MODE_START_FROM_CQE : MLX5_CQ_PERIOD_MODE_START_FROM_EQE;
+}
+
int mlx5e_ethtool_set_coalesce(struct mlx5e_priv *priv,
- struct ethtool_coalesce *coal)
+ struct ethtool_coalesce *coal,
+ struct kernel_ethtool_coalesce *kernel_coal,
+ struct netlink_ext_ack *extack)
{
struct dim_cq_moder *rx_moder, *tx_moder;
struct mlx5_core_dev *mdev = priv->mdev;
struct mlx5e_params new_params;
bool reset_rx, reset_tx;
bool reset = true;
+ u8 cq_period_mode;
int err = 0;
if (!MLX5_CAP_GEN(mdev, cq_moderation))
@@ -605,6 +627,12 @@ int mlx5e_ethtool_set_coalesce(struct mlx5e_priv *priv,
return -ERANGE;
}
+ if ((kernel_coal->use_cqe_mode_rx || kernel_coal->use_cqe_mode_tx) &&
+ !MLX5_CAP_GEN(priv->mdev, cq_period_start_from_cqe)) {
+ NL_SET_ERR_MSG_MOD(extack, "cqe_mode_rx/tx is not supported on this device");
+ return -EOPNOTSUPP;
+ }
+
mutex_lock(&priv->state_lock);
new_params = priv->channels.params;
@@ -621,6 +649,18 @@ int mlx5e_ethtool_set_coalesce(struct mlx5e_priv *priv,
reset_rx = !!coal->use_adaptive_rx_coalesce != priv->channels.params.rx_dim_enabled;
reset_tx = !!coal->use_adaptive_tx_coalesce != priv->channels.params.tx_dim_enabled;
+ cq_period_mode = cqe_mode_to_period_mode(kernel_coal->use_cqe_mode_rx);
+ if (cq_period_mode != rx_moder->cq_period_mode) {
+ mlx5e_set_rx_cq_mode_params(&new_params, cq_period_mode);
+ reset_rx = true;
+ }
+
+ cq_period_mode = cqe_mode_to_period_mode(kernel_coal->use_cqe_mode_tx);
+ if (cq_period_mode != tx_moder->cq_period_mode) {
+ mlx5e_set_tx_cq_mode_params(&new_params, cq_period_mode);
+ reset_tx = true;
+ }
+
if (reset_rx) {
u8 mode = MLX5E_GET_PFLAG(&new_params,
MLX5E_PFLAG_RX_CQE_BASED_MODER);
@@ -656,9 +696,9 @@ static int mlx5e_set_coalesce(struct net_device *netdev,
struct kernel_ethtool_coalesce *kernel_coal,
struct netlink_ext_ack *extack)
{
- struct mlx5e_priv *priv = netdev_priv(netdev);
+ struct mlx5e_priv *priv = netdev_priv(netdev);
- return mlx5e_ethtool_set_coalesce(priv, coal);
+ return mlx5e_ethtool_set_coalesce(priv, coal, kernel_coal, extack);
}
static void ptys2ethtool_supported_link(struct mlx5_core_dev *mdev,
@@ -1843,24 +1883,19 @@ static int set_pflag_cqe_based_moder(struct net_device *netdev, bool enable,
bool is_rx_cq)
{
struct mlx5e_priv *priv = netdev_priv(netdev);
- struct mlx5_core_dev *mdev = priv->mdev;
- struct mlx5e_params new_params;
- bool mode_changed;
u8 cq_period_mode, current_cq_period_mode;
+ struct mlx5e_params new_params;
- cq_period_mode = enable ?
- MLX5_CQ_PERIOD_MODE_START_FROM_CQE :
- MLX5_CQ_PERIOD_MODE_START_FROM_EQE;
+ if (enable && !MLX5_CAP_GEN(priv->mdev, cq_period_start_from_cqe))
+ return -EOPNOTSUPP;
+
+ cq_period_mode = cqe_mode_to_period_mode(enable);
+
current_cq_period_mode = is_rx_cq ?
priv->channels.params.rx_cq_moderation.cq_period_mode :
priv->channels.params.tx_cq_moderation.cq_period_mode;
- mode_changed = cq_period_mode != current_cq_period_mode;
- if (cq_period_mode == MLX5_CQ_PERIOD_MODE_START_FROM_CQE &&
- !MLX5_CAP_GEN(mdev, cq_period_start_from_cqe))
- return -EOPNOTSUPP;
-
- if (!mode_changed)
+ if (cq_period_mode == current_cq_period_mode)
return 0;
new_params = priv->channels.params;
@@ -1894,7 +1929,7 @@ int mlx5e_modify_rx_cqe_compression_locked(struct mlx5e_priv *priv, bool new_val
if (curr_val == new_val)
return 0;
- if (new_val && !priv->profile->rx_ptp_support && rx_filter) {
+ if (new_val && !mlx5e_profile_feature_cap(priv->profile, PTP_RX) && rx_filter) {
netdev_err(priv->netdev,
"Profile doesn't support enabling of CQE compression while hardware time-stamping is enabled.\n");
return -EINVAL;
@@ -2358,7 +2393,8 @@ static void mlx5e_get_rmon_stats(struct net_device *netdev,
const struct ethtool_ops mlx5e_ethtool_ops = {
.supported_coalesce_params = ETHTOOL_COALESCE_USECS |
ETHTOOL_COALESCE_MAX_FRAMES |
- ETHTOOL_COALESCE_USE_ADAPTIVE,
+ ETHTOOL_COALESCE_USE_ADAPTIVE |
+ ETHTOOL_COALESCE_USE_CQE,
.get_drvinfo = mlx5e_get_drvinfo,
.get_link = ethtool_op_get_link,
.get_link_ext_state = mlx5e_get_link_ext_state,
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
index 6557159..ac69e0a 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
@@ -37,6 +37,7 @@
#include <net/geneve.h>
#include <linux/bpf.h>
#include <linux/if_bridge.h>
+#include <linux/filter.h>
#include <net/page_pool.h>
#include <net/xdp_sock_drv.h>
#include "eswitch.h"
@@ -479,7 +480,7 @@ static int mlx5e_init_rxq_rq(struct mlx5e_channel *c, struct mlx5e_params *param
rq->mdev = mdev;
rq->hw_mtu = MLX5E_SW2HW_MTU(params, params->sw_mtu);
rq->xdpsq = &c->rq_xdpsq;
- rq->stats = &c->priv->channel_stats[c->ix].rq;
+ rq->stats = &c->priv->channel_stats[c->ix]->rq;
rq->ptp_cyc2time = mlx5_rq_ts_translator(mdev);
err = mlx5e_rq_set_handlers(rq, params, NULL);
if (err)
@@ -1087,8 +1088,6 @@ void mlx5e_deactivate_rq(struct mlx5e_rq *rq)
void mlx5e_close_rq(struct mlx5e_rq *rq)
{
cancel_work_sync(&rq->dim.work);
- if (rq->icosq)
- cancel_work_sync(&rq->icosq->recover_work);
cancel_work_sync(&rq->recover_work);
mlx5e_destroy_rq(rq);
mlx5e_free_rx_descs(rq);
@@ -1161,10 +1160,10 @@ static int mlx5e_alloc_xdpsq(struct mlx5e_channel *c,
sq->xsk_pool = xsk_pool;
sq->stats = sq->xsk_pool ?
- &c->priv->channel_stats[c->ix].xsksq :
+ &c->priv->channel_stats[c->ix]->xsksq :
is_redirect ?
- &c->priv->channel_stats[c->ix].xdpsq :
- &c->priv->channel_stats[c->ix].rq_xdpsq;
+ &c->priv->channel_stats[c->ix]->xdpsq :
+ &c->priv->channel_stats[c->ix]->rq_xdpsq;
param->wq.db_numa_node = cpu_to_node(c->cpu);
err = mlx5_wq_cyc_create(mdev, ¶m->wq, sqc_wq, wq, &sq->wq_ctrl);
@@ -1216,9 +1215,20 @@ static void mlx5e_icosq_err_cqe_work(struct work_struct *recover_work)
mlx5e_reporter_icosq_cqe_err(sq);
}
+static void mlx5e_async_icosq_err_cqe_work(struct work_struct *recover_work)
+{
+ struct mlx5e_icosq *sq = container_of(recover_work, struct mlx5e_icosq,
+ recover_work);
+
+ /* Not implemented yet. */
+
+ netdev_warn(sq->channel->netdev, "async_icosq recovery is not implemented\n");
+}
+
static int mlx5e_alloc_icosq(struct mlx5e_channel *c,
struct mlx5e_sq_param *param,
- struct mlx5e_icosq *sq)
+ struct mlx5e_icosq *sq,
+ work_func_t recover_work_func)
{
void *sqc_wq = MLX5_ADDR_OF(sqc, param->sqc, wq);
struct mlx5_core_dev *mdev = c->mdev;
@@ -1239,7 +1249,7 @@ static int mlx5e_alloc_icosq(struct mlx5e_channel *c,
if (err)
goto err_sq_wq_destroy;
- INIT_WORK(&sq->recover_work, mlx5e_icosq_err_cqe_work);
+ INIT_WORK(&sq->recover_work, recover_work_func);
return 0;
@@ -1575,13 +1585,14 @@ void mlx5e_tx_err_cqe_work(struct work_struct *recover_work)
mlx5e_reporter_tx_err_cqe(sq);
}
-int mlx5e_open_icosq(struct mlx5e_channel *c, struct mlx5e_params *params,
- struct mlx5e_sq_param *param, struct mlx5e_icosq *sq)
+static int mlx5e_open_icosq(struct mlx5e_channel *c, struct mlx5e_params *params,
+ struct mlx5e_sq_param *param, struct mlx5e_icosq *sq,
+ work_func_t recover_work_func)
{
struct mlx5e_create_sq_param csp = {};
int err;
- err = mlx5e_alloc_icosq(c, param, sq);
+ err = mlx5e_alloc_icosq(c, param, sq, recover_work_func);
if (err)
return err;
@@ -1620,7 +1631,7 @@ void mlx5e_deactivate_icosq(struct mlx5e_icosq *icosq)
synchronize_net(); /* Sync with NAPI. */
}
-void mlx5e_close_icosq(struct mlx5e_icosq *sq)
+static void mlx5e_close_icosq(struct mlx5e_icosq *sq)
{
struct mlx5e_channel *c = sq->channel;
@@ -1928,7 +1939,7 @@ static int mlx5e_open_sqs(struct mlx5e_channel *c,
err = mlx5e_open_txqsq(c, c->priv->tisn[c->lag_port][tc], txq_ix,
params, &cparam->txq_sq, &c->sq[tc], tc,
qos_queue_group_id,
- &c->priv->channel_stats[c->ix].sq[tc]);
+ &c->priv->channel_stats[c->ix]->sq[tc]);
if (err)
goto err_close_sqs;
}
@@ -2084,11 +2095,15 @@ static int mlx5e_open_queues(struct mlx5e_channel *c,
spin_lock_init(&c->async_icosq_lock);
- err = mlx5e_open_icosq(c, params, &cparam->async_icosq, &c->async_icosq);
+ err = mlx5e_open_icosq(c, params, &cparam->async_icosq, &c->async_icosq,
+ mlx5e_async_icosq_err_cqe_work);
if (err)
goto err_close_xdpsq_cq;
- err = mlx5e_open_icosq(c, params, &cparam->icosq, &c->icosq);
+ mutex_init(&c->icosq_recovery_lock);
+
+ err = mlx5e_open_icosq(c, params, &cparam->icosq, &c->icosq,
+ mlx5e_icosq_err_cqe_work);
if (err)
goto err_close_async_icosq;
@@ -2156,9 +2171,12 @@ static void mlx5e_close_queues(struct mlx5e_channel *c)
mlx5e_close_xdpsq(&c->xdpsq);
if (c->xdp)
mlx5e_close_xdpsq(&c->rq_xdpsq);
+ /* The same ICOSQ is used for UMRs for both RQ and XSKRQ. */
+ cancel_work_sync(&c->icosq.recover_work);
mlx5e_close_rq(&c->rq);
mlx5e_close_sqs(c);
mlx5e_close_icosq(&c->icosq);
+ mutex_destroy(&c->icosq_recovery_lock);
mlx5e_close_icosq(&c->async_icosq);
if (c->xdp)
mlx5e_close_cq(&c->rq_xdpsq.cq);
@@ -2176,6 +2194,30 @@ static u8 mlx5e_enumerate_lag_port(struct mlx5_core_dev *mdev, int ix)
return (ix + port_aff_bias) % mlx5e_get_num_lag_ports(mdev);
}
+static int mlx5e_channel_stats_alloc(struct mlx5e_priv *priv, int ix, int cpu)
+{
+ if (ix > priv->stats_nch) {
+ netdev_warn(priv->netdev, "Unexpected channel stats index %d > %d\n", ix,
+ priv->stats_nch);
+ return -EINVAL;
+ }
+
+ if (priv->channel_stats[ix])
+ return 0;
+
+ /* Asymmetric dynamic memory allocation.
+ * Freed in mlx5e_priv_arrays_free, not on channel closure.
+ */
+ mlx5e_dbg(DRV, priv, "Creating channel stats %d\n", ix);
+ priv->channel_stats[ix] = kvzalloc_node(sizeof(**priv->channel_stats),
+ GFP_KERNEL, cpu_to_node(cpu));
+ if (!priv->channel_stats[ix])
+ return -ENOMEM;
+ priv->stats_nch++;
+
+ return 0;
+}
+
static int mlx5e_open_channel(struct mlx5e_priv *priv, int ix,
struct mlx5e_params *params,
struct mlx5e_channel_param *cparam,
@@ -2193,6 +2235,10 @@ static int mlx5e_open_channel(struct mlx5e_priv *priv, int ix,
if (err)
return err;
+ err = mlx5e_channel_stats_alloc(priv, ix, cpu);
+ if (err)
+ return err;
+
c = kvzalloc_node(sizeof(*c), GFP_KERNEL, cpu_to_node(cpu));
if (!c)
return -ENOMEM;
@@ -2207,7 +2253,7 @@ static int mlx5e_open_channel(struct mlx5e_priv *priv, int ix,
c->mkey_be = cpu_to_be32(priv->mdev->mlx5e_res.hw_objs.mkey);
c->num_tc = mlx5e_get_dcb_num_tc(params);
c->xdp = !!params->xdp_prog;
- c->stats = &priv->channel_stats[ix].ch;
+ c->stats = &priv->channel_stats[ix]->ch;
c->aff_mask = irq_get_effective_affinity_mask(irq);
c->lag_port = mlx5e_enumerate_lag_port(priv->mdev, ix);
@@ -2598,7 +2644,7 @@ static void mlx5e_set_default_xps_cpumasks(struct mlx5e_priv *priv,
}
}
-int mlx5e_num_channels_changed(struct mlx5e_priv *priv)
+static int mlx5e_num_channels_changed(struct mlx5e_priv *priv)
{
u16 count = priv->channels.params.num_channels;
int err;
@@ -3371,7 +3417,7 @@ void mlx5e_fold_sw_stats64(struct mlx5e_priv *priv, struct rtnl_link_stats64 *s)
int i;
for (i = 0; i < priv->stats_nch; i++) {
- struct mlx5e_channel_stats *channel_stats = &priv->channel_stats[i];
+ struct mlx5e_channel_stats *channel_stats = priv->channel_stats[i];
struct mlx5e_rq_stats *xskrq_stats = &channel_stats->xskrq;
struct mlx5e_rq_stats *rq_stats = &channel_stats->rq;
int j;
@@ -3559,11 +3605,6 @@ static int set_feature_hw_gro(struct net_device *netdev, bool enable)
new_params = priv->channels.params;
if (enable) {
- if (MLX5E_GET_PFLAG(&new_params, MLX5E_PFLAG_RX_CQE_COMPRESS)) {
- netdev_warn(netdev, "Can't set HW-GRO when CQE compress is active\n");
- err = -EINVAL;
- goto out;
- }
new_params.packet_merge.type = MLX5E_PACKET_MERGE_SHAMPO;
new_params.packet_merge.shampo.match_criteria_type =
MLX5_RQC_SHAMPO_MATCH_CRITERIA_TYPE_EXTENDED;
@@ -3724,12 +3765,11 @@ static int set_feature_arfs(struct net_device *netdev, bool enable)
static int mlx5e_handle_feature(struct net_device *netdev,
netdev_features_t *features,
- netdev_features_t wanted_features,
netdev_features_t feature,
mlx5e_feature_handler feature_handler)
{
- netdev_features_t changes = wanted_features ^ netdev->features;
- bool enable = !!(wanted_features & feature);
+ netdev_features_t changes = *features ^ netdev->features;
+ bool enable = !!(*features & feature);
int err;
if (!(changes & feature))
@@ -3737,22 +3777,22 @@ static int mlx5e_handle_feature(struct net_device *netdev,
err = feature_handler(netdev, enable);
if (err) {
+ MLX5E_SET_FEATURE(features, feature, !enable);
netdev_err(netdev, "%s feature %pNF failed, err %d\n",
enable ? "Enable" : "Disable", &feature, err);
return err;
}
- MLX5E_SET_FEATURE(features, feature, enable);
return 0;
}
int mlx5e_set_features(struct net_device *netdev, netdev_features_t features)
{
- netdev_features_t oper_features = netdev->features;
+ netdev_features_t oper_features = features;
int err = 0;
#define MLX5E_HANDLE_FEATURE(feature, handler) \
- mlx5e_handle_feature(netdev, &oper_features, features, feature, handler)
+ mlx5e_handle_feature(netdev, &oper_features, feature, handler)
err |= MLX5E_HANDLE_FEATURE(NETIF_F_LRO, set_feature_lro);
err |= MLX5E_HANDLE_FEATURE(NETIF_F_GRO_HW, set_feature_hw_gro);
@@ -3826,6 +3866,11 @@ static netdev_features_t mlx5e_fix_features(struct net_device *netdev,
features &= ~NETIF_F_RXHASH;
if (netdev->features & NETIF_F_RXHASH)
netdev_warn(netdev, "Disabling rxhash, not supported when CQE compress is active\n");
+
+ if (features & NETIF_F_GRO_HW) {
+ netdev_warn(netdev, "Disabling HW-GRO, not supported when CQE compress is active\n");
+ features &= ~NETIF_F_GRO_HW;
+ }
}
if (mlx5e_is_uplink_rep(priv))
@@ -4038,7 +4083,7 @@ int mlx5e_hwstamp_set(struct mlx5e_priv *priv, struct ifreq *ifr)
goto err_unlock;
}
- if (!priv->profile->rx_ptp_support)
+ if (!mlx5e_profile_feature_cap(priv->profile, PTP_RX))
err = mlx5e_hwstamp_config_no_ptp_rx(priv,
config.rx_filter != HWTSTAMP_FILTER_NONE);
else
@@ -4773,15 +4818,22 @@ static void mlx5e_build_nic_netdev(struct net_device *netdev)
}
if (mlx5_vxlan_allowed(mdev->vxlan) || mlx5_geneve_tx_allowed(mdev)) {
- netdev->hw_features |= NETIF_F_GSO_UDP_TUNNEL;
- netdev->hw_enc_features |= NETIF_F_GSO_UDP_TUNNEL;
- netdev->vlan_features |= NETIF_F_GSO_UDP_TUNNEL;
+ netdev->hw_features |= NETIF_F_GSO_UDP_TUNNEL |
+ NETIF_F_GSO_UDP_TUNNEL_CSUM;
+ netdev->hw_enc_features |= NETIF_F_GSO_UDP_TUNNEL |
+ NETIF_F_GSO_UDP_TUNNEL_CSUM;
+ netdev->gso_partial_features = NETIF_F_GSO_UDP_TUNNEL_CSUM;
+ netdev->vlan_features |= NETIF_F_GSO_UDP_TUNNEL |
+ NETIF_F_GSO_UDP_TUNNEL_CSUM;
}
if (mlx5e_tunnel_proto_supported_tx(mdev, IPPROTO_GRE)) {
- netdev->hw_features |= NETIF_F_GSO_GRE;
- netdev->hw_enc_features |= NETIF_F_GSO_GRE;
- netdev->gso_partial_features |= NETIF_F_GSO_GRE;
+ netdev->hw_features |= NETIF_F_GSO_GRE |
+ NETIF_F_GSO_GRE_CSUM;
+ netdev->hw_enc_features |= NETIF_F_GSO_GRE |
+ NETIF_F_GSO_GRE_CSUM;
+ netdev->gso_partial_features |= NETIF_F_GSO_GRE |
+ NETIF_F_GSO_GRE_CSUM;
}
if (mlx5e_tunnel_proto_supported_tx(mdev, IPPROTO_IPIP)) {
@@ -5093,9 +5145,23 @@ static const struct mlx5e_profile mlx5e_nic_profile = {
.rq_groups = MLX5E_NUM_RQ_GROUPS(XSK),
.stats_grps = mlx5e_nic_stats_grps,
.stats_grps_num = mlx5e_nic_stats_grps_num,
- .rx_ptp_support = true,
+ .features = BIT(MLX5E_PROFILE_FEATURE_PTP_RX) |
+ BIT(MLX5E_PROFILE_FEATURE_PTP_TX) |
+ BIT(MLX5E_PROFILE_FEATURE_QOS_HTB),
};
+static int mlx5e_profile_max_num_channels(struct mlx5_core_dev *mdev,
+ const struct mlx5e_profile *profile)
+{
+ int nch;
+
+ nch = mlx5e_get_max_num_channels(mdev);
+
+ if (profile->max_nch_limit)
+ nch = min_t(int, nch, profile->max_nch_limit(mdev));
+ return nch;
+}
+
static unsigned int
mlx5e_calc_max_nch(struct mlx5_core_dev *mdev, struct net_device *netdev,
const struct mlx5e_profile *profile)
@@ -5104,7 +5170,7 @@ mlx5e_calc_max_nch(struct mlx5_core_dev *mdev, struct net_device *netdev,
unsigned int max_nch, tmp;
/* core resources */
- max_nch = mlx5e_get_max_num_channels(mdev);
+ max_nch = mlx5e_profile_max_num_channels(mdev, profile);
/* netdev rx queues */
tmp = netdev->num_rx_queues / max_t(u8, profile->rq_groups, 1);
@@ -5128,12 +5194,17 @@ int mlx5e_priv_init(struct mlx5e_priv *priv,
struct net_device *netdev,
struct mlx5_core_dev *mdev)
{
+ int nch, num_txqs, node, i;
+
+ num_txqs = netdev->num_tx_queues;
+ nch = mlx5e_calc_max_nch(mdev, netdev, profile);
+ node = dev_to_node(mlx5_core_dma_dev(mdev));
+
/* priv init */
priv->mdev = mdev;
priv->netdev = netdev;
priv->msglevel = MLX5E_MSG_LEVEL;
- priv->max_nch = mlx5e_calc_max_nch(mdev, netdev, profile);
- priv->stats_nch = priv->max_nch;
+ priv->max_nch = nch;
priv->max_opened_tc = 1;
if (!alloc_cpumask_var(&priv->scratchpad.cpumask, GFP_KERNEL))
@@ -5150,11 +5221,46 @@ int mlx5e_priv_init(struct mlx5e_priv *priv,
if (!priv->wq)
goto err_free_cpumask;
+ priv->txq2sq = kcalloc_node(num_txqs, sizeof(*priv->txq2sq), GFP_KERNEL, node);
+ if (!priv->txq2sq)
+ goto err_destroy_workqueue;
+
+ priv->tx_rates = kcalloc_node(num_txqs, sizeof(*priv->tx_rates), GFP_KERNEL, node);
+ if (!priv->tx_rates)
+ goto err_free_txq2sq;
+
+ priv->channel_tc2realtxq =
+ kcalloc_node(nch, sizeof(*priv->channel_tc2realtxq), GFP_KERNEL, node);
+ if (!priv->channel_tc2realtxq)
+ goto err_free_tx_rates;
+
+ for (i = 0; i < nch; i++) {
+ priv->channel_tc2realtxq[i] =
+ kcalloc_node(profile->max_tc, sizeof(**priv->channel_tc2realtxq),
+ GFP_KERNEL, node);
+ if (!priv->channel_tc2realtxq[i])
+ goto err_free_channel_tc2realtxq;
+ }
+
+ priv->channel_stats =
+ kcalloc_node(nch, sizeof(*priv->channel_stats), GFP_KERNEL, node);
+ if (!priv->channel_stats)
+ goto err_free_channel_tc2realtxq;
+
return 0;
+err_free_channel_tc2realtxq:
+ while (--i >= 0)
+ kfree(priv->channel_tc2realtxq[i]);
+ kfree(priv->channel_tc2realtxq);
+err_free_tx_rates:
+ kfree(priv->tx_rates);
+err_free_txq2sq:
+ kfree(priv->txq2sq);
+err_destroy_workqueue:
+ destroy_workqueue(priv->wq);
err_free_cpumask:
free_cpumask_var(priv->scratchpad.cpumask);
-
return -ENOMEM;
}
@@ -5166,6 +5272,14 @@ void mlx5e_priv_cleanup(struct mlx5e_priv *priv)
if (!priv->mdev)
return;
+ for (i = 0; i < priv->stats_nch; i++)
+ kvfree(priv->channel_stats[i]);
+ kfree(priv->channel_stats);
+ for (i = 0; i < priv->max_nch; i++)
+ kfree(priv->channel_tc2realtxq[i]);
+ kfree(priv->channel_tc2realtxq);
+ kfree(priv->tx_rates);
+ kfree(priv->txq2sq);
destroy_workqueue(priv->wq);
free_cpumask_var(priv->scratchpad.cpumask);
@@ -5181,13 +5295,44 @@ void mlx5e_priv_cleanup(struct mlx5e_priv *priv)
memset(priv, 0, sizeof(*priv));
}
+static unsigned int mlx5e_get_max_num_txqs(struct mlx5_core_dev *mdev,
+ const struct mlx5e_profile *profile)
+{
+ unsigned int nch, ptp_txqs, qos_txqs;
+
+ nch = mlx5e_profile_max_num_channels(mdev, profile);
+
+ ptp_txqs = MLX5_CAP_GEN(mdev, ts_cqe_to_dest_cqn) &&
+ mlx5e_profile_feature_cap(profile, PTP_TX) ?
+ profile->max_tc : 0;
+
+ qos_txqs = mlx5_qos_is_supported(mdev) &&
+ mlx5e_profile_feature_cap(profile, QOS_HTB) ?
+ mlx5e_qos_max_leaf_nodes(mdev) : 0;
+
+ return nch * profile->max_tc + ptp_txqs + qos_txqs;
+}
+
+static unsigned int mlx5e_get_max_num_rxqs(struct mlx5_core_dev *mdev,
+ const struct mlx5e_profile *profile)
+{
+ unsigned int nch;
+
+ nch = mlx5e_profile_max_num_channels(mdev, profile);
+
+ return nch * profile->rq_groups;
+}
+
struct net_device *
-mlx5e_create_netdev(struct mlx5_core_dev *mdev, const struct mlx5e_profile *profile,
- unsigned int txqs, unsigned int rxqs)
+mlx5e_create_netdev(struct mlx5_core_dev *mdev, const struct mlx5e_profile *profile)
{
struct net_device *netdev;
+ unsigned int txqs, rxqs;
int err;
+ txqs = mlx5e_get_max_num_txqs(mdev, profile);
+ rxqs = mlx5e_get_max_num_rxqs(mdev, profile);
+
netdev = alloc_etherdev_mqs(sizeof(struct mlx5e_priv), txqs, rxqs);
if (!netdev) {
mlx5_core_err(mdev, "alloc_etherdev_mqs() failed\n");
@@ -5432,22 +5577,10 @@ static int mlx5e_probe(struct auxiliary_device *adev,
struct mlx5_core_dev *mdev = edev->mdev;
struct net_device *netdev;
pm_message_t state = {};
- unsigned int txqs, rxqs, ptp_txqs = 0;
struct mlx5e_priv *priv;
- int qos_sqs = 0;
int err;
- int nch;
- if (MLX5_CAP_GEN(mdev, ts_cqe_to_dest_cqn))
- ptp_txqs = profile->max_tc;
-
- if (mlx5_qos_is_supported(mdev))
- qos_sqs = mlx5e_qos_max_leaf_nodes(mdev);
-
- nch = mlx5e_get_max_num_channels(mdev);
- txqs = nch * profile->max_tc + ptp_txqs + qos_sqs;
- rxqs = nch * profile->rq_groups;
- netdev = mlx5e_create_netdev(mdev, profile, txqs, rxqs);
+ netdev = mlx5e_create_netdev(mdev, profile);
if (!netdev) {
mlx5_core_err(mdev, "mlx5e_create_netdev failed\n");
return -ENOMEM;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c
index 48895d7..06d1f46 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c
@@ -50,6 +50,7 @@
#include "fs_core.h"
#include "lib/mlx5.h"
#include "lib/devcom.h"
+#include "lib/vxlan.h"
#define CREATE_TRACE_POINTS
#include "diag/en_rep_tracepoint.h"
#include "en_accel/ipsec.h"
@@ -219,16 +220,22 @@ static int mlx5e_rep_get_sset_count(struct net_device *dev, int sset)
}
}
-static void mlx5e_rep_get_ringparam(struct net_device *dev,
- struct ethtool_ringparam *param)
+static void
+mlx5e_rep_get_ringparam(struct net_device *dev,
+ struct ethtool_ringparam *param,
+ struct kernel_ethtool_ringparam *kernel_param,
+ struct netlink_ext_ack *extack)
{
struct mlx5e_priv *priv = netdev_priv(dev);
mlx5e_ethtool_get_ringparam(priv, param);
}
-static int mlx5e_rep_set_ringparam(struct net_device *dev,
- struct ethtool_ringparam *param)
+static int
+mlx5e_rep_set_ringparam(struct net_device *dev,
+ struct ethtool_ringparam *param,
+ struct kernel_ethtool_ringparam *kernel_param,
+ struct netlink_ext_ack *extack)
{
struct mlx5e_priv *priv = netdev_priv(dev);
@@ -258,7 +265,7 @@ static int mlx5e_rep_get_coalesce(struct net_device *netdev,
{
struct mlx5e_priv *priv = netdev_priv(netdev);
- return mlx5e_ethtool_get_coalesce(priv, coal);
+ return mlx5e_ethtool_get_coalesce(priv, coal, kernel_coal);
}
static int mlx5e_rep_set_coalesce(struct net_device *netdev,
@@ -268,7 +275,7 @@ static int mlx5e_rep_set_coalesce(struct net_device *netdev,
{
struct mlx5e_priv *priv = netdev_priv(netdev);
- return mlx5e_ethtool_set_coalesce(priv, coal);
+ return mlx5e_ethtool_set_coalesce(priv, coal, kernel_coal, extack);
}
static u32 mlx5e_rep_get_rxfh_key_size(struct net_device *netdev)
@@ -585,6 +592,12 @@ bool mlx5e_eswitch_vf_rep(const struct net_device *netdev)
return netdev->netdev_ops == &mlx5e_netdev_ops_rep;
}
+static int mlx5e_rep_max_nch_limit(struct mlx5_core_dev *mdev)
+{
+ return (1 << MLX5_CAP_GEN(mdev, log_max_tir)) /
+ mlx5_eswitch_get_total_vports(mdev);
+}
+
static void mlx5e_build_rep_params(struct net_device *netdev)
{
struct mlx5e_priv *priv = netdev_priv(netdev);
@@ -1027,6 +1040,7 @@ static void mlx5e_uplink_rep_enable(struct mlx5e_priv *priv)
rtnl_lock();
if (netif_running(netdev))
mlx5e_open(netdev);
+ udp_tunnel_nic_reset_ntf(priv->netdev);
netif_device_attach(netdev);
rtnl_unlock();
}
@@ -1048,6 +1062,7 @@ static void mlx5e_uplink_rep_disable(struct mlx5e_priv *priv)
mlx5_notifier_unregister(mdev, &priv->events_nb);
mlx5e_rep_tc_disable(priv);
mlx5_lag_remove_netdev(mdev, priv->netdev);
+ mlx5_vxlan_reset_to_default(mdev->vxlan);
}
static MLX5E_DEFINE_STATS_GRP(sw_rep, 0);
@@ -1107,7 +1122,7 @@ static const struct mlx5e_profile mlx5e_rep_profile = {
.rq_groups = MLX5E_NUM_RQ_GROUPS(REGULAR),
.stats_grps = mlx5e_rep_stats_grps,
.stats_grps_num = mlx5e_rep_stats_grps_num,
- .rx_ptp_support = false,
+ .max_nch_limit = mlx5e_rep_max_nch_limit,
};
static const struct mlx5e_profile mlx5e_uplink_rep_profile = {
@@ -1128,7 +1143,6 @@ static const struct mlx5e_profile mlx5e_uplink_rep_profile = {
.rq_groups = MLX5E_NUM_RQ_GROUPS(XSK),
.stats_grps = mlx5e_ul_rep_stats_grps,
.stats_grps_num = mlx5e_ul_rep_stats_grps_num,
- .rx_ptp_support = false,
};
/* e-Switch vport representors */
@@ -1179,14 +1193,10 @@ mlx5e_vport_vf_rep_load(struct mlx5_core_dev *dev, struct mlx5_eswitch_rep *rep)
struct devlink_port *dl_port;
struct net_device *netdev;
struct mlx5e_priv *priv;
- unsigned int txqs, rxqs;
- int nch, err;
+ int err;
profile = &mlx5e_rep_profile;
- nch = mlx5e_get_max_num_channels(dev);
- txqs = nch * profile->max_tc;
- rxqs = nch * profile->rq_groups;
- netdev = mlx5e_create_netdev(dev, profile, txqs, rxqs);
+ netdev = mlx5e_create_netdev(dev, profile);
if (!netdev) {
mlx5_core_warn(dev,
"Failed to create representor netdev for vport %d\n",
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
index 793511d..e86ccc2 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
@@ -37,6 +37,7 @@
#include <net/ip6_checksum.h>
#include <net/page_pool.h>
#include <net/inet_ecn.h>
+#include <net/gro.h>
#include <net/udp.h>
#include <net/tcp.h>
#include "en.h"
@@ -278,8 +279,8 @@ static inline int mlx5e_page_alloc_pool(struct mlx5e_rq *rq,
if (unlikely(!dma_info->page))
return -ENOMEM;
- dma_info->addr = dma_map_page(rq->pdev, dma_info->page, 0,
- PAGE_SIZE, rq->buff.map_dir);
+ dma_info->addr = dma_map_page_attrs(rq->pdev, dma_info->page, 0, PAGE_SIZE,
+ rq->buff.map_dir, DMA_ATTR_SKIP_CPU_SYNC);
if (unlikely(dma_mapping_error(rq->pdev, dma_info->addr))) {
page_pool_recycle_direct(rq->page_pool, dma_info->page);
dma_info->page = NULL;
@@ -300,7 +301,8 @@ static inline int mlx5e_page_alloc(struct mlx5e_rq *rq,
void mlx5e_page_dma_unmap(struct mlx5e_rq *rq, struct mlx5e_dma_info *dma_info)
{
- dma_unmap_page(rq->pdev, dma_info->addr, PAGE_SIZE, rq->buff.map_dir);
+ dma_unmap_page_attrs(rq->pdev, dma_info->addr, PAGE_SIZE, rq->buff.map_dir,
+ DMA_ATTR_SKIP_CPU_SYNC);
}
void mlx5e_page_release_dynamic(struct mlx5e_rq *rq,
@@ -618,7 +620,7 @@ static int mlx5e_alloc_rx_hd_mpwqe(struct mlx5e_rq *rq)
struct mlx5e_icosq *sq = rq->icosq;
int i, err, max_klm_entries, len;
- max_klm_entries = MLX5E_MAX_KLM_PER_WQE(rq->mdev);
+ max_klm_entries = MLX5E_MAX_KLM_PER_WQE;
klm_entries = bitmap_find_window(shampo->bitmap,
shampo->hd_per_wqe,
shampo->hd_per_wq, shampo->pi);
@@ -1602,6 +1604,12 @@ static void trigger_report(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe)
}
}
+static void mlx5e_handle_rx_err_cqe(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe)
+{
+ trigger_report(rq, cqe);
+ rq->stats->wqe_err++;
+}
+
static void mlx5e_handle_rx_cqe(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe)
{
struct mlx5_wq_cyc *wq = &rq->wqe.wq;
@@ -1615,8 +1623,7 @@ static void mlx5e_handle_rx_cqe(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe)
cqe_bcnt = be32_to_cpu(cqe->byte_cnt);
if (unlikely(MLX5E_RX_ERR_CQE(cqe))) {
- trigger_report(rq, cqe);
- rq->stats->wqe_err++;
+ mlx5e_handle_rx_err_cqe(rq, cqe);
goto free_wqe;
}
@@ -1669,7 +1676,7 @@ static void mlx5e_handle_rx_cqe_rep(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe)
cqe_bcnt = be32_to_cpu(cqe->byte_cnt);
if (unlikely(MLX5E_RX_ERR_CQE(cqe))) {
- rq->stats->wqe_err++;
+ mlx5e_handle_rx_err_cqe(rq, cqe);
goto free_wqe;
}
@@ -1718,8 +1725,7 @@ static void mlx5e_handle_rx_cqe_mpwrq_rep(struct mlx5e_rq *rq, struct mlx5_cqe64
wi->consumed_strides += cstrides;
if (unlikely(MLX5E_RX_ERR_CQE(cqe))) {
- trigger_report(rq, cqe);
- rq->stats->wqe_err++;
+ mlx5e_handle_rx_err_cqe(rq, cqe);
goto mpwrq_cqe_out;
}
@@ -1987,8 +1993,7 @@ static void mlx5e_handle_rx_cqe_mpwrq_shampo(struct mlx5e_rq *rq, struct mlx5_cq
wi->consumed_strides += cstrides;
if (unlikely(MLX5E_RX_ERR_CQE(cqe))) {
- trigger_report(rq, cqe);
- stats->wqe_err++;
+ mlx5e_handle_rx_err_cqe(rq, cqe);
goto mpwrq_cqe_out;
}
@@ -2057,8 +2062,7 @@ static void mlx5e_handle_rx_cqe_mpwrq(struct mlx5e_rq *rq, struct mlx5_cqe64 *cq
wi->consumed_strides += cstrides;
if (unlikely(MLX5E_RX_ERR_CQE(cqe))) {
- trigger_report(rq, cqe);
- rq->stats->wqe_err++;
+ mlx5e_handle_rx_err_cqe(rq, cqe);
goto mpwrq_cqe_out;
}
@@ -2188,7 +2192,7 @@ static inline void mlx5i_complete_rx_cqe(struct mlx5e_rq *rq,
priv = mlx5i_epriv(netdev);
tstamp = &priv->tstamp;
- stats = &priv->channel_stats[rq->ix].rq;
+ stats = rq->stats;
flags_rqpn = be32_to_cpu(cqe->flags_rqpn);
g = (flags_rqpn >> 28) & 3;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c
index 2a9bfc3..26e326f 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c
@@ -35,6 +35,7 @@
#include "en_accel/tls.h"
#include "en_accel/en_accel.h"
#include "en/ptp.h"
+#include "en/port.h"
static unsigned int stats_grps_num(struct mlx5e_priv *priv)
{
@@ -463,7 +464,7 @@ static MLX5E_DECLARE_STATS_GRP_OP_UPDATE_STATS(sw)
for (i = 0; i < priv->stats_nch; i++) {
struct mlx5e_channel_stats *channel_stats =
- &priv->channel_stats[i];
+ priv->channel_stats[i];
int j;
mlx5e_stats_grp_sw_update_stats_rq_stats(s, &channel_stats->rq);
@@ -1158,12 +1159,99 @@ static MLX5E_DECLARE_STATS_GRP_OP_UPDATE_STATS(phy)
mlx5_core_access_reg(mdev, in, sz, out, sz, MLX5_REG_PPCNT, 0, 0);
}
-void mlx5e_stats_fec_get(struct mlx5e_priv *priv,
- struct ethtool_fec_stats *fec_stats)
+static int fec_num_lanes(struct mlx5_core_dev *dev)
+{
+ u32 out[MLX5_ST_SZ_DW(pmlp_reg)] = {};
+ u32 in[MLX5_ST_SZ_DW(pmlp_reg)] = {};
+ int err;
+
+ MLX5_SET(pmlp_reg, in, local_port, 1);
+ err = mlx5_core_access_reg(dev, in, sizeof(in), out, sizeof(out),
+ MLX5_REG_PMLP, 0, 0);
+ if (err)
+ return 0;
+
+ return MLX5_GET(pmlp_reg, out, width);
+}
+
+static int fec_active_mode(struct mlx5_core_dev *mdev)
+{
+ unsigned long fec_active_long;
+ u32 fec_active;
+
+ if (mlx5e_get_fec_mode(mdev, &fec_active, NULL))
+ return MLX5E_FEC_NOFEC;
+
+ fec_active_long = fec_active;
+ return find_first_bit(&fec_active_long, sizeof(unsigned long) * BITS_PER_BYTE);
+}
+
+#define MLX5E_STATS_SET_FEC_BLOCK(idx) ({ \
+ fec_stats->corrected_blocks.lanes[(idx)] = \
+ MLX5E_READ_CTR64_BE_F(ppcnt, phys_layer_cntrs, \
+ fc_fec_corrected_blocks_lane##idx); \
+ fec_stats->uncorrectable_blocks.lanes[(idx)] = \
+ MLX5E_READ_CTR64_BE_F(ppcnt, phys_layer_cntrs, \
+ fc_fec_uncorrectable_blocks_lane##idx); \
+})
+
+static void fec_set_fc_stats(struct ethtool_fec_stats *fec_stats,
+ u32 *ppcnt, u8 lanes)
+{
+ if (lanes > 3) { /* 4 lanes */
+ MLX5E_STATS_SET_FEC_BLOCK(3);
+ MLX5E_STATS_SET_FEC_BLOCK(2);
+ }
+ if (lanes > 1) /* 2 lanes */
+ MLX5E_STATS_SET_FEC_BLOCK(1);
+ if (lanes > 0) /* 1 lane */
+ MLX5E_STATS_SET_FEC_BLOCK(0);
+}
+
+static void fec_set_rs_stats(struct ethtool_fec_stats *fec_stats, u32 *ppcnt)
+{
+ fec_stats->corrected_blocks.total =
+ MLX5E_READ_CTR64_BE_F(ppcnt, phys_layer_cntrs,
+ rs_fec_corrected_blocks);
+ fec_stats->uncorrectable_blocks.total =
+ MLX5E_READ_CTR64_BE_F(ppcnt, phys_layer_cntrs,
+ rs_fec_uncorrectable_blocks);
+}
+
+static void fec_set_block_stats(struct mlx5e_priv *priv,
+ struct ethtool_fec_stats *fec_stats)
+{
+ struct mlx5_core_dev *mdev = priv->mdev;
+ u32 out[MLX5_ST_SZ_DW(ppcnt_reg)] = {};
+ u32 in[MLX5_ST_SZ_DW(ppcnt_reg)] = {};
+ int sz = MLX5_ST_SZ_BYTES(ppcnt_reg);
+ int mode = fec_active_mode(mdev);
+
+ if (mode == MLX5E_FEC_NOFEC)
+ return;
+
+ MLX5_SET(ppcnt_reg, in, local_port, 1);
+ MLX5_SET(ppcnt_reg, in, grp, MLX5_PHYSICAL_LAYER_COUNTERS_GROUP);
+ if (mlx5_core_access_reg(mdev, in, sz, out, sz, MLX5_REG_PPCNT, 0, 0))
+ return;
+
+ switch (mode) {
+ case MLX5E_FEC_RS_528_514:
+ case MLX5E_FEC_RS_544_514:
+ case MLX5E_FEC_LLRS_272_257_1:
+ fec_set_rs_stats(fec_stats, out);
+ return;
+ case MLX5E_FEC_FIRECODE:
+ fec_set_fc_stats(fec_stats, out, fec_num_lanes(mdev));
+ }
+}
+
+static void fec_set_corrected_bits_total(struct mlx5e_priv *priv,
+ struct ethtool_fec_stats *fec_stats)
{
u32 ppcnt_phy_statistical[MLX5_ST_SZ_DW(ppcnt_reg)];
struct mlx5_core_dev *mdev = priv->mdev;
- u32 in[MLX5_ST_SZ_DW(ppcnt_reg)] = {0};
+ u32 in[MLX5_ST_SZ_DW(ppcnt_reg)] = {};
int sz = MLX5_ST_SZ_BYTES(ppcnt_reg);
if (!MLX5_CAP_PCAM_FEATURE(mdev, ppcnt_statistical_group))
@@ -1181,6 +1269,13 @@ void mlx5e_stats_fec_get(struct mlx5e_priv *priv,
phy_corrected_bits);
}
+void mlx5e_stats_fec_get(struct mlx5e_priv *priv,
+ struct ethtool_fec_stats *fec_stats)
+{
+ fec_set_corrected_bits_total(priv, fec_stats);
+ fec_set_block_stats(priv, fec_stats);
+}
+
#define PPORT_ETH_EXT_OFF(c) \
MLX5_BYTE_OFF(ppcnt_reg, \
counter_set.eth_extended_cntrs_grp_data_layout.c##_high)
@@ -2076,7 +2171,7 @@ static MLX5E_DECLARE_STATS_GRP_OP_FILL_STRS(ptp)
for (i = 0; i < NUM_PTP_CH_STATS; i++)
sprintf(data + (idx++) * ETH_GSTRING_LEN,
- ptp_ch_stats_desc[i].format);
+ "%s", ptp_ch_stats_desc[i].format);
if (priv->tx_ptp_opened) {
for (tc = 0; tc < priv->max_opened_tc; tc++)
@@ -2197,21 +2292,21 @@ static MLX5E_DECLARE_STATS_GRP_OP_FILL_STATS(channels)
for (i = 0; i < max_nch; i++)
for (j = 0; j < NUM_CH_STATS; j++)
data[idx++] =
- MLX5E_READ_CTR64_CPU(&priv->channel_stats[i].ch,
+ MLX5E_READ_CTR64_CPU(&priv->channel_stats[i]->ch,
ch_stats_desc, j);
for (i = 0; i < max_nch; i++) {
for (j = 0; j < NUM_RQ_STATS; j++)
data[idx++] =
- MLX5E_READ_CTR64_CPU(&priv->channel_stats[i].rq,
+ MLX5E_READ_CTR64_CPU(&priv->channel_stats[i]->rq,
rq_stats_desc, j);
for (j = 0; j < NUM_XSKRQ_STATS * is_xsk; j++)
data[idx++] =
- MLX5E_READ_CTR64_CPU(&priv->channel_stats[i].xskrq,
+ MLX5E_READ_CTR64_CPU(&priv->channel_stats[i]->xskrq,
xskrq_stats_desc, j);
for (j = 0; j < NUM_RQ_XDPSQ_STATS; j++)
data[idx++] =
- MLX5E_READ_CTR64_CPU(&priv->channel_stats[i].rq_xdpsq,
+ MLX5E_READ_CTR64_CPU(&priv->channel_stats[i]->rq_xdpsq,
rq_xdpsq_stats_desc, j);
}
@@ -2219,17 +2314,17 @@ static MLX5E_DECLARE_STATS_GRP_OP_FILL_STATS(channels)
for (i = 0; i < max_nch; i++)
for (j = 0; j < NUM_SQ_STATS; j++)
data[idx++] =
- MLX5E_READ_CTR64_CPU(&priv->channel_stats[i].sq[tc],
+ MLX5E_READ_CTR64_CPU(&priv->channel_stats[i]->sq[tc],
sq_stats_desc, j);
for (i = 0; i < max_nch; i++) {
for (j = 0; j < NUM_XSKSQ_STATS * is_xsk; j++)
data[idx++] =
- MLX5E_READ_CTR64_CPU(&priv->channel_stats[i].xsksq,
+ MLX5E_READ_CTR64_CPU(&priv->channel_stats[i]->xsksq,
xsksq_stats_desc, j);
for (j = 0; j < NUM_XDPSQ_STATS; j++)
data[idx++] =
- MLX5E_READ_CTR64_CPU(&priv->channel_stats[i].xdpsq,
+ MLX5E_READ_CTR64_CPU(&priv->channel_stats[i]->xdpsq,
xdpsq_stats_desc, j);
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c b/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c
index 3d45f4a..3d908a7 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c
@@ -39,10 +39,6 @@
#include <linux/rhashtable.h>
#include <linux/refcount.h>
#include <linux/completion.h>
-#include <linux/if_macvlan.h>
-#include <net/tc_act/tc_pedit.h>
-#include <net/tc_act/tc_csum.h>
-#include <net/psample.h>
#include <net/arp.h>
#include <net/ipv6_stubs.h>
#include <net/bareudp.h>
@@ -62,6 +58,7 @@
#include "en/mod_hdr.h"
#include "en/tc_tun_encap.h"
#include "en/tc/sample.h"
+#include "en/tc/act/act.h"
#include "lib/devcom.h"
#include "lib/geneve.h"
#include "lib/fs_chains.h"
@@ -70,9 +67,6 @@
#include "lag/lag.h"
#include "lag/mp.h"
-#define nic_chains(priv) ((priv)->fs.tc.chains)
-#define MLX5_MH_ACT_SZ MLX5_UN_SZ_BYTES(set_add_copy_action_in_auto)
-
#define MLX5E_TC_TABLE_NUM_GROUPS 4
#define MLX5E_TC_TABLE_MAX_GROUP_SIZE BIT(18)
@@ -209,12 +203,9 @@ mlx5e_tc_match_to_reg_set_and_get_id(struct mlx5_core_dev *mdev,
char *modact;
int err;
- err = alloc_mod_hdr_actions(mdev, ns, mod_hdr_acts);
- if (err)
- return err;
-
- modact = mod_hdr_acts->actions +
- (mod_hdr_acts->num_actions * MLX5_MH_ACT_SZ);
+ modact = mlx5e_mod_hdr_alloc(mdev, ns, mod_hdr_acts);
+ if (IS_ERR(modact))
+ return PTR_ERR(modact);
/* Firmware has 5bit length field and 0 means 32bits */
if (mlen == 32)
@@ -333,7 +324,7 @@ void mlx5e_tc_match_to_reg_mod_hdr_change(struct mlx5_core_dev *mdev,
int mlen = mlx5e_tc_attr_to_reg_mappings[type].mlen;
char *modact;
- modact = mod_hdr_acts->actions + (act_id * MLX5_MH_ACT_SZ);
+ modact = mlx5e_mod_hdr_get_item(mod_hdr_acts, act_id);
/* Firmware has 5bit length field and 0 means 32bits */
if (mlen == 32)
@@ -403,7 +394,7 @@ bool mlx5e_is_eswitch_flow(struct mlx5e_tc_flow *flow)
return flow_flag_test(flow, ESWITCH);
}
-static bool mlx5e_is_ft_flow(struct mlx5e_tc_flow *flow)
+bool mlx5e_is_ft_flow(struct mlx5e_tc_flow *flow)
{
return flow_flag_test(flow, FT);
}
@@ -413,7 +404,7 @@ bool mlx5e_is_offloaded_flow(struct mlx5e_tc_flow *flow)
return flow_flag_test(flow, OFFLOADED);
}
-static int get_flow_name_space(struct mlx5e_tc_flow *flow)
+int mlx5e_get_flow_namespace(struct mlx5e_tc_flow *flow)
{
return mlx5e_is_eswitch_flow(flow) ?
MLX5_FLOW_NAMESPACE_FDB : MLX5_FLOW_NAMESPACE_KERNEL;
@@ -424,7 +415,7 @@ get_mod_hdr_table(struct mlx5e_priv *priv, struct mlx5e_tc_flow *flow)
{
struct mlx5_eswitch *esw = priv->mdev->priv.eswitch;
- return get_flow_name_space(flow) == MLX5_FLOW_NAMESPACE_FDB ?
+ return mlx5e_get_flow_namespace(flow) == MLX5_FLOW_NAMESPACE_FDB ?
&esw->offloads.mod_hdr :
&priv->fs.tc.mod_hdr;
}
@@ -437,7 +428,7 @@ static int mlx5e_attach_mod_hdr(struct mlx5e_priv *priv,
struct mlx5e_mod_hdr_handle *mh;
mh = mlx5e_mod_hdr_attach(priv->mdev, get_mod_hdr_table(priv, flow),
- get_flow_name_space(flow),
+ mlx5e_get_flow_namespace(flow),
&parse_attr->mod_hdr_acts);
if (IS_ERR(mh))
return PTR_ERR(mh);
@@ -941,7 +932,7 @@ mlx5e_add_offloaded_nic_rule(struct mlx5e_priv *priv,
struct mlx5_flow_attr *attr)
{
struct mlx5_flow_context *flow_context = &spec->flow_context;
- struct mlx5_fs_chains *nic_chains = nic_chains(priv);
+ struct mlx5_fs_chains *nic_chains = mlx5e_nic_chains(priv);
struct mlx5_nic_flow_attr *nic_attr = attr->nic_attr;
struct mlx5e_tc_table *tc = &priv->fs.tc;
struct mlx5_flow_destination dest[2] = {};
@@ -1076,7 +1067,7 @@ mlx5e_tc_add_nic_flow(struct mlx5e_priv *priv,
if (attr->action & MLX5_FLOW_CONTEXT_ACTION_MOD_HDR) {
err = mlx5e_attach_mod_hdr(priv, flow, parse_attr);
- dealloc_mod_hdr_actions(&parse_attr->mod_hdr_acts);
+ mlx5e_mod_hdr_dealloc(&parse_attr->mod_hdr_acts);
if (err)
return err;
}
@@ -1095,7 +1086,7 @@ void mlx5e_del_offloaded_nic_rule(struct mlx5e_priv *priv,
struct mlx5_flow_handle *rule,
struct mlx5_flow_attr *attr)
{
- struct mlx5_fs_chains *nic_chains = nic_chains(priv);
+ struct mlx5_fs_chains *nic_chains = mlx5e_nic_chains(priv);
mlx5_del_flow_rules(rule);
@@ -1127,21 +1118,21 @@ static void mlx5e_tc_del_nic_flow(struct mlx5e_priv *priv,
mutex_lock(&priv->fs.tc.t_lock);
if (!mlx5e_tc_num_filters(priv, MLX5_TC_FLAG(NIC_OFFLOAD)) &&
!IS_ERR_OR_NULL(tc->t)) {
- mlx5_chains_put_table(nic_chains(priv), 0, 1, MLX5E_TC_FT_LEVEL);
+ mlx5_chains_put_table(mlx5e_nic_chains(priv), 0, 1, MLX5E_TC_FT_LEVEL);
priv->fs.tc.t = NULL;
}
mutex_unlock(&priv->fs.tc.t_lock);
- kvfree(attr->parse_attr);
-
if (attr->action & MLX5_FLOW_CONTEXT_ACTION_MOD_HDR)
mlx5e_detach_mod_hdr(priv, flow);
- mlx5_fc_destroy(priv->mdev, attr->counter);
+ if (attr->action & MLX5_FLOW_CONTEXT_ACTION_COUNT)
+ mlx5_fc_destroy(priv->mdev, attr->counter);
if (flow_flag_test(flow, HAIRPIN))
mlx5e_hairpin_flow_del(priv, flow);
+ kvfree(attr->parse_attr);
kfree(flow->attr);
}
@@ -1196,21 +1187,16 @@ void mlx5e_tc_unoffload_fdb_rules(struct mlx5_eswitch *esw,
if (attr->flags & MLX5_ESW_ATTR_FLAG_SLOW_PATH)
goto offload_rule_0;
- if (flow_flag_test(flow, CT)) {
- mlx5_tc_ct_delete_flow(get_ct_priv(flow->priv), flow, attr);
- return;
- }
-
- if (flow_flag_test(flow, SAMPLE)) {
- mlx5e_tc_sample_unoffload(get_sample_priv(flow->priv), flow->rule[0], attr);
- return;
- }
-
if (attr->esw_attr->split_count)
mlx5_eswitch_del_fwd_rule(esw, flow->rule[1], attr);
+ if (flow_flag_test(flow, CT))
+ mlx5_tc_ct_delete_flow(get_ct_priv(flow->priv), flow, attr);
+ else if (flow_flag_test(flow, SAMPLE))
+ mlx5e_tc_sample_unoffload(get_sample_priv(flow->priv), flow->rule[0], attr);
+ else
offload_rule_0:
- mlx5_eswitch_del_offloaded_rule(esw, flow->rule[0], attr);
+ mlx5_eswitch_del_offloaded_rule(esw, flow->rule[0], attr);
}
struct mlx5_flow_handle *
@@ -1308,8 +1294,6 @@ static void remove_unready_flow(struct mlx5e_tc_flow *flow)
mutex_unlock(&uplink_priv->unready_flows_lock);
}
-static bool same_hw_devs(struct mlx5e_priv *priv, struct mlx5e_priv *peer_priv);
-
bool mlx5e_tc_is_vf_tunnel(struct net_device *out_dev, struct net_device *route_dev)
{
struct mlx5_core_dev *out_mdev, *route_mdev;
@@ -1324,7 +1308,7 @@ bool mlx5e_tc_is_vf_tunnel(struct net_device *out_dev, struct net_device *route_
route_mdev->coredev_type != MLX5_COREDEV_VF)
return false;
- return same_hw_devs(out_priv, route_priv);
+ return mlx5e_same_hw_devs(out_priv, route_priv);
}
int mlx5e_tc_query_route_vport(struct net_device *out_dev, struct net_device *route_dev, u16 *vport)
@@ -1371,7 +1355,7 @@ int mlx5e_tc_add_flow_mod_hdr(struct mlx5e_priv *priv,
struct mlx5_modify_hdr *mod_hdr;
mod_hdr = mlx5_modify_header_alloc(priv->mdev,
- get_flow_name_space(flow),
+ mlx5e_get_flow_namespace(flow),
mod_hdr_acts->num_actions,
mod_hdr_acts->actions);
if (IS_ERR(mod_hdr))
@@ -1445,7 +1429,9 @@ mlx5e_tc_add_fdb_flow(struct mlx5e_priv *priv,
MLX5_FLOW_NAMESPACE_FDB, VPORT_TO_REG,
metadata);
if (err)
- return err;
+ goto err_out;
+
+ attr->action |= MLX5_FLOW_CONTEXT_ACTION_MOD_HDR;
}
}
@@ -1461,13 +1447,15 @@ mlx5e_tc_add_fdb_flow(struct mlx5e_priv *priv,
if (attr->chain) {
NL_SET_ERR_MSG_MOD(extack,
"Internal port rule is only supported on chain 0");
- return -EOPNOTSUPP;
+ err = -EOPNOTSUPP;
+ goto err_out;
}
if (attr->dest_chain) {
NL_SET_ERR_MSG_MOD(extack,
"Internal port rule offload doesn't support goto action");
- return -EOPNOTSUPP;
+ err = -EOPNOTSUPP;
+ goto err_out;
}
int_port = mlx5e_tc_int_port_get(mlx5e_get_int_port_priv(priv),
@@ -1475,8 +1463,10 @@ mlx5e_tc_add_fdb_flow(struct mlx5e_priv *priv,
flow_flag_test(flow, EGRESS) ?
MLX5E_TC_INT_PORT_EGRESS :
MLX5E_TC_INT_PORT_INGRESS);
- if (IS_ERR(int_port))
- return PTR_ERR(int_port);
+ if (IS_ERR(int_port)) {
+ err = PTR_ERR(int_port);
+ goto err_out;
+ }
esw_attr->int_port = int_port;
}
@@ -1624,15 +1614,12 @@ static void mlx5e_tc_del_fdb_flow(struct mlx5e_priv *priv,
mlx5_tc_ct_match_del(get_ct_priv(priv), &flow->attr->ct_attr);
if (attr->action & MLX5_FLOW_CONTEXT_ACTION_MOD_HDR) {
- dealloc_mod_hdr_actions(&attr->parse_attr->mod_hdr_acts);
+ mlx5e_mod_hdr_dealloc(&attr->parse_attr->mod_hdr_acts);
if (vf_tun && attr->modify_hdr)
mlx5_modify_header_dealloc(priv->mdev, attr->modify_hdr);
else
mlx5e_detach_mod_hdr(priv, flow);
}
- kfree(attr->sample_attr);
- kvfree(attr->parse_attr);
- kvfree(attr->esw_attr->rx_tun_attr);
if (attr->action & MLX5_FLOW_CONTEXT_ACTION_COUNT)
mlx5_fc_destroy(esw_attr->counter_dev, attr->counter);
@@ -1646,6 +1633,9 @@ static void mlx5e_tc_del_fdb_flow(struct mlx5e_priv *priv,
if (flow_flag_test(flow, L3_TO_L2_DECAP))
mlx5e_detach_decap(priv, flow);
+ kfree(attr->sample_attr);
+ kvfree(attr->esw_attr->rx_tun_attr);
+ kvfree(attr->parse_attr);
kfree(flow->attr);
}
@@ -1948,6 +1938,111 @@ u8 mlx5e_tc_get_ip_version(struct mlx5_flow_spec *spec, bool outer)
return ip_version;
}
+/* Tunnel device follows RFC 6040, see include/net/inet_ecn.h.
+ * And changes inner ip_ecn depending on inner and outer ip_ecn as follows:
+ * +---------+----------------------------------------+
+ * |Arriving | Arriving Outer Header |
+ * | Inner +---------+---------+---------+----------+
+ * | Header | Not-ECT | ECT(0) | ECT(1) | CE |
+ * +---------+---------+---------+---------+----------+
+ * | Not-ECT | Not-ECT | Not-ECT | Not-ECT | <drop> |
+ * | ECT(0) | ECT(0) | ECT(0) | ECT(1) | CE* |
+ * | ECT(1) | ECT(1) | ECT(1) | ECT(1)* | CE* |
+ * | CE | CE | CE | CE | CE |
+ * +---------+---------+---------+---------+----------+
+ *
+ * Tc matches on inner after decapsulation on tunnel device, but hw offload matches
+ * the inner ip_ecn value before hardware decap action.
+ *
+ * Cells marked are changed from original inner packet ip_ecn value during decap, and
+ * so matching those values on inner ip_ecn before decap will fail.
+ *
+ * The following helper allows offload when inner ip_ecn won't be changed by outer ip_ecn,
+ * except for the outer ip_ecn = CE, where in all cases inner ip_ecn will be changed to CE,
+ * and such we can drop the inner ip_ecn=CE match.
+ */
+
+static int mlx5e_tc_verify_tunnel_ecn(struct mlx5e_priv *priv,
+ struct flow_cls_offload *f,
+ bool *match_inner_ecn)
+{
+ u8 outer_ecn_mask = 0, outer_ecn_key = 0, inner_ecn_mask = 0, inner_ecn_key = 0;
+ struct flow_rule *rule = flow_cls_offload_flow_rule(f);
+ struct netlink_ext_ack *extack = f->common.extack;
+ struct flow_match_ip match;
+
+ *match_inner_ecn = true;
+
+ if (flow_rule_match_key(rule, FLOW_DISSECTOR_KEY_ENC_IP)) {
+ flow_rule_match_enc_ip(rule, &match);
+ outer_ecn_key = match.key->tos & INET_ECN_MASK;
+ outer_ecn_mask = match.mask->tos & INET_ECN_MASK;
+ }
+
+ if (flow_rule_match_key(rule, FLOW_DISSECTOR_KEY_IP)) {
+ flow_rule_match_ip(rule, &match);
+ inner_ecn_key = match.key->tos & INET_ECN_MASK;
+ inner_ecn_mask = match.mask->tos & INET_ECN_MASK;
+ }
+
+ if (outer_ecn_mask != 0 && outer_ecn_mask != INET_ECN_MASK) {
+ NL_SET_ERR_MSG_MOD(extack, "Partial match on enc_tos ecn bits isn't supported");
+ netdev_warn(priv->netdev, "Partial match on enc_tos ecn bits isn't supported");
+ return -EOPNOTSUPP;
+ }
+
+ if (!outer_ecn_mask) {
+ if (!inner_ecn_mask)
+ return 0;
+
+ NL_SET_ERR_MSG_MOD(extack,
+ "Matching on tos ecn bits without also matching enc_tos ecn bits isn't supported");
+ netdev_warn(priv->netdev,
+ "Matching on tos ecn bits without also matching enc_tos ecn bits isn't supported");
+ return -EOPNOTSUPP;
+ }
+
+ if (inner_ecn_mask && inner_ecn_mask != INET_ECN_MASK) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "Partial match on tos ecn bits with match on enc_tos ecn bits isn't supported");
+ netdev_warn(priv->netdev,
+ "Partial match on tos ecn bits with match on enc_tos ecn bits isn't supported");
+ return -EOPNOTSUPP;
+ }
+
+ if (!inner_ecn_mask)
+ return 0;
+
+ /* Both inner and outer have full mask on ecn */
+
+ if (outer_ecn_key == INET_ECN_ECT_1) {
+ /* inner ecn might change by DECAP action */
+
+ NL_SET_ERR_MSG_MOD(extack, "Match on enc_tos ecn = ECT(1) isn't supported");
+ netdev_warn(priv->netdev, "Match on enc_tos ecn = ECT(1) isn't supported");
+ return -EOPNOTSUPP;
+ }
+
+ if (outer_ecn_key != INET_ECN_CE)
+ return 0;
+
+ if (inner_ecn_key != INET_ECN_CE) {
+ /* Can't happen in software, as packet ecn will be changed to CE after decap */
+ NL_SET_ERR_MSG_MOD(extack,
+ "Match on tos enc_tos ecn = CE while match on tos ecn != CE isn't supported");
+ netdev_warn(priv->netdev,
+ "Match on tos enc_tos ecn = CE while match on tos ecn != CE isn't supported");
+ return -EOPNOTSUPP;
+ }
+
+ /* outer ecn = CE, inner ecn = CE, as decap will change inner ecn to CE in anycase,
+ * drop match on inner ecn
+ */
+ *match_inner_ecn = false;
+
+ return 0;
+}
+
static int parse_tunnel_attr(struct mlx5e_priv *priv,
struct mlx5e_tc_flow *flow,
struct mlx5_flow_spec *spec,
@@ -2053,16 +2148,14 @@ static void *get_match_outer_headers_value(struct mlx5_flow_spec *spec)
outer_headers);
}
-static void *get_match_headers_value(u32 flags,
- struct mlx5_flow_spec *spec)
+void *mlx5e_get_match_headers_value(u32 flags, struct mlx5_flow_spec *spec)
{
return (flags & MLX5_FLOW_CONTEXT_ACTION_DECAP) ?
get_match_inner_headers_value(spec) :
get_match_outer_headers_value(spec);
}
-static void *get_match_headers_criteria(u32 flags,
- struct mlx5_flow_spec *spec)
+void *mlx5e_get_match_headers_criteria(u32 flags, struct mlx5_flow_spec *spec)
{
return (flags & MLX5_FLOW_CONTEXT_ACTION_DECAP) ?
get_match_inner_headers_criteria(spec) :
@@ -2143,6 +2236,7 @@ static int __parse_cls_flower(struct mlx5e_priv *priv,
struct flow_rule *rule = flow_cls_offload_flow_rule(f);
struct flow_dissector *dissector = rule->match.dissector;
enum fs_flow_table_type fs_type;
+ bool match_inner_ecn = true;
u16 addr_type = 0;
u8 ip_proto = 0;
u8 *match_level;
@@ -2196,6 +2290,10 @@ static int __parse_cls_flower(struct mlx5e_priv *priv,
headers_c = get_match_inner_headers_criteria(spec);
headers_v = get_match_inner_headers_value(spec);
}
+
+ err = mlx5e_tc_verify_tunnel_ecn(priv, f, &match_inner_ecn);
+ if (err)
+ return err;
}
err = mlx5e_flower_parse_meta(filter_dev, f);
@@ -2419,10 +2517,12 @@ static int __parse_cls_flower(struct mlx5e_priv *priv,
struct flow_match_ip match;
flow_rule_match_ip(rule, &match);
- MLX5_SET(fte_match_set_lyr_2_4, headers_c, ip_ecn,
- match.mask->tos & 0x3);
- MLX5_SET(fte_match_set_lyr_2_4, headers_v, ip_ecn,
- match.key->tos & 0x3);
+ if (match_inner_ecn) {
+ MLX5_SET(fte_match_set_lyr_2_4, headers_c, ip_ecn,
+ match.mask->tos & 0x3);
+ MLX5_SET(fte_match_set_lyr_2_4, headers_v, ip_ecn,
+ match.key->tos & 0x3);
+ }
MLX5_SET(fte_match_set_lyr_2_4, headers_c, ip_dscp,
match.mask->tos >> 2);
@@ -2607,55 +2707,6 @@ static int parse_cls_flower(struct mlx5e_priv *priv,
return err;
}
-struct pedit_headers {
- struct ethhdr eth;
- struct vlan_hdr vlan;
- struct iphdr ip4;
- struct ipv6hdr ip6;
- struct tcphdr tcp;
- struct udphdr udp;
-};
-
-struct pedit_headers_action {
- struct pedit_headers vals;
- struct pedit_headers masks;
- u32 pedits;
-};
-
-static int pedit_header_offsets[] = {
- [FLOW_ACT_MANGLE_HDR_TYPE_ETH] = offsetof(struct pedit_headers, eth),
- [FLOW_ACT_MANGLE_HDR_TYPE_IP4] = offsetof(struct pedit_headers, ip4),
- [FLOW_ACT_MANGLE_HDR_TYPE_IP6] = offsetof(struct pedit_headers, ip6),
- [FLOW_ACT_MANGLE_HDR_TYPE_TCP] = offsetof(struct pedit_headers, tcp),
- [FLOW_ACT_MANGLE_HDR_TYPE_UDP] = offsetof(struct pedit_headers, udp),
-};
-
-#define pedit_header(_ph, _htype) ((void *)(_ph) + pedit_header_offsets[_htype])
-
-static int set_pedit_val(u8 hdr_type, u32 mask, u32 val, u32 offset,
- struct pedit_headers_action *hdrs,
- struct netlink_ext_ack *extack)
-{
- u32 *curr_pmask, *curr_pval;
-
- curr_pmask = (u32 *)(pedit_header(&hdrs->masks, hdr_type) + offset);
- curr_pval = (u32 *)(pedit_header(&hdrs->vals, hdr_type) + offset);
-
- if (*curr_pmask & mask) { /* disallow acting twice on the same location */
- NL_SET_ERR_MSG_MOD(extack,
- "curr_pmask and new mask same. Acting twice on same location");
- goto out_err;
- }
-
- *curr_pmask |= mask;
- *curr_pval |= (val & mask);
-
- return 0;
-
-out_err:
- return -EOPNOTSUPP;
-}
-
struct mlx5_fields {
u8 field;
u8 field_bsize;
@@ -2767,26 +2818,23 @@ static int offload_pedit_fields(struct mlx5e_priv *priv,
struct netlink_ext_ack *extack)
{
struct pedit_headers *set_masks, *add_masks, *set_vals, *add_vals;
- int i, action_size, first, last, next_z;
void *headers_c, *headers_v, *action, *vals_p;
u32 *s_masks_p, *a_masks_p, s_mask, a_mask;
struct mlx5e_tc_mod_hdr_acts *mod_acts;
- struct mlx5_fields *f;
unsigned long mask, field_mask;
- int err;
+ int i, first, last, next_z;
+ struct mlx5_fields *f;
u8 cmd;
mod_acts = &parse_attr->mod_hdr_acts;
- headers_c = get_match_headers_criteria(*action_flags, &parse_attr->spec);
- headers_v = get_match_headers_value(*action_flags, &parse_attr->spec);
+ headers_c = mlx5e_get_match_headers_criteria(*action_flags, &parse_attr->spec);
+ headers_v = mlx5e_get_match_headers_value(*action_flags, &parse_attr->spec);
set_masks = &hdrs[0].masks;
add_masks = &hdrs[1].masks;
set_vals = &hdrs[0].vals;
add_vals = &hdrs[1].vals;
- action_size = MLX5_UN_SZ_BYTES(set_add_copy_action_in_auto);
-
for (i = 0; i < ARRAY_SIZE(fields); i++) {
bool skip;
@@ -2854,18 +2902,16 @@ static int offload_pedit_fields(struct mlx5e_priv *priv,
return -EOPNOTSUPP;
}
- err = alloc_mod_hdr_actions(priv->mdev, namespace, mod_acts);
- if (err) {
+ action = mlx5e_mod_hdr_alloc(priv->mdev, namespace, mod_acts);
+ if (IS_ERR(action)) {
NL_SET_ERR_MSG_MOD(extack,
"too many pedit actions, can't offload");
mlx5_core_warn(priv->mdev,
"mlx5: parsed %d pedit actions, can't do more\n",
mod_acts->num_actions);
- return err;
+ return PTR_ERR(action);
}
- action = mod_acts->actions +
- (mod_acts->num_actions * action_size);
MLX5_SET(set_action_in, action, action_type, cmd);
MLX5_SET(set_action_in, action, field, f->field);
@@ -2895,141 +2941,8 @@ static int offload_pedit_fields(struct mlx5e_priv *priv,
return 0;
}
-static int mlx5e_flow_namespace_max_modify_action(struct mlx5_core_dev *mdev,
- int namespace)
-{
- if (namespace == MLX5_FLOW_NAMESPACE_FDB) /* FDB offloading */
- return MLX5_CAP_ESW_FLOWTABLE_FDB(mdev, max_modify_header_actions);
- else /* namespace is MLX5_FLOW_NAMESPACE_KERNEL - NIC offloading */
- return MLX5_CAP_FLOWTABLE_NIC_RX(mdev, max_modify_header_actions);
-}
-
-int alloc_mod_hdr_actions(struct mlx5_core_dev *mdev,
- int namespace,
- struct mlx5e_tc_mod_hdr_acts *mod_hdr_acts)
-{
- int action_size, new_num_actions, max_hw_actions;
- size_t new_sz, old_sz;
- void *ret;
-
- if (mod_hdr_acts->num_actions < mod_hdr_acts->max_actions)
- return 0;
-
- action_size = MLX5_UN_SZ_BYTES(set_add_copy_action_in_auto);
-
- max_hw_actions = mlx5e_flow_namespace_max_modify_action(mdev,
- namespace);
- new_num_actions = min(max_hw_actions,
- mod_hdr_acts->actions ?
- mod_hdr_acts->max_actions * 2 : 1);
- if (mod_hdr_acts->max_actions == new_num_actions)
- return -ENOSPC;
-
- new_sz = action_size * new_num_actions;
- old_sz = mod_hdr_acts->max_actions * action_size;
- ret = krealloc(mod_hdr_acts->actions, new_sz, GFP_KERNEL);
- if (!ret)
- return -ENOMEM;
-
- memset(ret + old_sz, 0, new_sz - old_sz);
- mod_hdr_acts->actions = ret;
- mod_hdr_acts->max_actions = new_num_actions;
-
- return 0;
-}
-
-void dealloc_mod_hdr_actions(struct mlx5e_tc_mod_hdr_acts *mod_hdr_acts)
-{
- kfree(mod_hdr_acts->actions);
- mod_hdr_acts->actions = NULL;
- mod_hdr_acts->num_actions = 0;
- mod_hdr_acts->max_actions = 0;
-}
-
static const struct pedit_headers zero_masks = {};
-static int
-parse_pedit_to_modify_hdr(struct mlx5e_priv *priv,
- const struct flow_action_entry *act, int namespace,
- struct mlx5e_tc_flow_parse_attr *parse_attr,
- struct pedit_headers_action *hdrs,
- struct netlink_ext_ack *extack)
-{
- u8 cmd = (act->id == FLOW_ACTION_MANGLE) ? 0 : 1;
- int err = -EOPNOTSUPP;
- u32 mask, val, offset;
- u8 htype;
-
- htype = act->mangle.htype;
- err = -EOPNOTSUPP; /* can't be all optimistic */
-
- if (htype == FLOW_ACT_MANGLE_UNSPEC) {
- NL_SET_ERR_MSG_MOD(extack, "legacy pedit isn't offloaded");
- goto out_err;
- }
-
- if (!mlx5e_flow_namespace_max_modify_action(priv->mdev, namespace)) {
- NL_SET_ERR_MSG_MOD(extack,
- "The pedit offload action is not supported");
- goto out_err;
- }
-
- mask = act->mangle.mask;
- val = act->mangle.val;
- offset = act->mangle.offset;
-
- err = set_pedit_val(htype, ~mask, val, offset, &hdrs[cmd], extack);
- if (err)
- goto out_err;
-
- hdrs[cmd].pedits++;
-
- return 0;
-out_err:
- return err;
-}
-
-static int
-parse_pedit_to_reformat(const struct flow_action_entry *act,
- struct mlx5e_tc_flow_parse_attr *parse_attr,
- struct netlink_ext_ack *extack)
-{
- u32 mask, val, offset;
- u32 *p;
-
- if (act->id != FLOW_ACTION_MANGLE) {
- NL_SET_ERR_MSG_MOD(extack, "Unsupported action id");
- return -EOPNOTSUPP;
- }
-
- if (act->mangle.htype != FLOW_ACT_MANGLE_HDR_TYPE_ETH) {
- NL_SET_ERR_MSG_MOD(extack, "Only Ethernet modification is supported");
- return -EOPNOTSUPP;
- }
-
- mask = ~act->mangle.mask;
- val = act->mangle.val;
- offset = act->mangle.offset;
- p = (u32 *)&parse_attr->eth;
- *(p + (offset >> 2)) |= (val & mask);
-
- return 0;
-}
-
-static int parse_tc_pedit_action(struct mlx5e_priv *priv,
- const struct flow_action_entry *act, int namespace,
- struct mlx5e_tc_flow_parse_attr *parse_attr,
- struct pedit_headers_action *hdrs,
- struct mlx5e_tc_flow *flow,
- struct netlink_ext_ack *extack)
-{
- if (flow && flow_flag_test(flow, L3_TO_L2_DECAP))
- return parse_pedit_to_reformat(act, parse_attr, extack);
-
- return parse_pedit_to_modify_hdr(priv, act, namespace,
- parse_attr, hdrs, extack);
-}
-
static int alloc_tc_pedit_action(struct mlx5e_priv *priv, int namespace,
struct mlx5e_tc_flow_parse_attr *parse_attr,
struct pedit_headers_action *hdrs,
@@ -3061,39 +2974,10 @@ static int alloc_tc_pedit_action(struct mlx5e_priv *priv, int namespace,
return 0;
out_dealloc_parsed_actions:
- dealloc_mod_hdr_actions(&parse_attr->mod_hdr_acts);
+ mlx5e_mod_hdr_dealloc(&parse_attr->mod_hdr_acts);
return err;
}
-static bool csum_offload_supported(struct mlx5e_priv *priv,
- u32 action,
- u32 update_flags,
- struct netlink_ext_ack *extack)
-{
- u32 prot_flags = TCA_CSUM_UPDATE_FLAG_IPV4HDR | TCA_CSUM_UPDATE_FLAG_TCP |
- TCA_CSUM_UPDATE_FLAG_UDP;
-
- /* The HW recalcs checksums only if re-writing headers */
- if (!(action & MLX5_FLOW_CONTEXT_ACTION_MOD_HDR)) {
- NL_SET_ERR_MSG_MOD(extack,
- "TC csum action is only offloaded with pedit");
- netdev_warn(priv->netdev,
- "TC csum action is only offloaded with pedit\n");
- return false;
- }
-
- if (update_flags & ~prot_flags) {
- NL_SET_ERR_MSG_MOD(extack,
- "can't offload TC csum action for some header/s");
- netdev_warn(priv->netdev,
- "can't offload TC csum action for some header/s - flags %#x\n",
- update_flags);
- return false;
- }
-
- return true;
-}
-
struct ip_ttl_word {
__u8 ttl;
__u8 protocol;
@@ -3216,8 +3100,8 @@ static bool modify_header_match_supported(struct mlx5e_priv *priv,
u8 ip_proto;
int i;
- headers_c = get_match_headers_criteria(actions, spec);
- headers_v = get_match_headers_value(actions, spec);
+ headers_c = mlx5e_get_match_headers_criteria(actions, spec);
+ headers_v = mlx5e_get_match_headers_value(actions, spec);
ethertype = MLX5_GET(fte_match_set_lyr_2_4, headers_v, ethertype);
/* for non-IP we only re-write MACs, so we're okay */
@@ -3324,7 +3208,7 @@ static bool same_port_devs(struct mlx5e_priv *priv, struct mlx5e_priv *peer_priv
return priv->mdev == peer_priv->mdev;
}
-static bool same_hw_devs(struct mlx5e_priv *priv, struct mlx5e_priv *peer_priv)
+bool mlx5e_same_hw_devs(struct mlx5e_priv *priv, struct mlx5e_priv *peer_priv)
{
struct mlx5_core_dev *fmdev, *pmdev;
u64 fsystem_guid, psystem_guid;
@@ -3338,126 +3222,45 @@ static bool same_hw_devs(struct mlx5e_priv *priv, struct mlx5e_priv *peer_priv)
return (fsystem_guid == psystem_guid);
}
-static bool same_vf_reps(struct mlx5e_priv *priv,
- struct net_device *out_dev)
-{
- return mlx5e_eswitch_vf_rep(priv->netdev) &&
- priv->netdev == out_dev;
-}
-
-static int add_vlan_rewrite_action(struct mlx5e_priv *priv, int namespace,
- const struct flow_action_entry *act,
- struct mlx5e_tc_flow_parse_attr *parse_attr,
- struct pedit_headers_action *hdrs,
- u32 *action, struct netlink_ext_ack *extack)
-{
- u16 mask16 = VLAN_VID_MASK;
- u16 val16 = act->vlan.vid & VLAN_VID_MASK;
- const struct flow_action_entry pedit_act = {
- .id = FLOW_ACTION_MANGLE,
- .mangle.htype = FLOW_ACT_MANGLE_HDR_TYPE_ETH,
- .mangle.offset = offsetof(struct vlan_ethhdr, h_vlan_TCI),
- .mangle.mask = ~(u32)be16_to_cpu(*(__be16 *)&mask16),
- .mangle.val = (u32)be16_to_cpu(*(__be16 *)&val16),
- };
- u8 match_prio_mask, match_prio_val;
- void *headers_c, *headers_v;
- int err;
-
- headers_c = get_match_headers_criteria(*action, &parse_attr->spec);
- headers_v = get_match_headers_value(*action, &parse_attr->spec);
-
- if (!(MLX5_GET(fte_match_set_lyr_2_4, headers_c, cvlan_tag) &&
- MLX5_GET(fte_match_set_lyr_2_4, headers_v, cvlan_tag))) {
- NL_SET_ERR_MSG_MOD(extack,
- "VLAN rewrite action must have VLAN protocol match");
- return -EOPNOTSUPP;
- }
-
- match_prio_mask = MLX5_GET(fte_match_set_lyr_2_4, headers_c, first_prio);
- match_prio_val = MLX5_GET(fte_match_set_lyr_2_4, headers_v, first_prio);
- if (act->vlan.prio != (match_prio_val & match_prio_mask)) {
- NL_SET_ERR_MSG_MOD(extack,
- "Changing VLAN prio is not supported");
- return -EOPNOTSUPP;
- }
-
- err = parse_tc_pedit_action(priv, &pedit_act, namespace, parse_attr, hdrs, NULL, extack);
- *action |= MLX5_FLOW_CONTEXT_ACTION_MOD_HDR;
-
- return err;
-}
-
static int
-add_vlan_prio_tag_rewrite_action(struct mlx5e_priv *priv,
- struct mlx5e_tc_flow_parse_attr *parse_attr,
- struct pedit_headers_action *hdrs,
- u32 *action, struct netlink_ext_ack *extack)
+parse_tc_actions(struct mlx5e_tc_act_parse_state *parse_state,
+ struct flow_action *flow_action)
{
- const struct flow_action_entry prio_tag_act = {
- .vlan.vid = 0,
- .vlan.prio =
- MLX5_GET(fte_match_set_lyr_2_4,
- get_match_headers_value(*action,
- &parse_attr->spec),
- first_prio) &
- MLX5_GET(fte_match_set_lyr_2_4,
- get_match_headers_criteria(*action,
- &parse_attr->spec),
- first_prio),
- };
-
- return add_vlan_rewrite_action(priv, MLX5_FLOW_NAMESPACE_FDB,
- &prio_tag_act, parse_attr, hdrs, action,
- extack);
-}
-
-static int validate_goto_chain(struct mlx5e_priv *priv,
- struct mlx5e_tc_flow *flow,
- const struct flow_action_entry *act,
- u32 actions,
- struct netlink_ext_ack *extack)
-{
- bool is_esw = mlx5e_is_eswitch_flow(flow);
+ struct netlink_ext_ack *extack = parse_state->extack;
+ struct mlx5e_tc_flow *flow = parse_state->flow;
struct mlx5_flow_attr *attr = flow->attr;
- bool ft_flow = mlx5e_is_ft_flow(flow);
- u32 dest_chain = act->chain_index;
- struct mlx5_fs_chains *chains;
- struct mlx5_eswitch *esw;
- u32 reformat_and_fwd;
- u32 max_chain;
+ enum mlx5_flow_namespace_type ns_type;
+ struct mlx5e_priv *priv = flow->priv;
+ const struct flow_action_entry *act;
+ struct mlx5e_tc_act *tc_act;
+ int err, i;
- esw = priv->mdev->priv.eswitch;
- chains = is_esw ? esw_chains(esw) : nic_chains(priv);
- max_chain = mlx5_chains_get_chain_range(chains);
- reformat_and_fwd = is_esw ?
- MLX5_CAP_ESW_FLOWTABLE_FDB(priv->mdev, reformat_and_fwd_to_table) :
- MLX5_CAP_FLOWTABLE_NIC_RX(priv->mdev, reformat_and_fwd_to_table);
+ ns_type = mlx5e_get_flow_namespace(flow);
- if (ft_flow) {
- NL_SET_ERR_MSG_MOD(extack, "Goto action is not supported");
- return -EOPNOTSUPP;
+ flow_action_for_each(i, act, flow_action) {
+ tc_act = mlx5e_tc_act_get(act->id, ns_type);
+ if (!tc_act) {
+ NL_SET_ERR_MSG_MOD(extack, "Not implemented offload action");
+ return -EOPNOTSUPP;
+ }
+
+ if (!tc_act->can_offload(parse_state, act, i))
+ return -EOPNOTSUPP;
+
+ err = tc_act->parse_action(parse_state, act, priv, attr);
+ if (err)
+ return err;
}
- if (!mlx5_chains_backwards_supported(chains) &&
- dest_chain <= attr->chain) {
- NL_SET_ERR_MSG_MOD(extack,
- "Goto lower numbered chain isn't supported");
- return -EOPNOTSUPP;
- }
+ flow_action_for_each(i, act, flow_action) {
+ tc_act = mlx5e_tc_act_get(act->id, ns_type);
+ if (!tc_act || !tc_act->post_parse ||
+ !tc_act->can_offload(parse_state, act, i))
+ continue;
- if (dest_chain > max_chain) {
- NL_SET_ERR_MSG_MOD(extack,
- "Requested destination chain is out of supported range");
- return -EOPNOTSUPP;
- }
-
- if (actions & (MLX5_FLOW_CONTEXT_ACTION_PACKET_REFORMAT |
- MLX5_FLOW_CONTEXT_ACTION_DECAP) &&
- !reformat_and_fwd) {
- NL_SET_ERR_MSG_MOD(extack,
- "Goto chain is not allowed if action has reformat or decap");
- return -EOPNOTSUPP;
+ err = tc_act->post_parse(parse_state, priv, attr);
+ if (err)
+ return err;
}
return 0;
@@ -3478,19 +3281,19 @@ actions_prepare_mod_hdr_actions(struct mlx5e_priv *priv,
!hdrs[TCA_PEDIT_KEY_EX_CMD_ADD].pedits)
return 0;
- ns_type = get_flow_name_space(flow);
+ ns_type = mlx5e_get_flow_namespace(flow);
err = alloc_tc_pedit_action(priv, ns_type, parse_attr, hdrs,
&attr->action, extack);
if (err)
return err;
- /* In case all pedit actions are skipped, remove the MOD_HDR flag. */
if (parse_attr->mod_hdr_acts.num_actions > 0)
return 0;
+ /* In case all pedit actions are skipped, remove the MOD_HDR flag. */
attr->action &= ~MLX5_FLOW_CONTEXT_ACTION_MOD_HDR;
- dealloc_mod_hdr_actions(&parse_attr->mod_hdr_acts);
+ mlx5e_mod_hdr_dealloc(&parse_attr->mod_hdr_acts);
if (ns_type != MLX5_FLOW_NAMESPACE_FDB)
return 0;
@@ -3503,19 +3306,9 @@ actions_prepare_mod_hdr_actions(struct mlx5e_priv *priv,
}
static int
-parse_tc_nic_actions(struct mlx5e_priv *priv,
- struct flow_action *flow_action,
- struct mlx5e_tc_flow *flow,
- struct netlink_ext_ack *extack)
+flow_action_supported(struct flow_action *flow_action,
+ struct netlink_ext_ack *extack)
{
- struct mlx5e_tc_flow_parse_attr *parse_attr;
- struct mlx5_flow_attr *attr = flow->attr;
- struct pedit_headers_action hdrs[2] = {};
- const struct flow_action_entry *act;
- struct mlx5_nic_flow_attr *nic_attr;
- u32 action = 0;
- int err, i;
-
if (!flow_action_has_entries(flow_action)) {
NL_SET_ERR_MSG_MOD(extack, "Flow action doesn't have any entries");
return -EINVAL;
@@ -3527,108 +3320,35 @@ parse_tc_nic_actions(struct mlx5e_priv *priv,
return -EOPNOTSUPP;
}
- nic_attr = attr->nic_attr;
- nic_attr->flow_tag = MLX5_FS_DEFAULT_FLOW_TAG;
+ return 0;
+}
+
+static int
+parse_tc_nic_actions(struct mlx5e_priv *priv,
+ struct flow_action *flow_action,
+ struct mlx5e_tc_flow *flow,
+ struct netlink_ext_ack *extack)
+{
+ struct mlx5e_tc_act_parse_state *parse_state;
+ struct mlx5e_tc_flow_parse_attr *parse_attr;
+ struct mlx5_flow_attr *attr = flow->attr;
+ struct pedit_headers_action *hdrs;
+ int err;
+
+ err = flow_action_supported(flow_action, extack);
+ if (err)
+ return err;
+
+ attr->nic_attr->flow_tag = MLX5_FS_DEFAULT_FLOW_TAG;
parse_attr = attr->parse_attr;
+ parse_state = &parse_attr->parse_state;
+ mlx5e_tc_act_init_parse_state(parse_state, flow, flow_action, extack);
+ parse_state->ct_priv = get_ct_priv(priv);
+ hdrs = parse_state->hdrs;
- flow_action_for_each(i, act, flow_action) {
- switch (act->id) {
- case FLOW_ACTION_ACCEPT:
- action |= MLX5_FLOW_CONTEXT_ACTION_FWD_DEST |
- MLX5_FLOW_CONTEXT_ACTION_COUNT;
- break;
- case FLOW_ACTION_DROP:
- action |= MLX5_FLOW_CONTEXT_ACTION_DROP |
- MLX5_FLOW_CONTEXT_ACTION_COUNT;
- break;
- case FLOW_ACTION_MANGLE:
- case FLOW_ACTION_ADD:
- err = parse_tc_pedit_action(priv, act, MLX5_FLOW_NAMESPACE_KERNEL,
- parse_attr, hdrs, NULL, extack);
- if (err)
- return err;
-
- action |= MLX5_FLOW_CONTEXT_ACTION_MOD_HDR;
- break;
- case FLOW_ACTION_VLAN_MANGLE:
- err = add_vlan_rewrite_action(priv,
- MLX5_FLOW_NAMESPACE_KERNEL,
- act, parse_attr, hdrs,
- &action, extack);
- if (err)
- return err;
-
- break;
- case FLOW_ACTION_CSUM:
- if (csum_offload_supported(priv, action,
- act->csum_flags,
- extack))
- break;
-
- return -EOPNOTSUPP;
- case FLOW_ACTION_REDIRECT: {
- struct net_device *peer_dev = act->dev;
-
- if (priv->netdev->netdev_ops == peer_dev->netdev_ops &&
- same_hw_devs(priv, netdev_priv(peer_dev))) {
- parse_attr->mirred_ifindex[0] = peer_dev->ifindex;
- flow_flag_set(flow, HAIRPIN);
- action |= MLX5_FLOW_CONTEXT_ACTION_FWD_DEST |
- MLX5_FLOW_CONTEXT_ACTION_COUNT;
- } else {
- NL_SET_ERR_MSG_MOD(extack,
- "device is not on same HW, can't offload");
- netdev_warn(priv->netdev, "device %s not on same HW, can't offload\n",
- peer_dev->name);
- return -EOPNOTSUPP;
- }
- }
- break;
- case FLOW_ACTION_MARK: {
- u32 mark = act->mark;
-
- if (mark & ~MLX5E_TC_FLOW_ID_MASK) {
- NL_SET_ERR_MSG_MOD(extack,
- "Bad flow mark - only 16 bit is supported");
- return -EOPNOTSUPP;
- }
-
- nic_attr->flow_tag = mark;
- action |= MLX5_FLOW_CONTEXT_ACTION_FWD_DEST;
- }
- break;
- case FLOW_ACTION_GOTO:
- err = validate_goto_chain(priv, flow, act, action,
- extack);
- if (err)
- return err;
-
- action |= MLX5_FLOW_CONTEXT_ACTION_FWD_DEST |
- MLX5_FLOW_CONTEXT_ACTION_COUNT;
- attr->dest_chain = act->chain_index;
- break;
- case FLOW_ACTION_CT:
- err = mlx5_tc_ct_parse_action(get_ct_priv(priv), attr,
- &parse_attr->mod_hdr_acts,
- act, extack);
- if (err)
- return err;
-
- flow_flag_set(flow, CT);
- break;
- default:
- NL_SET_ERR_MSG_MOD(extack,
- "The offload action is not supported in NIC action");
- return -EOPNOTSUPP;
- }
- }
-
- attr->action = action;
-
- if (attr->dest_chain && parse_attr->mirred_ifindex[0]) {
- NL_SET_ERR_MSG(extack, "Mirroring goto chain rules isn't supported");
- return -EOPNOTSUPP;
- }
+ err = parse_tc_actions(parse_state, flow_action);
+ if (err)
+ return err;
err = actions_prepare_mod_hdr_actions(priv, flow, attr, hdrs, extack);
if (err)
@@ -3650,147 +3370,7 @@ static bool is_merged_eswitch_vfs(struct mlx5e_priv *priv,
return (MLX5_CAP_ESW(priv->mdev, merged_eswitch) &&
mlx5e_eswitch_vf_rep(priv->netdev) &&
mlx5e_eswitch_vf_rep(peer_netdev) &&
- same_hw_devs(priv, peer_priv));
-}
-
-static int parse_tc_vlan_action(struct mlx5e_priv *priv,
- const struct flow_action_entry *act,
- struct mlx5_esw_flow_attr *attr,
- u32 *action,
- struct netlink_ext_ack *extack)
-{
- u8 vlan_idx = attr->total_vlan;
-
- if (vlan_idx >= MLX5_FS_VLAN_DEPTH) {
- NL_SET_ERR_MSG_MOD(extack, "Total vlans used is greater than supported");
- return -EOPNOTSUPP;
- }
-
- switch (act->id) {
- case FLOW_ACTION_VLAN_POP:
- if (vlan_idx) {
- if (!mlx5_eswitch_vlan_actions_supported(priv->mdev,
- MLX5_FS_VLAN_DEPTH)) {
- NL_SET_ERR_MSG_MOD(extack,
- "vlan pop action is not supported");
- return -EOPNOTSUPP;
- }
-
- *action |= MLX5_FLOW_CONTEXT_ACTION_VLAN_POP_2;
- } else {
- *action |= MLX5_FLOW_CONTEXT_ACTION_VLAN_POP;
- }
- break;
- case FLOW_ACTION_VLAN_PUSH:
- attr->vlan_vid[vlan_idx] = act->vlan.vid;
- attr->vlan_prio[vlan_idx] = act->vlan.prio;
- attr->vlan_proto[vlan_idx] = act->vlan.proto;
- if (!attr->vlan_proto[vlan_idx])
- attr->vlan_proto[vlan_idx] = htons(ETH_P_8021Q);
-
- if (vlan_idx) {
- if (!mlx5_eswitch_vlan_actions_supported(priv->mdev,
- MLX5_FS_VLAN_DEPTH)) {
- NL_SET_ERR_MSG_MOD(extack,
- "vlan push action is not supported for vlan depth > 1");
- return -EOPNOTSUPP;
- }
-
- *action |= MLX5_FLOW_CONTEXT_ACTION_VLAN_PUSH_2;
- } else {
- if (!mlx5_eswitch_vlan_actions_supported(priv->mdev, 1) &&
- (act->vlan.proto != htons(ETH_P_8021Q) ||
- act->vlan.prio)) {
- NL_SET_ERR_MSG_MOD(extack,
- "vlan push action is not supported");
- return -EOPNOTSUPP;
- }
-
- *action |= MLX5_FLOW_CONTEXT_ACTION_VLAN_PUSH;
- }
- break;
- default:
- NL_SET_ERR_MSG_MOD(extack, "Unexpected action id for VLAN");
- return -EINVAL;
- }
-
- attr->total_vlan = vlan_idx + 1;
-
- return 0;
-}
-
-static struct net_device *get_fdb_out_dev(struct net_device *uplink_dev,
- struct net_device *out_dev)
-{
- struct net_device *fdb_out_dev = out_dev;
- struct net_device *uplink_upper;
-
- rcu_read_lock();
- uplink_upper = netdev_master_upper_dev_get_rcu(uplink_dev);
- if (uplink_upper && netif_is_lag_master(uplink_upper) &&
- uplink_upper == out_dev) {
- fdb_out_dev = uplink_dev;
- } else if (netif_is_lag_master(out_dev)) {
- fdb_out_dev = bond_option_active_slave_get_rcu(netdev_priv(out_dev));
- if (fdb_out_dev &&
- (!mlx5e_eswitch_rep(fdb_out_dev) ||
- !netdev_port_same_parent_id(fdb_out_dev, uplink_dev)))
- fdb_out_dev = NULL;
- }
- rcu_read_unlock();
- return fdb_out_dev;
-}
-
-static int add_vlan_push_action(struct mlx5e_priv *priv,
- struct mlx5_flow_attr *attr,
- struct net_device **out_dev,
- u32 *action,
- struct netlink_ext_ack *extack)
-{
- struct net_device *vlan_dev = *out_dev;
- struct flow_action_entry vlan_act = {
- .id = FLOW_ACTION_VLAN_PUSH,
- .vlan.vid = vlan_dev_vlan_id(vlan_dev),
- .vlan.proto = vlan_dev_vlan_proto(vlan_dev),
- .vlan.prio = 0,
- };
- int err;
-
- err = parse_tc_vlan_action(priv, &vlan_act, attr->esw_attr, action, extack);
- if (err)
- return err;
-
- rcu_read_lock();
- *out_dev = dev_get_by_index_rcu(dev_net(vlan_dev), dev_get_iflink(vlan_dev));
- rcu_read_unlock();
- if (!*out_dev)
- return -ENODEV;
-
- if (is_vlan_dev(*out_dev))
- err = add_vlan_push_action(priv, attr, out_dev, action, extack);
-
- return err;
-}
-
-static int add_vlan_pop_action(struct mlx5e_priv *priv,
- struct mlx5_flow_attr *attr,
- u32 *action,
- struct netlink_ext_ack *extack)
-{
- struct flow_action_entry vlan_act = {
- .id = FLOW_ACTION_VLAN_POP,
- };
- int nest_level, err = 0;
-
- nest_level = attr->parse_attr->filter_dev->lower_level -
- priv->netdev->lower_level;
- while (nest_level--) {
- err = parse_tc_vlan_action(priv, &vlan_act, attr->esw_attr, action, extack);
- if (err)
- return err;
- }
-
- return err;
+ mlx5e_same_hw_devs(priv, peer_priv));
}
static bool same_hw_reps(struct mlx5e_priv *priv,
@@ -3802,7 +3382,7 @@ static bool same_hw_reps(struct mlx5e_priv *priv,
return mlx5e_eswitch_rep(priv->netdev) &&
mlx5e_eswitch_rep(peer_netdev) &&
- same_hw_devs(priv, peer_priv);
+ mlx5e_same_hw_devs(priv, peer_priv);
}
static bool is_lag_dev(struct mlx5e_priv *priv,
@@ -3826,66 +3406,6 @@ bool mlx5e_is_valid_eswitch_fwd_dev(struct mlx5e_priv *priv,
same_port_devs(priv, netdev_priv(out_dev));
}
-static bool is_duplicated_output_device(struct net_device *dev,
- struct net_device *out_dev,
- int *ifindexes, int if_count,
- struct netlink_ext_ack *extack)
-{
- int i;
-
- for (i = 0; i < if_count; i++) {
- if (ifindexes[i] == out_dev->ifindex) {
- NL_SET_ERR_MSG_MOD(extack,
- "can't duplicate output to same device");
- netdev_err(dev, "can't duplicate output to same device: %s\n",
- out_dev->name);
- return true;
- }
- }
-
- return false;
-}
-
-static int verify_uplink_forwarding(struct mlx5e_priv *priv,
- struct mlx5e_tc_flow *flow,
- struct net_device *out_dev,
- struct netlink_ext_ack *extack)
-{
- struct mlx5_esw_flow_attr *attr = flow->attr->esw_attr;
- struct mlx5_eswitch *esw = priv->mdev->priv.eswitch;
- struct mlx5e_rep_priv *rep_priv;
-
- /* Forwarding non encapsulated traffic between
- * uplink ports is allowed only if
- * termination_table_raw_traffic cap is set.
- *
- * Input vport was stored attr->in_rep.
- * In LAG case, *priv* is the private data of
- * uplink which may be not the input vport.
- */
- rep_priv = mlx5e_rep_to_rep_priv(attr->in_rep);
-
- if (!(mlx5e_eswitch_uplink_rep(rep_priv->netdev) &&
- mlx5e_eswitch_uplink_rep(out_dev)))
- return 0;
-
- if (!MLX5_CAP_ESW_FLOWTABLE_FDB(esw->dev,
- termination_table_raw_traffic)) {
- NL_SET_ERR_MSG_MOD(extack,
- "devices are both uplink, can't offload forwarding");
- pr_err("devices %s %s are both uplink, can't offload forwarding\n",
- priv->netdev->name, out_dev->name);
- return -EOPNOTSUPP;
- } else if (out_dev != rep_priv->netdev) {
- NL_SET_ERR_MSG_MOD(extack,
- "devices are not the same uplink, can't offload forwarding");
- pr_err("devices %s %s are both uplink but not the same, can't offload forwarding\n",
- priv->netdev->name, out_dev->name);
- return -EOPNOTSUPP;
- }
- return 0;
-}
-
int mlx5e_set_fwd_to_int_port_actions(struct mlx5e_priv *priv,
struct mlx5_flow_attr *attr,
int ifindex,
@@ -3925,386 +3445,33 @@ int mlx5e_set_fwd_to_int_port_actions(struct mlx5e_priv *priv,
return 0;
}
-static int parse_tc_fdb_actions(struct mlx5e_priv *priv,
- struct flow_action *flow_action,
- struct mlx5e_tc_flow *flow,
- struct netlink_ext_ack *extack)
+static int
+parse_tc_fdb_actions(struct mlx5e_priv *priv,
+ struct flow_action *flow_action,
+ struct mlx5e_tc_flow *flow,
+ struct netlink_ext_ack *extack)
{
- struct pedit_headers_action hdrs[2] = {};
- struct mlx5_eswitch *esw = priv->mdev->priv.eswitch;
+ struct mlx5e_tc_act_parse_state *parse_state;
struct mlx5e_tc_flow_parse_attr *parse_attr;
- struct mlx5e_rep_priv *rpriv = priv->ppriv;
- struct mlx5e_sample_attr sample_attr = {};
- const struct ip_tunnel_info *info = NULL;
struct mlx5_flow_attr *attr = flow->attr;
- int ifindexes[MLX5_MAX_FLOW_FWD_VPORTS];
- bool ft_flow = mlx5e_is_ft_flow(flow);
- const struct flow_action_entry *act;
struct mlx5_esw_flow_attr *esw_attr;
- bool encap = false, decap = false;
- u32 action = attr->action;
- int err, i, if_count = 0;
- bool ptype_host = false;
- bool mpls_push = false;
+ struct pedit_headers_action *hdrs;
+ int err;
- if (!flow_action_has_entries(flow_action)) {
- NL_SET_ERR_MSG_MOD(extack, "Flow action doesn't have any entries");
- return -EINVAL;
- }
-
- if (!flow_action_hw_stats_check(flow_action, extack,
- FLOW_ACTION_HW_STATS_DELAYED_BIT)) {
- NL_SET_ERR_MSG_MOD(extack, "Flow action HW stats type is not supported");
- return -EOPNOTSUPP;
- }
+ err = flow_action_supported(flow_action, extack);
+ if (err)
+ return err;
esw_attr = attr->esw_attr;
parse_attr = attr->parse_attr;
+ parse_state = &parse_attr->parse_state;
+ mlx5e_tc_act_init_parse_state(parse_state, flow, flow_action, extack);
+ parse_state->ct_priv = get_ct_priv(priv);
+ hdrs = parse_state->hdrs;
- flow_action_for_each(i, act, flow_action) {
- switch (act->id) {
- case FLOW_ACTION_ACCEPT:
- action |= MLX5_FLOW_CONTEXT_ACTION_FWD_DEST |
- MLX5_FLOW_CONTEXT_ACTION_COUNT;
- attr->flags |= MLX5_ESW_ATTR_FLAG_ACCEPT;
- break;
- case FLOW_ACTION_PTYPE:
- if (act->ptype != PACKET_HOST) {
- NL_SET_ERR_MSG_MOD(extack,
- "skbedit ptype is only supported with type host");
- return -EOPNOTSUPP;
- }
-
- ptype_host = true;
- break;
- case FLOW_ACTION_DROP:
- action |= MLX5_FLOW_CONTEXT_ACTION_DROP |
- MLX5_FLOW_CONTEXT_ACTION_COUNT;
- break;
- case FLOW_ACTION_TRAP:
- if (!flow_offload_has_one_action(flow_action)) {
- NL_SET_ERR_MSG_MOD(extack,
- "action trap is supported as a sole action only");
- return -EOPNOTSUPP;
- }
- action |= (MLX5_FLOW_CONTEXT_ACTION_FWD_DEST |
- MLX5_FLOW_CONTEXT_ACTION_COUNT);
- attr->flags |= MLX5_ESW_ATTR_FLAG_SLOW_PATH;
- break;
- case FLOW_ACTION_MPLS_PUSH:
- if (!MLX5_CAP_ESW_FLOWTABLE_FDB(priv->mdev,
- reformat_l2_to_l3_tunnel) ||
- act->mpls_push.proto != htons(ETH_P_MPLS_UC)) {
- NL_SET_ERR_MSG_MOD(extack,
- "mpls push is supported only for mpls_uc protocol");
- return -EOPNOTSUPP;
- }
- mpls_push = true;
- break;
- case FLOW_ACTION_MPLS_POP:
- /* we only support mpls pop if it is the first action
- * and the filter net device is bareudp. Subsequent
- * actions can be pedit and the last can be mirred
- * egress redirect.
- */
- if (i) {
- NL_SET_ERR_MSG_MOD(extack,
- "mpls pop supported only as first action");
- return -EOPNOTSUPP;
- }
- if (!netif_is_bareudp(parse_attr->filter_dev)) {
- NL_SET_ERR_MSG_MOD(extack,
- "mpls pop supported only on bareudp devices");
- return -EOPNOTSUPP;
- }
-
- parse_attr->eth.h_proto = act->mpls_pop.proto;
- action |= MLX5_FLOW_CONTEXT_ACTION_PACKET_REFORMAT;
- flow_flag_set(flow, L3_TO_L2_DECAP);
- break;
- case FLOW_ACTION_MANGLE:
- case FLOW_ACTION_ADD:
- err = parse_tc_pedit_action(priv, act, MLX5_FLOW_NAMESPACE_FDB,
- parse_attr, hdrs, flow, extack);
- if (err)
- return err;
-
- if (!flow_flag_test(flow, L3_TO_L2_DECAP)) {
- action |= MLX5_FLOW_CONTEXT_ACTION_MOD_HDR;
- esw_attr->split_count = esw_attr->out_count;
- }
- break;
- case FLOW_ACTION_CSUM:
- if (csum_offload_supported(priv, action,
- act->csum_flags, extack))
- break;
-
- return -EOPNOTSUPP;
- case FLOW_ACTION_REDIRECT_INGRESS: {
- struct net_device *out_dev;
-
- out_dev = act->dev;
- if (!out_dev)
- return -EOPNOTSUPP;
-
- if (!netif_is_ovs_master(out_dev)) {
- NL_SET_ERR_MSG_MOD(extack,
- "redirect to ingress is supported only for OVS internal ports");
- return -EOPNOTSUPP;
- }
-
- if (netif_is_ovs_master(parse_attr->filter_dev)) {
- NL_SET_ERR_MSG_MOD(extack,
- "redirect to ingress is not supported from internal port");
- return -EOPNOTSUPP;
- }
-
- if (!ptype_host) {
- NL_SET_ERR_MSG_MOD(extack,
- "redirect to int port ingress requires ptype=host action");
- return -EOPNOTSUPP;
- }
-
- if (esw_attr->out_count) {
- NL_SET_ERR_MSG_MOD(extack,
- "redirect to int port ingress is supported only as single destination");
- return -EOPNOTSUPP;
- }
-
- action |= MLX5_FLOW_CONTEXT_ACTION_FWD_DEST |
- MLX5_FLOW_CONTEXT_ACTION_COUNT;
-
- err = mlx5e_set_fwd_to_int_port_actions(priv, attr, out_dev->ifindex,
- MLX5E_TC_INT_PORT_INGRESS,
- &action, esw_attr->out_count);
- if (err)
- return err;
-
- esw_attr->out_count++;
-
- break;
- }
- case FLOW_ACTION_REDIRECT:
- case FLOW_ACTION_MIRRED: {
- struct mlx5e_priv *out_priv;
- struct net_device *out_dev;
-
- out_dev = act->dev;
- if (!out_dev) {
- /* out_dev is NULL when filters with
- * non-existing mirred device are replayed to
- * the driver.
- */
- return -EINVAL;
- }
-
- if (mpls_push && !netif_is_bareudp(out_dev)) {
- NL_SET_ERR_MSG_MOD(extack,
- "mpls is supported only through a bareudp device");
- return -EOPNOTSUPP;
- }
-
- if (ft_flow && out_dev == priv->netdev) {
- /* Ignore forward to self rules generated
- * by adding both mlx5 devs to the flow table
- * block on a normal nft offload setup.
- */
- return -EOPNOTSUPP;
- }
-
- if (esw_attr->out_count >= MLX5_MAX_FLOW_FWD_VPORTS) {
- NL_SET_ERR_MSG_MOD(extack,
- "can't support more output ports, can't offload forwarding");
- netdev_warn(priv->netdev,
- "can't support more than %d output ports, can't offload forwarding\n",
- esw_attr->out_count);
- return -EOPNOTSUPP;
- }
-
- action |= MLX5_FLOW_CONTEXT_ACTION_FWD_DEST |
- MLX5_FLOW_CONTEXT_ACTION_COUNT;
- if (encap) {
- parse_attr->mirred_ifindex[esw_attr->out_count] =
- out_dev->ifindex;
- parse_attr->tun_info[esw_attr->out_count] =
- mlx5e_dup_tun_info(info);
- if (!parse_attr->tun_info[esw_attr->out_count])
- return -ENOMEM;
- encap = false;
- esw_attr->dests[esw_attr->out_count].flags |=
- MLX5_ESW_DEST_ENCAP;
- esw_attr->out_count++;
- /* attr->dests[].rep is resolved when we
- * handle encap
- */
- } else if (netdev_port_same_parent_id(priv->netdev, out_dev)) {
- struct mlx5_eswitch *esw = priv->mdev->priv.eswitch;
- struct net_device *uplink_dev = mlx5_eswitch_uplink_get_proto_dev(esw, REP_ETH);
-
- if (is_duplicated_output_device(priv->netdev,
- out_dev,
- ifindexes,
- if_count,
- extack))
- return -EOPNOTSUPP;
-
- ifindexes[if_count] = out_dev->ifindex;
- if_count++;
-
- out_dev = get_fdb_out_dev(uplink_dev, out_dev);
- if (!out_dev)
- return -ENODEV;
-
- if (is_vlan_dev(out_dev)) {
- err = add_vlan_push_action(priv, attr,
- &out_dev,
- &action, extack);
- if (err)
- return err;
- }
-
- if (is_vlan_dev(parse_attr->filter_dev)) {
- err = add_vlan_pop_action(priv, attr,
- &action, extack);
- if (err)
- return err;
- }
-
- if (netif_is_macvlan(out_dev))
- out_dev = macvlan_dev_real_dev(out_dev);
-
- err = verify_uplink_forwarding(priv, flow, out_dev, extack);
- if (err)
- return err;
-
- if (!mlx5e_is_valid_eswitch_fwd_dev(priv, out_dev)) {
- NL_SET_ERR_MSG_MOD(extack,
- "devices are not on same switch HW, can't offload forwarding");
- return -EOPNOTSUPP;
- }
-
- if (same_vf_reps(priv, out_dev)) {
- NL_SET_ERR_MSG_MOD(extack,
- "can't forward from a VF to itself");
- return -EOPNOTSUPP;
- }
-
- out_priv = netdev_priv(out_dev);
- rpriv = out_priv->ppriv;
- esw_attr->dests[esw_attr->out_count].rep = rpriv->rep;
- esw_attr->dests[esw_attr->out_count].mdev = out_priv->mdev;
- esw_attr->out_count++;
- } else if (netif_is_ovs_master(out_dev)) {
- err = mlx5e_set_fwd_to_int_port_actions(priv, attr,
- out_dev->ifindex,
- MLX5E_TC_INT_PORT_EGRESS,
- &action,
- esw_attr->out_count);
- if (err)
- return err;
-
- esw_attr->out_count++;
- } else if (parse_attr->filter_dev != priv->netdev) {
- /* All mlx5 devices are called to configure
- * high level device filters. Therefore, the
- * *attempt* to install a filter on invalid
- * eswitch should not trigger an explicit error
- */
- return -EINVAL;
- } else {
- NL_SET_ERR_MSG_MOD(extack,
- "devices are not on same switch HW, can't offload forwarding");
- netdev_warn(priv->netdev,
- "devices %s %s not on same switch HW, can't offload forwarding\n",
- priv->netdev->name,
- out_dev->name);
- return -EOPNOTSUPP;
- }
- }
- break;
- case FLOW_ACTION_TUNNEL_ENCAP:
- info = act->tunnel;
- if (info) {
- encap = true;
- } else {
- NL_SET_ERR_MSG_MOD(extack,
- "Zero tunnel attributes is not supported");
- return -EOPNOTSUPP;
- }
-
- break;
- case FLOW_ACTION_VLAN_PUSH:
- case FLOW_ACTION_VLAN_POP:
- if (act->id == FLOW_ACTION_VLAN_PUSH &&
- (action & MLX5_FLOW_CONTEXT_ACTION_VLAN_POP)) {
- /* Replace vlan pop+push with vlan modify */
- action &= ~MLX5_FLOW_CONTEXT_ACTION_VLAN_POP;
- err = add_vlan_rewrite_action(priv,
- MLX5_FLOW_NAMESPACE_FDB,
- act, parse_attr, hdrs,
- &action, extack);
- } else {
- err = parse_tc_vlan_action(priv, act, esw_attr, &action, extack);
- }
- if (err)
- return err;
-
- esw_attr->split_count = esw_attr->out_count;
- break;
- case FLOW_ACTION_VLAN_MANGLE:
- err = add_vlan_rewrite_action(priv,
- MLX5_FLOW_NAMESPACE_FDB,
- act, parse_attr, hdrs,
- &action, extack);
- if (err)
- return err;
-
- esw_attr->split_count = esw_attr->out_count;
- break;
- case FLOW_ACTION_TUNNEL_DECAP:
- decap = true;
- break;
- case FLOW_ACTION_GOTO:
- err = validate_goto_chain(priv, flow, act, action,
- extack);
- if (err)
- return err;
-
- action |= MLX5_FLOW_CONTEXT_ACTION_FWD_DEST |
- MLX5_FLOW_CONTEXT_ACTION_COUNT;
- attr->dest_chain = act->chain_index;
- break;
- case FLOW_ACTION_CT:
- if (flow_flag_test(flow, SAMPLE)) {
- NL_SET_ERR_MSG_MOD(extack, "Sample action with connection tracking is not supported");
- return -EOPNOTSUPP;
- }
- err = mlx5_tc_ct_parse_action(get_ct_priv(priv), attr,
- &parse_attr->mod_hdr_acts,
- act, extack);
- if (err)
- return err;
-
- flow_flag_set(flow, CT);
- esw_attr->split_count = esw_attr->out_count;
- break;
- case FLOW_ACTION_SAMPLE:
- if (flow_flag_test(flow, CT)) {
- NL_SET_ERR_MSG_MOD(extack, "Sample action with connection tracking is not supported");
- return -EOPNOTSUPP;
- }
- sample_attr.rate = act->sample.rate;
- sample_attr.group_num = act->sample.psample_group->group_num;
- if (act->sample.truncate)
- sample_attr.trunc_size = act->sample.trunc_size;
- flow_flag_set(flow, SAMPLE);
- break;
- default:
- NL_SET_ERR_MSG_MOD(extack,
- "The offload action is not supported in FDB action");
- return -EOPNOTSUPP;
- }
- }
+ err = parse_tc_actions(parse_state, flow_action);
+ if (err)
+ return err;
/* Forward to/from internal port can only have 1 dest */
if ((netif_is_ovs_master(parse_attr->filter_dev) || esw_attr->dest_int_port) &&
@@ -4314,23 +3481,6 @@ static int parse_tc_fdb_actions(struct mlx5e_priv *priv,
return -EOPNOTSUPP;
}
- /* always set IP version for indirect table handling */
- attr->ip_version = mlx5e_tc_get_ip_version(&parse_attr->spec, true);
-
- if (MLX5_CAP_GEN(esw->dev, prio_tag_required) &&
- action & MLX5_FLOW_CONTEXT_ACTION_VLAN_POP) {
- /* For prio tag mode, replace vlan pop with rewrite vlan prio
- * tag rewrite.
- */
- action &= ~MLX5_FLOW_CONTEXT_ACTION_VLAN_POP;
- err = add_vlan_prio_tag_rewrite_action(priv, parse_attr, hdrs,
- &action, extack);
- if (err)
- return err;
- }
-
- attr->action = action;
-
err = actions_prepare_mod_hdr_actions(priv, flow, attr, hdrs, extack);
if (err)
return err;
@@ -4338,30 +3488,6 @@ static int parse_tc_fdb_actions(struct mlx5e_priv *priv,
if (!actions_match_supported(priv, flow_action, parse_attr, flow, extack))
return -EOPNOTSUPP;
- if (attr->dest_chain && decap) {
- /* It can be supported if we'll create a mapping for
- * the tunnel device only (without tunnel), and set
- * this tunnel id with this decap flow.
- *
- * On restore (miss), we'll just set this saved tunnel
- * device.
- */
-
- NL_SET_ERR_MSG(extack, "Decap with goto isn't supported");
- netdev_warn(priv->netdev, "Decap with goto isn't supported");
- return -EOPNOTSUPP;
- }
-
- /* Allocate sample attribute only when there is a sample action and
- * no errors after parsing.
- */
- if (flow_flag_test(flow, SAMPLE)) {
- attr->sample_attr = kzalloc(sizeof(*attr->sample_attr), GFP_KERNEL);
- if (!attr->sample_attr)
- return -ENOMEM;
- *attr->sample_attr = sample_attr;
- }
-
return 0;
}
@@ -4458,7 +3584,7 @@ mlx5e_alloc_flow(struct mlx5e_priv *priv, int attr_size,
flow->cookie = f->cookie;
flow->priv = priv;
- attr = mlx5_alloc_flow_attr(get_flow_name_space(flow));
+ attr = mlx5_alloc_flow_attr(mlx5e_get_flow_namespace(flow));
if (!attr)
goto err_free;
@@ -4553,6 +3679,9 @@ __mlx5e_add_fdb_flow(struct mlx5e_priv *priv,
if (err)
goto err_free;
+ /* always set IP version for indirect table handling */
+ flow->attr->ip_version = mlx5e_tc_get_ip_version(&parse_attr->spec, true);
+
err = parse_tc_fdb_actions(priv, &rule->action, flow, extack);
if (err)
goto err_free;
@@ -4714,7 +3843,7 @@ mlx5e_add_nic_flow(struct mlx5e_priv *priv,
err_free:
flow_flag_set(flow, FAILED);
- dealloc_mod_hdr_actions(&parse_attr->mod_hdr_acts);
+ mlx5e_mod_hdr_dealloc(&parse_attr->mod_hdr_acts);
mlx5e_flow_put(priv, flow);
out:
return err;
@@ -5014,14 +4143,8 @@ static int scan_tc_matchall_fdb_actions(struct mlx5e_priv *priv,
int mlx5e_tc_configure_matchall(struct mlx5e_priv *priv,
struct tc_cls_matchall_offload *ma)
{
- struct mlx5_eswitch *esw = priv->mdev->priv.eswitch;
struct netlink_ext_ack *extack = ma->common.extack;
- if (!mlx5_esw_qos_enabled(esw)) {
- NL_SET_ERR_MSG_MOD(extack, "QoS is not supported on this device");
- return -EOPNOTSUPP;
- }
-
if (ma->common.prio != 1) {
NL_SET_ERR_MSG_MOD(extack, "only priority 1 is supported");
return -EINVAL;
@@ -5063,7 +4186,7 @@ static void mlx5e_tc_hairpin_update_dead_peer(struct mlx5e_priv *priv,
u16 peer_vhca_id;
int bkt;
- if (!same_hw_devs(priv, peer_priv))
+ if (!mlx5e_same_hw_devs(priv, peer_priv))
return;
peer_vhca_id = MLX5_CAP_GEN(peer_mdev, vhca_id);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_tc.h b/drivers/net/ethernet/mellanox/mlx5/core/en_tc.h
index fdb2227..5ffae9b 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_tc.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_tc.h
@@ -151,7 +151,6 @@ enum {
int mlx5e_tc_esw_init(struct rhashtable *tc_ht);
void mlx5e_tc_esw_cleanup(struct rhashtable *tc_ht);
-bool mlx5e_is_eswitch_flow(struct mlx5e_tc_flow *flow);
int mlx5e_configure_flower(struct net_device *dev, struct mlx5e_priv *priv,
struct flow_cls_offload *f, unsigned long flags);
@@ -247,11 +246,6 @@ int mlx5e_tc_add_flow_mod_hdr(struct mlx5e_priv *priv,
struct mlx5e_tc_flow_parse_attr *parse_attr,
struct mlx5e_tc_flow *flow);
-int alloc_mod_hdr_actions(struct mlx5_core_dev *mdev,
- int namespace,
- struct mlx5e_tc_mod_hdr_acts *mod_hdr_acts);
-void dealloc_mod_hdr_actions(struct mlx5e_tc_mod_hdr_acts *mod_hdr_acts);
-
struct mlx5e_tc_flow;
u32 mlx5e_tc_get_flow_tun_id(struct mlx5e_tc_flow *flow);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eq.c b/drivers/net/ethernet/mellanox/mlx5/core/eq.c
index 792e0d6..48a45aa 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eq.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eq.c
@@ -19,6 +19,7 @@
#include "lib/clock.h"
#include "diag/fw_tracer.h"
#include "mlx5_irq.h"
+#include "devlink.h"
enum {
MLX5_EQE_OWNER_INIT_VAL = 0x1,
@@ -58,6 +59,8 @@ struct mlx5_eq_table {
struct mutex lock; /* sync async eqs creations */
int num_comp_eqs;
struct mlx5_irq_table *irq_table;
+ struct mlx5_irq **comp_irqs;
+ struct mlx5_irq *ctrl_irq;
#ifdef CONFIG_RFS_ACCEL
struct cpu_rmap *rmap;
#endif
@@ -265,8 +268,8 @@ create_map_eq(struct mlx5_core_dev *dev, struct mlx5_eq *eq,
u32 out[MLX5_ST_SZ_DW(create_eq_out)] = {0};
u8 log_eq_stride = ilog2(MLX5_EQE_SIZE);
struct mlx5_priv *priv = &dev->priv;
- u16 vecidx = param->irq_index;
__be64 *pas;
+ u16 vecidx;
void *eqc;
int inlen;
u32 *in;
@@ -288,20 +291,16 @@ create_map_eq(struct mlx5_core_dev *dev, struct mlx5_eq *eq,
mlx5_init_fbc(eq->frag_buf.frags, log_eq_stride, log_eq_size, &eq->fbc);
init_eq_buf(eq);
- eq->irq = mlx5_irq_request(dev, vecidx, param->affinity);
- if (IS_ERR(eq->irq)) {
- err = PTR_ERR(eq->irq);
- goto err_buf;
- }
-
+ eq->irq = param->irq;
vecidx = mlx5_irq_get_index(eq->irq);
+
inlen = MLX5_ST_SZ_BYTES(create_eq_in) +
MLX5_FLD_SZ_BYTES(create_eq_in, pas[0]) * eq->frag_buf.npages;
in = kvzalloc(inlen, GFP_KERNEL);
if (!in) {
err = -ENOMEM;
- goto err_irq;
+ goto err_buf;
}
pas = (__be64 *)MLX5_ADDR_OF(create_eq_in, in, pas);
@@ -345,8 +344,6 @@ create_map_eq(struct mlx5_core_dev *dev, struct mlx5_eq *eq,
err_in:
kvfree(in);
-err_irq:
- mlx5_irq_release(eq->irq);
err_buf:
mlx5_frag_buf_free(dev, &eq->frag_buf);
return err;
@@ -400,7 +397,6 @@ static int destroy_unmap_eq(struct mlx5_core_dev *dev, struct mlx5_eq *eq)
if (err)
mlx5_core_warn(dev, "failed to destroy a previously created eq: eqn %d\n",
eq->eqn);
- mlx5_irq_release(eq->irq);
mlx5_frag_buf_free(dev, &eq->frag_buf);
return err;
@@ -593,11 +589,8 @@ setup_async_eq(struct mlx5_core_dev *dev, struct mlx5_eq_async *eq,
eq->irq_nb.notifier_call = mlx5_eq_async_int;
spin_lock_init(&eq->lock);
- if (!zalloc_cpumask_var(¶m->affinity, GFP_KERNEL))
- return -ENOMEM;
err = create_async_eq(dev, &eq->core, param);
- free_cpumask_var(param->affinity);
if (err) {
mlx5_core_warn(dev, "failed to create %s EQ %d\n", name, err);
return err;
@@ -622,17 +615,38 @@ static void cleanup_async_eq(struct mlx5_core_dev *dev,
name, err);
}
+static u16 async_eq_depth_devlink_param_get(struct mlx5_core_dev *dev)
+{
+ struct devlink *devlink = priv_to_devlink(dev);
+ union devlink_param_value val;
+ int err;
+
+ err = devlink_param_driverinit_value_get(devlink,
+ DEVLINK_PARAM_GENERIC_ID_EVENT_EQ_SIZE,
+ &val);
+ if (!err)
+ return val.vu32;
+ mlx5_core_dbg(dev, "Failed to get param. using default. err = %d\n", err);
+ return MLX5_NUM_ASYNC_EQE;
+}
static int create_async_eqs(struct mlx5_core_dev *dev)
{
struct mlx5_eq_table *table = dev->priv.eq_table;
struct mlx5_eq_param param = {};
int err;
+ /* All the async_eqs are using single IRQ, request one IRQ and share its
+ * index among all the async_eqs of this device.
+ */
+ table->ctrl_irq = mlx5_ctrl_irq_request(dev);
+ if (IS_ERR(table->ctrl_irq))
+ return PTR_ERR(table->ctrl_irq);
+
MLX5_NB_INIT(&table->cq_err_nb, cq_err_event_notifier, CQ_ERROR);
mlx5_eq_notifier_register(dev, &table->cq_err_nb);
param = (struct mlx5_eq_param) {
- .irq_index = MLX5_IRQ_EQ_CTRL,
+ .irq = table->ctrl_irq,
.nent = MLX5_NUM_CMD_EQE,
.mask[0] = 1ull << MLX5_EVENT_TYPE_CMD,
};
@@ -645,8 +659,8 @@ static int create_async_eqs(struct mlx5_core_dev *dev)
mlx5_cmd_allowed_opcode(dev, CMD_ALLOWED_OPCODE_ALL);
param = (struct mlx5_eq_param) {
- .irq_index = MLX5_IRQ_EQ_CTRL,
- .nent = MLX5_NUM_ASYNC_EQE,
+ .irq = table->ctrl_irq,
+ .nent = async_eq_depth_devlink_param_get(dev),
};
gather_async_events_mask(dev, param.mask);
@@ -655,7 +669,7 @@ static int create_async_eqs(struct mlx5_core_dev *dev)
goto err2;
param = (struct mlx5_eq_param) {
- .irq_index = MLX5_IRQ_EQ_CTRL,
+ .irq = table->ctrl_irq,
.nent = /* TODO: sriov max_vf + */ 1,
.mask[0] = 1ull << MLX5_EVENT_TYPE_PAGE_REQUEST,
};
@@ -674,6 +688,7 @@ static int create_async_eqs(struct mlx5_core_dev *dev)
err1:
mlx5_cmd_allowed_opcode(dev, CMD_ALLOWED_OPCODE_ALL);
mlx5_eq_notifier_unregister(dev, &table->cq_err_nb);
+ mlx5_ctrl_irq_release(table->ctrl_irq);
return err;
}
@@ -688,6 +703,7 @@ static void destroy_async_eqs(struct mlx5_core_dev *dev)
cleanup_async_eq(dev, &table->cmd_eq, "cmd");
mlx5_cmd_allowed_opcode(dev, CMD_ALLOWED_OPCODE_ALL);
mlx5_eq_notifier_unregister(dev, &table->cq_err_nb);
+ mlx5_ctrl_irq_release(table->ctrl_irq);
}
struct mlx5_eq *mlx5_get_async_eq(struct mlx5_core_dev *dev)
@@ -715,12 +731,10 @@ mlx5_eq_create_generic(struct mlx5_core_dev *dev,
struct mlx5_eq *eq = kvzalloc(sizeof(*eq), GFP_KERNEL);
int err;
- if (!cpumask_available(param->affinity))
- return ERR_PTR(-EINVAL);
-
if (!eq)
return ERR_PTR(-ENOMEM);
+ param->irq = dev->priv.eq_table->ctrl_irq;
err = create_async_eq(dev, eq, param);
if (err) {
kvfree(eq);
@@ -780,6 +794,54 @@ void mlx5_eq_update_ci(struct mlx5_eq *eq, u32 cc, bool arm)
}
EXPORT_SYMBOL(mlx5_eq_update_ci);
+static void comp_irqs_release(struct mlx5_core_dev *dev)
+{
+ struct mlx5_eq_table *table = dev->priv.eq_table;
+
+ if (mlx5_core_is_sf(dev))
+ mlx5_irq_affinity_irqs_release(dev, table->comp_irqs, table->num_comp_eqs);
+ else
+ mlx5_irqs_release_vectors(table->comp_irqs, table->num_comp_eqs);
+ kfree(table->comp_irqs);
+}
+
+static int comp_irqs_request(struct mlx5_core_dev *dev)
+{
+ struct mlx5_eq_table *table = dev->priv.eq_table;
+ int ncomp_eqs = table->num_comp_eqs;
+ u16 *cpus;
+ int ret;
+ int i;
+
+ ncomp_eqs = table->num_comp_eqs;
+ table->comp_irqs = kcalloc(ncomp_eqs, sizeof(*table->comp_irqs), GFP_KERNEL);
+ if (!table->comp_irqs)
+ return -ENOMEM;
+ if (mlx5_core_is_sf(dev)) {
+ ret = mlx5_irq_affinity_irqs_request_auto(dev, ncomp_eqs, table->comp_irqs);
+ if (ret < 0)
+ goto free_irqs;
+ return ret;
+ }
+
+ cpus = kcalloc(ncomp_eqs, sizeof(*cpus), GFP_KERNEL);
+ if (!cpus) {
+ ret = -ENOMEM;
+ goto free_irqs;
+ }
+ for (i = 0; i < ncomp_eqs; i++)
+ cpus[i] = cpumask_local_spread(i, dev->priv.numa_node);
+ ret = mlx5_irqs_request_vectors(dev, cpus, ncomp_eqs, table->comp_irqs);
+ kfree(cpus);
+ if (ret < 0)
+ goto free_irqs;
+ return ret;
+
+free_irqs:
+ kfree(table->comp_irqs);
+ return ret;
+}
+
static void destroy_comp_eqs(struct mlx5_core_dev *dev)
{
struct mlx5_eq_table *table = dev->priv.eq_table;
@@ -794,6 +856,22 @@ static void destroy_comp_eqs(struct mlx5_core_dev *dev)
tasklet_disable(&eq->tasklet_ctx.task);
kfree(eq);
}
+ comp_irqs_release(dev);
+}
+
+static u16 comp_eq_depth_devlink_param_get(struct mlx5_core_dev *dev)
+{
+ struct devlink *devlink = priv_to_devlink(dev);
+ union devlink_param_value val;
+ int err;
+
+ err = devlink_param_driverinit_value_get(devlink,
+ DEVLINK_PARAM_GENERIC_ID_IO_EQ_SIZE,
+ &val);
+ if (!err)
+ return val.vu32;
+ mlx5_core_dbg(dev, "Failed to get param. using default. err = %d\n", err);
+ return MLX5_COMP_EQ_SIZE;
}
static int create_comp_eqs(struct mlx5_core_dev *dev)
@@ -805,12 +883,13 @@ static int create_comp_eqs(struct mlx5_core_dev *dev)
int err;
int i;
+ ncomp_eqs = comp_irqs_request(dev);
+ if (ncomp_eqs < 0)
+ return ncomp_eqs;
INIT_LIST_HEAD(&table->comp_eqs_list);
- ncomp_eqs = table->num_comp_eqs;
- nent = MLX5_COMP_EQ_SIZE;
+ nent = comp_eq_depth_devlink_param_get(dev);
for (i = 0; i < ncomp_eqs; i++) {
struct mlx5_eq_param param = {};
- int vecidx = i;
eq = kzalloc(sizeof(*eq), GFP_KERNEL);
if (!eq) {
@@ -825,18 +904,11 @@ static int create_comp_eqs(struct mlx5_core_dev *dev)
eq->irq_nb.notifier_call = mlx5_eq_comp_int;
param = (struct mlx5_eq_param) {
- .irq_index = vecidx,
+ .irq = table->comp_irqs[i],
.nent = nent,
};
- if (!zalloc_cpumask_var(¶m.affinity, GFP_KERNEL)) {
- err = -ENOMEM;
- goto clean_eq;
- }
- cpumask_set_cpu(cpumask_local_spread(i, dev->priv.numa_node),
- param.affinity);
err = create_map_eq(dev, &eq->core, ¶m);
- free_cpumask_var(param.affinity);
if (err)
goto clean_eq;
err = mlx5_eq_enable(dev, &eq->core, &eq->irq_nb);
@@ -850,7 +922,9 @@ static int create_comp_eqs(struct mlx5_core_dev *dev)
list_add_tail(&eq->list, &table->comp_eqs_list);
}
+ table->num_comp_eqs = ncomp_eqs;
return 0;
+
clean_eq:
kfree(eq);
clean:
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/esw/indir_table.c b/drivers/net/ethernet/mellanox/mlx5/core/esw/indir_table.c
index 425c918..c275fe0 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/esw/indir_table.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/esw/indir_table.c
@@ -14,6 +14,7 @@
#include "fs_core.h"
#include "esw/indir_table.h"
#include "lib/fs_chains.h"
+#include "en/mod_hdr.h"
#define MLX5_ESW_INDIR_TABLE_SIZE 128
#define MLX5_ESW_INDIR_TABLE_RECIRC_IDX_MAX (MLX5_ESW_INDIR_TABLE_SIZE - 2)
@@ -226,7 +227,7 @@ static int mlx5_esw_indir_table_rule_get(struct mlx5_eswitch *esw,
goto err_handle;
}
- dealloc_mod_hdr_actions(&mod_acts);
+ mlx5e_mod_hdr_dealloc(&mod_acts);
rule->handle = handle;
rule->vni = esw_attr->rx_tun_attr->vni;
rule->mh = flow_act.modify_hdr;
@@ -243,7 +244,7 @@ static int mlx5_esw_indir_table_rule_get(struct mlx5_eswitch *esw,
mlx5_modify_header_dealloc(esw->dev, flow_act.modify_hdr);
err_mod_hdr_alloc:
err_mod_hdr_regc1:
- dealloc_mod_hdr_actions(&mod_acts);
+ mlx5e_mod_hdr_dealloc(&mod_acts);
err_mod_hdr_regc0:
err_ethertype:
kfree(rule);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/esw/legacy.c b/drivers/net/ethernet/mellanox/mlx5/core/esw/legacy.c
index df277a6..9d17206 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/esw/legacy.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/esw/legacy.c
@@ -431,7 +431,7 @@ int mlx5_eswitch_set_vport_vlan(struct mlx5_eswitch *esw,
int err = 0;
if (!mlx5_esw_allowed(esw))
- return -EPERM;
+ return vlan ? -EPERM : 0;
if (vlan || qos)
set_flags = SET_VLAN_STRIP | SET_VLAN_INSERT;
@@ -522,9 +522,7 @@ int mlx5_eswitch_set_vport_rate(struct mlx5_eswitch *esw, u16 vport,
return PTR_ERR(evport);
mutex_lock(&esw->state_lock);
- err = mlx5_esw_qos_set_vport_min_rate(esw, evport, min_rate, NULL);
- if (!err)
- err = mlx5_esw_qos_set_vport_max_rate(esw, evport, max_rate, NULL);
+ err = mlx5_esw_qos_set_vport_rate(esw, evport, max_rate, min_rate);
mutex_unlock(&esw->state_lock);
return err;
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/esw/qos.c b/drivers/net/ethernet/mellanox/mlx5/core/esw/qos.c
index d377ddc..11bbcd5 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/esw/qos.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/esw/qos.c
@@ -204,10 +204,8 @@ static int esw_qos_normalize_groups_min_rate(struct mlx5_eswitch *esw, u32 divid
return 0;
}
-int mlx5_esw_qos_set_vport_min_rate(struct mlx5_eswitch *esw,
- struct mlx5_vport *evport,
- u32 min_rate,
- struct netlink_ext_ack *extack)
+static int esw_qos_set_vport_min_rate(struct mlx5_eswitch *esw, struct mlx5_vport *evport,
+ u32 min_rate, struct netlink_ext_ack *extack)
{
u32 fw_max_bw_share, previous_min_rate;
bool min_rate_supported;
@@ -231,10 +229,8 @@ int mlx5_esw_qos_set_vport_min_rate(struct mlx5_eswitch *esw,
return err;
}
-int mlx5_esw_qos_set_vport_max_rate(struct mlx5_eswitch *esw,
- struct mlx5_vport *evport,
- u32 max_rate,
- struct netlink_ext_ack *extack)
+static int esw_qos_set_vport_max_rate(struct mlx5_eswitch *esw, struct mlx5_vport *evport,
+ u32 max_rate, struct netlink_ext_ack *extack)
{
u32 act_max_rate = max_rate;
bool max_rate_supported;
@@ -432,16 +428,13 @@ static int esw_qos_vport_update_group(struct mlx5_eswitch *esw,
}
static struct mlx5_esw_rate_group *
-esw_qos_create_rate_group(struct mlx5_eswitch *esw, struct netlink_ext_ack *extack)
+__esw_qos_create_rate_group(struct mlx5_eswitch *esw, struct netlink_ext_ack *extack)
{
u32 tsar_ctx[MLX5_ST_SZ_DW(scheduling_context)] = {};
struct mlx5_esw_rate_group *group;
u32 divider;
int err;
- if (!MLX5_CAP_QOS(esw->dev, log_esw_max_sched_depth))
- return ERR_PTR(-EOPNOTSUPP);
-
group = kzalloc(sizeof(*group), GFP_KERNEL);
if (!group)
return ERR_PTR(-ENOMEM);
@@ -482,9 +475,32 @@ esw_qos_create_rate_group(struct mlx5_eswitch *esw, struct netlink_ext_ack *exta
return ERR_PTR(err);
}
-static int esw_qos_destroy_rate_group(struct mlx5_eswitch *esw,
- struct mlx5_esw_rate_group *group,
- struct netlink_ext_ack *extack)
+static int esw_qos_get(struct mlx5_eswitch *esw, struct netlink_ext_ack *extack);
+static void esw_qos_put(struct mlx5_eswitch *esw);
+
+static struct mlx5_esw_rate_group *
+esw_qos_create_rate_group(struct mlx5_eswitch *esw, struct netlink_ext_ack *extack)
+{
+ struct mlx5_esw_rate_group *group;
+ int err;
+
+ if (!MLX5_CAP_QOS(esw->dev, log_esw_max_sched_depth))
+ return ERR_PTR(-EOPNOTSUPP);
+
+ err = esw_qos_get(esw, extack);
+ if (err)
+ return ERR_PTR(err);
+
+ group = __esw_qos_create_rate_group(esw, extack);
+ if (IS_ERR(group))
+ esw_qos_put(esw);
+
+ return group;
+}
+
+static int __esw_qos_destroy_rate_group(struct mlx5_eswitch *esw,
+ struct mlx5_esw_rate_group *group,
+ struct netlink_ext_ack *extack)
{
u32 divider;
int err;
@@ -503,7 +519,21 @@ static int esw_qos_destroy_rate_group(struct mlx5_eswitch *esw,
NL_SET_ERR_MSG_MOD(extack, "E-Switch destroy TSAR_ID failed");
trace_mlx5_esw_group_qos_destroy(esw->dev, group, group->tsar_ix);
+
kfree(group);
+
+ return err;
+}
+
+static int esw_qos_destroy_rate_group(struct mlx5_eswitch *esw,
+ struct mlx5_esw_rate_group *group,
+ struct netlink_ext_ack *extack)
+{
+ int err;
+
+ err = __esw_qos_destroy_rate_group(esw, group, extack);
+ esw_qos_put(esw);
+
return err;
}
@@ -526,7 +556,7 @@ static bool esw_qos_element_type_supported(struct mlx5_core_dev *dev, int type)
return false;
}
-void mlx5_esw_qos_create(struct mlx5_eswitch *esw)
+static int esw_qos_create(struct mlx5_eswitch *esw, struct netlink_ext_ack *extack)
{
u32 tsar_ctx[MLX5_ST_SZ_DW(scheduling_context)] = {};
struct mlx5_core_dev *dev = esw->dev;
@@ -534,14 +564,10 @@ void mlx5_esw_qos_create(struct mlx5_eswitch *esw)
int err;
if (!MLX5_CAP_GEN(dev, qos) || !MLX5_CAP_QOS(dev, esw_scheduling))
- return;
+ return -EOPNOTSUPP;
if (!esw_qos_element_type_supported(dev, SCHEDULING_CONTEXT_ELEMENT_TYPE_TSAR))
- return;
-
- mutex_lock(&esw->state_lock);
- if (esw->qos.enabled)
- goto unlock;
+ return -EOPNOTSUPP;
MLX5_SET(scheduling_context, tsar_ctx, element_type,
SCHEDULING_CONTEXT_ELEMENT_TYPE_TSAR);
@@ -555,75 +581,94 @@ void mlx5_esw_qos_create(struct mlx5_eswitch *esw)
&esw->qos.root_tsar_ix);
if (err) {
esw_warn(dev, "E-Switch create root TSAR failed (%d)\n", err);
- goto unlock;
+ return err;
}
INIT_LIST_HEAD(&esw->qos.groups);
if (MLX5_CAP_QOS(dev, log_esw_max_sched_depth)) {
- esw->qos.group0 = esw_qos_create_rate_group(esw, NULL);
+ esw->qos.group0 = __esw_qos_create_rate_group(esw, extack);
if (IS_ERR(esw->qos.group0)) {
esw_warn(dev, "E-Switch create rate group 0 failed (%ld)\n",
PTR_ERR(esw->qos.group0));
+ err = PTR_ERR(esw->qos.group0);
goto err_group0;
}
}
- esw->qos.enabled = true;
-unlock:
- mutex_unlock(&esw->state_lock);
- return;
+ refcount_set(&esw->qos.refcnt, 1);
+
+ return 0;
err_group0:
- err = mlx5_destroy_scheduling_element_cmd(esw->dev,
- SCHEDULING_HIERARCHY_E_SWITCH,
- esw->qos.root_tsar_ix);
- if (err)
- esw_warn(esw->dev, "E-Switch destroy root TSAR failed (%d)\n", err);
- mutex_unlock(&esw->state_lock);
+ if (mlx5_destroy_scheduling_element_cmd(esw->dev, SCHEDULING_HIERARCHY_E_SWITCH,
+ esw->qos.root_tsar_ix))
+ esw_warn(esw->dev, "E-Switch destroy root TSAR failed.\n");
+
+ return err;
}
-void mlx5_esw_qos_destroy(struct mlx5_eswitch *esw)
+static void esw_qos_destroy(struct mlx5_eswitch *esw)
{
- struct devlink *devlink = priv_to_devlink(esw->dev);
int err;
- devlink_rate_nodes_destroy(devlink);
- mutex_lock(&esw->state_lock);
- if (!esw->qos.enabled)
- goto unlock;
-
if (esw->qos.group0)
- esw_qos_destroy_rate_group(esw, esw->qos.group0, NULL);
+ __esw_qos_destroy_rate_group(esw, esw->qos.group0, NULL);
err = mlx5_destroy_scheduling_element_cmd(esw->dev,
SCHEDULING_HIERARCHY_E_SWITCH,
esw->qos.root_tsar_ix);
if (err)
esw_warn(esw->dev, "E-Switch destroy root TSAR failed (%d)\n", err);
-
- esw->qos.enabled = false;
-unlock:
- mutex_unlock(&esw->state_lock);
}
-int mlx5_esw_qos_vport_enable(struct mlx5_eswitch *esw, struct mlx5_vport *vport,
- u32 max_rate, u32 bw_share)
+static int esw_qos_get(struct mlx5_eswitch *esw, struct netlink_ext_ack *extack)
+{
+ int err = 0;
+
+ lockdep_assert_held(&esw->state_lock);
+
+ if (!refcount_inc_not_zero(&esw->qos.refcnt)) {
+ /* esw_qos_create() set refcount to 1 only on success.
+ * No need to decrement on failure.
+ */
+ err = esw_qos_create(esw, extack);
+ }
+
+ return err;
+}
+
+static void esw_qos_put(struct mlx5_eswitch *esw)
+{
+ lockdep_assert_held(&esw->state_lock);
+ if (refcount_dec_and_test(&esw->qos.refcnt))
+ esw_qos_destroy(esw);
+}
+
+static int esw_qos_vport_enable(struct mlx5_eswitch *esw, struct mlx5_vport *vport,
+ u32 max_rate, u32 bw_share, struct netlink_ext_ack *extack)
{
int err;
lockdep_assert_held(&esw->state_lock);
- if (!esw->qos.enabled)
+ if (vport->qos.enabled)
return 0;
- if (vport->qos.enabled)
- return -EEXIST;
+ err = esw_qos_get(esw, extack);
+ if (err)
+ return err;
vport->qos.group = esw->qos.group0;
err = esw_qos_vport_create_sched_element(esw, vport, max_rate, bw_share);
- if (!err) {
- vport->qos.enabled = true;
- trace_mlx5_esw_vport_qos_create(vport, bw_share, max_rate);
- }
+ if (err)
+ goto err_out;
+
+ vport->qos.enabled = true;
+ trace_mlx5_esw_vport_qos_create(vport, bw_share, max_rate);
+
+ return 0;
+
+err_out:
+ esw_qos_put(esw);
return err;
}
@@ -633,7 +678,7 @@ void mlx5_esw_qos_vport_disable(struct mlx5_eswitch *esw, struct mlx5_vport *vpo
int err;
lockdep_assert_held(&esw->state_lock);
- if (!esw->qos.enabled || !vport->qos.enabled)
+ if (!vport->qos.enabled)
return;
WARN(vport->qos.group && vport->qos.group != esw->qos.group0,
"Disabling QoS on port before detaching it from group");
@@ -645,8 +690,27 @@ void mlx5_esw_qos_vport_disable(struct mlx5_eswitch *esw, struct mlx5_vport *vpo
esw_warn(esw->dev, "E-Switch destroy TSAR vport element failed (vport=%d,err=%d)\n",
vport->vport, err);
- vport->qos.enabled = false;
+ memset(&vport->qos, 0, sizeof(vport->qos));
trace_mlx5_esw_vport_qos_destroy(vport);
+
+ esw_qos_put(esw);
+}
+
+int mlx5_esw_qos_set_vport_rate(struct mlx5_eswitch *esw, struct mlx5_vport *vport,
+ u32 min_rate, u32 max_rate)
+{
+ int err;
+
+ lockdep_assert_held(&esw->state_lock);
+ err = esw_qos_vport_enable(esw, vport, 0, 0, NULL);
+ if (err)
+ return err;
+
+ err = esw_qos_set_vport_min_rate(esw, vport, min_rate, NULL);
+ if (!err)
+ err = esw_qos_set_vport_max_rate(esw, vport, max_rate, NULL);
+
+ return err;
}
int mlx5_esw_qos_modify_vport_rate(struct mlx5_eswitch *esw, u16 vport_num, u32 rate_mbps)
@@ -654,22 +718,29 @@ int mlx5_esw_qos_modify_vport_rate(struct mlx5_eswitch *esw, u16 vport_num, u32
u32 ctx[MLX5_ST_SZ_DW(scheduling_context)] = {};
struct mlx5_vport *vport;
u32 bitmask;
+ int err;
vport = mlx5_eswitch_get_vport(esw, vport_num);
if (IS_ERR(vport))
return PTR_ERR(vport);
- if (!vport->qos.enabled)
- return -EOPNOTSUPP;
+ mutex_lock(&esw->state_lock);
+ if (!vport->qos.enabled) {
+ /* Eswitch QoS wasn't enabled yet. Enable it and vport QoS. */
+ err = esw_qos_vport_enable(esw, vport, rate_mbps, vport->qos.bw_share, NULL);
+ } else {
+ MLX5_SET(scheduling_context, ctx, max_average_bw, rate_mbps);
- MLX5_SET(scheduling_context, ctx, max_average_bw, rate_mbps);
- bitmask = MODIFY_SCHEDULING_ELEMENT_IN_MODIFY_BITMASK_MAX_AVERAGE_BW;
+ bitmask = MODIFY_SCHEDULING_ELEMENT_IN_MODIFY_BITMASK_MAX_AVERAGE_BW;
+ err = mlx5_modify_scheduling_element_cmd(esw->dev,
+ SCHEDULING_HIERARCHY_E_SWITCH,
+ ctx,
+ vport->qos.esw_tsar_ix,
+ bitmask);
+ }
+ mutex_unlock(&esw->state_lock);
- return mlx5_modify_scheduling_element_cmd(esw->dev,
- SCHEDULING_HIERARCHY_E_SWITCH,
- ctx,
- vport->qos.esw_tsar_ix,
- bitmask);
+ return err;
}
#define MLX5_LINKSPEED_UNIT 125000 /* 1Mbps in Bps */
@@ -728,7 +799,12 @@ int mlx5_esw_devlink_rate_leaf_tx_share_set(struct devlink_rate *rate_leaf, void
return err;
mutex_lock(&esw->state_lock);
- err = mlx5_esw_qos_set_vport_min_rate(esw, vport, tx_share, extack);
+ err = esw_qos_vport_enable(esw, vport, 0, 0, extack);
+ if (err)
+ goto unlock;
+
+ err = esw_qos_set_vport_min_rate(esw, vport, tx_share, extack);
+unlock:
mutex_unlock(&esw->state_lock);
return err;
}
@@ -749,7 +825,12 @@ int mlx5_esw_devlink_rate_leaf_tx_max_set(struct devlink_rate *rate_leaf, void *
return err;
mutex_lock(&esw->state_lock);
- err = mlx5_esw_qos_set_vport_max_rate(esw, vport, tx_max, extack);
+ err = esw_qos_vport_enable(esw, vport, 0, 0, extack);
+ if (err)
+ goto unlock;
+
+ err = esw_qos_set_vport_max_rate(esw, vport, tx_max, extack);
+unlock:
mutex_unlock(&esw->state_lock);
return err;
}
@@ -846,7 +927,9 @@ int mlx5_esw_qos_vport_update_group(struct mlx5_eswitch *esw,
int err;
mutex_lock(&esw->state_lock);
- err = esw_qos_vport_update_group(esw, vport, group, extack);
+ err = esw_qos_vport_enable(esw, vport, 0, 0, extack);
+ if (!err)
+ err = esw_qos_vport_update_group(esw, vport, group, extack);
mutex_unlock(&esw->state_lock);
return err;
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/esw/qos.h b/drivers/net/ethernet/mellanox/mlx5/core/esw/qos.h
index 28451ab..0141e9d 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/esw/qos.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/esw/qos.h
@@ -6,18 +6,8 @@
#ifdef CONFIG_MLX5_ESWITCH
-int mlx5_esw_qos_set_vport_min_rate(struct mlx5_eswitch *esw,
- struct mlx5_vport *evport,
- u32 min_rate,
- struct netlink_ext_ack *extack);
-int mlx5_esw_qos_set_vport_max_rate(struct mlx5_eswitch *esw,
- struct mlx5_vport *evport,
- u32 max_rate,
- struct netlink_ext_ack *extack);
-void mlx5_esw_qos_create(struct mlx5_eswitch *esw);
-void mlx5_esw_qos_destroy(struct mlx5_eswitch *esw);
-int mlx5_esw_qos_vport_enable(struct mlx5_eswitch *esw, struct mlx5_vport *vport,
- u32 max_rate, u32 bw_share);
+int mlx5_esw_qos_set_vport_rate(struct mlx5_eswitch *esw, struct mlx5_vport *evport,
+ u32 max_rate, u32 min_rate);
void mlx5_esw_qos_vport_disable(struct mlx5_eswitch *esw, struct mlx5_vport *vport);
int mlx5_esw_devlink_rate_leaf_tx_share_set(struct devlink_rate *rate_leaf, void *priv,
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
index 51a8cec..458ec0b 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
@@ -781,9 +781,6 @@ static int esw_vport_setup(struct mlx5_eswitch *esw, struct mlx5_vport *vport)
if (err)
return err;
- /* Attach vport to the eswitch rate limiter */
- mlx5_esw_qos_vport_enable(esw, vport, vport->qos.max_rate, vport->qos.bw_share);
-
if (mlx5_esw_is_manager_vport(esw, vport_num))
return 0;
@@ -1260,8 +1257,6 @@ int mlx5_eswitch_enable_locked(struct mlx5_eswitch *esw, int mode, int num_vfs)
mlx5_eswitch_update_num_of_vfs(esw, num_vfs);
- mlx5_esw_qos_create(esw);
-
esw->mode = mode;
if (mode == MLX5_ESWITCH_LEGACY) {
@@ -1290,7 +1285,6 @@ int mlx5_eswitch_enable_locked(struct mlx5_eswitch *esw, int mode, int num_vfs)
if (mode == MLX5_ESWITCH_OFFLOADS)
mlx5_rescan_drivers(esw->dev);
- mlx5_esw_qos_destroy(esw);
mlx5_esw_acls_ns_cleanup(esw);
return err;
}
@@ -1338,6 +1332,7 @@ int mlx5_eswitch_enable(struct mlx5_eswitch *esw, int num_vfs)
void mlx5_eswitch_disable_locked(struct mlx5_eswitch *esw, bool clear_vf)
{
+ struct devlink *devlink = priv_to_devlink(esw->dev);
int old_mode;
lockdep_assert_held_write(&esw->mode_lock);
@@ -1367,7 +1362,8 @@ void mlx5_eswitch_disable_locked(struct mlx5_eswitch *esw, bool clear_vf)
if (old_mode == MLX5_ESWITCH_OFFLOADS)
mlx5_rescan_drivers(esw->dev);
- mlx5_esw_qos_destroy(esw);
+ devlink_rate_nodes_destroy(devlink);
+
mlx5_esw_acls_ns_cleanup(esw);
if (clear_vf)
@@ -1576,6 +1572,7 @@ int mlx5_eswitch_init(struct mlx5_core_dev *dev)
lockdep_register_key(&esw->mode_lock_key);
init_rwsem(&esw->mode_lock);
lockdep_set_class(&esw->mode_lock, &esw->mode_lock_key);
+ refcount_set(&esw->qos.refcnt, 0);
esw->enabled_vports = 0;
esw->mode = MLX5_ESWITCH_NONE;
@@ -1614,6 +1611,7 @@ void mlx5_eswitch_cleanup(struct mlx5_eswitch *esw)
esw->dev->priv.eswitch = NULL;
destroy_workqueue(esw->work_queue);
+ WARN_ON(refcount_read(&esw->qos.refcnt));
lockdep_unregister_key(&esw->mode_lock_key);
mutex_destroy(&esw->state_lock);
WARN_ON(!xa_empty(&esw->offloads.vhca_map));
@@ -1703,82 +1701,6 @@ bool mlx5_esw_is_sf_vport(struct mlx5_eswitch *esw, u16 vport_num)
return mlx5_esw_check_port_type(esw, vport_num, MLX5_ESW_VPT_SF);
}
-static bool
-is_port_function_supported(struct mlx5_eswitch *esw, u16 vport_num)
-{
- return vport_num == MLX5_VPORT_PF ||
- mlx5_eswitch_is_vf_vport(esw, vport_num) ||
- mlx5_esw_is_sf_vport(esw, vport_num);
-}
-
-int mlx5_devlink_port_function_hw_addr_get(struct devlink_port *port,
- u8 *hw_addr, int *hw_addr_len,
- struct netlink_ext_ack *extack)
-{
- struct mlx5_eswitch *esw;
- struct mlx5_vport *vport;
- int err = -EOPNOTSUPP;
- u16 vport_num;
-
- esw = mlx5_devlink_eswitch_get(port->devlink);
- if (IS_ERR(esw))
- return PTR_ERR(esw);
-
- vport_num = mlx5_esw_devlink_port_index_to_vport_num(port->index);
- if (!is_port_function_supported(esw, vport_num))
- return -EOPNOTSUPP;
-
- vport = mlx5_eswitch_get_vport(esw, vport_num);
- if (IS_ERR(vport)) {
- NL_SET_ERR_MSG_MOD(extack, "Invalid port");
- return PTR_ERR(vport);
- }
-
- mutex_lock(&esw->state_lock);
- if (vport->enabled) {
- ether_addr_copy(hw_addr, vport->info.mac);
- *hw_addr_len = ETH_ALEN;
- err = 0;
- }
- mutex_unlock(&esw->state_lock);
- return err;
-}
-
-int mlx5_devlink_port_function_hw_addr_set(struct devlink_port *port,
- const u8 *hw_addr, int hw_addr_len,
- struct netlink_ext_ack *extack)
-{
- struct mlx5_eswitch *esw;
- struct mlx5_vport *vport;
- int err = -EOPNOTSUPP;
- u16 vport_num;
-
- esw = mlx5_devlink_eswitch_get(port->devlink);
- if (IS_ERR(esw)) {
- NL_SET_ERR_MSG_MOD(extack, "Eswitch doesn't support set hw_addr");
- return PTR_ERR(esw);
- }
-
- vport_num = mlx5_esw_devlink_port_index_to_vport_num(port->index);
- if (!is_port_function_supported(esw, vport_num)) {
- NL_SET_ERR_MSG_MOD(extack, "Port doesn't support set hw_addr");
- return -EINVAL;
- }
- vport = mlx5_eswitch_get_vport(esw, vport_num);
- if (IS_ERR(vport)) {
- NL_SET_ERR_MSG_MOD(extack, "Invalid port");
- return PTR_ERR(vport);
- }
-
- mutex_lock(&esw->state_lock);
- if (vport->enabled)
- err = mlx5_esw_set_vport_mac_locked(esw, vport, hw_addr);
- else
- NL_SET_ERR_MSG_MOD(extack, "Eswitch vport is disabled");
- mutex_unlock(&esw->state_lock);
- return err;
-}
-
int mlx5_eswitch_set_vport_state(struct mlx5_eswitch *esw,
u16 vport, int link_state)
{
@@ -1835,8 +1757,10 @@ int mlx5_eswitch_get_vport_config(struct mlx5_eswitch *esw,
ivi->qos = evport->info.qos;
ivi->spoofchk = evport->info.spoofchk;
ivi->trusted = evport->info.trusted;
- ivi->min_tx_rate = evport->qos.min_rate;
- ivi->max_tx_rate = evport->qos.max_rate;
+ if (evport->qos.enabled) {
+ ivi->min_tx_rate = evport->qos.min_rate;
+ ivi->max_tx_rate = evport->qos.max_rate;
+ }
mutex_unlock(&esw->state_lock);
return 0;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
index 42f8ee2..ead5e8a 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
@@ -308,10 +308,14 @@ struct mlx5_eswitch {
atomic64_t user_count;
struct {
- bool enabled;
u32 root_tsar_ix;
struct mlx5_esw_rate_group *group0;
struct list_head groups; /* Protected by esw->state_lock */
+
+ /* Protected by esw->state_lock.
+ * Initially 0, meaning no QoS users and QoS is disabled.
+ */
+ refcount_t refcnt;
} qos;
struct mlx5_esw_bridge_offloads *br_offloads;
@@ -339,9 +343,6 @@ void mlx5_esw_match_metadata_free(struct mlx5_eswitch *esw, u32 metadata);
int mlx5_esw_qos_modify_vport_rate(struct mlx5_eswitch *esw, u16 vport_num, u32 rate_mbps);
-bool mlx5_esw_vport_match_metadata_supported(const struct mlx5_eswitch *esw);
-int mlx5_esw_offloads_vport_metadata_set(struct mlx5_eswitch *esw, bool enable);
-
/* E-Switch API */
int mlx5_eswitch_init(struct mlx5_core_dev *dev);
void mlx5_eswitch_cleanup(struct mlx5_eswitch *esw);
@@ -516,11 +517,6 @@ int mlx5_eswitch_del_vlan_action(struct mlx5_eswitch *esw,
int __mlx5_eswitch_set_vport_vlan(struct mlx5_eswitch *esw,
u16 vport, u16 vlan, u8 qos, u8 set_flags);
-static inline bool mlx5_esw_qos_enabled(struct mlx5_eswitch *esw)
-{
- return esw->qos.enabled;
-}
-
static inline bool mlx5_eswitch_vlan_actions_supported(struct mlx5_core_dev *dev,
u8 vlan_depth)
{
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
index 32bc08a..9a7b256 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
@@ -295,26 +295,28 @@ esw_setup_chain_src_port_rewrite(struct mlx5_flow_destination *dest,
int *i)
{
struct mlx5_esw_flow_attr *esw_attr = attr->esw_attr;
- int j, err;
+ int err;
if (!(attr->flags & MLX5_ESW_ATTR_FLAG_SRC_REWRITE))
return -EOPNOTSUPP;
- for (j = esw_attr->split_count; j < esw_attr->out_count; j++, (*i)++) {
- err = esw_setup_chain_dest(dest, flow_act, chains, attr->dest_chain, 1, 0, *i);
- if (err)
- goto err_setup_chain;
+ /* flow steering cannot handle more than one dest with the same ft
+ * in a single flow
+ */
+ if (esw_attr->out_count - esw_attr->split_count > 1)
+ return -EOPNOTSUPP;
- if (esw_attr->dests[j].pkt_reformat) {
- flow_act->action |= MLX5_FLOW_CONTEXT_ACTION_PACKET_REFORMAT;
- flow_act->pkt_reformat = esw_attr->dests[j].pkt_reformat;
- }
+ err = esw_setup_chain_dest(dest, flow_act, chains, attr->dest_chain, 1, 0, *i);
+ if (err)
+ return err;
+
+ if (esw_attr->dests[esw_attr->split_count].pkt_reformat) {
+ flow_act->action |= MLX5_FLOW_CONTEXT_ACTION_PACKET_REFORMAT;
+ flow_act->pkt_reformat = esw_attr->dests[esw_attr->split_count].pkt_reformat;
}
- return 0;
+ (*i)++;
-err_setup_chain:
- esw_put_dest_tables_loop(esw, attr, esw_attr->split_count, j);
- return err;
+ return 0;
}
static void esw_cleanup_chain_src_port_rewrite(struct mlx5_eswitch *esw,
@@ -3867,3 +3869,62 @@ u32 mlx5_eswitch_get_vport_metadata_for_set(struct mlx5_eswitch *esw,
return vport->metadata;
}
EXPORT_SYMBOL(mlx5_eswitch_get_vport_metadata_for_set);
+
+static bool
+is_port_function_supported(struct mlx5_eswitch *esw, u16 vport_num)
+{
+ return vport_num == MLX5_VPORT_PF ||
+ mlx5_eswitch_is_vf_vport(esw, vport_num) ||
+ mlx5_esw_is_sf_vport(esw, vport_num);
+}
+
+int mlx5_devlink_port_function_hw_addr_get(struct devlink_port *port,
+ u8 *hw_addr, int *hw_addr_len,
+ struct netlink_ext_ack *extack)
+{
+ struct mlx5_eswitch *esw;
+ struct mlx5_vport *vport;
+ u16 vport_num;
+
+ esw = mlx5_devlink_eswitch_get(port->devlink);
+ if (IS_ERR(esw))
+ return PTR_ERR(esw);
+
+ vport_num = mlx5_esw_devlink_port_index_to_vport_num(port->index);
+ if (!is_port_function_supported(esw, vport_num))
+ return -EOPNOTSUPP;
+
+ vport = mlx5_eswitch_get_vport(esw, vport_num);
+ if (IS_ERR(vport)) {
+ NL_SET_ERR_MSG_MOD(extack, "Invalid port");
+ return PTR_ERR(vport);
+ }
+
+ mutex_lock(&esw->state_lock);
+ ether_addr_copy(hw_addr, vport->info.mac);
+ *hw_addr_len = ETH_ALEN;
+ mutex_unlock(&esw->state_lock);
+ return 0;
+}
+
+int mlx5_devlink_port_function_hw_addr_set(struct devlink_port *port,
+ const u8 *hw_addr, int hw_addr_len,
+ struct netlink_ext_ack *extack)
+{
+ struct mlx5_eswitch *esw;
+ u16 vport_num;
+
+ esw = mlx5_devlink_eswitch_get(port->devlink);
+ if (IS_ERR(esw)) {
+ NL_SET_ERR_MSG_MOD(extack, "Eswitch doesn't support set hw_addr");
+ return PTR_ERR(esw);
+ }
+
+ vport_num = mlx5_esw_devlink_port_index_to_vport_num(port->index);
+ if (!is_port_function_supported(esw, vport_num)) {
+ NL_SET_ERR_MSG_MOD(extack, "Port doesn't support set hw_addr");
+ return -EINVAL;
+ }
+
+ return mlx5_eswitch_set_vport_mac(esw, vport_num, hw_addr);
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fs_cmd.c b/drivers/net/ethernet/mellanox/mlx5/core/fs_cmd.c
index 750b211..dafe341 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/fs_cmd.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/fs_cmd.c
@@ -451,7 +451,8 @@ static int mlx5_set_extended_dest(struct mlx5_core_dev *dev,
list_for_each_entry(dst, &fte->node.children, node.list) {
if (dst->dest_attr.type == MLX5_FLOW_DESTINATION_TYPE_COUNTER)
continue;
- if (dst->dest_attr.type == MLX5_FLOW_DESTINATION_TYPE_VPORT &&
+ if ((dst->dest_attr.type == MLX5_FLOW_DESTINATION_TYPE_VPORT ||
+ dst->dest_attr.type == MLX5_FLOW_DESTINATION_TYPE_UPLINK) &&
dst->dest_attr.vport.flags & MLX5_FLOW_DEST_VPORT_REFORMAT_ID)
num_encap++;
num_fwd_destinations++;
@@ -788,7 +789,8 @@ static int mlx5_cmd_packet_reformat_alloc(struct mlx5_flow_root_namespace *ns,
int err;
u32 *in;
- if (namespace == MLX5_FLOW_NAMESPACE_FDB)
+ if (namespace == MLX5_FLOW_NAMESPACE_FDB ||
+ namespace == MLX5_FLOW_NAMESPACE_FDB_BYPASS)
max_encap_size = MLX5_CAP_ESW(dev, max_encap_header_size);
else
max_encap_size = MLX5_CAP_FLOWTABLE(dev, max_encap_header_size);
@@ -860,6 +862,7 @@ static int mlx5_cmd_modify_header_alloc(struct mlx5_flow_root_namespace *ns,
switch (namespace) {
case MLX5_FLOW_NAMESPACE_FDB:
+ case MLX5_FLOW_NAMESPACE_FDB_BYPASS:
max_actions = MLX5_CAP_ESW_FLOWTABLE_FDB(dev, max_modify_header_actions);
table_type = FS_FT_FDB;
break;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c b/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c
index 386ab9a..b628917 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c
@@ -1525,7 +1525,8 @@ static bool mlx5_flow_dests_cmp(struct mlx5_flow_destination *d1,
struct mlx5_flow_destination *d2)
{
if (d1->type == d2->type) {
- if ((d1->type == MLX5_FLOW_DESTINATION_TYPE_VPORT &&
+ if (((d1->type == MLX5_FLOW_DESTINATION_TYPE_VPORT ||
+ d1->type == MLX5_FLOW_DESTINATION_TYPE_UPLINK) &&
d1->vport.num == d2->vport.num &&
d1->vport.flags == d2->vport.flags &&
((d1->vport.flags & MLX5_FLOW_DEST_VPORT_VHCA_ID) ?
@@ -2206,6 +2207,22 @@ struct mlx5_flow_namespace *mlx5_get_fdb_sub_ns(struct mlx5_core_dev *dev,
}
EXPORT_SYMBOL(mlx5_get_fdb_sub_ns);
+static bool is_nic_rx_ns(enum mlx5_flow_namespace_type type)
+{
+ switch (type) {
+ case MLX5_FLOW_NAMESPACE_BYPASS:
+ case MLX5_FLOW_NAMESPACE_LAG:
+ case MLX5_FLOW_NAMESPACE_OFFLOADS:
+ case MLX5_FLOW_NAMESPACE_ETHTOOL:
+ case MLX5_FLOW_NAMESPACE_KERNEL:
+ case MLX5_FLOW_NAMESPACE_LEFTOVERS:
+ case MLX5_FLOW_NAMESPACE_ANCHOR:
+ return true;
+ default:
+ return false;
+ }
+}
+
struct mlx5_flow_namespace *mlx5_get_flow_namespace(struct mlx5_core_dev *dev,
enum mlx5_flow_namespace_type type)
{
@@ -2235,31 +2252,39 @@ struct mlx5_flow_namespace *mlx5_get_flow_namespace(struct mlx5_core_dev *dev,
if (steering->sniffer_tx_root_ns)
return &steering->sniffer_tx_root_ns->ns;
return NULL;
- default:
+ case MLX5_FLOW_NAMESPACE_FDB_BYPASS:
+ root_ns = steering->fdb_root_ns;
+ prio = FDB_BYPASS_PATH;
break;
- }
-
- if (type == MLX5_FLOW_NAMESPACE_EGRESS ||
- type == MLX5_FLOW_NAMESPACE_EGRESS_KERNEL) {
+ case MLX5_FLOW_NAMESPACE_EGRESS:
+ case MLX5_FLOW_NAMESPACE_EGRESS_KERNEL:
root_ns = steering->egress_root_ns;
prio = type - MLX5_FLOW_NAMESPACE_EGRESS;
- } else if (type == MLX5_FLOW_NAMESPACE_RDMA_RX) {
+ break;
+ case MLX5_FLOW_NAMESPACE_RDMA_RX:
root_ns = steering->rdma_rx_root_ns;
prio = RDMA_RX_BYPASS_PRIO;
- } else if (type == MLX5_FLOW_NAMESPACE_RDMA_RX_KERNEL) {
+ break;
+ case MLX5_FLOW_NAMESPACE_RDMA_RX_KERNEL:
root_ns = steering->rdma_rx_root_ns;
prio = RDMA_RX_KERNEL_PRIO;
- } else if (type == MLX5_FLOW_NAMESPACE_RDMA_TX) {
+ break;
+ case MLX5_FLOW_NAMESPACE_RDMA_TX:
root_ns = steering->rdma_tx_root_ns;
- } else if (type == MLX5_FLOW_NAMESPACE_RDMA_RX_COUNTERS) {
+ break;
+ case MLX5_FLOW_NAMESPACE_RDMA_RX_COUNTERS:
root_ns = steering->rdma_rx_root_ns;
prio = RDMA_RX_COUNTERS_PRIO;
- } else if (type == MLX5_FLOW_NAMESPACE_RDMA_TX_COUNTERS) {
+ break;
+ case MLX5_FLOW_NAMESPACE_RDMA_TX_COUNTERS:
root_ns = steering->rdma_tx_root_ns;
prio = RDMA_TX_COUNTERS_PRIO;
- } else { /* Must be NIC RX */
+ break;
+ default: /* Must be NIC RX */
+ WARN_ON(!is_nic_rx_ns(type));
root_ns = steering->root_ns;
prio = type;
+ break;
}
if (!root_ns)
@@ -2822,6 +2847,28 @@ static int create_fdb_fast_path(struct mlx5_flow_steering *steering)
return 0;
}
+static int create_fdb_bypass(struct mlx5_flow_steering *steering)
+{
+ struct mlx5_flow_namespace *ns;
+ struct fs_prio *prio;
+ int i;
+
+ prio = fs_create_prio(&steering->fdb_root_ns->ns, FDB_BYPASS_PATH, 0);
+ if (IS_ERR(prio))
+ return PTR_ERR(prio);
+
+ ns = fs_create_namespace(prio, MLX5_FLOW_TABLE_MISS_ACTION_DEF);
+ if (IS_ERR(ns))
+ return PTR_ERR(ns);
+
+ for (i = 0; i < MLX5_BY_PASS_NUM_REGULAR_PRIOS; i++) {
+ prio = fs_create_prio(ns, i, 1);
+ if (IS_ERR(prio))
+ return PTR_ERR(prio);
+ }
+ return 0;
+}
+
static int init_fdb_root_ns(struct mlx5_flow_steering *steering)
{
struct fs_prio *maj_prio;
@@ -2831,12 +2878,10 @@ static int init_fdb_root_ns(struct mlx5_flow_steering *steering)
if (!steering->fdb_root_ns)
return -ENOMEM;
- maj_prio = fs_create_prio(&steering->fdb_root_ns->ns, FDB_BYPASS_PATH,
- 1);
- if (IS_ERR(maj_prio)) {
- err = PTR_ERR(maj_prio);
+ err = create_fdb_bypass(steering);
+ if (err)
goto out_err;
- }
+
err = create_fdb_fast_path(steering);
if (err)
goto out_err;
@@ -3038,6 +3083,11 @@ int mlx5_init_fs(struct mlx5_core_dev *dev)
steering->dev = dev;
dev->priv.steering = steering;
+ if (mlx5_fs_dr_is_supported(dev))
+ steering->mode = MLX5_FLOW_STEERING_MODE_SMFS;
+ else
+ steering->mode = MLX5_FLOW_STEERING_MODE_DMFS;
+
steering->fgs_cache = kmem_cache_create("mlx5_fs_fgs",
sizeof(struct mlx5_flow_group), 0,
0, NULL);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fs_core.h b/drivers/net/ethernet/mellanox/mlx5/core/fs_core.h
index 7711db2..5469b08 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/fs_core.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/fs_core.h
@@ -203,7 +203,7 @@ struct mlx5_ft_underlay_qp {
u32 qpn;
};
-#define MLX5_FTE_MATCH_PARAM_RESERVED reserved_at_c00
+#define MLX5_FTE_MATCH_PARAM_RESERVED reserved_at_e00
/* Calculate the fte_match_param length and without the reserved length.
* Make sure the reserved field is the last.
*/
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fs_counters.c b/drivers/net/ethernet/mellanox/mlx5/core/fs_counters.c
index 7e0e04c..b406e03 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/fs_counters.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/fs_counters.c
@@ -38,9 +38,10 @@
#include "fs_cmd.h"
#define MLX5_FC_STATS_PERIOD msecs_to_jiffies(1000)
+#define MLX5_FC_BULK_QUERY_ALLOC_PERIOD msecs_to_jiffies(180 * 1000)
/* Max number of counters to query in bulk read is 32K */
#define MLX5_SW_MAX_COUNTERS_BULK BIT(15)
-#define MLX5_SF_NUM_COUNTERS_BULK 8
+#define MLX5_INIT_COUNTERS_BULK 8
#define MLX5_FC_POOL_MAX_THRESHOLD BIT(18)
#define MLX5_FC_POOL_USED_BUFF_RATIO 10
@@ -145,13 +146,15 @@ static void mlx5_fc_stats_remove(struct mlx5_core_dev *dev,
spin_unlock(&fc_stats->counters_idr_lock);
}
+static int get_init_bulk_query_len(struct mlx5_core_dev *dev)
+{
+ return min_t(int, MLX5_INIT_COUNTERS_BULK,
+ (1 << MLX5_CAP_GEN(dev, log_max_flow_counter_bulk)));
+}
+
static int get_max_bulk_query_len(struct mlx5_core_dev *dev)
{
- int num_counters_bulk = mlx5_core_is_sf(dev) ?
- MLX5_SF_NUM_COUNTERS_BULK :
- MLX5_SW_MAX_COUNTERS_BULK;
-
- return min_t(int, num_counters_bulk,
+ return min_t(int, MLX5_SW_MAX_COUNTERS_BULK,
(1 << MLX5_CAP_GEN(dev, log_max_flow_counter_bulk)));
}
@@ -177,7 +180,7 @@ static void mlx5_fc_stats_query_counter_range(struct mlx5_core_dev *dev,
{
struct mlx5_fc_stats *fc_stats = &dev->priv.fc_stats;
bool query_more_counters = (first->id <= last_id);
- int max_bulk_len = get_max_bulk_query_len(dev);
+ int cur_bulk_len = fc_stats->bulk_query_len;
u32 *data = fc_stats->bulk_query_out;
struct mlx5_fc *counter = first;
u32 bulk_base_id;
@@ -189,7 +192,7 @@ static void mlx5_fc_stats_query_counter_range(struct mlx5_core_dev *dev,
bulk_base_id = counter->id & ~0x3;
/* number of counters to query inc. the last counter */
- bulk_len = min_t(int, max_bulk_len,
+ bulk_len = min_t(int, cur_bulk_len,
ALIGN(last_id - bulk_base_id + 1, 4));
err = mlx5_cmd_fc_bulk_query(dev, bulk_base_id, bulk_len,
@@ -230,6 +233,41 @@ static void mlx5_fc_release(struct mlx5_core_dev *dev, struct mlx5_fc *counter)
mlx5_fc_free(dev, counter);
}
+static void mlx5_fc_stats_bulk_query_size_increase(struct mlx5_core_dev *dev)
+{
+ struct mlx5_fc_stats *fc_stats = &dev->priv.fc_stats;
+ int max_bulk_len = get_max_bulk_query_len(dev);
+ unsigned long now = jiffies;
+ u32 *bulk_query_out_tmp;
+ int max_out_len;
+
+ if (fc_stats->bulk_query_alloc_failed &&
+ time_before(now, fc_stats->next_bulk_query_alloc))
+ return;
+
+ max_out_len = mlx5_cmd_fc_get_bulk_query_out_len(max_bulk_len);
+ bulk_query_out_tmp = kzalloc(max_out_len, GFP_KERNEL);
+ if (!bulk_query_out_tmp) {
+ mlx5_core_warn_once(dev,
+ "Can't increase flow counters bulk query buffer size, insufficient memory, bulk_size(%d)\n",
+ max_bulk_len);
+ fc_stats->bulk_query_alloc_failed = true;
+ fc_stats->next_bulk_query_alloc =
+ now + MLX5_FC_BULK_QUERY_ALLOC_PERIOD;
+ return;
+ }
+
+ kfree(fc_stats->bulk_query_out);
+ fc_stats->bulk_query_out = bulk_query_out_tmp;
+ fc_stats->bulk_query_len = max_bulk_len;
+ if (fc_stats->bulk_query_alloc_failed) {
+ mlx5_core_info(dev,
+ "Flow counters bulk query buffer size increased, bulk_size(%d)\n",
+ max_bulk_len);
+ fc_stats->bulk_query_alloc_failed = false;
+ }
+}
+
static void mlx5_fc_stats_work(struct work_struct *work)
{
struct mlx5_core_dev *dev = container_of(work, struct mlx5_core_dev,
@@ -247,15 +285,22 @@ static void mlx5_fc_stats_work(struct work_struct *work)
queue_delayed_work(fc_stats->wq, &fc_stats->work,
fc_stats->sampling_interval);
- llist_for_each_entry(counter, addlist, addlist)
+ llist_for_each_entry(counter, addlist, addlist) {
mlx5_fc_stats_insert(dev, counter);
+ fc_stats->num_counters++;
+ }
llist_for_each_entry_safe(counter, tmp, dellist, dellist) {
mlx5_fc_stats_remove(dev, counter);
mlx5_fc_release(dev, counter);
+ fc_stats->num_counters--;
}
+ if (fc_stats->bulk_query_len < get_max_bulk_query_len(dev) &&
+ fc_stats->num_counters > get_init_bulk_query_len(dev))
+ mlx5_fc_stats_bulk_query_size_increase(dev);
+
if (time_before(now, fc_stats->next_query) ||
list_empty(&fc_stats->counters))
return;
@@ -378,8 +423,8 @@ EXPORT_SYMBOL(mlx5_fc_destroy);
int mlx5_init_fc_stats(struct mlx5_core_dev *dev)
{
struct mlx5_fc_stats *fc_stats = &dev->priv.fc_stats;
- int max_bulk_len;
- int max_out_len;
+ int init_bulk_len;
+ int init_out_len;
spin_lock_init(&fc_stats->counters_idr_lock);
idr_init(&fc_stats->counters_idr);
@@ -387,11 +432,12 @@ int mlx5_init_fc_stats(struct mlx5_core_dev *dev)
init_llist_head(&fc_stats->addlist);
init_llist_head(&fc_stats->dellist);
- max_bulk_len = get_max_bulk_query_len(dev);
- max_out_len = mlx5_cmd_fc_get_bulk_query_out_len(max_bulk_len);
- fc_stats->bulk_query_out = kzalloc(max_out_len, GFP_KERNEL);
+ init_bulk_len = get_init_bulk_query_len(dev);
+ init_out_len = mlx5_cmd_fc_get_bulk_query_out_len(init_bulk_len);
+ fc_stats->bulk_query_out = kzalloc(init_out_len, GFP_KERNEL);
if (!fc_stats->bulk_query_out)
return -ENOMEM;
+ fc_stats->bulk_query_len = init_bulk_len;
fc_stats->wq = create_singlethread_workqueue("mlx5_fc");
if (!fc_stats->wq)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/health.c b/drivers/net/ethernet/mellanox/mlx5/core/health.c
index 3ca9988..737df40 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/health.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/health.c
@@ -420,6 +420,11 @@ static void print_health_info(struct mlx5_core_dev *dev)
if (!ioread8(&h->synd))
return;
+ if (ioread32be(&h->fw_ver) == 0xFFFFFFFF) {
+ mlx5_log(dev, LOGLEVEL_ERR, "PCI slot is unavailable\n");
+ return;
+ }
+
rfr_severity = ioread8(&h->rfr_severity);
severity = mlx5_health_get_severity(rfr_severity);
mlx5_log(dev, severity, "Health issue observed, %s, severity(%d) %s:\n",
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ethtool.c b/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ethtool.c
index 962d414..f4f7eaf 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ethtool.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ethtool.c
@@ -67,7 +67,9 @@ static void mlx5i_get_ethtool_stats(struct net_device *dev,
}
static int mlx5i_set_ringparam(struct net_device *dev,
- struct ethtool_ringparam *param)
+ struct ethtool_ringparam *param,
+ struct kernel_ethtool_ringparam *kernel_param,
+ struct netlink_ext_ack *extack)
{
struct mlx5e_priv *priv = mlx5i_epriv(dev);
@@ -75,7 +77,9 @@ static int mlx5i_set_ringparam(struct net_device *dev,
}
static void mlx5i_get_ringparam(struct net_device *dev,
- struct ethtool_ringparam *param)
+ struct ethtool_ringparam *param,
+ struct kernel_ethtool_ringparam *kernel_param,
+ struct netlink_ext_ack *extack)
{
struct mlx5e_priv *priv = mlx5i_epriv(dev);
@@ -105,7 +109,7 @@ static int mlx5i_set_coalesce(struct net_device *netdev,
{
struct mlx5e_priv *priv = mlx5i_epriv(netdev);
- return mlx5e_ethtool_set_coalesce(priv, coal);
+ return mlx5e_ethtool_set_coalesce(priv, coal, kernel_coal, extack);
}
static int mlx5i_get_coalesce(struct net_device *netdev,
@@ -115,7 +119,7 @@ static int mlx5i_get_coalesce(struct net_device *netdev,
{
struct mlx5e_priv *priv = mlx5i_epriv(netdev);
- return mlx5e_ethtool_get_coalesce(priv, coal);
+ return mlx5e_ethtool_get_coalesce(priv, coal, kernel_coal);
}
static int mlx5i_get_ts_info(struct net_device *netdev,
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c b/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c
index ea1efde..0a99a02 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c
@@ -110,14 +110,14 @@ void mlx5i_cleanup(struct mlx5e_priv *priv)
static void mlx5i_grp_sw_update_stats(struct mlx5e_priv *priv)
{
- struct mlx5e_sw_stats s = { 0 };
+ struct rtnl_link_stats64 s = {};
int i, j;
for (i = 0; i < priv->stats_nch; i++) {
struct mlx5e_channel_stats *channel_stats;
struct mlx5e_rq_stats *rq_stats;
- channel_stats = &priv->channel_stats[i];
+ channel_stats = priv->channel_stats[i];
rq_stats = &channel_stats->rq;
s.rx_packets += rq_stats->packets;
@@ -128,11 +128,17 @@ static void mlx5i_grp_sw_update_stats(struct mlx5e_priv *priv)
s.tx_packets += sq_stats->packets;
s.tx_bytes += sq_stats->bytes;
- s.tx_queue_dropped += sq_stats->dropped;
+ s.tx_dropped += sq_stats->dropped;
}
}
- memcpy(&priv->stats.sw, &s, sizeof(s));
+ memset(&priv->stats.sw, 0, sizeof(s));
+
+ priv->stats.sw.rx_packets = s.rx_packets;
+ priv->stats.sw.rx_bytes = s.rx_bytes;
+ priv->stats.sw.tx_packets = s.tx_packets;
+ priv->stats.sw.tx_bytes = s.tx_bytes;
+ priv->stats.sw.tx_queue_dropped = s.tx_dropped;
}
void mlx5i_get_stats(struct net_device *dev, struct rtnl_link_stats64 *stats)
@@ -443,7 +449,6 @@ static const struct mlx5e_profile mlx5i_nic_profile = {
.rq_groups = MLX5E_NUM_RQ_GROUPS(REGULAR),
.stats_grps = mlx5i_stats_grps,
.stats_grps_num = mlx5i_stats_grps_num,
- .rx_ptp_support = false,
};
/* mlx5i netdev NDos */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib_vlan.c b/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib_vlan.c
index 5308f23..0b86e78 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib_vlan.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib_vlan.c
@@ -350,7 +350,6 @@ static const struct mlx5e_profile mlx5i_pkey_nic_profile = {
.rx_handlers = &mlx5i_rx_handlers,
.max_tc = MLX5I_MAX_NUM_TC,
.rq_groups = MLX5E_NUM_RQ_GROUPS(REGULAR),
- .rx_ptp_support = false,
};
const struct mlx5e_profile *mlx5i_pkey_get_profile(void)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/irq_affinity.c b/drivers/net/ethernet/mellanox/mlx5/core/irq_affinity.c
new file mode 100644
index 0000000..380a208a
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/irq_affinity.c
@@ -0,0 +1,226 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+/* Copyright (c) 2021, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */
+
+#include "mlx5_core.h"
+#include "mlx5_irq.h"
+#include "pci_irq.h"
+
+static void cpu_put(struct mlx5_irq_pool *pool, int cpu)
+{
+ pool->irqs_per_cpu[cpu]--;
+}
+
+static void cpu_get(struct mlx5_irq_pool *pool, int cpu)
+{
+ pool->irqs_per_cpu[cpu]++;
+}
+
+/* Gets the least loaded CPU. e.g.: the CPU with least IRQs bound to it */
+static int cpu_get_least_loaded(struct mlx5_irq_pool *pool,
+ const struct cpumask *req_mask)
+{
+ int best_cpu = -1;
+ int cpu;
+
+ for_each_cpu_and(cpu, req_mask, cpu_online_mask) {
+ /* CPU has zero IRQs on it. No need to search any more CPUs. */
+ if (!pool->irqs_per_cpu[cpu]) {
+ best_cpu = cpu;
+ break;
+ }
+ if (best_cpu < 0)
+ best_cpu = cpu;
+ if (pool->irqs_per_cpu[cpu] < pool->irqs_per_cpu[best_cpu])
+ best_cpu = cpu;
+ }
+ if (best_cpu == -1) {
+ /* There isn't online CPUs in req_mask */
+ mlx5_core_err(pool->dev, "NO online CPUs in req_mask (%*pbl)\n",
+ cpumask_pr_args(req_mask));
+ best_cpu = cpumask_first(cpu_online_mask);
+ }
+ pool->irqs_per_cpu[best_cpu]++;
+ return best_cpu;
+}
+
+/* Creating an IRQ from irq_pool */
+static struct mlx5_irq *
+irq_pool_request_irq(struct mlx5_irq_pool *pool, const struct cpumask *req_mask)
+{
+ cpumask_var_t auto_mask;
+ struct mlx5_irq *irq;
+ u32 irq_index;
+ int err;
+
+ if (!zalloc_cpumask_var(&auto_mask, GFP_KERNEL))
+ return ERR_PTR(-ENOMEM);
+ err = xa_alloc(&pool->irqs, &irq_index, NULL, pool->xa_num_irqs, GFP_KERNEL);
+ if (err)
+ return ERR_PTR(err);
+ if (pool->irqs_per_cpu) {
+ if (cpumask_weight(req_mask) > 1)
+ /* if req_mask contain more then one CPU, set the least loadad CPU
+ * of req_mask
+ */
+ cpumask_set_cpu(cpu_get_least_loaded(pool, req_mask), auto_mask);
+ else
+ cpu_get(pool, cpumask_first(req_mask));
+ }
+ irq = mlx5_irq_alloc(pool, irq_index, cpumask_empty(auto_mask) ? req_mask : auto_mask);
+ free_cpumask_var(auto_mask);
+ return irq;
+}
+
+/* Looking for the IRQ with the smallest refcount that fits req_mask.
+ * If pool is sf_comp_pool, then we are looking for an IRQ with any of the
+ * requested CPUs in req_mask.
+ * for example: req_mask = 0xf, irq0_mask = 0x10, irq1_mask = 0x1. irq0_mask
+ * isn't subset of req_mask, so we will skip it. irq1_mask is subset of req_mask,
+ * we don't skip it.
+ * If pool is sf_ctrl_pool, then all IRQs have the same mask, so any IRQ will
+ * fit. And since mask is subset of itself, we will pass the first if bellow.
+ */
+static struct mlx5_irq *
+irq_pool_find_least_loaded(struct mlx5_irq_pool *pool, const struct cpumask *req_mask)
+{
+ int start = pool->xa_num_irqs.min;
+ int end = pool->xa_num_irqs.max;
+ struct mlx5_irq *irq = NULL;
+ struct mlx5_irq *iter;
+ int irq_refcount = 0;
+ unsigned long index;
+
+ lockdep_assert_held(&pool->lock);
+ xa_for_each_range(&pool->irqs, index, iter, start, end) {
+ struct cpumask *iter_mask = mlx5_irq_get_affinity_mask(iter);
+ int iter_refcount = mlx5_irq_read_locked(iter);
+
+ if (!cpumask_subset(iter_mask, req_mask))
+ /* skip IRQs with a mask which is not subset of req_mask */
+ continue;
+ if (iter_refcount < pool->min_threshold)
+ /* If we found an IRQ with less than min_thres, return it */
+ return iter;
+ if (!irq || iter_refcount < irq_refcount) {
+ /* In case we won't find an IRQ with less than min_thres,
+ * keep a pointer to the least used IRQ
+ */
+ irq_refcount = iter_refcount;
+ irq = iter;
+ }
+ }
+ return irq;
+}
+
+/**
+ * mlx5_irq_affinity_request - request an IRQ according to the given mask.
+ * @pool: IRQ pool to request from.
+ * @req_mask: cpumask requested for this IRQ.
+ *
+ * This function returns a pointer to IRQ, or ERR_PTR in case of error.
+ */
+struct mlx5_irq *
+mlx5_irq_affinity_request(struct mlx5_irq_pool *pool, const struct cpumask *req_mask)
+{
+ struct mlx5_irq *least_loaded_irq, *new_irq;
+
+ mutex_lock(&pool->lock);
+ least_loaded_irq = irq_pool_find_least_loaded(pool, req_mask);
+ if (least_loaded_irq &&
+ mlx5_irq_read_locked(least_loaded_irq) < pool->min_threshold)
+ goto out;
+ /* We didn't find an IRQ with less than min_thres, try to allocate a new IRQ */
+ new_irq = irq_pool_request_irq(pool, req_mask);
+ if (IS_ERR(new_irq)) {
+ if (!least_loaded_irq) {
+ /* We failed to create an IRQ and we didn't find an IRQ */
+ mlx5_core_err(pool->dev, "Didn't find a matching IRQ. err = %ld\n",
+ PTR_ERR(new_irq));
+ mutex_unlock(&pool->lock);
+ return new_irq;
+ }
+ /* We failed to create a new IRQ for the requested affinity,
+ * sharing existing IRQ.
+ */
+ goto out;
+ }
+ least_loaded_irq = new_irq;
+ goto unlock;
+out:
+ mlx5_irq_get_locked(least_loaded_irq);
+ if (mlx5_irq_read_locked(least_loaded_irq) > pool->max_threshold)
+ mlx5_core_dbg(pool->dev, "IRQ %u overloaded, pool_name: %s, %u EQs on this irq\n",
+ pci_irq_vector(pool->dev->pdev,
+ mlx5_irq_get_index(least_loaded_irq)), pool->name,
+ mlx5_irq_read_locked(least_loaded_irq) / MLX5_EQ_REFS_PER_IRQ);
+unlock:
+ mutex_unlock(&pool->lock);
+ return least_loaded_irq;
+}
+
+void mlx5_irq_affinity_irqs_release(struct mlx5_core_dev *dev, struct mlx5_irq **irqs,
+ int num_irqs)
+{
+ struct mlx5_irq_pool *pool = mlx5_irq_pool_get(dev);
+ int i;
+
+ for (i = 0; i < num_irqs; i++) {
+ int cpu = cpumask_first(mlx5_irq_get_affinity_mask(irqs[i]));
+
+ synchronize_irq(pci_irq_vector(pool->dev->pdev,
+ mlx5_irq_get_index(irqs[i])));
+ if (mlx5_irq_put(irqs[i]))
+ if (pool->irqs_per_cpu)
+ cpu_put(pool, cpu);
+ }
+}
+
+/**
+ * mlx5_irq_affinity_irqs_request_auto - request one or more IRQs for mlx5 device.
+ * @dev: mlx5 device that is requesting the IRQs.
+ * @nirqs: number of IRQs to request.
+ * @irqs: an output array of IRQs pointers.
+ *
+ * Each IRQ is bounded to at most 1 CPU.
+ * This function is requesting IRQs according to the default assignment.
+ * The default assignment policy is:
+ * - in each iteration, request the least loaded IRQ which is not bound to any
+ * CPU of the previous IRQs requested.
+ *
+ * This function returns the number of IRQs requested, (which might be smaller than
+ * @nirqs), if successful, or a negative error code in case of an error.
+ */
+int mlx5_irq_affinity_irqs_request_auto(struct mlx5_core_dev *dev, int nirqs,
+ struct mlx5_irq **irqs)
+{
+ struct mlx5_irq_pool *pool = mlx5_irq_pool_get(dev);
+ cpumask_var_t req_mask;
+ struct mlx5_irq *irq;
+ int i = 0;
+
+ if (!zalloc_cpumask_var(&req_mask, GFP_KERNEL))
+ return -ENOMEM;
+ cpumask_copy(req_mask, cpu_online_mask);
+ for (i = 0; i < nirqs; i++) {
+ if (mlx5_irq_pool_is_sf_pool(pool))
+ irq = mlx5_irq_affinity_request(pool, req_mask);
+ else
+ /* In case SF pool doesn't exists, fallback to the PF IRQs.
+ * The PF IRQs are already allocated and binded to CPU
+ * at this point. Hence, only an index is needed.
+ */
+ irq = mlx5_irq_request(dev, i, NULL);
+ if (IS_ERR(irq))
+ break;
+ irqs[i] = irq;
+ cpumask_clear_cpu(cpumask_first(mlx5_irq_get_affinity_mask(irq)), req_mask);
+ mlx5_core_dbg(pool->dev, "IRQ %u mapped to cpu %*pbl, %u EQs on this irq\n",
+ pci_irq_vector(dev->pdev, mlx5_irq_get_index(irq)),
+ cpumask_pr_args(mlx5_irq_get_affinity_mask(irq)),
+ mlx5_irq_read_locked(irq) / MLX5_EQ_REFS_PER_IRQ);
+ }
+ free_cpumask_var(req_mask);
+ if (!i)
+ return PTR_ERR(irq);
+ return i;
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lag/mp.c b/drivers/net/ethernet/mellanox/mlx5/core/lag/mp.c
index bf4d3cb..1ca01a5 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/lag/mp.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/lag/mp.c
@@ -268,10 +268,8 @@ static int mlx5_lag_fib_event(struct notifier_block *nb,
fen_info = container_of(info, struct fib_entry_notifier_info,
info);
fi = fen_info->fi;
- if (fi->nh) {
- NL_SET_ERR_MSG_MOD(info->extack, "IPv4 route with nexthop objects is not supported");
- return notifier_from_errno(-EINVAL);
- }
+ if (fi->nh)
+ return NOTIFY_DONE;
fib_dev = fib_info_nh(fen_info->fi, 0)->fib_nh_dev;
if (fib_dev != ldev->pf[MLX5_LAG_P1].netdev &&
fib_dev != ldev->pf[MLX5_LAG_P2].netdev) {
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lib/fs_chains.c b/drivers/net/ethernet/mellanox/mlx5/core/lib/fs_chains.c
index 97e5845..d5e4763 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/lib/fs_chains.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/lib/fs_chains.c
@@ -121,6 +121,9 @@ u32 mlx5_chains_get_nf_ft_chain(struct mlx5_fs_chains *chains)
u32 mlx5_chains_get_prio_range(struct mlx5_fs_chains *chains)
{
+ if (!mlx5_chains_prios_supported(chains))
+ return 1;
+
if (mlx5_chains_ignore_flow_level_supported(chains))
return UINT_MAX;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c
index 7df9c7f..2c774f3 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c
@@ -98,6 +98,8 @@ enum {
MLX5_ATOMIC_REQ_MODE_HOST_ENDIANNESS = 0x1,
};
+#define LOG_MAX_SUPPORTED_QPS 0xff
+
static struct mlx5_profile profile[] = {
[0] = {
.mask = 0,
@@ -109,7 +111,7 @@ static struct mlx5_profile profile[] = {
[2] = {
.mask = MLX5_PROF_MASK_QP_SIZE |
MLX5_PROF_MASK_MR_CACHE,
- .log_max_qp = 18,
+ .log_max_qp = LOG_MAX_SUPPORTED_QPS,
.mr_cache[0] = {
.size = 500,
.limit = 250
@@ -484,10 +486,26 @@ static int handle_hca_cap_odp(struct mlx5_core_dev *dev, void *set_ctx)
return set_caps(dev, set_ctx, MLX5_SET_HCA_CAP_OP_MOD_ODP);
}
+static int max_uc_list_get_devlink_param(struct mlx5_core_dev *dev)
+{
+ struct devlink *devlink = priv_to_devlink(dev);
+ union devlink_param_value val;
+ int err;
+
+ err = devlink_param_driverinit_value_get(devlink,
+ DEVLINK_PARAM_GENERIC_ID_MAX_MACS,
+ &val);
+ if (!err)
+ return val.vu32;
+ mlx5_core_dbg(dev, "Failed to get param. err = %d\n", err);
+ return err;
+}
+
static int handle_hca_cap(struct mlx5_core_dev *dev, void *set_ctx)
{
struct mlx5_profile *prof = &dev->profile;
void *set_hca_cap;
+ int max_uc_list;
int err;
err = mlx5_core_get_caps(dev, MLX5_CAP_GENERAL);
@@ -507,7 +525,9 @@ static int handle_hca_cap(struct mlx5_core_dev *dev, void *set_ctx)
to_fw_pkey_sz(dev, 128));
/* Check log_max_qp from HCA caps to set in current profile */
- if (MLX5_CAP_GEN_MAX(dev, log_max_qp) < prof->log_max_qp) {
+ if (prof->log_max_qp == LOG_MAX_SUPPORTED_QPS) {
+ prof->log_max_qp = MLX5_CAP_GEN_MAX(dev, log_max_qp);
+ } else if (MLX5_CAP_GEN_MAX(dev, log_max_qp) < prof->log_max_qp) {
mlx5_core_warn(dev, "log_max_qp value in current profile is %d, changing it to HCA capability limit (%d)\n",
prof->log_max_qp,
MLX5_CAP_GEN_MAX(dev, log_max_qp));
@@ -561,6 +581,11 @@ static int handle_hca_cap(struct mlx5_core_dev *dev, void *set_ctx)
if (MLX5_CAP_GEN(dev, roce_rw_supported))
MLX5_SET(cmd_hca_cap, set_hca_cap, roce, mlx5_is_roce_init_enabled(dev));
+ max_uc_list = max_uc_list_get_devlink_param(dev);
+ if (max_uc_list > 0)
+ MLX5_SET(cmd_hca_cap, set_hca_cap, log_max_current_uc_list,
+ ilog2(max_uc_list));
+
return set_caps(dev, set_ctx, MLX5_SET_HCA_CAP_OP_MOD_GENERAL_DEVICE);
}
@@ -1604,12 +1629,28 @@ static void remove_one(struct pci_dev *pdev)
mlx5_devlink_free(devlink);
}
+#define mlx5_pci_trace(dev, fmt, ...) ({ \
+ struct mlx5_core_dev *__dev = (dev); \
+ mlx5_core_info(__dev, "%s Device state = %d health sensors: %d pci_status: %d. " fmt, \
+ __func__, __dev->state, mlx5_health_check_fatal_sensors(__dev), \
+ __dev->pci_status, ##__VA_ARGS__); \
+})
+
+static const char *result2str(enum pci_ers_result result)
+{
+ return result == PCI_ERS_RESULT_NEED_RESET ? "need reset" :
+ result == PCI_ERS_RESULT_DISCONNECT ? "disconnect" :
+ result == PCI_ERS_RESULT_RECOVERED ? "recovered" :
+ "unknown";
+}
+
static pci_ers_result_t mlx5_pci_err_detected(struct pci_dev *pdev,
pci_channel_state_t state)
{
struct mlx5_core_dev *dev = pci_get_drvdata(pdev);
+ enum pci_ers_result res;
- mlx5_core_info(dev, "%s was called\n", __func__);
+ mlx5_pci_trace(dev, "Enter, pci channel state = %d\n", state);
mlx5_enter_error_state(dev, false);
mlx5_error_sw_reset(dev);
@@ -1617,8 +1658,11 @@ static pci_ers_result_t mlx5_pci_err_detected(struct pci_dev *pdev,
mlx5_drain_health_wq(dev);
mlx5_pci_disable_device(dev);
- return state == pci_channel_io_perm_failure ?
+ res = state == pci_channel_io_perm_failure ?
PCI_ERS_RESULT_DISCONNECT : PCI_ERS_RESULT_NEED_RESET;
+
+ mlx5_pci_trace(dev, "Exit, result = %d, %s\n", res, result2str(res));
+ return res;
}
/* wait for the device to show vital signs by waiting
@@ -1652,28 +1696,34 @@ static int wait_vital(struct pci_dev *pdev)
static pci_ers_result_t mlx5_pci_slot_reset(struct pci_dev *pdev)
{
+ enum pci_ers_result res = PCI_ERS_RESULT_DISCONNECT;
struct mlx5_core_dev *dev = pci_get_drvdata(pdev);
int err;
- mlx5_core_info(dev, "%s was called\n", __func__);
+ mlx5_pci_trace(dev, "Enter\n");
err = mlx5_pci_enable_device(dev);
if (err) {
mlx5_core_err(dev, "%s: mlx5_pci_enable_device failed with error code: %d\n",
__func__, err);
- return PCI_ERS_RESULT_DISCONNECT;
+ goto out;
}
pci_set_master(pdev);
pci_restore_state(pdev);
pci_save_state(pdev);
- if (wait_vital(pdev)) {
- mlx5_core_err(dev, "%s: wait_vital timed out\n", __func__);
- return PCI_ERS_RESULT_DISCONNECT;
+ err = wait_vital(pdev);
+ if (err) {
+ mlx5_core_err(dev, "%s: wait vital failed with error code: %d\n",
+ __func__, err);
+ goto out;
}
- return PCI_ERS_RESULT_RECOVERED;
+ res = PCI_ERS_RESULT_RECOVERED;
+out:
+ mlx5_pci_trace(dev, "Exit, err = %d, result = %d, %s\n", err, res, result2str(res));
+ return res;
}
static void mlx5_pci_resume(struct pci_dev *pdev)
@@ -1681,14 +1731,12 @@ static void mlx5_pci_resume(struct pci_dev *pdev)
struct mlx5_core_dev *dev = pci_get_drvdata(pdev);
int err;
- mlx5_core_info(dev, "%s was called\n", __func__);
+ mlx5_pci_trace(dev, "Enter, loading driver..\n");
err = mlx5_load_one(dev);
- if (err)
- mlx5_core_err(dev, "%s: mlx5_load_one failed with error code: %d\n",
- __func__, err);
- else
- mlx5_core_info(dev, "%s: device recovered\n", __func__);
+
+ mlx5_pci_trace(dev, "Done, err = %d, device %s\n", err,
+ !err ? "recovered" : "Failed");
}
static const struct pci_error_handlers mlx5_err_handler = {
@@ -1809,12 +1857,13 @@ void mlx5_disable_device(struct mlx5_core_dev *dev)
int mlx5_recover_device(struct mlx5_core_dev *dev)
{
- int ret = -EIO;
+ if (!mlx5_core_is_sf(dev)) {
+ mlx5_pci_disable_device(dev);
+ if (mlx5_pci_slot_reset(dev->pdev) != PCI_ERS_RESULT_RECOVERED)
+ return -EIO;
+ }
- mlx5_pci_disable_device(dev);
- if (mlx5_pci_slot_reset(dev->pdev) == PCI_ERS_RESULT_RECOVERED)
- ret = mlx5_load_one(dev);
- return ret;
+ return mlx5_load_one(dev);
}
static struct pci_driver mlx5_core_driver = {
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h b/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h
index bb67732..6f8baa0 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h
@@ -305,5 +305,6 @@ static inline u32 mlx5_sriov_get_vf_total_msix(struct pci_dev *pdev)
bool mlx5_eth_supported(struct mlx5_core_dev *dev);
bool mlx5_rdma_supported(struct mlx5_core_dev *dev);
bool mlx5_vnet_supported(struct mlx5_core_dev *dev);
+bool mlx5_same_hw_devs(struct mlx5_core_dev *dev, struct mlx5_core_dev *peer_dev);
#endif /* __MLX5_CORE_H__ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/mlx5_irq.h b/drivers/net/ethernet/mellanox/mlx5/core/mlx5_irq.h
index 8116815..23cb63f 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/mlx5_irq.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/mlx5_irq.h
@@ -22,12 +22,40 @@ int mlx5_set_msix_vec_count(struct mlx5_core_dev *dev, int devfn,
int msix_vec_count);
int mlx5_get_default_msix_vec_count(struct mlx5_core_dev *dev, int num_vfs);
+struct mlx5_irq *mlx5_ctrl_irq_request(struct mlx5_core_dev *dev);
+void mlx5_ctrl_irq_release(struct mlx5_irq *ctrl_irq);
struct mlx5_irq *mlx5_irq_request(struct mlx5_core_dev *dev, u16 vecidx,
struct cpumask *affinity);
-void mlx5_irq_release(struct mlx5_irq *irq);
+int mlx5_irqs_request_vectors(struct mlx5_core_dev *dev, u16 *cpus, int nirqs,
+ struct mlx5_irq **irqs);
+void mlx5_irqs_release_vectors(struct mlx5_irq **irqs, int nirqs);
int mlx5_irq_attach_nb(struct mlx5_irq *irq, struct notifier_block *nb);
int mlx5_irq_detach_nb(struct mlx5_irq *irq, struct notifier_block *nb);
struct cpumask *mlx5_irq_get_affinity_mask(struct mlx5_irq *irq);
int mlx5_irq_get_index(struct mlx5_irq *irq);
+struct mlx5_irq_pool;
+#ifdef CONFIG_MLX5_SF
+int mlx5_irq_affinity_irqs_request_auto(struct mlx5_core_dev *dev, int nirqs,
+ struct mlx5_irq **irqs);
+struct mlx5_irq *mlx5_irq_affinity_request(struct mlx5_irq_pool *pool,
+ const struct cpumask *req_mask);
+void mlx5_irq_affinity_irqs_release(struct mlx5_core_dev *dev, struct mlx5_irq **irqs,
+ int num_irqs);
+#else
+static inline int mlx5_irq_affinity_irqs_request_auto(struct mlx5_core_dev *dev, int nirqs,
+ struct mlx5_irq **irqs)
+{
+ return -EOPNOTSUPP;
+}
+
+static inline struct mlx5_irq *
+mlx5_irq_affinity_request(struct mlx5_irq_pool *pool, const struct cpumask *req_mask)
+{
+ return ERR_PTR(-EOPNOTSUPP);
+}
+
+static inline void mlx5_irq_affinity_irqs_release(struct mlx5_core_dev *dev,
+ struct mlx5_irq **irqs, int num_irqs) {}
+#endif
#endif /* __MLX5_IRQ_H__ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c b/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c
index 830444f..90fec06 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c
@@ -7,15 +7,12 @@
#include <linux/mlx5/driver.h>
#include "mlx5_core.h"
#include "mlx5_irq.h"
+#include "pci_irq.h"
#include "lib/sf.h"
#ifdef CONFIG_RFS_ACCEL
#include <linux/cpu_rmap.h>
#endif
-#define MLX5_MAX_IRQ_NAME (32)
-/* max irq_index is 2047, so four chars */
-#define MLX5_MAX_IRQ_IDX_CHARS (4)
-
#define MLX5_SFS_PER_CTRL_IRQ 64
#define MLX5_IRQ_CTRL_SF_MAX 8
/* min num of vectors for SFs to be enabled */
@@ -25,7 +22,6 @@
#define MLX5_EQ_SHARE_IRQ_MAX_CTRL (UINT_MAX)
#define MLX5_EQ_SHARE_IRQ_MIN_COMP (1)
#define MLX5_EQ_SHARE_IRQ_MIN_CTRL (4)
-#define MLX5_EQ_REFS_PER_IRQ (2)
struct mlx5_irq {
struct atomic_notifier_head nh;
@@ -37,16 +33,6 @@ struct mlx5_irq {
int irqn;
};
-struct mlx5_irq_pool {
- char name[MLX5_MAX_IRQ_NAME - MLX5_MAX_IRQ_IDX_CHARS];
- struct xa_limit xa_num_irqs;
- struct mutex lock; /* sync IRQs creations */
- struct xarray irqs;
- u32 max_threshold;
- u32 min_threshold;
- struct mlx5_core_dev *dev;
-};
-
struct mlx5_irq_table {
struct mlx5_irq_pool *pf_pool;
struct mlx5_irq_pool *sf_ctrl_pool;
@@ -153,18 +139,28 @@ static void irq_release(struct mlx5_irq *irq)
kfree(irq);
}
-static void irq_put(struct mlx5_irq *irq)
+int mlx5_irq_put(struct mlx5_irq *irq)
{
struct mlx5_irq_pool *pool = irq->pool;
+ int ret = 0;
mutex_lock(&pool->lock);
irq->refcount--;
- if (!irq->refcount)
+ if (!irq->refcount) {
irq_release(irq);
+ ret = 1;
+ }
mutex_unlock(&pool->lock);
+ return ret;
}
-static int irq_get_locked(struct mlx5_irq *irq)
+int mlx5_irq_read_locked(struct mlx5_irq *irq)
+{
+ lockdep_assert_held(&irq->pool->lock);
+ return irq->refcount;
+}
+
+int mlx5_irq_get_locked(struct mlx5_irq *irq)
{
lockdep_assert_held(&irq->pool->lock);
if (WARN_ON_ONCE(!irq->refcount))
@@ -178,7 +174,7 @@ static int irq_get(struct mlx5_irq *irq)
int err;
mutex_lock(&irq->pool->lock);
- err = irq_get_locked(irq);
+ err = mlx5_irq_get_locked(irq);
mutex_unlock(&irq->pool->lock);
return err;
}
@@ -210,12 +206,8 @@ static void irq_set_name(struct mlx5_irq_pool *pool, char *name, int vecidx)
snprintf(name, MLX5_MAX_IRQ_NAME, "mlx5_comp%d", vecidx);
}
-static bool irq_pool_is_sf_pool(struct mlx5_irq_pool *pool)
-{
- return !strncmp("mlx5_sf", pool->name, strlen("mlx5_sf"));
-}
-
-static struct mlx5_irq *irq_request(struct mlx5_irq_pool *pool, int i)
+struct mlx5_irq *mlx5_irq_alloc(struct mlx5_irq_pool *pool, int i,
+ const struct cpumask *affinity)
{
struct mlx5_core_dev *dev = pool->dev;
char name[MLX5_MAX_IRQ_NAME];
@@ -226,7 +218,7 @@ static struct mlx5_irq *irq_request(struct mlx5_irq_pool *pool, int i)
if (!irq)
return ERR_PTR(-ENOMEM);
irq->irqn = pci_irq_vector(dev->pdev, i);
- if (!irq_pool_is_sf_pool(pool))
+ if (!mlx5_irq_pool_is_sf_pool(pool))
irq_set_name(pool, name, i);
else
irq_sf_set_name(pool, name, i);
@@ -244,6 +236,10 @@ static struct mlx5_irq *irq_request(struct mlx5_irq_pool *pool, int i)
err = -ENOMEM;
goto err_cpumask;
}
+ if (affinity) {
+ cpumask_copy(irq->mask, affinity);
+ irq_set_affinity_hint(irq->irqn, irq->mask);
+ }
irq->pool = pool;
irq->refcount = 1;
irq->index = i;
@@ -255,6 +251,7 @@ static struct mlx5_irq *irq_request(struct mlx5_irq_pool *pool, int i)
}
return irq;
err_xa:
+ irq_set_affinity_hint(irq->irqn, NULL);
free_cpumask_var(irq->mask);
err_cpumask:
free_irq(irq->irqn, &irq->nh);
@@ -275,7 +272,7 @@ int mlx5_irq_attach_nb(struct mlx5_irq *irq, struct notifier_block *nb)
return -ENOENT;
ret = atomic_notifier_chain_register(&irq->nh, nb);
if (ret)
- irq_put(irq);
+ mlx5_irq_put(irq);
return ret;
}
@@ -284,7 +281,7 @@ int mlx5_irq_detach_nb(struct mlx5_irq *irq, struct notifier_block *nb)
int err = 0;
err = atomic_notifier_chain_unregister(&irq->nh, nb);
- irq_put(irq);
+ mlx5_irq_put(irq);
return err;
}
@@ -300,85 +297,6 @@ int mlx5_irq_get_index(struct mlx5_irq *irq)
/* irq_pool API */
-/* creating an irq from irq_pool */
-static struct mlx5_irq *irq_pool_create_irq(struct mlx5_irq_pool *pool,
- struct cpumask *affinity)
-{
- struct mlx5_irq *irq;
- u32 irq_index;
- int err;
-
- err = xa_alloc(&pool->irqs, &irq_index, NULL, pool->xa_num_irqs,
- GFP_KERNEL);
- if (err)
- return ERR_PTR(err);
- irq = irq_request(pool, irq_index);
- if (IS_ERR(irq))
- return irq;
- cpumask_copy(irq->mask, affinity);
- irq_set_affinity_hint(irq->irqn, irq->mask);
- return irq;
-}
-
-/* looking for the irq with the smallest refcount and the same affinity */
-static struct mlx5_irq *irq_pool_find_least_loaded(struct mlx5_irq_pool *pool,
- struct cpumask *affinity)
-{
- int start = pool->xa_num_irqs.min;
- int end = pool->xa_num_irqs.max;
- struct mlx5_irq *irq = NULL;
- struct mlx5_irq *iter;
- unsigned long index;
-
- lockdep_assert_held(&pool->lock);
- xa_for_each_range(&pool->irqs, index, iter, start, end) {
- if (!cpumask_equal(iter->mask, affinity))
- continue;
- if (iter->refcount < pool->min_threshold)
- return iter;
- if (!irq || iter->refcount < irq->refcount)
- irq = iter;
- }
- return irq;
-}
-
-/* requesting an irq from a given pool according to given affinity */
-static struct mlx5_irq *irq_pool_request_affinity(struct mlx5_irq_pool *pool,
- struct cpumask *affinity)
-{
- struct mlx5_irq *least_loaded_irq, *new_irq;
-
- mutex_lock(&pool->lock);
- least_loaded_irq = irq_pool_find_least_loaded(pool, affinity);
- if (least_loaded_irq &&
- least_loaded_irq->refcount < pool->min_threshold)
- goto out;
- new_irq = irq_pool_create_irq(pool, affinity);
- if (IS_ERR(new_irq)) {
- if (!least_loaded_irq) {
- mlx5_core_err(pool->dev, "Didn't find IRQ for cpu = %u\n",
- cpumask_first(affinity));
- mutex_unlock(&pool->lock);
- return new_irq;
- }
- /* We failed to create a new IRQ for the requested affinity,
- * sharing existing IRQ.
- */
- goto out;
- }
- least_loaded_irq = new_irq;
- goto unlock;
-out:
- irq_get_locked(least_loaded_irq);
- if (least_loaded_irq->refcount > pool->max_threshold)
- mlx5_core_dbg(pool->dev, "IRQ %u overloaded, pool_name: %s, %u EQs on this irq\n",
- least_loaded_irq->irqn, pool->name,
- least_loaded_irq->refcount / MLX5_EQ_REFS_PER_IRQ);
-unlock:
- mutex_unlock(&pool->lock);
- return least_loaded_irq;
-}
-
/* requesting an irq from a given pool according to given index */
static struct mlx5_irq *
irq_pool_request_vector(struct mlx5_irq_pool *pool, int vecidx,
@@ -389,42 +307,111 @@ irq_pool_request_vector(struct mlx5_irq_pool *pool, int vecidx,
mutex_lock(&pool->lock);
irq = xa_load(&pool->irqs, vecidx);
if (irq) {
- irq_get_locked(irq);
+ mlx5_irq_get_locked(irq);
goto unlock;
}
- irq = irq_request(pool, vecidx);
- if (IS_ERR(irq) || !affinity)
- goto unlock;
- cpumask_copy(irq->mask, affinity);
- if (!irq_pool_is_sf_pool(pool) && !pool->xa_num_irqs.max &&
- cpumask_empty(irq->mask))
- cpumask_set_cpu(0, irq->mask);
- irq_set_affinity_hint(irq->irqn, irq->mask);
+ irq = mlx5_irq_alloc(pool, vecidx, affinity);
unlock:
mutex_unlock(&pool->lock);
return irq;
}
-static struct mlx5_irq_pool *find_sf_irq_pool(struct mlx5_irq_table *irq_table,
- int i, struct cpumask *affinity)
+static struct mlx5_irq_pool *sf_ctrl_irq_pool_get(struct mlx5_irq_table *irq_table)
{
- if (cpumask_empty(affinity) && i == MLX5_IRQ_EQ_CTRL)
- return irq_table->sf_ctrl_pool;
+ return irq_table->sf_ctrl_pool;
+}
+
+static struct mlx5_irq_pool *sf_irq_pool_get(struct mlx5_irq_table *irq_table)
+{
return irq_table->sf_comp_pool;
}
-/**
- * mlx5_irq_release - release an IRQ back to the system.
- * @irq: irq to be released.
- */
-void mlx5_irq_release(struct mlx5_irq *irq)
+struct mlx5_irq_pool *mlx5_irq_pool_get(struct mlx5_core_dev *dev)
{
- synchronize_irq(irq->irqn);
- irq_put(irq);
+ struct mlx5_irq_table *irq_table = mlx5_irq_table_get(dev);
+ struct mlx5_irq_pool *pool = NULL;
+
+ if (mlx5_core_is_sf(dev))
+ pool = sf_irq_pool_get(irq_table);
+
+ /* In some configs, there won't be a pool of SFs IRQs. Hence, returning
+ * the PF IRQs pool in case the SF pool doesn't exist.
+ */
+ return pool ? pool : irq_table->pf_pool;
+}
+
+static struct mlx5_irq_pool *ctrl_irq_pool_get(struct mlx5_core_dev *dev)
+{
+ struct mlx5_irq_table *irq_table = mlx5_irq_table_get(dev);
+ struct mlx5_irq_pool *pool = NULL;
+
+ if (mlx5_core_is_sf(dev))
+ pool = sf_ctrl_irq_pool_get(irq_table);
+
+ /* In some configs, there won't be a pool of SFs IRQs. Hence, returning
+ * the PF IRQs pool in case the SF pool doesn't exist.
+ */
+ return pool ? pool : irq_table->pf_pool;
}
/**
- * mlx5_irq_request - request an IRQ for mlx5 device.
+ * mlx5_irqs_release - release one or more IRQs back to the system.
+ * @irqs: IRQs to be released.
+ * @nirqs: number of IRQs to be released.
+ */
+static void mlx5_irqs_release(struct mlx5_irq **irqs, int nirqs)
+{
+ int i;
+
+ for (i = 0; i < nirqs; i++) {
+ synchronize_irq(irqs[i]->irqn);
+ mlx5_irq_put(irqs[i]);
+ }
+}
+
+/**
+ * mlx5_ctrl_irq_release - release a ctrl IRQ back to the system.
+ * @ctrl_irq: ctrl IRQ to be released.
+ */
+void mlx5_ctrl_irq_release(struct mlx5_irq *ctrl_irq)
+{
+ mlx5_irqs_release(&ctrl_irq, 1);
+}
+
+/**
+ * mlx5_ctrl_irq_request - request a ctrl IRQ for mlx5 device.
+ * @dev: mlx5 device that requesting the IRQ.
+ *
+ * This function returns a pointer to IRQ, or ERR_PTR in case of error.
+ */
+struct mlx5_irq *mlx5_ctrl_irq_request(struct mlx5_core_dev *dev)
+{
+ struct mlx5_irq_pool *pool = ctrl_irq_pool_get(dev);
+ cpumask_var_t req_mask;
+ struct mlx5_irq *irq;
+
+ if (!zalloc_cpumask_var(&req_mask, GFP_KERNEL))
+ return ERR_PTR(-ENOMEM);
+ cpumask_copy(req_mask, cpu_online_mask);
+ if (!mlx5_irq_pool_is_sf_pool(pool)) {
+ /* In case we are allocating a control IRQ for PF/VF */
+ if (!pool->xa_num_irqs.max) {
+ cpumask_clear(req_mask);
+ /* In case we only have a single IRQ for PF/VF */
+ cpumask_set_cpu(cpumask_first(cpu_online_mask), req_mask);
+ }
+ /* Allocate the IRQ in the last index of the pool */
+ irq = irq_pool_request_vector(pool, pool->xa_num_irqs.max, req_mask);
+ } else {
+ irq = mlx5_irq_affinity_request(pool, req_mask);
+ }
+
+ free_cpumask_var(req_mask);
+ return irq;
+}
+
+/**
+ * mlx5_irq_request - request an IRQ for mlx5 PF/VF device.
* @dev: mlx5 device that requesting the IRQ.
* @vecidx: vector index of the IRQ. This argument is ignore if affinity is
* provided.
@@ -439,23 +426,8 @@ struct mlx5_irq *mlx5_irq_request(struct mlx5_core_dev *dev, u16 vecidx,
struct mlx5_irq_pool *pool;
struct mlx5_irq *irq;
- if (mlx5_core_is_sf(dev)) {
- pool = find_sf_irq_pool(irq_table, vecidx, affinity);
- if (!pool)
- /* we don't have IRQs for SFs, using the PF IRQs */
- goto pf_irq;
- if (cpumask_empty(affinity) && !strcmp(pool->name, "mlx5_sf_comp"))
- /* In case an SF user request IRQ with vecidx */
- irq = irq_pool_request_vector(pool, vecidx, NULL);
- else
- irq = irq_pool_request_affinity(pool, affinity);
- goto out;
- }
-pf_irq:
pool = irq_table->pf_pool;
- vecidx = (vecidx == MLX5_IRQ_EQ_CTRL) ? pool->xa_num_irqs.max : vecidx;
irq = irq_pool_request_vector(pool, vecidx, affinity);
-out:
if (IS_ERR(irq))
return irq;
mlx5_core_dbg(dev, "irq %u mapped to cpu %*pbl, %u EQs on this irq\n",
@@ -464,6 +436,51 @@ struct mlx5_irq *mlx5_irq_request(struct mlx5_core_dev *dev, u16 vecidx,
return irq;
}
+/**
+ * mlx5_irqs_release_vectors - release one or more IRQs back to the system.
+ * @irqs: IRQs to be released.
+ * @nirqs: number of IRQs to be released.
+ */
+void mlx5_irqs_release_vectors(struct mlx5_irq **irqs, int nirqs)
+{
+ mlx5_irqs_release(irqs, nirqs);
+}
+
+/**
+ * mlx5_irqs_request_vectors - request one or more IRQs for mlx5 device.
+ * @dev: mlx5 device that is requesting the IRQs.
+ * @cpus: CPUs array for binding the IRQs
+ * @nirqs: number of IRQs to request.
+ * @irqs: an output array of IRQs pointers.
+ *
+ * Each IRQ is bound to at most 1 CPU.
+ * This function is requests nirqs IRQs, starting from @vecidx.
+ *
+ * This function returns the number of IRQs requested, (which might be smaller than
+ * @nirqs), if successful, or a negative error code in case of an error.
+ */
+int mlx5_irqs_request_vectors(struct mlx5_core_dev *dev, u16 *cpus, int nirqs,
+ struct mlx5_irq **irqs)
+{
+ cpumask_var_t req_mask;
+ struct mlx5_irq *irq;
+ int i;
+
+ if (!zalloc_cpumask_var(&req_mask, GFP_KERNEL))
+ return -ENOMEM;
+ for (i = 0; i < nirqs; i++) {
+ cpumask_set_cpu(cpus[i], req_mask);
+ irq = mlx5_irq_request(dev, i, req_mask);
+ if (IS_ERR(irq))
+ break;
+ cpumask_clear(req_mask);
+ irqs[i] = irq;
+ }
+
+ free_cpumask_var(req_mask);
+ return i ? i : PTR_ERR(irq);
+}
+
static struct mlx5_irq_pool *
irq_pool_alloc(struct mlx5_core_dev *dev, int start, int size, char *name,
u32 min_threshold, u32 max_threshold)
@@ -479,7 +496,7 @@ irq_pool_alloc(struct mlx5_core_dev *dev, int start, int size, char *name,
pool->xa_num_irqs.max = start + size - 1;
if (name)
snprintf(pool->name, MLX5_MAX_IRQ_NAME - MLX5_MAX_IRQ_IDX_CHARS,
- name);
+ "%s", name);
pool->min_threshold = min_threshold * MLX5_EQ_REFS_PER_IRQ;
pool->max_threshold = max_threshold * MLX5_EQ_REFS_PER_IRQ;
mlx5_core_dbg(dev, "pool->name = %s, pool->size = %d, pool->start = %d",
@@ -500,6 +517,7 @@ static void irq_pool_free(struct mlx5_irq_pool *pool)
irq_release(irq);
xa_destroy(&pool->irqs);
mutex_destroy(&pool->lock);
+ kfree(pool->irqs_per_cpu);
kvfree(pool);
}
@@ -547,7 +565,17 @@ static int irq_pools_init(struct mlx5_core_dev *dev, int sf_vec, int pf_vec)
err = PTR_ERR(table->sf_comp_pool);
goto err_sf_ctrl;
}
+
+ table->sf_comp_pool->irqs_per_cpu = kcalloc(nr_cpu_ids, sizeof(u16), GFP_KERNEL);
+ if (!table->sf_comp_pool->irqs_per_cpu) {
+ err = -ENOMEM;
+ goto err_irqs_per_cpu;
+ }
+
return 0;
+
+err_irqs_per_cpu:
+ irq_pool_free(table->sf_comp_pool);
err_sf_ctrl:
irq_pool_free(table->sf_ctrl_pool);
err_pf:
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.h b/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.h
new file mode 100644
index 0000000..5c7e68b
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.h
@@ -0,0 +1,39 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+/* Copyright (c) 2021, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */
+
+#ifndef __PCI_IRQ_H__
+#define __PCI_IRQ_H__
+
+#include <linux/mlx5/driver.h>
+
+#define MLX5_MAX_IRQ_NAME (32)
+/* max irq_index is 2047, so four chars */
+#define MLX5_MAX_IRQ_IDX_CHARS (4)
+#define MLX5_EQ_REFS_PER_IRQ (2)
+
+struct mlx5_irq;
+
+struct mlx5_irq_pool {
+ char name[MLX5_MAX_IRQ_NAME - MLX5_MAX_IRQ_IDX_CHARS];
+ struct xa_limit xa_num_irqs;
+ struct mutex lock; /* sync IRQs creations */
+ struct xarray irqs;
+ u32 max_threshold;
+ u32 min_threshold;
+ u16 *irqs_per_cpu;
+ struct mlx5_core_dev *dev;
+};
+
+struct mlx5_irq_pool *mlx5_irq_pool_get(struct mlx5_core_dev *dev);
+static inline bool mlx5_irq_pool_is_sf_pool(struct mlx5_irq_pool *pool)
+{
+ return !strncmp("mlx5_sf", pool->name, strlen("mlx5_sf"));
+}
+
+struct mlx5_irq *mlx5_irq_alloc(struct mlx5_irq_pool *pool, int i,
+ const struct cpumask *affinity);
+int mlx5_irq_get_locked(struct mlx5_irq *irq);
+int mlx5_irq_read_locked(struct mlx5_irq *irq);
+int mlx5_irq_put(struct mlx5_irq *irq);
+
+#endif /* __PCI_IRQ_H__ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/sf/dev/dev.c b/drivers/net/ethernet/mellanox/mlx5/core/sf/dev/dev.c
index f37db7c..7da012f 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/sf/dev/dev.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/sf/dev/dev.c
@@ -30,10 +30,7 @@ bool mlx5_sf_dev_allocated(const struct mlx5_core_dev *dev)
{
struct mlx5_sf_dev_table *table = dev->priv.sf_dev_table;
- if (!mlx5_sf_dev_supported(dev))
- return false;
-
- return !xa_empty(&table->devices);
+ return table && !xa_empty(&table->devices);
}
static ssize_t sfnum_show(struct device *dev, struct device_attribute *attr, char *buf)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/sf/hw_table.c b/drivers/net/ethernet/mellanox/mlx5/core/sf/hw_table.c
index 252d601..17aa348 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/sf/hw_table.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/sf/hw_table.c
@@ -247,7 +247,7 @@ int mlx5_sf_hw_table_init(struct mlx5_core_dev *dev)
{
struct mlx5_sf_hw_table *table;
u16 max_ext_fn = 0;
- u16 ext_base_id;
+ u16 ext_base_id = 0;
u16 max_fn = 0;
u16 base_id;
int err;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_action.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_action.c
index 0793684..c61a5e8 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_action.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_action.c
@@ -1560,6 +1560,12 @@ dr_action_modify_check_is_ttl_modify(const void *sw_action)
return sw_field == MLX5_ACTION_IN_FIELD_OUT_IP_TTL;
}
+static bool dr_action_modify_ttl_ignore(struct mlx5dr_domain *dmn)
+{
+ return !mlx5dr_ste_supp_ttl_cs_recalc(&dmn->info.caps) &&
+ !MLX5_CAP_ESW_FLOWTABLE(dmn->mdev, fdb_ipv4_ttl_modify);
+}
+
static int dr_actions_convert_modify_header(struct mlx5dr_action *action,
u32 max_hw_actions,
u32 num_sw_actions,
@@ -1591,8 +1597,13 @@ static int dr_actions_convert_modify_header(struct mlx5dr_action *action,
if (ret)
return ret;
- if (!(*modify_ttl))
- *modify_ttl = dr_action_modify_check_is_ttl_modify(sw_action);
+ if (!(*modify_ttl) &&
+ dr_action_modify_check_is_ttl_modify(sw_action)) {
+ if (dr_action_modify_ttl_ignore(dmn))
+ continue;
+
+ *modify_ttl = true;
+ }
/* Convert SW action to HW action */
ret = dr_action_modify_sw_to_hw(dmn,
@@ -1631,7 +1642,7 @@ static int dr_actions_convert_modify_header(struct mlx5dr_action *action,
* modify actions doesn't exceeds the limit
*/
hw_idx++;
- if ((num_sw_actions + hw_idx - i) >= max_hw_actions) {
+ if (hw_idx >= max_hw_actions) {
mlx5dr_dbg(dmn, "Modify header action number exceeds HW limit\n");
return -EINVAL;
}
@@ -1642,6 +1653,10 @@ static int dr_actions_convert_modify_header(struct mlx5dr_action *action,
hw_idx++;
}
+ /* if the resulting HW actions list is empty, add NOP action */
+ if (!hw_idx)
+ hw_idx++;
+
*num_hw_actions = hw_idx;
return 0;
@@ -1792,7 +1807,7 @@ mlx5dr_action_create_dest_vport(struct mlx5dr_domain *dmn,
int mlx5dr_action_destroy(struct mlx5dr_action *action)
{
- if (refcount_read(&action->refcount) > 1)
+ if (WARN_ON_ONCE(refcount_read(&action->refcount) > 1))
return -EBUSY;
switch (action->action_type) {
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_cmd.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_cmd.c
index 1d8febe..4dd619d 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_cmd.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_cmd.c
@@ -132,6 +132,13 @@ int mlx5dr_cmd_query_device(struct mlx5_core_dev *mdev,
caps->isolate_vl_tc = MLX5_CAP_GEN(mdev, isolate_vl_tc_new);
+ /* geneve_tlv_option_0_exist is the indication of
+ * STE support for lookup type flex_parser_ok
+ */
+ caps->flex_parser_ok_bits_supp =
+ MLX5_CAP_FLOWTABLE(mdev,
+ flow_table_properties_nic_receive.ft_field_support.geneve_tlv_option_0_exist);
+
if (caps->flex_protocols & MLX5_FLEX_PARSER_ICMP_V4_ENABLED) {
caps->flex_parser_id_icmp_dw0 = MLX5_CAP_GEN(mdev, flex_parser_id_icmp_dw0);
caps->flex_parser_id_icmp_dw1 = MLX5_CAP_GEN(mdev, flex_parser_id_icmp_dw1);
@@ -152,7 +159,7 @@ int mlx5dr_cmd_query_device(struct mlx5_core_dev *mdev,
caps->flex_parser_id_mpls_over_gre =
MLX5_CAP_GEN(mdev, flex_parser_id_outer_first_mpls_over_gre);
- if (caps->flex_protocols & mlx5_FLEX_PARSER_MPLS_OVER_UDP_ENABLED)
+ if (caps->flex_protocols & MLX5_FLEX_PARSER_MPLS_OVER_UDP_ENABLED)
caps->flex_parser_id_mpls_over_udp =
MLX5_CAP_GEN(mdev, flex_parser_id_outer_first_mpls_over_udp_label);
@@ -599,7 +606,8 @@ static int mlx5dr_cmd_set_extended_dest(struct mlx5_core_dev *dev,
for (i = 0; i < fte->dests_size; i++) {
if (fte->dest_arr[i].type == MLX5_FLOW_DESTINATION_TYPE_COUNTER)
continue;
- if (fte->dest_arr[i].type == MLX5_FLOW_DESTINATION_TYPE_VPORT &&
+ if ((fte->dest_arr[i].type == MLX5_FLOW_DESTINATION_TYPE_VPORT ||
+ fte->dest_arr[i].type == MLX5_FLOW_DESTINATION_TYPE_UPLINK) &&
fte->dest_arr[i].vport.flags & MLX5_FLOW_DEST_VPORT_REFORMAT_ID)
num_encap++;
num_fwd_destinations++;
@@ -724,12 +732,19 @@ int mlx5dr_cmd_set_fte(struct mlx5_core_dev *dev,
case MLX5_FLOW_DESTINATION_TYPE_FLOW_TABLE:
id = fte->dest_arr[i].ft_id;
break;
+ case MLX5_FLOW_DESTINATION_TYPE_UPLINK:
case MLX5_FLOW_DESTINATION_TYPE_VPORT:
- id = fte->dest_arr[i].vport.num;
- MLX5_SET(dest_format_struct, in_dests,
- destination_eswitch_owner_vhca_id_valid,
- !!(fte->dest_arr[i].vport.flags &
- MLX5_FLOW_DEST_VPORT_VHCA_ID));
+ if (type == MLX5_FLOW_DESTINATION_TYPE_VPORT) {
+ id = fte->dest_arr[i].vport.num;
+ MLX5_SET(dest_format_struct, in_dests,
+ destination_eswitch_owner_vhca_id_valid,
+ !!(fte->dest_arr[i].vport.flags &
+ MLX5_FLOW_DEST_VPORT_VHCA_ID));
+ } else {
+ id = 0;
+ MLX5_SET(dest_format_struct, in_dests,
+ destination_eswitch_owner_vhca_id_valid, 1);
+ }
MLX5_SET(dest_format_struct, in_dests,
destination_eswitch_owner_vhca_id,
fte->dest_arr[i].vport.vhca_id);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_dbg.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_dbg.c
new file mode 100644
index 0000000..2784cd5
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_dbg.c
@@ -0,0 +1,649 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+// Copyright (c) 2021, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#include <linux/debugfs.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/seq_file.h>
+#include "dr_types.h"
+
+#define DR_DBG_PTR_TO_ID(p) ((u64)(uintptr_t)(p) & 0xFFFFFFFFULL)
+
+enum dr_dump_rec_type {
+ DR_DUMP_REC_TYPE_DOMAIN = 3000,
+ DR_DUMP_REC_TYPE_DOMAIN_INFO_FLEX_PARSER = 3001,
+ DR_DUMP_REC_TYPE_DOMAIN_INFO_DEV_ATTR = 3002,
+ DR_DUMP_REC_TYPE_DOMAIN_INFO_VPORT = 3003,
+ DR_DUMP_REC_TYPE_DOMAIN_INFO_CAPS = 3004,
+ DR_DUMP_REC_TYPE_DOMAIN_SEND_RING = 3005,
+
+ DR_DUMP_REC_TYPE_TABLE = 3100,
+ DR_DUMP_REC_TYPE_TABLE_RX = 3101,
+ DR_DUMP_REC_TYPE_TABLE_TX = 3102,
+
+ DR_DUMP_REC_TYPE_MATCHER = 3200,
+ DR_DUMP_REC_TYPE_MATCHER_MASK = 3201,
+ DR_DUMP_REC_TYPE_MATCHER_RX = 3202,
+ DR_DUMP_REC_TYPE_MATCHER_TX = 3203,
+ DR_DUMP_REC_TYPE_MATCHER_BUILDER = 3204,
+
+ DR_DUMP_REC_TYPE_RULE = 3300,
+ DR_DUMP_REC_TYPE_RULE_RX_ENTRY_V0 = 3301,
+ DR_DUMP_REC_TYPE_RULE_TX_ENTRY_V0 = 3302,
+ DR_DUMP_REC_TYPE_RULE_RX_ENTRY_V1 = 3303,
+ DR_DUMP_REC_TYPE_RULE_TX_ENTRY_V1 = 3304,
+
+ DR_DUMP_REC_TYPE_ACTION_ENCAP_L2 = 3400,
+ DR_DUMP_REC_TYPE_ACTION_ENCAP_L3 = 3401,
+ DR_DUMP_REC_TYPE_ACTION_MODIFY_HDR = 3402,
+ DR_DUMP_REC_TYPE_ACTION_DROP = 3403,
+ DR_DUMP_REC_TYPE_ACTION_QP = 3404,
+ DR_DUMP_REC_TYPE_ACTION_FT = 3405,
+ DR_DUMP_REC_TYPE_ACTION_CTR = 3406,
+ DR_DUMP_REC_TYPE_ACTION_TAG = 3407,
+ DR_DUMP_REC_TYPE_ACTION_VPORT = 3408,
+ DR_DUMP_REC_TYPE_ACTION_DECAP_L2 = 3409,
+ DR_DUMP_REC_TYPE_ACTION_DECAP_L3 = 3410,
+ DR_DUMP_REC_TYPE_ACTION_DEVX_TIR = 3411,
+ DR_DUMP_REC_TYPE_ACTION_PUSH_VLAN = 3412,
+ DR_DUMP_REC_TYPE_ACTION_POP_VLAN = 3413,
+ DR_DUMP_REC_TYPE_ACTION_SAMPLER = 3415,
+ DR_DUMP_REC_TYPE_ACTION_INSERT_HDR = 3420,
+ DR_DUMP_REC_TYPE_ACTION_REMOVE_HDR = 3421
+};
+
+void mlx5dr_dbg_tbl_add(struct mlx5dr_table *tbl)
+{
+ mutex_lock(&tbl->dmn->dump_info.dbg_mutex);
+ list_add_tail(&tbl->dbg_node, &tbl->dmn->dbg_tbl_list);
+ mutex_unlock(&tbl->dmn->dump_info.dbg_mutex);
+}
+
+void mlx5dr_dbg_tbl_del(struct mlx5dr_table *tbl)
+{
+ mutex_lock(&tbl->dmn->dump_info.dbg_mutex);
+ list_del(&tbl->dbg_node);
+ mutex_unlock(&tbl->dmn->dump_info.dbg_mutex);
+}
+
+void mlx5dr_dbg_rule_add(struct mlx5dr_rule *rule)
+{
+ struct mlx5dr_domain *dmn = rule->matcher->tbl->dmn;
+
+ mutex_lock(&dmn->dump_info.dbg_mutex);
+ list_add_tail(&rule->dbg_node, &rule->matcher->dbg_rule_list);
+ mutex_unlock(&dmn->dump_info.dbg_mutex);
+}
+
+void mlx5dr_dbg_rule_del(struct mlx5dr_rule *rule)
+{
+ struct mlx5dr_domain *dmn = rule->matcher->tbl->dmn;
+
+ mutex_lock(&dmn->dump_info.dbg_mutex);
+ list_del(&rule->dbg_node);
+ mutex_unlock(&dmn->dump_info.dbg_mutex);
+}
+
+static u64 dr_dump_icm_to_idx(u64 icm_addr)
+{
+ return (icm_addr >> 6) & 0xffffffff;
+}
+
+#define DR_HEX_SIZE 256
+
+static void
+dr_dump_hex_print(char hex[DR_HEX_SIZE], char *src, u32 size)
+{
+ if (WARN_ON_ONCE(DR_HEX_SIZE < 2 * size + 1))
+ size = DR_HEX_SIZE / 2 - 1; /* truncate */
+
+ bin2hex(hex, src, size);
+ hex[2 * size] = 0; /* NULL-terminate */
+}
+
+static int
+dr_dump_rule_action_mem(struct seq_file *file, const u64 rule_id,
+ struct mlx5dr_rule_action_member *action_mem)
+{
+ struct mlx5dr_action *action = action_mem->action;
+ const u64 action_id = DR_DBG_PTR_TO_ID(action);
+
+ switch (action->action_type) {
+ case DR_ACTION_TYP_DROP:
+ seq_printf(file, "%d,0x%llx,0x%llx\n",
+ DR_DUMP_REC_TYPE_ACTION_DROP, action_id, rule_id);
+ break;
+ case DR_ACTION_TYP_FT:
+ if (action->dest_tbl->is_fw_tbl)
+ seq_printf(file, "%d,0x%llx,0x%llx,0x%x\n",
+ DR_DUMP_REC_TYPE_ACTION_FT, action_id,
+ rule_id, action->dest_tbl->fw_tbl.id);
+ else
+ seq_printf(file, "%d,0x%llx,0x%llx,0x%x\n",
+ DR_DUMP_REC_TYPE_ACTION_FT, action_id,
+ rule_id, action->dest_tbl->tbl->table_id);
+
+ break;
+ case DR_ACTION_TYP_CTR:
+ seq_printf(file, "%d,0x%llx,0x%llx,0x%x\n",
+ DR_DUMP_REC_TYPE_ACTION_CTR, action_id, rule_id,
+ action->ctr->ctr_id + action->ctr->offset);
+ break;
+ case DR_ACTION_TYP_TAG:
+ seq_printf(file, "%d,0x%llx,0x%llx,0x%x\n",
+ DR_DUMP_REC_TYPE_ACTION_TAG, action_id, rule_id,
+ action->flow_tag->flow_tag);
+ break;
+ case DR_ACTION_TYP_MODIFY_HDR:
+ seq_printf(file, "%d,0x%llx,0x%llx,0x%x\n",
+ DR_DUMP_REC_TYPE_ACTION_MODIFY_HDR, action_id,
+ rule_id, action->rewrite->index);
+ break;
+ case DR_ACTION_TYP_VPORT:
+ seq_printf(file, "%d,0x%llx,0x%llx,0x%x\n",
+ DR_DUMP_REC_TYPE_ACTION_VPORT, action_id, rule_id,
+ action->vport->caps->num);
+ break;
+ case DR_ACTION_TYP_TNL_L2_TO_L2:
+ seq_printf(file, "%d,0x%llx,0x%llx\n",
+ DR_DUMP_REC_TYPE_ACTION_DECAP_L2, action_id,
+ rule_id);
+ break;
+ case DR_ACTION_TYP_TNL_L3_TO_L2:
+ seq_printf(file, "%d,0x%llx,0x%llx,0x%x\n",
+ DR_DUMP_REC_TYPE_ACTION_DECAP_L3, action_id,
+ rule_id, action->rewrite->index);
+ break;
+ case DR_ACTION_TYP_L2_TO_TNL_L2:
+ seq_printf(file, "%d,0x%llx,0x%llx,0x%x\n",
+ DR_DUMP_REC_TYPE_ACTION_ENCAP_L2, action_id,
+ rule_id, action->reformat->id);
+ break;
+ case DR_ACTION_TYP_L2_TO_TNL_L3:
+ seq_printf(file, "%d,0x%llx,0x%llx,0x%x\n",
+ DR_DUMP_REC_TYPE_ACTION_ENCAP_L3, action_id,
+ rule_id, action->reformat->id);
+ break;
+ case DR_ACTION_TYP_POP_VLAN:
+ seq_printf(file, "%d,0x%llx,0x%llx\n",
+ DR_DUMP_REC_TYPE_ACTION_POP_VLAN, action_id,
+ rule_id);
+ break;
+ case DR_ACTION_TYP_PUSH_VLAN:
+ seq_printf(file, "%d,0x%llx,0x%llx,0x%x\n",
+ DR_DUMP_REC_TYPE_ACTION_PUSH_VLAN, action_id,
+ rule_id, action->push_vlan->vlan_hdr);
+ break;
+ case DR_ACTION_TYP_INSERT_HDR:
+ seq_printf(file, "%d,0x%llx,0x%llx,0x%x,0x%x,0x%x\n",
+ DR_DUMP_REC_TYPE_ACTION_INSERT_HDR, action_id,
+ rule_id, action->reformat->id,
+ action->reformat->param_0,
+ action->reformat->param_1);
+ break;
+ case DR_ACTION_TYP_REMOVE_HDR:
+ seq_printf(file, "%d,0x%llx,0x%llx,0x%x,0x%x,0x%x\n",
+ DR_DUMP_REC_TYPE_ACTION_REMOVE_HDR, action_id,
+ rule_id, action->reformat->id,
+ action->reformat->param_0,
+ action->reformat->param_1);
+ break;
+ case DR_ACTION_TYP_SAMPLER:
+ seq_printf(file,
+ "%d,0x%llx,0x%llx,0x%x,0x%x,0x%x,0x%llx,0x%llx\n",
+ DR_DUMP_REC_TYPE_ACTION_SAMPLER, action_id, rule_id,
+ 0, 0, action->sampler->sampler_id,
+ action->sampler->rx_icm_addr,
+ action->sampler->tx_icm_addr);
+ break;
+ default:
+ return 0;
+ }
+
+ return 0;
+}
+
+static int
+dr_dump_rule_mem(struct seq_file *file, struct mlx5dr_ste *ste,
+ bool is_rx, const u64 rule_id, u8 format_ver)
+{
+ char hw_ste_dump[DR_HEX_SIZE];
+ u32 mem_rec_type;
+
+ if (format_ver == MLX5_STEERING_FORMAT_CONNECTX_5) {
+ mem_rec_type = is_rx ? DR_DUMP_REC_TYPE_RULE_RX_ENTRY_V0 :
+ DR_DUMP_REC_TYPE_RULE_TX_ENTRY_V0;
+ } else {
+ mem_rec_type = is_rx ? DR_DUMP_REC_TYPE_RULE_RX_ENTRY_V1 :
+ DR_DUMP_REC_TYPE_RULE_TX_ENTRY_V1;
+ }
+
+ dr_dump_hex_print(hw_ste_dump, (char *)ste->hw_ste, DR_STE_SIZE_REDUCED);
+
+ seq_printf(file, "%d,0x%llx,0x%llx,%s\n", mem_rec_type,
+ dr_dump_icm_to_idx(mlx5dr_ste_get_icm_addr(ste)), rule_id,
+ hw_ste_dump);
+
+ return 0;
+}
+
+static int
+dr_dump_rule_rx_tx(struct seq_file *file, struct mlx5dr_rule_rx_tx *rule_rx_tx,
+ bool is_rx, const u64 rule_id, u8 format_ver)
+{
+ struct mlx5dr_ste *ste_arr[DR_RULE_MAX_STES + DR_ACTION_MAX_STES];
+ struct mlx5dr_ste *curr_ste = rule_rx_tx->last_rule_ste;
+ int ret, i;
+
+ if (mlx5dr_rule_get_reverse_rule_members(ste_arr, curr_ste, &i))
+ return 0;
+
+ while (i--) {
+ ret = dr_dump_rule_mem(file, ste_arr[i], is_rx, rule_id,
+ format_ver);
+ if (ret < 0)
+ return ret;
+ }
+
+ return 0;
+}
+
+static int dr_dump_rule(struct seq_file *file, struct mlx5dr_rule *rule)
+{
+ struct mlx5dr_rule_action_member *action_mem;
+ const u64 rule_id = DR_DBG_PTR_TO_ID(rule);
+ struct mlx5dr_rule_rx_tx *rx = &rule->rx;
+ struct mlx5dr_rule_rx_tx *tx = &rule->tx;
+ u8 format_ver;
+ int ret;
+
+ format_ver = rule->matcher->tbl->dmn->info.caps.sw_format_ver;
+
+ seq_printf(file, "%d,0x%llx,0x%llx\n", DR_DUMP_REC_TYPE_RULE, rule_id,
+ DR_DBG_PTR_TO_ID(rule->matcher));
+
+ if (rx->nic_matcher) {
+ ret = dr_dump_rule_rx_tx(file, rx, true, rule_id, format_ver);
+ if (ret < 0)
+ return ret;
+ }
+
+ if (tx->nic_matcher) {
+ ret = dr_dump_rule_rx_tx(file, tx, false, rule_id, format_ver);
+ if (ret < 0)
+ return ret;
+ }
+
+ list_for_each_entry(action_mem, &rule->rule_actions_list, list) {
+ ret = dr_dump_rule_action_mem(file, rule_id, action_mem);
+ if (ret < 0)
+ return ret;
+ }
+
+ return 0;
+}
+
+static int
+dr_dump_matcher_mask(struct seq_file *file, struct mlx5dr_match_param *mask,
+ u8 criteria, const u64 matcher_id)
+{
+ char dump[DR_HEX_SIZE];
+
+ seq_printf(file, "%d,0x%llx,", DR_DUMP_REC_TYPE_MATCHER_MASK,
+ matcher_id);
+
+ if (criteria & DR_MATCHER_CRITERIA_OUTER) {
+ dr_dump_hex_print(dump, (char *)&mask->outer, sizeof(mask->outer));
+ seq_printf(file, "%s,", dump);
+ } else {
+ seq_puts(file, ",");
+ }
+
+ if (criteria & DR_MATCHER_CRITERIA_INNER) {
+ dr_dump_hex_print(dump, (char *)&mask->inner, sizeof(mask->inner));
+ seq_printf(file, "%s,", dump);
+ } else {
+ seq_puts(file, ",");
+ }
+
+ if (criteria & DR_MATCHER_CRITERIA_MISC) {
+ dr_dump_hex_print(dump, (char *)&mask->misc, sizeof(mask->misc));
+ seq_printf(file, "%s,", dump);
+ } else {
+ seq_puts(file, ",");
+ }
+
+ if (criteria & DR_MATCHER_CRITERIA_MISC2) {
+ dr_dump_hex_print(dump, (char *)&mask->misc2, sizeof(mask->misc2));
+ seq_printf(file, "%s,", dump);
+ } else {
+ seq_puts(file, ",");
+ }
+
+ if (criteria & DR_MATCHER_CRITERIA_MISC3) {
+ dr_dump_hex_print(dump, (char *)&mask->misc3, sizeof(mask->misc3));
+ seq_printf(file, "%s\n", dump);
+ } else {
+ seq_puts(file, ",\n");
+ }
+
+ return 0;
+}
+
+static int
+dr_dump_matcher_builder(struct seq_file *file, struct mlx5dr_ste_build *builder,
+ u32 index, bool is_rx, const u64 matcher_id)
+{
+ seq_printf(file, "%d,0x%llx,%d,%d,0x%x\n",
+ DR_DUMP_REC_TYPE_MATCHER_BUILDER, matcher_id, index, is_rx,
+ builder->lu_type);
+
+ return 0;
+}
+
+static int
+dr_dump_matcher_rx_tx(struct seq_file *file, bool is_rx,
+ struct mlx5dr_matcher_rx_tx *matcher_rx_tx,
+ const u64 matcher_id)
+{
+ enum dr_dump_rec_type rec_type;
+ int i, ret;
+
+ rec_type = is_rx ? DR_DUMP_REC_TYPE_MATCHER_RX :
+ DR_DUMP_REC_TYPE_MATCHER_TX;
+
+ seq_printf(file, "%d,0x%llx,0x%llx,%d,0x%llx,0x%llx\n",
+ rec_type, DR_DBG_PTR_TO_ID(matcher_rx_tx),
+ matcher_id, matcher_rx_tx->num_of_builders,
+ dr_dump_icm_to_idx(matcher_rx_tx->s_htbl->chunk->icm_addr),
+ dr_dump_icm_to_idx(matcher_rx_tx->e_anchor->chunk->icm_addr));
+
+ for (i = 0; i < matcher_rx_tx->num_of_builders; i++) {
+ ret = dr_dump_matcher_builder(file,
+ &matcher_rx_tx->ste_builder[i],
+ i, is_rx, matcher_id);
+ if (ret < 0)
+ return ret;
+ }
+
+ return 0;
+}
+
+static int
+dr_dump_matcher(struct seq_file *file, struct mlx5dr_matcher *matcher)
+{
+ struct mlx5dr_matcher_rx_tx *rx = &matcher->rx;
+ struct mlx5dr_matcher_rx_tx *tx = &matcher->tx;
+ u64 matcher_id;
+ int ret;
+
+ matcher_id = DR_DBG_PTR_TO_ID(matcher);
+
+ seq_printf(file, "%d,0x%llx,0x%llx,%d\n", DR_DUMP_REC_TYPE_MATCHER,
+ matcher_id, DR_DBG_PTR_TO_ID(matcher->tbl), matcher->prio);
+
+ ret = dr_dump_matcher_mask(file, &matcher->mask,
+ matcher->match_criteria, matcher_id);
+ if (ret < 0)
+ return ret;
+
+ if (rx->nic_tbl) {
+ ret = dr_dump_matcher_rx_tx(file, true, rx, matcher_id);
+ if (ret < 0)
+ return ret;
+ }
+
+ if (tx->nic_tbl) {
+ ret = dr_dump_matcher_rx_tx(file, false, tx, matcher_id);
+ if (ret < 0)
+ return ret;
+ }
+
+ return 0;
+}
+
+static int
+dr_dump_matcher_all(struct seq_file *file, struct mlx5dr_matcher *matcher)
+{
+ struct mlx5dr_rule *rule;
+ int ret;
+
+ ret = dr_dump_matcher(file, matcher);
+ if (ret < 0)
+ return ret;
+
+ list_for_each_entry(rule, &matcher->dbg_rule_list, dbg_node) {
+ ret = dr_dump_rule(file, rule);
+ if (ret < 0)
+ return ret;
+ }
+
+ return 0;
+}
+
+static int
+dr_dump_table_rx_tx(struct seq_file *file, bool is_rx,
+ struct mlx5dr_table_rx_tx *table_rx_tx,
+ const u64 table_id)
+{
+ enum dr_dump_rec_type rec_type;
+
+ rec_type = is_rx ? DR_DUMP_REC_TYPE_TABLE_RX :
+ DR_DUMP_REC_TYPE_TABLE_TX;
+
+ seq_printf(file, "%d,0x%llx,0x%llx\n", rec_type, table_id,
+ dr_dump_icm_to_idx(table_rx_tx->s_anchor->chunk->icm_addr));
+
+ return 0;
+}
+
+static int dr_dump_table(struct seq_file *file, struct mlx5dr_table *table)
+{
+ struct mlx5dr_table_rx_tx *rx = &table->rx;
+ struct mlx5dr_table_rx_tx *tx = &table->tx;
+ int ret;
+
+ seq_printf(file, "%d,0x%llx,0x%llx,%d,%d\n", DR_DUMP_REC_TYPE_TABLE,
+ DR_DBG_PTR_TO_ID(table), DR_DBG_PTR_TO_ID(table->dmn),
+ table->table_type, table->level);
+
+ if (rx->nic_dmn) {
+ ret = dr_dump_table_rx_tx(file, true, rx,
+ DR_DBG_PTR_TO_ID(table));
+ if (ret < 0)
+ return ret;
+ }
+
+ if (tx->nic_dmn) {
+ ret = dr_dump_table_rx_tx(file, false, tx,
+ DR_DBG_PTR_TO_ID(table));
+ if (ret < 0)
+ return ret;
+ }
+ return 0;
+}
+
+static int dr_dump_table_all(struct seq_file *file, struct mlx5dr_table *tbl)
+{
+ struct mlx5dr_matcher *matcher;
+ int ret;
+
+ ret = dr_dump_table(file, tbl);
+ if (ret < 0)
+ return ret;
+
+ list_for_each_entry(matcher, &tbl->matcher_list, list_node) {
+ ret = dr_dump_matcher_all(file, matcher);
+ if (ret < 0)
+ return ret;
+ }
+ return 0;
+}
+
+static int
+dr_dump_send_ring(struct seq_file *file, struct mlx5dr_send_ring *ring,
+ const u64 domain_id)
+{
+ seq_printf(file, "%d,0x%llx,0x%llx,0x%x,0x%x\n",
+ DR_DUMP_REC_TYPE_DOMAIN_SEND_RING, DR_DBG_PTR_TO_ID(ring),
+ domain_id, ring->cq->mcq.cqn, ring->qp->qpn);
+ return 0;
+}
+
+static int
+dr_dump_domain_info_flex_parser(struct seq_file *file,
+ const char *flex_parser_name,
+ const u8 flex_parser_value,
+ const u64 domain_id)
+{
+ seq_printf(file, "%d,0x%llx,%s,0x%x\n",
+ DR_DUMP_REC_TYPE_DOMAIN_INFO_FLEX_PARSER, domain_id,
+ flex_parser_name, flex_parser_value);
+ return 0;
+}
+
+static int
+dr_dump_domain_info_caps(struct seq_file *file, struct mlx5dr_cmd_caps *caps,
+ const u64 domain_id)
+{
+ struct mlx5dr_cmd_vport_cap *vport_caps;
+ unsigned long i, vports_num;
+
+ xa_for_each(&caps->vports.vports_caps_xa, vports_num, vport_caps)
+ ; /* count the number of vports in xarray */
+
+ seq_printf(file, "%d,0x%llx,0x%x,0x%llx,0x%llx,0x%x,%lu,%d\n",
+ DR_DUMP_REC_TYPE_DOMAIN_INFO_CAPS, domain_id, caps->gvmi,
+ caps->nic_rx_drop_address, caps->nic_tx_drop_address,
+ caps->flex_protocols, vports_num, caps->eswitch_manager);
+
+ xa_for_each(&caps->vports.vports_caps_xa, i, vport_caps) {
+ vport_caps = xa_load(&caps->vports.vports_caps_xa, i);
+
+ seq_printf(file, "%d,0x%llx,%lu,0x%x,0x%llx,0x%llx\n",
+ DR_DUMP_REC_TYPE_DOMAIN_INFO_VPORT, domain_id, i,
+ vport_caps->vport_gvmi, vport_caps->icm_address_rx,
+ vport_caps->icm_address_tx);
+ }
+ return 0;
+}
+
+static int
+dr_dump_domain_info(struct seq_file *file, struct mlx5dr_domain_info *info,
+ const u64 domain_id)
+{
+ int ret;
+
+ ret = dr_dump_domain_info_caps(file, &info->caps, domain_id);
+ if (ret < 0)
+ return ret;
+
+ ret = dr_dump_domain_info_flex_parser(file, "icmp_dw0",
+ info->caps.flex_parser_id_icmp_dw0,
+ domain_id);
+ if (ret < 0)
+ return ret;
+
+ ret = dr_dump_domain_info_flex_parser(file, "icmp_dw1",
+ info->caps.flex_parser_id_icmp_dw1,
+ domain_id);
+ if (ret < 0)
+ return ret;
+
+ ret = dr_dump_domain_info_flex_parser(file, "icmpv6_dw0",
+ info->caps.flex_parser_id_icmpv6_dw0,
+ domain_id);
+ if (ret < 0)
+ return ret;
+
+ ret = dr_dump_domain_info_flex_parser(file, "icmpv6_dw1",
+ info->caps.flex_parser_id_icmpv6_dw1,
+ domain_id);
+ if (ret < 0)
+ return ret;
+
+ return 0;
+}
+
+static int
+dr_dump_domain(struct seq_file *file, struct mlx5dr_domain *dmn)
+{
+ u64 domain_id = DR_DBG_PTR_TO_ID(dmn);
+ int ret;
+
+ seq_printf(file, "%d,0x%llx,%d,0%x,%d,%s\n", DR_DUMP_REC_TYPE_DOMAIN,
+ domain_id, dmn->type, dmn->info.caps.gvmi,
+ dmn->info.supp_sw_steering, pci_name(dmn->mdev->pdev));
+
+ ret = dr_dump_domain_info(file, &dmn->info, domain_id);
+ if (ret < 0)
+ return ret;
+
+ if (dmn->info.supp_sw_steering) {
+ ret = dr_dump_send_ring(file, dmn->send_ring, domain_id);
+ if (ret < 0)
+ return ret;
+ }
+
+ return 0;
+}
+
+static int dr_dump_domain_all(struct seq_file *file, struct mlx5dr_domain *dmn)
+{
+ struct mlx5dr_table *tbl;
+ int ret;
+
+ mutex_lock(&dmn->dump_info.dbg_mutex);
+ mlx5dr_domain_lock(dmn);
+
+ ret = dr_dump_domain(file, dmn);
+ if (ret < 0)
+ goto unlock_mutex;
+
+ list_for_each_entry(tbl, &dmn->dbg_tbl_list, dbg_node) {
+ ret = dr_dump_table_all(file, tbl);
+ if (ret < 0)
+ break;
+ }
+
+unlock_mutex:
+ mlx5dr_domain_unlock(dmn);
+ mutex_unlock(&dmn->dump_info.dbg_mutex);
+ return ret;
+}
+
+static int dr_dump_show(struct seq_file *file, void *priv)
+{
+ return dr_dump_domain_all(file, file->private);
+}
+DEFINE_SHOW_ATTRIBUTE(dr_dump);
+
+void mlx5dr_dbg_init_dump(struct mlx5dr_domain *dmn)
+{
+ struct mlx5_core_dev *dev = dmn->mdev;
+ char file_name[128];
+
+ if (dmn->type != MLX5DR_DOMAIN_TYPE_FDB) {
+ mlx5_core_warn(dev,
+ "Steering dump is not supported for NIC RX/TX domains\n");
+ return;
+ }
+
+ dmn->dump_info.steering_debugfs =
+ debugfs_create_dir("steering", dev->priv.dbg_root);
+ dmn->dump_info.fdb_debugfs =
+ debugfs_create_dir("fdb", dmn->dump_info.steering_debugfs);
+
+ sprintf(file_name, "dmn_%p", dmn);
+ debugfs_create_file(file_name, 0444, dmn->dump_info.fdb_debugfs,
+ dmn, &dr_dump_fops);
+
+ INIT_LIST_HEAD(&dmn->dbg_tbl_list);
+ mutex_init(&dmn->dump_info.dbg_mutex);
+}
+
+void mlx5dr_dbg_uninit_dump(struct mlx5dr_domain *dmn)
+{
+ debugfs_remove_recursive(dmn->dump_info.steering_debugfs);
+ mutex_destroy(&dmn->dump_info.dbg_mutex);
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_dbg.h b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_dbg.h
new file mode 100644
index 0000000..def6cf8
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_dbg.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+/* Copyright (c) 2021, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */
+
+struct mlx5dr_dbg_dump_info {
+ struct mutex dbg_mutex; /* protect dbg lists */
+ struct dentry *steering_debugfs;
+ struct dentry *fdb_debugfs;
+};
+
+void mlx5dr_dbg_init_dump(struct mlx5dr_domain *dmn);
+void mlx5dr_dbg_uninit_dump(struct mlx5dr_domain *dmn);
+void mlx5dr_dbg_tbl_add(struct mlx5dr_table *tbl);
+void mlx5dr_dbg_tbl_del(struct mlx5dr_table *tbl);
+void mlx5dr_dbg_rule_add(struct mlx5dr_rule *rule);
+void mlx5dr_dbg_rule_del(struct mlx5dr_rule *rule);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_domain.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_domain.c
index 8cbd36c..5fa7f9d 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_domain.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_domain.c
@@ -2,6 +2,7 @@
/* Copyright (c) 2019 Mellanox Technologies. */
#include <linux/mlx5/eswitch.h>
+#include <linux/err.h>
#include "dr_types.h"
#define DR_DOMAIN_SW_STEERING_SUPPORTED(dmn, dmn_type) \
@@ -72,9 +73,9 @@ static int dr_domain_init_resources(struct mlx5dr_domain *dmn)
}
dmn->uar = mlx5_get_uars_page(dmn->mdev);
- if (!dmn->uar) {
+ if (IS_ERR(dmn->uar)) {
mlx5dr_err(dmn, "Couldn't allocate UAR\n");
- ret = -ENOMEM;
+ ret = PTR_ERR(dmn->uar);
goto clean_pd;
}
@@ -163,9 +164,7 @@ static int dr_domain_query_vport(struct mlx5dr_domain *dmn,
static int dr_domain_query_esw_mngr(struct mlx5dr_domain *dmn)
{
- return dr_domain_query_vport(dmn,
- dmn->info.caps.is_ecpf ? MLX5_VPORT_ECPF : 0,
- false,
+ return dr_domain_query_vport(dmn, 0, false,
&dmn->info.caps.vports.esw_manager_caps);
}
@@ -396,7 +395,7 @@ mlx5dr_domain_create(struct mlx5_core_dev *mdev, enum mlx5dr_domain_type type)
}
dr_domain_init_csum_recalc_fts(dmn);
-
+ mlx5dr_dbg_init_dump(dmn);
return dmn;
uninit_caps:
@@ -432,11 +431,12 @@ int mlx5dr_domain_sync(struct mlx5dr_domain *dmn, u32 flags)
int mlx5dr_domain_destroy(struct mlx5dr_domain *dmn)
{
- if (refcount_read(&dmn->refcount) > 1)
+ if (WARN_ON_ONCE(refcount_read(&dmn->refcount) > 1))
return -EBUSY;
/* make sure resources are not used by the hardware */
mlx5dr_cmd_sync_steering(dmn->mdev);
+ mlx5dr_dbg_uninit_dump(dmn);
dr_domain_uninit_csum_recalc_fts(dmn);
dr_domain_uninit_resources(dmn);
dr_domain_caps_uninit(dmn);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_matcher.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_matcher.c
index 7933652..e87cf498 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_matcher.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_matcher.c
@@ -141,6 +141,19 @@ static bool dr_mask_is_tnl_geneve_tlv_opt(struct mlx5dr_match_misc3 *misc3)
}
static bool
+dr_matcher_supp_flex_parser_ok(struct mlx5dr_cmd_caps *caps)
+{
+ return caps->flex_parser_ok_bits_supp;
+}
+
+static bool dr_mask_is_tnl_geneve_tlv_opt_exist_set(struct mlx5dr_match_misc *misc,
+ struct mlx5dr_domain *dmn)
+{
+ return dr_matcher_supp_flex_parser_ok(&dmn->info.caps) &&
+ misc->geneve_tlv_option_0_exist;
+}
+
+static bool
dr_matcher_supp_tnl_geneve(struct mlx5dr_cmd_caps *caps)
{
return (caps->sw_format_ver == MLX5_STEERING_FORMAT_CONNECTX_6DX) ||
@@ -359,7 +372,7 @@ static bool dr_mask_is_tnl_mpls_over_gre(struct mlx5dr_match_param *mask,
static int dr_matcher_supp_tnl_mpls_over_udp(struct mlx5dr_cmd_caps *caps)
{
- return caps->flex_protocols & mlx5_FLEX_PARSER_MPLS_OVER_UDP_ENABLED;
+ return caps->flex_protocols & MLX5_FLEX_PARSER_MPLS_OVER_UDP_ENABLED;
}
static bool dr_mask_is_tnl_mpls_over_udp(struct mlx5dr_match_param *mask,
@@ -368,6 +381,12 @@ static bool dr_mask_is_tnl_mpls_over_udp(struct mlx5dr_match_param *mask,
return DR_MASK_IS_OUTER_MPLS_OVER_UDP_SET(&mask->misc2) &&
dr_matcher_supp_tnl_mpls_over_udp(&dmn->info.caps);
}
+
+static bool dr_mask_is_tnl_header_0_1_set(struct mlx5dr_match_misc5 *misc5)
+{
+ return misc5->tunnel_header_0 || misc5->tunnel_header_1;
+}
+
int mlx5dr_matcher_select_builders(struct mlx5dr_matcher *matcher,
struct mlx5dr_matcher_rx_tx *nic_matcher,
enum mlx5dr_ipv outer_ipv,
@@ -424,6 +443,9 @@ static int dr_matcher_set_ste_builders(struct mlx5dr_matcher *matcher,
if (matcher->match_criteria & DR_MATCHER_CRITERIA_MISC4)
mask.misc4 = matcher->mask.misc4;
+ if (matcher->match_criteria & DR_MATCHER_CRITERIA_MISC5)
+ mask.misc5 = matcher->mask.misc5;
+
ret = mlx5dr_ste_build_pre_check(dmn, matcher->match_criteria,
&matcher->mask, NULL);
if (ret)
@@ -443,7 +465,8 @@ static int dr_matcher_set_ste_builders(struct mlx5dr_matcher *matcher,
if (matcher->match_criteria & (DR_MATCHER_CRITERIA_OUTER |
DR_MATCHER_CRITERIA_MISC |
DR_MATCHER_CRITERIA_MISC2 |
- DR_MATCHER_CRITERIA_MISC3)) {
+ DR_MATCHER_CRITERIA_MISC3 |
+ DR_MATCHER_CRITERIA_MISC5)) {
inner = false;
if (dr_mask_is_wqe_metadata_set(&mask.misc2))
@@ -511,6 +534,10 @@ static int dr_matcher_set_ste_builders(struct mlx5dr_matcher *matcher,
mlx5dr_ste_build_tnl_geneve_tlv_opt(ste_ctx, &sb[idx++],
&mask, &dmn->info.caps,
inner, rx);
+ if (dr_mask_is_tnl_geneve_tlv_opt_exist_set(&mask.misc, dmn))
+ mlx5dr_ste_build_tnl_geneve_tlv_opt_exist(ste_ctx, &sb[idx++],
+ &mask, &dmn->info.caps,
+ inner, rx);
} else if (dr_mask_is_tnl_gtpu_any(&mask, dmn)) {
if (dr_mask_is_tnl_gtpu_flex_parser_0(&mask, dmn))
mlx5dr_ste_build_tnl_gtpu_flex_parser_0(ste_ctx, &sb[idx++],
@@ -525,6 +552,9 @@ static int dr_matcher_set_ste_builders(struct mlx5dr_matcher *matcher,
if (dr_mask_is_tnl_gtpu(&mask, dmn))
mlx5dr_ste_build_tnl_gtpu(ste_ctx, &sb[idx++],
&mask, inner, rx);
+ } else if (dr_mask_is_tnl_header_0_1_set(&mask.misc5)) {
+ mlx5dr_ste_build_tnl_header_0_1(ste_ctx, &sb[idx++],
+ &mask, inner, rx);
}
if (DR_MASK_IS_ETH_L4_MISC_SET(mask.misc3, outer))
@@ -653,10 +683,10 @@ static int dr_matcher_set_ste_builders(struct mlx5dr_matcher *matcher,
return 0;
}
-static int dr_matcher_connect(struct mlx5dr_domain *dmn,
- struct mlx5dr_matcher_rx_tx *curr_nic_matcher,
- struct mlx5dr_matcher_rx_tx *next_nic_matcher,
- struct mlx5dr_matcher_rx_tx *prev_nic_matcher)
+static int dr_nic_matcher_connect(struct mlx5dr_domain *dmn,
+ struct mlx5dr_matcher_rx_tx *curr_nic_matcher,
+ struct mlx5dr_matcher_rx_tx *next_nic_matcher,
+ struct mlx5dr_matcher_rx_tx *prev_nic_matcher)
{
struct mlx5dr_table_rx_tx *nic_tbl = curr_nic_matcher->nic_tbl;
struct mlx5dr_domain_rx_tx *nic_dmn = nic_tbl->nic_dmn;
@@ -712,58 +742,50 @@ static int dr_matcher_connect(struct mlx5dr_domain *dmn,
return 0;
}
-static int dr_matcher_add_to_tbl(struct mlx5dr_matcher *matcher)
+int mlx5dr_matcher_add_to_tbl_nic(struct mlx5dr_domain *dmn,
+ struct mlx5dr_matcher_rx_tx *nic_matcher)
{
- struct mlx5dr_matcher *next_matcher, *prev_matcher, *tmp_matcher;
- struct mlx5dr_table *tbl = matcher->tbl;
- struct mlx5dr_domain *dmn = tbl->dmn;
+ struct mlx5dr_matcher_rx_tx *next_nic_matcher, *prev_nic_matcher, *tmp_nic_matcher;
+ struct mlx5dr_table_rx_tx *nic_tbl = nic_matcher->nic_tbl;
bool first = true;
int ret;
- next_matcher = NULL;
- list_for_each_entry(tmp_matcher, &tbl->matcher_list, matcher_list) {
- if (tmp_matcher->prio >= matcher->prio) {
- next_matcher = tmp_matcher;
+ /* If the nic matcher is already on its parent nic table list,
+ * then it is already connected to the chain of nic matchers.
+ */
+ if (!list_empty(&nic_matcher->list_node))
+ return 0;
+
+ next_nic_matcher = NULL;
+ list_for_each_entry(tmp_nic_matcher, &nic_tbl->nic_matcher_list, list_node) {
+ if (tmp_nic_matcher->prio >= nic_matcher->prio) {
+ next_nic_matcher = tmp_nic_matcher;
break;
}
first = false;
}
- prev_matcher = NULL;
- if (next_matcher && !first)
- prev_matcher = list_prev_entry(next_matcher, matcher_list);
+ prev_nic_matcher = NULL;
+ if (next_nic_matcher && !first)
+ prev_nic_matcher = list_prev_entry(next_nic_matcher, list_node);
else if (!first)
- prev_matcher = list_last_entry(&tbl->matcher_list,
- struct mlx5dr_matcher,
- matcher_list);
+ prev_nic_matcher = list_last_entry(&nic_tbl->nic_matcher_list,
+ struct mlx5dr_matcher_rx_tx,
+ list_node);
- if (dmn->type == MLX5DR_DOMAIN_TYPE_FDB ||
- dmn->type == MLX5DR_DOMAIN_TYPE_NIC_RX) {
- ret = dr_matcher_connect(dmn, &matcher->rx,
- next_matcher ? &next_matcher->rx : NULL,
- prev_matcher ? &prev_matcher->rx : NULL);
- if (ret)
- return ret;
- }
+ ret = dr_nic_matcher_connect(dmn, nic_matcher,
+ next_nic_matcher, prev_nic_matcher);
+ if (ret)
+ return ret;
- if (dmn->type == MLX5DR_DOMAIN_TYPE_FDB ||
- dmn->type == MLX5DR_DOMAIN_TYPE_NIC_TX) {
- ret = dr_matcher_connect(dmn, &matcher->tx,
- next_matcher ? &next_matcher->tx : NULL,
- prev_matcher ? &prev_matcher->tx : NULL);
- if (ret)
- return ret;
- }
-
- if (prev_matcher)
- list_add(&matcher->matcher_list, &prev_matcher->matcher_list);
- else if (next_matcher)
- list_add_tail(&matcher->matcher_list,
- &next_matcher->matcher_list);
+ if (prev_nic_matcher)
+ list_add(&nic_matcher->list_node, &prev_nic_matcher->list_node);
+ else if (next_nic_matcher)
+ list_add_tail(&nic_matcher->list_node, &next_nic_matcher->list_node);
else
- list_add(&matcher->matcher_list, &tbl->matcher_list);
+ list_add(&nic_matcher->list_node, &nic_matcher->nic_tbl->nic_matcher_list);
- return 0;
+ return ret;
}
static void dr_matcher_uninit_nic(struct mlx5dr_matcher_rx_tx *nic_matcher)
@@ -822,6 +844,9 @@ static int dr_matcher_init_nic(struct mlx5dr_matcher *matcher,
struct mlx5dr_domain *dmn = matcher->tbl->dmn;
int ret;
+ nic_matcher->prio = matcher->prio;
+ INIT_LIST_HEAD(&nic_matcher->list_node);
+
ret = dr_matcher_set_all_ste_builders(matcher, nic_matcher);
if (ret)
return ret;
@@ -872,13 +897,12 @@ static int dr_matcher_init_fdb(struct mlx5dr_matcher *matcher)
return ret;
}
-static int dr_matcher_init(struct mlx5dr_matcher *matcher,
- struct mlx5dr_match_parameters *mask)
+static int dr_matcher_copy_param(struct mlx5dr_matcher *matcher,
+ struct mlx5dr_match_parameters *mask)
{
+ struct mlx5dr_domain *dmn = matcher->tbl->dmn;
struct mlx5dr_match_parameters consumed_mask;
- struct mlx5dr_table *tbl = matcher->tbl;
- struct mlx5dr_domain *dmn = tbl->dmn;
- int i, ret;
+ int i, ret = 0;
if (matcher->match_criteria >= DR_MATCHER_CRITERIA_MAX) {
mlx5dr_err(dmn, "Invalid match criteria attribute\n");
@@ -898,10 +922,36 @@ static int dr_matcher_init(struct mlx5dr_matcher *matcher,
consumed_mask.match_sz = mask->match_sz;
memcpy(consumed_mask.match_buf, mask->match_buf, mask->match_sz);
mlx5dr_ste_copy_param(matcher->match_criteria,
- &matcher->mask, &consumed_mask,
- true);
+ &matcher->mask, &consumed_mask, true);
+
+ /* Check that all mask data was consumed */
+ for (i = 0; i < consumed_mask.match_sz; i++) {
+ if (!((u8 *)consumed_mask.match_buf)[i])
+ continue;
+
+ mlx5dr_dbg(dmn,
+ "Match param mask contains unsupported parameters\n");
+ ret = -EOPNOTSUPP;
+ break;
+ }
+
+ kfree(consumed_mask.match_buf);
}
+ return ret;
+}
+
+static int dr_matcher_init(struct mlx5dr_matcher *matcher,
+ struct mlx5dr_match_parameters *mask)
+{
+ struct mlx5dr_table *tbl = matcher->tbl;
+ struct mlx5dr_domain *dmn = tbl->dmn;
+ int ret;
+
+ ret = dr_matcher_copy_param(matcher, mask);
+ if (ret)
+ return ret;
+
switch (dmn->type) {
case MLX5DR_DOMAIN_TYPE_NIC_RX:
matcher->rx.nic_tbl = &tbl->rx;
@@ -919,25 +969,25 @@ static int dr_matcher_init(struct mlx5dr_matcher *matcher,
default:
WARN_ON(true);
ret = -EINVAL;
- goto free_consumed_mask;
}
- /* Check that all mask data was consumed */
- for (i = 0; i < consumed_mask.match_sz; i++) {
- if (!((u8 *)consumed_mask.match_buf)[i])
- continue;
-
- mlx5dr_dbg(dmn, "Match param mask contains unsupported parameters\n");
- ret = -EOPNOTSUPP;
- goto free_consumed_mask;
- }
-
- ret = 0;
-free_consumed_mask:
- kfree(consumed_mask.match_buf);
return ret;
}
+static void dr_matcher_add_to_dbg_list(struct mlx5dr_matcher *matcher)
+{
+ mutex_lock(&matcher->tbl->dmn->dump_info.dbg_mutex);
+ list_add(&matcher->list_node, &matcher->tbl->matcher_list);
+ mutex_unlock(&matcher->tbl->dmn->dump_info.dbg_mutex);
+}
+
+static void dr_matcher_remove_from_dbg_list(struct mlx5dr_matcher *matcher)
+{
+ mutex_lock(&matcher->tbl->dmn->dump_info.dbg_mutex);
+ list_del(&matcher->list_node);
+ mutex_unlock(&matcher->tbl->dmn->dump_info.dbg_mutex);
+}
+
struct mlx5dr_matcher *
mlx5dr_matcher_create(struct mlx5dr_table *tbl,
u32 priority,
@@ -957,7 +1007,8 @@ mlx5dr_matcher_create(struct mlx5dr_table *tbl,
matcher->prio = priority;
matcher->match_criteria = match_criteria_enable;
refcount_set(&matcher->refcount, 1);
- INIT_LIST_HEAD(&matcher->matcher_list);
+ INIT_LIST_HEAD(&matcher->list_node);
+ INIT_LIST_HEAD(&matcher->dbg_rule_list);
mlx5dr_domain_lock(tbl->dmn);
@@ -965,16 +1016,12 @@ mlx5dr_matcher_create(struct mlx5dr_table *tbl,
if (ret)
goto free_matcher;
- ret = dr_matcher_add_to_tbl(matcher);
- if (ret)
- goto matcher_uninit;
+ dr_matcher_add_to_dbg_list(matcher);
mlx5dr_domain_unlock(tbl->dmn);
return matcher;
-matcher_uninit:
- dr_matcher_uninit(matcher);
free_matcher:
mlx5dr_domain_unlock(tbl->dmn);
kfree(matcher);
@@ -983,10 +1030,10 @@ mlx5dr_matcher_create(struct mlx5dr_table *tbl,
return NULL;
}
-static int dr_matcher_disconnect(struct mlx5dr_domain *dmn,
- struct mlx5dr_table_rx_tx *nic_tbl,
- struct mlx5dr_matcher_rx_tx *next_nic_matcher,
- struct mlx5dr_matcher_rx_tx *prev_nic_matcher)
+static int dr_matcher_disconnect_nic(struct mlx5dr_domain *dmn,
+ struct mlx5dr_table_rx_tx *nic_tbl,
+ struct mlx5dr_matcher_rx_tx *next_nic_matcher,
+ struct mlx5dr_matcher_rx_tx *prev_nic_matcher)
{
struct mlx5dr_domain_rx_tx *nic_dmn = nic_tbl->nic_dmn;
struct mlx5dr_htbl_connect_info info;
@@ -1013,43 +1060,34 @@ static int dr_matcher_disconnect(struct mlx5dr_domain *dmn,
&info, true);
}
-static int dr_matcher_remove_from_tbl(struct mlx5dr_matcher *matcher)
+int mlx5dr_matcher_remove_from_tbl_nic(struct mlx5dr_domain *dmn,
+ struct mlx5dr_matcher_rx_tx *nic_matcher)
{
- struct mlx5dr_matcher *prev_matcher, *next_matcher;
- struct mlx5dr_table *tbl = matcher->tbl;
- struct mlx5dr_domain *dmn = tbl->dmn;
- int ret = 0;
+ struct mlx5dr_matcher_rx_tx *prev_nic_matcher, *next_nic_matcher;
+ struct mlx5dr_table_rx_tx *nic_tbl = nic_matcher->nic_tbl;
+ int ret;
- if (list_is_last(&matcher->matcher_list, &tbl->matcher_list))
- next_matcher = NULL;
+ /* If the nic matcher is not on its parent nic table list,
+ * then it is detached - no need to disconnect it.
+ */
+ if (list_empty(&nic_matcher->list_node))
+ return 0;
+
+ if (list_is_last(&nic_matcher->list_node, &nic_tbl->nic_matcher_list))
+ next_nic_matcher = NULL;
else
- next_matcher = list_next_entry(matcher, matcher_list);
+ next_nic_matcher = list_next_entry(nic_matcher, list_node);
- if (matcher->matcher_list.prev == &tbl->matcher_list)
- prev_matcher = NULL;
+ if (nic_matcher->list_node.prev == &nic_tbl->nic_matcher_list)
+ prev_nic_matcher = NULL;
else
- prev_matcher = list_prev_entry(matcher, matcher_list);
+ prev_nic_matcher = list_prev_entry(nic_matcher, list_node);
- if (dmn->type == MLX5DR_DOMAIN_TYPE_FDB ||
- dmn->type == MLX5DR_DOMAIN_TYPE_NIC_RX) {
- ret = dr_matcher_disconnect(dmn, &tbl->rx,
- next_matcher ? &next_matcher->rx : NULL,
- prev_matcher ? &prev_matcher->rx : NULL);
- if (ret)
- return ret;
- }
+ ret = dr_matcher_disconnect_nic(dmn, nic_tbl, next_nic_matcher, prev_nic_matcher);
+ if (ret)
+ return ret;
- if (dmn->type == MLX5DR_DOMAIN_TYPE_FDB ||
- dmn->type == MLX5DR_DOMAIN_TYPE_NIC_TX) {
- ret = dr_matcher_disconnect(dmn, &tbl->tx,
- next_matcher ? &next_matcher->tx : NULL,
- prev_matcher ? &prev_matcher->tx : NULL);
- if (ret)
- return ret;
- }
-
- list_del(&matcher->matcher_list);
-
+ list_del_init(&nic_matcher->list_node);
return 0;
}
@@ -1057,12 +1095,12 @@ int mlx5dr_matcher_destroy(struct mlx5dr_matcher *matcher)
{
struct mlx5dr_table *tbl = matcher->tbl;
- if (refcount_read(&matcher->refcount) > 1)
+ if (WARN_ON_ONCE(refcount_read(&matcher->refcount) > 1))
return -EBUSY;
mlx5dr_domain_lock(tbl->dmn);
- dr_matcher_remove_from_tbl(matcher);
+ dr_matcher_remove_from_dbg_list(matcher);
dr_matcher_uninit(matcher);
refcount_dec(&matcher->tbl->refcount);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_rule.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_rule.c
index 6a390e9..b437457 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_rule.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_rule.c
@@ -5,11 +5,6 @@
#define DR_RULE_MAX_STE_CHAIN (DR_RULE_MAX_STES + DR_ACTION_MAX_STES)
-struct mlx5dr_rule_action_member {
- struct mlx5dr_action *action;
- struct list_head list;
-};
-
static int dr_rule_append_to_miss_list(struct mlx5dr_ste_ctx *ste_ctx,
struct mlx5dr_ste *new_last_ste,
struct list_head *miss_list,
@@ -979,14 +974,36 @@ static bool dr_rule_verify(struct mlx5dr_matcher *matcher,
return false;
}
}
+
+ if (match_criteria & DR_MATCHER_CRITERIA_MISC5) {
+ s_idx = offsetof(struct mlx5dr_match_param, misc5);
+ e_idx = min(s_idx + sizeof(param->misc5), value_size);
+
+ if (!dr_rule_cmp_value_to_mask(mask_p, param_p, s_idx, e_idx)) {
+ mlx5dr_err(matcher->tbl->dmn, "Rule misc5 parameters contains a value not specified by mask\n");
+ return false;
+ }
+ }
return true;
}
static int dr_rule_destroy_rule_nic(struct mlx5dr_rule *rule,
struct mlx5dr_rule_rx_tx *nic_rule)
{
+ /* Check if this nic rule was actually created, or was it skipped
+ * and only the other type of the RX/TX nic rule was created.
+ */
+ if (!nic_rule->last_rule_ste)
+ return 0;
+
mlx5dr_domain_nic_lock(nic_rule->nic_matcher->nic_tbl->nic_dmn);
dr_rule_clean_rule_members(rule, nic_rule);
+
+ nic_rule->nic_matcher->rules--;
+ if (!nic_rule->nic_matcher->rules)
+ mlx5dr_matcher_remove_from_tbl_nic(rule->matcher->tbl->dmn,
+ nic_rule->nic_matcher);
+
mlx5dr_domain_nic_unlock(nic_rule->nic_matcher->nic_tbl->nic_dmn);
return 0;
@@ -1003,6 +1020,8 @@ static int dr_rule_destroy_rule(struct mlx5dr_rule *rule)
{
struct mlx5dr_domain *dmn = rule->matcher->tbl->dmn;
+ mlx5dr_dbg_rule_del(rule);
+
switch (dmn->type) {
case MLX5DR_DOMAIN_TYPE_NIC_RX:
dr_rule_destroy_rule_nic(rule, &rule->rx);
@@ -1091,24 +1110,28 @@ dr_rule_create_rule_nic(struct mlx5dr_rule *rule,
mlx5dr_domain_nic_lock(nic_dmn);
+ ret = mlx5dr_matcher_add_to_tbl_nic(dmn, nic_matcher);
+ if (ret)
+ goto free_hw_ste;
+
ret = mlx5dr_matcher_select_builders(matcher,
nic_matcher,
dr_rule_get_ipv(¶m->outer),
dr_rule_get_ipv(¶m->inner));
if (ret)
- goto free_hw_ste;
+ goto remove_from_nic_tbl;
/* Set the tag values inside the ste array */
ret = mlx5dr_ste_build_ste_arr(matcher, nic_matcher, param, hw_ste_arr);
if (ret)
- goto free_hw_ste;
+ goto remove_from_nic_tbl;
/* Set the actions values/addresses inside the ste array */
ret = mlx5dr_actions_build_ste_arr(matcher, nic_matcher, actions,
num_actions, hw_ste_arr,
&new_hw_ste_arr_sz);
if (ret)
- goto free_hw_ste;
+ goto remove_from_nic_tbl;
cur_htbl = nic_matcher->s_htbl;
@@ -1155,6 +1178,8 @@ dr_rule_create_rule_nic(struct mlx5dr_rule *rule,
if (htbl)
mlx5dr_htbl_put(htbl);
+ nic_matcher->rules++;
+
mlx5dr_domain_nic_unlock(nic_dmn);
kfree(hw_ste_arr);
@@ -1168,6 +1193,10 @@ dr_rule_create_rule_nic(struct mlx5dr_rule *rule,
list_del(&ste_info->send_list);
kfree(ste_info);
}
+
+remove_from_nic_tbl:
+ mlx5dr_matcher_remove_from_tbl_nic(dmn, nic_matcher);
+
free_hw_ste:
mlx5dr_domain_nic_unlock(nic_dmn);
kfree(hw_ste_arr);
@@ -1257,6 +1286,8 @@ dr_rule_create_rule(struct mlx5dr_matcher *matcher,
if (ret)
goto remove_action_members;
+ INIT_LIST_HEAD(&rule->dbg_node);
+ mlx5dr_dbg_rule_add(rule);
return rule;
remove_action_members:
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste.c
index 219a547..7e61742 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste.c
@@ -719,6 +719,8 @@ static void dr_ste_copy_mask_misc(char *mask, struct mlx5dr_match_misc *spec, bo
spec->vxlan_vni = IFC_GET_CLR(fte_match_set_misc, mask, vxlan_vni, clr);
spec->geneve_vni = IFC_GET_CLR(fte_match_set_misc, mask, geneve_vni, clr);
+ spec->geneve_tlv_option_0_exist =
+ IFC_GET_CLR(fte_match_set_misc, mask, geneve_tlv_option_0_exist, clr);
spec->geneve_oam = IFC_GET_CLR(fte_match_set_misc, mask, geneve_oam, clr);
spec->outer_ipv6_flow_label =
@@ -880,6 +882,26 @@ static void dr_ste_copy_mask_misc4(char *mask, struct mlx5dr_match_misc4 *spec,
IFC_GET_CLR(fte_match_set_misc4, mask, prog_sample_field_value_3, clr);
}
+static void dr_ste_copy_mask_misc5(char *mask, struct mlx5dr_match_misc5 *spec, bool clr)
+{
+ spec->macsec_tag_0 =
+ IFC_GET_CLR(fte_match_set_misc5, mask, macsec_tag_0, clr);
+ spec->macsec_tag_1 =
+ IFC_GET_CLR(fte_match_set_misc5, mask, macsec_tag_1, clr);
+ spec->macsec_tag_2 =
+ IFC_GET_CLR(fte_match_set_misc5, mask, macsec_tag_2, clr);
+ spec->macsec_tag_3 =
+ IFC_GET_CLR(fte_match_set_misc5, mask, macsec_tag_3, clr);
+ spec->tunnel_header_0 =
+ IFC_GET_CLR(fte_match_set_misc5, mask, tunnel_header_0, clr);
+ spec->tunnel_header_1 =
+ IFC_GET_CLR(fte_match_set_misc5, mask, tunnel_header_1, clr);
+ spec->tunnel_header_2 =
+ IFC_GET_CLR(fte_match_set_misc5, mask, tunnel_header_2, clr);
+ spec->tunnel_header_3 =
+ IFC_GET_CLR(fte_match_set_misc5, mask, tunnel_header_3, clr);
+}
+
void mlx5dr_ste_copy_param(u8 match_criteria,
struct mlx5dr_match_param *set_param,
struct mlx5dr_match_parameters *mask,
@@ -966,6 +988,20 @@ void mlx5dr_ste_copy_param(u8 match_criteria,
}
dr_ste_copy_mask_misc4(buff, &set_param->misc4, clr);
}
+
+ param_location += sizeof(struct mlx5dr_match_misc4);
+
+ if (match_criteria & DR_MATCHER_CRITERIA_MISC5) {
+ if (mask->match_sz < param_location +
+ sizeof(struct mlx5dr_match_misc5)) {
+ memcpy(tail_param, data + param_location,
+ mask->match_sz - param_location);
+ buff = tail_param;
+ } else {
+ buff = data + param_location;
+ }
+ dr_ste_copy_mask_misc5(buff, &set_param->misc5, clr);
+ }
}
void mlx5dr_ste_build_eth_l2_src_dst(struct mlx5dr_ste_ctx *ste_ctx,
@@ -1180,6 +1216,21 @@ void mlx5dr_ste_build_tnl_geneve_tlv_opt(struct mlx5dr_ste_ctx *ste_ctx,
ste_ctx->build_tnl_geneve_tlv_opt_init(sb, mask);
}
+void mlx5dr_ste_build_tnl_geneve_tlv_opt_exist(struct mlx5dr_ste_ctx *ste_ctx,
+ struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask,
+ struct mlx5dr_cmd_caps *caps,
+ bool inner, bool rx)
+{
+ if (!ste_ctx->build_tnl_geneve_tlv_opt_exist_init)
+ return;
+
+ sb->rx = rx;
+ sb->caps = caps;
+ sb->inner = inner;
+ ste_ctx->build_tnl_geneve_tlv_opt_exist_init(sb, mask);
+}
+
void mlx5dr_ste_build_tnl_gtpu(struct mlx5dr_ste_ctx *ste_ctx,
struct mlx5dr_ste_build *sb,
struct mlx5dr_match_param *mask,
@@ -1269,6 +1320,16 @@ void mlx5dr_ste_build_flex_parser_1(struct mlx5dr_ste_ctx *ste_ctx,
ste_ctx->build_flex_parser_1_init(sb, mask);
}
+void mlx5dr_ste_build_tnl_header_0_1(struct mlx5dr_ste_ctx *ste_ctx,
+ struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask,
+ bool inner, bool rx)
+{
+ sb->rx = rx;
+ sb->inner = inner;
+ ste_ctx->build_tnl_header_0_1_init(sb, mask);
+}
+
static struct mlx5dr_ste_ctx *mlx5dr_ste_ctx_arr[] = {
[MLX5_STEERING_FORMAT_CONNECTX_5] = &ste_ctx_v0,
[MLX5_STEERING_FORMAT_CONNECTX_6DX] = &ste_ctx_v1,
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste.h b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste.h
index 2d52d065..ca8fa32 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste.h
@@ -135,12 +135,14 @@ struct mlx5dr_ste_ctx {
void DR_STE_CTX_BUILDER(tnl_vxlan_gpe);
void DR_STE_CTX_BUILDER(tnl_geneve);
void DR_STE_CTX_BUILDER(tnl_geneve_tlv_opt);
+ void DR_STE_CTX_BUILDER(tnl_geneve_tlv_opt_exist);
void DR_STE_CTX_BUILDER(register_0);
void DR_STE_CTX_BUILDER(register_1);
void DR_STE_CTX_BUILDER(src_gvmi_qpn);
void DR_STE_CTX_BUILDER(flex_parser_0);
void DR_STE_CTX_BUILDER(flex_parser_1);
void DR_STE_CTX_BUILDER(tnl_gtpu);
+ void DR_STE_CTX_BUILDER(tnl_header_0_1);
void DR_STE_CTX_BUILDER(tnl_gtpu_flex_parser_0);
void DR_STE_CTX_BUILDER(tnl_gtpu_flex_parser_1);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste_v0.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste_v0.c
index b0649c2..2d62950 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste_v0.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste_v0.c
@@ -80,6 +80,7 @@ enum {
DR_STE_V0_LU_TYPE_GENERAL_PURPOSE = 0x18,
DR_STE_V0_LU_TYPE_STEERING_REGISTERS_0 = 0x2f,
DR_STE_V0_LU_TYPE_STEERING_REGISTERS_1 = 0x30,
+ DR_STE_V0_LU_TYPE_TUNNEL_HEADER = 0x34,
DR_STE_V0_LU_TYPE_DONT_CARE = MLX5DR_STE_LU_TYPE_DONT_CARE,
};
@@ -1704,7 +1705,7 @@ static void dr_ste_v0_set_flex_parser(u32 *misc4_field_id,
u32 id = *misc4_field_id;
u8 *parser_ptr;
- if (parser_is_used[id])
+ if (id >= DR_NUM_OF_FLEX_PARSERS || parser_is_used[id])
return;
parser_is_used[id] = true;
@@ -1875,6 +1876,27 @@ dr_ste_v0_build_tnl_gtpu_flex_parser_1_init(struct mlx5dr_ste_build *sb,
sb->ste_build_tag_func = &dr_ste_v0_build_tnl_gtpu_flex_parser_1_tag;
}
+static int dr_ste_v0_build_tnl_header_0_1_tag(struct mlx5dr_match_param *value,
+ struct mlx5dr_ste_build *sb,
+ uint8_t *tag)
+{
+ struct mlx5dr_match_misc5 *misc5 = &value->misc5;
+
+ DR_STE_SET_TAG(tunnel_header, tag, tunnel_header_0, misc5, tunnel_header_0);
+ DR_STE_SET_TAG(tunnel_header, tag, tunnel_header_1, misc5, tunnel_header_1);
+
+ return 0;
+}
+
+static void dr_ste_v0_build_tnl_header_0_1_init(struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask)
+{
+ sb->lu_type = DR_STE_V0_LU_TYPE_TUNNEL_HEADER;
+ dr_ste_v0_build_tnl_header_0_1_tag(mask, sb, sb->bit_mask);
+ sb->byte_mask = mlx5dr_ste_conv_bit_to_byte_mask(sb->bit_mask);
+ sb->ste_build_tag_func = &dr_ste_v0_build_tnl_header_0_1_tag;
+}
+
struct mlx5dr_ste_ctx ste_ctx_v0 = {
/* Builders */
.build_eth_l2_src_dst_init = &dr_ste_v0_build_eth_l2_src_dst_init,
@@ -1903,6 +1925,7 @@ struct mlx5dr_ste_ctx ste_ctx_v0 = {
.build_flex_parser_0_init = &dr_ste_v0_build_flex_parser_0_init,
.build_flex_parser_1_init = &dr_ste_v0_build_flex_parser_1_init,
.build_tnl_gtpu_init = &dr_ste_v0_build_flex_parser_tnl_gtpu_init,
+ .build_tnl_header_0_1_init = &dr_ste_v0_build_tnl_header_0_1_init,
.build_tnl_gtpu_flex_parser_0_init = &dr_ste_v0_build_tnl_gtpu_flex_parser_0_init,
.build_tnl_gtpu_flex_parser_1_init = &dr_ste_v0_build_tnl_gtpu_flex_parser_1_init,
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste_v1.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste_v1.c
index cb9cf67..6ca0680 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste_v1.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste_v1.c
@@ -47,6 +47,7 @@ enum {
DR_STE_V1_LU_TYPE_ETHL3_IPV4_MISC_I = 0x000f,
DR_STE_V1_LU_TYPE_STEERING_REGISTERS_0 = 0x010f,
DR_STE_V1_LU_TYPE_STEERING_REGISTERS_1 = 0x0110,
+ DR_STE_V1_LU_TYPE_FLEX_PARSER_OK = 0x0011,
DR_STE_V1_LU_TYPE_FLEX_PARSER_0 = 0x0111,
DR_STE_V1_LU_TYPE_FLEX_PARSER_1 = 0x0112,
DR_STE_V1_LU_TYPE_ETHL4_MISC_O = 0x0113,
@@ -1713,6 +1714,27 @@ dr_ste_v1_build_flex_parser_tnl_geneve_init(struct mlx5dr_ste_build *sb,
sb->ste_build_tag_func = &dr_ste_v1_build_flex_parser_tnl_geneve_tag;
}
+static int dr_ste_v1_build_tnl_header_0_1_tag(struct mlx5dr_match_param *value,
+ struct mlx5dr_ste_build *sb,
+ uint8_t *tag)
+{
+ struct mlx5dr_match_misc5 *misc5 = &value->misc5;
+
+ DR_STE_SET_TAG(tunnel_header, tag, tunnel_header_0, misc5, tunnel_header_0);
+ DR_STE_SET_TAG(tunnel_header, tag, tunnel_header_1, misc5, tunnel_header_1);
+
+ return 0;
+}
+
+static void dr_ste_v1_build_tnl_header_0_1_init(struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask)
+{
+ sb->lu_type = DR_STE_V1_LU_TYPE_FLEX_PARSER_TNL_HEADER;
+ dr_ste_v1_build_tnl_header_0_1_tag(mask, sb, sb->bit_mask);
+ sb->byte_mask = mlx5dr_ste_conv_bit_to_byte_mask(sb->bit_mask);
+ sb->ste_build_tag_func = &dr_ste_v1_build_tnl_header_0_1_tag;
+}
+
static int dr_ste_v1_build_register_0_tag(struct mlx5dr_match_param *value,
struct mlx5dr_ste_build *sb,
u8 *tag)
@@ -1833,7 +1855,7 @@ static void dr_ste_v1_set_flex_parser(u32 *misc4_field_id,
u32 id = *misc4_field_id;
u8 *parser_ptr;
- if (parser_is_used[id])
+ if (id >= DR_NUM_OF_FLEX_PARSERS || parser_is_used[id])
return;
parser_is_used[id] = true;
@@ -1921,6 +1943,32 @@ dr_ste_v1_build_flex_parser_tnl_geneve_tlv_opt_init(struct mlx5dr_ste_build *sb,
sb->ste_build_tag_func = &dr_ste_v1_build_flex_parser_tnl_geneve_tlv_opt_tag;
}
+static int
+dr_ste_v1_build_flex_parser_tnl_geneve_tlv_opt_exist_tag(struct mlx5dr_match_param *value,
+ struct mlx5dr_ste_build *sb,
+ uint8_t *tag)
+{
+ u8 parser_id = sb->caps->flex_parser_id_geneve_tlv_option_0;
+ struct mlx5dr_match_misc *misc = &value->misc;
+
+ if (misc->geneve_tlv_option_0_exist) {
+ MLX5_SET(ste_flex_parser_ok, tag, flex_parsers_ok, 1 << parser_id);
+ misc->geneve_tlv_option_0_exist = 0;
+ }
+
+ return 0;
+}
+
+static void
+dr_ste_v1_build_flex_parser_tnl_geneve_tlv_opt_exist_init(struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask)
+{
+ sb->lu_type = DR_STE_V1_LU_TYPE_FLEX_PARSER_OK;
+ dr_ste_v1_build_flex_parser_tnl_geneve_tlv_opt_exist_tag(mask, sb, sb->bit_mask);
+ sb->byte_mask = mlx5dr_ste_conv_bit_to_byte_mask(sb->bit_mask);
+ sb->ste_build_tag_func = &dr_ste_v1_build_flex_parser_tnl_geneve_tlv_opt_exist_tag;
+}
+
static int dr_ste_v1_build_flex_parser_tnl_gtpu_tag(struct mlx5dr_match_param *value,
struct mlx5dr_ste_build *sb,
u8 *tag)
@@ -2020,12 +2068,14 @@ struct mlx5dr_ste_ctx ste_ctx_v1 = {
.build_tnl_vxlan_gpe_init = &dr_ste_v1_build_flex_parser_tnl_vxlan_gpe_init,
.build_tnl_geneve_init = &dr_ste_v1_build_flex_parser_tnl_geneve_init,
.build_tnl_geneve_tlv_opt_init = &dr_ste_v1_build_flex_parser_tnl_geneve_tlv_opt_init,
+ .build_tnl_geneve_tlv_opt_exist_init = &dr_ste_v1_build_flex_parser_tnl_geneve_tlv_opt_exist_init,
.build_register_0_init = &dr_ste_v1_build_register_0_init,
.build_register_1_init = &dr_ste_v1_build_register_1_init,
.build_src_gvmi_qpn_init = &dr_ste_v1_build_src_gvmi_qpn_init,
.build_flex_parser_0_init = &dr_ste_v1_build_flex_parser_0_init,
.build_flex_parser_1_init = &dr_ste_v1_build_flex_parser_1_init,
.build_tnl_gtpu_init = &dr_ste_v1_build_flex_parser_tnl_gtpu_init,
+ .build_tnl_header_0_1_init = &dr_ste_v1_build_tnl_header_0_1_init,
.build_tnl_gtpu_flex_parser_0_init = &dr_ste_v1_build_tnl_gtpu_flex_parser_0_init,
.build_tnl_gtpu_flex_parser_1_init = &dr_ste_v1_build_tnl_gtpu_flex_parser_1_init,
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_table.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_table.c
index 30ae3cd..8ca1106 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_table.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_table.c
@@ -3,12 +3,47 @@
#include "dr_types.h"
+static int dr_table_set_miss_action_nic(struct mlx5dr_domain *dmn,
+ struct mlx5dr_table_rx_tx *nic_tbl,
+ struct mlx5dr_action *action)
+{
+ struct mlx5dr_matcher_rx_tx *last_nic_matcher = NULL;
+ struct mlx5dr_htbl_connect_info info;
+ struct mlx5dr_ste_htbl *last_htbl;
+ int ret;
+
+ if (!list_empty(&nic_tbl->nic_matcher_list))
+ last_nic_matcher = list_last_entry(&nic_tbl->nic_matcher_list,
+ struct mlx5dr_matcher_rx_tx,
+ list_node);
+
+ if (last_nic_matcher)
+ last_htbl = last_nic_matcher->e_anchor;
+ else
+ last_htbl = nic_tbl->s_anchor;
+
+ if (action)
+ nic_tbl->default_icm_addr =
+ nic_tbl->nic_dmn->type == DR_DOMAIN_NIC_TYPE_RX ?
+ action->dest_tbl->tbl->rx.s_anchor->chunk->icm_addr :
+ action->dest_tbl->tbl->tx.s_anchor->chunk->icm_addr;
+ else
+ nic_tbl->default_icm_addr = nic_tbl->nic_dmn->default_icm_addr;
+
+ info.type = CONNECT_MISS;
+ info.miss_icm_addr = nic_tbl->default_icm_addr;
+
+ ret = mlx5dr_ste_htbl_init_and_postsend(dmn, nic_tbl->nic_dmn,
+ last_htbl, &info, true);
+ if (ret)
+ mlx5dr_dbg(dmn, "Failed to set NIC RX/TX miss action, ret %d\n", ret);
+
+ return ret;
+}
+
int mlx5dr_table_set_miss_action(struct mlx5dr_table *tbl,
struct mlx5dr_action *action)
{
- struct mlx5dr_matcher *last_matcher = NULL;
- struct mlx5dr_htbl_connect_info info;
- struct mlx5dr_ste_htbl *last_htbl;
int ret;
if (action && action->action_type != DR_ACTION_TYP_FT)
@@ -16,56 +51,18 @@ int mlx5dr_table_set_miss_action(struct mlx5dr_table *tbl,
mlx5dr_domain_lock(tbl->dmn);
- if (!list_empty(&tbl->matcher_list))
- last_matcher = list_last_entry(&tbl->matcher_list,
- struct mlx5dr_matcher,
- matcher_list);
-
if (tbl->dmn->type == MLX5DR_DOMAIN_TYPE_NIC_RX ||
tbl->dmn->type == MLX5DR_DOMAIN_TYPE_FDB) {
- if (last_matcher)
- last_htbl = last_matcher->rx.e_anchor;
- else
- last_htbl = tbl->rx.s_anchor;
-
- tbl->rx.default_icm_addr = action ?
- action->dest_tbl->tbl->rx.s_anchor->chunk->icm_addr :
- tbl->rx.nic_dmn->default_icm_addr;
-
- info.type = CONNECT_MISS;
- info.miss_icm_addr = tbl->rx.default_icm_addr;
-
- ret = mlx5dr_ste_htbl_init_and_postsend(tbl->dmn,
- tbl->rx.nic_dmn,
- last_htbl,
- &info, true);
- if (ret) {
- mlx5dr_dbg(tbl->dmn, "Failed to set RX miss action, ret %d\n", ret);
+ ret = dr_table_set_miss_action_nic(tbl->dmn, &tbl->rx, action);
+ if (ret)
goto out;
- }
}
if (tbl->dmn->type == MLX5DR_DOMAIN_TYPE_NIC_TX ||
tbl->dmn->type == MLX5DR_DOMAIN_TYPE_FDB) {
- if (last_matcher)
- last_htbl = last_matcher->tx.e_anchor;
- else
- last_htbl = tbl->tx.s_anchor;
-
- tbl->tx.default_icm_addr = action ?
- action->dest_tbl->tbl->tx.s_anchor->chunk->icm_addr :
- tbl->tx.nic_dmn->default_icm_addr;
-
- info.type = CONNECT_MISS;
- info.miss_icm_addr = tbl->tx.default_icm_addr;
-
- ret = mlx5dr_ste_htbl_init_and_postsend(tbl->dmn,
- tbl->tx.nic_dmn,
- last_htbl, &info, true);
- if (ret) {
- mlx5dr_dbg(tbl->dmn, "Failed to set TX miss action, ret %d\n", ret);
+ ret = dr_table_set_miss_action_nic(tbl->dmn, &tbl->tx, action);
+ if (ret)
goto out;
- }
}
/* Release old action */
@@ -122,6 +119,8 @@ static int dr_table_init_nic(struct mlx5dr_domain *dmn,
struct mlx5dr_htbl_connect_info info;
int ret;
+ INIT_LIST_HEAD(&nic_tbl->nic_matcher_list);
+
nic_tbl->default_icm_addr = nic_dmn->default_icm_addr;
nic_tbl->s_anchor = mlx5dr_ste_htbl_alloc(dmn->ste_icm_pool,
@@ -266,6 +265,8 @@ struct mlx5dr_table *mlx5dr_table_create(struct mlx5dr_domain *dmn, u32 level, u
if (ret)
goto uninit_tbl;
+ INIT_LIST_HEAD(&tbl->dbg_node);
+ mlx5dr_dbg_tbl_add(tbl);
return tbl;
uninit_tbl:
@@ -281,9 +282,10 @@ int mlx5dr_table_destroy(struct mlx5dr_table *tbl)
{
int ret;
- if (refcount_read(&tbl->refcount) > 1)
+ if (WARN_ON_ONCE(refcount_read(&tbl->refcount) > 1))
return -EBUSY;
+ mlx5dr_dbg_tbl_del(tbl);
ret = dr_table_destroy_sw_owned_tbl(tbl);
if (ret)
return ret;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_types.h b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_types.h
index 2333c24..1b3d484 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_types.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_types.h
@@ -11,6 +11,7 @@
#include "lib/mlx5.h"
#include "mlx5_ifc_dr.h"
#include "mlx5dr.h"
+#include "dr_dbg.h"
#define DR_RULE_MAX_STES 18
#define DR_ACTION_MAX_STES 5
@@ -104,7 +105,8 @@ enum mlx5dr_matcher_criteria {
DR_MATCHER_CRITERIA_MISC2 = 1 << 3,
DR_MATCHER_CRITERIA_MISC3 = 1 << 4,
DR_MATCHER_CRITERIA_MISC4 = 1 << 5,
- DR_MATCHER_CRITERIA_MAX = 1 << 6,
+ DR_MATCHER_CRITERIA_MISC5 = 1 << 6,
+ DR_MATCHER_CRITERIA_MAX = 1 << 7,
};
enum mlx5dr_action_type {
@@ -440,6 +442,11 @@ void mlx5dr_ste_build_tnl_geneve_tlv_opt(struct mlx5dr_ste_ctx *ste_ctx,
struct mlx5dr_match_param *mask,
struct mlx5dr_cmd_caps *caps,
bool inner, bool rx);
+void mlx5dr_ste_build_tnl_geneve_tlv_opt_exist(struct mlx5dr_ste_ctx *ste_ctx,
+ struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask,
+ struct mlx5dr_cmd_caps *caps,
+ bool inner, bool rx);
void mlx5dr_ste_build_tnl_gtpu(struct mlx5dr_ste_ctx *ste_ctx,
struct mlx5dr_ste_build *sb,
struct mlx5dr_match_param *mask,
@@ -454,6 +461,10 @@ void mlx5dr_ste_build_tnl_gtpu_flex_parser_1(struct mlx5dr_ste_ctx *ste_ctx,
struct mlx5dr_match_param *mask,
struct mlx5dr_cmd_caps *caps,
bool inner, bool rx);
+void mlx5dr_ste_build_tnl_header_0_1(struct mlx5dr_ste_ctx *ste_ctx,
+ struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask,
+ bool inner, bool rx);
void mlx5dr_ste_build_general_purpose(struct mlx5dr_ste_ctx *ste_ctx,
struct mlx5dr_ste_build *sb,
struct mlx5dr_match_param *mask,
@@ -494,57 +505,64 @@ struct mlx5dr_match_spec {
/* Incoming packet Ethertype - this is the Ethertype
* following the last VLAN tag of the packet
*/
- u32 ethertype:16;
u32 smac_15_0:16; /* Source MAC address of incoming packet */
+ u32 ethertype:16;
+
u32 dmac_47_16; /* Destination MAC address of incoming packet */
- /* VLAN ID of first VLAN tag in the incoming packet.
- * Valid only when cvlan_tag==1 or svlan_tag==1
- */
- u32 first_vid:12;
- /* CFI bit of first VLAN tag in the incoming packet.
- * Valid only when cvlan_tag==1 or svlan_tag==1
- */
- u32 first_cfi:1;
+
+ u32 dmac_15_0:16; /* Destination MAC address of incoming packet */
/* Priority of first VLAN tag in the incoming packet.
* Valid only when cvlan_tag==1 or svlan_tag==1
*/
u32 first_prio:3;
- u32 dmac_15_0:16; /* Destination MAC address of incoming packet */
- /* TCP flags. ;Bit 0: FIN;Bit 1: SYN;Bit 2: RST;Bit 3: PSH;Bit 4: ACK;
- * Bit 5: URG;Bit 6: ECE;Bit 7: CWR;Bit 8: NS
+ /* CFI bit of first VLAN tag in the incoming packet.
+ * Valid only when cvlan_tag==1 or svlan_tag==1
*/
- u32 tcp_flags:9;
- u32 ip_version:4; /* IP version */
- u32 frag:1; /* Packet is an IP fragment */
- /* The first vlan in the packet is s-vlan (0x8a88).
- * cvlan_tag and svlan_tag cannot be set together
+ u32 first_cfi:1;
+ /* VLAN ID of first VLAN tag in the incoming packet.
+ * Valid only when cvlan_tag==1 or svlan_tag==1
*/
- u32 svlan_tag:1;
- /* The first vlan in the packet is c-vlan (0x8100).
- * cvlan_tag and svlan_tag cannot be set together
- */
- u32 cvlan_tag:1;
- /* Explicit Congestion Notification derived from
- * Traffic Class/TOS field of IPv6/v4
- */
- u32 ip_ecn:2;
+ u32 first_vid:12;
+
+ u32 ip_protocol:8; /* IP protocol */
/* Differentiated Services Code Point derived from
* Traffic Class/TOS field of IPv6/v4
*/
u32 ip_dscp:6;
- u32 ip_protocol:8; /* IP protocol */
+ /* Explicit Congestion Notification derived from
+ * Traffic Class/TOS field of IPv6/v4
+ */
+ u32 ip_ecn:2;
+ /* The first vlan in the packet is c-vlan (0x8100).
+ * cvlan_tag and svlan_tag cannot be set together
+ */
+ u32 cvlan_tag:1;
+ /* The first vlan in the packet is s-vlan (0x8a88).
+ * cvlan_tag and svlan_tag cannot be set together
+ */
+ u32 svlan_tag:1;
+ u32 frag:1; /* Packet is an IP fragment */
+ u32 ip_version:4; /* IP version */
+ /* TCP flags. ;Bit 0: FIN;Bit 1: SYN;Bit 2: RST;Bit 3: PSH;Bit 4: ACK;
+ * Bit 5: URG;Bit 6: ECE;Bit 7: CWR;Bit 8: NS
+ */
+ u32 tcp_flags:9;
+
+ /* TCP source port.;tcp and udp sport/dport are mutually exclusive */
+ u32 tcp_sport:16;
/* TCP destination port.
* tcp and udp sport/dport are mutually exclusive
*/
u32 tcp_dport:16;
- /* TCP source port.;tcp and udp sport/dport are mutually exclusive */
- u32 tcp_sport:16;
+
+ u32 reserved_auto1:24;
u32 ttl_hoplimit:8;
- u32 reserved:24;
- /* UDP destination port.;tcp and udp sport/dport are mutually exclusive */
- u32 udp_dport:16;
+
/* UDP source port.;tcp and udp sport/dport are mutually exclusive */
u32 udp_sport:16;
+ /* UDP destination port.;tcp and udp sport/dport are mutually exclusive */
+ u32 udp_dport:16;
+
/* IPv6 source address of incoming packets
* For IPv4 address use bits 31:0 (rest of the bits are reserved)
* This field should be qualified by an appropriate ethertype
@@ -588,96 +606,114 @@ struct mlx5dr_match_spec {
};
struct mlx5dr_match_misc {
- u32 source_sqn:24; /* Source SQN */
- u32 source_vhca_port:4;
- /* used with GRE, sequence number exist when gre_s_present == 1 */
- u32 gre_s_present:1;
- /* used with GRE, key exist when gre_k_present == 1 */
- u32 gre_k_present:1;
- u32 reserved_auto1:1;
/* used with GRE, checksum exist when gre_c_present == 1 */
u32 gre_c_present:1;
+ u32 reserved_auto1:1;
+ /* used with GRE, key exist when gre_k_present == 1 */
+ u32 gre_k_present:1;
+ /* used with GRE, sequence number exist when gre_s_present == 1 */
+ u32 gre_s_present:1;
+ u32 source_vhca_port:4;
+ u32 source_sqn:24; /* Source SQN */
+
+ u32 source_eswitch_owner_vhca_id:16;
/* Source port.;0xffff determines wire port */
u32 source_port:16;
- u32 source_eswitch_owner_vhca_id:16;
- /* VLAN ID of first VLAN tag the inner header of the incoming packet.
- * Valid only when inner_second_cvlan_tag ==1 or inner_second_svlan_tag ==1
- */
- u32 inner_second_vid:12;
- /* CFI bit of first VLAN tag in the inner header of the incoming packet.
- * Valid only when inner_second_cvlan_tag ==1 or inner_second_svlan_tag ==1
- */
- u32 inner_second_cfi:1;
- /* Priority of second VLAN tag in the inner header of the incoming packet.
- * Valid only when inner_second_cvlan_tag ==1 or inner_second_svlan_tag ==1
- */
- u32 inner_second_prio:3;
- /* VLAN ID of first VLAN tag the outer header of the incoming packet.
- * Valid only when outer_second_cvlan_tag ==1 or outer_second_svlan_tag ==1
- */
- u32 outer_second_vid:12;
- /* CFI bit of first VLAN tag in the outer header of the incoming packet.
- * Valid only when outer_second_cvlan_tag ==1 or outer_second_svlan_tag ==1
- */
- u32 outer_second_cfi:1;
+
/* Priority of second VLAN tag in the outer header of the incoming packet.
* Valid only when outer_second_cvlan_tag ==1 or outer_second_svlan_tag ==1
*/
u32 outer_second_prio:3;
- u32 gre_protocol:16; /* GRE Protocol (outer) */
- u32 reserved_auto3:12;
- /* The second vlan in the inner header of the packet is s-vlan (0x8a88).
- * inner_second_cvlan_tag and inner_second_svlan_tag cannot be set together
+ /* CFI bit of first VLAN tag in the outer header of the incoming packet.
+ * Valid only when outer_second_cvlan_tag ==1 or outer_second_svlan_tag ==1
*/
- u32 inner_second_svlan_tag:1;
- /* The second vlan in the outer header of the packet is s-vlan (0x8a88).
+ u32 outer_second_cfi:1;
+ /* VLAN ID of first VLAN tag the outer header of the incoming packet.
+ * Valid only when outer_second_cvlan_tag ==1 or outer_second_svlan_tag ==1
+ */
+ u32 outer_second_vid:12;
+ /* Priority of second VLAN tag in the inner header of the incoming packet.
+ * Valid only when inner_second_cvlan_tag ==1 or inner_second_svlan_tag ==1
+ */
+ u32 inner_second_prio:3;
+ /* CFI bit of first VLAN tag in the inner header of the incoming packet.
+ * Valid only when inner_second_cvlan_tag ==1 or inner_second_svlan_tag ==1
+ */
+ u32 inner_second_cfi:1;
+ /* VLAN ID of first VLAN tag the inner header of the incoming packet.
+ * Valid only when inner_second_cvlan_tag ==1 or inner_second_svlan_tag ==1
+ */
+ u32 inner_second_vid:12;
+
+ u32 outer_second_cvlan_tag:1;
+ u32 inner_second_cvlan_tag:1;
+ /* The second vlan in the outer header of the packet is c-vlan (0x8100).
* outer_second_cvlan_tag and outer_second_svlan_tag cannot be set together
*/
u32 outer_second_svlan_tag:1;
/* The second vlan in the inner header of the packet is c-vlan (0x8100).
* inner_second_cvlan_tag and inner_second_svlan_tag cannot be set together
*/
- u32 inner_second_cvlan_tag:1;
- /* The second vlan in the outer header of the packet is c-vlan (0x8100).
+ u32 inner_second_svlan_tag:1;
+ /* The second vlan in the outer header of the packet is s-vlan (0x8a88).
* outer_second_cvlan_tag and outer_second_svlan_tag cannot be set together
*/
- u32 outer_second_cvlan_tag:1;
- u32 gre_key_l:8; /* GRE Key [7:0] (outer) */
+ u32 reserved_auto2:12;
+ /* The second vlan in the inner header of the packet is s-vlan (0x8a88).
+ * inner_second_cvlan_tag and inner_second_svlan_tag cannot be set together
+ */
+ u32 gre_protocol:16; /* GRE Protocol (outer) */
+
u32 gre_key_h:24; /* GRE Key[31:8] (outer) */
- u32 reserved_auto4:8;
+ u32 gre_key_l:8; /* GRE Key [7:0] (outer) */
+
u32 vxlan_vni:24; /* VXLAN VNI (outer) */
- u32 geneve_oam:1; /* GENEVE OAM field (outer) */
- u32 reserved_auto5:7;
+ u32 reserved_auto3:8;
+
u32 geneve_vni:24; /* GENEVE VNI field (outer) */
+ u32 reserved_auto4:6;
+ u32 geneve_tlv_option_0_exist:1;
+ u32 geneve_oam:1; /* GENEVE OAM field (outer) */
+
+ u32 reserved_auto5:12;
u32 outer_ipv6_flow_label:20; /* Flow label of incoming IPv6 packet (outer) */
+
u32 reserved_auto6:12;
u32 inner_ipv6_flow_label:20; /* Flow label of incoming IPv6 packet (inner) */
- u32 reserved_auto7:12;
- u32 geneve_protocol_type:16; /* GENEVE protocol type (outer) */
+
+ u32 reserved_auto7:10;
u32 geneve_opt_len:6; /* GENEVE OptLen (outer) */
- u32 reserved_auto8:10;
+ u32 geneve_protocol_type:16; /* GENEVE protocol type (outer) */
+
+ u32 reserved_auto8:8;
u32 bth_dst_qp:24; /* Destination QP in BTH header */
- u32 reserved_auto9:8;
- u8 reserved_auto10[20];
+
+ u32 reserved_auto9;
+ u32 outer_esp_spi;
+ u32 reserved_auto10[3];
};
struct mlx5dr_match_misc2 {
- u32 outer_first_mpls_ttl:8; /* First MPLS TTL (outer) */
- u32 outer_first_mpls_s_bos:1; /* First MPLS S_BOS (outer) */
- u32 outer_first_mpls_exp:3; /* First MPLS EXP (outer) */
u32 outer_first_mpls_label:20; /* First MPLS LABEL (outer) */
- u32 inner_first_mpls_ttl:8; /* First MPLS TTL (inner) */
- u32 inner_first_mpls_s_bos:1; /* First MPLS S_BOS (inner) */
- u32 inner_first_mpls_exp:3; /* First MPLS EXP (inner) */
+ u32 outer_first_mpls_exp:3; /* First MPLS EXP (outer) */
+ u32 outer_first_mpls_s_bos:1; /* First MPLS S_BOS (outer) */
+ u32 outer_first_mpls_ttl:8; /* First MPLS TTL (outer) */
+
u32 inner_first_mpls_label:20; /* First MPLS LABEL (inner) */
- u32 outer_first_mpls_over_gre_ttl:8; /* last MPLS TTL (outer) */
- u32 outer_first_mpls_over_gre_s_bos:1; /* last MPLS S_BOS (outer) */
- u32 outer_first_mpls_over_gre_exp:3; /* last MPLS EXP (outer) */
+ u32 inner_first_mpls_exp:3; /* First MPLS EXP (inner) */
+ u32 inner_first_mpls_s_bos:1; /* First MPLS S_BOS (inner) */
+ u32 inner_first_mpls_ttl:8; /* First MPLS TTL (inner) */
+
u32 outer_first_mpls_over_gre_label:20; /* last MPLS LABEL (outer) */
- u32 outer_first_mpls_over_udp_ttl:8; /* last MPLS TTL (outer) */
- u32 outer_first_mpls_over_udp_s_bos:1; /* last MPLS S_BOS (outer) */
- u32 outer_first_mpls_over_udp_exp:3; /* last MPLS EXP (outer) */
+ u32 outer_first_mpls_over_gre_exp:3; /* last MPLS EXP (outer) */
+ u32 outer_first_mpls_over_gre_s_bos:1; /* last MPLS S_BOS (outer) */
+ u32 outer_first_mpls_over_gre_ttl:8; /* last MPLS TTL (outer) */
+
u32 outer_first_mpls_over_udp_label:20; /* last MPLS LABEL (outer) */
+ u32 outer_first_mpls_over_udp_exp:3; /* last MPLS EXP (outer) */
+ u32 outer_first_mpls_over_udp_s_bos:1; /* last MPLS S_BOS (outer) */
+ u32 outer_first_mpls_over_udp_ttl:8; /* last MPLS TTL (outer) */
+
u32 metadata_reg_c_7; /* metadata_reg_c_7 */
u32 metadata_reg_c_6; /* metadata_reg_c_6 */
u32 metadata_reg_c_5; /* metadata_reg_c_5 */
@@ -687,7 +723,7 @@ struct mlx5dr_match_misc2 {
u32 metadata_reg_c_1; /* metadata_reg_c_1 */
u32 metadata_reg_c_0; /* metadata_reg_c_0 */
u32 metadata_reg_a; /* metadata_reg_a */
- u8 reserved_auto2[12];
+ u32 reserved_auto1[3];
};
struct mlx5dr_match_misc3 {
@@ -695,24 +731,34 @@ struct mlx5dr_match_misc3 {
u32 outer_tcp_seq_num;
u32 inner_tcp_ack_num;
u32 outer_tcp_ack_num;
- u32 outer_vxlan_gpe_vni:24;
+
u32 reserved_auto1:8;
- u32 reserved_auto2:16;
- u32 outer_vxlan_gpe_flags:8;
+ u32 outer_vxlan_gpe_vni:24;
+
u32 outer_vxlan_gpe_next_protocol:8;
+ u32 outer_vxlan_gpe_flags:8;
+ u32 reserved_auto2:16;
+
u32 icmpv4_header_data;
u32 icmpv6_header_data;
- u8 icmpv6_code;
- u8 icmpv6_type;
- u8 icmpv4_code;
+
u8 icmpv4_type;
+ u8 icmpv4_code;
+ u8 icmpv6_type;
+ u8 icmpv6_code;
+
u32 geneve_tlv_option_0_data;
- u8 gtpu_msg_flags;
- u8 gtpu_msg_type;
+
u32 gtpu_teid;
+
+ u8 gtpu_msg_type;
+ u8 gtpu_msg_flags;
+ u32 reserved_auto3:16;
+
u32 gtpu_dw_2;
u32 gtpu_first_ext_dw_0;
u32 gtpu_dw_0;
+ u32 reserved_auto4;
};
struct mlx5dr_match_misc4 {
@@ -724,6 +770,18 @@ struct mlx5dr_match_misc4 {
u32 prog_sample_field_id_2;
u32 prog_sample_field_value_3;
u32 prog_sample_field_id_3;
+ u32 reserved_auto1[8];
+};
+
+struct mlx5dr_match_misc5 {
+ u32 macsec_tag_0;
+ u32 macsec_tag_1;
+ u32 macsec_tag_2;
+ u32 macsec_tag_3;
+ u32 tunnel_header_0;
+ u32 tunnel_header_1;
+ u32 tunnel_header_2;
+ u32 tunnel_header_3;
};
struct mlx5dr_match_param {
@@ -733,6 +791,7 @@ struct mlx5dr_match_param {
struct mlx5dr_match_misc2 misc2;
struct mlx5dr_match_misc3 misc3;
struct mlx5dr_match_misc4 misc4;
+ struct mlx5dr_match_misc5 misc5;
};
#define DR_MASK_IS_ICMPV4_SET(_misc3) ((_misc3)->icmpv4_type || \
@@ -789,6 +848,7 @@ struct mlx5dr_cmd_caps {
u8 flex_parser_id_gtpu_teid;
u8 flex_parser_id_gtpu_dw_2;
u8 flex_parser_id_gtpu_first_ext_dw_0;
+ u8 flex_parser_ok_bits_supp;
u8 max_ft_level;
u16 roce_min_src_udp;
u8 sw_format_ver;
@@ -843,12 +903,15 @@ struct mlx5dr_domain {
struct mlx5dr_domain_info info;
struct xarray csum_fts_xa;
struct mlx5dr_ste_ctx *ste_ctx;
+ struct list_head dbg_tbl_list;
+ struct mlx5dr_dbg_dump_info dump_info;
};
struct mlx5dr_table_rx_tx {
struct mlx5dr_ste_htbl *s_anchor;
struct mlx5dr_domain_rx_tx *nic_dmn;
u64 default_icm_addr;
+ struct list_head nic_matcher_list;
};
struct mlx5dr_table {
@@ -862,6 +925,7 @@ struct mlx5dr_table {
struct list_head matcher_list;
struct mlx5dr_action *miss_action;
refcount_t refcount;
+ struct list_head dbg_node;
};
struct mlx5dr_matcher_rx_tx {
@@ -875,18 +939,21 @@ struct mlx5dr_matcher_rx_tx {
u8 num_of_builders_arr[DR_RULE_IPV_MAX][DR_RULE_IPV_MAX];
u64 default_icm_addr;
struct mlx5dr_table_rx_tx *nic_tbl;
+ u32 prio;
+ struct list_head list_node;
+ u32 rules;
};
struct mlx5dr_matcher {
struct mlx5dr_table *tbl;
struct mlx5dr_matcher_rx_tx rx;
struct mlx5dr_matcher_rx_tx tx;
- struct list_head matcher_list;
+ struct list_head list_node; /* Used for both matchers and dbg managing */
u32 prio;
struct mlx5dr_match_param mask;
u8 match_criteria;
refcount_t refcount;
- struct mlx5dv_flow_matcher *dv_matcher;
+ struct list_head dbg_rule_list;
};
struct mlx5dr_ste_action_modify_field {
@@ -958,6 +1025,11 @@ struct mlx5dr_action_flow_tag {
u32 flow_tag;
};
+struct mlx5dr_rule_action_member {
+ struct mlx5dr_action *action;
+ struct list_head list;
+};
+
struct mlx5dr_action {
enum mlx5dr_action_type action_type;
refcount_t refcount;
@@ -998,6 +1070,7 @@ struct mlx5dr_rule {
struct mlx5dr_rule_rx_tx rx;
struct mlx5dr_rule_rx_tx tx;
struct list_head rule_actions_list;
+ struct list_head dbg_node;
u32 flow_source;
};
@@ -1050,6 +1123,11 @@ static inline void mlx5dr_domain_unlock(struct mlx5dr_domain *dmn)
mlx5dr_domain_nic_unlock(&dmn->info.rx);
}
+int mlx5dr_matcher_add_to_tbl_nic(struct mlx5dr_domain *dmn,
+ struct mlx5dr_matcher_rx_tx *nic_matcher);
+int mlx5dr_matcher_remove_from_tbl_nic(struct mlx5dr_domain *dmn,
+ struct mlx5dr_matcher_rx_tx *nic_matcher);
+
int mlx5dr_matcher_select_builders(struct mlx5dr_matcher *matcher,
struct mlx5dr_matcher_rx_tx *nic_matcher,
enum mlx5dr_ipv outer_ipv,
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/fs_dr.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/fs_dr.c
index 2632d5a..a476da2 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/steering/fs_dr.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/fs_dr.c
@@ -1,6 +1,7 @@
// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
/* Copyright (c) 2019 Mellanox Technologies */
+#include <linux/mlx5/vport.h>
#include "mlx5_core.h"
#include "fs_core.h"
#include "fs_cmd.h"
@@ -194,6 +195,15 @@ static struct mlx5dr_action *create_vport_action(struct mlx5dr_domain *domain,
dest_attr->vport.vhca_id);
}
+static struct mlx5dr_action *create_uplink_action(struct mlx5dr_domain *domain,
+ struct mlx5_flow_rule *dst)
+{
+ struct mlx5_flow_destination *dest_attr = &dst->dest_attr;
+
+ return mlx5dr_action_create_dest_vport(domain, MLX5_VPORT_UPLINK, 1,
+ dest_attr->vport.vhca_id);
+}
+
static struct mlx5dr_action *create_ft_action(struct mlx5dr_domain *domain,
struct mlx5_flow_rule *dst)
{
@@ -218,7 +228,8 @@ static struct mlx5dr_action *create_action_push_vlan(struct mlx5dr_domain *domai
static bool contain_vport_reformat_action(struct mlx5_flow_rule *dst)
{
- return dst->dest_attr.type == MLX5_FLOW_DESTINATION_TYPE_VPORT &&
+ return (dst->dest_attr.type == MLX5_FLOW_DESTINATION_TYPE_VPORT ||
+ dst->dest_attr.type == MLX5_FLOW_DESTINATION_TYPE_UPLINK) &&
dst->dest_attr.vport.flags & MLX5_FLOW_DEST_VPORT_REFORMAT_ID;
}
@@ -411,8 +422,11 @@ static int mlx5_cmd_dr_create_fte(struct mlx5_flow_root_namespace *ns,
fs_dr_actions[fs_dr_num_actions++] = tmp_action;
term_actions[num_term_actions++].dest = tmp_action;
break;
+ case MLX5_FLOW_DESTINATION_TYPE_UPLINK:
case MLX5_FLOW_DESTINATION_TYPE_VPORT:
- tmp_action = create_vport_action(domain, dst);
+ tmp_action = type == MLX5_FLOW_DESTINATION_TYPE_VPORT ?
+ create_vport_action(domain, dst) :
+ create_uplink_action(domain, dst);
if (!tmp_action) {
err = -ENOMEM;
goto free_actions;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/mlx5_ifc_dr.h b/drivers/net/ethernet/mellanox/mlx5/core/steering/mlx5_ifc_dr.h
index d2a937f6..9604b20 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/steering/mlx5_ifc_dr.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/mlx5_ifc_dr.h
@@ -447,6 +447,14 @@ struct mlx5_ifc_ste_flex_parser_1_bits {
u8 flex_parser_4[0x20];
};
+struct mlx5_ifc_ste_flex_parser_ok_bits {
+ u8 flex_parser_3[0x20];
+ u8 flex_parser_2[0x20];
+ u8 flex_parsers_ok[0x8];
+ u8 reserved_at_48[0x18];
+ u8 flex_parser_0[0x20];
+};
+
struct mlx5_ifc_ste_flex_parser_tnl_bits {
u8 flex_parser_tunneling_header_63_32[0x20];
@@ -490,6 +498,14 @@ struct mlx5_ifc_ste_flex_parser_tnl_gtpu_bits {
u8 reserved_at_40[0x40];
};
+struct mlx5_ifc_ste_tunnel_header_bits {
+ u8 tunnel_header_0[0x20];
+
+ u8 tunnel_header_1[0x20];
+
+ u8 reserved_at_40[0x40];
+};
+
struct mlx5_ifc_ste_general_purpose_bits {
u8 general_purpose_lookup_field[0x20];
diff --git a/drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_ethtool.c b/drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_ethtool.c
index 92b798f..ceeb7f4 100644
--- a/drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_ethtool.c
+++ b/drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_ethtool.c
@@ -33,8 +33,11 @@ static void mlxbf_gige_get_regs(struct net_device *netdev,
memcpy_fromio(p, priv->base, MLXBF_GIGE_MMIO_REG_SZ);
}
-static void mlxbf_gige_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ering)
+static void
+mlxbf_gige_get_ringparam(struct net_device *netdev,
+ struct ethtool_ringparam *ering,
+ struct kernel_ethtool_ringparam *kernel_ering,
+ struct netlink_ext_ack *extack)
{
struct mlxbf_gige *priv = netdev_priv(netdev);
diff --git a/drivers/net/ethernet/mellanox/mlxsw/Kconfig b/drivers/net/ethernet/mellanox/mlxsw/Kconfig
index d1ae248..4683312 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/Kconfig
+++ b/drivers/net/ethernet/mellanox/mlxsw/Kconfig
@@ -66,7 +66,7 @@
default m
help
This driver supports Mellanox Technologies
- Spectrum/Spectrum-2/Spectrum-3 Ethernet Switch ASICs.
+ Spectrum/Spectrum-2/Spectrum-3/Spectrum-4 Ethernet Switch ASICs.
To compile this driver as a module, choose M here: the
module will be called mlxsw_spectrum.
diff --git a/drivers/net/ethernet/mellanox/mlxsw/cmd.h b/drivers/net/ethernet/mellanox/mlxsw/cmd.h
index 392ce3c..51b260d 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/cmd.h
+++ b/drivers/net/ethernet/mellanox/mlxsw/cmd.h
@@ -935,6 +935,18 @@ static inline int mlxsw_cmd_sw2hw_rdq(struct mlxsw_core *mlxsw_core,
*/
MLXSW_ITEM32(cmd_mbox, sw2hw_dq, cq, 0x00, 24, 8);
+enum mlxsw_cmd_mbox_sw2hw_dq_sdq_lp {
+ MLXSW_CMD_MBOX_SW2HW_DQ_SDQ_LP_WQE,
+ MLXSW_CMD_MBOX_SW2HW_DQ_SDQ_LP_IGNORE_WQE,
+};
+
+/* cmd_mbox_sw2hw_dq_sdq_lp
+ * SDQ local Processing
+ * 0: local processing by wqe.lp
+ * 1: local processing (ignoring wqe.lp)
+ */
+MLXSW_ITEM32(cmd_mbox, sw2hw_dq, sdq_lp, 0x00, 23, 1);
+
/* cmd_mbox_sw2hw_dq_sdq_tclass
* SDQ: CPU Egress TClass
* RDQ: Reserved
diff --git a/drivers/net/ethernet/mellanox/mlxsw/core.c b/drivers/net/ethernet/mellanox/mlxsw/core.c
index 3fd3812..866b935 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/core.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/core.c
@@ -47,7 +47,7 @@ static struct workqueue_struct *mlxsw_owq;
struct mlxsw_core_port {
struct devlink_port devlink_port;
void *port_driver_priv;
- u8 local_port;
+ u16 local_port;
};
void *mlxsw_core_port_driver_priv(struct mlxsw_core_port *mlxsw_core_port)
@@ -77,7 +77,7 @@ struct mlxsw_core {
bool enable_string_tlv;
} emad;
struct {
- u8 *mapping; /* lag_id+port_index to local_port mapping */
+ u16 *mapping; /* lag_id+port_index to local_port mapping */
} lag;
struct mlxsw_res res;
struct mlxsw_hwmon *hwmon;
@@ -160,7 +160,7 @@ static void mlxsw_ports_fini(struct mlxsw_core *mlxsw_core, bool reload)
devlink_resource_occ_get_unregister(devlink, MLXSW_CORE_RESOURCE_PORTS);
if (!reload)
- devlink_resources_unregister(priv_to_devlink(mlxsw_core), NULL);
+ devlink_resources_unregister(priv_to_devlink(mlxsw_core));
kfree(mlxsw_core->ports);
}
@@ -718,7 +718,7 @@ static void mlxsw_emad_process_response(struct mlxsw_core *mlxsw_core,
}
/* called with rcu read lock held */
-static void mlxsw_emad_rx_listener_func(struct sk_buff *skb, u8 local_port,
+static void mlxsw_emad_rx_listener_func(struct sk_buff *skb, u16 local_port,
void *priv)
{
struct mlxsw_core *mlxsw_core = priv;
@@ -1708,6 +1708,124 @@ static void mlxsw_core_health_listener_func(const struct mlxsw_reg_info *reg,
static const struct mlxsw_listener mlxsw_core_health_listener =
MLXSW_EVENTL(mlxsw_core_health_listener_func, MFDE, MFDE);
+static int
+mlxsw_core_health_fw_fatal_dump_fatal_cause(const char *mfde_pl,
+ struct devlink_fmsg *fmsg)
+{
+ u32 val, tile_v;
+ int err;
+
+ val = mlxsw_reg_mfde_fatal_cause_id_get(mfde_pl);
+ err = devlink_fmsg_u32_pair_put(fmsg, "cause_id", val);
+ if (err)
+ return err;
+ tile_v = mlxsw_reg_mfde_fatal_cause_tile_v_get(mfde_pl);
+ if (tile_v) {
+ val = mlxsw_reg_mfde_fatal_cause_tile_index_get(mfde_pl);
+ err = devlink_fmsg_u8_pair_put(fmsg, "tile_index", val);
+ if (err)
+ return err;
+ }
+
+ return 0;
+}
+
+static int
+mlxsw_core_health_fw_fatal_dump_fw_assert(const char *mfde_pl,
+ struct devlink_fmsg *fmsg)
+{
+ u32 val, tile_v;
+ int err;
+
+ val = mlxsw_reg_mfde_fw_assert_var0_get(mfde_pl);
+ err = devlink_fmsg_u32_pair_put(fmsg, "var0", val);
+ if (err)
+ return err;
+ val = mlxsw_reg_mfde_fw_assert_var1_get(mfde_pl);
+ err = devlink_fmsg_u32_pair_put(fmsg, "var1", val);
+ if (err)
+ return err;
+ val = mlxsw_reg_mfde_fw_assert_var2_get(mfde_pl);
+ err = devlink_fmsg_u32_pair_put(fmsg, "var2", val);
+ if (err)
+ return err;
+ val = mlxsw_reg_mfde_fw_assert_var3_get(mfde_pl);
+ err = devlink_fmsg_u32_pair_put(fmsg, "var3", val);
+ if (err)
+ return err;
+ val = mlxsw_reg_mfde_fw_assert_var4_get(mfde_pl);
+ err = devlink_fmsg_u32_pair_put(fmsg, "var4", val);
+ if (err)
+ return err;
+ val = mlxsw_reg_mfde_fw_assert_existptr_get(mfde_pl);
+ err = devlink_fmsg_u32_pair_put(fmsg, "existptr", val);
+ if (err)
+ return err;
+ val = mlxsw_reg_mfde_fw_assert_callra_get(mfde_pl);
+ err = devlink_fmsg_u32_pair_put(fmsg, "callra", val);
+ if (err)
+ return err;
+ val = mlxsw_reg_mfde_fw_assert_oe_get(mfde_pl);
+ err = devlink_fmsg_bool_pair_put(fmsg, "old_event", val);
+ if (err)
+ return err;
+ tile_v = mlxsw_reg_mfde_fw_assert_tile_v_get(mfde_pl);
+ if (tile_v) {
+ val = mlxsw_reg_mfde_fw_assert_tile_index_get(mfde_pl);
+ err = devlink_fmsg_u8_pair_put(fmsg, "tile_index", val);
+ if (err)
+ return err;
+ }
+ val = mlxsw_reg_mfde_fw_assert_ext_synd_get(mfde_pl);
+ err = devlink_fmsg_u32_pair_put(fmsg, "ext_synd", val);
+ if (err)
+ return err;
+
+ return 0;
+}
+
+static int
+mlxsw_core_health_fw_fatal_dump_kvd_im_stop(const char *mfde_pl,
+ struct devlink_fmsg *fmsg)
+{
+ u32 val;
+ int err;
+
+ val = mlxsw_reg_mfde_kvd_im_stop_oe_get(mfde_pl);
+ err = devlink_fmsg_bool_pair_put(fmsg, "old_event", val);
+ if (err)
+ return err;
+ val = mlxsw_reg_mfde_kvd_im_stop_pipes_mask_get(mfde_pl);
+ return devlink_fmsg_u32_pair_put(fmsg, "pipes_mask", val);
+}
+
+static int
+mlxsw_core_health_fw_fatal_dump_crspace_to(const char *mfde_pl,
+ struct devlink_fmsg *fmsg)
+{
+ u32 val;
+ int err;
+
+ val = mlxsw_reg_mfde_crspace_to_log_address_get(mfde_pl);
+ err = devlink_fmsg_u32_pair_put(fmsg, "log_address", val);
+ if (err)
+ return err;
+ val = mlxsw_reg_mfde_crspace_to_oe_get(mfde_pl);
+ err = devlink_fmsg_bool_pair_put(fmsg, "old_event", val);
+ if (err)
+ return err;
+ val = mlxsw_reg_mfde_crspace_to_log_id_get(mfde_pl);
+ err = devlink_fmsg_u8_pair_put(fmsg, "log_irisc_id", val);
+ if (err)
+ return err;
+ val = mlxsw_reg_mfde_crspace_to_log_ip_get(mfde_pl);
+ err = devlink_fmsg_u64_pair_put(fmsg, "log_ip", val);
+ if (err)
+ return err;
+
+ return 0;
+}
+
static int mlxsw_core_health_fw_fatal_dump(struct devlink_health_reporter *reporter,
struct devlink_fmsg *fmsg, void *priv_ctx,
struct netlink_ext_ack *extack)
@@ -1741,6 +1859,15 @@ static int mlxsw_core_health_fw_fatal_dump(struct devlink_health_reporter *repor
case MLXSW_REG_MFDE_EVENT_ID_KVD_IM_STOP:
val_str = "KVD insertion machine stopped";
break;
+ case MLXSW_REG_MFDE_EVENT_ID_TEST:
+ val_str = "Test";
+ break;
+ case MLXSW_REG_MFDE_EVENT_ID_FW_ASSERT:
+ val_str = "FW assert";
+ break;
+ case MLXSW_REG_MFDE_EVENT_ID_FATAL_CAUSE:
+ val_str = "Fatal cause";
+ break;
default:
val_str = NULL;
}
@@ -1749,6 +1876,38 @@ static int mlxsw_core_health_fw_fatal_dump(struct devlink_health_reporter *repor
if (err)
return err;
}
+
+ err = devlink_fmsg_arr_pair_nest_end(fmsg);
+ if (err)
+ return err;
+
+ err = devlink_fmsg_arr_pair_nest_start(fmsg, "severity");
+ if (err)
+ return err;
+
+ val = mlxsw_reg_mfde_severity_get(mfde_pl);
+ err = devlink_fmsg_u8_pair_put(fmsg, "id", val);
+ if (err)
+ return err;
+ switch (val) {
+ case MLXSW_REG_MFDE_SEVERITY_FATL:
+ val_str = "Fatal";
+ break;
+ case MLXSW_REG_MFDE_SEVERITY_NRML:
+ val_str = "Normal";
+ break;
+ case MLXSW_REG_MFDE_SEVERITY_INTR:
+ val_str = "Debug";
+ break;
+ default:
+ val_str = NULL;
+ }
+ if (val_str) {
+ err = devlink_fmsg_string_pair_put(fmsg, "desc", val_str);
+ if (err)
+ return err;
+ }
+
err = devlink_fmsg_arr_pair_nest_end(fmsg);
if (err)
return err;
@@ -1800,24 +1959,18 @@ static int mlxsw_core_health_fw_fatal_dump(struct devlink_health_reporter *repor
if (err)
return err;
- if (event_id == MLXSW_REG_MFDE_EVENT_ID_CRSPACE_TO) {
- val = mlxsw_reg_mfde_log_address_get(mfde_pl);
- err = devlink_fmsg_u32_pair_put(fmsg, "log_address", val);
- if (err)
- return err;
- val = mlxsw_reg_mfde_log_id_get(mfde_pl);
- err = devlink_fmsg_u8_pair_put(fmsg, "log_irisc_id", val);
- if (err)
- return err;
- val = mlxsw_reg_mfde_log_ip_get(mfde_pl);
- err = devlink_fmsg_u64_pair_put(fmsg, "log_ip", val);
- if (err)
- return err;
- } else if (event_id == MLXSW_REG_MFDE_EVENT_ID_KVD_IM_STOP) {
- val = mlxsw_reg_mfde_pipes_mask_get(mfde_pl);
- err = devlink_fmsg_u32_pair_put(fmsg, "pipes_mask", val);
- if (err)
- return err;
+ switch (event_id) {
+ case MLXSW_REG_MFDE_EVENT_ID_CRSPACE_TO:
+ return mlxsw_core_health_fw_fatal_dump_crspace_to(mfde_pl,
+ fmsg);
+ case MLXSW_REG_MFDE_EVENT_ID_KVD_IM_STOP:
+ return mlxsw_core_health_fw_fatal_dump_kvd_im_stop(mfde_pl,
+ fmsg);
+ case MLXSW_REG_MFDE_EVENT_ID_FW_ASSERT:
+ return mlxsw_core_health_fw_fatal_dump_fw_assert(mfde_pl, fmsg);
+ case MLXSW_REG_MFDE_EVENT_ID_FATAL_CAUSE:
+ return mlxsw_core_health_fw_fatal_dump_fatal_cause(mfde_pl,
+ fmsg);
}
return 0;
@@ -1959,7 +2112,7 @@ __mlxsw_core_bus_device_register(const struct mlxsw_bus_info *mlxsw_bus_info,
if (MLXSW_CORE_RES_VALID(mlxsw_core, MAX_LAG) &&
MLXSW_CORE_RES_VALID(mlxsw_core, MAX_LAG_MEMBERS)) {
- alloc_size = sizeof(u8) *
+ alloc_size = sizeof(*mlxsw_core->lag.mapping) *
MLXSW_CORE_RES_GET(mlxsw_core, MAX_LAG) *
MLXSW_CORE_RES_GET(mlxsw_core, MAX_LAG_MEMBERS);
mlxsw_core->lag.mapping = kzalloc(alloc_size, GFP_KERNEL);
@@ -2033,7 +2186,7 @@ __mlxsw_core_bus_device_register(const struct mlxsw_bus_info *mlxsw_bus_info,
mlxsw_ports_fini(mlxsw_core, reload);
err_ports_init:
if (!reload)
- devlink_resources_unregister(devlink, NULL);
+ devlink_resources_unregister(devlink);
err_register_resources:
mlxsw_bus->fini(bus_priv);
err_bus_init:
@@ -2099,7 +2252,7 @@ void mlxsw_core_bus_device_unregister(struct mlxsw_core *mlxsw_core,
kfree(mlxsw_core->lag.mapping);
mlxsw_ports_fini(mlxsw_core, reload);
if (!reload)
- devlink_resources_unregister(devlink, NULL);
+ devlink_resources_unregister(devlink);
mlxsw_core->bus->fini(mlxsw_core->bus_priv);
if (!reload)
devlink_free(devlink);
@@ -2108,7 +2261,7 @@ void mlxsw_core_bus_device_unregister(struct mlxsw_core *mlxsw_core,
reload_fail_deinit:
mlxsw_core_params_unregister(mlxsw_core);
- devlink_resources_unregister(devlink, NULL);
+ devlink_resources_unregister(devlink);
devlink_free(devlink);
}
EXPORT_SYMBOL(mlxsw_core_bus_device_unregister);
@@ -2130,7 +2283,7 @@ int mlxsw_core_skb_transmit(struct mlxsw_core *mlxsw_core, struct sk_buff *skb,
EXPORT_SYMBOL(mlxsw_core_skb_transmit);
void mlxsw_core_ptp_transmitted(struct mlxsw_core *mlxsw_core,
- struct sk_buff *skb, u8 local_port)
+ struct sk_buff *skb, u16 local_port)
{
if (mlxsw_core->driver->ptp_transmitted)
mlxsw_core->driver->ptp_transmitted(mlxsw_core, skb,
@@ -2208,7 +2361,7 @@ mlxsw_core_rx_listener_state_set(struct mlxsw_core *mlxsw_core,
rxl_item->enabled = enabled;
}
-static void mlxsw_core_event_listener_func(struct sk_buff *skb, u8 local_port,
+static void mlxsw_core_event_listener_func(struct sk_buff *skb, u16 local_port,
void *priv)
{
struct mlxsw_event_listener_item *event_listener_item = priv;
@@ -2641,7 +2794,7 @@ void mlxsw_core_skb_receive(struct mlxsw_core *mlxsw_core, struct sk_buff *skb,
{
struct mlxsw_rx_listener_item *rxl_item;
const struct mlxsw_rx_listener *rxl;
- u8 local_port;
+ u16 local_port;
bool found = false;
if (rx_info->is_lag) {
@@ -2699,7 +2852,7 @@ static int mlxsw_core_lag_mapping_index(struct mlxsw_core *mlxsw_core,
}
void mlxsw_core_lag_mapping_set(struct mlxsw_core *mlxsw_core,
- u16 lag_id, u8 port_index, u8 local_port)
+ u16 lag_id, u8 port_index, u16 local_port)
{
int index = mlxsw_core_lag_mapping_index(mlxsw_core,
lag_id, port_index);
@@ -2708,8 +2861,8 @@ void mlxsw_core_lag_mapping_set(struct mlxsw_core *mlxsw_core,
}
EXPORT_SYMBOL(mlxsw_core_lag_mapping_set);
-u8 mlxsw_core_lag_mapping_get(struct mlxsw_core *mlxsw_core,
- u16 lag_id, u8 port_index)
+u16 mlxsw_core_lag_mapping_get(struct mlxsw_core *mlxsw_core,
+ u16 lag_id, u8 port_index)
{
int index = mlxsw_core_lag_mapping_index(mlxsw_core,
lag_id, port_index);
@@ -2719,7 +2872,7 @@ u8 mlxsw_core_lag_mapping_get(struct mlxsw_core *mlxsw_core,
EXPORT_SYMBOL(mlxsw_core_lag_mapping_get);
void mlxsw_core_lag_mapping_clear(struct mlxsw_core *mlxsw_core,
- u16 lag_id, u8 local_port)
+ u16 lag_id, u16 local_port)
{
int i;
@@ -2747,7 +2900,7 @@ u64 mlxsw_core_res_get(struct mlxsw_core *mlxsw_core,
}
EXPORT_SYMBOL(mlxsw_core_res_get);
-static int __mlxsw_core_port_init(struct mlxsw_core *mlxsw_core, u8 local_port,
+static int __mlxsw_core_port_init(struct mlxsw_core *mlxsw_core, u16 local_port,
enum devlink_port_flavour flavour,
u32 port_number, bool split,
u32 split_port_subnumber,
@@ -2778,7 +2931,7 @@ static int __mlxsw_core_port_init(struct mlxsw_core *mlxsw_core, u8 local_port,
return err;
}
-static void __mlxsw_core_port_fini(struct mlxsw_core *mlxsw_core, u8 local_port)
+static void __mlxsw_core_port_fini(struct mlxsw_core *mlxsw_core, u16 local_port)
{
struct mlxsw_core_port *mlxsw_core_port =
&mlxsw_core->ports[local_port];
@@ -2788,7 +2941,7 @@ static void __mlxsw_core_port_fini(struct mlxsw_core *mlxsw_core, u8 local_port)
memset(mlxsw_core_port, 0, sizeof(*mlxsw_core_port));
}
-int mlxsw_core_port_init(struct mlxsw_core *mlxsw_core, u8 local_port,
+int mlxsw_core_port_init(struct mlxsw_core *mlxsw_core, u16 local_port,
u32 port_number, bool split,
u32 split_port_subnumber,
bool splittable, u32 lanes,
@@ -2810,7 +2963,7 @@ int mlxsw_core_port_init(struct mlxsw_core *mlxsw_core, u8 local_port,
}
EXPORT_SYMBOL(mlxsw_core_port_init);
-void mlxsw_core_port_fini(struct mlxsw_core *mlxsw_core, u8 local_port)
+void mlxsw_core_port_fini(struct mlxsw_core *mlxsw_core, u16 local_port)
{
atomic_dec(&mlxsw_core->active_ports_count);
@@ -2845,7 +2998,7 @@ void mlxsw_core_cpu_port_fini(struct mlxsw_core *mlxsw_core)
}
EXPORT_SYMBOL(mlxsw_core_cpu_port_fini);
-void mlxsw_core_port_eth_set(struct mlxsw_core *mlxsw_core, u8 local_port,
+void mlxsw_core_port_eth_set(struct mlxsw_core *mlxsw_core, u16 local_port,
void *port_driver_priv, struct net_device *dev)
{
struct mlxsw_core_port *mlxsw_core_port =
@@ -2857,7 +3010,7 @@ void mlxsw_core_port_eth_set(struct mlxsw_core *mlxsw_core, u8 local_port,
}
EXPORT_SYMBOL(mlxsw_core_port_eth_set);
-void mlxsw_core_port_ib_set(struct mlxsw_core *mlxsw_core, u8 local_port,
+void mlxsw_core_port_ib_set(struct mlxsw_core *mlxsw_core, u16 local_port,
void *port_driver_priv)
{
struct mlxsw_core_port *mlxsw_core_port =
@@ -2869,7 +3022,7 @@ void mlxsw_core_port_ib_set(struct mlxsw_core *mlxsw_core, u8 local_port,
}
EXPORT_SYMBOL(mlxsw_core_port_ib_set);
-void mlxsw_core_port_clear(struct mlxsw_core *mlxsw_core, u8 local_port,
+void mlxsw_core_port_clear(struct mlxsw_core *mlxsw_core, u16 local_port,
void *port_driver_priv)
{
struct mlxsw_core_port *mlxsw_core_port =
@@ -2882,7 +3035,7 @@ void mlxsw_core_port_clear(struct mlxsw_core *mlxsw_core, u8 local_port,
EXPORT_SYMBOL(mlxsw_core_port_clear);
enum devlink_port_type mlxsw_core_port_type_get(struct mlxsw_core *mlxsw_core,
- u8 local_port)
+ u16 local_port)
{
struct mlxsw_core_port *mlxsw_core_port =
&mlxsw_core->ports[local_port];
@@ -2895,7 +3048,7 @@ EXPORT_SYMBOL(mlxsw_core_port_type_get);
struct devlink_port *
mlxsw_core_port_devlink_port_get(struct mlxsw_core *mlxsw_core,
- u8 local_port)
+ u16 local_port)
{
struct mlxsw_core_port *mlxsw_core_port =
&mlxsw_core->ports[local_port];
@@ -2905,7 +3058,7 @@ mlxsw_core_port_devlink_port_get(struct mlxsw_core *mlxsw_core,
}
EXPORT_SYMBOL(mlxsw_core_port_devlink_port_get);
-bool mlxsw_core_port_is_xm(const struct mlxsw_core *mlxsw_core, u8 local_port)
+bool mlxsw_core_port_is_xm(const struct mlxsw_core *mlxsw_core, u16 local_port)
{
const struct mlxsw_bus_info *bus_info = mlxsw_core->bus_info;
int i;
diff --git a/drivers/net/ethernet/mellanox/mlxsw/core.h b/drivers/net/ethernet/mellanox/mlxsw/core.h
index 12023a5..f30bb86 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/core.h
+++ b/drivers/net/ethernet/mellanox/mlxsw/core.h
@@ -54,7 +54,7 @@ int mlxsw_core_bus_device_register(const struct mlxsw_bus_info *mlxsw_bus_info,
void mlxsw_core_bus_device_unregister(struct mlxsw_core *mlxsw_core, bool reload);
struct mlxsw_tx_info {
- u8 local_port;
+ u16 local_port;
bool is_emad;
};
@@ -67,7 +67,7 @@ struct mlxsw_rx_md_info {
u16 tx_sys_port;
u16 tx_lag_id;
};
- u8 tx_lag_port_index; /* Valid when 'tx_port_is_lag' is set. */
+ u16 tx_lag_port_index; /* Valid when 'tx_port_is_lag' is set. */
u8 tx_tc;
u8 latency_valid:1,
tx_congestion_valid:1,
@@ -82,11 +82,11 @@ bool mlxsw_core_skb_transmit_busy(struct mlxsw_core *mlxsw_core,
int mlxsw_core_skb_transmit(struct mlxsw_core *mlxsw_core, struct sk_buff *skb,
const struct mlxsw_tx_info *tx_info);
void mlxsw_core_ptp_transmitted(struct mlxsw_core *mlxsw_core,
- struct sk_buff *skb, u8 local_port);
+ struct sk_buff *skb, u16 local_port);
struct mlxsw_rx_listener {
- void (*func)(struct sk_buff *skb, u8 local_port, void *priv);
- u8 local_port;
+ void (*func)(struct sk_buff *skb, u16 local_port, void *priv);
+ u16 local_port;
u8 mirror_reason;
u16 trap_id;
};
@@ -209,7 +209,7 @@ struct mlxsw_rx_info {
u16 sys_port;
u16 lag_id;
} u;
- u8 lag_port_index;
+ u16 lag_port_index;
u8 mirror_reason;
int trap_id;
};
@@ -218,36 +218,36 @@ void mlxsw_core_skb_receive(struct mlxsw_core *mlxsw_core, struct sk_buff *skb,
struct mlxsw_rx_info *rx_info);
void mlxsw_core_lag_mapping_set(struct mlxsw_core *mlxsw_core,
- u16 lag_id, u8 port_index, u8 local_port);
-u8 mlxsw_core_lag_mapping_get(struct mlxsw_core *mlxsw_core,
- u16 lag_id, u8 port_index);
+ u16 lag_id, u8 port_index, u16 local_port);
+u16 mlxsw_core_lag_mapping_get(struct mlxsw_core *mlxsw_core,
+ u16 lag_id, u8 port_index);
void mlxsw_core_lag_mapping_clear(struct mlxsw_core *mlxsw_core,
- u16 lag_id, u8 local_port);
+ u16 lag_id, u16 local_port);
void *mlxsw_core_port_driver_priv(struct mlxsw_core_port *mlxsw_core_port);
-int mlxsw_core_port_init(struct mlxsw_core *mlxsw_core, u8 local_port,
+int mlxsw_core_port_init(struct mlxsw_core *mlxsw_core, u16 local_port,
u32 port_number, bool split, u32 split_port_subnumber,
bool splittable, u32 lanes,
const unsigned char *switch_id,
unsigned char switch_id_len);
-void mlxsw_core_port_fini(struct mlxsw_core *mlxsw_core, u8 local_port);
+void mlxsw_core_port_fini(struct mlxsw_core *mlxsw_core, u16 local_port);
int mlxsw_core_cpu_port_init(struct mlxsw_core *mlxsw_core,
void *port_driver_priv,
const unsigned char *switch_id,
unsigned char switch_id_len);
void mlxsw_core_cpu_port_fini(struct mlxsw_core *mlxsw_core);
-void mlxsw_core_port_eth_set(struct mlxsw_core *mlxsw_core, u8 local_port,
+void mlxsw_core_port_eth_set(struct mlxsw_core *mlxsw_core, u16 local_port,
void *port_driver_priv, struct net_device *dev);
-void mlxsw_core_port_ib_set(struct mlxsw_core *mlxsw_core, u8 local_port,
+void mlxsw_core_port_ib_set(struct mlxsw_core *mlxsw_core, u16 local_port,
void *port_driver_priv);
-void mlxsw_core_port_clear(struct mlxsw_core *mlxsw_core, u8 local_port,
+void mlxsw_core_port_clear(struct mlxsw_core *mlxsw_core, u16 local_port,
void *port_driver_priv);
enum devlink_port_type mlxsw_core_port_type_get(struct mlxsw_core *mlxsw_core,
- u8 local_port);
+ u16 local_port);
struct devlink_port *
mlxsw_core_port_devlink_port_get(struct mlxsw_core *mlxsw_core,
- u8 local_port);
-bool mlxsw_core_port_is_xm(const struct mlxsw_core *mlxsw_core, u8 local_port);
+ u16 local_port);
+bool mlxsw_core_port_is_xm(const struct mlxsw_core *mlxsw_core, u16 local_port);
struct mlxsw_env *mlxsw_core_env(const struct mlxsw_core *mlxsw_core);
int mlxsw_core_schedule_dw(struct delayed_work *dwork, unsigned long delay);
@@ -316,11 +316,11 @@ struct mlxsw_driver {
struct netlink_ext_ack *extack);
void (*fini)(struct mlxsw_core *mlxsw_core);
int (*basic_trap_groups_set)(struct mlxsw_core *mlxsw_core);
- int (*port_type_set)(struct mlxsw_core *mlxsw_core, u8 local_port,
+ int (*port_type_set)(struct mlxsw_core *mlxsw_core, u16 local_port,
enum devlink_port_type new_type);
- int (*port_split)(struct mlxsw_core *mlxsw_core, u8 local_port,
+ int (*port_split)(struct mlxsw_core *mlxsw_core, u16 local_port,
unsigned int count, struct netlink_ext_ack *extack);
- int (*port_unsplit)(struct mlxsw_core *mlxsw_core, u8 local_port,
+ int (*port_unsplit)(struct mlxsw_core *mlxsw_core, u16 local_port,
struct netlink_ext_ack *extack);
int (*sb_pool_get)(struct mlxsw_core *mlxsw_core,
unsigned int sb_index, u16 pool_index,
@@ -394,7 +394,7 @@ struct mlxsw_driver {
* is responsible for freeing the passed-in SKB.
*/
void (*ptp_transmitted)(struct mlxsw_core *mlxsw_core,
- struct sk_buff *skb, u8 local_port);
+ struct sk_buff *skb, u16 local_port);
u8 txhdr_len;
const struct mlxsw_config_profile *profile;
diff --git a/drivers/net/ethernet/mellanox/mlxsw/core_acl_flex_actions.c b/drivers/net/ethernet/mellanox/mlxsw/core_acl_flex_actions.c
index 78d9c01..77e82e6 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/core_acl_flex_actions.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/core_acl_flex_actions.c
@@ -113,7 +113,7 @@ static const struct rhashtable_params mlxsw_afa_set_ht_params = {
};
struct mlxsw_afa_fwd_entry_ht_key {
- u8 local_port;
+ u16 local_port;
};
struct mlxsw_afa_fwd_entry {
@@ -555,7 +555,7 @@ int mlxsw_afa_block_terminate(struct mlxsw_afa_block *block)
EXPORT_SYMBOL(mlxsw_afa_block_terminate);
static struct mlxsw_afa_fwd_entry *
-mlxsw_afa_fwd_entry_create(struct mlxsw_afa *mlxsw_afa, u8 local_port)
+mlxsw_afa_fwd_entry_create(struct mlxsw_afa *mlxsw_afa, u16 local_port)
{
struct mlxsw_afa_fwd_entry *fwd_entry;
int err;
@@ -598,7 +598,7 @@ static void mlxsw_afa_fwd_entry_destroy(struct mlxsw_afa *mlxsw_afa,
}
static struct mlxsw_afa_fwd_entry *
-mlxsw_afa_fwd_entry_get(struct mlxsw_afa *mlxsw_afa, u8 local_port)
+mlxsw_afa_fwd_entry_get(struct mlxsw_afa *mlxsw_afa, u16 local_port)
{
struct mlxsw_afa_fwd_entry_ht_key ht_key = {0};
struct mlxsw_afa_fwd_entry *fwd_entry;
@@ -647,7 +647,7 @@ mlxsw_afa_fwd_entry_ref_destructor(struct mlxsw_afa_block *block,
}
static struct mlxsw_afa_fwd_entry_ref *
-mlxsw_afa_fwd_entry_ref_create(struct mlxsw_afa_block *block, u8 local_port)
+mlxsw_afa_fwd_entry_ref_create(struct mlxsw_afa_block *block, u16 local_port)
{
struct mlxsw_afa_fwd_entry_ref *fwd_entry_ref;
struct mlxsw_afa_fwd_entry *fwd_entry;
@@ -1352,7 +1352,7 @@ EXPORT_SYMBOL(mlxsw_afa_block_append_trap_and_forward);
struct mlxsw_afa_mirror {
struct mlxsw_afa_resource resource;
int span_id;
- u8 local_in_port;
+ u16 local_in_port;
bool ingress;
};
@@ -1379,7 +1379,7 @@ mlxsw_afa_mirror_destructor(struct mlxsw_afa_block *block,
}
static struct mlxsw_afa_mirror *
-mlxsw_afa_mirror_create(struct mlxsw_afa_block *block, u8 local_in_port,
+mlxsw_afa_mirror_create(struct mlxsw_afa_block *block, u16 local_in_port,
const struct net_device *out_dev, bool ingress)
{
struct mlxsw_afa_mirror *mirror;
@@ -1423,7 +1423,7 @@ mlxsw_afa_block_append_allocated_mirror(struct mlxsw_afa_block *block,
}
int
-mlxsw_afa_block_append_mirror(struct mlxsw_afa_block *block, u8 local_in_port,
+mlxsw_afa_block_append_mirror(struct mlxsw_afa_block *block, u16 local_in_port,
const struct net_device *out_dev, bool ingress,
struct netlink_ext_ack *extack)
{
@@ -1663,7 +1663,7 @@ mlxsw_afa_forward_pack(char *payload, enum mlxsw_afa_forward_type type,
}
int mlxsw_afa_block_append_fwd(struct mlxsw_afa_block *block,
- u8 local_port, bool in_port,
+ u16 local_port, bool in_port,
struct netlink_ext_ack *extack)
{
struct mlxsw_afa_fwd_entry_ref *fwd_entry_ref;
@@ -2038,7 +2038,7 @@ static void mlxsw_afa_sampler_pack(char *payload, u8 mirror_agent, u32 rate)
struct mlxsw_afa_sampler {
struct mlxsw_afa_resource resource;
int span_id;
- u8 local_port;
+ u16 local_port;
bool ingress;
};
@@ -2061,7 +2061,7 @@ static void mlxsw_afa_sampler_destructor(struct mlxsw_afa_block *block,
}
static struct mlxsw_afa_sampler *
-mlxsw_afa_sampler_create(struct mlxsw_afa_block *block, u8 local_port,
+mlxsw_afa_sampler_create(struct mlxsw_afa_block *block, u16 local_port,
struct psample_group *psample_group, u32 rate,
u32 trunc_size, bool truncate, bool ingress,
struct netlink_ext_ack *extack)
@@ -2104,7 +2104,7 @@ mlxsw_afa_block_append_allocated_sampler(struct mlxsw_afa_block *block,
return 0;
}
-int mlxsw_afa_block_append_sampler(struct mlxsw_afa_block *block, u8 local_port,
+int mlxsw_afa_block_append_sampler(struct mlxsw_afa_block *block, u16 local_port,
struct psample_group *psample_group,
u32 rate, u32 trunc_size, bool truncate,
bool ingress,
diff --git a/drivers/net/ethernet/mellanox/mlxsw/core_acl_flex_actions.h b/drivers/net/ethernet/mellanox/mlxsw/core_acl_flex_actions.h
index b65bf98..16cbd6a 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/core_acl_flex_actions.h
+++ b/drivers/net/ethernet/mellanox/mlxsw/core_acl_flex_actions.h
@@ -17,24 +17,24 @@ struct mlxsw_afa_ops {
void (*kvdl_set_del)(void *priv, u32 kvdl_index, bool is_first);
int (*kvdl_set_activity_get)(void *priv, u32 kvdl_index,
bool *activity);
- int (*kvdl_fwd_entry_add)(void *priv, u32 *p_kvdl_index, u8 local_port);
+ int (*kvdl_fwd_entry_add)(void *priv, u32 *p_kvdl_index, u16 local_port);
void (*kvdl_fwd_entry_del)(void *priv, u32 kvdl_index);
int (*counter_index_get)(void *priv, unsigned int *p_counter_index);
void (*counter_index_put)(void *priv, unsigned int counter_index);
- int (*mirror_add)(void *priv, u8 local_in_port,
+ int (*mirror_add)(void *priv, u16 local_in_port,
const struct net_device *out_dev,
bool ingress, int *p_span_id);
- void (*mirror_del)(void *priv, u8 local_in_port, int span_id,
+ void (*mirror_del)(void *priv, u16 local_in_port, int span_id,
bool ingress);
int (*policer_add)(void *priv, u64 rate_bytes_ps, u32 burst,
u16 *p_policer_index,
struct netlink_ext_ack *extack);
void (*policer_del)(void *priv, u16 policer_index);
- int (*sampler_add)(void *priv, u8 local_port,
+ int (*sampler_add)(void *priv, u16 local_port,
struct psample_group *psample_group, u32 rate,
u32 trunc_size, bool truncate, bool ingress,
int *p_span_id, struct netlink_ext_ack *extack);
- void (*sampler_del)(void *priv, u8 local_port, int span_id,
+ void (*sampler_del)(void *priv, u16 local_port, int span_id,
bool ingress);
bool dummy_first_set;
};
@@ -62,12 +62,12 @@ int mlxsw_afa_block_append_trap(struct mlxsw_afa_block *block, u16 trap_id);
int mlxsw_afa_block_append_trap_and_forward(struct mlxsw_afa_block *block,
u16 trap_id);
int mlxsw_afa_block_append_mirror(struct mlxsw_afa_block *block,
- u8 local_in_port,
+ u16 local_in_port,
const struct net_device *out_dev,
bool ingress,
struct netlink_ext_ack *extack);
int mlxsw_afa_block_append_fwd(struct mlxsw_afa_block *block,
- u8 local_port, bool in_port,
+ u16 local_port, bool in_port,
struct netlink_ext_ack *extack);
int mlxsw_afa_block_append_vlan_modify(struct mlxsw_afa_block *block,
u16 vid, u8 pcp, u8 et,
@@ -98,7 +98,7 @@ int mlxsw_afa_block_append_police(struct mlxsw_afa_block *block,
u32 fa_index, u64 rate_bytes_ps, u32 burst,
u16 *p_policer_index,
struct netlink_ext_ack *extack);
-int mlxsw_afa_block_append_sampler(struct mlxsw_afa_block *block, u8 local_port,
+int mlxsw_afa_block_append_sampler(struct mlxsw_afa_block *block, u16 local_port,
struct psample_group *psample_group,
u32 rate, u32 trunc_size, bool truncate,
bool ingress,
diff --git a/drivers/net/ethernet/mellanox/mlxsw/core_acl_flex_keys.c b/drivers/net/ethernet/mellanox/mlxsw/core_acl_flex_keys.c
index f1b09c2..bd1a51a 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/core_acl_flex_keys.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/core_acl_flex_keys.c
@@ -32,8 +32,8 @@ static const struct mlxsw_afk_element_info mlxsw_afk_element_infos[] = {
MLXSW_AFK_ELEMENT_INFO_U32(IP_TTL_, 0x18, 0, 8),
MLXSW_AFK_ELEMENT_INFO_U32(IP_ECN, 0x18, 9, 2),
MLXSW_AFK_ELEMENT_INFO_U32(IP_DSCP, 0x18, 11, 6),
- MLXSW_AFK_ELEMENT_INFO_U32(VIRT_ROUTER_8_10, 0x18, 17, 3),
- MLXSW_AFK_ELEMENT_INFO_U32(VIRT_ROUTER_0_7, 0x18, 20, 8),
+ MLXSW_AFK_ELEMENT_INFO_U32(VIRT_ROUTER_MSB, 0x18, 17, 3),
+ MLXSW_AFK_ELEMENT_INFO_U32(VIRT_ROUTER_LSB, 0x18, 20, 8),
MLXSW_AFK_ELEMENT_INFO_BUF(SRC_IP_96_127, 0x20, 4),
MLXSW_AFK_ELEMENT_INFO_BUF(SRC_IP_64_95, 0x24, 4),
MLXSW_AFK_ELEMENT_INFO_BUF(SRC_IP_32_63, 0x28, 4),
diff --git a/drivers/net/ethernet/mellanox/mlxsw/core_acl_flex_keys.h b/drivers/net/ethernet/mellanox/mlxsw/core_acl_flex_keys.h
index a47a17c..3a037fe 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/core_acl_flex_keys.h
+++ b/drivers/net/ethernet/mellanox/mlxsw/core_acl_flex_keys.h
@@ -33,8 +33,8 @@ enum mlxsw_afk_element {
MLXSW_AFK_ELEMENT_IP_TTL_,
MLXSW_AFK_ELEMENT_IP_ECN,
MLXSW_AFK_ELEMENT_IP_DSCP,
- MLXSW_AFK_ELEMENT_VIRT_ROUTER_8_10,
- MLXSW_AFK_ELEMENT_VIRT_ROUTER_0_7,
+ MLXSW_AFK_ELEMENT_VIRT_ROUTER_MSB,
+ MLXSW_AFK_ELEMENT_VIRT_ROUTER_LSB,
MLXSW_AFK_ELEMENT_MAX,
};
diff --git a/drivers/net/ethernet/mellanox/mlxsw/item.h b/drivers/net/ethernet/mellanox/mlxsw/item.h
index ab70a87..cfafbeb 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/item.h
+++ b/drivers/net/ethernet/mellanox/mlxsw/item.h
@@ -367,6 +367,42 @@ mlxsw_##_type##_##_cname##_##_iname##_set(char *buf, u32 val) \
__mlxsw_item_set32(buf, &__ITEM_NAME(_type, _cname, _iname), 0, val); \
}
+#define LOCAL_PORT_LSB_SIZE 8
+#define LOCAL_PORT_MSB_SIZE 2
+
+#define MLXSW_ITEM32_LP(_type, _cname, _offset1, _shift1, _offset2, _shift2) \
+static struct mlxsw_item __ITEM_NAME(_type, _cname, local_port) = { \
+ .offset = _offset1, \
+ .shift = _shift1, \
+ .size = {.bits = LOCAL_PORT_LSB_SIZE,}, \
+ .name = #_type "_" #_cname "_local_port", \
+}; \
+static struct mlxsw_item __ITEM_NAME(_type, _cname, lp_msb) = { \
+ .offset = _offset2, \
+ .shift = _shift2, \
+ .size = {.bits = LOCAL_PORT_MSB_SIZE,}, \
+ .name = #_type "_" #_cname "_lp_msb", \
+}; \
+static inline u32 __maybe_unused \
+mlxsw_##_type##_##_cname##_local_port_get(const char *buf) \
+{ \
+ u32 local_port, lp_msb; \
+ \
+ local_port = __mlxsw_item_get32(buf, &__ITEM_NAME(_type, _cname, \
+ local_port), 0); \
+ lp_msb = __mlxsw_item_get32(buf, &__ITEM_NAME(_type, _cname, lp_msb), \
+ 0); \
+ return (lp_msb << LOCAL_PORT_LSB_SIZE) + local_port; \
+} \
+static inline void __maybe_unused \
+mlxsw_##_type##_##_cname##_local_port_set(char *buf, u32 val) \
+{ \
+ __mlxsw_item_set32(buf, &__ITEM_NAME(_type, _cname, local_port), 0, \
+ val & ((1 << LOCAL_PORT_LSB_SIZE) - 1)); \
+ __mlxsw_item_set32(buf, &__ITEM_NAME(_type, _cname, lp_msb), 0, \
+ val >> LOCAL_PORT_LSB_SIZE); \
+}
+
#define MLXSW_ITEM32_INDEXED(_type, _cname, _iname, _offset, _shift, _sizebits, \
_step, _instepoffset, _norealshift) \
static struct mlxsw_item __ITEM_NAME(_type, _cname, _iname) = { \
diff --git a/drivers/net/ethernet/mellanox/mlxsw/minimal.c b/drivers/net/ethernet/mellanox/mlxsw/minimal.c
index 5d4dfa5..10d13f5 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/minimal.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/minimal.c
@@ -38,7 +38,7 @@ struct mlxsw_m {
struct mlxsw_m_port {
struct net_device *dev;
struct mlxsw_m *mlxsw_m;
- u8 local_port;
+ u16 local_port;
u8 module;
};
@@ -180,7 +180,7 @@ static const struct ethtool_ops mlxsw_m_port_ethtool_ops = {
};
static int
-mlxsw_m_port_module_info_get(struct mlxsw_m *mlxsw_m, u8 local_port,
+mlxsw_m_port_module_info_get(struct mlxsw_m *mlxsw_m, u16 local_port,
u8 *p_module, u8 *p_width)
{
char pmlp_pl[MLXSW_REG_PMLP_LEN];
@@ -214,7 +214,7 @@ mlxsw_m_port_dev_addr_get(struct mlxsw_m_port *mlxsw_m_port)
}
static int
-mlxsw_m_port_create(struct mlxsw_m *mlxsw_m, u8 local_port, u8 module)
+mlxsw_m_port_create(struct mlxsw_m *mlxsw_m, u16 local_port, u8 module)
{
struct mlxsw_m_port *mlxsw_m_port;
struct net_device *dev;
@@ -277,7 +277,7 @@ mlxsw_m_port_create(struct mlxsw_m *mlxsw_m, u8 local_port, u8 module)
return err;
}
-static void mlxsw_m_port_remove(struct mlxsw_m *mlxsw_m, u8 local_port)
+static void mlxsw_m_port_remove(struct mlxsw_m *mlxsw_m, u16 local_port)
{
struct mlxsw_m_port *mlxsw_m_port = mlxsw_m->ports[local_port];
@@ -288,7 +288,7 @@ static void mlxsw_m_port_remove(struct mlxsw_m *mlxsw_m, u8 local_port)
mlxsw_core_port_fini(mlxsw_m->core, local_port);
}
-static int mlxsw_m_port_module_map(struct mlxsw_m *mlxsw_m, u8 local_port,
+static int mlxsw_m_port_module_map(struct mlxsw_m *mlxsw_m, u16 local_port,
u8 *last_module)
{
unsigned int max_ports = mlxsw_core_max_ports(mlxsw_m->core);
diff --git a/drivers/net/ethernet/mellanox/mlxsw/pci.c b/drivers/net/ethernet/mellanox/mlxsw/pci.c
index a15c95a..f91dde4 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/pci.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/pci.c
@@ -285,6 +285,7 @@ static int mlxsw_pci_sdq_init(struct mlxsw_pci *mlxsw_pci, char *mbox,
struct mlxsw_pci_queue *q)
{
int tclass;
+ int lp;
int i;
int err;
@@ -292,9 +293,12 @@ static int mlxsw_pci_sdq_init(struct mlxsw_pci *mlxsw_pci, char *mbox,
q->consumer_counter = 0;
tclass = q->num == MLXSW_PCI_SDQ_EMAD_INDEX ? MLXSW_PCI_SDQ_EMAD_TC :
MLXSW_PCI_SDQ_CTL_TC;
+ lp = q->num == MLXSW_PCI_SDQ_EMAD_INDEX ? MLXSW_CMD_MBOX_SW2HW_DQ_SDQ_LP_IGNORE_WQE :
+ MLXSW_CMD_MBOX_SW2HW_DQ_SDQ_LP_WQE;
/* Set CQ of same number of this SDQ. */
mlxsw_cmd_mbox_sw2hw_dq_cq_set(mbox, q->num);
+ mlxsw_cmd_mbox_sw2hw_dq_sdq_lp_set(mbox, lp);
mlxsw_cmd_mbox_sw2hw_dq_sdq_tclass_set(mbox, tclass);
mlxsw_cmd_mbox_sw2hw_dq_log2_dq_sz_set(mbox, 3); /* 8 pages */
for (i = 0; i < MLXSW_PCI_AQ_PAGES; i++) {
@@ -1678,7 +1682,7 @@ static int mlxsw_pci_skb_transmit(void *bus_priv, struct sk_buff *skb,
wqe = elem_info->elem;
mlxsw_pci_wqe_c_set(wqe, 1); /* always report completion */
- mlxsw_pci_wqe_lp_set(wqe, !!tx_info->is_emad);
+ mlxsw_pci_wqe_lp_set(wqe, 0);
mlxsw_pci_wqe_type_set(wqe, MLXSW_PCI_WQE_TYPE_ETHERNET);
err = mlxsw_pci_wqe_frag_map(mlxsw_pci, wqe, 0, skb->data,
@@ -1973,6 +1977,7 @@ int mlxsw_pci_driver_register(struct pci_driver *pci_driver)
{
pci_driver->probe = mlxsw_pci_probe;
pci_driver->remove = mlxsw_pci_remove;
+ pci_driver->shutdown = mlxsw_pci_remove;
return pci_register_driver(pci_driver);
}
EXPORT_SYMBOL(mlxsw_pci_driver_register);
diff --git a/drivers/net/ethernet/mellanox/mlxsw/pci.h b/drivers/net/ethernet/mellanox/mlxsw/pci.h
index 9899c1a..cacc2f9 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/pci.h
+++ b/drivers/net/ethernet/mellanox/mlxsw/pci.h
@@ -9,6 +9,7 @@
#define PCI_DEVICE_ID_MELLANOX_SPECTRUM 0xcb84
#define PCI_DEVICE_ID_MELLANOX_SPECTRUM2 0xcf6c
#define PCI_DEVICE_ID_MELLANOX_SPECTRUM3 0xcf70
+#define PCI_DEVICE_ID_MELLANOX_SPECTRUM4 0xcf80
#if IS_ENABLED(CONFIG_MLXSW_PCI)
diff --git a/drivers/net/ethernet/mellanox/mlxsw/reg.h b/drivers/net/ethernet/mellanox/mlxsw/reg.h
index 8d420eb..24cc650 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/reg.h
+++ b/drivers/net/ethernet/mellanox/mlxsw/reg.h
@@ -69,52 +69,6 @@ MLXSW_REG_DEFINE(spad, MLXSW_REG_SPAD_ID, MLXSW_REG_SPAD_LEN);
*/
MLXSW_ITEM_BUF(reg, spad, base_mac, 0x02, 6);
-/* SMID - Switch Multicast ID
- * --------------------------
- * The MID record maps from a MID (Multicast ID), which is a unique identifier
- * of the multicast group within the stacking domain, into a list of local
- * ports into which the packet is replicated.
- */
-#define MLXSW_REG_SMID_ID 0x2007
-#define MLXSW_REG_SMID_LEN 0x240
-
-MLXSW_REG_DEFINE(smid, MLXSW_REG_SMID_ID, MLXSW_REG_SMID_LEN);
-
-/* reg_smid_swid
- * Switch partition ID.
- * Access: Index
- */
-MLXSW_ITEM32(reg, smid, swid, 0x00, 24, 8);
-
-/* reg_smid_mid
- * Multicast identifier - global identifier that represents the multicast group
- * across all devices.
- * Access: Index
- */
-MLXSW_ITEM32(reg, smid, mid, 0x00, 0, 16);
-
-/* reg_smid_port
- * Local port memebership (1 bit per port).
- * Access: RW
- */
-MLXSW_ITEM_BIT_ARRAY(reg, smid, port, 0x20, 0x20, 1);
-
-/* reg_smid_port_mask
- * Local port mask (1 bit per port).
- * Access: W
- */
-MLXSW_ITEM_BIT_ARRAY(reg, smid, port_mask, 0x220, 0x20, 1);
-
-static inline void mlxsw_reg_smid_pack(char *payload, u16 mid,
- u8 port, bool set)
-{
- MLXSW_REG_ZERO(smid, payload);
- mlxsw_reg_smid_swid_set(payload, 0);
- mlxsw_reg_smid_mid_set(payload, mid);
- mlxsw_reg_smid_port_set(payload, port, set);
- mlxsw_reg_smid_port_mask_set(payload, port, 1);
-}
-
/* SSPR - Switch System Port Record Register
* -----------------------------------------
* Configures the system port to local port mapping.
@@ -141,7 +95,7 @@ MLXSW_ITEM32(reg, sspr, m, 0x00, 31, 1);
*
* Access: RW
*/
-MLXSW_ITEM32(reg, sspr, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, sspr, 0x00, 16, 0x00, 12);
/* reg_sspr_sub_port
* Virtual port within the physical port.
@@ -161,7 +115,7 @@ MLXSW_ITEM32(reg, sspr, sub_port, 0x00, 8, 8);
*/
MLXSW_ITEM32(reg, sspr, system_port, 0x04, 0, 16);
-static inline void mlxsw_reg_sspr_pack(char *payload, u8 local_port)
+static inline void mlxsw_reg_sspr_pack(char *payload, u16 local_port)
{
MLXSW_REG_ZERO(sspr, payload);
mlxsw_reg_sspr_m_set(payload, 1);
@@ -407,7 +361,7 @@ static inline void mlxsw_reg_sfd_uc_pack(char *payload, int rec_index,
enum mlxsw_reg_sfd_rec_policy policy,
const char *mac, u16 fid_vid,
enum mlxsw_reg_sfd_rec_action action,
- u8 local_port)
+ u16 local_port)
{
mlxsw_reg_sfd_rec_pack(payload, rec_index,
MLXSW_REG_SFD_REC_TYPE_UNICAST, mac, action);
@@ -417,15 +371,6 @@ static inline void mlxsw_reg_sfd_uc_pack(char *payload, int rec_index,
mlxsw_reg_sfd_uc_system_port_set(payload, rec_index, local_port);
}
-static inline void mlxsw_reg_sfd_uc_unpack(char *payload, int rec_index,
- char *mac, u16 *p_fid_vid,
- u8 *p_local_port)
-{
- mlxsw_reg_sfd_rec_mac_memcpy_from(payload, rec_index, mac);
- *p_fid_vid = mlxsw_reg_sfd_uc_fid_vid_get(payload, rec_index);
- *p_local_port = mlxsw_reg_sfd_uc_system_port_get(payload, rec_index);
-}
-
/* reg_sfd_uc_lag_sub_port
* LAG sub port.
* Must be 0 if multichannel VEPA is not enabled.
@@ -478,15 +423,6 @@ mlxsw_reg_sfd_uc_lag_pack(char *payload, int rec_index,
mlxsw_reg_sfd_uc_lag_lag_id_set(payload, rec_index, lag_id);
}
-static inline void mlxsw_reg_sfd_uc_lag_unpack(char *payload, int rec_index,
- char *mac, u16 *p_vid,
- u16 *p_lag_id)
-{
- mlxsw_reg_sfd_rec_mac_memcpy_from(payload, rec_index, mac);
- *p_vid = mlxsw_reg_sfd_uc_lag_fid_vid_get(payload, rec_index);
- *p_lag_id = mlxsw_reg_sfd_uc_lag_lag_id_get(payload, rec_index);
-}
-
/* reg_sfd_mc_pgi
*
* Multicast port group index - index into the port group table.
@@ -568,19 +504,43 @@ static inline void
mlxsw_reg_sfd_uc_tunnel_pack(char *payload, int rec_index,
enum mlxsw_reg_sfd_rec_policy policy,
const char *mac, u16 fid,
- enum mlxsw_reg_sfd_rec_action action, u32 uip,
+ enum mlxsw_reg_sfd_rec_action action,
enum mlxsw_reg_sfd_uc_tunnel_protocol proto)
{
mlxsw_reg_sfd_rec_pack(payload, rec_index,
MLXSW_REG_SFD_REC_TYPE_UNICAST_TUNNEL, mac,
action);
mlxsw_reg_sfd_rec_policy_set(payload, rec_index, policy);
- mlxsw_reg_sfd_uc_tunnel_uip_msb_set(payload, rec_index, uip >> 24);
- mlxsw_reg_sfd_uc_tunnel_uip_lsb_set(payload, rec_index, uip);
mlxsw_reg_sfd_uc_tunnel_fid_set(payload, rec_index, fid);
mlxsw_reg_sfd_uc_tunnel_protocol_set(payload, rec_index, proto);
}
+static inline void
+mlxsw_reg_sfd_uc_tunnel_pack4(char *payload, int rec_index,
+ enum mlxsw_reg_sfd_rec_policy policy,
+ const char *mac, u16 fid,
+ enum mlxsw_reg_sfd_rec_action action, u32 uip)
+{
+ mlxsw_reg_sfd_uc_tunnel_uip_msb_set(payload, rec_index, uip >> 24);
+ mlxsw_reg_sfd_uc_tunnel_uip_lsb_set(payload, rec_index, uip);
+ mlxsw_reg_sfd_uc_tunnel_pack(payload, rec_index, policy, mac, fid,
+ action,
+ MLXSW_REG_SFD_UC_TUNNEL_PROTOCOL_IPV4);
+}
+
+static inline void
+mlxsw_reg_sfd_uc_tunnel_pack6(char *payload, int rec_index, const char *mac,
+ u16 fid, enum mlxsw_reg_sfd_rec_action action,
+ u32 uip_ptr)
+{
+ mlxsw_reg_sfd_uc_tunnel_uip_lsb_set(payload, rec_index, uip_ptr);
+ /* Only static policy is supported for IPv6 unicast tunnel entry. */
+ mlxsw_reg_sfd_uc_tunnel_pack(payload, rec_index,
+ MLXSW_REG_SFD_REC_POLICY_STATIC_ENTRY,
+ mac, fid, action,
+ MLXSW_REG_SFD_UC_TUNNEL_PROTOCOL_IPV6);
+}
+
enum mlxsw_reg_tunnel_port {
MLXSW_REG_TUNNEL_PORT_NVE,
MLXSW_REG_TUNNEL_PORT_VPLS,
@@ -692,7 +652,7 @@ MLXSW_ITEM32_INDEXED(reg, sfn, mac_system_port, MLXSW_REG_SFN_BASE_LEN, 0, 16,
static inline void mlxsw_reg_sfn_mac_unpack(char *payload, int rec_index,
char *mac, u16 *p_vid,
- u8 *p_local_port)
+ u16 *p_local_port)
{
mlxsw_reg_sfn_rec_mac_memcpy_from(payload, rec_index, mac);
*p_vid = mlxsw_reg_sfn_mac_fid_get(payload, rec_index);
@@ -781,7 +741,7 @@ MLXSW_REG_DEFINE(spms, MLXSW_REG_SPMS_ID, MLXSW_REG_SPMS_LEN);
* Local port number.
* Access: Index
*/
-MLXSW_ITEM32(reg, spms, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, spms, 0x00, 16, 0x00, 12);
enum mlxsw_reg_spms_state {
MLXSW_REG_SPMS_STATE_NO_CHANGE,
@@ -800,7 +760,7 @@ enum mlxsw_reg_spms_state {
*/
MLXSW_ITEM_BIT_ARRAY(reg, spms, state, 0x04, 0x400, 2);
-static inline void mlxsw_reg_spms_pack(char *payload, u8 local_port)
+static inline void mlxsw_reg_spms_pack(char *payload, u16 local_port)
{
MLXSW_REG_ZERO(spms, payload);
mlxsw_reg_spms_local_port_set(payload, local_port);
@@ -833,7 +793,7 @@ MLXSW_ITEM32(reg, spvid, tport, 0x00, 24, 1);
* When tport = 1: Tunnel port.
* Access: Index
*/
-MLXSW_ITEM32(reg, spvid, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, spvid, 0x00, 16, 0x00, 12);
/* reg_spvid_sub_port
* Virtual port within the physical port.
@@ -868,7 +828,7 @@ MLXSW_ITEM32(reg, spvid, et_vlan, 0x04, 16, 2);
*/
MLXSW_ITEM32(reg, spvid, pvid, 0x04, 0, 12);
-static inline void mlxsw_reg_spvid_pack(char *payload, u8 local_port, u16 pvid,
+static inline void mlxsw_reg_spvid_pack(char *payload, u16 local_port, u16 pvid,
u8 et_vlan)
{
MLXSW_REG_ZERO(spvid, payload);
@@ -911,7 +871,7 @@ MLXSW_ITEM32(reg, spvm, pte, 0x00, 30, 1);
* Local port number.
* Access: Index
*/
-MLXSW_ITEM32(reg, spvm, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, spvm, 0x00, 16, 0x00, 12);
/* reg_spvm_sub_port
* Virtual port within the physical port.
@@ -959,7 +919,7 @@ MLXSW_ITEM32_INDEXED(reg, spvm, rec_vid,
MLXSW_REG_SPVM_BASE_LEN, 0, 12,
MLXSW_REG_SPVM_REC_LEN, 0, false);
-static inline void mlxsw_reg_spvm_pack(char *payload, u8 local_port,
+static inline void mlxsw_reg_spvm_pack(char *payload, u16 local_port,
u16 vid_begin, u16 vid_end,
bool is_member, bool untagged)
{
@@ -994,7 +954,7 @@ MLXSW_REG_DEFINE(spaft, MLXSW_REG_SPAFT_ID, MLXSW_REG_SPAFT_LEN);
*
* Note: CPU port is not supported (all tag types are allowed).
*/
-MLXSW_ITEM32(reg, spaft, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, spaft, 0x00, 16, 0x00, 12);
/* reg_spaft_sub_port
* Virtual port within the physical port.
@@ -1021,7 +981,7 @@ MLXSW_ITEM32(reg, spaft, allow_prio_tagged, 0x04, 30, 1);
*/
MLXSW_ITEM32(reg, spaft, allow_tagged, 0x04, 29, 1);
-static inline void mlxsw_reg_spaft_pack(char *payload, u8 local_port,
+static inline void mlxsw_reg_spaft_pack(char *payload, u16 local_port,
bool allow_untagged)
{
MLXSW_REG_ZERO(spaft, payload);
@@ -1126,76 +1086,6 @@ mlxsw_reg_sfgc_pack(char *payload, enum mlxsw_reg_sfgc_type type,
mlxsw_reg_sfgc_mid_set(payload, MLXSW_PORT_MID);
}
-/* SFTR - Switch Flooding Table Register
- * -------------------------------------
- * The switch flooding table is used for flooding packet replication. The table
- * defines a bit mask of ports for packet replication.
- */
-#define MLXSW_REG_SFTR_ID 0x2012
-#define MLXSW_REG_SFTR_LEN 0x420
-
-MLXSW_REG_DEFINE(sftr, MLXSW_REG_SFTR_ID, MLXSW_REG_SFTR_LEN);
-
-/* reg_sftr_swid
- * Switch partition ID with which to associate the port.
- * Access: Index
- */
-MLXSW_ITEM32(reg, sftr, swid, 0x00, 24, 8);
-
-/* reg_sftr_flood_table
- * Flooding table index to associate with the specific type on the specific
- * switch partition.
- * Access: Index
- */
-MLXSW_ITEM32(reg, sftr, flood_table, 0x00, 16, 6);
-
-/* reg_sftr_index
- * Index. Used as an index into the Flooding Table in case the table is
- * configured to use VID / FID or FID Offset.
- * Access: Index
- */
-MLXSW_ITEM32(reg, sftr, index, 0x00, 0, 16);
-
-/* reg_sftr_table_type
- * See mlxsw_flood_table_type
- * Access: RW
- */
-MLXSW_ITEM32(reg, sftr, table_type, 0x04, 16, 3);
-
-/* reg_sftr_range
- * Range of entries to update
- * Access: Index
- */
-MLXSW_ITEM32(reg, sftr, range, 0x04, 0, 16);
-
-/* reg_sftr_port
- * Local port membership (1 bit per port).
- * Access: RW
- */
-MLXSW_ITEM_BIT_ARRAY(reg, sftr, port, 0x20, 0x20, 1);
-
-/* reg_sftr_cpu_port_mask
- * CPU port mask (1 bit per port).
- * Access: W
- */
-MLXSW_ITEM_BIT_ARRAY(reg, sftr, port_mask, 0x220, 0x20, 1);
-
-static inline void mlxsw_reg_sftr_pack(char *payload,
- unsigned int flood_table,
- unsigned int index,
- enum mlxsw_flood_table_type table_type,
- unsigned int range, u8 port, bool set)
-{
- MLXSW_REG_ZERO(sftr, payload);
- mlxsw_reg_sftr_swid_set(payload, 0);
- mlxsw_reg_sftr_flood_table_set(payload, flood_table);
- mlxsw_reg_sftr_index_set(payload, index);
- mlxsw_reg_sftr_table_type_set(payload, table_type);
- mlxsw_reg_sftr_range_set(payload, range);
- mlxsw_reg_sftr_port_set(payload, port, set);
- mlxsw_reg_sftr_port_mask_set(payload, port, 1);
-}
-
/* SFDF - Switch Filtering DB Flush
* --------------------------------
* The switch filtering DB flush register is used to flush the FDB.
@@ -1347,7 +1237,7 @@ MLXSW_ITEM32(reg, sldr, num_ports, 0x04, 24, 8);
MLXSW_ITEM32_INDEXED(reg, sldr, system_port, 0x08, 0, 16, 4, 0, false);
static inline void mlxsw_reg_sldr_lag_add_port_pack(char *payload, u8 lag_id,
- u8 local_port)
+ u16 local_port)
{
MLXSW_REG_ZERO(sldr, payload);
mlxsw_reg_sldr_op_set(payload, MLXSW_REG_SLDR_OP_LAG_ADD_PORT_LIST);
@@ -1357,7 +1247,7 @@ static inline void mlxsw_reg_sldr_lag_add_port_pack(char *payload, u8 lag_id,
}
static inline void mlxsw_reg_sldr_lag_remove_port_pack(char *payload, u8 lag_id,
- u8 local_port)
+ u16 local_port)
{
MLXSW_REG_ZERO(sldr, payload);
mlxsw_reg_sldr_op_set(payload, MLXSW_REG_SLDR_OP_LAG_REMOVE_PORT_LIST);
@@ -1397,7 +1287,7 @@ MLXSW_ITEM32(reg, slcr, pp, 0x00, 24, 1);
* Reserved when pp = Global Configuration
* Access: Index
*/
-MLXSW_ITEM32(reg, slcr, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, slcr, 0x00, 16, 0x00, 12);
enum mlxsw_reg_slcr_type {
MLXSW_REG_SLCR_TYPE_CRC, /* default */
@@ -1515,7 +1405,7 @@ MLXSW_ITEM32(reg, slcor, col, 0x00, 30, 2);
* Not supported for CPU port
* Access: Index
*/
-MLXSW_ITEM32(reg, slcor, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, slcor, 0x00, 16, 0x00, 12);
/* reg_slcor_lag_id
* LAG Identifier. Index into the LAG descriptor table.
@@ -1531,7 +1421,7 @@ MLXSW_ITEM32(reg, slcor, lag_id, 0x00, 0, 10);
MLXSW_ITEM32(reg, slcor, port_index, 0x04, 0, 10);
static inline void mlxsw_reg_slcor_pack(char *payload,
- u8 local_port, u16 lag_id,
+ u16 local_port, u16 lag_id,
enum mlxsw_reg_slcor_col col)
{
MLXSW_REG_ZERO(slcor, payload);
@@ -1541,7 +1431,7 @@ static inline void mlxsw_reg_slcor_pack(char *payload,
}
static inline void mlxsw_reg_slcor_port_add_pack(char *payload,
- u8 local_port, u16 lag_id,
+ u16 local_port, u16 lag_id,
u8 port_index)
{
mlxsw_reg_slcor_pack(payload, local_port, lag_id,
@@ -1550,21 +1440,21 @@ static inline void mlxsw_reg_slcor_port_add_pack(char *payload,
}
static inline void mlxsw_reg_slcor_port_remove_pack(char *payload,
- u8 local_port, u16 lag_id)
+ u16 local_port, u16 lag_id)
{
mlxsw_reg_slcor_pack(payload, local_port, lag_id,
MLXSW_REG_SLCOR_COL_LAG_REMOVE_PORT);
}
static inline void mlxsw_reg_slcor_col_enable_pack(char *payload,
- u8 local_port, u16 lag_id)
+ u16 local_port, u16 lag_id)
{
mlxsw_reg_slcor_pack(payload, local_port, lag_id,
MLXSW_REG_SLCOR_COL_LAG_COLLECTOR_ENABLED);
}
static inline void mlxsw_reg_slcor_col_disable_pack(char *payload,
- u8 local_port, u16 lag_id)
+ u16 local_port, u16 lag_id)
{
mlxsw_reg_slcor_pack(payload, local_port, lag_id,
MLXSW_REG_SLCOR_COL_LAG_COLLECTOR_ENABLED);
@@ -1583,7 +1473,7 @@ MLXSW_REG_DEFINE(spmlr, MLXSW_REG_SPMLR_ID, MLXSW_REG_SPMLR_LEN);
* Local port number.
* Access: Index
*/
-MLXSW_ITEM32(reg, spmlr, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, spmlr, 0x00, 16, 0x00, 12);
/* reg_spmlr_sub_port
* Virtual port within the physical port.
@@ -1611,7 +1501,7 @@ enum mlxsw_reg_spmlr_learn_mode {
*/
MLXSW_ITEM32(reg, spmlr, learn_mode, 0x04, 30, 2);
-static inline void mlxsw_reg_spmlr_pack(char *payload, u8 local_port,
+static inline void mlxsw_reg_spmlr_pack(char *payload, u16 local_port,
enum mlxsw_reg_spmlr_learn_mode mode)
{
MLXSW_REG_ZERO(spmlr, payload);
@@ -1642,7 +1532,7 @@ MLXSW_ITEM32(reg, svfa, swid, 0x00, 24, 8);
*
* Note: Reserved for 802.1Q FIDs.
*/
-MLXSW_ITEM32(reg, svfa, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, svfa, 0x00, 16, 0x00, 12);
enum mlxsw_reg_svfa_mt {
MLXSW_REG_SVFA_MT_VID_TO_FID,
@@ -1696,7 +1586,7 @@ MLXSW_ITEM32(reg, svfa, counter_set_type, 0x08, 24, 8);
*/
MLXSW_ITEM32(reg, svfa, counter_index, 0x08, 0, 24);
-static inline void mlxsw_reg_svfa_pack(char *payload, u8 local_port,
+static inline void mlxsw_reg_svfa_pack(char *payload, u16 local_port,
enum mlxsw_reg_svfa_mt mt, bool valid,
u16 fid, u16 vid)
{
@@ -1733,7 +1623,7 @@ MLXSW_ITEM32(reg, spvtr, tport, 0x00, 24, 1);
* When tport = 1: tunnel port.
* Access: Index
*/
-MLXSW_ITEM32(reg, spvtr, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, spvtr, 0x00, 16, 0x00, 12);
/* reg_spvtr_ippe
* Ingress Port Prio Mode Update Enable.
@@ -1803,7 +1693,7 @@ enum mlxsw_reg_spvtr_epvid_mode {
MLXSW_ITEM32(reg, spvtr, epvid_mode, 0x04, 0, 4);
static inline void mlxsw_reg_spvtr_pack(char *payload, bool tport,
- u8 local_port,
+ u16 local_port,
enum mlxsw_reg_spvtr_ipvid_mode ipvid_mode)
{
MLXSW_REG_ZERO(spvtr, payload);
@@ -1828,7 +1718,7 @@ MLXSW_REG_DEFINE(svpe, MLXSW_REG_SVPE_ID, MLXSW_REG_SVPE_LEN);
*
* Note: CPU port is not supported (uses VLAN mode only).
*/
-MLXSW_ITEM32(reg, svpe, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, svpe, 0x00, 16, 0x00, 12);
/* reg_svpe_vp_en
* Virtual port enable.
@@ -1838,7 +1728,7 @@ MLXSW_ITEM32(reg, svpe, local_port, 0x00, 16, 8);
*/
MLXSW_ITEM32(reg, svpe, vp_en, 0x00, 8, 1);
-static inline void mlxsw_reg_svpe_pack(char *payload, u8 local_port,
+static inline void mlxsw_reg_svpe_pack(char *payload, u16 local_port,
bool enable)
{
MLXSW_REG_ZERO(svpe, payload);
@@ -1948,7 +1838,7 @@ MLXSW_REG_DEFINE(spvmlr, MLXSW_REG_SPVMLR_ID, MLXSW_REG_SPVMLR_LEN);
*
* Note: CPU port is not supported.
*/
-MLXSW_ITEM32(reg, spvmlr, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, spvmlr, 0x00, 16, 0x00, 12);
/* reg_spvmlr_num_rec
* Number of records to update.
@@ -1971,7 +1861,7 @@ MLXSW_ITEM32_INDEXED(reg, spvmlr, rec_learn_enable, MLXSW_REG_SPVMLR_BASE_LEN,
MLXSW_ITEM32_INDEXED(reg, spvmlr, rec_vid, MLXSW_REG_SPVMLR_BASE_LEN, 0, 12,
MLXSW_REG_SPVMLR_REC_LEN, 0x00, false);
-static inline void mlxsw_reg_spvmlr_pack(char *payload, u8 local_port,
+static inline void mlxsw_reg_spvmlr_pack(char *payload, u16 local_port,
u16 vid_begin, u16 vid_end,
bool learn_enable)
{
@@ -2009,7 +1899,7 @@ MLXSW_REG_DEFINE(spvc, MLXSW_REG_SPVC_ID, MLXSW_REG_SPVC_LEN);
* through Rx port i and a Tx port j then port i and port j must have the
* same configuration.
*/
-MLXSW_ITEM32(reg, spvc, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, spvc, 0x00, 16, 0x00, 12);
/* reg_spvc_inner_et2
* Vlan Tag1 EtherType2 enable.
@@ -2074,7 +1964,7 @@ MLXSW_ITEM32(reg, spvc, inner_et0, 0x08, 1, 1);
*/
MLXSW_ITEM32(reg, spvc, et0, 0x08, 0, 1);
-static inline void mlxsw_reg_spvc_pack(char *payload, u8 local_port, bool et1,
+static inline void mlxsw_reg_spvc_pack(char *payload, u16 local_port, bool et1,
bool et0)
{
MLXSW_REG_ZERO(spvc, payload);
@@ -2104,7 +1994,7 @@ MLXSW_REG_DEFINE(spevet, MLXSW_REG_SPEVET_ID, MLXSW_REG_SPEVET_LEN);
* Not supported to CPU port.
* Access: Index
*/
-MLXSW_ITEM32(reg, spevet, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, spevet, 0x00, 16, 0x00, 12);
/* reg_spevet_et_vlan
* Egress EtherType VLAN to push when SPVID.egr_et_set field set for the packet:
@@ -2115,7 +2005,7 @@ MLXSW_ITEM32(reg, spevet, local_port, 0x00, 16, 8);
*/
MLXSW_ITEM32(reg, spevet, et_vlan, 0x04, 16, 2);
-static inline void mlxsw_reg_spevet_pack(char *payload, u8 local_port,
+static inline void mlxsw_reg_spevet_pack(char *payload, u16 local_port,
u8 et_vlan)
{
MLXSW_REG_ZERO(spevet, payload);
@@ -2123,6 +2013,122 @@ static inline void mlxsw_reg_spevet_pack(char *payload, u8 local_port,
mlxsw_reg_spevet_et_vlan_set(payload, et_vlan);
}
+/* SFTR-V2 - Switch Flooding Table Version 2 Register
+ * --------------------------------------------------
+ * The switch flooding table is used for flooding packet replication. The table
+ * defines a bit mask of ports for packet replication.
+ */
+#define MLXSW_REG_SFTR2_ID 0x202F
+#define MLXSW_REG_SFTR2_LEN 0x120
+
+MLXSW_REG_DEFINE(sftr2, MLXSW_REG_SFTR2_ID, MLXSW_REG_SFTR2_LEN);
+
+/* reg_sftr2_swid
+ * Switch partition ID with which to associate the port.
+ * Access: Index
+ */
+MLXSW_ITEM32(reg, sftr2, swid, 0x00, 24, 8);
+
+/* reg_sftr2_flood_table
+ * Flooding table index to associate with the specific type on the specific
+ * switch partition.
+ * Access: Index
+ */
+MLXSW_ITEM32(reg, sftr2, flood_table, 0x00, 16, 6);
+
+/* reg_sftr2_index
+ * Index. Used as an index into the Flooding Table in case the table is
+ * configured to use VID / FID or FID Offset.
+ * Access: Index
+ */
+MLXSW_ITEM32(reg, sftr2, index, 0x00, 0, 16);
+
+/* reg_sftr2_table_type
+ * See mlxsw_flood_table_type
+ * Access: RW
+ */
+MLXSW_ITEM32(reg, sftr2, table_type, 0x04, 16, 3);
+
+/* reg_sftr2_range
+ * Range of entries to update
+ * Access: Index
+ */
+MLXSW_ITEM32(reg, sftr2, range, 0x04, 0, 16);
+
+/* reg_sftr2_port
+ * Local port membership (1 bit per port).
+ * Access: RW
+ */
+MLXSW_ITEM_BIT_ARRAY(reg, sftr2, port, 0x20, 0x80, 1);
+
+/* reg_sftr2_port_mask
+ * Local port mask (1 bit per port).
+ * Access: WO
+ */
+MLXSW_ITEM_BIT_ARRAY(reg, sftr2, port_mask, 0xA0, 0x80, 1);
+
+static inline void mlxsw_reg_sftr2_pack(char *payload,
+ unsigned int flood_table,
+ unsigned int index,
+ enum mlxsw_flood_table_type table_type,
+ unsigned int range, u16 port, bool set)
+{
+ MLXSW_REG_ZERO(sftr2, payload);
+ mlxsw_reg_sftr2_swid_set(payload, 0);
+ mlxsw_reg_sftr2_flood_table_set(payload, flood_table);
+ mlxsw_reg_sftr2_index_set(payload, index);
+ mlxsw_reg_sftr2_table_type_set(payload, table_type);
+ mlxsw_reg_sftr2_range_set(payload, range);
+ mlxsw_reg_sftr2_port_set(payload, port, set);
+ mlxsw_reg_sftr2_port_mask_set(payload, port, 1);
+}
+
+/* SMID-V2 - Switch Multicast ID Version 2 Register
+ * ------------------------------------------------
+ * The MID record maps from a MID (Multicast ID), which is a unique identifier
+ * of the multicast group within the stacking domain, into a list of local
+ * ports into which the packet is replicated.
+ */
+#define MLXSW_REG_SMID2_ID 0x2034
+#define MLXSW_REG_SMID2_LEN 0x120
+
+MLXSW_REG_DEFINE(smid2, MLXSW_REG_SMID2_ID, MLXSW_REG_SMID2_LEN);
+
+/* reg_smid2_swid
+ * Switch partition ID.
+ * Access: Index
+ */
+MLXSW_ITEM32(reg, smid2, swid, 0x00, 24, 8);
+
+/* reg_smid2_mid
+ * Multicast identifier - global identifier that represents the multicast group
+ * across all devices.
+ * Access: Index
+ */
+MLXSW_ITEM32(reg, smid2, mid, 0x00, 0, 16);
+
+/* reg_smid2_port
+ * Local port memebership (1 bit per port).
+ * Access: RW
+ */
+MLXSW_ITEM_BIT_ARRAY(reg, smid2, port, 0x20, 0x80, 1);
+
+/* reg_smid2_port_mask
+ * Local port mask (1 bit per port).
+ * Access: WO
+ */
+MLXSW_ITEM_BIT_ARRAY(reg, smid2, port_mask, 0xA0, 0x80, 1);
+
+static inline void mlxsw_reg_smid2_pack(char *payload, u16 mid, u16 port,
+ bool set)
+{
+ MLXSW_REG_ZERO(smid2, payload);
+ mlxsw_reg_smid2_swid_set(payload, 0);
+ mlxsw_reg_smid2_mid_set(payload, mid);
+ mlxsw_reg_smid2_port_set(payload, port, set);
+ mlxsw_reg_smid2_port_mask_set(payload, port, 1);
+}
+
/* CWTP - Congetion WRED ECN TClass Profile
* ----------------------------------------
* Configures the profiles for queues of egress port and traffic class
@@ -2139,7 +2145,7 @@ MLXSW_REG_DEFINE(cwtp, MLXSW_REG_CWTP_ID, MLXSW_REG_CWTP_LEN);
* Not supported for CPU port
* Access: Index
*/
-MLXSW_ITEM32(reg, cwtp, local_port, 0, 16, 8);
+MLXSW_ITEM32_LP(reg, cwtp, 0x00, 16, 0x00, 12);
/* reg_cwtp_traffic_class
* Traffic Class to configure
@@ -2173,7 +2179,7 @@ MLXSW_ITEM32_INDEXED(reg, cwtp, profile_max, MLXSW_REG_CWTP_BASE_LEN,
#define MLXSW_REG_CWTP_MAX_PROFILE 2
#define MLXSW_REG_CWTP_DEFAULT_PROFILE 1
-static inline void mlxsw_reg_cwtp_pack(char *payload, u8 local_port,
+static inline void mlxsw_reg_cwtp_pack(char *payload, u16 local_port,
u8 traffic_class)
{
int i;
@@ -2217,7 +2223,7 @@ MLXSW_REG_DEFINE(cwtpm, MLXSW_REG_CWTPM_ID, MLXSW_REG_CWTPM_LEN);
* Not supported for CPU port
* Access: Index
*/
-MLXSW_ITEM32(reg, cwtpm, local_port, 0, 16, 8);
+MLXSW_ITEM32_LP(reg, cwtpm, 0x00, 16, 0x00, 12);
/* reg_cwtpm_traffic_class
* Traffic Class to configure
@@ -2291,7 +2297,7 @@ MLXSW_ITEM32(reg, cwtpm, ntcp_r, 64, 0, 2);
#define MLXSW_REG_CWTPM_RESET_PROFILE 0
-static inline void mlxsw_reg_cwtpm_pack(char *payload, u8 local_port,
+static inline void mlxsw_reg_cwtpm_pack(char *payload, u16 local_port,
u8 traffic_class, u8 profile,
bool wred, bool ecn)
{
@@ -2363,7 +2369,7 @@ MLXSW_ITEM32(reg, ppbt, op, 0x00, 28, 3);
* Local port. Not including CPU port.
* Access: Index
*/
-MLXSW_ITEM32(reg, ppbt, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, ppbt, 0x00, 16, 0x00, 12);
/* reg_ppbt_g
* group - When set, the binding is of an ACL group. When cleared,
@@ -2382,7 +2388,7 @@ MLXSW_ITEM32(reg, ppbt, acl_info, 0x10, 0, 16);
static inline void mlxsw_reg_ppbt_pack(char *payload, enum mlxsw_reg_pxbt_e e,
enum mlxsw_reg_pxbt_op op,
- u8 local_port, u16 acl_info)
+ u16 local_port, u16 acl_info)
{
MLXSW_REG_ZERO(ppbt, payload);
mlxsw_reg_ppbt_e_set(payload, e);
@@ -3513,7 +3519,7 @@ MLXSW_REG_DEFINE(qpts, MLXSW_REG_QPTS_ID, MLXSW_REG_QPTS_LEN);
*
* Note: CPU port is supported.
*/
-MLXSW_ITEM32(reg, qpts, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, qpts, 0x00, 16, 0x00, 12);
enum mlxsw_reg_qpts_trust_state {
MLXSW_REG_QPTS_TRUST_STATE_PCP = 1,
@@ -3526,7 +3532,7 @@ enum mlxsw_reg_qpts_trust_state {
*/
MLXSW_ITEM32(reg, qpts, trust_state, 0x04, 0, 3);
-static inline void mlxsw_reg_qpts_pack(char *payload, u8 local_port,
+static inline void mlxsw_reg_qpts_pack(char *payload, u16 local_port,
enum mlxsw_reg_qpts_trust_state ts)
{
MLXSW_REG_ZERO(qpts, payload);
@@ -3717,7 +3723,7 @@ MLXSW_REG_DEFINE(qtct, MLXSW_REG_QTCT_ID, MLXSW_REG_QTCT_LEN);
*
* Note: CPU port is not supported.
*/
-MLXSW_ITEM32(reg, qtct, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, qtct, 0x00, 16, 0x00, 12);
/* reg_qtct_sub_port
* Virtual port within the physical port.
@@ -3742,7 +3748,7 @@ MLXSW_ITEM32(reg, qtct, switch_prio, 0x00, 0, 4);
*/
MLXSW_ITEM32(reg, qtct, tclass, 0x04, 0, 4);
-static inline void mlxsw_reg_qtct_pack(char *payload, u8 local_port,
+static inline void mlxsw_reg_qtct_pack(char *payload, u16 local_port,
u8 switch_prio, u8 tclass)
{
MLXSW_REG_ZERO(qtct, payload);
@@ -3766,7 +3772,7 @@ MLXSW_REG_DEFINE(qeec, MLXSW_REG_QEEC_ID, MLXSW_REG_QEEC_LEN);
*
* Note: CPU port is supported.
*/
-MLXSW_ITEM32(reg, qeec, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, qeec, 0x00, 16, 0x00, 12);
enum mlxsw_reg_qeec_hr {
MLXSW_REG_QEEC_HR_PORT,
@@ -3908,8 +3914,9 @@ MLXSW_ITEM32(reg, qeec, max_shaper_bs, 0x1C, 0, 6);
#define MLXSW_REG_QEEC_LOWEST_SHAPER_BS_SP1 5
#define MLXSW_REG_QEEC_LOWEST_SHAPER_BS_SP2 11
#define MLXSW_REG_QEEC_LOWEST_SHAPER_BS_SP3 11
+#define MLXSW_REG_QEEC_LOWEST_SHAPER_BS_SP4 11
-static inline void mlxsw_reg_qeec_pack(char *payload, u8 local_port,
+static inline void mlxsw_reg_qeec_pack(char *payload, u16 local_port,
enum mlxsw_reg_qeec_hr hr, u8 index,
u8 next_index)
{
@@ -3920,7 +3927,7 @@ static inline void mlxsw_reg_qeec_pack(char *payload, u8 local_port,
mlxsw_reg_qeec_next_element_index_set(payload, next_index);
}
-static inline void mlxsw_reg_qeec_ptps_pack(char *payload, u8 local_port,
+static inline void mlxsw_reg_qeec_ptps_pack(char *payload, u16 local_port,
bool ptps)
{
MLXSW_REG_ZERO(qeec, payload);
@@ -3944,7 +3951,7 @@ MLXSW_REG_DEFINE(qrwe, MLXSW_REG_QRWE_ID, MLXSW_REG_QRWE_LEN);
*
* Note: CPU port is supported. No support for router port.
*/
-MLXSW_ITEM32(reg, qrwe, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, qrwe, 0x00, 16, 0x00, 12);
/* reg_qrwe_dscp
* Whether to enable DSCP rewrite (default is 0, don't rewrite).
@@ -3958,7 +3965,7 @@ MLXSW_ITEM32(reg, qrwe, dscp, 0x04, 1, 1);
*/
MLXSW_ITEM32(reg, qrwe, pcp, 0x04, 0, 1);
-static inline void mlxsw_reg_qrwe_pack(char *payload, u8 local_port,
+static inline void mlxsw_reg_qrwe_pack(char *payload, u16 local_port,
bool rewrite_pcp, bool rewrite_dscp)
{
MLXSW_REG_ZERO(qrwe, payload);
@@ -3985,7 +3992,7 @@ MLXSW_REG_DEFINE(qpdsm, MLXSW_REG_QPDSM_ID, MLXSW_REG_QPDSM_LEN);
* Local Port. Supported for data packets from CPU port.
* Access: Index
*/
-MLXSW_ITEM32(reg, qpdsm, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, qpdsm, 0x00, 16, 0x00, 12);
/* reg_qpdsm_prio_entry_color0_e
* Enable update of the entry for color 0 and a given port.
@@ -4038,7 +4045,7 @@ MLXSW_ITEM32_INDEXED(reg, qpdsm, prio_entry_color2_dscp,
MLXSW_REG_QPDSM_BASE_LEN, 8, 6,
MLXSW_REG_QPDSM_PRIO_ENTRY_REC_LEN, 0x00, false);
-static inline void mlxsw_reg_qpdsm_pack(char *payload, u8 local_port)
+static inline void mlxsw_reg_qpdsm_pack(char *payload, u16 local_port)
{
MLXSW_REG_ZERO(qpdsm, payload);
mlxsw_reg_qpdsm_local_port_set(payload, local_port);
@@ -4071,7 +4078,7 @@ MLXSW_REG_DEFINE(qpdp, MLXSW_REG_QPDP_ID, MLXSW_REG_QPDP_LEN);
* Local Port. Supported for data packets from CPU port.
* Access: Index
*/
-MLXSW_ITEM32(reg, qpdp, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, qpdp, 0x00, 16, 0x00, 12);
/* reg_qpdp_switch_prio
* Default port Switch Priority (default 0)
@@ -4079,7 +4086,7 @@ MLXSW_ITEM32(reg, qpdp, local_port, 0x00, 16, 8);
*/
MLXSW_ITEM32(reg, qpdp, switch_prio, 0x04, 0, 4);
-static inline void mlxsw_reg_qpdp_pack(char *payload, u8 local_port,
+static inline void mlxsw_reg_qpdp_pack(char *payload, u16 local_port,
u8 switch_prio)
{
MLXSW_REG_ZERO(qpdp, payload);
@@ -4106,7 +4113,7 @@ MLXSW_REG_DEFINE(qpdpm, MLXSW_REG_QPDPM_ID, MLXSW_REG_QPDPM_LEN);
* Local Port. Supported for data packets from CPU port.
* Access: Index
*/
-MLXSW_ITEM32(reg, qpdpm, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, qpdpm, 0x00, 16, 0x00, 12);
/* reg_qpdpm_dscp_e
* Enable update of the specific entry. When cleared, the switch_prio and color
@@ -4125,7 +4132,7 @@ MLXSW_ITEM16_INDEXED(reg, qpdpm, dscp_entry_prio,
MLXSW_REG_QPDPM_BASE_LEN, 0, 4,
MLXSW_REG_QPDPM_DSCP_ENTRY_REC_LEN, 0x00, false);
-static inline void mlxsw_reg_qpdpm_pack(char *payload, u8 local_port)
+static inline void mlxsw_reg_qpdpm_pack(char *payload, u16 local_port)
{
MLXSW_REG_ZERO(qpdpm, payload);
mlxsw_reg_qpdpm_local_port_set(payload, local_port);
@@ -4157,7 +4164,7 @@ MLXSW_REG_DEFINE(qtctm, MLXSW_REG_QTCTM_ID, MLXSW_REG_QTCTM_LEN);
* No support for CPU port.
* Access: Index
*/
-MLXSW_ITEM32(reg, qtctm, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, qtctm, 0x00, 16, 0x00, 12);
/* reg_qtctm_mc
* Multicast Mode
@@ -4167,7 +4174,7 @@ MLXSW_ITEM32(reg, qtctm, local_port, 0x00, 16, 8);
MLXSW_ITEM32(reg, qtctm, mc, 0x04, 0, 1);
static inline void
-mlxsw_reg_qtctm_pack(char *payload, u8 local_port, bool mc)
+mlxsw_reg_qtctm_pack(char *payload, u16 local_port, bool mc)
{
MLXSW_REG_ZERO(qtctm, payload);
mlxsw_reg_qtctm_local_port_set(payload, local_port);
@@ -4300,7 +4307,7 @@ MLXSW_ITEM32(reg, pmlp, rxtx, 0x00, 31, 1);
* Local port number.
* Access: Index
*/
-MLXSW_ITEM32(reg, pmlp, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, pmlp, 0x00, 16, 0x00, 12);
/* reg_pmlp_width
* 0 - Unmap local port.
@@ -4331,7 +4338,7 @@ MLXSW_ITEM32_INDEXED(reg, pmlp, tx_lane, 0x04, 16, 4, 0x04, 0x00, false);
*/
MLXSW_ITEM32_INDEXED(reg, pmlp, rx_lane, 0x04, 24, 4, 0x04, 0x00, false);
-static inline void mlxsw_reg_pmlp_pack(char *payload, u8 local_port)
+static inline void mlxsw_reg_pmlp_pack(char *payload, u16 local_port)
{
MLXSW_REG_ZERO(pmlp, payload);
mlxsw_reg_pmlp_local_port_set(payload, local_port);
@@ -4350,7 +4357,7 @@ MLXSW_REG_DEFINE(pmtu, MLXSW_REG_PMTU_ID, MLXSW_REG_PMTU_LEN);
* Local port number.
* Access: Index
*/
-MLXSW_ITEM32(reg, pmtu, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, pmtu, 0x00, 16, 0x00, 12);
/* reg_pmtu_max_mtu
* Maximum MTU.
@@ -4378,7 +4385,7 @@ MLXSW_ITEM32(reg, pmtu, admin_mtu, 0x08, 16, 16);
*/
MLXSW_ITEM32(reg, pmtu, oper_mtu, 0x0C, 16, 16);
-static inline void mlxsw_reg_pmtu_pack(char *payload, u8 local_port,
+static inline void mlxsw_reg_pmtu_pack(char *payload, u16 local_port,
u16 new_mtu)
{
MLXSW_REG_ZERO(pmtu, payload);
@@ -4412,7 +4419,7 @@ MLXSW_ITEM32(reg, ptys, an_disable_admin, 0x00, 30, 1);
* Local port number.
* Access: Index
*/
-MLXSW_ITEM32(reg, ptys, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, ptys, 0x00, 16, 0x00, 12);
#define MLXSW_REG_PTYS_PROTO_MASK_IB BIT(0)
#define MLXSW_REG_PTYS_PROTO_MASK_ETH BIT(2)
@@ -4572,7 +4579,7 @@ enum mlxsw_reg_ptys_connector_type {
*/
MLXSW_ITEM32(reg, ptys, connector_type, 0x2C, 0, 4);
-static inline void mlxsw_reg_ptys_eth_pack(char *payload, u8 local_port,
+static inline void mlxsw_reg_ptys_eth_pack(char *payload, u16 local_port,
u32 proto_admin, bool autoneg)
{
MLXSW_REG_ZERO(ptys, payload);
@@ -4582,7 +4589,7 @@ static inline void mlxsw_reg_ptys_eth_pack(char *payload, u8 local_port,
mlxsw_reg_ptys_an_disable_admin_set(payload, !autoneg);
}
-static inline void mlxsw_reg_ptys_ext_eth_pack(char *payload, u8 local_port,
+static inline void mlxsw_reg_ptys_ext_eth_pack(char *payload, u16 local_port,
u32 proto_admin, bool autoneg)
{
MLXSW_REG_ZERO(ptys, payload);
@@ -4624,7 +4631,7 @@ static inline void mlxsw_reg_ptys_ext_eth_unpack(char *payload,
mlxsw_reg_ptys_ext_eth_proto_oper_get(payload);
}
-static inline void mlxsw_reg_ptys_ib_pack(char *payload, u8 local_port,
+static inline void mlxsw_reg_ptys_ib_pack(char *payload, u16 local_port,
u16 proto_admin, u16 link_width)
{
MLXSW_REG_ZERO(ptys, payload);
@@ -4672,7 +4679,7 @@ MLXSW_ITEM32(reg, ppad, single_base_mac, 0x00, 28, 1);
* port number, if single_base_mac = 0 then local_port is reserved
* Access: RW
*/
-MLXSW_ITEM32(reg, ppad, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, ppad, 0x00, 16, 0x00, 24);
/* reg_ppad_mac
* If single_base_mac = 0 - base MAC address, mac[7:0] is reserved.
@@ -4682,7 +4689,7 @@ MLXSW_ITEM32(reg, ppad, local_port, 0x00, 16, 8);
MLXSW_ITEM_BUF(reg, ppad, mac, 0x02, 6);
static inline void mlxsw_reg_ppad_pack(char *payload, bool single_base_mac,
- u8 local_port)
+ u16 local_port)
{
MLXSW_REG_ZERO(ppad, payload);
mlxsw_reg_ppad_single_base_mac_set(payload, !!single_base_mac);
@@ -4711,7 +4718,7 @@ MLXSW_ITEM32(reg, paos, swid, 0x00, 24, 8);
* Local port number.
* Access: Index
*/
-MLXSW_ITEM32(reg, paos, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, paos, 0x00, 16, 0x00, 12);
/* reg_paos_admin_status
* Port administrative state (the desired state of the port):
@@ -4756,7 +4763,7 @@ MLXSW_ITEM32(reg, paos, ee, 0x04, 30, 1);
*/
MLXSW_ITEM32(reg, paos, e, 0x04, 0, 2);
-static inline void mlxsw_reg_paos_pack(char *payload, u8 local_port,
+static inline void mlxsw_reg_paos_pack(char *payload, u16 local_port,
enum mlxsw_port_admin_status status)
{
MLXSW_REG_ZERO(paos, payload);
@@ -4782,7 +4789,7 @@ MLXSW_REG_DEFINE(pfcc, MLXSW_REG_PFCC_ID, MLXSW_REG_PFCC_LEN);
* Local port number.
* Access: Index
*/
-MLXSW_ITEM32(reg, pfcc, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, pfcc, 0x00, 16, 0x00, 12);
/* reg_pfcc_pnat
* Port number access type. Determines the way local_port is interpreted:
@@ -4899,7 +4906,7 @@ static inline void mlxsw_reg_pfcc_prio_pack(char *payload, u8 pfc_en)
mlxsw_reg_pfcc_pfcrx_set(payload, pfc_en);
}
-static inline void mlxsw_reg_pfcc_pack(char *payload, u8 local_port)
+static inline void mlxsw_reg_pfcc_pack(char *payload, u16 local_port)
{
MLXSW_REG_ZERO(pfcc, payload);
mlxsw_reg_pfcc_local_port_set(payload, local_port);
@@ -4928,11 +4935,9 @@ MLXSW_ITEM32(reg, ppcnt, swid, 0x00, 24, 8);
/* reg_ppcnt_local_port
* Local port number.
- * 255 indicates all ports on the device, and is only allowed
- * for Set() operation.
* Access: Index
*/
-MLXSW_ITEM32(reg, ppcnt, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, ppcnt, 0x00, 16, 0x00, 12);
/* reg_ppcnt_pnat
* Port number access type:
@@ -4981,6 +4986,14 @@ MLXSW_ITEM32(reg, ppcnt, grp, 0x00, 0, 6);
*/
MLXSW_ITEM32(reg, ppcnt, clr, 0x04, 31, 1);
+/* reg_ppcnt_lp_gl
+ * Local port global variable.
+ * 0: local_port 255 = all ports of the device.
+ * 1: local_port indicates local port number for all ports.
+ * Access: OP
+ */
+MLXSW_ITEM32(reg, ppcnt, lp_gl, 0x04, 30, 1);
+
/* reg_ppcnt_prio_tc
* Priority for counter set that support per priority, valid values: 0-7.
* Traffic class for counter set that support per traffic class,
@@ -5404,7 +5417,7 @@ MLXSW_ITEM64(reg, ppcnt, wred_discard,
MLXSW_ITEM64(reg, ppcnt, ecn_marked_tc,
MLXSW_REG_PPCNT_COUNTERS_OFFSET + 0x08, 0, 64);
-static inline void mlxsw_reg_ppcnt_pack(char *payload, u8 local_port,
+static inline void mlxsw_reg_ppcnt_pack(char *payload, u16 local_port,
enum mlxsw_reg_ppcnt_grp grp,
u8 prio_tc)
{
@@ -5414,6 +5427,7 @@ static inline void mlxsw_reg_ppcnt_pack(char *payload, u8 local_port,
mlxsw_reg_ppcnt_pnat_set(payload, 0);
mlxsw_reg_ppcnt_grp_set(payload, grp);
mlxsw_reg_ppcnt_clr_set(payload, 0);
+ mlxsw_reg_ppcnt_lp_gl_set(payload, 1);
mlxsw_reg_ppcnt_prio_tc_set(payload, prio_tc);
}
@@ -5430,7 +5444,7 @@ MLXSW_REG_DEFINE(plib, MLXSW_REG_PLIB_ID, MLXSW_REG_PLIB_LEN);
* Local port number.
* Access: Index
*/
-MLXSW_ITEM32(reg, plib, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, plib, 0x00, 16, 0x00, 12);
/* reg_plib_ib_port
* InfiniBand port remapping for local_port.
@@ -5468,7 +5482,7 @@ MLXSW_ITEM32(reg, pptb, mm, 0x00, 28, 2);
* Local port number.
* Access: Index
*/
-MLXSW_ITEM32(reg, pptb, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, pptb, 0x00, 16, 0x00, 12);
/* reg_pptb_um
* Enables the update of the untagged_buf field.
@@ -5515,7 +5529,7 @@ MLXSW_ITEM_BIT_ARRAY(reg, pptb, prio_to_buff_msb, 0x0C, 0x04, 4);
#define MLXSW_REG_PPTB_ALL_PRIO 0xFF
-static inline void mlxsw_reg_pptb_pack(char *payload, u8 local_port)
+static inline void mlxsw_reg_pptb_pack(char *payload, u16 local_port)
{
MLXSW_REG_ZERO(pptb, payload);
mlxsw_reg_pptb_mm_set(payload, MLXSW_REG_PPTB_MM_UM);
@@ -5545,7 +5559,7 @@ MLXSW_REG_DEFINE(pbmc, MLXSW_REG_PBMC_ID, MLXSW_REG_PBMC_LEN);
* Local port number.
* Access: Index
*/
-MLXSW_ITEM32(reg, pbmc, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, pbmc, 0x00, 16, 0x00, 12);
/* reg_pbmc_xoff_timer_value
* When device generates a pause frame, it uses this value as the pause
@@ -5612,7 +5626,7 @@ MLXSW_ITEM32_INDEXED(reg, pbmc, buf_xoff_threshold, 0x0C, 16, 16,
MLXSW_ITEM32_INDEXED(reg, pbmc, buf_xon_threshold, 0x0C, 0, 16,
0x08, 0x04, false);
-static inline void mlxsw_reg_pbmc_pack(char *payload, u8 local_port,
+static inline void mlxsw_reg_pbmc_pack(char *payload, u16 local_port,
u16 xoff_timer_value, u16 xoff_refresh)
{
MLXSW_REG_ZERO(pbmc, payload);
@@ -5661,7 +5675,7 @@ MLXSW_ITEM32(reg, pspa, swid, 0x00, 24, 8);
* Local port number.
* Access: Index
*/
-MLXSW_ITEM32(reg, pspa, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, pspa, 0x00, 16, 0x00, 0);
/* reg_pspa_sub_port
* Virtual port within the local port. Set to 0 when virtual ports are
@@ -5670,7 +5684,7 @@ MLXSW_ITEM32(reg, pspa, local_port, 0x00, 16, 8);
*/
MLXSW_ITEM32(reg, pspa, sub_port, 0x00, 8, 8);
-static inline void mlxsw_reg_pspa_pack(char *payload, u8 swid, u8 local_port)
+static inline void mlxsw_reg_pspa_pack(char *payload, u8 swid, u16 local_port)
{
MLXSW_REG_ZERO(pspa, payload);
mlxsw_reg_pspa_swid_set(payload, swid);
@@ -5772,7 +5786,7 @@ MLXSW_REG_DEFINE(pplr, MLXSW_REG_PPLR_ID, MLXSW_REG_PPLR_LEN);
* Local port number.
* Access: Index
*/
-MLXSW_ITEM32(reg, pplr, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, pplr, 0x00, 16, 0x00, 12);
/* Phy local loopback. When set the port's egress traffic is looped back
* to the receiver and the port transmitter is disabled.
@@ -5785,7 +5799,7 @@ MLXSW_ITEM32(reg, pplr, local_port, 0x00, 16, 8);
*/
MLXSW_ITEM32(reg, pplr, lb_en, 0x04, 0, 8);
-static inline void mlxsw_reg_pplr_pack(char *payload, u8 local_port,
+static inline void mlxsw_reg_pplr_pack(char *payload, u16 local_port,
bool phy_local)
{
MLXSW_REG_ZERO(pplr, payload);
@@ -5846,7 +5860,7 @@ MLXSW_ITEM32(reg, pmtdb, status, 0x00, 0, 4);
* the module.
* Access: RO
*/
-MLXSW_ITEM16_INDEXED(reg, pmtdb, port_num, 0x04, 0, 8, 0x02, 0x00, false);
+MLXSW_ITEM16_INDEXED(reg, pmtdb, port_num, 0x04, 0, 10, 0x02, 0x00, false);
static inline void mlxsw_reg_pmtdb_pack(char *payload, u8 slot_index, u8 module,
u8 ports_width, u8 num_ports)
@@ -5915,7 +5929,7 @@ MLXSW_REG_DEFINE(pddr, MLXSW_REG_PDDR_ID, MLXSW_REG_PDDR_LEN);
* Local port number.
* Access: Index
*/
-MLXSW_ITEM32(reg, pddr, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, pddr, 0x00, 16, 0x00, 12);
enum mlxsw_reg_pddr_page_select {
MLXSW_REG_PDDR_PAGE_SELECT_TROUBLESHOOTING_INFO = 1,
@@ -5944,7 +5958,7 @@ MLXSW_ITEM32(reg, pddr, trblsh_group_opcode, 0x08, 0, 16);
*/
MLXSW_ITEM32(reg, pddr, trblsh_status_opcode, 0x0C, 0, 16);
-static inline void mlxsw_reg_pddr_pack(char *payload, u8 local_port,
+static inline void mlxsw_reg_pddr_pack(char *payload, u16 local_port,
u8 page_select)
{
MLXSW_REG_ZERO(pddr, payload);
@@ -6014,7 +6028,7 @@ MLXSW_REG_DEFINE(pllp, MLXSW_REG_PLLP_ID, MLXSW_REG_PLLP_LEN);
* Local port number.
* Access: Index
*/
-MLXSW_ITEM32(reg, pllp, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, pllp, 0x00, 16, 0x00, 12);
/* reg_pllp_label_port
* Front panel label of the port.
@@ -6034,7 +6048,7 @@ MLXSW_ITEM32(reg, pllp, split_num, 0x04, 0, 4);
*/
MLXSW_ITEM32(reg, pllp, slot_index, 0x08, 0, 4);
-static inline void mlxsw_reg_pllp_pack(char *payload, u8 local_port)
+static inline void mlxsw_reg_pllp_pack(char *payload, u16 local_port)
{
MLXSW_REG_ZERO(pllp, payload);
mlxsw_reg_pllp_local_port_set(payload, local_port);
@@ -10245,7 +10259,7 @@ MLXSW_REG_DEFINE(mpar, MLXSW_REG_MPAR_ID, MLXSW_REG_MPAR_LEN);
* The local port to mirror the packets from.
* Access: Index
*/
-MLXSW_ITEM32(reg, mpar, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, mpar, 0x00, 16, 0x00, 4);
enum mlxsw_reg_mpar_i_e {
MLXSW_REG_MPAR_TYPE_EGRESS,
@@ -10282,7 +10296,7 @@ MLXSW_ITEM32(reg, mpar, pa_id, 0x04, 0, 4);
*/
MLXSW_ITEM32(reg, mpar, probability_rate, 0x08, 0, 32);
-static inline void mlxsw_reg_mpar_pack(char *payload, u8 local_port,
+static inline void mlxsw_reg_mpar_pack(char *payload, u16 local_port,
enum mlxsw_reg_mpar_i_e i_e,
bool enable, u8 pa_id,
u32 probability_rate)
@@ -10386,7 +10400,7 @@ MLXSW_REG_DEFINE(mlcr, MLXSW_REG_MLCR_ID, MLXSW_REG_MLCR_LEN);
* Local port number.
* Access: RW
*/
-MLXSW_ITEM32(reg, mlcr, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, mlcr, 0x00, 16, 0x00, 24);
#define MLXSW_REG_MLCR_DURATION_MAX 0xFFFF
@@ -10405,7 +10419,7 @@ MLXSW_ITEM32(reg, mlcr, beacon_duration, 0x04, 0, 16);
*/
MLXSW_ITEM32(reg, mlcr, beacon_remain, 0x08, 0, 16);
-static inline void mlxsw_reg_mlcr_pack(char *payload, u8 local_port,
+static inline void mlxsw_reg_mlcr_pack(char *payload, u16 local_port,
bool active)
{
MLXSW_REG_ZERO(mlcr, payload);
@@ -10778,7 +10792,7 @@ MLXSW_REG_DEFINE(mpsc, MLXSW_REG_MPSC_ID, MLXSW_REG_MPSC_LEN);
* Not supported for CPU port
* Access: Index
*/
-MLXSW_ITEM32(reg, mpsc, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, mpsc, 0x00, 16, 0x00, 12);
/* reg_mpsc_e
* Enable sampling on port local_port
@@ -10795,7 +10809,7 @@ MLXSW_ITEM32(reg, mpsc, e, 0x04, 30, 1);
*/
MLXSW_ITEM32(reg, mpsc, rate, 0x08, 0, 32);
-static inline void mlxsw_reg_mpsc_pack(char *payload, u8 local_port, bool e,
+static inline void mlxsw_reg_mpsc_pack(char *payload, u16 local_port, bool e,
u32 rate)
{
MLXSW_REG_ZERO(mpsc, payload);
@@ -11003,7 +11017,7 @@ MLXSW_REG_DEFINE(momte, MLXSW_REG_MOMTE_ID, MLXSW_REG_MOMTE_LEN);
* Local port number.
* Access: Index
*/
-MLXSW_ITEM32(reg, momte, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, momte, 0x00, 16, 0x00, 12);
enum mlxsw_reg_momte_type {
MLXSW_REG_MOMTE_TYPE_WRED = 0x20,
@@ -11030,7 +11044,7 @@ MLXSW_ITEM32(reg, momte, type, 0x04, 0, 8);
*/
MLXSW_ITEM_BIT_ARRAY(reg, momte, tclass_en, 0x08, 0x08, 1);
-static inline void mlxsw_reg_momte_pack(char *payload, u8 local_port,
+static inline void mlxsw_reg_momte_pack(char *payload, u16 local_port,
enum mlxsw_reg_momte_type type)
{
MLXSW_REG_ZERO(momte, payload);
@@ -11098,7 +11112,7 @@ MLXSW_REG_DEFINE(mtpptr, MLXSW_REG_MTPPTR_ID, MLXSW_REG_MTPPTR_LEN);
* Not supported for CPU port.
* Access: Index
*/
-MLXSW_ITEM32(reg, mtpptr, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, mtpptr, 0x00, 16, 0x00, 12);
enum mlxsw_reg_mtpptr_dir {
MLXSW_REG_MTPPTR_DIR_INGRESS,
@@ -11305,7 +11319,7 @@ mlxsw_reg_mgpir_unpack(char *payload, u8 *num_of_devices,
* -----------------------------------
*/
#define MLXSW_REG_MFDE_ID 0x9200
-#define MLXSW_REG_MFDE_LEN 0x18
+#define MLXSW_REG_MFDE_LEN 0x30
MLXSW_REG_DEFINE(mfde, MLXSW_REG_MFDE_ID, MLXSW_REG_MFDE_LEN);
@@ -11315,10 +11329,32 @@ MLXSW_REG_DEFINE(mfde, MLXSW_REG_MFDE_ID, MLXSW_REG_MFDE_LEN);
*/
MLXSW_ITEM32(reg, mfde, irisc_id, 0x00, 24, 8);
+enum mlxsw_reg_mfde_severity {
+ /* Unrecoverable switch behavior */
+ MLXSW_REG_MFDE_SEVERITY_FATL = 2,
+ /* Unexpected state with possible systemic failure */
+ MLXSW_REG_MFDE_SEVERITY_NRML = 3,
+ /* Unexpected state without systemic failure */
+ MLXSW_REG_MFDE_SEVERITY_INTR = 5,
+};
+
+/* reg_mfde_severity
+ * The severity of the event.
+ * Access: RO
+ */
+MLXSW_ITEM32(reg, mfde, severity, 0x00, 16, 8);
+
enum mlxsw_reg_mfde_event_id {
+ /* CRspace timeout */
MLXSW_REG_MFDE_EVENT_ID_CRSPACE_TO = 1,
/* KVD insertion machine stopped */
MLXSW_REG_MFDE_EVENT_ID_KVD_IM_STOP,
+ /* Triggered by MFGD.trigger_test */
+ MLXSW_REG_MFDE_EVENT_ID_TEST,
+ /* Triggered when firmware hits an assert */
+ MLXSW_REG_MFDE_EVENT_ID_FW_ASSERT,
+ /* Fatal error interrupt from hardware */
+ MLXSW_REG_MFDE_EVENT_ID_FATAL_CAUSE,
};
/* reg_mfde_event_id
@@ -11359,32 +11395,110 @@ MLXSW_ITEM32(reg, mfde, command_type, 0x04, 24, 2);
*/
MLXSW_ITEM32(reg, mfde, reg_attr_id, 0x04, 0, 16);
-/* reg_mfde_log_address
+/* reg_mfde_crspace_to_log_address
* crspace address accessed, which resulted in timeout.
- * Valid in case event_id == MLXSW_REG_MFDE_EVENT_ID_CRSPACE_TO
* Access: RO
*/
-MLXSW_ITEM32(reg, mfde, log_address, 0x10, 0, 32);
+MLXSW_ITEM32(reg, mfde, crspace_to_log_address, 0x10, 0, 32);
-/* reg_mfde_log_id
+/* reg_mfde_crspace_to_oe
+ * 0 - New event
+ * 1 - Old event, occurred before MFGD activation.
+ * Access: RO
+ */
+MLXSW_ITEM32(reg, mfde, crspace_to_oe, 0x14, 24, 1);
+
+/* reg_mfde_crspace_to_log_id
* Which irisc triggered the timeout.
- * Valid in case event_id == MLXSW_REG_MFDE_EVENT_ID_CRSPACE_TO
* Access: RO
*/
-MLXSW_ITEM32(reg, mfde, log_id, 0x14, 0, 4);
+MLXSW_ITEM32(reg, mfde, crspace_to_log_id, 0x14, 0, 4);
-/* reg_mfde_log_ip
+/* reg_mfde_crspace_to_log_ip
* IP (instruction pointer) that triggered the timeout.
- * Valid in case event_id == MLXSW_REG_MFDE_EVENT_ID_CRSPACE_TO
* Access: RO
*/
-MLXSW_ITEM64(reg, mfde, log_ip, 0x18, 0, 64);
+MLXSW_ITEM64(reg, mfde, crspace_to_log_ip, 0x18, 0, 64);
-/* reg_mfde_pipes_mask
+/* reg_mfde_kvd_im_stop_oe
+ * 0 - New event
+ * 1 - Old event, occurred before MFGD activation.
+ * Access: RO
+ */
+MLXSW_ITEM32(reg, mfde, kvd_im_stop_oe, 0x10, 24, 1);
+
+/* reg_mfde_kvd_im_stop_pipes_mask
* Bit per kvh pipe.
* Access: RO
*/
-MLXSW_ITEM32(reg, mfde, pipes_mask, 0x10, 0, 16);
+MLXSW_ITEM32(reg, mfde, kvd_im_stop_pipes_mask, 0x10, 0, 16);
+
+/* reg_mfde_fw_assert_var0-4
+ * Variables passed to assert.
+ * Access: RO
+ */
+MLXSW_ITEM32(reg, mfde, fw_assert_var0, 0x10, 0, 32);
+MLXSW_ITEM32(reg, mfde, fw_assert_var1, 0x14, 0, 32);
+MLXSW_ITEM32(reg, mfde, fw_assert_var2, 0x18, 0, 32);
+MLXSW_ITEM32(reg, mfde, fw_assert_var3, 0x1C, 0, 32);
+MLXSW_ITEM32(reg, mfde, fw_assert_var4, 0x20, 0, 32);
+
+/* reg_mfde_fw_assert_existptr
+ * The instruction pointer when assert was triggered.
+ * Access: RO
+ */
+MLXSW_ITEM32(reg, mfde, fw_assert_existptr, 0x24, 0, 32);
+
+/* reg_mfde_fw_assert_callra
+ * The return address after triggering assert.
+ * Access: RO
+ */
+MLXSW_ITEM32(reg, mfde, fw_assert_callra, 0x28, 0, 32);
+
+/* reg_mfde_fw_assert_oe
+ * 0 - New event
+ * 1 - Old event, occurred before MFGD activation.
+ * Access: RO
+ */
+MLXSW_ITEM32(reg, mfde, fw_assert_oe, 0x2C, 24, 1);
+
+/* reg_mfde_fw_assert_tile_v
+ * 0: The assert was from main
+ * 1: The assert was from a tile
+ * Access: RO
+ */
+MLXSW_ITEM32(reg, mfde, fw_assert_tile_v, 0x2C, 23, 1);
+
+/* reg_mfde_fw_assert_tile_index
+ * When tile_v=1, the tile_index that caused the assert.
+ * Access: RO
+ */
+MLXSW_ITEM32(reg, mfde, fw_assert_tile_index, 0x2C, 16, 6);
+
+/* reg_mfde_fw_assert_ext_synd
+ * A generated one-to-one identifier which is specific per-assert.
+ * Access: RO
+ */
+MLXSW_ITEM32(reg, mfde, fw_assert_ext_synd, 0x2C, 0, 16);
+
+/* reg_mfde_fatal_cause_id
+ * HW interrupt cause id.
+ * Access: RO
+ */
+MLXSW_ITEM32(reg, mfde, fatal_cause_id, 0x10, 0, 18);
+
+/* reg_mfde_fatal_cause_tile_v
+ * 0: The assert was from main
+ * 1: The assert was from a tile
+ * Access: RO
+ */
+MLXSW_ITEM32(reg, mfde, fatal_cause_tile_v, 0x14, 23, 1);
+
+/* reg_mfde_fatal_cause_tile_index
+ * When tile_v=1, the tile_index that caused the assert.
+ * Access: RO
+ */
+MLXSW_ITEM32(reg, mfde, fatal_cause_tile_index, 0x14, 16, 6);
/* TNGCR - Tunneling NVE General Configuration Register
* ----------------------------------------------------
@@ -11692,7 +11806,7 @@ MLXSW_REG_DEFINE(tnqdr, MLXSW_REG_TNQDR_ID, MLXSW_REG_TNQDR_LEN);
* Local port number (receive port). CPU port is supported.
* Access: Index
*/
-MLXSW_ITEM32(reg, tnqdr, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, tnqdr, 0x00, 16, 0x00, 12);
/* reg_tnqdr_dscp
* For encapsulation, the default DSCP.
@@ -11700,7 +11814,7 @@ MLXSW_ITEM32(reg, tnqdr, local_port, 0x00, 16, 8);
*/
MLXSW_ITEM32(reg, tnqdr, dscp, 0x04, 0, 6);
-static inline void mlxsw_reg_tnqdr_pack(char *payload, u8 local_port)
+static inline void mlxsw_reg_tnqdr_pack(char *payload, u16 local_port)
{
MLXSW_REG_ZERO(tnqdr, payload);
mlxsw_reg_tnqdr_local_port_set(payload, local_port);
@@ -12028,7 +12142,7 @@ MLXSW_REG_DEFINE(sbcm, MLXSW_REG_SBCM_ID, MLXSW_REG_SBCM_LEN);
* For Egress: excludes IP Router
* Access: Index
*/
-MLXSW_ITEM32(reg, sbcm, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, sbcm, 0x00, 16, 0x00, 4);
/* reg_sbcm_pg_buff
* PG buffer - Port PG (dir=ingress) / traffic class (dir=egress)
@@ -12082,7 +12196,7 @@ MLXSW_ITEM32(reg, sbcm, max_buff, 0x1C, 0, 24);
*/
MLXSW_ITEM32(reg, sbcm, pool, 0x24, 0, 4);
-static inline void mlxsw_reg_sbcm_pack(char *payload, u8 local_port, u8 pg_buff,
+static inline void mlxsw_reg_sbcm_pack(char *payload, u16 local_port, u8 pg_buff,
enum mlxsw_reg_sbxx_dir dir,
u32 min_buff, u32 max_buff,
bool infi_max, u8 pool)
@@ -12114,7 +12228,7 @@ MLXSW_REG_DEFINE(sbpm, MLXSW_REG_SBPM_ID, MLXSW_REG_SBPM_LEN);
* For Egress: excludes IP Router
* Access: Index
*/
-MLXSW_ITEM32(reg, sbpm, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, sbpm, 0x00, 16, 0x00, 12);
/* reg_sbpm_pool
* The pool associated to quota counting on the local_port.
@@ -12168,7 +12282,7 @@ MLXSW_ITEM32(reg, sbpm, min_buff, 0x18, 0, 24);
*/
MLXSW_ITEM32(reg, sbpm, max_buff, 0x1C, 0, 24);
-static inline void mlxsw_reg_sbpm_pack(char *payload, u8 local_port, u8 pool,
+static inline void mlxsw_reg_sbpm_pack(char *payload, u16 local_port, u8 pool,
enum mlxsw_reg_sbxx_dir dir, bool clr,
u32 min_buff, u32 max_buff)
{
@@ -12266,6 +12380,16 @@ MLXSW_REG_DEFINE(sbsr, MLXSW_REG_SBSR_ID, MLXSW_REG_SBSR_LEN);
*/
MLXSW_ITEM32(reg, sbsr, clr, 0x00, 31, 1);
+#define MLXSW_REG_SBSR_NUM_PORTS_IN_PAGE 256
+
+/* reg_sbsr_port_page
+ * Determines the range of the ports specified in the 'ingress_port_mask'
+ * and 'egress_port_mask' bit masks.
+ * {ingress,egress}_port_mask[x] is (256 * port_page) + x
+ * Access: Index
+ */
+MLXSW_ITEM32(reg, sbsr, port_page, 0x04, 0, 4);
+
/* reg_sbsr_ingress_port_mask
* Bit vector for all ingress network ports.
* Indicates which of the ports (for which the relevant bit is set)
@@ -12353,7 +12477,7 @@ MLXSW_REG_DEFINE(sbib, MLXSW_REG_SBIB_ID, MLXSW_REG_SBIB_LEN);
* Not supported for CPU port and router port
* Access: Index
*/
-MLXSW_ITEM32(reg, sbib, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, sbib, 0x00, 16, 0x00, 12);
/* reg_sbib_buff_size
* Units represented in cells
@@ -12363,7 +12487,7 @@ MLXSW_ITEM32(reg, sbib, local_port, 0x00, 16, 8);
*/
MLXSW_ITEM32(reg, sbib, buff_size, 0x08, 0, 24);
-static inline void mlxsw_reg_sbib_pack(char *payload, u8 local_port,
+static inline void mlxsw_reg_sbib_pack(char *payload, u16 local_port,
u32 buff_size)
{
MLXSW_REG_ZERO(sbib, payload);
@@ -12374,7 +12498,6 @@ static inline void mlxsw_reg_sbib_pack(char *payload, u8 local_port,
static const struct mlxsw_reg_info *mlxsw_reg_infos[] = {
MLXSW_REG(sgcr),
MLXSW_REG(spad),
- MLXSW_REG(smid),
MLXSW_REG(sspr),
MLXSW_REG(sfdat),
MLXSW_REG(sfd),
@@ -12384,7 +12507,6 @@ static const struct mlxsw_reg_info *mlxsw_reg_infos[] = {
MLXSW_REG(spvm),
MLXSW_REG(spaft),
MLXSW_REG(sfgc),
- MLXSW_REG(sftr),
MLXSW_REG(sfdf),
MLXSW_REG(sldr),
MLXSW_REG(slcr),
@@ -12397,6 +12519,8 @@ static const struct mlxsw_reg_info *mlxsw_reg_infos[] = {
MLXSW_REG(spvmlr),
MLXSW_REG(spvc),
MLXSW_REG(spevet),
+ MLXSW_REG(sftr2),
+ MLXSW_REG(smid2),
MLXSW_REG(cwtp),
MLXSW_REG(cwtpm),
MLXSW_REG(pgcr),
@@ -12556,7 +12680,7 @@ MLXSW_ITEM32(reg, pude, swid, 0x00, 24, 8);
* Local port number.
* Access: Index
*/
-MLXSW_ITEM32(reg, pude, local_port, 0x00, 16, 8);
+MLXSW_ITEM32_LP(reg, pude, 0x00, 16, 0x00, 12);
/* reg_pude_admin_status
* Port administrative state (the desired state).
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum.c
index 03e5bad..aa411de 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum.c
@@ -46,8 +46,8 @@
#include "spectrum_trap.h"
#define MLXSW_SP1_FWREV_MAJOR 13
-#define MLXSW_SP1_FWREV_MINOR 2008
-#define MLXSW_SP1_FWREV_SUBMINOR 3326
+#define MLXSW_SP1_FWREV_MINOR 2010
+#define MLXSW_SP1_FWREV_SUBMINOR 1006
#define MLXSW_SP1_FWREV_CAN_RESET_MINOR 1702
static const struct mlxsw_fw_rev mlxsw_sp1_fw_rev = {
@@ -63,8 +63,8 @@ static const struct mlxsw_fw_rev mlxsw_sp1_fw_rev = {
"." __stringify(MLXSW_SP1_FWREV_SUBMINOR) ".mfa2"
#define MLXSW_SP2_FWREV_MAJOR 29
-#define MLXSW_SP2_FWREV_MINOR 2008
-#define MLXSW_SP2_FWREV_SUBMINOR 3326
+#define MLXSW_SP2_FWREV_MINOR 2010
+#define MLXSW_SP2_FWREV_SUBMINOR 1006
static const struct mlxsw_fw_rev mlxsw_sp2_fw_rev = {
.major = MLXSW_SP2_FWREV_MAJOR,
@@ -78,8 +78,8 @@ static const struct mlxsw_fw_rev mlxsw_sp2_fw_rev = {
"." __stringify(MLXSW_SP2_FWREV_SUBMINOR) ".mfa2"
#define MLXSW_SP3_FWREV_MAJOR 30
-#define MLXSW_SP3_FWREV_MINOR 2008
-#define MLXSW_SP3_FWREV_SUBMINOR 3326
+#define MLXSW_SP3_FWREV_MINOR 2010
+#define MLXSW_SP3_FWREV_SUBMINOR 1006
static const struct mlxsw_fw_rev mlxsw_sp3_fw_rev = {
.major = MLXSW_SP3_FWREV_MAJOR,
@@ -95,6 +95,7 @@ static const struct mlxsw_fw_rev mlxsw_sp3_fw_rev = {
static const char mlxsw_sp1_driver_name[] = "mlxsw_spectrum";
static const char mlxsw_sp2_driver_name[] = "mlxsw_spectrum2";
static const char mlxsw_sp3_driver_name[] = "mlxsw_spectrum3";
+static const char mlxsw_sp4_driver_name[] = "mlxsw_spectrum4";
static const unsigned char mlxsw_sp1_mac_mask[ETH_ALEN] = {
0xff, 0xff, 0xff, 0xff, 0xfc, 0x00
@@ -303,7 +304,7 @@ int mlxsw_sp_port_admin_status_set(struct mlxsw_sp_port *mlxsw_sp_port,
}
static int mlxsw_sp_port_dev_addr_set(struct mlxsw_sp_port *mlxsw_sp_port,
- unsigned char *addr)
+ const unsigned char *addr)
{
struct mlxsw_sp *mlxsw_sp = mlxsw_sp_port->mlxsw_sp;
char ppad_pl[MLXSW_REG_PPAD_LEN];
@@ -352,7 +353,7 @@ static int mlxsw_sp_port_mtu_set(struct mlxsw_sp_port *mlxsw_sp_port, u16 mtu)
}
static int mlxsw_sp_port_swid_set(struct mlxsw_sp *mlxsw_sp,
- u8 local_port, u8 swid)
+ u16 local_port, u8 swid)
{
char pspa_pl[MLXSW_REG_PSPA_LEN];
@@ -483,7 +484,7 @@ mlxsw_sp_port_system_port_mapping_set(struct mlxsw_sp_port *mlxsw_sp_port)
}
static int
-mlxsw_sp_port_module_info_get(struct mlxsw_sp *mlxsw_sp, u8 local_port,
+mlxsw_sp_port_module_info_get(struct mlxsw_sp *mlxsw_sp, u16 local_port,
struct mlxsw_sp_port_mapping *port_mapping)
{
char pmlp_pl[MLXSW_REG_PMLP_LEN];
@@ -535,7 +536,7 @@ mlxsw_sp_port_module_info_get(struct mlxsw_sp *mlxsw_sp, u8 local_port,
}
static int
-mlxsw_sp_port_module_map(struct mlxsw_sp *mlxsw_sp, u8 local_port,
+mlxsw_sp_port_module_map(struct mlxsw_sp *mlxsw_sp, u16 local_port,
const struct mlxsw_sp_port_mapping *port_mapping)
{
char pmlp_pl[MLXSW_REG_PMLP_LEN];
@@ -560,7 +561,7 @@ mlxsw_sp_port_module_map(struct mlxsw_sp *mlxsw_sp, u8 local_port,
return err;
}
-static void mlxsw_sp_port_module_unmap(struct mlxsw_sp *mlxsw_sp, u8 local_port,
+static void mlxsw_sp_port_module_unmap(struct mlxsw_sp *mlxsw_sp, u16 local_port,
u8 module)
{
char pmlp_pl[MLXSW_REG_PMLP_LEN];
@@ -1474,7 +1475,7 @@ mlxsw_sp_port_vlan_classification_set(struct mlxsw_sp_port *mlxsw_sp_port,
}
static int mlxsw_sp_port_label_info_get(struct mlxsw_sp *mlxsw_sp,
- u8 local_port, u8 *port_number,
+ u16 local_port, u8 *port_number,
u8 *split_port_subnumber,
u8 *slot_index)
{
@@ -1490,7 +1491,7 @@ static int mlxsw_sp_port_label_info_get(struct mlxsw_sp *mlxsw_sp,
return 0;
}
-static int mlxsw_sp_port_create(struct mlxsw_sp *mlxsw_sp, u8 local_port,
+static int mlxsw_sp_port_create(struct mlxsw_sp *mlxsw_sp, u16 local_port,
bool split,
struct mlxsw_sp_port_mapping *port_mapping)
{
@@ -1781,7 +1782,7 @@ static int mlxsw_sp_port_create(struct mlxsw_sp *mlxsw_sp, u8 local_port,
return err;
}
-static void mlxsw_sp_port_remove(struct mlxsw_sp *mlxsw_sp, u8 local_port)
+static void mlxsw_sp_port_remove(struct mlxsw_sp *mlxsw_sp, u16 local_port)
{
struct mlxsw_sp_port *mlxsw_sp_port = mlxsw_sp->ports[local_port];
u8 module = mlxsw_sp_port->mapping.module;
@@ -1848,12 +1849,12 @@ static void mlxsw_sp_cpu_port_remove(struct mlxsw_sp *mlxsw_sp)
kfree(mlxsw_sp_port);
}
-static bool mlxsw_sp_local_port_valid(u8 local_port)
+static bool mlxsw_sp_local_port_valid(u16 local_port)
{
return local_port != MLXSW_PORT_CPU_PORT;
}
-static bool mlxsw_sp_port_created(struct mlxsw_sp *mlxsw_sp, u8 local_port)
+static bool mlxsw_sp_port_created(struct mlxsw_sp *mlxsw_sp, u16 local_port)
{
if (!mlxsw_sp_local_port_valid(local_port))
return false;
@@ -1971,7 +1972,7 @@ mlxsw_sp_port_split_create(struct mlxsw_sp *mlxsw_sp,
split_port_mapping = *port_mapping;
split_port_mapping.width /= count;
for (i = 0; i < count; i++) {
- u8 s_local_port = mlxsw_reg_pmtdb_port_num_get(pmtdb_pl, i);
+ u16 s_local_port = mlxsw_reg_pmtdb_port_num_get(pmtdb_pl, i);
if (!mlxsw_sp_local_port_valid(s_local_port))
continue;
@@ -1987,7 +1988,7 @@ mlxsw_sp_port_split_create(struct mlxsw_sp *mlxsw_sp,
err_port_create:
for (i--; i >= 0; i--) {
- u8 s_local_port = mlxsw_reg_pmtdb_port_num_get(pmtdb_pl, i);
+ u16 s_local_port = mlxsw_reg_pmtdb_port_num_get(pmtdb_pl, i);
if (mlxsw_sp_port_created(mlxsw_sp, s_local_port))
mlxsw_sp_port_remove(mlxsw_sp, s_local_port);
@@ -2004,7 +2005,7 @@ static void mlxsw_sp_port_unsplit_create(struct mlxsw_sp *mlxsw_sp,
/* Go over original unsplit ports in the gap and recreate them. */
for (i = 0; i < count; i++) {
- u8 local_port = mlxsw_reg_pmtdb_port_num_get(pmtdb_pl, i);
+ u16 local_port = mlxsw_reg_pmtdb_port_num_get(pmtdb_pl, i);
port_mapping = mlxsw_sp->port_mapping[local_port];
if (!port_mapping || !mlxsw_sp_local_port_valid(local_port))
@@ -2015,14 +2016,14 @@ static void mlxsw_sp_port_unsplit_create(struct mlxsw_sp *mlxsw_sp,
}
static struct mlxsw_sp_port *
-mlxsw_sp_port_get_by_local_port(struct mlxsw_sp *mlxsw_sp, u8 local_port)
+mlxsw_sp_port_get_by_local_port(struct mlxsw_sp *mlxsw_sp, u16 local_port)
{
if (mlxsw_sp->ports && mlxsw_sp->ports[local_port])
return mlxsw_sp->ports[local_port];
return NULL;
}
-static int mlxsw_sp_port_split(struct mlxsw_core *mlxsw_core, u8 local_port,
+static int mlxsw_sp_port_split(struct mlxsw_core *mlxsw_core, u16 local_port,
unsigned int count,
struct netlink_ext_ack *extack)
{
@@ -2065,7 +2066,7 @@ static int mlxsw_sp_port_split(struct mlxsw_core *mlxsw_core, u8 local_port,
port_mapping = mlxsw_sp_port->mapping;
for (i = 0; i < count; i++) {
- u8 s_local_port = mlxsw_reg_pmtdb_port_num_get(pmtdb_pl, i);
+ u16 s_local_port = mlxsw_reg_pmtdb_port_num_get(pmtdb_pl, i);
if (mlxsw_sp_port_created(mlxsw_sp, s_local_port))
mlxsw_sp_port_remove(mlxsw_sp, s_local_port);
@@ -2085,7 +2086,7 @@ static int mlxsw_sp_port_split(struct mlxsw_core *mlxsw_core, u8 local_port,
return err;
}
-static int mlxsw_sp_port_unsplit(struct mlxsw_core *mlxsw_core, u8 local_port,
+static int mlxsw_sp_port_unsplit(struct mlxsw_core *mlxsw_core, u16 local_port,
struct netlink_ext_ack *extack)
{
struct mlxsw_sp *mlxsw_sp = mlxsw_core_driver_priv(mlxsw_core);
@@ -2121,7 +2122,7 @@ static int mlxsw_sp_port_unsplit(struct mlxsw_core *mlxsw_core, u8 local_port,
}
for (i = 0; i < count; i++) {
- u8 s_local_port = mlxsw_reg_pmtdb_port_num_get(pmtdb_pl, i);
+ u16 s_local_port = mlxsw_reg_pmtdb_port_num_get(pmtdb_pl, i);
if (mlxsw_sp_port_created(mlxsw_sp, s_local_port))
mlxsw_sp_port_remove(mlxsw_sp, s_local_port);
@@ -2148,7 +2149,7 @@ static void mlxsw_sp_pude_event_func(const struct mlxsw_reg_info *reg,
struct mlxsw_sp_port *mlxsw_sp_port;
enum mlxsw_reg_pude_oper_status status;
unsigned int max_ports;
- u8 local_port;
+ u16 local_port;
max_ports = mlxsw_core_max_ports(mlxsw_sp->core);
local_port = mlxsw_reg_pude_local_port_get(pude_pl);
@@ -2174,7 +2175,7 @@ static void mlxsw_sp_pude_event_func(const struct mlxsw_reg_info *reg,
static void mlxsw_sp1_ptp_fifo_event_func(struct mlxsw_sp *mlxsw_sp,
char *mtpptr_pl, bool ingress)
{
- u8 local_port;
+ u16 local_port;
u8 num_rec;
int i;
@@ -2212,7 +2213,7 @@ static void mlxsw_sp1_ptp_egr_fifo_event_func(const struct mlxsw_reg_info *reg,
}
void mlxsw_sp_rx_listener_no_mark_func(struct sk_buff *skb,
- u8 local_port, void *priv)
+ u16 local_port, void *priv)
{
struct mlxsw_sp *mlxsw_sp = priv;
struct mlxsw_sp_port *mlxsw_sp_port = mlxsw_sp->ports[local_port];
@@ -2236,7 +2237,7 @@ void mlxsw_sp_rx_listener_no_mark_func(struct sk_buff *skb,
netif_receive_skb(skb);
}
-static void mlxsw_sp_rx_listener_mark_func(struct sk_buff *skb, u8 local_port,
+static void mlxsw_sp_rx_listener_mark_func(struct sk_buff *skb, u16 local_port,
void *priv)
{
skb->offload_fwd_mark = 1;
@@ -2244,7 +2245,7 @@ static void mlxsw_sp_rx_listener_mark_func(struct sk_buff *skb, u8 local_port,
}
static void mlxsw_sp_rx_listener_l3_mark_func(struct sk_buff *skb,
- u8 local_port, void *priv)
+ u16 local_port, void *priv)
{
skb->offload_l3_fwd_mark = 1;
skb->offload_fwd_mark = 1;
@@ -2252,7 +2253,7 @@ static void mlxsw_sp_rx_listener_l3_mark_func(struct sk_buff *skb,
}
void mlxsw_sp_ptp_receive(struct mlxsw_sp *mlxsw_sp, struct sk_buff *skb,
- u8 local_port)
+ u16 local_port)
{
mlxsw_sp->ptp_ops->receive(mlxsw_sp, skb, local_port);
}
@@ -2755,6 +2756,140 @@ static void mlxsw_sp_parsing_fini(struct mlxsw_sp *mlxsw_sp)
mutex_destroy(&mlxsw_sp->parsing.lock);
}
+struct mlxsw_sp_ipv6_addr_node {
+ struct in6_addr key;
+ struct rhash_head ht_node;
+ u32 kvdl_index;
+ refcount_t refcount;
+};
+
+static const struct rhashtable_params mlxsw_sp_ipv6_addr_ht_params = {
+ .key_offset = offsetof(struct mlxsw_sp_ipv6_addr_node, key),
+ .head_offset = offsetof(struct mlxsw_sp_ipv6_addr_node, ht_node),
+ .key_len = sizeof(struct in6_addr),
+ .automatic_shrinking = true,
+};
+
+static int
+mlxsw_sp_ipv6_addr_init(struct mlxsw_sp *mlxsw_sp, const struct in6_addr *addr6,
+ u32 *p_kvdl_index)
+{
+ struct mlxsw_sp_ipv6_addr_node *node;
+ char rips_pl[MLXSW_REG_RIPS_LEN];
+ int err;
+
+ err = mlxsw_sp_kvdl_alloc(mlxsw_sp,
+ MLXSW_SP_KVDL_ENTRY_TYPE_IPV6_ADDRESS, 1,
+ p_kvdl_index);
+ if (err)
+ return err;
+
+ mlxsw_reg_rips_pack(rips_pl, *p_kvdl_index, addr6);
+ err = mlxsw_reg_write(mlxsw_sp->core, MLXSW_REG(rips), rips_pl);
+ if (err)
+ goto err_rips_write;
+
+ node = kzalloc(sizeof(*node), GFP_KERNEL);
+ if (!node) {
+ err = -ENOMEM;
+ goto err_node_alloc;
+ }
+
+ node->key = *addr6;
+ node->kvdl_index = *p_kvdl_index;
+ refcount_set(&node->refcount, 1);
+
+ err = rhashtable_insert_fast(&mlxsw_sp->ipv6_addr_ht,
+ &node->ht_node,
+ mlxsw_sp_ipv6_addr_ht_params);
+ if (err)
+ goto err_rhashtable_insert;
+
+ return 0;
+
+err_rhashtable_insert:
+ kfree(node);
+err_node_alloc:
+err_rips_write:
+ mlxsw_sp_kvdl_free(mlxsw_sp, MLXSW_SP_KVDL_ENTRY_TYPE_IPV6_ADDRESS, 1,
+ *p_kvdl_index);
+ return err;
+}
+
+static void mlxsw_sp_ipv6_addr_fini(struct mlxsw_sp *mlxsw_sp,
+ struct mlxsw_sp_ipv6_addr_node *node)
+{
+ u32 kvdl_index = node->kvdl_index;
+
+ rhashtable_remove_fast(&mlxsw_sp->ipv6_addr_ht, &node->ht_node,
+ mlxsw_sp_ipv6_addr_ht_params);
+ kfree(node);
+ mlxsw_sp_kvdl_free(mlxsw_sp, MLXSW_SP_KVDL_ENTRY_TYPE_IPV6_ADDRESS, 1,
+ kvdl_index);
+}
+
+int mlxsw_sp_ipv6_addr_kvdl_index_get(struct mlxsw_sp *mlxsw_sp,
+ const struct in6_addr *addr6,
+ u32 *p_kvdl_index)
+{
+ struct mlxsw_sp_ipv6_addr_node *node;
+ int err = 0;
+
+ mutex_lock(&mlxsw_sp->ipv6_addr_ht_lock);
+ node = rhashtable_lookup_fast(&mlxsw_sp->ipv6_addr_ht, addr6,
+ mlxsw_sp_ipv6_addr_ht_params);
+ if (node) {
+ refcount_inc(&node->refcount);
+ *p_kvdl_index = node->kvdl_index;
+ goto out_unlock;
+ }
+
+ err = mlxsw_sp_ipv6_addr_init(mlxsw_sp, addr6, p_kvdl_index);
+
+out_unlock:
+ mutex_unlock(&mlxsw_sp->ipv6_addr_ht_lock);
+ return err;
+}
+
+void
+mlxsw_sp_ipv6_addr_put(struct mlxsw_sp *mlxsw_sp, const struct in6_addr *addr6)
+{
+ struct mlxsw_sp_ipv6_addr_node *node;
+
+ mutex_lock(&mlxsw_sp->ipv6_addr_ht_lock);
+ node = rhashtable_lookup_fast(&mlxsw_sp->ipv6_addr_ht, addr6,
+ mlxsw_sp_ipv6_addr_ht_params);
+ if (WARN_ON(!node))
+ goto out_unlock;
+
+ if (!refcount_dec_and_test(&node->refcount))
+ goto out_unlock;
+
+ mlxsw_sp_ipv6_addr_fini(mlxsw_sp, node);
+
+out_unlock:
+ mutex_unlock(&mlxsw_sp->ipv6_addr_ht_lock);
+}
+
+static int mlxsw_sp_ipv6_addr_ht_init(struct mlxsw_sp *mlxsw_sp)
+{
+ int err;
+
+ err = rhashtable_init(&mlxsw_sp->ipv6_addr_ht,
+ &mlxsw_sp_ipv6_addr_ht_params);
+ if (err)
+ return err;
+
+ mutex_init(&mlxsw_sp->ipv6_addr_ht_lock);
+ return 0;
+}
+
+static void mlxsw_sp_ipv6_addr_ht_fini(struct mlxsw_sp *mlxsw_sp)
+{
+ mutex_destroy(&mlxsw_sp->ipv6_addr_ht_lock);
+ rhashtable_destroy(&mlxsw_sp->ipv6_addr_ht);
+}
+
static int mlxsw_sp_init(struct mlxsw_core *mlxsw_core,
const struct mlxsw_bus_info *mlxsw_bus_info,
struct netlink_ext_ack *extack)
@@ -2843,6 +2978,12 @@ static int mlxsw_sp_init(struct mlxsw_core *mlxsw_core,
goto err_afa_init;
}
+ err = mlxsw_sp_ipv6_addr_ht_init(mlxsw_sp);
+ if (err) {
+ dev_err(mlxsw_sp->bus_info->dev, "Failed to initialize hash table for IPv6 addresses\n");
+ goto err_ipv6_addr_ht_init;
+ }
+
err = mlxsw_sp_nve_init(mlxsw_sp);
if (err) {
dev_err(mlxsw_sp->bus_info->dev, "Failed to initialize NVE\n");
@@ -2944,6 +3085,8 @@ static int mlxsw_sp_init(struct mlxsw_core *mlxsw_core,
err_acl_init:
mlxsw_sp_nve_fini(mlxsw_sp);
err_nve_init:
+ mlxsw_sp_ipv6_addr_ht_fini(mlxsw_sp);
+err_ipv6_addr_ht_init:
mlxsw_sp_afa_fini(mlxsw_sp);
err_afa_init:
mlxsw_sp_counter_pool_fini(mlxsw_sp);
@@ -3013,6 +3156,7 @@ static int mlxsw_sp2_init(struct mlxsw_core *mlxsw_core,
mlxsw_sp->mr_tcam_ops = &mlxsw_sp2_mr_tcam_ops;
mlxsw_sp->acl_rulei_ops = &mlxsw_sp2_acl_rulei_ops;
mlxsw_sp->acl_tcam_ops = &mlxsw_sp2_acl_tcam_ops;
+ mlxsw_sp->acl_bf_ops = &mlxsw_sp2_acl_bf_ops;
mlxsw_sp->nve_ops_arr = mlxsw_sp2_nve_ops_arr;
mlxsw_sp->mac_mask = mlxsw_sp2_mac_mask;
mlxsw_sp->sb_vals = &mlxsw_sp2_sb_vals;
@@ -3042,6 +3186,7 @@ static int mlxsw_sp3_init(struct mlxsw_core *mlxsw_core,
mlxsw_sp->mr_tcam_ops = &mlxsw_sp2_mr_tcam_ops;
mlxsw_sp->acl_rulei_ops = &mlxsw_sp2_acl_rulei_ops;
mlxsw_sp->acl_tcam_ops = &mlxsw_sp2_acl_tcam_ops;
+ mlxsw_sp->acl_bf_ops = &mlxsw_sp2_acl_bf_ops;
mlxsw_sp->nve_ops_arr = mlxsw_sp2_nve_ops_arr;
mlxsw_sp->mac_mask = mlxsw_sp2_mac_mask;
mlxsw_sp->sb_vals = &mlxsw_sp2_sb_vals;
@@ -3058,6 +3203,36 @@ static int mlxsw_sp3_init(struct mlxsw_core *mlxsw_core,
return mlxsw_sp_init(mlxsw_core, mlxsw_bus_info, extack);
}
+static int mlxsw_sp4_init(struct mlxsw_core *mlxsw_core,
+ const struct mlxsw_bus_info *mlxsw_bus_info,
+ struct netlink_ext_ack *extack)
+{
+ struct mlxsw_sp *mlxsw_sp = mlxsw_core_driver_priv(mlxsw_core);
+
+ mlxsw_sp->switchdev_ops = &mlxsw_sp2_switchdev_ops;
+ mlxsw_sp->kvdl_ops = &mlxsw_sp2_kvdl_ops;
+ mlxsw_sp->afa_ops = &mlxsw_sp2_act_afa_ops;
+ mlxsw_sp->afk_ops = &mlxsw_sp4_afk_ops;
+ mlxsw_sp->mr_tcam_ops = &mlxsw_sp2_mr_tcam_ops;
+ mlxsw_sp->acl_rulei_ops = &mlxsw_sp2_acl_rulei_ops;
+ mlxsw_sp->acl_tcam_ops = &mlxsw_sp2_acl_tcam_ops;
+ mlxsw_sp->acl_bf_ops = &mlxsw_sp4_acl_bf_ops;
+ mlxsw_sp->nve_ops_arr = mlxsw_sp2_nve_ops_arr;
+ mlxsw_sp->mac_mask = mlxsw_sp2_mac_mask;
+ mlxsw_sp->sb_vals = &mlxsw_sp2_sb_vals;
+ mlxsw_sp->sb_ops = &mlxsw_sp3_sb_ops;
+ mlxsw_sp->port_type_speed_ops = &mlxsw_sp2_port_type_speed_ops;
+ mlxsw_sp->ptp_ops = &mlxsw_sp2_ptp_ops;
+ mlxsw_sp->span_ops = &mlxsw_sp3_span_ops;
+ mlxsw_sp->policer_core_ops = &mlxsw_sp2_policer_core_ops;
+ mlxsw_sp->trap_ops = &mlxsw_sp2_trap_ops;
+ mlxsw_sp->mall_ops = &mlxsw_sp2_mall_ops;
+ mlxsw_sp->router_ops = &mlxsw_sp2_router_ops;
+ mlxsw_sp->lowest_shaper_bs = MLXSW_REG_QEEC_LOWEST_SHAPER_BS_SP4;
+
+ return mlxsw_sp_init(mlxsw_core, mlxsw_bus_info, extack);
+}
+
static void mlxsw_sp_fini(struct mlxsw_core *mlxsw_core)
{
struct mlxsw_sp *mlxsw_sp = mlxsw_core_driver_priv(mlxsw_core);
@@ -3075,6 +3250,7 @@ static void mlxsw_sp_fini(struct mlxsw_core *mlxsw_core)
mlxsw_sp_router_fini(mlxsw_sp);
mlxsw_sp_acl_fini(mlxsw_sp);
mlxsw_sp_nve_fini(mlxsw_sp);
+ mlxsw_sp_ipv6_addr_ht_fini(mlxsw_sp);
mlxsw_sp_afa_fini(mlxsw_sp);
mlxsw_sp_counter_pool_fini(mlxsw_sp);
mlxsw_sp_switchdev_fini(mlxsw_sp);
@@ -3336,7 +3512,7 @@ static int mlxsw_sp1_resources_register(struct mlxsw_core *mlxsw_core)
err_policer_resources_register:
err_resources_counter_register:
err_resources_span_register:
- devlink_resources_unregister(priv_to_devlink(mlxsw_core), NULL);
+ devlink_resources_unregister(priv_to_devlink(mlxsw_core));
return err;
}
@@ -3370,7 +3546,7 @@ static int mlxsw_sp2_resources_register(struct mlxsw_core *mlxsw_core)
err_policer_resources_register:
err_resources_counter_register:
err_resources_span_register:
- devlink_resources_unregister(priv_to_devlink(mlxsw_core), NULL);
+ devlink_resources_unregister(priv_to_devlink(mlxsw_core));
return err;
}
@@ -3486,7 +3662,7 @@ static void mlxsw_sp2_params_unregister(struct mlxsw_core *mlxsw_core)
}
static void mlxsw_sp_ptp_transmitted(struct mlxsw_core *mlxsw_core,
- struct sk_buff *skb, u8 local_port)
+ struct sk_buff *skb, u16 local_port)
{
struct mlxsw_sp *mlxsw_sp = mlxsw_core_driver_priv(mlxsw_core);
@@ -3616,6 +3792,45 @@ static struct mlxsw_driver mlxsw_sp3_driver = {
.temp_warn_enabled = true,
};
+static struct mlxsw_driver mlxsw_sp4_driver = {
+ .kind = mlxsw_sp4_driver_name,
+ .priv_size = sizeof(struct mlxsw_sp),
+ .init = mlxsw_sp4_init,
+ .fini = mlxsw_sp_fini,
+ .basic_trap_groups_set = mlxsw_sp_basic_trap_groups_set,
+ .port_split = mlxsw_sp_port_split,
+ .port_unsplit = mlxsw_sp_port_unsplit,
+ .sb_pool_get = mlxsw_sp_sb_pool_get,
+ .sb_pool_set = mlxsw_sp_sb_pool_set,
+ .sb_port_pool_get = mlxsw_sp_sb_port_pool_get,
+ .sb_port_pool_set = mlxsw_sp_sb_port_pool_set,
+ .sb_tc_pool_bind_get = mlxsw_sp_sb_tc_pool_bind_get,
+ .sb_tc_pool_bind_set = mlxsw_sp_sb_tc_pool_bind_set,
+ .sb_occ_snapshot = mlxsw_sp_sb_occ_snapshot,
+ .sb_occ_max_clear = mlxsw_sp_sb_occ_max_clear,
+ .sb_occ_port_pool_get = mlxsw_sp_sb_occ_port_pool_get,
+ .sb_occ_tc_port_bind_get = mlxsw_sp_sb_occ_tc_port_bind_get,
+ .trap_init = mlxsw_sp_trap_init,
+ .trap_fini = mlxsw_sp_trap_fini,
+ .trap_action_set = mlxsw_sp_trap_action_set,
+ .trap_group_init = mlxsw_sp_trap_group_init,
+ .trap_group_set = mlxsw_sp_trap_group_set,
+ .trap_policer_init = mlxsw_sp_trap_policer_init,
+ .trap_policer_fini = mlxsw_sp_trap_policer_fini,
+ .trap_policer_set = mlxsw_sp_trap_policer_set,
+ .trap_policer_counter_get = mlxsw_sp_trap_policer_counter_get,
+ .txhdr_construct = mlxsw_sp_txhdr_construct,
+ .resources_register = mlxsw_sp2_resources_register,
+ .params_register = mlxsw_sp2_params_register,
+ .params_unregister = mlxsw_sp2_params_unregister,
+ .ptp_transmitted = mlxsw_sp_ptp_transmitted,
+ .txhdr_len = MLXSW_TXHDR_LEN,
+ .profile = &mlxsw_sp2_config_profile,
+ .res_query_enabled = true,
+ .fw_fatal_enabled = true,
+ .temp_warn_enabled = true,
+};
+
bool mlxsw_sp_port_dev_check(const struct net_device *dev)
{
return dev->netdev_ops == &mlxsw_sp_port_netdev_ops;
@@ -4783,6 +4998,16 @@ static struct pci_driver mlxsw_sp3_pci_driver = {
.id_table = mlxsw_sp3_pci_id_table,
};
+static const struct pci_device_id mlxsw_sp4_pci_id_table[] = {
+ {PCI_VDEVICE(MELLANOX, PCI_DEVICE_ID_MELLANOX_SPECTRUM4), 0},
+ {0, },
+};
+
+static struct pci_driver mlxsw_sp4_pci_driver = {
+ .name = mlxsw_sp4_driver_name,
+ .id_table = mlxsw_sp4_pci_id_table,
+};
+
static int __init mlxsw_sp_module_init(void)
{
int err;
@@ -4802,6 +5027,10 @@ static int __init mlxsw_sp_module_init(void)
if (err)
goto err_sp3_core_driver_register;
+ err = mlxsw_core_driver_register(&mlxsw_sp4_driver);
+ if (err)
+ goto err_sp4_core_driver_register;
+
err = mlxsw_pci_driver_register(&mlxsw_sp1_pci_driver);
if (err)
goto err_sp1_pci_driver_register;
@@ -4814,13 +5043,21 @@ static int __init mlxsw_sp_module_init(void)
if (err)
goto err_sp3_pci_driver_register;
+ err = mlxsw_pci_driver_register(&mlxsw_sp4_pci_driver);
+ if (err)
+ goto err_sp4_pci_driver_register;
+
return 0;
+err_sp4_pci_driver_register:
+ mlxsw_pci_driver_unregister(&mlxsw_sp3_pci_driver);
err_sp3_pci_driver_register:
mlxsw_pci_driver_unregister(&mlxsw_sp2_pci_driver);
err_sp2_pci_driver_register:
mlxsw_pci_driver_unregister(&mlxsw_sp1_pci_driver);
err_sp1_pci_driver_register:
+ mlxsw_core_driver_unregister(&mlxsw_sp4_driver);
+err_sp4_core_driver_register:
mlxsw_core_driver_unregister(&mlxsw_sp3_driver);
err_sp3_core_driver_register:
mlxsw_core_driver_unregister(&mlxsw_sp2_driver);
@@ -4834,9 +5071,11 @@ static int __init mlxsw_sp_module_init(void)
static void __exit mlxsw_sp_module_exit(void)
{
+ mlxsw_pci_driver_unregister(&mlxsw_sp4_pci_driver);
mlxsw_pci_driver_unregister(&mlxsw_sp3_pci_driver);
mlxsw_pci_driver_unregister(&mlxsw_sp2_pci_driver);
mlxsw_pci_driver_unregister(&mlxsw_sp1_pci_driver);
+ mlxsw_core_driver_unregister(&mlxsw_sp4_driver);
mlxsw_core_driver_unregister(&mlxsw_sp3_driver);
mlxsw_core_driver_unregister(&mlxsw_sp2_driver);
mlxsw_core_driver_unregister(&mlxsw_sp1_driver);
@@ -4853,6 +5092,7 @@ MODULE_DESCRIPTION("Mellanox Spectrum driver");
MODULE_DEVICE_TABLE(pci, mlxsw_sp1_pci_id_table);
MODULE_DEVICE_TABLE(pci, mlxsw_sp2_pci_id_table);
MODULE_DEVICE_TABLE(pci, mlxsw_sp3_pci_id_table);
+MODULE_DEVICE_TABLE(pci, mlxsw_sp4_pci_id_table);
MODULE_FIRMWARE(MLXSW_SP1_FW_FILENAME);
MODULE_FIRMWARE(MLXSW_SP2_FW_FILENAME);
MODULE_FIRMWARE(MLXSW_SP3_FW_FILENAME);
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum.h b/drivers/net/ethernet/mellanox/mlxsw/spectrum.h
index 32fdd37..bb2442e 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum.h
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum.h
@@ -190,6 +190,7 @@ struct mlxsw_sp {
const struct mlxsw_sp_mr_tcam_ops *mr_tcam_ops;
const struct mlxsw_sp_acl_rulei_ops *acl_rulei_ops;
const struct mlxsw_sp_acl_tcam_ops *acl_tcam_ops;
+ const struct mlxsw_sp_acl_bf_ops *acl_bf_ops;
const struct mlxsw_sp_nve_ops **nve_ops_arr;
const struct mlxsw_sp_sb_vals *sb_vals;
const struct mlxsw_sp_sb_ops *sb_ops;
@@ -203,6 +204,8 @@ struct mlxsw_sp {
const struct mlxsw_listener *listeners;
size_t listeners_count;
u32 lowest_shaper_bs;
+ struct rhashtable ipv6_addr_ht;
+ struct mutex ipv6_addr_ht_lock; /* Protects ipv6_addr_ht */
};
struct mlxsw_sp_ptp_ops {
@@ -217,13 +220,13 @@ struct mlxsw_sp_ptp_ops {
* is responsible for freeing the passed-in SKB.
*/
void (*receive)(struct mlxsw_sp *mlxsw_sp, struct sk_buff *skb,
- u8 local_port);
+ u16 local_port);
/* Notify a driver that a timestamped packet was transmitted. Driver
* is responsible for freeing the passed-in SKB.
*/
void (*transmitted)(struct mlxsw_sp *mlxsw_sp, struct sk_buff *skb,
- u8 local_port);
+ u16 local_port);
int (*hwtstamp_get)(struct mlxsw_sp_port *mlxsw_sp_port,
struct hwtstamp_config *config);
@@ -261,7 +264,7 @@ enum mlxsw_sp_sample_trigger_type {
struct mlxsw_sp_sample_trigger {
enum mlxsw_sp_sample_trigger_type type;
- u8 local_port; /* Reserved when trigger type is not ingress / egress. */
+ u16 local_port; /* Reserved when trigger type is not ingress / egress. */
};
struct mlxsw_sp_sample_params {
@@ -308,7 +311,7 @@ struct mlxsw_sp_port {
struct net_device *dev;
struct mlxsw_sp_port_pcpu_stats __percpu *pcpu_stats;
struct mlxsw_sp *mlxsw_sp;
- u8 local_port;
+ u16 local_port;
u8 lagged:1,
split:1;
u16 pvid;
@@ -370,7 +373,7 @@ struct mlxsw_sp_port_type_speed_ops {
u32 (*to_ptys_speed_lanes)(struct mlxsw_sp *mlxsw_sp, u8 width,
const struct ethtool_link_ksettings *cmd);
void (*reg_ptys_eth_pack)(struct mlxsw_sp *mlxsw_sp, char *payload,
- u8 local_port, u32 proto_admin, bool autoneg);
+ u16 local_port, u32 proto_admin, bool autoneg);
void (*reg_ptys_eth_unpack)(struct mlxsw_sp *mlxsw_sp, char *payload,
u32 *p_eth_proto_cap,
u32 *p_eth_proto_admin,
@@ -441,7 +444,7 @@ static inline struct mlxsw_sp_port *
mlxsw_sp_port_lagged_get(struct mlxsw_sp *mlxsw_sp, u16 lag_id, u8 port_index)
{
struct mlxsw_sp_port *mlxsw_sp_port;
- u8 local_port;
+ u16 local_port;
local_port = mlxsw_core_lag_mapping_get(mlxsw_sp->core,
lag_id, port_index);
@@ -587,6 +590,11 @@ mlxsw_sp_sample_trigger_params_set(struct mlxsw_sp *mlxsw_sp,
void
mlxsw_sp_sample_trigger_params_unset(struct mlxsw_sp *mlxsw_sp,
const struct mlxsw_sp_sample_trigger *trigger);
+int mlxsw_sp_ipv6_addr_kvdl_index_get(struct mlxsw_sp *mlxsw_sp,
+ const struct in6_addr *addr6,
+ u32 *p_kvdl_index);
+void
+mlxsw_sp_ipv6_addr_put(struct mlxsw_sp *mlxsw_sp, const struct in6_addr *addr6);
extern const struct mlxsw_sp_sb_vals mlxsw_sp1_sb_vals;
extern const struct mlxsw_sp_sb_vals mlxsw_sp2_sb_vals;
@@ -621,9 +629,9 @@ extern struct notifier_block mlxsw_sp_switchdev_notifier;
/* spectrum.c */
void mlxsw_sp_rx_listener_no_mark_func(struct sk_buff *skb,
- u8 local_port, void *priv);
+ u16 local_port, void *priv);
void mlxsw_sp_ptp_receive(struct mlxsw_sp *mlxsw_sp, struct sk_buff *skb,
- u8 local_port);
+ u16 local_port);
int mlxsw_sp_port_speed_get(struct mlxsw_sp_port *mlxsw_sp_port, u32 *speed);
int mlxsw_sp_port_ets_set(struct mlxsw_sp_port *mlxsw_sp_port,
enum mlxsw_reg_qeec_hr hr, u8 index, u8 next_index,
@@ -729,7 +737,7 @@ void mlxsw_sp_rif_destroy_by_dev(struct mlxsw_sp *mlxsw_sp,
bool mlxsw_sp_rif_exists(struct mlxsw_sp *mlxsw_sp,
const struct net_device *dev);
u16 mlxsw_sp_rif_vid(struct mlxsw_sp *mlxsw_sp, const struct net_device *dev);
-u8 mlxsw_sp_router_port(const struct mlxsw_sp *mlxsw_sp);
+u16 mlxsw_sp_router_port(const struct mlxsw_sp *mlxsw_sp);
int mlxsw_sp_router_nve_promote_decap(struct mlxsw_sp *mlxsw_sp, u32 ul_tb_id,
enum mlxsw_sp_l3proto ul_proto,
const union mlxsw_sp_l3addr *ul_sip,
@@ -1099,6 +1107,11 @@ extern const struct mlxsw_afa_ops mlxsw_sp2_act_afa_ops;
/* spectrum_acl_flex_keys.c */
extern const struct mlxsw_afk_ops mlxsw_sp1_afk_ops;
extern const struct mlxsw_afk_ops mlxsw_sp2_afk_ops;
+extern const struct mlxsw_afk_ops mlxsw_sp4_afk_ops;
+
+/* spectrum_acl_bloom_filter.c */
+extern const struct mlxsw_sp_acl_bf_ops mlxsw_sp2_acl_bf_ops;
+extern const struct mlxsw_sp_acl_bf_ops mlxsw_sp4_acl_bf_ops;
/* spectrum_matchall.c */
struct mlxsw_sp_mall_ops {
@@ -1222,7 +1235,7 @@ bool mlxsw_sp_fid_vni_is_set(const struct mlxsw_sp_fid *fid);
void mlxsw_sp_fid_fdb_clear_offload(const struct mlxsw_sp_fid *fid,
const struct net_device *nve_dev);
int mlxsw_sp_fid_flood_set(struct mlxsw_sp_fid *fid,
- enum mlxsw_sp_flood_type packet_type, u8 local_port,
+ enum mlxsw_sp_flood_type packet_type, u16 local_port,
bool member);
int mlxsw_sp_fid_port_vid_map(struct mlxsw_sp_fid *fid,
struct mlxsw_sp_port *mlxsw_sp_port, u16 vid);
@@ -1310,6 +1323,17 @@ void mlxsw_sp_nve_flood_ip_del(struct mlxsw_sp *mlxsw_sp,
struct mlxsw_sp_fid *fid,
enum mlxsw_sp_l3proto proto,
union mlxsw_sp_l3addr *addr);
+int mlxsw_sp_nve_ipv6_addr_kvdl_set(struct mlxsw_sp *mlxsw_sp,
+ const struct in6_addr *addr6,
+ u32 *p_kvdl_index);
+void mlxsw_sp_nve_ipv6_addr_kvdl_unset(struct mlxsw_sp *mlxsw_sp,
+ const struct in6_addr *addr6);
+int
+mlxsw_sp_nve_ipv6_addr_map_replace(struct mlxsw_sp *mlxsw_sp, const char *mac,
+ u16 fid_index,
+ const struct in6_addr *new_addr6);
+void mlxsw_sp_nve_ipv6_addr_map_del(struct mlxsw_sp *mlxsw_sp, const char *mac,
+ u16 fid_index);
int mlxsw_sp_nve_fid_enable(struct mlxsw_sp *mlxsw_sp, struct mlxsw_sp_fid *fid,
struct mlxsw_sp_nve_params *params,
struct netlink_ext_ack *extack);
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum2_mr_tcam.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum2_mr_tcam.c
index a11d911..e4f4cde 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum2_mr_tcam.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum2_mr_tcam.c
@@ -45,8 +45,8 @@ static int mlxsw_sp2_mr_tcam_bind_group(struct mlxsw_sp *mlxsw_sp,
}
static const enum mlxsw_afk_element mlxsw_sp2_mr_tcam_usage_ipv4[] = {
- MLXSW_AFK_ELEMENT_VIRT_ROUTER_8_10,
- MLXSW_AFK_ELEMENT_VIRT_ROUTER_0_7,
+ MLXSW_AFK_ELEMENT_VIRT_ROUTER_MSB,
+ MLXSW_AFK_ELEMENT_VIRT_ROUTER_LSB,
MLXSW_AFK_ELEMENT_SRC_IP_0_31,
MLXSW_AFK_ELEMENT_DST_IP_0_31,
};
@@ -89,8 +89,8 @@ static void mlxsw_sp2_mr_tcam_ipv4_fini(struct mlxsw_sp2_mr_tcam *mr_tcam)
}
static const enum mlxsw_afk_element mlxsw_sp2_mr_tcam_usage_ipv6[] = {
- MLXSW_AFK_ELEMENT_VIRT_ROUTER_8_10,
- MLXSW_AFK_ELEMENT_VIRT_ROUTER_0_7,
+ MLXSW_AFK_ELEMENT_VIRT_ROUTER_MSB,
+ MLXSW_AFK_ELEMENT_VIRT_ROUTER_LSB,
MLXSW_AFK_ELEMENT_SRC_IP_96_127,
MLXSW_AFK_ELEMENT_SRC_IP_64_95,
MLXSW_AFK_ELEMENT_SRC_IP_32_63,
@@ -189,10 +189,10 @@ mlxsw_sp2_mr_tcam_rule_parse(struct mlxsw_sp_acl_rule *rule,
rulei = mlxsw_sp_acl_rule_rulei(rule);
rulei->priority = priority;
- mlxsw_sp_acl_rulei_keymask_u32(rulei, MLXSW_AFK_ELEMENT_VIRT_ROUTER_0_7,
+ mlxsw_sp_acl_rulei_keymask_u32(rulei, MLXSW_AFK_ELEMENT_VIRT_ROUTER_LSB,
key->vrid, GENMASK(7, 0));
mlxsw_sp_acl_rulei_keymask_u32(rulei,
- MLXSW_AFK_ELEMENT_VIRT_ROUTER_8_10,
+ MLXSW_AFK_ELEMENT_VIRT_ROUTER_MSB,
key->vrid >> 8, GENMASK(2, 0));
switch (key->proto) {
case MLXSW_SP_L3_PROTO_IPV4:
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_acl.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_acl.c
index 67cedfa..70c11bf 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_acl.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_acl.c
@@ -406,7 +406,7 @@ int mlxsw_sp_acl_rulei_act_fwd(struct mlxsw_sp *mlxsw_sp,
struct netlink_ext_ack *extack)
{
struct mlxsw_sp_port *mlxsw_sp_port;
- u8 local_port;
+ u16 local_port;
bool in_port;
if (out_dev) {
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_acl_bloom_filter.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_acl_bloom_filter.c
index dbd3beb..e2aced7 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_acl_bloom_filter.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_acl_bloom_filter.c
@@ -17,9 +17,9 @@ struct mlxsw_sp_acl_bf {
};
/* Bloom filter uses a crc-16 hash over chunks of data which contain 4 key
- * blocks, eRP ID and region ID. In Spectrum-2, region key is combined of up to
- * 12 key blocks, so there can be up to 3 chunks in the Bloom filter key,
- * depending on the actual number of key blocks used in the region.
+ * blocks, eRP ID and region ID. In Spectrum-2 and above, region key is combined
+ * of up to 12 key blocks, so there can be up to 3 chunks in the Bloom filter
+ * key, depending on the actual number of key blocks used in the region.
* The layout of the Bloom filter key is as follows:
*
* +-------------------------+------------------------+------------------------+
@@ -27,7 +27,9 @@ struct mlxsw_sp_acl_bf {
* +-------------------------+------------------------+------------------------+
*/
#define MLXSW_BLOOM_KEY_CHUNKS 3
-#define MLXSW_BLOOM_KEY_LEN 69
+
+/* Spectrum-2 and Spectrum-3 chunks */
+#define MLXSW_SP2_BLOOM_KEY_LEN 69
/* Each chunk size is 23 bytes. 18 bytes of it contain 4 key blocks, each is
* 36 bits, 2 bytes which hold eRP ID and region ID, and 3 bytes of zero
@@ -42,31 +44,21 @@ struct mlxsw_sp_acl_bf {
* | 0 | region ID | eRP ID | 4 Key blocks (18 Bytes) |
* +---------+-----------+----------+-----------------------------------+
*/
-#define MLXSW_BLOOM_CHUNK_PAD_BYTES 3
-#define MLXSW_BLOOM_CHUNK_KEY_BYTES 18
-#define MLXSW_BLOOM_KEY_CHUNK_BYTES 23
+#define MLXSW_SP2_BLOOM_CHUNK_PAD_BYTES 3
+#define MLXSW_SP2_BLOOM_CHUNK_KEY_BYTES 18
+#define MLXSW_SP2_BLOOM_KEY_CHUNK_BYTES 23
/* The offset of the key block within a chunk is 5 bytes as it comes after
* 3 bytes of zero padding and 16 bits of region ID and eRP ID.
*/
-#define MLXSW_BLOOM_CHUNK_KEY_OFFSET 5
+#define MLXSW_SP2_BLOOM_CHUNK_KEY_OFFSET 5
-/* Each chunk contains 4 key blocks. Chunk 2 uses key blocks 11-8,
- * and we need to populate it with 4 key blocks copied from the entry encoded
- * key. Since the encoded key contains a padding, key block 11 starts at offset
- * 2. block 7 that is used in chunk 1 starts at offset 20 as 4 key blocks take
- * 18 bytes.
- * This array defines key offsets for easy access when copying key blocks from
- * entry key to Bloom filter chunk.
- */
-static const u8 chunk_key_offsets[MLXSW_BLOOM_KEY_CHUNKS] = {2, 20, 38};
-
-/* This table is just the CRC of each possible byte. It is
- * computed, Msbit first, for the Bloom filter polynomial
- * which is 0x8529 (1 + x^3 + x^5 + x^8 + x^10 + x^15 and
+/* This table is just the CRC of each possible byte which is used for
+ * Spectrum-{2-3}. It is computed, Msbit first, for the Bloom filter
+ * polynomial which is 0x8529 (1 + x^3 + x^5 + x^8 + x^10 + x^15 and
* the implicit x^16).
*/
-static const u16 mlxsw_sp_acl_bf_crc_tab[256] = {
+static const u16 mlxsw_sp2_acl_bf_crc16_tab[256] = {
0x0000, 0x8529, 0x8f7b, 0x0a52, 0x9bdf, 0x1ef6, 0x14a4, 0x918d,
0xb297, 0x37be, 0x3dec, 0xb8c5, 0x2948, 0xac61, 0xa633, 0x231a,
0xe007, 0x652e, 0x6f7c, 0xea55, 0x7bd8, 0xfef1, 0xf4a3, 0x718a,
@@ -101,24 +93,146 @@ static const u16 mlxsw_sp_acl_bf_crc_tab[256] = {
0x0c4c, 0x8965, 0x8337, 0x061e, 0x9793, 0x12ba, 0x18e8, 0x9dc1,
};
-static u16 mlxsw_sp_acl_bf_crc_byte(u16 crc, u8 c)
+/* Spectrum-4 chunks */
+#define MLXSW_SP4_BLOOM_KEY_LEN 60
+
+/* In Spectrum-4, there is no padding. Each chunk size is 20 bytes.
+ * 18 bytes of it contain 4 key blocks, each is 36 bits, and 2 bytes which hold
+ * eRP ID and region ID.
+ * The layout of each chunk is as follows:
+ *
+ * +----------------------+-----------------------------------+
+ * | 2 bytes | 18 bytes |
+ * +-----------+----------+-----------------------------------+
+ * | 157:148 | 147:144 | 143:0 |
+ * +---------+-----------+----------+-------------------------+
+ * | region ID | eRP ID | 4 Key blocks (18 Bytes) |
+ * +-----------+----------+-----------------------------------+
+ */
+
+#define MLXSW_SP4_BLOOM_CHUNK_PAD_BYTES 0
+#define MLXSW_SP4_BLOOM_CHUNK_KEY_BYTES 18
+#define MLXSW_SP4_BLOOM_KEY_CHUNK_BYTES 20
+
+/* The offset of the key block within a chunk is 2 bytes as it comes after
+ * 16 bits of region ID and eRP ID.
+ */
+#define MLXSW_SP4_BLOOM_CHUNK_KEY_OFFSET 2
+
+/* For Spectrum-4, two hash functions are used, CRC-10 and CRC-6 based.
+ * The result is combination of the two calculations -
+ * 6 bit column are MSB (result of CRC-6),
+ * 10 bit row are LSB (result of CRC-10).
+ */
+
+/* This table is just the CRC of each possible byte which is used for
+ * Spectrum-4. It is computed, Msbit first, for the Bloom filter
+ * polynomial which is 0x1b (1 + x^1 + x^3 + x^4 and the implicit x^10).
+ */
+static const u16 mlxsw_sp4_acl_bf_crc10_tab[256] = {
+0x0000, 0x001b, 0x0036, 0x002d, 0x006c, 0x0077, 0x005a, 0x0041,
+0x00d8, 0x00c3, 0x00ee, 0x00f5, 0x00b4, 0x00af, 0x0082, 0x0099,
+0x01b0, 0x01ab, 0x0186, 0x019d, 0x01dc, 0x01c7, 0x01ea, 0x01f1,
+0x0168, 0x0173, 0x015e, 0x0145, 0x0104, 0x011f, 0x0132, 0x0129,
+0x0360, 0x037b, 0x0356, 0x034d, 0x030c, 0x0317, 0x033a, 0x0321,
+0x03b8, 0x03a3, 0x038e, 0x0395, 0x03d4, 0x03cf, 0x03e2, 0x03f9,
+0x02d0, 0x02cb, 0x02e6, 0x02fd, 0x02bc, 0x02a7, 0x028a, 0x0291,
+0x0208, 0x0213, 0x023e, 0x0225, 0x0264, 0x027f, 0x0252, 0x0249,
+0x02db, 0x02c0, 0x02ed, 0x02f6, 0x02b7, 0x02ac, 0x0281, 0x029a,
+0x0203, 0x0218, 0x0235, 0x022e, 0x026f, 0x0274, 0x0259, 0x0242,
+0x036b, 0x0370, 0x035d, 0x0346, 0x0307, 0x031c, 0x0331, 0x032a,
+0x03b3, 0x03a8, 0x0385, 0x039e, 0x03df, 0x03c4, 0x03e9, 0x03f2,
+0x01bb, 0x01a0, 0x018d, 0x0196, 0x01d7, 0x01cc, 0x01e1, 0x01fa,
+0x0163, 0x0178, 0x0155, 0x014e, 0x010f, 0x0114, 0x0139, 0x0122,
+0x000b, 0x0010, 0x003d, 0x0026, 0x0067, 0x007c, 0x0051, 0x004a,
+0x00d3, 0x00c8, 0x00e5, 0x00fe, 0x00bf, 0x00a4, 0x0089, 0x0092,
+0x01ad, 0x01b6, 0x019b, 0x0180, 0x01c1, 0x01da, 0x01f7, 0x01ec,
+0x0175, 0x016e, 0x0143, 0x0158, 0x0119, 0x0102, 0x012f, 0x0134,
+0x001d, 0x0006, 0x002b, 0x0030, 0x0071, 0x006a, 0x0047, 0x005c,
+0x00c5, 0x00de, 0x00f3, 0x00e8, 0x00a9, 0x00b2, 0x009f, 0x0084,
+0x02cd, 0x02d6, 0x02fb, 0x02e0, 0x02a1, 0x02ba, 0x0297, 0x028c,
+0x0215, 0x020e, 0x0223, 0x0238, 0x0279, 0x0262, 0x024f, 0x0254,
+0x037d, 0x0366, 0x034b, 0x0350, 0x0311, 0x030a, 0x0327, 0x033c,
+0x03a5, 0x03be, 0x0393, 0x0388, 0x03c9, 0x03d2, 0x03ff, 0x03e4,
+0x0376, 0x036d, 0x0340, 0x035b, 0x031a, 0x0301, 0x032c, 0x0337,
+0x03ae, 0x03b5, 0x0398, 0x0383, 0x03c2, 0x03d9, 0x03f4, 0x03ef,
+0x02c6, 0x02dd, 0x02f0, 0x02eb, 0x02aa, 0x02b1, 0x029c, 0x0287,
+0x021e, 0x0205, 0x0228, 0x0233, 0x0272, 0x0269, 0x0244, 0x025f,
+0x0016, 0x000d, 0x0020, 0x003b, 0x007a, 0x0061, 0x004c, 0x0057,
+0x00ce, 0x00d5, 0x00f8, 0x00e3, 0x00a2, 0x00b9, 0x0094, 0x008f,
+0x01a6, 0x01bd, 0x0190, 0x018b, 0x01ca, 0x01d1, 0x01fc, 0x01e7,
+0x017e, 0x0165, 0x0148, 0x0153, 0x0112, 0x0109, 0x0124, 0x013f,
+};
+
+/* This table is just the CRC of each possible byte which is used for
+ * Spectrum-4. It is computed, Msbit first, for the Bloom filter
+ * polynomial which is 0x2d (1 + x^2+ x^3 + x^5 and the implicit x^6).
+ */
+static const u8 mlxsw_sp4_acl_bf_crc6_tab[256] = {
+0x00, 0x2d, 0x37, 0x1a, 0x03, 0x2e, 0x34, 0x19,
+0x06, 0x2b, 0x31, 0x1c, 0x05, 0x28, 0x32, 0x1f,
+0x0c, 0x21, 0x3b, 0x16, 0x0f, 0x22, 0x38, 0x15,
+0x0a, 0x27, 0x3d, 0x10, 0x09, 0x24, 0x3e, 0x13,
+0x18, 0x35, 0x2f, 0x02, 0x1b, 0x36, 0x2c, 0x01,
+0x1e, 0x33, 0x29, 0x04, 0x1d, 0x30, 0x2a, 0x07,
+0x14, 0x39, 0x23, 0x0e, 0x17, 0x3a, 0x20, 0x0d,
+0x12, 0x3f, 0x25, 0x08, 0x11, 0x3c, 0x26, 0x0b,
+0x30, 0x1d, 0x07, 0x2a, 0x33, 0x1e, 0x04, 0x29,
+0x36, 0x1b, 0x01, 0x2c, 0x35, 0x18, 0x02, 0x2f,
+0x3c, 0x11, 0x0b, 0x26, 0x3f, 0x12, 0x08, 0x25,
+0x3a, 0x17, 0x0d, 0x20, 0x39, 0x14, 0x0e, 0x23,
+0x28, 0x05, 0x1f, 0x32, 0x2b, 0x06, 0x1c, 0x31,
+0x2e, 0x03, 0x19, 0x34, 0x2d, 0x00, 0x1a, 0x37,
+0x24, 0x09, 0x13, 0x3e, 0x27, 0x0a, 0x10, 0x3d,
+0x22, 0x0f, 0x15, 0x38, 0x21, 0x0c, 0x16, 0x3b,
+0x0d, 0x20, 0x3a, 0x17, 0x0e, 0x23, 0x39, 0x14,
+0x0b, 0x26, 0x3c, 0x11, 0x08, 0x25, 0x3f, 0x12,
+0x01, 0x2c, 0x36, 0x1b, 0x02, 0x2f, 0x35, 0x18,
+0x07, 0x2a, 0x30, 0x1d, 0x04, 0x29, 0x33, 0x1e,
+0x15, 0x38, 0x22, 0x0f, 0x16, 0x3b, 0x21, 0x0c,
+0x13, 0x3e, 0x24, 0x09, 0x10, 0x3d, 0x27, 0x0a,
+0x19, 0x34, 0x2e, 0x03, 0x1a, 0x37, 0x2d, 0x00,
+0x1f, 0x32, 0x28, 0x05, 0x1c, 0x31, 0x2b, 0x06,
+0x3d, 0x10, 0x0a, 0x27, 0x3e, 0x13, 0x09, 0x24,
+0x3b, 0x16, 0x0c, 0x21, 0x38, 0x15, 0x0f, 0x22,
+0x31, 0x1c, 0x06, 0x2b, 0x32, 0x1f, 0x05, 0x28,
+0x37, 0x1a, 0x00, 0x2d, 0x34, 0x19, 0x03, 0x2e,
+0x25, 0x08, 0x12, 0x3f, 0x26, 0x0b, 0x11, 0x3c,
+0x23, 0x0e, 0x14, 0x39, 0x20, 0x0d, 0x17, 0x3a,
+0x29, 0x04, 0x1e, 0x33, 0x2a, 0x07, 0x1d, 0x30,
+0x2f, 0x02, 0x18, 0x35, 0x2c, 0x01, 0x1b, 0x36,
+};
+
+/* Each chunk contains 4 key blocks. Chunk 2 uses key blocks 11-8,
+ * and we need to populate it with 4 key blocks copied from the entry encoded
+ * key. The original keys layout is same for Spectrum-{2,3,4}.
+ * Since the encoded key contains a 2 bytes padding, key block 11 starts at
+ * offset 2. block 7 that is used in chunk 1 starts at offset 20 as 4 key blocks
+ * take 18 bytes. See 'MLXSW_SP2_AFK_BLOCK_LAYOUT' for more details.
+ * This array defines key offsets for easy access when copying key blocks from
+ * entry key to Bloom filter chunk.
+ */
+static const u8 chunk_key_offsets[MLXSW_BLOOM_KEY_CHUNKS] = {2, 20, 38};
+
+static u16 mlxsw_sp2_acl_bf_crc16_byte(u16 crc, u8 c)
{
- return (crc << 8) ^ mlxsw_sp_acl_bf_crc_tab[(crc >> 8) ^ c];
+ return (crc << 8) ^ mlxsw_sp2_acl_bf_crc16_tab[(crc >> 8) ^ c];
}
-static u16 mlxsw_sp_acl_bf_crc(const u8 *buffer, size_t len)
+static u16 mlxsw_sp2_acl_bf_crc(const u8 *buffer, size_t len)
{
u16 crc = 0;
while (len--)
- crc = mlxsw_sp_acl_bf_crc_byte(crc, *buffer++);
+ crc = mlxsw_sp2_acl_bf_crc16_byte(crc, *buffer++);
return crc;
}
static void
-mlxsw_sp_acl_bf_key_encode(struct mlxsw_sp_acl_atcam_region *aregion,
- struct mlxsw_sp_acl_atcam_entry *aentry,
- char *output, u8 *len)
+__mlxsw_sp_acl_bf_key_encode(struct mlxsw_sp_acl_atcam_region *aregion,
+ struct mlxsw_sp_acl_atcam_entry *aentry,
+ char *output, u8 *len, u8 max_chunks, u8 pad_bytes,
+ u8 key_offset, u8 chunk_key_len, u8 chunk_len)
{
struct mlxsw_afk_key_info *key_info = aregion->region->key_info;
u8 chunk_index, chunk_count, block_count;
@@ -129,17 +243,160 @@ mlxsw_sp_acl_bf_key_encode(struct mlxsw_sp_acl_atcam_region *aregion,
chunk_count = 1 + ((block_count - 1) >> 2);
erp_region_id = cpu_to_be16(aentry->ht_key.erp_id |
(aregion->region->id << 4));
- for (chunk_index = MLXSW_BLOOM_KEY_CHUNKS - chunk_count;
- chunk_index < MLXSW_BLOOM_KEY_CHUNKS; chunk_index++) {
- memset(chunk, 0, MLXSW_BLOOM_CHUNK_PAD_BYTES);
- memcpy(chunk + MLXSW_BLOOM_CHUNK_PAD_BYTES, &erp_region_id,
+ for (chunk_index = max_chunks - chunk_count; chunk_index < max_chunks;
+ chunk_index++) {
+ memset(chunk, 0, pad_bytes);
+ memcpy(chunk + pad_bytes, &erp_region_id,
sizeof(erp_region_id));
- memcpy(chunk + MLXSW_BLOOM_CHUNK_KEY_OFFSET,
+ memcpy(chunk + key_offset,
&aentry->enc_key[chunk_key_offsets[chunk_index]],
- MLXSW_BLOOM_CHUNK_KEY_BYTES);
- chunk += MLXSW_BLOOM_KEY_CHUNK_BYTES;
+ chunk_key_len);
+ chunk += chunk_len;
}
- *len = chunk_count * MLXSW_BLOOM_KEY_CHUNK_BYTES;
+ *len = chunk_count * chunk_len;
+}
+
+static void
+mlxsw_sp2_acl_bf_key_encode(struct mlxsw_sp_acl_atcam_region *aregion,
+ struct mlxsw_sp_acl_atcam_entry *aentry,
+ char *output, u8 *len)
+{
+ __mlxsw_sp_acl_bf_key_encode(aregion, aentry, output, len,
+ MLXSW_BLOOM_KEY_CHUNKS,
+ MLXSW_SP2_BLOOM_CHUNK_PAD_BYTES,
+ MLXSW_SP2_BLOOM_CHUNK_KEY_OFFSET,
+ MLXSW_SP2_BLOOM_CHUNK_KEY_BYTES,
+ MLXSW_SP2_BLOOM_KEY_CHUNK_BYTES);
+}
+
+static unsigned int
+mlxsw_sp2_acl_bf_index_get(struct mlxsw_sp_acl_bf *bf,
+ struct mlxsw_sp_acl_atcam_region *aregion,
+ struct mlxsw_sp_acl_atcam_entry *aentry)
+{
+ char bf_key[MLXSW_SP2_BLOOM_KEY_LEN];
+ u8 bf_size;
+
+ mlxsw_sp2_acl_bf_key_encode(aregion, aentry, bf_key, &bf_size);
+ return mlxsw_sp2_acl_bf_crc(bf_key, bf_size);
+}
+
+static u16 mlxsw_sp4_acl_bf_crc10_byte(u16 crc, u8 c)
+{
+ u8 index = ((crc >> 2) ^ c) & 0xff;
+
+ return ((crc << 8) ^ mlxsw_sp4_acl_bf_crc10_tab[index]) & 0x3ff;
+}
+
+static u16 mlxsw_sp4_acl_bf_crc6_byte(u16 crc, u8 c)
+{
+ u8 index = (crc ^ c) & 0xff;
+
+ return ((crc << 6) ^ (mlxsw_sp4_acl_bf_crc6_tab[index] << 2)) & 0xfc;
+}
+
+static u16 mlxsw_sp4_acl_bf_crc(const u8 *buffer, size_t len)
+{
+ u16 crc_row = 0, crc_col = 0;
+
+ while (len--) {
+ crc_row = mlxsw_sp4_acl_bf_crc10_byte(crc_row, *buffer);
+ crc_col = mlxsw_sp4_acl_bf_crc6_byte(crc_col, *buffer);
+ buffer++;
+ }
+
+ crc_col >>= 2;
+
+ /* 6 bit column are MSB, 10 bit row are LSB */
+ return (crc_col << 10) | crc_row;
+}
+
+static void right_shift_array(char *arr, u8 len, u8 shift_bits)
+{
+ u8 byte_mask = 0xff >> shift_bits;
+ int i;
+
+ if (WARN_ON(!shift_bits || shift_bits >= 8))
+ return;
+
+ for (i = len - 1; i >= 0; i--) {
+ /* The first iteration looks like out-of-bounds access,
+ * but actually references a buffer that the array is shifted
+ * into. This move is legal as we never send the last chunk to
+ * this function.
+ */
+ arr[i + 1] &= byte_mask;
+ arr[i + 1] |= arr[i] << (8 - shift_bits);
+ arr[i] = arr[i] >> shift_bits;
+ }
+}
+
+static void mlxsw_sp4_bf_key_shift_chunks(u8 chunk_count, char *output)
+{
+ /* The chunks are suppoosed to be continuous, with no padding.
+ * Since region ID and eRP ID use 14 bits, and not fully 2 bytes,
+ * and in Spectrum-4 there is no padding, it is necessary to shift some
+ * chunks 2 bits right.
+ */
+ switch (chunk_count) {
+ case 2:
+ /* The chunks are copied as follow:
+ * +-------------+-----------------+
+ * | Chunk 0 | Chunk 1 |
+ * | IDs | keys |(**) IDs | keys |
+ * +-------------+-----------------+
+ * In (**), there are two unused bits, therefore, chunk 0 needs
+ * to be shifted two bits right.
+ */
+ right_shift_array(output, MLXSW_SP4_BLOOM_KEY_CHUNK_BYTES, 2);
+ break;
+ case 3:
+ /* The chunks are copied as follow:
+ * +-------------+-----------------+-----------------+
+ * | Chunk 0 | Chunk 1 | Chunk 2 |
+ * | IDs | keys |(**) IDs | keys |(**) IDs | keys |
+ * +-------------+-----------------+-----------------+
+ * In (**), there are two unused bits, therefore, chunk 1 needs
+ * to be shifted two bits right and chunk 0 needs to be shifted
+ * four bits right.
+ */
+ right_shift_array(output + MLXSW_SP4_BLOOM_KEY_CHUNK_BYTES,
+ MLXSW_SP4_BLOOM_KEY_CHUNK_BYTES, 2);
+ right_shift_array(output, MLXSW_SP4_BLOOM_KEY_CHUNK_BYTES, 4);
+ break;
+ default:
+ WARN_ON(chunk_count > MLXSW_BLOOM_KEY_CHUNKS);
+ }
+}
+
+static void
+mlxsw_sp4_acl_bf_key_encode(struct mlxsw_sp_acl_atcam_region *aregion,
+ struct mlxsw_sp_acl_atcam_entry *aentry,
+ char *output, u8 *len)
+{
+ struct mlxsw_afk_key_info *key_info = aregion->region->key_info;
+ u8 block_count = mlxsw_afk_key_info_blocks_count_get(key_info);
+ u8 chunk_count = 1 + ((block_count - 1) >> 2);
+
+ __mlxsw_sp_acl_bf_key_encode(aregion, aentry, output, len,
+ MLXSW_BLOOM_KEY_CHUNKS,
+ MLXSW_SP4_BLOOM_CHUNK_PAD_BYTES,
+ MLXSW_SP4_BLOOM_CHUNK_KEY_OFFSET,
+ MLXSW_SP4_BLOOM_CHUNK_KEY_BYTES,
+ MLXSW_SP4_BLOOM_KEY_CHUNK_BYTES);
+ mlxsw_sp4_bf_key_shift_chunks(chunk_count, output);
+}
+
+static unsigned int
+mlxsw_sp4_acl_bf_index_get(struct mlxsw_sp_acl_bf *bf,
+ struct mlxsw_sp_acl_atcam_region *aregion,
+ struct mlxsw_sp_acl_atcam_entry *aentry)
+{
+ char bf_key[MLXSW_SP4_BLOOM_KEY_LEN] = {};
+ u8 bf_size;
+
+ mlxsw_sp4_acl_bf_key_encode(aregion, aentry, bf_key, &bf_size);
+ return mlxsw_sp4_acl_bf_crc(bf_key, bf_size);
}
static unsigned int
@@ -150,18 +407,6 @@ mlxsw_sp_acl_bf_rule_count_index_get(struct mlxsw_sp_acl_bf *bf,
return erp_bank * bf->bank_size + bf_index;
}
-static unsigned int
-mlxsw_sp_acl_bf_index_get(struct mlxsw_sp_acl_bf *bf,
- struct mlxsw_sp_acl_atcam_region *aregion,
- struct mlxsw_sp_acl_atcam_entry *aentry)
-{
- char bf_key[MLXSW_BLOOM_KEY_LEN];
- u8 bf_size;
-
- mlxsw_sp_acl_bf_key_encode(aregion, aentry, bf_key, &bf_size);
- return mlxsw_sp_acl_bf_crc(bf_key, bf_size);
-}
-
int
mlxsw_sp_acl_bf_entry_add(struct mlxsw_sp *mlxsw_sp,
struct mlxsw_sp_acl_bf *bf,
@@ -176,7 +421,7 @@ mlxsw_sp_acl_bf_entry_add(struct mlxsw_sp *mlxsw_sp,
mutex_lock(&bf->lock);
- bf_index = mlxsw_sp_acl_bf_index_get(bf, aregion, aentry);
+ bf_index = mlxsw_sp->acl_bf_ops->index_get(bf, aregion, aentry);
rule_index = mlxsw_sp_acl_bf_rule_count_index_get(bf, erp_bank,
bf_index);
@@ -219,7 +464,7 @@ mlxsw_sp_acl_bf_entry_del(struct mlxsw_sp *mlxsw_sp,
mutex_lock(&bf->lock);
- bf_index = mlxsw_sp_acl_bf_index_get(bf, aregion, aentry);
+ bf_index = mlxsw_sp->acl_bf_ops->index_get(bf, aregion, aentry);
rule_index = mlxsw_sp_acl_bf_rule_count_index_get(bf, erp_bank,
bf_index);
@@ -267,3 +512,11 @@ void mlxsw_sp_acl_bf_fini(struct mlxsw_sp_acl_bf *bf)
mutex_destroy(&bf->lock);
kfree(bf);
}
+
+const struct mlxsw_sp_acl_bf_ops mlxsw_sp2_acl_bf_ops = {
+ .index_get = mlxsw_sp2_acl_bf_index_get,
+};
+
+const struct mlxsw_sp_acl_bf_ops mlxsw_sp4_acl_bf_ops = {
+ .index_get = mlxsw_sp4_acl_bf_index_get,
+};
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_acl_flex_actions.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_acl_flex_actions.c
index c72aa38..5080659 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_acl_flex_actions.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_acl_flex_actions.c
@@ -83,7 +83,7 @@ static int mlxsw_sp2_act_kvdl_set_activity_get(void *priv, u32 kvdl_index,
}
static int mlxsw_sp_act_kvdl_fwd_entry_add(void *priv, u32 *p_kvdl_index,
- u8 local_port)
+ u16 local_port)
{
struct mlxsw_sp *mlxsw_sp = priv;
char ppbs_pl[MLXSW_REG_PPBS_LEN];
@@ -132,7 +132,7 @@ mlxsw_sp_act_counter_index_put(void *priv, unsigned int counter_index)
}
static int
-mlxsw_sp_act_mirror_add(void *priv, u8 local_in_port,
+mlxsw_sp_act_mirror_add(void *priv, u16 local_in_port,
const struct net_device *out_dev,
bool ingress, int *p_span_id)
{
@@ -159,7 +159,7 @@ mlxsw_sp_act_mirror_add(void *priv, u8 local_in_port,
}
static void
-mlxsw_sp_act_mirror_del(void *priv, u8 local_in_port, int span_id, bool ingress)
+mlxsw_sp_act_mirror_del(void *priv, u16 local_in_port, int span_id, bool ingress)
{
struct mlxsw_sp_port *mlxsw_sp_port;
struct mlxsw_sp *mlxsw_sp = priv;
@@ -192,7 +192,7 @@ static void mlxsw_sp_act_policer_del(void *priv, u16 policer_index)
policer_index);
}
-static int mlxsw_sp1_act_sampler_add(void *priv, u8 local_port,
+static int mlxsw_sp1_act_sampler_add(void *priv, u16 local_port,
struct psample_group *psample_group,
u32 rate, u32 trunc_size, bool truncate,
bool ingress, int *p_span_id,
@@ -202,7 +202,7 @@ static int mlxsw_sp1_act_sampler_add(void *priv, u8 local_port,
return -EOPNOTSUPP;
}
-static void mlxsw_sp1_act_sampler_del(void *priv, u8 local_port, int span_id,
+static void mlxsw_sp1_act_sampler_del(void *priv, u16 local_port, int span_id,
bool ingress)
{
WARN_ON_ONCE(1);
@@ -224,7 +224,7 @@ const struct mlxsw_afa_ops mlxsw_sp1_act_afa_ops = {
.sampler_del = mlxsw_sp1_act_sampler_del,
};
-static int mlxsw_sp2_act_sampler_add(void *priv, u8 local_port,
+static int mlxsw_sp2_act_sampler_add(void *priv, u16 local_port,
struct psample_group *psample_group,
u32 rate, u32 trunc_size, bool truncate,
bool ingress, int *p_span_id,
@@ -272,7 +272,7 @@ static int mlxsw_sp2_act_sampler_add(void *priv, u8 local_port,
return err;
}
-static void mlxsw_sp2_act_sampler_del(void *priv, u8 local_port, int span_id,
+static void mlxsw_sp2_act_sampler_del(void *priv, u16 local_port, int span_id,
bool ingress)
{
struct mlxsw_sp_sample_trigger trigger = {
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_acl_flex_keys.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_acl_flex_keys.c
index 279c241..00c32320f 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_acl_flex_keys.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_acl_flex_keys.c
@@ -168,8 +168,8 @@ static struct mlxsw_afk_element_inst mlxsw_sp_afk_element_info_ipv4_2[] = {
};
static struct mlxsw_afk_element_inst mlxsw_sp_afk_element_info_ipv4_4[] = {
- MLXSW_AFK_ELEMENT_INST_U32(VIRT_ROUTER_0_7, 0x04, 24, 8),
- MLXSW_AFK_ELEMENT_INST_U32(VIRT_ROUTER_8_10, 0x00, 0, 3),
+ MLXSW_AFK_ELEMENT_INST_U32(VIRT_ROUTER_LSB, 0x04, 24, 8),
+ MLXSW_AFK_ELEMENT_INST_U32(VIRT_ROUTER_MSB, 0x00, 0, 3),
};
static struct mlxsw_afk_element_inst mlxsw_sp_afk_element_info_ipv6_0[] = {
@@ -311,3 +311,45 @@ const struct mlxsw_afk_ops mlxsw_sp2_afk_ops = {
.encode_block = mlxsw_sp2_afk_encode_block,
.clear_block = mlxsw_sp2_afk_clear_block,
};
+
+static struct mlxsw_afk_element_inst mlxsw_sp_afk_element_info_mac_5b[] = {
+ MLXSW_AFK_ELEMENT_INST_U32(VID, 0x04, 18, 12),
+ MLXSW_AFK_ELEMENT_INST_EXT_U32(SRC_SYS_PORT, 0x04, 0, 9, -1, true), /* RX_ACL_SYSTEM_PORT */
+};
+
+static struct mlxsw_afk_element_inst mlxsw_sp_afk_element_info_ipv4_4b[] = {
+ MLXSW_AFK_ELEMENT_INST_U32(VIRT_ROUTER_LSB, 0x04, 13, 8),
+ MLXSW_AFK_ELEMENT_INST_EXT_U32(VIRT_ROUTER_MSB, 0x04, 21, 4, 0, true),
+};
+
+static struct mlxsw_afk_element_inst mlxsw_sp_afk_element_info_ipv6_2b[] = {
+ MLXSW_AFK_ELEMENT_INST_BUF(DST_IP_96_127, 0x04, 4),
+};
+
+static const struct mlxsw_afk_block mlxsw_sp4_afk_blocks[] = {
+ MLXSW_AFK_BLOCK(0x10, mlxsw_sp_afk_element_info_mac_0),
+ MLXSW_AFK_BLOCK(0x11, mlxsw_sp_afk_element_info_mac_1),
+ MLXSW_AFK_BLOCK(0x12, mlxsw_sp_afk_element_info_mac_2),
+ MLXSW_AFK_BLOCK(0x13, mlxsw_sp_afk_element_info_mac_3),
+ MLXSW_AFK_BLOCK(0x14, mlxsw_sp_afk_element_info_mac_4),
+ MLXSW_AFK_BLOCK(0x1A, mlxsw_sp_afk_element_info_mac_5b),
+ MLXSW_AFK_BLOCK(0x38, mlxsw_sp_afk_element_info_ipv4_0),
+ MLXSW_AFK_BLOCK(0x39, mlxsw_sp_afk_element_info_ipv4_1),
+ MLXSW_AFK_BLOCK(0x3A, mlxsw_sp_afk_element_info_ipv4_2),
+ MLXSW_AFK_BLOCK(0x35, mlxsw_sp_afk_element_info_ipv4_4b),
+ MLXSW_AFK_BLOCK(0x40, mlxsw_sp_afk_element_info_ipv6_0),
+ MLXSW_AFK_BLOCK(0x41, mlxsw_sp_afk_element_info_ipv6_1),
+ MLXSW_AFK_BLOCK(0x47, mlxsw_sp_afk_element_info_ipv6_2b),
+ MLXSW_AFK_BLOCK(0x43, mlxsw_sp_afk_element_info_ipv6_3),
+ MLXSW_AFK_BLOCK(0x44, mlxsw_sp_afk_element_info_ipv6_4),
+ MLXSW_AFK_BLOCK(0x45, mlxsw_sp_afk_element_info_ipv6_5),
+ MLXSW_AFK_BLOCK(0x90, mlxsw_sp_afk_element_info_l4_0),
+ MLXSW_AFK_BLOCK(0x92, mlxsw_sp_afk_element_info_l4_2),
+};
+
+const struct mlxsw_afk_ops mlxsw_sp4_afk_ops = {
+ .blocks = mlxsw_sp4_afk_blocks,
+ .blocks_count = ARRAY_SIZE(mlxsw_sp4_afk_blocks),
+ .encode_block = mlxsw_sp2_afk_encode_block,
+ .clear_block = mlxsw_sp2_afk_clear_block,
+};
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_acl_tcam.h b/drivers/net/ethernet/mellanox/mlxsw/spectrum_acl_tcam.h
index a41df10..edbbc89 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_acl_tcam.h
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_acl_tcam.h
@@ -287,6 +287,12 @@ void mlxsw_sp_acl_erps_fini(struct mlxsw_sp *mlxsw_sp,
struct mlxsw_sp_acl_bf;
+struct mlxsw_sp_acl_bf_ops {
+ unsigned int (*index_get)(struct mlxsw_sp_acl_bf *bf,
+ struct mlxsw_sp_acl_atcam_region *aregion,
+ struct mlxsw_sp_acl_atcam_entry *aentry);
+};
+
int
mlxsw_sp_acl_bf_entry_add(struct mlxsw_sp *mlxsw_sp,
struct mlxsw_sp_acl_bf *bf,
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_buffers.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_buffers.c
index d78cf5a..98f26f5 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_buffers.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_buffers.c
@@ -160,7 +160,7 @@ static bool mlxsw_sp_sb_cm_exists(u8 pg_buff, enum mlxsw_reg_sbxx_dir dir)
}
static struct mlxsw_sp_sb_cm *mlxsw_sp_sb_cm_get(struct mlxsw_sp *mlxsw_sp,
- u8 local_port, u8 pg_buff,
+ u16 local_port, u8 pg_buff,
enum mlxsw_reg_sbxx_dir dir)
{
struct mlxsw_sp_sb_port *sb_port = &mlxsw_sp->sb->ports[local_port];
@@ -173,7 +173,7 @@ static struct mlxsw_sp_sb_cm *mlxsw_sp_sb_cm_get(struct mlxsw_sp *mlxsw_sp,
}
static struct mlxsw_sp_sb_pm *mlxsw_sp_sb_pm_get(struct mlxsw_sp *mlxsw_sp,
- u8 local_port, u16 pool_index)
+ u16 local_port, u16 pool_index)
{
return &mlxsw_sp->sb->ports[local_port].pms[pool_index];
}
@@ -202,7 +202,7 @@ static int mlxsw_sp_sb_pr_write(struct mlxsw_sp *mlxsw_sp, u16 pool_index,
return 0;
}
-static int mlxsw_sp_sb_cm_write(struct mlxsw_sp *mlxsw_sp, u8 local_port,
+static int mlxsw_sp_sb_cm_write(struct mlxsw_sp *mlxsw_sp, u16 local_port,
u8 pg_buff, u32 min_buff, u32 max_buff,
bool infi_max, u16 pool_index)
{
@@ -232,7 +232,7 @@ static int mlxsw_sp_sb_cm_write(struct mlxsw_sp *mlxsw_sp, u8 local_port,
return 0;
}
-static int mlxsw_sp_sb_pm_write(struct mlxsw_sp *mlxsw_sp, u8 local_port,
+static int mlxsw_sp_sb_pm_write(struct mlxsw_sp *mlxsw_sp, u16 local_port,
u16 pool_index, u32 min_buff, u32 max_buff)
{
const struct mlxsw_sp_sb_pool_des *des =
@@ -253,7 +253,7 @@ static int mlxsw_sp_sb_pm_write(struct mlxsw_sp *mlxsw_sp, u8 local_port,
return 0;
}
-static int mlxsw_sp_sb_pm_occ_clear(struct mlxsw_sp *mlxsw_sp, u8 local_port,
+static int mlxsw_sp_sb_pm_occ_clear(struct mlxsw_sp *mlxsw_sp, u16 local_port,
u16 pool_index, struct list_head *bulk_list)
{
const struct mlxsw_sp_sb_pool_des *des =
@@ -279,7 +279,7 @@ static void mlxsw_sp_sb_pm_occ_query_cb(struct mlxsw_core *mlxsw_core,
mlxsw_reg_sbpm_unpack(sbpm_pl, &pm->occ.cur, &pm->occ.max);
}
-static int mlxsw_sp_sb_pm_occ_query(struct mlxsw_sp *mlxsw_sp, u8 local_port,
+static int mlxsw_sp_sb_pm_occ_query(struct mlxsw_sp *mlxsw_sp, u16 local_port,
u16 pool_index, struct list_head *bulk_list)
{
const struct mlxsw_sp_sb_pool_des *des =
@@ -919,7 +919,7 @@ mlxsw_sp_sb_pool_is_static(struct mlxsw_sp *mlxsw_sp, u16 pool_index)
return pr->mode == MLXSW_REG_SBPR_MODE_STATIC;
}
-static int __mlxsw_sp_sb_cms_init(struct mlxsw_sp *mlxsw_sp, u8 local_port,
+static int __mlxsw_sp_sb_cms_init(struct mlxsw_sp *mlxsw_sp, u16 local_port,
enum mlxsw_reg_sbxx_dir dir,
const struct mlxsw_sp_sb_cm *cms,
size_t cms_len)
@@ -1037,7 +1037,7 @@ static const struct mlxsw_sp_sb_pm mlxsw_sp_cpu_port_sb_pms[] = {
MLXSW_SP_SB_PM(0, MLXSW_REG_SBXX_DYN_MAX_BUFF_MAX),
};
-static int mlxsw_sp_sb_pms_init(struct mlxsw_sp *mlxsw_sp, u8 local_port,
+static int mlxsw_sp_sb_pms_init(struct mlxsw_sp *mlxsw_sp, u16 local_port,
const struct mlxsw_sp_sb_pm *pms,
bool skip_ingress)
{
@@ -1416,7 +1416,7 @@ int mlxsw_sp_sb_port_pool_get(struct mlxsw_core_port *mlxsw_core_port,
struct mlxsw_sp_port *mlxsw_sp_port =
mlxsw_core_port_driver_priv(mlxsw_core_port);
struct mlxsw_sp *mlxsw_sp = mlxsw_sp_port->mlxsw_sp;
- u8 local_port = mlxsw_sp_port->local_port;
+ u16 local_port = mlxsw_sp_port->local_port;
struct mlxsw_sp_sb_pm *pm = mlxsw_sp_sb_pm_get(mlxsw_sp, local_port,
pool_index);
@@ -1432,7 +1432,7 @@ int mlxsw_sp_sb_port_pool_set(struct mlxsw_core_port *mlxsw_core_port,
struct mlxsw_sp_port *mlxsw_sp_port =
mlxsw_core_port_driver_priv(mlxsw_core_port);
struct mlxsw_sp *mlxsw_sp = mlxsw_sp_port->mlxsw_sp;
- u8 local_port = mlxsw_sp_port->local_port;
+ u16 local_port = mlxsw_sp_port->local_port;
u32 max_buff;
int err;
@@ -1458,7 +1458,7 @@ int mlxsw_sp_sb_tc_pool_bind_get(struct mlxsw_core_port *mlxsw_core_port,
struct mlxsw_sp_port *mlxsw_sp_port =
mlxsw_core_port_driver_priv(mlxsw_core_port);
struct mlxsw_sp *mlxsw_sp = mlxsw_sp_port->mlxsw_sp;
- u8 local_port = mlxsw_sp_port->local_port;
+ u16 local_port = mlxsw_sp_port->local_port;
u8 pg_buff = tc_index;
enum mlxsw_reg_sbxx_dir dir = (enum mlxsw_reg_sbxx_dir) pool_type;
struct mlxsw_sp_sb_cm *cm = mlxsw_sp_sb_cm_get(mlxsw_sp, local_port,
@@ -1479,7 +1479,7 @@ int mlxsw_sp_sb_tc_pool_bind_set(struct mlxsw_core_port *mlxsw_core_port,
struct mlxsw_sp_port *mlxsw_sp_port =
mlxsw_core_port_driver_priv(mlxsw_core_port);
struct mlxsw_sp *mlxsw_sp = mlxsw_sp_port->mlxsw_sp;
- u8 local_port = mlxsw_sp_port->local_port;
+ u16 local_port = mlxsw_sp_port->local_port;
const struct mlxsw_sp_sb_cm *cm;
u8 pg_buff = tc_index;
enum mlxsw_reg_sbxx_dir dir = (enum mlxsw_reg_sbxx_dir) pool_type;
@@ -1526,7 +1526,7 @@ int mlxsw_sp_sb_tc_pool_bind_set(struct mlxsw_core_port *mlxsw_core_port,
struct mlxsw_sp_sb_sr_occ_query_cb_ctx {
u8 masked_count;
- u8 local_port_1;
+ u16 local_port_1;
};
static void mlxsw_sp_sb_sr_occ_query_cb(struct mlxsw_core *mlxsw_core,
@@ -1536,7 +1536,7 @@ static void mlxsw_sp_sb_sr_occ_query_cb(struct mlxsw_core *mlxsw_core,
struct mlxsw_sp *mlxsw_sp = mlxsw_core_driver_priv(mlxsw_core);
struct mlxsw_sp_sb_sr_occ_query_cb_ctx cb_ctx;
u8 masked_count;
- u8 local_port;
+ u16 local_port;
int rec_index = 0;
struct mlxsw_sp_sb_cm *cm;
int i;
@@ -1582,13 +1582,12 @@ int mlxsw_sp_sb_occ_snapshot(struct mlxsw_core *mlxsw_core,
unsigned int sb_index)
{
struct mlxsw_sp *mlxsw_sp = mlxsw_core_driver_priv(mlxsw_core);
+ u16 local_port, local_port_1, last_local_port;
struct mlxsw_sp_sb_sr_occ_query_cb_ctx cb_ctx;
+ u8 masked_count, current_page = 0;
unsigned long cb_priv = 0;
LIST_HEAD(bulk_list);
char *sbsr_pl;
- u8 masked_count;
- u8 local_port_1;
- u8 local_port;
int i;
int err;
int err2;
@@ -1602,6 +1601,10 @@ int mlxsw_sp_sb_occ_snapshot(struct mlxsw_core *mlxsw_core,
local_port_1 = local_port;
masked_count = 0;
mlxsw_reg_sbsr_pack(sbsr_pl, false);
+ mlxsw_reg_sbsr_port_page_set(sbsr_pl, current_page);
+ last_local_port = current_page * MLXSW_REG_SBSR_NUM_PORTS_IN_PAGE +
+ MLXSW_REG_SBSR_NUM_PORTS_IN_PAGE - 1;
+
for (i = 0; i < MLXSW_SP_SB_ING_TC_COUNT; i++)
mlxsw_reg_sbsr_pg_buff_mask_set(sbsr_pl, i, 1);
for (i = 0; i < MLXSW_SP_SB_EG_TC_COUNT; i++)
@@ -1609,6 +1612,10 @@ int mlxsw_sp_sb_occ_snapshot(struct mlxsw_core *mlxsw_core,
for (; local_port < mlxsw_core_max_ports(mlxsw_core); local_port++) {
if (!mlxsw_sp->ports[local_port])
continue;
+ if (local_port > last_local_port) {
+ current_page++;
+ goto do_query;
+ }
if (local_port != MLXSW_PORT_CPU_PORT) {
/* Ingress quotas are not supported for the CPU port */
mlxsw_reg_sbsr_ingress_port_mask_set(sbsr_pl,
@@ -1651,10 +1658,11 @@ int mlxsw_sp_sb_occ_max_clear(struct mlxsw_core *mlxsw_core,
unsigned int sb_index)
{
struct mlxsw_sp *mlxsw_sp = mlxsw_core_driver_priv(mlxsw_core);
+ u16 local_port, last_local_port;
LIST_HEAD(bulk_list);
- char *sbsr_pl;
unsigned int masked_count;
- u8 local_port;
+ u8 current_page = 0;
+ char *sbsr_pl;
int i;
int err;
int err2;
@@ -1667,6 +1675,10 @@ int mlxsw_sp_sb_occ_max_clear(struct mlxsw_core *mlxsw_core,
next_batch:
masked_count = 0;
mlxsw_reg_sbsr_pack(sbsr_pl, true);
+ mlxsw_reg_sbsr_port_page_set(sbsr_pl, current_page);
+ last_local_port = current_page * MLXSW_REG_SBSR_NUM_PORTS_IN_PAGE +
+ MLXSW_REG_SBSR_NUM_PORTS_IN_PAGE - 1;
+
for (i = 0; i < MLXSW_SP_SB_ING_TC_COUNT; i++)
mlxsw_reg_sbsr_pg_buff_mask_set(sbsr_pl, i, 1);
for (i = 0; i < MLXSW_SP_SB_EG_TC_COUNT; i++)
@@ -1674,6 +1686,10 @@ int mlxsw_sp_sb_occ_max_clear(struct mlxsw_core *mlxsw_core,
for (; local_port < mlxsw_core_max_ports(mlxsw_core); local_port++) {
if (!mlxsw_sp->ports[local_port])
continue;
+ if (local_port > last_local_port) {
+ current_page++;
+ goto do_query;
+ }
if (local_port != MLXSW_PORT_CPU_PORT) {
/* Ingress quotas are not supported for the CPU port */
mlxsw_reg_sbsr_ingress_port_mask_set(sbsr_pl,
@@ -1715,7 +1731,7 @@ int mlxsw_sp_sb_occ_port_pool_get(struct mlxsw_core_port *mlxsw_core_port,
struct mlxsw_sp_port *mlxsw_sp_port =
mlxsw_core_port_driver_priv(mlxsw_core_port);
struct mlxsw_sp *mlxsw_sp = mlxsw_sp_port->mlxsw_sp;
- u8 local_port = mlxsw_sp_port->local_port;
+ u16 local_port = mlxsw_sp_port->local_port;
struct mlxsw_sp_sb_pm *pm = mlxsw_sp_sb_pm_get(mlxsw_sp, local_port,
pool_index);
@@ -1732,7 +1748,7 @@ int mlxsw_sp_sb_occ_tc_port_bind_get(struct mlxsw_core_port *mlxsw_core_port,
struct mlxsw_sp_port *mlxsw_sp_port =
mlxsw_core_port_driver_priv(mlxsw_core_port);
struct mlxsw_sp *mlxsw_sp = mlxsw_sp_port->mlxsw_sp;
- u8 local_port = mlxsw_sp_port->local_port;
+ u16 local_port = mlxsw_sp_port->local_port;
u8 pg_buff = tc_index;
enum mlxsw_reg_sbxx_dir dir = (enum mlxsw_reg_sbxx_dir) pool_type;
struct mlxsw_sp_sb_cm *cm = mlxsw_sp_sb_cm_get(mlxsw_sp, local_port,
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_ethtool.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_ethtool.c
index 84d4460..2053071 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_ethtool.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_ethtool.c
@@ -1491,7 +1491,7 @@ static u32 mlxsw_sp1_to_ptys_speed_lanes(struct mlxsw_sp *mlxsw_sp, u8 width,
static void
mlxsw_sp1_reg_ptys_eth_pack(struct mlxsw_sp *mlxsw_sp, char *payload,
- u8 local_port, u32 proto_admin, bool autoneg)
+ u16 local_port, u32 proto_admin, bool autoneg)
{
mlxsw_reg_ptys_eth_pack(payload, local_port, proto_admin, autoneg);
}
@@ -1969,7 +1969,7 @@ static u32 mlxsw_sp2_to_ptys_speed_lanes(struct mlxsw_sp *mlxsw_sp, u8 width,
static void
mlxsw_sp2_reg_ptys_eth_pack(struct mlxsw_sp *mlxsw_sp, char *payload,
- u8 local_port, u32 proto_admin,
+ u16 local_port, u32 proto_admin,
bool autoneg)
{
mlxsw_reg_ptys_ext_eth_pack(payload, local_port, proto_admin, autoneg);
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_fid.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_fid.c
index 004c422..ce80931 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_fid.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_fid.c
@@ -317,13 +317,13 @@ mlxsw_sp_fid_flood_table_lookup(const struct mlxsw_sp_fid *fid,
}
int mlxsw_sp_fid_flood_set(struct mlxsw_sp_fid *fid,
- enum mlxsw_sp_flood_type packet_type, u8 local_port,
+ enum mlxsw_sp_flood_type packet_type, u16 local_port,
bool member)
{
struct mlxsw_sp_fid_family *fid_family = fid->fid_family;
const struct mlxsw_sp_fid_ops *ops = fid_family->ops;
const struct mlxsw_sp_flood_table *flood_table;
- char *sftr_pl;
+ char *sftr2_pl;
int err;
if (WARN_ON(!fid_family->flood_tables || !ops->flood_index))
@@ -333,16 +333,16 @@ int mlxsw_sp_fid_flood_set(struct mlxsw_sp_fid *fid,
if (!flood_table)
return -ESRCH;
- sftr_pl = kmalloc(MLXSW_REG_SFTR_LEN, GFP_KERNEL);
- if (!sftr_pl)
+ sftr2_pl = kmalloc(MLXSW_REG_SFTR2_LEN, GFP_KERNEL);
+ if (!sftr2_pl)
return -ENOMEM;
- mlxsw_reg_sftr_pack(sftr_pl, flood_table->table_index,
- ops->flood_index(fid), flood_table->table_type, 1,
- local_port, member);
- err = mlxsw_reg_write(fid_family->mlxsw_sp->core, MLXSW_REG(sftr),
- sftr_pl);
- kfree(sftr_pl);
+ mlxsw_reg_sftr2_pack(sftr2_pl, flood_table->table_index,
+ ops->flood_index(fid), flood_table->table_type, 1,
+ local_port, member);
+ err = mlxsw_reg_write(fid_family->mlxsw_sp->core, MLXSW_REG(sftr2),
+ sftr2_pl);
+ kfree(sftr2_pl);
return err;
}
@@ -439,7 +439,7 @@ static int mlxsw_sp_fid_vni_op(struct mlxsw_sp *mlxsw_sp, u16 fid_index,
}
static int __mlxsw_sp_fid_port_vid_map(struct mlxsw_sp *mlxsw_sp, u16 fid_index,
- u8 local_port, u16 vid, bool valid)
+ u16 local_port, u16 vid, bool valid)
{
enum mlxsw_reg_svfa_mt mt = MLXSW_REG_SVFA_MT_PORT_VID_TO_FID;
char svfa_pl[MLXSW_REG_SVFA_LEN];
@@ -573,7 +573,7 @@ static int mlxsw_sp_fid_8021d_port_vid_map(struct mlxsw_sp_fid *fid,
u16 vid)
{
struct mlxsw_sp *mlxsw_sp = mlxsw_sp_port->mlxsw_sp;
- u8 local_port = mlxsw_sp_port->local_port;
+ u16 local_port = mlxsw_sp_port->local_port;
int err;
err = __mlxsw_sp_fid_port_vid_map(mlxsw_sp, fid->fid_index,
@@ -601,7 +601,7 @@ mlxsw_sp_fid_8021d_port_vid_unmap(struct mlxsw_sp_fid *fid,
struct mlxsw_sp_port *mlxsw_sp_port, u16 vid)
{
struct mlxsw_sp *mlxsw_sp = mlxsw_sp_port->mlxsw_sp;
- u8 local_port = mlxsw_sp_port->local_port;
+ u16 local_port = mlxsw_sp_port->local_port;
if (mlxsw_sp->fid_core->port_fid_mappings[local_port] == 1)
mlxsw_sp_port_vlan_mode_trans(mlxsw_sp_port);
@@ -784,7 +784,7 @@ static int mlxsw_sp_fid_rfid_port_vid_map(struct mlxsw_sp_fid *fid,
u16 vid)
{
struct mlxsw_sp *mlxsw_sp = mlxsw_sp_port->mlxsw_sp;
- u8 local_port = mlxsw_sp_port->local_port;
+ u16 local_port = mlxsw_sp_port->local_port;
int err;
/* We only need to transition the port to virtual mode since
@@ -808,7 +808,7 @@ mlxsw_sp_fid_rfid_port_vid_unmap(struct mlxsw_sp_fid *fid,
struct mlxsw_sp_port *mlxsw_sp_port, u16 vid)
{
struct mlxsw_sp *mlxsw_sp = mlxsw_sp_port->mlxsw_sp;
- u8 local_port = mlxsw_sp_port->local_port;
+ u16 local_port = mlxsw_sp_port->local_port;
if (mlxsw_sp->fid_core->port_fid_mappings[local_port] == 1)
mlxsw_sp_port_vlan_mode_trans(mlxsw_sp_port);
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_flower.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_flower.c
index be3791c..bb417db 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_flower.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_flower.c
@@ -203,7 +203,7 @@ static int mlxsw_sp_flower_parse_actions(struct mlxsw_sp *mlxsw_sp,
*/
burst = roundup_pow_of_two(act->police.burst);
err = mlxsw_sp_acl_rulei_act_police(mlxsw_sp, rulei,
- act->police.index,
+ act->hw_index,
act->police.rate_bytes_ps,
burst, extack);
if (err)
@@ -508,7 +508,8 @@ static int mlxsw_sp_flower_parse(struct mlxsw_sp *mlxsw_sp,
struct flow_match_vlan match;
flow_rule_match_vlan(rule, &match);
- if (mlxsw_sp_flow_block_is_egress_bound(block)) {
+ if (mlxsw_sp_flow_block_is_egress_bound(block) &&
+ match.mask->vlan_id) {
NL_SET_ERR_MSG_MOD(f->common.extack, "vlan_id key is not supported on egress");
return -EOPNOTSUPP;
}
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_ipip.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_ipip.c
index ad3926d..01cf5a6 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_ipip.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_ipip.c
@@ -568,37 +568,21 @@ static int
mlxsw_sp2_ipip_rem_addr_set_gre6(struct mlxsw_sp *mlxsw_sp,
struct mlxsw_sp_ipip_entry *ipip_entry)
{
- char rips_pl[MLXSW_REG_RIPS_LEN];
struct __ip6_tnl_parm parms6;
- int err;
-
- err = mlxsw_sp_kvdl_alloc(mlxsw_sp,
- MLXSW_SP_KVDL_ENTRY_TYPE_IPV6_ADDRESS, 1,
- &ipip_entry->dip_kvdl_index);
- if (err)
- return err;
parms6 = mlxsw_sp_ipip_netdev_parms6(ipip_entry->ol_dev);
- mlxsw_reg_rips_pack(rips_pl, ipip_entry->dip_kvdl_index,
- &parms6.raddr);
- err = mlxsw_reg_write(mlxsw_sp->core, MLXSW_REG(rips), rips_pl);
- if (err)
- goto err_rips_write;
-
- return 0;
-
-err_rips_write:
- mlxsw_sp_kvdl_free(mlxsw_sp, MLXSW_SP_KVDL_ENTRY_TYPE_IPV6_ADDRESS, 1,
- ipip_entry->dip_kvdl_index);
- return err;
+ return mlxsw_sp_ipv6_addr_kvdl_index_get(mlxsw_sp, &parms6.raddr,
+ &ipip_entry->dip_kvdl_index);
}
static void
mlxsw_sp2_ipip_rem_addr_unset_gre6(struct mlxsw_sp *mlxsw_sp,
const struct mlxsw_sp_ipip_entry *ipip_entry)
{
- mlxsw_sp_kvdl_free(mlxsw_sp, MLXSW_SP_KVDL_ENTRY_TYPE_IPV6_ADDRESS, 1,
- ipip_entry->dip_kvdl_index);
+ struct __ip6_tnl_parm parms6;
+
+ parms6 = mlxsw_sp_ipip_netdev_parms6(ipip_entry->ol_dev);
+ mlxsw_sp_ipv6_addr_put(mlxsw_sp, &parms6.raddr);
}
static const struct mlxsw_sp_ipip_ops mlxsw_sp2_ipip_gre6_ops = {
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_nve.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_nve.c
index 9eba8fa..d2b57a0 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_nve.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_nve.c
@@ -130,15 +130,25 @@ mlxsw_sp_nve_mc_record_ipv6_entry_add(struct mlxsw_sp_nve_mc_record *mc_record,
struct mlxsw_sp_nve_mc_entry *mc_entry,
const union mlxsw_sp_l3addr *addr)
{
- WARN_ON(1);
+ u32 kvdl_index;
+ int err;
- return -EINVAL;
+ err = mlxsw_sp_ipv6_addr_kvdl_index_get(mc_record->mlxsw_sp,
+ &addr->addr6, &kvdl_index);
+ if (err)
+ return err;
+
+ mc_entry->ipv6_entry.addr6 = addr->addr6;
+ mc_entry->ipv6_entry.addr6_kvdl_index = kvdl_index;
+ return 0;
}
static void
mlxsw_sp_nve_mc_record_ipv6_entry_del(const struct mlxsw_sp_nve_mc_record *mc_record,
const struct mlxsw_sp_nve_mc_entry *mc_entry)
{
+ mlxsw_sp_ipv6_addr_put(mc_record->mlxsw_sp,
+ &mc_entry->ipv6_entry.addr6);
}
static void
@@ -787,6 +797,142 @@ static void mlxsw_sp_nve_fdb_clear_offload(struct mlxsw_sp *mlxsw_sp,
ops->fdb_clear_offload(nve_dev, vni);
}
+struct mlxsw_sp_nve_ipv6_ht_key {
+ u8 mac[ETH_ALEN];
+ u16 fid_index;
+};
+
+struct mlxsw_sp_nve_ipv6_ht_node {
+ struct rhash_head ht_node;
+ struct list_head list;
+ struct mlxsw_sp_nve_ipv6_ht_key key;
+ struct in6_addr addr6;
+};
+
+static const struct rhashtable_params mlxsw_sp_nve_ipv6_ht_params = {
+ .key_len = sizeof(struct mlxsw_sp_nve_ipv6_ht_key),
+ .key_offset = offsetof(struct mlxsw_sp_nve_ipv6_ht_node, key),
+ .head_offset = offsetof(struct mlxsw_sp_nve_ipv6_ht_node, ht_node),
+};
+
+int mlxsw_sp_nve_ipv6_addr_kvdl_set(struct mlxsw_sp *mlxsw_sp,
+ const struct in6_addr *addr6,
+ u32 *p_kvdl_index)
+{
+ return mlxsw_sp_ipv6_addr_kvdl_index_get(mlxsw_sp, addr6, p_kvdl_index);
+}
+
+void mlxsw_sp_nve_ipv6_addr_kvdl_unset(struct mlxsw_sp *mlxsw_sp,
+ const struct in6_addr *addr6)
+{
+ mlxsw_sp_ipv6_addr_put(mlxsw_sp, addr6);
+}
+
+static struct mlxsw_sp_nve_ipv6_ht_node *
+mlxsw_sp_nve_ipv6_ht_node_lookup(struct mlxsw_sp *mlxsw_sp, const char *mac,
+ u16 fid_index)
+{
+ struct mlxsw_sp_nve_ipv6_ht_key key = {};
+
+ ether_addr_copy(key.mac, mac);
+ key.fid_index = fid_index;
+ return rhashtable_lookup_fast(&mlxsw_sp->nve->ipv6_ht, &key,
+ mlxsw_sp_nve_ipv6_ht_params);
+}
+
+static int mlxsw_sp_nve_ipv6_ht_insert(struct mlxsw_sp *mlxsw_sp,
+ const char *mac, u16 fid_index,
+ const struct in6_addr *addr6)
+{
+ struct mlxsw_sp_nve_ipv6_ht_node *ipv6_ht_node;
+ struct mlxsw_sp_nve *nve = mlxsw_sp->nve;
+ int err;
+
+ ipv6_ht_node = kzalloc(sizeof(*ipv6_ht_node), GFP_KERNEL);
+ if (!ipv6_ht_node)
+ return -ENOMEM;
+
+ ether_addr_copy(ipv6_ht_node->key.mac, mac);
+ ipv6_ht_node->key.fid_index = fid_index;
+ ipv6_ht_node->addr6 = *addr6;
+
+ err = rhashtable_insert_fast(&nve->ipv6_ht, &ipv6_ht_node->ht_node,
+ mlxsw_sp_nve_ipv6_ht_params);
+ if (err)
+ goto err_rhashtable_insert;
+
+ list_add(&ipv6_ht_node->list, &nve->ipv6_addr_list);
+
+ return 0;
+
+err_rhashtable_insert:
+ kfree(ipv6_ht_node);
+ return err;
+}
+
+static void
+mlxsw_sp_nve_ipv6_ht_remove(struct mlxsw_sp *mlxsw_sp,
+ struct mlxsw_sp_nve_ipv6_ht_node *ipv6_ht_node)
+{
+ struct mlxsw_sp_nve *nve = mlxsw_sp->nve;
+
+ list_del(&ipv6_ht_node->list);
+ rhashtable_remove_fast(&nve->ipv6_ht, &ipv6_ht_node->ht_node,
+ mlxsw_sp_nve_ipv6_ht_params);
+ kfree(ipv6_ht_node);
+}
+
+int
+mlxsw_sp_nve_ipv6_addr_map_replace(struct mlxsw_sp *mlxsw_sp, const char *mac,
+ u16 fid_index,
+ const struct in6_addr *new_addr6)
+{
+ struct mlxsw_sp_nve_ipv6_ht_node *ipv6_ht_node;
+
+ ASSERT_RTNL();
+
+ ipv6_ht_node = mlxsw_sp_nve_ipv6_ht_node_lookup(mlxsw_sp, mac,
+ fid_index);
+ if (!ipv6_ht_node)
+ return mlxsw_sp_nve_ipv6_ht_insert(mlxsw_sp, mac, fid_index,
+ new_addr6);
+
+ mlxsw_sp_ipv6_addr_put(mlxsw_sp, &ipv6_ht_node->addr6);
+ ipv6_ht_node->addr6 = *new_addr6;
+ return 0;
+}
+
+void mlxsw_sp_nve_ipv6_addr_map_del(struct mlxsw_sp *mlxsw_sp, const char *mac,
+ u16 fid_index)
+{
+ struct mlxsw_sp_nve_ipv6_ht_node *ipv6_ht_node;
+
+ ASSERT_RTNL();
+
+ ipv6_ht_node = mlxsw_sp_nve_ipv6_ht_node_lookup(mlxsw_sp, mac,
+ fid_index);
+ if (WARN_ON(!ipv6_ht_node))
+ return;
+
+ mlxsw_sp_nve_ipv6_ht_remove(mlxsw_sp, ipv6_ht_node);
+}
+
+static void mlxsw_sp_nve_ipv6_addr_flush_by_fid(struct mlxsw_sp *mlxsw_sp,
+ u16 fid_index)
+{
+ struct mlxsw_sp_nve_ipv6_ht_node *ipv6_ht_node, *tmp;
+ struct mlxsw_sp_nve *nve = mlxsw_sp->nve;
+
+ list_for_each_entry_safe(ipv6_ht_node, tmp, &nve->ipv6_addr_list,
+ list) {
+ if (ipv6_ht_node->key.fid_index != fid_index)
+ continue;
+
+ mlxsw_sp_ipv6_addr_put(mlxsw_sp, &ipv6_ht_node->addr6);
+ mlxsw_sp_nve_ipv6_ht_remove(mlxsw_sp, ipv6_ht_node);
+ }
+}
+
int mlxsw_sp_nve_fid_enable(struct mlxsw_sp *mlxsw_sp, struct mlxsw_sp_fid *fid,
struct mlxsw_sp_nve_params *params,
struct netlink_ext_ack *extack)
@@ -845,6 +991,7 @@ void mlxsw_sp_nve_fid_disable(struct mlxsw_sp *mlxsw_sp,
mlxsw_sp_nve_flood_ip_flush(mlxsw_sp, fid);
mlxsw_sp_nve_fdb_flush_by_fid(mlxsw_sp, fid_index);
+ mlxsw_sp_nve_ipv6_addr_flush_by_fid(mlxsw_sp, fid_index);
if (WARN_ON(mlxsw_sp_fid_nve_ifindex(fid, &nve_ifindex) ||
mlxsw_sp_fid_vni(fid, &vni)))
@@ -981,7 +1128,13 @@ int mlxsw_sp_nve_init(struct mlxsw_sp *mlxsw_sp)
err = rhashtable_init(&nve->mc_list_ht,
&mlxsw_sp_nve_mc_list_ht_params);
if (err)
- goto err_rhashtable_init;
+ goto err_mc_rhashtable_init;
+
+ err = rhashtable_init(&nve->ipv6_ht, &mlxsw_sp_nve_ipv6_ht_params);
+ if (err)
+ goto err_ipv6_rhashtable_init;
+
+ INIT_LIST_HEAD(&nve->ipv6_addr_list);
err = mlxsw_sp_nve_qos_init(mlxsw_sp);
if (err)
@@ -1000,8 +1153,10 @@ int mlxsw_sp_nve_init(struct mlxsw_sp *mlxsw_sp)
err_nve_resources_query:
err_nve_ecn_init:
err_nve_qos_init:
+ rhashtable_destroy(&nve->ipv6_ht);
+err_ipv6_rhashtable_init:
rhashtable_destroy(&nve->mc_list_ht);
-err_rhashtable_init:
+err_mc_rhashtable_init:
mlxsw_sp->nve = NULL;
kfree(nve);
return err;
@@ -1010,6 +1165,8 @@ int mlxsw_sp_nve_init(struct mlxsw_sp *mlxsw_sp)
void mlxsw_sp_nve_fini(struct mlxsw_sp *mlxsw_sp)
{
WARN_ON(mlxsw_sp->nve->num_nve_tunnels);
+ WARN_ON(!list_empty(&mlxsw_sp->nve->ipv6_addr_list));
+ rhashtable_destroy(&mlxsw_sp->nve->ipv6_ht);
rhashtable_destroy(&mlxsw_sp->nve->mc_list_ht);
kfree(mlxsw_sp->nve);
mlxsw_sp->nve = NULL;
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_nve.h b/drivers/net/ethernet/mellanox/mlxsw/spectrum_nve.h
index 98d1fdc..0d21de1 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_nve.h
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_nve.h
@@ -23,6 +23,8 @@ struct mlxsw_sp_nve_config {
struct mlxsw_sp_nve {
struct mlxsw_sp_nve_config config;
struct rhashtable mc_list_ht;
+ struct rhashtable ipv6_ht;
+ struct list_head ipv6_addr_list; /* Saves hash table nodes. */
struct mlxsw_sp *mlxsw_sp;
const struct mlxsw_sp_nve_ops **nve_ops_arr;
unsigned int num_nve_tunnels; /* Protected by RTNL */
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_nve_vxlan.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_nve_vxlan.c
index d018d2d..d309b77 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_nve_vxlan.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_nve_vxlan.c
@@ -10,8 +10,48 @@
#include "spectrum.h"
#include "spectrum_nve.h"
-#define MLXSW_SP_NVE_VXLAN_SUPPORTED_FLAGS (VXLAN_F_UDP_ZERO_CSUM_TX | \
+#define MLXSW_SP_NVE_VXLAN_IPV4_SUPPORTED_FLAGS (VXLAN_F_UDP_ZERO_CSUM_TX | \
VXLAN_F_LEARN)
+#define MLXSW_SP_NVE_VXLAN_IPV6_SUPPORTED_FLAGS (VXLAN_F_IPV6 | \
+ VXLAN_F_UDP_ZERO_CSUM6_TX | \
+ VXLAN_F_UDP_ZERO_CSUM6_RX)
+
+static bool mlxsw_sp_nve_vxlan_ipv4_flags_check(const struct vxlan_config *cfg,
+ struct netlink_ext_ack *extack)
+{
+ if (!(cfg->flags & VXLAN_F_UDP_ZERO_CSUM_TX)) {
+ NL_SET_ERR_MSG_MOD(extack, "VxLAN: Zero UDP checksum must be allowed for TX");
+ return false;
+ }
+
+ if (cfg->flags & ~MLXSW_SP_NVE_VXLAN_IPV4_SUPPORTED_FLAGS) {
+ NL_SET_ERR_MSG_MOD(extack, "VxLAN: Unsupported flag");
+ return false;
+ }
+
+ return true;
+}
+
+static bool mlxsw_sp_nve_vxlan_ipv6_flags_check(const struct vxlan_config *cfg,
+ struct netlink_ext_ack *extack)
+{
+ if (!(cfg->flags & VXLAN_F_UDP_ZERO_CSUM6_TX)) {
+ NL_SET_ERR_MSG_MOD(extack, "VxLAN: Zero UDP checksum must be allowed for TX");
+ return false;
+ }
+
+ if (!(cfg->flags & VXLAN_F_UDP_ZERO_CSUM6_RX)) {
+ NL_SET_ERR_MSG_MOD(extack, "VxLAN: Zero UDP checksum must be allowed for RX");
+ return false;
+ }
+
+ if (cfg->flags & ~MLXSW_SP_NVE_VXLAN_IPV6_SUPPORTED_FLAGS) {
+ NL_SET_ERR_MSG_MOD(extack, "VxLAN: Unsupported flag");
+ return false;
+ }
+
+ return true;
+}
static bool mlxsw_sp_nve_vxlan_can_offload(const struct mlxsw_sp_nve *nve,
const struct mlxsw_sp_nve_params *params,
@@ -20,11 +60,6 @@ static bool mlxsw_sp_nve_vxlan_can_offload(const struct mlxsw_sp_nve *nve,
struct vxlan_dev *vxlan = netdev_priv(params->dev);
struct vxlan_config *cfg = &vxlan->cfg;
- if (cfg->saddr.sa.sa_family != AF_INET) {
- NL_SET_ERR_MSG_MOD(extack, "VxLAN: Only IPv4 underlay is supported");
- return false;
- }
-
if (vxlan_addr_multicast(&cfg->remote_ip)) {
NL_SET_ERR_MSG_MOD(extack, "VxLAN: Multicast destination IP is not supported");
return false;
@@ -55,14 +90,15 @@ static bool mlxsw_sp_nve_vxlan_can_offload(const struct mlxsw_sp_nve *nve,
return false;
}
- if (!(cfg->flags & VXLAN_F_UDP_ZERO_CSUM_TX)) {
- NL_SET_ERR_MSG_MOD(extack, "VxLAN: UDP checksum is not supported");
- return false;
- }
-
- if (cfg->flags & ~MLXSW_SP_NVE_VXLAN_SUPPORTED_FLAGS) {
- NL_SET_ERR_MSG_MOD(extack, "VxLAN: Unsupported flag");
- return false;
+ switch (cfg->saddr.sa.sa_family) {
+ case AF_INET:
+ if (!mlxsw_sp_nve_vxlan_ipv4_flags_check(cfg, extack))
+ return false;
+ break;
+ case AF_INET6:
+ if (!mlxsw_sp_nve_vxlan_ipv6_flags_check(cfg, extack))
+ return false;
+ break;
}
if (cfg->ttl == 0) {
@@ -90,6 +126,22 @@ static bool mlxsw_sp1_nve_vxlan_can_offload(const struct mlxsw_sp_nve *nve,
return mlxsw_sp_nve_vxlan_can_offload(nve, params, extack);
}
+static void
+mlxsw_sp_nve_vxlan_ul_proto_sip_config(const struct vxlan_config *cfg,
+ struct mlxsw_sp_nve_config *config)
+{
+ switch (cfg->saddr.sa.sa_family) {
+ case AF_INET:
+ config->ul_proto = MLXSW_SP_L3_PROTO_IPV4;
+ config->ul_sip.addr4 = cfg->saddr.sin.sin_addr.s_addr;
+ break;
+ case AF_INET6:
+ config->ul_proto = MLXSW_SP_L3_PROTO_IPV6;
+ config->ul_sip.addr6 = cfg->saddr.sin6.sin6_addr;
+ break;
+ }
+}
+
static void mlxsw_sp_nve_vxlan_config(const struct mlxsw_sp_nve *nve,
const struct mlxsw_sp_nve_params *params,
struct mlxsw_sp_nve_config *config)
@@ -102,8 +154,7 @@ static void mlxsw_sp_nve_vxlan_config(const struct mlxsw_sp_nve *nve,
config->flowlabel = cfg->label;
config->learning_en = cfg->flags & VXLAN_F_LEARN ? 1 : 0;
config->ul_tb_id = RT_TABLE_MAIN;
- config->ul_proto = MLXSW_SP_L3_PROTO_IPV4;
- config->ul_sip.addr4 = cfg->saddr.sin.sin_addr.s_addr;
+ mlxsw_sp_nve_vxlan_ul_proto_sip_config(cfg, config);
config->udp_dport = cfg->dst_port;
}
@@ -111,6 +162,7 @@ static void
mlxsw_sp_nve_vxlan_config_prepare(char *tngcr_pl,
const struct mlxsw_sp_nve_config *config)
{
+ struct in6_addr addr6;
u8 udp_sport;
mlxsw_reg_tngcr_pack(tngcr_pl, MLXSW_REG_TNGCR_TYPE_VXLAN, true,
@@ -122,7 +174,18 @@ mlxsw_sp_nve_vxlan_config_prepare(char *tngcr_pl,
get_random_bytes(&udp_sport, sizeof(udp_sport));
udp_sport = (udp_sport % (0xee - 0x80 + 1)) + 0x80;
mlxsw_reg_tngcr_nve_udp_sport_prefix_set(tngcr_pl, udp_sport);
- mlxsw_reg_tngcr_usipv4_set(tngcr_pl, be32_to_cpu(config->ul_sip.addr4));
+
+ switch (config->ul_proto) {
+ case MLXSW_SP_L3_PROTO_IPV4:
+ mlxsw_reg_tngcr_usipv4_set(tngcr_pl,
+ be32_to_cpu(config->ul_sip.addr4));
+ break;
+ case MLXSW_SP_L3_PROTO_IPV6:
+ addr6 = config->ul_sip.addr6;
+ mlxsw_reg_tngcr_usipv6_memcpy_to(tngcr_pl,
+ (const char *)&addr6);
+ break;
+ }
}
static int
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_ptp.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_ptp.c
index 1a18038..0ff163f 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_ptp.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_ptp.c
@@ -36,7 +36,7 @@ struct mlxsw_sp_ptp_state {
};
struct mlxsw_sp1_ptp_key {
- u8 local_port;
+ u16 local_port;
u8 message_type;
u16 sequence_id;
u8 domain_number;
@@ -406,7 +406,7 @@ mlxsw_sp1_ptp_unmatched_remove(struct mlxsw_sp *mlxsw_sp,
* This case is similar to 2) above.
*/
static void mlxsw_sp1_ptp_packet_finish(struct mlxsw_sp *mlxsw_sp,
- struct sk_buff *skb, u8 local_port,
+ struct sk_buff *skb, u16 local_port,
bool ingress,
struct skb_shared_hwtstamps *hwtstamps)
{
@@ -524,7 +524,7 @@ static void mlxsw_sp1_ptp_got_piece(struct mlxsw_sp *mlxsw_sp,
}
static void mlxsw_sp1_ptp_got_packet(struct mlxsw_sp *mlxsw_sp,
- struct sk_buff *skb, u8 local_port,
+ struct sk_buff *skb, u16 local_port,
bool ingress)
{
struct mlxsw_sp_port *mlxsw_sp_port;
@@ -564,7 +564,7 @@ static void mlxsw_sp1_ptp_got_packet(struct mlxsw_sp *mlxsw_sp,
}
void mlxsw_sp1_ptp_got_timestamp(struct mlxsw_sp *mlxsw_sp, bool ingress,
- u8 local_port, u8 message_type,
+ u16 local_port, u8 message_type,
u8 domain_number, u16 sequence_id,
u64 timestamp)
{
@@ -599,14 +599,14 @@ void mlxsw_sp1_ptp_got_timestamp(struct mlxsw_sp *mlxsw_sp, bool ingress,
}
void mlxsw_sp1_ptp_receive(struct mlxsw_sp *mlxsw_sp, struct sk_buff *skb,
- u8 local_port)
+ u16 local_port)
{
skb_reset_mac_header(skb);
mlxsw_sp1_ptp_got_packet(mlxsw_sp, skb, local_port, true);
}
void mlxsw_sp1_ptp_transmitted(struct mlxsw_sp *mlxsw_sp,
- struct sk_buff *skb, u8 local_port)
+ struct sk_buff *skb, u16 local_port)
{
mlxsw_sp1_ptp_got_packet(mlxsw_sp, skb, local_port, false);
}
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_ptp.h b/drivers/net/ethernet/mellanox/mlxsw/spectrum_ptp.h
index 1d43a37..c06cd13 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_ptp.h
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_ptp.h
@@ -31,13 +31,13 @@ struct mlxsw_sp_ptp_state *mlxsw_sp1_ptp_init(struct mlxsw_sp *mlxsw_sp);
void mlxsw_sp1_ptp_fini(struct mlxsw_sp_ptp_state *ptp_state);
void mlxsw_sp1_ptp_receive(struct mlxsw_sp *mlxsw_sp, struct sk_buff *skb,
- u8 local_port);
+ u16 local_port);
void mlxsw_sp1_ptp_transmitted(struct mlxsw_sp *mlxsw_sp,
- struct sk_buff *skb, u8 local_port);
+ struct sk_buff *skb, u16 local_port);
void mlxsw_sp1_ptp_got_timestamp(struct mlxsw_sp *mlxsw_sp, bool ingress,
- u8 local_port, u8 message_type,
+ u16 local_port, u8 message_type,
u8 domain_number, u16 sequence_id,
u64 timestamp);
@@ -80,20 +80,20 @@ static inline void mlxsw_sp1_ptp_fini(struct mlxsw_sp_ptp_state *ptp_state)
}
static inline void mlxsw_sp1_ptp_receive(struct mlxsw_sp *mlxsw_sp,
- struct sk_buff *skb, u8 local_port)
+ struct sk_buff *skb, u16 local_port)
{
mlxsw_sp_rx_listener_no_mark_func(skb, local_port, mlxsw_sp);
}
static inline void mlxsw_sp1_ptp_transmitted(struct mlxsw_sp *mlxsw_sp,
- struct sk_buff *skb, u8 local_port)
+ struct sk_buff *skb, u16 local_port)
{
dev_kfree_skb_any(skb);
}
static inline void
mlxsw_sp1_ptp_got_timestamp(struct mlxsw_sp *mlxsw_sp, bool ingress,
- u8 local_port, u8 message_type,
+ u16 local_port, u8 message_type,
u8 domain_number,
u16 sequence_id, u64 timestamp)
{
@@ -159,13 +159,13 @@ static inline void mlxsw_sp2_ptp_fini(struct mlxsw_sp_ptp_state *ptp_state)
}
static inline void mlxsw_sp2_ptp_receive(struct mlxsw_sp *mlxsw_sp,
- struct sk_buff *skb, u8 local_port)
+ struct sk_buff *skb, u16 local_port)
{
mlxsw_sp_rx_listener_no_mark_func(skb, local_port, mlxsw_sp);
}
static inline void mlxsw_sp2_ptp_transmitted(struct mlxsw_sp *mlxsw_sp,
- struct sk_buff *skb, u8 local_port)
+ struct sk_buff *skb, u16 local_port)
{
dev_kfree_skb_any(skb);
}
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c
index c34833f..d40762c 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c
@@ -1307,6 +1307,10 @@ mlxsw_sp_router_ip2me_fib_entry_find(struct mlxsw_sp *mlxsw_sp, u32 tb_id,
addr_prefix_len = 32;
break;
case MLXSW_SP_L3_PROTO_IPV6:
+ addrp = &addr->addr6;
+ addr_len = 16;
+ addr_prefix_len = 128;
+ break;
default:
WARN_ON(1);
return NULL;
@@ -7002,6 +7006,8 @@ mlxsw_sp_fib6_entry_type_set_local(struct mlxsw_sp *mlxsw_sp,
{
struct mlxsw_sp_nexthop_group_info *nhgi = fib_entry->nh_group->nhgi;
union mlxsw_sp_l3addr dip = { .addr6 = rt->fib6_dst.addr };
+ u32 tb_id = mlxsw_sp_fix_tb_id(rt->fib6_table->tb6_id);
+ struct mlxsw_sp_router *router = mlxsw_sp->router;
int ifindex = nhgi->nexthops[0].ifindex;
struct mlxsw_sp_ipip_entry *ipip_entry;
@@ -7015,6 +7021,14 @@ mlxsw_sp_fib6_entry_type_set_local(struct mlxsw_sp *mlxsw_sp,
return mlxsw_sp_fib_entry_decap_init(mlxsw_sp, fib_entry,
ipip_entry);
}
+ if (mlxsw_sp_router_nve_is_decap(mlxsw_sp, tb_id,
+ MLXSW_SP_L3_PROTO_IPV6, &dip)) {
+ u32 tunnel_index;
+
+ tunnel_index = router->nve_decap_config.tunnel_index;
+ fib_entry->decap.tunnel_index = tunnel_index;
+ fib_entry->type = MLXSW_SP_FIB_ENTRY_TYPE_NVE_DECAP;
+ }
return 0;
}
@@ -8369,9 +8383,6 @@ mlxsw_sp_rif_mac_profile_find(const struct mlxsw_sp *mlxsw_sp, const char *mac)
int id;
idr_for_each_entry(&router->rif_mac_profiles_idr, profile, id) {
- if (!profile)
- continue;
-
if (ether_addr_equal_masked(profile->mac_prefix, mac,
mlxsw_sp->mac_mask))
return profile;
@@ -9344,7 +9355,7 @@ static int mlxsw_sp_rif_vlan_fid_op(struct mlxsw_sp_rif *rif,
return mlxsw_reg_write(mlxsw_sp->core, MLXSW_REG(ritr), ritr_pl);
}
-u8 mlxsw_sp_router_port(const struct mlxsw_sp *mlxsw_sp)
+u16 mlxsw_sp_router_port(const struct mlxsw_sp *mlxsw_sp)
{
return mlxsw_core_max_ports(mlxsw_sp->core) + 1;
}
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c
index f5f819a..f9671cc 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c
@@ -37,7 +37,7 @@ struct mlxsw_sp_span {
struct mlxsw_sp_span_analyzed_port {
struct list_head list; /* Member of analyzed_ports_list */
refcount_t ref_count;
- u8 local_port;
+ u16 local_port;
bool ingress;
};
@@ -46,7 +46,7 @@ struct mlxsw_sp_span_trigger_entry {
struct mlxsw_sp_span *span;
const struct mlxsw_sp_span_trigger_ops *ops;
refcount_t ref_count;
- u8 local_port;
+ u16 local_port;
enum mlxsw_sp_span_trigger trigger;
struct mlxsw_sp_span_trigger_parms parms;
};
@@ -179,7 +179,7 @@ mlxsw_sp_span_entry_phys_configure(struct mlxsw_sp_span_entry *span_entry,
{
struct mlxsw_sp_port *dest_port = sparms.dest_port;
struct mlxsw_sp *mlxsw_sp = dest_port->mlxsw_sp;
- u8 local_port = dest_port->local_port;
+ u16 local_port = dest_port->local_port;
char mpat_pl[MLXSW_REG_MPAT_LEN];
int pa_id = span_entry->id;
@@ -199,7 +199,7 @@ mlxsw_sp_span_entry_deconfigure_common(struct mlxsw_sp_span_entry *span_entry,
{
struct mlxsw_sp_port *dest_port = span_entry->parms.dest_port;
struct mlxsw_sp *mlxsw_sp = dest_port->mlxsw_sp;
- u8 local_port = dest_port->local_port;
+ u16 local_port = dest_port->local_port;
char mpat_pl[MLXSW_REG_MPAT_LEN];
int pa_id = span_entry->id;
@@ -480,7 +480,7 @@ mlxsw_sp_span_entry_gretap4_configure(struct mlxsw_sp_span_entry *span_entry,
{
struct mlxsw_sp_port *dest_port = sparms.dest_port;
struct mlxsw_sp *mlxsw_sp = dest_port->mlxsw_sp;
- u8 local_port = dest_port->local_port;
+ u16 local_port = dest_port->local_port;
char mpat_pl[MLXSW_REG_MPAT_LEN];
int pa_id = span_entry->id;
@@ -584,7 +584,7 @@ mlxsw_sp_span_entry_gretap6_configure(struct mlxsw_sp_span_entry *span_entry,
{
struct mlxsw_sp_port *dest_port = sparms.dest_port;
struct mlxsw_sp *mlxsw_sp = dest_port->mlxsw_sp;
- u8 local_port = dest_port->local_port;
+ u16 local_port = dest_port->local_port;
char mpat_pl[MLXSW_REG_MPAT_LEN];
int pa_id = span_entry->id;
@@ -650,7 +650,7 @@ mlxsw_sp_span_entry_vlan_configure(struct mlxsw_sp_span_entry *span_entry,
{
struct mlxsw_sp_port *dest_port = sparms.dest_port;
struct mlxsw_sp *mlxsw_sp = dest_port->mlxsw_sp;
- u8 local_port = dest_port->local_port;
+ u16 local_port = dest_port->local_port;
char mpat_pl[MLXSW_REG_MPAT_LEN];
int pa_id = span_entry->id;
@@ -997,7 +997,7 @@ static void mlxsw_sp_span_port_buffer_disable(struct mlxsw_sp_port *mlxsw_sp_por
}
static struct mlxsw_sp_span_analyzed_port *
-mlxsw_sp_span_analyzed_port_find(struct mlxsw_sp_span *span, u8 local_port,
+mlxsw_sp_span_analyzed_port_find(struct mlxsw_sp_span *span, u16 local_port,
bool ingress)
{
struct mlxsw_sp_span_analyzed_port *analyzed_port;
@@ -1165,7 +1165,7 @@ int mlxsw_sp_span_analyzed_port_get(struct mlxsw_sp_port *mlxsw_sp_port,
{
struct mlxsw_sp *mlxsw_sp = mlxsw_sp_port->mlxsw_sp;
struct mlxsw_sp_span_analyzed_port *analyzed_port;
- u8 local_port = mlxsw_sp_port->local_port;
+ u16 local_port = mlxsw_sp_port->local_port;
int err = 0;
mutex_lock(&mlxsw_sp->span->analyzed_ports_lock);
@@ -1193,7 +1193,7 @@ void mlxsw_sp_span_analyzed_port_put(struct mlxsw_sp_port *mlxsw_sp_port,
{
struct mlxsw_sp *mlxsw_sp = mlxsw_sp_port->mlxsw_sp;
struct mlxsw_sp_span_analyzed_port *analyzed_port;
- u8 local_port = mlxsw_sp_port->local_port;
+ u16 local_port = mlxsw_sp_port->local_port;
mutex_lock(&mlxsw_sp->span->analyzed_ports_lock);
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_switchdev.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_switchdev.c
index 81c7e8a..65c1724 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_switchdev.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_switchdev.c
@@ -865,17 +865,17 @@ static int mlxsw_sp_port_mc_disabled_set(struct mlxsw_sp_port *mlxsw_sp_port,
static int mlxsw_sp_smid_router_port_set(struct mlxsw_sp *mlxsw_sp,
u16 mid_idx, bool add)
{
- char *smid_pl;
+ char *smid2_pl;
int err;
- smid_pl = kmalloc(MLXSW_REG_SMID_LEN, GFP_KERNEL);
- if (!smid_pl)
+ smid2_pl = kmalloc(MLXSW_REG_SMID2_LEN, GFP_KERNEL);
+ if (!smid2_pl)
return -ENOMEM;
- mlxsw_reg_smid_pack(smid_pl, mid_idx,
- mlxsw_sp_router_port(mlxsw_sp), add);
- err = mlxsw_reg_write(mlxsw_sp->core, MLXSW_REG(smid), smid_pl);
- kfree(smid_pl);
+ mlxsw_reg_smid2_pack(smid2_pl, mid_idx,
+ mlxsw_sp_router_port(mlxsw_sp), add);
+ err = mlxsw_reg_write(mlxsw_sp->core, MLXSW_REG(smid2), smid2_pl);
+ kfree(smid2_pl);
return err;
}
@@ -980,7 +980,7 @@ mlxsw_sp_port_vlan_fid_join(struct mlxsw_sp_port_vlan *mlxsw_sp_port_vlan,
{
struct mlxsw_sp_port *mlxsw_sp_port = mlxsw_sp_port_vlan->mlxsw_sp_port;
struct mlxsw_sp_bridge_device *bridge_device;
- u8 local_port = mlxsw_sp_port->local_port;
+ u16 local_port = mlxsw_sp_port->local_port;
u16 vid = mlxsw_sp_port_vlan->vid;
struct mlxsw_sp_fid *fid;
int err;
@@ -1029,7 +1029,7 @@ mlxsw_sp_port_vlan_fid_leave(struct mlxsw_sp_port_vlan *mlxsw_sp_port_vlan)
{
struct mlxsw_sp_port *mlxsw_sp_port = mlxsw_sp_port_vlan->mlxsw_sp_port;
struct mlxsw_sp_fid *fid = mlxsw_sp_port_vlan->fid;
- u8 local_port = mlxsw_sp_port->local_port;
+ u16 local_port = mlxsw_sp_port->local_port;
u16 vid = mlxsw_sp_port_vlan->vid;
mlxsw_sp_port_vlan->fid = NULL;
@@ -1290,38 +1290,24 @@ static enum mlxsw_reg_sfd_op mlxsw_sp_sfd_op(bool adding)
MLXSW_REG_SFD_OP_WRITE_REMOVE;
}
-static int mlxsw_sp_port_fdb_tunnel_uc_op(struct mlxsw_sp *mlxsw_sp,
- const char *mac, u16 fid,
- enum mlxsw_sp_l3proto proto,
- const union mlxsw_sp_l3addr *addr,
- bool adding, bool dynamic)
+static int
+mlxsw_sp_port_fdb_tun_uc_op4(struct mlxsw_sp *mlxsw_sp, bool dynamic,
+ const char *mac, u16 fid, __be32 addr, bool adding)
{
- enum mlxsw_reg_sfd_uc_tunnel_protocol sfd_proto;
char *sfd_pl;
u8 num_rec;
u32 uip;
int err;
- switch (proto) {
- case MLXSW_SP_L3_PROTO_IPV4:
- uip = be32_to_cpu(addr->addr4);
- sfd_proto = MLXSW_REG_SFD_UC_TUNNEL_PROTOCOL_IPV4;
- break;
- case MLXSW_SP_L3_PROTO_IPV6:
- default:
- WARN_ON(1);
- return -EOPNOTSUPP;
- }
-
sfd_pl = kmalloc(MLXSW_REG_SFD_LEN, GFP_KERNEL);
if (!sfd_pl)
return -ENOMEM;
+ uip = be32_to_cpu(addr);
mlxsw_reg_sfd_pack(sfd_pl, mlxsw_sp_sfd_op(adding), 0);
- mlxsw_reg_sfd_uc_tunnel_pack(sfd_pl, 0,
- mlxsw_sp_sfd_rec_policy(dynamic), mac, fid,
- MLXSW_REG_SFD_REC_ACTION_NOP, uip,
- sfd_proto);
+ mlxsw_reg_sfd_uc_tunnel_pack4(sfd_pl, 0,
+ mlxsw_sp_sfd_rec_policy(dynamic), mac,
+ fid, MLXSW_REG_SFD_REC_ACTION_NOP, uip);
num_rec = mlxsw_reg_sfd_num_rec_get(sfd_pl);
err = mlxsw_reg_write(mlxsw_sp->core, MLXSW_REG(sfd), sfd_pl);
if (err)
@@ -1335,7 +1321,108 @@ static int mlxsw_sp_port_fdb_tunnel_uc_op(struct mlxsw_sp *mlxsw_sp,
return err;
}
-static int __mlxsw_sp_port_fdb_uc_op(struct mlxsw_sp *mlxsw_sp, u8 local_port,
+static int mlxsw_sp_port_fdb_tun_uc_op6_sfd_write(struct mlxsw_sp *mlxsw_sp,
+ const char *mac, u16 fid,
+ u32 kvdl_index, bool adding)
+{
+ char *sfd_pl;
+ u8 num_rec;
+ int err;
+
+ sfd_pl = kmalloc(MLXSW_REG_SFD_LEN, GFP_KERNEL);
+ if (!sfd_pl)
+ return -ENOMEM;
+
+ mlxsw_reg_sfd_pack(sfd_pl, mlxsw_sp_sfd_op(adding), 0);
+ mlxsw_reg_sfd_uc_tunnel_pack6(sfd_pl, 0, mac, fid,
+ MLXSW_REG_SFD_REC_ACTION_NOP, kvdl_index);
+ num_rec = mlxsw_reg_sfd_num_rec_get(sfd_pl);
+ err = mlxsw_reg_write(mlxsw_sp->core, MLXSW_REG(sfd), sfd_pl);
+ if (err)
+ goto out;
+
+ if (num_rec != mlxsw_reg_sfd_num_rec_get(sfd_pl))
+ err = -EBUSY;
+
+out:
+ kfree(sfd_pl);
+ return err;
+}
+
+static int mlxsw_sp_port_fdb_tun_uc_op6_add(struct mlxsw_sp *mlxsw_sp,
+ const char *mac, u16 fid,
+ const struct in6_addr *addr)
+{
+ u32 kvdl_index;
+ int err;
+
+ err = mlxsw_sp_nve_ipv6_addr_kvdl_set(mlxsw_sp, addr, &kvdl_index);
+ if (err)
+ return err;
+
+ err = mlxsw_sp_port_fdb_tun_uc_op6_sfd_write(mlxsw_sp, mac, fid,
+ kvdl_index, true);
+ if (err)
+ goto err_sfd_write;
+
+ err = mlxsw_sp_nve_ipv6_addr_map_replace(mlxsw_sp, mac, fid, addr);
+ if (err)
+ /* Replace can fail only for creating new mapping, so removing
+ * the FDB entry in the error path is OK.
+ */
+ goto err_addr_replace;
+
+ return 0;
+
+err_addr_replace:
+ mlxsw_sp_port_fdb_tun_uc_op6_sfd_write(mlxsw_sp, mac, fid, kvdl_index,
+ false);
+err_sfd_write:
+ mlxsw_sp_nve_ipv6_addr_kvdl_unset(mlxsw_sp, addr);
+ return err;
+}
+
+static void mlxsw_sp_port_fdb_tun_uc_op6_del(struct mlxsw_sp *mlxsw_sp,
+ const char *mac, u16 fid,
+ const struct in6_addr *addr)
+{
+ mlxsw_sp_nve_ipv6_addr_map_del(mlxsw_sp, mac, fid);
+ mlxsw_sp_port_fdb_tun_uc_op6_sfd_write(mlxsw_sp, mac, fid, 0, false);
+ mlxsw_sp_nve_ipv6_addr_kvdl_unset(mlxsw_sp, addr);
+}
+
+static int
+mlxsw_sp_port_fdb_tun_uc_op6(struct mlxsw_sp *mlxsw_sp, const char *mac,
+ u16 fid, const struct in6_addr *addr, bool adding)
+{
+ if (adding)
+ return mlxsw_sp_port_fdb_tun_uc_op6_add(mlxsw_sp, mac, fid,
+ addr);
+
+ mlxsw_sp_port_fdb_tun_uc_op6_del(mlxsw_sp, mac, fid, addr);
+ return 0;
+}
+
+static int mlxsw_sp_port_fdb_tunnel_uc_op(struct mlxsw_sp *mlxsw_sp,
+ const char *mac, u16 fid,
+ enum mlxsw_sp_l3proto proto,
+ const union mlxsw_sp_l3addr *addr,
+ bool adding, bool dynamic)
+{
+ switch (proto) {
+ case MLXSW_SP_L3_PROTO_IPV4:
+ return mlxsw_sp_port_fdb_tun_uc_op4(mlxsw_sp, dynamic, mac, fid,
+ addr->addr4, adding);
+ case MLXSW_SP_L3_PROTO_IPV6:
+ return mlxsw_sp_port_fdb_tun_uc_op6(mlxsw_sp, mac, fid,
+ &addr->addr6, adding);
+ default:
+ WARN_ON(1);
+ return -EOPNOTSUPP;
+ }
+}
+
+static int __mlxsw_sp_port_fdb_uc_op(struct mlxsw_sp *mlxsw_sp, u16 local_port,
const char *mac, u16 fid, bool adding,
enum mlxsw_reg_sfd_rec_action action,
enum mlxsw_reg_sfd_rec_policy policy)
@@ -1363,7 +1450,7 @@ static int __mlxsw_sp_port_fdb_uc_op(struct mlxsw_sp *mlxsw_sp, u8 local_port,
return err;
}
-static int mlxsw_sp_port_fdb_uc_op(struct mlxsw_sp *mlxsw_sp, u8 local_port,
+static int mlxsw_sp_port_fdb_uc_op(struct mlxsw_sp *mlxsw_sp, u16 local_port,
const char *mac, u16 fid, bool adding,
bool dynamic)
{
@@ -1477,30 +1564,30 @@ static int mlxsw_sp_port_smid_full_entry(struct mlxsw_sp *mlxsw_sp, u16 mid_idx,
long *ports_bitmap,
bool set_router_port)
{
- char *smid_pl;
+ char *smid2_pl;
int err, i;
- smid_pl = kmalloc(MLXSW_REG_SMID_LEN, GFP_KERNEL);
- if (!smid_pl)
+ smid2_pl = kmalloc(MLXSW_REG_SMID2_LEN, GFP_KERNEL);
+ if (!smid2_pl)
return -ENOMEM;
- mlxsw_reg_smid_pack(smid_pl, mid_idx, 0, false);
+ mlxsw_reg_smid2_pack(smid2_pl, mid_idx, 0, false);
for (i = 1; i < mlxsw_core_max_ports(mlxsw_sp->core); i++) {
if (mlxsw_sp->ports[i])
- mlxsw_reg_smid_port_mask_set(smid_pl, i, 1);
+ mlxsw_reg_smid2_port_mask_set(smid2_pl, i, 1);
}
- mlxsw_reg_smid_port_mask_set(smid_pl,
- mlxsw_sp_router_port(mlxsw_sp), 1);
+ mlxsw_reg_smid2_port_mask_set(smid2_pl,
+ mlxsw_sp_router_port(mlxsw_sp), 1);
for_each_set_bit(i, ports_bitmap, mlxsw_core_max_ports(mlxsw_sp->core))
- mlxsw_reg_smid_port_set(smid_pl, i, 1);
+ mlxsw_reg_smid2_port_set(smid2_pl, i, 1);
- mlxsw_reg_smid_port_set(smid_pl, mlxsw_sp_router_port(mlxsw_sp),
- set_router_port);
+ mlxsw_reg_smid2_port_set(smid2_pl, mlxsw_sp_router_port(mlxsw_sp),
+ set_router_port);
- err = mlxsw_reg_write(mlxsw_sp->core, MLXSW_REG(smid), smid_pl);
- kfree(smid_pl);
+ err = mlxsw_reg_write(mlxsw_sp->core, MLXSW_REG(smid2), smid2_pl);
+ kfree(smid2_pl);
return err;
}
@@ -1508,16 +1595,16 @@ static int mlxsw_sp_port_smid_set(struct mlxsw_sp_port *mlxsw_sp_port,
u16 mid_idx, bool add)
{
struct mlxsw_sp *mlxsw_sp = mlxsw_sp_port->mlxsw_sp;
- char *smid_pl;
+ char *smid2_pl;
int err;
- smid_pl = kmalloc(MLXSW_REG_SMID_LEN, GFP_KERNEL);
- if (!smid_pl)
+ smid2_pl = kmalloc(MLXSW_REG_SMID2_LEN, GFP_KERNEL);
+ if (!smid2_pl)
return -ENOMEM;
- mlxsw_reg_smid_pack(smid_pl, mid_idx, mlxsw_sp_port->local_port, add);
- err = mlxsw_reg_write(mlxsw_sp->core, MLXSW_REG(smid), smid_pl);
- kfree(smid_pl);
+ mlxsw_reg_smid2_pack(smid2_pl, mid_idx, mlxsw_sp_port->local_port, add);
+ err = mlxsw_reg_write(mlxsw_sp->core, MLXSW_REG(smid2), smid2_pl);
+ kfree(smid2_pl);
return err;
}
@@ -2536,7 +2623,7 @@ static void mlxsw_sp_fdb_notify_mac_process(struct mlxsw_sp *mlxsw_sp,
struct mlxsw_sp_port *mlxsw_sp_port;
enum switchdev_notifier_type type;
char mac[ETH_ALEN];
- u8 local_port;
+ u16 local_port;
u16 vid, fid;
bool do_notification = true;
int err;
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_trap.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_trap.c
index 26d01ad..47b061b 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_trap.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_trap.c
@@ -60,7 +60,7 @@ enum {
};
static int mlxsw_sp_rx_listener(struct mlxsw_sp *mlxsw_sp, struct sk_buff *skb,
- u8 local_port,
+ u16 local_port,
struct mlxsw_sp_port *mlxsw_sp_port)
{
struct mlxsw_sp_port_pcpu_stats *pcpu_stats;
@@ -85,7 +85,7 @@ static int mlxsw_sp_rx_listener(struct mlxsw_sp *mlxsw_sp, struct sk_buff *skb,
return 0;
}
-static void mlxsw_sp_rx_drop_listener(struct sk_buff *skb, u8 local_port,
+static void mlxsw_sp_rx_drop_listener(struct sk_buff *skb, u16 local_port,
void *trap_ctx)
{
struct devlink_port *in_devlink_port;
@@ -109,7 +109,7 @@ static void mlxsw_sp_rx_drop_listener(struct sk_buff *skb, u8 local_port,
consume_skb(skb);
}
-static void mlxsw_sp_rx_acl_drop_listener(struct sk_buff *skb, u8 local_port,
+static void mlxsw_sp_rx_acl_drop_listener(struct sk_buff *skb, u16 local_port,
void *trap_ctx)
{
u32 cookie_index = mlxsw_skb_cb(skb)->rx_md_info.cookie_index;
@@ -138,7 +138,7 @@ static void mlxsw_sp_rx_acl_drop_listener(struct sk_buff *skb, u8 local_port,
consume_skb(skb);
}
-static int __mlxsw_sp_rx_no_mark_listener(struct sk_buff *skb, u8 local_port,
+static int __mlxsw_sp_rx_no_mark_listener(struct sk_buff *skb, u16 local_port,
void *trap_ctx)
{
struct devlink_port *in_devlink_port;
@@ -164,7 +164,7 @@ static int __mlxsw_sp_rx_no_mark_listener(struct sk_buff *skb, u8 local_port,
return 0;
}
-static void mlxsw_sp_rx_no_mark_listener(struct sk_buff *skb, u8 local_port,
+static void mlxsw_sp_rx_no_mark_listener(struct sk_buff *skb, u16 local_port,
void *trap_ctx)
{
int err;
@@ -176,14 +176,14 @@ static void mlxsw_sp_rx_no_mark_listener(struct sk_buff *skb, u8 local_port,
netif_receive_skb(skb);
}
-static void mlxsw_sp_rx_mark_listener(struct sk_buff *skb, u8 local_port,
+static void mlxsw_sp_rx_mark_listener(struct sk_buff *skb, u16 local_port,
void *trap_ctx)
{
skb->offload_fwd_mark = 1;
mlxsw_sp_rx_no_mark_listener(skb, local_port, trap_ctx);
}
-static void mlxsw_sp_rx_l3_mark_listener(struct sk_buff *skb, u8 local_port,
+static void mlxsw_sp_rx_l3_mark_listener(struct sk_buff *skb, u16 local_port,
void *trap_ctx)
{
skb->offload_l3_fwd_mark = 1;
@@ -191,7 +191,7 @@ static void mlxsw_sp_rx_l3_mark_listener(struct sk_buff *skb, u8 local_port,
mlxsw_sp_rx_no_mark_listener(skb, local_port, trap_ctx);
}
-static void mlxsw_sp_rx_ptp_listener(struct sk_buff *skb, u8 local_port,
+static void mlxsw_sp_rx_ptp_listener(struct sk_buff *skb, u16 local_port,
void *trap_ctx)
{
struct mlxsw_sp *mlxsw_sp = devlink_trap_ctx_priv(trap_ctx);
@@ -212,7 +212,7 @@ static struct mlxsw_sp_port *
mlxsw_sp_sample_tx_port_get(struct mlxsw_sp *mlxsw_sp,
const struct mlxsw_rx_md_info *rx_md_info)
{
- u8 local_port;
+ u16 local_port;
if (!rx_md_info->tx_port_valid)
return NULL;
@@ -257,7 +257,7 @@ static void mlxsw_sp_psample_md_init(struct mlxsw_sp *mlxsw_sp,
md->latency <<= MLXSW_SP_MIRROR_LATENCY_SHIFT;
}
-static void mlxsw_sp_rx_sample_listener(struct sk_buff *skb, u8 local_port,
+static void mlxsw_sp_rx_sample_listener(struct sk_buff *skb, u16 local_port,
void *trap_ctx)
{
struct mlxsw_sp *mlxsw_sp = devlink_trap_ctx_priv(trap_ctx);
@@ -293,7 +293,7 @@ static void mlxsw_sp_rx_sample_listener(struct sk_buff *skb, u8 local_port,
consume_skb(skb);
}
-static void mlxsw_sp_rx_sample_tx_listener(struct sk_buff *skb, u8 local_port,
+static void mlxsw_sp_rx_sample_tx_listener(struct sk_buff *skb, u16 local_port,
void *trap_ctx)
{
struct mlxsw_rx_md_info *rx_md_info = &mlxsw_skb_cb(skb)->rx_md_info;
@@ -343,7 +343,7 @@ static void mlxsw_sp_rx_sample_tx_listener(struct sk_buff *skb, u8 local_port,
consume_skb(skb);
}
-static void mlxsw_sp_rx_sample_acl_listener(struct sk_buff *skb, u8 local_port,
+static void mlxsw_sp_rx_sample_acl_listener(struct sk_buff *skb, u16 local_port,
void *trap_ctx)
{
struct mlxsw_sp *mlxsw_sp = devlink_trap_ctx_priv(trap_ctx);
diff --git a/drivers/net/ethernet/micrel/ksz884x.c b/drivers/net/ethernet/micrel/ksz884x.c
index 99c0c14..d024983 100644
--- a/drivers/net/ethernet/micrel/ksz884x.c
+++ b/drivers/net/ethernet/micrel/ksz884x.c
@@ -6317,11 +6317,15 @@ static int netdev_set_pauseparam(struct net_device *dev,
* netdev_get_ringparam - get tx/rx ring parameters
* @dev: Network device.
* @ring: Ethtool RING settings data structure.
+ * @kernel_ring: Ethtool external RING settings data structure.
+ * @extack: Netlink handle.
*
* This procedure returns the TX/RX ring settings.
*/
static void netdev_get_ringparam(struct net_device *dev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct dev_priv *priv = netdev_priv(dev);
struct dev_info *hw_priv = priv->adapter;
diff --git a/drivers/net/ethernet/microchip/Kconfig b/drivers/net/ethernet/microchip/Kconfig
index 735eea1..ed7a35c 100644
--- a/drivers/net/ethernet/microchip/Kconfig
+++ b/drivers/net/ethernet/microchip/Kconfig
@@ -55,6 +55,7 @@
To compile this driver as a module, choose M here. The module will be
called lan743x.
+source "drivers/net/ethernet/microchip/lan966x/Kconfig"
source "drivers/net/ethernet/microchip/sparx5/Kconfig"
endif # NET_VENDOR_MICROCHIP
diff --git a/drivers/net/ethernet/microchip/Makefile b/drivers/net/ethernet/microchip/Makefile
index c77dc03..9faa414 100644
--- a/drivers/net/ethernet/microchip/Makefile
+++ b/drivers/net/ethernet/microchip/Makefile
@@ -9,4 +9,5 @@
lan743x-objs := lan743x_main.o lan743x_ethtool.o lan743x_ptp.o
+obj-$(CONFIG_LAN966X_SWITCH) += lan966x/
obj-$(CONFIG_SPARX5_SWITCH) += sparx5/
diff --git a/drivers/net/ethernet/microchip/lan743x_main.c b/drivers/net/ethernet/microchip/lan743x_main.c
index 7d76474..8c6390d9 100644
--- a/drivers/net/ethernet/microchip/lan743x_main.c
+++ b/drivers/net/ethernet/microchip/lan743x_main.c
@@ -1739,13 +1739,10 @@ static int lan743x_tx_ring_init(struct lan743x_tx *tx)
}
if (dma_set_mask_and_coherent(&tx->adapter->pdev->dev,
DMA_BIT_MASK(64))) {
- if (dma_set_mask_and_coherent(&tx->adapter->pdev->dev,
- DMA_BIT_MASK(32))) {
- dev_warn(&tx->adapter->pdev->dev,
- "lan743x_: No suitable DMA available\n");
- ret = -ENOMEM;
- goto cleanup;
- }
+ dev_warn(&tx->adapter->pdev->dev,
+ "lan743x_: No suitable DMA available\n");
+ ret = -ENOMEM;
+ goto cleanup;
}
ring_allocation_size = ALIGN(tx->ring_size *
sizeof(struct lan743x_tx_descriptor),
@@ -2284,13 +2281,10 @@ static int lan743x_rx_ring_init(struct lan743x_rx *rx)
}
if (dma_set_mask_and_coherent(&rx->adapter->pdev->dev,
DMA_BIT_MASK(64))) {
- if (dma_set_mask_and_coherent(&rx->adapter->pdev->dev,
- DMA_BIT_MASK(32))) {
- dev_warn(&rx->adapter->pdev->dev,
- "lan743x_: No suitable DMA available\n");
- ret = -ENOMEM;
- goto cleanup;
- }
+ dev_warn(&rx->adapter->pdev->dev,
+ "lan743x_: No suitable DMA available\n");
+ ret = -ENOMEM;
+ goto cleanup;
}
ring_allocation_size = ALIGN(rx->ring_size *
sizeof(struct lan743x_rx_descriptor),
diff --git a/drivers/net/ethernet/microchip/lan743x_ptp.c b/drivers/net/ethernet/microchip/lan743x_ptp.c
index 9380e39..8b7a8d8 100644
--- a/drivers/net/ethernet/microchip/lan743x_ptp.c
+++ b/drivers/net/ethernet/microchip/lan743x_ptp.c
@@ -1305,12 +1305,6 @@ int lan743x_ptp_ioctl(struct net_device *netdev, struct ifreq *ifr, int cmd)
if (copy_from_user(&config, ifr->ifr_data, sizeof(config)))
return -EFAULT;
- if (config.flags) {
- netif_warn(adapter, drv, adapter->netdev,
- "ignoring hwtstamp_config.flags == 0x%08X, expected 0\n",
- config.flags);
- }
-
switch (config.tx_type) {
case HWTSTAMP_TX_OFF:
for (index = 0; index < LAN743X_MAX_TX_CHANNELS;
diff --git a/drivers/net/ethernet/microchip/lan966x/Kconfig b/drivers/net/ethernet/microchip/lan966x/Kconfig
new file mode 100644
index 0000000..ac273f8
--- /dev/null
+++ b/drivers/net/ethernet/microchip/lan966x/Kconfig
@@ -0,0 +1,9 @@
+config LAN966X_SWITCH
+ tristate "Lan966x switch driver"
+ depends on HAS_IOMEM
+ depends on OF
+ depends on NET_SWITCHDEV
+ select PHYLINK
+ select PACKING
+ help
+ This driver supports the Lan966x network switch device.
diff --git a/drivers/net/ethernet/microchip/lan966x/Makefile b/drivers/net/ethernet/microchip/lan966x/Makefile
new file mode 100644
index 0000000..040cfff
--- /dev/null
+++ b/drivers/net/ethernet/microchip/lan966x/Makefile
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Makefile for the Microchip Lan966x network device drivers.
+#
+
+obj-$(CONFIG_LAN966X_SWITCH) += lan966x-switch.o
+
+lan966x-switch-objs := lan966x_main.o lan966x_phylink.o lan966x_port.o \
+ lan966x_mac.o lan966x_ethtool.o lan966x_switchdev.o \
+ lan966x_vlan.o lan966x_fdb.o lan966x_mdb.o
diff --git a/drivers/net/ethernet/microchip/lan966x/lan966x_ethtool.c b/drivers/net/ethernet/microchip/lan966x/lan966x_ethtool.c
new file mode 100644
index 0000000..614f12c
--- /dev/null
+++ b/drivers/net/ethernet/microchip/lan966x/lan966x_ethtool.c
@@ -0,0 +1,682 @@
+// SPDX-License-Identifier: GPL-2.0+
+
+#include <linux/netdevice.h>
+
+#include "lan966x_main.h"
+
+/* Number of traffic classes */
+#define LAN966X_NUM_TC 8
+#define LAN966X_STATS_CHECK_DELAY (2 * HZ)
+
+static const struct lan966x_stat_layout lan966x_stats_layout[] = {
+ { .name = "rx_octets", .offset = 0x00, },
+ { .name = "rx_unicast", .offset = 0x01, },
+ { .name = "rx_multicast", .offset = 0x02 },
+ { .name = "rx_broadcast", .offset = 0x03 },
+ { .name = "rx_short", .offset = 0x04 },
+ { .name = "rx_frag", .offset = 0x05 },
+ { .name = "rx_jabber", .offset = 0x06 },
+ { .name = "rx_crc", .offset = 0x07 },
+ { .name = "rx_symbol_err", .offset = 0x08 },
+ { .name = "rx_sz_64", .offset = 0x09 },
+ { .name = "rx_sz_65_127", .offset = 0x0a},
+ { .name = "rx_sz_128_255", .offset = 0x0b},
+ { .name = "rx_sz_256_511", .offset = 0x0c },
+ { .name = "rx_sz_512_1023", .offset = 0x0d },
+ { .name = "rx_sz_1024_1526", .offset = 0x0e },
+ { .name = "rx_sz_jumbo", .offset = 0x0f },
+ { .name = "rx_pause", .offset = 0x10 },
+ { .name = "rx_control", .offset = 0x11 },
+ { .name = "rx_long", .offset = 0x12 },
+ { .name = "rx_cat_drop", .offset = 0x13 },
+ { .name = "rx_red_prio_0", .offset = 0x14 },
+ { .name = "rx_red_prio_1", .offset = 0x15 },
+ { .name = "rx_red_prio_2", .offset = 0x16 },
+ { .name = "rx_red_prio_3", .offset = 0x17 },
+ { .name = "rx_red_prio_4", .offset = 0x18 },
+ { .name = "rx_red_prio_5", .offset = 0x19 },
+ { .name = "rx_red_prio_6", .offset = 0x1a },
+ { .name = "rx_red_prio_7", .offset = 0x1b },
+ { .name = "rx_yellow_prio_0", .offset = 0x1c },
+ { .name = "rx_yellow_prio_1", .offset = 0x1d },
+ { .name = "rx_yellow_prio_2", .offset = 0x1e },
+ { .name = "rx_yellow_prio_3", .offset = 0x1f },
+ { .name = "rx_yellow_prio_4", .offset = 0x20 },
+ { .name = "rx_yellow_prio_5", .offset = 0x21 },
+ { .name = "rx_yellow_prio_6", .offset = 0x22 },
+ { .name = "rx_yellow_prio_7", .offset = 0x23 },
+ { .name = "rx_green_prio_0", .offset = 0x24 },
+ { .name = "rx_green_prio_1", .offset = 0x25 },
+ { .name = "rx_green_prio_2", .offset = 0x26 },
+ { .name = "rx_green_prio_3", .offset = 0x27 },
+ { .name = "rx_green_prio_4", .offset = 0x28 },
+ { .name = "rx_green_prio_5", .offset = 0x29 },
+ { .name = "rx_green_prio_6", .offset = 0x2a },
+ { .name = "rx_green_prio_7", .offset = 0x2b },
+ { .name = "rx_assembly_err", .offset = 0x2c },
+ { .name = "rx_smd_err", .offset = 0x2d },
+ { .name = "rx_assembly_ok", .offset = 0x2e },
+ { .name = "rx_merge_frag", .offset = 0x2f },
+ { .name = "rx_pmac_octets", .offset = 0x30, },
+ { .name = "rx_pmac_unicast", .offset = 0x31, },
+ { .name = "rx_pmac_multicast", .offset = 0x32 },
+ { .name = "rx_pmac_broadcast", .offset = 0x33 },
+ { .name = "rx_pmac_short", .offset = 0x34 },
+ { .name = "rx_pmac_frag", .offset = 0x35 },
+ { .name = "rx_pmac_jabber", .offset = 0x36 },
+ { .name = "rx_pmac_crc", .offset = 0x37 },
+ { .name = "rx_pmac_symbol_err", .offset = 0x38 },
+ { .name = "rx_pmac_sz_64", .offset = 0x39 },
+ { .name = "rx_pmac_sz_65_127", .offset = 0x3a },
+ { .name = "rx_pmac_sz_128_255", .offset = 0x3b },
+ { .name = "rx_pmac_sz_256_511", .offset = 0x3c },
+ { .name = "rx_pmac_sz_512_1023", .offset = 0x3d },
+ { .name = "rx_pmac_sz_1024_1526", .offset = 0x3e },
+ { .name = "rx_pmac_sz_jumbo", .offset = 0x3f },
+ { .name = "rx_pmac_pause", .offset = 0x40 },
+ { .name = "rx_pmac_control", .offset = 0x41 },
+ { .name = "rx_pmac_long", .offset = 0x42 },
+
+ { .name = "tx_octets", .offset = 0x80, },
+ { .name = "tx_unicast", .offset = 0x81, },
+ { .name = "tx_multicast", .offset = 0x82 },
+ { .name = "tx_broadcast", .offset = 0x83 },
+ { .name = "tx_col", .offset = 0x84 },
+ { .name = "tx_drop", .offset = 0x85 },
+ { .name = "tx_pause", .offset = 0x86 },
+ { .name = "tx_sz_64", .offset = 0x87 },
+ { .name = "tx_sz_65_127", .offset = 0x88 },
+ { .name = "tx_sz_128_255", .offset = 0x89 },
+ { .name = "tx_sz_256_511", .offset = 0x8a },
+ { .name = "tx_sz_512_1023", .offset = 0x8b },
+ { .name = "tx_sz_1024_1526", .offset = 0x8c },
+ { .name = "tx_sz_jumbo", .offset = 0x8d },
+ { .name = "tx_yellow_prio_0", .offset = 0x8e },
+ { .name = "tx_yellow_prio_1", .offset = 0x8f },
+ { .name = "tx_yellow_prio_2", .offset = 0x90 },
+ { .name = "tx_yellow_prio_3", .offset = 0x91 },
+ { .name = "tx_yellow_prio_4", .offset = 0x92 },
+ { .name = "tx_yellow_prio_5", .offset = 0x93 },
+ { .name = "tx_yellow_prio_6", .offset = 0x94 },
+ { .name = "tx_yellow_prio_7", .offset = 0x95 },
+ { .name = "tx_green_prio_0", .offset = 0x96 },
+ { .name = "tx_green_prio_1", .offset = 0x97 },
+ { .name = "tx_green_prio_2", .offset = 0x98 },
+ { .name = "tx_green_prio_3", .offset = 0x99 },
+ { .name = "tx_green_prio_4", .offset = 0x9a },
+ { .name = "tx_green_prio_5", .offset = 0x9b },
+ { .name = "tx_green_prio_6", .offset = 0x9c },
+ { .name = "tx_green_prio_7", .offset = 0x9d },
+ { .name = "tx_aged", .offset = 0x9e },
+ { .name = "tx_llct", .offset = 0x9f },
+ { .name = "tx_ct", .offset = 0xa0 },
+ { .name = "tx_mm_hold", .offset = 0xa1 },
+ { .name = "tx_merge_frag", .offset = 0xa2 },
+ { .name = "tx_pmac_octets", .offset = 0xa3, },
+ { .name = "tx_pmac_unicast", .offset = 0xa4, },
+ { .name = "tx_pmac_multicast", .offset = 0xa5 },
+ { .name = "tx_pmac_broadcast", .offset = 0xa6 },
+ { .name = "tx_pmac_pause", .offset = 0xa7 },
+ { .name = "tx_pmac_sz_64", .offset = 0xa8 },
+ { .name = "tx_pmac_sz_65_127", .offset = 0xa9 },
+ { .name = "tx_pmac_sz_128_255", .offset = 0xaa },
+ { .name = "tx_pmac_sz_256_511", .offset = 0xab },
+ { .name = "tx_pmac_sz_512_1023", .offset = 0xac },
+ { .name = "tx_pmac_sz_1024_1526", .offset = 0xad },
+ { .name = "tx_pmac_sz_jumbo", .offset = 0xae },
+
+ { .name = "dr_local", .offset = 0x100 },
+ { .name = "dr_tail", .offset = 0x101 },
+ { .name = "dr_yellow_prio_0", .offset = 0x102 },
+ { .name = "dr_yellow_prio_1", .offset = 0x103 },
+ { .name = "dr_yellow_prio_2", .offset = 0x104 },
+ { .name = "dr_yellow_prio_3", .offset = 0x105 },
+ { .name = "dr_yellow_prio_4", .offset = 0x106 },
+ { .name = "dr_yellow_prio_5", .offset = 0x107 },
+ { .name = "dr_yellow_prio_6", .offset = 0x108 },
+ { .name = "dr_yellow_prio_7", .offset = 0x109 },
+ { .name = "dr_green_prio_0", .offset = 0x10a },
+ { .name = "dr_green_prio_1", .offset = 0x10b },
+ { .name = "dr_green_prio_2", .offset = 0x10c },
+ { .name = "dr_green_prio_3", .offset = 0x10d },
+ { .name = "dr_green_prio_4", .offset = 0x10e },
+ { .name = "dr_green_prio_5", .offset = 0x10f },
+ { .name = "dr_green_prio_6", .offset = 0x110 },
+ { .name = "dr_green_prio_7", .offset = 0x111 },
+};
+
+/* The following numbers are indexes into lan966x_stats_layout[] */
+#define SYS_COUNT_RX_OCT 0
+#define SYS_COUNT_RX_UC 1
+#define SYS_COUNT_RX_MC 2
+#define SYS_COUNT_RX_BC 3
+#define SYS_COUNT_RX_SHORT 4
+#define SYS_COUNT_RX_FRAG 5
+#define SYS_COUNT_RX_JABBER 6
+#define SYS_COUNT_RX_CRC 7
+#define SYS_COUNT_RX_SYMBOL_ERR 8
+#define SYS_COUNT_RX_SZ_64 9
+#define SYS_COUNT_RX_SZ_65_127 10
+#define SYS_COUNT_RX_SZ_128_255 11
+#define SYS_COUNT_RX_SZ_256_511 12
+#define SYS_COUNT_RX_SZ_512_1023 13
+#define SYS_COUNT_RX_SZ_1024_1526 14
+#define SYS_COUNT_RX_SZ_JUMBO 15
+#define SYS_COUNT_RX_PAUSE 16
+#define SYS_COUNT_RX_CONTROL 17
+#define SYS_COUNT_RX_LONG 18
+#define SYS_COUNT_RX_CAT_DROP 19
+#define SYS_COUNT_RX_RED_PRIO_0 20
+#define SYS_COUNT_RX_RED_PRIO_1 21
+#define SYS_COUNT_RX_RED_PRIO_2 22
+#define SYS_COUNT_RX_RED_PRIO_3 23
+#define SYS_COUNT_RX_RED_PRIO_4 24
+#define SYS_COUNT_RX_RED_PRIO_5 25
+#define SYS_COUNT_RX_RED_PRIO_6 26
+#define SYS_COUNT_RX_RED_PRIO_7 27
+#define SYS_COUNT_RX_YELLOW_PRIO_0 28
+#define SYS_COUNT_RX_YELLOW_PRIO_1 29
+#define SYS_COUNT_RX_YELLOW_PRIO_2 30
+#define SYS_COUNT_RX_YELLOW_PRIO_3 31
+#define SYS_COUNT_RX_YELLOW_PRIO_4 32
+#define SYS_COUNT_RX_YELLOW_PRIO_5 33
+#define SYS_COUNT_RX_YELLOW_PRIO_6 34
+#define SYS_COUNT_RX_YELLOW_PRIO_7 35
+#define SYS_COUNT_RX_GREEN_PRIO_0 36
+#define SYS_COUNT_RX_GREEN_PRIO_1 37
+#define SYS_COUNT_RX_GREEN_PRIO_2 38
+#define SYS_COUNT_RX_GREEN_PRIO_3 39
+#define SYS_COUNT_RX_GREEN_PRIO_4 40
+#define SYS_COUNT_RX_GREEN_PRIO_5 41
+#define SYS_COUNT_RX_GREEN_PRIO_6 42
+#define SYS_COUNT_RX_GREEN_PRIO_7 43
+#define SYS_COUNT_RX_ASSEMBLY_ERR 44
+#define SYS_COUNT_RX_SMD_ERR 45
+#define SYS_COUNT_RX_ASSEMBLY_OK 46
+#define SYS_COUNT_RX_MERGE_FRAG 47
+#define SYS_COUNT_RX_PMAC_OCT 48
+#define SYS_COUNT_RX_PMAC_UC 49
+#define SYS_COUNT_RX_PMAC_MC 50
+#define SYS_COUNT_RX_PMAC_BC 51
+#define SYS_COUNT_RX_PMAC_SHORT 52
+#define SYS_COUNT_RX_PMAC_FRAG 53
+#define SYS_COUNT_RX_PMAC_JABBER 54
+#define SYS_COUNT_RX_PMAC_CRC 55
+#define SYS_COUNT_RX_PMAC_SYMBOL_ERR 56
+#define SYS_COUNT_RX_PMAC_SZ_64 57
+#define SYS_COUNT_RX_PMAC_SZ_65_127 58
+#define SYS_COUNT_RX_PMAC_SZ_128_255 59
+#define SYS_COUNT_RX_PMAC_SZ_256_511 60
+#define SYS_COUNT_RX_PMAC_SZ_512_1023 61
+#define SYS_COUNT_RX_PMAC_SZ_1024_1526 62
+#define SYS_COUNT_RX_PMAC_SZ_JUMBO 63
+#define SYS_COUNT_RX_PMAC_PAUSE 64
+#define SYS_COUNT_RX_PMAC_CONTROL 65
+#define SYS_COUNT_RX_PMAC_LONG 66
+
+#define SYS_COUNT_TX_OCT 67
+#define SYS_COUNT_TX_UC 68
+#define SYS_COUNT_TX_MC 69
+#define SYS_COUNT_TX_BC 70
+#define SYS_COUNT_TX_COL 71
+#define SYS_COUNT_TX_DROP 72
+#define SYS_COUNT_TX_PAUSE 73
+#define SYS_COUNT_TX_SZ_64 74
+#define SYS_COUNT_TX_SZ_65_127 75
+#define SYS_COUNT_TX_SZ_128_255 76
+#define SYS_COUNT_TX_SZ_256_511 77
+#define SYS_COUNT_TX_SZ_512_1023 78
+#define SYS_COUNT_TX_SZ_1024_1526 79
+#define SYS_COUNT_TX_SZ_JUMBO 80
+#define SYS_COUNT_TX_YELLOW_PRIO_0 81
+#define SYS_COUNT_TX_YELLOW_PRIO_1 82
+#define SYS_COUNT_TX_YELLOW_PRIO_2 83
+#define SYS_COUNT_TX_YELLOW_PRIO_3 84
+#define SYS_COUNT_TX_YELLOW_PRIO_4 85
+#define SYS_COUNT_TX_YELLOW_PRIO_5 86
+#define SYS_COUNT_TX_YELLOW_PRIO_6 87
+#define SYS_COUNT_TX_YELLOW_PRIO_7 88
+#define SYS_COUNT_TX_GREEN_PRIO_0 89
+#define SYS_COUNT_TX_GREEN_PRIO_1 90
+#define SYS_COUNT_TX_GREEN_PRIO_2 91
+#define SYS_COUNT_TX_GREEN_PRIO_3 92
+#define SYS_COUNT_TX_GREEN_PRIO_4 93
+#define SYS_COUNT_TX_GREEN_PRIO_5 94
+#define SYS_COUNT_TX_GREEN_PRIO_6 95
+#define SYS_COUNT_TX_GREEN_PRIO_7 96
+#define SYS_COUNT_TX_AGED 97
+#define SYS_COUNT_TX_LLCT 98
+#define SYS_COUNT_TX_CT 99
+#define SYS_COUNT_TX_MM_HOLD 100
+#define SYS_COUNT_TX_MERGE_FRAG 101
+#define SYS_COUNT_TX_PMAC_OCT 102
+#define SYS_COUNT_TX_PMAC_UC 103
+#define SYS_COUNT_TX_PMAC_MC 104
+#define SYS_COUNT_TX_PMAC_BC 105
+#define SYS_COUNT_TX_PMAC_PAUSE 106
+#define SYS_COUNT_TX_PMAC_SZ_64 107
+#define SYS_COUNT_TX_PMAC_SZ_65_127 108
+#define SYS_COUNT_TX_PMAC_SZ_128_255 109
+#define SYS_COUNT_TX_PMAC_SZ_256_511 110
+#define SYS_COUNT_TX_PMAC_SZ_512_1023 111
+#define SYS_COUNT_TX_PMAC_SZ_1024_1526 112
+#define SYS_COUNT_TX_PMAC_SZ_JUMBO 113
+
+#define SYS_COUNT_DR_LOCAL 114
+#define SYS_COUNT_DR_TAIL 115
+#define SYS_COUNT_DR_YELLOW_PRIO_0 116
+#define SYS_COUNT_DR_YELLOW_PRIO_1 117
+#define SYS_COUNT_DR_YELLOW_PRIO_2 118
+#define SYS_COUNT_DR_YELLOW_PRIO_3 119
+#define SYS_COUNT_DR_YELLOW_PRIO_4 120
+#define SYS_COUNT_DR_YELLOW_PRIO_5 121
+#define SYS_COUNT_DR_YELLOW_PRIO_6 122
+#define SYS_COUNT_DR_YELLOW_PRIO_7 123
+#define SYS_COUNT_DR_GREEN_PRIO_0 124
+#define SYS_COUNT_DR_GREEN_PRIO_1 125
+#define SYS_COUNT_DR_GREEN_PRIO_2 126
+#define SYS_COUNT_DR_GREEN_PRIO_3 127
+#define SYS_COUNT_DR_GREEN_PRIO_4 128
+#define SYS_COUNT_DR_GREEN_PRIO_5 129
+#define SYS_COUNT_DR_GREEN_PRIO_6 130
+#define SYS_COUNT_DR_GREEN_PRIO_7 131
+
+/* Add a possibly wrapping 32 bit value to a 64 bit counter */
+static void lan966x_add_cnt(u64 *cnt, u32 val)
+{
+ if (val < (*cnt & U32_MAX))
+ *cnt += (u64)1 << 32; /* value has wrapped */
+
+ *cnt = (*cnt & ~(u64)U32_MAX) + val;
+}
+
+static void lan966x_stats_update(struct lan966x *lan966x)
+{
+ int i, j;
+
+ mutex_lock(&lan966x->stats_lock);
+
+ for (i = 0; i < lan966x->num_phys_ports; i++) {
+ uint idx = i * lan966x->num_stats;
+
+ lan_wr(SYS_STAT_CFG_STAT_VIEW_SET(i),
+ lan966x, SYS_STAT_CFG);
+
+ for (j = 0; j < lan966x->num_stats; j++) {
+ u32 offset = lan966x->stats_layout[j].offset;
+
+ lan966x_add_cnt(&lan966x->stats[idx++],
+ lan_rd(lan966x, SYS_CNT(offset)));
+ }
+ }
+
+ mutex_unlock(&lan966x->stats_lock);
+}
+
+static int lan966x_get_sset_count(struct net_device *dev, int sset)
+{
+ struct lan966x_port *port = netdev_priv(dev);
+ struct lan966x *lan966x = port->lan966x;
+
+ if (sset != ETH_SS_STATS)
+ return -EOPNOTSUPP;
+
+ return lan966x->num_stats;
+}
+
+static void lan966x_get_strings(struct net_device *netdev, u32 sset, u8 *data)
+{
+ struct lan966x_port *port = netdev_priv(netdev);
+ struct lan966x *lan966x = port->lan966x;
+ int i;
+
+ if (sset != ETH_SS_STATS)
+ return;
+
+ for (i = 0; i < lan966x->num_stats; i++)
+ memcpy(data + i * ETH_GSTRING_LEN,
+ lan966x->stats_layout[i].name, ETH_GSTRING_LEN);
+}
+
+static void lan966x_get_ethtool_stats(struct net_device *dev,
+ struct ethtool_stats *stats, u64 *data)
+{
+ struct lan966x_port *port = netdev_priv(dev);
+ struct lan966x *lan966x = port->lan966x;
+ int i;
+
+ /* check and update now */
+ lan966x_stats_update(lan966x);
+
+ /* Copy all counters */
+ for (i = 0; i < lan966x->num_stats; i++)
+ *data++ = lan966x->stats[port->chip_port *
+ lan966x->num_stats + i];
+}
+
+static void lan966x_get_eth_mac_stats(struct net_device *dev,
+ struct ethtool_eth_mac_stats *mac_stats)
+{
+ struct lan966x_port *port = netdev_priv(dev);
+ struct lan966x *lan966x = port->lan966x;
+ u32 idx;
+
+ lan966x_stats_update(lan966x);
+
+ idx = port->chip_port * lan966x->num_stats;
+
+ mutex_lock(&lan966x->stats_lock);
+
+ mac_stats->FramesTransmittedOK =
+ lan966x->stats[idx + SYS_COUNT_TX_UC] +
+ lan966x->stats[idx + SYS_COUNT_TX_MC] +
+ lan966x->stats[idx + SYS_COUNT_TX_BC] +
+ lan966x->stats[idx + SYS_COUNT_TX_PMAC_UC] +
+ lan966x->stats[idx + SYS_COUNT_TX_PMAC_MC] +
+ lan966x->stats[idx + SYS_COUNT_TX_PMAC_BC];
+ mac_stats->SingleCollisionFrames =
+ lan966x->stats[idx + SYS_COUNT_TX_COL];
+ mac_stats->MultipleCollisionFrames = 0;
+ mac_stats->FramesReceivedOK =
+ lan966x->stats[idx + SYS_COUNT_RX_UC] +
+ lan966x->stats[idx + SYS_COUNT_RX_MC] +
+ lan966x->stats[idx + SYS_COUNT_RX_BC];
+ mac_stats->FrameCheckSequenceErrors =
+ lan966x->stats[idx + SYS_COUNT_RX_CRC] +
+ lan966x->stats[idx + SYS_COUNT_RX_CRC];
+ mac_stats->AlignmentErrors = 0;
+ mac_stats->OctetsTransmittedOK =
+ lan966x->stats[idx + SYS_COUNT_TX_OCT] +
+ lan966x->stats[idx + SYS_COUNT_TX_PMAC_OCT];
+ mac_stats->FramesWithDeferredXmissions =
+ lan966x->stats[idx + SYS_COUNT_TX_MM_HOLD];
+ mac_stats->LateCollisions = 0;
+ mac_stats->FramesAbortedDueToXSColls = 0;
+ mac_stats->FramesLostDueToIntMACXmitError = 0;
+ mac_stats->CarrierSenseErrors = 0;
+ mac_stats->OctetsReceivedOK =
+ lan966x->stats[idx + SYS_COUNT_RX_OCT];
+ mac_stats->FramesLostDueToIntMACRcvError = 0;
+ mac_stats->MulticastFramesXmittedOK =
+ lan966x->stats[idx + SYS_COUNT_TX_MC] +
+ lan966x->stats[idx + SYS_COUNT_TX_PMAC_MC];
+ mac_stats->BroadcastFramesXmittedOK =
+ lan966x->stats[idx + SYS_COUNT_TX_BC] +
+ lan966x->stats[idx + SYS_COUNT_TX_PMAC_BC];
+ mac_stats->FramesWithExcessiveDeferral = 0;
+ mac_stats->MulticastFramesReceivedOK =
+ lan966x->stats[idx + SYS_COUNT_RX_MC];
+ mac_stats->BroadcastFramesReceivedOK =
+ lan966x->stats[idx + SYS_COUNT_RX_BC];
+ mac_stats->InRangeLengthErrors =
+ lan966x->stats[idx + SYS_COUNT_RX_FRAG] +
+ lan966x->stats[idx + SYS_COUNT_RX_JABBER] +
+ lan966x->stats[idx + SYS_COUNT_RX_CRC] +
+ lan966x->stats[idx + SYS_COUNT_RX_PMAC_FRAG] +
+ lan966x->stats[idx + SYS_COUNT_RX_PMAC_JABBER] +
+ lan966x->stats[idx + SYS_COUNT_RX_PMAC_CRC];
+ mac_stats->OutOfRangeLengthField =
+ lan966x->stats[idx + SYS_COUNT_RX_SHORT] +
+ lan966x->stats[idx + SYS_COUNT_RX_PMAC_SHORT] +
+ lan966x->stats[idx + SYS_COUNT_RX_LONG] +
+ lan966x->stats[idx + SYS_COUNT_RX_PMAC_LONG];
+ mac_stats->FrameTooLongErrors =
+ lan966x->stats[idx + SYS_COUNT_RX_LONG] +
+ lan966x->stats[idx + SYS_COUNT_RX_PMAC_LONG];
+
+ mutex_unlock(&lan966x->stats_lock);
+}
+
+static const struct ethtool_rmon_hist_range lan966x_rmon_ranges[] = {
+ { 0, 64 },
+ { 65, 127 },
+ { 128, 255 },
+ { 256, 511 },
+ { 512, 1023 },
+ { 1024, 1518 },
+ { 1519, 10239 },
+ {}
+};
+
+static void lan966x_get_eth_rmon_stats(struct net_device *dev,
+ struct ethtool_rmon_stats *rmon_stats,
+ const struct ethtool_rmon_hist_range **ranges)
+{
+ struct lan966x_port *port = netdev_priv(dev);
+ struct lan966x *lan966x = port->lan966x;
+ u32 idx;
+
+ lan966x_stats_update(lan966x);
+
+ idx = port->chip_port * lan966x->num_stats;
+
+ mutex_lock(&lan966x->stats_lock);
+
+ rmon_stats->undersize_pkts =
+ lan966x->stats[idx + SYS_COUNT_RX_SHORT] +
+ lan966x->stats[idx + SYS_COUNT_RX_PMAC_SHORT];
+ rmon_stats->oversize_pkts =
+ lan966x->stats[idx + SYS_COUNT_RX_LONG] +
+ lan966x->stats[idx + SYS_COUNT_RX_PMAC_LONG];
+ rmon_stats->fragments =
+ lan966x->stats[idx + SYS_COUNT_RX_FRAG] +
+ lan966x->stats[idx + SYS_COUNT_RX_PMAC_FRAG];
+ rmon_stats->jabbers =
+ lan966x->stats[idx + SYS_COUNT_RX_JABBER] +
+ lan966x->stats[idx + SYS_COUNT_RX_PMAC_JABBER];
+ rmon_stats->hist[0] =
+ lan966x->stats[idx + SYS_COUNT_RX_SZ_64] +
+ lan966x->stats[idx + SYS_COUNT_RX_PMAC_SZ_64];
+ rmon_stats->hist[1] =
+ lan966x->stats[idx + SYS_COUNT_RX_SZ_65_127] +
+ lan966x->stats[idx + SYS_COUNT_RX_PMAC_SZ_65_127];
+ rmon_stats->hist[2] =
+ lan966x->stats[idx + SYS_COUNT_RX_SZ_128_255] +
+ lan966x->stats[idx + SYS_COUNT_RX_PMAC_SZ_128_255];
+ rmon_stats->hist[3] =
+ lan966x->stats[idx + SYS_COUNT_RX_SZ_256_511] +
+ lan966x->stats[idx + SYS_COUNT_RX_PMAC_SZ_256_511];
+ rmon_stats->hist[4] =
+ lan966x->stats[idx + SYS_COUNT_RX_SZ_512_1023] +
+ lan966x->stats[idx + SYS_COUNT_RX_PMAC_SZ_512_1023];
+ rmon_stats->hist[5] =
+ lan966x->stats[idx + SYS_COUNT_RX_SZ_1024_1526] +
+ lan966x->stats[idx + SYS_COUNT_RX_PMAC_SZ_1024_1526];
+ rmon_stats->hist[6] =
+ lan966x->stats[idx + SYS_COUNT_RX_SZ_1024_1526] +
+ lan966x->stats[idx + SYS_COUNT_RX_PMAC_SZ_1024_1526];
+
+ rmon_stats->hist_tx[0] =
+ lan966x->stats[idx + SYS_COUNT_TX_SZ_64] +
+ lan966x->stats[idx + SYS_COUNT_TX_PMAC_SZ_64];
+ rmon_stats->hist_tx[1] =
+ lan966x->stats[idx + SYS_COUNT_TX_SZ_65_127] +
+ lan966x->stats[idx + SYS_COUNT_TX_PMAC_SZ_65_127];
+ rmon_stats->hist_tx[2] =
+ lan966x->stats[idx + SYS_COUNT_TX_SZ_128_255] +
+ lan966x->stats[idx + SYS_COUNT_TX_PMAC_SZ_128_255];
+ rmon_stats->hist_tx[3] =
+ lan966x->stats[idx + SYS_COUNT_TX_SZ_256_511] +
+ lan966x->stats[idx + SYS_COUNT_TX_PMAC_SZ_256_511];
+ rmon_stats->hist_tx[4] =
+ lan966x->stats[idx + SYS_COUNT_TX_SZ_512_1023] +
+ lan966x->stats[idx + SYS_COUNT_TX_PMAC_SZ_512_1023];
+ rmon_stats->hist_tx[5] =
+ lan966x->stats[idx + SYS_COUNT_TX_SZ_1024_1526] +
+ lan966x->stats[idx + SYS_COUNT_TX_PMAC_SZ_1024_1526];
+ rmon_stats->hist_tx[6] =
+ lan966x->stats[idx + SYS_COUNT_TX_SZ_1024_1526] +
+ lan966x->stats[idx + SYS_COUNT_TX_PMAC_SZ_1024_1526];
+
+ mutex_unlock(&lan966x->stats_lock);
+
+ *ranges = lan966x_rmon_ranges;
+}
+
+static int lan966x_get_link_ksettings(struct net_device *ndev,
+ struct ethtool_link_ksettings *cmd)
+{
+ struct lan966x_port *port = netdev_priv(ndev);
+
+ return phylink_ethtool_ksettings_get(port->phylink, cmd);
+}
+
+static int lan966x_set_link_ksettings(struct net_device *ndev,
+ const struct ethtool_link_ksettings *cmd)
+{
+ struct lan966x_port *port = netdev_priv(ndev);
+
+ return phylink_ethtool_ksettings_set(port->phylink, cmd);
+}
+
+static void lan966x_get_pauseparam(struct net_device *dev,
+ struct ethtool_pauseparam *pause)
+{
+ struct lan966x_port *port = netdev_priv(dev);
+
+ phylink_ethtool_get_pauseparam(port->phylink, pause);
+}
+
+static int lan966x_set_pauseparam(struct net_device *dev,
+ struct ethtool_pauseparam *pause)
+{
+ struct lan966x_port *port = netdev_priv(dev);
+
+ return phylink_ethtool_set_pauseparam(port->phylink, pause);
+}
+
+const struct ethtool_ops lan966x_ethtool_ops = {
+ .get_link_ksettings = lan966x_get_link_ksettings,
+ .set_link_ksettings = lan966x_set_link_ksettings,
+ .get_pauseparam = lan966x_get_pauseparam,
+ .set_pauseparam = lan966x_set_pauseparam,
+ .get_sset_count = lan966x_get_sset_count,
+ .get_strings = lan966x_get_strings,
+ .get_ethtool_stats = lan966x_get_ethtool_stats,
+ .get_eth_mac_stats = lan966x_get_eth_mac_stats,
+ .get_rmon_stats = lan966x_get_eth_rmon_stats,
+ .get_link = ethtool_op_get_link,
+};
+
+static void lan966x_check_stats_work(struct work_struct *work)
+{
+ struct delayed_work *del_work = to_delayed_work(work);
+ struct lan966x *lan966x = container_of(del_work, struct lan966x,
+ stats_work);
+
+ lan966x_stats_update(lan966x);
+
+ queue_delayed_work(lan966x->stats_queue, &lan966x->stats_work,
+ LAN966X_STATS_CHECK_DELAY);
+}
+
+void lan966x_stats_get(struct net_device *dev,
+ struct rtnl_link_stats64 *stats)
+{
+ struct lan966x_port *port = netdev_priv(dev);
+ struct lan966x *lan966x = port->lan966x;
+ u32 idx;
+ int i;
+
+ idx = port->chip_port * lan966x->num_stats;
+
+ mutex_lock(&lan966x->stats_lock);
+
+ stats->rx_bytes = lan966x->stats[idx + SYS_COUNT_RX_OCT] +
+ lan966x->stats[idx + SYS_COUNT_RX_PMAC_OCT];
+
+ stats->rx_packets = lan966x->stats[idx + SYS_COUNT_RX_SHORT] +
+ lan966x->stats[idx + SYS_COUNT_RX_FRAG] +
+ lan966x->stats[idx + SYS_COUNT_RX_JABBER] +
+ lan966x->stats[idx + SYS_COUNT_RX_CRC] +
+ lan966x->stats[idx + SYS_COUNT_RX_SYMBOL_ERR] +
+ lan966x->stats[idx + SYS_COUNT_RX_SZ_64] +
+ lan966x->stats[idx + SYS_COUNT_RX_SZ_65_127] +
+ lan966x->stats[idx + SYS_COUNT_RX_SZ_128_255] +
+ lan966x->stats[idx + SYS_COUNT_RX_SZ_256_511] +
+ lan966x->stats[idx + SYS_COUNT_RX_SZ_512_1023] +
+ lan966x->stats[idx + SYS_COUNT_RX_SZ_1024_1526] +
+ lan966x->stats[idx + SYS_COUNT_RX_SZ_JUMBO] +
+ lan966x->stats[idx + SYS_COUNT_RX_LONG] +
+ lan966x->stats[idx + SYS_COUNT_RX_PMAC_SHORT] +
+ lan966x->stats[idx + SYS_COUNT_RX_PMAC_FRAG] +
+ lan966x->stats[idx + SYS_COUNT_RX_PMAC_JABBER] +
+ lan966x->stats[idx + SYS_COUNT_RX_PMAC_SZ_64] +
+ lan966x->stats[idx + SYS_COUNT_RX_PMAC_SZ_65_127] +
+ lan966x->stats[idx + SYS_COUNT_RX_PMAC_SZ_128_255] +
+ lan966x->stats[idx + SYS_COUNT_RX_PMAC_SZ_256_511] +
+ lan966x->stats[idx + SYS_COUNT_RX_PMAC_SZ_512_1023] +
+ lan966x->stats[idx + SYS_COUNT_RX_PMAC_SZ_1024_1526] +
+ lan966x->stats[idx + SYS_COUNT_RX_PMAC_SZ_JUMBO];
+
+ stats->multicast = lan966x->stats[idx + SYS_COUNT_RX_MC] +
+ lan966x->stats[idx + SYS_COUNT_RX_PMAC_MC];
+
+ stats->rx_errors = lan966x->stats[idx + SYS_COUNT_RX_SHORT] +
+ lan966x->stats[idx + SYS_COUNT_RX_FRAG] +
+ lan966x->stats[idx + SYS_COUNT_RX_JABBER] +
+ lan966x->stats[idx + SYS_COUNT_RX_CRC] +
+ lan966x->stats[idx + SYS_COUNT_RX_SYMBOL_ERR] +
+ lan966x->stats[idx + SYS_COUNT_RX_LONG];
+
+ stats->rx_dropped = dev->stats.rx_dropped +
+ lan966x->stats[idx + SYS_COUNT_RX_LONG] +
+ lan966x->stats[idx + SYS_COUNT_DR_LOCAL] +
+ lan966x->stats[idx + SYS_COUNT_DR_TAIL];
+
+ for (i = 0; i < LAN966X_NUM_TC; i++) {
+ stats->rx_dropped +=
+ (lan966x->stats[idx + SYS_COUNT_DR_YELLOW_PRIO_0 + i] +
+ lan966x->stats[idx + SYS_COUNT_DR_GREEN_PRIO_0 + i]);
+ }
+
+ /* Get Tx stats */
+ stats->tx_bytes = lan966x->stats[idx + SYS_COUNT_TX_OCT] +
+ lan966x->stats[idx + SYS_COUNT_TX_PMAC_OCT];
+
+ stats->tx_packets = lan966x->stats[idx + SYS_COUNT_TX_SZ_64] +
+ lan966x->stats[idx + SYS_COUNT_TX_SZ_65_127] +
+ lan966x->stats[idx + SYS_COUNT_TX_SZ_128_255] +
+ lan966x->stats[idx + SYS_COUNT_TX_SZ_256_511] +
+ lan966x->stats[idx + SYS_COUNT_TX_SZ_512_1023] +
+ lan966x->stats[idx + SYS_COUNT_TX_SZ_1024_1526] +
+ lan966x->stats[idx + SYS_COUNT_TX_SZ_JUMBO] +
+ lan966x->stats[idx + SYS_COUNT_TX_PMAC_SZ_64] +
+ lan966x->stats[idx + SYS_COUNT_TX_PMAC_SZ_65_127] +
+ lan966x->stats[idx + SYS_COUNT_TX_PMAC_SZ_128_255] +
+ lan966x->stats[idx + SYS_COUNT_TX_PMAC_SZ_256_511] +
+ lan966x->stats[idx + SYS_COUNT_TX_PMAC_SZ_512_1023] +
+ lan966x->stats[idx + SYS_COUNT_TX_PMAC_SZ_1024_1526] +
+ lan966x->stats[idx + SYS_COUNT_TX_PMAC_SZ_JUMBO];
+
+ stats->tx_dropped = lan966x->stats[idx + SYS_COUNT_TX_DROP] +
+ lan966x->stats[idx + SYS_COUNT_TX_AGED];
+
+ stats->collisions = lan966x->stats[idx + SYS_COUNT_TX_COL];
+
+ mutex_unlock(&lan966x->stats_lock);
+}
+
+int lan966x_stats_init(struct lan966x *lan966x)
+{
+ char queue_name[32];
+
+ lan966x->stats_layout = lan966x_stats_layout;
+ lan966x->num_stats = ARRAY_SIZE(lan966x_stats_layout);
+ lan966x->stats = devm_kcalloc(lan966x->dev, lan966x->num_phys_ports *
+ lan966x->num_stats,
+ sizeof(u64), GFP_KERNEL);
+ if (!lan966x->stats)
+ return -ENOMEM;
+
+ /* Init stats worker */
+ mutex_init(&lan966x->stats_lock);
+ snprintf(queue_name, sizeof(queue_name), "%s-stats",
+ dev_name(lan966x->dev));
+ lan966x->stats_queue = create_singlethread_workqueue(queue_name);
+ INIT_DELAYED_WORK(&lan966x->stats_work, lan966x_check_stats_work);
+ queue_delayed_work(lan966x->stats_queue, &lan966x->stats_work,
+ LAN966X_STATS_CHECK_DELAY);
+
+ return 0;
+}
diff --git a/drivers/net/ethernet/microchip/lan966x/lan966x_fdb.c b/drivers/net/ethernet/microchip/lan966x/lan966x_fdb.c
new file mode 100644
index 0000000..da5ca71
--- /dev/null
+++ b/drivers/net/ethernet/microchip/lan966x/lan966x_fdb.c
@@ -0,0 +1,244 @@
+// SPDX-License-Identifier: GPL-2.0+
+
+#include <net/switchdev.h>
+
+#include "lan966x_main.h"
+
+struct lan966x_fdb_event_work {
+ struct work_struct work;
+ struct switchdev_notifier_fdb_info fdb_info;
+ struct net_device *dev;
+ struct lan966x *lan966x;
+ unsigned long event;
+};
+
+struct lan966x_fdb_entry {
+ struct list_head list;
+ unsigned char mac[ETH_ALEN] __aligned(2);
+ u16 vid;
+ u32 references;
+};
+
+static struct lan966x_fdb_entry *
+lan966x_fdb_find_entry(struct lan966x *lan966x,
+ struct switchdev_notifier_fdb_info *fdb_info)
+{
+ struct lan966x_fdb_entry *fdb_entry;
+
+ list_for_each_entry(fdb_entry, &lan966x->fdb_entries, list) {
+ if (fdb_entry->vid == fdb_info->vid &&
+ ether_addr_equal(fdb_entry->mac, fdb_info->addr))
+ return fdb_entry;
+ }
+
+ return NULL;
+}
+
+static void lan966x_fdb_add_entry(struct lan966x *lan966x,
+ struct switchdev_notifier_fdb_info *fdb_info)
+{
+ struct lan966x_fdb_entry *fdb_entry;
+
+ fdb_entry = lan966x_fdb_find_entry(lan966x, fdb_info);
+ if (fdb_entry) {
+ fdb_entry->references++;
+ return;
+ }
+
+ fdb_entry = kzalloc(sizeof(*fdb_entry), GFP_KERNEL);
+ if (!fdb_entry)
+ return;
+
+ ether_addr_copy(fdb_entry->mac, fdb_info->addr);
+ fdb_entry->vid = fdb_info->vid;
+ fdb_entry->references = 1;
+ list_add_tail(&fdb_entry->list, &lan966x->fdb_entries);
+}
+
+static bool lan966x_fdb_del_entry(struct lan966x *lan966x,
+ struct switchdev_notifier_fdb_info *fdb_info)
+{
+ struct lan966x_fdb_entry *fdb_entry, *tmp;
+
+ list_for_each_entry_safe(fdb_entry, tmp, &lan966x->fdb_entries,
+ list) {
+ if (fdb_entry->vid == fdb_info->vid &&
+ ether_addr_equal(fdb_entry->mac, fdb_info->addr)) {
+ fdb_entry->references--;
+ if (!fdb_entry->references) {
+ list_del(&fdb_entry->list);
+ kfree(fdb_entry);
+ return true;
+ }
+ break;
+ }
+ }
+
+ return false;
+}
+
+void lan966x_fdb_write_entries(struct lan966x *lan966x, u16 vid)
+{
+ struct lan966x_fdb_entry *fdb_entry;
+
+ list_for_each_entry(fdb_entry, &lan966x->fdb_entries, list) {
+ if (fdb_entry->vid != vid)
+ continue;
+
+ lan966x_mac_cpu_learn(lan966x, fdb_entry->mac, fdb_entry->vid);
+ }
+}
+
+void lan966x_fdb_erase_entries(struct lan966x *lan966x, u16 vid)
+{
+ struct lan966x_fdb_entry *fdb_entry;
+
+ list_for_each_entry(fdb_entry, &lan966x->fdb_entries, list) {
+ if (fdb_entry->vid != vid)
+ continue;
+
+ lan966x_mac_cpu_forget(lan966x, fdb_entry->mac, fdb_entry->vid);
+ }
+}
+
+static void lan966x_fdb_purge_entries(struct lan966x *lan966x)
+{
+ struct lan966x_fdb_entry *fdb_entry, *tmp;
+
+ list_for_each_entry_safe(fdb_entry, tmp, &lan966x->fdb_entries, list) {
+ list_del(&fdb_entry->list);
+ kfree(fdb_entry);
+ }
+}
+
+int lan966x_fdb_init(struct lan966x *lan966x)
+{
+ INIT_LIST_HEAD(&lan966x->fdb_entries);
+ lan966x->fdb_work = alloc_ordered_workqueue("lan966x_order", 0);
+ if (!lan966x->fdb_work)
+ return -ENOMEM;
+
+ return 0;
+}
+
+void lan966x_fdb_deinit(struct lan966x *lan966x)
+{
+ destroy_workqueue(lan966x->fdb_work);
+ lan966x_fdb_purge_entries(lan966x);
+}
+
+static void lan966x_fdb_event_work(struct work_struct *work)
+{
+ struct lan966x_fdb_event_work *fdb_work =
+ container_of(work, struct lan966x_fdb_event_work, work);
+ struct switchdev_notifier_fdb_info *fdb_info;
+ struct net_device *dev = fdb_work->dev;
+ struct lan966x_port *port;
+ struct lan966x *lan966x;
+ int ret;
+
+ fdb_info = &fdb_work->fdb_info;
+ lan966x = fdb_work->lan966x;
+
+ if (lan966x_netdevice_check(dev)) {
+ port = netdev_priv(dev);
+
+ switch (fdb_work->event) {
+ case SWITCHDEV_FDB_ADD_TO_DEVICE:
+ if (!fdb_info->added_by_user)
+ break;
+ lan966x_mac_add_entry(lan966x, port, fdb_info->addr,
+ fdb_info->vid);
+ break;
+ case SWITCHDEV_FDB_DEL_TO_DEVICE:
+ if (!fdb_info->added_by_user)
+ break;
+ lan966x_mac_del_entry(lan966x, fdb_info->addr,
+ fdb_info->vid);
+ break;
+ }
+ } else {
+ if (!netif_is_bridge_master(dev))
+ goto out;
+
+ /* In case the bridge is called */
+ switch (fdb_work->event) {
+ case SWITCHDEV_FDB_ADD_TO_DEVICE:
+ /* If there is no front port in this vlan, there is no
+ * point to copy the frame to CPU because it would be
+ * just dropped at later point. So add it only if
+ * there is a port but it is required to store the fdb
+ * entry for later point when a port actually gets in
+ * the vlan.
+ */
+ lan966x_fdb_add_entry(lan966x, fdb_info);
+ if (!lan966x_vlan_cpu_member_cpu_vlan_mask(lan966x,
+ fdb_info->vid))
+ break;
+
+ lan966x_mac_cpu_learn(lan966x, fdb_info->addr,
+ fdb_info->vid);
+ break;
+ case SWITCHDEV_FDB_DEL_TO_DEVICE:
+ ret = lan966x_fdb_del_entry(lan966x, fdb_info);
+ if (!lan966x_vlan_cpu_member_cpu_vlan_mask(lan966x,
+ fdb_info->vid))
+ break;
+
+ if (ret)
+ lan966x_mac_cpu_forget(lan966x, fdb_info->addr,
+ fdb_info->vid);
+ break;
+ }
+ }
+
+out:
+ kfree(fdb_work->fdb_info.addr);
+ kfree(fdb_work);
+ dev_put(dev);
+}
+
+int lan966x_handle_fdb(struct net_device *dev,
+ struct net_device *orig_dev,
+ unsigned long event, const void *ctx,
+ const struct switchdev_notifier_fdb_info *fdb_info)
+{
+ struct lan966x_port *port = netdev_priv(dev);
+ struct lan966x *lan966x = port->lan966x;
+ struct lan966x_fdb_event_work *fdb_work;
+
+ if (ctx && ctx != port)
+ return 0;
+
+ switch (event) {
+ case SWITCHDEV_FDB_ADD_TO_DEVICE:
+ case SWITCHDEV_FDB_DEL_TO_DEVICE:
+ if (lan966x_netdevice_check(orig_dev) &&
+ !fdb_info->added_by_user)
+ break;
+
+ fdb_work = kzalloc(sizeof(*fdb_work), GFP_ATOMIC);
+ if (!fdb_work)
+ return -ENOMEM;
+
+ fdb_work->dev = orig_dev;
+ fdb_work->lan966x = lan966x;
+ fdb_work->event = event;
+ INIT_WORK(&fdb_work->work, lan966x_fdb_event_work);
+ memcpy(&fdb_work->fdb_info, fdb_info, sizeof(fdb_work->fdb_info));
+ fdb_work->fdb_info.addr = kzalloc(ETH_ALEN, GFP_ATOMIC);
+ if (!fdb_work->fdb_info.addr)
+ goto err_addr_alloc;
+
+ ether_addr_copy((u8 *)fdb_work->fdb_info.addr, fdb_info->addr);
+ dev_hold(orig_dev);
+
+ queue_work(lan966x->fdb_work, &fdb_work->work);
+ break;
+ }
+
+ return 0;
+err_addr_alloc:
+ kfree(fdb_work);
+ return -ENOMEM;
+}
diff --git a/drivers/net/ethernet/microchip/lan966x/lan966x_ifh.h b/drivers/net/ethernet/microchip/lan966x/lan966x_ifh.h
new file mode 100644
index 0000000..ca33147
--- /dev/null
+++ b/drivers/net/ethernet/microchip/lan966x/lan966x_ifh.h
@@ -0,0 +1,173 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+
+#ifndef __LAN966X_IFH_H__
+#define __LAN966X_IFH_H__
+
+/* Fields with description (*) should just be cleared upon injection
+ * IFH is transmitted MSByte first (Highest bit pos sent as MSB of first byte)
+ */
+
+#define IFH_LEN 7
+
+/* Timestamp for frame */
+#define IFH_POS_TIMESTAMP 192
+
+/* Bypass analyzer with a prefilled IFH */
+#define IFH_POS_BYPASS 191
+
+/* Masqueraded injection with masq_port defining logical source port */
+#define IFH_POS_MASQ 190
+
+/* Masqueraded port number for injection */
+#define IFH_POS_MASQ_PORT 186
+
+/* Frame length (*) */
+#define IFH_POS_LEN 178
+
+/* Cell filling mode. Full(0),Etype(1), LlctOpt(2), Llct(3) */
+#define IFH_POS_WRDMODE 176
+
+/* Frame has 16 bits rtag removed compared to line data */
+#define IFH_POS_RTAG48 175
+
+/* Frame has a redundancy tag */
+#define IFH_POS_HAS_RED_TAG 174
+
+/* Frame has been cut through forwarded (*) */
+#define IFH_POS_CUTTHRU 173
+
+/* Rewriter command */
+#define IFH_POS_REW_CMD 163
+
+/* Enable OAM-related rewriting. PDU_TYPE encodes OAM type. */
+#define IFH_POS_REW_OAM 162
+
+/* PDU type. Encoding: (0-NONE, 1-Y1731_CCM, 2-MRP_TST, 3-MRP_ITST, 4-DLR_BCN,
+ * 5-DLR_ADV, 6-RTE_NULL_INJ, 7-IPV4, 8-IPV6, 9-Y1731_NON_CCM).
+ */
+#define IFH_POS_PDU_TYPE 158
+
+/* Update FCS before transmission */
+#define IFH_POS_FCS_UPD 157
+
+/* Classified DSCP value of frame */
+#define IFH_POS_DSCP 151
+
+/* Yellow indication */
+#define IFH_POS_DP 150
+
+/* Process in RTE/inbound */
+#define IFH_POS_RTE_INB_UPDATE 149
+
+/* Number of tags to pop from frame */
+#define IFH_POS_POP_CNT 147
+
+/* Number of tags in front of the ethertype */
+#define IFH_POS_ETYPE_OFS 145
+
+/* Logical source port of frame (*) */
+#define IFH_POS_SRCPORT 141
+
+/* Sequence number in redundancy tag */
+#define IFH_POS_SEQ_NUM 120
+
+/* Stagd flag and classified TCI of frame (PCP/DEI/VID) */
+#define IFH_POS_TCI 103
+
+/* Classified internal priority for queuing */
+#define IFH_POS_QOS_CLASS 100
+
+/* Bit mask with eight cpu copy classses */
+#define IFH_POS_CPUQ 92
+
+/* Relearn + learn flags (*) */
+#define IFH_POS_LEARN_FLAGS 90
+
+/* SFLOW identifier for frame (0-8: Tx port, 9: Rx sampling, 15: No sampling) */
+#define IFH_POS_SFLOW_ID 86
+
+/* Set if an ACL/S2 rule was hit (*).
+ * Super priority: acl_hit=0 and acl_hit(4)=1.
+ */
+#define IFH_POS_ACL_HIT 85
+
+/* S2 rule index hit (*) */
+#define IFH_POS_ACL_IDX 79
+
+/* ISDX as classified by S1 */
+#define IFH_POS_ISDX 71
+
+/* Destination ports for frame */
+#define IFH_POS_DSTS 62
+
+/* Storm policer to be applied: None/Uni/Multi/Broad (*) */
+#define IFH_POS_FLOOD 60
+
+/* Redundancy tag operation */
+#define IFH_POS_SEQ_OP 58
+
+/* Classified internal priority for resourcemgt, tagging etc */
+#define IFH_POS_IPV 55
+
+/* Frame is for AFI use */
+#define IFH_POS_AFI 54
+
+/* Internal aging value (*) */
+#define IFH_POS_AGED 52
+
+/* RTP Identifier */
+#define IFH_POS_RTP_ID 42
+
+/* RTP MRPD flow */
+#define IFH_POS_RTP_SUBID 41
+
+/* Profinet DataStatus or opcua GroupVersion MSB */
+#define IFH_POS_PN_DATA_STATUS 33
+
+/* Profinet transfer status (1 iff the status is 0) */
+#define IFH_POS_PN_TRANSF_STATUS_ZERO 32
+
+/* Profinet cycle counter or opcua NetworkMessageNumber */
+#define IFH_POS_PN_CC 16
+
+#define IFH_WID_TIMESTAMP 32
+#define IFH_WID_BYPASS 1
+#define IFH_WID_MASQ 1
+#define IFH_WID_MASQ_PORT 4
+#define IFH_WID_LEN 14
+#define IFH_WID_WRDMODE 2
+#define IFH_WID_RTAG48 1
+#define IFH_WID_HAS_RED_TAG 1
+#define IFH_WID_CUTTHRU 1
+#define IFH_WID_REW_CMD 10
+#define IFH_WID_REW_OAM 1
+#define IFH_WID_PDU_TYPE 4
+#define IFH_WID_FCS_UPD 1
+#define IFH_WID_DSCP 6
+#define IFH_WID_DP 1
+#define IFH_WID_RTE_INB_UPDATE 1
+#define IFH_WID_POP_CNT 2
+#define IFH_WID_ETYPE_OFS 2
+#define IFH_WID_SRCPORT 4
+#define IFH_WID_SEQ_NUM 16
+#define IFH_WID_TCI 17
+#define IFH_WID_QOS_CLASS 3
+#define IFH_WID_CPUQ 8
+#define IFH_WID_LEARN_FLAGS 2
+#define IFH_WID_SFLOW_ID 4
+#define IFH_WID_ACL_HIT 1
+#define IFH_WID_ACL_IDX 6
+#define IFH_WID_ISDX 8
+#define IFH_WID_DSTS 9
+#define IFH_WID_FLOOD 2
+#define IFH_WID_SEQ_OP 2
+#define IFH_WID_IPV 3
+#define IFH_WID_AFI 1
+#define IFH_WID_AGED 2
+#define IFH_WID_RTP_ID 10
+#define IFH_WID_RTP_SUBID 1
+#define IFH_WID_PN_DATA_STATUS 8
+#define IFH_WID_PN_TRANSF_STATUS_ZERO 1
+#define IFH_WID_PN_CC 16
+
+#endif /* __LAN966X_IFH_H__ */
diff --git a/drivers/net/ethernet/microchip/lan966x/lan966x_mac.c b/drivers/net/ethernet/microchip/lan966x/lan966x_mac.c
new file mode 100644
index 0000000..ca5f117
--- /dev/null
+++ b/drivers/net/ethernet/microchip/lan966x/lan966x_mac.c
@@ -0,0 +1,469 @@
+// SPDX-License-Identifier: GPL-2.0+
+
+#include <net/switchdev.h>
+#include "lan966x_main.h"
+
+#define LAN966X_MAC_COLUMNS 4
+#define MACACCESS_CMD_IDLE 0
+#define MACACCESS_CMD_LEARN 1
+#define MACACCESS_CMD_FORGET 2
+#define MACACCESS_CMD_AGE 3
+#define MACACCESS_CMD_GET_NEXT 4
+#define MACACCESS_CMD_INIT 5
+#define MACACCESS_CMD_READ 6
+#define MACACCESS_CMD_WRITE 7
+#define MACACCESS_CMD_SYNC_GET_NEXT 8
+
+#define LAN966X_MAC_INVALID_ROW -1
+
+struct lan966x_mac_entry {
+ struct list_head list;
+ unsigned char mac[ETH_ALEN] __aligned(2);
+ u16 vid;
+ u16 port_index;
+ int row;
+};
+
+struct lan966x_mac_raw_entry {
+ u32 mach;
+ u32 macl;
+ u32 maca;
+ bool processed;
+};
+
+static int lan966x_mac_get_status(struct lan966x *lan966x)
+{
+ return lan_rd(lan966x, ANA_MACACCESS);
+}
+
+static int lan966x_mac_wait_for_completion(struct lan966x *lan966x)
+{
+ u32 val;
+
+ return readx_poll_timeout(lan966x_mac_get_status,
+ lan966x, val,
+ (ANA_MACACCESS_MAC_TABLE_CMD_GET(val)) ==
+ MACACCESS_CMD_IDLE,
+ TABLE_UPDATE_SLEEP_US, TABLE_UPDATE_TIMEOUT_US);
+}
+
+static void lan966x_mac_select(struct lan966x *lan966x,
+ const unsigned char mac[ETH_ALEN],
+ unsigned int vid)
+{
+ u32 macl = 0, mach = 0;
+
+ /* Set the MAC address to handle and the vlan associated in a format
+ * understood by the hardware.
+ */
+ mach |= vid << 16;
+ mach |= mac[0] << 8;
+ mach |= mac[1] << 0;
+ macl |= mac[2] << 24;
+ macl |= mac[3] << 16;
+ macl |= mac[4] << 8;
+ macl |= mac[5] << 0;
+
+ lan_wr(macl, lan966x, ANA_MACLDATA);
+ lan_wr(mach, lan966x, ANA_MACHDATA);
+}
+
+static int __lan966x_mac_learn(struct lan966x *lan966x, int pgid,
+ bool cpu_copy,
+ const unsigned char mac[ETH_ALEN],
+ unsigned int vid,
+ enum macaccess_entry_type type)
+{
+ lan966x_mac_select(lan966x, mac, vid);
+
+ /* Issue a write command */
+ lan_wr(ANA_MACACCESS_VALID_SET(1) |
+ ANA_MACACCESS_CHANGE2SW_SET(0) |
+ ANA_MACACCESS_MAC_CPU_COPY_SET(cpu_copy) |
+ ANA_MACACCESS_DEST_IDX_SET(pgid) |
+ ANA_MACACCESS_ENTRYTYPE_SET(type) |
+ ANA_MACACCESS_MAC_TABLE_CMD_SET(MACACCESS_CMD_LEARN),
+ lan966x, ANA_MACACCESS);
+
+ return lan966x_mac_wait_for_completion(lan966x);
+}
+
+/* The mask of the front ports is encoded inside the mac parameter via a call
+ * to lan966x_mdb_encode_mac().
+ */
+int lan966x_mac_ip_learn(struct lan966x *lan966x,
+ bool cpu_copy,
+ const unsigned char mac[ETH_ALEN],
+ unsigned int vid,
+ enum macaccess_entry_type type)
+{
+ WARN_ON(type != ENTRYTYPE_MACV4 && type != ENTRYTYPE_MACV6);
+
+ return __lan966x_mac_learn(lan966x, 0, cpu_copy, mac, vid, type);
+}
+
+int lan966x_mac_learn(struct lan966x *lan966x, int port,
+ const unsigned char mac[ETH_ALEN],
+ unsigned int vid,
+ enum macaccess_entry_type type)
+{
+ WARN_ON(type != ENTRYTYPE_NORMAL && type != ENTRYTYPE_LOCKED);
+
+ return __lan966x_mac_learn(lan966x, port, false, mac, vid, type);
+}
+
+int lan966x_mac_forget(struct lan966x *lan966x,
+ const unsigned char mac[ETH_ALEN],
+ unsigned int vid,
+ enum macaccess_entry_type type)
+{
+ lan966x_mac_select(lan966x, mac, vid);
+
+ /* Issue a forget command */
+ lan_wr(ANA_MACACCESS_ENTRYTYPE_SET(type) |
+ ANA_MACACCESS_MAC_TABLE_CMD_SET(MACACCESS_CMD_FORGET),
+ lan966x, ANA_MACACCESS);
+
+ return lan966x_mac_wait_for_completion(lan966x);
+}
+
+int lan966x_mac_cpu_learn(struct lan966x *lan966x, const char *addr, u16 vid)
+{
+ return lan966x_mac_learn(lan966x, PGID_CPU, addr, vid, ENTRYTYPE_LOCKED);
+}
+
+int lan966x_mac_cpu_forget(struct lan966x *lan966x, const char *addr, u16 vid)
+{
+ return lan966x_mac_forget(lan966x, addr, vid, ENTRYTYPE_LOCKED);
+}
+
+void lan966x_mac_set_ageing(struct lan966x *lan966x,
+ u32 ageing)
+{
+ lan_rmw(ANA_AUTOAGE_AGE_PERIOD_SET(ageing / 2),
+ ANA_AUTOAGE_AGE_PERIOD,
+ lan966x, ANA_AUTOAGE);
+}
+
+void lan966x_mac_init(struct lan966x *lan966x)
+{
+ /* Clear the MAC table */
+ lan_wr(MACACCESS_CMD_INIT, lan966x, ANA_MACACCESS);
+ lan966x_mac_wait_for_completion(lan966x);
+
+ spin_lock_init(&lan966x->mac_lock);
+ INIT_LIST_HEAD(&lan966x->mac_entries);
+}
+
+static struct lan966x_mac_entry *lan966x_mac_alloc_entry(const unsigned char *mac,
+ u16 vid, u16 port_index)
+{
+ struct lan966x_mac_entry *mac_entry;
+
+ mac_entry = kzalloc(sizeof(*mac_entry), GFP_KERNEL);
+ if (!mac_entry)
+ return NULL;
+
+ memcpy(mac_entry->mac, mac, ETH_ALEN);
+ mac_entry->vid = vid;
+ mac_entry->port_index = port_index;
+ mac_entry->row = LAN966X_MAC_INVALID_ROW;
+ return mac_entry;
+}
+
+static struct lan966x_mac_entry *lan966x_mac_find_entry(struct lan966x *lan966x,
+ const unsigned char *mac,
+ u16 vid, u16 port_index)
+{
+ struct lan966x_mac_entry *res = NULL;
+ struct lan966x_mac_entry *mac_entry;
+
+ spin_lock(&lan966x->mac_lock);
+ list_for_each_entry(mac_entry, &lan966x->mac_entries, list) {
+ if (mac_entry->vid == vid &&
+ ether_addr_equal(mac, mac_entry->mac) &&
+ mac_entry->port_index == port_index) {
+ res = mac_entry;
+ break;
+ }
+ }
+ spin_unlock(&lan966x->mac_lock);
+
+ return res;
+}
+
+static int lan966x_mac_lookup(struct lan966x *lan966x,
+ const unsigned char mac[ETH_ALEN],
+ unsigned int vid, enum macaccess_entry_type type)
+{
+ int ret;
+
+ lan966x_mac_select(lan966x, mac, vid);
+
+ /* Issue a read command */
+ lan_wr(ANA_MACACCESS_ENTRYTYPE_SET(type) |
+ ANA_MACACCESS_VALID_SET(1) |
+ ANA_MACACCESS_MAC_TABLE_CMD_SET(MACACCESS_CMD_READ),
+ lan966x, ANA_MACACCESS);
+
+ ret = lan966x_mac_wait_for_completion(lan966x);
+ if (ret)
+ return ret;
+
+ return ANA_MACACCESS_VALID_GET(lan_rd(lan966x, ANA_MACACCESS));
+}
+
+static void lan966x_fdb_call_notifiers(enum switchdev_notifier_type type,
+ const char *mac, u16 vid,
+ struct net_device *dev)
+{
+ struct switchdev_notifier_fdb_info info = { 0 };
+
+ info.addr = mac;
+ info.vid = vid;
+ info.offloaded = true;
+ call_switchdev_notifiers(type, dev, &info.info, NULL);
+}
+
+int lan966x_mac_add_entry(struct lan966x *lan966x, struct lan966x_port *port,
+ const unsigned char *addr, u16 vid)
+{
+ struct lan966x_mac_entry *mac_entry;
+
+ if (lan966x_mac_lookup(lan966x, addr, vid, ENTRYTYPE_NORMAL))
+ return 0;
+
+ /* In case the entry already exists, don't add it again to SW,
+ * just update HW, but we need to look in the actual HW because
+ * it is possible for an entry to be learn by HW and before we
+ * get the interrupt the frame will reach CPU and the CPU will
+ * add the entry but without the extern_learn flag.
+ */
+ mac_entry = lan966x_mac_find_entry(lan966x, addr, vid, port->chip_port);
+ if (mac_entry)
+ return lan966x_mac_learn(lan966x, port->chip_port,
+ addr, vid, ENTRYTYPE_LOCKED);
+
+ mac_entry = lan966x_mac_alloc_entry(addr, vid, port->chip_port);
+ if (!mac_entry)
+ return -ENOMEM;
+
+ spin_lock(&lan966x->mac_lock);
+ list_add_tail(&mac_entry->list, &lan966x->mac_entries);
+ spin_unlock(&lan966x->mac_lock);
+
+ lan966x_mac_learn(lan966x, port->chip_port, addr, vid, ENTRYTYPE_LOCKED);
+ lan966x_fdb_call_notifiers(SWITCHDEV_FDB_OFFLOADED, addr, vid, port->dev);
+
+ return 0;
+}
+
+int lan966x_mac_del_entry(struct lan966x *lan966x, const unsigned char *addr,
+ u16 vid)
+{
+ struct lan966x_mac_entry *mac_entry, *tmp;
+
+ spin_lock(&lan966x->mac_lock);
+ list_for_each_entry_safe(mac_entry, tmp, &lan966x->mac_entries,
+ list) {
+ if (mac_entry->vid == vid &&
+ ether_addr_equal(addr, mac_entry->mac)) {
+ lan966x_mac_forget(lan966x, mac_entry->mac, mac_entry->vid,
+ ENTRYTYPE_LOCKED);
+
+ list_del(&mac_entry->list);
+ kfree(mac_entry);
+ }
+ }
+ spin_unlock(&lan966x->mac_lock);
+
+ return 0;
+}
+
+void lan966x_mac_purge_entries(struct lan966x *lan966x)
+{
+ struct lan966x_mac_entry *mac_entry, *tmp;
+
+ spin_lock(&lan966x->mac_lock);
+ list_for_each_entry_safe(mac_entry, tmp, &lan966x->mac_entries,
+ list) {
+ lan966x_mac_forget(lan966x, mac_entry->mac, mac_entry->vid,
+ ENTRYTYPE_LOCKED);
+
+ list_del(&mac_entry->list);
+ kfree(mac_entry);
+ }
+ spin_unlock(&lan966x->mac_lock);
+}
+
+static void lan966x_mac_notifiers(enum switchdev_notifier_type type,
+ unsigned char *mac, u32 vid,
+ struct net_device *dev)
+{
+ rtnl_lock();
+ lan966x_fdb_call_notifiers(type, mac, vid, dev);
+ rtnl_unlock();
+}
+
+static void lan966x_mac_process_raw_entry(struct lan966x_mac_raw_entry *raw_entry,
+ u8 *mac, u16 *vid, u32 *dest_idx)
+{
+ mac[0] = (raw_entry->mach >> 8) & 0xff;
+ mac[1] = (raw_entry->mach >> 0) & 0xff;
+ mac[2] = (raw_entry->macl >> 24) & 0xff;
+ mac[3] = (raw_entry->macl >> 16) & 0xff;
+ mac[4] = (raw_entry->macl >> 8) & 0xff;
+ mac[5] = (raw_entry->macl >> 0) & 0xff;
+
+ *vid = (raw_entry->mach >> 16) & 0xfff;
+ *dest_idx = ANA_MACACCESS_DEST_IDX_GET(raw_entry->maca);
+}
+
+static void lan966x_mac_irq_process(struct lan966x *lan966x, u32 row,
+ struct lan966x_mac_raw_entry *raw_entries)
+{
+ struct lan966x_mac_entry *mac_entry, *tmp;
+ unsigned char mac[ETH_ALEN] __aligned(2);
+ u32 dest_idx;
+ u32 column;
+ u16 vid;
+
+ spin_lock(&lan966x->mac_lock);
+ list_for_each_entry_safe(mac_entry, tmp, &lan966x->mac_entries, list) {
+ bool found = false;
+
+ if (mac_entry->row != row)
+ continue;
+
+ for (column = 0; column < LAN966X_MAC_COLUMNS; ++column) {
+ /* All the valid entries are at the start of the row,
+ * so when get one invalid entry it can just skip the
+ * rest of the columns
+ */
+ if (!ANA_MACACCESS_VALID_GET(raw_entries[column].maca))
+ break;
+
+ lan966x_mac_process_raw_entry(&raw_entries[column],
+ mac, &vid, &dest_idx);
+ WARN_ON(dest_idx > lan966x->num_phys_ports);
+
+ /* If the entry in SW is found, then there is nothing
+ * to do
+ */
+ if (mac_entry->vid == vid &&
+ ether_addr_equal(mac_entry->mac, mac) &&
+ mac_entry->port_index == dest_idx) {
+ raw_entries[column].processed = true;
+ found = true;
+ break;
+ }
+ }
+
+ if (!found) {
+ /* Notify the bridge that the entry doesn't exist
+ * anymore in the HW and remove the entry from the SW
+ * list
+ */
+ lan966x_mac_notifiers(SWITCHDEV_FDB_DEL_TO_BRIDGE,
+ mac_entry->mac, mac_entry->vid,
+ lan966x->ports[mac_entry->port_index]->dev);
+
+ list_del(&mac_entry->list);
+ kfree(mac_entry);
+ }
+ }
+ spin_unlock(&lan966x->mac_lock);
+
+ /* Now go to the list of columns and see if any entry was not in the SW
+ * list, then that means that the entry is new so it needs to notify the
+ * bridge.
+ */
+ for (column = 0; column < LAN966X_MAC_COLUMNS; ++column) {
+ /* All the valid entries are at the start of the row, so when
+ * get one invalid entry it can just skip the rest of the columns
+ */
+ if (!ANA_MACACCESS_VALID_GET(raw_entries[column].maca))
+ break;
+
+ /* If the entry already exists then don't do anything */
+ if (raw_entries[column].processed)
+ continue;
+
+ lan966x_mac_process_raw_entry(&raw_entries[column],
+ mac, &vid, &dest_idx);
+ WARN_ON(dest_idx > lan966x->num_phys_ports);
+
+ mac_entry = lan966x_mac_alloc_entry(mac, vid, dest_idx);
+ if (!mac_entry)
+ return;
+
+ mac_entry->row = row;
+
+ spin_lock(&lan966x->mac_lock);
+ list_add_tail(&mac_entry->list, &lan966x->mac_entries);
+ spin_unlock(&lan966x->mac_lock);
+
+ lan966x_mac_notifiers(SWITCHDEV_FDB_ADD_TO_BRIDGE,
+ mac, vid, lan966x->ports[dest_idx]->dev);
+ }
+}
+
+irqreturn_t lan966x_mac_irq_handler(struct lan966x *lan966x)
+{
+ struct lan966x_mac_raw_entry entry[LAN966X_MAC_COLUMNS] = { 0 };
+ u32 index, column;
+ bool stop = true;
+ u32 val;
+
+ /* Start the scan from 0, 0 */
+ lan_wr(ANA_MACTINDX_M_INDEX_SET(0) |
+ ANA_MACTINDX_BUCKET_SET(0),
+ lan966x, ANA_MACTINDX);
+
+ while (1) {
+ lan_rmw(ANA_MACACCESS_MAC_TABLE_CMD_SET(MACACCESS_CMD_SYNC_GET_NEXT),
+ ANA_MACACCESS_MAC_TABLE_CMD,
+ lan966x, ANA_MACACCESS);
+ lan966x_mac_wait_for_completion(lan966x);
+
+ val = lan_rd(lan966x, ANA_MACTINDX);
+ index = ANA_MACTINDX_M_INDEX_GET(val);
+ column = ANA_MACTINDX_BUCKET_GET(val);
+
+ /* The SYNC-GET-NEXT returns all the entries(4) in a row in
+ * which is suffered a change. By change it means that new entry
+ * was added or an entry was removed because of ageing.
+ * It would return all the columns for that row. And after that
+ * it would return the next row The stop conditions of the
+ * SYNC-GET-NEXT is when it reaches 'directly' to row 0
+ * column 3. So if SYNC-GET-NEXT returns row 0 and column 0
+ * then it is required to continue to read more even if it
+ * reaches row 0 and column 3.
+ */
+ if (index == 0 && column == 0)
+ stop = false;
+
+ if (column == LAN966X_MAC_COLUMNS - 1 &&
+ index == 0 && stop)
+ break;
+
+ entry[column].mach = lan_rd(lan966x, ANA_MACHDATA);
+ entry[column].macl = lan_rd(lan966x, ANA_MACLDATA);
+ entry[column].maca = lan_rd(lan966x, ANA_MACACCESS);
+
+ /* Once all the columns are read process them */
+ if (column == LAN966X_MAC_COLUMNS - 1) {
+ lan966x_mac_irq_process(lan966x, index, entry);
+ /* A row was processed so it is safe to assume that the
+ * next row/column can be the stop condition
+ */
+ stop = true;
+ }
+ }
+
+ lan_rmw(ANA_ANAINTR_INTR_SET(0),
+ ANA_ANAINTR_INTR,
+ lan966x, ANA_ANAINTR);
+
+ return IRQ_HANDLED;
+}
diff --git a/drivers/net/ethernet/microchip/lan966x/lan966x_main.c b/drivers/net/ethernet/microchip/lan966x/lan966x_main.c
new file mode 100644
index 0000000..2cb70da
--- /dev/null
+++ b/drivers/net/ethernet/microchip/lan966x/lan966x_main.c
@@ -0,0 +1,1002 @@
+// SPDX-License-Identifier: GPL-2.0+
+
+#include <linux/module.h>
+#include <linux/if_bridge.h>
+#include <linux/if_vlan.h>
+#include <linux/iopoll.h>
+#include <linux/of_platform.h>
+#include <linux/of_net.h>
+#include <linux/packing.h>
+#include <linux/phy/phy.h>
+#include <linux/reset.h>
+
+#include "lan966x_main.h"
+
+#define XTR_EOF_0 0x00000080U
+#define XTR_EOF_1 0x01000080U
+#define XTR_EOF_2 0x02000080U
+#define XTR_EOF_3 0x03000080U
+#define XTR_PRUNED 0x04000080U
+#define XTR_ABORT 0x05000080U
+#define XTR_ESCAPE 0x06000080U
+#define XTR_NOT_READY 0x07000080U
+#define XTR_VALID_BYTES(x) (4 - (((x) >> 24) & 3))
+
+#define READL_SLEEP_US 10
+#define READL_TIMEOUT_US 100000000
+
+#define IO_RANGES 2
+
+static const struct of_device_id lan966x_match[] = {
+ { .compatible = "microchip,lan966x-switch" },
+ { }
+};
+MODULE_DEVICE_TABLE(of, lan966x_match);
+
+struct lan966x_main_io_resource {
+ enum lan966x_target id;
+ phys_addr_t offset;
+ int range;
+};
+
+static const struct lan966x_main_io_resource lan966x_main_iomap[] = {
+ { TARGET_CPU, 0xc0000, 0 }, /* 0xe00c0000 */
+ { TARGET_ORG, 0, 1 }, /* 0xe2000000 */
+ { TARGET_GCB, 0x4000, 1 }, /* 0xe2004000 */
+ { TARGET_QS, 0x8000, 1 }, /* 0xe2008000 */
+ { TARGET_CHIP_TOP, 0x10000, 1 }, /* 0xe2010000 */
+ { TARGET_REW, 0x14000, 1 }, /* 0xe2014000 */
+ { TARGET_SYS, 0x28000, 1 }, /* 0xe2028000 */
+ { TARGET_DEV, 0x34000, 1 }, /* 0xe2034000 */
+ { TARGET_DEV + 1, 0x38000, 1 }, /* 0xe2038000 */
+ { TARGET_DEV + 2, 0x3c000, 1 }, /* 0xe203c000 */
+ { TARGET_DEV + 3, 0x40000, 1 }, /* 0xe2040000 */
+ { TARGET_DEV + 4, 0x44000, 1 }, /* 0xe2044000 */
+ { TARGET_DEV + 5, 0x48000, 1 }, /* 0xe2048000 */
+ { TARGET_DEV + 6, 0x4c000, 1 }, /* 0xe204c000 */
+ { TARGET_DEV + 7, 0x50000, 1 }, /* 0xe2050000 */
+ { TARGET_QSYS, 0x100000, 1 }, /* 0xe2100000 */
+ { TARGET_AFI, 0x120000, 1 }, /* 0xe2120000 */
+ { TARGET_ANA, 0x140000, 1 }, /* 0xe2140000 */
+};
+
+static int lan966x_create_targets(struct platform_device *pdev,
+ struct lan966x *lan966x)
+{
+ struct resource *iores[IO_RANGES];
+ void __iomem *begin[IO_RANGES];
+ int idx;
+
+ /* Initially map the entire range and after that update each target to
+ * point inside the region at the correct offset. It is possible that
+ * other devices access the same region so don't add any checks about
+ * this.
+ */
+ for (idx = 0; idx < IO_RANGES; idx++) {
+ iores[idx] = platform_get_resource(pdev, IORESOURCE_MEM,
+ idx);
+ if (!iores[idx]) {
+ dev_err(&pdev->dev, "Invalid resource\n");
+ return -EINVAL;
+ }
+
+ begin[idx] = devm_ioremap(&pdev->dev,
+ iores[idx]->start,
+ resource_size(iores[idx]));
+ if (!begin[idx]) {
+ dev_err(&pdev->dev, "Unable to get registers: %s\n",
+ iores[idx]->name);
+ return -ENOMEM;
+ }
+ }
+
+ for (idx = 0; idx < ARRAY_SIZE(lan966x_main_iomap); idx++) {
+ const struct lan966x_main_io_resource *iomap =
+ &lan966x_main_iomap[idx];
+
+ lan966x->regs[iomap->id] = begin[iomap->range] + iomap->offset;
+ }
+
+ return 0;
+}
+
+static int lan966x_port_set_mac_address(struct net_device *dev, void *p)
+{
+ struct lan966x_port *port = netdev_priv(dev);
+ struct lan966x *lan966x = port->lan966x;
+ const struct sockaddr *addr = p;
+ int ret;
+
+ /* Learn the new net device MAC address in the mac table. */
+ ret = lan966x_mac_cpu_learn(lan966x, addr->sa_data, HOST_PVID);
+ if (ret)
+ return ret;
+
+ /* Then forget the previous one. */
+ ret = lan966x_mac_cpu_forget(lan966x, dev->dev_addr, HOST_PVID);
+ if (ret)
+ return ret;
+
+ eth_hw_addr_set(dev, addr->sa_data);
+ return ret;
+}
+
+static int lan966x_port_get_phys_port_name(struct net_device *dev,
+ char *buf, size_t len)
+{
+ struct lan966x_port *port = netdev_priv(dev);
+ int ret;
+
+ ret = snprintf(buf, len, "p%d", port->chip_port);
+ if (ret >= len)
+ return -EINVAL;
+
+ return 0;
+}
+
+static int lan966x_port_open(struct net_device *dev)
+{
+ struct lan966x_port *port = netdev_priv(dev);
+ struct lan966x *lan966x = port->lan966x;
+ int err;
+
+ /* Enable receiving frames on the port, and activate auto-learning of
+ * MAC addresses.
+ */
+ lan_rmw(ANA_PORT_CFG_LEARNAUTO_SET(1) |
+ ANA_PORT_CFG_RECV_ENA_SET(1) |
+ ANA_PORT_CFG_PORTID_VAL_SET(port->chip_port),
+ ANA_PORT_CFG_LEARNAUTO |
+ ANA_PORT_CFG_RECV_ENA |
+ ANA_PORT_CFG_PORTID_VAL,
+ lan966x, ANA_PORT_CFG(port->chip_port));
+
+ err = phylink_fwnode_phy_connect(port->phylink, port->fwnode, 0);
+ if (err) {
+ netdev_err(dev, "Could not attach to PHY\n");
+ return err;
+ }
+
+ phylink_start(port->phylink);
+
+ return 0;
+}
+
+static int lan966x_port_stop(struct net_device *dev)
+{
+ struct lan966x_port *port = netdev_priv(dev);
+
+ lan966x_port_config_down(port);
+ phylink_stop(port->phylink);
+ phylink_disconnect_phy(port->phylink);
+
+ return 0;
+}
+
+static int lan966x_port_inj_status(struct lan966x *lan966x)
+{
+ return lan_rd(lan966x, QS_INJ_STATUS);
+}
+
+static int lan966x_port_inj_ready(struct lan966x *lan966x, u8 grp)
+{
+ u32 val;
+
+ return readx_poll_timeout(lan966x_port_inj_status, lan966x, val,
+ QS_INJ_STATUS_FIFO_RDY_GET(val) & BIT(grp),
+ READL_SLEEP_US, READL_TIMEOUT_US);
+}
+
+static int lan966x_port_ifh_xmit(struct sk_buff *skb,
+ __be32 *ifh,
+ struct net_device *dev)
+{
+ struct lan966x_port *port = netdev_priv(dev);
+ struct lan966x *lan966x = port->lan966x;
+ u32 i, count, last;
+ u8 grp = 0;
+ u32 val;
+ int err;
+
+ val = lan_rd(lan966x, QS_INJ_STATUS);
+ if (!(QS_INJ_STATUS_FIFO_RDY_GET(val) & BIT(grp)) ||
+ (QS_INJ_STATUS_WMARK_REACHED_GET(val) & BIT(grp)))
+ return NETDEV_TX_BUSY;
+
+ /* Write start of frame */
+ lan_wr(QS_INJ_CTRL_GAP_SIZE_SET(1) |
+ QS_INJ_CTRL_SOF_SET(1),
+ lan966x, QS_INJ_CTRL(grp));
+
+ /* Write IFH header */
+ for (i = 0; i < IFH_LEN; ++i) {
+ /* Wait until the fifo is ready */
+ err = lan966x_port_inj_ready(lan966x, grp);
+ if (err)
+ return NETDEV_TX_BUSY;
+
+ lan_wr((__force u32)ifh[i], lan966x, QS_INJ_WR(grp));
+ }
+
+ /* Write frame */
+ count = DIV_ROUND_UP(skb->len, 4);
+ last = skb->len % 4;
+ for (i = 0; i < count; ++i) {
+ /* Wait until the fifo is ready */
+ err = lan966x_port_inj_ready(lan966x, grp);
+ if (err)
+ return NETDEV_TX_BUSY;
+
+ lan_wr(((u32 *)skb->data)[i], lan966x, QS_INJ_WR(grp));
+ }
+
+ /* Add padding */
+ while (i < (LAN966X_BUFFER_MIN_SZ / 4)) {
+ /* Wait until the fifo is ready */
+ err = lan966x_port_inj_ready(lan966x, grp);
+ if (err)
+ return NETDEV_TX_BUSY;
+
+ lan_wr(0, lan966x, QS_INJ_WR(grp));
+ ++i;
+ }
+
+ /* Inidcate EOF and valid bytes in the last word */
+ lan_wr(QS_INJ_CTRL_GAP_SIZE_SET(1) |
+ QS_INJ_CTRL_VLD_BYTES_SET(skb->len < LAN966X_BUFFER_MIN_SZ ?
+ 0 : last) |
+ QS_INJ_CTRL_EOF_SET(1),
+ lan966x, QS_INJ_CTRL(grp));
+
+ /* Add dummy CRC */
+ lan_wr(0, lan966x, QS_INJ_WR(grp));
+ skb_tx_timestamp(skb);
+
+ dev->stats.tx_packets++;
+ dev->stats.tx_bytes += skb->len;
+
+ dev_consume_skb_any(skb);
+ return NETDEV_TX_OK;
+}
+
+static void lan966x_ifh_set_bypass(void *ifh, u64 bypass)
+{
+ packing(ifh, &bypass, IFH_POS_BYPASS + IFH_WID_BYPASS - 1,
+ IFH_POS_BYPASS, IFH_LEN * 4, PACK, 0);
+}
+
+static void lan966x_ifh_set_port(void *ifh, u64 bypass)
+{
+ packing(ifh, &bypass, IFH_POS_DSTS + IFH_WID_DSTS - 1,
+ IFH_POS_DSTS, IFH_LEN * 4, PACK, 0);
+}
+
+static void lan966x_ifh_set_qos_class(void *ifh, u64 bypass)
+{
+ packing(ifh, &bypass, IFH_POS_QOS_CLASS + IFH_WID_QOS_CLASS - 1,
+ IFH_POS_QOS_CLASS, IFH_LEN * 4, PACK, 0);
+}
+
+static void lan966x_ifh_set_ipv(void *ifh, u64 bypass)
+{
+ packing(ifh, &bypass, IFH_POS_IPV + IFH_WID_IPV - 1,
+ IFH_POS_IPV, IFH_LEN * 4, PACK, 0);
+}
+
+static void lan966x_ifh_set_vid(void *ifh, u64 vid)
+{
+ packing(ifh, &vid, IFH_POS_TCI + IFH_WID_TCI - 1,
+ IFH_POS_TCI, IFH_LEN * 4, PACK, 0);
+}
+
+static int lan966x_port_xmit(struct sk_buff *skb, struct net_device *dev)
+{
+ struct lan966x_port *port = netdev_priv(dev);
+ __be32 ifh[IFH_LEN];
+
+ memset(ifh, 0x0, sizeof(__be32) * IFH_LEN);
+
+ lan966x_ifh_set_bypass(ifh, 1);
+ lan966x_ifh_set_port(ifh, BIT_ULL(port->chip_port));
+ lan966x_ifh_set_qos_class(ifh, skb->priority >= 7 ? 0x7 : skb->priority);
+ lan966x_ifh_set_ipv(ifh, skb->priority >= 7 ? 0x7 : skb->priority);
+ lan966x_ifh_set_vid(ifh, skb_vlan_tag_get(skb));
+
+ return lan966x_port_ifh_xmit(skb, ifh, dev);
+}
+
+static int lan966x_port_change_mtu(struct net_device *dev, int new_mtu)
+{
+ struct lan966x_port *port = netdev_priv(dev);
+ struct lan966x *lan966x = port->lan966x;
+
+ lan_wr(DEV_MAC_MAXLEN_CFG_MAX_LEN_SET(new_mtu),
+ lan966x, DEV_MAC_MAXLEN_CFG(port->chip_port));
+ dev->mtu = new_mtu;
+
+ return 0;
+}
+
+static int lan966x_mc_unsync(struct net_device *dev, const unsigned char *addr)
+{
+ struct lan966x_port *port = netdev_priv(dev);
+ struct lan966x *lan966x = port->lan966x;
+
+ return lan966x_mac_forget(lan966x, addr, HOST_PVID, ENTRYTYPE_LOCKED);
+}
+
+static int lan966x_mc_sync(struct net_device *dev, const unsigned char *addr)
+{
+ struct lan966x_port *port = netdev_priv(dev);
+ struct lan966x *lan966x = port->lan966x;
+
+ return lan966x_mac_cpu_learn(lan966x, addr, HOST_PVID);
+}
+
+static void lan966x_port_set_rx_mode(struct net_device *dev)
+{
+ __dev_mc_sync(dev, lan966x_mc_sync, lan966x_mc_unsync);
+}
+
+static int lan966x_port_get_parent_id(struct net_device *dev,
+ struct netdev_phys_item_id *ppid)
+{
+ struct lan966x_port *port = netdev_priv(dev);
+ struct lan966x *lan966x = port->lan966x;
+
+ ppid->id_len = sizeof(lan966x->base_mac);
+ memcpy(&ppid->id, &lan966x->base_mac, ppid->id_len);
+
+ return 0;
+}
+
+static const struct net_device_ops lan966x_port_netdev_ops = {
+ .ndo_open = lan966x_port_open,
+ .ndo_stop = lan966x_port_stop,
+ .ndo_start_xmit = lan966x_port_xmit,
+ .ndo_change_mtu = lan966x_port_change_mtu,
+ .ndo_set_rx_mode = lan966x_port_set_rx_mode,
+ .ndo_get_phys_port_name = lan966x_port_get_phys_port_name,
+ .ndo_get_stats64 = lan966x_stats_get,
+ .ndo_set_mac_address = lan966x_port_set_mac_address,
+ .ndo_get_port_parent_id = lan966x_port_get_parent_id,
+};
+
+bool lan966x_netdevice_check(const struct net_device *dev)
+{
+ return dev->netdev_ops == &lan966x_port_netdev_ops;
+}
+
+static int lan966x_port_xtr_status(struct lan966x *lan966x, u8 grp)
+{
+ return lan_rd(lan966x, QS_XTR_RD(grp));
+}
+
+static int lan966x_port_xtr_ready(struct lan966x *lan966x, u8 grp)
+{
+ u32 val;
+
+ return read_poll_timeout(lan966x_port_xtr_status, val,
+ val != XTR_NOT_READY,
+ READL_SLEEP_US, READL_TIMEOUT_US, false,
+ lan966x, grp);
+}
+
+static int lan966x_rx_frame_word(struct lan966x *lan966x, u8 grp, u32 *rval)
+{
+ u32 bytes_valid;
+ u32 val;
+ int err;
+
+ val = lan_rd(lan966x, QS_XTR_RD(grp));
+ if (val == XTR_NOT_READY) {
+ err = lan966x_port_xtr_ready(lan966x, grp);
+ if (err)
+ return -EIO;
+ }
+
+ switch (val) {
+ case XTR_ABORT:
+ return -EIO;
+ case XTR_EOF_0:
+ case XTR_EOF_1:
+ case XTR_EOF_2:
+ case XTR_EOF_3:
+ case XTR_PRUNED:
+ bytes_valid = XTR_VALID_BYTES(val);
+ val = lan_rd(lan966x, QS_XTR_RD(grp));
+ if (val == XTR_ESCAPE)
+ *rval = lan_rd(lan966x, QS_XTR_RD(grp));
+ else
+ *rval = val;
+
+ return bytes_valid;
+ case XTR_ESCAPE:
+ *rval = lan_rd(lan966x, QS_XTR_RD(grp));
+
+ return 4;
+ default:
+ *rval = val;
+
+ return 4;
+ }
+}
+
+static void lan966x_ifh_get_src_port(void *ifh, u64 *src_port)
+{
+ packing(ifh, src_port, IFH_POS_SRCPORT + IFH_WID_SRCPORT - 1,
+ IFH_POS_SRCPORT, IFH_LEN * 4, UNPACK, 0);
+}
+
+static void lan966x_ifh_get_len(void *ifh, u64 *len)
+{
+ packing(ifh, len, IFH_POS_LEN + IFH_WID_LEN - 1,
+ IFH_POS_LEN, IFH_LEN * 4, UNPACK, 0);
+}
+
+static irqreturn_t lan966x_xtr_irq_handler(int irq, void *args)
+{
+ struct lan966x *lan966x = args;
+ int i, grp = 0, err = 0;
+
+ if (!(lan_rd(lan966x, QS_XTR_DATA_PRESENT) & BIT(grp)))
+ return IRQ_NONE;
+
+ do {
+ struct net_device *dev;
+ struct sk_buff *skb;
+ int sz = 0, buf_len;
+ u64 src_port, len;
+ u32 ifh[IFH_LEN];
+ u32 *buf;
+ u32 val;
+
+ for (i = 0; i < IFH_LEN; i++) {
+ err = lan966x_rx_frame_word(lan966x, grp, &ifh[i]);
+ if (err != 4)
+ goto recover;
+ }
+
+ err = 0;
+
+ lan966x_ifh_get_src_port(ifh, &src_port);
+ lan966x_ifh_get_len(ifh, &len);
+
+ WARN_ON(src_port >= lan966x->num_phys_ports);
+
+ dev = lan966x->ports[src_port]->dev;
+ skb = netdev_alloc_skb(dev, len);
+ if (unlikely(!skb)) {
+ netdev_err(dev, "Unable to allocate sk_buff\n");
+ err = -ENOMEM;
+ break;
+ }
+ buf_len = len - ETH_FCS_LEN;
+ buf = (u32 *)skb_put(skb, buf_len);
+
+ len = 0;
+ do {
+ sz = lan966x_rx_frame_word(lan966x, grp, &val);
+ if (sz < 0) {
+ kfree_skb(skb);
+ goto recover;
+ }
+
+ *buf++ = val;
+ len += sz;
+ } while (len < buf_len);
+
+ /* Read the FCS */
+ sz = lan966x_rx_frame_word(lan966x, grp, &val);
+ if (sz < 0) {
+ kfree_skb(skb);
+ goto recover;
+ }
+
+ /* Update the statistics if part of the FCS was read before */
+ len -= ETH_FCS_LEN - sz;
+
+ if (unlikely(dev->features & NETIF_F_RXFCS)) {
+ buf = (u32 *)skb_put(skb, ETH_FCS_LEN);
+ *buf = val;
+ }
+
+ skb->protocol = eth_type_trans(skb, dev);
+
+ if (lan966x->bridge_mask & BIT(src_port))
+ skb->offload_fwd_mark = 1;
+
+ netif_rx_ni(skb);
+ dev->stats.rx_bytes += len;
+ dev->stats.rx_packets++;
+
+recover:
+ if (sz < 0 || err)
+ lan_rd(lan966x, QS_XTR_RD(grp));
+
+ } while (lan_rd(lan966x, QS_XTR_DATA_PRESENT) & BIT(grp));
+
+ return IRQ_HANDLED;
+}
+
+static irqreturn_t lan966x_ana_irq_handler(int irq, void *args)
+{
+ struct lan966x *lan966x = args;
+
+ return lan966x_mac_irq_handler(lan966x);
+}
+
+static void lan966x_cleanup_ports(struct lan966x *lan966x)
+{
+ struct lan966x_port *port;
+ int p;
+
+ for (p = 0; p < lan966x->num_phys_ports; p++) {
+ port = lan966x->ports[p];
+ if (!port)
+ continue;
+
+ if (port->dev)
+ unregister_netdev(port->dev);
+
+ if (port->phylink) {
+ rtnl_lock();
+ lan966x_port_stop(port->dev);
+ rtnl_unlock();
+ phylink_destroy(port->phylink);
+ port->phylink = NULL;
+ }
+
+ if (port->fwnode)
+ fwnode_handle_put(port->fwnode);
+ }
+
+ disable_irq(lan966x->xtr_irq);
+ lan966x->xtr_irq = -ENXIO;
+
+ if (lan966x->ana_irq) {
+ disable_irq(lan966x->ana_irq);
+ lan966x->ana_irq = -ENXIO;
+ }
+}
+
+static int lan966x_probe_port(struct lan966x *lan966x, u32 p,
+ phy_interface_t phy_mode,
+ struct fwnode_handle *portnp)
+{
+ struct lan966x_port *port;
+ struct phylink *phylink;
+ struct net_device *dev;
+ int err;
+
+ if (p >= lan966x->num_phys_ports)
+ return -EINVAL;
+
+ dev = devm_alloc_etherdev_mqs(lan966x->dev,
+ sizeof(struct lan966x_port), 8, 1);
+ if (!dev)
+ return -ENOMEM;
+
+ SET_NETDEV_DEV(dev, lan966x->dev);
+ port = netdev_priv(dev);
+ port->dev = dev;
+ port->lan966x = lan966x;
+ port->chip_port = p;
+ lan966x->ports[p] = port;
+
+ dev->max_mtu = ETH_MAX_MTU;
+
+ dev->netdev_ops = &lan966x_port_netdev_ops;
+ dev->ethtool_ops = &lan966x_ethtool_ops;
+ dev->features |= NETIF_F_HW_VLAN_CTAG_TX |
+ NETIF_F_HW_VLAN_STAG_TX;
+ dev->needed_headroom = IFH_LEN * sizeof(u32);
+
+ eth_hw_addr_gen(dev, lan966x->base_mac, p + 1);
+
+ lan966x_mac_learn(lan966x, PGID_CPU, dev->dev_addr, HOST_PVID,
+ ENTRYTYPE_LOCKED);
+
+ port->phylink_config.dev = &port->dev->dev;
+ port->phylink_config.type = PHYLINK_NETDEV;
+ port->phylink_pcs.poll = true;
+ port->phylink_pcs.ops = &lan966x_phylink_pcs_ops;
+
+ port->phylink_config.mac_capabilities = MAC_ASYM_PAUSE | MAC_SYM_PAUSE |
+ MAC_10 | MAC_100 | MAC_1000FD | MAC_2500FD;
+
+ __set_bit(PHY_INTERFACE_MODE_MII,
+ port->phylink_config.supported_interfaces);
+ __set_bit(PHY_INTERFACE_MODE_GMII,
+ port->phylink_config.supported_interfaces);
+ __set_bit(PHY_INTERFACE_MODE_SGMII,
+ port->phylink_config.supported_interfaces);
+ __set_bit(PHY_INTERFACE_MODE_QSGMII,
+ port->phylink_config.supported_interfaces);
+ __set_bit(PHY_INTERFACE_MODE_1000BASEX,
+ port->phylink_config.supported_interfaces);
+ __set_bit(PHY_INTERFACE_MODE_2500BASEX,
+ port->phylink_config.supported_interfaces);
+
+ phylink = phylink_create(&port->phylink_config,
+ portnp,
+ phy_mode,
+ &lan966x_phylink_mac_ops);
+ if (IS_ERR(phylink)) {
+ port->dev = NULL;
+ return PTR_ERR(phylink);
+ }
+
+ port->phylink = phylink;
+ phylink_set_pcs(phylink, &port->phylink_pcs);
+
+ err = register_netdev(dev);
+ if (err) {
+ dev_err(lan966x->dev, "register_netdev failed\n");
+ return err;
+ }
+
+ lan966x_vlan_port_set_vlan_aware(port, 0);
+ lan966x_vlan_port_set_vid(port, HOST_PVID, false, false);
+ lan966x_vlan_port_apply(port);
+
+ return 0;
+}
+
+static void lan966x_init(struct lan966x *lan966x)
+{
+ u32 p, i;
+
+ /* MAC table initialization */
+ lan966x_mac_init(lan966x);
+
+ lan966x_vlan_init(lan966x);
+
+ /* Flush queues */
+ lan_wr(lan_rd(lan966x, QS_XTR_FLUSH) |
+ GENMASK(1, 0),
+ lan966x, QS_XTR_FLUSH);
+
+ /* Allow to drain */
+ mdelay(1);
+
+ /* All Queues normal */
+ lan_wr(lan_rd(lan966x, QS_XTR_FLUSH) &
+ ~(GENMASK(1, 0)),
+ lan966x, QS_XTR_FLUSH);
+
+ /* Set MAC age time to default value, the entry is aged after
+ * 2 * AGE_PERIOD
+ */
+ lan_wr(ANA_AUTOAGE_AGE_PERIOD_SET(BR_DEFAULT_AGEING_TIME / 2 / HZ),
+ lan966x, ANA_AUTOAGE);
+
+ /* Disable learning for frames discarded by VLAN ingress filtering */
+ lan_rmw(ANA_ADVLEARN_VLAN_CHK_SET(1),
+ ANA_ADVLEARN_VLAN_CHK,
+ lan966x, ANA_ADVLEARN);
+
+ /* Setup frame ageing - "2 sec" - The unit is 6.5 us on lan966x */
+ lan_wr(SYS_FRM_AGING_AGE_TX_ENA_SET(1) |
+ (20000000 / 65),
+ lan966x, SYS_FRM_AGING);
+
+ /* Map the 8 CPU extraction queues to CPU port */
+ lan_wr(0, lan966x, QSYS_CPU_GROUP_MAP);
+
+ /* Do byte-swap and expect status after last data word
+ * Extraction: Mode: manual extraction) | Byte_swap
+ */
+ lan_wr(QS_XTR_GRP_CFG_MODE_SET(1) |
+ QS_XTR_GRP_CFG_BYTE_SWAP_SET(1),
+ lan966x, QS_XTR_GRP_CFG(0));
+
+ /* Injection: Mode: manual injection | Byte_swap */
+ lan_wr(QS_INJ_GRP_CFG_MODE_SET(1) |
+ QS_INJ_GRP_CFG_BYTE_SWAP_SET(1),
+ lan966x, QS_INJ_GRP_CFG(0));
+
+ lan_rmw(QS_INJ_CTRL_GAP_SIZE_SET(0),
+ QS_INJ_CTRL_GAP_SIZE,
+ lan966x, QS_INJ_CTRL(0));
+
+ /* Enable IFH insertion/parsing on CPU ports */
+ lan_wr(SYS_PORT_MODE_INCL_INJ_HDR_SET(1) |
+ SYS_PORT_MODE_INCL_XTR_HDR_SET(1),
+ lan966x, SYS_PORT_MODE(CPU_PORT));
+
+ /* Setup flooding PGIDs */
+ lan_wr(ANA_FLOODING_IPMC_FLD_MC4_DATA_SET(PGID_MCIPV4) |
+ ANA_FLOODING_IPMC_FLD_MC4_CTRL_SET(PGID_MC) |
+ ANA_FLOODING_IPMC_FLD_MC6_DATA_SET(PGID_MC) |
+ ANA_FLOODING_IPMC_FLD_MC6_CTRL_SET(PGID_MC),
+ lan966x, ANA_FLOODING_IPMC);
+
+ /* There are 8 priorities */
+ for (i = 0; i < 8; ++i)
+ lan_rmw(ANA_FLOODING_FLD_MULTICAST_SET(PGID_MC) |
+ ANA_FLOODING_FLD_UNICAST_SET(PGID_UC) |
+ ANA_FLOODING_FLD_BROADCAST_SET(PGID_BC),
+ ANA_FLOODING_FLD_MULTICAST |
+ ANA_FLOODING_FLD_UNICAST |
+ ANA_FLOODING_FLD_BROADCAST,
+ lan966x, ANA_FLOODING(i));
+
+ for (i = 0; i < PGID_ENTRIES; ++i)
+ /* Set all the entries to obey VLAN_VLAN */
+ lan_rmw(ANA_PGID_CFG_OBEY_VLAN_SET(1),
+ ANA_PGID_CFG_OBEY_VLAN,
+ lan966x, ANA_PGID_CFG(i));
+
+ for (p = 0; p < lan966x->num_phys_ports; p++) {
+ /* Disable bridging by default */
+ lan_rmw(ANA_PGID_PGID_SET(0x0),
+ ANA_PGID_PGID,
+ lan966x, ANA_PGID(p + PGID_SRC));
+
+ /* Do not forward BPDU frames to the front ports and copy them
+ * to CPU
+ */
+ lan_wr(0xffff, lan966x, ANA_CPU_FWD_BPDU_CFG(p));
+ }
+
+ /* Set source buffer size for each priority and each port to 1500 bytes */
+ for (i = 0; i <= QSYS_Q_RSRV; ++i) {
+ lan_wr(1500 / 64, lan966x, QSYS_RES_CFG(i));
+ lan_wr(1500 / 64, lan966x, QSYS_RES_CFG(512 + i));
+ }
+
+ /* Enable switching to/from cpu port */
+ lan_wr(QSYS_SW_PORT_MODE_PORT_ENA_SET(1) |
+ QSYS_SW_PORT_MODE_SCH_NEXT_CFG_SET(1) |
+ QSYS_SW_PORT_MODE_INGRESS_DROP_MODE_SET(1),
+ lan966x, QSYS_SW_PORT_MODE(CPU_PORT));
+
+ /* Configure and enable the CPU port */
+ lan_rmw(ANA_PGID_PGID_SET(0),
+ ANA_PGID_PGID,
+ lan966x, ANA_PGID(CPU_PORT));
+ lan_rmw(ANA_PGID_PGID_SET(BIT(CPU_PORT)),
+ ANA_PGID_PGID,
+ lan966x, ANA_PGID(PGID_CPU));
+
+ /* Multicast to all other ports */
+ lan_rmw(GENMASK(lan966x->num_phys_ports - 1, 0),
+ ANA_PGID_PGID,
+ lan966x, ANA_PGID(PGID_MC));
+
+ /* This will be controlled by mrouter ports */
+ lan_rmw(GENMASK(lan966x->num_phys_ports - 1, 0),
+ ANA_PGID_PGID,
+ lan966x, ANA_PGID(PGID_MCIPV4));
+
+ /* Unicast to all other ports */
+ lan_rmw(GENMASK(lan966x->num_phys_ports - 1, 0),
+ ANA_PGID_PGID,
+ lan966x, ANA_PGID(PGID_UC));
+
+ /* Broadcast to the CPU port and to other ports */
+ lan_rmw(ANA_PGID_PGID_SET(BIT(CPU_PORT) | GENMASK(lan966x->num_phys_ports - 1, 0)),
+ ANA_PGID_PGID,
+ lan966x, ANA_PGID(PGID_BC));
+
+ lan_wr(REW_PORT_CFG_NO_REWRITE_SET(1),
+ lan966x, REW_PORT_CFG(CPU_PORT));
+
+ lan_rmw(ANA_ANAINTR_INTR_ENA_SET(1),
+ ANA_ANAINTR_INTR_ENA,
+ lan966x, ANA_ANAINTR);
+}
+
+static int lan966x_ram_init(struct lan966x *lan966x)
+{
+ return lan_rd(lan966x, SYS_RAM_INIT);
+}
+
+static int lan966x_reset_switch(struct lan966x *lan966x)
+{
+ struct reset_control *switch_reset, *phy_reset;
+ int val = 0;
+ int ret;
+
+ switch_reset = devm_reset_control_get_shared(lan966x->dev, "switch");
+ if (IS_ERR(switch_reset))
+ return dev_err_probe(lan966x->dev, PTR_ERR(switch_reset),
+ "Could not obtain switch reset");
+
+ phy_reset = devm_reset_control_get_shared(lan966x->dev, "phy");
+ if (IS_ERR(phy_reset))
+ return dev_err_probe(lan966x->dev, PTR_ERR(phy_reset),
+ "Could not obtain phy reset\n");
+
+ reset_control_reset(switch_reset);
+ reset_control_reset(phy_reset);
+
+ lan_wr(SYS_RESET_CFG_CORE_ENA_SET(0), lan966x, SYS_RESET_CFG);
+ lan_wr(SYS_RAM_INIT_RAM_INIT_SET(1), lan966x, SYS_RAM_INIT);
+ ret = readx_poll_timeout(lan966x_ram_init, lan966x,
+ val, (val & BIT(1)) == 0, READL_SLEEP_US,
+ READL_TIMEOUT_US);
+ if (ret)
+ return ret;
+
+ lan_wr(SYS_RESET_CFG_CORE_ENA_SET(1), lan966x, SYS_RESET_CFG);
+
+ return 0;
+}
+
+static int lan966x_probe(struct platform_device *pdev)
+{
+ struct fwnode_handle *ports, *portnp;
+ struct lan966x *lan966x;
+ u8 mac_addr[ETH_ALEN];
+ int err, i;
+
+ lan966x = devm_kzalloc(&pdev->dev, sizeof(*lan966x), GFP_KERNEL);
+ if (!lan966x)
+ return -ENOMEM;
+
+ platform_set_drvdata(pdev, lan966x);
+ lan966x->dev = &pdev->dev;
+
+ if (!device_get_mac_address(&pdev->dev, mac_addr)) {
+ ether_addr_copy(lan966x->base_mac, mac_addr);
+ } else {
+ pr_info("MAC addr was not set, use random MAC\n");
+ eth_random_addr(lan966x->base_mac);
+ lan966x->base_mac[5] &= 0xf0;
+ }
+
+ ports = device_get_named_child_node(&pdev->dev, "ethernet-ports");
+ if (!ports)
+ return dev_err_probe(&pdev->dev, -ENODEV,
+ "no ethernet-ports child found\n");
+
+ err = lan966x_create_targets(pdev, lan966x);
+ if (err)
+ return dev_err_probe(&pdev->dev, err,
+ "Failed to create targets");
+
+ err = lan966x_reset_switch(lan966x);
+ if (err)
+ return dev_err_probe(&pdev->dev, err, "Reset failed");
+
+ i = 0;
+ fwnode_for_each_available_child_node(ports, portnp)
+ ++i;
+
+ lan966x->num_phys_ports = i;
+ lan966x->ports = devm_kcalloc(&pdev->dev, lan966x->num_phys_ports,
+ sizeof(struct lan966x_port *),
+ GFP_KERNEL);
+ if (!lan966x->ports)
+ return -ENOMEM;
+
+ /* There QS system has 32KB of memory */
+ lan966x->shared_queue_sz = LAN966X_BUFFER_MEMORY;
+
+ /* set irq */
+ lan966x->xtr_irq = platform_get_irq_byname(pdev, "xtr");
+ if (lan966x->xtr_irq <= 0)
+ return -EINVAL;
+
+ err = devm_request_threaded_irq(&pdev->dev, lan966x->xtr_irq, NULL,
+ lan966x_xtr_irq_handler, IRQF_ONESHOT,
+ "frame extraction", lan966x);
+ if (err) {
+ pr_err("Unable to use xtr irq");
+ return -ENODEV;
+ }
+
+ lan966x->ana_irq = platform_get_irq_byname(pdev, "ana");
+ if (lan966x->ana_irq) {
+ err = devm_request_threaded_irq(&pdev->dev, lan966x->ana_irq, NULL,
+ lan966x_ana_irq_handler, IRQF_ONESHOT,
+ "ana irq", lan966x);
+ if (err)
+ return dev_err_probe(&pdev->dev, err, "Unable to use ana irq");
+ }
+
+ /* init switch */
+ lan966x_init(lan966x);
+ lan966x_stats_init(lan966x);
+
+ /* go over the child nodes */
+ fwnode_for_each_available_child_node(ports, portnp) {
+ phy_interface_t phy_mode;
+ struct phy *serdes;
+ u32 p;
+
+ if (fwnode_property_read_u32(portnp, "reg", &p))
+ continue;
+
+ phy_mode = fwnode_get_phy_mode(portnp);
+ err = lan966x_probe_port(lan966x, p, phy_mode, portnp);
+ if (err)
+ goto cleanup_ports;
+
+ /* Read needed configuration */
+ lan966x->ports[p]->config.portmode = phy_mode;
+ lan966x->ports[p]->fwnode = fwnode_handle_get(portnp);
+
+ serdes = devm_of_phy_get(lan966x->dev, to_of_node(portnp), NULL);
+ if (!IS_ERR(serdes))
+ lan966x->ports[p]->serdes = serdes;
+
+ lan966x_port_init(lan966x->ports[p]);
+ }
+
+ lan966x_mdb_init(lan966x);
+ err = lan966x_fdb_init(lan966x);
+ if (err)
+ goto cleanup_ports;
+
+ return 0;
+
+cleanup_ports:
+ fwnode_handle_put(portnp);
+
+ lan966x_cleanup_ports(lan966x);
+
+ cancel_delayed_work_sync(&lan966x->stats_work);
+ destroy_workqueue(lan966x->stats_queue);
+ mutex_destroy(&lan966x->stats_lock);
+
+ return err;
+}
+
+static int lan966x_remove(struct platform_device *pdev)
+{
+ struct lan966x *lan966x = platform_get_drvdata(pdev);
+
+ lan966x_cleanup_ports(lan966x);
+
+ cancel_delayed_work_sync(&lan966x->stats_work);
+ destroy_workqueue(lan966x->stats_queue);
+ mutex_destroy(&lan966x->stats_lock);
+
+ lan966x_mac_purge_entries(lan966x);
+ lan966x_mdb_deinit(lan966x);
+ lan966x_fdb_deinit(lan966x);
+
+ return 0;
+}
+
+static struct platform_driver lan966x_driver = {
+ .probe = lan966x_probe,
+ .remove = lan966x_remove,
+ .driver = {
+ .name = "lan966x-switch",
+ .of_match_table = lan966x_match,
+ },
+};
+
+static int __init lan966x_switch_driver_init(void)
+{
+ int ret;
+
+ lan966x_register_notifier_blocks();
+
+ ret = platform_driver_register(&lan966x_driver);
+ if (ret)
+ goto err;
+
+ return 0;
+
+err:
+ lan966x_unregister_notifier_blocks();
+ return ret;
+}
+
+static void __exit lan966x_switch_driver_exit(void)
+{
+ platform_driver_unregister(&lan966x_driver);
+ lan966x_unregister_notifier_blocks();
+}
+
+module_init(lan966x_switch_driver_init);
+module_exit(lan966x_switch_driver_exit);
+
+MODULE_DESCRIPTION("Microchip LAN966X switch driver");
+MODULE_AUTHOR("Horatiu Vultur <horatiu.vultur@microchip.com>");
+MODULE_LICENSE("Dual MIT/GPL");
diff --git a/drivers/net/ethernet/microchip/lan966x/lan966x_main.h b/drivers/net/ethernet/microchip/lan966x/lan966x_main.h
new file mode 100644
index 0000000..99c6d0a
--- /dev/null
+++ b/drivers/net/ethernet/microchip/lan966x/lan966x_main.h
@@ -0,0 +1,278 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+
+#ifndef __LAN966X_MAIN_H__
+#define __LAN966X_MAIN_H__
+
+#include <linux/etherdevice.h>
+#include <linux/if_vlan.h>
+#include <linux/jiffies.h>
+#include <linux/phy.h>
+#include <linux/phylink.h>
+#include <net/switchdev.h>
+
+#include "lan966x_regs.h"
+#include "lan966x_ifh.h"
+
+#define TABLE_UPDATE_SLEEP_US 10
+#define TABLE_UPDATE_TIMEOUT_US 100000
+
+#define LAN966X_BUFFER_CELL_SZ 64
+#define LAN966X_BUFFER_MEMORY (160 * 1024)
+#define LAN966X_BUFFER_MIN_SZ 60
+
+#define PGID_AGGR 64
+#define PGID_SRC 80
+#define PGID_ENTRIES 89
+
+#define UNAWARE_PVID 0
+#define HOST_PVID 4095
+
+/* Reserved amount for (SRC, PRIO) at index 8*SRC + PRIO */
+#define QSYS_Q_RSRV 95
+
+#define CPU_PORT 8
+
+/* Reserved PGIDs */
+#define PGID_CPU (PGID_AGGR - 6)
+#define PGID_UC (PGID_AGGR - 5)
+#define PGID_BC (PGID_AGGR - 4)
+#define PGID_MC (PGID_AGGR - 3)
+#define PGID_MCIPV4 (PGID_AGGR - 2)
+#define PGID_MCIPV6 (PGID_AGGR - 1)
+
+/* Non-reserved PGIDs, used for general purpose */
+#define PGID_GP_START (CPU_PORT + 1)
+#define PGID_GP_END PGID_CPU
+
+#define LAN966X_SPEED_NONE 0
+#define LAN966X_SPEED_2500 1
+#define LAN966X_SPEED_1000 1
+#define LAN966X_SPEED_100 2
+#define LAN966X_SPEED_10 3
+
+/* MAC table entry types.
+ * ENTRYTYPE_NORMAL is subject to aging.
+ * ENTRYTYPE_LOCKED is not subject to aging.
+ * ENTRYTYPE_MACv4 is not subject to aging. For IPv4 multicast.
+ * ENTRYTYPE_MACv6 is not subject to aging. For IPv6 multicast.
+ */
+enum macaccess_entry_type {
+ ENTRYTYPE_NORMAL = 0,
+ ENTRYTYPE_LOCKED,
+ ENTRYTYPE_MACV4,
+ ENTRYTYPE_MACV6,
+};
+
+struct lan966x_port;
+
+struct lan966x_stat_layout {
+ u32 offset;
+ char name[ETH_GSTRING_LEN];
+};
+
+struct lan966x {
+ struct device *dev;
+
+ u8 num_phys_ports;
+ struct lan966x_port **ports;
+
+ void __iomem *regs[NUM_TARGETS];
+
+ int shared_queue_sz;
+
+ u8 base_mac[ETH_ALEN];
+
+ struct net_device *bridge;
+ u16 bridge_mask;
+ u16 bridge_fwd_mask;
+
+ struct list_head mac_entries;
+ spinlock_t mac_lock; /* lock for mac_entries list */
+
+ u16 vlan_mask[VLAN_N_VID];
+ DECLARE_BITMAP(cpu_vlan_mask, VLAN_N_VID);
+
+ /* stats */
+ const struct lan966x_stat_layout *stats_layout;
+ u32 num_stats;
+
+ /* workqueue for reading stats */
+ struct mutex stats_lock;
+ u64 *stats;
+ struct delayed_work stats_work;
+ struct workqueue_struct *stats_queue;
+
+ /* interrupts */
+ int xtr_irq;
+ int ana_irq;
+
+ /* worqueue for fdb */
+ struct workqueue_struct *fdb_work;
+ struct list_head fdb_entries;
+
+ /* mdb */
+ struct list_head mdb_entries;
+ struct list_head pgid_entries;
+};
+
+struct lan966x_port_config {
+ phy_interface_t portmode;
+ const unsigned long *advertising;
+ int speed;
+ int duplex;
+ u32 pause;
+ bool inband;
+ bool autoneg;
+};
+
+struct lan966x_port {
+ struct net_device *dev;
+ struct lan966x *lan966x;
+
+ u8 chip_port;
+ u16 pvid;
+ u16 vid;
+ bool vlan_aware;
+
+ bool learn_ena;
+
+ struct phylink_config phylink_config;
+ struct phylink_pcs phylink_pcs;
+ struct lan966x_port_config config;
+ struct phylink *phylink;
+ struct phy *serdes;
+ struct fwnode_handle *fwnode;
+};
+
+extern const struct phylink_mac_ops lan966x_phylink_mac_ops;
+extern const struct phylink_pcs_ops lan966x_phylink_pcs_ops;
+extern const struct ethtool_ops lan966x_ethtool_ops;
+
+bool lan966x_netdevice_check(const struct net_device *dev);
+
+void lan966x_register_notifier_blocks(void);
+void lan966x_unregister_notifier_blocks(void);
+
+void lan966x_stats_get(struct net_device *dev,
+ struct rtnl_link_stats64 *stats);
+int lan966x_stats_init(struct lan966x *lan966x);
+
+void lan966x_port_config_down(struct lan966x_port *port);
+void lan966x_port_config_up(struct lan966x_port *port);
+void lan966x_port_status_get(struct lan966x_port *port,
+ struct phylink_link_state *state);
+int lan966x_port_pcs_set(struct lan966x_port *port,
+ struct lan966x_port_config *config);
+void lan966x_port_init(struct lan966x_port *port);
+
+int lan966x_mac_ip_learn(struct lan966x *lan966x,
+ bool cpu_copy,
+ const unsigned char mac[ETH_ALEN],
+ unsigned int vid,
+ enum macaccess_entry_type type);
+int lan966x_mac_learn(struct lan966x *lan966x, int port,
+ const unsigned char mac[ETH_ALEN],
+ unsigned int vid,
+ enum macaccess_entry_type type);
+int lan966x_mac_forget(struct lan966x *lan966x,
+ const unsigned char mac[ETH_ALEN],
+ unsigned int vid,
+ enum macaccess_entry_type type);
+int lan966x_mac_cpu_learn(struct lan966x *lan966x, const char *addr, u16 vid);
+int lan966x_mac_cpu_forget(struct lan966x *lan966x, const char *addr, u16 vid);
+void lan966x_mac_init(struct lan966x *lan966x);
+void lan966x_mac_set_ageing(struct lan966x *lan966x,
+ u32 ageing);
+int lan966x_mac_del_entry(struct lan966x *lan966x,
+ const unsigned char *addr,
+ u16 vid);
+int lan966x_mac_add_entry(struct lan966x *lan966x,
+ struct lan966x_port *port,
+ const unsigned char *addr,
+ u16 vid);
+void lan966x_mac_purge_entries(struct lan966x *lan966x);
+irqreturn_t lan966x_mac_irq_handler(struct lan966x *lan966x);
+
+void lan966x_vlan_init(struct lan966x *lan966x);
+void lan966x_vlan_port_apply(struct lan966x_port *port);
+bool lan966x_vlan_cpu_member_cpu_vlan_mask(struct lan966x *lan966x, u16 vid);
+void lan966x_vlan_port_set_vlan_aware(struct lan966x_port *port,
+ bool vlan_aware);
+int lan966x_vlan_port_set_vid(struct lan966x_port *port,
+ u16 vid,
+ bool pvid,
+ bool untagged);
+void lan966x_vlan_port_add_vlan(struct lan966x_port *port,
+ u16 vid,
+ bool pvid,
+ bool untagged);
+void lan966x_vlan_port_del_vlan(struct lan966x_port *port, u16 vid);
+void lan966x_vlan_cpu_add_vlan(struct lan966x *lan966x, u16 vid);
+void lan966x_vlan_cpu_del_vlan(struct lan966x *lan966x, u16 vid);
+
+void lan966x_fdb_write_entries(struct lan966x *lan966x, u16 vid);
+void lan966x_fdb_erase_entries(struct lan966x *lan966x, u16 vid);
+int lan966x_fdb_init(struct lan966x *lan966x);
+void lan966x_fdb_deinit(struct lan966x *lan966x);
+int lan966x_handle_fdb(struct net_device *dev,
+ struct net_device *orig_dev,
+ unsigned long event, const void *ctx,
+ const struct switchdev_notifier_fdb_info *fdb_info);
+
+void lan966x_mdb_init(struct lan966x *lan966x);
+void lan966x_mdb_deinit(struct lan966x *lan966x);
+int lan966x_handle_port_mdb_add(struct lan966x_port *port,
+ const struct switchdev_obj *obj);
+int lan966x_handle_port_mdb_del(struct lan966x_port *port,
+ const struct switchdev_obj *obj);
+void lan966x_mdb_erase_entries(struct lan966x *lan966x, u16 vid);
+void lan966x_mdb_write_entries(struct lan966x *lan966x, u16 vid);
+
+static inline void __iomem *lan_addr(void __iomem *base[],
+ int id, int tinst, int tcnt,
+ int gbase, int ginst,
+ int gcnt, int gwidth,
+ int raddr, int rinst,
+ int rcnt, int rwidth)
+{
+ WARN_ON((tinst) >= tcnt);
+ WARN_ON((ginst) >= gcnt);
+ WARN_ON((rinst) >= rcnt);
+ return base[id + (tinst)] +
+ gbase + ((ginst) * gwidth) +
+ raddr + ((rinst) * rwidth);
+}
+
+static inline u32 lan_rd(struct lan966x *lan966x, int id, int tinst, int tcnt,
+ int gbase, int ginst, int gcnt, int gwidth,
+ int raddr, int rinst, int rcnt, int rwidth)
+{
+ return readl(lan_addr(lan966x->regs, id, tinst, tcnt, gbase, ginst,
+ gcnt, gwidth, raddr, rinst, rcnt, rwidth));
+}
+
+static inline void lan_wr(u32 val, struct lan966x *lan966x,
+ int id, int tinst, int tcnt,
+ int gbase, int ginst, int gcnt, int gwidth,
+ int raddr, int rinst, int rcnt, int rwidth)
+{
+ writel(val, lan_addr(lan966x->regs, id, tinst, tcnt,
+ gbase, ginst, gcnt, gwidth,
+ raddr, rinst, rcnt, rwidth));
+}
+
+static inline void lan_rmw(u32 val, u32 mask, struct lan966x *lan966x,
+ int id, int tinst, int tcnt,
+ int gbase, int ginst, int gcnt, int gwidth,
+ int raddr, int rinst, int rcnt, int rwidth)
+{
+ u32 nval;
+
+ nval = readl(lan_addr(lan966x->regs, id, tinst, tcnt, gbase, ginst,
+ gcnt, gwidth, raddr, rinst, rcnt, rwidth));
+ nval = (nval & ~mask) | (val & mask);
+ writel(nval, lan_addr(lan966x->regs, id, tinst, tcnt, gbase, ginst,
+ gcnt, gwidth, raddr, rinst, rcnt, rwidth));
+}
+
+#endif /* __LAN966X_MAIN_H__ */
diff --git a/drivers/net/ethernet/microchip/lan966x/lan966x_mdb.c b/drivers/net/ethernet/microchip/lan966x/lan966x_mdb.c
new file mode 100644
index 0000000..c68d0a9
--- /dev/null
+++ b/drivers/net/ethernet/microchip/lan966x/lan966x_mdb.c
@@ -0,0 +1,506 @@
+// SPDX-License-Identifier: GPL-2.0+
+
+#include <net/switchdev.h>
+
+#include "lan966x_main.h"
+
+struct lan966x_pgid_entry {
+ struct list_head list;
+ int index;
+ refcount_t refcount;
+ u16 ports;
+};
+
+struct lan966x_mdb_entry {
+ struct list_head list;
+ unsigned char mac[ETH_ALEN];
+ u16 vid;
+ u16 ports;
+ struct lan966x_pgid_entry *pgid;
+ u8 cpu_copy;
+};
+
+void lan966x_mdb_init(struct lan966x *lan966x)
+{
+ INIT_LIST_HEAD(&lan966x->mdb_entries);
+ INIT_LIST_HEAD(&lan966x->pgid_entries);
+}
+
+static void lan966x_mdb_purge_mdb_entries(struct lan966x *lan966x)
+{
+ struct lan966x_mdb_entry *mdb_entry, *tmp;
+
+ list_for_each_entry_safe(mdb_entry, tmp, &lan966x->mdb_entries, list) {
+ list_del(&mdb_entry->list);
+ kfree(mdb_entry);
+ }
+}
+
+static void lan966x_mdb_purge_pgid_entries(struct lan966x *lan966x)
+{
+ struct lan966x_pgid_entry *pgid_entry, *tmp;
+
+ list_for_each_entry_safe(pgid_entry, tmp, &lan966x->pgid_entries, list) {
+ list_del(&pgid_entry->list);
+ kfree(pgid_entry);
+ }
+}
+
+void lan966x_mdb_deinit(struct lan966x *lan966x)
+{
+ lan966x_mdb_purge_mdb_entries(lan966x);
+ lan966x_mdb_purge_pgid_entries(lan966x);
+}
+
+static struct lan966x_mdb_entry *
+lan966x_mdb_entry_get(struct lan966x *lan966x,
+ const unsigned char *mac,
+ u16 vid)
+{
+ struct lan966x_mdb_entry *mdb_entry;
+
+ list_for_each_entry(mdb_entry, &lan966x->mdb_entries, list) {
+ if (ether_addr_equal(mdb_entry->mac, mac) &&
+ mdb_entry->vid == vid)
+ return mdb_entry;
+ }
+
+ return NULL;
+}
+
+static struct lan966x_mdb_entry *
+lan966x_mdb_entry_add(struct lan966x *lan966x,
+ const struct switchdev_obj_port_mdb *mdb)
+{
+ struct lan966x_mdb_entry *mdb_entry;
+
+ mdb_entry = kzalloc(sizeof(*mdb_entry), GFP_KERNEL);
+ if (!mdb_entry)
+ return ERR_PTR(-ENOMEM);
+
+ ether_addr_copy(mdb_entry->mac, mdb->addr);
+ mdb_entry->vid = mdb->vid;
+
+ list_add_tail(&mdb_entry->list, &lan966x->mdb_entries);
+
+ return mdb_entry;
+}
+
+static void lan966x_mdb_encode_mac(unsigned char *mac,
+ struct lan966x_mdb_entry *mdb_entry,
+ enum macaccess_entry_type type)
+{
+ ether_addr_copy(mac, mdb_entry->mac);
+
+ if (type == ENTRYTYPE_MACV4) {
+ mac[0] = 0;
+ mac[1] = mdb_entry->ports >> 8;
+ mac[2] = mdb_entry->ports & 0xff;
+ } else if (type == ENTRYTYPE_MACV6) {
+ mac[0] = mdb_entry->ports >> 8;
+ mac[1] = mdb_entry->ports & 0xff;
+ }
+}
+
+static int lan966x_mdb_ip_add(struct lan966x_port *port,
+ const struct switchdev_obj_port_mdb *mdb,
+ enum macaccess_entry_type type)
+{
+ bool cpu_port = netif_is_bridge_master(mdb->obj.orig_dev);
+ struct lan966x *lan966x = port->lan966x;
+ struct lan966x_mdb_entry *mdb_entry;
+ unsigned char mac[ETH_ALEN];
+ bool cpu_copy = false;
+
+ mdb_entry = lan966x_mdb_entry_get(lan966x, mdb->addr, mdb->vid);
+ if (!mdb_entry) {
+ mdb_entry = lan966x_mdb_entry_add(lan966x, mdb);
+ if (IS_ERR(mdb_entry))
+ return PTR_ERR(mdb_entry);
+ } else {
+ lan966x_mdb_encode_mac(mac, mdb_entry, type);
+ lan966x_mac_forget(lan966x, mac, mdb_entry->vid, type);
+ }
+
+ if (cpu_port)
+ mdb_entry->cpu_copy++;
+ else
+ mdb_entry->ports |= BIT(port->chip_port);
+
+ /* Copy the frame to CPU only if the CPU is in the VLAN */
+ if (lan966x_vlan_cpu_member_cpu_vlan_mask(lan966x, mdb_entry->vid) &&
+ mdb_entry->cpu_copy)
+ cpu_copy = true;
+
+ lan966x_mdb_encode_mac(mac, mdb_entry, type);
+ return lan966x_mac_ip_learn(lan966x, cpu_copy,
+ mac, mdb_entry->vid, type);
+}
+
+static int lan966x_mdb_ip_del(struct lan966x_port *port,
+ const struct switchdev_obj_port_mdb *mdb,
+ enum macaccess_entry_type type)
+{
+ bool cpu_port = netif_is_bridge_master(mdb->obj.orig_dev);
+ struct lan966x *lan966x = port->lan966x;
+ struct lan966x_mdb_entry *mdb_entry;
+ unsigned char mac[ETH_ALEN];
+ u16 ports;
+
+ mdb_entry = lan966x_mdb_entry_get(lan966x, mdb->addr, mdb->vid);
+ if (!mdb_entry)
+ return -ENOENT;
+
+ ports = mdb_entry->ports;
+ if (cpu_port) {
+ /* If there are still other references to the CPU port then
+ * there is no point to delete and add again the same entry
+ */
+ mdb_entry->cpu_copy--;
+ if (mdb_entry->cpu_copy)
+ return 0;
+ } else {
+ ports &= ~BIT(port->chip_port);
+ }
+
+ lan966x_mdb_encode_mac(mac, mdb_entry, type);
+ lan966x_mac_forget(lan966x, mac, mdb_entry->vid, type);
+
+ mdb_entry->ports = ports;
+
+ if (!mdb_entry->ports && !mdb_entry->cpu_copy) {
+ list_del(&mdb_entry->list);
+ kfree(mdb_entry);
+ return 0;
+ }
+
+ lan966x_mdb_encode_mac(mac, mdb_entry, type);
+ return lan966x_mac_ip_learn(lan966x, mdb_entry->cpu_copy,
+ mac, mdb_entry->vid, type);
+}
+
+static struct lan966x_pgid_entry *
+lan966x_pgid_entry_add(struct lan966x *lan966x, int index, u16 ports)
+{
+ struct lan966x_pgid_entry *pgid_entry;
+
+ pgid_entry = kzalloc(sizeof(*pgid_entry), GFP_KERNEL);
+ if (!pgid_entry)
+ return ERR_PTR(-ENOMEM);
+
+ pgid_entry->ports = ports;
+ pgid_entry->index = index;
+ refcount_set(&pgid_entry->refcount, 1);
+
+ list_add_tail(&pgid_entry->list, &lan966x->pgid_entries);
+
+ return pgid_entry;
+}
+
+static struct lan966x_pgid_entry *
+lan966x_pgid_entry_get(struct lan966x *lan966x,
+ struct lan966x_mdb_entry *mdb_entry)
+{
+ struct lan966x_pgid_entry *pgid_entry;
+ int index;
+
+ /* Try to find an existing pgid that uses the same ports as the
+ * mdb_entry
+ */
+ list_for_each_entry(pgid_entry, &lan966x->pgid_entries, list) {
+ if (pgid_entry->ports == mdb_entry->ports) {
+ refcount_inc(&pgid_entry->refcount);
+ return pgid_entry;
+ }
+ }
+
+ /* Try to find an empty pgid entry and allocate one in case it finds it,
+ * otherwise it means that there are no more resources
+ */
+ for (index = PGID_GP_START; index < PGID_GP_END; index++) {
+ bool used = false;
+
+ list_for_each_entry(pgid_entry, &lan966x->pgid_entries, list) {
+ if (pgid_entry->index == index) {
+ used = true;
+ break;
+ }
+ }
+
+ if (!used)
+ return lan966x_pgid_entry_add(lan966x, index,
+ mdb_entry->ports);
+ }
+
+ return ERR_PTR(-ENOSPC);
+}
+
+static void lan966x_pgid_entry_del(struct lan966x *lan966x,
+ struct lan966x_pgid_entry *pgid_entry)
+{
+ if (!refcount_dec_and_test(&pgid_entry->refcount))
+ return;
+
+ list_del(&pgid_entry->list);
+ kfree(pgid_entry);
+}
+
+static int lan966x_mdb_l2_add(struct lan966x_port *port,
+ const struct switchdev_obj_port_mdb *mdb,
+ enum macaccess_entry_type type)
+{
+ bool cpu_port = netif_is_bridge_master(mdb->obj.orig_dev);
+ struct lan966x *lan966x = port->lan966x;
+ struct lan966x_pgid_entry *pgid_entry;
+ struct lan966x_mdb_entry *mdb_entry;
+ unsigned char mac[ETH_ALEN];
+
+ mdb_entry = lan966x_mdb_entry_get(lan966x, mdb->addr, mdb->vid);
+ if (!mdb_entry) {
+ mdb_entry = lan966x_mdb_entry_add(lan966x, mdb);
+ if (IS_ERR(mdb_entry))
+ return PTR_ERR(mdb_entry);
+ } else {
+ lan966x_pgid_entry_del(lan966x, mdb_entry->pgid);
+ lan966x_mdb_encode_mac(mac, mdb_entry, type);
+ lan966x_mac_forget(lan966x, mac, mdb_entry->vid, type);
+ }
+
+ if (cpu_port) {
+ mdb_entry->ports |= BIT(CPU_PORT);
+ mdb_entry->cpu_copy++;
+ } else {
+ mdb_entry->ports |= BIT(port->chip_port);
+ }
+
+ pgid_entry = lan966x_pgid_entry_get(lan966x, mdb_entry);
+ if (IS_ERR(pgid_entry)) {
+ list_del(&mdb_entry->list);
+ kfree(mdb_entry);
+ return PTR_ERR(pgid_entry);
+ }
+ mdb_entry->pgid = pgid_entry;
+
+ /* Copy the frame to CPU only if the CPU is in the VLAN */
+ if (!lan966x_vlan_cpu_member_cpu_vlan_mask(lan966x, mdb_entry->vid) &&
+ mdb_entry->cpu_copy)
+ mdb_entry->ports &= BIT(CPU_PORT);
+
+ lan_rmw(ANA_PGID_PGID_SET(mdb_entry->ports),
+ ANA_PGID_PGID,
+ lan966x, ANA_PGID(pgid_entry->index));
+
+ return lan966x_mac_learn(lan966x, pgid_entry->index, mdb_entry->mac,
+ mdb_entry->vid, type);
+}
+
+static int lan966x_mdb_l2_del(struct lan966x_port *port,
+ const struct switchdev_obj_port_mdb *mdb,
+ enum macaccess_entry_type type)
+{
+ bool cpu_port = netif_is_bridge_master(mdb->obj.orig_dev);
+ struct lan966x *lan966x = port->lan966x;
+ struct lan966x_pgid_entry *pgid_entry;
+ struct lan966x_mdb_entry *mdb_entry;
+ unsigned char mac[ETH_ALEN];
+ u16 ports;
+
+ mdb_entry = lan966x_mdb_entry_get(lan966x, mdb->addr, mdb->vid);
+ if (!mdb_entry)
+ return -ENOENT;
+
+ ports = mdb_entry->ports;
+ if (cpu_port) {
+ /* If there are still other references to the CPU port then
+ * there is no point to delete and add again the same entry
+ */
+ mdb_entry->cpu_copy--;
+ if (mdb_entry->cpu_copy)
+ return 0;
+
+ ports &= ~BIT(CPU_PORT);
+ } else {
+ ports &= ~BIT(port->chip_port);
+ }
+
+ lan966x_mdb_encode_mac(mac, mdb_entry, type);
+ lan966x_mac_forget(lan966x, mac, mdb_entry->vid, type);
+ lan966x_pgid_entry_del(lan966x, mdb_entry->pgid);
+
+ mdb_entry->ports = ports;
+
+ if (!mdb_entry->ports) {
+ list_del(&mdb_entry->list);
+ kfree(mdb_entry);
+ return 0;
+ }
+
+ pgid_entry = lan966x_pgid_entry_get(lan966x, mdb_entry);
+ if (IS_ERR(pgid_entry)) {
+ list_del(&mdb_entry->list);
+ kfree(mdb_entry);
+ return PTR_ERR(pgid_entry);
+ }
+ mdb_entry->pgid = pgid_entry;
+
+ lan_rmw(ANA_PGID_PGID_SET(mdb_entry->ports),
+ ANA_PGID_PGID,
+ lan966x, ANA_PGID(pgid_entry->index));
+
+ return lan966x_mac_learn(lan966x, pgid_entry->index, mdb_entry->mac,
+ mdb_entry->vid, type);
+}
+
+static enum macaccess_entry_type
+lan966x_mdb_classify(const unsigned char *mac)
+{
+ if (mac[0] == 0x01 && mac[1] == 0x00 && mac[2] == 0x5e)
+ return ENTRYTYPE_MACV4;
+ if (mac[0] == 0x33 && mac[1] == 0x33)
+ return ENTRYTYPE_MACV6;
+ return ENTRYTYPE_LOCKED;
+}
+
+int lan966x_handle_port_mdb_add(struct lan966x_port *port,
+ const struct switchdev_obj *obj)
+{
+ const struct switchdev_obj_port_mdb *mdb = SWITCHDEV_OBJ_PORT_MDB(obj);
+ enum macaccess_entry_type type;
+
+ /* Split the way the entries are added for ipv4/ipv6 and for l2. The
+ * reason is that for ipv4/ipv6 it doesn't require to use any pgid
+ * entry, while for l2 is required to use pgid entries
+ */
+ type = lan966x_mdb_classify(mdb->addr);
+ if (type == ENTRYTYPE_MACV4 || type == ENTRYTYPE_MACV6)
+ return lan966x_mdb_ip_add(port, mdb, type);
+
+ return lan966x_mdb_l2_add(port, mdb, type);
+}
+
+int lan966x_handle_port_mdb_del(struct lan966x_port *port,
+ const struct switchdev_obj *obj)
+{
+ const struct switchdev_obj_port_mdb *mdb = SWITCHDEV_OBJ_PORT_MDB(obj);
+ enum macaccess_entry_type type;
+
+ /* Split the way the entries are removed for ipv4/ipv6 and for l2. The
+ * reason is that for ipv4/ipv6 it doesn't require to use any pgid
+ * entry, while for l2 is required to use pgid entries
+ */
+ type = lan966x_mdb_classify(mdb->addr);
+ if (type == ENTRYTYPE_MACV4 || type == ENTRYTYPE_MACV6)
+ return lan966x_mdb_ip_del(port, mdb, type);
+
+ return lan966x_mdb_l2_del(port, mdb, type);
+}
+
+static void lan966x_mdb_ip_cpu_copy(struct lan966x *lan966x,
+ struct lan966x_mdb_entry *mdb_entry,
+ enum macaccess_entry_type type)
+{
+ unsigned char mac[ETH_ALEN];
+
+ lan966x_mdb_encode_mac(mac, mdb_entry, type);
+ lan966x_mac_forget(lan966x, mac, mdb_entry->vid, type);
+ lan966x_mac_ip_learn(lan966x, true, mac, mdb_entry->vid, type);
+}
+
+static void lan966x_mdb_l2_cpu_copy(struct lan966x *lan966x,
+ struct lan966x_mdb_entry *mdb_entry,
+ enum macaccess_entry_type type)
+{
+ struct lan966x_pgid_entry *pgid_entry;
+ unsigned char mac[ETH_ALEN];
+
+ lan966x_pgid_entry_del(lan966x, mdb_entry->pgid);
+ lan966x_mdb_encode_mac(mac, mdb_entry, type);
+ lan966x_mac_forget(lan966x, mac, mdb_entry->vid, type);
+
+ mdb_entry->ports |= BIT(CPU_PORT);
+
+ pgid_entry = lan966x_pgid_entry_get(lan966x, mdb_entry);
+ if (IS_ERR(pgid_entry))
+ return;
+
+ mdb_entry->pgid = pgid_entry;
+
+ lan_rmw(ANA_PGID_PGID_SET(mdb_entry->ports),
+ ANA_PGID_PGID,
+ lan966x, ANA_PGID(pgid_entry->index));
+
+ lan966x_mac_learn(lan966x, pgid_entry->index, mdb_entry->mac,
+ mdb_entry->vid, type);
+}
+
+void lan966x_mdb_write_entries(struct lan966x *lan966x, u16 vid)
+{
+ struct lan966x_mdb_entry *mdb_entry;
+ enum macaccess_entry_type type;
+
+ list_for_each_entry(mdb_entry, &lan966x->mdb_entries, list) {
+ if (mdb_entry->vid != vid || !mdb_entry->cpu_copy)
+ continue;
+
+ type = lan966x_mdb_classify(mdb_entry->mac);
+ if (type == ENTRYTYPE_MACV4 || type == ENTRYTYPE_MACV6)
+ lan966x_mdb_ip_cpu_copy(lan966x, mdb_entry, type);
+ else
+ lan966x_mdb_l2_cpu_copy(lan966x, mdb_entry, type);
+ }
+}
+
+static void lan966x_mdb_ip_cpu_remove(struct lan966x *lan966x,
+ struct lan966x_mdb_entry *mdb_entry,
+ enum macaccess_entry_type type)
+{
+ unsigned char mac[ETH_ALEN];
+
+ lan966x_mdb_encode_mac(mac, mdb_entry, type);
+ lan966x_mac_forget(lan966x, mac, mdb_entry->vid, type);
+ lan966x_mac_ip_learn(lan966x, false, mac, mdb_entry->vid, type);
+}
+
+static void lan966x_mdb_l2_cpu_remove(struct lan966x *lan966x,
+ struct lan966x_mdb_entry *mdb_entry,
+ enum macaccess_entry_type type)
+{
+ struct lan966x_pgid_entry *pgid_entry;
+ unsigned char mac[ETH_ALEN];
+
+ lan966x_pgid_entry_del(lan966x, mdb_entry->pgid);
+ lan966x_mdb_encode_mac(mac, mdb_entry, type);
+ lan966x_mac_forget(lan966x, mac, mdb_entry->vid, type);
+
+ mdb_entry->ports &= ~BIT(CPU_PORT);
+
+ pgid_entry = lan966x_pgid_entry_get(lan966x, mdb_entry);
+ if (IS_ERR(pgid_entry))
+ return;
+
+ mdb_entry->pgid = pgid_entry;
+
+ lan_rmw(ANA_PGID_PGID_SET(mdb_entry->ports),
+ ANA_PGID_PGID,
+ lan966x, ANA_PGID(pgid_entry->index));
+
+ lan966x_mac_learn(lan966x, pgid_entry->index, mdb_entry->mac,
+ mdb_entry->vid, type);
+}
+
+void lan966x_mdb_erase_entries(struct lan966x *lan966x, u16 vid)
+{
+ struct lan966x_mdb_entry *mdb_entry;
+ enum macaccess_entry_type type;
+
+ list_for_each_entry(mdb_entry, &lan966x->mdb_entries, list) {
+ if (mdb_entry->vid != vid || !mdb_entry->cpu_copy)
+ continue;
+
+ type = lan966x_mdb_classify(mdb_entry->mac);
+ if (type == ENTRYTYPE_MACV4 || type == ENTRYTYPE_MACV6)
+ lan966x_mdb_ip_cpu_remove(lan966x, mdb_entry, type);
+ else
+ lan966x_mdb_l2_cpu_remove(lan966x, mdb_entry, type);
+ }
+}
diff --git a/drivers/net/ethernet/microchip/lan966x/lan966x_phylink.c b/drivers/net/ethernet/microchip/lan966x/lan966x_phylink.c
new file mode 100644
index 0000000..b66a9aa
--- /dev/null
+++ b/drivers/net/ethernet/microchip/lan966x/lan966x_phylink.c
@@ -0,0 +1,127 @@
+// SPDX-License-Identifier: GPL-2.0+
+
+#include <linux/module.h>
+#include <linux/phylink.h>
+#include <linux/device.h>
+#include <linux/netdevice.h>
+#include <linux/phy/phy.h>
+#include <linux/sfp.h>
+
+#include "lan966x_main.h"
+
+static void lan966x_phylink_mac_config(struct phylink_config *config,
+ unsigned int mode,
+ const struct phylink_link_state *state)
+{
+}
+
+static int lan966x_phylink_mac_prepare(struct phylink_config *config,
+ unsigned int mode,
+ phy_interface_t iface)
+{
+ struct lan966x_port *port = netdev_priv(to_net_dev(config->dev));
+ int err;
+
+ if (port->serdes) {
+ err = phy_set_mode_ext(port->serdes, PHY_MODE_ETHERNET,
+ iface);
+ if (err) {
+ netdev_err(to_net_dev(config->dev),
+ "Could not set mode of SerDes\n");
+ return err;
+ }
+ }
+
+ return 0;
+}
+
+static void lan966x_phylink_mac_link_up(struct phylink_config *config,
+ struct phy_device *phy,
+ unsigned int mode,
+ phy_interface_t interface,
+ int speed, int duplex,
+ bool tx_pause, bool rx_pause)
+{
+ struct lan966x_port *port = netdev_priv(to_net_dev(config->dev));
+ struct lan966x_port_config *port_config = &port->config;
+
+ port_config->duplex = duplex;
+ port_config->speed = speed;
+ port_config->pause = 0;
+ port_config->pause |= tx_pause ? MLO_PAUSE_TX : 0;
+ port_config->pause |= rx_pause ? MLO_PAUSE_RX : 0;
+
+ lan966x_port_config_up(port);
+}
+
+static void lan966x_phylink_mac_link_down(struct phylink_config *config,
+ unsigned int mode,
+ phy_interface_t interface)
+{
+ struct lan966x_port *port = netdev_priv(to_net_dev(config->dev));
+ struct lan966x *lan966x = port->lan966x;
+
+ lan966x_port_config_down(port);
+
+ /* Take PCS out of reset */
+ lan_rmw(DEV_CLOCK_CFG_PCS_RX_RST_SET(0) |
+ DEV_CLOCK_CFG_PCS_TX_RST_SET(0),
+ DEV_CLOCK_CFG_PCS_RX_RST |
+ DEV_CLOCK_CFG_PCS_TX_RST,
+ lan966x, DEV_CLOCK_CFG(port->chip_port));
+}
+
+static struct lan966x_port *lan966x_pcs_to_port(struct phylink_pcs *pcs)
+{
+ return container_of(pcs, struct lan966x_port, phylink_pcs);
+}
+
+static void lan966x_pcs_get_state(struct phylink_pcs *pcs,
+ struct phylink_link_state *state)
+{
+ struct lan966x_port *port = lan966x_pcs_to_port(pcs);
+
+ lan966x_port_status_get(port, state);
+}
+
+static int lan966x_pcs_config(struct phylink_pcs *pcs,
+ unsigned int mode,
+ phy_interface_t interface,
+ const unsigned long *advertising,
+ bool permit_pause_to_mac)
+{
+ struct lan966x_port *port = lan966x_pcs_to_port(pcs);
+ struct lan966x_port_config config;
+ int ret;
+
+ config = port->config;
+ config.portmode = interface;
+ config.inband = phylink_autoneg_inband(mode);
+ config.autoneg = phylink_test(advertising, Autoneg);
+ config.advertising = advertising;
+
+ ret = lan966x_port_pcs_set(port, &config);
+ if (ret)
+ netdev_err(port->dev, "port PCS config failed: %d\n", ret);
+
+ return ret;
+}
+
+static void lan966x_pcs_aneg_restart(struct phylink_pcs *pcs)
+{
+ /* Currently not used */
+}
+
+const struct phylink_mac_ops lan966x_phylink_mac_ops = {
+ .validate = phylink_generic_validate,
+ .mac_config = lan966x_phylink_mac_config,
+ .mac_prepare = lan966x_phylink_mac_prepare,
+ .mac_link_down = lan966x_phylink_mac_link_down,
+ .mac_link_up = lan966x_phylink_mac_link_up,
+};
+
+const struct phylink_pcs_ops lan966x_phylink_pcs_ops = {
+ .pcs_get_state = lan966x_pcs_get_state,
+ .pcs_config = lan966x_pcs_config,
+ .pcs_an_restart = lan966x_pcs_aneg_restart,
+};
diff --git a/drivers/net/ethernet/microchip/lan966x/lan966x_port.c b/drivers/net/ethernet/microchip/lan966x/lan966x_port.c
new file mode 100644
index 0000000..2375558
--- /dev/null
+++ b/drivers/net/ethernet/microchip/lan966x/lan966x_port.c
@@ -0,0 +1,406 @@
+// SPDX-License-Identifier: GPL-2.0+
+
+#include <linux/netdevice.h>
+#include <linux/phy/phy.h>
+
+#include "lan966x_main.h"
+
+/* Watermark encode */
+#define MULTIPLIER_BIT BIT(8)
+static u32 lan966x_wm_enc(u32 value)
+{
+ value /= LAN966X_BUFFER_CELL_SZ;
+
+ if (value >= MULTIPLIER_BIT) {
+ value /= 16;
+ if (value >= MULTIPLIER_BIT)
+ value = (MULTIPLIER_BIT - 1);
+
+ value |= MULTIPLIER_BIT;
+ }
+
+ return value;
+}
+
+static void lan966x_port_link_down(struct lan966x_port *port)
+{
+ struct lan966x *lan966x = port->lan966x;
+ u32 val, delay = 0;
+
+ /* 0.5: Disable any AFI */
+ lan_rmw(AFI_PORT_CFG_FC_SKIP_TTI_INJ_SET(1) |
+ AFI_PORT_CFG_FRM_OUT_MAX_SET(0),
+ AFI_PORT_CFG_FC_SKIP_TTI_INJ |
+ AFI_PORT_CFG_FRM_OUT_MAX,
+ lan966x, AFI_PORT_CFG(port->chip_port));
+
+ /* wait for reg afi_port_frm_out to become 0 for the port */
+ while (true) {
+ val = lan_rd(lan966x, AFI_PORT_FRM_OUT(port->chip_port));
+ if (!AFI_PORT_FRM_OUT_FRM_OUT_CNT_GET(val))
+ break;
+
+ usleep_range(USEC_PER_MSEC, 2 * USEC_PER_MSEC);
+ delay++;
+ if (delay == 2000) {
+ pr_err("AFI timeout chip port %u", port->chip_port);
+ break;
+ }
+ }
+
+ delay = 0;
+
+ /* 1: Reset the PCS Rx clock domain */
+ lan_rmw(DEV_CLOCK_CFG_PCS_RX_RST_SET(1),
+ DEV_CLOCK_CFG_PCS_RX_RST,
+ lan966x, DEV_CLOCK_CFG(port->chip_port));
+
+ /* 2: Disable MAC frame reception */
+ lan_rmw(DEV_MAC_ENA_CFG_RX_ENA_SET(0),
+ DEV_MAC_ENA_CFG_RX_ENA,
+ lan966x, DEV_MAC_ENA_CFG(port->chip_port));
+
+ /* 3: Disable traffic being sent to or from switch port */
+ lan_rmw(QSYS_SW_PORT_MODE_PORT_ENA_SET(0),
+ QSYS_SW_PORT_MODE_PORT_ENA,
+ lan966x, QSYS_SW_PORT_MODE(port->chip_port));
+
+ /* 4: Disable dequeuing from the egress queues */
+ lan_rmw(QSYS_PORT_MODE_DEQUEUE_DIS_SET(1),
+ QSYS_PORT_MODE_DEQUEUE_DIS,
+ lan966x, QSYS_PORT_MODE(port->chip_port));
+
+ /* 5: Disable Flowcontrol */
+ lan_rmw(SYS_PAUSE_CFG_PAUSE_ENA_SET(0),
+ SYS_PAUSE_CFG_PAUSE_ENA,
+ lan966x, SYS_PAUSE_CFG(port->chip_port));
+
+ /* 5.1: Disable PFC */
+ lan_rmw(QSYS_SW_PORT_MODE_TX_PFC_ENA_SET(0),
+ QSYS_SW_PORT_MODE_TX_PFC_ENA,
+ lan966x, QSYS_SW_PORT_MODE(port->chip_port));
+
+ /* 6: Wait a worst case time 8ms (jumbo/10Mbit) */
+ usleep_range(8 * USEC_PER_MSEC, 9 * USEC_PER_MSEC);
+
+ /* 7: Disable HDX backpressure */
+ lan_rmw(SYS_FRONT_PORT_MODE_HDX_MODE_SET(0),
+ SYS_FRONT_PORT_MODE_HDX_MODE,
+ lan966x, SYS_FRONT_PORT_MODE(port->chip_port));
+
+ /* 8: Flush the queues accociated with the port */
+ lan_rmw(QSYS_SW_PORT_MODE_AGING_MODE_SET(3),
+ QSYS_SW_PORT_MODE_AGING_MODE,
+ lan966x, QSYS_SW_PORT_MODE(port->chip_port));
+
+ /* 9: Enable dequeuing from the egress queues */
+ lan_rmw(QSYS_PORT_MODE_DEQUEUE_DIS_SET(0),
+ QSYS_PORT_MODE_DEQUEUE_DIS,
+ lan966x, QSYS_PORT_MODE(port->chip_port));
+
+ /* 10: Wait until flushing is complete */
+ while (true) {
+ val = lan_rd(lan966x, QSYS_SW_STATUS(port->chip_port));
+ if (!QSYS_SW_STATUS_EQ_AVAIL_GET(val))
+ break;
+
+ usleep_range(USEC_PER_MSEC, 2 * USEC_PER_MSEC);
+ delay++;
+ if (delay == 2000) {
+ pr_err("Flush timeout chip port %u", port->chip_port);
+ break;
+ }
+ }
+
+ /* 11: Reset the Port and MAC clock domains */
+ lan_rmw(DEV_MAC_ENA_CFG_TX_ENA_SET(0),
+ DEV_MAC_ENA_CFG_TX_ENA,
+ lan966x, DEV_MAC_ENA_CFG(port->chip_port));
+
+ lan_rmw(DEV_CLOCK_CFG_PORT_RST_SET(1),
+ DEV_CLOCK_CFG_PORT_RST,
+ lan966x, DEV_CLOCK_CFG(port->chip_port));
+
+ usleep_range(USEC_PER_MSEC, 2 * USEC_PER_MSEC);
+
+ lan_rmw(DEV_CLOCK_CFG_MAC_TX_RST_SET(1) |
+ DEV_CLOCK_CFG_MAC_RX_RST_SET(1) |
+ DEV_CLOCK_CFG_PORT_RST_SET(1),
+ DEV_CLOCK_CFG_MAC_TX_RST |
+ DEV_CLOCK_CFG_MAC_RX_RST |
+ DEV_CLOCK_CFG_PORT_RST,
+ lan966x, DEV_CLOCK_CFG(port->chip_port));
+
+ /* 12: Clear flushing */
+ lan_rmw(QSYS_SW_PORT_MODE_AGING_MODE_SET(2),
+ QSYS_SW_PORT_MODE_AGING_MODE,
+ lan966x, QSYS_SW_PORT_MODE(port->chip_port));
+
+ /* The port is disabled and flushed, now set up the port in the
+ * new operating mode
+ */
+}
+
+static void lan966x_port_link_up(struct lan966x_port *port)
+{
+ struct lan966x_port_config *config = &port->config;
+ struct lan966x *lan966x = port->lan966x;
+ int speed = 0, mode = 0;
+ int atop_wm = 0;
+
+ switch (config->speed) {
+ case SPEED_10:
+ speed = LAN966X_SPEED_10;
+ break;
+ case SPEED_100:
+ speed = LAN966X_SPEED_100;
+ break;
+ case SPEED_1000:
+ speed = LAN966X_SPEED_1000;
+ mode = DEV_MAC_MODE_CFG_GIGA_MODE_ENA_SET(1);
+ break;
+ case SPEED_2500:
+ speed = LAN966X_SPEED_2500;
+ mode = DEV_MAC_MODE_CFG_GIGA_MODE_ENA_SET(1);
+ break;
+ }
+
+ /* Also the GIGA_MODE_ENA(1) needs to be set regardless of the
+ * port speed for QSGMII ports.
+ */
+ if (config->portmode == PHY_INTERFACE_MODE_QSGMII)
+ mode = DEV_MAC_MODE_CFG_GIGA_MODE_ENA_SET(1);
+
+ lan_wr(config->duplex | mode,
+ lan966x, DEV_MAC_MODE_CFG(port->chip_port));
+
+ lan_rmw(DEV_MAC_IFG_CFG_TX_IFG_SET(config->duplex ? 6 : 5) |
+ DEV_MAC_IFG_CFG_RX_IFG1_SET(config->speed == SPEED_10 ? 2 : 1) |
+ DEV_MAC_IFG_CFG_RX_IFG2_SET(2),
+ DEV_MAC_IFG_CFG_TX_IFG |
+ DEV_MAC_IFG_CFG_RX_IFG1 |
+ DEV_MAC_IFG_CFG_RX_IFG2,
+ lan966x, DEV_MAC_IFG_CFG(port->chip_port));
+
+ lan_rmw(DEV_MAC_HDX_CFG_SEED_SET(4) |
+ DEV_MAC_HDX_CFG_SEED_LOAD_SET(1),
+ DEV_MAC_HDX_CFG_SEED |
+ DEV_MAC_HDX_CFG_SEED_LOAD,
+ lan966x, DEV_MAC_HDX_CFG(port->chip_port));
+
+ if (config->portmode == PHY_INTERFACE_MODE_GMII) {
+ if (config->speed == SPEED_1000)
+ lan_rmw(CHIP_TOP_CUPHY_PORT_CFG_GTX_CLK_ENA_SET(1),
+ CHIP_TOP_CUPHY_PORT_CFG_GTX_CLK_ENA,
+ lan966x,
+ CHIP_TOP_CUPHY_PORT_CFG(port->chip_port));
+ else
+ lan_rmw(CHIP_TOP_CUPHY_PORT_CFG_GTX_CLK_ENA_SET(0),
+ CHIP_TOP_CUPHY_PORT_CFG_GTX_CLK_ENA,
+ lan966x,
+ CHIP_TOP_CUPHY_PORT_CFG(port->chip_port));
+ }
+
+ /* No PFC */
+ lan_wr(ANA_PFC_CFG_FC_LINK_SPEED_SET(speed),
+ lan966x, ANA_PFC_CFG(port->chip_port));
+
+ lan_rmw(DEV_PCS1G_CFG_PCS_ENA_SET(1),
+ DEV_PCS1G_CFG_PCS_ENA,
+ lan966x, DEV_PCS1G_CFG(port->chip_port));
+
+ lan_rmw(DEV_PCS1G_SD_CFG_SD_ENA_SET(0),
+ DEV_PCS1G_SD_CFG_SD_ENA,
+ lan966x, DEV_PCS1G_SD_CFG(port->chip_port));
+
+ /* Set Pause WM hysteresis, start/stop are in 1518 byte units */
+ lan_wr(SYS_PAUSE_CFG_PAUSE_ENA_SET(1) |
+ SYS_PAUSE_CFG_PAUSE_STOP_SET(lan966x_wm_enc(4 * 1518)) |
+ SYS_PAUSE_CFG_PAUSE_START_SET(lan966x_wm_enc(6 * 1518)),
+ lan966x, SYS_PAUSE_CFG(port->chip_port));
+
+ /* Set SMAC of Pause frame (00:00:00:00:00:00) */
+ lan_wr(0, lan966x, DEV_FC_MAC_LOW_CFG(port->chip_port));
+ lan_wr(0, lan966x, DEV_FC_MAC_HIGH_CFG(port->chip_port));
+
+ /* Flow control */
+ lan_rmw(SYS_MAC_FC_CFG_FC_LINK_SPEED_SET(speed) |
+ SYS_MAC_FC_CFG_FC_LATENCY_CFG_SET(7) |
+ SYS_MAC_FC_CFG_ZERO_PAUSE_ENA_SET(1) |
+ SYS_MAC_FC_CFG_PAUSE_VAL_CFG_SET(0xffff) |
+ SYS_MAC_FC_CFG_RX_FC_ENA_SET(config->pause & MLO_PAUSE_RX ? 1 : 0) |
+ SYS_MAC_FC_CFG_TX_FC_ENA_SET(config->pause & MLO_PAUSE_TX ? 1 : 0),
+ SYS_MAC_FC_CFG_FC_LINK_SPEED |
+ SYS_MAC_FC_CFG_FC_LATENCY_CFG |
+ SYS_MAC_FC_CFG_ZERO_PAUSE_ENA |
+ SYS_MAC_FC_CFG_PAUSE_VAL_CFG |
+ SYS_MAC_FC_CFG_RX_FC_ENA |
+ SYS_MAC_FC_CFG_TX_FC_ENA,
+ lan966x, SYS_MAC_FC_CFG(port->chip_port));
+
+ /* Tail dropping watermark */
+ atop_wm = lan966x->shared_queue_sz;
+
+ /* The total memory size is diveded by number of front ports plus CPU
+ * port
+ */
+ lan_wr(lan966x_wm_enc(atop_wm / lan966x->num_phys_ports + 1), lan966x,
+ SYS_ATOP(port->chip_port));
+ lan_wr(lan966x_wm_enc(atop_wm), lan966x, SYS_ATOP_TOT_CFG);
+
+ /* This needs to be at the end */
+ /* Enable MAC module */
+ lan_wr(DEV_MAC_ENA_CFG_RX_ENA_SET(1) |
+ DEV_MAC_ENA_CFG_TX_ENA_SET(1),
+ lan966x, DEV_MAC_ENA_CFG(port->chip_port));
+
+ /* Take out the clock from reset */
+ lan_wr(DEV_CLOCK_CFG_LINK_SPEED_SET(speed),
+ lan966x, DEV_CLOCK_CFG(port->chip_port));
+
+ /* Core: Enable port for frame transfer */
+ lan_wr(QSYS_SW_PORT_MODE_PORT_ENA_SET(1) |
+ QSYS_SW_PORT_MODE_SCH_NEXT_CFG_SET(1) |
+ QSYS_SW_PORT_MODE_INGRESS_DROP_MODE_SET(1),
+ lan966x, QSYS_SW_PORT_MODE(port->chip_port));
+
+ lan_rmw(AFI_PORT_CFG_FC_SKIP_TTI_INJ_SET(0) |
+ AFI_PORT_CFG_FRM_OUT_MAX_SET(16),
+ AFI_PORT_CFG_FC_SKIP_TTI_INJ |
+ AFI_PORT_CFG_FRM_OUT_MAX,
+ lan966x, AFI_PORT_CFG(port->chip_port));
+}
+
+void lan966x_port_config_down(struct lan966x_port *port)
+{
+ lan966x_port_link_down(port);
+}
+
+void lan966x_port_config_up(struct lan966x_port *port)
+{
+ lan966x_port_link_up(port);
+}
+
+void lan966x_port_status_get(struct lan966x_port *port,
+ struct phylink_link_state *state)
+{
+ struct lan966x *lan966x = port->lan966x;
+ bool link_down;
+ u16 bmsr = 0;
+ u16 lp_adv;
+ u32 val;
+
+ val = lan_rd(lan966x, DEV_PCS1G_STICKY(port->chip_port));
+ link_down = DEV_PCS1G_STICKY_LINK_DOWN_STICKY_GET(val);
+ if (link_down)
+ lan_wr(val, lan966x, DEV_PCS1G_STICKY(port->chip_port));
+
+ /* Get both current Link and Sync status */
+ val = lan_rd(lan966x, DEV_PCS1G_LINK_STATUS(port->chip_port));
+ state->link = DEV_PCS1G_LINK_STATUS_LINK_STATUS_GET(val) &&
+ DEV_PCS1G_LINK_STATUS_SYNC_STATUS_GET(val);
+ state->link &= !link_down;
+
+ /* Get PCS ANEG status register */
+ val = lan_rd(lan966x, DEV_PCS1G_ANEG_STATUS(port->chip_port));
+ /* Aneg complete provides more information */
+ if (DEV_PCS1G_ANEG_STATUS_ANEG_COMPLETE_GET(val)) {
+ state->an_complete = true;
+
+ bmsr |= state->link ? BMSR_LSTATUS : 0;
+ bmsr |= BMSR_ANEGCOMPLETE;
+
+ lp_adv = DEV_PCS1G_ANEG_STATUS_LP_ADV_GET(val);
+ phylink_mii_c22_pcs_decode_state(state, bmsr, lp_adv);
+ } else {
+ if (!state->link)
+ return;
+
+ if (state->interface == PHY_INTERFACE_MODE_1000BASEX)
+ state->speed = SPEED_1000;
+ else if (state->interface == PHY_INTERFACE_MODE_2500BASEX)
+ state->speed = SPEED_2500;
+
+ state->duplex = DUPLEX_FULL;
+ }
+}
+
+int lan966x_port_pcs_set(struct lan966x_port *port,
+ struct lan966x_port_config *config)
+{
+ struct lan966x *lan966x = port->lan966x;
+ bool inband_aneg = false;
+ bool outband;
+
+ if (config->inband) {
+ if (config->portmode == PHY_INTERFACE_MODE_SGMII ||
+ config->portmode == PHY_INTERFACE_MODE_QSGMII)
+ inband_aneg = true; /* Cisco-SGMII in-band-aneg */
+ else if (config->portmode == PHY_INTERFACE_MODE_1000BASEX &&
+ config->autoneg)
+ inband_aneg = true; /* Clause-37 in-band-aneg */
+
+ outband = false;
+ } else {
+ outband = true;
+ }
+
+ /* Disable or enable inband */
+ lan_rmw(DEV_PCS1G_MODE_CFG_SGMII_MODE_ENA_SET(outband),
+ DEV_PCS1G_MODE_CFG_SGMII_MODE_ENA,
+ lan966x, DEV_PCS1G_MODE_CFG(port->chip_port));
+
+ /* Enable PCS */
+ lan_wr(DEV_PCS1G_CFG_PCS_ENA_SET(1),
+ lan966x, DEV_PCS1G_CFG(port->chip_port));
+
+ if (inband_aneg) {
+ int adv = phylink_mii_c22_pcs_encode_advertisement(config->portmode,
+ config->advertising);
+ if (adv >= 0)
+ /* Enable in-band aneg */
+ lan_wr(DEV_PCS1G_ANEG_CFG_ADV_ABILITY_SET(adv) |
+ DEV_PCS1G_ANEG_CFG_SW_RESOLVE_ENA_SET(1) |
+ DEV_PCS1G_ANEG_CFG_ENA_SET(1) |
+ DEV_PCS1G_ANEG_CFG_RESTART_ONE_SHOT_SET(1),
+ lan966x, DEV_PCS1G_ANEG_CFG(port->chip_port));
+ } else {
+ lan_wr(0, lan966x, DEV_PCS1G_ANEG_CFG(port->chip_port));
+ }
+
+ /* Take PCS out of reset */
+ lan_rmw(DEV_CLOCK_CFG_LINK_SPEED_SET(2) |
+ DEV_CLOCK_CFG_PCS_RX_RST_SET(0) |
+ DEV_CLOCK_CFG_PCS_TX_RST_SET(0),
+ DEV_CLOCK_CFG_LINK_SPEED |
+ DEV_CLOCK_CFG_PCS_RX_RST |
+ DEV_CLOCK_CFG_PCS_TX_RST,
+ lan966x, DEV_CLOCK_CFG(port->chip_port));
+
+ port->config = *config;
+
+ return 0;
+}
+
+void lan966x_port_init(struct lan966x_port *port)
+{
+ struct lan966x_port_config *config = &port->config;
+ struct lan966x *lan966x = port->lan966x;
+
+ lan_rmw(ANA_PORT_CFG_LEARN_ENA_SET(0),
+ ANA_PORT_CFG_LEARN_ENA,
+ lan966x, ANA_PORT_CFG(port->chip_port));
+
+ lan966x_port_config_down(port);
+
+ if (config->portmode != PHY_INTERFACE_MODE_QSGMII)
+ return;
+
+ lan_rmw(DEV_CLOCK_CFG_PCS_RX_RST_SET(0) |
+ DEV_CLOCK_CFG_PCS_TX_RST_SET(0) |
+ DEV_CLOCK_CFG_LINK_SPEED_SET(LAN966X_SPEED_1000),
+ DEV_CLOCK_CFG_PCS_RX_RST |
+ DEV_CLOCK_CFG_PCS_TX_RST |
+ DEV_CLOCK_CFG_LINK_SPEED,
+ lan966x, DEV_CLOCK_CFG(port->chip_port));
+}
diff --git a/drivers/net/ethernet/microchip/lan966x/lan966x_regs.h b/drivers/net/ethernet/microchip/lan966x/lan966x_regs.h
new file mode 100644
index 0000000..7975601
--- /dev/null
+++ b/drivers/net/ethernet/microchip/lan966x/lan966x_regs.h
@@ -0,0 +1,871 @@
+/* SPDX-License-Identifier: (GPL-2.0 OR MIT) */
+
+/* This file is autogenerated by cml-utils 2021-10-10 13:25:08 +0200.
+ * Commit ID: 26db2002924973d36a30b369c94f025a678fe9ea (dirty)
+ */
+
+#ifndef _LAN966X_REGS_H_
+#define _LAN966X_REGS_H_
+
+#include <linux/bitfield.h>
+#include <linux/types.h>
+#include <linux/bug.h>
+
+enum lan966x_target {
+ TARGET_AFI = 2,
+ TARGET_ANA = 3,
+ TARGET_CHIP_TOP = 5,
+ TARGET_CPU = 6,
+ TARGET_DEV = 13,
+ TARGET_GCB = 27,
+ TARGET_ORG = 36,
+ TARGET_QS = 42,
+ TARGET_QSYS = 46,
+ TARGET_REW = 47,
+ TARGET_SYS = 52,
+ NUM_TARGETS = 66
+};
+
+#define __REG(...) __VA_ARGS__
+
+/* AFI:PORT_TBL:PORT_FRM_OUT */
+#define AFI_PORT_FRM_OUT(g) __REG(TARGET_AFI, 0, 1, 98816, g, 10, 8, 0, 0, 1, 4)
+
+#define AFI_PORT_FRM_OUT_FRM_OUT_CNT GENMASK(26, 16)
+#define AFI_PORT_FRM_OUT_FRM_OUT_CNT_SET(x)\
+ FIELD_PREP(AFI_PORT_FRM_OUT_FRM_OUT_CNT, x)
+#define AFI_PORT_FRM_OUT_FRM_OUT_CNT_GET(x)\
+ FIELD_GET(AFI_PORT_FRM_OUT_FRM_OUT_CNT, x)
+
+/* AFI:PORT_TBL:PORT_CFG */
+#define AFI_PORT_CFG(g) __REG(TARGET_AFI, 0, 1, 98816, g, 10, 8, 4, 0, 1, 4)
+
+#define AFI_PORT_CFG_FC_SKIP_TTI_INJ BIT(16)
+#define AFI_PORT_CFG_FC_SKIP_TTI_INJ_SET(x)\
+ FIELD_PREP(AFI_PORT_CFG_FC_SKIP_TTI_INJ, x)
+#define AFI_PORT_CFG_FC_SKIP_TTI_INJ_GET(x)\
+ FIELD_GET(AFI_PORT_CFG_FC_SKIP_TTI_INJ, x)
+
+#define AFI_PORT_CFG_FRM_OUT_MAX GENMASK(9, 0)
+#define AFI_PORT_CFG_FRM_OUT_MAX_SET(x)\
+ FIELD_PREP(AFI_PORT_CFG_FRM_OUT_MAX, x)
+#define AFI_PORT_CFG_FRM_OUT_MAX_GET(x)\
+ FIELD_GET(AFI_PORT_CFG_FRM_OUT_MAX, x)
+
+/* ANA:ANA:ADVLEARN */
+#define ANA_ADVLEARN __REG(TARGET_ANA, 0, 1, 29824, 0, 1, 244, 0, 0, 1, 4)
+
+#define ANA_ADVLEARN_VLAN_CHK BIT(0)
+#define ANA_ADVLEARN_VLAN_CHK_SET(x)\
+ FIELD_PREP(ANA_ADVLEARN_VLAN_CHK, x)
+#define ANA_ADVLEARN_VLAN_CHK_GET(x)\
+ FIELD_GET(ANA_ADVLEARN_VLAN_CHK, x)
+
+/* ANA:ANA:VLANMASK */
+#define ANA_VLANMASK __REG(TARGET_ANA, 0, 1, 29824, 0, 1, 244, 8, 0, 1, 4)
+
+/* ANA:ANA:ANAINTR */
+#define ANA_ANAINTR __REG(TARGET_ANA, 0, 1, 29824, 0, 1, 244, 16, 0, 1, 4)
+
+#define ANA_ANAINTR_INTR BIT(1)
+#define ANA_ANAINTR_INTR_SET(x)\
+ FIELD_PREP(ANA_ANAINTR_INTR, x)
+#define ANA_ANAINTR_INTR_GET(x)\
+ FIELD_GET(ANA_ANAINTR_INTR, x)
+
+#define ANA_ANAINTR_INTR_ENA BIT(0)
+#define ANA_ANAINTR_INTR_ENA_SET(x)\
+ FIELD_PREP(ANA_ANAINTR_INTR_ENA, x)
+#define ANA_ANAINTR_INTR_ENA_GET(x)\
+ FIELD_GET(ANA_ANAINTR_INTR_ENA, x)
+
+/* ANA:ANA:AUTOAGE */
+#define ANA_AUTOAGE __REG(TARGET_ANA, 0, 1, 29824, 0, 1, 244, 44, 0, 1, 4)
+
+#define ANA_AUTOAGE_AGE_PERIOD GENMASK(20, 1)
+#define ANA_AUTOAGE_AGE_PERIOD_SET(x)\
+ FIELD_PREP(ANA_AUTOAGE_AGE_PERIOD, x)
+#define ANA_AUTOAGE_AGE_PERIOD_GET(x)\
+ FIELD_GET(ANA_AUTOAGE_AGE_PERIOD, x)
+
+/* ANA:ANA:FLOODING */
+#define ANA_FLOODING(r) __REG(TARGET_ANA, 0, 1, 29824, 0, 1, 244, 68, r, 8, 4)
+
+#define ANA_FLOODING_FLD_UNICAST GENMASK(17, 12)
+#define ANA_FLOODING_FLD_UNICAST_SET(x)\
+ FIELD_PREP(ANA_FLOODING_FLD_UNICAST, x)
+#define ANA_FLOODING_FLD_UNICAST_GET(x)\
+ FIELD_GET(ANA_FLOODING_FLD_UNICAST, x)
+
+#define ANA_FLOODING_FLD_BROADCAST GENMASK(11, 6)
+#define ANA_FLOODING_FLD_BROADCAST_SET(x)\
+ FIELD_PREP(ANA_FLOODING_FLD_BROADCAST, x)
+#define ANA_FLOODING_FLD_BROADCAST_GET(x)\
+ FIELD_GET(ANA_FLOODING_FLD_BROADCAST, x)
+
+#define ANA_FLOODING_FLD_MULTICAST GENMASK(5, 0)
+#define ANA_FLOODING_FLD_MULTICAST_SET(x)\
+ FIELD_PREP(ANA_FLOODING_FLD_MULTICAST, x)
+#define ANA_FLOODING_FLD_MULTICAST_GET(x)\
+ FIELD_GET(ANA_FLOODING_FLD_MULTICAST, x)
+
+/* ANA:ANA:FLOODING_IPMC */
+#define ANA_FLOODING_IPMC __REG(TARGET_ANA, 0, 1, 29824, 0, 1, 244, 100, 0, 1, 4)
+
+#define ANA_FLOODING_IPMC_FLD_MC4_CTRL GENMASK(23, 18)
+#define ANA_FLOODING_IPMC_FLD_MC4_CTRL_SET(x)\
+ FIELD_PREP(ANA_FLOODING_IPMC_FLD_MC4_CTRL, x)
+#define ANA_FLOODING_IPMC_FLD_MC4_CTRL_GET(x)\
+ FIELD_GET(ANA_FLOODING_IPMC_FLD_MC4_CTRL, x)
+
+#define ANA_FLOODING_IPMC_FLD_MC4_DATA GENMASK(17, 12)
+#define ANA_FLOODING_IPMC_FLD_MC4_DATA_SET(x)\
+ FIELD_PREP(ANA_FLOODING_IPMC_FLD_MC4_DATA, x)
+#define ANA_FLOODING_IPMC_FLD_MC4_DATA_GET(x)\
+ FIELD_GET(ANA_FLOODING_IPMC_FLD_MC4_DATA, x)
+
+#define ANA_FLOODING_IPMC_FLD_MC6_CTRL GENMASK(11, 6)
+#define ANA_FLOODING_IPMC_FLD_MC6_CTRL_SET(x)\
+ FIELD_PREP(ANA_FLOODING_IPMC_FLD_MC6_CTRL, x)
+#define ANA_FLOODING_IPMC_FLD_MC6_CTRL_GET(x)\
+ FIELD_GET(ANA_FLOODING_IPMC_FLD_MC6_CTRL, x)
+
+#define ANA_FLOODING_IPMC_FLD_MC6_DATA GENMASK(5, 0)
+#define ANA_FLOODING_IPMC_FLD_MC6_DATA_SET(x)\
+ FIELD_PREP(ANA_FLOODING_IPMC_FLD_MC6_DATA, x)
+#define ANA_FLOODING_IPMC_FLD_MC6_DATA_GET(x)\
+ FIELD_GET(ANA_FLOODING_IPMC_FLD_MC6_DATA, x)
+
+/* ANA:PGID:PGID */
+#define ANA_PGID(g) __REG(TARGET_ANA, 0, 1, 27648, g, 89, 8, 0, 0, 1, 4)
+
+#define ANA_PGID_PGID GENMASK(8, 0)
+#define ANA_PGID_PGID_SET(x)\
+ FIELD_PREP(ANA_PGID_PGID, x)
+#define ANA_PGID_PGID_GET(x)\
+ FIELD_GET(ANA_PGID_PGID, x)
+
+/* ANA:PGID:PGID_CFG */
+#define ANA_PGID_CFG(g) __REG(TARGET_ANA, 0, 1, 27648, g, 89, 8, 4, 0, 1, 4)
+
+#define ANA_PGID_CFG_OBEY_VLAN BIT(0)
+#define ANA_PGID_CFG_OBEY_VLAN_SET(x)\
+ FIELD_PREP(ANA_PGID_CFG_OBEY_VLAN, x)
+#define ANA_PGID_CFG_OBEY_VLAN_GET(x)\
+ FIELD_GET(ANA_PGID_CFG_OBEY_VLAN, x)
+
+/* ANA:ANA_TABLES:MACHDATA */
+#define ANA_MACHDATA __REG(TARGET_ANA, 0, 1, 27520, 0, 1, 128, 40, 0, 1, 4)
+
+/* ANA:ANA_TABLES:MACLDATA */
+#define ANA_MACLDATA __REG(TARGET_ANA, 0, 1, 27520, 0, 1, 128, 44, 0, 1, 4)
+
+/* ANA:ANA_TABLES:MACACCESS */
+#define ANA_MACACCESS __REG(TARGET_ANA, 0, 1, 27520, 0, 1, 128, 48, 0, 1, 4)
+
+#define ANA_MACACCESS_CHANGE2SW BIT(17)
+#define ANA_MACACCESS_CHANGE2SW_SET(x)\
+ FIELD_PREP(ANA_MACACCESS_CHANGE2SW, x)
+#define ANA_MACACCESS_CHANGE2SW_GET(x)\
+ FIELD_GET(ANA_MACACCESS_CHANGE2SW, x)
+
+#define ANA_MACACCESS_MAC_CPU_COPY BIT(16)
+#define ANA_MACACCESS_MAC_CPU_COPY_SET(x)\
+ FIELD_PREP(ANA_MACACCESS_MAC_CPU_COPY, x)
+#define ANA_MACACCESS_MAC_CPU_COPY_GET(x)\
+ FIELD_GET(ANA_MACACCESS_MAC_CPU_COPY, x)
+
+#define ANA_MACACCESS_VALID BIT(12)
+#define ANA_MACACCESS_VALID_SET(x)\
+ FIELD_PREP(ANA_MACACCESS_VALID, x)
+#define ANA_MACACCESS_VALID_GET(x)\
+ FIELD_GET(ANA_MACACCESS_VALID, x)
+
+#define ANA_MACACCESS_ENTRYTYPE GENMASK(11, 10)
+#define ANA_MACACCESS_ENTRYTYPE_SET(x)\
+ FIELD_PREP(ANA_MACACCESS_ENTRYTYPE, x)
+#define ANA_MACACCESS_ENTRYTYPE_GET(x)\
+ FIELD_GET(ANA_MACACCESS_ENTRYTYPE, x)
+
+#define ANA_MACACCESS_DEST_IDX GENMASK(9, 4)
+#define ANA_MACACCESS_DEST_IDX_SET(x)\
+ FIELD_PREP(ANA_MACACCESS_DEST_IDX, x)
+#define ANA_MACACCESS_DEST_IDX_GET(x)\
+ FIELD_GET(ANA_MACACCESS_DEST_IDX, x)
+
+#define ANA_MACACCESS_MAC_TABLE_CMD GENMASK(3, 0)
+#define ANA_MACACCESS_MAC_TABLE_CMD_SET(x)\
+ FIELD_PREP(ANA_MACACCESS_MAC_TABLE_CMD, x)
+#define ANA_MACACCESS_MAC_TABLE_CMD_GET(x)\
+ FIELD_GET(ANA_MACACCESS_MAC_TABLE_CMD, x)
+
+/* ANA:ANA_TABLES:MACTINDX */
+#define ANA_MACTINDX __REG(TARGET_ANA, 0, 1, 27520, 0, 1, 128, 52, 0, 1, 4)
+
+#define ANA_MACTINDX_BUCKET GENMASK(12, 11)
+#define ANA_MACTINDX_BUCKET_SET(x)\
+ FIELD_PREP(ANA_MACTINDX_BUCKET, x)
+#define ANA_MACTINDX_BUCKET_GET(x)\
+ FIELD_GET(ANA_MACTINDX_BUCKET, x)
+
+#define ANA_MACTINDX_M_INDEX GENMASK(10, 0)
+#define ANA_MACTINDX_M_INDEX_SET(x)\
+ FIELD_PREP(ANA_MACTINDX_M_INDEX, x)
+#define ANA_MACTINDX_M_INDEX_GET(x)\
+ FIELD_GET(ANA_MACTINDX_M_INDEX, x)
+
+/* ANA:ANA_TABLES:VLAN_PORT_MASK */
+#define ANA_VLAN_PORT_MASK __REG(TARGET_ANA, 0, 1, 27520, 0, 1, 128, 56, 0, 1, 4)
+
+#define ANA_VLAN_PORT_MASK_VLAN_PORT_MASK GENMASK(8, 0)
+#define ANA_VLAN_PORT_MASK_VLAN_PORT_MASK_SET(x)\
+ FIELD_PREP(ANA_VLAN_PORT_MASK_VLAN_PORT_MASK, x)
+#define ANA_VLAN_PORT_MASK_VLAN_PORT_MASK_GET(x)\
+ FIELD_GET(ANA_VLAN_PORT_MASK_VLAN_PORT_MASK, x)
+
+/* ANA:ANA_TABLES:VLANACCESS */
+#define ANA_VLANACCESS __REG(TARGET_ANA, 0, 1, 27520, 0, 1, 128, 60, 0, 1, 4)
+
+#define ANA_VLANACCESS_VLAN_TBL_CMD GENMASK(1, 0)
+#define ANA_VLANACCESS_VLAN_TBL_CMD_SET(x)\
+ FIELD_PREP(ANA_VLANACCESS_VLAN_TBL_CMD, x)
+#define ANA_VLANACCESS_VLAN_TBL_CMD_GET(x)\
+ FIELD_GET(ANA_VLANACCESS_VLAN_TBL_CMD, x)
+
+/* ANA:ANA_TABLES:VLANTIDX */
+#define ANA_VLANTIDX __REG(TARGET_ANA, 0, 1, 27520, 0, 1, 128, 64, 0, 1, 4)
+
+#define ANA_VLANTIDX_VLAN_PGID_CPU_DIS BIT(18)
+#define ANA_VLANTIDX_VLAN_PGID_CPU_DIS_SET(x)\
+ FIELD_PREP(ANA_VLANTIDX_VLAN_PGID_CPU_DIS, x)
+#define ANA_VLANTIDX_VLAN_PGID_CPU_DIS_GET(x)\
+ FIELD_GET(ANA_VLANTIDX_VLAN_PGID_CPU_DIS, x)
+
+#define ANA_VLANTIDX_V_INDEX GENMASK(11, 0)
+#define ANA_VLANTIDX_V_INDEX_SET(x)\
+ FIELD_PREP(ANA_VLANTIDX_V_INDEX, x)
+#define ANA_VLANTIDX_V_INDEX_GET(x)\
+ FIELD_GET(ANA_VLANTIDX_V_INDEX, x)
+
+/* ANA:PORT:VLAN_CFG */
+#define ANA_VLAN_CFG(g) __REG(TARGET_ANA, 0, 1, 28672, g, 9, 128, 0, 0, 1, 4)
+
+#define ANA_VLAN_CFG_VLAN_AWARE_ENA BIT(20)
+#define ANA_VLAN_CFG_VLAN_AWARE_ENA_SET(x)\
+ FIELD_PREP(ANA_VLAN_CFG_VLAN_AWARE_ENA, x)
+#define ANA_VLAN_CFG_VLAN_AWARE_ENA_GET(x)\
+ FIELD_GET(ANA_VLAN_CFG_VLAN_AWARE_ENA, x)
+
+#define ANA_VLAN_CFG_VLAN_POP_CNT GENMASK(19, 18)
+#define ANA_VLAN_CFG_VLAN_POP_CNT_SET(x)\
+ FIELD_PREP(ANA_VLAN_CFG_VLAN_POP_CNT, x)
+#define ANA_VLAN_CFG_VLAN_POP_CNT_GET(x)\
+ FIELD_GET(ANA_VLAN_CFG_VLAN_POP_CNT, x)
+
+#define ANA_VLAN_CFG_VLAN_VID GENMASK(11, 0)
+#define ANA_VLAN_CFG_VLAN_VID_SET(x)\
+ FIELD_PREP(ANA_VLAN_CFG_VLAN_VID, x)
+#define ANA_VLAN_CFG_VLAN_VID_GET(x)\
+ FIELD_GET(ANA_VLAN_CFG_VLAN_VID, x)
+
+/* ANA:PORT:DROP_CFG */
+#define ANA_DROP_CFG(g) __REG(TARGET_ANA, 0, 1, 28672, g, 9, 128, 4, 0, 1, 4)
+
+#define ANA_DROP_CFG_DROP_UNTAGGED_ENA BIT(6)
+#define ANA_DROP_CFG_DROP_UNTAGGED_ENA_SET(x)\
+ FIELD_PREP(ANA_DROP_CFG_DROP_UNTAGGED_ENA, x)
+#define ANA_DROP_CFG_DROP_UNTAGGED_ENA_GET(x)\
+ FIELD_GET(ANA_DROP_CFG_DROP_UNTAGGED_ENA, x)
+
+#define ANA_DROP_CFG_DROP_PRIO_S_TAGGED_ENA BIT(3)
+#define ANA_DROP_CFG_DROP_PRIO_S_TAGGED_ENA_SET(x)\
+ FIELD_PREP(ANA_DROP_CFG_DROP_PRIO_S_TAGGED_ENA, x)
+#define ANA_DROP_CFG_DROP_PRIO_S_TAGGED_ENA_GET(x)\
+ FIELD_GET(ANA_DROP_CFG_DROP_PRIO_S_TAGGED_ENA, x)
+
+#define ANA_DROP_CFG_DROP_PRIO_C_TAGGED_ENA BIT(2)
+#define ANA_DROP_CFG_DROP_PRIO_C_TAGGED_ENA_SET(x)\
+ FIELD_PREP(ANA_DROP_CFG_DROP_PRIO_C_TAGGED_ENA, x)
+#define ANA_DROP_CFG_DROP_PRIO_C_TAGGED_ENA_GET(x)\
+ FIELD_GET(ANA_DROP_CFG_DROP_PRIO_C_TAGGED_ENA, x)
+
+#define ANA_DROP_CFG_DROP_MC_SMAC_ENA BIT(0)
+#define ANA_DROP_CFG_DROP_MC_SMAC_ENA_SET(x)\
+ FIELD_PREP(ANA_DROP_CFG_DROP_MC_SMAC_ENA, x)
+#define ANA_DROP_CFG_DROP_MC_SMAC_ENA_GET(x)\
+ FIELD_GET(ANA_DROP_CFG_DROP_MC_SMAC_ENA, x)
+
+/* ANA:PORT:CPU_FWD_CFG */
+#define ANA_CPU_FWD_CFG(g) __REG(TARGET_ANA, 0, 1, 28672, g, 9, 128, 96, 0, 1, 4)
+
+#define ANA_CPU_FWD_CFG_SRC_COPY_ENA BIT(3)
+#define ANA_CPU_FWD_CFG_SRC_COPY_ENA_SET(x)\
+ FIELD_PREP(ANA_CPU_FWD_CFG_SRC_COPY_ENA, x)
+#define ANA_CPU_FWD_CFG_SRC_COPY_ENA_GET(x)\
+ FIELD_GET(ANA_CPU_FWD_CFG_SRC_COPY_ENA, x)
+
+/* ANA:PORT:CPU_FWD_BPDU_CFG */
+#define ANA_CPU_FWD_BPDU_CFG(g) __REG(TARGET_ANA, 0, 1, 28672, g, 9, 128, 100, 0, 1, 4)
+
+/* ANA:PORT:PORT_CFG */
+#define ANA_PORT_CFG(g) __REG(TARGET_ANA, 0, 1, 28672, g, 9, 128, 112, 0, 1, 4)
+
+#define ANA_PORT_CFG_LEARNAUTO BIT(6)
+#define ANA_PORT_CFG_LEARNAUTO_SET(x)\
+ FIELD_PREP(ANA_PORT_CFG_LEARNAUTO, x)
+#define ANA_PORT_CFG_LEARNAUTO_GET(x)\
+ FIELD_GET(ANA_PORT_CFG_LEARNAUTO, x)
+
+#define ANA_PORT_CFG_LEARN_ENA BIT(5)
+#define ANA_PORT_CFG_LEARN_ENA_SET(x)\
+ FIELD_PREP(ANA_PORT_CFG_LEARN_ENA, x)
+#define ANA_PORT_CFG_LEARN_ENA_GET(x)\
+ FIELD_GET(ANA_PORT_CFG_LEARN_ENA, x)
+
+#define ANA_PORT_CFG_RECV_ENA BIT(4)
+#define ANA_PORT_CFG_RECV_ENA_SET(x)\
+ FIELD_PREP(ANA_PORT_CFG_RECV_ENA, x)
+#define ANA_PORT_CFG_RECV_ENA_GET(x)\
+ FIELD_GET(ANA_PORT_CFG_RECV_ENA, x)
+
+#define ANA_PORT_CFG_PORTID_VAL GENMASK(3, 0)
+#define ANA_PORT_CFG_PORTID_VAL_SET(x)\
+ FIELD_PREP(ANA_PORT_CFG_PORTID_VAL, x)
+#define ANA_PORT_CFG_PORTID_VAL_GET(x)\
+ FIELD_GET(ANA_PORT_CFG_PORTID_VAL, x)
+
+/* ANA:PFC:PFC_CFG */
+#define ANA_PFC_CFG(g) __REG(TARGET_ANA, 0, 1, 30720, g, 8, 64, 0, 0, 1, 4)
+
+#define ANA_PFC_CFG_FC_LINK_SPEED GENMASK(1, 0)
+#define ANA_PFC_CFG_FC_LINK_SPEED_SET(x)\
+ FIELD_PREP(ANA_PFC_CFG_FC_LINK_SPEED, x)
+#define ANA_PFC_CFG_FC_LINK_SPEED_GET(x)\
+ FIELD_GET(ANA_PFC_CFG_FC_LINK_SPEED, x)
+
+/* CHIP_TOP:CUPHY_CFG:CUPHY_PORT_CFG */
+#define CHIP_TOP_CUPHY_PORT_CFG(r) __REG(TARGET_CHIP_TOP, 0, 1, 16, 0, 1, 20, 8, r, 2, 4)
+
+#define CHIP_TOP_CUPHY_PORT_CFG_GTX_CLK_ENA BIT(0)
+#define CHIP_TOP_CUPHY_PORT_CFG_GTX_CLK_ENA_SET(x)\
+ FIELD_PREP(CHIP_TOP_CUPHY_PORT_CFG_GTX_CLK_ENA, x)
+#define CHIP_TOP_CUPHY_PORT_CFG_GTX_CLK_ENA_GET(x)\
+ FIELD_GET(CHIP_TOP_CUPHY_PORT_CFG_GTX_CLK_ENA, x)
+
+/* DEV:PORT_MODE:CLOCK_CFG */
+#define DEV_CLOCK_CFG(t) __REG(TARGET_DEV, t, 8, 0, 0, 1, 28, 0, 0, 1, 4)
+
+#define DEV_CLOCK_CFG_MAC_TX_RST BIT(7)
+#define DEV_CLOCK_CFG_MAC_TX_RST_SET(x)\
+ FIELD_PREP(DEV_CLOCK_CFG_MAC_TX_RST, x)
+#define DEV_CLOCK_CFG_MAC_TX_RST_GET(x)\
+ FIELD_GET(DEV_CLOCK_CFG_MAC_TX_RST, x)
+
+#define DEV_CLOCK_CFG_MAC_RX_RST BIT(6)
+#define DEV_CLOCK_CFG_MAC_RX_RST_SET(x)\
+ FIELD_PREP(DEV_CLOCK_CFG_MAC_RX_RST, x)
+#define DEV_CLOCK_CFG_MAC_RX_RST_GET(x)\
+ FIELD_GET(DEV_CLOCK_CFG_MAC_RX_RST, x)
+
+#define DEV_CLOCK_CFG_PCS_TX_RST BIT(5)
+#define DEV_CLOCK_CFG_PCS_TX_RST_SET(x)\
+ FIELD_PREP(DEV_CLOCK_CFG_PCS_TX_RST, x)
+#define DEV_CLOCK_CFG_PCS_TX_RST_GET(x)\
+ FIELD_GET(DEV_CLOCK_CFG_PCS_TX_RST, x)
+
+#define DEV_CLOCK_CFG_PCS_RX_RST BIT(4)
+#define DEV_CLOCK_CFG_PCS_RX_RST_SET(x)\
+ FIELD_PREP(DEV_CLOCK_CFG_PCS_RX_RST, x)
+#define DEV_CLOCK_CFG_PCS_RX_RST_GET(x)\
+ FIELD_GET(DEV_CLOCK_CFG_PCS_RX_RST, x)
+
+#define DEV_CLOCK_CFG_PORT_RST BIT(3)
+#define DEV_CLOCK_CFG_PORT_RST_SET(x)\
+ FIELD_PREP(DEV_CLOCK_CFG_PORT_RST, x)
+#define DEV_CLOCK_CFG_PORT_RST_GET(x)\
+ FIELD_GET(DEV_CLOCK_CFG_PORT_RST, x)
+
+#define DEV_CLOCK_CFG_LINK_SPEED GENMASK(1, 0)
+#define DEV_CLOCK_CFG_LINK_SPEED_SET(x)\
+ FIELD_PREP(DEV_CLOCK_CFG_LINK_SPEED, x)
+#define DEV_CLOCK_CFG_LINK_SPEED_GET(x)\
+ FIELD_GET(DEV_CLOCK_CFG_LINK_SPEED, x)
+
+/* DEV:MAC_CFG_STATUS:MAC_ENA_CFG */
+#define DEV_MAC_ENA_CFG(t) __REG(TARGET_DEV, t, 8, 28, 0, 1, 44, 0, 0, 1, 4)
+
+#define DEV_MAC_ENA_CFG_RX_ENA BIT(4)
+#define DEV_MAC_ENA_CFG_RX_ENA_SET(x)\
+ FIELD_PREP(DEV_MAC_ENA_CFG_RX_ENA, x)
+#define DEV_MAC_ENA_CFG_RX_ENA_GET(x)\
+ FIELD_GET(DEV_MAC_ENA_CFG_RX_ENA, x)
+
+#define DEV_MAC_ENA_CFG_TX_ENA BIT(0)
+#define DEV_MAC_ENA_CFG_TX_ENA_SET(x)\
+ FIELD_PREP(DEV_MAC_ENA_CFG_TX_ENA, x)
+#define DEV_MAC_ENA_CFG_TX_ENA_GET(x)\
+ FIELD_GET(DEV_MAC_ENA_CFG_TX_ENA, x)
+
+/* DEV:MAC_CFG_STATUS:MAC_MODE_CFG */
+#define DEV_MAC_MODE_CFG(t) __REG(TARGET_DEV, t, 8, 28, 0, 1, 44, 4, 0, 1, 4)
+
+#define DEV_MAC_MODE_CFG_GIGA_MODE_ENA BIT(4)
+#define DEV_MAC_MODE_CFG_GIGA_MODE_ENA_SET(x)\
+ FIELD_PREP(DEV_MAC_MODE_CFG_GIGA_MODE_ENA, x)
+#define DEV_MAC_MODE_CFG_GIGA_MODE_ENA_GET(x)\
+ FIELD_GET(DEV_MAC_MODE_CFG_GIGA_MODE_ENA, x)
+
+/* DEV:MAC_CFG_STATUS:MAC_MAXLEN_CFG */
+#define DEV_MAC_MAXLEN_CFG(t) __REG(TARGET_DEV, t, 8, 28, 0, 1, 44, 8, 0, 1, 4)
+
+#define DEV_MAC_MAXLEN_CFG_MAX_LEN GENMASK(15, 0)
+#define DEV_MAC_MAXLEN_CFG_MAX_LEN_SET(x)\
+ FIELD_PREP(DEV_MAC_MAXLEN_CFG_MAX_LEN, x)
+#define DEV_MAC_MAXLEN_CFG_MAX_LEN_GET(x)\
+ FIELD_GET(DEV_MAC_MAXLEN_CFG_MAX_LEN, x)
+
+/* DEV:MAC_CFG_STATUS:MAC_IFG_CFG */
+#define DEV_MAC_IFG_CFG(t) __REG(TARGET_DEV, t, 8, 28, 0, 1, 44, 20, 0, 1, 4)
+
+#define DEV_MAC_IFG_CFG_TX_IFG GENMASK(12, 8)
+#define DEV_MAC_IFG_CFG_TX_IFG_SET(x)\
+ FIELD_PREP(DEV_MAC_IFG_CFG_TX_IFG, x)
+#define DEV_MAC_IFG_CFG_TX_IFG_GET(x)\
+ FIELD_GET(DEV_MAC_IFG_CFG_TX_IFG, x)
+
+#define DEV_MAC_IFG_CFG_RX_IFG2 GENMASK(7, 4)
+#define DEV_MAC_IFG_CFG_RX_IFG2_SET(x)\
+ FIELD_PREP(DEV_MAC_IFG_CFG_RX_IFG2, x)
+#define DEV_MAC_IFG_CFG_RX_IFG2_GET(x)\
+ FIELD_GET(DEV_MAC_IFG_CFG_RX_IFG2, x)
+
+#define DEV_MAC_IFG_CFG_RX_IFG1 GENMASK(3, 0)
+#define DEV_MAC_IFG_CFG_RX_IFG1_SET(x)\
+ FIELD_PREP(DEV_MAC_IFG_CFG_RX_IFG1, x)
+#define DEV_MAC_IFG_CFG_RX_IFG1_GET(x)\
+ FIELD_GET(DEV_MAC_IFG_CFG_RX_IFG1, x)
+
+/* DEV:MAC_CFG_STATUS:MAC_HDX_CFG */
+#define DEV_MAC_HDX_CFG(t) __REG(TARGET_DEV, t, 8, 28, 0, 1, 44, 24, 0, 1, 4)
+
+#define DEV_MAC_HDX_CFG_SEED GENMASK(23, 16)
+#define DEV_MAC_HDX_CFG_SEED_SET(x)\
+ FIELD_PREP(DEV_MAC_HDX_CFG_SEED, x)
+#define DEV_MAC_HDX_CFG_SEED_GET(x)\
+ FIELD_GET(DEV_MAC_HDX_CFG_SEED, x)
+
+#define DEV_MAC_HDX_CFG_SEED_LOAD BIT(12)
+#define DEV_MAC_HDX_CFG_SEED_LOAD_SET(x)\
+ FIELD_PREP(DEV_MAC_HDX_CFG_SEED_LOAD, x)
+#define DEV_MAC_HDX_CFG_SEED_LOAD_GET(x)\
+ FIELD_GET(DEV_MAC_HDX_CFG_SEED_LOAD, x)
+
+/* DEV:MAC_CFG_STATUS:MAC_FC_MAC_LOW_CFG */
+#define DEV_FC_MAC_LOW_CFG(t) __REG(TARGET_DEV, t, 8, 28, 0, 1, 44, 32, 0, 1, 4)
+
+/* DEV:MAC_CFG_STATUS:MAC_FC_MAC_HIGH_CFG */
+#define DEV_FC_MAC_HIGH_CFG(t) __REG(TARGET_DEV, t, 8, 28, 0, 1, 44, 36, 0, 1, 4)
+
+/* DEV:PCS1G_CFG_STATUS:PCS1G_CFG */
+#define DEV_PCS1G_CFG(t) __REG(TARGET_DEV, t, 8, 72, 0, 1, 68, 0, 0, 1, 4)
+
+#define DEV_PCS1G_CFG_PCS_ENA BIT(0)
+#define DEV_PCS1G_CFG_PCS_ENA_SET(x)\
+ FIELD_PREP(DEV_PCS1G_CFG_PCS_ENA, x)
+#define DEV_PCS1G_CFG_PCS_ENA_GET(x)\
+ FIELD_GET(DEV_PCS1G_CFG_PCS_ENA, x)
+
+/* DEV:PCS1G_CFG_STATUS:PCS1G_MODE_CFG */
+#define DEV_PCS1G_MODE_CFG(t) __REG(TARGET_DEV, t, 8, 72, 0, 1, 68, 4, 0, 1, 4)
+
+#define DEV_PCS1G_MODE_CFG_SGMII_MODE_ENA BIT(0)
+#define DEV_PCS1G_MODE_CFG_SGMII_MODE_ENA_SET(x)\
+ FIELD_PREP(DEV_PCS1G_MODE_CFG_SGMII_MODE_ENA, x)
+#define DEV_PCS1G_MODE_CFG_SGMII_MODE_ENA_GET(x)\
+ FIELD_GET(DEV_PCS1G_MODE_CFG_SGMII_MODE_ENA, x)
+
+/* DEV:PCS1G_CFG_STATUS:PCS1G_SD_CFG */
+#define DEV_PCS1G_SD_CFG(t) __REG(TARGET_DEV, t, 8, 72, 0, 1, 68, 8, 0, 1, 4)
+
+#define DEV_PCS1G_SD_CFG_SD_ENA BIT(0)
+#define DEV_PCS1G_SD_CFG_SD_ENA_SET(x)\
+ FIELD_PREP(DEV_PCS1G_SD_CFG_SD_ENA, x)
+#define DEV_PCS1G_SD_CFG_SD_ENA_GET(x)\
+ FIELD_GET(DEV_PCS1G_SD_CFG_SD_ENA, x)
+
+/* DEV:PCS1G_CFG_STATUS:PCS1G_ANEG_CFG */
+#define DEV_PCS1G_ANEG_CFG(t) __REG(TARGET_DEV, t, 8, 72, 0, 1, 68, 12, 0, 1, 4)
+
+#define DEV_PCS1G_ANEG_CFG_ADV_ABILITY GENMASK(31, 16)
+#define DEV_PCS1G_ANEG_CFG_ADV_ABILITY_SET(x)\
+ FIELD_PREP(DEV_PCS1G_ANEG_CFG_ADV_ABILITY, x)
+#define DEV_PCS1G_ANEG_CFG_ADV_ABILITY_GET(x)\
+ FIELD_GET(DEV_PCS1G_ANEG_CFG_ADV_ABILITY, x)
+
+#define DEV_PCS1G_ANEG_CFG_SW_RESOLVE_ENA BIT(8)
+#define DEV_PCS1G_ANEG_CFG_SW_RESOLVE_ENA_SET(x)\
+ FIELD_PREP(DEV_PCS1G_ANEG_CFG_SW_RESOLVE_ENA, x)
+#define DEV_PCS1G_ANEG_CFG_SW_RESOLVE_ENA_GET(x)\
+ FIELD_GET(DEV_PCS1G_ANEG_CFG_SW_RESOLVE_ENA, x)
+
+#define DEV_PCS1G_ANEG_CFG_RESTART_ONE_SHOT BIT(1)
+#define DEV_PCS1G_ANEG_CFG_RESTART_ONE_SHOT_SET(x)\
+ FIELD_PREP(DEV_PCS1G_ANEG_CFG_RESTART_ONE_SHOT, x)
+#define DEV_PCS1G_ANEG_CFG_RESTART_ONE_SHOT_GET(x)\
+ FIELD_GET(DEV_PCS1G_ANEG_CFG_RESTART_ONE_SHOT, x)
+
+#define DEV_PCS1G_ANEG_CFG_ENA BIT(0)
+#define DEV_PCS1G_ANEG_CFG_ENA_SET(x)\
+ FIELD_PREP(DEV_PCS1G_ANEG_CFG_ENA, x)
+#define DEV_PCS1G_ANEG_CFG_ENA_GET(x)\
+ FIELD_GET(DEV_PCS1G_ANEG_CFG_ENA, x)
+
+/* DEV:PCS1G_CFG_STATUS:PCS1G_ANEG_STATUS */
+#define DEV_PCS1G_ANEG_STATUS(t) __REG(TARGET_DEV, t, 8, 72, 0, 1, 68, 32, 0, 1, 4)
+
+#define DEV_PCS1G_ANEG_STATUS_LP_ADV GENMASK(31, 16)
+#define DEV_PCS1G_ANEG_STATUS_LP_ADV_SET(x)\
+ FIELD_PREP(DEV_PCS1G_ANEG_STATUS_LP_ADV, x)
+#define DEV_PCS1G_ANEG_STATUS_LP_ADV_GET(x)\
+ FIELD_GET(DEV_PCS1G_ANEG_STATUS_LP_ADV, x)
+
+#define DEV_PCS1G_ANEG_STATUS_ANEG_COMPLETE BIT(0)
+#define DEV_PCS1G_ANEG_STATUS_ANEG_COMPLETE_SET(x)\
+ FIELD_PREP(DEV_PCS1G_ANEG_STATUS_ANEG_COMPLETE, x)
+#define DEV_PCS1G_ANEG_STATUS_ANEG_COMPLETE_GET(x)\
+ FIELD_GET(DEV_PCS1G_ANEG_STATUS_ANEG_COMPLETE, x)
+
+/* DEV:PCS1G_CFG_STATUS:PCS1G_LINK_STATUS */
+#define DEV_PCS1G_LINK_STATUS(t) __REG(TARGET_DEV, t, 8, 72, 0, 1, 68, 40, 0, 1, 4)
+
+#define DEV_PCS1G_LINK_STATUS_LINK_STATUS BIT(4)
+#define DEV_PCS1G_LINK_STATUS_LINK_STATUS_SET(x)\
+ FIELD_PREP(DEV_PCS1G_LINK_STATUS_LINK_STATUS, x)
+#define DEV_PCS1G_LINK_STATUS_LINK_STATUS_GET(x)\
+ FIELD_GET(DEV_PCS1G_LINK_STATUS_LINK_STATUS, x)
+
+#define DEV_PCS1G_LINK_STATUS_SYNC_STATUS BIT(0)
+#define DEV_PCS1G_LINK_STATUS_SYNC_STATUS_SET(x)\
+ FIELD_PREP(DEV_PCS1G_LINK_STATUS_SYNC_STATUS, x)
+#define DEV_PCS1G_LINK_STATUS_SYNC_STATUS_GET(x)\
+ FIELD_GET(DEV_PCS1G_LINK_STATUS_SYNC_STATUS, x)
+
+/* DEV:PCS1G_CFG_STATUS:PCS1G_STICKY */
+#define DEV_PCS1G_STICKY(t) __REG(TARGET_DEV, t, 8, 72, 0, 1, 68, 48, 0, 1, 4)
+
+#define DEV_PCS1G_STICKY_LINK_DOWN_STICKY BIT(4)
+#define DEV_PCS1G_STICKY_LINK_DOWN_STICKY_SET(x)\
+ FIELD_PREP(DEV_PCS1G_STICKY_LINK_DOWN_STICKY, x)
+#define DEV_PCS1G_STICKY_LINK_DOWN_STICKY_GET(x)\
+ FIELD_GET(DEV_PCS1G_STICKY_LINK_DOWN_STICKY, x)
+
+/* DEVCPU_QS:XTR:XTR_GRP_CFG */
+#define QS_XTR_GRP_CFG(r) __REG(TARGET_QS, 0, 1, 0, 0, 1, 36, 0, r, 2, 4)
+
+#define QS_XTR_GRP_CFG_MODE GENMASK(3, 2)
+#define QS_XTR_GRP_CFG_MODE_SET(x)\
+ FIELD_PREP(QS_XTR_GRP_CFG_MODE, x)
+#define QS_XTR_GRP_CFG_MODE_GET(x)\
+ FIELD_GET(QS_XTR_GRP_CFG_MODE, x)
+
+#define QS_XTR_GRP_CFG_BYTE_SWAP BIT(0)
+#define QS_XTR_GRP_CFG_BYTE_SWAP_SET(x)\
+ FIELD_PREP(QS_XTR_GRP_CFG_BYTE_SWAP, x)
+#define QS_XTR_GRP_CFG_BYTE_SWAP_GET(x)\
+ FIELD_GET(QS_XTR_GRP_CFG_BYTE_SWAP, x)
+
+/* DEVCPU_QS:XTR:XTR_RD */
+#define QS_XTR_RD(r) __REG(TARGET_QS, 0, 1, 0, 0, 1, 36, 8, r, 2, 4)
+
+/* DEVCPU_QS:XTR:XTR_FLUSH */
+#define QS_XTR_FLUSH __REG(TARGET_QS, 0, 1, 0, 0, 1, 36, 24, 0, 1, 4)
+
+/* DEVCPU_QS:XTR:XTR_DATA_PRESENT */
+#define QS_XTR_DATA_PRESENT __REG(TARGET_QS, 0, 1, 0, 0, 1, 36, 28, 0, 1, 4)
+
+/* DEVCPU_QS:INJ:INJ_GRP_CFG */
+#define QS_INJ_GRP_CFG(r) __REG(TARGET_QS, 0, 1, 36, 0, 1, 40, 0, r, 2, 4)
+
+#define QS_INJ_GRP_CFG_MODE GENMASK(3, 2)
+#define QS_INJ_GRP_CFG_MODE_SET(x)\
+ FIELD_PREP(QS_INJ_GRP_CFG_MODE, x)
+#define QS_INJ_GRP_CFG_MODE_GET(x)\
+ FIELD_GET(QS_INJ_GRP_CFG_MODE, x)
+
+#define QS_INJ_GRP_CFG_BYTE_SWAP BIT(0)
+#define QS_INJ_GRP_CFG_BYTE_SWAP_SET(x)\
+ FIELD_PREP(QS_INJ_GRP_CFG_BYTE_SWAP, x)
+#define QS_INJ_GRP_CFG_BYTE_SWAP_GET(x)\
+ FIELD_GET(QS_INJ_GRP_CFG_BYTE_SWAP, x)
+
+/* DEVCPU_QS:INJ:INJ_WR */
+#define QS_INJ_WR(r) __REG(TARGET_QS, 0, 1, 36, 0, 1, 40, 8, r, 2, 4)
+
+/* DEVCPU_QS:INJ:INJ_CTRL */
+#define QS_INJ_CTRL(r) __REG(TARGET_QS, 0, 1, 36, 0, 1, 40, 16, r, 2, 4)
+
+#define QS_INJ_CTRL_GAP_SIZE GENMASK(24, 21)
+#define QS_INJ_CTRL_GAP_SIZE_SET(x)\
+ FIELD_PREP(QS_INJ_CTRL_GAP_SIZE, x)
+#define QS_INJ_CTRL_GAP_SIZE_GET(x)\
+ FIELD_GET(QS_INJ_CTRL_GAP_SIZE, x)
+
+#define QS_INJ_CTRL_EOF BIT(19)
+#define QS_INJ_CTRL_EOF_SET(x)\
+ FIELD_PREP(QS_INJ_CTRL_EOF, x)
+#define QS_INJ_CTRL_EOF_GET(x)\
+ FIELD_GET(QS_INJ_CTRL_EOF, x)
+
+#define QS_INJ_CTRL_SOF BIT(18)
+#define QS_INJ_CTRL_SOF_SET(x)\
+ FIELD_PREP(QS_INJ_CTRL_SOF, x)
+#define QS_INJ_CTRL_SOF_GET(x)\
+ FIELD_GET(QS_INJ_CTRL_SOF, x)
+
+#define QS_INJ_CTRL_VLD_BYTES GENMASK(17, 16)
+#define QS_INJ_CTRL_VLD_BYTES_SET(x)\
+ FIELD_PREP(QS_INJ_CTRL_VLD_BYTES, x)
+#define QS_INJ_CTRL_VLD_BYTES_GET(x)\
+ FIELD_GET(QS_INJ_CTRL_VLD_BYTES, x)
+
+/* DEVCPU_QS:INJ:INJ_STATUS */
+#define QS_INJ_STATUS __REG(TARGET_QS, 0, 1, 36, 0, 1, 40, 24, 0, 1, 4)
+
+#define QS_INJ_STATUS_WMARK_REACHED GENMASK(5, 4)
+#define QS_INJ_STATUS_WMARK_REACHED_SET(x)\
+ FIELD_PREP(QS_INJ_STATUS_WMARK_REACHED, x)
+#define QS_INJ_STATUS_WMARK_REACHED_GET(x)\
+ FIELD_GET(QS_INJ_STATUS_WMARK_REACHED, x)
+
+#define QS_INJ_STATUS_FIFO_RDY GENMASK(3, 2)
+#define QS_INJ_STATUS_FIFO_RDY_SET(x)\
+ FIELD_PREP(QS_INJ_STATUS_FIFO_RDY, x)
+#define QS_INJ_STATUS_FIFO_RDY_GET(x)\
+ FIELD_GET(QS_INJ_STATUS_FIFO_RDY, x)
+
+/* QSYS:SYSTEM:PORT_MODE */
+#define QSYS_PORT_MODE(r) __REG(TARGET_QSYS, 0, 1, 28008, 0, 1, 216, 0, r, 10, 4)
+
+#define QSYS_PORT_MODE_DEQUEUE_DIS BIT(1)
+#define QSYS_PORT_MODE_DEQUEUE_DIS_SET(x)\
+ FIELD_PREP(QSYS_PORT_MODE_DEQUEUE_DIS, x)
+#define QSYS_PORT_MODE_DEQUEUE_DIS_GET(x)\
+ FIELD_GET(QSYS_PORT_MODE_DEQUEUE_DIS, x)
+
+/* QSYS:SYSTEM:SWITCH_PORT_MODE */
+#define QSYS_SW_PORT_MODE(r) __REG(TARGET_QSYS, 0, 1, 28008, 0, 1, 216, 80, r, 9, 4)
+
+#define QSYS_SW_PORT_MODE_PORT_ENA BIT(18)
+#define QSYS_SW_PORT_MODE_PORT_ENA_SET(x)\
+ FIELD_PREP(QSYS_SW_PORT_MODE_PORT_ENA, x)
+#define QSYS_SW_PORT_MODE_PORT_ENA_GET(x)\
+ FIELD_GET(QSYS_SW_PORT_MODE_PORT_ENA, x)
+
+#define QSYS_SW_PORT_MODE_SCH_NEXT_CFG GENMASK(16, 14)
+#define QSYS_SW_PORT_MODE_SCH_NEXT_CFG_SET(x)\
+ FIELD_PREP(QSYS_SW_PORT_MODE_SCH_NEXT_CFG, x)
+#define QSYS_SW_PORT_MODE_SCH_NEXT_CFG_GET(x)\
+ FIELD_GET(QSYS_SW_PORT_MODE_SCH_NEXT_CFG, x)
+
+#define QSYS_SW_PORT_MODE_INGRESS_DROP_MODE BIT(12)
+#define QSYS_SW_PORT_MODE_INGRESS_DROP_MODE_SET(x)\
+ FIELD_PREP(QSYS_SW_PORT_MODE_INGRESS_DROP_MODE, x)
+#define QSYS_SW_PORT_MODE_INGRESS_DROP_MODE_GET(x)\
+ FIELD_GET(QSYS_SW_PORT_MODE_INGRESS_DROP_MODE, x)
+
+#define QSYS_SW_PORT_MODE_TX_PFC_ENA GENMASK(11, 4)
+#define QSYS_SW_PORT_MODE_TX_PFC_ENA_SET(x)\
+ FIELD_PREP(QSYS_SW_PORT_MODE_TX_PFC_ENA, x)
+#define QSYS_SW_PORT_MODE_TX_PFC_ENA_GET(x)\
+ FIELD_GET(QSYS_SW_PORT_MODE_TX_PFC_ENA, x)
+
+#define QSYS_SW_PORT_MODE_AGING_MODE GENMASK(1, 0)
+#define QSYS_SW_PORT_MODE_AGING_MODE_SET(x)\
+ FIELD_PREP(QSYS_SW_PORT_MODE_AGING_MODE, x)
+#define QSYS_SW_PORT_MODE_AGING_MODE_GET(x)\
+ FIELD_GET(QSYS_SW_PORT_MODE_AGING_MODE, x)
+
+/* QSYS:SYSTEM:SW_STATUS */
+#define QSYS_SW_STATUS(r) __REG(TARGET_QSYS, 0, 1, 28008, 0, 1, 216, 164, r, 9, 4)
+
+#define QSYS_SW_STATUS_EQ_AVAIL GENMASK(7, 0)
+#define QSYS_SW_STATUS_EQ_AVAIL_SET(x)\
+ FIELD_PREP(QSYS_SW_STATUS_EQ_AVAIL, x)
+#define QSYS_SW_STATUS_EQ_AVAIL_GET(x)\
+ FIELD_GET(QSYS_SW_STATUS_EQ_AVAIL, x)
+
+/* QSYS:SYSTEM:CPU_GROUP_MAP */
+#define QSYS_CPU_GROUP_MAP __REG(TARGET_QSYS, 0, 1, 28008, 0, 1, 216, 204, 0, 1, 4)
+
+/* QSYS:RES_CTRL:RES_CFG */
+#define QSYS_RES_CFG(g) __REG(TARGET_QSYS, 0, 1, 32768, g, 1024, 8, 0, 0, 1, 4)
+
+/* REW:PORT:PORT_VLAN_CFG */
+#define REW_PORT_VLAN_CFG(g) __REG(TARGET_REW, 0, 1, 0, g, 10, 128, 0, 0, 1, 4)
+
+#define REW_PORT_VLAN_CFG_PORT_TPID GENMASK(31, 16)
+#define REW_PORT_VLAN_CFG_PORT_TPID_SET(x)\
+ FIELD_PREP(REW_PORT_VLAN_CFG_PORT_TPID, x)
+#define REW_PORT_VLAN_CFG_PORT_TPID_GET(x)\
+ FIELD_GET(REW_PORT_VLAN_CFG_PORT_TPID, x)
+
+#define REW_PORT_VLAN_CFG_PORT_VID GENMASK(11, 0)
+#define REW_PORT_VLAN_CFG_PORT_VID_SET(x)\
+ FIELD_PREP(REW_PORT_VLAN_CFG_PORT_VID, x)
+#define REW_PORT_VLAN_CFG_PORT_VID_GET(x)\
+ FIELD_GET(REW_PORT_VLAN_CFG_PORT_VID, x)
+
+/* REW:PORT:TAG_CFG */
+#define REW_TAG_CFG(g) __REG(TARGET_REW, 0, 1, 0, g, 10, 128, 4, 0, 1, 4)
+
+#define REW_TAG_CFG_TAG_CFG GENMASK(8, 7)
+#define REW_TAG_CFG_TAG_CFG_SET(x)\
+ FIELD_PREP(REW_TAG_CFG_TAG_CFG, x)
+#define REW_TAG_CFG_TAG_CFG_GET(x)\
+ FIELD_GET(REW_TAG_CFG_TAG_CFG, x)
+
+#define REW_TAG_CFG_TAG_TPID_CFG GENMASK(6, 5)
+#define REW_TAG_CFG_TAG_TPID_CFG_SET(x)\
+ FIELD_PREP(REW_TAG_CFG_TAG_TPID_CFG, x)
+#define REW_TAG_CFG_TAG_TPID_CFG_GET(x)\
+ FIELD_GET(REW_TAG_CFG_TAG_TPID_CFG, x)
+
+/* REW:PORT:PORT_CFG */
+#define REW_PORT_CFG(g) __REG(TARGET_REW, 0, 1, 0, g, 10, 128, 8, 0, 1, 4)
+
+#define REW_PORT_CFG_NO_REWRITE BIT(0)
+#define REW_PORT_CFG_NO_REWRITE_SET(x)\
+ FIELD_PREP(REW_PORT_CFG_NO_REWRITE, x)
+#define REW_PORT_CFG_NO_REWRITE_GET(x)\
+ FIELD_GET(REW_PORT_CFG_NO_REWRITE, x)
+
+/* SYS:SYSTEM:RESET_CFG */
+#define SYS_RESET_CFG __REG(TARGET_SYS, 0, 1, 4128, 0, 1, 168, 0, 0, 1, 4)
+
+#define SYS_RESET_CFG_CORE_ENA BIT(0)
+#define SYS_RESET_CFG_CORE_ENA_SET(x)\
+ FIELD_PREP(SYS_RESET_CFG_CORE_ENA, x)
+#define SYS_RESET_CFG_CORE_ENA_GET(x)\
+ FIELD_GET(SYS_RESET_CFG_CORE_ENA, x)
+
+/* SYS:SYSTEM:PORT_MODE */
+#define SYS_PORT_MODE(r) __REG(TARGET_SYS, 0, 1, 4128, 0, 1, 168, 44, r, 10, 4)
+
+#define SYS_PORT_MODE_INCL_INJ_HDR GENMASK(5, 4)
+#define SYS_PORT_MODE_INCL_INJ_HDR_SET(x)\
+ FIELD_PREP(SYS_PORT_MODE_INCL_INJ_HDR, x)
+#define SYS_PORT_MODE_INCL_INJ_HDR_GET(x)\
+ FIELD_GET(SYS_PORT_MODE_INCL_INJ_HDR, x)
+
+#define SYS_PORT_MODE_INCL_XTR_HDR GENMASK(3, 2)
+#define SYS_PORT_MODE_INCL_XTR_HDR_SET(x)\
+ FIELD_PREP(SYS_PORT_MODE_INCL_XTR_HDR, x)
+#define SYS_PORT_MODE_INCL_XTR_HDR_GET(x)\
+ FIELD_GET(SYS_PORT_MODE_INCL_XTR_HDR, x)
+
+/* SYS:SYSTEM:FRONT_PORT_MODE */
+#define SYS_FRONT_PORT_MODE(r) __REG(TARGET_SYS, 0, 1, 4128, 0, 1, 168, 84, r, 8, 4)
+
+#define SYS_FRONT_PORT_MODE_HDX_MODE BIT(1)
+#define SYS_FRONT_PORT_MODE_HDX_MODE_SET(x)\
+ FIELD_PREP(SYS_FRONT_PORT_MODE_HDX_MODE, x)
+#define SYS_FRONT_PORT_MODE_HDX_MODE_GET(x)\
+ FIELD_GET(SYS_FRONT_PORT_MODE_HDX_MODE, x)
+
+/* SYS:SYSTEM:FRM_AGING */
+#define SYS_FRM_AGING __REG(TARGET_SYS, 0, 1, 4128, 0, 1, 168, 116, 0, 1, 4)
+
+#define SYS_FRM_AGING_AGE_TX_ENA BIT(20)
+#define SYS_FRM_AGING_AGE_TX_ENA_SET(x)\
+ FIELD_PREP(SYS_FRM_AGING_AGE_TX_ENA, x)
+#define SYS_FRM_AGING_AGE_TX_ENA_GET(x)\
+ FIELD_GET(SYS_FRM_AGING_AGE_TX_ENA, x)
+
+/* SYS:SYSTEM:STAT_CFG */
+#define SYS_STAT_CFG __REG(TARGET_SYS, 0, 1, 4128, 0, 1, 168, 120, 0, 1, 4)
+
+#define SYS_STAT_CFG_STAT_VIEW GENMASK(9, 0)
+#define SYS_STAT_CFG_STAT_VIEW_SET(x)\
+ FIELD_PREP(SYS_STAT_CFG_STAT_VIEW, x)
+#define SYS_STAT_CFG_STAT_VIEW_GET(x)\
+ FIELD_GET(SYS_STAT_CFG_STAT_VIEW, x)
+
+/* SYS:PAUSE_CFG:PAUSE_CFG */
+#define SYS_PAUSE_CFG(r) __REG(TARGET_SYS, 0, 1, 4296, 0, 1, 112, 0, r, 9, 4)
+
+#define SYS_PAUSE_CFG_PAUSE_START GENMASK(18, 10)
+#define SYS_PAUSE_CFG_PAUSE_START_SET(x)\
+ FIELD_PREP(SYS_PAUSE_CFG_PAUSE_START, x)
+#define SYS_PAUSE_CFG_PAUSE_START_GET(x)\
+ FIELD_GET(SYS_PAUSE_CFG_PAUSE_START, x)
+
+#define SYS_PAUSE_CFG_PAUSE_STOP GENMASK(9, 1)
+#define SYS_PAUSE_CFG_PAUSE_STOP_SET(x)\
+ FIELD_PREP(SYS_PAUSE_CFG_PAUSE_STOP, x)
+#define SYS_PAUSE_CFG_PAUSE_STOP_GET(x)\
+ FIELD_GET(SYS_PAUSE_CFG_PAUSE_STOP, x)
+
+#define SYS_PAUSE_CFG_PAUSE_ENA BIT(0)
+#define SYS_PAUSE_CFG_PAUSE_ENA_SET(x)\
+ FIELD_PREP(SYS_PAUSE_CFG_PAUSE_ENA, x)
+#define SYS_PAUSE_CFG_PAUSE_ENA_GET(x)\
+ FIELD_GET(SYS_PAUSE_CFG_PAUSE_ENA, x)
+
+/* SYS:PAUSE_CFG:ATOP */
+#define SYS_ATOP(r) __REG(TARGET_SYS, 0, 1, 4296, 0, 1, 112, 40, r, 9, 4)
+
+/* SYS:PAUSE_CFG:ATOP_TOT_CFG */
+#define SYS_ATOP_TOT_CFG __REG(TARGET_SYS, 0, 1, 4296, 0, 1, 112, 76, 0, 1, 4)
+
+/* SYS:PAUSE_CFG:MAC_FC_CFG */
+#define SYS_MAC_FC_CFG(r) __REG(TARGET_SYS, 0, 1, 4296, 0, 1, 112, 80, r, 8, 4)
+
+#define SYS_MAC_FC_CFG_FC_LINK_SPEED GENMASK(27, 26)
+#define SYS_MAC_FC_CFG_FC_LINK_SPEED_SET(x)\
+ FIELD_PREP(SYS_MAC_FC_CFG_FC_LINK_SPEED, x)
+#define SYS_MAC_FC_CFG_FC_LINK_SPEED_GET(x)\
+ FIELD_GET(SYS_MAC_FC_CFG_FC_LINK_SPEED, x)
+
+#define SYS_MAC_FC_CFG_FC_LATENCY_CFG GENMASK(25, 20)
+#define SYS_MAC_FC_CFG_FC_LATENCY_CFG_SET(x)\
+ FIELD_PREP(SYS_MAC_FC_CFG_FC_LATENCY_CFG, x)
+#define SYS_MAC_FC_CFG_FC_LATENCY_CFG_GET(x)\
+ FIELD_GET(SYS_MAC_FC_CFG_FC_LATENCY_CFG, x)
+
+#define SYS_MAC_FC_CFG_ZERO_PAUSE_ENA BIT(18)
+#define SYS_MAC_FC_CFG_ZERO_PAUSE_ENA_SET(x)\
+ FIELD_PREP(SYS_MAC_FC_CFG_ZERO_PAUSE_ENA, x)
+#define SYS_MAC_FC_CFG_ZERO_PAUSE_ENA_GET(x)\
+ FIELD_GET(SYS_MAC_FC_CFG_ZERO_PAUSE_ENA, x)
+
+#define SYS_MAC_FC_CFG_TX_FC_ENA BIT(17)
+#define SYS_MAC_FC_CFG_TX_FC_ENA_SET(x)\
+ FIELD_PREP(SYS_MAC_FC_CFG_TX_FC_ENA, x)
+#define SYS_MAC_FC_CFG_TX_FC_ENA_GET(x)\
+ FIELD_GET(SYS_MAC_FC_CFG_TX_FC_ENA, x)
+
+#define SYS_MAC_FC_CFG_RX_FC_ENA BIT(16)
+#define SYS_MAC_FC_CFG_RX_FC_ENA_SET(x)\
+ FIELD_PREP(SYS_MAC_FC_CFG_RX_FC_ENA, x)
+#define SYS_MAC_FC_CFG_RX_FC_ENA_GET(x)\
+ FIELD_GET(SYS_MAC_FC_CFG_RX_FC_ENA, x)
+
+#define SYS_MAC_FC_CFG_PAUSE_VAL_CFG GENMASK(15, 0)
+#define SYS_MAC_FC_CFG_PAUSE_VAL_CFG_SET(x)\
+ FIELD_PREP(SYS_MAC_FC_CFG_PAUSE_VAL_CFG, x)
+#define SYS_MAC_FC_CFG_PAUSE_VAL_CFG_GET(x)\
+ FIELD_GET(SYS_MAC_FC_CFG_PAUSE_VAL_CFG, x)
+
+/* SYS:STAT:CNT */
+#define SYS_CNT(g) __REG(TARGET_SYS, 0, 1, 0, g, 896, 4, 0, 0, 1, 4)
+
+/* SYS:RAM_CTRL:RAM_INIT */
+#define SYS_RAM_INIT __REG(TARGET_SYS, 0, 1, 4432, 0, 1, 4, 0, 0, 1, 4)
+
+#define SYS_RAM_INIT_RAM_INIT BIT(1)
+#define SYS_RAM_INIT_RAM_INIT_SET(x)\
+ FIELD_PREP(SYS_RAM_INIT_RAM_INIT, x)
+#define SYS_RAM_INIT_RAM_INIT_GET(x)\
+ FIELD_GET(SYS_RAM_INIT_RAM_INIT, x)
+
+#endif /* _LAN966X_REGS_H_ */
diff --git a/drivers/net/ethernet/microchip/lan966x/lan966x_switchdev.c b/drivers/net/ethernet/microchip/lan966x/lan966x_switchdev.c
new file mode 100644
index 0000000..7de55f6
--- /dev/null
+++ b/drivers/net/ethernet/microchip/lan966x/lan966x_switchdev.c
@@ -0,0 +1,544 @@
+// SPDX-License-Identifier: GPL-2.0+
+
+#include <linux/if_bridge.h>
+#include <net/switchdev.h>
+
+#include "lan966x_main.h"
+
+static struct notifier_block lan966x_netdevice_nb __read_mostly;
+static struct notifier_block lan966x_switchdev_nb __read_mostly;
+static struct notifier_block lan966x_switchdev_blocking_nb __read_mostly;
+
+static void lan966x_port_set_mcast_flood(struct lan966x_port *port,
+ bool enabled)
+{
+ u32 val = lan_rd(port->lan966x, ANA_PGID(PGID_MC));
+
+ val = ANA_PGID_PGID_GET(val);
+ if (enabled)
+ val |= BIT(port->chip_port);
+ else
+ val &= ~BIT(port->chip_port);
+
+ lan_rmw(ANA_PGID_PGID_SET(val),
+ ANA_PGID_PGID,
+ port->lan966x, ANA_PGID(PGID_MC));
+}
+
+static void lan966x_port_set_ucast_flood(struct lan966x_port *port,
+ bool enabled)
+{
+ u32 val = lan_rd(port->lan966x, ANA_PGID(PGID_UC));
+
+ val = ANA_PGID_PGID_GET(val);
+ if (enabled)
+ val |= BIT(port->chip_port);
+ else
+ val &= ~BIT(port->chip_port);
+
+ lan_rmw(ANA_PGID_PGID_SET(val),
+ ANA_PGID_PGID,
+ port->lan966x, ANA_PGID(PGID_UC));
+}
+
+static void lan966x_port_set_bcast_flood(struct lan966x_port *port,
+ bool enabled)
+{
+ u32 val = lan_rd(port->lan966x, ANA_PGID(PGID_BC));
+
+ val = ANA_PGID_PGID_GET(val);
+ if (enabled)
+ val |= BIT(port->chip_port);
+ else
+ val &= ~BIT(port->chip_port);
+
+ lan_rmw(ANA_PGID_PGID_SET(val),
+ ANA_PGID_PGID,
+ port->lan966x, ANA_PGID(PGID_BC));
+}
+
+static void lan966x_port_set_learning(struct lan966x_port *port, bool enabled)
+{
+ lan_rmw(ANA_PORT_CFG_LEARN_ENA_SET(enabled),
+ ANA_PORT_CFG_LEARN_ENA,
+ port->lan966x, ANA_PORT_CFG(port->chip_port));
+
+ port->learn_ena = enabled;
+}
+
+static void lan966x_port_bridge_flags(struct lan966x_port *port,
+ struct switchdev_brport_flags flags)
+{
+ if (flags.mask & BR_MCAST_FLOOD)
+ lan966x_port_set_mcast_flood(port,
+ !!(flags.val & BR_MCAST_FLOOD));
+
+ if (flags.mask & BR_FLOOD)
+ lan966x_port_set_ucast_flood(port,
+ !!(flags.val & BR_FLOOD));
+
+ if (flags.mask & BR_BCAST_FLOOD)
+ lan966x_port_set_bcast_flood(port,
+ !!(flags.val & BR_BCAST_FLOOD));
+
+ if (flags.mask & BR_LEARNING)
+ lan966x_port_set_learning(port,
+ !!(flags.val & BR_LEARNING));
+}
+
+static int lan966x_port_pre_bridge_flags(struct lan966x_port *port,
+ struct switchdev_brport_flags flags)
+{
+ if (flags.mask & ~(BR_MCAST_FLOOD | BR_FLOOD | BR_BCAST_FLOOD |
+ BR_LEARNING))
+ return -EINVAL;
+
+ return 0;
+}
+
+static void lan966x_update_fwd_mask(struct lan966x *lan966x)
+{
+ int i;
+
+ for (i = 0; i < lan966x->num_phys_ports; i++) {
+ struct lan966x_port *port = lan966x->ports[i];
+ unsigned long mask = 0;
+
+ if (port && lan966x->bridge_fwd_mask & BIT(i))
+ mask = lan966x->bridge_fwd_mask & ~BIT(i);
+
+ mask |= BIT(CPU_PORT);
+
+ lan_wr(ANA_PGID_PGID_SET(mask),
+ lan966x, ANA_PGID(PGID_SRC + i));
+ }
+}
+
+static void lan966x_port_stp_state_set(struct lan966x_port *port, u8 state)
+{
+ struct lan966x *lan966x = port->lan966x;
+ bool learn_ena = false;
+
+ if ((state == BR_STATE_FORWARDING || state == BR_STATE_LEARNING) &&
+ port->learn_ena)
+ learn_ena = true;
+
+ if (state == BR_STATE_FORWARDING)
+ lan966x->bridge_fwd_mask |= BIT(port->chip_port);
+ else
+ lan966x->bridge_fwd_mask &= ~BIT(port->chip_port);
+
+ lan_rmw(ANA_PORT_CFG_LEARN_ENA_SET(learn_ena),
+ ANA_PORT_CFG_LEARN_ENA,
+ lan966x, ANA_PORT_CFG(port->chip_port));
+
+ lan966x_update_fwd_mask(lan966x);
+}
+
+static void lan966x_port_ageing_set(struct lan966x_port *port,
+ unsigned long ageing_clock_t)
+{
+ unsigned long ageing_jiffies = clock_t_to_jiffies(ageing_clock_t);
+ u32 ageing_time = jiffies_to_msecs(ageing_jiffies) / 1000;
+
+ lan966x_mac_set_ageing(port->lan966x, ageing_time);
+}
+
+static int lan966x_port_attr_set(struct net_device *dev, const void *ctx,
+ const struct switchdev_attr *attr,
+ struct netlink_ext_ack *extack)
+{
+ struct lan966x_port *port = netdev_priv(dev);
+ int err = 0;
+
+ if (ctx && ctx != port)
+ return 0;
+
+ switch (attr->id) {
+ case SWITCHDEV_ATTR_ID_PORT_BRIDGE_FLAGS:
+ lan966x_port_bridge_flags(port, attr->u.brport_flags);
+ break;
+ case SWITCHDEV_ATTR_ID_PORT_PRE_BRIDGE_FLAGS:
+ err = lan966x_port_pre_bridge_flags(port, attr->u.brport_flags);
+ break;
+ case SWITCHDEV_ATTR_ID_PORT_STP_STATE:
+ lan966x_port_stp_state_set(port, attr->u.stp_state);
+ break;
+ case SWITCHDEV_ATTR_ID_BRIDGE_AGEING_TIME:
+ lan966x_port_ageing_set(port, attr->u.ageing_time);
+ break;
+ case SWITCHDEV_ATTR_ID_BRIDGE_VLAN_FILTERING:
+ lan966x_vlan_port_set_vlan_aware(port, attr->u.vlan_filtering);
+ lan966x_vlan_port_apply(port);
+ break;
+ default:
+ err = -EOPNOTSUPP;
+ break;
+ }
+
+ return err;
+}
+
+static int lan966x_port_bridge_join(struct lan966x_port *port,
+ struct net_device *bridge,
+ struct netlink_ext_ack *extack)
+{
+ struct switchdev_brport_flags flags = {0};
+ struct lan966x *lan966x = port->lan966x;
+ struct net_device *dev = port->dev;
+ int err;
+
+ if (!lan966x->bridge_mask) {
+ lan966x->bridge = bridge;
+ } else {
+ if (lan966x->bridge != bridge) {
+ NL_SET_ERR_MSG_MOD(extack, "Not allow to add port to different bridge");
+ return -ENODEV;
+ }
+ }
+
+ err = switchdev_bridge_port_offload(dev, dev, port,
+ &lan966x_switchdev_nb,
+ &lan966x_switchdev_blocking_nb,
+ false, extack);
+ if (err)
+ return err;
+
+ lan966x->bridge_mask |= BIT(port->chip_port);
+
+ flags.mask = BR_LEARNING | BR_FLOOD | BR_MCAST_FLOOD | BR_BCAST_FLOOD;
+ flags.val = flags.mask;
+ lan966x_port_bridge_flags(port, flags);
+
+ return 0;
+}
+
+static void lan966x_port_bridge_leave(struct lan966x_port *port,
+ struct net_device *bridge)
+{
+ struct switchdev_brport_flags flags = {0};
+ struct lan966x *lan966x = port->lan966x;
+
+ flags.mask = BR_LEARNING | BR_FLOOD | BR_MCAST_FLOOD | BR_BCAST_FLOOD;
+ flags.val = flags.mask & ~BR_LEARNING;
+ lan966x_port_bridge_flags(port, flags);
+
+ lan966x->bridge_mask &= ~BIT(port->chip_port);
+
+ if (!lan966x->bridge_mask)
+ lan966x->bridge = NULL;
+
+ /* Set the port back to host mode */
+ lan966x_vlan_port_set_vlan_aware(port, false);
+ lan966x_vlan_port_set_vid(port, HOST_PVID, false, false);
+ lan966x_vlan_port_apply(port);
+}
+
+static int lan966x_port_changeupper(struct net_device *dev,
+ struct netdev_notifier_changeupper_info *info)
+{
+ struct lan966x_port *port = netdev_priv(dev);
+ struct netlink_ext_ack *extack;
+ int err = 0;
+
+ extack = netdev_notifier_info_to_extack(&info->info);
+
+ if (netif_is_bridge_master(info->upper_dev)) {
+ if (info->linking)
+ err = lan966x_port_bridge_join(port, info->upper_dev,
+ extack);
+ else
+ lan966x_port_bridge_leave(port, info->upper_dev);
+ }
+
+ return err;
+}
+
+static int lan966x_port_prechangeupper(struct net_device *dev,
+ struct netdev_notifier_changeupper_info *info)
+{
+ struct lan966x_port *port = netdev_priv(dev);
+
+ if (netif_is_bridge_master(info->upper_dev) && !info->linking)
+ switchdev_bridge_port_unoffload(port->dev, port,
+ &lan966x_switchdev_nb,
+ &lan966x_switchdev_blocking_nb);
+
+ return NOTIFY_DONE;
+}
+
+static int lan966x_foreign_bridging_check(struct net_device *bridge,
+ struct netlink_ext_ack *extack)
+{
+ struct lan966x *lan966x = NULL;
+ bool has_foreign = false;
+ struct net_device *dev;
+ struct list_head *iter;
+
+ if (!netif_is_bridge_master(bridge))
+ return 0;
+
+ netdev_for_each_lower_dev(bridge, dev, iter) {
+ if (lan966x_netdevice_check(dev)) {
+ struct lan966x_port *port = netdev_priv(dev);
+
+ if (lan966x) {
+ /* Bridge already has at least one port of a
+ * lan966x switch inside it, check that it's
+ * the same instance of the driver.
+ */
+ if (port->lan966x != lan966x) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "Bridging between multiple lan966x switches disallowed");
+ return -EINVAL;
+ }
+ } else {
+ /* This is the first lan966x port inside this
+ * bridge
+ */
+ lan966x = port->lan966x;
+ }
+ } else {
+ has_foreign = true;
+ }
+
+ if (lan966x && has_foreign) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "Bridging lan966x ports with foreign interfaces disallowed");
+ return -EINVAL;
+ }
+ }
+
+ return 0;
+}
+
+static int lan966x_bridge_check(struct net_device *dev,
+ struct netdev_notifier_changeupper_info *info)
+{
+ return lan966x_foreign_bridging_check(info->upper_dev,
+ info->info.extack);
+}
+
+static int lan966x_netdevice_port_event(struct net_device *dev,
+ struct notifier_block *nb,
+ unsigned long event, void *ptr)
+{
+ int err = 0;
+
+ if (!lan966x_netdevice_check(dev)) {
+ if (event == NETDEV_CHANGEUPPER)
+ return lan966x_bridge_check(dev, ptr);
+ return 0;
+ }
+
+ switch (event) {
+ case NETDEV_PRECHANGEUPPER:
+ err = lan966x_port_prechangeupper(dev, ptr);
+ break;
+ case NETDEV_CHANGEUPPER:
+ err = lan966x_bridge_check(dev, ptr);
+ if (err)
+ return err;
+
+ err = lan966x_port_changeupper(dev, ptr);
+ break;
+ }
+
+ return err;
+}
+
+static int lan966x_netdevice_event(struct notifier_block *nb,
+ unsigned long event, void *ptr)
+{
+ struct net_device *dev = netdev_notifier_info_to_dev(ptr);
+ int ret;
+
+ ret = lan966x_netdevice_port_event(dev, nb, event, ptr);
+
+ return notifier_from_errno(ret);
+}
+
+static bool lan966x_foreign_dev_check(const struct net_device *dev,
+ const struct net_device *foreign_dev)
+{
+ struct lan966x_port *port = netdev_priv(dev);
+ struct lan966x *lan966x = port->lan966x;
+
+ if (netif_is_bridge_master(foreign_dev))
+ if (lan966x->bridge != foreign_dev)
+ return true;
+
+ return false;
+}
+
+static int lan966x_switchdev_event(struct notifier_block *nb,
+ unsigned long event, void *ptr)
+{
+ struct net_device *dev = switchdev_notifier_info_to_dev(ptr);
+ int err;
+
+ switch (event) {
+ case SWITCHDEV_PORT_ATTR_SET:
+ err = switchdev_handle_port_attr_set(dev, ptr,
+ lan966x_netdevice_check,
+ lan966x_port_attr_set);
+ return notifier_from_errno(err);
+ case SWITCHDEV_FDB_ADD_TO_DEVICE:
+ case SWITCHDEV_FDB_DEL_TO_DEVICE:
+ err = switchdev_handle_fdb_event_to_device(dev, event, ptr,
+ lan966x_netdevice_check,
+ lan966x_foreign_dev_check,
+ lan966x_handle_fdb,
+ NULL);
+ return notifier_from_errno(err);
+ }
+
+ return NOTIFY_DONE;
+}
+
+static int lan966x_handle_port_vlan_add(struct lan966x_port *port,
+ const struct switchdev_obj *obj)
+{
+ const struct switchdev_obj_port_vlan *v = SWITCHDEV_OBJ_PORT_VLAN(obj);
+ struct lan966x *lan966x = port->lan966x;
+
+ /* When adding a port to a vlan, we get a callback for the port but
+ * also for the bridge. When get the callback for the bridge just bail
+ * out. Then when the bridge is added to the vlan, then we get a
+ * callback here but in this case the flags has set:
+ * BRIDGE_VLAN_INFO_BRENTRY. In this case it means that the CPU
+ * port is added to the vlan, so the broadcast frames and unicast frames
+ * with dmac of the bridge should be foward to CPU.
+ */
+ if (netif_is_bridge_master(obj->orig_dev) &&
+ !(v->flags & BRIDGE_VLAN_INFO_BRENTRY))
+ return 0;
+
+ if (!netif_is_bridge_master(obj->orig_dev))
+ lan966x_vlan_port_add_vlan(port, v->vid,
+ v->flags & BRIDGE_VLAN_INFO_PVID,
+ v->flags & BRIDGE_VLAN_INFO_UNTAGGED);
+ else
+ lan966x_vlan_cpu_add_vlan(lan966x, v->vid);
+
+ return 0;
+}
+
+static int lan966x_handle_port_obj_add(struct net_device *dev, const void *ctx,
+ const struct switchdev_obj *obj,
+ struct netlink_ext_ack *extack)
+{
+ struct lan966x_port *port = netdev_priv(dev);
+ int err;
+
+ if (ctx && ctx != port)
+ return 0;
+
+ switch (obj->id) {
+ case SWITCHDEV_OBJ_ID_PORT_VLAN:
+ err = lan966x_handle_port_vlan_add(port, obj);
+ break;
+ case SWITCHDEV_OBJ_ID_PORT_MDB:
+ case SWITCHDEV_OBJ_ID_HOST_MDB:
+ err = lan966x_handle_port_mdb_add(port, obj);
+ break;
+ default:
+ err = -EOPNOTSUPP;
+ break;
+ }
+
+ return err;
+}
+
+static int lan966x_handle_port_vlan_del(struct lan966x_port *port,
+ const struct switchdev_obj *obj)
+{
+ const struct switchdev_obj_port_vlan *v = SWITCHDEV_OBJ_PORT_VLAN(obj);
+ struct lan966x *lan966x = port->lan966x;
+
+ if (!netif_is_bridge_master(obj->orig_dev))
+ lan966x_vlan_port_del_vlan(port, v->vid);
+ else
+ lan966x_vlan_cpu_del_vlan(lan966x, v->vid);
+
+ return 0;
+}
+
+static int lan966x_handle_port_obj_del(struct net_device *dev, const void *ctx,
+ const struct switchdev_obj *obj)
+{
+ struct lan966x_port *port = netdev_priv(dev);
+ int err;
+
+ if (ctx && ctx != port)
+ return 0;
+
+ switch (obj->id) {
+ case SWITCHDEV_OBJ_ID_PORT_VLAN:
+ err = lan966x_handle_port_vlan_del(port, obj);
+ break;
+ case SWITCHDEV_OBJ_ID_PORT_MDB:
+ case SWITCHDEV_OBJ_ID_HOST_MDB:
+ err = lan966x_handle_port_mdb_del(port, obj);
+ break;
+ default:
+ err = -EOPNOTSUPP;
+ break;
+ }
+
+ return err;
+}
+
+static int lan966x_switchdev_blocking_event(struct notifier_block *nb,
+ unsigned long event,
+ void *ptr)
+{
+ struct net_device *dev = switchdev_notifier_info_to_dev(ptr);
+ int err;
+
+ switch (event) {
+ case SWITCHDEV_PORT_OBJ_ADD:
+ err = switchdev_handle_port_obj_add(dev, ptr,
+ lan966x_netdevice_check,
+ lan966x_handle_port_obj_add);
+ return notifier_from_errno(err);
+ case SWITCHDEV_PORT_OBJ_DEL:
+ err = switchdev_handle_port_obj_del(dev, ptr,
+ lan966x_netdevice_check,
+ lan966x_handle_port_obj_del);
+ return notifier_from_errno(err);
+ case SWITCHDEV_PORT_ATTR_SET:
+ err = switchdev_handle_port_attr_set(dev, ptr,
+ lan966x_netdevice_check,
+ lan966x_port_attr_set);
+ return notifier_from_errno(err);
+ }
+
+ return NOTIFY_DONE;
+}
+
+static struct notifier_block lan966x_netdevice_nb __read_mostly = {
+ .notifier_call = lan966x_netdevice_event,
+};
+
+static struct notifier_block lan966x_switchdev_nb __read_mostly = {
+ .notifier_call = lan966x_switchdev_event,
+};
+
+static struct notifier_block lan966x_switchdev_blocking_nb __read_mostly = {
+ .notifier_call = lan966x_switchdev_blocking_event,
+};
+
+void lan966x_register_notifier_blocks(void)
+{
+ register_netdevice_notifier(&lan966x_netdevice_nb);
+ register_switchdev_notifier(&lan966x_switchdev_nb);
+ register_switchdev_blocking_notifier(&lan966x_switchdev_blocking_nb);
+}
+
+void lan966x_unregister_notifier_blocks(void)
+{
+ unregister_switchdev_blocking_notifier(&lan966x_switchdev_blocking_nb);
+ unregister_switchdev_notifier(&lan966x_switchdev_nb);
+ unregister_netdevice_notifier(&lan966x_netdevice_nb);
+}
diff --git a/drivers/net/ethernet/microchip/lan966x/lan966x_vlan.c b/drivers/net/ethernet/microchip/lan966x/lan966x_vlan.c
new file mode 100644
index 0000000..8d7260c
--- /dev/null
+++ b/drivers/net/ethernet/microchip/lan966x/lan966x_vlan.c
@@ -0,0 +1,317 @@
+// SPDX-License-Identifier: GPL-2.0+
+
+#include "lan966x_main.h"
+
+#define VLANACCESS_CMD_IDLE 0
+#define VLANACCESS_CMD_READ 1
+#define VLANACCESS_CMD_WRITE 2
+#define VLANACCESS_CMD_INIT 3
+
+static int lan966x_vlan_get_status(struct lan966x *lan966x)
+{
+ return lan_rd(lan966x, ANA_VLANACCESS);
+}
+
+static int lan966x_vlan_wait_for_completion(struct lan966x *lan966x)
+{
+ u32 val;
+
+ return readx_poll_timeout(lan966x_vlan_get_status,
+ lan966x, val,
+ (val & ANA_VLANACCESS_VLAN_TBL_CMD) ==
+ VLANACCESS_CMD_IDLE,
+ TABLE_UPDATE_SLEEP_US, TABLE_UPDATE_TIMEOUT_US);
+}
+
+static void lan966x_vlan_set_mask(struct lan966x *lan966x, u16 vid)
+{
+ u16 mask = lan966x->vlan_mask[vid];
+ bool cpu_dis;
+
+ cpu_dis = !(mask & BIT(CPU_PORT));
+
+ /* Set flags and the VID to configure */
+ lan_rmw(ANA_VLANTIDX_VLAN_PGID_CPU_DIS_SET(cpu_dis) |
+ ANA_VLANTIDX_V_INDEX_SET(vid),
+ ANA_VLANTIDX_VLAN_PGID_CPU_DIS |
+ ANA_VLANTIDX_V_INDEX,
+ lan966x, ANA_VLANTIDX);
+
+ /* Set the vlan port members mask */
+ lan_rmw(ANA_VLAN_PORT_MASK_VLAN_PORT_MASK_SET(mask),
+ ANA_VLAN_PORT_MASK_VLAN_PORT_MASK,
+ lan966x, ANA_VLAN_PORT_MASK);
+
+ /* Issue a write command */
+ lan_rmw(ANA_VLANACCESS_VLAN_TBL_CMD_SET(VLANACCESS_CMD_WRITE),
+ ANA_VLANACCESS_VLAN_TBL_CMD,
+ lan966x, ANA_VLANACCESS);
+
+ if (lan966x_vlan_wait_for_completion(lan966x))
+ dev_err(lan966x->dev, "Vlan set mask failed\n");
+}
+
+static void lan966x_vlan_port_add_vlan_mask(struct lan966x_port *port, u16 vid)
+{
+ struct lan966x *lan966x = port->lan966x;
+ u8 p = port->chip_port;
+
+ lan966x->vlan_mask[vid] |= BIT(p);
+ lan966x_vlan_set_mask(lan966x, vid);
+}
+
+static void lan966x_vlan_port_del_vlan_mask(struct lan966x_port *port, u16 vid)
+{
+ struct lan966x *lan966x = port->lan966x;
+ u8 p = port->chip_port;
+
+ lan966x->vlan_mask[vid] &= ~BIT(p);
+ lan966x_vlan_set_mask(lan966x, vid);
+}
+
+static bool lan966x_vlan_port_any_vlan_mask(struct lan966x *lan966x, u16 vid)
+{
+ return !!(lan966x->vlan_mask[vid] & ~BIT(CPU_PORT));
+}
+
+static void lan966x_vlan_cpu_add_vlan_mask(struct lan966x *lan966x, u16 vid)
+{
+ lan966x->vlan_mask[vid] |= BIT(CPU_PORT);
+ lan966x_vlan_set_mask(lan966x, vid);
+}
+
+static void lan966x_vlan_cpu_del_vlan_mask(struct lan966x *lan966x, u16 vid)
+{
+ lan966x->vlan_mask[vid] &= ~BIT(CPU_PORT);
+ lan966x_vlan_set_mask(lan966x, vid);
+}
+
+static void lan966x_vlan_cpu_add_cpu_vlan_mask(struct lan966x *lan966x, u16 vid)
+{
+ __set_bit(vid, lan966x->cpu_vlan_mask);
+}
+
+static void lan966x_vlan_cpu_del_cpu_vlan_mask(struct lan966x *lan966x, u16 vid)
+{
+ __clear_bit(vid, lan966x->cpu_vlan_mask);
+}
+
+bool lan966x_vlan_cpu_member_cpu_vlan_mask(struct lan966x *lan966x, u16 vid)
+{
+ return test_bit(vid, lan966x->cpu_vlan_mask);
+}
+
+static u16 lan966x_vlan_port_get_pvid(struct lan966x_port *port)
+{
+ struct lan966x *lan966x = port->lan966x;
+
+ if (!(lan966x->bridge_mask & BIT(port->chip_port)))
+ return HOST_PVID;
+
+ return port->vlan_aware ? port->pvid : UNAWARE_PVID;
+}
+
+int lan966x_vlan_port_set_vid(struct lan966x_port *port, u16 vid,
+ bool pvid, bool untagged)
+{
+ struct lan966x *lan966x = port->lan966x;
+
+ /* Egress vlan classification */
+ if (untagged && port->vid != vid) {
+ if (port->vid) {
+ dev_err(lan966x->dev,
+ "Port already has a native VLAN: %d\n",
+ port->vid);
+ return -EBUSY;
+ }
+ port->vid = vid;
+ }
+
+ /* Default ingress vlan classification */
+ if (pvid)
+ port->pvid = vid;
+
+ return 0;
+}
+
+static void lan966x_vlan_port_remove_vid(struct lan966x_port *port, u16 vid)
+{
+ if (port->pvid == vid)
+ port->pvid = 0;
+
+ if (port->vid == vid)
+ port->vid = 0;
+}
+
+void lan966x_vlan_port_set_vlan_aware(struct lan966x_port *port,
+ bool vlan_aware)
+{
+ port->vlan_aware = vlan_aware;
+}
+
+void lan966x_vlan_port_apply(struct lan966x_port *port)
+{
+ struct lan966x *lan966x = port->lan966x;
+ u16 pvid;
+ u32 val;
+
+ pvid = lan966x_vlan_port_get_pvid(port);
+
+ /* Ingress clasification (ANA_PORT_VLAN_CFG) */
+ /* Default vlan to classify for untagged frames (may be zero) */
+ val = ANA_VLAN_CFG_VLAN_VID_SET(pvid);
+ if (port->vlan_aware)
+ val |= ANA_VLAN_CFG_VLAN_AWARE_ENA_SET(1) |
+ ANA_VLAN_CFG_VLAN_POP_CNT_SET(1);
+
+ lan_rmw(val,
+ ANA_VLAN_CFG_VLAN_VID | ANA_VLAN_CFG_VLAN_AWARE_ENA |
+ ANA_VLAN_CFG_VLAN_POP_CNT,
+ lan966x, ANA_VLAN_CFG(port->chip_port));
+
+ /* Drop frames with multicast source address */
+ val = ANA_DROP_CFG_DROP_MC_SMAC_ENA_SET(1);
+ if (port->vlan_aware && !pvid)
+ /* If port is vlan-aware and tagged, drop untagged and priority
+ * tagged frames.
+ */
+ val |= ANA_DROP_CFG_DROP_UNTAGGED_ENA_SET(1) |
+ ANA_DROP_CFG_DROP_PRIO_S_TAGGED_ENA_SET(1) |
+ ANA_DROP_CFG_DROP_PRIO_C_TAGGED_ENA_SET(1);
+
+ lan_wr(val, lan966x, ANA_DROP_CFG(port->chip_port));
+
+ /* Egress configuration (REW_TAG_CFG): VLAN tag type to 8021Q */
+ val = REW_TAG_CFG_TAG_TPID_CFG_SET(0);
+ if (port->vlan_aware) {
+ if (port->vid)
+ /* Tag all frames except when VID == DEFAULT_VLAN */
+ val |= REW_TAG_CFG_TAG_CFG_SET(1);
+ else
+ val |= REW_TAG_CFG_TAG_CFG_SET(3);
+ }
+
+ /* Update only some bits in the register */
+ lan_rmw(val,
+ REW_TAG_CFG_TAG_TPID_CFG | REW_TAG_CFG_TAG_CFG,
+ lan966x, REW_TAG_CFG(port->chip_port));
+
+ /* Set default VLAN and tag type to 8021Q */
+ lan_rmw(REW_PORT_VLAN_CFG_PORT_TPID_SET(ETH_P_8021Q) |
+ REW_PORT_VLAN_CFG_PORT_VID_SET(port->vid),
+ REW_PORT_VLAN_CFG_PORT_TPID |
+ REW_PORT_VLAN_CFG_PORT_VID,
+ lan966x, REW_PORT_VLAN_CFG(port->chip_port));
+}
+
+void lan966x_vlan_port_add_vlan(struct lan966x_port *port,
+ u16 vid,
+ bool pvid,
+ bool untagged)
+{
+ struct lan966x *lan966x = port->lan966x;
+
+ /* If the CPU(br) is already part of the vlan then add the fdb
+ * entries in MAC table to copy the frames to the CPU(br).
+ * If the CPU(br) is not part of the vlan then it would
+ * just drop the frames.
+ */
+ if (lan966x_vlan_cpu_member_cpu_vlan_mask(lan966x, vid)) {
+ lan966x_vlan_cpu_add_vlan_mask(lan966x, vid);
+ lan966x_fdb_write_entries(lan966x, vid);
+ lan966x_mdb_write_entries(lan966x, vid);
+ }
+
+ lan966x_vlan_port_set_vid(port, vid, pvid, untagged);
+ lan966x_vlan_port_add_vlan_mask(port, vid);
+ lan966x_vlan_port_apply(port);
+}
+
+void lan966x_vlan_port_del_vlan(struct lan966x_port *port, u16 vid)
+{
+ struct lan966x *lan966x = port->lan966x;
+
+ lan966x_vlan_port_remove_vid(port, vid);
+ lan966x_vlan_port_del_vlan_mask(port, vid);
+ lan966x_vlan_port_apply(port);
+
+ /* In case there are no other ports in vlan then remove the CPU from
+ * that vlan but still keep it in the mask because it may be needed
+ * again then another port gets added in that vlan
+ */
+ if (!lan966x_vlan_port_any_vlan_mask(lan966x, vid)) {
+ lan966x_vlan_cpu_del_vlan_mask(lan966x, vid);
+ lan966x_fdb_erase_entries(lan966x, vid);
+ lan966x_mdb_erase_entries(lan966x, vid);
+ }
+}
+
+void lan966x_vlan_cpu_add_vlan(struct lan966x *lan966x, u16 vid)
+{
+ /* Add an entry in the MAC table for the CPU
+ * Add the CPU part of the vlan only if there is another port in that
+ * vlan otherwise all the broadcast frames in that vlan will go to CPU
+ * even if none of the ports are in the vlan and then the CPU will just
+ * need to discard these frames. It is required to store this
+ * information so when a front port is added then it would add also the
+ * CPU port.
+ */
+ if (lan966x_vlan_port_any_vlan_mask(lan966x, vid)) {
+ lan966x_vlan_cpu_add_vlan_mask(lan966x, vid);
+ lan966x_mdb_write_entries(lan966x, vid);
+ }
+
+ lan966x_vlan_cpu_add_cpu_vlan_mask(lan966x, vid);
+ lan966x_fdb_write_entries(lan966x, vid);
+}
+
+void lan966x_vlan_cpu_del_vlan(struct lan966x *lan966x, u16 vid)
+{
+ /* Remove the CPU part of the vlan */
+ lan966x_vlan_cpu_del_cpu_vlan_mask(lan966x, vid);
+ lan966x_vlan_cpu_del_vlan_mask(lan966x, vid);
+ lan966x_fdb_erase_entries(lan966x, vid);
+ lan966x_mdb_erase_entries(lan966x, vid);
+}
+
+void lan966x_vlan_init(struct lan966x *lan966x)
+{
+ u16 port, vid;
+
+ /* Clear VLAN table, by default all ports are members of all VLANS */
+ lan_rmw(ANA_VLANACCESS_VLAN_TBL_CMD_SET(VLANACCESS_CMD_INIT),
+ ANA_VLANACCESS_VLAN_TBL_CMD,
+ lan966x, ANA_VLANACCESS);
+ lan966x_vlan_wait_for_completion(lan966x);
+
+ for (vid = 1; vid < VLAN_N_VID; vid++) {
+ lan966x->vlan_mask[vid] = 0;
+ lan966x_vlan_set_mask(lan966x, vid);
+ }
+
+ /* Set all the ports + cpu to be part of HOST_PVID and UNAWARE_PVID */
+ lan966x->vlan_mask[HOST_PVID] =
+ GENMASK(lan966x->num_phys_ports - 1, 0) | BIT(CPU_PORT);
+ lan966x_vlan_set_mask(lan966x, HOST_PVID);
+
+ lan966x->vlan_mask[UNAWARE_PVID] =
+ GENMASK(lan966x->num_phys_ports - 1, 0) | BIT(CPU_PORT);
+ lan966x_vlan_set_mask(lan966x, UNAWARE_PVID);
+
+ lan966x_vlan_cpu_add_cpu_vlan_mask(lan966x, UNAWARE_PVID);
+
+ /* Configure the CPU port to be vlan aware */
+ lan_wr(ANA_VLAN_CFG_VLAN_VID_SET(0) |
+ ANA_VLAN_CFG_VLAN_AWARE_ENA_SET(1) |
+ ANA_VLAN_CFG_VLAN_POP_CNT_SET(1),
+ lan966x, ANA_VLAN_CFG(CPU_PORT));
+
+ /* Set vlan ingress filter mask to all ports */
+ lan_wr(GENMASK(lan966x->num_phys_ports, 0),
+ lan966x, ANA_VLANMASK);
+
+ for (port = 0; port < lan966x->num_phys_ports; port++) {
+ lan_wr(0, lan966x, REW_PORT_VLAN_CFG(port));
+ lan_wr(0, lan966x, REW_TAG_CFG(port));
+ }
+}
diff --git a/drivers/net/ethernet/microchip/sparx5/sparx5_main.c b/drivers/net/ethernet/microchip/sparx5/sparx5_main.c
index 4625d4f..1626627 100644
--- a/drivers/net/ethernet/microchip/sparx5/sparx5_main.c
+++ b/drivers/net/ethernet/microchip/sparx5/sparx5_main.c
@@ -292,6 +292,33 @@ static int sparx5_create_port(struct sparx5 *sparx5,
spx5_port->phylink_config.dev = &spx5_port->ndev->dev;
spx5_port->phylink_config.type = PHYLINK_NETDEV;
spx5_port->phylink_config.pcs_poll = true;
+ spx5_port->phylink_config.mac_capabilities = MAC_ASYM_PAUSE |
+ MAC_SYM_PAUSE | MAC_10 | MAC_100 | MAC_1000FD |
+ MAC_2500FD | MAC_5000FD | MAC_10000FD | MAC_25000FD;
+
+ __set_bit(PHY_INTERFACE_MODE_SGMII,
+ spx5_port->phylink_config.supported_interfaces);
+ __set_bit(PHY_INTERFACE_MODE_QSGMII,
+ spx5_port->phylink_config.supported_interfaces);
+ __set_bit(PHY_INTERFACE_MODE_1000BASEX,
+ spx5_port->phylink_config.supported_interfaces);
+ __set_bit(PHY_INTERFACE_MODE_2500BASEX,
+ spx5_port->phylink_config.supported_interfaces);
+
+ if (spx5_port->conf.bandwidth == SPEED_5000 ||
+ spx5_port->conf.bandwidth == SPEED_10000 ||
+ spx5_port->conf.bandwidth == SPEED_25000)
+ __set_bit(PHY_INTERFACE_MODE_5GBASER,
+ spx5_port->phylink_config.supported_interfaces);
+
+ if (spx5_port->conf.bandwidth == SPEED_10000 ||
+ spx5_port->conf.bandwidth == SPEED_25000)
+ __set_bit(PHY_INTERFACE_MODE_10GBASER,
+ spx5_port->phylink_config.supported_interfaces);
+
+ if (spx5_port->conf.bandwidth == SPEED_25000)
+ __set_bit(PHY_INTERFACE_MODE_25GBASER,
+ spx5_port->phylink_config.supported_interfaces);
phylink = phylink_create(&spx5_port->phylink_config,
of_fwnode_handle(config->node),
diff --git a/drivers/net/ethernet/microchip/sparx5/sparx5_phylink.c b/drivers/net/ethernet/microchip/sparx5/sparx5_phylink.c
index fb74752..8ba33bc 100644
--- a/drivers/net/ethernet/microchip/sparx5/sparx5_phylink.c
+++ b/drivers/net/ethernet/microchip/sparx5/sparx5_phylink.c
@@ -26,79 +26,6 @@ static bool port_conf_has_changed(struct sparx5_port_config *a, struct sparx5_po
return false;
}
-static void sparx5_phylink_validate(struct phylink_config *config,
- unsigned long *supported,
- struct phylink_link_state *state)
-{
- struct sparx5_port *port = netdev_priv(to_net_dev(config->dev));
- __ETHTOOL_DECLARE_LINK_MODE_MASK(mask) = { 0, };
-
- phylink_set(mask, Autoneg);
- phylink_set_port_modes(mask);
- phylink_set(mask, Pause);
- phylink_set(mask, Asym_Pause);
-
- switch (state->interface) {
- case PHY_INTERFACE_MODE_5GBASER:
- case PHY_INTERFACE_MODE_10GBASER:
- case PHY_INTERFACE_MODE_25GBASER:
- case PHY_INTERFACE_MODE_NA:
- if (port->conf.bandwidth == SPEED_5000)
- phylink_set(mask, 5000baseT_Full);
- if (port->conf.bandwidth == SPEED_10000) {
- phylink_set(mask, 5000baseT_Full);
- phylink_set(mask, 10000baseT_Full);
- phylink_set(mask, 10000baseCR_Full);
- phylink_set(mask, 10000baseSR_Full);
- phylink_set(mask, 10000baseLR_Full);
- phylink_set(mask, 10000baseLRM_Full);
- phylink_set(mask, 10000baseER_Full);
- }
- if (port->conf.bandwidth == SPEED_25000) {
- phylink_set(mask, 5000baseT_Full);
- phylink_set(mask, 10000baseT_Full);
- phylink_set(mask, 10000baseCR_Full);
- phylink_set(mask, 10000baseSR_Full);
- phylink_set(mask, 10000baseLR_Full);
- phylink_set(mask, 10000baseLRM_Full);
- phylink_set(mask, 10000baseER_Full);
- phylink_set(mask, 25000baseCR_Full);
- phylink_set(mask, 25000baseSR_Full);
- }
- if (state->interface != PHY_INTERFACE_MODE_NA)
- break;
- fallthrough;
- case PHY_INTERFACE_MODE_SGMII:
- case PHY_INTERFACE_MODE_QSGMII:
- phylink_set(mask, 10baseT_Half);
- phylink_set(mask, 10baseT_Full);
- phylink_set(mask, 100baseT_Half);
- phylink_set(mask, 100baseT_Full);
- phylink_set(mask, 1000baseT_Full);
- phylink_set(mask, 1000baseX_Full);
- if (state->interface != PHY_INTERFACE_MODE_NA)
- break;
- fallthrough;
- case PHY_INTERFACE_MODE_1000BASEX:
- case PHY_INTERFACE_MODE_2500BASEX:
- if (state->interface != PHY_INTERFACE_MODE_2500BASEX) {
- phylink_set(mask, 1000baseT_Full);
- phylink_set(mask, 1000baseX_Full);
- }
- if (state->interface == PHY_INTERFACE_MODE_2500BASEX ||
- state->interface == PHY_INTERFACE_MODE_NA) {
- phylink_set(mask, 2500baseT_Full);
- phylink_set(mask, 2500baseX_Full);
- }
- break;
- default:
- linkmode_zero(supported);
- return;
- }
- linkmode_and(supported, supported, mask);
- linkmode_and(state->advertising, state->advertising, mask);
-}
-
static void sparx5_phylink_mac_config(struct phylink_config *config,
unsigned int mode,
const struct phylink_link_state *state)
@@ -202,7 +129,7 @@ const struct phylink_pcs_ops sparx5_phylink_pcs_ops = {
};
const struct phylink_mac_ops sparx5_phylink_mac_ops = {
- .validate = sparx5_phylink_validate,
+ .validate = phylink_generic_validate,
.mac_config = sparx5_phylink_mac_config,
.mac_link_down = sparx5_phylink_mac_link_down,
.mac_link_up = sparx5_phylink_mac_link_up,
diff --git a/drivers/net/ethernet/microsoft/mana/Makefile b/drivers/net/ethernet/microsoft/mana/Makefile
index 0edd5bb..e16a422 100644
--- a/drivers/net/ethernet/microsoft/mana/Makefile
+++ b/drivers/net/ethernet/microsoft/mana/Makefile
@@ -3,4 +3,4 @@
# Makefile for the Microsoft Azure Network Adapter driver
obj-$(CONFIG_MICROSOFT_MANA) += mana.o
-mana-objs := gdma_main.o shm_channel.o hw_channel.o mana_en.o mana_ethtool.o
+mana-objs := gdma_main.o shm_channel.o hw_channel.o mana_en.o mana_ethtool.o mana_bpf.o
diff --git a/drivers/net/ethernet/microsoft/mana/mana.h b/drivers/net/ethernet/microsoft/mana/mana.h
index d047ee8..9a12607 100644
--- a/drivers/net/ethernet/microsoft/mana/mana.h
+++ b/drivers/net/ethernet/microsoft/mana/mana.h
@@ -289,6 +289,8 @@ struct mana_rxq {
struct mana_cq rx_cq;
+ struct completion fence_event;
+
struct net_device *ndev;
/* Total number of receive buffers to be allocated */
@@ -298,6 +300,9 @@ struct mana_rxq {
struct mana_stats stats;
+ struct bpf_prog __rcu *bpf_prog;
+ struct xdp_rxq_info xdp_rxq;
+
/* MUST BE THE LAST MEMBER:
* Each receive buffer has an associated mana_recv_buf_oob.
*/
@@ -353,6 +358,8 @@ struct mana_port_context {
/* This points to an array of num_queues of RQ pointers. */
struct mana_rxq **rxqs;
+ struct bpf_prog *bpf_prog;
+
/* Create num_queues EQs, SQs, SQ-CQs, RQs and RQ-CQs, respectively. */
unsigned int max_queues;
unsigned int num_queues;
@@ -367,6 +374,7 @@ struct mana_port_context {
struct mana_ethtool_stats eth_stats;
};
+int mana_start_xmit(struct sk_buff *skb, struct net_device *ndev);
int mana_config_rss(struct mana_port_context *ac, enum TRI_STATE rx,
bool update_hash, bool update_tab);
@@ -377,6 +385,13 @@ int mana_detach(struct net_device *ndev, bool from_close);
int mana_probe(struct gdma_dev *gd, bool resuming);
void mana_remove(struct gdma_dev *gd, bool suspending);
+void mana_xdp_tx(struct sk_buff *skb, struct net_device *ndev);
+u32 mana_run_xdp(struct net_device *ndev, struct mana_rxq *rxq,
+ struct xdp_buff *xdp, void *buf_va, uint pkt_len);
+struct bpf_prog *mana_xdp_get(struct mana_port_context *apc);
+void mana_chn_setxdp(struct mana_port_context *apc, struct bpf_prog *prog);
+int mana_bpf(struct net_device *ndev, struct netdev_bpf *bpf);
+
extern const struct ethtool_ops mana_ethtool_ops;
struct mana_obj_spec {
diff --git a/drivers/net/ethernet/microsoft/mana/mana_bpf.c b/drivers/net/ethernet/microsoft/mana/mana_bpf.c
new file mode 100644
index 0000000..1d2f948
--- /dev/null
+++ b/drivers/net/ethernet/microsoft/mana/mana_bpf.c
@@ -0,0 +1,162 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause
+/* Copyright (c) 2021, Microsoft Corporation. */
+
+#include <linux/inetdevice.h>
+#include <linux/etherdevice.h>
+#include <linux/mm.h>
+#include <linux/bpf.h>
+#include <linux/bpf_trace.h>
+#include <net/xdp.h>
+
+#include "mana.h"
+
+void mana_xdp_tx(struct sk_buff *skb, struct net_device *ndev)
+{
+ u16 txq_idx = skb_get_queue_mapping(skb);
+ struct netdev_queue *ndevtxq;
+ int rc;
+
+ __skb_push(skb, ETH_HLEN);
+
+ ndevtxq = netdev_get_tx_queue(ndev, txq_idx);
+ __netif_tx_lock(ndevtxq, smp_processor_id());
+
+ rc = mana_start_xmit(skb, ndev);
+
+ __netif_tx_unlock(ndevtxq);
+
+ if (dev_xmit_complete(rc))
+ return;
+
+ dev_kfree_skb_any(skb);
+ ndev->stats.tx_dropped++;
+}
+
+u32 mana_run_xdp(struct net_device *ndev, struct mana_rxq *rxq,
+ struct xdp_buff *xdp, void *buf_va, uint pkt_len)
+{
+ struct bpf_prog *prog;
+ u32 act = XDP_PASS;
+
+ rcu_read_lock();
+ prog = rcu_dereference(rxq->bpf_prog);
+
+ if (!prog)
+ goto out;
+
+ xdp_init_buff(xdp, PAGE_SIZE, &rxq->xdp_rxq);
+ xdp_prepare_buff(xdp, buf_va, XDP_PACKET_HEADROOM, pkt_len, false);
+
+ act = bpf_prog_run_xdp(prog, xdp);
+
+ switch (act) {
+ case XDP_PASS:
+ case XDP_TX:
+ case XDP_DROP:
+ break;
+
+ case XDP_ABORTED:
+ trace_xdp_exception(ndev, prog, act);
+ break;
+
+ default:
+ bpf_warn_invalid_xdp_action(ndev, prog, act);
+ }
+
+out:
+ rcu_read_unlock();
+
+ return act;
+}
+
+static unsigned int mana_xdp_fraglen(unsigned int len)
+{
+ return SKB_DATA_ALIGN(len) +
+ SKB_DATA_ALIGN(sizeof(struct skb_shared_info));
+}
+
+struct bpf_prog *mana_xdp_get(struct mana_port_context *apc)
+{
+ ASSERT_RTNL();
+
+ return apc->bpf_prog;
+}
+
+static struct bpf_prog *mana_chn_xdp_get(struct mana_port_context *apc)
+{
+ return rtnl_dereference(apc->rxqs[0]->bpf_prog);
+}
+
+/* Set xdp program on channels */
+void mana_chn_setxdp(struct mana_port_context *apc, struct bpf_prog *prog)
+{
+ struct bpf_prog *old_prog = mana_chn_xdp_get(apc);
+ unsigned int num_queues = apc->num_queues;
+ int i;
+
+ ASSERT_RTNL();
+
+ if (old_prog == prog)
+ return;
+
+ if (prog)
+ bpf_prog_add(prog, num_queues);
+
+ for (i = 0; i < num_queues; i++)
+ rcu_assign_pointer(apc->rxqs[i]->bpf_prog, prog);
+
+ if (old_prog)
+ for (i = 0; i < num_queues; i++)
+ bpf_prog_put(old_prog);
+}
+
+static int mana_xdp_set(struct net_device *ndev, struct bpf_prog *prog,
+ struct netlink_ext_ack *extack)
+{
+ struct mana_port_context *apc = netdev_priv(ndev);
+ struct bpf_prog *old_prog;
+ int buf_max;
+
+ old_prog = mana_xdp_get(apc);
+
+ if (!old_prog && !prog)
+ return 0;
+
+ buf_max = XDP_PACKET_HEADROOM + mana_xdp_fraglen(ndev->mtu + ETH_HLEN);
+ if (prog && buf_max > PAGE_SIZE) {
+ netdev_err(ndev, "XDP: mtu:%u too large, buf_max:%u\n",
+ ndev->mtu, buf_max);
+ NL_SET_ERR_MSG_MOD(extack, "XDP: mtu too large");
+
+ return -EOPNOTSUPP;
+ }
+
+ /* One refcnt of the prog is hold by the caller already, so
+ * don't increase refcnt for this one.
+ */
+ apc->bpf_prog = prog;
+
+ if (old_prog)
+ bpf_prog_put(old_prog);
+
+ if (apc->port_is_up)
+ mana_chn_setxdp(apc, prog);
+
+ return 0;
+}
+
+int mana_bpf(struct net_device *ndev, struct netdev_bpf *bpf)
+{
+ struct netlink_ext_ack *extack = bpf->extack;
+ int ret;
+
+ switch (bpf->command) {
+ case XDP_SETUP_PROG:
+ return mana_xdp_set(ndev, bpf->prog, extack);
+
+ default:
+ return -EOPNOTSUPP;
+ }
+
+ return ret;
+}
diff --git a/drivers/net/ethernet/microsoft/mana/mana_en.c b/drivers/net/ethernet/microsoft/mana/mana_en.c
index 72cbf45..498d0f9 100644
--- a/drivers/net/ethernet/microsoft/mana/mana_en.c
+++ b/drivers/net/ethernet/microsoft/mana/mana_en.c
@@ -1,6 +1,8 @@
// SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause
/* Copyright (c) 2021, Microsoft Corporation. */
+#include <uapi/linux/bpf.h>
+
#include <linux/inetdevice.h>
#include <linux/etherdevice.h>
#include <linux/ethtool.h>
@@ -125,7 +127,7 @@ static int mana_map_skb(struct sk_buff *skb, struct mana_port_context *apc,
return -ENOMEM;
}
-static int mana_start_xmit(struct sk_buff *skb, struct net_device *ndev)
+int mana_start_xmit(struct sk_buff *skb, struct net_device *ndev)
{
enum mana_tx_pkt_format pkt_fmt = MANA_SHORT_PKT_FMT;
struct mana_port_context *apc = netdev_priv(ndev);
@@ -378,6 +380,7 @@ static const struct net_device_ops mana_devops = {
.ndo_start_xmit = mana_start_xmit,
.ndo_validate_addr = eth_validate_addr,
.ndo_get_stats64 = mana_get_stats64,
+ .ndo_bpf = mana_bpf,
};
static void mana_cleanup_port_context(struct mana_port_context *apc)
@@ -749,6 +752,61 @@ static int mana_create_eq(struct mana_context *ac)
return err;
}
+static int mana_fence_rq(struct mana_port_context *apc, struct mana_rxq *rxq)
+{
+ struct mana_fence_rq_resp resp = {};
+ struct mana_fence_rq_req req = {};
+ int err;
+
+ init_completion(&rxq->fence_event);
+
+ mana_gd_init_req_hdr(&req.hdr, MANA_FENCE_RQ,
+ sizeof(req), sizeof(resp));
+ req.wq_obj_handle = rxq->rxobj;
+
+ err = mana_send_request(apc->ac, &req, sizeof(req), &resp,
+ sizeof(resp));
+ if (err) {
+ netdev_err(apc->ndev, "Failed to fence RQ %u: %d\n",
+ rxq->rxq_idx, err);
+ return err;
+ }
+
+ err = mana_verify_resp_hdr(&resp.hdr, MANA_FENCE_RQ, sizeof(resp));
+ if (err || resp.hdr.status) {
+ netdev_err(apc->ndev, "Failed to fence RQ %u: %d, 0x%x\n",
+ rxq->rxq_idx, err, resp.hdr.status);
+ if (!err)
+ err = -EPROTO;
+
+ return err;
+ }
+
+ if (wait_for_completion_timeout(&rxq->fence_event, 10 * HZ) == 0) {
+ netdev_err(apc->ndev, "Failed to fence RQ %u: timed out\n",
+ rxq->rxq_idx);
+ return -ETIMEDOUT;
+ }
+
+ return 0;
+}
+
+static void mana_fence_rqs(struct mana_port_context *apc)
+{
+ unsigned int rxq_idx;
+ struct mana_rxq *rxq;
+ int err;
+
+ for (rxq_idx = 0; rxq_idx < apc->num_queues; rxq_idx++) {
+ rxq = apc->rxqs[rxq_idx];
+ err = mana_fence_rq(apc, rxq);
+
+ /* In case of any error, use sleep instead. */
+ if (err)
+ msleep(100);
+ }
+}
+
static int mana_move_wq_tail(struct gdma_queue *wq, u32 num_units)
{
u32 used_space_old;
@@ -906,6 +964,25 @@ static void mana_post_pkt_rxq(struct mana_rxq *rxq)
WARN_ON_ONCE(recv_buf_oob->wqe_inf.wqe_size_in_bu != 1);
}
+static struct sk_buff *mana_build_skb(void *buf_va, uint pkt_len,
+ struct xdp_buff *xdp)
+{
+ struct sk_buff *skb = build_skb(buf_va, PAGE_SIZE);
+
+ if (!skb)
+ return NULL;
+
+ if (xdp->data_hard_start) {
+ skb_reserve(skb, xdp->data - xdp->data_hard_start);
+ skb_put(skb, xdp->data_end - xdp->data);
+ } else {
+ skb_reserve(skb, XDP_PACKET_HEADROOM);
+ skb_put(skb, pkt_len);
+ }
+
+ return skb;
+}
+
static void mana_rx_skb(void *buf_va, struct mana_rxcomp_oob *cqe,
struct mana_rxq *rxq)
{
@@ -914,8 +991,10 @@ static void mana_rx_skb(void *buf_va, struct mana_rxcomp_oob *cqe,
uint pkt_len = cqe->ppi[0].pkt_len;
u16 rxq_idx = rxq->rxq_idx;
struct napi_struct *napi;
+ struct xdp_buff xdp = {};
struct sk_buff *skb;
u32 hash_value;
+ u32 act;
rxq->rx_cq.work_done++;
napi = &rxq->rx_cq.napi;
@@ -925,15 +1004,16 @@ static void mana_rx_skb(void *buf_va, struct mana_rxcomp_oob *cqe,
return;
}
- skb = build_skb(buf_va, PAGE_SIZE);
+ act = mana_run_xdp(ndev, rxq, &xdp, buf_va, pkt_len);
- if (!skb) {
- free_page((unsigned long)buf_va);
- ++ndev->stats.rx_dropped;
- return;
- }
+ if (act != XDP_PASS && act != XDP_TX)
+ goto drop;
- skb_put(skb, pkt_len);
+ skb = mana_build_skb(buf_va, pkt_len, &xdp);
+
+ if (!skb)
+ goto drop;
+
skb->dev = napi->dev;
skb->protocol = eth_type_trans(skb, ndev);
@@ -954,12 +1034,24 @@ static void mana_rx_skb(void *buf_va, struct mana_rxcomp_oob *cqe,
skb_set_hash(skb, hash_value, PKT_HASH_TYPE_L3);
}
+ if (act == XDP_TX) {
+ skb_set_queue_mapping(skb, rxq_idx);
+ mana_xdp_tx(skb, ndev);
+ return;
+ }
+
napi_gro_receive(napi, skb);
u64_stats_update_begin(&rx_stats->syncp);
rx_stats->packets++;
rx_stats->bytes += pkt_len;
u64_stats_update_end(&rx_stats->syncp);
+ return;
+
+drop:
+ free_page((unsigned long)buf_va);
+ ++ndev->stats.rx_dropped;
+ return;
}
static void mana_process_rx_cqe(struct mana_rxq *rxq, struct mana_cq *cq,
@@ -988,7 +1080,7 @@ static void mana_process_rx_cqe(struct mana_rxq *rxq, struct mana_cq *cq,
return;
case CQE_RX_OBJECT_FENCE:
- netdev_err(ndev, "RX Fencing is unsupported\n");
+ complete(&rxq->fence_event);
return;
default:
@@ -1016,7 +1108,7 @@ static void mana_process_rx_cqe(struct mana_rxq *rxq, struct mana_cq *cq,
new_page = alloc_page(GFP_ATOMIC);
if (new_page) {
- da = dma_map_page(dev, new_page, 0, rxq->datasize,
+ da = dma_map_page(dev, new_page, XDP_PACKET_HEADROOM, rxq->datasize,
DMA_FROM_DEVICE);
if (dma_mapping_error(dev, da)) {
@@ -1291,6 +1383,9 @@ static void mana_destroy_rxq(struct mana_port_context *apc,
napi_synchronize(napi);
napi_disable(napi);
+
+ xdp_rxq_info_unreg(&rxq->xdp_rxq);
+
netif_napi_del(napi);
mana_destroy_wq_obj(apc, GDMA_RQ, rxq->rxobj);
@@ -1342,7 +1437,8 @@ static int mana_alloc_rx_wqe(struct mana_port_context *apc,
if (!page)
return -ENOMEM;
- da = dma_map_page(dev, page, 0, rxq->datasize, DMA_FROM_DEVICE);
+ da = dma_map_page(dev, page, XDP_PACKET_HEADROOM, rxq->datasize,
+ DMA_FROM_DEVICE);
if (dma_mapping_error(dev, da)) {
__free_page(page);
@@ -1485,6 +1581,12 @@ static struct mana_rxq *mana_create_rxq(struct mana_port_context *apc,
gc->cq_table[cq->gdma_id] = cq->gdma_cq;
netif_napi_add(ndev, &cq->napi, mana_poll, 1);
+
+ WARN_ON(xdp_rxq_info_reg(&rxq->xdp_rxq, ndev, rxq_idx,
+ cq->napi.napi_id));
+ WARN_ON(xdp_rxq_info_reg_mem_model(&rxq->xdp_rxq,
+ MEM_TYPE_PAGE_SHARED, NULL));
+
napi_enable(&cq->napi);
mana_gd_ring_cq(cq->gdma_cq, SET_ARM_BIT);
@@ -1572,6 +1674,7 @@ int mana_config_rss(struct mana_port_context *apc, enum TRI_STATE rx,
bool update_hash, bool update_tab)
{
u32 queue_idx;
+ int err;
int i;
if (update_tab) {
@@ -1581,7 +1684,13 @@ int mana_config_rss(struct mana_port_context *apc, enum TRI_STATE rx,
}
}
- return mana_cfg_vport_steering(apc, rx, true, update_hash, update_tab);
+ err = mana_cfg_vport_steering(apc, rx, true, update_hash, update_tab);
+ if (err)
+ return err;
+
+ mana_fence_rqs(apc);
+
+ return 0;
}
static int mana_init_port(struct net_device *ndev)
@@ -1650,6 +1759,8 @@ int mana_alloc_queues(struct net_device *ndev)
if (err)
goto destroy_vport;
+ mana_chn_setxdp(apc, mana_xdp_get(apc));
+
return 0;
destroy_vport:
@@ -1698,6 +1809,8 @@ static int mana_dealloc_queues(struct net_device *ndev)
if (apc->port_is_up)
return -EINVAL;
+ mana_chn_setxdp(apc, NULL);
+
/* No packet can be transmitted now since apc->port_is_up is false.
* There is still a tiny chance that mana_poll_tx_cq() can re-enable
* a txq because it may not timely see apc->port_is_up being cleared
@@ -1724,9 +1837,6 @@ static int mana_dealloc_queues(struct net_device *ndev)
return err;
}
- /* TODO: Implement RX fencing */
- ssleep(1);
-
mana_destroy_vport(apc);
return 0;
diff --git a/drivers/net/ethernet/mscc/Makefile b/drivers/net/ethernet/mscc/Makefile
index 722c276..41b34a5 100644
--- a/drivers/net/ethernet/mscc/Makefile
+++ b/drivers/net/ethernet/mscc/Makefile
@@ -7,9 +7,11 @@
ocelot_vcap.o \
ocelot_flower.o \
ocelot_ptp.o \
- ocelot_devlink.o
+ ocelot_devlink.o \
+ vsc7514_regs.o
mscc_ocelot_switch_lib-$(CONFIG_BRIDGE_MRP) += ocelot_mrp.o
obj-$(CONFIG_MSCC_OCELOT_SWITCH) += mscc_ocelot.o
mscc_ocelot-y := \
+ ocelot_fdma.o \
ocelot_vsc7514.o \
ocelot_net.o
diff --git a/drivers/net/ethernet/mscc/ocelot.c b/drivers/net/ethernet/mscc/ocelot.c
index 1e4ad95..b1311b6 100644
--- a/drivers/net/ethernet/mscc/ocelot.c
+++ b/drivers/net/ethernet/mscc/ocelot.c
@@ -61,9 +61,9 @@ static void ocelot_mact_select(struct ocelot *ocelot,
}
-int ocelot_mact_learn(struct ocelot *ocelot, int port,
- const unsigned char mac[ETH_ALEN],
- unsigned int vid, enum macaccess_entry_type type)
+static int __ocelot_mact_learn(struct ocelot *ocelot, int port,
+ const unsigned char mac[ETH_ALEN],
+ unsigned int vid, enum macaccess_entry_type type)
{
u32 cmd = ANA_TABLES_MACACCESS_VALID |
ANA_TABLES_MACACCESS_DEST_IDX(port) |
@@ -83,8 +83,6 @@ int ocelot_mact_learn(struct ocelot *ocelot, int port,
if (mc_ports & BIT(ocelot->num_phys_ports))
cmd |= ANA_TABLES_MACACCESS_MAC_CPU_COPY;
- mutex_lock(&ocelot->mact_lock);
-
ocelot_mact_select(ocelot, mac, vid);
/* Issue a write command */
@@ -92,9 +90,20 @@ int ocelot_mact_learn(struct ocelot *ocelot, int port,
err = ocelot_mact_wait_for_completion(ocelot);
+ return err;
+}
+
+int ocelot_mact_learn(struct ocelot *ocelot, int port,
+ const unsigned char mac[ETH_ALEN],
+ unsigned int vid, enum macaccess_entry_type type)
+{
+ int ret;
+
+ mutex_lock(&ocelot->mact_lock);
+ ret = __ocelot_mact_learn(ocelot, port, mac, vid, type);
mutex_unlock(&ocelot->mact_lock);
- return err;
+ return ret;
}
EXPORT_SYMBOL(ocelot_mact_learn);
@@ -120,6 +129,66 @@ int ocelot_mact_forget(struct ocelot *ocelot,
}
EXPORT_SYMBOL(ocelot_mact_forget);
+int ocelot_mact_lookup(struct ocelot *ocelot, int *dst_idx,
+ const unsigned char mac[ETH_ALEN],
+ unsigned int vid, enum macaccess_entry_type *type)
+{
+ int val;
+
+ mutex_lock(&ocelot->mact_lock);
+
+ ocelot_mact_select(ocelot, mac, vid);
+
+ /* Issue a read command with MACACCESS_VALID=1. */
+ ocelot_write(ocelot, ANA_TABLES_MACACCESS_VALID |
+ ANA_TABLES_MACACCESS_MAC_TABLE_CMD(MACACCESS_CMD_READ),
+ ANA_TABLES_MACACCESS);
+
+ if (ocelot_mact_wait_for_completion(ocelot)) {
+ mutex_unlock(&ocelot->mact_lock);
+ return -ETIMEDOUT;
+ }
+
+ /* Read back the entry flags */
+ val = ocelot_read(ocelot, ANA_TABLES_MACACCESS);
+
+ mutex_unlock(&ocelot->mact_lock);
+
+ if (!(val & ANA_TABLES_MACACCESS_VALID))
+ return -ENOENT;
+
+ *dst_idx = ANA_TABLES_MACACCESS_DEST_IDX_X(val);
+ *type = ANA_TABLES_MACACCESS_ENTRYTYPE_X(val);
+
+ return 0;
+}
+EXPORT_SYMBOL(ocelot_mact_lookup);
+
+int ocelot_mact_learn_streamdata(struct ocelot *ocelot, int dst_idx,
+ const unsigned char mac[ETH_ALEN],
+ unsigned int vid,
+ enum macaccess_entry_type type,
+ int sfid, int ssid)
+{
+ int ret;
+
+ mutex_lock(&ocelot->mact_lock);
+
+ ocelot_write(ocelot,
+ (sfid < 0 ? 0 : ANA_TABLES_STREAMDATA_SFID_VALID) |
+ ANA_TABLES_STREAMDATA_SFID(sfid) |
+ (ssid < 0 ? 0 : ANA_TABLES_STREAMDATA_SSID_VALID) |
+ ANA_TABLES_STREAMDATA_SSID(ssid),
+ ANA_TABLES_STREAMDATA);
+
+ ret = __ocelot_mact_learn(ocelot, dst_idx, mac, vid, type);
+
+ mutex_unlock(&ocelot->mact_lock);
+
+ return ret;
+}
+EXPORT_SYMBOL(ocelot_mact_learn_streamdata);
+
static void ocelot_mact_init(struct ocelot *ocelot)
{
/* Configure the learning mode entries attributes:
@@ -594,9 +663,17 @@ void ocelot_phylink_mac_link_down(struct ocelot *ocelot, int port,
struct ocelot_port *ocelot_port = ocelot->ports[port];
int err;
+ ocelot_port->speed = SPEED_UNKNOWN;
+
ocelot_port_rmwl(ocelot_port, 0, DEV_MAC_ENA_CFG_RX_ENA,
DEV_MAC_ENA_CFG);
+ if (ocelot->ops->cut_through_fwd) {
+ mutex_lock(&ocelot->fwd_domain_lock);
+ ocelot->ops->cut_through_fwd(ocelot);
+ mutex_unlock(&ocelot->fwd_domain_lock);
+ }
+
ocelot_fields_write(ocelot, port, QSYS_SWITCH_PORT_MODE_PORT_ENA, 0);
err = ocelot_port_flush(ocelot, port);
@@ -628,6 +705,8 @@ void ocelot_phylink_mac_link_up(struct ocelot *ocelot, int port,
int mac_speed, mode = 0;
u32 mac_fc_cfg;
+ ocelot_port->speed = speed;
+
/* The MAC might be integrated in systems where the MAC speed is fixed
* and it's the PCS who is performing the rate adaptation, so we have
* to write "1000Mbps" into the LINK_SPEED field of DEV_CLOCK_CFG
@@ -700,6 +779,15 @@ void ocelot_phylink_mac_link_up(struct ocelot *ocelot, int port,
ocelot_port_writel(ocelot_port, DEV_MAC_ENA_CFG_RX_ENA |
DEV_MAC_ENA_CFG_TX_ENA, DEV_MAC_ENA_CFG);
+ /* If the port supports cut-through forwarding, update the masks before
+ * enabling forwarding on the port.
+ */
+ if (ocelot->ops->cut_through_fwd) {
+ mutex_lock(&ocelot->fwd_domain_lock);
+ ocelot->ops->cut_through_fwd(ocelot);
+ mutex_unlock(&ocelot->fwd_domain_lock);
+ }
+
/* Core: Enable port for frame transfer */
ocelot_fields_write(ocelot, port,
QSYS_SWITCH_PORT_MODE_PORT_ENA, 1);
@@ -966,14 +1054,34 @@ static int ocelot_xtr_poll_xfh(struct ocelot *ocelot, int grp, u32 *xfh)
return 0;
}
-int ocelot_xtr_poll_frame(struct ocelot *ocelot, int grp, struct sk_buff **nskb)
+void ocelot_ptp_rx_timestamp(struct ocelot *ocelot, struct sk_buff *skb,
+ u64 timestamp)
{
struct skb_shared_hwtstamps *shhwtstamps;
u64 tod_in_ns, full_ts_in_ns;
+ struct timespec64 ts;
+
+ ocelot_ptp_gettime64(&ocelot->ptp_info, &ts);
+
+ tod_in_ns = ktime_set(ts.tv_sec, ts.tv_nsec);
+ if ((tod_in_ns & 0xffffffff) < timestamp)
+ full_ts_in_ns = (((tod_in_ns >> 32) - 1) << 32) |
+ timestamp;
+ else
+ full_ts_in_ns = (tod_in_ns & GENMASK_ULL(63, 32)) |
+ timestamp;
+
+ shhwtstamps = skb_hwtstamps(skb);
+ memset(shhwtstamps, 0, sizeof(struct skb_shared_hwtstamps));
+ shhwtstamps->hwtstamp = full_ts_in_ns;
+}
+EXPORT_SYMBOL(ocelot_ptp_rx_timestamp);
+
+int ocelot_xtr_poll_frame(struct ocelot *ocelot, int grp, struct sk_buff **nskb)
+{
u64 timestamp, src_port, len;
u32 xfh[OCELOT_TAG_LEN / 4];
struct net_device *dev;
- struct timespec64 ts;
struct sk_buff *skb;
int sz, buf_len;
u32 val, *buf;
@@ -1029,21 +1137,8 @@ int ocelot_xtr_poll_frame(struct ocelot *ocelot, int grp, struct sk_buff **nskb)
*buf = val;
}
- if (ocelot->ptp) {
- ocelot_ptp_gettime64(&ocelot->ptp_info, &ts);
-
- tod_in_ns = ktime_set(ts.tv_sec, ts.tv_nsec);
- if ((tod_in_ns & 0xffffffff) < timestamp)
- full_ts_in_ns = (((tod_in_ns >> 32) - 1) << 32) |
- timestamp;
- else
- full_ts_in_ns = (tod_in_ns & GENMASK_ULL(63, 32)) |
- timestamp;
-
- shhwtstamps = skb_hwtstamps(skb);
- memset(shhwtstamps, 0, sizeof(struct skb_shared_hwtstamps));
- shhwtstamps->hwtstamp = full_ts_in_ns;
- }
+ if (ocelot->ptp)
+ ocelot_ptp_rx_timestamp(ocelot, skb, timestamp);
/* Everything we see on an interface that is in the HW bridge
* has already been forwarded.
@@ -1076,6 +1171,18 @@ bool ocelot_can_inject(struct ocelot *ocelot, int grp)
}
EXPORT_SYMBOL(ocelot_can_inject);
+void ocelot_ifh_port_set(void *ifh, int port, u32 rew_op, u32 vlan_tag)
+{
+ ocelot_ifh_set_bypass(ifh, 1);
+ ocelot_ifh_set_dest(ifh, BIT_ULL(port));
+ ocelot_ifh_set_tag_type(ifh, IFH_TAG_TYPE_C);
+ if (vlan_tag)
+ ocelot_ifh_set_vlan_tci(ifh, vlan_tag);
+ if (rew_op)
+ ocelot_ifh_set_rew_op(ifh, rew_op);
+}
+EXPORT_SYMBOL(ocelot_ifh_port_set);
+
void ocelot_port_inject_frame(struct ocelot *ocelot, int port, int grp,
u32 rew_op, struct sk_buff *skb)
{
@@ -1085,11 +1192,7 @@ void ocelot_port_inject_frame(struct ocelot *ocelot, int port, int grp,
ocelot_write_rix(ocelot, QS_INJ_CTRL_GAP_SIZE(1) |
QS_INJ_CTRL_SOF, QS_INJ_CTRL, grp);
- ocelot_ifh_set_bypass(ifh, 1);
- ocelot_ifh_set_dest(ifh, BIT_ULL(port));
- ocelot_ifh_set_tag_type(ifh, IFH_TAG_TYPE_C);
- ocelot_ifh_set_vlan_tci(ifh, skb_vlan_tag_get(skb));
- ocelot_ifh_set_rew_op(ifh, rew_op);
+ ocelot_ifh_port_set(ifh, port, rew_op, skb_vlan_tag_get(skb));
for (i = 0; i < OCELOT_TAG_LEN / 4; i++)
ocelot_write_rix(ocelot, ifh[i], QS_INJ_WR, grp);
@@ -1238,6 +1341,43 @@ static int ocelot_mact_read(struct ocelot *ocelot, int port, int row, int col,
return 0;
}
+int ocelot_mact_flush(struct ocelot *ocelot, int port)
+{
+ int err;
+
+ mutex_lock(&ocelot->mact_lock);
+
+ /* Program ageing filter for a single port */
+ ocelot_write(ocelot, ANA_ANAGEFIL_PID_EN | ANA_ANAGEFIL_PID_VAL(port),
+ ANA_ANAGEFIL);
+
+ /* Flushing dynamic FDB entries requires two successive age scans */
+ ocelot_write(ocelot,
+ ANA_TABLES_MACACCESS_MAC_TABLE_CMD(MACACCESS_CMD_AGE),
+ ANA_TABLES_MACACCESS);
+
+ err = ocelot_mact_wait_for_completion(ocelot);
+ if (err) {
+ mutex_unlock(&ocelot->mact_lock);
+ return err;
+ }
+
+ /* And second... */
+ ocelot_write(ocelot,
+ ANA_TABLES_MACACCESS_MAC_TABLE_CMD(MACACCESS_CMD_AGE),
+ ANA_TABLES_MACACCESS);
+
+ err = ocelot_mact_wait_for_completion(ocelot);
+
+ /* Restore ageing filter */
+ ocelot_write(ocelot, 0, ANA_ANAGEFIL);
+
+ mutex_unlock(&ocelot->mact_lock);
+
+ return err;
+}
+EXPORT_SYMBOL_GPL(ocelot_mact_flush);
+
int ocelot_fdb_dump(struct ocelot *ocelot, int port,
dsa_fdb_dump_cb_t *cb, void *data)
{
@@ -1514,10 +1654,6 @@ int ocelot_hwstamp_set(struct ocelot *ocelot, int port, struct ifreq *ifr)
if (copy_from_user(&cfg, ifr->ifr_data, sizeof(cfg)))
return -EFAULT;
- /* reserved for future extensions */
- if (cfg.flags)
- return -EINVAL;
-
/* Tx type sanity check */
switch (cfg.tx_type) {
case HWTSTAMP_TX_ON:
@@ -1688,8 +1824,7 @@ int ocelot_get_ts_info(struct ocelot *ocelot, int port,
}
EXPORT_SYMBOL(ocelot_get_ts_info);
-static u32 ocelot_get_bond_mask(struct ocelot *ocelot, struct net_device *bond,
- bool only_active_ports)
+static u32 ocelot_get_bond_mask(struct ocelot *ocelot, struct net_device *bond)
{
u32 mask = 0;
int port;
@@ -1700,26 +1835,25 @@ static u32 ocelot_get_bond_mask(struct ocelot *ocelot, struct net_device *bond,
if (!ocelot_port)
continue;
- if (ocelot_port->bond == bond) {
- if (only_active_ports && !ocelot_port->lag_tx_active)
- continue;
-
+ if (ocelot_port->bond == bond)
mask |= BIT(port);
- }
}
return mask;
}
-static u32 ocelot_get_bridge_fwd_mask(struct ocelot *ocelot, int src_port,
- struct net_device *bridge)
+u32 ocelot_get_bridge_fwd_mask(struct ocelot *ocelot, int src_port)
{
struct ocelot_port *ocelot_port = ocelot->ports[src_port];
+ const struct net_device *bridge;
u32 mask = 0;
int port;
- if (!ocelot_port || ocelot_port->bridge != bridge ||
- ocelot_port->stp_state != BR_STATE_FORWARDING)
+ if (!ocelot_port || ocelot_port->stp_state != BR_STATE_FORWARDING)
+ return 0;
+
+ bridge = ocelot_port->bridge;
+ if (!bridge)
return 0;
for (port = 0; port < ocelot->num_phys_ports; port++) {
@@ -1735,8 +1869,9 @@ static u32 ocelot_get_bridge_fwd_mask(struct ocelot *ocelot, int src_port,
return mask;
}
+EXPORT_SYMBOL_GPL(ocelot_get_bridge_fwd_mask);
-static u32 ocelot_get_dsa_8021q_cpu_mask(struct ocelot *ocelot)
+u32 ocelot_get_dsa_8021q_cpu_mask(struct ocelot *ocelot)
{
u32 mask = 0;
int port;
@@ -1753,12 +1888,22 @@ static u32 ocelot_get_dsa_8021q_cpu_mask(struct ocelot *ocelot)
return mask;
}
+EXPORT_SYMBOL_GPL(ocelot_get_dsa_8021q_cpu_mask);
-void ocelot_apply_bridge_fwd_mask(struct ocelot *ocelot)
+void ocelot_apply_bridge_fwd_mask(struct ocelot *ocelot, bool joining)
{
unsigned long cpu_fwd_mask;
int port;
+ lockdep_assert_held(&ocelot->fwd_domain_lock);
+
+ /* If cut-through forwarding is supported, update the masks before a
+ * port joins the forwarding domain, to avoid potential underruns if it
+ * has the highest speed from the new domain.
+ */
+ if (joining && ocelot->ops->cut_through_fwd)
+ ocelot->ops->cut_through_fwd(ocelot);
+
/* If a DSA tag_8021q CPU exists, it needs to be included in the
* regular forwarding path of the front ports regardless of whether
* those are bridged or standalone.
@@ -1786,16 +1931,13 @@ void ocelot_apply_bridge_fwd_mask(struct ocelot *ocelot)
mask = GENMASK(ocelot->num_phys_ports - 1, 0);
mask &= ~cpu_fwd_mask;
} else if (ocelot_port->bridge) {
- struct net_device *bridge = ocelot_port->bridge;
struct net_device *bond = ocelot_port->bond;
- mask = ocelot_get_bridge_fwd_mask(ocelot, port, bridge);
+ mask = ocelot_get_bridge_fwd_mask(ocelot, port);
mask |= cpu_fwd_mask;
mask &= ~BIT(port);
- if (bond) {
- mask &= ~ocelot_get_bond_mask(ocelot, bond,
- false);
- }
+ if (bond)
+ mask &= ~ocelot_get_bond_mask(ocelot, bond);
} else {
/* Standalone ports forward only to DSA tag_8021q CPU
* ports (if those exist), or to the hardware CPU port
@@ -1806,6 +1948,16 @@ void ocelot_apply_bridge_fwd_mask(struct ocelot *ocelot)
ocelot_write_rix(ocelot, mask, ANA_PGID_PGID, PGID_SRC + port);
}
+
+ /* If cut-through forwarding is supported and a port is leaving, there
+ * is a chance that cut-through was disabled on the other ports due to
+ * the port which is leaving (it has a higher link speed). We need to
+ * update the cut-through masks of the remaining ports no earlier than
+ * after the port has left, to prevent underruns from happening between
+ * the cut-through update and the forwarding domain update.
+ */
+ if (!joining && ocelot->ops->cut_through_fwd)
+ ocelot->ops->cut_through_fwd(ocelot);
}
EXPORT_SYMBOL(ocelot_apply_bridge_fwd_mask);
@@ -1814,6 +1966,8 @@ void ocelot_bridge_stp_state_set(struct ocelot *ocelot, int port, u8 state)
struct ocelot_port *ocelot_port = ocelot->ports[port];
u32 learn_ena = 0;
+ mutex_lock(&ocelot->fwd_domain_lock);
+
ocelot_port->stp_state = state;
if ((state == BR_STATE_LEARNING || state == BR_STATE_FORWARDING) &&
@@ -1823,7 +1977,9 @@ void ocelot_bridge_stp_state_set(struct ocelot *ocelot, int port, u8 state)
ocelot_rmw_gix(ocelot, learn_ena, ANA_PORT_PORT_CFG_LEARN_ENA,
ANA_PORT_PORT_CFG, port);
- ocelot_apply_bridge_fwd_mask(ocelot);
+ ocelot_apply_bridge_fwd_mask(ocelot, state == BR_STATE_FORWARDING);
+
+ mutex_unlock(&ocelot->fwd_domain_lock);
}
EXPORT_SYMBOL(ocelot_bridge_stp_state_set);
@@ -2053,9 +2209,13 @@ void ocelot_port_bridge_join(struct ocelot *ocelot, int port,
{
struct ocelot_port *ocelot_port = ocelot->ports[port];
+ mutex_lock(&ocelot->fwd_domain_lock);
+
ocelot_port->bridge = bridge;
- ocelot_apply_bridge_fwd_mask(ocelot);
+ ocelot_apply_bridge_fwd_mask(ocelot, true);
+
+ mutex_unlock(&ocelot->fwd_domain_lock);
}
EXPORT_SYMBOL(ocelot_port_bridge_join);
@@ -2064,11 +2224,15 @@ void ocelot_port_bridge_leave(struct ocelot *ocelot, int port,
{
struct ocelot_port *ocelot_port = ocelot->ports[port];
+ mutex_lock(&ocelot->fwd_domain_lock);
+
ocelot_port->bridge = NULL;
ocelot_port_set_pvid(ocelot, port, NULL);
ocelot_port_manage_port_tag(ocelot, port);
- ocelot_apply_bridge_fwd_mask(ocelot);
+ ocelot_apply_bridge_fwd_mask(ocelot, false);
+
+ mutex_unlock(&ocelot->fwd_domain_lock);
}
EXPORT_SYMBOL(ocelot_port_bridge_leave);
@@ -2112,13 +2276,17 @@ static void ocelot_set_aggr_pgids(struct ocelot *ocelot)
if (!bond || (visited & BIT(lag)))
continue;
- bond_mask = ocelot_get_bond_mask(ocelot, bond, true);
+ bond_mask = ocelot_get_bond_mask(ocelot, bond);
for_each_set_bit(port, &bond_mask, ocelot->num_phys_ports) {
+ struct ocelot_port *ocelot_port = ocelot->ports[port];
+
// Destination mask
ocelot_write_rix(ocelot, bond_mask,
ANA_PGID_PGID, port);
- aggr_idx[num_active_ports++] = port;
+
+ if (ocelot_port->lag_tx_active)
+ aggr_idx[num_active_ports++] = port;
}
for_each_aggr_pgid(ocelot, i) {
@@ -2167,8 +2335,7 @@ static void ocelot_setup_logical_port_ids(struct ocelot *ocelot)
bond = ocelot_port->bond;
if (bond) {
- int lag = __ffs(ocelot_get_bond_mask(ocelot, bond,
- false));
+ int lag = __ffs(ocelot_get_bond_mask(ocelot, bond));
ocelot_rmw_gix(ocelot,
ANA_PORT_PORT_CFG_PORTID_VAL(lag),
@@ -2190,12 +2357,16 @@ int ocelot_port_lag_join(struct ocelot *ocelot, int port,
if (info->tx_type != NETDEV_LAG_TX_TYPE_HASH)
return -EOPNOTSUPP;
+ mutex_lock(&ocelot->fwd_domain_lock);
+
ocelot->ports[port]->bond = bond;
ocelot_setup_logical_port_ids(ocelot);
- ocelot_apply_bridge_fwd_mask(ocelot);
+ ocelot_apply_bridge_fwd_mask(ocelot, true);
ocelot_set_aggr_pgids(ocelot);
+ mutex_unlock(&ocelot->fwd_domain_lock);
+
return 0;
}
EXPORT_SYMBOL(ocelot_port_lag_join);
@@ -2203,11 +2374,15 @@ EXPORT_SYMBOL(ocelot_port_lag_join);
void ocelot_port_lag_leave(struct ocelot *ocelot, int port,
struct net_device *bond)
{
+ mutex_lock(&ocelot->fwd_domain_lock);
+
ocelot->ports[port]->bond = NULL;
ocelot_setup_logical_port_ids(ocelot);
- ocelot_apply_bridge_fwd_mask(ocelot);
+ ocelot_apply_bridge_fwd_mask(ocelot, false);
ocelot_set_aggr_pgids(ocelot);
+
+ mutex_unlock(&ocelot->fwd_domain_lock);
}
EXPORT_SYMBOL(ocelot_port_lag_leave);
@@ -2498,6 +2673,7 @@ int ocelot_init(struct ocelot *ocelot)
mutex_init(&ocelot->stats_lock);
mutex_init(&ocelot->ptp_lock);
mutex_init(&ocelot->mact_lock);
+ mutex_init(&ocelot->fwd_domain_lock);
spin_lock_init(&ocelot->ptp_clock_lock);
spin_lock_init(&ocelot->ts_id_lock);
snprintf(queue_name, sizeof(queue_name), "%s-stats",
@@ -2521,6 +2697,9 @@ int ocelot_init(struct ocelot *ocelot)
ocelot_vcap_init(ocelot);
ocelot_cpu_port_init(ocelot);
+ if (ocelot->ops->psfp_init)
+ ocelot->ops->psfp_init(ocelot);
+
for (port = 0; port < ocelot->num_phys_ports; port++) {
/* Clear all counters (5 groups) */
ocelot_write(ocelot, SYS_STAT_CFG_STAT_VIEW(port) |
diff --git a/drivers/net/ethernet/mscc/ocelot.h b/drivers/net/ethernet/mscc/ocelot.h
index e43da09..bf4eff6 100644
--- a/drivers/net/ethernet/mscc/ocelot.h
+++ b/drivers/net/ethernet/mscc/ocelot.h
@@ -32,6 +32,8 @@
#define OCELOT_PTP_QUEUE_SZ 128
+#define OCELOT_JUMBO_MTU 9000
+
struct ocelot_port_tc {
bool block_shared;
unsigned long offload_cnt;
@@ -55,19 +57,6 @@ struct ocelot_dump_ctx {
int idx;
};
-/* MAC table entry types.
- * ENTRYTYPE_NORMAL is subject to aging.
- * ENTRYTYPE_LOCKED is not subject to aging.
- * ENTRYTYPE_MACv4 is not subject to aging. For IPv4 multicast.
- * ENTRYTYPE_MACv6 is not subject to aging. For IPv6 multicast.
- */
-enum macaccess_entry_type {
- ENTRYTYPE_NORMAL = 0,
- ENTRYTYPE_LOCKED,
- ENTRYTYPE_MACv4,
- ENTRYTYPE_MACv6,
-};
-
/* A (PGID) port mask structure, encoding the 2^ocelot->num_phys_ports
* possibilities of egress port masks for L2 multicast traffic.
* For a switch with 9 user ports, there are 512 possible port masks, but the
diff --git a/drivers/net/ethernet/mscc/ocelot_fdma.c b/drivers/net/ethernet/mscc/ocelot_fdma.c
new file mode 100644
index 0000000..dffa597
--- /dev/null
+++ b/drivers/net/ethernet/mscc/ocelot_fdma.c
@@ -0,0 +1,894 @@
+// SPDX-License-Identifier: (GPL-2.0 OR MIT)
+/*
+ * Microsemi SoCs FDMA driver
+ *
+ * Copyright (c) 2021 Microchip
+ *
+ * Page recycling code is mostly taken from gianfar driver.
+ */
+
+#include <linux/align.h>
+#include <linux/bitops.h>
+#include <linux/dmapool.h>
+#include <linux/dsa/ocelot.h>
+#include <linux/netdevice.h>
+#include <linux/of_platform.h>
+#include <linux/skbuff.h>
+
+#include "ocelot_fdma.h"
+#include "ocelot_qs.h"
+
+DEFINE_STATIC_KEY_FALSE(ocelot_fdma_enabled);
+
+static void ocelot_fdma_writel(struct ocelot *ocelot, u32 reg, u32 data)
+{
+ regmap_write(ocelot->targets[FDMA], reg, data);
+}
+
+static u32 ocelot_fdma_readl(struct ocelot *ocelot, u32 reg)
+{
+ u32 retval;
+
+ regmap_read(ocelot->targets[FDMA], reg, &retval);
+
+ return retval;
+}
+
+static dma_addr_t ocelot_fdma_idx_dma(dma_addr_t base, u16 idx)
+{
+ return base + idx * sizeof(struct ocelot_fdma_dcb);
+}
+
+static u16 ocelot_fdma_dma_idx(dma_addr_t base, dma_addr_t dma)
+{
+ return (dma - base) / sizeof(struct ocelot_fdma_dcb);
+}
+
+static u16 ocelot_fdma_idx_next(u16 idx, u16 ring_sz)
+{
+ return unlikely(idx == ring_sz - 1) ? 0 : idx + 1;
+}
+
+static u16 ocelot_fdma_idx_prev(u16 idx, u16 ring_sz)
+{
+ return unlikely(idx == 0) ? ring_sz - 1 : idx - 1;
+}
+
+static int ocelot_fdma_rx_ring_free(struct ocelot_fdma *fdma)
+{
+ struct ocelot_fdma_rx_ring *rx_ring = &fdma->rx_ring;
+
+ if (rx_ring->next_to_use >= rx_ring->next_to_clean)
+ return OCELOT_FDMA_RX_RING_SIZE -
+ (rx_ring->next_to_use - rx_ring->next_to_clean) - 1;
+ else
+ return rx_ring->next_to_clean - rx_ring->next_to_use - 1;
+}
+
+static int ocelot_fdma_tx_ring_free(struct ocelot_fdma *fdma)
+{
+ struct ocelot_fdma_tx_ring *tx_ring = &fdma->tx_ring;
+
+ if (tx_ring->next_to_use >= tx_ring->next_to_clean)
+ return OCELOT_FDMA_TX_RING_SIZE -
+ (tx_ring->next_to_use - tx_ring->next_to_clean) - 1;
+ else
+ return tx_ring->next_to_clean - tx_ring->next_to_use - 1;
+}
+
+static bool ocelot_fdma_tx_ring_empty(struct ocelot_fdma *fdma)
+{
+ struct ocelot_fdma_tx_ring *tx_ring = &fdma->tx_ring;
+
+ return tx_ring->next_to_clean == tx_ring->next_to_use;
+}
+
+static void ocelot_fdma_activate_chan(struct ocelot *ocelot, dma_addr_t dma,
+ int chan)
+{
+ ocelot_fdma_writel(ocelot, MSCC_FDMA_DCB_LLP(chan), dma);
+ /* Barrier to force memory writes to DCB to be completed before starting
+ * the channel.
+ */
+ wmb();
+ ocelot_fdma_writel(ocelot, MSCC_FDMA_CH_ACTIVATE, BIT(chan));
+}
+
+static int ocelot_fdma_wait_chan_safe(struct ocelot *ocelot, int chan)
+{
+ unsigned long timeout;
+ u32 safe;
+
+ timeout = jiffies + usecs_to_jiffies(OCELOT_FDMA_CH_SAFE_TIMEOUT_US);
+ do {
+ safe = ocelot_fdma_readl(ocelot, MSCC_FDMA_CH_SAFE);
+ if (safe & BIT(chan))
+ return 0;
+ } while (time_after(jiffies, timeout));
+
+ return -ETIMEDOUT;
+}
+
+static void ocelot_fdma_dcb_set_data(struct ocelot_fdma_dcb *dcb,
+ dma_addr_t dma_addr,
+ size_t size)
+{
+ u32 offset = dma_addr & 0x3;
+
+ dcb->llp = 0;
+ dcb->datap = ALIGN_DOWN(dma_addr, 4);
+ dcb->datal = ALIGN_DOWN(size, 4);
+ dcb->stat = MSCC_FDMA_DCB_STAT_BLOCKO(offset);
+}
+
+static bool ocelot_fdma_rx_alloc_page(struct ocelot *ocelot,
+ struct ocelot_fdma_rx_buf *rxb)
+{
+ dma_addr_t mapping;
+ struct page *page;
+
+ page = dev_alloc_page();
+ if (unlikely(!page))
+ return false;
+
+ mapping = dma_map_page(ocelot->dev, page, 0, PAGE_SIZE,
+ DMA_FROM_DEVICE);
+ if (unlikely(dma_mapping_error(ocelot->dev, mapping))) {
+ __free_page(page);
+ return false;
+ }
+
+ rxb->page = page;
+ rxb->page_offset = 0;
+ rxb->dma_addr = mapping;
+
+ return true;
+}
+
+static int ocelot_fdma_alloc_rx_buffs(struct ocelot *ocelot, u16 alloc_cnt)
+{
+ struct ocelot_fdma *fdma = ocelot->fdma;
+ struct ocelot_fdma_rx_ring *rx_ring;
+ struct ocelot_fdma_rx_buf *rxb;
+ struct ocelot_fdma_dcb *dcb;
+ dma_addr_t dma_addr;
+ int ret = 0;
+ u16 idx;
+
+ rx_ring = &fdma->rx_ring;
+ idx = rx_ring->next_to_use;
+
+ while (alloc_cnt--) {
+ rxb = &rx_ring->bufs[idx];
+ /* try reuse page */
+ if (unlikely(!rxb->page)) {
+ if (unlikely(!ocelot_fdma_rx_alloc_page(ocelot, rxb))) {
+ dev_err_ratelimited(ocelot->dev,
+ "Failed to allocate rx\n");
+ ret = -ENOMEM;
+ break;
+ }
+ }
+
+ dcb = &rx_ring->dcbs[idx];
+ dma_addr = rxb->dma_addr + rxb->page_offset;
+ ocelot_fdma_dcb_set_data(dcb, dma_addr, OCELOT_FDMA_RXB_SIZE);
+
+ idx = ocelot_fdma_idx_next(idx, OCELOT_FDMA_RX_RING_SIZE);
+ /* Chain the DCB to the next one */
+ dcb->llp = ocelot_fdma_idx_dma(rx_ring->dcbs_dma, idx);
+ }
+
+ rx_ring->next_to_use = idx;
+ rx_ring->next_to_alloc = idx;
+
+ return ret;
+}
+
+static bool ocelot_fdma_tx_dcb_set_skb(struct ocelot *ocelot,
+ struct ocelot_fdma_tx_buf *tx_buf,
+ struct ocelot_fdma_dcb *dcb,
+ struct sk_buff *skb)
+{
+ dma_addr_t mapping;
+
+ mapping = dma_map_single(ocelot->dev, skb->data, skb->len,
+ DMA_TO_DEVICE);
+ if (unlikely(dma_mapping_error(ocelot->dev, mapping)))
+ return false;
+
+ dma_unmap_addr_set(tx_buf, dma_addr, mapping);
+
+ ocelot_fdma_dcb_set_data(dcb, mapping, OCELOT_FDMA_RX_SIZE);
+ tx_buf->skb = skb;
+ dcb->stat |= MSCC_FDMA_DCB_STAT_BLOCKL(skb->len);
+ dcb->stat |= MSCC_FDMA_DCB_STAT_SOF | MSCC_FDMA_DCB_STAT_EOF;
+
+ return true;
+}
+
+static bool ocelot_fdma_check_stop_rx(struct ocelot *ocelot)
+{
+ u32 llp;
+
+ /* Check if the FDMA hits the DCB with LLP == NULL */
+ llp = ocelot_fdma_readl(ocelot, MSCC_FDMA_DCB_LLP(MSCC_FDMA_XTR_CHAN));
+ if (unlikely(llp))
+ return false;
+
+ ocelot_fdma_writel(ocelot, MSCC_FDMA_CH_DISABLE,
+ BIT(MSCC_FDMA_XTR_CHAN));
+
+ return true;
+}
+
+static void ocelot_fdma_rx_set_llp(struct ocelot_fdma_rx_ring *rx_ring)
+{
+ struct ocelot_fdma_dcb *dcb;
+ unsigned int idx;
+
+ idx = ocelot_fdma_idx_prev(rx_ring->next_to_use,
+ OCELOT_FDMA_RX_RING_SIZE);
+ dcb = &rx_ring->dcbs[idx];
+ dcb->llp = 0;
+}
+
+static void ocelot_fdma_rx_restart(struct ocelot *ocelot)
+{
+ struct ocelot_fdma *fdma = ocelot->fdma;
+ struct ocelot_fdma_rx_ring *rx_ring;
+ const u8 chan = MSCC_FDMA_XTR_CHAN;
+ dma_addr_t new_llp, dma_base;
+ unsigned int idx;
+ u32 llp_prev;
+ int ret;
+
+ rx_ring = &fdma->rx_ring;
+ ret = ocelot_fdma_wait_chan_safe(ocelot, chan);
+ if (ret) {
+ dev_err_ratelimited(ocelot->dev,
+ "Unable to stop RX channel\n");
+ return;
+ }
+
+ ocelot_fdma_rx_set_llp(rx_ring);
+
+ /* FDMA stopped on the last DCB that contained a NULL LLP, since
+ * we processed some DCBs in RX, there is free space, and we must set
+ * DCB_LLP to point to the next DCB
+ */
+ llp_prev = ocelot_fdma_readl(ocelot, MSCC_FDMA_DCB_LLP_PREV(chan));
+ dma_base = rx_ring->dcbs_dma;
+
+ /* Get the next DMA addr located after LLP == NULL DCB */
+ idx = ocelot_fdma_dma_idx(dma_base, llp_prev);
+ idx = ocelot_fdma_idx_next(idx, OCELOT_FDMA_RX_RING_SIZE);
+ new_llp = ocelot_fdma_idx_dma(dma_base, idx);
+
+ /* Finally reactivate the channel */
+ ocelot_fdma_activate_chan(ocelot, new_llp, chan);
+}
+
+static bool ocelot_fdma_add_rx_frag(struct ocelot_fdma_rx_buf *rxb, u32 stat,
+ struct sk_buff *skb, bool first)
+{
+ int size = MSCC_FDMA_DCB_STAT_BLOCKL(stat);
+ struct page *page = rxb->page;
+
+ if (likely(first)) {
+ skb_put(skb, size);
+ } else {
+ skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags, page,
+ rxb->page_offset, size, OCELOT_FDMA_RX_SIZE);
+ }
+
+ /* Try to reuse page */
+ if (unlikely(page_ref_count(page) != 1 || page_is_pfmemalloc(page)))
+ return false;
+
+ /* Change offset to the other half */
+ rxb->page_offset ^= OCELOT_FDMA_RX_SIZE;
+
+ page_ref_inc(page);
+
+ return true;
+}
+
+static void ocelot_fdma_reuse_rx_page(struct ocelot *ocelot,
+ struct ocelot_fdma_rx_buf *old_rxb)
+{
+ struct ocelot_fdma_rx_ring *rx_ring = &ocelot->fdma->rx_ring;
+ struct ocelot_fdma_rx_buf *new_rxb;
+
+ new_rxb = &rx_ring->bufs[rx_ring->next_to_alloc];
+ rx_ring->next_to_alloc = ocelot_fdma_idx_next(rx_ring->next_to_alloc,
+ OCELOT_FDMA_RX_RING_SIZE);
+
+ /* Copy page reference */
+ *new_rxb = *old_rxb;
+
+ /* Sync for use by the device */
+ dma_sync_single_range_for_device(ocelot->dev, old_rxb->dma_addr,
+ old_rxb->page_offset,
+ OCELOT_FDMA_RX_SIZE, DMA_FROM_DEVICE);
+}
+
+static struct sk_buff *ocelot_fdma_get_skb(struct ocelot *ocelot, u32 stat,
+ struct ocelot_fdma_rx_buf *rxb,
+ struct sk_buff *skb)
+{
+ bool first = false;
+
+ /* Allocate skb head and data */
+ if (likely(!skb)) {
+ void *buff_addr = page_address(rxb->page) +
+ rxb->page_offset;
+
+ skb = build_skb(buff_addr, OCELOT_FDMA_SKBFRAG_SIZE);
+ if (unlikely(!skb)) {
+ dev_err_ratelimited(ocelot->dev,
+ "build_skb failed !\n");
+ return NULL;
+ }
+ first = true;
+ }
+
+ dma_sync_single_range_for_cpu(ocelot->dev, rxb->dma_addr,
+ rxb->page_offset, OCELOT_FDMA_RX_SIZE,
+ DMA_FROM_DEVICE);
+
+ if (ocelot_fdma_add_rx_frag(rxb, stat, skb, first)) {
+ /* Reuse the free half of the page for the next_to_alloc DCB*/
+ ocelot_fdma_reuse_rx_page(ocelot, rxb);
+ } else {
+ /* page cannot be reused, unmap it */
+ dma_unmap_page(ocelot->dev, rxb->dma_addr, PAGE_SIZE,
+ DMA_FROM_DEVICE);
+ }
+
+ /* clear rx buff content */
+ rxb->page = NULL;
+
+ return skb;
+}
+
+static bool ocelot_fdma_receive_skb(struct ocelot *ocelot, struct sk_buff *skb)
+{
+ struct net_device *ndev;
+ void *xfh = skb->data;
+ u64 timestamp;
+ u64 src_port;
+
+ skb_pull(skb, OCELOT_TAG_LEN);
+
+ ocelot_xfh_get_src_port(xfh, &src_port);
+ if (unlikely(src_port >= ocelot->num_phys_ports))
+ return false;
+
+ ndev = ocelot_port_to_netdev(ocelot, src_port);
+ if (unlikely(!ndev))
+ return false;
+
+ pskb_trim(skb, skb->len - ETH_FCS_LEN);
+
+ skb->dev = ndev;
+ skb->protocol = eth_type_trans(skb, skb->dev);
+ skb->dev->stats.rx_bytes += skb->len;
+ skb->dev->stats.rx_packets++;
+
+ if (ocelot->ptp) {
+ ocelot_xfh_get_rew_val(xfh, ×tamp);
+ ocelot_ptp_rx_timestamp(ocelot, skb, timestamp);
+ }
+
+ if (likely(!skb_defer_rx_timestamp(skb)))
+ netif_receive_skb(skb);
+
+ return true;
+}
+
+static int ocelot_fdma_rx_get(struct ocelot *ocelot, int budget)
+{
+ struct ocelot_fdma *fdma = ocelot->fdma;
+ struct ocelot_fdma_rx_ring *rx_ring;
+ struct ocelot_fdma_rx_buf *rxb;
+ struct ocelot_fdma_dcb *dcb;
+ struct sk_buff *skb;
+ int work_done = 0;
+ int cleaned_cnt;
+ u32 stat;
+ u16 idx;
+
+ cleaned_cnt = ocelot_fdma_rx_ring_free(fdma);
+ rx_ring = &fdma->rx_ring;
+ skb = rx_ring->skb;
+
+ while (budget--) {
+ idx = rx_ring->next_to_clean;
+ dcb = &rx_ring->dcbs[idx];
+ stat = dcb->stat;
+ if (MSCC_FDMA_DCB_STAT_BLOCKL(stat) == 0)
+ break;
+
+ /* New packet is a start of frame but we already got a skb set,
+ * we probably lost an EOF packet, free skb
+ */
+ if (unlikely(skb && (stat & MSCC_FDMA_DCB_STAT_SOF))) {
+ dev_kfree_skb(skb);
+ skb = NULL;
+ }
+
+ rxb = &rx_ring->bufs[idx];
+ /* Fetch next to clean buffer from the rx_ring */
+ skb = ocelot_fdma_get_skb(ocelot, stat, rxb, skb);
+ if (unlikely(!skb))
+ break;
+
+ work_done++;
+ cleaned_cnt++;
+
+ idx = ocelot_fdma_idx_next(idx, OCELOT_FDMA_RX_RING_SIZE);
+ rx_ring->next_to_clean = idx;
+
+ if (unlikely(stat & MSCC_FDMA_DCB_STAT_ABORT ||
+ stat & MSCC_FDMA_DCB_STAT_PD)) {
+ dev_err_ratelimited(ocelot->dev,
+ "DCB aborted or pruned\n");
+ dev_kfree_skb(skb);
+ skb = NULL;
+ continue;
+ }
+
+ /* We still need to process the other fragment of the packet
+ * before delivering it to the network stack
+ */
+ if (!(stat & MSCC_FDMA_DCB_STAT_EOF))
+ continue;
+
+ if (unlikely(!ocelot_fdma_receive_skb(ocelot, skb)))
+ dev_kfree_skb(skb);
+
+ skb = NULL;
+ }
+
+ rx_ring->skb = skb;
+
+ if (cleaned_cnt)
+ ocelot_fdma_alloc_rx_buffs(ocelot, cleaned_cnt);
+
+ return work_done;
+}
+
+static void ocelot_fdma_wakeup_netdev(struct ocelot *ocelot)
+{
+ struct ocelot_port_private *priv;
+ struct ocelot_port *ocelot_port;
+ struct net_device *dev;
+ int port;
+
+ for (port = 0; port < ocelot->num_phys_ports; port++) {
+ ocelot_port = ocelot->ports[port];
+ if (!ocelot_port)
+ continue;
+ priv = container_of(ocelot_port, struct ocelot_port_private,
+ port);
+ dev = priv->dev;
+
+ if (unlikely(netif_queue_stopped(dev)))
+ netif_wake_queue(dev);
+ }
+}
+
+static void ocelot_fdma_tx_cleanup(struct ocelot *ocelot, int budget)
+{
+ struct ocelot_fdma *fdma = ocelot->fdma;
+ struct ocelot_fdma_tx_ring *tx_ring;
+ struct ocelot_fdma_tx_buf *buf;
+ unsigned int new_null_llp_idx;
+ struct ocelot_fdma_dcb *dcb;
+ bool end_of_list = false;
+ struct sk_buff *skb;
+ dma_addr_t dma;
+ u32 dcb_llp;
+ u16 ntc;
+ int ret;
+
+ tx_ring = &fdma->tx_ring;
+
+ /* Purge the TX packets that have been sent up to the NULL llp or the
+ * end of done list.
+ */
+ while (!ocelot_fdma_tx_ring_empty(fdma)) {
+ ntc = tx_ring->next_to_clean;
+ dcb = &tx_ring->dcbs[ntc];
+ if (!(dcb->stat & MSCC_FDMA_DCB_STAT_PD))
+ break;
+
+ buf = &tx_ring->bufs[ntc];
+ skb = buf->skb;
+ dma_unmap_single(ocelot->dev, dma_unmap_addr(buf, dma_addr),
+ skb->len, DMA_TO_DEVICE);
+ napi_consume_skb(skb, budget);
+ dcb_llp = dcb->llp;
+
+ /* Only update after accessing all dcb fields */
+ tx_ring->next_to_clean = ocelot_fdma_idx_next(ntc,
+ OCELOT_FDMA_TX_RING_SIZE);
+
+ /* If we hit the NULL LLP, stop, we might need to reload FDMA */
+ if (dcb_llp == 0) {
+ end_of_list = true;
+ break;
+ }
+ }
+
+ /* No need to try to wake if there were no TX cleaned_cnt up. */
+ if (ocelot_fdma_tx_ring_free(fdma))
+ ocelot_fdma_wakeup_netdev(ocelot);
+
+ /* If there is still some DCBs to be processed by the FDMA or if the
+ * pending list is empty, there is no need to restart the FDMA.
+ */
+ if (!end_of_list || ocelot_fdma_tx_ring_empty(fdma))
+ return;
+
+ ret = ocelot_fdma_wait_chan_safe(ocelot, MSCC_FDMA_INJ_CHAN);
+ if (ret) {
+ dev_warn(ocelot->dev,
+ "Failed to wait for TX channel to stop\n");
+ return;
+ }
+
+ /* Set NULL LLP to be the last DCB used */
+ new_null_llp_idx = ocelot_fdma_idx_prev(tx_ring->next_to_use,
+ OCELOT_FDMA_TX_RING_SIZE);
+ dcb = &tx_ring->dcbs[new_null_llp_idx];
+ dcb->llp = 0;
+
+ dma = ocelot_fdma_idx_dma(tx_ring->dcbs_dma, tx_ring->next_to_clean);
+ ocelot_fdma_activate_chan(ocelot, dma, MSCC_FDMA_INJ_CHAN);
+}
+
+static int ocelot_fdma_napi_poll(struct napi_struct *napi, int budget)
+{
+ struct ocelot_fdma *fdma = container_of(napi, struct ocelot_fdma, napi);
+ struct ocelot *ocelot = fdma->ocelot;
+ int work_done = 0;
+ bool rx_stopped;
+
+ ocelot_fdma_tx_cleanup(ocelot, budget);
+
+ rx_stopped = ocelot_fdma_check_stop_rx(ocelot);
+
+ work_done = ocelot_fdma_rx_get(ocelot, budget);
+
+ if (rx_stopped)
+ ocelot_fdma_rx_restart(ocelot);
+
+ if (work_done < budget) {
+ napi_complete_done(&fdma->napi, work_done);
+ ocelot_fdma_writel(ocelot, MSCC_FDMA_INTR_ENA,
+ BIT(MSCC_FDMA_INJ_CHAN) |
+ BIT(MSCC_FDMA_XTR_CHAN));
+ }
+
+ return work_done;
+}
+
+static irqreturn_t ocelot_fdma_interrupt(int irq, void *dev_id)
+{
+ u32 ident, llp, frm, err, err_code;
+ struct ocelot *ocelot = dev_id;
+
+ ident = ocelot_fdma_readl(ocelot, MSCC_FDMA_INTR_IDENT);
+ frm = ocelot_fdma_readl(ocelot, MSCC_FDMA_INTR_FRM);
+ llp = ocelot_fdma_readl(ocelot, MSCC_FDMA_INTR_LLP);
+
+ ocelot_fdma_writel(ocelot, MSCC_FDMA_INTR_LLP, llp & ident);
+ ocelot_fdma_writel(ocelot, MSCC_FDMA_INTR_FRM, frm & ident);
+ if (frm || llp) {
+ ocelot_fdma_writel(ocelot, MSCC_FDMA_INTR_ENA, 0);
+ napi_schedule(&ocelot->fdma->napi);
+ }
+
+ err = ocelot_fdma_readl(ocelot, MSCC_FDMA_EVT_ERR);
+ if (unlikely(err)) {
+ err_code = ocelot_fdma_readl(ocelot, MSCC_FDMA_EVT_ERR_CODE);
+ dev_err_ratelimited(ocelot->dev,
+ "Error ! chans mask: %#x, code: %#x\n",
+ err, err_code);
+
+ ocelot_fdma_writel(ocelot, MSCC_FDMA_EVT_ERR, err);
+ ocelot_fdma_writel(ocelot, MSCC_FDMA_EVT_ERR_CODE, err_code);
+ }
+
+ return IRQ_HANDLED;
+}
+
+static void ocelot_fdma_send_skb(struct ocelot *ocelot,
+ struct ocelot_fdma *fdma, struct sk_buff *skb)
+{
+ struct ocelot_fdma_tx_ring *tx_ring = &fdma->tx_ring;
+ struct ocelot_fdma_tx_buf *tx_buf;
+ struct ocelot_fdma_dcb *dcb;
+ dma_addr_t dma;
+ u16 next_idx;
+
+ dcb = &tx_ring->dcbs[tx_ring->next_to_use];
+ tx_buf = &tx_ring->bufs[tx_ring->next_to_use];
+ if (!ocelot_fdma_tx_dcb_set_skb(ocelot, tx_buf, dcb, skb)) {
+ dev_kfree_skb_any(skb);
+ return;
+ }
+
+ next_idx = ocelot_fdma_idx_next(tx_ring->next_to_use,
+ OCELOT_FDMA_TX_RING_SIZE);
+ skb_tx_timestamp(skb);
+
+ /* If the FDMA TX chan is empty, then enqueue the DCB directly */
+ if (ocelot_fdma_tx_ring_empty(fdma)) {
+ dma = ocelot_fdma_idx_dma(tx_ring->dcbs_dma,
+ tx_ring->next_to_use);
+ ocelot_fdma_activate_chan(ocelot, dma, MSCC_FDMA_INJ_CHAN);
+ } else {
+ /* Chain the DCBs */
+ dcb->llp = ocelot_fdma_idx_dma(tx_ring->dcbs_dma, next_idx);
+ }
+
+ tx_ring->next_to_use = next_idx;
+}
+
+static int ocelot_fdma_prepare_skb(struct ocelot *ocelot, int port, u32 rew_op,
+ struct sk_buff *skb, struct net_device *dev)
+{
+ int needed_headroom = max_t(int, OCELOT_TAG_LEN - skb_headroom(skb), 0);
+ int needed_tailroom = max_t(int, ETH_FCS_LEN - skb_tailroom(skb), 0);
+ void *ifh;
+ int err;
+
+ if (unlikely(needed_headroom || needed_tailroom ||
+ skb_header_cloned(skb))) {
+ err = pskb_expand_head(skb, needed_headroom, needed_tailroom,
+ GFP_ATOMIC);
+ if (unlikely(err)) {
+ dev_kfree_skb_any(skb);
+ return 1;
+ }
+ }
+
+ err = skb_linearize(skb);
+ if (err) {
+ net_err_ratelimited("%s: skb_linearize error (%d)!\n",
+ dev->name, err);
+ dev_kfree_skb_any(skb);
+ return 1;
+ }
+
+ ifh = skb_push(skb, OCELOT_TAG_LEN);
+ skb_put(skb, ETH_FCS_LEN);
+ memset(ifh, 0, OCELOT_TAG_LEN);
+ ocelot_ifh_port_set(ifh, port, rew_op, skb_vlan_tag_get(skb));
+
+ return 0;
+}
+
+int ocelot_fdma_inject_frame(struct ocelot *ocelot, int port, u32 rew_op,
+ struct sk_buff *skb, struct net_device *dev)
+{
+ struct ocelot_fdma *fdma = ocelot->fdma;
+ int ret = NETDEV_TX_OK;
+
+ spin_lock(&fdma->tx_ring.xmit_lock);
+
+ if (ocelot_fdma_tx_ring_free(fdma) == 0) {
+ netif_stop_queue(dev);
+ ret = NETDEV_TX_BUSY;
+ goto out;
+ }
+
+ if (ocelot_fdma_prepare_skb(ocelot, port, rew_op, skb, dev))
+ goto out;
+
+ ocelot_fdma_send_skb(ocelot, fdma, skb);
+
+out:
+ spin_unlock(&fdma->tx_ring.xmit_lock);
+
+ return ret;
+}
+
+static void ocelot_fdma_free_rx_ring(struct ocelot *ocelot)
+{
+ struct ocelot_fdma *fdma = ocelot->fdma;
+ struct ocelot_fdma_rx_ring *rx_ring;
+ struct ocelot_fdma_rx_buf *rxb;
+ u16 idx;
+
+ rx_ring = &fdma->rx_ring;
+ idx = rx_ring->next_to_clean;
+
+ /* Free the pages held in the RX ring */
+ while (idx != rx_ring->next_to_use) {
+ rxb = &rx_ring->bufs[idx];
+ dma_unmap_page(ocelot->dev, rxb->dma_addr, PAGE_SIZE,
+ DMA_FROM_DEVICE);
+ __free_page(rxb->page);
+ idx = ocelot_fdma_idx_next(idx, OCELOT_FDMA_RX_RING_SIZE);
+ }
+
+ if (fdma->rx_ring.skb)
+ dev_kfree_skb_any(fdma->rx_ring.skb);
+}
+
+static void ocelot_fdma_free_tx_ring(struct ocelot *ocelot)
+{
+ struct ocelot_fdma *fdma = ocelot->fdma;
+ struct ocelot_fdma_tx_ring *tx_ring;
+ struct ocelot_fdma_tx_buf *txb;
+ struct sk_buff *skb;
+ u16 idx;
+
+ tx_ring = &fdma->tx_ring;
+ idx = tx_ring->next_to_clean;
+
+ while (idx != tx_ring->next_to_use) {
+ txb = &tx_ring->bufs[idx];
+ skb = txb->skb;
+ dma_unmap_single(ocelot->dev, dma_unmap_addr(txb, dma_addr),
+ skb->len, DMA_TO_DEVICE);
+ dev_kfree_skb_any(skb);
+ idx = ocelot_fdma_idx_next(idx, OCELOT_FDMA_TX_RING_SIZE);
+ }
+}
+
+static int ocelot_fdma_rings_alloc(struct ocelot *ocelot)
+{
+ struct ocelot_fdma *fdma = ocelot->fdma;
+ struct ocelot_fdma_dcb *dcbs;
+ unsigned int adjust;
+ dma_addr_t dcbs_dma;
+ int ret;
+
+ /* Create a pool of consistent memory blocks for hardware descriptors */
+ fdma->dcbs_base = dmam_alloc_coherent(ocelot->dev,
+ OCELOT_DCBS_HW_ALLOC_SIZE,
+ &fdma->dcbs_dma_base, GFP_KERNEL);
+ if (!fdma->dcbs_base)
+ return -ENOMEM;
+
+ /* DCBs must be aligned on a 32bit boundary */
+ dcbs = fdma->dcbs_base;
+ dcbs_dma = fdma->dcbs_dma_base;
+ if (!IS_ALIGNED(dcbs_dma, 4)) {
+ adjust = dcbs_dma & 0x3;
+ dcbs_dma = ALIGN(dcbs_dma, 4);
+ dcbs = (void *)dcbs + adjust;
+ }
+
+ /* TX queue */
+ fdma->tx_ring.dcbs = dcbs;
+ fdma->tx_ring.dcbs_dma = dcbs_dma;
+ spin_lock_init(&fdma->tx_ring.xmit_lock);
+
+ /* RX queue */
+ fdma->rx_ring.dcbs = dcbs + OCELOT_FDMA_TX_RING_SIZE;
+ fdma->rx_ring.dcbs_dma = dcbs_dma + OCELOT_FDMA_TX_DCB_SIZE;
+ ret = ocelot_fdma_alloc_rx_buffs(ocelot,
+ ocelot_fdma_tx_ring_free(fdma));
+ if (ret) {
+ ocelot_fdma_free_rx_ring(ocelot);
+ return ret;
+ }
+
+ /* Set the last DCB LLP as NULL, this is normally done when restarting
+ * the RX chan, but this is for the first run
+ */
+ ocelot_fdma_rx_set_llp(&fdma->rx_ring);
+
+ return 0;
+}
+
+void ocelot_fdma_netdev_init(struct ocelot *ocelot, struct net_device *dev)
+{
+ struct ocelot_fdma *fdma = ocelot->fdma;
+
+ dev->needed_headroom = OCELOT_TAG_LEN;
+ dev->needed_tailroom = ETH_FCS_LEN;
+
+ if (fdma->ndev)
+ return;
+
+ fdma->ndev = dev;
+ netif_napi_add(dev, &fdma->napi, ocelot_fdma_napi_poll,
+ OCELOT_FDMA_WEIGHT);
+}
+
+void ocelot_fdma_netdev_deinit(struct ocelot *ocelot, struct net_device *dev)
+{
+ struct ocelot_fdma *fdma = ocelot->fdma;
+
+ if (fdma->ndev == dev) {
+ netif_napi_del(&fdma->napi);
+ fdma->ndev = NULL;
+ }
+}
+
+void ocelot_fdma_init(struct platform_device *pdev, struct ocelot *ocelot)
+{
+ struct device *dev = ocelot->dev;
+ struct ocelot_fdma *fdma;
+ int ret;
+
+ fdma = devm_kzalloc(dev, sizeof(*fdma), GFP_KERNEL);
+ if (!fdma)
+ return;
+
+ ocelot->fdma = fdma;
+ ocelot->dev->coherent_dma_mask = DMA_BIT_MASK(32);
+
+ ocelot_fdma_writel(ocelot, MSCC_FDMA_INTR_ENA, 0);
+
+ fdma->ocelot = ocelot;
+ fdma->irq = platform_get_irq_byname(pdev, "fdma");
+ ret = devm_request_irq(dev, fdma->irq, ocelot_fdma_interrupt, 0,
+ dev_name(dev), ocelot);
+ if (ret)
+ goto err_free_fdma;
+
+ ret = ocelot_fdma_rings_alloc(ocelot);
+ if (ret)
+ goto err_free_irq;
+
+ static_branch_enable(&ocelot_fdma_enabled);
+
+ return;
+
+err_free_irq:
+ devm_free_irq(dev, fdma->irq, fdma);
+err_free_fdma:
+ devm_kfree(dev, fdma);
+
+ ocelot->fdma = NULL;
+}
+
+void ocelot_fdma_start(struct ocelot *ocelot)
+{
+ struct ocelot_fdma *fdma = ocelot->fdma;
+
+ /* Reconfigure for extraction and injection using DMA */
+ ocelot_write_rix(ocelot, QS_INJ_GRP_CFG_MODE(2), QS_INJ_GRP_CFG, 0);
+ ocelot_write_rix(ocelot, QS_INJ_CTRL_GAP_SIZE(0), QS_INJ_CTRL, 0);
+
+ ocelot_write_rix(ocelot, QS_XTR_GRP_CFG_MODE(2), QS_XTR_GRP_CFG, 0);
+
+ ocelot_fdma_writel(ocelot, MSCC_FDMA_INTR_LLP, 0xffffffff);
+ ocelot_fdma_writel(ocelot, MSCC_FDMA_INTR_FRM, 0xffffffff);
+
+ ocelot_fdma_writel(ocelot, MSCC_FDMA_INTR_LLP_ENA,
+ BIT(MSCC_FDMA_INJ_CHAN) | BIT(MSCC_FDMA_XTR_CHAN));
+ ocelot_fdma_writel(ocelot, MSCC_FDMA_INTR_FRM_ENA,
+ BIT(MSCC_FDMA_XTR_CHAN));
+ ocelot_fdma_writel(ocelot, MSCC_FDMA_INTR_ENA,
+ BIT(MSCC_FDMA_INJ_CHAN) | BIT(MSCC_FDMA_XTR_CHAN));
+
+ napi_enable(&fdma->napi);
+
+ ocelot_fdma_activate_chan(ocelot, ocelot->fdma->rx_ring.dcbs_dma,
+ MSCC_FDMA_XTR_CHAN);
+}
+
+void ocelot_fdma_deinit(struct ocelot *ocelot)
+{
+ struct ocelot_fdma *fdma = ocelot->fdma;
+
+ ocelot_fdma_writel(ocelot, MSCC_FDMA_INTR_ENA, 0);
+ ocelot_fdma_writel(ocelot, MSCC_FDMA_CH_FORCEDIS,
+ BIT(MSCC_FDMA_XTR_CHAN));
+ ocelot_fdma_writel(ocelot, MSCC_FDMA_CH_FORCEDIS,
+ BIT(MSCC_FDMA_INJ_CHAN));
+ napi_synchronize(&fdma->napi);
+ napi_disable(&fdma->napi);
+
+ ocelot_fdma_free_rx_ring(ocelot);
+ ocelot_fdma_free_tx_ring(ocelot);
+}
diff --git a/drivers/net/ethernet/mscc/ocelot_fdma.h b/drivers/net/ethernet/mscc/ocelot_fdma.h
new file mode 100644
index 0000000..2fc8e1d
--- /dev/null
+++ b/drivers/net/ethernet/mscc/ocelot_fdma.h
@@ -0,0 +1,166 @@
+/* SPDX-License-Identifier: (GPL-2.0 OR MIT) */
+/*
+ * Microsemi SoCs FDMA driver
+ *
+ * Copyright (c) 2021 Microchip
+ */
+#ifndef _MSCC_OCELOT_FDMA_H_
+#define _MSCC_OCELOT_FDMA_H_
+
+#include "ocelot.h"
+
+#define MSCC_FDMA_DCB_STAT_BLOCKO(x) (((x) << 20) & GENMASK(31, 20))
+#define MSCC_FDMA_DCB_STAT_BLOCKO_M GENMASK(31, 20)
+#define MSCC_FDMA_DCB_STAT_BLOCKO_X(x) (((x) & GENMASK(31, 20)) >> 20)
+#define MSCC_FDMA_DCB_STAT_PD BIT(19)
+#define MSCC_FDMA_DCB_STAT_ABORT BIT(18)
+#define MSCC_FDMA_DCB_STAT_EOF BIT(17)
+#define MSCC_FDMA_DCB_STAT_SOF BIT(16)
+#define MSCC_FDMA_DCB_STAT_BLOCKL_M GENMASK(15, 0)
+#define MSCC_FDMA_DCB_STAT_BLOCKL(x) ((x) & GENMASK(15, 0))
+
+#define MSCC_FDMA_DCB_LLP(x) ((x) * 4 + 0x0)
+#define MSCC_FDMA_DCB_LLP_PREV(x) ((x) * 4 + 0xA0)
+#define MSCC_FDMA_CH_SAFE 0xcc
+#define MSCC_FDMA_CH_ACTIVATE 0xd0
+#define MSCC_FDMA_CH_DISABLE 0xd4
+#define MSCC_FDMA_CH_FORCEDIS 0xd8
+#define MSCC_FDMA_EVT_ERR 0x164
+#define MSCC_FDMA_EVT_ERR_CODE 0x168
+#define MSCC_FDMA_INTR_LLP 0x16c
+#define MSCC_FDMA_INTR_LLP_ENA 0x170
+#define MSCC_FDMA_INTR_FRM 0x174
+#define MSCC_FDMA_INTR_FRM_ENA 0x178
+#define MSCC_FDMA_INTR_ENA 0x184
+#define MSCC_FDMA_INTR_IDENT 0x188
+
+#define MSCC_FDMA_INJ_CHAN 2
+#define MSCC_FDMA_XTR_CHAN 0
+
+#define OCELOT_FDMA_WEIGHT 32
+
+#define OCELOT_FDMA_CH_SAFE_TIMEOUT_US 10
+
+#define OCELOT_FDMA_RX_RING_SIZE 512
+#define OCELOT_FDMA_TX_RING_SIZE 128
+
+#define OCELOT_FDMA_RX_DCB_SIZE (OCELOT_FDMA_RX_RING_SIZE * \
+ sizeof(struct ocelot_fdma_dcb))
+#define OCELOT_FDMA_TX_DCB_SIZE (OCELOT_FDMA_TX_RING_SIZE * \
+ sizeof(struct ocelot_fdma_dcb))
+/* +4 allows for word alignment after allocation */
+#define OCELOT_DCBS_HW_ALLOC_SIZE (OCELOT_FDMA_RX_DCB_SIZE + \
+ OCELOT_FDMA_TX_DCB_SIZE + \
+ 4)
+
+#define OCELOT_FDMA_RX_SIZE (PAGE_SIZE / 2)
+
+#define OCELOT_FDMA_SKBFRAG_OVR (4 + SKB_DATA_ALIGN(sizeof(struct skb_shared_info)))
+#define OCELOT_FDMA_RXB_SIZE ALIGN_DOWN(OCELOT_FDMA_RX_SIZE - OCELOT_FDMA_SKBFRAG_OVR, 4)
+#define OCELOT_FDMA_SKBFRAG_SIZE (OCELOT_FDMA_RXB_SIZE + OCELOT_FDMA_SKBFRAG_OVR)
+
+DECLARE_STATIC_KEY_FALSE(ocelot_fdma_enabled);
+
+struct ocelot_fdma_dcb {
+ u32 llp;
+ u32 datap;
+ u32 datal;
+ u32 stat;
+} __packed;
+
+/**
+ * struct ocelot_fdma_tx_buf - TX buffer structure
+ * @skb: SKB currently used in the corresponding DCB.
+ * @dma_addr: SKB DMA mapped address.
+ */
+struct ocelot_fdma_tx_buf {
+ struct sk_buff *skb;
+ DEFINE_DMA_UNMAP_ADDR(dma_addr);
+};
+
+/**
+ * struct ocelot_fdma_tx_ring - TX ring description of DCBs
+ *
+ * @dcbs: DCBs allocated for the ring
+ * @dcbs_dma: DMA base address of the DCBs
+ * @bufs: List of TX buffer associated to the DCBs
+ * @xmit_lock: lock for concurrent xmit access
+ * @next_to_clean: Next DCB to be cleaned in tx_cleanup
+ * @next_to_use: Next available DCB to send SKB
+ */
+struct ocelot_fdma_tx_ring {
+ struct ocelot_fdma_dcb *dcbs;
+ dma_addr_t dcbs_dma;
+ struct ocelot_fdma_tx_buf bufs[OCELOT_FDMA_TX_RING_SIZE];
+ /* Protect concurrent xmit calls */
+ spinlock_t xmit_lock;
+ u16 next_to_clean;
+ u16 next_to_use;
+};
+
+/**
+ * struct ocelot_fdma_rx_buf - RX buffer structure
+ * @page: Struct page used in this buffer
+ * @page_offset: Current page offset (either 0 or PAGE_SIZE/2)
+ * @dma_addr: DMA address of the page
+ */
+struct ocelot_fdma_rx_buf {
+ struct page *page;
+ u32 page_offset;
+ dma_addr_t dma_addr;
+};
+
+/**
+ * struct ocelot_fdma_rx_ring - TX ring description of DCBs
+ *
+ * @dcbs: DCBs allocated for the ring
+ * @dcbs_dma: DMA base address of the DCBs
+ * @bufs: List of RX buffer associated to the DCBs
+ * @skb: SKB currently received by the netdev
+ * @next_to_clean: Next DCB to be cleaned NAPI polling
+ * @next_to_use: Next available DCB to send SKB
+ * @next_to_alloc: Next buffer that needs to be allocated (page reuse or alloc)
+ */
+struct ocelot_fdma_rx_ring {
+ struct ocelot_fdma_dcb *dcbs;
+ dma_addr_t dcbs_dma;
+ struct ocelot_fdma_rx_buf bufs[OCELOT_FDMA_RX_RING_SIZE];
+ struct sk_buff *skb;
+ u16 next_to_clean;
+ u16 next_to_use;
+ u16 next_to_alloc;
+};
+
+/**
+ * struct ocelot_fdma - FDMA context
+ *
+ * @irq: FDMA interrupt
+ * @ndev: Net device used to initialize NAPI
+ * @dcbs_base: Memory coherent DCBs
+ * @dcbs_dma_base: DMA base address of memory coherent DCBs
+ * @tx_ring: Injection ring
+ * @rx_ring: Extraction ring
+ * @napi: NAPI context
+ * @ocelot: Back-pointer to ocelot struct
+ */
+struct ocelot_fdma {
+ int irq;
+ struct net_device *ndev;
+ struct ocelot_fdma_dcb *dcbs_base;
+ dma_addr_t dcbs_dma_base;
+ struct ocelot_fdma_tx_ring tx_ring;
+ struct ocelot_fdma_rx_ring rx_ring;
+ struct napi_struct napi;
+ struct ocelot *ocelot;
+};
+
+void ocelot_fdma_init(struct platform_device *pdev, struct ocelot *ocelot);
+void ocelot_fdma_start(struct ocelot *ocelot);
+void ocelot_fdma_deinit(struct ocelot *ocelot);
+int ocelot_fdma_inject_frame(struct ocelot *fdma, int port, u32 rew_op,
+ struct sk_buff *skb, struct net_device *dev);
+void ocelot_fdma_netdev_init(struct ocelot *ocelot, struct net_device *dev);
+void ocelot_fdma_netdev_deinit(struct ocelot *ocelot,
+ struct net_device *dev);
+
+#endif
diff --git a/drivers/net/ethernet/mscc/ocelot_flower.c b/drivers/net/ethernet/mscc/ocelot_flower.c
index 769a815..beb9379 100644
--- a/drivers/net/ethernet/mscc/ocelot_flower.c
+++ b/drivers/net/ethernet/mscc/ocelot_flower.c
@@ -20,6 +20,9 @@
(1 * VCAP_BLOCK + (lookup) * VCAP_LOOKUP)
#define VCAP_IS2_CHAIN(lookup, pag) \
(2 * VCAP_BLOCK + (lookup) * VCAP_LOOKUP + (pag))
+/* PSFP chain and block ID */
+#define PSFP_BLOCK_ID OCELOT_NUM_VCAP_BLOCKS
+#define OCELOT_PSFP_CHAIN (3 * VCAP_BLOCK)
static int ocelot_chain_to_block(int chain, bool ingress)
{
@@ -46,6 +49,9 @@ static int ocelot_chain_to_block(int chain, bool ingress)
if (chain == VCAP_IS2_CHAIN(lookup, pag))
return VCAP_IS2;
+ if (chain == OCELOT_PSFP_CHAIN)
+ return PSFP_BLOCK_ID;
+
return -EOPNOTSUPP;
}
@@ -84,7 +90,8 @@ static bool ocelot_is_goto_target_valid(int goto_target, int chain,
goto_target == VCAP_IS1_CHAIN(1) ||
goto_target == VCAP_IS1_CHAIN(2) ||
goto_target == VCAP_IS2_CHAIN(0, 0) ||
- goto_target == VCAP_IS2_CHAIN(1, 0));
+ goto_target == VCAP_IS2_CHAIN(1, 0) ||
+ goto_target == OCELOT_PSFP_CHAIN);
if (chain == VCAP_IS1_CHAIN(0))
return (goto_target == VCAP_IS1_CHAIN(1));
@@ -111,7 +118,11 @@ static bool ocelot_is_goto_target_valid(int goto_target, int chain,
if (chain == VCAP_IS2_CHAIN(0, pag))
return (goto_target == VCAP_IS2_CHAIN(1, pag));
- /* VCAP IS2 lookup 1 cannot jump anywhere */
+ /* VCAP IS2 lookup 1 can goto to PSFP block if hardware support */
+ for (pag = 0; pag < VCAP_IS2_NUM_PAG; pag++)
+ if (chain == VCAP_IS2_CHAIN(1, pag))
+ return (goto_target == OCELOT_PSFP_CHAIN);
+
return false;
}
@@ -211,6 +222,7 @@ static int ocelot_flower_parse_action(struct ocelot *ocelot, int port,
const struct flow_action_entry *a;
enum ocelot_tag_tpid_sel tpid;
int i, chain, egress_port;
+ u32 pol_ix, pol_max;
u64 rate;
int err;
@@ -269,10 +281,14 @@ static int ocelot_flower_parse_action(struct ocelot *ocelot, int port,
filter->type = OCELOT_VCAP_FILTER_OFFLOAD;
break;
case FLOW_ACTION_POLICE:
+ if (filter->block_id == PSFP_BLOCK_ID) {
+ filter->type = OCELOT_PSFP_FILTER_OFFLOAD;
+ break;
+ }
if (filter->block_id != VCAP_IS2 ||
filter->lookup != 0) {
NL_SET_ERR_MSG_MOD(extack,
- "Police action can only be offloaded to VCAP IS2 lookup 0");
+ "Police action can only be offloaded to VCAP IS2 lookup 0 or PSFP");
return -EOPNOTSUPP;
}
if (filter->goto_target != -1) {
@@ -286,6 +302,20 @@ static int ocelot_flower_parse_action(struct ocelot *ocelot, int port,
return -EOPNOTSUPP;
}
filter->action.police_ena = true;
+
+ pol_ix = a->hw_index + ocelot->vcap_pol.base;
+ pol_max = ocelot->vcap_pol.max;
+
+ if (ocelot->vcap_pol.max2 && pol_ix > pol_max) {
+ pol_ix += ocelot->vcap_pol.base2 - pol_max - 1;
+ pol_max = ocelot->vcap_pol.max2;
+ }
+
+ if (pol_ix >= pol_max)
+ return -EINVAL;
+
+ filter->action.pol_ix = pol_ix;
+
rate = a->police.rate_bytes_ps;
filter->action.pol.rate = div_u64(rate, 1000) * 8;
filter->action.pol.burst = a->police.burst;
@@ -399,6 +429,14 @@ static int ocelot_flower_parse_action(struct ocelot *ocelot, int port,
filter->action.pcp_a_val = a->vlan.prio;
filter->type = OCELOT_VCAP_FILTER_OFFLOAD;
break;
+ case FLOW_ACTION_GATE:
+ if (filter->block_id != PSFP_BLOCK_ID) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "Gate action can only be offloaded to PSFP chain");
+ return -EOPNOTSUPP;
+ }
+ filter->type = OCELOT_PSFP_FILTER_OFFLOAD;
+ break;
default:
NL_SET_ERR_MSG_MOD(extack, "Cannot offload action");
return -EOPNOTSUPP;
@@ -407,7 +445,7 @@ static int ocelot_flower_parse_action(struct ocelot *ocelot, int port,
if (filter->goto_target == -1) {
if ((filter->block_id == VCAP_IS2 && filter->lookup == 1) ||
- chain == 0) {
+ chain == 0 || filter->block_id == PSFP_BLOCK_ID) {
allow_missing_goto_target = true;
} else {
NL_SET_ERR_MSG_MOD(extack, "Missing GOTO action");
@@ -689,6 +727,10 @@ static int ocelot_flower_parse(struct ocelot *ocelot, int port, bool ingress,
if (ret)
return ret;
+ /* PSFP filter need to parse key by stream identification function. */
+ if (filter->type == OCELOT_PSFP_FILTER_OFFLOAD)
+ return 0;
+
return ocelot_flower_parse_key(ocelot, port, ingress, f, filter);
}
@@ -792,6 +834,15 @@ int ocelot_cls_flower_replace(struct ocelot *ocelot, int port,
if (filter->type == OCELOT_VCAP_FILTER_DUMMY)
return ocelot_vcap_dummy_filter_add(ocelot, filter);
+ if (filter->type == OCELOT_PSFP_FILTER_OFFLOAD) {
+ kfree(filter);
+ if (ocelot->ops->psfp_filter_add)
+ return ocelot->ops->psfp_filter_add(ocelot, port, f);
+
+ NL_SET_ERR_MSG_MOD(extack, "PSFP chain is not supported in HW");
+ return -EOPNOTSUPP;
+ }
+
return ocelot_vcap_filter_add(ocelot, filter, f->common.extack);
}
EXPORT_SYMBOL_GPL(ocelot_cls_flower_replace);
@@ -807,6 +858,13 @@ int ocelot_cls_flower_destroy(struct ocelot *ocelot, int port,
if (block_id < 0)
return 0;
+ if (block_id == PSFP_BLOCK_ID) {
+ if (ocelot->ops->psfp_filter_del)
+ return ocelot->ops->psfp_filter_del(ocelot, f);
+
+ return -EOPNOTSUPP;
+ }
+
block = &ocelot->block[block_id];
filter = ocelot_vcap_block_find_filter_by_id(block, f->cookie, true);
@@ -825,12 +883,25 @@ int ocelot_cls_flower_stats(struct ocelot *ocelot, int port,
{
struct ocelot_vcap_filter *filter;
struct ocelot_vcap_block *block;
+ struct flow_stats stats = {0};
int block_id, ret;
block_id = ocelot_chain_to_block(f->common.chain_index, ingress);
if (block_id < 0)
return 0;
+ if (block_id == PSFP_BLOCK_ID) {
+ if (ocelot->ops->psfp_stats_get) {
+ ret = ocelot->ops->psfp_stats_get(ocelot, f, &stats);
+ if (ret)
+ return ret;
+
+ goto stats_update;
+ }
+
+ return -EOPNOTSUPP;
+ }
+
block = &ocelot->block[block_id];
filter = ocelot_vcap_block_find_filter_by_id(block, f->cookie, true);
@@ -841,7 +912,10 @@ int ocelot_cls_flower_stats(struct ocelot *ocelot, int port,
if (ret)
return ret;
- flow_stats_update(&f->stats, 0x0, filter->stats.pkts, 0, 0x0,
+ stats.pkts = filter->stats.pkts;
+
+stats_update:
+ flow_stats_update(&f->stats, 0x0, stats.pkts, stats.drops, 0x0,
FLOW_ACTION_HW_STATS_IMMEDIATE);
return 0;
}
diff --git a/drivers/net/ethernet/mscc/ocelot_net.c b/drivers/net/ethernet/mscc/ocelot_net.c
index eaeba60..8115c3d 100644
--- a/drivers/net/ethernet/mscc/ocelot_net.c
+++ b/drivers/net/ethernet/mscc/ocelot_net.c
@@ -15,6 +15,7 @@
#include <net/pkt_cls.h>
#include "ocelot.h"
#include "ocelot_vcap.h"
+#include "ocelot_fdma.h"
#define OCELOT_MAC_QUIRKS OCELOT_QUIRK_QSGMII_PORTS_MUST_BE_UP
@@ -457,7 +458,8 @@ static netdev_tx_t ocelot_port_xmit(struct sk_buff *skb, struct net_device *dev)
int port = priv->chip_port;
u32 rew_op = 0;
- if (!ocelot_can_inject(ocelot, 0))
+ if (!static_branch_unlikely(&ocelot_fdma_enabled) &&
+ !ocelot_can_inject(ocelot, 0))
return NETDEV_TX_BUSY;
/* Check if timestamping is needed */
@@ -475,9 +477,13 @@ static netdev_tx_t ocelot_port_xmit(struct sk_buff *skb, struct net_device *dev)
rew_op = ocelot_ptp_rew_op(skb);
}
- ocelot_port_inject_frame(ocelot, port, 0, rew_op, skb);
+ if (static_branch_unlikely(&ocelot_fdma_enabled)) {
+ ocelot_fdma_inject_frame(ocelot, port, rew_op, skb, dev);
+ } else {
+ ocelot_port_inject_frame(ocelot, port, 0, rew_op, skb);
- kfree_skb(skb);
+ consume_skb(skb);
+ }
return NETDEV_TX_OK;
}
@@ -764,10 +770,23 @@ static int ocelot_ioctl(struct net_device *dev, struct ifreq *ifr, int cmd)
return phy_mii_ioctl(dev->phydev, ifr, cmd);
}
+static int ocelot_change_mtu(struct net_device *dev, int new_mtu)
+{
+ struct ocelot_port_private *priv = netdev_priv(dev);
+ struct ocelot_port *ocelot_port = &priv->port;
+ struct ocelot *ocelot = ocelot_port->ocelot;
+
+ ocelot_port_set_maxlen(ocelot, priv->chip_port, new_mtu);
+ WRITE_ONCE(dev->mtu, new_mtu);
+
+ return 0;
+}
+
static const struct net_device_ops ocelot_port_netdev_ops = {
.ndo_open = ocelot_port_open,
.ndo_stop = ocelot_port_stop,
.ndo_start_xmit = ocelot_port_xmit,
+ .ndo_change_mtu = ocelot_change_mtu,
.ndo_set_rx_mode = ocelot_set_rx_mode,
.ndo_set_mac_address = ocelot_port_set_mac_address,
.ndo_get_stats64 = ocelot_get_stats64,
@@ -1498,40 +1517,6 @@ struct notifier_block ocelot_switchdev_blocking_nb __read_mostly = {
.notifier_call = ocelot_switchdev_blocking_event,
};
-static void vsc7514_phylink_validate(struct phylink_config *config,
- unsigned long *supported,
- struct phylink_link_state *state)
-{
- struct net_device *ndev = to_net_dev(config->dev);
- struct ocelot_port_private *priv = netdev_priv(ndev);
- struct ocelot_port *ocelot_port = &priv->port;
- __ETHTOOL_DECLARE_LINK_MODE_MASK(mask) = {};
-
- if (state->interface != PHY_INTERFACE_MODE_NA &&
- state->interface != ocelot_port->phy_mode) {
- linkmode_zero(supported);
- return;
- }
-
- phylink_set_port_modes(mask);
-
- phylink_set(mask, Pause);
- phylink_set(mask, Autoneg);
- phylink_set(mask, Asym_Pause);
- phylink_set(mask, 10baseT_Half);
- phylink_set(mask, 10baseT_Full);
- phylink_set(mask, 100baseT_Half);
- phylink_set(mask, 100baseT_Full);
- phylink_set(mask, 1000baseT_Half);
- phylink_set(mask, 1000baseT_Full);
- phylink_set(mask, 1000baseX_Full);
- phylink_set(mask, 2500baseT_Full);
- phylink_set(mask, 2500baseX_Full);
-
- linkmode_and(supported, supported, mask);
- linkmode_and(state->advertising, state->advertising, mask);
-}
-
static void vsc7514_phylink_mac_config(struct phylink_config *config,
unsigned int link_an_mode,
const struct phylink_link_state *state)
@@ -1590,7 +1575,7 @@ static void vsc7514_phylink_mac_link_up(struct phylink_config *config,
}
static const struct phylink_mac_ops ocelot_phylink_ops = {
- .validate = vsc7514_phylink_validate,
+ .validate = phylink_generic_validate,
.mac_config = vsc7514_phylink_mac_config,
.mac_link_down = vsc7514_phylink_mac_link_down,
.mac_link_up = vsc7514_phylink_mac_link_up,
@@ -1654,6 +1639,11 @@ static int ocelot_port_phylink_create(struct ocelot *ocelot, int port,
priv->phylink_config.dev = &priv->dev->dev;
priv->phylink_config.type = PHYLINK_NETDEV;
+ priv->phylink_config.mac_capabilities = MAC_ASYM_PAUSE | MAC_SYM_PAUSE |
+ MAC_10 | MAC_100 | MAC_1000FD | MAC_2500FD;
+
+ __set_bit(ocelot_port->phy_mode,
+ priv->phylink_config.supported_interfaces);
phylink = phylink_create(&priv->phylink_config,
of_fwnode_handle(portnp),
@@ -1699,12 +1689,16 @@ int ocelot_probe_port(struct ocelot *ocelot, int port, struct regmap *target,
dev->netdev_ops = &ocelot_port_netdev_ops;
dev->ethtool_ops = &ocelot_ethtool_ops;
+ dev->max_mtu = OCELOT_JUMBO_MTU;
dev->hw_features |= NETIF_F_HW_VLAN_CTAG_FILTER | NETIF_F_RXFCS |
NETIF_F_HW_TC;
dev->features |= NETIF_F_HW_VLAN_CTAG_FILTER | NETIF_F_HW_TC;
- eth_hw_addr_gen(dev, ocelot->base_mac, port);
+ err = of_get_ethdev_address(portnp, dev);
+ if (err)
+ eth_hw_addr_gen(dev, ocelot->base_mac, port);
+
ocelot_mact_learn(ocelot, PGID_CPU, dev->dev_addr,
OCELOT_VLAN_UNAWARE_PVID, ENTRYTYPE_LOCKED);
@@ -1714,14 +1708,20 @@ int ocelot_probe_port(struct ocelot *ocelot, int port, struct regmap *target,
if (err)
goto out;
+ if (ocelot->fdma)
+ ocelot_fdma_netdev_init(ocelot, dev);
+
err = register_netdev(dev);
if (err) {
dev_err(ocelot->dev, "register_netdev failed\n");
- goto out;
+ goto out_fdma_deinit;
}
return 0;
+out_fdma_deinit:
+ if (ocelot->fdma)
+ ocelot_fdma_netdev_deinit(ocelot, dev);
out:
ocelot->ports[port] = NULL;
free_netdev(dev);
@@ -1734,9 +1734,14 @@ void ocelot_release_port(struct ocelot_port *ocelot_port)
struct ocelot_port_private *priv = container_of(ocelot_port,
struct ocelot_port_private,
port);
+ struct ocelot *ocelot = ocelot_port->ocelot;
+ struct ocelot_fdma *fdma = ocelot->fdma;
unregister_netdev(priv->dev);
+ if (fdma)
+ ocelot_fdma_netdev_deinit(ocelot, priv->dev);
+
if (priv->phylink) {
rtnl_lock();
phylink_disconnect_phy(priv->phylink);
diff --git a/drivers/net/ethernet/mscc/ocelot_vcap.c b/drivers/net/ethernet/mscc/ocelot_vcap.c
index 337cd08..d354441 100644
--- a/drivers/net/ethernet/mscc/ocelot_vcap.c
+++ b/drivers/net/ethernet/mscc/ocelot_vcap.c
@@ -887,10 +887,18 @@ static void vcap_entry_set(struct ocelot *ocelot, int ix,
return es0_entry_set(ocelot, ix, filter);
}
-static int ocelot_vcap_policer_add(struct ocelot *ocelot, u32 pol_ix,
- struct ocelot_policer *pol)
+struct vcap_policer_entry {
+ struct list_head list;
+ refcount_t refcount;
+ u32 pol_ix;
+};
+
+int ocelot_vcap_policer_add(struct ocelot *ocelot, u32 pol_ix,
+ struct ocelot_policer *pol)
{
struct qos_policer_conf pp = { 0 };
+ struct vcap_policer_entry *tmp;
+ int ret;
if (!pol)
return -EINVAL;
@@ -899,57 +907,74 @@ static int ocelot_vcap_policer_add(struct ocelot *ocelot, u32 pol_ix,
pp.pir = pol->rate;
pp.pbs = pol->burst;
- return qos_policer_conf_set(ocelot, 0, pol_ix, &pp);
-}
-
-static void ocelot_vcap_policer_del(struct ocelot *ocelot,
- struct ocelot_vcap_block *block,
- u32 pol_ix)
-{
- struct ocelot_vcap_filter *filter;
- struct qos_policer_conf pp = {0};
- int index = -1;
-
- if (pol_ix < block->pol_lpr)
- return;
-
- list_for_each_entry(filter, &block->rules, list) {
- index++;
- if (filter->block_id == VCAP_IS2 &&
- filter->action.police_ena &&
- filter->action.pol_ix < pol_ix) {
- filter->action.pol_ix += 1;
- ocelot_vcap_policer_add(ocelot, filter->action.pol_ix,
- &filter->action.pol);
- is2_entry_set(ocelot, index, filter);
+ list_for_each_entry(tmp, &ocelot->vcap_pol.pol_list, list)
+ if (tmp->pol_ix == pol_ix) {
+ refcount_inc(&tmp->refcount);
+ return 0;
}
+
+ tmp = kzalloc(sizeof(*tmp), GFP_KERNEL);
+ if (!tmp)
+ return -ENOMEM;
+
+ ret = qos_policer_conf_set(ocelot, 0, pol_ix, &pp);
+ if (ret) {
+ kfree(tmp);
+ return ret;
}
- pp.mode = MSCC_QOS_RATE_MODE_DISABLED;
- qos_policer_conf_set(ocelot, 0, pol_ix, &pp);
+ tmp->pol_ix = pol_ix;
+ refcount_set(&tmp->refcount, 1);
+ list_add_tail(&tmp->list, &ocelot->vcap_pol.pol_list);
- block->pol_lpr++;
+ return 0;
}
+EXPORT_SYMBOL(ocelot_vcap_policer_add);
-static void ocelot_vcap_filter_add_to_block(struct ocelot *ocelot,
- struct ocelot_vcap_block *block,
- struct ocelot_vcap_filter *filter)
+int ocelot_vcap_policer_del(struct ocelot *ocelot, u32 pol_ix)
+{
+ struct qos_policer_conf pp = {0};
+ struct vcap_policer_entry *tmp, *n;
+ u8 z = 0;
+
+ list_for_each_entry_safe(tmp, n, &ocelot->vcap_pol.pol_list, list)
+ if (tmp->pol_ix == pol_ix) {
+ z = refcount_dec_and_test(&tmp->refcount);
+ if (z) {
+ list_del(&tmp->list);
+ kfree(tmp);
+ }
+ }
+
+ if (z) {
+ pp.mode = MSCC_QOS_RATE_MODE_DISABLED;
+ return qos_policer_conf_set(ocelot, 0, pol_ix, &pp);
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL(ocelot_vcap_policer_del);
+
+static int ocelot_vcap_filter_add_to_block(struct ocelot *ocelot,
+ struct ocelot_vcap_block *block,
+ struct ocelot_vcap_filter *filter)
{
struct ocelot_vcap_filter *tmp;
struct list_head *pos, *n;
+ int ret;
if (filter->block_id == VCAP_IS2 && filter->action.police_ena) {
- block->pol_lpr--;
- filter->action.pol_ix = block->pol_lpr;
- ocelot_vcap_policer_add(ocelot, filter->action.pol_ix,
- &filter->action.pol);
+ ret = ocelot_vcap_policer_add(ocelot, filter->action.pol_ix,
+ &filter->action.pol);
+ if (ret)
+ return ret;
}
block->count++;
if (list_empty(&block->rules)) {
list_add(&filter->list, &block->rules);
- return;
+ return 0;
}
list_for_each_safe(pos, n, &block->rules) {
@@ -958,6 +983,8 @@ static void ocelot_vcap_filter_add_to_block(struct ocelot *ocelot,
break;
}
list_add(&filter->list, pos->prev);
+
+ return 0;
}
static bool ocelot_vcap_filter_equal(const struct ocelot_vcap_filter *a,
@@ -1132,7 +1159,7 @@ int ocelot_vcap_filter_add(struct ocelot *ocelot,
struct netlink_ext_ack *extack)
{
struct ocelot_vcap_block *block = &ocelot->block[filter->block_id];
- int i, index;
+ int i, index, ret;
if (!ocelot_exclusive_mac_etype_filter_rules(ocelot, filter)) {
NL_SET_ERR_MSG_MOD(extack,
@@ -1141,7 +1168,9 @@ int ocelot_vcap_filter_add(struct ocelot *ocelot,
}
/* Add filter to the linked list */
- ocelot_vcap_filter_add_to_block(ocelot, block, filter);
+ ret = ocelot_vcap_filter_add_to_block(ocelot, block, filter);
+ if (ret)
+ return ret;
/* Get the index of the inserted filter */
index = ocelot_vcap_block_get_filter_index(block, filter);
@@ -1174,7 +1203,7 @@ static void ocelot_vcap_block_remove_filter(struct ocelot *ocelot,
if (ocelot_vcap_filter_equal(filter, tmp)) {
if (tmp->block_id == VCAP_IS2 &&
tmp->action.police_ena)
- ocelot_vcap_policer_del(ocelot, block,
+ ocelot_vcap_policer_del(ocelot,
tmp->action.pol_ix);
list_del(pos);
@@ -1366,13 +1395,13 @@ int ocelot_vcap_init(struct ocelot *ocelot)
struct vcap_props *vcap = &ocelot->vcap[i];
INIT_LIST_HEAD(&block->rules);
- block->pol_lpr = OCELOT_POLICER_DISCARD - 1;
ocelot_vcap_detect_constants(ocelot, vcap);
ocelot_vcap_init_one(ocelot, vcap);
}
INIT_LIST_HEAD(&ocelot->dummy_rules);
+ INIT_LIST_HEAD(&ocelot->vcap_pol.pol_list);
return 0;
}
diff --git a/drivers/net/ethernet/mscc/ocelot_vsc7514.c b/drivers/net/ethernet/mscc/ocelot_vsc7514.c
index 38103b0..4f4a495a 100644
--- a/drivers/net/ethernet/mscc/ocelot_vsc7514.c
+++ b/drivers/net/ethernet/mscc/ocelot_vsc7514.c
@@ -18,313 +18,24 @@
#include <soc/mscc/ocelot_vcap.h>
#include <soc/mscc/ocelot_hsio.h>
+#include <soc/mscc/vsc7514_regs.h>
+#include "ocelot_fdma.h"
#include "ocelot.h"
-static const u32 ocelot_ana_regmap[] = {
- REG(ANA_ADVLEARN, 0x009000),
- REG(ANA_VLANMASK, 0x009004),
- REG(ANA_PORT_B_DOMAIN, 0x009008),
- REG(ANA_ANAGEFIL, 0x00900c),
- REG(ANA_ANEVENTS, 0x009010),
- REG(ANA_STORMLIMIT_BURST, 0x009014),
- REG(ANA_STORMLIMIT_CFG, 0x009018),
- REG(ANA_ISOLATED_PORTS, 0x009028),
- REG(ANA_COMMUNITY_PORTS, 0x00902c),
- REG(ANA_AUTOAGE, 0x009030),
- REG(ANA_MACTOPTIONS, 0x009034),
- REG(ANA_LEARNDISC, 0x009038),
- REG(ANA_AGENCTRL, 0x00903c),
- REG(ANA_MIRRORPORTS, 0x009040),
- REG(ANA_EMIRRORPORTS, 0x009044),
- REG(ANA_FLOODING, 0x009048),
- REG(ANA_FLOODING_IPMC, 0x00904c),
- REG(ANA_SFLOW_CFG, 0x009050),
- REG(ANA_PORT_MODE, 0x009080),
- REG(ANA_PGID_PGID, 0x008c00),
- REG(ANA_TABLES_ANMOVED, 0x008b30),
- REG(ANA_TABLES_MACHDATA, 0x008b34),
- REG(ANA_TABLES_MACLDATA, 0x008b38),
- REG(ANA_TABLES_MACACCESS, 0x008b3c),
- REG(ANA_TABLES_MACTINDX, 0x008b40),
- REG(ANA_TABLES_VLANACCESS, 0x008b44),
- REG(ANA_TABLES_VLANTIDX, 0x008b48),
- REG(ANA_TABLES_ISDXACCESS, 0x008b4c),
- REG(ANA_TABLES_ISDXTIDX, 0x008b50),
- REG(ANA_TABLES_ENTRYLIM, 0x008b00),
- REG(ANA_TABLES_PTP_ID_HIGH, 0x008b54),
- REG(ANA_TABLES_PTP_ID_LOW, 0x008b58),
- REG(ANA_MSTI_STATE, 0x008e00),
- REG(ANA_PORT_VLAN_CFG, 0x007000),
- REG(ANA_PORT_DROP_CFG, 0x007004),
- REG(ANA_PORT_QOS_CFG, 0x007008),
- REG(ANA_PORT_VCAP_CFG, 0x00700c),
- REG(ANA_PORT_VCAP_S1_KEY_CFG, 0x007010),
- REG(ANA_PORT_VCAP_S2_CFG, 0x00701c),
- REG(ANA_PORT_PCP_DEI_MAP, 0x007020),
- REG(ANA_PORT_CPU_FWD_CFG, 0x007060),
- REG(ANA_PORT_CPU_FWD_BPDU_CFG, 0x007064),
- REG(ANA_PORT_CPU_FWD_GARP_CFG, 0x007068),
- REG(ANA_PORT_CPU_FWD_CCM_CFG, 0x00706c),
- REG(ANA_PORT_PORT_CFG, 0x007070),
- REG(ANA_PORT_POL_CFG, 0x007074),
- REG(ANA_PORT_PTP_CFG, 0x007078),
- REG(ANA_PORT_PTP_DLY1_CFG, 0x00707c),
- REG(ANA_OAM_UPM_LM_CNT, 0x007c00),
- REG(ANA_PORT_PTP_DLY2_CFG, 0x007080),
- REG(ANA_PFC_PFC_CFG, 0x008800),
- REG(ANA_PFC_PFC_TIMER, 0x008804),
- REG(ANA_IPT_OAM_MEP_CFG, 0x008000),
- REG(ANA_IPT_IPT, 0x008004),
- REG(ANA_PPT_PPT, 0x008ac0),
- REG(ANA_FID_MAP_FID_MAP, 0x000000),
- REG(ANA_AGGR_CFG, 0x0090b4),
- REG(ANA_CPUQ_CFG, 0x0090b8),
- REG(ANA_CPUQ_CFG2, 0x0090bc),
- REG(ANA_CPUQ_8021_CFG, 0x0090c0),
- REG(ANA_DSCP_CFG, 0x009100),
- REG(ANA_DSCP_REWR_CFG, 0x009200),
- REG(ANA_VCAP_RNG_TYPE_CFG, 0x009240),
- REG(ANA_VCAP_RNG_VAL_CFG, 0x009260),
- REG(ANA_VRAP_CFG, 0x009280),
- REG(ANA_VRAP_HDR_DATA, 0x009284),
- REG(ANA_VRAP_HDR_MASK, 0x009288),
- REG(ANA_DISCARD_CFG, 0x00928c),
- REG(ANA_FID_CFG, 0x009290),
- REG(ANA_POL_PIR_CFG, 0x004000),
- REG(ANA_POL_CIR_CFG, 0x004004),
- REG(ANA_POL_MODE_CFG, 0x004008),
- REG(ANA_POL_PIR_STATE, 0x00400c),
- REG(ANA_POL_CIR_STATE, 0x004010),
- REG(ANA_POL_STATE, 0x004014),
- REG(ANA_POL_FLOWC, 0x008b80),
- REG(ANA_POL_HYST, 0x008bec),
- REG(ANA_POL_MISC_CFG, 0x008bf0),
-};
-
-static const u32 ocelot_qs_regmap[] = {
- REG(QS_XTR_GRP_CFG, 0x000000),
- REG(QS_XTR_RD, 0x000008),
- REG(QS_XTR_FRM_PRUNING, 0x000010),
- REG(QS_XTR_FLUSH, 0x000018),
- REG(QS_XTR_DATA_PRESENT, 0x00001c),
- REG(QS_XTR_CFG, 0x000020),
- REG(QS_INJ_GRP_CFG, 0x000024),
- REG(QS_INJ_WR, 0x00002c),
- REG(QS_INJ_CTRL, 0x000034),
- REG(QS_INJ_STATUS, 0x00003c),
- REG(QS_INJ_ERR, 0x000040),
- REG(QS_INH_DBG, 0x000048),
-};
-
-static const u32 ocelot_qsys_regmap[] = {
- REG(QSYS_PORT_MODE, 0x011200),
- REG(QSYS_SWITCH_PORT_MODE, 0x011234),
- REG(QSYS_STAT_CNT_CFG, 0x011264),
- REG(QSYS_EEE_CFG, 0x011268),
- REG(QSYS_EEE_THRES, 0x011294),
- REG(QSYS_IGR_NO_SHARING, 0x011298),
- REG(QSYS_EGR_NO_SHARING, 0x01129c),
- REG(QSYS_SW_STATUS, 0x0112a0),
- REG(QSYS_EXT_CPU_CFG, 0x0112d0),
- REG(QSYS_PAD_CFG, 0x0112d4),
- REG(QSYS_CPU_GROUP_MAP, 0x0112d8),
- REG(QSYS_QMAP, 0x0112dc),
- REG(QSYS_ISDX_SGRP, 0x011400),
- REG(QSYS_TIMED_FRAME_ENTRY, 0x014000),
- REG(QSYS_TFRM_MISC, 0x011310),
- REG(QSYS_TFRM_PORT_DLY, 0x011314),
- REG(QSYS_TFRM_TIMER_CFG_1, 0x011318),
- REG(QSYS_TFRM_TIMER_CFG_2, 0x01131c),
- REG(QSYS_TFRM_TIMER_CFG_3, 0x011320),
- REG(QSYS_TFRM_TIMER_CFG_4, 0x011324),
- REG(QSYS_TFRM_TIMER_CFG_5, 0x011328),
- REG(QSYS_TFRM_TIMER_CFG_6, 0x01132c),
- REG(QSYS_TFRM_TIMER_CFG_7, 0x011330),
- REG(QSYS_TFRM_TIMER_CFG_8, 0x011334),
- REG(QSYS_RED_PROFILE, 0x011338),
- REG(QSYS_RES_QOS_MODE, 0x011378),
- REG(QSYS_RES_CFG, 0x012000),
- REG(QSYS_RES_STAT, 0x012004),
- REG(QSYS_EGR_DROP_MODE, 0x01137c),
- REG(QSYS_EQ_CTRL, 0x011380),
- REG(QSYS_EVENTS_CORE, 0x011384),
- REG(QSYS_CIR_CFG, 0x000000),
- REG(QSYS_EIR_CFG, 0x000004),
- REG(QSYS_SE_CFG, 0x000008),
- REG(QSYS_SE_DWRR_CFG, 0x00000c),
- REG(QSYS_SE_CONNECT, 0x00003c),
- REG(QSYS_SE_DLB_SENSE, 0x000040),
- REG(QSYS_CIR_STATE, 0x000044),
- REG(QSYS_EIR_STATE, 0x000048),
- REG(QSYS_SE_STATE, 0x00004c),
- REG(QSYS_HSCH_MISC_CFG, 0x011388),
-};
-
-static const u32 ocelot_rew_regmap[] = {
- REG(REW_PORT_VLAN_CFG, 0x000000),
- REG(REW_TAG_CFG, 0x000004),
- REG(REW_PORT_CFG, 0x000008),
- REG(REW_DSCP_CFG, 0x00000c),
- REG(REW_PCP_DEI_QOS_MAP_CFG, 0x000010),
- REG(REW_PTP_CFG, 0x000050),
- REG(REW_PTP_DLY1_CFG, 0x000054),
- REG(REW_DSCP_REMAP_DP1_CFG, 0x000690),
- REG(REW_DSCP_REMAP_CFG, 0x000790),
- REG(REW_STAT_CFG, 0x000890),
- REG(REW_PPT, 0x000680),
-};
-
-static const u32 ocelot_sys_regmap[] = {
- REG(SYS_COUNT_RX_OCTETS, 0x000000),
- REG(SYS_COUNT_RX_UNICAST, 0x000004),
- REG(SYS_COUNT_RX_MULTICAST, 0x000008),
- REG(SYS_COUNT_RX_BROADCAST, 0x00000c),
- REG(SYS_COUNT_RX_SHORTS, 0x000010),
- REG(SYS_COUNT_RX_FRAGMENTS, 0x000014),
- REG(SYS_COUNT_RX_JABBERS, 0x000018),
- REG(SYS_COUNT_RX_CRC_ALIGN_ERRS, 0x00001c),
- REG(SYS_COUNT_RX_SYM_ERRS, 0x000020),
- REG(SYS_COUNT_RX_64, 0x000024),
- REG(SYS_COUNT_RX_65_127, 0x000028),
- REG(SYS_COUNT_RX_128_255, 0x00002c),
- REG(SYS_COUNT_RX_256_1023, 0x000030),
- REG(SYS_COUNT_RX_1024_1526, 0x000034),
- REG(SYS_COUNT_RX_1527_MAX, 0x000038),
- REG(SYS_COUNT_RX_PAUSE, 0x00003c),
- REG(SYS_COUNT_RX_CONTROL, 0x000040),
- REG(SYS_COUNT_RX_LONGS, 0x000044),
- REG(SYS_COUNT_RX_CLASSIFIED_DROPS, 0x000048),
- REG(SYS_COUNT_TX_OCTETS, 0x000100),
- REG(SYS_COUNT_TX_UNICAST, 0x000104),
- REG(SYS_COUNT_TX_MULTICAST, 0x000108),
- REG(SYS_COUNT_TX_BROADCAST, 0x00010c),
- REG(SYS_COUNT_TX_COLLISION, 0x000110),
- REG(SYS_COUNT_TX_DROPS, 0x000114),
- REG(SYS_COUNT_TX_PAUSE, 0x000118),
- REG(SYS_COUNT_TX_64, 0x00011c),
- REG(SYS_COUNT_TX_65_127, 0x000120),
- REG(SYS_COUNT_TX_128_511, 0x000124),
- REG(SYS_COUNT_TX_512_1023, 0x000128),
- REG(SYS_COUNT_TX_1024_1526, 0x00012c),
- REG(SYS_COUNT_TX_1527_MAX, 0x000130),
- REG(SYS_COUNT_TX_AGING, 0x000170),
- REG(SYS_RESET_CFG, 0x000508),
- REG(SYS_CMID, 0x00050c),
- REG(SYS_VLAN_ETYPE_CFG, 0x000510),
- REG(SYS_PORT_MODE, 0x000514),
- REG(SYS_FRONT_PORT_MODE, 0x000548),
- REG(SYS_FRM_AGING, 0x000574),
- REG(SYS_STAT_CFG, 0x000578),
- REG(SYS_SW_STATUS, 0x00057c),
- REG(SYS_MISC_CFG, 0x0005ac),
- REG(SYS_REW_MAC_HIGH_CFG, 0x0005b0),
- REG(SYS_REW_MAC_LOW_CFG, 0x0005dc),
- REG(SYS_CM_ADDR, 0x000500),
- REG(SYS_CM_DATA, 0x000504),
- REG(SYS_PAUSE_CFG, 0x000608),
- REG(SYS_PAUSE_TOT_CFG, 0x000638),
- REG(SYS_ATOP, 0x00063c),
- REG(SYS_ATOP_TOT_CFG, 0x00066c),
- REG(SYS_MAC_FC_CFG, 0x000670),
- REG(SYS_MMGT, 0x00069c),
- REG(SYS_MMGT_FAST, 0x0006a0),
- REG(SYS_EVENTS_DIF, 0x0006a4),
- REG(SYS_EVENTS_CORE, 0x0006b4),
- REG(SYS_CNT, 0x000000),
- REG(SYS_PTP_STATUS, 0x0006b8),
- REG(SYS_PTP_TXSTAMP, 0x0006bc),
- REG(SYS_PTP_NXT, 0x0006c0),
- REG(SYS_PTP_CFG, 0x0006c4),
-};
-
-static const u32 ocelot_vcap_regmap[] = {
- /* VCAP_CORE_CFG */
- REG(VCAP_CORE_UPDATE_CTRL, 0x000000),
- REG(VCAP_CORE_MV_CFG, 0x000004),
- /* VCAP_CORE_CACHE */
- REG(VCAP_CACHE_ENTRY_DAT, 0x000008),
- REG(VCAP_CACHE_MASK_DAT, 0x000108),
- REG(VCAP_CACHE_ACTION_DAT, 0x000208),
- REG(VCAP_CACHE_CNT_DAT, 0x000308),
- REG(VCAP_CACHE_TG_DAT, 0x000388),
- /* VCAP_CONST */
- REG(VCAP_CONST_VCAP_VER, 0x000398),
- REG(VCAP_CONST_ENTRY_WIDTH, 0x00039c),
- REG(VCAP_CONST_ENTRY_CNT, 0x0003a0),
- REG(VCAP_CONST_ENTRY_SWCNT, 0x0003a4),
- REG(VCAP_CONST_ENTRY_TG_WIDTH, 0x0003a8),
- REG(VCAP_CONST_ACTION_DEF_CNT, 0x0003ac),
- REG(VCAP_CONST_ACTION_WIDTH, 0x0003b0),
- REG(VCAP_CONST_CNT_WIDTH, 0x0003b4),
- REG(VCAP_CONST_CORE_CNT, 0x0003b8),
- REG(VCAP_CONST_IF_CNT, 0x0003bc),
-};
-
-static const u32 ocelot_ptp_regmap[] = {
- REG(PTP_PIN_CFG, 0x000000),
- REG(PTP_PIN_TOD_SEC_MSB, 0x000004),
- REG(PTP_PIN_TOD_SEC_LSB, 0x000008),
- REG(PTP_PIN_TOD_NSEC, 0x00000c),
- REG(PTP_PIN_WF_HIGH_PERIOD, 0x000014),
- REG(PTP_PIN_WF_LOW_PERIOD, 0x000018),
- REG(PTP_CFG_MISC, 0x0000a0),
- REG(PTP_CLK_CFG_ADJ_CFG, 0x0000a4),
- REG(PTP_CLK_CFG_ADJ_FREQ, 0x0000a8),
-};
-
-static const u32 ocelot_dev_gmii_regmap[] = {
- REG(DEV_CLOCK_CFG, 0x0),
- REG(DEV_PORT_MISC, 0x4),
- REG(DEV_EVENTS, 0x8),
- REG(DEV_EEE_CFG, 0xc),
- REG(DEV_RX_PATH_DELAY, 0x10),
- REG(DEV_TX_PATH_DELAY, 0x14),
- REG(DEV_PTP_PREDICT_CFG, 0x18),
- REG(DEV_MAC_ENA_CFG, 0x1c),
- REG(DEV_MAC_MODE_CFG, 0x20),
- REG(DEV_MAC_MAXLEN_CFG, 0x24),
- REG(DEV_MAC_TAGS_CFG, 0x28),
- REG(DEV_MAC_ADV_CHK_CFG, 0x2c),
- REG(DEV_MAC_IFG_CFG, 0x30),
- REG(DEV_MAC_HDX_CFG, 0x34),
- REG(DEV_MAC_DBG_CFG, 0x38),
- REG(DEV_MAC_FC_MAC_LOW_CFG, 0x3c),
- REG(DEV_MAC_FC_MAC_HIGH_CFG, 0x40),
- REG(DEV_MAC_STICKY, 0x44),
- REG(PCS1G_CFG, 0x48),
- REG(PCS1G_MODE_CFG, 0x4c),
- REG(PCS1G_SD_CFG, 0x50),
- REG(PCS1G_ANEG_CFG, 0x54),
- REG(PCS1G_ANEG_NP_CFG, 0x58),
- REG(PCS1G_LB_CFG, 0x5c),
- REG(PCS1G_DBG_CFG, 0x60),
- REG(PCS1G_CDET_CFG, 0x64),
- REG(PCS1G_ANEG_STATUS, 0x68),
- REG(PCS1G_ANEG_NP_STATUS, 0x6c),
- REG(PCS1G_LINK_STATUS, 0x70),
- REG(PCS1G_LINK_DOWN_CNT, 0x74),
- REG(PCS1G_STICKY, 0x78),
- REG(PCS1G_DEBUG_STATUS, 0x7c),
- REG(PCS1G_LPI_CFG, 0x80),
- REG(PCS1G_LPI_WAKE_ERROR_CNT, 0x84),
- REG(PCS1G_LPI_STATUS, 0x88),
- REG(PCS1G_TSTPAT_MODE_CFG, 0x8c),
- REG(PCS1G_TSTPAT_STATUS, 0x90),
- REG(DEV_PCS_FX100_CFG, 0x94),
- REG(DEV_PCS_FX100_STATUS, 0x98),
-};
+#define VSC7514_VCAP_POLICER_BASE 128
+#define VSC7514_VCAP_POLICER_MAX 191
static const u32 *ocelot_regmap[TARGET_MAX] = {
- [ANA] = ocelot_ana_regmap,
- [QS] = ocelot_qs_regmap,
- [QSYS] = ocelot_qsys_regmap,
- [REW] = ocelot_rew_regmap,
- [SYS] = ocelot_sys_regmap,
- [S0] = ocelot_vcap_regmap,
- [S1] = ocelot_vcap_regmap,
- [S2] = ocelot_vcap_regmap,
- [PTP] = ocelot_ptp_regmap,
- [DEV_GMII] = ocelot_dev_gmii_regmap,
+ [ANA] = vsc7514_ana_regmap,
+ [QS] = vsc7514_qs_regmap,
+ [QSYS] = vsc7514_qsys_regmap,
+ [REW] = vsc7514_rew_regmap,
+ [SYS] = vsc7514_sys_regmap,
+ [S0] = vsc7514_vcap_regmap,
+ [S1] = vsc7514_vcap_regmap,
+ [S2] = vsc7514_vcap_regmap,
+ [PTP] = vsc7514_ptp_regmap,
+ [DEV_GMII] = vsc7514_dev_gmii_regmap,
};
static const struct reg_field ocelot_regfields[REGFIELD_MAX] = {
@@ -633,211 +344,6 @@ static const struct ocelot_ops ocelot_ops = {
.netdev_to_port = ocelot_netdev_to_port,
};
-static const struct vcap_field vsc7514_vcap_es0_keys[] = {
- [VCAP_ES0_EGR_PORT] = { 0, 4},
- [VCAP_ES0_IGR_PORT] = { 4, 4},
- [VCAP_ES0_RSV] = { 8, 2},
- [VCAP_ES0_L2_MC] = { 10, 1},
- [VCAP_ES0_L2_BC] = { 11, 1},
- [VCAP_ES0_VID] = { 12, 12},
- [VCAP_ES0_DP] = { 24, 1},
- [VCAP_ES0_PCP] = { 25, 3},
-};
-
-static const struct vcap_field vsc7514_vcap_es0_actions[] = {
- [VCAP_ES0_ACT_PUSH_OUTER_TAG] = { 0, 2},
- [VCAP_ES0_ACT_PUSH_INNER_TAG] = { 2, 1},
- [VCAP_ES0_ACT_TAG_A_TPID_SEL] = { 3, 2},
- [VCAP_ES0_ACT_TAG_A_VID_SEL] = { 5, 1},
- [VCAP_ES0_ACT_TAG_A_PCP_SEL] = { 6, 2},
- [VCAP_ES0_ACT_TAG_A_DEI_SEL] = { 8, 2},
- [VCAP_ES0_ACT_TAG_B_TPID_SEL] = { 10, 2},
- [VCAP_ES0_ACT_TAG_B_VID_SEL] = { 12, 1},
- [VCAP_ES0_ACT_TAG_B_PCP_SEL] = { 13, 2},
- [VCAP_ES0_ACT_TAG_B_DEI_SEL] = { 15, 2},
- [VCAP_ES0_ACT_VID_A_VAL] = { 17, 12},
- [VCAP_ES0_ACT_PCP_A_VAL] = { 29, 3},
- [VCAP_ES0_ACT_DEI_A_VAL] = { 32, 1},
- [VCAP_ES0_ACT_VID_B_VAL] = { 33, 12},
- [VCAP_ES0_ACT_PCP_B_VAL] = { 45, 3},
- [VCAP_ES0_ACT_DEI_B_VAL] = { 48, 1},
- [VCAP_ES0_ACT_RSV] = { 49, 24},
- [VCAP_ES0_ACT_HIT_STICKY] = { 73, 1},
-};
-
-static const struct vcap_field vsc7514_vcap_is1_keys[] = {
- [VCAP_IS1_HK_TYPE] = { 0, 1},
- [VCAP_IS1_HK_LOOKUP] = { 1, 2},
- [VCAP_IS1_HK_IGR_PORT_MASK] = { 3, 12},
- [VCAP_IS1_HK_RSV] = { 15, 9},
- [VCAP_IS1_HK_OAM_Y1731] = { 24, 1},
- [VCAP_IS1_HK_L2_MC] = { 25, 1},
- [VCAP_IS1_HK_L2_BC] = { 26, 1},
- [VCAP_IS1_HK_IP_MC] = { 27, 1},
- [VCAP_IS1_HK_VLAN_TAGGED] = { 28, 1},
- [VCAP_IS1_HK_VLAN_DBL_TAGGED] = { 29, 1},
- [VCAP_IS1_HK_TPID] = { 30, 1},
- [VCAP_IS1_HK_VID] = { 31, 12},
- [VCAP_IS1_HK_DEI] = { 43, 1},
- [VCAP_IS1_HK_PCP] = { 44, 3},
- /* Specific Fields for IS1 Half Key S1_NORMAL */
- [VCAP_IS1_HK_L2_SMAC] = { 47, 48},
- [VCAP_IS1_HK_ETYPE_LEN] = { 95, 1},
- [VCAP_IS1_HK_ETYPE] = { 96, 16},
- [VCAP_IS1_HK_IP_SNAP] = {112, 1},
- [VCAP_IS1_HK_IP4] = {113, 1},
- /* Layer-3 Information */
- [VCAP_IS1_HK_L3_FRAGMENT] = {114, 1},
- [VCAP_IS1_HK_L3_FRAG_OFS_GT0] = {115, 1},
- [VCAP_IS1_HK_L3_OPTIONS] = {116, 1},
- [VCAP_IS1_HK_L3_DSCP] = {117, 6},
- [VCAP_IS1_HK_L3_IP4_SIP] = {123, 32},
- /* Layer-4 Information */
- [VCAP_IS1_HK_TCP_UDP] = {155, 1},
- [VCAP_IS1_HK_TCP] = {156, 1},
- [VCAP_IS1_HK_L4_SPORT] = {157, 16},
- [VCAP_IS1_HK_L4_RNG] = {173, 8},
- /* Specific Fields for IS1 Half Key S1_5TUPLE_IP4 */
- [VCAP_IS1_HK_IP4_INNER_TPID] = { 47, 1},
- [VCAP_IS1_HK_IP4_INNER_VID] = { 48, 12},
- [VCAP_IS1_HK_IP4_INNER_DEI] = { 60, 1},
- [VCAP_IS1_HK_IP4_INNER_PCP] = { 61, 3},
- [VCAP_IS1_HK_IP4_IP4] = { 64, 1},
- [VCAP_IS1_HK_IP4_L3_FRAGMENT] = { 65, 1},
- [VCAP_IS1_HK_IP4_L3_FRAG_OFS_GT0] = { 66, 1},
- [VCAP_IS1_HK_IP4_L3_OPTIONS] = { 67, 1},
- [VCAP_IS1_HK_IP4_L3_DSCP] = { 68, 6},
- [VCAP_IS1_HK_IP4_L3_IP4_DIP] = { 74, 32},
- [VCAP_IS1_HK_IP4_L3_IP4_SIP] = {106, 32},
- [VCAP_IS1_HK_IP4_L3_PROTO] = {138, 8},
- [VCAP_IS1_HK_IP4_TCP_UDP] = {146, 1},
- [VCAP_IS1_HK_IP4_TCP] = {147, 1},
- [VCAP_IS1_HK_IP4_L4_RNG] = {148, 8},
- [VCAP_IS1_HK_IP4_IP_PAYLOAD_S1_5TUPLE] = {156, 32},
-};
-
-static const struct vcap_field vsc7514_vcap_is1_actions[] = {
- [VCAP_IS1_ACT_DSCP_ENA] = { 0, 1},
- [VCAP_IS1_ACT_DSCP_VAL] = { 1, 6},
- [VCAP_IS1_ACT_QOS_ENA] = { 7, 1},
- [VCAP_IS1_ACT_QOS_VAL] = { 8, 3},
- [VCAP_IS1_ACT_DP_ENA] = { 11, 1},
- [VCAP_IS1_ACT_DP_VAL] = { 12, 1},
- [VCAP_IS1_ACT_PAG_OVERRIDE_MASK] = { 13, 8},
- [VCAP_IS1_ACT_PAG_VAL] = { 21, 8},
- [VCAP_IS1_ACT_RSV] = { 29, 9},
- /* The fields below are incorrectly shifted by 2 in the manual */
- [VCAP_IS1_ACT_VID_REPLACE_ENA] = { 38, 1},
- [VCAP_IS1_ACT_VID_ADD_VAL] = { 39, 12},
- [VCAP_IS1_ACT_FID_SEL] = { 51, 2},
- [VCAP_IS1_ACT_FID_VAL] = { 53, 13},
- [VCAP_IS1_ACT_PCP_DEI_ENA] = { 66, 1},
- [VCAP_IS1_ACT_PCP_VAL] = { 67, 3},
- [VCAP_IS1_ACT_DEI_VAL] = { 70, 1},
- [VCAP_IS1_ACT_VLAN_POP_CNT_ENA] = { 71, 1},
- [VCAP_IS1_ACT_VLAN_POP_CNT] = { 72, 2},
- [VCAP_IS1_ACT_CUSTOM_ACE_TYPE_ENA] = { 74, 4},
- [VCAP_IS1_ACT_HIT_STICKY] = { 78, 1},
-};
-
-static const struct vcap_field vsc7514_vcap_is2_keys[] = {
- /* Common: 46 bits */
- [VCAP_IS2_TYPE] = { 0, 4},
- [VCAP_IS2_HK_FIRST] = { 4, 1},
- [VCAP_IS2_HK_PAG] = { 5, 8},
- [VCAP_IS2_HK_IGR_PORT_MASK] = { 13, 12},
- [VCAP_IS2_HK_RSV2] = { 25, 1},
- [VCAP_IS2_HK_HOST_MATCH] = { 26, 1},
- [VCAP_IS2_HK_L2_MC] = { 27, 1},
- [VCAP_IS2_HK_L2_BC] = { 28, 1},
- [VCAP_IS2_HK_VLAN_TAGGED] = { 29, 1},
- [VCAP_IS2_HK_VID] = { 30, 12},
- [VCAP_IS2_HK_DEI] = { 42, 1},
- [VCAP_IS2_HK_PCP] = { 43, 3},
- /* MAC_ETYPE / MAC_LLC / MAC_SNAP / OAM common */
- [VCAP_IS2_HK_L2_DMAC] = { 46, 48},
- [VCAP_IS2_HK_L2_SMAC] = { 94, 48},
- /* MAC_ETYPE (TYPE=000) */
- [VCAP_IS2_HK_MAC_ETYPE_ETYPE] = {142, 16},
- [VCAP_IS2_HK_MAC_ETYPE_L2_PAYLOAD0] = {158, 16},
- [VCAP_IS2_HK_MAC_ETYPE_L2_PAYLOAD1] = {174, 8},
- [VCAP_IS2_HK_MAC_ETYPE_L2_PAYLOAD2] = {182, 3},
- /* MAC_LLC (TYPE=001) */
- [VCAP_IS2_HK_MAC_LLC_L2_LLC] = {142, 40},
- /* MAC_SNAP (TYPE=010) */
- [VCAP_IS2_HK_MAC_SNAP_L2_SNAP] = {142, 40},
- /* MAC_ARP (TYPE=011) */
- [VCAP_IS2_HK_MAC_ARP_SMAC] = { 46, 48},
- [VCAP_IS2_HK_MAC_ARP_ADDR_SPACE_OK] = { 94, 1},
- [VCAP_IS2_HK_MAC_ARP_PROTO_SPACE_OK] = { 95, 1},
- [VCAP_IS2_HK_MAC_ARP_LEN_OK] = { 96, 1},
- [VCAP_IS2_HK_MAC_ARP_TARGET_MATCH] = { 97, 1},
- [VCAP_IS2_HK_MAC_ARP_SENDER_MATCH] = { 98, 1},
- [VCAP_IS2_HK_MAC_ARP_OPCODE_UNKNOWN] = { 99, 1},
- [VCAP_IS2_HK_MAC_ARP_OPCODE] = {100, 2},
- [VCAP_IS2_HK_MAC_ARP_L3_IP4_DIP] = {102, 32},
- [VCAP_IS2_HK_MAC_ARP_L3_IP4_SIP] = {134, 32},
- [VCAP_IS2_HK_MAC_ARP_DIP_EQ_SIP] = {166, 1},
- /* IP4_TCP_UDP / IP4_OTHER common */
- [VCAP_IS2_HK_IP4] = { 46, 1},
- [VCAP_IS2_HK_L3_FRAGMENT] = { 47, 1},
- [VCAP_IS2_HK_L3_FRAG_OFS_GT0] = { 48, 1},
- [VCAP_IS2_HK_L3_OPTIONS] = { 49, 1},
- [VCAP_IS2_HK_IP4_L3_TTL_GT0] = { 50, 1},
- [VCAP_IS2_HK_L3_TOS] = { 51, 8},
- [VCAP_IS2_HK_L3_IP4_DIP] = { 59, 32},
- [VCAP_IS2_HK_L3_IP4_SIP] = { 91, 32},
- [VCAP_IS2_HK_DIP_EQ_SIP] = {123, 1},
- /* IP4_TCP_UDP (TYPE=100) */
- [VCAP_IS2_HK_TCP] = {124, 1},
- [VCAP_IS2_HK_L4_DPORT] = {125, 16},
- [VCAP_IS2_HK_L4_SPORT] = {141, 16},
- [VCAP_IS2_HK_L4_RNG] = {157, 8},
- [VCAP_IS2_HK_L4_SPORT_EQ_DPORT] = {165, 1},
- [VCAP_IS2_HK_L4_SEQUENCE_EQ0] = {166, 1},
- [VCAP_IS2_HK_L4_FIN] = {167, 1},
- [VCAP_IS2_HK_L4_SYN] = {168, 1},
- [VCAP_IS2_HK_L4_RST] = {169, 1},
- [VCAP_IS2_HK_L4_PSH] = {170, 1},
- [VCAP_IS2_HK_L4_ACK] = {171, 1},
- [VCAP_IS2_HK_L4_URG] = {172, 1},
- [VCAP_IS2_HK_L4_1588_DOM] = {173, 8},
- [VCAP_IS2_HK_L4_1588_VER] = {181, 4},
- /* IP4_OTHER (TYPE=101) */
- [VCAP_IS2_HK_IP4_L3_PROTO] = {124, 8},
- [VCAP_IS2_HK_L3_PAYLOAD] = {132, 56},
- /* IP6_STD (TYPE=110) */
- [VCAP_IS2_HK_IP6_L3_TTL_GT0] = { 46, 1},
- [VCAP_IS2_HK_L3_IP6_SIP] = { 47, 128},
- [VCAP_IS2_HK_IP6_L3_PROTO] = {175, 8},
- /* OAM (TYPE=111) */
- [VCAP_IS2_HK_OAM_MEL_FLAGS] = {142, 7},
- [VCAP_IS2_HK_OAM_VER] = {149, 5},
- [VCAP_IS2_HK_OAM_OPCODE] = {154, 8},
- [VCAP_IS2_HK_OAM_FLAGS] = {162, 8},
- [VCAP_IS2_HK_OAM_MEPID] = {170, 16},
- [VCAP_IS2_HK_OAM_CCM_CNTS_EQ0] = {186, 1},
- [VCAP_IS2_HK_OAM_IS_Y1731] = {187, 1},
-};
-
-static const struct vcap_field vsc7514_vcap_is2_actions[] = {
- [VCAP_IS2_ACT_HIT_ME_ONCE] = { 0, 1},
- [VCAP_IS2_ACT_CPU_COPY_ENA] = { 1, 1},
- [VCAP_IS2_ACT_CPU_QU_NUM] = { 2, 3},
- [VCAP_IS2_ACT_MASK_MODE] = { 5, 2},
- [VCAP_IS2_ACT_MIRROR_ENA] = { 7, 1},
- [VCAP_IS2_ACT_LRN_DIS] = { 8, 1},
- [VCAP_IS2_ACT_POLICE_ENA] = { 9, 1},
- [VCAP_IS2_ACT_POLICE_IDX] = { 10, 9},
- [VCAP_IS2_ACT_POLICE_VCAP_ONLY] = { 19, 1},
- [VCAP_IS2_ACT_PORT_MASK] = { 20, 11},
- [VCAP_IS2_ACT_REW_OP] = { 31, 9},
- [VCAP_IS2_ACT_SMAC_REPLACE_ENA] = { 40, 1},
- [VCAP_IS2_ACT_RSV] = { 41, 2},
- [VCAP_IS2_ACT_ACL_ID] = { 43, 6},
- [VCAP_IS2_ACT_HIT_CNT] = { 49, 32},
-};
-
static struct vcap_props vsc7514_vcap_props[] = {
[VCAP_ES0] = {
.action_type_width = 0,
@@ -1045,6 +551,7 @@ static int mscc_ocelot_probe(struct platform_device *pdev)
{ S1, "s1" },
{ S2, "s2" },
{ PTP, "ptp", 1 },
+ { FDMA, "fdma", 1 },
};
if (!np && !pdev->dev.platform_data)
@@ -1080,6 +587,9 @@ static int mscc_ocelot_probe(struct platform_device *pdev)
ocelot->targets[io_target[i].id] = target;
}
+ if (ocelot->targets[FDMA])
+ ocelot_fdma_init(pdev, ocelot);
+
hsio = syscon_regmap_lookup_by_compatible("mscc,ocelot-hsio");
if (IS_ERR(hsio)) {
dev_err(&pdev->dev, "missing hsio syscon\n");
@@ -1129,6 +639,10 @@ static int mscc_ocelot_probe(struct platform_device *pdev)
ocelot->num_flooding_pgids = 1;
ocelot->vcap = vsc7514_vcap_props;
+
+ ocelot->vcap_pol.base = VSC7514_VCAP_POLICER_BASE;
+ ocelot->vcap_pol.max = VSC7514_VCAP_POLICER_MAX;
+
ocelot->npi = -1;
err = ocelot_init(ocelot);
@@ -1139,6 +653,9 @@ static int mscc_ocelot_probe(struct platform_device *pdev)
if (err)
goto out_ocelot_devlink_unregister;
+ if (ocelot->fdma)
+ ocelot_fdma_start(ocelot);
+
err = ocelot_devlink_sb_register(ocelot);
if (err)
goto out_ocelot_release_ports;
@@ -1179,6 +696,8 @@ static int mscc_ocelot_remove(struct platform_device *pdev)
{
struct ocelot *ocelot = platform_get_drvdata(pdev);
+ if (ocelot->fdma)
+ ocelot_fdma_deinit(ocelot);
devlink_unregister(ocelot->devlink);
ocelot_deinit_timestamp(ocelot);
ocelot_devlink_sb_unregister(ocelot);
diff --git a/drivers/net/ethernet/mscc/vsc7514_regs.c b/drivers/net/ethernet/mscc/vsc7514_regs.c
new file mode 100644
index 0000000..c2af4eb
--- /dev/null
+++ b/drivers/net/ethernet/mscc/vsc7514_regs.c
@@ -0,0 +1,523 @@
+// SPDX-License-Identifier: (GPL-2.0 OR MIT)
+/*
+ * Microsemi Ocelot Switch driver
+ *
+ * Copyright (c) 2017 Microsemi Corporation
+ * Copyright (c) 2021 Innovative Advantage
+ */
+#include <soc/mscc/ocelot_vcap.h>
+#include <soc/mscc/vsc7514_regs.h>
+#include "ocelot.h"
+
+const u32 vsc7514_ana_regmap[] = {
+ REG(ANA_ADVLEARN, 0x009000),
+ REG(ANA_VLANMASK, 0x009004),
+ REG(ANA_PORT_B_DOMAIN, 0x009008),
+ REG(ANA_ANAGEFIL, 0x00900c),
+ REG(ANA_ANEVENTS, 0x009010),
+ REG(ANA_STORMLIMIT_BURST, 0x009014),
+ REG(ANA_STORMLIMIT_CFG, 0x009018),
+ REG(ANA_ISOLATED_PORTS, 0x009028),
+ REG(ANA_COMMUNITY_PORTS, 0x00902c),
+ REG(ANA_AUTOAGE, 0x009030),
+ REG(ANA_MACTOPTIONS, 0x009034),
+ REG(ANA_LEARNDISC, 0x009038),
+ REG(ANA_AGENCTRL, 0x00903c),
+ REG(ANA_MIRRORPORTS, 0x009040),
+ REG(ANA_EMIRRORPORTS, 0x009044),
+ REG(ANA_FLOODING, 0x009048),
+ REG(ANA_FLOODING_IPMC, 0x00904c),
+ REG(ANA_SFLOW_CFG, 0x009050),
+ REG(ANA_PORT_MODE, 0x009080),
+ REG(ANA_PGID_PGID, 0x008c00),
+ REG(ANA_TABLES_ANMOVED, 0x008b30),
+ REG(ANA_TABLES_MACHDATA, 0x008b34),
+ REG(ANA_TABLES_MACLDATA, 0x008b38),
+ REG(ANA_TABLES_MACACCESS, 0x008b3c),
+ REG(ANA_TABLES_MACTINDX, 0x008b40),
+ REG(ANA_TABLES_VLANACCESS, 0x008b44),
+ REG(ANA_TABLES_VLANTIDX, 0x008b48),
+ REG(ANA_TABLES_ISDXACCESS, 0x008b4c),
+ REG(ANA_TABLES_ISDXTIDX, 0x008b50),
+ REG(ANA_TABLES_ENTRYLIM, 0x008b00),
+ REG(ANA_TABLES_PTP_ID_HIGH, 0x008b54),
+ REG(ANA_TABLES_PTP_ID_LOW, 0x008b58),
+ REG(ANA_MSTI_STATE, 0x008e00),
+ REG(ANA_PORT_VLAN_CFG, 0x007000),
+ REG(ANA_PORT_DROP_CFG, 0x007004),
+ REG(ANA_PORT_QOS_CFG, 0x007008),
+ REG(ANA_PORT_VCAP_CFG, 0x00700c),
+ REG(ANA_PORT_VCAP_S1_KEY_CFG, 0x007010),
+ REG(ANA_PORT_VCAP_S2_CFG, 0x00701c),
+ REG(ANA_PORT_PCP_DEI_MAP, 0x007020),
+ REG(ANA_PORT_CPU_FWD_CFG, 0x007060),
+ REG(ANA_PORT_CPU_FWD_BPDU_CFG, 0x007064),
+ REG(ANA_PORT_CPU_FWD_GARP_CFG, 0x007068),
+ REG(ANA_PORT_CPU_FWD_CCM_CFG, 0x00706c),
+ REG(ANA_PORT_PORT_CFG, 0x007070),
+ REG(ANA_PORT_POL_CFG, 0x007074),
+ REG(ANA_PORT_PTP_CFG, 0x007078),
+ REG(ANA_PORT_PTP_DLY1_CFG, 0x00707c),
+ REG(ANA_OAM_UPM_LM_CNT, 0x007c00),
+ REG(ANA_PORT_PTP_DLY2_CFG, 0x007080),
+ REG(ANA_PFC_PFC_CFG, 0x008800),
+ REG(ANA_PFC_PFC_TIMER, 0x008804),
+ REG(ANA_IPT_OAM_MEP_CFG, 0x008000),
+ REG(ANA_IPT_IPT, 0x008004),
+ REG(ANA_PPT_PPT, 0x008ac0),
+ REG(ANA_FID_MAP_FID_MAP, 0x000000),
+ REG(ANA_AGGR_CFG, 0x0090b4),
+ REG(ANA_CPUQ_CFG, 0x0090b8),
+ REG(ANA_CPUQ_CFG2, 0x0090bc),
+ REG(ANA_CPUQ_8021_CFG, 0x0090c0),
+ REG(ANA_DSCP_CFG, 0x009100),
+ REG(ANA_DSCP_REWR_CFG, 0x009200),
+ REG(ANA_VCAP_RNG_TYPE_CFG, 0x009240),
+ REG(ANA_VCAP_RNG_VAL_CFG, 0x009260),
+ REG(ANA_VRAP_CFG, 0x009280),
+ REG(ANA_VRAP_HDR_DATA, 0x009284),
+ REG(ANA_VRAP_HDR_MASK, 0x009288),
+ REG(ANA_DISCARD_CFG, 0x00928c),
+ REG(ANA_FID_CFG, 0x009290),
+ REG(ANA_POL_PIR_CFG, 0x004000),
+ REG(ANA_POL_CIR_CFG, 0x004004),
+ REG(ANA_POL_MODE_CFG, 0x004008),
+ REG(ANA_POL_PIR_STATE, 0x00400c),
+ REG(ANA_POL_CIR_STATE, 0x004010),
+ REG(ANA_POL_STATE, 0x004014),
+ REG(ANA_POL_FLOWC, 0x008b80),
+ REG(ANA_POL_HYST, 0x008bec),
+ REG(ANA_POL_MISC_CFG, 0x008bf0),
+};
+EXPORT_SYMBOL(vsc7514_ana_regmap);
+
+const u32 vsc7514_qs_regmap[] = {
+ REG(QS_XTR_GRP_CFG, 0x000000),
+ REG(QS_XTR_RD, 0x000008),
+ REG(QS_XTR_FRM_PRUNING, 0x000010),
+ REG(QS_XTR_FLUSH, 0x000018),
+ REG(QS_XTR_DATA_PRESENT, 0x00001c),
+ REG(QS_XTR_CFG, 0x000020),
+ REG(QS_INJ_GRP_CFG, 0x000024),
+ REG(QS_INJ_WR, 0x00002c),
+ REG(QS_INJ_CTRL, 0x000034),
+ REG(QS_INJ_STATUS, 0x00003c),
+ REG(QS_INJ_ERR, 0x000040),
+ REG(QS_INH_DBG, 0x000048),
+};
+EXPORT_SYMBOL(vsc7514_qs_regmap);
+
+const u32 vsc7514_qsys_regmap[] = {
+ REG(QSYS_PORT_MODE, 0x011200),
+ REG(QSYS_SWITCH_PORT_MODE, 0x011234),
+ REG(QSYS_STAT_CNT_CFG, 0x011264),
+ REG(QSYS_EEE_CFG, 0x011268),
+ REG(QSYS_EEE_THRES, 0x011294),
+ REG(QSYS_IGR_NO_SHARING, 0x011298),
+ REG(QSYS_EGR_NO_SHARING, 0x01129c),
+ REG(QSYS_SW_STATUS, 0x0112a0),
+ REG(QSYS_EXT_CPU_CFG, 0x0112d0),
+ REG(QSYS_PAD_CFG, 0x0112d4),
+ REG(QSYS_CPU_GROUP_MAP, 0x0112d8),
+ REG(QSYS_QMAP, 0x0112dc),
+ REG(QSYS_ISDX_SGRP, 0x011400),
+ REG(QSYS_TIMED_FRAME_ENTRY, 0x014000),
+ REG(QSYS_TFRM_MISC, 0x011310),
+ REG(QSYS_TFRM_PORT_DLY, 0x011314),
+ REG(QSYS_TFRM_TIMER_CFG_1, 0x011318),
+ REG(QSYS_TFRM_TIMER_CFG_2, 0x01131c),
+ REG(QSYS_TFRM_TIMER_CFG_3, 0x011320),
+ REG(QSYS_TFRM_TIMER_CFG_4, 0x011324),
+ REG(QSYS_TFRM_TIMER_CFG_5, 0x011328),
+ REG(QSYS_TFRM_TIMER_CFG_6, 0x01132c),
+ REG(QSYS_TFRM_TIMER_CFG_7, 0x011330),
+ REG(QSYS_TFRM_TIMER_CFG_8, 0x011334),
+ REG(QSYS_RED_PROFILE, 0x011338),
+ REG(QSYS_RES_QOS_MODE, 0x011378),
+ REG(QSYS_RES_CFG, 0x012000),
+ REG(QSYS_RES_STAT, 0x012004),
+ REG(QSYS_EGR_DROP_MODE, 0x01137c),
+ REG(QSYS_EQ_CTRL, 0x011380),
+ REG(QSYS_EVENTS_CORE, 0x011384),
+ REG(QSYS_CIR_CFG, 0x000000),
+ REG(QSYS_EIR_CFG, 0x000004),
+ REG(QSYS_SE_CFG, 0x000008),
+ REG(QSYS_SE_DWRR_CFG, 0x00000c),
+ REG(QSYS_SE_CONNECT, 0x00003c),
+ REG(QSYS_SE_DLB_SENSE, 0x000040),
+ REG(QSYS_CIR_STATE, 0x000044),
+ REG(QSYS_EIR_STATE, 0x000048),
+ REG(QSYS_SE_STATE, 0x00004c),
+ REG(QSYS_HSCH_MISC_CFG, 0x011388),
+};
+EXPORT_SYMBOL(vsc7514_qsys_regmap);
+
+const u32 vsc7514_rew_regmap[] = {
+ REG(REW_PORT_VLAN_CFG, 0x000000),
+ REG(REW_TAG_CFG, 0x000004),
+ REG(REW_PORT_CFG, 0x000008),
+ REG(REW_DSCP_CFG, 0x00000c),
+ REG(REW_PCP_DEI_QOS_MAP_CFG, 0x000010),
+ REG(REW_PTP_CFG, 0x000050),
+ REG(REW_PTP_DLY1_CFG, 0x000054),
+ REG(REW_DSCP_REMAP_DP1_CFG, 0x000690),
+ REG(REW_DSCP_REMAP_CFG, 0x000790),
+ REG(REW_STAT_CFG, 0x000890),
+ REG(REW_PPT, 0x000680),
+};
+EXPORT_SYMBOL(vsc7514_rew_regmap);
+
+const u32 vsc7514_sys_regmap[] = {
+ REG(SYS_COUNT_RX_OCTETS, 0x000000),
+ REG(SYS_COUNT_RX_UNICAST, 0x000004),
+ REG(SYS_COUNT_RX_MULTICAST, 0x000008),
+ REG(SYS_COUNT_RX_BROADCAST, 0x00000c),
+ REG(SYS_COUNT_RX_SHORTS, 0x000010),
+ REG(SYS_COUNT_RX_FRAGMENTS, 0x000014),
+ REG(SYS_COUNT_RX_JABBERS, 0x000018),
+ REG(SYS_COUNT_RX_CRC_ALIGN_ERRS, 0x00001c),
+ REG(SYS_COUNT_RX_SYM_ERRS, 0x000020),
+ REG(SYS_COUNT_RX_64, 0x000024),
+ REG(SYS_COUNT_RX_65_127, 0x000028),
+ REG(SYS_COUNT_RX_128_255, 0x00002c),
+ REG(SYS_COUNT_RX_256_1023, 0x000030),
+ REG(SYS_COUNT_RX_1024_1526, 0x000034),
+ REG(SYS_COUNT_RX_1527_MAX, 0x000038),
+ REG(SYS_COUNT_RX_PAUSE, 0x00003c),
+ REG(SYS_COUNT_RX_CONTROL, 0x000040),
+ REG(SYS_COUNT_RX_LONGS, 0x000044),
+ REG(SYS_COUNT_RX_CLASSIFIED_DROPS, 0x000048),
+ REG(SYS_COUNT_TX_OCTETS, 0x000100),
+ REG(SYS_COUNT_TX_UNICAST, 0x000104),
+ REG(SYS_COUNT_TX_MULTICAST, 0x000108),
+ REG(SYS_COUNT_TX_BROADCAST, 0x00010c),
+ REG(SYS_COUNT_TX_COLLISION, 0x000110),
+ REG(SYS_COUNT_TX_DROPS, 0x000114),
+ REG(SYS_COUNT_TX_PAUSE, 0x000118),
+ REG(SYS_COUNT_TX_64, 0x00011c),
+ REG(SYS_COUNT_TX_65_127, 0x000120),
+ REG(SYS_COUNT_TX_128_511, 0x000124),
+ REG(SYS_COUNT_TX_512_1023, 0x000128),
+ REG(SYS_COUNT_TX_1024_1526, 0x00012c),
+ REG(SYS_COUNT_TX_1527_MAX, 0x000130),
+ REG(SYS_COUNT_TX_AGING, 0x000170),
+ REG(SYS_RESET_CFG, 0x000508),
+ REG(SYS_CMID, 0x00050c),
+ REG(SYS_VLAN_ETYPE_CFG, 0x000510),
+ REG(SYS_PORT_MODE, 0x000514),
+ REG(SYS_FRONT_PORT_MODE, 0x000548),
+ REG(SYS_FRM_AGING, 0x000574),
+ REG(SYS_STAT_CFG, 0x000578),
+ REG(SYS_SW_STATUS, 0x00057c),
+ REG(SYS_MISC_CFG, 0x0005ac),
+ REG(SYS_REW_MAC_HIGH_CFG, 0x0005b0),
+ REG(SYS_REW_MAC_LOW_CFG, 0x0005dc),
+ REG(SYS_CM_ADDR, 0x000500),
+ REG(SYS_CM_DATA, 0x000504),
+ REG(SYS_PAUSE_CFG, 0x000608),
+ REG(SYS_PAUSE_TOT_CFG, 0x000638),
+ REG(SYS_ATOP, 0x00063c),
+ REG(SYS_ATOP_TOT_CFG, 0x00066c),
+ REG(SYS_MAC_FC_CFG, 0x000670),
+ REG(SYS_MMGT, 0x00069c),
+ REG(SYS_MMGT_FAST, 0x0006a0),
+ REG(SYS_EVENTS_DIF, 0x0006a4),
+ REG(SYS_EVENTS_CORE, 0x0006b4),
+ REG(SYS_CNT, 0x000000),
+ REG(SYS_PTP_STATUS, 0x0006b8),
+ REG(SYS_PTP_TXSTAMP, 0x0006bc),
+ REG(SYS_PTP_NXT, 0x0006c0),
+ REG(SYS_PTP_CFG, 0x0006c4),
+};
+EXPORT_SYMBOL(vsc7514_sys_regmap);
+
+const u32 vsc7514_vcap_regmap[] = {
+ /* VCAP_CORE_CFG */
+ REG(VCAP_CORE_UPDATE_CTRL, 0x000000),
+ REG(VCAP_CORE_MV_CFG, 0x000004),
+ /* VCAP_CORE_CACHE */
+ REG(VCAP_CACHE_ENTRY_DAT, 0x000008),
+ REG(VCAP_CACHE_MASK_DAT, 0x000108),
+ REG(VCAP_CACHE_ACTION_DAT, 0x000208),
+ REG(VCAP_CACHE_CNT_DAT, 0x000308),
+ REG(VCAP_CACHE_TG_DAT, 0x000388),
+ /* VCAP_CONST */
+ REG(VCAP_CONST_VCAP_VER, 0x000398),
+ REG(VCAP_CONST_ENTRY_WIDTH, 0x00039c),
+ REG(VCAP_CONST_ENTRY_CNT, 0x0003a0),
+ REG(VCAP_CONST_ENTRY_SWCNT, 0x0003a4),
+ REG(VCAP_CONST_ENTRY_TG_WIDTH, 0x0003a8),
+ REG(VCAP_CONST_ACTION_DEF_CNT, 0x0003ac),
+ REG(VCAP_CONST_ACTION_WIDTH, 0x0003b0),
+ REG(VCAP_CONST_CNT_WIDTH, 0x0003b4),
+ REG(VCAP_CONST_CORE_CNT, 0x0003b8),
+ REG(VCAP_CONST_IF_CNT, 0x0003bc),
+};
+EXPORT_SYMBOL(vsc7514_vcap_regmap);
+
+const u32 vsc7514_ptp_regmap[] = {
+ REG(PTP_PIN_CFG, 0x000000),
+ REG(PTP_PIN_TOD_SEC_MSB, 0x000004),
+ REG(PTP_PIN_TOD_SEC_LSB, 0x000008),
+ REG(PTP_PIN_TOD_NSEC, 0x00000c),
+ REG(PTP_PIN_WF_HIGH_PERIOD, 0x000014),
+ REG(PTP_PIN_WF_LOW_PERIOD, 0x000018),
+ REG(PTP_CFG_MISC, 0x0000a0),
+ REG(PTP_CLK_CFG_ADJ_CFG, 0x0000a4),
+ REG(PTP_CLK_CFG_ADJ_FREQ, 0x0000a8),
+};
+EXPORT_SYMBOL(vsc7514_ptp_regmap);
+
+const u32 vsc7514_dev_gmii_regmap[] = {
+ REG(DEV_CLOCK_CFG, 0x0),
+ REG(DEV_PORT_MISC, 0x4),
+ REG(DEV_EVENTS, 0x8),
+ REG(DEV_EEE_CFG, 0xc),
+ REG(DEV_RX_PATH_DELAY, 0x10),
+ REG(DEV_TX_PATH_DELAY, 0x14),
+ REG(DEV_PTP_PREDICT_CFG, 0x18),
+ REG(DEV_MAC_ENA_CFG, 0x1c),
+ REG(DEV_MAC_MODE_CFG, 0x20),
+ REG(DEV_MAC_MAXLEN_CFG, 0x24),
+ REG(DEV_MAC_TAGS_CFG, 0x28),
+ REG(DEV_MAC_ADV_CHK_CFG, 0x2c),
+ REG(DEV_MAC_IFG_CFG, 0x30),
+ REG(DEV_MAC_HDX_CFG, 0x34),
+ REG(DEV_MAC_DBG_CFG, 0x38),
+ REG(DEV_MAC_FC_MAC_LOW_CFG, 0x3c),
+ REG(DEV_MAC_FC_MAC_HIGH_CFG, 0x40),
+ REG(DEV_MAC_STICKY, 0x44),
+ REG(PCS1G_CFG, 0x48),
+ REG(PCS1G_MODE_CFG, 0x4c),
+ REG(PCS1G_SD_CFG, 0x50),
+ REG(PCS1G_ANEG_CFG, 0x54),
+ REG(PCS1G_ANEG_NP_CFG, 0x58),
+ REG(PCS1G_LB_CFG, 0x5c),
+ REG(PCS1G_DBG_CFG, 0x60),
+ REG(PCS1G_CDET_CFG, 0x64),
+ REG(PCS1G_ANEG_STATUS, 0x68),
+ REG(PCS1G_ANEG_NP_STATUS, 0x6c),
+ REG(PCS1G_LINK_STATUS, 0x70),
+ REG(PCS1G_LINK_DOWN_CNT, 0x74),
+ REG(PCS1G_STICKY, 0x78),
+ REG(PCS1G_DEBUG_STATUS, 0x7c),
+ REG(PCS1G_LPI_CFG, 0x80),
+ REG(PCS1G_LPI_WAKE_ERROR_CNT, 0x84),
+ REG(PCS1G_LPI_STATUS, 0x88),
+ REG(PCS1G_TSTPAT_MODE_CFG, 0x8c),
+ REG(PCS1G_TSTPAT_STATUS, 0x90),
+ REG(DEV_PCS_FX100_CFG, 0x94),
+ REG(DEV_PCS_FX100_STATUS, 0x98),
+};
+EXPORT_SYMBOL(vsc7514_dev_gmii_regmap);
+
+const struct vcap_field vsc7514_vcap_es0_keys[] = {
+ [VCAP_ES0_EGR_PORT] = { 0, 4 },
+ [VCAP_ES0_IGR_PORT] = { 4, 4 },
+ [VCAP_ES0_RSV] = { 8, 2 },
+ [VCAP_ES0_L2_MC] = { 10, 1 },
+ [VCAP_ES0_L2_BC] = { 11, 1 },
+ [VCAP_ES0_VID] = { 12, 12 },
+ [VCAP_ES0_DP] = { 24, 1 },
+ [VCAP_ES0_PCP] = { 25, 3 },
+};
+EXPORT_SYMBOL(vsc7514_vcap_es0_keys);
+
+const struct vcap_field vsc7514_vcap_es0_actions[] = {
+ [VCAP_ES0_ACT_PUSH_OUTER_TAG] = { 0, 2 },
+ [VCAP_ES0_ACT_PUSH_INNER_TAG] = { 2, 1 },
+ [VCAP_ES0_ACT_TAG_A_TPID_SEL] = { 3, 2 },
+ [VCAP_ES0_ACT_TAG_A_VID_SEL] = { 5, 1 },
+ [VCAP_ES0_ACT_TAG_A_PCP_SEL] = { 6, 2 },
+ [VCAP_ES0_ACT_TAG_A_DEI_SEL] = { 8, 2 },
+ [VCAP_ES0_ACT_TAG_B_TPID_SEL] = { 10, 2 },
+ [VCAP_ES0_ACT_TAG_B_VID_SEL] = { 12, 1 },
+ [VCAP_ES0_ACT_TAG_B_PCP_SEL] = { 13, 2 },
+ [VCAP_ES0_ACT_TAG_B_DEI_SEL] = { 15, 2 },
+ [VCAP_ES0_ACT_VID_A_VAL] = { 17, 12 },
+ [VCAP_ES0_ACT_PCP_A_VAL] = { 29, 3 },
+ [VCAP_ES0_ACT_DEI_A_VAL] = { 32, 1 },
+ [VCAP_ES0_ACT_VID_B_VAL] = { 33, 12 },
+ [VCAP_ES0_ACT_PCP_B_VAL] = { 45, 3 },
+ [VCAP_ES0_ACT_DEI_B_VAL] = { 48, 1 },
+ [VCAP_ES0_ACT_RSV] = { 49, 24 },
+ [VCAP_ES0_ACT_HIT_STICKY] = { 73, 1 },
+};
+EXPORT_SYMBOL(vsc7514_vcap_es0_actions);
+
+const struct vcap_field vsc7514_vcap_is1_keys[] = {
+ [VCAP_IS1_HK_TYPE] = { 0, 1 },
+ [VCAP_IS1_HK_LOOKUP] = { 1, 2 },
+ [VCAP_IS1_HK_IGR_PORT_MASK] = { 3, 12 },
+ [VCAP_IS1_HK_RSV] = { 15, 9 },
+ [VCAP_IS1_HK_OAM_Y1731] = { 24, 1 },
+ [VCAP_IS1_HK_L2_MC] = { 25, 1 },
+ [VCAP_IS1_HK_L2_BC] = { 26, 1 },
+ [VCAP_IS1_HK_IP_MC] = { 27, 1 },
+ [VCAP_IS1_HK_VLAN_TAGGED] = { 28, 1 },
+ [VCAP_IS1_HK_VLAN_DBL_TAGGED] = { 29, 1 },
+ [VCAP_IS1_HK_TPID] = { 30, 1 },
+ [VCAP_IS1_HK_VID] = { 31, 12 },
+ [VCAP_IS1_HK_DEI] = { 43, 1 },
+ [VCAP_IS1_HK_PCP] = { 44, 3 },
+ /* Specific Fields for IS1 Half Key S1_NORMAL */
+ [VCAP_IS1_HK_L2_SMAC] = { 47, 48 },
+ [VCAP_IS1_HK_ETYPE_LEN] = { 95, 1 },
+ [VCAP_IS1_HK_ETYPE] = { 96, 16 },
+ [VCAP_IS1_HK_IP_SNAP] = { 112, 1 },
+ [VCAP_IS1_HK_IP4] = { 113, 1 },
+ /* Layer-3 Information */
+ [VCAP_IS1_HK_L3_FRAGMENT] = { 114, 1 },
+ [VCAP_IS1_HK_L3_FRAG_OFS_GT0] = { 115, 1 },
+ [VCAP_IS1_HK_L3_OPTIONS] = { 116, 1 },
+ [VCAP_IS1_HK_L3_DSCP] = { 117, 6 },
+ [VCAP_IS1_HK_L3_IP4_SIP] = { 123, 32 },
+ /* Layer-4 Information */
+ [VCAP_IS1_HK_TCP_UDP] = { 155, 1 },
+ [VCAP_IS1_HK_TCP] = { 156, 1 },
+ [VCAP_IS1_HK_L4_SPORT] = { 157, 16 },
+ [VCAP_IS1_HK_L4_RNG] = { 173, 8 },
+ /* Specific Fields for IS1 Half Key S1_5TUPLE_IP4 */
+ [VCAP_IS1_HK_IP4_INNER_TPID] = { 47, 1 },
+ [VCAP_IS1_HK_IP4_INNER_VID] = { 48, 12 },
+ [VCAP_IS1_HK_IP4_INNER_DEI] = { 60, 1 },
+ [VCAP_IS1_HK_IP4_INNER_PCP] = { 61, 3 },
+ [VCAP_IS1_HK_IP4_IP4] = { 64, 1 },
+ [VCAP_IS1_HK_IP4_L3_FRAGMENT] = { 65, 1 },
+ [VCAP_IS1_HK_IP4_L3_FRAG_OFS_GT0] = { 66, 1 },
+ [VCAP_IS1_HK_IP4_L3_OPTIONS] = { 67, 1 },
+ [VCAP_IS1_HK_IP4_L3_DSCP] = { 68, 6 },
+ [VCAP_IS1_HK_IP4_L3_IP4_DIP] = { 74, 32 },
+ [VCAP_IS1_HK_IP4_L3_IP4_SIP] = { 106, 32 },
+ [VCAP_IS1_HK_IP4_L3_PROTO] = { 138, 8 },
+ [VCAP_IS1_HK_IP4_TCP_UDP] = { 146, 1 },
+ [VCAP_IS1_HK_IP4_TCP] = { 147, 1 },
+ [VCAP_IS1_HK_IP4_L4_RNG] = { 148, 8 },
+ [VCAP_IS1_HK_IP4_IP_PAYLOAD_S1_5TUPLE] = { 156, 32 },
+};
+EXPORT_SYMBOL(vsc7514_vcap_is1_keys);
+
+const struct vcap_field vsc7514_vcap_is1_actions[] = {
+ [VCAP_IS1_ACT_DSCP_ENA] = { 0, 1 },
+ [VCAP_IS1_ACT_DSCP_VAL] = { 1, 6 },
+ [VCAP_IS1_ACT_QOS_ENA] = { 7, 1 },
+ [VCAP_IS1_ACT_QOS_VAL] = { 8, 3 },
+ [VCAP_IS1_ACT_DP_ENA] = { 11, 1 },
+ [VCAP_IS1_ACT_DP_VAL] = { 12, 1 },
+ [VCAP_IS1_ACT_PAG_OVERRIDE_MASK] = { 13, 8 },
+ [VCAP_IS1_ACT_PAG_VAL] = { 21, 8 },
+ [VCAP_IS1_ACT_RSV] = { 29, 9 },
+ /* The fields below are incorrectly shifted by 2 in the manual */
+ [VCAP_IS1_ACT_VID_REPLACE_ENA] = { 38, 1 },
+ [VCAP_IS1_ACT_VID_ADD_VAL] = { 39, 12 },
+ [VCAP_IS1_ACT_FID_SEL] = { 51, 2 },
+ [VCAP_IS1_ACT_FID_VAL] = { 53, 13 },
+ [VCAP_IS1_ACT_PCP_DEI_ENA] = { 66, 1 },
+ [VCAP_IS1_ACT_PCP_VAL] = { 67, 3 },
+ [VCAP_IS1_ACT_DEI_VAL] = { 70, 1 },
+ [VCAP_IS1_ACT_VLAN_POP_CNT_ENA] = { 71, 1 },
+ [VCAP_IS1_ACT_VLAN_POP_CNT] = { 72, 2 },
+ [VCAP_IS1_ACT_CUSTOM_ACE_TYPE_ENA] = { 74, 4 },
+ [VCAP_IS1_ACT_HIT_STICKY] = { 78, 1 },
+};
+EXPORT_SYMBOL(vsc7514_vcap_is1_actions);
+
+const struct vcap_field vsc7514_vcap_is2_keys[] = {
+ /* Common: 46 bits */
+ [VCAP_IS2_TYPE] = { 0, 4 },
+ [VCAP_IS2_HK_FIRST] = { 4, 1 },
+ [VCAP_IS2_HK_PAG] = { 5, 8 },
+ [VCAP_IS2_HK_IGR_PORT_MASK] = { 13, 12 },
+ [VCAP_IS2_HK_RSV2] = { 25, 1 },
+ [VCAP_IS2_HK_HOST_MATCH] = { 26, 1 },
+ [VCAP_IS2_HK_L2_MC] = { 27, 1 },
+ [VCAP_IS2_HK_L2_BC] = { 28, 1 },
+ [VCAP_IS2_HK_VLAN_TAGGED] = { 29, 1 },
+ [VCAP_IS2_HK_VID] = { 30, 12 },
+ [VCAP_IS2_HK_DEI] = { 42, 1 },
+ [VCAP_IS2_HK_PCP] = { 43, 3 },
+ /* MAC_ETYPE / MAC_LLC / MAC_SNAP / OAM common */
+ [VCAP_IS2_HK_L2_DMAC] = { 46, 48 },
+ [VCAP_IS2_HK_L2_SMAC] = { 94, 48 },
+ /* MAC_ETYPE (TYPE=000) */
+ [VCAP_IS2_HK_MAC_ETYPE_ETYPE] = { 142, 16 },
+ [VCAP_IS2_HK_MAC_ETYPE_L2_PAYLOAD0] = { 158, 16 },
+ [VCAP_IS2_HK_MAC_ETYPE_L2_PAYLOAD1] = { 174, 8 },
+ [VCAP_IS2_HK_MAC_ETYPE_L2_PAYLOAD2] = { 182, 3 },
+ /* MAC_LLC (TYPE=001) */
+ [VCAP_IS2_HK_MAC_LLC_L2_LLC] = { 142, 40 },
+ /* MAC_SNAP (TYPE=010) */
+ [VCAP_IS2_HK_MAC_SNAP_L2_SNAP] = { 142, 40 },
+ /* MAC_ARP (TYPE=011) */
+ [VCAP_IS2_HK_MAC_ARP_SMAC] = { 46, 48 },
+ [VCAP_IS2_HK_MAC_ARP_ADDR_SPACE_OK] = { 94, 1 },
+ [VCAP_IS2_HK_MAC_ARP_PROTO_SPACE_OK] = { 95, 1 },
+ [VCAP_IS2_HK_MAC_ARP_LEN_OK] = { 96, 1 },
+ [VCAP_IS2_HK_MAC_ARP_TARGET_MATCH] = { 97, 1 },
+ [VCAP_IS2_HK_MAC_ARP_SENDER_MATCH] = { 98, 1 },
+ [VCAP_IS2_HK_MAC_ARP_OPCODE_UNKNOWN] = { 99, 1 },
+ [VCAP_IS2_HK_MAC_ARP_OPCODE] = { 100, 2 },
+ [VCAP_IS2_HK_MAC_ARP_L3_IP4_DIP] = { 102, 32 },
+ [VCAP_IS2_HK_MAC_ARP_L3_IP4_SIP] = { 134, 32 },
+ [VCAP_IS2_HK_MAC_ARP_DIP_EQ_SIP] = { 166, 1 },
+ /* IP4_TCP_UDP / IP4_OTHER common */
+ [VCAP_IS2_HK_IP4] = { 46, 1 },
+ [VCAP_IS2_HK_L3_FRAGMENT] = { 47, 1 },
+ [VCAP_IS2_HK_L3_FRAG_OFS_GT0] = { 48, 1 },
+ [VCAP_IS2_HK_L3_OPTIONS] = { 49, 1 },
+ [VCAP_IS2_HK_IP4_L3_TTL_GT0] = { 50, 1 },
+ [VCAP_IS2_HK_L3_TOS] = { 51, 8 },
+ [VCAP_IS2_HK_L3_IP4_DIP] = { 59, 32 },
+ [VCAP_IS2_HK_L3_IP4_SIP] = { 91, 32 },
+ [VCAP_IS2_HK_DIP_EQ_SIP] = { 123, 1 },
+ /* IP4_TCP_UDP (TYPE=100) */
+ [VCAP_IS2_HK_TCP] = { 124, 1 },
+ [VCAP_IS2_HK_L4_DPORT] = { 125, 16 },
+ [VCAP_IS2_HK_L4_SPORT] = { 141, 16 },
+ [VCAP_IS2_HK_L4_RNG] = { 157, 8 },
+ [VCAP_IS2_HK_L4_SPORT_EQ_DPORT] = { 165, 1 },
+ [VCAP_IS2_HK_L4_SEQUENCE_EQ0] = { 166, 1 },
+ [VCAP_IS2_HK_L4_FIN] = { 167, 1 },
+ [VCAP_IS2_HK_L4_SYN] = { 168, 1 },
+ [VCAP_IS2_HK_L4_RST] = { 169, 1 },
+ [VCAP_IS2_HK_L4_PSH] = { 170, 1 },
+ [VCAP_IS2_HK_L4_ACK] = { 171, 1 },
+ [VCAP_IS2_HK_L4_URG] = { 172, 1 },
+ [VCAP_IS2_HK_L4_1588_DOM] = { 173, 8 },
+ [VCAP_IS2_HK_L4_1588_VER] = { 181, 4 },
+ /* IP4_OTHER (TYPE=101) */
+ [VCAP_IS2_HK_IP4_L3_PROTO] = { 124, 8 },
+ [VCAP_IS2_HK_L3_PAYLOAD] = { 132, 56 },
+ /* IP6_STD (TYPE=110) */
+ [VCAP_IS2_HK_IP6_L3_TTL_GT0] = { 46, 1 },
+ [VCAP_IS2_HK_L3_IP6_SIP] = { 47, 128 },
+ [VCAP_IS2_HK_IP6_L3_PROTO] = { 175, 8 },
+ /* OAM (TYPE=111) */
+ [VCAP_IS2_HK_OAM_MEL_FLAGS] = { 142, 7 },
+ [VCAP_IS2_HK_OAM_VER] = { 149, 5 },
+ [VCAP_IS2_HK_OAM_OPCODE] = { 154, 8 },
+ [VCAP_IS2_HK_OAM_FLAGS] = { 162, 8 },
+ [VCAP_IS2_HK_OAM_MEPID] = { 170, 16 },
+ [VCAP_IS2_HK_OAM_CCM_CNTS_EQ0] = { 186, 1 },
+ [VCAP_IS2_HK_OAM_IS_Y1731] = { 187, 1 },
+};
+EXPORT_SYMBOL(vsc7514_vcap_is2_keys);
+
+const struct vcap_field vsc7514_vcap_is2_actions[] = {
+ [VCAP_IS2_ACT_HIT_ME_ONCE] = { 0, 1 },
+ [VCAP_IS2_ACT_CPU_COPY_ENA] = { 1, 1 },
+ [VCAP_IS2_ACT_CPU_QU_NUM] = { 2, 3 },
+ [VCAP_IS2_ACT_MASK_MODE] = { 5, 2 },
+ [VCAP_IS2_ACT_MIRROR_ENA] = { 7, 1 },
+ [VCAP_IS2_ACT_LRN_DIS] = { 8, 1 },
+ [VCAP_IS2_ACT_POLICE_ENA] = { 9, 1 },
+ [VCAP_IS2_ACT_POLICE_IDX] = { 10, 9 },
+ [VCAP_IS2_ACT_POLICE_VCAP_ONLY] = { 19, 1 },
+ [VCAP_IS2_ACT_PORT_MASK] = { 20, 11 },
+ [VCAP_IS2_ACT_REW_OP] = { 31, 9 },
+ [VCAP_IS2_ACT_SMAC_REPLACE_ENA] = { 40, 1 },
+ [VCAP_IS2_ACT_RSV] = { 41, 2 },
+ [VCAP_IS2_ACT_ACL_ID] = { 43, 6 },
+ [VCAP_IS2_ACT_HIT_CNT] = { 49, 32 },
+};
+EXPORT_SYMBOL(vsc7514_vcap_is2_actions);
diff --git a/drivers/net/ethernet/myricom/myri10ge/myri10ge.c b/drivers/net/ethernet/myricom/myri10ge/myri10ge.c
index 5736fcd..50ac3ee 100644
--- a/drivers/net/ethernet/myricom/myri10ge/myri10ge.c
+++ b/drivers/net/ethernet/myricom/myri10ge/myri10ge.c
@@ -1704,7 +1704,9 @@ myri10ge_set_pauseparam(struct net_device *netdev,
static void
myri10ge_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct myri10ge_priv *mgp = netdev_priv(netdev);
@@ -3740,7 +3742,6 @@ static int myri10ge_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
struct myri10ge_priv *mgp;
struct device *dev = &pdev->dev;
int status = -ENXIO;
- int dac_enabled;
unsigned hdr_offset, ss_offset;
static int board_number;
@@ -3780,15 +3781,8 @@ static int myri10ge_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
myri10ge_mask_surprise_down(pdev);
pci_set_master(pdev);
- dac_enabled = 1;
status = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64));
if (status != 0) {
- dac_enabled = 0;
- dev_err(&pdev->dev,
- "64-bit pci address mask was refused, trying 32-bit\n");
- status = dma_set_mask(&pdev->dev, DMA_BIT_MASK(32));
- }
- if (status != 0) {
dev_err(&pdev->dev, "Error %d setting DMA mask\n", status);
goto abort_with_enabled;
}
@@ -3872,10 +3866,7 @@ static int myri10ge_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
/* fake NETIF_F_HW_VLAN_CTAG_RX for good GRO performance */
netdev->hw_features |= NETIF_F_HW_VLAN_CTAG_RX;
- netdev->features = netdev->hw_features;
-
- if (dac_enabled)
- netdev->features |= NETIF_F_HIGHDMA;
+ netdev->features = netdev->hw_features | NETIF_F_HIGHDMA;
netdev->vlan_features |= mgp->features;
if (mgp->fw_ver_tiny < 37)
diff --git a/drivers/net/ethernet/natsemi/jazzsonic.c b/drivers/net/ethernet/natsemi/jazzsonic.c
index d74a80f..3f371fa 100644
--- a/drivers/net/ethernet/natsemi/jazzsonic.c
+++ b/drivers/net/ethernet/natsemi/jazzsonic.c
@@ -114,6 +114,7 @@ static int sonic_probe1(struct net_device *dev)
struct sonic_local *lp = netdev_priv(dev);
int err = -ENODEV;
int i;
+ unsigned char addr[ETH_ALEN];
if (!request_mem_region(dev->base_addr, SONIC_MEM_SIZE, jazz_sonic_string))
return -EBUSY;
@@ -143,9 +144,10 @@ static int sonic_probe1(struct net_device *dev)
SONIC_WRITE(SONIC_CEP,0);
for (i=0; i<3; i++) {
val = SONIC_READ(SONIC_CAP0-i);
- dev->dev_addr[i*2] = val;
- dev->dev_addr[i*2+1] = val >> 8;
+ addr[i*2] = val;
+ addr[i*2+1] = val >> 8;
}
+ eth_hw_addr_set(dev, addr);
lp->dma_bitmode = SONIC_BITMODE32;
diff --git a/drivers/net/ethernet/natsemi/macsonic.c b/drivers/net/ethernet/natsemi/macsonic.c
index 8709d70..b16f7c8 100644
--- a/drivers/net/ethernet/natsemi/macsonic.c
+++ b/drivers/net/ethernet/natsemi/macsonic.c
@@ -203,6 +203,7 @@ static void mac_onboard_sonic_ethernet_addr(struct net_device *dev)
struct sonic_local *lp = netdev_priv(dev);
const int prom_addr = ONBOARD_SONIC_PROM_BASE;
unsigned short val;
+ u8 addr[ETH_ALEN];
/*
* On NuBus boards we can sometimes look in the ROM resources.
@@ -213,7 +214,8 @@ static void mac_onboard_sonic_ethernet_addr(struct net_device *dev)
int i;
for (i = 0; i < 6; i++)
- dev->dev_addr[i] = SONIC_READ_PROM(i);
+ addr[i] = SONIC_READ_PROM(i);
+ eth_hw_addr_set(dev, addr);
if (!INVALID_MAC(dev->dev_addr))
return;
@@ -222,7 +224,8 @@ static void mac_onboard_sonic_ethernet_addr(struct net_device *dev)
* source has a rather long and detailed historical account of
* why this is so.
*/
- bit_reverse_addr(dev->dev_addr);
+ bit_reverse_addr(addr);
+ eth_hw_addr_set(dev, addr);
if (!INVALID_MAC(dev->dev_addr))
return;
@@ -243,14 +246,15 @@ static void mac_onboard_sonic_ethernet_addr(struct net_device *dev)
SONIC_WRITE(SONIC_CEP, 15);
val = SONIC_READ(SONIC_CAP2);
- dev->dev_addr[5] = val >> 8;
- dev->dev_addr[4] = val & 0xff;
+ addr[5] = val >> 8;
+ addr[4] = val & 0xff;
val = SONIC_READ(SONIC_CAP1);
- dev->dev_addr[3] = val >> 8;
- dev->dev_addr[2] = val & 0xff;
+ addr[3] = val >> 8;
+ addr[2] = val & 0xff;
val = SONIC_READ(SONIC_CAP0);
- dev->dev_addr[1] = val >> 8;
- dev->dev_addr[0] = val & 0xff;
+ addr[1] = val >> 8;
+ addr[0] = val & 0xff;
+ eth_hw_addr_set(dev, addr);
if (!INVALID_MAC(dev->dev_addr))
return;
@@ -355,13 +359,16 @@ static int mac_onboard_sonic_probe(struct net_device *dev)
static int mac_sonic_nubus_ethernet_addr(struct net_device *dev,
unsigned long prom_addr, int id)
{
+ u8 addr[ETH_ALEN];
int i;
+
for(i = 0; i < 6; i++)
- dev->dev_addr[i] = SONIC_READ_PROM(i);
+ addr[i] = SONIC_READ_PROM(i);
/* Some of the addresses are bit-reversed */
if (id != MACSONIC_DAYNA)
- bit_reverse_addr(dev->dev_addr);
+ bit_reverse_addr(addr);
+ eth_hw_addr_set(dev, addr);
return 0;
}
diff --git a/drivers/net/ethernet/natsemi/xtsonic.c b/drivers/net/ethernet/natsemi/xtsonic.c
index 0a02d8bd..52fef34 100644
--- a/drivers/net/ethernet/natsemi/xtsonic.c
+++ b/drivers/net/ethernet/natsemi/xtsonic.c
@@ -127,6 +127,7 @@ static int sonic_probe1(struct net_device *dev)
unsigned int base_addr = dev->base_addr;
int i;
int err = 0;
+ unsigned char addr[ETH_ALEN];
if (!request_mem_region(base_addr, 0x100, xtsonic_string))
return -EBUSY;
@@ -163,9 +164,10 @@ static int sonic_probe1(struct net_device *dev)
for (i=0; i<3; i++) {
unsigned int val = SONIC_READ(SONIC_CAP0-i);
- dev->dev_addr[i*2] = val;
- dev->dev_addr[i*2+1] = val >> 8;
+ addr[i*2] = val;
+ addr[i*2+1] = val >> 8;
}
+ eth_hw_addr_set(dev, addr);
lp->dma_bitmode = SONIC_BITMODE32;
diff --git a/drivers/net/ethernet/neterion/s2io.c b/drivers/net/ethernet/neterion/s2io.c
index d1c32c6..6dd451a 100644
--- a/drivers/net/ethernet/neterion/s2io.c
+++ b/drivers/net/ethernet/neterion/s2io.c
@@ -5461,8 +5461,11 @@ static int s2io_ethtool_set_led(struct net_device *dev,
return 0;
}
-static void s2io_ethtool_gringparam(struct net_device *dev,
- struct ethtool_ringparam *ering)
+static void
+s2io_ethtool_gringparam(struct net_device *dev,
+ struct ethtool_ringparam *ering,
+ struct kernel_ethtool_ringparam *kernel_ering,
+ struct netlink_ext_ack *extack)
{
struct s2io_nic *sp = netdev_priv(dev);
int i, tx_desc_count = 0, rx_desc_count = 0;
@@ -7655,7 +7658,6 @@ s2io_init_nic(struct pci_dev *pdev, const struct pci_device_id *pre)
struct s2io_nic *sp;
struct net_device *dev;
int i, j, ret;
- int dma_flag = false;
u32 mac_up, mac_down;
u64 val64 = 0, tmp64 = 0;
struct XENA_dev_config __iomem *bar0 = NULL;
@@ -7677,17 +7679,8 @@ s2io_init_nic(struct pci_dev *pdev, const struct pci_device_id *pre)
return ret;
}
- if (!dma_set_mask(&pdev->dev, DMA_BIT_MASK(64))) {
+ if (!dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64))) {
DBG_PRINT(INIT_DBG, "%s: Using 64bit DMA\n", __func__);
- dma_flag = true;
- if (dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(64))) {
- DBG_PRINT(ERR_DBG,
- "Unable to obtain 64bit DMA for coherent allocations\n");
- pci_disable_device(pdev);
- return -ENOMEM;
- }
- } else if (!dma_set_mask(&pdev->dev, DMA_BIT_MASK(32))) {
- DBG_PRINT(INIT_DBG, "%s: Using 32bit DMA\n", __func__);
} else {
pci_disable_device(pdev);
return -ENOMEM;
@@ -7717,7 +7710,6 @@ s2io_init_nic(struct pci_dev *pdev, const struct pci_device_id *pre)
sp = netdev_priv(dev);
sp->dev = dev;
sp->pdev = pdev;
- sp->high_dma_flag = dma_flag;
sp->device_enabled_once = false;
if (rx_ring_mode == 1)
sp->rxd_mode = RXD_MODE_1;
@@ -7865,9 +7857,8 @@ s2io_init_nic(struct pci_dev *pdev, const struct pci_device_id *pre)
NETIF_F_TSO | NETIF_F_TSO6 |
NETIF_F_RXCSUM | NETIF_F_LRO;
dev->features |= dev->hw_features |
- NETIF_F_HW_VLAN_CTAG_TX | NETIF_F_HW_VLAN_CTAG_RX;
- if (sp->high_dma_flag == true)
- dev->features |= NETIF_F_HIGHDMA;
+ NETIF_F_HW_VLAN_CTAG_TX | NETIF_F_HW_VLAN_CTAG_RX |
+ NETIF_F_HIGHDMA;
dev->watchdog_timeo = WATCH_DOG_TIMEOUT;
INIT_WORK(&sp->rst_timer_task, s2io_restart_nic);
INIT_WORK(&sp->set_link_task, s2io_set_link);
diff --git a/drivers/net/ethernet/neterion/s2io.h b/drivers/net/ethernet/neterion/s2io.h
index a4266d1..cb7080e 100644
--- a/drivers/net/ethernet/neterion/s2io.h
+++ b/drivers/net/ethernet/neterion/s2io.h
@@ -873,7 +873,6 @@ struct s2io_nic {
struct mac_addr def_mac_addr[256];
struct net_device_stats stats;
- int high_dma_flag;
int device_enabled_once;
char name[60];
diff --git a/drivers/net/ethernet/neterion/vxge/vxge-main.c b/drivers/net/ethernet/neterion/vxge/vxge-main.c
index 1969009..aa7c093 100644
--- a/drivers/net/ethernet/neterion/vxge/vxge-main.c
+++ b/drivers/net/ethernet/neterion/vxge/vxge-main.c
@@ -3159,10 +3159,6 @@ static int vxge_hwtstamp_set(struct vxgedev *vdev, void __user *data)
if (copy_from_user(&config, data, sizeof(config)))
return -EFAULT;
- /* reserved for future extensions */
- if (config.flags)
- return -EINVAL;
-
/* Transmit HW Timestamp not supported */
switch (config.tx_type) {
case HWTSTAMP_TX_OFF:
@@ -3353,7 +3349,7 @@ static const struct net_device_ops vxge_netdev_ops = {
};
static int vxge_device_register(struct __vxge_hw_device *hldev,
- struct vxge_config *config, int high_dma,
+ struct vxge_config *config,
int no_of_vpath, struct vxgedev **vdev_out)
{
struct net_device *ndev;
@@ -3425,11 +3421,7 @@ static int vxge_device_register(struct __vxge_hw_device *hldev,
vxge_debug_init(vxge_hw_device_trace_level_get(hldev),
"%s : checksumming enabled", __func__);
- if (high_dma) {
- ndev->features |= NETIF_F_HIGHDMA;
- vxge_debug_init(vxge_hw_device_trace_level_get(hldev),
- "%s : using High DMA", __func__);
- }
+ ndev->features |= NETIF_F_HIGHDMA;
/* MTU range: 68 - 9600 */
ndev->min_mtu = VXGE_HW_MIN_MTU;
@@ -4286,7 +4278,6 @@ vxge_probe(struct pci_dev *pdev, const struct pci_device_id *pre)
struct __vxge_hw_device *hldev;
enum vxge_hw_status status;
int ret;
- int high_dma = 0;
u64 vpath_mask = 0;
struct vxgedev *vdev;
struct vxge_config *ll_config = NULL;
@@ -4376,22 +4367,9 @@ vxge_probe(struct pci_dev *pdev, const struct pci_device_id *pre)
goto _exit0;
}
- if (!dma_set_mask(&pdev->dev, DMA_BIT_MASK(64))) {
+ if (!dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64))) {
vxge_debug_ll_config(VXGE_TRACE,
"%s : using 64bit DMA", __func__);
-
- high_dma = 1;
-
- if (dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(64))) {
- vxge_debug_init(VXGE_ERR,
- "%s : unable to obtain 64bit DMA for "
- "consistent allocations", __func__);
- ret = -ENOMEM;
- goto _exit1;
- }
- } else if (!dma_set_mask(&pdev->dev, DMA_BIT_MASK(32))) {
- vxge_debug_ll_config(VXGE_TRACE,
- "%s : using 32bit DMA", __func__);
} else {
ret = -ENOMEM;
goto _exit1;
@@ -4559,8 +4537,7 @@ vxge_probe(struct pci_dev *pdev, const struct pci_device_id *pre)
ll_config->tx_pause_enable = VXGE_PAUSE_CTRL_ENABLE;
ll_config->rx_pause_enable = VXGE_PAUSE_CTRL_ENABLE;
- ret = vxge_device_register(hldev, ll_config, high_dma, no_of_vpath,
- &vdev);
+ ret = vxge_device_register(hldev, ll_config, no_of_vpath, &vdev);
if (ret) {
ret = -EINVAL;
goto _exit4;
diff --git a/drivers/net/ethernet/netronome/nfp/flower/metadata.c b/drivers/net/ethernet/netronome/nfp/flower/metadata.c
index 2af9fae..f448c56 100644
--- a/drivers/net/ethernet/netronome/nfp/flower/metadata.c
+++ b/drivers/net/ethernet/netronome/nfp/flower/metadata.c
@@ -43,15 +43,14 @@ static int nfp_release_stats_entry(struct nfp_app *app, u32 stats_context_id)
struct circ_buf *ring;
ring = &priv->stats_ids.free_list;
- /* Check if buffer is full. */
- if (!CIRC_SPACE(ring->head, ring->tail,
- priv->stats_ring_size * NFP_FL_STATS_ELEM_RS -
- NFP_FL_STATS_ELEM_RS + 1))
+ /* Check if buffer is full, stats_ring_size must be power of 2 */
+ if (!CIRC_SPACE(ring->head, ring->tail, priv->stats_ring_size))
return -ENOBUFS;
- memcpy(&ring->buf[ring->head], &stats_context_id, NFP_FL_STATS_ELEM_RS);
- ring->head = (ring->head + NFP_FL_STATS_ELEM_RS) %
- (priv->stats_ring_size * NFP_FL_STATS_ELEM_RS);
+ /* Each increment of head represents size of NFP_FL_STATS_ELEM_RS */
+ memcpy(&ring->buf[ring->head * NFP_FL_STATS_ELEM_RS],
+ &stats_context_id, NFP_FL_STATS_ELEM_RS);
+ ring->head = (ring->head + 1) & (priv->stats_ring_size - 1);
return 0;
}
@@ -86,11 +85,14 @@ static int nfp_get_stats_entry(struct nfp_app *app, u32 *stats_context_id)
return -ENOENT;
}
- memcpy(&temp_stats_id, &ring->buf[ring->tail], NFP_FL_STATS_ELEM_RS);
+ /* Each increment of tail represents size of NFP_FL_STATS_ELEM_RS */
+ memcpy(&temp_stats_id, &ring->buf[ring->tail * NFP_FL_STATS_ELEM_RS],
+ NFP_FL_STATS_ELEM_RS);
*stats_context_id = temp_stats_id;
- memcpy(&ring->buf[ring->tail], &freed_stats_id, NFP_FL_STATS_ELEM_RS);
- ring->tail = (ring->tail + NFP_FL_STATS_ELEM_RS) %
- (priv->stats_ring_size * NFP_FL_STATS_ELEM_RS);
+ memcpy(&ring->buf[ring->tail * NFP_FL_STATS_ELEM_RS], &freed_stats_id,
+ NFP_FL_STATS_ELEM_RS);
+ /* stats_ring_size must be power of 2 */
+ ring->tail = (ring->tail + 1) & (priv->stats_ring_size - 1);
return 0;
}
@@ -138,13 +140,18 @@ static int nfp_release_mask_id(struct nfp_app *app, u8 mask_id)
struct circ_buf *ring;
ring = &priv->mask_ids.mask_id_free_list;
- /* Checking if buffer is full. */
+ /* Checking if buffer is full,
+ * NFP_FLOWER_MASK_ENTRY_RS must be power of 2
+ */
if (CIRC_SPACE(ring->head, ring->tail, NFP_FLOWER_MASK_ENTRY_RS) == 0)
return -ENOBUFS;
- memcpy(&ring->buf[ring->head], &mask_id, NFP_FLOWER_MASK_ELEMENT_RS);
- ring->head = (ring->head + NFP_FLOWER_MASK_ELEMENT_RS) %
- (NFP_FLOWER_MASK_ENTRY_RS * NFP_FLOWER_MASK_ELEMENT_RS);
+ /* Each increment of head represents size of
+ * NFP_FLOWER_MASK_ELEMENT_RS
+ */
+ memcpy(&ring->buf[ring->head * NFP_FLOWER_MASK_ELEMENT_RS], &mask_id,
+ NFP_FLOWER_MASK_ELEMENT_RS);
+ ring->head = (ring->head + 1) & (NFP_FLOWER_MASK_ENTRY_RS - 1);
priv->mask_ids.last_used[mask_id] = ktime_get();
@@ -171,7 +178,11 @@ static int nfp_mask_alloc(struct nfp_app *app, u8 *mask_id)
if (ring->head == ring->tail)
goto err_not_found;
- memcpy(&temp_id, &ring->buf[ring->tail], NFP_FLOWER_MASK_ELEMENT_RS);
+ /* Each increment of tail represents size of
+ * NFP_FLOWER_MASK_ELEMENT_RS
+ */
+ memcpy(&temp_id, &ring->buf[ring->tail * NFP_FLOWER_MASK_ELEMENT_RS],
+ NFP_FLOWER_MASK_ELEMENT_RS);
*mask_id = temp_id;
reuse_timeout = ktime_add_ns(priv->mask_ids.last_used[*mask_id],
@@ -180,9 +191,10 @@ static int nfp_mask_alloc(struct nfp_app *app, u8 *mask_id)
if (ktime_before(ktime_get(), reuse_timeout))
goto err_not_found;
- memcpy(&ring->buf[ring->tail], &freed_id, NFP_FLOWER_MASK_ELEMENT_RS);
- ring->tail = (ring->tail + NFP_FLOWER_MASK_ELEMENT_RS) %
- (NFP_FLOWER_MASK_ENTRY_RS * NFP_FLOWER_MASK_ELEMENT_RS);
+ memcpy(&ring->buf[ring->tail * NFP_FLOWER_MASK_ELEMENT_RS], &freed_id,
+ NFP_FLOWER_MASK_ELEMENT_RS);
+ /* NFP_FLOWER_MASK_ENTRY_RS must be power of 2 */
+ ring->tail = (ring->tail + 1) & (NFP_FLOWER_MASK_ENTRY_RS - 1);
return 0;
@@ -338,11 +350,6 @@ int nfp_compile_flow_metadata(struct nfp_app *app, u32 cookie,
nfp_flow->meta.mask_len,
&nfp_flow->meta.flags, &new_mask_id)) {
NL_SET_ERR_MSG_MOD(extack, "invalid entry: cannot allocate a new mask id");
- if (nfp_release_stats_entry(app, stats_cxt)) {
- NL_SET_ERR_MSG_MOD(extack, "invalid entry: cannot release stats context");
- err = -EINVAL;
- goto err_remove_rhash;
- }
err = -ENOENT;
goto err_remove_rhash;
}
@@ -359,21 +366,6 @@ int nfp_compile_flow_metadata(struct nfp_app *app, u32 cookie,
check_entry = nfp_flower_search_fl_table(app, cookie, netdev);
if (check_entry) {
NL_SET_ERR_MSG_MOD(extack, "invalid entry: cannot offload duplicate flow entry");
- if (nfp_release_stats_entry(app, stats_cxt)) {
- NL_SET_ERR_MSG_MOD(extack, "invalid entry: cannot release stats context");
- err = -EINVAL;
- goto err_remove_mask;
- }
-
- if (!nfp_flow->pre_tun_rule.dev &&
- !nfp_check_mask_remove(app, nfp_flow->mask_data,
- nfp_flow->meta.mask_len,
- NULL, &new_mask_id)) {
- NL_SET_ERR_MSG_MOD(extack, "invalid entry: cannot release mask id");
- err = -EINVAL;
- goto err_remove_mask;
- }
-
err = -EEXIST;
goto err_remove_mask;
}
diff --git a/drivers/net/ethernet/netronome/nfp/flower/offload.c b/drivers/net/ethernet/netronome/nfp/flower/offload.c
index 224089d..f97eff5 100644
--- a/drivers/net/ethernet/netronome/nfp/flower/offload.c
+++ b/drivers/net/ethernet/netronome/nfp/flower/offload.c
@@ -1867,6 +1867,9 @@ nfp_flower_indr_setup_tc_cb(struct net_device *netdev, struct Qdisc *sch, void *
void *data,
void (*cleanup)(struct flow_block_cb *block_cb))
{
+ if (!netdev)
+ return -EOPNOTSUPP;
+
if (!nfp_fl_is_netdev_to_offload(netdev))
return -EOPNOTSUPP;
diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c
index 850bfdf..79257ec 100644
--- a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c
+++ b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c
@@ -1944,7 +1944,7 @@ static int nfp_net_rx(struct nfp_net_rx_ring *rx_ring, int budget)
xdp_prog, act);
continue;
default:
- bpf_warn_invalid_xdp_action(act);
+ bpf_warn_invalid_xdp_action(dp->netdev, xdp_prog, act);
fallthrough;
case XDP_ABORTED:
trace_xdp_exception(dp->netdev, xdp_prog, act);
@@ -4097,7 +4097,7 @@ static void nfp_net_netdev_init(struct nfp_net *nn)
netdev->min_mtu = ETH_MIN_MTU;
netdev->max_mtu = nn->max_mtu;
- netdev->gso_max_segs = NFP_NET_LSO_MAX_SEGS;
+ netif_set_gso_max_segs(netdev, NFP_NET_LSO_MAX_SEGS);
netif_carrier_off(netdev);
diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net_ethtool.c b/drivers/net/ethernet/netronome/nfp/nfp_net_ethtool.c
index cf78829..e0c27471 100644
--- a/drivers/net/ethernet/netronome/nfp/nfp_net_ethtool.c
+++ b/drivers/net/ethernet/netronome/nfp/nfp_net_ethtool.c
@@ -381,7 +381,9 @@ nfp_net_set_link_ksettings(struct net_device *netdev,
}
static void nfp_net_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct nfp_net *nn = netdev_priv(netdev);
@@ -406,7 +408,9 @@ static int nfp_net_set_ring_size(struct nfp_net *nn, u32 rxd_cnt, u32 txd_cnt)
}
static int nfp_net_set_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct nfp_net *nn = netdev_priv(netdev);
u32 rxd_cnt, txd_cnt;
diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net_repr.c b/drivers/net/ethernet/netronome/nfp/nfp_net_repr.c
index 369f6ae7..181ac8e 100644
--- a/drivers/net/ethernet/netronome/nfp/nfp_net_repr.c
+++ b/drivers/net/ethernet/netronome/nfp/nfp_net_repr.c
@@ -286,8 +286,8 @@ nfp_repr_transfer_features(struct net_device *netdev, struct net_device *lower)
if (repr->dst->u.port_info.lower_dev != lower)
return;
- netdev->gso_max_size = lower->gso_max_size;
- netdev->gso_max_segs = lower->gso_max_segs;
+ netif_set_gso_max_size(netdev, lower->gso_max_size);
+ netif_set_gso_max_segs(netdev, lower->gso_max_segs);
netdev_update_features(netdev);
}
@@ -381,7 +381,7 @@ int nfp_repr_init(struct nfp_app *app, struct net_device *netdev,
/* Advertise but disable TSO by default. */
netdev->features &= ~(NETIF_F_TSO | NETIF_F_TSO6);
- netdev->gso_max_segs = NFP_NET_LSO_MAX_SEGS;
+ netif_set_gso_max_segs(netdev, NFP_NET_LSO_MAX_SEGS);
netdev->priv_flags |= IFF_NO_QUEUE | IFF_DISABLE_NETPOLL;
netdev->features |= NETIF_F_LLTX;
diff --git a/drivers/net/ethernet/nvidia/forcedeth.c b/drivers/net/ethernet/nvidia/forcedeth.c
index 9b530d7..660013f 100644
--- a/drivers/net/ethernet/nvidia/forcedeth.c
+++ b/drivers/net/ethernet/nvidia/forcedeth.c
@@ -4651,7 +4651,10 @@ static int nv_nway_reset(struct net_device *dev)
return ret;
}
-static void nv_get_ringparam(struct net_device *dev, struct ethtool_ringparam* ring)
+static void nv_get_ringparam(struct net_device *dev,
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct fe_priv *np = netdev_priv(dev);
@@ -4662,7 +4665,10 @@ static void nv_get_ringparam(struct net_device *dev, struct ethtool_ringparam* r
ring->tx_pending = np->tx_ring_size;
}
-static int nv_set_ringparam(struct net_device *dev, struct ethtool_ringparam* ring)
+static int nv_set_ringparam(struct net_device *dev,
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct fe_priv *np = netdev_priv(dev);
u8 __iomem *base = get_hwbase(dev);
diff --git a/drivers/net/ethernet/oki-semi/pch_gbe/pch_gbe_ethtool.c b/drivers/net/ethernet/oki-semi/pch_gbe/pch_gbe_ethtool.c
index 660b07c..84cc79e 100644
--- a/drivers/net/ethernet/oki-semi/pch_gbe/pch_gbe_ethtool.c
+++ b/drivers/net/ethernet/oki-semi/pch_gbe/pch_gbe_ethtool.c
@@ -270,9 +270,13 @@ static int pch_gbe_nway_reset(struct net_device *netdev)
* pch_gbe_get_ringparam - Report ring sizes
* @netdev: Network interface device structure
* @ring: Ring param structure
+ * @kernel_ring: Ring external param structure
+ * @extack: netlink handle
*/
static void pch_gbe_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct pch_gbe_adapter *adapter = netdev_priv(netdev);
struct pch_gbe_tx_ring *txdr = adapter->tx_ring;
@@ -288,12 +292,16 @@ static void pch_gbe_get_ringparam(struct net_device *netdev,
* pch_gbe_set_ringparam - Set ring sizes
* @netdev: Network interface device structure
* @ring: Ring param structure
+ * @kernel_ring: Ring external param structure
+ * @extack: netlink handle
* Returns
* 0: Successful.
* Negative value: Failed.
*/
static int pch_gbe_set_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct pch_gbe_adapter *adapter = netdev_priv(netdev);
struct pch_gbe_tx_ring *txdr, *tx_old;
diff --git a/drivers/net/ethernet/oki-semi/pch_gbe/pch_gbe_main.c b/drivers/net/ethernet/oki-semi/pch_gbe/pch_gbe_main.c
index 71d2342..1dc40c5 100644
--- a/drivers/net/ethernet/oki-semi/pch_gbe/pch_gbe_main.c
+++ b/drivers/net/ethernet/oki-semi/pch_gbe/pch_gbe_main.c
@@ -210,9 +210,6 @@ static int hwtstamp_ioctl(struct net_device *netdev, struct ifreq *ifr, int cmd)
if (copy_from_user(&cfg, ifr->ifr_data, sizeof(cfg)))
return -EFAULT;
- if (cfg.flags) /* reserved for future extensions */
- return -EINVAL;
-
/* Get ieee1588's dev information */
pdev = adapter->ptp_pdev;
diff --git a/drivers/net/ethernet/pasemi/pasemi_mac_ethtool.c b/drivers/net/ethernet/pasemi/pasemi_mac_ethtool.c
index e1a3048..4c7e0c9 100644
--- a/drivers/net/ethernet/pasemi/pasemi_mac_ethtool.c
+++ b/drivers/net/ethernet/pasemi/pasemi_mac_ethtool.c
@@ -69,7 +69,9 @@ pasemi_mac_ethtool_set_msglevel(struct net_device *netdev,
static void
pasemi_mac_ethtool_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ering)
+ struct ethtool_ringparam *ering,
+ struct kernel_ethtool_ringparam *kernel_ering,
+ struct netlink_ext_ack *extack)
{
struct pasemi_mac *mac = netdev_priv(netdev);
diff --git a/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c b/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c
index c54d735..386a5cf 100644
--- a/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c
+++ b/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c
@@ -512,7 +512,9 @@ static int ionic_set_coalesce(struct net_device *netdev,
}
static void ionic_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct ionic_lif *lif = netdev_priv(netdev);
@@ -523,7 +525,9 @@ static void ionic_get_ringparam(struct net_device *netdev,
}
static int ionic_set_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct ionic_lif *lif = netdev_priv(netdev);
struct ionic_queue_params qparam;
diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.c b/drivers/net/ethernet/pensando/ionic/ionic_lif.c
index 63f8a81..2ff7be1 100644
--- a/drivers/net/ethernet/pensando/ionic/ionic_lif.c
+++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.c
@@ -3135,7 +3135,7 @@ int ionic_lif_init(struct ionic_lif *lif)
return -EINVAL;
}
- lif->dbid_inuse = bitmap_alloc(lif->dbid_count, GFP_KERNEL);
+ lif->dbid_inuse = bitmap_zalloc(lif->dbid_count, GFP_KERNEL);
if (!lif->dbid_inuse) {
dev_err(dev, "Failed alloc doorbell id bitmap, aborting\n");
return -ENOMEM;
diff --git a/drivers/net/ethernet/qlogic/netxen/netxen_nic_ethtool.c b/drivers/net/ethernet/qlogic/netxen/netxen_nic_ethtool.c
index a075643..3c4a84e 100644
--- a/drivers/net/ethernet/qlogic/netxen/netxen_nic_ethtool.c
+++ b/drivers/net/ethernet/qlogic/netxen/netxen_nic_ethtool.c
@@ -392,7 +392,9 @@ netxen_nic_get_eeprom(struct net_device *dev, struct ethtool_eeprom *eeprom,
static void
netxen_nic_get_ringparam(struct net_device *dev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct netxen_adapter *adapter = netdev_priv(dev);
@@ -430,7 +432,9 @@ netxen_validate_ringparam(u32 val, u32 min, u32 max, char *r_name)
static int
netxen_nic_set_ringparam(struct net_device *dev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct netxen_adapter *adapter = netdev_priv(dev);
u16 max_rcv_desc = MAX_RCV_DESCRIPTORS_10G;
diff --git a/drivers/net/ethernet/qlogic/qed/qed_cxt.c b/drivers/net/ethernet/qlogic/qed/qed_cxt.c
index 452494f..65e2069 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_cxt.c
+++ b/drivers/net/ethernet/qlogic/qed/qed_cxt.c
@@ -1036,12 +1036,12 @@ static void qed_cid_map_free(struct qed_hwfn *p_hwfn)
u32 type, vf;
for (type = 0; type < MAX_CONN_TYPES; type++) {
- kfree(p_mngr->acquired[type].cid_map);
+ bitmap_free(p_mngr->acquired[type].cid_map);
p_mngr->acquired[type].max_count = 0;
p_mngr->acquired[type].start_cid = 0;
for (vf = 0; vf < MAX_NUM_VFS; vf++) {
- kfree(p_mngr->acquired_vf[type][vf].cid_map);
+ bitmap_free(p_mngr->acquired_vf[type][vf].cid_map);
p_mngr->acquired_vf[type][vf].max_count = 0;
p_mngr->acquired_vf[type][vf].start_cid = 0;
}
@@ -1054,15 +1054,10 @@ qed_cid_map_alloc_single(struct qed_hwfn *p_hwfn,
u32 cid_start,
u32 cid_count, struct qed_cid_acquired_map *p_map)
{
- u32 size;
-
if (!cid_count)
return 0;
- size = DIV_ROUND_UP(cid_count,
- sizeof(unsigned long) * BITS_PER_BYTE) *
- sizeof(unsigned long);
- p_map->cid_map = kzalloc(size, GFP_KERNEL);
+ p_map->cid_map = bitmap_zalloc(cid_count, GFP_KERNEL);
if (!p_map->cid_map)
return -ENOMEM;
@@ -1216,7 +1211,6 @@ void qed_cxt_mngr_setup(struct qed_hwfn *p_hwfn)
struct qed_cid_acquired_map *p_map;
struct qed_conn_type_cfg *p_cfg;
int type;
- u32 len;
/* Reset acquired cids */
for (type = 0; type < MAX_CONN_TYPES; type++) {
@@ -1225,11 +1219,7 @@ void qed_cxt_mngr_setup(struct qed_hwfn *p_hwfn)
p_cfg = &p_mngr->conn_cfg[type];
if (p_cfg->cid_count) {
p_map = &p_mngr->acquired[type];
- len = DIV_ROUND_UP(p_map->max_count,
- sizeof(unsigned long) *
- BITS_PER_BYTE) *
- sizeof(unsigned long);
- memset(p_map->cid_map, 0, len);
+ bitmap_zero(p_map->cid_map, p_map->max_count);
}
if (!p_cfg->cids_per_vf)
@@ -1237,11 +1227,7 @@ void qed_cxt_mngr_setup(struct qed_hwfn *p_hwfn)
for (vf = 0; vf < MAX_NUM_VFS; vf++) {
p_map = &p_mngr->acquired_vf[type][vf];
- len = DIV_ROUND_UP(p_map->max_count,
- sizeof(unsigned long) *
- BITS_PER_BYTE) *
- sizeof(unsigned long);
- memset(p_map->cid_map, 0, len);
+ bitmap_zero(p_map->cid_map, p_map->max_count);
}
}
}
diff --git a/drivers/net/ethernet/qlogic/qed/qed_hsi.h b/drivers/net/ethernet/qlogic/qed/qed_hsi.h
index f2cedbd..ed1a845 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_hsi.h
+++ b/drivers/net/ethernet/qlogic/qed/qed_hsi.h
@@ -2721,6 +2721,25 @@ void qed_memset_task_ctx(void *p_ctx_mem, u32 ctx_size, u8 ctx_type);
#define NUM_STORMS 6
/**
+ * qed_get_protocol_type_str(): Get a string for Protocol type.
+ *
+ * @protocol_type: Protocol type (using enum protocol_type).
+ *
+ * Return: String.
+ */
+const char *qed_get_protocol_type_str(u32 protocol_type);
+
+/**
+ * qed_get_ramrod_cmd_id_str(): Get a string for Ramrod command ID.
+ *
+ * @protocol_type: Protocol type (using enum protocol_type).
+ * @ramrod_cmd_id: Ramrod command ID (using per-protocol enum <protocol>_ramrod_cmd_id).
+ *
+ * Return: String.
+ */
+const char *qed_get_ramrod_cmd_id_str(u32 protocol_type, u32 ramrod_cmd_id);
+
+/**
* qed_set_rdma_error_level(): Sets the RDMA assert level.
* If the severity of the error will be
* above the level, the FW will assert.
diff --git a/drivers/net/ethernet/qlogic/qed/qed_init_fw_funcs.c b/drivers/net/ethernet/qlogic/qed/qed_init_fw_funcs.c
index 321c434..0ce37f2 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_init_fw_funcs.c
+++ b/drivers/net/ethernet/qlogic/qed/qed_init_fw_funcs.c
@@ -210,6 +210,82 @@ static u16 task_region_offsets[1][NUM_OF_CONNECTION_TYPES] = {
(XSEM_REG_FAST_MEMORY + SEM_FAST_REG_INT_RAM + \
XSTORM_PQ_INFO_OFFSET(pq_id))
+static const char * const s_protocol_types[] = {
+ "PROTOCOLID_ISCSI", "PROTOCOLID_FCOE", "PROTOCOLID_ROCE",
+ "PROTOCOLID_CORE", "PROTOCOLID_ETH", "PROTOCOLID_IWARP",
+ "PROTOCOLID_TOE", "PROTOCOLID_PREROCE", "PROTOCOLID_COMMON",
+ "PROTOCOLID_TCP", "PROTOCOLID_RDMA", "PROTOCOLID_SCSI",
+};
+
+static const char *s_ramrod_cmd_ids[][28] = {
+ {
+ "ISCSI_RAMROD_CMD_ID_UNUSED", "ISCSI_RAMROD_CMD_ID_INIT_FUNC",
+ "ISCSI_RAMROD_CMD_ID_DESTROY_FUNC",
+ "ISCSI_RAMROD_CMD_ID_OFFLOAD_CONN",
+ "ISCSI_RAMROD_CMD_ID_UPDATE_CONN",
+ "ISCSI_RAMROD_CMD_ID_TERMINATION_CONN",
+ "ISCSI_RAMROD_CMD_ID_CLEAR_SQ", "ISCSI_RAMROD_CMD_ID_MAC_UPDATE",
+ "ISCSI_RAMROD_CMD_ID_CONN_STATS", },
+ { "FCOE_RAMROD_CMD_ID_INIT_FUNC", "FCOE_RAMROD_CMD_ID_DESTROY_FUNC",
+ "FCOE_RAMROD_CMD_ID_STAT_FUNC",
+ "FCOE_RAMROD_CMD_ID_OFFLOAD_CONN",
+ "FCOE_RAMROD_CMD_ID_TERMINATE_CONN", },
+ { "RDMA_RAMROD_UNUSED", "RDMA_RAMROD_FUNC_INIT",
+ "RDMA_RAMROD_FUNC_CLOSE", "RDMA_RAMROD_REGISTER_MR",
+ "RDMA_RAMROD_DEREGISTER_MR", "RDMA_RAMROD_CREATE_CQ",
+ "RDMA_RAMROD_RESIZE_CQ", "RDMA_RAMROD_DESTROY_CQ",
+ "RDMA_RAMROD_CREATE_SRQ", "RDMA_RAMROD_MODIFY_SRQ",
+ "RDMA_RAMROD_DESTROY_SRQ", "RDMA_RAMROD_START_NS_TRACKING",
+ "RDMA_RAMROD_STOP_NS_TRACKING", "ROCE_RAMROD_CREATE_QP",
+ "ROCE_RAMROD_MODIFY_QP", "ROCE_RAMROD_QUERY_QP",
+ "ROCE_RAMROD_DESTROY_QP", "ROCE_RAMROD_CREATE_UD_QP",
+ "ROCE_RAMROD_DESTROY_UD_QP", "ROCE_RAMROD_FUNC_UPDATE",
+ "ROCE_RAMROD_SUSPEND_QP", "ROCE_RAMROD_QUERY_SUSPENDED_QP",
+ "ROCE_RAMROD_CREATE_SUSPENDED_QP", "ROCE_RAMROD_RESUME_QP",
+ "ROCE_RAMROD_SUSPEND_UD_QP", "ROCE_RAMROD_RESUME_UD_QP",
+ "ROCE_RAMROD_CREATE_SUSPENDED_UD_QP", "ROCE_RAMROD_FLUSH_DPT_QP", },
+ { "CORE_RAMROD_UNUSED", "CORE_RAMROD_RX_QUEUE_START",
+ "CORE_RAMROD_TX_QUEUE_START", "CORE_RAMROD_RX_QUEUE_STOP",
+ "CORE_RAMROD_TX_QUEUE_STOP",
+ "CORE_RAMROD_RX_QUEUE_FLUSH",
+ "CORE_RAMROD_TX_QUEUE_UPDATE", "CORE_RAMROD_QUEUE_STATS_QUERY", },
+ { "ETH_RAMROD_UNUSED", "ETH_RAMROD_VPORT_START",
+ "ETH_RAMROD_VPORT_UPDATE", "ETH_RAMROD_VPORT_STOP",
+ "ETH_RAMROD_RX_QUEUE_START", "ETH_RAMROD_RX_QUEUE_STOP",
+ "ETH_RAMROD_TX_QUEUE_START", "ETH_RAMROD_TX_QUEUE_STOP",
+ "ETH_RAMROD_FILTERS_UPDATE", "ETH_RAMROD_RX_QUEUE_UPDATE",
+ "ETH_RAMROD_RX_CREATE_OPENFLOW_ACTION",
+ "ETH_RAMROD_RX_ADD_OPENFLOW_FILTER",
+ "ETH_RAMROD_RX_DELETE_OPENFLOW_FILTER",
+ "ETH_RAMROD_RX_ADD_UDP_FILTER",
+ "ETH_RAMROD_RX_DELETE_UDP_FILTER",
+ "ETH_RAMROD_RX_CREATE_GFT_ACTION",
+ "ETH_RAMROD_RX_UPDATE_GFT_FILTER", "ETH_RAMROD_TX_QUEUE_UPDATE",
+ "ETH_RAMROD_RGFS_FILTER_ADD", "ETH_RAMROD_RGFS_FILTER_DEL",
+ "ETH_RAMROD_TGFS_FILTER_ADD", "ETH_RAMROD_TGFS_FILTER_DEL",
+ "ETH_RAMROD_GFS_COUNTERS_REPORT_REQUEST", },
+ { "RDMA_RAMROD_UNUSED", "RDMA_RAMROD_FUNC_INIT",
+ "RDMA_RAMROD_FUNC_CLOSE", "RDMA_RAMROD_REGISTER_MR",
+ "RDMA_RAMROD_DEREGISTER_MR", "RDMA_RAMROD_CREATE_CQ",
+ "RDMA_RAMROD_RESIZE_CQ", "RDMA_RAMROD_DESTROY_CQ",
+ "RDMA_RAMROD_CREATE_SRQ", "RDMA_RAMROD_MODIFY_SRQ",
+ "RDMA_RAMROD_DESTROY_SRQ", "RDMA_RAMROD_START_NS_TRACKING",
+ "RDMA_RAMROD_STOP_NS_TRACKING",
+ "IWARP_RAMROD_CMD_ID_TCP_OFFLOAD",
+ "IWARP_RAMROD_CMD_ID_MPA_OFFLOAD",
+ "IWARP_RAMROD_CMD_ID_MPA_OFFLOAD_SEND_RTR",
+ "IWARP_RAMROD_CMD_ID_CREATE_QP", "IWARP_RAMROD_CMD_ID_QUERY_QP",
+ "IWARP_RAMROD_CMD_ID_MODIFY_QP",
+ "IWARP_RAMROD_CMD_ID_DESTROY_QP",
+ "IWARP_RAMROD_CMD_ID_ABORT_TCP_OFFLOAD", },
+ { NULL }, /*TOE*/
+ { NULL }, /*PREROCE*/
+ { "COMMON_RAMROD_UNUSED", "COMMON_RAMROD_PF_START",
+ "COMMON_RAMROD_PF_STOP", "COMMON_RAMROD_VF_START",
+ "COMMON_RAMROD_VF_STOP", "COMMON_RAMROD_PF_UPDATE",
+ "COMMON_RAMROD_RL_UPDATE", "COMMON_RAMROD_EMPTY", }
+};
+
/******************** INTERNAL IMPLEMENTATION *********************/
/* Returns the external VOQ number */
@@ -1647,6 +1723,32 @@ void qed_enable_context_validation(struct qed_hwfn *p_hwfn,
qed_wr(p_hwfn, p_ptt, CDU_REG_TCFC_CTX_VALID0, ctx_validation);
}
+const char *qed_get_protocol_type_str(u32 protocol_type)
+{
+ if (protocol_type >= ARRAY_SIZE(s_protocol_types))
+ return "Invalid protocol type";
+
+ return s_protocol_types[protocol_type];
+}
+
+const char *qed_get_ramrod_cmd_id_str(u32 protocol_type, u32 ramrod_cmd_id)
+{
+ const char *ramrod_cmd_id_str;
+
+ if (protocol_type >= ARRAY_SIZE(s_ramrod_cmd_ids))
+ return "Invalid protocol type";
+
+ if (ramrod_cmd_id >= ARRAY_SIZE(s_ramrod_cmd_ids[0]))
+ return "Invalid Ramrod command ID";
+
+ ramrod_cmd_id_str = s_ramrod_cmd_ids[protocol_type][ramrod_cmd_id];
+
+ if (!ramrod_cmd_id_str)
+ return "Invalid Ramrod command ID";
+
+ return ramrod_cmd_id_str;
+}
+
static u32 qed_get_rdma_assert_ram_addr(struct qed_hwfn *p_hwfn, u8 storm_id)
{
switch (storm_id) {
diff --git a/drivers/net/ethernet/qlogic/qed/qed_int.c b/drivers/net/ethernet/qlogic/qed/qed_int.c
index 6958ade..82e74f6 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_int.c
+++ b/drivers/net/ethernet/qlogic/qed/qed_int.c
@@ -2399,3 +2399,25 @@ int qed_int_set_timer_res(struct qed_hwfn *p_hwfn, struct qed_ptt *p_ptt,
return rc;
}
+
+int qed_int_get_sb_dbg(struct qed_hwfn *p_hwfn, struct qed_ptt *p_ptt,
+ struct qed_sb_info *p_sb, struct qed_sb_info_dbg *p_info)
+{
+ u16 sbid = p_sb->igu_sb_id;
+ u32 i;
+
+ if (IS_VF(p_hwfn->cdev))
+ return -EINVAL;
+
+ if (sbid >= NUM_OF_SBS(p_hwfn->cdev))
+ return -EINVAL;
+
+ p_info->igu_prod = qed_rd(p_hwfn, p_ptt, IGU_REG_PRODUCER_MEMORY + sbid * 4);
+ p_info->igu_cons = qed_rd(p_hwfn, p_ptt, IGU_REG_CONSUMER_MEM + sbid * 4);
+
+ for (i = 0; i < PIS_PER_SB; i++)
+ p_info->pi[i] = (u16)qed_rd(p_hwfn, p_ptt,
+ CAU_REG_PI_MEMORY + sbid * 4 * PIS_PER_SB + i * 4);
+
+ return 0;
+}
diff --git a/drivers/net/ethernet/qlogic/qed/qed_int.h b/drivers/net/ethernet/qlogic/qed/qed_int.h
index 84c17e9..7e5127f 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_int.h
+++ b/drivers/net/ethernet/qlogic/qed/qed_int.h
@@ -186,6 +186,19 @@ void qed_int_disable_post_isr_release(struct qed_dev *cdev);
void qed_int_attn_clr_enable(struct qed_dev *cdev, bool clr_enable);
/**
+ * qed_int_get_sb_dbg: Read debug information regarding a given SB
+ *
+ * @p_hwfn: hw function pointer
+ * @p_ptt: ptt resource
+ * @p_sb: pointer to status block for which we want to get info
+ * @p_info: pointer to struct to fill with information regarding SB
+ *
+ * Return: 0 with status block info filled on success, otherwise return error
+ */
+int qed_int_get_sb_dbg(struct qed_hwfn *p_hwfn, struct qed_ptt *p_ptt,
+ struct qed_sb_info *p_sb, struct qed_sb_info_dbg *p_info);
+
+/**
* qed_db_rec_handler(): Doorbell Recovery handler.
* Run doorbell recovery in case of PF overflow (and flush DORQ if
* needed).
diff --git a/drivers/net/ethernet/qlogic/qed/qed_main.c b/drivers/net/ethernet/qlogic/qed/qed_main.c
index 7673b3e..c5003fa 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_main.c
+++ b/drivers/net/ethernet/qlogic/qed/qed_main.c
@@ -255,27 +255,6 @@ static void __exit qed_exit(void)
}
module_exit(qed_exit);
-/* Check if the DMA controller on the machine can properly handle the DMA
- * addressing required by the device.
- */
-static int qed_set_coherency_mask(struct qed_dev *cdev)
-{
- struct device *dev = &cdev->pdev->dev;
-
- if (dma_set_mask(dev, DMA_BIT_MASK(64)) == 0) {
- if (dma_set_coherent_mask(dev, DMA_BIT_MASK(64)) != 0) {
- DP_NOTICE(cdev,
- "Can't request 64-bit consistent allocations\n");
- return -EIO;
- }
- } else if (dma_set_mask(dev, DMA_BIT_MASK(32)) != 0) {
- DP_NOTICE(cdev, "Can't request 64b/32b DMA addresses\n");
- return -EIO;
- }
-
- return 0;
-}
-
static void qed_free_pci(struct qed_dev *cdev)
{
struct pci_dev *pdev = cdev->pdev;
@@ -351,9 +330,12 @@ static int qed_init_pci(struct qed_dev *cdev, struct pci_dev *pdev)
if (IS_PF(cdev) && !cdev->pci_params.pm_cap)
DP_NOTICE(cdev, "Cannot find power management capability\n");
- rc = qed_set_coherency_mask(cdev);
- if (rc)
+ rc = dma_set_mask_and_coherent(&cdev->pdev->dev, DMA_BIT_MASK(64));
+ if (rc) {
+ DP_NOTICE(cdev, "Can't request DMA addresses\n");
+ rc = -EIO;
goto err2;
+ }
cdev->pci_params.mem_start = pci_resource_start(pdev, 0);
cdev->pci_params.mem_end = pci_resource_end(pdev, 0);
@@ -447,7 +429,7 @@ int qed_fill_dev_info(struct qed_dev *cdev,
dev_info->wol_support = true;
dev_info->smart_an = qed_mcp_is_smart_an_supported(p_hwfn);
-
+ dev_info->esl = qed_mcp_is_esl_supported(p_hwfn);
dev_info->abs_pf_id = QED_LEADING_HWFN(cdev)->abs_pf_id;
} else {
qed_vf_get_fw_version(&cdev->hwfns[0], &dev_info->fw_major,
@@ -2936,6 +2918,30 @@ static int qed_update_mtu(struct qed_dev *cdev, u16 mtu)
return status;
}
+static int
+qed_get_sb_info(struct qed_dev *cdev, struct qed_sb_info *sb,
+ u16 qid, struct qed_sb_info_dbg *sb_dbg)
+{
+ struct qed_hwfn *hwfn = &cdev->hwfns[qid % cdev->num_hwfns];
+ struct qed_ptt *ptt;
+ int rc;
+
+ if (IS_VF(cdev))
+ return -EINVAL;
+
+ ptt = qed_ptt_acquire(hwfn);
+ if (!ptt) {
+ DP_NOTICE(hwfn, "Can't acquire PTT\n");
+ return -EAGAIN;
+ }
+
+ memset(sb_dbg, 0, sizeof(*sb_dbg));
+ rc = qed_int_get_sb_dbg(hwfn, ptt, sb, sb_dbg);
+
+ qed_ptt_release(hwfn, ptt);
+ return rc;
+}
+
static int qed_read_module_eeprom(struct qed_dev *cdev, char *buf,
u8 dev_addr, u32 offset, u32 len)
{
@@ -2978,11 +2984,54 @@ static int qed_set_grc_config(struct qed_dev *cdev, u32 cfg_id, u32 val)
return rc;
}
+static __printf(2, 3) void qed_mfw_report(struct qed_dev *cdev, char *fmt, ...)
+{
+ char buf[QED_MFW_REPORT_STR_SIZE];
+ struct qed_hwfn *p_hwfn;
+ struct qed_ptt *p_ptt;
+ va_list vl;
+
+ va_start(vl, fmt);
+ vsnprintf(buf, QED_MFW_REPORT_STR_SIZE, fmt, vl);
+ va_end(vl);
+
+ if (IS_PF(cdev)) {
+ p_hwfn = QED_LEADING_HWFN(cdev);
+ p_ptt = qed_ptt_acquire(p_hwfn);
+ if (p_ptt) {
+ qed_mcp_send_raw_debug_data(p_hwfn, p_ptt, buf, strlen(buf));
+ qed_ptt_release(p_hwfn, p_ptt);
+ }
+ }
+}
+
static u8 qed_get_affin_hwfn_idx(struct qed_dev *cdev)
{
return QED_AFFIN_HWFN_IDX(cdev);
}
+static int qed_get_esl_status(struct qed_dev *cdev, bool *esl_active)
+{
+ struct qed_hwfn *hwfn = QED_LEADING_HWFN(cdev);
+ struct qed_ptt *ptt;
+ int rc = 0;
+
+ *esl_active = false;
+
+ if (IS_VF(cdev))
+ return 0;
+
+ ptt = qed_ptt_acquire(hwfn);
+ if (!ptt)
+ return -EAGAIN;
+
+ rc = qed_mcp_get_esl_status(hwfn, ptt, esl_active);
+
+ qed_ptt_release(hwfn, ptt);
+
+ return rc;
+}
+
static struct qed_selftest_ops qed_selftest_ops_pass = {
.selftest_memory = &qed_selftest_memory,
.selftest_interrupt = &qed_selftest_interrupt,
@@ -3038,6 +3087,9 @@ const struct qed_common_ops qed_common_ops_pass = {
.read_nvm_cfg = &qed_nvm_flash_cfg_read,
.read_nvm_cfg_len = &qed_nvm_flash_cfg_len,
.set_grc_config = &qed_set_grc_config,
+ .mfw_report = &qed_mfw_report,
+ .get_sb_info = &qed_get_sb_info,
+ .get_esl_status = &qed_get_esl_status,
};
void qed_get_protocol_stats(struct qed_dev *cdev,
diff --git a/drivers/net/ethernet/qlogic/qed/qed_mcp.c b/drivers/net/ethernet/qlogic/qed/qed_mcp.c
index 64678a2..da1eada 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_mcp.c
+++ b/drivers/net/ethernet/qlogic/qed/qed_mcp.c
@@ -4158,3 +4158,25 @@ qed_mcp_send_raw_debug_data(struct qed_hwfn *p_hwfn,
return qed_mcp_send_debug_data(p_hwfn, p_ptt,
QED_MCP_DBG_DATA_TYPE_RAW, p_buf, size);
}
+
+bool qed_mcp_is_esl_supported(struct qed_hwfn *p_hwfn)
+{
+ return !!(p_hwfn->mcp_info->capabilities &
+ FW_MB_PARAM_FEATURE_SUPPORT_ENHANCED_SYS_LCK);
+}
+
+int qed_mcp_get_esl_status(struct qed_hwfn *p_hwfn, struct qed_ptt *p_ptt, bool *active)
+{
+ u32 resp = 0, param = 0;
+ int rc;
+
+ rc = qed_mcp_cmd(p_hwfn, p_ptt, DRV_MSG_CODE_GET_MANAGEMENT_STATUS, 0, &resp, ¶m);
+ if (rc) {
+ DP_NOTICE(p_hwfn, "Failed to send ESL command, rc = %d\n", rc);
+ return rc;
+ }
+
+ *active = !!(param & FW_MB_PARAM_MANAGEMENT_STATUS_LOCKDOWN_ENABLED);
+
+ return 0;
+}
diff --git a/drivers/net/ethernet/qlogic/qed/qed_mcp.h b/drivers/net/ethernet/qlogic/qed/qed_mcp.h
index 5647238..369e189 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_mcp.h
+++ b/drivers/net/ethernet/qlogic/qed/qed_mcp.h
@@ -15,6 +15,8 @@
#include "qed_hsi.h"
#include "qed_dev_api.h"
+#define QED_MFW_REPORT_STR_SIZE 256
+
struct qed_mcp_link_speed_params {
bool autoneg;
@@ -1337,4 +1339,24 @@ int qed_mcp_nvm_get_cfg(struct qed_hwfn *p_hwfn, struct qed_ptt *p_ptt,
int qed_mcp_nvm_set_cfg(struct qed_hwfn *p_hwfn, struct qed_ptt *p_ptt,
u16 option_id, u8 entity_id, u16 flags, u8 *p_buf,
u32 len);
+
+/**
+ * qed_mcp_is_esl_supported(): Return whether management firmware support ESL or not.
+ *
+ * @p_hwfn: hw function pointer
+ *
+ * Return: true if esl is supported, otherwise return false
+ */
+bool qed_mcp_is_esl_supported(struct qed_hwfn *p_hwfn);
+
+/**
+ * qed_mcp_get_esl_status(): Get enhanced system lockdown status
+ *
+ * @p_hwfn: hw function pointer
+ * @p_ptt: ptt resource pointer
+ * @active: ESL active status data pointer
+ *
+ * Return: 0 with esl status info on success, otherwise return error
+ */
+int qed_mcp_get_esl_status(struct qed_hwfn *p_hwfn, struct qed_ptt *p_ptt, bool *active);
#endif
diff --git a/drivers/net/ethernet/qlogic/qed/qed_mfw_hsi.h b/drivers/net/ethernet/qlogic/qed/qed_mfw_hsi.h
index 8a0e3c5..b70ee82 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_mfw_hsi.h
+++ b/drivers/net/ethernet/qlogic/qed/qed_mfw_hsi.h
@@ -1191,6 +1191,7 @@ enum drv_msg_code_enum {
DRV_MSG_CODE_CFG_VF_MSIX = DRV_MSG_CODE(0xc001),
DRV_MSG_CODE_CFG_PF_VFS_MSIX = DRV_MSG_CODE(0xc002),
DRV_MSG_CODE_DEBUG_DATA_SEND = DRV_MSG_CODE(0xc004),
+ DRV_MSG_CODE_GET_MANAGEMENT_STATUS = DRV_MSG_CODE(0xc007),
};
#define DRV_MSG_CODE_VMAC_TYPE_SHIFT 4
diff --git a/drivers/net/ethernet/qlogic/qed/qed_reg_addr.h b/drivers/net/ethernet/qlogic/qed/qed_reg_addr.h
index 6f1a52e..b5e35f4 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_reg_addr.h
+++ b/drivers/net/ethernet/qlogic/qed/qed_reg_addr.h
@@ -550,6 +550,8 @@
0x1 << 1)
#define IGU_REG_BLOCK_CONFIGURATION_PXP_TPH_INTERFACE_EN ( \
0x1 << 0)
+#define IGU_REG_PRODUCER_MEMORY 0x182000UL
+#define IGU_REG_CONSUMER_MEM 0x183000UL
#define IGU_REG_MAPPING_MEMORY \
0x184000UL
#define IGU_REG_STATISTIC_NUM_VF_MSG_SENT \
diff --git a/drivers/net/ethernet/qlogic/qed/qed_sp_commands.c b/drivers/net/ethernet/qlogic/qed/qed_sp_commands.c
index 648176d..3b54da9 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_sp_commands.c
+++ b/drivers/net/ethernet/qlogic/qed/qed_sp_commands.c
@@ -85,10 +85,12 @@ int qed_sp_init_request(struct qed_hwfn *p_hwfn,
goto err;
}
- DP_VERBOSE(p_hwfn, QED_MSG_SPQ,
- "Initialized: CID %08x cmd %02x protocol %02x data_addr %lu comp_mode [%s]\n",
- opaque_cid, cmd, protocol,
- (unsigned long)&p_ent->ramrod,
+ DP_VERBOSE(p_hwfn,
+ QED_MSG_SPQ,
+ "Initialized: CID %08x %s:[%02x] %s:%02x data_addr %llx comp_mode [%s]\n",
+ opaque_cid, qed_get_ramrod_cmd_id_str(protocol, cmd),
+ cmd, qed_get_protocol_type_str(protocol), protocol,
+ (unsigned long long)(uintptr_t)&p_ent->ramrod,
D_TRINE(p_ent->comp_mode, QED_SPQ_MODE_EBLOCK,
QED_SPQ_MODE_BLOCK, "MODE_EBLOCK", "MODE_BLOCK",
"MODE_CB"));
diff --git a/drivers/net/ethernet/qlogic/qed/qed_spq.c b/drivers/net/ethernet/qlogic/qed/qed_spq.c
index e047372..d01b924 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_spq.c
+++ b/drivers/net/ethernet/qlogic/qed/qed_spq.c
@@ -139,10 +139,13 @@ static int qed_spq_block(struct qed_hwfn *p_hwfn,
if (!p_ptt)
return -EBUSY;
qed_hw_err_notify(p_hwfn, p_ptt, QED_HW_ERR_RAMROD_FAIL,
- "Ramrod is stuck [CID %08x cmd %02x protocol %02x echo %04x]\n",
+ "Ramrod is stuck [CID %08x %s:%02x %s:%02x echo %04x]\n",
le32_to_cpu(p_ent->elem.hdr.cid),
+ qed_get_ramrod_cmd_id_str(p_ent->elem.hdr.protocol_id,
+ p_ent->elem.hdr.cmd_id),
p_ent->elem.hdr.cmd_id,
- p_ent->elem.hdr.protocol_id,
+ qed_get_protocol_type_str(p_ent->elem.hdr.protocol_id),
+ p_ent->elem.hdr.protocol_id,
le16_to_cpu(p_ent->elem.hdr.echo));
qed_ptt_release(p_hwfn, p_ptt);
@@ -170,13 +173,16 @@ static int qed_spq_fill_entry(struct qed_hwfn *p_hwfn,
return -EINVAL;
}
- DP_VERBOSE(p_hwfn, QED_MSG_SPQ,
- "Ramrod header: [CID 0x%08x CMD 0x%02x protocol 0x%02x] Data pointer: [%08x:%08x] Completion Mode: %s\n",
+ DP_VERBOSE(p_hwfn,
+ QED_MSG_SPQ,
+ "Ramrod hdr: [CID 0x%08x %s:0x%02x %s:0x%02x] Data ptr: [%08x:%08x] Cmpltion Mode: %s\n",
p_ent->elem.hdr.cid,
+ qed_get_ramrod_cmd_id_str(p_ent->elem.hdr.protocol_id,
+ p_ent->elem.hdr.cmd_id),
p_ent->elem.hdr.cmd_id,
- p_ent->elem.hdr.protocol_id,
- p_ent->elem.data_ptr.hi,
- p_ent->elem.data_ptr.lo,
+ qed_get_protocol_type_str(p_ent->elem.hdr.protocol_id),
+ p_ent->elem.hdr.protocol_id,
+ p_ent->elem.data_ptr.hi, p_ent->elem.data_ptr.lo,
D_TRINE(p_ent->comp_mode, QED_SPQ_MODE_EBLOCK,
QED_SPQ_MODE_BLOCK, "MODE_EBLOCK", "MODE_BLOCK",
"MODE_CB"));
@@ -271,17 +277,27 @@ qed_async_event_completion(struct qed_hwfn *p_hwfn,
{
qed_spq_async_comp_cb cb;
- if (!p_hwfn->p_spq || (p_eqe->protocol_id >= MAX_PROTOCOL_TYPE))
+ if (!p_hwfn->p_spq)
return -EINVAL;
+ if (p_eqe->protocol_id >= MAX_PROTOCOL_TYPE) {
+ DP_ERR(p_hwfn, "Wrong protocol: %s:%d\n",
+ qed_get_protocol_type_str(p_eqe->protocol_id),
+ p_eqe->protocol_id);
+
+ return -EINVAL;
+ }
+
cb = p_hwfn->p_spq->async_comp_cb[p_eqe->protocol_id];
if (cb) {
return cb(p_hwfn, p_eqe->opcode, p_eqe->echo,
&p_eqe->data, p_eqe->fw_return_code);
} else {
DP_NOTICE(p_hwfn,
- "Unknown Async completion for protocol: %d\n",
+ "Unknown Async completion for %s:%d\n",
+ qed_get_protocol_type_str(p_eqe->protocol_id),
p_eqe->protocol_id);
+
return -EINVAL;
}
}
@@ -830,8 +846,12 @@ int qed_spq_post(struct qed_hwfn *p_hwfn,
if (p_hwfn->cdev->recov_in_prog) {
DP_VERBOSE(p_hwfn,
QED_MSG_SPQ,
- "Recovery is in progress. Skip spq post [cmd %02x protocol %02x]\n",
- p_ent->elem.hdr.cmd_id, p_ent->elem.hdr.protocol_id);
+ "Recovery is in progress. Skip spq post [%s:%02x %s:%02x]\n",
+ qed_get_ramrod_cmd_id_str(p_ent->elem.hdr.protocol_id,
+ p_ent->elem.hdr.cmd_id),
+ p_ent->elem.hdr.cmd_id,
+ qed_get_protocol_type_str(p_ent->elem.hdr.protocol_id),
+ p_ent->elem.hdr.protocol_id);
/* Let the flow complete w/o any error handling */
qed_spq_recov_set_ret_code(p_ent, fw_return_code);
diff --git a/drivers/net/ethernet/qlogic/qede/qede_ethtool.c b/drivers/net/ethernet/qlogic/qede/qede_ethtool.c
index 8284c4c..97a7ab0 100644
--- a/drivers/net/ethernet/qlogic/qede/qede_ethtool.c
+++ b/drivers/net/ethernet/qlogic/qede/qede_ethtool.c
@@ -168,6 +168,8 @@ enum {
QEDE_PRI_FLAG_CMT,
QEDE_PRI_FLAG_SMART_AN_SUPPORT, /* MFW supports SmartAN */
QEDE_PRI_FLAG_RECOVER_ON_ERROR,
+ QEDE_PRI_FLAG_ESL_SUPPORT, /* MFW supports Enhanced System Lockdown */
+ QEDE_PRI_FLAG_ESL_ACTIVE, /* Enhanced System Lockdown Active status */
QEDE_PRI_FLAG_LEN,
};
@@ -175,6 +177,8 @@ static const char qede_private_arr[QEDE_PRI_FLAG_LEN][ETH_GSTRING_LEN] = {
"Coupled-Function",
"SmartAN capable",
"Recover on error",
+ "ESL capable",
+ "ESL active",
};
enum qede_ethtool_tests {
@@ -478,6 +482,7 @@ static int qede_get_sset_count(struct net_device *dev, int stringset)
static u32 qede_get_priv_flags(struct net_device *dev)
{
struct qede_dev *edev = netdev_priv(dev);
+ bool esl_active;
u32 flags = 0;
if (edev->dev_info.common.num_hwfns > 1)
@@ -489,6 +494,14 @@ static u32 qede_get_priv_flags(struct net_device *dev)
if (edev->err_flags & BIT(QEDE_ERR_IS_RECOVERABLE))
flags |= BIT(QEDE_PRI_FLAG_RECOVER_ON_ERROR);
+ if (edev->dev_info.common.esl)
+ flags |= BIT(QEDE_PRI_FLAG_ESL_SUPPORT);
+
+ edev->ops->common->get_esl_status(edev->cdev, &esl_active);
+
+ if (esl_active)
+ flags |= BIT(QEDE_PRI_FLAG_ESL_ACTIVE);
+
return flags;
}
@@ -888,7 +901,9 @@ int qede_set_coalesce(struct net_device *dev, struct ethtool_coalesce *coal,
}
static void qede_get_ringparam(struct net_device *dev,
- struct ethtool_ringparam *ering)
+ struct ethtool_ringparam *ering,
+ struct kernel_ethtool_ringparam *kernel_ering,
+ struct netlink_ext_ack *extack)
{
struct qede_dev *edev = netdev_priv(dev);
@@ -899,7 +914,9 @@ static void qede_get_ringparam(struct net_device *dev,
}
static int qede_set_ringparam(struct net_device *dev,
- struct ethtool_ringparam *ering)
+ struct ethtool_ringparam *ering,
+ struct kernel_ethtool_ringparam *kernel_ering,
+ struct netlink_ext_ack *extack)
{
struct qede_dev *edev = netdev_priv(dev);
diff --git a/drivers/net/ethernet/qlogic/qede/qede_fp.c b/drivers/net/ethernet/qlogic/qede/qede_fp.c
index 999abcf..b242000 100644
--- a/drivers/net/ethernet/qlogic/qede/qede_fp.c
+++ b/drivers/net/ethernet/qlogic/qede/qede_fp.c
@@ -10,6 +10,7 @@
#include <linux/bpf_trace.h>
#include <net/udp_tunnel.h>
#include <linux/ip.h>
+#include <net/gro.h>
#include <net/ipv6.h>
#include <net/tcp.h>
#include <linux/if_ether.h>
@@ -1152,7 +1153,7 @@ static bool qede_rx_xdp(struct qede_dev *edev,
qede_rx_bd_ring_consume(rxq);
break;
default:
- bpf_warn_invalid_xdp_action(act);
+ bpf_warn_invalid_xdp_action(edev->ndev, prog, act);
fallthrough;
case XDP_ABORTED:
trace_xdp_exception(edev->ndev, prog, act);
diff --git a/drivers/net/ethernet/qlogic/qede/qede_main.c b/drivers/net/ethernet/qlogic/qede/qede_main.c
index 06c6a58..b4e5a15 100644
--- a/drivers/net/ethernet/qlogic/qede/qede_main.c
+++ b/drivers/net/ethernet/qlogic/qede/qede_main.c
@@ -509,34 +509,95 @@ static int qede_ioctl(struct net_device *dev, struct ifreq *ifr, int cmd)
return 0;
}
-static void qede_tx_log_print(struct qede_dev *edev, struct qede_tx_queue *txq)
+static void qede_fp_sb_dump(struct qede_dev *edev, struct qede_fastpath *fp)
{
+ char *p_sb = (char *)fp->sb_info->sb_virt;
+ u32 sb_size, i;
+
+ sb_size = sizeof(struct status_block);
+
+ for (i = 0; i < sb_size; i += 8)
+ DP_NOTICE(edev,
+ "%02hhX %02hhX %02hhX %02hhX %02hhX %02hhX %02hhX %02hhX\n",
+ p_sb[i], p_sb[i + 1], p_sb[i + 2], p_sb[i + 3],
+ p_sb[i + 4], p_sb[i + 5], p_sb[i + 6], p_sb[i + 7]);
+}
+
+static void
+qede_txq_fp_log_metadata(struct qede_dev *edev,
+ struct qede_fastpath *fp, struct qede_tx_queue *txq)
+{
+ struct qed_chain *p_chain = &txq->tx_pbl;
+
+ /* Dump txq/fp/sb ids etc. other metadata */
DP_NOTICE(edev,
- "Txq[%d]: FW cons [host] %04x, SW cons %04x, SW prod %04x [Jiffies %lu]\n",
- txq->index, le16_to_cpu(*txq->hw_cons_ptr),
- qed_chain_get_cons_idx(&txq->tx_pbl),
- qed_chain_get_prod_idx(&txq->tx_pbl),
- jiffies);
+ "fpid 0x%x sbid 0x%x txqid [0x%x] ndev_qid [0x%x] cos [0x%x] p_chain %p cap %d size %d jiffies %lu HZ 0x%x\n",
+ fp->id, fp->sb_info->igu_sb_id, txq->index, txq->ndev_txq_id, txq->cos,
+ p_chain, p_chain->capacity, p_chain->size, jiffies, HZ);
+
+ /* Dump all the relevant prod/cons indexes */
+ DP_NOTICE(edev,
+ "hw cons %04x sw_tx_prod=0x%x, sw_tx_cons=0x%x, bd_prod 0x%x bd_cons 0x%x\n",
+ le16_to_cpu(*txq->hw_cons_ptr), txq->sw_tx_prod, txq->sw_tx_cons,
+ qed_chain_get_prod_idx(p_chain), qed_chain_get_cons_idx(p_chain));
+}
+
+static void
+qede_tx_log_print(struct qede_dev *edev, struct qede_fastpath *fp, struct qede_tx_queue *txq)
+{
+ struct qed_sb_info_dbg sb_dbg;
+ int rc;
+
+ /* sb info */
+ qede_fp_sb_dump(edev, fp);
+
+ memset(&sb_dbg, 0, sizeof(sb_dbg));
+ rc = edev->ops->common->get_sb_info(edev->cdev, fp->sb_info, (u16)fp->id, &sb_dbg);
+
+ DP_NOTICE(edev, "IGU: prod %08x cons %08x CAU Tx %04x\n",
+ sb_dbg.igu_prod, sb_dbg.igu_cons, sb_dbg.pi[TX_PI(txq->cos)]);
+
+ /* report to mfw */
+ edev->ops->common->mfw_report(edev->cdev,
+ "Txq[%d]: FW cons [host] %04x, SW cons %04x, SW prod %04x [Jiffies %lu]\n",
+ txq->index, le16_to_cpu(*txq->hw_cons_ptr),
+ qed_chain_get_cons_idx(&txq->tx_pbl),
+ qed_chain_get_prod_idx(&txq->tx_pbl), jiffies);
+ if (!rc)
+ edev->ops->common->mfw_report(edev->cdev,
+ "Txq[%d]: SB[0x%04x] - IGU: prod %08x cons %08x CAU Tx %04x\n",
+ txq->index, fp->sb_info->igu_sb_id,
+ sb_dbg.igu_prod, sb_dbg.igu_cons,
+ sb_dbg.pi[TX_PI(txq->cos)]);
}
static void qede_tx_timeout(struct net_device *dev, unsigned int txqueue)
{
struct qede_dev *edev = netdev_priv(dev);
- struct qede_tx_queue *txq;
- int cos;
+ int i;
netif_carrier_off(dev);
DP_NOTICE(edev, "TX timeout on queue %u!\n", txqueue);
- if (!(edev->fp_array[txqueue].type & QEDE_FASTPATH_TX))
- return;
+ for_each_queue(i) {
+ struct qede_tx_queue *txq;
+ struct qede_fastpath *fp;
+ int cos;
- for_each_cos_in_txq(edev, cos) {
- txq = &edev->fp_array[txqueue].txq[cos];
+ fp = &edev->fp_array[i];
+ if (!(fp->type & QEDE_FASTPATH_TX))
+ continue;
- if (qed_chain_get_cons_idx(&txq->tx_pbl) !=
- qed_chain_get_prod_idx(&txq->tx_pbl))
- qede_tx_log_print(edev, txq);
+ for_each_cos_in_txq(edev, cos) {
+ txq = &fp->txq[cos];
+
+ /* Dump basic metadata for all queues */
+ qede_txq_fp_log_metadata(edev, fp, txq);
+
+ if (qed_chain_get_cons_idx(&txq->tx_pbl) !=
+ qed_chain_get_prod_idx(&txq->tx_pbl))
+ qede_tx_log_print(edev, fp, txq);
+ }
}
if (IS_VF(edev))
diff --git a/drivers/net/ethernet/qlogic/qede/qede_ptp.c b/drivers/net/ethernet/qlogic/qede/qede_ptp.c
index 8c28fab..39176e7 100644
--- a/drivers/net/ethernet/qlogic/qede/qede_ptp.c
+++ b/drivers/net/ethernet/qlogic/qede/qede_ptp.c
@@ -304,11 +304,6 @@ int qede_ptp_hw_ts(struct qede_dev *edev, struct ifreq *ifr)
"HWTSTAMP IOCTL: Requested tx_type = %d, requested rx_filters = %d\n",
config.tx_type, config.rx_filter);
- if (config.flags) {
- DP_ERR(edev, "config.flags is reserved for future use\n");
- return -EINVAL;
- }
-
ptp->hw_ts_ioctl_called = 1;
ptp->tx_type = config.tx_type;
ptp->rx_filter = config.rx_filter;
diff --git a/drivers/net/ethernet/qlogic/qla3xxx.c b/drivers/net/ethernet/qlogic/qla3xxx.c
index 71523d7..b30589a 100644
--- a/drivers/net/ethernet/qlogic/qla3xxx.c
+++ b/drivers/net/ethernet/qlogic/qla3xxx.c
@@ -3750,7 +3750,7 @@ static int ql3xxx_probe(struct pci_dev *pdev,
struct net_device *ndev = NULL;
struct ql3_adapter *qdev = NULL;
static int cards_found;
- int pci_using_dac, err;
+ int err;
err = pci_enable_device(pdev);
if (err) {
@@ -3766,11 +3766,7 @@ static int ql3xxx_probe(struct pci_dev *pdev,
pci_set_master(pdev);
- if (!dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)))
- pci_using_dac = 1;
- else if (!(err = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32))))
- pci_using_dac = 0;
-
+ err = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64));
if (err) {
pr_err("%s no usable DMA configuration\n", pci_name(pdev));
goto err_out_free_regions;
@@ -3797,8 +3793,7 @@ static int ql3xxx_probe(struct pci_dev *pdev,
qdev->msg_enable = netif_msg_init(debug, default_msg);
- if (pci_using_dac)
- ndev->features |= NETIF_F_HIGHDMA;
+ ndev->features |= NETIF_F_HIGHDMA;
if (qdev->device_id == QL3032_DEVICE_ID)
ndev->features |= NETIF_F_IP_CSUM | NETIF_F_SG;
diff --git a/drivers/net/ethernet/qlogic/qlcnic/qlcnic.h b/drivers/net/ethernet/qlogic/qlcnic/qlcnic.h
index be7abee..b25102fde 100644
--- a/drivers/net/ethernet/qlogic/qlcnic/qlcnic.h
+++ b/drivers/net/ethernet/qlogic/qlcnic/qlcnic.h
@@ -1698,7 +1698,7 @@ int qlcnic_set_vxlan_port(struct qlcnic_adapter *adapter, u16 port);
int qlcnic_set_vxlan_parsing(struct qlcnic_adapter *adapter, u16 port);
int qlcnic_83xx_configure_opmode(struct qlcnic_adapter *adapter);
int qlcnic_read_mac_addr(struct qlcnic_adapter *);
-int qlcnic_setup_netdev(struct qlcnic_adapter *, struct net_device *, int);
+int qlcnic_setup_netdev(struct qlcnic_adapter *, struct net_device *);
void qlcnic_set_netdev_features(struct qlcnic_adapter *,
struct qlcnic_esw_func_cfg *);
void qlcnic_sriov_vf_set_multi(struct net_device *);
diff --git a/drivers/net/ethernet/qlogic/qlcnic/qlcnic_83xx_hw.h b/drivers/net/ethernet/qlogic/qlcnic/qlcnic_83xx_hw.h
index 6f1d9c1..23cd47d 100644
--- a/drivers/net/ethernet/qlogic/qlcnic/qlcnic_83xx_hw.h
+++ b/drivers/net/ethernet/qlogic/qlcnic/qlcnic_83xx_hw.h
@@ -609,7 +609,7 @@ int qlcnic_83xx_read_flash_descriptor_table(struct qlcnic_adapter *);
int qlcnic_83xx_flash_read32(struct qlcnic_adapter *, u32, u8 *, int);
int qlcnic_83xx_lockless_flash_read32(struct qlcnic_adapter *,
u32, u8 *, int);
-int qlcnic_83xx_init(struct qlcnic_adapter *, int);
+int qlcnic_83xx_init(struct qlcnic_adapter *);
int qlcnic_83xx_idc_ready_state_entry(struct qlcnic_adapter *);
void qlcnic_83xx_idc_poll_dev_state(struct work_struct *);
void qlcnic_83xx_idc_exit(struct qlcnic_adapter *);
diff --git a/drivers/net/ethernet/qlogic/qlcnic/qlcnic_83xx_init.c b/drivers/net/ethernet/qlogic/qlcnic/qlcnic_83xx_init.c
index 27dffa2..dbb8007 100644
--- a/drivers/net/ethernet/qlogic/qlcnic/qlcnic_83xx_init.c
+++ b/drivers/net/ethernet/qlogic/qlcnic/qlcnic_83xx_init.c
@@ -2432,7 +2432,7 @@ static void qlcnic_83xx_init_rings(struct qlcnic_adapter *adapter)
qlcnic_set_sds_ring_count(adapter, rx_cnt);
}
-int qlcnic_83xx_init(struct qlcnic_adapter *adapter, int pci_using_dac)
+int qlcnic_83xx_init(struct qlcnic_adapter *adapter)
{
struct qlcnic_hardware_context *ahw = adapter->ahw;
int err = 0;
@@ -2466,7 +2466,7 @@ int qlcnic_83xx_init(struct qlcnic_adapter *adapter, int pci_using_dac)
goto exit;
if (qlcnic_sriov_vf_check(adapter)) {
- err = qlcnic_sriov_vf_init(adapter, pci_using_dac);
+ err = qlcnic_sriov_vf_init(adapter);
if (err)
goto detach_mbx;
else
diff --git a/drivers/net/ethernet/qlogic/qlcnic/qlcnic_ethtool.c b/drivers/net/ethernet/qlogic/qlcnic/qlcnic_ethtool.c
index fc364b4..e10fe07 100644
--- a/drivers/net/ethernet/qlogic/qlcnic/qlcnic_ethtool.c
+++ b/drivers/net/ethernet/qlogic/qlcnic/qlcnic_ethtool.c
@@ -632,7 +632,9 @@ qlcnic_get_eeprom(struct net_device *dev, struct ethtool_eeprom *eeprom,
static void
qlcnic_get_ringparam(struct net_device *dev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct qlcnic_adapter *adapter = netdev_priv(dev);
@@ -663,7 +665,9 @@ qlcnic_validate_ringparam(u32 val, u32 min, u32 max, char *r_name)
static int
qlcnic_set_ringparam(struct net_device *dev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct qlcnic_adapter *adapter = netdev_priv(dev);
u16 num_rxd, num_jumbo_rxd, num_txd;
diff --git a/drivers/net/ethernet/qlogic/qlcnic/qlcnic_main.c b/drivers/net/ethernet/qlogic/qlcnic/qlcnic_main.c
index ed84f0f..d320567 100644
--- a/drivers/net/ethernet/qlogic/qlcnic/qlcnic_main.c
+++ b/drivers/net/ethernet/qlogic/qlcnic/qlcnic_main.c
@@ -2258,8 +2258,7 @@ static int qlcnic_set_real_num_queues(struct qlcnic_adapter *adapter,
}
int
-qlcnic_setup_netdev(struct qlcnic_adapter *adapter, struct net_device *netdev,
- int pci_using_dac)
+qlcnic_setup_netdev(struct qlcnic_adapter *adapter, struct net_device *netdev)
{
int err;
struct pci_dev *pdev = adapter->pdev;
@@ -2278,20 +2277,15 @@ qlcnic_setup_netdev(struct qlcnic_adapter *adapter, struct net_device *netdev,
netdev->features |= (NETIF_F_SG | NETIF_F_IP_CSUM | NETIF_F_RXCSUM |
NETIF_F_IPV6_CSUM | NETIF_F_GRO |
- NETIF_F_HW_VLAN_CTAG_RX);
+ NETIF_F_HW_VLAN_CTAG_RX | NETIF_F_HIGHDMA);
netdev->vlan_features |= (NETIF_F_SG | NETIF_F_IP_CSUM |
- NETIF_F_IPV6_CSUM);
+ NETIF_F_IPV6_CSUM | NETIF_F_HIGHDMA);
if (QLCNIC_IS_TSO_CAPABLE(adapter)) {
netdev->features |= (NETIF_F_TSO | NETIF_F_TSO6);
netdev->vlan_features |= (NETIF_F_TSO | NETIF_F_TSO6);
}
- if (pci_using_dac) {
- netdev->features |= NETIF_F_HIGHDMA;
- netdev->vlan_features |= NETIF_F_HIGHDMA;
- }
-
if (qlcnic_vlan_tx_check(adapter))
netdev->features |= (NETIF_F_HW_VLAN_CTAG_TX);
@@ -2341,20 +2335,6 @@ qlcnic_setup_netdev(struct qlcnic_adapter *adapter, struct net_device *netdev,
return 0;
}
-static int qlcnic_set_dma_mask(struct pci_dev *pdev, int *pci_using_dac)
-{
- if (!dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)))
- *pci_using_dac = 1;
- else if (!dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32)))
- *pci_using_dac = 0;
- else {
- dev_err(&pdev->dev, "Unable to set DMA mask, aborting\n");
- return -EIO;
- }
-
- return 0;
-}
-
void qlcnic_free_tx_rings(struct qlcnic_adapter *adapter)
{
int ring;
@@ -2441,8 +2421,8 @@ qlcnic_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
struct net_device *netdev = NULL;
struct qlcnic_adapter *adapter = NULL;
struct qlcnic_hardware_context *ahw;
- int err, pci_using_dac = -1;
char board_name[QLCNIC_MAX_BOARD_NAME_LEN + 19]; /* MAC + ": " + name */
+ int err;
err = pci_enable_device(pdev);
if (err)
@@ -2453,9 +2433,11 @@ qlcnic_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
goto err_out_disable_pdev;
}
- err = qlcnic_set_dma_mask(pdev, &pci_using_dac);
- if (err)
+ err = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64));
+ if (err) {
+ dev_err(&pdev->dev, "Unable to set DMA mask, aborting\n");
goto err_out_disable_pdev;
+ }
err = pci_request_regions(pdev, qlcnic_driver_name);
if (err)
@@ -2569,7 +2551,7 @@ qlcnic_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
} else if (qlcnic_83xx_check(adapter)) {
qlcnic_83xx_check_vf(adapter, ent);
adapter->portnum = adapter->ahw->pci_func;
- err = qlcnic_83xx_init(adapter, pci_using_dac);
+ err = qlcnic_83xx_init(adapter);
if (err) {
switch (err) {
case -ENOTRECOVERABLE:
@@ -2633,7 +2615,7 @@ qlcnic_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
if (adapter->portnum == 0)
qlcnic_set_drv_version(adapter);
- err = qlcnic_setup_netdev(adapter, netdev, pci_using_dac);
+ err = qlcnic_setup_netdev(adapter, netdev);
if (err)
goto err_out_disable_mbx_intr;
diff --git a/drivers/net/ethernet/qlogic/qlcnic/qlcnic_sriov.h b/drivers/net/ethernet/qlogic/qlcnic/qlcnic_sriov.h
index d0111cb3..c42b99c 100644
--- a/drivers/net/ethernet/qlogic/qlcnic/qlcnic_sriov.h
+++ b/drivers/net/ethernet/qlogic/qlcnic/qlcnic_sriov.h
@@ -188,7 +188,7 @@ int qlcnic_sriov_init(struct qlcnic_adapter *, int);
void qlcnic_sriov_cleanup(struct qlcnic_adapter *);
void __qlcnic_sriov_cleanup(struct qlcnic_adapter *);
void qlcnic_sriov_vf_register_map(struct qlcnic_hardware_context *);
-int qlcnic_sriov_vf_init(struct qlcnic_adapter *, int);
+int qlcnic_sriov_vf_init(struct qlcnic_adapter *);
void qlcnic_sriov_vf_set_ops(struct qlcnic_adapter *);
int qlcnic_sriov_func_to_index(struct qlcnic_adapter *, u8);
void qlcnic_sriov_handle_bc_event(struct qlcnic_adapter *, u32);
diff --git a/drivers/net/ethernet/qlogic/qlcnic/qlcnic_sriov_common.c b/drivers/net/ethernet/qlogic/qlcnic/qlcnic_sriov_common.c
index 42a44c9..9282321c 100644
--- a/drivers/net/ethernet/qlogic/qlcnic/qlcnic_sriov_common.c
+++ b/drivers/net/ethernet/qlogic/qlcnic/qlcnic_sriov_common.c
@@ -525,8 +525,7 @@ static int qlcnic_sriov_vf_init_driver(struct qlcnic_adapter *adapter)
return 0;
}
-static int qlcnic_sriov_setup_vf(struct qlcnic_adapter *adapter,
- int pci_using_dac)
+static int qlcnic_sriov_setup_vf(struct qlcnic_adapter *adapter)
{
int err;
@@ -571,7 +570,7 @@ static int qlcnic_sriov_setup_vf(struct qlcnic_adapter *adapter,
if (err)
goto err_out_send_channel_term;
- err = qlcnic_setup_netdev(adapter, adapter->netdev, pci_using_dac);
+ err = qlcnic_setup_netdev(adapter, adapter->netdev);
if (err)
goto err_out_send_channel_term;
@@ -614,7 +613,7 @@ static int qlcnic_sriov_check_dev_ready(struct qlcnic_adapter *adapter)
return 0;
}
-int qlcnic_sriov_vf_init(struct qlcnic_adapter *adapter, int pci_using_dac)
+int qlcnic_sriov_vf_init(struct qlcnic_adapter *adapter)
{
struct qlcnic_hardware_context *ahw = adapter->ahw;
int err;
@@ -631,7 +630,7 @@ int qlcnic_sriov_vf_init(struct qlcnic_adapter *adapter, int pci_using_dac)
if (err)
return err;
- err = qlcnic_sriov_setup_vf(adapter, pci_using_dac);
+ err = qlcnic_sriov_setup_vf(adapter);
if (err)
return err;
diff --git a/drivers/net/ethernet/qualcomm/emac/emac-ethtool.c b/drivers/net/ethernet/qualcomm/emac/emac-ethtool.c
index f72e13b..f502db9 100644
--- a/drivers/net/ethernet/qualcomm/emac/emac-ethtool.c
+++ b/drivers/net/ethernet/qualcomm/emac/emac-ethtool.c
@@ -133,7 +133,9 @@ static int emac_nway_reset(struct net_device *netdev)
}
static void emac_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct emac_adapter *adpt = netdev_priv(netdev);
@@ -144,7 +146,9 @@ static void emac_get_ringparam(struct net_device *netdev,
}
static int emac_set_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct emac_adapter *adpt = netdev_priv(netdev);
diff --git a/drivers/net/ethernet/qualcomm/qca_debug.c b/drivers/net/ethernet/qualcomm/qca_debug.c
index d59fff2..792ce9a 100644
--- a/drivers/net/ethernet/qualcomm/qca_debug.c
+++ b/drivers/net/ethernet/qualcomm/qca_debug.c
@@ -246,7 +246,9 @@ qcaspi_get_regs(struct net_device *dev, struct ethtool_regs *regs, void *p)
}
static void
-qcaspi_get_ringparam(struct net_device *dev, struct ethtool_ringparam *ring)
+qcaspi_get_ringparam(struct net_device *dev, struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct qcaspi *qca = netdev_priv(dev);
@@ -257,7 +259,9 @@ qcaspi_get_ringparam(struct net_device *dev, struct ethtool_ringparam *ring)
}
static int
-qcaspi_set_ringparam(struct net_device *dev, struct ethtool_ringparam *ring)
+qcaspi_set_ringparam(struct net_device *dev, struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
const struct net_device_ops *ops = dev->netdev_ops;
struct qcaspi *qca = netdev_priv(dev);
diff --git a/drivers/net/ethernet/realtek/8139cp.c b/drivers/net/ethernet/realtek/8139cp.c
index 4f39f84..ad7b9e9 100644
--- a/drivers/net/ethernet/realtek/8139cp.c
+++ b/drivers/net/ethernet/realtek/8139cp.c
@@ -1388,7 +1388,9 @@ static void cp_get_drvinfo (struct net_device *dev, struct ethtool_drvinfo *info
}
static void cp_get_ringparam(struct net_device *dev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
ring->rx_max_pending = CP_RX_RING_SIZE;
ring->tx_max_pending = CP_TX_RING_SIZE;
diff --git a/drivers/net/ethernet/realtek/r8169_main.c b/drivers/net/ethernet/realtek/r8169_main.c
index 86c44bc..7161a5b 100644
--- a/drivers/net/ethernet/realtek/r8169_main.c
+++ b/drivers/net/ethernet/realtek/r8169_main.c
@@ -615,6 +615,7 @@ struct rtl8169_private {
struct ring_info tx_skb[NUM_TX_DESC]; /* Tx data buffers */
u16 cp_cmd;
u32 irq_mask;
+ int irq;
struct clk *clk;
struct {
@@ -1873,7 +1874,9 @@ static int rtl8169_set_eee(struct net_device *dev, struct ethtool_eee *data)
}
static void rtl8169_get_ringparam(struct net_device *dev,
- struct ethtool_ringparam *data)
+ struct ethtool_ringparam *data,
+ struct kernel_ethtool_ringparam *kernel_data,
+ struct netlink_ext_ack *extack)
{
data->rx_max_pending = NUM_RX_DESC;
data->rx_pending = NUM_RX_DESC;
@@ -1969,8 +1972,11 @@ static enum mac_version rtl8169_get_mac_version(u16 xid, bool gmii)
{ 0x7cf, 0x641, RTL_GIGA_MAC_VER_63 },
/* 8125A family. */
- { 0x7cf, 0x608, RTL_GIGA_MAC_VER_60 },
- { 0x7c8, 0x608, RTL_GIGA_MAC_VER_61 },
+ { 0x7cf, 0x609, RTL_GIGA_MAC_VER_61 },
+ /* It seems only XID 609 made it to the mass market.
+ * { 0x7cf, 0x608, RTL_GIGA_MAC_VER_60 },
+ * { 0x7c8, 0x608, RTL_GIGA_MAC_VER_61 },
+ */
/* RTL8117 */
{ 0x7cf, 0x54b, RTL_GIGA_MAC_VER_53 },
@@ -1978,17 +1984,26 @@ static enum mac_version rtl8169_get_mac_version(u16 xid, bool gmii)
/* 8168EP family. */
{ 0x7cf, 0x502, RTL_GIGA_MAC_VER_51 },
- { 0x7cf, 0x501, RTL_GIGA_MAC_VER_50 },
- { 0x7cf, 0x500, RTL_GIGA_MAC_VER_49 },
+ /* It seems this chip version never made it to
+ * the wild. Let's disable detection.
+ * { 0x7cf, 0x501, RTL_GIGA_MAC_VER_50 },
+ * { 0x7cf, 0x500, RTL_GIGA_MAC_VER_49 },
+ */
/* 8168H family. */
{ 0x7cf, 0x541, RTL_GIGA_MAC_VER_46 },
- { 0x7cf, 0x540, RTL_GIGA_MAC_VER_45 },
+ /* It seems this chip version never made it to
+ * the wild. Let's disable detection.
+ * { 0x7cf, 0x540, RTL_GIGA_MAC_VER_45 },
+ */
/* 8168G family. */
{ 0x7cf, 0x5c8, RTL_GIGA_MAC_VER_44 },
{ 0x7cf, 0x509, RTL_GIGA_MAC_VER_42 },
- { 0x7cf, 0x4c1, RTL_GIGA_MAC_VER_41 },
+ /* It seems this chip version never made it to
+ * the wild. Let's disable detection.
+ * { 0x7cf, 0x4c1, RTL_GIGA_MAC_VER_41 },
+ */
{ 0x7cf, 0x4c0, RTL_GIGA_MAC_VER_40 },
/* 8168F family. */
@@ -4698,7 +4713,7 @@ static int rtl8169_close(struct net_device *dev)
cancel_work_sync(&tp->wk.work);
- free_irq(pci_irq_vector(pdev, 0), tp);
+ free_irq(tp->irq, tp);
phy_disconnect(tp->phydev);
@@ -4719,7 +4734,7 @@ static void rtl8169_netpoll(struct net_device *dev)
{
struct rtl8169_private *tp = netdev_priv(dev);
- rtl8169_interrupt(pci_irq_vector(tp->pci_dev, 0), tp);
+ rtl8169_interrupt(tp->irq, tp);
}
#endif
@@ -4753,8 +4768,7 @@ static int rtl_open(struct net_device *dev)
rtl_request_firmware(tp);
irqflags = pci_dev_msi_enabled(pdev) ? IRQF_NO_THREAD : IRQF_SHARED;
- retval = request_irq(pci_irq_vector(pdev, 0), rtl8169_interrupt,
- irqflags, dev->name, tp);
+ retval = request_irq(tp->irq, rtl8169_interrupt, irqflags, dev->name, tp);
if (retval < 0)
goto err_release_fw_2;
@@ -4771,7 +4785,7 @@ static int rtl_open(struct net_device *dev)
return retval;
err_free_irq:
- free_irq(pci_irq_vector(pdev, 0), tp);
+ free_irq(tp->irq, tp);
err_release_fw_2:
rtl_release_firmware(tp);
rtl8169_rx_clear(tp);
@@ -5272,12 +5286,6 @@ static int rtl_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
if (rc)
return rc;
- /* Disable ASPM L1 as that cause random device stop working
- * problems as well as full system hangs for some PCIe devices users.
- */
- rc = pci_disable_link_state(pdev, PCIE_LINK_STATE_L1);
- tp->aspm_manageable = !rc;
-
/* enable device (incl. PCI PM wakeup and hotplug setup) */
rc = pcim_enable_device(pdev);
if (rc < 0) {
@@ -5320,6 +5328,17 @@ static int rtl_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
tp->mac_version = chipset;
+ /* Disable ASPM L1 as that cause random device stop working
+ * problems as well as full system hangs for some PCIe devices users.
+ * Chips from RTL8168h partially have issues with L1.2, but seem
+ * to work fine with L1 and L1.1.
+ */
+ if (tp->mac_version >= RTL_GIGA_MAC_VER_45)
+ rc = pci_disable_link_state(pdev, PCIE_LINK_STATE_L1_2);
+ else
+ rc = pci_disable_link_state(pdev, PCIE_LINK_STATE_L1);
+ tp->aspm_manageable = !rc;
+
tp->dash_type = rtl_check_dash(tp);
tp->cp_cmd = RTL_R16(tp, CPlusCmd) & CPCMD_MASK;
@@ -5341,6 +5360,7 @@ static int rtl_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
dev_err(&pdev->dev, "Can't allocate interrupt\n");
return rc;
}
+ tp->irq = pci_irq_vector(pdev, 0);
INIT_WORK(&tp->wk.work, rtl_task);
@@ -5375,12 +5395,12 @@ static int rtl_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
*/
if (rtl_chip_supports_csum_v2(tp)) {
dev->hw_features |= NETIF_F_SG | NETIF_F_TSO | NETIF_F_TSO6;
- dev->gso_max_size = RTL_GSO_MAX_SIZE_V2;
- dev->gso_max_segs = RTL_GSO_MAX_SEGS_V2;
+ netif_set_gso_max_size(dev, RTL_GSO_MAX_SIZE_V2);
+ netif_set_gso_max_segs(dev, RTL_GSO_MAX_SEGS_V2);
} else {
dev->hw_features |= NETIF_F_SG | NETIF_F_TSO;
- dev->gso_max_size = RTL_GSO_MAX_SIZE_V1;
- dev->gso_max_segs = RTL_GSO_MAX_SEGS_V1;
+ netif_set_gso_max_size(dev, RTL_GSO_MAX_SIZE_V1);
+ netif_set_gso_max_segs(dev, RTL_GSO_MAX_SEGS_V1);
}
dev->hw_features |= NETIF_F_RXALL;
@@ -5416,8 +5436,7 @@ static int rtl_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
return rc;
netdev_info(dev, "%s, %pM, XID %03x, IRQ %d\n",
- rtl_chip_infos[chipset].name, dev->dev_addr, xid,
- pci_irq_vector(pdev, 0));
+ rtl_chip_infos[chipset].name, dev->dev_addr, xid, tp->irq);
if (jumbo_max)
netdev_info(dev, "jumbo features [frames: %d bytes, tx checksumming: %s]\n",
diff --git a/drivers/net/ethernet/renesas/ravb_main.c b/drivers/net/ethernet/renesas/ravb_main.c
index b4c597f..b215cde 100644
--- a/drivers/net/ethernet/renesas/ravb_main.c
+++ b/drivers/net/ethernet/renesas/ravb_main.c
@@ -30,8 +30,7 @@
#include <linux/spinlock.h>
#include <linux/sys_soc.h>
#include <linux/reset.h>
-
-#include <asm/div64.h>
+#include <linux/math64.h>
#include "ravb.h"
@@ -1605,7 +1604,9 @@ static void ravb_get_strings(struct net_device *ndev, u32 stringset, u8 *data)
}
static void ravb_get_ringparam(struct net_device *ndev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct ravb_private *priv = netdev_priv(ndev);
@@ -1616,7 +1617,9 @@ static void ravb_get_ringparam(struct net_device *ndev,
}
static int ravb_set_ringparam(struct net_device *ndev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct ravb_private *priv = netdev_priv(ndev);
const struct ravb_hw_info *info = priv->info;
@@ -2218,10 +2221,6 @@ static int ravb_hwtstamp_set(struct net_device *ndev, struct ifreq *req)
if (copy_from_user(&config, req->ifr_data, sizeof(config)))
return -EFAULT;
- /* Reserved for future extensions */
- if (config.flags)
- return -EINVAL;
-
switch (config.tx_type) {
case HWTSTAMP_TX_OFF:
tstamp_tx_ctrl = 0;
@@ -2488,8 +2487,7 @@ static int ravb_set_gti(struct net_device *ndev)
if (!rate)
return -EINVAL;
- inc = 1000000000ULL << 20;
- do_div(inc, rate);
+ inc = div64_ul(1000000000ULL << 20, rate);
if (inc < GTI_TIV_MIN || inc > GTI_TIV_MAX) {
dev_err(dev, "gti.tiv increment 0x%llx is outside the range 0x%x - 0x%x\n",
diff --git a/drivers/net/ethernet/renesas/sh_eth.c b/drivers/net/ethernet/renesas/sh_eth.c
index a3fbb22..d947a62 100644
--- a/drivers/net/ethernet/renesas/sh_eth.c
+++ b/drivers/net/ethernet/renesas/sh_eth.c
@@ -2296,7 +2296,9 @@ static void sh_eth_get_strings(struct net_device *ndev, u32 stringset, u8 *data)
}
static void sh_eth_get_ringparam(struct net_device *ndev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct sh_eth_private *mdp = netdev_priv(ndev);
@@ -2307,7 +2309,9 @@ static void sh_eth_get_ringparam(struct net_device *ndev,
}
static int sh_eth_set_ringparam(struct net_device *ndev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct sh_eth_private *mdp = netdev_priv(ndev);
int ret;
@@ -3364,8 +3368,7 @@ static int sh_eth_drv_probe(struct platform_device *pdev)
/* MDIO bus init */
ret = sh_mdio_init(mdp, pd);
if (ret) {
- if (ret != -EPROBE_DEFER)
- dev_err(&pdev->dev, "MDIO init failed: %d\n", ret);
+ dev_err_probe(&pdev->dev, ret, "MDIO init failed\n");
goto out_release;
}
diff --git a/drivers/net/ethernet/rocker/rocker_main.c b/drivers/net/ethernet/rocker/rocker_main.c
index ba40628..3fcea21 100644
--- a/drivers/net/ethernet/rocker/rocker_main.c
+++ b/drivers/net/ethernet/rocker/rocker_main.c
@@ -1995,17 +1995,6 @@ static int rocker_port_get_phys_port_name(struct net_device *dev,
return err ? -EOPNOTSUPP : 0;
}
-static int rocker_port_change_proto_down(struct net_device *dev,
- bool proto_down)
-{
- struct rocker_port *rocker_port = netdev_priv(dev);
-
- if (rocker_port->dev->flags & IFF_UP)
- rocker_port_set_enable(rocker_port, !proto_down);
- rocker_port->dev->proto_down = proto_down;
- return 0;
-}
-
static void rocker_port_neigh_destroy(struct net_device *dev,
struct neighbour *n)
{
@@ -2037,7 +2026,6 @@ static const struct net_device_ops rocker_port_netdev_ops = {
.ndo_set_mac_address = rocker_port_set_mac_address,
.ndo_change_mtu = rocker_port_change_mtu,
.ndo_get_phys_port_name = rocker_port_get_phys_port_name,
- .ndo_change_proto_down = rocker_port_change_proto_down,
.ndo_neigh_destroy = rocker_port_neigh_destroy,
.ndo_get_port_parent_id = rocker_port_get_port_parent_id,
};
@@ -2882,19 +2870,10 @@ static int rocker_probe(struct pci_dev *pdev, const struct pci_device_id *id)
goto err_pci_request_regions;
}
- err = dma_set_mask(&pdev->dev, DMA_BIT_MASK(64));
- if (!err) {
- err = dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(64));
- if (err) {
- dev_err(&pdev->dev, "dma_set_coherent_mask failed\n");
- goto err_pci_set_dma_mask;
- }
- } else {
- err = dma_set_mask(&pdev->dev, DMA_BIT_MASK(32));
- if (err) {
- dev_err(&pdev->dev, "dma_set_mask failed\n");
- goto err_pci_set_dma_mask;
- }
+ err = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64));
+ if (err) {
+ dev_err(&pdev->dev, "dma_set_mask failed\n");
+ goto err_pci_set_dma_mask;
}
if (pci_resource_len(pdev, 0) < ROCKER_PCI_BAR0_SIZE) {
diff --git a/drivers/net/ethernet/rocker/rocker_ofdpa.c b/drivers/net/ethernet/rocker/rocker_ofdpa.c
index 3e1ca7a..bc70c6ab 100644
--- a/drivers/net/ethernet/rocker/rocker_ofdpa.c
+++ b/drivers/net/ethernet/rocker/rocker_ofdpa.c
@@ -2783,7 +2783,8 @@ static void ofdpa_fib4_abort(struct rocker *rocker)
if (!ofdpa_port)
continue;
nh->fib_nh_flags &= ~RTNH_F_OFFLOAD;
- ofdpa_flow_tbl_del(ofdpa_port, OFDPA_OP_FLAG_REMOVE,
+ ofdpa_flow_tbl_del(ofdpa_port,
+ OFDPA_OP_FLAG_REMOVE | OFDPA_OP_FLAG_NOWAIT,
flow_entry);
}
spin_unlock_irqrestore(&ofdpa->flow_tbl_lock, flags);
diff --git a/drivers/net/ethernet/sfc/ef100_ethtool.c b/drivers/net/ethernet/sfc/ef100_ethtool.c
index 835c838..5dba412 100644
--- a/drivers/net/ethernet/sfc/ef100_ethtool.c
+++ b/drivers/net/ethernet/sfc/ef100_ethtool.c
@@ -20,8 +20,11 @@
/* This is the maximum number of descriptor rings supported by the QDMA */
#define EFX_EF100_MAX_DMAQ_SIZE 16384UL
-static void ef100_ethtool_get_ringparam(struct net_device *net_dev,
- struct ethtool_ringparam *ring)
+static void
+ef100_ethtool_get_ringparam(struct net_device *net_dev,
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct efx_nic *efx = netdev_priv(net_dev);
diff --git a/drivers/net/ethernet/sfc/ef100_nic.c b/drivers/net/ethernet/sfc/ef100_nic.c
index e77a5cb..f79b14a1 100644
--- a/drivers/net/ethernet/sfc/ef100_nic.c
+++ b/drivers/net/ethernet/sfc/ef100_nic.c
@@ -996,11 +996,11 @@ static int ef100_process_design_param(struct efx_nic *efx,
return 0;
case ESE_EF100_DP_GZ_TSO_MAX_PAYLOAD_LEN:
nic_data->tso_max_payload_len = min_t(u64, reader->value, GSO_MAX_SIZE);
- efx->net_dev->gso_max_size = nic_data->tso_max_payload_len;
+ netif_set_gso_max_size(efx->net_dev, nic_data->tso_max_payload_len);
return 0;
case ESE_EF100_DP_GZ_TSO_MAX_PAYLOAD_NUM_SEGS:
nic_data->tso_max_payload_num_segs = min_t(u64, reader->value, 0xffff);
- efx->net_dev->gso_max_segs = nic_data->tso_max_payload_num_segs;
+ netif_set_gso_max_segs(efx->net_dev, nic_data->tso_max_payload_num_segs);
return 0;
case ESE_EF100_DP_GZ_TSO_MAX_NUM_FRAMES:
nic_data->tso_max_frames = min_t(u64, reader->value, 0xffff);
@@ -1125,7 +1125,7 @@ static int ef100_probe_main(struct efx_nic *efx)
nic_data->tso_max_frames = ESE_EF100_DP_GZ_TSO_MAX_NUM_FRAMES_DEFAULT;
nic_data->tso_max_payload_num_segs = ESE_EF100_DP_GZ_TSO_MAX_PAYLOAD_NUM_SEGS_DEFAULT;
nic_data->tso_max_payload_len = ESE_EF100_DP_GZ_TSO_MAX_PAYLOAD_LEN_DEFAULT;
- net_dev->gso_max_segs = ESE_EF100_DP_GZ_TSO_MAX_HDR_NUM_SEGS_DEFAULT;
+ netif_set_gso_max_segs(net_dev, ESE_EF100_DP_GZ_TSO_MAX_HDR_NUM_SEGS_DEFAULT);
/* Read design parameters */
rc = ef100_check_design_params(efx);
if (rc) {
diff --git a/drivers/net/ethernet/sfc/efx.c b/drivers/net/ethernet/sfc/efx.c
index 6960a2f..302dc83 100644
--- a/drivers/net/ethernet/sfc/efx.c
+++ b/drivers/net/ethernet/sfc/efx.c
@@ -5,6 +5,7 @@
* Copyright 2005-2013 Solarflare Communications Inc.
*/
+#include <linux/filter.h>
#include <linux/module.h>
#include <linux/pci.h>
#include <linux/netdevice.h>
@@ -709,7 +710,7 @@ static int efx_register_netdev(struct efx_nic *efx)
if (efx_nic_rev(efx) >= EFX_REV_HUNT_A0)
net_dev->priv_flags |= IFF_UNICAST_FLT;
net_dev->ethtool_ops = &efx_ethtool_ops;
- net_dev->gso_max_segs = EFX_TSO_MAX_SEGS;
+ netif_set_gso_max_segs(net_dev, EFX_TSO_MAX_SEGS);
net_dev->min_mtu = EFX_MIN_MTU;
net_dev->max_mtu = EFX_MAX_MTU;
diff --git a/drivers/net/ethernet/sfc/efx_channels.c b/drivers/net/ethernet/sfc/efx_channels.c
index 3dbea02..ead550a 100644
--- a/drivers/net/ethernet/sfc/efx_channels.c
+++ b/drivers/net/ethernet/sfc/efx_channels.c
@@ -10,6 +10,7 @@
#include "net_driver.h"
#include <linux/module.h>
+#include <linux/filter.h>
#include "efx_channels.h"
#include "efx.h"
#include "efx_common.h"
@@ -818,11 +819,8 @@ int efx_realloc_channels(struct efx_nic *efx, u32 rxq_entries, u32 txq_entries)
old_txq_entries = efx->txq_entries;
efx->rxq_entries = rxq_entries;
efx->txq_entries = txq_entries;
- for (i = 0; i < efx->n_channels; i++) {
- channel = efx->channel[i];
- efx->channel[i] = other_channel[i];
- other_channel[i] = channel;
- }
+ for (i = 0; i < efx->n_channels; i++)
+ swap(efx->channel[i], other_channel[i]);
/* Restart buffer table allocation */
efx->next_buffer_table = next_buffer_table;
@@ -864,11 +862,8 @@ int efx_realloc_channels(struct efx_nic *efx, u32 rxq_entries, u32 txq_entries)
/* Swap back */
efx->rxq_entries = old_rxq_entries;
efx->txq_entries = old_txq_entries;
- for (i = 0; i < efx->n_channels; i++) {
- channel = efx->channel[i];
- efx->channel[i] = other_channel[i];
- other_channel[i] = channel;
- }
+ for (i = 0; i < efx->n_channels; i++)
+ swap(efx->channel[i], other_channel[i]);
goto out;
}
diff --git a/drivers/net/ethernet/sfc/efx_common.c b/drivers/net/ethernet/sfc/efx_common.c
index f187631..af37c99 100644
--- a/drivers/net/ethernet/sfc/efx_common.c
+++ b/drivers/net/ethernet/sfc/efx_common.c
@@ -9,6 +9,7 @@
*/
#include "net_driver.h"
+#include <linux/filter.h>
#include <linux/module.h>
#include <linux/netdevice.h>
#include <net/gre.h>
diff --git a/drivers/net/ethernet/sfc/ethtool.c b/drivers/net/ethernet/sfc/ethtool.c
index e002ce2..4850637 100644
--- a/drivers/net/ethernet/sfc/ethtool.c
+++ b/drivers/net/ethernet/sfc/ethtool.c
@@ -157,8 +157,11 @@ static int efx_ethtool_set_coalesce(struct net_device *net_dev,
return 0;
}
-static void efx_ethtool_get_ringparam(struct net_device *net_dev,
- struct ethtool_ringparam *ring)
+static void
+efx_ethtool_get_ringparam(struct net_device *net_dev,
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct efx_nic *efx = netdev_priv(net_dev);
@@ -168,8 +171,11 @@ static void efx_ethtool_get_ringparam(struct net_device *net_dev,
ring->tx_pending = efx->txq_entries;
}
-static int efx_ethtool_set_ringparam(struct net_device *net_dev,
- struct ethtool_ringparam *ring)
+static int
+efx_ethtool_set_ringparam(struct net_device *net_dev,
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct efx_nic *efx = netdev_priv(net_dev);
u32 txq_entries;
diff --git a/drivers/net/ethernet/sfc/falcon/efx.c b/drivers/net/ethernet/sfc/falcon/efx.c
index 314c9c6..60c595e 100644
--- a/drivers/net/ethernet/sfc/falcon/efx.c
+++ b/drivers/net/ethernet/sfc/falcon/efx.c
@@ -2267,7 +2267,7 @@ static int ef4_register_netdev(struct ef4_nic *efx)
net_dev->irq = efx->pci_dev->irq;
net_dev->netdev_ops = &ef4_netdev_ops;
net_dev->ethtool_ops = &ef4_ethtool_ops;
- net_dev->gso_max_segs = EF4_TSO_MAX_SEGS;
+ netif_set_gso_max_segs(net_dev, EF4_TSO_MAX_SEGS);
net_dev->min_mtu = EF4_MIN_MTU;
net_dev->max_mtu = EF4_MAX_MTU;
diff --git a/drivers/net/ethernet/sfc/falcon/ethtool.c b/drivers/net/ethernet/sfc/falcon/ethtool.c
index 137e8a7..907254b3 100644
--- a/drivers/net/ethernet/sfc/falcon/ethtool.c
+++ b/drivers/net/ethernet/sfc/falcon/ethtool.c
@@ -637,8 +637,11 @@ static int ef4_ethtool_set_coalesce(struct net_device *net_dev,
return 0;
}
-static void ef4_ethtool_get_ringparam(struct net_device *net_dev,
- struct ethtool_ringparam *ring)
+static void
+ef4_ethtool_get_ringparam(struct net_device *net_dev,
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct ef4_nic *efx = netdev_priv(net_dev);
@@ -648,8 +651,11 @@ static void ef4_ethtool_get_ringparam(struct net_device *net_dev,
ring->tx_pending = efx->txq_entries;
}
-static int ef4_ethtool_set_ringparam(struct net_device *net_dev,
- struct ethtool_ringparam *ring)
+static int
+ef4_ethtool_set_ringparam(struct net_device *net_dev,
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct ef4_nic *efx = netdev_priv(net_dev);
u32 txq_entries;
diff --git a/drivers/net/ethernet/sfc/falcon/rx.c b/drivers/net/ethernet/sfc/falcon/rx.c
index 11a6aee..0c6cc21 100644
--- a/drivers/net/ethernet/sfc/falcon/rx.c
+++ b/drivers/net/ethernet/sfc/falcon/rx.c
@@ -110,6 +110,8 @@ static struct page *ef4_reuse_page(struct ef4_rx_queue *rx_queue)
struct ef4_rx_page_state *state;
unsigned index;
+ if (unlikely(!rx_queue->page_ring))
+ return NULL;
index = rx_queue->page_remove & rx_queue->page_ptr_mask;
page = rx_queue->page_ring[index];
if (page == NULL)
@@ -293,6 +295,9 @@ static void ef4_recycle_rx_pages(struct ef4_channel *channel,
{
struct ef4_rx_queue *rx_queue = ef4_channel_get_rx_queue(channel);
+ if (unlikely(!rx_queue->page_ring))
+ return;
+
do {
ef4_recycle_rx_page(channel, rx_buf);
rx_buf = ef4_rx_buf_next(rx_queue, rx_buf);
diff --git a/drivers/net/ethernet/sfc/mcdi_port_common.c b/drivers/net/ethernet/sfc/mcdi_port_common.c
index c4fe3c4..899cc16 100644
--- a/drivers/net/ethernet/sfc/mcdi_port_common.c
+++ b/drivers/net/ethernet/sfc/mcdi_port_common.c
@@ -71,7 +71,6 @@ int efx_mcdi_set_link(struct efx_nic *efx, u32 capabilities,
u32 flags, u32 loopback_mode, u32 loopback_speed)
{
MCDI_DECLARE_BUF(inbuf, MC_CMD_SET_LINK_IN_LEN);
- int rc;
BUILD_BUG_ON(MC_CMD_SET_LINK_OUT_LEN != 0);
@@ -80,9 +79,8 @@ int efx_mcdi_set_link(struct efx_nic *efx, u32 capabilities,
MCDI_SET_DWORD(inbuf, SET_LINK_IN_LOOPBACK_MODE, loopback_mode);
MCDI_SET_DWORD(inbuf, SET_LINK_IN_LOOPBACK_SPEED, loopback_speed);
- rc = efx_mcdi_rpc(efx, MC_CMD_SET_LINK, inbuf, sizeof(inbuf),
+ return efx_mcdi_rpc(efx, MC_CMD_SET_LINK, inbuf, sizeof(inbuf),
NULL, 0, NULL);
- return rc;
}
int efx_mcdi_loopback_modes(struct efx_nic *efx, u64 *loopback_modes)
diff --git a/drivers/net/ethernet/sfc/ptp.c b/drivers/net/ethernet/sfc/ptp.c
index 797e518..f0ef515 100644
--- a/drivers/net/ethernet/sfc/ptp.c
+++ b/drivers/net/ethernet/sfc/ptp.c
@@ -1765,9 +1765,6 @@ static int efx_ptp_ts_init(struct efx_nic *efx, struct hwtstamp_config *init)
{
int rc;
- if (init->flags)
- return -EINVAL;
-
if ((init->tx_type != HWTSTAMP_TX_OFF) &&
(init->tx_type != HWTSTAMP_TX_ON))
return -ERANGE;
diff --git a/drivers/net/ethernet/sfc/rx.c b/drivers/net/ethernet/sfc/rx.c
index 6067509..2375cef 100644
--- a/drivers/net/ethernet/sfc/rx.c
+++ b/drivers/net/ethernet/sfc/rx.c
@@ -338,7 +338,7 @@ static bool efx_do_xdp(struct efx_nic *efx, struct efx_channel *channel,
break;
default:
- bpf_warn_invalid_xdp_action(xdp_act);
+ bpf_warn_invalid_xdp_action(efx->net_dev, xdp_prog, xdp_act);
efx_free_rx_buffers(rx_queue, rx_buf, 1);
channel->n_rx_xdp_bad_drops++;
trace_xdp_exception(efx->net_dev, xdp_prog, xdp_act);
diff --git a/drivers/net/ethernet/sfc/rx_common.c b/drivers/net/ethernet/sfc/rx_common.c
index 0983abc..633ca77 100644
--- a/drivers/net/ethernet/sfc/rx_common.c
+++ b/drivers/net/ethernet/sfc/rx_common.c
@@ -45,6 +45,8 @@ static struct page *efx_reuse_page(struct efx_rx_queue *rx_queue)
unsigned int index;
struct page *page;
+ if (unlikely(!rx_queue->page_ring))
+ return NULL;
index = rx_queue->page_remove & rx_queue->page_ptr_mask;
page = rx_queue->page_ring[index];
if (page == NULL)
@@ -114,6 +116,9 @@ void efx_recycle_rx_pages(struct efx_channel *channel,
{
struct efx_rx_queue *rx_queue = efx_channel_get_rx_queue(channel);
+ if (unlikely(!rx_queue->page_ring))
+ return;
+
do {
efx_recycle_rx_page(channel, rx_buf);
rx_buf = efx_rx_buf_next(rx_queue, rx_buf);
diff --git a/drivers/net/ethernet/smsc/smc9194.c b/drivers/net/ethernet/smsc/smc9194.c
index 0ce403f..af661c6 100644
--- a/drivers/net/ethernet/smsc/smc9194.c
+++ b/drivers/net/ethernet/smsc/smc9194.c
@@ -856,6 +856,7 @@ static int __init smc_probe(struct net_device *dev, int ioaddr)
word configuration_register;
word memory_info_register;
word memory_cfg_register;
+ u8 addr[ETH_ALEN];
/* Grab the region so that no one else tries to probe our ioports. */
if (!request_region(ioaddr, SMC_IO_EXTENT, DRV_NAME))
@@ -924,9 +925,10 @@ static int __init smc_probe(struct net_device *dev, int ioaddr)
word address;
address = inw( ioaddr + ADDR0 + i );
- dev->dev_addr[ i + 1] = address >> 8;
- dev->dev_addr[ i ] = address & 0xFF;
+ addr[i + 1] = address >> 8;
+ addr[i] = address & 0xFF;
}
+ eth_hw_addr_set(dev, addr);
/* get the memory information */
diff --git a/drivers/net/ethernet/socionext/netsec.c b/drivers/net/ethernet/socionext/netsec.c
index de7d8bf..556bd35 100644
--- a/drivers/net/ethernet/socionext/netsec.c
+++ b/drivers/net/ethernet/socionext/netsec.c
@@ -933,7 +933,7 @@ static u32 netsec_run_xdp(struct netsec_priv *priv, struct bpf_prog *prog,
}
break;
default:
- bpf_warn_invalid_xdp_action(act);
+ bpf_warn_invalid_xdp_action(priv->ndev, prog, act);
fallthrough;
case XDP_ABORTED:
trace_xdp_exception(priv->ndev, prog, act);
@@ -1977,11 +1977,12 @@ static int netsec_register_mdio(struct netsec_priv *priv, u32 phy_addr)
static int netsec_probe(struct platform_device *pdev)
{
- struct resource *mmio_res, *eeprom_res, *irq_res;
+ struct resource *mmio_res, *eeprom_res;
struct netsec_priv *priv;
u32 hw_ver, phy_addr = 0;
struct net_device *ndev;
int ret;
+ int irq;
mmio_res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
if (!mmio_res) {
@@ -1995,11 +1996,9 @@ static int netsec_probe(struct platform_device *pdev)
return -ENODEV;
}
- irq_res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
- if (!irq_res) {
- dev_err(&pdev->dev, "No IRQ resource found.\n");
- return -ENODEV;
- }
+ irq = platform_get_irq(pdev, 0);
+ if (irq < 0)
+ return irq;
ndev = alloc_etherdev(sizeof(*priv));
if (!ndev)
@@ -2010,7 +2009,7 @@ static int netsec_probe(struct platform_device *pdev)
spin_lock_init(&priv->reglock);
SET_NETDEV_DEV(ndev, &pdev->dev);
platform_set_drvdata(pdev, priv);
- ndev->irq = irq_res->start;
+ ndev->irq = irq;
priv->dev = &pdev->dev;
priv->ndev = ndev;
diff --git a/drivers/net/ethernet/stmicro/stmmac/common.h b/drivers/net/ethernet/stmicro/stmmac/common.h
index 9160f9e..6b5d96b 100644
--- a/drivers/net/ethernet/stmicro/stmmac/common.h
+++ b/drivers/net/ethernet/stmicro/stmmac/common.h
@@ -317,6 +317,7 @@ enum tx_frame_status {
tx_not_ls = 0x1,
tx_err = 0x2,
tx_dma_own = 0x4,
+ tx_err_bump_tc = 0x8,
};
enum dma_irq_status {
diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac-qcom-ethqos.c b/drivers/net/ethernet/stmicro/stmmac/dwmac-qcom-ethqos.c
index 5c74b62..2ffa0a1 100644
--- a/drivers/net/ethernet/stmicro/stmmac/dwmac-qcom-ethqos.c
+++ b/drivers/net/ethernet/stmicro/stmmac/dwmac-qcom-ethqos.c
@@ -113,8 +113,10 @@ static void rgmii_updatel(struct qcom_ethqos *ethqos,
rgmii_writel(ethqos, temp, offset);
}
-static void rgmii_dump(struct qcom_ethqos *ethqos)
+static void rgmii_dump(void *priv)
{
+ struct qcom_ethqos *ethqos = priv;
+
dev_dbg(ðqos->pdev->dev, "Rgmii register dump\n");
dev_dbg(ðqos->pdev->dev, "RGMII_IO_MACRO_CONFIG: %x\n",
rgmii_readl(ethqos, RGMII_IO_MACRO_CONFIG));
@@ -447,6 +449,24 @@ static void ethqos_fix_mac_speed(void *priv, unsigned int speed)
ethqos_configure(ethqos);
}
+static int ethqos_clks_config(void *priv, bool enabled)
+{
+ struct qcom_ethqos *ethqos = priv;
+ int ret = 0;
+
+ if (enabled) {
+ ret = clk_prepare_enable(ethqos->rgmii_clk);
+ if (ret) {
+ dev_err(ðqos->pdev->dev, "rgmii_clk enable failed\n");
+ return ret;
+ }
+ } else {
+ clk_disable_unprepare(ethqos->rgmii_clk);
+ }
+
+ return ret;
+}
+
static int qcom_ethqos_probe(struct platform_device *pdev)
{
struct device_node *np = pdev->dev.of_node;
@@ -466,6 +486,8 @@ static int qcom_ethqos_probe(struct platform_device *pdev)
return PTR_ERR(plat_dat);
}
+ plat_dat->clks_config = ethqos_clks_config;
+
ethqos = devm_kzalloc(&pdev->dev, sizeof(*ethqos), GFP_KERNEL);
if (!ethqos) {
ret = -ENOMEM;
@@ -489,7 +511,7 @@ static int qcom_ethqos_probe(struct platform_device *pdev)
goto err_mem;
}
- ret = clk_prepare_enable(ethqos->rgmii_clk);
+ ret = ethqos_clks_config(ethqos, true);
if (ret)
goto err_mem;
@@ -499,6 +521,7 @@ static int qcom_ethqos_probe(struct platform_device *pdev)
plat_dat->bsp_priv = ethqos;
plat_dat->fix_mac_speed = ethqos_fix_mac_speed;
+ plat_dat->dump_debug_regs = rgmii_dump;
plat_dat->has_gmac4 = 1;
plat_dat->pmt = 1;
plat_dat->tso_en = of_property_read_bool(np, "snps,tso");
@@ -507,12 +530,10 @@ static int qcom_ethqos_probe(struct platform_device *pdev)
if (ret)
goto err_clk;
- rgmii_dump(ethqos);
-
return ret;
err_clk:
- clk_disable_unprepare(ethqos->rgmii_clk);
+ ethqos_clks_config(ethqos, false);
err_mem:
stmmac_remove_config_dt(pdev, plat_dat);
@@ -530,7 +551,7 @@ static int qcom_ethqos_remove(struct platform_device *pdev)
return -ENODEV;
ret = stmmac_pltfr_remove(pdev);
- clk_disable_unprepare(ethqos->rgmii_clk);
+ ethqos_clks_config(ethqos, false);
return ret;
}
diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac4_descs.c b/drivers/net/ethernet/stmicro/stmmac/dwmac4_descs.c
index cbf4429..d3b4765 100644
--- a/drivers/net/ethernet/stmicro/stmmac/dwmac4_descs.c
+++ b/drivers/net/ethernet/stmicro/stmmac/dwmac4_descs.c
@@ -32,6 +32,8 @@ static int dwmac4_wrback_get_tx_status(void *data, struct stmmac_extra_stats *x,
return tx_not_ls;
if (unlikely(tdes3 & TDES3_ERROR_SUMMARY)) {
+ ret = tx_err;
+
if (unlikely(tdes3 & TDES3_JABBER_TIMEOUT))
x->tx_jabber++;
if (unlikely(tdes3 & TDES3_PACKET_FLUSHED))
@@ -53,16 +55,16 @@ static int dwmac4_wrback_get_tx_status(void *data, struct stmmac_extra_stats *x,
if (unlikely(tdes3 & TDES3_EXCESSIVE_DEFERRAL))
x->tx_deferred++;
- if (unlikely(tdes3 & TDES3_UNDERFLOW_ERROR))
+ if (unlikely(tdes3 & TDES3_UNDERFLOW_ERROR)) {
x->tx_underflow++;
+ ret |= tx_err_bump_tc;
+ }
if (unlikely(tdes3 & TDES3_IP_HDR_ERROR))
x->tx_ip_header_error++;
if (unlikely(tdes3 & TDES3_PAYLOAD_ERROR))
x->tx_payload_error++;
-
- ret = tx_err;
}
if (unlikely(tdes3 & TDES3_DEFERRED))
diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac.h b/drivers/net/ethernet/stmicro/stmmac/stmmac.h
index 873b9e3..40b5ed9 100644
--- a/drivers/net/ethernet/stmicro/stmmac/stmmac.h
+++ b/drivers/net/ethernet/stmicro/stmmac/stmmac.h
@@ -10,7 +10,6 @@
#define __STMMAC_H__
#define STMMAC_RESOURCE_NAME "stmmaceth"
-#define DRV_MODULE_VERSION "Jan_2016"
#include <linux/clk.h>
#include <linux/hrtimer.h>
@@ -23,6 +22,7 @@
#include <linux/net_tstamp.h>
#include <linux/reset.h>
#include <net/page_pool.h>
+#include <uapi/linux/bpf.h>
struct stmmac_resources {
void __iomem *addr;
@@ -175,11 +175,14 @@ struct stmmac_flow_entry {
/* Rx Frame Steering */
enum stmmac_rfs_type {
STMMAC_RFS_T_VLAN,
+ STMMAC_RFS_T_LLDP,
+ STMMAC_RFS_T_1588,
STMMAC_RFS_T_MAX,
};
struct stmmac_rfs_entry {
unsigned long cookie;
+ u16 etype;
int in_use;
int type;
int tc;
@@ -334,8 +337,8 @@ void stmmac_set_ethtool_ops(struct net_device *netdev);
int stmmac_init_tstamp_counter(struct stmmac_priv *priv, u32 systime_flags);
void stmmac_ptp_register(struct stmmac_priv *priv);
void stmmac_ptp_unregister(struct stmmac_priv *priv);
-int stmmac_open(struct net_device *dev);
-int stmmac_release(struct net_device *dev);
+int stmmac_xdp_open(struct net_device *dev);
+void stmmac_xdp_release(struct net_device *dev);
int stmmac_resume(struct device *dev);
int stmmac_suspend(struct device *dev);
int stmmac_dvr_remove(struct device *dev);
diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_ethtool.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_ethtool.c
index d894558..164dff5 100644
--- a/drivers/net/ethernet/stmicro/stmmac/stmmac_ethtool.c
+++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_ethtool.c
@@ -290,7 +290,6 @@ static void stmmac_ethtool_getdrvinfo(struct net_device *dev,
strlcpy(info->bus_info, pci_name(priv->plat->pdev),
sizeof(info->bus_info));
}
- strlcpy(info->version, DRV_MODULE_VERSION, sizeof(info->version));
}
static int stmmac_ethtool_get_link_ksettings(struct net_device *dev,
@@ -463,7 +462,9 @@ static int stmmac_nway_reset(struct net_device *dev)
}
static void stmmac_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct stmmac_priv *priv = netdev_priv(netdev);
@@ -474,7 +475,9 @@ static void stmmac_get_ringparam(struct net_device *netdev,
}
static int stmmac_set_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
if (ring->rx_mini_pending || ring->rx_jumbo_pending ||
ring->rx_pending < DMA_MIN_RX_SIZE ||
diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
index 8ded4be..63ff2da 100644
--- a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
+++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
@@ -132,6 +132,8 @@ static irqreturn_t stmmac_msi_intr_tx(int irq, void *data);
static irqreturn_t stmmac_msi_intr_rx(int irq, void *data);
static void stmmac_tx_timer_arm(struct stmmac_priv *priv, u32 queue);
static void stmmac_flush_tx_descriptors(struct stmmac_priv *priv, int queue);
+static void stmmac_set_dma_operation_mode(struct stmmac_priv *priv, u32 txmode,
+ u32 rxmode, u32 chan);
#ifdef CONFIG_DEBUG_FS
static const struct net_device_ops stmmac_netdev_ops;
@@ -518,14 +520,6 @@ bool stmmac_eee_init(struct stmmac_priv *priv)
return true;
}
-static inline u32 stmmac_cdc_adjust(struct stmmac_priv *priv)
-{
- /* Correct the clk domain crossing(CDC) error */
- if (priv->plat->has_gmac4 && priv->plat->clk_ptp_rate)
- return (2 * NSEC_PER_SEC) / priv->plat->clk_ptp_rate;
- return 0;
-}
-
/* stmmac_get_tx_hwtstamp - get HW TX timestamps
* @priv: driver private structure
* @p : descriptor pointer
@@ -557,7 +551,7 @@ static void stmmac_get_tx_hwtstamp(struct stmmac_priv *priv,
}
if (found) {
- ns -= stmmac_cdc_adjust(priv);
+ ns -= priv->plat->cdc_error_adj;
memset(&shhwtstamp, 0, sizeof(struct skb_shared_hwtstamps));
shhwtstamp.hwtstamp = ns_to_ktime(ns);
@@ -594,7 +588,7 @@ static void stmmac_get_rx_hwtstamp(struct stmmac_priv *priv, struct dma_desc *p,
if (stmmac_get_rx_timestamp_status(priv, p, np, priv->adv_ts)) {
stmmac_get_timestamp(priv, desc, priv->adv_ts, &ns);
- ns -= stmmac_cdc_adjust(priv);
+ ns -= priv->plat->cdc_error_adj;
netdev_dbg(priv->dev, "get valid RX hw timestamp %llu\n", ns);
shhwtstamp = skb_hwtstamps(skb);
@@ -644,10 +638,6 @@ static int stmmac_hwtstamp_set(struct net_device *dev, struct ifreq *ifr)
netdev_dbg(priv->dev, "%s config flags:0x%x, tx_type:0x%x, rx_filter:0x%x\n",
__func__, config.flags, config.tx_type, config.rx_filter);
- /* reserved for future extensions */
- if (config.flags)
- return -EINVAL;
-
if (config.tx_type != HWTSTAMP_TX_OFF &&
config.tx_type != HWTSTAMP_TX_ON)
return -ERANGE;
@@ -2394,7 +2384,7 @@ static bool stmmac_xdp_xmit_zc(struct stmmac_priv *priv, u32 queue, u32 budget)
bool work_done = true;
/* Avoids TX time-out as we are sharing with slow path */
- nq->trans_start = jiffies;
+ txq_trans_cond_update(nq);
budget = min(budget, stmmac_tx_avail(priv, queue));
@@ -2478,6 +2468,21 @@ static bool stmmac_xdp_xmit_zc(struct stmmac_priv *priv, u32 queue, u32 budget)
return !!budget && work_done;
}
+static void stmmac_bump_dma_threshold(struct stmmac_priv *priv, u32 chan)
+{
+ if (unlikely(priv->xstats.threshold != SF_DMA_MODE) && tc <= 256) {
+ tc += 64;
+
+ if (priv->plat->force_thresh_dma_mode)
+ stmmac_set_dma_operation_mode(priv, tc, tc, chan);
+ else
+ stmmac_set_dma_operation_mode(priv, tc, SF_DMA_MODE,
+ chan);
+
+ priv->xstats.threshold = tc;
+ }
+}
+
/**
* stmmac_tx_clean - to manage the transmission completion
* @priv: driver private structure
@@ -2543,6 +2548,8 @@ static int stmmac_tx_clean(struct stmmac_priv *priv, int budget, u32 queue)
/* ... verify the status error condition */
if (unlikely(status & tx_err)) {
priv->dev->stats.tx_errors++;
+ if (unlikely(status & tx_err_bump_tc))
+ stmmac_bump_dma_threshold(priv, queue);
} else {
priv->dev->stats.tx_packets++;
priv->xstats.tx_pkt_n++;
@@ -2793,21 +2800,7 @@ static void stmmac_dma_interrupt(struct stmmac_priv *priv)
for (chan = 0; chan < tx_channel_count; chan++) {
if (unlikely(status[chan] & tx_hard_error_bump_tc)) {
/* Try to bump up the dma threshold on this failure */
- if (unlikely(priv->xstats.threshold != SF_DMA_MODE) &&
- (tc <= 256)) {
- tc += 64;
- if (priv->plat->force_thresh_dma_mode)
- stmmac_set_dma_operation_mode(priv,
- tc,
- tc,
- chan);
- else
- stmmac_set_dma_operation_mode(priv,
- tc,
- SF_DMA_MODE,
- chan);
- priv->xstats.threshold = tc;
- }
+ stmmac_bump_dma_threshold(priv, chan);
} else if (unlikely(status[chan] == tx_hard_error)) {
stmmac_tx_err(priv, chan);
}
@@ -3677,7 +3670,7 @@ static int stmmac_request_irq(struct net_device *dev)
* 0 on success and an appropriate (-)ve integer as defined in errno.h
* file on failure.
*/
-int stmmac_open(struct net_device *dev)
+static int stmmac_open(struct net_device *dev)
{
struct stmmac_priv *priv = netdev_priv(dev);
int mode = priv->plat->phy_interface;
@@ -3801,7 +3794,7 @@ static void stmmac_fpe_stop_wq(struct stmmac_priv *priv)
* Description:
* This is the stop entry point of the driver.
*/
-int stmmac_release(struct net_device *dev)
+static int stmmac_release(struct net_device *dev)
{
struct stmmac_priv *priv = netdev_priv(dev);
u32 chan;
@@ -4697,7 +4690,7 @@ static int stmmac_xdp_xmit_back(struct stmmac_priv *priv,
__netif_tx_lock(nq, cpu);
/* Avoids TX time-out as we are sharing with slow path */
- nq->trans_start = jiffies;
+ txq_trans_cond_update(nq);
res = stmmac_xdp_xmit_xdpf(priv, queue, xdpf, false);
if (res == STMMAC_XDP_TX)
@@ -4730,7 +4723,7 @@ static int __stmmac_xdp_run_prog(struct stmmac_priv *priv,
res = STMMAC_XDP_REDIRECT;
break;
default:
- bpf_warn_invalid_xdp_action(act);
+ bpf_warn_invalid_xdp_action(priv->dev, prog, act);
fallthrough;
case XDP_ABORTED:
trace_xdp_exception(priv->dev, prog, act);
@@ -5761,21 +5754,7 @@ static irqreturn_t stmmac_msi_intr_tx(int irq, void *data)
if (unlikely(status & tx_hard_error_bump_tc)) {
/* Try to bump up the dma threshold on this failure */
- if (unlikely(priv->xstats.threshold != SF_DMA_MODE) &&
- tc <= 256) {
- tc += 64;
- if (priv->plat->force_thresh_dma_mode)
- stmmac_set_dma_operation_mode(priv,
- tc,
- tc,
- chan);
- else
- stmmac_set_dma_operation_mode(priv,
- tc,
- SF_DMA_MODE,
- chan);
- priv->xstats.threshold = tc;
- }
+ stmmac_bump_dma_threshold(priv, chan);
} else if (unlikely(status == tx_hard_error)) {
stmmac_tx_err(priv, chan);
}
@@ -6335,7 +6314,7 @@ static int stmmac_xdp_xmit(struct net_device *dev, int num_frames,
__netif_tx_lock(nq, cpu);
/* Avoids TX time-out as we are sharing with slow path */
- nq->trans_start = jiffies;
+ txq_trans_cond_update(nq);
for (i = 0; i < num_frames; i++) {
int res;
@@ -6471,6 +6450,140 @@ void stmmac_enable_tx_queue(struct stmmac_priv *priv, u32 queue)
spin_unlock_irqrestore(&ch->lock, flags);
}
+void stmmac_xdp_release(struct net_device *dev)
+{
+ struct stmmac_priv *priv = netdev_priv(dev);
+ u32 chan;
+
+ /* Disable NAPI process */
+ stmmac_disable_all_queues(priv);
+
+ for (chan = 0; chan < priv->plat->tx_queues_to_use; chan++)
+ hrtimer_cancel(&priv->tx_queue[chan].txtimer);
+
+ /* Free the IRQ lines */
+ stmmac_free_irq(dev, REQ_IRQ_ERR_ALL, 0);
+
+ /* Stop TX/RX DMA channels */
+ stmmac_stop_all_dma(priv);
+
+ /* Release and free the Rx/Tx resources */
+ free_dma_desc_resources(priv);
+
+ /* Disable the MAC Rx/Tx */
+ stmmac_mac_set(priv, priv->ioaddr, false);
+
+ /* set trans_start so we don't get spurious
+ * watchdogs during reset
+ */
+ netif_trans_update(dev);
+ netif_carrier_off(dev);
+}
+
+int stmmac_xdp_open(struct net_device *dev)
+{
+ struct stmmac_priv *priv = netdev_priv(dev);
+ u32 rx_cnt = priv->plat->rx_queues_to_use;
+ u32 tx_cnt = priv->plat->tx_queues_to_use;
+ u32 dma_csr_ch = max(rx_cnt, tx_cnt);
+ struct stmmac_rx_queue *rx_q;
+ struct stmmac_tx_queue *tx_q;
+ u32 buf_size;
+ bool sph_en;
+ u32 chan;
+ int ret;
+
+ ret = alloc_dma_desc_resources(priv);
+ if (ret < 0) {
+ netdev_err(dev, "%s: DMA descriptors allocation failed\n",
+ __func__);
+ goto dma_desc_error;
+ }
+
+ ret = init_dma_desc_rings(dev, GFP_KERNEL);
+ if (ret < 0) {
+ netdev_err(dev, "%s: DMA descriptors initialization failed\n",
+ __func__);
+ goto init_error;
+ }
+
+ /* DMA CSR Channel configuration */
+ for (chan = 0; chan < dma_csr_ch; chan++)
+ stmmac_init_chan(priv, priv->ioaddr, priv->plat->dma_cfg, chan);
+
+ /* Adjust Split header */
+ sph_en = (priv->hw->rx_csum > 0) && priv->sph;
+
+ /* DMA RX Channel Configuration */
+ for (chan = 0; chan < rx_cnt; chan++) {
+ rx_q = &priv->rx_queue[chan];
+
+ stmmac_init_rx_chan(priv, priv->ioaddr, priv->plat->dma_cfg,
+ rx_q->dma_rx_phy, chan);
+
+ rx_q->rx_tail_addr = rx_q->dma_rx_phy +
+ (rx_q->buf_alloc_num *
+ sizeof(struct dma_desc));
+ stmmac_set_rx_tail_ptr(priv, priv->ioaddr,
+ rx_q->rx_tail_addr, chan);
+
+ if (rx_q->xsk_pool && rx_q->buf_alloc_num) {
+ buf_size = xsk_pool_get_rx_frame_size(rx_q->xsk_pool);
+ stmmac_set_dma_bfsize(priv, priv->ioaddr,
+ buf_size,
+ rx_q->queue_index);
+ } else {
+ stmmac_set_dma_bfsize(priv, priv->ioaddr,
+ priv->dma_buf_sz,
+ rx_q->queue_index);
+ }
+
+ stmmac_enable_sph(priv, priv->ioaddr, sph_en, chan);
+ }
+
+ /* DMA TX Channel Configuration */
+ for (chan = 0; chan < tx_cnt; chan++) {
+ tx_q = &priv->tx_queue[chan];
+
+ stmmac_init_tx_chan(priv, priv->ioaddr, priv->plat->dma_cfg,
+ tx_q->dma_tx_phy, chan);
+
+ tx_q->tx_tail_addr = tx_q->dma_tx_phy;
+ stmmac_set_tx_tail_ptr(priv, priv->ioaddr,
+ tx_q->tx_tail_addr, chan);
+
+ hrtimer_init(&tx_q->txtimer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+ tx_q->txtimer.function = stmmac_tx_timer;
+ }
+
+ /* Enable the MAC Rx/Tx */
+ stmmac_mac_set(priv, priv->ioaddr, true);
+
+ /* Start Rx & Tx DMA Channels */
+ stmmac_start_all_dma(priv);
+
+ ret = stmmac_request_irq(dev);
+ if (ret)
+ goto irq_error;
+
+ /* Enable NAPI process*/
+ stmmac_enable_all_queues(priv);
+ netif_carrier_on(dev);
+ netif_tx_start_all_queues(dev);
+
+ return 0;
+
+irq_error:
+ for (chan = 0; chan < priv->plat->tx_queues_to_use; chan++)
+ hrtimer_cancel(&priv->tx_queue[chan].txtimer);
+
+ stmmac_hw_teardown(dev);
+init_error:
+ free_dma_desc_resources(priv);
+dma_desc_error:
+ return ret;
+}
+
int stmmac_xsk_wakeup(struct net_device *dev, u32 queue, u32 flags)
{
struct stmmac_priv *priv = netdev_priv(dev);
@@ -7094,6 +7207,9 @@ int stmmac_dvr_probe(struct device *device,
stmmac_init_fs(ndev);
#endif
+ if (priv->plat->dump_debug_regs)
+ priv->plat->dump_debug_regs(priv->plat->bsp_priv);
+
/* Let pm_runtime_put() disable the clocks.
* If CONFIG_PM is not enabled, the clocks will stay powered.
*/
diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_ptp.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_ptp.c
index be9b58b..0d24ebd 100644
--- a/drivers/net/ethernet/stmicro/stmmac/stmmac_ptp.c
+++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_ptp.c
@@ -309,6 +309,11 @@ void stmmac_ptp_register(struct stmmac_priv *priv)
if (priv->plat->ptp_max_adj)
stmmac_ptp_clock_ops.max_adj = priv->plat->ptp_max_adj;
+ /* Calculate the clock domain crossing (CDC) error if necessary */
+ priv->plat->cdc_error_adj = 0;
+ if (priv->plat->has_gmac4 && priv->plat->clk_ptp_rate)
+ priv->plat->cdc_error_adj = (2 * NSEC_PER_SEC) / priv->plat->clk_ptp_rate;
+
stmmac_ptp_clock_ops.n_per_out = priv->dma_cap.pps_out_num;
stmmac_ptp_clock_ops.n_ext_ts = priv->dma_cap.aux_snapshot_n;
diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_tc.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_tc.c
index d0a2b28..d61766e 100644
--- a/drivers/net/ethernet/stmicro/stmmac/stmmac_tc.c
+++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_tc.c
@@ -237,6 +237,8 @@ static int tc_rfs_init(struct stmmac_priv *priv)
int i;
priv->rfs_entries_max[STMMAC_RFS_T_VLAN] = 8;
+ priv->rfs_entries_max[STMMAC_RFS_T_LLDP] = 1;
+ priv->rfs_entries_max[STMMAC_RFS_T_1588] = 1;
for (i = 0; i < STMMAC_RFS_T_MAX; i++)
priv->rfs_entries_total += priv->rfs_entries_max[i];
@@ -451,6 +453,8 @@ static int tc_parse_flow_actions(struct stmmac_priv *priv,
return 0;
}
+#define ETHER_TYPE_FULL_MASK cpu_to_be16(~0)
+
static int tc_add_basic_flow(struct stmmac_priv *priv,
struct flow_cls_offload *cls,
struct stmmac_flow_entry *entry)
@@ -464,6 +468,7 @@ static int tc_add_basic_flow(struct stmmac_priv *priv,
return -EINVAL;
flow_rule_match_basic(rule, &match);
+
entry->ip_proto = match.key->ip_proto;
return 0;
}
@@ -724,6 +729,114 @@ static int tc_del_vlan_flow(struct stmmac_priv *priv,
return 0;
}
+static int tc_add_ethtype_flow(struct stmmac_priv *priv,
+ struct flow_cls_offload *cls)
+{
+ struct stmmac_rfs_entry *entry = tc_find_rfs(priv, cls, false);
+ struct flow_rule *rule = flow_cls_offload_flow_rule(cls);
+ struct flow_dissector *dissector = rule->match.dissector;
+ int tc = tc_classid_to_hwtc(priv->dev, cls->classid);
+ struct flow_match_basic match;
+
+ if (!entry) {
+ entry = tc_find_rfs(priv, cls, true);
+ if (!entry)
+ return -ENOENT;
+ }
+
+ /* Nothing to do here */
+ if (!dissector_uses_key(dissector, FLOW_DISSECTOR_KEY_BASIC))
+ return -EINVAL;
+
+ if (tc < 0) {
+ netdev_err(priv->dev, "Invalid traffic class\n");
+ return -EINVAL;
+ }
+
+ flow_rule_match_basic(rule, &match);
+
+ if (match.mask->n_proto) {
+ u16 etype = ntohs(match.key->n_proto);
+
+ if (match.mask->n_proto != ETHER_TYPE_FULL_MASK) {
+ netdev_err(priv->dev, "Only full mask is supported for EthType filter");
+ return -EINVAL;
+ }
+ switch (etype) {
+ case ETH_P_LLDP:
+ if (priv->rfs_entries_cnt[STMMAC_RFS_T_LLDP] >=
+ priv->rfs_entries_max[STMMAC_RFS_T_LLDP])
+ return -ENOENT;
+
+ entry->type = STMMAC_RFS_T_LLDP;
+ priv->rfs_entries_cnt[STMMAC_RFS_T_LLDP]++;
+
+ stmmac_rx_queue_routing(priv, priv->hw,
+ PACKET_DCBCPQ, tc);
+ break;
+ case ETH_P_1588:
+ if (priv->rfs_entries_cnt[STMMAC_RFS_T_1588] >=
+ priv->rfs_entries_max[STMMAC_RFS_T_1588])
+ return -ENOENT;
+
+ entry->type = STMMAC_RFS_T_1588;
+ priv->rfs_entries_cnt[STMMAC_RFS_T_1588]++;
+
+ stmmac_rx_queue_routing(priv, priv->hw,
+ PACKET_PTPQ, tc);
+ break;
+ default:
+ netdev_err(priv->dev, "EthType(0x%x) is not supported", etype);
+ return -EINVAL;
+ }
+
+ entry->in_use = true;
+ entry->cookie = cls->cookie;
+ entry->tc = tc;
+ entry->etype = etype;
+
+ return 0;
+ }
+
+ return -EINVAL;
+}
+
+static int tc_del_ethtype_flow(struct stmmac_priv *priv,
+ struct flow_cls_offload *cls)
+{
+ struct stmmac_rfs_entry *entry = tc_find_rfs(priv, cls, false);
+
+ if (!entry || !entry->in_use ||
+ entry->type < STMMAC_RFS_T_LLDP ||
+ entry->type > STMMAC_RFS_T_1588)
+ return -ENOENT;
+
+ switch (entry->etype) {
+ case ETH_P_LLDP:
+ stmmac_rx_queue_routing(priv, priv->hw,
+ PACKET_DCBCPQ, 0);
+ priv->rfs_entries_cnt[STMMAC_RFS_T_LLDP]--;
+ break;
+ case ETH_P_1588:
+ stmmac_rx_queue_routing(priv, priv->hw,
+ PACKET_PTPQ, 0);
+ priv->rfs_entries_cnt[STMMAC_RFS_T_1588]--;
+ break;
+ default:
+ netdev_err(priv->dev, "EthType(0x%x) is not supported",
+ entry->etype);
+ return -EINVAL;
+ }
+
+ entry->in_use = false;
+ entry->cookie = 0;
+ entry->tc = 0;
+ entry->etype = 0;
+ entry->type = 0;
+
+ return 0;
+}
+
static int tc_add_flow_cls(struct stmmac_priv *priv,
struct flow_cls_offload *cls)
{
@@ -733,6 +846,10 @@ static int tc_add_flow_cls(struct stmmac_priv *priv,
if (!ret)
return ret;
+ ret = tc_add_ethtype_flow(priv, cls);
+ if (!ret)
+ return ret;
+
return tc_add_vlan_flow(priv, cls);
}
@@ -745,6 +862,10 @@ static int tc_del_flow_cls(struct stmmac_priv *priv,
if (!ret)
return ret;
+ ret = tc_del_ethtype_flow(priv, cls);
+ if (!ret)
+ return ret;
+
return tc_del_vlan_flow(priv, cls);
}
diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_xdp.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_xdp.c
index 2a616c6..9d4d8c3 100644
--- a/drivers/net/ethernet/stmicro/stmmac/stmmac_xdp.c
+++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_xdp.c
@@ -119,7 +119,7 @@ int stmmac_xdp_set_prog(struct stmmac_priv *priv, struct bpf_prog *prog,
need_update = !!priv->xdp_prog != !!prog;
if (if_running && need_update)
- stmmac_release(dev);
+ stmmac_xdp_release(dev);
old_prog = xchg(&priv->xdp_prog, prog);
if (old_prog)
@@ -129,7 +129,7 @@ int stmmac_xdp_set_prog(struct stmmac_priv *priv, struct bpf_prog *prog,
priv->sph = priv->sph_cap && !stmmac_xdp_is_enabled(priv);
if (if_running && need_update)
- stmmac_open(dev);
+ stmmac_xdp_open(dev);
return 0;
}
diff --git a/drivers/net/ethernet/sun/cassini.c b/drivers/net/ethernet/sun/cassini.c
index d2d4f47..dba9f12 100644
--- a/drivers/net/ethernet/sun/cassini.c
+++ b/drivers/net/ethernet/sun/cassini.c
@@ -4893,8 +4893,8 @@ static int cas_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
unsigned long casreg_len;
struct net_device *dev;
struct cas *cp;
- int i, err, pci_using_dac;
u16 pci_cmd;
+ int i, err;
u8 orig_cacheline_size = 0, cas_cacheline_size = 0;
if (cas_version_printed++ == 0)
@@ -4965,23 +4965,10 @@ static int cas_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
/* Configure DMA attributes. */
- if (!dma_set_mask(&pdev->dev, DMA_BIT_MASK(64))) {
- pci_using_dac = 1;
- err = dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(64));
- if (err < 0) {
- dev_err(&pdev->dev, "Unable to obtain 64-bit DMA "
- "for consistent allocations\n");
- goto err_out_free_res;
- }
-
- } else {
- err = dma_set_mask(&pdev->dev, DMA_BIT_MASK(32));
- if (err) {
- dev_err(&pdev->dev, "No usable DMA configuration, "
- "aborting\n");
- goto err_out_free_res;
- }
- pci_using_dac = 0;
+ err = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64));
+ if (err) {
+ dev_err(&pdev->dev, "No usable DMA configuration, aborting\n");
+ goto err_out_free_res;
}
casreg_len = pci_resource_len(pdev, 0);
@@ -5087,8 +5074,7 @@ static int cas_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
if ((cp->cas_flags & CAS_FLAG_NO_HW_CSUM) == 0)
dev->features |= NETIF_F_HW_CSUM | NETIF_F_SG;
- if (pci_using_dac)
- dev->features |= NETIF_F_HIGHDMA;
+ dev->features |= NETIF_F_HIGHDMA;
/* MTU range: 60 - varies or 9000 */
dev->min_mtu = CAS_MIN_MTU;
diff --git a/drivers/net/ethernet/tehuti/tehuti.c b/drivers/net/ethernet/tehuti/tehuti.c
index 0775a55..985073e 100644
--- a/drivers/net/ethernet/tehuti/tehuti.c
+++ b/drivers/net/ethernet/tehuti/tehuti.c
@@ -1884,10 +1884,10 @@ bdx_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
{
struct net_device *ndev;
struct bdx_priv *priv;
- int err, pci_using_dac, port;
unsigned long pciaddr;
u32 regionSize;
struct pci_nic *nic;
+ int err, port;
ENTER;
@@ -1900,16 +1900,10 @@ bdx_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
if (err) /* it triggers interrupt, dunno why. */
goto err_pci; /* it's not a problem though */
- if (!(err = dma_set_mask(&pdev->dev, DMA_BIT_MASK(64))) &&
- !(err = dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(64)))) {
- pci_using_dac = 1;
- } else {
- if ((err = dma_set_mask(&pdev->dev, DMA_BIT_MASK(32))) ||
- (err = dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(32)))) {
- pr_err("No usable DMA configuration, aborting\n");
- goto err_dma;
- }
- pci_using_dac = 0;
+ err = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64));
+ if (err) {
+ pr_err("No usable DMA configuration, aborting\n");
+ goto err_dma;
}
err = pci_request_regions(pdev, BDX_DRV_NAME);
@@ -1982,16 +1976,14 @@ bdx_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
/* these fields are used for info purposes only
* so we can have them same for all ports of the board */
ndev->if_port = port;
- ndev->features = NETIF_F_IP_CSUM | NETIF_F_SG | NETIF_F_TSO
- | NETIF_F_HW_VLAN_CTAG_TX | NETIF_F_HW_VLAN_CTAG_RX |
- NETIF_F_HW_VLAN_CTAG_FILTER | NETIF_F_RXCSUM
- ;
+ ndev->features = NETIF_F_IP_CSUM | NETIF_F_SG | NETIF_F_TSO |
+ NETIF_F_HW_VLAN_CTAG_TX | NETIF_F_HW_VLAN_CTAG_RX |
+ NETIF_F_HW_VLAN_CTAG_FILTER | NETIF_F_RXCSUM |
+ NETIF_F_HIGHDMA;
+
ndev->hw_features = NETIF_F_IP_CSUM | NETIF_F_SG |
NETIF_F_TSO | NETIF_F_HW_VLAN_CTAG_TX;
- if (pci_using_dac)
- ndev->features |= NETIF_F_HIGHDMA;
-
/************** priv ****************/
priv = nic->priv[port] = netdev_priv(ndev);
@@ -2245,9 +2237,13 @@ static inline int bdx_tx_fifo_size_to_packets(int tx_size)
* bdx_get_ringparam - report ring sizes
* @netdev
* @ring
+ * @kernel_ring
+ * @extack
*/
static void
-bdx_get_ringparam(struct net_device *netdev, struct ethtool_ringparam *ring)
+bdx_get_ringparam(struct net_device *netdev, struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct bdx_priv *priv = netdev_priv(netdev);
@@ -2262,9 +2258,13 @@ bdx_get_ringparam(struct net_device *netdev, struct ethtool_ringparam *ring)
* bdx_set_ringparam - set ring sizes
* @netdev
* @ring
+ * @kernel_ring
+ * @extack
*/
static int
-bdx_set_ringparam(struct net_device *netdev, struct ethtool_ringparam *ring)
+bdx_set_ringparam(struct net_device *netdev, struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct bdx_priv *priv = netdev_priv(netdev);
int rx_size = 0;
diff --git a/drivers/net/ethernet/ti/am65-cpsw-ethtool.c b/drivers/net/ethernet/ti/am65-cpsw-ethtool.c
index b05de9b..d45b6bb 100644
--- a/drivers/net/ethernet/ti/am65-cpsw-ethtool.c
+++ b/drivers/net/ethernet/ti/am65-cpsw-ethtool.c
@@ -453,8 +453,11 @@ static int am65_cpsw_set_channels(struct net_device *ndev,
return am65_cpsw_nuss_update_tx_chns(common, chs->tx_count);
}
-static void am65_cpsw_get_ringparam(struct net_device *ndev,
- struct ethtool_ringparam *ering)
+static void
+am65_cpsw_get_ringparam(struct net_device *ndev,
+ struct ethtool_ringparam *ering,
+ struct kernel_ethtool_ringparam *kernel_ering,
+ struct netlink_ext_ack *extack)
{
struct am65_cpsw_common *common = am65_ndev_to_common(ndev);
diff --git a/drivers/net/ethernet/ti/am65-cpsw-nuss.c b/drivers/net/ethernet/ti/am65-cpsw-nuss.c
index ffbbda8..8251d7e 100644
--- a/drivers/net/ethernet/ti/am65-cpsw-nuss.c
+++ b/drivers/net/ethernet/ti/am65-cpsw-nuss.c
@@ -345,7 +345,7 @@ static void am65_cpsw_nuss_ndo_host_tx_timeout(struct net_device *ndev,
netif_txq = netdev_get_tx_queue(ndev, txqueue);
tx_chn = &common->tx_chns[txqueue];
- trans_start = netif_txq->trans_start;
+ trans_start = READ_ONCE(netif_txq->trans_start);
netdev_err(ndev, "txq:%d DRV_XOFF:%d tmo:%u dql_avail:%d free_desc:%zu\n",
txqueue,
diff --git a/drivers/net/ethernet/ti/cpmac.c b/drivers/net/ethernet/ti/cpmac.c
index 7449436..bef5e68 100644
--- a/drivers/net/ethernet/ti/cpmac.c
+++ b/drivers/net/ethernet/ti/cpmac.c
@@ -817,7 +817,9 @@ static void cpmac_tx_timeout(struct net_device *dev, unsigned int txqueue)
}
static void cpmac_get_ringparam(struct net_device *dev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct cpmac_priv *priv = netdev_priv(dev);
@@ -833,7 +835,9 @@ static void cpmac_get_ringparam(struct net_device *dev,
}
static int cpmac_set_ringparam(struct net_device *dev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct cpmac_priv *priv = netdev_priv(dev);
diff --git a/drivers/net/ethernet/ti/cpsw_ethtool.c b/drivers/net/ethernet/ti/cpsw_ethtool.c
index 158c8d3..aa42141 100644
--- a/drivers/net/ethernet/ti/cpsw_ethtool.c
+++ b/drivers/net/ethernet/ti/cpsw_ethtool.c
@@ -658,7 +658,9 @@ int cpsw_set_channels_common(struct net_device *ndev,
}
void cpsw_get_ringparam(struct net_device *ndev,
- struct ethtool_ringparam *ering)
+ struct ethtool_ringparam *ering,
+ struct kernel_ethtool_ringparam *kernel_ering,
+ struct netlink_ext_ack *extack)
{
struct cpsw_priv *priv = netdev_priv(ndev);
struct cpsw_common *cpsw = priv->cpsw;
@@ -671,7 +673,9 @@ void cpsw_get_ringparam(struct net_device *ndev,
}
int cpsw_set_ringparam(struct net_device *ndev,
- struct ethtool_ringparam *ering)
+ struct ethtool_ringparam *ering,
+ struct kernel_ethtool_ringparam *kernel_ering,
+ struct netlink_ext_ack *extack)
{
struct cpsw_common *cpsw = ndev_to_cpsw(ndev);
int descs_num, ret;
diff --git a/drivers/net/ethernet/ti/cpsw_priv.c b/drivers/net/ethernet/ti/cpsw_priv.c
index ecc2a6b..3537502 100644
--- a/drivers/net/ethernet/ti/cpsw_priv.c
+++ b/drivers/net/ethernet/ti/cpsw_priv.c
@@ -626,10 +626,6 @@ static int cpsw_hwtstamp_set(struct net_device *dev, struct ifreq *ifr)
if (copy_from_user(&cfg, ifr->ifr_data, sizeof(cfg)))
return -EFAULT;
- /* reserved for future extensions */
- if (cfg.flags)
- return -EINVAL;
-
if (cfg.tx_type != HWTSTAMP_TX_OFF && cfg.tx_type != HWTSTAMP_TX_ON)
return -ERANGE;
@@ -710,20 +706,26 @@ int cpsw_ndo_ioctl(struct net_device *dev, struct ifreq *req, int cmd)
struct cpsw_priv *priv = netdev_priv(dev);
struct cpsw_common *cpsw = priv->cpsw;
int slave_no = cpsw_slave_index(cpsw, priv);
+ struct phy_device *phy;
if (!netif_running(dev))
return -EINVAL;
- switch (cmd) {
- case SIOCSHWTSTAMP:
- return cpsw_hwtstamp_set(dev, req);
- case SIOCGHWTSTAMP:
- return cpsw_hwtstamp_get(dev, req);
+ phy = cpsw->slaves[slave_no].phy;
+
+ if (!phy_has_hwtstamp(phy)) {
+ switch (cmd) {
+ case SIOCSHWTSTAMP:
+ return cpsw_hwtstamp_set(dev, req);
+ case SIOCGHWTSTAMP:
+ return cpsw_hwtstamp_get(dev, req);
+ }
}
- if (!cpsw->slaves[slave_no].phy)
- return -EOPNOTSUPP;
- return phy_mii_ioctl(cpsw->slaves[slave_no].phy, req, cmd);
+ if (phy)
+ return phy_mii_ioctl(phy, req, cmd);
+
+ return -EOPNOTSUPP;
}
int cpsw_ndo_set_tx_maxrate(struct net_device *ndev, int queue, u32 rate)
@@ -1360,7 +1362,7 @@ int cpsw_run_xdp(struct cpsw_priv *priv, int ch, struct xdp_buff *xdp,
xdp_do_flush_map();
break;
default:
- bpf_warn_invalid_xdp_action(act);
+ bpf_warn_invalid_xdp_action(ndev, prog, act);
fallthrough;
case XDP_ABORTED:
trace_xdp_exception(ndev, prog, act);
diff --git a/drivers/net/ethernet/ti/cpsw_priv.h b/drivers/net/ethernet/ti/cpsw_priv.h
index 435668e..7455597 100644
--- a/drivers/net/ethernet/ti/cpsw_priv.h
+++ b/drivers/net/ethernet/ti/cpsw_priv.h
@@ -6,6 +6,8 @@
#ifndef DRIVERS_NET_ETHERNET_TI_CPSW_PRIV_H_
#define DRIVERS_NET_ETHERNET_TI_CPSW_PRIV_H_
+#include <uapi/linux/bpf.h>
+
#include "davinci_cpdma.h"
#define CPSW_DEBUG (NETIF_MSG_HW | NETIF_MSG_WOL | \
@@ -491,9 +493,13 @@ int cpsw_get_eee(struct net_device *ndev, struct ethtool_eee *edata);
int cpsw_set_eee(struct net_device *ndev, struct ethtool_eee *edata);
int cpsw_nway_reset(struct net_device *ndev);
void cpsw_get_ringparam(struct net_device *ndev,
- struct ethtool_ringparam *ering);
+ struct ethtool_ringparam *ering,
+ struct kernel_ethtool_ringparam *kernel_ering,
+ struct netlink_ext_ack *extack);
int cpsw_set_ringparam(struct net_device *ndev,
- struct ethtool_ringparam *ering);
+ struct ethtool_ringparam *ering,
+ struct kernel_ethtool_ringparam *kernel_ering,
+ struct netlink_ext_ack *extack);
int cpsw_set_channels_common(struct net_device *ndev,
struct ethtool_channels *chs,
cpdma_handler_fn rx_handler);
diff --git a/drivers/net/ethernet/ti/davinci_emac.c b/drivers/net/ethernet/ti/davinci_emac.c
index d55f061..31df326 100644
--- a/drivers/net/ethernet/ti/davinci_emac.c
+++ b/drivers/net/ethernet/ti/davinci_emac.c
@@ -1454,23 +1454,33 @@ static int emac_dev_open(struct net_device *ndev)
}
/* Request IRQ */
- while ((res = platform_get_resource(priv->pdev, IORESOURCE_IRQ,
- res_num))) {
- for (irq_num = res->start; irq_num <= res->end; irq_num++) {
- if (request_irq(irq_num, emac_irq, 0, ndev->name,
- ndev)) {
- dev_err(emac_dev,
- "DaVinci EMAC: request_irq() failed\n");
- ret = -EBUSY;
+ if (dev_of_node(&priv->pdev->dev)) {
+ while ((ret = platform_get_irq_optional(priv->pdev, res_num)) != -ENXIO) {
+ if (ret < 0)
+ goto rollback;
+ ret = request_irq(ret, emac_irq, 0, ndev->name, ndev);
+ if (ret) {
+ dev_err(emac_dev, "DaVinci EMAC: request_irq() failed\n");
goto rollback;
}
+ res_num++;
}
- res_num++;
+ } else {
+ while ((res = platform_get_resource(priv->pdev, IORESOURCE_IRQ, res_num))) {
+ for (irq_num = res->start; irq_num <= res->end; irq_num++) {
+ ret = request_irq(irq_num, emac_irq, 0, ndev->name, ndev);
+ if (ret) {
+ dev_err(emac_dev, "DaVinci EMAC: request_irq() failed\n");
+ goto rollback;
+ }
+ }
+ res_num++;
+ }
+ /* prepare counters for rollback in case of an error */
+ res_num--;
+ irq_num--;
}
- /* prepare counters for rollback in case of an error */
- res_num--;
- irq_num--;
/* Start/Enable EMAC hardware */
emac_hw_enable(priv);
@@ -1554,16 +1564,24 @@ static int emac_dev_open(struct net_device *ndev)
napi_disable(&priv->napi);
rollback:
- for (q = res_num; q >= 0; q--) {
- res = platform_get_resource(priv->pdev, IORESOURCE_IRQ, q);
- /* at the first iteration, irq_num is already set to the
- * right value
- */
- if (q != res_num)
- irq_num = res->end;
+ if (dev_of_node(&priv->pdev->dev)) {
+ for (q = res_num - 1; q >= 0; q--) {
+ irq_num = platform_get_irq(priv->pdev, q);
+ if (irq_num > 0)
+ free_irq(irq_num, ndev);
+ }
+ } else {
+ for (q = res_num; q >= 0; q--) {
+ res = platform_get_resource(priv->pdev, IORESOURCE_IRQ, q);
+ /* at the first iteration, irq_num is already set to the
+ * right value
+ */
+ if (q != res_num)
+ irq_num = res->end;
- for (m = irq_num; m >= res->start; m--)
- free_irq(m, ndev);
+ for (m = irq_num; m >= res->start; m--)
+ free_irq(m, ndev);
+ }
}
cpdma_ctlr_stop(priv->dma);
pm_runtime_put(&priv->pdev->dev);
@@ -1899,13 +1917,10 @@ static int davinci_emac_probe(struct platform_device *pdev)
goto err_free_txchan;
}
- res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
- if (!res) {
- dev_err(&pdev->dev, "error getting irq res\n");
- rc = -ENOENT;
+ rc = platform_get_irq(pdev, 0);
+ if (rc < 0)
goto err_free_rxchan;
- }
- ndev->irq = res->start;
+ ndev->irq = rc;
rc = davinci_emac_try_get_mac(pdev, res_ctrl ? 0 : 1, priv->mac_addr);
if (!rc)
diff --git a/drivers/net/ethernet/ti/netcp_ethss.c b/drivers/net/ethernet/ti/netcp_ethss.c
index 33c1592..751fb0b 100644
--- a/drivers/net/ethernet/ti/netcp_ethss.c
+++ b/drivers/net/ethernet/ti/netcp_ethss.c
@@ -2654,10 +2654,6 @@ static int gbe_hwtstamp_set(struct gbe_intf *gbe_intf, struct ifreq *ifr)
if (copy_from_user(&cfg, ifr->ifr_data, sizeof(cfg)))
return -EFAULT;
- /* reserved for future extensions */
- if (cfg.flags)
- return -EINVAL;
-
switch (cfg.tx_type) {
case HWTSTAMP_TX_OFF:
gbe_dev->tx_ts_enabled = 0;
diff --git a/drivers/net/ethernet/toshiba/spider_net.c b/drivers/net/ethernet/toshiba/spider_net.c
index f50f9a4..f47b835 100644
--- a/drivers/net/ethernet/toshiba/spider_net.c
+++ b/drivers/net/ethernet/toshiba/spider_net.c
@@ -595,24 +595,24 @@ spider_net_set_multi(struct net_device *netdev)
int i;
u32 reg;
struct spider_net_card *card = netdev_priv(netdev);
- DECLARE_BITMAP(bitmask, SPIDER_NET_MULTICAST_HASHES) = {};
+ DECLARE_BITMAP(bitmask, SPIDER_NET_MULTICAST_HASHES);
spider_net_set_promisc(card);
if (netdev->flags & IFF_ALLMULTI) {
- for (i = 0; i < SPIDER_NET_MULTICAST_HASHES; i++) {
- set_bit(i, bitmask);
- }
+ bitmap_fill(bitmask, SPIDER_NET_MULTICAST_HASHES);
goto write_hash;
}
+ bitmap_zero(bitmask, SPIDER_NET_MULTICAST_HASHES);
+
/* well, we know, what the broadcast hash value is: it's xfd
hash = spider_net_get_multicast_hash(netdev, netdev->broadcast); */
- set_bit(0xfd, bitmask);
+ __set_bit(0xfd, bitmask);
netdev_for_each_mc_addr(ha, netdev) {
hash = spider_net_get_multicast_hash(netdev, ha->addr);
- set_bit(hash, bitmask);
+ __set_bit(hash, bitmask);
}
write_hash:
diff --git a/drivers/net/ethernet/toshiba/spider_net_ethtool.c b/drivers/net/ethernet/toshiba/spider_net_ethtool.c
index 54f655a..93110db 100644
--- a/drivers/net/ethernet/toshiba/spider_net_ethtool.c
+++ b/drivers/net/ethernet/toshiba/spider_net_ethtool.c
@@ -110,7 +110,9 @@ spider_net_ethtool_nway_reset(struct net_device *netdev)
static void
spider_net_ethtool_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ering)
+ struct ethtool_ringparam *ering,
+ struct kernel_ethtool_ringparam *kernel_ering,
+ struct netlink_ext_ack *extack)
{
struct spider_net_card *card = netdev_priv(netdev);
diff --git a/drivers/net/ethernet/vertexcom/Kconfig b/drivers/net/ethernet/vertexcom/Kconfig
new file mode 100644
index 0000000..6e2cf06
--- /dev/null
+++ b/drivers/net/ethernet/vertexcom/Kconfig
@@ -0,0 +1,25 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Vertexcom network device configuration
+#
+
+config NET_VENDOR_VERTEXCOM
+ bool "Vertexcom devices"
+ default n
+ help
+ If you have a network (Ethernet) card belonging to this class, say Y.
+
+ Note that the answer to this question doesn't directly affect the
+ kernel: saying N will just cause the configurator to skip all
+ the questions about Vertexcom cards. If you say Y, you will be asked
+ for your specific card in the following questions.
+
+if NET_VENDOR_VERTEXCOM
+
+config MSE102X
+ tristate "Vertexcom MSE102x SPI"
+ depends on SPI
+ help
+ SPI driver for Vertexcom MSE102x SPI attached network chip.
+
+endif # NET_VENDOR_VERTEXCOM
diff --git a/drivers/net/ethernet/vertexcom/Makefile b/drivers/net/ethernet/vertexcom/Makefile
new file mode 100644
index 0000000..f8b12e3
--- /dev/null
+++ b/drivers/net/ethernet/vertexcom/Makefile
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Makefile for the Vertexcom network device drivers.
+#
+
+obj-$(CONFIG_MSE102X) += mse102x.o
diff --git a/drivers/net/ethernet/vertexcom/mse102x.c b/drivers/net/ethernet/vertexcom/mse102x.c
new file mode 100644
index 0000000..89a3178
--- /dev/null
+++ b/drivers/net/ethernet/vertexcom/mse102x.c
@@ -0,0 +1,769 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright (C) 2021 in-tech smart charging GmbH
+ *
+ * driver is based on micrel/ks8851_spi.c
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/interrupt.h>
+#include <linux/module.h>
+#include <linux/kernel.h>
+#include <linux/netdevice.h>
+#include <linux/etherdevice.h>
+#include <linux/ethtool.h>
+#include <linux/cache.h>
+#include <linux/debugfs.h>
+#include <linux/seq_file.h>
+
+#include <linux/spi/spi.h>
+#include <linux/of_net.h>
+
+#define MSG_DEFAULT (NETIF_MSG_DRV | NETIF_MSG_PROBE | NETIF_MSG_LINK | \
+ NETIF_MSG_TIMER)
+
+#define DRV_NAME "mse102x"
+
+#define DET_CMD 0x0001
+#define DET_SOF 0x0002
+#define DET_DFT 0x55AA
+
+#define CMD_SHIFT 12
+#define CMD_RTS (0x1 << CMD_SHIFT)
+#define CMD_CTR (0x2 << CMD_SHIFT)
+
+#define CMD_MASK GENMASK(15, CMD_SHIFT)
+#define LEN_MASK GENMASK(CMD_SHIFT - 1, 0)
+
+#define DET_CMD_LEN 4
+#define DET_SOF_LEN 2
+#define DET_DFT_LEN 2
+
+#define MIN_FREQ_HZ 6000000
+#define MAX_FREQ_HZ 7142857
+
+struct mse102x_stats {
+ u64 xfer_err;
+ u64 invalid_cmd;
+ u64 invalid_ctr;
+ u64 invalid_dft;
+ u64 invalid_len;
+ u64 invalid_rts;
+ u64 invalid_sof;
+ u64 tx_timeout;
+};
+
+static const char mse102x_gstrings_stats[][ETH_GSTRING_LEN] = {
+ "SPI transfer errors",
+ "Invalid command",
+ "Invalid CTR",
+ "Invalid DFT",
+ "Invalid frame length",
+ "Invalid RTS",
+ "Invalid SOF",
+ "TX timeout",
+};
+
+struct mse102x_net {
+ struct net_device *ndev;
+
+ u8 rxd[8];
+ u8 txd[8];
+
+ u32 msg_enable ____cacheline_aligned;
+
+ struct sk_buff_head txq;
+ struct mse102x_stats stats;
+};
+
+struct mse102x_net_spi {
+ struct mse102x_net mse102x;
+ struct mutex lock; /* Protect SPI frame transfer */
+ struct work_struct tx_work;
+ struct spi_device *spidev;
+ struct spi_message spi_msg;
+ struct spi_transfer spi_xfer;
+
+#ifdef CONFIG_DEBUG_FS
+ struct dentry *device_root;
+#endif
+};
+
+#define to_mse102x_spi(mse) container_of((mse), struct mse102x_net_spi, mse102x)
+
+#ifdef CONFIG_DEBUG_FS
+
+static int mse102x_info_show(struct seq_file *s, void *what)
+{
+ struct mse102x_net_spi *mses = s->private;
+
+ seq_printf(s, "TX ring size : %u\n",
+ skb_queue_len(&mses->mse102x.txq));
+
+ seq_printf(s, "IRQ : %d\n",
+ mses->spidev->irq);
+
+ seq_printf(s, "SPI effective speed : %lu\n",
+ (unsigned long)mses->spi_xfer.effective_speed_hz);
+ seq_printf(s, "SPI mode : %x\n",
+ mses->spidev->mode);
+
+ return 0;
+}
+DEFINE_SHOW_ATTRIBUTE(mse102x_info);
+
+static void mse102x_init_device_debugfs(struct mse102x_net_spi *mses)
+{
+ mses->device_root = debugfs_create_dir(dev_name(&mses->mse102x.ndev->dev),
+ NULL);
+
+ debugfs_create_file("info", S_IFREG | 0444, mses->device_root, mses,
+ &mse102x_info_fops);
+}
+
+static void mse102x_remove_device_debugfs(struct mse102x_net_spi *mses)
+{
+ debugfs_remove_recursive(mses->device_root);
+}
+
+#else /* CONFIG_DEBUG_FS */
+
+static void mse102x_init_device_debugfs(struct mse102x_net_spi *mses)
+{
+}
+
+static void mse102x_remove_device_debugfs(struct mse102x_net_spi *mses)
+{
+}
+
+#endif
+
+/* SPI register read/write calls.
+ *
+ * All these calls issue SPI transactions to access the chip's registers. They
+ * all require that the necessary lock is held to prevent accesses when the
+ * chip is busy transferring packet data.
+ */
+
+static void mse102x_tx_cmd_spi(struct mse102x_net *mse, u16 cmd)
+{
+ struct mse102x_net_spi *mses = to_mse102x_spi(mse);
+ struct spi_transfer *xfer = &mses->spi_xfer;
+ struct spi_message *msg = &mses->spi_msg;
+ __be16 txb[2];
+ int ret;
+
+ txb[0] = cpu_to_be16(DET_CMD);
+ txb[1] = cpu_to_be16(cmd);
+
+ xfer->tx_buf = txb;
+ xfer->rx_buf = NULL;
+ xfer->len = DET_CMD_LEN;
+
+ ret = spi_sync(mses->spidev, msg);
+ if (ret < 0) {
+ netdev_err(mse->ndev, "%s: spi_sync() failed: %d\n",
+ __func__, ret);
+ mse->stats.xfer_err++;
+ }
+}
+
+static int mse102x_rx_cmd_spi(struct mse102x_net *mse, u8 *rxb)
+{
+ struct mse102x_net_spi *mses = to_mse102x_spi(mse);
+ struct spi_transfer *xfer = &mses->spi_xfer;
+ struct spi_message *msg = &mses->spi_msg;
+ __be16 *txb = (__be16 *)mse->txd;
+ __be16 *cmd = (__be16 *)mse->rxd;
+ u8 *trx = mse->rxd;
+ int ret;
+
+ txb[0] = 0;
+ txb[1] = 0;
+
+ xfer->tx_buf = txb;
+ xfer->rx_buf = trx;
+ xfer->len = DET_CMD_LEN;
+
+ ret = spi_sync(mses->spidev, msg);
+ if (ret < 0) {
+ netdev_err(mse->ndev, "%s: spi_sync() failed: %d\n",
+ __func__, ret);
+ mse->stats.xfer_err++;
+ } else if (*cmd != cpu_to_be16(DET_CMD)) {
+ net_dbg_ratelimited("%s: Unexpected response (0x%04x)\n",
+ __func__, *cmd);
+ mse->stats.invalid_cmd++;
+ ret = -EIO;
+ } else {
+ memcpy(rxb, trx + 2, 2);
+ }
+
+ return ret;
+}
+
+static inline void mse102x_push_header(struct sk_buff *skb)
+{
+ __be16 *header = skb_push(skb, DET_SOF_LEN);
+
+ *header = cpu_to_be16(DET_SOF);
+}
+
+static inline void mse102x_put_footer(struct sk_buff *skb)
+{
+ __be16 *footer = skb_put(skb, DET_DFT_LEN);
+
+ *footer = cpu_to_be16(DET_DFT);
+}
+
+static int mse102x_tx_frame_spi(struct mse102x_net *mse, struct sk_buff *txp,
+ unsigned int pad)
+{
+ struct mse102x_net_spi *mses = to_mse102x_spi(mse);
+ struct spi_transfer *xfer = &mses->spi_xfer;
+ struct spi_message *msg = &mses->spi_msg;
+ struct sk_buff *tskb;
+ int ret;
+
+ netif_dbg(mse, tx_queued, mse->ndev, "%s: skb %p, %d@%p\n",
+ __func__, txp, txp->len, txp->data);
+
+ if ((skb_headroom(txp) < DET_SOF_LEN) ||
+ (skb_tailroom(txp) < DET_DFT_LEN + pad)) {
+ tskb = skb_copy_expand(txp, DET_SOF_LEN, DET_DFT_LEN + pad,
+ GFP_KERNEL);
+ if (!tskb)
+ return -ENOMEM;
+
+ dev_kfree_skb(txp);
+ txp = tskb;
+ }
+
+ mse102x_push_header(txp);
+
+ if (pad)
+ skb_put_zero(txp, pad);
+
+ mse102x_put_footer(txp);
+
+ xfer->tx_buf = txp->data;
+ xfer->rx_buf = NULL;
+ xfer->len = txp->len;
+
+ ret = spi_sync(mses->spidev, msg);
+ if (ret < 0) {
+ netdev_err(mse->ndev, "%s: spi_sync() failed: %d\n",
+ __func__, ret);
+ mse->stats.xfer_err++;
+ }
+
+ return ret;
+}
+
+static int mse102x_rx_frame_spi(struct mse102x_net *mse, u8 *buff,
+ unsigned int frame_len)
+{
+ struct mse102x_net_spi *mses = to_mse102x_spi(mse);
+ struct spi_transfer *xfer = &mses->spi_xfer;
+ struct spi_message *msg = &mses->spi_msg;
+ __be16 *sof = (__be16 *)buff;
+ __be16 *dft = (__be16 *)(buff + DET_SOF_LEN + frame_len);
+ int ret;
+
+ xfer->rx_buf = buff;
+ xfer->tx_buf = NULL;
+ xfer->len = DET_SOF_LEN + frame_len + DET_DFT_LEN;
+
+ ret = spi_sync(mses->spidev, msg);
+ if (ret < 0) {
+ netdev_err(mse->ndev, "%s: spi_sync() failed: %d\n",
+ __func__, ret);
+ mse->stats.xfer_err++;
+ } else if (*sof != cpu_to_be16(DET_SOF)) {
+ netdev_dbg(mse->ndev, "%s: SPI start of frame is invalid (0x%04x)\n",
+ __func__, *sof);
+ mse->stats.invalid_sof++;
+ ret = -EIO;
+ } else if (*dft != cpu_to_be16(DET_DFT)) {
+ netdev_dbg(mse->ndev, "%s: SPI frame tail is invalid (0x%04x)\n",
+ __func__, *dft);
+ mse->stats.invalid_dft++;
+ ret = -EIO;
+ }
+
+ return ret;
+}
+
+static void mse102x_dump_packet(const char *msg, int len, const char *data)
+{
+ printk(KERN_DEBUG ": %s - packet len:%d\n", msg, len);
+ print_hex_dump(KERN_DEBUG, "pk data: ", DUMP_PREFIX_OFFSET, 16, 1,
+ data, len, true);
+}
+
+static void mse102x_rx_pkt_spi(struct mse102x_net *mse)
+{
+ struct sk_buff *skb;
+ unsigned int rxalign;
+ unsigned int rxlen;
+ __be16 rx = 0;
+ u16 cmd_resp;
+ u8 *rxpkt;
+ int ret;
+
+ mse102x_tx_cmd_spi(mse, CMD_CTR);
+ ret = mse102x_rx_cmd_spi(mse, (u8 *)&rx);
+ cmd_resp = be16_to_cpu(rx);
+
+ if (ret || ((cmd_resp & CMD_MASK) != CMD_RTS)) {
+ usleep_range(50, 100);
+
+ mse102x_tx_cmd_spi(mse, CMD_CTR);
+ ret = mse102x_rx_cmd_spi(mse, (u8 *)&rx);
+ if (ret)
+ return;
+
+ cmd_resp = be16_to_cpu(rx);
+ if ((cmd_resp & CMD_MASK) != CMD_RTS) {
+ net_dbg_ratelimited("%s: Unexpected response (0x%04x)\n",
+ __func__, cmd_resp);
+ mse->stats.invalid_rts++;
+ return;
+ }
+
+ net_dbg_ratelimited("%s: Unexpected response to first CMD\n",
+ __func__);
+ }
+
+ rxlen = cmd_resp & LEN_MASK;
+ if (!rxlen) {
+ net_dbg_ratelimited("%s: No frame length defined\n", __func__);
+ mse->stats.invalid_len++;
+ return;
+ }
+
+ rxalign = ALIGN(rxlen + DET_SOF_LEN + DET_DFT_LEN, 4);
+ skb = netdev_alloc_skb_ip_align(mse->ndev, rxalign);
+ if (!skb)
+ return;
+
+ /* 2 bytes Start of frame (before ethernet header)
+ * 2 bytes Data frame tail (after ethernet frame)
+ * They are copied, but ignored.
+ */
+ rxpkt = skb_put(skb, rxlen) - DET_SOF_LEN;
+ if (mse102x_rx_frame_spi(mse, rxpkt, rxlen)) {
+ mse->ndev->stats.rx_errors++;
+ dev_kfree_skb(skb);
+ return;
+ }
+
+ if (netif_msg_pktdata(mse))
+ mse102x_dump_packet(__func__, skb->len, skb->data);
+
+ skb->protocol = eth_type_trans(skb, mse->ndev);
+ netif_rx_ni(skb);
+
+ mse->ndev->stats.rx_packets++;
+ mse->ndev->stats.rx_bytes += rxlen;
+}
+
+static int mse102x_tx_pkt_spi(struct mse102x_net *mse, struct sk_buff *txb,
+ unsigned long work_timeout)
+{
+ unsigned int pad = 0;
+ __be16 rx = 0;
+ u16 cmd_resp;
+ int ret;
+ bool first = true;
+
+ if (txb->len < 60)
+ pad = 60 - txb->len;
+
+ while (1) {
+ mse102x_tx_cmd_spi(mse, CMD_RTS | (txb->len + pad));
+ ret = mse102x_rx_cmd_spi(mse, (u8 *)&rx);
+ cmd_resp = be16_to_cpu(rx);
+
+ if (!ret) {
+ /* ready to send frame ? */
+ if (cmd_resp == CMD_CTR)
+ break;
+
+ net_dbg_ratelimited("%s: Unexpected response (0x%04x)\n",
+ __func__, cmd_resp);
+ mse->stats.invalid_ctr++;
+ }
+
+ /* It's not predictable how long / many retries it takes to
+ * send at least one packet, so TX timeouts are possible.
+ * That's the reason why the netdev watchdog is not used here.
+ */
+ if (time_after(jiffies, work_timeout))
+ return -ETIMEDOUT;
+
+ if (first) {
+ /* throttle at first issue */
+ netif_stop_queue(mse->ndev);
+ /* fast retry */
+ usleep_range(50, 100);
+ first = false;
+ } else {
+ msleep(20);
+ }
+ }
+
+ ret = mse102x_tx_frame_spi(mse, txb, pad);
+ if (ret)
+ net_dbg_ratelimited("%s: Failed to send (%d), drop frame\n",
+ __func__, ret);
+
+ return ret;
+}
+
+#define TX_QUEUE_MAX 10
+
+static void mse102x_tx_work(struct work_struct *work)
+{
+ /* Make sure timeout is sufficient to transfer TX_QUEUE_MAX frames */
+ unsigned long work_timeout = jiffies + msecs_to_jiffies(1000);
+ struct mse102x_net_spi *mses;
+ struct mse102x_net *mse;
+ struct sk_buff *txb;
+ int ret = 0;
+
+ mses = container_of(work, struct mse102x_net_spi, tx_work);
+ mse = &mses->mse102x;
+
+ while ((txb = skb_dequeue(&mse->txq))) {
+ mutex_lock(&mses->lock);
+ ret = mse102x_tx_pkt_spi(mse, txb, work_timeout);
+ mutex_unlock(&mses->lock);
+ if (ret) {
+ mse->ndev->stats.tx_dropped++;
+ } else {
+ mse->ndev->stats.tx_bytes += txb->len;
+ mse->ndev->stats.tx_packets++;
+ }
+
+ dev_kfree_skb(txb);
+ }
+
+ if (ret == -ETIMEDOUT) {
+ if (netif_msg_timer(mse))
+ netdev_err(mse->ndev, "tx work timeout\n");
+
+ mse->stats.tx_timeout++;
+ }
+
+ netif_wake_queue(mse->ndev);
+}
+
+static netdev_tx_t mse102x_start_xmit_spi(struct sk_buff *skb,
+ struct net_device *ndev)
+{
+ struct mse102x_net *mse = netdev_priv(ndev);
+ struct mse102x_net_spi *mses = to_mse102x_spi(mse);
+
+ netif_dbg(mse, tx_queued, ndev,
+ "%s: skb %p, %d@%p\n", __func__, skb, skb->len, skb->data);
+
+ skb_queue_tail(&mse->txq, skb);
+
+ if (skb_queue_len(&mse->txq) >= TX_QUEUE_MAX)
+ netif_stop_queue(ndev);
+
+ schedule_work(&mses->tx_work);
+
+ return NETDEV_TX_OK;
+}
+
+static void mse102x_init_mac(struct mse102x_net *mse, struct device_node *np)
+{
+ struct net_device *ndev = mse->ndev;
+ int ret = of_get_ethdev_address(np, ndev);
+
+ if (ret) {
+ eth_hw_addr_random(ndev);
+ netdev_err(ndev, "Using random MAC address: %pM\n",
+ ndev->dev_addr);
+ }
+}
+
+/* Assumption: this is called for every incoming packet */
+static irqreturn_t mse102x_irq(int irq, void *_mse)
+{
+ struct mse102x_net *mse = _mse;
+ struct mse102x_net_spi *mses = to_mse102x_spi(mse);
+
+ mutex_lock(&mses->lock);
+ mse102x_rx_pkt_spi(mse);
+ mutex_unlock(&mses->lock);
+
+ return IRQ_HANDLED;
+}
+
+static int mse102x_net_open(struct net_device *ndev)
+{
+ struct mse102x_net *mse = netdev_priv(ndev);
+ int ret;
+
+ ret = request_threaded_irq(ndev->irq, NULL, mse102x_irq, IRQF_ONESHOT,
+ ndev->name, mse);
+ if (ret < 0) {
+ netdev_err(ndev, "Failed to get irq: %d\n", ret);
+ return ret;
+ }
+
+ netif_dbg(mse, ifup, ndev, "opening\n");
+
+ netif_start_queue(ndev);
+
+ netif_carrier_on(ndev);
+
+ netif_dbg(mse, ifup, ndev, "network device up\n");
+
+ return 0;
+}
+
+static int mse102x_net_stop(struct net_device *ndev)
+{
+ struct mse102x_net *mse = netdev_priv(ndev);
+ struct mse102x_net_spi *mses = to_mse102x_spi(mse);
+
+ netif_info(mse, ifdown, ndev, "shutting down\n");
+
+ netif_carrier_off(mse->ndev);
+
+ /* stop any outstanding work */
+ flush_work(&mses->tx_work);
+
+ netif_stop_queue(ndev);
+
+ skb_queue_purge(&mse->txq);
+
+ free_irq(ndev->irq, mse);
+
+ return 0;
+}
+
+static const struct net_device_ops mse102x_netdev_ops = {
+ .ndo_open = mse102x_net_open,
+ .ndo_stop = mse102x_net_stop,
+ .ndo_start_xmit = mse102x_start_xmit_spi,
+ .ndo_set_mac_address = eth_mac_addr,
+ .ndo_validate_addr = eth_validate_addr,
+};
+
+/* ethtool support */
+
+static void mse102x_get_drvinfo(struct net_device *ndev,
+ struct ethtool_drvinfo *di)
+{
+ strscpy(di->driver, DRV_NAME, sizeof(di->driver));
+ strscpy(di->bus_info, dev_name(ndev->dev.parent), sizeof(di->bus_info));
+}
+
+static u32 mse102x_get_msglevel(struct net_device *ndev)
+{
+ struct mse102x_net *mse = netdev_priv(ndev);
+
+ return mse->msg_enable;
+}
+
+static void mse102x_set_msglevel(struct net_device *ndev, u32 to)
+{
+ struct mse102x_net *mse = netdev_priv(ndev);
+
+ mse->msg_enable = to;
+}
+
+static void mse102x_get_ethtool_stats(struct net_device *ndev,
+ struct ethtool_stats *estats, u64 *data)
+{
+ struct mse102x_net *mse = netdev_priv(ndev);
+ struct mse102x_stats *st = &mse->stats;
+
+ memcpy(data, st, ARRAY_SIZE(mse102x_gstrings_stats) * sizeof(u64));
+}
+
+static void mse102x_get_strings(struct net_device *ndev, u32 stringset, u8 *buf)
+{
+ switch (stringset) {
+ case ETH_SS_STATS:
+ memcpy(buf, &mse102x_gstrings_stats,
+ sizeof(mse102x_gstrings_stats));
+ break;
+ default:
+ WARN_ON(1);
+ break;
+ }
+}
+
+static int mse102x_get_sset_count(struct net_device *ndev, int sset)
+{
+ switch (sset) {
+ case ETH_SS_STATS:
+ return ARRAY_SIZE(mse102x_gstrings_stats);
+ default:
+ return -EINVAL;
+ }
+}
+
+static const struct ethtool_ops mse102x_ethtool_ops = {
+ .get_drvinfo = mse102x_get_drvinfo,
+ .get_link = ethtool_op_get_link,
+ .get_msglevel = mse102x_get_msglevel,
+ .set_msglevel = mse102x_set_msglevel,
+ .get_ethtool_stats = mse102x_get_ethtool_stats,
+ .get_strings = mse102x_get_strings,
+ .get_sset_count = mse102x_get_sset_count,
+};
+
+/* driver bus management functions */
+
+#ifdef CONFIG_PM_SLEEP
+
+static int mse102x_suspend(struct device *dev)
+{
+ struct mse102x_net *mse = dev_get_drvdata(dev);
+ struct net_device *ndev = mse->ndev;
+
+ if (netif_running(ndev)) {
+ netif_device_detach(ndev);
+ mse102x_net_stop(ndev);
+ }
+
+ return 0;
+}
+
+static int mse102x_resume(struct device *dev)
+{
+ struct mse102x_net *mse = dev_get_drvdata(dev);
+ struct net_device *ndev = mse->ndev;
+
+ if (netif_running(ndev)) {
+ mse102x_net_open(ndev);
+ netif_device_attach(ndev);
+ }
+
+ return 0;
+}
+#endif
+
+static SIMPLE_DEV_PM_OPS(mse102x_pm_ops, mse102x_suspend, mse102x_resume);
+
+static int mse102x_probe_spi(struct spi_device *spi)
+{
+ struct device *dev = &spi->dev;
+ struct mse102x_net_spi *mses;
+ struct net_device *ndev;
+ struct mse102x_net *mse;
+ int ret;
+
+ spi->bits_per_word = 8;
+ spi->mode |= SPI_MODE_3;
+ /* enforce minimum speed to ensure device functionality */
+ spi->master->min_speed_hz = MIN_FREQ_HZ;
+
+ if (!spi->max_speed_hz)
+ spi->max_speed_hz = MAX_FREQ_HZ;
+
+ if (spi->max_speed_hz < MIN_FREQ_HZ ||
+ spi->max_speed_hz > MAX_FREQ_HZ) {
+ dev_err(&spi->dev, "SPI max frequency out of range (min: %u, max: %u)\n",
+ MIN_FREQ_HZ, MAX_FREQ_HZ);
+ return -EINVAL;
+ }
+
+ ret = spi_setup(spi);
+ if (ret < 0) {
+ dev_err(&spi->dev, "Unable to setup SPI device: %d\n", ret);
+ return ret;
+ }
+
+ ndev = devm_alloc_etherdev(dev, sizeof(struct mse102x_net_spi));
+ if (!ndev)
+ return -ENOMEM;
+
+ ndev->needed_tailroom += ALIGN(DET_DFT_LEN, 4);
+ ndev->needed_headroom += ALIGN(DET_SOF_LEN, 4);
+ ndev->priv_flags &= ~IFF_TX_SKB_SHARING;
+ ndev->tx_queue_len = 100;
+
+ mse = netdev_priv(ndev);
+ mses = to_mse102x_spi(mse);
+
+ mses->spidev = spi;
+ mutex_init(&mses->lock);
+ INIT_WORK(&mses->tx_work, mse102x_tx_work);
+
+ /* initialise pre-made spi transfer messages */
+ spi_message_init(&mses->spi_msg);
+ spi_message_add_tail(&mses->spi_xfer, &mses->spi_msg);
+
+ ndev->irq = spi->irq;
+ mse->ndev = ndev;
+
+ /* set the default message enable */
+ mse->msg_enable = netif_msg_init(-1, MSG_DEFAULT);
+
+ skb_queue_head_init(&mse->txq);
+
+ SET_NETDEV_DEV(ndev, dev);
+
+ dev_set_drvdata(dev, mse);
+
+ netif_carrier_off(mse->ndev);
+ ndev->netdev_ops = &mse102x_netdev_ops;
+ ndev->ethtool_ops = &mse102x_ethtool_ops;
+
+ mse102x_init_mac(mse, dev->of_node);
+
+ ret = register_netdev(ndev);
+ if (ret) {
+ dev_err(dev, "failed to register network device: %d\n", ret);
+ return ret;
+ }
+
+ mse102x_init_device_debugfs(mses);
+
+ return 0;
+}
+
+static int mse102x_remove_spi(struct spi_device *spi)
+{
+ struct mse102x_net *mse = dev_get_drvdata(&spi->dev);
+ struct mse102x_net_spi *mses = to_mse102x_spi(mse);
+
+ if (netif_msg_drv(mse))
+ dev_info(&spi->dev, "remove\n");
+
+ mse102x_remove_device_debugfs(mses);
+ unregister_netdev(mse->ndev);
+
+ return 0;
+}
+
+static const struct of_device_id mse102x_match_table[] = {
+ { .compatible = "vertexcom,mse1021" },
+ { .compatible = "vertexcom,mse1022" },
+ { }
+};
+MODULE_DEVICE_TABLE(of, mse102x_match_table);
+
+static struct spi_driver mse102x_driver = {
+ .driver = {
+ .name = DRV_NAME,
+ .of_match_table = mse102x_match_table,
+ .pm = &mse102x_pm_ops,
+ },
+ .probe = mse102x_probe_spi,
+ .remove = mse102x_remove_spi,
+};
+module_spi_driver(mse102x_driver);
+
+MODULE_DESCRIPTION("MSE102x Network driver");
+MODULE_AUTHOR("Stefan Wahren <stefan.wahren@in-tech.com>");
+MODULE_LICENSE("GPL");
+MODULE_ALIAS("spi:" DRV_NAME);
diff --git a/drivers/net/ethernet/xilinx/ll_temac_main.c b/drivers/net/ethernet/xilinx/ll_temac_main.c
index e7065c9..b900ab5 100644
--- a/drivers/net/ethernet/xilinx/ll_temac_main.c
+++ b/drivers/net/ethernet/xilinx/ll_temac_main.c
@@ -1276,8 +1276,11 @@ static const struct attribute_group temac_attr_group = {
* ethtool support
*/
-static void ll_temac_ethtools_get_ringparam(struct net_device *ndev,
- struct ethtool_ringparam *ering)
+static void
+ll_temac_ethtools_get_ringparam(struct net_device *ndev,
+ struct ethtool_ringparam *ering,
+ struct kernel_ethtool_ringparam *kernel_ering,
+ struct netlink_ext_ack *extack)
{
struct temac_local *lp = netdev_priv(ndev);
@@ -1291,8 +1294,11 @@ static void ll_temac_ethtools_get_ringparam(struct net_device *ndev,
ering->tx_pending = lp->tx_bd_num;
}
-static int ll_temac_ethtools_set_ringparam(struct net_device *ndev,
- struct ethtool_ringparam *ering)
+static int
+ll_temac_ethtools_set_ringparam(struct net_device *ndev,
+ struct ethtool_ringparam *ering,
+ struct kernel_ethtool_ringparam *kernel_ering,
+ struct netlink_ext_ack *extack)
{
struct temac_local *lp = netdev_priv(ndev);
diff --git a/drivers/net/ethernet/xilinx/xilinx_axienet_main.c b/drivers/net/ethernet/xilinx/xilinx_axienet_main.c
index 9b068b8..23ac353 100644
--- a/drivers/net/ethernet/xilinx/xilinx_axienet_main.c
+++ b/drivers/net/ethernet/xilinx/xilinx_axienet_main.c
@@ -1323,8 +1323,11 @@ static void axienet_ethtools_get_regs(struct net_device *ndev,
data[39] = axienet_dma_in32(lp, XAXIDMA_RX_TDESC_OFFSET);
}
-static void axienet_ethtools_get_ringparam(struct net_device *ndev,
- struct ethtool_ringparam *ering)
+static void
+axienet_ethtools_get_ringparam(struct net_device *ndev,
+ struct ethtool_ringparam *ering,
+ struct kernel_ethtool_ringparam *kernel_ering,
+ struct netlink_ext_ack *extack)
{
struct axienet_local *lp = netdev_priv(ndev);
@@ -1338,8 +1341,11 @@ static void axienet_ethtools_get_ringparam(struct net_device *ndev,
ering->tx_pending = lp->tx_bd_num;
}
-static int axienet_ethtools_set_ringparam(struct net_device *ndev,
- struct ethtool_ringparam *ering)
+static int
+axienet_ethtools_set_ringparam(struct net_device *ndev,
+ struct ethtool_ringparam *ering,
+ struct kernel_ethtool_ringparam *kernel_ering,
+ struct netlink_ext_ack *extack)
{
struct axienet_local *lp = netdev_priv(ndev);
@@ -1503,65 +1509,6 @@ static const struct ethtool_ops axienet_ethtool_ops = {
.nway_reset = axienet_ethtools_nway_reset,
};
-static void axienet_validate(struct phylink_config *config,
- unsigned long *supported,
- struct phylink_link_state *state)
-{
- struct net_device *ndev = to_net_dev(config->dev);
- struct axienet_local *lp = netdev_priv(ndev);
- __ETHTOOL_DECLARE_LINK_MODE_MASK(mask) = { 0, };
-
- /* Only support the mode we are configured for */
- switch (state->interface) {
- case PHY_INTERFACE_MODE_NA:
- break;
- case PHY_INTERFACE_MODE_1000BASEX:
- case PHY_INTERFACE_MODE_SGMII:
- if (lp->switch_x_sgmii)
- break;
- fallthrough;
- default:
- if (state->interface != lp->phy_mode) {
- netdev_warn(ndev, "Cannot use PHY mode %s, supported: %s\n",
- phy_modes(state->interface),
- phy_modes(lp->phy_mode));
- linkmode_zero(supported);
- return;
- }
- }
-
- phylink_set(mask, Autoneg);
- phylink_set_port_modes(mask);
-
- phylink_set(mask, Asym_Pause);
- phylink_set(mask, Pause);
-
- switch (state->interface) {
- case PHY_INTERFACE_MODE_NA:
- case PHY_INTERFACE_MODE_1000BASEX:
- case PHY_INTERFACE_MODE_SGMII:
- case PHY_INTERFACE_MODE_GMII:
- case PHY_INTERFACE_MODE_RGMII:
- case PHY_INTERFACE_MODE_RGMII_ID:
- case PHY_INTERFACE_MODE_RGMII_RXID:
- case PHY_INTERFACE_MODE_RGMII_TXID:
- phylink_set(mask, 1000baseX_Full);
- phylink_set(mask, 1000baseT_Full);
- if (state->interface == PHY_INTERFACE_MODE_1000BASEX)
- break;
- fallthrough;
- case PHY_INTERFACE_MODE_MII:
- phylink_set(mask, 100baseT_Full);
- phylink_set(mask, 10baseT_Full);
- fallthrough;
- default:
- break;
- }
-
- linkmode_and(supported, supported, mask);
- linkmode_and(state->advertising, state->advertising, mask);
-}
-
static void axienet_mac_pcs_get_state(struct phylink_config *config,
struct phylink_link_state *state)
{
@@ -1687,7 +1634,7 @@ static void axienet_mac_link_up(struct phylink_config *config,
}
static const struct phylink_mac_ops axienet_phylink_ops = {
- .validate = axienet_validate,
+ .validate = phylink_generic_validate,
.mac_pcs_get_state = axienet_mac_pcs_get_state,
.mac_an_restart = axienet_mac_an_restart,
.mac_prepare = axienet_mac_prepare,
@@ -2104,6 +2051,17 @@ static int axienet_probe(struct platform_device *pdev)
lp->phylink_config.dev = &ndev->dev;
lp->phylink_config.type = PHYLINK_NETDEV;
+ lp->phylink_config.legacy_pre_march2020 = true;
+ lp->phylink_config.mac_capabilities = MAC_SYM_PAUSE | MAC_ASYM_PAUSE |
+ MAC_10FD | MAC_100FD | MAC_1000FD;
+
+ __set_bit(lp->phy_mode, lp->phylink_config.supported_interfaces);
+ if (lp->switch_x_sgmii) {
+ __set_bit(PHY_INTERFACE_MODE_1000BASEX,
+ lp->phylink_config.supported_interfaces);
+ __set_bit(PHY_INTERFACE_MODE_SGMII,
+ lp->phylink_config.supported_interfaces);
+ }
lp->phylink = phylink_create(&lp->phylink_config, pdev->dev.fwnode,
lp->phy_mode,
diff --git a/drivers/net/ethernet/xilinx/xilinx_emaclite.c b/drivers/net/ethernet/xilinx/xilinx_emaclite.c
index 0815de5..5195994 100644
--- a/drivers/net/ethernet/xilinx/xilinx_emaclite.c
+++ b/drivers/net/ethernet/xilinx/xilinx_emaclite.c
@@ -1133,14 +1133,11 @@ static int xemaclite_of_probe(struct platform_device *ofdev)
lp->ndev = ndev;
/* Get IRQ for the device */
- res = platform_get_resource(ofdev, IORESOURCE_IRQ, 0);
- if (!res) {
- dev_err(dev, "no IRQ found\n");
- rc = -ENXIO;
+ rc = platform_get_irq(ofdev, 0);
+ if (rc < 0)
goto error;
- }
- ndev->irq = res->start;
+ ndev->irq = rc;
res = platform_get_resource(ofdev, IORESOURCE_MEM, 0);
lp->base_addr = devm_ioremap_resource(&ofdev->dev, res);
diff --git a/drivers/net/ethernet/xscale/ixp4xx_eth.c b/drivers/net/ethernet/xscale/ixp4xx_eth.c
index 65fdad1..df77a22 100644
--- a/drivers/net/ethernet/xscale/ixp4xx_eth.c
+++ b/drivers/net/ethernet/xscale/ixp4xx_eth.c
@@ -382,9 +382,6 @@ static int hwtstamp_set(struct net_device *netdev, struct ifreq *ifr)
if (copy_from_user(&cfg, ifr->ifr_data, sizeof(cfg)))
return -EFAULT;
- if (cfg.flags) /* reserved for future extensions */
- return -EINVAL;
-
ret = ixp46x_ptp_find(&port->timesync_regs, &port->phc_index);
if (ret)
return ret;
diff --git a/drivers/net/fddi/skfp/hwmtm.c b/drivers/net/fddi/skfp/hwmtm.c
index 1070390..145767d 100644
--- a/drivers/net/fddi/skfp/hwmtm.c
+++ b/drivers/net/fddi/skfp/hwmtm.c
@@ -38,10 +38,10 @@
-------------------------------------------------------------
*/
#ifdef COMMON_MB_POOL
-static SMbuf *mb_start = 0 ;
-static SMbuf *mb_free = 0 ;
+static SMbuf *mb_start;
+static SMbuf *mb_free;
static int mb_init = FALSE ;
-static int call_count = 0 ;
+static int call_count;
#endif
/*
diff --git a/drivers/net/fddi/skfp/smt.c b/drivers/net/fddi/skfp/smt.c
index 6b68a53..72c31f0 100644
--- a/drivers/net/fddi/skfp/smt.c
+++ b/drivers/net/fddi/skfp/smt.c
@@ -1846,10 +1846,10 @@ void smt_swap_para(struct smt_header *sm, int len, int direction)
}
}
+
static void smt_string_swap(char *data, const char *format, int len)
{
const char *open_paren = NULL ;
- int x ;
while (len > 0 && *format) {
switch (*format) {
@@ -1876,19 +1876,13 @@ static void smt_string_swap(char *data, const char *format, int len)
len-- ;
break ;
case 's' :
- x = data[0] ;
- data[0] = data[1] ;
- data[1] = x ;
+ swap(data[0], data[1]) ;
data += 2 ;
len -= 2 ;
break ;
case 'l' :
- x = data[0] ;
- data[0] = data[3] ;
- data[3] = x ;
- x = data[1] ;
- data[1] = data[2] ;
- data[2] = x ;
+ swap(data[0], data[3]) ;
+ swap(data[1], data[2]) ;
data += 4 ;
len -= 4 ;
break ;
diff --git a/drivers/net/geneve.c b/drivers/net/geneve.c
index 1ab94b5..c1fdd72 100644
--- a/drivers/net/geneve.c
+++ b/drivers/net/geneve.c
@@ -17,6 +17,7 @@
#include <net/gro_cells.h>
#include <net/rtnetlink.h>
#include <net/geneve.h>
+#include <net/gro.h>
#include <net/protocol.h>
#define GENEVE_NETDEV_VER "0.6"
@@ -516,18 +517,15 @@ static struct sk_buff *geneve_gro_receive(struct sock *sk,
type = gh->proto_type;
- rcu_read_lock();
ptype = gro_find_receive_by_type(type);
if (!ptype)
- goto out_unlock;
+ goto out;
skb_gro_pull(skb, gh_len);
skb_gro_postpull_rcsum(skb, gh, gh_len);
pp = call_gro_receive(ptype->callbacks.gro_receive, head, skb);
flush = 0;
-out_unlock:
- rcu_read_unlock();
out:
skb_gro_flush_final(skb, pp, flush);
@@ -547,13 +545,10 @@ static int geneve_gro_complete(struct sock *sk, struct sk_buff *skb,
gh_len = geneve_hlen(gh);
type = gh->proto_type;
- rcu_read_lock();
ptype = gro_find_complete_by_type(type);
if (ptype)
err = ptype->callbacks.gro_complete(skb, nhoff + gh_len);
- rcu_read_unlock();
-
skb_set_inner_mac_header(skb, nhoff + gh_len);
return err;
diff --git a/drivers/net/hamradio/hdlcdrv.c b/drivers/net/hamradio/hdlcdrv.c
index b0edb91..8297411 100644
--- a/drivers/net/hamradio/hdlcdrv.c
+++ b/drivers/net/hamradio/hdlcdrv.c
@@ -30,6 +30,7 @@
/*****************************************************************************/
#include <linux/capability.h>
+#include <linux/compat.h>
#include <linux/module.h>
#include <linux/types.h>
#include <linux/net.h>
diff --git a/drivers/net/hamradio/scc.c b/drivers/net/hamradio/scc.c
index 3d59dac..f90830d 100644
--- a/drivers/net/hamradio/scc.c
+++ b/drivers/net/hamradio/scc.c
@@ -148,6 +148,7 @@
/* ----------------------------------------------------------------------- */
+#include <linux/compat.h>
#include <linux/module.h>
#include <linux/errno.h>
#include <linux/signal.h>
diff --git a/drivers/net/hyperv/netvsc.c b/drivers/net/hyperv/netvsc.c
index 396bc1c..5086cd0 100644
--- a/drivers/net/hyperv/netvsc.c
+++ b/drivers/net/hyperv/netvsc.c
@@ -155,7 +155,7 @@ static void free_netvsc_device(struct rcu_head *head)
kfree(nvdev->extension);
vfree(nvdev->recv_buf);
vfree(nvdev->send_buf);
- kfree(nvdev->send_section_map);
+ bitmap_free(nvdev->send_section_map);
for (i = 0; i < VRSS_CHANNEL_MAX; i++) {
xdp_rxq_info_unreg(&nvdev->chan_table[i].xdp_rxq);
@@ -336,7 +336,6 @@ static int netvsc_init_buf(struct hv_device *device,
struct net_device *ndev = hv_get_drvdata(device);
struct nvsp_message *init_packet;
unsigned int buf_size;
- size_t map_words;
int i, ret = 0;
/* Get receive buffer area. */
@@ -528,10 +527,9 @@ static int netvsc_init_buf(struct hv_device *device,
net_device->send_section_size, net_device->send_section_cnt);
/* Setup state for managing the send buffer. */
- map_words = DIV_ROUND_UP(net_device->send_section_cnt, BITS_PER_LONG);
-
- net_device->send_section_map = kcalloc(map_words, sizeof(ulong), GFP_KERNEL);
- if (net_device->send_section_map == NULL) {
+ net_device->send_section_map = bitmap_zalloc(net_device->send_section_cnt,
+ GFP_KERNEL);
+ if (!net_device->send_section_map) {
ret = -ENOMEM;
goto cleanup;
}
diff --git a/drivers/net/hyperv/netvsc_bpf.c b/drivers/net/hyperv/netvsc_bpf.c
index aa877da..7856905 100644
--- a/drivers/net/hyperv/netvsc_bpf.c
+++ b/drivers/net/hyperv/netvsc_bpf.c
@@ -68,7 +68,7 @@ u32 netvsc_run_xdp(struct net_device *ndev, struct netvsc_channel *nvchan,
break;
default:
- bpf_warn_invalid_xdp_action(act);
+ bpf_warn_invalid_xdp_action(ndev, prog, act);
}
out:
diff --git a/drivers/net/hyperv/netvsc_drv.c b/drivers/net/hyperv/netvsc_drv.c
index 7e66ae1..efa963b 100644
--- a/drivers/net/hyperv/netvsc_drv.c
+++ b/drivers/net/hyperv/netvsc_drv.c
@@ -1858,7 +1858,9 @@ static void __netvsc_get_ringparam(struct netvsc_device *nvdev,
}
static void netvsc_get_ringparam(struct net_device *ndev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct net_device_context *ndevctx = netdev_priv(ndev);
struct netvsc_device *nvdev = rtnl_dereference(ndevctx->nvdev);
@@ -1870,7 +1872,9 @@ static void netvsc_get_ringparam(struct net_device *ndev,
}
static int netvsc_set_ringparam(struct net_device *ndev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct net_device_context *ndevctx = netdev_priv(ndev);
struct netvsc_device *nvdev = rtnl_dereference(ndevctx->nvdev);
diff --git a/drivers/net/ieee802154/atusb.c b/drivers/net/ieee802154/atusb.c
index 23ee0b1..2f5e7b3 100644
--- a/drivers/net/ieee802154/atusb.c
+++ b/drivers/net/ieee802154/atusb.c
@@ -93,7 +93,9 @@ static int atusb_control_msg(struct atusb *atusb, unsigned int pipe,
ret = usb_control_msg(usb_dev, pipe, request, requesttype,
value, index, data, size, timeout);
- if (ret < 0) {
+ if (ret < size) {
+ ret = ret < 0 ? ret : -ENODATA;
+
atusb->err = ret;
dev_err(&usb_dev->dev,
"%s: req 0x%02x val 0x%x idx 0x%x, error %d\n",
@@ -861,9 +863,9 @@ static int atusb_get_and_show_build(struct atusb *atusb)
if (!build)
return -ENOMEM;
- ret = atusb_control_msg(atusb, usb_rcvctrlpipe(usb_dev, 0),
- ATUSB_BUILD, ATUSB_REQ_FROM_DEV, 0, 0,
- build, ATUSB_BUILD_SIZE, 1000);
+ /* We cannot call atusb_control_msg() here, since this request may read various length data */
+ ret = usb_control_msg(atusb->usb_dev, usb_rcvctrlpipe(usb_dev, 0), ATUSB_BUILD,
+ ATUSB_REQ_FROM_DEV, 0, 0, build, ATUSB_BUILD_SIZE, 1000);
if (ret >= 0) {
build[ret] = 0;
dev_info(&usb_dev->dev, "Firmware: build %s\n", build);
diff --git a/drivers/net/ifb.c b/drivers/net/ifb.c
index 31f522b..1c64d53 100644
--- a/drivers/net/ifb.c
+++ b/drivers/net/ifb.c
@@ -27,6 +27,7 @@
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/netdevice.h>
+#include <linux/ethtool.h>
#include <linux/etherdevice.h>
#include <linux/init.h>
#include <linux/interrupt.h>
@@ -36,30 +37,55 @@
#include <net/net_namespace.h>
#define TX_Q_LIMIT 32
+
+struct ifb_q_stats {
+ u64 packets;
+ u64 bytes;
+ struct u64_stats_sync sync;
+};
+
struct ifb_q_private {
struct net_device *dev;
struct tasklet_struct ifb_tasklet;
int tasklet_pending;
int txqnum;
struct sk_buff_head rq;
- u64 rx_packets;
- u64 rx_bytes;
- struct u64_stats_sync rsync;
-
- struct u64_stats_sync tsync;
- u64 tx_packets;
- u64 tx_bytes;
struct sk_buff_head tq;
+ struct ifb_q_stats rx_stats;
+ struct ifb_q_stats tx_stats;
} ____cacheline_aligned_in_smp;
struct ifb_dev_private {
struct ifb_q_private *tx_private;
};
+/* For ethtools stats. */
+struct ifb_q_stats_desc {
+ char desc[ETH_GSTRING_LEN];
+ size_t offset;
+};
+
+#define IFB_Q_STAT(m) offsetof(struct ifb_q_stats, m)
+
+static const struct ifb_q_stats_desc ifb_q_stats_desc[] = {
+ { "packets", IFB_Q_STAT(packets) },
+ { "bytes", IFB_Q_STAT(bytes) },
+};
+
+#define IFB_Q_STATS_LEN ARRAY_SIZE(ifb_q_stats_desc)
+
static netdev_tx_t ifb_xmit(struct sk_buff *skb, struct net_device *dev);
static int ifb_open(struct net_device *dev);
static int ifb_close(struct net_device *dev);
+static void ifb_update_q_stats(struct ifb_q_stats *stats, int len)
+{
+ u64_stats_update_begin(&stats->sync);
+ stats->packets++;
+ stats->bytes += len;
+ u64_stats_update_end(&stats->sync);
+}
+
static void ifb_ri_tasklet(struct tasklet_struct *t)
{
struct ifb_q_private *txp = from_tasklet(txp, t, ifb_tasklet);
@@ -83,10 +109,7 @@ static void ifb_ri_tasklet(struct tasklet_struct *t)
#endif
nf_skip_egress(skb, true);
- u64_stats_update_begin(&txp->tsync);
- txp->tx_packets++;
- txp->tx_bytes += skb->len;
- u64_stats_update_end(&txp->tsync);
+ ifb_update_q_stats(&txp->tx_stats, skb->len);
rcu_read_lock();
skb->dev = dev_get_by_index_rcu(dev_net(txp->dev), skb->skb_iif);
@@ -139,18 +162,18 @@ static void ifb_stats64(struct net_device *dev,
for (i = 0; i < dev->num_tx_queues; i++,txp++) {
do {
- start = u64_stats_fetch_begin_irq(&txp->rsync);
- packets = txp->rx_packets;
- bytes = txp->rx_bytes;
- } while (u64_stats_fetch_retry_irq(&txp->rsync, start));
+ start = u64_stats_fetch_begin_irq(&txp->rx_stats.sync);
+ packets = txp->rx_stats.packets;
+ bytes = txp->rx_stats.bytes;
+ } while (u64_stats_fetch_retry_irq(&txp->rx_stats.sync, start));
stats->rx_packets += packets;
stats->rx_bytes += bytes;
do {
- start = u64_stats_fetch_begin_irq(&txp->tsync);
- packets = txp->tx_packets;
- bytes = txp->tx_bytes;
- } while (u64_stats_fetch_retry_irq(&txp->tsync, start));
+ start = u64_stats_fetch_begin_irq(&txp->tx_stats.sync);
+ packets = txp->tx_stats.packets;
+ bytes = txp->tx_stats.bytes;
+ } while (u64_stats_fetch_retry_irq(&txp->tx_stats.sync, start));
stats->tx_packets += packets;
stats->tx_bytes += bytes;
}
@@ -173,14 +196,83 @@ static int ifb_dev_init(struct net_device *dev)
txp->dev = dev;
__skb_queue_head_init(&txp->rq);
__skb_queue_head_init(&txp->tq);
- u64_stats_init(&txp->rsync);
- u64_stats_init(&txp->tsync);
+ u64_stats_init(&txp->rx_stats.sync);
+ u64_stats_init(&txp->tx_stats.sync);
tasklet_setup(&txp->ifb_tasklet, ifb_ri_tasklet);
netif_tx_start_queue(netdev_get_tx_queue(dev, i));
}
return 0;
}
+static void ifb_get_strings(struct net_device *dev, u32 stringset, u8 *buf)
+{
+ u8 *p = buf;
+ int i, j;
+
+ switch (stringset) {
+ case ETH_SS_STATS:
+ for (i = 0; i < dev->real_num_rx_queues; i++)
+ for (j = 0; j < IFB_Q_STATS_LEN; j++)
+ ethtool_sprintf(&p, "rx_queue_%u_%.18s",
+ i, ifb_q_stats_desc[j].desc);
+
+ for (i = 0; i < dev->real_num_tx_queues; i++)
+ for (j = 0; j < IFB_Q_STATS_LEN; j++)
+ ethtool_sprintf(&p, "tx_queue_%u_%.18s",
+ i, ifb_q_stats_desc[j].desc);
+
+ break;
+ }
+}
+
+static int ifb_get_sset_count(struct net_device *dev, int sset)
+{
+ switch (sset) {
+ case ETH_SS_STATS:
+ return IFB_Q_STATS_LEN * (dev->real_num_rx_queues +
+ dev->real_num_tx_queues);
+ default:
+ return -EOPNOTSUPP;
+ }
+}
+
+static void ifb_fill_stats_data(u64 **data,
+ struct ifb_q_stats *q_stats)
+{
+ void *stats_base = (void *)q_stats;
+ unsigned int start;
+ size_t offset;
+ int j;
+
+ do {
+ start = u64_stats_fetch_begin_irq(&q_stats->sync);
+ for (j = 0; j < IFB_Q_STATS_LEN; j++) {
+ offset = ifb_q_stats_desc[j].offset;
+ (*data)[j] = *(u64 *)(stats_base + offset);
+ }
+ } while (u64_stats_fetch_retry_irq(&q_stats->sync, start));
+
+ *data += IFB_Q_STATS_LEN;
+}
+
+static void ifb_get_ethtool_stats(struct net_device *dev,
+ struct ethtool_stats *stats, u64 *data)
+{
+ struct ifb_dev_private *dp = netdev_priv(dev);
+ struct ifb_q_private *txp;
+ int i;
+
+ for (i = 0; i < dev->real_num_rx_queues; i++) {
+ txp = dp->tx_private + i;
+ ifb_fill_stats_data(&data, &txp->rx_stats);
+ }
+
+ for (i = 0; i < dev->real_num_tx_queues; i++) {
+ txp = dp->tx_private + i;
+ ifb_fill_stats_data(&data, &txp->tx_stats);
+ }
+}
+
static const struct net_device_ops ifb_netdev_ops = {
.ndo_open = ifb_open,
.ndo_stop = ifb_close,
@@ -190,6 +282,12 @@ static const struct net_device_ops ifb_netdev_ops = {
.ndo_init = ifb_dev_init,
};
+static const struct ethtool_ops ifb_ethtool_ops = {
+ .get_strings = ifb_get_strings,
+ .get_sset_count = ifb_get_sset_count,
+ .get_ethtool_stats = ifb_get_ethtool_stats,
+};
+
#define IFB_FEATURES (NETIF_F_HW_CSUM | NETIF_F_SG | NETIF_F_FRAGLIST | \
NETIF_F_GSO_SOFTWARE | NETIF_F_GSO_ENCAP_ALL | \
NETIF_F_HIGHDMA | NETIF_F_HW_VLAN_CTAG_TX | \
@@ -213,6 +311,7 @@ static void ifb_setup(struct net_device *dev)
{
/* Initialize the device structure. */
dev->netdev_ops = &ifb_netdev_ops;
+ dev->ethtool_ops = &ifb_ethtool_ops;
/* Fill in device structure with ethernet-generic values. */
ether_setup(dev);
@@ -241,10 +340,7 @@ static netdev_tx_t ifb_xmit(struct sk_buff *skb, struct net_device *dev)
struct ifb_dev_private *dp = netdev_priv(dev);
struct ifb_q_private *txp = dp->tx_private + skb_get_queue_mapping(skb);
- u64_stats_update_begin(&txp->rsync);
- txp->rx_packets++;
- txp->rx_bytes += skb->len;
- u64_stats_update_end(&txp->rsync);
+ ifb_update_q_stats(&txp->rx_stats, skb->len);
if (!skb->redirected || !skb->skb_iif) {
dev_kfree_skb(skb);
diff --git a/drivers/net/ipa/gsi.c b/drivers/net/ipa/gsi.c
index a2fcdb1..bc98104 100644
--- a/drivers/net/ipa/gsi.c
+++ b/drivers/net/ipa/gsi.c
@@ -93,6 +93,7 @@
#define GSI_CHANNEL_STOP_RETRIES 10
#define GSI_CHANNEL_MODEM_HALT_RETRIES 10
+#define GSI_CHANNEL_MODEM_FLOW_RETRIES 5 /* disable flow control only */
#define GSI_MHI_EVENT_ID_START 10 /* 1st reserved event id */
#define GSI_MHI_EVENT_ID_END 16 /* Last reserved event id */
@@ -339,10 +340,10 @@ static u32 gsi_ring_index(struct gsi_ring *ring, u32 offset)
* completion to be signaled. Returns true if the command completes
* or false if it times out.
*/
-static bool
-gsi_command(struct gsi *gsi, u32 reg, u32 val, struct completion *completion)
+static bool gsi_command(struct gsi *gsi, u32 reg, u32 val)
{
unsigned long timeout = msecs_to_jiffies(GSI_CMD_TIMEOUT);
+ struct completion *completion = &gsi->completion;
reinit_completion(completion);
@@ -366,8 +367,6 @@ gsi_evt_ring_state(struct gsi *gsi, u32 evt_ring_id)
static void gsi_evt_ring_command(struct gsi *gsi, u32 evt_ring_id,
enum gsi_evt_cmd_opcode opcode)
{
- struct gsi_evt_ring *evt_ring = &gsi->evt_ring[evt_ring_id];
- struct completion *completion = &evt_ring->completion;
struct device *dev = gsi->dev;
bool timeout;
u32 val;
@@ -378,7 +377,7 @@ static void gsi_evt_ring_command(struct gsi *gsi, u32 evt_ring_id,
val = u32_encode_bits(evt_ring_id, EV_CHID_FMASK);
val |= u32_encode_bits(opcode, EV_OPCODE_FMASK);
- timeout = !gsi_command(gsi, GSI_EV_CH_CMD_OFFSET, val, completion);
+ timeout = !gsi_command(gsi, GSI_EV_CH_CMD_OFFSET, val);
gsi_irq_ev_ctrl_disable(gsi);
@@ -478,7 +477,6 @@ static enum gsi_channel_state gsi_channel_state(struct gsi_channel *channel)
static void
gsi_channel_command(struct gsi_channel *channel, enum gsi_ch_cmd_opcode opcode)
{
- struct completion *completion = &channel->completion;
u32 channel_id = gsi_channel_id(channel);
struct gsi *gsi = channel->gsi;
struct device *dev = gsi->dev;
@@ -490,7 +488,7 @@ gsi_channel_command(struct gsi_channel *channel, enum gsi_ch_cmd_opcode opcode)
val = u32_encode_bits(channel_id, CH_CHID_FMASK);
val |= u32_encode_bits(opcode, CH_OPCODE_FMASK);
- timeout = !gsi_command(gsi, GSI_CH_CMD_OFFSET, val, completion);
+ timeout = !gsi_command(gsi, GSI_CH_CMD_OFFSET, val);
gsi_irq_ch_ctrl_disable(gsi);
@@ -1074,13 +1072,10 @@ static void gsi_isr_chan_ctrl(struct gsi *gsi)
while (channel_mask) {
u32 channel_id = __ffs(channel_mask);
- struct gsi_channel *channel;
channel_mask ^= BIT(channel_id);
- channel = &gsi->channel[channel_id];
-
- complete(&channel->completion);
+ complete(&gsi->completion);
}
}
@@ -1094,13 +1089,10 @@ static void gsi_isr_evt_ctrl(struct gsi *gsi)
while (event_mask) {
u32 evt_ring_id = __ffs(event_mask);
- struct gsi_evt_ring *evt_ring;
event_mask ^= BIT(evt_ring_id);
- evt_ring = &gsi->evt_ring[evt_ring_id];
-
- complete(&evt_ring->completion);
+ complete(&gsi->completion);
}
}
@@ -1110,7 +1102,7 @@ gsi_isr_glob_chan_err(struct gsi *gsi, u32 err_ee, u32 channel_id, u32 code)
{
if (code == GSI_OUT_OF_RESOURCES) {
dev_err(gsi->dev, "channel %u out of resources\n", channel_id);
- complete(&gsi->channel[channel_id].completion);
+ complete(&gsi->completion);
return;
}
@@ -1127,7 +1119,7 @@ gsi_isr_glob_evt_err(struct gsi *gsi, u32 err_ee, u32 evt_ring_id, u32 code)
struct gsi_evt_ring *evt_ring = &gsi->evt_ring[evt_ring_id];
u32 channel_id = gsi_channel_id(evt_ring->channel);
- complete(&evt_ring->completion);
+ complete(&gsi->completion);
dev_err(gsi->dev, "evt_ring for channel %u out of resources\n",
channel_id);
return;
@@ -1171,18 +1163,23 @@ static void gsi_isr_gp_int1(struct gsi *gsi)
u32 result;
u32 val;
- /* This interrupt is used to handle completions of the two GENERIC
- * GSI commands. We use these to allocate and halt channels on
- * the modem's behalf due to a hardware quirk on IPA v4.2. Once
- * allocated, the modem "owns" these channels, and as a result we
- * have no way of knowing the channel's state at any given time.
+ /* This interrupt is used to handle completions of GENERIC GSI
+ * commands. We use these to allocate and halt channels on the
+ * modem's behalf due to a hardware quirk on IPA v4.2. The modem
+ * "owns" channels even when the AP allocates them, and have no
+ * way of knowing whether a modem channel's state has been changed.
+ *
+ * We also use GENERIC commands to enable/disable channel flow
+ * control for IPA v4.2+.
*
* It is recommended that we halt the modem channels we allocated
* when shutting down, but it's possible the channel isn't running
* at the time we issue the HALT command. We'll get an error in
* that case, but it's harmless (the channel is already halted).
+ * Similarly, we could get an error back when updating flow control
+ * on a channel because it's not in the proper state.
*
- * For this reason, we silently ignore a CHANNEL_NOT_RUNNING error
+ * In either case, we silently ignore a CHANNEL_NOT_RUNNING error
* if we receive it.
*/
val = ioread32(gsi->virt + GSI_CNTXT_SCRATCH_0_OFFSET);
@@ -1648,19 +1645,25 @@ static void gsi_channel_teardown_one(struct gsi *gsi, u32 channel_id)
gsi_evt_ring_de_alloc_command(gsi, evt_ring_id);
}
+/* We use generic commands only to operate on modem channels. We don't have
+ * the ability to determine channel state for a modem channel, so we simply
+ * issue the command and wait for it to complete.
+ */
static int gsi_generic_command(struct gsi *gsi, u32 channel_id,
- enum gsi_generic_cmd_opcode opcode)
+ enum gsi_generic_cmd_opcode opcode,
+ u8 params)
{
- struct completion *completion = &gsi->completion;
bool timeout;
u32 val;
- /* The error global interrupt type is always enabled (until we
- * teardown), so we won't change that. A generic EE command
- * completes with a GSI global interrupt of type GP_INT1. We
- * only perform one generic command at a time (to allocate or
- * halt a modem channel) and only from this function. So we
- * enable the GP_INT1 IRQ type here while we're expecting it.
+ /* The error global interrupt type is always enabled (until we tear
+ * down), so we will keep it enabled.
+ *
+ * A generic EE command completes with a GSI global interrupt of
+ * type GP_INT1. We only perform one generic command at a time
+ * (to allocate, halt, or enable/disable flow control on a modem
+ * channel), and only from this function. So we enable the GP_INT1
+ * IRQ type here, and disable it again after the command completes.
*/
val = BIT(ERROR_INT) | BIT(GP_INT1);
iowrite32(val, gsi->virt + GSI_CNTXT_GLOB_IRQ_EN_OFFSET);
@@ -1674,8 +1677,9 @@ static int gsi_generic_command(struct gsi *gsi, u32 channel_id,
val = u32_encode_bits(opcode, GENERIC_OPCODE_FMASK);
val |= u32_encode_bits(channel_id, GENERIC_CHID_FMASK);
val |= u32_encode_bits(GSI_EE_MODEM, GENERIC_EE_FMASK);
+ val |= u32_encode_bits(params, GENERIC_PARAMS_FMASK);
- timeout = !gsi_command(gsi, GSI_GENERIC_CMD_OFFSET, val, completion);
+ timeout = !gsi_command(gsi, GSI_GENERIC_CMD_OFFSET, val);
/* Disable the GP_INT1 IRQ type again */
iowrite32(BIT(ERROR_INT), gsi->virt + GSI_CNTXT_GLOB_IRQ_EN_OFFSET);
@@ -1692,7 +1696,7 @@ static int gsi_generic_command(struct gsi *gsi, u32 channel_id,
static int gsi_modem_channel_alloc(struct gsi *gsi, u32 channel_id)
{
return gsi_generic_command(gsi, channel_id,
- GSI_GENERIC_ALLOCATE_CHANNEL);
+ GSI_GENERIC_ALLOCATE_CHANNEL, 0);
}
static void gsi_modem_channel_halt(struct gsi *gsi, u32 channel_id)
@@ -1702,7 +1706,7 @@ static void gsi_modem_channel_halt(struct gsi *gsi, u32 channel_id)
do
ret = gsi_generic_command(gsi, channel_id,
- GSI_GENERIC_HALT_CHANNEL);
+ GSI_GENERIC_HALT_CHANNEL, 0);
while (ret == -EAGAIN && retries--);
if (ret)
@@ -1710,6 +1714,32 @@ static void gsi_modem_channel_halt(struct gsi *gsi, u32 channel_id)
ret, channel_id);
}
+/* Enable or disable flow control for a modem GSI TX channel (IPA v4.2+) */
+void
+gsi_modem_channel_flow_control(struct gsi *gsi, u32 channel_id, bool enable)
+{
+ u32 retries = 0;
+ u32 command;
+ int ret;
+
+ command = enable ? GSI_GENERIC_ENABLE_FLOW_CONTROL
+ : GSI_GENERIC_DISABLE_FLOW_CONTROL;
+ /* Disabling flow control on IPA v4.11+ can return -EAGAIN if enable
+ * is underway. In this case we need to retry the command.
+ */
+ if (!enable && gsi->version >= IPA_VERSION_4_11)
+ retries = GSI_CHANNEL_MODEM_FLOW_RETRIES;
+
+ do
+ ret = gsi_generic_command(gsi, channel_id, command, 0);
+ while (ret == -EAGAIN && retries--);
+
+ if (ret)
+ dev_err(gsi->dev,
+ "error %d %sabling mode channel %u flow control\n",
+ ret, enable ? "en" : "dis", channel_id);
+}
+
/* Setup function for channels */
static int gsi_channel_setup(struct gsi *gsi)
{
@@ -1975,18 +2005,6 @@ static void gsi_channel_evt_ring_exit(struct gsi_channel *channel)
gsi_evt_ring_id_free(gsi, evt_ring_id);
}
-/* Init function for event rings; there is no gsi_evt_ring_exit() */
-static void gsi_evt_ring_init(struct gsi *gsi)
-{
- u32 evt_ring_id = 0;
-
- gsi->event_bitmap = gsi_event_bitmap_init(GSI_EVT_RING_COUNT_MAX);
- gsi->ieob_enabled_bitmap = 0;
- do
- init_completion(&gsi->evt_ring[evt_ring_id].completion);
- while (++evt_ring_id < GSI_EVT_RING_COUNT_MAX);
-}
-
static bool gsi_channel_data_valid(struct gsi *gsi,
const struct ipa_gsi_endpoint_data *data)
{
@@ -2069,7 +2087,6 @@ static int gsi_channel_init_one(struct gsi *gsi,
channel->tlv_count = data->channel.tlv_count;
channel->tre_count = tre_count;
channel->event_count = data->channel.event_count;
- init_completion(&channel->completion);
ret = gsi_channel_evt_ring_init(channel);
if (ret)
@@ -2129,7 +2146,8 @@ static int gsi_channel_init(struct gsi *gsi, u32 count,
/* IPA v4.2 requires the AP to allocate channels for the modem */
modem_alloc = gsi->version == IPA_VERSION_4_2;
- gsi_evt_ring_init(gsi); /* No matching exit required */
+ gsi->event_bitmap = gsi_event_bitmap_init(GSI_EVT_RING_COUNT_MAX);
+ gsi->ieob_enabled_bitmap = 0;
/* The endpoint data array is indexed by endpoint name */
for (i = 0; i < count; i++) {
diff --git a/drivers/net/ipa/gsi.h b/drivers/net/ipa/gsi.h
index 88b80dc..9cc6576 100644
--- a/drivers/net/ipa/gsi.h
+++ b/drivers/net/ipa/gsi.h
@@ -101,6 +101,7 @@ enum gsi_channel_state {
GSI_CHANNEL_STATE_STARTED = 0x2,
GSI_CHANNEL_STATE_STOPPED = 0x3,
GSI_CHANNEL_STATE_STOP_IN_PROC = 0x4,
+ GSI_CHANNEL_STATE_FLOW_CONTROLLED = 0x5, /* IPA v4.2-v4.9 */
GSI_CHANNEL_STATE_ERROR = 0xf,
};
@@ -114,8 +115,6 @@ struct gsi_channel {
u16 tre_count;
u16 event_count;
- struct completion completion; /* signals channel command completion */
-
struct gsi_ring tre_ring;
u32 evt_ring_id;
@@ -141,28 +140,27 @@ enum gsi_evt_ring_state {
struct gsi_evt_ring {
struct gsi_channel *channel;
- struct completion completion; /* signals event ring state changes */
struct gsi_ring ring;
};
struct gsi {
struct device *dev; /* Same as IPA device */
enum ipa_version version;
- struct net_device dummy_dev; /* needed for NAPI */
void __iomem *virt_raw; /* I/O mapped address range */
void __iomem *virt; /* Adjusted for most registers */
u32 irq;
u32 channel_count;
u32 evt_ring_count;
- struct gsi_channel channel[GSI_CHANNEL_COUNT_MAX];
- struct gsi_evt_ring evt_ring[GSI_EVT_RING_COUNT_MAX];
u32 event_bitmap; /* allocated event rings */
u32 modem_channel_bitmap; /* modem channels to allocate */
u32 type_enabled_bitmap; /* GSI IRQ types enabled */
u32 ieob_enabled_bitmap; /* IEOB IRQ enabled (event rings) */
- struct completion completion; /* for global EE commands */
int result; /* Negative errno (generic commands) */
+ struct completion completion; /* Signals GSI command completion */
struct mutex mutex; /* protects commands, programming */
+ struct gsi_channel channel[GSI_CHANNEL_COUNT_MAX];
+ struct gsi_evt_ring evt_ring[GSI_EVT_RING_COUNT_MAX];
+ struct net_device dummy_dev; /* needed for NAPI */
};
/**
@@ -219,6 +217,15 @@ int gsi_channel_start(struct gsi *gsi, u32 channel_id);
int gsi_channel_stop(struct gsi *gsi, u32 channel_id);
/**
+ * gsi_modem_channel_flow_control() - Set channel flow control state (IPA v4.2+)
+ * @gsi: GSI pointer returned by gsi_setup()
+ * @channel_id: Modem TX channel to control
+ * @enable: Whether to enable flow control (i.e., prevent flow)
+ */
+void gsi_modem_channel_flow_control(struct gsi *gsi, u32 channel_id,
+ bool enable);
+
+/**
* gsi_channel_reset() - Reset an allocated GSI channel
* @gsi: GSI pointer
* @channel_id: Channel to be reset
diff --git a/drivers/net/ipa/gsi_reg.h b/drivers/net/ipa/gsi_reg.h
index bf9593d..8906f43 100644
--- a/drivers/net/ipa/gsi_reg.h
+++ b/drivers/net/ipa/gsi_reg.h
@@ -313,11 +313,15 @@ enum gsi_evt_cmd_opcode {
#define GENERIC_OPCODE_FMASK GENMASK(4, 0)
#define GENERIC_CHID_FMASK GENMASK(9, 5)
#define GENERIC_EE_FMASK GENMASK(13, 10)
+#define GENERIC_PARAMS_FMASK GENMASK(31, 24) /* IPA v4.11+ */
/** enum gsi_generic_cmd_opcode - GENERIC_OPCODE field values in GENERIC_CMD */
enum gsi_generic_cmd_opcode {
GSI_GENERIC_HALT_CHANNEL = 0x1,
GSI_GENERIC_ALLOCATE_CHANNEL = 0x2,
+ GSI_GENERIC_ENABLE_FLOW_CONTROL = 0x3, /* IPA v4.2+ */
+ GSI_GENERIC_DISABLE_FLOW_CONTROL = 0x4, /* IPA v4.2+ */
+ GSI_GENERIC_QUERY_FLOW_CONTROL = 0x5, /* IPA v4.11+ */
};
/* The next register is present for IPA v3.5.1 and above */
diff --git a/drivers/net/ipa/ipa_data-v4.5.c b/drivers/net/ipa/ipa_data-v4.5.c
index e62ab9c..2da2c41 100644
--- a/drivers/net/ipa/ipa_data-v4.5.c
+++ b/drivers/net/ipa/ipa_data-v4.5.c
@@ -420,15 +420,10 @@ static const struct ipa_mem_data ipa_mem_data = {
/* Interconnect rates are in 1000 byte/second units */
static const struct ipa_interconnect_data ipa_interconnect_data[] = {
{
- .name = "memory-a",
+ .name = "memory",
.peak_bandwidth = 600000, /* 600 MBps */
.average_bandwidth = 150000, /* 150 MBps */
},
- {
- .name = "memory-b",
- .peak_bandwidth = 1804000, /* 1.804 GBps */
- .average_bandwidth = 150000, /* 150 MBps */
- },
/* Average rate is unused for the next two interconnects */
{
.name = "imem",
diff --git a/drivers/net/ipa/ipa_endpoint.c b/drivers/net/ipa/ipa_endpoint.c
index 03a1709..49d9a07 100644
--- a/drivers/net/ipa/ipa_endpoint.c
+++ b/drivers/net/ipa/ipa_endpoint.c
@@ -237,7 +237,8 @@ static struct gsi_trans *ipa_endpoint_trans_alloc(struct ipa_endpoint *endpoint,
}
/* suspend_delay represents suspend for RX, delay for TX endpoints.
- * Note that suspend is not supported starting with IPA v4.0.
+ * Note that suspend is not supported starting with IPA v4.0, and
+ * delay mode should not be used starting with IPA v4.2.
*/
static bool
ipa_endpoint_init_ctrl(struct ipa_endpoint *endpoint, bool suspend_delay)
@@ -248,11 +249,8 @@ ipa_endpoint_init_ctrl(struct ipa_endpoint *endpoint, bool suspend_delay)
u32 mask;
u32 val;
- /* Suspend is not supported for IPA v4.0+. Delay doesn't work
- * correctly on IPA v4.2.
- */
if (endpoint->toward_ipa)
- WARN_ON(ipa->version == IPA_VERSION_4_2);
+ WARN_ON(ipa->version >= IPA_VERSION_4_2);
else
WARN_ON(ipa->version >= IPA_VERSION_4_0);
@@ -270,15 +268,15 @@ ipa_endpoint_init_ctrl(struct ipa_endpoint *endpoint, bool suspend_delay)
return state;
}
-/* We currently don't care what the previous state was for delay mode */
+/* We don't care what the previous state was for delay mode */
static void
ipa_endpoint_program_delay(struct ipa_endpoint *endpoint, bool enable)
{
+ /* Delay mode should not be used for IPA v4.2+ */
+ WARN_ON(endpoint->ipa->version >= IPA_VERSION_4_2);
WARN_ON(!endpoint->toward_ipa);
- /* Delay mode doesn't work properly for IPA v4.2 */
- if (endpoint->ipa->version != IPA_VERSION_4_2)
- (void)ipa_endpoint_init_ctrl(endpoint, enable);
+ (void)ipa_endpoint_init_ctrl(endpoint, enable);
}
static bool ipa_endpoint_aggr_active(struct ipa_endpoint *endpoint)
@@ -355,26 +353,29 @@ ipa_endpoint_program_suspend(struct ipa_endpoint *endpoint, bool enable)
return suspended;
}
-/* Enable or disable delay or suspend mode on all modem endpoints */
+/* Put all modem RX endpoints into suspend mode, and stop transmission
+ * on all modem TX endpoints. Prior to IPA v4.2, endpoint DELAY mode is
+ * used for TX endpoints; starting with IPA v4.2 we use GSI channel flow
+ * control instead.
+ */
void ipa_endpoint_modem_pause_all(struct ipa *ipa, bool enable)
{
u32 endpoint_id;
- /* DELAY mode doesn't work correctly on IPA v4.2 */
- if (ipa->version == IPA_VERSION_4_2)
- return;
-
for (endpoint_id = 0; endpoint_id < IPA_ENDPOINT_MAX; endpoint_id++) {
struct ipa_endpoint *endpoint = &ipa->endpoint[endpoint_id];
if (endpoint->ee_id != GSI_EE_MODEM)
continue;
- /* Set TX delay mode or RX suspend mode */
- if (endpoint->toward_ipa)
+ if (!endpoint->toward_ipa)
+ (void)ipa_endpoint_program_suspend(endpoint, enable);
+ else if (ipa->version < IPA_VERSION_4_2)
ipa_endpoint_program_delay(endpoint, enable);
else
- (void)ipa_endpoint_program_suspend(endpoint, enable);
+ gsi_modem_channel_flow_control(&ipa->gsi,
+ endpoint->channel_id,
+ enable);
}
}
@@ -860,7 +861,7 @@ static void ipa_endpoint_init_hol_block_timer(struct ipa_endpoint *endpoint,
}
static void
-ipa_endpoint_init_hol_block_enable(struct ipa_endpoint *endpoint, bool enable)
+ipa_endpoint_init_hol_block_en(struct ipa_endpoint *endpoint, bool enable)
{
u32 endpoint_id = endpoint->endpoint_id;
u32 offset;
@@ -874,6 +875,19 @@ ipa_endpoint_init_hol_block_enable(struct ipa_endpoint *endpoint, bool enable)
iowrite32(val, endpoint->ipa->reg_virt + offset);
}
+/* Assumes HOL_BLOCK is in disabled state */
+static void ipa_endpoint_init_hol_block_enable(struct ipa_endpoint *endpoint,
+ u32 microseconds)
+{
+ ipa_endpoint_init_hol_block_timer(endpoint, microseconds);
+ ipa_endpoint_init_hol_block_en(endpoint, true);
+}
+
+static void ipa_endpoint_init_hol_block_disable(struct ipa_endpoint *endpoint)
+{
+ ipa_endpoint_init_hol_block_en(endpoint, false);
+}
+
void ipa_endpoint_modem_hol_block_clear_all(struct ipa *ipa)
{
u32 i;
@@ -884,9 +898,8 @@ void ipa_endpoint_modem_hol_block_clear_all(struct ipa *ipa)
if (endpoint->toward_ipa || endpoint->ee_id != GSI_EE_MODEM)
continue;
- ipa_endpoint_init_hol_block_enable(endpoint, false);
- ipa_endpoint_init_hol_block_timer(endpoint, 0);
- ipa_endpoint_init_hol_block_enable(endpoint, true);
+ ipa_endpoint_init_hol_block_disable(endpoint);
+ ipa_endpoint_init_hol_block_enable(endpoint, 0);
}
}
@@ -1141,18 +1154,19 @@ static void ipa_endpoint_skb_copy(struct ipa_endpoint *endpoint,
{
struct sk_buff *skb;
- skb = __dev_alloc_skb(len, GFP_ATOMIC);
- if (skb) {
- skb_put(skb, len);
- memcpy(skb->data, data, len);
- skb->truesize += extra;
- }
+ if (!endpoint->netdev)
+ return;
- /* Now receive it, or drop it if there's no netdev */
- if (endpoint->netdev)
- ipa_modem_skb_rx(endpoint->netdev, skb);
- else if (skb)
- dev_kfree_skb_any(skb);
+ skb = __dev_alloc_skb(len, GFP_ATOMIC);
+ if (!skb)
+ return;
+
+ /* Copy the data into the socket buffer and receive it */
+ skb_put(skb, len);
+ memcpy(skb->data, data, len);
+ skb->truesize += extra;
+
+ ipa_modem_skb_rx(endpoint->netdev, skb);
}
static bool ipa_endpoint_skb_build(struct ipa_endpoint *endpoint,
@@ -1519,10 +1533,19 @@ static void ipa_endpoint_reset(struct ipa_endpoint *endpoint)
static void ipa_endpoint_program(struct ipa_endpoint *endpoint)
{
- if (endpoint->toward_ipa)
- ipa_endpoint_program_delay(endpoint, false);
- else
+ if (endpoint->toward_ipa) {
+ /* Newer versions of IPA use GSI channel flow control
+ * instead of endpoint DELAY mode to prevent sending data.
+ * Flow control is disabled for newly-allocated channels,
+ * and we can assume flow control is not (ever) enabled
+ * for AP TX channels.
+ */
+ if (endpoint->ipa->version < IPA_VERSION_4_2)
+ ipa_endpoint_program_delay(endpoint, false);
+ } else {
+ /* Ensure suspend mode is off on all AP RX endpoints */
(void)ipa_endpoint_program_suspend(endpoint, false);
+ }
ipa_endpoint_init_cfg(endpoint);
ipa_endpoint_init_nat(endpoint);
ipa_endpoint_init_hdr(endpoint);
@@ -1530,6 +1553,8 @@ static void ipa_endpoint_program(struct ipa_endpoint *endpoint)
ipa_endpoint_init_hdr_metadata_mask(endpoint);
ipa_endpoint_init_mode(endpoint);
ipa_endpoint_init_aggr(endpoint);
+ if (!endpoint->toward_ipa)
+ ipa_endpoint_init_hol_block_disable(endpoint);
ipa_endpoint_init_deaggr(endpoint);
ipa_endpoint_init_rsrc_grp(endpoint);
ipa_endpoint_init_seq(endpoint);
diff --git a/drivers/net/ipa/ipa_main.c b/drivers/net/ipa/ipa_main.c
index a448ec1..3757ce3 100644
--- a/drivers/net/ipa/ipa_main.c
+++ b/drivers/net/ipa/ipa_main.c
@@ -734,7 +734,7 @@ static int ipa_probe(struct platform_device *pdev)
if (ret)
goto err_endpoint_exit;
- ret = ipa_modem_init(ipa, modem_init);
+ ret = ipa_smp2p_init(ipa, modem_init);
if (ret)
goto err_table_exit;
@@ -776,7 +776,7 @@ static int ipa_probe(struct platform_device *pdev)
ipa_deconfig(ipa);
err_power_put:
pm_runtime_put_noidle(dev);
- ipa_modem_exit(ipa);
+ ipa_smp2p_exit(ipa);
err_table_exit:
ipa_table_exit(ipa);
err_endpoint_exit:
@@ -827,7 +827,7 @@ static int ipa_remove(struct platform_device *pdev)
ipa_deconfig(ipa);
out_power_put:
pm_runtime_put_noidle(dev);
- ipa_modem_exit(ipa);
+ ipa_smp2p_exit(ipa);
ipa_table_exit(ipa);
ipa_endpoint_exit(ipa);
gsi_exit(&ipa->gsi);
diff --git a/drivers/net/ipa/ipa_mem.c b/drivers/net/ipa/ipa_mem.c
index 4337b09..1e9eae2 100644
--- a/drivers/net/ipa/ipa_mem.c
+++ b/drivers/net/ipa/ipa_mem.c
@@ -266,9 +266,7 @@ static bool ipa_mem_valid(struct ipa *ipa, const struct ipa_mem_data *mem_data)
}
/* Now see if any required regions are not defined */
- for (mem_id = find_first_zero_bit(regions, IPA_MEM_COUNT);
- mem_id < IPA_MEM_COUNT;
- mem_id = find_next_zero_bit(regions, IPA_MEM_COUNT, mem_id + 1)) {
+ for_each_clear_bit(mem_id, regions, IPA_MEM_COUNT) {
if (ipa_mem_id_required(ipa, mem_id))
dev_err(dev, "required memory region %u missing\n",
mem_id);
diff --git a/drivers/net/ipa/ipa_modem.c b/drivers/net/ipa/ipa_modem.c
index d0ab4d7..27d8709 100644
--- a/drivers/net/ipa/ipa_modem.c
+++ b/drivers/net/ipa/ipa_modem.c
@@ -442,16 +442,6 @@ static int ipa_modem_notify(struct notifier_block *nb, unsigned long action,
return NOTIFY_OK;
}
-int ipa_modem_init(struct ipa *ipa, bool modem_init)
-{
- return ipa_smp2p_init(ipa, modem_init);
-}
-
-void ipa_modem_exit(struct ipa *ipa)
-{
- ipa_smp2p_exit(ipa);
-}
-
int ipa_modem_config(struct ipa *ipa)
{
void *notifier;
diff --git a/drivers/net/ipa/ipa_modem.h b/drivers/net/ipa/ipa_modem.h
index 5e6e3d2..e64ccc2 100644
--- a/drivers/net/ipa/ipa_modem.h
+++ b/drivers/net/ipa/ipa_modem.h
@@ -18,9 +18,6 @@ void ipa_modem_skb_rx(struct net_device *netdev, struct sk_buff *skb);
void ipa_modem_suspend(struct net_device *netdev);
void ipa_modem_resume(struct net_device *netdev);
-int ipa_modem_init(struct ipa *ipa, bool modem_init);
-void ipa_modem_exit(struct ipa *ipa);
-
int ipa_modem_config(struct ipa *ipa);
void ipa_modem_deconfig(struct ipa *ipa);
diff --git a/drivers/net/ipa/ipa_table.c b/drivers/net/ipa/ipa_table.c
index 1da334f..2f5a58b 100644
--- a/drivers/net/ipa/ipa_table.c
+++ b/drivers/net/ipa/ipa_table.c
@@ -419,21 +419,26 @@ static void ipa_table_init_add(struct gsi_trans *trans, bool filter,
const struct ipa_mem *mem = ipa_mem_find(ipa, mem_id);
dma_addr_t hash_addr;
dma_addr_t addr;
+ u32 zero_offset;
u16 hash_count;
+ u32 zero_size;
u16 hash_size;
u16 count;
u16 size;
- /* The number of filtering endpoints determines number of entries
- * in the filter table. The hashed and non-hashed filter table
- * will have the same number of entries. The size of the route
- * table region determines the number of entries it has.
- */
+ /* Compute the number of table entries to initialize */
if (filter) {
- /* Include one extra "slot" to hold the filter map itself */
+ /* The number of filtering endpoints determines number of
+ * entries in the filter table; we also add one more "slot"
+ * to hold the bitmap itself. The size of the hashed filter
+ * table is either the same as the non-hashed one, or zero.
+ */
count = 1 + hweight32(ipa->filter_map);
hash_count = hash_mem->size ? count : 0;
} else {
+ /* The size of a route table region determines the number
+ * of entries it has.
+ */
count = mem->size / sizeof(__le64);
hash_count = hash_mem->size / sizeof(__le64);
}
@@ -445,13 +450,42 @@ static void ipa_table_init_add(struct gsi_trans *trans, bool filter,
ipa_cmd_table_init_add(trans, opcode, size, mem->offset, addr,
hash_size, hash_mem->offset, hash_addr);
+ if (!filter)
+ return;
+
+ /* Zero the unused space in the filter table */
+ zero_offset = mem->offset + size;
+ zero_size = mem->size - size;
+ ipa_cmd_dma_shared_mem_add(trans, zero_offset, zero_size,
+ ipa->zero_addr, true);
+ if (!hash_size)
+ return;
+
+ /* Zero the unused space in the hashed filter table */
+ zero_offset = hash_mem->offset + hash_size;
+ zero_size = hash_mem->size - hash_size;
+ ipa_cmd_dma_shared_mem_add(trans, zero_offset, zero_size,
+ ipa->zero_addr, true);
}
int ipa_table_setup(struct ipa *ipa)
{
struct gsi_trans *trans;
- trans = ipa_cmd_trans_alloc(ipa, 4);
+ /* We will need at most 8 TREs:
+ * - IPv4:
+ * - One for route table initialization (non-hashed and hashed)
+ * - One for filter table initialization (non-hashed and hashed)
+ * - One to zero unused entries in the non-hashed filter table
+ * - One to zero unused entries in the hashed filter table
+ * - IPv6:
+ * - One for route table initialization (non-hashed and hashed)
+ * - One for filter table initialization (non-hashed and hashed)
+ * - One to zero unused entries in the non-hashed filter table
+ * - One to zero unused entries in the hashed filter table
+ * All platforms support at least 8 TREs in a transaction.
+ */
+ trans = ipa_cmd_trans_alloc(ipa, 8);
if (!trans) {
dev_err(&ipa->pdev->dev, "no transaction for table setup\n");
return -EBUSY;
diff --git a/drivers/net/ipvlan/ipvlan_core.c b/drivers/net/ipvlan/ipvlan_core.c
index 6cd5010..c613900 100644
--- a/drivers/net/ipvlan/ipvlan_core.c
+++ b/drivers/net/ipvlan/ipvlan_core.c
@@ -291,8 +291,7 @@ void ipvlan_process_multicast(struct work_struct *work)
else
kfree_skb(skb);
}
- if (dev)
- dev_put(dev);
+ dev_put(dev);
cond_resched();
}
}
diff --git a/drivers/net/ipvlan/ipvlan_main.c b/drivers/net/ipvlan/ipvlan_main.c
index 1d2f4e7..696e245f 100644
--- a/drivers/net/ipvlan/ipvlan_main.c
+++ b/drivers/net/ipvlan/ipvlan_main.c
@@ -100,8 +100,7 @@ static void ipvlan_port_destroy(struct net_device *dev)
netdev_rx_handler_unregister(dev);
cancel_work_sync(&port->wq);
while ((skb = __skb_dequeue(&port->backlog)) != NULL) {
- if (skb->dev)
- dev_put(skb->dev);
+ dev_put(skb->dev);
kfree_skb(skb);
}
ida_destroy(&port->ida);
@@ -140,8 +139,8 @@ static int ipvlan_init(struct net_device *dev)
dev->vlan_features = phy_dev->vlan_features & IPVLAN_FEATURES;
dev->vlan_features |= IPVLAN_ALWAYS_ON_OFLOADS;
dev->hw_enc_features |= dev->features;
- dev->gso_max_size = phy_dev->gso_max_size;
- dev->gso_max_segs = phy_dev->gso_max_segs;
+ netif_set_gso_max_size(dev, phy_dev->gso_max_size);
+ netif_set_gso_max_segs(dev, phy_dev->gso_max_segs);
dev->hard_header_len = phy_dev->hard_header_len;
netdev_lockdep_set_classes(dev);
@@ -763,8 +762,8 @@ static int ipvlan_device_event(struct notifier_block *unused,
case NETDEV_FEAT_CHANGE:
list_for_each_entry(ipvlan, &port->ipvlans, pnode) {
- ipvlan->dev->gso_max_size = dev->gso_max_size;
- ipvlan->dev->gso_max_segs = dev->gso_max_segs;
+ netif_set_gso_max_size(ipvlan->dev, dev->gso_max_size);
+ netif_set_gso_max_segs(ipvlan->dev, dev->gso_max_segs);
netdev_update_features(ipvlan->dev);
}
break;
diff --git a/drivers/net/loopback.c b/drivers/net/loopback.c
index a1c77cc..ed0edf5 100644
--- a/drivers/net/loopback.c
+++ b/drivers/net/loopback.c
@@ -44,6 +44,7 @@
#include <linux/etherdevice.h>
#include <linux/skbuff.h>
#include <linux/ethtool.h>
+#include <net/sch_generic.h>
#include <net/sock.h>
#include <net/checksum.h>
#include <linux/if_ether.h> /* For the statistics structure. */
diff --git a/drivers/net/macvlan.c b/drivers/net/macvlan.c
index d2f830e..6ef5f77 100644
--- a/drivers/net/macvlan.c
+++ b/drivers/net/macvlan.c
@@ -900,8 +900,8 @@ static int macvlan_init(struct net_device *dev)
dev->vlan_features = lowerdev->vlan_features & MACVLAN_FEATURES;
dev->vlan_features |= ALWAYS_ON_OFFLOADS;
dev->hw_enc_features |= dev->features;
- dev->gso_max_size = lowerdev->gso_max_size;
- dev->gso_max_segs = lowerdev->gso_max_segs;
+ netif_set_gso_max_size(dev, lowerdev->gso_max_size);
+ netif_set_gso_max_segs(dev, lowerdev->gso_max_segs);
dev->hard_header_len = lowerdev->hard_header_len;
macvlan_set_lockdep_class(dev);
@@ -1171,7 +1171,6 @@ static const struct net_device_ops macvlan_netdev_ops = {
#endif
.ndo_get_iflink = macvlan_dev_get_iflink,
.ndo_features_check = passthru_features_check,
- .ndo_change_proto_down = dev_change_proto_down_generic,
};
void macvlan_common_setup(struct net_device *dev)
@@ -1182,7 +1181,7 @@ void macvlan_common_setup(struct net_device *dev)
dev->max_mtu = ETH_MAX_MTU;
dev->priv_flags &= ~IFF_TX_SKB_SHARING;
netif_keep_dst(dev);
- dev->priv_flags |= IFF_UNICAST_FLT;
+ dev->priv_flags |= IFF_UNICAST_FLT | IFF_CHANGE_PROTO_DOWN;
dev->netdev_ops = &macvlan_netdev_ops;
dev->needs_free_netdev = true;
dev->header_ops = &macvlan_hard_header_ops;
@@ -1748,8 +1747,8 @@ static int macvlan_device_event(struct notifier_block *unused,
break;
case NETDEV_FEAT_CHANGE:
list_for_each_entry(vlan, &port->vlans, list) {
- vlan->dev->gso_max_size = dev->gso_max_size;
- vlan->dev->gso_max_segs = dev->gso_max_segs;
+ netif_set_gso_max_size(vlan->dev, dev->gso_max_size);
+ netif_set_gso_max_segs(vlan->dev, dev->gso_max_segs);
netdev_update_features(vlan->dev);
}
break;
diff --git a/drivers/net/mctp/Kconfig b/drivers/net/mctp/Kconfig
index d8f966c..2929471 100644
--- a/drivers/net/mctp/Kconfig
+++ b/drivers/net/mctp/Kconfig
@@ -3,6 +3,24 @@
menu "MCTP Device Drivers"
+config MCTP_SERIAL
+ tristate "MCTP serial transport"
+ depends on TTY
+ select CRC_CCITT
+ help
+ This driver provides an MCTP-over-serial interface, through a
+ serial line-discipline, as defined by DMTF specification "DSP0253 -
+ MCTP Serial Transport Binding". By attaching the ldisc to a serial
+ device, we get a new net device to transport MCTP packets.
+
+ This allows communication with external MCTP endpoints which use
+ serial as their transport. It can also be used as an easy way to
+ provide MCTP connectivity between virtual machines, by forwarding
+ data between simple virtual serial devices.
+
+ Say y here if you need to connect to MCTP endpoints over serial. To
+ compile as a module, use m; the module will be called mctp-serial.
+
endmenu
endif
diff --git a/drivers/net/mctp/Makefile b/drivers/net/mctp/Makefile
index e69de29..d326226 100644
--- a/drivers/net/mctp/Makefile
+++ b/drivers/net/mctp/Makefile
@@ -0,0 +1 @@
+obj-$(CONFIG_MCTP_SERIAL) += mctp-serial.o
diff --git a/drivers/net/mctp/mctp-serial.c b/drivers/net/mctp/mctp-serial.c
new file mode 100644
index 0000000..eaa6fb3
--- /dev/null
+++ b/drivers/net/mctp/mctp-serial.c
@@ -0,0 +1,515 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Management Component Transport Protocol (MCTP) - serial transport
+ * binding. This driver is an implementation of the DMTF specificiation
+ * "DSP0253 - Management Component Transport Protocol (MCTP) Serial Transport
+ * Binding", available at:
+ *
+ * https://www.dmtf.org/sites/default/files/standards/documents/DSP0253_1.0.0.pdf
+ *
+ * This driver provides DSP0253-type MCTP-over-serial transport using a Linux
+ * tty device, by setting the N_MCTP line discipline on the tty.
+ *
+ * Copyright (c) 2021 Code Construct
+ */
+
+#include <linux/idr.h>
+#include <linux/if_arp.h>
+#include <linux/module.h>
+#include <linux/skbuff.h>
+#include <linux/tty.h>
+#include <linux/workqueue.h>
+#include <linux/crc-ccitt.h>
+
+#include <linux/mctp.h>
+#include <net/mctp.h>
+#include <net/pkt_sched.h>
+
+#define MCTP_SERIAL_MTU 68 /* base mtu (64) + mctp header */
+#define MCTP_SERIAL_FRAME_MTU (MCTP_SERIAL_MTU + 6) /* + serial framing */
+
+#define MCTP_SERIAL_VERSION 0x1 /* DSP0253 defines a single version: 1 */
+
+#define BUFSIZE MCTP_SERIAL_FRAME_MTU
+
+#define BYTE_FRAME 0x7e
+#define BYTE_ESC 0x7d
+
+static DEFINE_IDA(mctp_serial_ida);
+
+enum mctp_serial_state {
+ STATE_IDLE,
+ STATE_START,
+ STATE_HEADER,
+ STATE_DATA,
+ STATE_ESCAPE,
+ STATE_TRAILER,
+ STATE_DONE,
+ STATE_ERR,
+};
+
+struct mctp_serial {
+ struct net_device *netdev;
+ struct tty_struct *tty;
+
+ int idx;
+
+ /* protects our rx & tx state machines; held during both paths */
+ spinlock_t lock;
+
+ struct work_struct tx_work;
+ enum mctp_serial_state txstate, rxstate;
+ u16 txfcs, rxfcs, rxfcs_rcvd;
+ unsigned int txlen, rxlen;
+ unsigned int txpos, rxpos;
+ unsigned char txbuf[BUFSIZE],
+ rxbuf[BUFSIZE];
+};
+
+static bool needs_escape(unsigned char c)
+{
+ return c == BYTE_ESC || c == BYTE_FRAME;
+}
+
+static int next_chunk_len(struct mctp_serial *dev)
+{
+ int i;
+
+ /* either we have no bytes to send ... */
+ if (dev->txpos == dev->txlen)
+ return 0;
+
+ /* ... or the next byte to send is an escaped byte; requiring a
+ * single-byte chunk...
+ */
+ if (needs_escape(dev->txbuf[dev->txpos]))
+ return 1;
+
+ /* ... or we have one or more bytes up to the next escape - this chunk
+ * will be those non-escaped bytes, and does not include the escaped
+ * byte.
+ */
+ for (i = 1; i + dev->txpos + 1 < dev->txlen; i++) {
+ if (needs_escape(dev->txbuf[dev->txpos + i + 1]))
+ break;
+ }
+
+ return i;
+}
+
+static int write_chunk(struct mctp_serial *dev, unsigned char *buf, int len)
+{
+ return dev->tty->ops->write(dev->tty, buf, len);
+}
+
+static void mctp_serial_tx_work(struct work_struct *work)
+{
+ struct mctp_serial *dev = container_of(work, struct mctp_serial,
+ tx_work);
+ unsigned char c, buf[3];
+ unsigned long flags;
+ int len, txlen;
+
+ spin_lock_irqsave(&dev->lock, flags);
+
+ /* txstate represents the next thing to send */
+ switch (dev->txstate) {
+ case STATE_START:
+ dev->txpos = 0;
+ fallthrough;
+ case STATE_HEADER:
+ buf[0] = BYTE_FRAME;
+ buf[1] = MCTP_SERIAL_VERSION;
+ buf[2] = dev->txlen;
+
+ if (!dev->txpos)
+ dev->txfcs = crc_ccitt(0, buf + 1, 2);
+
+ txlen = write_chunk(dev, buf + dev->txpos, 3 - dev->txpos);
+ if (txlen <= 0) {
+ dev->txstate = STATE_ERR;
+ } else {
+ dev->txpos += txlen;
+ if (dev->txpos == 3) {
+ dev->txstate = STATE_DATA;
+ dev->txpos = 0;
+ }
+ }
+ break;
+
+ case STATE_ESCAPE:
+ buf[0] = dev->txbuf[dev->txpos] & ~0x20;
+ txlen = write_chunk(dev, buf, 1);
+ if (txlen <= 0) {
+ dev->txstate = STATE_ERR;
+ } else {
+ dev->txpos += txlen;
+ if (dev->txpos == dev->txlen) {
+ dev->txstate = STATE_TRAILER;
+ dev->txpos = 0;
+ }
+ }
+
+ break;
+
+ case STATE_DATA:
+ len = next_chunk_len(dev);
+ if (len) {
+ c = dev->txbuf[dev->txpos];
+ if (len == 1 && needs_escape(c)) {
+ buf[0] = BYTE_ESC;
+ buf[1] = c & ~0x20;
+ dev->txfcs = crc_ccitt_byte(dev->txfcs, c);
+ txlen = write_chunk(dev, buf, 2);
+ if (txlen == 2)
+ dev->txpos++;
+ else if (txlen == 1)
+ dev->txstate = STATE_ESCAPE;
+ else
+ dev->txstate = STATE_ERR;
+ } else {
+ txlen = write_chunk(dev,
+ dev->txbuf + dev->txpos,
+ len);
+ if (txlen <= 0) {
+ dev->txstate = STATE_ERR;
+ } else {
+ dev->txfcs = crc_ccitt(dev->txfcs,
+ dev->txbuf +
+ dev->txpos,
+ txlen);
+ dev->txpos += txlen;
+ }
+ }
+ if (dev->txstate == STATE_DATA &&
+ dev->txpos == dev->txlen) {
+ dev->txstate = STATE_TRAILER;
+ dev->txpos = 0;
+ }
+ break;
+ }
+ dev->txstate = STATE_TRAILER;
+ dev->txpos = 0;
+ fallthrough;
+
+ case STATE_TRAILER:
+ buf[0] = dev->txfcs >> 8;
+ buf[1] = dev->txfcs & 0xff;
+ buf[2] = BYTE_FRAME;
+ txlen = write_chunk(dev, buf + dev->txpos, 3 - dev->txpos);
+ if (txlen <= 0) {
+ dev->txstate = STATE_ERR;
+ } else {
+ dev->txpos += txlen;
+ if (dev->txpos == 3) {
+ dev->txstate = STATE_DONE;
+ dev->txpos = 0;
+ }
+ }
+ break;
+ default:
+ netdev_err_once(dev->netdev, "invalid tx state %d\n",
+ dev->txstate);
+ }
+
+ if (dev->txstate == STATE_DONE) {
+ dev->netdev->stats.tx_packets++;
+ dev->netdev->stats.tx_bytes += dev->txlen;
+ dev->txlen = 0;
+ dev->txpos = 0;
+ clear_bit(TTY_DO_WRITE_WAKEUP, &dev->tty->flags);
+ dev->txstate = STATE_IDLE;
+ spin_unlock_irqrestore(&dev->lock, flags);
+
+ netif_wake_queue(dev->netdev);
+ } else {
+ spin_unlock_irqrestore(&dev->lock, flags);
+ }
+}
+
+static netdev_tx_t mctp_serial_tx(struct sk_buff *skb, struct net_device *ndev)
+{
+ struct mctp_serial *dev = netdev_priv(ndev);
+ unsigned long flags;
+
+ WARN_ON(dev->txstate != STATE_IDLE);
+
+ if (skb->len > MCTP_SERIAL_MTU) {
+ dev->netdev->stats.tx_dropped++;
+ goto out;
+ }
+
+ spin_lock_irqsave(&dev->lock, flags);
+ netif_stop_queue(dev->netdev);
+ skb_copy_bits(skb, 0, dev->txbuf, skb->len);
+ dev->txpos = 0;
+ dev->txlen = skb->len;
+ dev->txstate = STATE_START;
+ spin_unlock_irqrestore(&dev->lock, flags);
+
+ set_bit(TTY_DO_WRITE_WAKEUP, &dev->tty->flags);
+ schedule_work(&dev->tx_work);
+
+out:
+ kfree_skb(skb);
+ return NETDEV_TX_OK;
+}
+
+static void mctp_serial_tty_write_wakeup(struct tty_struct *tty)
+{
+ struct mctp_serial *dev = tty->disc_data;
+
+ schedule_work(&dev->tx_work);
+}
+
+static void mctp_serial_rx(struct mctp_serial *dev)
+{
+ struct mctp_skb_cb *cb;
+ struct sk_buff *skb;
+
+ if (dev->rxfcs != dev->rxfcs_rcvd) {
+ dev->netdev->stats.rx_dropped++;
+ dev->netdev->stats.rx_crc_errors++;
+ return;
+ }
+
+ skb = netdev_alloc_skb(dev->netdev, dev->rxlen);
+ if (!skb) {
+ dev->netdev->stats.rx_dropped++;
+ return;
+ }
+
+ skb->protocol = htons(ETH_P_MCTP);
+ skb_put_data(skb, dev->rxbuf, dev->rxlen);
+ skb_reset_network_header(skb);
+
+ cb = __mctp_cb(skb);
+ cb->halen = 0;
+
+ netif_rx_ni(skb);
+ dev->netdev->stats.rx_packets++;
+ dev->netdev->stats.rx_bytes += dev->rxlen;
+}
+
+static void mctp_serial_push_header(struct mctp_serial *dev, unsigned char c)
+{
+ switch (dev->rxpos) {
+ case 0:
+ if (c == BYTE_FRAME)
+ dev->rxpos++;
+ else
+ dev->rxstate = STATE_ERR;
+ break;
+ case 1:
+ if (c == MCTP_SERIAL_VERSION) {
+ dev->rxpos++;
+ dev->rxfcs = crc_ccitt_byte(0, c);
+ } else {
+ dev->rxstate = STATE_ERR;
+ }
+ break;
+ case 2:
+ if (c > MCTP_SERIAL_FRAME_MTU) {
+ dev->rxstate = STATE_ERR;
+ } else {
+ dev->rxlen = c;
+ dev->rxpos = 0;
+ dev->rxstate = STATE_DATA;
+ dev->rxfcs = crc_ccitt_byte(dev->rxfcs, c);
+ }
+ break;
+ }
+}
+
+static void mctp_serial_push_trailer(struct mctp_serial *dev, unsigned char c)
+{
+ switch (dev->rxpos) {
+ case 0:
+ dev->rxfcs_rcvd = c << 8;
+ dev->rxpos++;
+ break;
+ case 1:
+ dev->rxfcs_rcvd |= c;
+ dev->rxpos++;
+ break;
+ case 2:
+ if (c != BYTE_FRAME) {
+ dev->rxstate = STATE_ERR;
+ } else {
+ mctp_serial_rx(dev);
+ dev->rxlen = 0;
+ dev->rxpos = 0;
+ dev->rxstate = STATE_IDLE;
+ }
+ break;
+ }
+}
+
+static void mctp_serial_push(struct mctp_serial *dev, unsigned char c)
+{
+ switch (dev->rxstate) {
+ case STATE_IDLE:
+ dev->rxstate = STATE_HEADER;
+ fallthrough;
+ case STATE_HEADER:
+ mctp_serial_push_header(dev, c);
+ break;
+
+ case STATE_ESCAPE:
+ c |= 0x20;
+ fallthrough;
+ case STATE_DATA:
+ if (dev->rxstate != STATE_ESCAPE && c == BYTE_ESC) {
+ dev->rxstate = STATE_ESCAPE;
+ } else {
+ dev->rxfcs = crc_ccitt_byte(dev->rxfcs, c);
+ dev->rxbuf[dev->rxpos] = c;
+ dev->rxpos++;
+ dev->rxstate = STATE_DATA;
+ if (dev->rxpos == dev->rxlen) {
+ dev->rxpos = 0;
+ dev->rxstate = STATE_TRAILER;
+ }
+ }
+ break;
+
+ case STATE_TRAILER:
+ mctp_serial_push_trailer(dev, c);
+ break;
+
+ case STATE_ERR:
+ if (c == BYTE_FRAME)
+ dev->rxstate = STATE_IDLE;
+ break;
+
+ default:
+ netdev_err_once(dev->netdev, "invalid rx state %d\n",
+ dev->rxstate);
+ }
+}
+
+static void mctp_serial_tty_receive_buf(struct tty_struct *tty,
+ const unsigned char *c,
+ const char *f, int len)
+{
+ struct mctp_serial *dev = tty->disc_data;
+ int i;
+
+ if (!netif_running(dev->netdev))
+ return;
+
+ /* we don't (currently) use the flag bytes, just data. */
+ for (i = 0; i < len; i++)
+ mctp_serial_push(dev, c[i]);
+}
+
+static const struct net_device_ops mctp_serial_netdev_ops = {
+ .ndo_start_xmit = mctp_serial_tx,
+};
+
+static void mctp_serial_setup(struct net_device *ndev)
+{
+ ndev->type = ARPHRD_MCTP;
+
+ /* we limit at the fixed MTU, which is also the MCTP-standard
+ * baseline MTU, so is also our minimum
+ */
+ ndev->mtu = MCTP_SERIAL_MTU;
+ ndev->max_mtu = MCTP_SERIAL_MTU;
+ ndev->min_mtu = MCTP_SERIAL_MTU;
+
+ ndev->hard_header_len = 0;
+ ndev->addr_len = 0;
+ ndev->tx_queue_len = DEFAULT_TX_QUEUE_LEN;
+ ndev->flags = IFF_NOARP;
+ ndev->netdev_ops = &mctp_serial_netdev_ops;
+ ndev->needs_free_netdev = true;
+}
+
+static int mctp_serial_open(struct tty_struct *tty)
+{
+ struct mctp_serial *dev;
+ struct net_device *ndev;
+ char name[32];
+ int idx, rc;
+
+ if (!capable(CAP_NET_ADMIN))
+ return -EPERM;
+
+ if (!tty->ops->write)
+ return -EOPNOTSUPP;
+
+ idx = ida_alloc(&mctp_serial_ida, GFP_KERNEL);
+ if (idx < 0)
+ return idx;
+
+ snprintf(name, sizeof(name), "mctpserial%d", idx);
+ ndev = alloc_netdev(sizeof(*dev), name, NET_NAME_ENUM,
+ mctp_serial_setup);
+ if (!ndev) {
+ rc = -ENOMEM;
+ goto free_ida;
+ }
+
+ dev = netdev_priv(ndev);
+ dev->idx = idx;
+ dev->tty = tty;
+ dev->netdev = ndev;
+ dev->txstate = STATE_IDLE;
+ dev->rxstate = STATE_IDLE;
+ spin_lock_init(&dev->lock);
+ INIT_WORK(&dev->tx_work, mctp_serial_tx_work);
+
+ rc = register_netdev(ndev);
+ if (rc)
+ goto free_netdev;
+
+ tty->receive_room = 64 * 1024;
+ tty->disc_data = dev;
+
+ return 0;
+
+free_netdev:
+ free_netdev(ndev);
+
+free_ida:
+ ida_free(&mctp_serial_ida, idx);
+ return rc;
+}
+
+static void mctp_serial_close(struct tty_struct *tty)
+{
+ struct mctp_serial *dev = tty->disc_data;
+ int idx = dev->idx;
+
+ unregister_netdev(dev->netdev);
+ cancel_work_sync(&dev->tx_work);
+ ida_free(&mctp_serial_ida, idx);
+}
+
+static struct tty_ldisc_ops mctp_ldisc = {
+ .owner = THIS_MODULE,
+ .num = N_MCTP,
+ .name = "mctp",
+ .open = mctp_serial_open,
+ .close = mctp_serial_close,
+ .receive_buf = mctp_serial_tty_receive_buf,
+ .write_wakeup = mctp_serial_tty_write_wakeup,
+};
+
+static int __init mctp_serial_init(void)
+{
+ return tty_register_ldisc(&mctp_ldisc);
+}
+
+static void __exit mctp_serial_exit(void)
+{
+ tty_unregister_ldisc(&mctp_ldisc);
+}
+
+module_init(mctp_serial_init);
+module_exit(mctp_serial_exit);
+
+MODULE_LICENSE("GPL v2");
+MODULE_AUTHOR("Jeremy Kerr <jk@codeconstruct.com.au>");
+MODULE_DESCRIPTION("MCTP Serial transport");
diff --git a/drivers/net/mdio/Kconfig b/drivers/net/mdio/Kconfig
index 6da1fcb..bfa1682 100644
--- a/drivers/net/mdio/Kconfig
+++ b/drivers/net/mdio/Kconfig
@@ -141,7 +141,7 @@
config MDIO_MSCC_MIIM
tristate "Microsemi MIIM interface support"
- depends on HAS_IOMEM
+ depends on HAS_IOMEM && REGMAP_MMIO
select MDIO_DEVRES
help
This driver supports the MIIM (MDIO) interface found in the network
diff --git a/drivers/net/mdio/mdio-ipq8064.c b/drivers/net/mdio/mdio-ipq8064.c
index bd1aea2..37e0d8b 100644
--- a/drivers/net/mdio/mdio-ipq8064.c
+++ b/drivers/net/mdio/mdio-ipq8064.c
@@ -127,7 +127,7 @@ ipq8064_mdio_probe(struct platform_device *pdev)
if (of_address_to_resource(np, 0, &res))
return -ENOMEM;
- base = ioremap(res.start, resource_size(&res));
+ base = devm_ioremap(&pdev->dev, res.start, resource_size(&res));
if (!base)
return -ENOMEM;
diff --git a/drivers/net/mdio/mdio-mscc-miim.c b/drivers/net/mdio/mdio-mscc-miim.c
index 17f98f6..7d2abaf 100644
--- a/drivers/net/mdio/mdio-mscc-miim.c
+++ b/drivers/net/mdio/mdio-mscc-miim.c
@@ -10,10 +10,12 @@
#include <linux/io.h>
#include <linux/iopoll.h>
#include <linux/kernel.h>
+#include <linux/mdio/mdio-mscc-miim.h>
#include <linux/module.h>
#include <linux/of_mdio.h>
#include <linux/phy.h>
#include <linux/platform_device.h>
+#include <linux/regmap.h>
#define MSCC_MIIM_REG_STATUS 0x0
#define MSCC_MIIM_STATUS_STAT_PENDING BIT(2)
@@ -35,37 +37,52 @@
#define MSCC_PHY_REG_PHY_STATUS 0x4
struct mscc_miim_dev {
- void __iomem *regs;
- void __iomem *phy_regs;
+ struct regmap *regs;
+ int mii_status_offset;
+ struct regmap *phy_regs;
+ int phy_reset_offset;
};
/* When high resolution timers aren't built-in: we can't use usleep_range() as
* we would sleep way too long. Use udelay() instead.
*/
-#define mscc_readl_poll_timeout(addr, val, cond, delay_us, timeout_us) \
-({ \
- if (!IS_ENABLED(CONFIG_HIGH_RES_TIMERS)) \
- readl_poll_timeout_atomic(addr, val, cond, delay_us, \
- timeout_us); \
- readl_poll_timeout(addr, val, cond, delay_us, timeout_us); \
+#define mscc_readx_poll_timeout(op, addr, val, cond, delay_us, timeout_us)\
+({ \
+ if (!IS_ENABLED(CONFIG_HIGH_RES_TIMERS)) \
+ readx_poll_timeout_atomic(op, addr, val, cond, delay_us, \
+ timeout_us); \
+ readx_poll_timeout(op, addr, val, cond, delay_us, timeout_us); \
})
+static int mscc_miim_status(struct mii_bus *bus)
+{
+ struct mscc_miim_dev *miim = bus->priv;
+ int val, ret;
+
+ ret = regmap_read(miim->regs,
+ MSCC_MIIM_REG_STATUS + miim->mii_status_offset, &val);
+ if (ret < 0) {
+ WARN_ONCE(1, "mscc miim status read error %d\n", ret);
+ return ret;
+ }
+
+ return val;
+}
+
static int mscc_miim_wait_ready(struct mii_bus *bus)
{
- struct mscc_miim_dev *miim = bus->priv;
u32 val;
- return mscc_readl_poll_timeout(miim->regs + MSCC_MIIM_REG_STATUS, val,
+ return mscc_readx_poll_timeout(mscc_miim_status, bus, val,
!(val & MSCC_MIIM_STATUS_STAT_BUSY), 50,
10000);
}
static int mscc_miim_wait_pending(struct mii_bus *bus)
{
- struct mscc_miim_dev *miim = bus->priv;
u32 val;
- return mscc_readl_poll_timeout(miim->regs + MSCC_MIIM_REG_STATUS, val,
+ return mscc_readx_poll_timeout(mscc_miim_status, bus, val,
!(val & MSCC_MIIM_STATUS_STAT_PENDING),
50, 10000);
}
@@ -80,15 +97,29 @@ static int mscc_miim_read(struct mii_bus *bus, int mii_id, int regnum)
if (ret)
goto out;
- writel(MSCC_MIIM_CMD_VLD | (mii_id << MSCC_MIIM_CMD_PHYAD_SHIFT) |
- (regnum << MSCC_MIIM_CMD_REGAD_SHIFT) | MSCC_MIIM_CMD_OPR_READ,
- miim->regs + MSCC_MIIM_REG_CMD);
+ ret = regmap_write(miim->regs,
+ MSCC_MIIM_REG_CMD + miim->mii_status_offset,
+ MSCC_MIIM_CMD_VLD |
+ (mii_id << MSCC_MIIM_CMD_PHYAD_SHIFT) |
+ (regnum << MSCC_MIIM_CMD_REGAD_SHIFT) |
+ MSCC_MIIM_CMD_OPR_READ);
+
+ if (ret < 0) {
+ WARN_ONCE(1, "mscc miim write cmd reg error %d\n", ret);
+ goto out;
+ }
ret = mscc_miim_wait_ready(bus);
if (ret)
goto out;
- val = readl(miim->regs + MSCC_MIIM_REG_DATA);
+ ret = regmap_read(miim->regs,
+ MSCC_MIIM_REG_DATA + miim->mii_status_offset, &val);
+ if (ret < 0) {
+ WARN_ONCE(1, "mscc miim read data reg error %d\n", ret);
+ goto out;
+ }
+
if (val & MSCC_MIIM_DATA_ERROR) {
ret = -EIO;
goto out;
@@ -109,12 +140,16 @@ static int mscc_miim_write(struct mii_bus *bus, int mii_id,
if (ret < 0)
goto out;
- writel(MSCC_MIIM_CMD_VLD | (mii_id << MSCC_MIIM_CMD_PHYAD_SHIFT) |
- (regnum << MSCC_MIIM_CMD_REGAD_SHIFT) |
- (value << MSCC_MIIM_CMD_WRDATA_SHIFT) |
- MSCC_MIIM_CMD_OPR_WRITE,
- miim->regs + MSCC_MIIM_REG_CMD);
+ ret = regmap_write(miim->regs,
+ MSCC_MIIM_REG_CMD + miim->mii_status_offset,
+ MSCC_MIIM_CMD_VLD |
+ (mii_id << MSCC_MIIM_CMD_PHYAD_SHIFT) |
+ (regnum << MSCC_MIIM_CMD_REGAD_SHIFT) |
+ (value << MSCC_MIIM_CMD_WRDATA_SHIFT) |
+ MSCC_MIIM_CMD_OPR_WRITE);
+ if (ret < 0)
+ WARN_ONCE(1, "mscc miim write error %d\n", ret);
out:
return ret;
}
@@ -122,51 +157,116 @@ static int mscc_miim_write(struct mii_bus *bus, int mii_id,
static int mscc_miim_reset(struct mii_bus *bus)
{
struct mscc_miim_dev *miim = bus->priv;
+ int offset = miim->phy_reset_offset;
+ int ret;
if (miim->phy_regs) {
- writel(0, miim->phy_regs + MSCC_PHY_REG_PHY_CFG);
- writel(0x1ff, miim->phy_regs + MSCC_PHY_REG_PHY_CFG);
+ ret = regmap_write(miim->phy_regs,
+ MSCC_PHY_REG_PHY_CFG + offset, 0);
+ if (ret < 0) {
+ WARN_ONCE(1, "mscc reset set error %d\n", ret);
+ return ret;
+ }
+
+ ret = regmap_write(miim->phy_regs,
+ MSCC_PHY_REG_PHY_CFG + offset, 0x1ff);
+ if (ret < 0) {
+ WARN_ONCE(1, "mscc reset clear error %d\n", ret);
+ return ret;
+ }
+
mdelay(500);
}
return 0;
}
+static const struct regmap_config mscc_miim_regmap_config = {
+ .reg_bits = 32,
+ .val_bits = 32,
+ .reg_stride = 4,
+};
+
+int mscc_miim_setup(struct device *dev, struct mii_bus **pbus, const char *name,
+ struct regmap *mii_regmap, int status_offset)
+{
+ struct mscc_miim_dev *miim;
+ struct mii_bus *bus;
+
+ bus = devm_mdiobus_alloc_size(dev, sizeof(*miim));
+ if (!bus)
+ return -ENOMEM;
+
+ bus->name = name;
+ bus->read = mscc_miim_read;
+ bus->write = mscc_miim_write;
+ bus->reset = mscc_miim_reset;
+ snprintf(bus->id, MII_BUS_ID_SIZE, "%s-mii", dev_name(dev));
+ bus->parent = dev;
+
+ miim = bus->priv;
+
+ *pbus = bus;
+
+ miim->regs = mii_regmap;
+ miim->mii_status_offset = status_offset;
+
+ *pbus = bus;
+
+ return 0;
+}
+EXPORT_SYMBOL(mscc_miim_setup);
+
static int mscc_miim_probe(struct platform_device *pdev)
{
- struct mscc_miim_dev *dev;
+ struct regmap *mii_regmap, *phy_regmap = NULL;
+ void __iomem *regs, *phy_regs;
+ struct mscc_miim_dev *miim;
struct resource *res;
struct mii_bus *bus;
int ret;
- bus = devm_mdiobus_alloc_size(&pdev->dev, sizeof(*dev));
- if (!bus)
- return -ENOMEM;
-
- bus->name = "mscc_miim";
- bus->read = mscc_miim_read;
- bus->write = mscc_miim_write;
- bus->reset = mscc_miim_reset;
- snprintf(bus->id, MII_BUS_ID_SIZE, "%s-mii", dev_name(&pdev->dev));
- bus->parent = &pdev->dev;
-
- dev = bus->priv;
- dev->regs = devm_platform_get_and_ioremap_resource(pdev, 0, NULL);
- if (IS_ERR(dev->regs)) {
+ regs = devm_platform_get_and_ioremap_resource(pdev, 0, NULL);
+ if (IS_ERR(regs)) {
dev_err(&pdev->dev, "Unable to map MIIM registers\n");
- return PTR_ERR(dev->regs);
+ return PTR_ERR(regs);
+ }
+
+ mii_regmap = devm_regmap_init_mmio(&pdev->dev, regs,
+ &mscc_miim_regmap_config);
+
+ if (IS_ERR(mii_regmap)) {
+ dev_err(&pdev->dev, "Unable to create MIIM regmap\n");
+ return PTR_ERR(mii_regmap);
}
/* This resource is optional */
res = platform_get_resource(pdev, IORESOURCE_MEM, 1);
if (res) {
- dev->phy_regs = devm_ioremap_resource(&pdev->dev, res);
- if (IS_ERR(dev->phy_regs)) {
+ phy_regs = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(phy_regs)) {
dev_err(&pdev->dev, "Unable to map internal phy registers\n");
- return PTR_ERR(dev->phy_regs);
+ return PTR_ERR(phy_regs);
+ }
+
+ phy_regmap = devm_regmap_init_mmio(&pdev->dev, phy_regs,
+ &mscc_miim_regmap_config);
+ if (IS_ERR(phy_regmap)) {
+ dev_err(&pdev->dev, "Unable to create phy register regmap\n");
+ return PTR_ERR(phy_regmap);
}
}
+ ret = mscc_miim_setup(&pdev->dev, &bus, "mscc_miim", mii_regmap, 0);
+ if (ret < 0) {
+ dev_err(&pdev->dev, "Unable to setup the MDIO bus\n");
+ return ret;
+ }
+
+ miim = bus->priv;
+ miim->phy_regs = phy_regmap;
+ miim->phy_reset_offset = 0;
+
ret = of_mdiobus_register(bus, pdev->dev.of_node);
if (ret < 0) {
dev_err(&pdev->dev, "Cannot register MDIO bus (%d)\n", ret);
diff --git a/drivers/net/netconsole.c b/drivers/net/netconsole.c
index ccecba9..ab8cd55 100644
--- a/drivers/net/netconsole.c
+++ b/drivers/net/netconsole.c
@@ -721,7 +721,7 @@ static int netconsole_netdev_event(struct notifier_block *this,
__netpoll_cleanup(&nt->np);
spin_lock_irqsave(&target_list_lock, flags);
- dev_put(nt->np.dev);
+ dev_put_track(nt->np.dev, &nt->np.dev_tracker);
nt->np.dev = NULL;
nt->enabled = false;
stopped = true;
diff --git a/drivers/net/netdevsim/dev.c b/drivers/net/netdevsim/dev.c
index 54345c0..08d7b46 100644
--- a/drivers/net/netdevsim/dev.c
+++ b/drivers/net/netdevsim/dev.c
@@ -1622,7 +1622,7 @@ int nsim_drv_probe(struct nsim_bus_dev *nsim_bus_dev)
devlink_params_unregister(devlink, nsim_devlink_params,
ARRAY_SIZE(nsim_devlink_params));
err_dl_unregister:
- devlink_resources_unregister(devlink, NULL);
+ devlink_resources_unregister(devlink);
err_vfc_free:
kfree(nsim_dev->vfconfigs);
err_devlink_free:
@@ -1668,7 +1668,7 @@ void nsim_drv_remove(struct nsim_bus_dev *nsim_bus_dev)
nsim_dev_debugfs_exit(nsim_dev);
devlink_params_unregister(devlink, nsim_devlink_params,
ARRAY_SIZE(nsim_devlink_params));
- devlink_resources_unregister(devlink, NULL);
+ devlink_resources_unregister(devlink);
kfree(nsim_dev->vfconfigs);
devlink_free(devlink);
dev_set_drvdata(&nsim_bus_dev->dev, NULL);
diff --git a/drivers/net/netdevsim/ethtool.c b/drivers/net/netdevsim/ethtool.c
index a6a713b..ffd9f84 100644
--- a/drivers/net/netdevsim/ethtool.c
+++ b/drivers/net/netdevsim/ethtool.c
@@ -65,7 +65,9 @@ static int nsim_set_coalesce(struct net_device *dev,
}
static void nsim_get_ringparam(struct net_device *dev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct netdevsim *ns = netdev_priv(dev);
@@ -73,7 +75,9 @@ static void nsim_get_ringparam(struct net_device *dev,
}
static int nsim_set_ringparam(struct net_device *dev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct netdevsim *ns = netdev_priv(dev);
diff --git a/drivers/net/pcs/pcs-lynx.c b/drivers/net/pcs/pcs-lynx.c
index af36cd6..fd34453 100644
--- a/drivers/net/pcs/pcs-lynx.c
+++ b/drivers/net/pcs/pcs-lynx.c
@@ -22,6 +22,11 @@
#define IF_MODE_SPEED_MSK GENMASK(3, 2)
#define IF_MODE_HALF_DUPLEX BIT(4)
+struct lynx_pcs {
+ struct phylink_pcs pcs;
+ struct mdio_device *mdio;
+};
+
enum sgmii_speed {
SGMII_SPEED_10 = 0,
SGMII_SPEED_100 = 1,
@@ -30,6 +35,15 @@ enum sgmii_speed {
};
#define phylink_pcs_to_lynx(pl_pcs) container_of((pl_pcs), struct lynx_pcs, pcs)
+#define lynx_to_phylink_pcs(lynx) (&(lynx)->pcs)
+
+struct mdio_device *lynx_get_mdio_device(struct phylink_pcs *pcs)
+{
+ struct lynx_pcs *lynx = phylink_pcs_to_lynx(pcs);
+
+ return lynx->mdio;
+}
+EXPORT_SYMBOL(lynx_get_mdio_device);
static void lynx_pcs_get_state_usxgmii(struct mdio_device *pcs,
struct phylink_link_state *state)
@@ -329,25 +343,27 @@ static const struct phylink_pcs_ops lynx_pcs_phylink_ops = {
.pcs_link_up = lynx_pcs_link_up,
};
-struct lynx_pcs *lynx_pcs_create(struct mdio_device *mdio)
+struct phylink_pcs *lynx_pcs_create(struct mdio_device *mdio)
{
- struct lynx_pcs *lynx_pcs;
+ struct lynx_pcs *lynx;
- lynx_pcs = kzalloc(sizeof(*lynx_pcs), GFP_KERNEL);
- if (!lynx_pcs)
+ lynx = kzalloc(sizeof(*lynx), GFP_KERNEL);
+ if (!lynx)
return NULL;
- lynx_pcs->mdio = mdio;
- lynx_pcs->pcs.ops = &lynx_pcs_phylink_ops;
- lynx_pcs->pcs.poll = true;
+ lynx->mdio = mdio;
+ lynx->pcs.ops = &lynx_pcs_phylink_ops;
+ lynx->pcs.poll = true;
- return lynx_pcs;
+ return lynx_to_phylink_pcs(lynx);
}
EXPORT_SYMBOL(lynx_pcs_create);
-void lynx_pcs_destroy(struct lynx_pcs *pcs)
+void lynx_pcs_destroy(struct phylink_pcs *pcs)
{
- kfree(pcs);
+ struct lynx_pcs *lynx = phylink_pcs_to_lynx(pcs);
+
+ kfree(lynx);
}
EXPORT_SYMBOL(lynx_pcs_destroy);
diff --git a/drivers/net/phy/dp83640.c b/drivers/net/phy/dp83640.c
index 705c166..c2d1a85 100644
--- a/drivers/net/phy/dp83640.c
+++ b/drivers/net/phy/dp83640.c
@@ -1235,9 +1235,6 @@ static int dp83640_hwtstamp(struct mii_timestamper *mii_ts, struct ifreq *ifr)
if (copy_from_user(&cfg, ifr->ifr_data, sizeof(cfg)))
return -EFAULT;
- if (cfg.flags) /* reserved for future extensions */
- return -EINVAL;
-
if (cfg.tx_type < 0 || cfg.tx_type > HWTSTAMP_TX_ONESTEP_SYNC)
return -ERANGE;
diff --git a/drivers/net/phy/dp83869.c b/drivers/net/phy/dp83869.c
index 7113925..b4ff9c5 100644
--- a/drivers/net/phy/dp83869.c
+++ b/drivers/net/phy/dp83869.c
@@ -16,6 +16,7 @@
#include <dt-bindings/net/ti-dp83869.h>
#define DP83869_PHY_ID 0x2000a0f1
+#define DP83561_PHY_ID 0x2000a1a4
#define DP83869_DEVADDR 0x1f
#define MII_DP83869_PHYCTRL 0x10
@@ -878,34 +879,35 @@ static int dp83869_phy_reset(struct phy_device *phydev)
return dp83869_config_init(phydev);
}
+
+#define DP83869_PHY_DRIVER(_id, _name) \
+{ \
+ PHY_ID_MATCH_MODEL(_id), \
+ .name = (_name), \
+ .probe = dp83869_probe, \
+ .config_init = dp83869_config_init, \
+ .soft_reset = dp83869_phy_reset, \
+ .config_intr = dp83869_config_intr, \
+ .handle_interrupt = dp83869_handle_interrupt, \
+ .read_status = dp83869_read_status, \
+ .get_tunable = dp83869_get_tunable, \
+ .set_tunable = dp83869_set_tunable, \
+ .get_wol = dp83869_get_wol, \
+ .set_wol = dp83869_set_wol, \
+ .suspend = genphy_suspend, \
+ .resume = genphy_resume, \
+}
+
static struct phy_driver dp83869_driver[] = {
- {
- PHY_ID_MATCH_MODEL(DP83869_PHY_ID),
- .name = "TI DP83869",
+ DP83869_PHY_DRIVER(DP83869_PHY_ID, "TI DP83869"),
+ DP83869_PHY_DRIVER(DP83561_PHY_ID, "TI DP83561-SP"),
- .probe = dp83869_probe,
- .config_init = dp83869_config_init,
- .soft_reset = dp83869_phy_reset,
-
- /* IRQ related */
- .config_intr = dp83869_config_intr,
- .handle_interrupt = dp83869_handle_interrupt,
- .read_status = dp83869_read_status,
-
- .get_tunable = dp83869_get_tunable,
- .set_tunable = dp83869_set_tunable,
-
- .get_wol = dp83869_get_wol,
- .set_wol = dp83869_set_wol,
-
- .suspend = genphy_suspend,
- .resume = genphy_resume,
- },
};
module_phy_driver(dp83869_driver);
static struct mdio_device_id __maybe_unused dp83869_tbl[] = {
{ PHY_ID_MATCH_MODEL(DP83869_PHY_ID) },
+ { PHY_ID_MATCH_MODEL(DP83561_PHY_ID) },
{ }
};
MODULE_DEVICE_TABLE(mdio, dp83869_tbl);
diff --git a/drivers/net/phy/marvell.c b/drivers/net/phy/marvell.c
index 4fcfca4..739859c 100644
--- a/drivers/net/phy/marvell.c
+++ b/drivers/net/phy/marvell.c
@@ -1225,28 +1225,28 @@ static int m88e1118_config_aneg(struct phy_device *phydev)
static int m88e1118_config_init(struct phy_device *phydev)
{
+ u16 leds;
int err;
- /* Change address */
- err = marvell_set_page(phydev, MII_MARVELL_MSCR_PAGE);
- if (err < 0)
- return err;
-
/* Enable 1000 Mbit */
- err = phy_write(phydev, 0x15, 0x1070);
+ err = phy_write_paged(phydev, MII_MARVELL_MSCR_PAGE,
+ MII_88E1121_PHY_MSCR_REG, 0x1070);
if (err < 0)
return err;
- /* Change address */
- err = marvell_set_page(phydev, MII_MARVELL_LED_PAGE);
- if (err < 0)
- return err;
+ if (phy_interface_is_rgmii(phydev)) {
+ err = m88e1121_config_aneg_rgmii_delays(phydev);
+ if (err < 0)
+ return err;
+ }
/* Adjust LED Control */
if (phydev->dev_flags & MARVELL_PHY_M1118_DNS323_LEDS)
- err = phy_write(phydev, 0x10, 0x1100);
+ leds = 0x1100;
else
- err = phy_write(phydev, 0x10, 0x021e);
+ leds = 0x021e;
+
+ err = phy_write_paged(phydev, MII_MARVELL_LED_PAGE, 0x10, leds);
if (err < 0)
return err;
@@ -1254,7 +1254,7 @@ static int m88e1118_config_init(struct phy_device *phydev)
if (err < 0)
return err;
- /* Reset address */
+ /* Reset page register */
err = marvell_set_page(phydev, MII_MARVELL_COPPER_PAGE);
if (err < 0)
return err;
diff --git a/drivers/net/phy/mdio_bus.c b/drivers/net/phy/mdio_bus.c
index c198722..58d6029 100644
--- a/drivers/net/phy/mdio_bus.c
+++ b/drivers/net/phy/mdio_bus.c
@@ -176,9 +176,11 @@ static void mdiobus_release(struct device *d)
{
struct mii_bus *bus = to_mii_bus(d);
- BUG_ON(bus->state != MDIOBUS_RELEASED &&
- /* for compatibility with error handling in drivers */
- bus->state != MDIOBUS_ALLOCATED);
+ WARN(bus->state != MDIOBUS_RELEASED &&
+ /* for compatibility with error handling in drivers */
+ bus->state != MDIOBUS_ALLOCATED,
+ "%s: not in RELEASED or ALLOCATED state\n",
+ bus->id);
kfree(bus);
}
@@ -532,8 +534,9 @@ int __mdiobus_register(struct mii_bus *bus, struct module *owner)
bus->parent->of_node->fwnode.flags |=
FWNODE_FLAG_NEEDS_CHILD_BOUND_ON_ADD;
- BUG_ON(bus->state != MDIOBUS_ALLOCATED &&
- bus->state != MDIOBUS_UNREGISTERED);
+ WARN(bus->state != MDIOBUS_ALLOCATED &&
+ bus->state != MDIOBUS_UNREGISTERED,
+ "%s: not in ALLOCATED or UNREGISTERED state\n", bus->id);
bus->owner = owner;
bus->dev.parent = bus->parent;
@@ -594,7 +597,7 @@ int __mdiobus_register(struct mii_bus *bus, struct module *owner)
mdiobus_setup_mdiodev_from_board_info(bus, mdiobus_create_device);
bus->state = MDIOBUS_REGISTERED;
- pr_info("%s: probed\n", bus->name);
+ dev_dbg(&bus->dev, "probed\n");
return 0;
error:
@@ -661,7 +664,8 @@ void mdiobus_free(struct mii_bus *bus)
return;
}
- BUG_ON(bus->state != MDIOBUS_UNREGISTERED);
+ WARN(bus->state != MDIOBUS_UNREGISTERED,
+ "%s: not in UNREGISTERED state\n", bus->id);
bus->state = MDIOBUS_RELEASED;
put_device(&bus->dev);
diff --git a/drivers/net/phy/micrel.c b/drivers/net/phy/micrel.c
index 44a24b9..4570cb9 100644
--- a/drivers/net/phy/micrel.c
+++ b/drivers/net/phy/micrel.c
@@ -66,6 +66,19 @@
#define KSZ8081_LMD_SHORT_INDICATOR BIT(12)
#define KSZ8081_LMD_DELTA_TIME_MASK GENMASK(8, 0)
+/* Lan8814 general Interrupt control/status reg in GPHY specific block. */
+#define LAN8814_INTC 0x18
+#define LAN8814_INTS 0x1B
+
+#define LAN8814_INT_LINK_DOWN BIT(2)
+#define LAN8814_INT_LINK_UP BIT(0)
+#define LAN8814_INT_LINK (LAN8814_INT_LINK_UP |\
+ LAN8814_INT_LINK_DOWN)
+
+#define LAN8814_INTR_CTRL_REG 0x34
+#define LAN8814_INTR_CTRL_REG_POLARITY BIT(1)
+#define LAN8814_INTR_CTRL_REG_INTR_ENABLE BIT(0)
+
/* PHY Control 1 */
#define MII_KSZPHY_CTRL_1 0x1e
#define KSZ8081_CTRL1_MDIX_STAT BIT(4)
@@ -1565,6 +1578,14 @@ static int ksz886x_cable_test_get_status(struct phy_device *phydev,
#define LAN_EXT_PAGE_ACCESS_ADDRESS_DATA 0x17
#define LAN_EXT_PAGE_ACCESS_CTRL_EP_FUNC 0x4000
+#define LAN8814_QSGMII_SOFT_RESET 0x43
+#define LAN8814_QSGMII_SOFT_RESET_BIT BIT(0)
+#define LAN8814_QSGMII_PCS1G_ANEG_CONFIG 0x13
+#define LAN8814_QSGMII_PCS1G_ANEG_CONFIG_ANEG_ENA BIT(3)
+#define LAN8814_ALIGN_SWAP 0x4a
+#define LAN8814_ALIGN_TX_A_B_SWAP 0x1
+#define LAN8814_ALIGN_TX_A_B_SWAP_MASK GENMASK(2, 0)
+
#define LAN8804_ALIGN_SWAP 0x4a
#define LAN8804_ALIGN_TX_A_B_SWAP 0x1
#define LAN8804_ALIGN_TX_A_B_SWAP_MASK GENMASK(2, 0)
@@ -1601,6 +1622,29 @@ static int lanphy_write_page_reg(struct phy_device *phydev, int page, u16 addr,
return 0;
}
+static int lan8814_config_init(struct phy_device *phydev)
+{
+ int val;
+
+ /* Reset the PHY */
+ val = lanphy_read_page_reg(phydev, 4, LAN8814_QSGMII_SOFT_RESET);
+ val |= LAN8814_QSGMII_SOFT_RESET_BIT;
+ lanphy_write_page_reg(phydev, 4, LAN8814_QSGMII_SOFT_RESET, val);
+
+ /* Disable ANEG with QSGMII PCS Host side */
+ val = lanphy_read_page_reg(phydev, 5, LAN8814_QSGMII_PCS1G_ANEG_CONFIG);
+ val &= ~LAN8814_QSGMII_PCS1G_ANEG_CONFIG_ANEG_ENA;
+ lanphy_write_page_reg(phydev, 5, LAN8814_QSGMII_PCS1G_ANEG_CONFIG, val);
+
+ /* MDI-X setting for swap A,B transmit */
+ val = lanphy_read_page_reg(phydev, 2, LAN8814_ALIGN_SWAP);
+ val &= ~LAN8814_ALIGN_TX_A_B_SWAP_MASK;
+ val |= LAN8814_ALIGN_TX_A_B_SWAP;
+ lanphy_write_page_reg(phydev, 2, LAN8814_ALIGN_SWAP, val);
+
+ return 0;
+}
+
static int lan8804_config_init(struct phy_device *phydev)
{
int val;
@@ -1620,6 +1664,58 @@ static int lan8804_config_init(struct phy_device *phydev)
return 0;
}
+static irqreturn_t lan8814_handle_interrupt(struct phy_device *phydev)
+{
+ int irq_status;
+
+ irq_status = phy_read(phydev, LAN8814_INTS);
+ if (irq_status < 0)
+ return IRQ_NONE;
+
+ if (!(irq_status & LAN8814_INT_LINK))
+ return IRQ_NONE;
+
+ phy_trigger_machine(phydev);
+
+ return IRQ_HANDLED;
+}
+
+static int lan8814_ack_interrupt(struct phy_device *phydev)
+{
+ /* bit[12..0] int status, which is a read and clear register. */
+ int rc;
+
+ rc = phy_read(phydev, LAN8814_INTS);
+
+ return (rc < 0) ? rc : 0;
+}
+
+static int lan8814_config_intr(struct phy_device *phydev)
+{
+ int err;
+
+ lanphy_write_page_reg(phydev, 4, LAN8814_INTR_CTRL_REG,
+ LAN8814_INTR_CTRL_REG_POLARITY |
+ LAN8814_INTR_CTRL_REG_INTR_ENABLE);
+
+ /* enable / disable interrupts */
+ if (phydev->interrupts == PHY_INTERRUPT_ENABLED) {
+ err = lan8814_ack_interrupt(phydev);
+ if (err)
+ return err;
+
+ err = phy_write(phydev, LAN8814_INTC, LAN8814_INT_LINK);
+ } else {
+ err = phy_write(phydev, LAN8814_INTC, 0);
+ if (err)
+ return err;
+
+ err = lan8814_ack_interrupt(phydev);
+ }
+
+ return err;
+}
+
static struct phy_driver ksphy_driver[] = {
{
.phy_id = PHY_ID_KS8737,
@@ -1793,6 +1889,7 @@ static struct phy_driver ksphy_driver[] = {
.phy_id = PHY_ID_LAN8814,
.phy_id_mask = MICREL_PHY_ID_MASK,
.name = "Microchip INDY Gigabit Quad PHY",
+ .config_init = lan8814_config_init,
.driver_data = &ksz9021_type,
.probe = kszphy_probe,
.soft_reset = genphy_soft_reset,
@@ -1802,6 +1899,8 @@ static struct phy_driver ksphy_driver[] = {
.get_stats = kszphy_get_stats,
.suspend = genphy_suspend,
.resume = kszphy_resume,
+ .config_intr = lan8814_config_intr,
+ .handle_interrupt = lan8814_handle_interrupt,
}, {
.phy_id = PHY_ID_LAN8804,
.phy_id_mask = MICREL_PHY_ID_MASK,
diff --git a/drivers/net/phy/mscc/mscc_ptp.c b/drivers/net/phy/mscc/mscc_ptp.c
index edb9516..34f8298 100644
--- a/drivers/net/phy/mscc/mscc_ptp.c
+++ b/drivers/net/phy/mscc/mscc_ptp.c
@@ -1057,9 +1057,6 @@ static int vsc85xx_hwtstamp(struct mii_timestamper *mii_ts, struct ifreq *ifr)
if (copy_from_user(&cfg, ifr->ifr_data, sizeof(cfg)))
return -EFAULT;
- if (cfg.flags)
- return -EINVAL;
-
switch (cfg.tx_type) {
case HWTSTAMP_TX_ONESTEP_SYNC:
one_step = true;
diff --git a/drivers/net/phy/nxp-c45-tja11xx.c b/drivers/net/phy/nxp-c45-tja11xx.c
index 91a327f..06fdbae 100644
--- a/drivers/net/phy/nxp-c45-tja11xx.c
+++ b/drivers/net/phy/nxp-c45-tja11xx.c
@@ -97,6 +97,11 @@
#define VEND1_TX_IPG_LENGTH 0xAFD1
#define COUNTER_EN BIT(15)
+#define VEND1_PTP_CONFIG 0x1102
+#define EXT_TRG_EDGE BIT(1)
+#define PPS_OUT_POL BIT(2)
+#define PPS_OUT_EN BIT(3)
+
#define VEND1_LTC_LOAD_CTRL 0x1105
#define READ_LTC BIT(2)
#define LOAD_LTC BIT(0)
@@ -132,6 +137,13 @@
#define VEND1_EGR_RING_DATA_3 0x1151
#define VEND1_EGR_RING_CTRL 0x1154
+#define VEND1_EXT_TRG_TS_DATA_0 0x1121
+#define VEND1_EXT_TRG_TS_DATA_1 0x1122
+#define VEND1_EXT_TRG_TS_DATA_2 0x1123
+#define VEND1_EXT_TRG_TS_DATA_3 0x1124
+#define VEND1_EXT_TRG_TS_DATA_4 0x1125
+#define VEND1_EXT_TRG_TS_CTRL 0x1126
+
#define RING_DATA_0_DOMAIN_NUMBER GENMASK(7, 0)
#define RING_DATA_0_MSG_TYPE GENMASK(11, 8)
#define RING_DATA_0_SEC_4_2 GENMASK(14, 2)
@@ -162,6 +174,17 @@
#define VEND1_RX_PIPE_DLY_NS 0x114B
#define VEND1_RX_PIPEDLY_SUBNS 0x114C
+#define VEND1_GPIO_FUNC_CONFIG_BASE 0x2C40
+#define GPIO_FUNC_EN BIT(15)
+#define GPIO_FUNC_PTP BIT(6)
+#define GPIO_SIGNAL_PTP_TRIGGER 0x01
+#define GPIO_SIGNAL_PPS_OUT 0x12
+#define GPIO_DISABLE 0
+#define GPIO_PPS_OUT_CFG (GPIO_FUNC_EN | GPIO_FUNC_PTP | \
+ GPIO_SIGNAL_PPS_OUT)
+#define GPIO_EXTTS_OUT_CFG (GPIO_FUNC_EN | GPIO_FUNC_PTP | \
+ GPIO_SIGNAL_PTP_TRIGGER)
+
#define RGMII_PERIOD_PS 8000U
#define PS_PER_DEGREE div_u64(RGMII_PERIOD_PS, 360)
#define MIN_ID_PS 1644U
@@ -199,6 +222,9 @@ struct nxp_c45_phy {
int hwts_rx;
u32 tx_delay;
u32 rx_delay;
+ struct timespec64 extts_ts;
+ int extts_index;
+ bool extts;
};
struct nxp_c45_phy_stats {
@@ -339,6 +365,21 @@ static bool nxp_c45_match_ts(struct ptp_header *header,
header->domain_number == hwts->domain_number;
}
+static void nxp_c45_get_extts(struct nxp_c45_phy *priv,
+ struct timespec64 *extts)
+{
+ extts->tv_nsec = phy_read_mmd(priv->phydev, MDIO_MMD_VEND1,
+ VEND1_EXT_TRG_TS_DATA_0);
+ extts->tv_nsec |= phy_read_mmd(priv->phydev, MDIO_MMD_VEND1,
+ VEND1_EXT_TRG_TS_DATA_1) << 16;
+ extts->tv_sec = phy_read_mmd(priv->phydev, MDIO_MMD_VEND1,
+ VEND1_EXT_TRG_TS_DATA_2);
+ extts->tv_sec |= phy_read_mmd(priv->phydev, MDIO_MMD_VEND1,
+ VEND1_EXT_TRG_TS_DATA_3) << 16;
+ phy_write_mmd(priv->phydev, MDIO_MMD_VEND1, VEND1_EXT_TRG_TS_CTRL,
+ RING_DONE);
+}
+
static bool nxp_c45_get_hwtxts(struct nxp_c45_phy *priv,
struct nxp_c45_hwts *hwts)
{
@@ -409,6 +450,7 @@ static long nxp_c45_do_aux_work(struct ptp_clock_info *ptp)
struct nxp_c45_phy *priv = container_of(ptp, struct nxp_c45_phy, caps);
bool poll_txts = nxp_c45_poll_txts(priv->phydev);
struct skb_shared_hwtstamps *shhwtstamps_rx;
+ struct ptp_clock_event event;
struct nxp_c45_hwts hwts;
bool reschedule = false;
struct timespec64 ts;
@@ -439,9 +481,181 @@ static long nxp_c45_do_aux_work(struct ptp_clock_info *ptp)
netif_rx_ni(skb);
}
+ if (priv->extts) {
+ nxp_c45_get_extts(priv, &ts);
+ if (timespec64_compare(&ts, &priv->extts_ts) != 0) {
+ priv->extts_ts = ts;
+ event.index = priv->extts_index;
+ event.type = PTP_CLOCK_EXTTS;
+ event.timestamp = ns_to_ktime(timespec64_to_ns(&ts));
+ ptp_clock_event(priv->ptp_clock, &event);
+ }
+ reschedule = true;
+ }
+
return reschedule ? 1 : -1;
}
+static void nxp_c45_gpio_config(struct nxp_c45_phy *priv,
+ int pin, u16 pin_cfg)
+{
+ struct phy_device *phydev = priv->phydev;
+
+ phy_write_mmd(phydev, MDIO_MMD_VEND1,
+ VEND1_GPIO_FUNC_CONFIG_BASE + pin, pin_cfg);
+}
+
+static int nxp_c45_perout_enable(struct nxp_c45_phy *priv,
+ struct ptp_perout_request *perout, int on)
+{
+ struct phy_device *phydev = priv->phydev;
+ int pin;
+
+ if (perout->flags & ~PTP_PEROUT_PHASE)
+ return -EOPNOTSUPP;
+
+ pin = ptp_find_pin(priv->ptp_clock, PTP_PF_PEROUT, perout->index);
+ if (pin < 0)
+ return pin;
+
+ if (!on) {
+ phy_clear_bits_mmd(phydev, MDIO_MMD_VEND1, VEND1_PTP_CONFIG,
+ PPS_OUT_EN);
+ phy_clear_bits_mmd(phydev, MDIO_MMD_VEND1, VEND1_PTP_CONFIG,
+ PPS_OUT_POL);
+
+ nxp_c45_gpio_config(priv, pin, GPIO_DISABLE);
+
+ return 0;
+ }
+
+ /* The PPS signal is fixed to 1 second and is always generated when the
+ * seconds counter is incremented. The start time is not configurable.
+ * If the clock is adjusted, the PPS signal is automatically readjusted.
+ */
+ if (perout->period.sec != 1 || perout->period.nsec != 0) {
+ phydev_warn(phydev, "The period can be set only to 1 second.");
+ return -EINVAL;
+ }
+
+ if (!(perout->flags & PTP_PEROUT_PHASE)) {
+ if (perout->start.sec != 0 || perout->start.nsec != 0) {
+ phydev_warn(phydev, "The start time is not configurable. Should be set to 0 seconds and 0 nanoseconds.");
+ return -EINVAL;
+ }
+ } else {
+ if (perout->phase.nsec != 0 &&
+ perout->phase.nsec != (NSEC_PER_SEC >> 1)) {
+ phydev_warn(phydev, "The phase can be set only to 0 or 500000000 nanoseconds.");
+ return -EINVAL;
+ }
+
+ if (perout->phase.nsec == 0)
+ phy_clear_bits_mmd(phydev, MDIO_MMD_VEND1,
+ VEND1_PTP_CONFIG, PPS_OUT_POL);
+ else
+ phy_set_bits_mmd(phydev, MDIO_MMD_VEND1,
+ VEND1_PTP_CONFIG, PPS_OUT_POL);
+ }
+
+ nxp_c45_gpio_config(priv, pin, GPIO_PPS_OUT_CFG);
+
+ phy_set_bits_mmd(phydev, MDIO_MMD_VEND1, VEND1_PTP_CONFIG, PPS_OUT_EN);
+
+ return 0;
+}
+
+static int nxp_c45_extts_enable(struct nxp_c45_phy *priv,
+ struct ptp_extts_request *extts, int on)
+{
+ int pin;
+
+ if (extts->flags & ~(PTP_ENABLE_FEATURE |
+ PTP_RISING_EDGE |
+ PTP_FALLING_EDGE |
+ PTP_STRICT_FLAGS))
+ return -EOPNOTSUPP;
+
+ /* Sampling on both edges is not supported */
+ if ((extts->flags & PTP_RISING_EDGE) &&
+ (extts->flags & PTP_FALLING_EDGE))
+ return -EOPNOTSUPP;
+
+ pin = ptp_find_pin(priv->ptp_clock, PTP_PF_EXTTS, extts->index);
+ if (pin < 0)
+ return pin;
+
+ if (!on) {
+ nxp_c45_gpio_config(priv, pin, GPIO_DISABLE);
+ priv->extts = false;
+
+ return 0;
+ }
+
+ if (extts->flags & PTP_RISING_EDGE)
+ phy_clear_bits_mmd(priv->phydev, MDIO_MMD_VEND1,
+ VEND1_PTP_CONFIG, EXT_TRG_EDGE);
+
+ if (extts->flags & PTP_FALLING_EDGE)
+ phy_set_bits_mmd(priv->phydev, MDIO_MMD_VEND1,
+ VEND1_PTP_CONFIG, EXT_TRG_EDGE);
+
+ nxp_c45_gpio_config(priv, pin, GPIO_EXTTS_OUT_CFG);
+ priv->extts = true;
+ priv->extts_index = extts->index;
+ ptp_schedule_worker(priv->ptp_clock, 0);
+
+ return 0;
+}
+
+static int nxp_c45_ptp_enable(struct ptp_clock_info *ptp,
+ struct ptp_clock_request *req, int on)
+{
+ struct nxp_c45_phy *priv = container_of(ptp, struct nxp_c45_phy, caps);
+
+ switch (req->type) {
+ case PTP_CLK_REQ_EXTTS:
+ return nxp_c45_extts_enable(priv, &req->extts, on);
+ case PTP_CLK_REQ_PEROUT:
+ return nxp_c45_perout_enable(priv, &req->perout, on);
+ default:
+ return -EOPNOTSUPP;
+ }
+}
+
+static struct ptp_pin_desc nxp_c45_ptp_pins[] = {
+ { "nxp_c45_gpio0", 0, PTP_PF_NONE},
+ { "nxp_c45_gpio1", 1, PTP_PF_NONE},
+ { "nxp_c45_gpio2", 2, PTP_PF_NONE},
+ { "nxp_c45_gpio3", 3, PTP_PF_NONE},
+ { "nxp_c45_gpio4", 4, PTP_PF_NONE},
+ { "nxp_c45_gpio5", 5, PTP_PF_NONE},
+ { "nxp_c45_gpio6", 6, PTP_PF_NONE},
+ { "nxp_c45_gpio7", 7, PTP_PF_NONE},
+ { "nxp_c45_gpio8", 8, PTP_PF_NONE},
+ { "nxp_c45_gpio9", 9, PTP_PF_NONE},
+ { "nxp_c45_gpio10", 10, PTP_PF_NONE},
+ { "nxp_c45_gpio11", 11, PTP_PF_NONE},
+};
+
+static int nxp_c45_ptp_verify_pin(struct ptp_clock_info *ptp, unsigned int pin,
+ enum ptp_pin_function func, unsigned int chan)
+{
+ if (pin >= ARRAY_SIZE(nxp_c45_ptp_pins))
+ return -EINVAL;
+
+ switch (func) {
+ case PTP_PF_NONE:
+ case PTP_PF_PEROUT:
+ case PTP_PF_EXTTS:
+ break;
+ default:
+ return -EOPNOTSUPP;
+ }
+
+ return 0;
+}
+
static int nxp_c45_init_ptp_clock(struct nxp_c45_phy *priv)
{
priv->caps = (struct ptp_clock_info) {
@@ -452,7 +666,13 @@ static int nxp_c45_init_ptp_clock(struct nxp_c45_phy *priv)
.adjtime = nxp_c45_ptp_adjtime,
.gettimex64 = nxp_c45_ptp_gettimex64,
.settime64 = nxp_c45_ptp_settime64,
+ .enable = nxp_c45_ptp_enable,
+ .verify = nxp_c45_ptp_verify_pin,
.do_aux_work = nxp_c45_do_aux_work,
+ .pin_config = nxp_c45_ptp_pins,
+ .n_pins = ARRAY_SIZE(nxp_c45_ptp_pins),
+ .n_ext_ts = 1,
+ .n_per_out = 1,
};
priv->ptp_clock = ptp_clock_register(&priv->caps,
diff --git a/drivers/net/phy/phy-core.c b/drivers/net/phy/phy-core.c
index 2870c33..271fc01 100644
--- a/drivers/net/phy/phy-core.c
+++ b/drivers/net/phy/phy-core.c
@@ -162,11 +162,11 @@ static const struct phy_setting settings[] = {
PHY_SETTING( 2500, FULL, 2500baseT_Full ),
PHY_SETTING( 2500, FULL, 2500baseX_Full ),
/* 1G */
- PHY_SETTING( 1000, FULL, 1000baseKX_Full ),
PHY_SETTING( 1000, FULL, 1000baseT_Full ),
PHY_SETTING( 1000, HALF, 1000baseT_Half ),
PHY_SETTING( 1000, FULL, 1000baseT1_Full ),
PHY_SETTING( 1000, FULL, 1000baseX_Full ),
+ PHY_SETTING( 1000, FULL, 1000baseKX_Full ),
/* 100M */
PHY_SETTING( 100, FULL, 100baseT_Full ),
PHY_SETTING( 100, FULL, 100baseT1_Full ),
diff --git a/drivers/net/phy/phylink.c b/drivers/net/phy/phylink.c
index ea82ea5..4202018 100644
--- a/drivers/net/phy/phylink.c
+++ b/drivers/net/phy/phylink.c
@@ -166,6 +166,307 @@ static const char *phylink_an_mode_str(unsigned int mode)
return mode < ARRAY_SIZE(modestr) ? modestr[mode] : "unknown";
}
+static void phylink_caps_to_linkmodes(unsigned long *linkmodes,
+ unsigned long caps)
+{
+ if (caps & MAC_SYM_PAUSE)
+ __set_bit(ETHTOOL_LINK_MODE_Pause_BIT, linkmodes);
+
+ if (caps & MAC_ASYM_PAUSE)
+ __set_bit(ETHTOOL_LINK_MODE_Asym_Pause_BIT, linkmodes);
+
+ if (caps & MAC_10HD)
+ __set_bit(ETHTOOL_LINK_MODE_10baseT_Half_BIT, linkmodes);
+
+ if (caps & MAC_10FD)
+ __set_bit(ETHTOOL_LINK_MODE_10baseT_Full_BIT, linkmodes);
+
+ if (caps & MAC_100HD) {
+ __set_bit(ETHTOOL_LINK_MODE_100baseT_Half_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_100baseFX_Half_BIT, linkmodes);
+ }
+
+ if (caps & MAC_100FD) {
+ __set_bit(ETHTOOL_LINK_MODE_100baseT_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_100baseT1_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_100baseFX_Full_BIT, linkmodes);
+ }
+
+ if (caps & MAC_1000HD)
+ __set_bit(ETHTOOL_LINK_MODE_1000baseT_Half_BIT, linkmodes);
+
+ if (caps & MAC_1000FD) {
+ __set_bit(ETHTOOL_LINK_MODE_1000baseT_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_1000baseKX_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_1000baseX_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_1000baseT1_Full_BIT, linkmodes);
+ }
+
+ if (caps & MAC_2500FD) {
+ __set_bit(ETHTOOL_LINK_MODE_2500baseT_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_2500baseX_Full_BIT, linkmodes);
+ }
+
+ if (caps & MAC_5000FD)
+ __set_bit(ETHTOOL_LINK_MODE_5000baseT_Full_BIT, linkmodes);
+
+ if (caps & MAC_10000FD) {
+ __set_bit(ETHTOOL_LINK_MODE_10000baseT_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_10000baseKX4_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_10000baseKR_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_10000baseR_FEC_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_10000baseCR_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_10000baseSR_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_10000baseLR_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_10000baseLRM_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_10000baseER_Full_BIT, linkmodes);
+ }
+
+ if (caps & MAC_25000FD) {
+ __set_bit(ETHTOOL_LINK_MODE_25000baseCR_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_25000baseKR_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_25000baseSR_Full_BIT, linkmodes);
+ }
+
+ if (caps & MAC_40000FD) {
+ __set_bit(ETHTOOL_LINK_MODE_40000baseKR4_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_40000baseCR4_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_40000baseSR4_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_40000baseLR4_Full_BIT, linkmodes);
+ }
+
+ if (caps & MAC_50000FD) {
+ __set_bit(ETHTOOL_LINK_MODE_50000baseCR2_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_50000baseKR2_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_50000baseSR2_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_50000baseKR_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_50000baseSR_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_50000baseCR_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_50000baseLR_ER_FR_Full_BIT,
+ linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_50000baseDR_Full_BIT, linkmodes);
+ }
+
+ if (caps & MAC_56000FD) {
+ __set_bit(ETHTOOL_LINK_MODE_56000baseKR4_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_56000baseCR4_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_56000baseSR4_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_56000baseLR4_Full_BIT, linkmodes);
+ }
+
+ if (caps & MAC_100000FD) {
+ __set_bit(ETHTOOL_LINK_MODE_100000baseKR4_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_100000baseSR4_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_100000baseCR4_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_100000baseLR4_ER4_Full_BIT,
+ linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_100000baseKR2_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_100000baseSR2_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_100000baseCR2_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_100000baseLR2_ER2_FR2_Full_BIT,
+ linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_100000baseDR2_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_100000baseKR_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_100000baseSR_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_100000baseLR_ER_FR_Full_BIT,
+ linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_100000baseCR_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_100000baseDR_Full_BIT, linkmodes);
+ }
+
+ if (caps & MAC_200000FD) {
+ __set_bit(ETHTOOL_LINK_MODE_200000baseKR4_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_200000baseSR4_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_200000baseLR4_ER4_FR4_Full_BIT,
+ linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_200000baseDR4_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_200000baseCR4_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_200000baseKR2_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_200000baseSR2_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_200000baseLR2_ER2_FR2_Full_BIT,
+ linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_200000baseDR2_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_200000baseCR2_Full_BIT, linkmodes);
+ }
+
+ if (caps & MAC_400000FD) {
+ __set_bit(ETHTOOL_LINK_MODE_400000baseKR8_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_400000baseSR8_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_400000baseLR8_ER8_FR8_Full_BIT,
+ linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_400000baseDR8_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_400000baseCR8_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_400000baseKR4_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_400000baseSR4_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_400000baseLR4_ER4_FR4_Full_BIT,
+ linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_400000baseDR4_Full_BIT, linkmodes);
+ __set_bit(ETHTOOL_LINK_MODE_400000baseCR4_Full_BIT, linkmodes);
+ }
+}
+
+/**
+ * phylink_get_linkmodes() - get acceptable link modes
+ * @linkmodes: ethtool linkmode mask (must be already initialised)
+ * @interface: phy interface mode defined by &typedef phy_interface_t
+ * @mac_capabilities: bitmask of MAC capabilities
+ *
+ * Set all possible pause, speed and duplex linkmodes in @linkmodes that
+ * are supported by the @interface mode and @mac_capabilities. @linkmodes
+ * must have been initialised previously.
+ */
+void phylink_get_linkmodes(unsigned long *linkmodes, phy_interface_t interface,
+ unsigned long mac_capabilities)
+{
+ unsigned long caps = MAC_SYM_PAUSE | MAC_ASYM_PAUSE;
+
+ switch (interface) {
+ case PHY_INTERFACE_MODE_USXGMII:
+ caps |= MAC_10000FD | MAC_5000FD | MAC_2500FD;
+ fallthrough;
+
+ case PHY_INTERFACE_MODE_RGMII_TXID:
+ case PHY_INTERFACE_MODE_RGMII_RXID:
+ case PHY_INTERFACE_MODE_RGMII_ID:
+ case PHY_INTERFACE_MODE_RGMII:
+ case PHY_INTERFACE_MODE_QSGMII:
+ case PHY_INTERFACE_MODE_SGMII:
+ case PHY_INTERFACE_MODE_GMII:
+ caps |= MAC_1000HD | MAC_1000FD;
+ fallthrough;
+
+ case PHY_INTERFACE_MODE_REVRMII:
+ case PHY_INTERFACE_MODE_RMII:
+ case PHY_INTERFACE_MODE_SMII:
+ case PHY_INTERFACE_MODE_REVMII:
+ case PHY_INTERFACE_MODE_MII:
+ caps |= MAC_10HD | MAC_10FD;
+ fallthrough;
+
+ case PHY_INTERFACE_MODE_100BASEX:
+ caps |= MAC_100HD | MAC_100FD;
+ break;
+
+ case PHY_INTERFACE_MODE_TBI:
+ case PHY_INTERFACE_MODE_MOCA:
+ case PHY_INTERFACE_MODE_RTBI:
+ case PHY_INTERFACE_MODE_1000BASEX:
+ caps |= MAC_1000HD;
+ fallthrough;
+ case PHY_INTERFACE_MODE_TRGMII:
+ caps |= MAC_1000FD;
+ break;
+
+ case PHY_INTERFACE_MODE_2500BASEX:
+ caps |= MAC_2500FD;
+ break;
+
+ case PHY_INTERFACE_MODE_5GBASER:
+ caps |= MAC_5000FD;
+ break;
+
+ case PHY_INTERFACE_MODE_XGMII:
+ case PHY_INTERFACE_MODE_RXAUI:
+ case PHY_INTERFACE_MODE_XAUI:
+ case PHY_INTERFACE_MODE_10GBASER:
+ case PHY_INTERFACE_MODE_10GKR:
+ caps |= MAC_10000FD;
+ break;
+
+ case PHY_INTERFACE_MODE_25GBASER:
+ caps |= MAC_25000FD;
+ break;
+
+ case PHY_INTERFACE_MODE_XLGMII:
+ caps |= MAC_40000FD;
+ break;
+
+ case PHY_INTERFACE_MODE_INTERNAL:
+ caps |= ~0;
+ break;
+
+ case PHY_INTERFACE_MODE_NA:
+ case PHY_INTERFACE_MODE_MAX:
+ break;
+ }
+
+ phylink_caps_to_linkmodes(linkmodes, caps & mac_capabilities);
+}
+EXPORT_SYMBOL_GPL(phylink_get_linkmodes);
+
+/**
+ * phylink_generic_validate() - generic validate() callback implementation
+ * @config: a pointer to a &struct phylink_config.
+ * @supported: ethtool bitmask for supported link modes.
+ * @state: a pointer to a &struct phylink_link_state.
+ *
+ * Generic implementation of the validate() callback that MAC drivers can
+ * use when they pass the range of supported interfaces and MAC capabilities.
+ * This makes use of phylink_get_linkmodes().
+ */
+void phylink_generic_validate(struct phylink_config *config,
+ unsigned long *supported,
+ struct phylink_link_state *state)
+{
+ __ETHTOOL_DECLARE_LINK_MODE_MASK(mask) = { 0, };
+
+ phylink_set_port_modes(mask);
+ phylink_set(mask, Autoneg);
+ phylink_get_linkmodes(mask, state->interface, config->mac_capabilities);
+
+ linkmode_and(supported, supported, mask);
+ linkmode_and(state->advertising, state->advertising, mask);
+}
+EXPORT_SYMBOL_GPL(phylink_generic_validate);
+
+static int phylink_validate_mac_and_pcs(struct phylink *pl,
+ unsigned long *supported,
+ struct phylink_link_state *state)
+{
+ struct phylink_pcs *pcs;
+ int ret;
+
+ /* Get the PCS for this interface mode */
+ if (pl->mac_ops->mac_select_pcs) {
+ pcs = pl->mac_ops->mac_select_pcs(pl->config, state->interface);
+ if (IS_ERR(pcs))
+ return PTR_ERR(pcs);
+ } else {
+ pcs = pl->pcs;
+ }
+
+ if (pcs) {
+ /* The PCS, if present, must be setup before phylink_create()
+ * has been called. If the ops is not initialised, print an
+ * error and backtrace rather than oopsing the kernel.
+ */
+ if (!pcs->ops) {
+ phylink_err(pl, "interface %s: uninitialised PCS\n",
+ phy_modes(state->interface));
+ dump_stack();
+ return -EINVAL;
+ }
+
+ /* Validate the link parameters with the PCS */
+ if (pcs->ops->pcs_validate) {
+ ret = pcs->ops->pcs_validate(pcs, supported, state);
+ if (ret < 0 || phylink_is_empty_linkmode(supported))
+ return -EINVAL;
+
+ /* Ensure the advertising mask is a subset of the
+ * supported mask.
+ */
+ linkmode_and(state->advertising, state->advertising,
+ supported);
+ }
+ }
+
+ /* Then validate the link parameters with the MAC */
+ pl->mac_ops->validate(pl->config, supported, state);
+
+ return phylink_is_empty_linkmode(supported) ? -EINVAL : 0;
+}
+
static int phylink_validate_any(struct phylink *pl, unsigned long *supported,
struct phylink_link_state *state)
{
@@ -181,9 +482,10 @@ static int phylink_validate_any(struct phylink *pl, unsigned long *supported,
t = *state;
t.interface = intf;
- pl->mac_ops->validate(pl->config, s, &t);
- linkmode_or(all_s, all_s, s);
- linkmode_or(all_adv, all_adv, t.advertising);
+ if (!phylink_validate_mac_and_pcs(pl, s, &t)) {
+ linkmode_or(all_s, all_s, s);
+ linkmode_or(all_adv, all_adv, t.advertising);
+ }
}
}
@@ -205,9 +507,7 @@ static int phylink_validate(struct phylink *pl, unsigned long *supported,
return -EINVAL;
}
- pl->mac_ops->validate(pl->config, supported, state);
-
- return phylink_is_empty_linkmode(supported) ? -EINVAL : 0;
+ return phylink_validate_mac_and_pcs(pl, supported, state);
}
static int phylink_parse_fixedlink(struct phylink *pl,
@@ -489,7 +789,7 @@ static void phylink_mac_pcs_an_restart(struct phylink *pl)
phylink_autoneg_inband(pl->cur_link_an_mode)) {
if (pl->pcs_ops)
pl->pcs_ops->pcs_an_restart(pl->pcs);
- else
+ else if (pl->config->legacy_pre_march2020)
pl->mac_ops->mac_an_restart(pl->config);
}
}
@@ -497,10 +797,21 @@ static void phylink_mac_pcs_an_restart(struct phylink *pl)
static void phylink_major_config(struct phylink *pl, bool restart,
const struct phylink_link_state *state)
{
+ struct phylink_pcs *pcs = NULL;
int err;
phylink_dbg(pl, "major config %s\n", phy_modes(state->interface));
+ if (pl->mac_ops->mac_select_pcs) {
+ pcs = pl->mac_ops->mac_select_pcs(pl->config, state->interface);
+ if (IS_ERR(pcs)) {
+ phylink_err(pl,
+ "mac_select_pcs unexpectedly failed: %pe\n",
+ pcs);
+ return;
+ }
+ }
+
if (pl->mac_ops->mac_prepare) {
err = pl->mac_ops->mac_prepare(pl->config, pl->cur_link_an_mode,
state->interface);
@@ -511,6 +822,12 @@ static void phylink_major_config(struct phylink *pl, bool restart,
}
}
+ /* If we have a new PCS, switch to the new PCS after preparing the MAC
+ * for the change.
+ */
+ if (pcs)
+ phylink_set_pcs(pl, pcs);
+
phylink_mac_config(pl, state);
if (pl->pcs_ops) {
@@ -550,7 +867,7 @@ static int phylink_change_inband_advert(struct phylink *pl)
if (test_bit(PHYLINK_DISABLE_STOPPED, &pl->phylink_disable_state))
return 0;
- if (!pl->pcs_ops) {
+ if (!pl->pcs_ops && pl->config->legacy_pre_march2020) {
/* Legacy method */
phylink_mac_config(pl, &pl->link_config);
phylink_mac_pcs_an_restart(pl);
@@ -601,7 +918,8 @@ static void phylink_mac_pcs_get_state(struct phylink *pl,
if (pl->pcs_ops)
pl->pcs_ops->pcs_get_state(pl->pcs, state);
- else if (pl->mac_ops->mac_pcs_get_state)
+ else if (pl->mac_ops->mac_pcs_get_state &&
+ pl->config->legacy_pre_march2020)
pl->mac_ops->mac_pcs_get_state(pl->config, state);
else
state->link = 0;
@@ -795,12 +1113,11 @@ static void phylink_resolve(struct work_struct *w)
}
phylink_major_config(pl, false, &link_state);
pl->link_config.interface = link_state.interface;
- } else if (!pl->pcs_ops) {
+ } else if (!pl->pcs_ops && pl->config->legacy_pre_march2020) {
/* The interface remains unchanged, only the speed,
* duplex or pause settings have changed. Call the
* old mac_config() method to configure the MAC/PCS
- * only if we do not have a PCS installed (an
- * unconverted user.)
+ * only if we do not have a legacy MAC driver.
*/
phylink_mac_config(pl, &link_state);
}
@@ -837,6 +1154,12 @@ static void phylink_run_resolve_and_disable(struct phylink *pl, int bit)
}
}
+static void phylink_enable_and_run_resolve(struct phylink *pl, int bit)
+{
+ clear_bit(bit, &pl->phylink_disable_state);
+ phylink_run_resolve(pl);
+}
+
static void phylink_fixed_poll(struct timer_list *t)
{
struct phylink *pl = container_of(t, struct phylink, link_poll);
@@ -896,6 +1219,14 @@ struct phylink *phylink_create(struct phylink_config *config,
struct phylink *pl;
int ret;
+ /* Validate the supplied configuration */
+ if (mac_ops->mac_select_pcs &&
+ phy_interface_empty(config->supported_interfaces)) {
+ dev_err(config->dev,
+ "phylink: error: empty supported_interfaces but mac_select_pcs() method present\n");
+ return ERR_PTR(-EINVAL);
+ }
+
pl = kzalloc(sizeof(*pl), GFP_KERNEL);
if (!pl)
return ERR_PTR(-ENOMEM);
@@ -963,9 +1294,10 @@ EXPORT_SYMBOL_GPL(phylink_create);
* @pl: a pointer to a &struct phylink returned from phylink_create()
* @pcs: a pointer to the &struct phylink_pcs
*
- * Bind the MAC PCS to phylink. This may be called after phylink_create(),
- * in mac_prepare() or mac_config() methods if it is desired to dynamically
- * change the PCS.
+ * Bind the MAC PCS to phylink. This may be called after phylink_create().
+ * If it is desired to dynamically change the PCS, then the preferred method
+ * is to use mac_select_pcs(), but it may also be called in mac_prepare()
+ * or mac_config().
*
* Please note that there are behavioural changes with the mac_config()
* callback if a PCS is present (denoting a newer setup) so removing a PCS
@@ -976,6 +1308,14 @@ void phylink_set_pcs(struct phylink *pl, struct phylink_pcs *pcs)
{
pl->pcs = pcs;
pl->pcs_ops = pcs->ops;
+
+ if (!pl->phylink_disable_state &&
+ pl->cfg_link_an_mode == MLO_AN_INBAND) {
+ if (pl->config->pcs_poll || pcs->poll)
+ mod_timer(&pl->link_poll, jiffies + HZ);
+ else
+ del_timer(&pl->link_poll);
+ }
}
EXPORT_SYMBOL_GPL(phylink_set_pcs);
@@ -1096,7 +1436,8 @@ static int phylink_bringup_phy(struct phylink *pl, struct phy_device *phy,
mutex_unlock(&phy->lock);
phylink_dbg(pl,
- "phy: setting supported %*pb advertising %*pb\n",
+ "phy: %s setting supported %*pb advertising %*pb\n",
+ phy_modes(interface),
__ETHTOOL_LINK_MODE_MASK_NBITS, pl->supported,
__ETHTOOL_LINK_MODE_MASK_NBITS, phy->advertising);
@@ -1214,6 +1555,12 @@ int phylink_fwnode_phy_connect(struct phylink *pl,
if (!phy_dev)
return -ENODEV;
+ /* Use PHY device/driver interface */
+ if (pl->link_interface == PHY_INTERFACE_MODE_NA) {
+ pl->link_interface = phy_dev->interface;
+ pl->link_config.interface = pl->link_interface;
+ }
+
ret = phy_attach_direct(pl->netdev, phy_dev, flags,
pl->link_interface);
if (ret) {
@@ -1314,8 +1661,7 @@ void phylink_start(struct phylink *pl)
*/
phylink_mac_initial_config(pl, true);
- clear_bit(PHYLINK_DISABLE_STOPPED, &pl->phylink_disable_state);
- phylink_run_resolve(pl);
+ phylink_enable_and_run_resolve(pl, PHYLINK_DISABLE_STOPPED);
if (pl->cfg_link_an_mode == MLO_AN_FIXED && pl->link_gpio) {
int irq = gpiod_to_irq(pl->link_gpio);
@@ -1456,8 +1802,7 @@ void phylink_resume(struct phylink *pl)
phylink_mac_initial_config(pl, true);
/* Re-enable and re-resolve the link parameters */
- clear_bit(PHYLINK_DISABLE_MAC_WOL, &pl->phylink_disable_state);
- phylink_run_resolve(pl);
+ phylink_enable_and_run_resolve(pl, PHYLINK_DISABLE_MAC_WOL);
} else {
phylink_start(pl);
}
@@ -2386,8 +2731,7 @@ static void phylink_sfp_link_up(void *upstream)
ASSERT_RTNL();
- clear_bit(PHYLINK_DISABLE_LINK, &pl->phylink_disable_state);
- phylink_run_resolve(pl);
+ phylink_enable_and_run_resolve(pl, PHYLINK_DISABLE_LINK);
}
/* The Broadcom BCM84881 in the Methode DM7052 is unable to provide a SGMII
@@ -2587,31 +2931,22 @@ void phylink_decode_usxgmii_word(struct phylink_link_state *state,
EXPORT_SYMBOL_GPL(phylink_decode_usxgmii_word);
/**
- * phylink_mii_c22_pcs_get_state() - read the MAC PCS state
- * @pcs: a pointer to a &struct mdio_device.
+ * phylink_mii_c22_pcs_decode_state() - Decode MAC PCS state from MII registers
* @state: a pointer to a &struct phylink_link_state.
+ * @bmsr: The value of the %MII_BMSR register
+ * @lpa: The value of the %MII_LPA register
*
* Helper for MAC PCS supporting the 802.3 clause 22 register set for
* clause 37 negotiation and/or SGMII control.
*
- * Read the MAC PCS state from the MII device configured in @config and
- * parse the Clause 37 or Cisco SGMII link partner negotiation word into
- * the phylink @state structure. This is suitable to be directly plugged
- * into the mac_pcs_get_state() member of the struct phylink_mac_ops
- * structure.
+ * Parse the Clause 37 or Cisco SGMII link partner negotiation word into
+ * the phylink @state structure. This is suitable to be used for implementing
+ * the mac_pcs_get_state() member of the struct phylink_mac_ops structure if
+ * accessing @bmsr and @lpa cannot be done with MDIO directly.
*/
-void phylink_mii_c22_pcs_get_state(struct mdio_device *pcs,
- struct phylink_link_state *state)
+void phylink_mii_c22_pcs_decode_state(struct phylink_link_state *state,
+ u16 bmsr, u16 lpa)
{
- int bmsr, lpa;
-
- bmsr = mdiodev_read(pcs, MII_BMSR);
- lpa = mdiodev_read(pcs, MII_LPA);
- if (bmsr < 0 || lpa < 0) {
- state->link = false;
- return;
- }
-
state->link = !!(bmsr & BMSR_LSTATUS);
state->an_complete = !!(bmsr & BMSR_ANEGCOMPLETE);
/* If there is no link or autonegotiation is disabled, the LP advertisement
@@ -2639,28 +2974,54 @@ void phylink_mii_c22_pcs_get_state(struct mdio_device *pcs,
break;
}
}
+EXPORT_SYMBOL_GPL(phylink_mii_c22_pcs_decode_state);
+
+/**
+ * phylink_mii_c22_pcs_get_state() - read the MAC PCS state
+ * @pcs: a pointer to a &struct mdio_device.
+ * @state: a pointer to a &struct phylink_link_state.
+ *
+ * Helper for MAC PCS supporting the 802.3 clause 22 register set for
+ * clause 37 negotiation and/or SGMII control.
+ *
+ * Read the MAC PCS state from the MII device configured in @config and
+ * parse the Clause 37 or Cisco SGMII link partner negotiation word into
+ * the phylink @state structure. This is suitable to be directly plugged
+ * into the mac_pcs_get_state() member of the struct phylink_mac_ops
+ * structure.
+ */
+void phylink_mii_c22_pcs_get_state(struct mdio_device *pcs,
+ struct phylink_link_state *state)
+{
+ int bmsr, lpa;
+
+ bmsr = mdiodev_read(pcs, MII_BMSR);
+ lpa = mdiodev_read(pcs, MII_LPA);
+ if (bmsr < 0 || lpa < 0) {
+ state->link = false;
+ return;
+ }
+
+ phylink_mii_c22_pcs_decode_state(state, bmsr, lpa);
+}
EXPORT_SYMBOL_GPL(phylink_mii_c22_pcs_get_state);
/**
- * phylink_mii_c22_pcs_set_advertisement() - configure the clause 37 PCS
+ * phylink_mii_c22_pcs_encode_advertisement() - configure the clause 37 PCS
* advertisement
- * @pcs: a pointer to a &struct mdio_device.
* @interface: the PHY interface mode being configured
* @advertising: the ethtool advertisement mask
*
* Helper for MAC PCS supporting the 802.3 clause 22 register set for
* clause 37 negotiation and/or SGMII control.
*
- * Configure the clause 37 PCS advertisement as specified by @state. This
- * does not trigger a renegotiation; phylink will do that via the
- * mac_an_restart() method of the struct phylink_mac_ops structure.
+ * Encode the clause 37 PCS advertisement as specified by @interface and
+ * @advertising.
*
- * Returns negative error code on failure to configure the advertisement,
- * zero if no change has been made, or one if the advertisement has changed.
+ * Return: The new value for @adv, or ``-EINVAL`` if it should not be changed.
*/
-int phylink_mii_c22_pcs_set_advertisement(struct mdio_device *pcs,
- phy_interface_t interface,
- const unsigned long *advertising)
+int phylink_mii_c22_pcs_encode_advertisement(phy_interface_t interface,
+ const unsigned long *advertising)
{
u16 adv;
@@ -2674,18 +3035,15 @@ int phylink_mii_c22_pcs_set_advertisement(struct mdio_device *pcs,
if (linkmode_test_bit(ETHTOOL_LINK_MODE_Asym_Pause_BIT,
advertising))
adv |= ADVERTISE_1000XPSE_ASYM;
-
- return mdiodev_modify_changed(pcs, MII_ADVERTISE, 0xffff, adv);
-
+ return adv;
case PHY_INTERFACE_MODE_SGMII:
- return mdiodev_modify_changed(pcs, MII_ADVERTISE, 0xffff, 0x0001);
-
+ return 0x0001;
default:
/* Nothing to do for other modes */
- return 0;
+ return -EINVAL;
}
}
-EXPORT_SYMBOL_GPL(phylink_mii_c22_pcs_set_advertisement);
+EXPORT_SYMBOL_GPL(phylink_mii_c22_pcs_encode_advertisement);
/**
* phylink_mii_c22_pcs_config() - configure clause 22 PCS
@@ -2703,16 +3061,18 @@ int phylink_mii_c22_pcs_config(struct mdio_device *pcs, unsigned int mode,
phy_interface_t interface,
const unsigned long *advertising)
{
- bool changed;
+ bool changed = 0;
u16 bmcr;
- int ret;
+ int ret, adv;
- ret = phylink_mii_c22_pcs_set_advertisement(pcs, interface,
- advertising);
- if (ret < 0)
- return ret;
-
- changed = ret > 0;
+ adv = phylink_mii_c22_pcs_encode_advertisement(interface, advertising);
+ if (adv >= 0) {
+ ret = mdiobus_modify_changed(pcs->bus, pcs->addr,
+ MII_ADVERTISE, 0xffff, adv);
+ if (ret < 0)
+ return ret;
+ changed = ret;
+ }
/* Ensure ISOLATE bit is disabled */
if (mode == MLO_AN_INBAND &&
@@ -2725,7 +3085,7 @@ int phylink_mii_c22_pcs_config(struct mdio_device *pcs, unsigned int mode,
if (ret < 0)
return ret;
- return changed ? 1 : 0;
+ return changed;
}
EXPORT_SYMBOL_GPL(phylink_mii_c22_pcs_config);
diff --git a/drivers/net/ppp/ppp_generic.c b/drivers/net/ppp/ppp_generic.c
index 1180a0e..4a365f1 100644
--- a/drivers/net/ppp/ppp_generic.c
+++ b/drivers/net/ppp/ppp_generic.c
@@ -69,6 +69,8 @@
#define MPHDRLEN 6 /* multilink protocol header length */
#define MPHDRLEN_SSN 4 /* ditto with short sequence numbers */
+#define PPP_PROTO_LEN 2
+
/*
* An instance of /dev/ppp can be associated with either a ppp
* interface unit or a ppp channel. In both cases, file->private_data
@@ -173,6 +175,7 @@ struct channel {
spinlock_t downl; /* protects `chan', file.xq dequeue */
struct ppp *ppp; /* ppp unit we're connected to */
struct net *chan_net; /* the net channel belongs to */
+ netns_tracker ns_tracker;
struct list_head clist; /* link in list of channels per unit */
rwlock_t upl; /* protects `ppp' and 'bridge' */
struct channel __rcu *bridge; /* "bridged" ppp channel */
@@ -497,6 +500,9 @@ static ssize_t ppp_write(struct file *file, const char __user *buf,
if (!pf)
return -ENXIO;
+ /* All PPP packets should start with the 2-byte protocol */
+ if (count < PPP_PROTO_LEN)
+ return -EINVAL;
ret = -ENOMEM;
skb = alloc_skb(count + pf->hdrlen, GFP_KERNEL);
if (!skb)
@@ -1764,7 +1770,7 @@ ppp_send_frame(struct ppp *ppp, struct sk_buff *skb)
}
++ppp->stats64.tx_packets;
- ppp->stats64.tx_bytes += skb->len - 2;
+ ppp->stats64.tx_bytes += skb->len - PPP_PROTO_LEN;
switch (proto) {
case PPP_IP:
@@ -2879,7 +2885,7 @@ int ppp_register_net_channel(struct net *net, struct ppp_channel *chan)
pch->ppp = NULL;
pch->chan = chan;
- pch->chan_net = get_net(net);
+ pch->chan_net = get_net_track(net, &pch->ns_tracker, GFP_KERNEL);
chan->ppp = pch;
init_ppp_file(&pch->file, CHANNEL);
pch->file.hdrlen = chan->hdrlen;
@@ -3519,7 +3525,7 @@ ppp_disconnect_channel(struct channel *pch)
*/
static void ppp_destroy_channel(struct channel *pch)
{
- put_net(pch->chan_net);
+ put_net_track(pch->chan_net, &pch->ns_tracker);
pch->chan_net = NULL;
atomic_dec(&channel_count);
diff --git a/drivers/net/tun.c b/drivers/net/tun.c
index 45a67e7..fed8544 100644
--- a/drivers/net/tun.c
+++ b/drivers/net/tun.c
@@ -1602,7 +1602,7 @@ static int tun_xdp_act(struct tun_struct *tun, struct bpf_prog *xdp_prog,
case XDP_PASS:
break;
default:
- bpf_warn_invalid_xdp_action(act);
+ bpf_warn_invalid_xdp_action(tun->dev, xdp_prog, act);
fallthrough;
case XDP_ABORTED:
trace_xdp_exception(tun->dev, xdp_prog, act);
diff --git a/drivers/net/usb/ax88179_178a.c b/drivers/net/usb/ax88179_178a.c
index ea8aa8c..1a627ba 100644
--- a/drivers/net/usb/ax88179_178a.c
+++ b/drivers/net/usb/ax88179_178a.c
@@ -1377,11 +1377,12 @@ static int ax88179_bind(struct usbnet *dev, struct usb_interface *intf)
dev->mii.phy_id = 0x03;
dev->mii.supports_gmii = 1;
- dev->net->features |= NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM |
- NETIF_F_RXCSUM;
+ dev->net->features |= NETIF_F_SG | NETIF_F_IP_CSUM |
+ NETIF_F_IPV6_CSUM | NETIF_F_RXCSUM | NETIF_F_TSO;
- dev->net->hw_features |= NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM |
- NETIF_F_RXCSUM;
+ dev->net->hw_features |= dev->net->features;
+
+ netif_set_gso_max_size(dev->net, 16384);
/* Enable checksum offload */
*tmp = AX_RXCOE_IP | AX_RXCOE_TCP | AX_RXCOE_UDP |
@@ -1526,17 +1527,19 @@ ax88179_tx_fixup(struct usbnet *dev, struct sk_buff *skb, gfp_t flags)
{
u32 tx_hdr1, tx_hdr2;
int frame_size = dev->maxpacket;
- int mss = skb_shinfo(skb)->gso_size;
int headroom;
void *ptr;
tx_hdr1 = skb->len;
- tx_hdr2 = mss;
+ tx_hdr2 = skb_shinfo(skb)->gso_size; /* Set TSO mss */
if (((skb->len + 8) % frame_size) == 0)
tx_hdr2 |= 0x80008000; /* Enable padding */
headroom = skb_headroom(skb) - 8;
+ if ((dev->net->features & NETIF_F_SG) && skb_linearize(skb))
+ return NULL;
+
if ((skb_header_cloned(skb) || headroom < 0) &&
pskb_expand_head(skb, headroom < 0 ? 8 : 0, 0, GFP_ATOMIC)) {
dev_kfree_skb_any(skb);
@@ -1547,6 +1550,8 @@ ax88179_tx_fixup(struct usbnet *dev, struct sk_buff *skb, gfp_t flags)
put_unaligned_le32(tx_hdr1, ptr);
put_unaligned_le32(tx_hdr2, ptr + 4);
+ usbnet_set_skb_tx_stats(skb, (skb_shinfo(skb)->gso_segs ?: 1), 0);
+
return skb;
}
diff --git a/drivers/net/usb/lan78xx.c b/drivers/net/usb/lan78xx.c
index 075f8ab..b8e20a3 100644
--- a/drivers/net/usb/lan78xx.c
+++ b/drivers/net/usb/lan78xx.c
@@ -67,7 +67,7 @@
#define DEFAULT_TSO_CSUM_ENABLE (true)
#define DEFAULT_VLAN_FILTER_ENABLE (true)
#define DEFAULT_VLAN_RX_OFFLOAD (true)
-#define TX_OVERHEAD (8)
+#define TX_ALIGNMENT (4)
#define RXW_PADDING 2
#define LAN78XX_USB_VENDOR_ID (0x0424)
@@ -92,6 +92,41 @@
WAKE_MCAST | WAKE_BCAST | \
WAKE_ARP | WAKE_MAGIC)
+#define LAN78XX_NAPI_WEIGHT 64
+
+#define TX_URB_NUM 10
+#define TX_SS_URB_NUM TX_URB_NUM
+#define TX_HS_URB_NUM TX_URB_NUM
+#define TX_FS_URB_NUM TX_URB_NUM
+
+/* A single URB buffer must be large enough to hold a complete jumbo packet
+ */
+#define TX_SS_URB_SIZE (32 * 1024)
+#define TX_HS_URB_SIZE (16 * 1024)
+#define TX_FS_URB_SIZE (10 * 1024)
+
+#define RX_SS_URB_NUM 30
+#define RX_HS_URB_NUM 10
+#define RX_FS_URB_NUM 10
+#define RX_SS_URB_SIZE TX_SS_URB_SIZE
+#define RX_HS_URB_SIZE TX_HS_URB_SIZE
+#define RX_FS_URB_SIZE TX_FS_URB_SIZE
+
+#define SS_BURST_CAP_SIZE RX_SS_URB_SIZE
+#define SS_BULK_IN_DELAY 0x2000
+#define HS_BURST_CAP_SIZE RX_HS_URB_SIZE
+#define HS_BULK_IN_DELAY 0x2000
+#define FS_BURST_CAP_SIZE RX_FS_URB_SIZE
+#define FS_BULK_IN_DELAY 0x2000
+
+#define TX_CMD_LEN 8
+#define TX_SKB_MIN_LEN (TX_CMD_LEN + ETH_HLEN)
+#define LAN78XX_TSO_SIZE(dev) ((dev)->tx_urb_size - TX_SKB_MIN_LEN)
+
+#define RX_CMD_LEN 10
+#define RX_SKB_MIN_LEN (RX_CMD_LEN + ETH_HLEN)
+#define RX_MAX_FRAME_LEN(mtu) ((mtu) + ETH_HLEN + VLAN_HLEN)
+
/* USB related defines */
#define BULK_IN_PIPE 1
#define BULK_OUT_PIPE 2
@@ -387,14 +422,22 @@ struct lan78xx_net {
struct usb_interface *intf;
void *driver_priv;
- int rx_qlen;
- int tx_qlen;
+ unsigned int tx_pend_data_len;
+ size_t n_tx_urbs;
+ size_t n_rx_urbs;
+ size_t tx_urb_size;
+ size_t rx_urb_size;
+
+ struct sk_buff_head rxq_free;
struct sk_buff_head rxq;
+ struct sk_buff_head rxq_done;
+ struct sk_buff_head rxq_overflow;
+ struct sk_buff_head txq_free;
struct sk_buff_head txq;
- struct sk_buff_head done;
struct sk_buff_head txq_pend;
- struct tasklet_struct bh;
+ struct napi_struct napi;
+
struct delayed_work wq;
int msg_enable;
@@ -406,8 +449,8 @@ struct lan78xx_net {
struct mutex phy_mutex; /* for phy access */
unsigned int pipe_in, pipe_out, pipe_intr;
- u32 hard_mtu; /* count any extra framing */
- size_t rx_urb_size; /* size for rx urbs */
+ unsigned int bulk_in_delay;
+ unsigned int burst_cap;
unsigned long flags;
@@ -445,6 +488,129 @@ static int msg_level = -1;
module_param(msg_level, int, 0);
MODULE_PARM_DESC(msg_level, "Override default message level");
+static struct sk_buff *lan78xx_get_buf(struct sk_buff_head *buf_pool)
+{
+ if (skb_queue_empty(buf_pool))
+ return NULL;
+
+ return skb_dequeue(buf_pool);
+}
+
+static void lan78xx_release_buf(struct sk_buff_head *buf_pool,
+ struct sk_buff *buf)
+{
+ buf->data = buf->head;
+ skb_reset_tail_pointer(buf);
+
+ buf->len = 0;
+ buf->data_len = 0;
+
+ skb_queue_tail(buf_pool, buf);
+}
+
+static void lan78xx_free_buf_pool(struct sk_buff_head *buf_pool)
+{
+ struct skb_data *entry;
+ struct sk_buff *buf;
+
+ while (!skb_queue_empty(buf_pool)) {
+ buf = skb_dequeue(buf_pool);
+ if (buf) {
+ entry = (struct skb_data *)buf->cb;
+ usb_free_urb(entry->urb);
+ dev_kfree_skb_any(buf);
+ }
+ }
+}
+
+static int lan78xx_alloc_buf_pool(struct sk_buff_head *buf_pool,
+ size_t n_urbs, size_t urb_size,
+ struct lan78xx_net *dev)
+{
+ struct skb_data *entry;
+ struct sk_buff *buf;
+ struct urb *urb;
+ int i;
+
+ skb_queue_head_init(buf_pool);
+
+ for (i = 0; i < n_urbs; i++) {
+ buf = alloc_skb(urb_size, GFP_ATOMIC);
+ if (!buf)
+ goto error;
+
+ if (skb_linearize(buf) != 0) {
+ dev_kfree_skb_any(buf);
+ goto error;
+ }
+
+ urb = usb_alloc_urb(0, GFP_ATOMIC);
+ if (!urb) {
+ dev_kfree_skb_any(buf);
+ goto error;
+ }
+
+ entry = (struct skb_data *)buf->cb;
+ entry->urb = urb;
+ entry->dev = dev;
+ entry->length = 0;
+ entry->num_of_packet = 0;
+
+ skb_queue_tail(buf_pool, buf);
+ }
+
+ return 0;
+
+error:
+ lan78xx_free_buf_pool(buf_pool);
+
+ return -ENOMEM;
+}
+
+static struct sk_buff *lan78xx_get_rx_buf(struct lan78xx_net *dev)
+{
+ return lan78xx_get_buf(&dev->rxq_free);
+}
+
+static void lan78xx_release_rx_buf(struct lan78xx_net *dev,
+ struct sk_buff *rx_buf)
+{
+ lan78xx_release_buf(&dev->rxq_free, rx_buf);
+}
+
+static void lan78xx_free_rx_resources(struct lan78xx_net *dev)
+{
+ lan78xx_free_buf_pool(&dev->rxq_free);
+}
+
+static int lan78xx_alloc_rx_resources(struct lan78xx_net *dev)
+{
+ return lan78xx_alloc_buf_pool(&dev->rxq_free,
+ dev->n_rx_urbs, dev->rx_urb_size, dev);
+}
+
+static struct sk_buff *lan78xx_get_tx_buf(struct lan78xx_net *dev)
+{
+ return lan78xx_get_buf(&dev->txq_free);
+}
+
+static void lan78xx_release_tx_buf(struct lan78xx_net *dev,
+ struct sk_buff *tx_buf)
+{
+ lan78xx_release_buf(&dev->txq_free, tx_buf);
+}
+
+static void lan78xx_free_tx_resources(struct lan78xx_net *dev)
+{
+ lan78xx_free_buf_pool(&dev->txq_free);
+}
+
+static int lan78xx_alloc_tx_resources(struct lan78xx_net *dev)
+{
+ return lan78xx_alloc_buf_pool(&dev->txq_free,
+ dev->n_tx_urbs, dev->tx_urb_size, dev);
+}
+
static int lan78xx_read_reg(struct lan78xx_net *dev, u32 index, u32 *data)
{
u32 *buf;
@@ -1202,6 +1368,8 @@ static int lan78xx_update_flowcontrol(struct lan78xx_net *dev, u8 duplex,
return 0;
}
+static void lan78xx_rx_urb_submit_all(struct lan78xx_net *dev);
+
static int lan78xx_mac_reset(struct lan78xx_net *dev)
{
unsigned long start_time = jiffies;
@@ -1333,7 +1501,9 @@ static int lan78xx_link_reset(struct lan78xx_net *dev)
jiffies + STAT_UPDATE_TIMER);
}
- tasklet_schedule(&dev->bh);
+ lan78xx_rx_urb_submit_all(dev);
+
+ napi_schedule(&dev->napi);
}
return 0;
@@ -2373,37 +2543,24 @@ static int unlink_urbs(struct lan78xx_net *dev, struct sk_buff_head *q)
static int lan78xx_change_mtu(struct net_device *netdev, int new_mtu)
{
struct lan78xx_net *dev = netdev_priv(netdev);
- int ll_mtu = new_mtu + netdev->hard_header_len;
- int old_hard_mtu = dev->hard_mtu;
- int old_rx_urb_size = dev->rx_urb_size;
+ int max_frame_len = RX_MAX_FRAME_LEN(new_mtu);
int ret;
/* no second zero-length packet read wanted after mtu-sized packets */
- if ((ll_mtu % dev->maxpacket) == 0)
+ if ((max_frame_len % dev->maxpacket) == 0)
return -EDOM;
ret = usb_autopm_get_interface(dev->intf);
if (ret < 0)
return ret;
- lan78xx_set_rx_max_frame_length(dev, new_mtu + VLAN_ETH_HLEN);
-
- netdev->mtu = new_mtu;
-
- dev->hard_mtu = netdev->mtu + netdev->hard_header_len;
- if (dev->rx_urb_size == old_hard_mtu) {
- dev->rx_urb_size = dev->hard_mtu;
- if (dev->rx_urb_size > old_rx_urb_size) {
- if (netif_running(dev->net)) {
- unlink_urbs(dev, &dev->rxq);
- tasklet_schedule(&dev->bh);
- }
- }
- }
+ ret = lan78xx_set_rx_max_frame_length(dev, max_frame_len);
+ if (!ret)
+ netdev->mtu = new_mtu;
usb_autopm_put_interface(dev->intf);
- return 0;
+ return ret;
}
static int lan78xx_set_mac_addr(struct net_device *netdev, void *p)
@@ -2559,6 +2716,44 @@ static void lan78xx_init_ltm(struct lan78xx_net *dev)
lan78xx_write_reg(dev, LTM_INACTIVE1, regs[5]);
}
+static int lan78xx_urb_config_init(struct lan78xx_net *dev)
+{
+ int result = 0;
+
+ switch (dev->udev->speed) {
+ case USB_SPEED_SUPER:
+ dev->rx_urb_size = RX_SS_URB_SIZE;
+ dev->tx_urb_size = TX_SS_URB_SIZE;
+ dev->n_rx_urbs = RX_SS_URB_NUM;
+ dev->n_tx_urbs = TX_SS_URB_NUM;
+ dev->bulk_in_delay = SS_BULK_IN_DELAY;
+ dev->burst_cap = SS_BURST_CAP_SIZE / SS_USB_PKT_SIZE;
+ break;
+ case USB_SPEED_HIGH:
+ dev->rx_urb_size = RX_HS_URB_SIZE;
+ dev->tx_urb_size = TX_HS_URB_SIZE;
+ dev->n_rx_urbs = RX_HS_URB_NUM;
+ dev->n_tx_urbs = TX_HS_URB_NUM;
+ dev->bulk_in_delay = HS_BULK_IN_DELAY;
+ dev->burst_cap = HS_BURST_CAP_SIZE / HS_USB_PKT_SIZE;
+ break;
+ case USB_SPEED_FULL:
+ dev->rx_urb_size = RX_FS_URB_SIZE;
+ dev->tx_urb_size = TX_FS_URB_SIZE;
+ dev->n_rx_urbs = RX_FS_URB_NUM;
+ dev->n_tx_urbs = TX_FS_URB_NUM;
+ dev->bulk_in_delay = FS_BULK_IN_DELAY;
+ dev->burst_cap = FS_BURST_CAP_SIZE / FS_USB_PKT_SIZE;
+ break;
+ default:
+ netdev_warn(dev->net, "USB bus speed not supported\n");
+ result = -EIO;
+ break;
+ }
+
+ return result;
+}
+
static int lan78xx_start_hw(struct lan78xx_net *dev, u32 reg, u32 hw_enable)
{
return lan78xx_update_reg(dev, reg, hw_enable, hw_enable);
@@ -2766,28 +2961,11 @@ static int lan78xx_reset(struct lan78xx_net *dev)
/* Init LTM */
lan78xx_init_ltm(dev);
- if (dev->udev->speed == USB_SPEED_SUPER) {
- buf = DEFAULT_BURST_CAP_SIZE / SS_USB_PKT_SIZE;
- dev->rx_urb_size = DEFAULT_BURST_CAP_SIZE;
- dev->rx_qlen = 4;
- dev->tx_qlen = 4;
- } else if (dev->udev->speed == USB_SPEED_HIGH) {
- buf = DEFAULT_BURST_CAP_SIZE / HS_USB_PKT_SIZE;
- dev->rx_urb_size = DEFAULT_BURST_CAP_SIZE;
- dev->rx_qlen = RX_MAX_QUEUE_MEMORY / dev->rx_urb_size;
- dev->tx_qlen = RX_MAX_QUEUE_MEMORY / dev->hard_mtu;
- } else {
- buf = DEFAULT_BURST_CAP_SIZE / FS_USB_PKT_SIZE;
- dev->rx_urb_size = DEFAULT_BURST_CAP_SIZE;
- dev->rx_qlen = 4;
- dev->tx_qlen = 4;
- }
-
- ret = lan78xx_write_reg(dev, BURST_CAP, buf);
+ ret = lan78xx_write_reg(dev, BURST_CAP, dev->burst_cap);
if (ret < 0)
return ret;
- ret = lan78xx_write_reg(dev, BULK_IN_DLY, DEFAULT_BULK_IN_DELAY);
+ ret = lan78xx_write_reg(dev, BULK_IN_DLY, dev->bulk_in_delay);
if (ret < 0)
return ret;
@@ -2900,7 +3078,7 @@ static int lan78xx_reset(struct lan78xx_net *dev)
return ret;
ret = lan78xx_set_rx_max_frame_length(dev,
- dev->net->mtu + VLAN_ETH_HLEN);
+ RX_MAX_FRAME_LEN(dev->net->mtu));
return ret;
}
@@ -2980,6 +3158,8 @@ static int lan78xx_open(struct net_device *net)
dev->link_on = false;
+ napi_enable(&dev->napi);
+
lan78xx_defer_kevent(dev, EVENT_LINK_RESET);
done:
mutex_unlock(&dev->dev_mutex);
@@ -3013,15 +3193,16 @@ static void lan78xx_terminate_urbs(struct lan78xx_net *dev)
dev->wait = NULL;
remove_wait_queue(&unlink_wakeup, &wait);
- while (!skb_queue_empty(&dev->done)) {
- struct skb_data *entry;
- struct sk_buff *skb;
+ /* empty Rx done, Rx overflow and Tx pend queues
+ */
+ while (!skb_queue_empty(&dev->rxq_done)) {
+ struct sk_buff *skb = skb_dequeue(&dev->rxq_done);
- skb = skb_dequeue(&dev->done);
- entry = (struct skb_data *)(skb->cb);
- usb_free_urb(entry->urb);
- dev_kfree_skb(skb);
+ lan78xx_release_rx_buf(dev, skb);
}
+
+ skb_queue_purge(&dev->rxq_overflow);
+ skb_queue_purge(&dev->txq_pend);
}
static int lan78xx_stop(struct net_device *net)
@@ -3037,7 +3218,7 @@ static int lan78xx_stop(struct net_device *net)
clear_bit(EVENT_DEV_OPEN, &dev->flags);
netif_stop_queue(net);
- tasklet_kill(&dev->bh);
+ napi_disable(&dev->napi);
lan78xx_terminate_urbs(dev);
@@ -3073,48 +3254,6 @@ static int lan78xx_stop(struct net_device *net)
return 0;
}
-static struct sk_buff *lan78xx_tx_prep(struct lan78xx_net *dev,
- struct sk_buff *skb, gfp_t flags)
-{
- u32 tx_cmd_a, tx_cmd_b;
- void *ptr;
-
- if (skb_cow_head(skb, TX_OVERHEAD)) {
- dev_kfree_skb_any(skb);
- return NULL;
- }
-
- if (skb_linearize(skb)) {
- dev_kfree_skb_any(skb);
- return NULL;
- }
-
- tx_cmd_a = (u32)(skb->len & TX_CMD_A_LEN_MASK_) | TX_CMD_A_FCS_;
-
- if (skb->ip_summed == CHECKSUM_PARTIAL)
- tx_cmd_a |= TX_CMD_A_IPE_ | TX_CMD_A_TPE_;
-
- tx_cmd_b = 0;
- if (skb_is_gso(skb)) {
- u16 mss = max(skb_shinfo(skb)->gso_size, TX_CMD_B_MSS_MIN_);
-
- tx_cmd_b = (mss << TX_CMD_B_MSS_SHIFT_) & TX_CMD_B_MSS_MASK_;
-
- tx_cmd_a |= TX_CMD_A_LSO_;
- }
-
- if (skb_vlan_tag_present(skb)) {
- tx_cmd_a |= TX_CMD_A_IVTG_;
- tx_cmd_b |= skb_vlan_tag_get(skb) & TX_CMD_B_VTAG_MASK_;
- }
-
- ptr = skb_push(skb, 8);
- put_unaligned_le32(tx_cmd_a, ptr);
- put_unaligned_le32(tx_cmd_b, ptr + 4);
-
- return skb;
-}
-
static enum skb_state defer_bh(struct lan78xx_net *dev, struct sk_buff *skb,
struct sk_buff_head *list, enum skb_state state)
{
@@ -3128,12 +3267,13 @@ static enum skb_state defer_bh(struct lan78xx_net *dev, struct sk_buff *skb,
__skb_unlink(skb, list);
spin_unlock(&list->lock);
- spin_lock(&dev->done.lock);
+ spin_lock(&dev->rxq_done.lock);
- __skb_queue_tail(&dev->done, skb);
- if (skb_queue_len(&dev->done) == 1)
- tasklet_schedule(&dev->bh);
- spin_unlock_irqrestore(&dev->done.lock, flags);
+ __skb_queue_tail(&dev->rxq_done, skb);
+ if (skb_queue_len(&dev->rxq_done) == 1)
+ napi_schedule(&dev->napi);
+
+ spin_unlock_irqrestore(&dev->rxq_done.lock, flags);
return old_state;
}
@@ -3148,7 +3288,7 @@ static void tx_complete(struct urb *urb)
dev->net->stats.tx_packets += entry->num_of_packet;
dev->net->stats.tx_bytes += entry->length;
} else {
- dev->net->stats.tx_errors++;
+ dev->net->stats.tx_errors += entry->num_of_packet;
switch (urb->status) {
case -EPIPE:
@@ -3181,7 +3321,15 @@ static void tx_complete(struct urb *urb)
usb_autopm_put_interface_async(dev->intf);
- defer_bh(dev, skb, &dev->txq, tx_done);
+ skb_unlink(skb, &dev->txq);
+
+ lan78xx_release_tx_buf(dev, skb);
+
+ /* Re-schedule NAPI if Tx data pending but no URBs in progress.
+ */
+ if (skb_queue_empty(&dev->txq) &&
+ !skb_queue_empty(&dev->txq_pend))
+ napi_schedule(&dev->napi);
}
static void lan78xx_queue_skb(struct sk_buff_head *list,
@@ -3193,36 +3341,97 @@ static void lan78xx_queue_skb(struct sk_buff_head *list,
entry->state = state;
}
+static unsigned int lan78xx_tx_urb_space(struct lan78xx_net *dev)
+{
+ return skb_queue_len(&dev->txq_free) * dev->tx_urb_size;
+}
+
+static unsigned int lan78xx_tx_pend_data_len(struct lan78xx_net *dev)
+{
+ return dev->tx_pend_data_len;
+}
+
+static void lan78xx_tx_pend_skb_add(struct lan78xx_net *dev,
+ struct sk_buff *skb,
+ unsigned int *tx_pend_data_len)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&dev->txq_pend.lock, flags);
+
+ __skb_queue_tail(&dev->txq_pend, skb);
+
+ dev->tx_pend_data_len += skb->len;
+ *tx_pend_data_len = dev->tx_pend_data_len;
+
+ spin_unlock_irqrestore(&dev->txq_pend.lock, flags);
+}
+
+static void lan78xx_tx_pend_skb_head_add(struct lan78xx_net *dev,
+ struct sk_buff *skb,
+ unsigned int *tx_pend_data_len)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&dev->txq_pend.lock, flags);
+
+ __skb_queue_head(&dev->txq_pend, skb);
+
+ dev->tx_pend_data_len += skb->len;
+ *tx_pend_data_len = dev->tx_pend_data_len;
+
+ spin_unlock_irqrestore(&dev->txq_pend.lock, flags);
+}
+
+static void lan78xx_tx_pend_skb_get(struct lan78xx_net *dev,
+ struct sk_buff **skb,
+ unsigned int *tx_pend_data_len)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&dev->txq_pend.lock, flags);
+
+ *skb = __skb_dequeue(&dev->txq_pend);
+ if (*skb)
+ dev->tx_pend_data_len -= (*skb)->len;
+ *tx_pend_data_len = dev->tx_pend_data_len;
+
+ spin_unlock_irqrestore(&dev->txq_pend.lock, flags);
+}
+
static netdev_tx_t
lan78xx_start_xmit(struct sk_buff *skb, struct net_device *net)
{
struct lan78xx_net *dev = netdev_priv(net);
- struct sk_buff *skb2 = NULL;
+ unsigned int tx_pend_data_len;
if (test_bit(EVENT_DEV_ASLEEP, &dev->flags))
schedule_delayed_work(&dev->wq, 0);
- if (skb) {
- skb_tx_timestamp(skb);
- skb2 = lan78xx_tx_prep(dev, skb, GFP_ATOMIC);
+ skb_tx_timestamp(skb);
+
+ lan78xx_tx_pend_skb_add(dev, skb, &tx_pend_data_len);
+
+ /* Set up a Tx URB if none is in progress */
+
+ if (skb_queue_empty(&dev->txq))
+ napi_schedule(&dev->napi);
+
+ /* Stop stack Tx queue if we have enough data to fill
+ * all the free Tx URBs.
+ */
+ if (tx_pend_data_len > lan78xx_tx_urb_space(dev)) {
+ netif_stop_queue(net);
+
+ netif_dbg(dev, hw, dev->net, "tx data len: %u, urb space %u",
+ tx_pend_data_len, lan78xx_tx_urb_space(dev));
+
+ /* Kick off transmission of pending data */
+
+ if (!skb_queue_empty(&dev->txq_free))
+ napi_schedule(&dev->napi);
}
- if (skb2) {
- skb_queue_tail(&dev->txq_pend, skb2);
-
- /* throttle TX patch at slower than SUPER SPEED USB */
- if ((dev->udev->speed < USB_SPEED_SUPER) &&
- (skb_queue_len(&dev->txq_pend) > 10))
- netif_stop_queue(net);
- } else {
- netif_dbg(dev, tx_err, dev->net,
- "lan78xx_tx_prep return NULL\n");
- dev->net->stats.tx_errors++;
- dev->net->stats.tx_dropped++;
- }
-
- tasklet_schedule(&dev->bh);
-
return NETDEV_TX_OK;
}
@@ -3278,9 +3487,6 @@ static int lan78xx_bind(struct lan78xx_net *dev, struct usb_interface *intf)
goto out1;
}
- dev->net->hard_header_len += TX_OVERHEAD;
- dev->hard_mtu = dev->net->mtu + dev->net->hard_header_len;
-
/* Init all registers */
ret = lan78xx_reset(dev);
if (ret) {
@@ -3359,8 +3565,6 @@ static void lan78xx_rx_vlan_offload(struct lan78xx_net *dev,
static void lan78xx_skb_return(struct lan78xx_net *dev, struct sk_buff *skb)
{
- int status;
-
dev->net->stats.rx_packets++;
dev->net->stats.rx_bytes += skb->len;
@@ -3373,21 +3577,21 @@ static void lan78xx_skb_return(struct lan78xx_net *dev, struct sk_buff *skb)
if (skb_defer_rx_timestamp(skb))
return;
- status = netif_rx(skb);
- if (status != NET_RX_SUCCESS)
- netif_dbg(dev, rx_err, dev->net,
- "netif_rx status %d\n", status);
+ napi_gro_receive(&dev->napi, skb);
}
-static int lan78xx_rx(struct lan78xx_net *dev, struct sk_buff *skb)
+static int lan78xx_rx(struct lan78xx_net *dev, struct sk_buff *skb,
+ int budget, int *work_done)
{
- if (skb->len < dev->net->hard_header_len)
+ if (skb->len < RX_SKB_MIN_LEN)
return 0;
+ /* Extract frames from the URB buffer and pass each one to
+ * the stack in a new NAPI SKB.
+ */
while (skb->len > 0) {
u32 rx_cmd_a, rx_cmd_b, align_count, size;
u16 rx_cmd_c;
- struct sk_buff *skb2;
unsigned char *packet;
rx_cmd_a = get_unaligned_le32(skb->data);
@@ -3409,41 +3613,36 @@ static int lan78xx_rx(struct lan78xx_net *dev, struct sk_buff *skb)
netif_dbg(dev, rx_err, dev->net,
"Error rx_cmd_a=0x%08x", rx_cmd_a);
} else {
- /* last frame in this batch */
- if (skb->len == size) {
- lan78xx_rx_csum_offload(dev, skb,
- rx_cmd_a, rx_cmd_b);
- lan78xx_rx_vlan_offload(dev, skb,
- rx_cmd_a, rx_cmd_b);
+ u32 frame_len = size - ETH_FCS_LEN;
+ struct sk_buff *skb2;
- skb_trim(skb, skb->len - 4); /* remove fcs */
- skb->truesize = size + sizeof(struct sk_buff);
-
- return 1;
- }
-
- skb2 = skb_clone(skb, GFP_ATOMIC);
- if (unlikely(!skb2)) {
- netdev_warn(dev->net, "Error allocating skb");
+ skb2 = napi_alloc_skb(&dev->napi, frame_len);
+ if (!skb2)
return 0;
- }
- skb2->len = size;
- skb2->data = packet;
- skb_set_tail_pointer(skb2, size);
+ memcpy(skb2->data, packet, frame_len);
+
+ skb_put(skb2, frame_len);
lan78xx_rx_csum_offload(dev, skb2, rx_cmd_a, rx_cmd_b);
lan78xx_rx_vlan_offload(dev, skb2, rx_cmd_a, rx_cmd_b);
- skb_trim(skb2, skb2->len - 4); /* remove fcs */
- skb2->truesize = size + sizeof(struct sk_buff);
-
- lan78xx_skb_return(dev, skb2);
+ /* Processing of the URB buffer must complete once
+ * it has started. If the NAPI work budget is exhausted
+ * while frames remain they are added to the overflow
+ * queue for delivery in the next NAPI polling cycle.
+ */
+ if (*work_done < budget) {
+ lan78xx_skb_return(dev, skb2);
+ ++(*work_done);
+ } else {
+ skb_queue_tail(&dev->rxq_overflow, skb2);
+ }
}
skb_pull(skb, size);
- /* padding bytes before the next frame starts */
+ /* skip padding bytes before the next frame starts */
if (skb->len)
skb_pull(skb, align_count);
}
@@ -3451,85 +3650,13 @@ static int lan78xx_rx(struct lan78xx_net *dev, struct sk_buff *skb)
return 1;
}
-static inline void rx_process(struct lan78xx_net *dev, struct sk_buff *skb)
+static inline void rx_process(struct lan78xx_net *dev, struct sk_buff *skb,
+ int budget, int *work_done)
{
- if (!lan78xx_rx(dev, skb)) {
+ if (!lan78xx_rx(dev, skb, budget, work_done)) {
+ netif_dbg(dev, rx_err, dev->net, "drop\n");
dev->net->stats.rx_errors++;
- goto done;
}
-
- if (skb->len) {
- lan78xx_skb_return(dev, skb);
- return;
- }
-
- netif_dbg(dev, rx_err, dev->net, "drop\n");
- dev->net->stats.rx_errors++;
-done:
- skb_queue_tail(&dev->done, skb);
-}
-
-static void rx_complete(struct urb *urb);
-
-static int rx_submit(struct lan78xx_net *dev, struct urb *urb, gfp_t flags)
-{
- struct sk_buff *skb;
- struct skb_data *entry;
- unsigned long lockflags;
- size_t size = dev->rx_urb_size;
- int ret = 0;
-
- skb = netdev_alloc_skb_ip_align(dev->net, size);
- if (!skb) {
- usb_free_urb(urb);
- return -ENOMEM;
- }
-
- entry = (struct skb_data *)skb->cb;
- entry->urb = urb;
- entry->dev = dev;
- entry->length = 0;
-
- usb_fill_bulk_urb(urb, dev->udev, dev->pipe_in,
- skb->data, size, rx_complete, skb);
-
- spin_lock_irqsave(&dev->rxq.lock, lockflags);
-
- if (netif_device_present(dev->net) &&
- netif_running(dev->net) &&
- !test_bit(EVENT_RX_HALT, &dev->flags) &&
- !test_bit(EVENT_DEV_ASLEEP, &dev->flags)) {
- ret = usb_submit_urb(urb, GFP_ATOMIC);
- switch (ret) {
- case 0:
- lan78xx_queue_skb(&dev->rxq, skb, rx_start);
- break;
- case -EPIPE:
- lan78xx_defer_kevent(dev, EVENT_RX_HALT);
- break;
- case -ENODEV:
- case -ENOENT:
- netif_dbg(dev, ifdown, dev->net, "device gone\n");
- netif_device_detach(dev->net);
- break;
- case -EHOSTUNREACH:
- ret = -ENOLINK;
- break;
- default:
- netif_dbg(dev, rx_err, dev->net,
- "rx submit, %d\n", ret);
- tasklet_schedule(&dev->bh);
- }
- } else {
- netif_dbg(dev, ifdown, dev->net, "rx: stopped\n");
- ret = -ENOLINK;
- }
- spin_unlock_irqrestore(&dev->rxq.lock, lockflags);
- if (ret) {
- dev_kfree_skb_any(skb);
- usb_free_urb(urb);
- }
- return ret;
}
static void rx_complete(struct urb *urb)
@@ -3540,13 +3667,18 @@ static void rx_complete(struct urb *urb)
int urb_status = urb->status;
enum skb_state state;
+ netif_dbg(dev, rx_status, dev->net,
+ "rx done: status %d", urb->status);
+
skb_put(skb, urb->actual_length);
state = rx_done;
- entry->urb = NULL;
+
+ if (urb != entry->urb)
+ netif_warn(dev, rx_err, dev->net, "URB pointer mismatch");
switch (urb_status) {
case 0:
- if (skb->len < dev->net->hard_header_len) {
+ if (skb->len < RX_SKB_MIN_LEN) {
state = rx_cleanup;
dev->net->stats.rx_errors++;
dev->net->stats.rx_length_errors++;
@@ -3564,16 +3696,12 @@ static void rx_complete(struct urb *urb)
netif_dbg(dev, ifdown, dev->net,
"rx shutdown, code %d\n", urb_status);
state = rx_cleanup;
- entry->urb = urb;
- urb = NULL;
break;
case -EPROTO:
case -ETIME:
case -EILSEQ:
dev->net->stats.rx_errors++;
state = rx_cleanup;
- entry->urb = urb;
- urb = NULL;
break;
/* data overrun ... flush fifo? */
@@ -3589,203 +3717,327 @@ static void rx_complete(struct urb *urb)
}
state = defer_bh(dev, skb, &dev->rxq, state);
+}
- if (urb) {
- if (netif_running(dev->net) &&
- !test_bit(EVENT_RX_HALT, &dev->flags) &&
- state != unlink_start) {
- rx_submit(dev, urb, GFP_ATOMIC);
- return;
+static int rx_submit(struct lan78xx_net *dev, struct sk_buff *skb, gfp_t flags)
+{
+ struct skb_data *entry = (struct skb_data *)skb->cb;
+ size_t size = dev->rx_urb_size;
+ struct urb *urb = entry->urb;
+ unsigned long lockflags;
+ int ret = 0;
+
+ usb_fill_bulk_urb(urb, dev->udev, dev->pipe_in,
+ skb->data, size, rx_complete, skb);
+
+ spin_lock_irqsave(&dev->rxq.lock, lockflags);
+
+ if (netif_device_present(dev->net) &&
+ netif_running(dev->net) &&
+ !test_bit(EVENT_RX_HALT, &dev->flags) &&
+ !test_bit(EVENT_DEV_ASLEEP, &dev->flags)) {
+ ret = usb_submit_urb(urb, flags);
+ switch (ret) {
+ case 0:
+ lan78xx_queue_skb(&dev->rxq, skb, rx_start);
+ break;
+ case -EPIPE:
+ lan78xx_defer_kevent(dev, EVENT_RX_HALT);
+ break;
+ case -ENODEV:
+ case -ENOENT:
+ netif_dbg(dev, ifdown, dev->net, "device gone\n");
+ netif_device_detach(dev->net);
+ break;
+ case -EHOSTUNREACH:
+ ret = -ENOLINK;
+ napi_schedule(&dev->napi);
+ break;
+ default:
+ netif_dbg(dev, rx_err, dev->net,
+ "rx submit, %d\n", ret);
+ napi_schedule(&dev->napi);
+ break;
}
- usb_free_urb(urb);
+ } else {
+ netif_dbg(dev, ifdown, dev->net, "rx: stopped\n");
+ ret = -ENOLINK;
}
- netif_dbg(dev, rx_err, dev->net, "no read resubmitted\n");
+ spin_unlock_irqrestore(&dev->rxq.lock, lockflags);
+
+ if (ret)
+ lan78xx_release_rx_buf(dev, skb);
+
+ return ret;
+}
+
+static void lan78xx_rx_urb_submit_all(struct lan78xx_net *dev)
+{
+ struct sk_buff *rx_buf;
+
+ /* Ensure the maximum number of Rx URBs is submitted
+ */
+ while ((rx_buf = lan78xx_get_rx_buf(dev)) != NULL) {
+ if (rx_submit(dev, rx_buf, GFP_ATOMIC) != 0)
+ break;
+ }
+}
+
+static void lan78xx_rx_urb_resubmit(struct lan78xx_net *dev,
+ struct sk_buff *rx_buf)
+{
+ /* reset SKB data pointers */
+
+ rx_buf->data = rx_buf->head;
+ skb_reset_tail_pointer(rx_buf);
+ rx_buf->len = 0;
+ rx_buf->data_len = 0;
+
+ rx_submit(dev, rx_buf, GFP_ATOMIC);
+}
+
+static void lan78xx_fill_tx_cmd_words(struct sk_buff *skb, u8 *buffer)
+{
+ u32 tx_cmd_a;
+ u32 tx_cmd_b;
+
+ tx_cmd_a = (u32)(skb->len & TX_CMD_A_LEN_MASK_) | TX_CMD_A_FCS_;
+
+ if (skb->ip_summed == CHECKSUM_PARTIAL)
+ tx_cmd_a |= TX_CMD_A_IPE_ | TX_CMD_A_TPE_;
+
+ tx_cmd_b = 0;
+ if (skb_is_gso(skb)) {
+ u16 mss = max(skb_shinfo(skb)->gso_size, TX_CMD_B_MSS_MIN_);
+
+ tx_cmd_b = (mss << TX_CMD_B_MSS_SHIFT_) & TX_CMD_B_MSS_MASK_;
+
+ tx_cmd_a |= TX_CMD_A_LSO_;
+ }
+
+ if (skb_vlan_tag_present(skb)) {
+ tx_cmd_a |= TX_CMD_A_IVTG_;
+ tx_cmd_b |= skb_vlan_tag_get(skb) & TX_CMD_B_VTAG_MASK_;
+ }
+
+ put_unaligned_le32(tx_cmd_a, buffer);
+ put_unaligned_le32(tx_cmd_b, buffer + 4);
+}
+
+static struct skb_data *lan78xx_tx_buf_fill(struct lan78xx_net *dev,
+ struct sk_buff *tx_buf)
+{
+ struct skb_data *entry = (struct skb_data *)tx_buf->cb;
+ int remain = dev->tx_urb_size;
+ u8 *tx_data = tx_buf->data;
+ u32 urb_len = 0;
+
+ entry->num_of_packet = 0;
+ entry->length = 0;
+
+ /* Work through the pending SKBs and copy the data of each SKB into
+ * the URB buffer if there room for all the SKB data.
+ *
+ * There must be at least DST+SRC+TYPE in the SKB (with padding enabled)
+ */
+ while (remain >= TX_SKB_MIN_LEN) {
+ unsigned int pending_bytes;
+ unsigned int align_bytes;
+ struct sk_buff *skb;
+ unsigned int len;
+
+ lan78xx_tx_pend_skb_get(dev, &skb, &pending_bytes);
+
+ if (!skb)
+ break;
+
+ align_bytes = (TX_ALIGNMENT - (urb_len % TX_ALIGNMENT)) %
+ TX_ALIGNMENT;
+ len = align_bytes + TX_CMD_LEN + skb->len;
+ if (len > remain) {
+ lan78xx_tx_pend_skb_head_add(dev, skb, &pending_bytes);
+ break;
+ }
+
+ tx_data += align_bytes;
+
+ lan78xx_fill_tx_cmd_words(skb, tx_data);
+ tx_data += TX_CMD_LEN;
+
+ len = skb->len;
+ if (skb_copy_bits(skb, 0, tx_data, len) < 0) {
+ struct net_device_stats *stats = &dev->net->stats;
+
+ stats->tx_dropped++;
+ dev_kfree_skb_any(skb);
+ tx_data -= TX_CMD_LEN;
+ continue;
+ }
+
+ tx_data += len;
+ entry->length += len;
+ entry->num_of_packet += skb_shinfo(skb)->gso_segs ?: 1;
+
+ dev_kfree_skb_any(skb);
+
+ urb_len = (u32)(tx_data - (u8 *)tx_buf->data);
+
+ remain = dev->tx_urb_size - urb_len;
+ }
+
+ skb_put(tx_buf, urb_len);
+
+ return entry;
}
static void lan78xx_tx_bh(struct lan78xx_net *dev)
{
- int length;
- struct urb *urb = NULL;
- struct skb_data *entry;
- unsigned long flags;
- struct sk_buff_head *tqp = &dev->txq_pend;
- struct sk_buff *skb, *skb2;
int ret;
- int count, pos;
- int skb_totallen, pkt_cnt;
- skb_totallen = 0;
- pkt_cnt = 0;
- count = 0;
- length = 0;
- spin_lock_irqsave(&tqp->lock, flags);
- skb_queue_walk(tqp, skb) {
- if (skb_is_gso(skb)) {
- if (!skb_queue_is_first(tqp, skb)) {
- /* handle previous packets first */
- break;
- }
- count = 1;
- length = skb->len - TX_OVERHEAD;
- __skb_unlink(skb, tqp);
- spin_unlock_irqrestore(&tqp->lock, flags);
- goto gso_skb;
- }
+ /* Start the stack Tx queue if it was stopped
+ */
+ netif_tx_lock(dev->net);
+ if (netif_queue_stopped(dev->net)) {
+ if (lan78xx_tx_pend_data_len(dev) < lan78xx_tx_urb_space(dev))
+ netif_wake_queue(dev->net);
+ }
+ netif_tx_unlock(dev->net);
- if ((skb_totallen + skb->len) > MAX_SINGLE_PACKET_SIZE)
+ /* Go through the Tx pending queue and set up URBs to transfer
+ * the data to the device. Stop if no more pending data or URBs,
+ * or if an error occurs when a URB is submitted.
+ */
+ do {
+ struct skb_data *entry;
+ struct sk_buff *tx_buf;
+ unsigned long flags;
+
+ if (skb_queue_empty(&dev->txq_pend))
break;
- skb_totallen = skb->len + roundup(skb_totallen, sizeof(u32));
- pkt_cnt++;
- }
- spin_unlock_irqrestore(&tqp->lock, flags);
- /* copy to a single skb */
- skb = alloc_skb(skb_totallen, GFP_ATOMIC);
- if (!skb)
- goto drop;
+ tx_buf = lan78xx_get_tx_buf(dev);
+ if (!tx_buf)
+ break;
- skb_put(skb, skb_totallen);
+ entry = lan78xx_tx_buf_fill(dev, tx_buf);
- for (count = pos = 0; count < pkt_cnt; count++) {
- skb2 = skb_dequeue(tqp);
- if (skb2) {
- length += (skb2->len - TX_OVERHEAD);
- memcpy(skb->data + pos, skb2->data, skb2->len);
- pos += roundup(skb2->len, sizeof(u32));
- dev_kfree_skb(skb2);
+ spin_lock_irqsave(&dev->txq.lock, flags);
+ ret = usb_autopm_get_interface_async(dev->intf);
+ if (ret < 0) {
+ spin_unlock_irqrestore(&dev->txq.lock, flags);
+ goto out;
}
- }
-gso_skb:
- urb = usb_alloc_urb(0, GFP_ATOMIC);
- if (!urb)
- goto drop;
+ usb_fill_bulk_urb(entry->urb, dev->udev, dev->pipe_out,
+ tx_buf->data, tx_buf->len, tx_complete,
+ tx_buf);
- entry = (struct skb_data *)skb->cb;
- entry->urb = urb;
- entry->dev = dev;
- entry->length = length;
- entry->num_of_packet = count;
-
- spin_lock_irqsave(&dev->txq.lock, flags);
- ret = usb_autopm_get_interface_async(dev->intf);
- if (ret < 0) {
- spin_unlock_irqrestore(&dev->txq.lock, flags);
- goto drop;
- }
-
- usb_fill_bulk_urb(urb, dev->udev, dev->pipe_out,
- skb->data, skb->len, tx_complete, skb);
-
- if (length % dev->maxpacket == 0) {
- /* send USB_ZERO_PACKET */
- urb->transfer_flags |= URB_ZERO_PACKET;
- }
+ if (tx_buf->len % dev->maxpacket == 0) {
+ /* send USB_ZERO_PACKET */
+ entry->urb->transfer_flags |= URB_ZERO_PACKET;
+ }
#ifdef CONFIG_PM
- /* if this triggers the device is still a sleep */
- if (test_bit(EVENT_DEV_ASLEEP, &dev->flags)) {
- /* transmission will be done in resume */
- usb_anchor_urb(urb, &dev->deferred);
- /* no use to process more packets */
- netif_stop_queue(dev->net);
- usb_put_urb(urb);
- spin_unlock_irqrestore(&dev->txq.lock, flags);
- netdev_dbg(dev->net, "Delaying transmission for resumption\n");
- return;
- }
-#endif
-
- ret = usb_submit_urb(urb, GFP_ATOMIC);
- switch (ret) {
- case 0:
- netif_trans_update(dev->net);
- lan78xx_queue_skb(&dev->txq, skb, tx_start);
- if (skb_queue_len(&dev->txq) >= dev->tx_qlen)
+ /* if device is asleep stop outgoing packet processing */
+ if (test_bit(EVENT_DEV_ASLEEP, &dev->flags)) {
+ usb_anchor_urb(entry->urb, &dev->deferred);
netif_stop_queue(dev->net);
- break;
- case -EPIPE:
- netif_stop_queue(dev->net);
- lan78xx_defer_kevent(dev, EVENT_TX_HALT);
- usb_autopm_put_interface_async(dev->intf);
- break;
- case -ENODEV:
- case -ENOENT:
- netif_dbg(dev, tx_err, dev->net,
- "tx: submit urb err %d (disconnected?)", ret);
- netif_device_detach(dev->net);
- break;
- default:
- usb_autopm_put_interface_async(dev->intf);
- netif_dbg(dev, tx_err, dev->net,
- "tx: submit urb err %d\n", ret);
- break;
- }
-
- spin_unlock_irqrestore(&dev->txq.lock, flags);
-
- if (ret) {
- netif_dbg(dev, tx_err, dev->net, "drop, code %d\n", ret);
-drop:
- dev->net->stats.tx_dropped++;
- if (skb)
- dev_kfree_skb_any(skb);
- usb_free_urb(urb);
- } else {
- netif_dbg(dev, tx_queued, dev->net,
- "> tx, len %d, type 0x%x\n", length, skb->protocol);
- }
-}
-
-static void lan78xx_rx_bh(struct lan78xx_net *dev)
-{
- struct urb *urb;
- int i;
-
- if (skb_queue_len(&dev->rxq) < dev->rx_qlen) {
- for (i = 0; i < 10; i++) {
- if (skb_queue_len(&dev->rxq) >= dev->rx_qlen)
- break;
- urb = usb_alloc_urb(0, GFP_ATOMIC);
- if (urb)
- if (rx_submit(dev, urb, GFP_ATOMIC) == -ENOLINK)
- return;
- }
-
- if (skb_queue_len(&dev->rxq) < dev->rx_qlen)
- tasklet_schedule(&dev->bh);
- }
- if (skb_queue_len(&dev->txq) < dev->tx_qlen)
- netif_wake_queue(dev->net);
-}
-
-static void lan78xx_bh(struct tasklet_struct *t)
-{
- struct lan78xx_net *dev = from_tasklet(dev, t, bh);
- struct sk_buff *skb;
- struct skb_data *entry;
-
- while ((skb = skb_dequeue(&dev->done))) {
- entry = (struct skb_data *)(skb->cb);
- switch (entry->state) {
- case rx_done:
- entry->state = rx_cleanup;
- rx_process(dev, skb);
- continue;
- case tx_done:
- usb_free_urb(entry->urb);
- dev_kfree_skb(skb);
- continue;
- case rx_cleanup:
- usb_free_urb(entry->urb);
- dev_kfree_skb(skb);
- continue;
- default:
- netdev_dbg(dev->net, "skb state %d\n", entry->state);
+ spin_unlock_irqrestore(&dev->txq.lock, flags);
+ netdev_dbg(dev->net,
+ "Delaying transmission for resumption\n");
return;
}
+#endif
+ ret = usb_submit_urb(entry->urb, GFP_ATOMIC);
+ switch (ret) {
+ case 0:
+ netif_trans_update(dev->net);
+ lan78xx_queue_skb(&dev->txq, tx_buf, tx_start);
+ break;
+ case -EPIPE:
+ netif_stop_queue(dev->net);
+ lan78xx_defer_kevent(dev, EVENT_TX_HALT);
+ usb_autopm_put_interface_async(dev->intf);
+ break;
+ case -ENODEV:
+ case -ENOENT:
+ netif_dbg(dev, tx_err, dev->net,
+ "tx submit urb err %d (disconnected?)", ret);
+ netif_device_detach(dev->net);
+ break;
+ default:
+ usb_autopm_put_interface_async(dev->intf);
+ netif_dbg(dev, tx_err, dev->net,
+ "tx submit urb err %d\n", ret);
+ break;
+ }
+
+ spin_unlock_irqrestore(&dev->txq.lock, flags);
+
+ if (ret) {
+ netdev_warn(dev->net, "failed to tx urb %d\n", ret);
+out:
+ dev->net->stats.tx_dropped += entry->num_of_packet;
+ lan78xx_release_tx_buf(dev, tx_buf);
+ }
+ } while (ret == 0);
+}
+
+static int lan78xx_bh(struct lan78xx_net *dev, int budget)
+{
+ struct sk_buff_head done;
+ struct sk_buff *rx_buf;
+ struct skb_data *entry;
+ unsigned long flags;
+ int work_done = 0;
+
+ /* Pass frames received in the last NAPI cycle before
+ * working on newly completed URBs.
+ */
+ while (!skb_queue_empty(&dev->rxq_overflow)) {
+ lan78xx_skb_return(dev, skb_dequeue(&dev->rxq_overflow));
+ ++work_done;
}
+ /* Take a snapshot of the done queue and move items to a
+ * temporary queue. Rx URB completions will continue to add
+ * to the done queue.
+ */
+ __skb_queue_head_init(&done);
+
+ spin_lock_irqsave(&dev->rxq_done.lock, flags);
+ skb_queue_splice_init(&dev->rxq_done, &done);
+ spin_unlock_irqrestore(&dev->rxq_done.lock, flags);
+
+ /* Extract receive frames from completed URBs and
+ * pass them to the stack. Re-submit each completed URB.
+ */
+ while ((work_done < budget) &&
+ (rx_buf = __skb_dequeue(&done))) {
+ entry = (struct skb_data *)(rx_buf->cb);
+ switch (entry->state) {
+ case rx_done:
+ rx_process(dev, rx_buf, budget, &work_done);
+ break;
+ case rx_cleanup:
+ break;
+ default:
+ netdev_dbg(dev->net, "rx buf state %d\n",
+ entry->state);
+ break;
+ }
+
+ lan78xx_rx_urb_resubmit(dev, rx_buf);
+ }
+
+ /* If budget was consumed before processing all the URBs put them
+ * back on the front of the done queue. They will be first to be
+ * processed in the next NAPI cycle.
+ */
+ spin_lock_irqsave(&dev->rxq_done.lock, flags);
+ skb_queue_splice(&done, &dev->rxq_done);
+ spin_unlock_irqrestore(&dev->rxq_done.lock, flags);
+
if (netif_device_present(dev->net) && netif_running(dev->net)) {
/* reset update timer delta */
if (timer_pending(&dev->stat_monitor) && (dev->delta != 1)) {
@@ -3794,12 +4046,61 @@ static void lan78xx_bh(struct tasklet_struct *t)
jiffies + STAT_UPDATE_TIMER);
}
- if (!skb_queue_empty(&dev->txq_pend))
- lan78xx_tx_bh(dev);
+ /* Submit all free Rx URBs */
if (!test_bit(EVENT_RX_HALT, &dev->flags))
- lan78xx_rx_bh(dev);
+ lan78xx_rx_urb_submit_all(dev);
+
+ /* Submit new Tx URBs */
+
+ lan78xx_tx_bh(dev);
}
+
+ return work_done;
+}
+
+static int lan78xx_poll(struct napi_struct *napi, int budget)
+{
+ struct lan78xx_net *dev = container_of(napi, struct lan78xx_net, napi);
+ int result = budget;
+ int work_done;
+
+ /* Don't do any work if the device is suspended */
+
+ if (test_bit(EVENT_DEV_ASLEEP, &dev->flags)) {
+ napi_complete_done(napi, 0);
+ return 0;
+ }
+
+ /* Process completed URBs and submit new URBs */
+
+ work_done = lan78xx_bh(dev, budget);
+
+ if (work_done < budget) {
+ napi_complete_done(napi, work_done);
+
+ /* Start a new polling cycle if data was received or
+ * data is waiting to be transmitted.
+ */
+ if (!skb_queue_empty(&dev->rxq_done)) {
+ napi_schedule(napi);
+ } else if (netif_carrier_ok(dev->net)) {
+ if (skb_queue_empty(&dev->txq) &&
+ !skb_queue_empty(&dev->txq_pend)) {
+ napi_schedule(napi);
+ } else {
+ netif_tx_lock(dev->net);
+ if (netif_queue_stopped(dev->net)) {
+ netif_wake_queue(dev->net);
+ napi_schedule(napi);
+ }
+ netif_tx_unlock(dev->net);
+ }
+ }
+ result = work_done;
+ }
+
+ return result;
}
static void lan78xx_delayedwork(struct work_struct *work)
@@ -3845,7 +4146,7 @@ static void lan78xx_delayedwork(struct work_struct *work)
status);
} else {
clear_bit(EVENT_RX_HALT, &dev->flags);
- tasklet_schedule(&dev->bh);
+ napi_schedule(&dev->napi);
}
}
@@ -3939,6 +4240,8 @@ static void lan78xx_disconnect(struct usb_interface *intf)
set_bit(EVENT_DEV_DISCONNECT, &dev->flags);
+ netif_napi_del(&dev->napi);
+
udev = interface_to_usbdev(intf);
net = dev->net;
@@ -3963,6 +4266,9 @@ static void lan78xx_disconnect(struct usb_interface *intf)
lan78xx_unbind(dev, intf);
+ lan78xx_free_tx_resources(dev);
+ lan78xx_free_rx_resources(dev);
+
usb_kill_urb(dev->urb_intr);
usb_free_urb(dev->urb_intr);
@@ -3975,14 +4281,16 @@ static void lan78xx_tx_timeout(struct net_device *net, unsigned int txqueue)
struct lan78xx_net *dev = netdev_priv(net);
unlink_urbs(dev, &dev->txq);
- tasklet_schedule(&dev->bh);
+ napi_schedule(&dev->napi);
}
static netdev_features_t lan78xx_features_check(struct sk_buff *skb,
struct net_device *netdev,
netdev_features_t features)
{
- if (skb->len + TX_OVERHEAD > MAX_SINGLE_PACKET_SIZE)
+ struct lan78xx_net *dev = netdev_priv(netdev);
+
+ if (skb->len > LAN78XX_TSO_SIZE(dev))
features &= ~NETIF_F_GSO_MASK;
features = vlan_features_check(skb, features);
@@ -4048,12 +4356,31 @@ static int lan78xx_probe(struct usb_interface *intf,
skb_queue_head_init(&dev->rxq);
skb_queue_head_init(&dev->txq);
- skb_queue_head_init(&dev->done);
+ skb_queue_head_init(&dev->rxq_done);
skb_queue_head_init(&dev->txq_pend);
+ skb_queue_head_init(&dev->rxq_overflow);
mutex_init(&dev->phy_mutex);
mutex_init(&dev->dev_mutex);
- tasklet_setup(&dev->bh, lan78xx_bh);
+ ret = lan78xx_urb_config_init(dev);
+ if (ret < 0)
+ goto out2;
+
+ ret = lan78xx_alloc_tx_resources(dev);
+ if (ret < 0)
+ goto out2;
+
+ ret = lan78xx_alloc_rx_resources(dev);
+ if (ret < 0)
+ goto out3;
+
+ /* MTU range: 68 - 9000 */
+ netdev->max_mtu = MAX_SINGLE_PACKET_SIZE;
+
+ netif_set_gso_max_size(netdev, LAN78XX_TSO_SIZE(dev));
+
+ netif_napi_add(netdev, &dev->napi, lan78xx_poll, LAN78XX_NAPI_WEIGHT);
+
INIT_DELAYED_WORK(&dev->wq, lan78xx_delayedwork);
init_usb_anchor(&dev->deferred);
@@ -4068,27 +4395,27 @@ static int lan78xx_probe(struct usb_interface *intf,
if (intf->cur_altsetting->desc.bNumEndpoints < 3) {
ret = -ENODEV;
- goto out2;
+ goto out4;
}
dev->pipe_in = usb_rcvbulkpipe(udev, BULK_IN_PIPE);
ep_blkin = usb_pipe_endpoint(udev, dev->pipe_in);
if (!ep_blkin || !usb_endpoint_is_bulk_in(&ep_blkin->desc)) {
ret = -ENODEV;
- goto out2;
+ goto out4;
}
dev->pipe_out = usb_sndbulkpipe(udev, BULK_OUT_PIPE);
ep_blkout = usb_pipe_endpoint(udev, dev->pipe_out);
if (!ep_blkout || !usb_endpoint_is_bulk_out(&ep_blkout->desc)) {
ret = -ENODEV;
- goto out2;
+ goto out4;
}
ep_intr = &intf->cur_altsetting->endpoint[2];
if (!usb_endpoint_is_int_in(&ep_intr->desc)) {
ret = -ENODEV;
- goto out2;
+ goto out4;
}
dev->pipe_intr = usb_rcvintpipe(dev->udev,
@@ -4096,30 +4423,25 @@ static int lan78xx_probe(struct usb_interface *intf,
ret = lan78xx_bind(dev, intf);
if (ret < 0)
- goto out2;
-
- if (netdev->mtu > (dev->hard_mtu - netdev->hard_header_len))
- netdev->mtu = dev->hard_mtu - netdev->hard_header_len;
-
- /* MTU range: 68 - 9000 */
- netdev->max_mtu = MAX_SINGLE_PACKET_SIZE;
- netif_set_gso_max_size(netdev, MAX_SINGLE_PACKET_SIZE - MAX_HEADER);
+ goto out4;
period = ep_intr->desc.bInterval;
maxp = usb_maxpacket(dev->udev, dev->pipe_intr, 0);
buf = kmalloc(maxp, GFP_KERNEL);
- if (buf) {
- dev->urb_intr = usb_alloc_urb(0, GFP_KERNEL);
- if (!dev->urb_intr) {
- ret = -ENOMEM;
- kfree(buf);
- goto out3;
- } else {
- usb_fill_int_urb(dev->urb_intr, dev->udev,
- dev->pipe_intr, buf, maxp,
- intr_complete, dev, period);
- dev->urb_intr->transfer_flags |= URB_FREE_BUFFER;
- }
+ if (!buf) {
+ ret = -ENOMEM;
+ goto out5;
+ }
+
+ dev->urb_intr = usb_alloc_urb(0, GFP_KERNEL);
+ if (!dev->urb_intr) {
+ ret = -ENOMEM;
+ goto out6;
+ } else {
+ usb_fill_int_urb(dev->urb_intr, dev->udev,
+ dev->pipe_intr, buf, maxp,
+ intr_complete, dev, period);
+ dev->urb_intr->transfer_flags |= URB_FREE_BUFFER;
}
dev->maxpacket = usb_maxpacket(dev->udev, dev->pipe_out, 1);
@@ -4127,7 +4449,7 @@ static int lan78xx_probe(struct usb_interface *intf,
/* Reject broken descriptors. */
if (dev->maxpacket == 0) {
ret = -ENODEV;
- goto out4;
+ goto out6;
}
/* driver requires remote-wakeup capability during autosuspend. */
@@ -4135,12 +4457,12 @@ static int lan78xx_probe(struct usb_interface *intf,
ret = lan78xx_phy_init(dev);
if (ret < 0)
- goto out4;
+ goto out7;
ret = register_netdev(netdev);
if (ret != 0) {
netif_err(dev, probe, netdev, "couldn't register the device\n");
- goto out5;
+ goto out8;
}
usb_set_intfdata(intf, dev);
@@ -4155,12 +4477,19 @@ static int lan78xx_probe(struct usb_interface *intf,
return 0;
-out5:
+out8:
phy_disconnect(netdev->phydev);
-out4:
+out7:
usb_free_urb(dev->urb_intr);
-out3:
+out6:
+ kfree(buf);
+out5:
lan78xx_unbind(dev, intf);
+out4:
+ netif_napi_del(&dev->napi);
+ lan78xx_free_rx_resources(dev);
+out3:
+ lan78xx_free_tx_resources(dev);
out2:
free_netdev(netdev);
out1:
@@ -4581,8 +4910,7 @@ static bool lan78xx_submit_deferred_urbs(struct lan78xx_net *dev)
if (!netif_device_present(dev->net) ||
!netif_carrier_ok(dev->net) ||
pipe_halted) {
- usb_free_urb(urb);
- dev_kfree_skb(skb);
+ lan78xx_release_tx_buf(dev, skb);
continue;
}
@@ -4592,15 +4920,14 @@ static bool lan78xx_submit_deferred_urbs(struct lan78xx_net *dev)
netif_trans_update(dev->net);
lan78xx_queue_skb(&dev->txq, skb, tx_start);
} else {
- usb_free_urb(urb);
- dev_kfree_skb(skb);
-
if (ret == -EPIPE) {
netif_stop_queue(dev->net);
pipe_halted = true;
} else if (ret == -ENODEV) {
netif_device_detach(dev->net);
}
+
+ lan78xx_release_tx_buf(dev, skb);
}
}
@@ -4632,8 +4959,7 @@ static int lan78xx_resume(struct usb_interface *intf)
if (ret < 0) {
if (ret == -ENODEV)
netif_device_detach(dev->net);
-
- netdev_warn(dev->net, "Failed to submit intr URB");
+ netdev_warn(dev->net, "Failed to submit intr URB");
}
}
@@ -4652,14 +4978,14 @@ static int lan78xx_resume(struct usb_interface *intf)
if (!pipe_halted &&
netif_device_present(dev->net) &&
- (skb_queue_len(&dev->txq) < dev->tx_qlen))
+ (lan78xx_tx_pend_data_len(dev) < lan78xx_tx_urb_space(dev)))
netif_start_queue(dev->net);
ret = lan78xx_start_tx_path(dev);
if (ret < 0)
goto out;
- tasklet_schedule(&dev->bh);
+ napi_schedule(&dev->napi);
if (!timer_pending(&dev->stat_monitor)) {
dev->delta = 1;
diff --git a/drivers/net/usb/mcs7830.c b/drivers/net/usb/mcs7830.c
index 326cc4e..fdda061 100644
--- a/drivers/net/usb/mcs7830.c
+++ b/drivers/net/usb/mcs7830.c
@@ -108,8 +108,16 @@ static const char driver_name[] = "MOSCHIP usb-ethernet driver";
static int mcs7830_get_reg(struct usbnet *dev, u16 index, u16 size, void *data)
{
- return usbnet_read_cmd(dev, MCS7830_RD_BREQ, MCS7830_RD_BMREQ,
- 0x0000, index, data, size);
+ int ret;
+
+ ret = usbnet_read_cmd(dev, MCS7830_RD_BREQ, MCS7830_RD_BMREQ,
+ 0x0000, index, data, size);
+ if (ret < 0)
+ return ret;
+ else if (ret < size)
+ return -ENODATA;
+
+ return ret;
}
static int mcs7830_set_reg(struct usbnet *dev, u16 index, u16 size, const void *data)
diff --git a/drivers/net/usb/pegasus.c b/drivers/net/usb/pegasus.c
index c4cd40b..feb247e 100644
--- a/drivers/net/usb/pegasus.c
+++ b/drivers/net/usb/pegasus.c
@@ -493,11 +493,11 @@ static void read_bulk_callback(struct urb *urb)
goto goon;
rx_status = buf[count - 2];
- if (rx_status & 0x1e) {
+ if (rx_status & 0x1c) {
netif_dbg(pegasus, rx_err, net,
"RX packet error %x\n", rx_status);
net->stats.rx_errors++;
- if (rx_status & 0x06) /* long or runt */
+ if (rx_status & 0x04) /* runt */
net->stats.rx_length_errors++;
if (rx_status & 0x08)
net->stats.rx_crc_errors++;
diff --git a/drivers/net/usb/r8152.c b/drivers/net/usb/r8152.c
index 3085e81..ee41088c 100644
--- a/drivers/net/usb/r8152.c
+++ b/drivers/net/usb/r8152.c
@@ -9018,7 +9018,9 @@ static int rtl8152_set_tunable(struct net_device *netdev,
}
static void rtl8152_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct r8152 *tp = netdev_priv(netdev);
@@ -9027,7 +9029,9 @@ static void rtl8152_get_ringparam(struct net_device *netdev,
}
static int rtl8152_set_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct r8152 *tp = netdev_priv(netdev);
@@ -9638,9 +9642,12 @@ static int rtl8152_probe(struct usb_interface *intf,
netdev->hw_features &= ~NETIF_F_RXCSUM;
}
- if (udev->parent &&
- le16_to_cpu(udev->parent->descriptor.idVendor) == VENDOR_ID_LENOVO) {
- tp->lenovo_macpassthru = 1;
+ if (le16_to_cpu(udev->descriptor.idVendor) == VENDOR_ID_LENOVO) {
+ switch (le16_to_cpu(udev->descriptor.idProduct)) {
+ case DEVICE_ID_THINKPAD_THUNDERBOLT3_DOCK_GEN2:
+ case DEVICE_ID_THINKPAD_USB_C_DOCK_GEN2:
+ tp->lenovo_macpassthru = 1;
+ }
}
if (le16_to_cpu(udev->descriptor.bcdDevice) == 0x3011 && udev->serial &&
diff --git a/drivers/net/usb/rndis_host.c b/drivers/net/usb/rndis_host.c
index 4a84f90..247f58c 100644
--- a/drivers/net/usb/rndis_host.c
+++ b/drivers/net/usb/rndis_host.c
@@ -609,6 +609,11 @@ static const struct usb_device_id products [] = {
USB_CLASS_COMM, 2 /* ACM */, 0x0ff),
.driver_info = (unsigned long) &rndis_poll_status_info,
}, {
+ /* Hytera Communications DMR radios' "Radio to PC Network" */
+ USB_VENDOR_AND_INTERFACE_INFO(0x238b,
+ USB_CLASS_COMM, 2 /* ACM */, 0x0ff),
+ .driver_info = (unsigned long)&rndis_info,
+}, {
/* RNDIS is MSFT's un-official variant of CDC ACM */
USB_INTERFACE_INFO(USB_CLASS_COMM, 2 /* ACM */, 0x0ff),
.driver_info = (unsigned long) &rndis_info,
diff --git a/drivers/net/veth.c b/drivers/net/veth.c
index 2acdb8a..354a963 100644
--- a/drivers/net/veth.c
+++ b/drivers/net/veth.c
@@ -134,29 +134,22 @@ static void veth_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *inf
static void veth_get_strings(struct net_device *dev, u32 stringset, u8 *buf)
{
- char *p = (char *)buf;
+ u8 *p = buf;
int i, j;
switch(stringset) {
case ETH_SS_STATS:
memcpy(p, ðtool_stats_keys, sizeof(ethtool_stats_keys));
p += sizeof(ethtool_stats_keys);
- for (i = 0; i < dev->real_num_rx_queues; i++) {
- for (j = 0; j < VETH_RQ_STATS_LEN; j++) {
- snprintf(p, ETH_GSTRING_LEN,
- "rx_queue_%u_%.18s",
- i, veth_rq_stats_desc[j].desc);
- p += ETH_GSTRING_LEN;
- }
- }
- for (i = 0; i < dev->real_num_tx_queues; i++) {
- for (j = 0; j < VETH_TQ_STATS_LEN; j++) {
- snprintf(p, ETH_GSTRING_LEN,
- "tx_queue_%u_%.18s",
- i, veth_tq_stats_desc[j].desc);
- p += ETH_GSTRING_LEN;
- }
- }
+ for (i = 0; i < dev->real_num_rx_queues; i++)
+ for (j = 0; j < VETH_RQ_STATS_LEN; j++)
+ ethtool_sprintf(&p, "rx_queue_%u_%.18s",
+ i, veth_rq_stats_desc[j].desc);
+
+ for (i = 0; i < dev->real_num_tx_queues; i++)
+ for (j = 0; j < VETH_TQ_STATS_LEN; j++)
+ ethtool_sprintf(&p, "tx_queue_%u_%.18s",
+ i, veth_tq_stats_desc[j].desc);
break;
}
}
@@ -342,7 +335,6 @@ static netdev_tx_t veth_xmit(struct sk_buff *skb, struct net_device *dev)
*/
use_napi = rcu_access_pointer(rq->napi) &&
veth_skb_is_eligible_for_gro(dev, rcv, skb);
- skb_record_rx_queue(skb, rxq);
}
skb_tx_timestamp(skb);
@@ -651,7 +643,7 @@ static struct xdp_frame *veth_xdp_rcv_one(struct veth_rq *rq,
rcu_read_unlock();
goto xdp_xmit;
default:
- bpf_warn_invalid_xdp_action(act);
+ bpf_warn_invalid_xdp_action(rq->dev, xdp_prog, act);
fallthrough;
case XDP_ABORTED:
trace_xdp_exception(rq->dev, xdp_prog, act);
@@ -801,7 +793,7 @@ static struct sk_buff *veth_xdp_rcv_skb(struct veth_rq *rq,
rcu_read_unlock();
goto xdp_xmit;
default:
- bpf_warn_invalid_xdp_action(act);
+ bpf_warn_invalid_xdp_action(rq->dev, xdp_prog, act);
fallthrough;
case XDP_ABORTED:
trace_xdp_exception(rq->dev, xdp_prog, act);
@@ -1693,8 +1685,8 @@ static int veth_newlink(struct net *src_net, struct net_device *dev,
if (ifmp && (dev->ifindex != 0))
peer->ifindex = ifmp->ifi_index;
- peer->gso_max_size = dev->gso_max_size;
- peer->gso_max_segs = dev->gso_max_segs;
+ netif_set_gso_max_size(peer, dev->gso_max_size);
+ netif_set_gso_max_segs(peer, dev->gso_max_segs);
err = register_netdevice(peer);
put_net(net);
diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index b107835..569eecf 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -812,7 +812,7 @@ static struct sk_buff *receive_small(struct net_device *dev,
rcu_read_unlock();
goto xdp_xmit;
default:
- bpf_warn_invalid_xdp_action(act);
+ bpf_warn_invalid_xdp_action(vi->dev, xdp_prog, act);
fallthrough;
case XDP_ABORTED:
trace_xdp_exception(vi->dev, xdp_prog, act);
@@ -1022,7 +1022,7 @@ static struct sk_buff *receive_mergeable(struct net_device *dev,
rcu_read_unlock();
goto xdp_xmit;
default:
- bpf_warn_invalid_xdp_action(act);
+ bpf_warn_invalid_xdp_action(vi->dev, xdp_prog, act);
fallthrough;
case XDP_ABORTED:
trace_xdp_exception(vi->dev, xdp_prog, act);
@@ -2171,7 +2171,9 @@ static void virtnet_cpu_notif_remove(struct virtnet_info *vi)
}
static void virtnet_get_ringparam(struct net_device *dev,
- struct ethtool_ringparam *ring)
+ struct ethtool_ringparam *ring,
+ struct kernel_ethtool_ringparam *kernel_ring,
+ struct netlink_ext_ack *extack)
{
struct virtnet_info *vi = netdev_priv(dev);
@@ -2691,7 +2693,7 @@ static void virtnet_tx_timeout(struct net_device *dev, unsigned int txqueue)
netdev_err(dev, "TX timeout on queue: %u, sq: %s, vq: 0x%x, name: %s, %u usecs ago\n",
txqueue, sq->name, sq->vq->index, sq->vq->name,
- jiffies_to_usecs(jiffies - txq->trans_start));
+ jiffies_to_usecs(jiffies - READ_ONCE(txq->trans_start)));
}
static const struct net_device_ops virtnet_netdev = {
diff --git a/drivers/net/vmxnet3/vmxnet3_drv.c b/drivers/net/vmxnet3/vmxnet3_drv.c
index fd407c0..d9d90ba 100644
--- a/drivers/net/vmxnet3/vmxnet3_drv.c
+++ b/drivers/net/vmxnet3/vmxnet3_drv.c
@@ -3159,14 +3159,14 @@ vmxnet3_change_mtu(struct net_device *netdev, int new_mtu)
static void
-vmxnet3_declare_features(struct vmxnet3_adapter *adapter, bool dma64)
+vmxnet3_declare_features(struct vmxnet3_adapter *adapter)
{
struct net_device *netdev = adapter->netdev;
netdev->hw_features = NETIF_F_SG | NETIF_F_RXCSUM |
NETIF_F_HW_CSUM | NETIF_F_HW_VLAN_CTAG_TX |
NETIF_F_HW_VLAN_CTAG_RX | NETIF_F_TSO | NETIF_F_TSO6 |
- NETIF_F_LRO;
+ NETIF_F_LRO | NETIF_F_HIGHDMA;
if (VMXNET3_VERSION_GE_4(adapter)) {
netdev->hw_features |= NETIF_F_GSO_UDP_TUNNEL |
@@ -3179,8 +3179,6 @@ vmxnet3_declare_features(struct vmxnet3_adapter *adapter, bool dma64)
NETIF_F_GSO_UDP_TUNNEL_CSUM;
}
- if (dma64)
- netdev->hw_features |= NETIF_F_HIGHDMA;
netdev->vlan_features = netdev->hw_features &
~(NETIF_F_HW_VLAN_CTAG_TX |
NETIF_F_HW_VLAN_CTAG_RX);
@@ -3397,7 +3395,6 @@ vmxnet3_probe_device(struct pci_dev *pdev,
#endif
};
int err;
- bool dma64;
u32 ver;
struct net_device *netdev;
struct vmxnet3_adapter *adapter;
@@ -3439,15 +3436,10 @@ vmxnet3_probe_device(struct pci_dev *pdev,
adapter->rx_ring_size = VMXNET3_DEF_RX_RING_SIZE;
adapter->rx_ring2_size = VMXNET3_DEF_RX_RING2_SIZE;
- if (dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)) == 0) {
- dma64 = true;
- } else {
- err = dma_set_mask(&pdev->dev, DMA_BIT_MASK(32));
- if (err) {
- dev_err(&pdev->dev, "dma_set_mask failed\n");
- goto err_set_mask;
- }
- dma64 = false;
+ err = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64));
+ if (err) {
+ dev_err(&pdev->dev, "dma_set_mask failed\n");
+ goto err_set_mask;
}
spin_lock_init(&adapter->cmd_lock);
@@ -3614,7 +3606,7 @@ vmxnet3_probe_device(struct pci_dev *pdev,
}
SET_NETDEV_DEV(netdev, &pdev->dev);
- vmxnet3_declare_features(adapter, dma64);
+ vmxnet3_declare_features(adapter);
adapter->rxdata_desc_size = VMXNET3_VERSION_GE_3(adapter) ?
VMXNET3_DEF_RXDATA_DESC_SIZE : 0;
diff --git a/drivers/net/vmxnet3/vmxnet3_ethtool.c b/drivers/net/vmxnet3/vmxnet3_ethtool.c
index 16f3a20..3172d46 100644
--- a/drivers/net/vmxnet3/vmxnet3_ethtool.c
+++ b/drivers/net/vmxnet3/vmxnet3_ethtool.c
@@ -575,10 +575,11 @@ vmxnet3_get_link_ksettings(struct net_device *netdev,
return 0;
}
-
static void
vmxnet3_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *param)
+ struct ethtool_ringparam *param,
+ struct kernel_ethtool_ringparam *kernel_param,
+ struct netlink_ext_ack *extack)
{
struct vmxnet3_adapter *adapter = netdev_priv(netdev);
@@ -595,10 +596,11 @@ vmxnet3_get_ringparam(struct net_device *netdev,
param->rx_jumbo_pending = adapter->rx_ring2_size;
}
-
static int
vmxnet3_set_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *param)
+ struct ethtool_ringparam *param,
+ struct kernel_ethtool_ringparam *kernel_param,
+ struct netlink_ext_ack *extack)
{
struct vmxnet3_adapter *adapter = netdev_priv(netdev);
u32 new_tx_ring_size, new_rx_ring_size, new_rx_ring2_size;
diff --git a/drivers/net/vrf.c b/drivers/net/vrf.c
index b2242a0..e0b1ab9 100644
--- a/drivers/net/vrf.c
+++ b/drivers/net/vrf.c
@@ -34,6 +34,7 @@
#include <net/addrconf.h>
#include <net/l3mdev.h>
#include <net/fib_rules.h>
+#include <net/sch_generic.h>
#include <net/netns/generic.h>
#include <net/netfilter/nf_conntrack.h>
@@ -814,9 +815,9 @@ static void vrf_rt6_release(struct net_device *dev, struct net_vrf *vrf)
*/
if (rt6) {
dst = &rt6->dst;
- dev_put(dst->dev);
+ dev_replace_track(dst->dev, net->loopback_dev,
+ &dst->dev_tracker, GFP_KERNEL);
dst->dev = net->loopback_dev;
- dev_hold(dst->dev);
dst_release(dst);
}
}
@@ -1061,9 +1062,9 @@ static void vrf_rtable_release(struct net_device *dev, struct net_vrf *vrf)
*/
if (rth) {
dst = &rth->dst;
- dev_put(dst->dev);
+ dev_replace_track(dst->dev, net->loopback_dev,
+ &dst->dev_tracker, GFP_KERNEL);
dst->dev = net->loopback_dev;
- dev_hold(dst->dev);
dst_release(dst);
}
}
diff --git a/drivers/net/vxlan.c b/drivers/net/vxlan.c
index 141635a..359d167 100644
--- a/drivers/net/vxlan.c
+++ b/drivers/net/vxlan.c
@@ -17,6 +17,7 @@
#include <linux/ethtool.h>
#include <net/arp.h>
#include <net/ndisc.h>
+#include <net/gro.h>
#include <net/ipv6_stubs.h>
#include <net/ip.h>
#include <net/icmp.h>
@@ -3233,7 +3234,6 @@ static const struct net_device_ops vxlan_netdev_ether_ops = {
.ndo_fdb_dump = vxlan_fdb_dump,
.ndo_fdb_get = vxlan_fdb_get,
.ndo_fill_metadata_dst = vxlan_fill_metadata_dst,
- .ndo_change_proto_down = dev_change_proto_down_generic,
};
static const struct net_device_ops vxlan_netdev_raw_ops = {
@@ -3304,7 +3304,7 @@ static void vxlan_setup(struct net_device *dev)
dev->hw_features |= NETIF_F_RXCSUM;
dev->hw_features |= NETIF_F_GSO_SOFTWARE;
netif_keep_dst(dev);
- dev->priv_flags |= IFF_NO_QUEUE;
+ dev->priv_flags |= IFF_NO_QUEUE | IFF_CHANGE_PROTO_DOWN;
/* MTU range: 68 - 65535 */
dev->min_mtu = ETH_MIN_MTU;
@@ -3747,7 +3747,7 @@ static int vxlan_config_validate(struct net *src_net, struct vxlan_config *conf,
if (!conf->dst_port) {
if (conf->flags & VXLAN_F_GPE)
- conf->dst_port = htons(4790); /* IANA VXLAN-GPE port */
+ conf->dst_port = htons(IANA_VXLAN_GPE_UDP_PORT);
else
conf->dst_port = htons(vxlan_port);
}
@@ -3810,8 +3810,8 @@ static void vxlan_config_apply(struct net_device *dev,
if (lowerdev) {
dst->remote_ifindex = conf->remote_ifindex;
- dev->gso_max_size = lowerdev->gso_max_size;
- dev->gso_max_segs = lowerdev->gso_max_segs;
+ netif_set_gso_max_size(dev, lowerdev->gso_max_size);
+ netif_set_gso_max_segs(dev, lowerdev->gso_max_segs);
needed_headroom = lowerdev->hard_header_len;
needed_headroom += lowerdev->needed_headroom;
diff --git a/drivers/net/wan/fsl_ucc_hdlc.c b/drivers/net/wan/fsl_ucc_hdlc.c
index cda1b4c..5ae2d27b 100644
--- a/drivers/net/wan/fsl_ucc_hdlc.c
+++ b/drivers/net/wan/fsl_ucc_hdlc.c
@@ -306,9 +306,8 @@ static int uhdlc_init(struct ucc_hdlc_private *priv)
else
bd_status = R_E_S | R_I_S | R_W_S;
- iowrite16be(bd_status, &priv->rx_bd_base[i].status);
- iowrite32be(priv->dma_rx_addr + i * MAX_RX_BUF_LENGTH,
- &priv->rx_bd_base[i].buf);
+ priv->rx_bd_base[i].status = cpu_to_be16(bd_status);
+ priv->rx_bd_base[i].buf = cpu_to_be32(priv->dma_rx_addr + i * MAX_RX_BUF_LENGTH);
}
for (i = 0; i < TX_BD_RING_LEN; i++) {
@@ -317,10 +316,10 @@ static int uhdlc_init(struct ucc_hdlc_private *priv)
else
bd_status = T_I_S | T_TC_S | T_W_S;
- iowrite16be(bd_status, &priv->tx_bd_base[i].status);
- iowrite32be(priv->dma_tx_addr + i * MAX_RX_BUF_LENGTH,
- &priv->tx_bd_base[i].buf);
+ priv->tx_bd_base[i].status = cpu_to_be16(bd_status);
+ priv->tx_bd_base[i].buf = cpu_to_be32(priv->dma_tx_addr + i * MAX_RX_BUF_LENGTH);
}
+ dma_wmb();
return 0;
@@ -352,10 +351,10 @@ static netdev_tx_t ucc_hdlc_tx(struct sk_buff *skb, struct net_device *dev)
{
hdlc_device *hdlc = dev_to_hdlc(dev);
struct ucc_hdlc_private *priv = (struct ucc_hdlc_private *)hdlc->priv;
- struct qe_bd __iomem *bd;
+ struct qe_bd *bd;
u16 bd_status;
unsigned long flags;
- u16 *proto_head;
+ __be16 *proto_head;
switch (dev->type) {
case ARPHRD_RAWHDLC:
@@ -368,14 +367,14 @@ static netdev_tx_t ucc_hdlc_tx(struct sk_buff *skb, struct net_device *dev)
skb_push(skb, HDLC_HEAD_LEN);
- proto_head = (u16 *)skb->data;
+ proto_head = (__be16 *)skb->data;
*proto_head = htons(DEFAULT_HDLC_HEAD);
dev->stats.tx_bytes += skb->len;
break;
case ARPHRD_PPP:
- proto_head = (u16 *)skb->data;
+ proto_head = (__be16 *)skb->data;
if (*proto_head != htons(DEFAULT_PPP_HEAD)) {
dev->stats.tx_dropped++;
dev_kfree_skb(skb);
@@ -398,9 +397,10 @@ static netdev_tx_t ucc_hdlc_tx(struct sk_buff *skb, struct net_device *dev)
netdev_sent_queue(dev, skb->len);
spin_lock_irqsave(&priv->lock, flags);
+ dma_rmb();
/* Start from the next BD that should be filled */
bd = priv->curtx_bd;
- bd_status = ioread16be(&bd->status);
+ bd_status = be16_to_cpu(bd->status);
/* Save the skb pointer so we can free it later */
priv->tx_skbuff[priv->skb_curtx] = skb;
@@ -415,8 +415,8 @@ static netdev_tx_t ucc_hdlc_tx(struct sk_buff *skb, struct net_device *dev)
/* set bd status and length */
bd_status = (bd_status & T_W_S) | T_R_S | T_I_S | T_L_S | T_TC_S;
- iowrite16be(skb->len, &bd->length);
- iowrite16be(bd_status, &bd->status);
+ bd->length = cpu_to_be16(skb->len);
+ bd->status = cpu_to_be16(bd_status);
/* Move to next BD in the ring */
if (!(bd_status & T_W_S))
@@ -458,8 +458,9 @@ static int hdlc_tx_done(struct ucc_hdlc_private *priv)
u16 bd_status;
int tx_restart = 0;
+ dma_rmb();
bd = priv->dirty_tx;
- bd_status = ioread16be(&bd->status);
+ bd_status = be16_to_cpu(bd->status);
/* Normal processing. */
while ((bd_status & T_R_S) == 0) {
@@ -503,7 +504,7 @@ static int hdlc_tx_done(struct ucc_hdlc_private *priv)
bd += 1;
else
bd = priv->tx_bd_base;
- bd_status = ioread16be(&bd->status);
+ bd_status = be16_to_cpu(bd->status);
}
priv->dirty_tx = bd;
@@ -524,8 +525,9 @@ static int hdlc_rx_done(struct ucc_hdlc_private *priv, int rx_work_limit)
u16 length, howmany = 0;
u8 *bdbuffer;
+ dma_rmb();
bd = priv->currx_bd;
- bd_status = ioread16be(&bd->status);
+ bd_status = be16_to_cpu(bd->status);
/* while there are received buffers and BD is full (~R_E) */
while (!((bd_status & (R_E_S)) || (--rx_work_limit < 0))) {
@@ -549,7 +551,7 @@ static int hdlc_rx_done(struct ucc_hdlc_private *priv, int rx_work_limit)
}
bdbuffer = priv->rx_buffer +
(priv->currx_bdnum * MAX_RX_BUF_LENGTH);
- length = ioread16be(&bd->length);
+ length = be16_to_cpu(bd->length);
switch (dev->type) {
case ARPHRD_RAWHDLC:
@@ -593,7 +595,7 @@ static int hdlc_rx_done(struct ucc_hdlc_private *priv, int rx_work_limit)
netif_receive_skb(skb);
recycle:
- iowrite16be((bd_status & R_W_S) | R_E_S | R_I_S, &bd->status);
+ bd->status = cpu_to_be16((bd_status & R_W_S) | R_E_S | R_I_S);
/* update to point at the next bd */
if (bd_status & R_W_S) {
@@ -608,8 +610,9 @@ static int hdlc_rx_done(struct ucc_hdlc_private *priv, int rx_work_limit)
bd += 1;
}
- bd_status = ioread16be(&bd->status);
+ bd_status = be16_to_cpu(bd->status);
}
+ dma_rmb();
priv->currx_bd = bd;
return howmany;
@@ -721,7 +724,7 @@ static int uhdlc_open(struct net_device *dev)
/* Enable the TDM port */
if (priv->tsa)
- utdm->si_regs->siglmr1_h |= (0x1 << utdm->tdm_port);
+ qe_setbits_8(&utdm->si_regs->siglmr1_h, 0x1 << utdm->tdm_port);
priv->hdlc_busy = 1;
netif_device_attach(priv->ndev);
@@ -812,7 +815,7 @@ static int uhdlc_close(struct net_device *dev)
(u8)QE_CR_PROTOCOL_UNSPECIFIED, 0);
if (priv->tsa)
- utdm->si_regs->siglmr1_h &= ~(0x1 << utdm->tdm_port);
+ qe_clrbits_8(&utdm->si_regs->siglmr1_h, 0x1 << utdm->tdm_port);
ucc_fast_disable(priv->uccf, COMM_DIR_RX | COMM_DIR_TX);
@@ -848,7 +851,7 @@ static int ucc_hdlc_attach(struct net_device *dev, unsigned short encoding,
#ifdef CONFIG_PM
static void store_clk_config(struct ucc_hdlc_private *priv)
{
- struct qe_mux *qe_mux_reg = &qe_immr->qmx;
+ struct qe_mux __iomem *qe_mux_reg = &qe_immr->qmx;
/* store si clk */
priv->cmxsi1cr_h = ioread32be(&qe_mux_reg->cmxsi1cr_h);
@@ -863,7 +866,7 @@ static void store_clk_config(struct ucc_hdlc_private *priv)
static void resume_clk_config(struct ucc_hdlc_private *priv)
{
- struct qe_mux *qe_mux_reg = &qe_immr->qmx;
+ struct qe_mux __iomem *qe_mux_reg = &qe_immr->qmx;
memcpy_toio(qe_mux_reg->cmxucr, priv->cmxucr, 4 * sizeof(u32));
@@ -990,9 +993,8 @@ static int uhdlc_resume(struct device *dev)
else
bd_status = R_E_S | R_I_S | R_W_S;
- iowrite16be(bd_status, &priv->rx_bd_base[i].status);
- iowrite32be(priv->dma_rx_addr + i * MAX_RX_BUF_LENGTH,
- &priv->rx_bd_base[i].buf);
+ priv->rx_bd_base[i].status = cpu_to_be16(bd_status);
+ priv->rx_bd_base[i].buf = cpu_to_be32(priv->dma_rx_addr + i * MAX_RX_BUF_LENGTH);
}
for (i = 0; i < TX_BD_RING_LEN; i++) {
@@ -1001,10 +1003,10 @@ static int uhdlc_resume(struct device *dev)
else
bd_status = T_I_S | T_TC_S | T_W_S;
- iowrite16be(bd_status, &priv->tx_bd_base[i].status);
- iowrite32be(priv->dma_tx_addr + i * MAX_RX_BUF_LENGTH,
- &priv->tx_bd_base[i].buf);
+ priv->tx_bd_base[i].status = cpu_to_be16(bd_status);
+ priv->tx_bd_base[i].buf = cpu_to_be32(priv->dma_tx_addr + i * MAX_RX_BUF_LENGTH);
}
+ dma_wmb();
/* if hdlc is busy enable TX and RX */
if (priv->hdlc_busy == 1) {
@@ -1018,7 +1020,7 @@ static int uhdlc_resume(struct device *dev)
/* Enable the TDM port */
if (priv->tsa)
- utdm->si_regs->siglmr1_h |= (0x1 << utdm->tdm_port);
+ qe_setbits_8(&utdm->si_regs->siglmr1_h, 0x1 << utdm->tdm_port);
}
napi_enable(&priv->napi);
diff --git a/drivers/net/wan/ixp4xx_hss.c b/drivers/net/wan/ixp4xx_hss.c
index 88a36a0..0b7d9f2 100644
--- a/drivers/net/wan/ixp4xx_hss.c
+++ b/drivers/net/wan/ixp4xx_hss.c
@@ -17,13 +17,19 @@
#include <linux/io.h>
#include <linux/kernel.h>
#include <linux/platform_device.h>
-#include <linux/platform_data/wan_ixp4xx_hss.h>
#include <linux/poll.h>
#include <linux/slab.h>
+#include <linux/gpio/consumer.h>
+#include <linux/of.h>
#include <linux/soc/ixp4xx/npe.h>
#include <linux/soc/ixp4xx/qmgr.h>
#include <linux/soc/ixp4xx/cpu.h>
+/* This is what all IXP4xx platforms we know uses, if more frequencies
+ * are needed, we need to migrate to the clock framework.
+ */
+#define IXP4XX_TIMER_FREQ 66666000
+
#define DEBUG_DESC 0
#define DEBUG_RX 0
#define DEBUG_TX 0
@@ -50,7 +56,6 @@
#define NAPI_WEIGHT 16
/* Queue IDs */
-#define HSS0_CHL_RXTRIG_QUEUE 12 /* orig size = 32 dwords */
#define HSS0_PKT_RX_QUEUE 13 /* orig size = 32 dwords */
#define HSS0_PKT_TX0_QUEUE 14 /* orig size = 16 dwords */
#define HSS0_PKT_TX1_QUEUE 15
@@ -62,7 +67,6 @@
#define HSS0_PKT_RXFREE3_QUEUE 21
#define HSS0_PKT_TXDONE_QUEUE 22 /* orig size = 64 dwords */
-#define HSS1_CHL_RXTRIG_QUEUE 10
#define HSS1_PKT_RX_QUEUE 0
#define HSS1_PKT_TX0_QUEUE 5
#define HSS1_PKT_TX1_QUEUE 6
@@ -252,9 +256,19 @@ typedef void buffer_t;
struct port {
struct device *dev;
struct npe *npe;
+ unsigned int txreadyq;
+ unsigned int rxtrigq;
+ unsigned int rxfreeq;
+ unsigned int rxq;
+ unsigned int txq;
+ unsigned int txdoneq;
+ struct gpio_desc *cts;
+ struct gpio_desc *rts;
+ struct gpio_desc *dcd;
+ struct gpio_desc *dtr;
+ struct gpio_desc *clk_internal;
struct net_device *netdev;
struct napi_struct napi;
- struct hss_plat_info *plat;
buffer_t *rx_buff_tab[RX_DESCS], *tx_buff_tab[TX_DESCS];
struct desc *desc_tab; /* coherent */
dma_addr_t desc_tab_phys;
@@ -322,14 +336,6 @@ static int ports_open;
static struct dma_pool *dma_pool;
static DEFINE_SPINLOCK(npe_lock);
-static const struct {
- int tx, txdone, rx, rxfree;
-} queue_ids[2] = {{HSS0_PKT_TX0_QUEUE, HSS0_PKT_TXDONE_QUEUE, HSS0_PKT_RX_QUEUE,
- HSS0_PKT_RXFREE0_QUEUE},
- {HSS1_PKT_TX0_QUEUE, HSS1_PKT_TXDONE_QUEUE, HSS1_PKT_RX_QUEUE,
- HSS1_PKT_RXFREE0_QUEUE},
-};
-
/*****************************************************************************
* utility functions
****************************************************************************/
@@ -645,7 +651,7 @@ static void hss_hdlc_rx_irq(void *pdev)
#if DEBUG_RX
printk(KERN_DEBUG "%s: hss_hdlc_rx_irq\n", dev->name);
#endif
- qmgr_disable_irq(queue_ids[port->id].rx);
+ qmgr_disable_irq(port->rxq);
napi_schedule(&port->napi);
}
@@ -653,8 +659,8 @@ static int hss_hdlc_poll(struct napi_struct *napi, int budget)
{
struct port *port = container_of(napi, struct port, napi);
struct net_device *dev = port->netdev;
- unsigned int rxq = queue_ids[port->id].rx;
- unsigned int rxfreeq = queue_ids[port->id].rxfree;
+ unsigned int rxq = port->rxq;
+ unsigned int rxfreeq = port->rxfreeq;
int received = 0;
#if DEBUG_RX
@@ -795,7 +801,7 @@ static void hss_hdlc_txdone_irq(void *pdev)
#if DEBUG_TX
printk(KERN_DEBUG DRV_NAME ": hss_hdlc_txdone_irq\n");
#endif
- while ((n_desc = queue_get_desc(queue_ids[port->id].txdone,
+ while ((n_desc = queue_get_desc(port->txdoneq,
port, 1)) >= 0) {
struct desc *desc;
int start;
@@ -813,8 +819,8 @@ static void hss_hdlc_txdone_irq(void *pdev)
free_buffer_irq(port->tx_buff_tab[n_desc]);
port->tx_buff_tab[n_desc] = NULL;
- start = qmgr_stat_below_low_watermark(port->plat->txreadyq);
- queue_put_desc(port->plat->txreadyq,
+ start = qmgr_stat_below_low_watermark(port->txreadyq);
+ queue_put_desc(port->txreadyq,
tx_desc_phys(port, n_desc), desc);
if (start) { /* TX-ready queue was empty */
#if DEBUG_TX
@@ -829,7 +835,7 @@ static void hss_hdlc_txdone_irq(void *pdev)
static int hss_hdlc_xmit(struct sk_buff *skb, struct net_device *dev)
{
struct port *port = dev_to_port(dev);
- unsigned int txreadyq = port->plat->txreadyq;
+ unsigned int txreadyq = port->txreadyq;
int len, offset, bytes, n;
void *mem;
u32 phys;
@@ -889,7 +895,7 @@ static int hss_hdlc_xmit(struct sk_buff *skb, struct net_device *dev)
desc->buf_len = desc->pkt_len = len;
wmb();
- queue_put_desc(queue_ids[port->id].tx, tx_desc_phys(port, n), desc);
+ queue_put_desc(port->txq, tx_desc_phys(port, n), desc);
if (qmgr_stat_below_low_watermark(txreadyq)) { /* empty */
#if DEBUG_TX
@@ -916,40 +922,40 @@ static int request_hdlc_queues(struct port *port)
{
int err;
- err = qmgr_request_queue(queue_ids[port->id].rxfree, RX_DESCS, 0, 0,
+ err = qmgr_request_queue(port->rxfreeq, RX_DESCS, 0, 0,
"%s:RX-free", port->netdev->name);
if (err)
return err;
- err = qmgr_request_queue(queue_ids[port->id].rx, RX_DESCS, 0, 0,
+ err = qmgr_request_queue(port->rxq, RX_DESCS, 0, 0,
"%s:RX", port->netdev->name);
if (err)
goto rel_rxfree;
- err = qmgr_request_queue(queue_ids[port->id].tx, TX_DESCS, 0, 0,
+ err = qmgr_request_queue(port->txq, TX_DESCS, 0, 0,
"%s:TX", port->netdev->name);
if (err)
goto rel_rx;
- err = qmgr_request_queue(port->plat->txreadyq, TX_DESCS, 0, 0,
+ err = qmgr_request_queue(port->txreadyq, TX_DESCS, 0, 0,
"%s:TX-ready", port->netdev->name);
if (err)
goto rel_tx;
- err = qmgr_request_queue(queue_ids[port->id].txdone, TX_DESCS, 0, 0,
+ err = qmgr_request_queue(port->txdoneq, TX_DESCS, 0, 0,
"%s:TX-done", port->netdev->name);
if (err)
goto rel_txready;
return 0;
rel_txready:
- qmgr_release_queue(port->plat->txreadyq);
+ qmgr_release_queue(port->txreadyq);
rel_tx:
- qmgr_release_queue(queue_ids[port->id].tx);
+ qmgr_release_queue(port->txq);
rel_rx:
- qmgr_release_queue(queue_ids[port->id].rx);
+ qmgr_release_queue(port->rxq);
rel_rxfree:
- qmgr_release_queue(queue_ids[port->id].rxfree);
+ qmgr_release_queue(port->rxfreeq);
printk(KERN_DEBUG "%s: unable to request hardware queues\n",
port->netdev->name);
return err;
@@ -957,11 +963,11 @@ static int request_hdlc_queues(struct port *port)
static void release_hdlc_queues(struct port *port)
{
- qmgr_release_queue(queue_ids[port->id].rxfree);
- qmgr_release_queue(queue_ids[port->id].rx);
- qmgr_release_queue(queue_ids[port->id].txdone);
- qmgr_release_queue(queue_ids[port->id].tx);
- qmgr_release_queue(port->plat->txreadyq);
+ qmgr_release_queue(port->rxfreeq);
+ qmgr_release_queue(port->rxq);
+ qmgr_release_queue(port->txdoneq);
+ qmgr_release_queue(port->txq);
+ qmgr_release_queue(port->txreadyq);
}
static int init_hdlc_queues(struct port *port)
@@ -1046,11 +1052,24 @@ static void destroy_hdlc_queues(struct port *port)
}
}
+static irqreturn_t hss_hdlc_dcd_irq(int irq, void *data)
+{
+ struct net_device *dev = data;
+ struct port *port = dev_to_port(dev);
+ int val;
+
+ val = gpiod_get_value(port->dcd);
+ hss_hdlc_set_carrier(dev, val);
+
+ return IRQ_HANDLED;
+}
+
static int hss_hdlc_open(struct net_device *dev)
{
struct port *port = dev_to_port(dev);
unsigned long flags;
int i, err = 0;
+ int val;
err = hdlc_open(dev);
if (err)
@@ -1069,32 +1088,44 @@ static int hss_hdlc_open(struct net_device *dev)
goto err_destroy_queues;
spin_lock_irqsave(&npe_lock, flags);
- if (port->plat->open) {
- err = port->plat->open(port->id, dev, hss_hdlc_set_carrier);
- if (err)
- goto err_unlock;
+
+ /* Set the carrier, the GPIO is flagged active low so this will return
+ * 1 if DCD is asserted.
+ */
+ val = gpiod_get_value(port->dcd);
+ hss_hdlc_set_carrier(dev, val);
+
+ /* Set up an IRQ for DCD */
+ err = request_irq(gpiod_to_irq(port->dcd), hss_hdlc_dcd_irq, 0, "IXP4xx HSS", dev);
+ if (err) {
+ dev_err(&dev->dev, "ixp4xx_hss: failed to request DCD IRQ (%i)\n", err);
+ goto err_unlock;
}
+ /* GPIOs are flagged active low so this asserts DTR and RTS */
+ gpiod_set_value(port->dtr, 1);
+ gpiod_set_value(port->rts, 1);
+
spin_unlock_irqrestore(&npe_lock, flags);
/* Populate queues with buffers, no failure after this point */
for (i = 0; i < TX_DESCS; i++)
- queue_put_desc(port->plat->txreadyq,
+ queue_put_desc(port->txreadyq,
tx_desc_phys(port, i), tx_desc_ptr(port, i));
for (i = 0; i < RX_DESCS; i++)
- queue_put_desc(queue_ids[port->id].rxfree,
+ queue_put_desc(port->rxfreeq,
rx_desc_phys(port, i), rx_desc_ptr(port, i));
napi_enable(&port->napi);
netif_start_queue(dev);
- qmgr_set_irq(queue_ids[port->id].rx, QUEUE_IRQ_SRC_NOT_EMPTY,
+ qmgr_set_irq(port->rxq, QUEUE_IRQ_SRC_NOT_EMPTY,
hss_hdlc_rx_irq, dev);
- qmgr_set_irq(queue_ids[port->id].txdone, QUEUE_IRQ_SRC_NOT_EMPTY,
+ qmgr_set_irq(port->txdoneq, QUEUE_IRQ_SRC_NOT_EMPTY,
hss_hdlc_txdone_irq, dev);
- qmgr_enable_irq(queue_ids[port->id].txdone);
+ qmgr_enable_irq(port->txdoneq);
ports_open++;
@@ -1125,15 +1156,15 @@ static int hss_hdlc_close(struct net_device *dev)
spin_lock_irqsave(&npe_lock, flags);
ports_open--;
- qmgr_disable_irq(queue_ids[port->id].rx);
+ qmgr_disable_irq(port->rxq);
netif_stop_queue(dev);
napi_disable(&port->napi);
hss_stop_hdlc(port);
- while (queue_get_desc(queue_ids[port->id].rxfree, port, 0) >= 0)
+ while (queue_get_desc(port->rxfreeq, port, 0) >= 0)
buffs--;
- while (queue_get_desc(queue_ids[port->id].rx, port, 0) >= 0)
+ while (queue_get_desc(port->rxq, port, 0) >= 0)
buffs--;
if (buffs)
@@ -1141,12 +1172,12 @@ static int hss_hdlc_close(struct net_device *dev)
buffs);
buffs = TX_DESCS;
- while (queue_get_desc(queue_ids[port->id].tx, port, 1) >= 0)
+ while (queue_get_desc(port->txq, port, 1) >= 0)
buffs--; /* cancel TX */
i = 0;
do {
- while (queue_get_desc(port->plat->txreadyq, port, 1) >= 0)
+ while (queue_get_desc(port->txreadyq, port, 1) >= 0)
buffs--;
if (!buffs)
break;
@@ -1159,10 +1190,12 @@ static int hss_hdlc_close(struct net_device *dev)
if (!buffs)
printk(KERN_DEBUG "Draining TX queues took %i cycles\n", i);
#endif
- qmgr_disable_irq(queue_ids[port->id].txdone);
+ qmgr_disable_irq(port->txdoneq);
- if (port->plat->close)
- port->plat->close(port->id, dev);
+ free_irq(gpiod_to_irq(port->dcd), dev);
+ /* GPIOs are flagged active low so this de-asserts DTR and RTS */
+ gpiod_set_value(port->dtr, 0);
+ gpiod_set_value(port->rts, 0);
spin_unlock_irqrestore(&npe_lock, flags);
destroy_hdlc_queues(port);
@@ -1254,6 +1287,21 @@ static void find_best_clock(u32 timer_freq, u32 rate, u32 *best, u32 *reg)
}
}
+static int hss_hdlc_set_clock(struct port *port, unsigned int clock_type)
+{
+ switch (clock_type) {
+ case CLOCK_DEFAULT:
+ case CLOCK_EXT:
+ gpiod_set_value(port->clk_internal, 0);
+ return CLOCK_EXT;
+ case CLOCK_INT:
+ gpiod_set_value(port->clk_internal, 1);
+ return CLOCK_INT;
+ default:
+ return -EINVAL;
+ }
+}
+
static int hss_hdlc_ioctl(struct net_device *dev, struct if_settings *ifs)
{
const size_t size = sizeof(sync_serial_settings);
@@ -1286,8 +1334,7 @@ static int hss_hdlc_ioctl(struct net_device *dev, struct if_settings *ifs)
return -EFAULT;
clk = new_line.clock_type;
- if (port->plat->set_clock)
- clk = port->plat->set_clock(port->id, clk);
+ hss_hdlc_set_clock(port, clk);
if (clk != CLOCK_EXT && clk != CLOCK_INT)
return -EINVAL; /* No such clock setting */
@@ -1297,7 +1344,7 @@ static int hss_hdlc_ioctl(struct net_device *dev, struct if_settings *ifs)
port->clock_type = clk; /* Update settings */
if (clk == CLOCK_INT) {
- find_best_clock(port->plat->timer_freq,
+ find_best_clock(IXP4XX_TIMER_FREQ,
new_line.clock_rate,
&port->clock_rate, &port->clock_reg);
} else {
@@ -1335,77 +1382,139 @@ static const struct net_device_ops hss_hdlc_ops = {
.ndo_siocwandev = hss_hdlc_ioctl,
};
-static int hss_init_one(struct platform_device *pdev)
+static int ixp4xx_hss_probe(struct platform_device *pdev)
{
+ struct of_phandle_args queue_spec;
+ struct of_phandle_args npe_spec;
+ struct device *dev = &pdev->dev;
+ struct net_device *ndev;
+ struct device_node *np;
struct port *port;
- struct net_device *dev;
hdlc_device *hdlc;
int err;
- port = kzalloc(sizeof(*port), GFP_KERNEL);
+ np = dev->of_node;
+
+ port = devm_kzalloc(dev, sizeof(*port), GFP_KERNEL);
if (!port)
return -ENOMEM;
- port->npe = npe_request(0);
+ err = of_parse_phandle_with_fixed_args(np, "intel,npe-handle", 1, 0,
+ &npe_spec);
+ if (err)
+ return dev_err_probe(dev, err, "no NPE engine specified\n");
+ /* NPE ID 0x00, 0x10, 0x20... */
+ port->npe = npe_request(npe_spec.args[0] << 4);
if (!port->npe) {
- err = -ENODEV;
- goto err_free;
+ dev_err(dev, "unable to obtain NPE instance\n");
+ return -ENODEV;
}
- dev = alloc_hdlcdev(port);
+ /* Get the TX ready queue as resource from queue manager */
+ err = of_parse_phandle_with_fixed_args(np, "intek,queue-chl-txready", 1, 0,
+ &queue_spec);
+ if (err)
+ return dev_err_probe(dev, err, "no txready queue phandle\n");
+ port->txreadyq = queue_spec.args[0];
+ /* Get the RX trig queue as resource from queue manager */
+ err = of_parse_phandle_with_fixed_args(np, "intek,queue-chl-rxtrig", 1, 0,
+ &queue_spec);
+ if (err)
+ return dev_err_probe(dev, err, "no rxtrig queue phandle\n");
+ port->rxtrigq = queue_spec.args[0];
+ /* Get the RX queue as resource from queue manager */
+ err = of_parse_phandle_with_fixed_args(np, "intek,queue-pkt-rx", 1, 0,
+ &queue_spec);
+ if (err)
+ return dev_err_probe(dev, err, "no RX queue phandle\n");
+ port->rxq = queue_spec.args[0];
+ /* Get the TX queue as resource from queue manager */
+ err = of_parse_phandle_with_fixed_args(np, "intek,queue-pkt-tx", 1, 0,
+ &queue_spec);
+ if (err)
+ return dev_err_probe(dev, err, "no RX queue phandle\n");
+ port->txq = queue_spec.args[0];
+ /* Get the RX free queue as resource from queue manager */
+ err = of_parse_phandle_with_fixed_args(np, "intek,queue-pkt-rxfree", 1, 0,
+ &queue_spec);
+ if (err)
+ return dev_err_probe(dev, err, "no RX free queue phandle\n");
+ port->rxfreeq = queue_spec.args[0];
+ /* Get the TX done queue as resource from queue manager */
+ err = of_parse_phandle_with_fixed_args(np, "intek,queue-pkt-txdone", 1, 0,
+ &queue_spec);
+ if (err)
+ return dev_err_probe(dev, err, "no TX done queue phandle\n");
+ port->txdoneq = queue_spec.args[0];
+
+ /* Obtain all the line control GPIOs */
+ port->cts = devm_gpiod_get(dev, "cts", GPIOD_OUT_LOW);
+ if (IS_ERR(port->cts))
+ return dev_err_probe(dev, PTR_ERR(port->cts), "unable to get CTS GPIO\n");
+ port->rts = devm_gpiod_get(dev, "rts", GPIOD_OUT_LOW);
+ if (IS_ERR(port->rts))
+ return dev_err_probe(dev, PTR_ERR(port->rts), "unable to get RTS GPIO\n");
+ port->dcd = devm_gpiod_get(dev, "dcd", GPIOD_IN);
+ if (IS_ERR(port->dcd))
+ return dev_err_probe(dev, PTR_ERR(port->dcd), "unable to get DCD GPIO\n");
+ port->dtr = devm_gpiod_get(dev, "dtr", GPIOD_OUT_LOW);
+ if (IS_ERR(port->dtr))
+ return dev_err_probe(dev, PTR_ERR(port->dtr), "unable to get DTR GPIO\n");
+ port->clk_internal = devm_gpiod_get(dev, "clk-internal", GPIOD_OUT_LOW);
+ if (IS_ERR(port->clk_internal))
+ return dev_err_probe(dev, PTR_ERR(port->clk_internal),
+ "unable to get CLK internal GPIO\n");
+
+ ndev = alloc_hdlcdev(port);
port->netdev = alloc_hdlcdev(port);
if (!port->netdev) {
err = -ENOMEM;
goto err_plat;
}
- SET_NETDEV_DEV(dev, &pdev->dev);
- hdlc = dev_to_hdlc(dev);
+ SET_NETDEV_DEV(ndev, &pdev->dev);
+ hdlc = dev_to_hdlc(ndev);
hdlc->attach = hss_hdlc_attach;
hdlc->xmit = hss_hdlc_xmit;
- dev->netdev_ops = &hss_hdlc_ops;
- dev->tx_queue_len = 100;
+ ndev->netdev_ops = &hss_hdlc_ops;
+ ndev->tx_queue_len = 100;
port->clock_type = CLOCK_EXT;
port->clock_rate = 0;
port->clock_reg = CLK42X_SPEED_2048KHZ;
port->id = pdev->id;
port->dev = &pdev->dev;
- port->plat = pdev->dev.platform_data;
- netif_napi_add(dev, &port->napi, hss_hdlc_poll, NAPI_WEIGHT);
+ netif_napi_add(ndev, &port->napi, hss_hdlc_poll, NAPI_WEIGHT);
- err = register_hdlc_device(dev);
+ err = register_hdlc_device(ndev);
if (err)
goto err_free_netdev;
platform_set_drvdata(pdev, port);
- netdev_info(dev, "initialized\n");
+ netdev_info(ndev, "initialized\n");
return 0;
err_free_netdev:
- free_netdev(dev);
+ free_netdev(ndev);
err_plat:
npe_release(port->npe);
-err_free:
- kfree(port);
return err;
}
-static int hss_remove_one(struct platform_device *pdev)
+static int ixp4xx_hss_remove(struct platform_device *pdev)
{
struct port *port = platform_get_drvdata(pdev);
unregister_hdlc_device(port->netdev);
free_netdev(port->netdev);
npe_release(port->npe);
- kfree(port);
return 0;
}
static struct platform_driver ixp4xx_hss_driver = {
.driver.name = DRV_NAME,
- .probe = hss_init_one,
- .remove = hss_remove_one,
+ .probe = ixp4xx_hss_probe,
+ .remove = ixp4xx_hss_remove,
};
static int __init hss_init_module(void)
diff --git a/drivers/net/wan/lmc/lmc_main.c b/drivers/net/wan/lmc/lmc_main.c
index ed687bf..6a142dc 100644
--- a/drivers/net/wan/lmc/lmc_main.c
+++ b/drivers/net/wan/lmc/lmc_main.c
@@ -550,7 +550,7 @@ static int lmc_siocdevprivate(struct net_device *dev, struct ifreq *ifr,
(timeout-- > 0))
cpu_relax();
- printk(KERN_DEBUG "%s: Waited %d for the Xilinx to clear it's memory\n", dev->name, 500000-timeout);
+ printk(KERN_DEBUG "%s: Waited %d for the Xilinx to clear its memory\n", dev->name, 500000-timeout);
for(pos = 0; pos < xc.len; pos++){
switch(data[pos]){
diff --git a/drivers/net/wireguard/queueing.h b/drivers/net/wireguard/queueing.h
index e238810..583adb3 100644
--- a/drivers/net/wireguard/queueing.h
+++ b/drivers/net/wireguard/queueing.h
@@ -79,9 +79,7 @@ static inline void wg_reset_packet(struct sk_buff *skb, bool encapsulating)
u8 sw_hash = skb->sw_hash;
u32 hash = skb->hash;
skb_scrub_packet(skb, true);
- memset(&skb->headers_start, 0,
- offsetof(struct sk_buff, headers_end) -
- offsetof(struct sk_buff, headers_start));
+ memset(&skb->headers, 0, sizeof(skb->headers));
if (encapsulating) {
skb->l4_hash = l4_hash;
skb->sw_hash = sw_hash;
diff --git a/drivers/net/wireless/ath/ar5523/ar5523.c b/drivers/net/wireless/ath/ar5523/ar5523.c
index 0e9bad3..141c1b5 100644
--- a/drivers/net/wireless/ath/ar5523/ar5523.c
+++ b/drivers/net/wireless/ath/ar5523/ar5523.c
@@ -153,6 +153,10 @@ static void ar5523_cmd_rx_cb(struct urb *urb)
ar5523_err(ar, "Invalid reply to WDCMSG_TARGET_START");
return;
}
+ if (!cmd->odata) {
+ ar5523_err(ar, "Unexpected WDCMSG_TARGET_START reply");
+ return;
+ }
memcpy(cmd->odata, hdr + 1, sizeof(u32));
cmd->olen = sizeof(u32);
cmd->res = 0;
diff --git a/drivers/net/wireless/ath/ath10k/core.c b/drivers/net/wireless/ath/ath10k/core.c
index 5935e09..8f5b8eb 100644
--- a/drivers/net/wireless/ath/ath10k/core.c
+++ b/drivers/net/wireless/ath/ath10k/core.c
@@ -12,6 +12,7 @@
#include <linux/dmi.h>
#include <linux/ctype.h>
#include <linux/pm_qos.h>
+#include <linux/nvmem-consumer.h>
#include <asm/byteorder.h>
#include "core.h"
@@ -89,6 +90,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
.rri_on_ddr = false,
.hw_filter_reset_required = true,
.fw_diag_ce_download = false,
+ .credit_size_workaround = false,
.tx_stats_over_pktlog = true,
.dynamic_sar_support = false,
},
@@ -124,6 +126,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
.rri_on_ddr = false,
.hw_filter_reset_required = true,
.fw_diag_ce_download = false,
+ .credit_size_workaround = false,
.tx_stats_over_pktlog = true,
.dynamic_sar_support = false,
},
@@ -160,6 +163,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
.rri_on_ddr = false,
.hw_filter_reset_required = true,
.fw_diag_ce_download = false,
+ .credit_size_workaround = false,
.tx_stats_over_pktlog = false,
.dynamic_sar_support = false,
},
@@ -190,6 +194,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
.num_wds_entries = 0x20,
.uart_pin_workaround = true,
.tx_stats_over_pktlog = false,
+ .credit_size_workaround = false,
.bmi_large_size_download = true,
.supports_peer_stats_info = true,
.dynamic_sar_support = true,
@@ -226,6 +231,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
.rri_on_ddr = false,
.hw_filter_reset_required = true,
.fw_diag_ce_download = false,
+ .credit_size_workaround = false,
.tx_stats_over_pktlog = false,
.dynamic_sar_support = false,
},
@@ -261,6 +267,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
.rri_on_ddr = false,
.hw_filter_reset_required = true,
.fw_diag_ce_download = false,
+ .credit_size_workaround = false,
.tx_stats_over_pktlog = false,
.dynamic_sar_support = false,
},
@@ -296,6 +303,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
.rri_on_ddr = false,
.hw_filter_reset_required = true,
.fw_diag_ce_download = false,
+ .credit_size_workaround = false,
.tx_stats_over_pktlog = false,
.dynamic_sar_support = false,
},
@@ -334,6 +342,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
.rri_on_ddr = false,
.hw_filter_reset_required = true,
.fw_diag_ce_download = true,
+ .credit_size_workaround = false,
.tx_stats_over_pktlog = false,
.supports_peer_stats_info = true,
.dynamic_sar_support = true,
@@ -376,6 +385,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
.rri_on_ddr = false,
.hw_filter_reset_required = true,
.fw_diag_ce_download = false,
+ .credit_size_workaround = false,
.tx_stats_over_pktlog = false,
.dynamic_sar_support = false,
},
@@ -424,6 +434,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
.rri_on_ddr = false,
.hw_filter_reset_required = true,
.fw_diag_ce_download = false,
+ .credit_size_workaround = false,
.tx_stats_over_pktlog = false,
.dynamic_sar_support = false,
},
@@ -469,6 +480,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
.rri_on_ddr = false,
.hw_filter_reset_required = true,
.fw_diag_ce_download = false,
+ .credit_size_workaround = false,
.tx_stats_over_pktlog = false,
.dynamic_sar_support = false,
},
@@ -504,6 +516,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
.rri_on_ddr = false,
.hw_filter_reset_required = true,
.fw_diag_ce_download = false,
+ .credit_size_workaround = false,
.tx_stats_over_pktlog = false,
.dynamic_sar_support = false,
},
@@ -541,6 +554,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
.rri_on_ddr = false,
.hw_filter_reset_required = true,
.fw_diag_ce_download = true,
+ .credit_size_workaround = false,
.tx_stats_over_pktlog = false,
.dynamic_sar_support = false,
},
@@ -570,6 +584,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
.ast_skid_limit = 0x10,
.num_wds_entries = 0x20,
.uart_pin_workaround = true,
+ .credit_size_workaround = true,
.dynamic_sar_support = false,
},
{
@@ -611,6 +626,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
.rri_on_ddr = false,
.hw_filter_reset_required = true,
.fw_diag_ce_download = false,
+ .credit_size_workaround = false,
.tx_stats_over_pktlog = false,
.dynamic_sar_support = false,
},
@@ -639,6 +655,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = {
.rri_on_ddr = true,
.hw_filter_reset_required = false,
.fw_diag_ce_download = false,
+ .credit_size_workaround = false,
.tx_stats_over_pktlog = false,
.dynamic_sar_support = true,
},
@@ -714,6 +731,7 @@ static void ath10k_send_suspend_complete(struct ath10k *ar)
static int ath10k_init_sdio(struct ath10k *ar, enum ath10k_firmware_mode mode)
{
+ bool mtu_workaround = ar->hw_params.credit_size_workaround;
int ret;
u32 param = 0;
@@ -731,7 +749,7 @@ static int ath10k_init_sdio(struct ath10k *ar, enum ath10k_firmware_mode mode)
param |= HI_ACS_FLAGS_SDIO_REDUCE_TX_COMPL_SET;
- if (mode == ATH10K_FIRMWARE_MODE_NORMAL)
+ if (mode == ATH10K_FIRMWARE_MODE_NORMAL && !mtu_workaround)
param |= HI_ACS_FLAGS_ALT_DATA_CREDIT_SIZE;
else
param &= ~HI_ACS_FLAGS_ALT_DATA_CREDIT_SIZE;
@@ -935,7 +953,8 @@ static int ath10k_core_get_board_id_from_otp(struct ath10k *ar)
}
if (ar->cal_mode == ATH10K_PRE_CAL_MODE_DT ||
- ar->cal_mode == ATH10K_PRE_CAL_MODE_FILE)
+ ar->cal_mode == ATH10K_PRE_CAL_MODE_FILE ||
+ ar->cal_mode == ATH10K_PRE_CAL_MODE_NVMEM)
bmi_board_id_param = BMI_PARAM_GET_FLASH_BOARD_ID;
else
bmi_board_id_param = BMI_PARAM_GET_EEPROM_BOARD_ID;
@@ -1726,7 +1745,8 @@ static int ath10k_download_and_run_otp(struct ath10k *ar)
/* As of now pre-cal is valid for 10_4 variants */
if (ar->cal_mode == ATH10K_PRE_CAL_MODE_DT ||
- ar->cal_mode == ATH10K_PRE_CAL_MODE_FILE)
+ ar->cal_mode == ATH10K_PRE_CAL_MODE_FILE ||
+ ar->cal_mode == ATH10K_PRE_CAL_MODE_NVMEM)
bmi_otp_exe_param = BMI_PARAM_FLASH_SECTION_ALL;
ret = ath10k_bmi_execute(ar, address, bmi_otp_exe_param, &result);
@@ -1853,6 +1873,39 @@ static int ath10k_download_cal_eeprom(struct ath10k *ar)
return ret;
}
+static int ath10k_download_cal_nvmem(struct ath10k *ar, const char *cell_name)
+{
+ struct nvmem_cell *cell;
+ void *buf;
+ size_t len;
+ int ret;
+
+ cell = devm_nvmem_cell_get(ar->dev, cell_name);
+ if (IS_ERR(cell)) {
+ ret = PTR_ERR(cell);
+ return ret;
+ }
+
+ buf = nvmem_cell_read(cell, &len);
+ if (IS_ERR(buf))
+ return PTR_ERR(buf);
+
+ if (ar->hw_params.cal_data_len != len) {
+ kfree(buf);
+ ath10k_warn(ar, "invalid calibration data length in nvmem-cell '%s': %zu != %u\n",
+ cell_name, len, ar->hw_params.cal_data_len);
+ return -EMSGSIZE;
+ }
+
+ ret = ath10k_download_board_data(ar, buf, len);
+ kfree(buf);
+ if (ret)
+ ath10k_warn(ar, "failed to download calibration data from nvmem-cell '%s': %d\n",
+ cell_name, ret);
+
+ return ret;
+}
+
int ath10k_core_fetch_firmware_api_n(struct ath10k *ar, const char *name,
struct ath10k_fw_file *fw_file)
{
@@ -2087,6 +2140,18 @@ static int ath10k_core_pre_cal_download(struct ath10k *ar)
{
int ret;
+ ret = ath10k_download_cal_nvmem(ar, "pre-calibration");
+ if (ret == 0) {
+ ar->cal_mode = ATH10K_PRE_CAL_MODE_NVMEM;
+ goto success;
+ } else if (ret == -EPROBE_DEFER) {
+ return ret;
+ }
+
+ ath10k_dbg(ar, ATH10K_DBG_BOOT,
+ "boot did not find a pre-calibration nvmem-cell, try file next: %d\n",
+ ret);
+
ret = ath10k_download_cal_file(ar, ar->pre_cal_file);
if (ret == 0) {
ar->cal_mode = ATH10K_PRE_CAL_MODE_FILE;
@@ -2153,6 +2218,18 @@ static int ath10k_download_cal_data(struct ath10k *ar)
"pre cal download procedure failed, try cal file: %d\n",
ret);
+ ret = ath10k_download_cal_nvmem(ar, "calibration");
+ if (ret == 0) {
+ ar->cal_mode = ATH10K_CAL_MODE_NVMEM;
+ goto done;
+ } else if (ret == -EPROBE_DEFER) {
+ return ret;
+ }
+
+ ath10k_dbg(ar, ATH10K_DBG_BOOT,
+ "boot did not find a calibration nvmem-cell, try file next: %d\n",
+ ret);
+
ret = ath10k_download_cal_file(ar, ar->cal_file);
if (ret == 0) {
ar->cal_mode = ATH10K_CAL_MODE_FILE;
diff --git a/drivers/net/wireless/ath/ath10k/core.h b/drivers/net/wireless/ath/ath10k/core.h
index 5aeff2d..9f6680b 100644
--- a/drivers/net/wireless/ath/ath10k/core.h
+++ b/drivers/net/wireless/ath/ath10k/core.h
@@ -877,8 +877,10 @@ enum ath10k_cal_mode {
ATH10K_CAL_MODE_FILE,
ATH10K_CAL_MODE_OTP,
ATH10K_CAL_MODE_DT,
+ ATH10K_CAL_MODE_NVMEM,
ATH10K_PRE_CAL_MODE_FILE,
ATH10K_PRE_CAL_MODE_DT,
+ ATH10K_PRE_CAL_MODE_NVMEM,
ATH10K_CAL_MODE_EEPROM,
};
@@ -898,10 +900,14 @@ static inline const char *ath10k_cal_mode_str(enum ath10k_cal_mode mode)
return "otp";
case ATH10K_CAL_MODE_DT:
return "dt";
+ case ATH10K_CAL_MODE_NVMEM:
+ return "nvmem";
case ATH10K_PRE_CAL_MODE_FILE:
return "pre-cal-file";
case ATH10K_PRE_CAL_MODE_DT:
return "pre-cal-dt";
+ case ATH10K_PRE_CAL_MODE_NVMEM:
+ return "pre-cal-nvmem";
case ATH10K_CAL_MODE_EEPROM:
return "eeprom";
}
diff --git a/drivers/net/wireless/ath/ath10k/coredump.c b/drivers/net/wireless/ath/ath10k/coredump.c
index 55e7e11..fe6b6f9 100644
--- a/drivers/net/wireless/ath/ath10k/coredump.c
+++ b/drivers/net/wireless/ath/ath10k/coredump.c
@@ -1522,7 +1522,7 @@ static struct ath10k_dump_file_data *ath10k_coredump_build(struct ath10k *ar)
mutex_lock(&ar->dump_mutex);
dump_data = (struct ath10k_dump_file_data *)(buf);
- strlcpy(dump_data->df_magic, "ATH10K-FW-DUMP",
+ strscpy(dump_data->df_magic, "ATH10K-FW-DUMP",
sizeof(dump_data->df_magic));
dump_data->len = cpu_to_le32(len);
@@ -1543,11 +1543,11 @@ static struct ath10k_dump_file_data *ath10k_coredump_build(struct ath10k *ar)
dump_data->vht_cap_info = cpu_to_le32(ar->vht_cap_info);
dump_data->num_rf_chains = cpu_to_le32(ar->num_rf_chains);
- strlcpy(dump_data->fw_ver, ar->hw->wiphy->fw_version,
+ strscpy(dump_data->fw_ver, ar->hw->wiphy->fw_version,
sizeof(dump_data->fw_ver));
dump_data->kernel_ver_code = 0;
- strlcpy(dump_data->kernel_ver, init_utsname()->release,
+ strscpy(dump_data->kernel_ver, init_utsname()->release,
sizeof(dump_data->kernel_ver));
dump_data->tv_sec = cpu_to_le64(crash_data->timestamp.tv_sec);
diff --git a/drivers/net/wireless/ath/ath10k/htt.h b/drivers/net/wireless/ath/ath10k/htt.h
index a6de08d..9a3a890 100644
--- a/drivers/net/wireless/ath/ath10k/htt.h
+++ b/drivers/net/wireless/ath/ath10k/htt.h
@@ -1401,115 +1401,6 @@ enum htt_dbg_stats_status {
};
/*
- * target -> host statistics upload
- *
- * The following field definitions describe the format of the HTT target
- * to host stats upload confirmation message.
- * The message contains a cookie echoed from the HTT host->target stats
- * upload request, which identifies which request the confirmation is
- * for, and a series of tag-length-value stats information elements.
- * The tag-length header for each stats info element also includes a
- * status field, to indicate whether the request for the stat type in
- * question was fully met, partially met, unable to be met, or invalid
- * (if the stat type in question is disabled in the target).
- * A special value of all 1's in this status field is used to indicate
- * the end of the series of stats info elements.
- *
- *
- * |31 16|15 8|7 5|4 0|
- * |------------------------------------------------------------|
- * | reserved | msg type |
- * |------------------------------------------------------------|
- * | cookie LSBs |
- * |------------------------------------------------------------|
- * | cookie MSBs |
- * |------------------------------------------------------------|
- * | stats entry length | reserved | S |stat type|
- * |------------------------------------------------------------|
- * | |
- * | type-specific stats info |
- * | |
- * |------------------------------------------------------------|
- * | stats entry length | reserved | S |stat type|
- * |------------------------------------------------------------|
- * | |
- * | type-specific stats info |
- * | |
- * |------------------------------------------------------------|
- * | n/a | reserved | 111 | n/a |
- * |------------------------------------------------------------|
- * Header fields:
- * - MSG_TYPE
- * Bits 7:0
- * Purpose: identifies this is a statistics upload confirmation message
- * Value: 0x9
- * - COOKIE_LSBS
- * Bits 31:0
- * Purpose: Provide a mechanism to match a target->host stats confirmation
- * message with its preceding host->target stats request message.
- * Value: LSBs of the opaque cookie specified by the host-side requestor
- * - COOKIE_MSBS
- * Bits 31:0
- * Purpose: Provide a mechanism to match a target->host stats confirmation
- * message with its preceding host->target stats request message.
- * Value: MSBs of the opaque cookie specified by the host-side requestor
- *
- * Stats Information Element tag-length header fields:
- * - STAT_TYPE
- * Bits 4:0
- * Purpose: identifies the type of statistics info held in the
- * following information element
- * Value: htt_dbg_stats_type
- * - STATUS
- * Bits 7:5
- * Purpose: indicate whether the requested stats are present
- * Value: htt_dbg_stats_status, including a special value (0x7) to mark
- * the completion of the stats entry series
- * - LENGTH
- * Bits 31:16
- * Purpose: indicate the stats information size
- * Value: This field specifies the number of bytes of stats information
- * that follows the element tag-length header.
- * It is expected but not required that this length is a multiple of
- * 4 bytes. Even if the length is not an integer multiple of 4, the
- * subsequent stats entry header will begin on a 4-byte aligned
- * boundary.
- */
-
-#define HTT_STATS_CONF_ITEM_INFO_STAT_TYPE_MASK 0x1F
-#define HTT_STATS_CONF_ITEM_INFO_STAT_TYPE_LSB 0
-#define HTT_STATS_CONF_ITEM_INFO_STATUS_MASK 0xE0
-#define HTT_STATS_CONF_ITEM_INFO_STATUS_LSB 5
-
-struct htt_stats_conf_item {
- union {
- u8 info;
- struct {
- u8 stat_type:5; /* %HTT_DBG_STATS_ */
- u8 status:3; /* %HTT_DBG_STATS_STATUS_ */
- } __packed;
- } __packed;
- u8 pad;
- __le16 length;
- u8 payload[]; /* roundup(length, 4) long */
-} __packed;
-
-struct htt_stats_conf {
- u8 pad[3];
- __le32 cookie_lsb;
- __le32 cookie_msb;
-
- /* each item has variable length! */
- struct htt_stats_conf_item items[];
-} __packed;
-
-static inline struct htt_stats_conf_item *htt_stats_conf_next_item(
- const struct htt_stats_conf_item *item)
-{
- return (void *)item + sizeof(*item) + roundup(item->length, 4);
-}
-
-/*
* host -> target FRAG DESCRIPTOR/MSDU_EXT DESC bank
*
* The following field definitions describe the format of the HTT host
@@ -1828,7 +1719,6 @@ struct htt_resp {
struct htt_rc_update rc_update;
struct htt_rx_test rx_test;
struct htt_pktlog_msg pktlog_msg;
- struct htt_stats_conf stats_conf;
struct htt_rx_pn_ind rx_pn_ind;
struct htt_rx_offload_ind rx_offload_ind;
struct htt_rx_in_ord_ind rx_in_ord_ind;
diff --git a/drivers/net/wireless/ath/ath10k/htt_tx.c b/drivers/net/wireless/ath/ath10k/htt_tx.c
index d6b8bdc..b793eac 100644
--- a/drivers/net/wireless/ath/ath10k/htt_tx.c
+++ b/drivers/net/wireless/ath/ath10k/htt_tx.c
@@ -147,6 +147,9 @@ void ath10k_htt_tx_dec_pending(struct ath10k_htt *htt)
htt->num_pending_tx--;
if (htt->num_pending_tx == htt->max_num_pending_tx - 1)
ath10k_mac_tx_unlock(htt->ar, ATH10K_TX_PAUSE_Q_FULL);
+
+ if (htt->num_pending_tx == 0)
+ wake_up(&htt->empty_tx_wq);
}
int ath10k_htt_tx_inc_pending(struct ath10k_htt *htt)
diff --git a/drivers/net/wireless/ath/ath10k/hw.h b/drivers/net/wireless/ath/ath10k/hw.h
index 6b03c77..591ef74 100644
--- a/drivers/net/wireless/ath/ath10k/hw.h
+++ b/drivers/net/wireless/ath/ath10k/hw.h
@@ -618,6 +618,9 @@ struct ath10k_hw_params {
*/
bool uart_pin_workaround;
+ /* Workaround for the credit size calculation */
+ bool credit_size_workaround;
+
/* tx stats support over pktlog */
bool tx_stats_over_pktlog;
diff --git a/drivers/net/wireless/ath/ath10k/mac.c b/drivers/net/wireless/ath/ath10k/mac.c
index 1f73fbf..b11aaee 100644
--- a/drivers/net/wireless/ath/ath10k/mac.c
+++ b/drivers/net/wireless/ath/ath10k/mac.c
@@ -6380,13 +6380,14 @@ static int ath10k_hw_scan(struct ieee80211_hw *hw,
scan_timeout = min_t(u32, arg.max_rest_time *
(arg.n_channels - 1) + (req->duration +
ATH10K_SCAN_CHANNEL_SWITCH_WMI_EVT_OVERHEAD) *
- arg.n_channels, arg.max_scan_time + 200);
-
+ arg.n_channels, arg.max_scan_time);
} else {
- /* Add a 200ms margin to account for event/command processing */
- scan_timeout = arg.max_scan_time + 200;
+ scan_timeout = arg.max_scan_time;
}
+ /* Add a 200ms margin to account for event/command processing */
+ scan_timeout += 200;
+
ret = ath10k_start_scan(ar, &arg);
if (ret) {
ath10k_warn(ar, "failed to start hw scan: %d\n", ret);
diff --git a/drivers/net/wireless/ath/ath10k/txrx.c b/drivers/net/wireless/ath/ath10k/txrx.c
index 7c9ea0c..6f8b642 100644
--- a/drivers/net/wireless/ath/ath10k/txrx.c
+++ b/drivers/net/wireless/ath/ath10k/txrx.c
@@ -82,8 +82,6 @@ int ath10k_txrx_tx_unref(struct ath10k_htt *htt,
flags = skb_cb->flags;
ath10k_htt_tx_free_msdu_id(htt, tx_done->msdu_id);
ath10k_htt_tx_dec_pending(htt);
- if (htt->num_pending_tx == 0)
- wake_up(&htt->empty_tx_wq);
spin_unlock_bh(&htt->tx_lock);
rcu_read_lock();
diff --git a/drivers/net/wireless/ath/ath10k/wmi.c b/drivers/net/wireless/ath/ath10k/wmi.c
index 7c1c265..62c453a 100644
--- a/drivers/net/wireless/ath/ath10k/wmi.c
+++ b/drivers/net/wireless/ath/ath10k/wmi.c
@@ -2611,9 +2611,36 @@ int ath10k_wmi_event_mgmt_rx(struct ath10k *ar, struct sk_buff *skb)
ath10k_mac_handle_beacon(ar, skb);
if (ieee80211_is_beacon(hdr->frame_control) ||
- ieee80211_is_probe_resp(hdr->frame_control))
+ ieee80211_is_probe_resp(hdr->frame_control)) {
+ struct ieee80211_mgmt *mgmt = (void *)skb->data;
+ enum cfg80211_bss_frame_type ftype;
+ u8 *ies;
+ int ies_ch;
+
status->boottime_ns = ktime_get_boottime_ns();
+ if (!ar->scan_channel)
+ goto drop;
+
+ ies = mgmt->u.beacon.variable;
+
+ if (ieee80211_is_beacon(mgmt->frame_control))
+ ftype = CFG80211_BSS_FTYPE_BEACON;
+ else
+ ftype = CFG80211_BSS_FTYPE_PRESP;
+
+ ies_ch = cfg80211_get_ies_channel_number(mgmt->u.beacon.variable,
+ skb_tail_pointer(skb) - ies,
+ sband->band, ftype);
+
+ if (ies_ch > 0 && ies_ch != channel) {
+ ath10k_dbg(ar, ATH10K_DBG_MGMT,
+ "channel mismatched ds channel %d scan channel %d\n",
+ ies_ch, channel);
+ goto drop;
+ }
+ }
+
ath10k_dbg(ar, ATH10K_DBG_MGMT,
"event mgmt rx skb %pK len %d ftype %02x stype %02x\n",
skb, skb->len,
@@ -2627,6 +2654,10 @@ int ath10k_wmi_event_mgmt_rx(struct ath10k *ar, struct sk_buff *skb)
ieee80211_rx_ni(ar->hw, skb);
return 0;
+
+drop:
+ dev_kfree_skb(skb);
+ return 0;
}
static int freq_to_idx(struct ath10k *ar, int freq)
diff --git a/drivers/net/wireless/ath/ath10k/wmi.h b/drivers/net/wireless/ath/ath10k/wmi.h
index 01bfd09..4abd12e 100644
--- a/drivers/net/wireless/ath/ath10k/wmi.h
+++ b/drivers/net/wireless/ath/ath10k/wmi.h
@@ -3478,7 +3478,9 @@ struct wmi_phyerr_event {
__le32 num_phyerrs;
__le32 tsf_l32;
__le32 tsf_u32;
- struct wmi_phyerr phyerrs[];
+
+ /* array of struct wmi_phyerr */
+ u8 phyerrs[];
} __packed;
struct wmi_10_4_phyerr_event {
diff --git a/drivers/net/wireless/ath/ath11k/ahb.c b/drivers/net/wireless/ath/ath11k/ahb.c
index 8c9c781..3fb0aa0 100644
--- a/drivers/net/wireless/ath/ath11k/ahb.c
+++ b/drivers/net/wireless/ath/ath11k/ahb.c
@@ -175,8 +175,11 @@ static void __ath11k_ahb_ext_irq_disable(struct ath11k_base *ab)
ath11k_ahb_ext_grp_disable(irq_grp);
- napi_synchronize(&irq_grp->napi);
- napi_disable(&irq_grp->napi);
+ if (irq_grp->napi_enabled) {
+ napi_synchronize(&irq_grp->napi);
+ napi_disable(&irq_grp->napi);
+ irq_grp->napi_enabled = false;
+ }
}
}
@@ -206,13 +209,13 @@ static void ath11k_ahb_clearbit32(struct ath11k_base *ab, u8 bit, u32 offset)
static void ath11k_ahb_ce_irq_enable(struct ath11k_base *ab, u16 ce_id)
{
- const struct ce_pipe_config *ce_config;
+ const struct ce_attr *ce_attr;
- ce_config = &ab->hw_params.target_ce_config[ce_id];
- if (__le32_to_cpu(ce_config->pipedir) & PIPEDIR_OUT)
+ ce_attr = &ab->hw_params.host_ce_config[ce_id];
+ if (ce_attr->src_nentries)
ath11k_ahb_setbit32(ab, ce_id, CE_HOST_IE_ADDRESS);
- if (__le32_to_cpu(ce_config->pipedir) & PIPEDIR_IN) {
+ if (ce_attr->dest_nentries) {
ath11k_ahb_setbit32(ab, ce_id, CE_HOST_IE_2_ADDRESS);
ath11k_ahb_setbit32(ab, ce_id + CE_HOST_IE_3_SHIFT,
CE_HOST_IE_3_ADDRESS);
@@ -221,13 +224,13 @@ static void ath11k_ahb_ce_irq_enable(struct ath11k_base *ab, u16 ce_id)
static void ath11k_ahb_ce_irq_disable(struct ath11k_base *ab, u16 ce_id)
{
- const struct ce_pipe_config *ce_config;
+ const struct ce_attr *ce_attr;
- ce_config = &ab->hw_params.target_ce_config[ce_id];
- if (__le32_to_cpu(ce_config->pipedir) & PIPEDIR_OUT)
+ ce_attr = &ab->hw_params.host_ce_config[ce_id];
+ if (ce_attr->src_nentries)
ath11k_ahb_clearbit32(ab, ce_id, CE_HOST_IE_ADDRESS);
- if (__le32_to_cpu(ce_config->pipedir) & PIPEDIR_IN) {
+ if (ce_attr->dest_nentries) {
ath11k_ahb_clearbit32(ab, ce_id, CE_HOST_IE_2_ADDRESS);
ath11k_ahb_clearbit32(ab, ce_id + CE_HOST_IE_3_SHIFT,
CE_HOST_IE_3_ADDRESS);
@@ -300,7 +303,10 @@ static void ath11k_ahb_ext_irq_enable(struct ath11k_base *ab)
for (i = 0; i < ATH11K_EXT_IRQ_GRP_NUM_MAX; i++) {
struct ath11k_ext_irq_grp *irq_grp = &ab->ext_irq_grp[i];
- napi_enable(&irq_grp->napi);
+ if (!irq_grp->napi_enabled) {
+ napi_enable(&irq_grp->napi);
+ irq_grp->napi_enabled = true;
+ }
ath11k_ahb_ext_grp_enable(irq_grp);
}
}
diff --git a/drivers/net/wireless/ath/ath11k/ce.c b/drivers/net/wireless/ath/ath11k/ce.c
index de8b632..aaa7b05 100644
--- a/drivers/net/wireless/ath/ath11k/ce.c
+++ b/drivers/net/wireless/ath/ath11k/ce.c
@@ -14,6 +14,7 @@ const struct ce_attr ath11k_host_ce_config_ipq8074[] = {
.src_nentries = 16,
.src_sz_max = 2048,
.dest_nentries = 0,
+ .send_cb = ath11k_htc_tx_completion_handler,
},
/* CE1: target->host HTT + HTC control */
@@ -40,6 +41,7 @@ const struct ce_attr ath11k_host_ce_config_ipq8074[] = {
.src_nentries = 32,
.src_sz_max = 2048,
.dest_nentries = 0,
+ .send_cb = ath11k_htc_tx_completion_handler,
},
/* CE4: host->target HTT */
@@ -73,11 +75,12 @@ const struct ce_attr ath11k_host_ce_config_ipq8074[] = {
.src_nentries = 32,
.src_sz_max = 2048,
.dest_nentries = 0,
+ .send_cb = ath11k_htc_tx_completion_handler,
},
/* CE8: target autonomous hif_memcpy */
{
- .flags = CE_ATTR_FLAGS,
+ .flags = CE_ATTR_FLAGS | CE_ATTR_DIS_INTR,
.src_nentries = 0,
.src_sz_max = 0,
.dest_nentries = 0,
@@ -89,6 +92,7 @@ const struct ce_attr ath11k_host_ce_config_ipq8074[] = {
.src_nentries = 32,
.src_sz_max = 2048,
.dest_nentries = 0,
+ .send_cb = ath11k_htc_tx_completion_handler,
},
/* CE10: target->host HTT */
@@ -142,6 +146,7 @@ const struct ce_attr ath11k_host_ce_config_qca6390[] = {
.src_nentries = 32,
.src_sz_max = 2048,
.dest_nentries = 0,
+ .send_cb = ath11k_htc_tx_completion_handler,
},
/* CE4: host->target HTT */
@@ -175,6 +180,7 @@ const struct ce_attr ath11k_host_ce_config_qca6390[] = {
.src_nentries = 32,
.src_sz_max = 2048,
.dest_nentries = 0,
+ .send_cb = ath11k_htc_tx_completion_handler,
},
/* CE8: target autonomous hif_memcpy */
@@ -220,6 +226,7 @@ const struct ce_attr ath11k_host_ce_config_qcn9074[] = {
.src_nentries = 32,
.src_sz_max = 2048,
.dest_nentries = 0,
+ .send_cb = ath11k_htc_tx_completion_handler,
},
/* CE4: host->target HTT */
@@ -489,18 +496,32 @@ static struct sk_buff *ath11k_ce_completed_send_next(struct ath11k_ce_pipe *pipe
return skb;
}
-static void ath11k_ce_send_done_cb(struct ath11k_ce_pipe *pipe)
+static void ath11k_ce_tx_process_cb(struct ath11k_ce_pipe *pipe)
{
struct ath11k_base *ab = pipe->ab;
struct sk_buff *skb;
+ struct sk_buff_head list;
+ __skb_queue_head_init(&list);
while (!IS_ERR(skb = ath11k_ce_completed_send_next(pipe))) {
if (!skb)
continue;
dma_unmap_single(ab->dev, ATH11K_SKB_CB(skb)->paddr, skb->len,
DMA_TO_DEVICE);
- dev_kfree_skb_any(skb);
+
+ if ((!pipe->send_cb) || ab->hw_params.credit_flow) {
+ dev_kfree_skb_any(skb);
+ continue;
+ }
+
+ __skb_queue_tail(&list, skb);
+ }
+
+ while ((skb = __skb_dequeue(&list))) {
+ ath11k_dbg(ab, ATH11K_DBG_AHB, "tx ce pipe %d len %d\n",
+ pipe->pipe_num, skb->len);
+ pipe->send_cb(ab, skb);
}
}
@@ -636,7 +657,7 @@ static int ath11k_ce_alloc_pipe(struct ath11k_base *ab, int ce_id)
pipe->attr_flags = attr->flags;
if (attr->src_nentries) {
- pipe->send_cb = ath11k_ce_send_done_cb;
+ pipe->send_cb = attr->send_cb;
nentries = roundup_pow_of_two(attr->src_nentries);
desc_sz = ath11k_hal_ce_get_desc_size(HAL_CE_DESC_SRC);
ring = ath11k_ce_alloc_ring(ab, nentries, desc_sz);
@@ -667,9 +688,10 @@ static int ath11k_ce_alloc_pipe(struct ath11k_base *ab, int ce_id)
void ath11k_ce_per_engine_service(struct ath11k_base *ab, u16 ce_id)
{
struct ath11k_ce_pipe *pipe = &ab->ce.ce_pipe[ce_id];
+ const struct ce_attr *attr = &ab->hw_params.host_ce_config[ce_id];
- if (pipe->send_cb)
- pipe->send_cb(pipe);
+ if (attr->src_nentries)
+ ath11k_ce_tx_process_cb(pipe);
if (pipe->recv_cb)
ath11k_ce_recv_process_cb(pipe);
@@ -678,9 +700,10 @@ void ath11k_ce_per_engine_service(struct ath11k_base *ab, u16 ce_id)
void ath11k_ce_poll_send_completed(struct ath11k_base *ab, u8 pipe_id)
{
struct ath11k_ce_pipe *pipe = &ab->ce.ce_pipe[pipe_id];
+ const struct ce_attr *attr = &ab->hw_params.host_ce_config[pipe_id];
- if ((pipe->attr_flags & CE_ATTR_DIS_INTR) && pipe->send_cb)
- pipe->send_cb(pipe);
+ if ((pipe->attr_flags & CE_ATTR_DIS_INTR) && attr->src_nentries)
+ ath11k_ce_tx_process_cb(pipe);
}
EXPORT_SYMBOL(ath11k_ce_per_engine_service);
@@ -953,6 +976,7 @@ int ath11k_ce_init_pipes(struct ath11k_base *ab)
void ath11k_ce_free_pipes(struct ath11k_base *ab)
{
struct ath11k_ce_pipe *pipe;
+ struct ath11k_ce_ring *ce_ring;
int desc_sz;
int i;
@@ -964,22 +988,24 @@ void ath11k_ce_free_pipes(struct ath11k_base *ab)
if (pipe->src_ring) {
desc_sz = ath11k_hal_ce_get_desc_size(HAL_CE_DESC_SRC);
+ ce_ring = pipe->src_ring;
dma_free_coherent(ab->dev,
pipe->src_ring->nentries * desc_sz +
CE_DESC_RING_ALIGN,
- pipe->src_ring->base_addr_owner_space,
- pipe->src_ring->base_addr_ce_space);
+ ce_ring->base_addr_owner_space_unaligned,
+ ce_ring->base_addr_ce_space_unaligned);
kfree(pipe->src_ring);
pipe->src_ring = NULL;
}
if (pipe->dest_ring) {
desc_sz = ath11k_hal_ce_get_desc_size(HAL_CE_DESC_DST);
+ ce_ring = pipe->dest_ring;
dma_free_coherent(ab->dev,
pipe->dest_ring->nentries * desc_sz +
CE_DESC_RING_ALIGN,
- pipe->dest_ring->base_addr_owner_space,
- pipe->dest_ring->base_addr_ce_space);
+ ce_ring->base_addr_owner_space_unaligned,
+ ce_ring->base_addr_ce_space_unaligned);
kfree(pipe->dest_ring);
pipe->dest_ring = NULL;
}
@@ -987,11 +1013,12 @@ void ath11k_ce_free_pipes(struct ath11k_base *ab)
if (pipe->status_ring) {
desc_sz =
ath11k_hal_ce_get_desc_size(HAL_CE_DESC_DST_STATUS);
+ ce_ring = pipe->status_ring;
dma_free_coherent(ab->dev,
pipe->status_ring->nentries * desc_sz +
CE_DESC_RING_ALIGN,
- pipe->status_ring->base_addr_owner_space,
- pipe->status_ring->base_addr_ce_space);
+ ce_ring->base_addr_owner_space_unaligned,
+ ce_ring->base_addr_ce_space_unaligned);
kfree(pipe->status_ring);
pipe->status_ring = NULL;
}
diff --git a/drivers/net/wireless/ath/ath11k/ce.h b/drivers/net/wireless/ath/ath11k/ce.h
index 713f766..8255b6c 100644
--- a/drivers/net/wireless/ath/ath11k/ce.h
+++ b/drivers/net/wireless/ath/ath11k/ce.h
@@ -101,6 +101,7 @@ struct ce_attr {
unsigned int dest_nentries;
void (*recv_cb)(struct ath11k_base *, struct sk_buff *);
+ void (*send_cb)(struct ath11k_base *, struct sk_buff *);
};
#define CE_DESC_RING_ALIGN 8
@@ -154,7 +155,7 @@ struct ath11k_ce_pipe {
unsigned int buf_sz;
unsigned int rx_buf_needed;
- void (*send_cb)(struct ath11k_ce_pipe *);
+ void (*send_cb)(struct ath11k_base *, struct sk_buff *);
void (*recv_cb)(struct ath11k_base *, struct sk_buff *);
struct tasklet_struct intr_tq;
diff --git a/drivers/net/wireless/ath/ath11k/core.c b/drivers/net/wireless/ath/ath11k/core.c
index b5a2af3..293563b 100644
--- a/drivers/net/wireless/ath/ath11k/core.c
+++ b/drivers/net/wireless/ath/ath11k/core.c
@@ -52,6 +52,9 @@ static const struct ath11k_hw_params ath11k_hw_params[] = {
.target_ce_count = 11,
.svc_to_ce_map = ath11k_target_service_to_ce_map_wlan_ipq8074,
.svc_to_ce_map_len = 21,
+ .rfkill_pin = 0,
+ .rfkill_cfg = 0,
+ .rfkill_on_level = 0,
.single_pdev_only = false,
.rxdma1_enable = true,
.num_rxmda_per_pdev = 1,
@@ -74,14 +77,26 @@ static const struct ath11k_hw_params ath11k_hw_params[] = {
BIT(NL80211_IFTYPE_AP) |
BIT(NL80211_IFTYPE_MESH_POINT),
.supports_monitor = true,
+ .full_monitor_mode = false,
.supports_shadow_regs = false,
.idle_ps = false,
+ .supports_sta_ps = false,
.cold_boot_calib = true,
+ .fw_mem_mode = 0,
+ .num_vdevs = 16 + 1,
+ .num_peers = 512,
.supports_suspend = false,
.hal_desc_sz = sizeof(struct hal_rx_desc_ipq8074),
+ .supports_regdb = false,
.fix_l1ss = true,
+ .credit_flow = false,
.max_tx_ring = DP_TCL_NUM_RING_MAX,
.hal_params = &ath11k_hw_hal_params_ipq8074,
+ .supports_dynamic_smps_6ghz = false,
+ .alloc_cacheable_memory = true,
+ .wakeup_mhi = false,
+ .supports_rssi_stats = false,
+ .fw_wmi_diag_event = false,
},
{
.hw_rev = ATH11K_HW_IPQ6018_HW10,
@@ -104,6 +119,9 @@ static const struct ath11k_hw_params ath11k_hw_params[] = {
.target_ce_count = 11,
.svc_to_ce_map = ath11k_target_service_to_ce_map_wlan_ipq6018,
.svc_to_ce_map_len = 19,
+ .rfkill_pin = 0,
+ .rfkill_cfg = 0,
+ .rfkill_on_level = 0,
.single_pdev_only = false,
.rxdma1_enable = true,
.num_rxmda_per_pdev = 1,
@@ -123,14 +141,26 @@ static const struct ath11k_hw_params ath11k_hw_params[] = {
BIT(NL80211_IFTYPE_AP) |
BIT(NL80211_IFTYPE_MESH_POINT),
.supports_monitor = true,
+ .full_monitor_mode = false,
.supports_shadow_regs = false,
.idle_ps = false,
+ .supports_sta_ps = false,
.cold_boot_calib = true,
+ .fw_mem_mode = 0,
+ .num_vdevs = 16 + 1,
+ .num_peers = 512,
.supports_suspend = false,
.hal_desc_sz = sizeof(struct hal_rx_desc_ipq8074),
+ .supports_regdb = false,
.fix_l1ss = true,
+ .credit_flow = false,
.max_tx_ring = DP_TCL_NUM_RING_MAX,
.hal_params = &ath11k_hw_hal_params_ipq8074,
+ .supports_dynamic_smps_6ghz = false,
+ .alloc_cacheable_memory = true,
+ .wakeup_mhi = false,
+ .supports_rssi_stats = false,
+ .fw_wmi_diag_event = false,
},
{
.name = "qca6390 hw2.0",
@@ -153,6 +183,9 @@ static const struct ath11k_hw_params ath11k_hw_params[] = {
.target_ce_count = 9,
.svc_to_ce_map = ath11k_target_service_to_ce_map_wlan_qca6390,
.svc_to_ce_map_len = 14,
+ .rfkill_pin = 48,
+ .rfkill_cfg = 0,
+ .rfkill_on_level = 1,
.single_pdev_only = true,
.rxdma1_enable = false,
.num_rxmda_per_pdev = 2,
@@ -171,14 +204,26 @@ static const struct ath11k_hw_params ath11k_hw_params[] = {
.interface_modes = BIT(NL80211_IFTYPE_STATION) |
BIT(NL80211_IFTYPE_AP),
.supports_monitor = false,
+ .full_monitor_mode = false,
.supports_shadow_regs = true,
.idle_ps = true,
+ .supports_sta_ps = true,
.cold_boot_calib = false,
+ .fw_mem_mode = 0,
+ .num_vdevs = 16 + 1,
+ .num_peers = 512,
.supports_suspend = true,
.hal_desc_sz = sizeof(struct hal_rx_desc_ipq8074),
+ .supports_regdb = true,
.fix_l1ss = true,
+ .credit_flow = true,
.max_tx_ring = DP_TCL_NUM_RING_MAX_QCA6390,
.hal_params = &ath11k_hw_hal_params_qca6390,
+ .supports_dynamic_smps_6ghz = false,
+ .alloc_cacheable_memory = false,
+ .wakeup_mhi = true,
+ .supports_rssi_stats = true,
+ .fw_wmi_diag_event = true,
},
{
.name = "qcn9074 hw1.0",
@@ -201,6 +246,9 @@ static const struct ath11k_hw_params ath11k_hw_params[] = {
.target_ce_count = 9,
.svc_to_ce_map = ath11k_target_service_to_ce_map_wlan_qcn9074,
.svc_to_ce_map_len = 18,
+ .rfkill_pin = 0,
+ .rfkill_cfg = 0,
+ .rfkill_on_level = 0,
.rxdma1_enable = true,
.num_rxmda_per_pdev = 1,
.rx_mac_buf_ring = false,
@@ -219,14 +267,26 @@ static const struct ath11k_hw_params ath11k_hw_params[] = {
BIT(NL80211_IFTYPE_AP) |
BIT(NL80211_IFTYPE_MESH_POINT),
.supports_monitor = true,
+ .full_monitor_mode = true,
.supports_shadow_regs = false,
.idle_ps = false,
+ .supports_sta_ps = false,
.cold_boot_calib = false,
+ .fw_mem_mode = 2,
+ .num_vdevs = 8,
+ .num_peers = 128,
.supports_suspend = false,
.hal_desc_sz = sizeof(struct hal_rx_desc_qcn9074),
+ .supports_regdb = false,
.fix_l1ss = true,
+ .credit_flow = false,
.max_tx_ring = DP_TCL_NUM_RING_MAX,
.hal_params = &ath11k_hw_hal_params_ipq8074,
+ .supports_dynamic_smps_6ghz = true,
+ .alloc_cacheable_memory = true,
+ .wakeup_mhi = false,
+ .supports_rssi_stats = false,
+ .fw_wmi_diag_event = false,
},
{
.name = "wcn6855 hw2.0",
@@ -249,6 +309,72 @@ static const struct ath11k_hw_params ath11k_hw_params[] = {
.target_ce_count = 9,
.svc_to_ce_map = ath11k_target_service_to_ce_map_wlan_qca6390,
.svc_to_ce_map_len = 14,
+ .rfkill_pin = 0,
+ .rfkill_cfg = 0,
+ .rfkill_on_level = 0,
+ .single_pdev_only = true,
+ .rxdma1_enable = false,
+ .num_rxmda_per_pdev = 2,
+ .rx_mac_buf_ring = true,
+ .vdev_start_delay = true,
+ .htt_peer_map_v2 = false,
+
+ .spectral = {
+ .fft_sz = 0,
+ .fft_pad_sz = 0,
+ .summary_pad_sz = 0,
+ .fft_hdr_len = 0,
+ .max_fft_bins = 0,
+ },
+
+ .interface_modes = BIT(NL80211_IFTYPE_STATION) |
+ BIT(NL80211_IFTYPE_AP),
+ .supports_monitor = false,
+ .full_monitor_mode = false,
+ .supports_shadow_regs = true,
+ .idle_ps = true,
+ .supports_sta_ps = true,
+ .cold_boot_calib = false,
+ .fw_mem_mode = 0,
+ .num_vdevs = 16 + 1,
+ .num_peers = 512,
+ .supports_suspend = true,
+ .hal_desc_sz = sizeof(struct hal_rx_desc_wcn6855),
+ .supports_regdb = true,
+ .fix_l1ss = false,
+ .credit_flow = true,
+ .max_tx_ring = DP_TCL_NUM_RING_MAX_QCA6390,
+ .hal_params = &ath11k_hw_hal_params_qca6390,
+ .supports_dynamic_smps_6ghz = false,
+ .alloc_cacheable_memory = false,
+ .wakeup_mhi = true,
+ .supports_rssi_stats = true,
+ .fw_wmi_diag_event = true,
+ },
+ {
+ .name = "wcn6855 hw2.1",
+ .hw_rev = ATH11K_HW_WCN6855_HW21,
+ .fw = {
+ .dir = "WCN6855/hw2.1",
+ .board_size = 256 * 1024,
+ .cal_offset = 128 * 1024,
+ },
+ .max_radios = 3,
+ .bdf_addr = 0x4B0C0000,
+ .hw_ops = &wcn6855_ops,
+ .ring_mask = &ath11k_hw_ring_mask_qca6390,
+ .internal_sleep_clock = true,
+ .regs = &wcn6855_regs,
+ .qmi_service_ins_id = ATH11K_QMI_WLFW_SERVICE_INS_ID_V01_QCA6390,
+ .host_ce_config = ath11k_host_ce_config_qca6390,
+ .ce_count = 9,
+ .target_ce_config = ath11k_target_ce_config_wlan_qca6390,
+ .target_ce_count = 9,
+ .svc_to_ce_map = ath11k_target_service_to_ce_map_wlan_qca6390,
+ .svc_to_ce_map_len = 14,
+ .rfkill_pin = 0,
+ .rfkill_cfg = 0,
+ .rfkill_on_level = 0,
.single_pdev_only = true,
.rxdma1_enable = false,
.num_rxmda_per_pdev = 2,
@@ -269,12 +395,23 @@ static const struct ath11k_hw_params ath11k_hw_params[] = {
.supports_monitor = false,
.supports_shadow_regs = true,
.idle_ps = true,
+ .supports_sta_ps = true,
.cold_boot_calib = false,
+ .fw_mem_mode = 0,
+ .num_vdevs = 16 + 1,
+ .num_peers = 512,
.supports_suspend = true,
.hal_desc_sz = sizeof(struct hal_rx_desc_wcn6855),
+ .supports_regdb = true,
.fix_l1ss = false,
+ .credit_flow = true,
.max_tx_ring = DP_TCL_NUM_RING_MAX_QCA6390,
.hal_params = &ath11k_hw_hal_params_qca6390,
+ .supports_dynamic_smps_6ghz = false,
+ .alloc_cacheable_memory = false,
+ .wakeup_mhi = true,
+ .supports_rssi_stats = true,
+ .fw_wmi_diag_event = true,
},
};
@@ -392,11 +529,26 @@ static int ath11k_core_create_board_name(struct ath11k_base *ab, char *name,
scnprintf(variant, sizeof(variant), ",variant=%s",
ab->qmi.target.bdf_ext);
- scnprintf(name, name_len,
- "bus=%s,qmi-chip-id=%d,qmi-board-id=%d%s",
- ath11k_bus_str(ab->hif.bus),
- ab->qmi.target.chip_id,
- ab->qmi.target.board_id, variant);
+ switch (ab->id.bdf_search) {
+ case ATH11K_BDF_SEARCH_BUS_AND_BOARD:
+ scnprintf(name, name_len,
+ "bus=%s,vendor=%04x,device=%04x,subsystem-vendor=%04x,subsystem-device=%04x,qmi-chip-id=%d,qmi-board-id=%d%s",
+ ath11k_bus_str(ab->hif.bus),
+ ab->id.vendor, ab->id.device,
+ ab->id.subsystem_vendor,
+ ab->id.subsystem_device,
+ ab->qmi.target.chip_id,
+ ab->qmi.target.board_id,
+ variant);
+ break;
+ default:
+ scnprintf(name, name_len,
+ "bus=%s,qmi-chip-id=%d,qmi-board-id=%d%s",
+ ath11k_bus_str(ab->hif.bus),
+ ab->qmi.target.chip_id,
+ ab->qmi.target.board_id, variant);
+ break;
+ }
ath11k_dbg(ab, ATH11K_DBG_BOOT, "boot using board name '%s'\n", name);
@@ -620,10 +772,12 @@ static int ath11k_core_fetch_board_data_api_n(struct ath11k_base *ab,
return ret;
}
-static int ath11k_core_fetch_board_data_api_1(struct ath11k_base *ab,
- struct ath11k_board_data *bd)
+int ath11k_core_fetch_board_data_api_1(struct ath11k_base *ab,
+ struct ath11k_board_data *bd,
+ const char *name)
{
- bd->fw = ath11k_core_firmware_request(ab, ATH11K_DEFAULT_BOARD_FILE);
+ bd->fw = ath11k_core_firmware_request(ab, name);
+
if (IS_ERR(bd->fw))
return PTR_ERR(bd->fw);
@@ -633,7 +787,7 @@ static int ath11k_core_fetch_board_data_api_1(struct ath11k_base *ab,
return 0;
}
-#define BOARD_NAME_SIZE 100
+#define BOARD_NAME_SIZE 200
int ath11k_core_fetch_bdf(struct ath11k_base *ab, struct ath11k_board_data *bd)
{
char boardname[BOARD_NAME_SIZE];
@@ -651,7 +805,7 @@ int ath11k_core_fetch_bdf(struct ath11k_base *ab, struct ath11k_board_data *bd)
goto success;
ab->bd_api = 1;
- ret = ath11k_core_fetch_board_data_api_1(ab, bd);
+ ret = ath11k_core_fetch_board_data_api_1(ab, bd, ATH11K_DEFAULT_BOARD_FILE);
if (ret) {
ath11k_err(ab, "failed to fetch board-2.bin or board.bin from %s\n",
ab->hw_params.fw.dir);
@@ -663,6 +817,18 @@ int ath11k_core_fetch_bdf(struct ath11k_base *ab, struct ath11k_board_data *bd)
return 0;
}
+int ath11k_core_fetch_regdb(struct ath11k_base *ab, struct ath11k_board_data *bd)
+{
+ int ret;
+
+ ret = ath11k_core_fetch_board_data_api_1(ab, bd, ATH11K_REGDB_FILE_NAME);
+ if (ret)
+ ath11k_dbg(ab, ATH11K_DBG_BOOT, "failed to fetch %s from %s\n",
+ ATH11K_REGDB_FILE_NAME, ab->hw_params.fw.dir);
+
+ return ret;
+}
+
static void ath11k_core_stop(struct ath11k_base *ab)
{
if (!test_bit(ATH11K_FLAG_CRASH_FLUSH, &ab->dev_flags))
@@ -893,6 +1059,27 @@ static int ath11k_core_start(struct ath11k_base *ab,
return ret;
}
+static int ath11k_core_rfkill_config(struct ath11k_base *ab)
+{
+ struct ath11k *ar;
+ int ret = 0, i;
+
+ if (!(ab->target_caps.sys_cap_info & WMI_SYS_CAP_INFO_RFKILL))
+ return 0;
+
+ for (i = 0; i < ab->num_radios; i++) {
+ ar = ab->pdevs[i].ar;
+
+ ret = ath11k_mac_rfkill_config(ar);
+ if (ret && ret != -EOPNOTSUPP) {
+ ath11k_warn(ab, "failed to configure rfkill: %d", ret);
+ return ret;
+ }
+ }
+
+ return ret;
+}
+
int ath11k_core_qmi_firmware_ready(struct ath11k_base *ab)
{
int ret;
@@ -939,6 +1126,13 @@ int ath11k_core_qmi_firmware_ready(struct ath11k_base *ab)
goto err_core_stop;
}
ath11k_hif_irq_enable(ab);
+
+ ret = ath11k_core_rfkill_config(ab);
+ if (ret && ret != -EOPNOTSUPP) {
+ ath11k_err(ab, "failed to config rfkill: %d\n", ret);
+ goto err_core_stop;
+ }
+
mutex_unlock(&ab->core_lock);
return 0;
@@ -969,7 +1163,7 @@ static int ath11k_core_reconfigure_on_crash(struct ath11k_base *ab)
ath11k_dp_free(ab);
ath11k_hal_srng_deinit(ab);
- ab->free_vdev_map = (1LL << (ab->num_radios * TARGET_NUM_VDEVS)) - 1;
+ ab->free_vdev_map = (1LL << (ab->num_radios * TARGET_NUM_VDEVS(ab))) - 1;
ret = ath11k_hal_srng_init(ab);
if (ret)
@@ -1003,6 +1197,8 @@ void ath11k_core_halt(struct ath11k *ar)
ath11k_mac_peer_cleanup_all(ar);
cancel_delayed_work_sync(&ar->scan.timeout);
cancel_work_sync(&ar->regd_update_work);
+ cancel_work_sync(&ab->update_11d_work);
+ cancel_work_sync(&ab->rfkill_work);
rcu_assign_pointer(ab->pdevs_active[ar->pdev_idx], NULL);
synchronize_rcu();
@@ -1010,6 +1206,56 @@ void ath11k_core_halt(struct ath11k *ar)
idr_init(&ar->txmgmt_idr);
}
+static void ath11k_rfkill_work(struct work_struct *work)
+{
+ struct ath11k_base *ab = container_of(work, struct ath11k_base, rfkill_work);
+ struct ath11k *ar;
+ bool rfkill_radio_on;
+ int i;
+
+ spin_lock_bh(&ab->base_lock);
+ rfkill_radio_on = ab->rfkill_radio_on;
+ spin_unlock_bh(&ab->base_lock);
+
+ for (i = 0; i < ab->num_radios; i++) {
+ ar = ab->pdevs[i].ar;
+ if (!ar)
+ continue;
+
+ /* notify cfg80211 radio state change */
+ ath11k_mac_rfkill_enable_radio(ar, rfkill_radio_on);
+ wiphy_rfkill_set_hw_state(ar->hw->wiphy, !rfkill_radio_on);
+ }
+}
+
+static void ath11k_update_11d(struct work_struct *work)
+{
+ struct ath11k_base *ab = container_of(work, struct ath11k_base, update_11d_work);
+ struct ath11k *ar;
+ struct ath11k_pdev *pdev;
+ struct wmi_set_current_country_params set_current_param = {};
+ int ret, i;
+
+ spin_lock_bh(&ab->base_lock);
+ memcpy(&set_current_param.alpha2, &ab->new_alpha2, 2);
+ spin_unlock_bh(&ab->base_lock);
+
+ ath11k_dbg(ab, ATH11K_DBG_WMI, "update 11d new cc %c%c\n",
+ set_current_param.alpha2[0],
+ set_current_param.alpha2[1]);
+
+ for (i = 0; i < ab->num_radios; i++) {
+ pdev = &ab->pdevs[i];
+ ar = pdev->ar;
+
+ ret = ath11k_wmi_send_set_current_country_cmd(ar, &set_current_param);
+ if (ret)
+ ath11k_warn(ar->ab,
+ "pdev id %d failed set current country code: %d\n",
+ i, ret);
+ }
+}
+
static void ath11k_core_restart(struct work_struct *work)
{
struct ath11k_base *ab = container_of(work, struct ath11k_base, restart_work);
@@ -1043,6 +1289,7 @@ static void ath11k_core_restart(struct work_struct *work)
idr_for_each(&ar->txmgmt_idr,
ath11k_mac_tx_mgmt_pending_free, ar);
idr_destroy(&ar->txmgmt_idr);
+ wake_up(&ar->txmgmt_empty_waitq);
}
wake_up(&ab->wmi_ab.tx_credits_wq);
@@ -1179,12 +1426,15 @@ struct ath11k_base *ath11k_core_alloc(struct device *dev, size_t priv_size,
mutex_init(&ab->core_lock);
spin_lock_init(&ab->base_lock);
+ mutex_init(&ab->vdev_id_11d_lock);
INIT_LIST_HEAD(&ab->peers);
init_waitqueue_head(&ab->peer_mapping_wq);
init_waitqueue_head(&ab->wmi_ab.tx_credits_wq);
init_waitqueue_head(&ab->qmi.cold_boot_waitq);
INIT_WORK(&ab->restart_work, ath11k_core_restart);
+ INIT_WORK(&ab->update_11d_work, ath11k_update_11d);
+ INIT_WORK(&ab->rfkill_work, ath11k_rfkill_work);
timer_setup(&ab->rx_replenish_retry, ath11k_ce_rx_replenish_retry, 0);
init_completion(&ab->htc_suspend);
init_completion(&ab->wow.wakeup_completed);
diff --git a/drivers/net/wireless/ath/ath11k/core.h b/drivers/net/wireless/ath/ath11k/core.h
index 31d234a..9e88ccc 100644
--- a/drivers/net/wireless/ath/ath11k/core.h
+++ b/drivers/net/wireless/ath/ath11k/core.h
@@ -47,6 +47,11 @@ enum ath11k_supported_bw {
ATH11K_BW_160 = 3,
};
+enum ath11k_bdf_search {
+ ATH11K_BDF_SEARCH_DEFAULT,
+ ATH11K_BDF_SEARCH_BUS_AND_BOARD,
+};
+
enum wme_ac {
WME_AC_BE,
WME_AC_BK,
@@ -112,6 +117,7 @@ enum ath11k_hw_rev {
ATH11K_HW_IPQ6018_HW10,
ATH11K_HW_QCN9074_HW10,
ATH11K_HW_WCN6855_HW20,
+ ATH11K_HW_WCN6855_HW21,
};
enum ath11k_firmware_mode {
@@ -136,6 +142,7 @@ struct ath11k_ext_irq_grp {
u32 num_irq;
u32 grp_id;
u64 timestamp;
+ bool napi_enabled;
struct napi_struct napi;
struct net_device napi_ndev;
};
@@ -194,6 +201,9 @@ enum ath11k_dev_flags {
ATH11K_FLAG_REGISTERED,
ATH11K_FLAG_QMI_FAIL,
ATH11K_FLAG_HTC_SUSPEND_COMPLETE,
+ ATH11K_FLAG_CE_IRQ_ENABLED,
+ ATH11K_FLAG_EXT_IRQ_ENABLED,
+ ATH11K_FLAG_FIXED_MEM_RGN,
};
enum ath11k_monitor_flags {
@@ -240,6 +250,7 @@ struct ath11k_vif {
bool is_started;
bool is_up;
bool spectral_enabled;
+ bool ps;
u32 aid;
u8 bssid[ETH_ALEN];
struct cfg80211_bitrate_mask bitrate_mask;
@@ -249,6 +260,8 @@ struct ath11k_vif {
int txpower;
bool rsnie_present;
bool wpaie_present;
+ bool bcca_zero_sent;
+ bool do_not_send_tmpl;
struct ieee80211_chanctx_conf chanctx;
};
@@ -370,10 +383,13 @@ struct ath11k_sta {
struct work_struct update_wk;
struct work_struct set_4addr_wk;
struct rate_info txrate;
+ u32 peer_nss;
struct rate_info last_txrate;
u64 rx_duration;
u64 tx_duration;
u8 rssi_comb;
+ s8 rssi_beacon;
+ s8 chain_signal[IEEE80211_MAX_CHAINS];
struct ath11k_htt_tx_stats *tx_stats;
struct ath11k_rx_peer_stats *rx_stats;
@@ -404,6 +420,10 @@ enum ath11k_state {
/* Antenna noise floor */
#define ATH11K_DEFAULT_NOISE_FLOOR -95
+#define ATH11K_INVALID_RSSI_FULL -1
+
+#define ATH11K_INVALID_RSSI_EMPTY -128
+
struct ath11k_fw_stats {
struct dentry *debugfs_fwstats;
u32 pdev_id;
@@ -539,6 +559,7 @@ struct ath11k {
/* protects txmgmt_idr data */
spinlock_t txmgmt_idr_lock;
atomic_t num_pending_mgmt_tx;
+ wait_queue_head_t txmgmt_empty_waitq;
/* cycle count is reported twice for each visited channel during scan.
* access protected by data_lock
@@ -577,6 +598,11 @@ struct ath11k {
#endif
bool dfs_block_radar_events;
struct ath11k_thermal thermal;
+ u32 vdev_id_11d_scan;
+ struct completion finish_11d_scan;
+ struct completion finish_11d_ch_list;
+ bool pending_11d;
+ bool regdom_set_by_user;
};
struct ath11k_band_cap {
@@ -703,6 +729,11 @@ struct ath11k_base {
/* Protects data like peers */
spinlock_t base_lock;
struct ath11k_pdev pdevs[MAX_RADIOS];
+ struct {
+ enum WMI_HOST_WLAN_BAND supported_bands;
+ u32 pdev_id;
+ } target_pdev_ids[MAX_RADIOS];
+ u8 target_pdev_count;
struct ath11k_pdev __rcu *pdevs_active[MAX_RADIOS];
struct ath11k_hal_reg_capabilities_ext hal_reg_cap[MAX_RADIOS];
unsigned long long free_vdev_map;
@@ -713,7 +744,6 @@ struct ath11k_base {
u32 wlan_init_status;
int irq_num[ATH11K_IRQ_NUM_MAX];
struct ath11k_ext_irq_grp ext_irq_grp[ATH11K_EXT_IRQ_GRP_NUM_MAX];
- struct napi_struct *napi;
struct ath11k_targ_cap target_caps;
u32 ext_service_bitmap[WMI_SERVICE_EXT_BM_SIZE];
bool pdevs_macaddr_valid;
@@ -746,6 +776,8 @@ struct ath11k_base {
struct completion driver_recovery;
struct workqueue_struct *workqueue;
struct work_struct restart_work;
+ struct work_struct update_11d_work;
+ u8 new_alpha2[3];
struct {
/* protected by data_lock */
u32 fw_crash_counter;
@@ -754,11 +786,25 @@ struct ath11k_base {
struct ath11k_dbring_cap *db_caps;
u32 num_db_cap;
+ struct work_struct rfkill_work;
+ /* true means radio is on */
+ bool rfkill_radio_on;
+
+ /* To synchronize 11d scan vdev id */
+ struct mutex vdev_id_11d_lock;
struct timer_list mon_reap_timer;
struct completion htc_suspend;
+ struct {
+ enum ath11k_bdf_search bdf_search;
+ u32 vendor;
+ u32 device;
+ u32 subsystem_vendor;
+ u32 subsystem_device;
+ } id;
+
/* must be last */
u8 drv_priv[0] __aligned(sizeof(void *));
};
@@ -934,6 +980,10 @@ struct ath11k_base *ath11k_core_alloc(struct device *dev, size_t priv_size,
void ath11k_core_free(struct ath11k_base *ath11k);
int ath11k_core_fetch_bdf(struct ath11k_base *ath11k,
struct ath11k_board_data *bd);
+int ath11k_core_fetch_regdb(struct ath11k_base *ab, struct ath11k_board_data *bd);
+int ath11k_core_fetch_board_data_api_1(struct ath11k_base *ab,
+ struct ath11k_board_data *bd,
+ const char *name);
void ath11k_core_free_bdf(struct ath11k_base *ab, struct ath11k_board_data *bd);
int ath11k_core_check_dt(struct ath11k_base *ath11k);
diff --git a/drivers/net/wireless/ath/ath11k/dbring.c b/drivers/net/wireless/ath/ath11k/dbring.c
index fd98ba5..eda67eb 100644
--- a/drivers/net/wireless/ath/ath11k/dbring.c
+++ b/drivers/net/wireless/ath/ath11k/dbring.c
@@ -6,6 +6,35 @@
#include "core.h"
#include "debug.h"
+#define ATH11K_DB_MAGIC_VALUE 0xdeadbeaf
+
+int ath11k_dbring_validate_buffer(struct ath11k *ar, void *buffer, u32 size)
+{
+ u32 *temp;
+ int idx;
+
+ size = size >> 2;
+
+ for (idx = 0, temp = buffer; idx < size; idx++, temp++) {
+ if (*temp == ATH11K_DB_MAGIC_VALUE)
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static void ath11k_dbring_fill_magic_value(struct ath11k *ar,
+ void *buffer, u32 size)
+{
+ u32 *temp;
+ int idx;
+
+ size = size >> 2;
+
+ for (idx = 0, temp = buffer; idx < size; idx++, temp++)
+ *temp++ = ATH11K_DB_MAGIC_VALUE;
+}
+
static int ath11k_dbring_bufs_replenish(struct ath11k *ar,
struct ath11k_dbring *ring,
struct ath11k_dbring_element *buff)
@@ -26,6 +55,7 @@ static int ath11k_dbring_bufs_replenish(struct ath11k *ar,
ptr_unaligned = buff->payload;
ptr_aligned = PTR_ALIGN(ptr_unaligned, ring->buf_align);
+ ath11k_dbring_fill_magic_value(ar, ptr_aligned, ring->buf_sz);
paddr = dma_map_single(ab->dev, ptr_aligned, ring->buf_sz,
DMA_FROM_DEVICE);
@@ -87,17 +117,23 @@ static int ath11k_dbring_fill_bufs(struct ath11k *ar,
req_entries = min(num_free, ring->bufs_max);
num_remain = req_entries;
align = ring->buf_align;
- size = sizeof(*buff) + ring->buf_sz + align - 1;
+ size = ring->buf_sz + align - 1;
while (num_remain > 0) {
- buff = kzalloc(size, GFP_ATOMIC);
+ buff = kzalloc(sizeof(*buff), GFP_ATOMIC);
if (!buff)
break;
+ buff->payload = kzalloc(size, GFP_ATOMIC);
+ if (!buff->payload) {
+ kfree(buff);
+ break;
+ }
ret = ath11k_dbring_bufs_replenish(ar, ring, buff);
if (ret) {
ath11k_warn(ar->ab, "failed to replenish db ring num_remain %d req_ent %d\n",
num_remain, req_entries);
+ kfree(buff->payload);
kfree(buff);
break;
}
@@ -282,7 +318,7 @@ int ath11k_dbring_buffer_release_event(struct ath11k_base *ab,
srng = &ab->hal.srng_list[ring->refill_srng.ring_id];
num_entry = ev->fixed.num_buf_release_entry;
- size = sizeof(*buff) + ring->buf_sz + ring->buf_align - 1;
+ size = ring->buf_sz + ring->buf_align - 1;
num_buff_reaped = 0;
spin_lock_bh(&srng->lock);
@@ -319,7 +355,8 @@ int ath11k_dbring_buffer_release_event(struct ath11k_base *ab,
ring->handler(ar, &handler_data);
}
- memset(buff, 0, size);
+ buff->paddr = 0;
+ memset(buff->payload, 0, size);
ath11k_dbring_bufs_replenish(ar, ring, buff);
}
@@ -346,6 +383,7 @@ void ath11k_dbring_buf_cleanup(struct ath11k *ar, struct ath11k_dbring *ring)
idr_remove(&ring->bufs_idr, buf_id);
dma_unmap_single(ar->ab->dev, buff->paddr,
ring->buf_sz, DMA_FROM_DEVICE);
+ kfree(buff->payload);
kfree(buff);
}
diff --git a/drivers/net/wireless/ath/ath11k/dbring.h b/drivers/net/wireless/ath/ath11k/dbring.h
index f7fce9e..ef906c6 100644
--- a/drivers/net/wireless/ath/ath11k/dbring.h
+++ b/drivers/net/wireless/ath/ath11k/dbring.h
@@ -13,7 +13,7 @@
struct ath11k_dbring_element {
dma_addr_t paddr;
- u8 payload[0];
+ u8 *payload;
};
struct ath11k_dbring_data {
@@ -76,4 +76,6 @@ int ath11k_dbring_get_cap(struct ath11k_base *ab,
struct ath11k_dbring_cap *db_cap);
void ath11k_dbring_srng_cleanup(struct ath11k *ar, struct ath11k_dbring *ring);
void ath11k_dbring_buf_cleanup(struct ath11k *ar, struct ath11k_dbring *ring);
+int ath11k_dbring_validate_buffer(struct ath11k *ar, void *data, u32 size);
+
#endif /* ATH11K_DBRING_H */
diff --git a/drivers/net/wireless/ath/ath11k/debug.c b/drivers/net/wireless/ath/ath11k/debug.c
index c86de95..958d874 100644
--- a/drivers/net/wireless/ath/ath11k/debug.c
+++ b/drivers/net/wireless/ath/ath11k/debug.c
@@ -17,7 +17,7 @@ void ath11k_info(struct ath11k_base *ab, const char *fmt, ...)
va_start(args, fmt);
vaf.va = &args;
dev_info(ab->dev, "%pV", &vaf);
- /* TODO: Trace the log */
+ trace_ath11k_log_info(ab, &vaf);
va_end(args);
}
EXPORT_SYMBOL(ath11k_info);
@@ -32,7 +32,7 @@ void ath11k_err(struct ath11k_base *ab, const char *fmt, ...)
va_start(args, fmt);
vaf.va = &args;
dev_err(ab->dev, "%pV", &vaf);
- /* TODO: Trace the log */
+ trace_ath11k_log_err(ab, &vaf);
va_end(args);
}
EXPORT_SYMBOL(ath11k_err);
@@ -47,7 +47,7 @@ void ath11k_warn(struct ath11k_base *ab, const char *fmt, ...)
va_start(args, fmt);
vaf.va = &args;
dev_warn_ratelimited(ab->dev, "%pV", &vaf);
- /* TODO: Trace the log */
+ trace_ath11k_log_warn(ab, &vaf);
va_end(args);
}
EXPORT_SYMBOL(ath11k_warn);
@@ -68,7 +68,7 @@ void __ath11k_dbg(struct ath11k_base *ab, enum ath11k_debug_mask mask,
if (ath11k_debug_mask & mask)
dev_printk(KERN_DEBUG, ab->dev, "%pV", &vaf);
- /* TODO: trace log */
+ trace_ath11k_log_dbg(ab, mask, &vaf);
va_end(args);
}
@@ -100,6 +100,10 @@ void ath11k_dbg_dump(struct ath11k_base *ab,
dev_printk(KERN_DEBUG, ab->dev, "%s\n", linebuf);
}
}
+
+ /* tracing code doesn't like null strings */
+ trace_ath11k_log_dbg_dump(ab, msg ? msg : "", prefix ? prefix : "",
+ buf, len);
}
EXPORT_SYMBOL(ath11k_dbg_dump);
diff --git a/drivers/net/wireless/ath/ath11k/debug.h b/drivers/net/wireless/ath/ath11k/debug.h
index 659a275..fbbd5fe 100644
--- a/drivers/net/wireless/ath/ath11k/debug.h
+++ b/drivers/net/wireless/ath/ath11k/debug.h
@@ -60,7 +60,8 @@ static inline void ath11k_dbg_dump(struct ath11k_base *ab,
#define ath11k_dbg(ar, dbg_mask, fmt, ...) \
do { \
- if (ath11k_debug_mask & dbg_mask) \
+ if ((ath11k_debug_mask & dbg_mask) || \
+ trace_ath11k_log_dbg_enabled()) \
__ath11k_dbg(ar, dbg_mask, fmt, ##__VA_ARGS__); \
} while (0)
diff --git a/drivers/net/wireless/ath/ath11k/debugfs.c b/drivers/net/wireless/ath/ath11k/debugfs.c
index 80afd35..198ade9 100644
--- a/drivers/net/wireless/ath/ath11k/debugfs.c
+++ b/drivers/net/wireless/ath/ath11k/debugfs.c
@@ -3,6 +3,8 @@
* Copyright (c) 2018-2020 The Linux Foundation. All rights reserved.
*/
+#include <linux/vmalloc.h>
+
#include "debugfs.h"
#include "core.h"
@@ -126,6 +128,11 @@ void ath11k_debugfs_fw_stats_process(struct ath11k_base *ab, struct sk_buff *skb
goto complete;
}
+ if (stats.stats_id == WMI_REQUEST_RSSI_PER_CHAIN_STAT) {
+ ar->debug.fw_stats_done = true;
+ goto complete;
+ }
+
if (stats.stats_id == WMI_REQUEST_VDEV_STAT) {
if (list_empty(&stats.vdevs)) {
ath11k_warn(ab, "empty vdev stats");
@@ -195,7 +202,7 @@ static int ath11k_debugfs_fw_stats_request(struct ath11k *ar,
* received 'update stats' event, we keep a 3 seconds timeout in case,
* fw_stats_done is not marked yet
*/
- timeout = jiffies + msecs_to_jiffies(3 * HZ);
+ timeout = jiffies + msecs_to_jiffies(3 * 1000);
ath11k_debugfs_fw_stats_reset(ar);
@@ -229,6 +236,38 @@ static int ath11k_debugfs_fw_stats_request(struct ath11k *ar,
return 0;
}
+int ath11k_debugfs_get_fw_stats(struct ath11k *ar, u32 pdev_id,
+ u32 vdev_id, u32 stats_id)
+{
+ struct ath11k_base *ab = ar->ab;
+ struct stats_request_params req_param;
+ int ret;
+
+ mutex_lock(&ar->conf_mutex);
+
+ if (ar->state != ATH11K_STATE_ON) {
+ ret = -ENETDOWN;
+ goto err_unlock;
+ }
+
+ req_param.pdev_id = pdev_id;
+ req_param.vdev_id = vdev_id;
+ req_param.stats_id = stats_id;
+
+ ret = ath11k_debugfs_fw_stats_request(ar, &req_param);
+ if (ret)
+ ath11k_warn(ab, "failed to request fw stats: %d\n", ret);
+
+ ath11k_dbg(ab, ATH11K_DBG_WMI,
+ "debug get fw stat pdev id %d vdev id %d stats id 0x%x\n",
+ pdev_id, vdev_id, stats_id);
+
+err_unlock:
+ mutex_unlock(&ar->conf_mutex);
+
+ return ret;
+}
+
static int ath11k_open_pdev_stats(struct inode *inode, struct file *file)
{
struct ath11k *ar = inode->i_private;
diff --git a/drivers/net/wireless/ath/ath11k/debugfs.h b/drivers/net/wireless/ath/ath11k/debugfs.h
index ec743a0..4c07403 100644
--- a/drivers/net/wireless/ath/ath11k/debugfs.h
+++ b/drivers/net/wireless/ath/ath11k/debugfs.h
@@ -117,6 +117,8 @@ void ath11k_debugfs_unregister(struct ath11k *ar);
void ath11k_debugfs_fw_stats_process(struct ath11k_base *ab, struct sk_buff *skb);
void ath11k_debugfs_fw_stats_init(struct ath11k *ar);
+int ath11k_debugfs_get_fw_stats(struct ath11k *ar, u32 pdev_id,
+ u32 vdev_id, u32 stats_id);
static inline bool ath11k_debugfs_is_pktlog_lite_mode_enabled(struct ath11k *ar)
{
@@ -216,6 +218,12 @@ static inline int ath11k_debugfs_rx_filter(struct ath11k *ar)
return 0;
}
+static inline int ath11k_debugfs_get_fw_stats(struct ath11k *ar,
+ u32 pdev_id, u32 vdev_id, u32 stats_id)
+{
+ return 0;
+}
+
#endif /* CONFIG_MAC80211_DEBUGFS*/
#endif /* _ATH11K_DEBUGFS_H_ */
diff --git a/drivers/net/wireless/ath/ath11k/debugfs_sta.c b/drivers/net/wireless/ath/ath11k/debugfs_sta.c
index fecd971..1b1acbd 100644
--- a/drivers/net/wireless/ath/ath11k/debugfs_sta.c
+++ b/drivers/net/wireless/ath/ath11k/debugfs_sta.c
@@ -126,85 +126,9 @@ void ath11k_debugfs_sta_add_tx_stats(struct ath11k_sta *arsta,
}
void ath11k_debugfs_sta_update_txcompl(struct ath11k *ar,
- struct sk_buff *msdu,
struct hal_tx_status *ts)
{
- struct ath11k_base *ab = ar->ab;
- struct ath11k_per_peer_tx_stats *peer_stats = &ar->cached_stats;
- enum hal_tx_rate_stats_pkt_type pkt_type;
- enum hal_tx_rate_stats_sgi sgi;
- enum hal_tx_rate_stats_bw bw;
- struct ath11k_peer *peer;
- struct ath11k_sta *arsta;
- struct ieee80211_sta *sta;
- u16 rate;
- u8 rate_idx = 0;
- int ret;
- u8 mcs;
-
- rcu_read_lock();
- spin_lock_bh(&ab->base_lock);
- peer = ath11k_peer_find_by_id(ab, ts->peer_id);
- if (!peer || !peer->sta) {
- ath11k_warn(ab, "failed to find the peer\n");
- spin_unlock_bh(&ab->base_lock);
- rcu_read_unlock();
- return;
- }
-
- sta = peer->sta;
- arsta = (struct ath11k_sta *)sta->drv_priv;
-
- memset(&arsta->txrate, 0, sizeof(arsta->txrate));
- pkt_type = FIELD_GET(HAL_TX_RATE_STATS_INFO0_PKT_TYPE,
- ts->rate_stats);
- mcs = FIELD_GET(HAL_TX_RATE_STATS_INFO0_MCS,
- ts->rate_stats);
- sgi = FIELD_GET(HAL_TX_RATE_STATS_INFO0_SGI,
- ts->rate_stats);
- bw = FIELD_GET(HAL_TX_RATE_STATS_INFO0_BW, ts->rate_stats);
-
- if (pkt_type == HAL_TX_RATE_STATS_PKT_TYPE_11A ||
- pkt_type == HAL_TX_RATE_STATS_PKT_TYPE_11B) {
- ret = ath11k_mac_hw_ratecode_to_legacy_rate(mcs,
- pkt_type,
- &rate_idx,
- &rate);
- if (ret < 0)
- goto err_out;
- arsta->txrate.legacy = rate;
- } else if (pkt_type == HAL_TX_RATE_STATS_PKT_TYPE_11N) {
- if (mcs > 7) {
- ath11k_warn(ab, "Invalid HT mcs index %d\n", mcs);
- goto err_out;
- }
-
- arsta->txrate.mcs = mcs + 8 * (arsta->last_txrate.nss - 1);
- arsta->txrate.flags = RATE_INFO_FLAGS_MCS;
- if (sgi)
- arsta->txrate.flags |= RATE_INFO_FLAGS_SHORT_GI;
- } else if (pkt_type == HAL_TX_RATE_STATS_PKT_TYPE_11AC) {
- if (mcs > 9) {
- ath11k_warn(ab, "Invalid VHT mcs index %d\n", mcs);
- goto err_out;
- }
-
- arsta->txrate.mcs = mcs;
- arsta->txrate.flags = RATE_INFO_FLAGS_VHT_MCS;
- if (sgi)
- arsta->txrate.flags |= RATE_INFO_FLAGS_SHORT_GI;
- } else if (pkt_type == HAL_TX_RATE_STATS_PKT_TYPE_11AX) {
- /* TODO */
- }
-
- arsta->txrate.nss = arsta->last_txrate.nss;
- arsta->txrate.bw = ath11k_mac_bw_to_mac80211_bw(bw);
-
- ath11k_debugfs_sta_add_tx_stats(arsta, peer_stats, rate_idx);
-
-err_out:
- spin_unlock_bh(&ab->base_lock);
- rcu_read_unlock();
+ ath11k_dp_tx_update_txcompl(ar, ts);
}
static ssize_t ath11k_dbg_sta_dump_tx_stats(struct file *file,
diff --git a/drivers/net/wireless/ath/ath11k/debugfs_sta.h b/drivers/net/wireless/ath/ath11k/debugfs_sta.h
index 18dc65d..e6c11b3 100644
--- a/drivers/net/wireless/ath/ath11k/debugfs_sta.h
+++ b/drivers/net/wireless/ath/ath11k/debugfs_sta.h
@@ -19,7 +19,6 @@ void ath11k_debugfs_sta_add_tx_stats(struct ath11k_sta *arsta,
struct ath11k_per_peer_tx_stats *peer_stats,
u8 legacy_rate_idx);
void ath11k_debugfs_sta_update_txcompl(struct ath11k *ar,
- struct sk_buff *msdu,
struct hal_tx_status *ts);
#else /* CONFIG_ATH11K_DEBUGFS */
@@ -34,7 +33,6 @@ ath11k_debugfs_sta_add_tx_stats(struct ath11k_sta *arsta,
}
static inline void ath11k_debugfs_sta_update_txcompl(struct ath11k *ar,
- struct sk_buff *msdu,
struct hal_tx_status *ts)
{
}
diff --git a/drivers/net/wireless/ath/ath11k/dp.c b/drivers/net/wireless/ath/ath11k/dp.c
index 8baaeeb..8b790ce 100644
--- a/drivers/net/wireless/ath/ath11k/dp.c
+++ b/drivers/net/wireless/ath/ath11k/dp.c
@@ -101,8 +101,11 @@ void ath11k_dp_srng_cleanup(struct ath11k_base *ab, struct dp_srng *ring)
if (!ring->vaddr_unaligned)
return;
- dma_free_coherent(ab->dev, ring->size, ring->vaddr_unaligned,
- ring->paddr_unaligned);
+ if (ring->cached)
+ kfree(ring->vaddr_unaligned);
+ else
+ dma_free_coherent(ab->dev, ring->size, ring->vaddr_unaligned,
+ ring->paddr_unaligned);
ring->vaddr_unaligned = NULL;
}
@@ -222,6 +225,7 @@ int ath11k_dp_srng_setup(struct ath11k_base *ab, struct dp_srng *ring,
int entry_sz = ath11k_hal_srng_get_entrysize(ab, type);
int max_entries = ath11k_hal_srng_get_max_entries(ab, type);
int ret;
+ bool cached = false;
if (max_entries < 0 || entry_sz < 0)
return -EINVAL;
@@ -230,9 +234,29 @@ int ath11k_dp_srng_setup(struct ath11k_base *ab, struct dp_srng *ring,
num_entries = max_entries;
ring->size = (num_entries * entry_sz) + HAL_RING_BASE_ALIGN - 1;
- ring->vaddr_unaligned = dma_alloc_coherent(ab->dev, ring->size,
- &ring->paddr_unaligned,
- GFP_KERNEL);
+
+ if (ab->hw_params.alloc_cacheable_memory) {
+ /* Allocate the reo dst and tx completion rings from cacheable memory */
+ switch (type) {
+ case HAL_REO_DST:
+ case HAL_WBM2SW_RELEASE:
+ cached = true;
+ break;
+ default:
+ cached = false;
+ }
+
+ if (cached) {
+ ring->vaddr_unaligned = kzalloc(ring->size, GFP_KERNEL);
+ ring->paddr_unaligned = virt_to_phys(ring->vaddr_unaligned);
+ }
+ }
+
+ if (!cached)
+ ring->vaddr_unaligned = dma_alloc_coherent(ab->dev, ring->size,
+ &ring->paddr_unaligned,
+ GFP_KERNEL);
+
if (!ring->vaddr_unaligned)
return -ENOMEM;
@@ -292,6 +316,11 @@ int ath11k_dp_srng_setup(struct ath11k_base *ab, struct dp_srng *ring,
return -EINVAL;
}
+ if (cached) {
+ params.flags |= HAL_SRNG_FLAGS_CACHED;
+ ring->cached = 1;
+ }
+
ret = ath11k_hal_srng_setup(ab, type, ring_num, mac_id, ¶ms);
if (ret < 0) {
ath11k_warn(ab, "failed to setup srng: %d ring_id %d\n",
@@ -742,13 +771,12 @@ int ath11k_dp_service_srng(struct ath11k_base *ab,
const struct ath11k_hw_hal_params *hal_params;
int grp_id = irq_grp->grp_id;
int work_done = 0;
- int i = 0, j;
+ int i, j;
int tot_work_done = 0;
- while (ab->hw_params.ring_mask->tx[grp_id] >> i) {
- if (ab->hw_params.ring_mask->tx[grp_id] & BIT(i))
- ath11k_dp_tx_completion_handler(ab, i);
- i++;
+ if (ab->hw_params.ring_mask->tx[grp_id]) {
+ i = __fls(ab->hw_params.ring_mask->tx[grp_id]);
+ ath11k_dp_tx_completion_handler(ab, i);
}
if (ab->hw_params.ring_mask->rx_err[grp_id]) {
@@ -1023,6 +1051,7 @@ int ath11k_dp_alloc(struct ath11k_base *ab)
INIT_LIST_HEAD(&dp->reo_cmd_list);
INIT_LIST_HEAD(&dp->reo_cmd_cache_flush_list);
+ INIT_LIST_HEAD(&dp->dp_full_mon_mpdu_list);
spin_lock_init(&dp->reo_cmd_lock);
dp->reo_cmd_cache_flush_count = 0;
diff --git a/drivers/net/wireless/ath/ath11k/dp.h b/drivers/net/wireless/ath/ath11k/dp.h
index 4794ca0..409d6cc 100644
--- a/drivers/net/wireless/ath/ath11k/dp.h
+++ b/drivers/net/wireless/ath/ath11k/dp.h
@@ -64,6 +64,7 @@ struct dp_srng {
dma_addr_t paddr;
int size;
u32 ring_id;
+ u8 cached;
};
struct dp_rxdma_ring {
@@ -88,6 +89,19 @@ struct dp_tx_ring {
int tx_status_tail;
};
+enum dp_mon_status_buf_state {
+ /* PPDU id matches in dst ring and status ring */
+ DP_MON_STATUS_MATCH,
+ /* status ring dma is not done */
+ DP_MON_STATUS_NO_DMA,
+ /* status ring is lagging, reap status ring */
+ DP_MON_STATUS_LAG,
+ /* status ring is leading, reap dst ring and drop */
+ DP_MON_STATUS_LEAD,
+ /* replinish monitor status ring */
+ DP_MON_STATUS_REPLINISH,
+};
+
struct ath11k_pdev_mon_stats {
u32 status_ppdu_state;
u32 status_ppdu_start;
@@ -103,6 +117,12 @@ struct ath11k_pdev_mon_stats {
u32 dup_mon_buf_cnt;
};
+struct dp_full_mon_mpdu {
+ struct list_head list;
+ struct sk_buff *head;
+ struct sk_buff *tail;
+};
+
struct dp_link_desc_bank {
void *vaddr_unaligned;
void *vaddr;
@@ -134,7 +154,11 @@ struct ath11k_mon_data {
u32 mon_last_buf_cookie;
u64 mon_last_linkdesc_paddr;
u16 chan_noise_floor;
-
+ bool hold_mon_dst_ring;
+ enum dp_mon_status_buf_state buf_state;
+ dma_addr_t mon_status_paddr;
+ struct dp_full_mon_mpdu *mon_mpdu;
+ struct hal_sw_mon_ring_entries sw_mon_entries;
struct ath11k_pdev_mon_stats rx_mon_stats;
/* lock for monitor data */
spinlock_t mon_lock;
@@ -244,6 +268,7 @@ struct ath11k_dp {
struct hal_wbm_idle_scatter_list scatter_list[DP_IDLE_SCATTER_BUFS_MAX];
struct list_head reo_cmd_list;
struct list_head reo_cmd_cache_flush_list;
+ struct list_head dp_full_mon_mpdu_list;
u32 reo_cmd_cache_flush_count;
/**
* protects access to below fields,
@@ -291,6 +316,7 @@ enum htt_h2t_msg_type {
HTT_H2T_MSG_TYPE_RX_RING_SELECTION_CFG = 0xc,
HTT_H2T_MSG_TYPE_EXT_STATS_CFG = 0x10,
HTT_H2T_MSG_TYPE_PPDU_STATS_CFG = 0x11,
+ HTT_H2T_MSG_TYPE_RX_FULL_MONITOR_MODE = 0x17,
};
#define HTT_VER_REQ_INFO_MSG_ID GENMASK(7, 0)
@@ -517,7 +543,8 @@ struct htt_ppdu_stats_cfg_cmd {
} __packed;
#define HTT_PPDU_STATS_CFG_MSG_TYPE GENMASK(7, 0)
-#define HTT_PPDU_STATS_CFG_PDEV_ID GENMASK(15, 8)
+#define HTT_PPDU_STATS_CFG_SOC_STATS BIT(8)
+#define HTT_PPDU_STATS_CFG_PDEV_ID GENMASK(15, 9)
#define HTT_PPDU_STATS_CFG_TLV_TYPE_BITMASK GENMASK(31, 16)
enum htt_ppdu_stats_tag_type {
@@ -955,6 +982,33 @@ struct htt_rx_ring_tlv_filter {
u32 pkt_filter_flags3; /* DATA */
};
+#define HTT_RX_FULL_MON_MODE_CFG_CMD_INFO0_MSG_TYPE GENMASK(7, 0)
+#define HTT_RX_FULL_MON_MODE_CFG_CMD_INFO0_PDEV_ID GENMASK(15, 8)
+
+#define HTT_RX_FULL_MON_MODE_CFG_CMD_CFG_ENABLE BIT(0)
+#define HTT_RX_FULL_MON_MODE_CFG_CMD_CFG_ZERO_MPDUS_END BIT(1)
+#define HTT_RX_FULL_MON_MODE_CFG_CMD_CFG_NON_ZERO_MPDUS_END BIT(2)
+#define HTT_RX_FULL_MON_MODE_CFG_CMD_CFG_RELEASE_RING GENMASK(10, 3)
+
+/**
+ * Enumeration for full monitor mode destination ring select
+ * 0 - REO destination ring select
+ * 1 - FW destination ring select
+ * 2 - SW destination ring select
+ * 3 - Release destination ring select
+ */
+enum htt_rx_full_mon_release_ring {
+ HTT_RX_MON_RING_REO,
+ HTT_RX_MON_RING_FW,
+ HTT_RX_MON_RING_SW,
+ HTT_RX_MON_RING_RELEASE,
+};
+
+struct htt_rx_full_monitor_mode_cfg_cmd {
+ u32 info0;
+ u32 cfg;
+} __packed;
+
/* HTT message target->host */
enum htt_t2h_msg_type {
diff --git a/drivers/net/wireless/ath/ath11k/dp_rx.c b/drivers/net/wireless/ath/ath11k/dp_rx.c
index c532084..c212a78 100644
--- a/drivers/net/wireless/ath/ath11k/dp_rx.c
+++ b/drivers/net/wireless/ath/ath11k/dp_rx.c
@@ -20,13 +20,15 @@
#define ATH11K_DP_RX_FRAGMENT_TIMEOUT_MS (2 * HZ)
-static u8 *ath11k_dp_rx_h_80211_hdr(struct ath11k_base *ab, struct hal_rx_desc *desc)
+static inline
+u8 *ath11k_dp_rx_h_80211_hdr(struct ath11k_base *ab, struct hal_rx_desc *desc)
{
return ab->hw_params.hw_ops->rx_desc_get_hdr_status(desc);
}
-static enum hal_encrypt_type ath11k_dp_rx_h_mpdu_start_enctype(struct ath11k_base *ab,
- struct hal_rx_desc *desc)
+static inline
+enum hal_encrypt_type ath11k_dp_rx_h_mpdu_start_enctype(struct ath11k_base *ab,
+ struct hal_rx_desc *desc)
{
if (!ab->hw_params.hw_ops->rx_desc_encrypt_valid(desc))
return HAL_ENCRYPT_TYPE_OPEN;
@@ -34,32 +36,34 @@ static enum hal_encrypt_type ath11k_dp_rx_h_mpdu_start_enctype(struct ath11k_bas
return ab->hw_params.hw_ops->rx_desc_get_encrypt_type(desc);
}
-static u8 ath11k_dp_rx_h_msdu_start_decap_type(struct ath11k_base *ab,
- struct hal_rx_desc *desc)
+static inline u8 ath11k_dp_rx_h_msdu_start_decap_type(struct ath11k_base *ab,
+ struct hal_rx_desc *desc)
{
return ab->hw_params.hw_ops->rx_desc_get_decap_type(desc);
}
-static u8 ath11k_dp_rx_h_msdu_start_mesh_ctl_present(struct ath11k_base *ab,
- struct hal_rx_desc *desc)
+static inline
+u8 ath11k_dp_rx_h_msdu_start_mesh_ctl_present(struct ath11k_base *ab,
+ struct hal_rx_desc *desc)
{
return ab->hw_params.hw_ops->rx_desc_get_mesh_ctl(desc);
}
-static bool ath11k_dp_rx_h_mpdu_start_seq_ctrl_valid(struct ath11k_base *ab,
- struct hal_rx_desc *desc)
+static inline
+bool ath11k_dp_rx_h_mpdu_start_seq_ctrl_valid(struct ath11k_base *ab,
+ struct hal_rx_desc *desc)
{
return ab->hw_params.hw_ops->rx_desc_get_mpdu_seq_ctl_vld(desc);
}
-static bool ath11k_dp_rx_h_mpdu_start_fc_valid(struct ath11k_base *ab,
- struct hal_rx_desc *desc)
+static inline bool ath11k_dp_rx_h_mpdu_start_fc_valid(struct ath11k_base *ab,
+ struct hal_rx_desc *desc)
{
return ab->hw_params.hw_ops->rx_desc_get_mpdu_fc_valid(desc);
}
-static bool ath11k_dp_rx_h_mpdu_start_more_frags(struct ath11k_base *ab,
- struct sk_buff *skb)
+static inline bool ath11k_dp_rx_h_mpdu_start_more_frags(struct ath11k_base *ab,
+ struct sk_buff *skb)
{
struct ieee80211_hdr *hdr;
@@ -67,8 +71,8 @@ static bool ath11k_dp_rx_h_mpdu_start_more_frags(struct ath11k_base *ab,
return ieee80211_has_morefrags(hdr->frame_control);
}
-static u16 ath11k_dp_rx_h_mpdu_start_frag_no(struct ath11k_base *ab,
- struct sk_buff *skb)
+static inline u16 ath11k_dp_rx_h_mpdu_start_frag_no(struct ath11k_base *ab,
+ struct sk_buff *skb)
{
struct ieee80211_hdr *hdr;
@@ -76,37 +80,37 @@ static u16 ath11k_dp_rx_h_mpdu_start_frag_no(struct ath11k_base *ab,
return le16_to_cpu(hdr->seq_ctrl) & IEEE80211_SCTL_FRAG;
}
-static u16 ath11k_dp_rx_h_mpdu_start_seq_no(struct ath11k_base *ab,
- struct hal_rx_desc *desc)
+static inline u16 ath11k_dp_rx_h_mpdu_start_seq_no(struct ath11k_base *ab,
+ struct hal_rx_desc *desc)
{
return ab->hw_params.hw_ops->rx_desc_get_mpdu_start_seq_no(desc);
}
-static void *ath11k_dp_rx_get_attention(struct ath11k_base *ab,
- struct hal_rx_desc *desc)
+static inline void *ath11k_dp_rx_get_attention(struct ath11k_base *ab,
+ struct hal_rx_desc *desc)
{
return ab->hw_params.hw_ops->rx_desc_get_attention(desc);
}
-static bool ath11k_dp_rx_h_attn_msdu_done(struct rx_attention *attn)
+static inline bool ath11k_dp_rx_h_attn_msdu_done(struct rx_attention *attn)
{
return !!FIELD_GET(RX_ATTENTION_INFO2_MSDU_DONE,
__le32_to_cpu(attn->info2));
}
-static bool ath11k_dp_rx_h_attn_l4_cksum_fail(struct rx_attention *attn)
+static inline bool ath11k_dp_rx_h_attn_l4_cksum_fail(struct rx_attention *attn)
{
return !!FIELD_GET(RX_ATTENTION_INFO1_TCP_UDP_CKSUM_FAIL,
__le32_to_cpu(attn->info1));
}
-static bool ath11k_dp_rx_h_attn_ip_cksum_fail(struct rx_attention *attn)
+static inline bool ath11k_dp_rx_h_attn_ip_cksum_fail(struct rx_attention *attn)
{
return !!FIELD_GET(RX_ATTENTION_INFO1_IP_CKSUM_FAIL,
__le32_to_cpu(attn->info1));
}
-static bool ath11k_dp_rx_h_attn_is_decrypted(struct rx_attention *attn)
+static inline bool ath11k_dp_rx_h_attn_is_decrypted(struct rx_attention *attn)
{
return (FIELD_GET(RX_ATTENTION_INFO2_DCRYPT_STATUS_CODE,
__le32_to_cpu(attn->info2)) ==
@@ -154,68 +158,68 @@ static bool ath11k_dp_rx_h_attn_msdu_len_err(struct ath11k_base *ab,
return errmap & DP_RX_MPDU_ERR_MSDU_LEN;
}
-static u16 ath11k_dp_rx_h_msdu_start_msdu_len(struct ath11k_base *ab,
- struct hal_rx_desc *desc)
+static inline u16 ath11k_dp_rx_h_msdu_start_msdu_len(struct ath11k_base *ab,
+ struct hal_rx_desc *desc)
{
return ab->hw_params.hw_ops->rx_desc_get_msdu_len(desc);
}
-static u8 ath11k_dp_rx_h_msdu_start_sgi(struct ath11k_base *ab,
- struct hal_rx_desc *desc)
+static inline u8 ath11k_dp_rx_h_msdu_start_sgi(struct ath11k_base *ab,
+ struct hal_rx_desc *desc)
{
return ab->hw_params.hw_ops->rx_desc_get_msdu_sgi(desc);
}
-static u8 ath11k_dp_rx_h_msdu_start_rate_mcs(struct ath11k_base *ab,
- struct hal_rx_desc *desc)
+static inline u8 ath11k_dp_rx_h_msdu_start_rate_mcs(struct ath11k_base *ab,
+ struct hal_rx_desc *desc)
{
return ab->hw_params.hw_ops->rx_desc_get_msdu_rate_mcs(desc);
}
-static u8 ath11k_dp_rx_h_msdu_start_rx_bw(struct ath11k_base *ab,
- struct hal_rx_desc *desc)
+static inline u8 ath11k_dp_rx_h_msdu_start_rx_bw(struct ath11k_base *ab,
+ struct hal_rx_desc *desc)
{
return ab->hw_params.hw_ops->rx_desc_get_msdu_rx_bw(desc);
}
-static u32 ath11k_dp_rx_h_msdu_start_freq(struct ath11k_base *ab,
- struct hal_rx_desc *desc)
+static inline u32 ath11k_dp_rx_h_msdu_start_freq(struct ath11k_base *ab,
+ struct hal_rx_desc *desc)
{
return ab->hw_params.hw_ops->rx_desc_get_msdu_freq(desc);
}
-static u8 ath11k_dp_rx_h_msdu_start_pkt_type(struct ath11k_base *ab,
- struct hal_rx_desc *desc)
+static inline u8 ath11k_dp_rx_h_msdu_start_pkt_type(struct ath11k_base *ab,
+ struct hal_rx_desc *desc)
{
return ab->hw_params.hw_ops->rx_desc_get_msdu_pkt_type(desc);
}
-static u8 ath11k_dp_rx_h_msdu_start_nss(struct ath11k_base *ab,
- struct hal_rx_desc *desc)
+static inline u8 ath11k_dp_rx_h_msdu_start_nss(struct ath11k_base *ab,
+ struct hal_rx_desc *desc)
{
return hweight8(ab->hw_params.hw_ops->rx_desc_get_msdu_nss(desc));
}
-static u8 ath11k_dp_rx_h_mpdu_start_tid(struct ath11k_base *ab,
- struct hal_rx_desc *desc)
+static inline u8 ath11k_dp_rx_h_mpdu_start_tid(struct ath11k_base *ab,
+ struct hal_rx_desc *desc)
{
return ab->hw_params.hw_ops->rx_desc_get_mpdu_tid(desc);
}
-static u16 ath11k_dp_rx_h_mpdu_start_peer_id(struct ath11k_base *ab,
- struct hal_rx_desc *desc)
+static inline u16 ath11k_dp_rx_h_mpdu_start_peer_id(struct ath11k_base *ab,
+ struct hal_rx_desc *desc)
{
return ab->hw_params.hw_ops->rx_desc_get_mpdu_peer_id(desc);
}
-static u8 ath11k_dp_rx_h_msdu_end_l3pad(struct ath11k_base *ab,
- struct hal_rx_desc *desc)
+static inline u8 ath11k_dp_rx_h_msdu_end_l3pad(struct ath11k_base *ab,
+ struct hal_rx_desc *desc)
{
return ab->hw_params.hw_ops->rx_desc_get_l3_pad_bytes(desc);
}
-static bool ath11k_dp_rx_h_msdu_end_first_msdu(struct ath11k_base *ab,
- struct hal_rx_desc *desc)
+static inline bool ath11k_dp_rx_h_msdu_end_first_msdu(struct ath11k_base *ab,
+ struct hal_rx_desc *desc)
{
return ab->hw_params.hw_ops->rx_desc_get_first_msdu(desc);
}
@@ -233,14 +237,14 @@ static void ath11k_dp_rx_desc_end_tlv_copy(struct ath11k_base *ab,
ab->hw_params.hw_ops->rx_desc_copy_attn_end_tlv(fdesc, ldesc);
}
-static u32 ath11k_dp_rxdesc_get_mpdulen_err(struct rx_attention *attn)
+static inline u32 ath11k_dp_rxdesc_get_mpdulen_err(struct rx_attention *attn)
{
return FIELD_GET(RX_ATTENTION_INFO1_MPDU_LEN_ERR,
__le32_to_cpu(attn->info1));
}
-static u8 *ath11k_dp_rxdesc_get_80211hdr(struct ath11k_base *ab,
- struct hal_rx_desc *rx_desc)
+static inline u8 *ath11k_dp_rxdesc_get_80211hdr(struct ath11k_base *ab,
+ struct hal_rx_desc *rx_desc)
{
u8 *rx_pkt_hdr;
@@ -249,8 +253,8 @@ static u8 *ath11k_dp_rxdesc_get_80211hdr(struct ath11k_base *ab,
return rx_pkt_hdr;
}
-static bool ath11k_dp_rxdesc_mpdu_valid(struct ath11k_base *ab,
- struct hal_rx_desc *rx_desc)
+static inline bool ath11k_dp_rxdesc_mpdu_valid(struct ath11k_base *ab,
+ struct hal_rx_desc *rx_desc)
{
u32 tlv_tag;
@@ -259,15 +263,15 @@ static bool ath11k_dp_rxdesc_mpdu_valid(struct ath11k_base *ab,
return tlv_tag == HAL_RX_MPDU_START;
}
-static u32 ath11k_dp_rxdesc_get_ppduid(struct ath11k_base *ab,
- struct hal_rx_desc *rx_desc)
+static inline u32 ath11k_dp_rxdesc_get_ppduid(struct ath11k_base *ab,
+ struct hal_rx_desc *rx_desc)
{
return ab->hw_params.hw_ops->rx_desc_get_mpdu_ppdu_id(rx_desc);
}
-static void ath11k_dp_rxdesc_set_msdu_len(struct ath11k_base *ab,
- struct hal_rx_desc *desc,
- u16 len)
+static inline void ath11k_dp_rxdesc_set_msdu_len(struct ath11k_base *ab,
+ struct hal_rx_desc *desc,
+ u16 len)
{
ab->hw_params.hw_ops->rx_desc_set_msdu_len(desc, len);
}
@@ -1356,25 +1360,6 @@ int ath11k_dp_htt_tlv_iter(struct ath11k_base *ab, const void *ptr, size_t len,
return 0;
}
-static inline u32 ath11k_he_gi_to_nl80211_he_gi(u8 sgi)
-{
- u32 ret = 0;
-
- switch (sgi) {
- case RX_MSDU_START_SGI_0_8_US:
- ret = NL80211_RATE_INFO_HE_GI_0_8;
- break;
- case RX_MSDU_START_SGI_1_6_US:
- ret = NL80211_RATE_INFO_HE_GI_1_6;
- break;
- case RX_MSDU_START_SGI_3_2_US:
- ret = NL80211_RATE_INFO_HE_GI_3_2;
- break;
- }
-
- return ret;
-}
-
static void
ath11k_update_per_peer_tx_stats(struct ath11k *ar,
struct htt_ppdu_stats *ppdu_stats, u8 user)
@@ -1493,14 +1478,15 @@ ath11k_update_per_peer_tx_stats(struct ath11k *ar,
arsta->txrate.mcs = mcs;
arsta->txrate.flags = RATE_INFO_FLAGS_HE_MCS;
arsta->txrate.he_dcm = dcm;
- arsta->txrate.he_gi = ath11k_he_gi_to_nl80211_he_gi(sgi);
- arsta->txrate.he_ru_alloc = ath11k_he_ru_tones_to_nl80211_he_ru_alloc(
- (user_rate->ru_end -
+ arsta->txrate.he_gi = ath11k_mac_he_gi_to_nl80211_he_gi(sgi);
+ arsta->txrate.he_ru_alloc = ath11k_mac_phy_he_ru_to_nl80211_he_ru_alloc
+ ((user_rate->ru_end -
user_rate->ru_start) + 1);
break;
}
arsta->txrate.nss = nss;
+
arsta->txrate.bw = ath11k_mac_bw_to_mac80211_bw(bw);
arsta->tx_duration += tx_duration;
memcpy(&arsta->last_txrate, &arsta->txrate, sizeof(struct rate_info));
@@ -2380,7 +2366,7 @@ static void ath11k_dp_rx_h_rate(struct ath11k *ar, struct hal_rx_desc *rx_desc,
}
rx_status->encoding = RX_ENC_HE;
rx_status->nss = nss;
- rx_status->he_gi = ath11k_he_gi_to_nl80211_he_gi(sgi);
+ rx_status->he_gi = ath11k_mac_he_gi_to_nl80211_he_gi(sgi);
rx_status->bw = ath11k_mac_bw_to_mac80211_bw(bw);
break;
}
@@ -2596,36 +2582,30 @@ static int ath11k_dp_rx_process_msdu(struct ath11k *ar,
static void ath11k_dp_rx_process_received_packets(struct ath11k_base *ab,
struct napi_struct *napi,
struct sk_buff_head *msdu_list,
- int *quota, int ring_id)
+ int mac_id)
{
- struct ath11k_skb_rxcb *rxcb;
struct sk_buff *msdu;
struct ath11k *ar;
struct ieee80211_rx_status rx_status = {0};
- u8 mac_id;
int ret;
if (skb_queue_empty(msdu_list))
return;
- rcu_read_lock();
+ if (unlikely(!rcu_access_pointer(ab->pdevs_active[mac_id]))) {
+ __skb_queue_purge(msdu_list);
+ return;
+ }
- while (*quota && (msdu = __skb_dequeue(msdu_list))) {
- rxcb = ATH11K_SKB_RXCB(msdu);
- mac_id = rxcb->mac_id;
- ar = ab->pdevs[mac_id].ar;
- if (!rcu_dereference(ab->pdevs_active[mac_id])) {
- dev_kfree_skb_any(msdu);
- continue;
- }
+ ar = ab->pdevs[mac_id].ar;
+ if (unlikely(test_bit(ATH11K_CAC_RUNNING, &ar->dev_flags))) {
+ __skb_queue_purge(msdu_list);
+ return;
+ }
- if (test_bit(ATH11K_CAC_RUNNING, &ar->dev_flags)) {
- dev_kfree_skb_any(msdu);
- continue;
- }
-
+ while ((msdu = __skb_dequeue(msdu_list))) {
ret = ath11k_dp_rx_process_msdu(ar, msdu, msdu_list, &rx_status);
- if (ret) {
+ if (unlikely(ret)) {
ath11k_dbg(ab, ATH11K_DBG_DATA,
"Unable to process msdu %d", ret);
dev_kfree_skb_any(msdu);
@@ -2633,10 +2613,7 @@ static void ath11k_dp_rx_process_received_packets(struct ath11k_base *ab,
}
ath11k_dp_rx_deliver_msdu(ar, napi, msdu, &rx_status);
- (*quota)--;
}
-
- rcu_read_unlock();
}
int ath11k_dp_process_rx(struct ath11k_base *ab, int ring_id,
@@ -2645,19 +2622,21 @@ int ath11k_dp_process_rx(struct ath11k_base *ab, int ring_id,
struct ath11k_dp *dp = &ab->dp;
struct dp_rxdma_ring *rx_ring;
int num_buffs_reaped[MAX_RADIOS] = {0};
- struct sk_buff_head msdu_list;
+ struct sk_buff_head msdu_list[MAX_RADIOS];
struct ath11k_skb_rxcb *rxcb;
int total_msdu_reaped = 0;
struct hal_srng *srng;
struct sk_buff *msdu;
- int quota = budget;
bool done = false;
int buf_id, mac_id;
struct ath11k *ar;
- u32 *rx_desc;
+ struct hal_reo_dest_ring *desc;
+ enum hal_reo_dest_ring_push_reason push_reason;
+ u32 cookie;
int i;
- __skb_queue_head_init(&msdu_list);
+ for (i = 0; i < MAX_RADIOS; i++)
+ __skb_queue_head_init(&msdu_list[i]);
srng = &ab->hal.srng_list[dp->reo_dst_ring[ring_id].ring_id];
@@ -2666,13 +2645,11 @@ int ath11k_dp_process_rx(struct ath11k_base *ab, int ring_id,
ath11k_hal_srng_access_begin(ab, srng);
try_again:
- while ((rx_desc = ath11k_hal_srng_dst_get_next_entry(ab, srng))) {
- struct hal_reo_dest_ring desc = *(struct hal_reo_dest_ring *)rx_desc;
- enum hal_reo_dest_ring_push_reason push_reason;
- u32 cookie;
-
+ while (likely(desc =
+ (struct hal_reo_dest_ring *)ath11k_hal_srng_dst_get_next_entry(ab,
+ srng))) {
cookie = FIELD_GET(BUFFER_ADDR_INFO1_SW_COOKIE,
- desc.buf_addr_info.info1);
+ desc->buf_addr_info.info1);
buf_id = FIELD_GET(DP_RXDMA_BUF_COOKIE_BUF_ID,
cookie);
mac_id = FIELD_GET(DP_RXDMA_BUF_COOKIE_PDEV_ID, cookie);
@@ -2681,7 +2658,7 @@ int ath11k_dp_process_rx(struct ath11k_base *ab, int ring_id,
rx_ring = &ar->dp.rx_refill_buf_ring;
spin_lock_bh(&rx_ring->idr_lock);
msdu = idr_find(&rx_ring->bufs_idr, buf_id);
- if (!msdu) {
+ if (unlikely(!msdu)) {
ath11k_warn(ab, "frame rx with invalid buf_id %d\n",
buf_id);
spin_unlock_bh(&rx_ring->idr_lock);
@@ -2697,37 +2674,41 @@ int ath11k_dp_process_rx(struct ath11k_base *ab, int ring_id,
DMA_FROM_DEVICE);
num_buffs_reaped[mac_id]++;
- total_msdu_reaped++;
push_reason = FIELD_GET(HAL_REO_DEST_RING_INFO0_PUSH_REASON,
- desc.info0);
- if (push_reason !=
- HAL_REO_DEST_RING_PUSH_REASON_ROUTING_INSTRUCTION) {
+ desc->info0);
+ if (unlikely(push_reason !=
+ HAL_REO_DEST_RING_PUSH_REASON_ROUTING_INSTRUCTION)) {
dev_kfree_skb_any(msdu);
ab->soc_stats.hal_reo_error[dp->reo_dst_ring[ring_id].ring_id]++;
continue;
}
- rxcb->is_first_msdu = !!(desc.rx_msdu_info.info0 &
+ rxcb->is_first_msdu = !!(desc->rx_msdu_info.info0 &
RX_MSDU_DESC_INFO0_FIRST_MSDU_IN_MPDU);
- rxcb->is_last_msdu = !!(desc.rx_msdu_info.info0 &
+ rxcb->is_last_msdu = !!(desc->rx_msdu_info.info0 &
RX_MSDU_DESC_INFO0_LAST_MSDU_IN_MPDU);
- rxcb->is_continuation = !!(desc.rx_msdu_info.info0 &
+ rxcb->is_continuation = !!(desc->rx_msdu_info.info0 &
RX_MSDU_DESC_INFO0_MSDU_CONTINUATION);
rxcb->peer_id = FIELD_GET(RX_MPDU_DESC_META_DATA_PEER_ID,
- desc.rx_mpdu_info.meta_data);
+ desc->rx_mpdu_info.meta_data);
rxcb->seq_no = FIELD_GET(RX_MPDU_DESC_INFO0_SEQ_NUM,
- desc.rx_mpdu_info.info0);
+ desc->rx_mpdu_info.info0);
rxcb->tid = FIELD_GET(HAL_REO_DEST_RING_INFO0_RX_QUEUE_NUM,
- desc.info0);
+ desc->info0);
rxcb->mac_id = mac_id;
- __skb_queue_tail(&msdu_list, msdu);
+ __skb_queue_tail(&msdu_list[mac_id], msdu);
- if (total_msdu_reaped >= quota && !rxcb->is_continuation) {
+ if (rxcb->is_continuation) {
+ done = false;
+ } else {
+ total_msdu_reaped++;
done = true;
- break;
}
+
+ if (total_msdu_reaped >= budget)
+ break;
}
/* Hw might have updated the head pointer after we cached it.
@@ -2736,7 +2717,7 @@ int ath11k_dp_process_rx(struct ath11k_base *ab, int ring_id,
* head pointer so that we can reap complete MPDU in the current
* rx processing.
*/
- if (!done && ath11k_hal_srng_dst_num_free(ab, srng, true)) {
+ if (unlikely(!done && ath11k_hal_srng_dst_num_free(ab, srng, true))) {
ath11k_hal_srng_access_end(ab, srng);
goto try_again;
}
@@ -2745,25 +2726,23 @@ int ath11k_dp_process_rx(struct ath11k_base *ab, int ring_id,
spin_unlock_bh(&srng->lock);
- if (!total_msdu_reaped)
+ if (unlikely(!total_msdu_reaped))
goto exit;
for (i = 0; i < ab->num_radios; i++) {
if (!num_buffs_reaped[i])
continue;
+ ath11k_dp_rx_process_received_packets(ab, napi, &msdu_list[i], i);
+
ar = ab->pdevs[i].ar;
rx_ring = &ar->dp.rx_refill_buf_ring;
ath11k_dp_rxbufs_replenish(ab, i, rx_ring, num_buffs_reaped[i],
ab->hw_params.hal_params->rx_buf_rbm);
}
-
- ath11k_dp_rx_process_received_packets(ab, napi, &msdu_list,
- "a, ring_id);
-
exit:
- return budget - quota;
+ return total_msdu_reaped;
}
static void ath11k_dp_rx_update_peer_stats(struct ath11k_sta *arsta,
@@ -2771,6 +2750,7 @@ static void ath11k_dp_rx_update_peer_stats(struct ath11k_sta *arsta,
{
struct ath11k_rx_peer_stats *rx_stats = arsta->rx_stats;
u32 num_msdu;
+ int i;
if (!rx_stats)
return;
@@ -2832,6 +2812,13 @@ static void ath11k_dp_rx_update_peer_stats(struct ath11k_sta *arsta,
rx_stats->ru_alloc_cnt[ppdu_info->ru_alloc] += num_msdu;
arsta->rssi_comb = ppdu_info->rssi_comb;
+
+ BUILD_BUG_ON(ARRAY_SIZE(arsta->chain_signal) >
+ ARRAY_SIZE(ppdu_info->rssi_chain_pri20));
+
+ for (i = 0; i < ARRAY_SIZE(arsta->chain_signal); i++)
+ arsta->chain_signal[i] = ppdu_info->rssi_chain_pri20[i];
+
rx_stats->rx_duration += ppdu_info->rx_duration;
arsta->rx_duration = rx_stats->rx_duration;
}
@@ -2945,6 +2932,43 @@ int ath11k_dp_rx_mon_status_bufs_replenish(struct ath11k_base *ab, int mac_id,
return req_entries - num_remain;
}
+#define ATH11K_DP_RX_FULL_MON_PPDU_ID_WRAP 32535
+
+static void
+ath11k_dp_rx_mon_update_status_buf_state(struct ath11k_mon_data *pmon,
+ struct hal_tlv_hdr *tlv)
+{
+ struct hal_rx_ppdu_start *ppdu_start;
+ u16 ppdu_id_diff, ppdu_id, tlv_len;
+ u8 *ptr;
+
+ /* PPDU id is part of second tlv, move ptr to second tlv */
+ tlv_len = FIELD_GET(HAL_TLV_HDR_LEN, tlv->tl);
+ ptr = (u8 *)tlv;
+ ptr += sizeof(*tlv) + tlv_len;
+ tlv = (struct hal_tlv_hdr *)ptr;
+
+ if (FIELD_GET(HAL_TLV_HDR_TAG, tlv->tl) != HAL_RX_PPDU_START)
+ return;
+
+ ptr += sizeof(*tlv);
+ ppdu_start = (struct hal_rx_ppdu_start *)ptr;
+ ppdu_id = FIELD_GET(HAL_RX_PPDU_START_INFO0_PPDU_ID,
+ __le32_to_cpu(ppdu_start->info0));
+
+ if (pmon->sw_mon_entries.ppdu_id < ppdu_id) {
+ pmon->buf_state = DP_MON_STATUS_LEAD;
+ ppdu_id_diff = ppdu_id - pmon->sw_mon_entries.ppdu_id;
+ if (ppdu_id_diff > ATH11K_DP_RX_FULL_MON_PPDU_ID_WRAP)
+ pmon->buf_state = DP_MON_STATUS_LAG;
+ } else if (pmon->sw_mon_entries.ppdu_id > ppdu_id) {
+ pmon->buf_state = DP_MON_STATUS_LAG;
+ ppdu_id_diff = pmon->sw_mon_entries.ppdu_id - ppdu_id;
+ if (ppdu_id_diff > ATH11K_DP_RX_FULL_MON_PPDU_ID_WRAP)
+ pmon->buf_state = DP_MON_STATUS_LEAD;
+ }
+}
+
static int ath11k_dp_rx_reap_mon_status_ring(struct ath11k_base *ab, int mac_id,
int *budget, struct sk_buff_head *skb_list)
{
@@ -2952,6 +2976,7 @@ static int ath11k_dp_rx_reap_mon_status_ring(struct ath11k_base *ab, int mac_id,
const struct ath11k_hw_hal_params *hal_params;
struct ath11k_pdev_dp *dp;
struct dp_rxdma_ring *rx_ring;
+ struct ath11k_mon_data *pmon;
struct hal_srng *srng;
void *rx_mon_status_desc;
struct sk_buff *skb;
@@ -2965,6 +2990,7 @@ static int ath11k_dp_rx_reap_mon_status_ring(struct ath11k_base *ab, int mac_id,
ar = ab->pdevs[ath11k_hw_mac_id_to_pdev_id(&ab->hw_params, mac_id)].ar;
dp = &ar->dp;
+ pmon = &dp->mon_data;
srng_id = ath11k_hw_mac_id_to_srng_id(&ab->hw_params, mac_id);
rx_ring = &dp->rx_mon_status_refill_ring[srng_id];
@@ -2977,8 +3003,10 @@ static int ath11k_dp_rx_reap_mon_status_ring(struct ath11k_base *ab, int mac_id,
*budget -= 1;
rx_mon_status_desc =
ath11k_hal_srng_src_peek(ab, srng);
- if (!rx_mon_status_desc)
+ if (!rx_mon_status_desc) {
+ pmon->buf_state = DP_MON_STATUS_REPLINISH;
break;
+ }
ath11k_hal_rx_buf_addr_info_get(rx_mon_status_desc, &paddr,
&cookie, &rbm);
@@ -2991,6 +3019,7 @@ static int ath11k_dp_rx_reap_mon_status_ring(struct ath11k_base *ab, int mac_id,
ath11k_warn(ab, "rx monitor status with invalid buf_id %d\n",
buf_id);
spin_unlock_bh(&rx_ring->idr_lock);
+ pmon->buf_state = DP_MON_STATUS_REPLINISH;
goto move_next;
}
@@ -3010,10 +3039,18 @@ static int ath11k_dp_rx_reap_mon_status_ring(struct ath11k_base *ab, int mac_id,
FIELD_GET(HAL_TLV_HDR_TAG,
tlv->tl));
dev_kfree_skb_any(skb);
+ pmon->buf_state = DP_MON_STATUS_NO_DMA;
goto move_next;
}
+ if (ab->hw_params.full_monitor_mode) {
+ ath11k_dp_rx_mon_update_status_buf_state(pmon, tlv);
+ if (paddr == pmon->mon_status_paddr)
+ pmon->buf_state = DP_MON_STATUS_MATCH;
+ }
__skb_queue_tail(skb_list, skb);
+ } else {
+ pmon->buf_state = DP_MON_STATUS_REPLINISH;
}
move_next:
skb = ath11k_dp_rx_alloc_mon_status_buf(ab, rx_ring,
@@ -3064,10 +3101,10 @@ int ath11k_dp_rx_process_mon_status(struct ath11k_base *ab, int mac_id,
if (!num_buffs_reaped)
goto exit;
- while ((skb = __skb_dequeue(&skb_list))) {
- memset(&ppdu_info, 0, sizeof(ppdu_info));
- ppdu_info.peer_id = HAL_INVALID_PEERID;
+ memset(&ppdu_info, 0, sizeof(ppdu_info));
+ ppdu_info.peer_id = HAL_INVALID_PEERID;
+ while ((skb = __skb_dequeue(&skb_list))) {
if (ath11k_debugfs_is_pktlog_lite_mode_enabled(ar)) {
log_type = ATH11K_PKTLOG_TYPE_LITE_RX;
rx_buf_sz = DP_RX_BUFFER_SIZE_LITE;
@@ -3095,10 +3132,7 @@ int ath11k_dp_rx_process_mon_status(struct ath11k_base *ab, int mac_id,
ath11k_dbg(ab, ATH11K_DBG_DATA,
"failed to find the peer with peer_id %d\n",
ppdu_info.peer_id);
- spin_unlock_bh(&ab->base_lock);
- rcu_read_unlock();
- dev_kfree_skb_any(skb);
- continue;
+ goto next_skb;
}
arsta = (struct ath11k_sta *)peer->sta->drv_priv;
@@ -3107,10 +3141,13 @@ int ath11k_dp_rx_process_mon_status(struct ath11k_base *ab, int mac_id,
if (ath11k_debugfs_is_pktlog_peer_valid(ar, peer->addr))
trace_ath11k_htt_rxdesc(ar, skb->data, log_type, rx_buf_sz);
+next_skb:
spin_unlock_bh(&ab->base_lock);
rcu_read_unlock();
dev_kfree_skb_any(skb);
+ memset(&ppdu_info, 0, sizeof(ppdu_info));
+ ppdu_info.peer_id = HAL_INVALID_PEERID;
}
exit:
return num_buffs_reaped;
@@ -3800,7 +3837,7 @@ int ath11k_dp_process_rx_err(struct ath11k_base *ab, struct napi_struct *napi,
ath11k_hal_rx_msdu_link_info_get(link_desc_va, &num_msdus, msdu_cookies,
&rbm);
if (rbm != HAL_RX_BUF_RBM_WBM_IDLE_DESC_LIST &&
- rbm != ab->hw_params.hal_params->rx_buf_rbm) {
+ rbm != HAL_RX_BUF_RBM_SW3_BM) {
ab->soc_stats.invalid_rbm++;
ath11k_warn(ab, "invalid return buffer manager %d\n", rbm);
ath11k_dp_rx_link_desc_return(ab, desc,
@@ -4829,7 +4866,7 @@ static struct sk_buff *
ath11k_dp_rx_mon_merg_msdus(struct ath11k *ar,
u32 mac_id, struct sk_buff *head_msdu,
struct sk_buff *last_msdu,
- struct ieee80211_rx_status *rxs)
+ struct ieee80211_rx_status *rxs, bool *fcs_err)
{
struct ath11k_base *ab = ar->ab;
struct sk_buff *msdu, *prev_buf;
@@ -4839,12 +4876,17 @@ ath11k_dp_rx_mon_merg_msdus(struct ath11k *ar,
u8 *dest, decap_format;
struct ieee80211_hdr_3addr *wh;
struct rx_attention *rx_attention;
+ u32 err_bitmap;
if (!head_msdu)
goto err_merge_fail;
rx_desc = (struct hal_rx_desc *)head_msdu->data;
rx_attention = ath11k_dp_rx_get_attention(ab, rx_desc);
+ err_bitmap = ath11k_dp_rx_h_attn_mpdu_err(rx_attention);
+
+ if (err_bitmap & DP_RX_MPDU_ERR_FCS)
+ *fcs_err = true;
if (ath11k_dp_rxdesc_get_mpdulen_err(rx_attention))
return NULL;
@@ -4933,9 +4975,10 @@ static int ath11k_dp_rx_mon_deliver(struct ath11k *ar, u32 mac_id,
struct ath11k_pdev_dp *dp = &ar->dp;
struct sk_buff *mon_skb, *skb_next, *header;
struct ieee80211_rx_status *rxs = &dp->rx_status;
+ bool fcs_err = false;
mon_skb = ath11k_dp_rx_mon_merg_msdus(ar, mac_id, head_msdu,
- tail_msdu, rxs);
+ tail_msdu, rxs, &fcs_err);
if (!mon_skb)
goto mon_deliver_fail;
@@ -4943,6 +4986,10 @@ static int ath11k_dp_rx_mon_deliver(struct ath11k *ar, u32 mac_id,
header = mon_skb;
rxs->flag = 0;
+
+ if (fcs_err)
+ rxs->flag = RX_FLAG_FAILED_FCS_CRC;
+
do {
skb_next = mon_skb->next;
if (!skb_next)
@@ -5091,6 +5138,357 @@ static void ath11k_dp_rx_mon_status_process_tlv(struct ath11k *ar,
}
}
+static u32
+ath11k_dp_rx_full_mon_mpdu_pop(struct ath11k *ar,
+ void *ring_entry, struct sk_buff **head_msdu,
+ struct sk_buff **tail_msdu,
+ struct hal_sw_mon_ring_entries *sw_mon_entries)
+{
+ struct ath11k_pdev_dp *dp = &ar->dp;
+ struct ath11k_mon_data *pmon = &dp->mon_data;
+ struct dp_rxdma_ring *rx_ring = &dp->rxdma_mon_buf_ring;
+ struct sk_buff *msdu = NULL, *last = NULL;
+ struct hal_sw_monitor_ring *sw_desc = ring_entry;
+ struct hal_rx_msdu_list msdu_list;
+ struct hal_rx_desc *rx_desc;
+ struct ath11k_skb_rxcb *rxcb;
+ void *rx_msdu_link_desc;
+ void *p_buf_addr_info, *p_last_buf_addr_info;
+ int buf_id, i = 0;
+ u32 rx_buf_size, rx_pkt_offset, l2_hdr_offset;
+ u32 rx_bufs_used = 0, msdu_cnt = 0;
+ u32 total_len = 0, frag_len = 0, sw_cookie;
+ u16 num_msdus = 0;
+ u8 rxdma_err, rbm;
+ bool is_frag, is_first_msdu;
+ bool drop_mpdu = false;
+
+ ath11k_hal_rx_sw_mon_ring_buf_paddr_get(ring_entry, sw_mon_entries);
+
+ sw_cookie = sw_mon_entries->mon_dst_sw_cookie;
+ sw_mon_entries->end_of_ppdu = false;
+ sw_mon_entries->drop_ppdu = false;
+ p_last_buf_addr_info = sw_mon_entries->dst_buf_addr_info;
+ msdu_cnt = sw_mon_entries->msdu_cnt;
+
+ sw_mon_entries->end_of_ppdu =
+ FIELD_GET(HAL_SW_MON_RING_INFO0_END_OF_PPDU, sw_desc->info0);
+ if (sw_mon_entries->end_of_ppdu)
+ return rx_bufs_used;
+
+ if (FIELD_GET(HAL_SW_MON_RING_INFO0_RXDMA_PUSH_REASON,
+ sw_desc->info0) ==
+ HAL_REO_DEST_RING_PUSH_REASON_ERR_DETECTED) {
+ rxdma_err =
+ FIELD_GET(HAL_SW_MON_RING_INFO0_RXDMA_ERROR_CODE,
+ sw_desc->info0);
+ if (rxdma_err == HAL_REO_ENTR_RING_RXDMA_ECODE_FLUSH_REQUEST_ERR ||
+ rxdma_err == HAL_REO_ENTR_RING_RXDMA_ECODE_MPDU_LEN_ERR ||
+ rxdma_err == HAL_REO_ENTR_RING_RXDMA_ECODE_OVERFLOW_ERR) {
+ pmon->rx_mon_stats.dest_mpdu_drop++;
+ drop_mpdu = true;
+ }
+ }
+
+ is_frag = false;
+ is_first_msdu = true;
+
+ do {
+ rx_msdu_link_desc =
+ (u8 *)pmon->link_desc_banks[sw_cookie].vaddr +
+ (sw_mon_entries->mon_dst_paddr -
+ pmon->link_desc_banks[sw_cookie].paddr);
+
+ ath11k_hal_rx_msdu_list_get(ar, rx_msdu_link_desc, &msdu_list,
+ &num_msdus);
+
+ for (i = 0; i < num_msdus; i++) {
+ buf_id = FIELD_GET(DP_RXDMA_BUF_COOKIE_BUF_ID,
+ msdu_list.sw_cookie[i]);
+
+ spin_lock_bh(&rx_ring->idr_lock);
+ msdu = idr_find(&rx_ring->bufs_idr, buf_id);
+ if (!msdu) {
+ ath11k_dbg(ar->ab, ATH11K_DBG_DATA,
+ "full mon msdu_pop: invalid buf_id %d\n",
+ buf_id);
+ spin_unlock_bh(&rx_ring->idr_lock);
+ break;
+ }
+ idr_remove(&rx_ring->bufs_idr, buf_id);
+ spin_unlock_bh(&rx_ring->idr_lock);
+
+ rxcb = ATH11K_SKB_RXCB(msdu);
+ if (!rxcb->unmapped) {
+ dma_unmap_single(ar->ab->dev, rxcb->paddr,
+ msdu->len +
+ skb_tailroom(msdu),
+ DMA_FROM_DEVICE);
+ rxcb->unmapped = 1;
+ }
+ if (drop_mpdu) {
+ ath11k_dbg(ar->ab, ATH11K_DBG_DATA,
+ "full mon: i %d drop msdu %p *ppdu_id %x\n",
+ i, msdu, sw_mon_entries->ppdu_id);
+ dev_kfree_skb_any(msdu);
+ msdu_cnt--;
+ goto next_msdu;
+ }
+
+ rx_desc = (struct hal_rx_desc *)msdu->data;
+
+ rx_pkt_offset = sizeof(struct hal_rx_desc);
+ l2_hdr_offset = ath11k_dp_rx_h_msdu_end_l3pad(ar->ab, rx_desc);
+
+ if (is_first_msdu) {
+ if (!ath11k_dp_rxdesc_mpdu_valid(ar->ab, rx_desc)) {
+ drop_mpdu = true;
+ dev_kfree_skb_any(msdu);
+ msdu = NULL;
+ goto next_msdu;
+ }
+ is_first_msdu = false;
+ }
+
+ ath11k_dp_mon_get_buf_len(&msdu_list.msdu_info[i],
+ &is_frag, &total_len,
+ &frag_len, &msdu_cnt);
+
+ rx_buf_size = rx_pkt_offset + l2_hdr_offset + frag_len;
+
+ ath11k_dp_pkt_set_pktlen(msdu, rx_buf_size);
+
+ if (!(*head_msdu))
+ *head_msdu = msdu;
+ else if (last)
+ last->next = msdu;
+
+ last = msdu;
+next_msdu:
+ rx_bufs_used++;
+ }
+
+ ath11k_dp_rx_mon_next_link_desc_get(rx_msdu_link_desc,
+ &sw_mon_entries->mon_dst_paddr,
+ &sw_mon_entries->mon_dst_sw_cookie,
+ &rbm,
+ &p_buf_addr_info);
+
+ if (ath11k_dp_rx_monitor_link_desc_return(ar,
+ p_last_buf_addr_info,
+ dp->mac_id))
+ ath11k_dbg(ar->ab, ATH11K_DBG_DATA,
+ "full mon: dp_rx_monitor_link_desc_return failed\n");
+
+ p_last_buf_addr_info = p_buf_addr_info;
+
+ } while (sw_mon_entries->mon_dst_paddr && msdu_cnt);
+
+ if (last)
+ last->next = NULL;
+
+ *tail_msdu = msdu;
+
+ return rx_bufs_used;
+}
+
+static int ath11k_dp_rx_full_mon_prepare_mpdu(struct ath11k_dp *dp,
+ struct dp_full_mon_mpdu *mon_mpdu,
+ struct sk_buff *head,
+ struct sk_buff *tail)
+{
+ mon_mpdu = kzalloc(sizeof(*mon_mpdu), GFP_ATOMIC);
+ if (!mon_mpdu)
+ return -ENOMEM;
+
+ list_add_tail(&mon_mpdu->list, &dp->dp_full_mon_mpdu_list);
+ mon_mpdu->head = head;
+ mon_mpdu->tail = tail;
+
+ return 0;
+}
+
+static void ath11k_dp_rx_full_mon_drop_ppdu(struct ath11k_dp *dp,
+ struct dp_full_mon_mpdu *mon_mpdu)
+{
+ struct dp_full_mon_mpdu *tmp;
+ struct sk_buff *tmp_msdu, *skb_next;
+
+ if (list_empty(&dp->dp_full_mon_mpdu_list))
+ return;
+
+ list_for_each_entry_safe(mon_mpdu, tmp, &dp->dp_full_mon_mpdu_list, list) {
+ list_del(&mon_mpdu->list);
+
+ tmp_msdu = mon_mpdu->head;
+ while (tmp_msdu) {
+ skb_next = tmp_msdu->next;
+ dev_kfree_skb_any(tmp_msdu);
+ tmp_msdu = skb_next;
+ }
+
+ kfree(mon_mpdu);
+ }
+}
+
+static int ath11k_dp_rx_full_mon_deliver_ppdu(struct ath11k *ar,
+ int mac_id,
+ struct ath11k_mon_data *pmon,
+ struct napi_struct *napi)
+{
+ struct ath11k_pdev_mon_stats *rx_mon_stats;
+ struct dp_full_mon_mpdu *tmp;
+ struct dp_full_mon_mpdu *mon_mpdu = pmon->mon_mpdu;
+ struct sk_buff *head_msdu, *tail_msdu;
+ struct ath11k_base *ab = ar->ab;
+ struct ath11k_dp *dp = &ab->dp;
+ int ret;
+
+ rx_mon_stats = &pmon->rx_mon_stats;
+
+ list_for_each_entry_safe(mon_mpdu, tmp, &dp->dp_full_mon_mpdu_list, list) {
+ list_del(&mon_mpdu->list);
+ head_msdu = mon_mpdu->head;
+ tail_msdu = mon_mpdu->tail;
+ if (head_msdu && tail_msdu) {
+ ret = ath11k_dp_rx_mon_deliver(ar, mac_id, head_msdu,
+ tail_msdu, napi);
+ rx_mon_stats->dest_mpdu_done++;
+ ath11k_dbg(ar->ab, ATH11K_DBG_DATA, "full mon: deliver ppdu\n");
+ }
+ kfree(mon_mpdu);
+ }
+
+ return ret;
+}
+
+static int
+ath11k_dp_rx_process_full_mon_status_ring(struct ath11k_base *ab, int mac_id,
+ struct napi_struct *napi, int budget)
+{
+ struct ath11k *ar = ab->pdevs[mac_id].ar;
+ struct ath11k_pdev_dp *dp = &ar->dp;
+ struct ath11k_mon_data *pmon = &dp->mon_data;
+ struct hal_sw_mon_ring_entries *sw_mon_entries;
+ int quota = 0, work = 0, count;
+
+ sw_mon_entries = &pmon->sw_mon_entries;
+
+ while (pmon->hold_mon_dst_ring) {
+ quota = ath11k_dp_rx_process_mon_status(ab, mac_id,
+ napi, 1);
+ if (pmon->buf_state == DP_MON_STATUS_MATCH) {
+ count = sw_mon_entries->status_buf_count;
+ if (count > 1) {
+ quota += ath11k_dp_rx_process_mon_status(ab, mac_id,
+ napi, count);
+ }
+
+ ath11k_dp_rx_full_mon_deliver_ppdu(ar, dp->mac_id,
+ pmon, napi);
+ pmon->hold_mon_dst_ring = false;
+ } else if (!pmon->mon_status_paddr ||
+ pmon->buf_state == DP_MON_STATUS_LEAD) {
+ sw_mon_entries->drop_ppdu = true;
+ pmon->hold_mon_dst_ring = false;
+ }
+
+ if (!quota)
+ break;
+
+ work += quota;
+ }
+
+ if (sw_mon_entries->drop_ppdu)
+ ath11k_dp_rx_full_mon_drop_ppdu(&ab->dp, pmon->mon_mpdu);
+
+ return work;
+}
+
+static int ath11k_dp_full_mon_process_rx(struct ath11k_base *ab, int mac_id,
+ struct napi_struct *napi, int budget)
+{
+ struct ath11k *ar = ab->pdevs[mac_id].ar;
+ struct ath11k_pdev_dp *dp = &ar->dp;
+ struct ath11k_mon_data *pmon = &dp->mon_data;
+ struct hal_sw_mon_ring_entries *sw_mon_entries;
+ struct ath11k_pdev_mon_stats *rx_mon_stats;
+ struct sk_buff *head_msdu, *tail_msdu;
+ void *mon_dst_srng = &ar->ab->hal.srng_list[dp->rxdma_mon_dst_ring.ring_id];
+ void *ring_entry;
+ u32 rx_bufs_used = 0, mpdu_rx_bufs_used;
+ int quota = 0, ret;
+ bool break_dst_ring = false;
+
+ spin_lock_bh(&pmon->mon_lock);
+
+ sw_mon_entries = &pmon->sw_mon_entries;
+ rx_mon_stats = &pmon->rx_mon_stats;
+
+ if (pmon->hold_mon_dst_ring) {
+ spin_unlock_bh(&pmon->mon_lock);
+ goto reap_status_ring;
+ }
+
+ ath11k_hal_srng_access_begin(ar->ab, mon_dst_srng);
+ while ((ring_entry = ath11k_hal_srng_dst_peek(ar->ab, mon_dst_srng))) {
+ head_msdu = NULL;
+ tail_msdu = NULL;
+
+ mpdu_rx_bufs_used = ath11k_dp_rx_full_mon_mpdu_pop(ar, ring_entry,
+ &head_msdu,
+ &tail_msdu,
+ sw_mon_entries);
+ rx_bufs_used += mpdu_rx_bufs_used;
+
+ if (!sw_mon_entries->end_of_ppdu) {
+ if (head_msdu) {
+ ret = ath11k_dp_rx_full_mon_prepare_mpdu(&ab->dp,
+ pmon->mon_mpdu,
+ head_msdu,
+ tail_msdu);
+ if (ret)
+ break_dst_ring = true;
+ }
+
+ goto next_entry;
+ } else {
+ if (!sw_mon_entries->ppdu_id &&
+ !sw_mon_entries->mon_status_paddr) {
+ break_dst_ring = true;
+ goto next_entry;
+ }
+ }
+
+ rx_mon_stats->dest_ppdu_done++;
+ pmon->mon_ppdu_status = DP_PPDU_STATUS_START;
+ pmon->buf_state = DP_MON_STATUS_LAG;
+ pmon->mon_status_paddr = sw_mon_entries->mon_status_paddr;
+ pmon->hold_mon_dst_ring = true;
+next_entry:
+ ring_entry = ath11k_hal_srng_dst_get_next_entry(ar->ab,
+ mon_dst_srng);
+ if (break_dst_ring)
+ break;
+ }
+
+ ath11k_hal_srng_access_end(ar->ab, mon_dst_srng);
+ spin_unlock_bh(&pmon->mon_lock);
+
+ if (rx_bufs_used) {
+ ath11k_dp_rxbufs_replenish(ar->ab, dp->mac_id,
+ &dp->rxdma_mon_buf_ring,
+ rx_bufs_used,
+ HAL_RX_BUF_RBM_SW3_BM);
+ }
+
+reap_status_ring:
+ quota = ath11k_dp_rx_process_full_mon_status_ring(ab, mac_id,
+ napi, budget);
+
+ return quota;
+}
+
static int ath11k_dp_mon_process_rx(struct ath11k_base *ab, int mac_id,
struct napi_struct *napi, int budget)
{
@@ -5113,10 +5511,14 @@ int ath11k_dp_rx_process_mon_rings(struct ath11k_base *ab, int mac_id,
struct ath11k *ar = ath11k_ab_to_ar(ab, mac_id);
int ret = 0;
- if (test_bit(ATH11K_FLAG_MONITOR_STARTED, &ar->monitor_flags))
+ if (test_bit(ATH11K_FLAG_MONITOR_STARTED, &ar->monitor_flags) &&
+ ab->hw_params.full_monitor_mode)
+ ret = ath11k_dp_full_mon_process_rx(ab, mac_id, napi, budget);
+ else if (test_bit(ATH11K_FLAG_MONITOR_STARTED, &ar->monitor_flags))
ret = ath11k_dp_mon_process_rx(ab, mac_id, napi, budget);
else
ret = ath11k_dp_rx_process_mon_status(ab, mac_id, napi, budget);
+
return ret;
}
diff --git a/drivers/net/wireless/ath/ath11k/dp_tx.c b/drivers/net/wireless/ath/ath11k/dp_tx.c
index 879fb2a..91d6244 100644
--- a/drivers/net/wireless/ath/ath11k/dp_tx.c
+++ b/drivers/net/wireless/ath/ath11k/dp_tx.c
@@ -9,6 +9,7 @@
#include "debugfs_sta.h"
#include "hw.h"
#include "peer.h"
+#include "mac.h"
static enum hal_tcl_encap_type
ath11k_dp_tx_get_encap_type(struct ath11k_vif *arvif, struct sk_buff *skb)
@@ -95,11 +96,11 @@ int ath11k_dp_tx(struct ath11k *ar, struct ath11k_vif *arvif,
u8 ring_selector = 0, ring_map = 0;
bool tcl_ring_retry;
- if (test_bit(ATH11K_FLAG_CRASH_FLUSH, &ar->ab->dev_flags))
+ if (unlikely(test_bit(ATH11K_FLAG_CRASH_FLUSH, &ar->ab->dev_flags)))
return -ESHUTDOWN;
- if (!(info->flags & IEEE80211_TX_CTL_HW_80211_ENCAP) &&
- !ieee80211_is_data(hdr->frame_control))
+ if (unlikely(!(info->flags & IEEE80211_TX_CTL_HW_80211_ENCAP) &&
+ !ieee80211_is_data(hdr->frame_control)))
return -ENOTSUPP;
pool_id = skb_get_queue_mapping(skb) & (ATH11K_HW_MAX_QUEUES - 1);
@@ -127,7 +128,7 @@ int ath11k_dp_tx(struct ath11k *ar, struct ath11k_vif *arvif,
DP_TX_IDR_SIZE - 1, GFP_ATOMIC);
spin_unlock_bh(&tx_ring->tx_idr_lock);
- if (ret < 0) {
+ if (unlikely(ret < 0)) {
if (ring_map == (BIT(ab->hw_params.max_tx_ring) - 1)) {
atomic_inc(&ab->soc_stats.tx_err.misc_fail);
return -ENOSPC;
@@ -152,7 +153,7 @@ int ath11k_dp_tx(struct ath11k *ar, struct ath11k_vif *arvif,
ti.meta_data_flags = arvif->tcl_metadata;
}
- if (ti.encap_type == HAL_TCL_ENCAP_TYPE_RAW) {
+ if (unlikely(ti.encap_type == HAL_TCL_ENCAP_TYPE_RAW)) {
if (skb_cb->flags & ATH11K_SKB_CIPHER_SET) {
ti.encrypt_type =
ath11k_dp_tx_get_encrypt_type(skb_cb->cipher);
@@ -173,8 +174,8 @@ int ath11k_dp_tx(struct ath11k *ar, struct ath11k_vif *arvif,
ti.bss_ast_idx = arvif->ast_idx;
ti.dscp_tid_tbl_idx = 0;
- if (skb->ip_summed == CHECKSUM_PARTIAL &&
- ti.encap_type != HAL_TCL_ENCAP_TYPE_RAW) {
+ if (likely(skb->ip_summed == CHECKSUM_PARTIAL &&
+ ti.encap_type != HAL_TCL_ENCAP_TYPE_RAW)) {
ti.flags0 |= FIELD_PREP(HAL_TCL_DATA_CMD_INFO1_IP4_CKSUM_EN, 1) |
FIELD_PREP(HAL_TCL_DATA_CMD_INFO1_UDP4_CKSUM_EN, 1) |
FIELD_PREP(HAL_TCL_DATA_CMD_INFO1_UDP6_CKSUM_EN, 1) |
@@ -211,7 +212,7 @@ int ath11k_dp_tx(struct ath11k *ar, struct ath11k_vif *arvif,
}
ti.paddr = dma_map_single(ab->dev, skb->data, skb->len, DMA_TO_DEVICE);
- if (dma_mapping_error(ab->dev, ti.paddr)) {
+ if (unlikely(dma_mapping_error(ab->dev, ti.paddr))) {
atomic_inc(&ab->soc_stats.tx_err.misc_fail);
ath11k_warn(ab, "failed to DMA map data Tx buffer\n");
ret = -ENOMEM;
@@ -231,7 +232,7 @@ int ath11k_dp_tx(struct ath11k *ar, struct ath11k_vif *arvif,
ath11k_hal_srng_access_begin(ab, tcl_ring);
hal_tcl_desc = (void *)ath11k_hal_srng_src_get_next_entry(ab, tcl_ring);
- if (!hal_tcl_desc) {
+ if (unlikely(!hal_tcl_desc)) {
/* NOTE: It is highly unlikely we'll be running out of tcl_ring
* desc because the desc is directly enqueued onto hw queue.
*/
@@ -245,7 +246,7 @@ int ath11k_dp_tx(struct ath11k *ar, struct ath11k_vif *arvif,
* checking this ring earlier for each pkt tx.
* Restart ring selection if some rings are not checked yet.
*/
- if (ring_map != (BIT(ab->hw_params.max_tx_ring) - 1) &&
+ if (unlikely(ring_map != (BIT(ab->hw_params.max_tx_ring)) - 1) &&
ab->hw_params.max_tx_ring > 1) {
tcl_ring_retry = true;
ring_selector++;
@@ -293,20 +294,18 @@ static void ath11k_dp_tx_free_txbuf(struct ath11k_base *ab, u8 mac_id,
struct sk_buff *msdu;
struct ath11k_skb_cb *skb_cb;
- spin_lock_bh(&tx_ring->tx_idr_lock);
- msdu = idr_find(&tx_ring->txbuf_idr, msdu_id);
- if (!msdu) {
+ spin_lock(&tx_ring->tx_idr_lock);
+ msdu = idr_remove(&tx_ring->txbuf_idr, msdu_id);
+ spin_unlock(&tx_ring->tx_idr_lock);
+
+ if (unlikely(!msdu)) {
ath11k_warn(ab, "tx completion for unknown msdu_id %d\n",
msdu_id);
- spin_unlock_bh(&tx_ring->tx_idr_lock);
return;
}
skb_cb = ATH11K_SKB_CB(msdu);
- idr_remove(&tx_ring->txbuf_idr, msdu_id);
- spin_unlock_bh(&tx_ring->tx_idr_lock);
-
dma_unmap_single(ab->dev, skb_cb->paddr, msdu->len, DMA_TO_DEVICE);
dev_kfree_skb_any(msdu);
@@ -325,12 +324,13 @@ ath11k_dp_tx_htt_tx_complete_buf(struct ath11k_base *ab,
struct ath11k_skb_cb *skb_cb;
struct ath11k *ar;
- spin_lock_bh(&tx_ring->tx_idr_lock);
- msdu = idr_find(&tx_ring->txbuf_idr, ts->msdu_id);
- if (!msdu) {
+ spin_lock(&tx_ring->tx_idr_lock);
+ msdu = idr_remove(&tx_ring->txbuf_idr, ts->msdu_id);
+ spin_unlock(&tx_ring->tx_idr_lock);
+
+ if (unlikely(!msdu)) {
ath11k_warn(ab, "htt tx completion for unknown msdu_id %d\n",
ts->msdu_id);
- spin_unlock_bh(&tx_ring->tx_idr_lock);
return;
}
@@ -339,9 +339,6 @@ ath11k_dp_tx_htt_tx_complete_buf(struct ath11k_base *ab,
ar = skb_cb->ar;
- idr_remove(&tx_ring->txbuf_idr, ts->msdu_id);
- spin_unlock_bh(&tx_ring->tx_idr_lock);
-
if (atomic_dec_and_test(&ar->dp.num_tx_pending))
wake_up(&ar->dp.tx_empty_waitq);
@@ -418,6 +415,105 @@ static void ath11k_dp_tx_cache_peer_stats(struct ath11k *ar,
}
}
+void ath11k_dp_tx_update_txcompl(struct ath11k *ar, struct hal_tx_status *ts)
+{
+ struct ath11k_base *ab = ar->ab;
+ struct ath11k_per_peer_tx_stats *peer_stats = &ar->cached_stats;
+ enum hal_tx_rate_stats_pkt_type pkt_type;
+ enum hal_tx_rate_stats_sgi sgi;
+ enum hal_tx_rate_stats_bw bw;
+ struct ath11k_peer *peer;
+ struct ath11k_sta *arsta;
+ struct ieee80211_sta *sta;
+ u16 rate, ru_tones;
+ u8 mcs, rate_idx, ofdma;
+ int ret;
+
+ spin_lock_bh(&ab->base_lock);
+ peer = ath11k_peer_find_by_id(ab, ts->peer_id);
+ if (!peer || !peer->sta) {
+ ath11k_dbg(ab, ATH11K_DBG_DP_TX,
+ "failed to find the peer by id %u\n", ts->peer_id);
+ goto err_out;
+ }
+
+ sta = peer->sta;
+ arsta = (struct ath11k_sta *)sta->drv_priv;
+
+ memset(&arsta->txrate, 0, sizeof(arsta->txrate));
+ pkt_type = FIELD_GET(HAL_TX_RATE_STATS_INFO0_PKT_TYPE,
+ ts->rate_stats);
+ mcs = FIELD_GET(HAL_TX_RATE_STATS_INFO0_MCS,
+ ts->rate_stats);
+ sgi = FIELD_GET(HAL_TX_RATE_STATS_INFO0_SGI,
+ ts->rate_stats);
+ bw = FIELD_GET(HAL_TX_RATE_STATS_INFO0_BW, ts->rate_stats);
+ ru_tones = FIELD_GET(HAL_TX_RATE_STATS_INFO0_TONES_IN_RU, ts->rate_stats);
+ ofdma = FIELD_GET(HAL_TX_RATE_STATS_INFO0_OFDMA_TX, ts->rate_stats);
+
+ /* This is to prefer choose the real NSS value arsta->last_txrate.nss,
+ * if it is invalid, then choose the NSS value while assoc.
+ */
+ if (arsta->last_txrate.nss)
+ arsta->txrate.nss = arsta->last_txrate.nss;
+ else
+ arsta->txrate.nss = arsta->peer_nss;
+
+ if (pkt_type == HAL_TX_RATE_STATS_PKT_TYPE_11A ||
+ pkt_type == HAL_TX_RATE_STATS_PKT_TYPE_11B) {
+ ret = ath11k_mac_hw_ratecode_to_legacy_rate(mcs,
+ pkt_type,
+ &rate_idx,
+ &rate);
+ if (ret < 0)
+ goto err_out;
+ arsta->txrate.legacy = rate;
+ } else if (pkt_type == HAL_TX_RATE_STATS_PKT_TYPE_11N) {
+ if (mcs > 7) {
+ ath11k_warn(ab, "Invalid HT mcs index %d\n", mcs);
+ goto err_out;
+ }
+
+ if (arsta->txrate.nss != 0)
+ arsta->txrate.mcs = mcs + 8 * (arsta->txrate.nss - 1);
+ arsta->txrate.flags = RATE_INFO_FLAGS_MCS;
+ if (sgi)
+ arsta->txrate.flags |= RATE_INFO_FLAGS_SHORT_GI;
+ } else if (pkt_type == HAL_TX_RATE_STATS_PKT_TYPE_11AC) {
+ if (mcs > 9) {
+ ath11k_warn(ab, "Invalid VHT mcs index %d\n", mcs);
+ goto err_out;
+ }
+
+ arsta->txrate.mcs = mcs;
+ arsta->txrate.flags = RATE_INFO_FLAGS_VHT_MCS;
+ if (sgi)
+ arsta->txrate.flags |= RATE_INFO_FLAGS_SHORT_GI;
+ } else if (pkt_type == HAL_TX_RATE_STATS_PKT_TYPE_11AX) {
+ if (mcs > 11) {
+ ath11k_warn(ab, "Invalid HE mcs index %d\n", mcs);
+ goto err_out;
+ }
+
+ arsta->txrate.mcs = mcs;
+ arsta->txrate.flags = RATE_INFO_FLAGS_HE_MCS;
+ arsta->txrate.he_gi = ath11k_mac_he_gi_to_nl80211_he_gi(sgi);
+ }
+
+ arsta->txrate.bw = ath11k_mac_bw_to_mac80211_bw(bw);
+ if (ofdma && pkt_type == HAL_TX_RATE_STATS_PKT_TYPE_11AX) {
+ arsta->txrate.bw = RATE_INFO_BW_HE_RU;
+ arsta->txrate.he_ru_alloc =
+ ath11k_mac_he_ru_tones_to_nl80211_he_ru_alloc(ru_tones);
+ }
+
+ if (ath11k_debugfs_is_extd_tx_stats_enabled(ar))
+ ath11k_debugfs_sta_add_tx_stats(arsta, peer_stats, rate_idx);
+
+err_out:
+ spin_unlock_bh(&ab->base_lock);
+}
+
static void ath11k_dp_tx_complete_msdu(struct ath11k *ar,
struct sk_buff *msdu,
struct hal_tx_status *ts)
@@ -435,16 +531,14 @@ static void ath11k_dp_tx_complete_msdu(struct ath11k *ar,
dma_unmap_single(ab->dev, skb_cb->paddr, msdu->len, DMA_TO_DEVICE);
- rcu_read_lock();
-
- if (!rcu_dereference(ab->pdevs_active[ar->pdev_idx])) {
+ if (unlikely(!rcu_access_pointer(ab->pdevs_active[ar->pdev_idx]))) {
dev_kfree_skb_any(msdu);
- goto exit;
+ return;
}
- if (!skb_cb->vif) {
+ if (unlikely(!skb_cb->vif)) {
dev_kfree_skb_any(msdu);
- goto exit;
+ return;
}
info = IEEE80211_SKB_CB(msdu);
@@ -465,7 +559,8 @@ static void ath11k_dp_tx_complete_msdu(struct ath11k *ar,
(info->flags & IEEE80211_TX_CTL_NO_ACK))
info->flags |= IEEE80211_TX_STAT_NOACK_TRANSMITTED;
- if (ath11k_debugfs_is_extd_tx_stats_enabled(ar)) {
+ if (unlikely(ath11k_debugfs_is_extd_tx_stats_enabled(ar)) ||
+ ab->hw_params.single_pdev_only) {
if (ts->flags & HAL_TX_STATUS_FLAGS_FIRST_MSDU) {
if (ar->last_ppdu_id == 0) {
ar->last_ppdu_id = ts->ppdu_id;
@@ -473,12 +568,12 @@ static void ath11k_dp_tx_complete_msdu(struct ath11k *ar,
ar->cached_ppdu_id == ar->last_ppdu_id) {
ar->cached_ppdu_id = ar->last_ppdu_id;
ar->cached_stats.is_ampdu = true;
- ath11k_debugfs_sta_update_txcompl(ar, msdu, ts);
+ ath11k_dp_tx_update_txcompl(ar, ts);
memset(&ar->cached_stats, 0,
sizeof(struct ath11k_per_peer_tx_stats));
} else {
ar->cached_stats.is_ampdu = false;
- ath11k_debugfs_sta_update_txcompl(ar, msdu, ts);
+ ath11k_dp_tx_update_txcompl(ar, ts);
memset(&ar->cached_stats, 0,
sizeof(struct ath11k_per_peer_tx_stats));
}
@@ -494,9 +589,6 @@ static void ath11k_dp_tx_complete_msdu(struct ath11k *ar,
*/
ieee80211_tx_status(ar->hw, msdu);
-
-exit:
- rcu_read_unlock();
}
static inline void ath11k_dp_tx_status_parse(struct ath11k_base *ab,
@@ -505,11 +597,11 @@ static inline void ath11k_dp_tx_status_parse(struct ath11k_base *ab,
{
ts->buf_rel_source =
FIELD_GET(HAL_WBM_RELEASE_INFO0_REL_SRC_MODULE, desc->info0);
- if (ts->buf_rel_source != HAL_WBM_REL_SRC_MODULE_FW &&
- ts->buf_rel_source != HAL_WBM_REL_SRC_MODULE_TQM)
+ if (unlikely(ts->buf_rel_source != HAL_WBM_REL_SRC_MODULE_FW &&
+ ts->buf_rel_source != HAL_WBM_REL_SRC_MODULE_TQM))
return;
- if (ts->buf_rel_source == HAL_WBM_REL_SRC_MODULE_FW)
+ if (unlikely(ts->buf_rel_source == HAL_WBM_REL_SRC_MODULE_FW))
return;
ts->status = FIELD_GET(HAL_WBM_RELEASE_INFO0_TQM_RELEASE_REASON,
@@ -556,8 +648,9 @@ void ath11k_dp_tx_completion_handler(struct ath11k_base *ab, int ring_id)
ATH11K_TX_COMPL_NEXT(tx_ring->tx_status_head);
}
- if ((ath11k_hal_srng_dst_peek(ab, status_ring) != NULL) &&
- (ATH11K_TX_COMPL_NEXT(tx_ring->tx_status_head) == tx_ring->tx_status_tail)) {
+ if (unlikely((ath11k_hal_srng_dst_peek(ab, status_ring) != NULL) &&
+ (ATH11K_TX_COMPL_NEXT(tx_ring->tx_status_head) ==
+ tx_ring->tx_status_tail))) {
/* TODO: Process pending tx_status messages when kfifo_is_full() */
ath11k_warn(ab, "Unable to process some of the tx_status ring desc because status_fifo is full\n");
}
@@ -580,7 +673,7 @@ void ath11k_dp_tx_completion_handler(struct ath11k_base *ab, int ring_id)
mac_id = FIELD_GET(DP_TX_DESC_ID_MAC_ID, desc_id);
msdu_id = FIELD_GET(DP_TX_DESC_ID_MSDU_ID, desc_id);
- if (ts.buf_rel_source == HAL_WBM_REL_SRC_MODULE_FW) {
+ if (unlikely(ts.buf_rel_source == HAL_WBM_REL_SRC_MODULE_FW)) {
ath11k_dp_tx_process_htt_tx_complete(ab,
(void *)tx_status,
mac_id, msdu_id,
@@ -588,16 +681,16 @@ void ath11k_dp_tx_completion_handler(struct ath11k_base *ab, int ring_id)
continue;
}
- spin_lock_bh(&tx_ring->tx_idr_lock);
- msdu = idr_find(&tx_ring->txbuf_idr, msdu_id);
- if (!msdu) {
+ spin_lock(&tx_ring->tx_idr_lock);
+ msdu = idr_remove(&tx_ring->txbuf_idr, msdu_id);
+ if (unlikely(!msdu)) {
ath11k_warn(ab, "tx completion for unknown msdu_id %d\n",
msdu_id);
- spin_unlock_bh(&tx_ring->tx_idr_lock);
+ spin_unlock(&tx_ring->tx_idr_lock);
continue;
}
- idr_remove(&tx_ring->txbuf_idr, msdu_id);
- spin_unlock_bh(&tx_ring->tx_idr_lock);
+
+ spin_unlock(&tx_ring->tx_idr_lock);
ar = ab->pdevs[mac_id].ar;
@@ -903,7 +996,7 @@ int ath11k_dp_tx_htt_h2t_ppdu_stats_req(struct ath11k *ar, u32 mask)
cmd->msg = FIELD_PREP(HTT_PPDU_STATS_CFG_MSG_TYPE,
HTT_H2T_MSG_TYPE_PPDU_STATS_CFG);
- pdev_mask = 1 << (i + 1);
+ pdev_mask = 1 << (ar->pdev_idx + i);
cmd->msg |= FIELD_PREP(HTT_PPDU_STATS_CFG_PDEV_ID, pdev_mask);
cmd->msg |= FIELD_PREP(HTT_PPDU_STATS_CFG_TLV_TYPE_BITMASK, mask);
@@ -993,6 +1086,7 @@ ath11k_dp_tx_htt_h2t_ext_stats_req(struct ath11k *ar, u8 type,
struct ath11k_dp *dp = &ab->dp;
struct sk_buff *skb;
struct htt_ext_stats_cfg_cmd *cmd;
+ u32 pdev_id;
int len = sizeof(*cmd);
int ret;
@@ -1006,7 +1100,12 @@ ath11k_dp_tx_htt_h2t_ext_stats_req(struct ath11k *ar, u8 type,
memset(cmd, 0, sizeof(*cmd));
cmd->hdr.msg_type = HTT_H2T_MSG_TYPE_EXT_STATS_CFG;
- cmd->hdr.pdev_mask = 1 << ar->pdev->pdev_id;
+ if (ab->hw_params.single_pdev_only)
+ pdev_id = ath11k_mac_get_target_pdev_id(ar);
+ else
+ pdev_id = ar->pdev->pdev_id;
+
+ cmd->hdr.pdev_mask = 1 << pdev_id;
cmd->hdr.stats_type = type;
cmd->cfg_param0 = cfg_params->cfg0;
@@ -1034,6 +1133,15 @@ int ath11k_dp_tx_htt_monitor_mode_ring_config(struct ath11k *ar, bool reset)
struct htt_rx_ring_tlv_filter tlv_filter = {0};
int ret = 0, ring_id = 0, i;
+ if (ab->hw_params.full_monitor_mode) {
+ ret = ath11k_dp_tx_htt_rx_full_mon_setup(ab,
+ dp->mac_id, !reset);
+ if (ret < 0) {
+ ath11k_err(ab, "failed to setup full monitor %d\n", ret);
+ return ret;
+ }
+ }
+
ring_id = dp->rxdma_mon_buf_ring.refill_buf_ring.ring_id;
if (!reset) {
@@ -1099,3 +1207,42 @@ int ath11k_dp_tx_htt_monitor_mode_ring_config(struct ath11k *ar, bool reset)
return ret;
}
+
+int ath11k_dp_tx_htt_rx_full_mon_setup(struct ath11k_base *ab, int mac_id,
+ bool config)
+{
+ struct htt_rx_full_monitor_mode_cfg_cmd *cmd;
+ struct sk_buff *skb;
+ int ret, len = sizeof(*cmd);
+
+ skb = ath11k_htc_alloc_skb(ab, len);
+ if (!skb)
+ return -ENOMEM;
+
+ skb_put(skb, len);
+ cmd = (struct htt_rx_full_monitor_mode_cfg_cmd *)skb->data;
+ memset(cmd, 0, sizeof(*cmd));
+ cmd->info0 = FIELD_PREP(HTT_RX_FULL_MON_MODE_CFG_CMD_INFO0_MSG_TYPE,
+ HTT_H2T_MSG_TYPE_RX_FULL_MONITOR_MODE);
+
+ cmd->info0 |= FIELD_PREP(HTT_RX_FULL_MON_MODE_CFG_CMD_INFO0_PDEV_ID, mac_id);
+
+ cmd->cfg = HTT_RX_FULL_MON_MODE_CFG_CMD_CFG_ENABLE |
+ FIELD_PREP(HTT_RX_FULL_MON_MODE_CFG_CMD_CFG_RELEASE_RING,
+ HTT_RX_MON_RING_SW);
+ if (config) {
+ cmd->cfg |= HTT_RX_FULL_MON_MODE_CFG_CMD_CFG_ZERO_MPDUS_END |
+ HTT_RX_FULL_MON_MODE_CFG_CMD_CFG_NON_ZERO_MPDUS_END;
+ }
+
+ ret = ath11k_htc_send(&ab->htc, ab->dp.eid, skb);
+ if (ret)
+ goto err_free;
+
+ return 0;
+
+err_free:
+ dev_kfree_skb_any(skb);
+
+ return ret;
+}
diff --git a/drivers/net/wireless/ath/ath11k/dp_tx.h b/drivers/net/wireless/ath/ath11k/dp_tx.h
index 698b907..e87d65b 100644
--- a/drivers/net/wireless/ath/ath11k/dp_tx.h
+++ b/drivers/net/wireless/ath/ath11k/dp_tx.h
@@ -15,6 +15,7 @@ struct ath11k_dp_htt_wbm_tx_status {
int ack_rssi;
};
+void ath11k_dp_tx_update_txcompl(struct ath11k *ar, struct hal_tx_status *ts);
int ath11k_dp_tx_htt_h2t_ver_req_msg(struct ath11k_base *ab);
int ath11k_dp_tx(struct ath11k *ar, struct ath11k_vif *arvif,
struct ath11k_sta *arsta, struct sk_buff *skb);
@@ -37,4 +38,6 @@ int ath11k_dp_tx_htt_rx_filter_setup(struct ath11k_base *ab, u32 ring_id,
int rx_buf_size,
struct htt_rx_ring_tlv_filter *tlv_filter);
+int ath11k_dp_tx_htt_rx_full_mon_setup(struct ath11k_base *ab, int mac_id,
+ bool config);
#endif
diff --git a/drivers/net/wireless/ath/ath11k/hal.c b/drivers/net/wireless/ath/ath11k/hal.c
index eaa0edc..2ec09ae 100644
--- a/drivers/net/wireless/ath/ath11k/hal.c
+++ b/drivers/net/wireless/ath/ath11k/hal.c
@@ -627,6 +627,21 @@ u32 *ath11k_hal_srng_dst_peek(struct ath11k_base *ab, struct hal_srng *srng)
return NULL;
}
+static void ath11k_hal_srng_prefetch_desc(struct ath11k_base *ab,
+ struct hal_srng *srng)
+{
+ u32 *desc;
+
+ /* prefetch only if desc is available */
+ desc = ath11k_hal_srng_dst_peek(ab, srng);
+ if (likely(desc)) {
+ dma_sync_single_for_cpu(ab->dev, virt_to_phys(desc),
+ (srng->entry_size * sizeof(u32)),
+ DMA_FROM_DEVICE);
+ prefetch(desc);
+ }
+}
+
u32 *ath11k_hal_srng_dst_get_next_entry(struct ath11k_base *ab,
struct hal_srng *srng)
{
@@ -639,8 +654,15 @@ u32 *ath11k_hal_srng_dst_get_next_entry(struct ath11k_base *ab,
desc = srng->ring_base_vaddr + srng->u.dst_ring.tp;
- srng->u.dst_ring.tp = (srng->u.dst_ring.tp + srng->entry_size) %
- srng->ring_size;
+ srng->u.dst_ring.tp += srng->entry_size;
+
+ /* wrap around to start of ring*/
+ if (srng->u.dst_ring.tp == srng->ring_size)
+ srng->u.dst_ring.tp = 0;
+
+ /* Try to prefetch the next descriptor in the ring */
+ if (srng->flags & HAL_SRNG_FLAGS_CACHED)
+ ath11k_hal_srng_prefetch_desc(ab, srng);
return desc;
}
@@ -775,11 +797,16 @@ void ath11k_hal_srng_access_begin(struct ath11k_base *ab, struct hal_srng *srng)
{
lockdep_assert_held(&srng->lock);
- if (srng->ring_dir == HAL_SRNG_DIR_SRC)
+ if (srng->ring_dir == HAL_SRNG_DIR_SRC) {
srng->u.src_ring.cached_tp =
*(volatile u32 *)srng->u.src_ring.tp_addr;
- else
+ } else {
srng->u.dst_ring.cached_hp = *srng->u.dst_ring.hp_addr;
+
+ /* Try to prefetch the next descriptor in the ring */
+ if (srng->flags & HAL_SRNG_FLAGS_CACHED)
+ ath11k_hal_srng_prefetch_desc(ab, srng);
+ }
}
/* Update cached ring head/tail pointers to HW. ath11k_hal_srng_access_begin()
@@ -947,6 +974,7 @@ int ath11k_hal_srng_setup(struct ath11k_base *ab, enum hal_ring_type type,
srng->msi_data = params->msi_data;
srng->initialized = 1;
spin_lock_init(&srng->lock);
+ lockdep_set_class(&srng->lock, hal->srng_key + ring_id);
for (i = 0; i < HAL_SRNG_NUM_REG_GRP; i++) {
srng->hwreg_base[i] = srng_config->reg_start[i] +
@@ -1233,6 +1261,24 @@ static int ath11k_hal_srng_create_config(struct ath11k_base *ab)
return 0;
}
+static void ath11k_hal_register_srng_key(struct ath11k_base *ab)
+{
+ struct ath11k_hal *hal = &ab->hal;
+ u32 ring_id;
+
+ for (ring_id = 0; ring_id < HAL_SRNG_RING_ID_MAX; ring_id++)
+ lockdep_register_key(hal->srng_key + ring_id);
+}
+
+static void ath11k_hal_unregister_srng_key(struct ath11k_base *ab)
+{
+ struct ath11k_hal *hal = &ab->hal;
+ u32 ring_id;
+
+ for (ring_id = 0; ring_id < HAL_SRNG_RING_ID_MAX; ring_id++)
+ lockdep_unregister_key(hal->srng_key + ring_id);
+}
+
int ath11k_hal_srng_init(struct ath11k_base *ab)
{
struct ath11k_hal *hal = &ab->hal;
@@ -1252,6 +1298,8 @@ int ath11k_hal_srng_init(struct ath11k_base *ab)
if (ret)
goto err_free_cont_rdp;
+ ath11k_hal_register_srng_key(ab);
+
return 0;
err_free_cont_rdp:
@@ -1266,6 +1314,7 @@ void ath11k_hal_srng_deinit(struct ath11k_base *ab)
{
struct ath11k_hal *hal = &ab->hal;
+ ath11k_hal_unregister_srng_key(ab);
ath11k_hal_free_cont_rdp(ab);
ath11k_hal_free_cont_wrp(ab);
kfree(hal->srng_config);
diff --git a/drivers/net/wireless/ath/ath11k/hal.h b/drivers/net/wireless/ath/ath11k/hal.h
index 35ed3a1..a7d9b4c 100644
--- a/drivers/net/wireless/ath/ath11k/hal.h
+++ b/drivers/net/wireless/ath/ath11k/hal.h
@@ -513,6 +513,7 @@ enum hal_srng_dir {
#define HAL_SRNG_FLAGS_DATA_TLV_SWAP 0x00000020
#define HAL_SRNG_FLAGS_LOW_THRESH_INTR_EN 0x00010000
#define HAL_SRNG_FLAGS_MSI_INTR 0x00020000
+#define HAL_SRNG_FLAGS_CACHED 0x20000000
#define HAL_SRNG_FLAGS_LMAC_RING 0x80000000
#define HAL_SRNG_TLV_HDR_TAG GENMASK(9, 1)
@@ -901,6 +902,8 @@ struct ath11k_hal {
/* shadow register configuration */
u32 shadow_reg_addr[HAL_SHADOW_NUM_REGS];
int num_shadow_reg_configured;
+
+ struct lock_class_key srng_key[HAL_SRNG_RING_ID_MAX];
};
u32 ath11k_hal_reo_qdesc_size(u32 ba_window_size, u8 tid);
diff --git a/drivers/net/wireless/ath/ath11k/hal_desc.h b/drivers/net/wireless/ath/ath11k/hal_desc.h
index 00b595b..4067676 100644
--- a/drivers/net/wireless/ath/ath11k/hal_desc.h
+++ b/drivers/net/wireless/ath/ath11k/hal_desc.h
@@ -858,6 +858,25 @@ struct hal_reo_entrance_ring {
* this ring has looped around the ring.
*/
+#define HAL_SW_MON_RING_INFO0_RXDMA_PUSH_REASON GENMASK(1, 0)
+#define HAL_SW_MON_RING_INFO0_RXDMA_ERROR_CODE GENMASK(6, 2)
+#define HAL_SW_MON_RING_INFO0_MPDU_FRAG_NUMBER GENMASK(10, 7)
+#define HAL_SW_MON_RING_INFO0_FRAMELESS_BAR BIT(11)
+#define HAL_SW_MON_RING_INFO0_STATUS_BUF_CNT GENMASK(15, 12)
+#define HAL_SW_MON_RING_INFO0_END_OF_PPDU BIT(16)
+
+#define HAL_SW_MON_RING_INFO1_PHY_PPDU_ID GENMASK(15, 0)
+#define HAL_SW_MON_RING_INFO1_RING_ID GENMASK(27, 20)
+#define HAL_SW_MON_RING_INFO1_LOOPING_COUNT GENMASK(31, 28)
+
+struct hal_sw_monitor_ring {
+ struct ath11k_buffer_addr buf_addr_info;
+ struct rx_mpdu_desc rx_mpdu_info;
+ struct ath11k_buffer_addr status_buf_addr_info;
+ u32 info0;
+ u32 info1;
+} __packed;
+
#define HAL_REO_CMD_HDR_INFO0_CMD_NUMBER GENMASK(15, 0)
#define HAL_REO_CMD_HDR_INFO0_STATUS_REQUIRED BIT(16)
diff --git a/drivers/net/wireless/ath/ath11k/hal_rx.c b/drivers/net/wireless/ath/ath11k/hal_rx.c
index 329c404c..a3b353a 100644
--- a/drivers/net/wireless/ath/ath11k/hal_rx.c
+++ b/drivers/net/wireless/ath/ath11k/hal_rx.c
@@ -29,8 +29,7 @@ static int ath11k_hal_reo_cmd_queue_stats(struct hal_tlv_hdr *tlv,
FIELD_PREP(HAL_TLV_HDR_LEN, sizeof(*desc));
desc = (struct hal_reo_get_queue_stats *)tlv->value;
- memset(&desc->queue_addr_lo, 0,
- (sizeof(*desc) - sizeof(struct hal_reo_cmd_hdr)));
+ memset_startat(desc, 0, queue_addr_lo);
desc->cmd.info0 &= ~HAL_REO_CMD_HDR_INFO0_STATUS_REQUIRED;
if (cmd->flag & HAL_REO_CMD_FLG_NEED_STATUS)
@@ -62,8 +61,7 @@ static int ath11k_hal_reo_cmd_flush_cache(struct ath11k_hal *hal, struct hal_tlv
FIELD_PREP(HAL_TLV_HDR_LEN, sizeof(*desc));
desc = (struct hal_reo_flush_cache *)tlv->value;
- memset(&desc->cache_addr_lo, 0,
- (sizeof(*desc) - sizeof(struct hal_reo_cmd_hdr)));
+ memset_startat(desc, 0, cache_addr_lo);
desc->cmd.info0 &= ~HAL_REO_CMD_HDR_INFO0_STATUS_REQUIRED;
if (cmd->flag & HAL_REO_CMD_FLG_NEED_STATUS)
@@ -101,8 +99,7 @@ static int ath11k_hal_reo_cmd_update_rx_queue(struct hal_tlv_hdr *tlv,
FIELD_PREP(HAL_TLV_HDR_LEN, sizeof(*desc));
desc = (struct hal_reo_update_rx_queue *)tlv->value;
- memset(&desc->queue_addr_lo, 0,
- (sizeof(*desc) - sizeof(struct hal_reo_cmd_hdr)));
+ memset_startat(desc, 0, queue_addr_lo);
desc->cmd.info0 &= ~HAL_REO_CMD_HDR_INFO0_STATUS_REQUIRED;
if (cmd->flag & HAL_REO_CMD_FLG_NEED_STATUS)
@@ -374,7 +371,7 @@ int ath11k_hal_wbm_desc_parse_err(struct ath11k_base *ab, void *desc,
ret_buf_mgr = FIELD_GET(BUFFER_ADDR_INFO1_RET_BUF_MGR,
wbm_desc->buf_addr_info.info1);
- if (ret_buf_mgr != ab->hw_params.hal_params->rx_buf_rbm) {
+ if (ret_buf_mgr != HAL_RX_BUF_RBM_SW3_BM) {
ab->soc_stats.invalid_rbm++;
return -EINVAL;
}
@@ -764,15 +761,17 @@ void ath11k_hal_reo_qdesc_setup(void *vaddr, int tid, u32 ba_window_size,
* size changes and also send WMI message to FW to change the REO
* queue descriptor in Rx peer entry as part of dp_rx_tid_update.
*/
- memset(ext_desc, 0, 3 * sizeof(*ext_desc));
+ memset(ext_desc, 0, sizeof(*ext_desc));
ath11k_hal_reo_set_desc_hdr(&ext_desc->desc_hdr, HAL_DESC_REO_OWNED,
HAL_DESC_REO_QUEUE_EXT_DESC,
REO_QUEUE_DESC_MAGIC_DEBUG_PATTERN_1);
ext_desc++;
+ memset(ext_desc, 0, sizeof(*ext_desc));
ath11k_hal_reo_set_desc_hdr(&ext_desc->desc_hdr, HAL_DESC_REO_OWNED,
HAL_DESC_REO_QUEUE_EXT_DESC,
REO_QUEUE_DESC_MAGIC_DEBUG_PATTERN_2);
ext_desc++;
+ memset(ext_desc, 0, sizeof(*ext_desc));
ath11k_hal_reo_set_desc_hdr(&ext_desc->desc_hdr, HAL_DESC_REO_OWNED,
HAL_DESC_REO_QUEUE_EXT_DESC,
REO_QUEUE_DESC_MAGIC_DEBUG_PATTERN_3);
@@ -1039,7 +1038,9 @@ ath11k_hal_rx_parse_mon_status_tlv(struct ath11k_base *ab,
ru_tones = FIELD_GET(HAL_RX_HE_SIG_B1_MU_INFO_INFO0_RU_ALLOCATION,
info0);
- ppdu_info->ru_alloc = ath11k_he_ru_tones_to_nl80211_he_ru_alloc(ru_tones);
+ ppdu_info->ru_alloc =
+ ath11k_mac_phy_he_ru_to_nl80211_he_ru_alloc(ru_tones);
+
ppdu_info->reception_type = HAL_RX_RECEPTION_TYPE_MU_MIMO;
break;
}
@@ -1080,6 +1081,9 @@ ath11k_hal_rx_parse_mon_status_tlv(struct ath11k_base *ab,
break;
}
case HAL_PHYRX_RSSI_LEGACY: {
+ int i;
+ bool db2dbm = test_bit(WMI_TLV_SERVICE_HW_DB2DBM_CONVERSION_SUPPORT,
+ ab->wmi_ab.svc_map);
struct hal_rx_phyrx_rssi_legacy_info *rssi =
(struct hal_rx_phyrx_rssi_legacy_info *)tlv_data;
@@ -1090,6 +1094,14 @@ ath11k_hal_rx_parse_mon_status_tlv(struct ath11k_base *ab,
ppdu_info->rssi_comb =
FIELD_GET(HAL_RX_PHYRX_RSSI_LEGACY_INFO_INFO1_RSSI_COMB,
__le32_to_cpu(rssi->info0));
+
+ if (db2dbm) {
+ for (i = 0; i < ARRAY_SIZE(rssi->preamble); i++) {
+ ppdu_info->rssi_chain_pri20[i] =
+ le32_get_bits(rssi->preamble[i].rssi_2040,
+ HAL_RX_PHYRX_RSSI_PREAMBLE_PRI20);
+ }
+ }
break;
}
case HAL_RX_MPDU_START: {
@@ -1186,3 +1198,47 @@ void ath11k_hal_rx_reo_ent_buf_paddr_get(void *rx_desc, dma_addr_t *paddr,
*pp_buf_addr = (void *)buf_addr_info;
}
+
+void
+ath11k_hal_rx_sw_mon_ring_buf_paddr_get(void *rx_desc,
+ struct hal_sw_mon_ring_entries *sw_mon_entries)
+{
+ struct hal_sw_monitor_ring *sw_mon_ring = rx_desc;
+ struct ath11k_buffer_addr *buf_addr_info;
+ struct ath11k_buffer_addr *status_buf_addr_info;
+ struct rx_mpdu_desc *rx_mpdu_desc_info_details;
+
+ rx_mpdu_desc_info_details = &sw_mon_ring->rx_mpdu_info;
+
+ sw_mon_entries->msdu_cnt = FIELD_GET(RX_MPDU_DESC_INFO0_MSDU_COUNT,
+ rx_mpdu_desc_info_details->info0);
+
+ buf_addr_info = &sw_mon_ring->buf_addr_info;
+ status_buf_addr_info = &sw_mon_ring->status_buf_addr_info;
+
+ sw_mon_entries->mon_dst_paddr = (((u64)FIELD_GET(BUFFER_ADDR_INFO1_ADDR,
+ buf_addr_info->info1)) << 32) |
+ FIELD_GET(BUFFER_ADDR_INFO0_ADDR,
+ buf_addr_info->info0);
+
+ sw_mon_entries->mon_status_paddr =
+ (((u64)FIELD_GET(BUFFER_ADDR_INFO1_ADDR,
+ status_buf_addr_info->info1)) << 32) |
+ FIELD_GET(BUFFER_ADDR_INFO0_ADDR,
+ status_buf_addr_info->info0);
+
+ sw_mon_entries->mon_dst_sw_cookie = FIELD_GET(BUFFER_ADDR_INFO1_SW_COOKIE,
+ buf_addr_info->info1);
+
+ sw_mon_entries->mon_status_sw_cookie = FIELD_GET(BUFFER_ADDR_INFO1_SW_COOKIE,
+ status_buf_addr_info->info1);
+
+ sw_mon_entries->status_buf_count = FIELD_GET(HAL_SW_MON_RING_INFO0_STATUS_BUF_CNT,
+ sw_mon_ring->info0);
+
+ sw_mon_entries->dst_buf_addr_info = buf_addr_info;
+ sw_mon_entries->status_buf_addr_info = status_buf_addr_info;
+
+ sw_mon_entries->ppdu_id =
+ FIELD_GET(HAL_SW_MON_RING_INFO1_PHY_PPDU_ID, sw_mon_ring->info1);
+}
diff --git a/drivers/net/wireless/ath/ath11k/hal_rx.h b/drivers/net/wireless/ath/ath11k/hal_rx.h
index 0f1f04b..571054c 100644
--- a/drivers/net/wireless/ath/ath11k/hal_rx.h
+++ b/drivers/net/wireless/ath/ath11k/hal_rx.h
@@ -77,6 +77,20 @@ enum hal_rx_mon_status {
HAL_RX_MON_STATUS_BUF_DONE,
};
+struct hal_sw_mon_ring_entries {
+ dma_addr_t mon_dst_paddr;
+ dma_addr_t mon_status_paddr;
+ u32 mon_dst_sw_cookie;
+ u32 mon_status_sw_cookie;
+ void *dst_buf_addr_info;
+ void *status_buf_addr_info;
+ u16 ppdu_id;
+ u8 status_buf_count;
+ u8 msdu_cnt;
+ bool end_of_ppdu;
+ bool drop_ppdu;
+};
+
struct hal_rx_mon_ppdu_info {
u32 ppdu_id;
u32 ppdu_ts;
@@ -98,6 +112,7 @@ struct hal_rx_mon_ppdu_info {
u8 ldpc;
u8 beamformed;
u8 rssi_comb;
+ u8 rssi_chain_pri20[HAL_RX_MAX_NSS];
u8 tid;
u8 dcm;
u8 ru_alloc;
@@ -248,8 +263,17 @@ struct hal_rx_he_sig_b2_ofdma_info {
#define HAL_RX_PHYRX_RSSI_LEGACY_INFO_INFO1_RSSI_COMB GENMASK(15, 8)
+#define HAL_RX_PHYRX_RSSI_PREAMBLE_PRI20 GENMASK(7, 0)
+
+struct hal_rx_phyrx_chain_rssi {
+ __le32 rssi_2040;
+ __le32 rssi_80;
+} __packed;
+
struct hal_rx_phyrx_rssi_legacy_info {
- __le32 rsvd[35];
+ __le32 rsvd[3];
+ struct hal_rx_phyrx_chain_rssi pre_rssi[HAL_RX_MAX_NSS];
+ struct hal_rx_phyrx_chain_rssi preamble[HAL_RX_MAX_NSS];
__le32 info0;
} __packed;
@@ -331,38 +355,14 @@ void ath11k_hal_rx_reo_ent_buf_paddr_get(void *rx_desc,
dma_addr_t *paddr, u32 *sw_cookie,
void **pp_buf_addr_info, u8 *rbm,
u32 *msdu_cnt);
+void
+ath11k_hal_rx_sw_mon_ring_buf_paddr_get(void *rx_desc,
+ struct hal_sw_mon_ring_entries *sw_mon_ent);
enum hal_rx_mon_status
ath11k_hal_rx_parse_mon_status(struct ath11k_base *ab,
struct hal_rx_mon_ppdu_info *ppdu_info,
struct sk_buff *skb);
-static inline u32 ath11k_he_ru_tones_to_nl80211_he_ru_alloc(u16 ru_tones)
-{
- u32 ret = 0;
-
- switch (ru_tones) {
- case RU_26:
- ret = NL80211_RATE_INFO_HE_RU_ALLOC_26;
- break;
- case RU_52:
- ret = NL80211_RATE_INFO_HE_RU_ALLOC_52;
- break;
- case RU_106:
- ret = NL80211_RATE_INFO_HE_RU_ALLOC_106;
- break;
- case RU_242:
- ret = NL80211_RATE_INFO_HE_RU_ALLOC_242;
- break;
- case RU_484:
- ret = NL80211_RATE_INFO_HE_RU_ALLOC_484;
- break;
- case RU_996:
- ret = NL80211_RATE_INFO_HE_RU_ALLOC_996;
- break;
- }
- return ret;
-}
-
#define REO_QUEUE_DESC_MAGIC_DEBUG_PATTERN_0 0xDDBEEF
#define REO_QUEUE_DESC_MAGIC_DEBUG_PATTERN_1 0xADBEEF
#define REO_QUEUE_DESC_MAGIC_DEBUG_PATTERN_2 0xBDBEEF
diff --git a/drivers/net/wireless/ath/ath11k/htc.c b/drivers/net/wireless/ath/ath11k/htc.c
index 54b1d34..6913b74 100644
--- a/drivers/net/wireless/ath/ath11k/htc.c
+++ b/drivers/net/wireless/ath/ath11k/htc.c
@@ -81,6 +81,8 @@ int ath11k_htc_send(struct ath11k_htc *htc,
struct ath11k_base *ab = htc->ab;
int credits = 0;
int ret;
+ bool credit_flow_enabled = (ab->hw_params.credit_flow &&
+ ep->tx_credit_flow_enabled);
if (eid >= ATH11K_HTC_EP_COUNT) {
ath11k_warn(ab, "Invalid endpoint id: %d\n", eid);
@@ -89,7 +91,7 @@ int ath11k_htc_send(struct ath11k_htc *htc,
skb_push(skb, sizeof(struct ath11k_htc_hdr));
- if (ep->tx_credit_flow_enabled) {
+ if (credit_flow_enabled) {
credits = DIV_ROUND_UP(skb->len, htc->target_credit_size);
spin_lock_bh(&htc->tx_lock);
if (ep->tx_credits < credits) {
@@ -126,7 +128,7 @@ int ath11k_htc_send(struct ath11k_htc *htc,
err_unmap:
dma_unmap_single(dev, skb_cb->paddr, skb->len, DMA_TO_DEVICE);
err_credits:
- if (ep->tx_credit_flow_enabled) {
+ if (credit_flow_enabled) {
spin_lock_bh(&htc->tx_lock);
ep->tx_credits += credits;
ath11k_dbg(ab, ATH11K_DBG_HTC,
@@ -203,23 +205,25 @@ static int ath11k_htc_process_trailer(struct ath11k_htc *htc,
break;
}
- switch (record->hdr.id) {
- case ATH11K_HTC_RECORD_CREDITS:
- len = sizeof(struct ath11k_htc_credit_report);
- if (record->hdr.len < len) {
- ath11k_warn(ab, "Credit report too long\n");
- status = -EINVAL;
+ if (ab->hw_params.credit_flow) {
+ switch (record->hdr.id) {
+ case ATH11K_HTC_RECORD_CREDITS:
+ len = sizeof(struct ath11k_htc_credit_report);
+ if (record->hdr.len < len) {
+ ath11k_warn(ab, "Credit report too long\n");
+ status = -EINVAL;
+ break;
+ }
+ ath11k_htc_process_credit_report(htc,
+ record->credit_report,
+ record->hdr.len,
+ src_eid);
+ break;
+ default:
+ ath11k_warn(ab, "Unhandled record: id:%d length:%d\n",
+ record->hdr.id, record->hdr.len);
break;
}
- ath11k_htc_process_credit_report(htc,
- record->credit_report,
- record->hdr.len,
- src_eid);
- break;
- default:
- ath11k_warn(ab, "Unhandled record: id:%d length:%d\n",
- record->hdr.id, record->hdr.len);
- break;
}
if (status)
@@ -245,6 +249,29 @@ static void ath11k_htc_suspend_complete(struct ath11k_base *ab, bool ack)
complete(&ab->htc_suspend);
}
+void ath11k_htc_tx_completion_handler(struct ath11k_base *ab,
+ struct sk_buff *skb)
+{
+ struct ath11k_htc *htc = &ab->htc;
+ struct ath11k_htc_ep *ep;
+ void (*ep_tx_complete)(struct ath11k_base *, struct sk_buff *);
+ u8 eid;
+
+ eid = ATH11K_SKB_CB(skb)->eid;
+ if (eid >= ATH11K_HTC_EP_COUNT)
+ return;
+
+ ep = &htc->endpoint[eid];
+ spin_lock_bh(&htc->tx_lock);
+ ep_tx_complete = ep->ep_ops.ep_tx_complete;
+ spin_unlock_bh(&htc->tx_lock);
+ if (!ep_tx_complete) {
+ dev_kfree_skb_any(skb);
+ return;
+ }
+ ep_tx_complete(htc->ab, skb);
+}
+
void ath11k_htc_rx_completion_handler(struct ath11k_base *ab,
struct sk_buff *skb)
{
@@ -607,6 +634,11 @@ int ath11k_htc_connect_service(struct ath11k_htc *htc,
disable_credit_flow_ctrl = true;
}
+ if (!ab->hw_params.credit_flow) {
+ flags |= ATH11K_HTC_CONN_FLAGS_DISABLE_CREDIT_FLOW_CTRL;
+ disable_credit_flow_ctrl = true;
+ }
+
req_msg->flags_len = FIELD_PREP(HTC_SVC_MSG_CONNECTIONFLAGS, flags);
req_msg->msg_svc_id |= FIELD_PREP(HTC_SVC_MSG_SERVICE_ID,
conn_req->service_id);
@@ -732,7 +764,10 @@ int ath11k_htc_start(struct ath11k_htc *htc)
msg->msg_id = FIELD_PREP(HTC_MSG_MESSAGEID,
ATH11K_HTC_MSG_SETUP_COMPLETE_EX_ID);
- ath11k_dbg(ab, ATH11K_DBG_HTC, "HTC is using TX credit flow control\n");
+ if (ab->hw_params.credit_flow)
+ ath11k_dbg(ab, ATH11K_DBG_HTC, "HTC is using TX credit flow control\n");
+ else
+ msg->flags |= ATH11K_GLOBAL_DISABLE_CREDIT_FLOW;
status = ath11k_htc_send(htc, ATH11K_HTC_EP_0, skb);
if (status) {
diff --git a/drivers/net/wireless/ath/ath11k/htc.h b/drivers/net/wireless/ath/ath11k/htc.h
index 6c8a469..f429b37 100644
--- a/drivers/net/wireless/ath/ath11k/htc.h
+++ b/drivers/net/wireless/ath/ath11k/htc.h
@@ -83,8 +83,8 @@ enum ath11k_htc_conn_flags {
ATH11K_HTC_CONN_FLAGS_THRESHOLD_LEVEL_ONE_HALF = 0x1,
ATH11K_HTC_CONN_FLAGS_THRESHOLD_LEVEL_THREE_FOURTHS = 0x2,
ATH11K_HTC_CONN_FLAGS_THRESHOLD_LEVEL_UNITY = 0x3,
- ATH11K_HTC_CONN_FLAGS_REDUCE_CREDIT_DRIBBLE = 1 << 2,
- ATH11K_HTC_CONN_FLAGS_DISABLE_CREDIT_FLOW_CTRL = 1 << 3
+ ATH11K_HTC_CONN_FLAGS_REDUCE_CREDIT_DRIBBLE = 0x4,
+ ATH11K_HTC_CONN_FLAGS_DISABLE_CREDIT_FLOW_CTRL = 0x8,
};
enum ath11k_htc_conn_svc_status {
@@ -116,6 +116,8 @@ struct ath11k_htc_conn_svc_resp {
u32 svc_meta_pad;
} __packed;
+#define ATH11K_GLOBAL_DISABLE_CREDIT_FLOW BIT(1)
+
struct ath11k_htc_setup_complete_extended {
u32 msg_id;
u32 flags;
@@ -305,5 +307,6 @@ int ath11k_htc_send(struct ath11k_htc *htc, enum ath11k_htc_ep_id eid,
struct sk_buff *ath11k_htc_alloc_skb(struct ath11k_base *ar, int size);
void ath11k_htc_rx_completion_handler(struct ath11k_base *ar,
struct sk_buff *skb);
-
+void ath11k_htc_tx_completion_handler(struct ath11k_base *ab,
+ struct sk_buff *skb);
#endif
diff --git a/drivers/net/wireless/ath/ath11k/hw.c b/drivers/net/wireless/ath/ath11k/hw.c
index da35fcf..3b0fdc1 100644
--- a/drivers/net/wireless/ath/ath11k/hw.c
+++ b/drivers/net/wireless/ath/ath11k/hw.c
@@ -150,18 +150,18 @@ static void ath11k_hw_ipq8074_reo_setup(struct ath11k_base *ab)
static void ath11k_init_wmi_config_ipq8074(struct ath11k_base *ab,
struct target_resource_config *config)
{
- config->num_vdevs = ab->num_radios * TARGET_NUM_VDEVS;
+ config->num_vdevs = ab->num_radios * TARGET_NUM_VDEVS(ab);
if (ab->num_radios == 2) {
- config->num_peers = TARGET_NUM_PEERS(DBS);
- config->num_tids = TARGET_NUM_TIDS(DBS);
+ config->num_peers = TARGET_NUM_PEERS(ab, DBS);
+ config->num_tids = TARGET_NUM_TIDS(ab, DBS);
} else if (ab->num_radios == 3) {
- config->num_peers = TARGET_NUM_PEERS(DBS_SBS);
- config->num_tids = TARGET_NUM_TIDS(DBS_SBS);
+ config->num_peers = TARGET_NUM_PEERS(ab, DBS_SBS);
+ config->num_tids = TARGET_NUM_TIDS(ab, DBS_SBS);
} else {
/* Control should not reach here */
- config->num_peers = TARGET_NUM_PEERS(SINGLE);
- config->num_tids = TARGET_NUM_TIDS(SINGLE);
+ config->num_peers = TARGET_NUM_PEERS(ab, SINGLE);
+ config->num_tids = TARGET_NUM_TIDS(ab, SINGLE);
}
config->num_offload_peers = TARGET_NUM_OFFLD_PEERS;
config->num_offload_reorder_buffs = TARGET_NUM_OFFLD_REORDER_BUFFS;
@@ -1061,8 +1061,6 @@ const struct ath11k_hw_ring_mask ath11k_hw_ring_mask_ipq8074 = {
const struct ath11k_hw_ring_mask ath11k_hw_ring_mask_qca6390 = {
.tx = {
ATH11K_TX_RING_MASK_0,
- ATH11K_TX_RING_MASK_1,
- ATH11K_TX_RING_MASK_2,
},
.rx_mon_status = {
0, 0, 0, 0,
diff --git a/drivers/net/wireless/ath/ath11k/hw.h b/drivers/net/wireless/ath/ath11k/hw.h
index 19223d3..29934b3 100644
--- a/drivers/net/wireless/ath/ath11k/hw.h
+++ b/drivers/net/wireless/ath/ath11k/hw.h
@@ -12,26 +12,26 @@
/* Target configuration defines */
/* Num VDEVS per radio */
-#define TARGET_NUM_VDEVS (16 + 1)
+#define TARGET_NUM_VDEVS(ab) (ab->hw_params.num_vdevs)
-#define TARGET_NUM_PEERS_PDEV (512 + TARGET_NUM_VDEVS)
+#define TARGET_NUM_PEERS_PDEV(ab) (ab->hw_params.num_peers + TARGET_NUM_VDEVS(ab))
/* Num of peers for Single Radio mode */
-#define TARGET_NUM_PEERS_SINGLE (TARGET_NUM_PEERS_PDEV)
+#define TARGET_NUM_PEERS_SINGLE(ab) (TARGET_NUM_PEERS_PDEV(ab))
/* Num of peers for DBS */
-#define TARGET_NUM_PEERS_DBS (2 * TARGET_NUM_PEERS_PDEV)
+#define TARGET_NUM_PEERS_DBS(ab) (2 * TARGET_NUM_PEERS_PDEV(ab))
/* Num of peers for DBS_SBS */
-#define TARGET_NUM_PEERS_DBS_SBS (3 * TARGET_NUM_PEERS_PDEV)
+#define TARGET_NUM_PEERS_DBS_SBS(ab) (3 * TARGET_NUM_PEERS_PDEV(ab))
/* Max num of stations (per radio) */
-#define TARGET_NUM_STATIONS 512
+#define TARGET_NUM_STATIONS(ab) (ab->hw_params.num_peers)
-#define TARGET_NUM_PEERS(x) TARGET_NUM_PEERS_##x
+#define TARGET_NUM_PEERS(ab, x) TARGET_NUM_PEERS_##x(ab)
#define TARGET_NUM_PEER_KEYS 2
-#define TARGET_NUM_TIDS(x) (2 * TARGET_NUM_PEERS(x) + \
- 4 * TARGET_NUM_VDEVS + 8)
+#define TARGET_NUM_TIDS(ab, x) (2 * TARGET_NUM_PEERS(ab, x) + \
+ 4 * TARGET_NUM_VDEVS(ab) + 8)
#define TARGET_AST_SKID_LIMIT 16
#define TARGET_NUM_OFFLD_PEERS 4
@@ -77,6 +77,7 @@
#define ATH11K_DEFAULT_CAL_FILE "caldata.bin"
#define ATH11K_AMSS_FILE "amss.bin"
#define ATH11K_M3_FILE "m3.bin"
+#define ATH11K_REGDB_FILE_NAME "regdb.bin"
enum ath11k_hw_rate_cck {
ATH11K_HW_RATE_CCK_LP_11M = 0,
@@ -151,6 +152,9 @@ struct ath11k_hw_params {
u32 svc_to_ce_map_len;
bool single_pdev_only;
+ u32 rfkill_pin;
+ u32 rfkill_cfg;
+ u32 rfkill_on_level;
bool rxdma1_enable;
int num_rxmda_per_pdev;
@@ -168,14 +172,26 @@ struct ath11k_hw_params {
u16 interface_modes;
bool supports_monitor;
+ bool full_monitor_mode;
bool supports_shadow_regs;
bool idle_ps;
+ bool supports_sta_ps;
bool cold_boot_calib;
+ int fw_mem_mode;
+ u32 num_vdevs;
+ u32 num_peers;
bool supports_suspend;
u32 hal_desc_sz;
+ bool supports_regdb;
bool fix_l1ss;
+ bool credit_flow;
u8 max_tx_ring;
const struct ath11k_hw_hal_params *hal_params;
+ bool supports_dynamic_smps_6ghz;
+ bool alloc_cacheable_memory;
+ bool wakeup_mhi;
+ bool supports_rssi_stats;
+ bool fw_wmi_diag_event;
};
struct ath11k_hw_ops {
diff --git a/drivers/net/wireless/ath/ath11k/mac.c b/drivers/net/wireless/ath/ath11k/mac.c
index 1cc5560..07f499d 100644
--- a/drivers/net/wireless/ath/ath11k/mac.c
+++ b/drivers/net/wireless/ath/ath11k/mac.c
@@ -1,6 +1,7 @@
// SPDX-License-Identifier: BSD-3-Clause-Clear
/*
* Copyright (c) 2018-2019 The Linux Foundation. All rights reserved.
+ * Copyright (c) 2021 Qualcomm Innovation Center, Inc. All rights reserved.
*/
#include <net/mac80211.h>
@@ -245,6 +246,93 @@ static const u32 ath11k_smps_map[] = {
static int ath11k_start_vdev_delay(struct ieee80211_hw *hw,
struct ieee80211_vif *vif);
+enum nl80211_he_ru_alloc ath11k_mac_phy_he_ru_to_nl80211_he_ru_alloc(u16 ru_phy)
+{
+ enum nl80211_he_ru_alloc ret;
+
+ switch (ru_phy) {
+ case RU_26:
+ ret = NL80211_RATE_INFO_HE_RU_ALLOC_26;
+ break;
+ case RU_52:
+ ret = NL80211_RATE_INFO_HE_RU_ALLOC_52;
+ break;
+ case RU_106:
+ ret = NL80211_RATE_INFO_HE_RU_ALLOC_106;
+ break;
+ case RU_242:
+ ret = NL80211_RATE_INFO_HE_RU_ALLOC_242;
+ break;
+ case RU_484:
+ ret = NL80211_RATE_INFO_HE_RU_ALLOC_484;
+ break;
+ case RU_996:
+ ret = NL80211_RATE_INFO_HE_RU_ALLOC_996;
+ break;
+ default:
+ ret = NL80211_RATE_INFO_HE_RU_ALLOC_26;
+ break;
+ }
+
+ return ret;
+}
+
+enum nl80211_he_ru_alloc ath11k_mac_he_ru_tones_to_nl80211_he_ru_alloc(u16 ru_tones)
+{
+ enum nl80211_he_ru_alloc ret;
+
+ switch (ru_tones) {
+ case 26:
+ ret = NL80211_RATE_INFO_HE_RU_ALLOC_26;
+ break;
+ case 52:
+ ret = NL80211_RATE_INFO_HE_RU_ALLOC_52;
+ break;
+ case 106:
+ ret = NL80211_RATE_INFO_HE_RU_ALLOC_106;
+ break;
+ case 242:
+ ret = NL80211_RATE_INFO_HE_RU_ALLOC_242;
+ break;
+ case 484:
+ ret = NL80211_RATE_INFO_HE_RU_ALLOC_484;
+ break;
+ case 996:
+ ret = NL80211_RATE_INFO_HE_RU_ALLOC_996;
+ break;
+ case (996 * 2):
+ ret = NL80211_RATE_INFO_HE_RU_ALLOC_2x996;
+ break;
+ default:
+ ret = NL80211_RATE_INFO_HE_RU_ALLOC_26;
+ break;
+ }
+
+ return ret;
+}
+
+enum nl80211_he_gi ath11k_mac_he_gi_to_nl80211_he_gi(u8 sgi)
+{
+ enum nl80211_he_gi ret;
+
+ switch (sgi) {
+ case RX_MSDU_START_SGI_0_8_US:
+ ret = NL80211_RATE_INFO_HE_GI_0_8;
+ break;
+ case RX_MSDU_START_SGI_1_6_US:
+ ret = NL80211_RATE_INFO_HE_GI_1_6;
+ break;
+ case RX_MSDU_START_SGI_3_2_US:
+ ret = NL80211_RATE_INFO_HE_GI_3_2;
+ break;
+ default:
+ ret = NL80211_RATE_INFO_HE_GI_0_8;
+ break;
+ }
+
+ return ret;
+}
+
u8 ath11k_mac_bw_to_mac80211_bw(u8 bw)
{
u8 ret = 0;
@@ -553,6 +641,67 @@ struct ath11k *ath11k_mac_get_ar_by_pdev_id(struct ath11k_base *ab, u32 pdev_id)
return NULL;
}
+struct ath11k_vif *ath11k_mac_get_vif_up(struct ath11k_base *ab)
+{
+ struct ath11k *ar;
+ struct ath11k_pdev *pdev;
+ struct ath11k_vif *arvif;
+ int i;
+
+ for (i = 0; i < ab->num_radios; i++) {
+ pdev = &ab->pdevs[i];
+ ar = pdev->ar;
+ list_for_each_entry(arvif, &ar->arvifs, list) {
+ if (arvif->is_up)
+ return arvif;
+ }
+ }
+
+ return NULL;
+}
+
+static bool ath11k_mac_band_match(enum nl80211_band band1, enum WMI_HOST_WLAN_BAND band2)
+{
+ return (((band1 == NL80211_BAND_2GHZ) && (band2 & WMI_HOST_WLAN_2G_CAP)) ||
+ (((band1 == NL80211_BAND_5GHZ) || (band1 == NL80211_BAND_6GHZ)) &&
+ (band2 & WMI_HOST_WLAN_5G_CAP)));
+}
+
+u8 ath11k_mac_get_target_pdev_id_from_vif(struct ath11k_vif *arvif)
+{
+ struct ath11k *ar = arvif->ar;
+ struct ath11k_base *ab = ar->ab;
+ struct ieee80211_vif *vif = arvif->vif;
+ struct cfg80211_chan_def def;
+ enum nl80211_band band;
+ u8 pdev_id = ab->target_pdev_ids[0].pdev_id;
+ int i;
+
+ if (WARN_ON(ath11k_mac_vif_chan(vif, &def)))
+ return pdev_id;
+
+ band = def.chan->band;
+
+ for (i = 0; i < ab->target_pdev_count; i++) {
+ if (ath11k_mac_band_match(band, ab->target_pdev_ids[i].supported_bands))
+ return ab->target_pdev_ids[i].pdev_id;
+ }
+
+ return pdev_id;
+}
+
+u8 ath11k_mac_get_target_pdev_id(struct ath11k *ar)
+{
+ struct ath11k_vif *arvif;
+
+ arvif = ath11k_mac_get_vif_up(ar->ab);
+
+ if (arvif)
+ return ath11k_mac_get_target_pdev_id_from_vif(arvif);
+ else
+ return ar->ab->target_pdev_ids[0].pdev_id;
+}
+
static void ath11k_pdev_caps_update(struct ath11k *ar)
{
struct ath11k_base *ab = ar->ab;
@@ -775,9 +924,9 @@ static int ath11k_mac_monitor_vdev_start(struct ath11k *ar, int vdev_id,
arg.channel.chan_radar = !!(channel->flags & IEEE80211_CHAN_RADAR);
arg.channel.min_power = 0;
- arg.channel.max_power = channel->max_power * 2;
- arg.channel.max_reg_power = channel->max_reg_power * 2;
- arg.channel.max_antenna_gain = channel->max_antenna_gain * 2;
+ arg.channel.max_power = channel->max_power;
+ arg.channel.max_reg_power = channel->max_reg_power;
+ arg.channel.max_antenna_gain = channel->max_antenna_gain;
arg.pref_tx_streams = ar->num_tx_chains;
arg.pref_rx_streams = ar->num_rx_chains;
@@ -1049,6 +1198,83 @@ static int ath11k_mac_monitor_stop(struct ath11k *ar)
return 0;
}
+static int ath11k_mac_vif_setup_ps(struct ath11k_vif *arvif)
+{
+ struct ath11k *ar = arvif->ar;
+ struct ieee80211_vif *vif = arvif->vif;
+ struct ieee80211_conf *conf = &ar->hw->conf;
+ enum wmi_sta_powersave_param param;
+ enum wmi_sta_ps_mode psmode;
+ int ret;
+ int timeout;
+ bool enable_ps;
+
+ lockdep_assert_held(&arvif->ar->conf_mutex);
+
+ if (arvif->vif->type != NL80211_IFTYPE_STATION)
+ return 0;
+
+ enable_ps = arvif->ps;
+
+ if (!arvif->is_started) {
+ /* mac80211 can update vif powersave state while disconnected.
+ * Firmware doesn't behave nicely and consumes more power than
+ * necessary if PS is disabled on a non-started vdev. Hence
+ * force-enable PS for non-running vdevs.
+ */
+ psmode = WMI_STA_PS_MODE_ENABLED;
+ } else if (enable_ps) {
+ psmode = WMI_STA_PS_MODE_ENABLED;
+ param = WMI_STA_PS_PARAM_INACTIVITY_TIME;
+
+ timeout = conf->dynamic_ps_timeout;
+ if (timeout == 0) {
+ /* firmware doesn't like 0 */
+ timeout = ieee80211_tu_to_usec(vif->bss_conf.beacon_int) / 1000;
+ }
+
+ ret = ath11k_wmi_set_sta_ps_param(ar, arvif->vdev_id, param,
+ timeout);
+ if (ret) {
+ ath11k_warn(ar->ab, "failed to set inactivity time for vdev %d: %i\n",
+ arvif->vdev_id, ret);
+ return ret;
+ }
+ } else {
+ psmode = WMI_STA_PS_MODE_DISABLED;
+ }
+
+ ath11k_dbg(ar->ab, ATH11K_DBG_MAC, "mac vdev %d psmode %s\n",
+ arvif->vdev_id, psmode ? "enable" : "disable");
+
+ ret = ath11k_wmi_pdev_set_ps_mode(ar, arvif->vdev_id, psmode);
+ if (ret) {
+ ath11k_warn(ar->ab, "failed to set sta power save mode %d for vdev %d: %d\n",
+ psmode, arvif->vdev_id, ret);
+ return ret;
+ }
+
+ return 0;
+}
+
+static int ath11k_mac_config_ps(struct ath11k *ar)
+{
+ struct ath11k_vif *arvif;
+ int ret = 0;
+
+ lockdep_assert_held(&ar->conf_mutex);
+
+ list_for_each_entry(arvif, &ar->arvifs, list) {
+ ret = ath11k_mac_vif_setup_ps(arvif);
+ if (ret) {
+ ath11k_warn(ar->ab, "failed to setup powersave: %d\n", ret);
+ break;
+ }
+ }
+
+ return ret;
+}
+
static int ath11k_mac_op_config(struct ieee80211_hw *hw, u32 changed)
{
struct ath11k *ar = hw->priv;
@@ -1137,11 +1363,15 @@ static int ath11k_mac_setup_bcn_tmpl(struct ath11k_vif *arvif)
if (cfg80211_find_ie(WLAN_EID_RSN, ies, (skb_tail_pointer(bcn) - ies)))
arvif->rsnie_present = true;
+ else
+ arvif->rsnie_present = false;
if (cfg80211_find_vendor_ie(WLAN_OUI_MICROSOFT,
WLAN_OUI_TYPE_MICROSOFT_WPA,
ies, (skb_tail_pointer(bcn) - ies)))
arvif->wpaie_present = true;
+ else
+ arvif->wpaie_present = false;
ret = ath11k_wmi_bcn_tmpl(ar, arvif->vdev_id, &offs, bcn);
@@ -1154,6 +1384,26 @@ static int ath11k_mac_setup_bcn_tmpl(struct ath11k_vif *arvif)
return ret;
}
+void ath11k_mac_bcn_tx_event(struct ath11k_vif *arvif)
+{
+ struct ieee80211_vif *vif = arvif->vif;
+
+ if (!vif->color_change_active && !arvif->bcca_zero_sent)
+ return;
+
+ if (vif->color_change_active && ieee80211_beacon_cntdwn_is_complete(vif)) {
+ arvif->bcca_zero_sent = true;
+ ieee80211_color_change_finish(vif);
+ return;
+ }
+
+ arvif->bcca_zero_sent = false;
+
+ if (vif->color_change_active)
+ ieee80211_beacon_update_cntdwn(vif);
+ ath11k_mac_setup_bcn_tmpl(arvif);
+}
+
static void ath11k_control_beaconing(struct ath11k_vif *arvif,
struct ieee80211_bss_conf *info)
{
@@ -1819,7 +2069,6 @@ static void ath11k_peer_assoc_h_he(struct ath11k *ar,
struct ath11k_vif *arvif = (void *)vif->drv_priv;
struct cfg80211_chan_def def;
const struct ieee80211_sta_he_cap *he_cap = &sta->he_cap;
- u8 ampdu_factor;
enum nl80211_band band;
u16 *he_mcs_mask;
u8 max_nss, he_mcs;
@@ -1827,6 +2076,9 @@ static void ath11k_peer_assoc_h_he(struct ath11k *ar,
int i, he_nss, nss_idx;
bool user_rate_valid = true;
u32 rx_nss, tx_nss, nss_160;
+ u8 ampdu_factor, rx_mcs_80, rx_mcs_160;
+ u16 mcs_160_map, mcs_80_map;
+ bool support_160;
if (WARN_ON(ath11k_mac_vif_chan(vif, &def)))
return;
@@ -1841,6 +2093,39 @@ static void ath11k_peer_assoc_h_he(struct ath11k *ar,
return;
arg->he_flag = true;
+ support_160 = !!(he_cap->he_cap_elem.phy_cap_info[0] &
+ IEEE80211_HE_PHY_CAP0_CHANNEL_WIDTH_SET_160MHZ_IN_5G);
+
+ /* Supported HE-MCS and NSS Set of peer he_cap is intersection with self he_cp */
+ mcs_160_map = le16_to_cpu(he_cap->he_mcs_nss_supp.rx_mcs_160);
+ mcs_80_map = le16_to_cpu(he_cap->he_mcs_nss_supp.rx_mcs_80);
+
+ if (support_160) {
+ for (i = 7; i >= 0; i--) {
+ u8 mcs_160 = (mcs_160_map >> (2 * i)) & 3;
+
+ if (mcs_160 != IEEE80211_VHT_MCS_NOT_SUPPORTED) {
+ rx_mcs_160 = i + 1;
+ break;
+ }
+ }
+ }
+
+ for (i = 7; i >= 0; i--) {
+ u8 mcs_80 = (mcs_80_map >> (2 * i)) & 3;
+
+ if (mcs_80 != IEEE80211_VHT_MCS_NOT_SUPPORTED) {
+ rx_mcs_80 = i + 1;
+ break;
+ }
+ }
+
+ if (support_160)
+ max_nss = min(rx_mcs_80, rx_mcs_160);
+ else
+ max_nss = rx_mcs_80;
+
+ arg->peer_nss = min(sta->rx_nss, max_nss);
memcpy_and_pad(&arg->peer_he_cap_macinfo,
sizeof(arg->peer_he_cap_macinfo),
@@ -2343,8 +2628,12 @@ static void ath11k_peer_assoc_prepare(struct ath11k *ar,
struct peer_assoc_params *arg,
bool reassoc)
{
+ struct ath11k_sta *arsta;
+
lockdep_assert_held(&ar->conf_mutex);
+ arsta = (struct ath11k_sta *)sta->drv_priv;
+
memset(arg, 0, sizeof(*arg));
reinit_completion(&ar->peer_assoc_done);
@@ -2361,6 +2650,8 @@ static void ath11k_peer_assoc_prepare(struct ath11k *ar,
ath11k_peer_assoc_h_qos(ar, vif, sta, arg);
ath11k_peer_assoc_h_smps(sta, arg);
+ arsta->peer_nss = arg->peer_nss;
+
/* TODO: amsdu_disable req? */
}
@@ -2397,6 +2688,8 @@ static void ath11k_bss_assoc(struct ieee80211_hw *hw,
struct ath11k_vif *arvif = (void *)vif->drv_priv;
struct peer_assoc_params peer_arg;
struct ieee80211_sta *ap_sta;
+ struct ath11k_peer *peer;
+ bool is_auth = false;
int ret;
lockdep_assert_held(&ar->conf_mutex);
@@ -2418,6 +2711,7 @@ static void ath11k_bss_assoc(struct ieee80211_hw *hw,
rcu_read_unlock();
+ peer_arg.is_assoc = true;
ret = ath11k_wmi_send_peer_assoc_cmd(ar, &peer_arg);
if (ret) {
ath11k_warn(ar->ab, "failed to run peer assoc for %pM vdev %i: %d\n",
@@ -2458,19 +2752,37 @@ static void ath11k_bss_assoc(struct ieee80211_hw *hw,
"mac vdev %d up (associated) bssid %pM aid %d\n",
arvif->vdev_id, bss_conf->bssid, bss_conf->aid);
- /* Authorize BSS Peer */
- ret = ath11k_wmi_set_peer_param(ar, arvif->bssid,
- arvif->vdev_id,
- WMI_PEER_AUTHORIZE,
- 1);
- if (ret)
- ath11k_warn(ar->ab, "Unable to authorize BSS peer: %d\n", ret);
+ spin_lock_bh(&ar->ab->base_lock);
+
+ peer = ath11k_peer_find(ar->ab, arvif->vdev_id, arvif->bssid);
+ if (peer && peer->is_authorized)
+ is_auth = true;
+
+ spin_unlock_bh(&ar->ab->base_lock);
+
+ if (is_auth) {
+ ret = ath11k_wmi_set_peer_param(ar, arvif->bssid,
+ arvif->vdev_id,
+ WMI_PEER_AUTHORIZE,
+ 1);
+ if (ret)
+ ath11k_warn(ar->ab, "Unable to authorize BSS peer: %d\n", ret);
+ }
ret = ath11k_wmi_send_obss_spr_cmd(ar, arvif->vdev_id,
&bss_conf->he_obss_pd);
if (ret)
ath11k_warn(ar->ab, "failed to set vdev %i OBSS PD parameters: %d\n",
arvif->vdev_id, ret);
+
+ ret = ath11k_wmi_vdev_set_param_cmd(ar, arvif->vdev_id,
+ WMI_VDEV_PARAM_DTIM_POLICY,
+ WMI_DTIM_POLICY_STICK);
+ if (ret)
+ ath11k_warn(ar->ab, "failed to set vdev %d dtim policy: %d\n",
+ arvif->vdev_id, ret);
+
+ ath11k_mac_11d_scan_stop_all(ar->ab);
}
static void ath11k_bss_disassoc(struct ieee80211_hw *hw,
@@ -2805,10 +3117,17 @@ static void ath11k_mac_op_bss_info_changed(struct ieee80211_hw *hw,
"Set staggered beacon mode for VDEV: %d\n",
arvif->vdev_id);
- ret = ath11k_mac_setup_bcn_tmpl(arvif);
- if (ret)
- ath11k_warn(ar->ab, "failed to update bcn template: %d\n",
- ret);
+ if (!arvif->do_not_send_tmpl || !arvif->bcca_zero_sent) {
+ ret = ath11k_mac_setup_bcn_tmpl(arvif);
+ if (ret)
+ ath11k_warn(ar->ab, "failed to update bcn template: %d\n",
+ ret);
+ }
+
+ if (arvif->bcca_zero_sent)
+ arvif->do_not_send_tmpl = true;
+ else
+ arvif->do_not_send_tmpl = false;
}
if (changed & (BSS_CHANGED_BEACON_INFO | BSS_CHANGED_BEACON)) {
@@ -2942,6 +3261,16 @@ static void ath11k_mac_op_bss_info_changed(struct ieee80211_hw *hw,
ath11k_mac_txpower_recalc(ar);
}
+ if (changed & BSS_CHANGED_PS &&
+ ar->ab->hw_params.supports_sta_ps) {
+ arvif->ps = vif->bss_conf.ps;
+
+ ret = ath11k_mac_config_ps(ar);
+ if (ret)
+ ath11k_warn(ar->ab, "failed to setup ps on vdev %i: %d\n",
+ arvif->vdev_id, ret);
+ }
+
if (changed & BSS_CHANGED_MCAST_RATE &&
!ath11k_mac_vif_chan(arvif->vif, &def)) {
band = def.chan->band;
@@ -3009,6 +3338,25 @@ static void ath11k_mac_op_bss_info_changed(struct ieee80211_hw *hw,
if (ret)
ath11k_warn(ar->ab, "failed to set bss color collision on vdev %i: %d\n",
arvif->vdev_id, ret);
+
+ param_id = WMI_VDEV_PARAM_BSS_COLOR;
+ if (info->he_bss_color.enabled)
+ param_value = info->he_bss_color.color <<
+ IEEE80211_HE_OPERATION_BSS_COLOR_OFFSET;
+ else
+ param_value = IEEE80211_HE_OPERATION_BSS_COLOR_DISABLED;
+
+ ret = ath11k_wmi_vdev_set_param_cmd(ar, arvif->vdev_id,
+ param_id,
+ param_value);
+ if (ret)
+ ath11k_warn(ar->ab,
+ "failed to set bss color param on vdev %i: %d\n",
+ arvif->vdev_id, ret);
+
+ ath11k_dbg(ar->ab, ATH11K_DBG_MAC,
+ "bss color param 0x%x set on vdev %i\n",
+ param_value, arvif->vdev_id);
} else if (vif->type == NL80211_IFTYPE_STATION) {
ret = ath11k_wmi_send_bss_color_change_enable_cmd(ar,
arvif->vdev_id,
@@ -3164,6 +3512,7 @@ static int ath11k_start_scan(struct ath11k *ar,
struct scan_req_params *arg)
{
int ret;
+ unsigned long timeout = 1 * HZ;
lockdep_assert_held(&ar->conf_mutex);
@@ -3174,7 +3523,14 @@ static int ath11k_start_scan(struct ath11k *ar,
if (ret)
return ret;
- ret = wait_for_completion_timeout(&ar->scan.started, 1 * HZ);
+ if (test_bit(WMI_TLV_SERVICE_11D_OFFLOAD, ar->ab->wmi_ab.svc_map)) {
+ timeout = 5 * HZ;
+
+ if (ar->supports_6ghz)
+ timeout += 5 * HZ;
+ }
+
+ ret = wait_for_completion_timeout(&ar->scan.started, timeout);
if (ret == 0) {
ret = ath11k_scan_stop(ar);
if (ret)
@@ -3231,15 +3587,38 @@ static int ath11k_mac_op_hw_scan(struct ieee80211_hw *hw,
if (ret)
goto exit;
+ /* Currently the pending_11d=true only happened 1 time while
+ * wlan interface up in ath11k_mac_11d_scan_start(), it is called by
+ * ath11k_mac_op_add_interface(), after wlan interface up,
+ * pending_11d=false always.
+ * If remove below wait, it always happened scan fail and lead connect
+ * fail while wlan interface up, because it has a 11d scan which is running
+ * in firmware, and lead this scan failed.
+ */
+ if (ar->pending_11d) {
+ long time_left;
+ unsigned long timeout = 5 * HZ;
+
+ if (ar->supports_6ghz)
+ timeout += 5 * HZ;
+
+ time_left = wait_for_completion_timeout(&ar->finish_11d_ch_list, timeout);
+ ath11k_dbg(ar->ab, ATH11K_DBG_MAC,
+ "mac wait 11d channel list time left %ld\n", time_left);
+ }
+
memset(&arg, 0, sizeof(arg));
ath11k_wmi_start_scan_init(ar, &arg);
arg.vdev_id = arvif->vdev_id;
arg.scan_id = ATH11K_SCAN_ID;
if (req->ie_len) {
+ arg.extraie.ptr = kmemdup(req->ie, req->ie_len, GFP_KERNEL);
+ if (!arg.extraie.ptr) {
+ ret = -ENOMEM;
+ goto exit;
+ }
arg.extraie.len = req->ie_len;
- arg.extraie.ptr = kzalloc(req->ie_len, GFP_KERNEL);
- memcpy(arg.extraie.ptr, req->ie, req->ie_len);
}
if (req->n_ssids) {
@@ -3255,10 +3634,24 @@ static int ath11k_mac_op_hw_scan(struct ieee80211_hw *hw,
if (req->n_channels) {
arg.num_chan = req->n_channels;
+ arg.chan_list = kcalloc(arg.num_chan, sizeof(*arg.chan_list),
+ GFP_KERNEL);
+
+ if (!arg.chan_list) {
+ ret = -ENOMEM;
+ goto exit;
+ }
+
for (i = 0; i < arg.num_chan; i++)
arg.chan_list[i] = req->channels[i]->center_freq;
}
+ if (req->flags & NL80211_SCAN_FLAG_RANDOM_ADDR) {
+ arg.scan_f_add_spoofed_mac_in_probe = 1;
+ ether_addr_copy(arg.mac_addr.addr, req->mac_addr);
+ ether_addr_copy(arg.mac_mask.addr, req->mac_addr_mask);
+ }
+
ret = ath11k_start_scan(ar, &arg);
if (ret) {
ath11k_warn(ar->ab, "failed to start hw scan: %d\n", ret);
@@ -3273,6 +3666,8 @@ static int ath11k_mac_op_hw_scan(struct ieee80211_hw *hw,
ATH11K_MAC_SCAN_TIMEOUT_MSECS));
exit:
+ kfree(arg.chan_list);
+
if (req->ie_len)
kfree(arg.extraie.ptr);
@@ -3316,9 +3711,7 @@ static int ath11k_install_key(struct ath11k_vif *arvif,
return 0;
if (cmd == DISABLE_KEY) {
- /* TODO: Check if FW expects value other than NONE for del */
- /* arg.key_cipher = WMI_CIPHER_NONE; */
- arg.key_len = 0;
+ arg.key_cipher = WMI_CIPHER_NONE;
arg.key_data = NULL;
goto install;
}
@@ -3450,7 +3843,7 @@ static int ath11k_mac_op_set_key(struct ieee80211_hw *hw, enum set_key_cmd cmd,
/* flush the fragments cache during key (re)install to
* ensure all frags in the new frag list belong to the same key.
*/
- if (peer && cmd == SET_KEY)
+ if (peer && sta && cmd == SET_KEY)
ath11k_peer_frags_flush(ar, peer);
spin_unlock_bh(&ab->base_lock);
@@ -3685,6 +4078,7 @@ static int ath11k_station_assoc(struct ath11k *ar,
ath11k_peer_assoc_prepare(ar, vif, sta, &peer_arg, reassoc);
+ peer_arg.is_assoc = true;
ret = ath11k_wmi_send_peer_assoc_cmd(ar, &peer_arg);
if (ret) {
ath11k_warn(ar->ab, "failed to run peer assoc for STA %pM vdev %i: %d\n",
@@ -3824,11 +4218,27 @@ static void ath11k_sta_rc_update_wk(struct work_struct *wk)
ath11k_mac_max_he_nss(he_mcs_mask)));
if (changed & IEEE80211_RC_BW_CHANGED) {
- err = ath11k_wmi_set_peer_param(ar, sta->addr, arvif->vdev_id,
- WMI_PEER_CHWIDTH, bw);
- if (err)
- ath11k_warn(ar->ab, "failed to update STA %pM peer bw %d: %d\n",
- sta->addr, bw, err);
+ /* Send peer assoc command before set peer bandwidth param to
+ * avoid the mismatch between the peer phymode and the peer
+ * bandwidth.
+ */
+ ath11k_peer_assoc_prepare(ar, arvif->vif, sta, &peer_arg, true);
+
+ peer_arg.is_assoc = false;
+ err = ath11k_wmi_send_peer_assoc_cmd(ar, &peer_arg);
+ if (err) {
+ ath11k_warn(ar->ab, "failed to send peer assoc for STA %pM vdev %i: %d\n",
+ sta->addr, arvif->vdev_id, err);
+ } else if (wait_for_completion_timeout(&ar->peer_assoc_done, 1 * HZ)) {
+ err = ath11k_wmi_set_peer_param(ar, sta->addr, arvif->vdev_id,
+ WMI_PEER_CHWIDTH, bw);
+ if (err)
+ ath11k_warn(ar->ab, "failed to update STA %pM peer bw %d: %d\n",
+ sta->addr, bw, err);
+ } else {
+ ath11k_warn(ar->ab, "failed to get peer assoc conf event for %pM vdev %i\n",
+ sta->addr, arvif->vdev_id);
+ }
}
if (changed & IEEE80211_RC_NSS_CHANGED) {
@@ -3896,6 +4306,7 @@ static void ath11k_sta_rc_update_wk(struct work_struct *wk)
ath11k_peer_assoc_prepare(ar, arvif->vif, sta,
&peer_arg, true);
+ peer_arg.is_assoc = false;
err = ath11k_wmi_send_peer_assoc_cmd(ar, &peer_arg);
if (err)
ath11k_warn(ar->ab, "failed to run peer assoc for STA %pM vdev %i: %d\n",
@@ -4095,6 +4506,10 @@ static int ath11k_mac_op_sta_state(struct ieee80211_hw *hw,
new_state == IEEE80211_STA_NOTEXIST)) {
ath11k_dp_peer_cleanup(ar, arvif->vdev_id, sta->addr);
+ if (ar->ab->hw_params.vdev_start_delay &&
+ vif->type == NL80211_IFTYPE_STATION)
+ goto free;
+
ret = ath11k_peer_delete(ar, arvif->vdev_id, sta->addr);
if (ret)
ath11k_warn(ar->ab, "Failed to delete peer: %pM for VDEV: %d\n",
@@ -4116,6 +4531,7 @@ static int ath11k_mac_op_sta_state(struct ieee80211_hw *hw,
}
spin_unlock_bh(&ar->ab->base_lock);
+free:
kfree(arsta->tx_stats);
arsta->tx_stats = NULL;
@@ -4131,6 +4547,34 @@ static int ath11k_mac_op_sta_state(struct ieee80211_hw *hw,
ath11k_warn(ar->ab, "Failed to associate station: %pM\n",
sta->addr);
} else if (old_state == IEEE80211_STA_ASSOC &&
+ new_state == IEEE80211_STA_AUTHORIZED) {
+ spin_lock_bh(&ar->ab->base_lock);
+
+ peer = ath11k_peer_find(ar->ab, arvif->vdev_id, sta->addr);
+ if (peer)
+ peer->is_authorized = true;
+
+ spin_unlock_bh(&ar->ab->base_lock);
+
+ if (vif->type == NL80211_IFTYPE_STATION && arvif->is_up) {
+ ret = ath11k_wmi_set_peer_param(ar, sta->addr,
+ arvif->vdev_id,
+ WMI_PEER_AUTHORIZE,
+ 1);
+ if (ret)
+ ath11k_warn(ar->ab, "Unable to authorize peer %pM vdev %d: %d\n",
+ sta->addr, arvif->vdev_id, ret);
+ }
+ } else if (old_state == IEEE80211_STA_AUTHORIZED &&
+ new_state == IEEE80211_STA_ASSOC) {
+ spin_lock_bh(&ar->ab->base_lock);
+
+ peer = ath11k_peer_find(ar->ab, arvif->vdev_id, sta->addr);
+ if (peer)
+ peer->is_authorized = false;
+
+ spin_unlock_bh(&ar->ab->base_lock);
+ } else if (old_state == IEEE80211_STA_ASSOC &&
new_state == IEEE80211_STA_AUTH &&
(vif->type == NL80211_IFTYPE_AP ||
vif->type == NL80211_IFTYPE_MESH_POINT ||
@@ -4561,6 +5005,10 @@ ath11k_create_vht_cap(struct ath11k *ar, u32 rate_cap_tx_chainmask,
vht_cap.vht_supported = 1;
vht_cap.cap = ar->pdev->cap.vht_cap;
+ if (ar->pdev->cap.nss_ratio_enabled)
+ vht_cap.vht_mcs.tx_highest |=
+ cpu_to_le16(IEEE80211_VHT_EXT_NSS_BW_CAPABLE);
+
ath11k_set_vht_txbf_cap(ar, &vht_cap.cap);
rxmcs_map = 0;
@@ -4926,23 +5374,47 @@ static int __ath11k_set_antenna(struct ath11k *ar, u32 tx_ant, u32 rx_ant)
return 0;
}
-int ath11k_mac_tx_mgmt_pending_free(int buf_id, void *skb, void *ctx)
+static void ath11k_mgmt_over_wmi_tx_drop(struct ath11k *ar, struct sk_buff *skb)
{
- struct sk_buff *msdu = skb;
+ int num_mgmt;
+
+ ieee80211_free_txskb(ar->hw, skb);
+
+ num_mgmt = atomic_dec_if_positive(&ar->num_pending_mgmt_tx);
+
+ if (num_mgmt < 0)
+ WARN_ON_ONCE(1);
+
+ if (!num_mgmt)
+ wake_up(&ar->txmgmt_empty_waitq);
+}
+
+static void ath11k_mac_tx_mgmt_free(struct ath11k *ar, int buf_id)
+{
+ struct sk_buff *msdu;
struct ieee80211_tx_info *info;
- struct ath11k *ar = ctx;
- struct ath11k_base *ab = ar->ab;
spin_lock_bh(&ar->txmgmt_idr_lock);
- idr_remove(&ar->txmgmt_idr, buf_id);
+ msdu = idr_remove(&ar->txmgmt_idr, buf_id);
spin_unlock_bh(&ar->txmgmt_idr_lock);
- dma_unmap_single(ab->dev, ATH11K_SKB_CB(msdu)->paddr, msdu->len,
+
+ if (!msdu)
+ return;
+
+ dma_unmap_single(ar->ab->dev, ATH11K_SKB_CB(msdu)->paddr, msdu->len,
DMA_TO_DEVICE);
info = IEEE80211_SKB_CB(msdu);
memset(&info->status, 0, sizeof(info->status));
- ieee80211_free_txskb(ar->hw, msdu);
+ ath11k_mgmt_over_wmi_tx_drop(ar, msdu);
+}
+
+int ath11k_mac_tx_mgmt_pending_free(int buf_id, void *skb, void *ctx)
+{
+ struct ath11k *ar = ctx;
+
+ ath11k_mac_tx_mgmt_free(ar, buf_id);
return 0;
}
@@ -4951,17 +5423,10 @@ static int ath11k_mac_vif_txmgmt_idr_remove(int buf_id, void *skb, void *ctx)
{
struct ieee80211_vif *vif = ctx;
struct ath11k_skb_cb *skb_cb = ATH11K_SKB_CB((struct sk_buff *)skb);
- struct sk_buff *msdu = skb;
struct ath11k *ar = skb_cb->ar;
- struct ath11k_base *ab = ar->ab;
- if (skb_cb->vif == vif) {
- spin_lock_bh(&ar->txmgmt_idr_lock);
- idr_remove(&ar->txmgmt_idr, buf_id);
- spin_unlock_bh(&ar->txmgmt_idr_lock);
- dma_unmap_single(ab->dev, skb_cb->paddr, msdu->len,
- DMA_TO_DEVICE);
- }
+ if (skb_cb->vif == vif)
+ ath11k_mac_tx_mgmt_free(ar, buf_id);
return 0;
}
@@ -4976,10 +5441,16 @@ static int ath11k_mac_mgmt_tx_wmi(struct ath11k *ar, struct ath11k_vif *arvif,
int buf_id;
int ret;
+ ATH11K_SKB_CB(skb)->ar = ar;
+
spin_lock_bh(&ar->txmgmt_idr_lock);
buf_id = idr_alloc(&ar->txmgmt_idr, skb, 0,
ATH11K_TX_MGMT_NUM_PENDING_MAX, GFP_ATOMIC);
spin_unlock_bh(&ar->txmgmt_idr_lock);
+
+ ath11k_dbg(ar->ab, ATH11K_DBG_MAC,
+ "mac tx mgmt frame, buf id %d\n", buf_id);
+
if (buf_id < 0)
return -ENOSPC;
@@ -5026,7 +5497,7 @@ static void ath11k_mgmt_over_wmi_tx_purge(struct ath11k *ar)
struct sk_buff *skb;
while ((skb = skb_dequeue(&ar->wmi_mgmt_tx_queue)) != NULL)
- ieee80211_free_txskb(ar->hw, skb);
+ ath11k_mgmt_over_wmi_tx_drop(ar, skb);
}
static void ath11k_mgmt_over_wmi_tx_work(struct work_struct *work)
@@ -5041,7 +5512,7 @@ static void ath11k_mgmt_over_wmi_tx_work(struct work_struct *work)
skb_cb = ATH11K_SKB_CB(skb);
if (!skb_cb->vif) {
ath11k_warn(ar->ab, "no vif found for mgmt frame\n");
- ieee80211_free_txskb(ar->hw, skb);
+ ath11k_mgmt_over_wmi_tx_drop(ar, skb);
continue;
}
@@ -5052,16 +5523,18 @@ static void ath11k_mgmt_over_wmi_tx_work(struct work_struct *work)
if (ret) {
ath11k_warn(ar->ab, "failed to tx mgmt frame, vdev_id %d :%d\n",
arvif->vdev_id, ret);
- ieee80211_free_txskb(ar->hw, skb);
+ ath11k_mgmt_over_wmi_tx_drop(ar, skb);
} else {
- atomic_inc(&ar->num_pending_mgmt_tx);
+ ath11k_dbg(ar->ab, ATH11K_DBG_MAC,
+ "mac tx mgmt frame, vdev_id %d\n",
+ arvif->vdev_id);
}
} else {
ath11k_warn(ar->ab,
"dropping mgmt frame for vdev %d, is_started %d\n",
arvif->vdev_id,
arvif->is_started);
- ieee80211_free_txskb(ar->hw, skb);
+ ath11k_mgmt_over_wmi_tx_drop(ar, skb);
}
}
}
@@ -5092,11 +5565,69 @@ static int ath11k_mac_mgmt_tx(struct ath11k *ar, struct sk_buff *skb,
}
skb_queue_tail(q, skb);
+ atomic_inc(&ar->num_pending_mgmt_tx);
ieee80211_queue_work(ar->hw, &ar->wmi_mgmt_tx_work);
return 0;
}
+int ath11k_mac_rfkill_config(struct ath11k *ar)
+{
+ struct ath11k_base *ab = ar->ab;
+ u32 param;
+ int ret;
+
+ if (ab->hw_params.rfkill_pin == 0)
+ return -EOPNOTSUPP;
+
+ ath11k_dbg(ab, ATH11K_DBG_MAC,
+ "mac rfkill_pin %d rfkill_cfg %d rfkill_on_level %d",
+ ab->hw_params.rfkill_pin, ab->hw_params.rfkill_cfg,
+ ab->hw_params.rfkill_on_level);
+
+ param = FIELD_PREP(WMI_RFKILL_CFG_RADIO_LEVEL,
+ ab->hw_params.rfkill_on_level) |
+ FIELD_PREP(WMI_RFKILL_CFG_GPIO_PIN_NUM,
+ ab->hw_params.rfkill_pin) |
+ FIELD_PREP(WMI_RFKILL_CFG_PIN_AS_GPIO,
+ ab->hw_params.rfkill_cfg);
+
+ ret = ath11k_wmi_pdev_set_param(ar, WMI_PDEV_PARAM_HW_RFKILL_CONFIG,
+ param, ar->pdev->pdev_id);
+ if (ret) {
+ ath11k_warn(ab,
+ "failed to set rfkill config 0x%x: %d\n",
+ param, ret);
+ return ret;
+ }
+
+ return 0;
+}
+
+int ath11k_mac_rfkill_enable_radio(struct ath11k *ar, bool enable)
+{
+ enum wmi_rfkill_enable_radio param;
+ int ret;
+
+ if (enable)
+ param = WMI_RFKILL_ENABLE_RADIO_ON;
+ else
+ param = WMI_RFKILL_ENABLE_RADIO_OFF;
+
+ ath11k_dbg(ar->ab, ATH11K_DBG_MAC, "mac %d rfkill enable %d",
+ ar->pdev_idx, param);
+
+ ret = ath11k_wmi_pdev_set_param(ar, WMI_PDEV_PARAM_RFKILL_ENABLE,
+ param, ar->pdev->pdev_id);
+ if (ret) {
+ ath11k_warn(ar->ab, "failed to set rfkill enable param %d: %d\n",
+ param, ret);
+ return ret;
+ }
+
+ return 0;
+}
+
static void ath11k_mac_op_tx(struct ieee80211_hw *hw,
struct ieee80211_tx_control *control,
struct sk_buff *skb)
@@ -5138,7 +5669,7 @@ static void ath11k_mac_op_tx(struct ieee80211_hw *hw,
arsta = (struct ath11k_sta *)control->sta->drv_priv;
ret = ath11k_dp_tx(ar, arvif, arsta, skb);
- if (ret) {
+ if (unlikely(ret)) {
ath11k_warn(ar->ab, "failed to transmit frame %d\n", ret);
ieee80211_free_txskb(ar->hw, skb);
}
@@ -5222,6 +5753,14 @@ static int ath11k_mac_op_start(struct ieee80211_hw *hw)
goto err;
}
+ if (test_bit(WMI_TLV_SERVICE_SPOOF_MAC_SUPPORT, ar->wmi->wmi_ab->svc_map)) {
+ ret = ath11k_wmi_scan_prob_req_oui(ar, ar->mac_addr);
+ if (ret) {
+ ath11k_err(ab, "failed to set prob req oui: %i\n", ret);
+ goto err;
+ }
+ }
+
ret = ath11k_wmi_pdev_set_param(ar, WMI_PDEV_PARAM_ARP_AC_OVERRIDE,
0, pdev->pdev_id);
if (ret) {
@@ -5320,6 +5859,8 @@ static void ath11k_mac_op_stop(struct ieee80211_hw *hw)
cancel_delayed_work_sync(&ar->scan.timeout);
cancel_work_sync(&ar->regd_update_work);
+ cancel_work_sync(&ar->ab->update_11d_work);
+ cancel_work_sync(&ar->ab->rfkill_work);
spin_lock_bh(&ar->data_lock);
list_for_each_entry_safe(ppdu_stats, tmp, &ar->ppdu_stats_info, list) {
@@ -5473,6 +6014,122 @@ static void ath11k_mac_op_update_vif_offload(struct ieee80211_hw *hw,
}
}
+static bool ath11k_mac_vif_ap_active_any(struct ath11k_base *ab)
+{
+ struct ath11k *ar;
+ struct ath11k_pdev *pdev;
+ struct ath11k_vif *arvif;
+ int i;
+
+ for (i = 0; i < ab->num_radios; i++) {
+ pdev = &ab->pdevs[i];
+ ar = pdev->ar;
+ list_for_each_entry(arvif, &ar->arvifs, list) {
+ if (arvif->is_up && arvif->vdev_type == WMI_VDEV_TYPE_AP)
+ return true;
+ }
+ }
+ return false;
+}
+
+void ath11k_mac_11d_scan_start(struct ath11k *ar, u32 vdev_id, bool wait)
+{
+ struct wmi_11d_scan_start_params param;
+ int ret;
+
+ mutex_lock(&ar->ab->vdev_id_11d_lock);
+
+ ath11k_dbg(ar->ab, ATH11K_DBG_MAC, "mac vdev id for 11d scan %d\n",
+ ar->vdev_id_11d_scan);
+
+ if (ar->regdom_set_by_user)
+ goto fin;
+
+ if (ar->vdev_id_11d_scan != ATH11K_11D_INVALID_VDEV_ID)
+ goto fin;
+
+ if (!test_bit(WMI_TLV_SERVICE_11D_OFFLOAD, ar->ab->wmi_ab.svc_map))
+ goto fin;
+
+ if (ath11k_mac_vif_ap_active_any(ar->ab))
+ goto fin;
+
+ param.vdev_id = vdev_id;
+ param.start_interval_msec = 0;
+ param.scan_period_msec = ATH11K_SCAN_11D_INTERVAL;
+
+ ath11k_dbg(ar->ab, ATH11K_DBG_MAC, "mac start 11d scan\n");
+
+ if (wait)
+ reinit_completion(&ar->finish_11d_scan);
+
+ ret = ath11k_wmi_send_11d_scan_start_cmd(ar, ¶m);
+ if (ret) {
+ ath11k_warn(ar->ab, "failed to start 11d scan vdev %d ret: %d\n",
+ vdev_id, ret);
+ } else {
+ ar->vdev_id_11d_scan = vdev_id;
+ if (wait) {
+ ar->pending_11d = true;
+ ret = wait_for_completion_timeout(&ar->finish_11d_scan,
+ 5 * HZ);
+ ath11k_dbg(ar->ab, ATH11K_DBG_MAC,
+ "mac 11d scan left time %d\n", ret);
+
+ if (!ret)
+ ar->pending_11d = false;
+ }
+ }
+
+fin:
+ mutex_unlock(&ar->ab->vdev_id_11d_lock);
+}
+
+void ath11k_mac_11d_scan_stop(struct ath11k *ar)
+{
+ int ret;
+ u32 vdev_id;
+
+ if (!test_bit(WMI_TLV_SERVICE_11D_OFFLOAD, ar->ab->wmi_ab.svc_map))
+ return;
+
+ ath11k_dbg(ar->ab, ATH11K_DBG_MAC, "mac stop 11d scan\n");
+
+ mutex_lock(&ar->ab->vdev_id_11d_lock);
+
+ ath11k_dbg(ar->ab, ATH11K_DBG_MAC, "mac stop 11d vdev id %d\n",
+ ar->vdev_id_11d_scan);
+
+ if (ar->vdev_id_11d_scan != ATH11K_11D_INVALID_VDEV_ID) {
+ vdev_id = ar->vdev_id_11d_scan;
+
+ ret = ath11k_wmi_send_11d_scan_stop_cmd(ar, vdev_id);
+ if (ret)
+ ath11k_warn(ar->ab,
+ "failed to stopt 11d scan vdev %d ret: %d\n",
+ vdev_id, ret);
+ else
+ ar->vdev_id_11d_scan = ATH11K_11D_INVALID_VDEV_ID;
+ }
+ mutex_unlock(&ar->ab->vdev_id_11d_lock);
+}
+
+void ath11k_mac_11d_scan_stop_all(struct ath11k_base *ab)
+{
+ struct ath11k *ar;
+ struct ath11k_pdev *pdev;
+ int i;
+
+ ath11k_dbg(ab, ATH11K_DBG_MAC, "mac stop soc 11d scan\n");
+
+ for (i = 0; i < ab->num_radios; i++) {
+ pdev = &ab->pdevs[i];
+ ar = pdev->ar;
+
+ ath11k_mac_11d_scan_stop(ar);
+ }
+}
+
static int ath11k_mac_op_add_interface(struct ieee80211_hw *hw,
struct ieee80211_vif *vif)
{
@@ -5484,7 +6141,7 @@ static int ath11k_mac_op_add_interface(struct ieee80211_hw *hw,
u32 param_id, param_value;
u16 nss;
int i;
- int ret;
+ int ret, fbret;
int bit;
vif->driver_flags |= IEEE80211_VIF_SUPPORTS_UAPSD;
@@ -5498,9 +6155,9 @@ static int ath11k_mac_op_add_interface(struct ieee80211_hw *hw,
goto err;
}
- if (ar->num_created_vdevs > (TARGET_NUM_VDEVS - 1)) {
+ if (ar->num_created_vdevs > (TARGET_NUM_VDEVS(ab) - 1)) {
ath11k_warn(ab, "failed to create vdev %u, reached max vdev limit %d\n",
- ar->num_created_vdevs, TARGET_NUM_VDEVS);
+ ar->num_created_vdevs, TARGET_NUM_VDEVS(ab));
ret = -EBUSY;
goto err;
}
@@ -5606,6 +6263,8 @@ static int ath11k_mac_op_add_interface(struct ieee80211_hw *hw,
arvif->vdev_id, ret);
goto err_peer_del;
}
+
+ ath11k_mac_11d_scan_stop_all(ar->ab);
break;
case WMI_VDEV_TYPE_STA:
param_id = WMI_STA_PS_PARAM_RX_WAKE_POLICY;
@@ -5638,12 +6297,16 @@ static int ath11k_mac_op_add_interface(struct ieee80211_hw *hw,
goto err_peer_del;
}
- ret = ath11k_wmi_pdev_set_ps_mode(ar, arvif->vdev_id, false);
+ ret = ath11k_wmi_pdev_set_ps_mode(ar, arvif->vdev_id,
+ WMI_STA_PS_MODE_DISABLED);
if (ret) {
ath11k_warn(ar->ab, "failed to disable vdev %d ps mode: %d\n",
arvif->vdev_id, ret);
goto err_peer_del;
}
+
+ ath11k_mac_11d_scan_start(ar, arvif->vdev_id, true);
+
break;
case WMI_VDEV_TYPE_MONITOR:
set_bit(ATH11K_FLAG_MONITOR_VDEV_CREATED, &ar->monitor_flags);
@@ -5686,17 +6349,17 @@ static int ath11k_mac_op_add_interface(struct ieee80211_hw *hw,
if (arvif->vdev_type == WMI_VDEV_TYPE_AP) {
reinit_completion(&ar->peer_delete_done);
- ret = ath11k_wmi_send_peer_delete_cmd(ar, vif->addr,
- arvif->vdev_id);
- if (ret) {
+ fbret = ath11k_wmi_send_peer_delete_cmd(ar, vif->addr,
+ arvif->vdev_id);
+ if (fbret) {
ath11k_warn(ar->ab, "failed to delete peer vdev_id %d addr %pM\n",
arvif->vdev_id, vif->addr);
goto err;
}
- ret = ath11k_wait_for_peer_delete_done(ar, arvif->vdev_id,
- vif->addr);
- if (ret)
+ fbret = ath11k_wait_for_peer_delete_done(ar, arvif->vdev_id,
+ vif->addr);
+ if (fbret)
goto err;
ar->num_peers--;
@@ -5745,6 +6408,9 @@ static void ath11k_mac_op_remove_interface(struct ieee80211_hw *hw,
ath11k_dbg(ab, ATH11K_DBG_MAC, "mac remove interface (vdev %d)\n",
arvif->vdev_id);
+ if (arvif->vdev_type == WMI_VDEV_TYPE_STA)
+ ath11k_mac_11d_scan_stop(ar);
+
if (arvif->vdev_type == WMI_VDEV_TYPE_AP) {
ret = ath11k_peer_delete(ar, arvif->vdev_id, vif->addr);
if (ret)
@@ -5831,7 +6497,6 @@ static void ath11k_mac_op_configure_filter(struct ieee80211_hw *hw,
mutex_lock(&ar->conf_mutex);
- changed_flags &= SUPPORTED_FILTERS;
*total_flags &= SUPPORTED_FILTERS;
ar->filter_flags = *total_flags;
@@ -5969,9 +6634,9 @@ ath11k_mac_vdev_start_restart(struct ath11k_vif *arvif,
ath11k_phymodes[chandef->chan->band][chandef->width];
arg.channel.min_power = 0;
- arg.channel.max_power = chandef->chan->max_power * 2;
- arg.channel.max_reg_power = chandef->chan->max_reg_power * 2;
- arg.channel.max_antenna_gain = chandef->chan->max_antenna_gain * 2;
+ arg.channel.max_power = chandef->chan->max_power;
+ arg.channel.max_reg_power = chandef->chan->max_reg_power;
+ arg.channel.max_antenna_gain = chandef->chan->max_antenna_gain;
arg.pref_tx_streams = ar->num_tx_chains;
arg.pref_rx_streams = ar->num_rx_chains;
@@ -6159,37 +6824,7 @@ ath11k_mac_update_vif_chan(struct ath11k *ar,
lockdep_assert_held(&ar->conf_mutex);
- for (i = 0; i < n_vifs; i++) {
- arvif = (void *)vifs[i].vif->drv_priv;
-
- if (vifs[i].vif->type == NL80211_IFTYPE_MONITOR)
- monitor_vif = true;
-
- ath11k_dbg(ab, ATH11K_DBG_MAC,
- "mac chanctx switch vdev_id %i freq %u->%u width %d->%d\n",
- arvif->vdev_id,
- vifs[i].old_ctx->def.chan->center_freq,
- vifs[i].new_ctx->def.chan->center_freq,
- vifs[i].old_ctx->def.width,
- vifs[i].new_ctx->def.width);
-
- if (WARN_ON(!arvif->is_started))
- continue;
-
- if (WARN_ON(!arvif->is_up))
- continue;
-
- ret = ath11k_wmi_vdev_down(ar, arvif->vdev_id);
- if (ret) {
- ath11k_warn(ab, "failed to down vdev %d: %d\n",
- arvif->vdev_id, ret);
- continue;
- }
-
- ar->num_started_vdevs--;
- }
-
- /* All relevant vdevs are downed and associated channel resources
+ /* Associated channel resources of all relevant vdevs
* should be available for the channel switch now.
*/
@@ -6468,6 +7103,19 @@ ath11k_mac_op_unassign_vif_chanctx(struct ieee80211_hw *hw,
arvif->is_started = false;
if (ab->hw_params.vdev_start_delay &&
+ arvif->vdev_type == WMI_VDEV_TYPE_STA) {
+ ret = ath11k_peer_delete(ar, arvif->vdev_id, arvif->bssid);
+ if (ret)
+ ath11k_warn(ar->ab,
+ "failed to delete peer %pM for vdev %d: %d\n",
+ arvif->bssid, arvif->vdev_id, ret);
+ else
+ ath11k_dbg(ar->ab, ATH11K_DBG_MAC,
+ "mac removed peer %pM vdev %d after vdev stop\n",
+ arvif->bssid, arvif->vdev_id);
+ }
+
+ if (ab->hw_params.vdev_start_delay &&
arvif->vdev_type == WMI_VDEV_TYPE_MONITOR)
ath11k_wmi_vdev_down(ar, arvif->vdev_id);
@@ -6481,6 +7129,9 @@ ath11k_mac_op_unassign_vif_chanctx(struct ieee80211_hw *hw,
ret);
}
+ if (arvif->vdev_type == WMI_VDEV_TYPE_STA)
+ ath11k_mac_11d_scan_start(ar, arvif->vdev_id, false);
+
mutex_unlock(&ar->conf_mutex);
}
@@ -6567,6 +7218,17 @@ static void ath11k_mac_op_flush(struct ieee80211_hw *hw, struct ieee80211_vif *v
ATH11K_FLUSH_TIMEOUT);
if (time_left == 0)
ath11k_warn(ar->ab, "failed to flush transmit queue %ld\n", time_left);
+
+ time_left = wait_event_timeout(ar->txmgmt_empty_waitq,
+ (atomic_read(&ar->num_pending_mgmt_tx) == 0),
+ ATH11K_FLUSH_TIMEOUT);
+ if (time_left == 0)
+ ath11k_warn(ar->ab, "failed to flush mgmt transmit queue %ld\n",
+ time_left);
+
+ ath11k_dbg(ar->ab, ATH11K_DBG_MAC,
+ "mac mgmt tx flush mgmt pending %d\n",
+ atomic_read(&ar->num_pending_mgmt_tx));
}
static int
@@ -7264,12 +7926,45 @@ static int ath11k_mac_op_get_survey(struct ieee80211_hw *hw, int idx,
return ret;
}
+static void ath11k_mac_put_chain_rssi(struct station_info *sinfo,
+ struct ath11k_sta *arsta,
+ char *pre,
+ bool clear)
+{
+ struct ath11k *ar = arsta->arvif->ar;
+ int i;
+ s8 rssi;
+
+ for (i = 0; i < ARRAY_SIZE(sinfo->chain_signal); i++) {
+ sinfo->chains &= ~BIT(i);
+ rssi = arsta->chain_signal[i];
+ if (clear)
+ arsta->chain_signal[i] = ATH11K_INVALID_RSSI_FULL;
+
+ ath11k_dbg(ar->ab, ATH11K_DBG_MAC,
+ "mac sta statistics %s rssi[%d] %d\n", pre, i, rssi);
+
+ if (rssi != ATH11K_DEFAULT_NOISE_FLOOR &&
+ rssi != ATH11K_INVALID_RSSI_FULL &&
+ rssi != ATH11K_INVALID_RSSI_EMPTY &&
+ rssi != 0) {
+ sinfo->chain_signal[i] = rssi;
+ sinfo->chains |= BIT(i);
+ sinfo->filled |= BIT_ULL(NL80211_STA_INFO_CHAIN_SIGNAL);
+ }
+ }
+}
+
static void ath11k_mac_op_sta_statistics(struct ieee80211_hw *hw,
struct ieee80211_vif *vif,
struct ieee80211_sta *sta,
struct station_info *sinfo)
{
struct ath11k_sta *arsta = (struct ath11k_sta *)sta->drv_priv;
+ struct ath11k *ar = arsta->arvif->ar;
+ s8 signal;
+ bool db2dbm = test_bit(WMI_TLV_SERVICE_HW_DB2DBM_CONVERSION_SUPPORT,
+ ar->ab->wmi_ab.svc_map);
sinfo->rx_duration = arsta->rx_duration;
sinfo->filled |= BIT_ULL(NL80211_STA_INFO_RX_DURATION);
@@ -7277,25 +7972,47 @@ static void ath11k_mac_op_sta_statistics(struct ieee80211_hw *hw,
sinfo->tx_duration = arsta->tx_duration;
sinfo->filled |= BIT_ULL(NL80211_STA_INFO_TX_DURATION);
- if (!arsta->txrate.legacy && !arsta->txrate.nss)
- return;
-
- if (arsta->txrate.legacy) {
- sinfo->txrate.legacy = arsta->txrate.legacy;
- } else {
- sinfo->txrate.mcs = arsta->txrate.mcs;
- sinfo->txrate.nss = arsta->txrate.nss;
- sinfo->txrate.bw = arsta->txrate.bw;
- sinfo->txrate.he_gi = arsta->txrate.he_gi;
- sinfo->txrate.he_dcm = arsta->txrate.he_dcm;
- sinfo->txrate.he_ru_alloc = arsta->txrate.he_ru_alloc;
+ if (arsta->txrate.legacy || arsta->txrate.nss) {
+ if (arsta->txrate.legacy) {
+ sinfo->txrate.legacy = arsta->txrate.legacy;
+ } else {
+ sinfo->txrate.mcs = arsta->txrate.mcs;
+ sinfo->txrate.nss = arsta->txrate.nss;
+ sinfo->txrate.bw = arsta->txrate.bw;
+ sinfo->txrate.he_gi = arsta->txrate.he_gi;
+ sinfo->txrate.he_dcm = arsta->txrate.he_dcm;
+ sinfo->txrate.he_ru_alloc = arsta->txrate.he_ru_alloc;
+ }
+ sinfo->txrate.flags = arsta->txrate.flags;
+ sinfo->filled |= BIT_ULL(NL80211_STA_INFO_TX_BITRATE);
}
- sinfo->txrate.flags = arsta->txrate.flags;
- sinfo->filled |= BIT_ULL(NL80211_STA_INFO_TX_BITRATE);
- /* TODO: Use real NF instead of default one. */
- sinfo->signal = arsta->rssi_comb + ATH11K_DEFAULT_NOISE_FLOOR;
- sinfo->filled |= BIT_ULL(NL80211_STA_INFO_SIGNAL);
+ ath11k_mac_put_chain_rssi(sinfo, arsta, "ppdu", false);
+
+ if (!(sinfo->filled & BIT_ULL(NL80211_STA_INFO_CHAIN_SIGNAL)) &&
+ arsta->arvif->vdev_type == WMI_VDEV_TYPE_STA &&
+ ar->ab->hw_params.supports_rssi_stats &&
+ !ath11k_debugfs_get_fw_stats(ar, ar->pdev->pdev_id, 0,
+ WMI_REQUEST_RSSI_PER_CHAIN_STAT)) {
+ ath11k_mac_put_chain_rssi(sinfo, arsta, "fw stats", true);
+ }
+
+ signal = arsta->rssi_comb;
+ if (!signal &&
+ arsta->arvif->vdev_type == WMI_VDEV_TYPE_STA &&
+ ar->ab->hw_params.supports_rssi_stats &&
+ !(ath11k_debugfs_get_fw_stats(ar, ar->pdev->pdev_id, 0,
+ WMI_REQUEST_VDEV_STAT)))
+ signal = arsta->rssi_beacon;
+
+ ath11k_dbg(ar->ab, ATH11K_DBG_MAC,
+ "mac sta statistics db2dbm %u rssi comb %d rssi beacon %d\n",
+ db2dbm, arsta->rssi_comb, arsta->rssi_beacon);
+
+ if (signal) {
+ sinfo->signal = db2dbm ? signal : signal + ATH11K_DEFAULT_NOISE_FLOOR;
+ sinfo->filled |= BIT_ULL(NL80211_STA_INFO_SIGNAL);
+ }
}
static const struct ieee80211_ops ath11k_ops = {
@@ -7633,6 +8350,9 @@ static int __ath11k_mac_register(struct ath11k *ar)
ar->hw->wiphy->interface_modes = ab->hw_params.interface_modes;
+ if (ab->hw_params.single_pdev_only && ar->supports_6ghz)
+ ieee80211_hw_set(ar->hw, SINGLE_SCAN_ON_ALL_BANDS);
+
ieee80211_hw_set(ar->hw, SIGNAL_DBM);
ieee80211_hw_set(ar->hw, SUPPORTS_PS);
ieee80211_hw_set(ar->hw, SUPPORTS_DYNAMIC_PS);
@@ -7672,7 +8392,8 @@ static int __ath11k_mac_register(struct ath11k *ar)
* for each band for a dual band capable radio. It will be tricky to
* handle it when the ht capability different for each band.
*/
- if (ht_cap & WMI_HT_CAP_DYNAMIC_SMPS || ar->supports_6ghz)
+ if (ht_cap & WMI_HT_CAP_DYNAMIC_SMPS ||
+ (ar->supports_6ghz && ab->hw_params.supports_dynamic_smps_6ghz))
ar->hw->wiphy->features |= NL80211_FEATURE_DYNAMIC_SMPS;
ar->hw->wiphy->max_scan_ssids = WLAN_SCAN_PARAMS_MAX_SSID;
@@ -7688,11 +8409,16 @@ static int __ath11k_mac_register(struct ath11k *ar)
ar->hw->wiphy->features |= NL80211_FEATURE_AP_MODE_CHAN_WIDTH_CHANGE |
NL80211_FEATURE_AP_SCAN;
- ar->max_num_stations = TARGET_NUM_STATIONS;
- ar->max_num_peers = TARGET_NUM_PEERS_PDEV;
+ ar->max_num_stations = TARGET_NUM_STATIONS(ab);
+ ar->max_num_peers = TARGET_NUM_PEERS_PDEV(ab);
ar->hw->wiphy->max_ap_assoc_sta = ar->max_num_stations;
+ if (test_bit(WMI_TLV_SERVICE_SPOOF_MAC_SUPPORT, ar->wmi->wmi_ab->svc_map)) {
+ ar->hw->wiphy->features |=
+ NL80211_FEATURE_SCAN_RANDOM_MAC_ADDR;
+ }
+
ar->hw->queues = ATH11K_HW_MAX_QUEUES;
ar->hw->wiphy->tx_queue_len = ATH11K_QUEUE_LEN;
ar->hw->offchannel_tx_hw_queue = ATH11K_HW_MAX_QUEUES - 1;
@@ -7703,6 +8429,9 @@ static int __ath11k_mac_register(struct ath11k *ar)
wiphy_ext_feature_set(ar->hw->wiphy, NL80211_EXT_FEATURE_CQM_RSSI_LIST);
wiphy_ext_feature_set(ar->hw->wiphy, NL80211_EXT_FEATURE_STA_TX_PWR);
+ if (test_bit(WMI_TLV_SERVICE_BSS_COLOR_OFFLOAD, ar->ab->wmi_ab.svc_map))
+ wiphy_ext_feature_set(ar->hw->wiphy,
+ NL80211_EXT_FEATURE_BSS_COLOR);
ar->hw->wiphy->cipher_suites = cipher_suites;
ar->hw->wiphy->n_cipher_suites = ARRAY_SIZE(cipher_suites);
@@ -7784,7 +8513,7 @@ int ath11k_mac_register(struct ath11k_base *ab)
/* Initialize channel counters frequency value in hertz */
ab->cc_freq_hz = IPQ8074_CC_FREQ_HERTZ;
- ab->free_vdev_map = (1LL << (ab->num_radios * TARGET_NUM_VDEVS)) - 1;
+ ab->free_vdev_map = (1LL << (ab->num_radios * TARGET_NUM_VDEVS(ab))) - 1;
for (i = 0; i < ab->num_radios; i++) {
pdev = &ab->pdevs[i];
@@ -7802,6 +8531,8 @@ int ath11k_mac_register(struct ath11k_base *ab)
ret = __ath11k_mac_register(ar);
if (ret)
goto err_cleanup;
+
+ init_waitqueue_head(&ar->txmgmt_empty_waitq);
}
return 0;
@@ -7879,6 +8610,9 @@ int ath11k_mac_allocate(struct ath11k_base *ab)
ar->monitor_vdev_id = -1;
clear_bit(ATH11K_FLAG_MONITOR_VDEV_CREATED, &ar->monitor_flags);
+ ar->vdev_id_11d_scan = ATH11K_11D_INVALID_VDEV_ID;
+ init_completion(&ar->finish_11d_scan);
+ init_completion(&ar->finish_11d_ch_list);
}
return 0;
diff --git a/drivers/net/wireless/ath/ath11k/mac.h b/drivers/net/wireless/ath/ath11k/mac.h
index 254ca4a..0e6c870 100644
--- a/drivers/net/wireless/ath/ath11k/mac.h
+++ b/drivers/net/wireless/ath/ath11k/mac.h
@@ -127,6 +127,13 @@ struct ath11k_generic_iter {
extern const struct htt_rx_ring_tlv_filter ath11k_mac_mon_status_filter_default;
+#define ATH11K_SCAN_11D_INTERVAL 600000
+#define ATH11K_11D_INVALID_VDEV_ID 0xFFFF
+
+void ath11k_mac_11d_scan_start(struct ath11k *ar, u32 vdev_id, bool wait);
+void ath11k_mac_11d_scan_stop(struct ath11k *ar);
+void ath11k_mac_11d_scan_stop_all(struct ath11k_base *ab);
+
void ath11k_mac_destroy(struct ath11k_base *ab);
void ath11k_mac_unregister(struct ath11k_base *ab);
int ath11k_mac_register(struct ath11k_base *ab);
@@ -140,10 +147,16 @@ u8 ath11k_mac_hw_rate_to_idx(const struct ieee80211_supported_band *sband,
void __ath11k_mac_scan_finish(struct ath11k *ar);
void ath11k_mac_scan_finish(struct ath11k *ar);
+int ath11k_mac_rfkill_enable_radio(struct ath11k *ar, bool enable);
+int ath11k_mac_rfkill_config(struct ath11k *ar);
struct ath11k_vif *ath11k_mac_get_arvif(struct ath11k *ar, u32 vdev_id);
struct ath11k_vif *ath11k_mac_get_arvif_by_vdev_id(struct ath11k_base *ab,
u32 vdev_id);
+u8 ath11k_mac_get_target_pdev_id(struct ath11k *ar);
+u8 ath11k_mac_get_target_pdev_id_from_vif(struct ath11k_vif *arvif);
+struct ath11k_vif *ath11k_mac_get_vif_up(struct ath11k_base *ab);
+
struct ath11k *ath11k_mac_get_ar_by_vdev_id(struct ath11k_base *ab, u32 vdev_id);
struct ath11k *ath11k_mac_get_ar_by_pdev_id(struct ath11k_base *ab, u32 pdev_id);
@@ -151,8 +164,12 @@ void ath11k_mac_drain_tx(struct ath11k *ar);
void ath11k_mac_peer_cleanup_all(struct ath11k *ar);
int ath11k_mac_tx_mgmt_pending_free(int buf_id, void *skb, void *ctx);
u8 ath11k_mac_bw_to_mac80211_bw(u8 bw);
+u32 ath11k_mac_he_gi_to_nl80211_he_gi(u8 sgi);
+enum nl80211_he_ru_alloc ath11k_mac_phy_he_ru_to_nl80211_he_ru_alloc(u16 ru_phy);
+enum nl80211_he_ru_alloc ath11k_mac_he_ru_tones_to_nl80211_he_ru_alloc(u16 ru_tones);
enum ath11k_supported_bw ath11k_mac_mac80211_bw_to_ath11k_bw(enum rate_info_bw bw);
enum hal_encrypt_type ath11k_dp_tx_get_encrypt_type(u32 cipher);
void ath11k_mac_handle_beacon(struct ath11k *ar, struct sk_buff *skb);
void ath11k_mac_handle_beacon_miss(struct ath11k *ar, u32 vdev_id);
+void ath11k_mac_bcn_tx_event(struct ath11k_vif *arvif);
#endif
diff --git a/drivers/net/wireless/ath/ath11k/mhi.c b/drivers/net/wireless/ath/ath11k/mhi.c
index 49c0b1a..e4250ba 100644
--- a/drivers/net/wireless/ath/ath11k/mhi.c
+++ b/drivers/net/wireless/ath/ath11k/mhi.c
@@ -3,6 +3,9 @@
#include <linux/msi.h>
#include <linux/pci.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/ioport.h>
#include "core.h"
#include "debug.h"
@@ -248,6 +251,7 @@ static int ath11k_mhi_get_msi(struct ath11k_pci *ab_pci)
u32 user_base_data, base_vector;
int ret, num_vectors, i;
int *irq;
+ unsigned int msi_data;
ret = ath11k_pci_get_user_msi_assignment(ab_pci,
"MHI", &num_vectors,
@@ -262,9 +266,15 @@ static int ath11k_mhi_get_msi(struct ath11k_pci *ab_pci)
if (!irq)
return -ENOMEM;
- for (i = 0; i < num_vectors; i++)
+ for (i = 0; i < num_vectors; i++) {
+ msi_data = base_vector;
+
+ if (test_bit(ATH11K_PCI_FLAG_MULTI_MSI_VECTORS, &ab_pci->flags))
+ msi_data += i;
+
irq[i] = ath11k_pci_get_msi_irq(ab->dev,
- base_vector + i);
+ msi_data);
+ }
ab_pci->mhi_ctrl->irq = irq;
ab_pci->mhi_ctrl->nr_irqs = num_vectors;
@@ -311,6 +321,26 @@ static void ath11k_mhi_op_write_reg(struct mhi_controller *mhi_cntrl,
writel(val, addr);
}
+static int ath11k_mhi_read_addr_from_dt(struct mhi_controller *mhi_ctrl)
+{
+ struct device_node *np;
+ struct resource res;
+ int ret;
+
+ np = of_find_node_by_type(NULL, "memory");
+ if (!np)
+ return -ENOENT;
+
+ ret = of_address_to_resource(np, 0, &res);
+ if (ret)
+ return ret;
+
+ mhi_ctrl->iova_start = res.start + 0x1000000;
+ mhi_ctrl->iova_stop = res.end;
+
+ return 0;
+}
+
int ath11k_mhi_register(struct ath11k_pci *ab_pci)
{
struct ath11k_base *ab = ab_pci->ab;
@@ -339,8 +369,18 @@ int ath11k_mhi_register(struct ath11k_pci *ab_pci)
return ret;
}
- mhi_ctrl->iova_start = 0;
- mhi_ctrl->iova_stop = 0xffffffff;
+ if (!test_bit(ATH11K_PCI_FLAG_MULTI_MSI_VECTORS, &ab_pci->flags))
+ mhi_ctrl->irq_flags = IRQF_SHARED | IRQF_NOBALANCING;
+
+ if (test_bit(ATH11K_FLAG_FIXED_MEM_RGN, &ab->dev_flags)) {
+ ret = ath11k_mhi_read_addr_from_dt(mhi_ctrl);
+ if (ret < 0)
+ return ret;
+ } else {
+ mhi_ctrl->iova_start = 0;
+ mhi_ctrl->iova_stop = 0xFFFFFFFF;
+ }
+
mhi_ctrl->sbl_size = SZ_512K;
mhi_ctrl->seg_len = SZ_512K;
mhi_ctrl->fbc_download = true;
@@ -356,6 +396,7 @@ int ath11k_mhi_register(struct ath11k_pci *ab_pci)
break;
case ATH11K_HW_QCA6390_HW20:
case ATH11K_HW_WCN6855_HW20:
+ case ATH11K_HW_WCN6855_HW21:
ath11k_mhi_config = &ath11k_mhi_config_qca6390;
break;
default:
diff --git a/drivers/net/wireless/ath/ath11k/pci.c b/drivers/net/wireless/ath/ath11k/pci.c
index 3d353e7..d73b522 100644
--- a/drivers/net/wireless/ath/ath11k/pci.c
+++ b/drivers/net/wireless/ath/ath11k/pci.c
@@ -6,6 +6,7 @@
#include <linux/module.h>
#include <linux/msi.h>
#include <linux/pci.h>
+#include <linux/of.h>
#include "pci.h"
#include "core.h"
@@ -16,7 +17,8 @@
#define ATH11K_PCI_BAR_NUM 0
#define ATH11K_PCI_DMA_MASK 32
-#define ATH11K_PCI_IRQ_CE0_OFFSET 3
+#define ATH11K_PCI_IRQ_CE0_OFFSET 3
+#define ATH11K_PCI_IRQ_DP_OFFSET 14
#define WINDOW_ENABLE_BIT 0x40000000
#define WINDOW_REG_ADDRESS 0x310c
@@ -25,7 +27,7 @@
#define WINDOW_RANGE_MASK GENMASK(18, 0)
#define TCSR_SOC_HW_VERSION 0x0224
-#define TCSR_SOC_HW_VERSION_MAJOR_MASK GENMASK(16, 8)
+#define TCSR_SOC_HW_VERSION_MAJOR_MASK GENMASK(11, 8)
#define TCSR_SOC_HW_VERSION_MINOR_MASK GENMASK(7, 0)
/* BAR0 + 4k is always accessible, and no
@@ -76,6 +78,17 @@ static const struct ath11k_msi_config ath11k_msi_config[] = {
},
};
+static const struct ath11k_msi_config msi_config_one_msi = {
+ .total_vectors = 1,
+ .total_users = 4,
+ .users = (struct ath11k_msi_user[]) {
+ { .name = "MHI", .num_vectors = 3, .base_vector = 0 },
+ { .name = "CE", .num_vectors = 1, .base_vector = 0 },
+ { .name = "WAKE", .num_vectors = 1, .base_vector = 0 },
+ { .name = "DP", .num_vectors = 1, .base_vector = 0 },
+ },
+};
+
static const char *irq_name[ATH11K_IRQ_NUM_MAX] = {
"bhi",
"mhi-er0",
@@ -182,7 +195,8 @@ void ath11k_pci_write32(struct ath11k_base *ab, u32 offset, u32 value)
/* for offset beyond BAR + 4K - 32, may
* need to wakeup MHI to access.
*/
- if (test_bit(ATH11K_PCI_FLAG_INIT_DONE, &ab_pci->flags) &&
+ if (ab->hw_params.wakeup_mhi &&
+ test_bit(ATH11K_PCI_FLAG_INIT_DONE, &ab_pci->flags) &&
offset >= ACCESS_ALWAYS_OFF)
mhi_device_get_sync(ab_pci->mhi_ctrl->mhi_dev);
@@ -206,7 +220,8 @@ void ath11k_pci_write32(struct ath11k_base *ab, u32 offset, u32 value)
}
}
- if (test_bit(ATH11K_PCI_FLAG_INIT_DONE, &ab_pci->flags) &&
+ if (ab->hw_params.wakeup_mhi &&
+ test_bit(ATH11K_PCI_FLAG_INIT_DONE, &ab_pci->flags) &&
offset >= ACCESS_ALWAYS_OFF)
mhi_device_put(ab_pci->mhi_ctrl->mhi_dev);
}
@@ -219,7 +234,8 @@ u32 ath11k_pci_read32(struct ath11k_base *ab, u32 offset)
/* for offset beyond BAR + 4K - 32, may
* need to wakeup MHI to access.
*/
- if (test_bit(ATH11K_PCI_FLAG_INIT_DONE, &ab_pci->flags) &&
+ if (ab->hw_params.wakeup_mhi &&
+ test_bit(ATH11K_PCI_FLAG_INIT_DONE, &ab_pci->flags) &&
offset >= ACCESS_ALWAYS_OFF)
mhi_device_get_sync(ab_pci->mhi_ctrl->mhi_dev);
@@ -243,7 +259,8 @@ u32 ath11k_pci_read32(struct ath11k_base *ab, u32 offset)
}
}
- if (test_bit(ATH11K_PCI_FLAG_INIT_DONE, &ab_pci->flags) &&
+ if (ab->hw_params.wakeup_mhi &&
+ test_bit(ATH11K_PCI_FLAG_INIT_DONE, &ab_pci->flags) &&
offset >= ACCESS_ALWAYS_OFF)
mhi_device_put(ab_pci->mhi_ctrl->mhi_dev);
@@ -481,11 +498,11 @@ int ath11k_pci_get_user_msi_assignment(struct ath11k_pci *ab_pci, char *user_nam
for (idx = 0; idx < msi_config->total_users; idx++) {
if (strcmp(user_name, msi_config->users[idx].name) == 0) {
*num_vectors = msi_config->users[idx].num_vectors;
- *user_base_data = msi_config->users[idx].base_vector
- + ab_pci->msi_ep_base_data;
- *base_vector = msi_config->users[idx].base_vector;
+ *base_vector = msi_config->users[idx].base_vector;
+ *user_base_data = *base_vector + ab_pci->msi_ep_base_data;
- ath11k_dbg(ab, ATH11K_DBG_PCI, "Assign MSI to user: %s, num_vectors: %d, user_base_data: %u, base_vector: %u\n",
+ ath11k_dbg(ab, ATH11K_DBG_PCI,
+ "Assign MSI to user: %s, num_vectors: %d, user_base_data: %u, base_vector: %u\n",
user_name, *num_vectors, *user_base_data,
*base_vector);
@@ -556,16 +573,30 @@ static void ath11k_pci_free_irq(struct ath11k_base *ab)
static void ath11k_pci_ce_irq_enable(struct ath11k_base *ab, u16 ce_id)
{
+ struct ath11k_pci *ab_pci = ath11k_pci_priv(ab);
u32 irq_idx;
+ /* In case of one MSI vector, we handle irq enable/disable in a
+ * uniform way since we only have one irq
+ */
+ if (!test_bit(ATH11K_PCI_FLAG_MULTI_MSI_VECTORS, &ab_pci->flags))
+ return;
+
irq_idx = ATH11K_PCI_IRQ_CE0_OFFSET + ce_id;
enable_irq(ab->irq_num[irq_idx]);
}
static void ath11k_pci_ce_irq_disable(struct ath11k_base *ab, u16 ce_id)
{
+ struct ath11k_pci *ab_pci = ath11k_pci_priv(ab);
u32 irq_idx;
+ /* In case of one MSI vector, we handle irq enable/disable in a
+ * uniform way since we only have one irq
+ */
+ if (!test_bit(ATH11K_PCI_FLAG_MULTI_MSI_VECTORS, &ab_pci->flags))
+ return;
+
irq_idx = ATH11K_PCI_IRQ_CE0_OFFSET + ce_id;
disable_irq_nosync(ab->irq_num[irq_idx]);
}
@@ -574,6 +605,8 @@ static void ath11k_pci_ce_irqs_disable(struct ath11k_base *ab)
{
int i;
+ clear_bit(ATH11K_FLAG_CE_IRQ_ENABLED, &ab->dev_flags);
+
for (i = 0; i < ab->hw_params.ce_count; i++) {
if (ath11k_ce_get_attr_flags(ab, i) & CE_ATTR_DIS_INTR)
continue;
@@ -598,20 +631,27 @@ static void ath11k_pci_sync_ce_irqs(struct ath11k_base *ab)
static void ath11k_pci_ce_tasklet(struct tasklet_struct *t)
{
struct ath11k_ce_pipe *ce_pipe = from_tasklet(ce_pipe, t, intr_tq);
+ int irq_idx = ATH11K_PCI_IRQ_CE0_OFFSET + ce_pipe->pipe_num;
ath11k_ce_per_engine_service(ce_pipe->ab, ce_pipe->pipe_num);
- ath11k_pci_ce_irq_enable(ce_pipe->ab, ce_pipe->pipe_num);
+ enable_irq(ce_pipe->ab->irq_num[irq_idx]);
}
static irqreturn_t ath11k_pci_ce_interrupt_handler(int irq, void *arg)
{
struct ath11k_ce_pipe *ce_pipe = arg;
+ struct ath11k_base *ab = ce_pipe->ab;
+ int irq_idx = ATH11K_PCI_IRQ_CE0_OFFSET + ce_pipe->pipe_num;
+
+ if (!test_bit(ATH11K_FLAG_CE_IRQ_ENABLED, &ab->dev_flags))
+ return IRQ_HANDLED;
/* last interrupt received for this CE */
ce_pipe->timestamp = jiffies;
- ath11k_pci_ce_irq_disable(ce_pipe->ab, ce_pipe->pipe_num);
+ disable_irq_nosync(ab->irq_num[irq_idx]);
+
tasklet_schedule(&ce_pipe->intr_tq);
return IRQ_HANDLED;
@@ -619,8 +659,15 @@ static irqreturn_t ath11k_pci_ce_interrupt_handler(int irq, void *arg)
static void ath11k_pci_ext_grp_disable(struct ath11k_ext_irq_grp *irq_grp)
{
+ struct ath11k_pci *ab_pci = ath11k_pci_priv(irq_grp->ab);
int i;
+ /* In case of one MSI vector, we handle irq enable/disable
+ * in a uniform way since we only have one irq
+ */
+ if (!test_bit(ATH11K_PCI_FLAG_MULTI_MSI_VECTORS, &ab_pci->flags))
+ return;
+
for (i = 0; i < irq_grp->num_irq; i++)
disable_irq_nosync(irq_grp->ab->irq_num[irq_grp->irqs[i]]);
}
@@ -629,20 +676,32 @@ static void __ath11k_pci_ext_irq_disable(struct ath11k_base *sc)
{
int i;
+ clear_bit(ATH11K_FLAG_EXT_IRQ_ENABLED, &sc->dev_flags);
+
for (i = 0; i < ATH11K_EXT_IRQ_GRP_NUM_MAX; i++) {
struct ath11k_ext_irq_grp *irq_grp = &sc->ext_irq_grp[i];
ath11k_pci_ext_grp_disable(irq_grp);
- napi_synchronize(&irq_grp->napi);
- napi_disable(&irq_grp->napi);
+ if (irq_grp->napi_enabled) {
+ napi_synchronize(&irq_grp->napi);
+ napi_disable(&irq_grp->napi);
+ irq_grp->napi_enabled = false;
+ }
}
}
static void ath11k_pci_ext_grp_enable(struct ath11k_ext_irq_grp *irq_grp)
{
+ struct ath11k_pci *ab_pci = ath11k_pci_priv(irq_grp->ab);
int i;
+ /* In case of one MSI vector, we handle irq enable/disable in a
+ * uniform way since we only have one irq
+ */
+ if (!test_bit(ATH11K_PCI_FLAG_MULTI_MSI_VECTORS, &ab_pci->flags))
+ return;
+
for (i = 0; i < irq_grp->num_irq; i++)
enable_irq(irq_grp->ab->irq_num[irq_grp->irqs[i]]);
}
@@ -651,10 +710,15 @@ static void ath11k_pci_ext_irq_enable(struct ath11k_base *ab)
{
int i;
+ set_bit(ATH11K_FLAG_EXT_IRQ_ENABLED, &ab->dev_flags);
+
for (i = 0; i < ATH11K_EXT_IRQ_GRP_NUM_MAX; i++) {
struct ath11k_ext_irq_grp *irq_grp = &ab->ext_irq_grp[i];
- napi_enable(&irq_grp->napi);
+ if (!irq_grp->napi_enabled) {
+ napi_enable(&irq_grp->napi);
+ irq_grp->napi_enabled = true;
+ }
ath11k_pci_ext_grp_enable(irq_grp);
}
}
@@ -686,11 +750,13 @@ static int ath11k_pci_ext_grp_napi_poll(struct napi_struct *napi, int budget)
napi);
struct ath11k_base *ab = irq_grp->ab;
int work_done;
+ int i;
work_done = ath11k_dp_service_srng(ab, irq_grp, budget);
if (work_done < budget) {
napi_complete_done(napi, work_done);
- ath11k_pci_ext_grp_enable(irq_grp);
+ for (i = 0; i < irq_grp->num_irq; i++)
+ enable_irq(irq_grp->ab->irq_num[irq_grp->irqs[i]]);
}
if (work_done > budget)
@@ -702,13 +768,19 @@ static int ath11k_pci_ext_grp_napi_poll(struct napi_struct *napi, int budget)
static irqreturn_t ath11k_pci_ext_interrupt_handler(int irq, void *arg)
{
struct ath11k_ext_irq_grp *irq_grp = arg;
+ struct ath11k_base *ab = irq_grp->ab;
+ int i;
+
+ if (!test_bit(ATH11K_FLAG_EXT_IRQ_ENABLED, &ab->dev_flags))
+ return IRQ_HANDLED;
ath11k_dbg(irq_grp->ab, ATH11K_DBG_PCI, "ext irq:%d\n", irq);
/* last interrupt received for this group */
irq_grp->timestamp = jiffies;
- ath11k_pci_ext_grp_disable(irq_grp);
+ for (i = 0; i < irq_grp->num_irq; i++)
+ disable_irq_nosync(irq_grp->ab->irq_num[irq_grp->irqs[i]]);
napi_schedule(&irq_grp->napi);
@@ -717,10 +789,10 @@ static irqreturn_t ath11k_pci_ext_interrupt_handler(int irq, void *arg)
static int ath11k_pci_ext_irq_config(struct ath11k_base *ab)
{
+ struct ath11k_pci *ab_pci = ath11k_pci_priv(ab);
int i, j, ret, num_vectors = 0;
- u32 user_base_data = 0, base_vector = 0, base_idx;
+ u32 user_base_data = 0, base_vector = 0;
- base_idx = ATH11K_PCI_IRQ_CE0_OFFSET + CE_COUNT_MAX;
ret = ath11k_pci_get_user_msi_assignment(ath11k_pci_priv(ab), "DP",
&num_vectors,
&user_base_data,
@@ -750,7 +822,7 @@ static int ath11k_pci_ext_irq_config(struct ath11k_base *ab)
}
irq_grp->num_irq = num_irq;
- irq_grp->irqs[0] = base_idx + i;
+ irq_grp->irqs[0] = ATH11K_PCI_IRQ_DP_OFFSET + i;
for (j = 0; j < irq_grp->num_irq; j++) {
int irq_idx = irq_grp->irqs[j];
@@ -764,23 +836,32 @@ static int ath11k_pci_ext_irq_config(struct ath11k_base *ab)
irq_set_status_flags(irq, IRQ_DISABLE_UNLAZY);
ret = request_irq(irq, ath11k_pci_ext_interrupt_handler,
- IRQF_SHARED,
+ ab_pci->irq_flags,
"DP_EXT_IRQ", irq_grp);
if (ret) {
ath11k_err(ab, "failed request irq %d: %d\n",
vector, ret);
return ret;
}
-
- disable_irq_nosync(ab->irq_num[irq_idx]);
}
+ ath11k_pci_ext_grp_disable(irq_grp);
}
return 0;
}
+static int ath11k_pci_set_irq_affinity_hint(struct ath11k_pci *ab_pci,
+ const struct cpumask *m)
+{
+ if (test_bit(ATH11K_PCI_FLAG_MULTI_MSI_VECTORS, &ab_pci->flags))
+ return 0;
+
+ return irq_set_affinity_hint(ab_pci->pdev->irq, m);
+}
+
static int ath11k_pci_config_irq(struct ath11k_base *ab)
{
+ struct ath11k_pci *ab_pci = ath11k_pci_priv(ab);
struct ath11k_ce_pipe *ce_pipe;
u32 msi_data_start;
u32 msi_data_count, msi_data_idx;
@@ -794,6 +875,12 @@ static int ath11k_pci_config_irq(struct ath11k_base *ab)
if (ret)
return ret;
+ ret = ath11k_pci_set_irq_affinity_hint(ab_pci, cpumask_of(0));
+ if (ret) {
+ ath11k_err(ab, "failed to set irq affinity %d\n", ret);
+ return ret;
+ }
+
/* Configure CE irqs */
for (i = 0, msi_data_idx = 0; i < ab->hw_params.ce_count; i++) {
if (ath11k_ce_get_attr_flags(ab, i) & CE_ATTR_DIS_INTR)
@@ -808,12 +895,12 @@ static int ath11k_pci_config_irq(struct ath11k_base *ab)
tasklet_setup(&ce_pipe->intr_tq, ath11k_pci_ce_tasklet);
ret = request_irq(irq, ath11k_pci_ce_interrupt_handler,
- IRQF_SHARED, irq_name[irq_idx],
+ ab_pci->irq_flags, irq_name[irq_idx],
ce_pipe);
if (ret) {
ath11k_err(ab, "failed to request irq %d: %d\n",
irq_idx, ret);
- return ret;
+ goto err_irq_affinity_cleanup;
}
ab->irq_num[irq_idx] = irq;
@@ -824,9 +911,13 @@ static int ath11k_pci_config_irq(struct ath11k_base *ab)
ret = ath11k_pci_ext_irq_config(ab);
if (ret)
- return ret;
+ goto err_irq_affinity_cleanup;
return 0;
+
+err_irq_affinity_cleanup:
+ ath11k_pci_set_irq_affinity_hint(ab_pci, NULL);
+ return ret;
}
static void ath11k_pci_init_qmi_ce_config(struct ath11k_base *ab)
@@ -848,6 +939,8 @@ static void ath11k_pci_ce_irqs_enable(struct ath11k_base *ab)
{
int i;
+ set_bit(ATH11K_FLAG_CE_IRQ_ENABLED, &ab->dev_flags);
+
for (i = 0; i < ab->hw_params.ce_count; i++) {
if (ath11k_ce_get_attr_flags(ab, i) & CE_ATTR_DIS_INTR)
continue;
@@ -892,15 +985,25 @@ static int ath11k_pci_alloc_msi(struct ath11k_pci *ab_pci)
msi_config->total_vectors,
msi_config->total_vectors,
PCI_IRQ_MSI);
- if (num_vectors != msi_config->total_vectors) {
- ath11k_err(ab, "failed to get %d MSI vectors, only %d available",
- msi_config->total_vectors, num_vectors);
-
- if (num_vectors >= 0)
- return -EINVAL;
- else
- return num_vectors;
+ if (num_vectors == msi_config->total_vectors) {
+ set_bit(ATH11K_PCI_FLAG_MULTI_MSI_VECTORS, &ab_pci->flags);
+ ab_pci->irq_flags = IRQF_SHARED;
+ } else {
+ num_vectors = pci_alloc_irq_vectors(ab_pci->pdev,
+ 1,
+ 1,
+ PCI_IRQ_MSI);
+ if (num_vectors < 0) {
+ ret = -EINVAL;
+ goto reset_msi_config;
+ }
+ clear_bit(ATH11K_PCI_FLAG_MULTI_MSI_VECTORS, &ab_pci->flags);
+ ab_pci->msi_config = &msi_config_one_msi;
+ ab_pci->irq_flags = IRQF_SHARED | IRQF_NOBALANCING;
+ ath11k_dbg(ab, ATH11K_DBG_PCI, "request MSI one vector\n");
}
+ ath11k_info(ab, "MSI vectors: %d\n", num_vectors);
+
ath11k_pci_msi_disable(ab_pci);
msi_desc = irq_get_msi_desc(ab_pci->pdev->irq);
@@ -921,6 +1024,7 @@ static int ath11k_pci_alloc_msi(struct ath11k_pci *ab_pci)
free_msi_vector:
pci_free_irq_vectors(ab_pci->pdev);
+reset_msi_config:
return ret;
}
@@ -929,6 +1033,25 @@ static void ath11k_pci_free_msi(struct ath11k_pci *ab_pci)
pci_free_irq_vectors(ab_pci->pdev);
}
+static int ath11k_pci_config_msi_data(struct ath11k_pci *ab_pci)
+{
+ struct msi_desc *msi_desc;
+
+ msi_desc = irq_get_msi_desc(ab_pci->pdev->irq);
+ if (!msi_desc) {
+ ath11k_err(ab_pci->ab, "msi_desc is NULL!\n");
+ pci_free_irq_vectors(ab_pci->pdev);
+ return -EINVAL;
+ }
+
+ ab_pci->msi_ep_base_data = msi_desc->msg.data;
+
+ ath11k_dbg(ab_pci->ab, ATH11K_DBG_PCI, "pci after request_irq msi_ep_base_data %d\n",
+ ab_pci->msi_ep_base_data);
+
+ return 0;
+}
+
static int ath11k_pci_claim(struct ath11k_pci *ab_pci, struct pci_dev *pdev)
{
struct ath11k_base *ab = ab_pci->ab;
@@ -1126,7 +1249,13 @@ static int ath11k_pci_start(struct ath11k_base *ab)
set_bit(ATH11K_PCI_FLAG_INIT_DONE, &ab_pci->flags);
- ath11k_pci_aspm_restore(ab_pci);
+ /* TODO: for now don't restore ASPM in case of single MSI
+ * vector as MHI register reading in M2 causes system hang.
+ */
+ if (test_bit(ATH11K_PCI_FLAG_MULTI_MSI_VECTORS, &ab_pci->flags))
+ ath11k_pci_aspm_restore(ab_pci);
+ else
+ ath11k_info(ab, "leaving PCI ASPM disabled to avoid MHI M2 problems\n");
ath11k_pci_ce_irqs_enable(ab);
ath11k_ce_rx_post_buf(ab);
@@ -1225,7 +1354,7 @@ static int ath11k_pci_probe(struct pci_dev *pdev,
{
struct ath11k_base *ab;
struct ath11k_pci *ab_pci;
- u32 soc_hw_version_major, soc_hw_version_minor;
+ u32 soc_hw_version_major, soc_hw_version_minor, addr;
int ret;
ab = ath11k_core_alloc(&pdev->dev, sizeof(*ab_pci), ATH11K_BUS_PCI,
@@ -1245,12 +1374,29 @@ static int ath11k_pci_probe(struct pci_dev *pdev,
pci_set_drvdata(pdev, ab);
spin_lock_init(&ab_pci->window_lock);
+ /* Set fixed_mem_region to true for platforms support reserved memory
+ * from DT. If memory is reserved from DT for FW, ath11k driver need not
+ * allocate memory.
+ */
+ ret = of_property_read_u32(ab->dev->of_node, "memory-region", &addr);
+ if (!ret)
+ set_bit(ATH11K_FLAG_FIXED_MEM_RGN, &ab->dev_flags);
+
ret = ath11k_pci_claim(ab_pci, pdev);
if (ret) {
ath11k_err(ab, "failed to claim device: %d\n", ret);
goto err_free_core;
}
+ ath11k_dbg(ab, ATH11K_DBG_BOOT, "pci probe %04x:%04x %04x:%04x\n",
+ pdev->vendor, pdev->device,
+ pdev->subsystem_vendor, pdev->subsystem_device);
+
+ ab->id.vendor = pdev->vendor;
+ ab->id.device = pdev->device;
+ ab->id.subsystem_vendor = pdev->subsystem_vendor;
+ ab->id.subsystem_device = pdev->subsystem_device;
+
switch (pci_dev->device) {
case QCA6390_DEVICE_ID:
ath11k_pci_read_hw_version(ab, &soc_hw_version_major,
@@ -1273,13 +1419,26 @@ static int ath11k_pci_probe(struct pci_dev *pdev,
ab->hw_rev = ATH11K_HW_QCN9074_HW10;
break;
case WCN6855_DEVICE_ID:
+ ab->id.bdf_search = ATH11K_BDF_SEARCH_BUS_AND_BOARD;
ath11k_pci_read_hw_version(ab, &soc_hw_version_major,
&soc_hw_version_minor);
switch (soc_hw_version_major) {
case 2:
- ab->hw_rev = ATH11K_HW_WCN6855_HW20;
+ switch (soc_hw_version_minor) {
+ case 0x00:
+ case 0x01:
+ ab->hw_rev = ATH11K_HW_WCN6855_HW20;
+ break;
+ case 0x10:
+ case 0x11:
+ ab->hw_rev = ATH11K_HW_WCN6855_HW21;
+ break;
+ default:
+ goto unsupported_wcn6855_soc;
+ }
break;
default:
+unsupported_wcn6855_soc:
dev_err(&pdev->dev, "Unsupported WCN6855 SOC hardware version: %d %d\n",
soc_hw_version_major, soc_hw_version_minor);
ret = -EOPNOTSUPP;
@@ -1328,6 +1487,17 @@ static int ath11k_pci_probe(struct pci_dev *pdev,
goto err_ce_free;
}
+ /* kernel may allocate a dummy vector before request_irq and
+ * then allocate a real vector when request_irq is called.
+ * So get msi_data here again to avoid spurious interrupt
+ * as msi_data will configured to srngs.
+ */
+ ret = ath11k_pci_config_msi_data(ab_pci);
+ if (ret) {
+ ath11k_err(ab, "failed to config msi_data: %d\n", ret);
+ goto err_free_irq;
+ }
+
ret = ath11k_core_init(ab);
if (ret) {
ath11k_err(ab, "failed to init core: %d\n", ret);
@@ -1364,6 +1534,8 @@ static void ath11k_pci_remove(struct pci_dev *pdev)
struct ath11k_base *ab = pci_get_drvdata(pdev);
struct ath11k_pci *ab_pci = ath11k_pci_priv(ab);
+ ath11k_pci_set_irq_affinity_hint(ab_pci, NULL);
+
if (test_bit(ATH11K_FLAG_QMI_FAIL, &ab->dev_flags)) {
ath11k_pci_power_down(ab);
ath11k_debugfs_soc_destroy(ab);
diff --git a/drivers/net/wireless/ath/ath11k/pci.h b/drivers/net/wireless/ath/ath11k/pci.h
index f3e6458..61d67b2 100644
--- a/drivers/net/wireless/ath/ath11k/pci.h
+++ b/drivers/net/wireless/ath/ath11k/pci.h
@@ -68,6 +68,7 @@ enum ath11k_pci_flags {
ATH11K_PCI_FLAG_INIT_DONE,
ATH11K_PCI_FLAG_IS_MSI_64,
ATH11K_PCI_ASPM_RESTORE,
+ ATH11K_PCI_FLAG_MULTI_MSI_VECTORS,
};
struct ath11k_pci {
@@ -87,6 +88,8 @@ struct ath11k_pci {
/* enum ath11k_pci_flags */
unsigned long flags;
u16 link_ctl;
+
+ unsigned long irq_flags;
};
static inline struct ath11k_pci *ath11k_pci_priv(struct ath11k_base *ab)
diff --git a/drivers/net/wireless/ath/ath11k/peer.h b/drivers/net/wireless/ath/ath11k/peer.h
index 619db00..63fe566 100644
--- a/drivers/net/wireless/ath/ath11k/peer.h
+++ b/drivers/net/wireless/ath/ath11k/peer.h
@@ -28,6 +28,7 @@ struct ath11k_peer {
u8 ucast_keyidx;
u16 sec_type;
u16 sec_type_grp;
+ bool is_authorized;
};
void ath11k_peer_unmap_event(struct ath11k_base *ab, u16 peer_id);
diff --git a/drivers/net/wireless/ath/ath11k/qmi.c b/drivers/net/wireless/ath/ath11k/qmi.c
index fa73118..65d3c6b 100644
--- a/drivers/net/wireless/ath/ath11k/qmi.c
+++ b/drivers/net/wireless/ath/ath11k/qmi.c
@@ -9,6 +9,8 @@
#include "core.h"
#include "debug.h"
#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/ioport.h>
#include <linux/firmware.h>
#define SLEEP_CLOCK_SELECT_INTERNAL_BIT 0x02
@@ -1582,11 +1584,55 @@ static struct qmi_elem_info qmi_wlanfw_cold_boot_cal_done_ind_msg_v01_ei[] = {
},
};
+static struct qmi_elem_info qmi_wlanfw_wlan_ini_req_msg_v01_ei[] = {
+ {
+ .data_type = QMI_OPT_FLAG,
+ .elem_len = 1,
+ .elem_size = sizeof(u8),
+ .array_type = NO_ARRAY,
+ .tlv_type = 0x10,
+ .offset = offsetof(struct qmi_wlanfw_wlan_ini_req_msg_v01,
+ enablefwlog_valid),
+ },
+ {
+ .data_type = QMI_UNSIGNED_1_BYTE,
+ .elem_len = 1,
+ .elem_size = sizeof(u8),
+ .array_type = NO_ARRAY,
+ .tlv_type = 0x10,
+ .offset = offsetof(struct qmi_wlanfw_wlan_ini_req_msg_v01,
+ enablefwlog),
+ },
+ {
+ .data_type = QMI_EOTI,
+ .array_type = NO_ARRAY,
+ .tlv_type = QMI_COMMON_TLV_TYPE,
+ },
+};
+
+static struct qmi_elem_info qmi_wlanfw_wlan_ini_resp_msg_v01_ei[] = {
+ {
+ .data_type = QMI_STRUCT,
+ .elem_len = 1,
+ .elem_size = sizeof(struct qmi_response_type_v01),
+ .array_type = NO_ARRAY,
+ .tlv_type = 0x02,
+ .offset = offsetof(struct qmi_wlanfw_wlan_ini_resp_msg_v01,
+ resp),
+ .ei_array = qmi_response_type_v01_ei,
+ },
+ {
+ .data_type = QMI_EOTI,
+ .array_type = NO_ARRAY,
+ .tlv_type = QMI_COMMON_TLV_TYPE,
+ },
+};
+
static int ath11k_qmi_host_cap_send(struct ath11k_base *ab)
{
struct qmi_wlanfw_host_cap_req_msg_v01 req;
struct qmi_wlanfw_host_cap_resp_msg_v01 resp;
- struct qmi_txn txn = {};
+ struct qmi_txn txn;
int ret = 0;
memset(&req, 0, sizeof(req));
@@ -1640,6 +1686,7 @@ static int ath11k_qmi_host_cap_send(struct ath11k_base *ab)
QMI_WLANFW_HOST_CAP_REQ_MSG_V01_MAX_LEN,
qmi_wlanfw_host_cap_req_msg_v01_ei, &req);
if (ret < 0) {
+ qmi_txn_cancel(&txn);
ath11k_warn(ab, "failed to send host capability request: %d\n", ret);
goto out;
}
@@ -1705,6 +1752,7 @@ static int ath11k_qmi_fw_ind_register_send(struct ath11k_base *ab)
QMI_WLANFW_IND_REGISTER_REQ_MSG_V01_MAX_LEN,
qmi_wlanfw_ind_register_req_msg_v01_ei, req);
if (ret < 0) {
+ qmi_txn_cancel(&txn);
ath11k_warn(ab, "failed to send indication register request: %d\n",
ret);
goto out;
@@ -1734,7 +1782,7 @@ static int ath11k_qmi_respond_fw_mem_request(struct ath11k_base *ab)
{
struct qmi_wlanfw_respond_mem_req_msg_v01 *req;
struct qmi_wlanfw_respond_mem_resp_msg_v01 resp;
- struct qmi_txn txn = {};
+ struct qmi_txn txn;
int ret = 0, i;
bool delayed;
@@ -1749,7 +1797,9 @@ static int ath11k_qmi_respond_fw_mem_request(struct ath11k_base *ab)
* failure to FW and FW will then request mulitple blocks of small
* chunk size memory.
*/
- if (!ab->bus_params.fixed_mem_region && ab->qmi.target_mem_delayed) {
+ if (!(ab->bus_params.fixed_mem_region ||
+ test_bit(ATH11K_FLAG_FIXED_MEM_RGN, &ab->dev_flags)) &&
+ ab->qmi.target_mem_delayed) {
delayed = true;
ath11k_dbg(ab, ATH11K_DBG_QMI, "qmi delays mem_request %d\n",
ab->qmi.mem_seg_count);
@@ -1783,6 +1833,7 @@ static int ath11k_qmi_respond_fw_mem_request(struct ath11k_base *ab)
QMI_WLANFW_RESPOND_MEM_REQ_MSG_V01_MAX_LEN,
qmi_wlanfw_respond_mem_req_msg_v01_ei, req);
if (ret < 0) {
+ qmi_txn_cancel(&txn);
ath11k_warn(ab, "failed to respond qmi memory request: %d\n",
ret);
goto out;
@@ -1815,10 +1866,12 @@ static void ath11k_qmi_free_target_mem_chunk(struct ath11k_base *ab)
{
int i;
- if (ab->bus_params.fixed_mem_region)
- return;
-
for (i = 0; i < ab->qmi.mem_seg_count; i++) {
+ if ((ab->bus_params.fixed_mem_region ||
+ test_bit(ATH11K_FLAG_FIXED_MEM_RGN, &ab->dev_flags)) &&
+ ab->qmi.target_mem[i].iaddr)
+ iounmap(ab->qmi.target_mem[i].iaddr);
+
if (!ab->qmi.target_mem[i].vaddr)
continue;
@@ -1866,10 +1919,44 @@ static int ath11k_qmi_alloc_target_mem_chunk(struct ath11k_base *ab)
static int ath11k_qmi_assign_target_mem_chunk(struct ath11k_base *ab)
{
- int i, idx;
+ struct device *dev = ab->dev;
+ struct device_node *hremote_node = NULL;
+ struct resource res;
+ u32 host_ddr_sz;
+ int i, idx, ret;
for (i = 0, idx = 0; i < ab->qmi.mem_seg_count; i++) {
switch (ab->qmi.target_mem[i].type) {
+ case HOST_DDR_REGION_TYPE:
+ hremote_node = of_parse_phandle(dev->of_node, "memory-region", 0);
+ if (!hremote_node) {
+ ath11k_dbg(ab, ATH11K_DBG_QMI,
+ "qmi fail to get hremote_node\n");
+ return ret;
+ }
+
+ ret = of_address_to_resource(hremote_node, 0, &res);
+ if (ret) {
+ ath11k_dbg(ab, ATH11K_DBG_QMI,
+ "qmi fail to get reg from hremote\n");
+ return ret;
+ }
+
+ if (res.end - res.start + 1 < ab->qmi.target_mem[i].size) {
+ ath11k_dbg(ab, ATH11K_DBG_QMI,
+ "qmi fail to assign memory of sz\n");
+ return -EINVAL;
+ }
+
+ ab->qmi.target_mem[idx].paddr = res.start;
+ ab->qmi.target_mem[idx].iaddr =
+ ioremap(ab->qmi.target_mem[idx].paddr,
+ ab->qmi.target_mem[i].size);
+ ab->qmi.target_mem[idx].size = ab->qmi.target_mem[i].size;
+ host_ddr_sz = ab->qmi.target_mem[i].size;
+ ab->qmi.target_mem[idx].type = ab->qmi.target_mem[i].type;
+ idx++;
+ break;
case BDF_MEM_REGION_TYPE:
ab->qmi.target_mem[idx].paddr = ab->hw_params.bdf_addr;
ab->qmi.target_mem[idx].vaddr = NULL;
@@ -1884,10 +1971,16 @@ static int ath11k_qmi_assign_target_mem_chunk(struct ath11k_base *ab)
}
if (ath11k_cold_boot_cal && ab->hw_params.cold_boot_calib) {
- ab->qmi.target_mem[idx].paddr =
- ATH11K_QMI_CALDB_ADDRESS;
- ab->qmi.target_mem[idx].vaddr =
- (void *)ATH11K_QMI_CALDB_ADDRESS;
+ if (hremote_node) {
+ ab->qmi.target_mem[idx].paddr =
+ res.start + host_ddr_sz;
+ ab->qmi.target_mem[idx].iaddr =
+ ioremap(ab->qmi.target_mem[idx].paddr,
+ ab->qmi.target_mem[i].size);
+ } else {
+ ab->qmi.target_mem[idx].paddr =
+ ATH11K_QMI_CALDB_ADDRESS;
+ }
} else {
ab->qmi.target_mem[idx].paddr = 0;
ab->qmi.target_mem[idx].vaddr = NULL;
@@ -1911,7 +2004,7 @@ static int ath11k_qmi_request_target_cap(struct ath11k_base *ab)
{
struct qmi_wlanfw_cap_req_msg_v01 req;
struct qmi_wlanfw_cap_resp_msg_v01 resp;
- struct qmi_txn txn = {};
+ struct qmi_txn txn;
int ret = 0;
int r;
@@ -1930,6 +2023,7 @@ static int ath11k_qmi_request_target_cap(struct ath11k_base *ab)
QMI_WLANFW_CAP_REQ_MSG_V01_MAX_LEN,
qmi_wlanfw_cap_req_msg_v01_ei, &req);
if (ret < 0) {
+ qmi_txn_cancel(&txn);
ath11k_warn(ab, "failed to send qmi cap request: %d\n",
ret);
goto out;
@@ -2000,7 +2094,7 @@ static int ath11k_qmi_load_file_target_mem(struct ath11k_base *ab,
{
struct qmi_wlanfw_bdf_download_req_msg_v01 *req;
struct qmi_wlanfw_bdf_download_resp_msg_v01 resp;
- struct qmi_txn txn = {};
+ struct qmi_txn txn;
const u8 *temp = data;
void __iomem *bdf_addr = NULL;
int ret;
@@ -2111,7 +2205,8 @@ static int ath11k_qmi_load_file_target_mem(struct ath11k_base *ab,
return ret;
}
-static int ath11k_qmi_load_bdf_qmi(struct ath11k_base *ab)
+static int ath11k_qmi_load_bdf_qmi(struct ath11k_base *ab,
+ bool regdb)
{
struct device *dev = ab->dev;
char filename[ATH11K_QMI_MAX_BDF_FILE_NAME_SIZE];
@@ -2122,13 +2217,21 @@ static int ath11k_qmi_load_bdf_qmi(struct ath11k_base *ab)
const u8 *tmp;
memset(&bd, 0, sizeof(bd));
- ret = ath11k_core_fetch_bdf(ab, &bd);
- if (ret) {
- ath11k_warn(ab, "qmi failed to fetch board file: %d\n", ret);
- goto out;
+
+ if (regdb) {
+ ret = ath11k_core_fetch_regdb(ab, &bd);
+ } else {
+ ret = ath11k_core_fetch_bdf(ab, &bd);
+ if (ret)
+ ath11k_warn(ab, "qmi failed to fetch board file: %d\n", ret);
}
- if (bd.len >= SELFMAG && memcmp(bd.data, ELFMAG, SELFMAG) == 0)
+ if (ret)
+ goto out;
+
+ if (regdb)
+ bdf_type = ATH11K_QMI_BDF_TYPE_REGDB;
+ else if (bd.len >= SELFMAG && memcmp(bd.data, ELFMAG, SELFMAG) == 0)
bdf_type = ATH11K_QMI_BDF_TYPE_ELF;
else
bdf_type = ATH11K_QMI_BDF_TYPE_BIN;
@@ -2143,8 +2246,8 @@ static int ath11k_qmi_load_bdf_qmi(struct ath11k_base *ab)
goto out;
}
- /* QCA6390 does not support cal data, skip it */
- if (bdf_type == ATH11K_QMI_BDF_TYPE_ELF)
+ /* QCA6390/WCN6855 does not support cal data, skip it */
+ if (bdf_type == ATH11K_QMI_BDF_TYPE_ELF || bdf_type == ATH11K_QMI_BDF_TYPE_REGDB)
goto out;
if (ab->qmi.target.eeprom_caldata) {
@@ -2245,7 +2348,7 @@ static int ath11k_qmi_wlanfw_m3_info_send(struct ath11k_base *ab)
struct m3_mem_region *m3_mem = &ab->qmi.m3_mem;
struct qmi_wlanfw_m3_info_req_msg_v01 req;
struct qmi_wlanfw_m3_info_resp_msg_v01 resp;
- struct qmi_txn txn = {};
+ struct qmi_txn txn;
int ret = 0;
memset(&req, 0, sizeof(req));
@@ -2277,6 +2380,7 @@ static int ath11k_qmi_wlanfw_m3_info_send(struct ath11k_base *ab)
QMI_WLANFW_M3_INFO_REQ_MSG_V01_MAX_MSG_LEN,
qmi_wlanfw_m3_info_req_msg_v01_ei, &req);
if (ret < 0) {
+ qmi_txn_cancel(&txn);
ath11k_warn(ab, "failed to send m3 information request: %d\n",
ret);
goto out;
@@ -2303,7 +2407,7 @@ static int ath11k_qmi_wlanfw_mode_send(struct ath11k_base *ab,
{
struct qmi_wlanfw_wlan_mode_req_msg_v01 req;
struct qmi_wlanfw_wlan_mode_resp_msg_v01 resp;
- struct qmi_txn txn = {};
+ struct qmi_txn txn;
int ret = 0;
memset(&req, 0, sizeof(req));
@@ -2325,6 +2429,7 @@ static int ath11k_qmi_wlanfw_mode_send(struct ath11k_base *ab,
QMI_WLANFW_WLAN_MODE_REQ_MSG_V01_MAX_LEN,
qmi_wlanfw_wlan_mode_req_msg_v01_ei, &req);
if (ret < 0) {
+ qmi_txn_cancel(&txn);
ath11k_warn(ab, "failed to send wlan mode request (mode %d): %d\n",
mode, ret);
goto out;
@@ -2358,7 +2463,7 @@ static int ath11k_qmi_wlanfw_wlan_cfg_send(struct ath11k_base *ab)
struct qmi_wlanfw_wlan_cfg_resp_msg_v01 resp;
struct ce_pipe_config *ce_cfg;
struct service_to_pipe *svc_cfg;
- struct qmi_txn txn = {};
+ struct qmi_txn txn;
int ret = 0, pipe_num;
ce_cfg = (struct ce_pipe_config *)ab->qmi.ce_cfg.tgt_ce;
@@ -2419,6 +2524,7 @@ static int ath11k_qmi_wlanfw_wlan_cfg_send(struct ath11k_base *ab)
QMI_WLANFW_WLAN_CFG_REQ_MSG_V01_MAX_LEN,
qmi_wlanfw_wlan_cfg_req_msg_v01_ei, req);
if (ret < 0) {
+ qmi_txn_cancel(&txn);
ath11k_warn(ab, "failed to send wlan config request: %d\n",
ret);
goto out;
@@ -2442,6 +2548,48 @@ static int ath11k_qmi_wlanfw_wlan_cfg_send(struct ath11k_base *ab)
return ret;
}
+static int ath11k_qmi_wlanfw_wlan_ini_send(struct ath11k_base *ab, bool enable)
+{
+ int ret;
+ struct qmi_txn txn;
+ struct qmi_wlanfw_wlan_ini_req_msg_v01 req = {};
+ struct qmi_wlanfw_wlan_ini_resp_msg_v01 resp = {};
+
+ req.enablefwlog_valid = true;
+ req.enablefwlog = enable ? 1 : 0;
+
+ ret = qmi_txn_init(&ab->qmi.handle, &txn,
+ qmi_wlanfw_wlan_ini_resp_msg_v01_ei, &resp);
+ if (ret < 0)
+ goto out;
+
+ ret = qmi_send_request(&ab->qmi.handle, NULL, &txn,
+ QMI_WLANFW_WLAN_INI_REQ_V01,
+ QMI_WLANFW_WLAN_INI_REQ_MSG_V01_MAX_LEN,
+ qmi_wlanfw_wlan_ini_req_msg_v01_ei, &req);
+ if (ret < 0) {
+ ath11k_warn(ab, "qmi failed to send wlan ini request, err = %d\n",
+ ret);
+ qmi_txn_cancel(&txn);
+ goto out;
+ }
+
+ ret = qmi_txn_wait(&txn, msecs_to_jiffies(ATH11K_QMI_WLANFW_TIMEOUT_MS));
+ if (ret < 0) {
+ ath11k_warn(ab, "qmi failed wlan ini request, err = %d\n", ret);
+ goto out;
+ }
+
+ if (resp.resp.result != QMI_RESULT_SUCCESS_V01) {
+ ath11k_warn(ab, "qmi wlan ini request failed, result: %d, err: %d\n",
+ resp.resp.result, resp.resp.error);
+ ret = -EINVAL;
+ }
+
+out:
+ return ret;
+}
+
void ath11k_qmi_firmware_stop(struct ath11k_base *ab)
{
int ret;
@@ -2462,6 +2610,14 @@ int ath11k_qmi_firmware_start(struct ath11k_base *ab,
ath11k_dbg(ab, ATH11K_DBG_QMI, "qmi firmware start\n");
+ if (ab->hw_params.fw_wmi_diag_event) {
+ ret = ath11k_qmi_wlanfw_wlan_ini_send(ab, true);
+ if (ret < 0) {
+ ath11k_warn(ab, "qmi failed to send wlan fw ini:%d\n", ret);
+ return ret;
+ }
+ }
+
ret = ath11k_qmi_wlanfw_wlan_cfg_send(ab);
if (ret < 0) {
ath11k_warn(ab, "qmi failed to send wlan cfg: %d\n", ret);
@@ -2573,7 +2729,10 @@ static int ath11k_qmi_event_load_bdf(struct ath11k_qmi *qmi)
return ret;
}
- ret = ath11k_qmi_load_bdf_qmi(ab);
+ if (ab->hw_params.supports_regdb)
+ ath11k_qmi_load_bdf_qmi(ab, true);
+
+ ret = ath11k_qmi_load_bdf_qmi(ab, false);
if (ret < 0) {
ath11k_warn(ab, "failed to load board data file: %d\n", ret);
return ret;
@@ -2614,7 +2773,8 @@ static void ath11k_qmi_msg_mem_request_cb(struct qmi_handle *qmi_hdl,
msg->mem_seg[i].type, msg->mem_seg[i].size);
}
- if (ab->bus_params.fixed_mem_region) {
+ if (ab->bus_params.fixed_mem_region ||
+ test_bit(ATH11K_FLAG_FIXED_MEM_RGN, &ab->dev_flags)) {
ret = ath11k_qmi_assign_target_mem_chunk(ab);
if (ret) {
ath11k_warn(ab, "failed to assign qmi target memory: %d\n",
@@ -2823,7 +2983,7 @@ int ath11k_qmi_init_service(struct ath11k_base *ab)
memset(&ab->qmi.target_mem, 0, sizeof(struct target_mem_chunk));
ab->qmi.ab = ab;
- ab->qmi.target_mem_mode = ATH11K_QMI_TARGET_MEM_MODE_DEFAULT;
+ ab->qmi.target_mem_mode = ab->hw_params.fw_mem_mode;
ret = qmi_handle_init(&ab->qmi.handle, ATH11K_QMI_RESP_LEN_MAX,
&ath11k_qmi_ops, ath11k_qmi_msg_handlers);
if (ret < 0) {
diff --git a/drivers/net/wireless/ath/ath11k/qmi.h b/drivers/net/wireless/ath/ath11k/qmi.h
index 3bb0f9e..ba2eff4 100644
--- a/drivers/net/wireless/ath/ath11k/qmi.h
+++ b/drivers/net/wireless/ath/ath11k/qmi.h
@@ -34,14 +34,13 @@
#define QMI_WLANFW_MAX_DATA_SIZE_V01 6144
#define ATH11K_FIRMWARE_MODE_OFF 4
-#define ATH11K_QMI_TARGET_MEM_MODE_DEFAULT 0
#define ATH11K_COLD_BOOT_FW_RESET_DELAY (40 * HZ)
struct ath11k_base;
enum ath11k_qmi_file_type {
ATH11K_QMI_FILE_TYPE_BDF_GOLDEN,
- ATH11K_QMI_FILE_TYPE_CALDATA,
+ ATH11K_QMI_FILE_TYPE_CALDATA = 2,
ATH11K_QMI_FILE_TYPE_EEPROM,
ATH11K_QMI_MAX_FILE_TYPE,
};
@@ -49,6 +48,7 @@ enum ath11k_qmi_file_type {
enum ath11k_qmi_bdf_type {
ATH11K_QMI_BDF_TYPE_BIN = 0,
ATH11K_QMI_BDF_TYPE_ELF = 1,
+ ATH11K_QMI_BDF_TYPE_REGDB = 4,
};
enum ath11k_qmi_event_type {
@@ -95,6 +95,7 @@ struct target_mem_chunk {
u32 type;
dma_addr_t paddr;
u32 *vaddr;
+ void __iomem *iaddr;
};
struct target_info {
@@ -427,10 +428,12 @@ struct qmi_wlanfw_m3_info_resp_msg_v01 {
#define QMI_WLANFW_WLAN_MODE_RESP_MSG_V01_MAX_LEN 7
#define QMI_WLANFW_WLAN_CFG_REQ_MSG_V01_MAX_LEN 803
#define QMI_WLANFW_WLAN_CFG_RESP_MSG_V01_MAX_LEN 7
+#define QMI_WLANFW_WLAN_INI_REQ_MSG_V01_MAX_LEN 4
#define QMI_WLANFW_WLAN_MODE_REQ_V01 0x0022
#define QMI_WLANFW_WLAN_MODE_RESP_V01 0x0022
#define QMI_WLANFW_WLAN_CFG_REQ_V01 0x0023
#define QMI_WLANFW_WLAN_CFG_RESP_V01 0x0023
+#define QMI_WLANFW_WLAN_INI_REQ_V01 0x002F
#define QMI_WLANFW_MAX_STR_LEN_V01 16
#define QMI_WLANFW_MAX_NUM_CE_V01 12
#define QMI_WLANFW_MAX_NUM_SVC_V01 24
@@ -472,6 +475,16 @@ struct qmi_wlanfw_wlan_cfg_resp_msg_v01 {
struct qmi_response_type_v01 resp;
};
+struct qmi_wlanfw_wlan_ini_req_msg_v01 {
+ /* Must be set to true if enablefwlog is being passed */
+ u8 enablefwlog_valid;
+ u8 enablefwlog;
+};
+
+struct qmi_wlanfw_wlan_ini_resp_msg_v01 {
+ struct qmi_response_type_v01 resp;
+};
+
int ath11k_qmi_firmware_start(struct ath11k_base *ab,
u32 mode);
void ath11k_qmi_firmware_stop(struct ath11k_base *ab);
diff --git a/drivers/net/wireless/ath/ath11k/reg.c b/drivers/net/wireless/ath/ath11k/reg.c
index a66b5bd..d6575fe 100644
--- a/drivers/net/wireless/ath/ath11k/reg.c
+++ b/drivers/net/wireless/ath/ath11k/reg.c
@@ -2,6 +2,8 @@
/*
* Copyright (c) 2018-2019 The Linux Foundation. All rights reserved.
*/
+#include <linux/rtnetlink.h>
+
#include "core.h"
#include "debug.h"
@@ -86,6 +88,9 @@ ath11k_reg_notifier(struct wiphy *wiphy, struct regulatory_request *request)
if (ret)
ath11k_warn(ar->ab,
"INIT Country code set to fw failed : %d\n", ret);
+
+ ath11k_mac_11d_scan_stop(ar);
+ ar->regdom_set_by_user = true;
}
int ath11k_reg_update_chan_list(struct ath11k *ar)
@@ -179,6 +184,11 @@ int ath11k_reg_update_chan_list(struct ath11k *ar)
ret = ath11k_wmi_send_scan_chan_list_cmd(ar, params);
kfree(params);
+ if (ar->pending_11d) {
+ complete(&ar->finish_11d_ch_list);
+ ar->pending_11d = false;
+ }
+
return ret;
}
@@ -244,8 +254,15 @@ int ath11k_regd_update(struct ath11k *ar)
goto err;
}
+ if (ar->pending_11d)
+ complete(&ar->finish_11d_scan);
+
rtnl_lock();
wiphy_lock(ar->hw->wiphy);
+
+ if (ar->pending_11d)
+ reinit_completion(&ar->finish_11d_ch_list);
+
ret = regulatory_set_wiphy_regd_sync(ar->hw->wiphy, regd_copy);
wiphy_unlock(ar->hw->wiphy);
rtnl_unlock();
@@ -456,6 +473,9 @@ ath11k_reg_adjust_bw(u16 start_freq, u16 end_freq, u16 max_bw)
{
u16 bw;
+ if (end_freq <= start_freq)
+ return 0;
+
bw = end_freq - start_freq;
bw = min_t(u16, bw, max_bw);
@@ -463,8 +483,10 @@ ath11k_reg_adjust_bw(u16 start_freq, u16 end_freq, u16 max_bw)
bw = 80;
else if (bw >= 40 && bw < 80)
bw = 40;
- else if (bw < 40)
+ else if (bw >= 20 && bw < 40)
bw = 20;
+ else
+ bw = 0;
return bw;
}
@@ -488,73 +510,77 @@ ath11k_reg_update_weather_radar_band(struct ath11k_base *ab,
struct cur_reg_rule *reg_rule,
u8 *rule_idx, u32 flags, u16 max_bw)
{
+ u32 start_freq;
u32 end_freq;
u16 bw;
u8 i;
i = *rule_idx;
+ /* there might be situations when even the input rule must be dropped */
+ i--;
+
+ /* frequencies below weather radar */
bw = ath11k_reg_adjust_bw(reg_rule->start_freq,
ETSI_WEATHER_RADAR_BAND_LOW, max_bw);
+ if (bw > 0) {
+ i++;
- ath11k_reg_update_rule(regd->reg_rules + i, reg_rule->start_freq,
- ETSI_WEATHER_RADAR_BAND_LOW, bw,
- reg_rule->ant_gain, reg_rule->reg_power,
- flags);
+ ath11k_reg_update_rule(regd->reg_rules + i,
+ reg_rule->start_freq,
+ ETSI_WEATHER_RADAR_BAND_LOW, bw,
+ reg_rule->ant_gain, reg_rule->reg_power,
+ flags);
- ath11k_dbg(ab, ATH11K_DBG_REG,
- "\t%d. (%d - %d @ %d) (%d, %d) (%d ms) (FLAGS %d)\n",
- i + 1, reg_rule->start_freq, ETSI_WEATHER_RADAR_BAND_LOW,
- bw, reg_rule->ant_gain, reg_rule->reg_power,
- regd->reg_rules[i].dfs_cac_ms,
- flags);
-
- if (reg_rule->end_freq > ETSI_WEATHER_RADAR_BAND_HIGH)
- end_freq = ETSI_WEATHER_RADAR_BAND_HIGH;
- else
- end_freq = reg_rule->end_freq;
-
- bw = ath11k_reg_adjust_bw(ETSI_WEATHER_RADAR_BAND_LOW, end_freq,
- max_bw);
-
- i++;
-
- ath11k_reg_update_rule(regd->reg_rules + i,
- ETSI_WEATHER_RADAR_BAND_LOW, end_freq, bw,
- reg_rule->ant_gain, reg_rule->reg_power,
- flags);
-
- regd->reg_rules[i].dfs_cac_ms = ETSI_WEATHER_RADAR_BAND_CAC_TIMEOUT;
-
- ath11k_dbg(ab, ATH11K_DBG_REG,
- "\t%d. (%d - %d @ %d) (%d, %d) (%d ms) (FLAGS %d)\n",
- i + 1, ETSI_WEATHER_RADAR_BAND_LOW, end_freq,
- bw, reg_rule->ant_gain, reg_rule->reg_power,
- regd->reg_rules[i].dfs_cac_ms,
- flags);
-
- if (end_freq == reg_rule->end_freq) {
- regd->n_reg_rules--;
- *rule_idx = i;
- return;
+ ath11k_dbg(ab, ATH11K_DBG_REG,
+ "\t%d. (%d - %d @ %d) (%d, %d) (%d ms) (FLAGS %d)\n",
+ i + 1, reg_rule->start_freq,
+ ETSI_WEATHER_RADAR_BAND_LOW, bw, reg_rule->ant_gain,
+ reg_rule->reg_power, regd->reg_rules[i].dfs_cac_ms,
+ flags);
}
+ /* weather radar frequencies */
+ start_freq = max_t(u32, reg_rule->start_freq,
+ ETSI_WEATHER_RADAR_BAND_LOW);
+ end_freq = min_t(u32, reg_rule->end_freq, ETSI_WEATHER_RADAR_BAND_HIGH);
+
+ bw = ath11k_reg_adjust_bw(start_freq, end_freq, max_bw);
+ if (bw > 0) {
+ i++;
+
+ ath11k_reg_update_rule(regd->reg_rules + i, start_freq,
+ end_freq, bw, reg_rule->ant_gain,
+ reg_rule->reg_power, flags);
+
+ regd->reg_rules[i].dfs_cac_ms = ETSI_WEATHER_RADAR_BAND_CAC_TIMEOUT;
+
+ ath11k_dbg(ab, ATH11K_DBG_REG,
+ "\t%d. (%d - %d @ %d) (%d, %d) (%d ms) (FLAGS %d)\n",
+ i + 1, start_freq, end_freq, bw,
+ reg_rule->ant_gain, reg_rule->reg_power,
+ regd->reg_rules[i].dfs_cac_ms, flags);
+ }
+
+ /* frequencies above weather radar */
bw = ath11k_reg_adjust_bw(ETSI_WEATHER_RADAR_BAND_HIGH,
reg_rule->end_freq, max_bw);
+ if (bw > 0) {
+ i++;
- i++;
+ ath11k_reg_update_rule(regd->reg_rules + i,
+ ETSI_WEATHER_RADAR_BAND_HIGH,
+ reg_rule->end_freq, bw,
+ reg_rule->ant_gain, reg_rule->reg_power,
+ flags);
- ath11k_reg_update_rule(regd->reg_rules + i, ETSI_WEATHER_RADAR_BAND_HIGH,
- reg_rule->end_freq, bw,
- reg_rule->ant_gain, reg_rule->reg_power,
- flags);
-
- ath11k_dbg(ab, ATH11K_DBG_REG,
- "\t%d. (%d - %d @ %d) (%d, %d) (%d ms) (FLAGS %d)\n",
- i + 1, ETSI_WEATHER_RADAR_BAND_HIGH, reg_rule->end_freq,
- bw, reg_rule->ant_gain, reg_rule->reg_power,
- regd->reg_rules[i].dfs_cac_ms,
- flags);
+ ath11k_dbg(ab, ATH11K_DBG_REG,
+ "\t%d. (%d - %d @ %d) (%d, %d) (%d ms) (FLAGS %d)\n",
+ i + 1, ETSI_WEATHER_RADAR_BAND_HIGH,
+ reg_rule->end_freq, bw, reg_rule->ant_gain,
+ reg_rule->reg_power, regd->reg_rules[i].dfs_cac_ms,
+ flags);
+ }
*rule_idx = i;
}
diff --git a/drivers/net/wireless/ath/ath11k/spectral.c b/drivers/net/wireless/ath/ath11k/spectral.c
index ac4da99..4100cc1 100644
--- a/drivers/net/wireless/ath/ath11k/spectral.c
+++ b/drivers/net/wireless/ath/ath11k/spectral.c
@@ -581,6 +581,7 @@ int ath11k_spectral_process_fft(struct ath11k *ar,
u16 length, freq;
u8 chan_width_mhz, bin_sz;
int ret;
+ u32 check_length;
lockdep_assert_held(&ar->spectral.lock);
@@ -614,6 +615,13 @@ int ath11k_spectral_process_fft(struct ath11k *ar,
return -EINVAL;
}
+ check_length = sizeof(*fft_report) + (num_bins * ab->hw_params.spectral.fft_sz);
+ ret = ath11k_dbring_validate_buffer(ar, data, check_length);
+ if (ret) {
+ ath11k_warn(ar->ab, "found magic value in fft data, dropping\n");
+ return ret;
+ }
+
ret = ath11k_spectral_pull_search(ar, data, &search);
if (ret) {
ath11k_warn(ab, "failed to pull search report %d\n", ret);
@@ -747,6 +755,12 @@ static int ath11k_spectral_process_data(struct ath11k *ar,
goto err;
}
+ ret = ath11k_dbring_validate_buffer(ar, data, tlv_len);
+ if (ret) {
+ ath11k_warn(ar->ab, "found magic value in spectral summary, dropping\n");
+ goto err;
+ }
+
summary = (struct spectral_summary_fft_report *)tlv;
ath11k_spectral_pull_summary(ar, ¶m->meta,
summary, &summ_rpt);
diff --git a/drivers/net/wireless/ath/ath11k/trace.c b/drivers/net/wireless/ath/ath11k/trace.c
index f0cc49b..6620650 100644
--- a/drivers/net/wireless/ath/ath11k/trace.c
+++ b/drivers/net/wireless/ath/ath11k/trace.c
@@ -7,3 +7,4 @@
#define CREATE_TRACE_POINTS
#include "trace.h"
+EXPORT_SYMBOL(__tracepoint_ath11k_log_dbg);
diff --git a/drivers/net/wireless/ath/ath11k/trace.h b/drivers/net/wireless/ath/ath11k/trace.h
index 25d18e9..a02e547 100644
--- a/drivers/net/wireless/ath/ath11k/trace.h
+++ b/drivers/net/wireless/ath/ath11k/trace.h
@@ -14,12 +14,24 @@
#if !defined(CONFIG_ATH11K_TRACING)
#undef TRACE_EVENT
#define TRACE_EVENT(name, proto, ...) \
+static inline void trace_ ## name(proto) {} \
+static inline bool trace_##name##_enabled(void) \
+{ \
+ return false; \
+}
+
+#undef DECLARE_EVENT_CLASS
+#define DECLARE_EVENT_CLASS(...)
+#undef DEFINE_EVENT
+#define DEFINE_EVENT(evt_class, name, proto, ...) \
static inline void trace_ ## name(proto) {}
#endif /* !CONFIG_ATH11K_TRACING || __CHECKER__ */
#undef TRACE_SYSTEM
#define TRACE_SYSTEM ath11k
+#define ATH11K_MSG_MAX 400
+
TRACE_EVENT(ath11k_htt_pktlog,
TP_PROTO(struct ath11k *ar, const void *buf, u16 buf_len,
u32 pktlog_checksum),
@@ -108,6 +120,194 @@ TRACE_EVENT(ath11k_htt_rxdesc,
)
);
+DECLARE_EVENT_CLASS(ath11k_log_event,
+ TP_PROTO(struct ath11k_base *ab, struct va_format *vaf),
+ TP_ARGS(ab, vaf),
+ TP_STRUCT__entry(
+ __string(device, dev_name(ab->dev))
+ __string(driver, dev_driver_string(ab->dev))
+ __dynamic_array(char, msg, ATH11K_MSG_MAX)
+ ),
+ TP_fast_assign(
+ __assign_str(device, dev_name(ab->dev));
+ __assign_str(driver, dev_driver_string(ab->dev));
+ WARN_ON_ONCE(vsnprintf(__get_dynamic_array(msg),
+ ATH11K_MSG_MAX,
+ vaf->fmt,
+ *vaf->va) >= ATH11K_MSG_MAX);
+ ),
+ TP_printk(
+ "%s %s %s",
+ __get_str(driver),
+ __get_str(device),
+ __get_str(msg)
+ )
+);
+
+DEFINE_EVENT(ath11k_log_event, ath11k_log_err,
+ TP_PROTO(struct ath11k_base *ab, struct va_format *vaf),
+ TP_ARGS(ab, vaf)
+);
+
+DEFINE_EVENT(ath11k_log_event, ath11k_log_warn,
+ TP_PROTO(struct ath11k_base *ab, struct va_format *vaf),
+ TP_ARGS(ab, vaf)
+);
+
+DEFINE_EVENT(ath11k_log_event, ath11k_log_info,
+ TP_PROTO(struct ath11k_base *ab, struct va_format *vaf),
+ TP_ARGS(ab, vaf)
+);
+
+TRACE_EVENT(ath11k_wmi_cmd,
+ TP_PROTO(struct ath11k_base *ab, int id, const void *buf, size_t buf_len),
+
+ TP_ARGS(ab, id, buf, buf_len),
+
+ TP_STRUCT__entry(
+ __string(device, dev_name(ab->dev))
+ __string(driver, dev_driver_string(ab->dev))
+ __field(unsigned int, id)
+ __field(size_t, buf_len)
+ __dynamic_array(u8, buf, buf_len)
+ ),
+
+ TP_fast_assign(
+ __assign_str(device, dev_name(ab->dev));
+ __assign_str(driver, dev_driver_string(ab->dev));
+ __entry->id = id;
+ __entry->buf_len = buf_len;
+ memcpy(__get_dynamic_array(buf), buf, buf_len);
+ ),
+
+ TP_printk(
+ "%s %s id %d len %zu",
+ __get_str(driver),
+ __get_str(device),
+ __entry->id,
+ __entry->buf_len
+ )
+);
+
+TRACE_EVENT(ath11k_wmi_event,
+ TP_PROTO(struct ath11k_base *ab, int id, const void *buf, size_t buf_len),
+
+ TP_ARGS(ab, id, buf, buf_len),
+
+ TP_STRUCT__entry(
+ __string(device, dev_name(ab->dev))
+ __string(driver, dev_driver_string(ab->dev))
+ __field(unsigned int, id)
+ __field(size_t, buf_len)
+ __dynamic_array(u8, buf, buf_len)
+ ),
+
+ TP_fast_assign(
+ __assign_str(device, dev_name(ab->dev));
+ __assign_str(driver, dev_driver_string(ab->dev));
+ __entry->id = id;
+ __entry->buf_len = buf_len;
+ memcpy(__get_dynamic_array(buf), buf, buf_len);
+ ),
+
+ TP_printk(
+ "%s %s id %d len %zu",
+ __get_str(driver),
+ __get_str(device),
+ __entry->id,
+ __entry->buf_len
+ )
+);
+
+TRACE_EVENT(ath11k_log_dbg,
+ TP_PROTO(struct ath11k_base *ab, unsigned int level, struct va_format *vaf),
+
+ TP_ARGS(ab, level, vaf),
+
+ TP_STRUCT__entry(
+ __string(device, dev_name(ab->dev))
+ __string(driver, dev_driver_string(ab->dev))
+ __field(unsigned int, level)
+ __dynamic_array(char, msg, ATH11K_MSG_MAX)
+ ),
+
+ TP_fast_assign(
+ __assign_str(device, dev_name(ab->dev));
+ __assign_str(driver, dev_driver_string(ab->dev));
+ __entry->level = level;
+ WARN_ON_ONCE(vsnprintf(__get_dynamic_array(msg),
+ ATH11K_MSG_MAX, vaf->fmt,
+ *vaf->va) >= ATH11K_MSG_MAX);
+ ),
+
+ TP_printk(
+ "%s %s %s",
+ __get_str(driver),
+ __get_str(device),
+ __get_str(msg)
+ )
+);
+
+TRACE_EVENT(ath11k_log_dbg_dump,
+ TP_PROTO(struct ath11k_base *ab, const char *msg, const char *prefix,
+ const void *buf, size_t buf_len),
+
+ TP_ARGS(ab, msg, prefix, buf, buf_len),
+
+ TP_STRUCT__entry(
+ __string(device, dev_name(ab->dev))
+ __string(driver, dev_driver_string(ab->dev))
+ __string(msg, msg)
+ __string(prefix, prefix)
+ __field(size_t, buf_len)
+ __dynamic_array(u8, buf, buf_len)
+ ),
+
+ TP_fast_assign(
+ __assign_str(device, dev_name(ab->dev));
+ __assign_str(driver, dev_driver_string(ab->dev));
+ __assign_str(msg, msg);
+ __assign_str(prefix, prefix);
+ __entry->buf_len = buf_len;
+ memcpy(__get_dynamic_array(buf), buf, buf_len);
+ ),
+
+ TP_printk(
+ "%s %s %s/%s\n",
+ __get_str(driver),
+ __get_str(device),
+ __get_str(prefix),
+ __get_str(msg)
+ )
+);
+
+TRACE_EVENT(ath11k_wmi_diag,
+ TP_PROTO(struct ath11k_base *ab, const void *data, size_t len),
+
+ TP_ARGS(ab, data, len),
+
+ TP_STRUCT__entry(
+ __string(device, dev_name(ab->dev))
+ __string(driver, dev_driver_string(ab->dev))
+ __field(u16, len)
+ __dynamic_array(u8, data, len)
+ ),
+
+ TP_fast_assign(
+ __assign_str(device, dev_name(ab->dev));
+ __assign_str(driver, dev_driver_string(ab->dev));
+ __entry->len = len;
+ memcpy(__get_dynamic_array(data), data, len);
+ ),
+
+ TP_printk(
+ "%s %s tlv diag len %d",
+ __get_str(driver),
+ __get_str(device),
+ __entry->len
+ )
+);
+
#endif /* _TRACE_H_ || TRACE_HEADER_MULTI_READ*/
/* we don't want to use include/trace/events */
diff --git a/drivers/net/wireless/ath/ath11k/wmi.c b/drivers/net/wireless/ath/ath11k/wmi.c
index 5ae2ef4..6b68ccf6 100644
--- a/drivers/net/wireless/ath/ath11k/wmi.c
+++ b/drivers/net/wireless/ath/ath11k/wmi.c
@@ -73,6 +73,14 @@ struct wmi_tlv_dma_buf_release_parse {
bool meta_data_done;
};
+struct wmi_tlv_fw_stats_parse {
+ const struct wmi_stats_event *ev;
+ const struct wmi_per_chain_rssi_stats *rssi;
+ struct ath11k_fw_stats *stats;
+ int rssi_num;
+ bool chain_rssi_done;
+};
+
static const struct wmi_tlv_policy wmi_tlv_policies[] = {
[WMI_TAG_ARRAY_BYTE]
= { .min_len = 0 },
@@ -120,6 +128,8 @@ static const struct wmi_tlv_policy wmi_tlv_policies[] = {
= { .min_len = sizeof(struct wmi_peer_assoc_conf_event) },
[WMI_TAG_STATS_EVENT]
= { .min_len = sizeof(struct wmi_stats_event) },
+ [WMI_TAG_RFKILL_EVENT] = {
+ .min_len = sizeof(struct wmi_rfkill_state_change_ev) },
[WMI_TAG_PDEV_CTL_FAILSAFE_CHECK_EVENT]
= { .min_len = sizeof(struct wmi_pdev_ctl_failsafe_chk_event) },
[WMI_TAG_HOST_SWFDA_EVENT] = {
@@ -128,6 +138,12 @@ static const struct wmi_tlv_policy wmi_tlv_policies[] = {
.min_len = sizeof(struct wmi_probe_resp_tx_status_event) },
[WMI_TAG_VDEV_DELETE_RESP_EVENT] = {
.min_len = sizeof(struct wmi_vdev_delete_resp_event) },
+ [WMI_TAG_OBSS_COLOR_COLLISION_EVT] = {
+ .min_len = sizeof(struct wmi_obss_color_collision_event) },
+ [WMI_TAG_11D_NEW_COUNTRY_EVENT] = {
+ .min_len = sizeof(struct wmi_11d_new_cc_ev) },
+ [WMI_TAG_PER_CHAIN_RSSI_STATS] = {
+ .min_len = sizeof(struct wmi_per_chain_rssi_stats) },
};
#define PRIMAP(_hw_mode_) \
@@ -249,6 +265,8 @@ static int ath11k_wmi_cmd_send_nowait(struct ath11k_pdev_wmi *wmi, struct sk_buf
cmd_hdr = (struct wmi_cmd_hdr *)skb->data;
cmd_hdr->cmd_id = cmd;
+ trace_ath11k_wmi_cmd(ab, cmd_id, skb->data, skb->len);
+
memset(skb_cb, 0, sizeof(*skb_cb));
ret = ath11k_htc_send(&ab->htc, wmi->eid, skb);
@@ -267,21 +285,39 @@ int ath11k_wmi_cmd_send(struct ath11k_pdev_wmi *wmi, struct sk_buff *skb,
{
struct ath11k_wmi_base *wmi_sc = wmi->wmi_ab;
int ret = -EOPNOTSUPP;
+ struct ath11k_base *ab = wmi_sc->ab;
might_sleep();
- wait_event_timeout(wmi_sc->tx_credits_wq, ({
- ret = ath11k_wmi_cmd_send_nowait(wmi, skb, cmd_id);
+ if (ab->hw_params.credit_flow) {
+ wait_event_timeout(wmi_sc->tx_credits_wq, ({
+ ret = ath11k_wmi_cmd_send_nowait(wmi, skb, cmd_id);
- if (ret && test_bit(ATH11K_FLAG_CRASH_FLUSH, &wmi_sc->ab->dev_flags))
- ret = -ESHUTDOWN;
+ if (ret && test_bit(ATH11K_FLAG_CRASH_FLUSH,
+ &wmi_sc->ab->dev_flags))
+ ret = -ESHUTDOWN;
- (ret != -EAGAIN);
- }), WMI_SEND_TIMEOUT_HZ);
+ (ret != -EAGAIN);
+ }), WMI_SEND_TIMEOUT_HZ);
+ } else {
+ wait_event_timeout(wmi->tx_ce_desc_wq, ({
+ ret = ath11k_wmi_cmd_send_nowait(wmi, skb, cmd_id);
+
+ if (ret && test_bit(ATH11K_FLAG_CRASH_FLUSH,
+ &wmi_sc->ab->dev_flags))
+ ret = -ESHUTDOWN;
+
+ (ret != -ENOBUFS);
+ }), WMI_SEND_TIMEOUT_HZ);
+ }
if (ret == -EAGAIN)
ath11k_warn(wmi_sc->ab, "wmi command %d timeout\n", cmd_id);
+ if (ret == -ENOBUFS)
+ ath11k_warn(wmi_sc->ab, "ce desc not available for wmi command %d\n",
+ cmd_id);
+
return ret;
}
@@ -315,6 +351,7 @@ ath11k_pull_mac_phy_cap_svc_ready_ext(struct ath11k_pdev_wmi *wmi_handle,
struct ath11k_pdev *pdev)
{
struct wmi_mac_phy_capabilities *mac_phy_caps;
+ struct ath11k_base *ab = wmi_handle->wmi_ab->ab;
struct ath11k_band_cap *cap_band;
struct ath11k_pdev_cap *pdev_cap = &pdev->cap;
u32 phy_map;
@@ -346,6 +383,10 @@ ath11k_pull_mac_phy_cap_svc_ready_ext(struct ath11k_pdev_wmi *wmi_handle,
pdev->pdev_id = mac_phy_caps->pdev_id;
pdev_cap->supported_bands |= mac_phy_caps->supported_bands;
pdev_cap->ampdu_density = mac_phy_caps->ampdu_density;
+ ab->target_pdev_ids[ab->target_pdev_count].supported_bands =
+ mac_phy_caps->supported_bands;
+ ab->target_pdev_ids[ab->target_pdev_count].pdev_id = mac_phy_caps->pdev_id;
+ ab->target_pdev_count++;
/* Take non-zero tx/rx chainmask. If tx/rx chainmask differs from
* band to band for a single radio, need to see how this should be
@@ -485,6 +526,8 @@ static int ath11k_pull_service_ready_tlv(struct ath11k_base *ab,
cap->default_dbs_hw_mode_index = ev->default_dbs_hw_mode_index;
cap->num_msdu_desc = ev->num_msdu_desc;
+ ath11k_dbg(ab, ATH11K_DBG_WMI, "wmi sys cap info 0x%x\n", cap->sys_cap_info);
+
return 0;
}
@@ -1244,7 +1287,8 @@ int ath11k_wmi_pdev_set_param(struct ath11k *ar, u32 param_id,
return ret;
}
-int ath11k_wmi_pdev_set_ps_mode(struct ath11k *ar, int vdev_id, u32 enable)
+int ath11k_wmi_pdev_set_ps_mode(struct ath11k *ar, int vdev_id,
+ enum wmi_sta_ps_mode psmode)
{
struct ath11k_pdev_wmi *wmi = ar->wmi;
struct wmi_pdev_set_ps_mode_cmd *cmd;
@@ -1259,7 +1303,7 @@ int ath11k_wmi_pdev_set_ps_mode(struct ath11k *ar, int vdev_id, u32 enable)
cmd->tlv_header = FIELD_PREP(WMI_TLV_TAG, WMI_TAG_STA_POWERSAVE_MODE_CMD) |
FIELD_PREP(WMI_TLV_LEN, sizeof(*cmd) - TLV_HDR_SIZE);
cmd->vdev_id = vdev_id;
- cmd->sta_ps_mode = enable;
+ cmd->sta_ps_mode = psmode;
ret = ath11k_wmi_cmd_send(wmi, skb, WMI_STA_POWERSAVE_MODE_CMDID);
if (ret) {
@@ -1269,7 +1313,7 @@ int ath11k_wmi_pdev_set_ps_mode(struct ath11k *ar, int vdev_id, u32 enable)
ath11k_dbg(ar->ab, ATH11K_DBG_WMI,
"WMI vdev set psmode %d vdev id %d\n",
- enable, vdev_id);
+ psmode, vdev_id);
return ret;
}
@@ -1612,6 +1656,15 @@ int ath11k_wmi_bcn_tmpl(struct ath11k *ar, u32 vdev_id,
void *ptr;
int ret, len;
size_t aligned_len = roundup(bcn->len, 4);
+ struct ieee80211_vif *vif;
+ struct ath11k_vif *arvif = ath11k_mac_get_arvif(ar, vdev_id);
+
+ if (!arvif) {
+ ath11k_warn(ar->ab, "failed to find arvif with vdev id %d\n", vdev_id);
+ return -EINVAL;
+ }
+
+ vif = arvif->vif;
len = sizeof(*cmd) + sizeof(*bcn_prb_info) + TLV_HDR_SIZE + aligned_len;
@@ -1624,8 +1677,12 @@ int ath11k_wmi_bcn_tmpl(struct ath11k *ar, u32 vdev_id,
FIELD_PREP(WMI_TLV_LEN, sizeof(*cmd) - TLV_HDR_SIZE);
cmd->vdev_id = vdev_id;
cmd->tim_ie_offset = offs->tim_offset;
- cmd->csa_switch_count_offset = offs->cntdwn_counter_offs[0];
- cmd->ext_csa_switch_count_offset = offs->cntdwn_counter_offs[1];
+
+ if (vif->csa_active) {
+ cmd->csa_switch_count_offset = offs->cntdwn_counter_offs[0];
+ cmd->ext_csa_switch_count_offset = offs->cntdwn_counter_offs[1];
+ }
+
cmd->buf_len = bcn->len;
ptr = skb->data + sizeof(*cmd);
@@ -1689,7 +1746,8 @@ int ath11k_wmi_vdev_install_key(struct ath11k *ar,
tlv = (struct wmi_tlv *)(skb->data + sizeof(*cmd));
tlv->header = FIELD_PREP(WMI_TLV_TAG, WMI_TAG_ARRAY_BYTE) |
FIELD_PREP(WMI_TLV_LEN, key_len_aligned);
- memcpy(tlv->value, (u8 *)arg->key_data, key_len_aligned);
+ if (arg->key_data)
+ memcpy(tlv->value, (u8 *)arg->key_data, key_len_aligned);
ret = ath11k_wmi_cmd_send(wmi, skb, WMI_VDEV_INSTALL_KEY_CMDID);
if (ret) {
@@ -1762,7 +1820,7 @@ ath11k_wmi_copy_peer_flags(struct wmi_peer_assoc_complete_cmd *cmd,
cmd->peer_flags |= WMI_PEER_AUTH;
if (param->need_ptk_4_way) {
cmd->peer_flags |= WMI_PEER_NEED_PTK_4_WAY;
- if (!hw_crypto_disabled)
+ if (!hw_crypto_disabled && param->is_assoc)
cmd->peer_flags &= ~WMI_PEER_AUTH;
}
if (param->need_gtk_2_way)
@@ -2069,7 +2127,7 @@ int ath11k_wmi_send_scan_start_cmd(struct ath11k *ar,
void *ptr;
int i, ret, len;
u32 *tmp_ptr;
- u8 extraie_len_with_pad = 0;
+ u16 extraie_len_with_pad = 0;
struct hint_short_ssid *s_ssid = NULL;
struct hint_bssid *hint_bssid = NULL;
@@ -2088,7 +2146,7 @@ int ath11k_wmi_send_scan_start_cmd(struct ath11k *ar,
len += sizeof(*bssid) * params->num_bssid;
len += TLV_HDR_SIZE;
- if (params->extraie.len)
+ if (params->extraie.len && params->extraie.len <= 0xFFFF)
extraie_len_with_pad =
roundup(params->extraie.len, sizeof(u32));
len += extraie_len_with_pad;
@@ -2137,6 +2195,8 @@ int ath11k_wmi_send_scan_start_cmd(struct ath11k *ar,
cmd->num_ssids = params->num_ssids;
cmd->ie_len = params->extraie.len;
cmd->n_probes = params->n_probes;
+ ether_addr_copy(cmd->mac_addr.addr, params->mac_addr.addr);
+ ether_addr_copy(cmd->mac_mask.addr, params->mac_mask.addr);
ptr += sizeof(*cmd);
@@ -2195,7 +2255,7 @@ int ath11k_wmi_send_scan_start_cmd(struct ath11k *ar,
FIELD_PREP(WMI_TLV_LEN, len);
ptr += TLV_HDR_SIZE;
- if (params->extraie.len)
+ if (extraie_len_with_pad)
memcpy(ptr, params->extraie.ptr,
params->extraie.len);
@@ -2386,6 +2446,8 @@ int ath11k_wmi_send_scan_chan_list_cmd(struct ath11k *ar,
tchan_info->reg_class_id);
*reg2 |= FIELD_PREP(WMI_CHAN_REG_INFO2_ANT_MAX,
tchan_info->antennamax);
+ *reg2 |= FIELD_PREP(WMI_CHAN_REG_INFO2_MAX_TX_PWR,
+ tchan_info->maxregpower);
ath11k_dbg(ar->ab, ATH11K_DBG_WMI,
"WMI chan scan list chan[%d] = %u, chan_info->info %8x\n",
@@ -2754,6 +2816,42 @@ ath11k_wmi_send_init_country_cmd(struct ath11k *ar,
return ret;
}
+int ath11k_wmi_send_set_current_country_cmd(struct ath11k *ar,
+ struct wmi_set_current_country_params *param)
+{
+ struct ath11k_pdev_wmi *wmi = ar->wmi;
+ struct wmi_set_current_country_cmd *cmd;
+ struct sk_buff *skb;
+ int ret;
+
+ skb = ath11k_wmi_alloc_skb(wmi->wmi_ab, sizeof(*cmd));
+ if (!skb)
+ return -ENOMEM;
+
+ cmd = (struct wmi_set_current_country_cmd *)skb->data;
+ cmd->tlv_header =
+ FIELD_PREP(WMI_TLV_TAG, WMI_TAG_SET_CURRENT_COUNTRY_CMD) |
+ FIELD_PREP(WMI_TLV_LEN, sizeof(*cmd) - TLV_HDR_SIZE);
+
+ cmd->pdev_id = ar->pdev->pdev_id;
+ memcpy(&cmd->new_alpha2, ¶m->alpha2, 3);
+ ret = ath11k_wmi_cmd_send(wmi, skb, WMI_SET_CURRENT_COUNTRY_CMDID);
+
+ ath11k_dbg(ar->ab, ATH11K_DBG_WMI,
+ "set current country pdev id %d alpha2 %c%c\n",
+ ar->pdev->pdev_id,
+ param->alpha2[0],
+ param->alpha2[1]);
+
+ if (ret) {
+ ath11k_warn(ar->ab,
+ "failed to send WMI_SET_CURRENT_COUNTRY_CMDID: %d\n", ret);
+ dev_kfree_skb(skb);
+ }
+
+ return ret;
+}
+
int
ath11k_wmi_send_thermal_mitigation_param_cmd(struct ath11k *ar,
struct thermal_mitigation_params *param)
@@ -2818,6 +2916,75 @@ ath11k_wmi_send_thermal_mitigation_param_cmd(struct ath11k *ar,
return ret;
}
+int ath11k_wmi_send_11d_scan_start_cmd(struct ath11k *ar,
+ struct wmi_11d_scan_start_params *param)
+{
+ struct ath11k_pdev_wmi *wmi = ar->wmi;
+ struct wmi_11d_scan_start_cmd *cmd;
+ struct sk_buff *skb;
+ int ret;
+
+ skb = ath11k_wmi_alloc_skb(wmi->wmi_ab, sizeof(*cmd));
+ if (!skb)
+ return -ENOMEM;
+
+ cmd = (struct wmi_11d_scan_start_cmd *)skb->data;
+ cmd->tlv_header =
+ FIELD_PREP(WMI_TLV_TAG, WMI_TAG_11D_SCAN_START_CMD) |
+ FIELD_PREP(WMI_TLV_LEN, sizeof(*cmd) - TLV_HDR_SIZE);
+
+ cmd->vdev_id = param->vdev_id;
+ cmd->scan_period_msec = param->scan_period_msec;
+ cmd->start_interval_msec = param->start_interval_msec;
+ ret = ath11k_wmi_cmd_send(wmi, skb, WMI_11D_SCAN_START_CMDID);
+
+ ath11k_dbg(ar->ab, ATH11K_DBG_WMI,
+ "send 11d scan start vdev id %d period %d ms internal %d ms\n",
+ cmd->vdev_id,
+ cmd->scan_period_msec,
+ cmd->start_interval_msec);
+
+ if (ret) {
+ ath11k_warn(ar->ab,
+ "failed to send WMI_11D_SCAN_START_CMDID: %d\n", ret);
+ dev_kfree_skb(skb);
+ }
+
+ return ret;
+}
+
+int ath11k_wmi_send_11d_scan_stop_cmd(struct ath11k *ar, u32 vdev_id)
+{
+ struct ath11k_pdev_wmi *wmi = ar->wmi;
+ struct wmi_11d_scan_stop_cmd *cmd;
+ struct sk_buff *skb;
+ int ret;
+
+ skb = ath11k_wmi_alloc_skb(wmi->wmi_ab, sizeof(*cmd));
+ if (!skb)
+ return -ENOMEM;
+
+ cmd = (struct wmi_11d_scan_stop_cmd *)skb->data;
+ cmd->tlv_header =
+ FIELD_PREP(WMI_TLV_TAG, WMI_TAG_11D_SCAN_STOP_CMD) |
+ FIELD_PREP(WMI_TLV_LEN, sizeof(*cmd) - TLV_HDR_SIZE);
+
+ cmd->vdev_id = vdev_id;
+ ret = ath11k_wmi_cmd_send(wmi, skb, WMI_11D_SCAN_STOP_CMDID);
+
+ ath11k_dbg(ar->ab, ATH11K_DBG_WMI,
+ "send 11d scan stop vdev id %d\n",
+ cmd->vdev_id);
+
+ if (ret) {
+ ath11k_warn(ar->ab,
+ "failed to send WMI_11D_SCAN_STOP_CMDID: %d\n", ret);
+ dev_kfree_skb(skb);
+ }
+
+ return ret;
+}
+
int ath11k_wmi_pdev_pktlog_enable(struct ath11k *ar, u32 pktlog_filter)
{
struct ath11k_pdev_wmi *wmi = ar->wmi;
@@ -3428,6 +3595,56 @@ int ath11k_wmi_fils_discovery(struct ath11k *ar, u32 vdev_id, u32 interval,
}
static void
+ath11k_wmi_obss_color_collision_event(struct ath11k_base *ab, struct sk_buff *skb)
+{
+ const void **tb;
+ const struct wmi_obss_color_collision_event *ev;
+ struct ath11k_vif *arvif;
+ int ret;
+
+ tb = ath11k_wmi_tlv_parse_alloc(ab, skb->data, skb->len, GFP_ATOMIC);
+ if (IS_ERR(tb)) {
+ ret = PTR_ERR(tb);
+ ath11k_warn(ab, "failed to parse tlv: %d\n", ret);
+ return;
+ }
+
+ rcu_read_lock();
+
+ ev = tb[WMI_TAG_OBSS_COLOR_COLLISION_EVT];
+ if (!ev) {
+ ath11k_warn(ab, "failed to fetch obss color collision ev");
+ goto exit;
+ }
+
+ arvif = ath11k_mac_get_arvif_by_vdev_id(ab, ev->vdev_id);
+ if (!arvif) {
+ ath11k_warn(ab, "failed to find arvif with vedv id %d in obss_color_collision_event\n",
+ ev->vdev_id);
+ goto exit;
+ }
+
+ switch (ev->evt_type) {
+ case WMI_BSS_COLOR_COLLISION_DETECTION:
+ ieeee80211_obss_color_collision_notify(arvif->vif, ev->obss_color_bitmap);
+ ath11k_dbg(ab, ATH11K_DBG_WMI,
+ "OBSS color collision detected vdev:%d, event:%d, bitmap:%08llx\n",
+ ev->vdev_id, ev->evt_type, ev->obss_color_bitmap);
+ break;
+ case WMI_BSS_COLOR_COLLISION_DISABLE:
+ case WMI_BSS_COLOR_FREE_SLOT_TIMER_EXPIRY:
+ case WMI_BSS_COLOR_FREE_SLOT_AVAILABLE:
+ break;
+ default:
+ ath11k_warn(ab, "received unknown obss color collision detection event\n");
+ }
+
+exit:
+ kfree(tb);
+ rcu_read_unlock();
+}
+
+static void
ath11k_fill_band_to_mac_param(struct ath11k_base *soc,
struct wmi_host_pdev_band_to_mac *band_to_mac)
{
@@ -4144,6 +4361,7 @@ static int ath11k_wmi_tlv_ext_soc_hal_reg_caps_parse(struct ath11k_base *soc,
svc_rdy_ext->param.num_phy = svc_rdy_ext->soc_hal_reg_caps->num_phy;
soc->num_radios = 0;
+ soc->target_pdev_count = 0;
phy_id_map = svc_rdy_ext->pref_hw_mode_caps.phy_id_map;
while (phy_id_map && soc->num_radios < MAX_RADIOS) {
@@ -4781,6 +4999,7 @@ static int wmi_process_mgmt_tx_comp(struct ath11k *ar, u32 desc_id,
struct sk_buff *msdu;
struct ieee80211_tx_info *info;
struct ath11k_skb_cb *skb_cb;
+ int num_mgmt;
spin_lock_bh(&ar->txmgmt_idr_lock);
msdu = idr_find(&ar->txmgmt_idr, desc_id);
@@ -4804,10 +5023,19 @@ static int wmi_process_mgmt_tx_comp(struct ath11k *ar, u32 desc_id,
ieee80211_tx_status_irqsafe(ar->hw, msdu);
+ num_mgmt = atomic_dec_if_positive(&ar->num_pending_mgmt_tx);
+
/* WARN when we received this event without doing any mgmt tx */
- if (atomic_dec_if_positive(&ar->num_pending_mgmt_tx) < 0)
+ if (num_mgmt < 0)
WARN_ON_ONCE(1);
+ ath11k_dbg(ar->ab, ATH11K_DBG_WMI,
+ "wmi mgmt tx comp pending %d desc id %d\n",
+ num_mgmt, desc_id);
+
+ if (!num_mgmt)
+ wake_up(&ar->txmgmt_empty_waitq);
+
return 0;
}
@@ -5343,47 +5571,107 @@ ath11k_wmi_pull_bcn_stats(const struct wmi_bcn_stats *src,
dst->tx_bcn_outage_cnt = src->tx_bcn_outage_cnt;
}
-int ath11k_wmi_pull_fw_stats(struct ath11k_base *ab, struct sk_buff *skb,
- struct ath11k_fw_stats *stats)
+static int ath11k_wmi_tlv_rssi_chain_parse(struct ath11k_base *ab,
+ u16 tag, u16 len,
+ const void *ptr, void *data)
{
- const void **tb;
- const struct wmi_stats_event *ev;
- const void *data;
- int i, ret;
- u32 len = skb->len;
+ struct wmi_tlv_fw_stats_parse *parse = data;
+ const struct wmi_stats_event *ev = parse->ev;
+ struct ath11k_fw_stats *stats = parse->stats;
+ struct ath11k *ar;
+ struct ath11k_vif *arvif;
+ struct ieee80211_sta *sta;
+ struct ath11k_sta *arsta;
+ const struct wmi_rssi_stats *stats_rssi = (const struct wmi_rssi_stats *)ptr;
+ int j, ret = 0;
- tb = ath11k_wmi_tlv_parse_alloc(ab, skb->data, len, GFP_ATOMIC);
- if (IS_ERR(tb)) {
- ret = PTR_ERR(tb);
- ath11k_warn(ab, "failed to parse tlv: %d\n", ret);
- return ret;
- }
-
- ev = tb[WMI_TAG_STATS_EVENT];
- data = tb[WMI_TAG_ARRAY_BYTE];
- if (!ev || !data) {
- ath11k_warn(ab, "failed to fetch update stats ev");
- kfree(tb);
+ if (tag != WMI_TAG_RSSI_STATS)
return -EPROTO;
+
+ rcu_read_lock();
+
+ ar = ath11k_mac_get_ar_by_pdev_id(ab, ev->pdev_id);
+ stats->stats_id = WMI_REQUEST_RSSI_PER_CHAIN_STAT;
+
+ ath11k_dbg(ab, ATH11K_DBG_WMI,
+ "wmi stats vdev id %d mac %pM\n",
+ stats_rssi->vdev_id, stats_rssi->peer_macaddr.addr);
+
+ arvif = ath11k_mac_get_arvif(ar, stats_rssi->vdev_id);
+ if (!arvif) {
+ ath11k_warn(ab, "not found vif for vdev id %d\n",
+ stats_rssi->vdev_id);
+ ret = -EPROTO;
+ goto exit;
}
ath11k_dbg(ab, ATH11K_DBG_WMI,
- "wmi stats update ev pdev_id %d pdev %i vdev %i bcn %i\n",
- ev->pdev_id,
- ev->num_pdev_stats, ev->num_vdev_stats,
- ev->num_bcn_stats);
+ "wmi stats bssid %pM vif %pK\n",
+ arvif->bssid, arvif->vif);
- stats->pdev_id = ev->pdev_id;
+ sta = ieee80211_find_sta_by_ifaddr(ar->hw,
+ arvif->bssid,
+ NULL);
+ if (!sta) {
+ ath11k_warn(ab, "not found station for bssid %pM\n",
+ arvif->bssid);
+ ret = -EPROTO;
+ goto exit;
+ }
+
+ arsta = (struct ath11k_sta *)sta->drv_priv;
+
+ BUILD_BUG_ON(ARRAY_SIZE(arsta->chain_signal) >
+ ARRAY_SIZE(stats_rssi->rssi_avg_beacon));
+
+ for (j = 0; j < ARRAY_SIZE(arsta->chain_signal); j++) {
+ arsta->chain_signal[j] = stats_rssi->rssi_avg_beacon[j];
+ ath11k_dbg(ab, ATH11K_DBG_WMI,
+ "wmi stats beacon rssi[%d] %d data rssi[%d] %d\n",
+ j,
+ stats_rssi->rssi_avg_beacon[j],
+ j,
+ stats_rssi->rssi_avg_data[j]);
+ }
+
+exit:
+ rcu_read_unlock();
+ return ret;
+}
+
+static int ath11k_wmi_tlv_fw_stats_data_parse(struct ath11k_base *ab,
+ struct wmi_tlv_fw_stats_parse *parse,
+ const void *ptr,
+ u16 len)
+{
+ struct ath11k_fw_stats *stats = parse->stats;
+ const struct wmi_stats_event *ev = parse->ev;
+ struct ath11k *ar;
+ struct ath11k_vif *arvif;
+ struct ieee80211_sta *sta;
+ struct ath11k_sta *arsta;
+ int i, ret = 0;
+ const void *data = ptr;
+
+ if (!ev) {
+ ath11k_warn(ab, "failed to fetch update stats ev");
+ return -EPROTO;
+ }
+
stats->stats_id = 0;
+ rcu_read_lock();
+
+ ar = ath11k_mac_get_ar_by_pdev_id(ab, ev->pdev_id);
+
for (i = 0; i < ev->num_pdev_stats; i++) {
const struct wmi_pdev_stats *src;
struct ath11k_fw_stats_pdev *dst;
src = data;
if (len < sizeof(*src)) {
- kfree(tb);
- return -EPROTO;
+ ret = -EPROTO;
+ goto exit;
}
stats->stats_id = WMI_REQUEST_PDEV_STAT;
@@ -5407,12 +5695,29 @@ int ath11k_wmi_pull_fw_stats(struct ath11k_base *ab, struct sk_buff *skb,
src = data;
if (len < sizeof(*src)) {
- kfree(tb);
- return -EPROTO;
+ ret = -EPROTO;
+ goto exit;
}
stats->stats_id = WMI_REQUEST_VDEV_STAT;
+ arvif = ath11k_mac_get_arvif(ar, src->vdev_id);
+ if (arvif) {
+ sta = ieee80211_find_sta_by_ifaddr(ar->hw,
+ arvif->bssid,
+ NULL);
+ if (sta) {
+ arsta = (struct ath11k_sta *)sta->drv_priv;
+ arsta->rssi_beacon = src->beacon_snr;
+ ath11k_dbg(ab, ATH11K_DBG_WMI,
+ "wmi stats vdev id %d snr %d\n",
+ src->vdev_id, src->beacon_snr);
+ } else {
+ ath11k_warn(ab, "not found station for bssid %pM\n",
+ arvif->bssid);
+ }
+ }
+
data += sizeof(*src);
len -= sizeof(*src);
@@ -5430,8 +5735,8 @@ int ath11k_wmi_pull_fw_stats(struct ath11k_base *ab, struct sk_buff *skb,
src = data;
if (len < sizeof(*src)) {
- kfree(tb);
- return -EPROTO;
+ ret = -EPROTO;
+ goto exit;
}
stats->stats_id = WMI_REQUEST_BCN_STAT;
@@ -5447,8 +5752,67 @@ int ath11k_wmi_pull_fw_stats(struct ath11k_base *ab, struct sk_buff *skb,
list_add_tail(&dst->list, &stats->bcn);
}
- kfree(tb);
- return 0;
+exit:
+ rcu_read_unlock();
+ return ret;
+}
+
+static int ath11k_wmi_tlv_fw_stats_parse(struct ath11k_base *ab,
+ u16 tag, u16 len,
+ const void *ptr, void *data)
+{
+ struct wmi_tlv_fw_stats_parse *parse = data;
+ int ret = 0;
+
+ switch (tag) {
+ case WMI_TAG_STATS_EVENT:
+ parse->ev = (struct wmi_stats_event *)ptr;
+ parse->stats->pdev_id = parse->ev->pdev_id;
+ break;
+ case WMI_TAG_ARRAY_BYTE:
+ ret = ath11k_wmi_tlv_fw_stats_data_parse(ab, parse, ptr, len);
+ break;
+ case WMI_TAG_PER_CHAIN_RSSI_STATS:
+ parse->rssi = (struct wmi_per_chain_rssi_stats *)ptr;
+
+ if (parse->ev->stats_id & WMI_REQUEST_RSSI_PER_CHAIN_STAT)
+ parse->rssi_num = parse->rssi->num_per_chain_rssi_stats;
+
+ ath11k_dbg(ab, ATH11K_DBG_WMI,
+ "wmi stats id 0x%x num chain %d\n",
+ parse->ev->stats_id,
+ parse->rssi_num);
+ break;
+ case WMI_TAG_ARRAY_STRUCT:
+ if (parse->rssi_num && !parse->chain_rssi_done) {
+ ret = ath11k_wmi_tlv_iter(ab, ptr, len,
+ ath11k_wmi_tlv_rssi_chain_parse,
+ parse);
+ if (ret) {
+ ath11k_warn(ab, "failed to parse rssi chain %d\n",
+ ret);
+ return ret;
+ }
+ parse->chain_rssi_done = true;
+ }
+ break;
+ default:
+ break;
+ }
+ return ret;
+}
+
+int ath11k_wmi_pull_fw_stats(struct ath11k_base *ab, struct sk_buff *skb,
+ struct ath11k_fw_stats *stats)
+{
+ struct wmi_tlv_fw_stats_parse parse = { };
+
+ stats->stats_id = 0;
+ parse.stats = stats;
+
+ return ath11k_wmi_tlv_iter(ab, skb->data, skb->len,
+ ath11k_wmi_tlv_fw_stats_parse,
+ &parse);
}
size_t ath11k_wmi_fw_stats_num_vdevs(struct list_head *head)
@@ -5810,15 +6174,79 @@ static void ath11k_wmi_op_ep_tx_credits(struct ath11k_base *ab)
wake_up(&ab->wmi_ab.tx_credits_wq);
}
+static int ath11k_reg_11d_new_cc_event(struct ath11k_base *ab, struct sk_buff *skb)
+{
+ const struct wmi_11d_new_cc_ev *ev;
+ const void **tb;
+ int ret;
+
+ tb = ath11k_wmi_tlv_parse_alloc(ab, skb->data, skb->len, GFP_ATOMIC);
+ if (IS_ERR(tb)) {
+ ret = PTR_ERR(tb);
+ ath11k_warn(ab, "failed to parse tlv: %d\n", ret);
+ return ret;
+ }
+
+ ev = tb[WMI_TAG_11D_NEW_COUNTRY_EVENT];
+ if (!ev) {
+ kfree(tb);
+ ath11k_warn(ab, "failed to fetch 11d new cc ev");
+ return -EPROTO;
+ }
+
+ spin_lock_bh(&ab->base_lock);
+ memcpy(&ab->new_alpha2, &ev->new_alpha2, 2);
+ spin_unlock_bh(&ab->base_lock);
+
+ ath11k_dbg(ab, ATH11K_DBG_WMI, "wmi 11d new cc %c%c\n",
+ ab->new_alpha2[0],
+ ab->new_alpha2[1]);
+
+ kfree(tb);
+
+ queue_work(ab->workqueue, &ab->update_11d_work);
+
+ return 0;
+}
+
static void ath11k_wmi_htc_tx_complete(struct ath11k_base *ab,
struct sk_buff *skb)
{
+ struct ath11k_pdev_wmi *wmi = NULL;
+ u32 i;
+ u8 wmi_ep_count;
+ u8 eid;
+
+ eid = ATH11K_SKB_CB(skb)->eid;
dev_kfree_skb(skb);
+
+ if (eid >= ATH11K_HTC_EP_COUNT)
+ return;
+
+ wmi_ep_count = ab->htc.wmi_ep_count;
+ if (wmi_ep_count > ab->hw_params.max_radios)
+ return;
+
+ for (i = 0; i < ab->htc.wmi_ep_count; i++) {
+ if (ab->wmi_ab.wmi[i].eid == eid) {
+ wmi = &ab->wmi_ab.wmi[i];
+ break;
+ }
+ }
+
+ if (wmi)
+ wake_up(&wmi->tx_ce_desc_wq);
}
static bool ath11k_reg_is_world_alpha(char *alpha)
{
- return alpha[0] == '0' && alpha[1] == '0';
+ if (alpha[0] == '0' && alpha[1] == '0')
+ return true;
+
+ if (alpha[0] == 'n' && alpha[1] == 'a')
+ return true;
+
+ return false;
}
static int ath11k_reg_chan_list_event(struct ath11k_base *ab, struct sk_buff *skb)
@@ -5911,7 +6339,7 @@ static int ath11k_reg_chan_list_event(struct ath11k_base *ab, struct sk_buff *sk
ar = ab->pdevs[pdev_idx].ar;
kfree(ab->new_regd[pdev_idx]);
ab->new_regd[pdev_idx] = regd;
- ieee80211_queue_work(ar->hw, &ar->regd_update_work);
+ queue_work(ab->workqueue, &ar->regd_update_work);
} else {
/* This regd would be applied during mac registration and is
* held constant throughout for regd intersection purpose
@@ -6111,6 +6539,7 @@ static void ath11k_vdev_start_resp_event(struct ath11k_base *ab, struct sk_buff
static void ath11k_bcn_tx_status_event(struct ath11k_base *ab, struct sk_buff *skb)
{
+ struct ath11k_vif *arvif;
u32 vdev_id, tx_status;
if (ath11k_pull_bcn_tx_status_ev(ab, skb->data, skb->len,
@@ -6118,6 +6547,17 @@ static void ath11k_bcn_tx_status_event(struct ath11k_base *ab, struct sk_buff *s
ath11k_warn(ab, "failed to extract bcn tx status");
return;
}
+
+ rcu_read_lock();
+ arvif = ath11k_mac_get_arvif_by_vdev_id(ab, vdev_id);
+ if (!arvif) {
+ ath11k_warn(ab, "invalid vdev id %d in bcn_tx_status",
+ vdev_id);
+ rcu_read_unlock();
+ return;
+ }
+ ath11k_mac_bcn_tx_event(arvif);
+ rcu_read_unlock();
}
static void ath11k_vdev_stopped_event(struct ath11k_base *ab, struct sk_buff *skb)
@@ -6398,6 +6838,7 @@ static void ath11k_peer_sta_kickout_event(struct ath11k_base *ab, struct sk_buff
struct ieee80211_sta *sta;
struct ath11k_peer *peer;
struct ath11k *ar;
+ u32 vdev_id;
if (ath11k_pull_peer_sta_kickout_ev(ab, skb, &arg) != 0) {
ath11k_warn(ab, "failed to extract peer sta kickout event");
@@ -6413,10 +6854,15 @@ static void ath11k_peer_sta_kickout_event(struct ath11k_base *ab, struct sk_buff
if (!peer) {
ath11k_warn(ab, "peer not found %pM\n",
arg.mac_addr);
+ spin_unlock_bh(&ab->base_lock);
goto exit;
}
- ar = ath11k_mac_get_ar_by_vdev_id(ab, peer->vdev_id);
+ vdev_id = peer->vdev_id;
+
+ spin_unlock_bh(&ab->base_lock);
+
+ ar = ath11k_mac_get_ar_by_vdev_id(ab, vdev_id);
if (!ar) {
ath11k_warn(ab, "invalid vdev id in peer sta kickout ev %d",
peer->vdev_id);
@@ -6437,7 +6883,6 @@ static void ath11k_peer_sta_kickout_event(struct ath11k_base *ab, struct sk_buff
ieee80211_report_low_ack(sta, 10);
exit:
- spin_unlock_bh(&ab->base_lock);
rcu_read_unlock();
}
@@ -6893,6 +7338,40 @@ ath11k_wmi_pdev_dfs_radar_detected_event(struct ath11k_base *ab, struct sk_buff
kfree(tb);
}
+static void ath11k_rfkill_state_change_event(struct ath11k_base *ab,
+ struct sk_buff *skb)
+{
+ const struct wmi_rfkill_state_change_ev *ev;
+ const void **tb;
+ int ret;
+
+ tb = ath11k_wmi_tlv_parse_alloc(ab, skb->data, skb->len, GFP_ATOMIC);
+ if (IS_ERR(tb)) {
+ ret = PTR_ERR(tb);
+ ath11k_warn(ab, "failed to parse tlv: %d\n", ret);
+ return;
+ }
+
+ ev = tb[WMI_TAG_RFKILL_EVENT];
+ if (!ev) {
+ kfree(tb);
+ return;
+ }
+
+ ath11k_dbg(ab, ATH11K_DBG_MAC,
+ "wmi tlv rfkill state change gpio %d type %d radio_state %d\n",
+ ev->gpio_pin_num,
+ ev->int_type,
+ ev->radio_state);
+
+ spin_lock_bh(&ab->base_lock);
+ ab->rfkill_radio_on = (ev->radio_state == WMI_RFKILL_RADIO_STATE_ON);
+ spin_unlock_bh(&ab->base_lock);
+
+ queue_work(ab->workqueue, &ab->rfkill_work);
+ kfree(tb);
+}
+
static void
ath11k_wmi_pdev_temperature_event(struct ath11k_base *ab,
struct sk_buff *skb)
@@ -7046,6 +7525,13 @@ static void ath11k_wmi_event_wow_wakeup_host(struct ath11k_base *ab, struct sk_b
complete(&ab->wow.wakeup_completed);
}
+static void
+ath11k_wmi_diag_event(struct ath11k_base *ab,
+ struct sk_buff *skb)
+{
+ trace_ath11k_wmi_diag(ab, skb->data, skb->len);
+}
+
static void ath11k_wmi_tlv_op_rx(struct ath11k_base *ab, struct sk_buff *skb)
{
struct wmi_cmd_hdr *cmd_hdr;
@@ -7054,6 +7540,8 @@ static void ath11k_wmi_tlv_op_rx(struct ath11k_base *ab, struct sk_buff *skb)
cmd_hdr = (struct wmi_cmd_hdr *)skb->data;
id = FIELD_GET(WMI_CMD_HDR_CMD_ID, (cmd_hdr->cmd_id));
+ trace_ath11k_wmi_event(ab, id, skb->data, skb->len);
+
if (skb_pull(skb, sizeof(struct wmi_cmd_hdr)) == NULL)
goto out;
@@ -7138,6 +7626,9 @@ static void ath11k_wmi_tlv_op_rx(struct ath11k_base *ab, struct sk_buff *skb)
case WMI_OFFLOAD_PROB_RESP_TX_STATUS_EVENTID:
ath11k_probe_resp_tx_status_event(ab, skb);
break;
+ case WMI_OBSS_COLOR_COLLISION_DETECTION_EVENTID:
+ ath11k_wmi_obss_color_collision_event(ab, skb);
+ break;
/* add Unsupported events here */
case WMI_TBTTOFFSET_EXT_UPDATE_EVENTID:
case WMI_PEER_OPER_MODE_CHANGE_EVENTID:
@@ -7157,6 +7648,15 @@ static void ath11k_wmi_tlv_op_rx(struct ath11k_base *ab, struct sk_buff *skb)
case WMI_WOW_WAKEUP_HOST_EVENTID:
ath11k_wmi_event_wow_wakeup_host(ab, skb);
break;
+ case WMI_11D_NEW_COUNTRY_EVENTID:
+ ath11k_reg_11d_new_cc_event(ab, skb);
+ break;
+ case WMI_RFKILL_STATE_CHANGE_EVENTID:
+ ath11k_rfkill_state_change_event(ab, skb);
+ break;
+ case WMI_DIAG_EVENTID:
+ ath11k_wmi_diag_event(ab, skb);
+ break;
/* TODO: Add remaining events */
default:
ath11k_dbg(ab, ATH11K_DBG_WMI, "Unknown eventid: 0x%x\n", id);
@@ -7199,6 +7699,7 @@ static int ath11k_connect_pdev_htc_service(struct ath11k_base *ab,
ab->wmi_ab.wmi_endpoint_id[pdev_idx] = conn_resp.eid;
ab->wmi_ab.wmi[pdev_idx].eid = conn_resp.eid;
ab->wmi_ab.max_msg_len[pdev_idx] = conn_resp.max_msg_len;
+ init_waitqueue_head(&ab->wmi_ab.wmi[pdev_idx].tx_ce_desc_wq);
return 0;
}
@@ -7414,3 +7915,31 @@ int ath11k_wmi_wow_enable(struct ath11k *ar)
return ath11k_wmi_cmd_send(ar->wmi, skb, WMI_WOW_ENABLE_CMDID);
}
+
+int ath11k_wmi_scan_prob_req_oui(struct ath11k *ar,
+ const u8 mac_addr[ETH_ALEN])
+{
+ struct sk_buff *skb;
+ struct wmi_scan_prob_req_oui_cmd *cmd;
+ u32 prob_req_oui;
+ int len;
+
+ prob_req_oui = (((u32)mac_addr[0]) << 16) |
+ (((u32)mac_addr[1]) << 8) | mac_addr[2];
+
+ len = sizeof(*cmd);
+ skb = ath11k_wmi_alloc_skb(ar->wmi->wmi_ab, len);
+ if (!skb)
+ return -ENOMEM;
+
+ cmd = (struct wmi_scan_prob_req_oui_cmd *)skb->data;
+ cmd->tlv_header = FIELD_PREP(WMI_TLV_TAG,
+ WMI_TAG_SCAN_PROB_REQ_OUI_CMD) |
+ FIELD_PREP(WMI_TLV_LEN, sizeof(*cmd) - TLV_HDR_SIZE);
+ cmd->prob_req_oui = prob_req_oui;
+
+ ath11k_dbg(ar->ab, ATH11K_DBG_WMI, "wmi scan prob req oui %d\n",
+ prob_req_oui);
+
+ return ath11k_wmi_cmd_send(ar->wmi, skb, WMI_SCAN_PROB_REQ_OUI_CMDID);
+}
diff --git a/drivers/net/wireless/ath/ath11k/wmi.h b/drivers/net/wireless/ath/ath11k/wmi.h
index 0584e68..2f26ec1a 100644
--- a/drivers/net/wireless/ath/ath11k/wmi.h
+++ b/drivers/net/wireless/ath/ath11k/wmi.h
@@ -113,10 +113,10 @@ enum wmi_host_hw_mode_priority {
WMI_HOST_HW_MODE_MAX_PRI
};
-enum {
+enum WMI_HOST_WLAN_BAND {
WMI_HOST_WLAN_2G_CAP = 0x1,
WMI_HOST_WLAN_5G_CAP = 0x2,
- WMI_HOST_WLAN_2G_5G_CAP = 0x3,
+ WMI_HOST_WLAN_2G_5G_CAP = WMI_HOST_WLAN_2G_CAP | WMI_HOST_WLAN_5G_CAP,
};
/* Parameters used for WMI_VDEV_PARAM_AUTORATE_MISC_CFG command.
@@ -774,6 +774,8 @@ enum wmi_tlv_event_id {
WMI_MDNS_STATS_EVENTID = WMI_TLV_CMD(WMI_GRP_MDNS_OFL),
WMI_SAP_OFL_ADD_STA_EVENTID = WMI_TLV_CMD(WMI_GRP_SAP_OFL),
WMI_SAP_OFL_DEL_STA_EVENTID,
+ WMI_OBSS_COLOR_COLLISION_DETECTION_EVENTID =
+ WMI_EVT_GRP_START_ID(WMI_GRP_OBSS_OFL),
WMI_OCB_SET_CONFIG_RESP_EVENTID = WMI_TLV_CMD(WMI_GRP_OCB),
WMI_OCB_GET_TSF_TIMER_RESP_EVENTID,
WMI_DCC_GET_STATS_RESP_EVENTID,
@@ -2167,6 +2169,13 @@ enum wmi_nss_ratio {
WMI_NSS_RATIO_2_NSS = 0x3,
};
+enum wmi_dtim_policy {
+ WMI_DTIM_POLICY_IGNORE = 1,
+ WMI_DTIM_POLICY_NORMAL = 2,
+ WMI_DTIM_POLICY_STICK = 3,
+ WMI_DTIM_POLICY_AUTO = 4,
+};
+
struct wmi_host_pdev_band_to_mac {
u32 pdev_id;
u32 start_freq;
@@ -2522,6 +2531,7 @@ struct ath11k_pdev_wmi {
enum ath11k_htc_ep_id eid;
const struct wmi_peer_flags_map *peer_flags;
u32 rx_decap_mode;
+ wait_queue_head_t tx_ce_desc_wq;
};
struct vdev_create_params {
@@ -3079,7 +3089,6 @@ enum scan_dwelltime_adaptive_mode {
#define WLAN_SCAN_MAX_NUM_SSID 10
#define WLAN_SCAN_MAX_NUM_BSSID 10
-#define WLAN_SCAN_MAX_NUM_CHANNELS 40
#define WLAN_SSID_MAX_LEN 32
@@ -3300,7 +3309,7 @@ struct scan_req_params {
u32 num_bssid;
u32 num_ssids;
u32 n_probes;
- u32 chan_list[WLAN_SCAN_MAX_NUM_CHANNELS];
+ u32 *chan_list;
u32 notify_scan_events;
struct wlan_ssid ssid[WLAN_SCAN_MAX_NUM_SSID];
struct wmi_mac_addr bssid_list[WLAN_SCAN_MAX_NUM_BSSID];
@@ -3311,6 +3320,8 @@ struct scan_req_params {
u32 num_hint_bssid;
struct hint_short_ssid hint_s_ssid[WLAN_SCAN_MAX_HINT_S_SSID];
struct hint_bssid hint_bssid[WLAN_SCAN_MAX_HINT_BSSID];
+ struct wmi_mac_addr mac_addr;
+ struct wmi_mac_addr mac_mask;
};
struct wmi_ssid_arg {
@@ -3617,6 +3628,7 @@ struct peer_assoc_params {
u32 peer_he_tx_mcs_set[WMI_HOST_MAX_HE_RATE_SET];
bool twt_responder;
bool twt_requester;
+ bool is_assoc;
struct ath11k_ppe_threshold peer_ppet;
};
@@ -3673,6 +3685,11 @@ struct wmi_scan_chan_list_cmd {
u32 pdev_id;
} __packed;
+struct wmi_scan_prob_req_oui_cmd {
+ u32 tlv_header;
+ u32 prob_req_oui;
+} __packed;
+
#define WMI_MGMT_SEND_DOWNLD_LEN 64
#define WMI_TX_PARAMS_DWORD0_POWER GENMASK(7, 0)
@@ -3766,6 +3783,16 @@ struct stats_request_params {
u32 pdev_id;
};
+struct wmi_set_current_country_params {
+ u8 alpha2[3];
+};
+
+struct wmi_set_current_country_cmd {
+ u32 tlv_header;
+ u32 pdev_id;
+ u32 new_alpha2;
+} __packed;
+
enum set_init_cc_type {
WMI_COUNTRY_INFO_TYPE_ALPHA,
WMI_COUNTRY_INFO_TYPE_COUNTRY_CODE,
@@ -3799,6 +3826,28 @@ struct wmi_init_country_cmd {
} cc_info;
} __packed;
+struct wmi_11d_scan_start_params {
+ u32 vdev_id;
+ u32 scan_period_msec;
+ u32 start_interval_msec;
+};
+
+struct wmi_11d_scan_start_cmd {
+ u32 tlv_header;
+ u32 vdev_id;
+ u32 scan_period_msec;
+ u32 start_interval_msec;
+} __packed;
+
+struct wmi_11d_scan_stop_cmd {
+ u32 tlv_header;
+ u32 vdev_id;
+} __packed;
+
+struct wmi_11d_new_cc_ev {
+ u32 new_alpha2;
+} __packed;
+
#define THERMAL_LEVELS 1
struct tt_level_config {
u32 tmplwm;
@@ -4390,6 +4439,17 @@ struct wmi_stats_event {
u32 num_peer_extd2_stats;
} __packed;
+struct wmi_rssi_stats {
+ u32 vdev_id;
+ u32 rssi_avg_beacon[WMI_MAX_CHAINS];
+ u32 rssi_avg_data[WMI_MAX_CHAINS];
+ struct wmi_mac_addr peer_macaddr;
+} __packed;
+
+struct wmi_per_chain_rssi_stats {
+ u32 num_per_chain_rssi_stats;
+} __packed;
+
struct wmi_pdev_ctl_failsafe_chk_event {
u32 pdev_id;
u32 ctl_failsafe_status;
@@ -4914,6 +4974,13 @@ struct wmi_pdev_obss_pd_bitmap_cmd {
#define ATH11K_BSS_COLOR_COLLISION_DETECTION_STA_PERIOD_MS 10000
#define ATH11K_BSS_COLOR_COLLISION_DETECTION_AP_PERIOD_MS 5000
+enum wmi_bss_color_collision {
+ WMI_BSS_COLOR_COLLISION_DISABLE = 0,
+ WMI_BSS_COLOR_COLLISION_DETECTION,
+ WMI_BSS_COLOR_FREE_SLOT_TIMER_EXPIRY,
+ WMI_BSS_COLOR_FREE_SLOT_AVAILABLE,
+};
+
struct wmi_obss_color_collision_cfg_params_cmd {
u32 tlv_header;
u32 vdev_id;
@@ -4931,6 +4998,12 @@ struct wmi_bss_color_change_enable_params_cmd {
u32 enable;
} __packed;
+struct wmi_obss_color_collision_event {
+ u32 vdev_id;
+ u32 evt_type;
+ u64 obss_color_bitmap;
+} __packed;
+
#define ATH11K_IPV4_TH_SEED_SIZE 5
#define ATH11K_IPV6_TH_SEED_SIZE 11
@@ -5142,6 +5215,31 @@ struct target_resource_config {
u32 twt_ap_sta_count;
};
+enum wmi_sys_cap_info_flags {
+ WMI_SYS_CAP_INFO_RXTX_LED = BIT(0),
+ WMI_SYS_CAP_INFO_RFKILL = BIT(1),
+};
+
+#define WMI_RFKILL_CFG_GPIO_PIN_NUM GENMASK(5, 0)
+#define WMI_RFKILL_CFG_RADIO_LEVEL BIT(6)
+#define WMI_RFKILL_CFG_PIN_AS_GPIO GENMASK(10, 7)
+
+enum wmi_rfkill_enable_radio {
+ WMI_RFKILL_ENABLE_RADIO_ON = 0,
+ WMI_RFKILL_ENABLE_RADIO_OFF = 1,
+};
+
+enum wmi_rfkill_radio_state {
+ WMI_RFKILL_RADIO_STATE_OFF = 1,
+ WMI_RFKILL_RADIO_STATE_ON = 2,
+};
+
+struct wmi_rfkill_state_change_ev {
+ u32 gpio_pin_num;
+ u32 int_type;
+ u32 radio_state;
+} __packed;
+
#define WMI_MAX_MEM_REQS 32
#define MAX_RADIOS 3
@@ -5351,7 +5449,8 @@ int ath11k_wmi_set_peer_param(struct ath11k *ar, const u8 *peer_addr,
u32 vdev_id, u32 param_id, u32 param_val);
int ath11k_wmi_pdev_set_param(struct ath11k *ar, u32 param_id,
u32 param_value, u8 pdev_id);
-int ath11k_wmi_pdev_set_ps_mode(struct ath11k *ar, int vdev_id, u32 enable);
+int ath11k_wmi_pdev_set_ps_mode(struct ath11k *ar, int vdev_id,
+ enum wmi_sta_ps_mode psmode);
int ath11k_wmi_wait_for_unified_ready(struct ath11k_base *ab);
int ath11k_wmi_cmd_init(struct ath11k_base *ab);
int ath11k_wmi_wait_for_service_ready(struct ath11k_base *ab);
@@ -5415,9 +5514,16 @@ int ath11k_wmi_delba_send(struct ath11k *ar, u32 vdev_id, const u8 *mac,
u32 tid, u32 initiator, u32 reason);
int ath11k_wmi_send_bcn_offload_control_cmd(struct ath11k *ar,
u32 vdev_id, u32 bcn_ctrl_op);
+int ath11k_wmi_send_set_current_country_cmd(struct ath11k *ar,
+ struct wmi_set_current_country_params *param);
int
ath11k_wmi_send_init_country_cmd(struct ath11k *ar,
struct wmi_init_country_params init_cc_param);
+
+int ath11k_wmi_send_11d_scan_start_cmd(struct ath11k *ar,
+ struct wmi_11d_scan_start_params *param);
+int ath11k_wmi_send_11d_scan_stop_cmd(struct ath11k *ar, u32 vdev_id);
+
int
ath11k_wmi_send_thermal_mitigation_param_cmd(struct ath11k *ar,
struct thermal_mitigation_params *param);
@@ -5474,5 +5580,6 @@ int ath11k_wmi_set_hw_mode(struct ath11k_base *ab,
enum wmi_host_hw_mode_config_type mode);
int ath11k_wmi_wow_host_wakeup_ind(struct ath11k *ar);
int ath11k_wmi_wow_enable(struct ath11k *ar);
-
+int ath11k_wmi_scan_prob_req_oui(struct ath11k *ar,
+ const u8 mac_addr[ETH_ALEN]);
#endif
diff --git a/drivers/net/wireless/ath/ath5k/base.c b/drivers/net/wireless/ath/ath5k/base.c
index cef17f3..66d123f 100644
--- a/drivers/net/wireless/ath/ath5k/base.c
+++ b/drivers/net/wireless/ath/ath5k/base.c
@@ -727,6 +727,43 @@ ath5k_get_rate_hw_value(const struct ieee80211_hw *hw,
return hw_rate;
}
+static bool ath5k_merge_ratetbl(struct ieee80211_sta *sta,
+ struct ath5k_buf *bf,
+ struct ieee80211_tx_info *tx_info)
+{
+ struct ieee80211_sta_rates *ratetbl;
+ u8 i;
+
+ if (!sta)
+ return false;
+
+ ratetbl = rcu_dereference(sta->rates);
+ if (!ratetbl)
+ return false;
+
+ if (tx_info->control.rates[0].idx < 0 ||
+ tx_info->control.rates[0].count == 0)
+ {
+ i = 0;
+ } else {
+ bf->rates[0] = tx_info->control.rates[0];
+ i = 1;
+ }
+
+ for ( ; i < IEEE80211_TX_MAX_RATES; i++) {
+ bf->rates[i].idx = ratetbl->rate[i].idx;
+ bf->rates[i].flags = ratetbl->rate[i].flags;
+ if (tx_info->control.use_rts)
+ bf->rates[i].count = ratetbl->rate[i].count_rts;
+ else if (tx_info->control.use_cts_prot)
+ bf->rates[i].count = ratetbl->rate[i].count_cts;
+ else
+ bf->rates[i].count = ratetbl->rate[i].count;
+ }
+
+ return true;
+}
+
static int
ath5k_txbuf_setup(struct ath5k_hw *ah, struct ath5k_buf *bf,
struct ath5k_txq *txq, int padsize,
@@ -737,6 +774,7 @@ ath5k_txbuf_setup(struct ath5k_hw *ah, struct ath5k_buf *bf,
struct ieee80211_tx_info *info = IEEE80211_SKB_CB(skb);
unsigned int pktlen, flags, keyidx = AR5K_TXKEYIX_INVALID;
struct ieee80211_rate *rate;
+ struct ieee80211_sta *sta;
unsigned int mrr_rate[3], mrr_tries[3];
int i, ret;
u16 hw_rate;
@@ -753,8 +791,16 @@ ath5k_txbuf_setup(struct ath5k_hw *ah, struct ath5k_buf *bf,
if (dma_mapping_error(ah->dev, bf->skbaddr))
return -ENOSPC;
- ieee80211_get_tx_rates(info->control.vif, (control) ? control->sta : NULL, skb, bf->rates,
- ARRAY_SIZE(bf->rates));
+ if (control)
+ sta = control->sta;
+ else
+ sta = NULL;
+
+ if (!ath5k_merge_ratetbl(sta, bf, info)) {
+ ieee80211_get_tx_rates(info->control.vif,
+ sta, skb, bf->rates,
+ ARRAY_SIZE(bf->rates));
+ }
rate = ath5k_get_rate(ah->hw, info, bf, 0);
diff --git a/drivers/net/wireless/ath/ath6kl/htc.h b/drivers/net/wireless/ath/ath6kl/htc.h
index 112d8a9..d3534a2 100644
--- a/drivers/net/wireless/ath/ath6kl/htc.h
+++ b/drivers/net/wireless/ath/ath6kl/htc.h
@@ -153,12 +153,19 @@
* implementations.
*/
struct htc_frame_hdr {
- u8 eid;
- u8 flags;
+ struct_group_tagged(htc_frame_look_ahead, header,
+ union {
+ struct {
+ u8 eid;
+ u8 flags;
- /* length of data (including trailer) that follows the header */
- __le16 payld_len;
+ /* length of data (including trailer) that follows the header */
+ __le16 payld_len;
+ };
+ u32 word;
+ };
+ );
/* end of 4-byte lookahead */
u8 ctrl[2];
diff --git a/drivers/net/wireless/ath/ath6kl/htc_mbox.c b/drivers/net/wireless/ath/ath6kl/htc_mbox.c
index 998947e..e387442 100644
--- a/drivers/net/wireless/ath/ath6kl/htc_mbox.c
+++ b/drivers/net/wireless/ath/ath6kl/htc_mbox.c
@@ -2260,19 +2260,16 @@ int ath6kl_htc_rxmsg_pending_handler(struct htc_target *target,
static struct htc_packet *htc_wait_for_ctrl_msg(struct htc_target *target)
{
struct htc_packet *packet = NULL;
- struct htc_frame_hdr *htc_hdr;
- u32 look_ahead;
+ struct htc_frame_look_ahead look_ahead;
- if (ath6kl_hif_poll_mboxmsg_rx(target->dev, &look_ahead,
+ if (ath6kl_hif_poll_mboxmsg_rx(target->dev, &look_ahead.word,
HTC_TARGET_RESPONSE_TIMEOUT))
return NULL;
ath6kl_dbg(ATH6KL_DBG_HTC,
- "htc rx wait ctrl look_ahead 0x%X\n", look_ahead);
+ "htc rx wait ctrl look_ahead 0x%X\n", look_ahead.word);
- htc_hdr = (struct htc_frame_hdr *)&look_ahead;
-
- if (htc_hdr->eid != ENDPOINT_0)
+ if (look_ahead.eid != ENDPOINT_0)
return NULL;
packet = htc_get_control_buf(target, false);
@@ -2281,8 +2278,8 @@ static struct htc_packet *htc_wait_for_ctrl_msg(struct htc_target *target)
return NULL;
packet->info.rx.rx_flags = 0;
- packet->info.rx.exp_hdr = look_ahead;
- packet->act_len = le16_to_cpu(htc_hdr->payld_len) + HTC_HDR_LENGTH;
+ packet->info.rx.exp_hdr = look_ahead.word;
+ packet->act_len = le16_to_cpu(look_ahead.payld_len) + HTC_HDR_LENGTH;
if (packet->act_len > packet->buf_len)
goto fail_ctrl_rx;
diff --git a/drivers/net/wireless/ath/ath9k/ar9002_mac.c b/drivers/net/wireless/ath/ath9k/ar9002_mac.c
index ce9a0a53..fba5a84 100644
--- a/drivers/net/wireless/ath/ath9k/ar9002_mac.c
+++ b/drivers/net/wireless/ath/ath9k/ar9002_mac.c
@@ -120,7 +120,7 @@ static bool ar9002_hw_get_isr(struct ath_hw *ah, enum ath9k_int *masked,
AR_ISR_TXEOL);
}
- ah->intr_txqs |= MS(s0_s, AR_ISR_S0_QCU_TXOK);
+ ah->intr_txqs = MS(s0_s, AR_ISR_S0_QCU_TXOK);
ah->intr_txqs |= MS(s0_s, AR_ISR_S0_QCU_TXDESC);
ah->intr_txqs |= MS(s1_s, AR_ISR_S1_QCU_TXERR);
ah->intr_txqs |= MS(s1_s, AR_ISR_S1_QCU_TXEOL);
diff --git a/drivers/net/wireless/ath/ath9k/ar9003_calib.c b/drivers/net/wireless/ath/ath9k/ar9003_calib.c
index 7e27a06..dc24da1 100644
--- a/drivers/net/wireless/ath/ath9k/ar9003_calib.c
+++ b/drivers/net/wireless/ath/ath9k/ar9003_calib.c
@@ -1005,24 +1005,20 @@ static void __ar955x_tx_iq_cal_sort(struct ath_hw *ah,
int i, int nmeasurement)
{
struct ath_common *common = ath9k_hw_common(ah);
- int im, ix, iy, temp;
+ int im, ix, iy;
for (im = 0; im < nmeasurement; im++) {
for (ix = 0; ix < MAXIQCAL - 1; ix++) {
for (iy = ix + 1; iy <= MAXIQCAL - 1; iy++) {
if (coeff->mag_coeff[i][im][iy] <
coeff->mag_coeff[i][im][ix]) {
- temp = coeff->mag_coeff[i][im][ix];
- coeff->mag_coeff[i][im][ix] =
- coeff->mag_coeff[i][im][iy];
- coeff->mag_coeff[i][im][iy] = temp;
+ swap(coeff->mag_coeff[i][im][ix],
+ coeff->mag_coeff[i][im][iy]);
}
if (coeff->phs_coeff[i][im][iy] <
coeff->phs_coeff[i][im][ix]) {
- temp = coeff->phs_coeff[i][im][ix];
- coeff->phs_coeff[i][im][ix] =
- coeff->phs_coeff[i][im][iy];
- coeff->phs_coeff[i][im][iy] = temp;
+ swap(coeff->phs_coeff[i][im][ix],
+ coeff->phs_coeff[i][im][iy]);
}
}
}
diff --git a/drivers/net/wireless/ath/ath9k/hif_usb.c b/drivers/net/wireless/ath/ath9k/hif_usb.c
index 860da13..f06eec9 100644
--- a/drivers/net/wireless/ath/ath9k/hif_usb.c
+++ b/drivers/net/wireless/ath/ath9k/hif_usb.c
@@ -590,6 +590,13 @@ static void ath9k_hif_usb_rx_stream(struct hif_device_usb *hif_dev,
return;
}
+ if (pkt_len > 2 * MAX_RX_BUF_SIZE) {
+ dev_err(&hif_dev->udev->dev,
+ "ath9k_htc: invalid pkt_len (%x)\n", pkt_len);
+ RX_STAT_INC(skb_dropped);
+ return;
+ }
+
pad_len = 4 - (pkt_len & 0x3);
if (pad_len == 4)
pad_len = 0;
diff --git a/drivers/net/wireless/ath/ath9k/htc.h b/drivers/net/wireless/ath/ath9k/htc.h
index 0a16342..6b45e63 100644
--- a/drivers/net/wireless/ath/ath9k/htc.h
+++ b/drivers/net/wireless/ath/ath9k/htc.h
@@ -281,6 +281,7 @@ struct ath9k_htc_rxbuf {
struct ath9k_htc_rx {
struct list_head rxbuf;
spinlock_t rxbuflock;
+ bool initialized;
};
#define ATH9K_HTC_TX_CLEANUP_INTERVAL 50 /* ms */
@@ -305,6 +306,7 @@ struct ath9k_htc_tx {
DECLARE_BITMAP(tx_slot, MAX_TX_BUF_NUM);
struct timer_list cleanup_timer;
spinlock_t tx_lock;
+ bool initialized;
};
struct ath9k_htc_tx_ctl {
diff --git a/drivers/net/wireless/ath/ath9k/htc_drv_txrx.c b/drivers/net/wireless/ath/ath9k/htc_drv_txrx.c
index 8e69e898..6a850a0 100644
--- a/drivers/net/wireless/ath/ath9k/htc_drv_txrx.c
+++ b/drivers/net/wireless/ath/ath9k/htc_drv_txrx.c
@@ -813,6 +813,11 @@ int ath9k_tx_init(struct ath9k_htc_priv *priv)
skb_queue_head_init(&priv->tx.data_vi_queue);
skb_queue_head_init(&priv->tx.data_vo_queue);
skb_queue_head_init(&priv->tx.tx_failed);
+
+ /* Allow ath9k_wmi_event_tasklet(WMI_TXSTATUS_EVENTID) to operate. */
+ smp_wmb();
+ priv->tx.initialized = true;
+
return 0;
}
@@ -1130,6 +1135,10 @@ void ath9k_htc_rxep(void *drv_priv, struct sk_buff *skb,
struct ath9k_htc_rxbuf *rxbuf = NULL, *tmp_buf = NULL;
unsigned long flags;
+ /* Check if ath9k_rx_init() completed. */
+ if (!data_race(priv->rx.initialized))
+ goto err;
+
spin_lock_irqsave(&priv->rx.rxbuflock, flags);
list_for_each_entry(tmp_buf, &priv->rx.rxbuf, list) {
if (!tmp_buf->in_process) {
@@ -1185,6 +1194,10 @@ int ath9k_rx_init(struct ath9k_htc_priv *priv)
list_add_tail(&rxbuf->list, &priv->rx.rxbuf);
}
+ /* Allow ath9k_htc_rxep() to operate. */
+ smp_wmb();
+ priv->rx.initialized = true;
+
return 0;
err:
diff --git a/drivers/net/wireless/ath/ath9k/wmi.c b/drivers/net/wireless/ath/ath9k/wmi.c
index fe29ad4..f315c54 100644
--- a/drivers/net/wireless/ath/ath9k/wmi.c
+++ b/drivers/net/wireless/ath/ath9k/wmi.c
@@ -169,6 +169,10 @@ void ath9k_wmi_event_tasklet(struct tasklet_struct *t)
&wmi->drv_priv->fatal_work);
break;
case WMI_TXSTATUS_EVENTID:
+ /* Check if ath9k_tx_init() completed. */
+ if (!data_race(priv->tx.initialized))
+ break;
+
spin_lock_bh(&priv->tx.tx_lock);
if (priv->tx.flags & ATH9K_HTC_OP_TX_DRAIN) {
spin_unlock_bh(&priv->tx.tx_lock);
diff --git a/drivers/net/wireless/ath/ath9k/xmit.c b/drivers/net/wireless/ath/ath9k/xmit.c
index 5691bd6..d0caf1d 100644
--- a/drivers/net/wireless/ath/ath9k/xmit.c
+++ b/drivers/net/wireless/ath/ath9k/xmit.c
@@ -154,11 +154,52 @@ static void ath_send_bar(struct ath_atx_tid *tid, u16 seqno)
seqno << IEEE80211_SEQ_SEQ_SHIFT);
}
+static bool ath_merge_ratetbl(struct ieee80211_sta *sta, struct ath_buf *bf,
+ struct ieee80211_tx_info *tx_info)
+{
+ struct ieee80211_sta_rates *ratetbl;
+ u8 i;
+
+ if (!sta)
+ return false;
+
+ ratetbl = rcu_dereference(sta->rates);
+ if (!ratetbl)
+ return false;
+
+ if (tx_info->control.rates[0].idx < 0 ||
+ tx_info->control.rates[0].count == 0)
+ {
+ i = 0;
+ } else {
+ bf->rates[0] = tx_info->control.rates[0];
+ i = 1;
+ }
+
+ for ( ; i < IEEE80211_TX_MAX_RATES; i++) {
+ bf->rates[i].idx = ratetbl->rate[i].idx;
+ bf->rates[i].flags = ratetbl->rate[i].flags;
+ if (tx_info->control.use_rts)
+ bf->rates[i].count = ratetbl->rate[i].count_rts;
+ else if (tx_info->control.use_cts_prot)
+ bf->rates[i].count = ratetbl->rate[i].count_cts;
+ else
+ bf->rates[i].count = ratetbl->rate[i].count;
+ }
+
+ return true;
+}
+
static void ath_set_rates(struct ieee80211_vif *vif, struct ieee80211_sta *sta,
struct ath_buf *bf)
{
- ieee80211_get_tx_rates(vif, sta, bf->bf_mpdu, bf->rates,
- ARRAY_SIZE(bf->rates));
+ struct ieee80211_tx_info *tx_info;
+
+ tx_info = IEEE80211_SKB_CB(bf->bf_mpdu);
+
+ if (!ath_merge_ratetbl(sta, bf, tx_info))
+ ieee80211_get_tx_rates(vif, sta, bf->bf_mpdu, bf->rates,
+ ARRAY_SIZE(bf->rates));
}
static void ath_txq_skb_done(struct ath_softc *sc, struct ath_txq *txq,
diff --git a/drivers/net/wireless/ath/carl9170/main.c b/drivers/net/wireless/ath/carl9170/main.c
index cca3b08..49f7ee1 100644
--- a/drivers/net/wireless/ath/carl9170/main.c
+++ b/drivers/net/wireless/ath/carl9170/main.c
@@ -307,8 +307,7 @@ static void carl9170_zap_queues(struct ar9170 *ar)
for (i = 0; i < ar->hw->queues; i++)
ar->tx_stats[i].limit = CARL9170_NUM_TX_LIMIT_HARD;
- for (i = 0; i < DIV_ROUND_UP(ar->fw.mem_blocks, BITS_PER_LONG); i++)
- ar->mem_bitmap[i] = 0;
+ bitmap_zero(ar->mem_bitmap, ar->fw.mem_blocks);
rcu_read_lock();
list_for_each_entry_rcu(cvif, &ar->vif_list, list) {
@@ -1968,9 +1967,7 @@ int carl9170_register(struct ar9170 *ar)
if (WARN_ON(ar->mem_bitmap))
return -EINVAL;
- ar->mem_bitmap = kcalloc(roundup(ar->fw.mem_blocks, BITS_PER_LONG),
- sizeof(unsigned long),
- GFP_KERNEL);
+ ar->mem_bitmap = bitmap_zalloc(ar->fw.mem_blocks, GFP_KERNEL);
if (!ar->mem_bitmap)
return -ENOMEM;
@@ -2085,7 +2082,7 @@ void carl9170_free(struct ar9170 *ar)
kfree_skb(ar->rx_failover);
ar->rx_failover = NULL;
- kfree(ar->mem_bitmap);
+ bitmap_free(ar->mem_bitmap);
ar->mem_bitmap = NULL;
kfree(ar->survey);
diff --git a/drivers/net/wireless/ath/carl9170/tx.c b/drivers/net/wireless/ath/carl9170/tx.c
index 88444fe..1b76f44 100644
--- a/drivers/net/wireless/ath/carl9170/tx.c
+++ b/drivers/net/wireless/ath/carl9170/tx.c
@@ -275,12 +275,12 @@ static void carl9170_tx_release(struct kref *ref)
if (WARN_ON_ONCE(!ar))
return;
- BUILD_BUG_ON(
- offsetof(struct ieee80211_tx_info, status.ack_signal) != 20);
-
- memset(&txinfo->status.ack_signal, 0,
- sizeof(struct ieee80211_tx_info) -
- offsetof(struct ieee80211_tx_info, status.ack_signal));
+ /*
+ * This does not call ieee80211_tx_info_clear_status() because
+ * carl9170_tx_fill_rateinfo() has filled the rate information
+ * before we get to this point.
+ */
+ memset_after(&txinfo->status, 0, rates);
if (atomic_read(&ar->tx_total_queued))
ar->tx_schedule = true;
diff --git a/drivers/net/wireless/ath/regd.h b/drivers/net/wireless/ath/regd.h
index 8d5a16b..774419c 100644
--- a/drivers/net/wireless/ath/regd.h
+++ b/drivers/net/wireless/ath/regd.h
@@ -126,6 +126,7 @@ enum CountryCode {
CTRY_KOREA_ROC = 410,
CTRY_KOREA_ROC2 = 411,
CTRY_KOREA_ROC3 = 412,
+ CTRY_KOREA_ROC4 = 413,
CTRY_KUWAIT = 414,
CTRY_LATVIA = 428,
CTRY_LEBANON = 422,
diff --git a/drivers/net/wireless/ath/regd_common.h b/drivers/net/wireless/ath/regd_common.h
index c4bd26e..cdb1e9a 100644
--- a/drivers/net/wireless/ath/regd_common.h
+++ b/drivers/net/wireless/ath/regd_common.h
@@ -76,6 +76,7 @@ enum EnumRd {
APL7_FCCA = 0x5C,
APL8_WORLD = 0x5D,
APL9_WORLD = 0x5E,
+ APL10_WORLD = 0x5F,
WOR0_WORLD = 0x60,
WOR1_WORLD = 0x61,
@@ -204,6 +205,7 @@ static struct reg_dmn_pair_mapping regDomainPairs[] = {
{APL6_WORLD, CTL_ETSI, CTL_ETSI},
{APL8_WORLD, CTL_ETSI, CTL_ETSI},
{APL9_WORLD, CTL_ETSI, CTL_ETSI},
+ {APL10_WORLD, CTL_ETSI, CTL_ETSI},
{APL3_FCCA, CTL_FCC, CTL_FCC},
{APL7_FCCA, CTL_FCC, CTL_FCC},
@@ -426,6 +428,7 @@ static struct country_code_to_enum_rd allCountries[] = {
{CTRY_KOREA_ROC, APL9_WORLD, "KR"},
{CTRY_KOREA_ROC2, APL2_WORLD, "K2"},
{CTRY_KOREA_ROC3, APL9_WORLD, "K3"},
+ {CTRY_KOREA_ROC4, APL10_WORLD, "K4"},
{CTRY_KUWAIT, ETSI3_WORLD, "KW"},
{CTRY_LATVIA, ETSI1_WORLD, "LV"},
{CTRY_LEBANON, NULL1_WORLD, "LB"},
diff --git a/drivers/net/wireless/ath/wcn36xx/dxe.c b/drivers/net/wireless/ath/wcn36xx/dxe.c
index aff04ef..4e9e139 100644
--- a/drivers/net/wireless/ath/wcn36xx/dxe.c
+++ b/drivers/net/wireless/ath/wcn36xx/dxe.c
@@ -272,6 +272,21 @@ static int wcn36xx_dxe_enable_ch_int(struct wcn36xx *wcn, u16 wcn_ch)
return 0;
}
+static void wcn36xx_dxe_disable_ch_int(struct wcn36xx *wcn, u16 wcn_ch)
+{
+ int reg_data = 0;
+
+ wcn36xx_dxe_read_register(wcn,
+ WCN36XX_DXE_INT_MASK_REG,
+ ®_data);
+
+ reg_data &= ~wcn_ch;
+
+ wcn36xx_dxe_write_register(wcn,
+ WCN36XX_DXE_INT_MASK_REG,
+ (int)reg_data);
+}
+
static int wcn36xx_dxe_fill_skb(struct device *dev,
struct wcn36xx_dxe_ctl *ctl,
gfp_t gfp)
@@ -834,6 +849,53 @@ int wcn36xx_dxe_tx_frame(struct wcn36xx *wcn,
return ret;
}
+static bool _wcn36xx_dxe_tx_channel_is_empty(struct wcn36xx_dxe_ch *ch)
+{
+ unsigned long flags;
+ struct wcn36xx_dxe_ctl *ctl_bd_start, *ctl_skb_start;
+ struct wcn36xx_dxe_ctl *ctl_bd, *ctl_skb;
+ bool ret = true;
+
+ spin_lock_irqsave(&ch->lock, flags);
+
+ /* Loop through ring buffer looking for nonempty entries. */
+ ctl_bd_start = ch->head_blk_ctl;
+ ctl_bd = ctl_bd_start;
+ ctl_skb_start = ctl_bd_start->next;
+ ctl_skb = ctl_skb_start;
+ do {
+ if (ctl_skb->skb) {
+ ret = false;
+ goto unlock;
+ }
+ ctl_bd = ctl_skb->next;
+ ctl_skb = ctl_bd->next;
+ } while (ctl_skb != ctl_skb_start);
+
+unlock:
+ spin_unlock_irqrestore(&ch->lock, flags);
+ return ret;
+}
+
+int wcn36xx_dxe_tx_flush(struct wcn36xx *wcn)
+{
+ int i = 0;
+
+ /* Called with mac80211 queues stopped. Wait for empty HW queues. */
+ do {
+ if (_wcn36xx_dxe_tx_channel_is_empty(&wcn->dxe_tx_l_ch) &&
+ _wcn36xx_dxe_tx_channel_is_empty(&wcn->dxe_tx_h_ch)) {
+ return 0;
+ }
+ /* This ieee80211_ops callback is specifically allowed to
+ * sleep.
+ */
+ usleep_range(1000, 1100);
+ } while (++i < 100);
+
+ return -EBUSY;
+}
+
int wcn36xx_dxe_init(struct wcn36xx *wcn)
{
int reg_data = 0, ret;
@@ -869,7 +931,6 @@ int wcn36xx_dxe_init(struct wcn36xx *wcn)
WCN36XX_DXE_WQ_TX_L);
wcn36xx_dxe_read_register(wcn, WCN36XX_DXE_REG_CH_EN, ®_data);
- wcn36xx_dxe_enable_ch_int(wcn, WCN36XX_INT_MASK_CHAN_TX_L);
/***************************************/
/* Init descriptors for TX HIGH channel */
@@ -893,9 +954,6 @@ int wcn36xx_dxe_init(struct wcn36xx *wcn)
wcn36xx_dxe_read_register(wcn, WCN36XX_DXE_REG_CH_EN, ®_data);
- /* Enable channel interrupts */
- wcn36xx_dxe_enable_ch_int(wcn, WCN36XX_INT_MASK_CHAN_TX_H);
-
/***************************************/
/* Init descriptors for RX LOW channel */
/***************************************/
@@ -905,7 +963,6 @@ int wcn36xx_dxe_init(struct wcn36xx *wcn)
goto out_err_rxl_ch;
}
-
/* For RX we need to preallocated buffers */
wcn36xx_dxe_ch_alloc_skb(wcn, &wcn->dxe_rx_l_ch);
@@ -928,9 +985,6 @@ int wcn36xx_dxe_init(struct wcn36xx *wcn)
WCN36XX_DXE_REG_CTL_RX_L,
WCN36XX_DXE_CH_DEFAULT_CTL_RX_L);
- /* Enable channel interrupts */
- wcn36xx_dxe_enable_ch_int(wcn, WCN36XX_INT_MASK_CHAN_RX_L);
-
/***************************************/
/* Init descriptors for RX HIGH channel */
/***************************************/
@@ -962,15 +1016,18 @@ int wcn36xx_dxe_init(struct wcn36xx *wcn)
WCN36XX_DXE_REG_CTL_RX_H,
WCN36XX_DXE_CH_DEFAULT_CTL_RX_H);
- /* Enable channel interrupts */
- wcn36xx_dxe_enable_ch_int(wcn, WCN36XX_INT_MASK_CHAN_RX_H);
-
ret = wcn36xx_dxe_request_irqs(wcn);
if (ret < 0)
goto out_err_irq;
timer_setup(&wcn->tx_ack_timer, wcn36xx_dxe_tx_timer, 0);
+ /* Enable channel interrupts */
+ wcn36xx_dxe_enable_ch_int(wcn, WCN36XX_INT_MASK_CHAN_TX_L);
+ wcn36xx_dxe_enable_ch_int(wcn, WCN36XX_INT_MASK_CHAN_TX_H);
+ wcn36xx_dxe_enable_ch_int(wcn, WCN36XX_INT_MASK_CHAN_RX_L);
+ wcn36xx_dxe_enable_ch_int(wcn, WCN36XX_INT_MASK_CHAN_RX_H);
+
return 0;
out_err_irq:
@@ -987,6 +1044,14 @@ int wcn36xx_dxe_init(struct wcn36xx *wcn)
void wcn36xx_dxe_deinit(struct wcn36xx *wcn)
{
+ int reg_data = 0;
+
+ /* Disable channel interrupts */
+ wcn36xx_dxe_disable_ch_int(wcn, WCN36XX_INT_MASK_CHAN_RX_H);
+ wcn36xx_dxe_disable_ch_int(wcn, WCN36XX_INT_MASK_CHAN_RX_L);
+ wcn36xx_dxe_disable_ch_int(wcn, WCN36XX_INT_MASK_CHAN_TX_H);
+ wcn36xx_dxe_disable_ch_int(wcn, WCN36XX_INT_MASK_CHAN_TX_L);
+
free_irq(wcn->tx_irq, wcn);
free_irq(wcn->rx_irq, wcn);
del_timer(&wcn->tx_ack_timer);
@@ -996,6 +1061,15 @@ void wcn36xx_dxe_deinit(struct wcn36xx *wcn)
wcn->tx_ack_skb = NULL;
}
+ /* Put the DXE block into reset before freeing memory */
+ reg_data = WCN36XX_DXE_REG_RESET;
+ wcn36xx_dxe_write_register(wcn, WCN36XX_DXE_REG_CSR_RESET, reg_data);
+
wcn36xx_dxe_ch_free_skbs(wcn, &wcn->dxe_rx_l_ch);
wcn36xx_dxe_ch_free_skbs(wcn, &wcn->dxe_rx_h_ch);
+
+ wcn36xx_dxe_deinit_descs(wcn->dev, &wcn->dxe_tx_l_ch);
+ wcn36xx_dxe_deinit_descs(wcn->dev, &wcn->dxe_tx_h_ch);
+ wcn36xx_dxe_deinit_descs(wcn->dev, &wcn->dxe_rx_l_ch);
+ wcn36xx_dxe_deinit_descs(wcn->dev, &wcn->dxe_rx_h_ch);
}
diff --git a/drivers/net/wireless/ath/wcn36xx/dxe.h b/drivers/net/wireless/ath/wcn36xx/dxe.h
index 31b81b7..26a31ed 100644
--- a/drivers/net/wireless/ath/wcn36xx/dxe.h
+++ b/drivers/net/wireless/ath/wcn36xx/dxe.h
@@ -466,5 +466,6 @@ int wcn36xx_dxe_tx_frame(struct wcn36xx *wcn,
struct wcn36xx_tx_bd *bd,
struct sk_buff *skb,
bool is_low);
+int wcn36xx_dxe_tx_flush(struct wcn36xx *wcn);
void wcn36xx_dxe_tx_ack_ind(struct wcn36xx *wcn, u32 status);
#endif /* _DXE_H_ */
diff --git a/drivers/net/wireless/ath/wcn36xx/hal.h b/drivers/net/wireless/ath/wcn36xx/hal.h
index 9bea2b0..2a1db97 100644
--- a/drivers/net/wireless/ath/wcn36xx/hal.h
+++ b/drivers/net/wireless/ath/wcn36xx/hal.h
@@ -3459,9 +3459,6 @@ struct wcn36xx_hal_missed_beacon_ind_msg {
/* Beacon Filtering data structures */
-/* The above structure would be followed by multiple of below mentioned
- * structure
- */
struct beacon_filter_ie {
u8 element_id;
u8 check_ie_presence;
@@ -3469,7 +3466,27 @@ struct beacon_filter_ie {
u8 value;
u8 bitmask;
u8 ref;
-};
+} __packed;
+
+#define WCN36XX_FILTER_CAPABILITY_MASK 0x73cf
+#define WCN36XX_FILTER_IE_DS_CHANNEL_MASK 0x00
+#define WCN36XX_FILTER_IE_ERP_FILTER_MASK 0xF8
+#define WCN36XX_FILTER_IE_EDCA_FILTER_MASK 0xF0
+#define WCN36XX_FILTER_IE_QOS_FILTER_MASK 0xF0
+#define WCN36XX_FILTER_IE_CHANNEL_SWITCH_MASK 0x00
+#define WCN36XX_FILTER_IE_HT_BYTE0_FILTER_MASK 0x00
+#define WCN36XX_FILTER_IE_HT_BYTE1_FILTER_MASK 0xF8
+#define WCN36XX_FILTER_IE_HT_BYTE2_FILTER_MASK 0xEB
+#define WCN36XX_FILTER_IE_HT_BYTE5_FILTER_MASK 0xFD
+#define WCN36XX_FILTER_IE_PWR_CONSTRAINT_MASK 0x00
+#define WCN36XX_FILTER_IE_OPMODE_NOTIF_MASK 0x00
+#define WCN36XX_FILTER_IE_VHTOP_CHWIDTH_MASK 0xFC
+#define WCN36XX_FILTER_IE_RSN_MASK 0x00
+#define WCN36XX_FILTER_IE_VENDOR_MASK 0x00
+
+/* The above structure would be followed by multiple of below mentioned
+ * structure
+ */
struct wcn36xx_hal_add_bcn_filter_req_msg {
struct wcn36xx_hal_msg_header header;
@@ -3480,14 +3497,14 @@ struct wcn36xx_hal_add_bcn_filter_req_msg {
u16 ie_num;
u8 bss_index;
u8 reserved;
-};
+} __packed;
struct wcn36xx_hal_rem_bcn_filter_req {
struct wcn36xx_hal_msg_header header;
u8 ie_Count;
u8 rem_ie_id[1];
-};
+} __packed;
#define WCN36XX_HAL_IPV4_ARP_REPLY_OFFLOAD 0
#define WCN36XX_HAL_IPV6_NEIGHBOR_DISCOVERY_OFFLOAD 1
diff --git a/drivers/net/wireless/ath/wcn36xx/main.c b/drivers/net/wireless/ath/wcn36xx/main.c
index b04533b..9575d73 100644
--- a/drivers/net/wireless/ath/wcn36xx/main.c
+++ b/drivers/net/wireless/ath/wcn36xx/main.c
@@ -402,6 +402,7 @@ static void wcn36xx_change_opchannel(struct wcn36xx *wcn, int ch)
static int wcn36xx_config(struct ieee80211_hw *hw, u32 changed)
{
struct wcn36xx *wcn = hw->priv;
+ int ret;
wcn36xx_dbg(WCN36XX_DBG_MAC, "mac config changed 0x%08x\n", changed);
@@ -417,17 +418,31 @@ static int wcn36xx_config(struct ieee80211_hw *hw, u32 changed)
* want to receive/transmit regular data packets, then
* simply stop the scan session and exit PS mode.
*/
- wcn36xx_smd_finish_scan(wcn, HAL_SYS_MODE_SCAN,
- wcn->sw_scan_vif);
- wcn->sw_scan_channel = 0;
+ if (wcn->sw_scan_channel)
+ wcn36xx_smd_end_scan(wcn, wcn->sw_scan_channel);
+ if (wcn->sw_scan_init) {
+ wcn36xx_smd_finish_scan(wcn, HAL_SYS_MODE_SCAN,
+ wcn->sw_scan_vif);
+ }
} else if (wcn->sw_scan) {
/* A scan is ongoing, do not change the operating
* channel, but start a scan session on the channel.
*/
- wcn36xx_smd_init_scan(wcn, HAL_SYS_MODE_SCAN,
- wcn->sw_scan_vif);
+ if (wcn->sw_scan_channel)
+ wcn36xx_smd_end_scan(wcn, wcn->sw_scan_channel);
+ if (!wcn->sw_scan_init) {
+ /* This can fail if we are unable to notify the
+ * operating channel.
+ */
+ ret = wcn36xx_smd_init_scan(wcn,
+ HAL_SYS_MODE_SCAN,
+ wcn->sw_scan_vif);
+ if (ret) {
+ mutex_unlock(&wcn->conf_mutex);
+ return -EIO;
+ }
+ }
wcn36xx_smd_start_scan(wcn, ch);
- wcn->sw_scan_channel = ch;
} else {
wcn36xx_change_opchannel(wcn, ch);
}
@@ -707,6 +722,8 @@ static void wcn36xx_sw_scan_start(struct ieee80211_hw *hw,
struct wcn36xx *wcn = hw->priv;
struct wcn36xx_vif *vif_priv = wcn36xx_vif_to_priv(vif);
+ wcn36xx_dbg(WCN36XX_DBG_MAC, "sw_scan_start");
+
wcn->sw_scan = true;
wcn->sw_scan_vif = vif;
wcn->sw_scan_channel = 0;
@@ -721,8 +738,15 @@ static void wcn36xx_sw_scan_complete(struct ieee80211_hw *hw,
{
struct wcn36xx *wcn = hw->priv;
+ wcn36xx_dbg(WCN36XX_DBG_MAC, "sw_scan_complete");
+
/* ensure that any scan session is finished */
- wcn36xx_smd_finish_scan(wcn, HAL_SYS_MODE_SCAN, wcn->sw_scan_vif);
+ if (wcn->sw_scan_channel)
+ wcn36xx_smd_end_scan(wcn, wcn->sw_scan_channel);
+ if (wcn->sw_scan_init) {
+ wcn36xx_smd_finish_scan(wcn, HAL_SYS_MODE_SCAN,
+ wcn->sw_scan_vif);
+ }
wcn->sw_scan = false;
wcn->sw_scan_opchannel = 0;
}
@@ -910,6 +934,8 @@ static void wcn36xx_bss_info_changed(struct ieee80211_hw *hw,
* place where AID is available.
*/
wcn36xx_smd_config_sta(wcn, vif, sta);
+ if (vif->type == NL80211_IFTYPE_STATION)
+ wcn36xx_smd_add_beacon_filter(wcn, vif);
wcn36xx_enable_keep_alive_null_packet(wcn, vif);
} else {
wcn36xx_dbg(WCN36XX_DBG_MAC,
@@ -1196,7 +1222,7 @@ static int wcn36xx_ampdu_action(struct ieee80211_hw *hw,
u16 tid = params->tid;
u16 *ssn = ¶ms->ssn;
int ret = 0;
- u8 session;
+ int session;
wcn36xx_dbg(WCN36XX_DBG_MAC, "mac ampdu action action %d tid %d\n",
action, tid);
@@ -1208,9 +1234,11 @@ static int wcn36xx_ampdu_action(struct ieee80211_hw *hw,
sta_priv->tid = tid;
session = wcn36xx_smd_add_ba_session(wcn, sta, tid, ssn, 0,
get_sta_index(vif, sta_priv));
+ if (session < 0) {
+ ret = session;
+ goto out;
+ }
wcn36xx_smd_add_ba(wcn, session);
- wcn36xx_smd_trigger_ba(wcn, get_sta_index(vif, sta_priv), tid,
- session);
break;
case IEEE80211_AMPDU_RX_STOP:
wcn36xx_smd_del_ba(wcn, tid, 0, get_sta_index(vif, sta_priv));
@@ -1220,6 +1248,18 @@ static int wcn36xx_ampdu_action(struct ieee80211_hw *hw,
sta_priv->ampdu_state[tid] = WCN36XX_AMPDU_START;
spin_unlock_bh(&sta_priv->ampdu_lock);
+ /* Replace the mac80211 ssn with the firmware one */
+ wcn36xx_dbg(WCN36XX_DBG_MAC, "mac ampdu ssn = %u\n", *ssn);
+ wcn36xx_smd_trigger_ba(wcn, get_sta_index(vif, sta_priv), tid, ssn);
+ wcn36xx_dbg(WCN36XX_DBG_MAC, "mac ampdu fw-ssn = %u\n", *ssn);
+
+ /* Start BA session */
+ session = wcn36xx_smd_add_ba_session(wcn, sta, tid, ssn, 1,
+ get_sta_index(vif, sta_priv));
+ if (session < 0) {
+ ret = session;
+ goto out;
+ }
ret = IEEE80211_AMPDU_TX_START_IMMEDIATE;
break;
case IEEE80211_AMPDU_TX_OPERATIONAL:
@@ -1227,8 +1267,6 @@ static int wcn36xx_ampdu_action(struct ieee80211_hw *hw,
sta_priv->ampdu_state[tid] = WCN36XX_AMPDU_OPERATIONAL;
spin_unlock_bh(&sta_priv->ampdu_lock);
- wcn36xx_smd_add_ba_session(wcn, sta, tid, ssn, 1,
- get_sta_index(vif, sta_priv));
break;
case IEEE80211_AMPDU_TX_STOP_FLUSH:
case IEEE80211_AMPDU_TX_STOP_FLUSH_CONT:
@@ -1244,6 +1282,7 @@ static int wcn36xx_ampdu_action(struct ieee80211_hw *hw,
wcn36xx_err("Unknown AMPDU action\n");
}
+out:
mutex_unlock(&wcn->conf_mutex);
return ret;
@@ -1277,6 +1316,16 @@ static void wcn36xx_ipv6_addr_change(struct ieee80211_hw *hw,
}
#endif
+static void wcn36xx_flush(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
+ u32 queues, bool drop)
+{
+ struct wcn36xx *wcn = hw->priv;
+
+ if (wcn36xx_dxe_tx_flush(wcn)) {
+ wcn36xx_err("Failed to flush hardware tx queues\n");
+ }
+}
+
static const struct ieee80211_ops wcn36xx_ops = {
.start = wcn36xx_start,
.stop = wcn36xx_stop,
@@ -1304,6 +1353,7 @@ static const struct ieee80211_ops wcn36xx_ops = {
#if IS_ENABLED(CONFIG_IPV6)
.ipv6_addr_change = wcn36xx_ipv6_addr_change,
#endif
+ .flush = wcn36xx_flush,
CFG80211_TESTMODE_CMD(wcn36xx_tm_cmd)
};
diff --git a/drivers/net/wireless/ath/wcn36xx/smd.c b/drivers/net/wireless/ath/wcn36xx/smd.c
index ed45e2cf..caeb689 100644
--- a/drivers/net/wireless/ath/wcn36xx/smd.c
+++ b/drivers/net/wireless/ath/wcn36xx/smd.c
@@ -722,6 +722,7 @@ int wcn36xx_smd_init_scan(struct wcn36xx *wcn, enum wcn36xx_hal_sys_mode mode,
wcn36xx_err("hal_init_scan response failed err=%d\n", ret);
goto out;
}
+ wcn->sw_scan_init = true;
out:
mutex_unlock(&wcn->hal_mutex);
return ret;
@@ -752,6 +753,7 @@ int wcn36xx_smd_start_scan(struct wcn36xx *wcn, u8 scan_channel)
wcn36xx_err("hal_start_scan response failed err=%d\n", ret);
goto out;
}
+ wcn->sw_scan_channel = scan_channel;
out:
mutex_unlock(&wcn->hal_mutex);
return ret;
@@ -782,6 +784,7 @@ int wcn36xx_smd_end_scan(struct wcn36xx *wcn, u8 scan_channel)
wcn36xx_err("hal_end_scan response failed err=%d\n", ret);
goto out;
}
+ wcn->sw_scan_channel = 0;
out:
mutex_unlock(&wcn->hal_mutex);
return ret;
@@ -823,6 +826,7 @@ int wcn36xx_smd_finish_scan(struct wcn36xx *wcn,
wcn36xx_err("hal_finish_scan response failed err=%d\n", ret);
goto out;
}
+ wcn->sw_scan_init = false;
out:
mutex_unlock(&wcn->hal_mutex);
return ret;
@@ -940,7 +944,7 @@ int wcn36xx_smd_update_channel_list(struct wcn36xx *wcn, struct cfg80211_scan_re
INIT_HAL_MSG((*msg_body), WCN36XX_HAL_UPDATE_CHANNEL_LIST_REQ);
- msg_body->num_channel = min_t(u8, req->n_channels, sizeof(msg_body->channels));
+ msg_body->num_channel = min_t(u8, req->n_channels, ARRAY_SIZE(msg_body->channels));
for (i = 0; i < msg_body->num_channel; i++) {
struct wcn36xx_hal_channel_param *param = &msg_body->channels[i];
u32 min_power = WCN36XX_HAL_DEFAULT_MIN_POWER;
@@ -2557,6 +2561,7 @@ int wcn36xx_smd_add_ba_session(struct wcn36xx *wcn,
&session_id);
if (ret) {
wcn36xx_err("hal_add_ba_session response failed err=%d\n", ret);
+ ret = -EINVAL;
goto out;
}
@@ -2622,27 +2627,43 @@ int wcn36xx_smd_del_ba(struct wcn36xx *wcn, u16 tid, u8 direction, u8 sta_index)
return ret;
}
-static int wcn36xx_smd_trigger_ba_rsp(void *buf, int len)
+static int wcn36xx_smd_trigger_ba_rsp(void *buf, int len, struct add_ba_info *ba_info)
{
+ struct wcn36xx_hal_trigger_ba_rsp_candidate *candidate;
struct wcn36xx_hal_trigger_ba_rsp_msg *rsp;
+ int i;
if (len < sizeof(*rsp))
return -EINVAL;
rsp = (struct wcn36xx_hal_trigger_ba_rsp_msg *) buf;
+
+ if (rsp->candidate_cnt < 1)
+ return rsp->status ? rsp->status : -EINVAL;
+
+ candidate = (struct wcn36xx_hal_trigger_ba_rsp_candidate *)(buf + sizeof(*rsp));
+
+ for (i = 0; i < STACFG_MAX_TC; i++) {
+ ba_info[i] = candidate->ba_info[i];
+ }
+
return rsp->status;
}
-int wcn36xx_smd_trigger_ba(struct wcn36xx *wcn, u8 sta_index, u16 tid, u8 session_id)
+int wcn36xx_smd_trigger_ba(struct wcn36xx *wcn, u8 sta_index, u16 tid, u16 *ssn)
{
struct wcn36xx_hal_trigger_ba_req_msg msg_body;
struct wcn36xx_hal_trigger_ba_req_candidate *candidate;
+ struct add_ba_info ba_info[STACFG_MAX_TC];
int ret;
+ if (tid >= STACFG_MAX_TC)
+ return -EINVAL;
+
mutex_lock(&wcn->hal_mutex);
INIT_HAL_MSG(msg_body, WCN36XX_HAL_TRIGGER_BA_REQ);
- msg_body.session_id = session_id;
+ msg_body.session_id = 0; /* not really used */
msg_body.candidate_cnt = 1;
msg_body.header.len += sizeof(*candidate);
PREPARE_HAL_BUF(wcn->hal_buf, msg_body);
@@ -2657,13 +2678,17 @@ int wcn36xx_smd_trigger_ba(struct wcn36xx *wcn, u8 sta_index, u16 tid, u8 sessio
wcn36xx_err("Sending hal_trigger_ba failed\n");
goto out;
}
- ret = wcn36xx_smd_trigger_ba_rsp(wcn->hal_buf, wcn->hal_rsp_len);
+ ret = wcn36xx_smd_trigger_ba_rsp(wcn->hal_buf, wcn->hal_rsp_len, ba_info);
if (ret) {
wcn36xx_err("hal_trigger_ba response failed err=%d\n", ret);
goto out;
}
out:
mutex_unlock(&wcn->hal_mutex);
+
+ if (ssn)
+ *ssn = ba_info[tid].starting_seq_num;
+
return ret;
}
@@ -2732,7 +2757,7 @@ static int wcn36xx_smd_missed_beacon_ind(struct wcn36xx *wcn,
wcn36xx_dbg(WCN36XX_DBG_HAL, "beacon missed bss_index %d\n",
tmp->bss_index);
vif = wcn36xx_priv_to_vif(tmp);
- ieee80211_connection_loss(vif);
+ ieee80211_beacon_loss(vif);
}
return 0;
}
@@ -2747,7 +2772,7 @@ static int wcn36xx_smd_missed_beacon_ind(struct wcn36xx *wcn,
wcn36xx_dbg(WCN36XX_DBG_HAL, "beacon missed bss_index %d\n",
rsp->bss_index);
vif = wcn36xx_priv_to_vif(tmp);
- ieee80211_connection_loss(vif);
+ ieee80211_beacon_loss(vif);
return 0;
}
}
@@ -3168,6 +3193,91 @@ int wcn36xx_smd_host_resume(struct wcn36xx *wcn)
return ret;
}
+#define BEACON_FILTER(eid, presence, offs, val, mask, ref_val) \
+ { \
+ .element_id = eid, \
+ .check_ie_presence = presence, \
+ .offset = offs, \
+ .value = val, \
+ .bitmask = mask, \
+ .ref = ref_val, \
+ }
+
+static const struct beacon_filter_ie bcn_filter_ies[] = {
+ BEACON_FILTER(WLAN_EID_DS_PARAMS, 0, 0, 0,
+ WCN36XX_FILTER_IE_DS_CHANNEL_MASK, 0),
+ BEACON_FILTER(WLAN_EID_ERP_INFO, 0, 0, 0,
+ WCN36XX_FILTER_IE_ERP_FILTER_MASK, 0),
+ BEACON_FILTER(WLAN_EID_EDCA_PARAM_SET, 0, 0, 0,
+ WCN36XX_FILTER_IE_EDCA_FILTER_MASK, 0),
+ BEACON_FILTER(WLAN_EID_QOS_CAPA, 0, 0, 0,
+ WCN36XX_FILTER_IE_QOS_FILTER_MASK, 0),
+ BEACON_FILTER(WLAN_EID_CHANNEL_SWITCH, 1, 0, 0,
+ WCN36XX_FILTER_IE_CHANNEL_SWITCH_MASK, 0),
+ BEACON_FILTER(WLAN_EID_HT_OPERATION, 0, 0, 0,
+ WCN36XX_FILTER_IE_HT_BYTE0_FILTER_MASK, 0),
+ BEACON_FILTER(WLAN_EID_HT_OPERATION, 0, 2, 0,
+ WCN36XX_FILTER_IE_HT_BYTE2_FILTER_MASK, 0),
+ BEACON_FILTER(WLAN_EID_HT_OPERATION, 0, 5, 0,
+ WCN36XX_FILTER_IE_HT_BYTE5_FILTER_MASK, 0),
+ BEACON_FILTER(WLAN_EID_PWR_CONSTRAINT, 0, 0, 0,
+ WCN36XX_FILTER_IE_PWR_CONSTRAINT_MASK, 0),
+ BEACON_FILTER(WLAN_EID_OPMODE_NOTIF, 0, 0, 0,
+ WCN36XX_FILTER_IE_OPMODE_NOTIF_MASK, 0),
+ BEACON_FILTER(WLAN_EID_VHT_OPERATION, 0, 0, 0,
+ WCN36XX_FILTER_IE_VHTOP_CHWIDTH_MASK, 0),
+ BEACON_FILTER(WLAN_EID_RSN, 1, 0, 0,
+ WCN36XX_FILTER_IE_RSN_MASK, 0),
+ BEACON_FILTER(WLAN_EID_VENDOR_SPECIFIC, 1, 0, 0,
+ WCN36XX_FILTER_IE_VENDOR_MASK, 0),
+};
+
+int wcn36xx_smd_add_beacon_filter(struct wcn36xx *wcn,
+ struct ieee80211_vif *vif)
+{
+ struct wcn36xx_hal_add_bcn_filter_req_msg msg_body, *body;
+ struct wcn36xx_vif *vif_priv = wcn36xx_vif_to_priv(vif);
+ u8 *payload;
+ size_t payload_size;
+ int ret;
+
+ if (!get_feat_caps(wcn->fw_feat_caps, BCN_FILTER))
+ return -EOPNOTSUPP;
+
+ mutex_lock(&wcn->hal_mutex);
+ INIT_HAL_MSG(msg_body, WCN36XX_HAL_ADD_BCN_FILTER_REQ);
+
+ PREPARE_HAL_BUF(wcn->hal_buf, msg_body);
+
+ body = (struct wcn36xx_hal_add_bcn_filter_req_msg *)wcn->hal_buf;
+ body->capability_info = vif->bss_conf.assoc_capability;
+ body->capability_mask = WCN36XX_FILTER_CAPABILITY_MASK;
+ body->beacon_interval = vif->bss_conf.beacon_int;
+ body->ie_num = ARRAY_SIZE(bcn_filter_ies);
+ body->bss_index = vif_priv->bss_index;
+
+ payload = ((u8 *)body) + body->header.len;
+ payload_size = sizeof(bcn_filter_ies);
+ memcpy(payload, &bcn_filter_ies, payload_size);
+
+ body->header.len += payload_size;
+
+ ret = wcn36xx_smd_send_and_wait(wcn, body->header.len);
+ if (ret) {
+ wcn36xx_err("Sending add bcn_filter failed\n");
+ goto out;
+ }
+
+ ret = wcn36xx_smd_rsp_status_check(wcn->hal_buf, wcn->hal_rsp_len);
+ if (ret) {
+ wcn36xx_err("add bcn filter response failed err=%d\n", ret);
+ goto out;
+ }
+out:
+ mutex_unlock(&wcn->hal_mutex);
+ return ret;
+}
+
int wcn36xx_smd_rsp_process(struct rpmsg_device *rpdev,
void *buf, int len, void *priv, u32 addr)
{
@@ -3223,6 +3333,7 @@ int wcn36xx_smd_rsp_process(struct rpmsg_device *rpdev,
case WCN36XX_HAL_ENTER_IMPS_RSP:
case WCN36XX_HAL_EXIT_IMPS_RSP:
case WCN36XX_HAL_UPDATE_CHANNEL_LIST_RSP:
+ case WCN36XX_HAL_ADD_BCN_FILTER_RSP:
memcpy(wcn->hal_buf, buf, len);
wcn->hal_rsp_len = len;
complete(&wcn->hal_rsp_compl);
diff --git a/drivers/net/wireless/ath/wcn36xx/smd.h b/drivers/net/wireless/ath/wcn36xx/smd.h
index 88e045d..957cfa8 100644
--- a/drivers/net/wireless/ath/wcn36xx/smd.h
+++ b/drivers/net/wireless/ath/wcn36xx/smd.h
@@ -137,7 +137,7 @@ int wcn36xx_smd_add_ba_session(struct wcn36xx *wcn,
u8 sta_index);
int wcn36xx_smd_add_ba(struct wcn36xx *wcn, u8 session_id);
int wcn36xx_smd_del_ba(struct wcn36xx *wcn, u16 tid, u8 direction, u8 sta_index);
-int wcn36xx_smd_trigger_ba(struct wcn36xx *wcn, u8 sta_index, u16 tid, u8 session_id);
+int wcn36xx_smd_trigger_ba(struct wcn36xx *wcn, u8 sta_index, u16 tid, u16 *ssn);
int wcn36xx_smd_update_cfg(struct wcn36xx *wcn, u32 cfg_id, u32 value);
@@ -167,4 +167,7 @@ int wcn36xx_smd_host_resume(struct wcn36xx *wcn);
int wcn36xx_smd_enter_imps(struct wcn36xx *wcn);
int wcn36xx_smd_exit_imps(struct wcn36xx *wcn);
+int wcn36xx_smd_add_beacon_filter(struct wcn36xx *wcn,
+ struct ieee80211_vif *vif);
+
#endif /* _SMD_H_ */
diff --git a/drivers/net/wireless/ath/wcn36xx/txrx.c b/drivers/net/wireless/ath/wcn36xx/txrx.c
index 75951cc..dd58dde 100644
--- a/drivers/net/wireless/ath/wcn36xx/txrx.c
+++ b/drivers/net/wireless/ath/wcn36xx/txrx.c
@@ -272,7 +272,6 @@ int wcn36xx_rx_skb(struct wcn36xx *wcn, struct sk_buff *skb)
const struct wcn36xx_rate *rate;
struct ieee80211_hdr *hdr;
struct wcn36xx_rx_bd *bd;
- struct ieee80211_supported_band *sband;
u16 fc, sn;
/*
@@ -314,8 +313,6 @@ int wcn36xx_rx_skb(struct wcn36xx *wcn, struct sk_buff *skb)
fc = __le16_to_cpu(hdr->frame_control);
sn = IEEE80211_SEQ_TO_SN(__le16_to_cpu(hdr->seq_ctrl));
- status.freq = WCN36XX_CENTER_FREQ(wcn);
- status.band = WCN36XX_BAND(wcn);
status.mactime = 10;
status.signal = -get_rssi0(bd);
status.antenna = 1;
@@ -327,18 +324,36 @@ int wcn36xx_rx_skb(struct wcn36xx *wcn, struct sk_buff *skb)
wcn36xx_dbg(WCN36XX_DBG_RX, "status.flags=%x\n", status.flag);
+ if (bd->scan_learn) {
+ /* If packet originate from hardware scanning, extract the
+ * band/channel from bd descriptor.
+ */
+ u8 hwch = (bd->reserved0 << 4) + bd->rx_ch;
+
+ if (bd->rf_band != 1 && hwch <= sizeof(ab_rx_ch_map) && hwch >= 1) {
+ status.band = NL80211_BAND_5GHZ;
+ status.freq = ieee80211_channel_to_frequency(ab_rx_ch_map[hwch - 1],
+ status.band);
+ } else {
+ status.band = NL80211_BAND_2GHZ;
+ status.freq = ieee80211_channel_to_frequency(hwch, status.band);
+ }
+ } else {
+ status.band = WCN36XX_BAND(wcn);
+ status.freq = WCN36XX_CENTER_FREQ(wcn);
+ }
+
if (bd->rate_id < ARRAY_SIZE(wcn36xx_rate_table)) {
rate = &wcn36xx_rate_table[bd->rate_id];
status.encoding = rate->encoding;
status.enc_flags = rate->encoding_flags;
status.bw = rate->bw;
status.rate_idx = rate->mcs_or_legacy_index;
- sband = wcn->hw->wiphy->bands[status.band];
status.nss = 1;
if (status.band == NL80211_BAND_5GHZ &&
status.encoding == RX_ENC_LEGACY &&
- status.rate_idx >= sband->n_bitrates) {
+ status.rate_idx >= 4) {
/* no dsss rates in 5Ghz rates table */
status.rate_idx -= 4;
}
@@ -353,22 +368,6 @@ int wcn36xx_rx_skb(struct wcn36xx *wcn, struct sk_buff *skb)
ieee80211_is_probe_resp(hdr->frame_control))
status.boottime_ns = ktime_get_boottime_ns();
- if (bd->scan_learn) {
- /* If packet originates from hardware scanning, extract the
- * band/channel from bd descriptor.
- */
- u8 hwch = (bd->reserved0 << 4) + bd->rx_ch;
-
- if (bd->rf_band != 1 && hwch <= sizeof(ab_rx_ch_map) && hwch >= 1) {
- status.band = NL80211_BAND_5GHZ;
- status.freq = ieee80211_channel_to_frequency(ab_rx_ch_map[hwch - 1],
- status.band);
- } else {
- status.band = NL80211_BAND_2GHZ;
- status.freq = ieee80211_channel_to_frequency(hwch, status.band);
- }
- }
-
memcpy(IEEE80211_SKB_RXCB(skb), &status, sizeof(status));
if (ieee80211_is_beacon(hdr->frame_control)) {
diff --git a/drivers/net/wireless/ath/wcn36xx/wcn36xx.h b/drivers/net/wireless/ath/wcn36xx/wcn36xx.h
index 1c8d918..fbd0558 100644
--- a/drivers/net/wireless/ath/wcn36xx/wcn36xx.h
+++ b/drivers/net/wireless/ath/wcn36xx/wcn36xx.h
@@ -248,6 +248,7 @@ struct wcn36xx {
struct cfg80211_scan_request *scan_req;
bool sw_scan;
u8 sw_scan_opchannel;
+ bool sw_scan_init;
u8 sw_scan_channel;
struct ieee80211_vif *sw_scan_vif;
struct mutex scan_lock;
diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c
index fb72777..ba52318 100644
--- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c
+++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c
@@ -3901,6 +3901,24 @@ static void brcmf_configure_wowl(struct brcmf_cfg80211_info *cfg,
cfg->wowl.active = true;
}
+static int brcmf_keepalive_start(struct brcmf_if *ifp, unsigned int interval)
+{
+ struct brcmf_mkeep_alive_pkt_le kalive = {0};
+ int ret = 0;
+
+ /* Configure Null function/data keepalive */
+ kalive.version = cpu_to_le16(1);
+ kalive.period_msec = cpu_to_le32(interval * MSEC_PER_SEC);
+ kalive.len_bytes = cpu_to_le16(0);
+ kalive.keep_alive_id = 0;
+
+ ret = brcmf_fil_iovar_data_set(ifp, "mkeep_alive", &kalive, sizeof(kalive));
+ if (ret)
+ brcmf_err("keep-alive packet config failed, ret=%d\n", ret);
+
+ return ret;
+}
+
static s32 brcmf_cfg80211_suspend(struct wiphy *wiphy,
struct cfg80211_wowlan *wowl)
{
@@ -3947,6 +3965,9 @@ static s32 brcmf_cfg80211_suspend(struct wiphy *wiphy,
} else {
/* Configure WOWL paramaters */
brcmf_configure_wowl(cfg, ifp, wowl);
+
+ /* Prevent disassociation due to inactivity with keep-alive */
+ brcmf_keepalive_start(ifp, 30);
}
exit:
diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/fwil_types.h b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/fwil_types.h
index ff2ef55..e69d1e5 100644
--- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/fwil_types.h
+++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/fwil_types.h
@@ -1052,4 +1052,23 @@ struct brcmf_gscan_config {
struct brcmf_gscan_bucket_config bucket[1];
};
+/**
+ * struct brcmf_mkeep_alive_pkt_le - configuration data for keep-alive frame.
+ *
+ * @version: version for mkeep_alive
+ * @length: length of fixed parameters in the structure.
+ * @period_msec: keep-alive period in milliseconds.
+ * @len_bytes: size of the data.
+ * @keep_alive_id: ID (0 - 3).
+ * @data: keep-alive frame data.
+ */
+struct brcmf_mkeep_alive_pkt_le {
+ __le16 version;
+ __le16 length;
+ __le32 period_msec;
+ __le16 len_bytes;
+ u8 keep_alive_id;
+ u8 data[0];
+} __packed;
+
#endif /* FWIL_TYPES_H_ */
diff --git a/drivers/net/wireless/intel/ipw2x00/ipw2200.c b/drivers/net/wireless/intel/ipw2x00/ipw2200.c
index 23037bf..5727c7c 100644
--- a/drivers/net/wireless/intel/ipw2x00/ipw2200.c
+++ b/drivers/net/wireless/intel/ipw2x00/ipw2200.c
@@ -2303,7 +2303,7 @@ static int ipw_send_ssid(struct ipw_priv *priv, u8 * ssid, int len)
ssid);
}
-static int ipw_send_adapter_address(struct ipw_priv *priv, u8 * mac)
+static int ipw_send_adapter_address(struct ipw_priv *priv, const u8 * mac)
{
if (!priv || !mac) {
IPW_ERROR("Invalid args\n");
diff --git a/drivers/net/wireless/intel/iwlwifi/Kconfig b/drivers/net/wireless/intel/iwlwifi/Kconfig
index 418ae4f..c21c0c6 100644
--- a/drivers/net/wireless/intel/iwlwifi/Kconfig
+++ b/drivers/net/wireless/intel/iwlwifi/Kconfig
@@ -2,6 +2,7 @@
config IWLWIFI
tristate "Intel Wireless WiFi Next Gen AGN - Wireless-N/Advanced-N/Ultimate-N (iwlwifi) "
depends on PCI && HAS_IOMEM && CFG80211
+ depends on IWLMEI || !IWLMEI
select FW_LOADER
help
Select to build the driver supporting the:
@@ -146,3 +147,28 @@
endmenu
endif
+
+config IWLMEI
+ tristate "Intel Management Engine communication over WLAN"
+ depends on INTEL_MEI
+ depends on PM
+ help
+ Enables the iwlmei kernel module.
+
+ CSME stands for Converged Security and Management Engine. It is a CPU
+ on the chipset and runs a dedicated firmware. AMT (Active Management
+ Technology) is one of the applications that run on that CPU. AMT
+ allows to control the platform remotely.
+
+ This kernel module allows to communicate with the Intel Management
+ Engine over Wifi. This is supported starting from Tiger Lake
+ platforms and has been tested on 9260 devices only.
+ If AMT is configured not to use the wireless device, this module is
+ harmless (and useless).
+ Enabling this option on a platform that has a different device and
+ has Wireless enabled on AMT can prevent WiFi from working correctly.
+
+ For more information see
+ <https://software.intel.com/en-us/manageability/>
+
+ If unsure, say N.
diff --git a/drivers/net/wireless/intel/iwlwifi/Makefile b/drivers/net/wireless/intel/iwlwifi/Makefile
index 0d4656e..75a703e 100644
--- a/drivers/net/wireless/intel/iwlwifi/Makefile
+++ b/drivers/net/wireless/intel/iwlwifi/Makefile
@@ -30,5 +30,6 @@
obj-$(CONFIG_IWLDVM) += dvm/
obj-$(CONFIG_IWLMVM) += mvm/
+obj-$(CONFIG_IWLMEI) += mei/
CFLAGS_iwl-devtrace.o := -I$(src)
diff --git a/drivers/net/wireless/intel/iwlwifi/cfg/22000.c b/drivers/net/wireless/intel/iwlwifi/cfg/22000.c
index 1572097..330ef04 100644
--- a/drivers/net/wireless/intel/iwlwifi/cfg/22000.c
+++ b/drivers/net/wireless/intel/iwlwifi/cfg/22000.c
@@ -9,7 +9,7 @@
#include "iwl-prph.h"
/* Highest firmware API version supported */
-#define IWL_22000_UCODE_API_MAX 67
+#define IWL_22000_UCODE_API_MAX 69
/* Lowest firmware API version supported */
#define IWL_22000_UCODE_API_MIN 39
@@ -54,7 +54,13 @@
#define IWL_BZ_A_GF4_A_FW_PRE "iwlwifi-bz-a0-gf4-a0-"
#define IWL_BZ_A_MR_A_FW_PRE "iwlwifi-bz-a0-mr-a0-"
#define IWL_BZ_A_FM_A_FW_PRE "iwlwifi-bz-a0-fm-a0-"
-#define IWL_GL_A_FM_A_FW_PRE "iwlwifi-gl-a0-fm7-a0-"
+#define IWL_GL_A_FM_A_FW_PRE "iwlwifi-gl-a0-fm-a0-"
+#define IWL_BZ_Z_GF_A_FW_PRE "iwlwifi-bz-z0-gf-a0-"
+#define IWL_BNJ_A_FM_A_FW_PRE "iwlwifi-BzBnj-a0-fm-a0-"
+#define IWL_BNJ_A_FM4_A_FW_PRE "iwlwifi-BzBnj-a0-fm4-a0-"
+#define IWL_BNJ_A_GF_A_FW_PRE "iwlwifi-BzBnj-a0-gf-a0-"
+#define IWL_BNJ_A_GF4_A_FW_PRE "iwlwifi-BzBnj-a0-gf4-a0-"
+#define IWL_BNJ_A_HR_B_FW_PRE "iwlwifi-BzBnj-a0-hr-b0-"
#define IWL_QU_B_HR_B_MODULE_FIRMWARE(api) \
@@ -113,6 +119,18 @@
IWL_BZ_A_FM_A_FW_PRE __stringify(api) ".ucode"
#define IWL_GL_A_FM_A_MODULE_FIRMWARE(api) \
IWL_GL_A_FM_A_FW_PRE __stringify(api) ".ucode"
+#define IWL_BZ_Z_GF_A_MODULE_FIRMWARE(api) \
+ IWL_BZ_Z_GF_A_FW_PRE __stringify(api) ".ucode"
+#define IWL_BNJ_A_FM_A_MODULE_FIRMWARE(api) \
+ IWL_BNJ_A_FM_A_FW_PRE __stringify(api) ".ucode"
+#define IWL_BNJ_A_FM4_A_MODULE_FIRMWARE(api) \
+ IWL_BNJ_A_FM4_A_FW_PRE __stringify(api) ".ucode"
+#define IWL_BNJ_A_GF_A_MODULE_FIRMWARE(api) \
+ IWL_BNJ_A_GF_A_FW_PRE __stringify(api) ".ucode"
+#define IWL_BNJ_A_GF4_A_MODULE_FIRMWARE(api) \
+ IWL_BNJ_A_GF4_A_FW_PRE __stringify(api) ".ucode"
+#define IWL_BNJ_A_HR_B_MODULE_FIRMWARE(api) \
+ IWL_BNJ_A_HR_B_FW_PRE __stringify(api) ".ucode"
static const struct iwl_base_params iwl_22000_base_params = {
.eeprom_size = OTP_LOW_IMAGE_SIZE_32K,
@@ -234,7 +252,7 @@ static const struct iwl_ht_params iwl_22000_ht_params = {
.dccm2_len = IWL_22000_DCCM2_LEN, \
.smem_offset = IWL_22000_SMEM_OFFSET, \
.smem_len = IWL_22000_SMEM_LEN, \
- .features = IWL_TX_CSUM_NETIF_FLAGS | NETIF_F_RXCSUM, \
+ .features = IWL_TX_CSUM_NETIF_FLAGS_BZ | NETIF_F_RXCSUM, \
.apmg_not_supported = true, \
.trans.mq_rx_supported = true, \
.vht_mu_mimo_supported = true, \
@@ -626,7 +644,7 @@ const struct iwl_cfg iwl_ax200_cfg_cc = {
};
const struct iwl_cfg killer1650s_2ax_cfg_qu_b0_hr_b0 = {
- .name = "Killer(R) Wi-Fi 6 AX1650i 160MHz Wireless Network Adapter (201NGW)",
+ .name = "Killer(R) Wi-Fi 6 AX1650s 160MHz Wireless Network Adapter (201NGW)",
.fw_name_pre = IWL_QU_B_HR_B_FW_PRE,
IWL_DEVICE_22500,
/*
@@ -639,7 +657,7 @@ const struct iwl_cfg killer1650s_2ax_cfg_qu_b0_hr_b0 = {
};
const struct iwl_cfg killer1650i_2ax_cfg_qu_b0_hr_b0 = {
- .name = "Killer(R) Wi-Fi 6 AX1650s 160MHz Wireless Network Adapter (201D2W)",
+ .name = "Killer(R) Wi-Fi 6 AX1650i 160MHz Wireless Network Adapter (201D2W)",
.fw_name_pre = IWL_QU_B_HR_B_FW_PRE,
IWL_DEVICE_22500,
/*
@@ -652,7 +670,7 @@ const struct iwl_cfg killer1650i_2ax_cfg_qu_b0_hr_b0 = {
};
const struct iwl_cfg killer1650s_2ax_cfg_qu_c0_hr_b0 = {
- .name = "Killer(R) Wi-Fi 6 AX1650i 160MHz Wireless Network Adapter (201NGW)",
+ .name = "Killer(R) Wi-Fi 6 AX1650s 160MHz Wireless Network Adapter (201NGW)",
.fw_name_pre = IWL_QU_C_HR_B_FW_PRE,
IWL_DEVICE_22500,
/*
@@ -665,7 +683,7 @@ const struct iwl_cfg killer1650s_2ax_cfg_qu_c0_hr_b0 = {
};
const struct iwl_cfg killer1650i_2ax_cfg_qu_c0_hr_b0 = {
- .name = "Killer(R) Wi-Fi 6 AX1650s 160MHz Wireless Network Adapter (201D2W)",
+ .name = "Killer(R) Wi-Fi 6 AX1650i 160MHz Wireless Network Adapter (201D2W)",
.fw_name_pre = IWL_QU_C_HR_B_FW_PRE,
IWL_DEVICE_22500,
/*
@@ -696,13 +714,6 @@ const struct iwl_cfg iwlax210_2ax_cfg_so_jf_b0 = {
.num_rbds = IWL_NUM_RBDS_NON_HE,
};
-const struct iwl_cfg iwlax210_2ax_cfg_so_hr_a0 = {
- .name = "Intel(R) Wi-Fi 6 AX210 160MHz",
- .fw_name_pre = IWL_SO_A_HR_B_FW_PRE,
- IWL_DEVICE_AX210,
- .num_rbds = IWL_NUM_RBDS_AX210_HE,
-};
-
const struct iwl_cfg iwlax211_2ax_cfg_so_gf_a0 = {
.name = iwl_ax211_name,
.fw_name_pre = IWL_SO_A_GF_A_FW_PRE,
@@ -879,6 +890,47 @@ const struct iwl_cfg iwl_cfg_gl_a0_fm_a0 = {
.num_rbds = IWL_NUM_RBDS_AX210_HE,
};
+const struct iwl_cfg iwl_cfg_bz_z0_gf_a0 = {
+ .fw_name_pre = IWL_BZ_Z_GF_A_FW_PRE,
+ .uhb_supported = true,
+ IWL_DEVICE_BZ,
+ .num_rbds = IWL_NUM_RBDS_AX210_HE,
+};
+
+const struct iwl_cfg iwl_cfg_bnj_a0_fm_a0 = {
+ .fw_name_pre = IWL_BNJ_A_FM_A_FW_PRE,
+ .uhb_supported = true,
+ IWL_DEVICE_BZ,
+ .num_rbds = IWL_NUM_RBDS_AX210_HE,
+};
+
+const struct iwl_cfg iwl_cfg_bnj_a0_fm4_a0 = {
+ .fw_name_pre = IWL_BNJ_A_FM4_A_FW_PRE,
+ .uhb_supported = true,
+ IWL_DEVICE_BZ,
+ .num_rbds = IWL_NUM_RBDS_AX210_HE,
+};
+
+const struct iwl_cfg iwl_cfg_bnj_a0_gf_a0 = {
+ .fw_name_pre = IWL_BNJ_A_GF_A_FW_PRE,
+ .uhb_supported = true,
+ IWL_DEVICE_BZ,
+ .num_rbds = IWL_NUM_RBDS_AX210_HE,
+};
+
+const struct iwl_cfg iwl_cfg_bnj_a0_gf4_a0 = {
+ .fw_name_pre = IWL_BNJ_A_GF4_A_FW_PRE,
+ .uhb_supported = true,
+ IWL_DEVICE_BZ,
+ .num_rbds = IWL_NUM_RBDS_AX210_HE,
+};
+
+const struct iwl_cfg iwl_cfg_bnj_a0_hr_b0 = {
+ .fw_name_pre = IWL_BNJ_A_HR_B_FW_PRE,
+ .uhb_supported = true,
+ IWL_DEVICE_BZ,
+ .num_rbds = IWL_NUM_RBDS_AX210_HE,
+};
MODULE_FIRMWARE(IWL_QU_B_HR_B_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
MODULE_FIRMWARE(IWL_QNJ_B_HR_B_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
MODULE_FIRMWARE(IWL_QU_C_HR_B_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
@@ -907,3 +959,8 @@ MODULE_FIRMWARE(IWL_BZ_A_GF4_A_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
MODULE_FIRMWARE(IWL_BZ_A_MR_A_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
MODULE_FIRMWARE(IWL_BZ_A_FM_A_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
MODULE_FIRMWARE(IWL_GL_A_FM_A_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
+MODULE_FIRMWARE(IWL_BNJ_A_FM_A_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
+MODULE_FIRMWARE(IWL_BNJ_A_FM4_A_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
+MODULE_FIRMWARE(IWL_BNJ_A_GF_A_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
+MODULE_FIRMWARE(IWL_BNJ_A_GF4_A_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
+MODULE_FIRMWARE(IWL_BNJ_A_HR_B_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
diff --git a/drivers/net/wireless/intel/iwlwifi/dvm/main.c b/drivers/net/wireless/intel/iwlwifi/dvm/main.c
index fbd57a2..90b9bec 100644
--- a/drivers/net/wireless/intel/iwlwifi/dvm/main.c
+++ b/drivers/net/wireless/intel/iwlwifi/dvm/main.c
@@ -1974,12 +1974,8 @@ static void iwl_nic_config(struct iwl_op_mode *op_mode)
/* SKU Control */
iwl_trans_set_bits_mask(priv->trans, CSR_HW_IF_CONFIG_REG,
- CSR_HW_IF_CONFIG_REG_MSK_MAC_DASH |
- CSR_HW_IF_CONFIG_REG_MSK_MAC_STEP,
- (CSR_HW_REV_STEP(priv->trans->hw_rev) <<
- CSR_HW_IF_CONFIG_REG_POS_MAC_STEP) |
- (CSR_HW_REV_DASH(priv->trans->hw_rev) <<
- CSR_HW_IF_CONFIG_REG_POS_MAC_DASH));
+ CSR_HW_IF_CONFIG_REG_MSK_MAC_STEP_DASH,
+ CSR_HW_REV_STEP_DASH(priv->trans->hw_rev));
/* write radio config values to register */
if (priv->nvm_data->radio_cfg_type <= EEPROM_RF_CONFIG_TYPE_MAX) {
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/acpi.c b/drivers/net/wireless/intel/iwlwifi/fw/acpi.c
index bf431fa..790c96d 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/acpi.c
+++ b/drivers/net/wireless/intel/iwlwifi/fw/acpi.c
@@ -242,17 +242,16 @@ union acpi_object *iwl_acpi_get_wifi_pkg_range(struct device *dev,
IWL_EXPORT_SYMBOL(iwl_acpi_get_wifi_pkg_range);
int iwl_acpi_get_tas(struct iwl_fw_runtime *fwrt,
- __le32 *block_list_array,
- int *block_list_size)
+ struct iwl_tas_config_cmd_v3 *cmd)
{
union acpi_object *wifi_pkg, *data;
- int ret, tbl_rev, i;
- bool enabled;
+ int ret, tbl_rev, i, block_list_size, enabled;
data = iwl_acpi_get_object(fwrt->dev, ACPI_WTAS_METHOD);
if (IS_ERR(data))
return PTR_ERR(data);
+ /* try to read wtas table revision 1 or revision 0*/
wifi_pkg = iwl_acpi_get_wifi_pkg(fwrt->dev, data,
ACPI_WTAS_WIFI_DATA_SIZE,
&tbl_rev);
@@ -261,40 +260,54 @@ int iwl_acpi_get_tas(struct iwl_fw_runtime *fwrt,
goto out_free;
}
- if (wifi_pkg->package.elements[0].type != ACPI_TYPE_INTEGER ||
- tbl_rev != 0) {
+ if (tbl_rev == 1 && wifi_pkg->package.elements[1].type ==
+ ACPI_TYPE_INTEGER) {
+ u32 tas_selection =
+ (u32)wifi_pkg->package.elements[1].integer.value;
+ u16 override_iec =
+ (tas_selection & ACPI_WTAS_OVERRIDE_IEC_MSK) >> ACPI_WTAS_OVERRIDE_IEC_POS;
+ u16 enabled_iec = (tas_selection & ACPI_WTAS_ENABLE_IEC_MSK) >>
+ ACPI_WTAS_ENABLE_IEC_POS;
+
+ enabled = tas_selection & ACPI_WTAS_ENABLED_MSK;
+ cmd->override_tas_iec = cpu_to_le16(override_iec);
+ cmd->enable_tas_iec = cpu_to_le16(enabled_iec);
+
+ } else if (tbl_rev == 0 &&
+ wifi_pkg->package.elements[1].type == ACPI_TYPE_INTEGER) {
+ enabled = !!wifi_pkg->package.elements[1].integer.value;
+ } else {
ret = -EINVAL;
goto out_free;
}
- enabled = !!wifi_pkg->package.elements[1].integer.value;
-
if (!enabled) {
- *block_list_size = -1;
IWL_DEBUG_RADIO(fwrt, "TAS not enabled\n");
ret = 0;
goto out_free;
}
+ IWL_DEBUG_RADIO(fwrt, "Reading TAS table revision %d\n", tbl_rev);
if (wifi_pkg->package.elements[2].type != ACPI_TYPE_INTEGER ||
wifi_pkg->package.elements[2].integer.value >
APCI_WTAS_BLACK_LIST_MAX) {
IWL_DEBUG_RADIO(fwrt, "TAS invalid array size %llu\n",
- wifi_pkg->package.elements[1].integer.value);
+ wifi_pkg->package.elements[2].integer.value);
ret = -EINVAL;
goto out_free;
}
- *block_list_size = wifi_pkg->package.elements[2].integer.value;
+ block_list_size = wifi_pkg->package.elements[2].integer.value;
+ cmd->block_list_size = cpu_to_le32(block_list_size);
- IWL_DEBUG_RADIO(fwrt, "TAS array size %d\n", *block_list_size);
- if (*block_list_size > APCI_WTAS_BLACK_LIST_MAX) {
+ IWL_DEBUG_RADIO(fwrt, "TAS array size %u\n", block_list_size);
+ if (block_list_size > APCI_WTAS_BLACK_LIST_MAX) {
IWL_DEBUG_RADIO(fwrt, "TAS invalid array size value %u\n",
- *block_list_size);
+ block_list_size);
ret = -EINVAL;
goto out_free;
}
- for (i = 0; i < *block_list_size; i++) {
+ for (i = 0; i < block_list_size; i++) {
u32 country;
if (wifi_pkg->package.elements[3 + i].type !=
@@ -306,11 +319,11 @@ int iwl_acpi_get_tas(struct iwl_fw_runtime *fwrt,
}
country = wifi_pkg->package.elements[3 + i].integer.value;
- block_list_array[i] = cpu_to_le32(country);
+ cmd->block_list_array[i] = cpu_to_le32(country);
IWL_DEBUG_RADIO(fwrt, "TAS block list country %d\n", country);
}
- ret = 0;
+ ret = 1;
out_free:
kfree(data);
return ret;
@@ -789,7 +802,7 @@ int iwl_sar_get_wgds_table(struct iwl_fw_runtime *fwrt)
* looking up in ACPI
*/
if (wifi_pkg->package.count !=
- min_size + profile_size * num_profiles) {
+ hdr_size + profile_size * num_profiles) {
ret = -EINVAL;
goto out_free;
}
@@ -852,6 +865,8 @@ int iwl_sar_get_wgds_table(struct iwl_fw_runtime *fwrt)
}
}
+ fwrt->geo_num_profiles = num_profiles;
+ fwrt->geo_enabled = true;
ret = 0;
out_free:
kfree(data);
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/acpi.h b/drivers/net/wireless/intel/iwlwifi/fw/acpi.h
index 4aaa8a6..22b3c66 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/acpi.h
+++ b/drivers/net/wireless/intel/iwlwifi/fw/acpi.h
@@ -65,10 +65,19 @@
#define ACPI_ECKV_WIFI_DATA_SIZE 2
/*
- * 1 type, 1 enabled, 1 block list size, 16 block list array
+ * TAS size: 1 elelment for type,
+ * 1 element for enabled field,
+ * 1 element for block list size,
+ * 16 elements for block list array
*/
#define APCI_WTAS_BLACK_LIST_MAX 16
#define ACPI_WTAS_WIFI_DATA_SIZE (3 + APCI_WTAS_BLACK_LIST_MAX)
+#define ACPI_WTAS_ENABLED_MSK 0x1
+#define ACPI_WTAS_OVERRIDE_IEC_MSK 0x2
+#define ACPI_WTAS_ENABLE_IEC_MSK 0x4
+#define ACPI_WTAS_OVERRIDE_IEC_POS 0x1
+#define ACPI_WTAS_ENABLE_IEC_POS 0x2
+
#define ACPI_PPAG_WIFI_DATA_SIZE_V1 ((IWL_NUM_CHAIN_LIMITS * \
IWL_NUM_SUB_BANDS_V1) + 2)
@@ -105,6 +114,11 @@ struct iwl_geo_profile {
struct iwl_geo_profile_band bands[ACPI_GEO_NUM_BANDS_REV2];
};
+/* Same thing as with SAR, all revisions fit in revision 2 */
+struct iwl_ppag_chain {
+ s8 subbands[ACPI_SAR_NUM_SUB_BANDS_REV2];
+};
+
enum iwl_dsm_funcs_rev_0 {
DSM_FUNC_QUERY = 0,
DSM_FUNC_DISABLE_SRD = 1,
@@ -198,8 +212,8 @@ int iwl_sar_geo_init(struct iwl_fw_runtime *fwrt,
struct iwl_per_chain_offset *table,
u32 n_bands, u32 n_profiles);
-int iwl_acpi_get_tas(struct iwl_fw_runtime *fwrt, __le32 *block_list_array,
- int *block_list_size);
+int iwl_acpi_get_tas(struct iwl_fw_runtime *fwrt,
+ struct iwl_tas_config_cmd_v3 *cmd);
__le32 iwl_acpi_get_lari_config_bitmap(struct iwl_fw_runtime *fwrt);
@@ -280,8 +294,7 @@ static inline bool iwl_sar_geo_support(struct iwl_fw_runtime *fwrt)
}
static inline int iwl_acpi_get_tas(struct iwl_fw_runtime *fwrt,
- __le32 *block_list_array,
- int *block_list_size)
+ struct iwl_tas_config_cmd_v3 *cmd)
{
return -ENOENT;
}
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/api/alive.h b/drivers/net/wireless/intel/iwlwifi/fw/api/alive.h
index c840a97..e00ab21 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/api/alive.h
+++ b/drivers/net/wireless/intel/iwlwifi/fw/api/alive.h
@@ -1,6 +1,6 @@
/* SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause */
/*
- * Copyright (C) 2012-2014, 2018, 2020 Intel Corporation
+ * Copyright (C) 2012-2014, 2018, 2020-2021 Intel Corporation
* Copyright (C) 2013-2015 Intel Mobile Communications GmbH
* Copyright (C) 2016-2017 Intel Deutschland GmbH
*/
@@ -97,6 +97,21 @@ struct iwl_alive_ntf_v5 {
struct iwl_sku_id sku_id;
} __packed; /* UCODE_ALIVE_NTFY_API_S_VER_5 */
+struct iwl_imr_alive_info {
+ __le64 base_addr;
+ __le32 size;
+ __le32 enabled;
+} __packed; /* IMR_ALIVE_INFO_API_S_VER_1 */
+
+struct iwl_alive_ntf_v6 {
+ __le16 status;
+ __le16 flags;
+ struct iwl_lmac_alive lmac_data[2];
+ struct iwl_umac_alive umac_data;
+ struct iwl_sku_id sku_id;
+ struct iwl_imr_alive_info imr;
+} __packed; /* UCODE_ALIVE_NTFY_API_S_VER_6 */
+
/**
* enum iwl_extended_cfg_flag - commands driver may send before
* finishing init flow
@@ -143,15 +158,6 @@ enum iwl_card_state_flags {
};
/**
- * struct iwl_radio_version_notif - information on the card state
- * ( CARD_STATE_NOTIFICATION = 0xa1 )
- * @flags: &enum iwl_card_state_flags
- */
-struct iwl_card_state_notif {
- __le32 flags;
-} __packed; /* CARD_STATE_NTFY_API_S_VER_1 */
-
-/**
* enum iwl_error_recovery_flags - flags for error recovery cmd
* @ERROR_RECOVERY_UPDATE_DB: update db from blob sent
* @ERROR_RECOVERY_END_OF_RECOVERY: end of recovery
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/api/commands.h b/drivers/net/wireless/intel/iwlwifi/fw/api/commands.h
index ee6b584..0703e41 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/api/commands.h
+++ b/drivers/net/wireless/intel/iwlwifi/fw/api/commands.h
@@ -51,7 +51,7 @@ enum iwl_legacy_cmds {
* @UCODE_ALIVE_NTFY:
* Alive data from the firmware, as described in
* &struct iwl_alive_ntf_v3 or &struct iwl_alive_ntf_v4 or
- * &struct iwl_alive_ntf_v5.
+ * &struct iwl_alive_ntf_v5 or &struct iwl_alive_ntf_v6.
*/
UCODE_ALIVE_NTFY = 0x1,
@@ -72,7 +72,8 @@ enum iwl_legacy_cmds {
/**
* @PHY_CONTEXT_CMD:
- * Add/modify/remove a PHY context, using &struct iwl_phy_context_cmd.
+ * Add/modify/remove a PHY context, using &struct iwl_phy_context_cmd
+ * or &struct iwl_phy_context_cmd_v1.
*/
PHY_CONTEXT_CMD = 0x8,
@@ -90,7 +91,8 @@ enum iwl_legacy_cmds {
/**
* @SCAN_CFG_CMD:
- * uses &struct iwl_scan_config_v1 or &struct iwl_scan_config
+ * uses &struct iwl_scan_config_v1, &struct iwl_scan_config_v2
+ * or &struct iwl_scan_config
*/
SCAN_CFG_CMD = 0xc,
@@ -356,7 +358,7 @@ enum iwl_legacy_cmds {
* &struct iwl_notif_statistics_v11,
* &struct iwl_notif_statistics_v10,
* &struct iwl_notif_statistics,
- * &struct iwl_statistics_operational_ntfy
+ * &struct iwl_statistics_operational_ntfy_ver_14
*/
STATISTICS_CMD = 0x9c,
@@ -365,6 +367,7 @@ enum iwl_legacy_cmds {
* one of &struct iwl_notif_statistics_v10,
* &struct iwl_notif_statistics_v11,
* &struct iwl_notif_statistic,
+ * &struct iwl_statistics_operational_ntfy_ver_14
* &struct iwl_statistics_operational_ntfy
*/
STATISTICS_NOTIFICATION = 0x9d,
@@ -383,13 +386,6 @@ enum iwl_legacy_cmds {
REDUCE_TX_POWER_CMD = 0x9f,
/**
- * @CARD_STATE_NOTIFICATION:
- * Card state (RF/CT kill) notification,
- * uses &struct iwl_card_state_notif
- */
- CARD_STATE_NOTIFICATION = 0xa1,
-
- /**
* @MISSED_BEACONS_NOTIFICATION: &struct iwl_missed_beacons_notif
*/
MISSED_BEACONS_NOTIFICATION = 0xa2,
@@ -612,6 +608,11 @@ enum iwl_system_subcmd_ids {
* @RFI_GET_FREQ_TABLE_CMD: &struct iwl_rfi_config_cmd
*/
RFI_GET_FREQ_TABLE_CMD = 0xc,
+
+ /**
+ * @SYSTEM_FEATURES_CONTROL_CMD: &struct iwl_system_features_control_cmd
+ */
+ SYSTEM_FEATURES_CONTROL_CMD = 0xd,
};
#endif /* __iwl_fw_api_commands_h__ */
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/api/d3.h b/drivers/net/wireless/intel/iwlwifi/fw/api/d3.h
index 1503119..4cd9ab2 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/api/d3.h
+++ b/drivers/net/wireless/intel/iwlwifi/fw/api/d3.h
@@ -554,7 +554,7 @@ struct iwl_wowlan_gtk_status_v1 {
} __packed; /* WOWLAN_GTK_MATERIAL_VER_1 */
/**
- * struct iwl_wowlan_gtk_status - GTK status
+ * struct iwl_wowlan_gtk_status_v2 - GTK status
* @key: GTK material
* @key_len: GTK legth, if set to 0, the key is not available
* @key_flags: information about the key:
@@ -565,7 +565,7 @@ struct iwl_wowlan_gtk_status_v1 {
* @tkip_mic_key: TKIP RX MIC key
* @rsc: TSC RSC counters
*/
-struct iwl_wowlan_gtk_status {
+struct iwl_wowlan_gtk_status_v2 {
u8 key[WOWLAN_KEY_MAX_SIZE];
u8 key_len;
u8 key_flags;
@@ -574,6 +574,41 @@ struct iwl_wowlan_gtk_status {
struct iwl_wowlan_rsc_tsc_params_cmd_ver_2 rsc;
} __packed; /* WOWLAN_GTK_MATERIAL_VER_2 */
+/**
+ * struct iwl_wowlan_all_rsc_tsc_v5 - key counters
+ * @ucast_rsc: unicast RSC values
+ * @mcast_rsc: multicast RSC values (per key map value)
+ * @sta_id: station ID
+ * @mcast_key_id_map: map of key id to @mcast_rsc entry
+ */
+struct iwl_wowlan_all_rsc_tsc_v5 {
+ __le64 ucast_rsc[IWL_MAX_TID_COUNT];
+ __le64 mcast_rsc[2][IWL_MAX_TID_COUNT];
+ __le32 sta_id;
+ u8 mcast_key_id_map[4];
+} __packed; /* ALL_TSC_RSC_API_S_VER_5 */
+
+/**
+ * struct iwl_wowlan_gtk_status_v3 - GTK status
+ * @key: GTK material
+ * @key_len: GTK length, if set to 0, the key is not available
+ * @key_flags: information about the key:
+ * bits[0:1]: key index assigned by the AP
+ * bits[2:6]: GTK index of the key in the internal DB
+ * bit[7]: Set iff this is the currently used GTK
+ * @reserved: padding
+ * @tkip_mic_key: TKIP RX MIC key
+ * @sc: RSC/TSC counters
+ */
+struct iwl_wowlan_gtk_status_v3 {
+ u8 key[WOWLAN_KEY_MAX_SIZE];
+ u8 key_len;
+ u8 key_flags;
+ u8 reserved[2];
+ u8 tkip_mic_key[IWL_MIC_KEY_SIZE];
+ struct iwl_wowlan_all_rsc_tsc_v5 sc;
+} __packed; /* WOWLAN_GTK_MATERIAL_VER_3 */
+
#define IWL_WOWLAN_GTK_IDX_MASK (BIT(0) | BIT(1))
/**
@@ -640,7 +675,7 @@ struct iwl_wowlan_status_v6 {
* @wake_packet: wakeup packet
*/
struct iwl_wowlan_status_v7 {
- struct iwl_wowlan_gtk_status gtk[WOWLAN_GTK_KEYS_NUM];
+ struct iwl_wowlan_gtk_status_v2 gtk[WOWLAN_GTK_KEYS_NUM];
struct iwl_wowlan_igtk_status igtk[WOWLAN_IGTK_KEYS_NUM];
__le64 replay_ctr;
__le16 pattern_number;
@@ -676,7 +711,7 @@ struct iwl_wowlan_status_v7 {
* @wake_packet: wakeup packet
*/
struct iwl_wowlan_status_v9 {
- struct iwl_wowlan_gtk_status gtk[WOWLAN_GTK_KEYS_NUM];
+ struct iwl_wowlan_gtk_status_v2 gtk[WOWLAN_GTK_KEYS_NUM];
struct iwl_wowlan_igtk_status igtk[WOWLAN_IGTK_KEYS_NUM];
__le64 replay_ctr;
__le16 pattern_number;
@@ -693,6 +728,44 @@ struct iwl_wowlan_status_v9 {
u8 wake_packet[]; /* can be truncated from _length to _bufsize */
} __packed; /* WOWLAN_STATUSES_RSP_API_S_VER_9 */
+/**
+ * struct iwl_wowlan_status_v12 - WoWLAN status
+ * @gtk: GTK data
+ * @igtk: IGTK data
+ * @replay_ctr: GTK rekey replay counter
+ * @pattern_number: number of the matched pattern
+ * @non_qos_seq_ctr: non-QoS sequence counter to use next.
+ * Reserved if the struct has version >= 10.
+ * @qos_seq_ctr: QoS sequence counters to use next
+ * @wakeup_reasons: wakeup reasons, see &enum iwl_wowlan_wakeup_reason
+ * @num_of_gtk_rekeys: number of GTK rekeys
+ * @transmitted_ndps: number of transmitted neighbor discovery packets
+ * @received_beacons: number of received beacons
+ * @wake_packet_length: wakeup packet length
+ * @wake_packet_bufsize: wakeup packet buffer size
+ * @tid_tear_down: bit mask of tids whose BA sessions were closed
+ * in suspend state
+ * @reserved: unused
+ * @wake_packet: wakeup packet
+ */
+struct iwl_wowlan_status_v12 {
+ struct iwl_wowlan_gtk_status_v3 gtk[WOWLAN_GTK_KEYS_NUM];
+ struct iwl_wowlan_igtk_status igtk[WOWLAN_IGTK_KEYS_NUM];
+ __le64 replay_ctr;
+ __le16 pattern_number;
+ __le16 non_qos_seq_ctr;
+ __le16 qos_seq_ctr[8];
+ __le32 wakeup_reasons;
+ __le32 num_of_gtk_rekeys;
+ __le32 transmitted_ndps;
+ __le32 received_beacons;
+ __le32 wake_packet_length;
+ __le32 wake_packet_bufsize;
+ u8 tid_tear_down;
+ u8 reserved[3];
+ u8 wake_packet[]; /* can be truncated from _length to _bufsize */
+} __packed; /* WOWLAN_STATUSES_RSP_API_S_VER_12 */
+
/* TODO: NetDetect API */
#endif /* __iwl_fw_api_d3_h__ */
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/api/datapath.h b/drivers/net/wireless/intel/iwlwifi/fw/api/datapath.h
index 985b0dc..89236f4 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/api/datapath.h
+++ b/drivers/net/wireless/intel/iwlwifi/fw/api/datapath.h
@@ -32,6 +32,11 @@ enum iwl_data_path_subcmd_ids {
STA_HE_CTXT_CMD = 0x7,
/**
+ * @RLC_CONFIG_CMD: &struct iwl_rlc_config_cmd
+ */
+ RLC_CONFIG_CMD = 0x8,
+
+ /**
* @RFH_QUEUE_CONFIG_CMD: &struct iwl_rfh_queue_config
*/
RFH_QUEUE_CONFIG_CMD = 0xD,
@@ -195,4 +200,61 @@ struct iwl_thermal_dual_chain_request {
__le32 event;
} __packed; /* THERMAL_DUAL_CHAIN_DISABLE_REQ_NTFY_API_S_VER_1 */
+enum iwl_rlc_chain_info {
+ IWL_RLC_CHAIN_INFO_DRIVER_FORCE = BIT(0),
+ IWL_RLC_CHAIN_INFO_VALID = 0x000e,
+ IWL_RLC_CHAIN_INFO_FORCE = 0x0070,
+ IWL_RLC_CHAIN_INFO_FORCE_MIMO = 0x0380,
+ IWL_RLC_CHAIN_INFO_COUNT = 0x0c00,
+ IWL_RLC_CHAIN_INFO_MIMO_COUNT = 0x3000,
+};
+
+/**
+ * struct iwl_rlc_properties - RLC properties
+ * @rx_chain_info: RX chain info, &enum iwl_rlc_chain_info
+ * @reserved: reserved
+ */
+struct iwl_rlc_properties {
+ __le32 rx_chain_info;
+ __le32 reserved;
+} __packed; /* RLC_PROPERTIES_S_VER_1 */
+
+enum iwl_sad_mode {
+ IWL_SAD_MODE_ENABLED = BIT(0),
+ IWL_SAD_MODE_DEFAULT_ANT_MSK = 0x6,
+ IWL_SAD_MODE_DEFAULT_ANT_FW = 0x0,
+ IWL_SAD_MODE_DEFAULT_ANT_A = 0x2,
+ IWL_SAD_MODE_DEFAULT_ANT_B = 0x4,
+};
+
+/**
+ * struct iwl_sad_properties - SAD properties
+ * @chain_a_sad_mode: chain A SAD mode, &enum iwl_sad_mode
+ * @chain_b_sad_mode: chain B SAD mode, &enum iwl_sad_mode
+ * @mac_id: MAC index
+ * @reserved: reserved
+ */
+struct iwl_sad_properties {
+ __le32 chain_a_sad_mode;
+ __le32 chain_b_sad_mode;
+ __le32 mac_id;
+ __le32 reserved;
+} __packed;
+
+/**
+ * struct iwl_rlc_config_cmd - RLC configuration
+ * @phy_id: PHY index
+ * @rlc: RLC properties, &struct iwl_rlc_properties
+ * @sad: SAD (single antenna diversity) options, &struct iwl_sad_properties
+ * @flags: flags, &enum iwl_rlc_flags
+ * @reserved: reserved
+ */
+struct iwl_rlc_config_cmd {
+ __le32 phy_id;
+ struct iwl_rlc_properties rlc;
+ struct iwl_sad_properties sad;
+ u8 flags;
+ u8 reserved[3];
+} __packed; /* RLC_CONFIG_CMD_API_S_VER_2 */
+
#endif /* __iwl_fw_api_datapath_h__ */
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/api/dbg-tlv.h b/drivers/net/wireless/intel/iwlwifi/fw/api/dbg-tlv.h
index 3988f5f..456b7ea 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/api/dbg-tlv.h
+++ b/drivers/net/wireless/intel/iwlwifi/fw/api/dbg-tlv.h
@@ -7,7 +7,6 @@
#include <linux/bitops.h>
-#define IWL_FW_INI_HW_SMEM_REGION_ID 15
#define IWL_FW_INI_MAX_REGION_ID 64
#define IWL_FW_INI_MAX_NAME 32
#define IWL_FW_INI_MAX_CFG_NAME 64
@@ -124,6 +123,9 @@ struct iwl_fw_ini_region_internal_buffer {
* @hdr: debug header
* @id: region id. Max id is &IWL_FW_INI_MAX_REGION_ID
* @type: region type. One of &enum iwl_fw_ini_region_type
+ * @sub_type: region sub type
+ * @sub_type_ver: region sub type version
+ * @reserved: not in use
* @name: region name
* @dev_addr: device address configuration. Used by
* &IWL_FW_INI_REGION_DEVICE_MEMORY, &IWL_FW_INI_REGION_PERIPHERY_MAC,
@@ -146,7 +148,10 @@ struct iwl_fw_ini_region_internal_buffer {
struct iwl_fw_ini_region_tlv {
struct iwl_fw_ini_header hdr;
__le32 id;
- __le32 type;
+ u8 type;
+ u8 sub_type;
+ u8 sub_type_ver;
+ u8 reserved;
u8 name[IWL_FW_INI_MAX_NAME];
union {
struct iwl_fw_ini_region_dev_addr dev_addr;
@@ -306,6 +311,7 @@ enum iwl_fw_ini_config_set_type {
* @IWL_FW_INI_ALLOCATION_ID_DBGC1: allocation meant for DBGC1 configuration
* @IWL_FW_INI_ALLOCATION_ID_DBGC2: allocation meant for DBGC2 configuration
* @IWL_FW_INI_ALLOCATION_ID_DBGC3: allocation meant for DBGC3 configuration
+ * @IWL_FW_INI_ALLOCATION_ID_DBGC4: allocation meant for DBGC4 configuration
* @IWL_FW_INI_ALLOCATION_NUM: number of allocation ids
*/
enum iwl_fw_ini_allocation_id {
@@ -313,6 +319,7 @@ enum iwl_fw_ini_allocation_id {
IWL_FW_INI_ALLOCATION_ID_DBGC1,
IWL_FW_INI_ALLOCATION_ID_DBGC2,
IWL_FW_INI_ALLOCATION_ID_DBGC3,
+ IWL_FW_INI_ALLOCATION_ID_DBGC4,
IWL_FW_INI_ALLOCATION_NUM,
}; /* FW_DEBUG_TLV_ALLOCATION_ID_E_VER_1 */
@@ -379,6 +386,17 @@ enum iwl_fw_ini_region_type {
IWL_FW_INI_REGION_NUM
}; /* FW_TLV_DEBUG_REGION_TYPE_API_E */
+enum iwl_fw_ini_region_device_memory_subtype {
+ IWL_FW_INI_REGION_DEVICE_MEMORY_SUBTYPE_HW_SMEM = 1,
+ IWL_FW_INI_REGION_DEVICE_MEMORY_SUBTYPE_UMAC_ERROR_TABLE = 5,
+ IWL_FW_INI_REGION_DEVICE_MEMORY_SUBTYPE_LMAC_1_ERROR_TABLE = 7,
+ IWL_FW_INI_REGION_DEVICE_MEMORY_SUBTYPE_LMAC_2_ERROR_TABLE = 10,
+ IWL_FW_INI_REGION_DEVICE_MEMORY_SUBTYPE_TCM_1_ERROR_TABLE = 14,
+ IWL_FW_INI_REGION_DEVICE_MEMORY_SUBTYPE_TCM_2_ERROR_TABLE = 16,
+ IWL_FW_INI_REGION_DEVICE_MEMORY_SUBTYPE_RCM_1_ERROR_TABLE = 18,
+ IWL_FW_INI_REGION_DEVICE_MEMORY_SUBTYPE_RCM_2_ERROR_TABLE = 20,
+}; /* FW_TLV_DEBUG_REGION_DEVICE_MEMORY_SUBTYPE_API_E */
+
/**
* enum iwl_fw_ini_time_point
*
@@ -465,4 +483,17 @@ enum iwl_fw_ini_trigger_apply_policy {
IWL_FW_INI_APPLY_POLICY_OVERRIDE_CFG = BIT(9),
IWL_FW_INI_APPLY_POLICY_OVERRIDE_DATA = BIT(10),
};
+
+/**
+ * enum iwl_fw_ini_trigger_reset_fw_policy - Determines how to handle reset
+ *
+ * @IWL_FW_INI_RESET_FW_MODE_NOTHING: do not stop FW and reload (default)
+ * @IWL_FW_INI_RESET_FW_MODE_STOP_FW_ONLY: stop FW without reload FW
+ * @IWL_FW_INI_RESET_FW_MODE_STOP_AND_RELOAD_FW: stop FW with reload FW
+ */
+enum iwl_fw_ini_trigger_reset_fw_policy {
+ IWL_FW_INI_RESET_FW_MODE_NOTHING = 0,
+ IWL_FW_INI_RESET_FW_MODE_STOP_FW_ONLY,
+ IWL_FW_INI_RESET_FW_MODE_STOP_AND_RELOAD_FW
+};
#endif
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/api/nvm-reg.h b/drivers/net/wireless/intel/iwlwifi/fw/api/nvm-reg.h
index 3551a3f1..4949fcf 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/api/nvm-reg.h
+++ b/drivers/net/wireless/intel/iwlwifi/fw/api/nvm-reg.h
@@ -34,6 +34,11 @@ enum iwl_regulatory_and_nvm_subcmd_ids {
TAS_CONFIG = 0x3,
/**
+ * @SAR_OFFSET_MAPPING_TABLE_CMD: &iwl_sar_offset_mapping_cmd
+ */
+ SAR_OFFSET_MAPPING_TABLE_CMD = 0x4,
+
+ /**
* @PNVM_INIT_COMPLETE_NTFY: &struct iwl_pnvm_init_complete_ntfy
*/
PNVM_INIT_COMPLETE_NTFY = 0xFE,
@@ -388,18 +393,33 @@ enum iwl_mcc_source {
MCC_SOURCE_GETTING_MCC_TEST_MODE = 0x11,
};
-#define IWL_TAS_BLACK_LIST_MAX 16
+#define IWL_TAS_BLOCK_LIST_MAX 16
/**
- * struct iwl_tas_config_cmd - configures the TAS
+ * struct iwl_tas_config_cmd_v2 - configures the TAS
* @block_list_size: size of relevant field in block_list_array
- * @block_list_array: block list countries (without TAS)
+ * @block_list_array: list of countries where TAS must be disabled
*/
-struct iwl_tas_config_cmd {
+struct iwl_tas_config_cmd_v2 {
__le32 block_list_size;
- __le32 block_list_array[IWL_TAS_BLACK_LIST_MAX];
+ __le32 block_list_array[IWL_TAS_BLOCK_LIST_MAX];
} __packed; /* TAS_CONFIG_CMD_API_S_VER_2 */
/**
+ * struct iwl_tas_config_cmd_v3 - configures the TAS
+ * @block_list_size: size of relevant field in block_list_array
+ * @block_list_array: list of countries where TAS must be disabled
+ * @override_tas_iec: indicates whether to override default value of IEC regulatory
+ * @enable_tas_iec: in case override_tas_iec is set -
+ * indicates whether IEC regulatory is enabled or disabled
+ */
+struct iwl_tas_config_cmd_v3 {
+ __le32 block_list_size;
+ __le32 block_list_array[IWL_TAS_BLOCK_LIST_MAX];
+ __le16 override_tas_iec;
+ __le16 enable_tas_iec;
+} __packed; /* TAS_CONFIG_CMD_API_S_VER_3 */
+
+/**
* enum iwl_lari_configs - bit masks for the various LARI config operations
* @LARI_CONFIG_DISABLE_11AC_UKRAINE_MSK: disable 11ac in ukraine
* @LARI_CONFIG_CHANGE_ETSI_TO_PASSIVE_MSK: ETSI 5.8GHz SRD passive scan
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/api/phy-ctxt.h b/drivers/net/wireless/intel/iwlwifi/fw/api/phy-ctxt.h
index 68b788b..e66f779 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/api/phy-ctxt.h
+++ b/drivers/net/wireless/intel/iwlwifi/fw/api/phy-ctxt.h
@@ -1,6 +1,6 @@
/* SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause */
/*
- * Copyright (C) 2012-2014, 2018, 2020 Intel Corporation
+ * Copyright (C) 2012-2014, 2018, 2020-2021 Intel Corporation
* Copyright (C) 2013-2015 Intel Mobile Communications GmbH
* Copyright (C) 2016-2017 Intel Deutschland GmbH
*/
@@ -150,11 +150,12 @@ struct iwl_phy_context_cmd {
/* COMMON_INDEX_HDR_API_S_VER_1 */
__le32 id_and_color;
__le32 action;
- /* PHY_CONTEXT_DATA_API_S_VER_3 */
+ /* PHY_CONTEXT_DATA_API_S_VER_3, PHY_CONTEXT_DATA_API_S_VER_4 */
struct iwl_fw_channel_info ci;
__le32 lmac_id;
- __le32 rxchain_info;
+ __le32 rxchain_info; /* reserved in _VER_4 */
__le32 dsp_cfg_flags;
__le32 reserved;
-} __packed; /* PHY_CONTEXT_CMD_API_VER_3 */
+} __packed; /* PHY_CONTEXT_CMD_API_VER_3, PHY_CONTEXT_CMD_API_VER_4 */
+
#endif /* __iwl_fw_api_phy_ctxt_h__ */
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/api/power.h b/drivers/net/wireless/intel/iwlwifi/fw/api/power.h
index 4d671c8..8131820 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/api/power.h
+++ b/drivers/net/wireless/intel/iwlwifi/fw/api/power.h
@@ -419,7 +419,7 @@ struct iwl_geo_tx_power_profiles_cmd_v1 {
* struct iwl_geo_tx_power_profile_cmd_v2 - struct for PER_CHAIN_LIMIT_OFFSET_CMD cmd.
* @ops: operations, value from &enum iwl_geo_per_chain_offset_operation
* @table: offset profile per band.
- * @table_revision: BIOS table revision.
+ * @table_revision: 0 for not-South Korea, 1 for South Korea (the name is misleading)
*/
struct iwl_geo_tx_power_profiles_cmd_v2 {
__le32 ops;
@@ -431,7 +431,7 @@ struct iwl_geo_tx_power_profiles_cmd_v2 {
* struct iwl_geo_tx_power_profile_cmd_v3 - struct for PER_CHAIN_LIMIT_OFFSET_CMD cmd.
* @ops: operations, value from &enum iwl_geo_per_chain_offset_operation
* @table: offset profile per band.
- * @table_revision: BIOS table revision.
+ * @table_revision: 0 for not-South Korea, 1 for South Korea (the name is misleading)
*/
struct iwl_geo_tx_power_profiles_cmd_v3 {
__le32 ops;
@@ -443,7 +443,7 @@ struct iwl_geo_tx_power_profiles_cmd_v3 {
* struct iwl_geo_tx_power_profile_cmd_v4 - struct for PER_CHAIN_LIMIT_OFFSET_CMD cmd.
* @ops: operations, value from &enum iwl_geo_per_chain_offset_operation
* @table: offset profile per band.
- * @table_revision: BIOS table revision.
+ * @table_revision: 0 for not-South Korea, 1 for South Korea (the name is misleading)
*/
struct iwl_geo_tx_power_profiles_cmd_v4 {
__le32 ops;
@@ -455,7 +455,7 @@ struct iwl_geo_tx_power_profiles_cmd_v4 {
* struct iwl_geo_tx_power_profile_cmd_v5 - struct for PER_CHAIN_LIMIT_OFFSET_CMD cmd.
* @ops: operations, value from &enum iwl_geo_per_chain_offset_operation
* @table: offset profile per band.
- * @table_revision: BIOS table revision.
+ * @table_revision: 0 for not-South Korea, 1 for South Korea (the name is misleading)
*/
struct iwl_geo_tx_power_profiles_cmd_v5 {
__le32 ops;
@@ -503,6 +503,20 @@ union iwl_ppag_table_cmd {
} v2;
} __packed;
+#define MCC_TO_SAR_OFFSET_TABLE_ROW_SIZE 26
+#define MCC_TO_SAR_OFFSET_TABLE_COL_SIZE 13
+
+/**
+ * struct iwl_sar_offset_mapping_cmd - struct for SAR_OFFSET_MAPPING_TABLE_CMD
+ * @offset_map: mapping a mcc to a geo sar group
+ * @reserved: reserved
+ */
+struct iwl_sar_offset_mapping_cmd {
+ u8 offset_map[MCC_TO_SAR_OFFSET_TABLE_ROW_SIZE]
+ [MCC_TO_SAR_OFFSET_TABLE_COL_SIZE];
+ u16 reserved;
+} __packed; /*SAR_OFFSET_MAPPING_TABLE_CMD_API_S*/
+
/**
* struct iwl_beacon_filter_cmd
* REPLY_BEACON_FILTERING_CMD = 0xd2 (command)
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/api/rs.h b/drivers/net/wireless/intel/iwlwifi/fw/api/rs.h
index a09081d..173a699 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/api/rs.h
+++ b/drivers/net/wireless/intel/iwlwifi/fw/api/rs.h
@@ -116,9 +116,20 @@ enum IWL_TLC_MNG_NSS {
IWL_TLC_NSS_MAX
};
-enum IWL_TLC_HT_BW_RATES {
- IWL_TLC_HT_BW_NONE_160,
- IWL_TLC_HT_BW_160,
+/**
+ * enum IWL_TLC_MCS_PER_BW - mcs index per BW
+ * @IWL_TLC_MCS_PER_BW_80: mcs for bw - 20Hhz, 40Hhz, 80Hhz
+ * @IWL_TLC_MCS_PER_BW_160: mcs for bw - 160Mhz
+ * @IWL_TLC_MCS_PER_BW_320: mcs for bw - 320Mhz
+ * @IWL_TLC_MCS_PER_BW_NUM_V3: number of entries up to version 3
+ * @IWL_TLC_MCS_PER_BW_NUM_V4: number of entries from version 4
+ */
+enum IWL_TLC_MCS_PER_BW {
+ IWL_TLC_MCS_PER_BW_80,
+ IWL_TLC_MCS_PER_BW_160,
+ IWL_TLC_MCS_PER_BW_320,
+ IWL_TLC_MCS_PER_BW_NUM_V3 = IWL_TLC_MCS_PER_BW_160 + 1,
+ IWL_TLC_MCS_PER_BW_NUM_V4 = IWL_TLC_MCS_PER_BW_320 + 1,
};
/**
@@ -131,8 +142,8 @@ enum IWL_TLC_HT_BW_RATES {
* @amsdu: TX amsdu is supported
* @flags: bitmask of &enum iwl_tlc_mng_cfg_flags
* @non_ht_rates: bitmap of supported legacy rates
- * @ht_rates: bitmap of &enum iwl_tlc_mng_ht_rates, per <nss, channel-width>
- * pair (0 - 80mhz width and below, 1 - 160mhz).
+ * @ht_rates: bitmap of &enum iwl_tlc_mng_ht_rates, per &enum IWL_TLC_MCS_PER_BW
+ * <nss, channel-width> pair (0 - 80mhz width and below, 1 - 160mhz).
* @max_mpdu_len: max MPDU length, in bytes
* @sgi_ch_width_supp: bitmap of SGI support per channel width
* use BIT(@enum iwl_tlc_mng_cfg_cw)
@@ -140,7 +151,7 @@ enum IWL_TLC_HT_BW_RATES {
* @max_tx_op: max TXOP in uSecs for all AC (BK, BE, VO, VI),
* set zero for no limit.
*/
-struct iwl_tlc_config_cmd {
+struct iwl_tlc_config_cmd_v3 {
u8 sta_id;
u8 reserved1[3];
u8 max_ch_width;
@@ -149,7 +160,7 @@ struct iwl_tlc_config_cmd {
u8 amsdu;
__le16 flags;
__le16 non_ht_rates;
- __le16 ht_rates[IWL_TLC_NSS_MAX][2];
+ __le16 ht_rates[IWL_TLC_NSS_MAX][IWL_TLC_MCS_PER_BW_NUM_V3];
__le16 max_mpdu_len;
u8 sgi_ch_width_supp;
u8 reserved2;
@@ -157,6 +168,37 @@ struct iwl_tlc_config_cmd {
} __packed; /* TLC_MNG_CONFIG_CMD_API_S_VER_3 */
/**
+ * struct tlc_config_cmd - TLC configuration
+ * @sta_id: station id
+ * @reserved1: reserved
+ * @max_ch_width: max supported channel width from &enum iwl_tlc_mng_cfg_cw
+ * @mode: &enum iwl_tlc_mng_cfg_mode
+ * @chains: bitmask of &enum iwl_tlc_mng_cfg_chains
+ * @sgi_ch_width_supp: bitmap of SGI support per channel width
+ * use BIT(&enum iwl_tlc_mng_cfg_cw)
+ * @flags: bitmask of &enum iwl_tlc_mng_cfg_flags
+ * @non_ht_rates: bitmap of supported legacy rates
+ * @ht_rates: bitmap of &enum iwl_tlc_mng_ht_rates, per <nss, channel-width>
+ * pair (0 - 80mhz width and below, 1 - 160mhz, 2 - 320mhz).
+ * @max_mpdu_len: max MPDU length, in bytes
+ * @max_tx_op: max TXOP in uSecs for all AC (BK, BE, VO, VI),
+ * set zero for no limit.
+ */
+struct iwl_tlc_config_cmd_v4 {
+ u8 sta_id;
+ u8 reserved1[3];
+ u8 max_ch_width;
+ u8 mode;
+ u8 chains;
+ u8 sgi_ch_width_supp;
+ __le16 flags;
+ __le16 non_ht_rates;
+ __le16 ht_rates[IWL_TLC_NSS_MAX][IWL_TLC_MCS_PER_BW_NUM_V4];
+ __le16 max_mpdu_len;
+ __le16 max_tx_op;
+} __packed; /* TLC_MNG_CONFIG_CMD_API_S_VER_4 */
+
+/**
* enum iwl_tlc_update_flags - updated fields
* @IWL_TLC_NOTIF_FLAG_RATE: last initial rate update
* @IWL_TLC_NOTIF_FLAG_AMSDU: umsdu parameters update
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/api/scan.h b/drivers/net/wireless/intel/iwlwifi/fw/api/scan.h
index 8b20037..5413087 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/api/scan.h
+++ b/drivers/net/wireless/intel/iwlwifi/fw/api/scan.h
@@ -82,6 +82,16 @@ enum iwl_scan_offload_band_selection {
IWL_SCAN_OFFLOAD_SELECT_ANY = 0xc,
};
+enum iwl_scan_offload_auth_alg {
+ IWL_AUTH_ALGO_UNSUPPORTED = 0x00,
+ IWL_AUTH_ALGO_NONE = 0x01,
+ IWL_AUTH_ALGO_PSK = 0x02,
+ IWL_AUTH_ALGO_8021X = 0x04,
+ IWL_AUTH_ALGO_SAE = 0x08,
+ IWL_AUTH_ALGO_8021X_SHA384 = 0x10,
+ IWL_AUTH_ALGO_OWE = 0x20,
+};
+
/**
* struct iwl_scan_offload_profile - SCAN_OFFLOAD_PROFILE_S
* @ssid_index: index to ssid list in fixed part
@@ -201,7 +211,7 @@ struct iwl_scan_channel_cfg_lmac {
__le32 iter_interval;
} __packed;
-/*
+/**
* struct iwl_scan_probe_segment - PROBE_SEGMENT_API_S_VER_1
* @offset: offset in the data block
* @len: length of the segment
@@ -211,7 +221,8 @@ struct iwl_scan_probe_segment {
__le16 len;
} __packed;
-/* iwl_scan_probe_req - PROBE_REQUEST_FRAME_API_S_VER_2
+/**
+ * struct iwl_scan_probe_req_v1 - PROBE_REQUEST_FRAME_API_S_VER_2
* @mac_header: first (and common) part of the probe
* @band_data: band specific data
* @common_data: last (and common) part of the probe
@@ -224,7 +235,8 @@ struct iwl_scan_probe_req_v1 {
u8 buf[SCAN_OFFLOAD_PROBE_REQ_SIZE];
} __packed;
-/* iwl_scan_probe_req - PROBE_REQUEST_FRAME_API_S_VER_v2
+/**
+ * struct iwl_scan_probe_req - PROBE_REQUEST_FRAME_API_S_VER_v2
* @mac_header: first (and common) part of the probe
* @band_data: band specific data
* @common_data: last (and common) part of the probe
@@ -247,7 +259,8 @@ enum iwl_scan_channel_flags {
IWL_SCAN_CHANNEL_FLAG_6G_PSC_NO_FILTER = BIT(6),
};
-/* struct iwl_scan_channel_opt - CHANNEL_OPTIMIZATION_API_S
+/**
+ * struct iwl_scan_channel_opt - CHANNEL_OPTIMIZATION_API_S
* @flags: enum iwl_scan_channel_flags
* @non_ebs_ratio: defines the ratio of number of scan iterations where EBS is
* involved.
@@ -492,7 +505,7 @@ struct iwl_scan_dwell {
} __packed;
/**
- * struct iwl_scan_config_v1
+ * struct iwl_scan_config_v1 - scan configuration command
* @flags: enum scan_config_flags
* @tx_chains: valid_tx antenna - ANT_* definitions
* @rx_chains: valid_rx antenna - ANT_* definitions
@@ -524,6 +537,21 @@ struct iwl_scan_config_v1 {
#define SCAN_LB_LMAC_IDX 0
#define SCAN_HB_LMAC_IDX 1
+/**
+ * struct iwl_scan_config_v2 - scan configuration command
+ * @flags: enum scan_config_flags
+ * @tx_chains: valid_tx antenna - ANT_* definitions
+ * @rx_chains: valid_rx antenna - ANT_* definitions
+ * @legacy_rates: default legacy rates - enum scan_config_rates
+ * @out_of_channel_time: default max out of serving channel time
+ * @suspend_time: default max suspend time
+ * @dwell: dwells for the scan
+ * @mac_addr: default mac address to be used in probes
+ * @bcast_sta_id: the index of the station in the fw
+ * @channel_flags: default channel flags - enum iwl_channel_flags
+ * scan_config_channel_flag
+ * @channel_array: default supported channels
+ */
struct iwl_scan_config_v2 {
__le32 flags;
__le32 tx_chains;
@@ -539,7 +567,7 @@ struct iwl_scan_config_v2 {
} __packed; /* SCAN_CONFIG_DB_CMD_API_S_2 */
/**
- * struct iwl_scan_config
+ * struct iwl_scan_config - scan configuration command
* @enable_cam_mode: whether to enable CAM mode.
* @enable_promiscouos_mode: whether to enable promiscouos mode
* @bcast_sta_id: the index of the station in the fw. Deprecated starting with
@@ -640,6 +668,10 @@ enum iwl_umac_scan_general_flags2 {
* @IWL_UMAC_SCAN_GEN_FLAGS_V2_6GHZ_PASSIVE_SCAN_FILTER_IN: in case
* &IWL_UMAC_SCAN_GEN_FLAGS_V2_6GHZ_PASSIVE_SCAN is enabled and scan is
* activated over 6GHz PSC channels, filter in beacons and probe responses.
+ * @IWL_UMAC_SCAN_GEN_FLAGS_V2_OCE: if set, send probe requests in a minimum
+ * rate of 5.5Mpbs, filter in broadcast probe responses and set the max
+ * channel time indication field in the FILS request parameters element
+ * (if included by the driver in the probe request IEs).
*/
enum iwl_umac_scan_general_flags_v2 {
IWL_UMAC_SCAN_GEN_FLAGS_V2_PERIODIC = BIT(0),
@@ -657,6 +689,20 @@ enum iwl_umac_scan_general_flags_v2 {
IWL_UMAC_SCAN_GEN_FLAGS_V2_TRIGGER_UHB_SCAN = BIT(12),
IWL_UMAC_SCAN_GEN_FLAGS_V2_6GHZ_PASSIVE_SCAN = BIT(13),
IWL_UMAC_SCAN_GEN_FLAGS_V2_6GHZ_PASSIVE_SCAN_FILTER_IN = BIT(14),
+ IWL_UMAC_SCAN_GEN_FLAGS_V2_OCE = BIT(15),
+};
+
+/**
+ * enum iwl_umac_scan_general_params_flags2 - UMAC scan general flags2
+ *
+ * @IWL_UMAC_SCAN_GEN_PARAMS_FLAGS2_RESPECT_P2P_GO_LB: scan event scheduling
+ * should be aware of a P2P GO operation on the 2GHz band.
+ * @IWL_UMAC_SCAN_GEN_PARAMS_FLAGS2_RESPECT_P2P_GO_HB: scan event scheduling
+ * should be aware of a P2P GO operation on the 5GHz or 6GHz band.
+ */
+enum iwl_umac_scan_general_params_flags2 {
+ IWL_UMAC_SCAN_GEN_PARAMS_FLAGS2_RESPECT_P2P_GO_LB = BIT(0),
+ IWL_UMAC_SCAN_GEN_PARAMS_FLAGS2_RESPECT_P2P_GO_HB = BIT(1),
};
/**
@@ -941,8 +987,8 @@ struct iwl_scan_channel_params_v6 {
} __packed; /* SCAN_CHANNEL_PARAMS_API_S_VER_6 */
/**
- * struct iwl_scan_general_params_v10
- * @flags: &enum iwl_umac_scan_flags
+ * struct iwl_scan_general_params_v11
+ * @flags: &enum iwl_umac_scan_general_flags_v2
* @reserved: reserved for future
* @scan_start_mac_id: report the scan start TSF time according to this mac TSF
* @active_dwell: dwell time for active scan per LMAC
@@ -952,7 +998,8 @@ struct iwl_scan_channel_params_v6 {
* for 5GHz channels
* @adwell_default_social_chn: adaptive dwell default number of
* APs per social channel
- * @reserved1: reserved for future
+ * @flags2: for version 11 see &enum iwl_umac_scan_general_params_flags2.
+ * Otherwise reserved.
* @adwell_max_budget: the maximal number of TUs that adaptive dwell
* can add to the total scan time
* @max_out_of_time: max out of serving channel time, per LMAC
@@ -963,7 +1010,7 @@ struct iwl_scan_channel_params_v6 {
* @num_of_fragments: number of fragments needed for full fragmented
* scan coverage.
*/
-struct iwl_scan_general_params_v10 {
+struct iwl_scan_general_params_v11 {
__le16 flags;
u8 reserved;
u8 scan_start_mac_id;
@@ -971,14 +1018,14 @@ struct iwl_scan_general_params_v10 {
u8 adwell_default_2g;
u8 adwell_default_5g;
u8 adwell_default_social_chn;
- u8 reserved1;
+ u8 flags2;
__le16 adwell_max_budget;
__le32 max_out_of_time[SCAN_TWO_LMACS];
__le32 suspend_time[SCAN_TWO_LMACS];
__le32 scan_priority;
u8 passive_dwell[SCAN_TWO_LMACS];
u8 num_of_fragments[SCAN_TWO_LMACS];
-} __packed; /* SCAN_GENERAL_PARAMS_API_S_VER_10 */
+} __packed; /* SCAN_GENERAL_PARAMS_API_S_VER_11 and *_VER_10 */
/**
* struct iwl_scan_periodic_parms_v1
@@ -994,31 +1041,31 @@ struct iwl_scan_periodic_parms_v1 {
/**
* struct iwl_scan_req_params_v12
- * @general_params: &struct iwl_scan_general_params_v10
+ * @general_params: &struct iwl_scan_general_params_v11
* @channel_params: &struct iwl_scan_channel_params_v4
* @periodic_params: &struct iwl_scan_periodic_parms_v1
* @probe_params: &struct iwl_scan_probe_params_v3
*/
struct iwl_scan_req_params_v12 {
- struct iwl_scan_general_params_v10 general_params;
+ struct iwl_scan_general_params_v11 general_params;
struct iwl_scan_channel_params_v4 channel_params;
struct iwl_scan_periodic_parms_v1 periodic_params;
struct iwl_scan_probe_params_v3 probe_params;
} __packed; /* SCAN_REQUEST_PARAMS_API_S_VER_12 */
/**
- * struct iwl_scan_req_params_v14
- * @general_params: &struct iwl_scan_general_params_v10
+ * struct iwl_scan_req_params_v15
+ * @general_params: &struct iwl_scan_general_params_v11
* @channel_params: &struct iwl_scan_channel_params_v6
* @periodic_params: &struct iwl_scan_periodic_parms_v1
* @probe_params: &struct iwl_scan_probe_params_v4
*/
-struct iwl_scan_req_params_v14 {
- struct iwl_scan_general_params_v10 general_params;
+struct iwl_scan_req_params_v15 {
+ struct iwl_scan_general_params_v11 general_params;
struct iwl_scan_channel_params_v6 channel_params;
struct iwl_scan_periodic_parms_v1 periodic_params;
struct iwl_scan_probe_params_v4 probe_params;
-} __packed; /* SCAN_REQUEST_PARAMS_API_S_VER_14 */
+} __packed; /* SCAN_REQUEST_PARAMS_API_S_VER_15 and *_VER_14 */
/**
* struct iwl_scan_req_umac_v12
@@ -1033,16 +1080,16 @@ struct iwl_scan_req_umac_v12 {
} __packed; /* SCAN_REQUEST_CMD_UMAC_API_S_VER_12 */
/**
- * struct iwl_scan_req_umac_v14
+ * struct iwl_scan_req_umac_v15
* @uid: scan id, &enum iwl_umac_scan_uid_offsets
* @ooc_priority: out of channel priority - &enum iwl_scan_priority
* @scan_params: scan parameters
*/
-struct iwl_scan_req_umac_v14 {
+struct iwl_scan_req_umac_v15 {
__le32 uid;
__le32 ooc_priority;
- struct iwl_scan_req_params_v14 scan_params;
-} __packed; /* SCAN_REQUEST_CMD_UMAC_API_S_VER_14 */
+ struct iwl_scan_req_params_v15 scan_params;
+} __packed; /* SCAN_REQUEST_CMD_UMAC_API_S_VER_15 and *_VER_14 */
/**
* struct iwl_umac_scan_abort
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/api/soc.h b/drivers/net/wireless/intel/iwlwifi/fw/api/soc.h
deleted file mode 100644
index c5df1171..0000000
--- a/drivers/net/wireless/intel/iwlwifi/fw/api/soc.h
+++ /dev/null
@@ -1,35 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause */
-/*
- * Copyright (C) 2012-2014, 2019-2020 Intel Corporation
- * Copyright (C) 2013-2015 Intel Mobile Communications GmbH
- * Copyright (C) 2016-2017 Intel Deutschland GmbH
- */
-#ifndef __iwl_fw_api_soc_h__
-#define __iwl_fw_api_soc_h__
-
-#define SOC_CONFIG_CMD_FLAGS_DISCRETE BIT(0)
-#define SOC_CONFIG_CMD_FLAGS_LOW_LATENCY BIT(1)
-
-#define SOC_FLAGS_LTR_APPLY_DELAY_MASK 0xc
-#define SOC_FLAGS_LTR_APPLY_DELAY_NONE 0
-#define SOC_FLAGS_LTR_APPLY_DELAY_200 1
-#define SOC_FLAGS_LTR_APPLY_DELAY_2500 2
-#define SOC_FLAGS_LTR_APPLY_DELAY_1820 3
-
-/**
- * struct iwl_soc_configuration_cmd - Set device stabilization latency
- *
- * @flags: soc settings flags. In VER_1, we can only set the DISCRETE
- * flag, because the FW treats the whole value as an integer. In
- * VER_2, we can set the bits independently.
- * @latency: time for SOC to ensure stable power & XTAL
- */
-struct iwl_soc_configuration_cmd {
- __le32 flags;
- __le32 latency;
-} __packed; /*
- * SOC_CONFIGURATION_CMD_S_VER_1 (see description above)
- * SOC_CONFIGURATION_CMD_S_VER_2
- */
-
-#endif /* __iwl_fw_api_soc_h__ */
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/api/stats.h b/drivers/net/wireless/intel/iwlwifi/fw/api/stats.h
index 18cca15..898e623 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/api/stats.h
+++ b/drivers/net/wireless/intel/iwlwifi/fw/api/stats.h
@@ -1,6 +1,6 @@
/* SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause */
/*
- * Copyright (C) 2012-2014, 2018, 2020 Intel Corporation
+ * Copyright (C) 2012-2014, 2018, 2020 - 2021 Intel Corporation
* Copyright (C) 2013-2015 Intel Mobile Communications GmbH
* Copyright (C) 2016-2017 Intel Deutschland GmbH
*/
@@ -432,6 +432,7 @@ enum iwl_fw_statistics_type {
FW_STATISTICS_HE,
}; /* FW_STATISTICS_TYPE_API_E_VER_1 */
+#define IWL_STATISTICS_TYPE_MSK 0x7f
/**
* struct iwl_statistics_ntfy_hdr
*
@@ -446,10 +447,97 @@ struct iwl_statistics_ntfy_hdr {
}; /* STATISTICS_NTFY_HDR_API_S_VER_1 */
/**
+ * struct iwl_statistics_ntfy_per_mac
+ *
+ * @beacon_filter_average_energy: Average energy [-dBm] of the 2
+ * antennas.
+ * @air_time: air time
+ * @beacon_counter: all beacons (both filtered and not filtered)
+ * @beacon_average_energy: all beacons (both filtered and not
+ * filtered)
+ * @beacon_rssi_a: beacon RSSI on antenna A
+ * @beacon_rssi_b: beacon RSSI on antenna B
+ * @rx_bytes: RX byte count
+ */
+struct iwl_statistics_ntfy_per_mac {
+ __le32 beacon_filter_average_energy;
+ __le32 air_time;
+ __le32 beacon_counter;
+ __le32 beacon_average_energy;
+ __le32 beacon_rssi_a;
+ __le32 beacon_rssi_b;
+ __le32 rx_bytes;
+} __packed; /* STATISTICS_NTFY_PER_MAC_API_S_VER_1 */
+
+#define IWL_STATS_MAX_BW_INDEX 5
+/** struct iwl_statistics_ntfy_per_phy
+ * @channel_load: channel load
+ * @channel_load_by_us: device contribution to MCLM
+ * @channel_load_not_by_us: other devices' contribution to MCLM
+ * @clt: CLT HW timer (TIM_CH_LOAD2)
+ * @act: active accumulator SW
+ * @elp: elapsed time accumulator SW
+ * @rx_detected_per_ch_width: number of deferred TX per channel width,
+ * 0 - 20, 1/2/3 - 40/80/160
+ * @success_per_ch_width: number of frames that got ACK/BACK/CTS
+ * per channel BW. note, BACK counted as 1
+ * @fail_per_ch_width: number of frames that didn't get ACK/BACK/CTS
+ * per channel BW. note BACK counted as 1
+ * @last_tx_ch_width_indx: last txed frame channel width index
+ */
+struct iwl_statistics_ntfy_per_phy {
+ __le32 channel_load;
+ __le32 channel_load_by_us;
+ __le32 channel_load_not_by_us;
+ __le32 clt;
+ __le32 act;
+ __le32 elp;
+ __le32 rx_detected_per_ch_width[IWL_STATS_MAX_BW_INDEX];
+ __le32 success_per_ch_width[IWL_STATS_MAX_BW_INDEX];
+ __le32 fail_per_ch_width[IWL_STATS_MAX_BW_INDEX];
+ __le32 last_tx_ch_width_indx;
+} __packed; /* STATISTICS_NTFY_PER_PHY_API_S_VER_1 */
+
+/**
+ * struct iwl_statistics_ntfy_per_sta
+ *
+ * @average_energy: in fact it is minus the energy..
+ */
+struct iwl_statistics_ntfy_per_sta {
+ __le32 average_energy;
+} __packed; /* STATISTICS_NTFY_PER_STA_API_S_VER_1 */
+
+#define IWL_STATS_MAX_PHY_OPERTINAL 3
+/**
* struct iwl_statistics_operational_ntfy
*
* @hdr: general statistics header
* @flags: bitmap of possible notification structures
+ * @per_mac_stats: per mac statistics, &struct iwl_statistics_ntfy_per_mac
+ * @per_phy_stats: per phy statistics, &struct iwl_statistics_ntfy_per_phy
+ * @per_sta_stats: per sta statistics, &struct iwl_statistics_ntfy_per_sta
+ * @rx_time: rx time
+ * @tx_time: usec the radio is transmitting.
+ * @on_time_rf: The total time in usec the RF is awake.
+ * @on_time_scan: usec the radio is awake due to scan.
+ */
+struct iwl_statistics_operational_ntfy {
+ struct iwl_statistics_ntfy_hdr hdr;
+ __le32 flags;
+ struct iwl_statistics_ntfy_per_mac per_mac_stats[MAC_INDEX_AUX];
+ struct iwl_statistics_ntfy_per_phy per_phy_stats[IWL_STATS_MAX_PHY_OPERTINAL];
+ struct iwl_statistics_ntfy_per_sta per_sta_stats[IWL_MVM_STATION_COUNT_MAX];
+ __le64 rx_time;
+ __le64 tx_time;
+ __le64 on_time_rf;
+ __le64 on_time_scan;
+} __packed; /* STATISTICS_OPERATIONAL_NTFY_API_S_VER_15 */
+
+/**
+ * struct iwl_statistics_operational_ntfy_ver_14
+ *
+ * @hdr: general statistics header
+ * @flags: bitmap of possible notification structures
* @mac_id: mac on which the beacon was received
* @beacon_filter_average_energy: Average energy [-dBm] of the 2
* antennas.
@@ -469,7 +557,7 @@ struct iwl_statistics_ntfy_hdr {
* @average_energy: in fact it is minus the energy..
* @reserved: reserved
*/
-struct iwl_statistics_operational_ntfy {
+struct iwl_statistics_operational_ntfy_ver_14 {
struct iwl_statistics_ntfy_hdr hdr;
__le32 flags;
__le32 mac_id;
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/api/system.h b/drivers/net/wireless/intel/iwlwifi/fw/api/system.h
new file mode 100644
index 0000000..acf5d4b9
--- /dev/null
+++ b/drivers/net/wireless/intel/iwlwifi/fw/api/system.h
@@ -0,0 +1,43 @@
+/* SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause */
+/*
+ * Copyright (C) 2012-2014, 2019-2021 Intel Corporation
+ * Copyright (C) 2013-2015 Intel Mobile Communications GmbH
+ * Copyright (C) 2016-2017 Intel Deutschland GmbH
+ */
+#ifndef __iwl_fw_api_system_h__
+#define __iwl_fw_api_system_h__
+
+#define SOC_CONFIG_CMD_FLAGS_DISCRETE BIT(0)
+#define SOC_CONFIG_CMD_FLAGS_LOW_LATENCY BIT(1)
+
+#define SOC_FLAGS_LTR_APPLY_DELAY_MASK 0xc
+#define SOC_FLAGS_LTR_APPLY_DELAY_NONE 0
+#define SOC_FLAGS_LTR_APPLY_DELAY_200 1
+#define SOC_FLAGS_LTR_APPLY_DELAY_2500 2
+#define SOC_FLAGS_LTR_APPLY_DELAY_1820 3
+
+/**
+ * struct iwl_soc_configuration_cmd - Set device stabilization latency
+ *
+ * @flags: soc settings flags. In VER_1, we can only set the DISCRETE
+ * flag, because the FW treats the whole value as an integer. In
+ * VER_2, we can set the bits independently.
+ * @latency: time for SOC to ensure stable power & XTAL
+ */
+struct iwl_soc_configuration_cmd {
+ __le32 flags;
+ __le32 latency;
+} __packed; /*
+ * SOC_CONFIGURATION_CMD_S_VER_1 (see description above)
+ * SOC_CONFIGURATION_CMD_S_VER_2
+ */
+
+/**
+ * struct iwl_system_features_control_cmd - system features control command
+ * @features: bitmap of features to disable
+ */
+struct iwl_system_features_control_cmd {
+ __le32 features[4];
+} __packed; /* SYSTEM_FEATURES_CONTROL_CMD_API_S_VER_1 */
+
+#endif /* __iwl_fw_api_system_h__ */
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/api/tx.h b/drivers/net/wireless/intel/iwlwifi/fw/api/tx.h
index 4a74c0e..e73cc73 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/api/tx.h
+++ b/drivers/net/wireless/intel/iwlwifi/fw/api/tx.h
@@ -177,6 +177,17 @@ enum iwl_tx_offload_assist_flags_pos {
#define IWL_TX_CMD_OFFLD_MH_MASK 0x1f
#define IWL_TX_CMD_OFFLD_IP_HDR_MASK 0x3f
+enum iwl_tx_offload_assist_bz {
+ IWL_TX_CMD_OFFLD_BZ_RESULT_OFFS = 0x000003ff,
+ IWL_TX_CMD_OFFLD_BZ_START_OFFS = 0x001ff800,
+ IWL_TX_CMD_OFFLD_BZ_MH_LEN = 0x07c00000,
+ IWL_TX_CMD_OFFLD_BZ_MH_PAD = 0x08000000,
+ IWL_TX_CMD_OFFLD_BZ_AMSDU = 0x10000000,
+ IWL_TX_CMD_OFFLD_BZ_ZERO2ONES = 0x20000000,
+ IWL_TX_CMD_OFFLD_BZ_ENABLE_CSUM = 0x40000000,
+ IWL_TX_CMD_OFFLD_BZ_PARTIAL_CSUM = 0x80000000,
+};
+
/* TODO: complete documentation for try_cnt and btkill_cnt */
/**
* struct iwl_tx_cmd - TX command struct to FW
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/dbg.c b/drivers/net/wireless/intel/iwlwifi/fw/dbg.c
index a39013c..7ad9cee9 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/dbg.c
+++ b/drivers/net/wireless/intel/iwlwifi/fw/dbg.c
@@ -880,7 +880,7 @@ iwl_fw_error_dump_file(struct iwl_fw_runtime *fwrt,
dump_info->hw_type =
cpu_to_le32(CSR_HW_REV_TYPE(fwrt->trans->hw_rev));
dump_info->hw_step =
- cpu_to_le32(CSR_HW_REV_STEP(fwrt->trans->hw_rev));
+ cpu_to_le32(fwrt->trans->hw_rev_step);
memcpy(dump_info->fw_human_readable, fwrt->fw->human_readable,
sizeof(dump_info->fw_human_readable));
strncpy(dump_info->dev_human_readable, fwrt->trans->name,
@@ -1165,8 +1165,7 @@ static int iwl_dump_ini_dev_mem_iter(struct iwl_fw_runtime *fwrt,
iwl_trans_read_mem_bytes(fwrt->trans, addr, range->data,
le32_to_cpu(reg->dev_addr.size));
- if ((le32_to_cpu(reg->id) & IWL_FW_INI_REGION_V2_MASK) ==
- IWL_FW_INI_HW_SMEM_REGION_ID &&
+ if (reg->sub_type == IWL_FW_INI_REGION_DEVICE_MEMORY_SUBTYPE_HW_SMEM &&
fwrt->sanitize_ops && fwrt->sanitize_ops->frob_txf)
fwrt->sanitize_ops->frob_txf(fwrt->sanitize_ctx,
range->data,
@@ -1565,7 +1564,7 @@ iwl_dump_ini_dbgi_sram_iter(struct iwl_fw_runtime *fwrt,
iwl_write_prph_no_grab(fwrt->trans, DBGI_SRAM_TARGET_ACCESS_CFG,
DBGI_SRAM_TARGET_ACCESS_CFG_RESET_ADDRESS_MSK);
for (i = 0; i < (le32_to_cpu(reg->dev_addr.size) / 4); i++) {
- prph_data = iwl_read_prph(fwrt->trans, (i % 2) ?
+ prph_data = iwl_read_prph_no_grab(fwrt->trans, (i % 2) ?
DBGI_SRAM_TARGET_ACCESS_RDATA_MSB :
DBGI_SRAM_TARGET_ACCESS_RDATA_LSB);
if (prph_data == 0x5a5a5a5a) {
@@ -1988,17 +1987,18 @@ static u32 iwl_dump_ini_mem(struct iwl_fw_runtime *fwrt, struct list_head *list,
{
struct iwl_fw_ini_region_tlv *reg = (void *)reg_data->reg_tlv->data;
struct iwl_fw_ini_dump_entry *entry;
- struct iwl_fw_error_dump_data *tlv;
+ struct iwl_fw_ini_error_dump_data *tlv;
struct iwl_fw_ini_error_dump_header *header;
- u32 type = le32_to_cpu(reg->type), id = le32_to_cpu(reg->id);
+ u32 type = reg->type;
+ u32 id = le32_to_cpu(reg->id);
u32 num_of_ranges, i, size;
void *range;
/*
- * The higher part of the ID in version 2 is irrelevant for
+ * The higher part of the ID from 2 is irrelevant for
* us, so mask it out.
*/
- if (le32_to_cpu(reg->hdr.version) == 2)
+ if (le32_to_cpu(reg->hdr.version) >= 2)
id &= IWL_FW_INI_REGION_V2_MASK;
if (!ops->get_num_of_ranges || !ops->get_size || !ops->fill_mem_hdr ||
@@ -2017,6 +2017,9 @@ static u32 iwl_dump_ini_mem(struct iwl_fw_runtime *fwrt, struct list_head *list,
tlv = (void *)entry->data;
tlv->type = reg->type;
+ tlv->sub_type = reg->sub_type;
+ tlv->sub_type_ver = reg->sub_type_ver;
+ tlv->reserved = reg->reserved;
tlv->len = cpu_to_le32(size);
IWL_DEBUG_FW(fwrt, "WRT: Collecting region: id=%d, type=%d\n", id,
@@ -2099,7 +2102,7 @@ static u32 iwl_dump_ini_info(struct iwl_fw_runtime *fwrt,
dump->ver_type = cpu_to_le32(fwrt->dump.fw_ver.type);
dump->ver_subtype = cpu_to_le32(fwrt->dump.fw_ver.subtype);
- dump->hw_step = cpu_to_le32(CSR_HW_REV_STEP(fwrt->trans->hw_rev));
+ dump->hw_step = cpu_to_le32(fwrt->trans->hw_rev_step);
/*
* Several HWs all have type == 0x42, so we'll override this value
@@ -2107,7 +2110,7 @@ static u32 iwl_dump_ini_info(struct iwl_fw_runtime *fwrt,
*/
hw_type = CSR_HW_REV_TYPE(fwrt->trans->hw_rev);
if (hw_type == IWL_AX210_HW_TYPE) {
- u32 prph_val = iwl_read_prph(fwrt->trans, WFPM_OTP_CFG1_ADDR_GEN2);
+ u32 prph_val = iwl_read_umac_prph(fwrt->trans, WFPM_OTP_CFG1_ADDR);
u32 is_jacket = !!(prph_val & WFPM_OTP_CFG1_IS_JACKET_BIT);
u32 is_cdb = !!(prph_val & WFPM_OTP_CFG1_IS_CDB_BIT);
u32 masked_bits = is_jacket | (is_cdb << 1);
@@ -2291,7 +2294,7 @@ static u32 iwl_dump_ini_trigger(struct iwl_fw_runtime *fwrt,
}
reg = (void *)reg_data.reg_tlv->data;
- reg_type = le32_to_cpu(reg->type);
+ reg_type = reg->type;
if (reg_type >= ARRAY_SIZE(iwl_dump_ini_region_ops))
continue;
@@ -2716,6 +2719,9 @@ static void iwl_fw_dbg_collect_sync(struct iwl_fw_runtime *fwrt, u8 wk_idx)
iwl_fw_dbg_stop_restart_recording(fwrt, ¶ms, false);
+ if (fwrt->trans->dbg.last_tp_resetfw == IWL_FW_INI_RESET_FW_MODE_STOP_FW_ONLY)
+ iwl_force_nmi(fwrt->trans);
+
out:
if (iwl_trans_dbg_ini_valid(fwrt->trans)) {
iwl_fw_error_dump_data_free(dump_data);
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/dump.c b/drivers/net/wireless/intel/iwlwifi/fw/dump.c
index 016b3a4c..b90f1e9 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/dump.c
+++ b/drivers/net/wireless/intel/iwlwifi/fw/dump.c
@@ -12,6 +12,7 @@
#include "iwl-io.h"
#include "iwl-prph.h"
#include "iwl-csr.h"
+#include "pnvm.h"
/*
* Note: This structure is read from the device with IO accesses,
@@ -19,53 +20,6 @@
* read with u32-sized accesses, any members with a different size
* need to be ordered correctly though!
*/
-struct iwl_error_event_table_v1 {
- u32 valid; /* (nonzero) valid, (0) log is empty */
- u32 error_id; /* type of error */
- u32 pc; /* program counter */
- u32 blink1; /* branch link */
- u32 blink2; /* branch link */
- u32 ilink1; /* interrupt link */
- u32 ilink2; /* interrupt link */
- u32 data1; /* error-specific data */
- u32 data2; /* error-specific data */
- u32 data3; /* error-specific data */
- u32 bcon_time; /* beacon timer */
- u32 tsf_low; /* network timestamp function timer */
- u32 tsf_hi; /* network timestamp function timer */
- u32 gp1; /* GP1 timer register */
- u32 gp2; /* GP2 timer register */
- u32 gp3; /* GP3 timer register */
- u32 ucode_ver; /* uCode version */
- u32 hw_ver; /* HW Silicon version */
- u32 brd_ver; /* HW board version */
- u32 log_pc; /* log program counter */
- u32 frame_ptr; /* frame pointer */
- u32 stack_ptr; /* stack pointer */
- u32 hcmd; /* last host command header */
- u32 isr0; /* isr status register LMPM_NIC_ISR0:
- * rxtx_flag */
- u32 isr1; /* isr status register LMPM_NIC_ISR1:
- * host_flag */
- u32 isr2; /* isr status register LMPM_NIC_ISR2:
- * enc_flag */
- u32 isr3; /* isr status register LMPM_NIC_ISR3:
- * time_flag */
- u32 isr4; /* isr status register LMPM_NIC_ISR4:
- * wico interrupt */
- u32 isr_pref; /* isr status register LMPM_NIC_PREF_STAT */
- u32 wait_event; /* wait event() caller address */
- u32 l2p_control; /* L2pControlField */
- u32 l2p_duration; /* L2pDurationField */
- u32 l2p_mhvalid; /* L2pMhValidBits */
- u32 l2p_addr_match; /* L2pAddrMatchStat */
- u32 lmpm_pmg_sel; /* indicate which clocks are turned on
- * (LMPM_PMG_SEL) */
- u32 u_timestamp; /* indicate when the date and time of the
- * compilation */
- u32 flow_handler; /* FH read/write pointers, RX credit */
-} __packed /* LOG_ERROR_TABLE_API_S_VER_1 */;
-
struct iwl_error_event_table {
u32 valid; /* (nonzero) valid, (0) log is empty */
u32 error_id; /* type of error */
@@ -147,6 +101,7 @@ static void iwl_fwrt_dump_umac_error_log(struct iwl_fw_runtime *fwrt)
struct iwl_trans *trans = fwrt->trans;
struct iwl_umac_error_event_table table = {};
u32 base = fwrt->trans->dbg.umac_error_event_table;
+ char pnvm_name[MAX_PNVM_NAME];
if (!base &&
!(fwrt->trans->dbg.error_event_table_tlv_status &
@@ -164,6 +119,13 @@ static void iwl_fwrt_dump_umac_error_log(struct iwl_fw_runtime *fwrt)
fwrt->trans->status, table.valid);
}
+ if ((table.error_id & ~FW_SYSASSERT_CPU_MASK) ==
+ FW_SYSASSERT_PNVM_MISSING) {
+ iwl_pnvm_get_fs_name(trans, pnvm_name, sizeof(pnvm_name));
+ IWL_ERR(fwrt, "PNVM data is missing, please install %s\n",
+ pnvm_name);
+ }
+
IWL_ERR(fwrt, "0x%08X | %s\n", table.error_id,
iwl_fw_lookup_assert_desc(table.error_id));
IWL_ERR(fwrt, "0x%08X | umac branchlink1\n", table.blink1);
@@ -212,7 +174,9 @@ static void iwl_fwrt_dump_lmac_error_log(struct iwl_fw_runtime *fwrt, u8 lmac_nu
IWL_ERR(trans, "HW error, resetting before reading\n");
/* reset the device */
- iwl_trans_sw_reset(trans);
+ err = iwl_trans_sw_reset(trans, true);
+ if (err)
+ return;
err = iwl_finish_nic_init(trans);
if (err)
@@ -295,21 +259,21 @@ struct iwl_tcm_error_event_table {
u32 reserved[4];
} __packed; /* TCM_LOG_ERROR_TABLE_API_S_VER_1 */
-static void iwl_fwrt_dump_tcm_error_log(struct iwl_fw_runtime *fwrt)
+static void iwl_fwrt_dump_tcm_error_log(struct iwl_fw_runtime *fwrt, int idx)
{
struct iwl_trans *trans = fwrt->trans;
struct iwl_tcm_error_event_table table = {};
- u32 base = fwrt->trans->dbg.tcm_error_event_table;
+ u32 base = fwrt->trans->dbg.tcm_error_event_table[idx];
int i;
+ u32 flag = idx ? IWL_ERROR_EVENT_TABLE_TCM2 :
+ IWL_ERROR_EVENT_TABLE_TCM1;
- if (!base ||
- !(fwrt->trans->dbg.error_event_table_tlv_status &
- IWL_ERROR_EVENT_TABLE_TCM))
+ if (!base || !(fwrt->trans->dbg.error_event_table_tlv_status & flag))
return;
iwl_trans_read_mem_bytes(trans, base, &table, sizeof(table));
- IWL_ERR(fwrt, "TCM status:\n");
+ IWL_ERR(fwrt, "TCM%d status:\n", idx + 1);
IWL_ERR(fwrt, "0x%08X | error ID\n", table.error_id);
IWL_ERR(fwrt, "0x%08X | tcm branchlink2\n", table.blink2);
IWL_ERR(fwrt, "0x%08X | tcm interruptlink1\n", table.ilink1);
@@ -328,13 +292,72 @@ static void iwl_fwrt_dump_tcm_error_log(struct iwl_fw_runtime *fwrt)
for (i = 0; i < ARRAY_SIZE(table.sw_status); i++)
IWL_ERR(fwrt, "0x%08X | tcm SW status[%d]\n",
table.sw_status[i], i);
+}
- if (trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_BZ) {
- u32 scratch = iwl_read32(trans, CSR_FUNC_SCRATCH);
+/*
+ * RCM error struct.
+ * Note: This structure is read from the device with IO accesses,
+ * and the reading already does the endian conversion. As it is
+ * read with u32-sized accesses, any members with a different size
+ * need to be ordered correctly though!
+ */
+struct iwl_rcm_error_event_table {
+ u32 valid;
+ u32 error_id;
+ u32 blink2;
+ u32 ilink1;
+ u32 ilink2;
+ u32 data1, data2, data3;
+ u32 logpc;
+ u32 frame_pointer;
+ u32 stack_pointer;
+ u32 msgid;
+ u32 isr;
+ u32 frame_hw_status;
+ u32 mbx_lmac_to_rcm_req;
+ u32 mbx_rcm_to_lmac_req;
+ u32 mh_ctl;
+ u32 mh_addr1_lo;
+ u32 mh_info;
+ u32 mh_err;
+ u32 reserved[3];
+} __packed; /* RCM_LOG_ERROR_TABLE_API_S_VER_1 */
- IWL_ERR(fwrt, "Function Scratch status:\n");
- IWL_ERR(fwrt, "0x%08X | Func Scratch\n", scratch);
- }
+static void iwl_fwrt_dump_rcm_error_log(struct iwl_fw_runtime *fwrt, int idx)
+{
+ struct iwl_trans *trans = fwrt->trans;
+ struct iwl_rcm_error_event_table table = {};
+ u32 base = fwrt->trans->dbg.rcm_error_event_table[idx];
+ u32 flag = idx ? IWL_ERROR_EVENT_TABLE_RCM2 :
+ IWL_ERROR_EVENT_TABLE_RCM1;
+
+ if (!base || !(fwrt->trans->dbg.error_event_table_tlv_status & flag))
+ return;
+
+ iwl_trans_read_mem_bytes(trans, base, &table, sizeof(table));
+
+ IWL_ERR(fwrt, "RCM%d status:\n", idx + 1);
+ IWL_ERR(fwrt, "0x%08X | error ID\n", table.error_id);
+ IWL_ERR(fwrt, "0x%08X | rcm branchlink2\n", table.blink2);
+ IWL_ERR(fwrt, "0x%08X | rcm interruptlink1\n", table.ilink1);
+ IWL_ERR(fwrt, "0x%08X | rcm interruptlink2\n", table.ilink2);
+ IWL_ERR(fwrt, "0x%08X | rcm data1\n", table.data1);
+ IWL_ERR(fwrt, "0x%08X | rcm data2\n", table.data2);
+ IWL_ERR(fwrt, "0x%08X | rcm data3\n", table.data3);
+ IWL_ERR(fwrt, "0x%08X | rcm log PC\n", table.logpc);
+ IWL_ERR(fwrt, "0x%08X | rcm frame pointer\n", table.frame_pointer);
+ IWL_ERR(fwrt, "0x%08X | rcm stack pointer\n", table.stack_pointer);
+ IWL_ERR(fwrt, "0x%08X | rcm msg ID\n", table.msgid);
+ IWL_ERR(fwrt, "0x%08X | rcm ISR status\n", table.isr);
+ IWL_ERR(fwrt, "0x%08X | frame HW status\n", table.frame_hw_status);
+ IWL_ERR(fwrt, "0x%08X | LMAC-to-RCM request mbox\n",
+ table.mbx_lmac_to_rcm_req);
+ IWL_ERR(fwrt, "0x%08X | RCM-to-LMAC request mbox\n",
+ table.mbx_rcm_to_lmac_req);
+ IWL_ERR(fwrt, "0x%08X | MAC header control\n", table.mh_ctl);
+ IWL_ERR(fwrt, "0x%08X | MAC header addr1 low\n", table.mh_addr1_lo);
+ IWL_ERR(fwrt, "0x%08X | MAC header info\n", table.mh_info);
+ IWL_ERR(fwrt, "0x%08X | MAC header error\n", table.mh_err);
}
static void iwl_fwrt_dump_iml_error_log(struct iwl_fw_runtime *fwrt)
@@ -418,8 +441,18 @@ void iwl_fwrt_dump_error_logs(struct iwl_fw_runtime *fwrt)
if (fwrt->trans->dbg.lmac_error_event_table[1])
iwl_fwrt_dump_lmac_error_log(fwrt, 1);
iwl_fwrt_dump_umac_error_log(fwrt);
- iwl_fwrt_dump_tcm_error_log(fwrt);
+ iwl_fwrt_dump_tcm_error_log(fwrt, 0);
+ iwl_fwrt_dump_rcm_error_log(fwrt, 0);
+ iwl_fwrt_dump_tcm_error_log(fwrt, 1);
+ iwl_fwrt_dump_rcm_error_log(fwrt, 1);
iwl_fwrt_dump_iml_error_log(fwrt);
iwl_fwrt_dump_fseq_regs(fwrt);
+
+ if (fwrt->trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_BZ) {
+ u32 scratch = iwl_read32(fwrt->trans, CSR_FUNC_SCRATCH);
+
+ IWL_ERR(fwrt, "Function Scratch status:\n");
+ IWL_ERR(fwrt, "0x%08X | Func Scratch\n", scratch);
+ }
}
IWL_EXPORT_SYMBOL(iwl_fwrt_dump_error_logs);
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/error-dump.h b/drivers/net/wireless/intel/iwlwifi/fw/error-dump.h
index 9036b32..079fa00 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/error-dump.h
+++ b/drivers/net/wireless/intel/iwlwifi/fw/error-dump.h
@@ -232,6 +232,24 @@ struct iwl_fw_error_dump_mem {
#define IWL_INI_DUMP_INFO_TYPE BIT(31)
/**
+ * struct iwl_fw_error_dump_data - data for one type
+ * @type: &enum iwl_fw_ini_region_type
+ * @sub_type: sub type id
+ * @sub_type_ver: sub type version
+ * @reserved: not in use
+ * @len: the length starting from %data
+ * @data: the data itself
+ */
+struct iwl_fw_ini_error_dump_data {
+ u8 type;
+ u8 sub_type;
+ u8 sub_type_ver;
+ u8 reserved;
+ __le32 len;
+ __u8 data[];
+} __packed;
+
+/**
* struct iwl_fw_ini_dump_entry
* @list: list of dump entries
* @size: size of the data
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/file.h b/drivers/net/wireless/intel/iwlwifi/fw/file.h
index 3d572f5..e4ebda6 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/file.h
+++ b/drivers/net/wireless/intel/iwlwifi/fw/file.h
@@ -98,7 +98,6 @@ enum iwl_ucode_tlv_type {
IWL_UCODE_TLV_PNVM_VERSION = 62,
IWL_UCODE_TLV_PNVM_SKU = 64,
- IWL_UCODE_TLV_TCM_DEBUG_ADDRS = 65,
IWL_UCODE_TLV_SEC_TABLE_ADDR = 66,
IWL_UCODE_TLV_D3_KEK_KCK_ADDR = 67,
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/img.c b/drivers/net/wireless/intel/iwlwifi/fw/img.c
index 24a9666..530674a 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/img.c
+++ b/drivers/net/wireless/intel/iwlwifi/fw/img.c
@@ -1,6 +1,6 @@
// SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause
/*
- * Copyright(c) 2019 - 2020 Intel Corporation
+ * Copyright(c) 2019 - 2021 Intel Corporation
*/
#include "img.h"
@@ -49,10 +49,9 @@ u8 iwl_fw_lookup_notif_ver(const struct iwl_fw *fw, u8 grp, u8 cmd, u8 def)
}
EXPORT_SYMBOL_GPL(iwl_fw_lookup_notif_ver);
-#define FW_SYSASSERT_CPU_MASK 0xf0000000
static const struct {
const char *name;
- u8 num;
+ u32 num;
} advanced_lookup[] = {
{ "NMI_INTERRUPT_WDG", 0x34 },
{ "SYSASSERT", 0x35 },
@@ -73,6 +72,7 @@ static const struct {
{ "NMI_INTERRUPT_ACTION_PT", 0x7C },
{ "NMI_INTERRUPT_UNKNOWN", 0x84 },
{ "NMI_INTERRUPT_INST_ACTION_PT", 0x86 },
+ { "PNVM_MISSING", FW_SYSASSERT_PNVM_MISSING },
{ "ADVANCED_SYSASSERT", 0 },
};
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/img.h b/drivers/net/wireless/intel/iwlwifi/fw/img.h
index 993bda1..fa7b178 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/img.h
+++ b/drivers/net/wireless/intel/iwlwifi/fw/img.h
@@ -279,4 +279,8 @@ u8 iwl_fw_lookup_cmd_ver(const struct iwl_fw *fw, u8 grp, u8 cmd, u8 def);
u8 iwl_fw_lookup_notif_ver(const struct iwl_fw *fw, u8 grp, u8 cmd, u8 def);
const char *iwl_fw_lookup_assert_desc(u32 num);
+
+#define FW_SYSASSERT_CPU_MASK 0xf0000000
+#define FW_SYSASSERT_PNVM_MISSING 0x0010070d
+
#endif /* __iwl_fw_img_h__ */
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/init.c b/drivers/net/wireless/intel/iwlwifi/fw/init.c
index 566957a..139ece8 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/init.c
+++ b/drivers/net/wireless/intel/iwlwifi/fw/init.c
@@ -8,7 +8,7 @@
#include "dbg.h"
#include "debugfs.h"
-#include "fw/api/soc.h"
+#include "fw/api/system.h"
#include "fw/api/commands.h"
#include "fw/api/rx.h"
#include "fw/api/datapath.h"
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/runtime.h b/drivers/net/wireless/intel/iwlwifi/fw/runtime.h
index 69799f1..3cb0ddb 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/runtime.h
+++ b/drivers/net/wireless/intel/iwlwifi/fw/runtime.h
@@ -156,8 +156,13 @@ struct iwl_fw_runtime {
u8 sar_chain_b_profile;
struct iwl_geo_profile geo_profiles[ACPI_NUM_GEO_PROFILES_REV3];
u32 geo_rev;
- union iwl_ppag_table_cmd ppag_table;
+ u32 geo_num_profiles;
+ bool geo_enabled;
+ struct iwl_ppag_chain ppag_chains[IWL_NUM_CHAIN_LIMITS];
+ u32 ppag_flags;
u32 ppag_ver;
+ struct iwl_sar_offset_mapping_cmd sgom_table;
+ bool sgom_enabled;
#endif
};
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/uefi.c b/drivers/net/wireless/intel/iwlwifi/fw/uefi.c
index 009dd4b..bd82c24 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/uefi.c
+++ b/drivers/net/wireless/intel/iwlwifi/fw/uefi.c
@@ -11,6 +11,7 @@
#include "fw/uefi.h"
#include "fw/api/alive.h"
#include <linux/efi.h>
+#include "fw/runtime.h"
#define IWL_EFI_VAR_GUID EFI_GUID(0x92daaf2f, 0xc02b, 0x455b, \
0xb2, 0xec, 0xf5, 0xa3, \
@@ -266,3 +267,90 @@ void *iwl_uefi_get_reduced_power(struct iwl_trans *trans, size_t *len)
return data;
}
+
+#ifdef CONFIG_ACPI
+static int iwl_uefi_sgom_parse(struct uefi_cnv_wlan_sgom_data *sgom_data,
+ struct iwl_fw_runtime *fwrt)
+{
+ int i, j;
+
+ if (sgom_data->revision != 1)
+ return -EINVAL;
+
+ memcpy(fwrt->sgom_table.offset_map, sgom_data->offset_map,
+ sizeof(fwrt->sgom_table.offset_map));
+
+ for (i = 0; i < MCC_TO_SAR_OFFSET_TABLE_ROW_SIZE; i++) {
+ for (j = 0; j < MCC_TO_SAR_OFFSET_TABLE_COL_SIZE; j++) {
+ /* since each byte is composed of to values, */
+ /* one for each letter, */
+ /* extract and check each of them separately */
+ u8 value = fwrt->sgom_table.offset_map[i][j];
+ u8 low = value & 0xF;
+ u8 high = (value & 0xF0) >> 4;
+
+ if (high > fwrt->geo_num_profiles)
+ high = 0;
+ if (low > fwrt->geo_num_profiles)
+ low = 0;
+ fwrt->sgom_table.offset_map[i][j] = (high << 4) | low;
+ }
+ }
+
+ fwrt->sgom_enabled = true;
+ return 0;
+}
+
+void iwl_uefi_get_sgom_table(struct iwl_trans *trans,
+ struct iwl_fw_runtime *fwrt)
+{
+ struct efivar_entry *sgom_efivar;
+ struct uefi_cnv_wlan_sgom_data *data;
+ unsigned long package_size;
+ int err, ret;
+
+ if (!fwrt->geo_enabled)
+ return;
+
+ sgom_efivar = kzalloc(sizeof(*sgom_efivar), GFP_KERNEL);
+ if (!sgom_efivar)
+ return;
+
+ memcpy(&sgom_efivar->var.VariableName, IWL_UEFI_SGOM_NAME,
+ sizeof(IWL_UEFI_SGOM_NAME));
+ sgom_efivar->var.VendorGuid = IWL_EFI_VAR_GUID;
+
+ /* TODO: we hardcode a maximum length here, because reading
+ * from the UEFI is not working. To implement this properly,
+ * we have to call efivar_entry_size().
+ */
+ package_size = IWL_HARDCODED_SGOM_SIZE;
+
+ data = kmalloc(package_size, GFP_KERNEL);
+ if (!data) {
+ data = ERR_PTR(-ENOMEM);
+ goto out;
+ }
+
+ err = efivar_entry_get(sgom_efivar, NULL, &package_size, data);
+ if (err) {
+ IWL_DEBUG_FW(trans,
+ "SGOM UEFI variable not found %d\n", err);
+ goto out_free;
+ }
+
+ IWL_DEBUG_FW(trans, "Read SGOM from UEFI with size %lu\n",
+ package_size);
+
+ ret = iwl_uefi_sgom_parse(data, fwrt);
+ if (ret < 0)
+ IWL_DEBUG_FW(trans, "Cannot read SGOM tables. rev is invalid\n");
+
+out_free:
+ kfree(data);
+
+out:
+ kfree(sgom_efivar);
+}
+IWL_EXPORT_SYMBOL(iwl_uefi_get_sgom_table);
+#endif /* CONFIG_ACPI */
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/uefi.h b/drivers/net/wireless/intel/iwlwifi/fw/uefi.h
index d552c656..09d2a97 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/uefi.h
+++ b/drivers/net/wireless/intel/iwlwifi/fw/uefi.h
@@ -7,6 +7,7 @@
#define IWL_UEFI_OEM_PNVM_NAME L"UefiCnvWlanOemSignedPnvm"
#define IWL_UEFI_REDUCED_POWER_NAME L"UefiCnvWlanReducedPower"
+#define IWL_UEFI_SGOM_NAME L"UefiCnvWlanSarGeoOffsetMapping"
/*
* TODO: we have these hardcoded values that the caller must pass,
@@ -16,6 +17,7 @@
*/
#define IWL_HARDCODED_PNVM_SIZE 4096
#define IWL_HARDCODED_REDUCE_POWER_SIZE 32768
+#define IWL_HARDCODED_SGOM_SIZE 339
struct pnvm_sku_package {
u8 rev;
@@ -25,6 +27,16 @@ struct pnvm_sku_package {
u8 data[];
} __packed;
+struct uefi_cnv_wlan_sgom_data {
+ u8 revision;
+ u8 offset_map[IWL_HARDCODED_SGOM_SIZE - 1];
+} __packed;
+
+/*
+ * This is known to be broken on v4.19 and to work on v5.4. Until we
+ * figure out why this is the case and how to make it work, simply
+ * disable the feature in old kernels.
+ */
#ifdef CONFIG_EFI
void *iwl_uefi_get_pnvm(struct iwl_trans *trans, size_t *len);
void *iwl_uefi_get_reduced_power(struct iwl_trans *trans, size_t *len);
@@ -42,4 +54,12 @@ void *iwl_uefi_get_reduced_power(struct iwl_trans *trans, size_t *len)
}
#endif /* CONFIG_EFI */
+#if defined(CONFIG_EFI) && defined(CONFIG_ACPI)
+void iwl_uefi_get_sgom_table(struct iwl_trans *trans, struct iwl_fw_runtime *fwrt);
+#else
+static inline
+void iwl_uefi_get_sgom_table(struct iwl_trans *trans, struct iwl_fw_runtime *fwrt)
+{
+}
+#endif
#endif /* __iwl_fw_uefi__ */
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-config.h b/drivers/net/wireless/intel/iwlwifi/iwl-config.h
index 665167a..e122b8b 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-config.h
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-config.h
@@ -84,6 +84,10 @@ enum iwl_nvm_type {
#define IWL_DEFAULT_MAX_TX_POWER 22
#define IWL_TX_CSUM_NETIF_FLAGS (NETIF_F_IPV6_CSUM | NETIF_F_IP_CSUM |\
NETIF_F_TSO | NETIF_F_TSO6)
+#define IWL_TX_CSUM_NETIF_FLAGS_BZ (NETIF_F_HW_CSUM | NETIF_F_TSO | NETIF_F_TSO6)
+#define IWL_CSUM_NETIF_FLAGS_MASK (IWL_TX_CSUM_NETIF_FLAGS | \
+ IWL_TX_CSUM_NETIF_FLAGS_BZ | \
+ NETIF_F_RXCSUM)
/* Antenna presence definitions */
#define ANT_NONE 0x0
@@ -448,6 +452,9 @@ struct iwl_cfg {
#define IWL_CFG_NO_CDB 0x0
#define IWL_CFG_CDB 0x1
+#define IWL_CFG_NO_JACKET 0x0
+#define IWL_CFG_IS_JACKET 0x1
+
#define IWL_SUBDEVICE_RF_ID(subdevice) ((u16)((subdevice) & 0x00F0) >> 4)
#define IWL_SUBDEVICE_NO_160(subdevice) ((u16)((subdevice) & 0x0200) >> 9)
#define IWL_SUBDEVICE_CORES(subdevice) ((u16)((subdevice) & 0x1C00) >> 10)
@@ -462,6 +469,7 @@ struct iwl_dev_info {
u8 no_160;
u8 cores;
u8 cdb;
+ u8 jacket;
const struct iwl_cfg *cfg;
const char *name;
};
@@ -610,7 +618,6 @@ extern const struct iwl_cfg killer1650x_2ax_cfg;
extern const struct iwl_cfg killer1650w_2ax_cfg;
extern const struct iwl_cfg iwl_qnj_b0_hr_b0_cfg;
extern const struct iwl_cfg iwlax210_2ax_cfg_so_jf_b0;
-extern const struct iwl_cfg iwlax210_2ax_cfg_so_hr_a0;
extern const struct iwl_cfg iwlax211_2ax_cfg_so_gf_a0;
extern const struct iwl_cfg iwlax211_2ax_cfg_so_gf_a0_long;
extern const struct iwl_cfg iwlax210_2ax_cfg_ty_gf_a0;
@@ -634,6 +641,12 @@ extern const struct iwl_cfg iwl_cfg_bz_a0_gf4_a0;
extern const struct iwl_cfg iwl_cfg_bz_a0_mr_a0;
extern const struct iwl_cfg iwl_cfg_bz_a0_fm_a0;
extern const struct iwl_cfg iwl_cfg_gl_a0_fm_a0;
+extern const struct iwl_cfg iwl_cfg_bz_z0_gf_a0;
+extern const struct iwl_cfg iwl_cfg_bnj_a0_fm_a0;
+extern const struct iwl_cfg iwl_cfg_bnj_a0_fm4_a0;
+extern const struct iwl_cfg iwl_cfg_bnj_a0_gf_a0;
+extern const struct iwl_cfg iwl_cfg_bnj_a0_gf4_a0;
+extern const struct iwl_cfg iwl_cfg_bnj_a0_hr_b0;
#endif /* CONFIG_IWLMVM */
#endif /* __IWL_CONFIG_H__ */
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-csr.h b/drivers/net/wireless/intel/iwlwifi/iwl-csr.h
index ff79a2e..f90d466 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-csr.h
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-csr.h
@@ -105,9 +105,14 @@
/* GIO Chicken Bits (PCI Express bus link power management) */
#define CSR_GIO_CHICKEN_BITS (CSR_BASE+0x100)
-/* Doorbell NMI (since Bz) */
+#define CSR_IPC_SLEEP_CONTROL (CSR_BASE + 0x114)
+#define CSR_IPC_SLEEP_CONTROL_SUSPEND 0x3
+#define CSR_IPC_SLEEP_CONTROL_RESUME 0
+
+/* Doorbell - since Bz
+ * connected to UREG_DOORBELL_TO_ISR6 (lower 16 bits only)
+ */
#define CSR_DOORBELL_VECTOR (CSR_BASE + 0x130)
-#define CSR_DOORBELL_VECTOR_NMI BIT(1)
/* host chicken bits */
#define CSR_HOST_CHICKEN (CSR_BASE + 0x204)
@@ -143,8 +148,7 @@
#define CSR_FUNC_SCRATCH_INIT_VALUE (0x01010101)
/* Bits for CSR_HW_IF_CONFIG_REG */
-#define CSR_HW_IF_CONFIG_REG_MSK_MAC_DASH (0x00000003)
-#define CSR_HW_IF_CONFIG_REG_MSK_MAC_STEP (0x0000000C)
+#define CSR_HW_IF_CONFIG_REG_MSK_MAC_STEP_DASH (0x0000000F)
#define CSR_HW_IF_CONFIG_REG_BIT_MONITOR_SRAM (0x00000080)
#define CSR_HW_IF_CONFIG_REG_MSK_BOARD_VER (0x000000C0)
#define CSR_HW_IF_CONFIG_REG_BIT_MAC_SI (0x00000100)
@@ -287,8 +291,7 @@
#define CSR_GP_CNTRL_REG_FLAG_SW_RESET BIT(31)
/* HW REV */
-#define CSR_HW_REV_DASH(_val) (((_val) & 0x0000003) >> 0)
-#define CSR_HW_REV_STEP(_val) (((_val) & 0x000000C) >> 2)
+#define CSR_HW_REV_STEP_DASH(_val) ((_val) & CSR_HW_IF_CONFIG_REG_MSK_MAC_STEP_DASH)
#define CSR_HW_REV_TYPE(_val) (((_val) & 0x000FFF0) >> 4)
/* HW RFID */
@@ -306,6 +309,7 @@ enum {
SILICON_A_STEP = 0,
SILICON_B_STEP,
SILICON_C_STEP,
+ SILICON_Z_STEP = 0xf,
};
@@ -328,10 +332,10 @@ enum {
#define CSR_HW_REV_TYPE_7265D (0x0000210)
#define CSR_HW_REV_TYPE_NONE (0x00001F0)
#define CSR_HW_REV_TYPE_QNJ (0x0000360)
-#define CSR_HW_REV_TYPE_QNJ_B0 (0x0000364)
-#define CSR_HW_REV_TYPE_QU_B0 (0x0000334)
-#define CSR_HW_REV_TYPE_QU_C0 (0x0000338)
-#define CSR_HW_REV_TYPE_QUZ (0x0000354)
+#define CSR_HW_REV_TYPE_QNJ_B0 (0x0000361)
+#define CSR_HW_REV_TYPE_QU_B0 (0x0000331)
+#define CSR_HW_REV_TYPE_QU_C0 (0x0000332)
+#define CSR_HW_REV_TYPE_QUZ (0x0000351)
#define CSR_HW_REV_TYPE_HR_CDB (0x0000340)
#define CSR_HW_REV_TYPE_SO (0x0000370)
#define CSR_HW_REV_TYPE_TY (0x0000420)
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-dbg-tlv.c b/drivers/net/wireless/intel/iwlwifi/iwl-dbg-tlv.c
index 7ab98b4..c73672d 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-dbg-tlv.c
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-dbg-tlv.c
@@ -59,7 +59,7 @@ dbg_ver_table[IWL_DBG_TLV_TYPE_NUM] = {
[IWL_DBG_TLV_TYPE_DEBUG_INFO] = {.min_ver = 1, .max_ver = 1,},
[IWL_DBG_TLV_TYPE_BUF_ALLOC] = {.min_ver = 1, .max_ver = 1,},
[IWL_DBG_TLV_TYPE_HCMD] = {.min_ver = 1, .max_ver = 1,},
- [IWL_DBG_TLV_TYPE_REGION] = {.min_ver = 1, .max_ver = 2,},
+ [IWL_DBG_TLV_TYPE_REGION] = {.min_ver = 1, .max_ver = 3,},
[IWL_DBG_TLV_TYPE_TRIGGER] = {.min_ver = 1, .max_ver = 1,},
[IWL_DBG_TLV_TYPE_CONF_SET] = {.min_ver = 1, .max_ver = 1,},
};
@@ -177,14 +177,14 @@ static int iwl_dbg_tlv_alloc_region(struct iwl_trans *trans,
const struct iwl_fw_ini_region_tlv *reg = (const void *)tlv->data;
struct iwl_ucode_tlv **active_reg;
u32 id = le32_to_cpu(reg->id);
- u32 type = le32_to_cpu(reg->type);
+ u8 type = reg->type;
u32 tlv_len = sizeof(*tlv) + le32_to_cpu(tlv->length);
/*
- * The higher part of the ID in version 2 is irrelevant for
+ * The higher part of the ID in from version 2 is irrelevant for
* us, so mask it out.
*/
- if (le32_to_cpu(reg->hdr.version) == 2)
+ if (le32_to_cpu(reg->hdr.version) >= 2)
id &= IWL_FW_INI_REGION_V2_MASK;
if (le32_to_cpu(tlv->length) < sizeof(*reg))
@@ -233,6 +233,7 @@ static int iwl_dbg_tlv_alloc_trigger(struct iwl_trans *trans,
const struct iwl_fw_ini_trigger_tlv *trig = (const void *)tlv->data;
struct iwl_fw_ini_trigger_tlv *dup_trig;
u32 tp = le32_to_cpu(trig->time_point);
+ u32 rf = le32_to_cpu(trig->reset_fw);
struct iwl_ucode_tlv *dup = NULL;
int ret;
@@ -247,6 +248,10 @@ static int iwl_dbg_tlv_alloc_trigger(struct iwl_trans *trans,
return -EINVAL;
}
+ IWL_DEBUG_FW(trans,
+ "WRT: time point %u for trigger TLV with reset_fw %u\n",
+ tp, rf);
+ trans->dbg.last_tp_resetfw = 0xFF;
if (!le32_to_cpu(trig->occurrences)) {
dup = kmemdup(tlv, sizeof(*tlv) + le32_to_cpu(tlv->length),
GFP_KERNEL);
@@ -300,14 +305,21 @@ static int (*dbg_tlv_alloc[])(struct iwl_trans *trans,
void iwl_dbg_tlv_alloc(struct iwl_trans *trans, const struct iwl_ucode_tlv *tlv,
bool ext)
{
- const struct iwl_fw_ini_header *hdr = (const void *)&tlv->data[0];
- u32 type = le32_to_cpu(tlv->type);
- u32 tlv_idx = type - IWL_UCODE_TLV_DEBUG_BASE;
- u32 domain = le32_to_cpu(hdr->domain);
enum iwl_ini_cfg_state *cfg_state = ext ?
&trans->dbg.external_ini_cfg : &trans->dbg.internal_ini_cfg;
+ const struct iwl_fw_ini_header *hdr = (const void *)&tlv->data[0];
+ u32 type;
+ u32 tlv_idx;
+ u32 domain;
int ret;
+ if (le32_to_cpu(tlv->length) < sizeof(*hdr))
+ return;
+
+ type = le32_to_cpu(tlv->type);
+ tlv_idx = type - IWL_UCODE_TLV_DEBUG_BASE;
+ domain = le32_to_cpu(hdr->domain);
+
if (domain != IWL_FW_INI_DOMAIN_ALWAYS_ON &&
!(domain & trans->dbg.domains_bitmap)) {
IWL_DEBUG_FW(trans,
@@ -473,7 +485,7 @@ void iwl_dbg_tlv_load_bin(struct device *dev, struct iwl_trans *trans)
int res;
if (!iwlwifi_mod_params.enable_ini ||
- trans->trans_cfg->device_family <= IWL_DEVICE_FAMILY_9000)
+ trans->trans_cfg->device_family <= IWL_DEVICE_FAMILY_8000)
return;
res = firmware_request_nowarn(&fw, yoyo_bin, dev);
@@ -1159,6 +1171,8 @@ iwl_dbg_tlv_tp_trigger(struct iwl_fw_runtime *fwrt, bool sync,
u32 num_data = iwl_tlv_array_len(&node->tlv, dump_data.trig,
data);
int ret, i;
+ u32 tp = le32_to_cpu(dump_data.trig->time_point);
+
if (!num_data) {
ret = iwl_fw_dbg_ini_collect(fwrt, &dump_data, sync);
@@ -1177,8 +1191,42 @@ iwl_dbg_tlv_tp_trigger(struct iwl_fw_runtime *fwrt, bool sync,
break;
}
}
- }
+ fwrt->trans->dbg.restart_required = FALSE;
+ IWL_DEBUG_INFO(fwrt, "WRT: tp %d, reset_fw %d\n",
+ tp, dump_data.trig->reset_fw);
+ IWL_DEBUG_INFO(fwrt, "WRT: restart_required %d, last_tp_resetfw %d\n",
+ fwrt->trans->dbg.restart_required,
+ fwrt->trans->dbg.last_tp_resetfw);
+
+ if (fwrt->trans->trans_cfg->device_family ==
+ IWL_DEVICE_FAMILY_9000) {
+ fwrt->trans->dbg.restart_required = TRUE;
+ } else if (tp == IWL_FW_INI_TIME_POINT_FW_ASSERT &&
+ fwrt->trans->dbg.last_tp_resetfw ==
+ IWL_FW_INI_RESET_FW_MODE_STOP_FW_ONLY) {
+ fwrt->trans->dbg.restart_required = FALSE;
+ fwrt->trans->dbg.last_tp_resetfw = 0xFF;
+ IWL_DEBUG_FW(fwrt, "WRT: FW_ASSERT due to reset_fw_mode-no restart\n");
+ } else if (le32_to_cpu(dump_data.trig->reset_fw) ==
+ IWL_FW_INI_RESET_FW_MODE_STOP_AND_RELOAD_FW) {
+ IWL_DEBUG_INFO(fwrt, "WRT: stop and reload firmware\n");
+ fwrt->trans->dbg.restart_required = TRUE;
+ } else if (le32_to_cpu(dump_data.trig->reset_fw) ==
+ IWL_FW_INI_RESET_FW_MODE_STOP_FW_ONLY) {
+ IWL_DEBUG_INFO(fwrt, "WRT: stop only and no reload firmware\n");
+ fwrt->trans->dbg.restart_required = FALSE;
+ fwrt->trans->dbg.last_tp_resetfw =
+ le32_to_cpu(dump_data.trig->reset_fw);
+ } else if (le32_to_cpu(dump_data.trig->reset_fw) ==
+ IWL_FW_INI_RESET_FW_MODE_NOTHING) {
+ IWL_DEBUG_INFO(fwrt,
+ "WRT: nothing need to be done after debug collection\n");
+ } else {
+ IWL_ERR(fwrt, "WRT: wrong resetfw %d\n",
+ le32_to_cpu(dump_data.trig->reset_fw));
+ }
+ }
return 0;
}
@@ -1244,7 +1292,7 @@ static void iwl_dbg_tlv_init_cfg(struct iwl_fw_runtime *fwrt)
}
reg = (void *)(*active_reg)->data;
- reg_type = le32_to_cpu(reg->type);
+ reg_type = reg->type;
if (reg_type != IWL_FW_INI_REGION_DRAM_BUFFER ||
!(BIT(le32_to_cpu(reg->dram_alloc_id)) & failed_alloc))
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-drv.c b/drivers/net/wireless/intel/iwlwifi/iwl-drv.c
index 5cec467..83e3b73 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-drv.c
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-drv.c
@@ -130,6 +130,9 @@ static void iwl_dealloc_ucode(struct iwl_drv *drv)
for (i = 0; i < IWL_UCODE_TYPE_MAX; i++)
iwl_free_fw_img(drv, drv->fw.img + i);
+
+ /* clear the data for the aborted load case */
+ memset(&drv->fw, 0, sizeof(drv->fw));
}
static int iwl_alloc_fw_desc(struct iwl_drv *drv, struct fw_desc *desc,
@@ -163,8 +166,8 @@ static int iwl_request_firmware(struct iwl_drv *drv, bool first)
char tag[8];
if (drv->trans->trans_cfg->device_family == IWL_DEVICE_FAMILY_9000 &&
- (CSR_HW_REV_STEP(drv->trans->hw_rev) != SILICON_B_STEP &&
- CSR_HW_REV_STEP(drv->trans->hw_rev) != SILICON_C_STEP)) {
+ (drv->trans->hw_rev_step != SILICON_B_STEP &&
+ drv->trans->hw_rev_step != SILICON_C_STEP)) {
IWL_ERR(drv,
"Only HW steps B and C are currently supported (0x%0x)\n",
drv->trans->hw_rev);
@@ -586,6 +589,66 @@ static void iwl_drv_set_dump_exclude(struct iwl_drv *drv,
excl->size = le32_to_cpu(fw->size);
}
+static void iwl_parse_dbg_tlv_assert_tables(struct iwl_drv *drv,
+ const struct iwl_ucode_tlv *tlv)
+{
+ const struct iwl_fw_ini_region_tlv *region;
+ u32 length = le32_to_cpu(tlv->length);
+ u32 addr;
+
+ if (length < offsetof(typeof(*region), special_mem) +
+ sizeof(region->special_mem))
+ return;
+
+ region = (void *)tlv->data;
+ addr = le32_to_cpu(region->special_mem.base_addr);
+ addr += le32_to_cpu(region->special_mem.offset);
+ addr &= ~FW_ADDR_CACHE_CONTROL;
+
+ if (region->type != IWL_FW_INI_REGION_SPECIAL_DEVICE_MEMORY)
+ return;
+
+ switch (region->sub_type) {
+ case IWL_FW_INI_REGION_DEVICE_MEMORY_SUBTYPE_UMAC_ERROR_TABLE:
+ drv->trans->dbg.umac_error_event_table = addr;
+ drv->trans->dbg.error_event_table_tlv_status |=
+ IWL_ERROR_EVENT_TABLE_UMAC;
+ break;
+ case IWL_FW_INI_REGION_DEVICE_MEMORY_SUBTYPE_LMAC_1_ERROR_TABLE:
+ drv->trans->dbg.lmac_error_event_table[0] = addr;
+ drv->trans->dbg.error_event_table_tlv_status |=
+ IWL_ERROR_EVENT_TABLE_LMAC1;
+ break;
+ case IWL_FW_INI_REGION_DEVICE_MEMORY_SUBTYPE_LMAC_2_ERROR_TABLE:
+ drv->trans->dbg.lmac_error_event_table[1] = addr;
+ drv->trans->dbg.error_event_table_tlv_status |=
+ IWL_ERROR_EVENT_TABLE_LMAC2;
+ break;
+ case IWL_FW_INI_REGION_DEVICE_MEMORY_SUBTYPE_TCM_1_ERROR_TABLE:
+ drv->trans->dbg.tcm_error_event_table[0] = addr;
+ drv->trans->dbg.error_event_table_tlv_status |=
+ IWL_ERROR_EVENT_TABLE_TCM1;
+ break;
+ case IWL_FW_INI_REGION_DEVICE_MEMORY_SUBTYPE_TCM_2_ERROR_TABLE:
+ drv->trans->dbg.tcm_error_event_table[1] = addr;
+ drv->trans->dbg.error_event_table_tlv_status |=
+ IWL_ERROR_EVENT_TABLE_TCM2;
+ break;
+ case IWL_FW_INI_REGION_DEVICE_MEMORY_SUBTYPE_RCM_1_ERROR_TABLE:
+ drv->trans->dbg.rcm_error_event_table[0] = addr;
+ drv->trans->dbg.error_event_table_tlv_status |=
+ IWL_ERROR_EVENT_TABLE_RCM1;
+ break;
+ case IWL_FW_INI_REGION_DEVICE_MEMORY_SUBTYPE_RCM_2_ERROR_TABLE:
+ drv->trans->dbg.rcm_error_event_table[1] = addr;
+ drv->trans->dbg.error_event_table_tlv_status |=
+ IWL_ERROR_EVENT_TABLE_RCM2;
+ break;
+ default:
+ break;
+ }
+}
+
static int iwl_parse_tlv_firmware(struct iwl_drv *drv,
const struct firmware *ucode_raw,
struct iwl_firmware_pieces *pieces,
@@ -1153,21 +1216,12 @@ static int iwl_parse_tlv_firmware(struct iwl_drv *drv,
IWL_ERROR_EVENT_TABLE_LMAC1;
break;
}
- case IWL_UCODE_TLV_TCM_DEBUG_ADDRS: {
- struct iwl_fw_tcm_error_addr *ptr = (void *)tlv_data;
-
- if (tlv_len != sizeof(*ptr))
- goto invalid_tlv_len;
- drv->trans->dbg.tcm_error_event_table =
- le32_to_cpu(ptr->addr) & ~FW_ADDR_CACHE_CONTROL;
- drv->trans->dbg.error_event_table_tlv_status |=
- IWL_ERROR_EVENT_TABLE_TCM;
- break;
- }
+ case IWL_UCODE_TLV_TYPE_REGIONS:
+ iwl_parse_dbg_tlv_assert_tables(drv, tlv);
+ fallthrough;
case IWL_UCODE_TLV_TYPE_DEBUG_INFO:
case IWL_UCODE_TLV_TYPE_BUFFER_ALLOCATION:
case IWL_UCODE_TLV_TYPE_HCMD:
- case IWL_UCODE_TLV_TYPE_REGIONS:
case IWL_UCODE_TLV_TYPE_TRIGGERS:
case IWL_UCODE_TLV_TYPE_CONF_SET:
if (iwlwifi_mod_params.enable_ini)
@@ -1375,6 +1429,7 @@ static void iwl_req_fw_callback(const struct firmware *ucode_raw, void *context)
int i;
bool load_module = false;
bool usniffer_images = false;
+ bool failure = true;
fw->ucode_capa.max_probe_length = IWL_DEFAULT_MAX_PROBE_LENGTH;
fw->ucode_capa.standard_phy_calibration_size =
@@ -1635,15 +1690,9 @@ static void iwl_req_fw_callback(const struct firmware *ucode_raw, void *context)
* else from proceeding if the module fails to load
* or hangs loading.
*/
- if (load_module) {
+ if (load_module)
request_module("%s", op->name);
-#ifdef CONFIG_IWLWIFI_OPMODE_MODULAR
- if (err)
- IWL_ERR(drv,
- "failed to load module %s (error %d), is dynamic loading enabled?\n",
- op->name, err);
-#endif
- }
+ failure = false;
goto free;
try_again:
@@ -1659,6 +1708,9 @@ static void iwl_req_fw_callback(const struct firmware *ucode_raw, void *context)
complete(&drv->request_firmware_complete);
device_release_driver(drv->trans->dev);
free:
+ if (failure)
+ iwl_dealloc_ucode(drv);
+
if (pieces) {
for (i = 0; i < ARRAY_SIZE(pieces->img); i++)
kfree(pieces->img[i].sec);
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-eeprom-parse.c b/drivers/net/wireless/intel/iwlwifi/iwl-eeprom-parse.c
index f12b865..d7a7835 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-eeprom-parse.c
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-eeprom-parse.c
@@ -10,6 +10,7 @@
#include "iwl-modparams.h"
#include "iwl-eeprom-parse.h"
+#if IS_ENABLED(CONFIG_IWLDVM)
/* EEPROM offset definitions */
/* indirect access definitions */
@@ -647,6 +648,7 @@ static int iwl_init_channel_map(struct device *dev, const struct iwl_cfg *cfg,
return n_channels;
}
+#endif
int iwl_init_sband_channels(struct iwl_nvm_data *data,
struct ieee80211_supported_band *sband,
@@ -750,6 +752,7 @@ void iwl_init_ht_hw_capab(struct iwl_trans *trans,
}
}
+#if IS_ENABLED(CONFIG_IWLDVM)
static void iwl_init_sbands(struct iwl_trans *trans, const struct iwl_cfg *cfg,
struct iwl_nvm_data *data,
const u8 *eeprom, size_t eeprom_size)
@@ -873,3 +876,4 @@ iwl_parse_eeprom_data(struct iwl_trans *trans, const struct iwl_cfg *cfg,
return NULL;
}
IWL_EXPORT_SYMBOL(iwl_parse_eeprom_data);
+#endif
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-fh.h b/drivers/net/wireless/intel/iwlwifi/iwl-fh.h
index aaa3b65..e6fd494 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-fh.h
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-fh.h
@@ -580,7 +580,7 @@ struct iwl_rb_status {
__le16 closed_fr_num;
__le16 finished_rb_num;
__le16 finished_fr_nam;
- __le32 __unused;
+ __le32 __spare;
} __packed;
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-io.c b/drivers/net/wireless/intel/iwlwifi/iwl-io.c
index 46917b4..253eac4 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-io.c
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-io.c
@@ -218,7 +218,7 @@ void iwl_force_nmi(struct iwl_trans *trans)
UREG_DOORBELL_TO_ISR6_NMI_BIT);
else
iwl_write32(trans, CSR_DOORBELL_VECTOR,
- CSR_DOORBELL_VECTOR_NMI);
+ UREG_DOORBELL_TO_ISR6_NMI_BIT);
}
IWL_EXPORT_SYMBOL(iwl_force_nmi);
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-nvm-parse.c b/drivers/net/wireless/intel/iwlwifi/iwl-nvm-parse.c
index f470f9a..dd58c8f 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-nvm-parse.c
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-nvm-parse.c
@@ -22,6 +22,7 @@
#include "fw/api/commands.h"
#include "fw/api/cmdhdr.h"
#include "fw/img.h"
+#include "mei/iwl-mei.h"
/* NVM offsets (in words) definitions */
enum nvm_offsets {
@@ -607,7 +608,8 @@ static const struct ieee80211_sband_iftype_data iwl_he_capa[] = {
.phy_cap_info[9] =
IEEE80211_HE_PHY_CAP9_RX_FULL_BW_SU_USING_MU_WITH_COMP_SIGB |
IEEE80211_HE_PHY_CAP9_RX_FULL_BW_SU_USING_MU_WITH_NON_COMP_SIGB |
- IEEE80211_HE_PHY_CAP9_NOMIMAL_PKT_PADDING_RESERVED,
+ (IEEE80211_HE_PHY_CAP9_NOMINAL_PKT_PADDING_RESERVED <<
+ IEEE80211_HE_PHY_CAP9_NOMINAL_PKT_PADDING_POS),
.phy_cap_info[10] =
IEEE80211_HE_PHY_CAP10_HE_MU_M1RU_MAX_LTF,
},
@@ -664,7 +666,8 @@ static const struct ieee80211_sband_iftype_data iwl_he_capa[] = {
IEEE80211_HE_PHY_CAP8_HE_ER_SU_PPDU_4XLTF_AND_08_US_GI |
IEEE80211_HE_PHY_CAP8_DCM_MAX_RU_242,
.phy_cap_info[9] =
- IEEE80211_HE_PHY_CAP9_NOMIMAL_PKT_PADDING_RESERVED,
+ IEEE80211_HE_PHY_CAP9_NOMINAL_PKT_PADDING_RESERVED
+ << IEEE80211_HE_PHY_CAP9_NOMINAL_PKT_PADDING_POS,
},
/*
* Set default Tx/Rx HE MCS NSS Support field.
@@ -1115,6 +1118,66 @@ iwl_nvm_no_wide_in_5ghz(struct iwl_trans *trans, const struct iwl_cfg *cfg,
}
struct iwl_nvm_data *
+iwl_parse_mei_nvm_data(struct iwl_trans *trans, const struct iwl_cfg *cfg,
+ const struct iwl_mei_nvm *mei_nvm,
+ const struct iwl_fw *fw)
+{
+ struct iwl_nvm_data *data;
+ u32 sbands_flags = 0;
+ u8 rx_chains = fw->valid_rx_ant;
+ u8 tx_chains = fw->valid_rx_ant;
+
+ if (cfg->uhb_supported)
+ data = kzalloc(struct_size(data, channels,
+ IWL_NVM_NUM_CHANNELS_UHB),
+ GFP_KERNEL);
+ else
+ data = kzalloc(struct_size(data, channels,
+ IWL_NVM_NUM_CHANNELS_EXT),
+ GFP_KERNEL);
+ if (!data)
+ return NULL;
+
+ BUILD_BUG_ON(ARRAY_SIZE(mei_nvm->channels) !=
+ IWL_NVM_NUM_CHANNELS_UHB);
+ data->nvm_version = mei_nvm->nvm_version;
+
+ iwl_set_radio_cfg(cfg, data, mei_nvm->radio_cfg);
+ if (data->valid_tx_ant)
+ tx_chains &= data->valid_tx_ant;
+ if (data->valid_rx_ant)
+ rx_chains &= data->valid_rx_ant;
+
+ data->sku_cap_mimo_disabled = false;
+ data->sku_cap_band_24ghz_enable = true;
+ data->sku_cap_band_52ghz_enable = true;
+ data->sku_cap_11n_enable =
+ !(iwlwifi_mod_params.disable_11n & IWL_DISABLE_HT_ALL);
+ data->sku_cap_11ac_enable = true;
+ data->sku_cap_11ax_enable =
+ mei_nvm->caps & MEI_NVM_CAPS_11AX_SUPPORT;
+
+ data->lar_enabled = mei_nvm->caps & MEI_NVM_CAPS_LARI_SUPPORT;
+
+ data->n_hw_addrs = mei_nvm->n_hw_addrs;
+ /* If no valid mac address was found - bail out */
+ if (iwl_set_hw_address(trans, cfg, data, NULL, NULL)) {
+ kfree(data);
+ return NULL;
+ }
+
+ if (data->lar_enabled &&
+ fw_has_capa(&fw->ucode_capa, IWL_UCODE_TLV_CAPA_LAR_SUPPORT))
+ sbands_flags |= IWL_NVM_SBANDS_FLAGS_LAR;
+
+ iwl_init_sbands(trans, data, mei_nvm->channels, tx_chains, rx_chains,
+ sbands_flags, true, fw);
+
+ return data;
+}
+IWL_EXPORT_SYMBOL(iwl_parse_mei_nvm_data);
+
+struct iwl_nvm_data *
iwl_parse_nvm_data(struct iwl_trans *trans, const struct iwl_cfg *cfg,
const struct iwl_fw *fw,
const __be16 *nvm_hw, const __le16 *nvm_sw,
@@ -1548,7 +1611,7 @@ int iwl_read_external_nvm(struct iwl_trans *trans,
/* nvm file validation, dword_buff[2] holds the file version */
if (trans->trans_cfg->device_family == IWL_DEVICE_FAMILY_8000 &&
- CSR_HW_REV_STEP(trans->hw_rev) == SILICON_C_STEP &&
+ trans->hw_rev_step == SILICON_C_STEP &&
le32_to_cpu(dword_buff[2]) < 0xE4A) {
ret = -EFAULT;
goto out;
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-nvm-parse.h b/drivers/net/wireless/intel/iwlwifi/iwl-nvm-parse.h
index e1f5a97..e01f775 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-nvm-parse.h
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-nvm-parse.h
@@ -1,6 +1,6 @@
/* SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause */
/*
- * Copyright (C) 2005-2015, 2018-2020 Intel Corporation
+ * Copyright (C) 2005-2015, 2018-2021 Intel Corporation
* Copyright (C) 2016-2017 Intel Deutschland GmbH
*/
#ifndef __iwl_nvm_parse_h__
@@ -8,6 +8,7 @@
#include <net/cfg80211.h>
#include "iwl-eeprom-parse.h"
+#include "mei/iwl-mei.h"
/**
* enum iwl_nvm_sbands_flags - modification flags for the channel profiles
@@ -81,4 +82,12 @@ void iwl_nvm_fixups(u32 hw_id, unsigned int section, u8 *data,
struct iwl_nvm_data *iwl_get_nvm(struct iwl_trans *trans,
const struct iwl_fw *fw);
+/**
+ * iwl_parse_mei_nvm_data - parse the mei_nvm_data and get an iwl_nvm_data
+ */
+struct iwl_nvm_data *
+iwl_parse_mei_nvm_data(struct iwl_trans *trans, const struct iwl_cfg *cfg,
+ const struct iwl_mei_nvm *mei_nvm,
+ const struct iwl_fw *fw);
+
#endif /* __iwl_nvm_parse_h__ */
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-prph.h b/drivers/net/wireless/intel/iwlwifi/iwl-prph.h
index a84ab02..95b3dae 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-prph.h
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-prph.h
@@ -347,9 +347,7 @@
#define RADIO_REG_SYS_MANUAL_DFT_0 0xAD4078
#define RFIC_REG_RD 0xAD0470
#define WFPM_CTRL_REG 0xA03030
-#define WFPM_CTRL_REG_GEN2 0xd03030
#define WFPM_OTP_CFG1_ADDR 0x00a03098
-#define WFPM_OTP_CFG1_ADDR_GEN2 0x00d03098
#define WFPM_OTP_CFG1_IS_JACKET_BIT BIT(4)
#define WFPM_OTP_CFG1_IS_CDB_BIT BIT(5)
@@ -455,6 +453,13 @@ enum {
#define UREG_DOORBELL_TO_ISR6_RESUME BIT(19)
#define UREG_DOORBELL_TO_ISR6_PNVM BIT(20)
+/*
+ * From BZ family driver triggers this bit for suspend and resume
+ * The driver should update CSR_IPC_SLEEP_CONTROL before triggering
+ * this interrupt with suspend/resume value
+ */
+#define UREG_DOORBELL_TO_ISR6_SLEEP_CTRL BIT(31)
+
#define CNVI_MBOX_C 0xA3400C
#define FSEQ_ERROR_CODE 0xA340C8
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-trans.h b/drivers/net/wireless/intel/iwlwifi/iwl-trans.h
index 4ebb187..1bcaa35 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-trans.h
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-trans.h
@@ -193,7 +193,10 @@ enum iwl_error_event_table_status {
IWL_ERROR_EVENT_TABLE_LMAC1 = BIT(0),
IWL_ERROR_EVENT_TABLE_LMAC2 = BIT(1),
IWL_ERROR_EVENT_TABLE_UMAC = BIT(2),
- IWL_ERROR_EVENT_TABLE_TCM = BIT(3),
+ IWL_ERROR_EVENT_TABLE_TCM1 = BIT(3),
+ IWL_ERROR_EVENT_TABLE_TCM2 = BIT(4),
+ IWL_ERROR_EVENT_TABLE_RCM1 = BIT(5),
+ IWL_ERROR_EVENT_TABLE_RCM2 = BIT(6),
};
/**
@@ -296,6 +299,8 @@ enum iwl_d3_status {
* are sent
* @STATUS_TRANS_IDLE: the trans is idle - general commands are not to be sent
* @STATUS_TRANS_DEAD: trans is dead - avoid any read/write operation
+ * @STATUS_SUPPRESS_CMD_ERROR_ONCE: suppress "FW error in SYNC CMD" once,
+ * e.g. for testing
*/
enum iwl_trans_status {
STATUS_SYNC_HCMD_ACTIVE,
@@ -308,6 +313,7 @@ enum iwl_trans_status {
STATUS_TRANS_GOING_IDLE,
STATUS_TRANS_IDLE,
STATUS_TRANS_DEAD,
+ STATUS_SUPPRESS_CMD_ERROR_ONCE,
};
static inline int
@@ -593,7 +599,7 @@ struct iwl_trans_ops {
void (*configure)(struct iwl_trans *trans,
const struct iwl_trans_config *trans_cfg);
void (*set_pmi)(struct iwl_trans *trans, bool state);
- void (*sw_reset)(struct iwl_trans *trans);
+ int (*sw_reset)(struct iwl_trans *trans, bool retake_ownership);
bool (*grab_nic_access)(struct iwl_trans *trans);
void (*release_nic_access)(struct iwl_trans *trans);
void (*set_bits_mask)(struct iwl_trans *trans, u32 reg, u32 mask,
@@ -725,7 +731,8 @@ struct iwl_self_init_dram {
* @trigger_tlv: array of pointers to triggers TLVs for debug
* @lmac_error_event_table: addrs of lmacs error tables
* @umac_error_event_table: addr of umac error table
- * @tcm_error_event_table: address of TCM error table
+ * @tcm_error_event_table: address(es) of TCM error table(s)
+ * @rcm_error_event_table: address(es) of RCM error table(s)
* @error_event_table_tlv_status: bitmap that indicates what error table
* pointers was recevied via TLV. uses enum &iwl_error_event_table_status
* @internal_ini_cfg: internal debug cfg state. Uses &enum iwl_ini_cfg_state
@@ -752,7 +759,8 @@ struct iwl_trans_debug {
u32 lmac_error_event_table[2];
u32 umac_error_event_table;
- u32 tcm_error_event_table;
+ u32 tcm_error_event_table[2];
+ u32 rcm_error_event_table[2];
unsigned int error_event_table_tlv_status;
enum iwl_ini_cfg_state internal_ini_cfg;
@@ -775,6 +783,8 @@ struct iwl_trans_debug {
u32 domains_bitmap;
u32 ucode_preset;
+ bool restart_required;
+ u32 last_tp_resetfw;
};
struct iwl_dma_ptr {
@@ -924,6 +934,7 @@ struct iwl_trans_txqs {
/**
* struct iwl_trans - transport common data
*
+ * @csme_own - true if we couldn't get ownership on the device
* @ops - pointer to iwl_trans_ops
* @op_mode - pointer to the op_mode
* @trans_cfg: the trans-specific configuration part
@@ -937,6 +948,7 @@ struct iwl_trans_txqs {
* @hw_id: a u32 with the ID of the device / sub-device.
* Set during transport allocation.
* @hw_id_str: a string with info about HW ID. Set during transport allocation.
+ * @hw_rev_step: The mac step of the HW
* @pm_support: set to true in start_hw if link pm is supported
* @ltr_enabled: set to true if the LTR is enabled
* @wide_cmd_header: true when ucode supports wide command header format
@@ -958,6 +970,7 @@ struct iwl_trans_txqs {
* @iwl_trans_txqs: transport tx queues data.
*/
struct iwl_trans {
+ bool csme_own;
const struct iwl_trans_ops *ops;
struct iwl_op_mode *op_mode;
const struct iwl_cfg_trans_params *trans_cfg;
@@ -969,6 +982,7 @@ struct iwl_trans {
struct device *dev;
u32 max_skb_frags;
u32 hw_rev;
+ u32 hw_rev_step;
u32 hw_rf_id;
u32 hw_id;
char hw_id_str[52];
@@ -1382,10 +1396,12 @@ static inline void iwl_trans_set_pmi(struct iwl_trans *trans, bool state)
trans->ops->set_pmi(trans, state);
}
-static inline void iwl_trans_sw_reset(struct iwl_trans *trans)
+static inline int iwl_trans_sw_reset(struct iwl_trans *trans,
+ bool retake_ownership)
{
if (trans->ops->sw_reset)
- trans->ops->sw_reset(trans);
+ return trans->ops->sw_reset(trans, retake_ownership);
+ return 0;
}
static inline void
diff --git a/drivers/net/wireless/intel/iwlwifi/mei/Makefile b/drivers/net/wireless/intel/iwlwifi/mei/Makefile
new file mode 100644
index 0000000..8e3ef03
--- /dev/null
+++ b/drivers/net/wireless/intel/iwlwifi/mei/Makefile
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: GPL-2.0-only
+obj-$(CONFIG_IWLMEI) += iwlmei.o
+iwlmei-y += main.o
+iwlmei-y += net.o
+iwlmei-$(CONFIG_IWLWIFI_DEVICE_TRACING) += trace.o
+CFLAGS_trace.o := -I$(src)
+
+ccflags-y += -I $(srctree)/$(src)/../
diff --git a/drivers/net/wireless/intel/iwlwifi/mei/internal.h b/drivers/net/wireless/intel/iwlwifi/mei/internal.h
new file mode 100644
index 0000000..92fea7d
--- /dev/null
+++ b/drivers/net/wireless/intel/iwlwifi/mei/internal.h
@@ -0,0 +1,20 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2021 Intel Corporation
+ */
+
+#ifndef __IWLMEI_INTERNAL_H_
+#define __IWLMEI_INTERNAL_H_
+
+#include <uapi/linux/if_ether.h>
+#include <linux/netdevice.h>
+
+#include "sap.h"
+
+rx_handler_result_t iwl_mei_rx_filter(struct sk_buff *skb,
+ const struct iwl_sap_oob_filters *filters,
+ bool *pass_to_csme);
+
+void iwl_mei_add_data_to_ring(struct sk_buff *skb, bool cb_tx);
+
+#endif /* __IWLMEI_INTERNAL_H_ */
diff --git a/drivers/net/wireless/intel/iwlwifi/mei/iwl-mei.h b/drivers/net/wireless/intel/iwlwifi/mei/iwl-mei.h
new file mode 100644
index 0000000..67122cf
--- /dev/null
+++ b/drivers/net/wireless/intel/iwlwifi/mei/iwl-mei.h
@@ -0,0 +1,505 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2021 Intel Corporation
+ */
+
+#ifndef __iwl_mei_h__
+#define __iwl_mei_h__
+
+#include <linux/if_ether.h>
+#include <linux/skbuff.h>
+#include <linux/ieee80211.h>
+
+/**
+ * DOC: Introduction
+ *
+ * iwlmei is the kernel module that is in charge of the commnunication between
+ * the iwlwifi driver and the CSME firmware's WLAN driver. This communication
+ * uses the SAP protocol defined in another file.
+ * iwlwifi can request or release ownership on the WiFi device through iwlmei.
+ * iwlmei may notify iwlwifi about certain events: what filter iwlwifi should
+ * use to passthrough inbound packets to the CSME firmware for example. iwlmei
+ * may also use iwlwifi to send traffic. This means that we need communication
+ * from iwlmei to iwlwifi and the other way around.
+ */
+
+/**
+ * DOC: Life cycle
+ *
+ * iwlmei exports symbols that are needed by iwlwifi so that iwlmei will always
+ * be loaded when iwlwifi is alive. iwlwifi registers itself to iwlmei and
+ * provides the pointers to the functions that iwlmei calls whenever needed.
+ * iwlwifi calls iwlmei through direct and context-free function calls.
+ * It is assumed that only one device is accessible to the CSME firmware and
+ * under the scope of iwlmei so that it is valid not to have any context passed
+ * to iwlmei's functions.
+ *
+ * There are cases in which iwlmei can't access the CSME firmware, because the
+ * CSME firmware is undergoing a reset, or the mei bus decided to unbind the
+ * device. In those cases, iwlmei will need not to send requests over the mei
+ * bus. Instead, it needs to cache the requests from iwlwifi and fulfill them
+ * when the mei bus is available again.
+ *
+ * iwlmei can call iwlwifi as long as iwlwifi is registered to iwlmei. When
+ * iwlwifi goes down (the PCI device is unbound, or the iwlwifi is unloaded)
+ * iwlwifi needs to unregister from iwlmei.
+ */
+
+/**
+ * DOC: Memory layout
+ *
+ * Since iwlwifi calls iwlmei without any context, iwlmei needs to hold a
+ * global pointer to its data (which is in the mei client device's private
+ * data area). If there was no bind on the mei bus, this pointer is NULL and
+ * iwlmei knows not access to the CSME firmware upon requests from iwlwifi.
+ *
+ * iwlmei needs to cache requests from iwlwifi when there is no mei client
+ * device available (when iwlmei has been removed from the mei bus). In this
+ * case, all iwlmei's data that resides in the mei client device's private data
+ * area is unavailable. For this specific case, a separate caching area is
+ * needed.
+ */
+
+/**
+ * DOC: Concurrency
+ *
+ * iwlwifi can call iwlmei at any time. iwlmei will take care to synchronize
+ * the calls from iwlwifi with its internal flows. iwlwifi must not call iwlmei
+ * in flows that cannot sleep. Moreover, iwlwifi must not call iwlmei in flows
+ * that originated from iwlmei.
+ */
+
+/**
+ * DOC: Probe and remove from mei bus driver
+ *
+ * When the mei bus driver enumerates its devices, it calls the iwlmei's probe
+ * function which will send the %SAP_ME_MSG_START message. The probe completes
+ * before the response (%SAP_ME_MSG_START_OK) is received. This response will
+ * be handle by the Rx path. Once it arrives, the connection to the CSME
+ * firmware is considered established and iwlwifi's requests can be treated
+ * against the CSME firmware.
+ *
+ * When the mei bus driver removes the device, iwlmei loses all the data that
+ * was attached to the mei client device. It clears the global pointer to the
+ * mei client device since it is not available anymore. This will cause all the
+ * requests coming from iwlwifi to be cached. This flow takes the global mutex
+ * to be synchronized with all the requests coming from iwlwifi.
+ */
+
+/**
+ * DOC: Driver load when CSME owns the device
+ *
+ * When the driver (iwlwifi) is loaded while CSME owns the device,
+ * it'll ask CSME to release the device through HW registers. CSME
+ * will release the device only in the case that there is no connection
+ * through the mei bus. If there is a mei bus connection, CSME will refuse
+ * to release the ownership on the device through the HW registers. In that
+ * case, iwlwifi must first request ownership using the SAP protocol.
+ *
+ * Once iwlwifi will request ownership through the SAP protocol, CSME will
+ * grant the ownership on the device through the HW registers as well.
+ * In order to request ownership over SAP, we first need to have an interface
+ * which means that we need to register to mac80211.
+ * This can't happen before we get the NVM that contains all the capabilities
+ * of the device. Reading the NVM usually requires the load the firmware, but
+ * this is impossible as long as we don't have ownership on the device.
+ * In order to solve this chicken and egg problem, the host driver can get
+ * the NVM through CSME which owns the device. It can send
+ * %SAP_MSG_NOTIF_GET_NVM, which will be replied by %SAP_MSG_NOTIF_NVM with
+ * the NVM's content that the host driver needs.
+ */
+
+/**
+ * DOC: CSME behavior regarding the ownership requests
+ *
+ * The ownership requests from the host can come in two different ways:
+ * - the HW registers in iwl_pcie_set_hw_ready
+ * - using the Software Arbitration Protocol (SAP)
+ *
+ * The host can ask CSME who owns the device with %SAP_MSG_NOTIF_WHO_OWNS_NIC,
+ * and it can request ownership with %SAP_MSG_NOTIF_HOST_ASKS_FOR_NIC_OWNERSHIP.
+ * The host will first use %SAP_MSG_NOTIF_WHO_OWNS_NIC to know what state
+ * CSME is in. In case CSME thinks it owns the device, the host can ask for
+ * ownership with %SAP_MSG_NOTIF_HOST_ASKS_FOR_NIC_OWNERSHIP.
+ *
+ * Here the table that describes CSME's behavior upon ownership request:
+ *
+ * +-------------------+------------+--------------+-----------------------------+------------+
+ * | State | HW reg bit | Reply for | Event | HW reg bit |
+ * | | before | WHO_OWNS_NIC | | after |
+ * +===================+============+==============+=============================+============+
+ * | WiAMT not | 0 | Host | HW register or | 0 |
+ * | operational | Host owner | | HOST_ASKS_FOR_NIC_OWNERSHIP | Host owner |
+ * +-------------------+------------+--------------+-----------------------------+------------+
+ * | Operational & | 1 | N/A | HW register | 0 |
+ * | SAP down & | CSME owner | | | Host owner |
+ * | no session active | | | | |
+ * +-------------------+------------+--------------+-----------------------------+------------+
+ * | Operational & | 1 | CSME | HW register | 1 |
+ * | SAP up | CSME owner | | | CSME owner |
+ * +-------------------+------------+--------------+-----------------------------+------------+
+ * | Operational & | 1 | CSME | HOST_ASKS_FOR_NIC_OWNERSHIP | 0 |
+ * | SAP up | CSME owner | | | Host owner |
+ * +-------------------+------------+--------------+-----------------------------+------------+
+ */
+
+/**
+ * DOC: Driver load when CSME is associated and a session is active
+ *
+ * A "session" is active when CSME is associated to an access point and the
+ * link is used to attach a remote driver or to control the system remotely.
+ * When a session is active, we want to make sure it won't disconnect when we
+ * take ownership on the device.
+ * In this case, the driver can get the device, but it'll need to make
+ * sure that it'll connect to the exact same AP (same BSSID).
+ * In order to do so, CSME will send the connection parameters through
+ * SAP and then the host can check if it can connect to this same AP.
+ * If yes, it can request ownership through SAP and connect quickly without
+ * scanning all the channels, but just probing the AP on the channel that
+ * CSME was connected to.
+ * In order to signal this specific scenario to iwlwifi, iwlmei will
+ * immediately require iwlwifi to report RF-Kill to the network stack. This
+ * RF-Kill will prevent the stack from getting the device, and it has a reason
+ * that tells the userspace that the device is in RF-Kill because it is not
+ * owned by the host. Once the userspace has configured the right profile,
+ * it'll be able to let iwlmei know that it can request ownership over SAP
+ * which will remove the RF-Kill, and finally allow the host to connect.
+ * The host has then 3 seconds to connect (including DHCP). Had the host
+ * failed to connect within those 3 seconds, CSME will take the device back.
+ */
+
+/**
+ * DOC: Datapath
+ *
+ * CSME can transmit packets, through the netdev that it gets from the wifi
+ * driver. It'll send packet in the 802.3 format and simply call
+ * dev_queue_xmit.
+ *
+ * For Rx, iwlmei registers a Rx handler that it attaches to the netdev. iwlmei
+ * may catch packets and send them to CSME, it can then either drop them so
+ * that they are invisible to user space, or let them go the user space.
+ *
+ * Packets transmitted by the user space do not need to be forwarded to CSME
+ * with the exception of the DHCP request. In order to know what IP is used
+ * by the user space, CSME needs to get the DHCP request. See
+ * iwl_mei_tx_copy_to_csme().
+ */
+
+/**
+ * enum iwl_mei_nvm_caps - capabilities for MEI NVM
+ * @MEI_NVM_CAPS_LARI_SUPPORT: Lari is supported
+ * @MEI_NVM_CAPS_11AX_SUPPORT: 11AX is supported
+ */
+enum iwl_mei_nvm_caps {
+ MEI_NVM_CAPS_LARI_SUPPORT = BIT(0),
+ MEI_NVM_CAPS_11AX_SUPPORT = BIT(1),
+};
+
+/**
+ * struct iwl_mei_nvm - used to pass the NVM from CSME
+ * @hw_addr: The MAC address
+ * @n_hw_addrs: The number of MAC addresses
+ * @reserved: For alignment.
+ * @radio_cfg: The radio configuration.
+ * @caps: See &enum iwl_mei_nvm_caps.
+ * @nvm_version: The version of the NVM.
+ * @channels: The data for each channel.
+ *
+ * If a field is added, it must correspond to the SAP structure.
+ */
+struct iwl_mei_nvm {
+ u8 hw_addr[ETH_ALEN];
+ u8 n_hw_addrs;
+ u8 reserved;
+ u32 radio_cfg;
+ u32 caps;
+ u32 nvm_version;
+ u32 channels[110];
+};
+
+/**
+ * enum iwl_mei_pairwise_cipher - cipher for UCAST key
+ * @IWL_MEI_CIPHER_NONE: none
+ * @IWL_MEI_CIPHER_CCMP: ccmp
+ * @IWL_MEI_CIPHER_GCMP: gcmp
+ * @IWL_MEI_CIPHER_GCMP_256: gcmp 256
+ *
+ * Note that those values are dictated by the CSME firmware API (see sap.h)
+ */
+enum iwl_mei_pairwise_cipher {
+ IWL_MEI_CIPHER_NONE = 0,
+ IWL_MEI_CIPHER_CCMP = 4,
+ IWL_MEI_CIPHER_GCMP = 8,
+ IWL_MEI_CIPHER_GCMP_256 = 9,
+};
+
+/**
+ * enum iwl_mei_akm_auth - a combination of AKM and AUTH method
+ * @IWL_MEI_AKM_AUTH_OPEN: No encryption
+ * @IWL_MEI_AKM_AUTH_RSNA: 1X profile
+ * @IWL_MEI_AKM_AUTH_RSNA_PSK: PSK profile
+ * @IWL_MEI_AKM_AUTH_SAE: SAE profile
+ *
+ * Note that those values are dictated by the CSME firmware API (see sap.h)
+ */
+enum iwl_mei_akm_auth {
+ IWL_MEI_AKM_AUTH_OPEN = 0,
+ IWL_MEI_AKM_AUTH_RSNA = 6,
+ IWL_MEI_AKM_AUTH_RSNA_PSK = 7,
+ IWL_MEI_AKM_AUTH_SAE = 9,
+};
+
+/**
+ * struct iwl_mei_conn_info - connection info
+ * @lp_state: link protection state
+ * @auth_mode: authentication mode
+ * @ssid_len: the length of SSID
+ * @ssid: the SSID
+ * @pairwise_cipher: the cipher used for unicast packets
+ * @channel: the associated channel
+ * @band: the associated band
+ * @bssid: the BSSID
+ */
+struct iwl_mei_conn_info {
+ u8 lp_state;
+ u8 auth_mode;
+ u8 ssid_len;
+ u8 channel;
+ u8 band;
+ u8 pairwise_cipher;
+ u8 bssid[ETH_ALEN];
+ u8 ssid[IEEE80211_MAX_SSID_LEN];
+};
+
+/**
+ * struct iwl_mei_colloc_info - collocated AP info
+ * @channel: the channel of the collocated AP
+ * @bssid: the BSSID of the collocated AP
+ */
+struct iwl_mei_colloc_info {
+ u8 channel;
+ u8 bssid[ETH_ALEN];
+};
+
+/*
+ * struct iwl_mei_ops - driver's operations called by iwlmei
+ * Operations will not be called more than once concurrently.
+ * It's not allowed to call iwlmei functions from this context.
+ *
+ * @me_conn_status: provide information about CSME's current connection.
+ * @rfkill: called when the wifi driver should report a change in the rfkill
+ * status.
+ * @roaming_forbidden: indicates whether roaming is forbidden.
+ * @sap_connected: indicate that SAP is now connected. Will be called in case
+ * the wifi driver registered to iwlmei before SAP connection succeeded or
+ * when the SAP connection is re-established.
+ * @nic_stolen: this means that device is no longer available. The device can
+ * still be used until the callback returns.
+ */
+struct iwl_mei_ops {
+ void (*me_conn_status)(void *priv,
+ const struct iwl_mei_conn_info *conn_info);
+ void (*rfkill)(void *priv, bool blocked);
+ void (*roaming_forbidden)(void *priv, bool forbidden);
+ void (*sap_connected)(void *priv);
+ void (*nic_stolen)(void *priv);
+};
+
+#if IS_ENABLED(CONFIG_IWLMEI)
+
+/**
+ * iwl_mei_is_connected() - is the connection to the CSME firmware established?
+ *
+ * Return: true if we have a SAP connection
+ */
+bool iwl_mei_is_connected(void);
+
+/**
+ * iwl_mei_get_nvm() - returns the NVM for the device
+ *
+ * It is the caller's responsibility to free the memory returned
+ * by this function.
+ * This function blocks (sleeps) until the NVM is ready.
+ *
+ * Return: the NVM as received from CSME
+ */
+struct iwl_mei_nvm *iwl_mei_get_nvm(void);
+
+/**
+ * iwl_mei_get_ownership() - request ownership
+ *
+ * This function blocks until ownership is granted or timeout expired.
+ *
+ * Return: 0 in case we could get ownership on the device
+ */
+int iwl_mei_get_ownership(void);
+
+/**
+ * iwl_mei_set_rfkill_state() - set SW and HW RF kill states
+ * @hw_rfkill: HW RF kill state.
+ * @sw_rfkill: SW RF kill state.
+ *
+ * This function must be called when SW RF kill is issued by the user.
+ */
+void iwl_mei_set_rfkill_state(bool hw_rfkill, bool sw_rfkill);
+
+/**
+ * iwl_mei_set_nic_info() - set mac address
+ * @mac_address: mac address to set
+ * @nvm_address: NVM mac adsress to set
+ *
+ * This function must be called upon mac address change.
+ */
+void iwl_mei_set_nic_info(const u8 *mac_address, const u8 *nvm_address);
+
+/**
+ * iwl_mei_set_country_code() - set new country code
+ * @mcc: the new applied MCC
+ *
+ * This function must be called upon country code update
+ */
+void iwl_mei_set_country_code(u16 mcc);
+
+/**
+ * iwl_mei_set_power_limit() - set TX power limit
+ * @power_limit: pointer to an array of 10 elements (le16) represents the power
+ * restrictions per chain.
+ *
+ * This function must be called upon power restrictions change
+ */
+void iwl_mei_set_power_limit(const __le16 *power_limit);
+
+/**
+ * iwl_mei_register() - register the wifi driver to iwlmei
+ * @priv: a pointer to the wifi driver's context. Cannot be NULL.
+ * @ops: the ops structure.
+ *
+ * Return: 0 unless something went wrong. It is illegal to call any
+ * other API function before this function is called and succeeds.
+ *
+ * Only one wifi driver instance (wifi device instance really)
+ * can register at a time.
+ */
+int iwl_mei_register(void *priv, const struct iwl_mei_ops *ops);
+
+/**
+ * iwl_mei_start_unregister() - unregister the wifi driver from iwlmei
+ *
+ * From this point on, iwlmei will not used the callbacks provided by
+ * the driver, but the device is still usable.
+ */
+void iwl_mei_start_unregister(void);
+
+/**
+ * iwl_mei_unregister_complete() - complete the unregistration
+ *
+ * Must be called after iwl_mei_start_unregister. When this function returns,
+ * the device is owned by CSME.
+ */
+void iwl_mei_unregister_complete(void);
+
+/**
+ * iwl_mei_set_netdev() - sets the netdev for Tx / Rx.
+ * @netdev: the net_device
+ *
+ * The caller should set the netdev to a non-NULL value when the
+ * interface is added. Packets might be sent to the driver immediately
+ * afterwards.
+ * The caller should set the netdev to NULL when the interface is removed.
+ * This function will call synchronize_net() after setting the netdev to NULL.
+ * Only when this function returns, can the caller assume that iwlmei will
+ * no longer inject packets into the netdev's Tx path.
+ *
+ * Context: This function can sleep and assumes rtnl_lock is taken.
+ * The netdev must be set to NULL before iwl_mei_start_unregister() is called.
+ */
+void iwl_mei_set_netdev(struct net_device *netdev);
+
+/**
+ * iwl_mei_tx_copy_to_csme() - must be called for each packet sent by
+ * the wifi driver.
+ * @skb: the skb sent
+ * @ivlen: the size of the IV that needs to be skipped after the MAC and
+ * before the SNAP header.
+ *
+ * This function doesn't take any lock, it simply tries to catch DHCP
+ * packets sent by the wifi driver. If the packet is a DHCP packet, it
+ * will send it to CSME. This function must not be called for virtual
+ * interfaces that are not monitored by CSME, meaning it must be called
+ * only for packets transmitted by the netdevice that was registered
+ * with iwl_mei_set_netdev().
+ */
+void iwl_mei_tx_copy_to_csme(struct sk_buff *skb, unsigned int ivlen);
+
+/**
+ * iwl_mei_host_associated() - must be called when iwlwifi associated.
+ * @conn_info: pointer to the connection info structure.
+ * @colloc_info: pointer to the collocated AP info. This is relevant only in
+ * case of UHB associated AP, otherwise set to NULL.
+ */
+void iwl_mei_host_associated(const struct iwl_mei_conn_info *conn_info,
+ const struct iwl_mei_colloc_info *colloc_info);
+
+/**
+ * iwl_mei_host_disassociated() - must be called when iwlwifi disassociated.
+ */
+void iwl_mei_host_disassociated(void);
+
+/**
+ * iwl_mei_device_down() - must be called when the device is down
+ */
+void iwl_mei_device_down(void);
+
+#else
+
+static inline bool iwl_mei_is_connected(void)
+{ return false; }
+
+static inline struct iwl_mei_nvm *iwl_mei_get_nvm(void)
+{ return NULL; }
+
+static inline int iwl_mei_get_ownership(void)
+{ return 0; }
+
+static inline void iwl_mei_set_rfkill_state(bool hw_rfkill, bool sw_rfkill)
+{}
+
+static inline void iwl_mei_set_nic_info(const u8 *mac_address, const u8 *nvm_address)
+{}
+
+static inline void iwl_mei_set_country_code(u16 mcc)
+{}
+
+static inline void iwl_mei_set_power_limit(__le16 *power_limit)
+{}
+
+static inline int iwl_mei_register(void *priv,
+ const struct iwl_mei_ops *ops)
+{ return 0; }
+
+static inline void iwl_mei_start_unregister(void)
+{}
+
+static inline void iwl_mei_unregister_complete(void)
+{}
+
+static inline void iwl_mei_set_netdev(struct net_device *netdev)
+{}
+
+static inline void iwl_mei_tx_copy_to_csme(struct sk_buff *skb,
+ unsigned int ivlen)
+{}
+
+static inline void iwl_mei_host_associated(const struct iwl_mei_conn_info *conn_info,
+ const struct iwl_mei_colloc_info *colloc_info)
+{}
+
+static inline void iwl_mei_host_disassociated(void)
+{}
+
+static inline void iwl_mei_device_down(void)
+{}
+
+#endif /* CONFIG_IWLMEI */
+
+#endif /* __iwl_mei_h__ */
diff --git a/drivers/net/wireless/intel/iwlwifi/mei/main.c b/drivers/net/wireless/intel/iwlwifi/mei/main.c
new file mode 100644
index 0000000..d9733aaf
--- /dev/null
+++ b/drivers/net/wireless/intel/iwlwifi/mei/main.c
@@ -0,0 +1,2001 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2021 Intel Corporation
+ */
+
+#include <linux/etherdevice.h>
+#include <linux/netdevice.h>
+#include <linux/ieee80211.h>
+#include <linux/rtnetlink.h>
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <linux/mei_cl_bus.h>
+#include <linux/rcupdate.h>
+#include <linux/debugfs.h>
+#include <linux/skbuff.h>
+#include <linux/wait.h>
+#include <linux/slab.h>
+#include <linux/mm.h>
+
+#include <net/cfg80211.h>
+
+#include "internal.h"
+#include "iwl-mei.h"
+#include "trace.h"
+#include "trace-data.h"
+#include "sap.h"
+
+MODULE_DESCRIPTION("The Intel(R) wireless / CSME firmware interface");
+MODULE_LICENSE("GPL");
+
+#define MEI_WLAN_UUID UUID_LE(0x13280904, 0x7792, 0x4fcb, \
+ 0xa1, 0xaa, 0x5e, 0x70, 0xcb, 0xb1, 0xe8, 0x65)
+
+/*
+ * Since iwlwifi calls iwlmei without any context, hold a pointer to the
+ * mei_cl_device structure here.
+ * Define a mutex that will synchronize all the flows between iwlwifi and
+ * iwlmei.
+ * Note that iwlmei can't have several instances, so it ok to have static
+ * variables here.
+ */
+static struct mei_cl_device *iwl_mei_global_cldev;
+static DEFINE_MUTEX(iwl_mei_mutex);
+static unsigned long iwl_mei_status;
+
+enum iwl_mei_status_bits {
+ IWL_MEI_STATUS_SAP_CONNECTED,
+};
+
+bool iwl_mei_is_connected(void)
+{
+ return test_bit(IWL_MEI_STATUS_SAP_CONNECTED, &iwl_mei_status);
+}
+EXPORT_SYMBOL_GPL(iwl_mei_is_connected);
+
+#define SAP_VERSION 3
+#define SAP_CONTROL_BLOCK_ID 0x21504153 /* SAP! in ASCII */
+
+struct iwl_sap_q_ctrl_blk {
+ __le32 wr_ptr;
+ __le32 rd_ptr;
+ __le32 size;
+};
+
+enum iwl_sap_q_idx {
+ SAP_QUEUE_IDX_NOTIF = 0,
+ SAP_QUEUE_IDX_DATA,
+ SAP_QUEUE_IDX_MAX,
+};
+
+struct iwl_sap_dir {
+ __le32 reserved;
+ struct iwl_sap_q_ctrl_blk q_ctrl_blk[SAP_QUEUE_IDX_MAX];
+};
+
+enum iwl_sap_dir_idx {
+ SAP_DIRECTION_HOST_TO_ME = 0,
+ SAP_DIRECTION_ME_TO_HOST,
+ SAP_DIRECTION_MAX,
+};
+
+struct iwl_sap_shared_mem_ctrl_blk {
+ __le32 sap_id;
+ __le32 size;
+ struct iwl_sap_dir dir[SAP_DIRECTION_MAX];
+};
+
+/*
+ * The shared area has the following layout:
+ *
+ * +-----------------------------------+
+ * |struct iwl_sap_shared_mem_ctrl_blk |
+ * +-----------------------------------+
+ * |Host -> ME data queue |
+ * +-----------------------------------+
+ * |Host -> ME notif queue |
+ * +-----------------------------------+
+ * |ME -> Host data queue |
+ * +-----------------------------------+
+ * |ME -> host notif queue |
+ * +-----------------------------------+
+ * |SAP control block id (SAP!) |
+ * +-----------------------------------+
+ */
+
+#define SAP_H2M_DATA_Q_SZ 48256
+#define SAP_M2H_DATA_Q_SZ 24128
+#define SAP_H2M_NOTIF_Q_SZ 2240
+#define SAP_M2H_NOTIF_Q_SZ 62720
+
+#define _IWL_MEI_SAP_SHARED_MEM_SZ \
+ (sizeof(struct iwl_sap_shared_mem_ctrl_blk) + \
+ SAP_H2M_DATA_Q_SZ + SAP_H2M_NOTIF_Q_SZ + \
+ SAP_M2H_DATA_Q_SZ + SAP_M2H_NOTIF_Q_SZ + 4)
+
+#define IWL_MEI_SAP_SHARED_MEM_SZ \
+ (roundup(_IWL_MEI_SAP_SHARED_MEM_SZ, PAGE_SIZE))
+
+struct iwl_mei_shared_mem_ptrs {
+ struct iwl_sap_shared_mem_ctrl_blk *ctrl;
+ void *q_head[SAP_DIRECTION_MAX][SAP_QUEUE_IDX_MAX];
+ size_t q_size[SAP_DIRECTION_MAX][SAP_QUEUE_IDX_MAX];
+};
+
+struct iwl_mei_filters {
+ struct rcu_head rcu_head;
+ struct iwl_sap_oob_filters filters;
+};
+
+/**
+ * struct iwl_mei - holds the private date for iwl_mei
+ *
+ * @get_nvm_wq: the wait queue for the get_nvm flow
+ * @send_csa_msg_wk: used to defer the transmission of the CHECK_SHARED_AREA
+ * message. Used so that we can send CHECK_SHARED_AREA from atomic
+ * contexts.
+ * @get_ownership_wq: the wait queue for the get_ownership_flow
+ * @shared_mem: the memory that is shared between CSME and the host
+ * @cldev: the pointer to the MEI client device
+ * @nvm: the data returned by the CSME for the NVM
+ * @filters: the filters sent by CSME
+ * @got_ownership: true if we own the device
+ * @amt_enabled: true if CSME has wireless enabled
+ * @csa_throttled: when true, we can't send CHECK_SHARED_AREA over the MEI
+ * bus, but rather need to wait until send_csa_msg_wk runs
+ * @csme_taking_ownership: true when CSME is taking ownership. Used to remember
+ * to send CSME_OWNERSHIP_CONFIRMED when the driver completes its down
+ * flow.
+ * @csa_throttle_end_wk: used when &csa_throttled is true
+ * @data_q_lock: protects the access to the data queues which are
+ * accessed without the mutex.
+ * @sap_seq_no: the sequence number for the SAP messages
+ * @seq_no: the sequence number for the SAP messages
+ * @dbgfs_dir: the debugfs dir entry
+ */
+struct iwl_mei {
+ wait_queue_head_t get_nvm_wq;
+ struct work_struct send_csa_msg_wk;
+ wait_queue_head_t get_ownership_wq;
+ struct iwl_mei_shared_mem_ptrs shared_mem;
+ struct mei_cl_device *cldev;
+ struct iwl_mei_nvm *nvm;
+ struct iwl_mei_filters __rcu *filters;
+ bool got_ownership;
+ bool amt_enabled;
+ bool csa_throttled;
+ bool csme_taking_ownership;
+ struct delayed_work csa_throttle_end_wk;
+ spinlock_t data_q_lock;
+
+ atomic_t sap_seq_no;
+ atomic_t seq_no;
+
+ struct dentry *dbgfs_dir;
+};
+
+/**
+ * struct iwl_mei_cache - cache for the parameters from iwlwifi
+ * @ops: Callbacks to iwlwifi.
+ * @netdev: The netdev that will be used to transmit / receive packets.
+ * @conn_info: The connection info message triggered by iwlwifi's association.
+ * @power_limit: pointer to an array of 10 elements (le16) represents the power
+ * restrictions per chain.
+ * @rf_kill: rf kill state.
+ * @mcc: MCC info
+ * @mac_address: interface MAC address.
+ * @nvm_address: NVM MAC address.
+ * @priv: A pointer to iwlwifi.
+ *
+ * This used to cache the configurations coming from iwlwifi's way. The data
+ * is cached here so that we can buffer the configuration even if we don't have
+ * a bind from the mei bus and hence, on iwl_mei structure.
+ */
+struct iwl_mei_cache {
+ const struct iwl_mei_ops *ops;
+ struct net_device __rcu *netdev;
+ const struct iwl_sap_notif_connection_info *conn_info;
+ const __le16 *power_limit;
+ u32 rf_kill;
+ u16 mcc;
+ u8 mac_address[6];
+ u8 nvm_address[6];
+ void *priv;
+};
+
+static struct iwl_mei_cache iwl_mei_cache = {
+ .rf_kill = SAP_HW_RFKILL_DEASSERTED | SAP_SW_RFKILL_DEASSERTED
+};
+
+static void iwl_mei_free_shared_mem(struct mei_cl_device *cldev)
+{
+ struct iwl_mei *mei = mei_cldev_get_drvdata(cldev);
+
+ if (mei_cldev_dma_unmap(cldev))
+ dev_err(&cldev->dev, "Couldn't unmap the shared mem properly\n");
+ memset(&mei->shared_mem, 0, sizeof(mei->shared_mem));
+}
+
+#define HBM_DMA_BUF_ID_WLAN 1
+
+static int iwl_mei_alloc_shared_mem(struct mei_cl_device *cldev)
+{
+ struct iwl_mei *mei = mei_cldev_get_drvdata(cldev);
+ struct iwl_mei_shared_mem_ptrs *mem = &mei->shared_mem;
+
+ mem->ctrl = mei_cldev_dma_map(cldev, HBM_DMA_BUF_ID_WLAN,
+ IWL_MEI_SAP_SHARED_MEM_SZ);
+
+ if (IS_ERR(mem->ctrl)) {
+ int ret = PTR_ERR(mem->ctrl);
+
+ dev_err(&cldev->dev, "Couldn't allocate the shared memory: %d\n",
+ ret);
+ mem->ctrl = NULL;
+
+ return ret;
+ }
+
+ memset(mem->ctrl, 0, IWL_MEI_SAP_SHARED_MEM_SZ);
+
+ return 0;
+}
+
+static void iwl_mei_init_shared_mem(struct iwl_mei *mei)
+{
+ struct iwl_mei_shared_mem_ptrs *mem = &mei->shared_mem;
+ struct iwl_sap_dir *h2m;
+ struct iwl_sap_dir *m2h;
+ int dir, queue;
+ u8 *q_head;
+
+ mem->ctrl->sap_id = cpu_to_le32(SAP_CONTROL_BLOCK_ID);
+
+ mem->ctrl->size = cpu_to_le32(sizeof(*mem->ctrl));
+
+ h2m = &mem->ctrl->dir[SAP_DIRECTION_HOST_TO_ME];
+ m2h = &mem->ctrl->dir[SAP_DIRECTION_ME_TO_HOST];
+
+ h2m->q_ctrl_blk[SAP_QUEUE_IDX_DATA].size =
+ cpu_to_le32(SAP_H2M_DATA_Q_SZ);
+ h2m->q_ctrl_blk[SAP_QUEUE_IDX_NOTIF].size =
+ cpu_to_le32(SAP_H2M_NOTIF_Q_SZ);
+ m2h->q_ctrl_blk[SAP_QUEUE_IDX_DATA].size =
+ cpu_to_le32(SAP_M2H_DATA_Q_SZ);
+ m2h->q_ctrl_blk[SAP_QUEUE_IDX_NOTIF].size =
+ cpu_to_le32(SAP_M2H_NOTIF_Q_SZ);
+
+ /* q_head points to the start of the first queue */
+ q_head = (void *)(mem->ctrl + 1);
+
+ /* Initialize the queue heads */
+ for (dir = 0; dir < SAP_DIRECTION_MAX; dir++) {
+ for (queue = 0; queue < SAP_QUEUE_IDX_MAX; queue++) {
+ mem->q_head[dir][queue] = q_head;
+ q_head +=
+ le32_to_cpu(mem->ctrl->dir[dir].q_ctrl_blk[queue].size);
+ mem->q_size[dir][queue] =
+ le32_to_cpu(mem->ctrl->dir[dir].q_ctrl_blk[queue].size);
+ }
+ }
+
+ *(__le32 *)q_head = cpu_to_le32(SAP_CONTROL_BLOCK_ID);
+}
+
+static ssize_t iwl_mei_write_cyclic_buf(struct mei_cl_device *cldev,
+ struct iwl_sap_q_ctrl_blk *notif_q,
+ u8 *q_head,
+ const struct iwl_sap_hdr *hdr,
+ u32 q_sz)
+{
+ u32 rd = le32_to_cpu(READ_ONCE(notif_q->rd_ptr));
+ u32 wr = le32_to_cpu(READ_ONCE(notif_q->wr_ptr));
+ size_t room_in_buf;
+ size_t tx_sz = sizeof(*hdr) + le16_to_cpu(hdr->len);
+
+ if (rd > q_sz || wr > q_sz) {
+ dev_err(&cldev->dev,
+ "Pointers are past the end of the buffer\n");
+ return -EINVAL;
+ }
+
+ room_in_buf = wr >= rd ? q_sz - wr + rd : rd - wr;
+
+ /* we don't have enough room for the data to write */
+ if (room_in_buf < tx_sz) {
+ dev_err(&cldev->dev,
+ "Not enough room in the buffer\n");
+ return -ENOSPC;
+ }
+
+ if (wr + tx_sz <= q_sz) {
+ memcpy(q_head + wr, hdr, tx_sz);
+ } else {
+ memcpy(q_head + wr, hdr, q_sz - wr);
+ memcpy(q_head, (u8 *)hdr + q_sz - wr, tx_sz - (q_sz - wr));
+ }
+
+ WRITE_ONCE(notif_q->wr_ptr, cpu_to_le32((wr + tx_sz) % q_sz));
+ return 0;
+}
+
+static bool iwl_mei_host_to_me_data_pending(const struct iwl_mei *mei)
+{
+ struct iwl_sap_q_ctrl_blk *notif_q;
+ struct iwl_sap_dir *dir;
+
+ dir = &mei->shared_mem.ctrl->dir[SAP_DIRECTION_HOST_TO_ME];
+ notif_q = &dir->q_ctrl_blk[SAP_QUEUE_IDX_DATA];
+
+ if (READ_ONCE(notif_q->wr_ptr) != READ_ONCE(notif_q->rd_ptr))
+ return true;
+
+ notif_q = &dir->q_ctrl_blk[SAP_QUEUE_IDX_NOTIF];
+ return READ_ONCE(notif_q->wr_ptr) != READ_ONCE(notif_q->rd_ptr);
+}
+
+static int iwl_mei_send_check_shared_area(struct mei_cl_device *cldev)
+{
+ struct iwl_mei *mei = mei_cldev_get_drvdata(cldev);
+ struct iwl_sap_me_msg_start msg = {
+ .hdr.type = cpu_to_le32(SAP_ME_MSG_CHECK_SHARED_AREA),
+ .hdr.seq_num = cpu_to_le32(atomic_inc_return(&mei->seq_no)),
+ };
+ int ret;
+
+ lockdep_assert_held(&iwl_mei_mutex);
+
+ if (mei->csa_throttled)
+ return 0;
+
+ trace_iwlmei_me_msg(&msg.hdr, true);
+ ret = mei_cldev_send(cldev, (void *)&msg, sizeof(msg));
+ if (ret != sizeof(msg)) {
+ dev_err(&cldev->dev,
+ "failed to send the SAP_ME_MSG_CHECK_SHARED_AREA message %d\n",
+ ret);
+ return ret;
+ }
+
+ mei->csa_throttled = true;
+
+ schedule_delayed_work(&mei->csa_throttle_end_wk,
+ msecs_to_jiffies(100));
+
+ return 0;
+}
+
+static void iwl_mei_csa_throttle_end_wk(struct work_struct *wk)
+{
+ struct iwl_mei *mei =
+ container_of(wk, struct iwl_mei, csa_throttle_end_wk.work);
+
+ mutex_lock(&iwl_mei_mutex);
+
+ mei->csa_throttled = false;
+
+ if (iwl_mei_host_to_me_data_pending(mei))
+ iwl_mei_send_check_shared_area(mei->cldev);
+
+ mutex_unlock(&iwl_mei_mutex);
+}
+
+static int iwl_mei_send_sap_msg_payload(struct mei_cl_device *cldev,
+ struct iwl_sap_hdr *hdr)
+{
+ struct iwl_mei *mei = mei_cldev_get_drvdata(cldev);
+ struct iwl_sap_q_ctrl_blk *notif_q;
+ struct iwl_sap_dir *dir;
+ void *q_head;
+ u32 q_sz;
+ int ret;
+
+ lockdep_assert_held(&iwl_mei_mutex);
+
+ if (!mei->shared_mem.ctrl) {
+ dev_err(&cldev->dev,
+ "No shared memory, can't send any SAP message\n");
+ return -EINVAL;
+ }
+
+ if (!iwl_mei_is_connected()) {
+ dev_err(&cldev->dev,
+ "Can't send a SAP message if we're not connected\n");
+ return -ENODEV;
+ }
+
+ hdr->seq_num = cpu_to_le32(atomic_inc_return(&mei->sap_seq_no));
+ dev_dbg(&cldev->dev, "Sending %d\n", hdr->type);
+
+ dir = &mei->shared_mem.ctrl->dir[SAP_DIRECTION_HOST_TO_ME];
+ notif_q = &dir->q_ctrl_blk[SAP_QUEUE_IDX_NOTIF];
+ q_head = mei->shared_mem.q_head[SAP_DIRECTION_HOST_TO_ME][SAP_QUEUE_IDX_NOTIF];
+ q_sz = mei->shared_mem.q_size[SAP_DIRECTION_HOST_TO_ME][SAP_QUEUE_IDX_NOTIF];
+ ret = iwl_mei_write_cyclic_buf(q_head, notif_q, q_head, hdr, q_sz);
+
+ if (ret < 0)
+ return ret;
+
+ trace_iwlmei_sap_cmd(hdr, true);
+
+ return iwl_mei_send_check_shared_area(cldev);
+}
+
+void iwl_mei_add_data_to_ring(struct sk_buff *skb, bool cb_tx)
+{
+ struct iwl_sap_q_ctrl_blk *notif_q;
+ struct iwl_sap_dir *dir;
+ struct iwl_mei *mei;
+ size_t room_in_buf;
+ size_t tx_sz;
+ size_t hdr_sz;
+ u32 q_sz;
+ u32 rd;
+ u32 wr;
+ void *q_head;
+
+ if (!iwl_mei_global_cldev)
+ return;
+
+ mei = mei_cldev_get_drvdata(iwl_mei_global_cldev);
+
+ /*
+ * We access this path for Rx packets (the more common case)
+ * and from Tx path when we send DHCP packets, the latter is
+ * very unlikely.
+ * Take the lock already here to make sure we see that remove()
+ * might have cleared the IWL_MEI_STATUS_SAP_CONNECTED bit.
+ */
+ spin_lock_bh(&mei->data_q_lock);
+
+ if (!iwl_mei_is_connected()) {
+ spin_unlock_bh(&mei->data_q_lock);
+ return;
+ }
+
+ /*
+ * We are in a RCU critical section and the remove from the CSME bus
+ * which would free this memory waits for the readers to complete (this
+ * is done in netdev_rx_handler_unregister).
+ */
+ dir = &mei->shared_mem.ctrl->dir[SAP_DIRECTION_HOST_TO_ME];
+ notif_q = &dir->q_ctrl_blk[SAP_QUEUE_IDX_DATA];
+ q_head = mei->shared_mem.q_head[SAP_DIRECTION_HOST_TO_ME][SAP_QUEUE_IDX_DATA];
+ q_sz = mei->shared_mem.q_size[SAP_DIRECTION_HOST_TO_ME][SAP_QUEUE_IDX_DATA];
+
+ rd = le32_to_cpu(READ_ONCE(notif_q->rd_ptr));
+ wr = le32_to_cpu(READ_ONCE(notif_q->wr_ptr));
+ hdr_sz = cb_tx ? sizeof(struct iwl_sap_cb_data) :
+ sizeof(struct iwl_sap_hdr);
+ tx_sz = skb->len + hdr_sz;
+
+ if (rd > q_sz || wr > q_sz) {
+ dev_err(&mei->cldev->dev,
+ "can't write the data: pointers are past the end of the buffer\n");
+ goto out;
+ }
+
+ room_in_buf = wr >= rd ? q_sz - wr + rd : rd - wr;
+
+ /* we don't have enough room for the data to write */
+ if (room_in_buf < tx_sz) {
+ dev_err(&mei->cldev->dev,
+ "Not enough room in the buffer for this data\n");
+ goto out;
+ }
+
+ if (skb_headroom(skb) < hdr_sz) {
+ dev_err(&mei->cldev->dev,
+ "Not enough headroom in the skb to write the SAP header\n");
+ goto out;
+ }
+
+ if (cb_tx) {
+ struct iwl_sap_cb_data *cb_hdr = skb_push(skb, sizeof(*cb_hdr));
+
+ cb_hdr->hdr.type = cpu_to_le16(SAP_MSG_CB_DATA_PACKET);
+ cb_hdr->hdr.len = cpu_to_le16(skb->len - sizeof(cb_hdr->hdr));
+ cb_hdr->hdr.seq_num = cpu_to_le32(atomic_inc_return(&mei->sap_seq_no));
+ cb_hdr->to_me_filt_status = cpu_to_le32(BIT(CB_TX_DHCP_FILT_IDX));
+ cb_hdr->data_len = cpu_to_le32(skb->len - sizeof(*cb_hdr));
+ trace_iwlmei_sap_data(skb, IWL_SAP_TX_DHCP);
+ } else {
+ struct iwl_sap_hdr *hdr = skb_push(skb, sizeof(*hdr));
+
+ hdr->type = cpu_to_le16(SAP_MSG_DATA_PACKET);
+ hdr->len = cpu_to_le16(skb->len - sizeof(*hdr));
+ hdr->seq_num = cpu_to_le32(atomic_inc_return(&mei->sap_seq_no));
+ trace_iwlmei_sap_data(skb, IWL_SAP_TX_DATA_FROM_AIR);
+ }
+
+ if (wr + tx_sz <= q_sz) {
+ skb_copy_bits(skb, 0, q_head + wr, tx_sz);
+ } else {
+ skb_copy_bits(skb, 0, q_head + wr, q_sz - wr);
+ skb_copy_bits(skb, q_sz - wr, q_head, tx_sz - (q_sz - wr));
+ }
+
+ WRITE_ONCE(notif_q->wr_ptr, cpu_to_le32((wr + tx_sz) % q_sz));
+
+out:
+ spin_unlock_bh(&mei->data_q_lock);
+}
+
+static int
+iwl_mei_send_sap_msg(struct mei_cl_device *cldev, u16 type)
+{
+ struct iwl_sap_hdr msg = {
+ .type = cpu_to_le16(type),
+ };
+
+ return iwl_mei_send_sap_msg_payload(cldev, &msg);
+}
+
+static void iwl_mei_send_csa_msg_wk(struct work_struct *wk)
+{
+ struct iwl_mei *mei =
+ container_of(wk, struct iwl_mei, send_csa_msg_wk);
+
+ if (!iwl_mei_is_connected())
+ return;
+
+ mutex_lock(&iwl_mei_mutex);
+
+ iwl_mei_send_check_shared_area(mei->cldev);
+
+ mutex_unlock(&iwl_mei_mutex);
+}
+
+/* Called in a RCU read critical section from netif_receive_skb */
+static rx_handler_result_t iwl_mei_rx_handler(struct sk_buff **pskb)
+{
+ struct sk_buff *skb = *pskb;
+ struct iwl_mei *mei =
+ rcu_dereference(skb->dev->rx_handler_data);
+ struct iwl_mei_filters *filters = rcu_dereference(mei->filters);
+ bool rx_for_csme = false;
+ rx_handler_result_t res;
+
+ /*
+ * remove() unregisters this handler and synchronize_net, so this
+ * should never happen.
+ */
+ if (!iwl_mei_is_connected()) {
+ dev_err(&mei->cldev->dev,
+ "Got an Rx packet, but we're not connected to SAP?\n");
+ return RX_HANDLER_PASS;
+ }
+
+ if (filters)
+ res = iwl_mei_rx_filter(skb, &filters->filters, &rx_for_csme);
+ else
+ res = RX_HANDLER_PASS;
+
+ /*
+ * The data is already on the ring of the shared area, all we
+ * need to do is to tell the CSME firmware to check what we have
+ * there.
+ */
+ if (rx_for_csme)
+ schedule_work(&mei->send_csa_msg_wk);
+
+ if (res != RX_HANDLER_PASS) {
+ trace_iwlmei_sap_data(skb, IWL_SAP_RX_DATA_DROPPED_FROM_AIR);
+ dev_kfree_skb(skb);
+ }
+
+ return res;
+}
+
+static void
+iwl_mei_handle_rx_start_ok(struct mei_cl_device *cldev,
+ const struct iwl_sap_me_msg_start_ok *rsp,
+ ssize_t len)
+{
+ struct iwl_mei *mei = mei_cldev_get_drvdata(cldev);
+
+ if (len != sizeof(*rsp)) {
+ dev_err(&cldev->dev,
+ "got invalid SAP_ME_MSG_START_OK from CSME firmware\n");
+ dev_err(&cldev->dev,
+ "size is incorrect: %zd instead of %zu\n",
+ len, sizeof(*rsp));
+ return;
+ }
+
+ if (rsp->supported_version != SAP_VERSION) {
+ dev_err(&cldev->dev,
+ "didn't get the expected version: got %d\n",
+ rsp->supported_version);
+ return;
+ }
+
+ mutex_lock(&iwl_mei_mutex);
+ set_bit(IWL_MEI_STATUS_SAP_CONNECTED, &iwl_mei_status);
+ /* wifi driver has registered already */
+ if (iwl_mei_cache.ops) {
+ iwl_mei_send_sap_msg(mei->cldev,
+ SAP_MSG_NOTIF_WIFIDR_UP);
+ iwl_mei_cache.ops->sap_connected(iwl_mei_cache.priv);
+ }
+
+ mutex_unlock(&iwl_mei_mutex);
+}
+
+static void iwl_mei_handle_csme_filters(struct mei_cl_device *cldev,
+ const struct iwl_sap_csme_filters *filters)
+{
+ struct iwl_mei *mei = mei_cldev_get_drvdata(iwl_mei_global_cldev);
+ struct iwl_mei_filters *new_filters;
+ struct iwl_mei_filters *old_filters;
+
+ old_filters =
+ rcu_dereference_protected(mei->filters,
+ lockdep_is_held(&iwl_mei_mutex));
+
+ new_filters = kzalloc(sizeof(*new_filters), GFP_KERNEL);
+ if (!new_filters)
+ return;
+
+ /* Copy the OOB filters */
+ new_filters->filters = filters->filters;
+
+ rcu_assign_pointer(mei->filters, new_filters);
+
+ if (old_filters)
+ kfree_rcu(old_filters, rcu_head);
+}
+
+static void
+iwl_mei_handle_conn_status(struct mei_cl_device *cldev,
+ const struct iwl_sap_notif_conn_status *status)
+{
+ struct iwl_mei *mei = mei_cldev_get_drvdata(cldev);
+ struct iwl_mei_conn_info conn_info = {
+ .lp_state = le32_to_cpu(status->link_prot_state),
+ .ssid_len = le32_to_cpu(status->conn_info.ssid_len),
+ .channel = status->conn_info.channel,
+ .band = status->conn_info.band,
+ .auth_mode = le32_to_cpu(status->conn_info.auth_mode),
+ .pairwise_cipher = le32_to_cpu(status->conn_info.pairwise_cipher),
+ };
+
+ if (!iwl_mei_cache.ops ||
+ conn_info.ssid_len > ARRAY_SIZE(conn_info.ssid))
+ return;
+
+ memcpy(conn_info.ssid, status->conn_info.ssid, conn_info.ssid_len);
+ ether_addr_copy(conn_info.bssid, status->conn_info.bssid);
+
+ iwl_mei_cache.ops->me_conn_status(iwl_mei_cache.priv, &conn_info);
+
+ /*
+ * Update the Rfkill state in case the host does not own the device:
+ * if we are in Link Protection, ask to not touch the device, else,
+ * unblock rfkill.
+ * If the host owns the device, inform the user space whether it can
+ * roam.
+ */
+ if (mei->got_ownership)
+ iwl_mei_cache.ops->roaming_forbidden(iwl_mei_cache.priv,
+ status->link_prot_state);
+ else
+ iwl_mei_cache.ops->rfkill(iwl_mei_cache.priv,
+ status->link_prot_state);
+}
+
+static void iwl_mei_set_init_conf(struct iwl_mei *mei)
+{
+ struct iwl_sap_notif_host_link_up link_msg = {
+ .hdr.type = cpu_to_le16(SAP_MSG_NOTIF_HOST_LINK_UP),
+ .hdr.len = cpu_to_le16(sizeof(link_msg) - sizeof(link_msg.hdr)),
+ };
+ struct iwl_sap_notif_country_code mcc_msg = {
+ .hdr.type = cpu_to_le16(SAP_MSG_NOTIF_COUNTRY_CODE),
+ .hdr.len = cpu_to_le16(sizeof(mcc_msg) - sizeof(mcc_msg.hdr)),
+ .mcc = cpu_to_le16(iwl_mei_cache.mcc),
+ };
+ struct iwl_sap_notif_sar_limits sar_msg = {
+ .hdr.type = cpu_to_le16(SAP_MSG_NOTIF_SAR_LIMITS),
+ .hdr.len = cpu_to_le16(sizeof(sar_msg) - sizeof(sar_msg.hdr)),
+ };
+ struct iwl_sap_notif_host_nic_info nic_info_msg = {
+ .hdr.type = cpu_to_le16(SAP_MSG_NOTIF_NIC_INFO),
+ .hdr.len = cpu_to_le16(sizeof(nic_info_msg) - sizeof(nic_info_msg.hdr)),
+ };
+ struct iwl_sap_msg_dw rfkill_msg = {
+ .hdr.type = cpu_to_le16(SAP_MSG_NOTIF_RADIO_STATE),
+ .hdr.len = cpu_to_le16(sizeof(rfkill_msg) - sizeof(rfkill_msg.hdr)),
+ .val = cpu_to_le32(iwl_mei_cache.rf_kill),
+ };
+
+ iwl_mei_send_sap_msg(mei->cldev, SAP_MSG_NOTIF_WHO_OWNS_NIC);
+
+ if (iwl_mei_cache.conn_info) {
+ link_msg.conn_info = *iwl_mei_cache.conn_info;
+ iwl_mei_send_sap_msg_payload(mei->cldev, &link_msg.hdr);
+ }
+
+ iwl_mei_send_sap_msg_payload(mei->cldev, &mcc_msg.hdr);
+
+ if (iwl_mei_cache.power_limit) {
+ memcpy(sar_msg.sar_chain_info_table, iwl_mei_cache.power_limit,
+ sizeof(sar_msg.sar_chain_info_table));
+ iwl_mei_send_sap_msg_payload(mei->cldev, &sar_msg.hdr);
+ }
+
+ ether_addr_copy(nic_info_msg.mac_address, iwl_mei_cache.mac_address);
+ ether_addr_copy(nic_info_msg.nvm_address, iwl_mei_cache.nvm_address);
+ iwl_mei_send_sap_msg_payload(mei->cldev, &nic_info_msg.hdr);
+
+ iwl_mei_send_sap_msg_payload(mei->cldev, &rfkill_msg.hdr);
+}
+
+static void iwl_mei_handle_amt_state(struct mei_cl_device *cldev,
+ const struct iwl_sap_msg_dw *dw)
+{
+ struct iwl_mei *mei = mei_cldev_get_drvdata(cldev);
+ struct net_device *netdev;
+
+ /*
+ * First take rtnl and only then the mutex to avoid an ABBA
+ * with iwl_mei_set_netdev()
+ */
+ rtnl_lock();
+ mutex_lock(&iwl_mei_mutex);
+
+ netdev = rcu_dereference_protected(iwl_mei_cache.netdev,
+ lockdep_is_held(&iwl_mei_mutex));
+
+ if (mei->amt_enabled == !!le32_to_cpu(dw->val))
+ goto out;
+
+ mei->amt_enabled = dw->val;
+
+ if (mei->amt_enabled) {
+ if (netdev)
+ netdev_rx_handler_register(netdev, iwl_mei_rx_handler, mei);
+
+ iwl_mei_set_init_conf(mei);
+ } else {
+ if (iwl_mei_cache.ops)
+ iwl_mei_cache.ops->rfkill(iwl_mei_cache.priv, false);
+ if (netdev)
+ netdev_rx_handler_unregister(netdev);
+ }
+
+out:
+ mutex_unlock(&iwl_mei_mutex);
+ rtnl_unlock();
+}
+
+static void iwl_mei_handle_nic_owner(struct mei_cl_device *cldev,
+ const struct iwl_sap_msg_dw *dw)
+{
+ struct iwl_mei *mei = mei_cldev_get_drvdata(cldev);
+
+ mei->got_ownership = dw->val != cpu_to_le32(SAP_NIC_OWNER_ME);
+}
+
+static void iwl_mei_handle_can_release_ownership(struct mei_cl_device *cldev,
+ const void *payload)
+{
+ /* We can get ownership and driver is registered, go ahead */
+ if (iwl_mei_cache.ops)
+ iwl_mei_send_sap_msg(cldev,
+ SAP_MSG_NOTIF_HOST_ASKS_FOR_NIC_OWNERSHIP);
+}
+
+static void iwl_mei_handle_csme_taking_ownership(struct mei_cl_device *cldev,
+ const void *payload)
+{
+ struct iwl_mei *mei = mei_cldev_get_drvdata(cldev);
+
+ dev_info(&cldev->dev, "CSME takes ownership\n");
+
+ mei->got_ownership = false;
+
+ /*
+ * Remember to send CSME_OWNERSHIP_CONFIRMED when the wifi driver
+ * is finished taking the device down.
+ */
+ mei->csme_taking_ownership = true;
+
+ if (iwl_mei_cache.ops)
+ iwl_mei_cache.ops->rfkill(iwl_mei_cache.priv, true);
+}
+
+static void iwl_mei_handle_nvm(struct mei_cl_device *cldev,
+ const struct iwl_sap_nvm *sap_nvm)
+{
+ struct iwl_mei *mei = mei_cldev_get_drvdata(cldev);
+ const struct iwl_mei_nvm *mei_nvm = (const void *)sap_nvm;
+ int i;
+
+ kfree(mei->nvm);
+ mei->nvm = kzalloc(sizeof(*mei_nvm), GFP_KERNEL);
+ if (!mei->nvm)
+ return;
+
+ ether_addr_copy(mei->nvm->hw_addr, sap_nvm->hw_addr);
+ mei->nvm->n_hw_addrs = sap_nvm->n_hw_addrs;
+ mei->nvm->radio_cfg = le32_to_cpu(sap_nvm->radio_cfg);
+ mei->nvm->caps = le32_to_cpu(sap_nvm->caps);
+ mei->nvm->nvm_version = le32_to_cpu(sap_nvm->nvm_version);
+
+ for (i = 0; i < ARRAY_SIZE(mei->nvm->channels); i++)
+ mei->nvm->channels[i] = le32_to_cpu(sap_nvm->channels[i]);
+
+ wake_up_all(&mei->get_nvm_wq);
+}
+
+static void iwl_mei_handle_rx_host_own_req(struct mei_cl_device *cldev,
+ const struct iwl_sap_msg_dw *dw)
+{
+ struct iwl_mei *mei = mei_cldev_get_drvdata(cldev);
+
+ /*
+ * This means that we can't use the wifi device right now, CSME is not
+ * ready to let us use it.
+ */
+ if (!dw->val) {
+ dev_info(&cldev->dev, "Ownership req denied\n");
+ return;
+ }
+
+ mei->got_ownership = true;
+ wake_up_all(&mei->get_ownership_wq);
+
+ iwl_mei_send_sap_msg(cldev,
+ SAP_MSG_NOTIF_HOST_OWNERSHIP_CONFIRMED);
+
+ /* We can now start the connection, unblock rfkill */
+ if (iwl_mei_cache.ops)
+ iwl_mei_cache.ops->rfkill(iwl_mei_cache.priv, false);
+}
+
+static void iwl_mei_handle_ping(struct mei_cl_device *cldev,
+ const struct iwl_sap_hdr *hdr)
+{
+ iwl_mei_send_sap_msg(cldev, SAP_MSG_NOTIF_PONG);
+}
+
+static void iwl_mei_handle_sap_msg(struct mei_cl_device *cldev,
+ const struct iwl_sap_hdr *hdr)
+{
+ u16 len = le16_to_cpu(hdr->len) + sizeof(*hdr);
+ u16 type = le16_to_cpu(hdr->type);
+
+ dev_dbg(&cldev->dev,
+ "Got a new SAP message: type %d, len %d, seq %d\n",
+ le16_to_cpu(hdr->type), len,
+ le32_to_cpu(hdr->seq_num));
+
+#define SAP_MSG_HANDLER(_cmd, _handler, _sz) \
+ case SAP_MSG_NOTIF_ ## _cmd: \
+ if (len < _sz) { \
+ dev_err(&cldev->dev, \
+ "Bad size for %d: %u < %u\n", \
+ le16_to_cpu(hdr->type), \
+ (unsigned int)len, \
+ (unsigned int)_sz); \
+ break; \
+ } \
+ mutex_lock(&iwl_mei_mutex); \
+ _handler(cldev, (const void *)hdr); \
+ mutex_unlock(&iwl_mei_mutex); \
+ break
+
+#define SAP_MSG_HANDLER_NO_LOCK(_cmd, _handler, _sz) \
+ case SAP_MSG_NOTIF_ ## _cmd: \
+ if (len < _sz) { \
+ dev_err(&cldev->dev, \
+ "Bad size for %d: %u < %u\n", \
+ le16_to_cpu(hdr->type), \
+ (unsigned int)len, \
+ (unsigned int)_sz); \
+ break; \
+ } \
+ _handler(cldev, (const void *)hdr); \
+ break
+
+#define SAP_MSG_HANDLER_NO_HANDLER(_cmd, _sz) \
+ case SAP_MSG_NOTIF_ ## _cmd: \
+ if (len < _sz) { \
+ dev_err(&cldev->dev, \
+ "Bad size for %d: %u < %u\n", \
+ le16_to_cpu(hdr->type), \
+ (unsigned int)len, \
+ (unsigned int)_sz); \
+ break; \
+ } \
+ break
+
+ switch (type) {
+ SAP_MSG_HANDLER(PING, iwl_mei_handle_ping, 0);
+ SAP_MSG_HANDLER(CSME_FILTERS,
+ iwl_mei_handle_csme_filters,
+ sizeof(struct iwl_sap_csme_filters));
+ SAP_MSG_HANDLER(CSME_CONN_STATUS,
+ iwl_mei_handle_conn_status,
+ sizeof(struct iwl_sap_notif_conn_status));
+ SAP_MSG_HANDLER_NO_LOCK(AMT_STATE,
+ iwl_mei_handle_amt_state,
+ sizeof(struct iwl_sap_msg_dw));
+ SAP_MSG_HANDLER_NO_HANDLER(PONG, 0);
+ SAP_MSG_HANDLER(NVM, iwl_mei_handle_nvm,
+ sizeof(struct iwl_sap_nvm));
+ SAP_MSG_HANDLER(CSME_REPLY_TO_HOST_OWNERSHIP_REQ,
+ iwl_mei_handle_rx_host_own_req,
+ sizeof(struct iwl_sap_msg_dw));
+ SAP_MSG_HANDLER(NIC_OWNER, iwl_mei_handle_nic_owner,
+ sizeof(struct iwl_sap_msg_dw));
+ SAP_MSG_HANDLER(CSME_CAN_RELEASE_OWNERSHIP,
+ iwl_mei_handle_can_release_ownership, 0);
+ SAP_MSG_HANDLER(CSME_TAKING_OWNERSHIP,
+ iwl_mei_handle_csme_taking_ownership, 0);
+ default:
+ /*
+ * This is not really an error, there are message that we decided
+ * to ignore, yet, it is useful to be able to leave a note if debug
+ * is enabled.
+ */
+ dev_dbg(&cldev->dev, "Unsupported message: type %d, len %d\n",
+ le16_to_cpu(hdr->type), len);
+ }
+
+#undef SAP_MSG_HANDLER
+#undef SAP_MSG_HANDLER_NO_LOCK
+}
+
+static void iwl_mei_read_from_q(const u8 *q_head, u32 q_sz,
+ u32 *_rd, u32 wr,
+ void *_buf, u32 len)
+{
+ u8 *buf = _buf;
+ u32 rd = *_rd;
+
+ if (rd + len <= q_sz) {
+ memcpy(buf, q_head + rd, len);
+ rd += len;
+ } else {
+ memcpy(buf, q_head + rd, q_sz - rd);
+ memcpy(buf + q_sz - rd, q_head, len - (q_sz - rd));
+ rd = len - (q_sz - rd);
+ }
+
+ *_rd = rd;
+}
+
+#define QOS_HDR_IV_SNAP_LEN (sizeof(struct ieee80211_qos_hdr) + \
+ IEEE80211_TKIP_IV_LEN + \
+ sizeof(rfc1042_header) + ETH_TLEN)
+
+static void iwl_mei_handle_sap_data(struct mei_cl_device *cldev,
+ const u8 *q_head, u32 q_sz,
+ u32 rd, u32 wr, ssize_t valid_rx_sz,
+ struct sk_buff_head *tx_skbs)
+{
+ struct iwl_sap_hdr hdr;
+ struct net_device *netdev =
+ rcu_dereference_protected(iwl_mei_cache.netdev,
+ lockdep_is_held(&iwl_mei_mutex));
+
+ if (!netdev)
+ return;
+
+ while (valid_rx_sz >= sizeof(hdr)) {
+ struct ethhdr *ethhdr;
+ unsigned char *data;
+ struct sk_buff *skb;
+ u16 len;
+
+ iwl_mei_read_from_q(q_head, q_sz, &rd, wr, &hdr, sizeof(hdr));
+ valid_rx_sz -= sizeof(hdr);
+ len = le16_to_cpu(hdr.len);
+
+ if (valid_rx_sz < len) {
+ dev_err(&cldev->dev,
+ "Data queue is corrupted: valid data len %zd, len %d\n",
+ valid_rx_sz, len);
+ break;
+ }
+
+ if (len < sizeof(*ethhdr)) {
+ dev_err(&cldev->dev,
+ "Data len is smaller than an ethernet header? len = %d\n",
+ len);
+ }
+
+ valid_rx_sz -= len;
+
+ if (le16_to_cpu(hdr.type) != SAP_MSG_DATA_PACKET) {
+ dev_err(&cldev->dev, "Unsupported Rx data: type %d, len %d\n",
+ le16_to_cpu(hdr.type), len);
+ continue;
+ }
+
+ /* We need enough room for the WiFi header + SNAP + IV */
+ skb = netdev_alloc_skb(netdev, len + QOS_HDR_IV_SNAP_LEN);
+
+ skb_reserve(skb, QOS_HDR_IV_SNAP_LEN);
+ ethhdr = skb_push(skb, sizeof(*ethhdr));
+
+ iwl_mei_read_from_q(q_head, q_sz, &rd, wr,
+ ethhdr, sizeof(*ethhdr));
+ len -= sizeof(*ethhdr);
+
+ skb_reset_mac_header(skb);
+ skb_reset_network_header(skb);
+ skb->protocol = ethhdr->h_proto;
+
+ data = skb_put(skb, len);
+ iwl_mei_read_from_q(q_head, q_sz, &rd, wr, data, len);
+
+ /*
+ * Enqueue the skb here so that it can be sent later when we
+ * do not hold the mutex. TX'ing a packet with a mutex held is
+ * possible, but it wouldn't be nice to forbid the TX path to
+ * call any of iwlmei's functions, since every API from iwlmei
+ * needs the mutex.
+ */
+ __skb_queue_tail(tx_skbs, skb);
+ }
+}
+
+static void iwl_mei_handle_sap_rx_cmd(struct mei_cl_device *cldev,
+ const u8 *q_head, u32 q_sz,
+ u32 rd, u32 wr, ssize_t valid_rx_sz)
+{
+ struct page *p = alloc_page(GFP_KERNEL);
+ struct iwl_sap_hdr *hdr;
+
+ if (!p)
+ return;
+
+ hdr = page_address(p);
+
+ while (valid_rx_sz >= sizeof(*hdr)) {
+ u16 len;
+
+ iwl_mei_read_from_q(q_head, q_sz, &rd, wr, hdr, sizeof(*hdr));
+ valid_rx_sz -= sizeof(*hdr);
+ len = le16_to_cpu(hdr->len);
+
+ if (valid_rx_sz < len)
+ break;
+
+ iwl_mei_read_from_q(q_head, q_sz, &rd, wr, hdr + 1, len);
+
+ trace_iwlmei_sap_cmd(hdr, false);
+ iwl_mei_handle_sap_msg(cldev, hdr);
+ valid_rx_sz -= len;
+ }
+
+ /* valid_rx_sz must be 0 now... */
+ if (valid_rx_sz)
+ dev_err(&cldev->dev,
+ "More data in the buffer although we read it all\n");
+
+ __free_page(p);
+}
+
+static void iwl_mei_handle_sap_rx(struct mei_cl_device *cldev,
+ struct iwl_sap_q_ctrl_blk *notif_q,
+ const u8 *q_head,
+ struct sk_buff_head *skbs,
+ u32 q_sz)
+{
+ u32 rd = le32_to_cpu(READ_ONCE(notif_q->rd_ptr));
+ u32 wr = le32_to_cpu(READ_ONCE(notif_q->wr_ptr));
+ ssize_t valid_rx_sz;
+
+ if (rd > q_sz || wr > q_sz) {
+ dev_err(&cldev->dev,
+ "Pointers are past the buffer limit\n");
+ return;
+ }
+
+ if (rd == wr)
+ return;
+
+ valid_rx_sz = wr > rd ? wr - rd : q_sz - rd + wr;
+
+ if (skbs)
+ iwl_mei_handle_sap_data(cldev, q_head, q_sz, rd, wr,
+ valid_rx_sz, skbs);
+ else
+ iwl_mei_handle_sap_rx_cmd(cldev, q_head, q_sz, rd, wr,
+ valid_rx_sz);
+
+ /* Increment the read pointer to point to the write pointer */
+ WRITE_ONCE(notif_q->rd_ptr, cpu_to_le32(wr));
+}
+
+static void iwl_mei_handle_check_shared_area(struct mei_cl_device *cldev)
+{
+ struct iwl_mei *mei = mei_cldev_get_drvdata(cldev);
+ struct iwl_sap_q_ctrl_blk *notif_q;
+ struct sk_buff_head tx_skbs;
+ struct iwl_sap_dir *dir;
+ void *q_head;
+ u32 q_sz;
+
+ if (!mei->shared_mem.ctrl)
+ return;
+
+ dir = &mei->shared_mem.ctrl->dir[SAP_DIRECTION_ME_TO_HOST];
+ notif_q = &dir->q_ctrl_blk[SAP_QUEUE_IDX_NOTIF];
+ q_head = mei->shared_mem.q_head[SAP_DIRECTION_ME_TO_HOST][SAP_QUEUE_IDX_NOTIF];
+ q_sz = mei->shared_mem.q_size[SAP_DIRECTION_ME_TO_HOST][SAP_QUEUE_IDX_NOTIF];
+
+ /*
+ * Do not hold the mutex here, but rather each and every message
+ * handler takes it.
+ * This allows message handlers to take it at a certain time.
+ */
+ iwl_mei_handle_sap_rx(cldev, notif_q, q_head, NULL, q_sz);
+
+ mutex_lock(&iwl_mei_mutex);
+ dir = &mei->shared_mem.ctrl->dir[SAP_DIRECTION_ME_TO_HOST];
+ notif_q = &dir->q_ctrl_blk[SAP_QUEUE_IDX_DATA];
+ q_head = mei->shared_mem.q_head[SAP_DIRECTION_ME_TO_HOST][SAP_QUEUE_IDX_DATA];
+ q_sz = mei->shared_mem.q_size[SAP_DIRECTION_ME_TO_HOST][SAP_QUEUE_IDX_DATA];
+
+ __skb_queue_head_init(&tx_skbs);
+
+ iwl_mei_handle_sap_rx(cldev, notif_q, q_head, &tx_skbs, q_sz);
+
+ if (skb_queue_empty(&tx_skbs)) {
+ mutex_unlock(&iwl_mei_mutex);
+ return;
+ }
+
+ /*
+ * Take the RCU read lock before we unlock the mutex to make sure that
+ * even if the netdev is replaced by another non-NULL netdev right after
+ * we unlock the mutex, the old netdev will still be valid when we
+ * transmit the frames. We can't allow to replace the netdev here because
+ * the skbs hold a pointer to the netdev.
+ */
+ rcu_read_lock();
+
+ mutex_unlock(&iwl_mei_mutex);
+
+ if (!rcu_access_pointer(iwl_mei_cache.netdev)) {
+ dev_err(&cldev->dev, "Can't Tx without a netdev\n");
+ skb_queue_purge(&tx_skbs);
+ goto out;
+ }
+
+ while (!skb_queue_empty(&tx_skbs)) {
+ struct sk_buff *skb = __skb_dequeue(&tx_skbs);
+
+ trace_iwlmei_sap_data(skb, IWL_SAP_RX_DATA_TO_AIR);
+ dev_queue_xmit(skb);
+ }
+
+out:
+ rcu_read_unlock();
+}
+
+static void iwl_mei_rx(struct mei_cl_device *cldev)
+{
+ struct iwl_sap_me_msg_hdr *hdr;
+ u8 msg[100];
+ ssize_t ret;
+
+ ret = mei_cldev_recv(cldev, (u8 *)&msg, sizeof(msg));
+ if (ret < 0) {
+ dev_err(&cldev->dev, "failed to receive data: %zd\n", ret);
+ return;
+ }
+
+ if (ret == 0) {
+ dev_err(&cldev->dev, "got an empty response\n");
+ return;
+ }
+
+ hdr = (void *)msg;
+ trace_iwlmei_me_msg(hdr, false);
+
+ switch (le32_to_cpu(hdr->type)) {
+ case SAP_ME_MSG_START_OK:
+ BUILD_BUG_ON(sizeof(struct iwl_sap_me_msg_start_ok) >
+ sizeof(msg));
+
+ iwl_mei_handle_rx_start_ok(cldev, (void *)msg, ret);
+ break;
+ case SAP_ME_MSG_CHECK_SHARED_AREA:
+ iwl_mei_handle_check_shared_area(cldev);
+ break;
+ default:
+ dev_err(&cldev->dev, "got a RX notification: %d\n",
+ le32_to_cpu(hdr->type));
+ break;
+ }
+}
+
+static int iwl_mei_send_start(struct mei_cl_device *cldev)
+{
+ struct iwl_mei *mei = mei_cldev_get_drvdata(cldev);
+ struct iwl_sap_me_msg_start msg = {
+ .hdr.type = cpu_to_le32(SAP_ME_MSG_START),
+ .hdr.seq_num = cpu_to_le32(atomic_inc_return(&mei->seq_no)),
+ .hdr.len = cpu_to_le32(sizeof(msg)),
+ .supported_versions[0] = SAP_VERSION,
+ .init_data_seq_num = cpu_to_le16(0x100),
+ .init_notif_seq_num = cpu_to_le16(0x800),
+ };
+ int ret;
+
+ trace_iwlmei_me_msg(&msg.hdr, true);
+ ret = mei_cldev_send(cldev, (void *)&msg, sizeof(msg));
+ if (ret != sizeof(msg)) {
+ dev_err(&cldev->dev,
+ "failed to send the SAP_ME_MSG_START message %d\n",
+ ret);
+ return ret;
+ }
+
+ return 0;
+}
+
+static int iwl_mei_enable(struct mei_cl_device *cldev)
+{
+ int ret;
+
+ ret = mei_cldev_enable(cldev);
+ if (ret < 0) {
+ dev_err(&cldev->dev, "failed to enable the device: %d\n", ret);
+ return ret;
+ }
+
+ ret = mei_cldev_register_rx_cb(cldev, iwl_mei_rx);
+ if (ret) {
+ dev_err(&cldev->dev,
+ "failed to register to the rx cb: %d\n", ret);
+ mei_cldev_disable(cldev);
+ return ret;
+ }
+
+ return 0;
+}
+
+struct iwl_mei_nvm *iwl_mei_get_nvm(void)
+{
+ struct iwl_mei_nvm *nvm = NULL;
+ struct iwl_mei *mei;
+ int ret;
+
+ mutex_lock(&iwl_mei_mutex);
+
+ if (!iwl_mei_is_connected())
+ goto out;
+
+ mei = mei_cldev_get_drvdata(iwl_mei_global_cldev);
+
+ if (!mei)
+ goto out;
+
+ ret = iwl_mei_send_sap_msg(iwl_mei_global_cldev,
+ SAP_MSG_NOTIF_GET_NVM);
+ if (ret)
+ goto out;
+
+ mutex_unlock(&iwl_mei_mutex);
+
+ ret = wait_event_timeout(mei->get_nvm_wq, mei->nvm, 2 * HZ);
+ if (!ret)
+ return NULL;
+
+ mutex_lock(&iwl_mei_mutex);
+
+ if (!iwl_mei_is_connected())
+ goto out;
+
+ mei = mei_cldev_get_drvdata(iwl_mei_global_cldev);
+
+ if (!mei)
+ goto out;
+
+ if (mei->nvm)
+ nvm = kmemdup(mei->nvm, sizeof(*mei->nvm), GFP_KERNEL);
+
+out:
+ mutex_unlock(&iwl_mei_mutex);
+ return nvm;
+}
+EXPORT_SYMBOL_GPL(iwl_mei_get_nvm);
+
+int iwl_mei_get_ownership(void)
+{
+ struct iwl_mei *mei;
+ int ret;
+
+ mutex_lock(&iwl_mei_mutex);
+
+ /* In case we didn't have a bind */
+ if (!iwl_mei_is_connected()) {
+ ret = 0;
+ goto out;
+ }
+
+ mei = mei_cldev_get_drvdata(iwl_mei_global_cldev);
+
+ if (!mei) {
+ ret = -ENODEV;
+ goto out;
+ }
+
+ if (!mei->amt_enabled) {
+ ret = 0;
+ goto out;
+ }
+
+ if (mei->got_ownership) {
+ ret = 0;
+ goto out;
+ }
+
+ ret = iwl_mei_send_sap_msg(mei->cldev,
+ SAP_MSG_NOTIF_HOST_ASKS_FOR_NIC_OWNERSHIP);
+ if (ret)
+ goto out;
+
+ mutex_unlock(&iwl_mei_mutex);
+
+ ret = wait_event_timeout(mei->get_ownership_wq,
+ mei->got_ownership, HZ / 2);
+ if (!ret)
+ return -ETIMEDOUT;
+
+ mutex_lock(&iwl_mei_mutex);
+
+ /* In case we didn't have a bind */
+ if (!iwl_mei_is_connected()) {
+ ret = 0;
+ goto out;
+ }
+
+ mei = mei_cldev_get_drvdata(iwl_mei_global_cldev);
+
+ if (!mei) {
+ ret = -ENODEV;
+ goto out;
+ }
+
+ ret = !mei->got_ownership;
+
+out:
+ mutex_unlock(&iwl_mei_mutex);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(iwl_mei_get_ownership);
+
+void iwl_mei_host_associated(const struct iwl_mei_conn_info *conn_info,
+ const struct iwl_mei_colloc_info *colloc_info)
+{
+ struct iwl_sap_notif_host_link_up msg = {
+ .hdr.type = cpu_to_le16(SAP_MSG_NOTIF_HOST_LINK_UP),
+ .hdr.len = cpu_to_le16(sizeof(msg) - sizeof(msg.hdr)),
+ .conn_info = {
+ .ssid_len = cpu_to_le32(conn_info->ssid_len),
+ .channel = conn_info->channel,
+ .band = conn_info->band,
+ .pairwise_cipher = cpu_to_le32(conn_info->pairwise_cipher),
+ .auth_mode = cpu_to_le32(conn_info->auth_mode),
+ },
+ };
+ struct iwl_mei *mei;
+
+ if (conn_info->ssid_len > ARRAY_SIZE(msg.conn_info.ssid))
+ return;
+
+ memcpy(msg.conn_info.ssid, conn_info->ssid, conn_info->ssid_len);
+ memcpy(msg.conn_info.bssid, conn_info->bssid, ETH_ALEN);
+
+ if (colloc_info) {
+ msg.colloc_channel = colloc_info->channel;
+ msg.colloc_band = colloc_info->channel <= 14 ? 0 : 1;
+ memcpy(msg.colloc_bssid, colloc_info->bssid, ETH_ALEN);
+ }
+
+ mutex_lock(&iwl_mei_mutex);
+
+ if (!iwl_mei_is_connected())
+ goto out;
+
+ mei = mei_cldev_get_drvdata(iwl_mei_global_cldev);
+
+ if (!mei)
+ goto out;
+
+ if (!mei->amt_enabled)
+ goto out;
+
+ iwl_mei_send_sap_msg_payload(mei->cldev, &msg.hdr);
+
+out:
+ kfree(iwl_mei_cache.conn_info);
+ iwl_mei_cache.conn_info =
+ kmemdup(&msg.conn_info, sizeof(msg.conn_info), GFP_KERNEL);
+ mutex_unlock(&iwl_mei_mutex);
+}
+EXPORT_SYMBOL_GPL(iwl_mei_host_associated);
+
+void iwl_mei_host_disassociated(void)
+{
+ struct iwl_mei *mei;
+ struct iwl_sap_notif_host_link_down msg = {
+ .hdr.type = cpu_to_le16(SAP_MSG_NOTIF_HOST_LINK_DOWN),
+ .hdr.len = cpu_to_le16(sizeof(msg) - sizeof(msg.hdr)),
+ .type = HOST_LINK_DOWN_TYPE_LONG,
+ };
+
+ mutex_lock(&iwl_mei_mutex);
+
+ if (!iwl_mei_is_connected())
+ goto out;
+
+ mei = mei_cldev_get_drvdata(iwl_mei_global_cldev);
+
+ if (!mei)
+ goto out;
+
+ iwl_mei_send_sap_msg_payload(mei->cldev, &msg.hdr);
+
+out:
+ kfree(iwl_mei_cache.conn_info);
+ iwl_mei_cache.conn_info = NULL;
+ mutex_unlock(&iwl_mei_mutex);
+}
+EXPORT_SYMBOL_GPL(iwl_mei_host_disassociated);
+
+void iwl_mei_set_rfkill_state(bool hw_rfkill, bool sw_rfkill)
+{
+ struct iwl_mei *mei;
+ u32 rfkill_state = 0;
+ struct iwl_sap_msg_dw msg = {
+ .hdr.type = cpu_to_le16(SAP_MSG_NOTIF_RADIO_STATE),
+ .hdr.len = cpu_to_le16(sizeof(msg) - sizeof(msg.hdr)),
+ };
+
+ if (!sw_rfkill)
+ rfkill_state |= SAP_SW_RFKILL_DEASSERTED;
+
+ if (!hw_rfkill)
+ rfkill_state |= SAP_HW_RFKILL_DEASSERTED;
+
+ mutex_lock(&iwl_mei_mutex);
+
+ if (!iwl_mei_is_connected())
+ goto out;
+
+ msg.val = cpu_to_le32(rfkill_state);
+
+ mei = mei_cldev_get_drvdata(iwl_mei_global_cldev);
+
+ if (!mei)
+ goto out;
+
+ iwl_mei_send_sap_msg_payload(mei->cldev, &msg.hdr);
+
+out:
+ iwl_mei_cache.rf_kill = rfkill_state;
+ mutex_unlock(&iwl_mei_mutex);
+}
+EXPORT_SYMBOL_GPL(iwl_mei_set_rfkill_state);
+
+void iwl_mei_set_nic_info(const u8 *mac_address, const u8 *nvm_address)
+{
+ struct iwl_mei *mei;
+ struct iwl_sap_notif_host_nic_info msg = {
+ .hdr.type = cpu_to_le16(SAP_MSG_NOTIF_NIC_INFO),
+ .hdr.len = cpu_to_le16(sizeof(msg) - sizeof(msg.hdr)),
+ };
+
+ mutex_lock(&iwl_mei_mutex);
+
+ if (!iwl_mei_is_connected())
+ goto out;
+
+ ether_addr_copy(msg.mac_address, mac_address);
+ ether_addr_copy(msg.nvm_address, nvm_address);
+
+ mei = mei_cldev_get_drvdata(iwl_mei_global_cldev);
+
+ if (!mei)
+ goto out;
+
+ iwl_mei_send_sap_msg_payload(mei->cldev, &msg.hdr);
+
+out:
+ ether_addr_copy(iwl_mei_cache.mac_address, mac_address);
+ ether_addr_copy(iwl_mei_cache.nvm_address, nvm_address);
+ mutex_unlock(&iwl_mei_mutex);
+}
+EXPORT_SYMBOL_GPL(iwl_mei_set_nic_info);
+
+void iwl_mei_set_country_code(u16 mcc)
+{
+ struct iwl_mei *mei;
+ struct iwl_sap_notif_country_code msg = {
+ .hdr.type = cpu_to_le16(SAP_MSG_NOTIF_COUNTRY_CODE),
+ .hdr.len = cpu_to_le16(sizeof(msg) - sizeof(msg.hdr)),
+ .mcc = cpu_to_le16(mcc),
+ };
+
+ mutex_lock(&iwl_mei_mutex);
+
+ if (!iwl_mei_is_connected())
+ goto out;
+
+ mei = mei_cldev_get_drvdata(iwl_mei_global_cldev);
+
+ if (!mei)
+ goto out;
+
+ iwl_mei_send_sap_msg_payload(mei->cldev, &msg.hdr);
+
+out:
+ iwl_mei_cache.mcc = mcc;
+ mutex_unlock(&iwl_mei_mutex);
+}
+EXPORT_SYMBOL_GPL(iwl_mei_set_country_code);
+
+void iwl_mei_set_power_limit(const __le16 *power_limit)
+{
+ struct iwl_mei *mei;
+ struct iwl_sap_notif_sar_limits msg = {
+ .hdr.type = cpu_to_le16(SAP_MSG_NOTIF_SAR_LIMITS),
+ .hdr.len = cpu_to_le16(sizeof(msg) - sizeof(msg.hdr)),
+ };
+
+ mutex_lock(&iwl_mei_mutex);
+
+ if (!iwl_mei_is_connected())
+ goto out;
+
+ mei = mei_cldev_get_drvdata(iwl_mei_global_cldev);
+
+ if (!mei)
+ goto out;
+
+ memcpy(msg.sar_chain_info_table, power_limit, sizeof(msg.sar_chain_info_table));
+
+ iwl_mei_send_sap_msg_payload(mei->cldev, &msg.hdr);
+
+out:
+ kfree(iwl_mei_cache.power_limit);
+ iwl_mei_cache.power_limit = kmemdup(power_limit,
+ sizeof(msg.sar_chain_info_table), GFP_KERNEL);
+ mutex_unlock(&iwl_mei_mutex);
+}
+EXPORT_SYMBOL_GPL(iwl_mei_set_power_limit);
+
+void iwl_mei_set_netdev(struct net_device *netdev)
+{
+ struct iwl_mei *mei;
+
+ mutex_lock(&iwl_mei_mutex);
+
+ if (!iwl_mei_is_connected()) {
+ rcu_assign_pointer(iwl_mei_cache.netdev, netdev);
+ goto out;
+ }
+
+ mei = mei_cldev_get_drvdata(iwl_mei_global_cldev);
+
+ if (!mei)
+ goto out;
+
+ if (!netdev) {
+ struct net_device *dev =
+ rcu_dereference_protected(iwl_mei_cache.netdev,
+ lockdep_is_held(&iwl_mei_mutex));
+
+ if (!dev)
+ goto out;
+
+ netdev_rx_handler_unregister(dev);
+ }
+
+ rcu_assign_pointer(iwl_mei_cache.netdev, netdev);
+
+ if (netdev && mei->amt_enabled)
+ netdev_rx_handler_register(netdev, iwl_mei_rx_handler, mei);
+
+out:
+ mutex_unlock(&iwl_mei_mutex);
+}
+EXPORT_SYMBOL_GPL(iwl_mei_set_netdev);
+
+void iwl_mei_device_down(void)
+{
+ struct iwl_mei *mei;
+
+ mutex_lock(&iwl_mei_mutex);
+
+ if (!iwl_mei_is_connected())
+ goto out;
+
+ mei = mei_cldev_get_drvdata(iwl_mei_global_cldev);
+
+ if (!mei)
+ goto out;
+
+ if (!mei->csme_taking_ownership)
+ goto out;
+
+ iwl_mei_send_sap_msg(mei->cldev,
+ SAP_MSG_NOTIF_CSME_OWNERSHIP_CONFIRMED);
+ mei->csme_taking_ownership = false;
+out:
+ mutex_unlock(&iwl_mei_mutex);
+}
+EXPORT_SYMBOL_GPL(iwl_mei_device_down);
+
+int iwl_mei_register(void *priv, const struct iwl_mei_ops *ops)
+{
+ int ret;
+
+ /*
+ * We must have a non-NULL priv pointer to not crash when there are
+ * multiple WiFi devices.
+ */
+ if (!priv)
+ return -EINVAL;
+
+ mutex_lock(&iwl_mei_mutex);
+
+ /* do not allow registration if someone else already registered */
+ if (iwl_mei_cache.priv || iwl_mei_cache.ops) {
+ ret = -EBUSY;
+ goto out;
+ }
+
+ iwl_mei_cache.priv = priv;
+ iwl_mei_cache.ops = ops;
+
+ if (iwl_mei_global_cldev) {
+ struct iwl_mei *mei =
+ mei_cldev_get_drvdata(iwl_mei_global_cldev);
+
+ /* we have already a SAP connection */
+ if (iwl_mei_is_connected())
+ iwl_mei_send_sap_msg(mei->cldev,
+ SAP_MSG_NOTIF_WIFIDR_UP);
+ }
+ ret = 0;
+
+out:
+ mutex_unlock(&iwl_mei_mutex);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(iwl_mei_register);
+
+void iwl_mei_start_unregister(void)
+{
+ mutex_lock(&iwl_mei_mutex);
+
+ /* At this point, the wifi driver should have removed the netdev */
+ if (rcu_access_pointer(iwl_mei_cache.netdev))
+ pr_err("Still had a netdev pointer set upon unregister\n");
+
+ kfree(iwl_mei_cache.conn_info);
+ iwl_mei_cache.conn_info = NULL;
+ kfree(iwl_mei_cache.power_limit);
+ iwl_mei_cache.power_limit = NULL;
+ iwl_mei_cache.ops = NULL;
+ /* leave iwl_mei_cache.priv non-NULL to prevent any new registration */
+
+ mutex_unlock(&iwl_mei_mutex);
+}
+EXPORT_SYMBOL_GPL(iwl_mei_start_unregister);
+
+void iwl_mei_unregister_complete(void)
+{
+ mutex_lock(&iwl_mei_mutex);
+
+ iwl_mei_cache.priv = NULL;
+
+ if (iwl_mei_global_cldev) {
+ struct iwl_mei *mei =
+ mei_cldev_get_drvdata(iwl_mei_global_cldev);
+
+ iwl_mei_send_sap_msg(mei->cldev, SAP_MSG_NOTIF_WIFIDR_DOWN);
+ mei->got_ownership = false;
+ }
+
+ mutex_unlock(&iwl_mei_mutex);
+}
+EXPORT_SYMBOL_GPL(iwl_mei_unregister_complete);
+
+#if IS_ENABLED(CONFIG_DEBUG_FS)
+
+static ssize_t
+iwl_mei_dbgfs_send_start_message_write(struct file *file,
+ const char __user *user_buf,
+ size_t count, loff_t *ppos)
+{
+ int ret;
+
+ mutex_lock(&iwl_mei_mutex);
+
+ if (!iwl_mei_global_cldev) {
+ ret = -ENODEV;
+ goto out;
+ }
+
+ ret = iwl_mei_send_start(iwl_mei_global_cldev);
+
+out:
+ mutex_unlock(&iwl_mei_mutex);
+ return ret ?: count;
+}
+
+static const struct file_operations iwl_mei_dbgfs_send_start_message_ops = {
+ .write = iwl_mei_dbgfs_send_start_message_write,
+ .open = simple_open,
+ .llseek = default_llseek,
+};
+
+static ssize_t iwl_mei_dbgfs_req_ownership_write(struct file *file,
+ const char __user *user_buf,
+ size_t count, loff_t *ppos)
+{
+ iwl_mei_get_ownership();
+
+ return count;
+}
+
+static const struct file_operations iwl_mei_dbgfs_req_ownership_ops = {
+ .write = iwl_mei_dbgfs_req_ownership_write,
+ .open = simple_open,
+ .llseek = default_llseek,
+};
+
+static void iwl_mei_dbgfs_register(struct iwl_mei *mei)
+{
+ mei->dbgfs_dir = debugfs_create_dir(KBUILD_MODNAME, NULL);
+
+ if (!mei->dbgfs_dir)
+ return;
+
+ debugfs_create_ulong("status", S_IRUSR,
+ mei->dbgfs_dir, &iwl_mei_status);
+ debugfs_create_file("send_start_message", S_IWUSR, mei->dbgfs_dir,
+ mei, &iwl_mei_dbgfs_send_start_message_ops);
+ debugfs_create_file("req_ownership", S_IWUSR, mei->dbgfs_dir,
+ mei, &iwl_mei_dbgfs_req_ownership_ops);
+}
+
+static void iwl_mei_dbgfs_unregister(struct iwl_mei *mei)
+{
+ debugfs_remove_recursive(mei->dbgfs_dir);
+ mei->dbgfs_dir = NULL;
+}
+
+#else
+
+static void iwl_mei_dbgfs_register(struct iwl_mei *mei) {}
+static void iwl_mei_dbgfs_unregister(struct iwl_mei *mei) {}
+
+#endif /* CONFIG_DEBUG_FS */
+
+/*
+ * iwl_mei_probe - the probe function called by the mei bus enumeration
+ *
+ * This allocates the data needed by iwlmei and sets a pointer to this data
+ * into the mei_cl_device's drvdata.
+ * It starts the SAP protocol by sending the SAP_ME_MSG_START without
+ * waiting for the answer. The answer will be caught later by the Rx callback.
+ */
+static int iwl_mei_probe(struct mei_cl_device *cldev,
+ const struct mei_cl_device_id *id)
+{
+ struct iwl_mei *mei;
+ int ret;
+
+ mei = devm_kzalloc(&cldev->dev, sizeof(*mei), GFP_KERNEL);
+ if (!mei)
+ return -ENOMEM;
+
+ init_waitqueue_head(&mei->get_nvm_wq);
+ INIT_WORK(&mei->send_csa_msg_wk, iwl_mei_send_csa_msg_wk);
+ INIT_DELAYED_WORK(&mei->csa_throttle_end_wk,
+ iwl_mei_csa_throttle_end_wk);
+ init_waitqueue_head(&mei->get_ownership_wq);
+ spin_lock_init(&mei->data_q_lock);
+
+ mei_cldev_set_drvdata(cldev, mei);
+ mei->cldev = cldev;
+
+ /*
+ * The CSME firmware needs to boot the internal WLAN client. Wait here
+ * so that the DMA map request will succeed.
+ */
+ msleep(20);
+
+ ret = iwl_mei_alloc_shared_mem(cldev);
+ if (ret)
+ goto free;
+
+ iwl_mei_init_shared_mem(mei);
+
+ ret = iwl_mei_enable(cldev);
+ if (ret)
+ goto free_shared_mem;
+
+ iwl_mei_dbgfs_register(mei);
+
+ /*
+ * We now have a Rx function in place, start the SAP procotol
+ * we expect to get the SAP_ME_MSG_START_OK response later on.
+ */
+ mutex_lock(&iwl_mei_mutex);
+ ret = iwl_mei_send_start(cldev);
+ mutex_unlock(&iwl_mei_mutex);
+ if (ret)
+ goto debugfs_unregister;
+
+ /* must be last */
+ iwl_mei_global_cldev = cldev;
+
+ return 0;
+
+debugfs_unregister:
+ iwl_mei_dbgfs_unregister(mei);
+ mei_cldev_disable(cldev);
+free_shared_mem:
+ iwl_mei_free_shared_mem(cldev);
+free:
+ mei_cldev_set_drvdata(cldev, NULL);
+ devm_kfree(&cldev->dev, mei);
+
+ return ret;
+}
+
+#define SEND_SAP_MAX_WAIT_ITERATION 10
+
+static void iwl_mei_remove(struct mei_cl_device *cldev)
+{
+ struct iwl_mei *mei = mei_cldev_get_drvdata(cldev);
+ int i;
+
+ /*
+ * We are being removed while the bus is active, it means we are
+ * going to suspend/ shutdown, so the NIC will disappear.
+ */
+ if (mei_cldev_enabled(cldev) && iwl_mei_cache.ops)
+ iwl_mei_cache.ops->nic_stolen(iwl_mei_cache.priv);
+
+ if (rcu_access_pointer(iwl_mei_cache.netdev)) {
+ struct net_device *dev;
+
+ /*
+ * First take rtnl and only then the mutex to avoid an ABBA
+ * with iwl_mei_set_netdev()
+ */
+ rtnl_lock();
+ mutex_lock(&iwl_mei_mutex);
+
+ /*
+ * If we are suspending and the wifi driver hasn't removed it's netdev
+ * yet, do it now. In any case, don't change the cache.netdev pointer.
+ */
+ dev = rcu_dereference_protected(iwl_mei_cache.netdev,
+ lockdep_is_held(&iwl_mei_mutex));
+
+ netdev_rx_handler_unregister(dev);
+ mutex_unlock(&iwl_mei_mutex);
+ rtnl_unlock();
+ }
+
+ mutex_lock(&iwl_mei_mutex);
+
+ /*
+ * Tell CSME that we are going down so that it won't access the
+ * memory anymore, make sure this message goes through immediately.
+ */
+ mei->csa_throttled = false;
+ iwl_mei_send_sap_msg(mei->cldev,
+ SAP_MSG_NOTIF_HOST_GOES_DOWN);
+
+ for (i = 0; i < SEND_SAP_MAX_WAIT_ITERATION; i++) {
+ if (!iwl_mei_host_to_me_data_pending(mei))
+ break;
+
+ msleep(5);
+ }
+
+ /*
+ * If we couldn't make sure that CSME saw the HOST_GOES_DOWN message,
+ * it means that it will probably keep reading memory that we are going
+ * to unmap and free, expect IOMMU error messages.
+ */
+ if (i == SEND_SAP_MAX_WAIT_ITERATION)
+ dev_err(&mei->cldev->dev,
+ "Couldn't get ACK from CSME on HOST_GOES_DOWN message\n");
+
+ mutex_unlock(&iwl_mei_mutex);
+
+ /*
+ * This looks strange, but this lock is taken here to make sure that
+ * iwl_mei_add_data_to_ring called from the Tx path sees that we
+ * clear the IWL_MEI_STATUS_SAP_CONNECTED bit.
+ * Rx isn't a problem because the rx_handler can't be called after
+ * having been unregistered.
+ */
+ spin_lock_bh(&mei->data_q_lock);
+ clear_bit(IWL_MEI_STATUS_SAP_CONNECTED, &iwl_mei_status);
+ spin_unlock_bh(&mei->data_q_lock);
+
+ if (iwl_mei_cache.ops)
+ iwl_mei_cache.ops->rfkill(iwl_mei_cache.priv, false);
+
+ /*
+ * mei_cldev_disable will return only after all the MEI Rx is done.
+ * It must be called when iwl_mei_mutex is *not* held, since it waits
+ * for our Rx handler to complete.
+ * After it returns, no new Rx will start.
+ */
+ mei_cldev_disable(cldev);
+
+ /*
+ * Since the netdev was already removed and the netdev's removal
+ * includes a call to synchronize_net() so that we know there won't be
+ * any new Rx that will trigger the following workers.
+ */
+ cancel_work_sync(&mei->send_csa_msg_wk);
+ cancel_delayed_work_sync(&mei->csa_throttle_end_wk);
+
+ /*
+ * If someone waits for the ownership, let him know that we are going
+ * down and that we are not connected anymore. He'll be able to take
+ * the device.
+ */
+ wake_up_all(&mei->get_ownership_wq);
+
+ mutex_lock(&iwl_mei_mutex);
+
+ iwl_mei_global_cldev = NULL;
+
+ wake_up_all(&mei->get_nvm_wq);
+
+ iwl_mei_free_shared_mem(cldev);
+
+ iwl_mei_dbgfs_unregister(mei);
+
+ mei_cldev_set_drvdata(cldev, NULL);
+
+ kfree(mei->nvm);
+
+ kfree(rcu_access_pointer(mei->filters));
+
+ devm_kfree(&cldev->dev, mei);
+
+ mutex_unlock(&iwl_mei_mutex);
+}
+
+static const struct mei_cl_device_id iwl_mei_tbl[] = {
+ { KBUILD_MODNAME, MEI_WLAN_UUID, MEI_CL_VERSION_ANY},
+
+ /* required last entry */
+ { }
+};
+
+/*
+ * Do not export the device table because this module is loaded by
+ * iwlwifi's dependency.
+ */
+
+static struct mei_cl_driver iwl_mei_cl_driver = {
+ .id_table = iwl_mei_tbl,
+ .name = KBUILD_MODNAME,
+ .probe = iwl_mei_probe,
+ .remove = iwl_mei_remove,
+};
+
+module_mei_cl_driver(iwl_mei_cl_driver);
diff --git a/drivers/net/wireless/intel/iwlwifi/mei/net.c b/drivers/net/wireless/intel/iwlwifi/mei/net.c
new file mode 100644
index 0000000..5f966af
--- /dev/null
+++ b/drivers/net/wireless/intel/iwlwifi/mei/net.c
@@ -0,0 +1,409 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2021 Intel Corporation
+ */
+
+#include <uapi/linux/if_ether.h>
+#include <uapi/linux/if_arp.h>
+#include <uapi/linux/icmp.h>
+
+#include <linux/etherdevice.h>
+#include <linux/netdevice.h>
+#include <linux/skbuff.h>
+#include <linux/ieee80211.h>
+
+#include <net/cfg80211.h>
+#include <net/ip.h>
+
+#include <linux/if_arp.h>
+#include <linux/icmp.h>
+#include <linux/udp.h>
+#include <linux/ip.h>
+#include <linux/mm.h>
+
+#include "internal.h"
+#include "sap.h"
+#include "iwl-mei.h"
+
+/*
+ * Returns true if further filtering should be stopped. Only in that case
+ * pass_to_csme and rx_handler_res are set. Otherwise, next level of filters
+ * should be checked.
+ */
+static bool iwl_mei_rx_filter_eth(const struct ethhdr *ethhdr,
+ const struct iwl_sap_oob_filters *filters,
+ bool *pass_to_csme,
+ rx_handler_result_t *rx_handler_res)
+{
+ const struct iwl_sap_eth_filter *filt;
+
+ /* This filter is not relevant for UCAST packet */
+ if (!is_multicast_ether_addr(ethhdr->h_dest) ||
+ is_broadcast_ether_addr(ethhdr->h_dest))
+ return false;
+
+ for (filt = &filters->eth_filters[0];
+ filt < &filters->eth_filters[0] + ARRAY_SIZE(filters->eth_filters);
+ filt++) {
+ /* Assume there are no enabled filter after a disabled one */
+ if (!(filt->flags & SAP_ETH_FILTER_ENABLED))
+ break;
+
+ if (compare_ether_header(filt->mac_address, ethhdr->h_dest))
+ continue;
+
+ /* Packet needs to reach the host's stack */
+ if (filt->flags & SAP_ETH_FILTER_COPY)
+ *rx_handler_res = RX_HANDLER_PASS;
+ else
+ *rx_handler_res = RX_HANDLER_CONSUMED;
+
+ /* We have an authoritative answer, stop filtering */
+ if (filt->flags & SAP_ETH_FILTER_STOP) {
+ *pass_to_csme = true;
+ return true;
+ }
+
+ return false;
+ }
+
+ /* MCAST frames that don't match layer 2 filters are not sent to ME */
+ *pass_to_csme = false;
+
+ return true;
+}
+
+/*
+ * Returns true iff the frame should be passed to CSME in which case
+ * rx_handler_res is set.
+ */
+static bool iwl_mei_rx_filter_arp(struct sk_buff *skb,
+ const struct iwl_sap_oob_filters *filters,
+ rx_handler_result_t *rx_handler_res)
+{
+ const struct iwl_sap_ipv4_filter *filt = &filters->ipv4_filter;
+ const struct arphdr *arp;
+ const __be32 *target_ip;
+ u32 flags = le32_to_cpu(filt->flags);
+
+ if (!pskb_may_pull(skb, arp_hdr_len(skb->dev)))
+ return false;
+
+ arp = arp_hdr(skb);
+
+ /* Handle only IPv4 over ethernet ARP frames */
+ if (arp->ar_hrd != htons(ARPHRD_ETHER) ||
+ arp->ar_pro != htons(ETH_P_IP))
+ return false;
+
+ /*
+ * After the ARP header, we have:
+ * src MAC address - 6 bytes
+ * src IP address - 4 bytes
+ * target MAC addess - 6 bytes
+ */
+ target_ip = (void *)((u8 *)(arp + 1) +
+ ETH_ALEN + sizeof(__be32) + ETH_ALEN);
+
+ /*
+ * ARP request is forwarded to ME only if IP address match in the
+ * ARP request's target ip field.
+ */
+ if (arp->ar_op == htons(ARPOP_REQUEST) &&
+ (filt->flags & cpu_to_le32(SAP_IPV4_FILTER_ARP_REQ_PASS)) &&
+ (filt->ipv4_addr == 0 || filt->ipv4_addr == *target_ip)) {
+ if (flags & SAP_IPV4_FILTER_ARP_REQ_COPY)
+ *rx_handler_res = RX_HANDLER_PASS;
+ else
+ *rx_handler_res = RX_HANDLER_CONSUMED;
+
+ return true;
+ }
+
+ /* ARP reply is always forwarded to ME regardless of the IP */
+ if (flags & SAP_IPV4_FILTER_ARP_RESP_PASS &&
+ arp->ar_op == htons(ARPOP_REPLY)) {
+ if (flags & SAP_IPV4_FILTER_ARP_RESP_COPY)
+ *rx_handler_res = RX_HANDLER_PASS;
+ else
+ *rx_handler_res = RX_HANDLER_CONSUMED;
+
+ return true;
+ }
+
+ return false;
+}
+
+static bool
+iwl_mei_rx_filter_tcp_udp(struct sk_buff *skb, bool ip_match,
+ const struct iwl_sap_oob_filters *filters,
+ rx_handler_result_t *rx_handler_res)
+{
+ const struct iwl_sap_flex_filter *filt;
+
+ for (filt = &filters->flex_filters[0];
+ filt < &filters->flex_filters[0] + ARRAY_SIZE(filters->flex_filters);
+ filt++) {
+ if (!(filt->flags & SAP_FLEX_FILTER_ENABLED))
+ break;
+
+ /*
+ * We are required to have a match on the IP level and we didn't
+ * have such match.
+ */
+ if ((filt->flags &
+ (SAP_FLEX_FILTER_IPV4 | SAP_FLEX_FILTER_IPV6)) &&
+ !ip_match)
+ continue;
+
+ if ((filt->flags & SAP_FLEX_FILTER_UDP) &&
+ ip_hdr(skb)->protocol != IPPROTO_UDP)
+ continue;
+
+ if ((filt->flags & SAP_FLEX_FILTER_TCP) &&
+ ip_hdr(skb)->protocol != IPPROTO_TCP)
+ continue;
+
+ /*
+ * We must have either a TCP header or a UDP header, both
+ * starts with a source port and then a destination port.
+ * Both are big endian words.
+ * Use a UDP header and that will work for TCP as well.
+ */
+ if ((filt->src_port && filt->src_port != udp_hdr(skb)->source) ||
+ (filt->dst_port && filt->dst_port != udp_hdr(skb)->dest))
+ continue;
+
+ if (filt->flags & SAP_FLEX_FILTER_COPY)
+ *rx_handler_res = RX_HANDLER_PASS;
+ else
+ *rx_handler_res = RX_HANDLER_CONSUMED;
+
+ return true;
+ }
+
+ return false;
+}
+
+static bool iwl_mei_rx_filter_ipv4(struct sk_buff *skb,
+ const struct iwl_sap_oob_filters *filters,
+ rx_handler_result_t *rx_handler_res)
+{
+ const struct iwl_sap_ipv4_filter *filt = &filters->ipv4_filter;
+ const struct iphdr *iphdr;
+ unsigned int iphdrlen;
+ bool match;
+
+ if (!pskb_may_pull(skb, skb_network_offset(skb) + sizeof(*iphdr)) ||
+ !pskb_may_pull(skb, skb_network_offset(skb) +
+ sizeof(ip_hdrlen(skb) - sizeof(*iphdr))))
+ return false;
+
+ iphdrlen = ip_hdrlen(skb);
+ iphdr = ip_hdr(skb);
+ match = !filters->ipv4_filter.ipv4_addr ||
+ filters->ipv4_filter.ipv4_addr == iphdr->daddr;
+
+ skb_set_transport_header(skb, skb_network_offset(skb) + iphdrlen);
+
+ switch (ip_hdr(skb)->protocol) {
+ case IPPROTO_UDP:
+ case IPPROTO_TCP:
+ /*
+ * UDP header is shorter than TCP header and we look at the first bytes
+ * of the header anyway (see below).
+ * If we have a truncated TCP packet, let CSME handle this.
+ */
+ if (!pskb_may_pull(skb, skb_transport_offset(skb) +
+ sizeof(struct udphdr)))
+ return false;
+
+ return iwl_mei_rx_filter_tcp_udp(skb, match,
+ filters, rx_handler_res);
+
+ case IPPROTO_ICMP: {
+ struct icmphdr *icmp;
+
+ if (!pskb_may_pull(skb, skb_transport_offset(skb) + sizeof(*icmp)))
+ return false;
+
+ icmp = icmp_hdr(skb);
+
+ /*
+ * Don't pass echo requests to ME even if it wants it as we
+ * want the host to answer.
+ */
+ if ((filt->flags & cpu_to_le32(SAP_IPV4_FILTER_ICMP_PASS)) &&
+ match && (icmp->type != ICMP_ECHO || icmp->code != 0)) {
+ if (filt->flags & cpu_to_le32(SAP_IPV4_FILTER_ICMP_COPY))
+ *rx_handler_res = RX_HANDLER_PASS;
+ else
+ *rx_handler_res = RX_HANDLER_CONSUMED;
+
+ return true;
+ }
+ break;
+ }
+ case IPPROTO_ICMPV6:
+ /* TODO: Should we have the same ICMP request logic here too? */
+ if ((filters->icmpv6_flags & cpu_to_le32(SAP_ICMPV6_FILTER_ENABLED) &&
+ match)) {
+ if (filters->icmpv6_flags &
+ cpu_to_le32(SAP_ICMPV6_FILTER_COPY))
+ *rx_handler_res = RX_HANDLER_PASS;
+ else
+ *rx_handler_res = RX_HANDLER_CONSUMED;
+
+ return true;
+ }
+ break;
+ default:
+ return false;
+ }
+
+ return false;
+}
+
+static bool iwl_mei_rx_filter_ipv6(struct sk_buff *skb,
+ const struct iwl_sap_oob_filters *filters,
+ rx_handler_result_t *rx_handler_res)
+{
+ *rx_handler_res = RX_HANDLER_PASS;
+
+ /* TODO */
+
+ return false;
+}
+
+static rx_handler_result_t
+iwl_mei_rx_pass_to_csme(struct sk_buff *skb,
+ const struct iwl_sap_oob_filters *filters,
+ bool *pass_to_csme)
+{
+ const struct ethhdr *ethhdr = (void *)skb_mac_header(skb);
+ rx_handler_result_t rx_handler_res = RX_HANDLER_PASS;
+ bool (*filt_handler)(struct sk_buff *skb,
+ const struct iwl_sap_oob_filters *filters,
+ rx_handler_result_t *rx_handler_res);
+
+ /*
+ * skb->data points the IP header / ARP header and the ETH header
+ * is in the headroom.
+ */
+ skb_reset_network_header(skb);
+
+ /*
+ * MCAST IP packets sent by us are received again here without
+ * an ETH header. Drop them here.
+ */
+ if (!skb_mac_offset(skb))
+ return RX_HANDLER_PASS;
+
+ if (skb_headroom(skb) < sizeof(*ethhdr))
+ return RX_HANDLER_PASS;
+
+ if (iwl_mei_rx_filter_eth(ethhdr, filters,
+ pass_to_csme, &rx_handler_res))
+ return rx_handler_res;
+
+ switch (skb->protocol) {
+ case htons(ETH_P_IP):
+ filt_handler = iwl_mei_rx_filter_ipv4;
+ break;
+ case htons(ETH_P_ARP):
+ filt_handler = iwl_mei_rx_filter_arp;
+ break;
+ case htons(ETH_P_IPV6):
+ filt_handler = iwl_mei_rx_filter_ipv6;
+ break;
+ default:
+ *pass_to_csme = false;
+ return rx_handler_res;
+ }
+
+ *pass_to_csme = filt_handler(skb, filters, &rx_handler_res);
+
+ return rx_handler_res;
+}
+
+rx_handler_result_t iwl_mei_rx_filter(struct sk_buff *orig_skb,
+ const struct iwl_sap_oob_filters *filters,
+ bool *pass_to_csme)
+{
+ rx_handler_result_t ret;
+ struct sk_buff *skb;
+
+ ret = iwl_mei_rx_pass_to_csme(orig_skb, filters, pass_to_csme);
+
+ if (!*pass_to_csme)
+ return RX_HANDLER_PASS;
+
+ if (ret == RX_HANDLER_PASS)
+ skb = skb_copy(orig_skb, GFP_ATOMIC);
+ else
+ skb = orig_skb;
+
+ /* CSME wants the MAC header as well, push it back */
+ skb_push(skb, skb->data - skb_mac_header(skb));
+
+ /*
+ * Add the packet that CSME wants to get to the ring. Don't send the
+ * Check Shared Area HECI message since this is not possible from the
+ * Rx context. The caller will schedule a worker to do just that.
+ */
+ iwl_mei_add_data_to_ring(skb, false);
+
+ /*
+ * In case we drop the packet, don't free it, the caller will do that
+ * for us
+ */
+ if (ret == RX_HANDLER_PASS)
+ dev_kfree_skb(skb);
+
+ return ret;
+}
+
+#define DHCP_SERVER_PORT 67
+#define DHCP_CLIENT_PORT 68
+void iwl_mei_tx_copy_to_csme(struct sk_buff *origskb, unsigned int ivlen)
+{
+ struct ieee80211_hdr *hdr;
+ struct sk_buff *skb;
+ struct ethhdr ethhdr;
+ struct ethhdr *eth;
+
+ /* Catch DHCP packets */
+ if (origskb->protocol != htons(ETH_P_IP) ||
+ ip_hdr(origskb)->protocol != IPPROTO_UDP ||
+ udp_hdr(origskb)->source != htons(DHCP_CLIENT_PORT) ||
+ udp_hdr(origskb)->dest != htons(DHCP_SERVER_PORT))
+ return;
+
+ /*
+ * We could be a bit less aggressive here and not copy everything, but
+ * this is very rare anyway, do don't bother much.
+ */
+ skb = skb_copy(origskb, GFP_ATOMIC);
+ if (!skb)
+ return;
+
+ skb->protocol = origskb->protocol;
+
+ hdr = (void *)skb->data;
+
+ memcpy(ethhdr.h_dest, ieee80211_get_DA(hdr), ETH_ALEN);
+ memcpy(ethhdr.h_source, ieee80211_get_SA(hdr), ETH_ALEN);
+
+ /*
+ * Remove the ieee80211 header + IV + SNAP but leave the ethertype
+ * We still have enough headroom for the sap header.
+ */
+ pskb_pull(skb, ieee80211_hdrlen(hdr->frame_control) + ivlen + 6);
+ eth = skb_push(skb, sizeof(ethhdr.h_dest) + sizeof(ethhdr.h_source));
+ memcpy(eth, ðhdr, sizeof(ethhdr.h_dest) + sizeof(ethhdr.h_source));
+
+ iwl_mei_add_data_to_ring(skb, true);
+
+ dev_kfree_skb(skb);
+}
+EXPORT_SYMBOL_GPL(iwl_mei_tx_copy_to_csme);
diff --git a/drivers/net/wireless/intel/iwlwifi/mei/sap.h b/drivers/net/wireless/intel/iwlwifi/mei/sap.h
new file mode 100644
index 0000000..11e3009
--- /dev/null
+++ b/drivers/net/wireless/intel/iwlwifi/mei/sap.h
@@ -0,0 +1,733 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2021 Intel Corporation
+ */
+
+#ifndef __sap_h__
+#define __sap_h__
+
+#include "mei/iwl-mei.h"
+
+/**
+ * DOC: Introduction
+ *
+ * SAP is the protocol used by the Intel Wireless driver (iwlwifi)
+ * and the wireless driver implemented in the CSME firmware.
+ * It allows to do several things:
+ * 1) Decide who is the owner of the device: CSME or the host
+ * 2) When the host is the owner of the device, CSME can still
+ * send and receive packets through iwlwifi.
+ *
+ * The protocol uses the ME interface (mei driver) to send
+ * messages to the CSME firmware. Those messages have a header
+ * &struct iwl_sap_me_msg_hdr and this header is followed
+ * by a payload.
+ *
+ * Since this messaging system cannot support high amounts of
+ * traffic, iwlwifi and the CSME firmware's WLAN driver have an
+ * addtional communication pipe to exchange information. The body
+ * of the message is copied to a shared area and the message that
+ * goes over the ME interface just signals the other side
+ * that a new message is waiting in the shared area. The ME
+ * interface is used only for signaling and not to transfer
+ * the payload.
+ *
+ * This shared area of memory is DMA'able mapped to be
+ * writable by both the CSME firmware and iwlwifi. It is
+ * mapped to address space of the device that controls the ME
+ * interface's DMA engine. Any data that iwlwifi needs to
+ * send to the CSME firmware needs to be copied to there.
+ */
+
+/**
+ * DOC: Initial Handshake
+ *
+ * Once we get a link to the CMSE's WLAN driver we start the handshake
+ * to establish the shared memory that will allow the communication between
+ * the CSME's WLAN driver and the host.
+ *
+ * 1) Host sends %SAP_ME_MSG_START message with the physical address
+ * of the shared area.
+ * 2) CSME replies with %SAP_ME_MSG_START_OK which includes the versions
+ * protocol versions supported by CSME.
+ */
+
+/**
+ * DOC: Host and driver state messages
+ *
+ * In order to let CSME konw about the host state and the host driver state,
+ * the host sends messages that let CSME know about the host's state.
+ * When the host driver is loaded, the host sends %SAP_MSG_NOTIF_WIFIDR_UP.
+ * When the host driver is unloaded, the host sends %SAP_MSG_NOTIF_WIFIDR_DOWN.
+ * When the iwlmei is unloaded, %SAP_MSG_NOTIF_HOST_GOES_DOWN is sent to let
+ * CSME know not to access the shared memory anymore since it'll be freed.
+ *
+ * CSME will reply to SAP_MSG_NOTIF_WIFIDR_UP by
+ * %SAP_MSG_NOTIF_AMT_STATE to let the host driver whether CSME can use the
+ * WiFi device or not followed by %SAP_MSG_NOTIF_CSME_CONN_STATUS to inform
+ * the host driver on the connection state of CSME.
+ *
+ * When host is associated to an AP, it must send %SAP_MSG_NOTIF_HOST_LINK_UP
+ * and when it disconnect from the AP, it must send
+ * %SAP_MSG_NOTIF_HOST_LINK_DOWN.
+ */
+
+/**
+ * DOC: Ownership
+ *
+ * The device can be controlled either by the CSME firmware or
+ * by the host driver: iwlwifi. There is a negotiaion between
+ * those two entities to determine who controls (or owns) the
+ * device. Since the CSME can control the device even when the
+ * OS is not working or even missing, the CSME can request the
+ * device if it comes to the conclusion that the OS's host driver
+ * is not operational. This is why the host driver needs to
+ * signal CSME that it is up and running. If the driver is
+ * unloaded, it'll signal CSME that it is going down so that
+ * CSME can take ownership.
+ */
+
+/**
+ * DOC: Ownership transfer
+ *
+ * When the host driver needs the device, it'll send the
+ * %SAP_MSG_NOTIF_HOST_ASKS_FOR_NIC_OWNERSHIP that will be replied by
+ * %SAP_MSG_NOTIF_CSME_REPLY_TO_HOST_OWNERSHIP_REQ which will let the
+ * host know whether the ownership is granted or no. If the ownership is
+ * granted, the hosts sends %SAP_MSG_NOTIF_HOST_OWNERSHIP_CONFIRMED.
+ *
+ * When CSME requests ownership, it'll send the
+ * %SAP_MSG_NOTIF_CSME_TAKING_OWNERSHIP and give some time to host to stop
+ * accessing the device. The host needs to send
+ * %SAP_MSG_NOTIF_CSME_OWNERSHIP_CONFIRMED to confirm that it won't access
+ * the device anymore. If the host failed to send this message fast enough,
+ * CSME will take ownership on the device anyway.
+ * When CSME is willing to release the ownership, it'll send
+ * %SAP_MSG_NOTIF_CSME_CAN_RELEASE_OWNERSHIP.
+ */
+
+/**
+ * DOC: Data messages
+ *
+ * Data messages must be sent and receives on a separate queue in the shared
+ * memory. Almost all the data messages use the %SAP_MSG_DATA_PACKET for both
+ * packets sent by CSME to the host to be sent to the AP or for packets
+ * received from the AP and sent by the host to CSME.
+ * CSME sends filters to the host to let the host what inbound packets it must
+ * send to CSME. Those filters are received by the host as a
+ * %SAP_MSG_NOTIF_CSME_FILTERS command.
+ * The only outbound packets that must be sent to CSME are the DHCP packets.
+ * Those packets must use the %SAP_MSG_CB_DATA_PACKET message.
+ */
+
+/**
+ * enum iwl_sap_me_msg_id - the ID of the ME message
+ * @SAP_ME_MSG_START: See &struct iwl_sap_me_msg_start.
+ * @SAP_ME_MSG_START_OK: See &struct iwl_sap_me_msg_start_ok.
+ * @SAP_ME_MSG_CHECK_SHARED_AREA: This message has no payload.
+ */
+enum iwl_sap_me_msg_id {
+ SAP_ME_MSG_START = 1,
+ SAP_ME_MSG_START_OK,
+ SAP_ME_MSG_CHECK_SHARED_AREA,
+};
+
+/**
+ * struct iwl_sap_me_msg_hdr - the header of the ME message
+ * @type: the type of the message, see &enum iwl_sap_me_msg_id.
+ * @seq_num: a sequence number used for debug only.
+ * @len: the length of the mssage.
+ */
+struct iwl_sap_me_msg_hdr {
+ __le32 type;
+ __le32 seq_num;
+ __le32 len;
+} __packed;
+
+/**
+ * struct iwl_sap_me_msg_start - used for the %SAP_ME_MSG_START message
+ * @hdr: See &struct iwl_sap_me_msg_hdr.
+ * @shared_mem: physical address of SAP shared memory area.
+ * @init_data_seq_num: seq_num of the first data packet HOST -> CSME.
+ * @init_notif_seq_num: seq_num of the first notification HOST -> CSME.
+ * @supported_versions: The host sends to the CSME a zero-terminated array
+ * of versions its supports.
+ *
+ * This message is sent by the host to CSME and will responded by the
+ * %SAP_ME_MSG_START_OK message.
+ */
+struct iwl_sap_me_msg_start {
+ struct iwl_sap_me_msg_hdr hdr;
+ __le64 shared_mem;
+ __le16 init_data_seq_num;
+ __le16 init_notif_seq_num;
+ u8 supported_versions[64];
+} __packed;
+
+/**
+ * struct iwl_sap_me_msg_start_ok - used for the %SAP_ME_MSG_START_OK
+ * @hdr: See &struct iwl_sap_me_msg_hdr
+ * @init_data_seq_num: Not used.
+ * @init_notif_seq_num: Not used
+ * @supported_version: The version that will be used.
+ * @reserved: For alignment.
+ *
+ * This message is sent by CSME to the host in response to the
+ * %SAP_ME_MSG_START message.
+ */
+struct iwl_sap_me_msg_start_ok {
+ struct iwl_sap_me_msg_hdr hdr;
+ __le16 init_data_seq_num;
+ __le16 init_notif_seq_num;
+ u8 supported_version;
+ u8 reserved[3];
+} __packed;
+
+/**
+ * enum iwl_sap_msg - SAP messages
+ * @SAP_MSG_NOTIF_BOTH_WAYS_MIN: Not used.
+ * @SAP_MSG_NOTIF_PING: No payload. Solicitate a response message (check-alive).
+ * @SAP_MSG_NOTIF_PONG: No payload. The response message.
+ * @SAP_MSG_NOTIF_BOTH_WAYS_MAX: Not used.
+ *
+ * @SAP_MSG_NOTIF_FROM_CSME_MIN: Not used.
+ * @SAP_MSG_NOTIF_CSME_FILTERS: TODO
+ * @SAP_MSG_NOTIF_AMT_STATE: Payload is a DW. Any non-zero value means
+ * that CSME is enabled.
+ * @SAP_MSG_NOTIF_CSME_REPLY_TO_HOST_OWNERSHIP_REQ: Payload is a DW. 0 means
+ * the host will not get ownership. Any other value means the host is
+ * the owner.
+ * @SAP_MSG_NOTIF_CSME_TAKING_OWNERSHIP: No payload.
+ * @SAP_MSG_NOTIF_TRIGGER_IP_REFRESH: No payload.
+ * @SAP_MSG_NOTIF_CSME_CAN_RELEASE_OWNERSHIP: No payload.
+ * @SAP_MSG_NOTIF_NIC_OWNER: Payload is a DW. See &enum iwl_sap_nic_owner.
+ * @SAP_MSG_NOTIF_CSME_CONN_STATUS: See &struct iwl_sap_notif_conn_status.
+ * @SAP_MSG_NOTIF_NVM: See &struct iwl_sap_nvm.
+ * @SAP_MSG_NOTIF_FROM_CSME_MAX: Not used.
+ *
+ * @SAP_MSG_NOTIF_FROM_HOST_MIN: Not used.
+ * @SAP_MSG_NOTIF_BAND_SELECTION: TODO
+ * @SAP_MSG_NOTIF_RADIO_STATE: Payload is a DW.
+ * See &enum iwl_sap_radio_state_bitmap.
+ * @SAP_MSG_NOTIF_NIC_INFO: See &struct iwl_sap_notif_host_nic_info.
+ * @SAP_MSG_NOTIF_HOST_ASKS_FOR_NIC_OWNERSHIP: No payload.
+ * @SAP_MSG_NOTIF_HOST_SUSPENDS: Payload is a DW. Bitmap described in
+ * &enum iwl_sap_notif_host_suspends_bitmap.
+ * @SAP_MSG_NOTIF_HOST_RESUMES: Payload is a DW. 0 or 1. 1 says that
+ * the CSME should re-initialize the init control block.
+ * @SAP_MSG_NOTIF_HOST_GOES_DOWN: No payload.
+ * @SAP_MSG_NOTIF_CSME_OWNERSHIP_CONFIRMED: No payload.
+ * @SAP_MSG_NOTIF_COUNTRY_CODE: See &struct iwl_sap_notif_country_code.
+ * @SAP_MSG_NOTIF_HOST_LINK_UP: See &struct iwl_sap_notif_host_link_up.
+ * @SAP_MSG_NOTIF_HOST_LINK_DOWN: See &struct iwl_sap_notif_host_link_down.
+ * @SAP_MSG_NOTIF_WHO_OWNS_NIC: No payload.
+ * @SAP_MSG_NOTIF_WIFIDR_DOWN: No payload.
+ * @SAP_MSG_NOTIF_WIFIDR_UP: No payload.
+ * @SAP_MSG_NOTIF_HOST_OWNERSHIP_CONFIRMED: No payload.
+ * @SAP_MSG_NOTIF_SAR_LIMITS: See &struct iwl_sap_notif_sar_limits.
+ * @SAP_MSG_NOTIF_GET_NVM: No payload. Triggers %SAP_MSG_NOTIF_NVM.
+ * @SAP_MSG_NOTIF_FROM_HOST_MAX: Not used.
+ *
+ * @SAP_MSG_DATA_MIN: Not used.
+ * @SAP_MSG_DATA_PACKET: Packets that passed the filters defined by
+ * %SAP_MSG_NOTIF_CSME_FILTERS. The payload is &struct iwl_sap_hdr with
+ * the payload of the packet immediately afterwards.
+ * @SAP_MSG_CB_DATA_PACKET: Indicates to CSME that we transmitted a specific
+ * packet. Used only for DHCP transmitted packets. See
+ * &struct iwl_sap_cb_data.
+ * @SAP_MSG_DATA_MAX: Not used.
+ */
+enum iwl_sap_msg {
+ SAP_MSG_NOTIF_BOTH_WAYS_MIN = 0,
+ SAP_MSG_NOTIF_PING = 1,
+ SAP_MSG_NOTIF_PONG = 2,
+ SAP_MSG_NOTIF_BOTH_WAYS_MAX,
+
+ SAP_MSG_NOTIF_FROM_CSME_MIN = 500,
+ SAP_MSG_NOTIF_CSME_FILTERS = SAP_MSG_NOTIF_FROM_CSME_MIN,
+ /* 501 is deprecated */
+ SAP_MSG_NOTIF_AMT_STATE = 502,
+ SAP_MSG_NOTIF_CSME_REPLY_TO_HOST_OWNERSHIP_REQ = 503,
+ SAP_MSG_NOTIF_CSME_TAKING_OWNERSHIP = 504,
+ SAP_MSG_NOTIF_TRIGGER_IP_REFRESH = 505,
+ SAP_MSG_NOTIF_CSME_CAN_RELEASE_OWNERSHIP = 506,
+ /* 507 is deprecated */
+ /* 508 is deprecated */
+ /* 509 is deprecated */
+ /* 510 is deprecated */
+ SAP_MSG_NOTIF_NIC_OWNER = 511,
+ SAP_MSG_NOTIF_CSME_CONN_STATUS = 512,
+ SAP_MSG_NOTIF_NVM = 513,
+ SAP_MSG_NOTIF_FROM_CSME_MAX,
+
+ SAP_MSG_NOTIF_FROM_HOST_MIN = 1000,
+ SAP_MSG_NOTIF_BAND_SELECTION = SAP_MSG_NOTIF_FROM_HOST_MIN,
+ SAP_MSG_NOTIF_RADIO_STATE = 1001,
+ SAP_MSG_NOTIF_NIC_INFO = 1002,
+ SAP_MSG_NOTIF_HOST_ASKS_FOR_NIC_OWNERSHIP = 1003,
+ SAP_MSG_NOTIF_HOST_SUSPENDS = 1004,
+ SAP_MSG_NOTIF_HOST_RESUMES = 1005,
+ SAP_MSG_NOTIF_HOST_GOES_DOWN = 1006,
+ SAP_MSG_NOTIF_CSME_OWNERSHIP_CONFIRMED = 1007,
+ SAP_MSG_NOTIF_COUNTRY_CODE = 1008,
+ SAP_MSG_NOTIF_HOST_LINK_UP = 1009,
+ SAP_MSG_NOTIF_HOST_LINK_DOWN = 1010,
+ SAP_MSG_NOTIF_WHO_OWNS_NIC = 1011,
+ SAP_MSG_NOTIF_WIFIDR_DOWN = 1012,
+ SAP_MSG_NOTIF_WIFIDR_UP = 1013,
+ /* 1014 is deprecated */
+ SAP_MSG_NOTIF_HOST_OWNERSHIP_CONFIRMED = 1015,
+ SAP_MSG_NOTIF_SAR_LIMITS = 1016,
+ SAP_MSG_NOTIF_GET_NVM = 1017,
+ SAP_MSG_NOTIF_FROM_HOST_MAX,
+
+ SAP_MSG_DATA_MIN = 2000,
+ SAP_MSG_DATA_PACKET = SAP_MSG_DATA_MIN,
+ SAP_MSG_CB_DATA_PACKET = 2001,
+ SAP_MSG_DATA_MAX,
+};
+
+/**
+ * struct iwl_sap_hdr - prefixes any SAP message
+ * @type: See &enum iwl_sap_msg.
+ * @len: The length of the message (header not included).
+ * @seq_num: For debug.
+ * @payload: The payload of the message.
+ */
+struct iwl_sap_hdr {
+ __le16 type;
+ __le16 len;
+ __le32 seq_num;
+ u8 payload[0];
+};
+
+/**
+ * struct iwl_sap_msg_dw - suits any DW long SAP message
+ * @hdr: The SAP header
+ * @val: The value of the DW.
+ */
+struct iwl_sap_msg_dw {
+ struct iwl_sap_hdr hdr;
+ __le32 val;
+};
+
+/**
+ * enum iwl_sap_nic_owner - used by %SAP_MSG_NOTIF_NIC_OWNER
+ * @SAP_NIC_OWNER_UNKNOWN: Not used.
+ * @SAP_NIC_OWNER_HOST: The host owns the NIC.
+ * @SAP_NIC_OWNER_ME: CSME owns the NIC.
+ */
+enum iwl_sap_nic_owner {
+ SAP_NIC_OWNER_UNKNOWN,
+ SAP_NIC_OWNER_HOST,
+ SAP_NIC_OWNER_ME,
+};
+
+enum iwl_sap_wifi_auth_type {
+ SAP_WIFI_AUTH_TYPE_OPEN = IWL_MEI_AKM_AUTH_OPEN,
+ SAP_WIFI_AUTH_TYPE_RSNA = IWL_MEI_AKM_AUTH_RSNA,
+ SAP_WIFI_AUTH_TYPE_RSNA_PSK = IWL_MEI_AKM_AUTH_RSNA_PSK,
+ SAP_WIFI_AUTH_TYPE_SAE = IWL_MEI_AKM_AUTH_SAE,
+ SAP_WIFI_AUTH_TYPE_MAX,
+};
+
+/**
+ * enum iwl_sap_wifi_cipher_alg
+ * @SAP_WIFI_CIPHER_ALG_NONE: TBD
+ * @SAP_WIFI_CIPHER_ALG_CCMP: TBD
+ * @SAP_WIFI_CIPHER_ALG_GCMP: TBD
+ * @SAP_WIFI_CIPHER_ALG_GCMP_256: TBD
+ */
+enum iwl_sap_wifi_cipher_alg {
+ SAP_WIFI_CIPHER_ALG_NONE = IWL_MEI_CIPHER_NONE,
+ SAP_WIFI_CIPHER_ALG_CCMP = IWL_MEI_CIPHER_CCMP,
+ SAP_WIFI_CIPHER_ALG_GCMP = IWL_MEI_CIPHER_GCMP,
+ SAP_WIFI_CIPHER_ALG_GCMP_256 = IWL_MEI_CIPHER_GCMP_256,
+};
+
+/**
+ * struct iwl_sap_notif_connection_info - nested in other structures
+ * @ssid_len: The length of the SSID.
+ * @ssid: The SSID.
+ * @auth_mode: The authentication mode. See &enum iwl_sap_wifi_auth_type.
+ * @pairwise_cipher: The cipher used for unicast packets.
+ * See &enum iwl_sap_wifi_cipher_alg.
+ * @channel: The channel on which we are associated.
+ * @band: The band on which we are associated.
+ * @reserved: For alignment.
+ * @bssid: The BSSID.
+ * @reserved1: For alignment.
+ */
+struct iwl_sap_notif_connection_info {
+ __le32 ssid_len;
+ u8 ssid[32];
+ __le32 auth_mode;
+ __le32 pairwise_cipher;
+ u8 channel;
+ u8 band;
+ __le16 reserved;
+ u8 bssid[6];
+ __le16 reserved1;
+} __packed;
+
+/**
+ * enum iwl_sap_scan_request - for the scan_request field
+ * @SCAN_REQUEST_FILTERING: Filtering is requested.
+ * @SCAN_REQUEST_FAST: Fast scan is requested.
+ */
+enum iwl_sap_scan_request {
+ SCAN_REQUEST_FILTERING = 1 << 0,
+ SCAN_REQUEST_FAST = 1 << 1,
+};
+
+/**
+ * struct iwl_sap_notif_conn_status - payload of %SAP_MSG_NOTIF_CSME_CONN_STATUS
+ * @hdr: The SAP header
+ * @link_prot_state: Non-zero if link protection is active.
+ * @scan_request: See &enum iwl_sap_scan_request.
+ * @conn_info: Information about the connection.
+ */
+struct iwl_sap_notif_conn_status {
+ struct iwl_sap_hdr hdr;
+ __le32 link_prot_state;
+ __le32 scan_request;
+ struct iwl_sap_notif_connection_info conn_info;
+} __packed;
+
+/**
+ * enum iwl_sap_radio_state_bitmap - used for %SAP_MSG_NOTIF_RADIO_STATE
+ * @SAP_SW_RFKILL_DEASSERTED: If set, SW RfKill is de-asserted
+ * @SAP_HW_RFKILL_DEASSERTED: If set, HW RfKill is de-asserted
+ *
+ * If both bits are set, then the radio is on.
+ */
+enum iwl_sap_radio_state_bitmap {
+ SAP_SW_RFKILL_DEASSERTED = 1 << 0,
+ SAP_HW_RFKILL_DEASSERTED = 1 << 1,
+};
+
+/**
+ * enum iwl_sap_notif_host_suspends_bitmap - used for %SAP_MSG_NOTIF_HOST_SUSPENDS
+ * @SAP_OFFER_NIC: TBD
+ * @SAP_FILTER_CONFIGURED: TBD
+ * @SAP_NLO_CONFIGURED: TBD
+ * @SAP_HOST_OWNS_NIC: TBD
+ * @SAP_LINK_PROTECTED: TBD
+ */
+enum iwl_sap_notif_host_suspends_bitmap {
+ SAP_OFFER_NIC = 1 << 0,
+ SAP_FILTER_CONFIGURED = 1 << 1,
+ SAP_NLO_CONFIGURED = 1 << 2,
+ SAP_HOST_OWNS_NIC = 1 << 3,
+ SAP_LINK_PROTECTED = 1 << 4,
+};
+
+/**
+ * struct iwl_sap_notif_country_code - payload of %SAP_MSG_NOTIF_COUNTRY_CODE
+ * @hdr: The SAP header
+ * @mcc: The country code.
+ * @source_id: TBD
+ * @reserved: For alignment.
+ * @diff_time: TBD
+ */
+struct iwl_sap_notif_country_code {
+ struct iwl_sap_hdr hdr;
+ __le16 mcc;
+ u8 source_id;
+ u8 reserved;
+ __le32 diff_time;
+} __packed;
+
+/**
+ * struct iwl_sap_notif_host_link_up - payload of %SAP_MSG_NOTIF_HOST_LINK_UP
+ * @hdr: The SAP header
+ * @conn_info: Information about the connection.
+ * @colloc_channel: The collocated channel
+ * @colloc_band: The band of the collocated channel.
+ * @reserved: For alignment.
+ * @colloc_bssid: The collocated BSSID.
+ * @reserved1: For alignment.
+ */
+struct iwl_sap_notif_host_link_up {
+ struct iwl_sap_hdr hdr;
+ struct iwl_sap_notif_connection_info conn_info;
+ u8 colloc_channel;
+ u8 colloc_band;
+ __le16 reserved;
+ u8 colloc_bssid[6];
+ __le16 reserved1;
+} __packed;
+
+/**
+ * enum iwl_sap_notif_link_down_type - used in &struct iwl_sap_notif_host_link_down
+ * @HOST_LINK_DOWN_TYPE_NONE: TBD
+ * @HOST_LINK_DOWN_TYPE_TEMPORARY: TBD
+ * @HOST_LINK_DOWN_TYPE_LONG: TBD
+ */
+enum iwl_sap_notif_link_down_type {
+ HOST_LINK_DOWN_TYPE_NONE,
+ HOST_LINK_DOWN_TYPE_TEMPORARY,
+ HOST_LINK_DOWN_TYPE_LONG,
+};
+
+/**
+ * struct iwl_sap_notif_host_link_down - payload for %SAP_MSG_NOTIF_HOST_LINK_DOWN
+ * @hdr: The SAP header
+ * @type: See &enum iwl_sap_notif_link_down_type.
+ * @reserved: For alignment.
+ * @reason_valid: If 0, ignore the next field.
+ * @reason: The reason of the disconnection.
+ */
+struct iwl_sap_notif_host_link_down {
+ struct iwl_sap_hdr hdr;
+ u8 type;
+ u8 reserved[2];
+ u8 reason_valid;
+ __le32 reason;
+} __packed;
+
+/**
+ * struct iwl_sap_notif_host_nic_info - payload for %SAP_MSG_NOTIF_NIC_INFO
+ * @hdr: The SAP header
+ * @mac_address: The MAC address as configured to the interface.
+ * @nvm_address: The MAC address as configured in the NVM.
+ */
+struct iwl_sap_notif_host_nic_info {
+ struct iwl_sap_hdr hdr;
+ u8 mac_address[6];
+ u8 nvm_address[6];
+} __packed;
+
+/**
+ * struct iwl_sap_notif_dw - payload is a dw
+ * @hdr: The SAP header.
+ * @dw: The payload.
+ */
+struct iwl_sap_notif_dw {
+ struct iwl_sap_hdr hdr;
+ __le32 dw;
+} __packed;
+
+/**
+ * struct iwl_sap_notif_sar_limits - payload for %SAP_MSG_NOTIF_SAR_LIMITS
+ * @hdr: The SAP header
+ * @sar_chain_info_table: Tx power limits.
+ */
+struct iwl_sap_notif_sar_limits {
+ struct iwl_sap_hdr hdr;
+ __le16 sar_chain_info_table[2][5];
+} __packed;
+
+/**
+ * enum iwl_sap_nvm_caps - capabilities for NVM SAP
+ * @SAP_NVM_CAPS_LARI_SUPPORT: Lari is supported
+ * @SAP_NVM_CAPS_11AX_SUPPORT: 11AX is supported
+ */
+enum iwl_sap_nvm_caps {
+ SAP_NVM_CAPS_LARI_SUPPORT = BIT(0),
+ SAP_NVM_CAPS_11AX_SUPPORT = BIT(1),
+};
+
+/**
+ * struct iwl_sap_nvm - payload for %SAP_MSG_NOTIF_NVM
+ * @hdr: The SAP header.
+ * @hw_addr: The MAC address
+ * @n_hw_addrs: The number of MAC addresses
+ * @reserved: For alignment.
+ * @radio_cfg: The radio configuration.
+ * @caps: See &enum iwl_sap_nvm_caps.
+ * @nvm_version: The version of the NVM.
+ * @channels: The data for each channel.
+ */
+struct iwl_sap_nvm {
+ struct iwl_sap_hdr hdr;
+ u8 hw_addr[6];
+ u8 n_hw_addrs;
+ u8 reserved;
+ __le32 radio_cfg;
+ __le32 caps;
+ __le32 nvm_version;
+ __le32 channels[110];
+} __packed;
+
+/**
+ * enum iwl_sap_eth_filter_flags - used in &struct iwl_sap_eth_filter
+ * @SAP_ETH_FILTER_STOP: Do not process further filters.
+ * @SAP_ETH_FILTER_COPY: Copy the packet to the CSME.
+ * @SAP_ETH_FILTER_ENABLED: If false, the filter should be ignored.
+ */
+enum iwl_sap_eth_filter_flags {
+ SAP_ETH_FILTER_STOP = BIT(0),
+ SAP_ETH_FILTER_COPY = BIT(1),
+ SAP_ETH_FILTER_ENABLED = BIT(2),
+};
+
+/**
+ * struct iwl_sap_eth_filter - a L2 filter
+ * @mac_address: Address to filter.
+ * @flags: See &enum iwl_sap_eth_filter_flags.
+ */
+struct iwl_sap_eth_filter {
+ u8 mac_address[6];
+ u8 flags;
+} __packed;
+
+/**
+ * enum iwl_sap_flex_filter_flags - used in &struct iwl_sap_flex_filter
+ * @SAP_FLEX_FILTER_COPY: Pass UDP / TCP packets to CSME.
+ * @SAP_FLEX_FILTER_ENABLED: If false, the filter should be ignored.
+ * @SAP_FLEX_FILTER_IPV4: Filter requires match on the IP address as well.
+ * @SAP_FLEX_FILTER_IPV6: Filter requires match on the IP address as well.
+ * @SAP_FLEX_FILTER_TCP: Filter should be applied on TCP packets.
+ * @SAP_FLEX_FILTER_UDP: Filter should be applied on UDP packets.
+ */
+enum iwl_sap_flex_filter_flags {
+ SAP_FLEX_FILTER_COPY = BIT(0),
+ SAP_FLEX_FILTER_ENABLED = BIT(1),
+ SAP_FLEX_FILTER_IPV6 = BIT(2),
+ SAP_FLEX_FILTER_IPV4 = BIT(3),
+ SAP_FLEX_FILTER_TCP = BIT(4),
+ SAP_FLEX_FILTER_UDP = BIT(5),
+};
+
+/**
+ * struct iwl_sap_flex_filter -
+ * @src_port: Source port in network format.
+ * @dst_port: Destination port in network format.
+ * @flags: Flags and protocol, see &enum iwl_sap_flex_filter_flags.
+ * @reserved: For alignment.
+ */
+struct iwl_sap_flex_filter {
+ __be16 src_port;
+ __be16 dst_port;
+ u8 flags;
+ u8 reserved;
+} __packed;
+
+/**
+ * enum iwl_sap_ipv4_filter_flags - used in &struct iwl_sap_ipv4_filter
+ * @SAP_IPV4_FILTER_ICMP_PASS: Pass ICMP packets to CSME.
+ * @SAP_IPV4_FILTER_ICMP_COPY: Pass ICMP packets to host.
+ * @SAP_IPV4_FILTER_ARP_REQ_PASS: Pass ARP requests to CSME.
+ * @SAP_IPV4_FILTER_ARP_REQ_COPY: Pass ARP requests to host.
+ * @SAP_IPV4_FILTER_ARP_RESP_PASS: Pass ARP responses to CSME.
+ * @SAP_IPV4_FILTER_ARP_RESP_COPY: Pass ARP responses to host.
+ */
+enum iwl_sap_ipv4_filter_flags {
+ SAP_IPV4_FILTER_ICMP_PASS = BIT(0),
+ SAP_IPV4_FILTER_ICMP_COPY = BIT(1),
+ SAP_IPV4_FILTER_ARP_REQ_PASS = BIT(2),
+ SAP_IPV4_FILTER_ARP_REQ_COPY = BIT(3),
+ SAP_IPV4_FILTER_ARP_RESP_PASS = BIT(4),
+ SAP_IPV4_FILTER_ARP_RESP_COPY = BIT(5),
+};
+
+/**
+ * struct iwl_sap_ipv4_filter-
+ * @ipv4_addr: The IP address to filer.
+ * @flags: See &enum iwl_sap_ipv4_filter_flags.
+ */
+struct iwl_sap_ipv4_filter {
+ __be32 ipv4_addr;
+ __le32 flags;
+} __packed;
+
+/**
+ * enum iwl_sap_ipv6_filter_flags -
+ * @SAP_IPV6_ADDR_FILTER_COPY: Pass packets to the host.
+ * @SAP_IPV6_ADDR_FILTER_ENABLED: If false, the filter should be ignored.
+ */
+enum iwl_sap_ipv6_filter_flags {
+ SAP_IPV6_ADDR_FILTER_COPY = BIT(0),
+ SAP_IPV6_ADDR_FILTER_ENABLED = BIT(1),
+};
+
+/**
+ * struct iwl_sap_ipv6_filter -
+ * @addr_lo24: Lowest 24 bits of the IPv6 address.
+ * @flags: See &enum iwl_sap_ipv6_filter_flags.
+ */
+struct iwl_sap_ipv6_filter {
+ u8 addr_lo24[3];
+ u8 flags;
+} __packed;
+
+/**
+ * enum iwl_sap_icmpv6_filter_flags -
+ * @SAP_ICMPV6_FILTER_ENABLED: If false, the filter should be ignored.
+ * @SAP_ICMPV6_FILTER_COPY: Pass packets to the host.
+ */
+enum iwl_sap_icmpv6_filter_flags {
+ SAP_ICMPV6_FILTER_ENABLED = BIT(0),
+ SAP_ICMPV6_FILTER_COPY = BIT(1),
+};
+
+/**
+ * enum iwl_sap_vlan_filter_flags -
+ * @SAP_VLAN_FILTER_VLAN_ID_MSK: TBD
+ * @SAP_VLAN_FILTER_ENABLED: If false, the filter should be ignored.
+ */
+enum iwl_sap_vlan_filter_flags {
+ SAP_VLAN_FILTER_VLAN_ID_MSK = 0x0FFF,
+ SAP_VLAN_FILTER_ENABLED = BIT(15),
+};
+
+/**
+ * struct iwl_sap_oob_filters - Out of band filters (for RX only)
+ * @flex_filters: Array of &struct iwl_sap_flex_filter.
+ * @icmpv6_flags: See &enum iwl_sap_icmpv6_filter_flags.
+ * @ipv6_filters: Array of &struct iwl_sap_ipv6_filter.
+ * @eth_filters: Array of &struct iwl_sap_eth_filter.
+ * @reserved: For alignment.
+ * @ipv4_filter: &struct iwl_sap_ipv4_filter.
+ * @vlan: See &enum iwl_sap_vlan_filter_flags.
+ */
+struct iwl_sap_oob_filters {
+ struct iwl_sap_flex_filter flex_filters[14];
+ __le32 icmpv6_flags;
+ struct iwl_sap_ipv6_filter ipv6_filters[4];
+ struct iwl_sap_eth_filter eth_filters[5];
+ u8 reserved;
+ struct iwl_sap_ipv4_filter ipv4_filter;
+ __le16 vlan[4];
+} __packed;
+
+/**
+ * struct iwl_sap_csme_filters - payload of %SAP_MSG_NOTIF_CSME_FILTERS
+ * @hdr: The SAP header.
+ * @mode: Not used.
+ * @mac_address: Not used.
+ * @reserved: For alignment.
+ * @cbfilters: Not used.
+ * @filters: Out of band filters.
+ */
+struct iwl_sap_csme_filters {
+ struct iwl_sap_hdr hdr;
+ __le32 mode;
+ u8 mac_address[6];
+ __le16 reserved;
+ u8 cbfilters[1728];
+ struct iwl_sap_oob_filters filters;
+} __packed;
+
+#define CB_TX_DHCP_FILT_IDX 30
+/**
+ * struct iwl_sap_cb_data - header to be added for transmitted packets.
+ * @hdr: The SAP header.
+ * @reserved: Not used.
+ * @to_me_filt_status: The filter that matches. Bit %CB_TX_DHCP_FILT_IDX should
+ * be set for DHCP (the only packet that uses this header).
+ * @reserved2: Not used.
+ * @data_len: The length of the payload.
+ * @payload: The payload of the transmitted packet.
+ */
+struct iwl_sap_cb_data {
+ struct iwl_sap_hdr hdr;
+ __le32 reserved[7];
+ __le32 to_me_filt_status;
+ __le32 reserved2;
+ __le32 data_len;
+ u8 payload[];
+};
+
+#endif /* __sap_h__ */
diff --git a/drivers/net/wireless/intel/iwlwifi/mei/trace-data.h b/drivers/net/wireless/intel/iwlwifi/mei/trace-data.h
new file mode 100644
index 0000000..83639c6
--- /dev/null
+++ b/drivers/net/wireless/intel/iwlwifi/mei/trace-data.h
@@ -0,0 +1,82 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright(c) 2021 Intel Corporation
+ */
+
+#if !defined(CONFIG_IWLWIFI_DEVICE_TRACING)
+
+#define trace_iwlmei_sap_data(...)
+
+#else
+
+#if !defined(__IWLWIFI_DEVICE_TRACE_IWLWIFI_SAP_DATA) || defined(TRACE_HEADER_MULTI_READ)
+
+#ifndef __IWLWIFI_DEVICE_TRACE_IWLWIFI_SAP_DATA
+enum iwl_sap_data_trace_type {
+ IWL_SAP_RX_DATA_TO_AIR,
+ IWL_SAP_TX_DATA_FROM_AIR,
+ IWL_SAP_RX_DATA_DROPPED_FROM_AIR,
+ IWL_SAP_TX_DHCP,
+};
+
+static inline size_t
+iwlmei_sap_data_offset(enum iwl_sap_data_trace_type trace_type)
+{
+ switch (trace_type) {
+ case IWL_SAP_RX_DATA_TO_AIR:
+ return 0;
+ case IWL_SAP_TX_DATA_FROM_AIR:
+ case IWL_SAP_RX_DATA_DROPPED_FROM_AIR:
+ return sizeof(struct iwl_sap_hdr);
+ case IWL_SAP_TX_DHCP:
+ return sizeof(struct iwl_sap_cb_data);
+ default:
+ WARN_ON_ONCE(1);
+ }
+
+ return 0;
+}
+#endif
+
+#define __IWLWIFI_DEVICE_TRACE_IWLWIFI_SAP_DATA
+
+#include <linux/tracepoint.h>
+#include <linux/skbuff.h>
+#include "sap.h"
+
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM iwlmei_sap_data
+
+TRACE_EVENT(iwlmei_sap_data,
+ TP_PROTO(const struct sk_buff *skb,
+ enum iwl_sap_data_trace_type trace_type),
+ TP_ARGS(skb, trace_type),
+ TP_STRUCT__entry(
+ __dynamic_array(u8, data,
+ skb->len - iwlmei_sap_data_offset(trace_type))
+ __field(u32, trace_type)
+ ),
+ TP_fast_assign(
+ size_t offset = iwlmei_sap_data_offset(trace_type);
+ __entry->trace_type = trace_type;
+ skb_copy_bits(skb, offset, __get_dynamic_array(data),
+ skb->len - offset);
+ ),
+ TP_printk("sap_data:trace_type %d len %d",
+ __entry->trace_type, __get_dynamic_array_len(data))
+);
+
+/*
+ * If you add something here, add a stub in case
+ * !defined(CONFIG_IWLWIFI_DEVICE_TRACING)
+ */
+
+#endif /* __IWLWIFI_DEVICE_TRACE_IWLWIFI_SAP_DATA */
+
+#undef TRACE_INCLUDE_PATH
+#define TRACE_INCLUDE_PATH .
+#undef TRACE_INCLUDE_FILE
+#define TRACE_INCLUDE_FILE trace-data
+#include <trace/define_trace.h>
+
+#endif /* CONFIG_IWLWIFI_DEVICE_TRACING */
diff --git a/drivers/net/wireless/intel/iwlwifi/mei/trace.c b/drivers/net/wireless/intel/iwlwifi/mei/trace.c
new file mode 100644
index 0000000..47ac32e
--- /dev/null
+++ b/drivers/net/wireless/intel/iwlwifi/mei/trace.c
@@ -0,0 +1,15 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2021 Intel Corporation
+ */
+
+#include <linux/module.h>
+
+/* sparse doesn't like tracepoint macros */
+#ifndef __CHECKER__
+
+#define CREATE_TRACE_POINTS
+#include "trace.h"
+#include "trace-data.h"
+
+#endif /* __CHECKER__ */
diff --git a/drivers/net/wireless/intel/iwlwifi/mei/trace.h b/drivers/net/wireless/intel/iwlwifi/mei/trace.h
new file mode 100644
index 0000000..45ecb22
--- /dev/null
+++ b/drivers/net/wireless/intel/iwlwifi/mei/trace.h
@@ -0,0 +1,76 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright(c) 2021 Intel Corporation
+ */
+
+#if !defined(CONFIG_IWLWIFI_DEVICE_TRACING)
+
+#define trace_iwlmei_sap_cmd(...)
+#define trace_iwlmei_me_msg(...)
+
+#else
+
+#if !defined(__IWLWIFI_DEVICE_TRACE_IWLWIFI_SAP_CMD) || defined(TRACE_HEADER_MULTI_READ)
+#define __IWLWIFI_DEVICE_TRACE_IWLWIFI_SAP_CMD
+
+#include <linux/tracepoint.h>
+
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM iwlmei_sap_cmd
+
+#include "mei/sap.h"
+
+TRACE_EVENT(iwlmei_sap_cmd,
+ TP_PROTO(const struct iwl_sap_hdr *sap_cmd, bool tx),
+ TP_ARGS(sap_cmd, tx),
+ TP_STRUCT__entry(
+ __dynamic_array(u8, cmd,
+ le16_to_cpu(sap_cmd->len) + sizeof(*sap_cmd))
+ __field(u8, tx)
+ __field(u16, type)
+ __field(u16, len)
+ __field(u32, seq)
+ ),
+ TP_fast_assign(
+ memcpy(__get_dynamic_array(cmd), sap_cmd,
+ le16_to_cpu(sap_cmd->len) + sizeof(*sap_cmd));
+ __entry->tx = tx;
+ __entry->type = le16_to_cpu(sap_cmd->type);
+ __entry->len = le16_to_cpu(sap_cmd->len);
+ __entry->seq = le32_to_cpu(sap_cmd->seq_num);
+ ),
+ TP_printk("sap_cmd %s: type %d len %d seq %d", __entry->tx ? "Tx" : "Rx",
+ __entry->type, __entry->len, __entry->seq)
+);
+
+TRACE_EVENT(iwlmei_me_msg,
+ TP_PROTO(const struct iwl_sap_me_msg_hdr *hdr, bool tx),
+ TP_ARGS(hdr, tx),
+ TP_STRUCT__entry(
+ __field(u8, type)
+ __field(u8, tx)
+ __field(u32, seq_num)
+ ),
+ TP_fast_assign(
+ __entry->type = le32_to_cpu(hdr->type);
+ __entry->seq_num = le32_to_cpu(hdr->seq_num);
+ __entry->tx = tx;
+ ),
+ TP_printk("ME message: %s: type %d seq %d", __entry->tx ? "Tx" : "Rx",
+ __entry->type, __entry->seq_num)
+);
+
+/*
+ * If you add something here, add a stub in case
+ * !defined(CONFIG_IWLWIFI_DEVICE_TRACING)
+ */
+
+#endif /* __IWLWIFI_DEVICE_TRACE_IWLWIFI_SAP_CMD */
+
+#undef TRACE_INCLUDE_PATH
+#define TRACE_INCLUDE_PATH .
+#undef TRACE_INCLUDE_FILE
+#define TRACE_INCLUDE_FILE trace
+#include <trace/define_trace.h>
+
+#endif /* CONFIG_IWLWIFI_DEVICE_TRACING */
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/Makefile b/drivers/net/wireless/intel/iwlwifi/mvm/Makefile
index 75fc2d9..11e814b 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/Makefile
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/Makefile
@@ -10,5 +10,6 @@
iwlmvm-$(CONFIG_IWLWIFI_DEBUGFS) += debugfs.o debugfs-vif.o
iwlmvm-$(CONFIG_IWLWIFI_LEDS) += led.o
iwlmvm-$(CONFIG_PM) += d3.o
+iwlmvm-$(CONFIG_IWLMEI) += vendor-cmd.o
ccflags-y += -I $(srctree)/$(src)/../
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/constants.h b/drivers/net/wireless/intel/iwlwifi/mvm/constants.h
index 9d0d01f2..c604f9f 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/constants.h
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/constants.h
@@ -107,7 +107,7 @@
#define IWL_MVM_FTM_NON_TB_MAX_TIME_BETWEEN_MSR 1000
#define IWL_MVM_D3_DEBUG false
#define IWL_MVM_USE_TWT true
-#define IWL_MVM_AMPDU_CONSEC_DROPS_DELBA 10
+#define IWL_MVM_AMPDU_CONSEC_DROPS_DELBA 20
#define IWL_MVM_USE_NSSN_SYNC 0
#define IWL_MVM_PHY_FILTER_CHAIN_A 0
#define IWL_MVM_PHY_FILTER_CHAIN_B 0
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/d3.c b/drivers/net/wireless/intel/iwlwifi/mvm/d3.c
index a19f646..b400867 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/d3.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/d3.c
@@ -1391,6 +1391,13 @@ struct iwl_wowlan_status_data {
u8 tid_tear_down;
struct {
+ /* including RX MIC key for TKIP */
+ u8 key[WOWLAN_KEY_MAX_SIZE];
+ u8 len;
+ u8 flags;
+ } gtk;
+
+ struct {
/*
* We store both the TKIP and AES representations
* coming from the firmware because we decode the
@@ -1400,11 +1407,15 @@ struct iwl_wowlan_status_data {
struct {
struct ieee80211_key_seq seq[IWL_MAX_TID_COUNT];
} tkip, aes;
- /* including RX MIC key for TKIP */
- u8 key[WOWLAN_KEY_MAX_SIZE];
- u8 len;
- u8 flags;
- } gtk;
+
+ /*
+ * We use -1 for when we have valid data but don't know
+ * the key ID from firmware, and thus it needs to be
+ * installed with the last key (depending on rekeying).
+ */
+ s8 key_id;
+ bool valid;
+ } gtk_seq[2];
struct {
/* Same as above */
@@ -1556,12 +1567,10 @@ static void iwl_mvm_report_wakeup_reasons(struct iwl_mvm *mvm,
kfree_skb(pkt);
}
-static void iwl_mvm_aes_sc_to_seq(struct aes_sc *sc,
- struct ieee80211_key_seq *seq)
+static void iwl_mvm_le64_to_aes_seq(__le64 le_pn, struct ieee80211_key_seq *seq)
{
- u64 pn;
+ u64 pn = le64_to_cpu(le_pn);
- pn = le64_to_cpu(sc->pn);
seq->ccmp.pn[0] = pn >> 40;
seq->ccmp.pn[1] = pn >> 32;
seq->ccmp.pn[2] = pn >> 24;
@@ -1570,6 +1579,20 @@ static void iwl_mvm_aes_sc_to_seq(struct aes_sc *sc,
seq->ccmp.pn[5] = pn;
}
+static void iwl_mvm_aes_sc_to_seq(struct aes_sc *sc,
+ struct ieee80211_key_seq *seq)
+{
+ iwl_mvm_le64_to_aes_seq(sc->pn, seq);
+}
+
+static void iwl_mvm_le64_to_tkip_seq(__le64 le_pn, struct ieee80211_key_seq *seq)
+{
+ u64 pn = le64_to_cpu(le_pn);
+
+ seq->tkip.iv16 = (u16)pn;
+ seq->tkip.iv32 = (u32)(pn >> 16);
+}
+
static void iwl_mvm_tkip_sc_to_seq(struct tkip_sc *sc,
struct ieee80211_key_seq *seq)
{
@@ -1630,10 +1653,12 @@ static void iwl_mvm_convert_key_counters(struct iwl_wowlan_status_data *status,
/* GTK RX counters */
for (i = 0; i < IWL_MAX_TID_COUNT; i++) {
iwl_mvm_tkip_sc_to_seq(&sc->tkip.multicast_rsc[i],
- &status->gtk.tkip.seq[i]);
+ &status->gtk_seq[0].tkip.seq[i]);
iwl_mvm_aes_sc_to_seq(&sc->aes.multicast_rsc[i],
- &status->gtk.aes.seq[i]);
+ &status->gtk_seq[0].aes.seq[i]);
}
+ status->gtk_seq[0].valid = true;
+ status->gtk_seq[0].key_id = -1;
/* PTK TX counter */
status->ptk.tkip.tx_pn = (u64)le16_to_cpu(sc->tkip.tsc.iv16) |
@@ -1649,24 +1674,103 @@ static void iwl_mvm_convert_key_counters(struct iwl_wowlan_status_data *status,
}
}
-static void iwl_mvm_set_key_rx_seq(struct iwl_mvm *mvm,
- struct ieee80211_key_conf *key,
- struct iwl_wowlan_status_data *status)
+static void
+iwl_mvm_convert_key_counters_v5_gtk_seq(struct iwl_wowlan_status_data *status,
+ struct iwl_wowlan_all_rsc_tsc_v5 *sc,
+ unsigned int idx, unsigned int key_id)
+{
+ int tid;
+
+ for (tid = 0; tid < IWL_MAX_TID_COUNT; tid++) {
+ iwl_mvm_le64_to_tkip_seq(sc->mcast_rsc[idx][tid],
+ &status->gtk_seq[idx].tkip.seq[tid]);
+ iwl_mvm_le64_to_aes_seq(sc->mcast_rsc[idx][tid],
+ &status->gtk_seq[idx].aes.seq[tid]);
+ }
+
+ status->gtk_seq[idx].valid = true;
+ status->gtk_seq[idx].key_id = key_id;
+}
+
+static void
+iwl_mvm_convert_key_counters_v5(struct iwl_wowlan_status_data *status,
+ struct iwl_wowlan_all_rsc_tsc_v5 *sc)
+{
+ int i, tid;
+
+ BUILD_BUG_ON(IWL_MAX_TID_COUNT > IWL_MAX_TID_COUNT);
+ BUILD_BUG_ON(IWL_MAX_TID_COUNT > IWL_NUM_RSC);
+ BUILD_BUG_ON(ARRAY_SIZE(sc->mcast_rsc) != ARRAY_SIZE(status->gtk_seq));
+
+ /* GTK RX counters */
+ for (i = 0; i < ARRAY_SIZE(sc->mcast_key_id_map); i++) {
+ u8 entry = sc->mcast_key_id_map[i];
+
+ if (entry < ARRAY_SIZE(sc->mcast_rsc))
+ iwl_mvm_convert_key_counters_v5_gtk_seq(status, sc,
+ entry, i);
+ }
+
+ /* PTK TX counters not needed, assigned in device */
+
+ /* PTK RX counters */
+ for (tid = 0; tid < IWL_MAX_TID_COUNT; tid++) {
+ iwl_mvm_le64_to_tkip_seq(sc->ucast_rsc[tid],
+ &status->ptk.tkip.seq[tid]);
+ iwl_mvm_le64_to_aes_seq(sc->ucast_rsc[tid],
+ &status->ptk.aes.seq[tid]);
+ }
+}
+
+static void iwl_mvm_set_key_rx_seq_idx(struct ieee80211_key_conf *key,
+ struct iwl_wowlan_status_data *status,
+ int idx)
{
switch (key->cipher) {
case WLAN_CIPHER_SUITE_CCMP:
case WLAN_CIPHER_SUITE_GCMP:
case WLAN_CIPHER_SUITE_GCMP_256:
- iwl_mvm_set_key_rx_seq_tids(key, status->gtk.aes.seq);
+ iwl_mvm_set_key_rx_seq_tids(key, status->gtk_seq[idx].aes.seq);
break;
case WLAN_CIPHER_SUITE_TKIP:
- iwl_mvm_set_key_rx_seq_tids(key, status->gtk.tkip.seq);
+ iwl_mvm_set_key_rx_seq_tids(key, status->gtk_seq[idx].tkip.seq);
break;
default:
WARN_ON(1);
}
}
+static void iwl_mvm_set_key_rx_seq(struct ieee80211_key_conf *key,
+ struct iwl_wowlan_status_data *status,
+ bool installed)
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(status->gtk_seq); i++) {
+ if (!status->gtk_seq[i].valid)
+ continue;
+
+ /* Handle the case where we know the key ID */
+ if (status->gtk_seq[i].key_id == key->keyidx) {
+ s8 new_key_id = -1;
+
+ if (status->num_of_gtk_rekeys)
+ new_key_id = status->gtk.flags &
+ IWL_WOWLAN_GTK_IDX_MASK;
+
+ /* Don't install a new key's value to an old key */
+ if (new_key_id != key->keyidx)
+ iwl_mvm_set_key_rx_seq_idx(key, status, i);
+ continue;
+ }
+
+ /* handle the case where we didn't, last key only */
+ if (status->gtk_seq[i].key_id == -1 &&
+ (!status->num_of_gtk_rekeys || installed))
+ iwl_mvm_set_key_rx_seq_idx(key, status, i);
+ }
+}
+
struct iwl_mvm_d3_gtk_iter_data {
struct iwl_mvm *mvm;
struct iwl_wowlan_status_data *status;
@@ -1740,8 +1844,9 @@ static void iwl_mvm_d3_update_keys(struct ieee80211_hw *hw,
if (data->status->num_of_gtk_rekeys)
ieee80211_remove_key(key);
- else if (data->last_gtk == key)
- iwl_mvm_set_key_rx_seq(data->mvm, key, data->status);
+
+ if (data->last_gtk == key)
+ iwl_mvm_set_key_rx_seq(key, data->status, false);
}
static bool iwl_mvm_setup_connection_keep(struct iwl_mvm *mvm,
@@ -1825,7 +1930,7 @@ static bool iwl_mvm_setup_connection_keep(struct iwl_mvm *mvm,
key = ieee80211_gtk_rekey_add(vif, &conf.conf);
if (IS_ERR(key))
return false;
- iwl_mvm_set_key_rx_seq(mvm, key, status);
+ iwl_mvm_set_key_rx_seq(key, status, true);
replay_ctr = cpu_to_be64(status->replay_ctr);
@@ -1893,9 +1998,10 @@ iwl_mvm_parse_wowlan_status_common_ ## _ver(struct iwl_mvm *mvm, \
iwl_mvm_parse_wowlan_status_common(v6)
iwl_mvm_parse_wowlan_status_common(v7)
iwl_mvm_parse_wowlan_status_common(v9)
+iwl_mvm_parse_wowlan_status_common(v12)
-static void iwl_mvm_convert_gtk(struct iwl_wowlan_status_data *status,
- struct iwl_wowlan_gtk_status *data)
+static void iwl_mvm_convert_gtk_v2(struct iwl_wowlan_status_data *status,
+ struct iwl_wowlan_gtk_status_v2 *data)
{
BUILD_BUG_ON(sizeof(status->gtk.key) < sizeof(data->key));
BUILD_BUG_ON(NL80211_TKIP_DATA_OFFSET_RX_MIC_KEY +
@@ -1913,6 +2019,26 @@ static void iwl_mvm_convert_gtk(struct iwl_wowlan_status_data *status,
data->tkip_mic_key, sizeof(data->tkip_mic_key));
}
+static void iwl_mvm_convert_gtk_v3(struct iwl_wowlan_status_data *status,
+ struct iwl_wowlan_gtk_status_v3 *data)
+{
+ /* The parts we need are identical in v2 and v3 */
+#define CHECK(_f) do { \
+ BUILD_BUG_ON(offsetof(struct iwl_wowlan_gtk_status_v2, _f) != \
+ offsetof(struct iwl_wowlan_gtk_status_v3, _f)); \
+ BUILD_BUG_ON(offsetofend(struct iwl_wowlan_gtk_status_v2, _f) !=\
+ offsetofend(struct iwl_wowlan_gtk_status_v3, _f)); \
+} while (0)
+
+ CHECK(key);
+ CHECK(key_len);
+ CHECK(key_flags);
+ CHECK(tkip_mic_key);
+#undef CHECK
+
+ iwl_mvm_convert_gtk_v2(status, (void *)data);
+}
+
static void iwl_mvm_convert_igtk(struct iwl_wowlan_status_data *status,
struct iwl_wowlan_igtk_status *data)
{
@@ -2012,7 +2138,7 @@ iwl_mvm_send_wowlan_get_status(struct iwl_mvm *mvm, u8 sta_id)
goto out_free_resp;
iwl_mvm_convert_key_counters(status, &v7->gtk[0].rsc.all_tsc_rsc);
- iwl_mvm_convert_gtk(status, &v7->gtk[0]);
+ iwl_mvm_convert_gtk_v2(status, &v7->gtk[0]);
iwl_mvm_convert_igtk(status, &v7->igtk[0]);
} else if (notif_ver == 9 || notif_ver == 10 || notif_ver == 11) {
struct iwl_wowlan_status_v9 *v9 = (void *)cmd.resp_pkt->data;
@@ -2025,10 +2151,22 @@ iwl_mvm_send_wowlan_get_status(struct iwl_mvm *mvm, u8 sta_id)
goto out_free_resp;
iwl_mvm_convert_key_counters(status, &v9->gtk[0].rsc.all_tsc_rsc);
- iwl_mvm_convert_gtk(status, &v9->gtk[0]);
+ iwl_mvm_convert_gtk_v2(status, &v9->gtk[0]);
iwl_mvm_convert_igtk(status, &v9->igtk[0]);
status->tid_tear_down = v9->tid_tear_down;
+ } else if (notif_ver == 12) {
+ struct iwl_wowlan_status_v12 *v12 = (void *)cmd.resp_pkt->data;
+
+ status = iwl_mvm_parse_wowlan_status_common_v12(mvm, v12, len);
+ if (IS_ERR(status))
+ goto out_free_resp;
+
+ iwl_mvm_convert_key_counters_v5(status, &v12->gtk[0].sc);
+ iwl_mvm_convert_gtk_v3(status, &v12->gtk[0]);
+ iwl_mvm_convert_igtk(status, &v12->igtk[0]);
+
+ status->tid_tear_down = v12->tid_tear_down;
} else {
IWL_ERR(mvm,
"Firmware advertises unknown WoWLAN status response %d!\n",
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/debugfs.c b/drivers/net/wireless/intel/iwlwifi/mvm/debugfs.c
index ff66001..fb4920b 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/debugfs.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/debugfs.c
@@ -1022,6 +1022,11 @@ static ssize_t iwl_dbgfs_fw_restart_write(struct iwl_mvm *mvm, char *buf,
if (mvm->fw_restart >= 0)
mvm->fw_restart++;
+ if (count == 6 && !strcmp(buf, "nolog\n")) {
+ set_bit(IWL_MVM_STATUS_SUPPRESS_ERROR_LOG_ONCE, &mvm->status);
+ set_bit(STATUS_SUPPRESS_CMD_ERROR_ONCE, &mvm->trans->status);
+ }
+
/* take the return value to make compiler happy - it will fail anyway */
ret = iwl_mvm_send_cmd_pdu(mvm,
WIDE_ID(LONG_GROUP, REPLY_ERROR),
@@ -1038,6 +1043,9 @@ static ssize_t iwl_dbgfs_fw_nmi_write(struct iwl_mvm *mvm, char *buf,
if (!iwl_mvm_firmware_running(mvm))
return -EIO;
+ if (count == 6 && !strcmp(buf, "nolog\n"))
+ set_bit(IWL_MVM_STATUS_SUPPRESS_ERROR_LOG_ONCE, &mvm->status);
+
iwl_force_nmi(mvm->trans);
return count;
@@ -2064,6 +2072,7 @@ void iwl_mvm_dbgfs_register(struct iwl_mvm *mvm)
MVM_DEBUGFS_ADD_FILE(prph_reg, mvm->debugfs_dir, 0600);
MVM_DEBUGFS_ADD_FILE(fw_dbg_conf, mvm->debugfs_dir, 0600);
MVM_DEBUGFS_ADD_FILE(fw_dbg_collect, mvm->debugfs_dir, 0200);
+ MVM_DEBUGFS_ADD_FILE(dbg_time_point, mvm->debugfs_dir, 0200);
MVM_DEBUGFS_ADD_FILE(send_echo_cmd, mvm->debugfs_dir, 0200);
MVM_DEBUGFS_ADD_FILE(indirection_tbl, mvm->debugfs_dir, 0200);
MVM_DEBUGFS_ADD_FILE(inject_packet, mvm->debugfs_dir, 0200);
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/ftm-initiator.c b/drivers/net/wireless/intel/iwlwifi/mvm/ftm-initiator.c
index 949fb790..628aee6 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/ftm-initiator.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/ftm-initiator.c
@@ -511,7 +511,7 @@ iwl_mvm_ftm_put_target(struct iwl_mvm *mvm, struct ieee80211_vif *vif,
rcu_read_lock();
sta = rcu_dereference(mvm->fw_id_to_mac_id[mvmvif->ap_sta_id]);
- if (sta->mfp)
+ if (sta->mfp && (peer->ftm.trigger_based || peer->ftm.non_trigger_based))
FTM_PUT_FLAG(PMF);
rcu_read_unlock();
@@ -1066,7 +1066,7 @@ static void iwl_mvm_ftm_rtt_smoothing(struct iwl_mvm *mvm,
overshoot = IWL_MVM_FTM_INITIATOR_SMOOTH_OVERSHOOT;
alpha = IWL_MVM_FTM_INITIATOR_SMOOTH_ALPHA;
- rtt_avg = (alpha * rtt + (100 - alpha) * resp->rtt_avg) / 100;
+ rtt_avg = div_s64(alpha * rtt + (100 - alpha) * resp->rtt_avg, 100);
IWL_DEBUG_INFO(mvm,
"%pM: prev rtt_avg=%lld, new rtt_avg=%lld, rtt=%lld\n",
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/fw-api.h b/drivers/net/wireless/intel/iwlwifi/mvm/fw-api.h
index 73a82f0..083f86f 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/fw-api.h
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/fw-api.h
@@ -15,7 +15,7 @@
#include "fw/api/datapath.h"
#include "fw/api/phy.h"
#include "fw/api/config.h"
-#include "fw/api/soc.h"
+#include "fw/api/system.h"
#include "fw/api/alive.h"
#include "fw/api/binding.h"
#include "fw/api/cmdhdr.h"
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/fw.c b/drivers/net/wireless/intel/iwlwifi/mvm/fw.c
index 863fec1..6f4690e 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/fw.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/fw.c
@@ -12,8 +12,6 @@
#include "iwl-op-mode.h"
#include "fw/img.h"
#include "iwl-debug.h"
-#include "iwl-csr.h" /* for iwl_mvm_rx_card_state_notif */
-#include "iwl-io.h" /* for iwl_mvm_rx_card_state_notif */
#include "iwl-prph.h"
#include "fw/acpi.h"
#include "fw/pnvm.h"
@@ -32,6 +30,9 @@
#define IWL_PPAG_MASK 3
#define IWL_PPAG_ETSI_MASK BIT(0)
+#define IWL_TAS_US_MCC 0x5553
+#define IWL_TAS_CANADA_MCC 0x4341
+
struct iwl_mvm_alive_data {
bool valid;
u32 scd_base_addr;
@@ -123,13 +124,15 @@ static bool iwl_alive_fn(struct iwl_notif_wait_data *notif_wait,
struct iwl_lmac_alive *lmac2 = NULL;
u16 status;
u32 lmac_error_event_table, umac_error_table;
+ u32 version = iwl_fw_lookup_notif_ver(mvm->fw, LEGACY_GROUP,
+ UCODE_ALIVE_NTFY, 0);
/*
* For v5 and above, we can check the version, for older
* versions we need to check the size.
*/
- if (iwl_fw_lookup_notif_ver(mvm->fw, LEGACY_GROUP,
- UCODE_ALIVE_NTFY, 0) == 5) {
+ if (version == 5 || version == 6) {
+ /* v5 and v6 are compatible (only IMR addition) */
struct iwl_alive_ntf_v5 *palive;
if (pkt_len < sizeof(*palive))
@@ -516,7 +519,6 @@ static void iwl_mvm_phy_filter_init(struct iwl_mvm *mvm,
cpu_to_le32(IWL_MVM_PHY_FILTER_CHAIN_D);
}
}
-
#else /* CONFIG_ACPI */
static void iwl_mvm_phy_filter_init(struct iwl_mvm *mvm,
@@ -525,6 +527,49 @@ static void iwl_mvm_phy_filter_init(struct iwl_mvm *mvm,
}
#endif /* CONFIG_ACPI */
+#if defined(CONFIG_ACPI) && defined(CONFIG_EFI)
+static int iwl_mvm_sgom_init(struct iwl_mvm *mvm)
+{
+ u8 cmd_ver;
+ int ret;
+ struct iwl_host_cmd cmd = {
+ .id = WIDE_ID(REGULATORY_AND_NVM_GROUP,
+ SAR_OFFSET_MAPPING_TABLE_CMD),
+ .flags = 0,
+ .data[0] = &mvm->fwrt.sgom_table,
+ .len[0] = sizeof(mvm->fwrt.sgom_table),
+ .dataflags[0] = IWL_HCMD_DFL_NOCOPY,
+ };
+
+ if (!mvm->fwrt.sgom_enabled) {
+ IWL_DEBUG_RADIO(mvm, "SGOM table is disabled\n");
+ return 0;
+ }
+
+ cmd_ver = iwl_fw_lookup_cmd_ver(mvm->fw, REGULATORY_AND_NVM_GROUP,
+ SAR_OFFSET_MAPPING_TABLE_CMD,
+ IWL_FW_CMD_VER_UNKNOWN);
+
+ if (cmd_ver != 2) {
+ IWL_DEBUG_RADIO(mvm, "command version is unsupported. version = %d\n",
+ cmd_ver);
+ return 0;
+ }
+
+ ret = iwl_mvm_send_cmd(mvm, &cmd);
+ if (ret < 0)
+ IWL_ERR(mvm, "failed to send SAR_OFFSET_MAPPING_CMD (%d)\n", ret);
+
+ return ret;
+}
+#else
+
+static int iwl_mvm_sgom_init(struct iwl_mvm *mvm)
+{
+ return 0;
+}
+#endif
+
static int iwl_send_phy_cfg_cmd(struct iwl_mvm *mvm)
{
struct iwl_phy_cfg_cmd_v3 phy_cfg_cmd;
@@ -757,6 +802,8 @@ int iwl_mvm_sar_select_profile(struct iwl_mvm *mvm, int prof_a, int prof_b)
if (ret)
return ret;
+ iwl_mei_set_power_limit(per_chain);
+
IWL_DEBUG_RADIO(mvm, "Sending REDUCE_TX_POWER_CMD per chain\n");
return iwl_mvm_send_cmd_pdu(mvm, REDUCE_TX_POWER_CMD, 0, len, &cmd);
}
@@ -820,6 +867,7 @@ static int iwl_mvm_sar_geo_init(struct iwl_mvm *mvm)
u16 len;
u32 n_bands;
u32 n_profiles;
+ u32 sk = 0;
int ret;
u8 cmd_ver = iwl_fw_lookup_cmd_ver(mvm->fw, PHY_OPS_GROUP,
PER_CHAIN_LIMIT_OFFSET_CMD,
@@ -879,19 +927,26 @@ static int iwl_mvm_sar_geo_init(struct iwl_mvm *mvm)
if (ret)
return 0;
+ /* Only set to South Korea if the table revision is 1 */
+ if (mvm->fwrt.geo_rev == 1)
+ sk = 1;
+
/*
- * Set the revision on versions that contain it.
+ * Set the table_revision to South Korea (1) or not (0). The
+ * element name is misleading, as it doesn't contain the table
+ * revision number, but whether the South Korea variation
+ * should be used.
* This must be done after calling iwl_sar_geo_init().
*/
if (cmd_ver == 5)
- cmd.v5.table_revision = cpu_to_le32(mvm->fwrt.geo_rev);
+ cmd.v5.table_revision = cpu_to_le32(sk);
else if (cmd_ver == 4)
- cmd.v4.table_revision = cpu_to_le32(mvm->fwrt.geo_rev);
+ cmd.v4.table_revision = cpu_to_le32(sk);
else if (cmd_ver == 3)
- cmd.v3.table_revision = cpu_to_le32(mvm->fwrt.geo_rev);
+ cmd.v3.table_revision = cpu_to_le32(sk);
else if (fw_has_api(&mvm->fwrt.fw->ucode_capa,
IWL_UCODE_TLV_API_SAR_TABLE_VER))
- cmd.v2.table_revision = cpu_to_le32(mvm->fwrt.geo_rev);
+ cmd.v2.table_revision = cpu_to_le32(sk);
return iwl_mvm_send_cmd_pdu(mvm,
WIDE_ID(PHY_OPS_GROUP,
@@ -904,13 +959,8 @@ static int iwl_mvm_get_ppag_table(struct iwl_mvm *mvm)
union acpi_object *wifi_pkg, *data, *flags;
int i, j, ret, tbl_rev, num_sub_bands;
int idx = 2;
- s8 *gain;
- /*
- * The 'flags' field is the same in v1 and in v2 so we can just
- * use v1 to access it.
- */
- mvm->fwrt.ppag_table.v1.flags = cpu_to_le32(0);
+ mvm->fwrt.ppag_flags = 0;
data = iwl_acpi_get_object(mvm->dev, ACPI_PPAG_METHOD);
if (IS_ERR(data))
@@ -922,8 +972,6 @@ static int iwl_mvm_get_ppag_table(struct iwl_mvm *mvm)
if (!IS_ERR(wifi_pkg)) {
if (tbl_rev == 1 || tbl_rev == 2) {
num_sub_bands = IWL_NUM_SUB_BANDS_V2;
- gain = mvm->fwrt.ppag_table.v2.gain[0];
- mvm->fwrt.ppag_ver = tbl_rev;
IWL_DEBUG_RADIO(mvm,
"Reading PPAG table v2 (tbl_rev=%d)\n",
tbl_rev);
@@ -943,8 +991,6 @@ static int iwl_mvm_get_ppag_table(struct iwl_mvm *mvm)
goto out_free;
}
num_sub_bands = IWL_NUM_SUB_BANDS_V1;
- gain = mvm->fwrt.ppag_table.v1.gain[0];
- mvm->fwrt.ppag_ver = 0;
IWL_DEBUG_RADIO(mvm, "Reading PPAG table v1 (tbl_rev=0)\n");
goto read_table;
}
@@ -952,6 +998,7 @@ static int iwl_mvm_get_ppag_table(struct iwl_mvm *mvm)
goto out_free;
read_table:
+ mvm->fwrt.ppag_ver = tbl_rev;
flags = &wifi_pkg->package.elements[1];
if (flags->type != ACPI_TYPE_INTEGER) {
@@ -959,10 +1006,9 @@ static int iwl_mvm_get_ppag_table(struct iwl_mvm *mvm)
goto out_free;
}
- mvm->fwrt.ppag_table.v1.flags = cpu_to_le32(flags->integer.value &
- IWL_PPAG_MASK);
+ mvm->fwrt.ppag_flags = flags->integer.value & IWL_PPAG_MASK;
- if (!mvm->fwrt.ppag_table.v1.flags) {
+ if (!mvm->fwrt.ppag_flags) {
ret = 0;
goto out_free;
}
@@ -982,15 +1028,15 @@ static int iwl_mvm_get_ppag_table(struct iwl_mvm *mvm)
goto out_free;
}
- gain[i * num_sub_bands + j] = ent->integer.value;
+ mvm->fwrt.ppag_chains[i].subbands[j] = ent->integer.value;
if ((j == 0 &&
- (gain[i * num_sub_bands + j] > ACPI_PPAG_MAX_LB ||
- gain[i * num_sub_bands + j] < ACPI_PPAG_MIN_LB)) ||
+ (mvm->fwrt.ppag_chains[i].subbands[j] > ACPI_PPAG_MAX_LB ||
+ mvm->fwrt.ppag_chains[i].subbands[j] < ACPI_PPAG_MIN_LB)) ||
(j != 0 &&
- (gain[i * num_sub_bands + j] > ACPI_PPAG_MAX_HB ||
- gain[i * num_sub_bands + j] < ACPI_PPAG_MIN_HB))) {
- mvm->fwrt.ppag_table.v1.flags = cpu_to_le32(0);
+ (mvm->fwrt.ppag_chains[i].subbands[j] > ACPI_PPAG_MAX_HB ||
+ mvm->fwrt.ppag_chains[i].subbands[j] < ACPI_PPAG_MIN_HB))) {
+ mvm->fwrt.ppag_flags = 0;
ret = -EINVAL;
goto out_free;
}
@@ -1005,6 +1051,7 @@ static int iwl_mvm_get_ppag_table(struct iwl_mvm *mvm)
int iwl_mvm_ppag_send_cmd(struct iwl_mvm *mvm)
{
+ union iwl_ppag_table_cmd cmd;
u8 cmd_ver;
int i, j, ret, num_sub_bands, cmd_size;
s8 *gain;
@@ -1014,37 +1061,39 @@ int iwl_mvm_ppag_send_cmd(struct iwl_mvm *mvm)
"PPAG capability not supported by FW, command not sent.\n");
return 0;
}
- if (!mvm->fwrt.ppag_table.v1.flags) {
+ if (!mvm->fwrt.ppag_flags) {
IWL_DEBUG_RADIO(mvm, "PPAG not enabled, command not sent.\n");
return 0;
}
+ /* The 'flags' field is the same in v1 and in v2 so we can just
+ * use v1 to access it.
+ */
+ cmd.v1.flags = cpu_to_le32(mvm->fwrt.ppag_flags);
cmd_ver = iwl_fw_lookup_cmd_ver(mvm->fw, PHY_OPS_GROUP,
PER_PLATFORM_ANT_GAIN_CMD,
IWL_FW_CMD_VER_UNKNOWN);
if (cmd_ver == 1) {
num_sub_bands = IWL_NUM_SUB_BANDS_V1;
- gain = mvm->fwrt.ppag_table.v1.gain[0];
- cmd_size = sizeof(mvm->fwrt.ppag_table.v1);
+ gain = cmd.v1.gain[0];
+ cmd_size = sizeof(cmd.v1);
if (mvm->fwrt.ppag_ver == 1 || mvm->fwrt.ppag_ver == 2) {
IWL_DEBUG_RADIO(mvm,
"PPAG table rev is %d but FW supports v1, sending truncated table\n",
mvm->fwrt.ppag_ver);
- mvm->fwrt.ppag_table.v1.flags &=
- cpu_to_le32(IWL_PPAG_ETSI_MASK);
+ cmd.v1.flags &= cpu_to_le32(IWL_PPAG_ETSI_MASK);
}
} else if (cmd_ver == 2 || cmd_ver == 3) {
num_sub_bands = IWL_NUM_SUB_BANDS_V2;
- gain = mvm->fwrt.ppag_table.v2.gain[0];
- cmd_size = sizeof(mvm->fwrt.ppag_table.v2);
+ gain = cmd.v2.gain[0];
+ cmd_size = sizeof(cmd.v2);
if (mvm->fwrt.ppag_ver == 0) {
IWL_DEBUG_RADIO(mvm,
"PPAG table is v1 but FW supports v2, sending padded table\n");
} else if (cmd_ver == 2 && mvm->fwrt.ppag_ver == 2) {
IWL_DEBUG_RADIO(mvm,
"PPAG table is v3 but FW supports v2, sending partial bitmap.\n");
- mvm->fwrt.ppag_table.v1.flags &=
- cpu_to_le32(IWL_PPAG_ETSI_MASK);
+ cmd.v1.flags &= cpu_to_le32(IWL_PPAG_ETSI_MASK);
}
} else {
IWL_DEBUG_RADIO(mvm, "Unsupported PPAG command version\n");
@@ -1053,6 +1102,8 @@ int iwl_mvm_ppag_send_cmd(struct iwl_mvm *mvm)
for (i = 0; i < IWL_NUM_CHAIN_LIMITS; i++) {
for (j = 0; j < num_sub_bands; j++) {
+ gain[i * num_sub_bands + j] =
+ mvm->fwrt.ppag_chains[i].subbands[j];
IWL_DEBUG_RADIO(mvm,
"PPAG table: chain[%d] band[%d]: gain = %d\n",
i, j, gain[i * num_sub_bands + j]);
@@ -1061,7 +1112,7 @@ int iwl_mvm_ppag_send_cmd(struct iwl_mvm *mvm)
IWL_DEBUG_RADIO(mvm, "Sending PER_PLATFORM_ANT_GAIN_CMD\n");
ret = iwl_mvm_send_cmd_pdu(mvm, WIDE_ID(PHY_OPS_GROUP,
PER_PLATFORM_ANT_GAIN_CMD),
- 0, cmd_size, &mvm->fwrt.ppag_table);
+ 0, cmd_size, &cmd);
if (ret < 0)
IWL_ERR(mvm, "failed to send PER_PLATFORM_ANT_GAIN_CMD (%d)\n",
ret);
@@ -1100,18 +1151,63 @@ static int iwl_mvm_ppag_init(struct iwl_mvm *mvm)
IWL_DEBUG_RADIO(mvm,
"System vendor '%s' is not in the approved list, disabling PPAG.\n",
dmi_get_system_info(DMI_SYS_VENDOR));
- mvm->fwrt.ppag_table.v1.flags = cpu_to_le32(0);
+ mvm->fwrt.ppag_flags = 0;
return 0;
}
return iwl_mvm_ppag_send_cmd(mvm);
}
+static const struct dmi_system_id dmi_tas_approved_list[] = {
+ { .ident = "HP",
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "HP"),
+ },
+ },
+ { .ident = "SAMSUNG",
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "SAMSUNG ELECTRONICS CO., LTD"),
+ },
+ },
+ { .ident = "LENOVO",
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "Lenovo"),
+ },
+ },
+ { .ident = "DELL",
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "Dell Inc."),
+ },
+ },
+
+ /* keep last */
+ {}
+};
+
+static bool iwl_mvm_add_to_tas_block_list(__le32 *list, __le32 *le_size, unsigned int mcc)
+{
+ int i;
+ u32 size = le32_to_cpu(*le_size);
+
+ /* Verify that there is room for another country */
+ if (size >= IWL_TAS_BLOCK_LIST_MAX)
+ return false;
+
+ for (i = 0; i < size; i++) {
+ if (list[i] == cpu_to_le32(mcc))
+ return true;
+ }
+
+ list[size++] = cpu_to_le32(mcc);
+ *le_size = cpu_to_le32(size);
+ return true;
+}
+
static void iwl_mvm_tas_init(struct iwl_mvm *mvm)
{
int ret;
- struct iwl_tas_config_cmd cmd = {};
- int list_size;
+ struct iwl_tas_config_cmd_v3 cmd = {};
+ int cmd_size;
BUILD_BUG_ON(ARRAY_SIZE(cmd.block_list_array) <
APCI_WTAS_BLACK_LIST_MAX);
@@ -1121,7 +1217,7 @@ static void iwl_mvm_tas_init(struct iwl_mvm *mvm)
return;
}
- ret = iwl_acpi_get_tas(&mvm->fwrt, cmd.block_list_array, &list_size);
+ ret = iwl_acpi_get_tas(&mvm->fwrt, &cmd);
if (ret < 0) {
IWL_DEBUG_RADIO(mvm,
"TAS table invalid or unavailable. (%d)\n",
@@ -1129,15 +1225,32 @@ static void iwl_mvm_tas_init(struct iwl_mvm *mvm)
return;
}
- if (list_size < 0)
+ if (ret == 0)
return;
- /* list size if TAS enabled can only be non-negative */
- cmd.block_list_size = cpu_to_le32((u32)list_size);
+ if (!dmi_check_system(dmi_tas_approved_list)) {
+ IWL_DEBUG_RADIO(mvm,
+ "System vendor '%s' is not in the approved list, disabling TAS in US and Canada.\n",
+ dmi_get_system_info(DMI_SYS_VENDOR));
+ if ((!iwl_mvm_add_to_tas_block_list(cmd.block_list_array,
+ &cmd.block_list_size, IWL_TAS_US_MCC)) ||
+ (!iwl_mvm_add_to_tas_block_list(cmd.block_list_array,
+ &cmd.block_list_size, IWL_TAS_CANADA_MCC))) {
+ IWL_DEBUG_RADIO(mvm,
+ "Unable to add US/Canada to TAS block list, disabling TAS\n");
+ return;
+ }
+ }
+
+ cmd_size = iwl_fw_lookup_cmd_ver(mvm->fw, REGULATORY_AND_NVM_GROUP,
+ TAS_CONFIG,
+ IWL_FW_CMD_VER_UNKNOWN) < 3 ?
+ sizeof(struct iwl_tas_config_cmd_v2) :
+ sizeof(struct iwl_tas_config_cmd_v3);
ret = iwl_mvm_send_cmd_pdu(mvm, WIDE_ID(REGULATORY_AND_NVM_GROUP,
TAS_CONFIG),
- 0, sizeof(cmd), &cmd);
+ 0, cmd_size, &cmd);
if (ret < 0)
IWL_DEBUG_RADIO(mvm, "failed to send TAS_CONFIG (%d)\n", ret);
}
@@ -1336,6 +1449,7 @@ static u8 iwl_mvm_eval_dsm_rfi(struct iwl_mvm *mvm)
void iwl_mvm_get_acpi_tables(struct iwl_mvm *mvm)
{
}
+
#endif /* CONFIG_ACPI */
void iwl_mvm_send_recovery_cmd(struct iwl_mvm *mvm, u32 flags)
@@ -1401,7 +1515,6 @@ static int iwl_mvm_load_rt_fw(struct iwl_mvm *mvm)
if (iwl_mvm_has_unified_ucode(mvm))
return iwl_run_unified_mvm_ucode(mvm);
- WARN_ON(!mvm->nvm_data);
ret = iwl_run_init_mvm_ucode(mvm);
if (ret) {
@@ -1631,6 +1744,10 @@ int iwl_mvm_up(struct iwl_mvm *mvm)
else if (ret < 0)
goto error;
+ ret = iwl_mvm_sgom_init(mvm);
+ if (ret)
+ goto error;
+
iwl_mvm_tas_init(mvm);
iwl_mvm_leds_sync(mvm);
@@ -1705,20 +1822,6 @@ int iwl_mvm_load_d3_fw(struct iwl_mvm *mvm)
return ret;
}
-void iwl_mvm_rx_card_state_notif(struct iwl_mvm *mvm,
- struct iwl_rx_cmd_buffer *rxb)
-{
- struct iwl_rx_packet *pkt = rxb_addr(rxb);
- struct iwl_card_state_notif *card_state_notif = (void *)pkt->data;
- u32 flags = le32_to_cpu(card_state_notif->flags);
-
- IWL_DEBUG_RF_KILL(mvm, "Card state received: HW:%s SW:%s CT:%s\n",
- (flags & HW_CARD_DISABLED) ? "Kill" : "On",
- (flags & SW_CARD_DISABLED) ? "Kill" : "On",
- (flags & CT_KILL_CARD_DISABLED) ?
- "Reached" : "Not reached");
-}
-
void iwl_mvm_rx_mfuart_notif(struct iwl_mvm *mvm,
struct iwl_rx_cmd_buffer *rxb)
{
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/mac80211.c b/drivers/net/wireless/intel/iwlwifi/mvm/mac80211.c
index 897e3b9..65f4fe3 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/mac80211.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/mac80211.c
@@ -1,6 +1,6 @@
// SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause
/*
- * Copyright (C) 2012-2014, 2018-2020 Intel Corporation
+ * Copyright (C) 2012-2014, 2018-2021 Intel Corporation
* Copyright (C) 2013-2015 Intel Mobile Communications GmbH
* Copyright (C) 2016-2017 Intel Deutschland GmbH
*/
@@ -191,6 +191,7 @@ struct ieee80211_regdomain *iwl_mvm_get_regdomain(struct wiphy *wiphy,
if (IS_ERR_OR_NULL(resp)) {
IWL_DEBUG_LAR(mvm, "Could not get update from FW %d\n",
PTR_ERR_OR_ZERO(resp));
+ resp = NULL;
goto out;
}
@@ -212,7 +213,6 @@ struct ieee80211_regdomain *iwl_mvm_get_regdomain(struct wiphy *wiphy,
__le16_to_cpu(resp->cap), resp_ver);
/* Store the return source id */
src_id = resp->source_id;
- kfree(resp);
if (IS_ERR_OR_NULL(regd)) {
IWL_DEBUG_LAR(mvm, "Could not get parse update from FW %d\n",
PTR_ERR_OR_ZERO(regd));
@@ -224,7 +224,10 @@ struct ieee80211_regdomain *iwl_mvm_get_regdomain(struct wiphy *wiphy,
mvm->lar_regdom_set = true;
mvm->mcc_src = src_id;
+ iwl_mei_set_country_code(__le16_to_cpu(resp->mcc));
+
out:
+ kfree(resp);
return regd;
}
@@ -638,14 +641,21 @@ int iwl_mvm_mac_setup_register(struct iwl_mvm *mvm)
}
if (iwl_mvm_is_oce_supported(mvm)) {
+ u8 scan_ver = iwl_fw_lookup_cmd_ver(mvm->fw,
+ IWL_ALWAYS_LONG_GROUP,
+ SCAN_REQ_UMAC, 0);
+
wiphy_ext_feature_set(hw->wiphy,
NL80211_EXT_FEATURE_ACCEPT_BCAST_PROBE_RESP);
wiphy_ext_feature_set(hw->wiphy,
NL80211_EXT_FEATURE_FILS_MAX_CHANNEL_TIME);
wiphy_ext_feature_set(hw->wiphy,
- NL80211_EXT_FEATURE_OCE_PROBE_REQ_DEFERRAL_SUPPRESSION);
- wiphy_ext_feature_set(hw->wiphy,
NL80211_EXT_FEATURE_OCE_PROBE_REQ_HIGH_TX_RATE);
+
+ /* Old firmware also supports probe deferral and suppression */
+ if (scan_ver < 15)
+ wiphy_ext_feature_set(hw->wiphy,
+ NL80211_EXT_FEATURE_OCE_PROBE_REQ_DEFERRAL_SUPPRESSION);
}
if (mvm->nvm_data->sku_cap_11ax_enable &&
@@ -707,8 +717,7 @@ int iwl_mvm_mac_setup_register(struct iwl_mvm *mvm)
hw->netdev_features |= mvm->cfg->features;
if (!iwl_mvm_is_csum_supported(mvm))
- hw->netdev_features &= ~(IWL_TX_CSUM_NETIF_FLAGS |
- NETIF_F_RXCSUM);
+ hw->netdev_features &= ~IWL_CSUM_NETIF_FLAGS_MASK;
if (mvm->cfg->vht_mu_mimo_supported)
wiphy_ext_feature_set(hw->wiphy,
@@ -718,6 +727,8 @@ int iwl_mvm_mac_setup_register(struct iwl_mvm *mvm)
wiphy_ext_feature_set(hw->wiphy,
NL80211_EXT_FEATURE_PROTECTED_TWT);
+ iwl_mvm_vendor_cmds_register(mvm);
+
hw->wiphy->available_antennas_tx = iwl_mvm_get_valid_tx_ant(mvm);
hw->wiphy->available_antennas_rx = iwl_mvm_get_valid_rx_ant(mvm);
@@ -1084,6 +1095,27 @@ int __iwl_mvm_mac_start(struct iwl_mvm *mvm)
lockdep_assert_held(&mvm->mutex);
+ ret = iwl_mvm_mei_get_ownership(mvm);
+ if (ret)
+ return ret;
+
+ if (mvm->mei_nvm_data) {
+ /* We got the NIC, we can now free the MEI NVM data */
+ kfree(mvm->mei_nvm_data);
+ mvm->mei_nvm_data = NULL;
+
+ /*
+ * We can't free the nvm_data we allocated based on the SAP
+ * data because we registered to cfg80211 with the channels
+ * allocated on mvm->nvm_data. Keep a pointer in temp_nvm_data
+ * just in order to be able free it later.
+ * NULLify nvm_data so that we will read the NVM from the
+ * firmware this time.
+ */
+ mvm->temp_nvm_data = mvm->nvm_data;
+ mvm->nvm_data = NULL;
+ }
+
if (test_bit(IWL_MVM_STATUS_HW_RESTART_REQUESTED, &mvm->status)) {
/*
* Now convert the HW_RESTART_REQUESTED flag to IN_HW_RESTART
@@ -1144,6 +1176,8 @@ static int iwl_mvm_mac_start(struct ieee80211_hw *hw)
mutex_unlock(&mvm->mutex);
+ iwl_mvm_mei_set_sw_rfkill_state(mvm);
+
return ret;
}
@@ -1261,6 +1295,8 @@ static void iwl_mvm_mac_stop(struct ieee80211_hw *hw)
*/
flush_work(&mvm->roc_done_wk);
+ iwl_mvm_mei_set_sw_rfkill_state(mvm);
+
mutex_lock(&mvm->mutex);
__iwl_mvm_mac_stop(mvm);
mutex_unlock(&mvm->mutex);
@@ -1531,6 +1567,15 @@ static int iwl_mvm_mac_add_interface(struct ieee80211_hw *hw,
mvm->monitor_on = true;
iwl_mvm_vif_dbgfs_register(mvm, vif);
+
+ if (!test_bit(IWL_MVM_STATUS_IN_HW_RESTART, &mvm->status) &&
+ vif->type == NL80211_IFTYPE_STATION && !vif->p2p &&
+ !mvm->csme_vif && mvm->mei_registered) {
+ iwl_mei_set_nic_info(vif->addr, mvm->nvm_data->hw_addr);
+ iwl_mei_set_netdev(ieee80211_vif_to_wdev(vif)->netdev);
+ mvm->csme_vif = vif;
+ }
+
goto out_unlock;
out_unbind:
@@ -1583,6 +1628,11 @@ static void iwl_mvm_mac_remove_interface(struct ieee80211_hw *hw,
mutex_lock(&mvm->mutex);
+ if (vif == mvm->csme_vif) {
+ iwl_mei_set_netdev(NULL);
+ mvm->csme_vif = NULL;
+ }
+
probe_data = rcu_dereference_protected(mvmvif->probe_resp_data,
lockdep_is_held(&mvm->mutex));
RCU_INIT_POINTER(mvmvif->probe_resp_data, NULL);
@@ -1688,6 +1738,7 @@ static void iwl_mvm_recalc_multicast(struct iwl_mvm *mvm)
struct iwl_mvm_mc_iter_data iter_data = {
.mvm = mvm,
};
+ int ret;
lockdep_assert_held(&mvm->mutex);
@@ -1697,6 +1748,22 @@ static void iwl_mvm_recalc_multicast(struct iwl_mvm *mvm)
ieee80211_iterate_active_interfaces_atomic(
mvm->hw, IEEE80211_IFACE_ITER_NORMAL,
iwl_mvm_mc_iface_iterator, &iter_data);
+
+ /*
+ * Send a (synchronous) ech command so that we wait for the
+ * multiple asynchronous MCAST_FILTER_CMD commands sent by
+ * the interface iterator. Otherwise, we might get here over
+ * and over again (by userspace just sending a lot of these)
+ * and the CPU can send them faster than the firmware can
+ * process them.
+ * Note that the CPU is still faster - but with this we'll
+ * actually send fewer commands overall because the CPU will
+ * not schedule the work in mac80211 as frequently if it's
+ * still running when rescheduled (possibly multiple times).
+ */
+ ret = iwl_mvm_send_cmd_pdu(mvm, ECHO_CMD, 0, 0, NULL);
+ if (ret)
+ IWL_ERR(mvm, "Failed to synchronize multicast groups update\n");
}
static u64 iwl_mvm_prepare_multicast(struct ieee80211_hw *hw,
@@ -2148,24 +2215,24 @@ static void iwl_mvm_cfg_he_sta(struct iwl_mvm *mvm,
}
flags |= STA_CTXT_HE_PACKET_EXT;
- } else if ((sta->he_cap.he_cap_elem.phy_cap_info[9] &
- IEEE80211_HE_PHY_CAP9_NOMIMAL_PKT_PADDING_MASK) !=
- IEEE80211_HE_PHY_CAP9_NOMIMAL_PKT_PADDING_RESERVED) {
+ } else if (u8_get_bits(sta->he_cap.he_cap_elem.phy_cap_info[9],
+ IEEE80211_HE_PHY_CAP9_NOMINAL_PKT_PADDING_MASK)
+ != IEEE80211_HE_PHY_CAP9_NOMINAL_PKT_PADDING_RESERVED) {
int low_th = -1;
int high_th = -1;
/* Take the PPE thresholds from the nominal padding info */
- switch (sta->he_cap.he_cap_elem.phy_cap_info[9] &
- IEEE80211_HE_PHY_CAP9_NOMIMAL_PKT_PADDING_MASK) {
- case IEEE80211_HE_PHY_CAP9_NOMIMAL_PKT_PADDING_0US:
+ switch (u8_get_bits(sta->he_cap.he_cap_elem.phy_cap_info[9],
+ IEEE80211_HE_PHY_CAP9_NOMINAL_PKT_PADDING_MASK)) {
+ case IEEE80211_HE_PHY_CAP9_NOMINAL_PKT_PADDING_0US:
low_th = IWL_HE_PKT_EXT_NONE;
high_th = IWL_HE_PKT_EXT_NONE;
break;
- case IEEE80211_HE_PHY_CAP9_NOMIMAL_PKT_PADDING_8US:
+ case IEEE80211_HE_PHY_CAP9_NOMINAL_PKT_PADDING_8US:
low_th = IWL_HE_PKT_EXT_BPSK;
high_th = IWL_HE_PKT_EXT_NONE;
break;
- case IEEE80211_HE_PHY_CAP9_NOMIMAL_PKT_PADDING_16US:
+ case IEEE80211_HE_PHY_CAP9_NOMINAL_PKT_PADDING_16US:
low_th = IWL_HE_PKT_EXT_NONE;
high_th = IWL_HE_PKT_EXT_BPSK;
break;
@@ -2393,6 +2460,7 @@ static void iwl_mvm_bss_info_changed_station(struct iwl_mvm *mvm,
IEEE80211_SMPS_DYNAMIC);
}
} else if (mvmvif->ap_sta_id != IWL_MVM_INVALID_STA) {
+ iwl_mvm_mei_host_disassociated(mvm);
/*
* If update fails - SF might be running in associated
* mode while disassociated - which is forbidden.
@@ -3129,6 +3197,69 @@ static void iwl_mvm_reset_cca_40mhz_workaround(struct iwl_mvm *mvm,
}
}
+static void iwl_mvm_mei_host_associated(struct iwl_mvm *mvm,
+ struct ieee80211_vif *vif,
+ struct iwl_mvm_sta *mvm_sta)
+{
+#if IS_ENABLED(CONFIG_IWLMEI)
+ struct iwl_mvm_vif *mvmvif = iwl_mvm_vif_from_mac80211(vif);
+ struct iwl_mei_conn_info conn_info = {
+ .ssid_len = vif->bss_conf.ssid_len,
+ .channel = vif->bss_conf.chandef.chan->hw_value,
+ };
+
+ if (test_bit(IWL_MVM_STATUS_IN_HW_RESTART, &mvm->status))
+ return;
+
+ if (!mvm->mei_registered)
+ return;
+
+ switch (mvm_sta->pairwise_cipher) {
+ case WLAN_CIPHER_SUITE_CCMP:
+ conn_info.pairwise_cipher = IWL_MEI_CIPHER_CCMP;
+ break;
+ case WLAN_CIPHER_SUITE_GCMP:
+ conn_info.pairwise_cipher = IWL_MEI_CIPHER_GCMP;
+ break;
+ case WLAN_CIPHER_SUITE_GCMP_256:
+ conn_info.pairwise_cipher = IWL_MEI_CIPHER_GCMP_256;
+ break;
+ case 0:
+ /* open profile */
+ break;
+ default:
+ /* cipher not supported, don't send anything to iwlmei */
+ return;
+ }
+
+ switch (mvmvif->rekey_data.akm) {
+ case WLAN_AKM_SUITE_SAE & 0xff:
+ conn_info.auth_mode = IWL_MEI_AKM_AUTH_SAE;
+ break;
+ case WLAN_AKM_SUITE_PSK & 0xff:
+ conn_info.auth_mode = IWL_MEI_AKM_AUTH_RSNA_PSK;
+ break;
+ case WLAN_AKM_SUITE_8021X & 0xff:
+ conn_info.auth_mode = IWL_MEI_AKM_AUTH_RSNA;
+ break;
+ case 0:
+ /* open profile */
+ conn_info.auth_mode = IWL_MEI_AKM_AUTH_OPEN;
+ break;
+ default:
+ /* auth method / AKM not supported */
+ /* TODO: All the FT vesions of these? */
+ return;
+ }
+
+ memcpy(conn_info.ssid, vif->bss_conf.ssid, vif->bss_conf.ssid_len);
+ memcpy(conn_info.bssid, vif->bss_conf.bssid, ETH_ALEN);
+
+ /* TODO: add support for collocated AP data */
+ iwl_mei_host_associated(&conn_info, NULL);
+#endif
+}
+
static int iwl_mvm_mac_sta_state(struct ieee80211_hw *hw,
struct ieee80211_vif *vif,
struct ieee80211_sta *sta,
@@ -3273,12 +3404,18 @@ static int iwl_mvm_mac_sta_state(struct ieee80211_hw *hw,
* multicast data frames can be forwarded to the driver
*/
iwl_mvm_mac_ctxt_changed(mvm, vif, false, NULL);
+ iwl_mvm_mei_host_associated(mvm, vif, mvm_sta);
}
iwl_mvm_rs_rate_init(mvm, sta, mvmvif->phy_ctxt->channel->band,
true);
} else if (old_state == IEEE80211_STA_AUTHORIZED &&
new_state == IEEE80211_STA_ASSOC) {
+ /* once we move into assoc state, need to update rate scale to
+ * disable using wide bandwidth
+ */
+ iwl_mvm_rs_rate_init(mvm, sta, mvmvif->phy_ctxt->channel->band,
+ false);
if (!sta->tdls) {
/* Multicast data frames are no longer allowed */
iwl_mvm_mac_ctxt_changed(mvm, vif, false, NULL);
@@ -3301,16 +3438,16 @@ static int iwl_mvm_mac_sta_state(struct ieee80211_hw *hw,
if (vif->type == NL80211_IFTYPE_AP) {
mvmvif->ap_assoc_sta_count--;
iwl_mvm_mac_ctxt_changed(mvm, vif, false, NULL);
- } else if (vif->type == NL80211_IFTYPE_STATION && !sta->tdls) {
- /* remove session protection if still running */
+ } else if (vif->type == NL80211_IFTYPE_STATION && !sta->tdls)
iwl_mvm_stop_session_protection(mvm, vif);
- }
ret = 0;
} else if (old_state == IEEE80211_STA_AUTH &&
new_state == IEEE80211_STA_NONE) {
ret = 0;
} else if (old_state == IEEE80211_STA_NONE &&
new_state == IEEE80211_STA_NOTEXIST) {
+ if (vif->type == NL80211_IFTYPE_STATION && !sta->tdls)
+ iwl_mvm_stop_session_protection(mvm, vif);
ret = iwl_mvm_rm_sta(mvm, vif, sta);
if (sta->tdls) {
iwl_mvm_recalc_tdls_state(mvm, vif, false);
@@ -3476,12 +3613,15 @@ static int __iwl_mvm_mac_set_key(struct ieee80211_hw *hw,
{
struct iwl_mvm_vif *mvmvif = iwl_mvm_vif_from_mac80211(vif);
struct iwl_mvm *mvm = IWL_MAC80211_GET_MVM(hw);
- struct iwl_mvm_sta *mvmsta;
+ struct iwl_mvm_sta *mvmsta = NULL;
struct iwl_mvm_key_pn *ptk_pn;
int keyidx = key->keyidx;
int ret, i;
u8 key_offset;
+ if (sta)
+ mvmsta = iwl_mvm_sta_from_mac80211(sta);
+
switch (key->cipher) {
case WLAN_CIPHER_SUITE_TKIP:
if (!mvm->trans->trans_cfg->gen2) {
@@ -3582,7 +3722,7 @@ static int __iwl_mvm_mac_set_key(struct ieee80211_hw *hw,
}
if (!test_bit(IWL_MVM_STATUS_IN_HW_RESTART, &mvm->status) &&
- sta && iwl_mvm_has_new_rx_api(mvm) &&
+ mvmsta && iwl_mvm_has_new_rx_api(mvm) &&
key->flags & IEEE80211_KEY_FLAG_PAIRWISE &&
(key->cipher == WLAN_CIPHER_SUITE_CCMP ||
key->cipher == WLAN_CIPHER_SUITE_GCMP ||
@@ -3590,7 +3730,6 @@ static int __iwl_mvm_mac_set_key(struct ieee80211_hw *hw,
struct ieee80211_key_seq seq;
int tid, q;
- mvmsta = iwl_mvm_sta_from_mac80211(sta);
WARN_ON(rcu_access_pointer(mvmsta->ptk_pn[keyidx]));
ptk_pn = kzalloc(struct_size(ptk_pn, q,
mvm->trans->num_rx_queues),
@@ -3617,6 +3756,9 @@ static int __iwl_mvm_mac_set_key(struct ieee80211_hw *hw,
else
key_offset = STA_KEY_IDX_INVALID;
+ if (mvmsta && key->flags & IEEE80211_KEY_FLAG_PAIRWISE)
+ mvmsta->pairwise_cipher = key->cipher;
+
IWL_DEBUG_MAC80211(mvm, "set hwcrypto key\n");
ret = iwl_mvm_set_sta_key(mvm, vif, sta, key, key_offset);
if (ret) {
@@ -3657,12 +3799,11 @@ static int __iwl_mvm_mac_set_key(struct ieee80211_hw *hw,
break;
}
- if (sta && iwl_mvm_has_new_rx_api(mvm) &&
+ if (mvmsta && iwl_mvm_has_new_rx_api(mvm) &&
key->flags & IEEE80211_KEY_FLAG_PAIRWISE &&
(key->cipher == WLAN_CIPHER_SUITE_CCMP ||
key->cipher == WLAN_CIPHER_SUITE_GCMP ||
key->cipher == WLAN_CIPHER_SUITE_GCMP_256)) {
- mvmsta = iwl_mvm_sta_from_mac80211(sta);
ptk_pn = rcu_dereference_protected(
mvmsta->ptk_pn[keyidx],
lockdep_is_held(&mvm->mutex));
@@ -5382,6 +5523,10 @@ static bool iwl_mvm_mac_can_aggregate(struct ieee80211_hw *hw,
{
struct iwl_mvm *mvm = IWL_MAC80211_GET_MVM(hw);
+ if (mvm->trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_BZ)
+ return iwl_mvm_tx_csum_bz(mvm, head, true) ==
+ iwl_mvm_tx_csum_bz(mvm, skb, true);
+
/* For now don't aggregate IPv6 in AMSDU */
if (skb->protocol != htons(ETH_P_IP))
return false;
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/mvm.h b/drivers/net/wireless/intel/iwlwifi/mvm/mvm.h
index a72d850..1dcbb0e 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/mvm.h
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/mvm.h
@@ -30,6 +30,7 @@
#include "fw/runtime.h"
#include "fw/dbg.h"
#include "fw/acpi.h"
+#include "mei/iwl-mei.h"
#include "iwl-nvm-parse.h"
#include <linux/average.h>
@@ -93,11 +94,10 @@ struct iwl_mvm_phy_ctxt {
enum nl80211_chan_width width;
- /*
- * TODO: This should probably be removed. Currently here only for rate
- * scaling algorithm
- */
struct ieee80211_channel *channel;
+
+ /* track for RLC config command */
+ u32 center_freq1;
};
struct iwl_mvm_time_event_data {
@@ -830,6 +830,18 @@ struct iwl_mvm {
const char *nvm_file_name;
struct iwl_nvm_data *nvm_data;
+ struct iwl_mei_nvm *mei_nvm_data;
+ struct iwl_mvm_csme_conn_info __rcu *csme_conn_info;
+ bool mei_rfkill_blocked;
+ bool mei_registered;
+ struct work_struct sap_connected_wk;
+
+ /*
+ * NVM built based on the SAP data but that we can't free even after
+ * we get ownership because it contains the cfg80211's channel.
+ */
+ struct iwl_nvm_data *temp_nvm_data;
+
/* NVM sections */
struct iwl_nvm_section nvm_sections[NVM_MAX_NUM_SECTIONS];
@@ -1021,6 +1033,8 @@ struct iwl_mvm {
/* Indicate if 32Khz external clock is valid */
u32 ext_clock_valid;
+ /* This vif used by CSME to send / receive traffic */
+ struct ieee80211_vif *csme_vif;
struct ieee80211_vif __rcu *csa_vif;
struct ieee80211_vif __rcu *csa_tx_blocked_vif;
u8 csa_tx_block_bcn_timeout;
@@ -1123,6 +1137,8 @@ struct iwl_mvm {
* @IWL_MVM_STATUS_FIRMWARE_RUNNING: firmware is running
* @IWL_MVM_STATUS_NEED_FLUSH_P2P: need to flush P2P bcast STA
* @IWL_MVM_STATUS_IN_D3: in D3 (or at least about to go into it)
+ * @IWL_MVM_STATUS_SUPPRESS_ERROR_LOG_ONCE: suppress one error log
+ * if this is set, when intentionally triggered
* @IWL_MVM_STATUS_STARTING: starting mac,
* used to disable restart flow while in STARTING state
*/
@@ -1136,9 +1152,15 @@ enum iwl_mvm_status {
IWL_MVM_STATUS_FIRMWARE_RUNNING,
IWL_MVM_STATUS_NEED_FLUSH_P2P,
IWL_MVM_STATUS_IN_D3,
+ IWL_MVM_STATUS_SUPPRESS_ERROR_LOG_ONCE,
IWL_MVM_STATUS_STARTING,
};
+struct iwl_mvm_csme_conn_info {
+ struct rcu_head rcu_head;
+ struct iwl_mei_conn_info conn_info;
+};
+
/* Keep track of completed init configuration */
enum iwl_mvm_init_status {
IWL_MVM_INIT_STATUS_THERMAL_INIT_COMPLETE = BIT(0),
@@ -1496,6 +1518,7 @@ void iwl_mvm_mac_itxq_xmit(struct ieee80211_hw *hw, struct ieee80211_txq *txq);
unsigned int iwl_mvm_max_amsdu_size(struct iwl_mvm *mvm,
struct ieee80211_sta *sta,
unsigned int tid);
+u32 iwl_mvm_tx_csum_bz(struct iwl_mvm *mvm, struct sk_buff *skb, bool amsdu);
#ifdef CONFIG_IWLWIFI_DEBUG
const char *iwl_mvm_get_tx_fail_reason(u32 status);
@@ -1601,8 +1624,6 @@ void iwl_mvm_rx_ba_notif(struct iwl_mvm *mvm, struct iwl_rx_cmd_buffer *rxb);
void iwl_mvm_rx_ant_coupling_notif(struct iwl_mvm *mvm,
struct iwl_rx_cmd_buffer *rxb);
void iwl_mvm_rx_fw_error(struct iwl_mvm *mvm, struct iwl_rx_cmd_buffer *rxb);
-void iwl_mvm_rx_card_state_notif(struct iwl_mvm *mvm,
- struct iwl_rx_cmd_buffer *rxb);
void iwl_mvm_rx_mfuart_notif(struct iwl_mvm *mvm,
struct iwl_rx_cmd_buffer *rxb);
void iwl_mvm_rx_shared_mem_cfg_notif(struct iwl_mvm *mvm,
@@ -1942,6 +1963,17 @@ void iwl_mvm_enter_ctkill(struct iwl_mvm *mvm);
int iwl_mvm_send_temp_report_ths_cmd(struct iwl_mvm *mvm);
int iwl_mvm_ctdp_command(struct iwl_mvm *mvm, u32 op, u32 budget);
+#if IS_ENABLED(CONFIG_IWLMEI)
+
+/* vendor commands */
+void iwl_mvm_vendor_cmds_register(struct iwl_mvm *mvm);
+
+#else
+
+static inline void iwl_mvm_vendor_cmds_register(struct iwl_mvm *mvm) {}
+
+#endif
+
/* Location Aware Regulatory */
struct iwl_mcc_update_resp *
iwl_mvm_update_mcc(struct iwl_mvm *mvm, const char *alpha2,
@@ -2161,4 +2193,47 @@ enum iwl_location_cipher iwl_mvm_cipher_to_location_cipher(u32 cipher)
return IWL_LOCATION_CIPHER_INVALID;
}
}
+
+struct iwl_mvm_csme_conn_info *iwl_mvm_get_csme_conn_info(struct iwl_mvm *mvm);
+static inline int iwl_mvm_mei_get_ownership(struct iwl_mvm *mvm)
+{
+ if (mvm->mei_registered)
+ return iwl_mei_get_ownership();
+ return 0;
+}
+
+static inline void iwl_mvm_mei_tx_copy_to_csme(struct iwl_mvm *mvm,
+ struct sk_buff *skb,
+ unsigned int ivlen)
+{
+ if (mvm->mei_registered)
+ iwl_mei_tx_copy_to_csme(skb, ivlen);
+}
+
+static inline void iwl_mvm_mei_host_disassociated(struct iwl_mvm *mvm)
+{
+ if (mvm->mei_registered)
+ iwl_mei_host_disassociated();
+}
+
+static inline void iwl_mvm_mei_device_down(struct iwl_mvm *mvm)
+{
+ if (mvm->mei_registered)
+ iwl_mei_device_down();
+}
+
+static inline void iwl_mvm_mei_set_sw_rfkill_state(struct iwl_mvm *mvm)
+{
+ bool sw_rfkill =
+ mvm->hw_registered ? rfkill_blocked(mvm->hw->wiphy->rfkill) : false;
+
+ if (mvm->mei_registered)
+ iwl_mei_set_rfkill_state(iwl_mvm_is_radio_killed(mvm),
+ sw_rfkill);
+}
+
+void iwl_mvm_send_roaming_forbidden_event(struct iwl_mvm *mvm,
+ struct ieee80211_vif *vif,
+ bool forbidden);
+
#endif /* __IWL_MVM_H__ */
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/ops.c b/drivers/net/wireless/intel/iwlwifi/mvm/ops.c
index cd08e28..87630d3 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/ops.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/ops.c
@@ -5,6 +5,7 @@
* Copyright (C) 2016-2017 Intel Deutschland GmbH
*/
#include <linux/module.h>
+#include <linux/rtnetlink.h>
#include <linux/vmalloc.h>
#include <net/mac80211.h>
@@ -26,6 +27,7 @@
#include "time-event.h"
#include "fw-api.h"
#include "fw/acpi.h"
+#include "fw/uefi.h"
#define DRV_DESCRIPTION "The new Intel(R) wireless AGN driver for Linux"
MODULE_DESCRIPTION(DRV_DESCRIPTION);
@@ -78,7 +80,7 @@ static void iwl_mvm_nic_config(struct iwl_op_mode *op_mode)
{
struct iwl_mvm *mvm = IWL_OP_MODE_GET_MVM(op_mode);
u8 radio_cfg_type, radio_cfg_step, radio_cfg_dash;
- u32 reg_val = 0;
+ u32 reg_val;
u32 phy_config = iwl_mvm_get_phy_config(mvm);
radio_cfg_type = (phy_config & FW_PHY_CFG_RADIO_TYPE) >>
@@ -89,10 +91,7 @@ static void iwl_mvm_nic_config(struct iwl_op_mode *op_mode)
FW_PHY_CFG_RADIO_DASH_POS;
/* SKU control */
- reg_val |= CSR_HW_REV_STEP(mvm->trans->hw_rev) <<
- CSR_HW_IF_CONFIG_REG_POS_MAC_STEP;
- reg_val |= CSR_HW_REV_DASH(mvm->trans->hw_rev) <<
- CSR_HW_IF_CONFIG_REG_POS_MAC_DASH;
+ reg_val = CSR_HW_REV_STEP_DASH(mvm->trans->hw_rev);
/* radio configuration */
reg_val |= radio_cfg_type << CSR_HW_IF_CONFIG_REG_POS_PHY_TYPE;
@@ -117,8 +116,7 @@ static void iwl_mvm_nic_config(struct iwl_op_mode *op_mode)
reg_val |= CSR_HW_IF_CONFIG_REG_D3_DEBUG;
iwl_trans_set_bits_mask(mvm->trans, CSR_HW_IF_CONFIG_REG,
- CSR_HW_IF_CONFIG_REG_MSK_MAC_DASH |
- CSR_HW_IF_CONFIG_REG_MSK_MAC_STEP |
+ CSR_HW_IF_CONFIG_REG_MSK_MAC_STEP_DASH |
CSR_HW_IF_CONFIG_REG_MSK_PHY_TYPE |
CSR_HW_IF_CONFIG_REG_MSK_PHY_STEP |
CSR_HW_IF_CONFIG_REG_MSK_PHY_DASH |
@@ -260,6 +258,7 @@ enum iwl_rx_handler_context {
/**
* struct iwl_rx_handlers handler for FW notification
* @cmd_id: command id
+ * @min_size: minimum size to expect for the notification
* @context: see &iwl_rx_handler_context
* @fn: the function is called when notification is received
*/
@@ -334,9 +333,6 @@ static const struct iwl_rx_handlers iwl_mvm_rx_handlers[] = {
iwl_mvm_rx_umac_scan_iter_complete_notif, RX_HANDLER_SYNC,
struct iwl_umac_scan_iter_complete_notif),
- RX_HANDLER(CARD_STATE_NOTIFICATION, iwl_mvm_rx_card_state_notif,
- RX_HANDLER_SYNC, struct iwl_card_state_notif),
-
RX_HANDLER(MISSED_BEACONS_NOTIFICATION, iwl_mvm_rx_missed_beacons_notif,
RX_HANDLER_SYNC, struct iwl_missed_beacons_notif),
@@ -457,7 +453,6 @@ static const struct iwl_hcmd_names iwl_mvm_legacy_names[] = {
HCMD_NAME(STATISTICS_NOTIFICATION),
HCMD_NAME(EOSP_NOTIFICATION),
HCMD_NAME(REDUCE_TX_POWER_CMD),
- HCMD_NAME(CARD_STATE_NOTIFICATION),
HCMD_NAME(MISSED_BEACONS_NOTIFICATION),
HCMD_NAME(TDLS_CONFIG_CMD),
HCMD_NAME(MAC_PM_POWER_TABLE),
@@ -502,6 +497,9 @@ static const struct iwl_hcmd_names iwl_mvm_system_names[] = {
HCMD_NAME(SHARED_MEM_CFG_CMD),
HCMD_NAME(INIT_EXTENDED_CFG_CMD),
HCMD_NAME(FW_ERROR_RECOVERY_CMD),
+ HCMD_NAME(RFI_CONFIG_CMD),
+ HCMD_NAME(RFI_GET_FREQ_TABLE_CMD),
+ HCMD_NAME(SYSTEM_FEATURES_CONTROL_CMD),
};
/* Please keep this array *SORTED* by hex value.
@@ -534,6 +532,7 @@ static const struct iwl_hcmd_names iwl_mvm_data_path_names[] = {
HCMD_NAME(UPDATE_MU_GROUPS_CMD),
HCMD_NAME(TRIGGER_RX_QUEUES_NOTIF_CMD),
HCMD_NAME(STA_HE_CTXT_CMD),
+ HCMD_NAME(RLC_CONFIG_CMD),
HCMD_NAME(RFH_QUEUE_CONFIG_CMD),
HCMD_NAME(TLC_MNG_CONFIG_CMD),
HCMD_NAME(CHEST_COLLECTOR_FILTER_CONFIG_CMD),
@@ -683,14 +682,45 @@ static const struct iwl_fw_runtime_ops iwl_mvm_fwrt_ops = {
static int iwl_mvm_start_get_nvm(struct iwl_mvm *mvm)
{
+ struct iwl_trans *trans = mvm->trans;
int ret;
+ if (trans->csme_own) {
+ if (WARN(!mvm->mei_registered,
+ "csme is owner, but we aren't registered to iwlmei\n"))
+ goto get_nvm_from_fw;
+
+ mvm->mei_nvm_data = iwl_mei_get_nvm();
+ if (mvm->mei_nvm_data) {
+ /*
+ * mvm->mei_nvm_data is set and because of that,
+ * we'll load the NVM from the FW when we'll get
+ * ownership.
+ */
+ mvm->nvm_data =
+ iwl_parse_mei_nvm_data(trans, trans->cfg,
+ mvm->mei_nvm_data, mvm->fw);
+ return 0;
+ }
+
+ IWL_ERR(mvm,
+ "Got a NULL NVM from CSME, trying to get it from the device\n");
+ }
+
+get_nvm_from_fw:
rtnl_lock();
wiphy_lock(mvm->hw->wiphy);
mutex_lock(&mvm->mutex);
- ret = iwl_run_init_mvm_ucode(mvm);
+ ret = iwl_trans_start_hw(mvm->trans);
+ if (ret) {
+ mutex_unlock(&mvm->mutex);
+ wiphy_unlock(mvm->hw->wiphy);
+ rtnl_unlock();
+ return ret;
+ }
+ ret = iwl_run_init_mvm_ucode(mvm);
if (ret && ret != -ERFKILL)
iwl_fw_dbg_error_collect(&mvm->fwrt, FW_DBG_TRIGGER_DRIVER);
if (!ret && iwl_mvm_is_lar_supported(mvm)) {
@@ -705,7 +735,7 @@ static int iwl_mvm_start_get_nvm(struct iwl_mvm *mvm)
wiphy_unlock(mvm->hw->wiphy);
rtnl_unlock();
- if (ret < 0)
+ if (ret)
IWL_ERR(mvm, "Failed to run INIT ucode: %d\n", ret);
return ret;
@@ -713,6 +743,7 @@ static int iwl_mvm_start_get_nvm(struct iwl_mvm *mvm)
static int iwl_mvm_start_post_nvm(struct iwl_mvm *mvm)
{
+ struct iwl_mvm_csme_conn_info *csme_conn_info __maybe_unused;
int ret;
iwl_mvm_toggle_tx_ant(mvm, &mvm->mgmt_last_antenna_idx);
@@ -720,10 +751,17 @@ static int iwl_mvm_start_post_nvm(struct iwl_mvm *mvm)
ret = iwl_mvm_mac_setup_register(mvm);
if (ret)
return ret;
+
mvm->hw_registered = true;
iwl_mvm_dbgfs_register(mvm);
+ wiphy_rfkill_set_hw_state_reason(mvm->hw->wiphy,
+ mvm->mei_rfkill_blocked,
+ RFKILL_HARD_BLOCK_NOT_OWNER);
+
+ iwl_mvm_mei_set_sw_rfkill_state(mvm);
+
return 0;
}
@@ -904,6 +942,109 @@ static const struct iwl_dump_sanitize_ops iwl_mvm_sanitize_ops = {
.frob_mem = iwl_mvm_frob_mem,
};
+static void iwl_mvm_me_conn_status(void *priv, const struct iwl_mei_conn_info *conn_info)
+{
+ struct iwl_mvm *mvm = priv;
+ struct iwl_mvm_csme_conn_info *prev_conn_info, *curr_conn_info;
+
+ /*
+ * This is protected by the guarantee that this function will not be
+ * called twice on two different threads
+ */
+ prev_conn_info = rcu_dereference_protected(mvm->csme_conn_info, true);
+
+ curr_conn_info = kzalloc(sizeof(*curr_conn_info), GFP_KERNEL);
+ if (!curr_conn_info)
+ return;
+
+ curr_conn_info->conn_info = *conn_info;
+
+ rcu_assign_pointer(mvm->csme_conn_info, curr_conn_info);
+
+ if (prev_conn_info)
+ kfree_rcu(prev_conn_info, rcu_head);
+}
+
+static void iwl_mvm_mei_rfkill(void *priv, bool blocked)
+{
+ struct iwl_mvm *mvm = priv;
+
+ mvm->mei_rfkill_blocked = blocked;
+ if (!mvm->hw_registered)
+ return;
+
+ wiphy_rfkill_set_hw_state_reason(mvm->hw->wiphy,
+ mvm->mei_rfkill_blocked,
+ RFKILL_HARD_BLOCK_NOT_OWNER);
+}
+
+static void iwl_mvm_mei_roaming_forbidden(void *priv, bool forbidden)
+{
+ struct iwl_mvm *mvm = priv;
+
+ if (!mvm->hw_registered || !mvm->csme_vif)
+ return;
+
+ iwl_mvm_send_roaming_forbidden_event(mvm, mvm->csme_vif, forbidden);
+}
+
+static void iwl_mvm_sap_connected_wk(struct work_struct *wk)
+{
+ struct iwl_mvm *mvm =
+ container_of(wk, struct iwl_mvm, sap_connected_wk);
+ int ret;
+
+ ret = iwl_mvm_start_get_nvm(mvm);
+ if (ret)
+ goto out_free;
+
+ ret = iwl_mvm_start_post_nvm(mvm);
+ if (ret)
+ goto out_free;
+
+ return;
+
+out_free:
+ IWL_ERR(mvm, "Couldn't get started...\n");
+ iwl_mei_start_unregister();
+ iwl_mei_unregister_complete();
+ iwl_fw_flush_dumps(&mvm->fwrt);
+ iwl_mvm_thermal_exit(mvm);
+ iwl_fw_runtime_free(&mvm->fwrt);
+ iwl_phy_db_free(mvm->phy_db);
+ kfree(mvm->scan_cmd);
+ iwl_trans_op_mode_leave(mvm->trans);
+ kfree(mvm->nvm_data);
+ kfree(mvm->mei_nvm_data);
+
+ ieee80211_free_hw(mvm->hw);
+}
+
+static void iwl_mvm_mei_sap_connected(void *priv)
+{
+ struct iwl_mvm *mvm = priv;
+
+ if (!mvm->hw_registered)
+ schedule_work(&mvm->sap_connected_wk);
+}
+
+static void iwl_mvm_mei_nic_stolen(void *priv)
+{
+ struct iwl_mvm *mvm = priv;
+
+ rtnl_lock();
+ cfg80211_shutdown_all_interfaces(mvm->hw->wiphy);
+ rtnl_unlock();
+}
+
+static const struct iwl_mei_ops mei_ops = {
+ .me_conn_status = iwl_mvm_me_conn_status,
+ .rfkill = iwl_mvm_mei_rfkill,
+ .roaming_forbidden = iwl_mvm_mei_roaming_forbidden,
+ .sap_connected = iwl_mvm_mei_sap_connected,
+ .nic_stolen = iwl_mvm_mei_nic_stolen,
+};
+
static struct iwl_op_mode *
iwl_op_mode_mvm_start(struct iwl_trans *trans, const struct iwl_cfg *cfg,
const struct iwl_fw *fw, struct dentry *dbgfs_dir)
@@ -915,9 +1056,9 @@ iwl_op_mode_mvm_start(struct iwl_trans *trans, const struct iwl_cfg *cfg,
static const u8 no_reclaim_cmds[] = {
TX_CMD,
};
- int err, scan_size;
+ int scan_size;
u32 min_backoff;
- enum iwl_amsdu_size rb_size_default;
+ struct iwl_mvm_csme_conn_info *csme_conn_info __maybe_unused;
/*
* We use IWL_MVM_STATION_COUNT_MAX to check the validity of the station
@@ -956,6 +1097,7 @@ iwl_op_mode_mvm_start(struct iwl_trans *trans, const struct iwl_cfg *cfg,
&iwl_mvm_sanitize_ops, mvm, dbgfs_dir);
iwl_mvm_get_acpi_tables(mvm);
+ iwl_uefi_get_sgom_table(trans, &mvm->fwrt);
mvm->init_status = 0;
@@ -1017,6 +1159,7 @@ iwl_op_mode_mvm_start(struct iwl_trans *trans, const struct iwl_cfg *cfg,
INIT_WORK(&mvm->async_handlers_wk, iwl_mvm_async_handlers_wk);
INIT_WORK(&mvm->roc_done_wk, iwl_mvm_roc_done_wk);
+ INIT_WORK(&mvm->sap_connected_wk, iwl_mvm_sap_connected_wk);
INIT_DELAYED_WORK(&mvm->tdls_cs.dwork, iwl_mvm_tdls_ch_switch_work);
INIT_DELAYED_WORK(&mvm->scan_timeout_dwork, iwl_mvm_scan_timeout_wk);
INIT_WORK(&mvm->add_stream_wk, iwl_mvm_add_new_dqa_stream_wk);
@@ -1058,14 +1201,9 @@ iwl_op_mode_mvm_start(struct iwl_trans *trans, const struct iwl_cfg *cfg,
trans_cfg.no_reclaim_cmds = no_reclaim_cmds;
trans_cfg.n_no_reclaim_cmds = ARRAY_SIZE(no_reclaim_cmds);
- if (mvm->trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_AX210)
- rb_size_default = IWL_AMSDU_2K;
- else
- rb_size_default = IWL_AMSDU_4K;
-
switch (iwlwifi_mod_params.amsdu_size) {
case IWL_AMSDU_DEF:
- trans_cfg.rx_buf_size = rb_size_default;
+ trans_cfg.rx_buf_size = IWL_AMSDU_4K;
break;
case IWL_AMSDU_4K:
trans_cfg.rx_buf_size = IWL_AMSDU_4K;
@@ -1079,7 +1217,7 @@ iwl_op_mode_mvm_start(struct iwl_trans *trans, const struct iwl_cfg *cfg,
default:
pr_err("%s: Unsupported amsdu_size: %d\n", KBUILD_MODNAME,
iwlwifi_mod_params.amsdu_size);
- trans_cfg.rx_buf_size = rb_size_default;
+ trans_cfg.rx_buf_size = IWL_AMSDU_4K;
}
trans->wide_cmd_header = true;
@@ -1139,10 +1277,6 @@ iwl_op_mode_mvm_start(struct iwl_trans *trans, const struct iwl_cfg *cfg,
IWL_DEBUG_EEPROM(mvm->trans->dev,
"working without external nvm file\n");
- err = iwl_trans_start_hw(mvm->trans);
- if (err)
- goto out_free;
-
scan_size = iwl_mvm_scan_size(mvm);
mvm->scan_cmd = kmalloc(scan_size, GFP_KERNEL);
@@ -1167,8 +1301,20 @@ iwl_op_mode_mvm_start(struct iwl_trans *trans, const struct iwl_cfg *cfg,
mvm->debugfs_dir = dbgfs_dir;
- if (iwl_mvm_start_get_nvm(mvm))
+ mvm->mei_registered = !iwl_mei_register(mvm, &mei_ops);
+
+ if (iwl_mvm_start_get_nvm(mvm)) {
+ /*
+ * Getting NVM failed while CSME is the owner, but we are
+ * registered to MEI, we'll get the NVM later when it'll be
+ * possible to get it from CSME.
+ */
+ if (trans->csme_own && mvm->mei_registered)
+ return op_mode;
+
goto out_thermal_exit;
+ }
+
if (iwl_mvm_start_post_nvm(mvm))
goto out_thermal_exit;
@@ -1177,6 +1323,10 @@ iwl_op_mode_mvm_start(struct iwl_trans *trans, const struct iwl_cfg *cfg,
out_thermal_exit:
iwl_mvm_thermal_exit(mvm);
+ if (mvm->mei_registered) {
+ iwl_mei_start_unregister();
+ iwl_mei_unregister_complete();
+ }
out_free:
iwl_fw_flush_dumps(&mvm->fwrt);
iwl_fw_runtime_free(&mvm->fwrt);
@@ -1203,6 +1353,7 @@ void iwl_mvm_stop_device(struct iwl_mvm *mvm)
iwl_trans_stop_device(mvm->trans);
iwl_free_fw_paging(&mvm->fwrt);
iwl_fw_dump_conf_clear(&mvm->fwrt);
+ iwl_mvm_mei_device_down(mvm);
}
static void iwl_op_mode_mvm_stop(struct iwl_op_mode *op_mode)
@@ -1210,11 +1361,33 @@ static void iwl_op_mode_mvm_stop(struct iwl_op_mode *op_mode)
struct iwl_mvm *mvm = IWL_OP_MODE_GET_MVM(op_mode);
int i;
+ if (mvm->mei_registered) {
+ rtnl_lock();
+ iwl_mei_set_netdev(NULL);
+ rtnl_unlock();
+ iwl_mei_start_unregister();
+ }
+
+ /*
+ * After we unregister from mei, the worker can't be scheduled
+ * anymore.
+ */
+ cancel_work_sync(&mvm->sap_connected_wk);
+
iwl_mvm_leds_exit(mvm);
iwl_mvm_thermal_exit(mvm);
- ieee80211_unregister_hw(mvm->hw);
+ /*
+ * If we couldn't get ownership on the device and we couldn't
+ * get the NVM from CSME, we haven't registered to mac80211.
+ * In that case, we didn't fail op_mode_start, because we are
+ * waiting for CSME to allow us to get the NVM to register to
+ * mac80211. If that didn't happen, we haven't registered to
+ * mac80211, hence the if below.
+ */
+ if (mvm->hw_registered)
+ ieee80211_unregister_hw(mvm->hw);
kfree(mvm->scan_cmd);
kfree(mvm->mcast_filter_cmd);
@@ -1229,6 +1402,9 @@ static void iwl_op_mode_mvm_stop(struct iwl_op_mode *op_mode)
mvm->phy_db = NULL;
kfree(mvm->nvm_data);
+ kfree(mvm->mei_nvm_data);
+ kfree(rcu_access_pointer(mvm->csme_conn_info));
+ kfree(mvm->temp_nvm_data);
for (i = 0; i < NVM_MAX_NUM_SECTIONS; i++)
kfree(mvm->nvm_sections[i].data);
@@ -1237,6 +1413,9 @@ static void iwl_op_mode_mvm_stop(struct iwl_op_mode *op_mode)
iwl_fw_runtime_free(&mvm->fwrt);
mutex_destroy(&mvm->mutex);
+ if (mvm->mei_registered)
+ iwl_mei_unregister_complete();
+
ieee80211_free_hw(mvm->hw);
}
@@ -1519,6 +1698,12 @@ void iwl_mvm_set_hw_ctkill_state(struct iwl_mvm *mvm, bool state)
iwl_mvm_set_rfkill_state(mvm);
}
+struct iwl_mvm_csme_conn_info *iwl_mvm_get_csme_conn_info(struct iwl_mvm *mvm)
+{
+ return rcu_dereference_protected(mvm->csme_conn_info,
+ lockdep_is_held(&mvm->mutex));
+}
+
static bool iwl_mvm_set_hw_rfkill_state(struct iwl_op_mode *op_mode, bool state)
{
struct iwl_mvm *mvm = IWL_OP_MODE_GET_MVM(op_mode);
@@ -1657,9 +1842,16 @@ void iwl_mvm_nic_restart(struct iwl_mvm *mvm, bool fw_error)
iwl_fw_error_collect(&mvm->fwrt, false);
- if (fw_error && mvm->fw_restart > 0)
+ if (fw_error && mvm->fw_restart > 0) {
mvm->fw_restart--;
- ieee80211_restart_hw(mvm->hw);
+ ieee80211_restart_hw(mvm->hw);
+ } else if (mvm->fwrt.trans->dbg.restart_required) {
+ IWL_DEBUG_INFO(mvm, "FW restart requested after debug collection\n");
+ mvm->fwrt.trans->dbg.restart_required = FALSE;
+ ieee80211_restart_hw(mvm->hw);
+ } else if (mvm->trans->trans_cfg->device_family <= IWL_DEVICE_FAMILY_8000) {
+ ieee80211_restart_hw(mvm->hw);
+ }
}
}
@@ -1667,7 +1859,9 @@ static void iwl_mvm_nic_error(struct iwl_op_mode *op_mode, bool sync)
{
struct iwl_mvm *mvm = IWL_OP_MODE_GET_MVM(op_mode);
- if (!test_bit(STATUS_TRANS_DEAD, &mvm->trans->status))
+ if (!test_bit(STATUS_TRANS_DEAD, &mvm->trans->status) &&
+ !test_and_clear_bit(IWL_MVM_STATUS_SUPPRESS_ERROR_LOG_ONCE,
+ &mvm->status))
iwl_mvm_dump_nic_error_log(mvm);
if (sync) {
@@ -1688,7 +1882,7 @@ static void iwl_mvm_nic_error(struct iwl_op_mode *op_mode, bool sync)
if (!test_bit(IWL_MVM_STATUS_FIRMWARE_RUNNING, &mvm->status))
return;
- iwl_mvm_nic_restart(mvm, true);
+ iwl_mvm_nic_restart(mvm, false);
}
static void iwl_mvm_cmd_queue_full(struct iwl_op_mode *op_mode)
@@ -1737,6 +1931,9 @@ static void iwl_mvm_rx_mq_rss(struct iwl_op_mode *op_mode,
struct iwl_rx_packet *pkt = rxb_addr(rxb);
u16 cmd = WIDE_ID(pkt->hdr.group_id, pkt->hdr.cmd);
+ if (unlikely(queue >= mvm->trans->num_rx_queues))
+ return;
+
if (unlikely(cmd == WIDE_ID(LEGACY_GROUP, FRAME_RELEASE)))
iwl_mvm_rx_frame_release(mvm, napi, rxb, queue);
else if (unlikely(cmd == WIDE_ID(DATA_PATH_GROUP,
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/phy-ctxt.c b/drivers/net/wireless/intel/iwlwifi/mvm/phy-ctxt.c
index 035336a..9af40b0 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/phy-ctxt.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/phy-ctxt.c
@@ -157,8 +157,43 @@ static void iwl_mvm_phy_ctxt_cmd_data(struct iwl_mvm *mvm,
/* Set the channel info data */
iwl_mvm_set_chan_info_chandef(mvm, &cmd->ci, chandef);
- iwl_mvm_phy_ctxt_set_rxchain(mvm, ctxt, &cmd->rxchain_info,
+ /* we only support RLC command version 2 */
+ if (iwl_fw_lookup_cmd_ver(mvm->fw, DATA_PATH_GROUP,
+ RLC_CONFIG_CMD, 0) < 2)
+ iwl_mvm_phy_ctxt_set_rxchain(mvm, ctxt, &cmd->rxchain_info,
+ chains_static, chains_dynamic);
+}
+
+static int iwl_mvm_phy_send_rlc(struct iwl_mvm *mvm,
+ struct iwl_mvm_phy_ctxt *ctxt,
+ u8 chains_static, u8 chains_dynamic)
+{
+ struct iwl_rlc_config_cmd cmd = {
+ .phy_id = cpu_to_le32(ctxt->id),
+ };
+
+ if (iwl_fw_lookup_cmd_ver(mvm->fw, DATA_PATH_GROUP,
+ RLC_CONFIG_CMD, 0) < 2)
+ return 0;
+
+ BUILD_BUG_ON(IWL_RLC_CHAIN_INFO_DRIVER_FORCE !=
+ PHY_RX_CHAIN_DRIVER_FORCE_MSK);
+ BUILD_BUG_ON(IWL_RLC_CHAIN_INFO_VALID !=
+ PHY_RX_CHAIN_VALID_MSK);
+ BUILD_BUG_ON(IWL_RLC_CHAIN_INFO_FORCE !=
+ PHY_RX_CHAIN_FORCE_SEL_MSK);
+ BUILD_BUG_ON(IWL_RLC_CHAIN_INFO_FORCE_MIMO !=
+ PHY_RX_CHAIN_FORCE_MIMO_SEL_MSK);
+ BUILD_BUG_ON(IWL_RLC_CHAIN_INFO_COUNT != PHY_RX_CHAIN_CNT_MSK);
+ BUILD_BUG_ON(IWL_RLC_CHAIN_INFO_MIMO_COUNT !=
+ PHY_RX_CHAIN_MIMO_CNT_MSK);
+
+ iwl_mvm_phy_ctxt_set_rxchain(mvm, ctxt, &cmd.rlc.rx_chain_info,
chains_static, chains_dynamic);
+
+ return iwl_mvm_send_cmd_pdu(mvm, iwl_cmd_id(RLC_CONFIG_CMD,
+ DATA_PATH_GROUP, 2),
+ 0, sizeof(cmd), &cmd);
}
/*
@@ -177,7 +212,7 @@ static int iwl_mvm_phy_ctxt_apply(struct iwl_mvm *mvm,
int ver = iwl_fw_lookup_cmd_ver(mvm->fw, IWL_ALWAYS_LONG_GROUP,
PHY_CONTEXT_CMD, 1);
- if (ver == 3) {
+ if (ver == 3 || ver == 4) {
struct iwl_phy_context_cmd cmd = {};
/* Set the command header fields */
@@ -211,9 +246,16 @@ static int iwl_mvm_phy_ctxt_apply(struct iwl_mvm *mvm,
}
- if (ret)
+ if (ret) {
IWL_ERR(mvm, "PHY ctxt cmd error. ret=%d\n", ret);
- return ret;
+ return ret;
+ }
+
+ if (action != FW_CTXT_ACTION_REMOVE)
+ return iwl_mvm_phy_send_rlc(mvm, ctxt, chains_static,
+ chains_dynamic);
+
+ return 0;
}
/*
@@ -228,6 +270,8 @@ int iwl_mvm_phy_ctxt_add(struct iwl_mvm *mvm, struct iwl_mvm_phy_ctxt *ctxt,
lockdep_assert_held(&mvm->mutex);
ctxt->channel = chandef->chan;
+ ctxt->width = chandef->width;
+ ctxt->center_freq1 = chandef->center_freq1;
return iwl_mvm_phy_ctxt_apply(mvm, ctxt, chandef,
chains_static, chains_dynamic,
@@ -257,6 +301,14 @@ int iwl_mvm_phy_ctxt_changed(struct iwl_mvm *mvm, struct iwl_mvm_phy_ctxt *ctxt,
lockdep_assert_held(&mvm->mutex);
+ if (iwl_fw_lookup_cmd_ver(mvm->fw, DATA_PATH_GROUP,
+ RLC_CONFIG_CMD, 0) >= 2 &&
+ ctxt->channel == chandef->chan &&
+ ctxt->width == chandef->width &&
+ ctxt->center_freq1 == chandef->center_freq1)
+ return iwl_mvm_phy_send_rlc(mvm, ctxt, chains_static,
+ chains_dynamic);
+
if (fw_has_capa(&mvm->fw->ucode_capa,
IWL_UCODE_TLV_CAPA_BINDING_CDB_SUPPORT) &&
ctxt->channel->band != chandef->chan->band) {
@@ -275,6 +327,8 @@ int iwl_mvm_phy_ctxt_changed(struct iwl_mvm *mvm, struct iwl_mvm_phy_ctxt *ctxt,
ctxt->channel = chandef->chan;
ctxt->width = chandef->width;
+ ctxt->center_freq1 = chandef->center_freq1;
+
return iwl_mvm_phy_ctxt_apply(mvm, ctxt, chandef,
chains_static, chains_dynamic,
action);
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/rfi.c b/drivers/net/wireless/intel/iwlwifi/mvm/rfi.c
index 4434421..f054ce7 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/rfi.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/rfi.c
@@ -7,39 +7,57 @@
#include "fw/api/commands.h"
#include "fw/api/phy-ctxt.h"
-/**
+/*
* DDR needs frequency in units of 16.666MHz, so provide FW with the
* frequency values in the adjusted format.
*/
static const struct iwl_rfi_lut_entry iwl_rfi_table[IWL_RFI_LUT_SIZE] = {
- /* LPDDR4 */
+ /* frequency 2667MHz */
+ {cpu_to_le16(160), {50, 58, 60, 62, 64, 52, 54, 56},
+ {PHY_BAND_5, PHY_BAND_5, PHY_BAND_5, PHY_BAND_5, PHY_BAND_5,
+ PHY_BAND_5, PHY_BAND_5, PHY_BAND_5,}},
+
+ /* frequency 2933MHz */
+ {cpu_to_le16(176), {149, 151, 153, 157, 159, 161, 165, 163, 167, 169,
+ 171, 173, 175},
+ {PHY_BAND_5, PHY_BAND_5, PHY_BAND_5, PHY_BAND_5, PHY_BAND_5,
+ PHY_BAND_5, PHY_BAND_5, PHY_BAND_5, PHY_BAND_5, PHY_BAND_5,
+ PHY_BAND_5, PHY_BAND_5, PHY_BAND_5,}},
+
+ /* frequency 3200MHz */
+ {cpu_to_le16(192), {79, 81, 83, 85, 87, 89, 91, 93},
+ {PHY_BAND_6, PHY_BAND_6, PHY_BAND_6, PHY_BAND_6, PHY_BAND_6,
+ PHY_BAND_6, PHY_BAND_6, PHY_BAND_6,}},
/* frequency 3733MHz */
- {cpu_to_le16(223), {114, 116, 118, 120, 122,},
- {PHY_BAND_5, PHY_BAND_5, PHY_BAND_5, PHY_BAND_5, PHY_BAND_5,}},
+ {cpu_to_le16(223), {114, 116, 118, 120, 122, 106, 110, 124, 126},
+ {PHY_BAND_5, PHY_BAND_5, PHY_BAND_5, PHY_BAND_5, PHY_BAND_5,
+ PHY_BAND_5, PHY_BAND_5, PHY_BAND_5, PHY_BAND_5,}},
+
+ /* frequency 4000MHz */
+ {cpu_to_le16(240), {114, 151, 155, 157, 159, 161, 165},
+ {PHY_BAND_5, PHY_BAND_5, PHY_BAND_5, PHY_BAND_5, PHY_BAND_5,
+ PHY_BAND_5, PHY_BAND_5,}},
/* frequency 4267MHz */
{cpu_to_le16(256), {79, 83, 85, 87, 89, 91, 93,},
{PHY_BAND_6, PHY_BAND_6, PHY_BAND_6, PHY_BAND_6, PHY_BAND_6,
PHY_BAND_6, PHY_BAND_6,}},
- /* DDR5ePOR */
-
- /* frequency 4000MHz */
- {cpu_to_le16(240), {3, 5, 7, 9, 11, 13, 15,},
- {PHY_BAND_6, PHY_BAND_6, PHY_BAND_6, PHY_BAND_6, PHY_BAND_6,
- PHY_BAND_6, PHY_BAND_6,}},
-
/* frequency 4400MHz */
{cpu_to_le16(264), {111, 119, 123, 125, 129, 131, 133, 135, 143,},
{PHY_BAND_6, PHY_BAND_6, PHY_BAND_6, PHY_BAND_6, PHY_BAND_6,
PHY_BAND_6, PHY_BAND_6, PHY_BAND_6, PHY_BAND_6,}},
- /* LPDDR5iPOR */
-
/* frequency 5200MHz */
- {cpu_to_le16(312), {36, 38, 40, 42, 50,},
- {PHY_BAND_5, PHY_BAND_5, PHY_BAND_5, PHY_BAND_5, PHY_BAND_5,}},
+ {cpu_to_le16(312), {36, 38, 40, 42, 44, 46, 50,},
+ {PHY_BAND_5, PHY_BAND_5, PHY_BAND_5, PHY_BAND_5, PHY_BAND_5,
+ PHY_BAND_5, PHY_BAND_5,}},
+
+ /* frequency 5600MHz */
+ {cpu_to_le16(336), {106, 110, 112, 114, 116, 118, 120, 122},
+ {PHY_BAND_5, PHY_BAND_5, PHY_BAND_5, PHY_BAND_5, PHY_BAND_5,
+ PHY_BAND_5, PHY_BAND_5, PHY_BAND_5,}},
/* frequency 6000MHz */
{cpu_to_le16(360), {3, 5, 7, 9, 11, 13, 15,},
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/rs-fw.c b/drivers/net/wireless/intel/iwlwifi/mvm/rs-fw.c
index 9587024..66808c5 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/rs-fw.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/rs-fw.c
@@ -129,7 +129,7 @@ int rs_fw_vht_highest_rx_mcs_index(const struct ieee80211_sta_vht_cap *vht_cap,
static void
rs_fw_vht_set_enabled_rates(const struct ieee80211_sta *sta,
const struct ieee80211_sta_vht_cap *vht_cap,
- struct iwl_tlc_config_cmd *cmd)
+ struct iwl_tlc_config_cmd_v4 *cmd)
{
u16 supp;
int i, highest_mcs;
@@ -154,7 +154,7 @@ rs_fw_vht_set_enabled_rates(const struct ieee80211_sta *sta,
if (sta->bandwidth == IEEE80211_STA_RX_BW_20)
supp &= ~BIT(IWL_TLC_MNG_HT_RATE_MCS9);
- cmd->ht_rates[i][IWL_TLC_HT_BW_NONE_160] = cpu_to_le16(supp);
+ cmd->ht_rates[i][IWL_TLC_MCS_PER_BW_80] = cpu_to_le16(supp);
/*
* Check if VHT extended NSS indicates that the bandwidth/NSS
* configuration is supported - only for MCS 0 since we already
@@ -164,8 +164,8 @@ rs_fw_vht_set_enabled_rates(const struct ieee80211_sta *sta,
ieee80211_get_vht_max_nss(&ieee_vht_cap,
IEEE80211_VHT_CHANWIDTH_160MHZ,
0, true, nss) >= nss)
- cmd->ht_rates[i][IWL_TLC_HT_BW_160] =
- cmd->ht_rates[i][IWL_TLC_HT_BW_NONE_160];
+ cmd->ht_rates[i][IWL_TLC_MCS_PER_BW_160] =
+ cmd->ht_rates[i][IWL_TLC_MCS_PER_BW_80];
}
}
@@ -189,7 +189,7 @@ static u16 rs_fw_he_ieee80211_mcs_to_rs_mcs(u16 mcs)
static void
rs_fw_he_set_enabled_rates(const struct ieee80211_sta *sta,
struct ieee80211_supported_band *sband,
- struct iwl_tlc_config_cmd *cmd)
+ struct iwl_tlc_config_cmd_v4 *cmd)
{
const struct ieee80211_sta_he_cap *he_cap = &sta->he_cap;
u16 mcs_160 = le16_to_cpu(he_cap->he_mcs_nss_supp.rx_mcs_160);
@@ -219,7 +219,7 @@ rs_fw_he_set_enabled_rates(const struct ieee80211_sta *sta,
}
if (_mcs_80 > _tx_mcs_80)
_mcs_80 = _tx_mcs_80;
- cmd->ht_rates[i][IWL_TLC_HT_BW_NONE_160] =
+ cmd->ht_rates[i][IWL_TLC_MCS_PER_BW_80] =
cpu_to_le16(rs_fw_he_ieee80211_mcs_to_rs_mcs(_mcs_80));
/* If one side doesn't support - mark both as not supporting */
@@ -230,14 +230,14 @@ rs_fw_he_set_enabled_rates(const struct ieee80211_sta *sta,
}
if (_mcs_160 > _tx_mcs_160)
_mcs_160 = _tx_mcs_160;
- cmd->ht_rates[i][IWL_TLC_HT_BW_160] =
+ cmd->ht_rates[i][IWL_TLC_MCS_PER_BW_160] =
cpu_to_le16(rs_fw_he_ieee80211_mcs_to_rs_mcs(_mcs_160));
}
}
static void rs_fw_set_supp_rates(struct ieee80211_sta *sta,
struct ieee80211_supported_band *sband,
- struct iwl_tlc_config_cmd *cmd)
+ struct iwl_tlc_config_cmd_v4 *cmd)
{
int i;
u16 supp = 0;
@@ -263,15 +263,15 @@ static void rs_fw_set_supp_rates(struct ieee80211_sta *sta,
rs_fw_vht_set_enabled_rates(sta, vht_cap, cmd);
} else if (ht_cap->ht_supported) {
cmd->mode = IWL_TLC_MNG_MODE_HT;
- cmd->ht_rates[IWL_TLC_NSS_1][IWL_TLC_HT_BW_NONE_160] =
+ cmd->ht_rates[IWL_TLC_NSS_1][IWL_TLC_MCS_PER_BW_80] =
cpu_to_le16(ht_cap->mcs.rx_mask[0]);
/* the station support only a single receive chain */
if (sta->smps_mode == IEEE80211_SMPS_STATIC)
- cmd->ht_rates[IWL_TLC_NSS_2][IWL_TLC_HT_BW_NONE_160] =
+ cmd->ht_rates[IWL_TLC_NSS_2][IWL_TLC_MCS_PER_BW_80] =
0;
else
- cmd->ht_rates[IWL_TLC_NSS_2][IWL_TLC_HT_BW_NONE_160] =
+ cmd->ht_rates[IWL_TLC_NSS_2][IWL_TLC_MCS_PER_BW_80] =
cpu_to_le16(ht_cap->mcs.rx_mask[1]);
}
}
@@ -291,8 +291,12 @@ void iwl_mvm_tlc_update_notif(struct iwl_mvm *mvm,
notif = (void *)pkt->data;
sta = rcu_dereference(mvm->fw_id_to_mac_id[notif->sta_id]);
if (IS_ERR_OR_NULL(sta)) {
- IWL_ERR(mvm, "Invalid sta id (%d) in FW TLC notification\n",
- notif->sta_id);
+ /* can happen in remove station flow where mvm removed internally
+ * the station before removing from FW
+ */
+ IWL_DEBUG_RATE(mvm,
+ "Invalid mvm RCU pointer for sta id (%d) in TLC notification\n",
+ notif->sta_id);
goto out;
}
@@ -311,18 +315,19 @@ void iwl_mvm_tlc_update_notif(struct iwl_mvm *mvm,
if (flags & IWL_TLC_NOTIF_FLAG_RATE) {
char pretty_rate[100];
- if (iwl_fw_lookup_notif_ver(mvm->fw, DATA_PATH_GROUP,
- TLC_MNG_UPDATE_NOTIF, 0) < 3) {
- rs_pretty_print_rate_v1(pretty_rate, sizeof(pretty_rate),
- le32_to_cpu(notif->rate));
- IWL_DEBUG_RATE(mvm,
- "Got rate in old format. Rate: %s. Converting.\n",
- pretty_rate);
- lq_sta->last_rate_n_flags =
- iwl_new_rate_from_v1(le32_to_cpu(notif->rate));
- } else {
- lq_sta->last_rate_n_flags = le32_to_cpu(notif->rate);
- }
+ if (iwl_fw_lookup_notif_ver(mvm->fw, DATA_PATH_GROUP,
+ TLC_MNG_UPDATE_NOTIF, 0) < 3) {
+ rs_pretty_print_rate_v1(pretty_rate,
+ sizeof(pretty_rate),
+ le32_to_cpu(notif->rate));
+ IWL_DEBUG_RATE(mvm,
+ "Got rate in old format. Rate: %s. Converting.\n",
+ pretty_rate);
+ lq_sta->last_rate_n_flags =
+ iwl_new_rate_from_v1(le32_to_cpu(notif->rate));
+ } else {
+ lq_sta->last_rate_n_flags = le32_to_cpu(notif->rate);
+ }
rs_pretty_print_rate(pretty_rate, sizeof(pretty_rate),
lq_sta->last_rate_n_flags);
IWL_DEBUG_RATE(mvm, "new rate: %s\n", pretty_rate);
@@ -418,23 +423,18 @@ void rs_fw_rate_init(struct iwl_mvm *mvm, struct ieee80211_sta *sta,
u32 cmd_id = iwl_cmd_id(TLC_MNG_CONFIG_CMD, DATA_PATH_GROUP, 0);
struct ieee80211_supported_band *sband = hw->wiphy->bands[band];
u16 max_amsdu_len = rs_fw_get_max_amsdu_len(sta);
- struct iwl_tlc_config_cmd cfg_cmd = {
+ struct iwl_tlc_config_cmd_v4 cfg_cmd = {
.sta_id = mvmsta->sta_id,
.max_ch_width = update ?
rs_fw_bw_from_sta_bw(sta) : RATE_MCS_CHAN_WIDTH_20,
.flags = cpu_to_le16(rs_fw_get_config_flags(mvm, sta, sband)),
.chains = rs_fw_set_active_chains(iwl_mvm_get_valid_tx_ant(mvm)),
.sgi_ch_width_supp = rs_fw_sgi_cw_support(sta),
- .max_mpdu_len = cpu_to_le16(max_amsdu_len),
- .amsdu = iwl_mvm_is_csum_supported(mvm),
+ .max_mpdu_len = iwl_mvm_is_csum_supported(mvm) ?
+ cpu_to_le16(max_amsdu_len) : 0,
};
int ret;
- u16 cmd_size = sizeof(cfg_cmd);
-
- /* In old versions of the API the struct is 4 bytes smaller */
- if (iwl_fw_lookup_cmd_ver(mvm->fw, DATA_PATH_GROUP,
- TLC_MNG_CONFIG_CMD, 0) < 3)
- cmd_size -= 4;
+ int cmd_ver;
memset(lq_sta, 0, offsetof(typeof(*lq_sta), pers));
@@ -449,8 +449,41 @@ void rs_fw_rate_init(struct iwl_mvm *mvm, struct ieee80211_sta *sta,
*/
sta->max_amsdu_len = max_amsdu_len;
- ret = iwl_mvm_send_cmd_pdu(mvm, cmd_id, CMD_ASYNC, cmd_size,
- &cfg_cmd);
+ cmd_ver = iwl_fw_lookup_cmd_ver(mvm->fw, DATA_PATH_GROUP,
+ TLC_MNG_CONFIG_CMD, 0);
+ if (cmd_ver == 4) {
+ ret = iwl_mvm_send_cmd_pdu(mvm, cmd_id, CMD_ASYNC,
+ sizeof(cfg_cmd), &cfg_cmd);
+ } else if (cmd_ver < 4) {
+ struct iwl_tlc_config_cmd_v3 cfg_cmd_v3 = {
+ .sta_id = cfg_cmd.sta_id,
+ .max_ch_width = cfg_cmd.max_ch_width,
+ .mode = cfg_cmd.mode,
+ .chains = cfg_cmd.chains,
+ .amsdu = !!cfg_cmd.max_mpdu_len,
+ .flags = cfg_cmd.flags,
+ .non_ht_rates = cfg_cmd.non_ht_rates,
+ .ht_rates[0][0] = cfg_cmd.ht_rates[0][0],
+ .ht_rates[0][1] = cfg_cmd.ht_rates[0][1],
+ .ht_rates[1][0] = cfg_cmd.ht_rates[1][0],
+ .ht_rates[1][1] = cfg_cmd.ht_rates[1][1],
+ .sgi_ch_width_supp = cfg_cmd.sgi_ch_width_supp,
+ .max_mpdu_len = cfg_cmd.max_mpdu_len,
+ };
+
+ u16 cmd_size = sizeof(cfg_cmd_v3);
+
+ /* In old versions of the API the struct is 4 bytes smaller */
+ if (iwl_fw_lookup_cmd_ver(mvm->fw, DATA_PATH_GROUP,
+ TLC_MNG_CONFIG_CMD, 0) < 3)
+ cmd_size -= 4;
+
+ ret = iwl_mvm_send_cmd_pdu(mvm, cmd_id, CMD_ASYNC, cmd_size,
+ &cfg_cmd_v3);
+ } else {
+ ret = -EINVAL;
+ }
+
if (ret)
IWL_ERR(mvm, "Failed to send rate scale config (%d)\n", ret);
}
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/rx.c b/drivers/net/wireless/intel/iwlwifi/mvm/rx.c
index d22f40a..64446a1 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/rx.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/rx.c
@@ -527,40 +527,19 @@ struct iwl_mvm_stat_data {
u8 *beacon_average_energy;
};
-static void iwl_mvm_stat_iterator(void *_data, u8 *mac,
- struct ieee80211_vif *vif)
+struct iwl_mvm_stat_data_all_macs {
+ struct iwl_mvm *mvm;
+ __le32 flags;
+ struct iwl_statistics_ntfy_per_mac *per_mac_stats;
+};
+
+static void iwl_mvm_update_vif_sig(struct ieee80211_vif *vif, int sig)
{
- struct iwl_mvm_stat_data *data = _data;
- struct iwl_mvm *mvm = data->mvm;
- int sig = -data->beacon_filter_average_energy;
- int last_event;
+ struct iwl_mvm_vif *mvmvif = iwl_mvm_vif_from_mac80211(vif);
+ struct iwl_mvm *mvm = mvmvif->mvm;
int thold = vif->bss_conf.cqm_rssi_thold;
int hyst = vif->bss_conf.cqm_rssi_hyst;
- u16 id = le32_to_cpu(data->mac_id);
- struct iwl_mvm_vif *mvmvif = iwl_mvm_vif_from_mac80211(vif);
- u16 vif_id = mvmvif->id;
-
- /* This doesn't need the MAC ID check since it's not taking the
- * data copied into the "data" struct, but rather the data from
- * the notification directly.
- */
- mvmvif->beacon_stats.num_beacons =
- le32_to_cpu(data->beacon_counter[vif_id]);
- mvmvif->beacon_stats.avg_signal =
- -data->beacon_average_energy[vif_id];
-
- /* make sure that beacon statistics don't go backwards with TCM
- * request to clear statistics
- */
- if (le32_to_cpu(data->flags) & IWL_STATISTICS_REPLY_FLG_CLEAR)
- mvmvif->beacon_stats.accu_num_beacons +=
- mvmvif->beacon_stats.num_beacons;
-
- if (mvmvif->id != id)
- return;
-
- if (vif->type != NL80211_IFTYPE_STATION)
- return;
+ int last_event;
if (sig == 0) {
IWL_DEBUG_RX(mvm, "RSSI is 0 - skip signal based decision\n");
@@ -618,6 +597,73 @@ static void iwl_mvm_stat_iterator(void *_data, u8 *mac,
}
}
+static void iwl_mvm_stat_iterator(void *_data, u8 *mac,
+ struct ieee80211_vif *vif)
+{
+ struct iwl_mvm_stat_data *data = _data;
+ int sig = -data->beacon_filter_average_energy;
+ u16 id = le32_to_cpu(data->mac_id);
+ struct iwl_mvm_vif *mvmvif = iwl_mvm_vif_from_mac80211(vif);
+ u16 vif_id = mvmvif->id;
+
+ /* This doesn't need the MAC ID check since it's not taking the
+ * data copied into the "data" struct, but rather the data from
+ * the notification directly.
+ */
+ mvmvif->beacon_stats.num_beacons =
+ le32_to_cpu(data->beacon_counter[vif_id]);
+ mvmvif->beacon_stats.avg_signal =
+ -data->beacon_average_energy[vif_id];
+
+ if (mvmvif->id != id)
+ return;
+
+ if (vif->type != NL80211_IFTYPE_STATION)
+ return;
+
+ /* make sure that beacon statistics don't go backwards with TCM
+ * request to clear statistics
+ */
+ if (le32_to_cpu(data->flags) & IWL_STATISTICS_REPLY_FLG_CLEAR)
+ mvmvif->beacon_stats.accu_num_beacons +=
+ mvmvif->beacon_stats.num_beacons;
+
+ iwl_mvm_update_vif_sig(vif, sig);
+}
+
+static void iwl_mvm_stat_iterator_all_macs(void *_data, u8 *mac,
+ struct ieee80211_vif *vif)
+{
+ struct iwl_mvm_stat_data_all_macs *data = _data;
+ struct iwl_statistics_ntfy_per_mac *mac_stats;
+ int sig;
+ struct iwl_mvm_vif *mvmvif = iwl_mvm_vif_from_mac80211(vif);
+ u16 vif_id = mvmvif->id;
+
+ if (WARN_ONCE(vif_id > MAC_INDEX_AUX, "invalid vif id: %d", vif_id))
+ return;
+
+ if (vif->type != NL80211_IFTYPE_STATION)
+ return;
+
+ mac_stats = &data->per_mac_stats[vif_id];
+
+ mvmvif->beacon_stats.num_beacons =
+ le32_to_cpu(mac_stats->beacon_counter);
+ mvmvif->beacon_stats.avg_signal =
+ -le32_to_cpu(mac_stats->beacon_average_energy);
+
+ /* make sure that beacon statistics don't go backwards with TCM
+ * request to clear statistics
+ */
+ if (le32_to_cpu(data->flags) & IWL_STATISTICS_REPLY_FLG_CLEAR)
+ mvmvif->beacon_stats.accu_num_beacons +=
+ mvmvif->beacon_stats.num_beacons;
+
+ sig = -le32_to_cpu(mac_stats->beacon_filter_average_energy);
+ iwl_mvm_update_vif_sig(vif, sig);
+}
+
static inline void
iwl_mvm_rx_stats_check_trigger(struct iwl_mvm *mvm, struct iwl_rx_packet *pkt)
{
@@ -684,47 +730,41 @@ iwl_mvm_update_tcm_from_stats(struct iwl_mvm *mvm, __le32 *air_time_le,
}
static void
-iwl_mvm_handle_rx_statistics_tlv(struct iwl_mvm *mvm,
- struct iwl_rx_packet *pkt)
+iwl_mvm_stats_ver_15(struct iwl_mvm *mvm,
+ struct iwl_statistics_operational_ntfy *stats)
+{
+ struct iwl_mvm_stat_data_all_macs data = {
+ .mvm = mvm,
+ .flags = stats->flags,
+ .per_mac_stats = stats->per_mac_stats,
+ };
+
+ ieee80211_iterate_active_interfaces(mvm->hw,
+ IEEE80211_IFACE_ITER_NORMAL,
+ iwl_mvm_stat_iterator_all_macs,
+ &data);
+}
+
+static void
+iwl_mvm_stats_ver_14(struct iwl_mvm *mvm,
+ struct iwl_statistics_operational_ntfy_ver_14 *stats)
{
struct iwl_mvm_stat_data data = {
.mvm = mvm,
};
+
u8 beacon_average_energy[MAC_INDEX_AUX];
- u8 average_energy[IWL_MVM_STATION_COUNT_MAX];
- struct iwl_statistics_operational_ntfy *stats;
- int expected_size;
__le32 flags;
int i;
- expected_size = sizeof(*stats);
- if (WARN_ONCE(iwl_rx_packet_payload_len(pkt) < expected_size,
- "received invalid statistics size (%d)!, expected_size: %d\n",
- iwl_rx_packet_payload_len(pkt), expected_size))
- return;
-
- stats = (void *)&pkt->data;
-
- if (WARN_ONCE(stats->hdr.type != FW_STATISTICS_OPERATIONAL ||
- stats->hdr.version !=
- iwl_fw_lookup_notif_ver(mvm->fw, LONG_GROUP, STATISTICS_CMD, 0),
- "received unsupported hdr type %d, version %d\n",
- stats->hdr.type, stats->hdr.version))
- return;
-
flags = stats->flags;
- mvm->radio_stats.rx_time = le64_to_cpu(stats->rx_time);
- mvm->radio_stats.tx_time = le64_to_cpu(stats->tx_time);
- mvm->radio_stats.on_time_rf = le64_to_cpu(stats->on_time_rf);
- mvm->radio_stats.on_time_scan = le64_to_cpu(stats->on_time_scan);
-
- iwl_mvm_rx_stats_check_trigger(mvm, pkt);
data.mac_id = stats->mac_id;
data.beacon_filter_average_energy =
le32_to_cpu(stats->beacon_filter_average_energy);
data.flags = flags;
data.beacon_counter = stats->beacon_counter;
+
for (i = 0; i < ARRAY_SIZE(beacon_average_energy); i++)
beacon_average_energy[i] =
le32_to_cpu(stats->beacon_average_energy[i]);
@@ -735,9 +775,105 @@ iwl_mvm_handle_rx_statistics_tlv(struct iwl_mvm *mvm,
IEEE80211_IFACE_ITER_NORMAL,
iwl_mvm_stat_iterator,
&data);
+}
- for (i = 0; i < ARRAY_SIZE(average_energy); i++)
- average_energy[i] = le32_to_cpu(stats->average_energy[i]);
+static bool iwl_mvm_verify_stats_len(struct iwl_mvm *mvm,
+ struct iwl_rx_packet *pkt,
+ u32 expected_size)
+{
+ struct iwl_statistics_ntfy_hdr *hdr;
+
+ if (WARN_ONCE(iwl_rx_packet_payload_len(pkt) < expected_size,
+ "received invalid statistics size (%d)!, expected_size: %d\n",
+ iwl_rx_packet_payload_len(pkt), expected_size))
+ return false;
+
+ hdr = (void *)&pkt->data;
+
+ if (WARN_ONCE((hdr->type & IWL_STATISTICS_TYPE_MSK) != FW_STATISTICS_OPERATIONAL ||
+ hdr->version !=
+ iwl_fw_lookup_notif_ver(mvm->fw, LEGACY_GROUP, STATISTICS_NOTIFICATION, 0),
+ "received unsupported hdr type %d, version %d\n",
+ hdr->type, hdr->version))
+ return false;
+
+ if (WARN_ONCE(le16_to_cpu(hdr->size) != expected_size,
+ "received invalid statistics size in header (%d)!, expected_size: %d\n",
+ le16_to_cpu(hdr->size), expected_size))
+ return false;
+
+ return true;
+}
+
+static void
+iwl_mvm_handle_rx_statistics_tlv(struct iwl_mvm *mvm,
+ struct iwl_rx_packet *pkt)
+{
+ u8 average_energy[IWL_MVM_STATION_COUNT_MAX];
+ __le32 air_time[MAC_INDEX_AUX];
+ __le32 rx_bytes[MAC_INDEX_AUX];
+ __le32 flags = 0;
+ int i;
+ u32 notif_ver = iwl_fw_lookup_notif_ver(mvm->fw, LEGACY_GROUP,
+ STATISTICS_NOTIFICATION, 0);
+
+ if (WARN_ONCE(notif_ver > 15,
+ "invalid statistics version id: %d\n", notif_ver))
+ return;
+
+ if (notif_ver == 14) {
+ struct iwl_statistics_operational_ntfy_ver_14 *stats =
+ (void *)pkt->data;
+
+ if (!iwl_mvm_verify_stats_len(mvm, pkt, sizeof(*stats)))
+ return;
+
+ iwl_mvm_stats_ver_14(mvm, stats);
+
+ flags = stats->flags;
+ mvm->radio_stats.rx_time = le64_to_cpu(stats->rx_time);
+ mvm->radio_stats.tx_time = le64_to_cpu(stats->tx_time);
+ mvm->radio_stats.on_time_rf = le64_to_cpu(stats->on_time_rf);
+ mvm->radio_stats.on_time_scan =
+ le64_to_cpu(stats->on_time_scan);
+
+ for (i = 0; i < ARRAY_SIZE(average_energy); i++)
+ average_energy[i] = le32_to_cpu(stats->average_energy[i]);
+
+ for (i = 0; i < ARRAY_SIZE(air_time); i++) {
+ air_time[i] = stats->air_time[i];
+ rx_bytes[i] = stats->rx_bytes[i];
+ }
+ }
+
+ if (notif_ver == 15) {
+ struct iwl_statistics_operational_ntfy *stats =
+ (void *)pkt->data;
+
+ if (!iwl_mvm_verify_stats_len(mvm, pkt, sizeof(*stats)))
+ return;
+
+ iwl_mvm_stats_ver_15(mvm, stats);
+
+ flags = stats->flags;
+ mvm->radio_stats.rx_time = le64_to_cpu(stats->rx_time);
+ mvm->radio_stats.tx_time = le64_to_cpu(stats->tx_time);
+ mvm->radio_stats.on_time_rf = le64_to_cpu(stats->on_time_rf);
+ mvm->radio_stats.on_time_scan =
+ le64_to_cpu(stats->on_time_scan);
+
+ for (i = 0; i < ARRAY_SIZE(average_energy); i++)
+ average_energy[i] =
+ le32_to_cpu(stats->per_sta_stats[i].average_energy);
+
+ for (i = 0; i < ARRAY_SIZE(air_time); i++) {
+ air_time[i] = stats->per_mac_stats[i].air_time;
+ rx_bytes[i] = stats->per_mac_stats[i].rx_bytes;
+ }
+ }
+
+ iwl_mvm_rx_stats_check_trigger(mvm, pkt);
+
ieee80211_iterate_stations_atomic(mvm->hw, iwl_mvm_stats_energy_iter,
average_energy);
/*
@@ -746,8 +882,7 @@ iwl_mvm_handle_rx_statistics_tlv(struct iwl_mvm *mvm,
* request and once in statistics notification.
*/
if (le32_to_cpu(flags) & IWL_STATISTICS_REPLY_FLG_CLEAR)
- iwl_mvm_update_tcm_from_stats(mvm, stats->air_time,
- stats->rx_bytes);
+ iwl_mvm_update_tcm_from_stats(mvm, air_time, rx_bytes);
}
void iwl_mvm_handle_rx_statistics(struct iwl_mvm *mvm,
@@ -761,8 +896,8 @@ void iwl_mvm_handle_rx_statistics(struct iwl_mvm *mvm,
u8 *energy;
/* From ver 14 and up we use TLV statistics format */
- if (iwl_fw_lookup_notif_ver(mvm->fw, LONG_GROUP,
- STATISTICS_CMD, 0) >= 14)
+ if (iwl_fw_lookup_notif_ver(mvm->fw, LEGACY_GROUP,
+ STATISTICS_NOTIFICATION, 0) >= 14)
return iwl_mvm_handle_rx_statistics_tlv(mvm, pkt);
if (!iwl_mvm_has_new_rx_stats_api(mvm)) {
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/rxmq.c b/drivers/net/wireless/intel/iwlwifi/mvm/rxmq.c
index e0601f8..295629c 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/rxmq.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/rxmq.c
@@ -121,12 +121,39 @@ static int iwl_mvm_create_skb(struct iwl_mvm *mvm, struct sk_buff *skb,
struct iwl_rx_mpdu_desc *desc = (void *)pkt->data;
unsigned int headlen, fraglen, pad_len = 0;
unsigned int hdrlen = ieee80211_hdrlen(hdr->frame_control);
+ u8 mic_crc_len = u8_get_bits(desc->mac_flags1,
+ IWL_RX_MPDU_MFLG1_MIC_CRC_LEN_MASK) << 1;
if (desc->mac_flags2 & IWL_RX_MPDU_MFLG2_PAD) {
len -= 2;
pad_len = 2;
}
+ /*
+ * For non monitor interface strip the bytes the RADA might not have
+ * removed. As monitor interface cannot exist with other interfaces
+ * this removal is safe.
+ */
+ if (mic_crc_len && !ieee80211_hw_check(mvm->hw, RX_INCLUDES_FCS)) {
+ u32 pkt_flags = le32_to_cpu(pkt->len_n_flags);
+
+ /*
+ * If RADA was not enabled then decryption was not performed so
+ * the MIC cannot be removed.
+ */
+ if (!(pkt_flags & FH_RSCSR_RADA_EN)) {
+ if (WARN_ON(crypt_len > mic_crc_len))
+ return -EINVAL;
+
+ mic_crc_len -= crypt_len;
+ }
+
+ if (WARN_ON(mic_crc_len > len))
+ return -EINVAL;
+
+ len -= mic_crc_len;
+ }
+
/* If frame is small enough to fit in skb->head, pull it completely.
* If not, only pull ieee80211_hdr (including crypto if present, and
* an additional 8 bytes for SNAP/ethertype, see below) so that
@@ -149,18 +176,8 @@ static int iwl_mvm_create_skb(struct iwl_mvm *mvm, struct sk_buff *skb,
*/
hdrlen += crypt_len;
- if (WARN_ONCE(headlen < hdrlen,
- "invalid packet lengths (hdrlen=%d, len=%d, crypt_len=%d)\n",
- hdrlen, len, crypt_len)) {
- /*
- * We warn and trace because we want to be able to see
- * it in trace-cmd as well.
- */
- IWL_DEBUG_RX(mvm,
- "invalid packet lengths (hdrlen=%d, len=%d, crypt_len=%d)\n",
- hdrlen, len, crypt_len);
+ if (unlikely(headlen < hdrlen))
return -EINVAL;
- }
skb_put_data(skb, hdr, hdrlen);
skb_put_data(skb, (u8 *)hdr + hdrlen + pad_len, headlen - hdrlen);
@@ -172,8 +189,12 @@ static int iwl_mvm_create_skb(struct iwl_mvm *mvm, struct sk_buff *skb,
* in the cases the hardware didn't handle, since it's rare to see
* such packets, even though the hardware did calculate the checksum
* in this case, just starting after the MAC header instead.
+ *
+ * Starting from Bz hardware, it calculates starting directly after
+ * the MAC header, so that matches mac80211's expectation.
*/
- if (skb->ip_summed == CHECKSUM_COMPLETE) {
+ if (skb->ip_summed == CHECKSUM_COMPLETE &&
+ mvm->trans->trans_cfg->device_family < IWL_DEVICE_FAMILY_BZ) {
struct {
u8 hdr[6];
__be16 type;
@@ -766,8 +787,11 @@ static void iwl_mvm_release_frames_from_notif(struct iwl_mvm *mvm,
rcu_read_lock();
ba_data = rcu_dereference(mvm->baid_map[baid]);
- if (WARN_ON_ONCE(!ba_data))
+ if (!ba_data) {
+ WARN(!(flags & IWL_MVM_RELEASE_FROM_RSS_SYNC),
+ "BAID %d not found in map\n", baid);
goto out;
+ }
sta = rcu_dereference(mvm->fw_id_to_mac_id[ba_data->sta_id]);
if (WARN_ON_ONCE(IS_ERR_OR_NULL(sta)))
@@ -1961,8 +1985,7 @@ void iwl_mvm_rx_mpdu_mq(struct iwl_mvm *mvm, struct napi_struct *napi,
} else if (format == RATE_MCS_VHT_MSK) {
u8 stbc = (rate_n_flags & RATE_MCS_STBC_MSK) >>
RATE_MCS_STBC_POS;
- rx_status->nss =
- ((rate_n_flags & RATE_MCS_NSS_MSK) >>
+ rx_status->nss = ((rate_n_flags & RATE_MCS_NSS_MSK) >>
RATE_MCS_NSS_POS) + 1;
rx_status->rate_idx = rate_n_flags & RATE_MCS_CODE_MSK;
rx_status->encoding = RX_ENC_VHT;
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/scan.c b/drivers/net/wireless/intel/iwlwifi/mvm/scan.c
index a138b5c..5f92a09 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/scan.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/scan.c
@@ -579,7 +579,9 @@ iwl_mvm_config_sched_scan_profiles(struct iwl_mvm *mvm,
profile->ssid_index = i;
/* Support any cipher and auth algorithm */
profile->unicast_cipher = 0xff;
- profile->auth_alg = 0xff;
+ profile->auth_alg = IWL_AUTH_ALGO_UNSUPPORTED |
+ IWL_AUTH_ALGO_NONE | IWL_AUTH_ALGO_PSK | IWL_AUTH_ALGO_8021X |
+ IWL_AUTH_ALGO_SAE | IWL_AUTH_ALGO_8021X_SHA384 | IWL_AUTH_ALGO_OWE;
profile->network_type = IWL_NETWORK_TYPE_ANY;
profile->band_selection = IWL_SCAN_OFFLOAD_SELECT_ANY;
profile->client_bitmap = SCAN_CLIENT_SCHED_SCAN;
@@ -1394,8 +1396,8 @@ static u32 iwl_mvm_scan_umac_ooc_priority(struct iwl_mvm_scan_params *params)
}
static void
-iwl_mvm_scan_umac_dwell_v10(struct iwl_mvm *mvm,
- struct iwl_scan_general_params_v10 *general_params,
+iwl_mvm_scan_umac_dwell_v11(struct iwl_mvm *mvm,
+ struct iwl_scan_general_params_v11 *general_params,
struct iwl_mvm_scan_params *params)
{
struct iwl_mvm_scan_timing_params *timing, *hb_timing;
@@ -1826,8 +1828,6 @@ iwl_mvm_umac_scan_cfg_channels_v6_6g(struct iwl_mvm_scan_params *params,
}
}
- flags = bssid_bitmap | (s_ssid_bitmap << 16);
-
if (cfg80211_channel_is_psc(params->channels[i]) &&
psc_no_listen)
flags |= IWL_UHB_CHAN_CFG_FLAG_PSC_CHAN_NO_LISTEN;
@@ -1869,8 +1869,11 @@ iwl_mvm_umac_scan_cfg_channels_v6_6g(struct iwl_mvm_scan_params *params,
(s_max > 1 || b_max > 3));
}
if ((allow_passive && force_passive) ||
- (!flags && !cfg80211_channel_is_psc(params->channels[i])))
+ (!(bssid_bitmap | s_ssid_bitmap) &&
+ !cfg80211_channel_is_psc(params->channels[i])))
flags |= IWL_UHB_CHAN_CFG_FLAG_FORCE_PASSIVE;
+ else
+ flags |= bssid_bitmap | (s_ssid_bitmap << 16);
channel_cfg[i].flags |= cpu_to_le32(flags);
}
@@ -1924,22 +1927,19 @@ static void iwl_mvm_scan_6ghz_passive_scan(struct iwl_mvm *mvm,
}
/*
- * 6GHz passive scan is allowed while associated in a defined time
- * interval following HW reset or resume flow
+ * 6GHz passive scan is allowed in a defined time interval following HW
+ * reset or resume flow, or while not associated and a large interval
+ * has passed since the last 6GHz passive scan.
*/
- if (vif->bss_conf.assoc &&
+ if ((vif->bss_conf.assoc ||
+ time_after(mvm->last_6ghz_passive_scan_jiffies +
+ (IWL_MVM_6GHZ_PASSIVE_SCAN_TIMEOUT * HZ), jiffies)) &&
(time_before(mvm->last_reset_or_resume_time_jiffies +
(IWL_MVM_6GHZ_PASSIVE_SCAN_ASSOC_TIMEOUT * HZ),
jiffies))) {
- IWL_DEBUG_SCAN(mvm, "6GHz passive scan: associated\n");
- return;
- }
-
- /* No need for 6GHz passive scan if not enough time elapsed */
- if (time_after(mvm->last_6ghz_passive_scan_jiffies +
- (IWL_MVM_6GHZ_PASSIVE_SCAN_TIMEOUT * HZ), jiffies)) {
- IWL_DEBUG_SCAN(mvm,
- "6GHz passive scan: timeout did not expire\n");
+ IWL_DEBUG_SCAN(mvm, "6GHz passive scan: %s\n",
+ vif->bss_conf.assoc ? "associated" :
+ "timeout did not expire");
return;
}
@@ -2037,6 +2037,12 @@ static u16 iwl_mvm_scan_umac_flags_v2(struct iwl_mvm *mvm,
if (params->enable_6ghz_passive)
flags |= IWL_UMAC_SCAN_GEN_FLAGS_V2_6GHZ_PASSIVE_SCAN;
+ if (iwl_mvm_is_oce_supported(mvm) &&
+ (params->flags & (NL80211_SCAN_FLAG_ACCEPT_BCAST_PROBE_RESP |
+ NL80211_SCAN_FLAG_OCE_PROBE_REQ_HIGH_TX_RATE |
+ NL80211_SCAN_FLAG_FILS_MAX_CHANNEL_TIME)))
+ flags |= IWL_UMAC_SCAN_GEN_FLAGS_V2_OCE;
+
return flags;
}
@@ -2238,15 +2244,15 @@ static int iwl_mvm_scan_umac(struct iwl_mvm *mvm, struct ieee80211_vif *vif,
}
static void
-iwl_mvm_scan_umac_fill_general_p_v10(struct iwl_mvm *mvm,
+iwl_mvm_scan_umac_fill_general_p_v11(struct iwl_mvm *mvm,
struct iwl_mvm_scan_params *params,
struct ieee80211_vif *vif,
- struct iwl_scan_general_params_v10 *gp,
+ struct iwl_scan_general_params_v11 *gp,
u16 gen_flags)
{
struct iwl_mvm_vif *scan_vif = iwl_mvm_vif_from_mac80211(vif);
- iwl_mvm_scan_umac_dwell_v10(mvm, gp, params);
+ iwl_mvm_scan_umac_dwell_v11(mvm, gp, params);
gp->flags = cpu_to_le16(gen_flags);
@@ -2350,7 +2356,7 @@ static int iwl_mvm_scan_umac_v12(struct iwl_mvm *mvm, struct ieee80211_vif *vif,
cmd->uid = cpu_to_le32(uid);
gen_flags = iwl_mvm_scan_umac_flags_v2(mvm, params, vif, type);
- iwl_mvm_scan_umac_fill_general_p_v10(mvm, params, vif,
+ iwl_mvm_scan_umac_fill_general_p_v11(mvm, params, vif,
&scan_p->general_params,
gen_flags);
@@ -2367,12 +2373,13 @@ static int iwl_mvm_scan_umac_v12(struct iwl_mvm *mvm, struct ieee80211_vif *vif,
return 0;
}
-static int iwl_mvm_scan_umac_v14(struct iwl_mvm *mvm, struct ieee80211_vif *vif,
- struct iwl_mvm_scan_params *params, int type,
- int uid)
+static int iwl_mvm_scan_umac_v14_and_above(struct iwl_mvm *mvm,
+ struct ieee80211_vif *vif,
+ struct iwl_mvm_scan_params *params,
+ int type, int uid, u32 version)
{
- struct iwl_scan_req_umac_v14 *cmd = mvm->scan_cmd;
- struct iwl_scan_req_params_v14 *scan_p = &cmd->scan_params;
+ struct iwl_scan_req_umac_v15 *cmd = mvm->scan_cmd;
+ struct iwl_scan_req_params_v15 *scan_p = &cmd->scan_params;
struct iwl_scan_channel_params_v6 *cp = &scan_p->channel_params;
struct iwl_scan_probe_params_v4 *pb = &scan_p->probe_params;
int ret;
@@ -2385,7 +2392,7 @@ static int iwl_mvm_scan_umac_v14(struct iwl_mvm *mvm, struct ieee80211_vif *vif,
cmd->uid = cpu_to_le32(uid);
gen_flags = iwl_mvm_scan_umac_flags_v2(mvm, params, vif, type);
- iwl_mvm_scan_umac_fill_general_p_v10(mvm, params, vif,
+ iwl_mvm_scan_umac_fill_general_p_v11(mvm, params, vif,
&scan_p->general_params,
gen_flags);
@@ -2425,6 +2432,20 @@ static int iwl_mvm_scan_umac_v14(struct iwl_mvm *mvm, struct ieee80211_vif *vif,
return 0;
}
+static int iwl_mvm_scan_umac_v14(struct iwl_mvm *mvm, struct ieee80211_vif *vif,
+ struct iwl_mvm_scan_params *params, int type,
+ int uid)
+{
+ return iwl_mvm_scan_umac_v14_and_above(mvm, vif, params, type, uid, 14);
+}
+
+static int iwl_mvm_scan_umac_v15(struct iwl_mvm *mvm, struct ieee80211_vif *vif,
+ struct iwl_mvm_scan_params *params, int type,
+ int uid)
+{
+ return iwl_mvm_scan_umac_v14_and_above(mvm, vif, params, type, uid, 15);
+}
+
static int iwl_mvm_num_scans(struct iwl_mvm *mvm)
{
return hweight32(mvm->scan_status & IWL_MVM_SCAN_MASK);
@@ -2498,7 +2519,7 @@ static int iwl_mvm_check_running_scans(struct iwl_mvm *mvm, int type)
return -EIO;
}
-#define SCAN_TIMEOUT 20000
+#define SCAN_TIMEOUT 30000
void iwl_mvm_scan_timeout_wk(struct work_struct *work)
{
@@ -2540,6 +2561,7 @@ struct iwl_scan_umac_handler {
static const struct iwl_scan_umac_handler iwl_scan_umac_handlers[] = {
/* set the newest version first to shorten the list traverse time */
+ IWL_SCAN_UMAC_HANDLER(15),
IWL_SCAN_UMAC_HANDLER(14),
IWL_SCAN_UMAC_HANDLER(12),
};
@@ -2940,15 +2962,14 @@ static int iwl_mvm_scan_stop_wait(struct iwl_mvm *mvm, int type)
1 * HZ);
}
-#define IWL_SCAN_REQ_UMAC_HANDLE_SIZE(_ver) { \
- case (_ver): return sizeof(struct iwl_scan_req_umac_v##_ver); \
-}
-
static int iwl_scan_req_umac_get_size(u8 scan_ver)
{
switch (scan_ver) {
- IWL_SCAN_REQ_UMAC_HANDLE_SIZE(14);
- IWL_SCAN_REQ_UMAC_HANDLE_SIZE(12);
+ case 12:
+ return sizeof(struct iwl_scan_req_umac_v12);
+ case 14:
+ case 15:
+ return sizeof(struct iwl_scan_req_umac_v15);
}
return 0;
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/sta.c b/drivers/net/wireless/intel/iwlwifi/mvm/sta.c
index a64874c..feab0bf 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/sta.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/sta.c
@@ -2684,6 +2684,16 @@ int iwl_mvm_sta_rx_agg(struct iwl_mvm *mvm, struct ieee80211_sta *sta,
RCU_INIT_POINTER(mvm->baid_map[baid], NULL);
kfree_rcu(baid_data, rcu_head);
IWL_DEBUG_HT(mvm, "BAID %d is free\n", baid);
+
+ /*
+ * After we've deleted it, do another queue sync
+ * so if an IWL_MVM_RXQ_NSSN_SYNC was concurrently
+ * running it won't find a new session in the old
+ * BAID. It can find the NULL pointer for the BAID,
+ * but we must not have it find a different session.
+ */
+ iwl_mvm_sync_rx_queues_internal(mvm, IWL_MVM_RXQ_EMPTY,
+ true, NULL, 0);
}
return 0;
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/sta.h b/drivers/net/wireless/intel/iwlwifi/mvm/sta.h
index 32b4d19..e34b82b 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/sta.h
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/sta.h
@@ -1,6 +1,6 @@
/* SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause */
/*
- * Copyright (C) 2012-2014, 2018-2020 Intel Corporation
+ * Copyright (C) 2012-2014, 2018-2021 Intel Corporation
* Copyright (C) 2013-2014 Intel Mobile Communications GmbH
* Copyright (C) 2015-2016 Intel Deutschland GmbH
*/
@@ -373,6 +373,7 @@ struct iwl_mvm_rxq_dup_data {
* @tx_ant: the index of the antenna to use for data tx to this station. Only
* used during connection establishment (e.g. for the 4 way handshake
* exchange).
+ * @pairwise_cipher: used to feed iwlmei upon authorization
*
* When mac80211 creates a station it reserves some space (hw->sta_data_size)
* in the structure for use by driver. This structure is placed in that
@@ -415,6 +416,7 @@ struct iwl_mvm_sta {
u8 sleep_tx_count;
u8 avg_energy;
u8 tx_ant;
+ u32 pairwise_cipher;
};
u16 iwl_mvm_tid_queued(struct iwl_mvm *mvm, struct iwl_mvm_tid_data *tid_data);
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/time-event.c b/drivers/net/wireless/intel/iwlwifi/mvm/time-event.c
index e91f8e8..ab06dcd 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/time-event.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/time-event.c
@@ -49,14 +49,13 @@ void iwl_mvm_roc_done_wk(struct work_struct *wk)
struct iwl_mvm *mvm = container_of(wk, struct iwl_mvm, roc_done_wk);
/*
- * Clear the ROC_RUNNING /ROC_AUX_RUNNING status bit.
+ * Clear the ROC_RUNNING status bit.
* This will cause the TX path to drop offchannel transmissions.
* That would also be done by mac80211, but it is racy, in particular
* in the case that the time event actually completed in the firmware
* (which is handled in iwl_mvm_te_handle_notif).
*/
clear_bit(IWL_MVM_STATUS_ROC_RUNNING, &mvm->status);
- clear_bit(IWL_MVM_STATUS_ROC_AUX_RUNNING, &mvm->status);
synchronize_net();
@@ -82,9 +81,19 @@ void iwl_mvm_roc_done_wk(struct work_struct *wk)
mvmvif = iwl_mvm_vif_from_mac80211(mvm->p2p_device_vif);
iwl_mvm_flush_sta(mvm, &mvmvif->bcast_sta, true);
}
- } else {
+ }
+
+ /*
+ * Clear the ROC_AUX_RUNNING status bit.
+ * This will cause the TX path to drop offchannel transmissions.
+ * That would also be done by mac80211, but it is racy, in particular
+ * in the case that the time event actually completed in the firmware
+ * (which is handled in iwl_mvm_te_handle_notif).
+ */
+ if (test_and_clear_bit(IWL_MVM_STATUS_ROC_AUX_RUNNING, &mvm->status)) {
/* do the same in case of hot spot 2.0 */
iwl_mvm_flush_sta(mvm, &mvm->aux_sta, true);
+
/* In newer version of this command an aux station is added only
* in cases of dedicated tx queue and need to be removed in end
* of use */
@@ -687,11 +696,14 @@ static bool __iwl_mvm_remove_time_event(struct iwl_mvm *mvm,
iwl_mvm_te_clear_data(mvm, te_data);
spin_unlock_bh(&mvm->time_event_lock);
- /* When session protection is supported, the te_data->id field
+ /* When session protection is used, the te_data->id field
* is reused to save session protection's configuration.
+ * For AUX ROC, HOT_SPOT_CMD is used and the te_data->id field is set
+ * to HOT_SPOT_CMD.
*/
if (fw_has_capa(&mvm->fw->ucode_capa,
- IWL_UCODE_TLV_CAPA_SESSION_PROT_CMD)) {
+ IWL_UCODE_TLV_CAPA_SESSION_PROT_CMD) &&
+ id != HOT_SPOT_CMD) {
if (mvmvif && id < SESSION_PROTECT_CONF_MAX_ID) {
/* Session protection is still ongoing. Cancel it */
iwl_mvm_cancel_session_protection(mvm, mvmvif, id);
@@ -1027,7 +1039,7 @@ void iwl_mvm_stop_roc(struct iwl_mvm *mvm, struct ieee80211_vif *vif)
iwl_mvm_p2p_roc_finished(mvm);
} else {
iwl_mvm_remove_aux_roc_te(mvm, mvmvif,
- &mvmvif->time_event_data);
+ &mvmvif->hs_time_event_data);
iwl_mvm_roc_finished(mvm);
}
@@ -1158,15 +1170,10 @@ void iwl_mvm_schedule_session_protection(struct iwl_mvm *mvm,
cpu_to_le32(FW_CMD_ID_AND_COLOR(mvmvif->id,
mvmvif->color)),
.action = cpu_to_le32(FW_CTXT_ACTION_ADD),
+ .conf_id = cpu_to_le32(SESSION_PROTECT_CONF_ASSOC),
.duration_tu = cpu_to_le32(MSEC_TO_TU(duration)),
};
- /* The time_event_data.id field is reused to save session
- * protection's configuration.
- */
- mvmvif->time_event_data.id = SESSION_PROTECT_CONF_ASSOC;
- cmd.conf_id = cpu_to_le32(mvmvif->time_event_data.id);
-
lockdep_assert_held(&mvm->mutex);
spin_lock_bh(&mvm->time_event_lock);
@@ -1180,6 +1187,11 @@ void iwl_mvm_schedule_session_protection(struct iwl_mvm *mvm,
}
iwl_mvm_te_clear_data(mvm, te_data);
+ /*
+ * The time_event_data.id field is reused to save session
+ * protection's configuration.
+ */
+ te_data->id = le32_to_cpu(cmd.conf_id);
te_data->duration = le32_to_cpu(cmd.duration_tu);
te_data->vif = vif;
spin_unlock_bh(&mvm->time_event_lock);
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/tx.c b/drivers/net/wireless/intel/iwlwifi/mvm/tx.c
index 76e0b7b..6fa2c12 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/tx.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/tx.c
@@ -1,6 +1,6 @@
// SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause
/*
- * Copyright (C) 2012-2014, 2018-2020 Intel Corporation
+ * Copyright (C) 2012-2014, 2018-2021 Intel Corporation
* Copyright (C) 2013-2015 Intel Mobile Communications GmbH
* Copyright (C) 2016-2017 Intel Deutschland GmbH
*/
@@ -39,11 +39,11 @@ iwl_mvm_bar_check_trigger(struct iwl_mvm *mvm, const u8 *addr,
#define OPT_HDR(type, skb, off) \
(type *)(skb_network_header(skb) + (off))
-static u16 iwl_mvm_tx_csum(struct iwl_mvm *mvm, struct sk_buff *skb,
- struct ieee80211_hdr *hdr,
- struct ieee80211_tx_info *info,
- u16 offload_assist)
+static u16 iwl_mvm_tx_csum_pre_bz(struct iwl_mvm *mvm, struct sk_buff *skb,
+ struct ieee80211_tx_info *info, bool amsdu)
{
+ struct ieee80211_hdr *hdr = (void *)skb->data;
+ u16 offload_assist = 0;
#if IS_ENABLED(CONFIG_INET)
u16 mh_len = ieee80211_hdrlen(hdr->frame_control);
u8 protocol = 0;
@@ -106,8 +106,7 @@ static u16 iwl_mvm_tx_csum(struct iwl_mvm *mvm, struct sk_buff *skb,
offload_assist |= (4 << TX_CMD_OFFLD_IP_HDR);
/* Do IPv4 csum for AMSDU only (no IP csum for Ipv6) */
- if (skb->protocol == htons(ETH_P_IP) &&
- (offload_assist & BIT(TX_CMD_OFFLD_AMSDU))) {
+ if (skb->protocol == htons(ETH_P_IP) && amsdu) {
ip_hdr(skb)->check = 0;
offload_assist |= BIT(TX_CMD_OFFLD_L3_EN);
}
@@ -132,9 +131,63 @@ static u16 iwl_mvm_tx_csum(struct iwl_mvm *mvm, struct sk_buff *skb,
out:
#endif
+ if (amsdu)
+ offload_assist |= BIT(TX_CMD_OFFLD_AMSDU);
+ else if (ieee80211_hdrlen(hdr->frame_control) % 4)
+ /* padding is inserted later in transport */
+ offload_assist |= BIT(TX_CMD_OFFLD_PAD);
+
return offload_assist;
}
+u32 iwl_mvm_tx_csum_bz(struct iwl_mvm *mvm, struct sk_buff *skb, bool amsdu)
+{
+ struct ieee80211_hdr *hdr = (void *)skb->data;
+ u32 offload_assist = IWL_TX_CMD_OFFLD_BZ_PARTIAL_CSUM;
+ unsigned int hdrlen = ieee80211_hdrlen(hdr->frame_control);
+ unsigned int csum_start = skb_checksum_start_offset(skb);
+
+ offload_assist |= u32_encode_bits(hdrlen / 2,
+ IWL_TX_CMD_OFFLD_BZ_MH_LEN);
+ if (amsdu)
+ offload_assist |= IWL_TX_CMD_OFFLD_BZ_AMSDU;
+ else if (hdrlen % 4)
+ /* padding is inserted later in transport */
+ offload_assist |= IWL_TX_CMD_OFFLD_BZ_MH_PAD;
+
+ if (skb->ip_summed != CHECKSUM_PARTIAL)
+ return offload_assist;
+
+ offload_assist |= IWL_TX_CMD_OFFLD_BZ_ENABLE_CSUM |
+ IWL_TX_CMD_OFFLD_BZ_ZERO2ONES;
+
+ /*
+ * mac80211 will always calculate checksum in software for
+ * non-fast-xmit, and so we can only do offloaded checksum
+ * for fast-xmit frames. In this case, we always have the
+ * RFC 1042 header present. skb_checksum_start_offset()
+ * returns the offset from the beginning, but the hardware
+ * needs it from after the header & SNAP header.
+ */
+ csum_start -= hdrlen + 8;
+
+ offload_assist |= u32_encode_bits(csum_start,
+ IWL_TX_CMD_OFFLD_BZ_START_OFFS);
+ offload_assist |= u32_encode_bits(csum_start + skb->csum_offset,
+ IWL_TX_CMD_OFFLD_BZ_RESULT_OFFS);
+
+ return offload_assist;
+}
+
+static u32 iwl_mvm_tx_csum(struct iwl_mvm *mvm, struct sk_buff *skb,
+ struct ieee80211_tx_info *info,
+ bool amsdu)
+{
+ if (mvm->trans->trans_cfg->device_family < IWL_DEVICE_FAMILY_BZ)
+ return iwl_mvm_tx_csum_pre_bz(mvm, skb, info, amsdu);
+ return iwl_mvm_tx_csum_bz(mvm, skb, amsdu);
+}
+
/*
* Sets most of the Tx cmd's fields
*/
@@ -146,7 +199,7 @@ void iwl_mvm_set_tx_cmd(struct iwl_mvm *mvm, struct sk_buff *skb,
__le16 fc = hdr->frame_control;
u32 tx_flags = le32_to_cpu(tx_cmd->tx_flags);
u32 len = skb->len + FCS_LEN;
- u16 offload_assist = 0;
+ bool amsdu = false;
u8 ac;
if (!(info->flags & IEEE80211_TX_CTL_NO_ACK) ||
@@ -166,8 +219,7 @@ void iwl_mvm_set_tx_cmd(struct iwl_mvm *mvm, struct sk_buff *skb,
u8 *qc = ieee80211_get_qos_ctl(hdr);
tx_cmd->tid_tspec = qc[0] & 0xf;
tx_flags &= ~TX_CMD_FLG_SEQ_CTL;
- if (*qc & IEEE80211_QOS_CTL_A_MSDU_PRESENT)
- offload_assist |= BIT(TX_CMD_OFFLD_AMSDU);
+ amsdu = *qc & IEEE80211_QOS_CTL_A_MSDU_PRESENT;
} else if (ieee80211_is_back_req(fc)) {
struct ieee80211_bar *bar = (void *)skb->data;
u16 control = le16_to_cpu(bar->control);
@@ -234,14 +286,8 @@ void iwl_mvm_set_tx_cmd(struct iwl_mvm *mvm, struct sk_buff *skb,
tx_cmd->life_time = cpu_to_le32(TX_CMD_LIFE_TIME_INFINITE);
tx_cmd->sta_id = sta_id;
- /* padding is inserted later in transport */
- if (ieee80211_hdrlen(fc) % 4 &&
- !(offload_assist & BIT(TX_CMD_OFFLD_AMSDU)))
- offload_assist |= BIT(TX_CMD_OFFLD_PAD);
-
- tx_cmd->offload_assist |=
- cpu_to_le16(iwl_mvm_tx_csum(mvm, skb, hdr, info,
- offload_assist));
+ tx_cmd->offload_assist =
+ cpu_to_le16(iwl_mvm_tx_csum_pre_bz(mvm, skb, info, amsdu));
}
static u32 iwl_mvm_get_tx_ant(struct iwl_mvm *mvm,
@@ -463,27 +509,18 @@ iwl_mvm_set_tx_params(struct iwl_mvm *mvm, struct sk_buff *skb,
dev_cmd->hdr.cmd = TX_CMD;
if (iwl_mvm_has_new_tx_api(mvm)) {
- u16 offload_assist = 0;
u32 rate_n_flags = 0;
u16 flags = 0;
struct iwl_mvm_sta *mvmsta = sta ?
iwl_mvm_sta_from_mac80211(sta) : NULL;
+ bool amsdu = false;
if (ieee80211_is_data_qos(hdr->frame_control)) {
u8 *qc = ieee80211_get_qos_ctl(hdr);
- if (*qc & IEEE80211_QOS_CTL_A_MSDU_PRESENT)
- offload_assist |= BIT(TX_CMD_OFFLD_AMSDU);
+ amsdu = *qc & IEEE80211_QOS_CTL_A_MSDU_PRESENT;
}
- offload_assist = iwl_mvm_tx_csum(mvm, skb, hdr, info,
- offload_assist);
-
- /* padding is inserted later in transport */
- if (ieee80211_hdrlen(hdr->frame_control) % 4 &&
- !(offload_assist & BIT(TX_CMD_OFFLD_AMSDU)))
- offload_assist |= BIT(TX_CMD_OFFLD_PAD);
-
if (!info->control.hw_key)
flags |= IWL_TX_FLAGS_ENCRYPT_DIS;
@@ -503,8 +540,10 @@ iwl_mvm_set_tx_params(struct iwl_mvm *mvm, struct sk_buff *skb,
if (mvm->trans->trans_cfg->device_family >=
IWL_DEVICE_FAMILY_AX210) {
struct iwl_tx_cmd_gen3 *cmd = (void *)dev_cmd->payload;
+ u32 offload_assist = iwl_mvm_tx_csum(mvm, skb,
+ info, amsdu);
- cmd->offload_assist |= cpu_to_le32(offload_assist);
+ cmd->offload_assist = cpu_to_le32(offload_assist);
/* Total # bytes to be transmitted */
cmd->len = cpu_to_le16((u16)skb->len);
@@ -516,8 +555,11 @@ iwl_mvm_set_tx_params(struct iwl_mvm *mvm, struct sk_buff *skb,
cmd->rate_n_flags = cpu_to_le32(rate_n_flags);
} else {
struct iwl_tx_cmd_gen2 *cmd = (void *)dev_cmd->payload;
+ u16 offload_assist = iwl_mvm_tx_csum_pre_bz(mvm, skb,
+ info,
+ amsdu);
- cmd->offload_assist |= cpu_to_le16(offload_assist);
+ cmd->offload_assist = cpu_to_le16(offload_assist);
/* Total # bytes to be transmitted */
cmd->len = cpu_to_le16((u16)skb->len);
@@ -1129,6 +1171,11 @@ static int iwl_mvm_tx_mpdu(struct iwl_mvm *mvm, struct sk_buff *skb,
/* From now on, we cannot access info->control */
iwl_mvm_skb_prepare_status(skb, dev_cmd);
+ if (ieee80211_is_data(fc))
+ iwl_mvm_mei_tx_copy_to_csme(mvm, skb,
+ info->control.hw_key ?
+ info->control.hw_key->iv_len : 0);
+
if (iwl_trans_tx(mvm->trans, skb, dev_cmd, txq_id))
goto drop_unlock_sta;
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/utils.c b/drivers/net/wireless/intel/iwlwifi/mvm/utils.c
index caf1dcf..1f3e90e 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/utils.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/utils.c
@@ -340,25 +340,64 @@ void iwl_mvm_update_smps(struct iwl_mvm *mvm, struct ieee80211_vif *vif,
ieee80211_request_smps(vif, smps_mode);
}
+static bool iwl_wait_stats_complete(struct iwl_notif_wait_data *notif_wait,
+ struct iwl_rx_packet *pkt, void *data)
+{
+ WARN_ON(pkt->hdr.cmd != STATISTICS_NOTIFICATION);
+
+ return true;
+}
+
int iwl_mvm_request_statistics(struct iwl_mvm *mvm, bool clear)
{
struct iwl_statistics_cmd scmd = {
.flags = clear ? cpu_to_le32(IWL_STATISTICS_FLG_CLEAR) : 0,
};
+
struct iwl_host_cmd cmd = {
.id = STATISTICS_CMD,
.len[0] = sizeof(scmd),
.data[0] = &scmd,
- .flags = CMD_WANT_SKB,
};
int ret;
- ret = iwl_mvm_send_cmd(mvm, &cmd);
- if (ret)
- return ret;
+ /* From version 15 - STATISTICS_NOTIFICATION, the reply for
+ * STATISTICS_CMD is empty, and the response is with
+ * STATISTICS_NOTIFICATION notification
+ */
+ if (iwl_fw_lookup_notif_ver(mvm->fw, LEGACY_GROUP,
+ STATISTICS_NOTIFICATION, 0) < 15) {
+ cmd.flags = CMD_WANT_SKB;
- iwl_mvm_handle_rx_statistics(mvm, cmd.resp_pkt);
- iwl_free_resp(&cmd);
+ ret = iwl_mvm_send_cmd(mvm, &cmd);
+ if (ret)
+ return ret;
+
+ iwl_mvm_handle_rx_statistics(mvm, cmd.resp_pkt);
+ iwl_free_resp(&cmd);
+ } else {
+ struct iwl_notification_wait stats_wait;
+ static const u16 stats_complete[] = {
+ STATISTICS_NOTIFICATION,
+ };
+
+ iwl_init_notification_wait(&mvm->notif_wait, &stats_wait,
+ stats_complete, ARRAY_SIZE(stats_complete),
+ iwl_wait_stats_complete, NULL);
+
+ ret = iwl_mvm_send_cmd(mvm, &cmd);
+ if (ret) {
+ iwl_remove_notification(&mvm->notif_wait, &stats_wait);
+ return ret;
+ }
+
+ /* 200ms should be enough for FW to collect data from all
+ * LMACs and send STATISTICS_NOTIFICATION to host
+ */
+ ret = iwl_wait_notification(&mvm->notif_wait, &stats_wait, HZ / 5);
+ if (ret)
+ return ret;
+ }
if (clear)
iwl_mvm_accu_radio_stats(mvm);
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/vendor-cmd.c b/drivers/net/wireless/intel/iwlwifi/mvm/vendor-cmd.c
new file mode 100644
index 0000000..7845036
--- /dev/null
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/vendor-cmd.c
@@ -0,0 +1,152 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2021 Intel Corporation
+ */
+#include "mvm.h"
+#include <linux/nl80211-vnd-intel.h>
+#include <net/netlink.h>
+
+static const struct nla_policy
+iwl_mvm_vendor_attr_policy[NUM_IWL_MVM_VENDOR_ATTR] = {
+ [IWL_MVM_VENDOR_ATTR_ROAMING_FORBIDDEN] = { .type = NLA_U8 },
+ [IWL_MVM_VENDOR_ATTR_AUTH_MODE] = { .type = NLA_U32 },
+ [IWL_MVM_VENDOR_ATTR_CHANNEL_NUM] = { .type = NLA_U8 },
+ [IWL_MVM_VENDOR_ATTR_SSID] = { .type = NLA_BINARY,
+ .len = IEEE80211_MAX_SSID_LEN },
+ [IWL_MVM_VENDOR_ATTR_BAND] = { .type = NLA_U8 },
+ [IWL_MVM_VENDOR_ATTR_COLLOC_CHANNEL] = { .type = NLA_U8 },
+ [IWL_MVM_VENDOR_ATTR_COLLOC_ADDR] = { .type = NLA_BINARY, .len = ETH_ALEN },
+};
+
+static int iwl_mvm_vendor_get_csme_conn_info(struct wiphy *wiphy,
+ struct wireless_dev *wdev,
+ const void *data, int data_len)
+{
+ struct ieee80211_hw *hw = wiphy_to_ieee80211_hw(wiphy);
+ struct iwl_mvm *mvm = IWL_MAC80211_GET_MVM(hw);
+ struct iwl_mvm_csme_conn_info *csme_conn_info;
+ struct sk_buff *skb;
+ int err = 0;
+
+ mutex_lock(&mvm->mutex);
+ csme_conn_info = iwl_mvm_get_csme_conn_info(mvm);
+
+ if (!csme_conn_info) {
+ err = -EINVAL;
+ goto out_unlock;
+ }
+
+ skb = cfg80211_vendor_cmd_alloc_reply_skb(wiphy, 200);
+ if (!skb) {
+ err = -ENOMEM;
+ goto out_unlock;
+ }
+
+ if (nla_put_u32(skb, IWL_MVM_VENDOR_ATTR_AUTH_MODE,
+ csme_conn_info->conn_info.auth_mode) ||
+ nla_put(skb, IWL_MVM_VENDOR_ATTR_SSID,
+ csme_conn_info->conn_info.ssid_len,
+ csme_conn_info->conn_info.ssid) ||
+ nla_put_u32(skb, IWL_MVM_VENDOR_ATTR_STA_CIPHER,
+ csme_conn_info->conn_info.pairwise_cipher) ||
+ nla_put_u8(skb, IWL_MVM_VENDOR_ATTR_CHANNEL_NUM,
+ csme_conn_info->conn_info.channel) ||
+ nla_put(skb, IWL_MVM_VENDOR_ATTR_ADDR, ETH_ALEN,
+ csme_conn_info->conn_info.bssid)) {
+ kfree_skb(skb);
+ err = -ENOBUFS;
+ }
+
+out_unlock:
+ mutex_unlock(&mvm->mutex);
+ if (err)
+ return err;
+
+ return cfg80211_vendor_cmd_reply(skb);
+}
+
+static int iwl_mvm_vendor_host_get_ownership(struct wiphy *wiphy,
+ struct wireless_dev *wdev,
+ const void *data, int data_len)
+{
+ struct ieee80211_hw *hw = wiphy_to_ieee80211_hw(wiphy);
+ struct iwl_mvm *mvm = IWL_MAC80211_GET_MVM(hw);
+
+ mutex_lock(&mvm->mutex);
+ iwl_mvm_mei_get_ownership(mvm);
+ mutex_unlock(&mvm->mutex);
+
+ return 0;
+}
+
+static const struct wiphy_vendor_command iwl_mvm_vendor_commands[] = {
+ {
+ .info = {
+ .vendor_id = INTEL_OUI,
+ .subcmd = IWL_MVM_VENDOR_CMD_GET_CSME_CONN_INFO,
+ },
+ .doit = iwl_mvm_vendor_get_csme_conn_info,
+ .flags = WIPHY_VENDOR_CMD_NEED_WDEV,
+ .policy = iwl_mvm_vendor_attr_policy,
+ .maxattr = MAX_IWL_MVM_VENDOR_ATTR,
+ },
+ {
+ .info = {
+ .vendor_id = INTEL_OUI,
+ .subcmd = IWL_MVM_VENDOR_CMD_HOST_GET_OWNERSHIP,
+ },
+ .doit = iwl_mvm_vendor_host_get_ownership,
+ .flags = WIPHY_VENDOR_CMD_NEED_WDEV,
+ .policy = iwl_mvm_vendor_attr_policy,
+ .maxattr = MAX_IWL_MVM_VENDOR_ATTR,
+ },
+};
+
+enum iwl_mvm_vendor_events_idx {
+ /* 0x0 - 0x3 are deprecated */
+ IWL_MVM_VENDOR_EVENT_IDX_ROAMING_FORBIDDEN = 4,
+ NUM_IWL_MVM_VENDOR_EVENT_IDX
+};
+
+static const struct nl80211_vendor_cmd_info
+iwl_mvm_vendor_events[NUM_IWL_MVM_VENDOR_EVENT_IDX] = {
+ [IWL_MVM_VENDOR_EVENT_IDX_ROAMING_FORBIDDEN] = {
+ .vendor_id = INTEL_OUI,
+ .subcmd = IWL_MVM_VENDOR_CMD_ROAMING_FORBIDDEN_EVENT,
+ },
+};
+
+void iwl_mvm_vendor_cmds_register(struct iwl_mvm *mvm)
+{
+ mvm->hw->wiphy->vendor_commands = iwl_mvm_vendor_commands;
+ mvm->hw->wiphy->n_vendor_commands = ARRAY_SIZE(iwl_mvm_vendor_commands);
+ mvm->hw->wiphy->vendor_events = iwl_mvm_vendor_events;
+ mvm->hw->wiphy->n_vendor_events = ARRAY_SIZE(iwl_mvm_vendor_events);
+}
+
+void iwl_mvm_send_roaming_forbidden_event(struct iwl_mvm *mvm,
+ struct ieee80211_vif *vif,
+ bool forbidden)
+{
+ struct sk_buff *msg =
+ cfg80211_vendor_event_alloc(mvm->hw->wiphy,
+ ieee80211_vif_to_wdev(vif),
+ 200, IWL_MVM_VENDOR_EVENT_IDX_ROAMING_FORBIDDEN,
+ GFP_ATOMIC);
+ if (!msg)
+ return;
+
+ if (WARN_ON(!vif))
+ return;
+
+ if (nla_put(msg, IWL_MVM_VENDOR_ATTR_VIF_ADDR,
+ ETH_ALEN, vif->addr) ||
+ nla_put_u8(msg, IWL_MVM_VENDOR_ATTR_ROAMING_FORBIDDEN, forbidden))
+ goto nla_put_failure;
+
+ cfg80211_vendor_event(msg, GFP_ATOMIC);
+ return;
+
+ nla_put_failure:
+ kfree_skb(msg);
+}
diff --git a/drivers/net/wireless/intel/iwlwifi/pcie/drv.c b/drivers/net/wireless/intel/iwlwifi/pcie/drv.c
index 5ce07f2..5178e85 100644
--- a/drivers/net/wireless/intel/iwlwifi/pcie/drv.c
+++ b/drivers/net/wireless/intel/iwlwifi/pcie/drv.c
@@ -510,16 +510,16 @@ static const struct pci_device_id iwl_hw_card_ids[] = {
MODULE_DEVICE_TABLE(pci, iwl_hw_card_ids);
#define _IWL_DEV_INFO(_device, _subdevice, _mac_type, _mac_step, _rf_type, \
- _rf_id, _no_160, _cores, _cdb, _cfg, _name) \
+ _rf_id, _no_160, _cores, _cdb, _jacket, _cfg, _name) \
{ .device = (_device), .subdevice = (_subdevice), .cfg = &(_cfg), \
.name = _name, .mac_type = _mac_type, .rf_type = _rf_type, \
.no_160 = _no_160, .cores = _cores, .rf_id = _rf_id, \
- .mac_step = _mac_step, .cdb = _cdb }
+ .mac_step = _mac_step, .cdb = _cdb, .jacket = _jacket }
#define IWL_DEV_INFO(_device, _subdevice, _cfg, _name) \
_IWL_DEV_INFO(_device, _subdevice, IWL_CFG_ANY, IWL_CFG_ANY, \
IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_ANY, \
- IWL_CFG_ANY, _cfg, _name)
+ IWL_CFG_ANY, IWL_CFG_ANY, _cfg, _name)
static const struct iwl_dev_info iwl_dev_info_table[] = {
#if IS_ENABLED(CONFIG_IWLMVM)
@@ -562,6 +562,7 @@ static const struct iwl_dev_info iwl_dev_info_table[] = {
IWL_DEV_INFO(0x43F0, 0x1652, killer1650i_2ax_cfg_qu_b0_hr_b0, iwl_ax201_killer_1650i_name),
IWL_DEV_INFO(0x43F0, 0x2074, iwl_ax201_cfg_qu_hr, NULL),
IWL_DEV_INFO(0x43F0, 0x4070, iwl_ax201_cfg_qu_hr, NULL),
+ IWL_DEV_INFO(0x43F0, 0x1651, killer1650s_2ax_cfg_qu_b0_hr_b0, iwl_ax201_killer_1650s_name),
IWL_DEV_INFO(0xA0F0, 0x0070, iwl_ax201_cfg_qu_hr, NULL),
IWL_DEV_INFO(0xA0F0, 0x0074, iwl_ax201_cfg_qu_hr, NULL),
IWL_DEV_INFO(0xA0F0, 0x0078, iwl_ax201_cfg_qu_hr, NULL),
@@ -665,97 +666,111 @@ static const struct iwl_dev_info iwl_dev_info_table[] = {
IWL_DEV_INFO(0x2726, 0x0510, iwlax211_cfg_snj_gf_a0, NULL),
IWL_DEV_INFO(0x2726, 0x1651, iwl_cfg_snj_hr_b0, iwl_ax201_killer_1650s_name),
IWL_DEV_INFO(0x2726, 0x1652, iwl_cfg_snj_hr_b0, iwl_ax201_killer_1650i_name),
- IWL_DEV_INFO(0x2726, 0x1671, iwlax211_cfg_snj_gf_a0, iwl_ax211_killer_1675s_name),
- IWL_DEV_INFO(0x2726, 0x1672, iwlax211_cfg_snj_gf_a0, iwl_ax211_killer_1675i_name),
IWL_DEV_INFO(0x2726, 0x1691, iwlax411_2ax_cfg_sosnj_gf4_a0, iwl_ax411_killer_1690s_name),
IWL_DEV_INFO(0x2726, 0x1692, iwlax411_2ax_cfg_sosnj_gf4_a0, iwl_ax411_killer_1690i_name),
IWL_DEV_INFO(0x7F70, 0x1691, iwlax411_2ax_cfg_sosnj_gf4_a0, iwl_ax411_killer_1690s_name),
IWL_DEV_INFO(0x7F70, 0x1692, iwlax411_2ax_cfg_sosnj_gf4_a0, iwl_ax411_killer_1690i_name),
+ /* SO with GF2 */
+ IWL_DEV_INFO(0x2726, 0x1671, iwlax211_2ax_cfg_so_gf_a0, iwl_ax211_killer_1675s_name),
+ IWL_DEV_INFO(0x2726, 0x1672, iwlax211_2ax_cfg_so_gf_a0, iwl_ax211_killer_1675i_name),
+ IWL_DEV_INFO(0x51F0, 0x1671, iwlax211_2ax_cfg_so_gf_a0, iwl_ax211_killer_1675s_name),
+ IWL_DEV_INFO(0x51F0, 0x1672, iwlax211_2ax_cfg_so_gf_a0, iwl_ax211_killer_1675i_name),
+ IWL_DEV_INFO(0x54F0, 0x1671, iwlax211_2ax_cfg_so_gf_a0, iwl_ax211_killer_1675s_name),
+ IWL_DEV_INFO(0x54F0, 0x1672, iwlax211_2ax_cfg_so_gf_a0, iwl_ax211_killer_1675i_name),
+ IWL_DEV_INFO(0x7A70, 0x1671, iwlax211_2ax_cfg_so_gf_a0, iwl_ax211_killer_1675s_name),
+ IWL_DEV_INFO(0x7A70, 0x1672, iwlax211_2ax_cfg_so_gf_a0, iwl_ax211_killer_1675i_name),
+ IWL_DEV_INFO(0x7AF0, 0x1671, iwlax211_2ax_cfg_so_gf_a0, iwl_ax211_killer_1675s_name),
+ IWL_DEV_INFO(0x7AF0, 0x1672, iwlax211_2ax_cfg_so_gf_a0, iwl_ax211_killer_1675i_name),
+
+ /* MA with GF2 */
+ IWL_DEV_INFO(0x7E40, 0x1671, iwl_cfg_ma_a0_gf_a0, iwl_ax211_killer_1675s_name),
+ IWL_DEV_INFO(0x7E40, 0x1672, iwl_cfg_ma_a0_gf_a0, iwl_ax211_killer_1675i_name),
+
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_PU, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1,
- IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_2ac_cfg_soc, iwl9461_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_PU, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1,
- IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_2ac_cfg_soc, iwl9461_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_PU, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV,
- IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_2ac_cfg_soc, iwl9462_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_PU, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV,
- IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_2ac_cfg_soc, iwl9462_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_PU, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
- IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_2ac_cfg_soc, iwl9560_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_PU, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
- IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_2ac_cfg_soc, iwl9560_name),
_IWL_DEV_INFO(0x2526, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_PNJ, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1,
- IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9260_2ac_cfg, iwl9461_160_name),
_IWL_DEV_INFO(0x2526, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_PNJ, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1,
- IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9260_2ac_cfg, iwl9461_name),
_IWL_DEV_INFO(0x2526, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_PNJ, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV,
- IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9260_2ac_cfg, iwl9462_160_name),
_IWL_DEV_INFO(0x2526, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_PNJ, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV,
- IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9260_2ac_cfg, iwl9462_name),
_IWL_DEV_INFO(0x2526, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_TH, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_TH, IWL_CFG_ANY,
- IWL_CFG_160, IWL_CFG_CORES_BT_GNSS, IWL_CFG_NO_CDB,
+ IWL_CFG_160, IWL_CFG_CORES_BT_GNSS, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9260_2ac_cfg, iwl9270_160_name),
_IWL_DEV_INFO(0x2526, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_TH, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_TH, IWL_CFG_ANY,
- IWL_CFG_NO_160, IWL_CFG_CORES_BT_GNSS, IWL_CFG_NO_CDB,
+ IWL_CFG_NO_160, IWL_CFG_CORES_BT_GNSS, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9260_2ac_cfg, iwl9270_name),
_IWL_DEV_INFO(0x271B, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_TH, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_TH1, IWL_CFG_ANY,
- IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9260_2ac_cfg, iwl9162_160_name),
_IWL_DEV_INFO(0x271B, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_TH, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_TH1, IWL_CFG_ANY,
- IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9260_2ac_cfg, iwl9162_name),
_IWL_DEV_INFO(0x2526, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_TH, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_TH, IWL_CFG_ANY,
- IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9260_2ac_cfg, iwl9260_160_name),
_IWL_DEV_INFO(0x2526, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_TH, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_TH, IWL_CFG_ANY,
- IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9260_2ac_cfg, iwl9260_name),
/* Qu with Jf */
@@ -763,176 +778,176 @@ static const struct iwl_dev_info iwl_dev_info_table[] = {
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QU, SILICON_B_STEP,
IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1,
- IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qu_b0_jf_b0_cfg, iwl9461_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QU, SILICON_B_STEP,
IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1,
- IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qu_b0_jf_b0_cfg, iwl9461_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QU, SILICON_B_STEP,
IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV,
- IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qu_b0_jf_b0_cfg, iwl9462_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QU, SILICON_B_STEP,
IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV,
- IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qu_b0_jf_b0_cfg, iwl9462_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QU, SILICON_B_STEP,
IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
- IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qu_b0_jf_b0_cfg, iwl9560_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QU, SILICON_B_STEP,
IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
- IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qu_b0_jf_b0_cfg, iwl9560_name),
_IWL_DEV_INFO(IWL_CFG_ANY, 0x1551,
IWL_CFG_MAC_TYPE_QU, SILICON_B_STEP,
IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
- IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qu_b0_jf_b0_cfg, iwl9560_killer_1550s_name),
_IWL_DEV_INFO(IWL_CFG_ANY, 0x1552,
IWL_CFG_MAC_TYPE_QU, SILICON_B_STEP,
IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
- IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qu_b0_jf_b0_cfg, iwl9560_killer_1550i_name),
/* Qu C step */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QU, SILICON_C_STEP,
IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1,
- IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qu_c0_jf_b0_cfg, iwl9461_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QU, SILICON_C_STEP,
IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1,
- IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qu_c0_jf_b0_cfg, iwl9461_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QU, SILICON_C_STEP,
IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV,
- IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qu_c0_jf_b0_cfg, iwl9462_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QU, SILICON_C_STEP,
IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV,
- IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qu_c0_jf_b0_cfg, iwl9462_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QU, SILICON_C_STEP,
IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
- IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qu_c0_jf_b0_cfg, iwl9560_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QU, SILICON_C_STEP,
IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
- IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qu_c0_jf_b0_cfg, iwl9560_name),
_IWL_DEV_INFO(IWL_CFG_ANY, 0x1551,
IWL_CFG_MAC_TYPE_QU, SILICON_C_STEP,
IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
- IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qu_c0_jf_b0_cfg, iwl9560_killer_1550s_name),
_IWL_DEV_INFO(IWL_CFG_ANY, 0x1552,
IWL_CFG_MAC_TYPE_QU, SILICON_C_STEP,
IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
- IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qu_c0_jf_b0_cfg, iwl9560_killer_1550i_name),
/* QuZ */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QUZ, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1,
- IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_quz_a0_jf_b0_cfg, iwl9461_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QUZ, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1,
- IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_quz_a0_jf_b0_cfg, iwl9461_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QUZ, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV,
- IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_quz_a0_jf_b0_cfg, iwl9462_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QUZ, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV,
- IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_quz_a0_jf_b0_cfg, iwl9462_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QUZ, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
- IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_quz_a0_jf_b0_cfg, iwl9560_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QUZ, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
- IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_quz_a0_jf_b0_cfg, iwl9560_name),
_IWL_DEV_INFO(IWL_CFG_ANY, 0x1551,
IWL_CFG_MAC_TYPE_QUZ, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
- IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_quz_a0_jf_b0_cfg, iwl9560_killer_1550s_name),
_IWL_DEV_INFO(IWL_CFG_ANY, 0x1552,
IWL_CFG_MAC_TYPE_QUZ, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
- IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_quz_a0_jf_b0_cfg, iwl9560_killer_1550i_name),
/* QnJ */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QNJ, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1,
- IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qnj_b0_jf_b0_cfg, iwl9461_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QNJ, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1,
- IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qnj_b0_jf_b0_cfg, iwl9461_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QNJ, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV,
- IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qnj_b0_jf_b0_cfg, iwl9462_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QNJ, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV,
- IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qnj_b0_jf_b0_cfg, iwl9462_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QNJ, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
- IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qnj_b0_jf_b0_cfg, iwl9560_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QNJ, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
- IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qnj_b0_jf_b0_cfg, iwl9560_name),
_IWL_DEV_INFO(IWL_CFG_ANY, 0x1551,
IWL_CFG_MAC_TYPE_QNJ, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
- IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qnj_b0_jf_b0_cfg, iwl9560_killer_1550s_name),
_IWL_DEV_INFO(IWL_CFG_ANY, 0x1552,
IWL_CFG_MAC_TYPE_QNJ, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
- IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qnj_b0_jf_b0_cfg, iwl9560_killer_1550i_name),
/* Qu with Hr */
@@ -940,303 +955,352 @@ static const struct iwl_dev_info iwl_dev_info_table[] = {
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QU, SILICON_B_STEP,
IWL_CFG_RF_TYPE_HR1, IWL_CFG_ANY,
- IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB,
+ IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_qu_b0_hr1_b0, iwl_ax101_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QU, SILICON_B_STEP,
IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY,
- IWL_CFG_NO_160, IWL_CFG_ANY, IWL_CFG_NO_CDB,
+ IWL_CFG_NO_160, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_qu_b0_hr_b0, iwl_ax203_name),
/* Qu C step */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QU, SILICON_C_STEP,
IWL_CFG_RF_TYPE_HR1, IWL_CFG_ANY,
- IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB,
+ IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_qu_c0_hr1_b0, iwl_ax101_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QU, SILICON_C_STEP,
IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY,
- IWL_CFG_NO_160, IWL_CFG_ANY, IWL_CFG_NO_CDB,
+ IWL_CFG_NO_160, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_qu_c0_hr_b0, iwl_ax203_name),
+ _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
+ IWL_CFG_MAC_TYPE_QU, SILICON_C_STEP,
+ IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY,
+ IWL_CFG_160, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
+ iwl_qu_c0_hr_b0, iwl_ax201_name),
/* QuZ */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QUZ, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_HR1, IWL_CFG_ANY,
- IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB,
+ IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_quz_a0_hr1_b0, iwl_ax101_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QUZ, SILICON_B_STEP,
IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY,
- IWL_CFG_NO_160, IWL_CFG_ANY, IWL_CFG_NO_CDB,
+ IWL_CFG_NO_160, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_quz_a0_hr_b0, iwl_ax203_name),
/* QnJ with Hr */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QNJ, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY,
- IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB,
+ IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_qnj_b0_hr_b0_cfg, iwl_ax201_name),
/* SnJ with Jf */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SNJ, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1,
- IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_snj_a0_jf_b0, iwl9461_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SNJ, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1,
- IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_snj_a0_jf_b0, iwl9461_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SNJ, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV,
- IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_snj_a0_jf_b0, iwl9462_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SNJ, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV,
- IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_snj_a0_jf_b0, iwl9462_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SNJ, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
- IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_snj_a0_jf_b0, iwl9560_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SNJ, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
- IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_snj_a0_jf_b0, iwl9560_name),
/* SnJ with Hr */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SNJ, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_HR1, IWL_CFG_ANY,
- IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB,
+ IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_snj_hr_b0, iwl_ax101_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SNJ, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY,
- IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB,
+ IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_snj_hr_b0, iwl_ax201_name),
/* Ma */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_MA, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY,
- IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB,
+ IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_ma_a0_hr_b0, iwl_ax201_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_MA, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_GF, IWL_CFG_ANY,
- IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB,
+ IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_ma_a0_gf_a0, iwl_ax211_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_MA, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_GF, IWL_CFG_ANY,
- IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_CDB,
+ IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_CDB, IWL_CFG_ANY,
iwl_cfg_ma_a0_gf4_a0, iwl_ax211_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_MA, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_MR, IWL_CFG_ANY,
- IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB,
+ IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_ma_a0_mr_a0, iwl_ax221_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_MA, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_FM, IWL_CFG_ANY,
- IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB,
+ IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_ma_a0_fm_a0, iwl_ax231_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SNJ, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_MR, IWL_CFG_ANY,
- IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB,
+ IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_snj_a0_mr_a0, iwl_ax221_name),
/* So with Hr */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SO, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY,
- IWL_CFG_NO_160, IWL_CFG_ANY, IWL_CFG_NO_CDB,
+ IWL_CFG_NO_160, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_so_a0_hr_a0, iwl_ax203_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SO, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_HR1, IWL_CFG_ANY,
- IWL_CFG_160, IWL_CFG_ANY, IWL_CFG_NO_CDB,
+ IWL_CFG_160, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_so_a0_hr_a0, iwl_ax101_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SO, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY,
- IWL_CFG_160, IWL_CFG_ANY, IWL_CFG_NO_CDB,
+ IWL_CFG_160, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_so_a0_hr_a0, iwl_ax201_name),
/* So-F with Hr */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SOF, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY,
- IWL_CFG_NO_160, IWL_CFG_ANY, IWL_CFG_NO_CDB,
+ IWL_CFG_NO_160, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_so_a0_hr_a0, iwl_ax203_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SOF, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_HR1, IWL_CFG_ANY,
- IWL_CFG_160, IWL_CFG_ANY, IWL_CFG_NO_CDB,
+ IWL_CFG_160, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_so_a0_hr_a0, iwl_ax101_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SOF, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY,
- IWL_CFG_160, IWL_CFG_ANY, IWL_CFG_NO_CDB,
+ IWL_CFG_160, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_so_a0_hr_a0, iwl_ax201_name),
/* So-F with Gf */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SOF, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_GF, IWL_CFG_ANY,
- IWL_CFG_160, IWL_CFG_ANY, IWL_CFG_NO_CDB,
+ IWL_CFG_160, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwlax211_2ax_cfg_so_gf_a0, iwl_ax211_name),
+ _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
+ IWL_CFG_MAC_TYPE_SOF, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_GF, IWL_CFG_ANY,
+ IWL_CFG_160, IWL_CFG_ANY, IWL_CFG_CDB, IWL_CFG_ANY,
+ iwlax411_2ax_cfg_so_gf4_a0, iwl_ax411_name),
/* Bz */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_BZ, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY,
- IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB,
+ IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_bz_a0_hr_b0, iwl_bz_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_BZ, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_GF, IWL_CFG_ANY,
- IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB,
+ IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_bz_a0_gf_a0, iwl_bz_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_BZ, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_GF, IWL_CFG_ANY,
- IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_CDB,
+ IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_CDB, IWL_CFG_ANY,
iwl_cfg_bz_a0_gf4_a0, iwl_bz_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_BZ, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_MR, IWL_CFG_ANY,
- IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB,
+ IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_bz_a0_mr_a0, iwl_bz_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_BZ, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_FM, IWL_CFG_ANY,
- IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB,
+ IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_bz_a0_fm_a0, iwl_bz_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_GL, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_FM, IWL_CFG_ANY,
- IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB,
+ IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_NO_JACKET,
iwl_cfg_gl_a0_fm_a0, iwl_bz_name),
+/* BZ Z step */
+ _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
+ IWL_CFG_MAC_TYPE_BZ, SILICON_Z_STEP,
+ IWL_CFG_RF_TYPE_GF, IWL_CFG_ANY,
+ IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
+ iwl_cfg_bz_z0_gf_a0, iwl_bz_name),
+
+/* BNJ */
+ _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
+ IWL_CFG_MAC_TYPE_GL, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_FM, IWL_CFG_ANY,
+ IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_IS_JACKET,
+ iwl_cfg_bnj_a0_fm_a0, iwl_bz_name),
+ _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
+ IWL_CFG_MAC_TYPE_GL, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_FM, IWL_CFG_ANY,
+ IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_CDB, IWL_CFG_IS_JACKET,
+ iwl_cfg_bnj_a0_fm4_a0, iwl_bz_name),
+ _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
+ IWL_CFG_MAC_TYPE_GL, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_GF, IWL_CFG_ANY,
+ IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_IS_JACKET,
+ iwl_cfg_bnj_a0_gf_a0, iwl_bz_name),
+ _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
+ IWL_CFG_MAC_TYPE_GL, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_GF, IWL_CFG_ANY,
+ IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_CDB, IWL_CFG_IS_JACKET,
+ iwl_cfg_bnj_a0_gf4_a0, iwl_bz_name),
+ _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
+ IWL_CFG_MAC_TYPE_GL, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_HR1, IWL_CFG_ANY,
+ IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_IS_JACKET,
+ iwl_cfg_bnj_a0_hr_b0, iwl_bz_name),
+
/* SoF with JF2 */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SOF, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
- IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwlax210_2ax_cfg_so_jf_b0, iwl9560_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SOF, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
- IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwlax210_2ax_cfg_so_jf_b0, iwl9560_name),
/* SoF with JF */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SOF, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1,
- IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwlax210_2ax_cfg_so_jf_b0, iwl9461_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SOF, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV,
- IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwlax210_2ax_cfg_so_jf_b0, iwl9462_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SOF, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1,
- IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwlax210_2ax_cfg_so_jf_b0, iwl9461_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SOF, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV,
- IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwlax210_2ax_cfg_so_jf_b0, iwl9462_name),
/* SoF with JF2 */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SOF, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
- IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwlax210_2ax_cfg_so_jf_b0, iwl9560_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SOF, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
- IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwlax210_2ax_cfg_so_jf_b0, iwl9560_name),
/* SoF with JF */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SOF, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1,
- IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwlax210_2ax_cfg_so_jf_b0, iwl9461_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SOF, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV,
- IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwlax210_2ax_cfg_so_jf_b0, iwl9462_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SOF, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1,
- IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwlax210_2ax_cfg_so_jf_b0, iwl9461_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SOF, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV,
- IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwlax210_2ax_cfg_so_jf_b0, iwl9462_name),
/* So with GF */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SO, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_GF, IWL_CFG_ANY,
- IWL_CFG_160, IWL_CFG_ANY, IWL_CFG_NO_CDB,
+ IWL_CFG_160, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwlax211_2ax_cfg_so_gf_a0, iwl_ax211_name),
+ _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
+ IWL_CFG_MAC_TYPE_SO, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_GF, IWL_CFG_ANY,
+ IWL_CFG_160, IWL_CFG_ANY, IWL_CFG_CDB, IWL_CFG_ANY,
+ iwlax411_2ax_cfg_so_gf4_a0, iwl_ax411_name),
/* So with JF2 */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SO, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
- IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwlax210_2ax_cfg_so_jf_b0, iwl9560_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SO, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
- IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwlax210_2ax_cfg_so_jf_b0, iwl9560_name),
/* So with JF */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SO, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1,
- IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwlax210_2ax_cfg_so_jf_b0, iwl9461_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SO, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV,
- IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwlax210_2ax_cfg_so_jf_b0, iwl9462_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SO, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1,
- IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwlax210_2ax_cfg_so_jf_b0, iwl9461_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SO, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV,
- IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB,
+ IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwlax210_2ax_cfg_so_jf_b0, iwl9462_name)
#endif /* CONFIG_IWLMVM */
@@ -1249,22 +1313,14 @@ static const struct iwl_dev_info iwl_dev_info_table[] = {
static int get_crf_id(struct iwl_trans *iwl_trans)
{
int ret = 0;
- u32 wfpm_ctrl_addr;
- u32 wfpm_otp_cfg_addr;
u32 sd_reg_ver_addr;
u32 cdb = 0;
u32 val;
- if (iwl_trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_AX210) {
- wfpm_ctrl_addr = WFPM_CTRL_REG_GEN2;
- wfpm_otp_cfg_addr = WFPM_OTP_CFG1_ADDR_GEN2;
+ if (iwl_trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_AX210)
sd_reg_ver_addr = SD_REG_VER_GEN2;
- /* Qu/Pu families have other addresses */
- } else {
- wfpm_ctrl_addr = WFPM_CTRL_REG;
- wfpm_otp_cfg_addr = WFPM_OTP_CFG1_ADDR;
+ else
sd_reg_ver_addr = SD_REG_VER;
- }
if (!iwl_trans_grab_nic_access(iwl_trans)) {
IWL_ERR(iwl_trans, "Failed to grab nic access before reading crf id\n");
@@ -1273,15 +1329,15 @@ static int get_crf_id(struct iwl_trans *iwl_trans)
}
/* Enable access to peripheral registers */
- val = iwl_read_umac_prph_no_grab(iwl_trans, wfpm_ctrl_addr);
+ val = iwl_read_umac_prph_no_grab(iwl_trans, WFPM_CTRL_REG);
val |= ENABLE_WFPM;
- iwl_write_umac_prph_no_grab(iwl_trans, wfpm_ctrl_addr, val);
+ iwl_write_umac_prph_no_grab(iwl_trans, WFPM_CTRL_REG, val);
/* Read crf info */
val = iwl_read_prph_no_grab(iwl_trans, sd_reg_ver_addr);
/* Read cdb info (also contains the jacket info if needed in the future */
- cdb = iwl_read_umac_prph_no_grab(iwl_trans, wfpm_otp_cfg_addr);
+ cdb = iwl_read_umac_prph_no_grab(iwl_trans, WFPM_OTP_CFG1_ADDR);
/* Map between crf id to rf id */
switch (REG_CRF_ID_TYPE(val)) {
@@ -1337,7 +1393,7 @@ static int get_crf_id(struct iwl_trans *iwl_trans)
static const struct iwl_dev_info *
iwl_pci_find_dev_info(u16 device, u16 subsystem_device,
u16 mac_type, u8 mac_step,
- u16 rf_type, u8 cdb, u8 rf_id, u8 no_160, u8 cores)
+ u16 rf_type, u8 cdb, u8 jacket, u8 rf_id, u8 no_160, u8 cores)
{
int num_devices = ARRAY_SIZE(iwl_dev_info_table);
int i;
@@ -1372,6 +1428,10 @@ iwl_pci_find_dev_info(u16 device, u16 subsystem_device,
dev_info->cdb != cdb)
continue;
+ if (dev_info->jacket != (u8)IWL_CFG_ANY &&
+ dev_info->jacket != jacket)
+ continue;
+
if (dev_info->rf_id != (u8)IWL_CFG_ANY &&
dev_info->rf_id != rf_id)
continue;
@@ -1426,15 +1486,18 @@ static int iwl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
* first trying to load the firmware etc. and potentially only
* detecting any problems when the first interface is brought up.
*/
- ret = iwl_finish_nic_init(iwl_trans);
- if (ret)
- goto out_free_trans;
- if (iwl_trans_grab_nic_access(iwl_trans)) {
- /* all good */
- iwl_trans_release_nic_access(iwl_trans);
- } else {
- ret = -EIO;
- goto out_free_trans;
+ ret = iwl_pcie_prepare_card_hw(iwl_trans);
+ if (!ret) {
+ ret = iwl_finish_nic_init(iwl_trans);
+ if (ret)
+ goto out_free_trans;
+ if (iwl_trans_grab_nic_access(iwl_trans)) {
+ /* all good */
+ iwl_trans_release_nic_access(iwl_trans);
+ } else {
+ ret = -EIO;
+ goto out_free_trans;
+ }
}
iwl_trans->hw_rf_id = iwl_read32(iwl_trans, CSR_HW_RF_ID);
@@ -1453,9 +1516,10 @@ static int iwl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
dev_info = iwl_pci_find_dev_info(pdev->device, pdev->subsystem_device,
CSR_HW_REV_TYPE(iwl_trans->hw_rev),
- CSR_HW_REV_STEP(iwl_trans->hw_rev),
+ iwl_trans->hw_rev_step,
CSR_HW_RFID_TYPE(iwl_trans->hw_rf_id),
CSR_HW_RFID_IS_CDB(iwl_trans->hw_rf_id),
+ CSR_HW_RFID_IS_JACKET(iwl_trans->hw_rf_id),
IWL_SUBDEVICE_RF_ID(pdev->subsystem_device),
IWL_SUBDEVICE_NO_160(pdev->subsystem_device),
IWL_SUBDEVICE_CORES(pdev->subsystem_device));
@@ -1494,21 +1558,6 @@ static int iwl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
(iwl_trans->hw_rev & CSR_HW_REV_TYPE_MSK) == CSR_HW_REV_TYPE_7265D)
iwl_trans->cfg = cfg_7265d;
- if (cfg == &iwlax210_2ax_cfg_so_hr_a0) {
- if (iwl_trans->hw_rev == CSR_HW_REV_TYPE_TY) {
- iwl_trans->cfg = &iwlax210_2ax_cfg_ty_gf_a0;
- } else if (CSR_HW_RFID_TYPE(iwl_trans->hw_rf_id) ==
- CSR_HW_RFID_TYPE(CSR_HW_RF_ID_TYPE_JF)) {
- iwl_trans->cfg = &iwlax210_2ax_cfg_so_jf_b0;
- } else if (CSR_HW_RFID_TYPE(iwl_trans->hw_rf_id) ==
- CSR_HW_RFID_TYPE(CSR_HW_RF_ID_TYPE_GF)) {
- iwl_trans->cfg = &iwlax211_2ax_cfg_so_gf_a0;
- } else if (CSR_HW_RFID_TYPE(iwl_trans->hw_rf_id) ==
- CSR_HW_RFID_TYPE(CSR_HW_RF_ID_TYPE_GF4)) {
- iwl_trans->cfg = &iwlax411_2ax_cfg_so_gf4_a0;
- }
- }
-
/*
* This is a hack to switch from Qu B0 to Qu C0. We need to
* do this for all cfgs that use Qu B0, except for those using
@@ -1569,6 +1618,10 @@ static int iwl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
goto out_free_trans;
pci_set_drvdata(pdev, iwl_trans);
+
+ /* try to get ownership so that we'll know if we don't own it */
+ iwl_pcie_prepare_card_hw(iwl_trans);
+
iwl_trans->drv = iwl_drv_start(iwl_trans);
if (IS_ERR(iwl_trans->drv)) {
diff --git a/drivers/net/wireless/intel/iwlwifi/pcie/rx.c b/drivers/net/wireless/intel/iwlwifi/pcie/rx.c
index 14602d6..8247014 100644
--- a/drivers/net/wireless/intel/iwlwifi/pcie/rx.c
+++ b/drivers/net/wireless/intel/iwlwifi/pcie/rx.c
@@ -2266,7 +2266,12 @@ irqreturn_t iwl_pcie_irq_msix_handler(int irq, void *dev_id)
}
}
- if (inta_hw & MSIX_HW_INT_CAUSES_REG_WAKEUP) {
+ /*
+ * In some rare cases when the HW is in a bad state, we may
+ * get this interrupt too early, when prph_info is still NULL.
+ * So make sure that it's not NULL to prevent crashing.
+ */
+ if (inta_hw & MSIX_HW_INT_CAUSES_REG_WAKEUP && trans_pcie->prph_info) {
u32 sleep_notif =
le32_to_cpu(trans_pcie->prph_info->sleep_notif);
if (sleep_notif == IWL_D3_SLEEP_STATUS_SUSPEND ||
diff --git a/drivers/net/wireless/intel/iwlwifi/pcie/trans-gen2.c b/drivers/net/wireless/intel/iwlwifi/pcie/trans-gen2.c
index 645cb4d..0febdca 100644
--- a/drivers/net/wireless/intel/iwlwifi/pcie/trans-gen2.c
+++ b/drivers/net/wireless/intel/iwlwifi/pcie/trans-gen2.c
@@ -81,7 +81,7 @@ static void iwl_pcie_gen2_apm_stop(struct iwl_trans *trans, bool op_mode_leave)
/* Stop device's DMA activity */
iwl_pcie_apm_stop_master(trans);
- iwl_trans_sw_reset(trans);
+ iwl_trans_sw_reset(trans, false);
/*
* Clear "initialization complete" bit to move adapter from
@@ -105,9 +105,12 @@ static void iwl_trans_pcie_fw_reset_handshake(struct iwl_trans *trans)
if (trans->trans_cfg->device_family < IWL_DEVICE_FAMILY_AX210)
iwl_write_umac_prph(trans, UREG_NIC_SET_NMI_DRIVER,
UREG_NIC_SET_NMI_DRIVER_RESET_HANDSHAKE);
- else
+ else if (trans->trans_cfg->device_family == IWL_DEVICE_FAMILY_AX210)
iwl_write_umac_prph(trans, UREG_DOORBELL_TO_ISR6,
UREG_DOORBELL_TO_ISR6_RESET_HANDSHAKE);
+ else
+ iwl_write32(trans, CSR_DOORBELL_VECTOR,
+ UREG_DOORBELL_TO_ISR6_RESET_HANDSHAKE);
/* wait 200ms */
ret = wait_event_timeout(trans_pcie->fw_reset_waitq,
@@ -166,7 +169,8 @@ void _iwl_trans_pcie_gen2_stop_device(struct iwl_trans *trans)
/* Stop the device, and put it in low power state */
iwl_pcie_gen2_apm_stop(trans, false);
- iwl_trans_sw_reset(trans);
+ /* re-take ownership to prevent other users from stealing the device */
+ iwl_trans_sw_reset(trans, true);
/*
* Upon stop, the IVAR table gets erased, so msi-x won't
@@ -196,9 +200,6 @@ void _iwl_trans_pcie_gen2_stop_device(struct iwl_trans *trans)
* interrupt
*/
iwl_enable_rfkill_int(trans);
-
- /* re-take ownership to prevent other users from stealing the device */
- iwl_pcie_prepare_card_hw(trans);
}
void iwl_trans_pcie_gen2_stop_device(struct iwl_trans *trans)
diff --git a/drivers/net/wireless/intel/iwlwifi/pcie/trans.c b/drivers/net/wireless/intel/iwlwifi/pcie/trans.c
index 1efb53f..a63386a 100644
--- a/drivers/net/wireless/intel/iwlwifi/pcie/trans.c
+++ b/drivers/net/wireless/intel/iwlwifi/pcie/trans.c
@@ -24,6 +24,7 @@
#include "fw/error-dump.h"
#include "fw/dbg.h"
#include "fw/api/tx.h"
+#include "mei/iwl-mei.h"
#include "internal.h"
#include "iwl-fh.h"
#include "iwl-context-info-gen3.h"
@@ -126,7 +127,8 @@ void iwl_trans_pcie_dump_regs(struct iwl_trans *trans)
kfree(buf);
}
-static void iwl_trans_pcie_sw_reset(struct iwl_trans *trans)
+static int iwl_trans_pcie_sw_reset(struct iwl_trans *trans,
+ bool retake_ownership)
{
/* Reset entire device - do controller reset (results in SHRD_HW_RST) */
if (trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_BZ)
@@ -136,6 +138,11 @@ static void iwl_trans_pcie_sw_reset(struct iwl_trans *trans)
iwl_set_bit(trans, CSR_RESET,
CSR_RESET_REG_FLAG_SW_RESET);
usleep_range(5000, 6000);
+
+ if (retake_ownership)
+ return iwl_pcie_prepare_card_hw(trans);
+
+ return 0;
}
static void iwl_pcie_free_fw_monitor(struct iwl_trans *trans)
@@ -381,9 +388,11 @@ static void iwl_pcie_apm_lp_xtal_enable(struct iwl_trans *trans)
__iwl_trans_pcie_set_bit(trans, CSR_GP_CNTRL,
CSR_GP_CNTRL_REG_FLAG_XTAL_ON);
- iwl_trans_pcie_sw_reset(trans);
+ ret = iwl_trans_pcie_sw_reset(trans, true);
- ret = iwl_finish_nic_init(trans);
+ if (!ret)
+ ret = iwl_finish_nic_init(trans);
+
if (WARN_ON(ret)) {
/* Release XTAL ON request */
__iwl_trans_pcie_clear_bit(trans, CSR_GP_CNTRL,
@@ -408,7 +417,10 @@ static void iwl_pcie_apm_lp_xtal_enable(struct iwl_trans *trans)
apmg_xtal_cfg_reg |
SHR_APMG_XTAL_CFG_XTAL_ON_REQ);
- iwl_trans_pcie_sw_reset(trans);
+ ret = iwl_trans_pcie_sw_reset(trans, true);
+ if (ret)
+ IWL_ERR(trans,
+ "iwl_pcie_apm_lp_xtal_enable: failed to retake NIC ownership\n");
/* Enable LP XTAL by indirect access through CSR */
apmg_gp1_reg = iwl_trans_pcie_read_shr(trans, SHR_APMG_GP1_REG);
@@ -514,7 +526,7 @@ static void iwl_pcie_apm_stop(struct iwl_trans *trans, bool op_mode_leave)
return;
}
- iwl_trans_pcie_sw_reset(trans);
+ iwl_trans_pcie_sw_reset(trans, false);
/*
* Clear "initialization complete" bit to move adapter from
@@ -594,8 +606,10 @@ int iwl_pcie_prepare_card_hw(struct iwl_trans *trans)
ret = iwl_pcie_set_hw_ready(trans);
/* If the card is ready, exit 0 */
- if (ret >= 0)
+ if (ret >= 0) {
+ trans->csme_own = false;
return 0;
+ }
iwl_set_bit(trans, CSR_DBG_LINK_PWR_MGMT_REG,
CSR_RESET_LINK_PWR_MGMT_DISABLED);
@@ -608,8 +622,22 @@ int iwl_pcie_prepare_card_hw(struct iwl_trans *trans)
do {
ret = iwl_pcie_set_hw_ready(trans);
- if (ret >= 0)
+ if (ret >= 0) {
+ trans->csme_own = false;
return 0;
+ }
+
+ if (iwl_mei_is_connected()) {
+ IWL_DEBUG_INFO(trans,
+ "Couldn't prepare the card but SAP is connected\n");
+ trans->csme_own = true;
+ if (trans->trans_cfg->device_family !=
+ IWL_DEVICE_FAMILY_9000)
+ IWL_ERR(trans,
+ "SAP not supported for this NIC family\n");
+
+ return -EBUSY;
+ }
usleep_range(200, 1000);
t += 200;
@@ -1244,7 +1272,8 @@ static void _iwl_trans_pcie_stop_device(struct iwl_trans *trans)
/* Stop the device, and put it in low power state */
iwl_pcie_apm_stop(trans, false);
- iwl_trans_pcie_sw_reset(trans);
+ /* re-take ownership to prevent other users from stealing the device */
+ iwl_trans_pcie_sw_reset(trans, true);
/*
* Upon stop, the IVAR table gets erased, so msi-x won't
@@ -1274,9 +1303,6 @@ static void _iwl_trans_pcie_stop_device(struct iwl_trans *trans)
* interrupt
*/
iwl_enable_rfkill_int(trans);
-
- /* re-take ownership to prevent other users from stealing the device */
- iwl_pcie_prepare_card_hw(trans);
}
void iwl_pcie_synchronize_irqs(struct iwl_trans *trans)
@@ -1482,33 +1508,54 @@ void iwl_pcie_d3_complete_suspend(struct iwl_trans *trans,
iwl_pcie_set_pwr(trans, true);
}
+static int iwl_pcie_d3_handshake(struct iwl_trans *trans, bool suspend)
+{
+ struct iwl_trans_pcie *trans_pcie = IWL_TRANS_GET_PCIE_TRANS(trans);
+ int ret;
+
+ if (trans->trans_cfg->device_family == IWL_DEVICE_FAMILY_AX210) {
+ iwl_write_umac_prph(trans, UREG_DOORBELL_TO_ISR6,
+ suspend ? UREG_DOORBELL_TO_ISR6_SUSPEND :
+ UREG_DOORBELL_TO_ISR6_RESUME);
+ } else if (trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_BZ) {
+ iwl_write32(trans, CSR_IPC_SLEEP_CONTROL,
+ suspend ? CSR_IPC_SLEEP_CONTROL_SUSPEND :
+ CSR_IPC_SLEEP_CONTROL_RESUME);
+ iwl_write_umac_prph(trans, UREG_DOORBELL_TO_ISR6,
+ UREG_DOORBELL_TO_ISR6_SLEEP_CTRL);
+ } else {
+ return 0;
+ }
+
+ ret = wait_event_timeout(trans_pcie->sx_waitq,
+ trans_pcie->sx_complete, 2 * HZ);
+
+ /* Invalidate it toward next suspend or resume */
+ trans_pcie->sx_complete = false;
+
+ if (!ret) {
+ IWL_ERR(trans, "Timeout %s D3\n",
+ suspend ? "entering" : "exiting");
+ return -ETIMEDOUT;
+ }
+
+ return 0;
+}
+
static int iwl_trans_pcie_d3_suspend(struct iwl_trans *trans, bool test,
bool reset)
{
int ret;
- struct iwl_trans_pcie *trans_pcie = IWL_TRANS_GET_PCIE_TRANS(trans);
if (!reset)
/* Enable persistence mode to avoid reset */
iwl_set_bit(trans, CSR_HW_IF_CONFIG_REG,
CSR_HW_IF_CONFIG_REG_PERSIST_MODE);
- if (trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_AX210) {
- iwl_write_umac_prph(trans, UREG_DOORBELL_TO_ISR6,
- UREG_DOORBELL_TO_ISR6_SUSPEND);
+ ret = iwl_pcie_d3_handshake(trans, true);
+ if (ret)
+ return ret;
- ret = wait_event_timeout(trans_pcie->sx_waitq,
- trans_pcie->sx_complete, 2 * HZ);
- /*
- * Invalidate it toward resume.
- */
- trans_pcie->sx_complete = false;
-
- if (!ret) {
- IWL_ERR(trans, "Timeout entering D3\n");
- return -ETIMEDOUT;
- }
- }
iwl_pcie_d3_complete_suspend(trans, test, reset);
return 0;
@@ -1525,6 +1572,7 @@ static int iwl_trans_pcie_d3_resume(struct iwl_trans *trans,
if (test) {
iwl_enable_interrupts(trans);
*status = IWL_D3_STATUS_ALIVE;
+ ret = 0;
goto out;
}
@@ -1573,25 +1621,10 @@ static int iwl_trans_pcie_d3_resume(struct iwl_trans *trans,
*status = IWL_D3_STATUS_ALIVE;
out:
- if (*status == IWL_D3_STATUS_ALIVE &&
- trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_AX210) {
- trans_pcie->sx_complete = false;
- iwl_write_umac_prph(trans, UREG_DOORBELL_TO_ISR6,
- UREG_DOORBELL_TO_ISR6_RESUME);
+ if (*status == IWL_D3_STATUS_ALIVE)
+ ret = iwl_pcie_d3_handshake(trans, false);
- ret = wait_event_timeout(trans_pcie->sx_waitq,
- trans_pcie->sx_complete, 2 * HZ);
- /*
- * Invalidate it toward next suspend.
- */
- trans_pcie->sx_complete = false;
-
- if (!ret) {
- IWL_ERR(trans, "Timeout exiting D3\n");
- return -ETIMEDOUT;
- }
- }
- return 0;
+ return ret;
}
static void
@@ -1778,9 +1811,7 @@ static int iwl_pcie_gen2_force_power_gating(struct iwl_trans *trans)
iwl_clear_bits_prph(trans, HPM_HIPM_GEN_CFG,
HPM_HIPM_GEN_CFG_CR_FORCE_ACTIVE);
- iwl_trans_pcie_sw_reset(trans);
-
- return 0;
+ return iwl_trans_pcie_sw_reset(trans, true);
}
static int _iwl_trans_pcie_start_hw(struct iwl_trans *trans)
@@ -1800,7 +1831,9 @@ static int _iwl_trans_pcie_start_hw(struct iwl_trans *trans)
if (err)
return err;
- iwl_trans_pcie_sw_reset(trans);
+ err = iwl_trans_pcie_sw_reset(trans, true);
+ if (err)
+ return err;
if (trans->trans_cfg->device_family == IWL_DEVICE_FAMILY_22000 &&
trans->trans_cfg->integrated) {
@@ -3599,8 +3632,9 @@ struct iwl_trans *iwl_trans_pcie_alloc(struct pci_dev *pdev,
* in the old format.
*/
if (cfg_trans->device_family >= IWL_DEVICE_FAMILY_8000)
- trans->hw_rev = (trans->hw_rev & 0xfff0) |
- (CSR_HW_REV_STEP(trans->hw_rev << 2) << 2);
+ trans->hw_rev_step = trans->hw_rev & 0xF;
+ else
+ trans->hw_rev_step = (trans->hw_rev & 0xC) >> 2;
IWL_DEBUG_INFO(trans, "HW REV: 0x%0x\n", trans->hw_rev);
diff --git a/drivers/net/wireless/intel/iwlwifi/queue/tx.c b/drivers/net/wireless/intel/iwlwifi/queue/tx.c
index 451b060..0730657 100644
--- a/drivers/net/wireless/intel/iwlwifi/queue/tx.c
+++ b/drivers/net/wireless/intel/iwlwifi/queue/tx.c
@@ -1072,6 +1072,7 @@ int iwl_txq_alloc(struct iwl_trans *trans, struct iwl_txq *txq, int slots_num,
return 0;
err_free_tfds:
dma_free_coherent(trans->dev, tfd_sz, txq->tfds, txq->dma_addr);
+ txq->tfds = NULL;
error:
if (txq->entries && cmd_queue)
for (i = 0; i < slots_num; i++)
@@ -1752,8 +1753,11 @@ static int iwl_trans_txq_send_hcmd_sync(struct iwl_trans *trans,
}
if (test_bit(STATUS_FW_ERROR, &trans->status)) {
- IWL_ERR(trans, "FW error in SYNC CMD %s\n", cmd_str);
- dump_stack();
+ if (!test_and_clear_bit(STATUS_SUPPRESS_CMD_ERROR_ONCE,
+ &trans->status)) {
+ IWL_ERR(trans, "FW error in SYNC CMD %s\n", cmd_str);
+ dump_stack();
+ }
ret = -EIO;
goto cancel;
}
diff --git a/drivers/net/wireless/intersil/hostap/hostap_hw.c b/drivers/net/wireless/intersil/hostap/hostap_hw.c
index e459e71..b74f4cb 100644
--- a/drivers/net/wireless/intersil/hostap/hostap_hw.c
+++ b/drivers/net/wireless/intersil/hostap/hostap_hw.c
@@ -1815,8 +1815,9 @@ static int prism2_tx_80211(struct sk_buff *skb, struct net_device *dev)
memset(&txdesc, 0, sizeof(txdesc));
/* skb->data starts with txdesc->frame_control */
- hdr_len = 24;
- skb_copy_from_linear_data(skb, &txdesc.frame_control, hdr_len);
+ hdr_len = sizeof(txdesc.header);
+ BUILD_BUG_ON(hdr_len != 24);
+ skb_copy_from_linear_data(skb, &txdesc.header, hdr_len);
if (ieee80211_is_data(txdesc.frame_control) &&
ieee80211_has_a4(txdesc.frame_control) &&
skb->len >= 30) {
diff --git a/drivers/net/wireless/intersil/hostap/hostap_wlan.h b/drivers/net/wireless/intersil/hostap/hostap_wlan.h
index dd2603d..c25cd21 100644
--- a/drivers/net/wireless/intersil/hostap/hostap_wlan.h
+++ b/drivers/net/wireless/intersil/hostap/hostap_wlan.h
@@ -115,12 +115,14 @@ struct hfa384x_tx_frame {
__le16 tx_control; /* HFA384X_TX_CTRL_ flags */
/* 802.11 */
- __le16 frame_control; /* parts not used */
- __le16 duration_id;
- u8 addr1[ETH_ALEN];
- u8 addr2[ETH_ALEN]; /* filled by firmware */
- u8 addr3[ETH_ALEN];
- __le16 seq_ctrl; /* filled by firmware */
+ struct_group(header,
+ __le16 frame_control; /* parts not used */
+ __le16 duration_id;
+ u8 addr1[ETH_ALEN];
+ u8 addr2[ETH_ALEN]; /* filled by firmware */
+ u8 addr3[ETH_ALEN];
+ __le16 seq_ctrl; /* filled by firmware */
+ );
u8 addr4[ETH_ALEN];
__le16 data_len;
diff --git a/drivers/net/wireless/intersil/p54/txrx.c b/drivers/net/wireless/intersil/p54/txrx.c
index 873fea5..8414aa2 100644
--- a/drivers/net/wireless/intersil/p54/txrx.c
+++ b/drivers/net/wireless/intersil/p54/txrx.c
@@ -431,11 +431,7 @@ static void p54_rx_frame_sent(struct p54_common *priv, struct sk_buff *skb)
* Clear manually, ieee80211_tx_info_clear_status would
* clear the counts too and we need them.
*/
- memset(&info->status.ack_signal, 0,
- sizeof(struct ieee80211_tx_info) -
- offsetof(struct ieee80211_tx_info, status.ack_signal));
- BUILD_BUG_ON(offsetof(struct ieee80211_tx_info,
- status.ack_signal) != 20);
+ memset_after(&info->status, 0, rates);
if (entry_hdr->flags & cpu_to_le16(P54_HDR_FLAG_DATA_ALIGN))
pad = entry_data->align[0];
diff --git a/drivers/net/wireless/mac80211_hwsim.c b/drivers/net/wireless/mac80211_hwsim.c
index 23219f3..0307a66 100644
--- a/drivers/net/wireless/mac80211_hwsim.c
+++ b/drivers/net/wireless/mac80211_hwsim.c
@@ -1276,7 +1276,7 @@ static void mac80211_hwsim_tx_frame_nl(struct ieee80211_hw *hw,
hdr->frame_control |= cpu_to_le16(IEEE80211_FCTL_PM);
/* If the queue contains MAX_QUEUE skb's drop some */
if (skb_queue_len(&data->pending) >= MAX_QUEUE) {
- /* Droping until WARN_QUEUE level */
+ /* Dropping until WARN_QUEUE level */
while (skb_queue_len(&data->pending) >= WARN_QUEUE) {
ieee80211_free_txskb(hw, skb_dequeue(&data->pending));
data->tx_dropped++;
diff --git a/drivers/net/wireless/marvell/libertas/host.h b/drivers/net/wireless/marvell/libertas/host.h
index dfa2246..ceff4b9 100644
--- a/drivers/net/wireless/marvell/libertas/host.h
+++ b/drivers/net/wireless/marvell/libertas/host.h
@@ -308,10 +308,12 @@ struct txpd {
__le32 tx_packet_location;
/* Tx packet length */
__le16 tx_packet_length;
- /* First 2 byte of destination MAC address */
- u8 tx_dest_addr_high[2];
- /* Last 4 byte of destination MAC address */
- u8 tx_dest_addr_low[4];
+ struct_group_attr(tx_dest_addr, __packed,
+ /* First 2 byte of destination MAC address */
+ u8 tx_dest_addr_high[2];
+ /* Last 4 byte of destination MAC address */
+ u8 tx_dest_addr_low[4];
+ );
/* Pkt Priority */
u8 priority;
/* Pkt Trasnit Power control */
diff --git a/drivers/net/wireless/marvell/libertas/tx.c b/drivers/net/wireless/marvell/libertas/tx.c
index aeb4817..27304a9 100644
--- a/drivers/net/wireless/marvell/libertas/tx.c
+++ b/drivers/net/wireless/marvell/libertas/tx.c
@@ -113,6 +113,7 @@ netdev_tx_t lbs_hard_start_xmit(struct sk_buff *skb, struct net_device *dev)
p802x_hdr = skb->data;
pkt_len = skb->len;
+ BUILD_BUG_ON(sizeof(txpd->tx_dest_addr) != ETH_ALEN);
if (priv->wdev->iftype == NL80211_IFTYPE_MONITOR) {
struct tx_radiotap_hdr *rtap_hdr = (void *)skb->data;
@@ -124,10 +125,10 @@ netdev_tx_t lbs_hard_start_xmit(struct sk_buff *skb, struct net_device *dev)
pkt_len -= sizeof(*rtap_hdr);
/* copy destination address from 802.11 header */
- memcpy(txpd->tx_dest_addr_high, p802x_hdr + 4, ETH_ALEN);
+ memcpy(&txpd->tx_dest_addr, p802x_hdr + 4, ETH_ALEN);
} else {
/* copy destination address from 802.3 header */
- memcpy(txpd->tx_dest_addr_high, p802x_hdr, ETH_ALEN);
+ memcpy(&txpd->tx_dest_addr, p802x_hdr, ETH_ALEN);
}
txpd->tx_packet_length = cpu_to_le16(pkt_len);
diff --git a/drivers/net/wireless/marvell/libertas_tf/libertas_tf.h b/drivers/net/wireless/marvell/libertas_tf/libertas_tf.h
index 5d72654..631b5da 100644
--- a/drivers/net/wireless/marvell/libertas_tf/libertas_tf.h
+++ b/drivers/net/wireless/marvell/libertas_tf/libertas_tf.h
@@ -268,10 +268,12 @@ struct txpd {
__le32 tx_packet_location;
/* Tx packet length */
__le16 tx_packet_length;
- /* First 2 byte of destination MAC address */
- u8 tx_dest_addr_high[2];
- /* Last 4 byte of destination MAC address */
- u8 tx_dest_addr_low[4];
+ struct_group_attr(tx_dest_addr, __packed,
+ /* First 2 byte of destination MAC address */
+ u8 tx_dest_addr_high[2];
+ /* Last 4 byte of destination MAC address */
+ u8 tx_dest_addr_low[4];
+ );
/* Pkt Priority */
u8 priority;
/* Pkt Trasnit Power control */
@@ -280,7 +282,7 @@ struct txpd {
u8 pktdelay_2ms;
/* reserved */
u8 reserved1;
-};
+} __packed;
/* RxPD Descriptor */
struct rxpd {
@@ -311,7 +313,7 @@ struct rxpd {
/* Pkt Priority */
u8 priority;
u8 reserved[3];
-};
+} __packed;
struct cmd_header {
__le16 command;
@@ -377,14 +379,14 @@ struct cmd_ds_mac_control {
struct cmd_header hdr;
__le16 action;
u16 reserved;
-};
+} __packed;
struct cmd_ds_802_11_mac_address {
struct cmd_header hdr;
__le16 action;
uint8_t macadd[ETH_ALEN];
-};
+} __packed;
struct cmd_ds_mac_multicast_addr {
struct cmd_header hdr;
@@ -392,27 +394,27 @@ struct cmd_ds_mac_multicast_addr {
__le16 action;
__le16 nr_of_adrs;
u8 maclist[ETH_ALEN * MRVDRV_MAX_MULTICAST_LIST_SIZE];
-};
+} __packed;
struct cmd_ds_set_mode {
struct cmd_header hdr;
__le16 mode;
-};
+} __packed;
struct cmd_ds_set_bssid {
struct cmd_header hdr;
u8 bssid[6];
u8 activate;
-};
+} __packed;
struct cmd_ds_802_11_radio_control {
struct cmd_header hdr;
__le16 action;
__le16 control;
-};
+} __packed;
struct cmd_ds_802_11_rf_channel {
@@ -423,20 +425,20 @@ struct cmd_ds_802_11_rf_channel {
__le16 rftype; /* unused */
__le16 reserved; /* unused */
u8 channellist[32]; /* unused */
-};
+} __packed;
struct cmd_ds_set_boot2_ver {
struct cmd_header hdr;
__le16 action;
__le16 version;
-};
+} __packed;
struct cmd_ds_802_11_reset {
struct cmd_header hdr;
__le16 action;
-};
+} __packed;
struct cmd_ds_802_11_beacon_control {
struct cmd_header hdr;
@@ -444,14 +446,14 @@ struct cmd_ds_802_11_beacon_control {
__le16 action;
__le16 beacon_enable;
__le16 beacon_period;
-};
+} __packed;
struct cmd_ds_802_11_beacon_set {
struct cmd_header hdr;
__le16 len;
u8 beacon[MRVL_MAX_BCN_SIZE];
-};
+} __packed;
struct cmd_ctrl_node;
diff --git a/drivers/net/wireless/marvell/libertas_tf/main.c b/drivers/net/wireless/marvell/libertas_tf/main.c
index 7149221..02a1e1f 100644
--- a/drivers/net/wireless/marvell/libertas_tf/main.c
+++ b/drivers/net/wireless/marvell/libertas_tf/main.c
@@ -232,7 +232,8 @@ static void lbtf_tx_work(struct work_struct *work)
ieee80211_get_tx_rate(priv->hw, info)->hw_value);
/* copy destination address from 802.11 header */
- memcpy(txpd->tx_dest_addr_high, skb->data + sizeof(struct txpd) + 4,
+ BUILD_BUG_ON(sizeof(txpd->tx_dest_addr) != ETH_ALEN);
+ memcpy(&txpd->tx_dest_addr, skb->data + sizeof(struct txpd) + 4,
ETH_ALEN);
txpd->tx_packet_length = cpu_to_le16(len);
txpd->tx_packet_location = cpu_to_le32(sizeof(struct txpd));
diff --git a/drivers/net/wireless/marvell/mwifiex/fw.h b/drivers/net/wireless/marvell/mwifiex/fw.h
index 2ff23ab..63c25c6 100644
--- a/drivers/net/wireless/marvell/mwifiex/fw.h
+++ b/drivers/net/wireless/marvell/mwifiex/fw.h
@@ -2071,9 +2071,11 @@ struct mwifiex_ie_types_robust_coex {
__le32 mode;
} __packed;
+#define MWIFIEX_VERSION_STR_LENGTH 128
+
struct host_cmd_ds_version_ext {
u8 version_str_sel;
- char version_str[128];
+ char version_str[MWIFIEX_VERSION_STR_LENGTH];
} __packed;
struct host_cmd_ds_mgmt_frame_reg {
diff --git a/drivers/net/wireless/marvell/mwifiex/init.c b/drivers/net/wireless/marvell/mwifiex/init.c
index f006a3d..88c72d1 100644
--- a/drivers/net/wireless/marvell/mwifiex/init.c
+++ b/drivers/net/wireless/marvell/mwifiex/init.c
@@ -332,7 +332,7 @@ void mwifiex_set_trans_start(struct net_device *dev)
int i;
for (i = 0; i < dev->num_tx_queues; i++)
- netdev_get_tx_queue(dev, i)->trans_start = jiffies;
+ txq_trans_cond_update(netdev_get_tx_queue(dev, i));
netif_trans_update(dev);
}
diff --git a/drivers/net/wireless/marvell/mwifiex/main.c b/drivers/net/wireless/marvell/mwifiex/main.c
index 19b996c..ace7371 100644
--- a/drivers/net/wireless/marvell/mwifiex/main.c
+++ b/drivers/net/wireless/marvell/mwifiex/main.c
@@ -226,6 +226,23 @@ static int mwifiex_process_rx(struct mwifiex_adapter *adapter)
return 0;
}
+static void maybe_quirk_fw_disable_ds(struct mwifiex_adapter *adapter)
+{
+ struct mwifiex_private *priv = mwifiex_get_priv(adapter, MWIFIEX_BSS_ROLE_STA);
+ struct mwifiex_ver_ext ver_ext;
+
+ if (test_and_set_bit(MWIFIEX_IS_REQUESTING_FW_VEREXT, &adapter->work_flags))
+ return;
+
+ memset(&ver_ext, 0, sizeof(ver_ext));
+ ver_ext.version_str_sel = 1;
+ if (mwifiex_send_cmd(priv, HostCmd_CMD_VERSION_EXT,
+ HostCmd_ACT_GEN_GET, 0, &ver_ext, false)) {
+ mwifiex_dbg(priv->adapter, MSG,
+ "Checking hardware revision failed.\n");
+ }
+}
+
/*
* The main process.
*
@@ -356,6 +373,7 @@ int mwifiex_main_process(struct mwifiex_adapter *adapter)
if (adapter->hw_status == MWIFIEX_HW_STATUS_INIT_DONE) {
adapter->hw_status = MWIFIEX_HW_STATUS_READY;
mwifiex_init_fw_complete(adapter);
+ maybe_quirk_fw_disable_ds(adapter);
}
}
diff --git a/drivers/net/wireless/marvell/mwifiex/main.h b/drivers/net/wireless/marvell/mwifiex/main.h
index 90012cb..332dd1c 100644
--- a/drivers/net/wireless/marvell/mwifiex/main.h
+++ b/drivers/net/wireless/marvell/mwifiex/main.h
@@ -524,6 +524,7 @@ enum mwifiex_adapter_work_flags {
MWIFIEX_IS_SUSPENDED,
MWIFIEX_IS_HS_CONFIGURED,
MWIFIEX_IS_HS_ENABLING,
+ MWIFIEX_IS_REQUESTING_FW_VEREXT,
};
struct mwifiex_band_config {
@@ -646,7 +647,7 @@ struct mwifiex_private {
struct wireless_dev wdev;
struct mwifiex_chan_freq_power cfp;
u32 versionstrsel;
- char version_str[128];
+ char version_str[MWIFIEX_VERSION_STR_LENGTH];
#ifdef CONFIG_DEBUG_FS
struct dentry *dfs_dev_dir;
#endif
@@ -1055,6 +1056,8 @@ struct mwifiex_adapter {
void *devdump_data;
int devdump_len;
struct timer_list devdump_timer;
+
+ bool ignore_btcoex_events;
};
void mwifiex_process_tx_queue(struct mwifiex_adapter *adapter);
diff --git a/drivers/net/wireless/marvell/mwifiex/pcie.c b/drivers/net/wireless/marvell/mwifiex/pcie.c
index c3f5583..d5fb294 100644
--- a/drivers/net/wireless/marvell/mwifiex/pcie.c
+++ b/drivers/net/wireless/marvell/mwifiex/pcie.c
@@ -3152,6 +3152,9 @@ static int mwifiex_init_pcie(struct mwifiex_adapter *adapter)
if (ret)
goto err_alloc_buffers;
+ if (pdev->device == PCIE_DEVICE_ID_MARVELL_88W8897)
+ adapter->ignore_btcoex_events = true;
+
return 0;
err_alloc_buffers:
diff --git a/drivers/net/wireless/marvell/mwifiex/sta_cmdresp.c b/drivers/net/wireless/marvell/mwifiex/sta_cmdresp.c
index 6b5d35d..1a4ae8a 100644
--- a/drivers/net/wireless/marvell/mwifiex/sta_cmdresp.c
+++ b/drivers/net/wireless/marvell/mwifiex/sta_cmdresp.c
@@ -708,11 +708,35 @@ static int mwifiex_ret_ver_ext(struct mwifiex_private *priv,
{
struct host_cmd_ds_version_ext *ver_ext = &resp->params.verext;
+ if (test_and_clear_bit(MWIFIEX_IS_REQUESTING_FW_VEREXT, &priv->adapter->work_flags)) {
+ if (strncmp(ver_ext->version_str, "ChipRev:20, BB:9b(10.00), RF:40(21)",
+ MWIFIEX_VERSION_STR_LENGTH) == 0) {
+ struct mwifiex_ds_auto_ds auto_ds = {
+ .auto_ds = DEEP_SLEEP_OFF,
+ };
+
+ mwifiex_dbg(priv->adapter, MSG,
+ "Bad HW revision detected, disabling deep sleep\n");
+
+ if (mwifiex_send_cmd(priv, HostCmd_CMD_802_11_PS_MODE_ENH,
+ DIS_AUTO_PS, BITMAP_AUTO_DS, &auto_ds, false)) {
+ mwifiex_dbg(priv->adapter, MSG,
+ "Disabling deep sleep failed.\n");
+ }
+ }
+
+ return 0;
+ }
+
if (version_ext) {
version_ext->version_str_sel = ver_ext->version_str_sel;
memcpy(version_ext->version_str, ver_ext->version_str,
- sizeof(char) * 128);
- memcpy(priv->version_str, ver_ext->version_str, 128);
+ MWIFIEX_VERSION_STR_LENGTH);
+ memcpy(priv->version_str, ver_ext->version_str,
+ MWIFIEX_VERSION_STR_LENGTH);
+
+ /* Ensure the version string from the firmware is 0-terminated */
+ priv->version_str[MWIFIEX_VERSION_STR_LENGTH - 1] = '\0';
}
return 0;
}
diff --git a/drivers/net/wireless/marvell/mwifiex/sta_event.c b/drivers/net/wireless/marvell/mwifiex/sta_event.c
index 68c6326..7d42c5d 100644
--- a/drivers/net/wireless/marvell/mwifiex/sta_event.c
+++ b/drivers/net/wireless/marvell/mwifiex/sta_event.c
@@ -365,10 +365,12 @@ static void mwifiex_process_uap_tx_pause(struct mwifiex_private *priv,
sta_ptr = mwifiex_get_sta_entry(priv, tp->peermac);
if (sta_ptr && sta_ptr->tx_pause != tp->tx_pause) {
sta_ptr->tx_pause = tp->tx_pause;
+ spin_unlock_bh(&priv->sta_list_spinlock);
mwifiex_update_ralist_tx_pause(priv, tp->peermac,
tp->tx_pause);
+ } else {
+ spin_unlock_bh(&priv->sta_list_spinlock);
}
- spin_unlock_bh(&priv->sta_list_spinlock);
}
}
@@ -400,11 +402,13 @@ static void mwifiex_process_sta_tx_pause(struct mwifiex_private *priv,
sta_ptr = mwifiex_get_sta_entry(priv, tp->peermac);
if (sta_ptr && sta_ptr->tx_pause != tp->tx_pause) {
sta_ptr->tx_pause = tp->tx_pause;
+ spin_unlock_bh(&priv->sta_list_spinlock);
mwifiex_update_ralist_tx_pause(priv,
tp->peermac,
tp->tx_pause);
+ } else {
+ spin_unlock_bh(&priv->sta_list_spinlock);
}
- spin_unlock_bh(&priv->sta_list_spinlock);
}
}
}
@@ -1058,6 +1062,9 @@ int mwifiex_process_sta_event(struct mwifiex_private *priv)
break;
case EVENT_BT_COEX_WLAN_PARA_CHANGE:
dev_dbg(adapter->dev, "EVENT: BT coex wlan param update\n");
+ if (adapter->ignore_btcoex_events)
+ break;
+
mwifiex_bt_coex_wlan_param_update_event(priv,
adapter->event_skb);
break;
diff --git a/drivers/net/wireless/marvell/mwifiex/usb.c b/drivers/net/wireless/marvell/mwifiex/usb.c
index 9736aa0..8f01fcb 100644
--- a/drivers/net/wireless/marvell/mwifiex/usb.c
+++ b/drivers/net/wireless/marvell/mwifiex/usb.c
@@ -130,7 +130,8 @@ static int mwifiex_usb_recv(struct mwifiex_adapter *adapter,
default:
mwifiex_dbg(adapter, ERROR,
"unknown recv_type %#x\n", recv_type);
- return -1;
+ ret = -1;
+ goto exit_restore_skb;
}
break;
case MWIFIEX_USB_EP_DATA:
diff --git a/drivers/net/wireless/marvell/mwl8k.c b/drivers/net/wireless/marvell/mwl8k.c
index 529e325..864a2ba 100644
--- a/drivers/net/wireless/marvell/mwl8k.c
+++ b/drivers/net/wireless/marvell/mwl8k.c
@@ -4225,9 +4225,11 @@ struct mwl8k_cmd_set_key {
__le32 key_info;
__le32 key_id;
__le16 key_len;
- __u8 key_material[MAX_ENCR_KEY_LENGTH];
- __u8 tkip_tx_mic_key[MIC_KEY_LENGTH];
- __u8 tkip_rx_mic_key[MIC_KEY_LENGTH];
+ struct {
+ __u8 key_material[MAX_ENCR_KEY_LENGTH];
+ __u8 tkip_tx_mic_key[MIC_KEY_LENGTH];
+ __u8 tkip_rx_mic_key[MIC_KEY_LENGTH];
+ } tkip;
__le16 tkip_rsc_low;
__le32 tkip_rsc_high;
__le16 tkip_tsc_low;
@@ -4375,7 +4377,7 @@ static int mwl8k_cmd_encryption_set_key(struct ieee80211_hw *hw,
goto done;
}
- memcpy(cmd->key_material, key->key, keymlen);
+ memcpy(&cmd->tkip, key->key, keymlen);
cmd->action = cpu_to_le32(action);
rc = mwl8k_post_pervif_cmd(hw, vif, &cmd->header);
diff --git a/drivers/net/wireless/mediatek/mt76/debugfs.c b/drivers/net/wireless/mediatek/mt76/debugfs.c
index b8bcf22..47e9911 100644
--- a/drivers/net/wireless/mediatek/mt76/debugfs.c
+++ b/drivers/net/wireless/mediatek/mt76/debugfs.c
@@ -82,7 +82,7 @@ static int mt76_rx_queues_read(struct seq_file *s, void *data)
queued = mt76_is_usb(dev) ? q->ndesc - q->queued : q->queued;
seq_printf(s, " %9d | %9d | %9d | %9d |\n",
- i, q->queued, q->head, q->tail);
+ i, queued, q->head, q->tail);
}
return 0;
diff --git a/drivers/net/wireless/mediatek/mt76/dma.c b/drivers/net/wireless/mediatek/mt76/dma.c
index 5e1c150..3a9af89 100644
--- a/drivers/net/wireless/mediatek/mt76/dma.c
+++ b/drivers/net/wireless/mediatek/mt76/dma.c
@@ -572,9 +572,7 @@ mt76_dma_rx_process(struct mt76_dev *dev, struct mt76_queue *q, int budget)
if (data_len < len + q->buf_offset) {
dev_kfree_skb(q->rx_head);
q->rx_head = NULL;
-
- skb_free_frag(data);
- continue;
+ goto free_frag;
}
if (q->rx_head) {
@@ -582,11 +580,14 @@ mt76_dma_rx_process(struct mt76_dev *dev, struct mt76_queue *q, int budget)
continue;
}
+ if (!more && dev->drv->rx_check &&
+ !(dev->drv->rx_check(dev, data, len)))
+ goto free_frag;
+
skb = build_skb(data, q->buf_size);
- if (!skb) {
- skb_free_frag(data);
- continue;
- }
+ if (!skb)
+ goto free_frag;
+
skb_reserve(skb, q->buf_offset);
if (q == &dev->q_rx[MT_RXQ_MCU]) {
@@ -603,6 +604,10 @@ mt76_dma_rx_process(struct mt76_dev *dev, struct mt76_queue *q, int budget)
}
dev->drv->rx_skb(dev, q - dev->q_rx, skb);
+ continue;
+
+free_frag:
+ skb_free_frag(data);
}
mt76_dma_rx_fill(dev, q);
diff --git a/drivers/net/wireless/mediatek/mt76/eeprom.c b/drivers/net/wireless/mediatek/mt76/eeprom.c
index 2d58aa3..a499861 100644
--- a/drivers/net/wireless/mediatek/mt76/eeprom.c
+++ b/drivers/net/wireless/mediatek/mt76/eeprom.c
@@ -65,6 +65,8 @@ int mt76_get_of_eeprom(struct mt76_dev *dev, void *eep, int offset, int len)
offset = be32_to_cpup(list);
ret = mtd_read(mtd, offset, len, &retlen, eep);
put_mtd_device(mtd);
+ if (mtd_is_bitflip(ret))
+ ret = 0;
if (ret) {
dev_err(dev->dev, "reading EEPROM from mtd %s failed: %i\n",
part, ret);
diff --git a/drivers/net/wireless/mediatek/mt76/mac80211.c b/drivers/net/wireless/mediatek/mt76/mac80211.c
index 62807dc..8bb1c7a 100644
--- a/drivers/net/wireless/mediatek/mt76/mac80211.c
+++ b/drivers/net/wireless/mediatek/mt76/mac80211.c
@@ -185,7 +185,6 @@ const struct cfg80211_sar_capa mt76_sar_capa = {
.num_freq_ranges = ARRAY_SIZE(mt76_sar_freq_ranges),
.freq_ranges = &mt76_sar_freq_ranges[0],
};
-EXPORT_SYMBOL_GPL(mt76_sar_capa);
static int mt76_led_init(struct mt76_dev *dev)
{
@@ -393,7 +392,7 @@ mt76_check_sband(struct mt76_phy *phy, struct mt76_sband *msband,
phy->hw->wiphy->bands[band] = NULL;
}
-static void
+static int
mt76_phy_init(struct mt76_phy *phy, struct ieee80211_hw *hw)
{
struct mt76_dev *dev = phy->dev;
@@ -411,8 +410,15 @@ mt76_phy_init(struct mt76_phy *phy, struct ieee80211_hw *hw)
wiphy_ext_feature_set(wiphy, NL80211_EXT_FEATURE_AIRTIME_FAIRNESS);
wiphy_ext_feature_set(wiphy, NL80211_EXT_FEATURE_AQL);
- wiphy->available_antennas_tx = dev->phy.antenna_mask;
- wiphy->available_antennas_rx = dev->phy.antenna_mask;
+ wiphy->available_antennas_tx = phy->antenna_mask;
+ wiphy->available_antennas_rx = phy->antenna_mask;
+
+ wiphy->sar_capa = &mt76_sar_capa;
+ phy->frp = devm_kcalloc(dev->dev, wiphy->sar_capa->num_freq_ranges,
+ sizeof(struct mt76_freq_range_power),
+ GFP_KERNEL);
+ if (!phy->frp)
+ return -ENOMEM;
hw->txq_data_size = sizeof(struct mt76_txq);
hw->uapsd_max_sp_len = IEEE80211_WMM_IE_STA_QOSINFO_SP_ALL;
@@ -432,6 +438,8 @@ mt76_phy_init(struct mt76_phy *phy, struct ieee80211_hw *hw)
ieee80211_hw_set(hw, MFP_CAPABLE);
ieee80211_hw_set(hw, AP_LINK_PS);
ieee80211_hw_set(hw, REPORTS_TX_ACK_STATUS);
+
+ return 0;
}
struct mt76_phy *
@@ -472,7 +480,9 @@ int mt76_register_phy(struct mt76_phy *phy, bool vht,
{
int ret;
- mt76_phy_init(phy, phy->hw);
+ ret = mt76_phy_init(phy, phy->hw);
+ if (ret)
+ return ret;
if (phy->cap.has_2ghz) {
ret = mt76_init_sband_2g(phy, rates, n_rates);
@@ -591,7 +601,9 @@ int mt76_register_device(struct mt76_dev *dev, bool vht,
int ret;
dev_set_drvdata(dev->dev, dev);
- mt76_phy_init(phy, hw);
+ ret = mt76_phy_init(phy, hw);
+ if (ret)
+ return ret;
if (phy->cap.has_2ghz) {
ret = mt76_init_sband_2g(phy, rates, n_rates);
@@ -1163,10 +1175,12 @@ mt76_check_sta(struct mt76_dev *dev, struct sk_buff *skb)
if (ps)
set_bit(MT_WCID_FLAG_PS, &wcid->flags);
- else
- clear_bit(MT_WCID_FLAG_PS, &wcid->flags);
dev->drv->sta_ps(dev, sta, ps);
+
+ if (!ps)
+ clear_bit(MT_WCID_FLAG_PS, &wcid->flags);
+
ieee80211_sta_ps_transition(sta, ps);
}
@@ -1348,6 +1362,59 @@ int mt76_get_txpower(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
}
EXPORT_SYMBOL_GPL(mt76_get_txpower);
+int mt76_init_sar_power(struct ieee80211_hw *hw,
+ const struct cfg80211_sar_specs *sar)
+{
+ struct mt76_phy *phy = hw->priv;
+ const struct cfg80211_sar_capa *capa = hw->wiphy->sar_capa;
+ int i;
+
+ if (sar->type != NL80211_SAR_TYPE_POWER || !sar->num_sub_specs)
+ return -EINVAL;
+
+ for (i = 0; i < sar->num_sub_specs; i++) {
+ u32 index = sar->sub_specs[i].freq_range_index;
+ /* SAR specifies power limitaton in 0.25dbm */
+ s32 power = sar->sub_specs[i].power >> 1;
+
+ if (power > 127 || power < -127)
+ power = 127;
+
+ phy->frp[index].range = &capa->freq_ranges[index];
+ phy->frp[index].power = power;
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(mt76_init_sar_power);
+
+int mt76_get_sar_power(struct mt76_phy *phy,
+ struct ieee80211_channel *chan,
+ int power)
+{
+ const struct cfg80211_sar_capa *capa = phy->hw->wiphy->sar_capa;
+ int freq, i;
+
+ if (!capa || !phy->frp)
+ return power;
+
+ if (power > 127 || power < -127)
+ power = 127;
+
+ freq = ieee80211_channel_to_frequency(chan->hw_value, chan->band);
+ for (i = 0 ; i < capa->num_freq_ranges; i++) {
+ if (phy->frp[i].range &&
+ freq >= phy->frp[i].range->start_freq &&
+ freq < phy->frp[i].range->end_freq) {
+ power = min_t(int, phy->frp[i].power, power);
+ break;
+ }
+ }
+
+ return power;
+}
+EXPORT_SYMBOL_GPL(mt76_get_sar_power);
+
static void
__mt76_csa_finish(void *priv, u8 *mac, struct ieee80211_vif *vif)
{
@@ -1494,7 +1561,6 @@ EXPORT_SYMBOL_GPL(mt76_init_queue);
u16 mt76_calculate_default_rate(struct mt76_phy *phy, int rateidx)
{
int offset = 0;
- struct ieee80211_rate *rate;
if (phy->chandef.chan->band != NL80211_BAND_2GHZ)
offset = 4;
@@ -1503,9 +1569,11 @@ u16 mt76_calculate_default_rate(struct mt76_phy *phy, int rateidx)
if (rateidx < 0)
rateidx = 0;
- rate = &mt76_rates[offset + rateidx];
+ rateidx += offset;
+ if (rateidx >= ARRAY_SIZE(mt76_rates))
+ rateidx = offset;
- return rate->hw_value;
+ return mt76_rates[rateidx].hw_value;
}
EXPORT_SYMBOL_GPL(mt76_calculate_default_rate);
diff --git a/drivers/net/wireless/mediatek/mt76/mt76.h b/drivers/net/wireless/mediatek/mt76/mt76.h
index e2da720..404c3d1 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76.h
+++ b/drivers/net/wireless/mediatek/mt76/mt76.h
@@ -373,6 +373,8 @@ struct mt76_driver_ops {
bool (*tx_status_data)(struct mt76_dev *dev, u8 *update);
+ bool (*rx_check)(struct mt76_dev *dev, void *data, int len);
+
void (*rx_skb)(struct mt76_dev *dev, enum mt76_rxq_id q,
struct sk_buff *skb);
@@ -495,6 +497,8 @@ struct mt76_usb {
};
#define MT76S_XMIT_BUF_SZ (16 * PAGE_SIZE)
+#define MT76S_NUM_TX_ENTRIES 256
+#define MT76S_NUM_RX_ENTRIES 512
struct mt76_sdio {
struct mt76_worker txrx_worker;
struct mt76_worker status_worker;
@@ -599,6 +603,8 @@ struct mt76_testmode_data {
u8 tx_power[4];
u8 tx_power_control;
+ u8 addr[3][ETH_ALEN];
+
u32 tx_pending;
u32 tx_queued;
u16 tx_queued_limit;
@@ -808,7 +814,6 @@ struct mt76_ethtool_worker_info {
}
extern struct ieee80211_rate mt76_rates[12];
-extern const struct cfg80211_sar_capa mt76_sar_capa;
#define __mt76_rr(dev, ...) (dev)->bus->rr((dev), __VA_ARGS__)
#define __mt76_wr(dev, ...) (dev)->bus->wr((dev), __VA_ARGS__)
@@ -1157,6 +1162,11 @@ int mt76_get_min_avg_rssi(struct mt76_dev *dev, bool ext_phy);
int mt76_get_txpower(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
int *dbm);
+int mt76_init_sar_power(struct ieee80211_hw *hw,
+ const struct cfg80211_sar_specs *sar);
+int mt76_get_sar_power(struct mt76_phy *phy,
+ struct ieee80211_channel *chan,
+ int power);
void mt76_csa_check(struct mt76_dev *dev);
void mt76_csa_finish(struct mt76_dev *dev);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7603/mac.c b/drivers/net/wireless/mediatek/mt76/mt7603/mac.c
index fe03e31..a272d64 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7603/mac.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7603/mac.c
@@ -202,10 +202,11 @@ void mt7603_filter_tx(struct mt7603_dev *dev, int idx, bool abort)
FIELD_PREP(MT_DMA_FQCR0_DEST_PORT_ID, port) |
FIELD_PREP(MT_DMA_FQCR0_DEST_QUEUE_ID, queue));
- WARN_ON_ONCE(!mt76_poll(dev, MT_DMA_FQCR0, MT_DMA_FQCR0_BUSY,
- 0, 5000));
+ mt76_poll(dev, MT_DMA_FQCR0, MT_DMA_FQCR0_BUSY, 0, 15000);
}
+ WARN_ON_ONCE(mt76_rr(dev, MT_DMA_FQCR0) & MT_DMA_FQCR0_BUSY);
+
mt76_wr(dev, MT_TX_ABORT, 0);
mt7603_wtbl_set_skip_tx(dev, idx, false);
@@ -525,6 +526,10 @@ mt7603_mac_fill_rx(struct mt7603_dev *dev, struct sk_buff *skb)
if (rxd2 & MT_RXD2_NORMAL_TKIP_MIC_ERR)
status->flag |= RX_FLAG_MMIC_ERROR;
+ /* ICV error or CCMP/BIP/WPI MIC error */
+ if (rxd2 & MT_RXD2_NORMAL_ICV_ERR)
+ status->flag |= RX_FLAG_ONLY_MONITOR;
+
if (FIELD_GET(MT_RXD2_NORMAL_SEC_MODE, rxd2) != 0 &&
!(rxd2 & (MT_RXD2_NORMAL_CLM | MT_RXD2_NORMAL_CM))) {
status->flag |= RX_FLAG_DECRYPTED;
diff --git a/drivers/net/wireless/mediatek/mt76/mt7603/main.c b/drivers/net/wireless/mediatek/mt76/mt7603/main.c
index 7ac4cd24..2b546bc 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7603/main.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7603/main.c
@@ -133,13 +133,15 @@ void mt7603_init_edcca(struct mt7603_dev *dev)
}
static int
-mt7603_set_channel(struct mt7603_dev *dev, struct cfg80211_chan_def *def)
+mt7603_set_channel(struct ieee80211_hw *hw, struct cfg80211_chan_def *def)
{
+ struct mt7603_dev *dev = hw->priv;
u8 *rssi_data = (u8 *)dev->mt76.eeprom.data;
int idx, ret;
u8 bw = MT_BW_20;
bool failed = false;
+ ieee80211_stop_queues(hw);
cancel_delayed_work_sync(&dev->mphy.mac_work);
tasklet_disable(&dev->mt76.pre_tbtt_tasklet);
@@ -205,9 +207,28 @@ mt7603_set_channel(struct mt7603_dev *dev, struct cfg80211_chan_def *def)
if (failed)
mt7603_mac_work(&dev->mphy.mac_work.work);
+ ieee80211_wake_queues(hw);
+
return ret;
}
+static int mt7603_set_sar_specs(struct ieee80211_hw *hw,
+ const struct cfg80211_sar_specs *sar)
+{
+ struct mt7603_dev *dev = hw->priv;
+ struct mt76_phy *mphy = &dev->mphy;
+ int err;
+
+ if (!cfg80211_chandef_valid(&mphy->chandef))
+ return -EINVAL;
+
+ err = mt76_init_sar_power(hw, sar);
+ if (err)
+ return err;
+
+ return mt7603_set_channel(hw, &mphy->chandef);
+}
+
static int
mt7603_config(struct ieee80211_hw *hw, u32 changed)
{
@@ -215,11 +236,8 @@ mt7603_config(struct ieee80211_hw *hw, u32 changed)
int ret = 0;
if (changed & (IEEE80211_CONF_CHANGE_CHANNEL |
- IEEE80211_CONF_CHANGE_POWER)) {
- ieee80211_stop_queues(hw);
- ret = mt7603_set_channel(dev, &hw->conf.chandef);
- ieee80211_wake_queues(hw);
- }
+ IEEE80211_CONF_CHANGE_POWER))
+ ret = mt7603_set_channel(hw, &hw->conf.chandef);
if (changed & IEEE80211_CONF_CHANGE_MONITOR) {
mutex_lock(&dev->mt76.mutex);
@@ -700,6 +718,7 @@ const struct ieee80211_ops mt7603_ops = {
.set_tim = mt76_set_tim,
.get_survey = mt76_get_survey,
.get_antenna = mt76_get_antenna,
+ .set_sar_specs = mt7603_set_sar_specs,
};
MODULE_LICENSE("Dual BSD/GPL");
diff --git a/drivers/net/wireless/mediatek/mt76/mt7603/mcu.c b/drivers/net/wireless/mediatek/mt76/mt7603/mcu.c
index 6abfe6b..7884b95 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7603/mcu.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7603/mcu.c
@@ -403,7 +403,7 @@ int mt7603_mcu_set_channel(struct mt7603_dev *dev)
.tx_streams = n_chains,
.rx_streams = n_chains,
};
- s8 tx_power;
+ s8 tx_power = hw->conf.power_level * 2;
int i, ret;
if (dev->mphy.chandef.width == NL80211_CHAN_WIDTH_40) {
@@ -414,7 +414,7 @@ int mt7603_mcu_set_channel(struct mt7603_dev *dev)
req.center_chan -= 2;
}
- tx_power = hw->conf.power_level * 2;
+ tx_power = mt76_get_sar_power(&dev->mphy, chandef->chan, tx_power);
if (dev->mphy.antenna_mask == 3)
tx_power -= 6;
tx_power = min(tx_power, dev->tx_power_limit);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/debugfs.c b/drivers/net/wireless/mediatek/mt76/mt7615/debugfs.c
index 6fd6f06..b535280 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/debugfs.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/debugfs.c
@@ -359,6 +359,9 @@ mt7615_queues_acq(struct seq_file *s, void *data)
int acs = i / MT7615_MAX_WMM_SETS;
u32 ctrl, val, qlen = 0;
+ if (wmm_idx == 3 && is_mt7663(&dev->mt76))
+ continue;
+
val = mt76_rr(dev, MT_PLE_AC_QEMPTY(acs, wmm_idx));
ctrl = BIT(31) | BIT(15) | (acs << 8);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/init.c b/drivers/net/wireless/mediatek/mt76/mt7615/init.c
index 47f23ac..a753c74 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/init.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/init.c
@@ -194,6 +194,7 @@ mt7615_check_offload_capability(struct mt7615_dev *dev)
ieee80211_hw_set(hw, SUPPORTS_PS);
ieee80211_hw_set(hw, SUPPORTS_DYNAMIC_PS);
+ wiphy->flags &= ~WIPHY_FLAG_4ADDR_STATION;
wiphy->max_remain_on_channel_duration = 5000;
wiphy->features |= NL80211_FEATURE_SCHED_SCAN_RANDOM_MAC_ADDR |
NL80211_FEATURE_SCAN_RANDOM_MAC_ADDR |
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/mac.c b/drivers/net/wireless/mediatek/mt76/mt7615/mac.c
index 423f690..ec25e5a 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/mac.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/mac.c
@@ -249,6 +249,82 @@ static void mt7615_mac_fill_tm_rx(struct mt7615_phy *phy, __le32 *rxv)
#endif
}
+/* The HW does not translate the mac header to 802.3 for mesh point */
+static int mt7615_reverse_frag0_hdr_trans(struct sk_buff *skb, u16 hdr_gap)
+{
+ struct mt76_rx_status *status = (struct mt76_rx_status *)skb->cb;
+ struct mt7615_sta *msta = (struct mt7615_sta *)status->wcid;
+ struct ieee80211_sta *sta;
+ struct ieee80211_vif *vif;
+ struct ieee80211_hdr hdr;
+ struct ethhdr eth_hdr;
+ __le32 *rxd = (__le32 *)skb->data;
+ __le32 qos_ctrl, ht_ctrl;
+
+ if (FIELD_GET(MT_RXD1_NORMAL_ADDR_TYPE, le32_to_cpu(rxd[1])) !=
+ MT_RXD1_NORMAL_U2M)
+ return -EINVAL;
+
+ if (!(le32_to_cpu(rxd[0]) & MT_RXD0_NORMAL_GROUP_4))
+ return -EINVAL;
+
+ if (!msta || !msta->vif)
+ return -EINVAL;
+
+ sta = container_of((void *)msta, struct ieee80211_sta, drv_priv);
+ vif = container_of((void *)msta->vif, struct ieee80211_vif, drv_priv);
+
+ /* store the info from RXD and ethhdr to avoid being overridden */
+ memcpy(ð_hdr, skb->data + hdr_gap, sizeof(eth_hdr));
+ hdr.frame_control = FIELD_GET(MT_RXD4_FRAME_CONTROL, rxd[4]);
+ hdr.seq_ctrl = FIELD_GET(MT_RXD6_SEQ_CTRL, rxd[6]);
+ qos_ctrl = FIELD_GET(MT_RXD6_QOS_CTL, rxd[6]);
+ ht_ctrl = FIELD_GET(MT_RXD7_HT_CONTROL, rxd[7]);
+
+ hdr.duration_id = 0;
+ ether_addr_copy(hdr.addr1, vif->addr);
+ ether_addr_copy(hdr.addr2, sta->addr);
+ switch (le16_to_cpu(hdr.frame_control) &
+ (IEEE80211_FCTL_TODS | IEEE80211_FCTL_FROMDS)) {
+ case 0:
+ ether_addr_copy(hdr.addr3, vif->bss_conf.bssid);
+ break;
+ case IEEE80211_FCTL_FROMDS:
+ ether_addr_copy(hdr.addr3, eth_hdr.h_source);
+ break;
+ case IEEE80211_FCTL_TODS:
+ ether_addr_copy(hdr.addr3, eth_hdr.h_dest);
+ break;
+ case IEEE80211_FCTL_TODS | IEEE80211_FCTL_FROMDS:
+ ether_addr_copy(hdr.addr3, eth_hdr.h_dest);
+ ether_addr_copy(hdr.addr4, eth_hdr.h_source);
+ break;
+ default:
+ break;
+ }
+
+ skb_pull(skb, hdr_gap + sizeof(struct ethhdr) - 2);
+ if (eth_hdr.h_proto == htons(ETH_P_AARP) ||
+ eth_hdr.h_proto == htons(ETH_P_IPX))
+ ether_addr_copy(skb_push(skb, ETH_ALEN), bridge_tunnel_header);
+ else if (eth_hdr.h_proto >= htons(ETH_P_802_3_MIN))
+ ether_addr_copy(skb_push(skb, ETH_ALEN), rfc1042_header);
+ else
+ skb_pull(skb, 2);
+
+ if (ieee80211_has_order(hdr.frame_control))
+ memcpy(skb_push(skb, 2), &ht_ctrl, 2);
+ if (ieee80211_is_data_qos(hdr.frame_control))
+ memcpy(skb_push(skb, 2), &qos_ctrl, 2);
+ if (ieee80211_has_a4(hdr.frame_control))
+ memcpy(skb_push(skb, sizeof(hdr)), &hdr, sizeof(hdr));
+ else
+ memcpy(skb_push(skb, sizeof(hdr) - 6), &hdr, sizeof(hdr) - 6);
+
+ status->flag &= ~(RX_FLAG_RADIOTAP_HE | RX_FLAG_RADIOTAP_HE_MU);
+ return 0;
+}
+
static int mt7615_mac_fill_rx(struct mt7615_dev *dev, struct sk_buff *skb)
{
struct mt76_rx_status *status = (struct mt76_rx_status *)skb->cb;
@@ -263,6 +339,7 @@ static int mt7615_mac_fill_rx(struct mt7615_dev *dev, struct sk_buff *skb)
u32 rxd2 = le32_to_cpu(rxd[2]);
u32 csum_mask = MT_RXD0_NORMAL_IP_SUM | MT_RXD0_NORMAL_UDP_TCP_SUM;
bool unicast, hdr_trans, remove_pad, insert_ccmp_hdr = false;
+ u16 hdr_gap;
int phy_idx;
int i, idx;
u8 chfreq, amsdu_info, qos_ctl = 0;
@@ -286,9 +363,16 @@ static int mt7615_mac_fill_rx(struct mt7615_dev *dev, struct sk_buff *skb)
if (rxd2 & MT_RXD2_NORMAL_AMSDU_ERR)
return -EINVAL;
+ hdr_trans = rxd1 & MT_RXD1_NORMAL_HDR_TRANS;
+ if (hdr_trans && (rxd2 & MT_RXD2_NORMAL_CM))
+ return -EINVAL;
+
+ /* ICV error or CCMP/BIP/WPI MIC error */
+ if (rxd2 & MT_RXD2_NORMAL_ICV_ERR)
+ status->flag |= RX_FLAG_ONLY_MONITOR;
+
unicast = (rxd1 & MT_RXD1_NORMAL_ADDR_TYPE) == MT_RXD1_NORMAL_U2M;
idx = FIELD_GET(MT_RXD2_NORMAL_WLAN_IDX, rxd2);
- hdr_trans = rxd1 & MT_RXD1_NORMAL_HDR_TRANS;
status->wcid = mt7615_rx_get_wcid(dev, idx, unicast);
if (status->wcid) {
@@ -503,16 +587,42 @@ static int mt7615_mac_fill_rx(struct mt7615_dev *dev, struct sk_buff *skb)
return -EINVAL;
}
- skb_pull(skb, (u8 *)rxd - skb->data + 2 * remove_pad);
-
amsdu_info = FIELD_GET(MT_RXD1_NORMAL_PAYLOAD_FORMAT, rxd1);
status->amsdu = !!amsdu_info;
if (status->amsdu) {
status->first_amsdu = amsdu_info == MT_RXD1_FIRST_AMSDU_FRAME;
status->last_amsdu = amsdu_info == MT_RXD1_LAST_AMSDU_FRAME;
- if (!hdr_trans) {
- memmove(skb->data + 2, skb->data,
- ieee80211_get_hdrlen_from_skb(skb));
+ }
+
+ hdr_gap = (u8 *)rxd - skb->data + 2 * remove_pad;
+ if (hdr_trans && ieee80211_has_morefrags(fc)) {
+ if (mt7615_reverse_frag0_hdr_trans(skb, hdr_gap))
+ return -EINVAL;
+ hdr_trans = false;
+ } else {
+ int pad_start = 0;
+
+ skb_pull(skb, hdr_gap);
+ if (!hdr_trans && status->amsdu) {
+ pad_start = ieee80211_get_hdrlen_from_skb(skb);
+ } else if (hdr_trans && (rxd2 & MT_RXD2_NORMAL_HDR_TRANS_ERROR)) {
+ /*
+ * When header translation failure is indicated,
+ * the hardware will insert an extra 2-byte field
+ * containing the data length after the protocol
+ * type field.
+ */
+ pad_start = 12;
+ if (get_unaligned_be16(skb->data + pad_start) == ETH_P_8021Q)
+ pad_start += 4;
+
+ if (get_unaligned_be16(skb->data + pad_start) !=
+ skb->len - pad_start - 2)
+ pad_start = 0;
+ }
+
+ if (pad_start) {
+ memmove(skb->data + 2, skb->data, pad_start);
skb_pull(skb, 2);
}
}
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/mac.h b/drivers/net/wireless/mediatek/mt76/mt7615/mac.h
index 46f283e..e241c61 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/mac.h
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/mac.h
@@ -86,6 +86,8 @@ enum rx_pkt_type {
#define MT_RXD6_SEQ_CTRL GENMASK(15, 0)
#define MT_RXD6_QOS_CTL GENMASK(31, 16)
+#define MT_RXD7_HT_CONTROL GENMASK(31, 0)
+
#define MT_RXV1_ACID_DET_H BIT(31)
#define MT_RXV1_ACID_DET_L BIT(30)
#define MT_RXV1_VHTA2_B8_B3 GENMASK(29, 24)
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/main.c b/drivers/net/wireless/mediatek/mt76/mt7615/main.c
index 890d9b0..82d625a 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/main.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/main.c
@@ -73,7 +73,7 @@ static int mt7615_start(struct ieee80211_hw *hw)
goto out;
}
- ret = mt7615_mcu_set_chan_info(phy, MCU_EXT_CMD_SET_RX_PATH);
+ ret = mt7615_mcu_set_chan_info(phy, MCU_EXT_CMD(SET_RX_PATH));
if (ret)
goto out;
@@ -141,9 +141,6 @@ static int get_omac_idx(enum nl80211_iftype type, u64 mask)
if (i)
return i - 1;
- if (type != NL80211_IFTYPE_STATION)
- break;
-
/* next, try to find a free repeater entry for the sta */
i = get_free_idx(mask >> REPEATER_BSSID_START, 0,
REPEATER_BSSID_MAX - REPEATER_BSSID_START);
@@ -211,11 +208,9 @@ static int mt7615_add_interface(struct ieee80211_hw *hw,
mvif->mt76.omac_idx = idx;
mvif->mt76.band_idx = ext_phy;
- if (mt7615_ext_phy(dev))
- mvif->mt76.wmm_idx = ext_phy * (MT7615_MAX_WMM_SETS / 2) +
- mvif->mt76.idx % (MT7615_MAX_WMM_SETS / 2);
- else
- mvif->mt76.wmm_idx = mvif->mt76.idx % MT7615_MAX_WMM_SETS;
+ mvif->mt76.wmm_idx = vif->type != NL80211_IFTYPE_AP;
+ if (ext_phy)
+ mvif->mt76.wmm_idx += 2;
dev->mt76.vif_mask |= BIT(mvif->mt76.idx);
dev->omac_mask |= BIT_ULL(mvif->mt76.omac_idx);
@@ -331,7 +326,7 @@ int mt7615_set_channel(struct mt7615_phy *phy)
goto out;
}
- ret = mt7615_mcu_set_chan_info(phy, MCU_EXT_CMD_CHANNEL_SWITCH);
+ ret = mt7615_mcu_set_chan_info(phy, MCU_EXT_CMD(CHANNEL_SWITCH));
if (ret)
goto out;
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/mcu.c b/drivers/net/wireless/mediatek/mt76/mt7615/mcu.c
index 25f9cbe..759dcf0 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/mcu.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/mcu.c
@@ -87,7 +87,7 @@ struct mt7663_fw_buf {
void mt7615_mcu_fill_msg(struct mt7615_dev *dev, struct sk_buff *skb,
int cmd, int *wait_seq)
{
- int txd_len, mcu_cmd = cmd & MCU_CMD_MASK;
+ int txd_len, mcu_cmd = FIELD_GET(__MCU_CMD_FIELD_ID, cmd);
struct mt7615_uni_txd *uni_txd;
struct mt7615_mcu_txd *mcu_txd;
u8 seq, q_idx, pkt_fmt;
@@ -103,10 +103,10 @@ void mt7615_mcu_fill_msg(struct mt7615_dev *dev, struct sk_buff *skb,
if (wait_seq)
*wait_seq = seq;
- txd_len = cmd & MCU_UNI_PREFIX ? sizeof(*uni_txd) : sizeof(*mcu_txd);
+ txd_len = cmd & __MCU_CMD_FIELD_UNI ? sizeof(*uni_txd) : sizeof(*mcu_txd);
txd = (__le32 *)skb_push(skb, txd_len);
- if (cmd != MCU_CMD_FW_SCATTER) {
+ if (cmd != MCU_CMD(FW_SCATTER)) {
q_idx = MT_TX_MCU_PORT_RX_Q0;
pkt_fmt = MT_TX_TYPE_CMD;
} else {
@@ -124,7 +124,7 @@ void mt7615_mcu_fill_msg(struct mt7615_dev *dev, struct sk_buff *skb,
FIELD_PREP(MT_TXD1_PKT_FMT, pkt_fmt);
txd[1] = cpu_to_le32(val);
- if (cmd & MCU_UNI_PREFIX) {
+ if (cmd & __MCU_CMD_FIELD_UNI) {
uni_txd = (struct mt7615_uni_txd *)txd;
uni_txd->len = cpu_to_le16(skb->len - sizeof(uni_txd->txd));
uni_txd->option = MCU_CMD_UNI_EXT_ACK;
@@ -142,28 +142,17 @@ void mt7615_mcu_fill_msg(struct mt7615_dev *dev, struct sk_buff *skb,
mcu_txd->s2d_index = MCU_S2D_H2N;
mcu_txd->pkt_type = MCU_PKT_ID;
mcu_txd->seq = seq;
+ mcu_txd->cid = mcu_cmd;
+ mcu_txd->ext_cid = FIELD_GET(__MCU_CMD_FIELD_EXT_ID, cmd);
- switch (cmd & ~MCU_CMD_MASK) {
- case MCU_FW_PREFIX:
+ if (mcu_txd->ext_cid || (cmd & __MCU_CMD_FIELD_CE)) {
+ if (cmd & __MCU_CMD_FIELD_QUERY)
+ mcu_txd->set_query = MCU_Q_QUERY;
+ else
+ mcu_txd->set_query = MCU_Q_SET;
+ mcu_txd->ext_cid_ack = !!mcu_txd->ext_cid;
+ } else {
mcu_txd->set_query = MCU_Q_NA;
- mcu_txd->cid = mcu_cmd;
- break;
- case MCU_CE_PREFIX:
- if (cmd & MCU_QUERY_MASK)
- mcu_txd->set_query = MCU_Q_QUERY;
- else
- mcu_txd->set_query = MCU_Q_SET;
- mcu_txd->cid = mcu_cmd;
- break;
- default:
- mcu_txd->cid = MCU_CMD_EXT_CID;
- if (cmd & MCU_QUERY_PREFIX)
- mcu_txd->set_query = MCU_Q_QUERY;
- else
- mcu_txd->set_query = MCU_Q_SET;
- mcu_txd->ext_cid = mcu_cmd;
- mcu_txd->ext_cid_ack = 1;
- break;
}
}
EXPORT_SYMBOL_GPL(mt7615_mcu_fill_msg);
@@ -184,42 +173,32 @@ int mt7615_mcu_parse_response(struct mt76_dev *mdev, int cmd,
if (seq != rxd->seq)
return -EAGAIN;
- switch (cmd) {
- case MCU_CMD_PATCH_SEM_CONTROL:
+ if (cmd == MCU_CMD(PATCH_SEM_CONTROL)) {
skb_pull(skb, sizeof(*rxd) - 4);
ret = *skb->data;
- break;
- case MCU_EXT_CMD_GET_TEMP:
+ } else if (cmd == MCU_EXT_CMD(THERMAL_CTRL)) {
skb_pull(skb, sizeof(*rxd));
ret = le32_to_cpu(*(__le32 *)skb->data);
- break;
- case MCU_EXT_CMD_RF_REG_ACCESS | MCU_QUERY_PREFIX:
+ } else if (cmd == MCU_EXT_QUERY(RF_REG_ACCESS)) {
skb_pull(skb, sizeof(*rxd));
ret = le32_to_cpu(*(__le32 *)&skb->data[8]);
- break;
- case MCU_UNI_CMD_DEV_INFO_UPDATE:
- case MCU_UNI_CMD_BSS_INFO_UPDATE:
- case MCU_UNI_CMD_STA_REC_UPDATE:
- case MCU_UNI_CMD_HIF_CTRL:
- case MCU_UNI_CMD_OFFLOAD:
- case MCU_UNI_CMD_SUSPEND: {
+ } else if (cmd == MCU_UNI_CMD(DEV_INFO_UPDATE) ||
+ cmd == MCU_UNI_CMD(BSS_INFO_UPDATE) ||
+ cmd == MCU_UNI_CMD(STA_REC_UPDATE) ||
+ cmd == MCU_UNI_CMD(HIF_CTRL) ||
+ cmd == MCU_UNI_CMD(OFFLOAD) ||
+ cmd == MCU_UNI_CMD(SUSPEND)) {
struct mt7615_mcu_uni_event *event;
skb_pull(skb, sizeof(*rxd));
event = (struct mt7615_mcu_uni_event *)skb->data;
ret = le32_to_cpu(event->status);
- break;
- }
- case MCU_CMD_REG_READ: {
+ } else if (cmd == MCU_CE_QUERY(REG_READ)) {
struct mt7615_mcu_reg_event *event;
skb_pull(skb, sizeof(*rxd));
event = (struct mt7615_mcu_reg_event *)skb->data;
ret = (int)le32_to_cpu(event->val);
- break;
- }
- default:
- break;
}
return ret;
@@ -253,8 +232,7 @@ u32 mt7615_rf_rr(struct mt7615_dev *dev, u32 wf, u32 reg)
.address = cpu_to_le32(reg),
};
- return mt76_mcu_send_msg(&dev->mt76,
- MCU_EXT_CMD_RF_REG_ACCESS | MCU_QUERY_PREFIX,
+ return mt76_mcu_send_msg(&dev->mt76, MCU_EXT_QUERY(RF_REG_ACCESS),
&req, sizeof(req), true);
}
@@ -270,8 +248,8 @@ int mt7615_rf_wr(struct mt7615_dev *dev, u32 wf, u32 reg, u32 val)
.data = cpu_to_le32(val),
};
- return mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD_RF_REG_ACCESS, &req,
- sizeof(req), false);
+ return mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD(RF_REG_ACCESS),
+ &req, sizeof(req), false);
}
void mt7622_trigger_hif_int(struct mt7615_dev *dev, bool en)
@@ -658,8 +636,8 @@ mt7615_mcu_muar_config(struct mt7615_dev *dev, struct ieee80211_vif *vif,
if (enable)
ether_addr_copy(req.addr, addr);
- return mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD_MUAR_UPDATE, &req,
- sizeof(req), true);
+ return mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD(MUAR_UPDATE),
+ &req, sizeof(req), true);
}
static int
@@ -702,7 +680,7 @@ mt7615_mcu_add_dev(struct mt7615_phy *phy, struct ieee80211_vif *vif,
return mt7615_mcu_muar_config(dev, vif, false, enable);
memcpy(data.tlv.omac_addr, vif->addr, ETH_ALEN);
- return mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD_DEV_INFO_UPDATE,
+ return mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD(DEV_INFO_UPDATE),
&data, sizeof(data), true);
}
@@ -771,7 +749,7 @@ mt7615_mcu_add_beacon_offload(struct mt7615_dev *dev,
dev_kfree_skb(skb);
out:
- return mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD_BCN_OFFLOAD, &req,
+ return mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD(BCN_OFFLOAD), &req,
sizeof(req), true);
}
@@ -802,8 +780,8 @@ mt7615_mcu_ctrl_pm_state(struct mt7615_dev *dev, int band, int state)
.band_idx = band,
};
- return mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD_PM_STATE_CTRL, &req,
- sizeof(req), true);
+ return mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD(PM_STATE_CTRL),
+ &req, sizeof(req), true);
}
static int
@@ -843,7 +821,7 @@ mt7615_mcu_bss_basic_tlv(struct sk_buff *skb, struct ieee80211_vif *vif,
bss = (struct bss_info_basic *)tlv;
bss->network_type = cpu_to_le32(type);
- bss->bmc_tx_wlan_idx = wlan_idx;
+ bss->bmc_wcid_lo = wlan_idx;
bss->wmm_idx = mvif->mt76.wmm_idx;
bss->active = enable;
@@ -944,7 +922,7 @@ mt7615_mcu_add_bss(struct mt7615_phy *phy, struct ieee80211_vif *vif,
mt7615_mcu_bss_ext_tlv(skb, mvif);
return mt76_mcu_skb_send_msg(&dev->mt76, skb,
- MCU_EXT_CMD_BSS_INFO_UPDATE, true);
+ MCU_EXT_CMD(BSS_INFO_UPDATE), true);
}
static int
@@ -966,8 +944,8 @@ mt7615_mcu_wtbl_tx_ba(struct mt7615_dev *dev,
mt76_connac_mcu_wtbl_ba_tlv(&dev->mt76, skb, params, enable, true,
NULL, wtbl_hdr);
- err = mt76_mcu_skb_send_msg(&dev->mt76, skb, MCU_EXT_CMD_WTBL_UPDATE,
- true);
+ err = mt76_mcu_skb_send_msg(&dev->mt76, skb,
+ MCU_EXT_CMD(WTBL_UPDATE), true);
if (err < 0)
return err;
@@ -979,7 +957,7 @@ mt7615_mcu_wtbl_tx_ba(struct mt7615_dev *dev,
mt76_connac_mcu_sta_ba_tlv(skb, params, enable, true);
return mt76_mcu_skb_send_msg(&dev->mt76, skb,
- MCU_EXT_CMD_STA_REC_UPDATE, true);
+ MCU_EXT_CMD(STA_REC_UPDATE), true);
}
static int
@@ -1001,7 +979,7 @@ mt7615_mcu_wtbl_rx_ba(struct mt7615_dev *dev,
mt76_connac_mcu_sta_ba_tlv(skb, params, enable, false);
err = mt76_mcu_skb_send_msg(&dev->mt76, skb,
- MCU_EXT_CMD_STA_REC_UPDATE, true);
+ MCU_EXT_CMD(STA_REC_UPDATE), true);
if (err < 0 || !enable)
return err;
@@ -1014,8 +992,8 @@ mt7615_mcu_wtbl_rx_ba(struct mt7615_dev *dev,
mt76_connac_mcu_wtbl_ba_tlv(&dev->mt76, skb, params, enable, false,
NULL, wtbl_hdr);
- return mt76_mcu_skb_send_msg(&dev->mt76, skb, MCU_EXT_CMD_WTBL_UPDATE,
- true);
+ return mt76_mcu_skb_send_msg(&dev->mt76, skb,
+ MCU_EXT_CMD(WTBL_UPDATE), true);
}
static int
@@ -1057,7 +1035,7 @@ mt7615_mcu_wtbl_sta_add(struct mt7615_phy *phy, struct ieee80211_vif *vif,
NULL, wtbl_hdr);
}
- cmd = enable ? MCU_EXT_CMD_WTBL_UPDATE : MCU_EXT_CMD_STA_REC_UPDATE;
+ cmd = enable ? MCU_EXT_CMD(WTBL_UPDATE) : MCU_EXT_CMD(STA_REC_UPDATE);
skb = enable ? wskb : sskb;
err = mt76_mcu_skb_send_msg(&dev->mt76, skb, cmd, true);
@@ -1068,7 +1046,7 @@ mt7615_mcu_wtbl_sta_add(struct mt7615_phy *phy, struct ieee80211_vif *vif,
return err;
}
- cmd = enable ? MCU_EXT_CMD_STA_REC_UPDATE : MCU_EXT_CMD_WTBL_UPDATE;
+ cmd = enable ? MCU_EXT_CMD(STA_REC_UPDATE) : MCU_EXT_CMD(WTBL_UPDATE);
skb = enable ? sskb : wskb;
return mt76_mcu_skb_send_msg(&dev->mt76, skb, cmd, true);
@@ -1090,8 +1068,8 @@ mt7615_mcu_wtbl_update_hdr_trans(struct mt7615_dev *dev,
mt76_connac_mcu_wtbl_hdr_trans_tlv(skb, vif, &msta->wcid, NULL,
wtbl_hdr);
- return mt76_mcu_skb_send_msg(&dev->mt76, skb, MCU_EXT_CMD_WTBL_UPDATE,
- true);
+ return mt76_mcu_skb_send_msg(&dev->mt76, skb,
+ MCU_EXT_CMD(WTBL_UPDATE), true);
}
static const struct mt7615_mcu_ops wtbl_update_ops = {
@@ -1136,7 +1114,7 @@ mt7615_mcu_sta_ba(struct mt7615_dev *dev,
sta_wtbl, wtbl_hdr);
return mt76_mcu_skb_send_msg(&dev->mt76, skb,
- MCU_EXT_CMD_STA_REC_UPDATE, true);
+ MCU_EXT_CMD(STA_REC_UPDATE), true);
}
static int
@@ -1179,7 +1157,7 @@ mt7615_mcu_add_sta(struct mt7615_phy *phy, struct ieee80211_vif *vif,
struct ieee80211_sta *sta, bool enable)
{
return __mt7615_mcu_add_sta(phy->mt76, vif, sta, enable,
- MCU_EXT_CMD_STA_REC_UPDATE, false);
+ MCU_EXT_CMD(STA_REC_UPDATE), false);
}
static int
@@ -1191,7 +1169,7 @@ mt7615_mcu_sta_update_hdr_trans(struct mt7615_dev *dev,
return mt76_connac_mcu_sta_update_hdr_trans(&dev->mt76,
vif, &msta->wcid,
- MCU_EXT_CMD_STA_REC_UPDATE);
+ MCU_EXT_CMD(STA_REC_UPDATE));
}
static const struct mt7615_mcu_ops sta_update_ops = {
@@ -1285,7 +1263,7 @@ mt7615_mcu_uni_add_beacon_offload(struct mt7615_dev *dev,
dev_kfree_skb(skb);
out:
- return mt76_mcu_send_msg(&dev->mt76, MCU_UNI_CMD_BSS_INFO_UPDATE,
+ return mt76_mcu_send_msg(&dev->mt76, MCU_UNI_CMD(BSS_INFO_UPDATE),
&req, sizeof(req), true);
}
@@ -1314,7 +1292,7 @@ mt7615_mcu_uni_add_sta(struct mt7615_phy *phy, struct ieee80211_vif *vif,
struct ieee80211_sta *sta, bool enable)
{
return __mt7615_mcu_add_sta(phy->mt76, vif, sta, enable,
- MCU_UNI_CMD_STA_REC_UPDATE, true);
+ MCU_UNI_CMD(STA_REC_UPDATE), true);
}
static int
@@ -1348,7 +1326,7 @@ mt7615_mcu_uni_rx_ba(struct mt7615_dev *dev,
mt76_connac_mcu_sta_ba_tlv(skb, params, enable, false);
err = mt76_mcu_skb_send_msg(&dev->mt76, skb,
- MCU_UNI_CMD_STA_REC_UPDATE, true);
+ MCU_UNI_CMD(STA_REC_UPDATE), true);
if (err < 0 || !enable)
return err;
@@ -1369,7 +1347,7 @@ mt7615_mcu_uni_rx_ba(struct mt7615_dev *dev,
sta_wtbl, wtbl_hdr);
return mt76_mcu_skb_send_msg(&dev->mt76, skb,
- MCU_UNI_CMD_STA_REC_UPDATE, true);
+ MCU_UNI_CMD(STA_REC_UPDATE), true);
}
static int
@@ -1381,7 +1359,7 @@ mt7615_mcu_sta_uni_update_hdr_trans(struct mt7615_dev *dev,
return mt76_connac_mcu_sta_update_hdr_trans(&dev->mt76,
vif, &msta->wcid,
- MCU_UNI_CMD_STA_REC_UPDATE);
+ MCU_UNI_CMD(STA_REC_UPDATE));
}
static const struct mt7615_mcu_ops uni_update_ops = {
@@ -1399,7 +1377,7 @@ static const struct mt7615_mcu_ops uni_update_ops = {
int mt7615_mcu_restart(struct mt76_dev *dev)
{
- return mt76_mcu_send_msg(dev, MCU_CMD_RESTART_DL_REQ, NULL, 0, true);
+ return mt76_mcu_send_msg(dev, MCU_CMD(RESTART_DL_REQ), NULL, 0, true);
}
EXPORT_SYMBOL_GPL(mt7615_mcu_restart);
@@ -1445,7 +1423,7 @@ static int mt7615_load_patch(struct mt7615_dev *dev, u32 addr, const char *name)
goto out;
}
- ret = mt76_mcu_send_firmware(&dev->mt76, MCU_CMD_FW_SCATTER,
+ ret = mt76_mcu_send_firmware(&dev->mt76, MCU_CMD(FW_SCATTER),
fw->data + sizeof(*hdr), len);
if (ret) {
dev_err(dev->mt76.dev, "Failed to send firmware to device\n");
@@ -1508,7 +1486,7 @@ mt7615_mcu_send_ram_firmware(struct mt7615_dev *dev,
return err;
}
- err = mt76_mcu_send_firmware(&dev->mt76, MCU_CMD_FW_SCATTER,
+ err = mt76_mcu_send_firmware(&dev->mt76, MCU_CMD(FW_SCATTER),
data + offset, len);
if (err) {
dev_err(dev->mt76.dev, "Failed to send firmware to device\n");
@@ -1644,7 +1622,7 @@ static int mt7615_load_firmware(struct mt7615_dev *dev)
if (!mt76_poll_msec(dev, MT_TOP_MISC2, MT_TOP_MISC2_FW_STATE,
FIELD_PREP(MT_TOP_MISC2_FW_STATE,
- FW_STATE_CR4_RDY), 500)) {
+ FW_STATE_RDY), 500)) {
dev_err(dev->mt76.dev, "Timeout for initializing firmware\n");
return -EIO;
}
@@ -1694,8 +1672,8 @@ int mt7615_mcu_fw_log_2_host(struct mt7615_dev *dev, u8 ctrl)
.ctrl_val = ctrl
};
- return mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD_FW_LOG_2_HOST, &data,
- sizeof(data), true);
+ return mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD(FW_LOG_2_HOST),
+ &data, sizeof(data), true);
}
static int mt7615_mcu_cal_cache_apply(struct mt7615_dev *dev)
@@ -1707,7 +1685,7 @@ static int mt7615_mcu_cal_cache_apply(struct mt7615_dev *dev)
.cache_enable = true
};
- return mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD_CAL_CACHE, &data,
+ return mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD(CAL_CACHE), &data,
sizeof(data), false);
}
@@ -1756,7 +1734,7 @@ static int mt7663_load_n9(struct mt7615_dev *dev, const char *name)
goto out;
}
- ret = mt76_mcu_send_firmware(&dev->mt76, MCU_CMD_FW_SCATTER,
+ ret = mt76_mcu_send_firmware(&dev->mt76, MCU_CMD(FW_SCATTER),
fw->data + offset, len);
if (ret) {
dev_err(dev->mt76.dev, "Failed to send firmware\n");
@@ -1977,7 +1955,7 @@ int mt7615_mcu_set_eeprom(struct mt7615_dev *dev)
skb_put_data(skb, eep + offset, eep_len);
return mt76_mcu_skb_send_msg(&dev->mt76, skb,
- MCU_EXT_CMD_EFUSE_BUFFER_MODE, true);
+ MCU_EXT_CMD(EFUSE_BUFFER_MODE), true);
}
int mt7615_mcu_set_wmm(struct mt7615_dev *dev, u8 queue,
@@ -2013,8 +1991,8 @@ int mt7615_mcu_set_wmm(struct mt7615_dev *dev, u8 queue,
if (params->cw_max)
req.cw_max = cpu_to_le16(fls(params->cw_max));
- return mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD_EDCA_UPDATE, &req,
- sizeof(req), true);
+ return mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD(EDCA_UPDATE),
+ &req, sizeof(req), true);
}
int mt7615_mcu_set_dbdc(struct mt7615_dev *dev)
@@ -2072,7 +2050,7 @@ int mt7615_mcu_set_dbdc(struct mt7615_dev *dev)
ADD_DBDC_ENTRY(DBDC_TYPE_MGMT, 1, 1);
out:
- return mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD_DBDC_CTRL, &req,
+ return mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD(DBDC_CTRL), &req,
sizeof(req), true);
}
@@ -2082,8 +2060,8 @@ int mt7615_mcu_del_wtbl_all(struct mt7615_dev *dev)
.operation = WTBL_RESET_ALL,
};
- return mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD_WTBL_UPDATE, &req,
- sizeof(req), true);
+ return mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD(WTBL_UPDATE),
+ &req, sizeof(req), true);
}
int mt7615_mcu_rdd_cmd(struct mt7615_dev *dev,
@@ -2103,8 +2081,8 @@ int mt7615_mcu_rdd_cmd(struct mt7615_dev *dev,
.val = val,
};
- return mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD_SET_RDD_CTRL, &req,
- sizeof(req), true);
+ return mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD(SET_RDD_CTRL),
+ &req, sizeof(req), true);
}
int mt7615_mcu_set_fcc5_lpn(struct mt7615_dev *dev, int val)
@@ -2117,8 +2095,8 @@ int mt7615_mcu_set_fcc5_lpn(struct mt7615_dev *dev, int val)
.min_lpn = cpu_to_le16(val),
};
- return mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD_SET_RDD_TH, &req,
- sizeof(req), true);
+ return mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD(SET_RADAR_TH),
+ &req, sizeof(req), true);
}
int mt7615_mcu_set_pulse_th(struct mt7615_dev *dev,
@@ -2146,8 +2124,8 @@ int mt7615_mcu_set_pulse_th(struct mt7615_dev *dev,
#undef __req_field
};
- return mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD_SET_RDD_TH, &req,
- sizeof(req), true);
+ return mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD(SET_RADAR_TH),
+ &req, sizeof(req), true);
}
int mt7615_mcu_set_radar_th(struct mt7615_dev *dev, int index,
@@ -2193,8 +2171,8 @@ int mt7615_mcu_set_radar_th(struct mt7615_dev *dev, int index,
#undef __req_field_u32
};
- return mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD_SET_RDD_TH, &req,
- sizeof(req), true);
+ return mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD(SET_RADAR_TH),
+ &req, sizeof(req), true);
}
int mt7615_mcu_rdd_send_pattern(struct mt7615_dev *dev)
@@ -2225,7 +2203,7 @@ int mt7615_mcu_rdd_send_pattern(struct mt7615_dev *dev)
req.pattern[i].start_time = cpu_to_le32(ts);
}
- return mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD_SET_RDD_PATTERN,
+ return mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD(SET_RDD_PATTERN),
&req, sizeof(req), false);
}
@@ -2394,8 +2372,8 @@ int mt7615_mcu_get_temperature(struct mt7615_dev *dev)
u8 rsv[3];
} req = {};
- return mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD_GET_TEMP, &req,
- sizeof(req), true);
+ return mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD(THERMAL_CTRL),
+ &req, sizeof(req), true);
}
int mt7615_mcu_set_test_param(struct mt7615_dev *dev, u8 param, bool test_mode,
@@ -2415,8 +2393,8 @@ int mt7615_mcu_set_test_param(struct mt7615_dev *dev, u8 param, bool test_mode,
.value = cpu_to_le32(val),
};
- return mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD_ATE_CTRL, &req,
- sizeof(req), false);
+ return mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD(ATE_CTRL),
+ &req, sizeof(req), false);
}
int mt7615_mcu_set_sku_en(struct mt7615_phy *phy, bool enable)
@@ -2434,8 +2412,8 @@ int mt7615_mcu_set_sku_en(struct mt7615_phy *phy, bool enable)
};
return mt76_mcu_send_msg(&dev->mt76,
- MCU_EXT_CMD_TX_POWER_FEATURE_CTRL, &req,
- sizeof(req), true);
+ MCU_EXT_CMD(TX_POWER_FEATURE_CTRL),
+ &req, sizeof(req), true);
}
static int mt7615_find_freq_idx(const u16 *freqs, int n_freqs, u16 cur)
@@ -2574,7 +2552,7 @@ int mt7615_mcu_apply_rx_dcoc(struct mt7615_phy *phy)
out:
req.center_freq = cpu_to_le16(center_freq);
- ret = mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD_RXDCOC_CAL, &req,
+ ret = mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD(RXDCOC_CAL), &req,
sizeof(req), true);
if ((chandef->width == NL80211_CHAN_WIDTH_80P80 ||
@@ -2695,8 +2673,8 @@ int mt7615_mcu_apply_tx_dpd(struct mt7615_phy *phy)
out:
req.center_freq = cpu_to_le16(center_freq);
- ret = mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD_TXDPD_CAL, &req,
- sizeof(req), true);
+ ret = mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD(TXDPD_CAL),
+ &req, sizeof(req), true);
if ((chandef->width == NL80211_CHAN_WIDTH_80P80 ||
chandef->width == NL80211_CHAN_WIDTH_160) && !req.is_freq2) {
@@ -2724,7 +2702,7 @@ int mt7615_mcu_set_rx_hdr_trans_blacklist(struct mt7615_dev *dev)
.etype = cpu_to_le16(ETH_P_PAE),
};
- return mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD_RX_HDR_TRANS,
+ return mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD(RX_HDR_TRANS),
&req, sizeof(req), false);
}
@@ -2759,13 +2737,13 @@ int mt7615_mcu_set_bss_pm(struct mt7615_dev *dev, struct ieee80211_vif *vif,
if (vif->type != NL80211_IFTYPE_STATION)
return 0;
- err = mt76_mcu_send_msg(&dev->mt76, MCU_CMD_SET_BSS_ABORT, &req_hdr,
- sizeof(req_hdr), false);
+ err = mt76_mcu_send_msg(&dev->mt76, MCU_CE_CMD(SET_BSS_ABORT),
+ &req_hdr, sizeof(req_hdr), false);
if (err < 0 || !enable)
return err;
- return mt76_mcu_send_msg(&dev->mt76, MCU_CMD_SET_BSS_CONNECTED, &req,
- sizeof(req), false);
+ return mt76_mcu_send_msg(&dev->mt76, MCU_CE_CMD(SET_BSS_CONNECTED),
+ &req, sizeof(req), false);
}
int mt7615_mcu_set_roc(struct mt7615_phy *phy, struct ieee80211_vif *vif,
@@ -2784,6 +2762,6 @@ int mt7615_mcu_set_roc(struct mt7615_phy *phy, struct ieee80211_vif *vif,
phy->roc_grant = false;
- return mt76_mcu_send_msg(&dev->mt76, MCU_CMD_SET_ROC, &req,
- sizeof(req), false);
+ return mt76_mcu_send_msg(&dev->mt76, MCU_CE_CMD(SET_ROC),
+ &req, sizeof(req), false);
}
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/mcu.h b/drivers/net/wireless/mediatek/mt76/mt7615/mcu.h
index 98c383e..47863ae 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/mcu.h
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/mcu.h
@@ -76,35 +76,6 @@ struct mt7615_uni_txd {
u8 reserved2[4];
} __packed __aligned(4);
-/* event table */
-enum {
- MCU_EVENT_TARGET_ADDRESS_LEN = 0x01,
- MCU_EVENT_FW_START = 0x01,
- MCU_EVENT_GENERIC = 0x01,
- MCU_EVENT_ACCESS_REG = 0x02,
- MCU_EVENT_MT_PATCH_SEM = 0x04,
- MCU_EVENT_REG_ACCESS = 0x05,
- MCU_EVENT_SCAN_DONE = 0x0d,
- MCU_EVENT_ROC = 0x10,
- MCU_EVENT_BSS_ABSENCE = 0x11,
- MCU_EVENT_BSS_BEACON_LOSS = 0x13,
- MCU_EVENT_CH_PRIVILEGE = 0x18,
- MCU_EVENT_SCHED_SCAN_DONE = 0x23,
- MCU_EVENT_EXT = 0xed,
- MCU_EVENT_RESTART_DL = 0xef,
- MCU_EVENT_COREDUMP = 0xf0,
-};
-
-/* ext event table */
-enum {
- MCU_EXT_EVENT_PS_SYNC = 0x5,
- MCU_EXT_EVENT_FW_LOG_2_HOST = 0x13,
- MCU_EXT_EVENT_THERMAL_PROTECT = 0x22,
- MCU_EXT_EVENT_ASSERT_DUMP = 0x23,
- MCU_EXT_EVENT_RDD_REPORT = 0x3a,
- MCU_EXT_EVENT_CSA_NOTIFY = 0x4f,
-};
-
enum {
MT_SKU_CCK_1_2 = 0,
MT_SKU_CCK_55_11,
@@ -234,20 +205,6 @@ struct mt7615_mcu_rdd_report {
#define MCU_PKT_ID 0xa0
enum {
- MCU_Q_QUERY,
- MCU_Q_SET,
- MCU_Q_RESERVED,
- MCU_Q_NA
-};
-
-enum {
- MCU_S2D_H2N,
- MCU_S2D_C2N,
- MCU_S2D_H2C,
- MCU_S2D_H2CN
-};
-
-enum {
MCU_ATE_SET_FREQ_OFFSET = 0xa,
MCU_ATE_SET_TX_POWER_CONTROL = 0x15,
};
@@ -281,21 +238,6 @@ struct mt7615_roc_tlv {
} __packed;
enum {
- PATCH_NOT_DL_SEM_FAIL = 0x0,
- PATCH_IS_DL = 0x1,
- PATCH_NOT_DL_SEM_SUCCESS = 0x2,
- PATCH_REL_SEM_SUCCESS = 0x3
-};
-
-enum {
- FW_STATE_INITIAL = 0,
- FW_STATE_FW_DOWNLOAD = 1,
- FW_STATE_NORMAL_OPERATION = 2,
- FW_STATE_NORMAL_TRX = 3,
- FW_STATE_CR4_RDY = 7
-};
-
-enum {
FW_STATE_PWR_ON = 1,
FW_STATE_N9_RDY = 2,
};
@@ -312,73 +254,4 @@ enum {
__DBDC_TYPE_MAX,
};
-struct bss_info_omac {
- __le16 tag;
- __le16 len;
- u8 hw_bss_idx;
- u8 omac_idx;
- u8 band_idx;
- u8 rsv0;
- __le32 conn_type;
- u32 rsv1;
-} __packed;
-
-struct bss_info_basic {
- __le16 tag;
- __le16 len;
- __le32 network_type;
- u8 active;
- u8 rsv0;
- __le16 bcn_interval;
- u8 bssid[ETH_ALEN];
- u8 wmm_idx;
- u8 dtim_period;
- u8 bmc_tx_wlan_idx;
- u8 cipher; /* not used */
- u8 phymode; /* not used */
- u8 rsv1[5];
-} __packed;
-
-struct bss_info_rf_ch {
- __le16 tag;
- __le16 len;
- u8 pri_ch;
- u8 central_ch0;
- u8 central_ch1;
- u8 bw;
-} __packed;
-
-struct bss_info_ext_bss {
- __le16 tag;
- __le16 len;
- __le32 mbss_tsf_offset; /* in unit of us */
- u8 rsv[8];
-} __packed;
-
-enum {
- BSS_INFO_OMAC,
- BSS_INFO_BASIC,
- BSS_INFO_RF_CH, /* optional, for BT/LTE coex */
- BSS_INFO_PM, /* sta only */
- BSS_INFO_UAPSD, /* sta only */
- BSS_INFO_ROAM_DETECTION, /* obsoleted */
- BSS_INFO_LQ_RM, /* obsoleted */
- BSS_INFO_EXT_BSS,
- BSS_INFO_BMC_INFO, /* for bmc rate control in CR4 */
- BSS_INFO_SYNC_MODE, /* obsoleted */
- BSS_INFO_RA,
- BSS_INFO_MAX_NUM
-};
-
-enum {
- CH_SWITCH_NORMAL = 0,
- CH_SWITCH_SCAN = 3,
- CH_SWITCH_MCC = 4,
- CH_SWITCH_DFS = 5,
- CH_SWITCH_BACKGROUND_SCAN_START = 6,
- CH_SWITCH_BACKGROUND_SCAN_RUNNING = 7,
- CH_SWITCH_BACKGROUND_SCAN_STOP = 8,
- CH_SWITCH_SCAN_BYPASS_DPD = 9
-};
-
#endif
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/mmio.c b/drivers/net/wireless/mediatek/mt76/mt7615/mmio.c
index 71719c7..33f72f3 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/mmio.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/mmio.c
@@ -135,6 +135,7 @@ static void mt7615_irq_tasklet(struct tasklet_struct *t)
if (is_mt7663(&dev->mt76)) {
mcu_int = mt76_rr(dev, MT_MCU2HOST_INT_STATUS);
mcu_int &= MT7663_MCU_CMD_ERROR_MASK;
+ mt76_wr(dev, MT_MCU2HOST_INT_STATUS, mcu_int);
} else {
mcu_int = mt76_rr(dev, MT_MCU_CMD);
mcu_int &= MT_MCU_CMD_ERROR_MASK;
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/pci_init.c b/drivers/net/wireless/mediatek/mt76/mt7615/pci_init.c
index a2465b4..87b4aa5 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/pci_init.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/pci_init.c
@@ -28,8 +28,6 @@ static void mt7615_pci_init_work(struct work_struct *work)
return;
mt7615_init_work(dev);
- if (dev->dbdc_support)
- mt7615_register_ext_phy(dev);
}
static int mt7615_init_hardware(struct mt7615_dev *dev)
@@ -160,6 +158,12 @@ int mt7615_register_device(struct mt7615_dev *dev)
mt7615_init_txpower(dev, &dev->mphy.sband_2g.sband);
mt7615_init_txpower(dev, &dev->mphy.sband_5g.sband);
+ if (dev->dbdc_support) {
+ ret = mt7615_register_ext_phy(dev);
+ if (ret)
+ return ret;
+ }
+
return mt7615_init_debugfs(dev);
}
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/testmode.c b/drivers/net/wireless/mediatek/mt76/mt7615/testmode.c
index 59d9926..a3d1cfa 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/testmode.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/testmode.c
@@ -91,7 +91,7 @@ mt7615_tm_set_tx_power(struct mt7615_phy *phy)
}
return mt76_mcu_skb_send_msg(&dev->mt76, skb,
- MCU_EXT_CMD_SET_TX_POWER_CTRL, false);
+ MCU_EXT_CMD(SET_TX_POWER_CTRL), false);
}
static void
@@ -185,36 +185,35 @@ mt7615_tm_set_tx_antenna(struct mt7615_phy *phy, bool en)
for (i = 0; i < 4; i++) {
mt76_rmw_field(dev, MT_WF_PHY_RFINTF3_0(i),
MT_WF_PHY_RFINTF3_0_ANT,
- (td->tx_antenna_mask & BIT(i)) ? 0 : 0xa);
-
+ (mask & BIT(i)) ? 0 : 0xa);
}
/* 2.4 GHz band */
mt76_rmw_field(dev, MT_ANT_SWITCH_CON(3), MT_ANT_SWITCH_CON_MODE(0),
- (td->tx_antenna_mask & BIT(0)) ? 0x8 : 0x1b);
+ (mask & BIT(0)) ? 0x8 : 0x1b);
mt76_rmw_field(dev, MT_ANT_SWITCH_CON(4), MT_ANT_SWITCH_CON_MODE(2),
- (td->tx_antenna_mask & BIT(1)) ? 0xe : 0x1b);
+ (mask & BIT(1)) ? 0xe : 0x1b);
mt76_rmw_field(dev, MT_ANT_SWITCH_CON(6), MT_ANT_SWITCH_CON_MODE1(0),
- (td->tx_antenna_mask & BIT(2)) ? 0x0 : 0xf);
+ (mask & BIT(2)) ? 0x0 : 0xf);
mt76_rmw_field(dev, MT_ANT_SWITCH_CON(7), MT_ANT_SWITCH_CON_MODE1(2),
- (td->tx_antenna_mask & BIT(3)) ? 0x6 : 0xf);
+ (mask & BIT(3)) ? 0x6 : 0xf);
/* 5 GHz band */
mt76_rmw_field(dev, MT_ANT_SWITCH_CON(4), MT_ANT_SWITCH_CON_MODE(1),
- (td->tx_antenna_mask & BIT(0)) ? 0xd : 0x1b);
+ (mask & BIT(0)) ? 0xd : 0x1b);
mt76_rmw_field(dev, MT_ANT_SWITCH_CON(2), MT_ANT_SWITCH_CON_MODE(3),
- (td->tx_antenna_mask & BIT(1)) ? 0x13 : 0x1b);
+ (mask & BIT(1)) ? 0x13 : 0x1b);
mt76_rmw_field(dev, MT_ANT_SWITCH_CON(7), MT_ANT_SWITCH_CON_MODE1(1),
- (td->tx_antenna_mask & BIT(2)) ? 0x5 : 0xf);
+ (mask & BIT(2)) ? 0x5 : 0xf);
mt76_rmw_field(dev, MT_ANT_SWITCH_CON(8), MT_ANT_SWITCH_CON_MODE1(3),
- (td->tx_antenna_mask & BIT(3)) ? 0xb : 0xf);
+ (mask & BIT(3)) ? 0xb : 0xf);
for (i = 0; i < 4; i++) {
u32 val;
val = mt7615_rf_rr(dev, i, 0x48);
val &= ~(0x3ff << 20);
- if (td->tx_antenna_mask & BIT(i))
+ if (mask & BIT(i))
val |= 3 << 20;
else
val |= (2 << 28) | (2 << 26) | (8 << 20);
@@ -229,7 +228,7 @@ mt7615_tm_set_tx_frames(struct mt7615_phy *phy, bool en)
struct ieee80211_tx_info *info;
struct sk_buff *skb = phy->mt76->test.tx_skb;
- mt7615_mcu_set_chan_info(phy, MCU_EXT_CMD_SET_RX_PATH);
+ mt7615_mcu_set_chan_info(phy, MCU_EXT_CMD(SET_RX_PATH));
mt7615_tm_set_tx_antenna(phy, en);
mt7615_tm_set_rx_enable(dev, !en);
if (!en || !skb)
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/usb_mcu.c b/drivers/net/wireless/mediatek/mt76/mt7615/usb_mcu.c
index 028ff43..0ebb4c3 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/usb_mcu.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/usb_mcu.c
@@ -21,7 +21,7 @@ mt7663u_mcu_send_message(struct mt76_dev *mdev, struct sk_buff *skb,
int ret, ep, len, pad;
mt7615_mcu_fill_msg(dev, skb, cmd, seq);
- if (cmd != MCU_CMD_FW_SCATTER)
+ if (cmd != MCU_CMD(FW_SCATTER))
ep = MT_EP_OUT_INBAND_CMD;
else
ep = MT_EP_OUT_AC_BE;
diff --git a/drivers/net/wireless/mediatek/mt76/mt76_connac_mac.c b/drivers/net/wireless/mediatek/mt76/mt76_connac_mac.c
index af43bcb..306e9ea 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76_connac_mac.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76_connac_mac.c
@@ -7,9 +7,6 @@ int mt76_connac_pm_wake(struct mt76_phy *phy, struct mt76_connac_pm *pm)
{
struct mt76_dev *dev = phy->dev;
- if (!pm->enable)
- return 0;
-
if (mt76_is_usb(dev))
return 0;
diff --git a/drivers/net/wireless/mediatek/mt76/mt76_connac_mcu.c b/drivers/net/wireless/mediatek/mt76/mt76_connac_mcu.c
index 26b4b87..f79e3d5 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76_connac_mcu.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76_connac_mcu.c
@@ -13,8 +13,8 @@ int mt76_connac_mcu_start_firmware(struct mt76_dev *dev, u32 addr, u32 option)
.addr = cpu_to_le32(addr),
};
- return mt76_mcu_send_msg(dev, MCU_CMD_FW_START_REQ, &req, sizeof(req),
- true);
+ return mt76_mcu_send_msg(dev, MCU_CMD(FW_START_REQ), &req,
+ sizeof(req), true);
}
EXPORT_SYMBOL_GPL(mt76_connac_mcu_start_firmware);
@@ -27,8 +27,8 @@ int mt76_connac_mcu_patch_sem_ctrl(struct mt76_dev *dev, bool get)
.op = cpu_to_le32(op),
};
- return mt76_mcu_send_msg(dev, MCU_CMD_PATCH_SEM_CONTROL, &req,
- sizeof(req), true);
+ return mt76_mcu_send_msg(dev, MCU_CMD(PATCH_SEM_CONTROL),
+ &req, sizeof(req), true);
}
EXPORT_SYMBOL_GPL(mt76_connac_mcu_patch_sem_ctrl);
@@ -41,8 +41,8 @@ int mt76_connac_mcu_start_patch(struct mt76_dev *dev)
.check_crc = 0,
};
- return mt76_mcu_send_msg(dev, MCU_CMD_PATCH_FINISH_REQ, &req,
- sizeof(req), true);
+ return mt76_mcu_send_msg(dev, MCU_CMD(PATCH_FINISH_REQ),
+ &req, sizeof(req), true);
}
EXPORT_SYMBOL_GPL(mt76_connac_mcu_start_patch);
@@ -64,9 +64,9 @@ int mt76_connac_mcu_init_download(struct mt76_dev *dev, u32 addr, u32 len,
if (is_mt7921(dev) &&
(req.addr == cpu_to_le32(MCU_PATCH_ADDRESS) || addr == 0x900000))
- cmd = MCU_CMD_PATCH_START_REQ;
+ cmd = MCU_CMD(PATCH_START_REQ);
else
- cmd = MCU_CMD_TARGET_ADDRESS_LEN_REQ;
+ cmd = MCU_CMD(TARGET_ADDRESS_LEN_REQ);
return mt76_mcu_send_msg(dev, cmd, &req, sizeof(req), true);
}
@@ -160,7 +160,8 @@ int mt76_connac_mcu_set_channel_domain(struct mt76_phy *phy)
memcpy(__skb_push(skb, sizeof(hdr)), &hdr, sizeof(hdr));
- return mt76_mcu_skb_send_msg(dev, skb, MCU_CMD_SET_CHAN_DOMAIN, false);
+ return mt76_mcu_skb_send_msg(dev, skb, MCU_CE_CMD(SET_CHAN_DOMAIN),
+ false);
}
EXPORT_SYMBOL_GPL(mt76_connac_mcu_set_channel_domain);
@@ -176,7 +177,7 @@ int mt76_connac_mcu_set_mac_enable(struct mt76_dev *dev, int band, bool enable,
.band = band,
};
- return mt76_mcu_send_msg(dev, MCU_EXT_CMD_MAC_INIT_CTRL, &req_mac,
+ return mt76_mcu_send_msg(dev, MCU_EXT_CMD(MAC_INIT_CTRL), &req_mac,
sizeof(req_mac), true);
}
EXPORT_SYMBOL_GPL(mt76_connac_mcu_set_mac_enable);
@@ -198,8 +199,8 @@ int mt76_connac_mcu_set_vif_ps(struct mt76_dev *dev, struct ieee80211_vif *vif)
if (vif->type != NL80211_IFTYPE_STATION)
return -EOPNOTSUPP;
- return mt76_mcu_send_msg(dev, MCU_CMD_SET_PS_PROFILE, &req,
- sizeof(req), false);
+ return mt76_mcu_send_msg(dev, MCU_CE_CMD(SET_PS_PROFILE),
+ &req, sizeof(req), false);
}
EXPORT_SYMBOL_GPL(mt76_connac_mcu_set_vif_ps);
@@ -218,7 +219,7 @@ int mt76_connac_mcu_set_rts_thresh(struct mt76_dev *dev, u32 val, u8 band)
.pkt_thresh = cpu_to_le32(0x2),
};
- return mt76_mcu_send_msg(dev, MCU_EXT_CMD_PROTECT_CTRL, &req,
+ return mt76_mcu_send_msg(dev, MCU_EXT_CMD(PROTECT_CTRL), &req,
sizeof(req), true);
}
EXPORT_SYMBOL_GPL(mt76_connac_mcu_set_rts_thresh);
@@ -257,11 +258,8 @@ mt76_connac_mcu_add_nested_tlv(struct sk_buff *skb, int tag, int len,
ntlv = le16_to_cpu(ntlv_hdr->tlv_num);
ntlv_hdr->tlv_num = cpu_to_le16(ntlv + 1);
- if (sta_hdr) {
- u16 size = le16_to_cpu(sta_hdr->len);
-
- sta_hdr->len = cpu_to_le16(size + len);
- }
+ if (sta_hdr)
+ le16_add_cpu(&sta_hdr->len, len);
return ptlv;
}
@@ -1071,7 +1069,7 @@ int mt76_connac_mcu_uni_add_dev(struct mt76_phy *phy,
memcpy(dev_req.tlv.omac_addr, vif->addr, ETH_ALEN);
- cmd = enable ? MCU_UNI_CMD_DEV_INFO_UPDATE : MCU_UNI_CMD_BSS_INFO_UPDATE;
+ cmd = enable ? MCU_UNI_CMD(DEV_INFO_UPDATE) : MCU_UNI_CMD(BSS_INFO_UPDATE);
data = enable ? (void *)&dev_req : (void *)&basic_req;
len = enable ? sizeof(dev_req) : sizeof(basic_req);
@@ -1079,7 +1077,7 @@ int mt76_connac_mcu_uni_add_dev(struct mt76_phy *phy,
if (err < 0)
return err;
- cmd = enable ? MCU_UNI_CMD_BSS_INFO_UPDATE : MCU_UNI_CMD_DEV_INFO_UPDATE;
+ cmd = enable ? MCU_UNI_CMD(BSS_INFO_UPDATE) : MCU_UNI_CMD(DEV_INFO_UPDATE);
data = enable ? (void *)&basic_req : (void *)&dev_req;
len = enable ? sizeof(basic_req) : sizeof(dev_req);
@@ -1131,7 +1129,8 @@ int mt76_connac_mcu_sta_ba(struct mt76_dev *dev, struct mt76_vif *mvif,
mt76_connac_mcu_wtbl_ba_tlv(dev, skb, params, enable, tx, sta_wtbl,
wtbl_hdr);
- ret = mt76_mcu_skb_send_msg(dev, skb, MCU_UNI_CMD_STA_REC_UPDATE, true);
+ ret = mt76_mcu_skb_send_msg(dev, skb,
+ MCU_UNI_CMD(STA_REC_UPDATE), true);
if (ret)
return ret;
@@ -1141,8 +1140,8 @@ int mt76_connac_mcu_sta_ba(struct mt76_dev *dev, struct mt76_vif *mvif,
mt76_connac_mcu_sta_ba_tlv(skb, params, enable, tx);
- return mt76_mcu_skb_send_msg(dev, skb, MCU_UNI_CMD_STA_REC_UPDATE,
- true);
+ return mt76_mcu_skb_send_msg(dev, skb,
+ MCU_UNI_CMD(STA_REC_UPDATE), true);
}
EXPORT_SYMBOL_GPL(mt76_connac_mcu_sta_ba);
@@ -1179,7 +1178,7 @@ mt76_connac_get_phy_mode(struct mt76_phy *phy, struct ieee80211_vif *vif,
if (ht_cap->ht_supported)
mode |= PHY_MODE_GN;
- if (he_cap->has_he)
+ if (he_cap && he_cap->has_he)
mode |= PHY_MODE_AX_24G;
} else if (band == NL80211_BAND_5GHZ || band == NL80211_BAND_6GHZ) {
mode |= PHY_MODE_A;
@@ -1190,12 +1189,8 @@ mt76_connac_get_phy_mode(struct mt76_phy *phy, struct ieee80211_vif *vif,
if (vht_cap->vht_supported)
mode |= PHY_MODE_AC;
- if (he_cap->has_he) {
- if (band == NL80211_BAND_6GHZ)
- mode |= PHY_MODE_AX_6G;
- else
- mode |= PHY_MODE_AX_5G;
- }
+ if (he_cap && he_cap->has_he && band == NL80211_BAND_5GHZ)
+ mode |= PHY_MODE_AX_5G;
}
return mode;
@@ -1318,7 +1313,7 @@ int mt76_connac_mcu_uni_add_bss(struct mt76_phy *phy,
idx = mvif->omac_idx > EXT_BSSID_START ? HW_BSSID_0 : mvif->omac_idx;
basic_req.basic.hw_bss_idx = idx;
if (band == NL80211_BAND_6GHZ)
- basic_req.basic.phymode_ext = BIT(0);
+ basic_req.basic.phymode_ext = PHY_MODE_AX_6G;
basic_phy = mt76_connac_get_phy_mode_v2(phy, vif, band, NULL);
basic_req.basic.nonht_basic_phy = cpu_to_le16(basic_phy);
@@ -1352,7 +1347,7 @@ int mt76_connac_mcu_uni_add_bss(struct mt76_phy *phy,
basic_req.basic.sta_idx = cpu_to_le16(wcid->idx);
basic_req.basic.conn_state = !enable;
- err = mt76_mcu_send_msg(mdev, MCU_UNI_CMD_BSS_INFO_UPDATE, &basic_req,
+ err = mt76_mcu_send_msg(mdev, MCU_UNI_CMD(BSS_INFO_UPDATE), &basic_req,
sizeof(basic_req), true);
if (err < 0)
return err;
@@ -1390,7 +1385,7 @@ int mt76_connac_mcu_uni_add_bss(struct mt76_phy *phy,
mt76_connac_mcu_uni_bss_he_tlv(phy, vif,
(struct tlv *)&he_req.he);
- err = mt76_mcu_send_msg(mdev, MCU_UNI_CMD_BSS_INFO_UPDATE,
+ err = mt76_mcu_send_msg(mdev, MCU_UNI_CMD(BSS_INFO_UPDATE),
&he_req, sizeof(he_req), true);
if (err < 0)
return err;
@@ -1428,7 +1423,7 @@ int mt76_connac_mcu_uni_add_bss(struct mt76_phy *phy,
else if (rlm_req.rlm.control_channel > rlm_req.rlm.center_chan)
rlm_req.rlm.sco = 3; /* SCB */
- return mt76_mcu_send_msg(mdev, MCU_UNI_CMD_BSS_INFO_UPDATE, &rlm_req,
+ return mt76_mcu_send_msg(mdev, MCU_UNI_CMD(BSS_INFO_UPDATE), &rlm_req,
sizeof(rlm_req), true);
}
EXPORT_SYMBOL_GPL(mt76_connac_mcu_uni_add_bss);
@@ -1522,7 +1517,8 @@ int mt76_connac_mcu_hw_scan(struct mt76_phy *phy, struct ieee80211_vif *vif,
req->scan_func |= SCAN_FUNC_RANDOM_MAC;
}
- err = mt76_mcu_skb_send_msg(mdev, skb, MCU_CMD_START_HW_SCAN, false);
+ err = mt76_mcu_skb_send_msg(mdev, skb, MCU_CE_CMD(START_HW_SCAN),
+ false);
if (err < 0)
clear_bit(MT76_HW_SCANNING, &phy->state);
@@ -1550,8 +1546,8 @@ int mt76_connac_mcu_cancel_hw_scan(struct mt76_phy *phy,
ieee80211_scan_completed(phy->hw, &info);
}
- return mt76_mcu_send_msg(phy->dev, MCU_CMD_CANCEL_HW_SCAN, &req,
- sizeof(req), false);
+ return mt76_mcu_send_msg(phy->dev, MCU_CE_CMD(CANCEL_HW_SCAN),
+ &req, sizeof(req), false);
}
EXPORT_SYMBOL_GPL(mt76_connac_mcu_cancel_hw_scan);
@@ -1637,7 +1633,8 @@ int mt76_connac_mcu_sched_scan_req(struct mt76_phy *phy,
memcpy(skb_put(skb, sreq->ie_len), sreq->ie, sreq->ie_len);
}
- return mt76_mcu_skb_send_msg(mdev, skb, MCU_CMD_SCHED_SCAN_REQ, false);
+ return mt76_mcu_skb_send_msg(mdev, skb, MCU_CE_CMD(SCHED_SCAN_REQ),
+ false);
}
EXPORT_SYMBOL_GPL(mt76_connac_mcu_sched_scan_req);
@@ -1657,8 +1654,8 @@ int mt76_connac_mcu_sched_scan_enable(struct mt76_phy *phy,
else
clear_bit(MT76_HW_SCHED_SCANNING, &phy->state);
- return mt76_mcu_send_msg(phy->dev, MCU_CMD_SCHED_SCAN_ENABLE, &req,
- sizeof(req), false);
+ return mt76_mcu_send_msg(phy->dev, MCU_CE_CMD(SCHED_SCAN_ENABLE),
+ &req, sizeof(req), false);
}
EXPORT_SYMBOL_GPL(mt76_connac_mcu_sched_scan_enable);
@@ -1670,8 +1667,8 @@ int mt76_connac_mcu_chip_config(struct mt76_dev *dev)
memcpy(req.data, "assert", 7);
- return mt76_mcu_send_msg(dev, MCU_CMD_CHIP_CONFIG, &req, sizeof(req),
- false);
+ return mt76_mcu_send_msg(dev, MCU_CE_CMD(CHIP_CONFIG),
+ &req, sizeof(req), false);
}
EXPORT_SYMBOL_GPL(mt76_connac_mcu_chip_config);
@@ -1683,8 +1680,8 @@ int mt76_connac_mcu_set_deep_sleep(struct mt76_dev *dev, bool enable)
snprintf(req.data, sizeof(req.data), "KeepFullPwr %d", !enable);
- return mt76_mcu_send_msg(dev, MCU_CMD_CHIP_CONFIG, &req, sizeof(req),
- false);
+ return mt76_mcu_send_msg(dev, MCU_CE_CMD(CHIP_CONFIG),
+ &req, sizeof(req), false);
}
EXPORT_SYMBOL_GPL(mt76_connac_mcu_set_deep_sleep);
@@ -1786,8 +1783,8 @@ int mt76_connac_mcu_get_nic_capability(struct mt76_phy *phy)
struct sk_buff *skb;
int ret, i;
- ret = mt76_mcu_send_and_get_msg(phy->dev, MCU_CMD_GET_NIC_CAPAB, NULL,
- 0, true, &skb);
+ ret = mt76_mcu_send_and_get_msg(phy->dev, MCU_CE_CMD(GET_NIC_CAPAB),
+ NULL, 0, true, &skb);
if (ret)
return ret;
@@ -1885,30 +1882,6 @@ mt76_connac_mcu_build_sku(struct mt76_dev *dev, s8 *sku,
}
}
-static s8 mt76_connac_get_sar_power(struct mt76_phy *phy,
- struct ieee80211_channel *chan,
- s8 target_power)
-{
- const struct cfg80211_sar_capa *capa = phy->hw->wiphy->sar_capa;
- struct mt76_freq_range_power *frp = phy->frp;
- int freq, i;
-
- if (!capa || !frp)
- return target_power;
-
- freq = ieee80211_channel_to_frequency(chan->hw_value, chan->band);
- for (i = 0 ; i < capa->num_freq_ranges; i++) {
- if (frp[i].range &&
- freq >= frp[i].range->start_freq &&
- freq < frp[i].range->end_freq) {
- target_power = min_t(s8, frp[i].power, target_power);
- break;
- }
- }
-
- return target_power;
-}
-
static s8 mt76_connac_get_ch_power(struct mt76_phy *phy,
struct ieee80211_channel *chan,
s8 target_power)
@@ -2008,12 +1981,12 @@ mt76_connac_mcu_rate_txpower_band(struct mt76_phy *phy,
}
batch_size = DIV_ROUND_UP(n_chan, batch_len);
- if (!phy->cap.has_5ghz)
- last_ch = chan_list_2ghz[n_chan - 1];
- else if (phy->cap.has_6ghz)
- last_ch = chan_list_6ghz[n_chan - 1];
+ if (phy->cap.has_6ghz)
+ last_ch = chan_list_6ghz[ARRAY_SIZE(chan_list_6ghz) - 1];
+ else if (phy->cap.has_5ghz)
+ last_ch = chan_list_5ghz[ARRAY_SIZE(chan_list_5ghz) - 1];
else
- last_ch = chan_list_5ghz[n_chan - 1];
+ last_ch = chan_list_2ghz[ARRAY_SIZE(chan_list_2ghz) - 1];
for (i = 0; i < batch_size; i++) {
struct mt76_connac_tx_power_limit_tlv tx_power_tlv = {};
@@ -2053,8 +2026,7 @@ mt76_connac_mcu_rate_txpower_band(struct mt76_phy *phy,
reg_power = mt76_connac_get_ch_power(phy, &chan,
tx_power);
- sar_power = mt76_connac_get_sar_power(phy, &chan,
- reg_power);
+ sar_power = mt76_get_sar_power(phy, &chan, reg_power);
mt76_get_rate_power_limits(phy, &chan, &limits,
sar_power);
@@ -2070,7 +2042,8 @@ mt76_connac_mcu_rate_txpower_band(struct mt76_phy *phy,
memcpy(skb->data, &tx_power_tlv, sizeof(tx_power_tlv));
err = mt76_mcu_skb_send_msg(dev, skb,
- MCU_CMD_SET_RATE_TX_POWER, false);
+ MCU_CE_CMD(SET_RATE_TX_POWER),
+ false);
if (err < 0)
return err;
}
@@ -2143,7 +2116,7 @@ int mt76_connac_mcu_update_arp_filter(struct mt76_dev *dev,
memcpy(addr, &info->arp_addr_list[i], sizeof(__be32));
}
- return mt76_mcu_skb_send_msg(dev, skb, MCU_UNI_CMD_OFFLOAD, true);
+ return mt76_mcu_skb_send_msg(dev, skb, MCU_UNI_CMD(OFFLOAD), true);
}
EXPORT_SYMBOL_GPL(mt76_connac_mcu_update_arp_filter);
@@ -2162,8 +2135,8 @@ int mt76_connac_mcu_set_p2p_oppps(struct ieee80211_hw *hw,
.bss_idx = mvif->idx,
};
- return mt76_mcu_send_msg(phy->dev, MCU_CMD_SET_P2P_OPPPS, &req,
- sizeof(req), false);
+ return mt76_mcu_send_msg(phy->dev, MCU_CE_CMD(SET_P2P_OPPPS),
+ &req, sizeof(req), false);
}
EXPORT_SYMBOL_GPL(mt76_connac_mcu_set_p2p_oppps);
@@ -2249,7 +2222,8 @@ int mt76_connac_mcu_update_gtk_rekey(struct ieee80211_hw *hw,
memcpy(gtk_tlv->kck, key->kck, NL80211_KCK_LEN);
memcpy(gtk_tlv->replay_ctr, key->replay_ctr, NL80211_REPLAY_CTR_LEN);
- return mt76_mcu_skb_send_msg(phy->dev, skb, MCU_UNI_CMD_OFFLOAD, true);
+ return mt76_mcu_skb_send_msg(phy->dev, skb,
+ MCU_UNI_CMD(OFFLOAD), true);
}
EXPORT_SYMBOL_GPL(mt76_connac_mcu_update_gtk_rekey);
@@ -2275,8 +2249,8 @@ mt76_connac_mcu_set_arp_filter(struct mt76_dev *dev, struct ieee80211_vif *vif,
},
};
- return mt76_mcu_send_msg(dev, MCU_UNI_CMD_OFFLOAD, &req, sizeof(req),
- true);
+ return mt76_mcu_send_msg(dev, MCU_UNI_CMD(OFFLOAD), &req,
+ sizeof(req), true);
}
static int
@@ -2301,8 +2275,8 @@ mt76_connac_mcu_set_gtk_rekey(struct mt76_dev *dev, struct ieee80211_vif *vif,
},
};
- return mt76_mcu_send_msg(dev, MCU_UNI_CMD_OFFLOAD, &req, sizeof(req),
- true);
+ return mt76_mcu_send_msg(dev, MCU_UNI_CMD(OFFLOAD), &req,
+ sizeof(req), true);
}
static int
@@ -2331,8 +2305,8 @@ mt76_connac_mcu_set_suspend_mode(struct mt76_dev *dev,
},
};
- return mt76_mcu_send_msg(dev, MCU_UNI_CMD_SUSPEND, &req, sizeof(req),
- true);
+ return mt76_mcu_send_msg(dev, MCU_UNI_CMD(SUSPEND), &req,
+ sizeof(req), true);
}
static int
@@ -2366,7 +2340,7 @@ mt76_connac_mcu_set_wow_pattern(struct mt76_dev *dev,
memcpy(ptlv->pattern, pattern->pattern, pattern->pattern_len);
memcpy(ptlv->mask, pattern->mask, DIV_ROUND_UP(pattern->pattern_len, 8));
- return mt76_mcu_skb_send_msg(dev, skb, MCU_UNI_CMD_SUSPEND, true);
+ return mt76_mcu_skb_send_msg(dev, skb, MCU_UNI_CMD(SUSPEND), true);
}
static int
@@ -2418,8 +2392,8 @@ mt76_connac_mcu_set_wow_ctrl(struct mt76_phy *phy, struct ieee80211_vif *vif,
else if (mt76_is_sdio(dev))
req.wow_ctrl_tlv.wakeup_hif = WOW_GPIO;
- return mt76_mcu_send_msg(dev, MCU_UNI_CMD_SUSPEND, &req, sizeof(req),
- true);
+ return mt76_mcu_send_msg(dev, MCU_UNI_CMD(SUSPEND), &req,
+ sizeof(req), true);
}
int mt76_connac_mcu_set_hif_suspend(struct mt76_dev *dev, bool suspend)
@@ -2452,8 +2426,8 @@ int mt76_connac_mcu_set_hif_suspend(struct mt76_dev *dev, bool suspend)
else if (mt76_is_sdio(dev))
req.hdr.hif_type = 0;
- return mt76_mcu_send_msg(dev, MCU_UNI_CMD_HIF_CTRL, &req, sizeof(req),
- true);
+ return mt76_mcu_send_msg(dev, MCU_UNI_CMD(HIF_CTRL), &req,
+ sizeof(req), true);
}
EXPORT_SYMBOL_GPL(mt76_connac_mcu_set_hif_suspend);
@@ -2461,7 +2435,7 @@ void mt76_connac_mcu_set_suspend_iter(void *priv, u8 *mac,
struct ieee80211_vif *vif)
{
struct mt76_phy *phy = priv;
- bool suspend = test_bit(MT76_STATE_SUSPEND, &phy->state);
+ bool suspend = !test_bit(MT76_STATE_RUNNING, &phy->state);
struct ieee80211_hw *hw = phy->hw;
struct cfg80211_wowlan *wowlan = hw->wiphy->wowlan_config;
int i;
@@ -2488,8 +2462,8 @@ u32 mt76_connac_mcu_reg_rr(struct mt76_dev *dev, u32 offset)
.addr = cpu_to_le32(offset),
};
- return mt76_mcu_send_msg(dev, MCU_CMD_REG_READ, &req, sizeof(req),
- true);
+ return mt76_mcu_send_msg(dev, MCU_CE_QUERY(REG_READ), &req,
+ sizeof(req), true);
}
EXPORT_SYMBOL_GPL(mt76_connac_mcu_reg_rr);
@@ -2503,7 +2477,8 @@ void mt76_connac_mcu_reg_wr(struct mt76_dev *dev, u32 offset, u32 val)
.val = cpu_to_le32(val),
};
- mt76_mcu_send_msg(dev, MCU_CMD_REG_WRITE, &req, sizeof(req), false);
+ mt76_mcu_send_msg(dev, MCU_CE_CMD(REG_WRITE), &req,
+ sizeof(req), false);
}
EXPORT_SYMBOL_GPL(mt76_connac_mcu_reg_wr);
diff --git a/drivers/net/wireless/mediatek/mt76/mt76_connac_mcu.h b/drivers/net/wireless/mediatek/mt76/mt76_connac_mcu.h
index 4e2c9daf..5baf837 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76_connac_mcu.h
+++ b/drivers/net/wireless/mediatek/mt76/mt76_connac_mcu.h
@@ -11,6 +11,76 @@ struct tlv {
__le16 len;
} __packed;
+struct bss_info_omac {
+ __le16 tag;
+ __le16 len;
+ u8 hw_bss_idx;
+ u8 omac_idx;
+ u8 band_idx;
+ u8 rsv0;
+ __le32 conn_type;
+ u32 rsv1;
+} __packed;
+
+struct bss_info_basic {
+ __le16 tag;
+ __le16 len;
+ __le32 network_type;
+ u8 active;
+ u8 rsv0;
+ __le16 bcn_interval;
+ u8 bssid[ETH_ALEN];
+ u8 wmm_idx;
+ u8 dtim_period;
+ u8 bmc_wcid_lo;
+ u8 cipher;
+ u8 phy_mode;
+ u8 max_bssid; /* max BSSID. range: 1 ~ 8, 0: MBSSID disabled */
+ u8 non_tx_bssid;/* non-transmitted BSSID, 0: transmitted BSSID */
+ u8 bmc_wcid_hi; /* high Byte and version */
+ u8 rsv[2];
+} __packed;
+
+struct bss_info_rf_ch {
+ __le16 tag;
+ __le16 len;
+ u8 pri_ch;
+ u8 center_ch0;
+ u8 center_ch1;
+ u8 bw;
+ u8 he_ru26_block; /* 1: don't send HETB in RU26, 0: allow */
+ u8 he_all_disable; /* 1: disallow all HETB, 0: allow */
+ u8 rsv[2];
+} __packed;
+
+struct bss_info_ext_bss {
+ __le16 tag;
+ __le16 len;
+ __le32 mbss_tsf_offset; /* in unit of us */
+ u8 rsv[8];
+} __packed;
+
+enum {
+ BSS_INFO_OMAC,
+ BSS_INFO_BASIC,
+ BSS_INFO_RF_CH, /* optional, for BT/LTE coex */
+ BSS_INFO_PM, /* sta only */
+ BSS_INFO_UAPSD, /* sta only */
+ BSS_INFO_ROAM_DETECT, /* obsoleted */
+ BSS_INFO_LQ_RM, /* obsoleted */
+ BSS_INFO_EXT_BSS,
+ BSS_INFO_BMC_RATE, /* for bmc rate control in CR4 */
+ BSS_INFO_SYNC_MODE, /* obsoleted */
+ BSS_INFO_RA,
+ BSS_INFO_HW_AMSDU,
+ BSS_INFO_BSS_COLOR,
+ BSS_INFO_HE_BASIC,
+ BSS_INFO_PROTECT_INFO,
+ BSS_INFO_OFFLOAD,
+ BSS_INFO_11V_MBSSID,
+ BSS_INFO_MAX_NUM
+};
+
/* sta_rec */
struct sta_ntlv_hdr {
@@ -54,7 +124,7 @@ struct sta_rec_vht {
__le32 vht_cap;
__le16 vht_rx_mcs_map;
__le16 vht_tx_mcs_map;
- /* mt7921 */
+ /* mt7915 - mt7921 */
u8 rts_bw_sig;
u8 rsv[3];
} __packed;
@@ -152,6 +222,191 @@ struct sta_rec_he_6g_capa {
u8 rsv[2];
} __packed;
+struct sec_key {
+ u8 cipher_id;
+ u8 cipher_len;
+ u8 key_id;
+ u8 key_len;
+ u8 key[32];
+} __packed;
+
+struct sta_rec_sec {
+ __le16 tag;
+ __le16 len;
+ u8 add;
+ u8 n_cipher;
+ u8 rsv[2];
+
+ struct sec_key key[2];
+} __packed;
+
+struct sta_rec_bf {
+ __le16 tag;
+ __le16 len;
+
+ __le16 pfmu; /* 0xffff: no access right for PFMU */
+ bool su_mu; /* 0: SU, 1: MU */
+ u8 bf_cap; /* 0: iBF, 1: eBF */
+ u8 sounding_phy; /* 0: legacy, 1: OFDM, 2: HT, 4: VHT */
+ u8 ndpa_rate;
+ u8 ndp_rate;
+ u8 rept_poll_rate;
+ u8 tx_mode; /* 0: legacy, 1: OFDM, 2: HT, 4: VHT ... */
+ u8 ncol;
+ u8 nrow;
+ u8 bw; /* 0: 20M, 1: 40M, 2: 80M, 3: 160M */
+
+ u8 mem_total;
+ u8 mem_20m;
+ struct {
+ u8 row;
+ u8 col: 6, row_msb: 2;
+ } mem[4];
+
+ __le16 smart_ant;
+ u8 se_idx;
+ u8 auto_sounding; /* b7: low traffic indicator
+ * b6: Stop sounding for this entry
+ * b5 ~ b0: postpone sounding
+ */
+ u8 ibf_timeout;
+ u8 ibf_dbw;
+ u8 ibf_ncol;
+ u8 ibf_nrow;
+ u8 nrow_bw160;
+ u8 ncol_bw160;
+ u8 ru_start_idx;
+ u8 ru_end_idx;
+
+ bool trigger_su;
+ bool trigger_mu;
+ bool ng16_su;
+ bool ng16_mu;
+ bool codebook42_su;
+ bool codebook75_mu;
+
+ u8 he_ltf;
+ u8 rsv[3];
+} __packed;
+
+struct sta_rec_bfee {
+ __le16 tag;
+ __le16 len;
+ bool fb_identity_matrix; /* 1: feedback identity matrix */
+ bool ignore_feedback; /* 1: ignore */
+ u8 rsv[2];
+} __packed;
+
+struct sta_rec_muru {
+ __le16 tag;
+ __le16 len;
+
+ struct {
+ bool ofdma_dl_en;
+ bool ofdma_ul_en;
+ bool mimo_dl_en;
+ bool mimo_ul_en;
+ u8 rsv[4];
+ } cfg;
+
+ struct {
+ u8 punc_pream_rx;
+ bool he_20m_in_40m_2g;
+ bool he_20m_in_160m;
+ bool he_80m_in_160m;
+ bool lt16_sigb;
+ bool rx_su_comp_sigb;
+ bool rx_su_non_comp_sigb;
+ u8 rsv;
+ } ofdma_dl;
+
+ struct {
+ u8 t_frame_dur;
+ u8 mu_cascading;
+ u8 uo_ra;
+ u8 he_2x996_tone;
+ u8 rx_t_frame_11ac;
+ u8 rsv[3];
+ } ofdma_ul;
+
+ struct {
+ bool vht_mu_bfee;
+ bool partial_bw_dl_mimo;
+ u8 rsv[2];
+ } mimo_dl;
+
+ struct {
+ bool full_ul_mimo;
+ bool partial_ul_mimo;
+ u8 rsv[2];
+ } mimo_ul;
+} __packed;
+
+struct sta_phy {
+ u8 type;
+ u8 flag;
+ u8 stbc;
+ u8 sgi;
+ u8 bw;
+ u8 ldpc;
+ u8 mcs;
+ u8 nss;
+ u8 he_ltf;
+};
+
+struct sta_rec_ra {
+ __le16 tag;
+ __le16 len;
+
+ u8 valid;
+ u8 auto_rate;
+ u8 phy_mode;
+ u8 channel;
+ u8 bw;
+ u8 disable_cck;
+ u8 ht_mcs32;
+ u8 ht_gf;
+ u8 ht_mcs[4];
+ u8 mmps_mode;
+ u8 gband_256;
+ u8 af;
+ u8 auth_wapi_mode;
+ u8 rate_len;
+
+ u8 supp_mode;
+ u8 supp_cck_rate;
+ u8 supp_ofdm_rate;
+ __le32 supp_ht_mcs;
+ __le16 supp_vht_mcs[4];
+
+ u8 op_mode;
+ u8 op_vht_chan_width;
+ u8 op_vht_rx_nss;
+ u8 op_vht_rx_nss_type;
+
+ __le32 sta_cap;
+
+ struct sta_phy phy;
+} __packed;
+
+struct sta_rec_ra_fixed {
+ __le16 tag;
+ __le16 len;
+
+ __le32 field;
+ u8 op_mode;
+ u8 op_vht_chan_width;
+ u8 op_vht_rx_nss;
+ u8 op_vht_rx_nss_type;
+
+ struct sta_phy phy;
+
+ u8 spe_en;
+ u8 short_preamble;
+ u8 is_5g;
+ u8 mmps_mode;
+} __packed;
+
/* wtbl_rec */
struct wtbl_req_hdr {
@@ -234,6 +489,7 @@ struct wtbl_ba {
__le16 sn;
u8 ba_en;
u8 ba_winsize_idx;
+ /* originator & recipient */
__le16 ba_winsize;
/* recipient only */
u8 peer_addr[ETH_ALEN];
@@ -304,12 +560,17 @@ struct wtbl_raw {
#define MT76_CONNAC_STA_UPDATE_MAX_SIZE (sizeof(struct sta_req_hdr) + \
sizeof(struct sta_rec_basic) + \
+ sizeof(struct sta_rec_bf) + \
sizeof(struct sta_rec_ht) + \
sizeof(struct sta_rec_he) + \
sizeof(struct sta_rec_ba) + \
sizeof(struct sta_rec_vht) + \
sizeof(struct sta_rec_uapsd) + \
sizeof(struct sta_rec_amsdu) + \
+ sizeof(struct sta_rec_muru) + \
+ sizeof(struct sta_rec_bfee) + \
+ sizeof(struct sta_rec_ra) + \
+ sizeof(struct sta_rec_ra_fixed) + \
sizeof(struct sta_rec_he_6g_capa) + \
sizeof(struct tlv) + \
MT76_CONNAC_WTBL_UPDATE_MAX_SIZE)
@@ -423,7 +684,8 @@ enum {
#define PHY_MODE_AC BIT(5)
#define PHY_MODE_AX_24G BIT(6)
#define PHY_MODE_AX_5G BIT(7)
-#define PHY_MODE_AX_6G BIT(8)
+
+#define PHY_MODE_AX_6G BIT(0) /* phymode_ext */
#define MODE_CCK BIT(0)
#define MODE_OFDM BIT(1)
@@ -431,6 +693,21 @@ enum {
#define MODE_VHT BIT(3)
#define MODE_HE BIT(4)
+#define STA_CAP_WMM BIT(0)
+#define STA_CAP_SGI_20 BIT(4)
+#define STA_CAP_SGI_40 BIT(5)
+#define STA_CAP_TX_STBC BIT(6)
+#define STA_CAP_RX_STBC BIT(7)
+#define STA_CAP_VHT_SGI_80 BIT(16)
+#define STA_CAP_VHT_SGI_160 BIT(17)
+#define STA_CAP_VHT_TX_STBC BIT(18)
+#define STA_CAP_VHT_RX_STBC BIT(19)
+#define STA_CAP_VHT_LDPC BIT(23)
+#define STA_CAP_LDPC BIT(24)
+#define STA_CAP_HT BIT(26)
+#define STA_CAP_VHT BIT(27)
+#define STA_CAP_HE BIT(28)
+
enum {
PHY_TYPE_HR_DSSS_INDEX = 0,
PHY_TYPE_ERP_INDEX,
@@ -489,6 +766,121 @@ enum {
DEV_INFO_MAX_NUM
};
+/* event table */
+enum {
+ MCU_EVENT_TARGET_ADDRESS_LEN = 0x01,
+ MCU_EVENT_FW_START = 0x01,
+ MCU_EVENT_GENERIC = 0x01,
+ MCU_EVENT_ACCESS_REG = 0x02,
+ MCU_EVENT_MT_PATCH_SEM = 0x04,
+ MCU_EVENT_REG_ACCESS = 0x05,
+ MCU_EVENT_LP_INFO = 0x07,
+ MCU_EVENT_SCAN_DONE = 0x0d,
+ MCU_EVENT_TX_DONE = 0x0f,
+ MCU_EVENT_ROC = 0x10,
+ MCU_EVENT_BSS_ABSENCE = 0x11,
+ MCU_EVENT_BSS_BEACON_LOSS = 0x13,
+ MCU_EVENT_CH_PRIVILEGE = 0x18,
+ MCU_EVENT_SCHED_SCAN_DONE = 0x23,
+ MCU_EVENT_DBG_MSG = 0x27,
+ MCU_EVENT_TXPWR = 0xd0,
+ MCU_EVENT_EXT = 0xed,
+ MCU_EVENT_RESTART_DL = 0xef,
+ MCU_EVENT_COREDUMP = 0xf0,
+};
+
+/* ext event table */
+enum {
+ MCU_EXT_EVENT_PS_SYNC = 0x5,
+ MCU_EXT_EVENT_FW_LOG_2_HOST = 0x13,
+ MCU_EXT_EVENT_THERMAL_PROTECT = 0x22,
+ MCU_EXT_EVENT_ASSERT_DUMP = 0x23,
+ MCU_EXT_EVENT_RDD_REPORT = 0x3a,
+ MCU_EXT_EVENT_CSA_NOTIFY = 0x4f,
+ MCU_EXT_EVENT_BCC_NOTIFY = 0x75,
+ MCU_EXT_EVENT_MURU_CTRL = 0x9f,
+};
+
+enum {
+ MCU_Q_QUERY,
+ MCU_Q_SET,
+ MCU_Q_RESERVED,
+ MCU_Q_NA
+};
+
+enum {
+ MCU_S2D_H2N,
+ MCU_S2D_C2N,
+ MCU_S2D_H2C,
+ MCU_S2D_H2CN
+};
+
+enum {
+ PATCH_NOT_DL_SEM_FAIL,
+ PATCH_IS_DL,
+ PATCH_NOT_DL_SEM_SUCCESS,
+ PATCH_REL_SEM_SUCCESS
+};
+
+enum {
+ FW_STATE_INITIAL,
+ FW_STATE_FW_DOWNLOAD,
+ FW_STATE_NORMAL_OPERATION,
+ FW_STATE_NORMAL_TRX,
+ FW_STATE_RDY = 7
+};
+
+enum {
+ CH_SWITCH_NORMAL = 0,
+ CH_SWITCH_SCAN = 3,
+ CH_SWITCH_MCC = 4,
+ CH_SWITCH_DFS = 5,
+ CH_SWITCH_BACKGROUND_SCAN_START = 6,
+ CH_SWITCH_BACKGROUND_SCAN_RUNNING = 7,
+ CH_SWITCH_BACKGROUND_SCAN_STOP = 8,
+ CH_SWITCH_SCAN_BYPASS_DPD = 9
+};
+
+enum {
+ THERMAL_SENSOR_TEMP_QUERY,
+ THERMAL_SENSOR_MANUAL_CTRL,
+ THERMAL_SENSOR_INFO_QUERY,
+ THERMAL_SENSOR_TASK_CTRL,
+};
+
+enum mcu_cipher_type {
+ MCU_CIPHER_NONE = 0,
+ MCU_CIPHER_WEP40,
+ MCU_CIPHER_WEP104,
+ MCU_CIPHER_WEP128,
+ MCU_CIPHER_TKIP,
+ MCU_CIPHER_AES_CCMP,
+ MCU_CIPHER_CCMP_256,
+ MCU_CIPHER_GCMP,
+ MCU_CIPHER_GCMP_256,
+ MCU_CIPHER_WAPI,
+ MCU_CIPHER_BIP_CMAC_128,
+};
+
+enum {
+ EE_MODE_EFUSE,
+ EE_MODE_BUFFER,
+};
+
+enum {
+ EE_FORMAT_BIN,
+ EE_FORMAT_WHOLE,
+ EE_FORMAT_MULTIPLE,
+};
+
+enum {
+ MCU_PHY_STATE_TX_RATE,
+ MCU_PHY_STATE_RX_RATE,
+ MCU_PHY_STATE_RSSI,
+ MCU_PHY_STATE_CONTENTION_RX_RATE,
+ MCU_PHY_STATE_OFDMLQ_CNINFO,
+};
+
#define MCU_CMD_ACK BIT(0)
#define MCU_CMD_UNI BIT(1)
#define MCU_CMD_QUERY BIT(2)
@@ -496,29 +888,51 @@ enum {
#define MCU_CMD_UNI_EXT_ACK (MCU_CMD_ACK | MCU_CMD_UNI | \
MCU_CMD_QUERY)
-#define MCU_FW_PREFIX BIT(31)
-#define MCU_UNI_PREFIX BIT(30)
-#define MCU_CE_PREFIX BIT(29)
-#define MCU_QUERY_PREFIX BIT(28)
-#define MCU_CMD_MASK ~(MCU_FW_PREFIX | MCU_UNI_PREFIX | \
- MCU_CE_PREFIX | MCU_QUERY_PREFIX)
+#define __MCU_CMD_FIELD_ID GENMASK(7, 0)
+#define __MCU_CMD_FIELD_EXT_ID GENMASK(15, 8)
+#define __MCU_CMD_FIELD_QUERY BIT(16)
+#define __MCU_CMD_FIELD_UNI BIT(17)
+#define __MCU_CMD_FIELD_CE BIT(18)
+#define __MCU_CMD_FIELD_WA BIT(19)
-#define MCU_QUERY_MASK BIT(16)
+#define MCU_CMD(_t) FIELD_PREP(__MCU_CMD_FIELD_ID, \
+ MCU_CMD_##_t)
+#define MCU_EXT_CMD(_t) (MCU_CMD(EXT_CID) | \
+ FIELD_PREP(__MCU_CMD_FIELD_EXT_ID, \
+ MCU_EXT_CMD_##_t))
+#define MCU_EXT_QUERY(_t) (MCU_EXT_CMD(_t) | __MCU_CMD_FIELD_QUERY)
+#define MCU_UNI_CMD(_t) (__MCU_CMD_FIELD_UNI | \
+ FIELD_PREP(__MCU_CMD_FIELD_ID, \
+ MCU_UNI_CMD_##_t))
+#define MCU_CE_CMD(_t) (__MCU_CMD_FIELD_CE | \
+ FIELD_PREP(__MCU_CMD_FIELD_ID, \
+ MCU_CE_CMD_##_t))
+#define MCU_CE_QUERY(_t) (MCU_CE_CMD(_t) | __MCU_CMD_FIELD_QUERY)
+
+#define MCU_WA_CMD(_t) (MCU_CMD(_t) | __MCU_CMD_FIELD_WA)
+#define MCU_WA_EXT_CMD(_t) (MCU_EXT_CMD(_t) | __MCU_CMD_FIELD_WA)
+#define MCU_WA_PARAM_CMD(_t) (MCU_WA_CMD(WA_PARAM) | \
+ FIELD_PREP(__MCU_CMD_FIELD_EXT_ID, \
+ MCU_WA_PARAM_CMD_##_t))
enum {
MCU_EXT_CMD_EFUSE_ACCESS = 0x01,
MCU_EXT_CMD_RF_REG_ACCESS = 0x02,
+ MCU_EXT_CMD_RF_TEST = 0x04,
MCU_EXT_CMD_PM_STATE_CTRL = 0x07,
MCU_EXT_CMD_CHANNEL_SWITCH = 0x08,
MCU_EXT_CMD_SET_TX_POWER_CTRL = 0x11,
MCU_EXT_CMD_FW_LOG_2_HOST = 0x13,
+ MCU_EXT_CMD_TXBF_ACTION = 0x1e,
MCU_EXT_CMD_EFUSE_BUFFER_MODE = 0x21,
+ MCU_EXT_CMD_THERMAL_PROT = 0x23,
MCU_EXT_CMD_STA_REC_UPDATE = 0x25,
MCU_EXT_CMD_BSS_INFO_UPDATE = 0x26,
MCU_EXT_CMD_EDCA_UPDATE = 0x27,
MCU_EXT_CMD_DEV_INFO_UPDATE = 0x2A,
- MCU_EXT_CMD_GET_TEMP = 0x2c,
+ MCU_EXT_CMD_THERMAL_CTRL = 0x2c,
MCU_EXT_CMD_WTBL_UPDATE = 0x32,
+ MCU_EXT_CMD_SET_DRR_CTRL = 0x36,
MCU_EXT_CMD_SET_RDD_CTRL = 0x3a,
MCU_EXT_CMD_ATE_CTRL = 0x3d,
MCU_EXT_CMD_PROTECT_CTRL = 0x3e,
@@ -527,59 +941,74 @@ enum {
MCU_EXT_CMD_RX_HDR_TRANS = 0x47,
MCU_EXT_CMD_MUAR_UPDATE = 0x48,
MCU_EXT_CMD_BCN_OFFLOAD = 0x49,
+ MCU_EXT_CMD_RX_AIRTIME_CTRL = 0x4a,
MCU_EXT_CMD_SET_RX_PATH = 0x4e,
+ MCU_EXT_CMD_EFUSE_FREE_BLOCK = 0x4f,
MCU_EXT_CMD_TX_POWER_FEATURE_CTRL = 0x58,
MCU_EXT_CMD_RXDCOC_CAL = 0x59,
+ MCU_EXT_CMD_GET_MIB_INFO = 0x5a,
MCU_EXT_CMD_TXDPD_CAL = 0x60,
MCU_EXT_CMD_CAL_CACHE = 0x67,
- MCU_EXT_CMD_SET_RDD_TH = 0x7c,
+ MCU_EXT_CMD_SET_RADAR_TH = 0x7c,
MCU_EXT_CMD_SET_RDD_PATTERN = 0x7d,
+ MCU_EXT_CMD_MWDS_SUPPORT = 0x80,
+ MCU_EXT_CMD_SET_SER_TRIGGER = 0x81,
+ MCU_EXT_CMD_SCS_CTRL = 0x82,
+ MCU_EXT_CMD_TWT_AGRT_UPDATE = 0x94,
+ MCU_EXT_CMD_FW_DBG_CTRL = 0x95,
+ MCU_EXT_CMD_SET_RDD_TH = 0x9d,
+ MCU_EXT_CMD_MURU_CTRL = 0x9f,
+ MCU_EXT_CMD_SET_SPR = 0xa8,
+ MCU_EXT_CMD_GROUP_PRE_CAL_INFO = 0xab,
+ MCU_EXT_CMD_DPD_PRE_CAL_INFO = 0xac,
+ MCU_EXT_CMD_PHY_STAT_INFO = 0xad,
};
enum {
- MCU_UNI_CMD_DEV_INFO_UPDATE = MCU_UNI_PREFIX | 0x01,
- MCU_UNI_CMD_BSS_INFO_UPDATE = MCU_UNI_PREFIX | 0x02,
- MCU_UNI_CMD_STA_REC_UPDATE = MCU_UNI_PREFIX | 0x03,
- MCU_UNI_CMD_SUSPEND = MCU_UNI_PREFIX | 0x05,
- MCU_UNI_CMD_OFFLOAD = MCU_UNI_PREFIX | 0x06,
- MCU_UNI_CMD_HIF_CTRL = MCU_UNI_PREFIX | 0x07,
+ MCU_UNI_CMD_DEV_INFO_UPDATE = 0x01,
+ MCU_UNI_CMD_BSS_INFO_UPDATE = 0x02,
+ MCU_UNI_CMD_STA_REC_UPDATE = 0x03,
+ MCU_UNI_CMD_SUSPEND = 0x05,
+ MCU_UNI_CMD_OFFLOAD = 0x06,
+ MCU_UNI_CMD_HIF_CTRL = 0x07,
};
enum {
- MCU_CMD_TARGET_ADDRESS_LEN_REQ = MCU_FW_PREFIX | 0x01,
- MCU_CMD_FW_START_REQ = MCU_FW_PREFIX | 0x02,
+ MCU_CMD_TARGET_ADDRESS_LEN_REQ = 0x01,
+ MCU_CMD_FW_START_REQ = 0x02,
MCU_CMD_INIT_ACCESS_REG = 0x3,
- MCU_CMD_NIC_POWER_CTRL = MCU_FW_PREFIX | 0x4,
- MCU_CMD_PATCH_START_REQ = MCU_FW_PREFIX | 0x05,
- MCU_CMD_PATCH_FINISH_REQ = MCU_FW_PREFIX | 0x07,
- MCU_CMD_PATCH_SEM_CONTROL = MCU_FW_PREFIX | 0x10,
+ MCU_CMD_NIC_POWER_CTRL = 0x4,
+ MCU_CMD_PATCH_START_REQ = 0x05,
+ MCU_CMD_PATCH_FINISH_REQ = 0x07,
+ MCU_CMD_PATCH_SEM_CONTROL = 0x10,
+ MCU_CMD_WA_PARAM = 0xc4,
MCU_CMD_EXT_CID = 0xed,
- MCU_CMD_FW_SCATTER = MCU_FW_PREFIX | 0xee,
- MCU_CMD_RESTART_DL_REQ = MCU_FW_PREFIX | 0xef,
+ MCU_CMD_FW_SCATTER = 0xee,
+ MCU_CMD_RESTART_DL_REQ = 0xef,
};
/* offload mcu commands */
enum {
- MCU_CMD_TEST_CTRL = MCU_CE_PREFIX | 0x01,
- MCU_CMD_START_HW_SCAN = MCU_CE_PREFIX | 0x03,
- MCU_CMD_SET_PS_PROFILE = MCU_CE_PREFIX | 0x05,
- MCU_CMD_SET_CHAN_DOMAIN = MCU_CE_PREFIX | 0x0f,
- MCU_CMD_SET_BSS_CONNECTED = MCU_CE_PREFIX | 0x16,
- MCU_CMD_SET_BSS_ABORT = MCU_CE_PREFIX | 0x17,
- MCU_CMD_CANCEL_HW_SCAN = MCU_CE_PREFIX | 0x1b,
- MCU_CMD_SET_ROC = MCU_CE_PREFIX | 0x1d,
- MCU_CMD_SET_P2P_OPPPS = MCU_CE_PREFIX | 0x33,
- MCU_CMD_SET_RATE_TX_POWER = MCU_CE_PREFIX | 0x5d,
- MCU_CMD_SCHED_SCAN_ENABLE = MCU_CE_PREFIX | 0x61,
- MCU_CMD_SCHED_SCAN_REQ = MCU_CE_PREFIX | 0x62,
- MCU_CMD_GET_NIC_CAPAB = MCU_CE_PREFIX | 0x8a,
- MCU_CMD_SET_MU_EDCA_PARMS = MCU_CE_PREFIX | 0xb0,
- MCU_CMD_REG_WRITE = MCU_CE_PREFIX | 0xc0,
- MCU_CMD_REG_READ = MCU_CE_PREFIX | MCU_QUERY_MASK | 0xc0,
- MCU_CMD_CHIP_CONFIG = MCU_CE_PREFIX | 0xca,
- MCU_CMD_FWLOG_2_HOST = MCU_CE_PREFIX | 0xc5,
- MCU_CMD_GET_WTBL = MCU_CE_PREFIX | 0xcd,
- MCU_CMD_GET_TXPWR = MCU_CE_PREFIX | 0xd0,
+ MCU_CE_CMD_TEST_CTRL = 0x01,
+ MCU_CE_CMD_START_HW_SCAN = 0x03,
+ MCU_CE_CMD_SET_PS_PROFILE = 0x05,
+ MCU_CE_CMD_SET_CHAN_DOMAIN = 0x0f,
+ MCU_CE_CMD_SET_BSS_CONNECTED = 0x16,
+ MCU_CE_CMD_SET_BSS_ABORT = 0x17,
+ MCU_CE_CMD_CANCEL_HW_SCAN = 0x1b,
+ MCU_CE_CMD_SET_ROC = 0x1d,
+ MCU_CE_CMD_SET_P2P_OPPPS = 0x33,
+ MCU_CE_CMD_SET_RATE_TX_POWER = 0x5d,
+ MCU_CE_CMD_SCHED_SCAN_ENABLE = 0x61,
+ MCU_CE_CMD_SCHED_SCAN_REQ = 0x62,
+ MCU_CE_CMD_GET_NIC_CAPAB = 0x8a,
+ MCU_CE_CMD_SET_MU_EDCA_PARMS = 0xb0,
+ MCU_CE_CMD_REG_WRITE = 0xc0,
+ MCU_CE_CMD_REG_READ = 0xc0,
+ MCU_CE_CMD_CHIP_CONFIG = 0xca,
+ MCU_CE_CMD_FWLOG_2_HOST = 0xc5,
+ MCU_CE_CMD_GET_WTBL = 0xcd,
+ MCU_CE_CMD_GET_TXPWR = 0xd0,
};
enum {
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x0/init.c b/drivers/net/wireless/mediatek/mt76/mt76x0/init.c
index 0bac39b..66d47c7 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x0/init.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x0/init.c
@@ -237,7 +237,10 @@ int mt76x0_register_device(struct mt76x02_dev *dev)
{
int ret;
- mt76x02_init_device(dev);
+ ret = mt76x02_init_device(dev);
+ if (ret)
+ return ret;
+
mt76x02_config_mac_addr_list(dev);
ret = mt76_register_device(&dev->mt76, true, mt76x02_rates,
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x0/main.c b/drivers/net/wireless/mediatek/mt76/mt76x0/main.c
index 700ae9c..07380cc 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x0/main.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x0/main.c
@@ -31,6 +31,32 @@ mt76x0_set_channel(struct mt76x02_dev *dev, struct cfg80211_chan_def *chandef)
mt76_txq_schedule_all(&dev->mphy);
}
+int mt76x0_set_sar_specs(struct ieee80211_hw *hw,
+ const struct cfg80211_sar_specs *sar)
+{
+ int err = -EINVAL, power = hw->conf.power_level * 2;
+ struct mt76x02_dev *dev = hw->priv;
+ struct mt76_phy *mphy = &dev->mphy;
+
+ mutex_lock(&dev->mt76.mutex);
+ if (!cfg80211_chandef_valid(&mphy->chandef))
+ goto out;
+
+ err = mt76_init_sar_power(hw, sar);
+ if (err)
+ goto out;
+
+ dev->txpower_conf = mt76_get_sar_power(mphy, mphy->chandef.chan,
+ power);
+ if (test_bit(MT76_STATE_RUNNING, &mphy->state))
+ mt76x0_phy_set_txpower(dev);
+out:
+ mutex_unlock(&dev->mt76.mutex);
+
+ return err;
+}
+EXPORT_SYMBOL_GPL(mt76x0_set_sar_specs);
+
int mt76x0_config(struct ieee80211_hw *hw, u32 changed)
{
struct mt76x02_dev *dev = hw->priv;
@@ -44,9 +70,13 @@ int mt76x0_config(struct ieee80211_hw *hw, u32 changed)
}
if (changed & IEEE80211_CONF_CHANGE_POWER) {
- dev->txpower_conf = hw->conf.power_level * 2;
+ struct mt76_phy *mphy = &dev->mphy;
- if (test_bit(MT76_STATE_RUNNING, &dev->mphy.state))
+ dev->txpower_conf = hw->conf.power_level * 2;
+ dev->txpower_conf = mt76_get_sar_power(mphy,
+ mphy->chandef.chan,
+ dev->txpower_conf);
+ if (test_bit(MT76_STATE_RUNNING, &mphy->state))
mt76x0_phy_set_txpower(dev);
}
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x0/mt76x0.h b/drivers/net/wireless/mediatek/mt76/mt76x0/mt76x0.h
index 6953f25..99dcb8f 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x0/mt76x0.h
+++ b/drivers/net/wireless/mediatek/mt76/mt76x0/mt76x0.h
@@ -49,6 +49,8 @@ void mt76x0_chip_onoff(struct mt76x02_dev *dev, bool enable, bool reset);
void mt76x0_mac_stop(struct mt76x02_dev *dev);
int mt76x0_config(struct ieee80211_hw *hw, u32 changed);
+int mt76x0_set_sar_specs(struct ieee80211_hw *hw,
+ const struct cfg80211_sar_specs *sar);
/* PHY */
void mt76x0_phy_init(struct mt76x02_dev *dev);
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x0/pci.c b/drivers/net/wireless/mediatek/mt76/mt76x0/pci.c
index f19228f..9277ff3 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x0/pci.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x0/pci.c
@@ -85,6 +85,7 @@ static const struct ieee80211_ops mt76x0e_ops = {
.set_rts_threshold = mt76x02_set_rts_threshold,
.get_antenna = mt76_get_antenna,
.reconfig_complete = mt76x02_reconfig_complete,
+ .set_sar_specs = mt76x0_set_sar_specs,
};
static int mt76x0e_init_hardware(struct mt76x02_dev *dev, bool resume)
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x0/usb.c b/drivers/net/wireless/mediatek/mt76/mt76x0/usb.c
index f2b2fa7..436daf6 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x0/usb.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x0/usb.c
@@ -141,6 +141,7 @@ static const struct ieee80211_ops mt76x0u_ops = {
.set_tim = mt76_set_tim,
.release_buffered_frames = mt76_release_buffered_frames,
.get_antenna = mt76_get_antenna,
+ .set_sar_specs = mt76x0_set_sar_specs,
};
static int mt76x0u_init_hardware(struct mt76x02_dev *dev, bool reset)
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x02.h b/drivers/net/wireless/mediatek/mt76/mt76x02.h
index 4d58c2c..44d1a92 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x02.h
+++ b/drivers/net/wireless/mediatek/mt76/mt76x02.h
@@ -133,7 +133,7 @@ struct mt76x02_dev {
extern struct ieee80211_rate mt76x02_rates[12];
-void mt76x02_init_device(struct mt76x02_dev *dev);
+int mt76x02_init_device(struct mt76x02_dev *dev);
void mt76x02_configure_filter(struct ieee80211_hw *hw,
unsigned int changed_flags,
unsigned int *total_flags, u64 multicast);
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x02_util.c b/drivers/net/wireless/mediatek/mt76/mt76x02_util.c
index 1f17d86..dd30f53 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x02_util.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x02_util.c
@@ -138,7 +138,7 @@ mt76x02_led_set_brightness(struct led_classdev *led_cdev,
mt76x02_led_set_config(mdev, 0xff, 0);
}
-void mt76x02_init_device(struct mt76x02_dev *dev)
+int mt76x02_init_device(struct mt76x02_dev *dev)
{
struct ieee80211_hw *hw = mt76_hw(dev);
struct wiphy *wiphy = hw->wiphy;
@@ -197,6 +197,8 @@ void mt76x02_init_device(struct mt76x02_dev *dev)
dev->mphy.chainmask = 0x101;
dev->mphy.antenna_mask = 1;
}
+
+ return 0;
}
EXPORT_SYMBOL_GPL(mt76x02_init_device);
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x2/init.c b/drivers/net/wireless/mediatek/mt76/mt76x2/init.c
index a92a479..7b01a06 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x2/init.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x2/init.c
@@ -8,6 +8,35 @@
#include "eeprom.h"
#include "../mt76x02_phy.h"
+int mt76x2_set_sar_specs(struct ieee80211_hw *hw,
+ const struct cfg80211_sar_specs *sar)
+{
+ int err = -EINVAL, power = hw->conf.power_level * 2;
+ struct mt76x02_dev *dev = hw->priv;
+ struct mt76_phy *mphy = &dev->mphy;
+
+ mutex_lock(&dev->mt76.mutex);
+ if (!cfg80211_chandef_valid(&mphy->chandef))
+ goto out;
+
+ err = mt76_init_sar_power(hw, sar);
+ if (err)
+ goto out;
+
+ dev->txpower_conf = mt76_get_sar_power(mphy, mphy->chandef.chan,
+ power);
+ /* convert to per-chain power for 2x2 devices */
+ dev->txpower_conf -= 6;
+
+ if (test_bit(MT76_STATE_RUNNING, &mphy->state))
+ mt76x2_phy_set_txpower(dev);
+out:
+ mutex_unlock(&dev->mt76.mutex);
+
+ return err;
+}
+EXPORT_SYMBOL_GPL(mt76x2_set_sar_specs);
+
static void
mt76x2_set_wlan_state(struct mt76x02_dev *dev, bool enable)
{
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x2/mt76x2.h b/drivers/net/wireless/mediatek/mt76/mt76x2/mt76x2.h
index d01f47c..be12173 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x2/mt76x2.h
+++ b/drivers/net/wireless/mediatek/mt76/mt76x2/mt76x2.h
@@ -41,6 +41,8 @@ extern const struct ieee80211_ops mt76x2_ops;
int mt76x2_register_device(struct mt76x02_dev *dev);
int mt76x2_resume_device(struct mt76x02_dev *dev);
+int mt76x2_set_sar_specs(struct ieee80211_hw *hw,
+ const struct cfg80211_sar_specs *sar);
void mt76x2_phy_power_on(struct mt76x02_dev *dev);
void mt76x2_stop_hardware(struct mt76x02_dev *dev);
int mt76x2_eeprom_init(struct mt76x02_dev *dev);
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x2/pci_init.c b/drivers/net/wireless/mediatek/mt76/mt76x2/pci_init.c
index c6fa8cf..e38e8e5 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x2/pci_init.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x2/pci_init.c
@@ -292,8 +292,9 @@ int mt76x2_register_device(struct mt76x02_dev *dev)
int ret;
INIT_DELAYED_WORK(&dev->cal_work, mt76x2_phy_calibrate);
-
- mt76x02_init_device(dev);
+ ret = mt76x02_init_device(dev);
+ if (ret)
+ return ret;
ret = mt76x2_init_hardware(dev);
if (ret)
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x2/pci_main.c b/drivers/net/wireless/mediatek/mt76/mt76x2/pci_main.c
index 933125b..b38bb7a 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x2/pci_main.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x2/pci_main.c
@@ -78,8 +78,12 @@ mt76x2_config(struct ieee80211_hw *hw, u32 changed)
}
if (changed & IEEE80211_CONF_CHANGE_POWER) {
- dev->txpower_conf = hw->conf.power_level * 2;
+ struct mt76_phy *mphy = &dev->mphy;
+ dev->txpower_conf = hw->conf.power_level * 2;
+ dev->txpower_conf = mt76_get_sar_power(mphy,
+ mphy->chandef.chan,
+ dev->txpower_conf);
/* convert to per-chain power for 2x2 devices */
dev->txpower_conf -= 6;
@@ -155,5 +159,6 @@ const struct ieee80211_ops mt76x2_ops = {
.get_antenna = mt76_get_antenna,
.set_rts_threshold = mt76x02_set_rts_threshold,
.reconfig_complete = mt76x02_reconfig_complete,
+ .set_sar_specs = mt76x2_set_sar_specs,
};
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x2/usb_init.c b/drivers/net/wireless/mediatek/mt76/mt76x2/usb_init.c
index 85dcdc2..33a1436 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x2/usb_init.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x2/usb_init.c
@@ -194,7 +194,9 @@ int mt76x2u_register_device(struct mt76x02_dev *dev)
int err;
INIT_DELAYED_WORK(&dev->cal_work, mt76x2u_phy_calibrate);
- mt76x02_init_device(dev);
+ err = mt76x02_init_device(dev);
+ if (err)
+ return err;
err = mt76x2u_init_eeprom(dev);
if (err < 0)
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x2/usb_main.c b/drivers/net/wireless/mediatek/mt76/mt76x2/usb_main.c
index b668369..ac07ed1 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x2/usb_main.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x2/usb_main.c
@@ -78,12 +78,16 @@ mt76x2u_config(struct ieee80211_hw *hw, u32 changed)
}
if (changed & IEEE80211_CONF_CHANGE_POWER) {
- dev->txpower_conf = hw->conf.power_level * 2;
+ struct mt76_phy *mphy = &dev->mphy;
+ dev->txpower_conf = hw->conf.power_level * 2;
+ dev->txpower_conf = mt76_get_sar_power(mphy,
+ mphy->chandef.chan,
+ dev->txpower_conf);
/* convert to per-chain power for 2x2 devices */
dev->txpower_conf -= 6;
- if (test_bit(MT76_STATE_RUNNING, &dev->mphy.state))
+ if (test_bit(MT76_STATE_RUNNING, &mphy->state))
mt76x2_phy_set_txpower(dev);
}
@@ -121,4 +125,5 @@ const struct ieee80211_ops mt76x2u_ops = {
.set_tim = mt76_set_tim,
.release_buffered_frames = mt76_release_buffered_frames,
.get_antenna = mt76_get_antenna,
+ .set_sar_specs = mt76x2_set_sar_specs,
};
diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/debugfs.c b/drivers/net/wireless/mediatek/mt76/mt7915/debugfs.c
index a15aa25..e96d1c31 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7915/debugfs.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7915/debugfs.c
@@ -82,6 +82,225 @@ DEFINE_DEBUGFS_ATTRIBUTE(fops_radar_trigger, NULL,
mt7915_radar_trigger, "%lld\n");
static int
+mt7915_muru_debug_set(void *data, u64 val)
+{
+ struct mt7915_dev *dev = data;
+
+ dev->muru_debug = val;
+ mt7915_mcu_muru_debug_set(dev, data);
+
+ return 0;
+}
+
+static int
+mt7915_muru_debug_get(void *data, u64 *val)
+{
+ struct mt7915_dev *dev = data;
+
+ *val = dev->muru_debug;
+
+ return 0;
+}
+
+DEFINE_DEBUGFS_ATTRIBUTE(fops_muru_debug, mt7915_muru_debug_get,
+ mt7915_muru_debug_set, "%lld\n");
+
+static int mt7915_muru_stats_show(struct seq_file *file, void *data)
+{
+ struct mt7915_phy *phy = file->private;
+ struct mt7915_dev *dev = phy->dev;
+ struct mt7915_mcu_muru_stats mu_stats = {};
+ static const char * const dl_non_he_type[] = {
+ "CCK", "OFDM", "HT MIX", "HT GF",
+ "VHT SU", "VHT 2MU", "VHT 3MU", "VHT 4MU"
+ };
+ static const char * const dl_he_type[] = {
+ "HE SU", "HE EXT", "HE 2MU", "HE 3MU", "HE 4MU",
+ "HE 2RU", "HE 3RU", "HE 4RU", "HE 5-8RU", "HE 9-16RU",
+ "HE >16RU"
+ };
+ static const char * const ul_he_type[] = {
+ "HE 2MU", "HE 3MU", "HE 4MU", "HE SU", "HE 2RU",
+ "HE 3RU", "HE 4RU", "HE 5-8RU", "HE 9-16RU", "HE >16RU"
+ };
+ int ret, i;
+ u64 total_ppdu_cnt, sub_total_cnt;
+
+ if (!dev->muru_debug) {
+ seq_puts(file, "Please enable muru_debug first.\n");
+ return 0;
+ }
+
+ mutex_lock(&dev->mt76.mutex);
+
+ ret = mt7915_mcu_muru_debug_get(phy, &mu_stats);
+ if (ret)
+ goto exit;
+
+ /* Non-HE Downlink*/
+ seq_puts(file, "[Non-HE]\nDownlink\nData Type: ");
+
+ for (i = 0; i < 5; i++)
+ seq_printf(file, "%8s | ", dl_non_he_type[i]);
+
+#define __dl_u32(s) le32_to_cpu(mu_stats.dl.s)
+ seq_puts(file, "\nTotal Count:");
+ seq_printf(file, "%8u | %8u | %8u | %8u | %8u | ",
+ __dl_u32(cck_cnt),
+ __dl_u32(ofdm_cnt),
+ __dl_u32(htmix_cnt),
+ __dl_u32(htgf_cnt),
+ __dl_u32(vht_su_cnt));
+
+ seq_puts(file, "\nDownlink MU-MIMO\nData Type: ");
+
+ for (i = 5; i < 8; i++)
+ seq_printf(file, "%8s | ", dl_non_he_type[i]);
+
+ seq_puts(file, "\nTotal Count:");
+ seq_printf(file, "%8u | %8u | %8u | ",
+ __dl_u32(vht_2mu_cnt),
+ __dl_u32(vht_3mu_cnt),
+ __dl_u32(vht_4mu_cnt));
+
+ sub_total_cnt = __dl_u32(vht_2mu_cnt) +
+ __dl_u32(vht_3mu_cnt) +
+ __dl_u32(vht_4mu_cnt);
+
+ seq_printf(file, "\nTotal non-HE MU-MIMO DL PPDU count: %lld",
+ sub_total_cnt);
+
+ total_ppdu_cnt = sub_total_cnt +
+ __dl_u32(cck_cnt) +
+ __dl_u32(ofdm_cnt) +
+ __dl_u32(htmix_cnt) +
+ __dl_u32(htgf_cnt) +
+ __dl_u32(vht_su_cnt);
+
+ seq_printf(file, "\nAll non-HE DL PPDU count: %lld", total_ppdu_cnt);
+
+ /* HE Downlink */
+ seq_puts(file, "\n\n[HE]\nDownlink\nData Type: ");
+
+ for (i = 0; i < 2; i++)
+ seq_printf(file, "%8s | ", dl_he_type[i]);
+
+ seq_puts(file, "\nTotal Count:");
+ seq_printf(file, "%8u | %8u | ",
+ __dl_u32(he_su_cnt),
+ __dl_u32(he_ext_su_cnt));
+
+ seq_puts(file, "\nDownlink MU-MIMO\nData Type: ");
+
+ for (i = 2; i < 5; i++)
+ seq_printf(file, "%8s | ", dl_he_type[i]);
+
+ seq_puts(file, "\nTotal Count:");
+ seq_printf(file, "%8u | %8u | %8u | ",
+ __dl_u32(he_2mu_cnt),
+ __dl_u32(he_3mu_cnt),
+ __dl_u32(he_4mu_cnt));
+
+ seq_puts(file, "\nDownlink OFDMA\nData Type: ");
+
+ for (i = 5; i < 11; i++)
+ seq_printf(file, "%8s | ", dl_he_type[i]);
+
+ seq_puts(file, "\nTotal Count:");
+ seq_printf(file, "%8u | %8u | %8u | %8u | %9u | %8u | ",
+ __dl_u32(he_2ru_cnt),
+ __dl_u32(he_3ru_cnt),
+ __dl_u32(he_4ru_cnt),
+ __dl_u32(he_5to8ru_cnt),
+ __dl_u32(he_9to16ru_cnt),
+ __dl_u32(he_gtr16ru_cnt));
+
+ sub_total_cnt = __dl_u32(he_2mu_cnt) +
+ __dl_u32(he_3mu_cnt) +
+ __dl_u32(he_4mu_cnt);
+ total_ppdu_cnt = sub_total_cnt;
+
+ seq_printf(file, "\nTotal HE MU-MIMO DL PPDU count: %lld",
+ sub_total_cnt);
+
+ sub_total_cnt = __dl_u32(he_2ru_cnt) +
+ __dl_u32(he_3ru_cnt) +
+ __dl_u32(he_4ru_cnt) +
+ __dl_u32(he_5to8ru_cnt) +
+ __dl_u32(he_9to16ru_cnt) +
+ __dl_u32(he_gtr16ru_cnt);
+ total_ppdu_cnt += sub_total_cnt;
+
+ seq_printf(file, "\nTotal HE OFDMA DL PPDU count: %lld",
+ sub_total_cnt);
+
+ total_ppdu_cnt += __dl_u32(he_su_cnt) +
+ __dl_u32(he_ext_su_cnt);
+
+ seq_printf(file, "\nAll HE DL PPDU count: %lld", total_ppdu_cnt);
+#undef __dl_u32
+
+ /* HE Uplink */
+ seq_puts(file, "\n\nUplink");
+ seq_puts(file, "\nTrigger-based Uplink MU-MIMO\nData Type: ");
+
+ for (i = 0; i < 3; i++)
+ seq_printf(file, "%8s | ", ul_he_type[i]);
+
+#define __ul_u32(s) le32_to_cpu(mu_stats.ul.s)
+ seq_puts(file, "\nTotal Count:");
+ seq_printf(file, "%8u | %8u | %8u | ",
+ __ul_u32(hetrig_2mu_cnt),
+ __ul_u32(hetrig_3mu_cnt),
+ __ul_u32(hetrig_4mu_cnt));
+
+ seq_puts(file, "\nTrigger-based Uplink OFDMA\nData Type: ");
+
+ for (i = 3; i < 10; i++)
+ seq_printf(file, "%8s | ", ul_he_type[i]);
+
+ seq_puts(file, "\nTotal Count:");
+ seq_printf(file, "%8u | %8u | %8u | %8u | %8u | %9u | %7u | ",
+ __ul_u32(hetrig_su_cnt),
+ __ul_u32(hetrig_2ru_cnt),
+ __ul_u32(hetrig_3ru_cnt),
+ __ul_u32(hetrig_4ru_cnt),
+ __ul_u32(hetrig_5to8ru_cnt),
+ __ul_u32(hetrig_9to16ru_cnt),
+ __ul_u32(hetrig_gtr16ru_cnt));
+
+ sub_total_cnt = __ul_u32(hetrig_2mu_cnt) +
+ __ul_u32(hetrig_3mu_cnt) +
+ __ul_u32(hetrig_4mu_cnt);
+ total_ppdu_cnt = sub_total_cnt;
+
+ seq_printf(file, "\nTotal HE MU-MIMO UL TB PPDU count: %lld",
+ sub_total_cnt);
+
+ sub_total_cnt = __ul_u32(hetrig_2ru_cnt) +
+ __ul_u32(hetrig_3ru_cnt) +
+ __ul_u32(hetrig_4ru_cnt) +
+ __ul_u32(hetrig_5to8ru_cnt) +
+ __ul_u32(hetrig_9to16ru_cnt) +
+ __ul_u32(hetrig_gtr16ru_cnt);
+ total_ppdu_cnt += sub_total_cnt;
+
+ seq_printf(file, "\nTotal HE OFDMA UL TB PPDU count: %lld",
+ sub_total_cnt);
+
+ total_ppdu_cnt += __ul_u32(hetrig_su_cnt);
+
+ seq_printf(file, "\nAll HE UL TB PPDU count: %lld\n", total_ppdu_cnt);
+#undef __ul_u32
+
+exit:
+ mutex_unlock(&dev->mt76.mutex);
+
+ return ret;
+}
+DEFINE_SHOW_ATTRIBUTE(mt7915_muru_stats);
+
+static int
mt7915_fw_debug_wm_set(void *data, u64 val)
{
struct mt7915_dev *dev = data;
@@ -355,8 +574,8 @@ mt7915_sta_hw_queue_read(void *data, struct ieee80211_sta *sta)
qlen = mt76_get_field(dev, MT_PLE_BASE + MT_FL_Q3_CTRL,
GENMASK(11, 0));
seq_printf(s, "\tSTA %pM wcid %d: AC%d%d queued:%d\n",
- sta->addr, msta->wcid.idx, msta->vif->wmm_idx,
- ac, qlen);
+ sta->addr, msta->wcid.idx,
+ msta->vif->mt76.wmm_idx, ac, qlen);
}
}
@@ -528,7 +747,9 @@ int mt7915_init_debugfs(struct mt7915_phy *phy)
dir = mt76_register_debugfs_fops(phy->mt76, NULL);
if (!dir)
return -ENOMEM;
-
+ debugfs_create_file("muru_debug", 0600, dir, dev, &fops_muru_debug);
+ debugfs_create_file("muru_stats", 0400, dir, phy,
+ &mt7915_muru_stats_fops);
debugfs_create_file("hw-queues", 0400, dir, phy,
&mt7915_hw_queues_fops);
debugfs_create_file("xmit-queues", 0400, dir, phy,
diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/eeprom.c b/drivers/net/wireless/mediatek/mt76/mt7915/eeprom.c
index ee3d644..edd74d0 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7915/eeprom.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7915/eeprom.c
@@ -1,6 +1,7 @@
// SPDX-License-Identifier: ISC
/* Copyright (C) 2020 MediaTek Inc. */
+#include <linux/firmware.h>
#include "mt7915.h"
#include "eeprom.h"
@@ -10,6 +11,9 @@ static int mt7915_eeprom_load_precal(struct mt7915_dev *dev)
u8 *eeprom = mdev->eeprom.data;
u32 val = eeprom[MT_EE_DO_PRE_CAL];
+ if (!dev->flash_mode)
+ return 0;
+
if (val != (MT_EE_WIFI_CAL_DPD | MT_EE_WIFI_CAL_GROUP))
return 0;
@@ -21,30 +25,6 @@ static int mt7915_eeprom_load_precal(struct mt7915_dev *dev)
return mt76_get_of_eeprom(mdev, dev->cal, MT_EE_PRECAL, val);
}
-static int mt7915_eeprom_load(struct mt7915_dev *dev)
-{
- int ret;
-
- ret = mt76_eeprom_init(&dev->mt76, MT7915_EEPROM_SIZE);
- if (ret < 0)
- return ret;
-
- if (ret) {
- dev->flash_mode = true;
- ret = mt7915_eeprom_load_precal(dev);
- } else {
- u32 block_num, i;
-
- block_num = DIV_ROUND_UP(MT7915_EEPROM_SIZE,
- MT7915_EEPROM_BLOCK_SIZE);
- for (i = 0; i < block_num; i++)
- mt7915_mcu_get_eeprom(dev,
- i * MT7915_EEPROM_BLOCK_SIZE);
- }
-
- return ret;
-}
-
static int mt7915_check_eeprom(struct mt7915_dev *dev)
{
u8 *eeprom = dev->mt76.eeprom.data;
@@ -58,6 +38,66 @@ static int mt7915_check_eeprom(struct mt7915_dev *dev)
}
}
+static int
+mt7915_eeprom_load_default(struct mt7915_dev *dev)
+{
+ char *default_bin = MT7915_EEPROM_DEFAULT;
+ u8 *eeprom = dev->mt76.eeprom.data;
+ const struct firmware *fw = NULL;
+ int ret;
+
+ if (dev->dbdc_support)
+ default_bin = MT7915_EEPROM_DEFAULT_DBDC;
+
+ ret = request_firmware(&fw, default_bin, dev->mt76.dev);
+ if (ret)
+ return ret;
+
+ if (!fw || !fw->data) {
+ dev_err(dev->mt76.dev, "Invalid default bin\n");
+ ret = -EINVAL;
+ goto out;
+ }
+
+ memcpy(eeprom, fw->data, MT7915_EEPROM_SIZE);
+ dev->flash_mode = true;
+
+out:
+ release_firmware(fw);
+
+ return ret;
+}
+
+static int mt7915_eeprom_load(struct mt7915_dev *dev)
+{
+ int ret;
+
+ ret = mt76_eeprom_init(&dev->mt76, MT7915_EEPROM_SIZE);
+ if (ret < 0)
+ return ret;
+
+ if (ret) {
+ dev->flash_mode = true;
+ } else {
+ u8 free_block_num;
+ u32 block_num, i;
+
+ mt7915_mcu_get_eeprom_free_block(dev, &free_block_num);
+ /* efuse info not enough */
+ if (free_block_num >= 29)
+ return -EINVAL;
+
+ /* read eeprom data from efuse */
+ block_num = DIV_ROUND_UP(MT7915_EEPROM_SIZE,
+ MT7915_EEPROM_BLOCK_SIZE);
+ for (i = 0; i < block_num; i++)
+ mt7915_mcu_get_eeprom(dev,
+ i * MT7915_EEPROM_BLOCK_SIZE);
+ }
+
+ return mt7915_check_eeprom(dev);
+}
+
void mt7915_eeprom_parse_band_config(struct mt7915_phy *phy)
{
struct mt7915_dev *dev = phy->dev;
@@ -117,10 +157,17 @@ int mt7915_eeprom_init(struct mt7915_dev *dev)
int ret;
ret = mt7915_eeprom_load(dev);
- if (ret < 0)
- return ret;
+ if (ret < 0) {
+ if (ret != -EINVAL)
+ return ret;
- ret = mt7915_check_eeprom(dev);
+ dev_warn(dev->mt76.dev, "eeprom load fail, use default bin\n");
+ ret = mt7915_eeprom_load_default(dev);
+ if (ret)
+ return ret;
+ }
+
+ ret = mt7915_eeprom_load_precal(dev);
if (ret)
return ret;
diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/init.c b/drivers/net/wireless/mediatek/mt76/mt7915/init.c
index 4fa8e7b..d054cde 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7915/init.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7915/init.c
@@ -853,7 +853,8 @@ mt7915_init_he_caps(struct mt7915_phy *phy, enum nl80211_band band,
mt7915_gen_ppe_thresh(he_cap->ppe_thres, nss);
} else {
he_cap_elem->phy_cap_info[9] |=
- IEEE80211_HE_PHY_CAP9_NOMIMAL_PKT_PADDING_16US;
+ u8_encode_bits(IEEE80211_HE_PHY_CAP9_NOMINAL_PKT_PADDING_16US,
+ IEEE80211_HE_PHY_CAP9_NOMINAL_PKT_PADDING_MASK);
}
idx++;
}
diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/mac.c b/drivers/net/wireless/mediatek/mt76/mt7915/mac.c
index 809dc18..48f1155 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7915/mac.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7915/mac.c
@@ -268,10 +268,9 @@ mt7915_mac_decode_he_radiotap_ru(struct mt76_rx_status *status,
}
static void
-mt7915_mac_decode_he_mu_radiotap(struct sk_buff *skb,
- struct mt76_rx_status *status,
- __le32 *rxv)
+mt7915_mac_decode_he_mu_radiotap(struct sk_buff *skb, __le32 *rxv)
{
+ struct mt76_rx_status *status = (struct mt76_rx_status *)skb->cb;
static const struct ieee80211_radiotap_he_mu mu_known = {
.flags1 = HE_BITS(MU_FLAGS1_SIG_B_MCS_KNOWN) |
HE_BITS(MU_FLAGS1_SIG_B_DCM_KNOWN) |
@@ -281,6 +280,8 @@ mt7915_mac_decode_he_mu_radiotap(struct sk_buff *skb,
};
struct ieee80211_radiotap_he_mu *he_mu = NULL;
+ status->flag |= RX_FLAG_RADIOTAP_HE_MU;
+
he_mu = skb_push(skb, sizeof(mu_known));
memcpy(he_mu, &mu_known, sizeof(mu_known));
@@ -308,10 +309,9 @@ mt7915_mac_decode_he_mu_radiotap(struct sk_buff *skb,
}
static void
-mt7915_mac_decode_he_radiotap(struct sk_buff *skb,
- struct mt76_rx_status *status,
- __le32 *rxv, u32 phy)
+mt7915_mac_decode_he_radiotap(struct sk_buff *skb, __le32 *rxv, u32 mode)
{
+ struct mt76_rx_status *status = (struct mt76_rx_status *)skb->cb;
static const struct ieee80211_radiotap_he known = {
.data1 = HE_BITS(DATA1_DATA_MCS_KNOWN) |
HE_BITS(DATA1_DATA_DCM_KNOWN) |
@@ -329,6 +329,8 @@ mt7915_mac_decode_he_radiotap(struct sk_buff *skb,
struct ieee80211_radiotap_he *he = NULL;
u32 ltf_size = le32_get_bits(rxv[2], MT_CRXV_HE_LTF_SIZE) + 1;
+ status->flag |= RX_FLAG_RADIOTAP_HE;
+
he = skb_push(skb, sizeof(known));
memcpy(he, &known, sizeof(known));
@@ -343,7 +345,7 @@ mt7915_mac_decode_he_radiotap(struct sk_buff *skb,
he->data6 = HE_PREP(DATA6_TXOP, TXOP_DUR, rxv[14]) |
HE_PREP(DATA6_DOPPLER, DOPPLER, rxv[14]);
- switch (phy) {
+ switch (mode) {
case MT_PHY_TYPE_HE_SU:
he->data1 |= HE_BITS(DATA1_FORMAT_SU) |
HE_BITS(DATA1_UL_DL_KNOWN) |
@@ -366,6 +368,7 @@ mt7915_mac_decode_he_radiotap(struct sk_buff *skb,
he->data4 |= HE_PREP(DATA4_MU_STA_ID, MU_AID, rxv[7]);
mt7915_mac_decode_he_radiotap_ru(status, he, rxv);
+ mt7915_mac_decode_he_mu_radiotap(skb, rxv);
break;
case MT_PHY_TYPE_HE_TB:
he->data1 |= HE_BITS(DATA1_FORMAT_TRIG) |
@@ -384,6 +387,81 @@ mt7915_mac_decode_he_radiotap(struct sk_buff *skb,
}
}
+/* The HW does not translate the mac header to 802.3 for mesh point */
+static int mt7915_reverse_frag0_hdr_trans(struct sk_buff *skb, u16 hdr_gap)
+{
+ struct mt76_rx_status *status = (struct mt76_rx_status *)skb->cb;
+ struct mt7915_sta *msta = (struct mt7915_sta *)status->wcid;
+ struct ieee80211_sta *sta;
+ struct ieee80211_vif *vif;
+ struct ieee80211_hdr hdr;
+ struct ethhdr eth_hdr;
+ __le32 *rxd = (__le32 *)skb->data;
+ __le32 qos_ctrl, ht_ctrl;
+
+ if (FIELD_GET(MT_RXD3_NORMAL_ADDR_TYPE, le32_to_cpu(rxd[3])) !=
+ MT_RXD3_NORMAL_U2M)
+ return -EINVAL;
+
+ if (!(le32_to_cpu(rxd[1]) & MT_RXD1_NORMAL_GROUP_4))
+ return -EINVAL;
+
+ if (!msta || !msta->vif)
+ return -EINVAL;
+
+ sta = container_of((void *)msta, struct ieee80211_sta, drv_priv);
+ vif = container_of((void *)msta->vif, struct ieee80211_vif, drv_priv);
+
+ /* store the info from RXD and ethhdr to avoid being overridden */
+ memcpy(ð_hdr, skb->data + hdr_gap, sizeof(eth_hdr));
+ hdr.frame_control = FIELD_GET(MT_RXD6_FRAME_CONTROL, rxd[6]);
+ hdr.seq_ctrl = FIELD_GET(MT_RXD8_SEQ_CTRL, rxd[8]);
+ qos_ctrl = FIELD_GET(MT_RXD8_QOS_CTL, rxd[8]);
+ ht_ctrl = FIELD_GET(MT_RXD9_HT_CONTROL, rxd[9]);
+
+ hdr.duration_id = 0;
+ ether_addr_copy(hdr.addr1, vif->addr);
+ ether_addr_copy(hdr.addr2, sta->addr);
+ switch (le16_to_cpu(hdr.frame_control) &
+ (IEEE80211_FCTL_TODS | IEEE80211_FCTL_FROMDS)) {
+ case 0:
+ ether_addr_copy(hdr.addr3, vif->bss_conf.bssid);
+ break;
+ case IEEE80211_FCTL_FROMDS:
+ ether_addr_copy(hdr.addr3, eth_hdr.h_source);
+ break;
+ case IEEE80211_FCTL_TODS:
+ ether_addr_copy(hdr.addr3, eth_hdr.h_dest);
+ break;
+ case IEEE80211_FCTL_TODS | IEEE80211_FCTL_FROMDS:
+ ether_addr_copy(hdr.addr3, eth_hdr.h_dest);
+ ether_addr_copy(hdr.addr4, eth_hdr.h_source);
+ break;
+ default:
+ break;
+ }
+
+ skb_pull(skb, hdr_gap + sizeof(struct ethhdr) - 2);
+ if (eth_hdr.h_proto == htons(ETH_P_AARP) ||
+ eth_hdr.h_proto == htons(ETH_P_IPX))
+ ether_addr_copy(skb_push(skb, ETH_ALEN), bridge_tunnel_header);
+ else if (eth_hdr.h_proto >= htons(ETH_P_802_3_MIN))
+ ether_addr_copy(skb_push(skb, ETH_ALEN), rfc1042_header);
+ else
+ skb_pull(skb, 2);
+
+ if (ieee80211_has_order(hdr.frame_control))
+ memcpy(skb_push(skb, 2), &ht_ctrl, 2);
+ if (ieee80211_is_data_qos(hdr.frame_control))
+ memcpy(skb_push(skb, 2), &qos_ctrl, 2);
+ if (ieee80211_has_a4(hdr.frame_control))
+ memcpy(skb_push(skb, sizeof(hdr)), &hdr, sizeof(hdr));
+ else
+ memcpy(skb_push(skb, sizeof(hdr) - 6), &hdr, sizeof(hdr) - 6);
+
+ return 0;
+}
+
static int
mt7915_mac_fill_rx(struct mt7915_dev *dev, struct sk_buff *skb)
{
@@ -391,7 +469,6 @@ mt7915_mac_fill_rx(struct mt7915_dev *dev, struct sk_buff *skb)
struct mt76_phy *mphy = &dev->mt76.phy;
struct mt7915_phy *phy = &dev->phy;
struct ieee80211_supported_band *sband;
- struct ieee80211_hdr *hdr;
__le32 *rxd = (__le32 *)skb->data;
__le32 *rxv = NULL;
u32 mode = 0;
@@ -404,6 +481,7 @@ mt7915_mac_fill_rx(struct mt7915_dev *dev, struct sk_buff *skb)
bool unicast, insert_ccmp_hdr = false;
u8 remove_pad, amsdu_info;
bool hdr_trans;
+ u16 hdr_gap;
u16 seq_ctrl = 0;
u8 qos_ctl = 0;
__le16 fc = 0;
@@ -426,9 +504,16 @@ mt7915_mac_fill_rx(struct mt7915_dev *dev, struct sk_buff *skb)
if (rxd2 & MT_RXD2_NORMAL_AMSDU_ERR)
return -EINVAL;
+ hdr_trans = rxd2 & MT_RXD2_NORMAL_HDR_TRANS;
+ if (hdr_trans && (rxd1 & MT_RXD1_NORMAL_CM))
+ return -EINVAL;
+
+ /* ICV error or CCMP/BIP/WPI MIC error */
+ if (rxd1 & MT_RXD1_NORMAL_ICV_ERR)
+ status->flag |= RX_FLAG_ONLY_MONITOR;
+
unicast = FIELD_GET(MT_RXD3_NORMAL_ADDR_TYPE, rxd3) == MT_RXD3_NORMAL_U2M;
idx = FIELD_GET(MT_RXD1_NORMAL_WLAN_IDX, rxd1);
- hdr_trans = rxd2 & MT_RXD2_NORMAL_HDR_TRANS;
status->wcid = mt7915_rx_get_wcid(dev, idx, unicast);
if (status->wcid) {
@@ -604,15 +689,12 @@ mt7915_mac_fill_rx(struct mt7915_dev *dev, struct sk_buff *skb)
return -EINVAL;
break;
case MT_PHY_TYPE_HE_MU:
- status->flag |= RX_FLAG_RADIOTAP_HE_MU;
- fallthrough;
case MT_PHY_TYPE_HE_SU:
case MT_PHY_TYPE_HE_EXT_SU:
case MT_PHY_TYPE_HE_TB:
status->nss =
FIELD_GET(MT_PRXV_NSTS, v0) + 1;
status->encoding = RX_ENC_HE;
- status->flag |= RX_FLAG_RADIOTAP_HE;
i &= GENMASK(3, 0);
if (gi <= NL80211_RATE_INFO_HE_GI_3_2)
@@ -654,27 +736,55 @@ mt7915_mac_fill_rx(struct mt7915_dev *dev, struct sk_buff *skb)
}
}
- skb_pull(skb, (u8 *)rxd - skb->data + 2 * remove_pad);
-
amsdu_info = FIELD_GET(MT_RXD4_NORMAL_PAYLOAD_FORMAT, rxd4);
status->amsdu = !!amsdu_info;
if (status->amsdu) {
status->first_amsdu = amsdu_info == MT_RXD4_FIRST_AMSDU_FRAME;
status->last_amsdu = amsdu_info == MT_RXD4_LAST_AMSDU_FRAME;
- if (!hdr_trans) {
- memmove(skb->data + 2, skb->data,
- ieee80211_get_hdrlen_from_skb(skb));
+ }
+
+ hdr_gap = (u8 *)rxd - skb->data + 2 * remove_pad;
+ if (hdr_trans && ieee80211_has_morefrags(fc)) {
+ if (mt7915_reverse_frag0_hdr_trans(skb, hdr_gap))
+ return -EINVAL;
+ hdr_trans = false;
+ } else {
+ int pad_start = 0;
+
+ skb_pull(skb, hdr_gap);
+ if (!hdr_trans && status->amsdu) {
+ pad_start = ieee80211_get_hdrlen_from_skb(skb);
+ } else if (hdr_trans && (rxd2 & MT_RXD2_NORMAL_HDR_TRANS_ERROR)) {
+ /*
+ * When header translation failure is indicated,
+ * the hardware will insert an extra 2-byte field
+ * containing the data length after the protocol
+ * type field.
+ */
+ pad_start = 12;
+ if (get_unaligned_be16(skb->data + pad_start) == ETH_P_8021Q)
+ pad_start += 4;
+
+ if (get_unaligned_be16(skb->data + pad_start) !=
+ skb->len - pad_start - 2)
+ pad_start = 0;
+ }
+
+ if (pad_start) {
+ memmove(skb->data + 2, skb->data, pad_start);
skb_pull(skb, 2);
}
}
- if (insert_ccmp_hdr && !hdr_trans) {
- u8 key_id = FIELD_GET(MT_RXD1_NORMAL_KEY_ID, rxd1);
-
- mt76_insert_ccmp_hdr(skb, key_id);
- }
-
if (!hdr_trans) {
+ struct ieee80211_hdr *hdr;
+
+ if (insert_ccmp_hdr) {
+ u8 key_id = FIELD_GET(MT_RXD1_NORMAL_KEY_ID, rxd1);
+
+ mt76_insert_ccmp_hdr(skb, key_id);
+ }
+
hdr = mt76_skb_get_hdr(skb);
fc = hdr->frame_control;
if (ieee80211_is_data_qos(fc)) {
@@ -682,16 +792,11 @@ mt7915_mac_fill_rx(struct mt7915_dev *dev, struct sk_buff *skb)
qos_ctl = *ieee80211_get_qos_ctl(hdr);
}
} else {
- status->flag &= ~(RX_FLAG_RADIOTAP_HE |
- RX_FLAG_RADIOTAP_HE_MU);
status->flag |= RX_FLAG_8023;
}
- if (rxv && status->flag & RX_FLAG_RADIOTAP_HE) {
- mt7915_mac_decode_he_radiotap(skb, status, rxv, mode);
- if (status->flag & RX_FLAG_RADIOTAP_HE_MU)
- mt7915_mac_decode_he_mu_radiotap(skb, status, rxv);
- }
+ if (rxv && mode >= MT_PHY_TYPE_HE_SU && !(status->flag & RX_FLAG_8023))
+ mt7915_mac_decode_he_radiotap(skb, rxv, mode);
if (!status->wcid || !ieee80211_is_data_qos(fc))
return 0;
@@ -1044,8 +1149,8 @@ void mt7915_mac_write_txwi(struct mt7915_dev *dev, __le32 *txwi,
if (vif) {
struct mt7915_vif *mvif = (struct mt7915_vif *)vif->drv_priv;
- omac_idx = mvif->omac_idx;
- wmm_idx = mvif->wmm_idx;
+ omac_idx = mvif->mt76.omac_idx;
+ wmm_idx = mvif->mt76.wmm_idx;
}
if (ext_phy && dev->mt76.phy2)
@@ -1181,7 +1286,7 @@ int mt7915_tx_prepare_skb(struct mt76_dev *mdev, void *txwi_ptr,
if (vif) {
struct mt7915_vif *mvif = (struct mt7915_vif *)vif->drv_priv;
- txp->bss_idx = mvif->idx;
+ txp->bss_idx = mvif->mt76.idx;
}
txp->token = cpu_to_le16(id);
@@ -1268,15 +1373,16 @@ mt7915_txwi_free(struct mt7915_dev *dev, struct mt76_txwi_cache *t,
}
static void
-mt7915_mac_tx_free(struct mt7915_dev *dev, struct sk_buff *skb)
+mt7915_mac_tx_free(struct mt7915_dev *dev, void *data, int len)
{
- struct mt7915_tx_free *free = (struct mt7915_tx_free *)skb->data;
+ struct mt7915_tx_free *free = (struct mt7915_tx_free *)data;
struct mt76_dev *mdev = &dev->mt76;
struct mt76_phy *mphy_ext = mdev->phy2;
struct mt76_txwi_cache *txwi;
struct ieee80211_sta *sta = NULL;
LIST_HEAD(free_list);
- struct sk_buff *tmp;
+ struct sk_buff *skb, *tmp;
+ void *end = data + len;
u8 i, count;
bool wake = false;
@@ -1294,6 +1400,9 @@ mt7915_mac_tx_free(struct mt7915_dev *dev, struct sk_buff *skb)
* Should avoid accessing WTBL to get Tx airtime, and use it instead.
*/
count = FIELD_GET(MT_TX_FREE_MSDU_CNT, le16_to_cpu(free->ctrl));
+ if (WARN_ON_ONCE((void *)&free->info[count] > end))
+ return;
+
for (i = 0; i < count; i++) {
u32 msdu, info = le32_to_cpu(free->info[i]);
@@ -1336,8 +1445,6 @@ mt7915_mac_tx_free(struct mt7915_dev *dev, struct sk_buff *skb)
mt76_worker_schedule(&dev->mt76.tx_worker);
- napi_consume_skb(skb, 1);
-
list_for_each_entry_safe(skb, tmp, &free_list, list) {
skb_list_del_init(skb);
napi_consume_skb(skb, 1);
@@ -1512,6 +1619,27 @@ static void mt7915_mac_add_txs(struct mt7915_dev *dev, void *data)
rcu_read_unlock();
}
+bool mt7915_rx_check(struct mt76_dev *mdev, void *data, int len)
+{
+ struct mt7915_dev *dev = container_of(mdev, struct mt7915_dev, mt76);
+ __le32 *rxd = (__le32 *)data;
+ __le32 *end = (__le32 *)&rxd[len / 4];
+ enum rx_pkt_type type;
+
+ type = FIELD_GET(MT_RXD0_PKT_TYPE, le32_to_cpu(rxd[0]));
+ switch (type) {
+ case PKT_TYPE_TXRX_NOTIFY:
+ mt7915_mac_tx_free(dev, data, len);
+ return false;
+ case PKT_TYPE_TXS:
+ for (rxd += 2; rxd + 8 <= end; rxd += 8)
+ mt7915_mac_add_txs(dev, rxd);
+ return false;
+ default:
+ return true;
+ }
+}
+
void mt7915_queue_rx_skb(struct mt76_dev *mdev, enum mt76_rxq_id q,
struct sk_buff *skb)
{
@@ -1524,7 +1652,8 @@ void mt7915_queue_rx_skb(struct mt76_dev *mdev, enum mt76_rxq_id q,
switch (type) {
case PKT_TYPE_TXRX_NOTIFY:
- mt7915_mac_tx_free(dev, skb);
+ mt7915_mac_tx_free(dev, skb->data, skb->len);
+ napi_consume_skb(skb, 1);
break;
case PKT_TYPE_RX_EVENT:
mt7915_mcu_rx_event(dev, skb);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/main.c b/drivers/net/wireless/mediatek/mt76/mt7915/main.c
index 057ab27..8ac6f59 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7915/main.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7915/main.c
@@ -203,8 +203,8 @@ static int mt7915_add_interface(struct ieee80211_hw *hw,
is_zero_ether_addr(vif->addr))
phy->monitor_vif = vif;
- mvif->idx = ffs(~dev->mt76.vif_mask) - 1;
- if (mvif->idx >= MT7915_MAX_INTERFACES) {
+ mvif->mt76.idx = ffs(~dev->mt76.vif_mask) - 1;
+ if (mvif->mt76.idx >= MT7915_MAX_INTERFACES) {
ret = -ENOSPC;
goto out;
}
@@ -214,29 +214,27 @@ static int mt7915_add_interface(struct ieee80211_hw *hw,
ret = -ENOSPC;
goto out;
}
- mvif->omac_idx = idx;
+ mvif->mt76.omac_idx = idx;
mvif->phy = phy;
- mvif->band_idx = ext_phy;
+ mvif->mt76.band_idx = ext_phy;
- if (dev->mt76.phy2)
- mvif->wmm_idx = ext_phy * (MT7915_MAX_WMM_SETS / 2) +
- mvif->idx % (MT7915_MAX_WMM_SETS / 2);
- else
- mvif->wmm_idx = mvif->idx % MT7915_MAX_WMM_SETS;
+ mvif->mt76.wmm_idx = vif->type != NL80211_IFTYPE_AP;
+ if (ext_phy)
+ mvif->mt76.wmm_idx += 2;
ret = mt7915_mcu_add_dev_info(phy, vif, true);
if (ret)
goto out;
- dev->mt76.vif_mask |= BIT(mvif->idx);
- phy->omac_mask |= BIT_ULL(mvif->omac_idx);
+ dev->mt76.vif_mask |= BIT(mvif->mt76.idx);
+ phy->omac_mask |= BIT_ULL(mvif->mt76.omac_idx);
- idx = MT7915_WTBL_RESERVED - mvif->idx;
+ idx = MT7915_WTBL_RESERVED - mvif->mt76.idx;
INIT_LIST_HEAD(&mvif->sta.rc_list);
INIT_LIST_HEAD(&mvif->sta.poll_list);
mvif->sta.wcid.idx = idx;
- mvif->sta.wcid.ext_phy = mvif->band_idx;
+ mvif->sta.wcid.ext_phy = mvif->mt76.band_idx;
mvif->sta.wcid.hw_key_idx = -1;
mvif->sta.wcid.tx_info |= MT_WCID_TX_INFO_SET;
mt76_packet_id_init(&mvif->sta.wcid);
@@ -251,7 +249,7 @@ static int mt7915_add_interface(struct ieee80211_hw *hw,
}
if (vif->type != NL80211_IFTYPE_AP &&
- (!mvif->omac_idx || mvif->omac_idx > 3))
+ (!mvif->mt76.omac_idx || mvif->mt76.omac_idx > 3))
vif->offload_flags = 0;
vif->offload_flags |= IEEE80211_OFFLOAD_ENCAP_4ADDR;
@@ -288,8 +286,8 @@ static void mt7915_remove_interface(struct ieee80211_hw *hw,
rcu_assign_pointer(dev->mt76.wcid[idx], NULL);
mutex_lock(&dev->mt76.mutex);
- dev->mt76.vif_mask &= ~BIT(mvif->idx);
- phy->omac_mask &= ~BIT_ULL(mvif->omac_idx);
+ dev->mt76.vif_mask &= ~BIT(mvif->mt76.idx);
+ phy->omac_mask &= ~BIT_ULL(mvif->mt76.omac_idx);
mutex_unlock(&dev->mt76.mutex);
spin_lock_bh(&dev->sta_poll_lock);
@@ -425,6 +423,28 @@ static int mt7915_set_key(struct ieee80211_hw *hw, enum set_key_cmd cmd,
return err;
}
+static int mt7915_set_sar_specs(struct ieee80211_hw *hw,
+ const struct cfg80211_sar_specs *sar)
+{
+ struct mt7915_phy *phy = mt7915_hw_phy(hw);
+ struct mt7915_dev *dev = mt7915_hw_dev(hw);
+ int err = -EINVAL;
+
+ mutex_lock(&dev->mt76.mutex);
+ if (!cfg80211_chandef_valid(&phy->mt76->chandef))
+ goto out;
+
+ err = mt76_init_sar_power(hw, sar);
+ if (err)
+ goto out;
+
+ err = mt7915_mcu_set_txpower_sku(phy);
+out:
+ mutex_unlock(&dev->mt76.mutex);
+
+ return err;
+}
+
static int mt7915_config(struct ieee80211_hw *hw, u32 changed)
{
struct mt7915_dev *dev = mt7915_hw_dev(hw);
@@ -556,7 +576,7 @@ mt7915_update_bss_color(struct ieee80211_hw *hw,
case NL80211_IFTYPE_AP: {
struct mt7915_vif *mvif = (struct mt7915_vif *)vif->drv_priv;
- if (mvif->omac_idx > HW_BSSID_MAX)
+ if (mvif->mt76.omac_idx > HW_BSSID_MAX)
return;
fallthrough;
}
@@ -655,7 +675,7 @@ int mt7915_mac_sta_add(struct mt76_dev *mdev, struct ieee80211_vif *vif,
msta->vif = mvif;
msta->wcid.sta = 1;
msta->wcid.idx = idx;
- msta->wcid.ext_phy = mvif->band_idx;
+ msta->wcid.ext_phy = mvif->mt76.band_idx;
msta->wcid.tx_info |= MT_WCID_TX_INFO_SET;
msta->jiffies = jiffies;
@@ -838,7 +858,8 @@ u64 __mt7915_get_tsf(struct ieee80211_hw *hw, struct mt7915_vif *mvif)
lockdep_assert_held(&dev->mt76.mutex);
- n = mvif->omac_idx > HW_BSSID_MAX ? HW_BSSID_0 : mvif->omac_idx;
+ n = mvif->mt76.omac_idx > HW_BSSID_MAX ? HW_BSSID_0
+ : mvif->mt76.omac_idx;
/* TSF software read */
mt76_rmw(dev, MT_LPON_TCR(band, n), MT_LPON_TCR_SW_MODE,
MT_LPON_TCR_SW_READ);
@@ -878,7 +899,8 @@ mt7915_set_tsf(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
mutex_lock(&dev->mt76.mutex);
- n = mvif->omac_idx > HW_BSSID_MAX ? HW_BSSID_0 : mvif->omac_idx;
+ n = mvif->mt76.omac_idx > HW_BSSID_MAX ? HW_BSSID_0
+ : mvif->mt76.omac_idx;
mt76_wr(dev, MT_LPON_UTTR0(band), tsf.t32[0]);
mt76_wr(dev, MT_LPON_UTTR1(band), tsf.t32[1]);
/* TSF software overwrite */
@@ -904,7 +926,8 @@ mt7915_offset_tsf(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
mutex_lock(&dev->mt76.mutex);
- n = mvif->omac_idx > HW_BSSID_MAX ? HW_BSSID_0 : mvif->omac_idx;
+ n = mvif->mt76.omac_idx > HW_BSSID_MAX ? HW_BSSID_0
+ : mvif->mt76.omac_idx;
mt76_wr(dev, MT_LPON_UTTR0(band), tsf.t32[0]);
mt76_wr(dev, MT_LPON_UTTR1(band), tsf.t32[1]);
/* TSF software adjust*/
@@ -1195,7 +1218,7 @@ static void mt7915_ethtool_worker(void *wi_data, struct ieee80211_sta *sta)
struct mt76_ethtool_worker_info *wi = wi_data;
struct mt7915_sta *msta = (struct mt7915_sta *)sta->drv_priv;
- if (msta->vif->idx != wi->idx)
+ if (msta->vif->mt76.idx != wi->idx)
return;
mt76_ethtool_worker(wi, &msta->stats);
@@ -1211,7 +1234,7 @@ void mt7915_get_et_stats(struct ieee80211_hw *hw,
struct mt7915_vif *mvif = (struct mt7915_vif *)vif->drv_priv;
struct mt76_ethtool_worker_info wi = {
.data = data,
- .idx = mvif->idx,
+ .idx = mvif->mt76.idx,
};
struct mib_stats *mib = &phy->mib;
/* See mt7915_ampdu_stat_read_phy, etc */
@@ -1331,6 +1354,7 @@ const struct ieee80211_ops mt7915_ops = {
.sw_scan_complete = mt76_sw_scan_complete,
.release_buffered_frames = mt76_release_buffered_frames,
.get_txpower = mt76_get_txpower,
+ .set_sar_specs = mt7915_set_sar_specs,
.channel_switch_beacon = mt7915_channel_switch_beacon,
.get_stats = mt7915_get_stats,
.get_et_sset_count = mt7915_get_et_sset_count,
diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/mcu.c b/drivers/net/wireless/mediatek/mt76/mt7915/mcu.c
index 852d5d9..0911b6f 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7915/mcu.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7915/mcu.c
@@ -582,10 +582,10 @@ mt7915_mcu_alloc_sta_req(struct mt7915_dev *dev, struct mt7915_vif *mvif,
struct mt7915_sta *msta, int len)
{
struct sta_req_hdr hdr = {
- .bss_idx = mvif->idx,
+ .bss_idx = mvif->mt76.idx,
.wlan_idx_lo = msta ? to_wcid_lo(msta->wcid.idx) : 0,
.wlan_idx_hi = msta ? to_wcid_hi(msta->wcid.idx) : 0,
- .muar_idx = msta && msta->wcid.sta ? mvif->omac_idx : 0xe,
+ .muar_idx = msta && msta->wcid.sta ? mvif->mt76.omac_idx : 0xe,
.is_tlv_append = 1,
};
struct sk_buff *skb;
@@ -613,7 +613,7 @@ mt7915_mcu_alloc_wtbl_req(struct mt7915_dev *dev, struct mt7915_sta *msta,
if (!nskb) {
nskb = mt76_mcu_msg_alloc(&dev->mt76, NULL,
- MT7915_WTBL_UPDATE_MAX_SIZE);
+ MT76_CONNAC_WTBL_UPDATE_MAX_SIZE);
if (!nskb)
return ERR_PTR(-ENOMEM);
@@ -725,7 +725,7 @@ mt7915_mcu_bss_basic_tlv(struct sk_buff *skb, struct ieee80211_vif *vif,
bss->network_type = cpu_to_le32(type);
bss->bmc_wcid_lo = to_wcid_lo(wlan_idx);
bss->bmc_wcid_hi = to_wcid_hi(wlan_idx);
- bss->wmm_idx = mvif->wmm_idx;
+ bss->wmm_idx = mvif->mt76.wmm_idx;
bss->active = enable;
if (vif->type != NL80211_IFTYPE_MONITOR) {
@@ -769,10 +769,11 @@ mt7915_mcu_bss_omac_tlv(struct sk_buff *skb, struct ieee80211_vif *vif)
}
omac = (struct bss_info_omac *)tlv;
- idx = mvif->omac_idx > EXT_BSSID_START ? HW_BSSID_0 : mvif->omac_idx;
+ idx = mvif->mt76.omac_idx > EXT_BSSID_START ? HW_BSSID_0
+ : mvif->mt76.omac_idx;
omac->conn_type = cpu_to_le32(type);
- omac->omac_idx = mvif->omac_idx;
- omac->band_idx = mvif->band_idx;
+ omac->omac_idx = mvif->mt76.omac_idx;
+ omac->band_idx = mvif->mt76.band_idx;
omac->hw_bss_idx = idx;
}
@@ -937,7 +938,7 @@ mt7915_mcu_bss_ext_tlv(struct sk_buff *skb, struct mt7915_vif *mvif)
int ext_bss_idx, tsf_offset;
struct tlv *tlv;
- ext_bss_idx = mvif->omac_idx - EXT_BSSID_START;
+ ext_bss_idx = mvif->mt76.omac_idx - EXT_BSSID_START;
if (ext_bss_idx < 0)
return;
@@ -973,7 +974,7 @@ mt7915_mcu_muar_config(struct mt7915_phy *phy, struct ieee80211_vif *vif,
{
struct mt7915_dev *dev = phy->dev;
struct mt7915_vif *mvif = (struct mt7915_vif *)vif->drv_priv;
- u32 idx = mvif->omac_idx - REPEATER_BSSID_START;
+ u32 idx = mvif->mt76.omac_idx - REPEATER_BSSID_START;
u32 mask = phy->omac_mask >> 32 & ~BIT(idx);
const u8 *addr = vif->addr;
struct {
@@ -1011,7 +1012,7 @@ int mt7915_mcu_add_bss_info(struct mt7915_phy *phy,
struct mt7915_vif *mvif = (struct mt7915_vif *)vif->drv_priv;
struct sk_buff *skb;
- if (mvif->omac_idx >= REPEATER_BSSID_START) {
+ if (mvif->mt76.omac_idx >= REPEATER_BSSID_START) {
mt7915_mcu_muar_config(phy, vif, false, enable);
mt7915_mcu_muar_config(phy, vif, true, enable);
}
@@ -1039,8 +1040,8 @@ int mt7915_mcu_add_bss_info(struct mt7915_phy *phy,
if (vif->bss_conf.he_support)
mt7915_mcu_bss_he_tlv(skb, vif, phy);
- if (mvif->omac_idx >= EXT_BSSID_START &&
- mvif->omac_idx < REPEATER_BSSID_START)
+ if (mvif->mt76.omac_idx >= EXT_BSSID_START &&
+ mvif->mt76.omac_idx < REPEATER_BSSID_START)
mt7915_mcu_bss_ext_tlv(skb, mvif);
}
out:
@@ -1204,7 +1205,7 @@ mt7915_mcu_sta_ba(struct mt7915_dev *dev,
msta->wcid.amsdu = false;
skb = mt7915_mcu_alloc_sta_req(dev, mvif, msta,
- MT7915_STA_UPDATE_MAX_SIZE);
+ MT76_CONNAC_STA_UPDATE_MAX_SIZE);
if (IS_ERR(skb))
return PTR_ERR(skb);
@@ -1223,7 +1224,7 @@ mt7915_mcu_sta_ba(struct mt7915_dev *dev,
return ret;
skb = mt7915_mcu_alloc_sta_req(dev, mvif, msta,
- MT7915_STA_UPDATE_MAX_SIZE);
+ MT76_CONNAC_STA_UPDATE_MAX_SIZE);
if (IS_ERR(skb))
return PTR_ERR(skb);
@@ -1265,7 +1266,7 @@ mt7915_mcu_wtbl_generic_tlv(struct sk_buff *skb, struct ieee80211_vif *vif,
if (sta) {
memcpy(generic->peer_addr, sta->addr, ETH_ALEN);
generic->partial_aid = cpu_to_le16(sta->aid);
- generic->muar_idx = mvif->omac_idx;
+ generic->muar_idx = mvif->mt76.omac_idx;
generic->qos = sta->wme;
} else {
/* use BSSID in station mode */
@@ -1738,7 +1739,8 @@ int mt7915_mcu_sta_update_hdr_trans(struct mt7915_dev *dev,
struct wtbl_req_hdr *wtbl_hdr;
struct sk_buff *skb;
- skb = mt76_mcu_msg_alloc(&dev->mt76, NULL, MT7915_WTBL_UPDATE_MAX_SIZE);
+ skb = mt76_mcu_msg_alloc(&dev->mt76, NULL,
+ MT76_CONNAC_WTBL_UPDATE_MAX_SIZE);
if (!skb)
return -ENOMEM;
@@ -1752,33 +1754,6 @@ int mt7915_mcu_sta_update_hdr_trans(struct mt7915_dev *dev,
true);
}
-int mt7915_mcu_add_smps(struct mt7915_dev *dev, struct ieee80211_vif *vif,
- struct ieee80211_sta *sta)
-{
- struct mt7915_vif *mvif = (struct mt7915_vif *)vif->drv_priv;
- struct mt7915_sta *msta = (struct mt7915_sta *)sta->drv_priv;
- struct wtbl_req_hdr *wtbl_hdr;
- struct tlv *sta_wtbl;
- struct sk_buff *skb;
-
- skb = mt7915_mcu_alloc_sta_req(dev, mvif, msta,
- MT7915_STA_UPDATE_MAX_SIZE);
- if (IS_ERR(skb))
- return PTR_ERR(skb);
-
- sta_wtbl = mt7915_mcu_add_tlv(skb, STA_REC_WTBL, sizeof(struct tlv));
-
- wtbl_hdr = mt7915_mcu_alloc_wtbl_req(dev, msta, WTBL_SET, sta_wtbl,
- &skb);
- if (IS_ERR(wtbl_hdr))
- return PTR_ERR(wtbl_hdr);
-
- mt7915_mcu_wtbl_smps_tlv(skb, sta, sta_wtbl, wtbl_hdr);
-
- return mt76_mcu_skb_send_msg(&dev->mt76, skb,
- MCU_EXT_CMD(STA_REC_UPDATE), true);
-}
-
static inline bool
mt7915_is_ebf_supported(struct mt7915_phy *phy, struct ieee80211_vif *vif,
struct ieee80211_sta *sta, bool bfee)
@@ -1954,7 +1929,7 @@ mt7915_mcu_sta_bfer_tlv(struct mt7915_dev *dev, struct sk_buff *skb,
{
struct mt7915_vif *mvif = (struct mt7915_vif *)vif->drv_priv;
struct mt7915_phy *phy =
- mvif->band_idx ? mt7915_ext_phy(dev) : &dev->phy;
+ mvif->mt76.band_idx ? mt7915_ext_phy(dev) : &dev->phy;
int tx_ant = hweight8(phy->mt76->chainmask) - 1;
struct sta_rec_bf *bf;
struct tlv *tlv;
@@ -2021,7 +1996,7 @@ mt7915_mcu_sta_bfee_tlv(struct mt7915_dev *dev, struct sk_buff *skb,
{
struct mt7915_vif *mvif = (struct mt7915_vif *)vif->drv_priv;
struct mt7915_phy *phy =
- mvif->band_idx ? mt7915_ext_phy(dev) : &dev->phy;
+ mvif->mt76.band_idx ? mt7915_ext_phy(dev) : &dev->phy;
int tx_ant = hweight8(phy->mt76->chainmask) - 1;
struct sta_rec_bfee *bfee;
struct tlv *tlv;
@@ -2049,6 +2024,21 @@ mt7915_mcu_sta_bfee_tlv(struct mt7915_dev *dev, struct sk_buff *skb,
bfee->fb_identity_matrix = (nrow == 1 && tx_ant == 2);
}
+static enum mcu_mmps_mode
+mt7915_mcu_get_mmps_mode(enum ieee80211_smps_mode smps)
+{
+ switch (smps) {
+ case IEEE80211_SMPS_OFF:
+ return MCU_MMPS_DISABLE;
+ case IEEE80211_SMPS_STATIC:
+ return MCU_MMPS_STATIC;
+ case IEEE80211_SMPS_DYNAMIC:
+ return MCU_MMPS_DYNAMIC;
+ default:
+ return MCU_MMPS_DISABLE;
+ }
+}
+
int mt7915_mcu_set_fixed_rate_ctrl(struct mt7915_dev *dev,
struct ieee80211_vif *vif,
struct ieee80211_sta *sta,
@@ -2076,7 +2066,11 @@ int mt7915_mcu_set_fixed_rate_ctrl(struct mt7915_dev *dev,
case RATE_PARAM_FIXED_MCS:
case RATE_PARAM_FIXED_GI:
case RATE_PARAM_FIXED_HE_LTF:
- ra->phy = *phy;
+ if (phy)
+ ra->phy = *phy;
+ break;
+ case RATE_PARAM_MMPS_UPDATE:
+ ra->mmps_mode = mt7915_mcu_get_mmps_mode(sta->smps_mode);
break;
default:
break;
@@ -2087,6 +2081,39 @@ int mt7915_mcu_set_fixed_rate_ctrl(struct mt7915_dev *dev,
MCU_EXT_CMD(STA_REC_UPDATE), true);
}
+int mt7915_mcu_add_smps(struct mt7915_dev *dev, struct ieee80211_vif *vif,
+ struct ieee80211_sta *sta)
+{
+ struct mt7915_vif *mvif = (struct mt7915_vif *)vif->drv_priv;
+ struct mt7915_sta *msta = (struct mt7915_sta *)sta->drv_priv;
+ struct wtbl_req_hdr *wtbl_hdr;
+ struct tlv *sta_wtbl;
+ struct sk_buff *skb;
+ int ret;
+
+ skb = mt7915_mcu_alloc_sta_req(dev, mvif, msta,
+ MT76_CONNAC_STA_UPDATE_MAX_SIZE);
+ if (IS_ERR(skb))
+ return PTR_ERR(skb);
+
+ sta_wtbl = mt7915_mcu_add_tlv(skb, STA_REC_WTBL, sizeof(struct tlv));
+
+ wtbl_hdr = mt7915_mcu_alloc_wtbl_req(dev, msta, WTBL_SET, sta_wtbl,
+ &skb);
+ if (IS_ERR(wtbl_hdr))
+ return PTR_ERR(wtbl_hdr);
+
+ mt7915_mcu_wtbl_smps_tlv(skb, sta, sta_wtbl, wtbl_hdr);
+
+ ret = mt76_mcu_skb_send_msg(&dev->mt76, skb,
+ MCU_EXT_CMD(STA_REC_UPDATE), true);
+ if (ret)
+ return ret;
+
+ return mt7915_mcu_set_fixed_rate_ctrl(dev, vif, sta, NULL,
+ RATE_PARAM_MMPS_UPDATE);
+}
+
static int
mt7915_mcu_add_rate_ctrl_fixed(struct mt7915_dev *dev,
struct ieee80211_vif *vif,
@@ -2278,7 +2305,7 @@ int mt7915_mcu_add_rate_ctrl(struct mt7915_dev *dev, struct ieee80211_vif *vif,
int ret;
skb = mt7915_mcu_alloc_sta_req(dev, mvif, msta,
- MT7915_STA_UPDATE_MAX_SIZE);
+ MT76_CONNAC_STA_UPDATE_MAX_SIZE);
if (IS_ERR(skb))
return PTR_ERR(skb);
@@ -2323,7 +2350,7 @@ mt7915_mcu_add_group(struct mt7915_dev *dev, struct ieee80211_vif *vif,
u8 rsv1[8];
} __packed req = {
.action = cpu_to_le32(MT_STA_BSS_GROUP),
- .val = cpu_to_le32(mvif->idx % 16),
+ .val = cpu_to_le32(mvif->mt76.idx % 16),
};
msta = sta ? (struct mt7915_sta *)sta->drv_priv : &mvif->sta;
@@ -2345,7 +2372,7 @@ int mt7915_mcu_add_sta(struct mt7915_dev *dev, struct ieee80211_vif *vif,
msta = sta ? (struct mt7915_sta *)sta->drv_priv : &mvif->sta;
skb = mt7915_mcu_alloc_sta_req(dev, mvif, msta,
- MT7915_STA_UPDATE_MAX_SIZE);
+ MT76_CONNAC_STA_UPDATE_MAX_SIZE);
if (IS_ERR(skb))
return PTR_ERR(skb);
@@ -2389,53 +2416,6 @@ int mt7915_mcu_add_sta(struct mt7915_dev *dev, struct ieee80211_vif *vif,
MCU_EXT_CMD(STA_REC_UPDATE), true);
}
-int mt7915_mcu_set_fixed_rate(struct mt7915_dev *dev,
- struct ieee80211_sta *sta, u32 rate)
-{
- struct mt7915_sta *msta = (struct mt7915_sta *)sta->drv_priv;
- struct mt7915_vif *mvif = msta->vif;
- struct sta_rec_ra_fixed *ra;
- struct sk_buff *skb;
- struct tlv *tlv;
- int len = sizeof(struct sta_req_hdr) + sizeof(*ra);
-
- skb = mt7915_mcu_alloc_sta_req(dev, mvif, msta, len);
- if (IS_ERR(skb))
- return PTR_ERR(skb);
-
- tlv = mt7915_mcu_add_tlv(skb, STA_REC_RA_UPDATE, sizeof(*ra));
- ra = (struct sta_rec_ra_fixed *)tlv;
-
- if (!rate) {
- ra->field = cpu_to_le32(RATE_PARAM_AUTO);
- goto out;
- }
-
- ra->field = cpu_to_le32(RATE_PARAM_FIXED);
- ra->phy.type = FIELD_GET(RATE_CFG_PHY_TYPE, rate);
- ra->phy.bw = FIELD_GET(RATE_CFG_BW, rate);
- ra->phy.nss = FIELD_GET(RATE_CFG_NSS, rate);
- ra->phy.mcs = FIELD_GET(RATE_CFG_MCS, rate);
- ra->phy.stbc = FIELD_GET(RATE_CFG_STBC, rate);
-
- if (ra->phy.bw)
- ra->phy.ldpc = 7;
- else
- ra->phy.ldpc = FIELD_GET(RATE_CFG_LDPC, rate) * 7;
-
- /* HT/VHT - SGI: 1, LGI: 0; HE - SGI: 0, MGI: 1, LGI: 2 */
- if (ra->phy.type > MT_PHY_TYPE_VHT) {
- ra->phy.he_ltf = FIELD_GET(RATE_CFG_HE_LTF, rate) * 85;
- ra->phy.sgi = FIELD_GET(RATE_CFG_GI, rate) * 85;
- } else {
- ra->phy.sgi = FIELD_GET(RATE_CFG_GI, rate) * 15;
- }
-
-out:
- return mt76_mcu_skb_send_msg(&dev->mt76, skb,
- MCU_EXT_CMD(STA_REC_UPDATE), true);
-}
-
int mt7915_mcu_add_dev_info(struct mt7915_phy *phy,
struct ieee80211_vif *vif, bool enable)
{
@@ -2458,8 +2438,8 @@ int mt7915_mcu_add_dev_info(struct mt7915_phy *phy,
} __packed tlv;
} data = {
.hdr = {
- .omac_idx = mvif->omac_idx,
- .dbdc_idx = mvif->band_idx,
+ .omac_idx = mvif->mt76.omac_idx,
+ .dbdc_idx = mvif->mt76.band_idx,
.tlv_num = cpu_to_le16(1),
.is_tlv_append = 1,
},
@@ -2467,11 +2447,11 @@ int mt7915_mcu_add_dev_info(struct mt7915_phy *phy,
.tag = cpu_to_le16(DEV_INFO_ACTIVE),
.len = cpu_to_le16(sizeof(struct req_tlv)),
.active = enable,
- .dbdc_idx = mvif->band_idx,
+ .dbdc_idx = mvif->mt76.band_idx,
},
};
- if (mvif->omac_idx >= REPEATER_BSSID_START)
+ if (mvif->mt76.omac_idx >= REPEATER_BSSID_START)
return mt7915_mcu_muar_config(phy, vif, false, enable);
memcpy(data.tlv.omac_addr, vif->addr, ETH_ALEN);
@@ -2643,7 +2623,7 @@ int mt7915_mcu_add_beacon(struct ieee80211_hw *hw,
return -EINVAL;
}
- if (mvif->band_idx) {
+ if (mvif->mt76.band_idx) {
info = IEEE80211_SKB_CB(skb);
info->hw_queue |= MT_TX_HW_QUEUE_EXT_PHY;
}
@@ -2967,7 +2947,7 @@ static int mt7915_load_firmware(struct mt7915_dev *dev)
if (!mt76_poll_msec(dev, MT_TOP_MISC, MT_TOP_MISC_FW_STATE,
FIELD_PREP(MT_TOP_MISC_FW_STATE,
- FW_STATE_WACPU_RDY), 1000)) {
+ FW_STATE_RDY), 1000)) {
dev_err(dev->mt76.dev, "Timeout for initializing firmware\n");
return -EIO;
}
@@ -3014,6 +2994,47 @@ int mt7915_mcu_fw_dbg_ctrl(struct mt7915_dev *dev, u32 module, u8 level)
sizeof(data), false);
}
+int mt7915_mcu_muru_debug_set(struct mt7915_dev *dev, bool enabled)
+{
+ struct {
+ __le32 cmd;
+ u8 enable;
+ } data = {
+ .cmd = cpu_to_le32(MURU_SET_TXC_TX_STATS_EN),
+ .enable = enabled,
+ };
+
+ return mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD(MURU_CTRL), &data,
+ sizeof(data), false);
+}
+
+int mt7915_mcu_muru_debug_get(struct mt7915_phy *phy, void *ms)
+{
+ struct mt7915_dev *dev = phy->dev;
+ struct sk_buff *skb;
+ struct mt7915_mcu_muru_stats *mu_stats =
+ (struct mt7915_mcu_muru_stats *)ms;
+ int ret;
+
+ struct {
+ __le32 cmd;
+ u8 band_idx;
+ } req = {
+ .cmd = cpu_to_le32(MURU_GET_TXC_TX_STATS),
+ .band_idx = phy != &dev->phy,
+ };
+
+ ret = mt76_mcu_send_and_get_msg(&dev->mt76, MCU_EXT_CMD(MURU_CTRL),
+ &req, sizeof(req), true, &skb);
+ if (ret)
+ return ret;
+
+ memcpy(mu_stats, skb->data, sizeof(struct mt7915_mcu_muru_stats));
+ dev_kfree_skb(skb);
+
+ return 0;
+}
+
static int mt7915_mcu_set_mwds(struct mt7915_dev *dev, bool enabled)
{
struct {
@@ -3264,7 +3285,7 @@ int mt7915_mcu_set_tx(struct mt7915_dev *dev, struct ieee80211_vif *vif)
struct edca *e = &req.edca[ac];
e->set = WMM_PARAM_SET;
- e->queue = ac + mvif->wmm_idx * MT7915_MAX_WMM_SETS;
+ e->queue = ac + mvif->mt76.wmm_idx * MT7915_MAX_WMM_SETS;
e->aifs = q->aifs;
e->txop = cpu_to_le16(q->txop);
@@ -3579,6 +3600,30 @@ int mt7915_mcu_get_eeprom(struct mt7915_dev *dev, u32 offset)
return 0;
}
+int mt7915_mcu_get_eeprom_free_block(struct mt7915_dev *dev, u8 *block_num)
+{
+ struct {
+ u8 _rsv;
+ u8 version;
+ u8 die_idx;
+ u8 _rsv2;
+ } __packed req = {
+ .version = 1,
+ };
+ struct sk_buff *skb;
+ int ret;
+
+ ret = mt76_mcu_send_and_get_msg(&dev->mt76, MCU_EXT_QUERY(EFUSE_FREE_BLOCK), &req,
+ sizeof(req), true, &skb);
+ if (ret)
+ return ret;
+
+ *block_num = *(u8 *)skb->data;
+ dev_kfree_skb(skb);
+
+ return 0;
+}
+
static int mt7915_mcu_set_pre_cal(struct mt7915_dev *dev, u8 idx,
u8 *data, u32 len, int cmd)
{
@@ -3854,11 +3899,11 @@ int mt7915_mcu_set_txpower_sku(struct mt7915_phy *phy)
struct mt76_power_limits limits_array;
s8 *la = (s8 *)&limits_array;
int i, idx, n_chains = hweight8(mphy->antenna_mask);
- int tx_power;
+ int tx_power = hw->conf.power_level * 2;
- tx_power = hw->conf.power_level * 2 -
- mt76_tx_power_nss_delta(n_chains);
-
+ tx_power = mt76_get_sar_power(mphy, mphy->chandef.chan,
+ tx_power);
+ tx_power -= mt76_tx_power_nss_delta(n_chains);
tx_power = mt76_get_rate_power_limits(mphy, mphy->chandef.chan,
&limits_array, tx_power);
mphy->txpower_cur = tx_power;
@@ -4045,7 +4090,7 @@ int mt7915_mcu_add_obss_spr(struct mt7915_dev *dev, struct ieee80211_vif *vif,
} __packed req = {
.action = MT_SPR_ENABLE,
.arg_num = 1,
- .band_idx = mvif->band_idx,
+ .band_idx = mvif->mt76.band_idx,
.val = cpu_to_le32(enable),
};
@@ -4066,7 +4111,7 @@ int mt7915_mcu_get_rx_rate(struct mt7915_phy *phy, struct ieee80211_vif *vif,
__le16 wcid;
} __packed req = {
.category = MCU_PHY_STATE_CONTENTION_RX_RATE,
- .band = mvif->band_idx,
+ .band = mvif->mt76.band_idx,
.wcid = cpu_to_le16(msta->wcid.idx),
};
struct ieee80211_supported_band *sband;
@@ -4206,11 +4251,11 @@ int mt7915_mcu_twt_agrt_update(struct mt7915_dev *dev,
} __packed req = {
.tbl_idx = flow->table_id,
.cmd = cmd,
- .own_mac_idx = mvif->omac_idx,
+ .own_mac_idx = mvif->mt76.omac_idx,
.flowid = flow->id,
.peer_id = cpu_to_le16(flow->wcid),
.duration = flow->duration,
- .bss_idx = mvif->idx,
+ .bss_idx = mvif->mt76.idx,
.start_tsf = cpu_to_le64(flow->tsf),
.mantissa = flow->mantissa,
.exponent = flow->exp,
diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/mcu.h b/drivers/net/wireless/mediatek/mt76/mt7915/mcu.h
index 1f5a64b..92268e6 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7915/mcu.h
+++ b/drivers/net/wireless/mediatek/mt76/mt7915/mcu.h
@@ -4,6 +4,8 @@
#ifndef __MT7915_MCU_H
#define __MT7915_MCU_H
+#include "../mt76_connac_mcu.h"
+
struct mt7915_mcu_txd {
__le32 txd[8];
@@ -23,29 +25,6 @@ struct mt7915_mcu_txd {
u32 reserved[5];
} __packed __aligned(4);
-/* event table */
-enum {
- MCU_EVENT_TARGET_ADDRESS_LEN = 0x01,
- MCU_EVENT_FW_START = 0x01,
- MCU_EVENT_GENERIC = 0x01,
- MCU_EVENT_ACCESS_REG = 0x02,
- MCU_EVENT_MT_PATCH_SEM = 0x04,
- MCU_EVENT_CH_PRIVILEGE = 0x18,
- MCU_EVENT_EXT = 0xed,
- MCU_EVENT_RESTART_DL = 0xef,
-};
-
-/* ext event table */
-enum {
- MCU_EXT_EVENT_PS_SYNC = 0x5,
- MCU_EXT_EVENT_FW_LOG_2_HOST = 0x13,
- MCU_EXT_EVENT_THERMAL_PROTECT = 0x22,
- MCU_EXT_EVENT_ASSERT_DUMP = 0x23,
- MCU_EXT_EVENT_RDD_REPORT = 0x3a,
- MCU_EXT_EVENT_CSA_NOTIFY = 0x4f,
- MCU_EXT_EVENT_BCC_NOTIFY = 0x75,
-};
-
enum {
MCU_ATE_SET_TRX = 0x1,
MCU_ATE_SET_FREQ_OFFSET = 0xa,
@@ -206,6 +185,44 @@ struct mt7915_mcu_tx {
struct edca edca[IEEE80211_NUM_ACS];
} __packed;
+struct mt7915_mcu_muru_stats {
+ __le32 event_id;
+ struct {
+ __le32 cck_cnt;
+ __le32 ofdm_cnt;
+ __le32 htmix_cnt;
+ __le32 htgf_cnt;
+ __le32 vht_su_cnt;
+ __le32 vht_2mu_cnt;
+ __le32 vht_3mu_cnt;
+ __le32 vht_4mu_cnt;
+ __le32 he_su_cnt;
+ __le32 he_ext_su_cnt;
+ __le32 he_2ru_cnt;
+ __le32 he_2mu_cnt;
+ __le32 he_3ru_cnt;
+ __le32 he_3mu_cnt;
+ __le32 he_4ru_cnt;
+ __le32 he_4mu_cnt;
+ __le32 he_5to8ru_cnt;
+ __le32 he_9to16ru_cnt;
+ __le32 he_gtr16ru_cnt;
+ } dl;
+
+ struct {
+ __le32 hetrig_su_cnt;
+ __le32 hetrig_2ru_cnt;
+ __le32 hetrig_3ru_cnt;
+ __le32 hetrig_4ru_cnt;
+ __le32 hetrig_5to8ru_cnt;
+ __le32 hetrig_9to16ru_cnt;
+ __le32 hetrig_gtr16ru_cnt;
+ __le32 hetrig_2mu_cnt;
+ __le32 hetrig_3mu_cnt;
+ __le32 hetrig_4mu_cnt;
+ } ul;
+};
+
#define WMM_AIFS_SET BIT(0)
#define WMM_CW_MIN_SET BIT(1)
#define WMM_CW_MAX_SET BIT(2)
@@ -216,83 +233,11 @@ struct mt7915_mcu_tx {
#define MCU_PKT_ID 0xa0
enum {
- MCU_Q_QUERY,
- MCU_Q_SET,
- MCU_Q_RESERVED,
- MCU_Q_NA
-};
-
-enum {
- MCU_S2D_H2N,
- MCU_S2D_C2N,
- MCU_S2D_H2C,
- MCU_S2D_H2CN
-};
-
-enum {
MCU_FW_LOG_WM,
MCU_FW_LOG_WA,
MCU_FW_LOG_TO_HOST,
};
-#define __MCU_CMD_FIELD_ID GENMASK(7, 0)
-#define __MCU_CMD_FIELD_EXT_ID GENMASK(15, 8)
-#define __MCU_CMD_FIELD_QUERY BIT(16)
-#define __MCU_CMD_FIELD_WA BIT(17)
-
-enum {
- MCU_CMD_TARGET_ADDRESS_LEN_REQ = 0x01,
- MCU_CMD_FW_START_REQ = 0x02,
- MCU_CMD_INIT_ACCESS_REG = 0x3,
- MCU_CMD_NIC_POWER_CTRL = 0x4,
- MCU_CMD_PATCH_START_REQ = 0x05,
- MCU_CMD_PATCH_FINISH_REQ = 0x07,
- MCU_CMD_PATCH_SEM_CONTROL = 0x10,
- MCU_CMD_WA_PARAM = 0xC4,
- MCU_CMD_EXT_CID = 0xED,
- MCU_CMD_FW_SCATTER = 0xEE,
- MCU_CMD_RESTART_DL_REQ = 0xEF,
-};
-
-enum {
- MCU_EXT_CMD_EFUSE_ACCESS = 0x01,
- MCU_EXT_CMD_RF_TEST = 0x04,
- MCU_EXT_CMD_PM_STATE_CTRL = 0x07,
- MCU_EXT_CMD_CHANNEL_SWITCH = 0x08,
- MCU_EXT_CMD_FW_LOG_2_HOST = 0x13,
- MCU_EXT_CMD_TXBF_ACTION = 0x1e,
- MCU_EXT_CMD_EFUSE_BUFFER_MODE = 0x21,
- MCU_EXT_CMD_THERMAL_PROT = 0x23,
- MCU_EXT_CMD_STA_REC_UPDATE = 0x25,
- MCU_EXT_CMD_BSS_INFO_UPDATE = 0x26,
- MCU_EXT_CMD_EDCA_UPDATE = 0x27,
- MCU_EXT_CMD_DEV_INFO_UPDATE = 0x2A,
- MCU_EXT_CMD_THERMAL_CTRL = 0x2c,
- MCU_EXT_CMD_WTBL_UPDATE = 0x32,
- MCU_EXT_CMD_SET_DRR_CTRL = 0x36,
- MCU_EXT_CMD_SET_RDD_CTRL = 0x3a,
- MCU_EXT_CMD_ATE_CTRL = 0x3d,
- MCU_EXT_CMD_PROTECT_CTRL = 0x3e,
- MCU_EXT_CMD_MAC_INIT_CTRL = 0x46,
- MCU_EXT_CMD_RX_HDR_TRANS = 0x47,
- MCU_EXT_CMD_MUAR_UPDATE = 0x48,
- MCU_EXT_CMD_RX_AIRTIME_CTRL = 0x4a,
- MCU_EXT_CMD_SET_RX_PATH = 0x4e,
- MCU_EXT_CMD_TX_POWER_FEATURE_CTRL = 0x58,
- MCU_EXT_CMD_GET_MIB_INFO = 0x5a,
- MCU_EXT_CMD_MWDS_SUPPORT = 0x80,
- MCU_EXT_CMD_SET_SER_TRIGGER = 0x81,
- MCU_EXT_CMD_SCS_CTRL = 0x82,
- MCU_EXT_CMD_TWT_AGRT_UPDATE = 0x94,
- MCU_EXT_CMD_FW_DBG_CTRL = 0x95,
- MCU_EXT_CMD_SET_RDD_TH = 0x9d,
- MCU_EXT_CMD_MURU_CTRL = 0x9f,
- MCU_EXT_CMD_SET_SPR = 0xa8,
- MCU_EXT_CMD_GROUP_PRE_CAL_INFO = 0xab,
- MCU_EXT_CMD_DPD_PRE_CAL_INFO = 0xac,
- MCU_EXT_CMD_PHY_STAT_INFO = 0xad,
-};
-
enum {
MCU_TWT_AGRT_ADD,
MCU_TWT_AGRT_MODIFY,
@@ -314,55 +259,11 @@ enum {
MCU_WA_PARAM_RED = 0x0e,
};
-#define MCU_CMD(_t) FIELD_PREP(__MCU_CMD_FIELD_ID, MCU_CMD_##_t)
-#define MCU_EXT_CMD(_t) (MCU_CMD(EXT_CID) | \
- FIELD_PREP(__MCU_CMD_FIELD_EXT_ID, \
- MCU_EXT_CMD_##_t))
-#define MCU_EXT_QUERY(_t) (MCU_EXT_CMD(_t) | __MCU_CMD_FIELD_QUERY)
-
-#define MCU_WA_CMD(_t) (MCU_CMD(_t) | __MCU_CMD_FIELD_WA)
-#define MCU_WA_EXT_CMD(_t) (MCU_EXT_CMD(_t) | __MCU_CMD_FIELD_WA)
-#define MCU_WA_PARAM_CMD(_t) (MCU_WA_CMD(WA_PARAM) | \
- FIELD_PREP(__MCU_CMD_FIELD_EXT_ID, \
- MCU_WA_PARAM_CMD_##_t))
-
-enum {
- PATCH_SEM_RELEASE,
- PATCH_SEM_GET
-};
-
-enum {
- PATCH_NOT_DL_SEM_FAIL,
- PATCH_IS_DL,
- PATCH_NOT_DL_SEM_SUCCESS,
- PATCH_REL_SEM_SUCCESS
-};
-
-enum {
- FW_STATE_INITIAL,
- FW_STATE_FW_DOWNLOAD,
- FW_STATE_NORMAL_OPERATION,
- FW_STATE_NORMAL_TRX,
- FW_STATE_WACPU_RDY = 7
-};
-
-enum {
- EE_MODE_EFUSE,
- EE_MODE_BUFFER,
-};
-
-enum {
- EE_FORMAT_BIN,
- EE_FORMAT_WHOLE,
- EE_FORMAT_MULTIPLE,
-};
-
-enum {
- MCU_PHY_STATE_TX_RATE,
- MCU_PHY_STATE_RX_RATE,
- MCU_PHY_STATE_RSSI,
- MCU_PHY_STATE_CONTENTION_RX_RATE,
- MCU_PHY_STATE_OFDMLQ_CNINFO,
+enum mcu_mmps_mode {
+ MCU_MMPS_STATIC,
+ MCU_MMPS_DYNAMIC,
+ MCU_MMPS_RSV,
+ MCU_MMPS_DISABLE,
};
#define STA_TYPE_STA BIT(0)
@@ -389,11 +290,6 @@ enum {
#define CONN_STATE_PORT_SECURE 2
enum {
- DEV_INFO_ACTIVE,
- DEV_INFO_MAX_NUM
-};
-
-enum {
SCS_SEND_DATA,
SCS_SET_MANUAL_PD_TH,
SCS_CONFIG,
@@ -403,75 +299,6 @@ enum {
SCS_GET_GLO_ADDR_EVENT,
};
-enum {
- CMD_CBW_20MHZ = IEEE80211_STA_RX_BW_20,
- CMD_CBW_40MHZ = IEEE80211_STA_RX_BW_40,
- CMD_CBW_80MHZ = IEEE80211_STA_RX_BW_80,
- CMD_CBW_160MHZ = IEEE80211_STA_RX_BW_160,
- CMD_CBW_10MHZ,
- CMD_CBW_5MHZ,
- CMD_CBW_8080MHZ,
-
- CMD_HE_MCS_BW80 = 0,
- CMD_HE_MCS_BW160,
- CMD_HE_MCS_BW8080,
- CMD_HE_MCS_BW_NUM
-};
-
-struct tlv {
- __le16 tag;
- __le16 len;
-} __packed;
-
-struct bss_info_omac {
- __le16 tag;
- __le16 len;
- u8 hw_bss_idx;
- u8 omac_idx;
- u8 band_idx;
- u8 rsv0;
- __le32 conn_type;
- u32 rsv1;
-} __packed;
-
-struct bss_info_basic {
- __le16 tag;
- __le16 len;
- __le32 network_type;
- u8 active;
- u8 rsv0;
- __le16 bcn_interval;
- u8 bssid[ETH_ALEN];
- u8 wmm_idx;
- u8 dtim_period;
- u8 bmc_wcid_lo;
- u8 cipher;
- u8 phy_mode;
- u8 max_bssid; /* max BSSID. range: 1 ~ 8, 0: MBSSID disabled */
- u8 non_tx_bssid;/* non-transmitted BSSID, 0: transmitted BSSID */
- u8 bmc_wcid_hi; /* high Byte and version */
- u8 rsv[2];
-} __packed;
-
-struct bss_info_rf_ch {
- __le16 tag;
- __le16 len;
- u8 pri_ch;
- u8 center_ch0;
- u8 center_ch1;
- u8 bw;
- u8 he_ru26_block; /* 1: don't send HETB in RU26, 0: allow */
- u8 he_all_disable; /* 1: disallow all HETB, 0: allow */
- u8 rsv[2];
-} __packed;
-
-struct bss_info_ext_bss {
- __le16 tag;
- __le16 len;
- __le32 mbss_tsf_offset; /* in unit of us */
- u8 rsv[8];
-} __packed;
-
struct bss_info_bmc_rate {
__le16 tag;
__le16 len;
@@ -581,385 +408,8 @@ enum {
};
enum {
- BSS_INFO_OMAC,
- BSS_INFO_BASIC,
- BSS_INFO_RF_CH, /* optional, for BT/LTE coex */
- BSS_INFO_PM, /* sta only */
- BSS_INFO_UAPSD, /* sta only */
- BSS_INFO_ROAM_DETECT, /* obsoleted */
- BSS_INFO_LQ_RM, /* obsoleted */
- BSS_INFO_EXT_BSS,
- BSS_INFO_BMC_RATE, /* for bmc rate control in CR4 */
- BSS_INFO_SYNC_MODE, /* obsoleted */
- BSS_INFO_RA,
- BSS_INFO_HW_AMSDU,
- BSS_INFO_BSS_COLOR,
- BSS_INFO_HE_BASIC,
- BSS_INFO_PROTECT_INFO,
- BSS_INFO_OFFLOAD,
- BSS_INFO_11V_MBSSID,
- BSS_INFO_MAX_NUM
-};
-
-enum {
- WTBL_RESET_AND_SET = 1,
- WTBL_SET,
- WTBL_QUERY,
- WTBL_RESET_ALL
-};
-
-struct wtbl_req_hdr {
- u8 wlan_idx_lo;
- u8 operation;
- __le16 tlv_num;
- u8 wlan_idx_hi;
- u8 rsv[3];
-} __packed;
-
-struct wtbl_generic {
- __le16 tag;
- __le16 len;
- u8 peer_addr[ETH_ALEN];
- u8 muar_idx;
- u8 skip_tx;
- u8 cf_ack;
- u8 qos;
- u8 mesh;
- u8 adm;
- __le16 partial_aid;
- u8 baf_en;
- u8 aad_om;
-} __packed;
-
-struct wtbl_rx {
- __le16 tag;
- __le16 len;
- u8 rcid;
- u8 rca1;
- u8 rca2;
- u8 rv;
- u8 rsv[4];
-} __packed;
-
-struct wtbl_ht {
- __le16 tag;
- __le16 len;
- u8 ht;
- u8 ldpc;
- u8 af;
- u8 mm;
- u8 rsv[4];
-} __packed;
-
-struct wtbl_vht {
- __le16 tag;
- __le16 len;
- u8 ldpc;
- u8 dyn_bw;
- u8 vht;
- u8 txop_ps;
- u8 rsv[4];
-} __packed;
-
-struct wtbl_hdr_trans {
- __le16 tag;
- __le16 len;
- u8 to_ds;
- u8 from_ds;
- u8 no_rx_trans;
- u8 _rsv;
-};
-
-enum {
- MT_BA_TYPE_INVALID,
- MT_BA_TYPE_ORIGINATOR,
- MT_BA_TYPE_RECIPIENT
-};
-
-enum {
- RST_BA_MAC_TID_MATCH,
- RST_BA_MAC_MATCH,
- RST_BA_NO_MATCH
-};
-
-struct wtbl_ba {
- __le16 tag;
- __le16 len;
- /* common */
- u8 tid;
- u8 ba_type;
- u8 rsv0[2];
- /* originator only */
- __le16 sn;
- u8 ba_en;
- u8 ba_winsize_idx;
- /* originator & recipient */
- __le16 ba_winsize;
- /* recipient only */
- u8 peer_addr[ETH_ALEN];
- u8 rst_ba_tid;
- u8 rst_ba_sel;
- u8 rst_ba_sb;
- u8 band_idx;
- u8 rsv1[4];
-} __packed;
-
-struct wtbl_smps {
- __le16 tag;
- __le16 len;
- u8 smps;
- u8 rsv[3];
-} __packed;
-
-enum {
- WTBL_GENERIC,
- WTBL_RX,
- WTBL_HT,
- WTBL_VHT,
- WTBL_PEER_PS, /* not used */
- WTBL_TX_PS,
- WTBL_HDR_TRANS,
- WTBL_SEC_KEY,
- WTBL_BA,
- WTBL_RDG, /* obsoleted */
- WTBL_PROTECT, /* not used */
- WTBL_CLEAR, /* not used */
- WTBL_BF,
- WTBL_SMPS,
- WTBL_RAW_DATA, /* debug only */
- WTBL_PN,
- WTBL_SPE,
- WTBL_MAX_NUM
-};
-
-struct sta_ntlv_hdr {
- u8 rsv[2];
- __le16 tlv_num;
-} __packed;
-
-struct sta_req_hdr {
- u8 bss_idx;
- u8 wlan_idx_lo;
- __le16 tlv_num;
- u8 is_tlv_append;
- u8 muar_idx;
- u8 wlan_idx_hi;
- u8 rsv;
-} __packed;
-
-struct sta_rec_basic {
- __le16 tag;
- __le16 len;
- __le32 conn_type;
- u8 conn_state;
- u8 qos;
- __le16 aid;
- u8 peer_addr[ETH_ALEN];
- __le16 extra_info;
-} __packed;
-
-struct sta_rec_ht {
- __le16 tag;
- __le16 len;
- __le16 ht_cap;
- u16 rsv;
-} __packed;
-
-struct sta_rec_vht {
- __le16 tag;
- __le16 len;
- __le32 vht_cap;
- __le16 vht_rx_mcs_map;
- __le16 vht_tx_mcs_map;
- u8 rts_bw_sig;
- u8 rsv[3];
-} __packed;
-
-struct sta_rec_uapsd {
- __le16 tag;
- __le16 len;
- u8 dac_map;
- u8 tac_map;
- u8 max_sp;
- u8 rsv0;
- __le16 listen_interval;
- u8 rsv1[2];
-} __packed;
-
-struct sta_rec_muru {
- __le16 tag;
- __le16 len;
-
- struct {
- bool ofdma_dl_en;
- bool ofdma_ul_en;
- bool mimo_dl_en;
- bool mimo_ul_en;
- u8 rsv[4];
- } cfg;
-
- struct {
- u8 punc_pream_rx;
- bool he_20m_in_40m_2g;
- bool he_20m_in_160m;
- bool he_80m_in_160m;
- bool lt16_sigb;
- bool rx_su_comp_sigb;
- bool rx_su_non_comp_sigb;
- u8 rsv;
- } ofdma_dl;
-
- struct {
- u8 t_frame_dur;
- u8 mu_cascading;
- u8 uo_ra;
- u8 he_2x996_tone;
- u8 rx_t_frame_11ac;
- u8 rsv[3];
- } ofdma_ul;
-
- struct {
- bool vht_mu_bfee;
- bool partial_bw_dl_mimo;
- u8 rsv[2];
- } mimo_dl;
-
- struct {
- bool full_ul_mimo;
- bool partial_ul_mimo;
- u8 rsv[2];
- } mimo_ul;
-} __packed;
-
-struct sta_rec_he {
- __le16 tag;
- __le16 len;
-
- __le32 he_cap;
-
- u8 t_frame_dur;
- u8 max_ampdu_exp;
- u8 bw_set;
- u8 device_class;
- u8 dcm_tx_mode;
- u8 dcm_tx_max_nss;
- u8 dcm_rx_mode;
- u8 dcm_rx_max_nss;
- u8 dcm_max_ru;
- u8 punc_pream_rx;
- u8 pkt_ext;
- u8 rsv1;
-
- __le16 max_nss_mcs[CMD_HE_MCS_BW_NUM];
-
- u8 rsv2[2];
-} __packed;
-
-struct sta_rec_ba {
- __le16 tag;
- __le16 len;
- u8 tid;
- u8 ba_type;
- u8 amsdu;
- u8 ba_en;
- __le16 ssn;
- __le16 winsize;
-} __packed;
-
-struct sta_rec_amsdu {
- __le16 tag;
- __le16 len;
- u8 max_amsdu_num;
- u8 max_mpdu_size;
- u8 amsdu_en;
- u8 rsv;
-} __packed;
-
-struct sec_key {
- u8 cipher_id;
- u8 cipher_len;
- u8 key_id;
- u8 key_len;
- u8 key[32];
-} __packed;
-
-struct sta_rec_sec {
- __le16 tag;
- __le16 len;
- u8 add;
- u8 n_cipher;
- u8 rsv[2];
-
- struct sec_key key[2];
-} __packed;
-
-struct sta_phy {
- u8 type;
- u8 flag;
- u8 stbc;
- u8 sgi;
- u8 bw;
- u8 ldpc;
- u8 mcs;
- u8 nss;
- u8 he_ltf;
-};
-
-struct sta_rec_ra {
- __le16 tag;
- __le16 len;
-
- u8 valid;
- u8 auto_rate;
- u8 phy_mode;
- u8 channel;
- u8 bw;
- u8 disable_cck;
- u8 ht_mcs32;
- u8 ht_gf;
- u8 ht_mcs[4];
- u8 mmps_mode;
- u8 gband_256;
- u8 af;
- u8 auth_wapi_mode;
- u8 rate_len;
-
- u8 supp_mode;
- u8 supp_cck_rate;
- u8 supp_ofdm_rate;
- __le32 supp_ht_mcs;
- __le16 supp_vht_mcs[4];
-
- u8 op_mode;
- u8 op_vht_chan_width;
- u8 op_vht_rx_nss;
- u8 op_vht_rx_nss_type;
-
- __le32 sta_cap;
-
- struct sta_phy phy;
-} __packed;
-
-struct sta_rec_ra_fixed {
- __le16 tag;
- __le16 len;
-
- __le32 field;
- u8 op_mode;
- u8 op_vht_chan_width;
- u8 op_vht_rx_nss;
- u8 op_vht_rx_nss_type;
-
- struct sta_phy phy;
-
- u8 spe_en;
- u8 short_preamble;
- u8 is_5g;
- u8 mmps_mode;
-} __packed;
-
-enum {
RATE_PARAM_FIXED = 3,
+ RATE_PARAM_MMPS_UPDATE = 5,
RATE_PARAM_FIXED_HE_LTF = 7,
RATE_PARAM_FIXED_MCS,
RATE_PARAM_FIXED_GI = 11,
@@ -975,120 +425,6 @@ enum {
#define RATE_CFG_PHY_TYPE GENMASK(27, 24)
#define RATE_CFG_HE_LTF GENMASK(31, 28)
-struct sta_rec_bf {
- __le16 tag;
- __le16 len;
-
- __le16 pfmu; /* 0xffff: no access right for PFMU */
- bool su_mu; /* 0: SU, 1: MU */
- u8 bf_cap; /* 0: iBF, 1: eBF */
- u8 sounding_phy; /* 0: legacy, 1: OFDM, 2: HT, 4: VHT */
- u8 ndpa_rate;
- u8 ndp_rate;
- u8 rept_poll_rate;
- u8 tx_mode; /* 0: legacy, 1: OFDM, 2: HT, 4: VHT ... */
- u8 ncol;
- u8 nrow;
- u8 bw; /* 0: 20M, 1: 40M, 2: 80M, 3: 160M */
-
- u8 mem_total;
- u8 mem_20m;
- struct {
- u8 row;
- u8 col: 6, row_msb: 2;
- } mem[4];
-
- __le16 smart_ant;
- u8 se_idx;
- u8 auto_sounding; /* b7: low traffic indicator
- * b6: Stop sounding for this entry
- * b5 ~ b0: postpone sounding
- */
- u8 ibf_timeout;
- u8 ibf_dbw;
- u8 ibf_ncol;
- u8 ibf_nrow;
- u8 nrow_bw160;
- u8 ncol_bw160;
- u8 ru_start_idx;
- u8 ru_end_idx;
-
- bool trigger_su;
- bool trigger_mu;
- bool ng16_su;
- bool ng16_mu;
- bool codebook42_su;
- bool codebook75_mu;
-
- u8 he_ltf;
- u8 rsv[3];
-} __packed;
-
-struct sta_rec_bfee {
- __le16 tag;
- __le16 len;
- bool fb_identity_matrix; /* 1: feedback identity matrix */
- bool ignore_feedback; /* 1: ignore */
- u8 rsv[2];
-} __packed;
-
-enum {
- STA_REC_BASIC,
- STA_REC_RA,
- STA_REC_RA_CMM_INFO,
- STA_REC_RA_UPDATE,
- STA_REC_BF,
- STA_REC_AMSDU,
- STA_REC_BA,
- STA_REC_RED, /* not used */
- STA_REC_TX_PROC, /* for hdr trans and CSO in CR4 */
- STA_REC_HT,
- STA_REC_VHT,
- STA_REC_APPS,
- STA_REC_KEY,
- STA_REC_WTBL,
- STA_REC_HE,
- STA_REC_HW_AMSDU,
- STA_REC_WTBL_AADOM,
- STA_REC_KEY_V2,
- STA_REC_MURU,
- STA_REC_MUEDCA,
- STA_REC_BFEE,
- STA_REC_MAX_NUM
-};
-
-enum mcu_cipher_type {
- MCU_CIPHER_NONE = 0,
- MCU_CIPHER_WEP40,
- MCU_CIPHER_WEP104,
- MCU_CIPHER_WEP128,
- MCU_CIPHER_TKIP,
- MCU_CIPHER_AES_CCMP,
- MCU_CIPHER_CCMP_256,
- MCU_CIPHER_GCMP,
- MCU_CIPHER_GCMP_256,
- MCU_CIPHER_WAPI,
- MCU_CIPHER_BIP_CMAC_128,
-};
-
-enum {
- CH_SWITCH_NORMAL = 0,
- CH_SWITCH_SCAN = 3,
- CH_SWITCH_MCC = 4,
- CH_SWITCH_DFS = 5,
- CH_SWITCH_BACKGROUND_SCAN_START = 6,
- CH_SWITCH_BACKGROUND_SCAN_RUNNING = 7,
- CH_SWITCH_BACKGROUND_SCAN_STOP = 8,
- CH_SWITCH_SCAN_BYPASS_DPD = 9
-};
-
-enum {
- THERMAL_SENSOR_TEMP_QUERY,
- THERMAL_SENSOR_MANUAL_CTRL,
- THERMAL_SENSOR_INFO_QUERY,
- THERMAL_SENSOR_TASK_CTRL,
-};
-
enum {
THERMAL_PROTECT_PARAMETER_CTRL,
THERMAL_PROTECT_BASIC_INFO,
@@ -1116,28 +452,11 @@ enum {
MURU_PLATFORM_TYPE_PERF_LEVEL_2,
};
-#define MT7915_WTBL_UPDATE_MAX_SIZE (sizeof(struct wtbl_req_hdr) + \
- sizeof(struct wtbl_generic) + \
- sizeof(struct wtbl_rx) + \
- sizeof(struct wtbl_ht) + \
- sizeof(struct wtbl_vht) + \
- sizeof(struct wtbl_hdr_trans) +\
- sizeof(struct wtbl_ba) + \
- sizeof(struct wtbl_smps))
-
-#define MT7915_STA_UPDATE_MAX_SIZE (sizeof(struct sta_req_hdr) + \
- sizeof(struct sta_rec_basic) + \
- sizeof(struct sta_rec_bf) + \
- sizeof(struct sta_rec_ht) + \
- sizeof(struct sta_rec_he) + \
- sizeof(struct sta_rec_ba) + \
- sizeof(struct sta_rec_vht) + \
- sizeof(struct sta_rec_uapsd) + \
- sizeof(struct sta_rec_amsdu) + \
- sizeof(struct sta_rec_muru) + \
- sizeof(struct sta_rec_bfee) + \
- sizeof(struct tlv) + \
- MT7915_WTBL_UPDATE_MAX_SIZE)
+/* tx cmd tx statistics */
+enum {
+ MURU_SET_TXC_TX_STATS_EN = 150,
+ MURU_GET_TXC_TX_STATS = 151,
+};
#define MT7915_BSS_UPDATE_MAX_SIZE (sizeof(struct sta_req_hdr) + \
sizeof(struct bss_info_omac) + \
@@ -1154,62 +473,4 @@ enum {
sizeof(struct bss_info_bcn_mbss) + \
sizeof(struct bss_info_bcn_cont))
-#define PHY_MODE_A BIT(0)
-#define PHY_MODE_B BIT(1)
-#define PHY_MODE_G BIT(2)
-#define PHY_MODE_GN BIT(3)
-#define PHY_MODE_AN BIT(4)
-#define PHY_MODE_AC BIT(5)
-#define PHY_MODE_AX_24G BIT(6)
-#define PHY_MODE_AX_5G BIT(7)
-#define PHY_MODE_AX_6G BIT(8)
-
-#define MODE_CCK BIT(0)
-#define MODE_OFDM BIT(1)
-#define MODE_HT BIT(2)
-#define MODE_VHT BIT(3)
-#define MODE_HE BIT(4)
-
-#define STA_CAP_WMM BIT(0)
-#define STA_CAP_SGI_20 BIT(4)
-#define STA_CAP_SGI_40 BIT(5)
-#define STA_CAP_TX_STBC BIT(6)
-#define STA_CAP_RX_STBC BIT(7)
-#define STA_CAP_VHT_SGI_80 BIT(16)
-#define STA_CAP_VHT_SGI_160 BIT(17)
-#define STA_CAP_VHT_TX_STBC BIT(18)
-#define STA_CAP_VHT_RX_STBC BIT(19)
-#define STA_CAP_VHT_LDPC BIT(23)
-#define STA_CAP_LDPC BIT(24)
-#define STA_CAP_HT BIT(26)
-#define STA_CAP_VHT BIT(27)
-#define STA_CAP_HE BIT(28)
-
-/* HE MAC */
-#define STA_REC_HE_CAP_HTC BIT(0)
-#define STA_REC_HE_CAP_BQR BIT(1)
-#define STA_REC_HE_CAP_BSR BIT(2)
-#define STA_REC_HE_CAP_OM BIT(3)
-#define STA_REC_HE_CAP_AMSDU_IN_AMPDU BIT(4)
-/* HE PHY */
-#define STA_REC_HE_CAP_DUAL_BAND BIT(5)
-#define STA_REC_HE_CAP_LDPC BIT(6)
-#define STA_REC_HE_CAP_TRIG_CQI_FK BIT(7)
-#define STA_REC_HE_CAP_PARTIAL_BW_EXT_RANGE BIT(8)
-/* STBC */
-#define STA_REC_HE_CAP_LE_EQ_80M_TX_STBC BIT(9)
-#define STA_REC_HE_CAP_LE_EQ_80M_RX_STBC BIT(10)
-#define STA_REC_HE_CAP_GT_80M_TX_STBC BIT(11)
-#define STA_REC_HE_CAP_GT_80M_RX_STBC BIT(12)
-/* GI */
-#define STA_REC_HE_CAP_SU_PPDU_1LTF_8US_GI BIT(13)
-#define STA_REC_HE_CAP_SU_MU_PPDU_4LTF_8US_GI BIT(14)
-#define STA_REC_HE_CAP_ER_SU_PPDU_1LTF_8US_GI BIT(15)
-#define STA_REC_HE_CAP_ER_SU_PPDU_4LTF_8US_GI BIT(16)
-#define STA_REC_HE_CAP_NDP_4LTF_3DOT2MS_GI BIT(17)
-/* 242 TONE */
-#define STA_REC_HE_CAP_BW20_RU242_SUPPORT BIT(18)
-#define STA_REC_HE_CAP_TX_1024QAM_UNDER_RU242 BIT(19)
-#define STA_REC_HE_CAP_RX_1024QAM_UNDER_RU242 BIT(20)
-
#endif
diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/mt7915.h b/drivers/net/wireless/mediatek/mt76/mt7915/mt7915.h
index e69b4c8..42d8873 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7915/mt7915.h
+++ b/drivers/net/wireless/mediatek/mt76/mt7915/mt7915.h
@@ -30,6 +30,9 @@
#define MT7915_FIRMWARE_WM "mediatek/mt7915_wm.bin"
#define MT7915_ROM_PATCH "mediatek/mt7915_rom_patch.bin"
+#define MT7915_EEPROM_DEFAULT "mediatek/mt7915_eeprom.bin"
+#define MT7915_EEPROM_DEFAULT_DBDC "mediatek/mt7915_eeprom_dbdc.bin"
+
#define MT7915_EEPROM_SIZE 3584
#define MT7915_EEPROM_BLOCK_SIZE 16
#define MT7915_TOKEN_SIZE 8192
@@ -121,10 +124,7 @@ struct mt7915_vif_cap {
};
struct mt7915_vif {
- u16 idx;
- u8 omac_idx;
- u8 band_idx;
- u8 wmm_idx;
+ struct mt76_vif mt76; /* must be first */
struct mt7915_vif_cap cap;
struct mt7915_sta sta;
@@ -270,6 +270,7 @@ struct mt7915_dev {
bool dbdc_support;
bool flash_mode;
+ bool muru_debug;
bool ibf;
u8 fw_debug_wm;
u8 fw_debug_wa;
@@ -283,20 +284,6 @@ struct mt7915_dev {
};
enum {
- HW_BSSID_0 = 0x0,
- HW_BSSID_1,
- HW_BSSID_2,
- HW_BSSID_3,
- HW_BSSID_MAX = HW_BSSID_3,
- EXT_BSSID_START = 0x10,
- EXT_BSSID_1,
- EXT_BSSID_15 = 0x1f,
- EXT_BSSID_MAX = EXT_BSSID_15,
- REPEATER_BSSID_START = 0x20,
- REPEATER_BSSID_MAX = 0x3f,
-};
-
-enum {
MT_CTX0,
MT_HIF0 = 0x0,
@@ -423,6 +410,7 @@ int mt7915_mcu_set_fixed_rate_ctrl(struct mt7915_dev *dev,
void *data, u32 field);
int mt7915_mcu_set_eeprom(struct mt7915_dev *dev);
int mt7915_mcu_get_eeprom(struct mt7915_dev *dev, u32 offset);
+int mt7915_mcu_get_eeprom_free_block(struct mt7915_dev *dev, u8 *block_num);
int mt7915_mcu_set_mac(struct mt7915_dev *dev, int band, bool enable,
bool hdr_trans);
int mt7915_mcu_set_test_param(struct mt7915_dev *dev, u8 param, bool test_mode,
@@ -515,6 +503,7 @@ void mt7915_tx_token_put(struct mt7915_dev *dev);
int mt7915_init_tx_queues(struct mt7915_phy *phy, int idx, int n_desc);
void mt7915_queue_rx_skb(struct mt76_dev *mdev, enum mt76_rxq_id q,
struct sk_buff *skb);
+bool mt7915_rx_check(struct mt76_dev *mdev, void *data, int len);
void mt7915_sta_ps(struct mt76_dev *mdev, struct ieee80211_sta *sta, bool ps);
void mt7915_stats_work(struct work_struct *work);
int mt76_dfs_start_rdd(struct mt7915_dev *dev, bool force);
@@ -522,6 +511,8 @@ int mt7915_dfs_init_radar_detector(struct mt7915_phy *phy);
void mt7915_set_stream_he_caps(struct mt7915_phy *phy);
void mt7915_set_stream_vht_txbf_caps(struct mt7915_phy *phy);
void mt7915_update_channel(struct mt76_phy *mphy);
+int mt7915_mcu_muru_debug_set(struct mt7915_dev *dev, bool enable);
+int mt7915_mcu_muru_debug_get(struct mt7915_phy *phy, void *ms);
int mt7915_init_debugfs(struct mt7915_phy *phy);
#ifdef CONFIG_MAC80211_DEBUGFS
void mt7915_sta_add_debugfs(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/pci.c b/drivers/net/wireless/mediatek/mt76/mt7915/pci.c
index 0af4cdb..8130ea4 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7915/pci.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7915/pci.c
@@ -230,6 +230,7 @@ static int mt7915_pci_probe(struct pci_dev *pdev,
.tx_prepare_skb = mt7915_tx_prepare_skb,
.tx_complete_skb = mt7915_tx_complete_skb,
.rx_skb = mt7915_queue_rx_skb,
+ .rx_check = mt7915_rx_check,
.rx_poll_complete = mt7915_rx_poll_complete,
.sta_ps = mt7915_sta_ps,
.sta_add = mt7915_mac_sta_add,
diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/testmode.c b/drivers/net/wireless/mediatek/mt76/mt7915/testmode.c
index 89aae32..af80c2c 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7915/testmode.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7915/testmode.c
@@ -361,16 +361,15 @@ mt7915_tm_reg_backup_restore(struct mt7915_phy *phy)
return;
}
- if (b)
- return;
+ if (!b) {
+ b = devm_kzalloc(dev->mt76.dev, 4 * n_regs, GFP_KERNEL);
+ if (!b)
+ return;
- b = devm_kzalloc(dev->mt76.dev, 4 * n_regs, GFP_KERNEL);
- if (!b)
- return;
-
- phy->test.reg_backup = b;
- for (i = 0; i < n_regs; i++)
- b[i] = mt76_rr(dev, reg_backup_list[i].band[ext_phy]);
+ phy->test.reg_backup = b;
+ for (i = 0; i < n_regs; i++)
+ b[i] = mt76_rr(dev, reg_backup_list[i].band[ext_phy]);
+ }
mt76_clear(dev, MT_AGG_PCR0(ext_phy, 0), MT_AGG_PCR0_MM_PROT |
MT_AGG_PCR0_GF_PROT | MT_AGG_PCR0_ERP_PROT |
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/debugfs.c b/drivers/net/wireless/mediatek/mt76/mt7921/debugfs.c
index 7cdfdf8..86fd729 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7921/debugfs.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7921/debugfs.c
@@ -276,7 +276,7 @@ mt7921_pm_set(void *data, u64 val)
struct mt7921_dev *dev = data;
struct mt76_connac_pm *pm = &dev->pm;
- mt7921_mutex_acquire(dev);
+ mutex_lock(&dev->mt76.mutex);
if (val == pm->enable)
goto out;
@@ -285,7 +285,11 @@ mt7921_pm_set(void *data, u64 val)
pm->stats.last_wake_event = jiffies;
pm->stats.last_doze_event = jiffies;
}
- pm->enable = val;
+ /* make sure the chip is awake here and ps_work is scheduled
+ * just at end of the this routine.
+ */
+ pm->enable = false;
+ mt76_connac_pm_wake(&dev->mphy, pm);
ieee80211_iterate_active_interfaces(mt76_hw(dev),
IEEE80211_IFACE_ITER_RESUME_ALL,
@@ -293,8 +297,10 @@ mt7921_pm_set(void *data, u64 val)
mt76_connac_mcu_set_deep_sleep(&dev->mt76, pm->ds_enable);
+ pm->enable = val;
+ mt76_connac_power_save_sched(&dev->mphy, pm);
out:
- mt7921_mutex_release(dev);
+ mutex_unlock(&dev->mt76.mutex);
return 0;
}
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/init.c b/drivers/net/wireless/mediatek/mt76/mt7921/init.c
index 210998f..ad59ef9 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7921/init.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7921/init.c
@@ -76,14 +76,6 @@ mt7921_init_wiphy(struct ieee80211_hw *hw)
wiphy->max_sched_scan_reqs = 1;
wiphy->flags |= WIPHY_FLAG_HAS_CHANNEL_SWITCH;
wiphy->reg_notifier = mt7921_regd_notifier;
- wiphy->sar_capa = &mt76_sar_capa;
-
- phy->mt76->frp = devm_kcalloc(dev->mt76.dev,
- wiphy->sar_capa->num_freq_ranges,
- sizeof(struct mt76_freq_range_power),
- GFP_KERNEL);
- if (!phy->mt76->frp)
- return -ENOMEM;
wiphy->features |= NL80211_FEATURE_SCHED_SCAN_RANDOM_MAC_ADDR |
NL80211_FEATURE_SCAN_RANDOM_MAC_ADDR;
@@ -264,6 +256,10 @@ int mt7921_register_device(struct mt7921_dev *dev)
IEEE80211_VHT_CAP_SU_BEAMFORMEE_CAPABLE |
IEEE80211_VHT_CAP_MU_BEAMFORMEE_CAPABLE |
(3 << IEEE80211_VHT_CAP_BEAMFORMEE_STS_SHIFT);
+ if (is_mt7922(&dev->mt76))
+ dev->mphy.sband_5g.sband.vht_cap.cap |=
+ IEEE80211_VHT_CAP_SUPP_CHAN_WIDTH_160MHZ |
+ IEEE80211_VHT_CAP_SHORT_GI_160;
dev->mphy.hw->wiphy->available_antennas_rx = dev->mphy.chainmask;
dev->mphy.hw->wiphy->available_antennas_tx = dev->mphy.chainmask;
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/mac.c b/drivers/net/wireless/mediatek/mt76/mt7921/mac.c
index db3302b..ec10f95 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7921/mac.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7921/mac.c
@@ -218,10 +218,9 @@ mt7921_mac_decode_he_radiotap_ru(struct mt76_rx_status *status,
}
static void
-mt7921_mac_decode_he_mu_radiotap(struct sk_buff *skb,
- struct mt76_rx_status *status,
- __le32 *rxv)
+mt7921_mac_decode_he_mu_radiotap(struct sk_buff *skb, __le32 *rxv)
{
+ struct mt76_rx_status *status = (struct mt76_rx_status *)skb->cb;
static const struct ieee80211_radiotap_he_mu mu_known = {
.flags1 = HE_BITS(MU_FLAGS1_SIG_B_MCS_KNOWN) |
HE_BITS(MU_FLAGS1_SIG_B_DCM_KNOWN) |
@@ -233,6 +232,8 @@ mt7921_mac_decode_he_mu_radiotap(struct sk_buff *skb,
};
struct ieee80211_radiotap_he_mu *he_mu;
+ status->flag |= RX_FLAG_RADIOTAP_HE_MU;
+
he_mu = skb_push(skb, sizeof(mu_known));
memcpy(he_mu, &mu_known, sizeof(mu_known));
@@ -263,10 +264,9 @@ mt7921_mac_decode_he_mu_radiotap(struct sk_buff *skb,
}
static void
-mt7921_mac_decode_he_radiotap(struct sk_buff *skb,
- struct mt76_rx_status *status,
- __le32 *rxv, u32 phy)
+mt7921_mac_decode_he_radiotap(struct sk_buff *skb, __le32 *rxv, u32 mode)
{
+ struct mt76_rx_status *status = (struct mt76_rx_status *)skb->cb;
static const struct ieee80211_radiotap_he known = {
.data1 = HE_BITS(DATA1_DATA_MCS_KNOWN) |
HE_BITS(DATA1_DATA_DCM_KNOWN) |
@@ -284,6 +284,8 @@ mt7921_mac_decode_he_radiotap(struct sk_buff *skb,
struct ieee80211_radiotap_he *he = NULL;
u32 ltf_size = le32_get_bits(rxv[2], MT_CRXV_HE_LTF_SIZE) + 1;
+ status->flag |= RX_FLAG_RADIOTAP_HE;
+
he = skb_push(skb, sizeof(known));
memcpy(he, &known, sizeof(known));
@@ -298,7 +300,7 @@ mt7921_mac_decode_he_radiotap(struct sk_buff *skb,
he->data6 = HE_PREP(DATA6_TXOP, TXOP_DUR, rxv[14]) |
HE_PREP(DATA6_DOPPLER, DOPPLER, rxv[14]);
- switch (phy) {
+ switch (mode) {
case MT_PHY_TYPE_HE_SU:
he->data1 |= HE_BITS(DATA1_FORMAT_SU) |
HE_BITS(DATA1_UL_DL_KNOWN) |
@@ -322,6 +324,7 @@ mt7921_mac_decode_he_radiotap(struct sk_buff *skb,
he->data4 |= HE_PREP(DATA4_MU_STA_ID, MU_AID, rxv[7]);
mt7921_mac_decode_he_radiotap_ru(status, he, rxv);
+ mt7921_mac_decode_he_mu_radiotap(skb, rxv);
break;
case MT_PHY_TYPE_HE_TB:
he->data1 |= HE_BITS(DATA1_FORMAT_TRIG) |
@@ -395,6 +398,81 @@ mt7921_mac_assoc_rssi(struct mt7921_dev *dev, struct sk_buff *skb)
mt7921_mac_rssi_iter, skb);
}
+/* The HW does not translate the mac header to 802.3 for mesh point */
+static int mt7921_reverse_frag0_hdr_trans(struct sk_buff *skb, u16 hdr_gap)
+{
+ struct mt76_rx_status *status = (struct mt76_rx_status *)skb->cb;
+ struct mt7921_sta *msta = (struct mt7921_sta *)status->wcid;
+ struct ieee80211_sta *sta;
+ struct ieee80211_vif *vif;
+ struct ieee80211_hdr hdr;
+ struct ethhdr eth_hdr;
+ __le32 *rxd = (__le32 *)skb->data;
+ __le32 qos_ctrl, ht_ctrl;
+
+ if (FIELD_GET(MT_RXD3_NORMAL_ADDR_TYPE, le32_to_cpu(rxd[3])) !=
+ MT_RXD3_NORMAL_U2M)
+ return -EINVAL;
+
+ if (!(le32_to_cpu(rxd[1]) & MT_RXD1_NORMAL_GROUP_4))
+ return -EINVAL;
+
+ if (!msta || !msta->vif)
+ return -EINVAL;
+
+ sta = container_of((void *)msta, struct ieee80211_sta, drv_priv);
+ vif = container_of((void *)msta->vif, struct ieee80211_vif, drv_priv);
+
+ /* store the info from RXD and ethhdr to avoid being overridden */
+ memcpy(ð_hdr, skb->data + hdr_gap, sizeof(eth_hdr));
+ hdr.frame_control = FIELD_GET(MT_RXD6_FRAME_CONTROL, rxd[6]);
+ hdr.seq_ctrl = FIELD_GET(MT_RXD8_SEQ_CTRL, rxd[8]);
+ qos_ctrl = FIELD_GET(MT_RXD8_QOS_CTL, rxd[8]);
+ ht_ctrl = FIELD_GET(MT_RXD9_HT_CONTROL, rxd[9]);
+
+ hdr.duration_id = 0;
+ ether_addr_copy(hdr.addr1, vif->addr);
+ ether_addr_copy(hdr.addr2, sta->addr);
+ switch (le16_to_cpu(hdr.frame_control) &
+ (IEEE80211_FCTL_TODS | IEEE80211_FCTL_FROMDS)) {
+ case 0:
+ ether_addr_copy(hdr.addr3, vif->bss_conf.bssid);
+ break;
+ case IEEE80211_FCTL_FROMDS:
+ ether_addr_copy(hdr.addr3, eth_hdr.h_source);
+ break;
+ case IEEE80211_FCTL_TODS:
+ ether_addr_copy(hdr.addr3, eth_hdr.h_dest);
+ break;
+ case IEEE80211_FCTL_TODS | IEEE80211_FCTL_FROMDS:
+ ether_addr_copy(hdr.addr3, eth_hdr.h_dest);
+ ether_addr_copy(hdr.addr4, eth_hdr.h_source);
+ break;
+ default:
+ break;
+ }
+
+ skb_pull(skb, hdr_gap + sizeof(struct ethhdr) - 2);
+ if (eth_hdr.h_proto == htons(ETH_P_AARP) ||
+ eth_hdr.h_proto == htons(ETH_P_IPX))
+ ether_addr_copy(skb_push(skb, ETH_ALEN), bridge_tunnel_header);
+ else if (eth_hdr.h_proto >= htons(ETH_P_802_3_MIN))
+ ether_addr_copy(skb_push(skb, ETH_ALEN), rfc1042_header);
+ else
+ skb_pull(skb, 2);
+
+ if (ieee80211_has_order(hdr.frame_control))
+ memcpy(skb_push(skb, 2), &ht_ctrl, 2);
+ if (ieee80211_is_data_qos(hdr.frame_control))
+ memcpy(skb_push(skb, 2), &qos_ctrl, 2);
+ if (ieee80211_has_a4(hdr.frame_control))
+ memcpy(skb_push(skb, sizeof(hdr)), &hdr, sizeof(hdr));
+ else
+ memcpy(skb_push(skb, sizeof(hdr) - 6), &hdr, sizeof(hdr) - 6);
+
+ return 0;
+}
+
static int
mt7921_mac_fill_rx(struct mt7921_dev *dev, struct sk_buff *skb)
{
@@ -402,11 +480,11 @@ mt7921_mac_fill_rx(struct mt7921_dev *dev, struct sk_buff *skb)
struct mt76_rx_status *status = (struct mt76_rx_status *)skb->cb;
bool hdr_trans, unicast, insert_ccmp_hdr = false;
u8 chfreq, qos_ctl = 0, remove_pad, amsdu_info;
+ u16 hdr_gap;
__le32 *rxv = NULL, *rxd = (__le32 *)skb->data;
struct mt76_phy *mphy = &dev->mt76.phy;
struct mt7921_phy *phy = &dev->phy;
struct ieee80211_supported_band *sband;
- struct ieee80211_hdr *hdr;
u32 rxd0 = le32_to_cpu(rxd[0]);
u32 rxd1 = le32_to_cpu(rxd[1]);
u32 rxd2 = le32_to_cpu(rxd[2]);
@@ -428,10 +506,17 @@ mt7921_mac_fill_rx(struct mt7921_dev *dev, struct sk_buff *skb)
if (rxd2 & MT_RXD2_NORMAL_AMSDU_ERR)
return -EINVAL;
+ hdr_trans = rxd2 & MT_RXD2_NORMAL_HDR_TRANS;
+ if (hdr_trans && (rxd1 & MT_RXD1_NORMAL_CM))
+ return -EINVAL;
+
+ /* ICV error or CCMP/BIP/WPI MIC error */
+ if (rxd1 & MT_RXD1_NORMAL_ICV_ERR)
+ status->flag |= RX_FLAG_ONLY_MONITOR;
+
chfreq = FIELD_GET(MT_RXD3_NORMAL_CH_FREQ, rxd3);
unicast = FIELD_GET(MT_RXD3_NORMAL_ADDR_TYPE, rxd3) == MT_RXD3_NORMAL_U2M;
idx = FIELD_GET(MT_RXD1_NORMAL_WLAN_IDX, rxd1);
- hdr_trans = rxd2 & MT_RXD2_NORMAL_HDR_TRANS;
status->wcid = mt7921_rx_get_wcid(dev, idx, unicast);
if (status->wcid) {
@@ -612,15 +697,12 @@ mt7921_mac_fill_rx(struct mt7921_dev *dev, struct sk_buff *skb)
return -EINVAL;
break;
case MT_PHY_TYPE_HE_MU:
- status->flag |= RX_FLAG_RADIOTAP_HE_MU;
- fallthrough;
case MT_PHY_TYPE_HE_SU:
case MT_PHY_TYPE_HE_EXT_SU:
case MT_PHY_TYPE_HE_TB:
status->nss =
FIELD_GET(MT_PRXV_NSTS, v0) + 1;
status->encoding = RX_ENC_HE;
- status->flag |= RX_FLAG_RADIOTAP_HE;
i &= GENMASK(3, 0);
if (gi <= NL80211_RATE_INFO_HE_GI_3_2)
@@ -668,14 +750,21 @@ mt7921_mac_fill_rx(struct mt7921_dev *dev, struct sk_buff *skb)
}
}
- skb_pull(skb, (u8 *)rxd - skb->data + 2 * remove_pad);
-
amsdu_info = FIELD_GET(MT_RXD4_NORMAL_PAYLOAD_FORMAT, rxd4);
status->amsdu = !!amsdu_info;
if (status->amsdu) {
status->first_amsdu = amsdu_info == MT_RXD4_FIRST_AMSDU_FRAME;
status->last_amsdu = amsdu_info == MT_RXD4_LAST_AMSDU_FRAME;
- if (!hdr_trans) {
+ }
+
+ hdr_gap = (u8 *)rxd - skb->data + 2 * remove_pad;
+ if (hdr_trans && ieee80211_has_morefrags(fc)) {
+ if (mt7921_reverse_frag0_hdr_trans(skb, hdr_gap))
+ return -EINVAL;
+ hdr_trans = false;
+ } else {
+ skb_pull(skb, hdr_gap);
+ if (!hdr_trans && status->amsdu) {
memmove(skb->data + 2, skb->data,
ieee80211_get_hdrlen_from_skb(skb));
skb_pull(skb, 2);
@@ -683,6 +772,8 @@ mt7921_mac_fill_rx(struct mt7921_dev *dev, struct sk_buff *skb)
}
if (!hdr_trans) {
+ struct ieee80211_hdr *hdr;
+
if (insert_ccmp_hdr) {
u8 key_id = FIELD_GET(MT_RXD1_NORMAL_KEY_ID, rxd1);
@@ -696,19 +787,13 @@ mt7921_mac_fill_rx(struct mt7921_dev *dev, struct sk_buff *skb)
qos_ctl = *ieee80211_get_qos_ctl(hdr);
}
} else {
- status->flag &= ~(RX_FLAG_RADIOTAP_HE |
- RX_FLAG_RADIOTAP_HE_MU);
status->flag |= RX_FLAG_8023;
}
mt7921_mac_assoc_rssi(dev, skb);
- if (rxv && status->flag & RX_FLAG_RADIOTAP_HE) {
- mt7921_mac_decode_he_radiotap(skb, status, rxv, mode);
-
- if (status->flag & RX_FLAG_RADIOTAP_HE_MU)
- mt7921_mac_decode_he_mu_radiotap(skb, status, rxv);
- }
+ if (rxv && mode >= MT_PHY_TYPE_HE_SU && !(status->flag & RX_FLAG_8023))
+ mt7921_mac_decode_he_radiotap(skb, rxv, mode);
if (!status->wcid || !ieee80211_is_data_qos(fc))
return 0;
@@ -903,7 +988,7 @@ void mt7921_mac_write_txwi(struct mt7921_dev *dev, __le32 *txwi,
mt7921_mac_write_txwi_80211(dev, txwi, skb, key);
if (txwi[2] & cpu_to_le32(MT_TXD2_FIX_RATE)) {
- int rateidx = ffs(vif->bss_conf.basic_rates) - 1;
+ int rateidx = vif ? ffs(vif->bss_conf.basic_rates) - 1 : 0;
u16 rate, mode;
/* hardware won't add HTC for mgmt/ctrl frame */
@@ -1065,7 +1150,7 @@ mt7921_mac_add_txs_skb(struct mt7921_dev *dev, struct mt76_wcid *wcid, int pid,
return !!skb;
}
-static void mt7921_mac_add_txs(struct mt7921_dev *dev, void *data)
+void mt7921_mac_add_txs(struct mt7921_dev *dev, void *data)
{
struct mt7921_sta *msta = NULL;
struct mt76_wcid *wcid;
@@ -1314,6 +1399,7 @@ void mt7921_mac_reset_work(struct work_struct *work)
}
dev->hw_full_reset = false;
+ pm->suspended = false;
ieee80211_wake_queues(hw);
ieee80211_iterate_active_interfaces(hw,
IEEE80211_IFACE_ITER_RESUME_ALL,
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/main.c b/drivers/net/wireless/mediatek/mt76/mt7921/main.c
index 633c6d2..7a8d259 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7921/main.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7921/main.c
@@ -128,11 +128,23 @@ mt7921_init_he_caps(struct mt7921_phy *phy, enum nl80211_band band,
IEEE80211_HE_PHY_CAP9_RX_1024_QAM_LESS_THAN_242_TONE_RU |
IEEE80211_HE_PHY_CAP9_RX_FULL_BW_SU_USING_MU_WITH_COMP_SIGB |
IEEE80211_HE_PHY_CAP9_RX_FULL_BW_SU_USING_MU_WITH_NON_COMP_SIGB;
+
+ if (is_mt7922(phy->mt76->dev)) {
+ he_cap_elem->phy_cap_info[0] |=
+ IEEE80211_HE_PHY_CAP0_CHANNEL_WIDTH_SET_160MHZ_IN_5G;
+ he_cap_elem->phy_cap_info[8] |=
+ IEEE80211_HE_PHY_CAP8_20MHZ_IN_160MHZ_HE_PPDU |
+ IEEE80211_HE_PHY_CAP8_80MHZ_IN_160MHZ_HE_PPDU;
+ }
break;
}
he_mcs->rx_mcs_80 = cpu_to_le16(mcs_map);
he_mcs->tx_mcs_80 = cpu_to_le16(mcs_map);
+ if (is_mt7922(phy->mt76->dev)) {
+ he_mcs->rx_mcs_160 = cpu_to_le16(mcs_map);
+ he_mcs->tx_mcs_160 = cpu_to_le16(mcs_map);
+ }
memset(he_cap->ppe_thres, 0, sizeof(he_cap->ppe_thres));
if (he_cap_elem->phy_cap_info[6] &
@@ -140,7 +152,8 @@ mt7921_init_he_caps(struct mt7921_phy *phy, enum nl80211_band band,
mt7921_gen_ppe_thresh(he_cap->ppe_thres, nss);
} else {
he_cap_elem->phy_cap_info[9] |=
- IEEE80211_HE_PHY_CAP9_NOMIMAL_PKT_PADDING_16US;
+ u8_encode_bits(IEEE80211_HE_PHY_CAP9_NOMINAL_PKT_PADDING_16US,
+ IEEE80211_HE_PHY_CAP9_NOMINAL_PKT_PADDING_MASK);
}
if (band == NL80211_BAND_6GHZ) {
@@ -166,7 +179,7 @@ mt7921_init_he_caps(struct mt7921_phy *phy, enum nl80211_band band,
if (vht_cap->cap & IEEE80211_VHT_CAP_RX_ANTENNA_PATTERN)
cap |= IEEE80211_HE_6GHZ_CAP_RX_ANTPAT_CONS;
- data->he_6ghz_capa.capa = cpu_to_le16(cap);
+ data[idx].he_6ghz_capa.capa = cpu_to_le16(cap);
}
idx++;
}
@@ -221,7 +234,7 @@ int __mt7921_start(struct mt7921_phy *phy)
if (err)
return err;
- err = mt7921_mcu_set_chan_info(phy, MCU_EXT_CMD_SET_RX_PATH);
+ err = mt7921_mcu_set_chan_info(phy, MCU_EXT_CMD(SET_RX_PATH));
if (err)
return err;
@@ -318,12 +331,6 @@ static int mt7921_add_interface(struct ieee80211_hw *hw,
mtxq->wcid = &mvif->sta.wcid;
}
- if (vif->type != NL80211_IFTYPE_AP &&
- (!mvif->mt76.omac_idx || mvif->mt76.omac_idx > 3))
- vif->offload_flags = 0;
-
- vif->offload_flags |= IEEE80211_OFFLOAD_ENCAP_4ADDR;
-
out:
mt7921_mutex_release(dev);
@@ -369,7 +376,7 @@ static int mt7921_set_channel(struct mt7921_phy *phy)
mt76_set_channel(phy->mt76);
- ret = mt7921_mcu_set_chan_info(phy, MCU_EXT_CMD_CHANNEL_SWITCH);
+ ret = mt7921_mcu_set_chan_info(phy, MCU_EXT_CMD(CHANNEL_SWITCH));
if (ret)
goto out;
@@ -462,7 +469,7 @@ static int mt7921_config(struct ieee80211_hw *hw, u32 changed)
{
struct mt7921_dev *dev = mt7921_hw_dev(hw);
struct mt7921_phy *phy = mt7921_hw_phy(hw);
- int ret;
+ int ret = 0;
if (changed & IEEE80211_CONF_CHANGE_CHANNEL) {
ieee80211_stop_queues(hw);
@@ -474,8 +481,11 @@ static int mt7921_config(struct ieee80211_hw *hw, u32 changed)
mt7921_mutex_acquire(dev);
- if (changed & IEEE80211_CONF_CHANGE_POWER)
- mt76_connac_mcu_set_rate_txpower(phy->mt76);
+ if (changed & IEEE80211_CONF_CHANGE_POWER) {
+ ret = mt76_connac_mcu_set_rate_txpower(phy->mt76);
+ if (ret)
+ goto out;
+ }
if (changed & IEEE80211_CONF_CHANGE_MONITOR) {
bool enabled = !!(hw->conf.flags & IEEE80211_CONF_MONITOR);
@@ -490,9 +500,10 @@ static int mt7921_config(struct ieee80211_hw *hw, u32 changed)
mt76_wr(dev, MT_WF_RFCR(0), phy->rxfilter);
}
+out:
mt7921_mutex_release(dev);
- return 0;
+ return ret;
}
static int
@@ -1238,7 +1249,6 @@ static int mt7921_suspend(struct ieee80211_hw *hw,
{
struct mt7921_dev *dev = mt7921_hw_dev(hw);
struct mt7921_phy *phy = mt7921_hw_phy(hw);
- int err;
cancel_delayed_work_sync(&phy->scan_work);
cancel_delayed_work_sync(&phy->mt76->mac_work);
@@ -1249,34 +1259,24 @@ static int mt7921_suspend(struct ieee80211_hw *hw,
mt7921_mutex_acquire(dev);
clear_bit(MT76_STATE_RUNNING, &phy->mt76->state);
-
- set_bit(MT76_STATE_SUSPEND, &phy->mt76->state);
ieee80211_iterate_active_interfaces(hw,
IEEE80211_IFACE_ITER_RESUME_ALL,
mt76_connac_mcu_set_suspend_iter,
&dev->mphy);
- err = mt76_connac_mcu_set_hif_suspend(&dev->mt76, true);
-
mt7921_mutex_release(dev);
- return err;
+ return 0;
}
static int mt7921_resume(struct ieee80211_hw *hw)
{
struct mt7921_dev *dev = mt7921_hw_dev(hw);
struct mt7921_phy *phy = mt7921_hw_phy(hw);
- int err;
mt7921_mutex_acquire(dev);
- err = mt76_connac_mcu_set_hif_suspend(&dev->mt76, false);
- if (err < 0)
- goto out;
-
set_bit(MT76_STATE_RUNNING, &phy->mt76->state);
- clear_bit(MT76_STATE_SUSPEND, &phy->mt76->state);
ieee80211_iterate_active_interfaces(hw,
IEEE80211_IFACE_ITER_RESUME_ALL,
mt76_connac_mcu_set_suspend_iter,
@@ -1284,11 +1284,10 @@ static int mt7921_resume(struct ieee80211_hw *hw)
ieee80211_queue_delayed_work(hw, &phy->mt76->mac_work,
MT7921_WATCHDOG_TIME);
-out:
mt7921_mutex_release(dev);
- return err;
+ return 0;
}
static void mt7921_set_wakeup(struct ieee80211_hw *hw, bool enabled)
@@ -1334,41 +1333,23 @@ static void mt7921_sta_set_decap_offload(struct ieee80211_hw *hw,
clear_bit(MT_WCID_FLAG_HDR_TRANS, &msta->wcid.flags);
mt76_connac_mcu_sta_update_hdr_trans(&dev->mt76, vif, &msta->wcid,
- MCU_UNI_CMD_STA_REC_UPDATE);
+ MCU_UNI_CMD(STA_REC_UPDATE));
}
static int mt7921_set_sar_specs(struct ieee80211_hw *hw,
const struct cfg80211_sar_specs *sar)
{
- const struct cfg80211_sar_capa *capa = hw->wiphy->sar_capa;
struct mt7921_dev *dev = mt7921_hw_dev(hw);
- struct mt76_freq_range_power *data, *frp;
struct mt76_phy *mphy = hw->priv;
int err;
- u32 i;
-
- if (sar->type != NL80211_SAR_TYPE_POWER || !sar->num_sub_specs)
- return -EINVAL;
mt7921_mutex_acquire(dev);
-
- data = mphy->frp;
-
- for (i = 0; i < sar->num_sub_specs; i++) {
- u32 index = sar->sub_specs[i].freq_range_index;
- /* SAR specifies power limitaton in 0.25dbm */
- s32 power = sar->sub_specs[i].power >> 1;
-
- if (power > 127 || power < -127)
- power = 127;
-
- frp = &data[index];
- frp->range = &capa->freq_ranges[index];
- frp->power = power;
- }
+ err = mt76_init_sar_power(hw, sar);
+ if (err)
+ goto out;
err = mt76_connac_mcu_set_rate_txpower(mphy);
-
+out:
mt7921_mutex_release(dev);
return err;
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/mcu.c b/drivers/net/wireless/mediatek/mt76/mt7921/mcu.c
index 6ada1eb..ef1e1ef 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7921/mcu.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7921/mcu.c
@@ -93,9 +93,6 @@ struct mt7921_fw_region {
#define PATCH_SEC_ENC_SCRAMBLE_INFO_MASK GENMASK(15, 0)
#define PATCH_SEC_ENC_AES_KEY_MASK GENMASK(7, 0)
-#define to_wcid_lo(id) FIELD_GET(GENMASK(7, 0), (u16)id)
-#define to_wcid_hi(id) FIELD_GET(GENMASK(9, 8), (u16)id)
-
static enum mcu_cipher_type
mt7921_mcu_get_cipher(int cipher)
{
@@ -163,8 +160,8 @@ mt7921_mcu_parse_eeprom(struct mt76_dev *dev, struct sk_buff *skb)
int mt7921_mcu_parse_response(struct mt76_dev *mdev, int cmd,
struct sk_buff *skb, int seq)
{
+ int mcu_cmd = FIELD_GET(__MCU_CMD_FIELD_ID, cmd);
struct mt7921_mcu_rxd *rxd;
- int mcu_cmd = cmd & MCU_CMD_MASK;
int ret = 0;
if (!skb) {
@@ -179,24 +176,20 @@ int mt7921_mcu_parse_response(struct mt76_dev *mdev, int cmd,
if (seq != rxd->seq)
return -EAGAIN;
- switch (cmd) {
- case MCU_CMD_PATCH_SEM_CONTROL:
+ if (cmd == MCU_CMD(PATCH_SEM_CONTROL)) {
skb_pull(skb, sizeof(*rxd) - 4);
ret = *skb->data;
- break;
- case MCU_EXT_CMD_GET_TEMP:
+ } else if (cmd == MCU_EXT_CMD(THERMAL_CTRL)) {
skb_pull(skb, sizeof(*rxd) + 4);
ret = le32_to_cpu(*(__le32 *)skb->data);
- break;
- case MCU_EXT_CMD_EFUSE_ACCESS:
+ } else if (cmd == MCU_EXT_CMD(EFUSE_ACCESS)) {
ret = mt7921_mcu_parse_eeprom(mdev, skb);
- break;
- case MCU_UNI_CMD_DEV_INFO_UPDATE:
- case MCU_UNI_CMD_BSS_INFO_UPDATE:
- case MCU_UNI_CMD_STA_REC_UPDATE:
- case MCU_UNI_CMD_HIF_CTRL:
- case MCU_UNI_CMD_OFFLOAD:
- case MCU_UNI_CMD_SUSPEND: {
+ } else if (cmd == MCU_UNI_CMD(DEV_INFO_UPDATE) ||
+ cmd == MCU_UNI_CMD(BSS_INFO_UPDATE) ||
+ cmd == MCU_UNI_CMD(STA_REC_UPDATE) ||
+ cmd == MCU_UNI_CMD(HIF_CTRL) ||
+ cmd == MCU_UNI_CMD(OFFLOAD) ||
+ cmd == MCU_UNI_CMD(SUSPEND)) {
struct mt7921_mcu_uni_event *event;
skb_pull(skb, sizeof(*rxd));
@@ -205,19 +198,14 @@ int mt7921_mcu_parse_response(struct mt76_dev *mdev, int cmd,
/* skip invalid event */
if (mcu_cmd != event->cid)
ret = -EAGAIN;
- break;
- }
- case MCU_CMD_REG_READ: {
+ } else if (cmd == MCU_CE_QUERY(REG_READ)) {
struct mt7921_mcu_reg_event *event;
skb_pull(skb, sizeof(*rxd));
event = (struct mt7921_mcu_reg_event *)skb->data;
ret = (int)le32_to_cpu(event->val);
- break;
- }
- default:
+ } else {
skb_pull(skb, sizeof(struct mt7921_mcu_rxd));
- break;
}
return ret;
@@ -228,32 +216,28 @@ int mt7921_mcu_fill_message(struct mt76_dev *mdev, struct sk_buff *skb,
int cmd, int *wait_seq)
{
struct mt7921_dev *dev = container_of(mdev, struct mt7921_dev, mt76);
- int txd_len, mcu_cmd = cmd & MCU_CMD_MASK;
+ int txd_len, mcu_cmd = FIELD_GET(__MCU_CMD_FIELD_ID, cmd);
struct mt7921_uni_txd *uni_txd;
struct mt7921_mcu_txd *mcu_txd;
__le32 *txd;
u32 val;
u8 seq;
- switch (cmd) {
- case MCU_UNI_CMD_HIF_CTRL:
- case MCU_UNI_CMD_SUSPEND:
- case MCU_UNI_CMD_OFFLOAD:
- mdev->mcu.timeout = HZ / 3;
- break;
- default:
+ if (cmd == MCU_UNI_CMD(HIF_CTRL) ||
+ cmd == MCU_UNI_CMD(SUSPEND) ||
+ cmd == MCU_UNI_CMD(OFFLOAD))
+ mdev->mcu.timeout = HZ;
+ else
mdev->mcu.timeout = 3 * HZ;
- break;
- }
seq = ++dev->mt76.mcu.msg_seq & 0xf;
if (!seq)
seq = ++dev->mt76.mcu.msg_seq & 0xf;
- if (cmd == MCU_CMD_FW_SCATTER)
+ if (cmd == MCU_CMD(FW_SCATTER))
goto exit;
- txd_len = cmd & MCU_UNI_PREFIX ? sizeof(*uni_txd) : sizeof(*mcu_txd);
+ txd_len = cmd & __MCU_CMD_FIELD_UNI ? sizeof(*uni_txd) : sizeof(*mcu_txd);
txd = (__le32 *)skb_push(skb, txd_len);
val = FIELD_PREP(MT_TXD0_TX_BYTES, skb->len) |
@@ -265,7 +249,7 @@ int mt7921_mcu_fill_message(struct mt76_dev *mdev, struct sk_buff *skb,
FIELD_PREP(MT_TXD1_HDR_FORMAT, MT_HDR_FORMAT_CMD);
txd[1] = cpu_to_le32(val);
- if (cmd & MCU_UNI_PREFIX) {
+ if (cmd & __MCU_CMD_FIELD_UNI) {
uni_txd = (struct mt7921_uni_txd *)txd;
uni_txd->len = cpu_to_le16(skb->len - sizeof(uni_txd->txd));
uni_txd->option = MCU_CMD_UNI_EXT_ACK;
@@ -283,33 +267,19 @@ int mt7921_mcu_fill_message(struct mt76_dev *mdev, struct sk_buff *skb,
MT_TX_MCU_PORT_RX_Q0));
mcu_txd->pkt_type = MCU_PKT_ID;
mcu_txd->seq = seq;
-
- switch (cmd & ~MCU_CMD_MASK) {
- case MCU_FW_PREFIX:
- mcu_txd->set_query = MCU_Q_NA;
- mcu_txd->cid = mcu_cmd;
- break;
- case MCU_CE_PREFIX:
- if (cmd & MCU_QUERY_MASK)
- mcu_txd->set_query = MCU_Q_QUERY;
- else
- mcu_txd->set_query = MCU_Q_SET;
- mcu_txd->cid = mcu_cmd;
- break;
- default:
- mcu_txd->cid = MCU_CMD_EXT_CID;
- if (cmd & MCU_QUERY_PREFIX || cmd == MCU_EXT_CMD_EFUSE_ACCESS)
- mcu_txd->set_query = MCU_Q_QUERY;
- else
- mcu_txd->set_query = MCU_Q_SET;
- mcu_txd->ext_cid = mcu_cmd;
- mcu_txd->ext_cid_ack = 1;
- break;
- }
-
+ mcu_txd->cid = mcu_cmd;
mcu_txd->s2d_index = MCU_S2D_H2N;
- WARN_ON(cmd == MCU_EXT_CMD_EFUSE_ACCESS &&
- mcu_txd->set_query != MCU_Q_QUERY);
+ mcu_txd->ext_cid = FIELD_GET(__MCU_CMD_FIELD_EXT_ID, cmd);
+
+ if (mcu_txd->ext_cid || (cmd & __MCU_CMD_FIELD_CE)) {
+ if (cmd & __MCU_CMD_FIELD_QUERY)
+ mcu_txd->set_query = MCU_Q_QUERY;
+ else
+ mcu_txd->set_query = MCU_Q_SET;
+ mcu_txd->ext_cid_ack = !!mcu_txd->ext_cid;
+ } else {
+ mcu_txd->set_query = MCU_Q_NA;
+ }
exit:
if (wait_seq)
@@ -419,6 +389,17 @@ mt7921_mcu_low_power_event(struct mt7921_dev *dev, struct sk_buff *skb)
}
static void
+mt7921_mcu_tx_done_event(struct mt7921_dev *dev, struct sk_buff *skb)
+{
+ struct mt7921_mcu_tx_done_event *event;
+
+ skb_pull(skb, sizeof(struct mt7921_mcu_rxd));
+ event = (struct mt7921_mcu_tx_done_event *)skb->data;
+
+ mt7921_mac_add_txs(dev, event->txs);
+}
+
+static void
mt7921_mcu_rx_unsolicited_event(struct mt7921_dev *dev, struct sk_buff *skb)
{
struct mt7921_mcu_rxd *rxd = (struct mt7921_mcu_rxd *)skb->data;
@@ -445,6 +426,9 @@ mt7921_mcu_rx_unsolicited_event(struct mt7921_dev *dev, struct sk_buff *skb)
case MCU_EVENT_LP_INFO:
mt7921_mcu_low_power_event(dev, skb);
break;
+ case MCU_EVENT_TX_DONE:
+ mt7921_mcu_tx_done_event(dev, skb);
+ break;
default:
break;
}
@@ -567,7 +551,7 @@ int mt7921_mcu_add_key(struct mt7921_dev *dev, struct ieee80211_vif *vif,
return ret;
return mt76_mcu_skb_send_msg(&dev->mt76, skb,
- MCU_UNI_CMD_STA_REC_UPDATE, true);
+ MCU_UNI_CMD(STA_REC_UPDATE), true);
}
int mt7921_mcu_uni_tx_ba(struct mt7921_dev *dev,
@@ -602,7 +586,7 @@ int mt7921_mcu_restart(struct mt76_dev *dev)
.power_mode = 1,
};
- return mt76_mcu_send_msg(dev, MCU_CMD_NIC_POWER_CTRL, &req,
+ return mt76_mcu_send_msg(dev, MCU_CMD(NIC_POWER_CTRL), &req,
sizeof(req), false);
}
EXPORT_SYMBOL_GPL(mt7921_mcu_restart);
@@ -708,7 +692,7 @@ static int mt7921_load_patch(struct mt7921_dev *dev)
goto out;
}
- ret = __mt76_mcu_send_firmware(&dev->mt76, MCU_CMD_FW_SCATTER,
+ ret = __mt76_mcu_send_firmware(&dev->mt76, MCU_CMD(FW_SCATTER),
dl, len, max_len);
if (ret) {
dev_err(dev->mt76.dev, "Failed to send patch\n");
@@ -720,6 +704,17 @@ static int mt7921_load_patch(struct mt7921_dev *dev)
if (ret)
dev_err(dev->mt76.dev, "Failed to start patch\n");
+ if (mt76_is_sdio(&dev->mt76)) {
+ /* activate again */
+ ret = __mt7921_mcu_fw_pmctrl(dev);
+ if (ret)
+ return ret;
+
+ ret = __mt7921_mcu_drv_pmctrl(dev);
+ if (ret)
+ return ret;
+ }
+
out:
sem = mt76_connac_mcu_patch_sem_ctrl(&dev->mt76, false);
switch (sem) {
@@ -782,7 +777,7 @@ mt7921_mcu_send_ram_firmware(struct mt7921_dev *dev,
return err;
}
- err = __mt76_mcu_send_firmware(&dev->mt76, MCU_CMD_FW_SCATTER,
+ err = __mt76_mcu_send_firmware(&dev->mt76, MCU_CMD(FW_SCATTER),
data + offset, len, max_len);
if (err) {
dev_err(dev->mt76.dev, "Failed to send firmware.\n");
@@ -882,7 +877,7 @@ static int mt7921_load_firmware(struct mt7921_dev *dev)
dev->mt76.hw->wiphy->wowlan = &mt76_connac_wowlan_support;
#endif /* CONFIG_PM */
- dev_err(dev->mt76.dev, "Firmware init done\n");
+ dev_dbg(dev->mt76.dev, "Firmware init done\n");
return 0;
}
@@ -896,8 +891,8 @@ int mt7921_mcu_fw_log_2_host(struct mt7921_dev *dev, u8 ctrl)
.ctrl_val = ctrl
};
- return mt76_mcu_send_msg(&dev->mt76, MCU_CMD_FWLOG_2_HOST, &data,
- sizeof(data), false);
+ return mt76_mcu_send_msg(&dev->mt76, MCU_CE_CMD(FWLOG_2_HOST),
+ &data, sizeof(data), false);
}
int mt7921_run_firmware(struct mt7921_dev *dev)
@@ -997,8 +992,8 @@ int mt7921_mcu_set_tx(struct mt7921_dev *dev, struct ieee80211_vif *vif)
e->cw_max = cpu_to_le16(10);
}
- ret = mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD_EDCA_UPDATE, &req,
- sizeof(req), true);
+ ret = mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD(EDCA_UPDATE),
+ &req, sizeof(req), true);
if (ret)
return ret;
@@ -1022,8 +1017,8 @@ int mt7921_mcu_set_tx(struct mt7921_dev *dev, struct ieee80211_vif *vif)
e->timer = q->mu_edca_timer;
}
- return mt76_mcu_send_msg(&dev->mt76, MCU_CMD_SET_MU_EDCA_PARMS, &req_mu,
- sizeof(req_mu), false);
+ return mt76_mcu_send_msg(&dev->mt76, MCU_CE_CMD(SET_MU_EDCA_PARMS),
+ &req_mu, sizeof(req_mu), false);
}
int mt7921_mcu_set_chan_info(struct mt7921_phy *phy, int cmd)
@@ -1070,7 +1065,7 @@ int mt7921_mcu_set_chan_info(struct mt7921_phy *phy, int cmd)
else
req.switch_reason = CH_SWITCH_NORMAL;
- if (cmd == MCU_EXT_CMD_CHANNEL_SWITCH)
+ if (cmd == MCU_EXT_CMD(CHANNEL_SWITCH))
req.rx_streams = hweight8(req.rx_streams);
if (chandef->width == NL80211_CHAN_WIDTH_80P80) {
@@ -1093,7 +1088,7 @@ int mt7921_mcu_set_eeprom(struct mt7921_dev *dev)
.format = EE_FORMAT_WHOLE,
};
- return mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD_EFUSE_BUFFER_MODE,
+ return mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD(EFUSE_BUFFER_MODE),
&req, sizeof(req), true);
}
EXPORT_SYMBOL_GPL(mt7921_mcu_set_eeprom);
@@ -1108,8 +1103,9 @@ int mt7921_mcu_get_eeprom(struct mt7921_dev *dev, u32 offset)
int ret;
u8 *buf;
- ret = mt76_mcu_send_and_get_msg(&dev->mt76, MCU_EXT_CMD_EFUSE_ACCESS, &req,
- sizeof(req), true, &skb);
+ ret = mt76_mcu_send_and_get_msg(&dev->mt76,
+ MCU_EXT_QUERY(EFUSE_ACCESS),
+ &req, sizeof(req), true, &skb);
if (ret)
return ret;
@@ -1154,7 +1150,7 @@ int mt7921_mcu_uni_bss_ps(struct mt7921_dev *dev, struct ieee80211_vif *vif)
if (vif->type != NL80211_IFTYPE_STATION)
return -EOPNOTSUPP;
- return mt76_mcu_send_msg(&dev->mt76, MCU_UNI_CMD_BSS_INFO_UPDATE,
+ return mt76_mcu_send_msg(&dev->mt76, MCU_UNI_CMD(BSS_INFO_UPDATE),
&ps_req, sizeof(ps_req), true);
}
@@ -1190,7 +1186,7 @@ mt7921_mcu_uni_bss_bcnft(struct mt7921_dev *dev, struct ieee80211_vif *vif,
if (vif->type != NL80211_IFTYPE_STATION)
return 0;
- return mt76_mcu_send_msg(&dev->mt76, MCU_UNI_CMD_BSS_INFO_UPDATE,
+ return mt76_mcu_send_msg(&dev->mt76, MCU_UNI_CMD(BSS_INFO_UPDATE),
&bcnft_req, sizeof(bcnft_req), true);
}
@@ -1226,13 +1222,13 @@ mt7921_mcu_set_bss_pm(struct mt7921_dev *dev, struct ieee80211_vif *vif,
if (vif->type != NL80211_IFTYPE_STATION)
return 0;
- err = mt76_mcu_send_msg(&dev->mt76, MCU_CMD_SET_BSS_ABORT, &req_hdr,
- sizeof(req_hdr), false);
+ err = mt76_mcu_send_msg(&dev->mt76, MCU_CE_CMD(SET_BSS_ABORT),
+ &req_hdr, sizeof(req_hdr), false);
if (err < 0 || !enable)
return err;
- return mt76_mcu_send_msg(&dev->mt76, MCU_CMD_SET_BSS_CONNECTED, &req,
- sizeof(req), false);
+ return mt76_mcu_send_msg(&dev->mt76, MCU_CE_CMD(SET_BSS_CONNECTED),
+ &req, sizeof(req), false);
}
int mt7921_mcu_sta_update(struct mt7921_dev *dev, struct ieee80211_sta *sta,
@@ -1245,7 +1241,7 @@ int mt7921_mcu_sta_update(struct mt7921_dev *dev, struct ieee80211_sta *sta,
.sta = sta,
.vif = vif,
.enable = enable,
- .cmd = MCU_UNI_CMD_STA_REC_UPDATE,
+ .cmd = MCU_UNI_CMD(STA_REC_UPDATE),
.state = state,
.offload_fw = true,
.rcpi = to_rcpi(rssi),
@@ -1342,7 +1338,7 @@ int mt7921_get_txpwr_info(struct mt7921_dev *dev, struct mt7921_txpwr *txpwr)
struct sk_buff *skb;
int ret;
- ret = mt76_mcu_send_and_get_msg(&dev->mt76, MCU_CMD_GET_TXPWR,
+ ret = mt76_mcu_send_and_get_msg(&dev->mt76, MCU_CE_CMD(GET_TXPWR),
&req, sizeof(req), true, &skb);
if (ret)
return ret;
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/mcu.h b/drivers/net/wireless/mediatek/mt76/mt7921/mcu.h
index edc0c73..77cc0cc 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7921/mcu.h
+++ b/drivers/net/wireless/mediatek/mt76/mt7921/mcu.h
@@ -76,20 +76,32 @@ struct mt7921_uni_txd {
u8 reserved2[4];
} __packed __aligned(4);
-/* event table */
-enum {
- MCU_EVENT_REG_ACCESS = 0x05,
- MCU_EVENT_LP_INFO = 0x07,
- MCU_EVENT_SCAN_DONE = 0x0d,
- MCU_EVENT_TX_DONE = 0x0f,
- MCU_EVENT_BSS_ABSENCE = 0x11,
- MCU_EVENT_BSS_BEACON_LOSS = 0x13,
- MCU_EVENT_CH_PRIVILEGE = 0x18,
- MCU_EVENT_SCHED_SCAN_DONE = 0x23,
- MCU_EVENT_DBG_MSG = 0x27,
- MCU_EVENT_TXPWR = 0xd0,
- MCU_EVENT_COREDUMP = 0xf0,
-};
+struct mt7921_mcu_tx_done_event {
+ u8 pid;
+ u8 status;
+ __le16 seq;
+
+ u8 wlan_idx;
+ u8 tx_cnt;
+ __le16 tx_rate;
+
+ u8 flag;
+ u8 tid;
+ u8 rsp_rate;
+ u8 mcs;
+
+ u8 bw;
+ u8 tx_pwr;
+ u8 reason;
+ u8 rsv0[1];
+
+ __le32 delay;
+ __le32 timestamp;
+ __le32 applied_flag;
+ u8 txs[28];
+
+ u8 rsv1[32];
+} __packed;
/* ext event table */
enum {
@@ -126,20 +138,6 @@ struct mt7921_mcu_eeprom_info {
#define MCU_PQ_ID(p, q) (((p) << 15) | ((q) << 10))
#define MCU_PKT_ID 0xa0
-enum {
- MCU_Q_QUERY,
- MCU_Q_SET,
- MCU_Q_RESERVED,
- MCU_Q_NA
-};
-
-enum {
- MCU_S2D_H2N,
- MCU_S2D_C2N,
- MCU_S2D_H2C,
- MCU_S2D_H2CN
-};
-
struct mt7921_mcu_uni_event {
u8 cid;
u8 pad[3];
@@ -147,109 +145,10 @@ struct mt7921_mcu_uni_event {
} __packed;
enum {
- PATCH_NOT_DL_SEM_FAIL,
- PATCH_IS_DL,
- PATCH_NOT_DL_SEM_SUCCESS,
- PATCH_REL_SEM_SUCCESS
-};
-
-enum {
- FW_STATE_INITIAL,
- FW_STATE_FW_DOWNLOAD,
- FW_STATE_NORMAL_OPERATION,
- FW_STATE_NORMAL_TRX,
- FW_STATE_WACPU_RDY = 7
-};
-
-enum {
- EE_MODE_EFUSE,
- EE_MODE_BUFFER,
-};
-
-enum {
- EE_FORMAT_BIN,
- EE_FORMAT_WHOLE,
- EE_FORMAT_MULTIPLE,
-};
-
-enum {
- MCU_PHY_STATE_TX_RATE,
- MCU_PHY_STATE_RX_RATE,
- MCU_PHY_STATE_RSSI,
- MCU_PHY_STATE_CONTENTION_RX_RATE,
- MCU_PHY_STATE_OFDMLQ_CNINFO,
-};
-
-struct sec_key {
- u8 cipher_id;
- u8 cipher_len;
- u8 key_id;
- u8 key_len;
- u8 key[32];
-} __packed;
-
-struct sta_rec_sec {
- __le16 tag;
- __le16 len;
- u8 add;
- u8 n_cipher;
- u8 rsv[2];
-
- struct sec_key key[2];
-} __packed;
-
-enum mcu_cipher_type {
- MCU_CIPHER_NONE = 0,
- MCU_CIPHER_WEP40,
- MCU_CIPHER_WEP104,
- MCU_CIPHER_WEP128,
- MCU_CIPHER_TKIP,
- MCU_CIPHER_AES_CCMP,
- MCU_CIPHER_CCMP_256,
- MCU_CIPHER_GCMP,
- MCU_CIPHER_GCMP_256,
- MCU_CIPHER_WAPI,
- MCU_CIPHER_BIP_CMAC_128,
-};
-
-enum {
- CH_SWITCH_NORMAL = 0,
- CH_SWITCH_SCAN = 3,
- CH_SWITCH_MCC = 4,
- CH_SWITCH_DFS = 5,
- CH_SWITCH_BACKGROUND_SCAN_START = 6,
- CH_SWITCH_BACKGROUND_SCAN_RUNNING = 7,
- CH_SWITCH_BACKGROUND_SCAN_STOP = 8,
- CH_SWITCH_SCAN_BYPASS_DPD = 9
-};
-
-enum {
- THERMAL_SENSOR_TEMP_QUERY,
- THERMAL_SENSOR_MANUAL_CTRL,
- THERMAL_SENSOR_INFO_QUERY,
- THERMAL_SENSOR_TASK_CTRL,
-};
-
-enum {
MT_EBF = BIT(0), /* explicit beamforming */
MT_IBF = BIT(1) /* implicit beamforming */
};
-#define STA_CAP_WMM BIT(0)
-#define STA_CAP_SGI_20 BIT(4)
-#define STA_CAP_SGI_40 BIT(5)
-#define STA_CAP_TX_STBC BIT(6)
-#define STA_CAP_RX_STBC BIT(7)
-#define STA_CAP_VHT_SGI_80 BIT(16)
-#define STA_CAP_VHT_SGI_160 BIT(17)
-#define STA_CAP_VHT_TX_STBC BIT(18)
-#define STA_CAP_VHT_RX_STBC BIT(19)
-#define STA_CAP_VHT_LDPC BIT(23)
-#define STA_CAP_LDPC BIT(24)
-#define STA_CAP_HT BIT(26)
-#define STA_CAP_VHT BIT(27)
-#define STA_CAP_HE BIT(28)
-
struct mt7921_mcu_reg_event {
__le32 reg;
__le32 val;
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/mt7921.h b/drivers/net/wireless/mediatek/mt76/mt7921/mt7921.h
index e9c7c3a..9664780 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7921/mt7921.h
+++ b/drivers/net/wireless/mediatek/mt76/mt7921/mt7921.h
@@ -446,6 +446,7 @@ int mt7921_mcu_restart(struct mt76_dev *dev);
void mt7921e_queue_rx_skb(struct mt76_dev *mdev, enum mt76_rxq_id q,
struct sk_buff *skb);
+int mt7921e_driver_own(struct mt7921_dev *dev);
int mt7921e_mac_reset(struct mt7921_dev *dev);
int mt7921e_mcu_init(struct mt7921_dev *dev);
int mt7921s_wfsys_reset(struct mt7921_dev *dev);
@@ -463,4 +464,5 @@ int mt7921s_tx_prepare_skb(struct mt76_dev *mdev, void *txwi_ptr,
struct mt76_tx_info *tx_info);
void mt7921s_tx_complete_skb(struct mt76_dev *mdev, struct mt76_queue_entry *e);
bool mt7921s_tx_status_data(struct mt76_dev *mdev, u8 *update);
+void mt7921_mac_add_txs(struct mt7921_dev *dev, void *data);
#endif
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/pci.c b/drivers/net/wireless/mediatek/mt76/mt7921/pci.c
index 305b63f..9dae2f5 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7921/pci.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7921/pci.c
@@ -15,6 +15,8 @@
static const struct pci_device_id mt7921_pci_device_table[] = {
{ PCI_DEVICE(PCI_VENDOR_ID_MEDIATEK, 0x7961) },
{ PCI_DEVICE(PCI_VENDOR_ID_MEDIATEK, 0x7922) },
+ { PCI_DEVICE(PCI_VENDOR_ID_MEDIATEK, 0x0608) },
+ { PCI_DEVICE(PCI_VENDOR_ID_MEDIATEK, 0x0616) },
{ },
};
@@ -188,7 +190,7 @@ static int mt7921_pci_probe(struct pci_dev *pdev,
tasklet_init(&dev->irq_tasklet, mt7921_irq_tasklet, (unsigned long)dev);
mdev->rev = (mt7921_l1_rr(dev, MT_HW_CHIPID) << 16) |
(mt7921_l1_rr(dev, MT_HW_REV) & 0xff);
- dev_err(mdev->dev, "ASIC revision: %04x\n", mdev->rev);
+ dev_info(mdev->dev, "ASIC revision: %04x\n", mdev->rev);
mt76_wr(dev, MT_WFDMA0_HOST_INT_ENA, 0);
@@ -235,7 +237,6 @@ static int mt7921_pci_suspend(struct pci_dev *pdev, pm_message_t state)
struct mt76_dev *mdev = pci_get_drvdata(pdev);
struct mt7921_dev *dev = container_of(mdev, struct mt7921_dev, mt76);
struct mt76_connac_pm *pm = &dev->pm;
- bool hif_suspend;
int i, err;
pm->suspended = true;
@@ -246,12 +247,9 @@ static int mt7921_pci_suspend(struct pci_dev *pdev, pm_message_t state)
if (err < 0)
goto restore_suspend;
- hif_suspend = !test_bit(MT76_STATE_SUSPEND, &dev->mphy.state);
- if (hif_suspend) {
- err = mt76_connac_mcu_set_hif_suspend(mdev, true);
- if (err)
- goto restore_suspend;
- }
+ err = mt76_connac_mcu_set_hif_suspend(mdev, true);
+ if (err)
+ goto restore_suspend;
/* always enable deep sleep during suspend to reduce
* power consumption
@@ -302,8 +300,7 @@ static int mt7921_pci_suspend(struct pci_dev *pdev, pm_message_t state)
if (!pm->ds_enable)
mt76_connac_mcu_set_deep_sleep(&dev->mt76, false);
- if (hif_suspend)
- mt76_connac_mcu_set_hif_suspend(mdev, false);
+ mt76_connac_mcu_set_hif_suspend(mdev, false);
restore_suspend:
pm->suspended = false;
@@ -318,7 +315,6 @@ static int mt7921_pci_resume(struct pci_dev *pdev)
struct mt76_connac_pm *pm = &dev->pm;
int i, err;
- pm->suspended = false;
err = pci_set_power_state(pdev, PCI_D0);
if (err)
return err;
@@ -356,8 +352,11 @@ static int mt7921_pci_resume(struct pci_dev *pdev)
if (!pm->ds_enable)
mt76_connac_mcu_set_deep_sleep(&dev->mt76, false);
- if (!test_bit(MT76_STATE_SUSPEND, &dev->mphy.state))
- err = mt76_connac_mcu_set_hif_suspend(mdev, false);
+ err = mt76_connac_mcu_set_hif_suspend(mdev, false);
+ if (err)
+ return err;
+
+ pm->suspended = false;
return err;
}
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/pci_mac.c b/drivers/net/wireless/mediatek/mt76/mt7921/pci_mac.c
index f9547d2..85286cc 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7921/pci_mac.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7921/pci_mac.c
@@ -321,6 +321,10 @@ int mt7921e_mac_reset(struct mt7921_dev *dev)
MT_INT_MCU_CMD);
mt76_wr(dev, MT_PCIE_MAC_INT_ENABLE, 0xff);
+ err = mt7921e_driver_own(dev);
+ if (err)
+ return err;
+
err = mt7921_run_firmware(dev);
if (err)
goto out;
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/pci_mcu.c b/drivers/net/wireless/mediatek/mt76/mt7921/pci_mcu.c
index 583a89a..a020352 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7921/pci_mcu.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7921/pci_mcu.c
@@ -4,7 +4,7 @@
#include "mt7921.h"
#include "mcu.h"
-static int mt7921e_driver_own(struct mt7921_dev *dev)
+int mt7921e_driver_own(struct mt7921_dev *dev)
{
u32 reg = mt7921_reg_map_l1(dev, MT_TOP_LPCR_HOST_BAND0);
@@ -30,7 +30,7 @@ mt7921_mcu_send_message(struct mt76_dev *mdev, struct sk_buff *skb,
if (ret)
return ret;
- if (cmd == MCU_CMD_FW_SCATTER)
+ if (cmd == MCU_CMD(FW_SCATTER))
txq = MT_MCUQ_FWDL;
return mt76_tx_queue_skb_raw(dev, mdev->q_mcu[txq], skb, 0);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/sdio.c b/drivers/net/wireless/mediatek/mt76/mt7921/sdio.c
index ddf0eeb..65d6939 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7921/sdio.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7921/sdio.c
@@ -62,6 +62,10 @@ static int mt7921s_parse_intr(struct mt76_dev *dev, struct mt76s_intr *intr)
if (err < 0)
return err;
+ if (irq_data->rx.num[0] > 16 ||
+ irq_data->rx.num[1] > 128)
+ return -EINVAL;
+
intr->isr = irq_data->isr;
intr->rec_mb = irq_data->rec_mb;
intr->tx.wtqcr = irq_data->tx.wtqcr;
@@ -203,10 +207,11 @@ static int mt7921s_suspend(struct device *__dev)
struct mt7921_dev *dev = sdio_get_drvdata(func);
struct mt76_connac_pm *pm = &dev->pm;
struct mt76_dev *mdev = &dev->mt76;
- bool hif_suspend;
int err;
pm->suspended = true;
+ set_bit(MT76_STATE_SUSPEND, &mdev->phy.state);
+
cancel_delayed_work_sync(&pm->ps_work);
cancel_work_sync(&pm->wake_work);
@@ -214,13 +219,6 @@ static int mt7921s_suspend(struct device *__dev)
if (err < 0)
goto restore_suspend;
- hif_suspend = !test_bit(MT76_STATE_SUSPEND, &dev->mphy.state);
- if (hif_suspend) {
- err = mt76_connac_mcu_set_hif_suspend(mdev, true);
- if (err)
- goto restore_suspend;
- }
-
/* always enable deep sleep during suspend to reduce
* power consumption
*/
@@ -228,35 +226,45 @@ static int mt7921s_suspend(struct device *__dev)
mt76_txq_schedule_all(&dev->mphy);
mt76_worker_disable(&mdev->tx_worker);
- mt76_worker_disable(&mdev->sdio.txrx_worker);
mt76_worker_disable(&mdev->sdio.status_worker);
- mt76_worker_disable(&mdev->sdio.net_worker);
cancel_work_sync(&mdev->sdio.stat_work);
clear_bit(MT76_READING_STATS, &dev->mphy.state);
-
mt76_tx_status_check(mdev, true);
+ mt76_worker_schedule(&mdev->sdio.txrx_worker);
+ wait_event_timeout(dev->mt76.sdio.wait,
+ mt76s_txqs_empty(&dev->mt76), 5 * HZ);
+
+ /* It is supposed that SDIO bus is idle at the point */
+ err = mt76_connac_mcu_set_hif_suspend(mdev, true);
+ if (err)
+ goto restore_worker;
+
+ mt76_worker_disable(&mdev->sdio.txrx_worker);
+ mt76_worker_disable(&mdev->sdio.net_worker);
+
err = mt7921_mcu_fw_pmctrl(dev);
if (err)
- goto restore_worker;
+ goto restore_txrx_worker;
sdio_set_host_pm_flags(func, MMC_PM_KEEP_POWER);
return 0;
+restore_txrx_worker:
+ mt76_worker_enable(&mdev->sdio.net_worker);
+ mt76_worker_enable(&mdev->sdio.txrx_worker);
+ mt76_connac_mcu_set_hif_suspend(mdev, false);
+
restore_worker:
mt76_worker_enable(&mdev->tx_worker);
- mt76_worker_enable(&mdev->sdio.txrx_worker);
mt76_worker_enable(&mdev->sdio.status_worker);
- mt76_worker_enable(&mdev->sdio.net_worker);
if (!pm->ds_enable)
mt76_connac_mcu_set_deep_sleep(mdev, false);
- if (hif_suspend)
- mt76_connac_mcu_set_hif_suspend(mdev, false);
-
restore_suspend:
+ clear_bit(MT76_STATE_SUSPEND, &mdev->phy.state);
pm->suspended = false;
return err;
@@ -270,7 +278,7 @@ static int mt7921s_resume(struct device *__dev)
struct mt76_dev *mdev = &dev->mt76;
int err;
- pm->suspended = false;
+ clear_bit(MT76_STATE_SUSPEND, &mdev->phy.state);
err = mt7921_mcu_drv_pmctrl(dev);
if (err < 0)
@@ -285,8 +293,11 @@ static int mt7921s_resume(struct device *__dev)
if (!pm->ds_enable)
mt76_connac_mcu_set_deep_sleep(mdev, false);
- if (!test_bit(MT76_STATE_SUSPEND, &dev->mphy.state))
- err = mt76_connac_mcu_set_hif_suspend(mdev, false);
+ err = mt76_connac_mcu_set_hif_suspend(mdev, false);
+ if (err)
+ return err;
+
+ pm->suspended = false;
return err;
}
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/sdio_mac.c b/drivers/net/wireless/mediatek/mt76/mt7921/sdio_mac.c
index bdec508..ccaf813 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7921/sdio_mac.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7921/sdio_mac.c
@@ -219,5 +219,5 @@ bool mt7921s_tx_status_data(struct mt76_dev *mdev, u8 *update)
mt7921_mac_sta_poll(dev);
mt7921_mutex_release(dev);
- return 0;
+ return false;
}
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/sdio_mcu.c b/drivers/net/wireless/mediatek/mt76/mt7921/sdio_mcu.c
index 437cdda..d20f2ff 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7921/sdio_mcu.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7921/sdio_mcu.c
@@ -33,7 +33,7 @@ mt7921s_mcu_send_message(struct mt76_dev *mdev, struct sk_buff *skb,
if (ret)
return ret;
- if (cmd == MCU_CMD_FW_SCATTER)
+ if (cmd == MCU_CMD(FW_SCATTER))
type = MT7921_SDIO_FWDL;
mt7921_skb_add_sdio_hdr(skb, type);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/testmode.c b/drivers/net/wireless/mediatek/mt76/mt7921/testmode.c
index 8bd4387..bdec868 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7921/testmode.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7921/testmode.c
@@ -66,7 +66,7 @@ mt7921_tm_set(struct mt7921_dev *dev, struct mt7921_tm_cmd *req)
if (!mt76_testmode_enabled(phy))
goto out;
- ret = mt76_mcu_send_msg(&dev->mt76, MCU_CMD_TEST_CTRL, &cmd,
+ ret = mt76_mcu_send_msg(&dev->mt76, MCU_CE_CMD(TEST_CTRL), &cmd,
sizeof(cmd), false);
if (ret)
goto out;
@@ -95,7 +95,7 @@ mt7921_tm_query(struct mt7921_dev *dev, struct mt7921_tm_cmd *req,
struct sk_buff *skb;
int ret;
- ret = mt76_mcu_send_and_get_msg(&dev->mt76, MCU_CMD_TEST_CTRL,
+ ret = mt76_mcu_send_and_get_msg(&dev->mt76, MCU_CE_CMD(TEST_CTRL),
&cmd, sizeof(cmd), true, &skb);
if (ret)
goto out;
diff --git a/drivers/net/wireless/mediatek/mt76/sdio.c b/drivers/net/wireless/mediatek/mt76/sdio.c
index c99acc2..54f72d2 100644
--- a/drivers/net/wireless/mediatek/mt76/sdio.c
+++ b/drivers/net/wireless/mediatek/mt76/sdio.c
@@ -305,12 +305,12 @@ int mt76s_alloc_rx_queue(struct mt76_dev *dev, enum mt76_rxq_id qid)
spin_lock_init(&q->lock);
q->entry = devm_kcalloc(dev->dev,
- MT_NUM_RX_ENTRIES, sizeof(*q->entry),
+ MT76S_NUM_RX_ENTRIES, sizeof(*q->entry),
GFP_KERNEL);
if (!q->entry)
return -ENOMEM;
- q->ndesc = MT_NUM_RX_ENTRIES;
+ q->ndesc = MT76S_NUM_RX_ENTRIES;
q->head = q->tail = 0;
q->queued = 0;
@@ -328,12 +328,12 @@ static struct mt76_queue *mt76s_alloc_tx_queue(struct mt76_dev *dev)
spin_lock_init(&q->lock);
q->entry = devm_kcalloc(dev->dev,
- MT_NUM_TX_ENTRIES, sizeof(*q->entry),
+ MT76S_NUM_TX_ENTRIES, sizeof(*q->entry),
GFP_KERNEL);
if (!q->entry)
return ERR_PTR(-ENOMEM);
- q->ndesc = MT_NUM_TX_ENTRIES;
+ q->ndesc = MT76S_NUM_TX_ENTRIES;
return q;
}
@@ -479,7 +479,8 @@ static void mt76s_status_worker(struct mt76_worker *w)
resched = true;
if (dev->drv->tx_status_data &&
- !test_and_set_bit(MT76_READING_STATS, &dev->phy.state))
+ !test_and_set_bit(MT76_READING_STATS, &dev->phy.state) &&
+ !test_bit(MT76_STATE_SUSPEND, &dev->phy.state))
queue_work(dev->wq, &dev->sdio.stat_work);
} while (nframes > 0);
diff --git a/drivers/net/wireless/mediatek/mt76/sdio_txrx.c b/drivers/net/wireless/mediatek/mt76/sdio_txrx.c
index 649a567..801590a 100644
--- a/drivers/net/wireless/mediatek/mt76/sdio_txrx.c
+++ b/drivers/net/wireless/mediatek/mt76/sdio_txrx.c
@@ -317,7 +317,8 @@ void mt76s_txrx_worker(struct mt76_sdio *sdio)
if (ret > 0)
nframes += ret;
- if (test_bit(MT76_MCU_RESET, &dev->phy.state)) {
+ if (test_bit(MT76_MCU_RESET, &dev->phy.state) ||
+ test_bit(MT76_STATE_SUSPEND, &dev->phy.state)) {
if (!mt76s_txqs_empty(dev))
continue;
else
diff --git a/drivers/net/wireless/mediatek/mt76/testmode.c b/drivers/net/wireless/mediatek/mt76/testmode.c
index 66afc2b..1a01ad7 100644
--- a/drivers/net/wireless/mediatek/mt76/testmode.c
+++ b/drivers/net/wireless/mediatek/mt76/testmode.c
@@ -126,9 +126,9 @@ int mt76_testmode_alloc_skb(struct mt76_phy *phy, u32 len)
hdr = __skb_put_zero(head, head_len);
hdr->frame_control = cpu_to_le16(fc);
- memcpy(hdr->addr1, phy->macaddr, sizeof(phy->macaddr));
- memcpy(hdr->addr2, phy->macaddr, sizeof(phy->macaddr));
- memcpy(hdr->addr3, phy->macaddr, sizeof(phy->macaddr));
+ memcpy(hdr->addr1, td->addr[0], ETH_ALEN);
+ memcpy(hdr->addr2, td->addr[1], ETH_ALEN);
+ memcpy(hdr->addr3, td->addr[2], ETH_ALEN);
skb_set_queue_mapping(head, IEEE80211_AC_BE);
info = IEEE80211_SKB_CB(head);
@@ -318,6 +318,10 @@ mt76_testmode_init_defaults(struct mt76_phy *phy)
td->tx_count = 1;
td->tx_rate_mode = MT76_TM_TX_MODE_OFDM;
td->tx_rate_nss = 1;
+
+ memcpy(td->addr[0], phy->macaddr, ETH_ALEN);
+ memcpy(td->addr[1], phy->macaddr, ETH_ALEN);
+ memcpy(td->addr[2], phy->macaddr, ETH_ALEN);
}
static int
@@ -493,6 +497,20 @@ int mt76_testmode_cmd(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
}
}
+ if (tb[MT76_TM_ATTR_MAC_ADDRS]) {
+ struct nlattr *cur;
+ int idx = 0;
+ int rem;
+
+ nla_for_each_nested(cur, tb[MT76_TM_ATTR_MAC_ADDRS], rem) {
+ if (nla_len(cur) != ETH_ALEN || idx >= 3)
+ goto out;
+
+ memcpy(td->addr[idx], nla_data(cur), ETH_ALEN);
+ idx++;
+ }
+ }
+
if (dev->test_ops->set_params) {
err = dev->test_ops->set_params(phy, tb, state);
if (err)
@@ -635,6 +653,18 @@ int mt76_testmode_dump(struct ieee80211_hw *hw, struct sk_buff *msg,
nla_nest_end(msg, a);
}
+ if (mt76_testmode_param_present(td, MT76_TM_ATTR_MAC_ADDRS)) {
+ a = nla_nest_start(msg, MT76_TM_ATTR_MAC_ADDRS);
+ if (!a)
+ goto out;
+
+ for (i = 0; i < 3; i++)
+ if (nla_put(msg, i, ETH_ALEN, td->addr[i]))
+ goto out;
+
+ nla_nest_end(msg, a);
+ }
+
err = 0;
out:
diff --git a/drivers/net/wireless/mediatek/mt76/testmode.h b/drivers/net/wireless/mediatek/mt76/testmode.h
index d1f9c03..0590c35 100644
--- a/drivers/net/wireless/mediatek/mt76/testmode.h
+++ b/drivers/net/wireless/mediatek/mt76/testmode.h
@@ -7,6 +7,8 @@
#define MT76_TM_TIMEOUT 10
+#include <net/netlink.h>
+
/**
* enum mt76_testmode_attr - testmode attributes inside NL80211_ATTR_TESTDATA
*
@@ -45,6 +47,8 @@
* @MT76_TM_ATTR_TX_TIME: packet transmission time, in unit of us (u32)
*
* @MT76_TM_ATTR_DRV_DATA: driver specific netlink attrs (nested)
+ *
+ * @MT76_TM_ATTR_MAC_ADDRS: array of nested MAC addresses (nested)
*/
enum mt76_testmode_attr {
MT76_TM_ATTR_UNSPEC,
@@ -81,6 +85,8 @@ enum mt76_testmode_attr {
MT76_TM_ATTR_DRV_DATA,
+ MT76_TM_ATTR_MAC_ADDRS,
+
/* keep last */
NUM_MT76_TM_ATTRS,
MT76_TM_ATTR_MAX = NUM_MT76_TM_ATTRS - 1,
diff --git a/drivers/net/wireless/microchip/wilc1000/cfg80211.c b/drivers/net/wireless/microchip/wilc1000/cfg80211.c
index dc4bfe7..8d8378b 100644
--- a/drivers/net/wireless/microchip/wilc1000/cfg80211.c
+++ b/drivers/net/wireless/microchip/wilc1000/cfg80211.c
@@ -1737,23 +1737,15 @@ int wilc_cfg80211_init(struct wilc **wilc, struct device *dev, int io_type,
INIT_LIST_HEAD(&wl->rxq_head.list);
INIT_LIST_HEAD(&wl->vif_list);
- wl->hif_workqueue = create_singlethread_workqueue("WILC_wq");
- if (!wl->hif_workqueue) {
- ret = -ENOMEM;
- goto free_cfg;
- }
vif = wilc_netdev_ifc_init(wl, "wlan%d", WILC_STATION_MODE,
NL80211_IFTYPE_STATION, false);
if (IS_ERR(vif)) {
ret = PTR_ERR(vif);
- goto free_hq;
+ goto free_cfg;
}
return 0;
-free_hq:
- destroy_workqueue(wl->hif_workqueue);
-
free_cfg:
wilc_wlan_cfg_deinit(wl);
diff --git a/drivers/net/wireless/microchip/wilc1000/hif.c b/drivers/net/wireless/microchip/wilc1000/hif.c
index e69b9c7..71b44cf 100644
--- a/drivers/net/wireless/microchip/wilc1000/hif.c
+++ b/drivers/net/wireless/microchip/wilc1000/hif.c
@@ -1312,7 +1312,7 @@ int wilc_set_mac_address(struct wilc_vif *vif, u8 *mac_addr)
result = wilc_send_config_pkt(vif, WILC_SET_CFG, &wid, 1);
if (result)
- netdev_err(vif->ndev, "Failed to get mac address\n");
+ netdev_err(vif->ndev, "Failed to set mac address\n");
return result;
}
@@ -1929,6 +1929,7 @@ int wilc_edit_station(struct wilc_vif *vif, const u8 *mac,
int wilc_set_power_mgmt(struct wilc_vif *vif, bool enabled, u32 timeout)
{
+ struct wilc *wilc = vif->wilc;
struct wid wid;
int result;
s8 power_mode;
@@ -1944,6 +1945,8 @@ int wilc_set_power_mgmt(struct wilc_vif *vif, bool enabled, u32 timeout)
result = wilc_send_config_pkt(vif, WILC_SET_CFG, &wid, 1);
if (result)
netdev_err(vif->ndev, "Failed to send power management\n");
+ else
+ wilc->power_save_mode = enabled;
return result;
}
diff --git a/drivers/net/wireless/microchip/wilc1000/netdev.c b/drivers/net/wireless/microchip/wilc1000/netdev.c
index 690572e..643bdda 100644
--- a/drivers/net/wireless/microchip/wilc1000/netdev.c
+++ b/drivers/net/wireless/microchip/wilc1000/netdev.c
@@ -27,7 +27,7 @@ static irqreturn_t isr_uh_routine(int irq, void *user_data)
struct wilc *wilc = user_data;
if (wilc->close) {
- pr_err("Can't handle UH interrupt");
+ pr_err("Can't handle UH interrupt\n");
return IRQ_HANDLED;
}
return IRQ_WAKE_THREAD;
@@ -56,7 +56,7 @@ static int init_irq(struct net_device *dev)
ret = request_threaded_irq(wl->dev_irq_num, isr_uh_routine,
isr_bh_routine,
IRQF_TRIGGER_FALLING | IRQF_ONESHOT,
- "WILC_IRQ", wl);
+ dev->name, wl);
if (ret) {
netdev_err(dev, "Failed to request IRQ [%d]\n", ret);
return ret;
@@ -468,7 +468,7 @@ static int wlan_initialize_threads(struct net_device *dev)
struct wilc *wilc = vif->wilc;
wilc->txq_thread = kthread_run(wilc_txq_task, (void *)wilc,
- "K_TXQ_TASK");
+ "%s-tx", dev->name);
if (IS_ERR(wilc->txq_thread)) {
netdev_err(dev, "couldn't create TXQ thread\n");
wilc->close = 0;
@@ -574,6 +574,7 @@ static int wilc_mac_open(struct net_device *ndev)
struct wilc *wl = vif->wilc;
int ret = 0;
struct mgmt_frame_regs mgmt_regs = {};
+ u8 addr[ETH_ALEN] __aligned(2);
if (!wl || !wl->dev) {
netdev_err(ndev, "device not ready\n");
@@ -596,10 +597,9 @@ static int wilc_mac_open(struct net_device *ndev)
vif->idx);
if (is_valid_ether_addr(ndev->dev_addr)) {
- wilc_set_mac_address(vif, ndev->dev_addr);
+ ether_addr_copy(addr, ndev->dev_addr);
+ wilc_set_mac_address(vif, addr);
} else {
- u8 addr[ETH_ALEN];
-
wilc_get_mac_address(vif, addr);
eth_hw_addr_set(ndev, addr);
}
@@ -905,7 +905,6 @@ void wilc_netdev_cleanup(struct wilc *wilc)
wilc_wlan_cfg_deinit(wilc);
wlan_deinit_locks(wilc);
- kfree(wilc->bus_data);
wiphy_unregister(wilc->wiphy);
wiphy_free(wilc->wiphy);
}
@@ -962,8 +961,15 @@ struct wilc_vif *wilc_netdev_ifc_init(struct wilc *wl, const char *name,
ret = register_netdev(ndev);
if (ret) {
- free_netdev(ndev);
- return ERR_PTR(-EFAULT);
+ ret = -EFAULT;
+ goto error;
+ }
+
+ wl->hif_workqueue = alloc_ordered_workqueue("%s-wq", WQ_MEM_RECLAIM,
+ ndev->name);
+ if (!wl->hif_workqueue) {
+ ret = -ENOMEM;
+ goto error;
}
ndev->needs_free_netdev = true;
@@ -977,6 +983,10 @@ struct wilc_vif *wilc_netdev_ifc_init(struct wilc *wl, const char *name,
synchronize_srcu(&wl->srcu);
return vif;
+
+ error:
+ free_netdev(ndev);
+ return ERR_PTR(ret);
}
MODULE_LICENSE("GPL");
diff --git a/drivers/net/wireless/microchip/wilc1000/netdev.h b/drivers/net/wireless/microchip/wilc1000/netdev.h
index b9a88b3e..a067274 100644
--- a/drivers/net/wireless/microchip/wilc1000/netdev.h
+++ b/drivers/net/wireless/microchip/wilc1000/netdev.h
@@ -212,6 +212,8 @@ struct wilc {
s8 mac_status;
struct clk *rtc_clk;
bool initialized;
+ u32 chipid;
+ bool power_save_mode;
int dev_irq_num;
int close;
u8 vif_num;
diff --git a/drivers/net/wireless/microchip/wilc1000/sdio.c b/drivers/net/wireless/microchip/wilc1000/sdio.c
index 26ebf66..ec595dbd 100644
--- a/drivers/net/wireless/microchip/wilc1000/sdio.c
+++ b/drivers/net/wireless/microchip/wilc1000/sdio.c
@@ -167,9 +167,11 @@ static int wilc_sdio_probe(struct sdio_func *func,
static void wilc_sdio_remove(struct sdio_func *func)
{
struct wilc *wilc = sdio_get_drvdata(func);
+ struct wilc_sdio *sdio_priv = wilc->bus_data;
clk_disable_unprepare(wilc->rtc_clk);
wilc_netdev_cleanup(wilc);
+ kfree(sdio_priv);
}
static int wilc_sdio_reset(struct wilc *wilc)
diff --git a/drivers/net/wireless/microchip/wilc1000/spi.c b/drivers/net/wireless/microchip/wilc1000/spi.c
index 640850f..2c2ed4b 100644
--- a/drivers/net/wireless/microchip/wilc1000/spi.c
+++ b/drivers/net/wireless/microchip/wilc1000/spi.c
@@ -8,10 +8,13 @@
#include <linux/spi/spi.h>
#include <linux/crc7.h>
#include <linux/crc-itu-t.h>
+#include <linux/gpio/consumer.h>
#include "netdev.h"
#include "cfg80211.h"
+#define SPI_MODALIAS "wilc1000_spi"
+
static bool enable_crc7; /* protect SPI commands with CRC7 */
module_param(enable_crc7, bool, 0644);
MODULE_PARM_DESC(enable_crc7,
@@ -43,6 +46,10 @@ struct wilc_spi {
bool probing_crc; /* true if we're probing chip's CRC config */
bool crc7_enabled; /* true if crc7 is currently enabled */
bool crc16_enabled; /* true if crc16 is currently enabled */
+ struct wilc_gpios {
+ struct gpio_desc *enable; /* ENABLE GPIO or NULL */
+ struct gpio_desc *reset; /* RESET GPIO or NULL */
+ } gpios;
};
static const struct wilc_hif_func wilc_hif_spi;
@@ -99,8 +106,6 @@ static int wilc_spi_reset(struct wilc *wilc);
#define DATA_PKT_LOG_SZ DATA_PKT_LOG_SZ_MAX
#define DATA_PKT_SZ (1 << DATA_PKT_LOG_SZ)
-#define USE_SPI_DMA 0
-
#define WILC_SPI_COMMAND_STAT_SUCCESS 0
#define WILC_GET_RESP_HDR_START(h) (((h) >> 4) & 0xf)
@@ -152,6 +157,50 @@ struct wilc_spi_special_cmd_rsp {
u8 status;
} __packed;
+static int wilc_parse_gpios(struct wilc *wilc)
+{
+ struct spi_device *spi = to_spi_device(wilc->dev);
+ struct wilc_spi *spi_priv = wilc->bus_data;
+ struct wilc_gpios *gpios = &spi_priv->gpios;
+
+ /* get ENABLE pin and deassert it (if it is defined): */
+ gpios->enable = devm_gpiod_get_optional(&spi->dev,
+ "enable", GPIOD_OUT_LOW);
+ /* get RESET pin and assert it (if it is defined): */
+ if (gpios->enable) {
+ /* if enable pin exists, reset must exist as well */
+ gpios->reset = devm_gpiod_get(&spi->dev,
+ "reset", GPIOD_OUT_HIGH);
+ if (IS_ERR(gpios->reset)) {
+ dev_err(&spi->dev, "missing reset gpio.\n");
+ return PTR_ERR(gpios->reset);
+ }
+ } else {
+ gpios->reset = devm_gpiod_get_optional(&spi->dev,
+ "reset", GPIOD_OUT_HIGH);
+ }
+ return 0;
+}
+
+static void wilc_wlan_power(struct wilc *wilc, bool on)
+{
+ struct wilc_spi *spi_priv = wilc->bus_data;
+ struct wilc_gpios *gpios = &spi_priv->gpios;
+
+ if (on) {
+ /* assert ENABLE: */
+ gpiod_set_value(gpios->enable, 1);
+ mdelay(5);
+ /* deassert RESET: */
+ gpiod_set_value(gpios->reset, 0);
+ } else {
+ /* assert RESET: */
+ gpiod_set_value(gpios->reset, 1);
+ /* deassert ENABLE: */
+ gpiod_set_value(gpios->enable, 0);
+ }
+}
+
static int wilc_bus_probe(struct spi_device *spi)
{
int ret;
@@ -171,6 +220,10 @@ static int wilc_bus_probe(struct spi_device *spi)
wilc->bus_data = spi_priv;
wilc->dev_irq_num = spi->irq;
+ ret = wilc_parse_gpios(wilc);
+ if (ret < 0)
+ goto netdev_cleanup;
+
wilc->rtc_clk = devm_clk_get_optional(&spi->dev, "rtc");
if (IS_ERR(wilc->rtc_clk)) {
ret = PTR_ERR(wilc->rtc_clk);
@@ -190,9 +243,11 @@ static int wilc_bus_probe(struct spi_device *spi)
static int wilc_bus_remove(struct spi_device *spi)
{
struct wilc *wilc = spi_get_drvdata(spi);
+ struct wilc_spi *spi_priv = wilc->bus_data;
clk_disable_unprepare(wilc->rtc_clk);
wilc_netdev_cleanup(wilc);
+ kfree(spi_priv);
return 0;
}
@@ -203,11 +258,18 @@ static const struct of_device_id wilc_of_match[] = {
};
MODULE_DEVICE_TABLE(of, wilc_of_match);
+static const struct spi_device_id wilc_spi_id[] = {
+ { "wilc1000", 0 },
+ { /* sentinel */ }
+};
+MODULE_DEVICE_TABLE(spi, wilc_spi_id);
+
static struct spi_driver wilc_spi_driver = {
.driver = {
- .name = MODALIAS,
+ .name = SPI_MODALIAS,
.of_match_table = wilc_of_match,
},
+ .id_table = wilc_spi_id,
.probe = wilc_bus_probe,
.remove = wilc_bus_remove,
};
@@ -240,7 +302,6 @@ static int wilc_spi_tx(struct wilc *wilc, u8 *b, u32 len)
memset(&msg, 0, sizeof(msg));
spi_message_init(&msg);
msg.spi = spi;
- msg.is_dma_mapped = USE_SPI_DMA;
spi_message_add_tail(&tr, &msg);
ret = spi_sync(spi, &msg);
@@ -284,7 +345,6 @@ static int wilc_spi_rx(struct wilc *wilc, u8 *rb, u32 rlen)
memset(&msg, 0, sizeof(msg));
spi_message_init(&msg);
msg.spi = spi;
- msg.is_dma_mapped = USE_SPI_DMA;
spi_message_add_tail(&tr, &msg);
ret = spi_sync(spi, &msg);
@@ -323,7 +383,6 @@ static int wilc_spi_tx_rx(struct wilc *wilc, u8 *wb, u8 *rb, u32 rlen)
memset(&msg, 0, sizeof(msg));
spi_message_init(&msg);
msg.spi = spi;
- msg.is_dma_mapped = USE_SPI_DMA;
spi_message_add_tail(&tr, &msg);
ret = spi_sync(spi, &msg);
@@ -977,9 +1036,10 @@ static int wilc_spi_reset(struct wilc *wilc)
static int wilc_spi_deinit(struct wilc *wilc)
{
- /*
- * TODO:
- */
+ struct wilc_spi *spi_priv = wilc->bus_data;
+
+ spi_priv->isinit = false;
+ wilc_wlan_power(wilc, false);
return 0;
}
@@ -1000,6 +1060,8 @@ static int wilc_spi_init(struct wilc *wilc, bool resume)
dev_err(&spi->dev, "Fail cmd read chip id...\n");
}
+ wilc_wlan_power(wilc, true);
+
/*
* configure protocol
*/
diff --git a/drivers/net/wireless/microchip/wilc1000/wlan.c b/drivers/net/wireless/microchip/wilc1000/wlan.c
index ea81ef1..fb5633a 100644
--- a/drivers/net/wireless/microchip/wilc1000/wlan.c
+++ b/drivers/net/wireless/microchip/wilc1000/wlan.c
@@ -20,13 +20,13 @@ static inline bool is_wilc1000(u32 id)
static inline void acquire_bus(struct wilc *wilc, enum bus_acquire acquire)
{
mutex_lock(&wilc->hif_cs);
- if (acquire == WILC_BUS_ACQUIRE_AND_WAKEUP)
+ if (acquire == WILC_BUS_ACQUIRE_AND_WAKEUP && wilc->power_save_mode)
chip_wakeup(wilc);
}
static inline void release_bus(struct wilc *wilc, enum bus_release release)
{
- if (release == WILC_BUS_RELEASE_ALLOW_SLEEP)
+ if (release == WILC_BUS_RELEASE_ALLOW_SLEEP && wilc->power_save_mode)
chip_allow_sleep(wilc);
mutex_unlock(&wilc->hif_cs);
}
@@ -626,7 +626,6 @@ void chip_wakeup(struct wilc *wilc)
u32 clk_status_val = 0, trials = 0;
u32 wakeup_reg, wakeup_bit;
u32 clk_status_reg, clk_status_bit;
- u32 to_host_from_fw_reg, to_host_from_fw_bit;
u32 from_host_to_fw_reg, from_host_to_fw_bit;
const struct wilc_hif_func *hif_func = wilc->hif_func;
@@ -637,8 +636,6 @@ void chip_wakeup(struct wilc *wilc)
clk_status_bit = WILC_SDIO_CLK_STATUS_BIT;
from_host_to_fw_reg = WILC_SDIO_HOST_TO_FW_REG;
from_host_to_fw_bit = WILC_SDIO_HOST_TO_FW_BIT;
- to_host_from_fw_reg = WILC_SDIO_FW_TO_HOST_REG;
- to_host_from_fw_bit = WILC_SDIO_FW_TO_HOST_BIT;
} else {
wakeup_reg = WILC_SPI_WAKEUP_REG;
wakeup_bit = WILC_SPI_WAKEUP_BIT;
@@ -646,8 +643,6 @@ void chip_wakeup(struct wilc *wilc)
clk_status_bit = WILC_SPI_CLK_STATUS_BIT;
from_host_to_fw_reg = WILC_SPI_HOST_TO_FW_REG;
from_host_to_fw_bit = WILC_SPI_HOST_TO_FW_BIT;
- to_host_from_fw_reg = WILC_SPI_FW_TO_HOST_REG;
- to_host_from_fw_bit = WILC_SPI_FW_TO_HOST_BIT;
}
/* indicate host wakeup */
@@ -1229,7 +1224,8 @@ int wilc_wlan_stop(struct wilc *wilc, struct wilc_vif *vif)
ret = 0;
release:
- release_bus(wilc, WILC_BUS_RELEASE_ALLOW_SLEEP);
+ /* host comm is disabled - we can't issue sleep command anymore: */
+ release_bus(wilc, WILC_BUS_RELEASE_ONLY);
return ret;
}
@@ -1258,7 +1254,7 @@ void wilc_wlan_cleanup(struct net_device *dev)
wilc->rx_buffer = NULL;
kfree(wilc->tx_buffer);
wilc->tx_buffer = NULL;
- wilc->hif_func->hif_deinit(NULL);
+ wilc->hif_func->hif_deinit(wilc);
}
static int wilc_wlan_cfg_commit(struct wilc_vif *vif, int type,
@@ -1447,31 +1443,30 @@ static int init_chip(struct net_device *dev)
u32 wilc_get_chipid(struct wilc *wilc, bool update)
{
- static u32 chipid;
- u32 tempchipid = 0;
+ u32 chipid = 0;
u32 rfrevid = 0;
- if (chipid == 0 || update) {
- wilc->hif_func->hif_read_reg(wilc, WILC_CHIPID, &tempchipid);
+ if (wilc->chipid == 0 || update) {
+ wilc->hif_func->hif_read_reg(wilc, WILC_CHIPID, &chipid);
wilc->hif_func->hif_read_reg(wilc, WILC_RF_REVISION_ID,
&rfrevid);
- if (!is_wilc1000(tempchipid)) {
- chipid = 0;
- return chipid;
+ if (!is_wilc1000(chipid)) {
+ wilc->chipid = 0;
+ return wilc->chipid;
}
- if (tempchipid == WILC_1000_BASE_ID_2A) { /* 0x1002A0 */
+ if (chipid == WILC_1000_BASE_ID_2A) { /* 0x1002A0 */
if (rfrevid != 0x1)
- tempchipid = WILC_1000_BASE_ID_2A_REV1;
- } else if (tempchipid == WILC_1000_BASE_ID_2B) { /* 0x1002B0 */
+ chipid = WILC_1000_BASE_ID_2A_REV1;
+ } else if (chipid == WILC_1000_BASE_ID_2B) { /* 0x1002B0 */
if (rfrevid == 0x4)
- tempchipid = WILC_1000_BASE_ID_2B_REV1;
+ chipid = WILC_1000_BASE_ID_2B_REV1;
else if (rfrevid != 0x3)
- tempchipid = WILC_1000_BASE_ID_2B_REV2;
+ chipid = WILC_1000_BASE_ID_2B_REV2;
}
- chipid = tempchipid;
+ wilc->chipid = chipid;
}
- return chipid;
+ return wilc->chipid;
}
int wilc_wlan_init(struct net_device *dev)
diff --git a/drivers/net/wireless/microchip/wilc1000/wlan.h b/drivers/net/wireless/microchip/wilc1000/wlan.h
index 13fde63..eb79781 100644
--- a/drivers/net/wireless/microchip/wilc1000/wlan.h
+++ b/drivers/net/wireless/microchip/wilc1000/wlan.h
@@ -213,8 +213,6 @@
#define WILC_RX_BUFF_SIZE (96 * 1024)
#define WILC_TX_BUFF_SIZE (64 * 1024)
-#define MODALIAS "WILC_SPI"
-
#define NQUEUES 4
#define AC_BUFFER_SIZE 1000
diff --git a/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_core.c b/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_core.c
index a42e208..06d59ff 100644
--- a/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_core.c
+++ b/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_core.c
@@ -4859,7 +4859,7 @@ rtl8xxxu_fill_txdesc_v1(struct ieee80211_hw *hw, struct ieee80211_hdr *hdr,
* rts_rate is zero if RTS/CTS or CTS to SELF are not enabled
*/
tx_desc->txdw4 |= cpu_to_le32(rts_rate << TXDESC32_RTS_RATE_SHIFT);
- if (rate_flags & IEEE80211_TX_RC_USE_RTS_CTS) {
+ if (ampdu_enable || (rate_flags & IEEE80211_TX_RC_USE_RTS_CTS)) {
tx_desc->txdw4 |= cpu_to_le32(TXDESC32_RTS_CTS_ENABLE);
tx_desc->txdw4 |= cpu_to_le32(TXDESC32_HW_RTS_ENABLE);
} else if (rate_flags & IEEE80211_TX_RC_USE_CTS_PROTECT) {
@@ -4930,7 +4930,7 @@ rtl8xxxu_fill_txdesc_v2(struct ieee80211_hw *hw, struct ieee80211_hdr *hdr,
/*
* rts_rate is zero if RTS/CTS or CTS to SELF are not enabled
*/
- if (rate_flags & IEEE80211_TX_RC_USE_RTS_CTS) {
+ if (ampdu_enable || (rate_flags & IEEE80211_TX_RC_USE_RTS_CTS)) {
tx_desc40->txdw3 |= cpu_to_le32(TXDESC40_RTS_CTS_ENABLE);
tx_desc40->txdw3 |= cpu_to_le32(TXDESC40_HW_RTS_ENABLE);
} else if (rate_flags & IEEE80211_TX_RC_USE_CTS_PROTECT) {
diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192cu/hw.c b/drivers/net/wireless/realtek/rtlwifi/rtl8192cu/hw.c
index 6312fdd..eaba661 100644
--- a/drivers/net/wireless/realtek/rtlwifi/rtl8192cu/hw.c
+++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192cu/hw.c
@@ -1000,6 +1000,7 @@ int rtl92cu_hw_init(struct ieee80211_hw *hw)
_initpabias(hw);
rtl92c_dm_init(hw);
exit:
+ local_irq_disable();
local_irq_restore(flags);
return err;
}
diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192de/phy.c b/drivers/net/wireless/realtek/rtlwifi/rtl8192de/phy.c
index 9b83c71..51fe51b 100644
--- a/drivers/net/wireless/realtek/rtlwifi/rtl8192de/phy.c
+++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192de/phy.c
@@ -899,7 +899,7 @@ static u8 _rtl92c_phy_get_rightchnlplace(u8 chnl)
u8 place = chnl;
if (chnl > 14) {
- for (place = 14; place < sizeof(channel5g); place++) {
+ for (place = 14; place < ARRAY_SIZE(channel5g); place++) {
if (channel5g[place] == chnl) {
place++;
break;
@@ -1366,7 +1366,7 @@ u8 rtl92d_get_rightchnlplace_for_iqk(u8 chnl)
u8 place;
if (chnl > 14) {
- for (place = 14; place < sizeof(channel_all); place++) {
+ for (place = 14; place < ARRAY_SIZE(channel_all); place++) {
if (channel_all[place] == chnl)
return place - 13;
}
@@ -2428,7 +2428,7 @@ static bool _rtl92d_is_legal_5g_channel(struct ieee80211_hw *hw, u8 channel)
int i;
- for (i = 0; i < sizeof(channel5g); i++)
+ for (i = 0; i < ARRAY_SIZE(channel5g); i++)
if (channel == channel5g[i])
return true;
return false;
@@ -2692,9 +2692,8 @@ void rtl92d_phy_reset_iqk_result(struct ieee80211_hw *hw)
u8 i;
rtl_dbg(rtlpriv, COMP_INIT, DBG_LOUD,
- "settings regs %d default regs %d\n",
- (int)(sizeof(rtlphy->iqk_matrix) /
- sizeof(struct iqk_matrix_regs)),
+ "settings regs %zu default regs %d\n",
+ ARRAY_SIZE(rtlphy->iqk_matrix),
IQK_MATRIX_REG_NUM);
/* 0xe94, 0xe9c, 0xea4, 0xeac, 0xeb4, 0xebc, 0xec4, 0xecc */
for (i = 0; i < IQK_MATRIX_SETTINGS_NUM; i++) {
@@ -2861,16 +2860,14 @@ u8 rtl92d_phy_sw_chnl(struct ieee80211_hw *hw)
case BAND_ON_5G:
/* Get first channel error when change between
* 5G and 2.4G band. */
- if (channel <= 14)
+ if (WARN_ONCE(channel <= 14, "rtl8192de: 5G but channel<=14\n"))
return 0;
- WARN_ONCE((channel <= 14), "rtl8192de: 5G but channel<=14\n");
break;
case BAND_ON_2_4G:
/* Get first channel error when change between
* 5G and 2.4G band. */
- if (channel > 14)
+ if (WARN_ONCE(channel > 14, "rtl8192de: 2G but channel>14\n"))
return 0;
- WARN_ONCE((channel > 14), "rtl8192de: 2G but channel>14\n");
break;
default:
WARN_ONCE(true, "rtl8192de: Invalid WirelessMode(%#x)!!\n",
diff --git a/drivers/net/wireless/realtek/rtlwifi/wifi.h b/drivers/net/wireless/realtek/rtlwifi/wifi.h
index aa07856..31f9e9e 100644
--- a/drivers/net/wireless/realtek/rtlwifi/wifi.h
+++ b/drivers/net/wireless/realtek/rtlwifi/wifi.h
@@ -108,7 +108,6 @@
#define CHANNEL_GROUP_IDX_5GM 6
#define CHANNEL_GROUP_IDX_5GH 9
#define CHANNEL_GROUP_MAX_5G 9
-#define CHANNEL_MAX_NUMBER_2G 14
#define AVG_THERMAL_NUM 8
#define AVG_THERMAL_NUM_88E 4
#define AVG_THERMAL_NUM_8723BE 4
diff --git a/drivers/net/wireless/realtek/rtw88/Makefile b/drivers/net/wireless/realtek/rtw88/Makefile
index 73d6807..834c66e 100644
--- a/drivers/net/wireless/realtek/rtw88/Makefile
+++ b/drivers/net/wireless/realtek/rtw88/Makefile
@@ -15,6 +15,7 @@
ps.o \
sec.o \
bf.o \
+ sar.o \
regd.o
rtw88_core-$(CONFIG_PM) += wow.o
diff --git a/drivers/net/wireless/realtek/rtw88/bf.c b/drivers/net/wireless/realtek/rtw88/bf.c
index aff70e4..df750b3 100644
--- a/drivers/net/wireless/realtek/rtw88/bf.c
+++ b/drivers/net/wireless/realtek/rtw88/bf.c
@@ -130,7 +130,8 @@ void rtw_bf_cfg_sounding(struct rtw_dev *rtwdev, struct rtw_vif *vif,
BIT_WMAC_USE_NDPARATE |
(csi_rsc << 13);
- rtw_write8(rtwdev, REG_SND_PTCL_CTRL, RTW_SND_CTRL_SOUNDING);
+ rtw_write8_mask(rtwdev, REG_SND_PTCL_CTRL, BIT_MASK_BEAMFORM,
+ RTW_SND_CTRL_SOUNDING);
rtw_write8(rtwdev, REG_SND_PTCL_CTRL + 3, 0x26);
rtw_write8_clr(rtwdev, REG_RXFLTMAP1, BIT_RXFLTMAP1_BF_REPORT_POLL);
rtw_write8_clr(rtwdev, REG_RXFLTMAP4, BIT_RXFLTMAP4_BF_REPORT_POLL);
@@ -177,7 +178,7 @@ void rtw_bf_del_bfer_entry_mu(struct rtw_dev *rtwdev)
void rtw_bf_del_sounding(struct rtw_dev *rtwdev)
{
- rtw_write8(rtwdev, REG_SND_PTCL_CTRL, 0);
+ rtw_write8_mask(rtwdev, REG_SND_PTCL_CTRL, BIT_MASK_BEAMFORM, 0);
}
void rtw_bf_enable_bfee_su(struct rtw_dev *rtwdev, struct rtw_vif *vif,
@@ -204,7 +205,8 @@ void rtw_bf_enable_bfee_su(struct rtw_dev *rtwdev, struct rtw_vif *vif,
}
/* Sounding protocol control */
- rtw_write8(rtwdev, REG_SND_PTCL_CTRL, RTW_SND_CTRL_SOUNDING);
+ rtw_write8_mask(rtwdev, REG_SND_PTCL_CTRL, BIT_MASK_BEAMFORM,
+ RTW_SND_CTRL_SOUNDING);
/* MAC address/Partial AID of Beamformer */
for (i = 0; i < ETH_ALEN; i++)
@@ -273,7 +275,8 @@ void rtw_bf_remove_bfee_su(struct rtw_dev *rtwdev,
struct rtw_bf_info *bfinfo = &rtwdev->bf_info;
rtw_dbg(rtwdev, RTW_DBG_BF, "remove as a su bfee\n");
- rtw_write8(rtwdev, REG_SND_PTCL_CTRL, RTW_SND_CTRL_REMOVE);
+ rtw_write8_mask(rtwdev, REG_SND_PTCL_CTRL, BIT_MASK_BEAMFORM,
+ RTW_SND_CTRL_REMOVE);
switch (bfee->su_reg_index) {
case 0:
@@ -298,7 +301,8 @@ void rtw_bf_remove_bfee_mu(struct rtw_dev *rtwdev,
{
struct rtw_bf_info *bfinfo = &rtwdev->bf_info;
- rtw_write8(rtwdev, REG_SND_PTCL_CTRL, RTW_SND_CTRL_REMOVE);
+ rtw_write8_mask(rtwdev, REG_SND_PTCL_CTRL, BIT_MASK_BEAMFORM,
+ RTW_SND_CTRL_REMOVE);
rtw_bf_del_bfer_entry_mu(rtwdev);
diff --git a/drivers/net/wireless/realtek/rtw88/bf.h b/drivers/net/wireless/realtek/rtw88/bf.h
index 17855ed..7b40c2c 100644
--- a/drivers/net/wireless/realtek/rtw88/bf.h
+++ b/drivers/net/wireless/realtek/rtw88/bf.h
@@ -13,6 +13,9 @@
#define REG_ASSOCIATED_BFMER1_INFO 0x06EC
#define REG_TX_CSI_RPT_PARAM_BW20 0x06F4
#define REG_SND_PTCL_CTRL 0x0718
+#define BIT_DIS_CHK_VHTSIGB_CRC BIT(6)
+#define BIT_DIS_CHK_VHTSIGA_CRC BIT(5)
+#define BIT_MASK_BEAMFORM (GENMASK(4, 0) | BIT(7))
#define REG_MU_TX_CTL 0x14C0
#define REG_MU_STA_GID_VLD 0x14C4
#define REG_MU_STA_USER_POS_INFO 0x14C8
@@ -42,8 +45,8 @@
#define BIT_RXFLTMAP4_BF_REPORT_POLL BIT(4)
#define RTW_NDP_RX_STANDBY_TIME 0x70
-#define RTW_SND_CTRL_REMOVE 0xD8
-#define RTW_SND_CTRL_SOUNDING 0xDB
+#define RTW_SND_CTRL_REMOVE 0x98
+#define RTW_SND_CTRL_SOUNDING 0x9B
enum csi_seg_len {
HAL_CSI_SEG_4K = 0,
diff --git a/drivers/net/wireless/realtek/rtw88/debug.c b/drivers/net/wireless/realtek/rtw88/debug.c
index 682b235..e429428 100644
--- a/drivers/net/wireless/realtek/rtw88/debug.c
+++ b/drivers/net/wireless/realtek/rtw88/debug.c
@@ -152,6 +152,22 @@ static int rtw_debugfs_get_rf_read(struct seq_file *m, void *v)
return 0;
}
+static int rtw_debugfs_get_fix_rate(struct seq_file *m, void *v)
+{
+ struct rtw_debugfs_priv *debugfs_priv = m->private;
+ struct rtw_dev *rtwdev = debugfs_priv->rtwdev;
+ struct rtw_dm_info *dm_info = &rtwdev->dm_info;
+ u8 fix_rate = dm_info->fix_rate;
+
+ if (fix_rate >= DESC_RATE_MAX) {
+ seq_printf(m, "Fix rate disabled, fix_rate = %u\n", fix_rate);
+ return 0;
+ }
+
+ seq_printf(m, "Data frames fixed at desc rate %u\n", fix_rate);
+ return 0;
+}
+
static int rtw_debugfs_copy_from_user(char tmp[], int size,
const char __user *buffer, size_t count,
int num)
@@ -437,6 +453,31 @@ static ssize_t rtw_debugfs_set_rf_read(struct file *filp,
return count;
}
+static ssize_t rtw_debugfs_set_fix_rate(struct file *filp,
+ const char __user *buffer,
+ size_t count, loff_t *loff)
+{
+ struct seq_file *seqpriv = (struct seq_file *)filp->private_data;
+ struct rtw_debugfs_priv *debugfs_priv = seqpriv->private;
+ struct rtw_dev *rtwdev = debugfs_priv->rtwdev;
+ struct rtw_dm_info *dm_info = &rtwdev->dm_info;
+ u8 fix_rate;
+ char tmp[32 + 1];
+ int ret;
+
+ rtw_debugfs_copy_from_user(tmp, sizeof(tmp), buffer, count, 1);
+
+ ret = kstrtou8(tmp, 0, &fix_rate);
+ if (ret) {
+ rtw_warn(rtwdev, "invalid args, [rate]\n");
+ return ret;
+ }
+
+ dm_info->fix_rate = fix_rate;
+
+ return count;
+}
+
static int rtw_debug_get_mac_page(struct seq_file *m, void *v)
{
struct rtw_debugfs_priv *debugfs_priv = m->private;
@@ -590,9 +631,11 @@ static int rtw_debugfs_get_tx_pwr_tbl(struct seq_file *m, void *v)
u8 ch = hal->current_channel;
u8 regd = rtw_regd_get(rtwdev);
+ seq_printf(m, "channel: %u\n", ch);
+ seq_printf(m, "bandwidth: %u\n", bw);
seq_printf(m, "regulatory: %s\n", rtw_get_regd_string(regd));
- seq_printf(m, "%-4s %-10s %-3s%6s %-4s %4s (%-4s %-4s) %-4s\n",
- "path", "rate", "pwr", "", "base", "", "byr", "lmt", "rem");
+ seq_printf(m, "%-4s %-10s %-9s %-9s (%-4s %-4s %-4s) %-4s\n",
+ "path", "rate", "pwr", "base", "byr", "lmt", "sar", "rem");
mutex_lock(&hal->tx_power_mutex);
for (path = RF_PATH_A; path <= RF_PATH_B; path++) {
@@ -614,13 +657,15 @@ static int rtw_debugfs_get_tx_pwr_tbl(struct seq_file *m, void *v)
seq_printf(m, "%4c ", path + 'A');
rtw_print_rate(m, rate);
- seq_printf(m, " %3u(0x%02x) %4u %4d (%4d %4d) %4d\n",
+ seq_printf(m, " %3u(0x%02x) %4u %4d (%4d %4d %4d) %4d\n",
hal->tx_pwr_tbl[path][rate],
hal->tx_pwr_tbl[path][rate],
pwr_param.pwr_base,
- min_t(s8, pwr_param.pwr_offset,
- pwr_param.pwr_limit),
+ min3(pwr_param.pwr_offset,
+ pwr_param.pwr_limit,
+ pwr_param.pwr_sar),
pwr_param.pwr_offset, pwr_param.pwr_limit,
+ pwr_param.pwr_sar,
pwr_param.pwr_remnant);
}
}
@@ -904,6 +949,39 @@ static int rtw_debugfs_get_fw_crash(struct seq_file *m, void *v)
return 0;
}
+static ssize_t rtw_debugfs_set_force_lowest_basic_rate(struct file *filp,
+ const char __user *buffer,
+ size_t count, loff_t *loff)
+{
+ struct seq_file *seqpriv = (struct seq_file *)filp->private_data;
+ struct rtw_debugfs_priv *debugfs_priv = seqpriv->private;
+ struct rtw_dev *rtwdev = debugfs_priv->rtwdev;
+ bool input;
+ int err;
+
+ err = kstrtobool_from_user(buffer, count, &input);
+ if (err)
+ return err;
+
+ if (input)
+ set_bit(RTW_FLAG_FORCE_LOWEST_RATE, rtwdev->flags);
+ else
+ clear_bit(RTW_FLAG_FORCE_LOWEST_RATE, rtwdev->flags);
+
+ return count;
+}
+
+static int rtw_debugfs_get_force_lowest_basic_rate(struct seq_file *m, void *v)
+{
+ struct rtw_debugfs_priv *debugfs_priv = m->private;
+ struct rtw_dev *rtwdev = debugfs_priv->rtwdev;
+
+ seq_printf(m, "force lowest basic rate: %d\n",
+ test_bit(RTW_FLAG_FORCE_LOWEST_RATE, rtwdev->flags));
+
+ return 0;
+}
+
static ssize_t rtw_debugfs_set_dm_cap(struct file *filp,
const char __user *buffer,
size_t count, loff_t *loff)
@@ -1061,6 +1139,11 @@ static struct rtw_debugfs_priv rtw_debug_priv_read_reg = {
.cb_read = rtw_debugfs_get_read_reg,
};
+static struct rtw_debugfs_priv rtw_debug_priv_fix_rate = {
+ .cb_write = rtw_debugfs_set_fix_rate,
+ .cb_read = rtw_debugfs_get_fix_rate,
+};
+
static struct rtw_debugfs_priv rtw_debug_priv_dump_cam = {
.cb_write = rtw_debugfs_set_single_input,
.cb_read = rtw_debugfs_get_dump_cam,
@@ -1094,6 +1177,11 @@ static struct rtw_debugfs_priv rtw_debug_priv_fw_crash = {
.cb_read = rtw_debugfs_get_fw_crash,
};
+static struct rtw_debugfs_priv rtw_debug_priv_force_lowest_basic_rate = {
+ .cb_write = rtw_debugfs_set_force_lowest_basic_rate,
+ .cb_read = rtw_debugfs_get_force_lowest_basic_rate,
+};
+
static struct rtw_debugfs_priv rtw_debug_priv_dm_cap = {
.cb_write = rtw_debugfs_set_dm_cap,
.cb_read = rtw_debugfs_get_dm_cap,
@@ -1126,6 +1214,7 @@ void rtw_debugfs_init(struct rtw_dev *rtwdev)
rtw_debugfs_add_rw(read_reg);
rtw_debugfs_add_w(rf_write);
rtw_debugfs_add_rw(rf_read);
+ rtw_debugfs_add_rw(fix_rate);
rtw_debugfs_add_rw(dump_cam);
rtw_debugfs_add_rw(rsvd_page);
rtw_debugfs_add_r(phy_info);
@@ -1174,6 +1263,7 @@ void rtw_debugfs_init(struct rtw_dev *rtwdev)
rtw_debugfs_add_r(tx_pwr_tbl);
rtw_debugfs_add_rw(edcca_enable);
rtw_debugfs_add_rw(fw_crash);
+ rtw_debugfs_add_rw(force_lowest_basic_rate);
rtw_debugfs_add_rw(dm_cap);
}
diff --git a/drivers/net/wireless/realtek/rtw88/debug.h b/drivers/net/wireless/realtek/rtw88/debug.h
index 47c57f3..61f8369 100644
--- a/drivers/net/wireless/realtek/rtw88/debug.h
+++ b/drivers/net/wireless/realtek/rtw88/debug.h
@@ -22,6 +22,7 @@ enum rtw_debug_mask {
RTW_DBG_CFO = 0x00002000,
RTW_DBG_PATH_DIV = 0x00004000,
RTW_DBG_ADAPTIVITY = 0x00008000,
+ RTW_DBG_HW_SCAN = 0x00010000,
RTW_DBG_ALL = 0xffffffff
};
diff --git a/drivers/net/wireless/realtek/rtw88/fw.c b/drivers/net/wireless/realtek/rtw88/fw.c
index 0c4f2a2..2f7c036 100644
--- a/drivers/net/wireless/realtek/rtw88/fw.c
+++ b/drivers/net/wireless/realtek/rtw88/fw.c
@@ -28,6 +28,12 @@ static void rtw_fw_c2h_cmd_handle_ext(struct rtw_dev *rtwdev,
case C2H_CCX_RPT:
rtw_tx_report_handle(rtwdev, skb, C2H_CCX_RPT);
break;
+ case C2H_SCAN_STATUS_RPT:
+ rtw_hw_scan_status_report(rtwdev, skb);
+ break;
+ case C2H_CHAN_SWITCH:
+ rtw_hw_scan_chan_switch(rtwdev, skb);
+ break;
default:
break;
}
@@ -1777,3 +1783,385 @@ void rtw_fw_scan_notify(struct rtw_dev *rtwdev, bool start)
rtw_fw_send_h2c_command(rtwdev, h2c_pkt);
}
+
+static void rtw_append_probe_req_ie(struct rtw_dev *rtwdev, struct sk_buff *skb,
+ struct sk_buff_head *list,
+ struct rtw_vif *rtwvif)
+{
+ struct ieee80211_scan_ies *ies = rtwvif->scan_ies;
+ struct rtw_chip_info *chip = rtwdev->chip;
+ struct sk_buff *new;
+ u8 idx;
+
+ for (idx = NL80211_BAND_2GHZ; idx < NUM_NL80211_BANDS; idx++) {
+ if (!(BIT(idx) & chip->band))
+ continue;
+ new = skb_copy(skb, GFP_KERNEL);
+ skb_put_data(new, ies->ies[idx], ies->len[idx]);
+ skb_put_data(new, ies->common_ies, ies->common_ie_len);
+ skb_queue_tail(list, new);
+ }
+}
+
+static int _rtw_hw_scan_update_probe_req(struct rtw_dev *rtwdev, u8 num_ssids,
+ struct sk_buff_head *probe_req_list)
+{
+ struct rtw_chip_info *chip = rtwdev->chip;
+ struct sk_buff *skb, *tmp;
+ u8 page_offset = 1, *buf, page_size = chip->page_size;
+ u8 pages = page_offset + num_ssids * RTW_PROBE_PG_CNT;
+ u16 pg_addr = rtwdev->fifo.rsvd_h2c_info_addr, loc;
+ u16 buf_offset = page_size * page_offset;
+ u8 tx_desc_sz = chip->tx_pkt_desc_sz;
+ unsigned int pkt_len;
+ int ret;
+
+ buf = kzalloc(page_size * pages, GFP_KERNEL);
+ if (!buf)
+ return -ENOMEM;
+
+ buf_offset -= tx_desc_sz;
+ skb_queue_walk_safe(probe_req_list, skb, tmp) {
+ skb_unlink(skb, probe_req_list);
+ rtw_fill_rsvd_page_desc(rtwdev, skb, RSVD_PROBE_REQ);
+ if (skb->len > page_size * RTW_PROBE_PG_CNT) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ memcpy(buf + buf_offset, skb->data, skb->len);
+ pkt_len = skb->len - tx_desc_sz;
+ loc = pg_addr - rtwdev->fifo.rsvd_boundary + page_offset;
+ __rtw_fw_update_pkt(rtwdev, RTW_PACKET_PROBE_REQ, pkt_len, loc);
+
+ buf_offset += RTW_PROBE_PG_CNT * page_size;
+ page_offset += RTW_PROBE_PG_CNT;
+ kfree_skb(skb);
+ }
+
+ ret = rtw_fw_write_data_rsvd_page(rtwdev, pg_addr, buf, buf_offset);
+ if (ret) {
+ rtw_err(rtwdev, "Download probe request to firmware failed\n");
+ goto out;
+ }
+
+ rtwdev->scan_info.probe_pg_size = page_offset;
+out:
+ kfree(buf);
+
+ return ret;
+}
+
+static int rtw_hw_scan_update_probe_req(struct rtw_dev *rtwdev,
+ struct rtw_vif *rtwvif)
+{
+ struct cfg80211_scan_request *req = rtwvif->scan_req;
+ struct sk_buff_head list;
+ struct sk_buff *skb;
+ u8 num = req->n_ssids, i;
+
+ skb_queue_head_init(&list);
+ for (i = 0; i < num; i++) {
+ skb = ieee80211_probereq_get(rtwdev->hw, rtwvif->mac_addr,
+ req->ssids[i].ssid,
+ req->ssids[i].ssid_len,
+ req->ie_len);
+ rtw_append_probe_req_ie(rtwdev, skb, &list, rtwvif);
+ kfree_skb(skb);
+ }
+
+ return _rtw_hw_scan_update_probe_req(rtwdev, num, &list);
+}
+
+static int rtw_add_chan_info(struct rtw_dev *rtwdev, struct rtw_chan_info *info,
+ struct rtw_chan_list *list, u8 *buf)
+{
+ u8 *chan = &buf[list->size];
+ u8 info_size = RTW_CH_INFO_SIZE;
+
+ if (list->size > list->buf_size)
+ return -ENOMEM;
+
+ CH_INFO_SET_CH(chan, info->channel);
+ CH_INFO_SET_PRI_CH_IDX(chan, info->pri_ch_idx);
+ CH_INFO_SET_BW(chan, info->bw);
+ CH_INFO_SET_TIMEOUT(chan, info->timeout);
+ CH_INFO_SET_ACTION_ID(chan, info->action_id);
+ CH_INFO_SET_EXTRA_INFO(chan, info->extra_info);
+ if (info->extra_info) {
+ EXTRA_CH_INFO_SET_ID(chan, RTW_SCAN_EXTRA_ID_DFS);
+ EXTRA_CH_INFO_SET_INFO(chan, RTW_SCAN_EXTRA_ACTION_SCAN);
+ EXTRA_CH_INFO_SET_SIZE(chan, RTW_EX_CH_INFO_SIZE -
+ RTW_EX_CH_INFO_HDR_SIZE);
+ EXTRA_CH_INFO_SET_DFS_EXT_TIME(chan, RTW_DFS_CHAN_TIME);
+ info_size += RTW_EX_CH_INFO_SIZE;
+ }
+ list->size += info_size;
+ list->ch_num++;
+
+ return 0;
+}
+
+static int rtw_add_chan_list(struct rtw_dev *rtwdev, struct rtw_vif *rtwvif,
+ struct rtw_chan_list *list, u8 *buf)
+{
+ struct cfg80211_scan_request *req = rtwvif->scan_req;
+ struct rtw_fifo_conf *fifo = &rtwdev->fifo;
+ struct ieee80211_channel *channel;
+ int i, ret = 0;
+
+ for (i = 0; i < req->n_channels; i++) {
+ struct rtw_chan_info ch_info = {0};
+
+ channel = req->channels[i];
+ ch_info.channel = channel->hw_value;
+ ch_info.bw = RTW_SCAN_WIDTH;
+ ch_info.pri_ch_idx = RTW_PRI_CH_IDX;
+ ch_info.timeout = req->duration_mandatory ?
+ req->duration : RTW_CHANNEL_TIME;
+
+ if (channel->flags & (IEEE80211_CHAN_RADAR | IEEE80211_CHAN_NO_IR)) {
+ ch_info.action_id = RTW_CHANNEL_RADAR;
+ ch_info.extra_info = 1;
+ /* Overwrite duration for passive scans if necessary */
+ ch_info.timeout = ch_info.timeout > RTW_PASS_CHAN_TIME ?
+ ch_info.timeout : RTW_PASS_CHAN_TIME;
+ } else {
+ ch_info.action_id = RTW_CHANNEL_ACTIVE;
+ }
+
+ ret = rtw_add_chan_info(rtwdev, &ch_info, list, buf);
+ if (ret)
+ return ret;
+ }
+
+ if (list->size > fifo->rsvd_pg_num << TX_PAGE_SIZE_SHIFT) {
+ rtw_err(rtwdev, "List exceeds rsvd page total size\n");
+ return -EINVAL;
+ }
+
+ list->addr = fifo->rsvd_h2c_info_addr + rtwdev->scan_info.probe_pg_size;
+ ret = rtw_fw_write_data_rsvd_page(rtwdev, list->addr, buf, list->size);
+ if (ret)
+ rtw_err(rtwdev, "Download channel list failed\n");
+
+ return ret;
+}
+
+static void rtw_fw_set_scan_offload(struct rtw_dev *rtwdev,
+ struct rtw_ch_switch_option *opt,
+ struct rtw_vif *rtwvif,
+ struct rtw_chan_list *list)
+{
+ struct rtw_hw_scan_info *scan_info = &rtwdev->scan_info;
+ struct cfg80211_scan_request *req = rtwvif->scan_req;
+ struct rtw_fifo_conf *fifo = &rtwdev->fifo;
+ /* reserve one dummy page at the beginning for tx descriptor */
+ u8 pkt_loc = fifo->rsvd_h2c_info_addr - fifo->rsvd_boundary + 1;
+ bool random_seq = req->flags & NL80211_SCAN_FLAG_RANDOM_SN;
+ u8 h2c_pkt[H2C_PKT_SIZE] = {0};
+
+ rtw_h2c_pkt_set_header(h2c_pkt, H2C_PKT_SCAN_OFFLOAD);
+ SET_PKT_H2C_TOTAL_LEN(h2c_pkt, H2C_PKT_CH_SWITCH_LEN);
+
+ SCAN_OFFLOAD_SET_START(h2c_pkt, opt->switch_en);
+ SCAN_OFFLOAD_SET_BACK_OP_EN(h2c_pkt, opt->back_op_en);
+ SCAN_OFFLOAD_SET_RANDOM_SEQ_EN(h2c_pkt, random_seq);
+ SCAN_OFFLOAD_SET_NO_CCK_EN(h2c_pkt, req->no_cck);
+ SCAN_OFFLOAD_SET_CH_NUM(h2c_pkt, list->ch_num);
+ SCAN_OFFLOAD_SET_CH_INFO_SIZE(h2c_pkt, list->size);
+ SCAN_OFFLOAD_SET_CH_INFO_LOC(h2c_pkt, list->addr - fifo->rsvd_boundary);
+ SCAN_OFFLOAD_SET_OP_CH(h2c_pkt, scan_info->op_chan);
+ SCAN_OFFLOAD_SET_OP_PRI_CH_IDX(h2c_pkt, scan_info->op_pri_ch_idx);
+ SCAN_OFFLOAD_SET_OP_BW(h2c_pkt, scan_info->op_bw);
+ SCAN_OFFLOAD_SET_OP_PORT_ID(h2c_pkt, rtwvif->port);
+ SCAN_OFFLOAD_SET_OP_DWELL_TIME(h2c_pkt, req->duration_mandatory ?
+ req->duration : RTW_CHANNEL_TIME);
+ SCAN_OFFLOAD_SET_OP_GAP_TIME(h2c_pkt, RTW_OFF_CHAN_TIME);
+ SCAN_OFFLOAD_SET_SSID_NUM(h2c_pkt, req->n_ssids);
+ SCAN_OFFLOAD_SET_PKT_LOC(h2c_pkt, pkt_loc);
+
+ rtw_fw_send_h2c_packet(rtwdev, h2c_pkt);
+}
+
+void rtw_hw_scan_start(struct rtw_dev *rtwdev, struct ieee80211_vif *vif,
+ struct ieee80211_scan_request *scan_req)
+{
+ struct rtw_vif *rtwvif = (struct rtw_vif *)vif->drv_priv;
+ struct cfg80211_scan_request *req = &scan_req->req;
+ u8 mac_addr[ETH_ALEN];
+
+ rtwdev->scan_info.scanning_vif = vif;
+ rtwvif->scan_ies = &scan_req->ies;
+ rtwvif->scan_req = req;
+
+ ieee80211_stop_queues(rtwdev->hw);
+ if (req->flags & NL80211_SCAN_FLAG_RANDOM_ADDR)
+ get_random_mask_addr(mac_addr, req->mac_addr,
+ req->mac_addr_mask);
+ else
+ ether_addr_copy(mac_addr, vif->addr);
+
+ rtw_core_scan_start(rtwdev, rtwvif, mac_addr, true);
+
+ rtwdev->hal.rcr &= ~BIT_CBSSID_BCN;
+ rtw_write32(rtwdev, REG_RCR, rtwdev->hal.rcr);
+}
+
+void rtw_hw_scan_complete(struct rtw_dev *rtwdev, struct ieee80211_vif *vif,
+ bool aborted)
+{
+ struct cfg80211_scan_info info = {
+ .aborted = aborted,
+ };
+ struct rtw_vif *rtwvif;
+
+ if (!vif)
+ return;
+
+ rtwdev->hal.rcr |= BIT_CBSSID_BCN;
+ rtw_write32(rtwdev, REG_RCR, rtwdev->hal.rcr);
+
+ rtw_core_scan_complete(rtwdev, vif);
+
+ ieee80211_wake_queues(rtwdev->hw);
+ ieee80211_scan_completed(rtwdev->hw, &info);
+
+ rtwvif = (struct rtw_vif *)vif->drv_priv;
+ rtwvif->scan_req = NULL;
+ rtwvif->scan_ies = NULL;
+ rtwdev->scan_info.scanning_vif = NULL;
+}
+
+static int rtw_hw_scan_prehandle(struct rtw_dev *rtwdev, struct rtw_vif *rtwvif,
+ struct rtw_chan_list *list)
+{
+ struct cfg80211_scan_request *req = rtwvif->scan_req;
+ int size = req->n_channels * (RTW_CH_INFO_SIZE + RTW_EX_CH_INFO_SIZE);
+ u8 *buf;
+ int ret;
+
+ buf = kmalloc(size, GFP_KERNEL);
+ if (!buf)
+ return -ENOMEM;
+
+ ret = rtw_hw_scan_update_probe_req(rtwdev, rtwvif);
+ if (ret) {
+ rtw_err(rtwdev, "Update probe request failed\n");
+ goto out;
+ }
+
+ list->buf_size = size;
+ list->size = 0;
+ list->ch_num = 0;
+ ret = rtw_add_chan_list(rtwdev, rtwvif, list, buf);
+out:
+ kfree(buf);
+
+ return ret;
+}
+
+int rtw_hw_scan_offload(struct rtw_dev *rtwdev, struct ieee80211_vif *vif,
+ bool enable)
+{
+ struct rtw_vif *rtwvif = vif ? (struct rtw_vif *)vif->drv_priv : NULL;
+ struct rtw_ch_switch_option cs_option = {0};
+ struct rtw_chan_list chan_list = {0};
+ int ret = 0;
+
+ if (!rtwvif)
+ return -EINVAL;
+
+ cs_option.switch_en = enable;
+ cs_option.back_op_en = rtwvif->net_type == RTW_NET_MGD_LINKED;
+ if (enable) {
+ ret = rtw_hw_scan_prehandle(rtwdev, rtwvif, &chan_list);
+ if (ret)
+ goto out;
+ }
+ rtw_fw_set_scan_offload(rtwdev, &cs_option, rtwvif, &chan_list);
+out:
+ return ret;
+}
+
+void rtw_hw_scan_abort(struct rtw_dev *rtwdev, struct ieee80211_vif *vif)
+{
+ if (!rtw_fw_feature_check(&rtwdev->fw, FW_FEATURE_SCAN_OFFLOAD))
+ return;
+
+ rtw_hw_scan_offload(rtwdev, vif, false);
+ rtw_hw_scan_complete(rtwdev, vif, true);
+}
+
+void rtw_hw_scan_status_report(struct rtw_dev *rtwdev, struct sk_buff *skb)
+{
+ struct ieee80211_vif *vif = rtwdev->scan_info.scanning_vif;
+ struct rtw_c2h_cmd *c2h;
+ bool aborted;
+ u8 rc;
+
+ if (!test_bit(RTW_FLAG_SCANNING, rtwdev->flags))
+ return;
+
+ c2h = get_c2h_from_skb(skb);
+ rc = GET_SCAN_REPORT_RETURN_CODE(c2h->payload);
+ aborted = rc != RTW_SCAN_REPORT_SUCCESS;
+ rtw_hw_scan_complete(rtwdev, vif, aborted);
+
+ if (aborted)
+ rtw_info(rtwdev, "HW scan aborted with code: %d\n", rc);
+}
+
+void rtw_store_op_chan(struct rtw_dev *rtwdev)
+{
+ struct rtw_hw_scan_info *scan_info = &rtwdev->scan_info;
+ struct rtw_hal *hal = &rtwdev->hal;
+
+ scan_info->op_chan = hal->current_channel;
+ scan_info->op_bw = hal->current_band_width;
+ scan_info->op_pri_ch_idx = hal->current_primary_channel_index;
+}
+
+static bool rtw_is_op_chan(struct rtw_dev *rtwdev, u8 channel)
+{
+ struct rtw_hw_scan_info *scan_info = &rtwdev->scan_info;
+
+ return channel == scan_info->op_chan;
+}
+
+void rtw_hw_scan_chan_switch(struct rtw_dev *rtwdev, struct sk_buff *skb)
+{
+ struct rtw_hal *hal = &rtwdev->hal;
+ struct rtw_c2h_cmd *c2h;
+ enum rtw_scan_notify_id id;
+ u8 chan, status;
+
+ c2h = get_c2h_from_skb(skb);
+ chan = GET_CHAN_SWITCH_CENTRAL_CH(c2h->payload);
+ id = GET_CHAN_SWITCH_ID(c2h->payload);
+ status = GET_CHAN_SWITCH_STATUS(c2h->payload);
+
+ if (id == RTW_SCAN_NOTIFY_ID_POSTSWITCH) {
+ if (rtw_is_op_chan(rtwdev, chan))
+ ieee80211_wake_queues(rtwdev->hw);
+ hal->current_channel = chan;
+ hal->current_band_type = chan > 14 ? RTW_BAND_5G : RTW_BAND_2G;
+ } else if (id == RTW_SCAN_NOTIFY_ID_PRESWITCH) {
+ if (IS_CH_5G_BAND(chan)) {
+ rtw_coex_switchband_notify(rtwdev, COEX_SWITCH_TO_5G);
+ } else if (IS_CH_2G_BAND(chan)) {
+ u8 chan_type;
+
+ if (test_bit(RTW_FLAG_SCANNING, rtwdev->flags))
+ chan_type = COEX_SWITCH_TO_24G;
+ else
+ chan_type = COEX_SWITCH_TO_24G_NOFORSCAN;
+ rtw_coex_switchband_notify(rtwdev, chan_type);
+ }
+ if (rtw_is_op_chan(rtwdev, chan))
+ ieee80211_stop_queues(rtwdev->hw);
+ }
+
+ rtw_dbg(rtwdev, RTW_DBG_HW_SCAN,
+ "Chan switch: %x, id: %x, status: %x\n", chan, id, status);
+}
diff --git a/drivers/net/wireless/realtek/rtw88/fw.h b/drivers/net/wireless/realtek/rtw88/fw.h
index 09c7afb9..654c3c2 100644
--- a/drivers/net/wireless/realtek/rtw88/fw.h
+++ b/drivers/net/wireless/realtek/rtw88/fw.h
@@ -32,6 +32,17 @@
#define SCAN_NOTIFY_TIMEOUT msecs_to_jiffies(10)
+#define RTW_CHANNEL_TIME 45
+#define RTW_OFF_CHAN_TIME 100
+#define RTW_PASS_CHAN_TIME 105
+#define RTW_DFS_CHAN_TIME 20
+#define RTW_CH_INFO_SIZE 4
+#define RTW_EX_CH_INFO_SIZE 3
+#define RTW_EX_CH_INFO_HDR_SIZE 2
+#define RTW_SCAN_WIDTH 0
+#define RTW_PRI_CH_IDX 1
+#define RTW_PROBE_PG_CNT 2
+
enum rtw_c2h_cmd_id {
C2H_CCX_TX_RPT = 0x03,
C2H_BT_INFO = 0x09,
@@ -48,7 +59,9 @@ enum rtw_c2h_cmd_id {
};
enum rtw_c2h_cmd_id_ext {
+ C2H_SCAN_STATUS_RPT = 0x3,
C2H_CCX_RPT = 0x0f,
+ C2H_CHAN_SWITCH = 0x22,
};
struct rtw_c2h_cmd {
@@ -98,9 +111,11 @@ enum rtw_fw_feature {
FW_FEATURE_LPS_C2H = BIT(1),
FW_FEATURE_LCLK = BIT(2),
FW_FEATURE_PG = BIT(3),
+ FW_FEATURE_TX_WAKE = BIT(4),
FW_FEATURE_BCN_FILTER = BIT(5),
FW_FEATURE_NOTIFY_SCAN = BIT(6),
FW_FEATURE_ADAPTIVITY = BIT(7),
+ FW_FEATURE_SCAN_OFFLOAD = BIT(8),
FW_FEATURE_MAX = BIT(31),
};
@@ -196,6 +211,51 @@ struct rtw_fw_wow_disconnect_para {
u8 retry_count;
};
+enum rtw_channel_type {
+ RTW_CHANNEL_PASSIVE,
+ RTW_CHANNEL_ACTIVE,
+ RTW_CHANNEL_RADAR,
+};
+
+enum rtw_scan_extra_id {
+ RTW_SCAN_EXTRA_ID_DFS,
+};
+
+enum rtw_scan_extra_info {
+ RTW_SCAN_EXTRA_ACTION_SCAN,
+};
+
+enum rtw_scan_report_code {
+ RTW_SCAN_REPORT_SUCCESS = 0x00,
+ RTW_SCAN_REPORT_ERR_PHYDM = 0x01,
+ RTW_SCAN_REPORT_ERR_ID = 0x02,
+ RTW_SCAN_REPORT_ERR_TX = 0x03,
+ RTW_SCAN_REPORT_CANCELED = 0x10,
+ RTW_SCAN_REPORT_CANCELED_EXT = 0x11,
+ RTW_SCAN_REPORT_FW_DISABLED = 0xF0,
+};
+
+enum rtw_scan_notify_id {
+ RTW_SCAN_NOTIFY_ID_PRESWITCH = 0x00,
+ RTW_SCAN_NOTIFY_ID_POSTSWITCH = 0x01,
+ RTW_SCAN_NOTIFY_ID_PROBE_PRETX = 0x02,
+ RTW_SCAN_NOTIFY_ID_PROBE_ISSUETX = 0x03,
+ RTW_SCAN_NOTIFY_ID_NULL0_PRETX = 0x04,
+ RTW_SCAN_NOTIFY_ID_NULL0_ISSUETX = 0x05,
+ RTW_SCAN_NOTIFY_ID_NULL0_POSTTX = 0x06,
+ RTW_SCAN_NOTIFY_ID_NULL1_PRETX = 0x07,
+ RTW_SCAN_NOTIFY_ID_NULL1_ISSUETX = 0x08,
+ RTW_SCAN_NOTIFY_ID_NULL1_POSTTX = 0x09,
+ RTW_SCAN_NOTIFY_ID_DWELLEXT = 0x0A,
+};
+
+enum rtw_scan_notify_status {
+ RTW_SCAN_NOTIFY_STATUS_SUCCESS = 0x00,
+ RTW_SCAN_NOTIFY_STATUS_FAILURE = 0x01,
+ RTW_SCAN_NOTIFY_STATUS_RESOURCE = 0x02,
+ RTW_SCAN_NOTIFY_STATUS_TIMEOUT = 0x03,
+};
+
struct rtw_ch_switch_option {
u8 periodic_option;
u32 tsf_high;
@@ -209,6 +269,8 @@ struct rtw_ch_switch_option {
u8 slow_period;
u8 slow_period_sel;
u8 nlo_en;
+ bool switch_en;
+ bool back_op_en;
};
struct rtw_fw_hdr {
@@ -265,6 +327,11 @@ struct rtw_fw_hdr_legacy {
#define GET_CCX_REPORT_SEQNUM_V1(c2h_payload) (c2h_payload[8] & 0xfc)
#define GET_CCX_REPORT_STATUS_V1(c2h_payload) (c2h_payload[9] & 0xc0)
+#define GET_SCAN_REPORT_RETURN_CODE(c2h_payload) (c2h_payload[2] & 0xff)
+
+#define GET_CHAN_SWITCH_CENTRAL_CH(c2h_payload) (c2h_payload[2])
+#define GET_CHAN_SWITCH_ID(c2h_payload) (c2h_payload[3])
+#define GET_CHAN_SWITCH_STATUS(c2h_payload) (c2h_payload[4])
#define GET_RA_REPORT_RATE(c2h_payload) (c2h_payload[0] & 0x7f)
#define GET_RA_REPORT_SGI(c2h_payload) ((c2h_payload[0] & 0x80) >> 7)
#define GET_RA_REPORT_BW(c2h_payload) (c2h_payload[6])
@@ -284,6 +351,7 @@ struct rtw_fw_hdr_legacy {
#define H2C_PKT_CH_SWITCH 0x02
#define H2C_PKT_UPDATE_PKT 0x0C
+#define H2C_PKT_SCAN_OFFLOAD 0x19
#define H2C_PKT_CH_SWITCH_LEN 0x20
#define H2C_PKT_UPDATE_PKT_LEN 0x4
@@ -334,6 +402,30 @@ static inline void rtw_h2c_pkt_set_header(u8 *h2c_pkt, u8 sub_id)
le32p_replace_bits((__le32 *)(pkt) + 0x00, value, GENMASK(23, 16))
#define CHSW_INFO_SET_ACTION_ID(pkt, value) \
le32p_replace_bits((__le32 *)(pkt) + 0x00, value, GENMASK(30, 24))
+#define CHSW_INFO_SET_EXTRA_INFO(pkt, value) \
+ le32p_replace_bits((__le32 *)(pkt) + 0x00, value, BIT(31))
+
+#define CH_INFO_SET_CH(pkt, value) \
+ u8p_replace_bits((u8 *)(pkt) + 0x00, value, GENMASK(7, 0))
+#define CH_INFO_SET_PRI_CH_IDX(pkt, value) \
+ u8p_replace_bits((u8 *)(pkt) + 0x01, value, GENMASK(3, 0))
+#define CH_INFO_SET_BW(pkt, value) \
+ u8p_replace_bits((u8 *)(pkt) + 0x01, value, GENMASK(7, 4))
+#define CH_INFO_SET_TIMEOUT(pkt, value) \
+ u8p_replace_bits((u8 *)(pkt) + 0x02, value, GENMASK(7, 0))
+#define CH_INFO_SET_ACTION_ID(pkt, value) \
+ u8p_replace_bits((u8 *)(pkt) + 0x03, value, GENMASK(6, 0))
+#define CH_INFO_SET_EXTRA_INFO(pkt, value) \
+ u8p_replace_bits((u8 *)(pkt) + 0x03, value, BIT(7))
+
+#define EXTRA_CH_INFO_SET_ID(pkt, value) \
+ u8p_replace_bits((u8 *)(pkt) + 0x04, value, GENMASK(6, 0))
+#define EXTRA_CH_INFO_SET_INFO(pkt, value) \
+ u8p_replace_bits((u8 *)(pkt) + 0x04, value, BIT(7))
+#define EXTRA_CH_INFO_SET_SIZE(pkt, value) \
+ u8p_replace_bits((u8 *)(pkt) + 0x05, value, GENMASK(7, 0))
+#define EXTRA_CH_INFO_SET_DFS_EXT_TIME(pkt, value) \
+ u8p_replace_bits((u8 *)(pkt) + 0x06, value, GENMASK(7, 0))
#define UPDATE_PKT_SET_SIZE(h2c_pkt, value) \
le32p_replace_bits((__le32 *)(h2c_pkt) + 0x02, value, GENMASK(15, 0))
@@ -350,12 +442,18 @@ static inline void rtw_h2c_pkt_set_header(u8 *h2c_pkt, u8 sub_id)
le32p_replace_bits((__le32 *)(h2c_pkt) + 0x02, value, BIT(2))
#define CH_SWITCH_SET_PERIODIC_OPT(h2c_pkt, value) \
le32p_replace_bits((__le32 *)(h2c_pkt) + 0x02, value, GENMASK(4, 3))
+#define CH_SWITCH_SET_SCAN_MODE(h2c_pkt, value) \
+ le32p_replace_bits((__le32 *)(h2c_pkt) + 0x02, value, BIT(5))
+#define CH_SWITCH_SET_BACK_OP_EN(h2c_pkt, value) \
+ le32p_replace_bits((__le32 *)(h2c_pkt) + 0x02, value, BIT(6))
#define CH_SWITCH_SET_INFO_LOC(h2c_pkt, value) \
le32p_replace_bits((__le32 *)(h2c_pkt) + 0x02, value, GENMASK(15, 8))
#define CH_SWITCH_SET_CH_NUM(h2c_pkt, value) \
le32p_replace_bits((__le32 *)(h2c_pkt) + 0x02, value, GENMASK(23, 16))
#define CH_SWITCH_SET_PRI_CH_IDX(h2c_pkt, value) \
le32p_replace_bits((__le32 *)(h2c_pkt) + 0x02, value, GENMASK(27, 24))
+#define CH_SWITCH_SET_DEST_BW(h2c_pkt, value) \
+ le32p_replace_bits((__le32 *)(h2c_pkt) + 0x02, value, GENMASK(31, 28))
#define CH_SWITCH_SET_DEST_CH(h2c_pkt, value) \
le32p_replace_bits((__le32 *)(h2c_pkt) + 0x03, value, GENMASK(7, 0))
#define CH_SWITCH_SET_NORMAL_PERIOD(h2c_pkt, value) \
@@ -375,6 +473,41 @@ static inline void rtw_h2c_pkt_set_header(u8 *h2c_pkt, u8 sub_id)
#define CH_SWITCH_SET_INFO_SIZE(h2c_pkt, value) \
le32p_replace_bits((__le32 *)(h2c_pkt) + 0x06, value, GENMASK(15, 0))
+#define SCAN_OFFLOAD_SET_START(h2c_pkt, value) \
+ le32p_replace_bits((__le32 *)(h2c_pkt) + 0x02, value, BIT(0))
+#define SCAN_OFFLOAD_SET_BACK_OP_EN(h2c_pkt, value) \
+ le32p_replace_bits((__le32 *)(h2c_pkt) + 0x02, value, BIT(1))
+#define SCAN_OFFLOAD_SET_RANDOM_SEQ_EN(h2c_pkt, value) \
+ le32p_replace_bits((__le32 *)(h2c_pkt) + 0x02, value, BIT(2))
+#define SCAN_OFFLOAD_SET_NO_CCK_EN(h2c_pkt, value) \
+ le32p_replace_bits((__le32 *)(h2c_pkt) + 0x02, value, BIT(3))
+#define SCAN_OFFLOAD_SET_VERBOSE(h2c_pkt, value) \
+ le32p_replace_bits((__le32 *)(h2c_pkt) + 0x02, value, BIT(4))
+#define SCAN_OFFLOAD_SET_CH_NUM(h2c_pkt, value) \
+ le32p_replace_bits((__le32 *)(h2c_pkt) + 0x02, value, GENMASK(15, 8))
+#define SCAN_OFFLOAD_SET_CH_INFO_SIZE(h2c_pkt, value) \
+ le32p_replace_bits((__le32 *)(h2c_pkt) + 0x02, value, GENMASK(31, 16))
+#define SCAN_OFFLOAD_SET_CH_INFO_LOC(h2c_pkt, value) \
+ le32p_replace_bits((__le32 *)(h2c_pkt) + 0x03, value, GENMASK(7, 0))
+#define SCAN_OFFLOAD_SET_OP_CH(h2c_pkt, value) \
+ le32p_replace_bits((__le32 *)(h2c_pkt) + 0x03, value, GENMASK(15, 8))
+#define SCAN_OFFLOAD_SET_OP_PRI_CH_IDX(h2c_pkt, value) \
+ le32p_replace_bits((__le32 *)(h2c_pkt) + 0x03, value, GENMASK(19, 16))
+#define SCAN_OFFLOAD_SET_OP_BW(h2c_pkt, value) \
+ le32p_replace_bits((__le32 *)(h2c_pkt) + 0x03, value, GENMASK(23, 20))
+#define SCAN_OFFLOAD_SET_OP_PORT_ID(h2c_pkt, value) \
+ le32p_replace_bits((__le32 *)(h2c_pkt) + 0x03, value, GENMASK(26, 24))
+#define SCAN_OFFLOAD_SET_OP_DWELL_TIME(h2c_pkt, value) \
+ le32p_replace_bits((__le32 *)(h2c_pkt) + 0x04, value, GENMASK(15, 0))
+#define SCAN_OFFLOAD_SET_OP_GAP_TIME(h2c_pkt, value) \
+ le32p_replace_bits((__le32 *)(h2c_pkt) + 0x04, value, GENMASK(31, 16))
+#define SCAN_OFFLOAD_SET_MODE(h2c_pkt, value) \
+ le32p_replace_bits((__le32 *)(h2c_pkt) + 0x05, value, GENMASK(3, 0))
+#define SCAN_OFFLOAD_SET_SSID_NUM(h2c_pkt, value) \
+ le32p_replace_bits((__le32 *)(h2c_pkt) + 0x05, value, GENMASK(7, 4))
+#define SCAN_OFFLOAD_SET_PKT_LOC(h2c_pkt, value) \
+ le32p_replace_bits((__le32 *)(h2c_pkt) + 0x05, value, GENMASK(15, 8))
+
/* Command H2C */
#define H2C_CMD_RSVD_PAGE 0x0
#define H2C_CMD_MEDIA_STATUS_RPT 0x01
@@ -686,4 +819,14 @@ int rtw_fw_dump_fifo(struct rtw_dev *rtwdev, u8 fifo_sel, u32 addr, u32 size,
u32 *buffer);
void rtw_fw_scan_notify(struct rtw_dev *rtwdev, bool start);
void rtw_fw_adaptivity(struct rtw_dev *rtwdev);
+void rtw_store_op_chan(struct rtw_dev *rtwdev);
+void rtw_hw_scan_start(struct rtw_dev *rtwdev, struct ieee80211_vif *vif,
+ struct ieee80211_scan_request *req);
+void rtw_hw_scan_complete(struct rtw_dev *rtwdev, struct ieee80211_vif *vif,
+ bool aborted);
+int rtw_hw_scan_offload(struct rtw_dev *rtwdev, struct ieee80211_vif *vif,
+ bool enable);
+void rtw_hw_scan_status_report(struct rtw_dev *rtwdev, struct sk_buff *skb);
+void rtw_hw_scan_chan_switch(struct rtw_dev *rtwdev, struct sk_buff *skb);
+void rtw_hw_scan_abort(struct rtw_dev *rtwdev, struct ieee80211_vif *vif);
#endif
diff --git a/drivers/net/wireless/realtek/rtw88/mac80211.c b/drivers/net/wireless/realtek/rtw88/mac80211.c
index 6f56298..ae7d97d 100644
--- a/drivers/net/wireless/realtek/rtw88/mac80211.c
+++ b/drivers/net/wireless/realtek/rtw88/mac80211.c
@@ -13,6 +13,7 @@
#include "bf.h"
#include "debug.h"
#include "wow.h"
+#include "sar.h"
static void rtw_ops_tx(struct ieee80211_hw *hw,
struct ieee80211_tx_control *control,
@@ -161,6 +162,7 @@ static int rtw_ops_add_interface(struct ieee80211_hw *hw,
rtwvif->stats.rx_unicast = 0;
rtwvif->stats.tx_cnt = 0;
rtwvif->stats.rx_cnt = 0;
+ rtwvif->scan_req = NULL;
memset(&rtwvif->bfee, 0, sizeof(struct rtw_bfee));
rtwvif->conf = &rtw_vif_port[port];
rtw_txq_init(rtwdev, vif->txq);
@@ -372,9 +374,15 @@ static void rtw_ops_bss_info_changed(struct ieee80211_hw *hw,
rtw_coex_media_status_notify(rtwdev, conf->assoc);
if (rtw_bf_support)
rtw_bf_assoc(rtwdev, vif, conf);
+ rtw_store_op_chan(rtwdev);
} else {
rtw_leave_lps(rtwdev);
rtw_bf_disassoc(rtwdev, vif, conf);
+ /* Abort ongoing scan if cancel_scan isn't issued
+ * when disconnected by peer
+ */
+ if (test_bit(RTW_FLAG_SCANNING, rtwdev->flags))
+ rtw_hw_scan_abort(rtwdev, vif);
}
config |= PORT_SET_NET_TYPE;
@@ -594,22 +602,9 @@ static void rtw_ops_sw_scan_start(struct ieee80211_hw *hw,
{
struct rtw_dev *rtwdev = hw->priv;
struct rtw_vif *rtwvif = (struct rtw_vif *)vif->drv_priv;
- u32 config = 0;
mutex_lock(&rtwdev->mutex);
-
- rtw_leave_lps(rtwdev);
-
- ether_addr_copy(rtwvif->mac_addr, mac_addr);
- config |= PORT_SET_MAC_ADDR;
- rtw_vif_port_config(rtwdev, rtwvif, config);
-
- rtw_coex_scan_notify(rtwdev, COEX_SCAN_START);
- rtw_core_fw_scan_notify(rtwdev, true);
-
- set_bit(RTW_FLAG_DIG_DISABLE, rtwdev->flags);
- set_bit(RTW_FLAG_SCANNING, rtwdev->flags);
-
+ rtw_core_scan_start(rtwdev, rtwvif, mac_addr, false);
mutex_unlock(&rtwdev->mutex);
}
@@ -617,22 +612,9 @@ static void rtw_ops_sw_scan_complete(struct ieee80211_hw *hw,
struct ieee80211_vif *vif)
{
struct rtw_dev *rtwdev = hw->priv;
- struct rtw_vif *rtwvif = (struct rtw_vif *)vif->drv_priv;
- u32 config = 0;
mutex_lock(&rtwdev->mutex);
-
- clear_bit(RTW_FLAG_SCANNING, rtwdev->flags);
- clear_bit(RTW_FLAG_DIG_DISABLE, rtwdev->flags);
-
- rtw_core_fw_scan_notify(rtwdev, false);
-
- ether_addr_copy(rtwvif->mac_addr, vif->addr);
- config |= PORT_SET_MAC_ADDR;
- rtw_vif_port_config(rtwdev, rtwvif, config);
-
- rtw_coex_scan_notify(rtwdev, COEX_SCAN_FINISH);
-
+ rtw_core_scan_complete(rtwdev, vif);
mutex_unlock(&rtwdev->mutex);
}
@@ -815,6 +797,56 @@ static void rtw_reconfig_complete(struct ieee80211_hw *hw,
mutex_unlock(&rtwdev->mutex);
}
+static int rtw_ops_hw_scan(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
+ struct ieee80211_scan_request *req)
+{
+ struct rtw_dev *rtwdev = hw->priv;
+ int ret;
+
+ if (!rtw_fw_feature_check(&rtwdev->fw, FW_FEATURE_SCAN_OFFLOAD))
+ return 1;
+
+ if (test_bit(RTW_FLAG_SCANNING, rtwdev->flags))
+ return -EBUSY;
+
+ mutex_lock(&rtwdev->mutex);
+ rtw_hw_scan_start(rtwdev, vif, req);
+ ret = rtw_hw_scan_offload(rtwdev, vif, true);
+ if (ret) {
+ rtw_hw_scan_abort(rtwdev, vif);
+ rtw_err(rtwdev, "HW scan failed with status: %d\n", ret);
+ }
+ mutex_unlock(&rtwdev->mutex);
+
+ return ret;
+}
+
+static void rtw_ops_cancel_hw_scan(struct ieee80211_hw *hw,
+ struct ieee80211_vif *vif)
+{
+ struct rtw_dev *rtwdev = hw->priv;
+
+ if (!rtw_fw_feature_check(&rtwdev->fw, FW_FEATURE_SCAN_OFFLOAD))
+ return;
+
+ if (!test_bit(RTW_FLAG_SCANNING, rtwdev->flags))
+ return;
+
+ mutex_lock(&rtwdev->mutex);
+ rtw_hw_scan_abort(rtwdev, vif);
+ mutex_unlock(&rtwdev->mutex);
+}
+
+static int rtw_ops_set_sar_specs(struct ieee80211_hw *hw,
+ const struct cfg80211_sar_specs *sar)
+{
+ struct rtw_dev *rtwdev = hw->priv;
+
+ rtw_set_sar_specs(rtwdev, sar);
+
+ return 0;
+}
+
const struct ieee80211_ops rtw_ops = {
.tx = rtw_ops_tx,
.wake_tx_queue = rtw_ops_wake_tx_queue,
@@ -842,6 +874,9 @@ const struct ieee80211_ops rtw_ops = {
.set_antenna = rtw_ops_set_antenna,
.get_antenna = rtw_ops_get_antenna,
.reconfig_complete = rtw_reconfig_complete,
+ .hw_scan = rtw_ops_hw_scan,
+ .cancel_hw_scan = rtw_ops_cancel_hw_scan,
+ .set_sar_specs = rtw_ops_set_sar_specs,
#ifdef CONFIG_PM
.suspend = rtw_ops_suspend,
.resume = rtw_ops_resume,
diff --git a/drivers/net/wireless/realtek/rtw88/main.c b/drivers/net/wireless/realtek/rtw88/main.c
index a0d4d6e..3825211 100644
--- a/drivers/net/wireless/realtek/rtw88/main.c
+++ b/drivers/net/wireless/realtek/rtw88/main.c
@@ -17,6 +17,7 @@
#include "tx.h"
#include "debug.h"
#include "bf.h"
+#include "sar.h"
bool rtw_disable_lps_deep_mode;
EXPORT_SYMBOL(rtw_disable_lps_deep_mode);
@@ -637,6 +638,19 @@ static void rtw_txq_ba_work(struct work_struct *work)
rtw_iterate_stas_atomic(rtwdev, rtw_txq_ba_iter, &data);
}
+void rtw_set_rx_freq_band(struct rtw_rx_pkt_stat *pkt_stat, u8 channel)
+{
+ if (IS_CH_2G_BAND(channel))
+ pkt_stat->band = NL80211_BAND_2GHZ;
+ else if (IS_CH_5G_BAND(channel))
+ pkt_stat->band = NL80211_BAND_5GHZ;
+ else
+ return;
+
+ pkt_stat->freq = ieee80211_channel_to_frequency(channel, pkt_stat->band);
+}
+EXPORT_SYMBOL(rtw_set_rx_freq_band);
+
void rtw_get_channel_params(struct cfg80211_chan_def *chandef,
struct rtw_channel_params *chan_params)
{
@@ -735,8 +749,28 @@ void rtw_set_channel(struct rtw_dev *rtwdev)
hal->current_band_width = bandwidth;
hal->current_channel = center_chan;
+ hal->current_primary_channel_index = primary_chan_idx;
hal->current_band_type = center_chan > 14 ? RTW_BAND_5G : RTW_BAND_2G;
+ switch (center_chan) {
+ case 1 ... 14:
+ hal->sar_band = RTW_SAR_BAND_0;
+ break;
+ case 36 ... 64:
+ hal->sar_band = RTW_SAR_BAND_1;
+ break;
+ case 100 ... 144:
+ hal->sar_band = RTW_SAR_BAND_3;
+ break;
+ case 149 ... 177:
+ hal->sar_band = RTW_SAR_BAND_4;
+ break;
+ default:
+ WARN(1, "unknown ch(%u) to SAR band\n", center_chan);
+ hal->sar_band = RTW_SAR_BAND_0;
+ break;
+ }
+
for (i = RTW_CHANNEL_WIDTH_20; i <= RTW_MAX_CHANNEL_WIDTH; i++)
hal->cch_by_bw[i] = ch_param.cch_by_bw[i];
@@ -1278,6 +1312,50 @@ void rtw_core_fw_scan_notify(struct rtw_dev *rtwdev, bool start)
}
}
+void rtw_core_scan_start(struct rtw_dev *rtwdev, struct rtw_vif *rtwvif,
+ const u8 *mac_addr, bool hw_scan)
+{
+ u32 config = 0;
+ int ret = 0;
+
+ rtw_leave_lps(rtwdev);
+
+ if (hw_scan && rtwvif->net_type == RTW_NET_NO_LINK) {
+ ret = rtw_leave_ips(rtwdev);
+ if (ret) {
+ rtw_err(rtwdev, "failed to leave idle state\n");
+ return;
+ }
+ }
+
+ ether_addr_copy(rtwvif->mac_addr, mac_addr);
+ config |= PORT_SET_MAC_ADDR;
+ rtw_vif_port_config(rtwdev, rtwvif, config);
+
+ rtw_coex_scan_notify(rtwdev, COEX_SCAN_START);
+ rtw_core_fw_scan_notify(rtwdev, true);
+
+ set_bit(RTW_FLAG_DIG_DISABLE, rtwdev->flags);
+ set_bit(RTW_FLAG_SCANNING, rtwdev->flags);
+}
+
+void rtw_core_scan_complete(struct rtw_dev *rtwdev, struct ieee80211_vif *vif)
+{
+ struct rtw_vif *rtwvif = (struct rtw_vif *)vif->drv_priv;
+ u32 config = 0;
+
+ clear_bit(RTW_FLAG_SCANNING, rtwdev->flags);
+ clear_bit(RTW_FLAG_DIG_DISABLE, rtwdev->flags);
+
+ rtw_core_fw_scan_notify(rtwdev, false);
+
+ ether_addr_copy(rtwvif->mac_addr, vif->addr);
+ config |= PORT_SET_MAC_ADDR;
+ rtw_vif_port_config(rtwdev, rtwvif, config);
+
+ rtw_coex_scan_notify(rtwdev, COEX_SCAN_FINISH);
+}
+
int rtw_core_start(struct rtw_dev *rtwdev)
{
int ret;
@@ -1862,13 +1940,14 @@ int rtw_core_init(struct rtw_dev *rtwdev)
rtwdev->sec.total_cam_num = 32;
rtwdev->hal.current_channel = 1;
+ rtwdev->dm_info.fix_rate = U8_MAX;
set_bit(RTW_BC_MC_MACID, rtwdev->mac_id_map);
rtw_stats_init(rtwdev);
/* default rx filter setting */
rtwdev->hal.rcr = BIT_APP_FCS | BIT_APP_MIC | BIT_APP_ICV |
- BIT_HTC_LOC_CTRL | BIT_APP_PHYSTS |
+ BIT_PKTCTL_DLEN | BIT_HTC_LOC_CTRL | BIT_APP_PHYSTS |
BIT_AB | BIT_AM | BIT_APM;
ret = rtw_load_firmware(rtwdev, RTW_NORMAL_FW);
@@ -1951,6 +2030,7 @@ int rtw_register_hw(struct rtw_dev *rtwdev, struct ieee80211_hw *hw)
ieee80211_hw_set(hw, SUPPORTS_AMSDU_IN_AMPDU);
ieee80211_hw_set(hw, HAS_RATE_CONTROL);
ieee80211_hw_set(hw, TX_AMSDU);
+ ieee80211_hw_set(hw, SINGLE_SCAN_ON_ALL_BANDS);
hw->wiphy->interface_modes = BIT(NL80211_IFTYPE_STATION) |
BIT(NL80211_IFTYPE_AP) |
@@ -1963,8 +2043,12 @@ int rtw_register_hw(struct rtw_dev *rtwdev, struct ieee80211_hw *hw)
WIPHY_FLAG_TDLS_EXTERNAL_SETUP;
hw->wiphy->features |= NL80211_FEATURE_SCAN_RANDOM_MAC_ADDR;
+ hw->wiphy->max_scan_ssids = RTW_SCAN_MAX_SSIDS;
+ hw->wiphy->max_scan_ie_len = RTW_SCAN_MAX_IE_LEN;
wiphy_ext_feature_set(hw->wiphy, NL80211_EXT_FEATURE_CAN_REPLACE_PTK0);
+ wiphy_ext_feature_set(hw->wiphy, NL80211_EXT_FEATURE_SCAN_RANDOM_SN);
+ wiphy_ext_feature_set(hw->wiphy, NL80211_EXT_FEATURE_SET_SCAN_DWELL);
#ifdef CONFIG_PM
hw->wiphy->wowlan = rtwdev->chip->wowlan_stub;
@@ -1973,6 +2057,8 @@ int rtw_register_hw(struct rtw_dev *rtwdev, struct ieee80211_hw *hw)
rtw_set_supported_band(hw, rtwdev->chip);
SET_IEEE80211_PERM_ADDR(hw, rtwdev->efuse.addr);
+ hw->wiphy->sar_capa = &rtw_sar_capa;
+
ret = rtw_regd_init(rtwdev);
if (ret) {
rtw_err(rtwdev, "failed to init regd\n");
diff --git a/drivers/net/wireless/realtek/rtw88/main.h b/drivers/net/wireless/realtek/rtw88/main.h
index bbdd535..dc1cd9b 100644
--- a/drivers/net/wireless/realtek/rtw88/main.h
+++ b/drivers/net/wireless/realtek/rtw88/main.h
@@ -22,6 +22,9 @@
#define RTW_MAX_SEC_CAM_NUM 32
#define MAX_PG_CAM_BACKUP_NUM 8
+#define RTW_SCAN_MAX_SSIDS 4
+#define RTW_SCAN_MAX_IE_LEN 128
+
#define RTW_MAX_PATTERN_NUM 12
#define RTW_MAX_PATTERN_MASK_SIZE 16
#define RTW_MAX_PATTERN_SIZE 128
@@ -81,11 +84,9 @@ struct rtw_hci {
IS_CH_5G_BAND_3(channel) || IS_CH_5G_BAND_4(channel))
enum rtw_supported_band {
- RTW_BAND_2G = 1 << 0,
- RTW_BAND_5G = 1 << 1,
- RTW_BAND_60G = 1 << 2,
-
- RTW_BAND_MAX,
+ RTW_BAND_2G = BIT(NL80211_BAND_2GHZ),
+ RTW_BAND_5G = BIT(NL80211_BAND_5GHZ),
+ RTW_BAND_60G = BIT(NL80211_BAND_60GHZ),
};
/* now, support upto 80M bw */
@@ -364,6 +365,7 @@ enum rtw_flags {
RTW_FLAG_WOWLAN,
RTW_FLAG_RESTARTING,
RTW_FLAG_RESTART_TRIGGERING,
+ RTW_FLAG_FORCE_LOWEST_RATE,
NUM_OF_RTW_FLAGS,
};
@@ -629,6 +631,8 @@ struct rtw_rx_pkt_stat {
s8 rx_snr[RTW_RF_PATH_MAX];
u8 rx_evm[RTW_RF_PATH_MAX];
s8 cfo_tail[RTW_RF_PATH_MAX];
+ u16 freq;
+ u8 band;
struct rtw_sta_info *si;
struct ieee80211_vif *vif;
@@ -799,6 +803,8 @@ struct rtw_vif {
struct list_head rsvd_page_list;
struct ieee80211_tx_queue_params tx_params[IEEE80211_NUM_ACS];
const struct rtw_vif_port *conf;
+ struct cfg80211_scan_request *scan_req;
+ struct ieee80211_scan_ies *scan_ies;
struct rtw_traffic_stats stats;
@@ -1630,6 +1636,7 @@ struct rtw_dm_info {
u8 cck_gi_u_bnd;
u8 cck_gi_l_bnd;
+ u8 fix_rate;
u8 tx_rate;
u32 rrsr_val_init;
u32 rrsr_mask_min;
@@ -1806,6 +1813,33 @@ struct rtw_fw_state {
u32 feature;
};
+enum rtw_sar_sources {
+ RTW_SAR_SOURCE_NONE,
+ RTW_SAR_SOURCE_COMMON,
+};
+
+enum rtw_sar_bands {
+ RTW_SAR_BAND_0,
+ RTW_SAR_BAND_1,
+ /* RTW_SAR_BAND_2, not used now */
+ RTW_SAR_BAND_3,
+ RTW_SAR_BAND_4,
+
+ RTW_SAR_BAND_NR,
+};
+
+/* the union is reserved for other knids of SAR sources
+ * which might not re-use same format with array common.
+ */
+union rtw_sar_cfg {
+ s8 common[RTW_SAR_BAND_NR];
+};
+
+struct rtw_sar {
+ enum rtw_sar_sources src;
+ union rtw_sar_cfg cfg[RTW_RF_PATH_MAX][RTW_RATE_SECTION_MAX];
+};
+
struct rtw_hal {
u32 rcr;
@@ -1817,6 +1851,7 @@ struct rtw_hal {
u8 ps_mode;
u8 current_channel;
+ u8 current_primary_channel_index;
u8 current_band_width;
u8 current_band_type;
@@ -1853,6 +1888,9 @@ struct rtw_hal {
[RTW_MAX_CHANNEL_NUM_5G];
s8 tx_pwr_tbl[RTW_RF_PATH_MAX]
[DESC_RATE_MAX];
+
+ enum rtw_sar_bands sar_band;
+ struct rtw_sar sar;
};
struct rtw_path_div {
@@ -1863,12 +1901,37 @@ struct rtw_path_div {
u16 path_b_cnt;
};
+struct rtw_chan_info {
+ int pri_ch_idx;
+ int action_id;
+ int bw;
+ u8 extra_info;
+ u8 channel;
+ u16 timeout;
+};
+
+struct rtw_chan_list {
+ u32 buf_size;
+ u32 ch_num;
+ u32 size;
+ u16 addr;
+};
+
+struct rtw_hw_scan_info {
+ struct ieee80211_vif *scanning_vif;
+ u8 probe_pg_size;
+ u8 op_pri_ch_idx;
+ u8 op_chan;
+ u8 op_bw;
+};
+
struct rtw_dev {
struct ieee80211_hw *hw;
struct device *dev;
struct rtw_hci hci;
+ struct rtw_hw_scan_info scan_info;
struct rtw_chip_info *chip;
struct rtw_hal hal;
struct rtw_fifo_conf fifo;
@@ -2021,6 +2084,7 @@ static inline int rtw_chip_dump_fw_crash(struct rtw_dev *rtwdev)
return 0;
}
+void rtw_set_rx_freq_band(struct rtw_rx_pkt_stat *pkt_stat, u8 channel);
void rtw_get_channel_params(struct cfg80211_chan_def *chandef,
struct rtw_channel_params *ch_param);
bool check_hw_ready(struct rtw_dev *rtwdev, u32 addr, u32 mask, u32 target);
@@ -2035,6 +2099,9 @@ void rtw_vif_port_config(struct rtw_dev *rtwdev, struct rtw_vif *rtwvif,
u32 config);
void rtw_tx_report_purge_timer(struct timer_list *t);
void rtw_update_sta_info(struct rtw_dev *rtwdev, struct rtw_sta_info *si);
+void rtw_core_scan_start(struct rtw_dev *rtwdev, struct rtw_vif *rtwvif,
+ const u8 *mac_addr, bool hw_scan);
+void rtw_core_scan_complete(struct rtw_dev *rtwdev, struct ieee80211_vif *vif);
int rtw_core_start(struct rtw_dev *rtwdev);
void rtw_core_stop(struct rtw_dev *rtwdev);
int rtw_chip_info_setup(struct rtw_dev *rtwdev);
diff --git a/drivers/net/wireless/realtek/rtw88/pci.c b/drivers/net/wireless/realtek/rtw88/pci.c
index a7a6ebf..a0991d3 100644
--- a/drivers/net/wireless/realtek/rtw88/pci.c
+++ b/drivers/net/wireless/realtek/rtw88/pci.c
@@ -2,7 +2,6 @@
/* Copyright(c) 2018-2019 Realtek Corporation
*/
-#include <linux/dmi.h>
#include <linux/module.h>
#include <linux/pci.h>
#include "main.h"
@@ -612,6 +611,9 @@ static void rtw_pci_deep_ps_enter(struct rtw_dev *rtwdev)
bool tx_empty = true;
u8 queue;
+ if (rtw_fw_feature_check(&rtwdev->fw, FW_FEATURE_TX_WAKE))
+ goto enter_deep_ps;
+
lockdep_assert_held(&rtwpci->irq_lock);
/* Deep PS state is not allowed to TX-DMA */
@@ -637,7 +639,7 @@ static void rtw_pci_deep_ps_enter(struct rtw_dev *rtwdev)
"TX path not empty, cannot enter deep power save state\n");
return;
}
-
+enter_deep_ps:
set_bit(RTW_FLAG_LEISURE_PS_DEEP, rtwdev->flags);
rtw_power_mode_change(rtwdev, true);
}
@@ -808,7 +810,8 @@ static void rtw_pci_tx_kick_off_queue(struct rtw_dev *rtwdev, u8 queue)
bd_idx = rtw_pci_tx_queue_idx_addr[queue];
spin_lock_bh(&rtwpci->irq_lock);
- rtw_pci_deep_ps_leave(rtwdev);
+ if (!rtw_fw_feature_check(&rtwdev->fw, FW_FEATURE_TX_WAKE))
+ rtw_pci_deep_ps_leave(rtwdev);
rtw_write16(rtwdev, bd_idx, ring->r.wp & TRX_BD_IDX_MASK);
spin_unlock_bh(&rtwpci->irq_lock);
}
@@ -1409,7 +1412,11 @@ static void rtw_pci_link_ps(struct rtw_dev *rtwdev, bool enter)
* throughput. This is probably because the ASPM behavior slightly
* varies from different SOC.
*/
- if (rtwpci->link_ctrl & PCI_EXP_LNKCTL_ASPM_L1)
+ if (!(rtwpci->link_ctrl & PCI_EXP_LNKCTL_ASPM_L1))
+ return;
+
+ if ((enter && atomic_dec_if_positive(&rtwpci->link_usage) == 0) ||
+ (!enter && atomic_inc_return(&rtwpci->link_usage) == 1))
rtw_pci_aspm_set(rtwdev, enter);
}
@@ -1658,6 +1665,9 @@ static int rtw_pci_napi_poll(struct napi_struct *napi, int budget)
priv);
int work_done = 0;
+ if (rtwpci->rx_no_aspm)
+ rtw_pci_link_ps(rtwdev, false);
+
while (work_done < budget) {
u32 work_done_once;
@@ -1681,6 +1691,8 @@ static int rtw_pci_napi_poll(struct napi_struct *napi, int budget)
if (rtw_pci_get_hw_rx_ring_nr(rtwdev, rtwpci))
napi_schedule(napi);
}
+ if (rtwpci->rx_no_aspm)
+ rtw_pci_link_ps(rtwdev, true);
return work_done;
}
@@ -1702,50 +1714,13 @@ static void rtw_pci_napi_deinit(struct rtw_dev *rtwdev)
netif_napi_del(&rtwpci->napi);
}
-enum rtw88_quirk_dis_pci_caps {
- QUIRK_DIS_PCI_CAP_MSI,
- QUIRK_DIS_PCI_CAP_ASPM,
-};
-
-static int disable_pci_caps(const struct dmi_system_id *dmi)
-{
- uintptr_t dis_caps = (uintptr_t)dmi->driver_data;
-
- if (dis_caps & BIT(QUIRK_DIS_PCI_CAP_MSI))
- rtw_disable_msi = true;
- if (dis_caps & BIT(QUIRK_DIS_PCI_CAP_ASPM))
- rtw_pci_disable_aspm = true;
-
- return 1;
-}
-
-static const struct dmi_system_id rtw88_pci_quirks[] = {
- {
- .callback = disable_pci_caps,
- .ident = "Protempo Ltd L116HTN6SPW",
- .matches = {
- DMI_MATCH(DMI_SYS_VENDOR, "Protempo Ltd"),
- DMI_MATCH(DMI_PRODUCT_NAME, "L116HTN6SPW"),
- },
- .driver_data = (void *)BIT(QUIRK_DIS_PCI_CAP_ASPM),
- },
- {
- .callback = disable_pci_caps,
- .ident = "HP HP Pavilion Laptop 14-ce0xxx",
- .matches = {
- DMI_MATCH(DMI_SYS_VENDOR, "HP"),
- DMI_MATCH(DMI_PRODUCT_NAME, "HP Pavilion Laptop 14-ce0xxx"),
- },
- .driver_data = (void *)BIT(QUIRK_DIS_PCI_CAP_ASPM),
- },
- {}
-};
-
int rtw_pci_probe(struct pci_dev *pdev,
const struct pci_device_id *id)
{
+ struct pci_dev *bridge = pci_upstream_bridge(pdev);
struct ieee80211_hw *hw;
struct rtw_dev *rtwdev;
+ struct rtw_pci *rtwpci;
int drv_data_size;
int ret;
@@ -1763,6 +1738,9 @@ int rtw_pci_probe(struct pci_dev *pdev,
rtwdev->hci.ops = &rtw_pci_ops;
rtwdev->hci.type = RTW_HCI_TYPE_PCIE;
+ rtwpci = (struct rtw_pci *)rtwdev->priv;
+ atomic_set(&rtwpci->link_usage, 1);
+
ret = rtw_core_init(rtwdev);
if (ret)
goto err_release_hw;
@@ -1791,7 +1769,10 @@ int rtw_pci_probe(struct pci_dev *pdev,
goto err_destroy_pci;
}
- dmi_check_system(rtw88_pci_quirks);
+ /* Disable PCIe ASPM L1 while doing NAPI poll for 8821CE */
+ if (pdev->device == 0xc821 && bridge->vendor == PCI_VENDOR_ID_INTEL)
+ rtwpci->rx_no_aspm = true;
+
rtw_pci_phy_cfg(rtwdev);
ret = rtw_register_hw(rtwdev, hw);
diff --git a/drivers/net/wireless/realtek/rtw88/pci.h b/drivers/net/wireless/realtek/rtw88/pci.h
index 66f78eb..0c37efd 100644
--- a/drivers/net/wireless/realtek/rtw88/pci.h
+++ b/drivers/net/wireless/realtek/rtw88/pci.h
@@ -223,6 +223,8 @@ struct rtw_pci {
struct rtw_pci_tx_ring tx_rings[RTK_MAX_TX_QUEUE_NUM];
struct rtw_pci_rx_ring rx_rings[RTK_MAX_RX_QUEUE_NUM];
u16 link_ctrl;
+ atomic_t link_usage;
+ bool rx_no_aspm;
DECLARE_BITMAP(flags, NUM_OF_RTW_PCI_FLAGS);
void __iomem *mmap;
diff --git a/drivers/net/wireless/realtek/rtw88/phy.c b/drivers/net/wireless/realtek/rtw88/phy.c
index bfddfcb..e505d17 100644
--- a/drivers/net/wireless/realtek/rtw88/phy.c
+++ b/drivers/net/wireless/realtek/rtw88/phy.c
@@ -10,6 +10,7 @@
#include "phy.h"
#include "debug.h"
#include "regd.h"
+#include "sar.h"
struct phy_cfg_pair {
u32 addr;
@@ -2004,6 +2005,25 @@ static u8 rtw_phy_get_5g_tx_power_index(struct rtw_dev *rtwdev,
return tx_power;
}
+/* return RTW_RATE_SECTION_MAX to indicate rate is invalid */
+static u8 rtw_phy_rate_to_rate_section(u8 rate)
+{
+ if (rate >= DESC_RATE1M && rate <= DESC_RATE11M)
+ return RTW_RATE_SECTION_CCK;
+ else if (rate >= DESC_RATE6M && rate <= DESC_RATE54M)
+ return RTW_RATE_SECTION_OFDM;
+ else if (rate >= DESC_RATEMCS0 && rate <= DESC_RATEMCS7)
+ return RTW_RATE_SECTION_HT_1S;
+ else if (rate >= DESC_RATEMCS8 && rate <= DESC_RATEMCS15)
+ return RTW_RATE_SECTION_HT_2S;
+ else if (rate >= DESC_RATEVHT1SS_MCS0 && rate <= DESC_RATEVHT1SS_MCS9)
+ return RTW_RATE_SECTION_VHT_1S;
+ else if (rate >= DESC_RATEVHT2SS_MCS0 && rate <= DESC_RATEVHT2SS_MCS9)
+ return RTW_RATE_SECTION_VHT_2S;
+ else
+ return RTW_RATE_SECTION_MAX;
+}
+
static s8 rtw_phy_get_tx_power_limit(struct rtw_dev *rtwdev, u8 band,
enum rtw_bandwidth bw, u8 rf_path,
u8 rate, u8 channel, u8 regd)
@@ -2011,7 +2031,7 @@ static s8 rtw_phy_get_tx_power_limit(struct rtw_dev *rtwdev, u8 band,
struct rtw_hal *hal = &rtwdev->hal;
u8 *cch_by_bw = hal->cch_by_bw;
s8 power_limit = (s8)rtwdev->chip->max_power_index;
- u8 rs;
+ u8 rs = rtw_phy_rate_to_rate_section(rate);
int ch_idx;
u8 cur_bw, cur_ch;
s8 cur_lmt;
@@ -2019,19 +2039,7 @@ static s8 rtw_phy_get_tx_power_limit(struct rtw_dev *rtwdev, u8 band,
if (regd > RTW_REGD_WW)
return power_limit;
- if (rate >= DESC_RATE1M && rate <= DESC_RATE11M)
- rs = RTW_RATE_SECTION_CCK;
- else if (rate >= DESC_RATE6M && rate <= DESC_RATE54M)
- rs = RTW_RATE_SECTION_OFDM;
- else if (rate >= DESC_RATEMCS0 && rate <= DESC_RATEMCS7)
- rs = RTW_RATE_SECTION_HT_1S;
- else if (rate >= DESC_RATEMCS8 && rate <= DESC_RATEMCS15)
- rs = RTW_RATE_SECTION_HT_2S;
- else if (rate >= DESC_RATEVHT1SS_MCS0 && rate <= DESC_RATEVHT1SS_MCS9)
- rs = RTW_RATE_SECTION_VHT_1S;
- else if (rate >= DESC_RATEVHT2SS_MCS0 && rate <= DESC_RATEVHT2SS_MCS9)
- rs = RTW_RATE_SECTION_VHT_2S;
- else
+ if (rs == RTW_RATE_SECTION_MAX)
goto err;
/* only 20M BW with cck and ofdm */
@@ -2065,6 +2073,27 @@ static s8 rtw_phy_get_tx_power_limit(struct rtw_dev *rtwdev, u8 band,
return (s8)rtwdev->chip->max_power_index;
}
+static s8 rtw_phy_get_tx_power_sar(struct rtw_dev *rtwdev, u8 sar_band,
+ u8 rf_path, u8 rate)
+{
+ u8 rs = rtw_phy_rate_to_rate_section(rate);
+ struct rtw_sar_arg arg = {
+ .sar_band = sar_band,
+ .path = rf_path,
+ .rs = rs,
+ };
+
+ if (rs == RTW_RATE_SECTION_MAX)
+ goto err;
+
+ return rtw_query_sar(rtwdev, &arg);
+
+err:
+ WARN(1, "invalid arguments, sar_band=%d, path=%d, rate=%d\n",
+ sar_band, rf_path, rate);
+ return (s8)rtwdev->chip->max_power_index;
+}
+
void rtw_get_tx_power_params(struct rtw_dev *rtwdev, u8 path, u8 rate, u8 bw,
u8 ch, u8 regd, struct rtw_power_params *pwr_param)
{
@@ -2076,6 +2105,7 @@ void rtw_get_tx_power_params(struct rtw_dev *rtwdev, u8 path, u8 rate, u8 bw,
s8 *offset = &pwr_param->pwr_offset;
s8 *limit = &pwr_param->pwr_limit;
s8 *remnant = &pwr_param->pwr_remnant;
+ s8 *sar = &pwr_param->pwr_sar;
pwr_idx = &rtwdev->efuse.txpwr_idx_table[path];
group = rtw_get_channel_group(ch, rate);
@@ -2099,6 +2129,7 @@ void rtw_get_tx_power_params(struct rtw_dev *rtwdev, u8 path, u8 rate, u8 bw,
rate, ch, regd);
*remnant = (rate <= DESC_RATE11M ? dm_info->txagc_remnant_cck :
dm_info->txagc_remnant_ofdm);
+ *sar = rtw_phy_get_tx_power_sar(rtwdev, hal->sar_band, path, rate);
}
u8
@@ -2113,7 +2144,9 @@ rtw_phy_get_tx_power_index(struct rtw_dev *rtwdev, u8 rf_path, u8 rate,
channel, regd, &pwr_param);
tx_power = pwr_param.pwr_base;
- offset = min_t(s8, pwr_param.pwr_offset, pwr_param.pwr_limit);
+ offset = min3(pwr_param.pwr_offset,
+ pwr_param.pwr_limit,
+ pwr_param.pwr_sar);
if (rtwdev->chip->en_dis_dpd)
offset += rtw_phy_get_dis_dpd_by_rate_diff(rtwdev, rate);
diff --git a/drivers/net/wireless/realtek/rtw88/phy.h b/drivers/net/wireless/realtek/rtw88/phy.h
index 02d1ec4..b6c5ae6 100644
--- a/drivers/net/wireless/realtek/rtw88/phy.h
+++ b/drivers/net/wireless/realtek/rtw88/phy.h
@@ -148,6 +148,7 @@ struct rtw_power_params {
s8 pwr_offset;
s8 pwr_limit;
s8 pwr_remnant;
+ s8 pwr_sar;
};
void
diff --git a/drivers/net/wireless/realtek/rtw88/ps.c b/drivers/net/wireless/realtek/rtw88/ps.c
index 3f0ac33..bfa64c0 100644
--- a/drivers/net/wireless/realtek/rtw88/ps.c
+++ b/drivers/net/wireless/realtek/rtw88/ps.c
@@ -83,6 +83,9 @@ void rtw_power_mode_change(struct rtw_dev *rtwdev, bool enter)
/* Each request require an ack from firmware */
request |= POWER_MODE_ACK;
+ if (rtw_fw_feature_check(&rtwdev->fw, FW_FEATURE_TX_WAKE))
+ request |= POWER_TX_WAKE;
+
rtw_write8(rtwdev, rtwdev->hci.rpwm_addr, request);
/* Check firmware get the power requset and ack via cpwm register */
diff --git a/drivers/net/wireless/realtek/rtw88/ps.h b/drivers/net/wireless/realtek/rtw88/ps.h
index 7819391..c194386 100644
--- a/drivers/net/wireless/realtek/rtw88/ps.h
+++ b/drivers/net/wireless/realtek/rtw88/ps.h
@@ -9,6 +9,7 @@
#define POWER_MODE_ACK BIT(6)
#define POWER_MODE_PG BIT(4)
+#define POWER_TX_WAKE BIT(1)
#define POWER_MODE_LCLK BIT(0)
#define LEAVE_LPS_TRY_CNT 5
diff --git a/drivers/net/wireless/realtek/rtw88/rtw8821c.c b/drivers/net/wireless/realtek/rtw88/rtw8821c.c
index 80a6f4d..db078df 100644
--- a/drivers/net/wireless/realtek/rtw88/rtw8821c.c
+++ b/drivers/net/wireless/realtek/rtw88/rtw8821c.c
@@ -223,7 +223,8 @@ static int rtw8821c_mac_init(struct rtw_dev *rtwdev)
rtw_write8(rtwdev, REG_TCR + 1, WLAN_TX_FUNC_CFG1);
rtw_write8(rtwdev, REG_ACKTO_CCK, 0x40);
rtw_write8_set(rtwdev, REG_WMAC_TRXPTCL_CTL_H, BIT(1));
- rtw_write8_set(rtwdev, REG_SND_PTCL_CTRL, BIT(6));
+ rtw_write8_set(rtwdev, REG_SND_PTCL_CTRL,
+ BIT_DIS_CHK_VHTSIGB_CRC);
rtw_write32(rtwdev, REG_WMAC_OPTION_FUNCTION + 8, WLAN_MAC_OPT_FUNC2);
rtw_write8(rtwdev, REG_WMAC_OPTION_FUNCTION + 4, WLAN_MAC_OPT_NORM_FUNC1);
diff --git a/drivers/net/wireless/realtek/rtw88/rtw8821c.h b/drivers/net/wireless/realtek/rtw88/rtw8821c.h
index 112faa6..d9fbddd 100644
--- a/drivers/net/wireless/realtek/rtw88/rtw8821c.h
+++ b/drivers/net/wireless/realtek/rtw88/rtw8821c.h
@@ -131,7 +131,7 @@ _rtw_write32s_mask(struct rtw_dev *rtwdev, u32 addr, u32 mask, u32 data)
#define WLAN_TX_FUNC_CFG2 0x30
#define WLAN_MAC_OPT_NORM_FUNC1 0x98
#define WLAN_MAC_OPT_LB_FUNC1 0x80
-#define WLAN_MAC_OPT_FUNC2 0x30810041
+#define WLAN_MAC_OPT_FUNC2 0xb0810041
#define WLAN_SIFS_CFG (WLAN_SIFS_CCK_CONT_TX | \
(WLAN_SIFS_OFDM_CONT_TX << BIT_SHIFT_SIFS_OFDM_CTX) | \
diff --git a/drivers/net/wireless/realtek/rtw88/rtw8822b.c b/drivers/net/wireless/realtek/rtw88/rtw8822b.c
index c409c8c..dd4fbb8 100644
--- a/drivers/net/wireless/realtek/rtw88/rtw8822b.c
+++ b/drivers/net/wireless/realtek/rtw88/rtw8822b.c
@@ -205,7 +205,7 @@ static void rtw8822b_phy_set_param(struct rtw_dev *rtwdev)
#define WLAN_TX_FUNC_CFG2 0x30
#define WLAN_MAC_OPT_NORM_FUNC1 0x98
#define WLAN_MAC_OPT_LB_FUNC1 0x80
-#define WLAN_MAC_OPT_FUNC2 0x30810041
+#define WLAN_MAC_OPT_FUNC2 0xb0810041
#define WLAN_SIFS_CFG (WLAN_SIFS_CCK_CONT_TX | \
(WLAN_SIFS_OFDM_CONT_TX << BIT_SHIFT_SIFS_OFDM_CTX) | \
@@ -262,6 +262,8 @@ static int rtw8822b_mac_init(struct rtw_dev *rtwdev)
rtw_write8(rtwdev, REG_TCR + 1, WLAN_TX_FUNC_CFG1);
rtw_write32(rtwdev, REG_WMAC_OPTION_FUNCTION + 8, WLAN_MAC_OPT_FUNC2);
rtw_write8(rtwdev, REG_WMAC_OPTION_FUNCTION + 4, WLAN_MAC_OPT_NORM_FUNC1);
+ rtw_write8_set(rtwdev, REG_SND_PTCL_CTRL,
+ BIT_DIS_CHK_VHTSIGB_CRC);
return 0;
}
diff --git a/drivers/net/wireless/realtek/rtw88/rtw8822c.c b/drivers/net/wireless/realtek/rtw88/rtw8822c.c
index 46b881e..35c46e5 100644
--- a/drivers/net/wireless/realtek/rtw88/rtw8822c.c
+++ b/drivers/net/wireless/realtek/rtw88/rtw8822c.c
@@ -1962,7 +1962,7 @@ static void rtw8822c_phy_set_param(struct rtw_dev *rtwdev)
#define WLAN_TX_FUNC_CFG2 0x30
#define WLAN_MAC_OPT_NORM_FUNC1 0x98
#define WLAN_MAC_OPT_LB_FUNC1 0x80
-#define WLAN_MAC_OPT_FUNC2 0x30810041
+#define WLAN_MAC_OPT_FUNC2 0xb0810041
#define WLAN_MAC_INT_MIG_CFG 0x33330000
#define WLAN_SIFS_CFG (WLAN_SIFS_CCK_CONT_TX | \
@@ -2102,6 +2102,8 @@ static int rtw8822c_mac_init(struct rtw_dev *rtwdev)
BIT_RXPSF_CONT_ERRCHKEN);
value16 = BIT_SET_RXPSF_ERRTHR(value16, 0x07);
rtw_write16(rtwdev, REG_RXPSF_CTRL, value16);
+ rtw_write8_set(rtwdev, REG_SND_PTCL_CTRL,
+ BIT_DIS_CHK_VHTSIGB_CRC);
/* Interrupt migration configuration */
rtw_write32(rtwdev, REG_INT_MIG, WLAN_MAC_INT_MIG_CFG);
@@ -2533,6 +2535,7 @@ static void query_phy_status_page0(struct rtw_dev *rtwdev, u8 *phy_status,
s8 rx_power[RTW_RF_PATH_MAX];
s8 min_rx_power = -120;
u8 rssi;
+ u8 channel;
int path;
rx_power[RF_PATH_A] = GET_PHY_STAT_P0_PWDB_A(phy_status);
@@ -2553,6 +2556,11 @@ static void query_phy_status_page0(struct rtw_dev *rtwdev, u8 *phy_status,
rx_power[RF_PATH_A] -= 110;
rx_power[RF_PATH_B] -= 110;
+ channel = GET_PHY_STAT_P0_CHANNEL(phy_status);
+ if (channel == 0)
+ channel = rtwdev->hal.current_channel;
+ rtw_set_rx_freq_band(pkt_stat, channel);
+
pkt_stat->rx_power[RF_PATH_A] = rx_power[RF_PATH_A];
pkt_stat->rx_power[RF_PATH_B] = rx_power[RF_PATH_B];
@@ -2578,6 +2586,7 @@ static void query_phy_status_page1(struct rtw_dev *rtwdev, u8 *phy_status,
u8 evm_dbm = 0;
u8 rssi;
int path;
+ u8 channel;
if (pkt_stat->rate > DESC_RATE11M && pkt_stat->rate < DESC_RATEMCS0)
rxsc = GET_PHY_STAT_P1_L_RXSC(phy_status);
@@ -2591,6 +2600,9 @@ static void query_phy_status_page1(struct rtw_dev *rtwdev, u8 *phy_status,
else
bw = RTW_CHANNEL_WIDTH_20;
+ channel = GET_PHY_STAT_P1_CHANNEL(phy_status);
+ rtw_set_rx_freq_band(pkt_stat, channel);
+
pkt_stat->rx_power[RF_PATH_A] = GET_PHY_STAT_P1_PWDB_A(phy_status) - 110;
pkt_stat->rx_power[RF_PATH_B] = GET_PHY_STAT_P1_PWDB_B(phy_status) - 110;
pkt_stat->rssi = rtw_phy_rf_power_2_rssi(pkt_stat->rx_power, 2);
diff --git a/drivers/net/wireless/realtek/rtw88/rtw8822c.h b/drivers/net/wireless/realtek/rtw88/rtw8822c.h
index 3df62741..8201955 100644
--- a/drivers/net/wireless/realtek/rtw88/rtw8822c.h
+++ b/drivers/net/wireless/realtek/rtw88/rtw8822c.h
@@ -137,6 +137,8 @@ const struct rtw_table name ## _tbl = { \
le32_get_bits(*((__le32 *)(phy_stat) + 0x04), GENMASK(7, 0))
#define GET_PHY_STAT_P0_GAIN_A(phy_stat) \
le32_get_bits(*((__le32 *)(phy_stat) + 0x00), GENMASK(21, 16))
+#define GET_PHY_STAT_P0_CHANNEL(phy_stat) \
+ le32_get_bits(*((__le32 *)(phy_stat) + 0x01), GENMASK(23, 16))
#define GET_PHY_STAT_P0_GAIN_B(phy_stat) \
le32_get_bits(*((__le32 *)(phy_stat) + 0x04), GENMASK(29, 24))
@@ -149,6 +151,8 @@ const struct rtw_table name ## _tbl = { \
le32_get_bits(*((__le32 *)(phy_stat) + 0x01), GENMASK(11, 8))
#define GET_PHY_STAT_P1_HT_RXSC(phy_stat) \
le32_get_bits(*((__le32 *)(phy_stat) + 0x01), GENMASK(15, 12))
+#define GET_PHY_STAT_P1_CHANNEL(phy_stat) \
+ le32_get_bits(*((__le32 *)(phy_stat) + 0x01), GENMASK(23, 16))
#define GET_PHY_STAT_P1_RXEVM_A(phy_stat) \
le32_get_bits(*((__le32 *)(phy_stat) + 0x04), GENMASK(7, 0))
#define GET_PHY_STAT_P1_RXEVM_B(phy_stat) \
diff --git a/drivers/net/wireless/realtek/rtw88/rx.c b/drivers/net/wireless/realtek/rtw88/rx.c
index 7087e38..d2d607e 100644
--- a/drivers/net/wireless/realtek/rtw88/rx.c
+++ b/drivers/net/wireless/realtek/rtw88/rx.c
@@ -6,6 +6,7 @@
#include "rx.h"
#include "ps.h"
#include "debug.h"
+#include "fw.h"
void rtw_rx_stats(struct rtw_dev *rtwdev, struct ieee80211_vif *vif,
struct sk_buff *skb)
@@ -138,6 +139,13 @@ static void rtw_rx_addr_match(struct rtw_dev *rtwdev,
rtw_iterate_vifs_atomic(rtwdev, rtw_rx_addr_match_iter, &data);
}
+static void rtw_set_rx_freq_by_pktstat(struct rtw_rx_pkt_stat *pkt_stat,
+ struct ieee80211_rx_status *rx_status)
+{
+ rx_status->freq = pkt_stat->freq;
+ rx_status->band = pkt_stat->band;
+}
+
void rtw_rx_fill_rx_status(struct rtw_dev *rtwdev,
struct rtw_rx_pkt_stat *pkt_stat,
struct ieee80211_hdr *hdr,
@@ -150,6 +158,8 @@ void rtw_rx_fill_rx_status(struct rtw_dev *rtwdev,
memset(rx_status, 0, sizeof(*rx_status));
rx_status->freq = hw->conf.chandef.chan->center_freq;
rx_status->band = hw->conf.chandef.chan->band;
+ if (rtw_fw_feature_check(&rtwdev->fw, FW_FEATURE_SCAN_OFFLOAD))
+ rtw_set_rx_freq_by_pktstat(pkt_stat, rx_status);
if (pkt_stat->crc_err)
rx_status->flag |= RX_FLAG_FAILED_FCS_CRC;
if (pkt_stat->decrypted)
diff --git a/drivers/net/wireless/realtek/rtw88/sar.c b/drivers/net/wireless/realtek/rtw88/sar.c
new file mode 100644
index 0000000..3383726
--- /dev/null
+++ b/drivers/net/wireless/realtek/rtw88/sar.c
@@ -0,0 +1,114 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause
+/* Copyright(c) 2018-2021 Realtek Corporation
+ */
+
+#include "sar.h"
+#include "phy.h"
+#include "debug.h"
+
+s8 rtw_query_sar(struct rtw_dev *rtwdev, const struct rtw_sar_arg *arg)
+{
+ const struct rtw_hal *hal = &rtwdev->hal;
+ const struct rtw_sar *sar = &hal->sar;
+
+ switch (sar->src) {
+ default:
+ rtw_warn(rtwdev, "unknown SAR source: %d\n", sar->src);
+ fallthrough;
+ case RTW_SAR_SOURCE_NONE:
+ return (s8)rtwdev->chip->max_power_index;
+ case RTW_SAR_SOURCE_COMMON:
+ return sar->cfg[arg->path][arg->rs].common[arg->sar_band];
+ }
+}
+
+static int rtw_apply_sar(struct rtw_dev *rtwdev, const struct rtw_sar *new)
+{
+ struct rtw_hal *hal = &rtwdev->hal;
+ struct rtw_sar *sar = &hal->sar;
+
+ if (sar->src != RTW_SAR_SOURCE_NONE && new->src != sar->src) {
+ rtw_warn(rtwdev, "SAR source: %d is in use\n", sar->src);
+ return -EBUSY;
+ }
+
+ *sar = *new;
+ rtw_phy_set_tx_power_level(rtwdev, hal->current_channel);
+
+ return 0;
+}
+
+static s8 rtw_sar_to_phy(struct rtw_dev *rtwdev, u8 fct, s32 sar,
+ const struct rtw_sar_arg *arg)
+{
+ struct rtw_hal *hal = &rtwdev->hal;
+ u8 txgi = rtwdev->chip->txgi_factor;
+ u8 max = rtwdev->chip->max_power_index;
+ s32 tmp;
+ s8 base;
+
+ tmp = fct > txgi ? sar >> (fct - txgi) : sar << (txgi - fct);
+ base = arg->sar_band == RTW_SAR_BAND_0 ?
+ hal->tx_pwr_by_rate_base_2g[arg->path][arg->rs] :
+ hal->tx_pwr_by_rate_base_5g[arg->path][arg->rs];
+
+ return (s8)clamp_t(s32, tmp, -max - 1, max) - base;
+}
+
+static const struct cfg80211_sar_freq_ranges rtw_common_sar_freq_ranges[] = {
+ [RTW_SAR_BAND_0] = { .start_freq = 2412, .end_freq = 2484, },
+ [RTW_SAR_BAND_1] = { .start_freq = 5180, .end_freq = 5320, },
+ [RTW_SAR_BAND_3] = { .start_freq = 5500, .end_freq = 5720, },
+ [RTW_SAR_BAND_4] = { .start_freq = 5745, .end_freq = 5825, },
+};
+
+static_assert(ARRAY_SIZE(rtw_common_sar_freq_ranges) == RTW_SAR_BAND_NR);
+
+const struct cfg80211_sar_capa rtw_sar_capa = {
+ .type = NL80211_SAR_TYPE_POWER,
+ .num_freq_ranges = RTW_SAR_BAND_NR,
+ .freq_ranges = rtw_common_sar_freq_ranges,
+};
+
+int rtw_set_sar_specs(struct rtw_dev *rtwdev,
+ const struct cfg80211_sar_specs *sar)
+{
+ struct rtw_sar_arg arg = {0};
+ struct rtw_sar new = {0};
+ u32 idx, i, j, k;
+ s32 power;
+ s8 val;
+
+ if (sar->type != NL80211_SAR_TYPE_POWER)
+ return -EINVAL;
+
+ memset(&new, rtwdev->chip->max_power_index, sizeof(new));
+ new.src = RTW_SAR_SOURCE_COMMON;
+
+ for (i = 0; i < sar->num_sub_specs; i++) {
+ idx = sar->sub_specs[i].freq_range_index;
+ if (idx >= RTW_SAR_BAND_NR)
+ return -EINVAL;
+
+ power = sar->sub_specs[i].power;
+ rtw_info(rtwdev, "On freq %u to %u, set SAR %d in 1/%lu dBm\n",
+ rtw_common_sar_freq_ranges[idx].start_freq,
+ rtw_common_sar_freq_ranges[idx].end_freq,
+ power, BIT(RTW_COMMON_SAR_FCT));
+
+ for (j = 0; j < RTW_RF_PATH_MAX; j++) {
+ for (k = 0; k < RTW_RATE_SECTION_MAX; k++) {
+ arg = (struct rtw_sar_arg){
+ .sar_band = idx,
+ .path = j,
+ .rs = k,
+ };
+ val = rtw_sar_to_phy(rtwdev, RTW_COMMON_SAR_FCT,
+ power, &arg);
+ new.cfg[j][k].common[idx] = val;
+ }
+ }
+ }
+
+ return rtw_apply_sar(rtwdev, &new);
+}
diff --git a/drivers/net/wireless/realtek/rtw88/sar.h b/drivers/net/wireless/realtek/rtw88/sar.h
new file mode 100644
index 0000000..e01e7bb
--- /dev/null
+++ b/drivers/net/wireless/realtek/rtw88/sar.h
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause */
+/* Copyright(c) 2018-2021 Realtek Corporation
+ */
+
+#include "main.h"
+
+/* NL80211_SAR_TYPE_POWER means unit is in 0.25 dBm,
+ * where 0.25 = 1/4 = 2^(-2), so make factor 2.
+ */
+#define RTW_COMMON_SAR_FCT 2
+
+struct rtw_sar_arg {
+ u8 sar_band;
+ u8 path;
+ u8 rs;
+};
+
+extern const struct cfg80211_sar_capa rtw_sar_capa;
+
+s8 rtw_query_sar(struct rtw_dev *rtwdev, const struct rtw_sar_arg *arg);
+int rtw_set_sar_specs(struct rtw_dev *rtwdev,
+ const struct cfg80211_sar_specs *sar);
diff --git a/drivers/net/wireless/realtek/rtw88/tx.c b/drivers/net/wireless/realtek/rtw88/tx.c
index 3a101aa..efcc1b0 100644
--- a/drivers/net/wireless/realtek/rtw88/tx.c
+++ b/drivers/net/wireless/realtek/rtw88/tx.c
@@ -233,17 +233,34 @@ void rtw_tx_report_handle(struct rtw_dev *rtwdev, struct sk_buff *skb, int src)
spin_unlock_irqrestore(&tx_report->q_lock, flags);
}
+static u8 rtw_get_mgmt_rate(struct rtw_dev *rtwdev, struct sk_buff *skb,
+ u8 lowest_rate, bool ignore_rate)
+{
+ struct ieee80211_tx_info *tx_info = IEEE80211_SKB_CB(skb);
+ struct ieee80211_vif *vif = tx_info->control.vif;
+ bool force_lowest = test_bit(RTW_FLAG_FORCE_LOWEST_RATE, rtwdev->flags);
+
+ if (!vif || !vif->bss_conf.basic_rates || ignore_rate || force_lowest)
+ return lowest_rate;
+
+ return __ffs(vif->bss_conf.basic_rates) + lowest_rate;
+}
+
static void rtw_tx_pkt_info_update_rate(struct rtw_dev *rtwdev,
struct rtw_tx_pkt_info *pkt_info,
- struct sk_buff *skb)
+ struct sk_buff *skb,
+ bool ignore_rate)
{
if (rtwdev->hal.current_band_type == RTW_BAND_2G) {
pkt_info->rate_id = RTW_RATEID_B_20M;
- pkt_info->rate = DESC_RATE1M;
+ pkt_info->rate = rtw_get_mgmt_rate(rtwdev, skb, DESC_RATE1M,
+ ignore_rate);
} else {
pkt_info->rate_id = RTW_RATEID_G;
- pkt_info->rate = DESC_RATE6M;
+ pkt_info->rate = rtw_get_mgmt_rate(rtwdev, skb, DESC_RATE6M,
+ ignore_rate);
}
+
pkt_info->use_rate = true;
pkt_info->dis_rate_fallback = true;
}
@@ -280,7 +297,7 @@ static void rtw_tx_mgmt_pkt_info_update(struct rtw_dev *rtwdev,
struct ieee80211_sta *sta,
struct sk_buff *skb)
{
- rtw_tx_pkt_info_update_rate(rtwdev, pkt_info, skb);
+ rtw_tx_pkt_info_update_rate(rtwdev, pkt_info, skb, false);
pkt_info->dis_qselseq = true;
pkt_info->en_hwseq = true;
pkt_info->hw_ssn_sel = 0;
@@ -295,7 +312,9 @@ static void rtw_tx_data_pkt_info_update(struct rtw_dev *rtwdev,
struct ieee80211_hdr *hdr = (struct ieee80211_hdr *)skb->data;
struct ieee80211_tx_info *info = IEEE80211_SKB_CB(skb);
struct ieee80211_hw *hw = rtwdev->hw;
+ struct rtw_dm_info *dm_info = &rtwdev->dm_info;
struct rtw_sta_info *si;
+ u8 fix_rate;
u16 seq;
u8 ampdu_factor = 0;
u8 ampdu_density = 0;
@@ -347,6 +366,13 @@ static void rtw_tx_data_pkt_info_update(struct rtw_dev *rtwdev,
pkt_info->bw = bw;
pkt_info->stbc = stbc;
pkt_info->ldpc = ldpc;
+
+ fix_rate = dm_info->fix_rate;
+ if (fix_rate < DESC_RATE_MAX) {
+ pkt_info->rate = fix_rate;
+ pkt_info->dis_rate_fallback = true;
+ pkt_info->use_rate = true;
+ }
}
void rtw_tx_pkt_info_update(struct rtw_dev *rtwdev,
@@ -404,7 +430,7 @@ void rtw_tx_rsvd_page_pkt_info_update(struct rtw_dev *rtwdev,
if (type != RSVD_BEACON && type != RSVD_DUMMY)
pkt_info->qsel = TX_DESC_QSEL_MGMT;
- rtw_tx_pkt_info_update_rate(rtwdev, pkt_info, skb);
+ rtw_tx_pkt_info_update_rate(rtwdev, pkt_info, skb, true);
bmc = is_broadcast_ether_addr(hdr->addr1) ||
is_multicast_ether_addr(hdr->addr1);
diff --git a/drivers/net/wireless/realtek/rtw89/cam.c b/drivers/net/wireless/realtek/rtw89/cam.c
index ad7a815..bd34e4b 100644
--- a/drivers/net/wireless/realtek/rtw89/cam.c
+++ b/drivers/net/wireless/realtek/rtw89/cam.c
@@ -219,6 +219,7 @@ static int rtw89_cam_attach_sec_cam(struct rtw89_dev *rtwdev,
struct ieee80211_key_conf *key,
struct rtw89_sec_cam_entry *sec_cam)
{
+ struct rtw89_sta *rtwsta = sta_to_rtwsta_safe(sta);
struct rtw89_vif *rtwvif;
struct rtw89_addr_cam_entry *addr_cam;
u8 key_idx = 0;
@@ -243,7 +244,7 @@ static int rtw89_cam_attach_sec_cam(struct rtw89_dev *rtwdev,
addr_cam->sec_ent[key_idx] = sec_cam->sec_cam_idx;
addr_cam->sec_entries[key_idx] = sec_cam;
set_bit(key_idx, addr_cam->sec_cam_map);
- ret = rtw89_fw_h2c_cam(rtwdev, rtwvif);
+ ret = rtw89_fw_h2c_cam(rtwdev, rtwvif, rtwsta, NULL);
if (ret) {
rtw89_err(rtwdev, "failed to update addr cam sec entry: %d\n",
ret);
@@ -371,6 +372,7 @@ int rtw89_cam_sec_key_del(struct rtw89_dev *rtwdev,
struct ieee80211_key_conf *key,
bool inform_fw)
{
+ struct rtw89_sta *rtwsta = sta_to_rtwsta_safe(sta);
struct rtw89_cam_info *cam_info = &rtwdev->cam_info;
struct rtw89_vif *rtwvif;
struct rtw89_addr_cam_entry *addr_cam;
@@ -394,7 +396,7 @@ int rtw89_cam_sec_key_del(struct rtw89_dev *rtwdev,
clear_bit(key_idx, addr_cam->sec_cam_map);
addr_cam->sec_entries[key_idx] = NULL;
if (inform_fw) {
- ret = rtw89_fw_h2c_cam(rtwdev, rtwvif);
+ ret = rtw89_fw_h2c_cam(rtwdev, rtwvif, rtwsta, NULL);
if (ret)
rtw89_err(rtwdev, "failed to update cam del key: %d\n", ret);
}
@@ -536,12 +538,8 @@ static int rtw89_cam_init_bssid_cam(struct rtw89_dev *rtwdev,
void rtw89_cam_bssid_changed(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif)
{
- struct ieee80211_vif *vif = rtwvif_to_vif(rtwvif);
- struct rtw89_addr_cam_entry *addr_cam = &rtwvif->addr_cam;
struct rtw89_bssid_cam_entry *bssid_cam = &rtwvif->bssid_cam;
- if (vif->type == NL80211_IFTYPE_STATION)
- ether_addr_copy(addr_cam->tma, rtwvif->bssid);
ether_addr_copy(bssid_cam->bssid, rtwvif->bssid);
}
@@ -593,7 +591,7 @@ int rtw89_cam_fill_bssid_cam_info(struct rtw89_dev *rtwdev,
return 0;
}
-static u8 rtw89_cam_addr_hash(u8 start, u8 *addr)
+static u8 rtw89_cam_addr_hash(u8 start, const u8 *addr)
{
u8 hash = 0;
u8 i;
@@ -606,15 +604,18 @@ static u8 rtw89_cam_addr_hash(u8 start, u8 *addr)
void rtw89_cam_fill_addr_cam_info(struct rtw89_dev *rtwdev,
struct rtw89_vif *rtwvif,
+ struct rtw89_sta *rtwsta,
+ const u8 *scan_mac_addr,
u8 *cmd)
{
struct ieee80211_vif *vif = rtwvif_to_vif(rtwvif);
- struct ieee80211_sta *sta;
- struct rtw89_sta *rtwsta;
struct rtw89_addr_cam_entry *addr_cam = &rtwvif->addr_cam;
+ struct ieee80211_sta *sta = rtwsta_to_sta_safe(rtwsta);
+ const u8 *sma = scan_mac_addr ? scan_mac_addr : rtwvif->mac_addr;
u8 sma_hash, tma_hash, addr_msk_start;
u8 sma_start = 0;
u8 tma_start = 0;
+ u8 *tma = sta ? sta->addr : rtwvif->bssid;
if (addr_cam->addr_mask != 0) {
addr_msk_start = __ffs(addr_cam->addr_mask);
@@ -623,8 +624,8 @@ void rtw89_cam_fill_addr_cam_info(struct rtw89_dev *rtwdev,
else if (addr_cam->mask_sel == RTW89_TMA)
tma_start = addr_msk_start;
}
- sma_hash = rtw89_cam_addr_hash(sma_start, rtwvif->mac_addr);
- tma_hash = rtw89_cam_addr_hash(tma_start, addr_cam->tma);
+ sma_hash = rtw89_cam_addr_hash(sma_start, sma);
+ tma_hash = rtw89_cam_addr_hash(tma_start, tma);
FWCMD_SET_ADDR_IDX(cmd, addr_cam->addr_cam_idx);
FWCMD_SET_ADDR_OFFSET(cmd, addr_cam->offset);
@@ -642,19 +643,19 @@ void rtw89_cam_fill_addr_cam_info(struct rtw89_dev *rtwdev,
FWCMD_SET_ADDR_BSSID_CAM_IDX(cmd, addr_cam->bssid_cam_idx);
- FWCMD_SET_ADDR_SMA0(cmd, rtwvif->mac_addr[0]);
- FWCMD_SET_ADDR_SMA1(cmd, rtwvif->mac_addr[1]);
- FWCMD_SET_ADDR_SMA2(cmd, rtwvif->mac_addr[2]);
- FWCMD_SET_ADDR_SMA3(cmd, rtwvif->mac_addr[3]);
- FWCMD_SET_ADDR_SMA4(cmd, rtwvif->mac_addr[4]);
- FWCMD_SET_ADDR_SMA5(cmd, rtwvif->mac_addr[5]);
+ FWCMD_SET_ADDR_SMA0(cmd, sma[0]);
+ FWCMD_SET_ADDR_SMA1(cmd, sma[1]);
+ FWCMD_SET_ADDR_SMA2(cmd, sma[2]);
+ FWCMD_SET_ADDR_SMA3(cmd, sma[3]);
+ FWCMD_SET_ADDR_SMA4(cmd, sma[4]);
+ FWCMD_SET_ADDR_SMA5(cmd, sma[5]);
- FWCMD_SET_ADDR_TMA0(cmd, addr_cam->tma[0]);
- FWCMD_SET_ADDR_TMA1(cmd, addr_cam->tma[1]);
- FWCMD_SET_ADDR_TMA2(cmd, addr_cam->tma[2]);
- FWCMD_SET_ADDR_TMA3(cmd, addr_cam->tma[3]);
- FWCMD_SET_ADDR_TMA4(cmd, addr_cam->tma[4]);
- FWCMD_SET_ADDR_TMA5(cmd, addr_cam->tma[5]);
+ FWCMD_SET_ADDR_TMA0(cmd, tma[0]);
+ FWCMD_SET_ADDR_TMA1(cmd, tma[1]);
+ FWCMD_SET_ADDR_TMA2(cmd, tma[2]);
+ FWCMD_SET_ADDR_TMA3(cmd, tma[3]);
+ FWCMD_SET_ADDR_TMA4(cmd, tma[4]);
+ FWCMD_SET_ADDR_TMA5(cmd, tma[5]);
FWCMD_SET_ADDR_PORT_INT(cmd, rtwvif->port);
FWCMD_SET_ADDR_TSF_SYNC(cmd, rtwvif->port);
@@ -662,15 +663,11 @@ void rtw89_cam_fill_addr_cam_info(struct rtw89_dev *rtwdev,
FWCMD_SET_ADDR_LSIG_TXOP(cmd, rtwvif->lsig_txop);
FWCMD_SET_ADDR_TGT_IND(cmd, rtwvif->tgt_ind);
FWCMD_SET_ADDR_FRM_TGT_IND(cmd, rtwvif->frm_tgt_ind);
-
- if (vif->type == NL80211_IFTYPE_STATION) {
- sta = rtwvif->mgd.ap;
- if (sta) {
- rtwsta = (struct rtw89_sta *)sta->drv_priv;
- FWCMD_SET_ADDR_MACID(cmd, rtwsta->mac_id);
- FWCMD_SET_ADDR_AID12(cmd, vif->bss_conf.aid & 0xfff);
- }
- }
+ FWCMD_SET_ADDR_MACID(cmd, rtwsta ? rtwsta->mac_id : rtwvif->mac_id);
+ if (rtwvif->net_type == RTW89_NET_TYPE_INFRA)
+ FWCMD_SET_ADDR_AID12(cmd, vif->bss_conf.aid & 0xfff);
+ else if (rtwvif->net_type == RTW89_NET_TYPE_AP_MODE)
+ FWCMD_SET_ADDR_AID12(cmd, sta ? sta->aid & 0xfff : 0);
FWCMD_SET_ADDR_WOL_PATTERN(cmd, rtwvif->wowlan_pattern);
FWCMD_SET_ADDR_WOL_UC(cmd, rtwvif->wowlan_uc);
FWCMD_SET_ADDR_WOL_MAGIC(cmd, rtwvif->wowlan_magic);
diff --git a/drivers/net/wireless/realtek/rtw89/cam.h b/drivers/net/wireless/realtek/rtw89/cam.h
index 90a20a5..33a3ad5 100644
--- a/drivers/net/wireless/realtek/rtw89/cam.h
+++ b/drivers/net/wireless/realtek/rtw89/cam.h
@@ -9,145 +9,347 @@
#define RTW89_SEC_CAM_LEN 20
-#define FWCMD_SET_ADDR_IDX(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 1, value, GENMASK(7, 0))
-#define FWCMD_SET_ADDR_OFFSET(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 1, value, GENMASK(15, 8))
-#define FWCMD_SET_ADDR_LEN(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 1, value, GENMASK(23, 16))
-#define FWCMD_SET_ADDR_VALID(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 2, value, BIT(0))
-#define FWCMD_SET_ADDR_NET_TYPE(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 2, value, GENMASK(2, 1))
-#define FWCMD_SET_ADDR_BCN_HIT_COND(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 2, value, GENMASK(4, 3))
-#define FWCMD_SET_ADDR_HIT_RULE(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 2, value, GENMASK(6, 5))
-#define FWCMD_SET_ADDR_BB_SEL(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 2, value, BIT(7))
-#define FWCMD_SET_ADDR_ADDR_MASK(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 2, value, GENMASK(13, 8))
-#define FWCMD_SET_ADDR_MASK_SEL(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 2, value, GENMASK(15, 14))
-#define FWCMD_SET_ADDR_SMA_HASH(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 2, value, GENMASK(23, 16))
-#define FWCMD_SET_ADDR_TMA_HASH(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 2, value, GENMASK(31, 24))
-#define FWCMD_SET_ADDR_BSSID_CAM_IDX(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 3, value, GENMASK(5, 0))
-#define FWCMD_SET_ADDR_SMA0(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 4, value, GENMASK(7, 0))
-#define FWCMD_SET_ADDR_SMA1(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 4, value, GENMASK(15, 8))
-#define FWCMD_SET_ADDR_SMA2(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 4, value, GENMASK(23, 16))
-#define FWCMD_SET_ADDR_SMA3(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 4, value, GENMASK(31, 24))
-#define FWCMD_SET_ADDR_SMA4(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 5, value, GENMASK(7, 0))
-#define FWCMD_SET_ADDR_SMA5(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 5, value, GENMASK(15, 8))
-#define FWCMD_SET_ADDR_TMA0(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 5, value, GENMASK(23, 16))
-#define FWCMD_SET_ADDR_TMA1(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 5, value, GENMASK(31, 24))
-#define FWCMD_SET_ADDR_TMA2(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 6, value, GENMASK(7, 0))
-#define FWCMD_SET_ADDR_TMA3(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 6, value, GENMASK(15, 8))
-#define FWCMD_SET_ADDR_TMA4(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 6, value, GENMASK(23, 16))
-#define FWCMD_SET_ADDR_TMA5(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 6, value, GENMASK(31, 24))
-#define FWCMD_SET_ADDR_MACID(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 8, value, GENMASK(7, 0))
-#define FWCMD_SET_ADDR_PORT_INT(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 8, value, GENMASK(10, 8))
-#define FWCMD_SET_ADDR_TSF_SYNC(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 8, value, GENMASK(13, 11))
-#define FWCMD_SET_ADDR_TF_TRS(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 8, value, BIT(14))
-#define FWCMD_SET_ADDR_LSIG_TXOP(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 8, value, BIT(15))
-#define FWCMD_SET_ADDR_TGT_IND(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 8, value, GENMASK(26, 24))
-#define FWCMD_SET_ADDR_FRM_TGT_IND(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 8, value, GENMASK(29, 27))
-#define FWCMD_SET_ADDR_AID12(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 9, value, GENMASK(11, 0))
-#define FWCMD_SET_ADDR_AID12_0(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 9, value, GENMASK(7, 0))
-#define FWCMD_SET_ADDR_AID12_1(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 9, value, GENMASK(11, 8))
-#define FWCMD_SET_ADDR_WOL_PATTERN(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 9, value, BIT(12))
-#define FWCMD_SET_ADDR_WOL_UC(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 9, value, BIT(13))
-#define FWCMD_SET_ADDR_WOL_MAGIC(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 9, value, BIT(14))
-#define FWCMD_SET_ADDR_WAPI(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 9, value, BIT(15))
-#define FWCMD_SET_ADDR_SEC_ENT_MODE(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 9, value, GENMASK(17, 16))
-#define FWCMD_SET_ADDR_SEC_ENT0_KEYID(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 9, value, GENMASK(19, 18))
-#define FWCMD_SET_ADDR_SEC_ENT1_KEYID(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 9, value, GENMASK(21, 20))
-#define FWCMD_SET_ADDR_SEC_ENT2_KEYID(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 9, value, GENMASK(23, 22))
-#define FWCMD_SET_ADDR_SEC_ENT3_KEYID(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 9, value, GENMASK(25, 24))
-#define FWCMD_SET_ADDR_SEC_ENT4_KEYID(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 9, value, GENMASK(27, 26))
-#define FWCMD_SET_ADDR_SEC_ENT5_KEYID(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 9, value, GENMASK(29, 28))
-#define FWCMD_SET_ADDR_SEC_ENT6_KEYID(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 9, value, GENMASK(31, 30))
-#define FWCMD_SET_ADDR_SEC_ENT_VALID(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 10, value, GENMASK(7, 0))
-#define FWCMD_SET_ADDR_SEC_ENT0(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 10, value, GENMASK(15, 8))
-#define FWCMD_SET_ADDR_SEC_ENT1(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 10, value, GENMASK(23, 16))
-#define FWCMD_SET_ADDR_SEC_ENT2(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 10, value, GENMASK(31, 24))
-#define FWCMD_SET_ADDR_SEC_ENT3(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 11, value, GENMASK(7, 0))
-#define FWCMD_SET_ADDR_SEC_ENT4(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 11, value, GENMASK(15, 8))
-#define FWCMD_SET_ADDR_SEC_ENT5(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 11, value, GENMASK(23, 16))
-#define FWCMD_SET_ADDR_SEC_ENT6(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 11, value, GENMASK(31, 24))
-#define FWCMD_SET_ADDR_BSSID_IDX(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 12, value, GENMASK(7, 0))
-#define FWCMD_SET_ADDR_BSSID_OFFSET(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 12, value, GENMASK(15, 8))
-#define FWCMD_SET_ADDR_BSSID_LEN(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 12, value, GENMASK(23, 16))
-#define FWCMD_SET_ADDR_BSSID_VALID(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 13, value, BIT(0))
-#define FWCMD_SET_ADDR_BSSID_BB_SEL(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 13, value, BIT(1))
-#define FWCMD_SET_ADDR_BSSID_BSS_COLOR(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 13, value, GENMASK(13, 8))
-#define FWCMD_SET_ADDR_BSSID_BSSID0(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 13, value, GENMASK(23, 16))
-#define FWCMD_SET_ADDR_BSSID_BSSID1(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 13, value, GENMASK(31, 24))
-#define FWCMD_SET_ADDR_BSSID_BSSID2(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 14, value, GENMASK(7, 0))
-#define FWCMD_SET_ADDR_BSSID_BSSID3(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 14, value, GENMASK(15, 8))
-#define FWCMD_SET_ADDR_BSSID_BSSID4(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 14, value, GENMASK(23, 16))
-#define FWCMD_SET_ADDR_BSSID_BSSID5(cmd, value) \
- le32p_replace_bits((__le32 *)(cmd) + 14, value, GENMASK(31, 24))
+static inline void FWCMD_SET_ADDR_IDX(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 1, value, GENMASK(7, 0));
+}
+
+static inline void FWCMD_SET_ADDR_OFFSET(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 1, value, GENMASK(15, 8));
+}
+
+static inline void FWCMD_SET_ADDR_LEN(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 1, value, GENMASK(23, 16));
+}
+
+static inline void FWCMD_SET_ADDR_VALID(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 2, value, BIT(0));
+}
+
+static inline void FWCMD_SET_ADDR_NET_TYPE(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 2, value, GENMASK(2, 1));
+}
+
+static inline void FWCMD_SET_ADDR_BCN_HIT_COND(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 2, value, GENMASK(4, 3));
+}
+
+static inline void FWCMD_SET_ADDR_HIT_RULE(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 2, value, GENMASK(6, 5));
+}
+
+static inline void FWCMD_SET_ADDR_BB_SEL(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 2, value, BIT(7));
+}
+
+static inline void FWCMD_SET_ADDR_ADDR_MASK(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 2, value, GENMASK(13, 8));
+}
+
+static inline void FWCMD_SET_ADDR_MASK_SEL(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 2, value, GENMASK(15, 14));
+}
+
+static inline void FWCMD_SET_ADDR_SMA_HASH(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 2, value, GENMASK(23, 16));
+}
+
+static inline void FWCMD_SET_ADDR_TMA_HASH(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 2, value, GENMASK(31, 24));
+}
+
+static inline void FWCMD_SET_ADDR_BSSID_CAM_IDX(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 3, value, GENMASK(5, 0));
+}
+
+static inline void FWCMD_SET_ADDR_SMA0(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 4, value, GENMASK(7, 0));
+}
+
+static inline void FWCMD_SET_ADDR_SMA1(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 4, value, GENMASK(15, 8));
+}
+
+static inline void FWCMD_SET_ADDR_SMA2(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 4, value, GENMASK(23, 16));
+}
+
+static inline void FWCMD_SET_ADDR_SMA3(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 4, value, GENMASK(31, 24));
+}
+
+static inline void FWCMD_SET_ADDR_SMA4(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 5, value, GENMASK(7, 0));
+}
+
+static inline void FWCMD_SET_ADDR_SMA5(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 5, value, GENMASK(15, 8));
+}
+
+static inline void FWCMD_SET_ADDR_TMA0(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 5, value, GENMASK(23, 16));
+}
+
+static inline void FWCMD_SET_ADDR_TMA1(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 5, value, GENMASK(31, 24));
+}
+
+static inline void FWCMD_SET_ADDR_TMA2(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 6, value, GENMASK(7, 0));
+}
+
+static inline void FWCMD_SET_ADDR_TMA3(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 6, value, GENMASK(15, 8));
+}
+
+static inline void FWCMD_SET_ADDR_TMA4(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 6, value, GENMASK(23, 16));
+}
+
+static inline void FWCMD_SET_ADDR_TMA5(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 6, value, GENMASK(31, 24));
+}
+
+static inline void FWCMD_SET_ADDR_MACID(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 8, value, GENMASK(7, 0));
+}
+
+static inline void FWCMD_SET_ADDR_PORT_INT(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 8, value, GENMASK(10, 8));
+}
+
+static inline void FWCMD_SET_ADDR_TSF_SYNC(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 8, value, GENMASK(13, 11));
+}
+
+static inline void FWCMD_SET_ADDR_TF_TRS(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 8, value, BIT(14));
+}
+
+static inline void FWCMD_SET_ADDR_LSIG_TXOP(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 8, value, BIT(15));
+}
+
+static inline void FWCMD_SET_ADDR_TGT_IND(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 8, value, GENMASK(26, 24));
+}
+
+static inline void FWCMD_SET_ADDR_FRM_TGT_IND(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 8, value, GENMASK(29, 27));
+}
+
+static inline void FWCMD_SET_ADDR_AID12(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 9, value, GENMASK(11, 0));
+}
+
+static inline void FWCMD_SET_ADDR_AID12_0(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 9, value, GENMASK(7, 0));
+}
+
+static inline void FWCMD_SET_ADDR_AID12_1(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 9, value, GENMASK(11, 8));
+}
+
+static inline void FWCMD_SET_ADDR_WOL_PATTERN(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 9, value, BIT(12));
+}
+
+static inline void FWCMD_SET_ADDR_WOL_UC(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 9, value, BIT(13));
+}
+
+static inline void FWCMD_SET_ADDR_WOL_MAGIC(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 9, value, BIT(14));
+}
+
+static inline void FWCMD_SET_ADDR_WAPI(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 9, value, BIT(15));
+}
+
+static inline void FWCMD_SET_ADDR_SEC_ENT_MODE(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 9, value, GENMASK(17, 16));
+}
+
+static inline void FWCMD_SET_ADDR_SEC_ENT0_KEYID(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 9, value, GENMASK(19, 18));
+}
+
+static inline void FWCMD_SET_ADDR_SEC_ENT1_KEYID(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 9, value, GENMASK(21, 20));
+}
+
+static inline void FWCMD_SET_ADDR_SEC_ENT2_KEYID(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 9, value, GENMASK(23, 22));
+}
+
+static inline void FWCMD_SET_ADDR_SEC_ENT3_KEYID(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 9, value, GENMASK(25, 24));
+}
+
+static inline void FWCMD_SET_ADDR_SEC_ENT4_KEYID(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 9, value, GENMASK(27, 26));
+}
+
+static inline void FWCMD_SET_ADDR_SEC_ENT5_KEYID(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 9, value, GENMASK(29, 28));
+}
+
+static inline void FWCMD_SET_ADDR_SEC_ENT6_KEYID(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 9, value, GENMASK(31, 30));
+}
+
+static inline void FWCMD_SET_ADDR_SEC_ENT_VALID(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 10, value, GENMASK(7, 0));
+}
+
+static inline void FWCMD_SET_ADDR_SEC_ENT0(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 10, value, GENMASK(15, 8));
+}
+
+static inline void FWCMD_SET_ADDR_SEC_ENT1(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 10, value, GENMASK(23, 16));
+}
+
+static inline void FWCMD_SET_ADDR_SEC_ENT2(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 10, value, GENMASK(31, 24));
+}
+
+static inline void FWCMD_SET_ADDR_SEC_ENT3(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 11, value, GENMASK(7, 0));
+}
+
+static inline void FWCMD_SET_ADDR_SEC_ENT4(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 11, value, GENMASK(15, 8));
+}
+
+static inline void FWCMD_SET_ADDR_SEC_ENT5(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 11, value, GENMASK(23, 16));
+}
+
+static inline void FWCMD_SET_ADDR_SEC_ENT6(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 11, value, GENMASK(31, 24));
+}
+
+static inline void FWCMD_SET_ADDR_BSSID_IDX(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 12, value, GENMASK(7, 0));
+}
+
+static inline void FWCMD_SET_ADDR_BSSID_OFFSET(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 12, value, GENMASK(15, 8));
+}
+
+static inline void FWCMD_SET_ADDR_BSSID_LEN(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 12, value, GENMASK(23, 16));
+}
+
+static inline void FWCMD_SET_ADDR_BSSID_VALID(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 13, value, BIT(0));
+}
+
+static inline void FWCMD_SET_ADDR_BSSID_BB_SEL(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 13, value, BIT(1));
+}
+
+static inline void FWCMD_SET_ADDR_BSSID_BSS_COLOR(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 13, value, GENMASK(13, 8));
+}
+
+static inline void FWCMD_SET_ADDR_BSSID_BSSID0(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 13, value, GENMASK(23, 16));
+}
+
+static inline void FWCMD_SET_ADDR_BSSID_BSSID1(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 13, value, GENMASK(31, 24));
+}
+
+static inline void FWCMD_SET_ADDR_BSSID_BSSID2(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 14, value, GENMASK(7, 0));
+}
+
+static inline void FWCMD_SET_ADDR_BSSID_BSSID3(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 14, value, GENMASK(15, 8));
+}
+
+static inline void FWCMD_SET_ADDR_BSSID_BSSID4(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 14, value, GENMASK(23, 16));
+}
+
+static inline void FWCMD_SET_ADDR_BSSID_BSSID5(void *cmd, u32 value)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 14, value, GENMASK(31, 24));
+}
int rtw89_cam_init(struct rtw89_dev *rtwdev, struct rtw89_vif *vif);
void rtw89_cam_deinit(struct rtw89_dev *rtwdev, struct rtw89_vif *vif);
void rtw89_cam_fill_addr_cam_info(struct rtw89_dev *rtwdev,
- struct rtw89_vif *vif, u8 *cmd);
+ struct rtw89_vif *vif,
+ struct rtw89_sta *rtwsta,
+ const u8 *scan_mac_addr, u8 *cmd);
int rtw89_cam_fill_bssid_cam_info(struct rtw89_dev *rtwdev,
struct rtw89_vif *vif, u8 *cmd);
int rtw89_cam_sec_key_add(struct rtw89_dev *rtwdev,
diff --git a/drivers/net/wireless/realtek/rtw89/coex.c b/drivers/net/wireless/realtek/rtw89/coex.c
index abe4b65..9f7d4f8 100644
--- a/drivers/net/wireless/realtek/rtw89/coex.c
+++ b/drivers/net/wireless/realtek/rtw89/coex.c
@@ -540,8 +540,31 @@ static void _update_bt_scbd(struct rtw89_dev *rtwdev, bool only_update);
static void _send_fw_cmd(struct rtw89_dev *rtwdev, u8 h2c_class, u8 h2c_func,
void *param, u16 len)
{
- rtw89_fw_h2c_raw_with_hdr(rtwdev, h2c_class, h2c_func, param, len,
- false, true);
+ struct rtw89_btc *btc = &rtwdev->btc;
+ struct rtw89_btc_btf_fwinfo *pfwinfo = &btc->fwinfo;
+ struct rtw89_btc_cx *cx = &btc->cx;
+ struct rtw89_btc_wl_info *wl = &cx->wl;
+ int ret;
+
+ if (!wl->status.map.init_ok) {
+ rtw89_debug(rtwdev, RTW89_DBG_BTC,
+ "[BTC], %s(): return by btc not init!!\n", __func__);
+ pfwinfo->cnt_h2c_fail++;
+ return;
+ } else if ((wl->status.map.rf_off_pre == 1 && wl->status.map.rf_off == 1) ||
+ (wl->status.map.lps_pre == 1 && wl->status.map.lps == 1)) {
+ rtw89_debug(rtwdev, RTW89_DBG_BTC,
+ "[BTC], %s(): return by wl off!!\n", __func__);
+ pfwinfo->cnt_h2c_fail++;
+ return;
+ }
+
+ pfwinfo->cnt_h2c++;
+
+ ret = rtw89_fw_h2c_raw_with_hdr(rtwdev, h2c_class, h2c_func, param, len,
+ false, true);
+ if (ret != 0)
+ pfwinfo->cnt_h2c_fail++;
}
static void _reset_btc_var(struct rtw89_dev *rtwdev, u8 type)
@@ -1095,6 +1118,10 @@ static u32 _chk_btc_report(struct rtw89_dev *rtwdev,
diff_t = pcysta->tavg_cycle[CXT_WL] - wl_slot_set;
_chk_btc_err(rtwdev, BTC_DCNT_WL_SLOT_DRIFT, diff_t);
}
+
+ _chk_btc_err(rtwdev, BTC_DCNT_W1_FREEZE, pcysta->slot_cnt[CXST_W1]);
+ _chk_btc_err(rtwdev, BTC_DCNT_W1_FREEZE, pcysta->slot_cnt[CXST_W1]);
+ _chk_btc_err(rtwdev, BTC_DCNT_CYCLE_FREEZE, (u32)pcysta->cycles);
}
if (rpt_type == BTC_RPT_TYPE_CTRL) {
@@ -1103,6 +1130,18 @@ static u32 _chk_btc_report(struct rtw89_dev *rtwdev,
wl->ver_info.fw_coex = prpt->wl_fw_coex_ver;
wl->ver_info.fw = prpt->wl_fw_ver;
dm->wl_fw_cx_offload = !!(prpt->wl_fw_cx_offload);
+
+ _chk_btc_err(rtwdev, BTC_DCNT_RPT_FREEZE,
+ pfwinfo->event[BTF_EVNT_RPT]);
+
+ /* To avoid I/O if WL LPS or power-off */
+ if (wl->status.map.lps != BTC_LPS_RF_OFF && !wl->status.map.rf_off) {
+ rtwdev->chip->ops->btc_update_bt_cnt(rtwdev);
+ _chk_btc_err(rtwdev, BTC_DCNT_BTCNT_FREEZE, 0);
+
+ btc->cx.cnt_bt[BTC_BCNT_POLUT] =
+ rtw89_mac_get_plt_cnt(rtwdev, RTW89_MAC_0);
+ }
}
if (rpt_type >= BTC_RPT_TYPE_BT_VER &&
@@ -1596,7 +1635,7 @@ static void _set_rf_trx_para(struct rtw89_dev *rtwdev)
_set_bt_rx_gain(rtwdev, para.bt_rx_gain);
if (bt->enable.now == 0 || wl->status.map.rf_off == 1 ||
- wl->status.map.lps == 1)
+ wl->status.map.lps == BTC_LPS_RF_OFF)
wl_stb_chg = 0;
else
wl_stb_chg = 1;
@@ -4199,16 +4238,16 @@ void rtw89_btc_ntfy_radio_state(struct rtw89_dev *rtwdev, enum btc_rfctrl rf_sta
switch (rf_state) {
case BTC_RFCTRL_WL_OFF:
wl->status.map.rf_off = 1;
- wl->status.map.lps = 0;
+ wl->status.map.lps = BTC_LPS_OFF;
break;
case BTC_RFCTRL_FW_CTRL:
wl->status.map.rf_off = 0;
- wl->status.map.lps = 1;
+ wl->status.map.lps = BTC_LPS_RF_OFF;
break;
case BTC_RFCTRL_WL_ON:
default:
wl->status.map.rf_off = 0;
- wl->status.map.lps = 0;
+ wl->status.map.lps = BTC_LPS_OFF;
break;
}
@@ -4494,6 +4533,8 @@ void rtw89_btc_c2h_handle(struct rtw89_dev *rtwdev, struct sk_buff *skb,
struct rtw89_btc_btf_fwinfo *pfwinfo = &btc->fwinfo;
u8 *buf = &skb->data[RTW89_C2H_HEADER_LEN];
+ len -= RTW89_C2H_HEADER_LEN;
+
rtw89_debug(rtwdev, RTW89_DBG_BTC,
"[BTC], %s(): C2H BT len:%d class:%d fun:%d\n",
__func__, len, class, func);
@@ -4512,14 +4553,12 @@ void rtw89_btc_c2h_handle(struct rtw89_dev *rtwdev, struct sk_buff *skb,
rtw89_debug(rtwdev, RTW89_DBG_BTC,
"[BTC], handle C2H BT INFO with data %8ph\n", buf);
btc->cx.cnt_bt[BTC_BCNT_INFOUPDATE]++;
- rtw89_leave_ps_mode(rtwdev);
_update_bt_info(rtwdev, buf, len);
break;
case BTF_EVNT_BT_SCBD:
rtw89_debug(rtwdev, RTW89_DBG_BTC,
"[BTC], handle C2H BT SCBD with data %8ph\n", buf);
btc->cx.cnt_bt[BTC_BCNT_SCBDUPDATE]++;
- rtw89_leave_ps_mode(rtwdev);
_update_bt_scbd(rtwdev, false);
break;
case BTF_EVNT_BT_PSD:
@@ -4765,7 +4804,6 @@ static void _show_bt_profile_info(struct rtw89_dev *rtwdev, struct seq_file *m)
static void _show_bt_info(struct rtw89_dev *rtwdev, struct seq_file *m)
{
- const struct rtw89_chip_info *chip = rtwdev->chip;
struct rtw89_btc *btc = &rtwdev->btc;
struct rtw89_btc_cx *cx = &btc->cx;
struct rtw89_btc_bt_info *bt = &cx->bt;
@@ -4773,7 +4811,6 @@ static void _show_bt_info(struct rtw89_dev *rtwdev, struct seq_file *m)
struct rtw89_btc_module *module = &btc->mdinfo;
struct rtw89_btc_bt_link_info *bt_linfo = &bt->link_info;
u8 *afh = bt_linfo->afh_map;
- u16 polt_cnt = 0;
if (!(btc->dm.coex_info_map & BTC_COEX_INFO_BT))
return;
@@ -4849,17 +4886,11 @@ static void _show_bt_info(struct rtw89_dev *rtwdev, struct seq_file *m)
cx->cnt_bt[BTC_BCNT_INFOUPDATE],
cx->cnt_bt[BTC_BCNT_INFOSAME]);
- if (wl->status.map.lps || wl->status.map.rf_off)
- return;
-
- chip->ops->btc_update_bt_cnt(rtwdev);
- _chk_btc_err(rtwdev, BTC_DCNT_BTCNT_FREEZE, 0);
-
seq_printf(m,
" %-15s : Hi-rx = %d, Hi-tx = %d, Lo-rx = %d, Lo-tx = %d (bt_polut_wl_tx = %d)\n",
"[trx_req_cnt]", cx->cnt_bt[BTC_BCNT_HIPRI_RX],
cx->cnt_bt[BTC_BCNT_HIPRI_TX], cx->cnt_bt[BTC_BCNT_LOPRI_RX],
- cx->cnt_bt[BTC_BCNT_LOPRI_TX], polt_cnt);
+ cx->cnt_bt[BTC_BCNT_LOPRI_TX], cx->cnt_bt[BTC_BCNT_POLUT]);
}
#define CASE_BTC_RSN_STR(e) case BTC_RSN_ ## e: return #e
@@ -5227,8 +5258,6 @@ static void _show_fbtc_cysta(struct rtw89_dev *rtwdev, struct seq_file *m)
pcysta->bcn_cnt[CXBCN_BT_SLOT],
pcysta->bcn_cnt[CXBCN_BT_OK]);
- _chk_btc_err(rtwdev, BTC_DCNT_CYCLE_FREEZE, (u32)pcysta->cycles);
-
for (i = 0; i < CXST_MAX; i++) {
if (!pcysta->slot_cnt[i])
continue;
@@ -5252,9 +5281,6 @@ static void _show_fbtc_cysta(struct rtw89_dev *rtwdev, struct seq_file *m)
}
seq_puts(m, "\n");
- _chk_btc_err(rtwdev, BTC_DCNT_W1_FREEZE, pcysta->slot_cnt[CXST_W1]);
- _chk_btc_err(rtwdev, BTC_DCNT_B1_FREEZE, pcysta->slot_cnt[CXST_B1]);
-
seq_printf(m, " %-15s : avg_t[wl:%d/bt:%d/lk:%d.%03d]",
"[cycle_time]",
pcysta->tavg_cycle[CXT_WL],
@@ -5606,9 +5632,6 @@ static void _show_summary(struct rtw89_dev *rtwdev, struct seq_file *m)
pfwinfo->event[BTF_EVNT_RPT], prptctrl->rpt_cnt,
prptctrl->rpt_enable, dm->error.val);
- _chk_btc_err(rtwdev, BTC_DCNT_RPT_FREEZE,
- pfwinfo->event[BTF_EVNT_RPT]);
-
if (dm->error.map.wl_fw_hang)
seq_puts(m, " (WL FW Hang!!)");
seq_puts(m, "\n");
diff --git a/drivers/net/wireless/realtek/rtw89/coex.h b/drivers/net/wireless/realtek/rtw89/coex.h
index 4b4565d..c3a722d 100644
--- a/drivers/net/wireless/realtek/rtw89/coex.h
+++ b/drivers/net/wireless/realtek/rtw89/coex.h
@@ -130,6 +130,12 @@ enum btc_rfctrl {
BTC_RFCTRL_MAX
};
+enum btc_lps_state {
+ BTC_LPS_OFF = 0,
+ BTC_LPS_RF_OFF = 1,
+ BTC_LPS_RF_ON = 2
+};
+
void rtw89_btc_ntfy_poweron(struct rtw89_dev *rtwdev);
void rtw89_btc_ntfy_poweroff(struct rtw89_dev *rtwdev);
void rtw89_btc_ntfy_init(struct rtw89_dev *rtwdev, u8 mode);
diff --git a/drivers/net/wireless/realtek/rtw89/core.c b/drivers/net/wireless/realtek/rtw89/core.c
index d02ec5a..a0737ee 100644
--- a/drivers/net/wireless/realtek/rtw89/core.c
+++ b/drivers/net/wireless/realtek/rtw89/core.c
@@ -1,6 +1,8 @@
// SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause
/* Copyright(c) 2019-2020 Realtek Corporation
*/
+#include <linux/ip.h>
+#include <linux/udp.h>
#include "coex.h"
#include "core.h"
@@ -143,20 +145,15 @@ static void rtw89_get_channel_params(struct cfg80211_chan_def *chandef,
{
struct ieee80211_channel *channel = chandef->chan;
enum nl80211_chan_width width = chandef->width;
- u8 *cch_by_bw = chan_param->cch_by_bw;
u32 primary_freq, center_freq;
u8 center_chan;
u8 bandwidth = RTW89_CHANNEL_WIDTH_20;
u8 primary_chan_idx = 0;
- u8 i;
center_chan = channel->hw_value;
primary_freq = channel->center_freq;
center_freq = chandef->center_freq1;
- /* assign the center channel used while 20M bw is selected */
- cch_by_bw[RTW89_CHANNEL_WIDTH_20] = channel->hw_value;
-
switch (width) {
case NL80211_CHAN_WIDTH_20_NOHT:
case NL80211_CHAN_WIDTH_20:
@@ -183,10 +180,6 @@ static void rtw89_get_channel_params(struct cfg80211_chan_def *chandef,
primary_chan_idx = RTW89_SC_20_UPMOST;
center_chan -= 6;
}
- /* assign the center channel used
- * while 40M bw is selected
- */
- cch_by_bw[RTW89_CHANNEL_WIDTH_40] = center_chan + 4;
} else {
if (center_freq - primary_freq == 10) {
primary_chan_idx = RTW89_SC_20_LOWER;
@@ -195,10 +188,6 @@ static void rtw89_get_channel_params(struct cfg80211_chan_def *chandef,
primary_chan_idx = RTW89_SC_20_LOWEST;
center_chan += 6;
}
- /* assign the center channel used
- * while 40M bw is selected
- */
- cch_by_bw[RTW89_CHANNEL_WIDTH_40] = center_chan - 4;
}
break;
default:
@@ -210,12 +199,6 @@ static void rtw89_get_channel_params(struct cfg80211_chan_def *chandef,
chan_param->primary_chan = channel->hw_value;
chan_param->bandwidth = bandwidth;
chan_param->pri_ch_idx = primary_chan_idx;
-
- /* assign the center channel used while current bw is selected */
- cch_by_bw[bandwidth] = center_chan;
-
- for (i = bandwidth + 1; i <= RTW89_MAX_CHANNEL_WIDTH; i++)
- cch_by_bw[i] = 0;
}
void rtw89_set_channel(struct rtw89_dev *rtwdev)
@@ -228,7 +211,6 @@ void rtw89_set_channel(struct rtw89_dev *rtwdev)
u8 center_chan, bandwidth;
u8 band_type;
bool band_changed;
- u8 i;
rtw89_get_channel_params(&hw->conf.chandef, &ch_param);
if (WARN(ch_param.center_chan == 0, "Invalid channel\n"))
@@ -242,6 +224,7 @@ void rtw89_set_channel(struct rtw89_dev *rtwdev)
hal->current_band_width = bandwidth;
hal->current_channel = center_chan;
+ hal->prev_primary_channel = hal->current_primary_channel;
hal->current_primary_channel = ch_param.primary_chan;
hal->current_band_type = band_type;
@@ -260,9 +243,6 @@ void rtw89_set_channel(struct rtw89_dev *rtwdev)
break;
}
- for (i = RTW89_CHANNEL_WIDTH_20; i <= RTW89_MAX_CHANNEL_WIDTH; i++)
- hal->cch_by_bw[i] = ch_param.cch_by_bw[i];
-
rtw89_chip_set_channel_prepare(rtwdev, &bak);
chip->ops->set_channel(rtwdev, &ch_param);
@@ -881,8 +861,11 @@ static void rtw89_core_parse_phy_status_ie01(struct rtw89_dev *rtwdev, u8 *addr,
{
s16 cfo;
+ phy_ppdu->chan_idx = RTW89_GET_PHY_STS_IE01_CH_IDX(addr);
+ if (phy_ppdu->rate < RTW89_HW_RATE_OFDM6)
+ return;
/* sign conversion for S(12,2) */
- cfo = sign_extend32(RTW89_GET_PHY_STS_IE0_CFO(addr), 11);
+ cfo = sign_extend32(RTW89_GET_PHY_STS_IE01_CFO(addr), 11);
rtw89_phy_cfo_parse(rtwdev, cfo, phy_ppdu);
}
@@ -908,6 +891,7 @@ static void rtw89_core_update_phy_ppdu(struct rtw89_rx_phy_ppdu *phy_ppdu)
s8 *rssi = phy_ppdu->rssi;
u8 *buf = phy_ppdu->buf;
+ phy_ppdu->ie = RTW89_GET_PHY_STS_IE_MAP(buf);
phy_ppdu->rssi_avg = RTW89_GET_PHY_STS_RSSI_AVG(buf);
rssi[RF_PATH_A] = RTW89_RSSI_RAW_TO_DBM(RTW89_GET_PHY_STS_RSSI_A(buf));
rssi[RF_PATH_B] = RTW89_RSSI_RAW_TO_DBM(RTW89_GET_PHY_STS_RSSI_B(buf));
@@ -936,8 +920,9 @@ static int rtw89_core_rx_parse_phy_sts(struct rtw89_dev *rtwdev,
u16 ie_len;
u8 *pos, *end;
- if (!phy_ppdu->to_self)
- return 0;
+ /* mark invalid reports and bypass them */
+ if (phy_ppdu->ie < RTW89_CCK_PKT)
+ return -EINVAL;
pos = (u8 *)phy_ppdu->buf + PHY_STS_HDR_LEN;
end = (u8 *)phy_ppdu->buf + phy_ppdu->len;
@@ -1000,9 +985,7 @@ static bool rtw89_core_rx_ppdu_match(struct rtw89_dev *rtwdev,
data_rate_mode = GET_DATA_RATE_MODE(data_rate);
if (data_rate_mode == DATA_RATE_MODE_NON_HT) {
rate_idx = GET_DATA_RATE_NOT_HT_IDX(data_rate);
- /* No 4 CCK rates for 5G */
- if (status->band == NL80211_BAND_5GHZ)
- rate_idx -= 4;
+ /* rate_idx is still hardware value here */
} else if (data_rate_mode == DATA_RATE_MODE_HT) {
rate_idx = GET_DATA_RATE_HT_IDX(data_rate);
} else if (data_rate_mode == DATA_RATE_MODE_VHT) {
@@ -1081,6 +1064,29 @@ static void rtw89_core_rx_stats(struct rtw89_dev *rtwdev,
rtw89_iterate_vifs_bh(rtwdev, rtw89_vif_rx_stats_iter, &iter_data);
}
+static void rtw89_correct_cck_chan(struct rtw89_dev *rtwdev,
+ struct ieee80211_rx_status *status)
+{
+ u16 chan = rtwdev->hal.prev_primary_channel;
+ u8 band = chan <= 14 ? NL80211_BAND_2GHZ : NL80211_BAND_5GHZ;
+
+ if (status->band != NL80211_BAND_2GHZ &&
+ status->encoding == RX_ENC_LEGACY &&
+ status->rate_idx < RTW89_HW_RATE_OFDM6) {
+ status->freq = ieee80211_channel_to_frequency(chan, band);
+ status->band = band;
+ }
+}
+
+static void rtw89_core_hw_to_sband_rate(struct ieee80211_rx_status *rx_status)
+{
+ if (rx_status->band == NL80211_BAND_2GHZ ||
+ rx_status->encoding != RX_ENC_LEGACY)
+ return;
+ /* No 4 CCK rates for non-2G */
+ rx_status->rate_idx -= 4;
+}
+
static void rtw89_core_rx_pending_skb(struct rtw89_dev *rtwdev,
struct rtw89_rx_phy_ppdu *phy_ppdu,
struct rtw89_rx_desc_info *desc_info,
@@ -1099,6 +1105,8 @@ static void rtw89_core_rx_pending_skb(struct rtw89_dev *rtwdev,
rx_status = IEEE80211_SKB_RXCB(skb_ppdu);
if (rtw89_core_rx_ppdu_match(rtwdev, desc_info, rx_status))
rtw89_chip_query_ppdu(rtwdev, phy_ppdu, rx_status);
+ rtw89_correct_cck_chan(rtwdev, rx_status);
+ rtw89_core_hw_to_sband_rate(rx_status);
rtw89_core_rx_stats(rtwdev, phy_ppdu, desc_info, skb_ppdu);
ieee80211_rx_napi(rtwdev->hw, NULL, skb_ppdu, &rtwdev->napi);
rtwdev->napi_budget_countdown--;
@@ -1112,6 +1120,7 @@ static void rtw89_core_rx_process_ppdu_sts(struct rtw89_dev *rtwdev,
struct rtw89_rx_phy_ppdu phy_ppdu = {.buf = skb->data, .valid = false,
.len = skb->len,
.to_self = desc_info->addr1_match,
+ .rate = desc_info->data_rate,
.mac_id = desc_info->mac_id};
int ret;
@@ -1267,12 +1276,7 @@ static void rtw89_core_update_rx_status(struct rtw89_dev *rtwdev,
if (data_rate_mode == DATA_RATE_MODE_NON_HT) {
rx_status->encoding = RX_ENC_LEGACY;
rx_status->rate_idx = GET_DATA_RATE_NOT_HT_IDX(data_rate);
- /* No 4 CCK rates for 5G */
- if (rx_status->band == NL80211_BAND_5GHZ)
- rx_status->rate_idx -= 4;
- if (rtwdev->scanning)
- rx_status->rate_idx = min_t(u8, rx_status->rate_idx,
- ARRAY_SIZE(rtw89_bitrates) - 5);
+ /* convert rate_idx after we get the correct band */
} else if (data_rate_mode == DATA_RATE_MODE_HT) {
rx_status->encoding = RX_ENC_HT;
rx_status->rate_idx = GET_DATA_RATE_HT_IDX(data_rate);
@@ -1324,10 +1328,13 @@ static void rtw89_core_flush_ppdu_rx_queue(struct rtw89_dev *rtwdev,
{
struct rtw89_ppdu_sts_info *ppdu_sts = &rtwdev->ppdu_sts;
u8 band = desc_info->bb_sel ? RTW89_PHY_1 : RTW89_PHY_0;
+ struct ieee80211_rx_status *rx_status;
struct sk_buff *skb_ppdu, *tmp;
skb_queue_walk_safe(&ppdu_sts->rx_queue[band], skb_ppdu, tmp) {
skb_unlink(skb_ppdu, &ppdu_sts->rx_queue[band]);
+ rx_status = IEEE80211_SKB_RXCB(skb_ppdu);
+ rtw89_core_hw_to_sband_rate(rx_status);
rtw89_core_rx_stats(rtwdev, NULL, desc_info, skb_ppdu);
ieee80211_rx_napi(rtwdev->hw, NULL, skb_ppdu, &rtwdev->napi);
rtwdev->napi_budget_countdown--;
@@ -1360,6 +1367,7 @@ void rtw89_core_rx(struct rtw89_dev *rtwdev,
BIT(desc_info->frame_type) & PPDU_FILTER_BITMAP) {
skb_queue_tail(&ppdu_sts->rx_queue[band], skb);
} else {
+ rtw89_core_hw_to_sband_rate(rx_status);
rtw89_core_rx_stats(rtwdev, NULL, desc_info, skb);
ieee80211_rx_napi(rtwdev->hw, NULL, skb, &rtwdev->napi);
rtwdev->napi_budget_countdown--;
@@ -1825,7 +1833,8 @@ int rtw89_core_sta_add(struct rtw89_dev *rtwdev,
ewma_rssi_init(&rtwsta->avg_rssi);
if (vif->type == NL80211_IFTYPE_STATION) {
- rtwvif->mgd.ap = sta;
+ /* for station mode, assign the mac_id from itself */
+ rtwsta->mac_id = rtwvif->mac_id;
rtw89_btc_ntfy_role_info(rtwdev, rtwvif, rtwsta,
BTC_ROLE_MSTS_STA_CONN_START);
rtw89_chip_rfk_channel(rtwdev);
@@ -1851,6 +1860,7 @@ int rtw89_core_sta_disconnect(struct rtw89_dev *rtwdev,
struct ieee80211_sta *sta)
{
struct rtw89_vif *rtwvif = (struct rtw89_vif *)vif->drv_priv;
+ struct rtw89_sta *rtwsta = (struct rtw89_sta *)sta->drv_priv;
int ret;
rtw89_mac_bf_monitor_calc(rtwdev, sta, true);
@@ -1872,7 +1882,7 @@ int rtw89_core_sta_disconnect(struct rtw89_dev *rtwdev,
}
/* update cam aid mac_id net_type */
- rtw89_fw_h2c_cam(rtwdev, rtwvif);
+ rtw89_fw_h2c_cam(rtwdev, rtwvif, rtwsta, NULL);
if (ret) {
rtw89_warn(rtwdev, "failed to send h2c cam\n");
return ret;
@@ -1897,10 +1907,6 @@ int rtw89_core_sta_assoc(struct rtw89_dev *rtwdev,
return ret;
}
- /* for station mode, assign the mac_id from itself */
- if (vif->type == NL80211_IFTYPE_STATION)
- rtwsta->mac_id = rtwvif->mac_id;
-
ret = rtw89_fw_h2c_join_info(rtwdev, rtwvif, 0);
if (ret) {
rtw89_warn(rtwdev, "failed to send h2c join info\n");
@@ -1908,7 +1914,7 @@ int rtw89_core_sta_assoc(struct rtw89_dev *rtwdev,
}
/* update cam aid mac_id net_type */
- rtw89_fw_h2c_cam(rtwdev, rtwvif);
+ rtw89_fw_h2c_cam(rtwdev, rtwvif, rtwsta, NULL);
if (ret) {
rtw89_warn(rtwdev, "failed to send h2c cam\n");
return ret;
@@ -2115,7 +2121,8 @@ static void rtw89_init_he_cap(struct rtw89_dev *rtwdev,
IEEE80211_HE_PHY_CAP9_RX_1024_QAM_LESS_THAN_242_TONE_RU |
IEEE80211_HE_PHY_CAP9_RX_FULL_BW_SU_USING_MU_WITH_COMP_SIGB |
IEEE80211_HE_PHY_CAP9_RX_FULL_BW_SU_USING_MU_WITH_NON_COMP_SIGB |
- IEEE80211_HE_PHY_CAP9_NOMIMAL_PKT_PADDING_16US;
+ u8_encode_bits(IEEE80211_HE_PHY_CAP9_NOMINAL_PKT_PADDING_16US,
+ IEEE80211_HE_PHY_CAP9_NOMINAL_PKT_PADDING_MASK);
if (i == NL80211_IFTYPE_STATION)
phy_cap_info[9] |= IEEE80211_HE_PHY_CAP9_TX_1024_QAM_LESS_THAN_242_TONE_RU;
he_cap->he_mcs_nss_supp.rx_mcs_80 = cpu_to_le16(mcs_map);
diff --git a/drivers/net/wireless/realtek/rtw89/core.h b/drivers/net/wireless/realtek/rtw89/core.h
index c2885e4..7c84556 100644
--- a/drivers/net/wireless/realtek/rtw89/core.h
+++ b/drivers/net/wireless/realtek/rtw89/core.h
@@ -411,12 +411,13 @@ enum rtw89_regulation_type {
RTW89_NA = 4,
RTW89_IC = 5,
RTW89_KCC = 6,
- RTW89_NCC = 7,
- RTW89_CHILE = 8,
- RTW89_ACMA = 9,
- RTW89_MEXICO = 10,
+ RTW89_ACMA = 7,
+ RTW89_NCC = 8,
+ RTW89_MEXICO = 9,
+ RTW89_CHILE = 10,
RTW89_UKRAINE = 11,
RTW89_CN = 12,
+ RTW89_QATAR = 13,
RTW89_REGD_NUM,
};
@@ -472,6 +473,9 @@ struct rtw89_rx_phy_ppdu {
u8 rssi_avg;
s8 rssi[RF_PATH_MAX];
u8 mac_id;
+ u8 chan_idx;
+ u8 ie;
+ u16 rate;
bool to_self;
bool valid;
};
@@ -543,7 +547,6 @@ enum rtw89_ps_mode {
RTW89_PS_MODE_PWR_GATED = 3,
};
-#define RTW89_MAX_CHANNEL_WIDTH RTW89_CHANNEL_WIDTH_80
#define RTW89_2G_BW_NUM (RTW89_CHANNEL_WIDTH_40 + 1)
#define RTW89_5G_BW_NUM (RTW89_CHANNEL_WIDTH_80 + 1)
#define RTW89_PPE_BW_NUM (RTW89_CHANNEL_WIDTH_80 + 1)
@@ -570,7 +573,6 @@ struct rtw89_channel_params {
u8 primary_chan;
u8 bandwidth;
u8 pri_ch_idx;
- u8 cch_by_bw[RTW89_MAX_CHANNEL_WIDTH + 1];
};
struct rtw89_channel_help_params {
@@ -803,6 +805,7 @@ enum rtw89_btc_bt_state_cnt {
BTC_BCNT_HIPRI_RX,
BTC_BCNT_LOPRI_TX,
BTC_BCNT_LOPRI_RX,
+ BTC_BCNT_POLUT,
BTC_BCNT_RATECHG,
BTC_BCNT_NUM
};
@@ -1865,7 +1868,6 @@ struct rtw89_addr_cam_entry {
u8 wapi : 1;
u8 mask_sel : 2;
u8 bssid_cam_idx: 6;
- u8 tma[ETH_ALEN];
u8 sma[ETH_ALEN];
u8 sec_ent_mode;
@@ -1934,14 +1936,6 @@ struct rtw89_vif {
bool wowlan_magic;
bool is_hesta;
bool last_a_ctrl;
- union {
- struct {
- struct ieee80211_sta *ap;
- } mgd;
- struct {
- struct list_head sta_list;
- } ap;
- };
struct rtw89_addr_cam_entry addr_cam;
struct rtw89_bssid_cam_entry bssid_cam;
struct ieee80211_tx_queue_params tx_params[IEEE80211_NUM_ACS];
@@ -2354,14 +2348,11 @@ struct rtw89_hal {
u32 rx_fltr;
u8 cv;
u8 current_channel;
+ u8 prev_primary_channel;
u8 current_primary_channel;
enum rtw89_subband current_subband;
u8 current_band_width;
u8 current_band_type;
- /* center channel for different available bandwidth,
- * val of (bw > current_band_width) is invalid
- */
- u8 cch_by_bw[RTW89_MAX_CHANNEL_WIDTH + 1];
u32 sw_amsdu_max_size;
u32 antenna_tx;
u32 antenna_rx;
@@ -3127,6 +3118,16 @@ static inline struct ieee80211_sta *rtwsta_to_sta(struct rtw89_sta *rtwsta)
return container_of(p, struct ieee80211_sta, drv_priv);
}
+static inline struct ieee80211_sta *rtwsta_to_sta_safe(struct rtw89_sta *rtwsta)
+{
+ return rtwsta ? rtwsta_to_sta(rtwsta) : NULL;
+}
+
+static inline struct rtw89_sta *sta_to_rtwsta_safe(struct ieee80211_sta *sta)
+{
+ return sta ? (struct rtw89_sta *)sta->drv_priv : NULL;
+}
+
static inline
void rtw89_chip_set_channel_prepare(struct rtw89_dev *rtwdev,
struct rtw89_channel_help_params *p)
diff --git a/drivers/net/wireless/realtek/rtw89/debug.c b/drivers/net/wireless/realtek/rtw89/debug.c
index 29eb188..22bd1d0 100644
--- a/drivers/net/wireless/realtek/rtw89/debug.c
+++ b/drivers/net/wireless/realtek/rtw89/debug.c
@@ -2,6 +2,8 @@
/* Copyright(c) 2019-2020 Realtek Corporation
*/
+#include <linux/vmalloc.h>
+
#include "coex.h"
#include "debug.h"
#include "fw.h"
@@ -723,6 +725,7 @@ rtw89_debug_priv_mac_mem_dump_select(struct file *filp,
}
static const u32 mac_mem_base_addr_table[RTW89_MAC_MEM_MAX] = {
+ [RTW89_MAC_MEM_AXIDMA] = AXIDMA_BASE_ADDR,
[RTW89_MAC_MEM_SHARED_BUF] = SHARED_BUF_BASE_ADDR,
[RTW89_MAC_MEM_DMAC_TBL] = DMAC_TBL_BASE_ADDR,
[RTW89_MAC_MEM_SHCUT_MACHDR] = SHCUT_MACHDR_BASE_ADDR,
@@ -735,6 +738,10 @@ static const u32 mac_mem_base_addr_table[RTW89_MAC_MEM_MAX] = {
[RTW89_MAC_MEM_BA_CAM] = BA_CAM_BASE_ADDR,
[RTW89_MAC_MEM_BCN_IE_CAM0] = BCN_IE_CAM0_BASE_ADDR,
[RTW89_MAC_MEM_BCN_IE_CAM1] = BCN_IE_CAM1_BASE_ADDR,
+ [RTW89_MAC_MEM_TXD_FIFO_0] = TXD_FIFO_0_BASE_ADDR,
+ [RTW89_MAC_MEM_TXD_FIFO_1] = TXD_FIFO_1_BASE_ADDR,
+ [RTW89_MAC_MEM_TXDATA_FIFO_0] = TXDATA_FIFO_0_BASE_ADDR,
+ [RTW89_MAC_MEM_TXDATA_FIFO_1] = TXDATA_FIFO_1_BASE_ADDR,
};
static void rtw89_debug_dump_mac_mem(struct seq_file *m,
@@ -814,7 +821,7 @@ rtw89_debug_priv_mac_dbg_port_dump_select(struct file *filp,
return -EINVAL;
}
- enable = set == 0 ? false : true;
+ enable = set != 0;
switch (sel) {
case 0:
debugfs_priv->dbgpkg_en.ss_dbg = enable;
@@ -2280,7 +2287,7 @@ static void rtw89_sta_info_get_iter(void *data, struct ieee80211_sta *sta)
switch (status->encoding) {
case RX_ENC_LEGACY:
seq_printf(m, "Legacy %d", status->rate_idx +
- (status->band == NL80211_BAND_5GHZ ? 4 : 0));
+ (status->band != NL80211_BAND_2GHZ ? 4 : 0));
break;
case RX_ENC_HT:
seq_printf(m, "HT MCS-%d%s", status->rate_idx,
diff --git a/drivers/net/wireless/realtek/rtw89/fw.c b/drivers/net/wireless/realtek/rtw89/fw.c
index 65ef3dc..8a57b75 100644
--- a/drivers/net/wireless/realtek/rtw89/fw.c
+++ b/drivers/net/wireless/realtek/rtw89/fw.c
@@ -523,7 +523,8 @@ void rtw89_unload_firmware(struct rtw89_dev *rtwdev)
}
#define H2C_CAM_LEN 60
-int rtw89_fw_h2c_cam(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif)
+int rtw89_fw_h2c_cam(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif,
+ struct rtw89_sta *rtwsta, const u8 *scan_mac_addr)
{
struct sk_buff *skb;
@@ -533,7 +534,7 @@ int rtw89_fw_h2c_cam(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif)
return -ENOMEM;
}
skb_put(skb, H2C_CAM_LEN);
- rtw89_cam_fill_addr_cam_info(rtwdev, rtwvif, skb->data);
+ rtw89_cam_fill_addr_cam_info(rtwdev, rtwvif, rtwsta, scan_mac_addr, skb->data);
rtw89_cam_fill_bssid_cam_info(rtwdev, rtwvif, skb->data);
rtw89_h2c_pkt_set_hdr(rtwdev, skb, FWCMD_TYPE_H2C,
@@ -780,7 +781,7 @@ static void __get_sta_he_pkt_padding(struct rtw89_dev *rtwdev,
if (!ppe_th) {
u8 pad;
- pad = FIELD_GET(IEEE80211_HE_PHY_CAP9_NOMIMAL_PKT_PADDING_MASK,
+ pad = FIELD_GET(IEEE80211_HE_PHY_CAP9_NOMINAL_PKT_PADDING_MASK,
sta->he_cap.he_cap_elem.phy_cap_info[9]);
for (i = 0; i < RTW89_PPE_BW_NUM; i++)
diff --git a/drivers/net/wireless/realtek/rtw89/fw.h b/drivers/net/wireless/realtek/rtw89/fw.h
index 36e8d0d..2d36dc2 100644
--- a/drivers/net/wireless/realtek/rtw89/fw.h
+++ b/drivers/net/wireless/realtek/rtw89/fw.h
@@ -156,863 +156,1125 @@ struct rtw89_h2creg_sch_tx_en {
u16 rsvd:15;
} __packed;
-#define RTW89_SET_FWCMD_RA_IS_DIS(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x00, val, BIT(0))
-#define RTW89_SET_FWCMD_RA_MODE(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x00, val, GENMASK(5, 1))
-#define RTW89_SET_FWCMD_RA_BW_CAP(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x00, val, GENMASK(7, 6))
-#define RTW89_SET_FWCMD_RA_MACID(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x00, val, GENMASK(15, 8))
-#define RTW89_SET_FWCMD_RA_DCM(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x00, val, BIT(16))
-#define RTW89_SET_FWCMD_RA_ER(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x00, val, BIT(17))
-#define RTW89_SET_FWCMD_RA_INIT_RATE_LV(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x00, val, GENMASK(19, 18))
-#define RTW89_SET_FWCMD_RA_UPD_ALL(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x00, val, BIT(20))
-#define RTW89_SET_FWCMD_RA_SGI(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x00, val, BIT(21))
-#define RTW89_SET_FWCMD_RA_LDPC(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x00, val, BIT(22))
-#define RTW89_SET_FWCMD_RA_STBC(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x00, val, BIT(23))
-#define RTW89_SET_FWCMD_RA_SS_NUM(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x00, val, GENMASK(26, 24))
-#define RTW89_SET_FWCMD_RA_GILTF(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x00, val, GENMASK(29, 27))
-#define RTW89_SET_FWCMD_RA_UPD_BW_NSS_MASK(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x00, val, BIT(30))
-#define RTW89_SET_FWCMD_RA_UPD_MASK(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x00, val, BIT(31))
-#define RTW89_SET_FWCMD_RA_MASK_0(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x01, val, GENMASK(7, 0))
-#define RTW89_SET_FWCMD_RA_MASK_1(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x01, val, GENMASK(15, 8))
-#define RTW89_SET_FWCMD_RA_MASK_2(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x01, val, GENMASK(23, 16))
-#define RTW89_SET_FWCMD_RA_MASK_3(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x01, val, GENMASK(31, 24))
-#define RTW89_SET_FWCMD_RA_MASK_4(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x02, val, GENMASK(7, 0))
-#define RTW89_SET_FWCMD_RA_BFEE_CSI_CTL(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x02, val, BIT(31))
-#define RTW89_SET_FWCMD_RA_BAND_NUM(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x03, val, GENMASK(7, 0))
-#define RTW89_SET_FWCMD_RA_RA_CSI_RATE_EN(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x03, val, BIT(8))
-#define RTW89_SET_FWCMD_RA_FIXED_CSI_RATE_EN(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x03, val, BIT(9))
-#define RTW89_SET_FWCMD_RA_CR_TBL_SEL(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x03, val, BIT(10))
-#define RTW89_SET_FWCMD_RA_FIXED_CSI_MCS_SS_IDX(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x03, val, GENMASK(23, 16))
-#define RTW89_SET_FWCMD_RA_FIXED_CSI_MODE(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x03, val, GENMASK(25, 24))
-#define RTW89_SET_FWCMD_RA_FIXED_CSI_GI_LTF(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x03, val, GENMASK(28, 26))
-#define RTW89_SET_FWCMD_RA_FIXED_CSI_BW(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x03, val, GENMASK(31, 29))
+static inline void RTW89_SET_FWCMD_RA_IS_DIS(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x00, val, BIT(0));
+}
-#define RTW89_SET_FWCMD_SEC_IDX(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x00, val, GENMASK(7, 0))
-#define RTW89_SET_FWCMD_SEC_OFFSET(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x00, val, GENMASK(15, 8))
-#define RTW89_SET_FWCMD_SEC_LEN(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x00, val, GENMASK(23, 16))
-#define RTW89_SET_FWCMD_SEC_TYPE(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x01, val, GENMASK(3, 0))
-#define RTW89_SET_FWCMD_SEC_EXT_KEY(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x01, val, BIT(4))
-#define RTW89_SET_FWCMD_SEC_SPP_MODE(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x01, val, BIT(5))
-#define RTW89_SET_FWCMD_SEC_KEY0(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x02, val, GENMASK(31, 0))
-#define RTW89_SET_FWCMD_SEC_KEY1(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x03, val, GENMASK(31, 0))
-#define RTW89_SET_FWCMD_SEC_KEY2(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x04, val, GENMASK(31, 0))
-#define RTW89_SET_FWCMD_SEC_KEY3(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x05, val, GENMASK(31, 0))
+static inline void RTW89_SET_FWCMD_RA_MODE(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x00, val, GENMASK(5, 1));
+}
-#define RTW89_SET_EDCA_SEL(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x00, val, GENMASK(1, 0))
-#define RTW89_SET_EDCA_BAND(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x00, val, BIT(3))
-#define RTW89_SET_EDCA_WMM(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x00, val, BIT(4))
-#define RTW89_SET_EDCA_AC(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x00, val, GENMASK(6, 5))
-#define RTW89_SET_EDCA_PARAM(cmd, val) \
- le32p_replace_bits((__le32 *)(cmd) + 0x01, val, GENMASK(31, 0))
+static inline void RTW89_SET_FWCMD_RA_BW_CAP(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x00, val, GENMASK(7, 6));
+}
+
+static inline void RTW89_SET_FWCMD_RA_MACID(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x00, val, GENMASK(15, 8));
+}
+
+static inline void RTW89_SET_FWCMD_RA_DCM(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x00, val, BIT(16));
+}
+
+static inline void RTW89_SET_FWCMD_RA_ER(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x00, val, BIT(17));
+}
+
+static inline void RTW89_SET_FWCMD_RA_INIT_RATE_LV(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x00, val, GENMASK(19, 18));
+}
+
+static inline void RTW89_SET_FWCMD_RA_UPD_ALL(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x00, val, BIT(20));
+}
+
+static inline void RTW89_SET_FWCMD_RA_SGI(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x00, val, BIT(21));
+}
+
+static inline void RTW89_SET_FWCMD_RA_LDPC(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x00, val, BIT(22));
+}
+
+static inline void RTW89_SET_FWCMD_RA_STBC(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x00, val, BIT(23));
+}
+
+static inline void RTW89_SET_FWCMD_RA_SS_NUM(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x00, val, GENMASK(26, 24));
+}
+
+static inline void RTW89_SET_FWCMD_RA_GILTF(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x00, val, GENMASK(29, 27));
+}
+
+static inline void RTW89_SET_FWCMD_RA_UPD_BW_NSS_MASK(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x00, val, BIT(30));
+}
+
+static inline void RTW89_SET_FWCMD_RA_UPD_MASK(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x00, val, BIT(31));
+}
+
+static inline void RTW89_SET_FWCMD_RA_MASK_0(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x01, val, GENMASK(7, 0));
+}
+
+static inline void RTW89_SET_FWCMD_RA_MASK_1(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x01, val, GENMASK(15, 8));
+}
+
+static inline void RTW89_SET_FWCMD_RA_MASK_2(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x01, val, GENMASK(23, 16));
+}
+
+static inline void RTW89_SET_FWCMD_RA_MASK_3(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x01, val, GENMASK(31, 24));
+}
+
+static inline void RTW89_SET_FWCMD_RA_MASK_4(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x02, val, GENMASK(7, 0));
+}
+
+static inline void RTW89_SET_FWCMD_RA_BFEE_CSI_CTL(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x02, val, BIT(31));
+}
+
+static inline void RTW89_SET_FWCMD_RA_BAND_NUM(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x03, val, GENMASK(7, 0));
+}
+
+static inline void RTW89_SET_FWCMD_RA_RA_CSI_RATE_EN(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x03, val, BIT(8));
+}
+
+static inline void RTW89_SET_FWCMD_RA_FIXED_CSI_RATE_EN(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x03, val, BIT(9));
+}
+
+static inline void RTW89_SET_FWCMD_RA_CR_TBL_SEL(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x03, val, BIT(10));
+}
+
+static inline void RTW89_SET_FWCMD_RA_FIXED_CSI_MCS_SS_IDX(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x03, val, GENMASK(23, 16));
+}
+
+static inline void RTW89_SET_FWCMD_RA_FIXED_CSI_MODE(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x03, val, GENMASK(25, 24));
+}
+
+static inline void RTW89_SET_FWCMD_RA_FIXED_CSI_GI_LTF(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x03, val, GENMASK(28, 26));
+}
+
+static inline void RTW89_SET_FWCMD_RA_FIXED_CSI_BW(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x03, val, GENMASK(31, 29));
+}
+
+static inline void RTW89_SET_FWCMD_SEC_IDX(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x00, val, GENMASK(7, 0));
+}
+
+static inline void RTW89_SET_FWCMD_SEC_OFFSET(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x00, val, GENMASK(15, 8));
+}
+
+static inline void RTW89_SET_FWCMD_SEC_LEN(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x00, val, GENMASK(23, 16));
+}
+
+static inline void RTW89_SET_FWCMD_SEC_TYPE(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x01, val, GENMASK(3, 0));
+}
+
+static inline void RTW89_SET_FWCMD_SEC_EXT_KEY(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x01, val, BIT(4));
+}
+
+static inline void RTW89_SET_FWCMD_SEC_SPP_MODE(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x01, val, BIT(5));
+}
+
+static inline void RTW89_SET_FWCMD_SEC_KEY0(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x02, val, GENMASK(31, 0));
+}
+
+static inline void RTW89_SET_FWCMD_SEC_KEY1(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x03, val, GENMASK(31, 0));
+}
+
+static inline void RTW89_SET_FWCMD_SEC_KEY2(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x04, val, GENMASK(31, 0));
+}
+
+static inline void RTW89_SET_FWCMD_SEC_KEY3(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x05, val, GENMASK(31, 0));
+}
+
+static inline void RTW89_SET_EDCA_SEL(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x00, val, GENMASK(1, 0));
+}
+
+static inline void RTW89_SET_EDCA_BAND(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x00, val, BIT(3));
+}
+
+static inline void RTW89_SET_EDCA_WMM(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x00, val, BIT(4));
+}
+
+static inline void RTW89_SET_EDCA_AC(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x00, val, GENMASK(6, 5));
+}
+
+static inline void RTW89_SET_EDCA_PARAM(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)(cmd) + 0x01, val, GENMASK(31, 0));
+}
#define FW_EDCA_PARAM_TXOPLMT_MSK GENMASK(26, 16)
#define FW_EDCA_PARAM_CWMAX_MSK GENMASK(15, 12)
#define FW_EDCA_PARAM_CWMIN_MSK GENMASK(11, 8)
#define FW_EDCA_PARAM_AIFS_MSK GENMASK(7, 0)
#define GET_FWSECTION_HDR_SEC_SIZE(fwhdr) \
- le32_get_bits(*((__le32 *)(fwhdr) + 1), GENMASK(23, 0))
+ le32_get_bits(*((const __le32 *)(fwhdr) + 1), GENMASK(23, 0))
#define GET_FWSECTION_HDR_CHECKSUM(fwhdr) \
- le32_get_bits(*((__le32 *)(fwhdr) + 1), BIT(28))
+ le32_get_bits(*((const __le32 *)(fwhdr) + 1), BIT(28))
#define GET_FWSECTION_HDR_REDL(fwhdr) \
- le32_get_bits(*((__le32 *)(fwhdr) + 1), BIT(29))
+ le32_get_bits(*((const __le32 *)(fwhdr) + 1), BIT(29))
#define GET_FWSECTION_HDR_DL_ADDR(fwhdr) \
- le32_get_bits(*((__le32 *)(fwhdr)), GENMASK(31, 0))
+ le32_get_bits(*((const __le32 *)(fwhdr)), GENMASK(31, 0))
#define GET_FW_HDR_MAJOR_VERSION(fwhdr) \
- le32_get_bits(*((__le32 *)(fwhdr) + 1), GENMASK(7, 0))
+ le32_get_bits(*((const __le32 *)(fwhdr) + 1), GENMASK(7, 0))
#define GET_FW_HDR_MINOR_VERSION(fwhdr) \
- le32_get_bits(*((__le32 *)(fwhdr) + 1), GENMASK(15, 8))
+ le32_get_bits(*((const __le32 *)(fwhdr) + 1), GENMASK(15, 8))
#define GET_FW_HDR_SUBVERSION(fwhdr) \
- le32_get_bits(*((__le32 *)(fwhdr) + 1), GENMASK(23, 16))
+ le32_get_bits(*((const __le32 *)(fwhdr) + 1), GENMASK(23, 16))
#define GET_FW_HDR_SUBINDEX(fwhdr) \
- le32_get_bits(*((__le32 *)(fwhdr) + 1), GENMASK(31, 24))
+ le32_get_bits(*((const __le32 *)(fwhdr) + 1), GENMASK(31, 24))
#define GET_FW_HDR_MONTH(fwhdr) \
- le32_get_bits(*((__le32 *)(fwhdr) + 4), GENMASK(7, 0))
+ le32_get_bits(*((const __le32 *)(fwhdr) + 4), GENMASK(7, 0))
#define GET_FW_HDR_DATE(fwhdr) \
- le32_get_bits(*((__le32 *)(fwhdr) + 4), GENMASK(15, 8))
+ le32_get_bits(*((const __le32 *)(fwhdr) + 4), GENMASK(15, 8))
#define GET_FW_HDR_HOUR(fwhdr) \
- le32_get_bits(*((__le32 *)(fwhdr) + 4), GENMASK(23, 16))
+ le32_get_bits(*((const __le32 *)(fwhdr) + 4), GENMASK(23, 16))
#define GET_FW_HDR_MIN(fwhdr) \
- le32_get_bits(*((__le32 *)(fwhdr) + 4), GENMASK(31, 24))
+ le32_get_bits(*((const __le32 *)(fwhdr) + 4), GENMASK(31, 24))
#define GET_FW_HDR_YEAR(fwhdr) \
- le32_get_bits(*((__le32 *)(fwhdr) + 5), GENMASK(31, 0))
+ le32_get_bits(*((const __le32 *)(fwhdr) + 5), GENMASK(31, 0))
#define GET_FW_HDR_SEC_NUM(fwhdr) \
- le32_get_bits(*((__le32 *)(fwhdr) + 6), GENMASK(15, 8))
+ le32_get_bits(*((const __le32 *)(fwhdr) + 6), GENMASK(15, 8))
#define GET_FW_HDR_CMD_VERSERION(fwhdr) \
- le32_get_bits(*((__le32 *)(fwhdr) + 7), GENMASK(31, 24))
+ le32_get_bits(*((const __le32 *)(fwhdr) + 7), GENMASK(31, 24))
static inline void SET_FW_HDR_PART_SIZE(void *fwhdr, u32 val)
{
le32p_replace_bits((__le32 *)fwhdr + 7, val, GENMASK(15, 0));
}
-#define SET_CTRL_INFO_MACID(table, val) \
- le32p_replace_bits((__le32 *)(table) + 0, val, GENMASK(6, 0))
-#define SET_CTRL_INFO_OPERATION(table, val) \
- le32p_replace_bits((__le32 *)(table) + 0, val, BIT(7))
+static inline void SET_CTRL_INFO_MACID(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 0, val, GENMASK(6, 0));
+}
+
+static inline void SET_CTRL_INFO_OPERATION(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 0, val, BIT(7));
+}
#define SET_CMC_TBL_MASK_DATARATE GENMASK(8, 0)
-#define SET_CMC_TBL_DATARATE(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 1, val, GENMASK(8, 0)); \
- le32p_replace_bits((__le32 *)(table) + 9, SET_CMC_TBL_MASK_DATARATE, \
- GENMASK(8, 0)); \
-} while (0)
+static inline void SET_CMC_TBL_DATARATE(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 1, val, GENMASK(8, 0));
+ le32p_replace_bits((__le32 *)(table) + 9, SET_CMC_TBL_MASK_DATARATE,
+ GENMASK(8, 0));
+}
#define SET_CMC_TBL_MASK_FORCE_TXOP BIT(0)
-#define SET_CMC_TBL_FORCE_TXOP(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 1, val, BIT(9)); \
- le32p_replace_bits((__le32 *)(table) + 9, SET_CMC_TBL_MASK_FORCE_TXOP, \
- BIT(9)); \
-} while (0)
+static inline void SET_CMC_TBL_FORCE_TXOP(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 1, val, BIT(9));
+ le32p_replace_bits((__le32 *)(table) + 9, SET_CMC_TBL_MASK_FORCE_TXOP,
+ BIT(9));
+}
#define SET_CMC_TBL_MASK_DATA_BW GENMASK(1, 0)
-#define SET_CMC_TBL_DATA_BW(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 1, val, GENMASK(11, 10)); \
- le32p_replace_bits((__le32 *)(table) + 9, SET_CMC_TBL_MASK_DATA_BW, \
- GENMASK(11, 10)); \
-} while (0)
+static inline void SET_CMC_TBL_DATA_BW(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 1, val, GENMASK(11, 10));
+ le32p_replace_bits((__le32 *)(table) + 9, SET_CMC_TBL_MASK_DATA_BW,
+ GENMASK(11, 10));
+}
#define SET_CMC_TBL_MASK_DATA_GI_LTF GENMASK(2, 0)
-#define SET_CMC_TBL_DATA_GI_LTF(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 1, val, GENMASK(14, 12)); \
- le32p_replace_bits((__le32 *)(table) + 9, SET_CMC_TBL_MASK_DATA_GI_LTF, \
- GENMASK(14, 12)); \
-} while (0)
+static inline void SET_CMC_TBL_DATA_GI_LTF(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 1, val, GENMASK(14, 12));
+ le32p_replace_bits((__le32 *)(table) + 9, SET_CMC_TBL_MASK_DATA_GI_LTF,
+ GENMASK(14, 12));
+}
#define SET_CMC_TBL_MASK_DARF_TC_INDEX BIT(0)
-#define SET_CMC_TBL_DARF_TC_INDEX(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 1, val, BIT(15)); \
- le32p_replace_bits((__le32 *)(table) + 9, SET_CMC_TBL_MASK_DARF_TC_INDEX, \
- BIT(15)); \
-} while (0)
+static inline void SET_CMC_TBL_DARF_TC_INDEX(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 1, val, BIT(15));
+ le32p_replace_bits((__le32 *)(table) + 9, SET_CMC_TBL_MASK_DARF_TC_INDEX,
+ BIT(15));
+}
#define SET_CMC_TBL_MASK_ARFR_CTRL GENMASK(3, 0)
-#define SET_CMC_TBL_ARFR_CTRL(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 1, val, GENMASK(19, 16)); \
- le32p_replace_bits((__le32 *)(table) + 9, SET_CMC_TBL_MASK_ARFR_CTRL, \
- GENMASK(19, 16)); \
-} while (0)
+static inline void SET_CMC_TBL_ARFR_CTRL(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 1, val, GENMASK(19, 16));
+ le32p_replace_bits((__le32 *)(table) + 9, SET_CMC_TBL_MASK_ARFR_CTRL,
+ GENMASK(19, 16));
+}
#define SET_CMC_TBL_MASK_ACQ_RPT_EN BIT(0)
-#define SET_CMC_TBL_ACQ_RPT_EN(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 1, val, BIT(20)); \
- le32p_replace_bits((__le32 *)(table) + 9, SET_CMC_TBL_MASK_ACQ_RPT_EN, \
- BIT(20)); \
-} while (0)
+static inline void SET_CMC_TBL_ACQ_RPT_EN(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 1, val, BIT(20));
+ le32p_replace_bits((__le32 *)(table) + 9, SET_CMC_TBL_MASK_ACQ_RPT_EN,
+ BIT(20));
+}
#define SET_CMC_TBL_MASK_MGQ_RPT_EN BIT(0)
-#define SET_CMC_TBL_MGQ_RPT_EN(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 1, val, BIT(21)); \
- le32p_replace_bits((__le32 *)(table) + 9, SET_CMC_TBL_MASK_MGQ_RPT_EN, \
- BIT(21)); \
-} while (0)
+static inline void SET_CMC_TBL_MGQ_RPT_EN(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 1, val, BIT(21));
+ le32p_replace_bits((__le32 *)(table) + 9, SET_CMC_TBL_MASK_MGQ_RPT_EN,
+ BIT(21));
+}
#define SET_CMC_TBL_MASK_ULQ_RPT_EN BIT(0)
-#define SET_CMC_TBL_ULQ_RPT_EN(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 1, val, BIT(22)); \
- le32p_replace_bits((__le32 *)(table) + 9, SET_CMC_TBL_MASK_ULQ_RPT_EN, \
- BIT(22)); \
-} while (0)
+static inline void SET_CMC_TBL_ULQ_RPT_EN(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 1, val, BIT(22));
+ le32p_replace_bits((__le32 *)(table) + 9, SET_CMC_TBL_MASK_ULQ_RPT_EN,
+ BIT(22));
+}
#define SET_CMC_TBL_MASK_TWTQ_RPT_EN BIT(0)
-#define SET_CMC_TBL_TWTQ_RPT_EN(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 1, val, BIT(23)); \
- le32p_replace_bits((__le32 *)(table) + 9, SET_CMC_TBL_MASK_TWTQ_RPT_EN, \
- BIT(23)); \
-} while (0)
+static inline void SET_CMC_TBL_TWTQ_RPT_EN(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 1, val, BIT(23));
+ le32p_replace_bits((__le32 *)(table) + 9, SET_CMC_TBL_MASK_TWTQ_RPT_EN,
+ BIT(23));
+}
#define SET_CMC_TBL_MASK_DISRTSFB BIT(0)
-#define SET_CMC_TBL_DISRTSFB(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 1, val, BIT(25)); \
- le32p_replace_bits((__le32 *)(table) + 9, SET_CMC_TBL_MASK_DISRTSFB, \
- BIT(25)); \
-} while (0)
+static inline void SET_CMC_TBL_DISRTSFB(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 1, val, BIT(25));
+ le32p_replace_bits((__le32 *)(table) + 9, SET_CMC_TBL_MASK_DISRTSFB,
+ BIT(25));
+}
#define SET_CMC_TBL_MASK_DISDATAFB BIT(0)
-#define SET_CMC_TBL_DISDATAFB(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 1, val, BIT(26)); \
- le32p_replace_bits((__le32 *)(table) + 9, SET_CMC_TBL_MASK_DISDATAFB, \
- BIT(26)); \
-} while (0)
+static inline void SET_CMC_TBL_DISDATAFB(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 1, val, BIT(26));
+ le32p_replace_bits((__le32 *)(table) + 9, SET_CMC_TBL_MASK_DISDATAFB,
+ BIT(26));
+}
#define SET_CMC_TBL_MASK_TRYRATE BIT(0)
-#define SET_CMC_TBL_TRYRATE(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 1, val, BIT(27)); \
- le32p_replace_bits((__le32 *)(table) + 9, SET_CMC_TBL_MASK_TRYRATE, \
- BIT(27)); \
-} while (0)
+static inline void SET_CMC_TBL_TRYRATE(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 1, val, BIT(27));
+ le32p_replace_bits((__le32 *)(table) + 9, SET_CMC_TBL_MASK_TRYRATE,
+ BIT(27));
+}
#define SET_CMC_TBL_MASK_AMPDU_DENSITY GENMASK(3, 0)
-#define SET_CMC_TBL_AMPDU_DENSITY(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 1, val, GENMASK(31, 28)); \
- le32p_replace_bits((__le32 *)(table) + 9, SET_CMC_TBL_MASK_AMPDU_DENSITY, \
- GENMASK(31, 28)); \
-} while (0)
+static inline void SET_CMC_TBL_AMPDU_DENSITY(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 1, val, GENMASK(31, 28));
+ le32p_replace_bits((__le32 *)(table) + 9, SET_CMC_TBL_MASK_AMPDU_DENSITY,
+ GENMASK(31, 28));
+}
#define SET_CMC_TBL_MASK_DATA_RTY_LOWEST_RATE GENMASK(8, 0)
-#define SET_CMC_TBL_DATA_RTY_LOWEST_RATE(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 2, val, GENMASK(8, 0)); \
- le32p_replace_bits((__le32 *)(table) + 10, SET_CMC_TBL_MASK_DATA_RTY_LOWEST_RATE, \
- GENMASK(8, 0)); \
-} while (0)
+static inline void SET_CMC_TBL_DATA_RTY_LOWEST_RATE(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 2, val, GENMASK(8, 0));
+ le32p_replace_bits((__le32 *)(table) + 10, SET_CMC_TBL_MASK_DATA_RTY_LOWEST_RATE,
+ GENMASK(8, 0));
+}
#define SET_CMC_TBL_MASK_AMPDU_TIME_SEL BIT(0)
-#define SET_CMC_TBL_AMPDU_TIME_SEL(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 2, val, BIT(9)); \
- le32p_replace_bits((__le32 *)(table) + 10, SET_CMC_TBL_MASK_AMPDU_TIME_SEL, \
- BIT(9)); \
-} while (0)
+static inline void SET_CMC_TBL_AMPDU_TIME_SEL(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 2, val, BIT(9));
+ le32p_replace_bits((__le32 *)(table) + 10, SET_CMC_TBL_MASK_AMPDU_TIME_SEL,
+ BIT(9));
+}
#define SET_CMC_TBL_MASK_AMPDU_LEN_SEL BIT(0)
-#define SET_CMC_TBL_AMPDU_LEN_SEL(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 2, val, BIT(10)); \
- le32p_replace_bits((__le32 *)(table) + 10, SET_CMC_TBL_MASK_AMPDU_LEN_SEL, \
- BIT(10)); \
-} while (0)
+static inline void SET_CMC_TBL_AMPDU_LEN_SEL(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 2, val, BIT(10));
+ le32p_replace_bits((__le32 *)(table) + 10, SET_CMC_TBL_MASK_AMPDU_LEN_SEL,
+ BIT(10));
+}
#define SET_CMC_TBL_MASK_RTS_TXCNT_LMT_SEL BIT(0)
-#define SET_CMC_TBL_RTS_TXCNT_LMT_SEL(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 2, val, BIT(11)); \
- le32p_replace_bits((__le32 *)(table) + 10, SET_CMC_TBL_MASK_RTS_TXCNT_LMT_SEL, \
- BIT(11)); \
-} while (0)
+static inline void SET_CMC_TBL_RTS_TXCNT_LMT_SEL(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 2, val, BIT(11));
+ le32p_replace_bits((__le32 *)(table) + 10, SET_CMC_TBL_MASK_RTS_TXCNT_LMT_SEL,
+ BIT(11));
+}
#define SET_CMC_TBL_MASK_RTS_TXCNT_LMT GENMASK(3, 0)
-#define SET_CMC_TBL_RTS_TXCNT_LMT(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 2, val, GENMASK(15, 12)); \
- le32p_replace_bits((__le32 *)(table) + 10, SET_CMC_TBL_MASK_RTS_TXCNT_LMT, \
- GENMASK(15, 12)); \
-} while (0)
+static inline void SET_CMC_TBL_RTS_TXCNT_LMT(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 2, val, GENMASK(15, 12));
+ le32p_replace_bits((__le32 *)(table) + 10, SET_CMC_TBL_MASK_RTS_TXCNT_LMT,
+ GENMASK(15, 12));
+}
#define SET_CMC_TBL_MASK_RTSRATE GENMASK(8, 0)
-#define SET_CMC_TBL_RTSRATE(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 2, val, GENMASK(24, 16)); \
- le32p_replace_bits((__le32 *)(table) + 10, SET_CMC_TBL_MASK_RTSRATE, \
- GENMASK(24, 16)); \
-} while (0)
+static inline void SET_CMC_TBL_RTSRATE(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 2, val, GENMASK(24, 16));
+ le32p_replace_bits((__le32 *)(table) + 10, SET_CMC_TBL_MASK_RTSRATE,
+ GENMASK(24, 16));
+}
#define SET_CMC_TBL_MASK_VCS_STBC BIT(0)
-#define SET_CMC_TBL_VCS_STBC(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 2, val, BIT(27)); \
- le32p_replace_bits((__le32 *)(table) + 10, SET_CMC_TBL_MASK_VCS_STBC, \
- BIT(27)); \
-} while (0)
+static inline void SET_CMC_TBL_VCS_STBC(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 2, val, BIT(27));
+ le32p_replace_bits((__le32 *)(table) + 10, SET_CMC_TBL_MASK_VCS_STBC,
+ BIT(27));
+}
#define SET_CMC_TBL_MASK_RTS_RTY_LOWEST_RATE GENMASK(3, 0)
-#define SET_CMC_TBL_RTS_RTY_LOWEST_RATE(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 2, val, GENMASK(31, 28)); \
- le32p_replace_bits((__le32 *)(table) + 10, SET_CMC_TBL_MASK_RTS_RTY_LOWEST_RATE, \
- GENMASK(31, 28)); \
-} while (0)
+static inline void SET_CMC_TBL_RTS_RTY_LOWEST_RATE(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 2, val, GENMASK(31, 28));
+ le32p_replace_bits((__le32 *)(table) + 10, SET_CMC_TBL_MASK_RTS_RTY_LOWEST_RATE,
+ GENMASK(31, 28));
+}
#define SET_CMC_TBL_MASK_DATA_TX_CNT_LMT GENMASK(5, 0)
-#define SET_CMC_TBL_DATA_TX_CNT_LMT(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 3, val, GENMASK(5, 0)); \
- le32p_replace_bits((__le32 *)(table) + 11, SET_CMC_TBL_MASK_DATA_TX_CNT_LMT, \
- GENMASK(5, 0)); \
-} while (0)
+static inline void SET_CMC_TBL_DATA_TX_CNT_LMT(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 3, val, GENMASK(5, 0));
+ le32p_replace_bits((__le32 *)(table) + 11, SET_CMC_TBL_MASK_DATA_TX_CNT_LMT,
+ GENMASK(5, 0));
+}
#define SET_CMC_TBL_MASK_DATA_TXCNT_LMT_SEL BIT(0)
-#define SET_CMC_TBL_DATA_TXCNT_LMT_SEL(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 3, val, BIT(6)); \
- le32p_replace_bits((__le32 *)(table) + 11, SET_CMC_TBL_MASK_DATA_TXCNT_LMT_SEL, \
- BIT(6)); \
-} while (0)
+static inline void SET_CMC_TBL_DATA_TXCNT_LMT_SEL(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 3, val, BIT(6));
+ le32p_replace_bits((__le32 *)(table) + 11, SET_CMC_TBL_MASK_DATA_TXCNT_LMT_SEL,
+ BIT(6));
+}
#define SET_CMC_TBL_MASK_MAX_AGG_NUM_SEL BIT(0)
-#define SET_CMC_TBL_MAX_AGG_NUM_SEL(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 3, val, BIT(7)); \
- le32p_replace_bits((__le32 *)(table) + 11, SET_CMC_TBL_MASK_MAX_AGG_NUM_SEL, \
- BIT(7)); \
-} while (0)
+static inline void SET_CMC_TBL_MAX_AGG_NUM_SEL(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 3, val, BIT(7));
+ le32p_replace_bits((__le32 *)(table) + 11, SET_CMC_TBL_MASK_MAX_AGG_NUM_SEL,
+ BIT(7));
+}
#define SET_CMC_TBL_MASK_RTS_EN BIT(0)
-#define SET_CMC_TBL_RTS_EN(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 3, val, BIT(8)); \
- le32p_replace_bits((__le32 *)(table) + 11, SET_CMC_TBL_MASK_RTS_EN, \
- BIT(8)); \
-} while (0)
+static inline void SET_CMC_TBL_RTS_EN(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 3, val, BIT(8));
+ le32p_replace_bits((__le32 *)(table) + 11, SET_CMC_TBL_MASK_RTS_EN,
+ BIT(8));
+}
#define SET_CMC_TBL_MASK_CTS2SELF_EN BIT(0)
-#define SET_CMC_TBL_CTS2SELF_EN(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 3, val, BIT(9)); \
- le32p_replace_bits((__le32 *)(table) + 11, SET_CMC_TBL_MASK_CTS2SELF_EN, \
- BIT(9)); \
-} while (0)
+static inline void SET_CMC_TBL_CTS2SELF_EN(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 3, val, BIT(9));
+ le32p_replace_bits((__le32 *)(table) + 11, SET_CMC_TBL_MASK_CTS2SELF_EN,
+ BIT(9));
+}
#define SET_CMC_TBL_MASK_CCA_RTS GENMASK(1, 0)
-#define SET_CMC_TBL_CCA_RTS(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 3, val, GENMASK(11, 10)); \
- le32p_replace_bits((__le32 *)(table) + 11, SET_CMC_TBL_MASK_CCA_RTS, \
- GENMASK(11, 10)); \
-} while (0)
+static inline void SET_CMC_TBL_CCA_RTS(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 3, val, GENMASK(11, 10));
+ le32p_replace_bits((__le32 *)(table) + 11, SET_CMC_TBL_MASK_CCA_RTS,
+ GENMASK(11, 10));
+}
#define SET_CMC_TBL_MASK_HW_RTS_EN BIT(0)
-#define SET_CMC_TBL_HW_RTS_EN(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 3, val, BIT(12)); \
- le32p_replace_bits((__le32 *)(table) + 11, SET_CMC_TBL_MASK_HW_RTS_EN, \
- BIT(12)); \
-} while (0)
+static inline void SET_CMC_TBL_HW_RTS_EN(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 3, val, BIT(12));
+ le32p_replace_bits((__le32 *)(table) + 11, SET_CMC_TBL_MASK_HW_RTS_EN,
+ BIT(12));
+}
#define SET_CMC_TBL_MASK_RTS_DROP_DATA_MODE GENMASK(1, 0)
-#define SET_CMC_TBL_RTS_DROP_DATA_MODE(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 3, val, GENMASK(14, 13)); \
- le32p_replace_bits((__le32 *)(table) + 11, SET_CMC_TBL_MASK_RTS_DROP_DATA_MODE, \
- GENMASK(14, 13)); \
-} while (0)
+static inline void SET_CMC_TBL_RTS_DROP_DATA_MODE(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 3, val, GENMASK(14, 13));
+ le32p_replace_bits((__le32 *)(table) + 11, SET_CMC_TBL_MASK_RTS_DROP_DATA_MODE,
+ GENMASK(14, 13));
+}
#define SET_CMC_TBL_MASK_AMPDU_MAX_LEN GENMASK(10, 0)
-#define SET_CMC_TBL_AMPDU_MAX_LEN(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 3, val, GENMASK(26, 16)); \
- le32p_replace_bits((__le32 *)(table) + 11, SET_CMC_TBL_MASK_AMPDU_MAX_LEN, \
- GENMASK(26, 16)); \
-} while (0)
+static inline void SET_CMC_TBL_AMPDU_MAX_LEN(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 3, val, GENMASK(26, 16));
+ le32p_replace_bits((__le32 *)(table) + 11, SET_CMC_TBL_MASK_AMPDU_MAX_LEN,
+ GENMASK(26, 16));
+}
#define SET_CMC_TBL_MASK_UL_MU_DIS BIT(0)
-#define SET_CMC_TBL_UL_MU_DIS(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 3, val, BIT(27)); \
- le32p_replace_bits((__le32 *)(table) + 11, SET_CMC_TBL_MASK_UL_MU_DIS, \
- BIT(27)); \
-} while (0)
+static inline void SET_CMC_TBL_UL_MU_DIS(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 3, val, BIT(27));
+ le32p_replace_bits((__le32 *)(table) + 11, SET_CMC_TBL_MASK_UL_MU_DIS,
+ BIT(27));
+}
#define SET_CMC_TBL_MASK_AMPDU_MAX_TIME GENMASK(3, 0)
-#define SET_CMC_TBL_AMPDU_MAX_TIME(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 3, val, GENMASK(31, 28)); \
- le32p_replace_bits((__le32 *)(table) + 11, SET_CMC_TBL_MASK_AMPDU_MAX_TIME, \
- GENMASK(31, 28)); \
-} while (0)
+static inline void SET_CMC_TBL_AMPDU_MAX_TIME(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 3, val, GENMASK(31, 28));
+ le32p_replace_bits((__le32 *)(table) + 11, SET_CMC_TBL_MASK_AMPDU_MAX_TIME,
+ GENMASK(31, 28));
+}
#define SET_CMC_TBL_MASK_MAX_AGG_NUM GENMASK(7, 0)
-#define SET_CMC_TBL_MAX_AGG_NUM(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 4, val, GENMASK(7, 0)); \
- le32p_replace_bits((__le32 *)(table) + 12, SET_CMC_TBL_MASK_MAX_AGG_NUM, \
- GENMASK(7, 0)); \
-} while (0)
+static inline void SET_CMC_TBL_MAX_AGG_NUM(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 4, val, GENMASK(7, 0));
+ le32p_replace_bits((__le32 *)(table) + 12, SET_CMC_TBL_MASK_MAX_AGG_NUM,
+ GENMASK(7, 0));
+}
#define SET_CMC_TBL_MASK_BA_BMAP GENMASK(1, 0)
-#define SET_CMC_TBL_BA_BMAP(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 4, val, GENMASK(9, 8)); \
- le32p_replace_bits((__le32 *)(table) + 12, SET_CMC_TBL_MASK_BA_BMAP, \
- GENMASK(9, 8)); \
-} while (0)
+static inline void SET_CMC_TBL_BA_BMAP(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 4, val, GENMASK(9, 8));
+ le32p_replace_bits((__le32 *)(table) + 12, SET_CMC_TBL_MASK_BA_BMAP,
+ GENMASK(9, 8));
+}
#define SET_CMC_TBL_MASK_VO_LFTIME_SEL GENMASK(2, 0)
-#define SET_CMC_TBL_VO_LFTIME_SEL(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 4, val, GENMASK(18, 16)); \
- le32p_replace_bits((__le32 *)(table) + 12, SET_CMC_TBL_MASK_VO_LFTIME_SEL, \
- GENMASK(18, 16)); \
-} while (0)
+static inline void SET_CMC_TBL_VO_LFTIME_SEL(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 4, val, GENMASK(18, 16));
+ le32p_replace_bits((__le32 *)(table) + 12, SET_CMC_TBL_MASK_VO_LFTIME_SEL,
+ GENMASK(18, 16));
+}
#define SET_CMC_TBL_MASK_VI_LFTIME_SEL GENMASK(2, 0)
-#define SET_CMC_TBL_VI_LFTIME_SEL(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 4, val, GENMASK(21, 19)); \
- le32p_replace_bits((__le32 *)(table) + 12, SET_CMC_TBL_MASK_VI_LFTIME_SEL, \
- GENMASK(21, 19)); \
-} while (0)
+static inline void SET_CMC_TBL_VI_LFTIME_SEL(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 4, val, GENMASK(21, 19));
+ le32p_replace_bits((__le32 *)(table) + 12, SET_CMC_TBL_MASK_VI_LFTIME_SEL,
+ GENMASK(21, 19));
+}
#define SET_CMC_TBL_MASK_BE_LFTIME_SEL GENMASK(2, 0)
-#define SET_CMC_TBL_BE_LFTIME_SEL(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 4, val, GENMASK(24, 22)); \
- le32p_replace_bits((__le32 *)(table) + 12, SET_CMC_TBL_MASK_BE_LFTIME_SEL, \
- GENMASK(24, 22)); \
-} while (0)
+static inline void SET_CMC_TBL_BE_LFTIME_SEL(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 4, val, GENMASK(24, 22));
+ le32p_replace_bits((__le32 *)(table) + 12, SET_CMC_TBL_MASK_BE_LFTIME_SEL,
+ GENMASK(24, 22));
+}
#define SET_CMC_TBL_MASK_BK_LFTIME_SEL GENMASK(2, 0)
-#define SET_CMC_TBL_BK_LFTIME_SEL(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 4, val, GENMASK(27, 25)); \
- le32p_replace_bits((__le32 *)(table) + 12, SET_CMC_TBL_MASK_BK_LFTIME_SEL, \
- GENMASK(27, 25)); \
-} while (0)
+static inline void SET_CMC_TBL_BK_LFTIME_SEL(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 4, val, GENMASK(27, 25));
+ le32p_replace_bits((__le32 *)(table) + 12, SET_CMC_TBL_MASK_BK_LFTIME_SEL,
+ GENMASK(27, 25));
+}
#define SET_CMC_TBL_MASK_SECTYPE GENMASK(3, 0)
-#define SET_CMC_TBL_SECTYPE(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 4, val, GENMASK(31, 28)); \
- le32p_replace_bits((__le32 *)(table) + 12, SET_CMC_TBL_MASK_SECTYPE, \
- GENMASK(31, 28)); \
-} while (0)
+static inline void SET_CMC_TBL_SECTYPE(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 4, val, GENMASK(31, 28));
+ le32p_replace_bits((__le32 *)(table) + 12, SET_CMC_TBL_MASK_SECTYPE,
+ GENMASK(31, 28));
+}
#define SET_CMC_TBL_MASK_MULTI_PORT_ID GENMASK(2, 0)
-#define SET_CMC_TBL_MULTI_PORT_ID(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 5, val, GENMASK(2, 0)); \
- le32p_replace_bits((__le32 *)(table) + 13, SET_CMC_TBL_MASK_MULTI_PORT_ID, \
- GENMASK(2, 0)); \
-} while (0)
+static inline void SET_CMC_TBL_MULTI_PORT_ID(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 5, val, GENMASK(2, 0));
+ le32p_replace_bits((__le32 *)(table) + 13, SET_CMC_TBL_MASK_MULTI_PORT_ID,
+ GENMASK(2, 0));
+}
#define SET_CMC_TBL_MASK_BMC BIT(0)
-#define SET_CMC_TBL_BMC(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 5, val, BIT(3)); \
- le32p_replace_bits((__le32 *)(table) + 13, SET_CMC_TBL_MASK_BMC, \
- BIT(3)); \
-} while (0)
+static inline void SET_CMC_TBL_BMC(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 5, val, BIT(3));
+ le32p_replace_bits((__le32 *)(table) + 13, SET_CMC_TBL_MASK_BMC,
+ BIT(3));
+}
#define SET_CMC_TBL_MASK_MBSSID GENMASK(3, 0)
-#define SET_CMC_TBL_MBSSID(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 5, val, GENMASK(7, 4)); \
- le32p_replace_bits((__le32 *)(table) + 13, SET_CMC_TBL_MASK_MBSSID, \
- GENMASK(7, 4)); \
-} while (0)
+static inline void SET_CMC_TBL_MBSSID(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 5, val, GENMASK(7, 4));
+ le32p_replace_bits((__le32 *)(table) + 13, SET_CMC_TBL_MASK_MBSSID,
+ GENMASK(7, 4));
+}
#define SET_CMC_TBL_MASK_NAVUSEHDR BIT(0)
-#define SET_CMC_TBL_NAVUSEHDR(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 5, val, BIT(8)); \
- le32p_replace_bits((__le32 *)(table) + 13, SET_CMC_TBL_MASK_NAVUSEHDR, \
- BIT(8)); \
-} while (0)
+static inline void SET_CMC_TBL_NAVUSEHDR(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 5, val, BIT(8));
+ le32p_replace_bits((__le32 *)(table) + 13, SET_CMC_TBL_MASK_NAVUSEHDR,
+ BIT(8));
+}
#define SET_CMC_TBL_MASK_TXPWR_MODE GENMASK(2, 0)
-#define SET_CMC_TBL_TXPWR_MODE(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 5, val, GENMASK(11, 9)); \
- le32p_replace_bits((__le32 *)(table) + 13, SET_CMC_TBL_MASK_TXPWR_MODE, \
- GENMASK(11, 9)); \
-} while (0)
+static inline void SET_CMC_TBL_TXPWR_MODE(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 5, val, GENMASK(11, 9));
+ le32p_replace_bits((__le32 *)(table) + 13, SET_CMC_TBL_MASK_TXPWR_MODE,
+ GENMASK(11, 9));
+}
#define SET_CMC_TBL_MASK_DATA_DCM BIT(0)
-#define SET_CMC_TBL_DATA_DCM(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 5, val, BIT(12)); \
- le32p_replace_bits((__le32 *)(table) + 13, SET_CMC_TBL_MASK_DATA_DCM, \
- BIT(12)); \
-} while (0)
+static inline void SET_CMC_TBL_DATA_DCM(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 5, val, BIT(12));
+ le32p_replace_bits((__le32 *)(table) + 13, SET_CMC_TBL_MASK_DATA_DCM,
+ BIT(12));
+}
#define SET_CMC_TBL_MASK_DATA_ER BIT(0)
-#define SET_CMC_TBL_DATA_ER(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 5, val, BIT(13)); \
- le32p_replace_bits((__le32 *)(table) + 13, SET_CMC_TBL_MASK_DATA_ER, \
- BIT(13)); \
-} while (0)
+static inline void SET_CMC_TBL_DATA_ER(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 5, val, BIT(13));
+ le32p_replace_bits((__le32 *)(table) + 13, SET_CMC_TBL_MASK_DATA_ER,
+ BIT(13));
+}
#define SET_CMC_TBL_MASK_DATA_LDPC BIT(0)
-#define SET_CMC_TBL_DATA_LDPC(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 5, val, BIT(14)); \
- le32p_replace_bits((__le32 *)(table) + 13, SET_CMC_TBL_MASK_DATA_LDPC, \
- BIT(14)); \
-} while (0)
+static inline void SET_CMC_TBL_DATA_LDPC(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 5, val, BIT(14));
+ le32p_replace_bits((__le32 *)(table) + 13, SET_CMC_TBL_MASK_DATA_LDPC,
+ BIT(14));
+}
#define SET_CMC_TBL_MASK_DATA_STBC BIT(0)
-#define SET_CMC_TBL_DATA_STBC(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 5, val, BIT(15)); \
- le32p_replace_bits((__le32 *)(table) + 13, SET_CMC_TBL_MASK_DATA_STBC, \
- BIT(15)); \
-} while (0)
+static inline void SET_CMC_TBL_DATA_STBC(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 5, val, BIT(15));
+ le32p_replace_bits((__le32 *)(table) + 13, SET_CMC_TBL_MASK_DATA_STBC,
+ BIT(15));
+}
#define SET_CMC_TBL_MASK_A_CTRL_BQR BIT(0)
-#define SET_CMC_TBL_A_CTRL_BQR(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 5, val, BIT(16)); \
- le32p_replace_bits((__le32 *)(table) + 13, SET_CMC_TBL_MASK_A_CTRL_BQR, \
- BIT(16)); \
-} while (0)
+static inline void SET_CMC_TBL_A_CTRL_BQR(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 5, val, BIT(16));
+ le32p_replace_bits((__le32 *)(table) + 13, SET_CMC_TBL_MASK_A_CTRL_BQR,
+ BIT(16));
+}
#define SET_CMC_TBL_MASK_A_CTRL_UPH BIT(0)
-#define SET_CMC_TBL_A_CTRL_UPH(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 5, val, BIT(17)); \
- le32p_replace_bits((__le32 *)(table) + 13, SET_CMC_TBL_MASK_A_CTRL_UPH, \
- BIT(17)); \
-} while (0)
+static inline void SET_CMC_TBL_A_CTRL_UPH(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 5, val, BIT(17));
+ le32p_replace_bits((__le32 *)(table) + 13, SET_CMC_TBL_MASK_A_CTRL_UPH,
+ BIT(17));
+}
#define SET_CMC_TBL_MASK_A_CTRL_BSR BIT(0)
-#define SET_CMC_TBL_A_CTRL_BSR(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 5, val, BIT(18)); \
- le32p_replace_bits((__le32 *)(table) + 13, SET_CMC_TBL_MASK_A_CTRL_BSR, \
- BIT(18)); \
-} while (0)
+static inline void SET_CMC_TBL_A_CTRL_BSR(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 5, val, BIT(18));
+ le32p_replace_bits((__le32 *)(table) + 13, SET_CMC_TBL_MASK_A_CTRL_BSR,
+ BIT(18));
+}
#define SET_CMC_TBL_MASK_A_CTRL_CAS BIT(0)
-#define SET_CMC_TBL_A_CTRL_CAS(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 5, val, BIT(19)); \
- le32p_replace_bits((__le32 *)(table) + 13, SET_CMC_TBL_MASK_A_CTRL_CAS, \
- BIT(19)); \
-} while (0)
+static inline void SET_CMC_TBL_A_CTRL_CAS(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 5, val, BIT(19));
+ le32p_replace_bits((__le32 *)(table) + 13, SET_CMC_TBL_MASK_A_CTRL_CAS,
+ BIT(19));
+}
#define SET_CMC_TBL_MASK_DATA_BW_ER BIT(0)
-#define SET_CMC_TBL_DATA_BW_ER(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 5, val, BIT(20)); \
- le32p_replace_bits((__le32 *)(table) + 13, SET_CMC_TBL_MASK_DATA_BW_ER, \
- BIT(20)); \
-} while (0)
+static inline void SET_CMC_TBL_DATA_BW_ER(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 5, val, BIT(20));
+ le32p_replace_bits((__le32 *)(table) + 13, SET_CMC_TBL_MASK_DATA_BW_ER,
+ BIT(20));
+}
#define SET_CMC_TBL_MASK_LSIG_TXOP_EN BIT(0)
-#define SET_CMC_TBL_LSIG_TXOP_EN(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 5, val, BIT(21)); \
- le32p_replace_bits((__le32 *)(table) + 13, SET_CMC_TBL_MASK_LSIG_TXOP_EN, \
- BIT(21)); \
-} while (0)
+static inline void SET_CMC_TBL_LSIG_TXOP_EN(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 5, val, BIT(21));
+ le32p_replace_bits((__le32 *)(table) + 13, SET_CMC_TBL_MASK_LSIG_TXOP_EN,
+ BIT(21));
+}
#define SET_CMC_TBL_MASK_CTRL_CNT_VLD BIT(0)
-#define SET_CMC_TBL_CTRL_CNT_VLD(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 5, val, BIT(27)); \
- le32p_replace_bits((__le32 *)(table) + 13, SET_CMC_TBL_MASK_CTRL_CNT_VLD, \
- BIT(27)); \
-} while (0)
+static inline void SET_CMC_TBL_CTRL_CNT_VLD(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 5, val, BIT(27));
+ le32p_replace_bits((__le32 *)(table) + 13, SET_CMC_TBL_MASK_CTRL_CNT_VLD,
+ BIT(27));
+}
#define SET_CMC_TBL_MASK_CTRL_CNT GENMASK(3, 0)
-#define SET_CMC_TBL_CTRL_CNT(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 5, val, GENMASK(31, 28)); \
- le32p_replace_bits((__le32 *)(table) + 13, SET_CMC_TBL_MASK_CTRL_CNT, \
- GENMASK(31, 28)); \
-} while (0)
+static inline void SET_CMC_TBL_CTRL_CNT(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 5, val, GENMASK(31, 28));
+ le32p_replace_bits((__le32 *)(table) + 13, SET_CMC_TBL_MASK_CTRL_CNT,
+ GENMASK(31, 28));
+}
#define SET_CMC_TBL_MASK_RESP_REF_RATE GENMASK(8, 0)
-#define SET_CMC_TBL_RESP_REF_RATE(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 6, val, GENMASK(8, 0)); \
- le32p_replace_bits((__le32 *)(table) + 14, SET_CMC_TBL_MASK_RESP_REF_RATE, \
- GENMASK(8, 0)); \
-} while (0)
+static inline void SET_CMC_TBL_RESP_REF_RATE(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 6, val, GENMASK(8, 0));
+ le32p_replace_bits((__le32 *)(table) + 14, SET_CMC_TBL_MASK_RESP_REF_RATE,
+ GENMASK(8, 0));
+}
#define SET_CMC_TBL_MASK_ALL_ACK_SUPPORT BIT(0)
-#define SET_CMC_TBL_ALL_ACK_SUPPORT(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 6, val, BIT(12)); \
- le32p_replace_bits((__le32 *)(table) + 14, SET_CMC_TBL_MASK_ALL_ACK_SUPPORT, \
- BIT(12)); \
-} while (0)
+static inline void SET_CMC_TBL_ALL_ACK_SUPPORT(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 6, val, BIT(12));
+ le32p_replace_bits((__le32 *)(table) + 14, SET_CMC_TBL_MASK_ALL_ACK_SUPPORT,
+ BIT(12));
+}
#define SET_CMC_TBL_MASK_BSR_QUEUE_SIZE_FORMAT BIT(0)
-#define SET_CMC_TBL_BSR_QUEUE_SIZE_FORMAT(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 6, val, BIT(13)); \
- le32p_replace_bits((__le32 *)(table) + 14, SET_CMC_TBL_MASK_BSR_QUEUE_SIZE_FORMAT, \
- BIT(13)); \
-} while (0)
+static inline void SET_CMC_TBL_BSR_QUEUE_SIZE_FORMAT(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 6, val, BIT(13));
+ le32p_replace_bits((__le32 *)(table) + 14, SET_CMC_TBL_MASK_BSR_QUEUE_SIZE_FORMAT,
+ BIT(13));
+}
#define SET_CMC_TBL_MASK_NTX_PATH_EN GENMASK(3, 0)
-#define SET_CMC_TBL_NTX_PATH_EN(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 6, val, GENMASK(19, 16)); \
- le32p_replace_bits((__le32 *)(table) + 14, SET_CMC_TBL_MASK_NTX_PATH_EN, \
- GENMASK(19, 16)); \
-} while (0)
+static inline void SET_CMC_TBL_NTX_PATH_EN(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 6, val, GENMASK(19, 16));
+ le32p_replace_bits((__le32 *)(table) + 14, SET_CMC_TBL_MASK_NTX_PATH_EN,
+ GENMASK(19, 16));
+}
#define SET_CMC_TBL_MASK_PATH_MAP_A GENMASK(1, 0)
-#define SET_CMC_TBL_PATH_MAP_A(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 6, val, GENMASK(21, 20)); \
- le32p_replace_bits((__le32 *)(table) + 14, SET_CMC_TBL_MASK_PATH_MAP_A, \
- GENMASK(21, 20)); \
-} while (0)
+static inline void SET_CMC_TBL_PATH_MAP_A(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 6, val, GENMASK(21, 20));
+ le32p_replace_bits((__le32 *)(table) + 14, SET_CMC_TBL_MASK_PATH_MAP_A,
+ GENMASK(21, 20));
+}
#define SET_CMC_TBL_MASK_PATH_MAP_B GENMASK(1, 0)
-#define SET_CMC_TBL_PATH_MAP_B(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 6, val, GENMASK(23, 22)); \
- le32p_replace_bits((__le32 *)(table) + 14, SET_CMC_TBL_MASK_PATH_MAP_B, \
- GENMASK(23, 22)); \
-} while (0)
+static inline void SET_CMC_TBL_PATH_MAP_B(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 6, val, GENMASK(23, 22));
+ le32p_replace_bits((__le32 *)(table) + 14, SET_CMC_TBL_MASK_PATH_MAP_B,
+ GENMASK(23, 22));
+}
#define SET_CMC_TBL_MASK_PATH_MAP_C GENMASK(1, 0)
-#define SET_CMC_TBL_PATH_MAP_C(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 6, val, GENMASK(25, 24)); \
- le32p_replace_bits((__le32 *)(table) + 14, SET_CMC_TBL_MASK_PATH_MAP_C, \
- GENMASK(25, 24)); \
-} while (0)
+static inline void SET_CMC_TBL_PATH_MAP_C(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 6, val, GENMASK(25, 24));
+ le32p_replace_bits((__le32 *)(table) + 14, SET_CMC_TBL_MASK_PATH_MAP_C,
+ GENMASK(25, 24));
+}
#define SET_CMC_TBL_MASK_PATH_MAP_D GENMASK(1, 0)
-#define SET_CMC_TBL_PATH_MAP_D(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 6, val, GENMASK(27, 26)); \
- le32p_replace_bits((__le32 *)(table) + 14, SET_CMC_TBL_MASK_PATH_MAP_D, \
- GENMASK(27, 26)); \
-} while (0)
+static inline void SET_CMC_TBL_PATH_MAP_D(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 6, val, GENMASK(27, 26));
+ le32p_replace_bits((__le32 *)(table) + 14, SET_CMC_TBL_MASK_PATH_MAP_D,
+ GENMASK(27, 26));
+}
#define SET_CMC_TBL_MASK_ANTSEL_A BIT(0)
-#define SET_CMC_TBL_ANTSEL_A(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 6, val, BIT(28)); \
- le32p_replace_bits((__le32 *)(table) + 14, SET_CMC_TBL_MASK_ANTSEL_A, \
- BIT(28)); \
-} while (0)
+static inline void SET_CMC_TBL_ANTSEL_A(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 6, val, BIT(28));
+ le32p_replace_bits((__le32 *)(table) + 14, SET_CMC_TBL_MASK_ANTSEL_A,
+ BIT(28));
+}
#define SET_CMC_TBL_MASK_ANTSEL_B BIT(0)
-#define SET_CMC_TBL_ANTSEL_B(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 6, val, BIT(29)); \
- le32p_replace_bits((__le32 *)(table) + 14, SET_CMC_TBL_MASK_ANTSEL_B, \
- BIT(29)); \
-} while (0)
+static inline void SET_CMC_TBL_ANTSEL_B(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 6, val, BIT(29));
+ le32p_replace_bits((__le32 *)(table) + 14, SET_CMC_TBL_MASK_ANTSEL_B,
+ BIT(29));
+}
#define SET_CMC_TBL_MASK_ANTSEL_C BIT(0)
-#define SET_CMC_TBL_ANTSEL_C(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 6, val, BIT(30)); \
- le32p_replace_bits((__le32 *)(table) + 14, SET_CMC_TBL_MASK_ANTSEL_C, \
- BIT(30)); \
-} while (0)
+static inline void SET_CMC_TBL_ANTSEL_C(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 6, val, BIT(30));
+ le32p_replace_bits((__le32 *)(table) + 14, SET_CMC_TBL_MASK_ANTSEL_C,
+ BIT(30));
+}
#define SET_CMC_TBL_MASK_ANTSEL_D BIT(0)
-#define SET_CMC_TBL_ANTSEL_D(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 6, val, BIT(31)); \
- le32p_replace_bits((__le32 *)(table) + 14, SET_CMC_TBL_MASK_ANTSEL_D, \
- BIT(31)); \
-} while (0)
+static inline void SET_CMC_TBL_ANTSEL_D(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 6, val, BIT(31));
+ le32p_replace_bits((__le32 *)(table) + 14, SET_CMC_TBL_MASK_ANTSEL_D,
+ BIT(31));
+}
#define SET_CMC_TBL_MASK_ADDR_CAM_INDEX GENMASK(7, 0)
-#define SET_CMC_TBL_ADDR_CAM_INDEX(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 7, val, GENMASK(7, 0)); \
- le32p_replace_bits((__le32 *)(table) + 15, SET_CMC_TBL_MASK_ADDR_CAM_INDEX, \
- GENMASK(7, 0)); \
-} while (0)
+static inline void SET_CMC_TBL_ADDR_CAM_INDEX(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 7, val, GENMASK(7, 0));
+ le32p_replace_bits((__le32 *)(table) + 15, SET_CMC_TBL_MASK_ADDR_CAM_INDEX,
+ GENMASK(7, 0));
+}
#define SET_CMC_TBL_MASK_PAID GENMASK(8, 0)
-#define SET_CMC_TBL_PAID(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 7, val, GENMASK(16, 8)); \
- le32p_replace_bits((__le32 *)(table) + 15, SET_CMC_TBL_MASK_PAID, \
- GENMASK(16, 8)); \
-} while (0)
+static inline void SET_CMC_TBL_PAID(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 7, val, GENMASK(16, 8));
+ le32p_replace_bits((__le32 *)(table) + 15, SET_CMC_TBL_MASK_PAID,
+ GENMASK(16, 8));
+}
#define SET_CMC_TBL_MASK_ULDL BIT(0)
-#define SET_CMC_TBL_ULDL(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 7, val, BIT(17)); \
- le32p_replace_bits((__le32 *)(table) + 15, SET_CMC_TBL_MASK_ULDL, \
- BIT(17)); \
-} while (0)
+static inline void SET_CMC_TBL_ULDL(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 7, val, BIT(17));
+ le32p_replace_bits((__le32 *)(table) + 15, SET_CMC_TBL_MASK_ULDL,
+ BIT(17));
+}
#define SET_CMC_TBL_MASK_DOPPLER_CTRL GENMASK(1, 0)
-#define SET_CMC_TBL_DOPPLER_CTRL(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 7, val, GENMASK(19, 18)); \
- le32p_replace_bits((__le32 *)(table) + 15, SET_CMC_TBL_MASK_DOPPLER_CTRL, \
- GENMASK(19, 18)); \
-} while (0)
+static inline void SET_CMC_TBL_DOPPLER_CTRL(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 7, val, GENMASK(19, 18));
+ le32p_replace_bits((__le32 *)(table) + 15, SET_CMC_TBL_MASK_DOPPLER_CTRL,
+ GENMASK(19, 18));
+}
#define SET_CMC_TBL_MASK_NOMINAL_PKT_PADDING GENMASK(1, 0)
-#define SET_CMC_TBL_NOMINAL_PKT_PADDING(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 7, val, GENMASK(21, 20)); \
- le32p_replace_bits((__le32 *)(table) + 15, SET_CMC_TBL_MASK_NOMINAL_PKT_PADDING, \
- GENMASK(21, 20)); \
-} while (0)
-#define SET_CMC_TBL_NOMINAL_PKT_PADDING40(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 7, val, GENMASK(23, 22)); \
- le32p_replace_bits((__le32 *)(table) + 15, SET_CMC_TBL_MASK_NOMINAL_PKT_PADDING, \
- GENMASK(23, 22)); \
-} while (0)
+static inline void SET_CMC_TBL_NOMINAL_PKT_PADDING(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 7, val, GENMASK(21, 20));
+ le32p_replace_bits((__le32 *)(table) + 15, SET_CMC_TBL_MASK_NOMINAL_PKT_PADDING,
+ GENMASK(21, 20));
+}
+
+static inline void SET_CMC_TBL_NOMINAL_PKT_PADDING40(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 7, val, GENMASK(23, 22));
+ le32p_replace_bits((__le32 *)(table) + 15, SET_CMC_TBL_MASK_NOMINAL_PKT_PADDING,
+ GENMASK(23, 22));
+}
#define SET_CMC_TBL_MASK_TXPWR_TOLERENCE GENMASK(3, 0)
-#define SET_CMC_TBL_TXPWR_TOLERENCE(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 7, val, GENMASK(27, 24)); \
- le32p_replace_bits((__le32 *)(table) + 15, SET_CMC_TBL_MASK_TXPWR_TOLERENCE, \
- GENMASK(27, 24)); \
-} while (0)
-#define SET_CMC_TBL_NOMINAL_PKT_PADDING80(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 7, val, GENMASK(31, 30)); \
- le32p_replace_bits((__le32 *)(table) + 15, SET_CMC_TBL_MASK_NOMINAL_PKT_PADDING, \
- GENMASK(31, 30)); \
-} while (0)
+static inline void SET_CMC_TBL_TXPWR_TOLERENCE(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 7, val, GENMASK(27, 24));
+ le32p_replace_bits((__le32 *)(table) + 15, SET_CMC_TBL_MASK_TXPWR_TOLERENCE,
+ GENMASK(27, 24));
+}
+
+static inline void SET_CMC_TBL_NOMINAL_PKT_PADDING80(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 7, val, GENMASK(31, 30));
+ le32p_replace_bits((__le32 *)(table) + 15, SET_CMC_TBL_MASK_NOMINAL_PKT_PADDING,
+ GENMASK(31, 30));
+}
#define SET_CMC_TBL_MASK_NC GENMASK(2, 0)
-#define SET_CMC_TBL_NC(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 8, val, GENMASK(2, 0)); \
- le32p_replace_bits((__le32 *)(table) + 16, SET_CMC_TBL_MASK_NC, \
- GENMASK(2, 0)); \
-} while (0)
+static inline void SET_CMC_TBL_NC(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 8, val, GENMASK(2, 0));
+ le32p_replace_bits((__le32 *)(table) + 16, SET_CMC_TBL_MASK_NC,
+ GENMASK(2, 0));
+}
#define SET_CMC_TBL_MASK_NR GENMASK(2, 0)
-#define SET_CMC_TBL_NR(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 8, val, GENMASK(5, 3)); \
- le32p_replace_bits((__le32 *)(table) + 16, SET_CMC_TBL_MASK_NR, \
- GENMASK(5, 3)); \
-} while (0)
+static inline void SET_CMC_TBL_NR(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 8, val, GENMASK(5, 3));
+ le32p_replace_bits((__le32 *)(table) + 16, SET_CMC_TBL_MASK_NR,
+ GENMASK(5, 3));
+}
#define SET_CMC_TBL_MASK_NG GENMASK(1, 0)
-#define SET_CMC_TBL_NG(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 8, val, GENMASK(7, 6)); \
- le32p_replace_bits((__le32 *)(table) + 16, SET_CMC_TBL_MASK_NG, \
- GENMASK(7, 6)); \
-} while (0)
+static inline void SET_CMC_TBL_NG(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 8, val, GENMASK(7, 6));
+ le32p_replace_bits((__le32 *)(table) + 16, SET_CMC_TBL_MASK_NG,
+ GENMASK(7, 6));
+}
#define SET_CMC_TBL_MASK_CB GENMASK(1, 0)
-#define SET_CMC_TBL_CB(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 8, val, GENMASK(9, 8)); \
- le32p_replace_bits((__le32 *)(table) + 16, SET_CMC_TBL_MASK_CB, \
- GENMASK(9, 8)); \
-} while (0)
+static inline void SET_CMC_TBL_CB(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 8, val, GENMASK(9, 8));
+ le32p_replace_bits((__le32 *)(table) + 16, SET_CMC_TBL_MASK_CB,
+ GENMASK(9, 8));
+}
#define SET_CMC_TBL_MASK_CS GENMASK(1, 0)
-#define SET_CMC_TBL_CS(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 8, val, GENMASK(11, 10)); \
- le32p_replace_bits((__le32 *)(table) + 16, SET_CMC_TBL_MASK_CS, \
- GENMASK(11, 10)); \
-} while (0)
+static inline void SET_CMC_TBL_CS(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 8, val, GENMASK(11, 10));
+ le32p_replace_bits((__le32 *)(table) + 16, SET_CMC_TBL_MASK_CS,
+ GENMASK(11, 10));
+}
#define SET_CMC_TBL_MASK_CSI_TXBF_EN BIT(0)
-#define SET_CMC_TBL_CSI_TXBF_EN(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 8, val, BIT(12)); \
- le32p_replace_bits((__le32 *)(table) + 16, SET_CMC_TBL_MASK_CSI_TXBF_EN, \
- BIT(12)); \
-} while (0)
+static inline void SET_CMC_TBL_CSI_TXBF_EN(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 8, val, BIT(12));
+ le32p_replace_bits((__le32 *)(table) + 16, SET_CMC_TBL_MASK_CSI_TXBF_EN,
+ BIT(12));
+}
#define SET_CMC_TBL_MASK_CSI_STBC_EN BIT(0)
-#define SET_CMC_TBL_CSI_STBC_EN(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 8, val, BIT(13)); \
- le32p_replace_bits((__le32 *)(table) + 16, SET_CMC_TBL_MASK_CSI_STBC_EN, \
- BIT(13)); \
-} while (0)
+static inline void SET_CMC_TBL_CSI_STBC_EN(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 8, val, BIT(13));
+ le32p_replace_bits((__le32 *)(table) + 16, SET_CMC_TBL_MASK_CSI_STBC_EN,
+ BIT(13));
+}
#define SET_CMC_TBL_MASK_CSI_LDPC_EN BIT(0)
-#define SET_CMC_TBL_CSI_LDPC_EN(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 8, val, BIT(14)); \
- le32p_replace_bits((__le32 *)(table) + 16, SET_CMC_TBL_MASK_CSI_LDPC_EN, \
- BIT(14)); \
-} while (0)
+static inline void SET_CMC_TBL_CSI_LDPC_EN(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 8, val, BIT(14));
+ le32p_replace_bits((__le32 *)(table) + 16, SET_CMC_TBL_MASK_CSI_LDPC_EN,
+ BIT(14));
+}
#define SET_CMC_TBL_MASK_CSI_PARA_EN BIT(0)
-#define SET_CMC_TBL_CSI_PARA_EN(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 8, val, BIT(15)); \
- le32p_replace_bits((__le32 *)(table) + 16, SET_CMC_TBL_MASK_CSI_PARA_EN, \
- BIT(15)); \
-} while (0)
+static inline void SET_CMC_TBL_CSI_PARA_EN(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 8, val, BIT(15));
+ le32p_replace_bits((__le32 *)(table) + 16, SET_CMC_TBL_MASK_CSI_PARA_EN,
+ BIT(15));
+}
#define SET_CMC_TBL_MASK_CSI_FIX_RATE GENMASK(8, 0)
-#define SET_CMC_TBL_CSI_FIX_RATE(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 8, val, GENMASK(24, 16)); \
- le32p_replace_bits((__le32 *)(table) + 16, SET_CMC_TBL_MASK_CSI_FIX_RATE, \
- GENMASK(24, 16)); \
-} while (0)
+static inline void SET_CMC_TBL_CSI_FIX_RATE(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 8, val, GENMASK(24, 16));
+ le32p_replace_bits((__le32 *)(table) + 16, SET_CMC_TBL_MASK_CSI_FIX_RATE,
+ GENMASK(24, 16));
+}
#define SET_CMC_TBL_MASK_CSI_GI_LTF GENMASK(2, 0)
-#define SET_CMC_TBL_CSI_GI_LTF(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 8, val, GENMASK(27, 25)); \
- le32p_replace_bits((__le32 *)(table) + 16, SET_CMC_TBL_MASK_CSI_GI_LTF, \
- GENMASK(27, 25)); \
-} while (0)
+static inline void SET_CMC_TBL_CSI_GI_LTF(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 8, val, GENMASK(27, 25));
+ le32p_replace_bits((__le32 *)(table) + 16, SET_CMC_TBL_MASK_CSI_GI_LTF,
+ GENMASK(27, 25));
+}
#define SET_CMC_TBL_MASK_CSI_GID_SEL BIT(0)
-#define SET_CMC_TBL_CSI_GID_SEL(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 8, val, BIT(29)); \
- le32p_replace_bits((__le32 *)(table) + 16, SET_CMC_TBL_MASK_CSI_GID_SEL, \
- BIT(29)); \
-} while (0)
+static inline void SET_CMC_TBL_CSI_GID_SEL(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 8, val, BIT(29));
+ le32p_replace_bits((__le32 *)(table) + 16, SET_CMC_TBL_MASK_CSI_GID_SEL,
+ BIT(29));
+}
#define SET_CMC_TBL_MASK_CSI_BW GENMASK(1, 0)
-#define SET_CMC_TBL_CSI_BW(table, val) \
-do { \
- le32p_replace_bits((__le32 *)(table) + 8, val, GENMASK(31, 30)); \
- le32p_replace_bits((__le32 *)(table) + 16, SET_CMC_TBL_MASK_CSI_BW, \
- GENMASK(31, 30)); \
-} while (0)
+static inline void SET_CMC_TBL_CSI_BW(void *table, u32 val)
+{
+ le32p_replace_bits((__le32 *)(table) + 8, val, GENMASK(31, 30));
+ le32p_replace_bits((__le32 *)(table) + 16, SET_CMC_TBL_MASK_CSI_BW,
+ GENMASK(31, 30));
+}
-#define SET_FWROLE_MAINTAIN_MACID(h2c, val) \
- le32p_replace_bits((__le32 *)h2c, val, GENMASK(7, 0))
-#define SET_FWROLE_MAINTAIN_SELF_ROLE(h2c, val) \
- le32p_replace_bits((__le32 *)h2c, val, GENMASK(9, 8))
-#define SET_FWROLE_MAINTAIN_UPD_MODE(h2c, val) \
- le32p_replace_bits((__le32 *)h2c, val, GENMASK(12, 10))
-#define SET_FWROLE_MAINTAIN_WIFI_ROLE(h2c, val) \
- le32p_replace_bits((__le32 *)h2c, val, GENMASK(16, 13))
+static inline void SET_FWROLE_MAINTAIN_MACID(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)h2c, val, GENMASK(7, 0));
+}
-#define SET_JOININFO_MACID(h2c, val) \
- le32p_replace_bits((__le32 *)h2c, val, GENMASK(7, 0))
-#define SET_JOININFO_OP(h2c, val) \
- le32p_replace_bits((__le32 *)h2c, val, BIT(8))
-#define SET_JOININFO_BAND(h2c, val) \
- le32p_replace_bits((__le32 *)h2c, val, BIT(9))
-#define SET_JOININFO_WMM(h2c, val) \
- le32p_replace_bits((__le32 *)h2c, val, GENMASK(11, 10))
-#define SET_JOININFO_TGR(h2c, val) \
- le32p_replace_bits((__le32 *)h2c, val, BIT(12))
-#define SET_JOININFO_ISHESTA(h2c, val) \
- le32p_replace_bits((__le32 *)h2c, val, BIT(13))
-#define SET_JOININFO_DLBW(h2c, val) \
- le32p_replace_bits((__le32 *)h2c, val, GENMASK(15, 14))
-#define SET_JOININFO_TF_MAC_PAD(h2c, val) \
- le32p_replace_bits((__le32 *)h2c, val, GENMASK(17, 16))
-#define SET_JOININFO_DL_T_PE(h2c, val) \
- le32p_replace_bits((__le32 *)h2c, val, GENMASK(20, 18))
-#define SET_JOININFO_PORT_ID(h2c, val) \
- le32p_replace_bits((__le32 *)h2c, val, GENMASK(23, 21))
-#define SET_JOININFO_NET_TYPE(h2c, val) \
- le32p_replace_bits((__le32 *)h2c, val, GENMASK(25, 24))
-#define SET_JOININFO_WIFI_ROLE(h2c, val) \
- le32p_replace_bits((__le32 *)h2c, val, GENMASK(29, 26))
-#define SET_JOININFO_SELF_ROLE(h2c, val) \
- le32p_replace_bits((__le32 *)h2c, val, GENMASK(31, 30))
+static inline void SET_FWROLE_MAINTAIN_SELF_ROLE(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)h2c, val, GENMASK(9, 8));
+}
-#define SET_GENERAL_PKT_MACID(h2c, val) \
- le32p_replace_bits((__le32 *)h2c, val, GENMASK(7, 0))
-#define SET_GENERAL_PKT_PROBRSP_ID(h2c, val) \
- le32p_replace_bits((__le32 *)h2c, val, GENMASK(15, 8))
-#define SET_GENERAL_PKT_PSPOLL_ID(h2c, val) \
- le32p_replace_bits((__le32 *)h2c, val, GENMASK(23, 16))
-#define SET_GENERAL_PKT_NULL_ID(h2c, val) \
- le32p_replace_bits((__le32 *)h2c, val, GENMASK(31, 24))
-#define SET_GENERAL_PKT_QOS_NULL_ID(h2c, val) \
- le32p_replace_bits((__le32 *)(h2c) + 1, val, GENMASK(7, 0))
-#define SET_GENERAL_PKT_CTS2SELF_ID(h2c, val) \
- le32p_replace_bits((__le32 *)(h2c) + 1, val, GENMASK(15, 8))
+static inline void SET_FWROLE_MAINTAIN_UPD_MODE(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)h2c, val, GENMASK(12, 10));
+}
-#define SET_LOG_CFG_LEVEL(h2c, val) \
- le32p_replace_bits((__le32 *)h2c, val, GENMASK(7, 0))
-#define SET_LOG_CFG_PATH(h2c, val) \
- le32p_replace_bits((__le32 *)h2c, val, GENMASK(15, 8))
-#define SET_LOG_CFG_COMP(h2c, val) \
- le32p_replace_bits((__le32 *)(h2c) + 1, val, GENMASK(31, 0))
-#define SET_LOG_CFG_COMP_EXT(h2c, val) \
- le32p_replace_bits((__le32 *)(h2c) + 2, val, GENMASK(31, 0))
+static inline void SET_FWROLE_MAINTAIN_WIFI_ROLE(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)h2c, val, GENMASK(16, 13));
+}
-#define SET_BA_CAM_VALID(h2c, val) \
- le32p_replace_bits((__le32 *)h2c, val, BIT(0))
-#define SET_BA_CAM_INIT_REQ(h2c, val) \
- le32p_replace_bits((__le32 *)h2c, val, BIT(1))
-#define SET_BA_CAM_ENTRY_IDX(h2c, val) \
- le32p_replace_bits((__le32 *)h2c, val, GENMASK(3, 2))
-#define SET_BA_CAM_TID(h2c, val) \
- le32p_replace_bits((__le32 *)h2c, val, GENMASK(7, 4))
-#define SET_BA_CAM_MACID(h2c, val) \
- le32p_replace_bits((__le32 *)h2c, val, GENMASK(15, 8))
-#define SET_BA_CAM_BMAP_SIZE(h2c, val) \
- le32p_replace_bits((__le32 *)h2c, val, GENMASK(19, 16))
-#define SET_BA_CAM_SSN(h2c, val) \
- le32p_replace_bits((__le32 *)h2c, val, GENMASK(31, 20))
+static inline void SET_JOININFO_MACID(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)h2c, val, GENMASK(7, 0));
+}
-#define SET_LPS_PARM_MACID(h2c, val) \
- le32p_replace_bits((__le32 *)h2c, val, GENMASK(7, 0))
-#define SET_LPS_PARM_PSMODE(h2c, val) \
- le32p_replace_bits((__le32 *)h2c, val, GENMASK(15, 8))
-#define SET_LPS_PARM_RLBM(h2c, val) \
- le32p_replace_bits((__le32 *)h2c, val, GENMASK(19, 16))
-#define SET_LPS_PARM_SMARTPS(h2c, val) \
- le32p_replace_bits((__le32 *)h2c, val, GENMASK(23, 20))
-#define SET_LPS_PARM_AWAKEINTERVAL(h2c, val) \
- le32p_replace_bits((__le32 *)h2c, val, GENMASK(31, 24))
-#define SET_LPS_PARM_VOUAPSD(h2c, val) \
- le32p_replace_bits((__le32 *)(h2c) + 1, val, BIT(0))
-#define SET_LPS_PARM_VIUAPSD(h2c, val) \
- le32p_replace_bits((__le32 *)(h2c) + 1, val, BIT(1))
-#define SET_LPS_PARM_BEUAPSD(h2c, val) \
- le32p_replace_bits((__le32 *)(h2c) + 1, val, BIT(2))
-#define SET_LPS_PARM_BKUAPSD(h2c, val) \
- le32p_replace_bits((__le32 *)(h2c) + 1, val, BIT(3))
-#define SET_LPS_PARM_LASTRPWM(h2c, val) \
- le32p_replace_bits((__le32 *)(h2c) + 1, val, GENMASK(15, 8))
+static inline void SET_JOININFO_OP(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)h2c, val, BIT(8));
+}
+
+static inline void SET_JOININFO_BAND(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)h2c, val, BIT(9));
+}
+
+static inline void SET_JOININFO_WMM(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)h2c, val, GENMASK(11, 10));
+}
+
+static inline void SET_JOININFO_TGR(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)h2c, val, BIT(12));
+}
+
+static inline void SET_JOININFO_ISHESTA(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)h2c, val, BIT(13));
+}
+
+static inline void SET_JOININFO_DLBW(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)h2c, val, GENMASK(15, 14));
+}
+
+static inline void SET_JOININFO_TF_MAC_PAD(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)h2c, val, GENMASK(17, 16));
+}
+
+static inline void SET_JOININFO_DL_T_PE(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)h2c, val, GENMASK(20, 18));
+}
+
+static inline void SET_JOININFO_PORT_ID(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)h2c, val, GENMASK(23, 21));
+}
+
+static inline void SET_JOININFO_NET_TYPE(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)h2c, val, GENMASK(25, 24));
+}
+
+static inline void SET_JOININFO_WIFI_ROLE(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)h2c, val, GENMASK(29, 26));
+}
+
+static inline void SET_JOININFO_SELF_ROLE(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)h2c, val, GENMASK(31, 30));
+}
+
+static inline void SET_GENERAL_PKT_MACID(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)h2c, val, GENMASK(7, 0));
+}
+
+static inline void SET_GENERAL_PKT_PROBRSP_ID(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)h2c, val, GENMASK(15, 8));
+}
+
+static inline void SET_GENERAL_PKT_PSPOLL_ID(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)h2c, val, GENMASK(23, 16));
+}
+
+static inline void SET_GENERAL_PKT_NULL_ID(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)h2c, val, GENMASK(31, 24));
+}
+
+static inline void SET_GENERAL_PKT_QOS_NULL_ID(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)(h2c) + 1, val, GENMASK(7, 0));
+}
+
+static inline void SET_GENERAL_PKT_CTS2SELF_ID(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)(h2c) + 1, val, GENMASK(15, 8));
+}
+
+static inline void SET_LOG_CFG_LEVEL(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)h2c, val, GENMASK(7, 0));
+}
+
+static inline void SET_LOG_CFG_PATH(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)h2c, val, GENMASK(15, 8));
+}
+
+static inline void SET_LOG_CFG_COMP(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)(h2c) + 1, val, GENMASK(31, 0));
+}
+
+static inline void SET_LOG_CFG_COMP_EXT(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)(h2c) + 2, val, GENMASK(31, 0));
+}
+
+static inline void SET_BA_CAM_VALID(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)h2c, val, BIT(0));
+}
+
+static inline void SET_BA_CAM_INIT_REQ(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)h2c, val, BIT(1));
+}
+
+static inline void SET_BA_CAM_ENTRY_IDX(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)h2c, val, GENMASK(3, 2));
+}
+
+static inline void SET_BA_CAM_TID(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)h2c, val, GENMASK(7, 4));
+}
+
+static inline void SET_BA_CAM_MACID(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)h2c, val, GENMASK(15, 8));
+}
+
+static inline void SET_BA_CAM_BMAP_SIZE(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)h2c, val, GENMASK(19, 16));
+}
+
+static inline void SET_BA_CAM_SSN(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)h2c, val, GENMASK(31, 20));
+}
+
+static inline void SET_LPS_PARM_MACID(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)h2c, val, GENMASK(7, 0));
+}
+
+static inline void SET_LPS_PARM_PSMODE(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)h2c, val, GENMASK(15, 8));
+}
+
+static inline void SET_LPS_PARM_RLBM(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)h2c, val, GENMASK(19, 16));
+}
+
+static inline void SET_LPS_PARM_SMARTPS(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)h2c, val, GENMASK(23, 20));
+}
+
+static inline void SET_LPS_PARM_AWAKEINTERVAL(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)h2c, val, GENMASK(31, 24));
+}
+
+static inline void SET_LPS_PARM_VOUAPSD(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)(h2c) + 1, val, BIT(0));
+}
+
+static inline void SET_LPS_PARM_VIUAPSD(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)(h2c) + 1, val, BIT(1));
+}
+
+static inline void SET_LPS_PARM_BEUAPSD(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)(h2c) + 1, val, BIT(2));
+}
+
+static inline void SET_LPS_PARM_BKUAPSD(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)(h2c) + 1, val, BIT(3));
+}
+
+static inline void SET_LPS_PARM_LASTRPWM(void *h2c, u32 val)
+{
+ le32p_replace_bits((__le32 *)(h2c) + 1, val, GENMASK(15, 8));
+}
enum rtw89_btc_btf_h2c_class {
BTFC_SET = 0x10,
@@ -1054,165 +1316,322 @@ enum rtw89_btc_cxdrvinfo {
CXDRVINFO_MAX,
};
-#define RTW89_SET_FWCMD_CXHDR_TYPE(cmd, val) \
- u8p_replace_bits((u8 *)(cmd) + 0, val, GENMASK(7, 0))
-#define RTW89_SET_FWCMD_CXHDR_LEN(cmd, val) \
- u8p_replace_bits((u8 *)(cmd) + 1, val, GENMASK(7, 0))
+static inline void RTW89_SET_FWCMD_CXHDR_TYPE(void *cmd, u8 val)
+{
+ u8p_replace_bits((u8 *)(cmd) + 0, val, GENMASK(7, 0));
+}
-#define RTW89_SET_FWCMD_CXINIT_ANT_TYPE(cmd, val) \
- u8p_replace_bits((u8 *)(cmd) + 2, val, GENMASK(7, 0))
-#define RTW89_SET_FWCMD_CXINIT_ANT_NUM(cmd, val) \
- u8p_replace_bits((u8 *)(cmd) + 3, val, GENMASK(7, 0))
-#define RTW89_SET_FWCMD_CXINIT_ANT_ISO(cmd, val) \
- u8p_replace_bits((u8 *)(cmd) + 4, val, GENMASK(7, 0))
-#define RTW89_SET_FWCMD_CXINIT_ANT_POS(cmd, val) \
- u8p_replace_bits((u8 *)(cmd) + 5, val, BIT(0))
-#define RTW89_SET_FWCMD_CXINIT_ANT_DIVERSITY(cmd, val) \
- u8p_replace_bits((u8 *)(cmd) + 5, val, BIT(1))
-#define RTW89_SET_FWCMD_CXINIT_MOD_RFE(cmd, val) \
- u8p_replace_bits((u8 *)(cmd) + 6, val, GENMASK(7, 0))
-#define RTW89_SET_FWCMD_CXINIT_MOD_CV(cmd, val) \
- u8p_replace_bits((u8 *)(cmd) + 7, val, GENMASK(7, 0))
-#define RTW89_SET_FWCMD_CXINIT_MOD_BT_SOLO(cmd, val) \
- u8p_replace_bits((u8 *)(cmd) + 8, val, BIT(0))
-#define RTW89_SET_FWCMD_CXINIT_MOD_BT_POS(cmd, val) \
- u8p_replace_bits((u8 *)(cmd) + 8, val, BIT(1))
-#define RTW89_SET_FWCMD_CXINIT_MOD_SW_TYPE(cmd, val) \
- u8p_replace_bits((u8 *)(cmd) + 8, val, BIT(2))
-#define RTW89_SET_FWCMD_CXINIT_WL_GCH(cmd, val) \
- u8p_replace_bits((u8 *)(cmd) + 10, val, GENMASK(7, 0))
-#define RTW89_SET_FWCMD_CXINIT_WL_ONLY(cmd, val) \
- u8p_replace_bits((u8 *)(cmd) + 11, val, BIT(0))
-#define RTW89_SET_FWCMD_CXINIT_WL_INITOK(cmd, val) \
- u8p_replace_bits((u8 *)(cmd) + 11, val, BIT(1))
-#define RTW89_SET_FWCMD_CXINIT_DBCC_EN(cmd, val) \
- u8p_replace_bits((u8 *)(cmd) + 11, val, BIT(2))
-#define RTW89_SET_FWCMD_CXINIT_CX_OTHER(cmd, val) \
- u8p_replace_bits((u8 *)(cmd) + 11, val, BIT(3))
-#define RTW89_SET_FWCMD_CXINIT_BT_ONLY(cmd, val) \
- u8p_replace_bits((u8 *)(cmd) + 11, val, BIT(4))
+static inline void RTW89_SET_FWCMD_CXHDR_LEN(void *cmd, u8 val)
+{
+ u8p_replace_bits((u8 *)(cmd) + 1, val, GENMASK(7, 0));
+}
-#define RTW89_SET_FWCMD_CXROLE_CONNECT_CNT(cmd, val) \
- u8p_replace_bits((u8 *)(cmd) + 2, val, GENMASK(7, 0))
-#define RTW89_SET_FWCMD_CXROLE_LINK_MODE(cmd, val) \
- u8p_replace_bits((u8 *)(cmd) + 3, val, GENMASK(7, 0))
-#define RTW89_SET_FWCMD_CXROLE_ROLE_NONE(cmd, val) \
- le16p_replace_bits((__le16 *)((u8 *)(cmd) + 4), val, BIT(0))
-#define RTW89_SET_FWCMD_CXROLE_ROLE_STA(cmd, val) \
- le16p_replace_bits((__le16 *)((u8 *)(cmd) + 4), val, BIT(1))
-#define RTW89_SET_FWCMD_CXROLE_ROLE_AP(cmd, val) \
- le16p_replace_bits((__le16 *)((u8 *)(cmd) + 4), val, BIT(2))
-#define RTW89_SET_FWCMD_CXROLE_ROLE_VAP(cmd, val) \
- le16p_replace_bits((__le16 *)((u8 *)(cmd) + 4), val, BIT(3))
-#define RTW89_SET_FWCMD_CXROLE_ROLE_ADHOC(cmd, val) \
- le16p_replace_bits((__le16 *)((u8 *)(cmd) + 4), val, BIT(4))
-#define RTW89_SET_FWCMD_CXROLE_ROLE_ADHOC_MASTER(cmd, val) \
- le16p_replace_bits((__le16 *)((u8 *)(cmd) + 4), val, BIT(5))
-#define RTW89_SET_FWCMD_CXROLE_ROLE_MESH(cmd, val) \
- le16p_replace_bits((__le16 *)((u8 *)(cmd) + 4), val, BIT(6))
-#define RTW89_SET_FWCMD_CXROLE_ROLE_MONITOR(cmd, val) \
- le16p_replace_bits((__le16 *)((u8 *)(cmd) + 4), val, BIT(7))
-#define RTW89_SET_FWCMD_CXROLE_ROLE_P2P_DEV(cmd, val) \
- le16p_replace_bits((__le16 *)((u8 *)(cmd) + 4), val, BIT(8))
-#define RTW89_SET_FWCMD_CXROLE_ROLE_P2P_GC(cmd, val) \
- le16p_replace_bits((__le16 *)((u8 *)(cmd) + 4), val, BIT(9))
-#define RTW89_SET_FWCMD_CXROLE_ROLE_P2P_GO(cmd, val) \
- le16p_replace_bits((__le16 *)((u8 *)(cmd) + 4), val, BIT(10))
-#define RTW89_SET_FWCMD_CXROLE_ROLE_NAN(cmd, val) \
- le16p_replace_bits((__le16 *)((u8 *)(cmd) + 4), val, BIT(11))
-#define RTW89_SET_FWCMD_CXROLE_ACT_CONNECTED(cmd, val, n) \
- u8p_replace_bits((u8 *)(cmd) + (6 + 12 * (n)), val, BIT(0))
-#define RTW89_SET_FWCMD_CXROLE_ACT_PID(cmd, val, n) \
- u8p_replace_bits((u8 *)(cmd) + (6 + 12 * (n)), val, GENMASK(3, 1))
-#define RTW89_SET_FWCMD_CXROLE_ACT_PHY(cmd, val, n) \
- u8p_replace_bits((u8 *)(cmd) + (6 + 12 * (n)), val, BIT(4))
-#define RTW89_SET_FWCMD_CXROLE_ACT_NOA(cmd, val, n) \
- u8p_replace_bits((u8 *)(cmd) + (6 + 12 * (n)), val, BIT(5))
-#define RTW89_SET_FWCMD_CXROLE_ACT_BAND(cmd, val, n) \
- u8p_replace_bits((u8 *)(cmd) + (6 + 12 * (n)), val, GENMASK(7, 6))
-#define RTW89_SET_FWCMD_CXROLE_ACT_CLIENT_PS(cmd, val, n) \
- u8p_replace_bits((u8 *)(cmd) + (7 + 12 * (n)), val, BIT(0))
-#define RTW89_SET_FWCMD_CXROLE_ACT_BW(cmd, val, n) \
- u8p_replace_bits((u8 *)(cmd) + (7 + 12 * (n)), val, GENMASK(7, 1))
-#define RTW89_SET_FWCMD_CXROLE_ACT_ROLE(cmd, val, n) \
- u8p_replace_bits((u8 *)(cmd) + (8 + 12 * (n)), val, GENMASK(7, 0))
-#define RTW89_SET_FWCMD_CXROLE_ACT_CH(cmd, val, n) \
- u8p_replace_bits((u8 *)(cmd) + (9 + 12 * (n)), val, GENMASK(7, 0))
-#define RTW89_SET_FWCMD_CXROLE_ACT_TX_LVL(cmd, val, n) \
- le16p_replace_bits((__le16 *)((u8 *)(cmd) + (10 + 12 * (n))), val, GENMASK(15, 0))
-#define RTW89_SET_FWCMD_CXROLE_ACT_RX_LVL(cmd, val, n) \
- le16p_replace_bits((__le16 *)((u8 *)(cmd) + (12 + 12 * (n))), val, GENMASK(15, 0))
-#define RTW89_SET_FWCMD_CXROLE_ACT_TX_RATE(cmd, val, n) \
- le16p_replace_bits((__le16 *)((u8 *)(cmd) + (14 + 12 * (n))), val, GENMASK(15, 0))
-#define RTW89_SET_FWCMD_CXROLE_ACT_RX_RATE(cmd, val, n) \
- le16p_replace_bits((__le16 *)((u8 *)(cmd) + (16 + 12 * (n))), val, GENMASK(15, 0))
+static inline void RTW89_SET_FWCMD_CXINIT_ANT_TYPE(void *cmd, u8 val)
+{
+ u8p_replace_bits((u8 *)(cmd) + 2, val, GENMASK(7, 0));
+}
-#define RTW89_SET_FWCMD_CXCTRL_MANUAL(cmd, val) \
- le32p_replace_bits((__le32 *)((u8 *)(cmd) + 2), val, BIT(0))
-#define RTW89_SET_FWCMD_CXCTRL_IGNORE_BT(cmd, val) \
- le32p_replace_bits((__le32 *)((u8 *)(cmd) + 2), val, BIT(1))
-#define RTW89_SET_FWCMD_CXCTRL_ALWAYS_FREERUN(cmd, val) \
- le32p_replace_bits((__le32 *)((u8 *)(cmd) + 2), val, BIT(2))
-#define RTW89_SET_FWCMD_CXCTRL_TRACE_STEP(cmd, val) \
- le32p_replace_bits((__le32 *)((u8 *)(cmd) + 2), val, GENMASK(18, 3))
+static inline void RTW89_SET_FWCMD_CXINIT_ANT_NUM(void *cmd, u8 val)
+{
+ u8p_replace_bits((u8 *)(cmd) + 3, val, GENMASK(7, 0));
+}
-#define RTW89_SET_FWCMD_CXRFK_STATE(cmd, val) \
- le32p_replace_bits((__le32 *)((u8 *)(cmd) + 2), val, GENMASK(1, 0))
-#define RTW89_SET_FWCMD_CXRFK_PATH_MAP(cmd, val) \
- le32p_replace_bits((__le32 *)((u8 *)(cmd) + 2), val, GENMASK(5, 2))
-#define RTW89_SET_FWCMD_CXRFK_PHY_MAP(cmd, val) \
- le32p_replace_bits((__le32 *)((u8 *)(cmd) + 2), val, GENMASK(7, 6))
-#define RTW89_SET_FWCMD_CXRFK_BAND(cmd, val) \
- le32p_replace_bits((__le32 *)((u8 *)(cmd) + 2), val, GENMASK(9, 8))
-#define RTW89_SET_FWCMD_CXRFK_TYPE(cmd, val) \
- le32p_replace_bits((__le32 *)((u8 *)(cmd) + 2), val, GENMASK(17, 10))
+static inline void RTW89_SET_FWCMD_CXINIT_ANT_ISO(void *cmd, u8 val)
+{
+ u8p_replace_bits((u8 *)(cmd) + 4, val, GENMASK(7, 0));
+}
+
+static inline void RTW89_SET_FWCMD_CXINIT_ANT_POS(void *cmd, u8 val)
+{
+ u8p_replace_bits((u8 *)(cmd) + 5, val, BIT(0));
+}
+
+static inline void RTW89_SET_FWCMD_CXINIT_ANT_DIVERSITY(void *cmd, u8 val)
+{
+ u8p_replace_bits((u8 *)(cmd) + 5, val, BIT(1));
+}
+
+static inline void RTW89_SET_FWCMD_CXINIT_MOD_RFE(void *cmd, u8 val)
+{
+ u8p_replace_bits((u8 *)(cmd) + 6, val, GENMASK(7, 0));
+}
+
+static inline void RTW89_SET_FWCMD_CXINIT_MOD_CV(void *cmd, u8 val)
+{
+ u8p_replace_bits((u8 *)(cmd) + 7, val, GENMASK(7, 0));
+}
+
+static inline void RTW89_SET_FWCMD_CXINIT_MOD_BT_SOLO(void *cmd, u8 val)
+{
+ u8p_replace_bits((u8 *)(cmd) + 8, val, BIT(0));
+}
+
+static inline void RTW89_SET_FWCMD_CXINIT_MOD_BT_POS(void *cmd, u8 val)
+{
+ u8p_replace_bits((u8 *)(cmd) + 8, val, BIT(1));
+}
+
+static inline void RTW89_SET_FWCMD_CXINIT_MOD_SW_TYPE(void *cmd, u8 val)
+{
+ u8p_replace_bits((u8 *)(cmd) + 8, val, BIT(2));
+}
+
+static inline void RTW89_SET_FWCMD_CXINIT_WL_GCH(void *cmd, u8 val)
+{
+ u8p_replace_bits((u8 *)(cmd) + 10, val, GENMASK(7, 0));
+}
+
+static inline void RTW89_SET_FWCMD_CXINIT_WL_ONLY(void *cmd, u8 val)
+{
+ u8p_replace_bits((u8 *)(cmd) + 11, val, BIT(0));
+}
+
+static inline void RTW89_SET_FWCMD_CXINIT_WL_INITOK(void *cmd, u8 val)
+{
+ u8p_replace_bits((u8 *)(cmd) + 11, val, BIT(1));
+}
+
+static inline void RTW89_SET_FWCMD_CXINIT_DBCC_EN(void *cmd, u8 val)
+{
+ u8p_replace_bits((u8 *)(cmd) + 11, val, BIT(2));
+}
+
+static inline void RTW89_SET_FWCMD_CXINIT_CX_OTHER(void *cmd, u8 val)
+{
+ u8p_replace_bits((u8 *)(cmd) + 11, val, BIT(3));
+}
+
+static inline void RTW89_SET_FWCMD_CXINIT_BT_ONLY(void *cmd, u8 val)
+{
+ u8p_replace_bits((u8 *)(cmd) + 11, val, BIT(4));
+}
+
+static inline void RTW89_SET_FWCMD_CXROLE_CONNECT_CNT(void *cmd, u8 val)
+{
+ u8p_replace_bits((u8 *)(cmd) + 2, val, GENMASK(7, 0));
+}
+
+static inline void RTW89_SET_FWCMD_CXROLE_LINK_MODE(void *cmd, u8 val)
+{
+ u8p_replace_bits((u8 *)(cmd) + 3, val, GENMASK(7, 0));
+}
+
+static inline void RTW89_SET_FWCMD_CXROLE_ROLE_NONE(void *cmd, u16 val)
+{
+ le16p_replace_bits((__le16 *)((u8 *)(cmd) + 4), val, BIT(0));
+}
+
+static inline void RTW89_SET_FWCMD_CXROLE_ROLE_STA(void *cmd, u16 val)
+{
+ le16p_replace_bits((__le16 *)((u8 *)(cmd) + 4), val, BIT(1));
+}
+
+static inline void RTW89_SET_FWCMD_CXROLE_ROLE_AP(void *cmd, u16 val)
+{
+ le16p_replace_bits((__le16 *)((u8 *)(cmd) + 4), val, BIT(2));
+}
+
+static inline void RTW89_SET_FWCMD_CXROLE_ROLE_VAP(void *cmd, u16 val)
+{
+ le16p_replace_bits((__le16 *)((u8 *)(cmd) + 4), val, BIT(3));
+}
+
+static inline void RTW89_SET_FWCMD_CXROLE_ROLE_ADHOC(void *cmd, u16 val)
+{
+ le16p_replace_bits((__le16 *)((u8 *)(cmd) + 4), val, BIT(4));
+}
+
+static inline void RTW89_SET_FWCMD_CXROLE_ROLE_ADHOC_MASTER(void *cmd, u16 val)
+{
+ le16p_replace_bits((__le16 *)((u8 *)(cmd) + 4), val, BIT(5));
+}
+
+static inline void RTW89_SET_FWCMD_CXROLE_ROLE_MESH(void *cmd, u16 val)
+{
+ le16p_replace_bits((__le16 *)((u8 *)(cmd) + 4), val, BIT(6));
+}
+
+static inline void RTW89_SET_FWCMD_CXROLE_ROLE_MONITOR(void *cmd, u16 val)
+{
+ le16p_replace_bits((__le16 *)((u8 *)(cmd) + 4), val, BIT(7));
+}
+
+static inline void RTW89_SET_FWCMD_CXROLE_ROLE_P2P_DEV(void *cmd, u16 val)
+{
+ le16p_replace_bits((__le16 *)((u8 *)(cmd) + 4), val, BIT(8));
+}
+
+static inline void RTW89_SET_FWCMD_CXROLE_ROLE_P2P_GC(void *cmd, u16 val)
+{
+ le16p_replace_bits((__le16 *)((u8 *)(cmd) + 4), val, BIT(9));
+}
+
+static inline void RTW89_SET_FWCMD_CXROLE_ROLE_P2P_GO(void *cmd, u16 val)
+{
+ le16p_replace_bits((__le16 *)((u8 *)(cmd) + 4), val, BIT(10));
+}
+
+static inline void RTW89_SET_FWCMD_CXROLE_ROLE_NAN(void *cmd, u16 val)
+{
+ le16p_replace_bits((__le16 *)((u8 *)(cmd) + 4), val, BIT(11));
+}
+
+static inline void RTW89_SET_FWCMD_CXROLE_ACT_CONNECTED(void *cmd, u8 val, int n)
+{
+ u8p_replace_bits((u8 *)(cmd) + (6 + 12 * (n)), val, BIT(0));
+}
+
+static inline void RTW89_SET_FWCMD_CXROLE_ACT_PID(void *cmd, u8 val, int n)
+{
+ u8p_replace_bits((u8 *)(cmd) + (6 + 12 * (n)), val, GENMASK(3, 1));
+}
+
+static inline void RTW89_SET_FWCMD_CXROLE_ACT_PHY(void *cmd, u8 val, int n)
+{
+ u8p_replace_bits((u8 *)(cmd) + (6 + 12 * (n)), val, BIT(4));
+}
+
+static inline void RTW89_SET_FWCMD_CXROLE_ACT_NOA(void *cmd, u8 val, int n)
+{
+ u8p_replace_bits((u8 *)(cmd) + (6 + 12 * (n)), val, BIT(5));
+}
+
+static inline void RTW89_SET_FWCMD_CXROLE_ACT_BAND(void *cmd, u8 val, int n)
+{
+ u8p_replace_bits((u8 *)(cmd) + (6 + 12 * (n)), val, GENMASK(7, 6));
+}
+
+static inline void RTW89_SET_FWCMD_CXROLE_ACT_CLIENT_PS(void *cmd, u8 val, int n)
+{
+ u8p_replace_bits((u8 *)(cmd) + (7 + 12 * (n)), val, BIT(0));
+}
+
+static inline void RTW89_SET_FWCMD_CXROLE_ACT_BW(void *cmd, u8 val, int n)
+{
+ u8p_replace_bits((u8 *)(cmd) + (7 + 12 * (n)), val, GENMASK(7, 1));
+}
+
+static inline void RTW89_SET_FWCMD_CXROLE_ACT_ROLE(void *cmd, u8 val, int n)
+{
+ u8p_replace_bits((u8 *)(cmd) + (8 + 12 * (n)), val, GENMASK(7, 0));
+}
+
+static inline void RTW89_SET_FWCMD_CXROLE_ACT_CH(void *cmd, u8 val, int n)
+{
+ u8p_replace_bits((u8 *)(cmd) + (9 + 12 * (n)), val, GENMASK(7, 0));
+}
+
+static inline void RTW89_SET_FWCMD_CXROLE_ACT_TX_LVL(void *cmd, u16 val, int n)
+{
+ le16p_replace_bits((__le16 *)((u8 *)(cmd) + (10 + 12 * (n))), val, GENMASK(15, 0));
+}
+
+static inline void RTW89_SET_FWCMD_CXROLE_ACT_RX_LVL(void *cmd, u16 val, int n)
+{
+ le16p_replace_bits((__le16 *)((u8 *)(cmd) + (12 + 12 * (n))), val, GENMASK(15, 0));
+}
+
+static inline void RTW89_SET_FWCMD_CXROLE_ACT_TX_RATE(void *cmd, u16 val, int n)
+{
+ le16p_replace_bits((__le16 *)((u8 *)(cmd) + (14 + 12 * (n))), val, GENMASK(15, 0));
+}
+
+static inline void RTW89_SET_FWCMD_CXROLE_ACT_RX_RATE(void *cmd, u16 val, int n)
+{
+ le16p_replace_bits((__le16 *)((u8 *)(cmd) + (16 + 12 * (n))), val, GENMASK(15, 0));
+}
+
+static inline void RTW89_SET_FWCMD_CXCTRL_MANUAL(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)((u8 *)(cmd) + 2), val, BIT(0));
+}
+
+static inline void RTW89_SET_FWCMD_CXCTRL_IGNORE_BT(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)((u8 *)(cmd) + 2), val, BIT(1));
+}
+
+static inline void RTW89_SET_FWCMD_CXCTRL_ALWAYS_FREERUN(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)((u8 *)(cmd) + 2), val, BIT(2));
+}
+
+static inline void RTW89_SET_FWCMD_CXCTRL_TRACE_STEP(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)((u8 *)(cmd) + 2), val, GENMASK(18, 3));
+}
+
+static inline void RTW89_SET_FWCMD_CXRFK_STATE(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)((u8 *)(cmd) + 2), val, GENMASK(1, 0));
+}
+
+static inline void RTW89_SET_FWCMD_CXRFK_PATH_MAP(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)((u8 *)(cmd) + 2), val, GENMASK(5, 2));
+}
+
+static inline void RTW89_SET_FWCMD_CXRFK_PHY_MAP(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)((u8 *)(cmd) + 2), val, GENMASK(7, 6));
+}
+
+static inline void RTW89_SET_FWCMD_CXRFK_BAND(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)((u8 *)(cmd) + 2), val, GENMASK(9, 8));
+}
+
+static inline void RTW89_SET_FWCMD_CXRFK_TYPE(void *cmd, u32 val)
+{
+ le32p_replace_bits((__le32 *)((u8 *)(cmd) + 2), val, GENMASK(17, 10));
+}
#define RTW89_C2H_HEADER_LEN 8
#define RTW89_GET_C2H_CATEGORY(c2h) \
- le32_get_bits(*((__le32 *)c2h), GENMASK(1, 0))
+ le32_get_bits(*((const __le32 *)c2h), GENMASK(1, 0))
#define RTW89_GET_C2H_CLASS(c2h) \
- le32_get_bits(*((__le32 *)c2h), GENMASK(7, 2))
+ le32_get_bits(*((const __le32 *)c2h), GENMASK(7, 2))
#define RTW89_GET_C2H_FUNC(c2h) \
- le32_get_bits(*((__le32 *)c2h), GENMASK(15, 8))
+ le32_get_bits(*((const __le32 *)c2h), GENMASK(15, 8))
#define RTW89_GET_C2H_LEN(c2h) \
- le32_get_bits(*((__le32 *)(c2h) + 1), GENMASK(13, 0))
+ le32_get_bits(*((const __le32 *)(c2h) + 1), GENMASK(13, 0))
#define RTW89_GET_C2H_LOG_SRT_PRT(c2h) (char *)((__le32 *)(c2h) + 2)
#define RTW89_GET_C2H_LOG_LEN(len) ((len) - RTW89_C2H_HEADER_LEN)
#define RTW89_GET_MAC_C2H_DONE_ACK_CAT(c2h) \
- le32_get_bits(*((__le32 *)(c2h) + 2), GENMASK(1, 0))
+ le32_get_bits(*((const __le32 *)(c2h) + 2), GENMASK(1, 0))
#define RTW89_GET_MAC_C2H_DONE_ACK_CLASS(c2h) \
- le32_get_bits(*((__le32 *)(c2h) + 2), GENMASK(7, 2))
+ le32_get_bits(*((const __le32 *)(c2h) + 2), GENMASK(7, 2))
#define RTW89_GET_MAC_C2H_DONE_ACK_FUNC(c2h) \
- le32_get_bits(*((__le32 *)(c2h) + 2), GENMASK(15, 8))
+ le32_get_bits(*((const __le32 *)(c2h) + 2), GENMASK(15, 8))
#define RTW89_GET_MAC_C2H_DONE_ACK_H2C_RETURN(c2h) \
- le32_get_bits(*((__le32 *)(c2h) + 2), GENMASK(23, 16))
+ le32_get_bits(*((const __le32 *)(c2h) + 2), GENMASK(23, 16))
#define RTW89_GET_MAC_C2H_DONE_ACK_H2C_SEQ(c2h) \
- le32_get_bits(*((__le32 *)(c2h) + 2), GENMASK(31, 24))
+ le32_get_bits(*((const __le32 *)(c2h) + 2), GENMASK(31, 24))
#define RTW89_GET_MAC_C2H_REV_ACK_CAT(c2h) \
- le32_get_bits(*((__le32 *)(c2h) + 2), GENMASK(1, 0))
+ le32_get_bits(*((const __le32 *)(c2h) + 2), GENMASK(1, 0))
#define RTW89_GET_MAC_C2H_REV_ACK_CLASS(c2h) \
- le32_get_bits(*((__le32 *)(c2h) + 2), GENMASK(7, 2))
+ le32_get_bits(*((const __le32 *)(c2h) + 2), GENMASK(7, 2))
#define RTW89_GET_MAC_C2H_REV_ACK_FUNC(c2h) \
- le32_get_bits(*((__le32 *)(c2h) + 2), GENMASK(15, 8))
+ le32_get_bits(*((const __le32 *)(c2h) + 2), GENMASK(15, 8))
#define RTW89_GET_MAC_C2H_REV_ACK_H2C_SEQ(c2h) \
- le32_get_bits(*((__le32 *)(c2h) + 2), GENMASK(23, 16))
+ le32_get_bits(*((const __le32 *)(c2h) + 2), GENMASK(23, 16))
#define RTW89_GET_PHY_C2H_RA_RPT_MACID(c2h) \
- le32_get_bits(*((__le32 *)(c2h) + 2), GENMASK(15, 0))
+ le32_get_bits(*((const __le32 *)(c2h) + 2), GENMASK(15, 0))
#define RTW89_GET_PHY_C2H_RA_RPT_RETRY_RATIO(c2h) \
- le32_get_bits(*((__le32 *)(c2h) + 2), GENMASK(23, 16))
+ le32_get_bits(*((const __le32 *)(c2h) + 2), GENMASK(23, 16))
#define RTW89_GET_PHY_C2H_RA_RPT_MCSNSS(c2h) \
- le32_get_bits(*((__le32 *)(c2h) + 3), GENMASK(6, 0))
+ le32_get_bits(*((const __le32 *)(c2h) + 3), GENMASK(6, 0))
#define RTW89_GET_PHY_C2H_RA_RPT_MD_SEL(c2h) \
- le32_get_bits(*((__le32 *)(c2h) + 3), GENMASK(9, 8))
+ le32_get_bits(*((const __le32 *)(c2h) + 3), GENMASK(9, 8))
#define RTW89_GET_PHY_C2H_RA_RPT_GILTF(c2h) \
- le32_get_bits(*((__le32 *)(c2h) + 3), GENMASK(12, 10))
+ le32_get_bits(*((const __le32 *)(c2h) + 3), GENMASK(12, 10))
#define RTW89_GET_PHY_C2H_RA_RPT_BW(c2h) \
- le32_get_bits(*((__le32 *)(c2h) + 3), GENMASK(14, 13))
+ le32_get_bits(*((const __le32 *)(c2h) + 3), GENMASK(14, 13))
/* VHT, HE, HT-old: [6:4]: NSS, [3:0]: MCS
* HT-new: [6:5]: NA, [4:0]: MCS
@@ -1337,7 +1756,8 @@ int rtw89_fw_h2c_assoc_cmac_tbl(struct rtw89_dev *rtwdev,
struct ieee80211_sta *sta);
int rtw89_fw_h2c_txtime_cmac_tbl(struct rtw89_dev *rtwdev,
struct rtw89_sta *rtwsta);
-int rtw89_fw_h2c_cam(struct rtw89_dev *rtwdev, struct rtw89_vif *vif);
+int rtw89_fw_h2c_cam(struct rtw89_dev *rtwdev, struct rtw89_vif *vif,
+ struct rtw89_sta *rtwsta, const u8 *scan_mac_addr);
void rtw89_fw_c2h_irqsafe(struct rtw89_dev *rtwdev, struct sk_buff *c2h);
void rtw89_fw_c2h_work(struct work_struct *work);
int rtw89_fw_h2c_vif_maintain(struct rtw89_dev *rtwdev,
diff --git a/drivers/net/wireless/realtek/rtw89/mac.c b/drivers/net/wireless/realtek/rtw89/mac.c
index afcd07a..b98c47e 100644
--- a/drivers/net/wireless/realtek/rtw89/mac.c
+++ b/drivers/net/wireless/realtek/rtw89/mac.c
@@ -1093,7 +1093,6 @@ static int cmac_func_en(struct rtw89_dev *rtwdev, u8 mac_idx, bool en)
static int dmac_func_en(struct rtw89_dev *rtwdev)
{
u32 val32;
- u32 ret = 0;
val32 = (B_AX_MAC_FUNC_EN | B_AX_DMAC_FUNC_EN | B_AX_MAC_SEC_EN |
B_AX_DISPATCHER_EN | B_AX_DLE_CPUIO_EN | B_AX_PKT_IN_EN |
@@ -1107,7 +1106,7 @@ static int dmac_func_en(struct rtw89_dev *rtwdev)
B_AX_WD_RLS_CLK_EN);
rtw89_write32(rtwdev, R_AX_DMAC_CLK_EN, val32);
- return ret;
+ return 0;
}
static int chip_func_en(struct rtw89_dev *rtwdev)
@@ -2991,7 +2990,7 @@ int rtw89_mac_vif_init(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif)
if (ret)
return ret;
- ret = rtw89_fw_h2c_cam(rtwdev, rtwvif);
+ ret = rtw89_fw_h2c_cam(rtwdev, rtwvif, NULL, NULL);
if (ret)
return ret;
@@ -3012,7 +3011,7 @@ int rtw89_mac_vif_deinit(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif)
rtw89_cam_deinit(rtwdev, rtwvif);
- ret = rtw89_fw_h2c_cam(rtwdev, rtwvif);
+ ret = rtw89_fw_h2c_cam(rtwdev, rtwvif, NULL, NULL);
if (ret)
return ret;
@@ -3451,6 +3450,18 @@ bool rtw89_mac_get_ctrl_path(struct rtw89_dev *rtwdev)
return FIELD_GET(B_AX_LTE_MUX_CTRL_PATH >> 24, val);
}
+u16 rtw89_mac_get_plt_cnt(struct rtw89_dev *rtwdev, u8 band)
+{
+ u32 reg;
+ u16 cnt;
+
+ reg = rtw89_mac_reg_by_idx(R_AX_BT_PLT, band);
+ cnt = rtw89_read32_mask(rtwdev, reg, B_AX_BT_PLT_PKT_CNT_MASK);
+ rtw89_write16_set(rtwdev, reg, B_AX_BT_PLT_RST);
+
+ return cnt;
+}
+
static void rtw89_mac_bfee_ctrl(struct rtw89_dev *rtwdev, u8 mac_idx, bool en)
{
u32 reg;
@@ -3695,7 +3706,7 @@ void _rtw89_mac_bf_monitor_track(struct rtw89_dev *rtwdev)
{
struct rtw89_traffic_stats *stats = &rtwdev->stats;
struct rtw89_vif *rtwvif;
- bool en = stats->tx_tfc_lv > stats->rx_tfc_lv ? false : true;
+ bool en = stats->tx_tfc_lv <= stats->rx_tfc_lv;
bool old = test_bit(RTW89_FLAG_BFEE_EN, rtwdev->flags);
if (en == old)
diff --git a/drivers/net/wireless/realtek/rtw89/mac.h b/drivers/net/wireless/realtek/rtw89/mac.h
index 6f3db8a..b7d13ed 100644
--- a/drivers/net/wireless/realtek/rtw89/mac.h
+++ b/drivers/net/wireless/realtek/rtw89/mac.h
@@ -227,6 +227,7 @@ enum rtw89_mac_dbg_port_sel {
/* SRAM mem dump */
#define R_AX_INDIR_ACCESS_ENTRY 0x40000
+#define AXIDMA_BASE_ADDR 0x18006000
#define STA_SCHED_BASE_ADDR 0x18808000
#define RXPLD_FLTR_CAM_BASE_ADDR 0x18813000
#define SECURITY_CAM_BASE_ADDR 0x18814000
@@ -240,10 +241,15 @@ enum rtw89_mac_dbg_port_sel {
#define DMAC_TBL_BASE_ADDR 0x18800000
#define SHCUT_MACHDR_BASE_ADDR 0x18800800
#define BCN_IE_CAM1_BASE_ADDR 0x188A0000
+#define TXD_FIFO_0_BASE_ADDR 0x18856200
+#define TXD_FIFO_1_BASE_ADDR 0x188A1080
+#define TXDATA_FIFO_0_BASE_ADDR 0x18856000
+#define TXDATA_FIFO_1_BASE_ADDR 0x188A1000
#define CCTL_INFO_SIZE 32
enum rtw89_mac_mem_sel {
+ RTW89_MAC_MEM_AXIDMA,
RTW89_MAC_MEM_SHARED_BUF,
RTW89_MAC_MEM_DMAC_TBL,
RTW89_MAC_MEM_SHCUT_MACHDR,
@@ -256,6 +262,10 @@ enum rtw89_mac_mem_sel {
RTW89_MAC_MEM_BA_CAM,
RTW89_MAC_MEM_BCN_IE_CAM0,
RTW89_MAC_MEM_BCN_IE_CAM1,
+ RTW89_MAC_MEM_TXD_FIFO_0,
+ RTW89_MAC_MEM_TXD_FIFO_1,
+ RTW89_MAC_MEM_TXDATA_FIFO_0,
+ RTW89_MAC_MEM_TXDATA_FIFO_1,
/* keep last */
RTW89_MAC_MEM_LAST,
@@ -778,6 +788,7 @@ int rtw89_mac_coex_init(struct rtw89_dev *rtwdev, const struct rtw89_mac_ax_coex
int rtw89_mac_cfg_gnt(struct rtw89_dev *rtwdev,
const struct rtw89_mac_ax_coex_gnt *gnt_cfg);
int rtw89_mac_cfg_plt(struct rtw89_dev *rtwdev, struct rtw89_mac_ax_plt *plt);
+u16 rtw89_mac_get_plt_cnt(struct rtw89_dev *rtwdev, u8 band);
void rtw89_mac_cfg_sb(struct rtw89_dev *rtwdev, u32 val);
u32 rtw89_mac_get_sb(struct rtw89_dev *rtwdev);
bool rtw89_mac_get_ctrl_path(struct rtw89_dev *rtwdev);
diff --git a/drivers/net/wireless/realtek/rtw89/mac80211.c b/drivers/net/wireless/realtek/rtw89/mac80211.c
index 16dc6fb..a322259 100644
--- a/drivers/net/wireless/realtek/rtw89/mac80211.c
+++ b/drivers/net/wireless/realtek/rtw89/mac80211.c
@@ -27,6 +27,7 @@ static void rtw89_ops_tx(struct ieee80211_hw *hw,
if (ret) {
rtw89_err(rtwdev, "failed to transmit skb: %d\n", ret);
ieee80211_free_txskb(hw, skb);
+ return;
}
rtw89_core_tx_kick_off(rtwdev, qsel);
}
@@ -336,7 +337,7 @@ static void rtw89_ops_bss_info_changed(struct ieee80211_hw *hw,
if (changed & BSS_CHANGED_BSSID) {
ether_addr_copy(rtwvif->bssid, conf->bssid);
rtw89_cam_bssid_changed(rtwdev, rtwvif);
- rtw89_fw_h2c_cam(rtwdev, rtwvif);
+ rtw89_fw_h2c_cam(rtwdev, rtwvif, NULL, NULL);
}
if (changed & BSS_CHANGED_ERP_SLOT)
@@ -615,6 +616,7 @@ static void rtw89_ops_sw_scan_start(struct ieee80211_hw *hw,
const u8 *mac_addr)
{
struct rtw89_dev *rtwdev = hw->priv;
+ struct rtw89_vif *rtwvif = (struct rtw89_vif *)vif->drv_priv;
struct rtw89_hal *hal = &rtwdev->hal;
mutex_lock(&rtwdev->mutex);
@@ -623,6 +625,7 @@ static void rtw89_ops_sw_scan_start(struct ieee80211_hw *hw,
rtw89_btc_ntfy_scan_start(rtwdev, RTW89_PHY_0, hal->current_band_type);
rtw89_chip_rfk_scan(rtwdev, true);
rtw89_hci_recalc_int_mit(rtwdev);
+ rtw89_fw_h2c_cam(rtwdev, rtwvif, NULL, mac_addr);
mutex_unlock(&rtwdev->mutex);
}
@@ -630,8 +633,10 @@ static void rtw89_ops_sw_scan_complete(struct ieee80211_hw *hw,
struct ieee80211_vif *vif)
{
struct rtw89_dev *rtwdev = hw->priv;
+ struct rtw89_vif *rtwvif = (struct rtw89_vif *)vif->drv_priv;
mutex_lock(&rtwdev->mutex);
+ rtw89_fw_h2c_cam(rtwdev, rtwvif, NULL, NULL);
rtw89_chip_rfk_scan(rtwdev, false);
rtw89_btc_ntfy_scan_finish(rtwdev, RTW89_PHY_0);
rtwdev->scanning = false;
diff --git a/drivers/net/wireless/realtek/rtw89/phy.c b/drivers/net/wireless/realtek/rtw89/phy.c
index ab13485..1470098 100644
--- a/drivers/net/wireless/realtek/rtw89/phy.c
+++ b/drivers/net/wireless/realtek/rtw89/phy.c
@@ -654,6 +654,12 @@ rtw89_phy_cofig_rf_reg_store(struct rtw89_dev *rtwdev,
u16 idx = info->curr_idx % RTW89_H2C_RF_PAGE_SIZE;
u8 page = info->curr_idx / RTW89_H2C_RF_PAGE_SIZE;
+ if (page >= RTW89_H2C_RF_PAGE_NUM) {
+ rtw89_warn(rtwdev, "RF parameters exceed size. path=%d, idx=%d",
+ rf_path, info->curr_idx);
+ return;
+ }
+
info->rtw89_phy_config_rf_h2c[page][idx] =
cpu_to_le32((reg->addr << 20) | reg->data);
info->curr_idx++;
@@ -662,30 +668,29 @@ rtw89_phy_cofig_rf_reg_store(struct rtw89_dev *rtwdev,
static int rtw89_phy_config_rf_reg_fw(struct rtw89_dev *rtwdev,
struct rtw89_fw_h2c_rf_reg_info *info)
{
- u16 page = info->curr_idx / RTW89_H2C_RF_PAGE_SIZE;
- u16 len = (info->curr_idx % RTW89_H2C_RF_PAGE_SIZE) * 4;
+ u16 remain = info->curr_idx;
+ u16 len = 0;
u8 i;
int ret = 0;
- if (page > RTW89_H2C_RF_PAGE_NUM) {
+ if (remain > RTW89_H2C_RF_PAGE_NUM * RTW89_H2C_RF_PAGE_SIZE) {
rtw89_warn(rtwdev,
- "rf reg h2c total page num %d larger than %d (RTW89_H2C_RF_PAGE_NUM)\n",
- page, RTW89_H2C_RF_PAGE_NUM);
- return -EINVAL;
+ "rf reg h2c total len %d larger than %d\n",
+ remain, RTW89_H2C_RF_PAGE_NUM * RTW89_H2C_RF_PAGE_SIZE);
+ ret = -EINVAL;
+ goto out;
}
- for (i = 0; i < page; i++) {
- ret = rtw89_fw_h2c_rf_reg(rtwdev, info,
- RTW89_H2C_RF_PAGE_SIZE * 4, i);
+ for (i = 0; i < RTW89_H2C_RF_PAGE_NUM && remain; i++, remain -= len) {
+ len = remain > RTW89_H2C_RF_PAGE_SIZE ? RTW89_H2C_RF_PAGE_SIZE : remain;
+ ret = rtw89_fw_h2c_rf_reg(rtwdev, info, len * 4, i);
if (ret)
- return ret;
+ goto out;
}
- ret = rtw89_fw_h2c_rf_reg(rtwdev, info, len, i);
- if (ret)
- return ret;
+out:
info->curr_idx = 0;
- return 0;
+ return ret;
}
static void rtw89_phy_config_rf_reg(struct rtw89_dev *rtwdev,
@@ -1099,9 +1104,15 @@ s8 rtw89_phy_read_txpwr_limit(struct rtw89_dev *rtwdev,
switch (band) {
case RTW89_BAND_2G:
lmt = (*chip->txpwr_lmt_2g)[bw][ntx][rs][bf][regd][ch_idx];
+ if (!lmt)
+ lmt = (*chip->txpwr_lmt_2g)[bw][ntx][rs][bf]
+ [RTW89_WW][ch_idx];
break;
case RTW89_BAND_5G:
lmt = (*chip->txpwr_lmt_5g)[bw][ntx][rs][bf][regd][ch_idx];
+ if (!lmt)
+ lmt = (*chip->txpwr_lmt_5g)[bw][ntx][rs][bf]
+ [RTW89_WW][ch_idx];
break;
default:
rtw89_warn(rtwdev, "unknown band type: %d\n", band);
@@ -1224,9 +1235,15 @@ static s8 rtw89_phy_read_txpwr_limit_ru(struct rtw89_dev *rtwdev,
switch (band) {
case RTW89_BAND_2G:
lmt_ru = (*chip->txpwr_lmt_ru_2g)[ru][ntx][regd][ch_idx];
+ if (!lmt_ru)
+ lmt_ru = (*chip->txpwr_lmt_ru_2g)[ru][ntx]
+ [RTW89_WW][ch_idx];
break;
case RTW89_BAND_5G:
lmt_ru = (*chip->txpwr_lmt_ru_5g)[ru][ntx][regd][ch_idx];
+ if (!lmt_ru)
+ lmt_ru = (*chip->txpwr_lmt_ru_5g)[ru][ntx]
+ [RTW89_WW][ch_idx];
break;
default:
rtw89_warn(rtwdev, "unknown band type: %d\n", band);
@@ -1767,7 +1784,7 @@ static void rtw89_phy_cfo_dm(struct rtw89_dev *rtwdev)
}
rtw89_phy_cfo_crystal_cap_adjust(rtwdev, new_cfo);
cfo->cfo_avg_pre = new_cfo;
- x_cap_update = cfo->crystal_cap == pre_x_cap ? false : true;
+ x_cap_update = cfo->crystal_cap != pre_x_cap;
rtw89_debug(rtwdev, RTW89_DBG_CFO, "Xcap_up=%d\n", x_cap_update);
rtw89_debug(rtwdev, RTW89_DBG_CFO, "Xcap: D:%x C:%x->%x, ofst=%d\n",
cfo->def_x_cap, pre_x_cap, cfo->crystal_cap,
@@ -2404,6 +2421,116 @@ void rtw89_phy_env_monitor_track(struct rtw89_dev *rtwdev)
env->ccx_watchdog_result, chk_result);
}
+static bool rtw89_physts_ie_page_valid(enum rtw89_phy_status_bitmap *ie_page)
+{
+ if (*ie_page > RTW89_PHYSTS_BITMAP_NUM ||
+ *ie_page == RTW89_RSVD_9)
+ return false;
+ else if (*ie_page > RTW89_RSVD_9)
+ *ie_page -= 1;
+
+ return true;
+}
+
+static u32 rtw89_phy_get_ie_bitmap_addr(enum rtw89_phy_status_bitmap ie_page)
+{
+ static const u8 ie_page_shift = 2;
+
+ return R_PHY_STS_BITMAP_ADDR_START + (ie_page << ie_page_shift);
+}
+
+static u32 rtw89_physts_get_ie_bitmap(struct rtw89_dev *rtwdev,
+ enum rtw89_phy_status_bitmap ie_page)
+{
+ u32 addr;
+
+ if (!rtw89_physts_ie_page_valid(&ie_page))
+ return 0;
+
+ addr = rtw89_phy_get_ie_bitmap_addr(ie_page);
+
+ return rtw89_phy_read32(rtwdev, addr);
+}
+
+static void rtw89_physts_set_ie_bitmap(struct rtw89_dev *rtwdev,
+ enum rtw89_phy_status_bitmap ie_page,
+ u32 val)
+{
+ const struct rtw89_chip_info *chip = rtwdev->chip;
+ u32 addr;
+
+ if (!rtw89_physts_ie_page_valid(&ie_page))
+ return;
+
+ if (chip->chip_id == RTL8852A)
+ val &= B_PHY_STS_BITMAP_MSK_52A;
+
+ addr = rtw89_phy_get_ie_bitmap_addr(ie_page);
+ rtw89_phy_write32(rtwdev, addr, val);
+}
+
+static void rtw89_physts_enable_ie_bitmap(struct rtw89_dev *rtwdev,
+ enum rtw89_phy_status_bitmap bitmap,
+ enum rtw89_phy_status_ie_type ie,
+ bool enable)
+{
+ u32 val = rtw89_physts_get_ie_bitmap(rtwdev, bitmap);
+
+ if (enable)
+ val |= BIT(ie);
+ else
+ val &= ~BIT(ie);
+
+ rtw89_physts_set_ie_bitmap(rtwdev, bitmap, val);
+}
+
+static void rtw89_physts_enable_fail_report(struct rtw89_dev *rtwdev,
+ bool enable,
+ enum rtw89_phy_idx phy_idx)
+{
+ if (enable) {
+ rtw89_phy_write32_clr(rtwdev, R_PLCP_HISTOGRAM,
+ B_STS_DIS_TRIG_BY_FAIL);
+ rtw89_phy_write32_clr(rtwdev, R_PLCP_HISTOGRAM,
+ B_STS_DIS_TRIG_BY_BRK);
+ } else {
+ rtw89_phy_write32_set(rtwdev, R_PLCP_HISTOGRAM,
+ B_STS_DIS_TRIG_BY_FAIL);
+ rtw89_phy_write32_set(rtwdev, R_PLCP_HISTOGRAM,
+ B_STS_DIS_TRIG_BY_BRK);
+ }
+}
+
+static void rtw89_physts_parsing_init(struct rtw89_dev *rtwdev)
+{
+ const struct rtw89_chip_info *chip = rtwdev->chip;
+ u8 i;
+
+ if (chip->chip_id == RTL8852A && rtwdev->hal.cv == CHIP_CBV)
+ rtw89_physts_enable_fail_report(rtwdev, false, RTW89_PHY_0);
+
+ for (i = 0; i < RTW89_PHYSTS_BITMAP_NUM; i++) {
+ if (i >= RTW89_CCK_PKT)
+ rtw89_physts_enable_ie_bitmap(rtwdev, i,
+ RTW89_PHYSTS_IE09_FTR_0,
+ true);
+ if ((i >= RTW89_CCK_BRK && i <= RTW89_VHT_MU) ||
+ (i >= RTW89_RSVD_9 && i <= RTW89_CCK_PKT))
+ continue;
+ rtw89_physts_enable_ie_bitmap(rtwdev, i,
+ RTW89_PHYSTS_IE24_OFDM_TD_PATH_A,
+ true);
+ }
+ rtw89_physts_enable_ie_bitmap(rtwdev, RTW89_VHT_PKT,
+ RTW89_PHYSTS_IE13_DL_MU_DEF, true);
+ rtw89_physts_enable_ie_bitmap(rtwdev, RTW89_HE_PKT,
+ RTW89_PHYSTS_IE13_DL_MU_DEF, true);
+
+ /* force IE01 for channel index, only channel field is valid */
+ rtw89_physts_enable_ie_bitmap(rtwdev, RTW89_CCK_PKT,
+ RTW89_PHYSTS_IE01_CMN_OFDM, true);
+}
+
static void rtw89_phy_dig_read_gain_table(struct rtw89_dev *rtwdev, int type)
{
const struct rtw89_chip_info *chip = rtwdev->chip;
@@ -2839,6 +2966,7 @@ void rtw89_phy_dm_init(struct rtw89_dev *rtwdev)
rtw89_chip_bb_sethw(rtwdev);
rtw89_phy_env_monitor_init(rtwdev);
+ rtw89_physts_parsing_init(rtwdev);
rtw89_phy_dig_init(rtwdev);
rtw89_phy_cfo_init(rtwdev);
diff --git a/drivers/net/wireless/realtek/rtw89/phy.h b/drivers/net/wireless/realtek/rtw89/phy.h
index 3701293..b1f059b 100644
--- a/drivers/net/wireless/realtek/rtw89/phy.h
+++ b/drivers/net/wireless/realtek/rtw89/phy.h
@@ -134,6 +134,66 @@ enum rtw89_ccx_unit {
RTW89_CCX_32_US = 3
};
+enum rtw89_phy_status_ie_type {
+ RTW89_PHYSTS_IE00_CMN_CCK = 0,
+ RTW89_PHYSTS_IE01_CMN_OFDM = 1,
+ RTW89_PHYSTS_IE02_CMN_EXT_AX = 2,
+ RTW89_PHYSTS_IE03_CMN_EXT_SEG_1 = 3,
+ RTW89_PHYSTS_IE04_CMN_EXT_PATH_A = 4,
+ RTW89_PHYSTS_IE05_CMN_EXT_PATH_B = 5,
+ RTW89_PHYSTS_IE06_CMN_EXT_PATH_C = 6,
+ RTW89_PHYSTS_IE07_CMN_EXT_PATH_D = 7,
+ RTW89_PHYSTS_IE08_FTR_CH = 8,
+ RTW89_PHYSTS_IE09_FTR_0 = 9,
+ RTW89_PHYSTS_IE10_FTR_PLCP_EXT = 10,
+ RTW89_PHYSTS_IE11_FTR_PLCP_HISTOGRAM = 11,
+ RTW89_PHYSTS_IE12_MU_EIGEN_INFO = 12,
+ RTW89_PHYSTS_IE13_DL_MU_DEF = 13,
+ RTW89_PHYSTS_IE14_TB_UL_CQI = 14,
+ RTW89_PHYSTS_IE15_TB_UL_DEF = 15,
+ RTW89_PHYSTS_IE16_RSVD16 = 16,
+ RTW89_PHYSTS_IE17_TB_UL_CTRL = 17,
+ RTW89_PHYSTS_IE18_DBG_OFDM_FD_CMN = 18,
+ RTW89_PHYSTS_IE19_DBG_OFDM_TD_CMN = 19,
+ RTW89_PHYSTS_IE20_DBG_OFDM_FD_USER_SEG_0 = 20,
+ RTW89_PHYSTS_IE21_DBG_OFDM_FD_USER_SEG_1 = 21,
+ RTW89_PHYSTS_IE22_DBG_OFDM_FD_USER_AGC = 22,
+ RTW89_PHYSTS_IE23_RSVD23 = 23,
+ RTW89_PHYSTS_IE24_OFDM_TD_PATH_A = 24,
+ RTW89_PHYSTS_IE25_OFDM_TD_PATH_B = 25,
+ RTW89_PHYSTS_IE26_OFDM_TD_PATH_C = 26,
+ RTW89_PHYSTS_IE27_OFDM_TD_PATH_D = 27,
+ RTW89_PHYSTS_IE28_DBG_CCK_PATH_A = 28,
+ RTW89_PHYSTS_IE29_DBG_CCK_PATH_B = 29,
+ RTW89_PHYSTS_IE30_DBG_CCK_PATH_C = 30,
+ RTW89_PHYSTS_IE31_DBG_CCK_PATH_D = 31,
+
+ /* keep last */
+ RTW89_PHYSTS_IE_NUM,
+ RTW89_PHYSTS_IE_MAX = RTW89_PHYSTS_IE_NUM - 1
+};
+
+enum rtw89_phy_status_bitmap {
+ RTW89_TD_SEARCH_FAIL = 0,
+ RTW89_BRK_BY_TX_PKT = 1,
+ RTW89_CCA_SPOOF = 2,
+ RTW89_OFDM_BRK = 3,
+ RTW89_CCK_BRK = 4,
+ RTW89_DL_MU_SPOOFING = 5,
+ RTW89_HE_MU = 6,
+ RTW89_VHT_MU = 7,
+ RTW89_UL_TB_SPOOFING = 8,
+ RTW89_RSVD_9 = 9,
+ RTW89_TRIG_BASE_PPDU = 10,
+ RTW89_CCK_PKT = 11,
+ RTW89_LEGACY_OFDM_PKT = 12,
+ RTW89_HT_PKT = 13,
+ RTW89_VHT_PKT = 14,
+ RTW89_HE_PKT = 15,
+
+ RTW89_PHYSTS_BITMAP_NUM
+};
+
enum rtw89_dig_gain_type {
RTW89_DIG_GAIN_LNA_G = 0,
RTW89_DIG_GAIN_TIA_G = 1,
diff --git a/drivers/net/wireless/realtek/rtw89/reg.h b/drivers/net/wireless/realtek/rtw89/reg.h
index 365d8c8..e0a416d 100644
--- a/drivers/net/wireless/realtek/rtw89/reg.h
+++ b/drivers/net/wireless/realtek/rtw89/reg.h
@@ -1658,7 +1658,7 @@
#define R_RSTB_WATCH_DOG 0x000C
#define B_P0_RSTB_WATCH_DOG BIT(0)
#define B_P1_RSTB_WATCH_DOG BIT(1)
-#define B_UPD_P0_EN BIT(30)
+#define B_UPD_P0_EN BIT(31)
#define R_ANAPAR_PW15 0x030C
#define B_ANAPAR_PW15 GENMASK(31, 24)
#define B_ANAPAR_PW15_H GENMASK(27, 24)
@@ -1674,6 +1674,29 @@
#define B_UPD_CLK_ADC_VAL GENMASK(26, 25)
#define R_RSTB_ASYNC 0x0704
#define B_RSTB_ASYNC_ALL BIT(1)
+#define R_MAC_PIN_SEL 0x0734
+#define B_CH_IDX_SEG0 GENMASK(23, 16)
+#define R_PLCP_HISTOGRAM 0x0738
+#define B_STS_DIS_TRIG_BY_BRK BIT(2)
+#define B_STS_DIS_TRIG_BY_FAIL BIT(3)
+#define R_PHY_STS_BITMAP_ADDR_START R_PHY_STS_BITMAP_SEARCH_FAIL
+#define B_PHY_STS_BITMAP_ADDR_MASK GENMASK(6, 2)
+#define R_PHY_STS_BITMAP_SEARCH_FAIL 0x073C
+#define B_PHY_STS_BITMAP_MSK_52A 0x337cff3f
+#define R_PHY_STS_BITMAP_R2T 0x0740
+#define R_PHY_STS_BITMAP_CCA_SPOOF 0x0744
+#define R_PHY_STS_BITMAP_OFDM_BRK 0x0748
+#define R_PHY_STS_BITMAP_CCK_BRK 0x074C
+#define R_PHY_STS_BITMAP_DL_MU_SPOOF 0x0750
+#define R_PHY_STS_BITMAP_HE_MU 0x0754
+#define R_PHY_STS_BITMAP_VHT_MU 0x0758
+#define R_PHY_STS_BITMAP_UL_TB_SPOOF 0x075C
+#define R_PHY_STS_BITMAP_TRIGBASE 0x0760
+#define R_PHY_STS_BITMAP_CCK 0x0764
+#define R_PHY_STS_BITMAP_LEGACY 0x0768
+#define R_PHY_STS_BITMAP_HT 0x076C
+#define R_PHY_STS_BITMAP_VHT 0x0770
+#define R_PHY_STS_BITMAP_HE 0x0774
#define R_PMAC_GNT 0x0980
#define B_PMAC_GNT_TXEN BIT(0)
#define B_PMAC_GNT_RXEN BIT(16)
diff --git a/drivers/net/wireless/realtek/rtw89/regd.c b/drivers/net/wireless/realtek/rtw89/regd.c
index f00b94e..4c37e59 100644
--- a/drivers/net/wireless/realtek/rtw89/regd.c
+++ b/drivers/net/wireless/realtek/rtw89/regd.c
@@ -15,243 +15,244 @@ static const struct rtw89_regulatory rtw89_ww_regd =
COUNTRY_REGD("00", RTW89_WW, RTW89_WW);
static const struct rtw89_regulatory rtw89_regd_map[] = {
- COUNTRY_REGD("AR", RTW89_FCC, RTW89_FCC),
- COUNTRY_REGD("BO", RTW89_WW, RTW89_FCC),
+ COUNTRY_REGD("AR", RTW89_MEXICO, RTW89_MEXICO),
+ COUNTRY_REGD("BO", RTW89_FCC, RTW89_FCC),
COUNTRY_REGD("BR", RTW89_FCC, RTW89_FCC),
- COUNTRY_REGD("CL", RTW89_WW, RTW89_CHILE),
+ COUNTRY_REGD("CL", RTW89_CHILE, RTW89_CHILE),
COUNTRY_REGD("CO", RTW89_FCC, RTW89_FCC),
COUNTRY_REGD("CR", RTW89_FCC, RTW89_FCC),
COUNTRY_REGD("EC", RTW89_FCC, RTW89_FCC),
- COUNTRY_REGD("SV", RTW89_WW, RTW89_FCC),
+ COUNTRY_REGD("SV", RTW89_FCC, RTW89_FCC),
COUNTRY_REGD("GT", RTW89_FCC, RTW89_FCC),
- COUNTRY_REGD("HN", RTW89_WW, RTW89_FCC),
- COUNTRY_REGD("MX", RTW89_FCC, RTW89_MEXICO),
+ COUNTRY_REGD("HN", RTW89_FCC, RTW89_FCC),
+ COUNTRY_REGD("MX", RTW89_MEXICO, RTW89_MEXICO),
COUNTRY_REGD("NI", RTW89_FCC, RTW89_FCC),
COUNTRY_REGD("PA", RTW89_FCC, RTW89_FCC),
COUNTRY_REGD("PY", RTW89_FCC, RTW89_FCC),
COUNTRY_REGD("PE", RTW89_FCC, RTW89_FCC),
COUNTRY_REGD("US", RTW89_FCC, RTW89_FCC),
- COUNTRY_REGD("UY", RTW89_WW, RTW89_FCC),
- COUNTRY_REGD("VE", RTW89_WW, RTW89_FCC),
+ COUNTRY_REGD("UY", RTW89_FCC, RTW89_FCC),
+ COUNTRY_REGD("VE", RTW89_FCC, RTW89_FCC),
COUNTRY_REGD("PR", RTW89_FCC, RTW89_FCC),
COUNTRY_REGD("DO", RTW89_FCC, RTW89_FCC),
- COUNTRY_REGD("AT", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("BE", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("CY", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("CZ", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("DK", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("EE", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("FI", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("FR", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("DE", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("GR", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("HU", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("IS", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("IE", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("IT", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("LV", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("LI", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("LT", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("LU", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("MT", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("MC", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("NL", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("NO", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("PL", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("PT", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("SK", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("SI", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("ES", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("SE", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("CH", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("GB", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("AL", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("AZ", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("BH", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("BA", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("BG", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("HR", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("EG", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("GH", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("IQ", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("IL", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("JO", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("KZ", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("KE", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("KW", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("KG", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("LB", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("LS", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("MK", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("MA", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("MZ", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("NA", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("NG", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("OM", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("QA", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("RO", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("RU", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("SA", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("SN", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("RS", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("ME", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("ZA", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("TR", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("UA", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("AE", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("YE", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("ZW", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("BD", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("KH", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("CN", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("HK", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("IN", RTW89_WW, RTW89_ETSI),
+ COUNTRY_REGD("AT", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("BE", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("CY", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("CZ", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("DK", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("EE", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("FI", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("FR", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("DE", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("GR", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("HU", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("IS", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("IE", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("IT", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("LV", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("LI", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("LT", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("LU", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("MT", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("MC", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("NL", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("NO", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("PL", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("PT", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("SK", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("SI", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("ES", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("SE", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("CH", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("GB", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("AL", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("AZ", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("BH", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("BA", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("BG", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("HR", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("EG", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("GH", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("IQ", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("IL", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("JO", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("KZ", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("KE", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("KW", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("KG", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("LB", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("LS", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("MK", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("MA", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("MZ", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("NA", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("NG", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("OM", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("QA", RTW89_QATAR, RTW89_QATAR),
+ COUNTRY_REGD("RO", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("RU", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("SA", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("SN", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("RS", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("ME", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("ZA", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("TR", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("UA", RTW89_UKRAINE, RTW89_UKRAINE),
+ COUNTRY_REGD("AE", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("YE", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("ZW", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("BD", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("KH", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("CN", RTW89_CN, RTW89_CN),
+ COUNTRY_REGD("HK", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("IN", RTW89_ETSI, RTW89_ETSI),
COUNTRY_REGD("ID", RTW89_ETSI, RTW89_ETSI),
COUNTRY_REGD("KR", RTW89_KCC, RTW89_KCC),
- COUNTRY_REGD("MY", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("PK", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("PH", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("SG", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("LK", RTW89_WW, RTW89_ETSI),
+ COUNTRY_REGD("MY", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("PK", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("PH", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("SG", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("LK", RTW89_ETSI, RTW89_ETSI),
COUNTRY_REGD("TW", RTW89_FCC, RTW89_FCC),
- COUNTRY_REGD("TH", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("VN", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("AU", RTW89_WW, RTW89_ACMA),
- COUNTRY_REGD("NZ", RTW89_WW, RTW89_ACMA),
- COUNTRY_REGD("PG", RTW89_WW, RTW89_ETSI),
+ COUNTRY_REGD("TH", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("VN", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("AU", RTW89_ACMA, RTW89_ACMA),
+ COUNTRY_REGD("NZ", RTW89_ACMA, RTW89_ACMA),
+ COUNTRY_REGD("PG", RTW89_ETSI, RTW89_ETSI),
COUNTRY_REGD("CA", RTW89_IC, RTW89_IC),
COUNTRY_REGD("JP", RTW89_MKK, RTW89_MKK),
- COUNTRY_REGD("JM", RTW89_WW, RTW89_FCC),
+ COUNTRY_REGD("JM", RTW89_FCC, RTW89_FCC),
COUNTRY_REGD("AN", RTW89_FCC, RTW89_FCC),
COUNTRY_REGD("TT", RTW89_FCC, RTW89_FCC),
- COUNTRY_REGD("TN", RTW89_WW, RTW89_ETSI),
+ COUNTRY_REGD("TN", RTW89_ETSI, RTW89_ETSI),
COUNTRY_REGD("AF", RTW89_ETSI, RTW89_ETSI),
- COUNTRY_REGD("DZ", RTW89_WW, RTW89_ETSI),
+ COUNTRY_REGD("DZ", RTW89_ETSI, RTW89_ETSI),
COUNTRY_REGD("AS", RTW89_FCC, RTW89_FCC),
- COUNTRY_REGD("AD", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("AO", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("AI", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("AQ", RTW89_WW, RTW89_ETSI),
+ COUNTRY_REGD("AD", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("AO", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("AI", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("AQ", RTW89_ETSI, RTW89_ETSI),
COUNTRY_REGD("AG", RTW89_FCC, RTW89_FCC),
- COUNTRY_REGD("AM", RTW89_WW, RTW89_ETSI),
+ COUNTRY_REGD("AM", RTW89_ETSI, RTW89_ETSI),
COUNTRY_REGD("AW", RTW89_FCC, RTW89_FCC),
COUNTRY_REGD("BS", RTW89_FCC, RTW89_FCC),
COUNTRY_REGD("BB", RTW89_FCC, RTW89_FCC),
- COUNTRY_REGD("BY", RTW89_WW, RTW89_ETSI),
+ COUNTRY_REGD("BY", RTW89_ETSI, RTW89_ETSI),
COUNTRY_REGD("BZ", RTW89_FCC, RTW89_FCC),
- COUNTRY_REGD("BJ", RTW89_WW, RTW89_ETSI),
+ COUNTRY_REGD("BJ", RTW89_ETSI, RTW89_ETSI),
COUNTRY_REGD("BM", RTW89_FCC, RTW89_FCC),
- COUNTRY_REGD("BT", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("BW", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("BV", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("IO", RTW89_WW, RTW89_ETSI),
+ COUNTRY_REGD("BT", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("BW", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("BV", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("IO", RTW89_ETSI, RTW89_ETSI),
COUNTRY_REGD("VG", RTW89_FCC, RTW89_FCC),
- COUNTRY_REGD("BN", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("BF", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("MM", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("BI", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("CM", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("CV", RTW89_WW, RTW89_ETSI),
+ COUNTRY_REGD("BN", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("BF", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("MM", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("BI", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("CM", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("CV", RTW89_ETSI, RTW89_ETSI),
COUNTRY_REGD("KY", RTW89_FCC, RTW89_FCC),
- COUNTRY_REGD("CF", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("TD", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("CX", RTW89_WW, RTW89_ACMA),
- COUNTRY_REGD("CC", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("KM", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("CG", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("CD", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("CK", RTW89_WW, RTW89_ETSI),
+ COUNTRY_REGD("CF", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("TD", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("CX", RTW89_ACMA, RTW89_ACMA),
+ COUNTRY_REGD("CC", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("KM", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("CG", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("CD", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("CK", RTW89_ETSI, RTW89_ETSI),
COUNTRY_REGD("CI", RTW89_ETSI, RTW89_ETSI),
- COUNTRY_REGD("DJ", RTW89_WW, RTW89_ETSI),
+ COUNTRY_REGD("DJ", RTW89_ETSI, RTW89_ETSI),
COUNTRY_REGD("DM", RTW89_FCC, RTW89_FCC),
- COUNTRY_REGD("GQ", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("ER", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("ET", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("FK", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("FO", RTW89_WW, RTW89_ETSI),
+ COUNTRY_REGD("GQ", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("ER", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("ET", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("FK", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("FO", RTW89_ETSI, RTW89_ETSI),
COUNTRY_REGD("FJ", RTW89_FCC, RTW89_FCC),
- COUNTRY_REGD("GF", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("PF", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("TF", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("GA", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("GM", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("GE", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("GI", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("GL", RTW89_WW, RTW89_ETSI),
+ COUNTRY_REGD("GF", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("PF", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("TF", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("GA", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("GM", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("GE", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("GI", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("GL", RTW89_ETSI, RTW89_ETSI),
COUNTRY_REGD("GD", RTW89_FCC, RTW89_FCC),
- COUNTRY_REGD("GP", RTW89_WW, RTW89_ETSI),
+ COUNTRY_REGD("GP", RTW89_ETSI, RTW89_ETSI),
COUNTRY_REGD("GU", RTW89_FCC, RTW89_FCC),
- COUNTRY_REGD("GG", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("GN", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("GW", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("GY", RTW89_FCC, RTW89_NCC),
+ COUNTRY_REGD("GG", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("GN", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("GW", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("GY", RTW89_NCC, RTW89_NCC),
COUNTRY_REGD("HT", RTW89_FCC, RTW89_FCC),
- COUNTRY_REGD("HM", RTW89_WW, RTW89_ACMA),
- COUNTRY_REGD("VA", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("IM", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("JE", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("KI", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("LA", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("LR", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("LY", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("MO", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("MG", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("MW", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("MV", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("ML", RTW89_WW, RTW89_ETSI),
+ COUNTRY_REGD("HM", RTW89_ACMA, RTW89_ACMA),
+ COUNTRY_REGD("VA", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("IM", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("JE", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("KI", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("LA", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("LR", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("LY", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("MO", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("MG", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("MW", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("MV", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("ML", RTW89_ETSI, RTW89_ETSI),
COUNTRY_REGD("MH", RTW89_FCC, RTW89_FCC),
- COUNTRY_REGD("MQ", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("MR", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("MU", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("YT", RTW89_WW, RTW89_ETSI),
+ COUNTRY_REGD("MQ", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("MR", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("MU", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("YT", RTW89_ETSI, RTW89_ETSI),
COUNTRY_REGD("FM", RTW89_FCC, RTW89_FCC),
- COUNTRY_REGD("MD", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("MN", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("MS", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("NR", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("NP", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("NC", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("NE", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("NU", RTW89_WW, RTW89_ACMA),
- COUNTRY_REGD("NF", RTW89_WW, RTW89_ACMA),
+ COUNTRY_REGD("MD", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("MN", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("MS", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("NR", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("NP", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("NC", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("NE", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("NU", RTW89_ACMA, RTW89_ACMA),
+ COUNTRY_REGD("NF", RTW89_ACMA, RTW89_ACMA),
COUNTRY_REGD("MP", RTW89_FCC, RTW89_FCC),
COUNTRY_REGD("PW", RTW89_FCC, RTW89_FCC),
- COUNTRY_REGD("RE", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("RW", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("SH", RTW89_WW, RTW89_ETSI),
+ COUNTRY_REGD("RE", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("RW", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("SH", RTW89_ETSI, RTW89_ETSI),
COUNTRY_REGD("KN", RTW89_FCC, RTW89_FCC),
COUNTRY_REGD("LC", RTW89_FCC, RTW89_FCC),
COUNTRY_REGD("MF", RTW89_FCC, RTW89_FCC),
COUNTRY_REGD("SX", RTW89_FCC, RTW89_FCC),
- COUNTRY_REGD("PM", RTW89_WW, RTW89_ETSI),
+ COUNTRY_REGD("PM", RTW89_ETSI, RTW89_ETSI),
COUNTRY_REGD("VC", RTW89_FCC, RTW89_FCC),
COUNTRY_REGD("WS", RTW89_FCC, RTW89_FCC),
- COUNTRY_REGD("SM", RTW89_WW, RTW89_ETSI),
+ COUNTRY_REGD("SM", RTW89_ETSI, RTW89_ETSI),
COUNTRY_REGD("ST", RTW89_FCC, RTW89_FCC),
COUNTRY_REGD("SC", RTW89_FCC, RTW89_FCC),
- COUNTRY_REGD("SL", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("SB", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("SO", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("GS", RTW89_WW, RTW89_ETSI),
+ COUNTRY_REGD("SL", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("SB", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("SO", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("GS", RTW89_ETSI, RTW89_ETSI),
COUNTRY_REGD("SR", RTW89_FCC, RTW89_FCC),
- COUNTRY_REGD("SJ", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("SZ", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("TJ", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("TZ", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("TG", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("TK", RTW89_WW, RTW89_ACMA),
- COUNTRY_REGD("TO", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("TM", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("TC", RTW89_WW, RTW89_ETSI),
+ COUNTRY_REGD("SJ", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("SZ", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("TJ", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("TZ", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("TG", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("TK", RTW89_ACMA, RTW89_ACMA),
+ COUNTRY_REGD("TO", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("TM", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("TC", RTW89_ETSI, RTW89_ETSI),
COUNTRY_REGD("TV", RTW89_ETSI, RTW89_NA),
- COUNTRY_REGD("UG", RTW89_WW, RTW89_ETSI),
+ COUNTRY_REGD("UG", RTW89_ETSI, RTW89_ETSI),
COUNTRY_REGD("VI", RTW89_FCC, RTW89_FCC),
- COUNTRY_REGD("UZ", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("VU", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("WF", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("EH", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("ZM", RTW89_WW, RTW89_ETSI),
- COUNTRY_REGD("IR", RTW89_WW, RTW89_ETSI),
+ COUNTRY_REGD("UZ", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("VU", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("WF", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("EH", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("ZM", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("IR", RTW89_ETSI, RTW89_ETSI),
+ COUNTRY_REGD("PS", RTW89_ETSI, RTW89_ETSI),
};
static const struct rtw89_regulatory *rtw89_regd_find_reg_by_name(char *alpha2)
diff --git a/drivers/net/wireless/realtek/rtw89/rtw8852a.c b/drivers/net/wireless/realtek/rtw89/rtw8852a.c
index 5c6ffca..6b75e4b 100644
--- a/drivers/net/wireless/realtek/rtw89/rtw8852a.c
+++ b/drivers/net/wireless/realtek/rtw89/rtw8852a.c
@@ -1053,10 +1053,10 @@ static void rtw8852a_set_channel_bb(struct rtw89_dev *rtwdev,
struct rtw89_channel_params *param,
enum rtw89_phy_idx phy_idx)
{
- bool cck_en = param->center_chan > 14 ? false : true;
+ bool cck_en = param->center_chan <= 14;
u8 pri_ch_idx = param->pri_ch_idx;
- if (param->center_chan <= 14)
+ if (cck_en)
rtw8852a_ctrl_sco_cck(rtwdev, param->center_chan,
param->primary_chan, param->bandwidth);
@@ -1069,6 +1069,8 @@ static void rtw8852a_set_channel_bb(struct rtw89_dev *rtwdev,
rtw8852a_bbrst_for_rfk(rtwdev, phy_idx);
}
rtw8852a_spur_elimination(rtwdev, param->center_chan);
+ rtw89_phy_write32_mask(rtwdev, R_MAC_PIN_SEL, B_CH_IDX_SEG0,
+ param->primary_chan);
rtw8852a_bb_reset_all(rtwdev, phy_idx);
}
@@ -1927,6 +1929,21 @@ void rtw8852a_btc_wl_s1_standby(struct rtw89_dev *rtwdev, bool state)
rtw89_write_rf(rtwdev, RF_PATH_B, RR_LUTWE, RFREG_MASK, 0x0);
}
+static void rtw8852a_fill_freq_with_ppdu(struct rtw89_dev *rtwdev,
+ struct rtw89_rx_phy_ppdu *phy_ppdu,
+ struct ieee80211_rx_status *status)
+{
+ u16 chan = phy_ppdu->chan_idx;
+ u8 band;
+
+ if (chan == 0)
+ return;
+
+ band = chan <= 14 ? NL80211_BAND_2GHZ : NL80211_BAND_5GHZ;
+ status->freq = ieee80211_channel_to_frequency(chan, band);
+ status->band = band;
+}
+
static void rtw8852a_query_ppdu(struct rtw89_dev *rtwdev,
struct rtw89_rx_phy_ppdu *phy_ppdu,
struct ieee80211_rx_status *status)
@@ -1939,6 +1956,8 @@ static void rtw8852a_query_ppdu(struct rtw89_dev *rtwdev,
status->chains |= BIT(path);
status->chain_signal[path] = rx_power[path];
}
+ if (phy_ppdu->valid)
+ rtw8852a_fill_freq_with_ppdu(rtwdev, phy_ppdu, status);
}
static const struct rtw89_chip_ops rtw8852a_chip_ops = {
@@ -2012,7 +2031,7 @@ const struct rtw89_chip_info rtw8852a_chip_info = {
.limit_efuse_size = 1152,
.phycap_addr = 0x580,
.phycap_size = 128,
- .para_ver = 0x05050764,
+ .para_ver = 0x05050864,
.wlcx_desired = 0x05050000,
.btcx_desired = 0x5,
.scbd = 0x1,
diff --git a/drivers/net/wireless/realtek/rtw89/rtw8852a_table.c b/drivers/net/wireless/realtek/rtw89/rtw8852a_table.c
index 3a4fe72..253b5f8 100644
--- a/drivers/net/wireless/realtek/rtw89/rtw8852a_table.c
+++ b/drivers/net/wireless/realtek/rtw89/rtw8852a_table.c
@@ -43384,5248 +43384,6991 @@ static const u8 _txpwr_track_delta_swingidx_2g_cck_a_p[] = {
const s8 rtw89_8852a_txpwr_lmt_2g[RTW89_2G_BW_NUM][RTW89_NTX_NUM]
[RTW89_RS_LMT_NUM][RTW89_BF_NUM]
[RTW89_REGD_NUM][RTW89_2G_CH_NUM] = {
- [0][0][0][0][0][0] = 56,
- [0][0][0][0][0][1] = 56,
- [0][0][0][0][0][2] = 56,
- [0][0][0][0][0][3] = 56,
- [0][0][0][0][0][4] = 56,
- [0][0][0][0][0][5] = 56,
- [0][0][0][0][0][6] = 56,
- [0][0][0][0][0][7] = 56,
- [0][0][0][0][0][8] = 56,
- [0][0][0][0][0][9] = 56,
- [0][0][0][0][0][10] = 56,
- [0][0][0][0][0][11] = 56,
- [0][0][0][0][0][12] = 48,
- [0][0][0][0][0][13] = 76,
- [0][1][0][0][0][0] = 44,
- [0][1][0][0][0][1] = 44,
- [0][1][0][0][0][2] = 44,
- [0][1][0][0][0][3] = 44,
- [0][1][0][0][0][4] = 44,
- [0][1][0][0][0][5] = 44,
- [0][1][0][0][0][6] = 44,
- [0][1][0][0][0][7] = 44,
- [0][1][0][0][0][8] = 44,
- [0][1][0][0][0][9] = 44,
- [0][1][0][0][0][10] = 44,
- [0][1][0][0][0][11] = 44,
- [0][1][0][0][0][12] = 38,
- [0][1][0][0][0][13] = 64,
- [1][0][0][0][0][0] = 0,
- [1][0][0][0][0][1] = 0,
- [1][0][0][0][0][2] = 58,
- [1][0][0][0][0][3] = 58,
- [1][0][0][0][0][4] = 58,
- [1][0][0][0][0][5] = 58,
- [1][0][0][0][0][6] = 46,
- [1][0][0][0][0][7] = 46,
- [1][0][0][0][0][8] = 46,
- [1][0][0][0][0][9] = 32,
- [1][0][0][0][0][10] = 32,
- [1][0][0][0][0][11] = 0,
- [1][0][0][0][0][12] = 0,
- [1][0][0][0][0][13] = 0,
- [1][1][0][0][0][0] = 0,
- [1][1][0][0][0][1] = 0,
- [1][1][0][0][0][2] = 46,
- [1][1][0][0][0][3] = 46,
- [1][1][0][0][0][4] = 46,
- [1][1][0][0][0][5] = 46,
- [1][1][0][0][0][6] = 46,
- [1][1][0][0][0][7] = 46,
- [1][1][0][0][0][8] = 46,
- [1][1][0][0][0][9] = 24,
- [1][1][0][0][0][10] = 24,
- [1][1][0][0][0][11] = 0,
- [1][1][0][0][0][12] = 0,
- [1][1][0][0][0][13] = 0,
- [0][0][1][0][0][0] = 58,
- [0][0][1][0][0][1] = 58,
- [0][0][1][0][0][2] = 58,
- [0][0][1][0][0][3] = 58,
- [0][0][1][0][0][4] = 58,
- [0][0][1][0][0][5] = 58,
- [0][0][1][0][0][6] = 58,
- [0][0][1][0][0][7] = 58,
- [0][0][1][0][0][8] = 58,
- [0][0][1][0][0][9] = 58,
- [0][0][1][0][0][10] = 58,
- [0][0][1][0][0][11] = 56,
- [0][0][1][0][0][12] = 52,
- [0][0][1][0][0][13] = 0,
- [0][1][1][0][0][0] = 46,
- [0][1][1][0][0][1] = 46,
- [0][1][1][0][0][2] = 46,
- [0][1][1][0][0][3] = 46,
- [0][1][1][0][0][4] = 46,
- [0][1][1][0][0][5] = 46,
- [0][1][1][0][0][6] = 46,
- [0][1][1][0][0][7] = 46,
- [0][1][1][0][0][8] = 46,
- [0][1][1][0][0][9] = 46,
- [0][1][1][0][0][10] = 46,
- [0][1][1][0][0][11] = 42,
- [0][1][1][0][0][12] = 40,
- [0][1][1][0][0][13] = 0,
- [0][0][2][0][0][0] = 58,
- [0][0][2][0][0][1] = 58,
- [0][0][2][0][0][2] = 58,
- [0][0][2][0][0][3] = 58,
- [0][0][2][0][0][4] = 58,
- [0][0][2][0][0][5] = 58,
- [0][0][2][0][0][6] = 58,
- [0][0][2][0][0][7] = 58,
- [0][0][2][0][0][8] = 58,
- [0][0][2][0][0][9] = 58,
- [0][0][2][0][0][10] = 58,
- [0][0][2][0][0][11] = 54,
- [0][0][2][0][0][12] = 50,
- [0][0][2][0][0][13] = 0,
- [0][1][2][0][0][0] = 46,
- [0][1][2][0][0][1] = 46,
- [0][1][2][0][0][2] = 46,
- [0][1][2][0][0][3] = 46,
- [0][1][2][0][0][4] = 46,
- [0][1][2][0][0][5] = 46,
- [0][1][2][0][0][6] = 46,
- [0][1][2][0][0][7] = 46,
- [0][1][2][0][0][8] = 46,
- [0][1][2][0][0][9] = 46,
- [0][1][2][0][0][10] = 46,
- [0][1][2][0][0][11] = 42,
- [0][1][2][0][0][12] = 40,
- [0][1][2][0][0][13] = 0,
- [0][1][2][1][0][0] = 34,
- [0][1][2][1][0][1] = 34,
- [0][1][2][1][0][2] = 34,
- [0][1][2][1][0][3] = 34,
- [0][1][2][1][0][4] = 34,
- [0][1][2][1][0][5] = 34,
- [0][1][2][1][0][6] = 34,
- [0][1][2][1][0][7] = 34,
- [0][1][2][1][0][8] = 34,
- [0][1][2][1][0][9] = 34,
- [0][1][2][1][0][10] = 34,
- [0][1][2][1][0][11] = 34,
- [0][1][2][1][0][12] = 34,
- [0][1][2][1][0][13] = 0,
- [1][0][2][0][0][0] = 0,
- [1][0][2][0][0][1] = 0,
- [1][0][2][0][0][2] = 56,
- [1][0][2][0][0][3] = 56,
- [1][0][2][0][0][4] = 58,
- [1][0][2][0][0][5] = 58,
- [1][0][2][0][0][6] = 54,
- [1][0][2][0][0][7] = 50,
- [1][0][2][0][0][8] = 50,
- [1][0][2][0][0][9] = 42,
- [1][0][2][0][0][10] = 40,
- [1][0][2][0][0][11] = 0,
- [1][0][2][0][0][12] = 0,
- [1][0][2][0][0][13] = 0,
- [1][1][2][0][0][0] = 0,
- [1][1][2][0][0][1] = 0,
- [1][1][2][0][0][2] = 46,
- [1][1][2][0][0][3] = 46,
- [1][1][2][0][0][4] = 46,
- [1][1][2][0][0][5] = 46,
- [1][1][2][0][0][6] = 46,
- [1][1][2][0][0][7] = 46,
- [1][1][2][0][0][8] = 46,
- [1][1][2][0][0][9] = 38,
- [1][1][2][0][0][10] = 36,
- [1][1][2][0][0][11] = 0,
- [1][1][2][0][0][12] = 0,
- [1][1][2][0][0][13] = 0,
- [1][1][2][1][0][0] = 0,
- [1][1][2][1][0][1] = 0,
- [1][1][2][1][0][2] = 34,
- [1][1][2][1][0][3] = 34,
- [1][1][2][1][0][4] = 34,
- [1][1][2][1][0][5] = 34,
- [1][1][2][1][0][6] = 34,
- [1][1][2][1][0][7] = 34,
- [1][1][2][1][0][8] = 34,
- [1][1][2][1][0][9] = 34,
- [1][1][2][1][0][10] = 34,
- [1][1][2][1][0][11] = 0,
- [1][1][2][1][0][12] = 0,
- [1][1][2][1][0][13] = 0,
- [0][0][0][0][2][0] = 76,
- [0][0][0][0][1][0] = 56,
- [0][0][0][0][3][0] = 68,
- [0][0][0][0][5][0] = 76,
- [0][0][0][0][6][0] = 56,
- [0][0][0][0][9][0] = 56,
- [0][0][0][0][8][0] = 60,
- [0][0][0][0][11][0] = 56,
- [0][0][0][0][2][1] = 76,
- [0][0][0][0][1][1] = 56,
- [0][0][0][0][3][1] = 68,
- [0][0][0][0][5][1] = 76,
- [0][0][0][0][6][1] = 56,
- [0][0][0][0][9][1] = 56,
- [0][0][0][0][8][1] = 60,
- [0][0][0][0][11][1] = 56,
- [0][0][0][0][2][2] = 76,
- [0][0][0][0][1][2] = 56,
- [0][0][0][0][3][2] = 68,
- [0][0][0][0][5][2] = 76,
- [0][0][0][0][6][2] = 56,
- [0][0][0][0][9][2] = 56,
- [0][0][0][0][8][2] = 60,
- [0][0][0][0][11][2] = 56,
- [0][0][0][0][2][3] = 76,
- [0][0][0][0][1][3] = 56,
- [0][0][0][0][3][3] = 68,
- [0][0][0][0][5][3] = 76,
- [0][0][0][0][6][3] = 56,
- [0][0][0][0][9][3] = 56,
- [0][0][0][0][8][3] = 60,
- [0][0][0][0][11][3] = 56,
- [0][0][0][0][2][4] = 76,
- [0][0][0][0][1][4] = 56,
- [0][0][0][0][3][4] = 68,
- [0][0][0][0][5][4] = 76,
- [0][0][0][0][6][4] = 56,
- [0][0][0][0][9][4] = 56,
- [0][0][0][0][8][4] = 60,
- [0][0][0][0][11][4] = 56,
- [0][0][0][0][2][5] = 76,
- [0][0][0][0][1][5] = 56,
- [0][0][0][0][3][5] = 68,
- [0][0][0][0][5][5] = 76,
- [0][0][0][0][6][5] = 56,
- [0][0][0][0][9][5] = 56,
- [0][0][0][0][8][5] = 60,
- [0][0][0][0][11][5] = 56,
- [0][0][0][0][2][6] = 76,
- [0][0][0][0][1][6] = 56,
- [0][0][0][0][3][6] = 68,
- [0][0][0][0][5][6] = 76,
- [0][0][0][0][6][6] = 56,
- [0][0][0][0][9][6] = 56,
- [0][0][0][0][8][6] = 60,
- [0][0][0][0][11][6] = 56,
- [0][0][0][0][2][7] = 76,
- [0][0][0][0][1][7] = 56,
- [0][0][0][0][3][7] = 68,
- [0][0][0][0][5][7] = 76,
- [0][0][0][0][6][7] = 56,
- [0][0][0][0][9][7] = 56,
- [0][0][0][0][8][7] = 60,
- [0][0][0][0][11][7] = 56,
- [0][0][0][0][2][8] = 76,
- [0][0][0][0][1][8] = 56,
- [0][0][0][0][3][8] = 68,
- [0][0][0][0][5][8] = 76,
- [0][0][0][0][6][8] = 56,
- [0][0][0][0][9][8] = 56,
- [0][0][0][0][8][8] = 60,
- [0][0][0][0][11][8] = 56,
- [0][0][0][0][2][9] = 76,
- [0][0][0][0][1][9] = 56,
- [0][0][0][0][3][9] = 68,
- [0][0][0][0][5][9] = 76,
- [0][0][0][0][6][9] = 56,
- [0][0][0][0][9][9] = 56,
- [0][0][0][0][8][9] = 60,
- [0][0][0][0][11][9] = 56,
- [0][0][0][0][2][10] = 76,
- [0][0][0][0][1][10] = 56,
- [0][0][0][0][3][10] = 68,
- [0][0][0][0][5][10] = 76,
- [0][0][0][0][6][10] = 56,
- [0][0][0][0][9][10] = 56,
- [0][0][0][0][8][10] = 60,
- [0][0][0][0][11][10] = 56,
- [0][0][0][0][2][11] = 68,
- [0][0][0][0][1][11] = 56,
- [0][0][0][0][3][11] = 68,
- [0][0][0][0][5][11] = 68,
- [0][0][0][0][6][11] = 56,
- [0][0][0][0][9][11] = 56,
- [0][0][0][0][8][11] = 60,
- [0][0][0][0][11][11] = 56,
- [0][0][0][0][2][12] = 48,
- [0][0][0][0][1][12] = 56,
- [0][0][0][0][3][12] = 68,
- [0][0][0][0][5][12] = 48,
- [0][0][0][0][6][12] = 56,
- [0][0][0][0][9][12] = 56,
- [0][0][0][0][8][12] = 60,
- [0][0][0][0][11][12] = 56,
- [0][0][0][0][2][13] = 127,
- [0][0][0][0][1][13] = 127,
- [0][0][0][0][3][13] = 76,
- [0][0][0][0][5][13] = 127,
- [0][0][0][0][6][13] = 127,
- [0][0][0][0][9][13] = 127,
- [0][0][0][0][8][13] = 127,
- [0][0][0][0][11][13] = 127,
- [0][1][0][0][2][0] = 74,
- [0][1][0][0][1][0] = 44,
- [0][1][0][0][3][0] = 56,
- [0][1][0][0][5][0] = 74,
- [0][1][0][0][6][0] = 44,
- [0][1][0][0][9][0] = 44,
- [0][1][0][0][8][0] = 48,
- [0][1][0][0][11][0] = 44,
- [0][1][0][0][2][1] = 76,
- [0][1][0][0][1][1] = 44,
- [0][1][0][0][3][1] = 56,
- [0][1][0][0][5][1] = 76,
- [0][1][0][0][6][1] = 44,
- [0][1][0][0][9][1] = 44,
- [0][1][0][0][8][1] = 48,
- [0][1][0][0][11][1] = 44,
- [0][1][0][0][2][2] = 76,
- [0][1][0][0][1][2] = 44,
- [0][1][0][0][3][2] = 56,
- [0][1][0][0][5][2] = 76,
- [0][1][0][0][6][2] = 44,
- [0][1][0][0][9][2] = 44,
- [0][1][0][0][8][2] = 48,
- [0][1][0][0][11][2] = 44,
- [0][1][0][0][2][3] = 76,
- [0][1][0][0][1][3] = 44,
- [0][1][0][0][3][3] = 56,
- [0][1][0][0][5][3] = 76,
- [0][1][0][0][6][3] = 44,
- [0][1][0][0][9][3] = 44,
- [0][1][0][0][8][3] = 48,
- [0][1][0][0][11][3] = 44,
- [0][1][0][0][2][4] = 76,
- [0][1][0][0][1][4] = 44,
- [0][1][0][0][3][4] = 56,
- [0][1][0][0][5][4] = 76,
- [0][1][0][0][6][4] = 44,
- [0][1][0][0][9][4] = 44,
- [0][1][0][0][8][4] = 48,
- [0][1][0][0][11][4] = 44,
- [0][1][0][0][2][5] = 76,
- [0][1][0][0][1][5] = 44,
- [0][1][0][0][3][5] = 56,
- [0][1][0][0][5][5] = 76,
- [0][1][0][0][6][5] = 44,
- [0][1][0][0][9][5] = 44,
- [0][1][0][0][8][5] = 48,
- [0][1][0][0][11][5] = 44,
- [0][1][0][0][2][6] = 76,
- [0][1][0][0][1][6] = 44,
- [0][1][0][0][3][6] = 56,
- [0][1][0][0][5][6] = 76,
- [0][1][0][0][6][6] = 44,
- [0][1][0][0][9][6] = 44,
- [0][1][0][0][8][6] = 48,
- [0][1][0][0][11][6] = 44,
- [0][1][0][0][2][7] = 76,
- [0][1][0][0][1][7] = 44,
- [0][1][0][0][3][7] = 56,
- [0][1][0][0][5][7] = 76,
- [0][1][0][0][6][7] = 44,
- [0][1][0][0][9][7] = 44,
- [0][1][0][0][8][7] = 48,
- [0][1][0][0][11][7] = 44,
- [0][1][0][0][2][8] = 76,
- [0][1][0][0][1][8] = 44,
- [0][1][0][0][3][8] = 56,
- [0][1][0][0][5][8] = 76,
- [0][1][0][0][6][8] = 44,
- [0][1][0][0][9][8] = 44,
- [0][1][0][0][8][8] = 48,
- [0][1][0][0][11][8] = 44,
- [0][1][0][0][2][9] = 76,
- [0][1][0][0][1][9] = 44,
- [0][1][0][0][3][9] = 56,
- [0][1][0][0][5][9] = 76,
- [0][1][0][0][6][9] = 44,
- [0][1][0][0][9][9] = 44,
- [0][1][0][0][8][9] = 48,
- [0][1][0][0][11][9] = 44,
- [0][1][0][0][2][10] = 62,
- [0][1][0][0][1][10] = 44,
- [0][1][0][0][3][10] = 56,
- [0][1][0][0][5][10] = 62,
- [0][1][0][0][6][10] = 44,
- [0][1][0][0][9][10] = 44,
- [0][1][0][0][8][10] = 48,
- [0][1][0][0][11][10] = 44,
- [0][1][0][0][2][11] = 52,
- [0][1][0][0][1][11] = 44,
- [0][1][0][0][3][11] = 56,
- [0][1][0][0][5][11] = 52,
- [0][1][0][0][6][11] = 44,
- [0][1][0][0][9][11] = 44,
- [0][1][0][0][8][11] = 48,
- [0][1][0][0][11][11] = 44,
- [0][1][0][0][2][12] = 38,
- [0][1][0][0][1][12] = 44,
- [0][1][0][0][3][12] = 56,
- [0][1][0][0][5][12] = 38,
- [0][1][0][0][6][12] = 44,
- [0][1][0][0][9][12] = 44,
- [0][1][0][0][8][12] = 48,
- [0][1][0][0][11][12] = 44,
- [0][1][0][0][2][13] = 127,
- [0][1][0][0][1][13] = 127,
- [0][1][0][0][3][13] = 64,
- [0][1][0][0][5][13] = 127,
- [0][1][0][0][6][13] = 127,
- [0][1][0][0][9][13] = 127,
- [0][1][0][0][8][13] = 127,
- [0][1][0][0][11][13] = 127,
- [1][0][0][0][2][0] = 127,
- [1][0][0][0][1][0] = 127,
- [1][0][0][0][3][0] = 127,
- [1][0][0][0][5][0] = 127,
- [1][0][0][0][6][0] = 127,
- [1][0][0][0][9][0] = 127,
- [1][0][0][0][8][0] = 127,
- [1][0][0][0][11][0] = 127,
- [1][0][0][0][2][1] = 127,
- [1][0][0][0][1][1] = 127,
- [1][0][0][0][3][1] = 127,
- [1][0][0][0][5][1] = 127,
- [1][0][0][0][6][1] = 127,
- [1][0][0][0][9][1] = 127,
- [1][0][0][0][8][1] = 127,
- [1][0][0][0][11][1] = 127,
- [1][0][0][0][2][2] = 60,
- [1][0][0][0][1][2] = 58,
- [1][0][0][0][3][2] = 68,
- [1][0][0][0][5][2] = 60,
- [1][0][0][0][6][2] = 58,
- [1][0][0][0][9][2] = 58,
- [1][0][0][0][8][2] = 60,
- [1][0][0][0][11][2] = 58,
- [1][0][0][0][2][3] = 60,
- [1][0][0][0][1][3] = 58,
- [1][0][0][0][3][3] = 68,
- [1][0][0][0][5][3] = 60,
- [1][0][0][0][6][3] = 58,
- [1][0][0][0][9][3] = 58,
- [1][0][0][0][8][3] = 60,
- [1][0][0][0][11][3] = 58,
- [1][0][0][0][2][4] = 60,
- [1][0][0][0][1][4] = 58,
- [1][0][0][0][3][4] = 68,
- [1][0][0][0][5][4] = 60,
- [1][0][0][0][6][4] = 58,
- [1][0][0][0][9][4] = 58,
- [1][0][0][0][8][4] = 60,
- [1][0][0][0][11][4] = 58,
- [1][0][0][0][2][5] = 60,
- [1][0][0][0][1][5] = 58,
- [1][0][0][0][3][5] = 68,
- [1][0][0][0][5][5] = 60,
- [1][0][0][0][6][5] = 58,
- [1][0][0][0][9][5] = 58,
- [1][0][0][0][8][5] = 60,
- [1][0][0][0][11][5] = 58,
- [1][0][0][0][2][6] = 46,
- [1][0][0][0][1][6] = 58,
- [1][0][0][0][3][6] = 68,
- [1][0][0][0][5][6] = 46,
- [1][0][0][0][6][6] = 58,
- [1][0][0][0][9][6] = 58,
- [1][0][0][0][8][6] = 60,
- [1][0][0][0][11][6] = 58,
- [1][0][0][0][2][7] = 46,
- [1][0][0][0][1][7] = 58,
- [1][0][0][0][3][7] = 68,
- [1][0][0][0][5][7] = 46,
- [1][0][0][0][6][7] = 58,
- [1][0][0][0][9][7] = 58,
- [1][0][0][0][8][7] = 60,
- [1][0][0][0][11][7] = 58,
- [1][0][0][0][2][8] = 46,
- [1][0][0][0][1][8] = 58,
- [1][0][0][0][3][8] = 68,
- [1][0][0][0][5][8] = 46,
- [1][0][0][0][6][8] = 58,
- [1][0][0][0][9][8] = 58,
- [1][0][0][0][8][8] = 60,
- [1][0][0][0][11][8] = 58,
- [1][0][0][0][2][9] = 32,
- [1][0][0][0][1][9] = 58,
- [1][0][0][0][3][9] = 68,
- [1][0][0][0][5][9] = 32,
- [1][0][0][0][6][9] = 58,
- [1][0][0][0][9][9] = 58,
- [1][0][0][0][8][9] = 60,
- [1][0][0][0][11][9] = 58,
- [1][0][0][0][2][10] = 32,
- [1][0][0][0][1][10] = 58,
- [1][0][0][0][3][10] = 68,
- [1][0][0][0][5][10] = 32,
- [1][0][0][0][6][10] = 58,
- [1][0][0][0][9][10] = 58,
- [1][0][0][0][8][10] = 60,
- [1][0][0][0][11][10] = 58,
- [1][0][0][0][2][11] = 127,
- [1][0][0][0][1][11] = 127,
- [1][0][0][0][3][11] = 127,
- [1][0][0][0][5][11] = 127,
- [1][0][0][0][6][11] = 127,
- [1][0][0][0][9][11] = 127,
- [1][0][0][0][8][11] = 127,
- [1][0][0][0][11][11] = 127,
- [1][0][0][0][2][12] = 127,
- [1][0][0][0][1][12] = 127,
- [1][0][0][0][3][12] = 127,
- [1][0][0][0][5][12] = 127,
- [1][0][0][0][6][12] = 127,
- [1][0][0][0][9][12] = 127,
- [1][0][0][0][8][12] = 127,
- [1][0][0][0][11][12] = 127,
- [1][0][0][0][2][13] = 127,
- [1][0][0][0][1][13] = 127,
- [1][0][0][0][3][13] = 127,
- [1][0][0][0][5][13] = 127,
- [1][0][0][0][6][13] = 127,
- [1][0][0][0][9][13] = 127,
- [1][0][0][0][8][13] = 127,
- [1][0][0][0][11][13] = 127,
- [1][1][0][0][2][0] = 127,
- [1][1][0][0][1][0] = 127,
- [1][1][0][0][3][0] = 127,
- [1][1][0][0][5][0] = 127,
- [1][1][0][0][6][0] = 127,
- [1][1][0][0][9][0] = 127,
- [1][1][0][0][8][0] = 127,
- [1][1][0][0][11][0] = 127,
- [1][1][0][0][2][1] = 127,
- [1][1][0][0][1][1] = 127,
- [1][1][0][0][3][1] = 127,
- [1][1][0][0][5][1] = 127,
- [1][1][0][0][6][1] = 127,
- [1][1][0][0][9][1] = 127,
- [1][1][0][0][8][1] = 127,
- [1][1][0][0][11][1] = 127,
- [1][1][0][0][2][2] = 48,
- [1][1][0][0][1][2] = 46,
- [1][1][0][0][3][2] = 56,
- [1][1][0][0][5][2] = 48,
- [1][1][0][0][6][2] = 46,
- [1][1][0][0][9][2] = 46,
- [1][1][0][0][8][2] = 48,
- [1][1][0][0][11][2] = 46,
- [1][1][0][0][2][3] = 48,
- [1][1][0][0][1][3] = 46,
- [1][1][0][0][3][3] = 56,
- [1][1][0][0][5][3] = 48,
- [1][1][0][0][6][3] = 46,
- [1][1][0][0][9][3] = 46,
- [1][1][0][0][8][3] = 48,
- [1][1][0][0][11][3] = 46,
- [1][1][0][0][2][4] = 48,
- [1][1][0][0][1][4] = 46,
- [1][1][0][0][3][4] = 56,
- [1][1][0][0][5][4] = 48,
- [1][1][0][0][6][4] = 46,
- [1][1][0][0][9][4] = 46,
- [1][1][0][0][8][4] = 48,
- [1][1][0][0][11][4] = 46,
- [1][1][0][0][2][5] = 58,
- [1][1][0][0][1][5] = 46,
- [1][1][0][0][3][5] = 56,
- [1][1][0][0][5][5] = 58,
- [1][1][0][0][6][5] = 46,
- [1][1][0][0][9][5] = 46,
- [1][1][0][0][8][5] = 48,
- [1][1][0][0][11][5] = 46,
- [1][1][0][0][2][6] = 46,
- [1][1][0][0][1][6] = 46,
- [1][1][0][0][3][6] = 56,
- [1][1][0][0][5][6] = 46,
- [1][1][0][0][6][6] = 46,
- [1][1][0][0][9][6] = 46,
- [1][1][0][0][8][6] = 48,
- [1][1][0][0][11][6] = 46,
- [1][1][0][0][2][7] = 46,
- [1][1][0][0][1][7] = 46,
- [1][1][0][0][3][7] = 56,
- [1][1][0][0][5][7] = 46,
- [1][1][0][0][6][7] = 46,
- [1][1][0][0][9][7] = 46,
- [1][1][0][0][8][7] = 48,
- [1][1][0][0][11][7] = 46,
- [1][1][0][0][2][8] = 46,
- [1][1][0][0][1][8] = 46,
- [1][1][0][0][3][8] = 56,
- [1][1][0][0][5][8] = 46,
- [1][1][0][0][6][8] = 46,
- [1][1][0][0][9][8] = 46,
- [1][1][0][0][8][8] = 48,
- [1][1][0][0][11][8] = 46,
- [1][1][0][0][2][9] = 24,
- [1][1][0][0][1][9] = 46,
- [1][1][0][0][3][9] = 56,
- [1][1][0][0][5][9] = 24,
- [1][1][0][0][6][9] = 46,
- [1][1][0][0][9][9] = 46,
- [1][1][0][0][8][9] = 48,
- [1][1][0][0][11][9] = 46,
- [1][1][0][0][2][10] = 24,
- [1][1][0][0][1][10] = 46,
- [1][1][0][0][3][10] = 56,
- [1][1][0][0][5][10] = 24,
- [1][1][0][0][6][10] = 46,
- [1][1][0][0][9][10] = 46,
- [1][1][0][0][8][10] = 48,
- [1][1][0][0][11][10] = 46,
- [1][1][0][0][2][11] = 127,
- [1][1][0][0][1][11] = 127,
- [1][1][0][0][3][11] = 127,
- [1][1][0][0][5][11] = 127,
- [1][1][0][0][6][11] = 127,
- [1][1][0][0][9][11] = 127,
- [1][1][0][0][8][11] = 127,
- [1][1][0][0][11][11] = 127,
- [1][1][0][0][2][12] = 127,
- [1][1][0][0][1][12] = 127,
- [1][1][0][0][3][12] = 127,
- [1][1][0][0][5][12] = 127,
- [1][1][0][0][6][12] = 127,
- [1][1][0][0][9][12] = 127,
- [1][1][0][0][8][12] = 127,
- [1][1][0][0][11][12] = 127,
- [1][1][0][0][2][13] = 127,
- [1][1][0][0][1][13] = 127,
- [1][1][0][0][3][13] = 127,
- [1][1][0][0][5][13] = 127,
- [1][1][0][0][6][13] = 127,
- [1][1][0][0][9][13] = 127,
- [1][1][0][0][8][13] = 127,
- [1][1][0][0][11][13] = 127,
- [0][0][1][0][2][0] = 66,
- [0][0][1][0][1][0] = 58,
- [0][0][1][0][3][0] = 76,
- [0][0][1][0][5][0] = 66,
- [0][0][1][0][6][0] = 58,
- [0][0][1][0][9][0] = 58,
- [0][0][1][0][8][0] = 60,
- [0][0][1][0][11][0] = 58,
- [0][0][1][0][2][1] = 66,
- [0][0][1][0][1][1] = 58,
- [0][0][1][0][3][1] = 76,
- [0][0][1][0][5][1] = 66,
- [0][0][1][0][6][1] = 58,
- [0][0][1][0][9][1] = 58,
- [0][0][1][0][8][1] = 60,
- [0][0][1][0][11][1] = 58,
- [0][0][1][0][2][2] = 70,
- [0][0][1][0][1][2] = 58,
- [0][0][1][0][3][2] = 76,
- [0][0][1][0][5][2] = 70,
- [0][0][1][0][6][2] = 58,
- [0][0][1][0][9][2] = 58,
- [0][0][1][0][8][2] = 60,
- [0][0][1][0][11][2] = 58,
- [0][0][1][0][2][3] = 74,
- [0][0][1][0][1][3] = 58,
- [0][0][1][0][3][3] = 76,
- [0][0][1][0][5][3] = 74,
- [0][0][1][0][6][3] = 58,
- [0][0][1][0][9][3] = 58,
- [0][0][1][0][8][3] = 60,
- [0][0][1][0][11][3] = 58,
- [0][0][1][0][2][4] = 78,
- [0][0][1][0][1][4] = 58,
- [0][0][1][0][3][4] = 76,
- [0][0][1][0][5][4] = 78,
- [0][0][1][0][6][4] = 58,
- [0][0][1][0][9][4] = 58,
- [0][0][1][0][8][4] = 60,
- [0][0][1][0][11][4] = 58,
- [0][0][1][0][2][5] = 78,
- [0][0][1][0][1][5] = 58,
- [0][0][1][0][3][5] = 76,
- [0][0][1][0][5][5] = 78,
- [0][0][1][0][6][5] = 58,
- [0][0][1][0][9][5] = 58,
- [0][0][1][0][8][5] = 60,
- [0][0][1][0][11][5] = 58,
- [0][0][1][0][2][6] = 78,
- [0][0][1][0][1][6] = 58,
- [0][0][1][0][3][6] = 76,
- [0][0][1][0][5][6] = 78,
- [0][0][1][0][6][6] = 58,
- [0][0][1][0][9][6] = 58,
- [0][0][1][0][8][6] = 60,
- [0][0][1][0][11][6] = 58,
- [0][0][1][0][2][7] = 74,
- [0][0][1][0][1][7] = 58,
- [0][0][1][0][3][7] = 76,
- [0][0][1][0][5][7] = 74,
- [0][0][1][0][6][7] = 58,
- [0][0][1][0][9][7] = 58,
- [0][0][1][0][8][7] = 60,
- [0][0][1][0][11][7] = 58,
- [0][0][1][0][2][8] = 70,
- [0][0][1][0][1][8] = 58,
- [0][0][1][0][3][8] = 76,
- [0][0][1][0][5][8] = 70,
- [0][0][1][0][6][8] = 58,
- [0][0][1][0][9][8] = 58,
- [0][0][1][0][8][8] = 60,
- [0][0][1][0][11][8] = 58,
- [0][0][1][0][2][9] = 66,
- [0][0][1][0][1][9] = 58,
- [0][0][1][0][3][9] = 76,
- [0][0][1][0][5][9] = 66,
- [0][0][1][0][6][9] = 58,
- [0][0][1][0][9][9] = 58,
- [0][0][1][0][8][9] = 60,
- [0][0][1][0][11][9] = 58,
- [0][0][1][0][2][10] = 66,
- [0][0][1][0][1][10] = 58,
- [0][0][1][0][3][10] = 76,
- [0][0][1][0][5][10] = 66,
- [0][0][1][0][6][10] = 58,
- [0][0][1][0][9][10] = 58,
- [0][0][1][0][8][10] = 60,
- [0][0][1][0][11][10] = 58,
- [0][0][1][0][2][11] = 56,
- [0][0][1][0][1][11] = 58,
- [0][0][1][0][3][11] = 76,
- [0][0][1][0][5][11] = 56,
- [0][0][1][0][6][11] = 58,
- [0][0][1][0][9][11] = 58,
- [0][0][1][0][8][11] = 60,
- [0][0][1][0][11][11] = 58,
- [0][0][1][0][2][12] = 52,
- [0][0][1][0][1][12] = 58,
- [0][0][1][0][3][12] = 76,
- [0][0][1][0][5][12] = 52,
- [0][0][1][0][6][12] = 58,
- [0][0][1][0][9][12] = 58,
- [0][0][1][0][8][12] = 60,
- [0][0][1][0][11][12] = 58,
- [0][0][1][0][2][13] = 127,
- [0][0][1][0][1][13] = 127,
- [0][0][1][0][3][13] = 127,
- [0][0][1][0][5][13] = 127,
- [0][0][1][0][6][13] = 127,
- [0][0][1][0][9][13] = 127,
- [0][0][1][0][8][13] = 127,
- [0][0][1][0][11][13] = 127,
- [0][1][1][0][2][0] = 62,
- [0][1][1][0][1][0] = 46,
- [0][1][1][0][3][0] = 64,
- [0][1][1][0][5][0] = 62,
- [0][1][1][0][6][0] = 46,
- [0][1][1][0][9][0] = 46,
- [0][1][1][0][8][0] = 48,
- [0][1][1][0][11][0] = 46,
- [0][1][1][0][2][1] = 62,
- [0][1][1][0][1][1] = 46,
- [0][1][1][0][3][1] = 64,
- [0][1][1][0][5][1] = 62,
- [0][1][1][0][6][1] = 46,
- [0][1][1][0][9][1] = 46,
- [0][1][1][0][8][1] = 48,
- [0][1][1][0][11][1] = 46,
- [0][1][1][0][2][2] = 66,
- [0][1][1][0][1][2] = 46,
- [0][1][1][0][3][2] = 64,
- [0][1][1][0][5][2] = 66,
- [0][1][1][0][6][2] = 46,
- [0][1][1][0][9][2] = 46,
- [0][1][1][0][8][2] = 48,
- [0][1][1][0][11][2] = 46,
- [0][1][1][0][2][3] = 70,
- [0][1][1][0][1][3] = 46,
- [0][1][1][0][3][3] = 64,
- [0][1][1][0][5][3] = 70,
- [0][1][1][0][6][3] = 46,
- [0][1][1][0][9][3] = 46,
- [0][1][1][0][8][3] = 48,
- [0][1][1][0][11][3] = 46,
- [0][1][1][0][2][4] = 78,
- [0][1][1][0][1][4] = 46,
- [0][1][1][0][3][4] = 64,
- [0][1][1][0][5][4] = 78,
- [0][1][1][0][6][4] = 46,
- [0][1][1][0][9][4] = 46,
- [0][1][1][0][8][4] = 48,
- [0][1][1][0][11][4] = 46,
- [0][1][1][0][2][5] = 78,
- [0][1][1][0][1][5] = 46,
- [0][1][1][0][3][5] = 64,
- [0][1][1][0][5][5] = 78,
- [0][1][1][0][6][5] = 46,
- [0][1][1][0][9][5] = 46,
- [0][1][1][0][8][5] = 48,
- [0][1][1][0][11][5] = 46,
- [0][1][1][0][2][6] = 78,
- [0][1][1][0][1][6] = 46,
- [0][1][1][0][3][6] = 64,
- [0][1][1][0][5][6] = 78,
- [0][1][1][0][6][6] = 46,
- [0][1][1][0][9][6] = 46,
- [0][1][1][0][8][6] = 48,
- [0][1][1][0][11][6] = 46,
- [0][1][1][0][2][7] = 70,
- [0][1][1][0][1][7] = 46,
- [0][1][1][0][3][7] = 64,
- [0][1][1][0][5][7] = 70,
- [0][1][1][0][6][7] = 46,
- [0][1][1][0][9][7] = 46,
- [0][1][1][0][8][7] = 48,
- [0][1][1][0][11][7] = 46,
- [0][1][1][0][2][8] = 66,
- [0][1][1][0][1][8] = 46,
- [0][1][1][0][3][8] = 64,
- [0][1][1][0][5][8] = 66,
- [0][1][1][0][6][8] = 46,
- [0][1][1][0][9][8] = 46,
- [0][1][1][0][8][8] = 48,
- [0][1][1][0][11][8] = 46,
- [0][1][1][0][2][9] = 62,
- [0][1][1][0][1][9] = 46,
- [0][1][1][0][3][9] = 64,
- [0][1][1][0][5][9] = 62,
- [0][1][1][0][6][9] = 46,
- [0][1][1][0][9][9] = 46,
- [0][1][1][0][8][9] = 48,
- [0][1][1][0][11][9] = 46,
- [0][1][1][0][2][10] = 62,
- [0][1][1][0][1][10] = 46,
- [0][1][1][0][3][10] = 64,
- [0][1][1][0][5][10] = 62,
- [0][1][1][0][6][10] = 46,
- [0][1][1][0][9][10] = 46,
- [0][1][1][0][8][10] = 48,
- [0][1][1][0][11][10] = 46,
- [0][1][1][0][2][11] = 42,
- [0][1][1][0][1][11] = 46,
- [0][1][1][0][3][11] = 64,
- [0][1][1][0][5][11] = 42,
- [0][1][1][0][6][11] = 46,
- [0][1][1][0][9][11] = 46,
- [0][1][1][0][8][11] = 48,
- [0][1][1][0][11][11] = 46,
- [0][1][1][0][2][12] = 40,
- [0][1][1][0][1][12] = 46,
- [0][1][1][0][3][12] = 64,
- [0][1][1][0][5][12] = 40,
- [0][1][1][0][6][12] = 46,
- [0][1][1][0][9][12] = 46,
- [0][1][1][0][8][12] = 48,
- [0][1][1][0][11][12] = 46,
- [0][1][1][0][2][13] = 127,
- [0][1][1][0][1][13] = 127,
- [0][1][1][0][3][13] = 127,
- [0][1][1][0][5][13] = 127,
- [0][1][1][0][6][13] = 127,
- [0][1][1][0][9][13] = 127,
- [0][1][1][0][8][13] = 127,
- [0][1][1][0][11][13] = 127,
- [0][0][2][0][2][0] = 66,
- [0][0][2][0][1][0] = 58,
- [0][0][2][0][3][0] = 76,
- [0][0][2][0][5][0] = 66,
- [0][0][2][0][6][0] = 58,
- [0][0][2][0][9][0] = 58,
- [0][0][2][0][8][0] = 60,
- [0][0][2][0][11][0] = 58,
- [0][0][2][0][2][1] = 66,
- [0][0][2][0][1][1] = 58,
- [0][0][2][0][3][1] = 76,
- [0][0][2][0][5][1] = 66,
- [0][0][2][0][6][1] = 58,
- [0][0][2][0][9][1] = 58,
- [0][0][2][0][8][1] = 60,
- [0][0][2][0][11][1] = 58,
- [0][0][2][0][2][2] = 70,
- [0][0][2][0][1][2] = 58,
- [0][0][2][0][3][2] = 76,
- [0][0][2][0][5][2] = 70,
- [0][0][2][0][6][2] = 58,
- [0][0][2][0][9][2] = 58,
- [0][0][2][0][8][2] = 60,
- [0][0][2][0][11][2] = 58,
- [0][0][2][0][2][3] = 74,
- [0][0][2][0][1][3] = 58,
- [0][0][2][0][3][3] = 76,
- [0][0][2][0][5][3] = 74,
- [0][0][2][0][6][3] = 58,
- [0][0][2][0][9][3] = 58,
- [0][0][2][0][8][3] = 60,
- [0][0][2][0][11][3] = 58,
- [0][0][2][0][2][4] = 76,
- [0][0][2][0][1][4] = 58,
- [0][0][2][0][3][4] = 76,
- [0][0][2][0][5][4] = 76,
- [0][0][2][0][6][4] = 58,
- [0][0][2][0][9][4] = 58,
- [0][0][2][0][8][4] = 60,
- [0][0][2][0][11][4] = 58,
- [0][0][2][0][2][5] = 76,
- [0][0][2][0][1][5] = 58,
- [0][0][2][0][3][5] = 76,
- [0][0][2][0][5][5] = 76,
- [0][0][2][0][6][5] = 58,
- [0][0][2][0][9][5] = 58,
- [0][0][2][0][8][5] = 60,
- [0][0][2][0][11][5] = 58,
- [0][0][2][0][2][6] = 76,
- [0][0][2][0][1][6] = 58,
- [0][0][2][0][3][6] = 76,
- [0][0][2][0][5][6] = 76,
- [0][0][2][0][6][6] = 58,
- [0][0][2][0][9][6] = 58,
- [0][0][2][0][8][6] = 60,
- [0][0][2][0][11][6] = 58,
- [0][0][2][0][2][7] = 74,
- [0][0][2][0][1][7] = 58,
- [0][0][2][0][3][7] = 76,
- [0][0][2][0][5][7] = 74,
- [0][0][2][0][6][7] = 58,
- [0][0][2][0][9][7] = 58,
- [0][0][2][0][8][7] = 60,
- [0][0][2][0][11][7] = 58,
- [0][0][2][0][2][8] = 70,
- [0][0][2][0][1][8] = 58,
- [0][0][2][0][3][8] = 76,
- [0][0][2][0][5][8] = 70,
- [0][0][2][0][6][8] = 58,
- [0][0][2][0][9][8] = 58,
- [0][0][2][0][8][8] = 60,
- [0][0][2][0][11][8] = 58,
- [0][0][2][0][2][9] = 66,
- [0][0][2][0][1][9] = 58,
- [0][0][2][0][3][9] = 76,
- [0][0][2][0][5][9] = 66,
- [0][0][2][0][6][9] = 58,
- [0][0][2][0][9][9] = 58,
- [0][0][2][0][8][9] = 60,
- [0][0][2][0][11][9] = 58,
- [0][0][2][0][2][10] = 66,
- [0][0][2][0][1][10] = 58,
- [0][0][2][0][3][10] = 76,
- [0][0][2][0][5][10] = 66,
- [0][0][2][0][6][10] = 58,
- [0][0][2][0][9][10] = 58,
- [0][0][2][0][8][10] = 60,
- [0][0][2][0][11][10] = 58,
- [0][0][2][0][2][11] = 54,
- [0][0][2][0][1][11] = 58,
- [0][0][2][0][3][11] = 76,
- [0][0][2][0][5][11] = 54,
- [0][0][2][0][6][11] = 58,
- [0][0][2][0][9][11] = 58,
- [0][0][2][0][8][11] = 60,
- [0][0][2][0][11][11] = 58,
- [0][0][2][0][2][12] = 50,
- [0][0][2][0][1][12] = 58,
- [0][0][2][0][3][12] = 76,
- [0][0][2][0][5][12] = 50,
- [0][0][2][0][6][12] = 58,
- [0][0][2][0][9][12] = 58,
- [0][0][2][0][8][12] = 60,
- [0][0][2][0][11][12] = 58,
- [0][0][2][0][2][13] = 127,
- [0][0][2][0][1][13] = 127,
- [0][0][2][0][3][13] = 127,
- [0][0][2][0][5][13] = 127,
- [0][0][2][0][6][13] = 127,
- [0][0][2][0][9][13] = 127,
- [0][0][2][0][8][13] = 127,
- [0][0][2][0][11][13] = 127,
- [0][1][2][0][2][0] = 62,
- [0][1][2][0][1][0] = 46,
- [0][1][2][0][3][0] = 64,
- [0][1][2][0][5][0] = 62,
- [0][1][2][0][6][0] = 46,
- [0][1][2][0][9][0] = 46,
- [0][1][2][0][8][0] = 48,
- [0][1][2][0][11][0] = 46,
- [0][1][2][0][2][1] = 62,
- [0][1][2][0][1][1] = 46,
- [0][1][2][0][3][1] = 64,
- [0][1][2][0][5][1] = 62,
- [0][1][2][0][6][1] = 46,
- [0][1][2][0][9][1] = 46,
- [0][1][2][0][8][1] = 48,
- [0][1][2][0][11][1] = 46,
- [0][1][2][0][2][2] = 66,
- [0][1][2][0][1][2] = 46,
- [0][1][2][0][3][2] = 64,
- [0][1][2][0][5][2] = 66,
- [0][1][2][0][6][2] = 46,
- [0][1][2][0][9][2] = 46,
- [0][1][2][0][8][2] = 48,
- [0][1][2][0][11][2] = 46,
- [0][1][2][0][2][3] = 70,
- [0][1][2][0][1][3] = 46,
- [0][1][2][0][3][3] = 64,
- [0][1][2][0][5][3] = 70,
- [0][1][2][0][6][3] = 46,
- [0][1][2][0][9][3] = 46,
- [0][1][2][0][8][3] = 48,
- [0][1][2][0][11][3] = 46,
- [0][1][2][0][2][4] = 76,
- [0][1][2][0][1][4] = 46,
- [0][1][2][0][3][4] = 64,
- [0][1][2][0][5][4] = 76,
- [0][1][2][0][6][4] = 46,
- [0][1][2][0][9][4] = 46,
- [0][1][2][0][8][4] = 48,
- [0][1][2][0][11][4] = 46,
- [0][1][2][0][2][5] = 76,
- [0][1][2][0][1][5] = 46,
- [0][1][2][0][3][5] = 64,
- [0][1][2][0][5][5] = 76,
- [0][1][2][0][6][5] = 46,
- [0][1][2][0][9][5] = 46,
- [0][1][2][0][8][5] = 48,
- [0][1][2][0][11][5] = 46,
- [0][1][2][0][2][6] = 76,
- [0][1][2][0][1][6] = 46,
- [0][1][2][0][3][6] = 64,
- [0][1][2][0][5][6] = 76,
- [0][1][2][0][6][6] = 46,
- [0][1][2][0][9][6] = 46,
- [0][1][2][0][8][6] = 48,
- [0][1][2][0][11][6] = 46,
- [0][1][2][0][2][7] = 68,
- [0][1][2][0][1][7] = 46,
- [0][1][2][0][3][7] = 64,
- [0][1][2][0][5][7] = 68,
- [0][1][2][0][6][7] = 46,
- [0][1][2][0][9][7] = 46,
- [0][1][2][0][8][7] = 48,
- [0][1][2][0][11][7] = 46,
- [0][1][2][0][2][8] = 64,
- [0][1][2][0][1][8] = 46,
- [0][1][2][0][3][8] = 64,
- [0][1][2][0][5][8] = 64,
- [0][1][2][0][6][8] = 46,
- [0][1][2][0][9][8] = 46,
- [0][1][2][0][8][8] = 48,
- [0][1][2][0][11][8] = 46,
- [0][1][2][0][2][9] = 60,
- [0][1][2][0][1][9] = 46,
- [0][1][2][0][3][9] = 64,
- [0][1][2][0][5][9] = 60,
- [0][1][2][0][6][9] = 46,
- [0][1][2][0][9][9] = 46,
- [0][1][2][0][8][9] = 48,
- [0][1][2][0][11][9] = 46,
- [0][1][2][0][2][10] = 60,
- [0][1][2][0][1][10] = 46,
- [0][1][2][0][3][10] = 64,
- [0][1][2][0][5][10] = 60,
- [0][1][2][0][6][10] = 46,
- [0][1][2][0][9][10] = 46,
- [0][1][2][0][8][10] = 48,
- [0][1][2][0][11][10] = 46,
- [0][1][2][0][2][11] = 42,
- [0][1][2][0][1][11] = 46,
- [0][1][2][0][3][11] = 64,
- [0][1][2][0][5][11] = 42,
- [0][1][2][0][6][11] = 46,
- [0][1][2][0][9][11] = 46,
- [0][1][2][0][8][11] = 48,
- [0][1][2][0][11][11] = 46,
- [0][1][2][0][2][12] = 40,
- [0][1][2][0][1][12] = 46,
- [0][1][2][0][3][12] = 64,
- [0][1][2][0][5][12] = 40,
- [0][1][2][0][6][12] = 46,
- [0][1][2][0][9][12] = 46,
- [0][1][2][0][8][12] = 48,
- [0][1][2][0][11][12] = 46,
- [0][1][2][0][2][13] = 127,
- [0][1][2][0][1][13] = 127,
- [0][1][2][0][3][13] = 127,
- [0][1][2][0][5][13] = 127,
- [0][1][2][0][6][13] = 127,
- [0][1][2][0][9][13] = 127,
- [0][1][2][0][8][13] = 127,
- [0][1][2][0][11][13] = 127,
- [0][1][2][1][2][0] = 62,
- [0][1][2][1][1][0] = 34,
- [0][1][2][1][3][0] = 64,
- [0][1][2][1][5][0] = 62,
- [0][1][2][1][6][0] = 34,
- [0][1][2][1][9][0] = 34,
- [0][1][2][1][8][0] = 36,
- [0][1][2][1][11][0] = 34,
- [0][1][2][1][2][1] = 62,
- [0][1][2][1][1][1] = 34,
- [0][1][2][1][3][1] = 64,
- [0][1][2][1][5][1] = 62,
- [0][1][2][1][6][1] = 34,
- [0][1][2][1][9][1] = 34,
- [0][1][2][1][8][1] = 36,
- [0][1][2][1][11][1] = 34,
- [0][1][2][1][2][2] = 66,
- [0][1][2][1][1][2] = 34,
- [0][1][2][1][3][2] = 64,
- [0][1][2][1][5][2] = 66,
- [0][1][2][1][6][2] = 34,
- [0][1][2][1][9][2] = 34,
- [0][1][2][1][8][2] = 36,
- [0][1][2][1][11][2] = 34,
- [0][1][2][1][2][3] = 70,
- [0][1][2][1][1][3] = 34,
- [0][1][2][1][3][3] = 64,
- [0][1][2][1][5][3] = 70,
- [0][1][2][1][6][3] = 34,
- [0][1][2][1][9][3] = 34,
- [0][1][2][1][8][3] = 36,
- [0][1][2][1][11][3] = 34,
- [0][1][2][1][2][4] = 76,
- [0][1][2][1][1][4] = 34,
- [0][1][2][1][3][4] = 64,
- [0][1][2][1][5][4] = 76,
- [0][1][2][1][6][4] = 34,
- [0][1][2][1][9][4] = 34,
- [0][1][2][1][8][4] = 36,
- [0][1][2][1][11][4] = 34,
- [0][1][2][1][2][5] = 76,
- [0][1][2][1][1][5] = 34,
- [0][1][2][1][3][5] = 64,
- [0][1][2][1][5][5] = 76,
- [0][1][2][1][6][5] = 34,
- [0][1][2][1][9][5] = 34,
- [0][1][2][1][8][5] = 36,
- [0][1][2][1][11][5] = 34,
- [0][1][2][1][2][6] = 76,
- [0][1][2][1][1][6] = 34,
- [0][1][2][1][3][6] = 64,
- [0][1][2][1][5][6] = 76,
- [0][1][2][1][6][6] = 34,
- [0][1][2][1][9][6] = 34,
- [0][1][2][1][8][6] = 36,
- [0][1][2][1][11][6] = 34,
- [0][1][2][1][2][7] = 68,
- [0][1][2][1][1][7] = 34,
- [0][1][2][1][3][7] = 64,
- [0][1][2][1][5][7] = 68,
- [0][1][2][1][6][7] = 34,
- [0][1][2][1][9][7] = 34,
- [0][1][2][1][8][7] = 36,
- [0][1][2][1][11][7] = 34,
- [0][1][2][1][2][8] = 64,
- [0][1][2][1][1][8] = 34,
- [0][1][2][1][3][8] = 64,
- [0][1][2][1][5][8] = 64,
- [0][1][2][1][6][8] = 34,
- [0][1][2][1][9][8] = 34,
- [0][1][2][1][8][8] = 36,
- [0][1][2][1][11][8] = 34,
- [0][1][2][1][2][9] = 60,
- [0][1][2][1][1][9] = 34,
- [0][1][2][1][3][9] = 64,
- [0][1][2][1][5][9] = 60,
- [0][1][2][1][6][9] = 34,
- [0][1][2][1][9][9] = 34,
- [0][1][2][1][8][9] = 36,
- [0][1][2][1][11][9] = 34,
- [0][1][2][1][2][10] = 60,
- [0][1][2][1][1][10] = 34,
- [0][1][2][1][3][10] = 64,
- [0][1][2][1][5][10] = 60,
- [0][1][2][1][6][10] = 34,
- [0][1][2][1][9][10] = 34,
- [0][1][2][1][8][10] = 36,
- [0][1][2][1][11][10] = 34,
- [0][1][2][1][2][11] = 42,
- [0][1][2][1][1][11] = 34,
- [0][1][2][1][3][11] = 64,
- [0][1][2][1][5][11] = 42,
- [0][1][2][1][6][11] = 34,
- [0][1][2][1][9][11] = 34,
- [0][1][2][1][8][11] = 36,
- [0][1][2][1][11][11] = 34,
- [0][1][2][1][2][12] = 40,
- [0][1][2][1][1][12] = 34,
- [0][1][2][1][3][12] = 64,
- [0][1][2][1][5][12] = 40,
- [0][1][2][1][6][12] = 34,
- [0][1][2][1][9][12] = 34,
- [0][1][2][1][8][12] = 36,
- [0][1][2][1][11][12] = 34,
- [0][1][2][1][2][13] = 127,
- [0][1][2][1][1][13] = 127,
- [0][1][2][1][3][13] = 127,
- [0][1][2][1][5][13] = 127,
- [0][1][2][1][6][13] = 127,
- [0][1][2][1][9][13] = 127,
- [0][1][2][1][8][13] = 127,
- [0][1][2][1][11][13] = 127,
- [1][0][2][0][2][0] = 127,
- [1][0][2][0][1][0] = 127,
- [1][0][2][0][3][0] = 127,
- [1][0][2][0][5][0] = 127,
- [1][0][2][0][6][0] = 127,
- [1][0][2][0][9][0] = 127,
- [1][0][2][0][8][0] = 127,
- [1][0][2][0][11][0] = 127,
- [1][0][2][0][2][1] = 127,
- [1][0][2][0][1][1] = 127,
- [1][0][2][0][3][1] = 127,
- [1][0][2][0][5][1] = 127,
- [1][0][2][0][6][1] = 127,
- [1][0][2][0][9][1] = 127,
- [1][0][2][0][8][1] = 127,
- [1][0][2][0][11][1] = 127,
- [1][0][2][0][2][2] = 56,
- [1][0][2][0][1][2] = 58,
- [1][0][2][0][3][2] = 76,
- [1][0][2][0][5][2] = 56,
- [1][0][2][0][6][2] = 58,
- [1][0][2][0][9][2] = 58,
- [1][0][2][0][8][2] = 60,
- [1][0][2][0][11][2] = 58,
- [1][0][2][0][2][3] = 56,
- [1][0][2][0][1][3] = 58,
- [1][0][2][0][3][3] = 76,
- [1][0][2][0][5][3] = 56,
- [1][0][2][0][6][3] = 58,
- [1][0][2][0][9][3] = 58,
- [1][0][2][0][8][3] = 60,
- [1][0][2][0][11][3] = 58,
- [1][0][2][0][2][4] = 60,
- [1][0][2][0][1][4] = 58,
- [1][0][2][0][3][4] = 76,
- [1][0][2][0][5][4] = 60,
- [1][0][2][0][6][4] = 58,
- [1][0][2][0][9][4] = 58,
- [1][0][2][0][8][4] = 60,
- [1][0][2][0][11][4] = 58,
- [1][0][2][0][2][5] = 64,
- [1][0][2][0][1][5] = 58,
- [1][0][2][0][3][5] = 76,
- [1][0][2][0][5][5] = 64,
- [1][0][2][0][6][5] = 58,
- [1][0][2][0][9][5] = 58,
- [1][0][2][0][8][5] = 60,
- [1][0][2][0][11][5] = 58,
- [1][0][2][0][2][6] = 54,
- [1][0][2][0][1][6] = 58,
- [1][0][2][0][3][6] = 76,
- [1][0][2][0][5][6] = 54,
- [1][0][2][0][6][6] = 58,
- [1][0][2][0][9][6] = 58,
- [1][0][2][0][8][6] = 60,
- [1][0][2][0][11][6] = 58,
- [1][0][2][0][2][7] = 50,
- [1][0][2][0][1][7] = 58,
- [1][0][2][0][3][7] = 76,
- [1][0][2][0][5][7] = 50,
- [1][0][2][0][6][7] = 58,
- [1][0][2][0][9][7] = 58,
- [1][0][2][0][8][7] = 60,
- [1][0][2][0][11][7] = 58,
- [1][0][2][0][2][8] = 50,
- [1][0][2][0][1][8] = 58,
- [1][0][2][0][3][8] = 76,
- [1][0][2][0][5][8] = 50,
- [1][0][2][0][6][8] = 58,
- [1][0][2][0][9][8] = 58,
- [1][0][2][0][8][8] = 60,
- [1][0][2][0][11][8] = 58,
- [1][0][2][0][2][9] = 42,
- [1][0][2][0][1][9] = 58,
- [1][0][2][0][3][9] = 76,
- [1][0][2][0][5][9] = 42,
- [1][0][2][0][6][9] = 58,
- [1][0][2][0][9][9] = 58,
- [1][0][2][0][8][9] = 60,
- [1][0][2][0][11][9] = 58,
- [1][0][2][0][2][10] = 40,
- [1][0][2][0][1][10] = 58,
- [1][0][2][0][3][10] = 76,
- [1][0][2][0][5][10] = 40,
- [1][0][2][0][6][10] = 58,
- [1][0][2][0][9][10] = 58,
- [1][0][2][0][8][10] = 60,
- [1][0][2][0][11][10] = 58,
- [1][0][2][0][2][11] = 127,
- [1][0][2][0][1][11] = 127,
- [1][0][2][0][3][11] = 127,
- [1][0][2][0][5][11] = 127,
- [1][0][2][0][6][11] = 127,
- [1][0][2][0][9][11] = 127,
- [1][0][2][0][8][11] = 127,
- [1][0][2][0][11][11] = 127,
- [1][0][2][0][2][12] = 127,
- [1][0][2][0][1][12] = 127,
- [1][0][2][0][3][12] = 127,
- [1][0][2][0][5][12] = 127,
- [1][0][2][0][6][12] = 127,
- [1][0][2][0][9][12] = 127,
- [1][0][2][0][8][12] = 127,
- [1][0][2][0][11][12] = 127,
- [1][0][2][0][2][13] = 127,
- [1][0][2][0][1][13] = 127,
- [1][0][2][0][3][13] = 127,
- [1][0][2][0][5][13] = 127,
- [1][0][2][0][6][13] = 127,
- [1][0][2][0][9][13] = 127,
- [1][0][2][0][8][13] = 127,
- [1][0][2][0][11][13] = 127,
- [1][1][2][0][2][0] = 127,
- [1][1][2][0][1][0] = 127,
- [1][1][2][0][3][0] = 127,
- [1][1][2][0][5][0] = 127,
- [1][1][2][0][6][0] = 127,
- [1][1][2][0][9][0] = 127,
- [1][1][2][0][8][0] = 127,
- [1][1][2][0][11][0] = 127,
- [1][1][2][0][2][1] = 127,
- [1][1][2][0][1][1] = 127,
- [1][1][2][0][3][1] = 127,
- [1][1][2][0][5][1] = 127,
- [1][1][2][0][6][1] = 127,
- [1][1][2][0][9][1] = 127,
- [1][1][2][0][8][1] = 127,
- [1][1][2][0][11][1] = 127,
- [1][1][2][0][2][2] = 52,
- [1][1][2][0][1][2] = 46,
- [1][1][2][0][3][2] = 64,
- [1][1][2][0][5][2] = 52,
- [1][1][2][0][6][2] = 46,
- [1][1][2][0][9][2] = 46,
- [1][1][2][0][8][2] = 48,
- [1][1][2][0][11][2] = 46,
- [1][1][2][0][2][3] = 52,
- [1][1][2][0][1][3] = 46,
- [1][1][2][0][3][3] = 64,
- [1][1][2][0][5][3] = 52,
- [1][1][2][0][6][3] = 46,
- [1][1][2][0][9][3] = 46,
- [1][1][2][0][8][3] = 48,
- [1][1][2][0][11][3] = 46,
- [1][1][2][0][2][4] = 56,
- [1][1][2][0][1][4] = 46,
- [1][1][2][0][3][4] = 64,
- [1][1][2][0][5][4] = 56,
- [1][1][2][0][6][4] = 46,
- [1][1][2][0][9][4] = 46,
- [1][1][2][0][8][4] = 48,
- [1][1][2][0][11][4] = 46,
- [1][1][2][0][2][5] = 60,
- [1][1][2][0][1][5] = 46,
- [1][1][2][0][3][5] = 64,
- [1][1][2][0][5][5] = 60,
- [1][1][2][0][6][5] = 46,
- [1][1][2][0][9][5] = 46,
- [1][1][2][0][8][5] = 48,
- [1][1][2][0][11][5] = 46,
- [1][1][2][0][2][6] = 54,
- [1][1][2][0][1][6] = 46,
- [1][1][2][0][3][6] = 64,
- [1][1][2][0][5][6] = 52,
- [1][1][2][0][6][6] = 46,
- [1][1][2][0][9][6] = 46,
- [1][1][2][0][8][6] = 48,
- [1][1][2][0][11][6] = 46,
- [1][1][2][0][2][7] = 50,
- [1][1][2][0][1][7] = 46,
- [1][1][2][0][3][7] = 64,
- [1][1][2][0][5][7] = 48,
- [1][1][2][0][6][7] = 46,
- [1][1][2][0][9][7] = 46,
- [1][1][2][0][8][7] = 48,
- [1][1][2][0][11][7] = 46,
- [1][1][2][0][2][8] = 50,
- [1][1][2][0][1][8] = 46,
- [1][1][2][0][3][8] = 64,
- [1][1][2][0][5][8] = 48,
- [1][1][2][0][6][8] = 46,
- [1][1][2][0][9][8] = 46,
- [1][1][2][0][8][8] = 48,
- [1][1][2][0][11][8] = 46,
- [1][1][2][0][2][9] = 38,
- [1][1][2][0][1][9] = 46,
- [1][1][2][0][3][9] = 64,
- [1][1][2][0][5][9] = 38,
- [1][1][2][0][6][9] = 46,
- [1][1][2][0][9][9] = 46,
- [1][1][2][0][8][9] = 48,
- [1][1][2][0][11][9] = 46,
- [1][1][2][0][2][10] = 36,
- [1][1][2][0][1][10] = 46,
- [1][1][2][0][3][10] = 64,
- [1][1][2][0][5][10] = 36,
- [1][1][2][0][6][10] = 46,
- [1][1][2][0][9][10] = 46,
- [1][1][2][0][8][10] = 48,
- [1][1][2][0][11][10] = 46,
- [1][1][2][0][2][11] = 127,
- [1][1][2][0][1][11] = 127,
- [1][1][2][0][3][11] = 127,
- [1][1][2][0][5][11] = 127,
- [1][1][2][0][6][11] = 127,
- [1][1][2][0][9][11] = 127,
- [1][1][2][0][8][11] = 127,
- [1][1][2][0][11][11] = 127,
- [1][1][2][0][2][12] = 127,
- [1][1][2][0][1][12] = 127,
- [1][1][2][0][3][12] = 127,
- [1][1][2][0][5][12] = 127,
- [1][1][2][0][6][12] = 127,
- [1][1][2][0][9][12] = 127,
- [1][1][2][0][8][12] = 127,
- [1][1][2][0][11][12] = 127,
- [1][1][2][0][2][13] = 127,
- [1][1][2][0][1][13] = 127,
- [1][1][2][0][3][13] = 127,
- [1][1][2][0][5][13] = 127,
- [1][1][2][0][6][13] = 127,
- [1][1][2][0][9][13] = 127,
- [1][1][2][0][8][13] = 127,
- [1][1][2][0][11][13] = 127,
- [1][1][2][1][2][0] = 127,
- [1][1][2][1][1][0] = 127,
- [1][1][2][1][3][0] = 127,
- [1][1][2][1][5][0] = 127,
- [1][1][2][1][6][0] = 127,
- [1][1][2][1][9][0] = 127,
- [1][1][2][1][8][0] = 127,
- [1][1][2][1][11][0] = 127,
- [1][1][2][1][2][1] = 127,
- [1][1][2][1][1][1] = 127,
- [1][1][2][1][3][1] = 127,
- [1][1][2][1][5][1] = 127,
- [1][1][2][1][6][1] = 127,
- [1][1][2][1][9][1] = 127,
- [1][1][2][1][8][1] = 127,
- [1][1][2][1][11][1] = 127,
- [1][1][2][1][2][2] = 52,
- [1][1][2][1][1][2] = 34,
- [1][1][2][1][3][2] = 64,
- [1][1][2][1][5][2] = 52,
- [1][1][2][1][6][2] = 34,
- [1][1][2][1][9][2] = 34,
- [1][1][2][1][8][2] = 36,
- [1][1][2][1][11][2] = 34,
- [1][1][2][1][2][3] = 52,
- [1][1][2][1][1][3] = 34,
- [1][1][2][1][3][3] = 64,
- [1][1][2][1][5][3] = 52,
- [1][1][2][1][6][3] = 34,
- [1][1][2][1][9][3] = 34,
- [1][1][2][1][8][3] = 36,
- [1][1][2][1][11][3] = 34,
- [1][1][2][1][2][4] = 56,
- [1][1][2][1][1][4] = 34,
- [1][1][2][1][3][4] = 64,
- [1][1][2][1][5][4] = 56,
- [1][1][2][1][6][4] = 34,
- [1][1][2][1][9][4] = 34,
- [1][1][2][1][8][4] = 36,
- [1][1][2][1][11][4] = 34,
- [1][1][2][1][2][5] = 60,
- [1][1][2][1][1][5] = 34,
- [1][1][2][1][3][5] = 64,
- [1][1][2][1][5][5] = 60,
- [1][1][2][1][6][5] = 34,
- [1][1][2][1][9][5] = 34,
- [1][1][2][1][8][5] = 36,
- [1][1][2][1][11][5] = 34,
- [1][1][2][1][2][6] = 54,
- [1][1][2][1][1][6] = 34,
- [1][1][2][1][3][6] = 64,
- [1][1][2][1][5][6] = 52,
- [1][1][2][1][6][6] = 34,
- [1][1][2][1][9][6] = 34,
- [1][1][2][1][8][6] = 36,
- [1][1][2][1][11][6] = 34,
- [1][1][2][1][2][7] = 50,
- [1][1][2][1][1][7] = 34,
- [1][1][2][1][3][7] = 64,
- [1][1][2][1][5][7] = 48,
- [1][1][2][1][6][7] = 34,
- [1][1][2][1][9][7] = 34,
- [1][1][2][1][8][7] = 36,
- [1][1][2][1][11][7] = 34,
- [1][1][2][1][2][8] = 50,
- [1][1][2][1][1][8] = 34,
- [1][1][2][1][3][8] = 64,
- [1][1][2][1][5][8] = 48,
- [1][1][2][1][6][8] = 34,
- [1][1][2][1][9][8] = 34,
- [1][1][2][1][8][8] = 36,
- [1][1][2][1][11][8] = 34,
- [1][1][2][1][2][9] = 38,
- [1][1][2][1][1][9] = 34,
- [1][1][2][1][3][9] = 64,
- [1][1][2][1][5][9] = 38,
- [1][1][2][1][6][9] = 34,
- [1][1][2][1][9][9] = 34,
- [1][1][2][1][8][9] = 36,
- [1][1][2][1][11][9] = 34,
- [1][1][2][1][2][10] = 36,
- [1][1][2][1][1][10] = 34,
- [1][1][2][1][3][10] = 64,
- [1][1][2][1][5][10] = 36,
- [1][1][2][1][6][10] = 34,
- [1][1][2][1][9][10] = 34,
- [1][1][2][1][8][10] = 36,
- [1][1][2][1][11][10] = 34,
- [1][1][2][1][2][11] = 127,
- [1][1][2][1][1][11] = 127,
- [1][1][2][1][3][11] = 127,
- [1][1][2][1][5][11] = 127,
- [1][1][2][1][6][11] = 127,
- [1][1][2][1][9][11] = 127,
- [1][1][2][1][8][11] = 127,
- [1][1][2][1][11][11] = 127,
- [1][1][2][1][2][12] = 127,
- [1][1][2][1][1][12] = 127,
- [1][1][2][1][3][12] = 127,
- [1][1][2][1][5][12] = 127,
- [1][1][2][1][6][12] = 127,
- [1][1][2][1][9][12] = 127,
- [1][1][2][1][8][12] = 127,
- [1][1][2][1][11][12] = 127,
- [1][1][2][1][2][13] = 127,
- [1][1][2][1][1][13] = 127,
- [1][1][2][1][3][13] = 127,
- [1][1][2][1][5][13] = 127,
- [1][1][2][1][6][13] = 127,
- [1][1][2][1][9][13] = 127,
- [1][1][2][1][8][13] = 127,
- [1][1][2][1][11][13] = 127,
+ [0][0][0][0][RTW89_WW][0] = 56,
+ [0][0][0][0][RTW89_WW][1] = 56,
+ [0][0][0][0][RTW89_WW][2] = 56,
+ [0][0][0][0][RTW89_WW][3] = 56,
+ [0][0][0][0][RTW89_WW][4] = 56,
+ [0][0][0][0][RTW89_WW][5] = 56,
+ [0][0][0][0][RTW89_WW][6] = 56,
+ [0][0][0][0][RTW89_WW][7] = 56,
+ [0][0][0][0][RTW89_WW][8] = 56,
+ [0][0][0][0][RTW89_WW][9] = 56,
+ [0][0][0][0][RTW89_WW][10] = 56,
+ [0][0][0][0][RTW89_WW][11] = 56,
+ [0][0][0][0][RTW89_WW][12] = 48,
+ [0][0][0][0][RTW89_WW][13] = 76,
+ [0][1][0][0][RTW89_WW][0] = 44,
+ [0][1][0][0][RTW89_WW][1] = 44,
+ [0][1][0][0][RTW89_WW][2] = 44,
+ [0][1][0][0][RTW89_WW][3] = 44,
+ [0][1][0][0][RTW89_WW][4] = 44,
+ [0][1][0][0][RTW89_WW][5] = 44,
+ [0][1][0][0][RTW89_WW][6] = 44,
+ [0][1][0][0][RTW89_WW][7] = 44,
+ [0][1][0][0][RTW89_WW][8] = 44,
+ [0][1][0][0][RTW89_WW][9] = 44,
+ [0][1][0][0][RTW89_WW][10] = 44,
+ [0][1][0][0][RTW89_WW][11] = 44,
+ [0][1][0][0][RTW89_WW][12] = 38,
+ [0][1][0][0][RTW89_WW][13] = 64,
+ [1][0][0][0][RTW89_WW][0] = 0,
+ [1][0][0][0][RTW89_WW][1] = 0,
+ [1][0][0][0][RTW89_WW][2] = 58,
+ [1][0][0][0][RTW89_WW][3] = 58,
+ [1][0][0][0][RTW89_WW][4] = 58,
+ [1][0][0][0][RTW89_WW][5] = 58,
+ [1][0][0][0][RTW89_WW][6] = 46,
+ [1][0][0][0][RTW89_WW][7] = 46,
+ [1][0][0][0][RTW89_WW][8] = 46,
+ [1][0][0][0][RTW89_WW][9] = 32,
+ [1][0][0][0][RTW89_WW][10] = 32,
+ [1][0][0][0][RTW89_WW][11] = 0,
+ [1][0][0][0][RTW89_WW][12] = 0,
+ [1][0][0][0][RTW89_WW][13] = 0,
+ [1][1][0][0][RTW89_WW][0] = 0,
+ [1][1][0][0][RTW89_WW][1] = 0,
+ [1][1][0][0][RTW89_WW][2] = 46,
+ [1][1][0][0][RTW89_WW][3] = 46,
+ [1][1][0][0][RTW89_WW][4] = 46,
+ [1][1][0][0][RTW89_WW][5] = 46,
+ [1][1][0][0][RTW89_WW][6] = 46,
+ [1][1][0][0][RTW89_WW][7] = 46,
+ [1][1][0][0][RTW89_WW][8] = 46,
+ [1][1][0][0][RTW89_WW][9] = 24,
+ [1][1][0][0][RTW89_WW][10] = 24,
+ [1][1][0][0][RTW89_WW][11] = 0,
+ [1][1][0][0][RTW89_WW][12] = 0,
+ [1][1][0][0][RTW89_WW][13] = 0,
+ [0][0][1][0][RTW89_WW][0] = 58,
+ [0][0][1][0][RTW89_WW][1] = 58,
+ [0][0][1][0][RTW89_WW][2] = 58,
+ [0][0][1][0][RTW89_WW][3] = 58,
+ [0][0][1][0][RTW89_WW][4] = 58,
+ [0][0][1][0][RTW89_WW][5] = 58,
+ [0][0][1][0][RTW89_WW][6] = 58,
+ [0][0][1][0][RTW89_WW][7] = 58,
+ [0][0][1][0][RTW89_WW][8] = 58,
+ [0][0][1][0][RTW89_WW][9] = 58,
+ [0][0][1][0][RTW89_WW][10] = 58,
+ [0][0][1][0][RTW89_WW][11] = 56,
+ [0][0][1][0][RTW89_WW][12] = 52,
+ [0][0][1][0][RTW89_WW][13] = 0,
+ [0][1][1][0][RTW89_WW][0] = 46,
+ [0][1][1][0][RTW89_WW][1] = 46,
+ [0][1][1][0][RTW89_WW][2] = 46,
+ [0][1][1][0][RTW89_WW][3] = 46,
+ [0][1][1][0][RTW89_WW][4] = 46,
+ [0][1][1][0][RTW89_WW][5] = 46,
+ [0][1][1][0][RTW89_WW][6] = 46,
+ [0][1][1][0][RTW89_WW][7] = 46,
+ [0][1][1][0][RTW89_WW][8] = 46,
+ [0][1][1][0][RTW89_WW][9] = 46,
+ [0][1][1][0][RTW89_WW][10] = 46,
+ [0][1][1][0][RTW89_WW][11] = 42,
+ [0][1][1][0][RTW89_WW][12] = 40,
+ [0][1][1][0][RTW89_WW][13] = 0,
+ [0][0][2][0][RTW89_WW][0] = 58,
+ [0][0][2][0][RTW89_WW][1] = 58,
+ [0][0][2][0][RTW89_WW][2] = 58,
+ [0][0][2][0][RTW89_WW][3] = 58,
+ [0][0][2][0][RTW89_WW][4] = 58,
+ [0][0][2][0][RTW89_WW][5] = 58,
+ [0][0][2][0][RTW89_WW][6] = 58,
+ [0][0][2][0][RTW89_WW][7] = 58,
+ [0][0][2][0][RTW89_WW][8] = 58,
+ [0][0][2][0][RTW89_WW][9] = 58,
+ [0][0][2][0][RTW89_WW][10] = 58,
+ [0][0][2][0][RTW89_WW][11] = 54,
+ [0][0][2][0][RTW89_WW][12] = 50,
+ [0][0][2][0][RTW89_WW][13] = 0,
+ [0][1][2][0][RTW89_WW][0] = 46,
+ [0][1][2][0][RTW89_WW][1] = 46,
+ [0][1][2][0][RTW89_WW][2] = 46,
+ [0][1][2][0][RTW89_WW][3] = 46,
+ [0][1][2][0][RTW89_WW][4] = 46,
+ [0][1][2][0][RTW89_WW][5] = 46,
+ [0][1][2][0][RTW89_WW][6] = 46,
+ [0][1][2][0][RTW89_WW][7] = 46,
+ [0][1][2][0][RTW89_WW][8] = 46,
+ [0][1][2][0][RTW89_WW][9] = 46,
+ [0][1][2][0][RTW89_WW][10] = 46,
+ [0][1][2][0][RTW89_WW][11] = 42,
+ [0][1][2][0][RTW89_WW][12] = 40,
+ [0][1][2][0][RTW89_WW][13] = 0,
+ [0][1][2][1][RTW89_WW][0] = 34,
+ [0][1][2][1][RTW89_WW][1] = 34,
+ [0][1][2][1][RTW89_WW][2] = 34,
+ [0][1][2][1][RTW89_WW][3] = 34,
+ [0][1][2][1][RTW89_WW][4] = 34,
+ [0][1][2][1][RTW89_WW][5] = 34,
+ [0][1][2][1][RTW89_WW][6] = 34,
+ [0][1][2][1][RTW89_WW][7] = 34,
+ [0][1][2][1][RTW89_WW][8] = 34,
+ [0][1][2][1][RTW89_WW][9] = 34,
+ [0][1][2][1][RTW89_WW][10] = 34,
+ [0][1][2][1][RTW89_WW][11] = 34,
+ [0][1][2][1][RTW89_WW][12] = 34,
+ [0][1][2][1][RTW89_WW][13] = 0,
+ [1][0][2][0][RTW89_WW][0] = 0,
+ [1][0][2][0][RTW89_WW][1] = 0,
+ [1][0][2][0][RTW89_WW][2] = 56,
+ [1][0][2][0][RTW89_WW][3] = 56,
+ [1][0][2][0][RTW89_WW][4] = 58,
+ [1][0][2][0][RTW89_WW][5] = 58,
+ [1][0][2][0][RTW89_WW][6] = 54,
+ [1][0][2][0][RTW89_WW][7] = 50,
+ [1][0][2][0][RTW89_WW][8] = 50,
+ [1][0][2][0][RTW89_WW][9] = 42,
+ [1][0][2][0][RTW89_WW][10] = 40,
+ [1][0][2][0][RTW89_WW][11] = 0,
+ [1][0][2][0][RTW89_WW][12] = 0,
+ [1][0][2][0][RTW89_WW][13] = 0,
+ [1][1][2][0][RTW89_WW][0] = 0,
+ [1][1][2][0][RTW89_WW][1] = 0,
+ [1][1][2][0][RTW89_WW][2] = 46,
+ [1][1][2][0][RTW89_WW][3] = 46,
+ [1][1][2][0][RTW89_WW][4] = 46,
+ [1][1][2][0][RTW89_WW][5] = 46,
+ [1][1][2][0][RTW89_WW][6] = 46,
+ [1][1][2][0][RTW89_WW][7] = 46,
+ [1][1][2][0][RTW89_WW][8] = 46,
+ [1][1][2][0][RTW89_WW][9] = 38,
+ [1][1][2][0][RTW89_WW][10] = 36,
+ [1][1][2][0][RTW89_WW][11] = 0,
+ [1][1][2][0][RTW89_WW][12] = 0,
+ [1][1][2][0][RTW89_WW][13] = 0,
+ [1][1][2][1][RTW89_WW][0] = 0,
+ [1][1][2][1][RTW89_WW][1] = 0,
+ [1][1][2][1][RTW89_WW][2] = 34,
+ [1][1][2][1][RTW89_WW][3] = 34,
+ [1][1][2][1][RTW89_WW][4] = 34,
+ [1][1][2][1][RTW89_WW][5] = 34,
+ [1][1][2][1][RTW89_WW][6] = 34,
+ [1][1][2][1][RTW89_WW][7] = 34,
+ [1][1][2][1][RTW89_WW][8] = 34,
+ [1][1][2][1][RTW89_WW][9] = 34,
+ [1][1][2][1][RTW89_WW][10] = 34,
+ [1][1][2][1][RTW89_WW][11] = 0,
+ [1][1][2][1][RTW89_WW][12] = 0,
+ [1][1][2][1][RTW89_WW][13] = 0,
+ [0][0][0][0][RTW89_FCC][0] = 76,
+ [0][0][0][0][RTW89_ETSI][0] = 56,
+ [0][0][0][0][RTW89_MKK][0] = 68,
+ [0][0][0][0][RTW89_IC][0] = 76,
+ [0][0][0][0][RTW89_KCC][0] = 76,
+ [0][0][0][0][RTW89_ACMA][0] = 56,
+ [0][0][0][0][RTW89_CHILE][0] = 60,
+ [0][0][0][0][RTW89_UKRAINE][0] = 56,
+ [0][0][0][0][RTW89_MEXICO][0] = 76,
+ [0][0][0][0][RTW89_CN][0] = 56,
+ [0][0][0][0][RTW89_QATAR][0] = 56,
+ [0][0][0][0][RTW89_FCC][1] = 76,
+ [0][0][0][0][RTW89_ETSI][1] = 56,
+ [0][0][0][0][RTW89_MKK][1] = 68,
+ [0][0][0][0][RTW89_IC][1] = 76,
+ [0][0][0][0][RTW89_KCC][1] = 76,
+ [0][0][0][0][RTW89_ACMA][1] = 56,
+ [0][0][0][0][RTW89_CHILE][1] = 60,
+ [0][0][0][0][RTW89_UKRAINE][1] = 56,
+ [0][0][0][0][RTW89_MEXICO][1] = 76,
+ [0][0][0][0][RTW89_CN][1] = 56,
+ [0][0][0][0][RTW89_QATAR][1] = 56,
+ [0][0][0][0][RTW89_FCC][2] = 76,
+ [0][0][0][0][RTW89_ETSI][2] = 56,
+ [0][0][0][0][RTW89_MKK][2] = 68,
+ [0][0][0][0][RTW89_IC][2] = 76,
+ [0][0][0][0][RTW89_KCC][2] = 76,
+ [0][0][0][0][RTW89_ACMA][2] = 56,
+ [0][0][0][0][RTW89_CHILE][2] = 60,
+ [0][0][0][0][RTW89_UKRAINE][2] = 56,
+ [0][0][0][0][RTW89_MEXICO][2] = 76,
+ [0][0][0][0][RTW89_CN][2] = 56,
+ [0][0][0][0][RTW89_QATAR][2] = 56,
+ [0][0][0][0][RTW89_FCC][3] = 76,
+ [0][0][0][0][RTW89_ETSI][3] = 56,
+ [0][0][0][0][RTW89_MKK][3] = 68,
+ [0][0][0][0][RTW89_IC][3] = 76,
+ [0][0][0][0][RTW89_KCC][3] = 76,
+ [0][0][0][0][RTW89_ACMA][3] = 56,
+ [0][0][0][0][RTW89_CHILE][3] = 60,
+ [0][0][0][0][RTW89_UKRAINE][3] = 56,
+ [0][0][0][0][RTW89_MEXICO][3] = 76,
+ [0][0][0][0][RTW89_CN][3] = 56,
+ [0][0][0][0][RTW89_QATAR][3] = 56,
+ [0][0][0][0][RTW89_FCC][4] = 76,
+ [0][0][0][0][RTW89_ETSI][4] = 56,
+ [0][0][0][0][RTW89_MKK][4] = 68,
+ [0][0][0][0][RTW89_IC][4] = 76,
+ [0][0][0][0][RTW89_KCC][4] = 76,
+ [0][0][0][0][RTW89_ACMA][4] = 56,
+ [0][0][0][0][RTW89_CHILE][4] = 60,
+ [0][0][0][0][RTW89_UKRAINE][4] = 56,
+ [0][0][0][0][RTW89_MEXICO][4] = 76,
+ [0][0][0][0][RTW89_CN][4] = 56,
+ [0][0][0][0][RTW89_QATAR][4] = 56,
+ [0][0][0][0][RTW89_FCC][5] = 76,
+ [0][0][0][0][RTW89_ETSI][5] = 56,
+ [0][0][0][0][RTW89_MKK][5] = 68,
+ [0][0][0][0][RTW89_IC][5] = 76,
+ [0][0][0][0][RTW89_KCC][5] = 76,
+ [0][0][0][0][RTW89_ACMA][5] = 56,
+ [0][0][0][0][RTW89_CHILE][5] = 60,
+ [0][0][0][0][RTW89_UKRAINE][5] = 56,
+ [0][0][0][0][RTW89_MEXICO][5] = 76,
+ [0][0][0][0][RTW89_CN][5] = 56,
+ [0][0][0][0][RTW89_QATAR][5] = 56,
+ [0][0][0][0][RTW89_FCC][6] = 76,
+ [0][0][0][0][RTW89_ETSI][6] = 56,
+ [0][0][0][0][RTW89_MKK][6] = 68,
+ [0][0][0][0][RTW89_IC][6] = 76,
+ [0][0][0][0][RTW89_KCC][6] = 76,
+ [0][0][0][0][RTW89_ACMA][6] = 56,
+ [0][0][0][0][RTW89_CHILE][6] = 60,
+ [0][0][0][0][RTW89_UKRAINE][6] = 56,
+ [0][0][0][0][RTW89_MEXICO][6] = 76,
+ [0][0][0][0][RTW89_CN][6] = 56,
+ [0][0][0][0][RTW89_QATAR][6] = 56,
+ [0][0][0][0][RTW89_FCC][7] = 76,
+ [0][0][0][0][RTW89_ETSI][7] = 56,
+ [0][0][0][0][RTW89_MKK][7] = 68,
+ [0][0][0][0][RTW89_IC][7] = 76,
+ [0][0][0][0][RTW89_KCC][7] = 76,
+ [0][0][0][0][RTW89_ACMA][7] = 56,
+ [0][0][0][0][RTW89_CHILE][7] = 60,
+ [0][0][0][0][RTW89_UKRAINE][7] = 56,
+ [0][0][0][0][RTW89_MEXICO][7] = 76,
+ [0][0][0][0][RTW89_CN][7] = 56,
+ [0][0][0][0][RTW89_QATAR][7] = 56,
+ [0][0][0][0][RTW89_FCC][8] = 76,
+ [0][0][0][0][RTW89_ETSI][8] = 56,
+ [0][0][0][0][RTW89_MKK][8] = 68,
+ [0][0][0][0][RTW89_IC][8] = 76,
+ [0][0][0][0][RTW89_KCC][8] = 76,
+ [0][0][0][0][RTW89_ACMA][8] = 56,
+ [0][0][0][0][RTW89_CHILE][8] = 60,
+ [0][0][0][0][RTW89_UKRAINE][8] = 56,
+ [0][0][0][0][RTW89_MEXICO][8] = 76,
+ [0][0][0][0][RTW89_CN][8] = 56,
+ [0][0][0][0][RTW89_QATAR][8] = 56,
+ [0][0][0][0][RTW89_FCC][9] = 76,
+ [0][0][0][0][RTW89_ETSI][9] = 56,
+ [0][0][0][0][RTW89_MKK][9] = 68,
+ [0][0][0][0][RTW89_IC][9] = 76,
+ [0][0][0][0][RTW89_KCC][9] = 76,
+ [0][0][0][0][RTW89_ACMA][9] = 56,
+ [0][0][0][0][RTW89_CHILE][9] = 60,
+ [0][0][0][0][RTW89_UKRAINE][9] = 56,
+ [0][0][0][0][RTW89_MEXICO][9] = 76,
+ [0][0][0][0][RTW89_CN][9] = 56,
+ [0][0][0][0][RTW89_QATAR][9] = 56,
+ [0][0][0][0][RTW89_FCC][10] = 76,
+ [0][0][0][0][RTW89_ETSI][10] = 56,
+ [0][0][0][0][RTW89_MKK][10] = 68,
+ [0][0][0][0][RTW89_IC][10] = 76,
+ [0][0][0][0][RTW89_KCC][10] = 76,
+ [0][0][0][0][RTW89_ACMA][10] = 56,
+ [0][0][0][0][RTW89_CHILE][10] = 60,
+ [0][0][0][0][RTW89_UKRAINE][10] = 56,
+ [0][0][0][0][RTW89_MEXICO][10] = 76,
+ [0][0][0][0][RTW89_CN][10] = 56,
+ [0][0][0][0][RTW89_QATAR][10] = 56,
+ [0][0][0][0][RTW89_FCC][11] = 68,
+ [0][0][0][0][RTW89_ETSI][11] = 56,
+ [0][0][0][0][RTW89_MKK][11] = 68,
+ [0][0][0][0][RTW89_IC][11] = 68,
+ [0][0][0][0][RTW89_KCC][11] = 76,
+ [0][0][0][0][RTW89_ACMA][11] = 56,
+ [0][0][0][0][RTW89_CHILE][11] = 60,
+ [0][0][0][0][RTW89_UKRAINE][11] = 56,
+ [0][0][0][0][RTW89_MEXICO][11] = 68,
+ [0][0][0][0][RTW89_CN][11] = 56,
+ [0][0][0][0][RTW89_QATAR][11] = 56,
+ [0][0][0][0][RTW89_FCC][12] = 48,
+ [0][0][0][0][RTW89_ETSI][12] = 56,
+ [0][0][0][0][RTW89_MKK][12] = 68,
+ [0][0][0][0][RTW89_IC][12] = 48,
+ [0][0][0][0][RTW89_KCC][12] = 76,
+ [0][0][0][0][RTW89_ACMA][12] = 56,
+ [0][0][0][0][RTW89_CHILE][12] = 48,
+ [0][0][0][0][RTW89_UKRAINE][12] = 56,
+ [0][0][0][0][RTW89_MEXICO][12] = 48,
+ [0][0][0][0][RTW89_CN][12] = 56,
+ [0][0][0][0][RTW89_QATAR][12] = 56,
+ [0][0][0][0][RTW89_FCC][13] = 127,
+ [0][0][0][0][RTW89_ETSI][13] = 127,
+ [0][0][0][0][RTW89_MKK][13] = 76,
+ [0][0][0][0][RTW89_IC][13] = 127,
+ [0][0][0][0][RTW89_KCC][13] = 127,
+ [0][0][0][0][RTW89_ACMA][13] = 127,
+ [0][0][0][0][RTW89_CHILE][13] = 127,
+ [0][0][0][0][RTW89_UKRAINE][13] = 127,
+ [0][0][0][0][RTW89_MEXICO][13] = 127,
+ [0][0][0][0][RTW89_CN][13] = 127,
+ [0][0][0][0][RTW89_QATAR][13] = 127,
+ [0][1][0][0][RTW89_FCC][0] = 74,
+ [0][1][0][0][RTW89_ETSI][0] = 44,
+ [0][1][0][0][RTW89_MKK][0] = 56,
+ [0][1][0][0][RTW89_IC][0] = 74,
+ [0][1][0][0][RTW89_KCC][0] = 68,
+ [0][1][0][0][RTW89_ACMA][0] = 44,
+ [0][1][0][0][RTW89_CHILE][0] = 48,
+ [0][1][0][0][RTW89_UKRAINE][0] = 44,
+ [0][1][0][0][RTW89_MEXICO][0] = 74,
+ [0][1][0][0][RTW89_CN][0] = 44,
+ [0][1][0][0][RTW89_QATAR][0] = 44,
+ [0][1][0][0][RTW89_FCC][1] = 76,
+ [0][1][0][0][RTW89_ETSI][1] = 44,
+ [0][1][0][0][RTW89_MKK][1] = 56,
+ [0][1][0][0][RTW89_IC][1] = 76,
+ [0][1][0][0][RTW89_KCC][1] = 68,
+ [0][1][0][0][RTW89_ACMA][1] = 44,
+ [0][1][0][0][RTW89_CHILE][1] = 48,
+ [0][1][0][0][RTW89_UKRAINE][1] = 44,
+ [0][1][0][0][RTW89_MEXICO][1] = 76,
+ [0][1][0][0][RTW89_CN][1] = 44,
+ [0][1][0][0][RTW89_QATAR][1] = 44,
+ [0][1][0][0][RTW89_FCC][2] = 76,
+ [0][1][0][0][RTW89_ETSI][2] = 44,
+ [0][1][0][0][RTW89_MKK][2] = 56,
+ [0][1][0][0][RTW89_IC][2] = 76,
+ [0][1][0][0][RTW89_KCC][2] = 68,
+ [0][1][0][0][RTW89_ACMA][2] = 44,
+ [0][1][0][0][RTW89_CHILE][2] = 48,
+ [0][1][0][0][RTW89_UKRAINE][2] = 44,
+ [0][1][0][0][RTW89_MEXICO][2] = 76,
+ [0][1][0][0][RTW89_CN][2] = 44,
+ [0][1][0][0][RTW89_QATAR][2] = 44,
+ [0][1][0][0][RTW89_FCC][3] = 76,
+ [0][1][0][0][RTW89_ETSI][3] = 44,
+ [0][1][0][0][RTW89_MKK][3] = 56,
+ [0][1][0][0][RTW89_IC][3] = 76,
+ [0][1][0][0][RTW89_KCC][3] = 68,
+ [0][1][0][0][RTW89_ACMA][3] = 44,
+ [0][1][0][0][RTW89_CHILE][3] = 48,
+ [0][1][0][0][RTW89_UKRAINE][3] = 44,
+ [0][1][0][0][RTW89_MEXICO][3] = 76,
+ [0][1][0][0][RTW89_CN][3] = 44,
+ [0][1][0][0][RTW89_QATAR][3] = 44,
+ [0][1][0][0][RTW89_FCC][4] = 76,
+ [0][1][0][0][RTW89_ETSI][4] = 44,
+ [0][1][0][0][RTW89_MKK][4] = 56,
+ [0][1][0][0][RTW89_IC][4] = 76,
+ [0][1][0][0][RTW89_KCC][4] = 68,
+ [0][1][0][0][RTW89_ACMA][4] = 44,
+ [0][1][0][0][RTW89_CHILE][4] = 48,
+ [0][1][0][0][RTW89_UKRAINE][4] = 44,
+ [0][1][0][0][RTW89_MEXICO][4] = 76,
+ [0][1][0][0][RTW89_CN][4] = 44,
+ [0][1][0][0][RTW89_QATAR][4] = 44,
+ [0][1][0][0][RTW89_FCC][5] = 76,
+ [0][1][0][0][RTW89_ETSI][5] = 44,
+ [0][1][0][0][RTW89_MKK][5] = 56,
+ [0][1][0][0][RTW89_IC][5] = 76,
+ [0][1][0][0][RTW89_KCC][5] = 68,
+ [0][1][0][0][RTW89_ACMA][5] = 44,
+ [0][1][0][0][RTW89_CHILE][5] = 48,
+ [0][1][0][0][RTW89_UKRAINE][5] = 44,
+ [0][1][0][0][RTW89_MEXICO][5] = 76,
+ [0][1][0][0][RTW89_CN][5] = 44,
+ [0][1][0][0][RTW89_QATAR][5] = 44,
+ [0][1][0][0][RTW89_FCC][6] = 76,
+ [0][1][0][0][RTW89_ETSI][6] = 44,
+ [0][1][0][0][RTW89_MKK][6] = 56,
+ [0][1][0][0][RTW89_IC][6] = 76,
+ [0][1][0][0][RTW89_KCC][6] = 68,
+ [0][1][0][0][RTW89_ACMA][6] = 44,
+ [0][1][0][0][RTW89_CHILE][6] = 48,
+ [0][1][0][0][RTW89_UKRAINE][6] = 44,
+ [0][1][0][0][RTW89_MEXICO][6] = 76,
+ [0][1][0][0][RTW89_CN][6] = 44,
+ [0][1][0][0][RTW89_QATAR][6] = 44,
+ [0][1][0][0][RTW89_FCC][7] = 76,
+ [0][1][0][0][RTW89_ETSI][7] = 44,
+ [0][1][0][0][RTW89_MKK][7] = 56,
+ [0][1][0][0][RTW89_IC][7] = 76,
+ [0][1][0][0][RTW89_KCC][7] = 68,
+ [0][1][0][0][RTW89_ACMA][7] = 44,
+ [0][1][0][0][RTW89_CHILE][7] = 48,
+ [0][1][0][0][RTW89_UKRAINE][7] = 44,
+ [0][1][0][0][RTW89_MEXICO][7] = 76,
+ [0][1][0][0][RTW89_CN][7] = 44,
+ [0][1][0][0][RTW89_QATAR][7] = 44,
+ [0][1][0][0][RTW89_FCC][8] = 76,
+ [0][1][0][0][RTW89_ETSI][8] = 44,
+ [0][1][0][0][RTW89_MKK][8] = 56,
+ [0][1][0][0][RTW89_IC][8] = 76,
+ [0][1][0][0][RTW89_KCC][8] = 68,
+ [0][1][0][0][RTW89_ACMA][8] = 44,
+ [0][1][0][0][RTW89_CHILE][8] = 48,
+ [0][1][0][0][RTW89_UKRAINE][8] = 44,
+ [0][1][0][0][RTW89_MEXICO][8] = 76,
+ [0][1][0][0][RTW89_CN][8] = 44,
+ [0][1][0][0][RTW89_QATAR][8] = 44,
+ [0][1][0][0][RTW89_FCC][9] = 76,
+ [0][1][0][0][RTW89_ETSI][9] = 44,
+ [0][1][0][0][RTW89_MKK][9] = 56,
+ [0][1][0][0][RTW89_IC][9] = 76,
+ [0][1][0][0][RTW89_KCC][9] = 68,
+ [0][1][0][0][RTW89_ACMA][9] = 44,
+ [0][1][0][0][RTW89_CHILE][9] = 48,
+ [0][1][0][0][RTW89_UKRAINE][9] = 44,
+ [0][1][0][0][RTW89_MEXICO][9] = 76,
+ [0][1][0][0][RTW89_CN][9] = 44,
+ [0][1][0][0][RTW89_QATAR][9] = 44,
+ [0][1][0][0][RTW89_FCC][10] = 62,
+ [0][1][0][0][RTW89_ETSI][10] = 44,
+ [0][1][0][0][RTW89_MKK][10] = 56,
+ [0][1][0][0][RTW89_IC][10] = 62,
+ [0][1][0][0][RTW89_KCC][10] = 68,
+ [0][1][0][0][RTW89_ACMA][10] = 44,
+ [0][1][0][0][RTW89_CHILE][10] = 48,
+ [0][1][0][0][RTW89_UKRAINE][10] = 44,
+ [0][1][0][0][RTW89_MEXICO][10] = 62,
+ [0][1][0][0][RTW89_CN][10] = 44,
+ [0][1][0][0][RTW89_QATAR][10] = 44,
+ [0][1][0][0][RTW89_FCC][11] = 52,
+ [0][1][0][0][RTW89_ETSI][11] = 44,
+ [0][1][0][0][RTW89_MKK][11] = 56,
+ [0][1][0][0][RTW89_IC][11] = 52,
+ [0][1][0][0][RTW89_KCC][11] = 68,
+ [0][1][0][0][RTW89_ACMA][11] = 44,
+ [0][1][0][0][RTW89_CHILE][11] = 48,
+ [0][1][0][0][RTW89_UKRAINE][11] = 44,
+ [0][1][0][0][RTW89_MEXICO][11] = 52,
+ [0][1][0][0][RTW89_CN][11] = 44,
+ [0][1][0][0][RTW89_QATAR][11] = 44,
+ [0][1][0][0][RTW89_FCC][12] = 38,
+ [0][1][0][0][RTW89_ETSI][12] = 44,
+ [0][1][0][0][RTW89_MKK][12] = 56,
+ [0][1][0][0][RTW89_IC][12] = 38,
+ [0][1][0][0][RTW89_KCC][12] = 68,
+ [0][1][0][0][RTW89_ACMA][12] = 44,
+ [0][1][0][0][RTW89_CHILE][12] = 38,
+ [0][1][0][0][RTW89_UKRAINE][12] = 44,
+ [0][1][0][0][RTW89_MEXICO][12] = 38,
+ [0][1][0][0][RTW89_CN][12] = 44,
+ [0][1][0][0][RTW89_QATAR][12] = 44,
+ [0][1][0][0][RTW89_FCC][13] = 127,
+ [0][1][0][0][RTW89_ETSI][13] = 127,
+ [0][1][0][0][RTW89_MKK][13] = 64,
+ [0][1][0][0][RTW89_IC][13] = 127,
+ [0][1][0][0][RTW89_KCC][13] = 127,
+ [0][1][0][0][RTW89_ACMA][13] = 127,
+ [0][1][0][0][RTW89_CHILE][13] = 127,
+ [0][1][0][0][RTW89_UKRAINE][13] = 127,
+ [0][1][0][0][RTW89_MEXICO][13] = 127,
+ [0][1][0][0][RTW89_CN][13] = 127,
+ [0][1][0][0][RTW89_QATAR][13] = 127,
+ [1][0][0][0][RTW89_FCC][0] = 127,
+ [1][0][0][0][RTW89_ETSI][0] = 127,
+ [1][0][0][0][RTW89_MKK][0] = 127,
+ [1][0][0][0][RTW89_IC][0] = 127,
+ [1][0][0][0][RTW89_KCC][0] = 127,
+ [1][0][0][0][RTW89_ACMA][0] = 127,
+ [1][0][0][0][RTW89_CHILE][0] = 127,
+ [1][0][0][0][RTW89_UKRAINE][0] = 127,
+ [1][0][0][0][RTW89_MEXICO][0] = 127,
+ [1][0][0][0][RTW89_CN][0] = 127,
+ [1][0][0][0][RTW89_QATAR][0] = 127,
+ [1][0][0][0][RTW89_FCC][1] = 127,
+ [1][0][0][0][RTW89_ETSI][1] = 127,
+ [1][0][0][0][RTW89_MKK][1] = 127,
+ [1][0][0][0][RTW89_IC][1] = 127,
+ [1][0][0][0][RTW89_KCC][1] = 127,
+ [1][0][0][0][RTW89_ACMA][1] = 127,
+ [1][0][0][0][RTW89_CHILE][1] = 127,
+ [1][0][0][0][RTW89_UKRAINE][1] = 127,
+ [1][0][0][0][RTW89_MEXICO][1] = 127,
+ [1][0][0][0][RTW89_CN][1] = 127,
+ [1][0][0][0][RTW89_QATAR][1] = 127,
+ [1][0][0][0][RTW89_FCC][2] = 60,
+ [1][0][0][0][RTW89_ETSI][2] = 58,
+ [1][0][0][0][RTW89_MKK][2] = 68,
+ [1][0][0][0][RTW89_IC][2] = 60,
+ [1][0][0][0][RTW89_KCC][2] = 70,
+ [1][0][0][0][RTW89_ACMA][2] = 58,
+ [1][0][0][0][RTW89_CHILE][2] = 60,
+ [1][0][0][0][RTW89_UKRAINE][2] = 58,
+ [1][0][0][0][RTW89_MEXICO][2] = 60,
+ [1][0][0][0][RTW89_CN][2] = 58,
+ [1][0][0][0][RTW89_QATAR][2] = 58,
+ [1][0][0][0][RTW89_FCC][3] = 60,
+ [1][0][0][0][RTW89_ETSI][3] = 58,
+ [1][0][0][0][RTW89_MKK][3] = 68,
+ [1][0][0][0][RTW89_IC][3] = 60,
+ [1][0][0][0][RTW89_KCC][3] = 70,
+ [1][0][0][0][RTW89_ACMA][3] = 58,
+ [1][0][0][0][RTW89_CHILE][3] = 60,
+ [1][0][0][0][RTW89_UKRAINE][3] = 58,
+ [1][0][0][0][RTW89_MEXICO][3] = 60,
+ [1][0][0][0][RTW89_CN][3] = 58,
+ [1][0][0][0][RTW89_QATAR][3] = 58,
+ [1][0][0][0][RTW89_FCC][4] = 60,
+ [1][0][0][0][RTW89_ETSI][4] = 58,
+ [1][0][0][0][RTW89_MKK][4] = 68,
+ [1][0][0][0][RTW89_IC][4] = 60,
+ [1][0][0][0][RTW89_KCC][4] = 70,
+ [1][0][0][0][RTW89_ACMA][4] = 58,
+ [1][0][0][0][RTW89_CHILE][4] = 60,
+ [1][0][0][0][RTW89_UKRAINE][4] = 58,
+ [1][0][0][0][RTW89_MEXICO][4] = 60,
+ [1][0][0][0][RTW89_CN][4] = 58,
+ [1][0][0][0][RTW89_QATAR][4] = 58,
+ [1][0][0][0][RTW89_FCC][5] = 60,
+ [1][0][0][0][RTW89_ETSI][5] = 58,
+ [1][0][0][0][RTW89_MKK][5] = 68,
+ [1][0][0][0][RTW89_IC][5] = 60,
+ [1][0][0][0][RTW89_KCC][5] = 70,
+ [1][0][0][0][RTW89_ACMA][5] = 58,
+ [1][0][0][0][RTW89_CHILE][5] = 60,
+ [1][0][0][0][RTW89_UKRAINE][5] = 58,
+ [1][0][0][0][RTW89_MEXICO][5] = 60,
+ [1][0][0][0][RTW89_CN][5] = 58,
+ [1][0][0][0][RTW89_QATAR][5] = 58,
+ [1][0][0][0][RTW89_FCC][6] = 46,
+ [1][0][0][0][RTW89_ETSI][6] = 58,
+ [1][0][0][0][RTW89_MKK][6] = 68,
+ [1][0][0][0][RTW89_IC][6] = 46,
+ [1][0][0][0][RTW89_KCC][6] = 70,
+ [1][0][0][0][RTW89_ACMA][6] = 58,
+ [1][0][0][0][RTW89_CHILE][6] = 46,
+ [1][0][0][0][RTW89_UKRAINE][6] = 58,
+ [1][0][0][0][RTW89_MEXICO][6] = 46,
+ [1][0][0][0][RTW89_CN][6] = 58,
+ [1][0][0][0][RTW89_QATAR][6] = 58,
+ [1][0][0][0][RTW89_FCC][7] = 46,
+ [1][0][0][0][RTW89_ETSI][7] = 58,
+ [1][0][0][0][RTW89_MKK][7] = 68,
+ [1][0][0][0][RTW89_IC][7] = 46,
+ [1][0][0][0][RTW89_KCC][7] = 70,
+ [1][0][0][0][RTW89_ACMA][7] = 58,
+ [1][0][0][0][RTW89_CHILE][7] = 46,
+ [1][0][0][0][RTW89_UKRAINE][7] = 58,
+ [1][0][0][0][RTW89_MEXICO][7] = 46,
+ [1][0][0][0][RTW89_CN][7] = 58,
+ [1][0][0][0][RTW89_QATAR][7] = 58,
+ [1][0][0][0][RTW89_FCC][8] = 46,
+ [1][0][0][0][RTW89_ETSI][8] = 58,
+ [1][0][0][0][RTW89_MKK][8] = 68,
+ [1][0][0][0][RTW89_IC][8] = 46,
+ [1][0][0][0][RTW89_KCC][8] = 70,
+ [1][0][0][0][RTW89_ACMA][8] = 58,
+ [1][0][0][0][RTW89_CHILE][8] = 46,
+ [1][0][0][0][RTW89_UKRAINE][8] = 58,
+ [1][0][0][0][RTW89_MEXICO][8] = 46,
+ [1][0][0][0][RTW89_CN][8] = 58,
+ [1][0][0][0][RTW89_QATAR][8] = 58,
+ [1][0][0][0][RTW89_FCC][9] = 32,
+ [1][0][0][0][RTW89_ETSI][9] = 58,
+ [1][0][0][0][RTW89_MKK][9] = 68,
+ [1][0][0][0][RTW89_IC][9] = 32,
+ [1][0][0][0][RTW89_KCC][9] = 70,
+ [1][0][0][0][RTW89_ACMA][9] = 58,
+ [1][0][0][0][RTW89_CHILE][9] = 32,
+ [1][0][0][0][RTW89_UKRAINE][9] = 58,
+ [1][0][0][0][RTW89_MEXICO][9] = 32,
+ [1][0][0][0][RTW89_CN][9] = 58,
+ [1][0][0][0][RTW89_QATAR][9] = 58,
+ [1][0][0][0][RTW89_FCC][10] = 32,
+ [1][0][0][0][RTW89_ETSI][10] = 58,
+ [1][0][0][0][RTW89_MKK][10] = 68,
+ [1][0][0][0][RTW89_IC][10] = 32,
+ [1][0][0][0][RTW89_KCC][10] = 70,
+ [1][0][0][0][RTW89_ACMA][10] = 58,
+ [1][0][0][0][RTW89_CHILE][10] = 32,
+ [1][0][0][0][RTW89_UKRAINE][10] = 58,
+ [1][0][0][0][RTW89_MEXICO][10] = 32,
+ [1][0][0][0][RTW89_CN][10] = 58,
+ [1][0][0][0][RTW89_QATAR][10] = 58,
+ [1][0][0][0][RTW89_FCC][11] = 127,
+ [1][0][0][0][RTW89_ETSI][11] = 127,
+ [1][0][0][0][RTW89_MKK][11] = 127,
+ [1][0][0][0][RTW89_IC][11] = 127,
+ [1][0][0][0][RTW89_KCC][11] = 127,
+ [1][0][0][0][RTW89_ACMA][11] = 127,
+ [1][0][0][0][RTW89_CHILE][11] = 127,
+ [1][0][0][0][RTW89_UKRAINE][11] = 127,
+ [1][0][0][0][RTW89_MEXICO][11] = 127,
+ [1][0][0][0][RTW89_CN][11] = 127,
+ [1][0][0][0][RTW89_QATAR][11] = 127,
+ [1][0][0][0][RTW89_FCC][12] = 127,
+ [1][0][0][0][RTW89_ETSI][12] = 127,
+ [1][0][0][0][RTW89_MKK][12] = 127,
+ [1][0][0][0][RTW89_IC][12] = 127,
+ [1][0][0][0][RTW89_KCC][12] = 127,
+ [1][0][0][0][RTW89_ACMA][12] = 127,
+ [1][0][0][0][RTW89_CHILE][12] = 127,
+ [1][0][0][0][RTW89_UKRAINE][12] = 127,
+ [1][0][0][0][RTW89_MEXICO][12] = 127,
+ [1][0][0][0][RTW89_CN][12] = 127,
+ [1][0][0][0][RTW89_QATAR][12] = 127,
+ [1][0][0][0][RTW89_FCC][13] = 127,
+ [1][0][0][0][RTW89_ETSI][13] = 127,
+ [1][0][0][0][RTW89_MKK][13] = 127,
+ [1][0][0][0][RTW89_IC][13] = 127,
+ [1][0][0][0][RTW89_KCC][13] = 127,
+ [1][0][0][0][RTW89_ACMA][13] = 127,
+ [1][0][0][0][RTW89_CHILE][13] = 127,
+ [1][0][0][0][RTW89_UKRAINE][13] = 127,
+ [1][0][0][0][RTW89_MEXICO][13] = 127,
+ [1][0][0][0][RTW89_CN][13] = 127,
+ [1][0][0][0][RTW89_QATAR][13] = 127,
+ [1][1][0][0][RTW89_FCC][0] = 127,
+ [1][1][0][0][RTW89_ETSI][0] = 127,
+ [1][1][0][0][RTW89_MKK][0] = 127,
+ [1][1][0][0][RTW89_IC][0] = 127,
+ [1][1][0][0][RTW89_KCC][0] = 127,
+ [1][1][0][0][RTW89_ACMA][0] = 127,
+ [1][1][0][0][RTW89_CHILE][0] = 127,
+ [1][1][0][0][RTW89_UKRAINE][0] = 127,
+ [1][1][0][0][RTW89_MEXICO][0] = 127,
+ [1][1][0][0][RTW89_CN][0] = 127,
+ [1][1][0][0][RTW89_QATAR][0] = 127,
+ [1][1][0][0][RTW89_FCC][1] = 127,
+ [1][1][0][0][RTW89_ETSI][1] = 127,
+ [1][1][0][0][RTW89_MKK][1] = 127,
+ [1][1][0][0][RTW89_IC][1] = 127,
+ [1][1][0][0][RTW89_KCC][1] = 127,
+ [1][1][0][0][RTW89_ACMA][1] = 127,
+ [1][1][0][0][RTW89_CHILE][1] = 127,
+ [1][1][0][0][RTW89_UKRAINE][1] = 127,
+ [1][1][0][0][RTW89_MEXICO][1] = 127,
+ [1][1][0][0][RTW89_CN][1] = 127,
+ [1][1][0][0][RTW89_QATAR][1] = 127,
+ [1][1][0][0][RTW89_FCC][2] = 48,
+ [1][1][0][0][RTW89_ETSI][2] = 46,
+ [1][1][0][0][RTW89_MKK][2] = 56,
+ [1][1][0][0][RTW89_IC][2] = 48,
+ [1][1][0][0][RTW89_KCC][2] = 58,
+ [1][1][0][0][RTW89_ACMA][2] = 46,
+ [1][1][0][0][RTW89_CHILE][2] = 48,
+ [1][1][0][0][RTW89_UKRAINE][2] = 46,
+ [1][1][0][0][RTW89_MEXICO][2] = 48,
+ [1][1][0][0][RTW89_CN][2] = 46,
+ [1][1][0][0][RTW89_QATAR][2] = 46,
+ [1][1][0][0][RTW89_FCC][3] = 48,
+ [1][1][0][0][RTW89_ETSI][3] = 46,
+ [1][1][0][0][RTW89_MKK][3] = 56,
+ [1][1][0][0][RTW89_IC][3] = 48,
+ [1][1][0][0][RTW89_KCC][3] = 58,
+ [1][1][0][0][RTW89_ACMA][3] = 46,
+ [1][1][0][0][RTW89_CHILE][3] = 48,
+ [1][1][0][0][RTW89_UKRAINE][3] = 46,
+ [1][1][0][0][RTW89_MEXICO][3] = 48,
+ [1][1][0][0][RTW89_CN][3] = 46,
+ [1][1][0][0][RTW89_QATAR][3] = 46,
+ [1][1][0][0][RTW89_FCC][4] = 48,
+ [1][1][0][0][RTW89_ETSI][4] = 46,
+ [1][1][0][0][RTW89_MKK][4] = 56,
+ [1][1][0][0][RTW89_IC][4] = 48,
+ [1][1][0][0][RTW89_KCC][4] = 58,
+ [1][1][0][0][RTW89_ACMA][4] = 46,
+ [1][1][0][0][RTW89_CHILE][4] = 48,
+ [1][1][0][0][RTW89_UKRAINE][4] = 46,
+ [1][1][0][0][RTW89_MEXICO][4] = 48,
+ [1][1][0][0][RTW89_CN][4] = 46,
+ [1][1][0][0][RTW89_QATAR][4] = 46,
+ [1][1][0][0][RTW89_FCC][5] = 58,
+ [1][1][0][0][RTW89_ETSI][5] = 46,
+ [1][1][0][0][RTW89_MKK][5] = 56,
+ [1][1][0][0][RTW89_IC][5] = 58,
+ [1][1][0][0][RTW89_KCC][5] = 58,
+ [1][1][0][0][RTW89_ACMA][5] = 46,
+ [1][1][0][0][RTW89_CHILE][5] = 48,
+ [1][1][0][0][RTW89_UKRAINE][5] = 46,
+ [1][1][0][0][RTW89_MEXICO][5] = 58,
+ [1][1][0][0][RTW89_CN][5] = 46,
+ [1][1][0][0][RTW89_QATAR][5] = 46,
+ [1][1][0][0][RTW89_FCC][6] = 46,
+ [1][1][0][0][RTW89_ETSI][6] = 46,
+ [1][1][0][0][RTW89_MKK][6] = 56,
+ [1][1][0][0][RTW89_IC][6] = 46,
+ [1][1][0][0][RTW89_KCC][6] = 58,
+ [1][1][0][0][RTW89_ACMA][6] = 46,
+ [1][1][0][0][RTW89_CHILE][6] = 46,
+ [1][1][0][0][RTW89_UKRAINE][6] = 46,
+ [1][1][0][0][RTW89_MEXICO][6] = 46,
+ [1][1][0][0][RTW89_CN][6] = 46,
+ [1][1][0][0][RTW89_QATAR][6] = 46,
+ [1][1][0][0][RTW89_FCC][7] = 46,
+ [1][1][0][0][RTW89_ETSI][7] = 46,
+ [1][1][0][0][RTW89_MKK][7] = 56,
+ [1][1][0][0][RTW89_IC][7] = 46,
+ [1][1][0][0][RTW89_KCC][7] = 58,
+ [1][1][0][0][RTW89_ACMA][7] = 46,
+ [1][1][0][0][RTW89_CHILE][7] = 46,
+ [1][1][0][0][RTW89_UKRAINE][7] = 46,
+ [1][1][0][0][RTW89_MEXICO][7] = 46,
+ [1][1][0][0][RTW89_CN][7] = 46,
+ [1][1][0][0][RTW89_QATAR][7] = 46,
+ [1][1][0][0][RTW89_FCC][8] = 46,
+ [1][1][0][0][RTW89_ETSI][8] = 46,
+ [1][1][0][0][RTW89_MKK][8] = 56,
+ [1][1][0][0][RTW89_IC][8] = 46,
+ [1][1][0][0][RTW89_KCC][8] = 58,
+ [1][1][0][0][RTW89_ACMA][8] = 46,
+ [1][1][0][0][RTW89_CHILE][8] = 46,
+ [1][1][0][0][RTW89_UKRAINE][8] = 46,
+ [1][1][0][0][RTW89_MEXICO][8] = 46,
+ [1][1][0][0][RTW89_CN][8] = 46,
+ [1][1][0][0][RTW89_QATAR][8] = 46,
+ [1][1][0][0][RTW89_FCC][9] = 24,
+ [1][1][0][0][RTW89_ETSI][9] = 46,
+ [1][1][0][0][RTW89_MKK][9] = 56,
+ [1][1][0][0][RTW89_IC][9] = 24,
+ [1][1][0][0][RTW89_KCC][9] = 58,
+ [1][1][0][0][RTW89_ACMA][9] = 46,
+ [1][1][0][0][RTW89_CHILE][9] = 24,
+ [1][1][0][0][RTW89_UKRAINE][9] = 46,
+ [1][1][0][0][RTW89_MEXICO][9] = 24,
+ [1][1][0][0][RTW89_CN][9] = 46,
+ [1][1][0][0][RTW89_QATAR][9] = 46,
+ [1][1][0][0][RTW89_FCC][10] = 24,
+ [1][1][0][0][RTW89_ETSI][10] = 46,
+ [1][1][0][0][RTW89_MKK][10] = 56,
+ [1][1][0][0][RTW89_IC][10] = 24,
+ [1][1][0][0][RTW89_KCC][10] = 58,
+ [1][1][0][0][RTW89_ACMA][10] = 46,
+ [1][1][0][0][RTW89_CHILE][10] = 24,
+ [1][1][0][0][RTW89_UKRAINE][10] = 46,
+ [1][1][0][0][RTW89_MEXICO][10] = 24,
+ [1][1][0][0][RTW89_CN][10] = 46,
+ [1][1][0][0][RTW89_QATAR][10] = 46,
+ [1][1][0][0][RTW89_FCC][11] = 127,
+ [1][1][0][0][RTW89_ETSI][11] = 127,
+ [1][1][0][0][RTW89_MKK][11] = 127,
+ [1][1][0][0][RTW89_IC][11] = 127,
+ [1][1][0][0][RTW89_KCC][11] = 127,
+ [1][1][0][0][RTW89_ACMA][11] = 127,
+ [1][1][0][0][RTW89_CHILE][11] = 127,
+ [1][1][0][0][RTW89_UKRAINE][11] = 127,
+ [1][1][0][0][RTW89_MEXICO][11] = 127,
+ [1][1][0][0][RTW89_CN][11] = 127,
+ [1][1][0][0][RTW89_QATAR][11] = 127,
+ [1][1][0][0][RTW89_FCC][12] = 127,
+ [1][1][0][0][RTW89_ETSI][12] = 127,
+ [1][1][0][0][RTW89_MKK][12] = 127,
+ [1][1][0][0][RTW89_IC][12] = 127,
+ [1][1][0][0][RTW89_KCC][12] = 127,
+ [1][1][0][0][RTW89_ACMA][12] = 127,
+ [1][1][0][0][RTW89_CHILE][12] = 127,
+ [1][1][0][0][RTW89_UKRAINE][12] = 127,
+ [1][1][0][0][RTW89_MEXICO][12] = 127,
+ [1][1][0][0][RTW89_CN][12] = 127,
+ [1][1][0][0][RTW89_QATAR][12] = 127,
+ [1][1][0][0][RTW89_FCC][13] = 127,
+ [1][1][0][0][RTW89_ETSI][13] = 127,
+ [1][1][0][0][RTW89_MKK][13] = 127,
+ [1][1][0][0][RTW89_IC][13] = 127,
+ [1][1][0][0][RTW89_KCC][13] = 127,
+ [1][1][0][0][RTW89_ACMA][13] = 127,
+ [1][1][0][0][RTW89_CHILE][13] = 127,
+ [1][1][0][0][RTW89_UKRAINE][13] = 127,
+ [1][1][0][0][RTW89_MEXICO][13] = 127,
+ [1][1][0][0][RTW89_CN][13] = 127,
+ [1][1][0][0][RTW89_QATAR][13] = 127,
+ [0][0][1][0][RTW89_FCC][0] = 66,
+ [0][0][1][0][RTW89_ETSI][0] = 58,
+ [0][0][1][0][RTW89_MKK][0] = 76,
+ [0][0][1][0][RTW89_IC][0] = 66,
+ [0][0][1][0][RTW89_KCC][0] = 76,
+ [0][0][1][0][RTW89_ACMA][0] = 58,
+ [0][0][1][0][RTW89_CHILE][0] = 60,
+ [0][0][1][0][RTW89_UKRAINE][0] = 58,
+ [0][0][1][0][RTW89_MEXICO][0] = 66,
+ [0][0][1][0][RTW89_CN][0] = 58,
+ [0][0][1][0][RTW89_QATAR][0] = 58,
+ [0][0][1][0][RTW89_FCC][1] = 66,
+ [0][0][1][0][RTW89_ETSI][1] = 58,
+ [0][0][1][0][RTW89_MKK][1] = 76,
+ [0][0][1][0][RTW89_IC][1] = 66,
+ [0][0][1][0][RTW89_KCC][1] = 76,
+ [0][0][1][0][RTW89_ACMA][1] = 58,
+ [0][0][1][0][RTW89_CHILE][1] = 60,
+ [0][0][1][0][RTW89_UKRAINE][1] = 58,
+ [0][0][1][0][RTW89_MEXICO][1] = 66,
+ [0][0][1][0][RTW89_CN][1] = 58,
+ [0][0][1][0][RTW89_QATAR][1] = 58,
+ [0][0][1][0][RTW89_FCC][2] = 70,
+ [0][0][1][0][RTW89_ETSI][2] = 58,
+ [0][0][1][0][RTW89_MKK][2] = 76,
+ [0][0][1][0][RTW89_IC][2] = 70,
+ [0][0][1][0][RTW89_KCC][2] = 76,
+ [0][0][1][0][RTW89_ACMA][2] = 58,
+ [0][0][1][0][RTW89_CHILE][2] = 60,
+ [0][0][1][0][RTW89_UKRAINE][2] = 58,
+ [0][0][1][0][RTW89_MEXICO][2] = 70,
+ [0][0][1][0][RTW89_CN][2] = 58,
+ [0][0][1][0][RTW89_QATAR][2] = 58,
+ [0][0][1][0][RTW89_FCC][3] = 74,
+ [0][0][1][0][RTW89_ETSI][3] = 58,
+ [0][0][1][0][RTW89_MKK][3] = 76,
+ [0][0][1][0][RTW89_IC][3] = 74,
+ [0][0][1][0][RTW89_KCC][3] = 76,
+ [0][0][1][0][RTW89_ACMA][3] = 58,
+ [0][0][1][0][RTW89_CHILE][3] = 60,
+ [0][0][1][0][RTW89_UKRAINE][3] = 58,
+ [0][0][1][0][RTW89_MEXICO][3] = 74,
+ [0][0][1][0][RTW89_CN][3] = 58,
+ [0][0][1][0][RTW89_QATAR][3] = 58,
+ [0][0][1][0][RTW89_FCC][4] = 78,
+ [0][0][1][0][RTW89_ETSI][4] = 58,
+ [0][0][1][0][RTW89_MKK][4] = 76,
+ [0][0][1][0][RTW89_IC][4] = 78,
+ [0][0][1][0][RTW89_KCC][4] = 76,
+ [0][0][1][0][RTW89_ACMA][4] = 58,
+ [0][0][1][0][RTW89_CHILE][4] = 60,
+ [0][0][1][0][RTW89_UKRAINE][4] = 58,
+ [0][0][1][0][RTW89_MEXICO][4] = 78,
+ [0][0][1][0][RTW89_CN][4] = 58,
+ [0][0][1][0][RTW89_QATAR][4] = 58,
+ [0][0][1][0][RTW89_FCC][5] = 78,
+ [0][0][1][0][RTW89_ETSI][5] = 58,
+ [0][0][1][0][RTW89_MKK][5] = 76,
+ [0][0][1][0][RTW89_IC][5] = 78,
+ [0][0][1][0][RTW89_KCC][5] = 76,
+ [0][0][1][0][RTW89_ACMA][5] = 58,
+ [0][0][1][0][RTW89_CHILE][5] = 60,
+ [0][0][1][0][RTW89_UKRAINE][5] = 58,
+ [0][0][1][0][RTW89_MEXICO][5] = 78,
+ [0][0][1][0][RTW89_CN][5] = 58,
+ [0][0][1][0][RTW89_QATAR][5] = 58,
+ [0][0][1][0][RTW89_FCC][6] = 78,
+ [0][0][1][0][RTW89_ETSI][6] = 58,
+ [0][0][1][0][RTW89_MKK][6] = 76,
+ [0][0][1][0][RTW89_IC][6] = 78,
+ [0][0][1][0][RTW89_KCC][6] = 76,
+ [0][0][1][0][RTW89_ACMA][6] = 58,
+ [0][0][1][0][RTW89_CHILE][6] = 60,
+ [0][0][1][0][RTW89_UKRAINE][6] = 58,
+ [0][0][1][0][RTW89_MEXICO][6] = 78,
+ [0][0][1][0][RTW89_CN][6] = 58,
+ [0][0][1][0][RTW89_QATAR][6] = 58,
+ [0][0][1][0][RTW89_FCC][7] = 74,
+ [0][0][1][0][RTW89_ETSI][7] = 58,
+ [0][0][1][0][RTW89_MKK][7] = 76,
+ [0][0][1][0][RTW89_IC][7] = 74,
+ [0][0][1][0][RTW89_KCC][7] = 76,
+ [0][0][1][0][RTW89_ACMA][7] = 58,
+ [0][0][1][0][RTW89_CHILE][7] = 60,
+ [0][0][1][0][RTW89_UKRAINE][7] = 58,
+ [0][0][1][0][RTW89_MEXICO][7] = 74,
+ [0][0][1][0][RTW89_CN][7] = 58,
+ [0][0][1][0][RTW89_QATAR][7] = 58,
+ [0][0][1][0][RTW89_FCC][8] = 70,
+ [0][0][1][0][RTW89_ETSI][8] = 58,
+ [0][0][1][0][RTW89_MKK][8] = 76,
+ [0][0][1][0][RTW89_IC][8] = 70,
+ [0][0][1][0][RTW89_KCC][8] = 76,
+ [0][0][1][0][RTW89_ACMA][8] = 58,
+ [0][0][1][0][RTW89_CHILE][8] = 60,
+ [0][0][1][0][RTW89_UKRAINE][8] = 58,
+ [0][0][1][0][RTW89_MEXICO][8] = 70,
+ [0][0][1][0][RTW89_CN][8] = 58,
+ [0][0][1][0][RTW89_QATAR][8] = 58,
+ [0][0][1][0][RTW89_FCC][9] = 66,
+ [0][0][1][0][RTW89_ETSI][9] = 58,
+ [0][0][1][0][RTW89_MKK][9] = 76,
+ [0][0][1][0][RTW89_IC][9] = 66,
+ [0][0][1][0][RTW89_KCC][9] = 76,
+ [0][0][1][0][RTW89_ACMA][9] = 58,
+ [0][0][1][0][RTW89_CHILE][9] = 60,
+ [0][0][1][0][RTW89_UKRAINE][9] = 58,
+ [0][0][1][0][RTW89_MEXICO][9] = 66,
+ [0][0][1][0][RTW89_CN][9] = 58,
+ [0][0][1][0][RTW89_QATAR][9] = 58,
+ [0][0][1][0][RTW89_FCC][10] = 66,
+ [0][0][1][0][RTW89_ETSI][10] = 58,
+ [0][0][1][0][RTW89_MKK][10] = 76,
+ [0][0][1][0][RTW89_IC][10] = 66,
+ [0][0][1][0][RTW89_KCC][10] = 76,
+ [0][0][1][0][RTW89_ACMA][10] = 58,
+ [0][0][1][0][RTW89_CHILE][10] = 60,
+ [0][0][1][0][RTW89_UKRAINE][10] = 58,
+ [0][0][1][0][RTW89_MEXICO][10] = 66,
+ [0][0][1][0][RTW89_CN][10] = 58,
+ [0][0][1][0][RTW89_QATAR][10] = 58,
+ [0][0][1][0][RTW89_FCC][11] = 56,
+ [0][0][1][0][RTW89_ETSI][11] = 58,
+ [0][0][1][0][RTW89_MKK][11] = 76,
+ [0][0][1][0][RTW89_IC][11] = 56,
+ [0][0][1][0][RTW89_KCC][11] = 76,
+ [0][0][1][0][RTW89_ACMA][11] = 58,
+ [0][0][1][0][RTW89_CHILE][11] = 56,
+ [0][0][1][0][RTW89_UKRAINE][11] = 58,
+ [0][0][1][0][RTW89_MEXICO][11] = 56,
+ [0][0][1][0][RTW89_CN][11] = 58,
+ [0][0][1][0][RTW89_QATAR][11] = 58,
+ [0][0][1][0][RTW89_FCC][12] = 52,
+ [0][0][1][0][RTW89_ETSI][12] = 58,
+ [0][0][1][0][RTW89_MKK][12] = 76,
+ [0][0][1][0][RTW89_IC][12] = 52,
+ [0][0][1][0][RTW89_KCC][12] = 76,
+ [0][0][1][0][RTW89_ACMA][12] = 58,
+ [0][0][1][0][RTW89_CHILE][12] = 52,
+ [0][0][1][0][RTW89_UKRAINE][12] = 58,
+ [0][0][1][0][RTW89_MEXICO][12] = 52,
+ [0][0][1][0][RTW89_CN][12] = 58,
+ [0][0][1][0][RTW89_QATAR][12] = 58,
+ [0][0][1][0][RTW89_FCC][13] = 127,
+ [0][0][1][0][RTW89_ETSI][13] = 127,
+ [0][0][1][0][RTW89_MKK][13] = 127,
+ [0][0][1][0][RTW89_IC][13] = 127,
+ [0][0][1][0][RTW89_KCC][13] = 127,
+ [0][0][1][0][RTW89_ACMA][13] = 127,
+ [0][0][1][0][RTW89_CHILE][13] = 127,
+ [0][0][1][0][RTW89_UKRAINE][13] = 127,
+ [0][0][1][0][RTW89_MEXICO][13] = 127,
+ [0][0][1][0][RTW89_CN][13] = 127,
+ [0][0][1][0][RTW89_QATAR][13] = 127,
+ [0][1][1][0][RTW89_FCC][0] = 62,
+ [0][1][1][0][RTW89_ETSI][0] = 46,
+ [0][1][1][0][RTW89_MKK][0] = 64,
+ [0][1][1][0][RTW89_IC][0] = 62,
+ [0][1][1][0][RTW89_KCC][0] = 70,
+ [0][1][1][0][RTW89_ACMA][0] = 46,
+ [0][1][1][0][RTW89_CHILE][0] = 48,
+ [0][1][1][0][RTW89_UKRAINE][0] = 46,
+ [0][1][1][0][RTW89_MEXICO][0] = 62,
+ [0][1][1][0][RTW89_CN][0] = 46,
+ [0][1][1][0][RTW89_QATAR][0] = 46,
+ [0][1][1][0][RTW89_FCC][1] = 62,
+ [0][1][1][0][RTW89_ETSI][1] = 46,
+ [0][1][1][0][RTW89_MKK][1] = 64,
+ [0][1][1][0][RTW89_IC][1] = 62,
+ [0][1][1][0][RTW89_KCC][1] = 70,
+ [0][1][1][0][RTW89_ACMA][1] = 46,
+ [0][1][1][0][RTW89_CHILE][1] = 48,
+ [0][1][1][0][RTW89_UKRAINE][1] = 46,
+ [0][1][1][0][RTW89_MEXICO][1] = 62,
+ [0][1][1][0][RTW89_CN][1] = 46,
+ [0][1][1][0][RTW89_QATAR][1] = 46,
+ [0][1][1][0][RTW89_FCC][2] = 66,
+ [0][1][1][0][RTW89_ETSI][2] = 46,
+ [0][1][1][0][RTW89_MKK][2] = 64,
+ [0][1][1][0][RTW89_IC][2] = 66,
+ [0][1][1][0][RTW89_KCC][2] = 70,
+ [0][1][1][0][RTW89_ACMA][2] = 46,
+ [0][1][1][0][RTW89_CHILE][2] = 48,
+ [0][1][1][0][RTW89_UKRAINE][2] = 46,
+ [0][1][1][0][RTW89_MEXICO][2] = 66,
+ [0][1][1][0][RTW89_CN][2] = 46,
+ [0][1][1][0][RTW89_QATAR][2] = 46,
+ [0][1][1][0][RTW89_FCC][3] = 70,
+ [0][1][1][0][RTW89_ETSI][3] = 46,
+ [0][1][1][0][RTW89_MKK][3] = 64,
+ [0][1][1][0][RTW89_IC][3] = 70,
+ [0][1][1][0][RTW89_KCC][3] = 70,
+ [0][1][1][0][RTW89_ACMA][3] = 46,
+ [0][1][1][0][RTW89_CHILE][3] = 48,
+ [0][1][1][0][RTW89_UKRAINE][3] = 46,
+ [0][1][1][0][RTW89_MEXICO][3] = 70,
+ [0][1][1][0][RTW89_CN][3] = 46,
+ [0][1][1][0][RTW89_QATAR][3] = 46,
+ [0][1][1][0][RTW89_FCC][4] = 78,
+ [0][1][1][0][RTW89_ETSI][4] = 46,
+ [0][1][1][0][RTW89_MKK][4] = 64,
+ [0][1][1][0][RTW89_IC][4] = 78,
+ [0][1][1][0][RTW89_KCC][4] = 70,
+ [0][1][1][0][RTW89_ACMA][4] = 46,
+ [0][1][1][0][RTW89_CHILE][4] = 48,
+ [0][1][1][0][RTW89_UKRAINE][4] = 46,
+ [0][1][1][0][RTW89_MEXICO][4] = 78,
+ [0][1][1][0][RTW89_CN][4] = 46,
+ [0][1][1][0][RTW89_QATAR][4] = 46,
+ [0][1][1][0][RTW89_FCC][5] = 78,
+ [0][1][1][0][RTW89_ETSI][5] = 46,
+ [0][1][1][0][RTW89_MKK][5] = 64,
+ [0][1][1][0][RTW89_IC][5] = 78,
+ [0][1][1][0][RTW89_KCC][5] = 70,
+ [0][1][1][0][RTW89_ACMA][5] = 46,
+ [0][1][1][0][RTW89_CHILE][5] = 48,
+ [0][1][1][0][RTW89_UKRAINE][5] = 46,
+ [0][1][1][0][RTW89_MEXICO][5] = 78,
+ [0][1][1][0][RTW89_CN][5] = 46,
+ [0][1][1][0][RTW89_QATAR][5] = 46,
+ [0][1][1][0][RTW89_FCC][6] = 78,
+ [0][1][1][0][RTW89_ETSI][6] = 46,
+ [0][1][1][0][RTW89_MKK][6] = 64,
+ [0][1][1][0][RTW89_IC][6] = 78,
+ [0][1][1][0][RTW89_KCC][6] = 70,
+ [0][1][1][0][RTW89_ACMA][6] = 46,
+ [0][1][1][0][RTW89_CHILE][6] = 48,
+ [0][1][1][0][RTW89_UKRAINE][6] = 46,
+ [0][1][1][0][RTW89_MEXICO][6] = 78,
+ [0][1][1][0][RTW89_CN][6] = 46,
+ [0][1][1][0][RTW89_QATAR][6] = 46,
+ [0][1][1][0][RTW89_FCC][7] = 70,
+ [0][1][1][0][RTW89_ETSI][7] = 46,
+ [0][1][1][0][RTW89_MKK][7] = 64,
+ [0][1][1][0][RTW89_IC][7] = 70,
+ [0][1][1][0][RTW89_KCC][7] = 70,
+ [0][1][1][0][RTW89_ACMA][7] = 46,
+ [0][1][1][0][RTW89_CHILE][7] = 48,
+ [0][1][1][0][RTW89_UKRAINE][7] = 46,
+ [0][1][1][0][RTW89_MEXICO][7] = 70,
+ [0][1][1][0][RTW89_CN][7] = 46,
+ [0][1][1][0][RTW89_QATAR][7] = 46,
+ [0][1][1][0][RTW89_FCC][8] = 66,
+ [0][1][1][0][RTW89_ETSI][8] = 46,
+ [0][1][1][0][RTW89_MKK][8] = 64,
+ [0][1][1][0][RTW89_IC][8] = 66,
+ [0][1][1][0][RTW89_KCC][8] = 70,
+ [0][1][1][0][RTW89_ACMA][8] = 46,
+ [0][1][1][0][RTW89_CHILE][8] = 48,
+ [0][1][1][0][RTW89_UKRAINE][8] = 46,
+ [0][1][1][0][RTW89_MEXICO][8] = 66,
+ [0][1][1][0][RTW89_CN][8] = 46,
+ [0][1][1][0][RTW89_QATAR][8] = 46,
+ [0][1][1][0][RTW89_FCC][9] = 62,
+ [0][1][1][0][RTW89_ETSI][9] = 46,
+ [0][1][1][0][RTW89_MKK][9] = 64,
+ [0][1][1][0][RTW89_IC][9] = 62,
+ [0][1][1][0][RTW89_KCC][9] = 70,
+ [0][1][1][0][RTW89_ACMA][9] = 46,
+ [0][1][1][0][RTW89_CHILE][9] = 48,
+ [0][1][1][0][RTW89_UKRAINE][9] = 46,
+ [0][1][1][0][RTW89_MEXICO][9] = 62,
+ [0][1][1][0][RTW89_CN][9] = 46,
+ [0][1][1][0][RTW89_QATAR][9] = 46,
+ [0][1][1][0][RTW89_FCC][10] = 62,
+ [0][1][1][0][RTW89_ETSI][10] = 46,
+ [0][1][1][0][RTW89_MKK][10] = 64,
+ [0][1][1][0][RTW89_IC][10] = 62,
+ [0][1][1][0][RTW89_KCC][10] = 70,
+ [0][1][1][0][RTW89_ACMA][10] = 46,
+ [0][1][1][0][RTW89_CHILE][10] = 48,
+ [0][1][1][0][RTW89_UKRAINE][10] = 46,
+ [0][1][1][0][RTW89_MEXICO][10] = 62,
+ [0][1][1][0][RTW89_CN][10] = 46,
+ [0][1][1][0][RTW89_QATAR][10] = 46,
+ [0][1][1][0][RTW89_FCC][11] = 42,
+ [0][1][1][0][RTW89_ETSI][11] = 46,
+ [0][1][1][0][RTW89_MKK][11] = 64,
+ [0][1][1][0][RTW89_IC][11] = 42,
+ [0][1][1][0][RTW89_KCC][11] = 70,
+ [0][1][1][0][RTW89_ACMA][11] = 46,
+ [0][1][1][0][RTW89_CHILE][11] = 42,
+ [0][1][1][0][RTW89_UKRAINE][11] = 46,
+ [0][1][1][0][RTW89_MEXICO][11] = 42,
+ [0][1][1][0][RTW89_CN][11] = 46,
+ [0][1][1][0][RTW89_QATAR][11] = 46,
+ [0][1][1][0][RTW89_FCC][12] = 40,
+ [0][1][1][0][RTW89_ETSI][12] = 46,
+ [0][1][1][0][RTW89_MKK][12] = 64,
+ [0][1][1][0][RTW89_IC][12] = 40,
+ [0][1][1][0][RTW89_KCC][12] = 70,
+ [0][1][1][0][RTW89_ACMA][12] = 46,
+ [0][1][1][0][RTW89_CHILE][12] = 40,
+ [0][1][1][0][RTW89_UKRAINE][12] = 46,
+ [0][1][1][0][RTW89_MEXICO][12] = 40,
+ [0][1][1][0][RTW89_CN][12] = 46,
+ [0][1][1][0][RTW89_QATAR][12] = 46,
+ [0][1][1][0][RTW89_FCC][13] = 127,
+ [0][1][1][0][RTW89_ETSI][13] = 127,
+ [0][1][1][0][RTW89_MKK][13] = 127,
+ [0][1][1][0][RTW89_IC][13] = 127,
+ [0][1][1][0][RTW89_KCC][13] = 127,
+ [0][1][1][0][RTW89_ACMA][13] = 127,
+ [0][1][1][0][RTW89_CHILE][13] = 127,
+ [0][1][1][0][RTW89_UKRAINE][13] = 127,
+ [0][1][1][0][RTW89_MEXICO][13] = 127,
+ [0][1][1][0][RTW89_CN][13] = 127,
+ [0][1][1][0][RTW89_QATAR][13] = 127,
+ [0][0][2][0][RTW89_FCC][0] = 66,
+ [0][0][2][0][RTW89_ETSI][0] = 58,
+ [0][0][2][0][RTW89_MKK][0] = 76,
+ [0][0][2][0][RTW89_IC][0] = 66,
+ [0][0][2][0][RTW89_KCC][0] = 76,
+ [0][0][2][0][RTW89_ACMA][0] = 58,
+ [0][0][2][0][RTW89_CHILE][0] = 60,
+ [0][0][2][0][RTW89_UKRAINE][0] = 58,
+ [0][0][2][0][RTW89_MEXICO][0] = 66,
+ [0][0][2][0][RTW89_CN][0] = 58,
+ [0][0][2][0][RTW89_QATAR][0] = 58,
+ [0][0][2][0][RTW89_FCC][1] = 66,
+ [0][0][2][0][RTW89_ETSI][1] = 58,
+ [0][0][2][0][RTW89_MKK][1] = 76,
+ [0][0][2][0][RTW89_IC][1] = 66,
+ [0][0][2][0][RTW89_KCC][1] = 76,
+ [0][0][2][0][RTW89_ACMA][1] = 58,
+ [0][0][2][0][RTW89_CHILE][1] = 60,
+ [0][0][2][0][RTW89_UKRAINE][1] = 58,
+ [0][0][2][0][RTW89_MEXICO][1] = 66,
+ [0][0][2][0][RTW89_CN][1] = 58,
+ [0][0][2][0][RTW89_QATAR][1] = 58,
+ [0][0][2][0][RTW89_FCC][2] = 70,
+ [0][0][2][0][RTW89_ETSI][2] = 58,
+ [0][0][2][0][RTW89_MKK][2] = 76,
+ [0][0][2][0][RTW89_IC][2] = 70,
+ [0][0][2][0][RTW89_KCC][2] = 76,
+ [0][0][2][0][RTW89_ACMA][2] = 58,
+ [0][0][2][0][RTW89_CHILE][2] = 60,
+ [0][0][2][0][RTW89_UKRAINE][2] = 58,
+ [0][0][2][0][RTW89_MEXICO][2] = 70,
+ [0][0][2][0][RTW89_CN][2] = 58,
+ [0][0][2][0][RTW89_QATAR][2] = 58,
+ [0][0][2][0][RTW89_FCC][3] = 74,
+ [0][0][2][0][RTW89_ETSI][3] = 58,
+ [0][0][2][0][RTW89_MKK][3] = 76,
+ [0][0][2][0][RTW89_IC][3] = 74,
+ [0][0][2][0][RTW89_KCC][3] = 76,
+ [0][0][2][0][RTW89_ACMA][3] = 58,
+ [0][0][2][0][RTW89_CHILE][3] = 60,
+ [0][0][2][0][RTW89_UKRAINE][3] = 58,
+ [0][0][2][0][RTW89_MEXICO][3] = 74,
+ [0][0][2][0][RTW89_CN][3] = 58,
+ [0][0][2][0][RTW89_QATAR][3] = 58,
+ [0][0][2][0][RTW89_FCC][4] = 76,
+ [0][0][2][0][RTW89_ETSI][4] = 58,
+ [0][0][2][0][RTW89_MKK][4] = 76,
+ [0][0][2][0][RTW89_IC][4] = 76,
+ [0][0][2][0][RTW89_KCC][4] = 76,
+ [0][0][2][0][RTW89_ACMA][4] = 58,
+ [0][0][2][0][RTW89_CHILE][4] = 60,
+ [0][0][2][0][RTW89_UKRAINE][4] = 58,
+ [0][0][2][0][RTW89_MEXICO][4] = 76,
+ [0][0][2][0][RTW89_CN][4] = 58,
+ [0][0][2][0][RTW89_QATAR][4] = 58,
+ [0][0][2][0][RTW89_FCC][5] = 76,
+ [0][0][2][0][RTW89_ETSI][5] = 58,
+ [0][0][2][0][RTW89_MKK][5] = 76,
+ [0][0][2][0][RTW89_IC][5] = 76,
+ [0][0][2][0][RTW89_KCC][5] = 76,
+ [0][0][2][0][RTW89_ACMA][5] = 58,
+ [0][0][2][0][RTW89_CHILE][5] = 60,
+ [0][0][2][0][RTW89_UKRAINE][5] = 58,
+ [0][0][2][0][RTW89_MEXICO][5] = 76,
+ [0][0][2][0][RTW89_CN][5] = 58,
+ [0][0][2][0][RTW89_QATAR][5] = 58,
+ [0][0][2][0][RTW89_FCC][6] = 76,
+ [0][0][2][0][RTW89_ETSI][6] = 58,
+ [0][0][2][0][RTW89_MKK][6] = 76,
+ [0][0][2][0][RTW89_IC][6] = 76,
+ [0][0][2][0][RTW89_KCC][6] = 76,
+ [0][0][2][0][RTW89_ACMA][6] = 58,
+ [0][0][2][0][RTW89_CHILE][6] = 60,
+ [0][0][2][0][RTW89_UKRAINE][6] = 58,
+ [0][0][2][0][RTW89_MEXICO][6] = 76,
+ [0][0][2][0][RTW89_CN][6] = 58,
+ [0][0][2][0][RTW89_QATAR][6] = 58,
+ [0][0][2][0][RTW89_FCC][7] = 74,
+ [0][0][2][0][RTW89_ETSI][7] = 58,
+ [0][0][2][0][RTW89_MKK][7] = 76,
+ [0][0][2][0][RTW89_IC][7] = 74,
+ [0][0][2][0][RTW89_KCC][7] = 76,
+ [0][0][2][0][RTW89_ACMA][7] = 58,
+ [0][0][2][0][RTW89_CHILE][7] = 60,
+ [0][0][2][0][RTW89_UKRAINE][7] = 58,
+ [0][0][2][0][RTW89_MEXICO][7] = 74,
+ [0][0][2][0][RTW89_CN][7] = 58,
+ [0][0][2][0][RTW89_QATAR][7] = 58,
+ [0][0][2][0][RTW89_FCC][8] = 70,
+ [0][0][2][0][RTW89_ETSI][8] = 58,
+ [0][0][2][0][RTW89_MKK][8] = 76,
+ [0][0][2][0][RTW89_IC][8] = 70,
+ [0][0][2][0][RTW89_KCC][8] = 76,
+ [0][0][2][0][RTW89_ACMA][8] = 58,
+ [0][0][2][0][RTW89_CHILE][8] = 60,
+ [0][0][2][0][RTW89_UKRAINE][8] = 58,
+ [0][0][2][0][RTW89_MEXICO][8] = 70,
+ [0][0][2][0][RTW89_CN][8] = 58,
+ [0][0][2][0][RTW89_QATAR][8] = 58,
+ [0][0][2][0][RTW89_FCC][9] = 66,
+ [0][0][2][0][RTW89_ETSI][9] = 58,
+ [0][0][2][0][RTW89_MKK][9] = 76,
+ [0][0][2][0][RTW89_IC][9] = 66,
+ [0][0][2][0][RTW89_KCC][9] = 76,
+ [0][0][2][0][RTW89_ACMA][9] = 58,
+ [0][0][2][0][RTW89_CHILE][9] = 60,
+ [0][0][2][0][RTW89_UKRAINE][9] = 58,
+ [0][0][2][0][RTW89_MEXICO][9] = 66,
+ [0][0][2][0][RTW89_CN][9] = 58,
+ [0][0][2][0][RTW89_QATAR][9] = 58,
+ [0][0][2][0][RTW89_FCC][10] = 66,
+ [0][0][2][0][RTW89_ETSI][10] = 58,
+ [0][0][2][0][RTW89_MKK][10] = 76,
+ [0][0][2][0][RTW89_IC][10] = 66,
+ [0][0][2][0][RTW89_KCC][10] = 76,
+ [0][0][2][0][RTW89_ACMA][10] = 58,
+ [0][0][2][0][RTW89_CHILE][10] = 60,
+ [0][0][2][0][RTW89_UKRAINE][10] = 58,
+ [0][0][2][0][RTW89_MEXICO][10] = 66,
+ [0][0][2][0][RTW89_CN][10] = 58,
+ [0][0][2][0][RTW89_QATAR][10] = 58,
+ [0][0][2][0][RTW89_FCC][11] = 54,
+ [0][0][2][0][RTW89_ETSI][11] = 58,
+ [0][0][2][0][RTW89_MKK][11] = 76,
+ [0][0][2][0][RTW89_IC][11] = 54,
+ [0][0][2][0][RTW89_KCC][11] = 76,
+ [0][0][2][0][RTW89_ACMA][11] = 58,
+ [0][0][2][0][RTW89_CHILE][11] = 54,
+ [0][0][2][0][RTW89_UKRAINE][11] = 58,
+ [0][0][2][0][RTW89_MEXICO][11] = 54,
+ [0][0][2][0][RTW89_CN][11] = 58,
+ [0][0][2][0][RTW89_QATAR][11] = 58,
+ [0][0][2][0][RTW89_FCC][12] = 50,
+ [0][0][2][0][RTW89_ETSI][12] = 58,
+ [0][0][2][0][RTW89_MKK][12] = 76,
+ [0][0][2][0][RTW89_IC][12] = 50,
+ [0][0][2][0][RTW89_KCC][12] = 74,
+ [0][0][2][0][RTW89_ACMA][12] = 58,
+ [0][0][2][0][RTW89_CHILE][12] = 50,
+ [0][0][2][0][RTW89_UKRAINE][12] = 58,
+ [0][0][2][0][RTW89_MEXICO][12] = 50,
+ [0][0][2][0][RTW89_CN][12] = 58,
+ [0][0][2][0][RTW89_QATAR][12] = 58,
+ [0][0][2][0][RTW89_FCC][13] = 127,
+ [0][0][2][0][RTW89_ETSI][13] = 127,
+ [0][0][2][0][RTW89_MKK][13] = 127,
+ [0][0][2][0][RTW89_IC][13] = 127,
+ [0][0][2][0][RTW89_KCC][13] = 127,
+ [0][0][2][0][RTW89_ACMA][13] = 127,
+ [0][0][2][0][RTW89_CHILE][13] = 127,
+ [0][0][2][0][RTW89_UKRAINE][13] = 127,
+ [0][0][2][0][RTW89_MEXICO][13] = 127,
+ [0][0][2][0][RTW89_CN][13] = 127,
+ [0][0][2][0][RTW89_QATAR][13] = 127,
+ [0][1][2][0][RTW89_FCC][0] = 62,
+ [0][1][2][0][RTW89_ETSI][0] = 46,
+ [0][1][2][0][RTW89_MKK][0] = 64,
+ [0][1][2][0][RTW89_IC][0] = 62,
+ [0][1][2][0][RTW89_KCC][0] = 68,
+ [0][1][2][0][RTW89_ACMA][0] = 46,
+ [0][1][2][0][RTW89_CHILE][0] = 48,
+ [0][1][2][0][RTW89_UKRAINE][0] = 46,
+ [0][1][2][0][RTW89_MEXICO][0] = 62,
+ [0][1][2][0][RTW89_CN][0] = 46,
+ [0][1][2][0][RTW89_QATAR][0] = 46,
+ [0][1][2][0][RTW89_FCC][1] = 62,
+ [0][1][2][0][RTW89_ETSI][1] = 46,
+ [0][1][2][0][RTW89_MKK][1] = 64,
+ [0][1][2][0][RTW89_IC][1] = 62,
+ [0][1][2][0][RTW89_KCC][1] = 70,
+ [0][1][2][0][RTW89_ACMA][1] = 46,
+ [0][1][2][0][RTW89_CHILE][1] = 48,
+ [0][1][2][0][RTW89_UKRAINE][1] = 46,
+ [0][1][2][0][RTW89_MEXICO][1] = 62,
+ [0][1][2][0][RTW89_CN][1] = 46,
+ [0][1][2][0][RTW89_QATAR][1] = 46,
+ [0][1][2][0][RTW89_FCC][2] = 66,
+ [0][1][2][0][RTW89_ETSI][2] = 46,
+ [0][1][2][0][RTW89_MKK][2] = 64,
+ [0][1][2][0][RTW89_IC][2] = 66,
+ [0][1][2][0][RTW89_KCC][2] = 70,
+ [0][1][2][0][RTW89_ACMA][2] = 46,
+ [0][1][2][0][RTW89_CHILE][2] = 48,
+ [0][1][2][0][RTW89_UKRAINE][2] = 46,
+ [0][1][2][0][RTW89_MEXICO][2] = 66,
+ [0][1][2][0][RTW89_CN][2] = 46,
+ [0][1][2][0][RTW89_QATAR][2] = 46,
+ [0][1][2][0][RTW89_FCC][3] = 70,
+ [0][1][2][0][RTW89_ETSI][3] = 46,
+ [0][1][2][0][RTW89_MKK][3] = 64,
+ [0][1][2][0][RTW89_IC][3] = 70,
+ [0][1][2][0][RTW89_KCC][3] = 70,
+ [0][1][2][0][RTW89_ACMA][3] = 46,
+ [0][1][2][0][RTW89_CHILE][3] = 48,
+ [0][1][2][0][RTW89_UKRAINE][3] = 46,
+ [0][1][2][0][RTW89_MEXICO][3] = 70,
+ [0][1][2][0][RTW89_CN][3] = 46,
+ [0][1][2][0][RTW89_QATAR][3] = 46,
+ [0][1][2][0][RTW89_FCC][4] = 76,
+ [0][1][2][0][RTW89_ETSI][4] = 46,
+ [0][1][2][0][RTW89_MKK][4] = 64,
+ [0][1][2][0][RTW89_IC][4] = 76,
+ [0][1][2][0][RTW89_KCC][4] = 70,
+ [0][1][2][0][RTW89_ACMA][4] = 46,
+ [0][1][2][0][RTW89_CHILE][4] = 48,
+ [0][1][2][0][RTW89_UKRAINE][4] = 46,
+ [0][1][2][0][RTW89_MEXICO][4] = 76,
+ [0][1][2][0][RTW89_CN][4] = 46,
+ [0][1][2][0][RTW89_QATAR][4] = 46,
+ [0][1][2][0][RTW89_FCC][5] = 76,
+ [0][1][2][0][RTW89_ETSI][5] = 46,
+ [0][1][2][0][RTW89_MKK][5] = 64,
+ [0][1][2][0][RTW89_IC][5] = 76,
+ [0][1][2][0][RTW89_KCC][5] = 70,
+ [0][1][2][0][RTW89_ACMA][5] = 46,
+ [0][1][2][0][RTW89_CHILE][5] = 48,
+ [0][1][2][0][RTW89_UKRAINE][5] = 46,
+ [0][1][2][0][RTW89_MEXICO][5] = 76,
+ [0][1][2][0][RTW89_CN][5] = 46,
+ [0][1][2][0][RTW89_QATAR][5] = 46,
+ [0][1][2][0][RTW89_FCC][6] = 76,
+ [0][1][2][0][RTW89_ETSI][6] = 46,
+ [0][1][2][0][RTW89_MKK][6] = 64,
+ [0][1][2][0][RTW89_IC][6] = 76,
+ [0][1][2][0][RTW89_KCC][6] = 70,
+ [0][1][2][0][RTW89_ACMA][6] = 46,
+ [0][1][2][0][RTW89_CHILE][6] = 48,
+ [0][1][2][0][RTW89_UKRAINE][6] = 46,
+ [0][1][2][0][RTW89_MEXICO][6] = 76,
+ [0][1][2][0][RTW89_CN][6] = 46,
+ [0][1][2][0][RTW89_QATAR][6] = 46,
+ [0][1][2][0][RTW89_FCC][7] = 68,
+ [0][1][2][0][RTW89_ETSI][7] = 46,
+ [0][1][2][0][RTW89_MKK][7] = 64,
+ [0][1][2][0][RTW89_IC][7] = 68,
+ [0][1][2][0][RTW89_KCC][7] = 70,
+ [0][1][2][0][RTW89_ACMA][7] = 46,
+ [0][1][2][0][RTW89_CHILE][7] = 48,
+ [0][1][2][0][RTW89_UKRAINE][7] = 46,
+ [0][1][2][0][RTW89_MEXICO][7] = 68,
+ [0][1][2][0][RTW89_CN][7] = 46,
+ [0][1][2][0][RTW89_QATAR][7] = 46,
+ [0][1][2][0][RTW89_FCC][8] = 64,
+ [0][1][2][0][RTW89_ETSI][8] = 46,
+ [0][1][2][0][RTW89_MKK][8] = 64,
+ [0][1][2][0][RTW89_IC][8] = 64,
+ [0][1][2][0][RTW89_KCC][8] = 70,
+ [0][1][2][0][RTW89_ACMA][8] = 46,
+ [0][1][2][0][RTW89_CHILE][8] = 48,
+ [0][1][2][0][RTW89_UKRAINE][8] = 46,
+ [0][1][2][0][RTW89_MEXICO][8] = 64,
+ [0][1][2][0][RTW89_CN][8] = 46,
+ [0][1][2][0][RTW89_QATAR][8] = 46,
+ [0][1][2][0][RTW89_FCC][9] = 60,
+ [0][1][2][0][RTW89_ETSI][9] = 46,
+ [0][1][2][0][RTW89_MKK][9] = 64,
+ [0][1][2][0][RTW89_IC][9] = 60,
+ [0][1][2][0][RTW89_KCC][9] = 70,
+ [0][1][2][0][RTW89_ACMA][9] = 46,
+ [0][1][2][0][RTW89_CHILE][9] = 48,
+ [0][1][2][0][RTW89_UKRAINE][9] = 46,
+ [0][1][2][0][RTW89_MEXICO][9] = 60,
+ [0][1][2][0][RTW89_CN][9] = 46,
+ [0][1][2][0][RTW89_QATAR][9] = 46,
+ [0][1][2][0][RTW89_FCC][10] = 60,
+ [0][1][2][0][RTW89_ETSI][10] = 46,
+ [0][1][2][0][RTW89_MKK][10] = 64,
+ [0][1][2][0][RTW89_IC][10] = 60,
+ [0][1][2][0][RTW89_KCC][10] = 70,
+ [0][1][2][0][RTW89_ACMA][10] = 46,
+ [0][1][2][0][RTW89_CHILE][10] = 48,
+ [0][1][2][0][RTW89_UKRAINE][10] = 46,
+ [0][1][2][0][RTW89_MEXICO][10] = 60,
+ [0][1][2][0][RTW89_CN][10] = 46,
+ [0][1][2][0][RTW89_QATAR][10] = 46,
+ [0][1][2][0][RTW89_FCC][11] = 42,
+ [0][1][2][0][RTW89_ETSI][11] = 46,
+ [0][1][2][0][RTW89_MKK][11] = 64,
+ [0][1][2][0][RTW89_IC][11] = 42,
+ [0][1][2][0][RTW89_KCC][11] = 70,
+ [0][1][2][0][RTW89_ACMA][11] = 46,
+ [0][1][2][0][RTW89_CHILE][11] = 42,
+ [0][1][2][0][RTW89_UKRAINE][11] = 46,
+ [0][1][2][0][RTW89_MEXICO][11] = 42,
+ [0][1][2][0][RTW89_CN][11] = 46,
+ [0][1][2][0][RTW89_QATAR][11] = 46,
+ [0][1][2][0][RTW89_FCC][12] = 40,
+ [0][1][2][0][RTW89_ETSI][12] = 46,
+ [0][1][2][0][RTW89_MKK][12] = 64,
+ [0][1][2][0][RTW89_IC][12] = 40,
+ [0][1][2][0][RTW89_KCC][12] = 68,
+ [0][1][2][0][RTW89_ACMA][12] = 46,
+ [0][1][2][0][RTW89_CHILE][12] = 40,
+ [0][1][2][0][RTW89_UKRAINE][12] = 46,
+ [0][1][2][0][RTW89_MEXICO][12] = 40,
+ [0][1][2][0][RTW89_CN][12] = 46,
+ [0][1][2][0][RTW89_QATAR][12] = 46,
+ [0][1][2][0][RTW89_FCC][13] = 127,
+ [0][1][2][0][RTW89_ETSI][13] = 127,
+ [0][1][2][0][RTW89_MKK][13] = 127,
+ [0][1][2][0][RTW89_IC][13] = 127,
+ [0][1][2][0][RTW89_KCC][13] = 127,
+ [0][1][2][0][RTW89_ACMA][13] = 127,
+ [0][1][2][0][RTW89_CHILE][13] = 127,
+ [0][1][2][0][RTW89_UKRAINE][13] = 127,
+ [0][1][2][0][RTW89_MEXICO][13] = 127,
+ [0][1][2][0][RTW89_CN][13] = 127,
+ [0][1][2][0][RTW89_QATAR][13] = 127,
+ [0][1][2][1][RTW89_FCC][0] = 62,
+ [0][1][2][1][RTW89_ETSI][0] = 34,
+ [0][1][2][1][RTW89_MKK][0] = 64,
+ [0][1][2][1][RTW89_IC][0] = 62,
+ [0][1][2][1][RTW89_KCC][0] = 68,
+ [0][1][2][1][RTW89_ACMA][0] = 34,
+ [0][1][2][1][RTW89_CHILE][0] = 36,
+ [0][1][2][1][RTW89_UKRAINE][0] = 34,
+ [0][1][2][1][RTW89_MEXICO][0] = 62,
+ [0][1][2][1][RTW89_CN][0] = 34,
+ [0][1][2][1][RTW89_QATAR][0] = 34,
+ [0][1][2][1][RTW89_FCC][1] = 62,
+ [0][1][2][1][RTW89_ETSI][1] = 34,
+ [0][1][2][1][RTW89_MKK][1] = 64,
+ [0][1][2][1][RTW89_IC][1] = 62,
+ [0][1][2][1][RTW89_KCC][1] = 70,
+ [0][1][2][1][RTW89_ACMA][1] = 34,
+ [0][1][2][1][RTW89_CHILE][1] = 36,
+ [0][1][2][1][RTW89_UKRAINE][1] = 34,
+ [0][1][2][1][RTW89_MEXICO][1] = 62,
+ [0][1][2][1][RTW89_CN][1] = 34,
+ [0][1][2][1][RTW89_QATAR][1] = 34,
+ [0][1][2][1][RTW89_FCC][2] = 66,
+ [0][1][2][1][RTW89_ETSI][2] = 34,
+ [0][1][2][1][RTW89_MKK][2] = 64,
+ [0][1][2][1][RTW89_IC][2] = 66,
+ [0][1][2][1][RTW89_KCC][2] = 70,
+ [0][1][2][1][RTW89_ACMA][2] = 34,
+ [0][1][2][1][RTW89_CHILE][2] = 36,
+ [0][1][2][1][RTW89_UKRAINE][2] = 34,
+ [0][1][2][1][RTW89_MEXICO][2] = 66,
+ [0][1][2][1][RTW89_CN][2] = 34,
+ [0][1][2][1][RTW89_QATAR][2] = 34,
+ [0][1][2][1][RTW89_FCC][3] = 70,
+ [0][1][2][1][RTW89_ETSI][3] = 34,
+ [0][1][2][1][RTW89_MKK][3] = 64,
+ [0][1][2][1][RTW89_IC][3] = 70,
+ [0][1][2][1][RTW89_KCC][3] = 70,
+ [0][1][2][1][RTW89_ACMA][3] = 34,
+ [0][1][2][1][RTW89_CHILE][3] = 36,
+ [0][1][2][1][RTW89_UKRAINE][3] = 34,
+ [0][1][2][1][RTW89_MEXICO][3] = 70,
+ [0][1][2][1][RTW89_CN][3] = 34,
+ [0][1][2][1][RTW89_QATAR][3] = 34,
+ [0][1][2][1][RTW89_FCC][4] = 76,
+ [0][1][2][1][RTW89_ETSI][4] = 34,
+ [0][1][2][1][RTW89_MKK][4] = 64,
+ [0][1][2][1][RTW89_IC][4] = 76,
+ [0][1][2][1][RTW89_KCC][4] = 70,
+ [0][1][2][1][RTW89_ACMA][4] = 34,
+ [0][1][2][1][RTW89_CHILE][4] = 36,
+ [0][1][2][1][RTW89_UKRAINE][4] = 34,
+ [0][1][2][1][RTW89_MEXICO][4] = 76,
+ [0][1][2][1][RTW89_CN][4] = 34,
+ [0][1][2][1][RTW89_QATAR][4] = 34,
+ [0][1][2][1][RTW89_FCC][5] = 76,
+ [0][1][2][1][RTW89_ETSI][5] = 34,
+ [0][1][2][1][RTW89_MKK][5] = 64,
+ [0][1][2][1][RTW89_IC][5] = 76,
+ [0][1][2][1][RTW89_KCC][5] = 70,
+ [0][1][2][1][RTW89_ACMA][5] = 34,
+ [0][1][2][1][RTW89_CHILE][5] = 36,
+ [0][1][2][1][RTW89_UKRAINE][5] = 34,
+ [0][1][2][1][RTW89_MEXICO][5] = 76,
+ [0][1][2][1][RTW89_CN][5] = 34,
+ [0][1][2][1][RTW89_QATAR][5] = 34,
+ [0][1][2][1][RTW89_FCC][6] = 76,
+ [0][1][2][1][RTW89_ETSI][6] = 34,
+ [0][1][2][1][RTW89_MKK][6] = 64,
+ [0][1][2][1][RTW89_IC][6] = 76,
+ [0][1][2][1][RTW89_KCC][6] = 70,
+ [0][1][2][1][RTW89_ACMA][6] = 34,
+ [0][1][2][1][RTW89_CHILE][6] = 36,
+ [0][1][2][1][RTW89_UKRAINE][6] = 34,
+ [0][1][2][1][RTW89_MEXICO][6] = 76,
+ [0][1][2][1][RTW89_CN][6] = 34,
+ [0][1][2][1][RTW89_QATAR][6] = 34,
+ [0][1][2][1][RTW89_FCC][7] = 68,
+ [0][1][2][1][RTW89_ETSI][7] = 34,
+ [0][1][2][1][RTW89_MKK][7] = 64,
+ [0][1][2][1][RTW89_IC][7] = 68,
+ [0][1][2][1][RTW89_KCC][7] = 70,
+ [0][1][2][1][RTW89_ACMA][7] = 34,
+ [0][1][2][1][RTW89_CHILE][7] = 36,
+ [0][1][2][1][RTW89_UKRAINE][7] = 34,
+ [0][1][2][1][RTW89_MEXICO][7] = 68,
+ [0][1][2][1][RTW89_CN][7] = 34,
+ [0][1][2][1][RTW89_QATAR][7] = 34,
+ [0][1][2][1][RTW89_FCC][8] = 64,
+ [0][1][2][1][RTW89_ETSI][8] = 34,
+ [0][1][2][1][RTW89_MKK][8] = 64,
+ [0][1][2][1][RTW89_IC][8] = 64,
+ [0][1][2][1][RTW89_KCC][8] = 70,
+ [0][1][2][1][RTW89_ACMA][8] = 34,
+ [0][1][2][1][RTW89_CHILE][8] = 36,
+ [0][1][2][1][RTW89_UKRAINE][8] = 34,
+ [0][1][2][1][RTW89_MEXICO][8] = 64,
+ [0][1][2][1][RTW89_CN][8] = 34,
+ [0][1][2][1][RTW89_QATAR][8] = 34,
+ [0][1][2][1][RTW89_FCC][9] = 60,
+ [0][1][2][1][RTW89_ETSI][9] = 34,
+ [0][1][2][1][RTW89_MKK][9] = 64,
+ [0][1][2][1][RTW89_IC][9] = 60,
+ [0][1][2][1][RTW89_KCC][9] = 70,
+ [0][1][2][1][RTW89_ACMA][9] = 34,
+ [0][1][2][1][RTW89_CHILE][9] = 36,
+ [0][1][2][1][RTW89_UKRAINE][9] = 34,
+ [0][1][2][1][RTW89_MEXICO][9] = 60,
+ [0][1][2][1][RTW89_CN][9] = 34,
+ [0][1][2][1][RTW89_QATAR][9] = 34,
+ [0][1][2][1][RTW89_FCC][10] = 60,
+ [0][1][2][1][RTW89_ETSI][10] = 34,
+ [0][1][2][1][RTW89_MKK][10] = 64,
+ [0][1][2][1][RTW89_IC][10] = 60,
+ [0][1][2][1][RTW89_KCC][10] = 70,
+ [0][1][2][1][RTW89_ACMA][10] = 34,
+ [0][1][2][1][RTW89_CHILE][10] = 36,
+ [0][1][2][1][RTW89_UKRAINE][10] = 34,
+ [0][1][2][1][RTW89_MEXICO][10] = 60,
+ [0][1][2][1][RTW89_CN][10] = 34,
+ [0][1][2][1][RTW89_QATAR][10] = 34,
+ [0][1][2][1][RTW89_FCC][11] = 42,
+ [0][1][2][1][RTW89_ETSI][11] = 34,
+ [0][1][2][1][RTW89_MKK][11] = 64,
+ [0][1][2][1][RTW89_IC][11] = 42,
+ [0][1][2][1][RTW89_KCC][11] = 70,
+ [0][1][2][1][RTW89_ACMA][11] = 34,
+ [0][1][2][1][RTW89_CHILE][11] = 36,
+ [0][1][2][1][RTW89_UKRAINE][11] = 34,
+ [0][1][2][1][RTW89_MEXICO][11] = 42,
+ [0][1][2][1][RTW89_CN][11] = 34,
+ [0][1][2][1][RTW89_QATAR][11] = 34,
+ [0][1][2][1][RTW89_FCC][12] = 40,
+ [0][1][2][1][RTW89_ETSI][12] = 34,
+ [0][1][2][1][RTW89_MKK][12] = 64,
+ [0][1][2][1][RTW89_IC][12] = 40,
+ [0][1][2][1][RTW89_KCC][12] = 68,
+ [0][1][2][1][RTW89_ACMA][12] = 34,
+ [0][1][2][1][RTW89_CHILE][12] = 36,
+ [0][1][2][1][RTW89_UKRAINE][12] = 34,
+ [0][1][2][1][RTW89_MEXICO][12] = 40,
+ [0][1][2][1][RTW89_CN][12] = 34,
+ [0][1][2][1][RTW89_QATAR][12] = 34,
+ [0][1][2][1][RTW89_FCC][13] = 127,
+ [0][1][2][1][RTW89_ETSI][13] = 127,
+ [0][1][2][1][RTW89_MKK][13] = 127,
+ [0][1][2][1][RTW89_IC][13] = 127,
+ [0][1][2][1][RTW89_KCC][13] = 127,
+ [0][1][2][1][RTW89_ACMA][13] = 127,
+ [0][1][2][1][RTW89_CHILE][13] = 127,
+ [0][1][2][1][RTW89_UKRAINE][13] = 127,
+ [0][1][2][1][RTW89_MEXICO][13] = 127,
+ [0][1][2][1][RTW89_CN][13] = 127,
+ [0][1][2][1][RTW89_QATAR][13] = 127,
+ [1][0][2][0][RTW89_FCC][0] = 127,
+ [1][0][2][0][RTW89_ETSI][0] = 127,
+ [1][0][2][0][RTW89_MKK][0] = 127,
+ [1][0][2][0][RTW89_IC][0] = 127,
+ [1][0][2][0][RTW89_KCC][0] = 127,
+ [1][0][2][0][RTW89_ACMA][0] = 127,
+ [1][0][2][0][RTW89_CHILE][0] = 127,
+ [1][0][2][0][RTW89_UKRAINE][0] = 127,
+ [1][0][2][0][RTW89_MEXICO][0] = 127,
+ [1][0][2][0][RTW89_CN][0] = 127,
+ [1][0][2][0][RTW89_QATAR][0] = 127,
+ [1][0][2][0][RTW89_FCC][1] = 127,
+ [1][0][2][0][RTW89_ETSI][1] = 127,
+ [1][0][2][0][RTW89_MKK][1] = 127,
+ [1][0][2][0][RTW89_IC][1] = 127,
+ [1][0][2][0][RTW89_KCC][1] = 127,
+ [1][0][2][0][RTW89_ACMA][1] = 127,
+ [1][0][2][0][RTW89_CHILE][1] = 127,
+ [1][0][2][0][RTW89_UKRAINE][1] = 127,
+ [1][0][2][0][RTW89_MEXICO][1] = 127,
+ [1][0][2][0][RTW89_CN][1] = 127,
+ [1][0][2][0][RTW89_QATAR][1] = 127,
+ [1][0][2][0][RTW89_FCC][2] = 56,
+ [1][0][2][0][RTW89_ETSI][2] = 58,
+ [1][0][2][0][RTW89_MKK][2] = 68,
+ [1][0][2][0][RTW89_IC][2] = 56,
+ [1][0][2][0][RTW89_KCC][2] = 68,
+ [1][0][2][0][RTW89_ACMA][2] = 58,
+ [1][0][2][0][RTW89_CHILE][2] = 56,
+ [1][0][2][0][RTW89_UKRAINE][2] = 58,
+ [1][0][2][0][RTW89_MEXICO][2] = 56,
+ [1][0][2][0][RTW89_CN][2] = 58,
+ [1][0][2][0][RTW89_QATAR][2] = 58,
+ [1][0][2][0][RTW89_FCC][3] = 56,
+ [1][0][2][0][RTW89_ETSI][3] = 58,
+ [1][0][2][0][RTW89_MKK][3] = 68,
+ [1][0][2][0][RTW89_IC][3] = 56,
+ [1][0][2][0][RTW89_KCC][3] = 68,
+ [1][0][2][0][RTW89_ACMA][3] = 58,
+ [1][0][2][0][RTW89_CHILE][3] = 56,
+ [1][0][2][0][RTW89_UKRAINE][3] = 58,
+ [1][0][2][0][RTW89_MEXICO][3] = 56,
+ [1][0][2][0][RTW89_CN][3] = 58,
+ [1][0][2][0][RTW89_QATAR][3] = 58,
+ [1][0][2][0][RTW89_FCC][4] = 60,
+ [1][0][2][0][RTW89_ETSI][4] = 58,
+ [1][0][2][0][RTW89_MKK][4] = 68,
+ [1][0][2][0][RTW89_IC][4] = 60,
+ [1][0][2][0][RTW89_KCC][4] = 68,
+ [1][0][2][0][RTW89_ACMA][4] = 58,
+ [1][0][2][0][RTW89_CHILE][4] = 60,
+ [1][0][2][0][RTW89_UKRAINE][4] = 58,
+ [1][0][2][0][RTW89_MEXICO][4] = 60,
+ [1][0][2][0][RTW89_CN][4] = 58,
+ [1][0][2][0][RTW89_QATAR][4] = 58,
+ [1][0][2][0][RTW89_FCC][5] = 64,
+ [1][0][2][0][RTW89_ETSI][5] = 58,
+ [1][0][2][0][RTW89_MKK][5] = 68,
+ [1][0][2][0][RTW89_IC][5] = 64,
+ [1][0][2][0][RTW89_KCC][5] = 68,
+ [1][0][2][0][RTW89_ACMA][5] = 58,
+ [1][0][2][0][RTW89_CHILE][5] = 60,
+ [1][0][2][0][RTW89_UKRAINE][5] = 58,
+ [1][0][2][0][RTW89_MEXICO][5] = 64,
+ [1][0][2][0][RTW89_CN][5] = 58,
+ [1][0][2][0][RTW89_QATAR][5] = 58,
+ [1][0][2][0][RTW89_FCC][6] = 54,
+ [1][0][2][0][RTW89_ETSI][6] = 58,
+ [1][0][2][0][RTW89_MKK][6] = 68,
+ [1][0][2][0][RTW89_IC][6] = 54,
+ [1][0][2][0][RTW89_KCC][6] = 68,
+ [1][0][2][0][RTW89_ACMA][6] = 58,
+ [1][0][2][0][RTW89_CHILE][6] = 54,
+ [1][0][2][0][RTW89_UKRAINE][6] = 58,
+ [1][0][2][0][RTW89_MEXICO][6] = 54,
+ [1][0][2][0][RTW89_CN][6] = 58,
+ [1][0][2][0][RTW89_QATAR][6] = 58,
+ [1][0][2][0][RTW89_FCC][7] = 50,
+ [1][0][2][0][RTW89_ETSI][7] = 58,
+ [1][0][2][0][RTW89_MKK][7] = 68,
+ [1][0][2][0][RTW89_IC][7] = 50,
+ [1][0][2][0][RTW89_KCC][7] = 68,
+ [1][0][2][0][RTW89_ACMA][7] = 58,
+ [1][0][2][0][RTW89_CHILE][7] = 50,
+ [1][0][2][0][RTW89_UKRAINE][7] = 58,
+ [1][0][2][0][RTW89_MEXICO][7] = 50,
+ [1][0][2][0][RTW89_CN][7] = 58,
+ [1][0][2][0][RTW89_QATAR][7] = 58,
+ [1][0][2][0][RTW89_FCC][8] = 50,
+ [1][0][2][0][RTW89_ETSI][8] = 58,
+ [1][0][2][0][RTW89_MKK][8] = 68,
+ [1][0][2][0][RTW89_IC][8] = 50,
+ [1][0][2][0][RTW89_KCC][8] = 68,
+ [1][0][2][0][RTW89_ACMA][8] = 58,
+ [1][0][2][0][RTW89_CHILE][8] = 50,
+ [1][0][2][0][RTW89_UKRAINE][8] = 58,
+ [1][0][2][0][RTW89_MEXICO][8] = 50,
+ [1][0][2][0][RTW89_CN][8] = 58,
+ [1][0][2][0][RTW89_QATAR][8] = 58,
+ [1][0][2][0][RTW89_FCC][9] = 42,
+ [1][0][2][0][RTW89_ETSI][9] = 58,
+ [1][0][2][0][RTW89_MKK][9] = 68,
+ [1][0][2][0][RTW89_IC][9] = 42,
+ [1][0][2][0][RTW89_KCC][9] = 68,
+ [1][0][2][0][RTW89_ACMA][9] = 58,
+ [1][0][2][0][RTW89_CHILE][9] = 42,
+ [1][0][2][0][RTW89_UKRAINE][9] = 58,
+ [1][0][2][0][RTW89_MEXICO][9] = 42,
+ [1][0][2][0][RTW89_CN][9] = 58,
+ [1][0][2][0][RTW89_QATAR][9] = 58,
+ [1][0][2][0][RTW89_FCC][10] = 40,
+ [1][0][2][0][RTW89_ETSI][10] = 58,
+ [1][0][2][0][RTW89_MKK][10] = 68,
+ [1][0][2][0][RTW89_IC][10] = 40,
+ [1][0][2][0][RTW89_KCC][10] = 68,
+ [1][0][2][0][RTW89_ACMA][10] = 58,
+ [1][0][2][0][RTW89_CHILE][10] = 40,
+ [1][0][2][0][RTW89_UKRAINE][10] = 58,
+ [1][0][2][0][RTW89_MEXICO][10] = 40,
+ [1][0][2][0][RTW89_CN][10] = 58,
+ [1][0][2][0][RTW89_QATAR][10] = 58,
+ [1][0][2][0][RTW89_FCC][11] = 127,
+ [1][0][2][0][RTW89_ETSI][11] = 127,
+ [1][0][2][0][RTW89_MKK][11] = 127,
+ [1][0][2][0][RTW89_IC][11] = 127,
+ [1][0][2][0][RTW89_KCC][11] = 127,
+ [1][0][2][0][RTW89_ACMA][11] = 127,
+ [1][0][2][0][RTW89_CHILE][11] = 127,
+ [1][0][2][0][RTW89_UKRAINE][11] = 127,
+ [1][0][2][0][RTW89_MEXICO][11] = 127,
+ [1][0][2][0][RTW89_CN][11] = 127,
+ [1][0][2][0][RTW89_QATAR][11] = 127,
+ [1][0][2][0][RTW89_FCC][12] = 127,
+ [1][0][2][0][RTW89_ETSI][12] = 127,
+ [1][0][2][0][RTW89_MKK][12] = 127,
+ [1][0][2][0][RTW89_IC][12] = 127,
+ [1][0][2][0][RTW89_KCC][12] = 127,
+ [1][0][2][0][RTW89_ACMA][12] = 127,
+ [1][0][2][0][RTW89_CHILE][12] = 127,
+ [1][0][2][0][RTW89_UKRAINE][12] = 127,
+ [1][0][2][0][RTW89_MEXICO][12] = 127,
+ [1][0][2][0][RTW89_CN][12] = 127,
+ [1][0][2][0][RTW89_QATAR][12] = 127,
+ [1][0][2][0][RTW89_FCC][13] = 127,
+ [1][0][2][0][RTW89_ETSI][13] = 127,
+ [1][0][2][0][RTW89_MKK][13] = 127,
+ [1][0][2][0][RTW89_IC][13] = 127,
+ [1][0][2][0][RTW89_KCC][13] = 127,
+ [1][0][2][0][RTW89_ACMA][13] = 127,
+ [1][0][2][0][RTW89_CHILE][13] = 127,
+ [1][0][2][0][RTW89_UKRAINE][13] = 127,
+ [1][0][2][0][RTW89_MEXICO][13] = 127,
+ [1][0][2][0][RTW89_CN][13] = 127,
+ [1][0][2][0][RTW89_QATAR][13] = 127,
+ [1][1][2][0][RTW89_FCC][0] = 127,
+ [1][1][2][0][RTW89_ETSI][0] = 127,
+ [1][1][2][0][RTW89_MKK][0] = 127,
+ [1][1][2][0][RTW89_IC][0] = 127,
+ [1][1][2][0][RTW89_KCC][0] = 127,
+ [1][1][2][0][RTW89_ACMA][0] = 127,
+ [1][1][2][0][RTW89_CHILE][0] = 127,
+ [1][1][2][0][RTW89_UKRAINE][0] = 127,
+ [1][1][2][0][RTW89_MEXICO][0] = 127,
+ [1][1][2][0][RTW89_CN][0] = 127,
+ [1][1][2][0][RTW89_QATAR][0] = 127,
+ [1][1][2][0][RTW89_FCC][1] = 127,
+ [1][1][2][0][RTW89_ETSI][1] = 127,
+ [1][1][2][0][RTW89_MKK][1] = 127,
+ [1][1][2][0][RTW89_IC][1] = 127,
+ [1][1][2][0][RTW89_KCC][1] = 127,
+ [1][1][2][0][RTW89_ACMA][1] = 127,
+ [1][1][2][0][RTW89_CHILE][1] = 127,
+ [1][1][2][0][RTW89_UKRAINE][1] = 127,
+ [1][1][2][0][RTW89_MEXICO][1] = 127,
+ [1][1][2][0][RTW89_CN][1] = 127,
+ [1][1][2][0][RTW89_QATAR][1] = 127,
+ [1][1][2][0][RTW89_FCC][2] = 52,
+ [1][1][2][0][RTW89_ETSI][2] = 46,
+ [1][1][2][0][RTW89_MKK][2] = 64,
+ [1][1][2][0][RTW89_IC][2] = 52,
+ [1][1][2][0][RTW89_KCC][2] = 66,
+ [1][1][2][0][RTW89_ACMA][2] = 46,
+ [1][1][2][0][RTW89_CHILE][2] = 48,
+ [1][1][2][0][RTW89_UKRAINE][2] = 46,
+ [1][1][2][0][RTW89_MEXICO][2] = 52,
+ [1][1][2][0][RTW89_CN][2] = 46,
+ [1][1][2][0][RTW89_QATAR][2] = 46,
+ [1][1][2][0][RTW89_FCC][3] = 52,
+ [1][1][2][0][RTW89_ETSI][3] = 46,
+ [1][1][2][0][RTW89_MKK][3] = 64,
+ [1][1][2][0][RTW89_IC][3] = 52,
+ [1][1][2][0][RTW89_KCC][3] = 68,
+ [1][1][2][0][RTW89_ACMA][3] = 46,
+ [1][1][2][0][RTW89_CHILE][3] = 48,
+ [1][1][2][0][RTW89_UKRAINE][3] = 46,
+ [1][1][2][0][RTW89_MEXICO][3] = 52,
+ [1][1][2][0][RTW89_CN][3] = 46,
+ [1][1][2][0][RTW89_QATAR][3] = 46,
+ [1][1][2][0][RTW89_FCC][4] = 56,
+ [1][1][2][0][RTW89_ETSI][4] = 46,
+ [1][1][2][0][RTW89_MKK][4] = 64,
+ [1][1][2][0][RTW89_IC][4] = 56,
+ [1][1][2][0][RTW89_KCC][4] = 68,
+ [1][1][2][0][RTW89_ACMA][4] = 46,
+ [1][1][2][0][RTW89_CHILE][4] = 48,
+ [1][1][2][0][RTW89_UKRAINE][4] = 46,
+ [1][1][2][0][RTW89_MEXICO][4] = 56,
+ [1][1][2][0][RTW89_CN][4] = 46,
+ [1][1][2][0][RTW89_QATAR][4] = 46,
+ [1][1][2][0][RTW89_FCC][5] = 60,
+ [1][1][2][0][RTW89_ETSI][5] = 46,
+ [1][1][2][0][RTW89_MKK][5] = 64,
+ [1][1][2][0][RTW89_IC][5] = 60,
+ [1][1][2][0][RTW89_KCC][5] = 68,
+ [1][1][2][0][RTW89_ACMA][5] = 46,
+ [1][1][2][0][RTW89_CHILE][5] = 48,
+ [1][1][2][0][RTW89_UKRAINE][5] = 46,
+ [1][1][2][0][RTW89_MEXICO][5] = 60,
+ [1][1][2][0][RTW89_CN][5] = 46,
+ [1][1][2][0][RTW89_QATAR][5] = 46,
+ [1][1][2][0][RTW89_FCC][6] = 54,
+ [1][1][2][0][RTW89_ETSI][6] = 46,
+ [1][1][2][0][RTW89_MKK][6] = 64,
+ [1][1][2][0][RTW89_IC][6] = 52,
+ [1][1][2][0][RTW89_KCC][6] = 68,
+ [1][1][2][0][RTW89_ACMA][6] = 46,
+ [1][1][2][0][RTW89_CHILE][6] = 48,
+ [1][1][2][0][RTW89_UKRAINE][6] = 46,
+ [1][1][2][0][RTW89_MEXICO][6] = 54,
+ [1][1][2][0][RTW89_CN][6] = 46,
+ [1][1][2][0][RTW89_QATAR][6] = 46,
+ [1][1][2][0][RTW89_FCC][7] = 50,
+ [1][1][2][0][RTW89_ETSI][7] = 46,
+ [1][1][2][0][RTW89_MKK][7] = 64,
+ [1][1][2][0][RTW89_IC][7] = 48,
+ [1][1][2][0][RTW89_KCC][7] = 68,
+ [1][1][2][0][RTW89_ACMA][7] = 46,
+ [1][1][2][0][RTW89_CHILE][7] = 48,
+ [1][1][2][0][RTW89_UKRAINE][7] = 46,
+ [1][1][2][0][RTW89_MEXICO][7] = 50,
+ [1][1][2][0][RTW89_CN][7] = 46,
+ [1][1][2][0][RTW89_QATAR][7] = 46,
+ [1][1][2][0][RTW89_FCC][8] = 50,
+ [1][1][2][0][RTW89_ETSI][8] = 46,
+ [1][1][2][0][RTW89_MKK][8] = 64,
+ [1][1][2][0][RTW89_IC][8] = 48,
+ [1][1][2][0][RTW89_KCC][8] = 68,
+ [1][1][2][0][RTW89_ACMA][8] = 46,
+ [1][1][2][0][RTW89_CHILE][8] = 48,
+ [1][1][2][0][RTW89_UKRAINE][8] = 46,
+ [1][1][2][0][RTW89_MEXICO][8] = 50,
+ [1][1][2][0][RTW89_CN][8] = 46,
+ [1][1][2][0][RTW89_QATAR][8] = 46,
+ [1][1][2][0][RTW89_FCC][9] = 38,
+ [1][1][2][0][RTW89_ETSI][9] = 46,
+ [1][1][2][0][RTW89_MKK][9] = 64,
+ [1][1][2][0][RTW89_IC][9] = 38,
+ [1][1][2][0][RTW89_KCC][9] = 68,
+ [1][1][2][0][RTW89_ACMA][9] = 46,
+ [1][1][2][0][RTW89_CHILE][9] = 38,
+ [1][1][2][0][RTW89_UKRAINE][9] = 46,
+ [1][1][2][0][RTW89_MEXICO][9] = 38,
+ [1][1][2][0][RTW89_CN][9] = 46,
+ [1][1][2][0][RTW89_QATAR][9] = 46,
+ [1][1][2][0][RTW89_FCC][10] = 36,
+ [1][1][2][0][RTW89_ETSI][10] = 46,
+ [1][1][2][0][RTW89_MKK][10] = 64,
+ [1][1][2][0][RTW89_IC][10] = 36,
+ [1][1][2][0][RTW89_KCC][10] = 66,
+ [1][1][2][0][RTW89_ACMA][10] = 46,
+ [1][1][2][0][RTW89_CHILE][10] = 36,
+ [1][1][2][0][RTW89_UKRAINE][10] = 46,
+ [1][1][2][0][RTW89_MEXICO][10] = 36,
+ [1][1][2][0][RTW89_CN][10] = 46,
+ [1][1][2][0][RTW89_QATAR][10] = 46,
+ [1][1][2][0][RTW89_FCC][11] = 127,
+ [1][1][2][0][RTW89_ETSI][11] = 127,
+ [1][1][2][0][RTW89_MKK][11] = 127,
+ [1][1][2][0][RTW89_IC][11] = 127,
+ [1][1][2][0][RTW89_KCC][11] = 127,
+ [1][1][2][0][RTW89_ACMA][11] = 127,
+ [1][1][2][0][RTW89_CHILE][11] = 127,
+ [1][1][2][0][RTW89_UKRAINE][11] = 127,
+ [1][1][2][0][RTW89_MEXICO][11] = 127,
+ [1][1][2][0][RTW89_CN][11] = 127,
+ [1][1][2][0][RTW89_QATAR][11] = 127,
+ [1][1][2][0][RTW89_FCC][12] = 127,
+ [1][1][2][0][RTW89_ETSI][12] = 127,
+ [1][1][2][0][RTW89_MKK][12] = 127,
+ [1][1][2][0][RTW89_IC][12] = 127,
+ [1][1][2][0][RTW89_KCC][12] = 127,
+ [1][1][2][0][RTW89_ACMA][12] = 127,
+ [1][1][2][0][RTW89_CHILE][12] = 127,
+ [1][1][2][0][RTW89_UKRAINE][12] = 127,
+ [1][1][2][0][RTW89_MEXICO][12] = 127,
+ [1][1][2][0][RTW89_CN][12] = 127,
+ [1][1][2][0][RTW89_QATAR][12] = 127,
+ [1][1][2][0][RTW89_FCC][13] = 127,
+ [1][1][2][0][RTW89_ETSI][13] = 127,
+ [1][1][2][0][RTW89_MKK][13] = 127,
+ [1][1][2][0][RTW89_IC][13] = 127,
+ [1][1][2][0][RTW89_KCC][13] = 127,
+ [1][1][2][0][RTW89_ACMA][13] = 127,
+ [1][1][2][0][RTW89_CHILE][13] = 127,
+ [1][1][2][0][RTW89_UKRAINE][13] = 127,
+ [1][1][2][0][RTW89_MEXICO][13] = 127,
+ [1][1][2][0][RTW89_CN][13] = 127,
+ [1][1][2][0][RTW89_QATAR][13] = 127,
+ [1][1][2][1][RTW89_FCC][0] = 127,
+ [1][1][2][1][RTW89_ETSI][0] = 127,
+ [1][1][2][1][RTW89_MKK][0] = 127,
+ [1][1][2][1][RTW89_IC][0] = 127,
+ [1][1][2][1][RTW89_KCC][0] = 127,
+ [1][1][2][1][RTW89_ACMA][0] = 127,
+ [1][1][2][1][RTW89_CHILE][0] = 127,
+ [1][1][2][1][RTW89_UKRAINE][0] = 127,
+ [1][1][2][1][RTW89_MEXICO][0] = 127,
+ [1][1][2][1][RTW89_CN][0] = 127,
+ [1][1][2][1][RTW89_QATAR][0] = 127,
+ [1][1][2][1][RTW89_FCC][1] = 127,
+ [1][1][2][1][RTW89_ETSI][1] = 127,
+ [1][1][2][1][RTW89_MKK][1] = 127,
+ [1][1][2][1][RTW89_IC][1] = 127,
+ [1][1][2][1][RTW89_KCC][1] = 127,
+ [1][1][2][1][RTW89_ACMA][1] = 127,
+ [1][1][2][1][RTW89_CHILE][1] = 127,
+ [1][1][2][1][RTW89_UKRAINE][1] = 127,
+ [1][1][2][1][RTW89_MEXICO][1] = 127,
+ [1][1][2][1][RTW89_CN][1] = 127,
+ [1][1][2][1][RTW89_QATAR][1] = 127,
+ [1][1][2][1][RTW89_FCC][2] = 52,
+ [1][1][2][1][RTW89_ETSI][2] = 34,
+ [1][1][2][1][RTW89_MKK][2] = 64,
+ [1][1][2][1][RTW89_IC][2] = 52,
+ [1][1][2][1][RTW89_KCC][2] = 66,
+ [1][1][2][1][RTW89_ACMA][2] = 34,
+ [1][1][2][1][RTW89_CHILE][2] = 36,
+ [1][1][2][1][RTW89_UKRAINE][2] = 34,
+ [1][1][2][1][RTW89_MEXICO][2] = 52,
+ [1][1][2][1][RTW89_CN][2] = 34,
+ [1][1][2][1][RTW89_QATAR][2] = 34,
+ [1][1][2][1][RTW89_FCC][3] = 52,
+ [1][1][2][1][RTW89_ETSI][3] = 34,
+ [1][1][2][1][RTW89_MKK][3] = 64,
+ [1][1][2][1][RTW89_IC][3] = 52,
+ [1][1][2][1][RTW89_KCC][3] = 68,
+ [1][1][2][1][RTW89_ACMA][3] = 34,
+ [1][1][2][1][RTW89_CHILE][3] = 36,
+ [1][1][2][1][RTW89_UKRAINE][3] = 34,
+ [1][1][2][1][RTW89_MEXICO][3] = 52,
+ [1][1][2][1][RTW89_CN][3] = 34,
+ [1][1][2][1][RTW89_QATAR][3] = 34,
+ [1][1][2][1][RTW89_FCC][4] = 56,
+ [1][1][2][1][RTW89_ETSI][4] = 34,
+ [1][1][2][1][RTW89_MKK][4] = 64,
+ [1][1][2][1][RTW89_IC][4] = 56,
+ [1][1][2][1][RTW89_KCC][4] = 68,
+ [1][1][2][1][RTW89_ACMA][4] = 34,
+ [1][1][2][1][RTW89_CHILE][4] = 36,
+ [1][1][2][1][RTW89_UKRAINE][4] = 34,
+ [1][1][2][1][RTW89_MEXICO][4] = 56,
+ [1][1][2][1][RTW89_CN][4] = 34,
+ [1][1][2][1][RTW89_QATAR][4] = 34,
+ [1][1][2][1][RTW89_FCC][5] = 60,
+ [1][1][2][1][RTW89_ETSI][5] = 34,
+ [1][1][2][1][RTW89_MKK][5] = 64,
+ [1][1][2][1][RTW89_IC][5] = 60,
+ [1][1][2][1][RTW89_KCC][5] = 68,
+ [1][1][2][1][RTW89_ACMA][5] = 34,
+ [1][1][2][1][RTW89_CHILE][5] = 36,
+ [1][1][2][1][RTW89_UKRAINE][5] = 34,
+ [1][1][2][1][RTW89_MEXICO][5] = 60,
+ [1][1][2][1][RTW89_CN][5] = 34,
+ [1][1][2][1][RTW89_QATAR][5] = 34,
+ [1][1][2][1][RTW89_FCC][6] = 54,
+ [1][1][2][1][RTW89_ETSI][6] = 34,
+ [1][1][2][1][RTW89_MKK][6] = 64,
+ [1][1][2][1][RTW89_IC][6] = 52,
+ [1][1][2][1][RTW89_KCC][6] = 68,
+ [1][1][2][1][RTW89_ACMA][6] = 34,
+ [1][1][2][1][RTW89_CHILE][6] = 36,
+ [1][1][2][1][RTW89_UKRAINE][6] = 34,
+ [1][1][2][1][RTW89_MEXICO][6] = 54,
+ [1][1][2][1][RTW89_CN][6] = 34,
+ [1][1][2][1][RTW89_QATAR][6] = 34,
+ [1][1][2][1][RTW89_FCC][7] = 50,
+ [1][1][2][1][RTW89_ETSI][7] = 34,
+ [1][1][2][1][RTW89_MKK][7] = 64,
+ [1][1][2][1][RTW89_IC][7] = 48,
+ [1][1][2][1][RTW89_KCC][7] = 68,
+ [1][1][2][1][RTW89_ACMA][7] = 34,
+ [1][1][2][1][RTW89_CHILE][7] = 36,
+ [1][1][2][1][RTW89_UKRAINE][7] = 34,
+ [1][1][2][1][RTW89_MEXICO][7] = 50,
+ [1][1][2][1][RTW89_CN][7] = 34,
+ [1][1][2][1][RTW89_QATAR][7] = 34,
+ [1][1][2][1][RTW89_FCC][8] = 50,
+ [1][1][2][1][RTW89_ETSI][8] = 34,
+ [1][1][2][1][RTW89_MKK][8] = 64,
+ [1][1][2][1][RTW89_IC][8] = 48,
+ [1][1][2][1][RTW89_KCC][8] = 68,
+ [1][1][2][1][RTW89_ACMA][8] = 34,
+ [1][1][2][1][RTW89_CHILE][8] = 36,
+ [1][1][2][1][RTW89_UKRAINE][8] = 34,
+ [1][1][2][1][RTW89_MEXICO][8] = 50,
+ [1][1][2][1][RTW89_CN][8] = 34,
+ [1][1][2][1][RTW89_QATAR][8] = 34,
+ [1][1][2][1][RTW89_FCC][9] = 38,
+ [1][1][2][1][RTW89_ETSI][9] = 34,
+ [1][1][2][1][RTW89_MKK][9] = 64,
+ [1][1][2][1][RTW89_IC][9] = 38,
+ [1][1][2][1][RTW89_KCC][9] = 68,
+ [1][1][2][1][RTW89_ACMA][9] = 34,
+ [1][1][2][1][RTW89_CHILE][9] = 36,
+ [1][1][2][1][RTW89_UKRAINE][9] = 34,
+ [1][1][2][1][RTW89_MEXICO][9] = 38,
+ [1][1][2][1][RTW89_CN][9] = 34,
+ [1][1][2][1][RTW89_QATAR][9] = 34,
+ [1][1][2][1][RTW89_FCC][10] = 36,
+ [1][1][2][1][RTW89_ETSI][10] = 34,
+ [1][1][2][1][RTW89_MKK][10] = 64,
+ [1][1][2][1][RTW89_IC][10] = 36,
+ [1][1][2][1][RTW89_KCC][10] = 66,
+ [1][1][2][1][RTW89_ACMA][10] = 34,
+ [1][1][2][1][RTW89_CHILE][10] = 36,
+ [1][1][2][1][RTW89_UKRAINE][10] = 34,
+ [1][1][2][1][RTW89_MEXICO][10] = 36,
+ [1][1][2][1][RTW89_CN][10] = 34,
+ [1][1][2][1][RTW89_QATAR][10] = 34,
+ [1][1][2][1][RTW89_FCC][11] = 127,
+ [1][1][2][1][RTW89_ETSI][11] = 127,
+ [1][1][2][1][RTW89_MKK][11] = 127,
+ [1][1][2][1][RTW89_IC][11] = 127,
+ [1][1][2][1][RTW89_KCC][11] = 127,
+ [1][1][2][1][RTW89_ACMA][11] = 127,
+ [1][1][2][1][RTW89_CHILE][11] = 127,
+ [1][1][2][1][RTW89_UKRAINE][11] = 127,
+ [1][1][2][1][RTW89_MEXICO][11] = 127,
+ [1][1][2][1][RTW89_CN][11] = 127,
+ [1][1][2][1][RTW89_QATAR][11] = 127,
+ [1][1][2][1][RTW89_FCC][12] = 127,
+ [1][1][2][1][RTW89_ETSI][12] = 127,
+ [1][1][2][1][RTW89_MKK][12] = 127,
+ [1][1][2][1][RTW89_IC][12] = 127,
+ [1][1][2][1][RTW89_KCC][12] = 127,
+ [1][1][2][1][RTW89_ACMA][12] = 127,
+ [1][1][2][1][RTW89_CHILE][12] = 127,
+ [1][1][2][1][RTW89_UKRAINE][12] = 127,
+ [1][1][2][1][RTW89_MEXICO][12] = 127,
+ [1][1][2][1][RTW89_CN][12] = 127,
+ [1][1][2][1][RTW89_QATAR][12] = 127,
+ [1][1][2][1][RTW89_FCC][13] = 127,
+ [1][1][2][1][RTW89_ETSI][13] = 127,
+ [1][1][2][1][RTW89_MKK][13] = 127,
+ [1][1][2][1][RTW89_IC][13] = 127,
+ [1][1][2][1][RTW89_KCC][13] = 127,
+ [1][1][2][1][RTW89_ACMA][13] = 127,
+ [1][1][2][1][RTW89_CHILE][13] = 127,
+ [1][1][2][1][RTW89_UKRAINE][13] = 127,
+ [1][1][2][1][RTW89_MEXICO][13] = 127,
+ [1][1][2][1][RTW89_CN][13] = 127,
+ [1][1][2][1][RTW89_QATAR][13] = 127,
};
const s8 rtw89_8852a_txpwr_lmt_5g[RTW89_5G_BW_NUM][RTW89_NTX_NUM]
[RTW89_RS_LMT_NUM][RTW89_BF_NUM]
[RTW89_REGD_NUM][RTW89_5G_CH_NUM] = {
- [0][0][1][0][0][0] = 30,
- [0][0][1][0][0][2] = 30,
- [0][0][1][0][0][4] = 30,
- [0][0][1][0][0][6] = 30,
- [0][0][1][0][0][8] = 52,
- [0][0][1][0][0][10] = 52,
- [0][0][1][0][0][12] = 52,
- [0][0][1][0][0][14] = 52,
- [0][0][1][0][0][15] = 52,
- [0][0][1][0][0][17] = 52,
- [0][0][1][0][0][19] = 52,
- [0][0][1][0][0][21] = 52,
- [0][0][1][0][0][23] = 52,
- [0][0][1][0][0][25] = 52,
- [0][0][1][0][0][27] = 52,
- [0][0][1][0][0][29] = 52,
- [0][0][1][0][0][31] = 52,
- [0][0][1][0][0][33] = 52,
- [0][0][1][0][0][35] = 52,
- [0][0][1][0][0][37] = 54,
- [0][0][1][0][0][38] = 28,
- [0][0][1][0][0][40] = 28,
- [0][0][1][0][0][42] = 28,
- [0][0][1][0][0][44] = 28,
- [0][0][1][0][0][46] = 28,
- [0][1][1][0][0][0] = 18,
- [0][1][1][0][0][2] = 18,
- [0][1][1][0][0][4] = 18,
- [0][1][1][0][0][6] = 18,
- [0][1][1][0][0][8] = 40,
- [0][1][1][0][0][10] = 40,
- [0][1][1][0][0][12] = 40,
- [0][1][1][0][0][14] = 40,
- [0][1][1][0][0][15] = 40,
- [0][1][1][0][0][17] = 40,
- [0][1][1][0][0][19] = 40,
- [0][1][1][0][0][21] = 40,
- [0][1][1][0][0][23] = 40,
- [0][1][1][0][0][25] = 40,
- [0][1][1][0][0][27] = 40,
- [0][1][1][0][0][29] = 40,
- [0][1][1][0][0][31] = 40,
- [0][1][1][0][0][33] = 40,
- [0][1][1][0][0][35] = 40,
- [0][1][1][0][0][37] = 42,
- [0][1][1][0][0][38] = 16,
- [0][1][1][0][0][40] = 16,
- [0][1][1][0][0][42] = 16,
- [0][1][1][0][0][44] = 16,
- [0][1][1][0][0][46] = 16,
- [0][0][2][0][0][0] = 30,
- [0][0][2][0][0][2] = 30,
- [0][0][2][0][0][4] = 30,
- [0][0][2][0][0][6] = 30,
- [0][0][2][0][0][8] = 52,
- [0][0][2][0][0][10] = 52,
- [0][0][2][0][0][12] = 52,
- [0][0][2][0][0][14] = 52,
- [0][0][2][0][0][15] = 52,
- [0][0][2][0][0][17] = 52,
- [0][0][2][0][0][19] = 52,
- [0][0][2][0][0][21] = 52,
- [0][0][2][0][0][23] = 52,
- [0][0][2][0][0][25] = 52,
- [0][0][2][0][0][27] = 52,
- [0][0][2][0][0][29] = 52,
- [0][0][2][0][0][31] = 52,
- [0][0][2][0][0][33] = 52,
- [0][0][2][0][0][35] = 52,
- [0][0][2][0][0][37] = 54,
- [0][0][2][0][0][38] = 28,
- [0][0][2][0][0][40] = 28,
- [0][0][2][0][0][42] = 28,
- [0][0][2][0][0][44] = 28,
- [0][0][2][0][0][46] = 28,
- [0][1][2][0][0][0] = 18,
- [0][1][2][0][0][2] = 18,
- [0][1][2][0][0][4] = 18,
- [0][1][2][0][0][6] = 18,
- [0][1][2][0][0][8] = 40,
- [0][1][2][0][0][10] = 40,
- [0][1][2][0][0][12] = 40,
- [0][1][2][0][0][14] = 40,
- [0][1][2][0][0][15] = 40,
- [0][1][2][0][0][17] = 40,
- [0][1][2][0][0][19] = 40,
- [0][1][2][0][0][21] = 40,
- [0][1][2][0][0][23] = 40,
- [0][1][2][0][0][25] = 40,
- [0][1][2][0][0][27] = 40,
- [0][1][2][0][0][29] = 40,
- [0][1][2][0][0][31] = 40,
- [0][1][2][0][0][33] = 40,
- [0][1][2][0][0][35] = 40,
- [0][1][2][0][0][37] = 42,
- [0][1][2][0][0][38] = 16,
- [0][1][2][0][0][40] = 16,
- [0][1][2][0][0][42] = 16,
- [0][1][2][0][0][44] = 16,
- [0][1][2][0][0][46] = 16,
- [0][1][2][1][0][0] = 6,
- [0][1][2][1][0][2] = 6,
- [0][1][2][1][0][4] = 6,
- [0][1][2][1][0][6] = 6,
- [0][1][2][1][0][8] = 28,
- [0][1][2][1][0][10] = 28,
- [0][1][2][1][0][12] = 28,
- [0][1][2][1][0][14] = 28,
- [0][1][2][1][0][15] = 28,
- [0][1][2][1][0][17] = 28,
- [0][1][2][1][0][19] = 28,
- [0][1][2][1][0][21] = 28,
- [0][1][2][1][0][23] = 28,
- [0][1][2][1][0][25] = 28,
- [0][1][2][1][0][27] = 28,
- [0][1][2][1][0][29] = 28,
- [0][1][2][1][0][31] = 28,
- [0][1][2][1][0][33] = 28,
- [0][1][2][1][0][35] = 28,
- [0][1][2][1][0][37] = 30,
- [0][1][2][1][0][38] = 4,
- [0][1][2][1][0][40] = 4,
- [0][1][2][1][0][42] = 4,
- [0][1][2][1][0][44] = 4,
- [0][1][2][1][0][46] = 4,
- [1][0][2][0][0][1] = 30,
- [1][0][2][0][0][5] = 30,
- [1][0][2][0][0][9] = 52,
- [1][0][2][0][0][13] = 52,
- [1][0][2][0][0][16] = 52,
- [1][0][2][0][0][20] = 52,
- [1][0][2][0][0][24] = 52,
- [1][0][2][0][0][28] = 52,
- [1][0][2][0][0][32] = 52,
- [1][0][2][0][0][36] = 54,
- [1][0][2][0][0][39] = 28,
- [1][0][2][0][0][43] = 28,
- [1][1][2][0][0][1] = 18,
- [1][1][2][0][0][5] = 18,
- [1][1][2][0][0][9] = 40,
- [1][1][2][0][0][13] = 40,
- [1][1][2][0][0][16] = 40,
- [1][1][2][0][0][20] = 40,
- [1][1][2][0][0][24] = 40,
- [1][1][2][0][0][28] = 40,
- [1][1][2][0][0][32] = 40,
- [1][1][2][0][0][36] = 42,
- [1][1][2][0][0][39] = 16,
- [1][1][2][0][0][43] = 16,
- [1][1][2][1][0][1] = 6,
- [1][1][2][1][0][5] = 6,
- [1][1][2][1][0][9] = 28,
- [1][1][2][1][0][13] = 28,
- [1][1][2][1][0][16] = 28,
- [1][1][2][1][0][20] = 28,
- [1][1][2][1][0][24] = 28,
- [1][1][2][1][0][28] = 28,
- [1][1][2][1][0][32] = 28,
- [1][1][2][1][0][36] = 30,
- [1][1][2][1][0][39] = 4,
- [1][1][2][1][0][43] = 4,
- [2][0][2][0][0][3] = 30,
- [2][0][2][0][0][11] = 52,
- [2][0][2][0][0][18] = 52,
- [2][0][2][0][0][26] = 52,
- [2][0][2][0][0][34] = 54,
- [2][0][2][0][0][41] = 28,
- [2][1][2][0][0][3] = 18,
- [2][1][2][0][0][11] = 40,
- [2][1][2][0][0][18] = 40,
- [2][1][2][0][0][26] = 40,
- [2][1][2][0][0][34] = 42,
- [2][1][2][0][0][41] = 16,
- [2][1][2][1][0][3] = 6,
- [2][1][2][1][0][11] = 28,
- [2][1][2][1][0][18] = 28,
- [2][1][2][1][0][26] = 28,
- [2][1][2][1][0][34] = 30,
- [2][1][2][1][0][41] = 4,
- [0][0][1][0][2][0] = 76,
- [0][0][1][0][1][0] = 58,
- [0][0][1][0][3][0] = 62,
- [0][0][1][0][5][0] = 62,
- [0][0][1][0][6][0] = 58,
- [0][0][1][0][9][0] = 58,
- [0][0][1][0][8][0] = 30,
- [0][0][1][0][11][0] = 52,
- [0][0][1][0][2][2] = 76,
- [0][0][1][0][1][2] = 58,
- [0][0][1][0][3][2] = 62,
- [0][0][1][0][5][2] = 62,
- [0][0][1][0][6][2] = 58,
- [0][0][1][0][9][2] = 58,
- [0][0][1][0][8][2] = 30,
- [0][0][1][0][11][2] = 52,
- [0][0][1][0][2][4] = 76,
- [0][0][1][0][1][4] = 58,
- [0][0][1][0][3][4] = 62,
- [0][0][1][0][5][4] = 62,
- [0][0][1][0][6][4] = 58,
- [0][0][1][0][9][4] = 58,
- [0][0][1][0][8][4] = 30,
- [0][0][1][0][11][4] = 52,
- [0][0][1][0][2][6] = 76,
- [0][0][1][0][1][6] = 58,
- [0][0][1][0][3][6] = 62,
- [0][0][1][0][5][6] = 62,
- [0][0][1][0][6][6] = 54,
- [0][0][1][0][9][6] = 58,
- [0][0][1][0][8][6] = 30,
- [0][0][1][0][11][6] = 52,
- [0][0][1][0][2][8] = 76,
- [0][0][1][0][1][8] = 58,
- [0][0][1][0][3][8] = 62,
- [0][0][1][0][5][8] = 64,
- [0][0][1][0][6][8] = 58,
- [0][0][1][0][9][8] = 58,
- [0][0][1][0][8][8] = 54,
- [0][0][1][0][11][8] = 52,
- [0][0][1][0][2][10] = 76,
- [0][0][1][0][1][10] = 58,
- [0][0][1][0][3][10] = 62,
- [0][0][1][0][5][10] = 64,
- [0][0][1][0][6][10] = 58,
- [0][0][1][0][9][10] = 58,
- [0][0][1][0][8][10] = 54,
- [0][0][1][0][11][10] = 52,
- [0][0][1][0][2][12] = 76,
- [0][0][1][0][1][12] = 58,
- [0][0][1][0][3][12] = 62,
- [0][0][1][0][5][12] = 64,
- [0][0][1][0][6][12] = 58,
- [0][0][1][0][9][12] = 58,
- [0][0][1][0][8][12] = 54,
- [0][0][1][0][11][12] = 52,
- [0][0][1][0][2][14] = 76,
- [0][0][1][0][1][14] = 58,
- [0][0][1][0][3][14] = 62,
- [0][0][1][0][5][14] = 64,
- [0][0][1][0][6][14] = 58,
- [0][0][1][0][9][14] = 58,
- [0][0][1][0][8][14] = 54,
- [0][0][1][0][11][14] = 52,
- [0][0][1][0][2][15] = 76,
- [0][0][1][0][1][15] = 58,
- [0][0][1][0][3][15] = 76,
- [0][0][1][0][5][15] = 76,
- [0][0][1][0][6][15] = 58,
- [0][0][1][0][9][15] = 58,
- [0][0][1][0][8][15] = 54,
- [0][0][1][0][11][15] = 52,
- [0][0][1][0][2][17] = 76,
- [0][0][1][0][1][17] = 58,
- [0][0][1][0][3][17] = 76,
- [0][0][1][0][5][17] = 76,
- [0][0][1][0][6][17] = 58,
- [0][0][1][0][9][17] = 58,
- [0][0][1][0][8][17] = 54,
- [0][0][1][0][11][17] = 52,
- [0][0][1][0][2][19] = 76,
- [0][0][1][0][1][19] = 58,
- [0][0][1][0][3][19] = 76,
- [0][0][1][0][5][19] = 76,
- [0][0][1][0][6][19] = 58,
- [0][0][1][0][9][19] = 58,
- [0][0][1][0][8][19] = 54,
- [0][0][1][0][11][19] = 52,
- [0][0][1][0][2][21] = 76,
- [0][0][1][0][1][21] = 58,
- [0][0][1][0][3][21] = 76,
- [0][0][1][0][5][21] = 76,
- [0][0][1][0][6][21] = 58,
- [0][0][1][0][9][21] = 58,
- [0][0][1][0][8][21] = 54,
- [0][0][1][0][11][21] = 52,
- [0][0][1][0][2][23] = 76,
- [0][0][1][0][1][23] = 58,
- [0][0][1][0][3][23] = 76,
- [0][0][1][0][5][23] = 76,
- [0][0][1][0][6][23] = 58,
- [0][0][1][0][9][23] = 58,
- [0][0][1][0][8][23] = 54,
- [0][0][1][0][11][23] = 52,
- [0][0][1][0][2][25] = 76,
- [0][0][1][0][1][25] = 58,
- [0][0][1][0][3][25] = 76,
- [0][0][1][0][5][25] = 127,
- [0][0][1][0][6][25] = 58,
- [0][0][1][0][9][25] = 127,
- [0][0][1][0][8][25] = 54,
- [0][0][1][0][11][25] = 52,
- [0][0][1][0][2][27] = 76,
- [0][0][1][0][1][27] = 58,
- [0][0][1][0][3][27] = 76,
- [0][0][1][0][5][27] = 127,
- [0][0][1][0][6][27] = 58,
- [0][0][1][0][9][27] = 127,
- [0][0][1][0][8][27] = 54,
- [0][0][1][0][11][27] = 52,
- [0][0][1][0][2][29] = 76,
- [0][0][1][0][1][29] = 58,
- [0][0][1][0][3][29] = 76,
- [0][0][1][0][5][29] = 127,
- [0][0][1][0][6][29] = 58,
- [0][0][1][0][9][29] = 127,
- [0][0][1][0][8][29] = 54,
- [0][0][1][0][11][29] = 52,
- [0][0][1][0][2][31] = 76,
- [0][0][1][0][1][31] = 58,
- [0][0][1][0][3][31] = 76,
- [0][0][1][0][5][31] = 76,
- [0][0][1][0][6][31] = 58,
- [0][0][1][0][9][31] = 58,
- [0][0][1][0][8][31] = 54,
- [0][0][1][0][11][31] = 52,
- [0][0][1][0][2][33] = 76,
- [0][0][1][0][1][33] = 58,
- [0][0][1][0][3][33] = 76,
- [0][0][1][0][5][33] = 76,
- [0][0][1][0][6][33] = 58,
- [0][0][1][0][9][33] = 58,
- [0][0][1][0][8][33] = 54,
- [0][0][1][0][11][33] = 52,
- [0][0][1][0][2][35] = 74,
- [0][0][1][0][1][35] = 58,
- [0][0][1][0][3][35] = 76,
- [0][0][1][0][5][35] = 74,
- [0][0][1][0][6][35] = 58,
- [0][0][1][0][9][35] = 58,
- [0][0][1][0][8][35] = 54,
- [0][0][1][0][11][35] = 52,
- [0][0][1][0][2][37] = 76,
- [0][0][1][0][1][37] = 127,
- [0][0][1][0][3][37] = 76,
- [0][0][1][0][5][37] = 76,
- [0][0][1][0][6][37] = 58,
- [0][0][1][0][9][37] = 76,
- [0][0][1][0][8][37] = 54,
- [0][0][1][0][11][37] = 127,
- [0][0][1][0][2][38] = 76,
- [0][0][1][0][1][38] = 28,
- [0][0][1][0][3][38] = 127,
- [0][0][1][0][5][38] = 76,
- [0][0][1][0][6][38] = 28,
- [0][0][1][0][9][38] = 76,
- [0][0][1][0][8][38] = 54,
- [0][0][1][0][11][38] = 52,
- [0][0][1][0][2][40] = 76,
- [0][0][1][0][1][40] = 28,
- [0][0][1][0][3][40] = 127,
- [0][0][1][0][5][40] = 76,
- [0][0][1][0][6][40] = 28,
- [0][0][1][0][9][40] = 76,
- [0][0][1][0][8][40] = 54,
- [0][0][1][0][11][40] = 52,
- [0][0][1][0][2][42] = 76,
- [0][0][1][0][1][42] = 28,
- [0][0][1][0][3][42] = 127,
- [0][0][1][0][5][42] = 76,
- [0][0][1][0][6][42] = 28,
- [0][0][1][0][9][42] = 76,
- [0][0][1][0][8][42] = 54,
- [0][0][1][0][11][42] = 52,
- [0][0][1][0][2][44] = 76,
- [0][0][1][0][1][44] = 28,
- [0][0][1][0][3][44] = 127,
- [0][0][1][0][5][44] = 76,
- [0][0][1][0][6][44] = 28,
- [0][0][1][0][9][44] = 76,
- [0][0][1][0][8][44] = 54,
- [0][0][1][0][11][44] = 52,
- [0][0][1][0][2][46] = 76,
- [0][0][1][0][1][46] = 28,
- [0][0][1][0][3][46] = 127,
- [0][0][1][0][5][46] = 76,
- [0][0][1][0][6][46] = 28,
- [0][0][1][0][9][46] = 76,
- [0][0][1][0][8][46] = 54,
- [0][0][1][0][11][46] = 52,
- [0][1][1][0][2][0] = 68,
- [0][1][1][0][1][0] = 46,
- [0][1][1][0][3][0] = 50,
- [0][1][1][0][5][0] = 40,
- [0][1][1][0][6][0] = 46,
- [0][1][1][0][9][0] = 46,
- [0][1][1][0][8][0] = 18,
- [0][1][1][0][11][0] = 40,
- [0][1][1][0][2][2] = 68,
- [0][1][1][0][1][2] = 46,
- [0][1][1][0][3][2] = 50,
- [0][1][1][0][5][2] = 40,
- [0][1][1][0][6][2] = 46,
- [0][1][1][0][9][2] = 46,
- [0][1][1][0][8][2] = 18,
- [0][1][1][0][11][2] = 40,
- [0][1][1][0][2][4] = 68,
- [0][1][1][0][1][4] = 46,
- [0][1][1][0][3][4] = 50,
- [0][1][1][0][5][4] = 40,
- [0][1][1][0][6][4] = 46,
- [0][1][1][0][9][4] = 46,
- [0][1][1][0][8][4] = 18,
- [0][1][1][0][11][4] = 40,
- [0][1][1][0][2][6] = 68,
- [0][1][1][0][1][6] = 46,
- [0][1][1][0][3][6] = 50,
- [0][1][1][0][5][6] = 40,
- [0][1][1][0][6][6] = 36,
- [0][1][1][0][9][6] = 46,
- [0][1][1][0][8][6] = 18,
- [0][1][1][0][11][6] = 40,
- [0][1][1][0][2][8] = 68,
- [0][1][1][0][1][8] = 46,
- [0][1][1][0][3][8] = 50,
- [0][1][1][0][5][8] = 52,
- [0][1][1][0][6][8] = 46,
- [0][1][1][0][9][8] = 46,
- [0][1][1][0][8][8] = 42,
- [0][1][1][0][11][8] = 40,
- [0][1][1][0][2][10] = 68,
- [0][1][1][0][1][10] = 46,
- [0][1][1][0][3][10] = 50,
- [0][1][1][0][5][10] = 52,
- [0][1][1][0][6][10] = 46,
- [0][1][1][0][9][10] = 46,
- [0][1][1][0][8][10] = 42,
- [0][1][1][0][11][10] = 40,
- [0][1][1][0][2][12] = 68,
- [0][1][1][0][1][12] = 46,
- [0][1][1][0][3][12] = 50,
- [0][1][1][0][5][12] = 52,
- [0][1][1][0][6][12] = 46,
- [0][1][1][0][9][12] = 46,
- [0][1][1][0][8][12] = 42,
- [0][1][1][0][11][12] = 40,
- [0][1][1][0][2][14] = 68,
- [0][1][1][0][1][14] = 46,
- [0][1][1][0][3][14] = 50,
- [0][1][1][0][5][14] = 52,
- [0][1][1][0][6][14] = 46,
- [0][1][1][0][9][14] = 46,
- [0][1][1][0][8][14] = 42,
- [0][1][1][0][11][14] = 40,
- [0][1][1][0][2][15] = 68,
- [0][1][1][0][1][15] = 46,
- [0][1][1][0][3][15] = 70,
- [0][1][1][0][5][15] = 68,
- [0][1][1][0][6][15] = 46,
- [0][1][1][0][9][15] = 46,
- [0][1][1][0][8][15] = 42,
- [0][1][1][0][11][15] = 40,
- [0][1][1][0][2][17] = 68,
- [0][1][1][0][1][17] = 46,
- [0][1][1][0][3][17] = 70,
- [0][1][1][0][5][17] = 68,
- [0][1][1][0][6][17] = 46,
- [0][1][1][0][9][17] = 46,
- [0][1][1][0][8][17] = 42,
- [0][1][1][0][11][17] = 40,
- [0][1][1][0][2][19] = 68,
- [0][1][1][0][1][19] = 46,
- [0][1][1][0][3][19] = 70,
- [0][1][1][0][5][19] = 68,
- [0][1][1][0][6][19] = 46,
- [0][1][1][0][9][19] = 46,
- [0][1][1][0][8][19] = 42,
- [0][1][1][0][11][19] = 40,
- [0][1][1][0][2][21] = 68,
- [0][1][1][0][1][21] = 46,
- [0][1][1][0][3][21] = 70,
- [0][1][1][0][5][21] = 68,
- [0][1][1][0][6][21] = 46,
- [0][1][1][0][9][21] = 46,
- [0][1][1][0][8][21] = 42,
- [0][1][1][0][11][21] = 40,
- [0][1][1][0][2][23] = 68,
- [0][1][1][0][1][23] = 46,
- [0][1][1][0][3][23] = 70,
- [0][1][1][0][5][23] = 68,
- [0][1][1][0][6][23] = 46,
- [0][1][1][0][9][23] = 46,
- [0][1][1][0][8][23] = 42,
- [0][1][1][0][11][23] = 40,
- [0][1][1][0][2][25] = 68,
- [0][1][1][0][1][25] = 46,
- [0][1][1][0][3][25] = 70,
- [0][1][1][0][5][25] = 127,
- [0][1][1][0][6][25] = 46,
- [0][1][1][0][9][25] = 127,
- [0][1][1][0][8][25] = 42,
- [0][1][1][0][11][25] = 40,
- [0][1][1][0][2][27] = 68,
- [0][1][1][0][1][27] = 46,
- [0][1][1][0][3][27] = 70,
- [0][1][1][0][5][27] = 127,
- [0][1][1][0][6][27] = 46,
- [0][1][1][0][9][27] = 127,
- [0][1][1][0][8][27] = 42,
- [0][1][1][0][11][27] = 40,
- [0][1][1][0][2][29] = 68,
- [0][1][1][0][1][29] = 46,
- [0][1][1][0][3][29] = 70,
- [0][1][1][0][5][29] = 127,
- [0][1][1][0][6][29] = 46,
- [0][1][1][0][9][29] = 127,
- [0][1][1][0][8][29] = 42,
- [0][1][1][0][11][29] = 40,
- [0][1][1][0][2][31] = 68,
- [0][1][1][0][1][31] = 46,
- [0][1][1][0][3][31] = 70,
- [0][1][1][0][5][31] = 68,
- [0][1][1][0][6][31] = 46,
- [0][1][1][0][9][31] = 46,
- [0][1][1][0][8][31] = 42,
- [0][1][1][0][11][31] = 40,
- [0][1][1][0][2][33] = 68,
- [0][1][1][0][1][33] = 46,
- [0][1][1][0][3][33] = 70,
- [0][1][1][0][5][33] = 68,
- [0][1][1][0][6][33] = 46,
- [0][1][1][0][9][33] = 46,
- [0][1][1][0][8][33] = 42,
- [0][1][1][0][11][33] = 40,
- [0][1][1][0][2][35] = 66,
- [0][1][1][0][1][35] = 46,
- [0][1][1][0][3][35] = 70,
- [0][1][1][0][5][35] = 66,
- [0][1][1][0][6][35] = 46,
- [0][1][1][0][9][35] = 46,
- [0][1][1][0][8][35] = 42,
- [0][1][1][0][11][35] = 40,
- [0][1][1][0][2][37] = 68,
- [0][1][1][0][1][37] = 127,
- [0][1][1][0][3][37] = 70,
- [0][1][1][0][5][37] = 68,
- [0][1][1][0][6][37] = 46,
- [0][1][1][0][9][37] = 68,
- [0][1][1][0][8][37] = 42,
- [0][1][1][0][11][37] = 127,
- [0][1][1][0][2][38] = 76,
- [0][1][1][0][1][38] = 16,
- [0][1][1][0][3][38] = 127,
- [0][1][1][0][5][38] = 76,
- [0][1][1][0][6][38] = 16,
- [0][1][1][0][9][38] = 76,
- [0][1][1][0][8][38] = 42,
- [0][1][1][0][11][38] = 40,
- [0][1][1][0][2][40] = 76,
- [0][1][1][0][1][40] = 16,
- [0][1][1][0][3][40] = 127,
- [0][1][1][0][5][40] = 76,
- [0][1][1][0][6][40] = 16,
- [0][1][1][0][9][40] = 76,
- [0][1][1][0][8][40] = 42,
- [0][1][1][0][11][40] = 40,
- [0][1][1][0][2][42] = 76,
- [0][1][1][0][1][42] = 16,
- [0][1][1][0][3][42] = 127,
- [0][1][1][0][5][42] = 76,
- [0][1][1][0][6][42] = 16,
- [0][1][1][0][9][42] = 76,
- [0][1][1][0][8][42] = 42,
- [0][1][1][0][11][42] = 40,
- [0][1][1][0][2][44] = 76,
- [0][1][1][0][1][44] = 16,
- [0][1][1][0][3][44] = 127,
- [0][1][1][0][5][44] = 76,
- [0][1][1][0][6][44] = 16,
- [0][1][1][0][9][44] = 76,
- [0][1][1][0][8][44] = 42,
- [0][1][1][0][11][44] = 40,
- [0][1][1][0][2][46] = 76,
- [0][1][1][0][1][46] = 16,
- [0][1][1][0][3][46] = 127,
- [0][1][1][0][5][46] = 76,
- [0][1][1][0][6][46] = 16,
- [0][1][1][0][9][46] = 76,
- [0][1][1][0][8][46] = 42,
- [0][1][1][0][11][46] = 40,
- [0][0][2][0][2][0] = 76,
- [0][0][2][0][1][0] = 58,
- [0][0][2][0][3][0] = 62,
- [0][0][2][0][5][0] = 62,
- [0][0][2][0][6][0] = 58,
- [0][0][2][0][9][0] = 58,
- [0][0][2][0][8][0] = 30,
- [0][0][2][0][11][0] = 52,
- [0][0][2][0][2][2] = 76,
- [0][0][2][0][1][2] = 58,
- [0][0][2][0][3][2] = 62,
- [0][0][2][0][5][2] = 62,
- [0][0][2][0][6][2] = 58,
- [0][0][2][0][9][2] = 58,
- [0][0][2][0][8][2] = 30,
- [0][0][2][0][11][2] = 52,
- [0][0][2][0][2][4] = 76,
- [0][0][2][0][1][4] = 58,
- [0][0][2][0][3][4] = 62,
- [0][0][2][0][5][4] = 62,
- [0][0][2][0][6][4] = 58,
- [0][0][2][0][9][4] = 58,
- [0][0][2][0][8][4] = 30,
- [0][0][2][0][11][4] = 52,
- [0][0][2][0][2][6] = 76,
- [0][0][2][0][1][6] = 58,
- [0][0][2][0][3][6] = 62,
- [0][0][2][0][5][6] = 62,
- [0][0][2][0][6][6] = 54,
- [0][0][2][0][9][6] = 58,
- [0][0][2][0][8][6] = 30,
- [0][0][2][0][11][6] = 52,
- [0][0][2][0][2][8] = 76,
- [0][0][2][0][1][8] = 58,
- [0][0][2][0][3][8] = 62,
- [0][0][2][0][5][8] = 64,
- [0][0][2][0][6][8] = 58,
- [0][0][2][0][9][8] = 58,
- [0][0][2][0][8][8] = 54,
- [0][0][2][0][11][8] = 52,
- [0][0][2][0][2][10] = 76,
- [0][0][2][0][1][10] = 58,
- [0][0][2][0][3][10] = 62,
- [0][0][2][0][5][10] = 64,
- [0][0][2][0][6][10] = 58,
- [0][0][2][0][9][10] = 58,
- [0][0][2][0][8][10] = 54,
- [0][0][2][0][11][10] = 52,
- [0][0][2][0][2][12] = 76,
- [0][0][2][0][1][12] = 58,
- [0][0][2][0][3][12] = 62,
- [0][0][2][0][5][12] = 64,
- [0][0][2][0][6][12] = 58,
- [0][0][2][0][9][12] = 58,
- [0][0][2][0][8][12] = 54,
- [0][0][2][0][11][12] = 52,
- [0][0][2][0][2][14] = 76,
- [0][0][2][0][1][14] = 58,
- [0][0][2][0][3][14] = 62,
- [0][0][2][0][5][14] = 64,
- [0][0][2][0][6][14] = 58,
- [0][0][2][0][9][14] = 58,
- [0][0][2][0][8][14] = 54,
- [0][0][2][0][11][14] = 52,
- [0][0][2][0][2][15] = 74,
- [0][0][2][0][1][15] = 58,
- [0][0][2][0][3][15] = 76,
- [0][0][2][0][5][15] = 74,
- [0][0][2][0][6][15] = 58,
- [0][0][2][0][9][15] = 58,
- [0][0][2][0][8][15] = 54,
- [0][0][2][0][11][15] = 52,
- [0][0][2][0][2][17] = 76,
- [0][0][2][0][1][17] = 58,
- [0][0][2][0][3][17] = 76,
- [0][0][2][0][5][17] = 76,
- [0][0][2][0][6][17] = 58,
- [0][0][2][0][9][17] = 58,
- [0][0][2][0][8][17] = 54,
- [0][0][2][0][11][17] = 52,
- [0][0][2][0][2][19] = 76,
- [0][0][2][0][1][19] = 58,
- [0][0][2][0][3][19] = 76,
- [0][0][2][0][5][19] = 76,
- [0][0][2][0][6][19] = 58,
- [0][0][2][0][9][19] = 58,
- [0][0][2][0][8][19] = 54,
- [0][0][2][0][11][19] = 52,
- [0][0][2][0][2][21] = 76,
- [0][0][2][0][1][21] = 58,
- [0][0][2][0][3][21] = 76,
- [0][0][2][0][5][21] = 76,
- [0][0][2][0][6][21] = 58,
- [0][0][2][0][9][21] = 58,
- [0][0][2][0][8][21] = 54,
- [0][0][2][0][11][21] = 52,
- [0][0][2][0][2][23] = 76,
- [0][0][2][0][1][23] = 58,
- [0][0][2][0][3][23] = 76,
- [0][0][2][0][5][23] = 76,
- [0][0][2][0][6][23] = 58,
- [0][0][2][0][9][23] = 58,
- [0][0][2][0][8][23] = 54,
- [0][0][2][0][11][23] = 52,
- [0][0][2][0][2][25] = 76,
- [0][0][2][0][1][25] = 58,
- [0][0][2][0][3][25] = 76,
- [0][0][2][0][5][25] = 127,
- [0][0][2][0][6][25] = 58,
- [0][0][2][0][9][25] = 127,
- [0][0][2][0][8][25] = 54,
- [0][0][2][0][11][25] = 52,
- [0][0][2][0][2][27] = 76,
- [0][0][2][0][1][27] = 58,
- [0][0][2][0][3][27] = 76,
- [0][0][2][0][5][27] = 127,
- [0][0][2][0][6][27] = 58,
- [0][0][2][0][9][27] = 127,
- [0][0][2][0][8][27] = 54,
- [0][0][2][0][11][27] = 52,
- [0][0][2][0][2][29] = 76,
- [0][0][2][0][1][29] = 58,
- [0][0][2][0][3][29] = 76,
- [0][0][2][0][5][29] = 127,
- [0][0][2][0][6][29] = 58,
- [0][0][2][0][9][29] = 127,
- [0][0][2][0][8][29] = 54,
- [0][0][2][0][11][29] = 52,
- [0][0][2][0][2][31] = 76,
- [0][0][2][0][1][31] = 58,
- [0][0][2][0][3][31] = 76,
- [0][0][2][0][5][31] = 76,
- [0][0][2][0][6][31] = 58,
- [0][0][2][0][9][31] = 58,
- [0][0][2][0][8][31] = 54,
- [0][0][2][0][11][31] = 52,
- [0][0][2][0][2][33] = 76,
- [0][0][2][0][1][33] = 58,
- [0][0][2][0][3][33] = 76,
- [0][0][2][0][5][33] = 76,
- [0][0][2][0][6][33] = 58,
- [0][0][2][0][9][33] = 58,
- [0][0][2][0][8][33] = 54,
- [0][0][2][0][11][33] = 52,
- [0][0][2][0][2][35] = 70,
- [0][0][2][0][1][35] = 58,
- [0][0][2][0][3][35] = 76,
- [0][0][2][0][5][35] = 70,
- [0][0][2][0][6][35] = 58,
- [0][0][2][0][9][35] = 58,
- [0][0][2][0][8][35] = 54,
- [0][0][2][0][11][35] = 52,
- [0][0][2][0][2][37] = 76,
- [0][0][2][0][1][37] = 127,
- [0][0][2][0][3][37] = 76,
- [0][0][2][0][5][37] = 76,
- [0][0][2][0][6][37] = 58,
- [0][0][2][0][9][37] = 76,
- [0][0][2][0][8][37] = 54,
- [0][0][2][0][11][37] = 127,
- [0][0][2][0][2][38] = 76,
- [0][0][2][0][1][38] = 28,
- [0][0][2][0][3][38] = 127,
- [0][0][2][0][5][38] = 76,
- [0][0][2][0][6][38] = 28,
- [0][0][2][0][9][38] = 76,
- [0][0][2][0][8][38] = 54,
- [0][0][2][0][11][38] = 52,
- [0][0][2][0][2][40] = 76,
- [0][0][2][0][1][40] = 28,
- [0][0][2][0][3][40] = 127,
- [0][0][2][0][5][40] = 76,
- [0][0][2][0][6][40] = 28,
- [0][0][2][0][9][40] = 76,
- [0][0][2][0][8][40] = 54,
- [0][0][2][0][11][40] = 52,
- [0][0][2][0][2][42] = 76,
- [0][0][2][0][1][42] = 28,
- [0][0][2][0][3][42] = 127,
- [0][0][2][0][5][42] = 76,
- [0][0][2][0][6][42] = 28,
- [0][0][2][0][9][42] = 76,
- [0][0][2][0][8][42] = 54,
- [0][0][2][0][11][42] = 52,
- [0][0][2][0][2][44] = 76,
- [0][0][2][0][1][44] = 28,
- [0][0][2][0][3][44] = 127,
- [0][0][2][0][5][44] = 76,
- [0][0][2][0][6][44] = 28,
- [0][0][2][0][9][44] = 76,
- [0][0][2][0][8][44] = 54,
- [0][0][2][0][11][44] = 52,
- [0][0][2][0][2][46] = 76,
- [0][0][2][0][1][46] = 28,
- [0][0][2][0][3][46] = 127,
- [0][0][2][0][5][46] = 76,
- [0][0][2][0][6][46] = 28,
- [0][0][2][0][9][46] = 76,
- [0][0][2][0][8][46] = 54,
- [0][0][2][0][11][46] = 52,
- [0][1][2][0][2][0] = 68,
- [0][1][2][0][1][0] = 46,
- [0][1][2][0][3][0] = 50,
- [0][1][2][0][5][0] = 40,
- [0][1][2][0][6][0] = 46,
- [0][1][2][0][9][0] = 46,
- [0][1][2][0][8][0] = 18,
- [0][1][2][0][11][0] = 40,
- [0][1][2][0][2][2] = 68,
- [0][1][2][0][1][2] = 46,
- [0][1][2][0][3][2] = 50,
- [0][1][2][0][5][2] = 40,
- [0][1][2][0][6][2] = 46,
- [0][1][2][0][9][2] = 46,
- [0][1][2][0][8][2] = 18,
- [0][1][2][0][11][2] = 40,
- [0][1][2][0][2][4] = 68,
- [0][1][2][0][1][4] = 46,
- [0][1][2][0][3][4] = 50,
- [0][1][2][0][5][4] = 40,
- [0][1][2][0][6][4] = 46,
- [0][1][2][0][9][4] = 46,
- [0][1][2][0][8][4] = 18,
- [0][1][2][0][11][4] = 40,
- [0][1][2][0][2][6] = 68,
- [0][1][2][0][1][6] = 46,
- [0][1][2][0][3][6] = 50,
- [0][1][2][0][5][6] = 40,
- [0][1][2][0][6][6] = 36,
- [0][1][2][0][9][6] = 46,
- [0][1][2][0][8][6] = 18,
- [0][1][2][0][11][6] = 40,
- [0][1][2][0][2][8] = 68,
- [0][1][2][0][1][8] = 46,
- [0][1][2][0][3][8] = 50,
- [0][1][2][0][5][8] = 52,
- [0][1][2][0][6][8] = 46,
- [0][1][2][0][9][8] = 46,
- [0][1][2][0][8][8] = 42,
- [0][1][2][0][11][8] = 40,
- [0][1][2][0][2][10] = 68,
- [0][1][2][0][1][10] = 46,
- [0][1][2][0][3][10] = 50,
- [0][1][2][0][5][10] = 52,
- [0][1][2][0][6][10] = 46,
- [0][1][2][0][9][10] = 46,
- [0][1][2][0][8][10] = 42,
- [0][1][2][0][11][10] = 40,
- [0][1][2][0][2][12] = 68,
- [0][1][2][0][1][12] = 46,
- [0][1][2][0][3][12] = 50,
- [0][1][2][0][5][12] = 52,
- [0][1][2][0][6][12] = 46,
- [0][1][2][0][9][12] = 46,
- [0][1][2][0][8][12] = 42,
- [0][1][2][0][11][12] = 40,
- [0][1][2][0][2][14] = 68,
- [0][1][2][0][1][14] = 46,
- [0][1][2][0][3][14] = 50,
- [0][1][2][0][5][14] = 52,
- [0][1][2][0][6][14] = 46,
- [0][1][2][0][9][14] = 46,
- [0][1][2][0][8][14] = 42,
- [0][1][2][0][11][14] = 40,
- [0][1][2][0][2][15] = 68,
- [0][1][2][0][1][15] = 46,
- [0][1][2][0][3][15] = 70,
- [0][1][2][0][5][15] = 68,
- [0][1][2][0][6][15] = 46,
- [0][1][2][0][9][15] = 46,
- [0][1][2][0][8][15] = 42,
- [0][1][2][0][11][15] = 40,
- [0][1][2][0][2][17] = 68,
- [0][1][2][0][1][17] = 46,
- [0][1][2][0][3][17] = 70,
- [0][1][2][0][5][17] = 68,
- [0][1][2][0][6][17] = 46,
- [0][1][2][0][9][17] = 46,
- [0][1][2][0][8][17] = 42,
- [0][1][2][0][11][17] = 40,
- [0][1][2][0][2][19] = 68,
- [0][1][2][0][1][19] = 46,
- [0][1][2][0][3][19] = 70,
- [0][1][2][0][5][19] = 68,
- [0][1][2][0][6][19] = 46,
- [0][1][2][0][9][19] = 46,
- [0][1][2][0][8][19] = 42,
- [0][1][2][0][11][19] = 40,
- [0][1][2][0][2][21] = 68,
- [0][1][2][0][1][21] = 46,
- [0][1][2][0][3][21] = 70,
- [0][1][2][0][5][21] = 68,
- [0][1][2][0][6][21] = 46,
- [0][1][2][0][9][21] = 46,
- [0][1][2][0][8][21] = 42,
- [0][1][2][0][11][21] = 40,
- [0][1][2][0][2][23] = 68,
- [0][1][2][0][1][23] = 46,
- [0][1][2][0][3][23] = 70,
- [0][1][2][0][5][23] = 68,
- [0][1][2][0][6][23] = 46,
- [0][1][2][0][9][23] = 46,
- [0][1][2][0][8][23] = 42,
- [0][1][2][0][11][23] = 40,
- [0][1][2][0][2][25] = 68,
- [0][1][2][0][1][25] = 46,
- [0][1][2][0][3][25] = 70,
- [0][1][2][0][5][25] = 127,
- [0][1][2][0][6][25] = 46,
- [0][1][2][0][9][25] = 127,
- [0][1][2][0][8][25] = 42,
- [0][1][2][0][11][25] = 40,
- [0][1][2][0][2][27] = 68,
- [0][1][2][0][1][27] = 46,
- [0][1][2][0][3][27] = 70,
- [0][1][2][0][5][27] = 127,
- [0][1][2][0][6][27] = 46,
- [0][1][2][0][9][27] = 127,
- [0][1][2][0][8][27] = 42,
- [0][1][2][0][11][27] = 40,
- [0][1][2][0][2][29] = 68,
- [0][1][2][0][1][29] = 46,
- [0][1][2][0][3][29] = 70,
- [0][1][2][0][5][29] = 127,
- [0][1][2][0][6][29] = 46,
- [0][1][2][0][9][29] = 127,
- [0][1][2][0][8][29] = 42,
- [0][1][2][0][11][29] = 40,
- [0][1][2][0][2][31] = 68,
- [0][1][2][0][1][31] = 46,
- [0][1][2][0][3][31] = 70,
- [0][1][2][0][5][31] = 68,
- [0][1][2][0][6][31] = 46,
- [0][1][2][0][9][31] = 46,
- [0][1][2][0][8][31] = 42,
- [0][1][2][0][11][31] = 40,
- [0][1][2][0][2][33] = 68,
- [0][1][2][0][1][33] = 46,
- [0][1][2][0][3][33] = 70,
- [0][1][2][0][5][33] = 68,
- [0][1][2][0][6][33] = 46,
- [0][1][2][0][9][33] = 46,
- [0][1][2][0][8][33] = 42,
- [0][1][2][0][11][33] = 40,
- [0][1][2][0][2][35] = 64,
- [0][1][2][0][1][35] = 46,
- [0][1][2][0][3][35] = 70,
- [0][1][2][0][5][35] = 64,
- [0][1][2][0][6][35] = 46,
- [0][1][2][0][9][35] = 46,
- [0][1][2][0][8][35] = 42,
- [0][1][2][0][11][35] = 40,
- [0][1][2][0][2][37] = 68,
- [0][1][2][0][1][37] = 127,
- [0][1][2][0][3][37] = 70,
- [0][1][2][0][5][37] = 68,
- [0][1][2][0][6][37] = 46,
- [0][1][2][0][9][37] = 68,
- [0][1][2][0][8][37] = 42,
- [0][1][2][0][11][37] = 127,
- [0][1][2][0][2][38] = 76,
- [0][1][2][0][1][38] = 16,
- [0][1][2][0][3][38] = 127,
- [0][1][2][0][5][38] = 76,
- [0][1][2][0][6][38] = 16,
- [0][1][2][0][9][38] = 76,
- [0][1][2][0][8][38] = 42,
- [0][1][2][0][11][38] = 40,
- [0][1][2][0][2][40] = 76,
- [0][1][2][0][1][40] = 16,
- [0][1][2][0][3][40] = 127,
- [0][1][2][0][5][40] = 76,
- [0][1][2][0][6][40] = 16,
- [0][1][2][0][9][40] = 76,
- [0][1][2][0][8][40] = 42,
- [0][1][2][0][11][40] = 40,
- [0][1][2][0][2][42] = 76,
- [0][1][2][0][1][42] = 16,
- [0][1][2][0][3][42] = 127,
- [0][1][2][0][5][42] = 76,
- [0][1][2][0][6][42] = 16,
- [0][1][2][0][9][42] = 76,
- [0][1][2][0][8][42] = 42,
- [0][1][2][0][11][42] = 40,
- [0][1][2][0][2][44] = 76,
- [0][1][2][0][1][44] = 16,
- [0][1][2][0][3][44] = 127,
- [0][1][2][0][5][44] = 76,
- [0][1][2][0][6][44] = 16,
- [0][1][2][0][9][44] = 76,
- [0][1][2][0][8][44] = 42,
- [0][1][2][0][11][44] = 40,
- [0][1][2][0][2][46] = 76,
- [0][1][2][0][1][46] = 16,
- [0][1][2][0][3][46] = 127,
- [0][1][2][0][5][46] = 76,
- [0][1][2][0][6][46] = 16,
- [0][1][2][0][9][46] = 76,
- [0][1][2][0][8][46] = 42,
- [0][1][2][0][11][46] = 40,
- [0][1][2][1][2][0] = 68,
- [0][1][2][1][1][0] = 34,
- [0][1][2][1][3][0] = 50,
- [0][1][2][1][5][0] = 38,
- [0][1][2][1][6][0] = 34,
- [0][1][2][1][9][0] = 34,
- [0][1][2][1][8][0] = 6,
- [0][1][2][1][11][0] = 28,
- [0][1][2][1][2][2] = 68,
- [0][1][2][1][1][2] = 34,
- [0][1][2][1][3][2] = 50,
- [0][1][2][1][5][2] = 38,
- [0][1][2][1][6][2] = 34,
- [0][1][2][1][9][2] = 34,
- [0][1][2][1][8][2] = 6,
- [0][1][2][1][11][2] = 28,
- [0][1][2][1][2][4] = 68,
- [0][1][2][1][1][4] = 34,
- [0][1][2][1][3][4] = 50,
- [0][1][2][1][5][4] = 38,
- [0][1][2][1][6][4] = 34,
- [0][1][2][1][9][4] = 34,
- [0][1][2][1][8][4] = 6,
- [0][1][2][1][11][4] = 28,
- [0][1][2][1][2][6] = 68,
- [0][1][2][1][1][6] = 34,
- [0][1][2][1][3][6] = 50,
- [0][1][2][1][5][6] = 38,
- [0][1][2][1][6][6] = 34,
- [0][1][2][1][9][6] = 34,
- [0][1][2][1][8][6] = 6,
- [0][1][2][1][11][6] = 28,
- [0][1][2][1][2][8] = 68,
- [0][1][2][1][1][8] = 34,
- [0][1][2][1][3][8] = 50,
- [0][1][2][1][5][8] = 38,
- [0][1][2][1][6][8] = 34,
- [0][1][2][1][9][8] = 34,
- [0][1][2][1][8][8] = 30,
- [0][1][2][1][11][8] = 28,
- [0][1][2][1][2][10] = 68,
- [0][1][2][1][1][10] = 34,
- [0][1][2][1][3][10] = 50,
- [0][1][2][1][5][10] = 38,
- [0][1][2][1][6][10] = 34,
- [0][1][2][1][9][10] = 34,
- [0][1][2][1][8][10] = 30,
- [0][1][2][1][11][10] = 28,
- [0][1][2][1][2][12] = 68,
- [0][1][2][1][1][12] = 34,
- [0][1][2][1][3][12] = 50,
- [0][1][2][1][5][12] = 38,
- [0][1][2][1][6][12] = 34,
- [0][1][2][1][9][12] = 34,
- [0][1][2][1][8][12] = 30,
- [0][1][2][1][11][12] = 28,
- [0][1][2][1][2][14] = 68,
- [0][1][2][1][1][14] = 34,
- [0][1][2][1][3][14] = 50,
- [0][1][2][1][5][14] = 38,
- [0][1][2][1][6][14] = 34,
- [0][1][2][1][9][14] = 34,
- [0][1][2][1][8][14] = 30,
- [0][1][2][1][11][14] = 28,
- [0][1][2][1][2][15] = 68,
- [0][1][2][1][1][15] = 34,
- [0][1][2][1][3][15] = 70,
- [0][1][2][1][5][15] = 62,
- [0][1][2][1][6][15] = 34,
- [0][1][2][1][9][15] = 34,
- [0][1][2][1][8][15] = 30,
- [0][1][2][1][11][15] = 28,
- [0][1][2][1][2][17] = 68,
- [0][1][2][1][1][17] = 34,
- [0][1][2][1][3][17] = 70,
- [0][1][2][1][5][17] = 62,
- [0][1][2][1][6][17] = 34,
- [0][1][2][1][9][17] = 34,
- [0][1][2][1][8][17] = 30,
- [0][1][2][1][11][17] = 28,
- [0][1][2][1][2][19] = 68,
- [0][1][2][1][1][19] = 34,
- [0][1][2][1][3][19] = 70,
- [0][1][2][1][5][19] = 62,
- [0][1][2][1][6][19] = 34,
- [0][1][2][1][9][19] = 34,
- [0][1][2][1][8][19] = 30,
- [0][1][2][1][11][19] = 28,
- [0][1][2][1][2][21] = 68,
- [0][1][2][1][1][21] = 34,
- [0][1][2][1][3][21] = 70,
- [0][1][2][1][5][21] = 62,
- [0][1][2][1][6][21] = 34,
- [0][1][2][1][9][21] = 34,
- [0][1][2][1][8][21] = 30,
- [0][1][2][1][11][21] = 28,
- [0][1][2][1][2][23] = 68,
- [0][1][2][1][1][23] = 34,
- [0][1][2][1][3][23] = 70,
- [0][1][2][1][5][23] = 62,
- [0][1][2][1][6][23] = 34,
- [0][1][2][1][9][23] = 34,
- [0][1][2][1][8][23] = 30,
- [0][1][2][1][11][23] = 28,
- [0][1][2][1][2][25] = 68,
- [0][1][2][1][1][25] = 34,
- [0][1][2][1][3][25] = 70,
- [0][1][2][1][5][25] = 127,
- [0][1][2][1][6][25] = 34,
- [0][1][2][1][9][25] = 127,
- [0][1][2][1][8][25] = 30,
- [0][1][2][1][11][25] = 28,
- [0][1][2][1][2][27] = 68,
- [0][1][2][1][1][27] = 34,
- [0][1][2][1][3][27] = 70,
- [0][1][2][1][5][27] = 127,
- [0][1][2][1][6][27] = 34,
- [0][1][2][1][9][27] = 127,
- [0][1][2][1][8][27] = 30,
- [0][1][2][1][11][27] = 28,
- [0][1][2][1][2][29] = 68,
- [0][1][2][1][1][29] = 34,
- [0][1][2][1][3][29] = 70,
- [0][1][2][1][5][29] = 127,
- [0][1][2][1][6][29] = 34,
- [0][1][2][1][9][29] = 127,
- [0][1][2][1][8][29] = 30,
- [0][1][2][1][11][29] = 28,
- [0][1][2][1][2][31] = 68,
- [0][1][2][1][1][31] = 34,
- [0][1][2][1][3][31] = 70,
- [0][1][2][1][5][31] = 62,
- [0][1][2][1][6][31] = 34,
- [0][1][2][1][9][31] = 34,
- [0][1][2][1][8][31] = 30,
- [0][1][2][1][11][31] = 28,
- [0][1][2][1][2][33] = 68,
- [0][1][2][1][1][33] = 34,
- [0][1][2][1][3][33] = 70,
- [0][1][2][1][5][33] = 62,
- [0][1][2][1][6][33] = 34,
- [0][1][2][1][9][33] = 34,
- [0][1][2][1][8][33] = 30,
- [0][1][2][1][11][33] = 28,
- [0][1][2][1][2][35] = 64,
- [0][1][2][1][1][35] = 34,
- [0][1][2][1][3][35] = 70,
- [0][1][2][1][5][35] = 62,
- [0][1][2][1][6][35] = 34,
- [0][1][2][1][9][35] = 34,
- [0][1][2][1][8][35] = 30,
- [0][1][2][1][11][35] = 28,
- [0][1][2][1][2][37] = 68,
- [0][1][2][1][1][37] = 127,
- [0][1][2][1][3][37] = 70,
- [0][1][2][1][5][37] = 62,
- [0][1][2][1][6][37] = 34,
- [0][1][2][1][9][37] = 68,
- [0][1][2][1][8][37] = 30,
- [0][1][2][1][11][37] = 127,
- [0][1][2][1][2][38] = 76,
- [0][1][2][1][1][38] = 4,
- [0][1][2][1][3][38] = 127,
- [0][1][2][1][5][38] = 76,
- [0][1][2][1][6][38] = 4,
- [0][1][2][1][9][38] = 76,
- [0][1][2][1][8][38] = 30,
- [0][1][2][1][11][38] = 28,
- [0][1][2][1][2][40] = 76,
- [0][1][2][1][1][40] = 4,
- [0][1][2][1][3][40] = 127,
- [0][1][2][1][5][40] = 76,
- [0][1][2][1][6][40] = 4,
- [0][1][2][1][9][40] = 76,
- [0][1][2][1][8][40] = 30,
- [0][1][2][1][11][40] = 28,
- [0][1][2][1][2][42] = 76,
- [0][1][2][1][1][42] = 4,
- [0][1][2][1][3][42] = 127,
- [0][1][2][1][5][42] = 76,
- [0][1][2][1][6][42] = 4,
- [0][1][2][1][9][42] = 76,
- [0][1][2][1][8][42] = 30,
- [0][1][2][1][11][42] = 28,
- [0][1][2][1][2][44] = 76,
- [0][1][2][1][1][44] = 4,
- [0][1][2][1][3][44] = 127,
- [0][1][2][1][5][44] = 76,
- [0][1][2][1][6][44] = 4,
- [0][1][2][1][9][44] = 76,
- [0][1][2][1][8][44] = 30,
- [0][1][2][1][11][44] = 28,
- [0][1][2][1][2][46] = 76,
- [0][1][2][1][1][46] = 4,
- [0][1][2][1][3][46] = 127,
- [0][1][2][1][5][46] = 76,
- [0][1][2][1][6][46] = 4,
- [0][1][2][1][9][46] = 76,
- [0][1][2][1][8][46] = 30,
- [0][1][2][1][11][46] = 28,
- [1][0][2][0][2][1] = 68,
- [1][0][2][0][1][1] = 64,
- [1][0][2][0][3][1] = 62,
- [1][0][2][0][5][1] = 64,
- [1][0][2][0][6][1] = 64,
- [1][0][2][0][9][1] = 64,
- [1][0][2][0][8][1] = 30,
- [1][0][2][0][11][1] = 52,
- [1][0][2][0][2][5] = 72,
- [1][0][2][0][1][5] = 64,
- [1][0][2][0][3][5] = 62,
- [1][0][2][0][5][5] = 64,
- [1][0][2][0][6][5] = 60,
- [1][0][2][0][9][5] = 64,
- [1][0][2][0][8][5] = 30,
- [1][0][2][0][11][5] = 52,
- [1][0][2][0][2][9] = 72,
- [1][0][2][0][1][9] = 64,
- [1][0][2][0][3][9] = 62,
- [1][0][2][0][5][9] = 64,
- [1][0][2][0][6][9] = 64,
- [1][0][2][0][9][9] = 64,
- [1][0][2][0][8][9] = 54,
- [1][0][2][0][11][9] = 52,
- [1][0][2][0][2][13] = 66,
- [1][0][2][0][1][13] = 64,
- [1][0][2][0][3][13] = 62,
- [1][0][2][0][5][13] = 64,
- [1][0][2][0][6][13] = 64,
- [1][0][2][0][9][13] = 64,
- [1][0][2][0][8][13] = 54,
- [1][0][2][0][11][13] = 52,
- [1][0][2][0][2][16] = 62,
- [1][0][2][0][1][16] = 64,
- [1][0][2][0][3][16] = 72,
- [1][0][2][0][5][16] = 62,
- [1][0][2][0][6][16] = 64,
- [1][0][2][0][9][16] = 64,
- [1][0][2][0][8][16] = 54,
- [1][0][2][0][11][16] = 52,
- [1][0][2][0][2][20] = 72,
- [1][0][2][0][1][20] = 64,
- [1][0][2][0][3][20] = 72,
- [1][0][2][0][5][20] = 72,
- [1][0][2][0][6][20] = 64,
- [1][0][2][0][9][20] = 64,
- [1][0][2][0][8][20] = 54,
- [1][0][2][0][11][20] = 52,
- [1][0][2][0][2][24] = 72,
- [1][0][2][0][1][24] = 64,
- [1][0][2][0][3][24] = 72,
- [1][0][2][0][5][24] = 127,
- [1][0][2][0][6][24] = 64,
- [1][0][2][0][9][24] = 127,
- [1][0][2][0][8][24] = 54,
- [1][0][2][0][11][24] = 52,
- [1][0][2][0][2][28] = 72,
- [1][0][2][0][1][28] = 64,
- [1][0][2][0][3][28] = 72,
- [1][0][2][0][5][28] = 127,
- [1][0][2][0][6][28] = 64,
- [1][0][2][0][9][28] = 127,
- [1][0][2][0][8][28] = 54,
- [1][0][2][0][11][28] = 52,
- [1][0][2][0][2][32] = 72,
- [1][0][2][0][1][32] = 64,
- [1][0][2][0][3][32] = 72,
- [1][0][2][0][5][32] = 72,
- [1][0][2][0][6][32] = 64,
- [1][0][2][0][9][32] = 64,
- [1][0][2][0][8][32] = 54,
- [1][0][2][0][11][32] = 52,
- [1][0][2][0][2][36] = 72,
- [1][0][2][0][1][36] = 127,
- [1][0][2][0][3][36] = 72,
- [1][0][2][0][5][36] = 72,
- [1][0][2][0][6][36] = 64,
- [1][0][2][0][9][36] = 72,
- [1][0][2][0][8][36] = 54,
- [1][0][2][0][11][36] = 127,
- [1][0][2][0][2][39] = 72,
- [1][0][2][0][1][39] = 28,
- [1][0][2][0][3][39] = 127,
- [1][0][2][0][5][39] = 72,
- [1][0][2][0][6][39] = 28,
- [1][0][2][0][9][39] = 72,
- [1][0][2][0][8][39] = 54,
- [1][0][2][0][11][39] = 52,
- [1][0][2][0][2][43] = 72,
- [1][0][2][0][1][43] = 28,
- [1][0][2][0][3][43] = 127,
- [1][0][2][0][5][43] = 72,
- [1][0][2][0][6][43] = 28,
- [1][0][2][0][9][43] = 72,
- [1][0][2][0][8][43] = 54,
- [1][0][2][0][11][43] = 52,
- [1][1][2][0][2][1] = 58,
- [1][1][2][0][1][1] = 52,
- [1][1][2][0][3][1] = 50,
- [1][1][2][0][5][1] = 52,
- [1][1][2][0][6][1] = 52,
- [1][1][2][0][9][1] = 52,
- [1][1][2][0][8][1] = 18,
- [1][1][2][0][11][1] = 40,
- [1][1][2][0][2][5] = 72,
- [1][1][2][0][1][5] = 52,
- [1][1][2][0][3][5] = 50,
- [1][1][2][0][5][5] = 52,
- [1][1][2][0][6][5] = 46,
- [1][1][2][0][9][5] = 52,
- [1][1][2][0][8][5] = 18,
- [1][1][2][0][11][5] = 40,
- [1][1][2][0][2][9] = 72,
- [1][1][2][0][1][9] = 52,
- [1][1][2][0][3][9] = 50,
- [1][1][2][0][5][9] = 52,
- [1][1][2][0][6][9] = 52,
- [1][1][2][0][9][9] = 52,
- [1][1][2][0][8][9] = 42,
- [1][1][2][0][11][9] = 40,
- [1][1][2][0][2][13] = 58,
- [1][1][2][0][1][13] = 52,
- [1][1][2][0][3][13] = 50,
- [1][1][2][0][5][13] = 52,
- [1][1][2][0][6][13] = 52,
- [1][1][2][0][9][13] = 52,
- [1][1][2][0][8][13] = 42,
- [1][1][2][0][11][13] = 40,
- [1][1][2][0][2][16] = 56,
- [1][1][2][0][1][16] = 52,
- [1][1][2][0][3][16] = 72,
- [1][1][2][0][5][16] = 56,
- [1][1][2][0][6][16] = 52,
- [1][1][2][0][9][16] = 52,
- [1][1][2][0][8][16] = 42,
- [1][1][2][0][11][16] = 40,
- [1][1][2][0][2][20] = 72,
- [1][1][2][0][1][20] = 52,
- [1][1][2][0][3][20] = 72,
- [1][1][2][0][5][20] = 72,
- [1][1][2][0][6][20] = 52,
- [1][1][2][0][9][20] = 52,
- [1][1][2][0][8][20] = 42,
- [1][1][2][0][11][20] = 40,
- [1][1][2][0][2][24] = 72,
- [1][1][2][0][1][24] = 52,
- [1][1][2][0][3][24] = 72,
- [1][1][2][0][5][24] = 127,
- [1][1][2][0][6][24] = 52,
- [1][1][2][0][9][24] = 127,
- [1][1][2][0][8][24] = 42,
- [1][1][2][0][11][24] = 40,
- [1][1][2][0][2][28] = 72,
- [1][1][2][0][1][28] = 52,
- [1][1][2][0][3][28] = 72,
- [1][1][2][0][5][28] = 127,
- [1][1][2][0][6][28] = 52,
- [1][1][2][0][9][28] = 127,
- [1][1][2][0][8][28] = 42,
- [1][1][2][0][11][28] = 40,
- [1][1][2][0][2][32] = 68,
- [1][1][2][0][1][32] = 52,
- [1][1][2][0][3][32] = 72,
- [1][1][2][0][5][32] = 68,
- [1][1][2][0][6][32] = 52,
- [1][1][2][0][9][32] = 52,
- [1][1][2][0][8][32] = 42,
- [1][1][2][0][11][32] = 40,
- [1][1][2][0][2][36] = 72,
- [1][1][2][0][1][36] = 127,
- [1][1][2][0][3][36] = 72,
- [1][1][2][0][5][36] = 72,
- [1][1][2][0][6][36] = 52,
- [1][1][2][0][9][36] = 72,
- [1][1][2][0][8][36] = 42,
- [1][1][2][0][11][36] = 127,
- [1][1][2][0][2][39] = 72,
- [1][1][2][0][1][39] = 16,
- [1][1][2][0][3][39] = 127,
- [1][1][2][0][5][39] = 72,
- [1][1][2][0][6][39] = 16,
- [1][1][2][0][9][39] = 72,
- [1][1][2][0][8][39] = 42,
- [1][1][2][0][11][39] = 40,
- [1][1][2][0][2][43] = 72,
- [1][1][2][0][1][43] = 16,
- [1][1][2][0][3][43] = 127,
- [1][1][2][0][5][43] = 72,
- [1][1][2][0][6][43] = 16,
- [1][1][2][0][9][43] = 72,
- [1][1][2][0][8][43] = 42,
- [1][1][2][0][11][43] = 40,
- [1][1][2][1][2][1] = 58,
- [1][1][2][1][1][1] = 40,
- [1][1][2][1][3][1] = 50,
- [1][1][2][1][5][1] = 40,
- [1][1][2][1][6][1] = 40,
- [1][1][2][1][9][1] = 40,
- [1][1][2][1][8][1] = 6,
- [1][1][2][1][11][1] = 28,
- [1][1][2][1][2][5] = 68,
- [1][1][2][1][1][5] = 40,
- [1][1][2][1][3][5] = 50,
- [1][1][2][1][5][5] = 40,
- [1][1][2][1][6][5] = 40,
- [1][1][2][1][9][5] = 40,
- [1][1][2][1][8][5] = 6,
- [1][1][2][1][11][5] = 28,
- [1][1][2][1][2][9] = 68,
- [1][1][2][1][1][9] = 40,
- [1][1][2][1][3][9] = 50,
- [1][1][2][1][5][9] = 40,
- [1][1][2][1][6][9] = 40,
- [1][1][2][1][9][9] = 40,
- [1][1][2][1][8][9] = 30,
- [1][1][2][1][11][9] = 28,
- [1][1][2][1][2][13] = 58,
- [1][1][2][1][1][13] = 40,
- [1][1][2][1][3][13] = 50,
- [1][1][2][1][5][13] = 40,
- [1][1][2][1][6][13] = 40,
- [1][1][2][1][9][13] = 40,
- [1][1][2][1][8][13] = 30,
- [1][1][2][1][11][13] = 28,
- [1][1][2][1][2][16] = 56,
- [1][1][2][1][1][16] = 40,
- [1][1][2][1][3][16] = 72,
- [1][1][2][1][5][16] = 56,
- [1][1][2][1][6][16] = 40,
- [1][1][2][1][9][16] = 40,
- [1][1][2][1][8][16] = 30,
- [1][1][2][1][11][16] = 28,
- [1][1][2][1][2][20] = 68,
- [1][1][2][1][1][20] = 40,
- [1][1][2][1][3][20] = 72,
- [1][1][2][1][5][20] = 68,
- [1][1][2][1][6][20] = 40,
- [1][1][2][1][9][20] = 40,
- [1][1][2][1][8][20] = 30,
- [1][1][2][1][11][20] = 28,
- [1][1][2][1][2][24] = 68,
- [1][1][2][1][1][24] = 40,
- [1][1][2][1][3][24] = 72,
- [1][1][2][1][5][24] = 127,
- [1][1][2][1][6][24] = 40,
- [1][1][2][1][9][24] = 127,
- [1][1][2][1][8][24] = 30,
- [1][1][2][1][11][24] = 28,
- [1][1][2][1][2][28] = 68,
- [1][1][2][1][1][28] = 40,
- [1][1][2][1][3][28] = 72,
- [1][1][2][1][5][28] = 127,
- [1][1][2][1][6][28] = 40,
- [1][1][2][1][9][28] = 127,
- [1][1][2][1][8][28] = 30,
- [1][1][2][1][11][28] = 28,
- [1][1][2][1][2][32] = 68,
- [1][1][2][1][1][32] = 40,
- [1][1][2][1][3][32] = 72,
- [1][1][2][1][5][32] = 68,
- [1][1][2][1][6][32] = 40,
- [1][1][2][1][9][32] = 40,
- [1][1][2][1][8][32] = 30,
- [1][1][2][1][11][32] = 28,
- [1][1][2][1][2][36] = 68,
- [1][1][2][1][1][36] = 127,
- [1][1][2][1][3][36] = 72,
- [1][1][2][1][5][36] = 68,
- [1][1][2][1][6][36] = 40,
- [1][1][2][1][9][36] = 68,
- [1][1][2][1][8][36] = 30,
- [1][1][2][1][11][36] = 127,
- [1][1][2][1][2][39] = 72,
- [1][1][2][1][1][39] = 4,
- [1][1][2][1][3][39] = 127,
- [1][1][2][1][5][39] = 72,
- [1][1][2][1][6][39] = 4,
- [1][1][2][1][9][39] = 72,
- [1][1][2][1][8][39] = 30,
- [1][1][2][1][11][39] = 28,
- [1][1][2][1][2][43] = 72,
- [1][1][2][1][1][43] = 4,
- [1][1][2][1][3][43] = 127,
- [1][1][2][1][5][43] = 72,
- [1][1][2][1][6][43] = 4,
- [1][1][2][1][9][43] = 72,
- [1][1][2][1][8][43] = 30,
- [1][1][2][1][11][43] = 28,
- [2][0][2][0][2][3] = 64,
- [2][0][2][0][1][3] = 64,
- [2][0][2][0][3][3] = 64,
- [2][0][2][0][5][3] = 62,
- [2][0][2][0][6][3] = 64,
- [2][0][2][0][9][3] = 64,
- [2][0][2][0][8][3] = 30,
- [2][0][2][0][11][3] = 52,
- [2][0][2][0][2][11] = 64,
- [2][0][2][0][1][11] = 64,
- [2][0][2][0][3][11] = 64,
- [2][0][2][0][5][11] = 62,
- [2][0][2][0][6][11] = 64,
- [2][0][2][0][9][11] = 64,
- [2][0][2][0][8][11] = 54,
- [2][0][2][0][11][11] = 52,
- [2][0][2][0][2][18] = 62,
- [2][0][2][0][1][18] = 64,
- [2][0][2][0][3][18] = 72,
- [2][0][2][0][5][18] = 66,
- [2][0][2][0][6][18] = 64,
- [2][0][2][0][9][18] = 64,
- [2][0][2][0][8][18] = 54,
- [2][0][2][0][11][18] = 52,
- [2][0][2][0][2][26] = 72,
- [2][0][2][0][1][26] = 64,
- [2][0][2][0][3][26] = 72,
- [2][0][2][0][5][26] = 127,
- [2][0][2][0][6][26] = 64,
- [2][0][2][0][9][26] = 127,
- [2][0][2][0][8][26] = 54,
- [2][0][2][0][11][26] = 52,
- [2][0][2][0][2][34] = 72,
- [2][0][2][0][1][34] = 127,
- [2][0][2][0][3][34] = 72,
- [2][0][2][0][5][34] = 72,
- [2][0][2][0][6][34] = 64,
- [2][0][2][0][9][34] = 72,
- [2][0][2][0][8][34] = 54,
- [2][0][2][0][11][34] = 127,
- [2][0][2][0][2][41] = 72,
- [2][0][2][0][1][41] = 28,
- [2][0][2][0][3][41] = 127,
- [2][0][2][0][5][41] = 72,
- [2][0][2][0][6][41] = 28,
- [2][0][2][0][9][41] = 72,
- [2][0][2][0][8][41] = 54,
- [2][0][2][0][11][41] = 52,
- [2][1][2][0][2][3] = 56,
- [2][1][2][0][1][3] = 52,
- [2][1][2][0][3][3] = 52,
- [2][1][2][0][5][3] = 52,
- [2][1][2][0][6][3] = 52,
- [2][1][2][0][9][3] = 52,
- [2][1][2][0][8][3] = 18,
- [2][1][2][0][11][3] = 40,
- [2][1][2][0][2][11] = 56,
- [2][1][2][0][1][11] = 52,
- [2][1][2][0][3][11] = 52,
- [2][1][2][0][5][11] = 52,
- [2][1][2][0][6][11] = 52,
- [2][1][2][0][9][11] = 52,
- [2][1][2][0][8][11] = 42,
- [2][1][2][0][11][11] = 40,
- [2][1][2][0][2][18] = 56,
- [2][1][2][0][1][18] = 52,
- [2][1][2][0][3][18] = 72,
- [2][1][2][0][5][18] = 56,
- [2][1][2][0][6][18] = 52,
- [2][1][2][0][9][18] = 52,
- [2][1][2][0][8][18] = 42,
- [2][1][2][0][11][18] = 40,
- [2][1][2][0][2][26] = 72,
- [2][1][2][0][1][26] = 52,
- [2][1][2][0][3][26] = 72,
- [2][1][2][0][5][26] = 127,
- [2][1][2][0][6][26] = 52,
- [2][1][2][0][9][26] = 127,
- [2][1][2][0][8][26] = 42,
- [2][1][2][0][11][26] = 40,
- [2][1][2][0][2][34] = 72,
- [2][1][2][0][1][34] = 127,
- [2][1][2][0][3][34] = 72,
- [2][1][2][0][5][34] = 72,
- [2][1][2][0][6][34] = 52,
- [2][1][2][0][9][34] = 72,
- [2][1][2][0][8][34] = 42,
- [2][1][2][0][11][34] = 127,
- [2][1][2][0][2][41] = 72,
- [2][1][2][0][1][41] = 16,
- [2][1][2][0][3][41] = 127,
- [2][1][2][0][5][41] = 72,
- [2][1][2][0][6][41] = 16,
- [2][1][2][0][9][41] = 72,
- [2][1][2][0][8][41] = 42,
- [2][1][2][0][11][41] = 40,
- [2][1][2][1][2][3] = 56,
- [2][1][2][1][1][3] = 40,
- [2][1][2][1][3][3] = 52,
- [2][1][2][1][5][3] = 40,
- [2][1][2][1][6][3] = 40,
- [2][1][2][1][9][3] = 40,
- [2][1][2][1][8][3] = 6,
- [2][1][2][1][11][3] = 28,
- [2][1][2][1][2][11] = 56,
- [2][1][2][1][1][11] = 40,
- [2][1][2][1][3][11] = 52,
- [2][1][2][1][5][11] = 40,
- [2][1][2][1][6][11] = 40,
- [2][1][2][1][9][11] = 40,
- [2][1][2][1][8][11] = 30,
- [2][1][2][1][11][11] = 28,
- [2][1][2][1][2][18] = 56,
- [2][1][2][1][1][18] = 40,
- [2][1][2][1][3][18] = 72,
- [2][1][2][1][5][18] = 56,
- [2][1][2][1][6][18] = 40,
- [2][1][2][1][9][18] = 40,
- [2][1][2][1][8][18] = 30,
- [2][1][2][1][11][18] = 28,
- [2][1][2][1][2][26] = 68,
- [2][1][2][1][1][26] = 40,
- [2][1][2][1][3][26] = 72,
- [2][1][2][1][5][26] = 127,
- [2][1][2][1][6][26] = 40,
- [2][1][2][1][9][26] = 127,
- [2][1][2][1][8][26] = 30,
- [2][1][2][1][11][26] = 28,
- [2][1][2][1][2][34] = 68,
- [2][1][2][1][1][34] = 127,
- [2][1][2][1][3][34] = 72,
- [2][1][2][1][5][34] = 68,
- [2][1][2][1][6][34] = 40,
- [2][1][2][1][9][34] = 68,
- [2][1][2][1][8][34] = 30,
- [2][1][2][1][11][34] = 127,
- [2][1][2][1][2][41] = 72,
- [2][1][2][1][1][41] = 4,
- [2][1][2][1][3][41] = 127,
- [2][1][2][1][5][41] = 72,
- [2][1][2][1][6][41] = 4,
- [2][1][2][1][9][41] = 72,
- [2][1][2][1][8][41] = 30,
- [2][1][2][1][11][41] = 28,
+ [0][0][1][0][RTW89_WW][0] = 30,
+ [0][0][1][0][RTW89_WW][2] = 30,
+ [0][0][1][0][RTW89_WW][4] = 30,
+ [0][0][1][0][RTW89_WW][6] = 30,
+ [0][0][1][0][RTW89_WW][8] = 52,
+ [0][0][1][0][RTW89_WW][10] = 52,
+ [0][0][1][0][RTW89_WW][12] = 52,
+ [0][0][1][0][RTW89_WW][14] = 52,
+ [0][0][1][0][RTW89_WW][15] = 52,
+ [0][0][1][0][RTW89_WW][17] = 52,
+ [0][0][1][0][RTW89_WW][19] = 52,
+ [0][0][1][0][RTW89_WW][21] = 52,
+ [0][0][1][0][RTW89_WW][23] = 52,
+ [0][0][1][0][RTW89_WW][25] = 52,
+ [0][0][1][0][RTW89_WW][27] = 52,
+ [0][0][1][0][RTW89_WW][29] = 52,
+ [0][0][1][0][RTW89_WW][31] = 52,
+ [0][0][1][0][RTW89_WW][33] = 52,
+ [0][0][1][0][RTW89_WW][35] = 52,
+ [0][0][1][0][RTW89_WW][37] = 54,
+ [0][0][1][0][RTW89_WW][38] = 28,
+ [0][0][1][0][RTW89_WW][40] = 28,
+ [0][0][1][0][RTW89_WW][42] = 28,
+ [0][0][1][0][RTW89_WW][44] = 28,
+ [0][0][1][0][RTW89_WW][46] = 28,
+ [0][1][1][0][RTW89_WW][0] = 18,
+ [0][1][1][0][RTW89_WW][2] = 18,
+ [0][1][1][0][RTW89_WW][4] = 18,
+ [0][1][1][0][RTW89_WW][6] = 18,
+ [0][1][1][0][RTW89_WW][8] = 40,
+ [0][1][1][0][RTW89_WW][10] = 40,
+ [0][1][1][0][RTW89_WW][12] = 40,
+ [0][1][1][0][RTW89_WW][14] = 40,
+ [0][1][1][0][RTW89_WW][15] = 40,
+ [0][1][1][0][RTW89_WW][17] = 40,
+ [0][1][1][0][RTW89_WW][19] = 40,
+ [0][1][1][0][RTW89_WW][21] = 40,
+ [0][1][1][0][RTW89_WW][23] = 40,
+ [0][1][1][0][RTW89_WW][25] = 40,
+ [0][1][1][0][RTW89_WW][27] = 40,
+ [0][1][1][0][RTW89_WW][29] = 40,
+ [0][1][1][0][RTW89_WW][31] = 40,
+ [0][1][1][0][RTW89_WW][33] = 40,
+ [0][1][1][0][RTW89_WW][35] = 40,
+ [0][1][1][0][RTW89_WW][37] = 42,
+ [0][1][1][0][RTW89_WW][38] = 16,
+ [0][1][1][0][RTW89_WW][40] = 16,
+ [0][1][1][0][RTW89_WW][42] = 16,
+ [0][1][1][0][RTW89_WW][44] = 16,
+ [0][1][1][0][RTW89_WW][46] = 16,
+ [0][0][2][0][RTW89_WW][0] = 30,
+ [0][0][2][0][RTW89_WW][2] = 30,
+ [0][0][2][0][RTW89_WW][4] = 30,
+ [0][0][2][0][RTW89_WW][6] = 30,
+ [0][0][2][0][RTW89_WW][8] = 52,
+ [0][0][2][0][RTW89_WW][10] = 52,
+ [0][0][2][0][RTW89_WW][12] = 52,
+ [0][0][2][0][RTW89_WW][14] = 52,
+ [0][0][2][0][RTW89_WW][15] = 52,
+ [0][0][2][0][RTW89_WW][17] = 52,
+ [0][0][2][0][RTW89_WW][19] = 52,
+ [0][0][2][0][RTW89_WW][21] = 52,
+ [0][0][2][0][RTW89_WW][23] = 52,
+ [0][0][2][0][RTW89_WW][25] = 52,
+ [0][0][2][0][RTW89_WW][27] = 52,
+ [0][0][2][0][RTW89_WW][29] = 52,
+ [0][0][2][0][RTW89_WW][31] = 52,
+ [0][0][2][0][RTW89_WW][33] = 52,
+ [0][0][2][0][RTW89_WW][35] = 52,
+ [0][0][2][0][RTW89_WW][37] = 54,
+ [0][0][2][0][RTW89_WW][38] = 28,
+ [0][0][2][0][RTW89_WW][40] = 28,
+ [0][0][2][0][RTW89_WW][42] = 28,
+ [0][0][2][0][RTW89_WW][44] = 28,
+ [0][0][2][0][RTW89_WW][46] = 28,
+ [0][1][2][0][RTW89_WW][0] = 18,
+ [0][1][2][0][RTW89_WW][2] = 18,
+ [0][1][2][0][RTW89_WW][4] = 18,
+ [0][1][2][0][RTW89_WW][6] = 18,
+ [0][1][2][0][RTW89_WW][8] = 40,
+ [0][1][2][0][RTW89_WW][10] = 40,
+ [0][1][2][0][RTW89_WW][12] = 40,
+ [0][1][2][0][RTW89_WW][14] = 40,
+ [0][1][2][0][RTW89_WW][15] = 40,
+ [0][1][2][0][RTW89_WW][17] = 40,
+ [0][1][2][0][RTW89_WW][19] = 40,
+ [0][1][2][0][RTW89_WW][21] = 40,
+ [0][1][2][0][RTW89_WW][23] = 40,
+ [0][1][2][0][RTW89_WW][25] = 40,
+ [0][1][2][0][RTW89_WW][27] = 40,
+ [0][1][2][0][RTW89_WW][29] = 40,
+ [0][1][2][0][RTW89_WW][31] = 40,
+ [0][1][2][0][RTW89_WW][33] = 40,
+ [0][1][2][0][RTW89_WW][35] = 40,
+ [0][1][2][0][RTW89_WW][37] = 42,
+ [0][1][2][0][RTW89_WW][38] = 16,
+ [0][1][2][0][RTW89_WW][40] = 16,
+ [0][1][2][0][RTW89_WW][42] = 16,
+ [0][1][2][0][RTW89_WW][44] = 16,
+ [0][1][2][0][RTW89_WW][46] = 16,
+ [0][1][2][1][RTW89_WW][0] = 6,
+ [0][1][2][1][RTW89_WW][2] = 6,
+ [0][1][2][1][RTW89_WW][4] = 6,
+ [0][1][2][1][RTW89_WW][6] = 6,
+ [0][1][2][1][RTW89_WW][8] = 28,
+ [0][1][2][1][RTW89_WW][10] = 28,
+ [0][1][2][1][RTW89_WW][12] = 28,
+ [0][1][2][1][RTW89_WW][14] = 28,
+ [0][1][2][1][RTW89_WW][15] = 28,
+ [0][1][2][1][RTW89_WW][17] = 28,
+ [0][1][2][1][RTW89_WW][19] = 28,
+ [0][1][2][1][RTW89_WW][21] = 28,
+ [0][1][2][1][RTW89_WW][23] = 28,
+ [0][1][2][1][RTW89_WW][25] = 28,
+ [0][1][2][1][RTW89_WW][27] = 28,
+ [0][1][2][1][RTW89_WW][29] = 28,
+ [0][1][2][1][RTW89_WW][31] = 28,
+ [0][1][2][1][RTW89_WW][33] = 28,
+ [0][1][2][1][RTW89_WW][35] = 28,
+ [0][1][2][1][RTW89_WW][37] = 30,
+ [0][1][2][1][RTW89_WW][38] = 4,
+ [0][1][2][1][RTW89_WW][40] = 4,
+ [0][1][2][1][RTW89_WW][42] = 4,
+ [0][1][2][1][RTW89_WW][44] = 4,
+ [0][1][2][1][RTW89_WW][46] = 4,
+ [1][0][2][0][RTW89_WW][1] = 30,
+ [1][0][2][0][RTW89_WW][5] = 30,
+ [1][0][2][0][RTW89_WW][9] = 52,
+ [1][0][2][0][RTW89_WW][13] = 52,
+ [1][0][2][0][RTW89_WW][16] = 52,
+ [1][0][2][0][RTW89_WW][20] = 52,
+ [1][0][2][0][RTW89_WW][24] = 52,
+ [1][0][2][0][RTW89_WW][28] = 52,
+ [1][0][2][0][RTW89_WW][32] = 52,
+ [1][0][2][0][RTW89_WW][36] = 54,
+ [1][0][2][0][RTW89_WW][39] = 28,
+ [1][0][2][0][RTW89_WW][43] = 28,
+ [1][1][2][0][RTW89_WW][1] = 18,
+ [1][1][2][0][RTW89_WW][5] = 18,
+ [1][1][2][0][RTW89_WW][9] = 40,
+ [1][1][2][0][RTW89_WW][13] = 40,
+ [1][1][2][0][RTW89_WW][16] = 40,
+ [1][1][2][0][RTW89_WW][20] = 40,
+ [1][1][2][0][RTW89_WW][24] = 40,
+ [1][1][2][0][RTW89_WW][28] = 40,
+ [1][1][2][0][RTW89_WW][32] = 40,
+ [1][1][2][0][RTW89_WW][36] = 42,
+ [1][1][2][0][RTW89_WW][39] = 16,
+ [1][1][2][0][RTW89_WW][43] = 16,
+ [1][1][2][1][RTW89_WW][1] = 6,
+ [1][1][2][1][RTW89_WW][5] = 6,
+ [1][1][2][1][RTW89_WW][9] = 28,
+ [1][1][2][1][RTW89_WW][13] = 28,
+ [1][1][2][1][RTW89_WW][16] = 28,
+ [1][1][2][1][RTW89_WW][20] = 28,
+ [1][1][2][1][RTW89_WW][24] = 28,
+ [1][1][2][1][RTW89_WW][28] = 28,
+ [1][1][2][1][RTW89_WW][32] = 28,
+ [1][1][2][1][RTW89_WW][36] = 30,
+ [1][1][2][1][RTW89_WW][39] = 4,
+ [1][1][2][1][RTW89_WW][43] = 4,
+ [2][0][2][0][RTW89_WW][3] = 30,
+ [2][0][2][0][RTW89_WW][11] = 52,
+ [2][0][2][0][RTW89_WW][18] = 52,
+ [2][0][2][0][RTW89_WW][26] = 52,
+ [2][0][2][0][RTW89_WW][34] = 54,
+ [2][0][2][0][RTW89_WW][41] = 28,
+ [2][1][2][0][RTW89_WW][3] = 18,
+ [2][1][2][0][RTW89_WW][11] = 40,
+ [2][1][2][0][RTW89_WW][18] = 40,
+ [2][1][2][0][RTW89_WW][26] = 40,
+ [2][1][2][0][RTW89_WW][34] = 42,
+ [2][1][2][0][RTW89_WW][41] = 16,
+ [2][1][2][1][RTW89_WW][3] = 6,
+ [2][1][2][1][RTW89_WW][11] = 28,
+ [2][1][2][1][RTW89_WW][18] = 28,
+ [2][1][2][1][RTW89_WW][26] = 28,
+ [2][1][2][1][RTW89_WW][34] = 30,
+ [2][1][2][1][RTW89_WW][41] = 4,
+ [0][0][1][0][RTW89_FCC][0] = 76,
+ [0][0][1][0][RTW89_ETSI][0] = 58,
+ [0][0][1][0][RTW89_MKK][0] = 62,
+ [0][0][1][0][RTW89_IC][0] = 62,
+ [0][0][1][0][RTW89_KCC][0] = 76,
+ [0][0][1][0][RTW89_ACMA][0] = 58,
+ [0][0][1][0][RTW89_CHILE][0] = 30,
+ [0][0][1][0][RTW89_UKRAINE][0] = 52,
+ [0][0][1][0][RTW89_MEXICO][0] = 62,
+ [0][0][1][0][RTW89_CN][0] = 58,
+ [0][0][1][0][RTW89_QATAR][0] = 58,
+ [0][0][1][0][RTW89_FCC][2] = 76,
+ [0][0][1][0][RTW89_ETSI][2] = 58,
+ [0][0][1][0][RTW89_MKK][2] = 62,
+ [0][0][1][0][RTW89_IC][2] = 62,
+ [0][0][1][0][RTW89_KCC][2] = 76,
+ [0][0][1][0][RTW89_ACMA][2] = 58,
+ [0][0][1][0][RTW89_CHILE][2] = 30,
+ [0][0][1][0][RTW89_UKRAINE][2] = 52,
+ [0][0][1][0][RTW89_MEXICO][2] = 62,
+ [0][0][1][0][RTW89_CN][2] = 58,
+ [0][0][1][0][RTW89_QATAR][2] = 58,
+ [0][0][1][0][RTW89_FCC][4] = 76,
+ [0][0][1][0][RTW89_ETSI][4] = 58,
+ [0][0][1][0][RTW89_MKK][4] = 62,
+ [0][0][1][0][RTW89_IC][4] = 62,
+ [0][0][1][0][RTW89_KCC][4] = 76,
+ [0][0][1][0][RTW89_ACMA][4] = 58,
+ [0][0][1][0][RTW89_CHILE][4] = 30,
+ [0][0][1][0][RTW89_UKRAINE][4] = 52,
+ [0][0][1][0][RTW89_MEXICO][4] = 62,
+ [0][0][1][0][RTW89_CN][4] = 58,
+ [0][0][1][0][RTW89_QATAR][4] = 58,
+ [0][0][1][0][RTW89_FCC][6] = 76,
+ [0][0][1][0][RTW89_ETSI][6] = 58,
+ [0][0][1][0][RTW89_MKK][6] = 62,
+ [0][0][1][0][RTW89_IC][6] = 62,
+ [0][0][1][0][RTW89_KCC][6] = 58,
+ [0][0][1][0][RTW89_ACMA][6] = 58,
+ [0][0][1][0][RTW89_CHILE][6] = 30,
+ [0][0][1][0][RTW89_UKRAINE][6] = 52,
+ [0][0][1][0][RTW89_MEXICO][6] = 62,
+ [0][0][1][0][RTW89_CN][6] = 58,
+ [0][0][1][0][RTW89_QATAR][6] = 58,
+ [0][0][1][0][RTW89_FCC][8] = 76,
+ [0][0][1][0][RTW89_ETSI][8] = 58,
+ [0][0][1][0][RTW89_MKK][8] = 62,
+ [0][0][1][0][RTW89_IC][8] = 64,
+ [0][0][1][0][RTW89_KCC][8] = 76,
+ [0][0][1][0][RTW89_ACMA][8] = 58,
+ [0][0][1][0][RTW89_CHILE][8] = 54,
+ [0][0][1][0][RTW89_UKRAINE][8] = 52,
+ [0][0][1][0][RTW89_MEXICO][8] = 76,
+ [0][0][1][0][RTW89_CN][8] = 58,
+ [0][0][1][0][RTW89_QATAR][8] = 58,
+ [0][0][1][0][RTW89_FCC][10] = 76,
+ [0][0][1][0][RTW89_ETSI][10] = 58,
+ [0][0][1][0][RTW89_MKK][10] = 62,
+ [0][0][1][0][RTW89_IC][10] = 64,
+ [0][0][1][0][RTW89_KCC][10] = 76,
+ [0][0][1][0][RTW89_ACMA][10] = 58,
+ [0][0][1][0][RTW89_CHILE][10] = 54,
+ [0][0][1][0][RTW89_UKRAINE][10] = 52,
+ [0][0][1][0][RTW89_MEXICO][10] = 76,
+ [0][0][1][0][RTW89_CN][10] = 58,
+ [0][0][1][0][RTW89_QATAR][10] = 58,
+ [0][0][1][0][RTW89_FCC][12] = 76,
+ [0][0][1][0][RTW89_ETSI][12] = 58,
+ [0][0][1][0][RTW89_MKK][12] = 62,
+ [0][0][1][0][RTW89_IC][12] = 64,
+ [0][0][1][0][RTW89_KCC][12] = 76,
+ [0][0][1][0][RTW89_ACMA][12] = 58,
+ [0][0][1][0][RTW89_CHILE][12] = 54,
+ [0][0][1][0][RTW89_UKRAINE][12] = 52,
+ [0][0][1][0][RTW89_MEXICO][12] = 76,
+ [0][0][1][0][RTW89_CN][12] = 58,
+ [0][0][1][0][RTW89_QATAR][12] = 58,
+ [0][0][1][0][RTW89_FCC][14] = 76,
+ [0][0][1][0][RTW89_ETSI][14] = 58,
+ [0][0][1][0][RTW89_MKK][14] = 62,
+ [0][0][1][0][RTW89_IC][14] = 64,
+ [0][0][1][0][RTW89_KCC][14] = 76,
+ [0][0][1][0][RTW89_ACMA][14] = 58,
+ [0][0][1][0][RTW89_CHILE][14] = 54,
+ [0][0][1][0][RTW89_UKRAINE][14] = 52,
+ [0][0][1][0][RTW89_MEXICO][14] = 76,
+ [0][0][1][0][RTW89_CN][14] = 58,
+ [0][0][1][0][RTW89_QATAR][14] = 58,
+ [0][0][1][0][RTW89_FCC][15] = 76,
+ [0][0][1][0][RTW89_ETSI][15] = 58,
+ [0][0][1][0][RTW89_MKK][15] = 76,
+ [0][0][1][0][RTW89_IC][15] = 76,
+ [0][0][1][0][RTW89_KCC][15] = 76,
+ [0][0][1][0][RTW89_ACMA][15] = 58,
+ [0][0][1][0][RTW89_CHILE][15] = 54,
+ [0][0][1][0][RTW89_UKRAINE][15] = 52,
+ [0][0][1][0][RTW89_MEXICO][15] = 76,
+ [0][0][1][0][RTW89_CN][15] = 127,
+ [0][0][1][0][RTW89_QATAR][15] = 52,
+ [0][0][1][0][RTW89_FCC][17] = 76,
+ [0][0][1][0][RTW89_ETSI][17] = 58,
+ [0][0][1][0][RTW89_MKK][17] = 76,
+ [0][0][1][0][RTW89_IC][17] = 76,
+ [0][0][1][0][RTW89_KCC][17] = 76,
+ [0][0][1][0][RTW89_ACMA][17] = 58,
+ [0][0][1][0][RTW89_CHILE][17] = 54,
+ [0][0][1][0][RTW89_UKRAINE][17] = 52,
+ [0][0][1][0][RTW89_MEXICO][17] = 76,
+ [0][0][1][0][RTW89_CN][17] = 127,
+ [0][0][1][0][RTW89_QATAR][17] = 52,
+ [0][0][1][0][RTW89_FCC][19] = 76,
+ [0][0][1][0][RTW89_ETSI][19] = 58,
+ [0][0][1][0][RTW89_MKK][19] = 76,
+ [0][0][1][0][RTW89_IC][19] = 76,
+ [0][0][1][0][RTW89_KCC][19] = 76,
+ [0][0][1][0][RTW89_ACMA][19] = 58,
+ [0][0][1][0][RTW89_CHILE][19] = 54,
+ [0][0][1][0][RTW89_UKRAINE][19] = 52,
+ [0][0][1][0][RTW89_MEXICO][19] = 76,
+ [0][0][1][0][RTW89_CN][19] = 127,
+ [0][0][1][0][RTW89_QATAR][19] = 52,
+ [0][0][1][0][RTW89_FCC][21] = 76,
+ [0][0][1][0][RTW89_ETSI][21] = 58,
+ [0][0][1][0][RTW89_MKK][21] = 76,
+ [0][0][1][0][RTW89_IC][21] = 76,
+ [0][0][1][0][RTW89_KCC][21] = 76,
+ [0][0][1][0][RTW89_ACMA][21] = 58,
+ [0][0][1][0][RTW89_CHILE][21] = 54,
+ [0][0][1][0][RTW89_UKRAINE][21] = 52,
+ [0][0][1][0][RTW89_MEXICO][21] = 76,
+ [0][0][1][0][RTW89_CN][21] = 127,
+ [0][0][1][0][RTW89_QATAR][21] = 52,
+ [0][0][1][0][RTW89_FCC][23] = 76,
+ [0][0][1][0][RTW89_ETSI][23] = 58,
+ [0][0][1][0][RTW89_MKK][23] = 76,
+ [0][0][1][0][RTW89_IC][23] = 76,
+ [0][0][1][0][RTW89_KCC][23] = 76,
+ [0][0][1][0][RTW89_ACMA][23] = 58,
+ [0][0][1][0][RTW89_CHILE][23] = 54,
+ [0][0][1][0][RTW89_UKRAINE][23] = 52,
+ [0][0][1][0][RTW89_MEXICO][23] = 76,
+ [0][0][1][0][RTW89_CN][23] = 127,
+ [0][0][1][0][RTW89_QATAR][23] = 52,
+ [0][0][1][0][RTW89_FCC][25] = 76,
+ [0][0][1][0][RTW89_ETSI][25] = 58,
+ [0][0][1][0][RTW89_MKK][25] = 76,
+ [0][0][1][0][RTW89_IC][25] = 127,
+ [0][0][1][0][RTW89_KCC][25] = 76,
+ [0][0][1][0][RTW89_ACMA][25] = 127,
+ [0][0][1][0][RTW89_CHILE][25] = 54,
+ [0][0][1][0][RTW89_UKRAINE][25] = 52,
+ [0][0][1][0][RTW89_MEXICO][25] = 76,
+ [0][0][1][0][RTW89_CN][25] = 127,
+ [0][0][1][0][RTW89_QATAR][25] = 52,
+ [0][0][1][0][RTW89_FCC][27] = 76,
+ [0][0][1][0][RTW89_ETSI][27] = 58,
+ [0][0][1][0][RTW89_MKK][27] = 76,
+ [0][0][1][0][RTW89_IC][27] = 127,
+ [0][0][1][0][RTW89_KCC][27] = 76,
+ [0][0][1][0][RTW89_ACMA][27] = 127,
+ [0][0][1][0][RTW89_CHILE][27] = 54,
+ [0][0][1][0][RTW89_UKRAINE][27] = 52,
+ [0][0][1][0][RTW89_MEXICO][27] = 76,
+ [0][0][1][0][RTW89_CN][27] = 127,
+ [0][0][1][0][RTW89_QATAR][27] = 52,
+ [0][0][1][0][RTW89_FCC][29] = 76,
+ [0][0][1][0][RTW89_ETSI][29] = 58,
+ [0][0][1][0][RTW89_MKK][29] = 76,
+ [0][0][1][0][RTW89_IC][29] = 127,
+ [0][0][1][0][RTW89_KCC][29] = 76,
+ [0][0][1][0][RTW89_ACMA][29] = 127,
+ [0][0][1][0][RTW89_CHILE][29] = 54,
+ [0][0][1][0][RTW89_UKRAINE][29] = 52,
+ [0][0][1][0][RTW89_MEXICO][29] = 76,
+ [0][0][1][0][RTW89_CN][29] = 127,
+ [0][0][1][0][RTW89_QATAR][29] = 52,
+ [0][0][1][0][RTW89_FCC][31] = 76,
+ [0][0][1][0][RTW89_ETSI][31] = 58,
+ [0][0][1][0][RTW89_MKK][31] = 76,
+ [0][0][1][0][RTW89_IC][31] = 76,
+ [0][0][1][0][RTW89_KCC][31] = 76,
+ [0][0][1][0][RTW89_ACMA][31] = 58,
+ [0][0][1][0][RTW89_CHILE][31] = 54,
+ [0][0][1][0][RTW89_UKRAINE][31] = 52,
+ [0][0][1][0][RTW89_MEXICO][31] = 76,
+ [0][0][1][0][RTW89_CN][31] = 127,
+ [0][0][1][0][RTW89_QATAR][31] = 52,
+ [0][0][1][0][RTW89_FCC][33] = 76,
+ [0][0][1][0][RTW89_ETSI][33] = 58,
+ [0][0][1][0][RTW89_MKK][33] = 76,
+ [0][0][1][0][RTW89_IC][33] = 76,
+ [0][0][1][0][RTW89_KCC][33] = 76,
+ [0][0][1][0][RTW89_ACMA][33] = 58,
+ [0][0][1][0][RTW89_CHILE][33] = 54,
+ [0][0][1][0][RTW89_UKRAINE][33] = 52,
+ [0][0][1][0][RTW89_MEXICO][33] = 76,
+ [0][0][1][0][RTW89_CN][33] = 127,
+ [0][0][1][0][RTW89_QATAR][33] = 52,
+ [0][0][1][0][RTW89_FCC][35] = 74,
+ [0][0][1][0][RTW89_ETSI][35] = 58,
+ [0][0][1][0][RTW89_MKK][35] = 76,
+ [0][0][1][0][RTW89_IC][35] = 74,
+ [0][0][1][0][RTW89_KCC][35] = 76,
+ [0][0][1][0][RTW89_ACMA][35] = 58,
+ [0][0][1][0][RTW89_CHILE][35] = 54,
+ [0][0][1][0][RTW89_UKRAINE][35] = 52,
+ [0][0][1][0][RTW89_MEXICO][35] = 74,
+ [0][0][1][0][RTW89_CN][35] = 127,
+ [0][0][1][0][RTW89_QATAR][35] = 52,
+ [0][0][1][0][RTW89_FCC][37] = 76,
+ [0][0][1][0][RTW89_ETSI][37] = 127,
+ [0][0][1][0][RTW89_MKK][37] = 76,
+ [0][0][1][0][RTW89_IC][37] = 76,
+ [0][0][1][0][RTW89_KCC][37] = 76,
+ [0][0][1][0][RTW89_ACMA][37] = 76,
+ [0][0][1][0][RTW89_CHILE][37] = 54,
+ [0][0][1][0][RTW89_UKRAINE][37] = 127,
+ [0][0][1][0][RTW89_MEXICO][37] = 76,
+ [0][0][1][0][RTW89_CN][37] = 127,
+ [0][0][1][0][RTW89_QATAR][37] = 127,
+ [0][0][1][0][RTW89_FCC][38] = 76,
+ [0][0][1][0][RTW89_ETSI][38] = 28,
+ [0][0][1][0][RTW89_MKK][38] = 127,
+ [0][0][1][0][RTW89_IC][38] = 76,
+ [0][0][1][0][RTW89_KCC][38] = 76,
+ [0][0][1][0][RTW89_ACMA][38] = 76,
+ [0][0][1][0][RTW89_CHILE][38] = 54,
+ [0][0][1][0][RTW89_UKRAINE][38] = 28,
+ [0][0][1][0][RTW89_MEXICO][38] = 76,
+ [0][0][1][0][RTW89_CN][38] = 72,
+ [0][0][1][0][RTW89_QATAR][38] = 28,
+ [0][0][1][0][RTW89_FCC][40] = 76,
+ [0][0][1][0][RTW89_ETSI][40] = 28,
+ [0][0][1][0][RTW89_MKK][40] = 127,
+ [0][0][1][0][RTW89_IC][40] = 76,
+ [0][0][1][0][RTW89_KCC][40] = 76,
+ [0][0][1][0][RTW89_ACMA][40] = 76,
+ [0][0][1][0][RTW89_CHILE][40] = 54,
+ [0][0][1][0][RTW89_UKRAINE][40] = 28,
+ [0][0][1][0][RTW89_MEXICO][40] = 76,
+ [0][0][1][0][RTW89_CN][40] = 76,
+ [0][0][1][0][RTW89_QATAR][40] = 28,
+ [0][0][1][0][RTW89_FCC][42] = 76,
+ [0][0][1][0][RTW89_ETSI][42] = 28,
+ [0][0][1][0][RTW89_MKK][42] = 127,
+ [0][0][1][0][RTW89_IC][42] = 76,
+ [0][0][1][0][RTW89_KCC][42] = 76,
+ [0][0][1][0][RTW89_ACMA][42] = 76,
+ [0][0][1][0][RTW89_CHILE][42] = 54,
+ [0][0][1][0][RTW89_UKRAINE][42] = 28,
+ [0][0][1][0][RTW89_MEXICO][42] = 76,
+ [0][0][1][0][RTW89_CN][42] = 76,
+ [0][0][1][0][RTW89_QATAR][42] = 28,
+ [0][0][1][0][RTW89_FCC][44] = 76,
+ [0][0][1][0][RTW89_ETSI][44] = 28,
+ [0][0][1][0][RTW89_MKK][44] = 127,
+ [0][0][1][0][RTW89_IC][44] = 76,
+ [0][0][1][0][RTW89_KCC][44] = 76,
+ [0][0][1][0][RTW89_ACMA][44] = 76,
+ [0][0][1][0][RTW89_CHILE][44] = 54,
+ [0][0][1][0][RTW89_UKRAINE][44] = 28,
+ [0][0][1][0][RTW89_MEXICO][44] = 76,
+ [0][0][1][0][RTW89_CN][44] = 76,
+ [0][0][1][0][RTW89_QATAR][44] = 28,
+ [0][0][1][0][RTW89_FCC][46] = 76,
+ [0][0][1][0][RTW89_ETSI][46] = 28,
+ [0][0][1][0][RTW89_MKK][46] = 127,
+ [0][0][1][0][RTW89_IC][46] = 76,
+ [0][0][1][0][RTW89_KCC][46] = 76,
+ [0][0][1][0][RTW89_ACMA][46] = 76,
+ [0][0][1][0][RTW89_CHILE][46] = 54,
+ [0][0][1][0][RTW89_UKRAINE][46] = 28,
+ [0][0][1][0][RTW89_MEXICO][46] = 76,
+ [0][0][1][0][RTW89_CN][46] = 76,
+ [0][0][1][0][RTW89_QATAR][46] = 28,
+ [0][1][1][0][RTW89_FCC][0] = 68,
+ [0][1][1][0][RTW89_ETSI][0] = 46,
+ [0][1][1][0][RTW89_MKK][0] = 50,
+ [0][1][1][0][RTW89_IC][0] = 40,
+ [0][1][1][0][RTW89_KCC][0] = 72,
+ [0][1][1][0][RTW89_ACMA][0] = 46,
+ [0][1][1][0][RTW89_CHILE][0] = 18,
+ [0][1][1][0][RTW89_UKRAINE][0] = 40,
+ [0][1][1][0][RTW89_MEXICO][0] = 50,
+ [0][1][1][0][RTW89_CN][0] = 46,
+ [0][1][1][0][RTW89_QATAR][0] = 46,
+ [0][1][1][0][RTW89_FCC][2] = 68,
+ [0][1][1][0][RTW89_ETSI][2] = 46,
+ [0][1][1][0][RTW89_MKK][2] = 50,
+ [0][1][1][0][RTW89_IC][2] = 40,
+ [0][1][1][0][RTW89_KCC][2] = 72,
+ [0][1][1][0][RTW89_ACMA][2] = 46,
+ [0][1][1][0][RTW89_CHILE][2] = 18,
+ [0][1][1][0][RTW89_UKRAINE][2] = 40,
+ [0][1][1][0][RTW89_MEXICO][2] = 50,
+ [0][1][1][0][RTW89_CN][2] = 46,
+ [0][1][1][0][RTW89_QATAR][2] = 46,
+ [0][1][1][0][RTW89_FCC][4] = 68,
+ [0][1][1][0][RTW89_ETSI][4] = 46,
+ [0][1][1][0][RTW89_MKK][4] = 50,
+ [0][1][1][0][RTW89_IC][4] = 40,
+ [0][1][1][0][RTW89_KCC][4] = 72,
+ [0][1][1][0][RTW89_ACMA][4] = 46,
+ [0][1][1][0][RTW89_CHILE][4] = 18,
+ [0][1][1][0][RTW89_UKRAINE][4] = 40,
+ [0][1][1][0][RTW89_MEXICO][4] = 50,
+ [0][1][1][0][RTW89_CN][4] = 46,
+ [0][1][1][0][RTW89_QATAR][4] = 46,
+ [0][1][1][0][RTW89_FCC][6] = 68,
+ [0][1][1][0][RTW89_ETSI][6] = 46,
+ [0][1][1][0][RTW89_MKK][6] = 50,
+ [0][1][1][0][RTW89_IC][6] = 40,
+ [0][1][1][0][RTW89_KCC][6] = 44,
+ [0][1][1][0][RTW89_ACMA][6] = 46,
+ [0][1][1][0][RTW89_CHILE][6] = 18,
+ [0][1][1][0][RTW89_UKRAINE][6] = 40,
+ [0][1][1][0][RTW89_MEXICO][6] = 50,
+ [0][1][1][0][RTW89_CN][6] = 46,
+ [0][1][1][0][RTW89_QATAR][6] = 46,
+ [0][1][1][0][RTW89_FCC][8] = 68,
+ [0][1][1][0][RTW89_ETSI][8] = 46,
+ [0][1][1][0][RTW89_MKK][8] = 50,
+ [0][1][1][0][RTW89_IC][8] = 52,
+ [0][1][1][0][RTW89_KCC][8] = 72,
+ [0][1][1][0][RTW89_ACMA][8] = 46,
+ [0][1][1][0][RTW89_CHILE][8] = 42,
+ [0][1][1][0][RTW89_UKRAINE][8] = 40,
+ [0][1][1][0][RTW89_MEXICO][8] = 68,
+ [0][1][1][0][RTW89_CN][8] = 46,
+ [0][1][1][0][RTW89_QATAR][8] = 46,
+ [0][1][1][0][RTW89_FCC][10] = 68,
+ [0][1][1][0][RTW89_ETSI][10] = 46,
+ [0][1][1][0][RTW89_MKK][10] = 50,
+ [0][1][1][0][RTW89_IC][10] = 52,
+ [0][1][1][0][RTW89_KCC][10] = 72,
+ [0][1][1][0][RTW89_ACMA][10] = 46,
+ [0][1][1][0][RTW89_CHILE][10] = 42,
+ [0][1][1][0][RTW89_UKRAINE][10] = 40,
+ [0][1][1][0][RTW89_MEXICO][10] = 68,
+ [0][1][1][0][RTW89_CN][10] = 46,
+ [0][1][1][0][RTW89_QATAR][10] = 46,
+ [0][1][1][0][RTW89_FCC][12] = 68,
+ [0][1][1][0][RTW89_ETSI][12] = 46,
+ [0][1][1][0][RTW89_MKK][12] = 50,
+ [0][1][1][0][RTW89_IC][12] = 52,
+ [0][1][1][0][RTW89_KCC][12] = 72,
+ [0][1][1][0][RTW89_ACMA][12] = 46,
+ [0][1][1][0][RTW89_CHILE][12] = 42,
+ [0][1][1][0][RTW89_UKRAINE][12] = 40,
+ [0][1][1][0][RTW89_MEXICO][12] = 68,
+ [0][1][1][0][RTW89_CN][12] = 46,
+ [0][1][1][0][RTW89_QATAR][12] = 46,
+ [0][1][1][0][RTW89_FCC][14] = 68,
+ [0][1][1][0][RTW89_ETSI][14] = 46,
+ [0][1][1][0][RTW89_MKK][14] = 50,
+ [0][1][1][0][RTW89_IC][14] = 52,
+ [0][1][1][0][RTW89_KCC][14] = 72,
+ [0][1][1][0][RTW89_ACMA][14] = 46,
+ [0][1][1][0][RTW89_CHILE][14] = 42,
+ [0][1][1][0][RTW89_UKRAINE][14] = 40,
+ [0][1][1][0][RTW89_MEXICO][14] = 68,
+ [0][1][1][0][RTW89_CN][14] = 46,
+ [0][1][1][0][RTW89_QATAR][14] = 46,
+ [0][1][1][0][RTW89_FCC][15] = 68,
+ [0][1][1][0][RTW89_ETSI][15] = 46,
+ [0][1][1][0][RTW89_MKK][15] = 70,
+ [0][1][1][0][RTW89_IC][15] = 68,
+ [0][1][1][0][RTW89_KCC][15] = 72,
+ [0][1][1][0][RTW89_ACMA][15] = 46,
+ [0][1][1][0][RTW89_CHILE][15] = 42,
+ [0][1][1][0][RTW89_UKRAINE][15] = 40,
+ [0][1][1][0][RTW89_MEXICO][15] = 68,
+ [0][1][1][0][RTW89_CN][15] = 127,
+ [0][1][1][0][RTW89_QATAR][15] = 40,
+ [0][1][1][0][RTW89_FCC][17] = 68,
+ [0][1][1][0][RTW89_ETSI][17] = 46,
+ [0][1][1][0][RTW89_MKK][17] = 70,
+ [0][1][1][0][RTW89_IC][17] = 68,
+ [0][1][1][0][RTW89_KCC][17] = 72,
+ [0][1][1][0][RTW89_ACMA][17] = 46,
+ [0][1][1][0][RTW89_CHILE][17] = 42,
+ [0][1][1][0][RTW89_UKRAINE][17] = 40,
+ [0][1][1][0][RTW89_MEXICO][17] = 68,
+ [0][1][1][0][RTW89_CN][17] = 127,
+ [0][1][1][0][RTW89_QATAR][17] = 40,
+ [0][1][1][0][RTW89_FCC][19] = 68,
+ [0][1][1][0][RTW89_ETSI][19] = 46,
+ [0][1][1][0][RTW89_MKK][19] = 70,
+ [0][1][1][0][RTW89_IC][19] = 68,
+ [0][1][1][0][RTW89_KCC][19] = 72,
+ [0][1][1][0][RTW89_ACMA][19] = 46,
+ [0][1][1][0][RTW89_CHILE][19] = 42,
+ [0][1][1][0][RTW89_UKRAINE][19] = 40,
+ [0][1][1][0][RTW89_MEXICO][19] = 68,
+ [0][1][1][0][RTW89_CN][19] = 127,
+ [0][1][1][0][RTW89_QATAR][19] = 40,
+ [0][1][1][0][RTW89_FCC][21] = 68,
+ [0][1][1][0][RTW89_ETSI][21] = 46,
+ [0][1][1][0][RTW89_MKK][21] = 70,
+ [0][1][1][0][RTW89_IC][21] = 68,
+ [0][1][1][0][RTW89_KCC][21] = 72,
+ [0][1][1][0][RTW89_ACMA][21] = 46,
+ [0][1][1][0][RTW89_CHILE][21] = 42,
+ [0][1][1][0][RTW89_UKRAINE][21] = 40,
+ [0][1][1][0][RTW89_MEXICO][21] = 68,
+ [0][1][1][0][RTW89_CN][21] = 127,
+ [0][1][1][0][RTW89_QATAR][21] = 40,
+ [0][1][1][0][RTW89_FCC][23] = 68,
+ [0][1][1][0][RTW89_ETSI][23] = 46,
+ [0][1][1][0][RTW89_MKK][23] = 70,
+ [0][1][1][0][RTW89_IC][23] = 68,
+ [0][1][1][0][RTW89_KCC][23] = 72,
+ [0][1][1][0][RTW89_ACMA][23] = 46,
+ [0][1][1][0][RTW89_CHILE][23] = 42,
+ [0][1][1][0][RTW89_UKRAINE][23] = 40,
+ [0][1][1][0][RTW89_MEXICO][23] = 68,
+ [0][1][1][0][RTW89_CN][23] = 127,
+ [0][1][1][0][RTW89_QATAR][23] = 40,
+ [0][1][1][0][RTW89_FCC][25] = 68,
+ [0][1][1][0][RTW89_ETSI][25] = 46,
+ [0][1][1][0][RTW89_MKK][25] = 70,
+ [0][1][1][0][RTW89_IC][25] = 127,
+ [0][1][1][0][RTW89_KCC][25] = 72,
+ [0][1][1][0][RTW89_ACMA][25] = 127,
+ [0][1][1][0][RTW89_CHILE][25] = 42,
+ [0][1][1][0][RTW89_UKRAINE][25] = 40,
+ [0][1][1][0][RTW89_MEXICO][25] = 68,
+ [0][1][1][0][RTW89_CN][25] = 127,
+ [0][1][1][0][RTW89_QATAR][25] = 40,
+ [0][1][1][0][RTW89_FCC][27] = 68,
+ [0][1][1][0][RTW89_ETSI][27] = 46,
+ [0][1][1][0][RTW89_MKK][27] = 70,
+ [0][1][1][0][RTW89_IC][27] = 127,
+ [0][1][1][0][RTW89_KCC][27] = 72,
+ [0][1][1][0][RTW89_ACMA][27] = 127,
+ [0][1][1][0][RTW89_CHILE][27] = 42,
+ [0][1][1][0][RTW89_UKRAINE][27] = 40,
+ [0][1][1][0][RTW89_MEXICO][27] = 68,
+ [0][1][1][0][RTW89_CN][27] = 127,
+ [0][1][1][0][RTW89_QATAR][27] = 40,
+ [0][1][1][0][RTW89_FCC][29] = 68,
+ [0][1][1][0][RTW89_ETSI][29] = 46,
+ [0][1][1][0][RTW89_MKK][29] = 70,
+ [0][1][1][0][RTW89_IC][29] = 127,
+ [0][1][1][0][RTW89_KCC][29] = 72,
+ [0][1][1][0][RTW89_ACMA][29] = 127,
+ [0][1][1][0][RTW89_CHILE][29] = 42,
+ [0][1][1][0][RTW89_UKRAINE][29] = 40,
+ [0][1][1][0][RTW89_MEXICO][29] = 68,
+ [0][1][1][0][RTW89_CN][29] = 127,
+ [0][1][1][0][RTW89_QATAR][29] = 40,
+ [0][1][1][0][RTW89_FCC][31] = 68,
+ [0][1][1][0][RTW89_ETSI][31] = 46,
+ [0][1][1][0][RTW89_MKK][31] = 70,
+ [0][1][1][0][RTW89_IC][31] = 68,
+ [0][1][1][0][RTW89_KCC][31] = 72,
+ [0][1][1][0][RTW89_ACMA][31] = 46,
+ [0][1][1][0][RTW89_CHILE][31] = 42,
+ [0][1][1][0][RTW89_UKRAINE][31] = 40,
+ [0][1][1][0][RTW89_MEXICO][31] = 68,
+ [0][1][1][0][RTW89_CN][31] = 127,
+ [0][1][1][0][RTW89_QATAR][31] = 40,
+ [0][1][1][0][RTW89_FCC][33] = 68,
+ [0][1][1][0][RTW89_ETSI][33] = 46,
+ [0][1][1][0][RTW89_MKK][33] = 70,
+ [0][1][1][0][RTW89_IC][33] = 68,
+ [0][1][1][0][RTW89_KCC][33] = 72,
+ [0][1][1][0][RTW89_ACMA][33] = 46,
+ [0][1][1][0][RTW89_CHILE][33] = 42,
+ [0][1][1][0][RTW89_UKRAINE][33] = 40,
+ [0][1][1][0][RTW89_MEXICO][33] = 68,
+ [0][1][1][0][RTW89_CN][33] = 127,
+ [0][1][1][0][RTW89_QATAR][33] = 40,
+ [0][1][1][0][RTW89_FCC][35] = 66,
+ [0][1][1][0][RTW89_ETSI][35] = 46,
+ [0][1][1][0][RTW89_MKK][35] = 70,
+ [0][1][1][0][RTW89_IC][35] = 66,
+ [0][1][1][0][RTW89_KCC][35] = 72,
+ [0][1][1][0][RTW89_ACMA][35] = 46,
+ [0][1][1][0][RTW89_CHILE][35] = 42,
+ [0][1][1][0][RTW89_UKRAINE][35] = 40,
+ [0][1][1][0][RTW89_MEXICO][35] = 66,
+ [0][1][1][0][RTW89_CN][35] = 127,
+ [0][1][1][0][RTW89_QATAR][35] = 40,
+ [0][1][1][0][RTW89_FCC][37] = 68,
+ [0][1][1][0][RTW89_ETSI][37] = 127,
+ [0][1][1][0][RTW89_MKK][37] = 70,
+ [0][1][1][0][RTW89_IC][37] = 68,
+ [0][1][1][0][RTW89_KCC][37] = 72,
+ [0][1][1][0][RTW89_ACMA][37] = 68,
+ [0][1][1][0][RTW89_CHILE][37] = 42,
+ [0][1][1][0][RTW89_UKRAINE][37] = 127,
+ [0][1][1][0][RTW89_MEXICO][37] = 68,
+ [0][1][1][0][RTW89_CN][37] = 127,
+ [0][1][1][0][RTW89_QATAR][37] = 127,
+ [0][1][1][0][RTW89_FCC][38] = 76,
+ [0][1][1][0][RTW89_ETSI][38] = 16,
+ [0][1][1][0][RTW89_MKK][38] = 127,
+ [0][1][1][0][RTW89_IC][38] = 76,
+ [0][1][1][0][RTW89_KCC][38] = 72,
+ [0][1][1][0][RTW89_ACMA][38] = 76,
+ [0][1][1][0][RTW89_CHILE][38] = 42,
+ [0][1][1][0][RTW89_UKRAINE][38] = 16,
+ [0][1][1][0][RTW89_MEXICO][38] = 76,
+ [0][1][1][0][RTW89_CN][38] = 72,
+ [0][1][1][0][RTW89_QATAR][38] = 16,
+ [0][1][1][0][RTW89_FCC][40] = 76,
+ [0][1][1][0][RTW89_ETSI][40] = 16,
+ [0][1][1][0][RTW89_MKK][40] = 127,
+ [0][1][1][0][RTW89_IC][40] = 76,
+ [0][1][1][0][RTW89_KCC][40] = 72,
+ [0][1][1][0][RTW89_ACMA][40] = 76,
+ [0][1][1][0][RTW89_CHILE][40] = 42,
+ [0][1][1][0][RTW89_UKRAINE][40] = 16,
+ [0][1][1][0][RTW89_MEXICO][40] = 76,
+ [0][1][1][0][RTW89_CN][40] = 76,
+ [0][1][1][0][RTW89_QATAR][40] = 16,
+ [0][1][1][0][RTW89_FCC][42] = 76,
+ [0][1][1][0][RTW89_ETSI][42] = 16,
+ [0][1][1][0][RTW89_MKK][42] = 127,
+ [0][1][1][0][RTW89_IC][42] = 76,
+ [0][1][1][0][RTW89_KCC][42] = 72,
+ [0][1][1][0][RTW89_ACMA][42] = 76,
+ [0][1][1][0][RTW89_CHILE][42] = 42,
+ [0][1][1][0][RTW89_UKRAINE][42] = 16,
+ [0][1][1][0][RTW89_MEXICO][42] = 76,
+ [0][1][1][0][RTW89_CN][42] = 76,
+ [0][1][1][0][RTW89_QATAR][42] = 16,
+ [0][1][1][0][RTW89_FCC][44] = 76,
+ [0][1][1][0][RTW89_ETSI][44] = 16,
+ [0][1][1][0][RTW89_MKK][44] = 127,
+ [0][1][1][0][RTW89_IC][44] = 76,
+ [0][1][1][0][RTW89_KCC][44] = 72,
+ [0][1][1][0][RTW89_ACMA][44] = 76,
+ [0][1][1][0][RTW89_CHILE][44] = 42,
+ [0][1][1][0][RTW89_UKRAINE][44] = 16,
+ [0][1][1][0][RTW89_MEXICO][44] = 76,
+ [0][1][1][0][RTW89_CN][44] = 76,
+ [0][1][1][0][RTW89_QATAR][44] = 16,
+ [0][1][1][0][RTW89_FCC][46] = 76,
+ [0][1][1][0][RTW89_ETSI][46] = 16,
+ [0][1][1][0][RTW89_MKK][46] = 127,
+ [0][1][1][0][RTW89_IC][46] = 76,
+ [0][1][1][0][RTW89_KCC][46] = 72,
+ [0][1][1][0][RTW89_ACMA][46] = 76,
+ [0][1][1][0][RTW89_CHILE][46] = 42,
+ [0][1][1][0][RTW89_UKRAINE][46] = 16,
+ [0][1][1][0][RTW89_MEXICO][46] = 76,
+ [0][1][1][0][RTW89_CN][46] = 76,
+ [0][1][1][0][RTW89_QATAR][46] = 16,
+ [0][0][2][0][RTW89_FCC][0] = 76,
+ [0][0][2][0][RTW89_ETSI][0] = 58,
+ [0][0][2][0][RTW89_MKK][0] = 62,
+ [0][0][2][0][RTW89_IC][0] = 62,
+ [0][0][2][0][RTW89_KCC][0] = 76,
+ [0][0][2][0][RTW89_ACMA][0] = 58,
+ [0][0][2][0][RTW89_CHILE][0] = 30,
+ [0][0][2][0][RTW89_UKRAINE][0] = 52,
+ [0][0][2][0][RTW89_MEXICO][0] = 62,
+ [0][0][2][0][RTW89_CN][0] = 58,
+ [0][0][2][0][RTW89_QATAR][0] = 58,
+ [0][0][2][0][RTW89_FCC][2] = 76,
+ [0][0][2][0][RTW89_ETSI][2] = 58,
+ [0][0][2][0][RTW89_MKK][2] = 62,
+ [0][0][2][0][RTW89_IC][2] = 62,
+ [0][0][2][0][RTW89_KCC][2] = 76,
+ [0][0][2][0][RTW89_ACMA][2] = 58,
+ [0][0][2][0][RTW89_CHILE][2] = 30,
+ [0][0][2][0][RTW89_UKRAINE][2] = 52,
+ [0][0][2][0][RTW89_MEXICO][2] = 62,
+ [0][0][2][0][RTW89_CN][2] = 58,
+ [0][0][2][0][RTW89_QATAR][2] = 58,
+ [0][0][2][0][RTW89_FCC][4] = 76,
+ [0][0][2][0][RTW89_ETSI][4] = 58,
+ [0][0][2][0][RTW89_MKK][4] = 62,
+ [0][0][2][0][RTW89_IC][4] = 62,
+ [0][0][2][0][RTW89_KCC][4] = 76,
+ [0][0][2][0][RTW89_ACMA][4] = 58,
+ [0][0][2][0][RTW89_CHILE][4] = 30,
+ [0][0][2][0][RTW89_UKRAINE][4] = 52,
+ [0][0][2][0][RTW89_MEXICO][4] = 62,
+ [0][0][2][0][RTW89_CN][4] = 58,
+ [0][0][2][0][RTW89_QATAR][4] = 58,
+ [0][0][2][0][RTW89_FCC][6] = 76,
+ [0][0][2][0][RTW89_ETSI][6] = 58,
+ [0][0][2][0][RTW89_MKK][6] = 62,
+ [0][0][2][0][RTW89_IC][6] = 62,
+ [0][0][2][0][RTW89_KCC][6] = 58,
+ [0][0][2][0][RTW89_ACMA][6] = 58,
+ [0][0][2][0][RTW89_CHILE][6] = 30,
+ [0][0][2][0][RTW89_UKRAINE][6] = 52,
+ [0][0][2][0][RTW89_MEXICO][6] = 62,
+ [0][0][2][0][RTW89_CN][6] = 58,
+ [0][0][2][0][RTW89_QATAR][6] = 58,
+ [0][0][2][0][RTW89_FCC][8] = 76,
+ [0][0][2][0][RTW89_ETSI][8] = 58,
+ [0][0][2][0][RTW89_MKK][8] = 62,
+ [0][0][2][0][RTW89_IC][8] = 64,
+ [0][0][2][0][RTW89_KCC][8] = 76,
+ [0][0][2][0][RTW89_ACMA][8] = 58,
+ [0][0][2][0][RTW89_CHILE][8] = 54,
+ [0][0][2][0][RTW89_UKRAINE][8] = 52,
+ [0][0][2][0][RTW89_MEXICO][8] = 76,
+ [0][0][2][0][RTW89_CN][8] = 58,
+ [0][0][2][0][RTW89_QATAR][8] = 58,
+ [0][0][2][0][RTW89_FCC][10] = 76,
+ [0][0][2][0][RTW89_ETSI][10] = 58,
+ [0][0][2][0][RTW89_MKK][10] = 62,
+ [0][0][2][0][RTW89_IC][10] = 64,
+ [0][0][2][0][RTW89_KCC][10] = 76,
+ [0][0][2][0][RTW89_ACMA][10] = 58,
+ [0][0][2][0][RTW89_CHILE][10] = 54,
+ [0][0][2][0][RTW89_UKRAINE][10] = 52,
+ [0][0][2][0][RTW89_MEXICO][10] = 76,
+ [0][0][2][0][RTW89_CN][10] = 58,
+ [0][0][2][0][RTW89_QATAR][10] = 58,
+ [0][0][2][0][RTW89_FCC][12] = 76,
+ [0][0][2][0][RTW89_ETSI][12] = 58,
+ [0][0][2][0][RTW89_MKK][12] = 62,
+ [0][0][2][0][RTW89_IC][12] = 64,
+ [0][0][2][0][RTW89_KCC][12] = 76,
+ [0][0][2][0][RTW89_ACMA][12] = 58,
+ [0][0][2][0][RTW89_CHILE][12] = 54,
+ [0][0][2][0][RTW89_UKRAINE][12] = 52,
+ [0][0][2][0][RTW89_MEXICO][12] = 76,
+ [0][0][2][0][RTW89_CN][12] = 58,
+ [0][0][2][0][RTW89_QATAR][12] = 58,
+ [0][0][2][0][RTW89_FCC][14] = 76,
+ [0][0][2][0][RTW89_ETSI][14] = 58,
+ [0][0][2][0][RTW89_MKK][14] = 62,
+ [0][0][2][0][RTW89_IC][14] = 64,
+ [0][0][2][0][RTW89_KCC][14] = 76,
+ [0][0][2][0][RTW89_ACMA][14] = 58,
+ [0][0][2][0][RTW89_CHILE][14] = 54,
+ [0][0][2][0][RTW89_UKRAINE][14] = 52,
+ [0][0][2][0][RTW89_MEXICO][14] = 76,
+ [0][0][2][0][RTW89_CN][14] = 58,
+ [0][0][2][0][RTW89_QATAR][14] = 58,
+ [0][0][2][0][RTW89_FCC][15] = 74,
+ [0][0][2][0][RTW89_ETSI][15] = 58,
+ [0][0][2][0][RTW89_MKK][15] = 76,
+ [0][0][2][0][RTW89_IC][15] = 74,
+ [0][0][2][0][RTW89_KCC][15] = 76,
+ [0][0][2][0][RTW89_ACMA][15] = 58,
+ [0][0][2][0][RTW89_CHILE][15] = 54,
+ [0][0][2][0][RTW89_UKRAINE][15] = 52,
+ [0][0][2][0][RTW89_MEXICO][15] = 74,
+ [0][0][2][0][RTW89_CN][15] = 127,
+ [0][0][2][0][RTW89_QATAR][15] = 52,
+ [0][0][2][0][RTW89_FCC][17] = 76,
+ [0][0][2][0][RTW89_ETSI][17] = 58,
+ [0][0][2][0][RTW89_MKK][17] = 76,
+ [0][0][2][0][RTW89_IC][17] = 76,
+ [0][0][2][0][RTW89_KCC][17] = 76,
+ [0][0][2][0][RTW89_ACMA][17] = 58,
+ [0][0][2][0][RTW89_CHILE][17] = 54,
+ [0][0][2][0][RTW89_UKRAINE][17] = 52,
+ [0][0][2][0][RTW89_MEXICO][17] = 76,
+ [0][0][2][0][RTW89_CN][17] = 127,
+ [0][0][2][0][RTW89_QATAR][17] = 52,
+ [0][0][2][0][RTW89_FCC][19] = 76,
+ [0][0][2][0][RTW89_ETSI][19] = 58,
+ [0][0][2][0][RTW89_MKK][19] = 76,
+ [0][0][2][0][RTW89_IC][19] = 76,
+ [0][0][2][0][RTW89_KCC][19] = 76,
+ [0][0][2][0][RTW89_ACMA][19] = 58,
+ [0][0][2][0][RTW89_CHILE][19] = 54,
+ [0][0][2][0][RTW89_UKRAINE][19] = 52,
+ [0][0][2][0][RTW89_MEXICO][19] = 76,
+ [0][0][2][0][RTW89_CN][19] = 127,
+ [0][0][2][0][RTW89_QATAR][19] = 52,
+ [0][0][2][0][RTW89_FCC][21] = 76,
+ [0][0][2][0][RTW89_ETSI][21] = 58,
+ [0][0][2][0][RTW89_MKK][21] = 76,
+ [0][0][2][0][RTW89_IC][21] = 76,
+ [0][0][2][0][RTW89_KCC][21] = 76,
+ [0][0][2][0][RTW89_ACMA][21] = 58,
+ [0][0][2][0][RTW89_CHILE][21] = 54,
+ [0][0][2][0][RTW89_UKRAINE][21] = 52,
+ [0][0][2][0][RTW89_MEXICO][21] = 76,
+ [0][0][2][0][RTW89_CN][21] = 127,
+ [0][0][2][0][RTW89_QATAR][21] = 52,
+ [0][0][2][0][RTW89_FCC][23] = 76,
+ [0][0][2][0][RTW89_ETSI][23] = 58,
+ [0][0][2][0][RTW89_MKK][23] = 76,
+ [0][0][2][0][RTW89_IC][23] = 76,
+ [0][0][2][0][RTW89_KCC][23] = 76,
+ [0][0][2][0][RTW89_ACMA][23] = 58,
+ [0][0][2][0][RTW89_CHILE][23] = 54,
+ [0][0][2][0][RTW89_UKRAINE][23] = 52,
+ [0][0][2][0][RTW89_MEXICO][23] = 76,
+ [0][0][2][0][RTW89_CN][23] = 127,
+ [0][0][2][0][RTW89_QATAR][23] = 52,
+ [0][0][2][0][RTW89_FCC][25] = 76,
+ [0][0][2][0][RTW89_ETSI][25] = 58,
+ [0][0][2][0][RTW89_MKK][25] = 76,
+ [0][0][2][0][RTW89_IC][25] = 127,
+ [0][0][2][0][RTW89_KCC][25] = 76,
+ [0][0][2][0][RTW89_ACMA][25] = 127,
+ [0][0][2][0][RTW89_CHILE][25] = 54,
+ [0][0][2][0][RTW89_UKRAINE][25] = 52,
+ [0][0][2][0][RTW89_MEXICO][25] = 76,
+ [0][0][2][0][RTW89_CN][25] = 127,
+ [0][0][2][0][RTW89_QATAR][25] = 52,
+ [0][0][2][0][RTW89_FCC][27] = 76,
+ [0][0][2][0][RTW89_ETSI][27] = 58,
+ [0][0][2][0][RTW89_MKK][27] = 76,
+ [0][0][2][0][RTW89_IC][27] = 127,
+ [0][0][2][0][RTW89_KCC][27] = 76,
+ [0][0][2][0][RTW89_ACMA][27] = 127,
+ [0][0][2][0][RTW89_CHILE][27] = 54,
+ [0][0][2][0][RTW89_UKRAINE][27] = 52,
+ [0][0][2][0][RTW89_MEXICO][27] = 76,
+ [0][0][2][0][RTW89_CN][27] = 127,
+ [0][0][2][0][RTW89_QATAR][27] = 52,
+ [0][0][2][0][RTW89_FCC][29] = 76,
+ [0][0][2][0][RTW89_ETSI][29] = 58,
+ [0][0][2][0][RTW89_MKK][29] = 76,
+ [0][0][2][0][RTW89_IC][29] = 127,
+ [0][0][2][0][RTW89_KCC][29] = 76,
+ [0][0][2][0][RTW89_ACMA][29] = 127,
+ [0][0][2][0][RTW89_CHILE][29] = 54,
+ [0][0][2][0][RTW89_UKRAINE][29] = 52,
+ [0][0][2][0][RTW89_MEXICO][29] = 76,
+ [0][0][2][0][RTW89_CN][29] = 127,
+ [0][0][2][0][RTW89_QATAR][29] = 52,
+ [0][0][2][0][RTW89_FCC][31] = 76,
+ [0][0][2][0][RTW89_ETSI][31] = 58,
+ [0][0][2][0][RTW89_MKK][31] = 76,
+ [0][0][2][0][RTW89_IC][31] = 76,
+ [0][0][2][0][RTW89_KCC][31] = 76,
+ [0][0][2][0][RTW89_ACMA][31] = 58,
+ [0][0][2][0][RTW89_CHILE][31] = 54,
+ [0][0][2][0][RTW89_UKRAINE][31] = 52,
+ [0][0][2][0][RTW89_MEXICO][31] = 76,
+ [0][0][2][0][RTW89_CN][31] = 127,
+ [0][0][2][0][RTW89_QATAR][31] = 52,
+ [0][0][2][0][RTW89_FCC][33] = 76,
+ [0][0][2][0][RTW89_ETSI][33] = 58,
+ [0][0][2][0][RTW89_MKK][33] = 76,
+ [0][0][2][0][RTW89_IC][33] = 76,
+ [0][0][2][0][RTW89_KCC][33] = 76,
+ [0][0][2][0][RTW89_ACMA][33] = 58,
+ [0][0][2][0][RTW89_CHILE][33] = 54,
+ [0][0][2][0][RTW89_UKRAINE][33] = 52,
+ [0][0][2][0][RTW89_MEXICO][33] = 76,
+ [0][0][2][0][RTW89_CN][33] = 127,
+ [0][0][2][0][RTW89_QATAR][33] = 52,
+ [0][0][2][0][RTW89_FCC][35] = 70,
+ [0][0][2][0][RTW89_ETSI][35] = 58,
+ [0][0][2][0][RTW89_MKK][35] = 76,
+ [0][0][2][0][RTW89_IC][35] = 70,
+ [0][0][2][0][RTW89_KCC][35] = 76,
+ [0][0][2][0][RTW89_ACMA][35] = 58,
+ [0][0][2][0][RTW89_CHILE][35] = 54,
+ [0][0][2][0][RTW89_UKRAINE][35] = 52,
+ [0][0][2][0][RTW89_MEXICO][35] = 70,
+ [0][0][2][0][RTW89_CN][35] = 127,
+ [0][0][2][0][RTW89_QATAR][35] = 52,
+ [0][0][2][0][RTW89_FCC][37] = 76,
+ [0][0][2][0][RTW89_ETSI][37] = 127,
+ [0][0][2][0][RTW89_MKK][37] = 76,
+ [0][0][2][0][RTW89_IC][37] = 76,
+ [0][0][2][0][RTW89_KCC][37] = 76,
+ [0][0][2][0][RTW89_ACMA][37] = 76,
+ [0][0][2][0][RTW89_CHILE][37] = 54,
+ [0][0][2][0][RTW89_UKRAINE][37] = 127,
+ [0][0][2][0][RTW89_MEXICO][37] = 76,
+ [0][0][2][0][RTW89_CN][37] = 127,
+ [0][0][2][0][RTW89_QATAR][37] = 127,
+ [0][0][2][0][RTW89_FCC][38] = 76,
+ [0][0][2][0][RTW89_ETSI][38] = 28,
+ [0][0][2][0][RTW89_MKK][38] = 127,
+ [0][0][2][0][RTW89_IC][38] = 76,
+ [0][0][2][0][RTW89_KCC][38] = 76,
+ [0][0][2][0][RTW89_ACMA][38] = 76,
+ [0][0][2][0][RTW89_CHILE][38] = 54,
+ [0][0][2][0][RTW89_UKRAINE][38] = 28,
+ [0][0][2][0][RTW89_MEXICO][38] = 76,
+ [0][0][2][0][RTW89_CN][38] = 68,
+ [0][0][2][0][RTW89_QATAR][38] = 28,
+ [0][0][2][0][RTW89_FCC][40] = 76,
+ [0][0][2][0][RTW89_ETSI][40] = 28,
+ [0][0][2][0][RTW89_MKK][40] = 127,
+ [0][0][2][0][RTW89_IC][40] = 76,
+ [0][0][2][0][RTW89_KCC][40] = 76,
+ [0][0][2][0][RTW89_ACMA][40] = 76,
+ [0][0][2][0][RTW89_CHILE][40] = 54,
+ [0][0][2][0][RTW89_UKRAINE][40] = 28,
+ [0][0][2][0][RTW89_MEXICO][40] = 76,
+ [0][0][2][0][RTW89_CN][40] = 76,
+ [0][0][2][0][RTW89_QATAR][40] = 28,
+ [0][0][2][0][RTW89_FCC][42] = 76,
+ [0][0][2][0][RTW89_ETSI][42] = 28,
+ [0][0][2][0][RTW89_MKK][42] = 127,
+ [0][0][2][0][RTW89_IC][42] = 76,
+ [0][0][2][0][RTW89_KCC][42] = 76,
+ [0][0][2][0][RTW89_ACMA][42] = 76,
+ [0][0][2][0][RTW89_CHILE][42] = 54,
+ [0][0][2][0][RTW89_UKRAINE][42] = 28,
+ [0][0][2][0][RTW89_MEXICO][42] = 76,
+ [0][0][2][0][RTW89_CN][42] = 76,
+ [0][0][2][0][RTW89_QATAR][42] = 28,
+ [0][0][2][0][RTW89_FCC][44] = 76,
+ [0][0][2][0][RTW89_ETSI][44] = 28,
+ [0][0][2][0][RTW89_MKK][44] = 127,
+ [0][0][2][0][RTW89_IC][44] = 76,
+ [0][0][2][0][RTW89_KCC][44] = 76,
+ [0][0][2][0][RTW89_ACMA][44] = 76,
+ [0][0][2][0][RTW89_CHILE][44] = 54,
+ [0][0][2][0][RTW89_UKRAINE][44] = 28,
+ [0][0][2][0][RTW89_MEXICO][44] = 76,
+ [0][0][2][0][RTW89_CN][44] = 76,
+ [0][0][2][0][RTW89_QATAR][44] = 28,
+ [0][0][2][0][RTW89_FCC][46] = 76,
+ [0][0][2][0][RTW89_ETSI][46] = 28,
+ [0][0][2][0][RTW89_MKK][46] = 127,
+ [0][0][2][0][RTW89_IC][46] = 76,
+ [0][0][2][0][RTW89_KCC][46] = 76,
+ [0][0][2][0][RTW89_ACMA][46] = 76,
+ [0][0][2][0][RTW89_CHILE][46] = 54,
+ [0][0][2][0][RTW89_UKRAINE][46] = 28,
+ [0][0][2][0][RTW89_MEXICO][46] = 76,
+ [0][0][2][0][RTW89_CN][46] = 76,
+ [0][0][2][0][RTW89_QATAR][46] = 28,
+ [0][1][2][0][RTW89_FCC][0] = 68,
+ [0][1][2][0][RTW89_ETSI][0] = 46,
+ [0][1][2][0][RTW89_MKK][0] = 50,
+ [0][1][2][0][RTW89_IC][0] = 40,
+ [0][1][2][0][RTW89_KCC][0] = 68,
+ [0][1][2][0][RTW89_ACMA][0] = 46,
+ [0][1][2][0][RTW89_CHILE][0] = 18,
+ [0][1][2][0][RTW89_UKRAINE][0] = 40,
+ [0][1][2][0][RTW89_MEXICO][0] = 50,
+ [0][1][2][0][RTW89_CN][0] = 46,
+ [0][1][2][0][RTW89_QATAR][0] = 46,
+ [0][1][2][0][RTW89_FCC][2] = 68,
+ [0][1][2][0][RTW89_ETSI][2] = 46,
+ [0][1][2][0][RTW89_MKK][2] = 50,
+ [0][1][2][0][RTW89_IC][2] = 40,
+ [0][1][2][0][RTW89_KCC][2] = 68,
+ [0][1][2][0][RTW89_ACMA][2] = 46,
+ [0][1][2][0][RTW89_CHILE][2] = 18,
+ [0][1][2][0][RTW89_UKRAINE][2] = 40,
+ [0][1][2][0][RTW89_MEXICO][2] = 50,
+ [0][1][2][0][RTW89_CN][2] = 46,
+ [0][1][2][0][RTW89_QATAR][2] = 46,
+ [0][1][2][0][RTW89_FCC][4] = 68,
+ [0][1][2][0][RTW89_ETSI][4] = 46,
+ [0][1][2][0][RTW89_MKK][4] = 50,
+ [0][1][2][0][RTW89_IC][4] = 40,
+ [0][1][2][0][RTW89_KCC][4] = 68,
+ [0][1][2][0][RTW89_ACMA][4] = 46,
+ [0][1][2][0][RTW89_CHILE][4] = 18,
+ [0][1][2][0][RTW89_UKRAINE][4] = 40,
+ [0][1][2][0][RTW89_MEXICO][4] = 50,
+ [0][1][2][0][RTW89_CN][4] = 46,
+ [0][1][2][0][RTW89_QATAR][4] = 46,
+ [0][1][2][0][RTW89_FCC][6] = 68,
+ [0][1][2][0][RTW89_ETSI][6] = 46,
+ [0][1][2][0][RTW89_MKK][6] = 50,
+ [0][1][2][0][RTW89_IC][6] = 40,
+ [0][1][2][0][RTW89_KCC][6] = 38,
+ [0][1][2][0][RTW89_ACMA][6] = 46,
+ [0][1][2][0][RTW89_CHILE][6] = 18,
+ [0][1][2][0][RTW89_UKRAINE][6] = 40,
+ [0][1][2][0][RTW89_MEXICO][6] = 50,
+ [0][1][2][0][RTW89_CN][6] = 46,
+ [0][1][2][0][RTW89_QATAR][6] = 46,
+ [0][1][2][0][RTW89_FCC][8] = 68,
+ [0][1][2][0][RTW89_ETSI][8] = 46,
+ [0][1][2][0][RTW89_MKK][8] = 50,
+ [0][1][2][0][RTW89_IC][8] = 52,
+ [0][1][2][0][RTW89_KCC][8] = 68,
+ [0][1][2][0][RTW89_ACMA][8] = 46,
+ [0][1][2][0][RTW89_CHILE][8] = 42,
+ [0][1][2][0][RTW89_UKRAINE][8] = 40,
+ [0][1][2][0][RTW89_MEXICO][8] = 68,
+ [0][1][2][0][RTW89_CN][8] = 46,
+ [0][1][2][0][RTW89_QATAR][8] = 46,
+ [0][1][2][0][RTW89_FCC][10] = 68,
+ [0][1][2][0][RTW89_ETSI][10] = 46,
+ [0][1][2][0][RTW89_MKK][10] = 50,
+ [0][1][2][0][RTW89_IC][10] = 52,
+ [0][1][2][0][RTW89_KCC][10] = 68,
+ [0][1][2][0][RTW89_ACMA][10] = 46,
+ [0][1][2][0][RTW89_CHILE][10] = 42,
+ [0][1][2][0][RTW89_UKRAINE][10] = 40,
+ [0][1][2][0][RTW89_MEXICO][10] = 68,
+ [0][1][2][0][RTW89_CN][10] = 46,
+ [0][1][2][0][RTW89_QATAR][10] = 46,
+ [0][1][2][0][RTW89_FCC][12] = 68,
+ [0][1][2][0][RTW89_ETSI][12] = 46,
+ [0][1][2][0][RTW89_MKK][12] = 50,
+ [0][1][2][0][RTW89_IC][12] = 52,
+ [0][1][2][0][RTW89_KCC][12] = 68,
+ [0][1][2][0][RTW89_ACMA][12] = 46,
+ [0][1][2][0][RTW89_CHILE][12] = 42,
+ [0][1][2][0][RTW89_UKRAINE][12] = 40,
+ [0][1][2][0][RTW89_MEXICO][12] = 68,
+ [0][1][2][0][RTW89_CN][12] = 46,
+ [0][1][2][0][RTW89_QATAR][12] = 46,
+ [0][1][2][0][RTW89_FCC][14] = 68,
+ [0][1][2][0][RTW89_ETSI][14] = 46,
+ [0][1][2][0][RTW89_MKK][14] = 50,
+ [0][1][2][0][RTW89_IC][14] = 52,
+ [0][1][2][0][RTW89_KCC][14] = 68,
+ [0][1][2][0][RTW89_ACMA][14] = 46,
+ [0][1][2][0][RTW89_CHILE][14] = 42,
+ [0][1][2][0][RTW89_UKRAINE][14] = 40,
+ [0][1][2][0][RTW89_MEXICO][14] = 68,
+ [0][1][2][0][RTW89_CN][14] = 46,
+ [0][1][2][0][RTW89_QATAR][14] = 46,
+ [0][1][2][0][RTW89_FCC][15] = 68,
+ [0][1][2][0][RTW89_ETSI][15] = 46,
+ [0][1][2][0][RTW89_MKK][15] = 70,
+ [0][1][2][0][RTW89_IC][15] = 68,
+ [0][1][2][0][RTW89_KCC][15] = 66,
+ [0][1][2][0][RTW89_ACMA][15] = 46,
+ [0][1][2][0][RTW89_CHILE][15] = 42,
+ [0][1][2][0][RTW89_UKRAINE][15] = 40,
+ [0][1][2][0][RTW89_MEXICO][15] = 68,
+ [0][1][2][0][RTW89_CN][15] = 127,
+ [0][1][2][0][RTW89_QATAR][15] = 40,
+ [0][1][2][0][RTW89_FCC][17] = 68,
+ [0][1][2][0][RTW89_ETSI][17] = 46,
+ [0][1][2][0][RTW89_MKK][17] = 70,
+ [0][1][2][0][RTW89_IC][17] = 68,
+ [0][1][2][0][RTW89_KCC][17] = 66,
+ [0][1][2][0][RTW89_ACMA][17] = 46,
+ [0][1][2][0][RTW89_CHILE][17] = 42,
+ [0][1][2][0][RTW89_UKRAINE][17] = 40,
+ [0][1][2][0][RTW89_MEXICO][17] = 68,
+ [0][1][2][0][RTW89_CN][17] = 127,
+ [0][1][2][0][RTW89_QATAR][17] = 40,
+ [0][1][2][0][RTW89_FCC][19] = 68,
+ [0][1][2][0][RTW89_ETSI][19] = 46,
+ [0][1][2][0][RTW89_MKK][19] = 70,
+ [0][1][2][0][RTW89_IC][19] = 68,
+ [0][1][2][0][RTW89_KCC][19] = 66,
+ [0][1][2][0][RTW89_ACMA][19] = 46,
+ [0][1][2][0][RTW89_CHILE][19] = 42,
+ [0][1][2][0][RTW89_UKRAINE][19] = 40,
+ [0][1][2][0][RTW89_MEXICO][19] = 68,
+ [0][1][2][0][RTW89_CN][19] = 127,
+ [0][1][2][0][RTW89_QATAR][19] = 40,
+ [0][1][2][0][RTW89_FCC][21] = 68,
+ [0][1][2][0][RTW89_ETSI][21] = 46,
+ [0][1][2][0][RTW89_MKK][21] = 70,
+ [0][1][2][0][RTW89_IC][21] = 68,
+ [0][1][2][0][RTW89_KCC][21] = 66,
+ [0][1][2][0][RTW89_ACMA][21] = 46,
+ [0][1][2][0][RTW89_CHILE][21] = 42,
+ [0][1][2][0][RTW89_UKRAINE][21] = 40,
+ [0][1][2][0][RTW89_MEXICO][21] = 68,
+ [0][1][2][0][RTW89_CN][21] = 127,
+ [0][1][2][0][RTW89_QATAR][21] = 40,
+ [0][1][2][0][RTW89_FCC][23] = 68,
+ [0][1][2][0][RTW89_ETSI][23] = 46,
+ [0][1][2][0][RTW89_MKK][23] = 70,
+ [0][1][2][0][RTW89_IC][23] = 68,
+ [0][1][2][0][RTW89_KCC][23] = 66,
+ [0][1][2][0][RTW89_ACMA][23] = 46,
+ [0][1][2][0][RTW89_CHILE][23] = 42,
+ [0][1][2][0][RTW89_UKRAINE][23] = 40,
+ [0][1][2][0][RTW89_MEXICO][23] = 68,
+ [0][1][2][0][RTW89_CN][23] = 127,
+ [0][1][2][0][RTW89_QATAR][23] = 40,
+ [0][1][2][0][RTW89_FCC][25] = 68,
+ [0][1][2][0][RTW89_ETSI][25] = 46,
+ [0][1][2][0][RTW89_MKK][25] = 70,
+ [0][1][2][0][RTW89_IC][25] = 127,
+ [0][1][2][0][RTW89_KCC][25] = 66,
+ [0][1][2][0][RTW89_ACMA][25] = 127,
+ [0][1][2][0][RTW89_CHILE][25] = 42,
+ [0][1][2][0][RTW89_UKRAINE][25] = 40,
+ [0][1][2][0][RTW89_MEXICO][25] = 68,
+ [0][1][2][0][RTW89_CN][25] = 127,
+ [0][1][2][0][RTW89_QATAR][25] = 40,
+ [0][1][2][0][RTW89_FCC][27] = 68,
+ [0][1][2][0][RTW89_ETSI][27] = 46,
+ [0][1][2][0][RTW89_MKK][27] = 70,
+ [0][1][2][0][RTW89_IC][27] = 127,
+ [0][1][2][0][RTW89_KCC][27] = 66,
+ [0][1][2][0][RTW89_ACMA][27] = 127,
+ [0][1][2][0][RTW89_CHILE][27] = 42,
+ [0][1][2][0][RTW89_UKRAINE][27] = 40,
+ [0][1][2][0][RTW89_MEXICO][27] = 68,
+ [0][1][2][0][RTW89_CN][27] = 127,
+ [0][1][2][0][RTW89_QATAR][27] = 40,
+ [0][1][2][0][RTW89_FCC][29] = 68,
+ [0][1][2][0][RTW89_ETSI][29] = 46,
+ [0][1][2][0][RTW89_MKK][29] = 70,
+ [0][1][2][0][RTW89_IC][29] = 127,
+ [0][1][2][0][RTW89_KCC][29] = 66,
+ [0][1][2][0][RTW89_ACMA][29] = 127,
+ [0][1][2][0][RTW89_CHILE][29] = 42,
+ [0][1][2][0][RTW89_UKRAINE][29] = 40,
+ [0][1][2][0][RTW89_MEXICO][29] = 68,
+ [0][1][2][0][RTW89_CN][29] = 127,
+ [0][1][2][0][RTW89_QATAR][29] = 40,
+ [0][1][2][0][RTW89_FCC][31] = 68,
+ [0][1][2][0][RTW89_ETSI][31] = 46,
+ [0][1][2][0][RTW89_MKK][31] = 70,
+ [0][1][2][0][RTW89_IC][31] = 68,
+ [0][1][2][0][RTW89_KCC][31] = 66,
+ [0][1][2][0][RTW89_ACMA][31] = 46,
+ [0][1][2][0][RTW89_CHILE][31] = 42,
+ [0][1][2][0][RTW89_UKRAINE][31] = 40,
+ [0][1][2][0][RTW89_MEXICO][31] = 68,
+ [0][1][2][0][RTW89_CN][31] = 127,
+ [0][1][2][0][RTW89_QATAR][31] = 40,
+ [0][1][2][0][RTW89_FCC][33] = 68,
+ [0][1][2][0][RTW89_ETSI][33] = 46,
+ [0][1][2][0][RTW89_MKK][33] = 70,
+ [0][1][2][0][RTW89_IC][33] = 68,
+ [0][1][2][0][RTW89_KCC][33] = 66,
+ [0][1][2][0][RTW89_ACMA][33] = 46,
+ [0][1][2][0][RTW89_CHILE][33] = 42,
+ [0][1][2][0][RTW89_UKRAINE][33] = 40,
+ [0][1][2][0][RTW89_MEXICO][33] = 68,
+ [0][1][2][0][RTW89_CN][33] = 127,
+ [0][1][2][0][RTW89_QATAR][33] = 40,
+ [0][1][2][0][RTW89_FCC][35] = 64,
+ [0][1][2][0][RTW89_ETSI][35] = 46,
+ [0][1][2][0][RTW89_MKK][35] = 70,
+ [0][1][2][0][RTW89_IC][35] = 64,
+ [0][1][2][0][RTW89_KCC][35] = 66,
+ [0][1][2][0][RTW89_ACMA][35] = 46,
+ [0][1][2][0][RTW89_CHILE][35] = 42,
+ [0][1][2][0][RTW89_UKRAINE][35] = 40,
+ [0][1][2][0][RTW89_MEXICO][35] = 64,
+ [0][1][2][0][RTW89_CN][35] = 127,
+ [0][1][2][0][RTW89_QATAR][35] = 40,
+ [0][1][2][0][RTW89_FCC][37] = 68,
+ [0][1][2][0][RTW89_ETSI][37] = 127,
+ [0][1][2][0][RTW89_MKK][37] = 70,
+ [0][1][2][0][RTW89_IC][37] = 68,
+ [0][1][2][0][RTW89_KCC][37] = 66,
+ [0][1][2][0][RTW89_ACMA][37] = 68,
+ [0][1][2][0][RTW89_CHILE][37] = 42,
+ [0][1][2][0][RTW89_UKRAINE][37] = 127,
+ [0][1][2][0][RTW89_MEXICO][37] = 68,
+ [0][1][2][0][RTW89_CN][37] = 127,
+ [0][1][2][0][RTW89_QATAR][37] = 127,
+ [0][1][2][0][RTW89_FCC][38] = 76,
+ [0][1][2][0][RTW89_ETSI][38] = 16,
+ [0][1][2][0][RTW89_MKK][38] = 127,
+ [0][1][2][0][RTW89_IC][38] = 76,
+ [0][1][2][0][RTW89_KCC][38] = 66,
+ [0][1][2][0][RTW89_ACMA][38] = 76,
+ [0][1][2][0][RTW89_CHILE][38] = 42,
+ [0][1][2][0][RTW89_UKRAINE][38] = 16,
+ [0][1][2][0][RTW89_MEXICO][38] = 76,
+ [0][1][2][0][RTW89_CN][38] = 68,
+ [0][1][2][0][RTW89_QATAR][38] = 16,
+ [0][1][2][0][RTW89_FCC][40] = 76,
+ [0][1][2][0][RTW89_ETSI][40] = 16,
+ [0][1][2][0][RTW89_MKK][40] = 127,
+ [0][1][2][0][RTW89_IC][40] = 76,
+ [0][1][2][0][RTW89_KCC][40] = 66,
+ [0][1][2][0][RTW89_ACMA][40] = 76,
+ [0][1][2][0][RTW89_CHILE][40] = 42,
+ [0][1][2][0][RTW89_UKRAINE][40] = 16,
+ [0][1][2][0][RTW89_MEXICO][40] = 76,
+ [0][1][2][0][RTW89_CN][40] = 76,
+ [0][1][2][0][RTW89_QATAR][40] = 16,
+ [0][1][2][0][RTW89_FCC][42] = 76,
+ [0][1][2][0][RTW89_ETSI][42] = 16,
+ [0][1][2][0][RTW89_MKK][42] = 127,
+ [0][1][2][0][RTW89_IC][42] = 76,
+ [0][1][2][0][RTW89_KCC][42] = 66,
+ [0][1][2][0][RTW89_ACMA][42] = 76,
+ [0][1][2][0][RTW89_CHILE][42] = 42,
+ [0][1][2][0][RTW89_UKRAINE][42] = 16,
+ [0][1][2][0][RTW89_MEXICO][42] = 76,
+ [0][1][2][0][RTW89_CN][42] = 76,
+ [0][1][2][0][RTW89_QATAR][42] = 16,
+ [0][1][2][0][RTW89_FCC][44] = 76,
+ [0][1][2][0][RTW89_ETSI][44] = 16,
+ [0][1][2][0][RTW89_MKK][44] = 127,
+ [0][1][2][0][RTW89_IC][44] = 76,
+ [0][1][2][0][RTW89_KCC][44] = 66,
+ [0][1][2][0][RTW89_ACMA][44] = 76,
+ [0][1][2][0][RTW89_CHILE][44] = 42,
+ [0][1][2][0][RTW89_UKRAINE][44] = 16,
+ [0][1][2][0][RTW89_MEXICO][44] = 76,
+ [0][1][2][0][RTW89_CN][44] = 76,
+ [0][1][2][0][RTW89_QATAR][44] = 16,
+ [0][1][2][0][RTW89_FCC][46] = 76,
+ [0][1][2][0][RTW89_ETSI][46] = 16,
+ [0][1][2][0][RTW89_MKK][46] = 127,
+ [0][1][2][0][RTW89_IC][46] = 76,
+ [0][1][2][0][RTW89_KCC][46] = 66,
+ [0][1][2][0][RTW89_ACMA][46] = 76,
+ [0][1][2][0][RTW89_CHILE][46] = 42,
+ [0][1][2][0][RTW89_UKRAINE][46] = 16,
+ [0][1][2][0][RTW89_MEXICO][46] = 76,
+ [0][1][2][0][RTW89_CN][46] = 76,
+ [0][1][2][0][RTW89_QATAR][46] = 16,
+ [0][1][2][1][RTW89_FCC][0] = 68,
+ [0][1][2][1][RTW89_ETSI][0] = 34,
+ [0][1][2][1][RTW89_MKK][0] = 50,
+ [0][1][2][1][RTW89_IC][0] = 38,
+ [0][1][2][1][RTW89_KCC][0] = 68,
+ [0][1][2][1][RTW89_ACMA][0] = 34,
+ [0][1][2][1][RTW89_CHILE][0] = 6,
+ [0][1][2][1][RTW89_UKRAINE][0] = 28,
+ [0][1][2][1][RTW89_MEXICO][0] = 50,
+ [0][1][2][1][RTW89_CN][0] = 34,
+ [0][1][2][1][RTW89_QATAR][0] = 34,
+ [0][1][2][1][RTW89_FCC][2] = 68,
+ [0][1][2][1][RTW89_ETSI][2] = 34,
+ [0][1][2][1][RTW89_MKK][2] = 50,
+ [0][1][2][1][RTW89_IC][2] = 38,
+ [0][1][2][1][RTW89_KCC][2] = 68,
+ [0][1][2][1][RTW89_ACMA][2] = 34,
+ [0][1][2][1][RTW89_CHILE][2] = 6,
+ [0][1][2][1][RTW89_UKRAINE][2] = 28,
+ [0][1][2][1][RTW89_MEXICO][2] = 50,
+ [0][1][2][1][RTW89_CN][2] = 34,
+ [0][1][2][1][RTW89_QATAR][2] = 34,
+ [0][1][2][1][RTW89_FCC][4] = 68,
+ [0][1][2][1][RTW89_ETSI][4] = 34,
+ [0][1][2][1][RTW89_MKK][4] = 50,
+ [0][1][2][1][RTW89_IC][4] = 38,
+ [0][1][2][1][RTW89_KCC][4] = 68,
+ [0][1][2][1][RTW89_ACMA][4] = 34,
+ [0][1][2][1][RTW89_CHILE][4] = 6,
+ [0][1][2][1][RTW89_UKRAINE][4] = 28,
+ [0][1][2][1][RTW89_MEXICO][4] = 50,
+ [0][1][2][1][RTW89_CN][4] = 34,
+ [0][1][2][1][RTW89_QATAR][4] = 34,
+ [0][1][2][1][RTW89_FCC][6] = 68,
+ [0][1][2][1][RTW89_ETSI][6] = 34,
+ [0][1][2][1][RTW89_MKK][6] = 50,
+ [0][1][2][1][RTW89_IC][6] = 38,
+ [0][1][2][1][RTW89_KCC][6] = 38,
+ [0][1][2][1][RTW89_ACMA][6] = 34,
+ [0][1][2][1][RTW89_CHILE][6] = 6,
+ [0][1][2][1][RTW89_UKRAINE][6] = 28,
+ [0][1][2][1][RTW89_MEXICO][6] = 50,
+ [0][1][2][1][RTW89_CN][6] = 34,
+ [0][1][2][1][RTW89_QATAR][6] = 34,
+ [0][1][2][1][RTW89_FCC][8] = 68,
+ [0][1][2][1][RTW89_ETSI][8] = 34,
+ [0][1][2][1][RTW89_MKK][8] = 50,
+ [0][1][2][1][RTW89_IC][8] = 38,
+ [0][1][2][1][RTW89_KCC][8] = 68,
+ [0][1][2][1][RTW89_ACMA][8] = 34,
+ [0][1][2][1][RTW89_CHILE][8] = 30,
+ [0][1][2][1][RTW89_UKRAINE][8] = 28,
+ [0][1][2][1][RTW89_MEXICO][8] = 68,
+ [0][1][2][1][RTW89_CN][8] = 34,
+ [0][1][2][1][RTW89_QATAR][8] = 34,
+ [0][1][2][1][RTW89_FCC][10] = 68,
+ [0][1][2][1][RTW89_ETSI][10] = 34,
+ [0][1][2][1][RTW89_MKK][10] = 50,
+ [0][1][2][1][RTW89_IC][10] = 38,
+ [0][1][2][1][RTW89_KCC][10] = 68,
+ [0][1][2][1][RTW89_ACMA][10] = 34,
+ [0][1][2][1][RTW89_CHILE][10] = 30,
+ [0][1][2][1][RTW89_UKRAINE][10] = 28,
+ [0][1][2][1][RTW89_MEXICO][10] = 68,
+ [0][1][2][1][RTW89_CN][10] = 34,
+ [0][1][2][1][RTW89_QATAR][10] = 34,
+ [0][1][2][1][RTW89_FCC][12] = 68,
+ [0][1][2][1][RTW89_ETSI][12] = 34,
+ [0][1][2][1][RTW89_MKK][12] = 50,
+ [0][1][2][1][RTW89_IC][12] = 38,
+ [0][1][2][1][RTW89_KCC][12] = 68,
+ [0][1][2][1][RTW89_ACMA][12] = 34,
+ [0][1][2][1][RTW89_CHILE][12] = 30,
+ [0][1][2][1][RTW89_UKRAINE][12] = 28,
+ [0][1][2][1][RTW89_MEXICO][12] = 68,
+ [0][1][2][1][RTW89_CN][12] = 34,
+ [0][1][2][1][RTW89_QATAR][12] = 34,
+ [0][1][2][1][RTW89_FCC][14] = 68,
+ [0][1][2][1][RTW89_ETSI][14] = 34,
+ [0][1][2][1][RTW89_MKK][14] = 50,
+ [0][1][2][1][RTW89_IC][14] = 38,
+ [0][1][2][1][RTW89_KCC][14] = 68,
+ [0][1][2][1][RTW89_ACMA][14] = 34,
+ [0][1][2][1][RTW89_CHILE][14] = 30,
+ [0][1][2][1][RTW89_UKRAINE][14] = 28,
+ [0][1][2][1][RTW89_MEXICO][14] = 68,
+ [0][1][2][1][RTW89_CN][14] = 34,
+ [0][1][2][1][RTW89_QATAR][14] = 34,
+ [0][1][2][1][RTW89_FCC][15] = 68,
+ [0][1][2][1][RTW89_ETSI][15] = 34,
+ [0][1][2][1][RTW89_MKK][15] = 70,
+ [0][1][2][1][RTW89_IC][15] = 62,
+ [0][1][2][1][RTW89_KCC][15] = 66,
+ [0][1][2][1][RTW89_ACMA][15] = 34,
+ [0][1][2][1][RTW89_CHILE][15] = 30,
+ [0][1][2][1][RTW89_UKRAINE][15] = 28,
+ [0][1][2][1][RTW89_MEXICO][15] = 68,
+ [0][1][2][1][RTW89_CN][15] = 127,
+ [0][1][2][1][RTW89_QATAR][15] = 28,
+ [0][1][2][1][RTW89_FCC][17] = 68,
+ [0][1][2][1][RTW89_ETSI][17] = 34,
+ [0][1][2][1][RTW89_MKK][17] = 70,
+ [0][1][2][1][RTW89_IC][17] = 62,
+ [0][1][2][1][RTW89_KCC][17] = 66,
+ [0][1][2][1][RTW89_ACMA][17] = 34,
+ [0][1][2][1][RTW89_CHILE][17] = 30,
+ [0][1][2][1][RTW89_UKRAINE][17] = 28,
+ [0][1][2][1][RTW89_MEXICO][17] = 68,
+ [0][1][2][1][RTW89_CN][17] = 127,
+ [0][1][2][1][RTW89_QATAR][17] = 28,
+ [0][1][2][1][RTW89_FCC][19] = 68,
+ [0][1][2][1][RTW89_ETSI][19] = 34,
+ [0][1][2][1][RTW89_MKK][19] = 70,
+ [0][1][2][1][RTW89_IC][19] = 62,
+ [0][1][2][1][RTW89_KCC][19] = 66,
+ [0][1][2][1][RTW89_ACMA][19] = 34,
+ [0][1][2][1][RTW89_CHILE][19] = 30,
+ [0][1][2][1][RTW89_UKRAINE][19] = 28,
+ [0][1][2][1][RTW89_MEXICO][19] = 68,
+ [0][1][2][1][RTW89_CN][19] = 127,
+ [0][1][2][1][RTW89_QATAR][19] = 28,
+ [0][1][2][1][RTW89_FCC][21] = 68,
+ [0][1][2][1][RTW89_ETSI][21] = 34,
+ [0][1][2][1][RTW89_MKK][21] = 70,
+ [0][1][2][1][RTW89_IC][21] = 62,
+ [0][1][2][1][RTW89_KCC][21] = 66,
+ [0][1][2][1][RTW89_ACMA][21] = 34,
+ [0][1][2][1][RTW89_CHILE][21] = 30,
+ [0][1][2][1][RTW89_UKRAINE][21] = 28,
+ [0][1][2][1][RTW89_MEXICO][21] = 68,
+ [0][1][2][1][RTW89_CN][21] = 127,
+ [0][1][2][1][RTW89_QATAR][21] = 28,
+ [0][1][2][1][RTW89_FCC][23] = 68,
+ [0][1][2][1][RTW89_ETSI][23] = 34,
+ [0][1][2][1][RTW89_MKK][23] = 70,
+ [0][1][2][1][RTW89_IC][23] = 62,
+ [0][1][2][1][RTW89_KCC][23] = 66,
+ [0][1][2][1][RTW89_ACMA][23] = 34,
+ [0][1][2][1][RTW89_CHILE][23] = 30,
+ [0][1][2][1][RTW89_UKRAINE][23] = 28,
+ [0][1][2][1][RTW89_MEXICO][23] = 68,
+ [0][1][2][1][RTW89_CN][23] = 127,
+ [0][1][2][1][RTW89_QATAR][23] = 28,
+ [0][1][2][1][RTW89_FCC][25] = 68,
+ [0][1][2][1][RTW89_ETSI][25] = 34,
+ [0][1][2][1][RTW89_MKK][25] = 70,
+ [0][1][2][1][RTW89_IC][25] = 127,
+ [0][1][2][1][RTW89_KCC][25] = 66,
+ [0][1][2][1][RTW89_ACMA][25] = 127,
+ [0][1][2][1][RTW89_CHILE][25] = 30,
+ [0][1][2][1][RTW89_UKRAINE][25] = 28,
+ [0][1][2][1][RTW89_MEXICO][25] = 68,
+ [0][1][2][1][RTW89_CN][25] = 127,
+ [0][1][2][1][RTW89_QATAR][25] = 28,
+ [0][1][2][1][RTW89_FCC][27] = 68,
+ [0][1][2][1][RTW89_ETSI][27] = 34,
+ [0][1][2][1][RTW89_MKK][27] = 70,
+ [0][1][2][1][RTW89_IC][27] = 127,
+ [0][1][2][1][RTW89_KCC][27] = 66,
+ [0][1][2][1][RTW89_ACMA][27] = 127,
+ [0][1][2][1][RTW89_CHILE][27] = 30,
+ [0][1][2][1][RTW89_UKRAINE][27] = 28,
+ [0][1][2][1][RTW89_MEXICO][27] = 68,
+ [0][1][2][1][RTW89_CN][27] = 127,
+ [0][1][2][1][RTW89_QATAR][27] = 28,
+ [0][1][2][1][RTW89_FCC][29] = 68,
+ [0][1][2][1][RTW89_ETSI][29] = 34,
+ [0][1][2][1][RTW89_MKK][29] = 70,
+ [0][1][2][1][RTW89_IC][29] = 127,
+ [0][1][2][1][RTW89_KCC][29] = 66,
+ [0][1][2][1][RTW89_ACMA][29] = 127,
+ [0][1][2][1][RTW89_CHILE][29] = 30,
+ [0][1][2][1][RTW89_UKRAINE][29] = 28,
+ [0][1][2][1][RTW89_MEXICO][29] = 68,
+ [0][1][2][1][RTW89_CN][29] = 127,
+ [0][1][2][1][RTW89_QATAR][29] = 28,
+ [0][1][2][1][RTW89_FCC][31] = 68,
+ [0][1][2][1][RTW89_ETSI][31] = 34,
+ [0][1][2][1][RTW89_MKK][31] = 70,
+ [0][1][2][1][RTW89_IC][31] = 62,
+ [0][1][2][1][RTW89_KCC][31] = 66,
+ [0][1][2][1][RTW89_ACMA][31] = 34,
+ [0][1][2][1][RTW89_CHILE][31] = 30,
+ [0][1][2][1][RTW89_UKRAINE][31] = 28,
+ [0][1][2][1][RTW89_MEXICO][31] = 68,
+ [0][1][2][1][RTW89_CN][31] = 127,
+ [0][1][2][1][RTW89_QATAR][31] = 28,
+ [0][1][2][1][RTW89_FCC][33] = 68,
+ [0][1][2][1][RTW89_ETSI][33] = 34,
+ [0][1][2][1][RTW89_MKK][33] = 70,
+ [0][1][2][1][RTW89_IC][33] = 62,
+ [0][1][2][1][RTW89_KCC][33] = 66,
+ [0][1][2][1][RTW89_ACMA][33] = 34,
+ [0][1][2][1][RTW89_CHILE][33] = 30,
+ [0][1][2][1][RTW89_UKRAINE][33] = 28,
+ [0][1][2][1][RTW89_MEXICO][33] = 68,
+ [0][1][2][1][RTW89_CN][33] = 127,
+ [0][1][2][1][RTW89_QATAR][33] = 28,
+ [0][1][2][1][RTW89_FCC][35] = 64,
+ [0][1][2][1][RTW89_ETSI][35] = 34,
+ [0][1][2][1][RTW89_MKK][35] = 70,
+ [0][1][2][1][RTW89_IC][35] = 62,
+ [0][1][2][1][RTW89_KCC][35] = 66,
+ [0][1][2][1][RTW89_ACMA][35] = 34,
+ [0][1][2][1][RTW89_CHILE][35] = 30,
+ [0][1][2][1][RTW89_UKRAINE][35] = 28,
+ [0][1][2][1][RTW89_MEXICO][35] = 64,
+ [0][1][2][1][RTW89_CN][35] = 127,
+ [0][1][2][1][RTW89_QATAR][35] = 28,
+ [0][1][2][1][RTW89_FCC][37] = 68,
+ [0][1][2][1][RTW89_ETSI][37] = 127,
+ [0][1][2][1][RTW89_MKK][37] = 70,
+ [0][1][2][1][RTW89_IC][37] = 62,
+ [0][1][2][1][RTW89_KCC][37] = 66,
+ [0][1][2][1][RTW89_ACMA][37] = 68,
+ [0][1][2][1][RTW89_CHILE][37] = 30,
+ [0][1][2][1][RTW89_UKRAINE][37] = 127,
+ [0][1][2][1][RTW89_MEXICO][37] = 68,
+ [0][1][2][1][RTW89_CN][37] = 127,
+ [0][1][2][1][RTW89_QATAR][37] = 127,
+ [0][1][2][1][RTW89_FCC][38] = 76,
+ [0][1][2][1][RTW89_ETSI][38] = 4,
+ [0][1][2][1][RTW89_MKK][38] = 127,
+ [0][1][2][1][RTW89_IC][38] = 76,
+ [0][1][2][1][RTW89_KCC][38] = 66,
+ [0][1][2][1][RTW89_ACMA][38] = 76,
+ [0][1][2][1][RTW89_CHILE][38] = 30,
+ [0][1][2][1][RTW89_UKRAINE][38] = 4,
+ [0][1][2][1][RTW89_MEXICO][38] = 76,
+ [0][1][2][1][RTW89_CN][38] = 68,
+ [0][1][2][1][RTW89_QATAR][38] = 4,
+ [0][1][2][1][RTW89_FCC][40] = 76,
+ [0][1][2][1][RTW89_ETSI][40] = 4,
+ [0][1][2][1][RTW89_MKK][40] = 127,
+ [0][1][2][1][RTW89_IC][40] = 76,
+ [0][1][2][1][RTW89_KCC][40] = 66,
+ [0][1][2][1][RTW89_ACMA][40] = 76,
+ [0][1][2][1][RTW89_CHILE][40] = 30,
+ [0][1][2][1][RTW89_UKRAINE][40] = 4,
+ [0][1][2][1][RTW89_MEXICO][40] = 76,
+ [0][1][2][1][RTW89_CN][40] = 70,
+ [0][1][2][1][RTW89_QATAR][40] = 4,
+ [0][1][2][1][RTW89_FCC][42] = 76,
+ [0][1][2][1][RTW89_ETSI][42] = 4,
+ [0][1][2][1][RTW89_MKK][42] = 127,
+ [0][1][2][1][RTW89_IC][42] = 76,
+ [0][1][2][1][RTW89_KCC][42] = 66,
+ [0][1][2][1][RTW89_ACMA][42] = 76,
+ [0][1][2][1][RTW89_CHILE][42] = 30,
+ [0][1][2][1][RTW89_UKRAINE][42] = 4,
+ [0][1][2][1][RTW89_MEXICO][42] = 76,
+ [0][1][2][1][RTW89_CN][42] = 70,
+ [0][1][2][1][RTW89_QATAR][42] = 4,
+ [0][1][2][1][RTW89_FCC][44] = 76,
+ [0][1][2][1][RTW89_ETSI][44] = 4,
+ [0][1][2][1][RTW89_MKK][44] = 127,
+ [0][1][2][1][RTW89_IC][44] = 76,
+ [0][1][2][1][RTW89_KCC][44] = 66,
+ [0][1][2][1][RTW89_ACMA][44] = 76,
+ [0][1][2][1][RTW89_CHILE][44] = 30,
+ [0][1][2][1][RTW89_UKRAINE][44] = 4,
+ [0][1][2][1][RTW89_MEXICO][44] = 76,
+ [0][1][2][1][RTW89_CN][44] = 70,
+ [0][1][2][1][RTW89_QATAR][44] = 4,
+ [0][1][2][1][RTW89_FCC][46] = 76,
+ [0][1][2][1][RTW89_ETSI][46] = 4,
+ [0][1][2][1][RTW89_MKK][46] = 127,
+ [0][1][2][1][RTW89_IC][46] = 76,
+ [0][1][2][1][RTW89_KCC][46] = 66,
+ [0][1][2][1][RTW89_ACMA][46] = 76,
+ [0][1][2][1][RTW89_CHILE][46] = 30,
+ [0][1][2][1][RTW89_UKRAINE][46] = 4,
+ [0][1][2][1][RTW89_MEXICO][46] = 76,
+ [0][1][2][1][RTW89_CN][46] = 70,
+ [0][1][2][1][RTW89_QATAR][46] = 4,
+ [1][0][2][0][RTW89_FCC][1] = 68,
+ [1][0][2][0][RTW89_ETSI][1] = 64,
+ [1][0][2][0][RTW89_MKK][1] = 62,
+ [1][0][2][0][RTW89_IC][1] = 64,
+ [1][0][2][0][RTW89_KCC][1] = 72,
+ [1][0][2][0][RTW89_ACMA][1] = 64,
+ [1][0][2][0][RTW89_CHILE][1] = 30,
+ [1][0][2][0][RTW89_UKRAINE][1] = 52,
+ [1][0][2][0][RTW89_MEXICO][1] = 62,
+ [1][0][2][0][RTW89_CN][1] = 64,
+ [1][0][2][0][RTW89_QATAR][1] = 64,
+ [1][0][2][0][RTW89_FCC][5] = 72,
+ [1][0][2][0][RTW89_ETSI][5] = 64,
+ [1][0][2][0][RTW89_MKK][5] = 62,
+ [1][0][2][0][RTW89_IC][5] = 64,
+ [1][0][2][0][RTW89_KCC][5] = 72,
+ [1][0][2][0][RTW89_ACMA][5] = 64,
+ [1][0][2][0][RTW89_CHILE][5] = 30,
+ [1][0][2][0][RTW89_UKRAINE][5] = 52,
+ [1][0][2][0][RTW89_MEXICO][5] = 62,
+ [1][0][2][0][RTW89_CN][5] = 64,
+ [1][0][2][0][RTW89_QATAR][5] = 64,
+ [1][0][2][0][RTW89_FCC][9] = 72,
+ [1][0][2][0][RTW89_ETSI][9] = 64,
+ [1][0][2][0][RTW89_MKK][9] = 62,
+ [1][0][2][0][RTW89_IC][9] = 64,
+ [1][0][2][0][RTW89_KCC][9] = 72,
+ [1][0][2][0][RTW89_ACMA][9] = 64,
+ [1][0][2][0][RTW89_CHILE][9] = 54,
+ [1][0][2][0][RTW89_UKRAINE][9] = 52,
+ [1][0][2][0][RTW89_MEXICO][9] = 72,
+ [1][0][2][0][RTW89_CN][9] = 64,
+ [1][0][2][0][RTW89_QATAR][9] = 64,
+ [1][0][2][0][RTW89_FCC][13] = 66,
+ [1][0][2][0][RTW89_ETSI][13] = 64,
+ [1][0][2][0][RTW89_MKK][13] = 62,
+ [1][0][2][0][RTW89_IC][13] = 64,
+ [1][0][2][0][RTW89_KCC][13] = 68,
+ [1][0][2][0][RTW89_ACMA][13] = 64,
+ [1][0][2][0][RTW89_CHILE][13] = 54,
+ [1][0][2][0][RTW89_UKRAINE][13] = 52,
+ [1][0][2][0][RTW89_MEXICO][13] = 66,
+ [1][0][2][0][RTW89_CN][13] = 64,
+ [1][0][2][0][RTW89_QATAR][13] = 64,
+ [1][0][2][0][RTW89_FCC][16] = 62,
+ [1][0][2][0][RTW89_ETSI][16] = 64,
+ [1][0][2][0][RTW89_MKK][16] = 72,
+ [1][0][2][0][RTW89_IC][16] = 62,
+ [1][0][2][0][RTW89_KCC][16] = 72,
+ [1][0][2][0][RTW89_ACMA][16] = 64,
+ [1][0][2][0][RTW89_CHILE][16] = 54,
+ [1][0][2][0][RTW89_UKRAINE][16] = 52,
+ [1][0][2][0][RTW89_MEXICO][16] = 62,
+ [1][0][2][0][RTW89_CN][16] = 127,
+ [1][0][2][0][RTW89_QATAR][16] = 52,
+ [1][0][2][0][RTW89_FCC][20] = 72,
+ [1][0][2][0][RTW89_ETSI][20] = 64,
+ [1][0][2][0][RTW89_MKK][20] = 72,
+ [1][0][2][0][RTW89_IC][20] = 72,
+ [1][0][2][0][RTW89_KCC][20] = 72,
+ [1][0][2][0][RTW89_ACMA][20] = 64,
+ [1][0][2][0][RTW89_CHILE][20] = 54,
+ [1][0][2][0][RTW89_UKRAINE][20] = 52,
+ [1][0][2][0][RTW89_MEXICO][20] = 72,
+ [1][0][2][0][RTW89_CN][20] = 127,
+ [1][0][2][0][RTW89_QATAR][20] = 52,
+ [1][0][2][0][RTW89_FCC][24] = 72,
+ [1][0][2][0][RTW89_ETSI][24] = 64,
+ [1][0][2][0][RTW89_MKK][24] = 72,
+ [1][0][2][0][RTW89_IC][24] = 127,
+ [1][0][2][0][RTW89_KCC][24] = 72,
+ [1][0][2][0][RTW89_ACMA][24] = 127,
+ [1][0][2][0][RTW89_CHILE][24] = 54,
+ [1][0][2][0][RTW89_UKRAINE][24] = 52,
+ [1][0][2][0][RTW89_MEXICO][24] = 72,
+ [1][0][2][0][RTW89_CN][24] = 127,
+ [1][0][2][0][RTW89_QATAR][24] = 52,
+ [1][0][2][0][RTW89_FCC][28] = 72,
+ [1][0][2][0][RTW89_ETSI][28] = 64,
+ [1][0][2][0][RTW89_MKK][28] = 72,
+ [1][0][2][0][RTW89_IC][28] = 127,
+ [1][0][2][0][RTW89_KCC][28] = 72,
+ [1][0][2][0][RTW89_ACMA][28] = 127,
+ [1][0][2][0][RTW89_CHILE][28] = 54,
+ [1][0][2][0][RTW89_UKRAINE][28] = 52,
+ [1][0][2][0][RTW89_MEXICO][28] = 72,
+ [1][0][2][0][RTW89_CN][28] = 127,
+ [1][0][2][0][RTW89_QATAR][28] = 52,
+ [1][0][2][0][RTW89_FCC][32] = 72,
+ [1][0][2][0][RTW89_ETSI][32] = 64,
+ [1][0][2][0][RTW89_MKK][32] = 72,
+ [1][0][2][0][RTW89_IC][32] = 72,
+ [1][0][2][0][RTW89_KCC][32] = 72,
+ [1][0][2][0][RTW89_ACMA][32] = 64,
+ [1][0][2][0][RTW89_CHILE][32] = 54,
+ [1][0][2][0][RTW89_UKRAINE][32] = 52,
+ [1][0][2][0][RTW89_MEXICO][32] = 72,
+ [1][0][2][0][RTW89_CN][32] = 127,
+ [1][0][2][0][RTW89_QATAR][32] = 52,
+ [1][0][2][0][RTW89_FCC][36] = 72,
+ [1][0][2][0][RTW89_ETSI][36] = 127,
+ [1][0][2][0][RTW89_MKK][36] = 72,
+ [1][0][2][0][RTW89_IC][36] = 72,
+ [1][0][2][0][RTW89_KCC][36] = 72,
+ [1][0][2][0][RTW89_ACMA][36] = 72,
+ [1][0][2][0][RTW89_CHILE][36] = 54,
+ [1][0][2][0][RTW89_UKRAINE][36] = 127,
+ [1][0][2][0][RTW89_MEXICO][36] = 72,
+ [1][0][2][0][RTW89_CN][36] = 127,
+ [1][0][2][0][RTW89_QATAR][36] = 127,
+ [1][0][2][0][RTW89_FCC][39] = 72,
+ [1][0][2][0][RTW89_ETSI][39] = 28,
+ [1][0][2][0][RTW89_MKK][39] = 127,
+ [1][0][2][0][RTW89_IC][39] = 72,
+ [1][0][2][0][RTW89_KCC][39] = 72,
+ [1][0][2][0][RTW89_ACMA][39] = 72,
+ [1][0][2][0][RTW89_CHILE][39] = 54,
+ [1][0][2][0][RTW89_UKRAINE][39] = 28,
+ [1][0][2][0][RTW89_MEXICO][39] = 72,
+ [1][0][2][0][RTW89_CN][39] = 68,
+ [1][0][2][0][RTW89_QATAR][39] = 28,
+ [1][0][2][0][RTW89_FCC][43] = 72,
+ [1][0][2][0][RTW89_ETSI][43] = 28,
+ [1][0][2][0][RTW89_MKK][43] = 127,
+ [1][0][2][0][RTW89_IC][43] = 72,
+ [1][0][2][0][RTW89_KCC][43] = 72,
+ [1][0][2][0][RTW89_ACMA][43] = 72,
+ [1][0][2][0][RTW89_CHILE][43] = 54,
+ [1][0][2][0][RTW89_UKRAINE][43] = 28,
+ [1][0][2][0][RTW89_MEXICO][43] = 72,
+ [1][0][2][0][RTW89_CN][43] = 72,
+ [1][0][2][0][RTW89_QATAR][43] = 28,
+ [1][1][2][0][RTW89_FCC][1] = 58,
+ [1][1][2][0][RTW89_ETSI][1] = 52,
+ [1][1][2][0][RTW89_MKK][1] = 50,
+ [1][1][2][0][RTW89_IC][1] = 52,
+ [1][1][2][0][RTW89_KCC][1] = 66,
+ [1][1][2][0][RTW89_ACMA][1] = 52,
+ [1][1][2][0][RTW89_CHILE][1] = 18,
+ [1][1][2][0][RTW89_UKRAINE][1] = 40,
+ [1][1][2][0][RTW89_MEXICO][1] = 50,
+ [1][1][2][0][RTW89_CN][1] = 52,
+ [1][1][2][0][RTW89_QATAR][1] = 52,
+ [1][1][2][0][RTW89_FCC][5] = 72,
+ [1][1][2][0][RTW89_ETSI][5] = 52,
+ [1][1][2][0][RTW89_MKK][5] = 50,
+ [1][1][2][0][RTW89_IC][5] = 52,
+ [1][1][2][0][RTW89_KCC][5] = 50,
+ [1][1][2][0][RTW89_ACMA][5] = 52,
+ [1][1][2][0][RTW89_CHILE][5] = 18,
+ [1][1][2][0][RTW89_UKRAINE][5] = 40,
+ [1][1][2][0][RTW89_MEXICO][5] = 50,
+ [1][1][2][0][RTW89_CN][5] = 52,
+ [1][1][2][0][RTW89_QATAR][5] = 52,
+ [1][1][2][0][RTW89_FCC][9] = 72,
+ [1][1][2][0][RTW89_ETSI][9] = 52,
+ [1][1][2][0][RTW89_MKK][9] = 50,
+ [1][1][2][0][RTW89_IC][9] = 52,
+ [1][1][2][0][RTW89_KCC][9] = 66,
+ [1][1][2][0][RTW89_ACMA][9] = 52,
+ [1][1][2][0][RTW89_CHILE][9] = 42,
+ [1][1][2][0][RTW89_UKRAINE][9] = 40,
+ [1][1][2][0][RTW89_MEXICO][9] = 72,
+ [1][1][2][0][RTW89_CN][9] = 52,
+ [1][1][2][0][RTW89_QATAR][9] = 52,
+ [1][1][2][0][RTW89_FCC][13] = 58,
+ [1][1][2][0][RTW89_ETSI][13] = 52,
+ [1][1][2][0][RTW89_MKK][13] = 50,
+ [1][1][2][0][RTW89_IC][13] = 52,
+ [1][1][2][0][RTW89_KCC][13] = 66,
+ [1][1][2][0][RTW89_ACMA][13] = 52,
+ [1][1][2][0][RTW89_CHILE][13] = 42,
+ [1][1][2][0][RTW89_UKRAINE][13] = 40,
+ [1][1][2][0][RTW89_MEXICO][13] = 58,
+ [1][1][2][0][RTW89_CN][13] = 52,
+ [1][1][2][0][RTW89_QATAR][13] = 52,
+ [1][1][2][0][RTW89_FCC][16] = 56,
+ [1][1][2][0][RTW89_ETSI][16] = 52,
+ [1][1][2][0][RTW89_MKK][16] = 72,
+ [1][1][2][0][RTW89_IC][16] = 56,
+ [1][1][2][0][RTW89_KCC][16] = 64,
+ [1][1][2][0][RTW89_ACMA][16] = 52,
+ [1][1][2][0][RTW89_CHILE][16] = 42,
+ [1][1][2][0][RTW89_UKRAINE][16] = 40,
+ [1][1][2][0][RTW89_MEXICO][16] = 56,
+ [1][1][2][0][RTW89_CN][16] = 127,
+ [1][1][2][0][RTW89_QATAR][16] = 40,
+ [1][1][2][0][RTW89_FCC][20] = 72,
+ [1][1][2][0][RTW89_ETSI][20] = 52,
+ [1][1][2][0][RTW89_MKK][20] = 72,
+ [1][1][2][0][RTW89_IC][20] = 72,
+ [1][1][2][0][RTW89_KCC][20] = 66,
+ [1][1][2][0][RTW89_ACMA][20] = 52,
+ [1][1][2][0][RTW89_CHILE][20] = 42,
+ [1][1][2][0][RTW89_UKRAINE][20] = 40,
+ [1][1][2][0][RTW89_MEXICO][20] = 72,
+ [1][1][2][0][RTW89_CN][20] = 127,
+ [1][1][2][0][RTW89_QATAR][20] = 40,
+ [1][1][2][0][RTW89_FCC][24] = 72,
+ [1][1][2][0][RTW89_ETSI][24] = 52,
+ [1][1][2][0][RTW89_MKK][24] = 72,
+ [1][1][2][0][RTW89_IC][24] = 127,
+ [1][1][2][0][RTW89_KCC][24] = 66,
+ [1][1][2][0][RTW89_ACMA][24] = 127,
+ [1][1][2][0][RTW89_CHILE][24] = 42,
+ [1][1][2][0][RTW89_UKRAINE][24] = 40,
+ [1][1][2][0][RTW89_MEXICO][24] = 72,
+ [1][1][2][0][RTW89_CN][24] = 127,
+ [1][1][2][0][RTW89_QATAR][24] = 40,
+ [1][1][2][0][RTW89_FCC][28] = 72,
+ [1][1][2][0][RTW89_ETSI][28] = 52,
+ [1][1][2][0][RTW89_MKK][28] = 72,
+ [1][1][2][0][RTW89_IC][28] = 127,
+ [1][1][2][0][RTW89_KCC][28] = 66,
+ [1][1][2][0][RTW89_ACMA][28] = 127,
+ [1][1][2][0][RTW89_CHILE][28] = 42,
+ [1][1][2][0][RTW89_UKRAINE][28] = 40,
+ [1][1][2][0][RTW89_MEXICO][28] = 72,
+ [1][1][2][0][RTW89_CN][28] = 127,
+ [1][1][2][0][RTW89_QATAR][28] = 40,
+ [1][1][2][0][RTW89_FCC][32] = 68,
+ [1][1][2][0][RTW89_ETSI][32] = 52,
+ [1][1][2][0][RTW89_MKK][32] = 72,
+ [1][1][2][0][RTW89_IC][32] = 68,
+ [1][1][2][0][RTW89_KCC][32] = 66,
+ [1][1][2][0][RTW89_ACMA][32] = 52,
+ [1][1][2][0][RTW89_CHILE][32] = 42,
+ [1][1][2][0][RTW89_UKRAINE][32] = 40,
+ [1][1][2][0][RTW89_MEXICO][32] = 68,
+ [1][1][2][0][RTW89_CN][32] = 127,
+ [1][1][2][0][RTW89_QATAR][32] = 40,
+ [1][1][2][0][RTW89_FCC][36] = 72,
+ [1][1][2][0][RTW89_ETSI][36] = 127,
+ [1][1][2][0][RTW89_MKK][36] = 72,
+ [1][1][2][0][RTW89_IC][36] = 72,
+ [1][1][2][0][RTW89_KCC][36] = 66,
+ [1][1][2][0][RTW89_ACMA][36] = 72,
+ [1][1][2][0][RTW89_CHILE][36] = 42,
+ [1][1][2][0][RTW89_UKRAINE][36] = 127,
+ [1][1][2][0][RTW89_MEXICO][36] = 72,
+ [1][1][2][0][RTW89_CN][36] = 127,
+ [1][1][2][0][RTW89_QATAR][36] = 127,
+ [1][1][2][0][RTW89_FCC][39] = 72,
+ [1][1][2][0][RTW89_ETSI][39] = 16,
+ [1][1][2][0][RTW89_MKK][39] = 127,
+ [1][1][2][0][RTW89_IC][39] = 72,
+ [1][1][2][0][RTW89_KCC][39] = 66,
+ [1][1][2][0][RTW89_ACMA][39] = 72,
+ [1][1][2][0][RTW89_CHILE][39] = 42,
+ [1][1][2][0][RTW89_UKRAINE][39] = 16,
+ [1][1][2][0][RTW89_MEXICO][39] = 72,
+ [1][1][2][0][RTW89_CN][39] = 68,
+ [1][1][2][0][RTW89_QATAR][39] = 16,
+ [1][1][2][0][RTW89_FCC][43] = 72,
+ [1][1][2][0][RTW89_ETSI][43] = 16,
+ [1][1][2][0][RTW89_MKK][43] = 127,
+ [1][1][2][0][RTW89_IC][43] = 72,
+ [1][1][2][0][RTW89_KCC][43] = 66,
+ [1][1][2][0][RTW89_ACMA][43] = 72,
+ [1][1][2][0][RTW89_CHILE][43] = 42,
+ [1][1][2][0][RTW89_UKRAINE][43] = 16,
+ [1][1][2][0][RTW89_MEXICO][43] = 72,
+ [1][1][2][0][RTW89_CN][43] = 72,
+ [1][1][2][0][RTW89_QATAR][43] = 16,
+ [1][1][2][1][RTW89_FCC][1] = 58,
+ [1][1][2][1][RTW89_ETSI][1] = 40,
+ [1][1][2][1][RTW89_MKK][1] = 50,
+ [1][1][2][1][RTW89_IC][1] = 40,
+ [1][1][2][1][RTW89_KCC][1] = 66,
+ [1][1][2][1][RTW89_ACMA][1] = 40,
+ [1][1][2][1][RTW89_CHILE][1] = 6,
+ [1][1][2][1][RTW89_UKRAINE][1] = 28,
+ [1][1][2][1][RTW89_MEXICO][1] = 50,
+ [1][1][2][1][RTW89_CN][1] = 40,
+ [1][1][2][1][RTW89_QATAR][1] = 40,
+ [1][1][2][1][RTW89_FCC][5] = 68,
+ [1][1][2][1][RTW89_ETSI][5] = 40,
+ [1][1][2][1][RTW89_MKK][5] = 50,
+ [1][1][2][1][RTW89_IC][5] = 40,
+ [1][1][2][1][RTW89_KCC][5] = 50,
+ [1][1][2][1][RTW89_ACMA][5] = 40,
+ [1][1][2][1][RTW89_CHILE][5] = 6,
+ [1][1][2][1][RTW89_UKRAINE][5] = 28,
+ [1][1][2][1][RTW89_MEXICO][5] = 50,
+ [1][1][2][1][RTW89_CN][5] = 40,
+ [1][1][2][1][RTW89_QATAR][5] = 40,
+ [1][1][2][1][RTW89_FCC][9] = 68,
+ [1][1][2][1][RTW89_ETSI][9] = 40,
+ [1][1][2][1][RTW89_MKK][9] = 50,
+ [1][1][2][1][RTW89_IC][9] = 40,
+ [1][1][2][1][RTW89_KCC][9] = 66,
+ [1][1][2][1][RTW89_ACMA][9] = 40,
+ [1][1][2][1][RTW89_CHILE][9] = 30,
+ [1][1][2][1][RTW89_UKRAINE][9] = 28,
+ [1][1][2][1][RTW89_MEXICO][9] = 68,
+ [1][1][2][1][RTW89_CN][9] = 40,
+ [1][1][2][1][RTW89_QATAR][9] = 40,
+ [1][1][2][1][RTW89_FCC][13] = 58,
+ [1][1][2][1][RTW89_ETSI][13] = 40,
+ [1][1][2][1][RTW89_MKK][13] = 50,
+ [1][1][2][1][RTW89_IC][13] = 40,
+ [1][1][2][1][RTW89_KCC][13] = 66,
+ [1][1][2][1][RTW89_ACMA][13] = 40,
+ [1][1][2][1][RTW89_CHILE][13] = 30,
+ [1][1][2][1][RTW89_UKRAINE][13] = 28,
+ [1][1][2][1][RTW89_MEXICO][13] = 58,
+ [1][1][2][1][RTW89_CN][13] = 40,
+ [1][1][2][1][RTW89_QATAR][13] = 40,
+ [1][1][2][1][RTW89_FCC][16] = 56,
+ [1][1][2][1][RTW89_ETSI][16] = 40,
+ [1][1][2][1][RTW89_MKK][16] = 72,
+ [1][1][2][1][RTW89_IC][16] = 56,
+ [1][1][2][1][RTW89_KCC][16] = 64,
+ [1][1][2][1][RTW89_ACMA][16] = 40,
+ [1][1][2][1][RTW89_CHILE][16] = 30,
+ [1][1][2][1][RTW89_UKRAINE][16] = 28,
+ [1][1][2][1][RTW89_MEXICO][16] = 56,
+ [1][1][2][1][RTW89_CN][16] = 127,
+ [1][1][2][1][RTW89_QATAR][16] = 28,
+ [1][1][2][1][RTW89_FCC][20] = 68,
+ [1][1][2][1][RTW89_ETSI][20] = 40,
+ [1][1][2][1][RTW89_MKK][20] = 72,
+ [1][1][2][1][RTW89_IC][20] = 68,
+ [1][1][2][1][RTW89_KCC][20] = 66,
+ [1][1][2][1][RTW89_ACMA][20] = 40,
+ [1][1][2][1][RTW89_CHILE][20] = 30,
+ [1][1][2][1][RTW89_UKRAINE][20] = 28,
+ [1][1][2][1][RTW89_MEXICO][20] = 68,
+ [1][1][2][1][RTW89_CN][20] = 127,
+ [1][1][2][1][RTW89_QATAR][20] = 28,
+ [1][1][2][1][RTW89_FCC][24] = 68,
+ [1][1][2][1][RTW89_ETSI][24] = 40,
+ [1][1][2][1][RTW89_MKK][24] = 72,
+ [1][1][2][1][RTW89_IC][24] = 127,
+ [1][1][2][1][RTW89_KCC][24] = 66,
+ [1][1][2][1][RTW89_ACMA][24] = 127,
+ [1][1][2][1][RTW89_CHILE][24] = 30,
+ [1][1][2][1][RTW89_UKRAINE][24] = 28,
+ [1][1][2][1][RTW89_MEXICO][24] = 68,
+ [1][1][2][1][RTW89_CN][24] = 127,
+ [1][1][2][1][RTW89_QATAR][24] = 28,
+ [1][1][2][1][RTW89_FCC][28] = 68,
+ [1][1][2][1][RTW89_ETSI][28] = 40,
+ [1][1][2][1][RTW89_MKK][28] = 72,
+ [1][1][2][1][RTW89_IC][28] = 127,
+ [1][1][2][1][RTW89_KCC][28] = 66,
+ [1][1][2][1][RTW89_ACMA][28] = 127,
+ [1][1][2][1][RTW89_CHILE][28] = 30,
+ [1][1][2][1][RTW89_UKRAINE][28] = 28,
+ [1][1][2][1][RTW89_MEXICO][28] = 68,
+ [1][1][2][1][RTW89_CN][28] = 127,
+ [1][1][2][1][RTW89_QATAR][28] = 28,
+ [1][1][2][1][RTW89_FCC][32] = 68,
+ [1][1][2][1][RTW89_ETSI][32] = 40,
+ [1][1][2][1][RTW89_MKK][32] = 72,
+ [1][1][2][1][RTW89_IC][32] = 68,
+ [1][1][2][1][RTW89_KCC][32] = 66,
+ [1][1][2][1][RTW89_ACMA][32] = 40,
+ [1][1][2][1][RTW89_CHILE][32] = 30,
+ [1][1][2][1][RTW89_UKRAINE][32] = 28,
+ [1][1][2][1][RTW89_MEXICO][32] = 68,
+ [1][1][2][1][RTW89_CN][32] = 127,
+ [1][1][2][1][RTW89_QATAR][32] = 28,
+ [1][1][2][1][RTW89_FCC][36] = 68,
+ [1][1][2][1][RTW89_ETSI][36] = 127,
+ [1][1][2][1][RTW89_MKK][36] = 72,
+ [1][1][2][1][RTW89_IC][36] = 68,
+ [1][1][2][1][RTW89_KCC][36] = 66,
+ [1][1][2][1][RTW89_ACMA][36] = 68,
+ [1][1][2][1][RTW89_CHILE][36] = 30,
+ [1][1][2][1][RTW89_UKRAINE][36] = 127,
+ [1][1][2][1][RTW89_MEXICO][36] = 68,
+ [1][1][2][1][RTW89_CN][36] = 127,
+ [1][1][2][1][RTW89_QATAR][36] = 127,
+ [1][1][2][1][RTW89_FCC][39] = 72,
+ [1][1][2][1][RTW89_ETSI][39] = 4,
+ [1][1][2][1][RTW89_MKK][39] = 127,
+ [1][1][2][1][RTW89_IC][39] = 72,
+ [1][1][2][1][RTW89_KCC][39] = 66,
+ [1][1][2][1][RTW89_ACMA][39] = 72,
+ [1][1][2][1][RTW89_CHILE][39] = 30,
+ [1][1][2][1][RTW89_UKRAINE][39] = 4,
+ [1][1][2][1][RTW89_MEXICO][39] = 72,
+ [1][1][2][1][RTW89_CN][39] = 62,
+ [1][1][2][1][RTW89_QATAR][39] = 4,
+ [1][1][2][1][RTW89_FCC][43] = 72,
+ [1][1][2][1][RTW89_ETSI][43] = 4,
+ [1][1][2][1][RTW89_MKK][43] = 127,
+ [1][1][2][1][RTW89_IC][43] = 72,
+ [1][1][2][1][RTW89_KCC][43] = 66,
+ [1][1][2][1][RTW89_ACMA][43] = 72,
+ [1][1][2][1][RTW89_CHILE][43] = 30,
+ [1][1][2][1][RTW89_UKRAINE][43] = 4,
+ [1][1][2][1][RTW89_MEXICO][43] = 72,
+ [1][1][2][1][RTW89_CN][43] = 72,
+ [1][1][2][1][RTW89_QATAR][43] = 4,
+ [2][0][2][0][RTW89_FCC][3] = 64,
+ [2][0][2][0][RTW89_ETSI][3] = 64,
+ [2][0][2][0][RTW89_MKK][3] = 64,
+ [2][0][2][0][RTW89_IC][3] = 62,
+ [2][0][2][0][RTW89_KCC][3] = 72,
+ [2][0][2][0][RTW89_ACMA][3] = 64,
+ [2][0][2][0][RTW89_CHILE][3] = 30,
+ [2][0][2][0][RTW89_UKRAINE][3] = 52,
+ [2][0][2][0][RTW89_MEXICO][3] = 62,
+ [2][0][2][0][RTW89_CN][3] = 64,
+ [2][0][2][0][RTW89_QATAR][3] = 64,
+ [2][0][2][0][RTW89_FCC][11] = 64,
+ [2][0][2][0][RTW89_ETSI][11] = 64,
+ [2][0][2][0][RTW89_MKK][11] = 64,
+ [2][0][2][0][RTW89_IC][11] = 62,
+ [2][0][2][0][RTW89_KCC][11] = 72,
+ [2][0][2][0][RTW89_ACMA][11] = 64,
+ [2][0][2][0][RTW89_CHILE][11] = 54,
+ [2][0][2][0][RTW89_UKRAINE][11] = 52,
+ [2][0][2][0][RTW89_MEXICO][11] = 64,
+ [2][0][2][0][RTW89_CN][11] = 64,
+ [2][0][2][0][RTW89_QATAR][11] = 64,
+ [2][0][2][0][RTW89_FCC][18] = 62,
+ [2][0][2][0][RTW89_ETSI][18] = 64,
+ [2][0][2][0][RTW89_MKK][18] = 72,
+ [2][0][2][0][RTW89_IC][18] = 66,
+ [2][0][2][0][RTW89_KCC][18] = 70,
+ [2][0][2][0][RTW89_ACMA][18] = 64,
+ [2][0][2][0][RTW89_CHILE][18] = 54,
+ [2][0][2][0][RTW89_UKRAINE][18] = 52,
+ [2][0][2][0][RTW89_MEXICO][18] = 62,
+ [2][0][2][0][RTW89_CN][18] = 127,
+ [2][0][2][0][RTW89_QATAR][18] = 52,
+ [2][0][2][0][RTW89_FCC][26] = 72,
+ [2][0][2][0][RTW89_ETSI][26] = 64,
+ [2][0][2][0][RTW89_MKK][26] = 72,
+ [2][0][2][0][RTW89_IC][26] = 127,
+ [2][0][2][0][RTW89_KCC][26] = 72,
+ [2][0][2][0][RTW89_ACMA][26] = 127,
+ [2][0][2][0][RTW89_CHILE][26] = 54,
+ [2][0][2][0][RTW89_UKRAINE][26] = 52,
+ [2][0][2][0][RTW89_MEXICO][26] = 72,
+ [2][0][2][0][RTW89_CN][26] = 127,
+ [2][0][2][0][RTW89_QATAR][26] = 52,
+ [2][0][2][0][RTW89_FCC][34] = 72,
+ [2][0][2][0][RTW89_ETSI][34] = 127,
+ [2][0][2][0][RTW89_MKK][34] = 72,
+ [2][0][2][0][RTW89_IC][34] = 72,
+ [2][0][2][0][RTW89_KCC][34] = 72,
+ [2][0][2][0][RTW89_ACMA][34] = 72,
+ [2][0][2][0][RTW89_CHILE][34] = 54,
+ [2][0][2][0][RTW89_UKRAINE][34] = 127,
+ [2][0][2][0][RTW89_MEXICO][34] = 72,
+ [2][0][2][0][RTW89_CN][34] = 127,
+ [2][0][2][0][RTW89_QATAR][34] = 127,
+ [2][0][2][0][RTW89_FCC][41] = 72,
+ [2][0][2][0][RTW89_ETSI][41] = 28,
+ [2][0][2][0][RTW89_MKK][41] = 127,
+ [2][0][2][0][RTW89_IC][41] = 72,
+ [2][0][2][0][RTW89_KCC][41] = 68,
+ [2][0][2][0][RTW89_ACMA][41] = 72,
+ [2][0][2][0][RTW89_CHILE][41] = 54,
+ [2][0][2][0][RTW89_UKRAINE][41] = 28,
+ [2][0][2][0][RTW89_MEXICO][41] = 72,
+ [2][0][2][0][RTW89_CN][41] = 68,
+ [2][0][2][0][RTW89_QATAR][41] = 28,
+ [2][1][2][0][RTW89_FCC][3] = 56,
+ [2][1][2][0][RTW89_ETSI][3] = 52,
+ [2][1][2][0][RTW89_MKK][3] = 52,
+ [2][1][2][0][RTW89_IC][3] = 52,
+ [2][1][2][0][RTW89_KCC][3] = 66,
+ [2][1][2][0][RTW89_ACMA][3] = 52,
+ [2][1][2][0][RTW89_CHILE][3] = 18,
+ [2][1][2][0][RTW89_UKRAINE][3] = 40,
+ [2][1][2][0][RTW89_MEXICO][3] = 50,
+ [2][1][2][0][RTW89_CN][3] = 52,
+ [2][1][2][0][RTW89_QATAR][3] = 52,
+ [2][1][2][0][RTW89_FCC][11] = 56,
+ [2][1][2][0][RTW89_ETSI][11] = 52,
+ [2][1][2][0][RTW89_MKK][11] = 52,
+ [2][1][2][0][RTW89_IC][11] = 52,
+ [2][1][2][0][RTW89_KCC][11] = 64,
+ [2][1][2][0][RTW89_ACMA][11] = 52,
+ [2][1][2][0][RTW89_CHILE][11] = 42,
+ [2][1][2][0][RTW89_UKRAINE][11] = 40,
+ [2][1][2][0][RTW89_MEXICO][11] = 56,
+ [2][1][2][0][RTW89_CN][11] = 52,
+ [2][1][2][0][RTW89_QATAR][11] = 52,
+ [2][1][2][0][RTW89_FCC][18] = 56,
+ [2][1][2][0][RTW89_ETSI][18] = 52,
+ [2][1][2][0][RTW89_MKK][18] = 72,
+ [2][1][2][0][RTW89_IC][18] = 56,
+ [2][1][2][0][RTW89_KCC][18] = 58,
+ [2][1][2][0][RTW89_ACMA][18] = 52,
+ [2][1][2][0][RTW89_CHILE][18] = 42,
+ [2][1][2][0][RTW89_UKRAINE][18] = 40,
+ [2][1][2][0][RTW89_MEXICO][18] = 56,
+ [2][1][2][0][RTW89_CN][18] = 127,
+ [2][1][2][0][RTW89_QATAR][18] = 40,
+ [2][1][2][0][RTW89_FCC][26] = 72,
+ [2][1][2][0][RTW89_ETSI][26] = 52,
+ [2][1][2][0][RTW89_MKK][26] = 72,
+ [2][1][2][0][RTW89_IC][26] = 127,
+ [2][1][2][0][RTW89_KCC][26] = 64,
+ [2][1][2][0][RTW89_ACMA][26] = 127,
+ [2][1][2][0][RTW89_CHILE][26] = 42,
+ [2][1][2][0][RTW89_UKRAINE][26] = 40,
+ [2][1][2][0][RTW89_MEXICO][26] = 72,
+ [2][1][2][0][RTW89_CN][26] = 127,
+ [2][1][2][0][RTW89_QATAR][26] = 40,
+ [2][1][2][0][RTW89_FCC][34] = 72,
+ [2][1][2][0][RTW89_ETSI][34] = 127,
+ [2][1][2][0][RTW89_MKK][34] = 72,
+ [2][1][2][0][RTW89_IC][34] = 72,
+ [2][1][2][0][RTW89_KCC][34] = 64,
+ [2][1][2][0][RTW89_ACMA][34] = 72,
+ [2][1][2][0][RTW89_CHILE][34] = 42,
+ [2][1][2][0][RTW89_UKRAINE][34] = 127,
+ [2][1][2][0][RTW89_MEXICO][34] = 72,
+ [2][1][2][0][RTW89_CN][34] = 127,
+ [2][1][2][0][RTW89_QATAR][34] = 127,
+ [2][1][2][0][RTW89_FCC][41] = 72,
+ [2][1][2][0][RTW89_ETSI][41] = 16,
+ [2][1][2][0][RTW89_MKK][41] = 127,
+ [2][1][2][0][RTW89_IC][41] = 72,
+ [2][1][2][0][RTW89_KCC][41] = 58,
+ [2][1][2][0][RTW89_ACMA][41] = 72,
+ [2][1][2][0][RTW89_CHILE][41] = 42,
+ [2][1][2][0][RTW89_UKRAINE][41] = 16,
+ [2][1][2][0][RTW89_MEXICO][41] = 72,
+ [2][1][2][0][RTW89_CN][41] = 68,
+ [2][1][2][0][RTW89_QATAR][41] = 16,
+ [2][1][2][1][RTW89_FCC][3] = 56,
+ [2][1][2][1][RTW89_ETSI][3] = 40,
+ [2][1][2][1][RTW89_MKK][3] = 52,
+ [2][1][2][1][RTW89_IC][3] = 40,
+ [2][1][2][1][RTW89_KCC][3] = 66,
+ [2][1][2][1][RTW89_ACMA][3] = 40,
+ [2][1][2][1][RTW89_CHILE][3] = 6,
+ [2][1][2][1][RTW89_UKRAINE][3] = 28,
+ [2][1][2][1][RTW89_MEXICO][3] = 50,
+ [2][1][2][1][RTW89_CN][3] = 40,
+ [2][1][2][1][RTW89_QATAR][3] = 40,
+ [2][1][2][1][RTW89_FCC][11] = 56,
+ [2][1][2][1][RTW89_ETSI][11] = 40,
+ [2][1][2][1][RTW89_MKK][11] = 52,
+ [2][1][2][1][RTW89_IC][11] = 40,
+ [2][1][2][1][RTW89_KCC][11] = 64,
+ [2][1][2][1][RTW89_ACMA][11] = 40,
+ [2][1][2][1][RTW89_CHILE][11] = 30,
+ [2][1][2][1][RTW89_UKRAINE][11] = 28,
+ [2][1][2][1][RTW89_MEXICO][11] = 56,
+ [2][1][2][1][RTW89_CN][11] = 40,
+ [2][1][2][1][RTW89_QATAR][11] = 40,
+ [2][1][2][1][RTW89_FCC][18] = 56,
+ [2][1][2][1][RTW89_ETSI][18] = 40,
+ [2][1][2][1][RTW89_MKK][18] = 72,
+ [2][1][2][1][RTW89_IC][18] = 56,
+ [2][1][2][1][RTW89_KCC][18] = 58,
+ [2][1][2][1][RTW89_ACMA][18] = 40,
+ [2][1][2][1][RTW89_CHILE][18] = 30,
+ [2][1][2][1][RTW89_UKRAINE][18] = 28,
+ [2][1][2][1][RTW89_MEXICO][18] = 56,
+ [2][1][2][1][RTW89_CN][18] = 127,
+ [2][1][2][1][RTW89_QATAR][18] = 28,
+ [2][1][2][1][RTW89_FCC][26] = 68,
+ [2][1][2][1][RTW89_ETSI][26] = 40,
+ [2][1][2][1][RTW89_MKK][26] = 72,
+ [2][1][2][1][RTW89_IC][26] = 127,
+ [2][1][2][1][RTW89_KCC][26] = 64,
+ [2][1][2][1][RTW89_ACMA][26] = 127,
+ [2][1][2][1][RTW89_CHILE][26] = 30,
+ [2][1][2][1][RTW89_UKRAINE][26] = 28,
+ [2][1][2][1][RTW89_MEXICO][26] = 68,
+ [2][1][2][1][RTW89_CN][26] = 127,
+ [2][1][2][1][RTW89_QATAR][26] = 28,
+ [2][1][2][1][RTW89_FCC][34] = 68,
+ [2][1][2][1][RTW89_ETSI][34] = 127,
+ [2][1][2][1][RTW89_MKK][34] = 72,
+ [2][1][2][1][RTW89_IC][34] = 68,
+ [2][1][2][1][RTW89_KCC][34] = 64,
+ [2][1][2][1][RTW89_ACMA][34] = 68,
+ [2][1][2][1][RTW89_CHILE][34] = 30,
+ [2][1][2][1][RTW89_UKRAINE][34] = 127,
+ [2][1][2][1][RTW89_MEXICO][34] = 68,
+ [2][1][2][1][RTW89_CN][34] = 127,
+ [2][1][2][1][RTW89_QATAR][34] = 127,
+ [2][1][2][1][RTW89_FCC][41] = 72,
+ [2][1][2][1][RTW89_ETSI][41] = 4,
+ [2][1][2][1][RTW89_MKK][41] = 127,
+ [2][1][2][1][RTW89_IC][41] = 72,
+ [2][1][2][1][RTW89_KCC][41] = 58,
+ [2][1][2][1][RTW89_ACMA][41] = 72,
+ [2][1][2][1][RTW89_CHILE][41] = 30,
+ [2][1][2][1][RTW89_UKRAINE][41] = 4,
+ [2][1][2][1][RTW89_MEXICO][41] = 72,
+ [2][1][2][1][RTW89_CN][41] = 64,
+ [2][1][2][1][RTW89_QATAR][41] = 4,
};
const s8 rtw89_8852a_txpwr_lmt_ru_2g[RTW89_RU_NUM][RTW89_NTX_NUM]
[RTW89_REGD_NUM][RTW89_2G_CH_NUM] = {
- [0][0][0][0] = 32,
- [0][0][0][1] = 32,
- [0][0][0][2] = 32,
- [0][0][0][3] = 32,
- [0][0][0][4] = 32,
- [0][0][0][5] = 32,
- [0][0][0][6] = 32,
- [0][0][0][7] = 32,
- [0][0][0][8] = 32,
- [0][0][0][9] = 32,
- [0][0][0][10] = 32,
- [0][0][0][11] = 32,
- [0][0][0][12] = 32,
- [0][0][0][13] = 0,
- [0][1][0][0] = 20,
- [0][1][0][1] = 20,
- [0][1][0][2] = 20,
- [0][1][0][3] = 20,
- [0][1][0][4] = 20,
- [0][1][0][5] = 20,
- [0][1][0][6] = 20,
- [0][1][0][7] = 20,
- [0][1][0][8] = 20,
- [0][1][0][9] = 20,
- [0][1][0][10] = 20,
- [0][1][0][11] = 20,
- [0][1][0][12] = 20,
- [0][1][0][13] = 0,
- [1][0][0][0] = 42,
- [1][0][0][1] = 42,
- [1][0][0][2] = 42,
- [1][0][0][3] = 42,
- [1][0][0][4] = 42,
- [1][0][0][5] = 42,
- [1][0][0][6] = 42,
- [1][0][0][7] = 42,
- [1][0][0][8] = 42,
- [1][0][0][9] = 42,
- [1][0][0][10] = 42,
- [1][0][0][11] = 42,
- [1][0][0][12] = 36,
- [1][0][0][13] = 0,
- [1][1][0][0] = 30,
- [1][1][0][1] = 30,
- [1][1][0][2] = 30,
- [1][1][0][3] = 30,
- [1][1][0][4] = 30,
- [1][1][0][5] = 30,
- [1][1][0][6] = 30,
- [1][1][0][7] = 30,
- [1][1][0][8] = 30,
- [1][1][0][9] = 30,
- [1][1][0][10] = 30,
- [1][1][0][11] = 30,
- [1][1][0][12] = 30,
- [1][1][0][13] = 0,
- [2][0][0][0] = 52,
- [2][0][0][1] = 52,
- [2][0][0][2] = 52,
- [2][0][0][3] = 52,
- [2][0][0][4] = 52,
- [2][0][0][5] = 52,
- [2][0][0][6] = 52,
- [2][0][0][7] = 52,
- [2][0][0][8] = 52,
- [2][0][0][9] = 52,
- [2][0][0][10] = 52,
- [2][0][0][11] = 52,
- [2][0][0][12] = 40,
- [2][0][0][13] = 0,
- [2][1][0][0] = 40,
- [2][1][0][1] = 40,
- [2][1][0][2] = 40,
- [2][1][0][3] = 40,
- [2][1][0][4] = 40,
- [2][1][0][5] = 40,
- [2][1][0][6] = 40,
- [2][1][0][7] = 40,
- [2][1][0][8] = 40,
- [2][1][0][9] = 40,
- [2][1][0][10] = 40,
- [2][1][0][11] = 40,
- [2][1][0][12] = 26,
- [2][1][0][13] = 0,
- [0][0][2][0] = 70,
- [0][0][1][0] = 32,
- [0][0][3][0] = 40,
- [0][0][5][0] = 70,
- [0][0][6][0] = 32,
- [0][0][9][0] = 32,
- [0][0][8][0] = 60,
- [0][0][11][0] = 32,
- [0][0][2][1] = 70,
- [0][0][1][1] = 32,
- [0][0][3][1] = 40,
- [0][0][5][1] = 70,
- [0][0][6][1] = 32,
- [0][0][9][1] = 32,
- [0][0][8][1] = 60,
- [0][0][11][1] = 32,
- [0][0][2][2] = 74,
- [0][0][1][2] = 32,
- [0][0][3][2] = 40,
- [0][0][5][2] = 74,
- [0][0][6][2] = 32,
- [0][0][9][2] = 32,
- [0][0][8][2] = 60,
- [0][0][11][2] = 32,
- [0][0][2][3] = 78,
- [0][0][1][3] = 32,
- [0][0][3][3] = 40,
- [0][0][5][3] = 78,
- [0][0][6][3] = 32,
- [0][0][9][3] = 32,
- [0][0][8][3] = 60,
- [0][0][11][3] = 32,
- [0][0][2][4] = 78,
- [0][0][1][4] = 32,
- [0][0][3][4] = 40,
- [0][0][5][4] = 78,
- [0][0][6][4] = 32,
- [0][0][9][4] = 32,
- [0][0][8][4] = 60,
- [0][0][11][4] = 32,
- [0][0][2][5] = 78,
- [0][0][1][5] = 32,
- [0][0][3][5] = 40,
- [0][0][5][5] = 78,
- [0][0][6][5] = 32,
- [0][0][9][5] = 32,
- [0][0][8][5] = 60,
- [0][0][11][5] = 32,
- [0][0][2][6] = 78,
- [0][0][1][6] = 32,
- [0][0][3][6] = 40,
- [0][0][5][6] = 78,
- [0][0][6][6] = 32,
- [0][0][9][6] = 32,
- [0][0][8][6] = 60,
- [0][0][11][6] = 32,
- [0][0][2][7] = 78,
- [0][0][1][7] = 32,
- [0][0][3][7] = 40,
- [0][0][5][7] = 78,
- [0][0][6][7] = 32,
- [0][0][9][7] = 32,
- [0][0][8][7] = 60,
- [0][0][11][7] = 32,
- [0][0][2][8] = 74,
- [0][0][1][8] = 32,
- [0][0][3][8] = 40,
- [0][0][5][8] = 74,
- [0][0][6][8] = 32,
- [0][0][9][8] = 32,
- [0][0][8][8] = 60,
- [0][0][11][8] = 32,
- [0][0][2][9] = 70,
- [0][0][1][9] = 32,
- [0][0][3][9] = 40,
- [0][0][5][9] = 70,
- [0][0][6][9] = 32,
- [0][0][9][9] = 32,
- [0][0][8][9] = 60,
- [0][0][11][9] = 32,
- [0][0][2][10] = 70,
- [0][0][1][10] = 32,
- [0][0][3][10] = 40,
- [0][0][5][10] = 70,
- [0][0][6][10] = 32,
- [0][0][9][10] = 32,
- [0][0][8][10] = 60,
- [0][0][11][10] = 32,
- [0][0][2][11] = 58,
- [0][0][1][11] = 32,
- [0][0][3][11] = 40,
- [0][0][5][11] = 58,
- [0][0][6][11] = 32,
- [0][0][9][11] = 32,
- [0][0][8][11] = 60,
- [0][0][11][11] = 32,
- [0][0][2][12] = 34,
- [0][0][1][12] = 32,
- [0][0][3][12] = 40,
- [0][0][5][12] = 34,
- [0][0][6][12] = 32,
- [0][0][9][12] = 32,
- [0][0][8][12] = 60,
- [0][0][11][12] = 32,
- [0][0][2][13] = 127,
- [0][0][1][13] = 127,
- [0][0][3][13] = 127,
- [0][0][5][13] = 127,
- [0][0][6][13] = 127,
- [0][0][9][13] = 127,
- [0][0][8][13] = 127,
- [0][0][11][13] = 127,
- [0][1][2][0] = 64,
- [0][1][1][0] = 20,
- [0][1][3][0] = 28,
- [0][1][5][0] = 64,
- [0][1][6][0] = 20,
- [0][1][9][0] = 20,
- [0][1][8][0] = 48,
- [0][1][11][0] = 20,
- [0][1][2][1] = 64,
- [0][1][1][1] = 20,
- [0][1][3][1] = 28,
- [0][1][5][1] = 64,
- [0][1][6][1] = 20,
- [0][1][9][1] = 20,
- [0][1][8][1] = 48,
- [0][1][11][1] = 20,
- [0][1][2][2] = 68,
- [0][1][1][2] = 20,
- [0][1][3][2] = 28,
- [0][1][5][2] = 68,
- [0][1][6][2] = 20,
- [0][1][9][2] = 20,
- [0][1][8][2] = 48,
- [0][1][11][2] = 20,
- [0][1][2][3] = 72,
- [0][1][1][3] = 20,
- [0][1][3][3] = 28,
- [0][1][5][3] = 72,
- [0][1][6][3] = 20,
- [0][1][9][3] = 20,
- [0][1][8][3] = 48,
- [0][1][11][3] = 20,
- [0][1][2][4] = 76,
- [0][1][1][4] = 20,
- [0][1][3][4] = 28,
- [0][1][5][4] = 76,
- [0][1][6][4] = 20,
- [0][1][9][4] = 20,
- [0][1][8][4] = 48,
- [0][1][11][4] = 20,
- [0][1][2][5] = 78,
- [0][1][1][5] = 20,
- [0][1][3][5] = 28,
- [0][1][5][5] = 78,
- [0][1][6][5] = 20,
- [0][1][9][5] = 20,
- [0][1][8][5] = 48,
- [0][1][11][5] = 20,
- [0][1][2][6] = 76,
- [0][1][1][6] = 20,
- [0][1][3][6] = 28,
- [0][1][5][6] = 76,
- [0][1][6][6] = 20,
- [0][1][9][6] = 20,
- [0][1][8][6] = 48,
- [0][1][11][6] = 20,
- [0][1][2][7] = 72,
- [0][1][1][7] = 20,
- [0][1][3][7] = 28,
- [0][1][5][7] = 72,
- [0][1][6][7] = 20,
- [0][1][9][7] = 20,
- [0][1][8][7] = 48,
- [0][1][11][7] = 20,
- [0][1][2][8] = 68,
- [0][1][1][8] = 20,
- [0][1][3][8] = 28,
- [0][1][5][8] = 68,
- [0][1][6][8] = 20,
- [0][1][9][8] = 20,
- [0][1][8][8] = 48,
- [0][1][11][8] = 20,
- [0][1][2][9] = 64,
- [0][1][1][9] = 20,
- [0][1][3][9] = 28,
- [0][1][5][9] = 64,
- [0][1][6][9] = 20,
- [0][1][9][9] = 20,
- [0][1][8][9] = 48,
- [0][1][11][9] = 20,
- [0][1][2][10] = 64,
- [0][1][1][10] = 20,
- [0][1][3][10] = 28,
- [0][1][5][10] = 64,
- [0][1][6][10] = 20,
- [0][1][9][10] = 20,
- [0][1][8][10] = 48,
- [0][1][11][10] = 20,
- [0][1][2][11] = 54,
- [0][1][1][11] = 20,
- [0][1][3][11] = 28,
- [0][1][5][11] = 54,
- [0][1][6][11] = 20,
- [0][1][9][11] = 20,
- [0][1][8][11] = 48,
- [0][1][11][11] = 20,
- [0][1][2][12] = 32,
- [0][1][1][12] = 20,
- [0][1][3][12] = 28,
- [0][1][5][12] = 32,
- [0][1][6][12] = 20,
- [0][1][9][12] = 20,
- [0][1][8][12] = 48,
- [0][1][11][12] = 20,
- [0][1][2][13] = 127,
- [0][1][1][13] = 127,
- [0][1][3][13] = 127,
- [0][1][5][13] = 127,
- [0][1][6][13] = 127,
- [0][1][9][13] = 127,
- [0][1][8][13] = 127,
- [0][1][11][13] = 127,
- [1][0][2][0] = 72,
- [1][0][1][0] = 42,
- [1][0][3][0] = 50,
- [1][0][5][0] = 72,
- [1][0][6][0] = 42,
- [1][0][9][0] = 42,
- [1][0][8][0] = 60,
- [1][0][11][0] = 42,
- [1][0][2][1] = 72,
- [1][0][1][1] = 42,
- [1][0][3][1] = 50,
- [1][0][5][1] = 72,
- [1][0][6][1] = 42,
- [1][0][9][1] = 42,
- [1][0][8][1] = 60,
- [1][0][11][1] = 42,
- [1][0][2][2] = 76,
- [1][0][1][2] = 42,
- [1][0][3][2] = 50,
- [1][0][5][2] = 76,
- [1][0][6][2] = 42,
- [1][0][9][2] = 42,
- [1][0][8][2] = 60,
- [1][0][11][2] = 42,
- [1][0][2][3] = 78,
- [1][0][1][3] = 42,
- [1][0][3][3] = 50,
- [1][0][5][3] = 78,
- [1][0][6][3] = 42,
- [1][0][9][3] = 42,
- [1][0][8][3] = 60,
- [1][0][11][3] = 42,
- [1][0][2][4] = 78,
- [1][0][1][4] = 42,
- [1][0][3][4] = 50,
- [1][0][5][4] = 78,
- [1][0][6][4] = 42,
- [1][0][9][4] = 42,
- [1][0][8][4] = 60,
- [1][0][11][4] = 42,
- [1][0][2][5] = 78,
- [1][0][1][5] = 42,
- [1][0][3][5] = 50,
- [1][0][5][5] = 78,
- [1][0][6][5] = 42,
- [1][0][9][5] = 42,
- [1][0][8][5] = 60,
- [1][0][11][5] = 42,
- [1][0][2][6] = 78,
- [1][0][1][6] = 42,
- [1][0][3][6] = 50,
- [1][0][5][6] = 78,
- [1][0][6][6] = 42,
- [1][0][9][6] = 42,
- [1][0][8][6] = 60,
- [1][0][11][6] = 42,
- [1][0][2][7] = 78,
- [1][0][1][7] = 42,
- [1][0][3][7] = 50,
- [1][0][5][7] = 78,
- [1][0][6][7] = 42,
- [1][0][9][7] = 42,
- [1][0][8][7] = 60,
- [1][0][11][7] = 42,
- [1][0][2][8] = 78,
- [1][0][1][8] = 42,
- [1][0][3][8] = 50,
- [1][0][5][8] = 78,
- [1][0][6][8] = 42,
- [1][0][9][8] = 42,
- [1][0][8][8] = 60,
- [1][0][11][8] = 42,
- [1][0][2][9] = 74,
- [1][0][1][9] = 42,
- [1][0][3][9] = 50,
- [1][0][5][9] = 74,
- [1][0][6][9] = 42,
- [1][0][9][9] = 42,
- [1][0][8][9] = 60,
- [1][0][11][9] = 42,
- [1][0][2][10] = 74,
- [1][0][1][10] = 42,
- [1][0][3][10] = 50,
- [1][0][5][10] = 74,
- [1][0][6][10] = 42,
- [1][0][9][10] = 42,
- [1][0][8][10] = 60,
- [1][0][11][10] = 42,
- [1][0][2][11] = 64,
- [1][0][1][11] = 42,
- [1][0][3][11] = 50,
- [1][0][5][11] = 64,
- [1][0][6][11] = 42,
- [1][0][9][11] = 42,
- [1][0][8][11] = 60,
- [1][0][11][11] = 42,
- [1][0][2][12] = 36,
- [1][0][1][12] = 42,
- [1][0][3][12] = 50,
- [1][0][5][12] = 36,
- [1][0][6][12] = 42,
- [1][0][9][12] = 42,
- [1][0][8][12] = 60,
- [1][0][11][12] = 42,
- [1][0][2][13] = 127,
- [1][0][1][13] = 127,
- [1][0][3][13] = 127,
- [1][0][5][13] = 127,
- [1][0][6][13] = 127,
- [1][0][9][13] = 127,
- [1][0][8][13] = 127,
- [1][0][11][13] = 127,
- [1][1][2][0] = 66,
- [1][1][1][0] = 30,
- [1][1][3][0] = 38,
- [1][1][5][0] = 66,
- [1][1][6][0] = 30,
- [1][1][9][0] = 30,
- [1][1][8][0] = 48,
- [1][1][11][0] = 30,
- [1][1][2][1] = 66,
- [1][1][1][1] = 30,
- [1][1][3][1] = 38,
- [1][1][5][1] = 66,
- [1][1][6][1] = 30,
- [1][1][9][1] = 30,
- [1][1][8][1] = 48,
- [1][1][11][1] = 30,
- [1][1][2][2] = 70,
- [1][1][1][2] = 30,
- [1][1][3][2] = 38,
- [1][1][5][2] = 70,
- [1][1][6][2] = 30,
- [1][1][9][2] = 30,
- [1][1][8][2] = 48,
- [1][1][11][2] = 30,
- [1][1][2][3] = 74,
- [1][1][1][3] = 30,
- [1][1][3][3] = 38,
- [1][1][5][3] = 74,
- [1][1][6][3] = 30,
- [1][1][9][3] = 30,
- [1][1][8][3] = 48,
- [1][1][11][3] = 30,
- [1][1][2][4] = 78,
- [1][1][1][4] = 30,
- [1][1][3][4] = 38,
- [1][1][5][4] = 78,
- [1][1][6][4] = 30,
- [1][1][9][4] = 30,
- [1][1][8][4] = 48,
- [1][1][11][4] = 30,
- [1][1][2][5] = 78,
- [1][1][1][5] = 30,
- [1][1][3][5] = 38,
- [1][1][5][5] = 78,
- [1][1][6][5] = 30,
- [1][1][9][5] = 30,
- [1][1][8][5] = 48,
- [1][1][11][5] = 30,
- [1][1][2][6] = 78,
- [1][1][1][6] = 30,
- [1][1][3][6] = 38,
- [1][1][5][6] = 78,
- [1][1][6][6] = 30,
- [1][1][9][6] = 30,
- [1][1][8][6] = 48,
- [1][1][11][6] = 30,
- [1][1][2][7] = 74,
- [1][1][1][7] = 30,
- [1][1][3][7] = 38,
- [1][1][5][7] = 74,
- [1][1][6][7] = 30,
- [1][1][9][7] = 30,
- [1][1][8][7] = 48,
- [1][1][11][7] = 30,
- [1][1][2][8] = 70,
- [1][1][1][8] = 30,
- [1][1][3][8] = 38,
- [1][1][5][8] = 70,
- [1][1][6][8] = 30,
- [1][1][9][8] = 30,
- [1][1][8][8] = 48,
- [1][1][11][8] = 30,
- [1][1][2][9] = 66,
- [1][1][1][9] = 30,
- [1][1][3][9] = 38,
- [1][1][5][9] = 66,
- [1][1][6][9] = 30,
- [1][1][9][9] = 30,
- [1][1][8][9] = 48,
- [1][1][11][9] = 30,
- [1][1][2][10] = 66,
- [1][1][1][10] = 30,
- [1][1][3][10] = 38,
- [1][1][5][10] = 66,
- [1][1][6][10] = 30,
- [1][1][9][10] = 30,
- [1][1][8][10] = 48,
- [1][1][11][10] = 30,
- [1][1][2][11] = 60,
- [1][1][1][11] = 30,
- [1][1][3][11] = 38,
- [1][1][5][11] = 60,
- [1][1][6][11] = 30,
- [1][1][9][11] = 30,
- [1][1][8][11] = 48,
- [1][1][11][11] = 30,
- [1][1][2][12] = 32,
- [1][1][1][12] = 30,
- [1][1][3][12] = 38,
- [1][1][5][12] = 32,
- [1][1][6][12] = 30,
- [1][1][9][12] = 30,
- [1][1][8][12] = 48,
- [1][1][11][12] = 30,
- [1][1][2][13] = 127,
- [1][1][1][13] = 127,
- [1][1][3][13] = 127,
- [1][1][5][13] = 127,
- [1][1][6][13] = 127,
- [1][1][9][13] = 127,
- [1][1][8][13] = 127,
- [1][1][11][13] = 127,
- [2][0][2][0] = 76,
- [2][0][1][0] = 52,
- [2][0][3][0] = 64,
- [2][0][5][0] = 76,
- [2][0][6][0] = 52,
- [2][0][9][0] = 52,
- [2][0][8][0] = 60,
- [2][0][11][0] = 52,
- [2][0][2][1] = 76,
- [2][0][1][1] = 52,
- [2][0][3][1] = 64,
- [2][0][5][1] = 76,
- [2][0][6][1] = 52,
- [2][0][9][1] = 52,
- [2][0][8][1] = 60,
- [2][0][11][1] = 52,
- [2][0][2][2] = 78,
- [2][0][1][2] = 52,
- [2][0][3][2] = 64,
- [2][0][5][2] = 78,
- [2][0][6][2] = 52,
- [2][0][9][2] = 52,
- [2][0][8][2] = 60,
- [2][0][11][2] = 52,
- [2][0][2][3] = 78,
- [2][0][1][3] = 52,
- [2][0][3][3] = 64,
- [2][0][5][3] = 78,
- [2][0][6][3] = 52,
- [2][0][9][3] = 52,
- [2][0][8][3] = 60,
- [2][0][11][3] = 52,
- [2][0][2][4] = 78,
- [2][0][1][4] = 52,
- [2][0][3][4] = 64,
- [2][0][5][4] = 78,
- [2][0][6][4] = 52,
- [2][0][9][4] = 52,
- [2][0][8][4] = 60,
- [2][0][11][4] = 52,
- [2][0][2][5] = 78,
- [2][0][1][5] = 52,
- [2][0][3][5] = 64,
- [2][0][5][5] = 78,
- [2][0][6][5] = 52,
- [2][0][9][5] = 52,
- [2][0][8][5] = 60,
- [2][0][11][5] = 52,
- [2][0][2][6] = 78,
- [2][0][1][6] = 52,
- [2][0][3][6] = 64,
- [2][0][5][6] = 78,
- [2][0][6][6] = 52,
- [2][0][9][6] = 52,
- [2][0][8][6] = 60,
- [2][0][11][6] = 52,
- [2][0][2][7] = 78,
- [2][0][1][7] = 52,
- [2][0][3][7] = 64,
- [2][0][5][7] = 78,
- [2][0][6][7] = 52,
- [2][0][9][7] = 52,
- [2][0][8][7] = 60,
- [2][0][11][7] = 52,
- [2][0][2][8] = 78,
- [2][0][1][8] = 52,
- [2][0][3][8] = 64,
- [2][0][5][8] = 78,
- [2][0][6][8] = 52,
- [2][0][9][8] = 52,
- [2][0][8][8] = 60,
- [2][0][11][8] = 52,
- [2][0][2][9] = 76,
- [2][0][1][9] = 52,
- [2][0][3][9] = 64,
- [2][0][5][9] = 76,
- [2][0][6][9] = 52,
- [2][0][9][9] = 52,
- [2][0][8][9] = 60,
- [2][0][11][9] = 52,
- [2][0][2][10] = 76,
- [2][0][1][10] = 52,
- [2][0][3][10] = 64,
- [2][0][5][10] = 76,
- [2][0][6][10] = 52,
- [2][0][9][10] = 52,
- [2][0][8][10] = 60,
- [2][0][11][10] = 52,
- [2][0][2][11] = 68,
- [2][0][1][11] = 52,
- [2][0][3][11] = 64,
- [2][0][5][11] = 68,
- [2][0][6][11] = 52,
- [2][0][9][11] = 52,
- [2][0][8][11] = 60,
- [2][0][11][11] = 52,
- [2][0][2][12] = 40,
- [2][0][1][12] = 52,
- [2][0][3][12] = 64,
- [2][0][5][12] = 40,
- [2][0][6][12] = 52,
- [2][0][9][12] = 52,
- [2][0][8][12] = 60,
- [2][0][11][12] = 52,
- [2][0][2][13] = 127,
- [2][0][1][13] = 127,
- [2][0][3][13] = 127,
- [2][0][5][13] = 127,
- [2][0][6][13] = 127,
- [2][0][9][13] = 127,
- [2][0][8][13] = 127,
- [2][0][11][13] = 127,
- [2][1][2][0] = 68,
- [2][1][1][0] = 40,
- [2][1][3][0] = 52,
- [2][1][5][0] = 68,
- [2][1][6][0] = 40,
- [2][1][9][0] = 40,
- [2][1][8][0] = 48,
- [2][1][11][0] = 40,
- [2][1][2][1] = 68,
- [2][1][1][1] = 40,
- [2][1][3][1] = 52,
- [2][1][5][1] = 68,
- [2][1][6][1] = 40,
- [2][1][9][1] = 40,
- [2][1][8][1] = 48,
- [2][1][11][1] = 40,
- [2][1][2][2] = 72,
- [2][1][1][2] = 40,
- [2][1][3][2] = 52,
- [2][1][5][2] = 72,
- [2][1][6][2] = 40,
- [2][1][9][2] = 40,
- [2][1][8][2] = 48,
- [2][1][11][2] = 40,
- [2][1][2][3] = 76,
- [2][1][1][3] = 40,
- [2][1][3][3] = 52,
- [2][1][5][3] = 76,
- [2][1][6][3] = 40,
- [2][1][9][3] = 40,
- [2][1][8][3] = 48,
- [2][1][11][3] = 40,
- [2][1][2][4] = 78,
- [2][1][1][4] = 40,
- [2][1][3][4] = 52,
- [2][1][5][4] = 78,
- [2][1][6][4] = 40,
- [2][1][9][4] = 40,
- [2][1][8][4] = 48,
- [2][1][11][4] = 40,
- [2][1][2][5] = 78,
- [2][1][1][5] = 40,
- [2][1][3][5] = 52,
- [2][1][5][5] = 78,
- [2][1][6][5] = 40,
- [2][1][9][5] = 40,
- [2][1][8][5] = 48,
- [2][1][11][5] = 40,
- [2][1][2][6] = 78,
- [2][1][1][6] = 40,
- [2][1][3][6] = 52,
- [2][1][5][6] = 78,
- [2][1][6][6] = 40,
- [2][1][9][6] = 40,
- [2][1][8][6] = 48,
- [2][1][11][6] = 40,
- [2][1][2][7] = 78,
- [2][1][1][7] = 40,
- [2][1][3][7] = 52,
- [2][1][5][7] = 78,
- [2][1][6][7] = 40,
- [2][1][9][7] = 40,
- [2][1][8][7] = 48,
- [2][1][11][7] = 40,
- [2][1][2][8] = 74,
- [2][1][1][8] = 40,
- [2][1][3][8] = 52,
- [2][1][5][8] = 74,
- [2][1][6][8] = 40,
- [2][1][9][8] = 40,
- [2][1][8][8] = 48,
- [2][1][11][8] = 40,
- [2][1][2][9] = 70,
- [2][1][1][9] = 40,
- [2][1][3][9] = 52,
- [2][1][5][9] = 70,
- [2][1][6][9] = 40,
- [2][1][9][9] = 40,
- [2][1][8][9] = 48,
- [2][1][11][9] = 40,
- [2][1][2][10] = 70,
- [2][1][1][10] = 40,
- [2][1][3][10] = 52,
- [2][1][5][10] = 70,
- [2][1][6][10] = 40,
- [2][1][9][10] = 40,
- [2][1][8][10] = 48,
- [2][1][11][10] = 40,
- [2][1][2][11] = 48,
- [2][1][1][11] = 40,
- [2][1][3][11] = 52,
- [2][1][5][11] = 48,
- [2][1][6][11] = 40,
- [2][1][9][11] = 40,
- [2][1][8][11] = 48,
- [2][1][11][11] = 40,
- [2][1][2][12] = 26,
- [2][1][1][12] = 40,
- [2][1][3][12] = 52,
- [2][1][5][12] = 26,
- [2][1][6][12] = 40,
- [2][1][9][12] = 40,
- [2][1][8][12] = 48,
- [2][1][11][12] = 40,
- [2][1][2][13] = 127,
- [2][1][1][13] = 127,
- [2][1][3][13] = 127,
- [2][1][5][13] = 127,
- [2][1][6][13] = 127,
- [2][1][9][13] = 127,
- [2][1][8][13] = 127,
- [2][1][11][13] = 127,
+ [0][0][RTW89_WW][0] = 32,
+ [0][0][RTW89_WW][1] = 32,
+ [0][0][RTW89_WW][2] = 32,
+ [0][0][RTW89_WW][3] = 32,
+ [0][0][RTW89_WW][4] = 32,
+ [0][0][RTW89_WW][5] = 32,
+ [0][0][RTW89_WW][6] = 32,
+ [0][0][RTW89_WW][7] = 32,
+ [0][0][RTW89_WW][8] = 32,
+ [0][0][RTW89_WW][9] = 32,
+ [0][0][RTW89_WW][10] = 32,
+ [0][0][RTW89_WW][11] = 32,
+ [0][0][RTW89_WW][12] = 32,
+ [0][0][RTW89_WW][13] = 0,
+ [0][1][RTW89_WW][0] = 20,
+ [0][1][RTW89_WW][1] = 20,
+ [0][1][RTW89_WW][2] = 20,
+ [0][1][RTW89_WW][3] = 20,
+ [0][1][RTW89_WW][4] = 20,
+ [0][1][RTW89_WW][5] = 20,
+ [0][1][RTW89_WW][6] = 20,
+ [0][1][RTW89_WW][7] = 20,
+ [0][1][RTW89_WW][8] = 20,
+ [0][1][RTW89_WW][9] = 20,
+ [0][1][RTW89_WW][10] = 20,
+ [0][1][RTW89_WW][11] = 20,
+ [0][1][RTW89_WW][12] = 20,
+ [0][1][RTW89_WW][13] = 0,
+ [1][0][RTW89_WW][0] = 42,
+ [1][0][RTW89_WW][1] = 42,
+ [1][0][RTW89_WW][2] = 42,
+ [1][0][RTW89_WW][3] = 42,
+ [1][0][RTW89_WW][4] = 42,
+ [1][0][RTW89_WW][5] = 42,
+ [1][0][RTW89_WW][6] = 42,
+ [1][0][RTW89_WW][7] = 42,
+ [1][0][RTW89_WW][8] = 42,
+ [1][0][RTW89_WW][9] = 42,
+ [1][0][RTW89_WW][10] = 42,
+ [1][0][RTW89_WW][11] = 42,
+ [1][0][RTW89_WW][12] = 36,
+ [1][0][RTW89_WW][13] = 0,
+ [1][1][RTW89_WW][0] = 30,
+ [1][1][RTW89_WW][1] = 30,
+ [1][1][RTW89_WW][2] = 30,
+ [1][1][RTW89_WW][3] = 30,
+ [1][1][RTW89_WW][4] = 30,
+ [1][1][RTW89_WW][5] = 30,
+ [1][1][RTW89_WW][6] = 30,
+ [1][1][RTW89_WW][7] = 30,
+ [1][1][RTW89_WW][8] = 30,
+ [1][1][RTW89_WW][9] = 30,
+ [1][1][RTW89_WW][10] = 30,
+ [1][1][RTW89_WW][11] = 30,
+ [1][1][RTW89_WW][12] = 30,
+ [1][1][RTW89_WW][13] = 0,
+ [2][0][RTW89_WW][0] = 52,
+ [2][0][RTW89_WW][1] = 52,
+ [2][0][RTW89_WW][2] = 52,
+ [2][0][RTW89_WW][3] = 52,
+ [2][0][RTW89_WW][4] = 52,
+ [2][0][RTW89_WW][5] = 52,
+ [2][0][RTW89_WW][6] = 52,
+ [2][0][RTW89_WW][7] = 52,
+ [2][0][RTW89_WW][8] = 52,
+ [2][0][RTW89_WW][9] = 52,
+ [2][0][RTW89_WW][10] = 52,
+ [2][0][RTW89_WW][11] = 52,
+ [2][0][RTW89_WW][12] = 40,
+ [2][0][RTW89_WW][13] = 0,
+ [2][1][RTW89_WW][0] = 40,
+ [2][1][RTW89_WW][1] = 40,
+ [2][1][RTW89_WW][2] = 40,
+ [2][1][RTW89_WW][3] = 40,
+ [2][1][RTW89_WW][4] = 40,
+ [2][1][RTW89_WW][5] = 40,
+ [2][1][RTW89_WW][6] = 40,
+ [2][1][RTW89_WW][7] = 40,
+ [2][1][RTW89_WW][8] = 40,
+ [2][1][RTW89_WW][9] = 40,
+ [2][1][RTW89_WW][10] = 40,
+ [2][1][RTW89_WW][11] = 40,
+ [2][1][RTW89_WW][12] = 26,
+ [2][1][RTW89_WW][13] = 0,
+ [0][0][RTW89_FCC][0] = 70,
+ [0][0][RTW89_ETSI][0] = 32,
+ [0][0][RTW89_MKK][0] = 40,
+ [0][0][RTW89_IC][0] = 70,
+ [0][0][RTW89_KCC][0] = 46,
+ [0][0][RTW89_ACMA][0] = 32,
+ [0][0][RTW89_CHILE][0] = 60,
+ [0][0][RTW89_UKRAINE][0] = 32,
+ [0][0][RTW89_MEXICO][0] = 70,
+ [0][0][RTW89_CN][0] = 32,
+ [0][0][RTW89_QATAR][0] = 32,
+ [0][0][RTW89_FCC][1] = 70,
+ [0][0][RTW89_ETSI][1] = 32,
+ [0][0][RTW89_MKK][1] = 40,
+ [0][0][RTW89_IC][1] = 70,
+ [0][0][RTW89_KCC][1] = 46,
+ [0][0][RTW89_ACMA][1] = 32,
+ [0][0][RTW89_CHILE][1] = 60,
+ [0][0][RTW89_UKRAINE][1] = 32,
+ [0][0][RTW89_MEXICO][1] = 70,
+ [0][0][RTW89_CN][1] = 32,
+ [0][0][RTW89_QATAR][1] = 32,
+ [0][0][RTW89_FCC][2] = 74,
+ [0][0][RTW89_ETSI][2] = 32,
+ [0][0][RTW89_MKK][2] = 40,
+ [0][0][RTW89_IC][2] = 74,
+ [0][0][RTW89_KCC][2] = 46,
+ [0][0][RTW89_ACMA][2] = 32,
+ [0][0][RTW89_CHILE][2] = 60,
+ [0][0][RTW89_UKRAINE][2] = 32,
+ [0][0][RTW89_MEXICO][2] = 74,
+ [0][0][RTW89_CN][2] = 32,
+ [0][0][RTW89_QATAR][2] = 32,
+ [0][0][RTW89_FCC][3] = 78,
+ [0][0][RTW89_ETSI][3] = 32,
+ [0][0][RTW89_MKK][3] = 40,
+ [0][0][RTW89_IC][3] = 78,
+ [0][0][RTW89_KCC][3] = 46,
+ [0][0][RTW89_ACMA][3] = 32,
+ [0][0][RTW89_CHILE][3] = 60,
+ [0][0][RTW89_UKRAINE][3] = 32,
+ [0][0][RTW89_MEXICO][3] = 78,
+ [0][0][RTW89_CN][3] = 32,
+ [0][0][RTW89_QATAR][3] = 32,
+ [0][0][RTW89_FCC][4] = 78,
+ [0][0][RTW89_ETSI][4] = 32,
+ [0][0][RTW89_MKK][4] = 40,
+ [0][0][RTW89_IC][4] = 78,
+ [0][0][RTW89_KCC][4] = 46,
+ [0][0][RTW89_ACMA][4] = 32,
+ [0][0][RTW89_CHILE][4] = 60,
+ [0][0][RTW89_UKRAINE][4] = 32,
+ [0][0][RTW89_MEXICO][4] = 78,
+ [0][0][RTW89_CN][4] = 32,
+ [0][0][RTW89_QATAR][4] = 32,
+ [0][0][RTW89_FCC][5] = 78,
+ [0][0][RTW89_ETSI][5] = 32,
+ [0][0][RTW89_MKK][5] = 40,
+ [0][0][RTW89_IC][5] = 78,
+ [0][0][RTW89_KCC][5] = 46,
+ [0][0][RTW89_ACMA][5] = 32,
+ [0][0][RTW89_CHILE][5] = 60,
+ [0][0][RTW89_UKRAINE][5] = 32,
+ [0][0][RTW89_MEXICO][5] = 78,
+ [0][0][RTW89_CN][5] = 32,
+ [0][0][RTW89_QATAR][5] = 32,
+ [0][0][RTW89_FCC][6] = 78,
+ [0][0][RTW89_ETSI][6] = 32,
+ [0][0][RTW89_MKK][6] = 40,
+ [0][0][RTW89_IC][6] = 78,
+ [0][0][RTW89_KCC][6] = 46,
+ [0][0][RTW89_ACMA][6] = 32,
+ [0][0][RTW89_CHILE][6] = 60,
+ [0][0][RTW89_UKRAINE][6] = 32,
+ [0][0][RTW89_MEXICO][6] = 78,
+ [0][0][RTW89_CN][6] = 32,
+ [0][0][RTW89_QATAR][6] = 32,
+ [0][0][RTW89_FCC][7] = 78,
+ [0][0][RTW89_ETSI][7] = 32,
+ [0][0][RTW89_MKK][7] = 40,
+ [0][0][RTW89_IC][7] = 78,
+ [0][0][RTW89_KCC][7] = 46,
+ [0][0][RTW89_ACMA][7] = 32,
+ [0][0][RTW89_CHILE][7] = 60,
+ [0][0][RTW89_UKRAINE][7] = 32,
+ [0][0][RTW89_MEXICO][7] = 78,
+ [0][0][RTW89_CN][7] = 32,
+ [0][0][RTW89_QATAR][7] = 32,
+ [0][0][RTW89_FCC][8] = 74,
+ [0][0][RTW89_ETSI][8] = 32,
+ [0][0][RTW89_MKK][8] = 40,
+ [0][0][RTW89_IC][8] = 74,
+ [0][0][RTW89_KCC][8] = 46,
+ [0][0][RTW89_ACMA][8] = 32,
+ [0][0][RTW89_CHILE][8] = 60,
+ [0][0][RTW89_UKRAINE][8] = 32,
+ [0][0][RTW89_MEXICO][8] = 74,
+ [0][0][RTW89_CN][8] = 32,
+ [0][0][RTW89_QATAR][8] = 32,
+ [0][0][RTW89_FCC][9] = 70,
+ [0][0][RTW89_ETSI][9] = 32,
+ [0][0][RTW89_MKK][9] = 40,
+ [0][0][RTW89_IC][9] = 70,
+ [0][0][RTW89_KCC][9] = 46,
+ [0][0][RTW89_ACMA][9] = 32,
+ [0][0][RTW89_CHILE][9] = 60,
+ [0][0][RTW89_UKRAINE][9] = 32,
+ [0][0][RTW89_MEXICO][9] = 70,
+ [0][0][RTW89_CN][9] = 32,
+ [0][0][RTW89_QATAR][9] = 32,
+ [0][0][RTW89_FCC][10] = 70,
+ [0][0][RTW89_ETSI][10] = 32,
+ [0][0][RTW89_MKK][10] = 40,
+ [0][0][RTW89_IC][10] = 70,
+ [0][0][RTW89_KCC][10] = 46,
+ [0][0][RTW89_ACMA][10] = 32,
+ [0][0][RTW89_CHILE][10] = 60,
+ [0][0][RTW89_UKRAINE][10] = 32,
+ [0][0][RTW89_MEXICO][10] = 70,
+ [0][0][RTW89_CN][10] = 32,
+ [0][0][RTW89_QATAR][10] = 32,
+ [0][0][RTW89_FCC][11] = 58,
+ [0][0][RTW89_ETSI][11] = 32,
+ [0][0][RTW89_MKK][11] = 40,
+ [0][0][RTW89_IC][11] = 58,
+ [0][0][RTW89_KCC][11] = 46,
+ [0][0][RTW89_ACMA][11] = 32,
+ [0][0][RTW89_CHILE][11] = 58,
+ [0][0][RTW89_UKRAINE][11] = 32,
+ [0][0][RTW89_MEXICO][11] = 58,
+ [0][0][RTW89_CN][11] = 32,
+ [0][0][RTW89_QATAR][11] = 32,
+ [0][0][RTW89_FCC][12] = 34,
+ [0][0][RTW89_ETSI][12] = 32,
+ [0][0][RTW89_MKK][12] = 40,
+ [0][0][RTW89_IC][12] = 34,
+ [0][0][RTW89_KCC][12] = 46,
+ [0][0][RTW89_ACMA][12] = 32,
+ [0][0][RTW89_CHILE][12] = 34,
+ [0][0][RTW89_UKRAINE][12] = 32,
+ [0][0][RTW89_MEXICO][12] = 34,
+ [0][0][RTW89_CN][12] = 32,
+ [0][0][RTW89_QATAR][12] = 32,
+ [0][0][RTW89_FCC][13] = 127,
+ [0][0][RTW89_ETSI][13] = 127,
+ [0][0][RTW89_MKK][13] = 127,
+ [0][0][RTW89_IC][13] = 127,
+ [0][0][RTW89_KCC][13] = 127,
+ [0][0][RTW89_ACMA][13] = 127,
+ [0][0][RTW89_CHILE][13] = 127,
+ [0][0][RTW89_UKRAINE][13] = 127,
+ [0][0][RTW89_MEXICO][13] = 127,
+ [0][0][RTW89_CN][13] = 127,
+ [0][0][RTW89_QATAR][13] = 127,
+ [0][1][RTW89_FCC][0] = 64,
+ [0][1][RTW89_ETSI][0] = 20,
+ [0][1][RTW89_MKK][0] = 28,
+ [0][1][RTW89_IC][0] = 64,
+ [0][1][RTW89_KCC][0] = 32,
+ [0][1][RTW89_ACMA][0] = 20,
+ [0][1][RTW89_CHILE][0] = 48,
+ [0][1][RTW89_UKRAINE][0] = 20,
+ [0][1][RTW89_MEXICO][0] = 64,
+ [0][1][RTW89_CN][0] = 20,
+ [0][1][RTW89_QATAR][0] = 20,
+ [0][1][RTW89_FCC][1] = 64,
+ [0][1][RTW89_ETSI][1] = 20,
+ [0][1][RTW89_MKK][1] = 28,
+ [0][1][RTW89_IC][1] = 64,
+ [0][1][RTW89_KCC][1] = 32,
+ [0][1][RTW89_ACMA][1] = 20,
+ [0][1][RTW89_CHILE][1] = 48,
+ [0][1][RTW89_UKRAINE][1] = 20,
+ [0][1][RTW89_MEXICO][1] = 64,
+ [0][1][RTW89_CN][1] = 20,
+ [0][1][RTW89_QATAR][1] = 20,
+ [0][1][RTW89_FCC][2] = 68,
+ [0][1][RTW89_ETSI][2] = 20,
+ [0][1][RTW89_MKK][2] = 28,
+ [0][1][RTW89_IC][2] = 68,
+ [0][1][RTW89_KCC][2] = 32,
+ [0][1][RTW89_ACMA][2] = 20,
+ [0][1][RTW89_CHILE][2] = 48,
+ [0][1][RTW89_UKRAINE][2] = 20,
+ [0][1][RTW89_MEXICO][2] = 68,
+ [0][1][RTW89_CN][2] = 20,
+ [0][1][RTW89_QATAR][2] = 20,
+ [0][1][RTW89_FCC][3] = 72,
+ [0][1][RTW89_ETSI][3] = 20,
+ [0][1][RTW89_MKK][3] = 28,
+ [0][1][RTW89_IC][3] = 72,
+ [0][1][RTW89_KCC][3] = 32,
+ [0][1][RTW89_ACMA][3] = 20,
+ [0][1][RTW89_CHILE][3] = 48,
+ [0][1][RTW89_UKRAINE][3] = 20,
+ [0][1][RTW89_MEXICO][3] = 72,
+ [0][1][RTW89_CN][3] = 20,
+ [0][1][RTW89_QATAR][3] = 20,
+ [0][1][RTW89_FCC][4] = 76,
+ [0][1][RTW89_ETSI][4] = 20,
+ [0][1][RTW89_MKK][4] = 28,
+ [0][1][RTW89_IC][4] = 76,
+ [0][1][RTW89_KCC][4] = 32,
+ [0][1][RTW89_ACMA][4] = 20,
+ [0][1][RTW89_CHILE][4] = 48,
+ [0][1][RTW89_UKRAINE][4] = 20,
+ [0][1][RTW89_MEXICO][4] = 76,
+ [0][1][RTW89_CN][4] = 20,
+ [0][1][RTW89_QATAR][4] = 20,
+ [0][1][RTW89_FCC][5] = 78,
+ [0][1][RTW89_ETSI][5] = 20,
+ [0][1][RTW89_MKK][5] = 28,
+ [0][1][RTW89_IC][5] = 78,
+ [0][1][RTW89_KCC][5] = 32,
+ [0][1][RTW89_ACMA][5] = 20,
+ [0][1][RTW89_CHILE][5] = 48,
+ [0][1][RTW89_UKRAINE][5] = 20,
+ [0][1][RTW89_MEXICO][5] = 78,
+ [0][1][RTW89_CN][5] = 20,
+ [0][1][RTW89_QATAR][5] = 20,
+ [0][1][RTW89_FCC][6] = 76,
+ [0][1][RTW89_ETSI][6] = 20,
+ [0][1][RTW89_MKK][6] = 28,
+ [0][1][RTW89_IC][6] = 76,
+ [0][1][RTW89_KCC][6] = 32,
+ [0][1][RTW89_ACMA][6] = 20,
+ [0][1][RTW89_CHILE][6] = 48,
+ [0][1][RTW89_UKRAINE][6] = 20,
+ [0][1][RTW89_MEXICO][6] = 76,
+ [0][1][RTW89_CN][6] = 20,
+ [0][1][RTW89_QATAR][6] = 20,
+ [0][1][RTW89_FCC][7] = 72,
+ [0][1][RTW89_ETSI][7] = 20,
+ [0][1][RTW89_MKK][7] = 28,
+ [0][1][RTW89_IC][7] = 72,
+ [0][1][RTW89_KCC][7] = 32,
+ [0][1][RTW89_ACMA][7] = 20,
+ [0][1][RTW89_CHILE][7] = 48,
+ [0][1][RTW89_UKRAINE][7] = 20,
+ [0][1][RTW89_MEXICO][7] = 72,
+ [0][1][RTW89_CN][7] = 20,
+ [0][1][RTW89_QATAR][7] = 20,
+ [0][1][RTW89_FCC][8] = 68,
+ [0][1][RTW89_ETSI][8] = 20,
+ [0][1][RTW89_MKK][8] = 28,
+ [0][1][RTW89_IC][8] = 68,
+ [0][1][RTW89_KCC][8] = 32,
+ [0][1][RTW89_ACMA][8] = 20,
+ [0][1][RTW89_CHILE][8] = 48,
+ [0][1][RTW89_UKRAINE][8] = 20,
+ [0][1][RTW89_MEXICO][8] = 68,
+ [0][1][RTW89_CN][8] = 20,
+ [0][1][RTW89_QATAR][8] = 20,
+ [0][1][RTW89_FCC][9] = 64,
+ [0][1][RTW89_ETSI][9] = 20,
+ [0][1][RTW89_MKK][9] = 28,
+ [0][1][RTW89_IC][9] = 64,
+ [0][1][RTW89_KCC][9] = 32,
+ [0][1][RTW89_ACMA][9] = 20,
+ [0][1][RTW89_CHILE][9] = 48,
+ [0][1][RTW89_UKRAINE][9] = 20,
+ [0][1][RTW89_MEXICO][9] = 64,
+ [0][1][RTW89_CN][9] = 20,
+ [0][1][RTW89_QATAR][9] = 20,
+ [0][1][RTW89_FCC][10] = 64,
+ [0][1][RTW89_ETSI][10] = 20,
+ [0][1][RTW89_MKK][10] = 28,
+ [0][1][RTW89_IC][10] = 64,
+ [0][1][RTW89_KCC][10] = 32,
+ [0][1][RTW89_ACMA][10] = 20,
+ [0][1][RTW89_CHILE][10] = 48,
+ [0][1][RTW89_UKRAINE][10] = 20,
+ [0][1][RTW89_MEXICO][10] = 64,
+ [0][1][RTW89_CN][10] = 20,
+ [0][1][RTW89_QATAR][10] = 20,
+ [0][1][RTW89_FCC][11] = 54,
+ [0][1][RTW89_ETSI][11] = 20,
+ [0][1][RTW89_MKK][11] = 28,
+ [0][1][RTW89_IC][11] = 54,
+ [0][1][RTW89_KCC][11] = 32,
+ [0][1][RTW89_ACMA][11] = 20,
+ [0][1][RTW89_CHILE][11] = 48,
+ [0][1][RTW89_UKRAINE][11] = 20,
+ [0][1][RTW89_MEXICO][11] = 54,
+ [0][1][RTW89_CN][11] = 20,
+ [0][1][RTW89_QATAR][11] = 20,
+ [0][1][RTW89_FCC][12] = 32,
+ [0][1][RTW89_ETSI][12] = 20,
+ [0][1][RTW89_MKK][12] = 28,
+ [0][1][RTW89_IC][12] = 32,
+ [0][1][RTW89_KCC][12] = 32,
+ [0][1][RTW89_ACMA][12] = 20,
+ [0][1][RTW89_CHILE][12] = 32,
+ [0][1][RTW89_UKRAINE][12] = 20,
+ [0][1][RTW89_MEXICO][12] = 32,
+ [0][1][RTW89_CN][12] = 20,
+ [0][1][RTW89_QATAR][12] = 20,
+ [0][1][RTW89_FCC][13] = 127,
+ [0][1][RTW89_ETSI][13] = 127,
+ [0][1][RTW89_MKK][13] = 127,
+ [0][1][RTW89_IC][13] = 127,
+ [0][1][RTW89_KCC][13] = 127,
+ [0][1][RTW89_ACMA][13] = 127,
+ [0][1][RTW89_CHILE][13] = 127,
+ [0][1][RTW89_UKRAINE][13] = 127,
+ [0][1][RTW89_MEXICO][13] = 127,
+ [0][1][RTW89_CN][13] = 127,
+ [0][1][RTW89_QATAR][13] = 127,
+ [1][0][RTW89_FCC][0] = 72,
+ [1][0][RTW89_ETSI][0] = 42,
+ [1][0][RTW89_MKK][0] = 50,
+ [1][0][RTW89_IC][0] = 72,
+ [1][0][RTW89_KCC][0] = 58,
+ [1][0][RTW89_ACMA][0] = 42,
+ [1][0][RTW89_CHILE][0] = 60,
+ [1][0][RTW89_UKRAINE][0] = 42,
+ [1][0][RTW89_MEXICO][0] = 72,
+ [1][0][RTW89_CN][0] = 42,
+ [1][0][RTW89_QATAR][0] = 42,
+ [1][0][RTW89_FCC][1] = 72,
+ [1][0][RTW89_ETSI][1] = 42,
+ [1][0][RTW89_MKK][1] = 50,
+ [1][0][RTW89_IC][1] = 72,
+ [1][0][RTW89_KCC][1] = 58,
+ [1][0][RTW89_ACMA][1] = 42,
+ [1][0][RTW89_CHILE][1] = 60,
+ [1][0][RTW89_UKRAINE][1] = 42,
+ [1][0][RTW89_MEXICO][1] = 72,
+ [1][0][RTW89_CN][1] = 42,
+ [1][0][RTW89_QATAR][1] = 42,
+ [1][0][RTW89_FCC][2] = 76,
+ [1][0][RTW89_ETSI][2] = 42,
+ [1][0][RTW89_MKK][2] = 50,
+ [1][0][RTW89_IC][2] = 76,
+ [1][0][RTW89_KCC][2] = 58,
+ [1][0][RTW89_ACMA][2] = 42,
+ [1][0][RTW89_CHILE][2] = 60,
+ [1][0][RTW89_UKRAINE][2] = 42,
+ [1][0][RTW89_MEXICO][2] = 76,
+ [1][0][RTW89_CN][2] = 42,
+ [1][0][RTW89_QATAR][2] = 42,
+ [1][0][RTW89_FCC][3] = 78,
+ [1][0][RTW89_ETSI][3] = 42,
+ [1][0][RTW89_MKK][3] = 50,
+ [1][0][RTW89_IC][3] = 78,
+ [1][0][RTW89_KCC][3] = 58,
+ [1][0][RTW89_ACMA][3] = 42,
+ [1][0][RTW89_CHILE][3] = 60,
+ [1][0][RTW89_UKRAINE][3] = 42,
+ [1][0][RTW89_MEXICO][3] = 78,
+ [1][0][RTW89_CN][3] = 42,
+ [1][0][RTW89_QATAR][3] = 42,
+ [1][0][RTW89_FCC][4] = 78,
+ [1][0][RTW89_ETSI][4] = 42,
+ [1][0][RTW89_MKK][4] = 50,
+ [1][0][RTW89_IC][4] = 78,
+ [1][0][RTW89_KCC][4] = 58,
+ [1][0][RTW89_ACMA][4] = 42,
+ [1][0][RTW89_CHILE][4] = 60,
+ [1][0][RTW89_UKRAINE][4] = 42,
+ [1][0][RTW89_MEXICO][4] = 78,
+ [1][0][RTW89_CN][4] = 42,
+ [1][0][RTW89_QATAR][4] = 42,
+ [1][0][RTW89_FCC][5] = 78,
+ [1][0][RTW89_ETSI][5] = 42,
+ [1][0][RTW89_MKK][5] = 50,
+ [1][0][RTW89_IC][5] = 78,
+ [1][0][RTW89_KCC][5] = 58,
+ [1][0][RTW89_ACMA][5] = 42,
+ [1][0][RTW89_CHILE][5] = 60,
+ [1][0][RTW89_UKRAINE][5] = 42,
+ [1][0][RTW89_MEXICO][5] = 78,
+ [1][0][RTW89_CN][5] = 42,
+ [1][0][RTW89_QATAR][5] = 42,
+ [1][0][RTW89_FCC][6] = 78,
+ [1][0][RTW89_ETSI][6] = 42,
+ [1][0][RTW89_MKK][6] = 50,
+ [1][0][RTW89_IC][6] = 78,
+ [1][0][RTW89_KCC][6] = 58,
+ [1][0][RTW89_ACMA][6] = 42,
+ [1][0][RTW89_CHILE][6] = 60,
+ [1][0][RTW89_UKRAINE][6] = 42,
+ [1][0][RTW89_MEXICO][6] = 78,
+ [1][0][RTW89_CN][6] = 42,
+ [1][0][RTW89_QATAR][6] = 42,
+ [1][0][RTW89_FCC][7] = 78,
+ [1][0][RTW89_ETSI][7] = 42,
+ [1][0][RTW89_MKK][7] = 50,
+ [1][0][RTW89_IC][7] = 78,
+ [1][0][RTW89_KCC][7] = 58,
+ [1][0][RTW89_ACMA][7] = 42,
+ [1][0][RTW89_CHILE][7] = 60,
+ [1][0][RTW89_UKRAINE][7] = 42,
+ [1][0][RTW89_MEXICO][7] = 78,
+ [1][0][RTW89_CN][7] = 42,
+ [1][0][RTW89_QATAR][7] = 42,
+ [1][0][RTW89_FCC][8] = 78,
+ [1][0][RTW89_ETSI][8] = 42,
+ [1][0][RTW89_MKK][8] = 50,
+ [1][0][RTW89_IC][8] = 78,
+ [1][0][RTW89_KCC][8] = 58,
+ [1][0][RTW89_ACMA][8] = 42,
+ [1][0][RTW89_CHILE][8] = 60,
+ [1][0][RTW89_UKRAINE][8] = 42,
+ [1][0][RTW89_MEXICO][8] = 78,
+ [1][0][RTW89_CN][8] = 42,
+ [1][0][RTW89_QATAR][8] = 42,
+ [1][0][RTW89_FCC][9] = 74,
+ [1][0][RTW89_ETSI][9] = 42,
+ [1][0][RTW89_MKK][9] = 50,
+ [1][0][RTW89_IC][9] = 74,
+ [1][0][RTW89_KCC][9] = 58,
+ [1][0][RTW89_ACMA][9] = 42,
+ [1][0][RTW89_CHILE][9] = 60,
+ [1][0][RTW89_UKRAINE][9] = 42,
+ [1][0][RTW89_MEXICO][9] = 74,
+ [1][0][RTW89_CN][9] = 42,
+ [1][0][RTW89_QATAR][9] = 42,
+ [1][0][RTW89_FCC][10] = 74,
+ [1][0][RTW89_ETSI][10] = 42,
+ [1][0][RTW89_MKK][10] = 50,
+ [1][0][RTW89_IC][10] = 74,
+ [1][0][RTW89_KCC][10] = 58,
+ [1][0][RTW89_ACMA][10] = 42,
+ [1][0][RTW89_CHILE][10] = 60,
+ [1][0][RTW89_UKRAINE][10] = 42,
+ [1][0][RTW89_MEXICO][10] = 74,
+ [1][0][RTW89_CN][10] = 42,
+ [1][0][RTW89_QATAR][10] = 42,
+ [1][0][RTW89_FCC][11] = 64,
+ [1][0][RTW89_ETSI][11] = 42,
+ [1][0][RTW89_MKK][11] = 50,
+ [1][0][RTW89_IC][11] = 64,
+ [1][0][RTW89_KCC][11] = 58,
+ [1][0][RTW89_ACMA][11] = 42,
+ [1][0][RTW89_CHILE][11] = 60,
+ [1][0][RTW89_UKRAINE][11] = 42,
+ [1][0][RTW89_MEXICO][11] = 64,
+ [1][0][RTW89_CN][11] = 42,
+ [1][0][RTW89_QATAR][11] = 42,
+ [1][0][RTW89_FCC][12] = 36,
+ [1][0][RTW89_ETSI][12] = 42,
+ [1][0][RTW89_MKK][12] = 50,
+ [1][0][RTW89_IC][12] = 36,
+ [1][0][RTW89_KCC][12] = 58,
+ [1][0][RTW89_ACMA][12] = 42,
+ [1][0][RTW89_CHILE][12] = 36,
+ [1][0][RTW89_UKRAINE][12] = 42,
+ [1][0][RTW89_MEXICO][12] = 36,
+ [1][0][RTW89_CN][12] = 42,
+ [1][0][RTW89_QATAR][12] = 42,
+ [1][0][RTW89_FCC][13] = 127,
+ [1][0][RTW89_ETSI][13] = 127,
+ [1][0][RTW89_MKK][13] = 127,
+ [1][0][RTW89_IC][13] = 127,
+ [1][0][RTW89_KCC][13] = 127,
+ [1][0][RTW89_ACMA][13] = 127,
+ [1][0][RTW89_CHILE][13] = 127,
+ [1][0][RTW89_UKRAINE][13] = 127,
+ [1][0][RTW89_MEXICO][13] = 127,
+ [1][0][RTW89_CN][13] = 127,
+ [1][0][RTW89_QATAR][13] = 127,
+ [1][1][RTW89_FCC][0] = 66,
+ [1][1][RTW89_ETSI][0] = 30,
+ [1][1][RTW89_MKK][0] = 38,
+ [1][1][RTW89_IC][0] = 66,
+ [1][1][RTW89_KCC][0] = 44,
+ [1][1][RTW89_ACMA][0] = 30,
+ [1][1][RTW89_CHILE][0] = 48,
+ [1][1][RTW89_UKRAINE][0] = 30,
+ [1][1][RTW89_MEXICO][0] = 66,
+ [1][1][RTW89_CN][0] = 30,
+ [1][1][RTW89_QATAR][0] = 30,
+ [1][1][RTW89_FCC][1] = 66,
+ [1][1][RTW89_ETSI][1] = 30,
+ [1][1][RTW89_MKK][1] = 38,
+ [1][1][RTW89_IC][1] = 66,
+ [1][1][RTW89_KCC][1] = 44,
+ [1][1][RTW89_ACMA][1] = 30,
+ [1][1][RTW89_CHILE][1] = 48,
+ [1][1][RTW89_UKRAINE][1] = 30,
+ [1][1][RTW89_MEXICO][1] = 66,
+ [1][1][RTW89_CN][1] = 30,
+ [1][1][RTW89_QATAR][1] = 30,
+ [1][1][RTW89_FCC][2] = 70,
+ [1][1][RTW89_ETSI][2] = 30,
+ [1][1][RTW89_MKK][2] = 38,
+ [1][1][RTW89_IC][2] = 70,
+ [1][1][RTW89_KCC][2] = 44,
+ [1][1][RTW89_ACMA][2] = 30,
+ [1][1][RTW89_CHILE][2] = 48,
+ [1][1][RTW89_UKRAINE][2] = 30,
+ [1][1][RTW89_MEXICO][2] = 70,
+ [1][1][RTW89_CN][2] = 30,
+ [1][1][RTW89_QATAR][2] = 30,
+ [1][1][RTW89_FCC][3] = 74,
+ [1][1][RTW89_ETSI][3] = 30,
+ [1][1][RTW89_MKK][3] = 38,
+ [1][1][RTW89_IC][3] = 74,
+ [1][1][RTW89_KCC][3] = 44,
+ [1][1][RTW89_ACMA][3] = 30,
+ [1][1][RTW89_CHILE][3] = 48,
+ [1][1][RTW89_UKRAINE][3] = 30,
+ [1][1][RTW89_MEXICO][3] = 74,
+ [1][1][RTW89_CN][3] = 30,
+ [1][1][RTW89_QATAR][3] = 30,
+ [1][1][RTW89_FCC][4] = 78,
+ [1][1][RTW89_ETSI][4] = 30,
+ [1][1][RTW89_MKK][4] = 38,
+ [1][1][RTW89_IC][4] = 78,
+ [1][1][RTW89_KCC][4] = 44,
+ [1][1][RTW89_ACMA][4] = 30,
+ [1][1][RTW89_CHILE][4] = 48,
+ [1][1][RTW89_UKRAINE][4] = 30,
+ [1][1][RTW89_MEXICO][4] = 78,
+ [1][1][RTW89_CN][4] = 30,
+ [1][1][RTW89_QATAR][4] = 30,
+ [1][1][RTW89_FCC][5] = 78,
+ [1][1][RTW89_ETSI][5] = 30,
+ [1][1][RTW89_MKK][5] = 38,
+ [1][1][RTW89_IC][5] = 78,
+ [1][1][RTW89_KCC][5] = 44,
+ [1][1][RTW89_ACMA][5] = 30,
+ [1][1][RTW89_CHILE][5] = 48,
+ [1][1][RTW89_UKRAINE][5] = 30,
+ [1][1][RTW89_MEXICO][5] = 78,
+ [1][1][RTW89_CN][5] = 30,
+ [1][1][RTW89_QATAR][5] = 30,
+ [1][1][RTW89_FCC][6] = 78,
+ [1][1][RTW89_ETSI][6] = 30,
+ [1][1][RTW89_MKK][6] = 38,
+ [1][1][RTW89_IC][6] = 78,
+ [1][1][RTW89_KCC][6] = 44,
+ [1][1][RTW89_ACMA][6] = 30,
+ [1][1][RTW89_CHILE][6] = 48,
+ [1][1][RTW89_UKRAINE][6] = 30,
+ [1][1][RTW89_MEXICO][6] = 78,
+ [1][1][RTW89_CN][6] = 30,
+ [1][1][RTW89_QATAR][6] = 30,
+ [1][1][RTW89_FCC][7] = 74,
+ [1][1][RTW89_ETSI][7] = 30,
+ [1][1][RTW89_MKK][7] = 38,
+ [1][1][RTW89_IC][7] = 74,
+ [1][1][RTW89_KCC][7] = 44,
+ [1][1][RTW89_ACMA][7] = 30,
+ [1][1][RTW89_CHILE][7] = 48,
+ [1][1][RTW89_UKRAINE][7] = 30,
+ [1][1][RTW89_MEXICO][7] = 74,
+ [1][1][RTW89_CN][7] = 30,
+ [1][1][RTW89_QATAR][7] = 30,
+ [1][1][RTW89_FCC][8] = 70,
+ [1][1][RTW89_ETSI][8] = 30,
+ [1][1][RTW89_MKK][8] = 38,
+ [1][1][RTW89_IC][8] = 70,
+ [1][1][RTW89_KCC][8] = 44,
+ [1][1][RTW89_ACMA][8] = 30,
+ [1][1][RTW89_CHILE][8] = 48,
+ [1][1][RTW89_UKRAINE][8] = 30,
+ [1][1][RTW89_MEXICO][8] = 70,
+ [1][1][RTW89_CN][8] = 30,
+ [1][1][RTW89_QATAR][8] = 30,
+ [1][1][RTW89_FCC][9] = 66,
+ [1][1][RTW89_ETSI][9] = 30,
+ [1][1][RTW89_MKK][9] = 38,
+ [1][1][RTW89_IC][9] = 66,
+ [1][1][RTW89_KCC][9] = 44,
+ [1][1][RTW89_ACMA][9] = 30,
+ [1][1][RTW89_CHILE][9] = 48,
+ [1][1][RTW89_UKRAINE][9] = 30,
+ [1][1][RTW89_MEXICO][9] = 66,
+ [1][1][RTW89_CN][9] = 30,
+ [1][1][RTW89_QATAR][9] = 30,
+ [1][1][RTW89_FCC][10] = 66,
+ [1][1][RTW89_ETSI][10] = 30,
+ [1][1][RTW89_MKK][10] = 38,
+ [1][1][RTW89_IC][10] = 66,
+ [1][1][RTW89_KCC][10] = 44,
+ [1][1][RTW89_ACMA][10] = 30,
+ [1][1][RTW89_CHILE][10] = 48,
+ [1][1][RTW89_UKRAINE][10] = 30,
+ [1][1][RTW89_MEXICO][10] = 66,
+ [1][1][RTW89_CN][10] = 30,
+ [1][1][RTW89_QATAR][10] = 30,
+ [1][1][RTW89_FCC][11] = 60,
+ [1][1][RTW89_ETSI][11] = 30,
+ [1][1][RTW89_MKK][11] = 38,
+ [1][1][RTW89_IC][11] = 60,
+ [1][1][RTW89_KCC][11] = 44,
+ [1][1][RTW89_ACMA][11] = 30,
+ [1][1][RTW89_CHILE][11] = 48,
+ [1][1][RTW89_UKRAINE][11] = 30,
+ [1][1][RTW89_MEXICO][11] = 60,
+ [1][1][RTW89_CN][11] = 30,
+ [1][1][RTW89_QATAR][11] = 30,
+ [1][1][RTW89_FCC][12] = 32,
+ [1][1][RTW89_ETSI][12] = 30,
+ [1][1][RTW89_MKK][12] = 38,
+ [1][1][RTW89_IC][12] = 32,
+ [1][1][RTW89_KCC][12] = 44,
+ [1][1][RTW89_ACMA][12] = 30,
+ [1][1][RTW89_CHILE][12] = 32,
+ [1][1][RTW89_UKRAINE][12] = 30,
+ [1][1][RTW89_MEXICO][12] = 32,
+ [1][1][RTW89_CN][12] = 30,
+ [1][1][RTW89_QATAR][12] = 30,
+ [1][1][RTW89_FCC][13] = 127,
+ [1][1][RTW89_ETSI][13] = 127,
+ [1][1][RTW89_MKK][13] = 127,
+ [1][1][RTW89_IC][13] = 127,
+ [1][1][RTW89_KCC][13] = 127,
+ [1][1][RTW89_ACMA][13] = 127,
+ [1][1][RTW89_CHILE][13] = 127,
+ [1][1][RTW89_UKRAINE][13] = 127,
+ [1][1][RTW89_MEXICO][13] = 127,
+ [1][1][RTW89_CN][13] = 127,
+ [1][1][RTW89_QATAR][13] = 127,
+ [2][0][RTW89_FCC][0] = 76,
+ [2][0][RTW89_ETSI][0] = 52,
+ [2][0][RTW89_MKK][0] = 64,
+ [2][0][RTW89_IC][0] = 76,
+ [2][0][RTW89_KCC][0] = 70,
+ [2][0][RTW89_ACMA][0] = 52,
+ [2][0][RTW89_CHILE][0] = 60,
+ [2][0][RTW89_UKRAINE][0] = 52,
+ [2][0][RTW89_MEXICO][0] = 76,
+ [2][0][RTW89_CN][0] = 52,
+ [2][0][RTW89_QATAR][0] = 52,
+ [2][0][RTW89_FCC][1] = 76,
+ [2][0][RTW89_ETSI][1] = 52,
+ [2][0][RTW89_MKK][1] = 64,
+ [2][0][RTW89_IC][1] = 76,
+ [2][0][RTW89_KCC][1] = 70,
+ [2][0][RTW89_ACMA][1] = 52,
+ [2][0][RTW89_CHILE][1] = 60,
+ [2][0][RTW89_UKRAINE][1] = 52,
+ [2][0][RTW89_MEXICO][1] = 76,
+ [2][0][RTW89_CN][1] = 52,
+ [2][0][RTW89_QATAR][1] = 52,
+ [2][0][RTW89_FCC][2] = 78,
+ [2][0][RTW89_ETSI][2] = 52,
+ [2][0][RTW89_MKK][2] = 64,
+ [2][0][RTW89_IC][2] = 78,
+ [2][0][RTW89_KCC][2] = 70,
+ [2][0][RTW89_ACMA][2] = 52,
+ [2][0][RTW89_CHILE][2] = 60,
+ [2][0][RTW89_UKRAINE][2] = 52,
+ [2][0][RTW89_MEXICO][2] = 78,
+ [2][0][RTW89_CN][2] = 52,
+ [2][0][RTW89_QATAR][2] = 52,
+ [2][0][RTW89_FCC][3] = 78,
+ [2][0][RTW89_ETSI][3] = 52,
+ [2][0][RTW89_MKK][3] = 64,
+ [2][0][RTW89_IC][3] = 78,
+ [2][0][RTW89_KCC][3] = 70,
+ [2][0][RTW89_ACMA][3] = 52,
+ [2][0][RTW89_CHILE][3] = 60,
+ [2][0][RTW89_UKRAINE][3] = 52,
+ [2][0][RTW89_MEXICO][3] = 78,
+ [2][0][RTW89_CN][3] = 52,
+ [2][0][RTW89_QATAR][3] = 52,
+ [2][0][RTW89_FCC][4] = 78,
+ [2][0][RTW89_ETSI][4] = 52,
+ [2][0][RTW89_MKK][4] = 64,
+ [2][0][RTW89_IC][4] = 78,
+ [2][0][RTW89_KCC][4] = 70,
+ [2][0][RTW89_ACMA][4] = 52,
+ [2][0][RTW89_CHILE][4] = 60,
+ [2][0][RTW89_UKRAINE][4] = 52,
+ [2][0][RTW89_MEXICO][4] = 78,
+ [2][0][RTW89_CN][4] = 52,
+ [2][0][RTW89_QATAR][4] = 52,
+ [2][0][RTW89_FCC][5] = 78,
+ [2][0][RTW89_ETSI][5] = 52,
+ [2][0][RTW89_MKK][5] = 64,
+ [2][0][RTW89_IC][5] = 78,
+ [2][0][RTW89_KCC][5] = 70,
+ [2][0][RTW89_ACMA][5] = 52,
+ [2][0][RTW89_CHILE][5] = 60,
+ [2][0][RTW89_UKRAINE][5] = 52,
+ [2][0][RTW89_MEXICO][5] = 78,
+ [2][0][RTW89_CN][5] = 52,
+ [2][0][RTW89_QATAR][5] = 52,
+ [2][0][RTW89_FCC][6] = 78,
+ [2][0][RTW89_ETSI][6] = 52,
+ [2][0][RTW89_MKK][6] = 64,
+ [2][0][RTW89_IC][6] = 78,
+ [2][0][RTW89_KCC][6] = 70,
+ [2][0][RTW89_ACMA][6] = 52,
+ [2][0][RTW89_CHILE][6] = 60,
+ [2][0][RTW89_UKRAINE][6] = 52,
+ [2][0][RTW89_MEXICO][6] = 78,
+ [2][0][RTW89_CN][6] = 52,
+ [2][0][RTW89_QATAR][6] = 52,
+ [2][0][RTW89_FCC][7] = 78,
+ [2][0][RTW89_ETSI][7] = 52,
+ [2][0][RTW89_MKK][7] = 64,
+ [2][0][RTW89_IC][7] = 78,
+ [2][0][RTW89_KCC][7] = 70,
+ [2][0][RTW89_ACMA][7] = 52,
+ [2][0][RTW89_CHILE][7] = 60,
+ [2][0][RTW89_UKRAINE][7] = 52,
+ [2][0][RTW89_MEXICO][7] = 78,
+ [2][0][RTW89_CN][7] = 52,
+ [2][0][RTW89_QATAR][7] = 52,
+ [2][0][RTW89_FCC][8] = 78,
+ [2][0][RTW89_ETSI][8] = 52,
+ [2][0][RTW89_MKK][8] = 64,
+ [2][0][RTW89_IC][8] = 78,
+ [2][0][RTW89_KCC][8] = 70,
+ [2][0][RTW89_ACMA][8] = 52,
+ [2][0][RTW89_CHILE][8] = 60,
+ [2][0][RTW89_UKRAINE][8] = 52,
+ [2][0][RTW89_MEXICO][8] = 78,
+ [2][0][RTW89_CN][8] = 52,
+ [2][0][RTW89_QATAR][8] = 52,
+ [2][0][RTW89_FCC][9] = 76,
+ [2][0][RTW89_ETSI][9] = 52,
+ [2][0][RTW89_MKK][9] = 64,
+ [2][0][RTW89_IC][9] = 76,
+ [2][0][RTW89_KCC][9] = 70,
+ [2][0][RTW89_ACMA][9] = 52,
+ [2][0][RTW89_CHILE][9] = 60,
+ [2][0][RTW89_UKRAINE][9] = 52,
+ [2][0][RTW89_MEXICO][9] = 76,
+ [2][0][RTW89_CN][9] = 52,
+ [2][0][RTW89_QATAR][9] = 52,
+ [2][0][RTW89_FCC][10] = 76,
+ [2][0][RTW89_ETSI][10] = 52,
+ [2][0][RTW89_MKK][10] = 64,
+ [2][0][RTW89_IC][10] = 76,
+ [2][0][RTW89_KCC][10] = 70,
+ [2][0][RTW89_ACMA][10] = 52,
+ [2][0][RTW89_CHILE][10] = 60,
+ [2][0][RTW89_UKRAINE][10] = 52,
+ [2][0][RTW89_MEXICO][10] = 76,
+ [2][0][RTW89_CN][10] = 52,
+ [2][0][RTW89_QATAR][10] = 52,
+ [2][0][RTW89_FCC][11] = 68,
+ [2][0][RTW89_ETSI][11] = 52,
+ [2][0][RTW89_MKK][11] = 64,
+ [2][0][RTW89_IC][11] = 68,
+ [2][0][RTW89_KCC][11] = 70,
+ [2][0][RTW89_ACMA][11] = 52,
+ [2][0][RTW89_CHILE][11] = 60,
+ [2][0][RTW89_UKRAINE][11] = 52,
+ [2][0][RTW89_MEXICO][11] = 68,
+ [2][0][RTW89_CN][11] = 52,
+ [2][0][RTW89_QATAR][11] = 52,
+ [2][0][RTW89_FCC][12] = 40,
+ [2][0][RTW89_ETSI][12] = 52,
+ [2][0][RTW89_MKK][12] = 64,
+ [2][0][RTW89_IC][12] = 40,
+ [2][0][RTW89_KCC][12] = 70,
+ [2][0][RTW89_ACMA][12] = 52,
+ [2][0][RTW89_CHILE][12] = 40,
+ [2][0][RTW89_UKRAINE][12] = 52,
+ [2][0][RTW89_MEXICO][12] = 40,
+ [2][0][RTW89_CN][12] = 52,
+ [2][0][RTW89_QATAR][12] = 52,
+ [2][0][RTW89_FCC][13] = 127,
+ [2][0][RTW89_ETSI][13] = 127,
+ [2][0][RTW89_MKK][13] = 127,
+ [2][0][RTW89_IC][13] = 127,
+ [2][0][RTW89_KCC][13] = 127,
+ [2][0][RTW89_ACMA][13] = 127,
+ [2][0][RTW89_CHILE][13] = 127,
+ [2][0][RTW89_UKRAINE][13] = 127,
+ [2][0][RTW89_MEXICO][13] = 127,
+ [2][0][RTW89_CN][13] = 127,
+ [2][0][RTW89_QATAR][13] = 127,
+ [2][1][RTW89_FCC][0] = 68,
+ [2][1][RTW89_ETSI][0] = 40,
+ [2][1][RTW89_MKK][0] = 52,
+ [2][1][RTW89_IC][0] = 68,
+ [2][1][RTW89_KCC][0] = 56,
+ [2][1][RTW89_ACMA][0] = 40,
+ [2][1][RTW89_CHILE][0] = 48,
+ [2][1][RTW89_UKRAINE][0] = 40,
+ [2][1][RTW89_MEXICO][0] = 68,
+ [2][1][RTW89_CN][0] = 40,
+ [2][1][RTW89_QATAR][0] = 40,
+ [2][1][RTW89_FCC][1] = 68,
+ [2][1][RTW89_ETSI][1] = 40,
+ [2][1][RTW89_MKK][1] = 52,
+ [2][1][RTW89_IC][1] = 68,
+ [2][1][RTW89_KCC][1] = 56,
+ [2][1][RTW89_ACMA][1] = 40,
+ [2][1][RTW89_CHILE][1] = 48,
+ [2][1][RTW89_UKRAINE][1] = 40,
+ [2][1][RTW89_MEXICO][1] = 68,
+ [2][1][RTW89_CN][1] = 40,
+ [2][1][RTW89_QATAR][1] = 40,
+ [2][1][RTW89_FCC][2] = 72,
+ [2][1][RTW89_ETSI][2] = 40,
+ [2][1][RTW89_MKK][2] = 52,
+ [2][1][RTW89_IC][2] = 72,
+ [2][1][RTW89_KCC][2] = 56,
+ [2][1][RTW89_ACMA][2] = 40,
+ [2][1][RTW89_CHILE][2] = 48,
+ [2][1][RTW89_UKRAINE][2] = 40,
+ [2][1][RTW89_MEXICO][2] = 72,
+ [2][1][RTW89_CN][2] = 40,
+ [2][1][RTW89_QATAR][2] = 40,
+ [2][1][RTW89_FCC][3] = 76,
+ [2][1][RTW89_ETSI][3] = 40,
+ [2][1][RTW89_MKK][3] = 52,
+ [2][1][RTW89_IC][3] = 76,
+ [2][1][RTW89_KCC][3] = 56,
+ [2][1][RTW89_ACMA][3] = 40,
+ [2][1][RTW89_CHILE][3] = 48,
+ [2][1][RTW89_UKRAINE][3] = 40,
+ [2][1][RTW89_MEXICO][3] = 76,
+ [2][1][RTW89_CN][3] = 40,
+ [2][1][RTW89_QATAR][3] = 40,
+ [2][1][RTW89_FCC][4] = 78,
+ [2][1][RTW89_ETSI][4] = 40,
+ [2][1][RTW89_MKK][4] = 52,
+ [2][1][RTW89_IC][4] = 78,
+ [2][1][RTW89_KCC][4] = 56,
+ [2][1][RTW89_ACMA][4] = 40,
+ [2][1][RTW89_CHILE][4] = 48,
+ [2][1][RTW89_UKRAINE][4] = 40,
+ [2][1][RTW89_MEXICO][4] = 78,
+ [2][1][RTW89_CN][4] = 40,
+ [2][1][RTW89_QATAR][4] = 40,
+ [2][1][RTW89_FCC][5] = 78,
+ [2][1][RTW89_ETSI][5] = 40,
+ [2][1][RTW89_MKK][5] = 52,
+ [2][1][RTW89_IC][5] = 78,
+ [2][1][RTW89_KCC][5] = 56,
+ [2][1][RTW89_ACMA][5] = 40,
+ [2][1][RTW89_CHILE][5] = 48,
+ [2][1][RTW89_UKRAINE][5] = 40,
+ [2][1][RTW89_MEXICO][5] = 78,
+ [2][1][RTW89_CN][5] = 40,
+ [2][1][RTW89_QATAR][5] = 40,
+ [2][1][RTW89_FCC][6] = 78,
+ [2][1][RTW89_ETSI][6] = 40,
+ [2][1][RTW89_MKK][6] = 52,
+ [2][1][RTW89_IC][6] = 78,
+ [2][1][RTW89_KCC][6] = 56,
+ [2][1][RTW89_ACMA][6] = 40,
+ [2][1][RTW89_CHILE][6] = 48,
+ [2][1][RTW89_UKRAINE][6] = 40,
+ [2][1][RTW89_MEXICO][6] = 78,
+ [2][1][RTW89_CN][6] = 40,
+ [2][1][RTW89_QATAR][6] = 40,
+ [2][1][RTW89_FCC][7] = 78,
+ [2][1][RTW89_ETSI][7] = 40,
+ [2][1][RTW89_MKK][7] = 52,
+ [2][1][RTW89_IC][7] = 78,
+ [2][1][RTW89_KCC][7] = 56,
+ [2][1][RTW89_ACMA][7] = 40,
+ [2][1][RTW89_CHILE][7] = 48,
+ [2][1][RTW89_UKRAINE][7] = 40,
+ [2][1][RTW89_MEXICO][7] = 78,
+ [2][1][RTW89_CN][7] = 40,
+ [2][1][RTW89_QATAR][7] = 40,
+ [2][1][RTW89_FCC][8] = 74,
+ [2][1][RTW89_ETSI][8] = 40,
+ [2][1][RTW89_MKK][8] = 52,
+ [2][1][RTW89_IC][8] = 74,
+ [2][1][RTW89_KCC][8] = 56,
+ [2][1][RTW89_ACMA][8] = 40,
+ [2][1][RTW89_CHILE][8] = 48,
+ [2][1][RTW89_UKRAINE][8] = 40,
+ [2][1][RTW89_MEXICO][8] = 74,
+ [2][1][RTW89_CN][8] = 40,
+ [2][1][RTW89_QATAR][8] = 40,
+ [2][1][RTW89_FCC][9] = 70,
+ [2][1][RTW89_ETSI][9] = 40,
+ [2][1][RTW89_MKK][9] = 52,
+ [2][1][RTW89_IC][9] = 70,
+ [2][1][RTW89_KCC][9] = 56,
+ [2][1][RTW89_ACMA][9] = 40,
+ [2][1][RTW89_CHILE][9] = 48,
+ [2][1][RTW89_UKRAINE][9] = 40,
+ [2][1][RTW89_MEXICO][9] = 70,
+ [2][1][RTW89_CN][9] = 40,
+ [2][1][RTW89_QATAR][9] = 40,
+ [2][1][RTW89_FCC][10] = 70,
+ [2][1][RTW89_ETSI][10] = 40,
+ [2][1][RTW89_MKK][10] = 52,
+ [2][1][RTW89_IC][10] = 70,
+ [2][1][RTW89_KCC][10] = 56,
+ [2][1][RTW89_ACMA][10] = 40,
+ [2][1][RTW89_CHILE][10] = 48,
+ [2][1][RTW89_UKRAINE][10] = 40,
+ [2][1][RTW89_MEXICO][10] = 70,
+ [2][1][RTW89_CN][10] = 40,
+ [2][1][RTW89_QATAR][10] = 40,
+ [2][1][RTW89_FCC][11] = 48,
+ [2][1][RTW89_ETSI][11] = 40,
+ [2][1][RTW89_MKK][11] = 52,
+ [2][1][RTW89_IC][11] = 48,
+ [2][1][RTW89_KCC][11] = 56,
+ [2][1][RTW89_ACMA][11] = 40,
+ [2][1][RTW89_CHILE][11] = 48,
+ [2][1][RTW89_UKRAINE][11] = 40,
+ [2][1][RTW89_MEXICO][11] = 48,
+ [2][1][RTW89_CN][11] = 40,
+ [2][1][RTW89_QATAR][11] = 40,
+ [2][1][RTW89_FCC][12] = 26,
+ [2][1][RTW89_ETSI][12] = 40,
+ [2][1][RTW89_MKK][12] = 52,
+ [2][1][RTW89_IC][12] = 26,
+ [2][1][RTW89_KCC][12] = 56,
+ [2][1][RTW89_ACMA][12] = 40,
+ [2][1][RTW89_CHILE][12] = 26,
+ [2][1][RTW89_UKRAINE][12] = 40,
+ [2][1][RTW89_MEXICO][12] = 26,
+ [2][1][RTW89_CN][12] = 40,
+ [2][1][RTW89_QATAR][12] = 40,
+ [2][1][RTW89_FCC][13] = 127,
+ [2][1][RTW89_ETSI][13] = 127,
+ [2][1][RTW89_MKK][13] = 127,
+ [2][1][RTW89_IC][13] = 127,
+ [2][1][RTW89_KCC][13] = 127,
+ [2][1][RTW89_ACMA][13] = 127,
+ [2][1][RTW89_CHILE][13] = 127,
+ [2][1][RTW89_UKRAINE][13] = 127,
+ [2][1][RTW89_MEXICO][13] = 127,
+ [2][1][RTW89_CN][13] = 127,
+ [2][1][RTW89_QATAR][13] = 127,
};
const s8 rtw89_8852a_txpwr_lmt_ru_5g[RTW89_RU_NUM][RTW89_NTX_NUM]
[RTW89_REGD_NUM][RTW89_5G_CH_NUM] = {
- [0][0][0][0] = 22,
- [0][0][0][2] = 22,
- [0][0][0][4] = 22,
- [0][0][0][6] = 22,
- [0][0][0][8] = 24,
- [0][0][0][10] = 24,
- [0][0][0][12] = 24,
- [0][0][0][14] = 24,
- [0][0][0][15] = 24,
- [0][0][0][17] = 24,
- [0][0][0][19] = 24,
- [0][0][0][21] = 24,
- [0][0][0][23] = 24,
- [0][0][0][25] = 24,
- [0][0][0][27] = 24,
- [0][0][0][29] = 24,
- [0][0][0][31] = 24,
- [0][0][0][33] = 24,
- [0][0][0][35] = 24,
- [0][0][0][37] = 24,
- [0][0][0][38] = 28,
- [0][0][0][40] = 28,
- [0][0][0][42] = 28,
- [0][0][0][44] = 28,
- [0][0][0][46] = 28,
- [0][1][0][0] = 8,
- [0][1][0][2] = 8,
- [0][1][0][4] = 8,
- [0][1][0][6] = 8,
- [0][1][0][8] = 12,
- [0][1][0][10] = 12,
- [0][1][0][12] = 12,
- [0][1][0][14] = 12,
- [0][1][0][15] = 12,
- [0][1][0][17] = 12,
- [0][1][0][19] = 12,
- [0][1][0][21] = 12,
- [0][1][0][23] = 12,
- [0][1][0][25] = 12,
- [0][1][0][27] = 12,
- [0][1][0][29] = 12,
- [0][1][0][31] = 12,
- [0][1][0][33] = 12,
- [0][1][0][35] = 12,
- [0][1][0][37] = 12,
- [0][1][0][38] = 16,
- [0][1][0][40] = 16,
- [0][1][0][42] = 16,
- [0][1][0][44] = 16,
- [0][1][0][46] = 16,
- [1][0][0][0] = 30,
- [1][0][0][2] = 30,
- [1][0][0][4] = 30,
- [1][0][0][6] = 30,
- [1][0][0][8] = 36,
- [1][0][0][10] = 36,
- [1][0][0][12] = 36,
- [1][0][0][14] = 36,
- [1][0][0][15] = 36,
- [1][0][0][17] = 36,
- [1][0][0][19] = 36,
- [1][0][0][21] = 36,
- [1][0][0][23] = 36,
- [1][0][0][25] = 36,
- [1][0][0][27] = 36,
- [1][0][0][29] = 36,
- [1][0][0][31] = 36,
- [1][0][0][33] = 36,
- [1][0][0][35] = 36,
- [1][0][0][37] = 36,
- [1][0][0][38] = 28,
- [1][0][0][40] = 28,
- [1][0][0][42] = 28,
- [1][0][0][44] = 28,
- [1][0][0][46] = 28,
- [1][1][0][0] = 18,
- [1][1][0][2] = 18,
- [1][1][0][4] = 18,
- [1][1][0][6] = 18,
- [1][1][0][8] = 22,
- [1][1][0][10] = 22,
- [1][1][0][12] = 22,
- [1][1][0][14] = 22,
- [1][1][0][15] = 22,
- [1][1][0][17] = 22,
- [1][1][0][19] = 22,
- [1][1][0][21] = 22,
- [1][1][0][23] = 22,
- [1][1][0][25] = 22,
- [1][1][0][27] = 22,
- [1][1][0][29] = 22,
- [1][1][0][31] = 22,
- [1][1][0][33] = 22,
- [1][1][0][35] = 22,
- [1][1][0][37] = 22,
- [1][1][0][38] = 16,
- [1][1][0][40] = 16,
- [1][1][0][42] = 16,
- [1][1][0][44] = 16,
- [1][1][0][46] = 16,
- [2][0][0][0] = 30,
- [2][0][0][2] = 30,
- [2][0][0][4] = 30,
- [2][0][0][6] = 30,
- [2][0][0][8] = 46,
- [2][0][0][10] = 46,
- [2][0][0][12] = 46,
- [2][0][0][14] = 46,
- [2][0][0][15] = 46,
- [2][0][0][17] = 46,
- [2][0][0][19] = 46,
- [2][0][0][21] = 46,
- [2][0][0][23] = 46,
- [2][0][0][25] = 46,
- [2][0][0][27] = 46,
- [2][0][0][29] = 46,
- [2][0][0][31] = 46,
- [2][0][0][33] = 46,
- [2][0][0][35] = 46,
- [2][0][0][37] = 46,
- [2][0][0][38] = 28,
- [2][0][0][40] = 28,
- [2][0][0][42] = 28,
- [2][0][0][44] = 28,
- [2][0][0][46] = 28,
- [2][1][0][0] = 18,
- [2][1][0][2] = 18,
- [2][1][0][4] = 18,
- [2][1][0][6] = 18,
- [2][1][0][8] = 32,
- [2][1][0][10] = 32,
- [2][1][0][12] = 32,
- [2][1][0][14] = 32,
- [2][1][0][15] = 32,
- [2][1][0][17] = 32,
- [2][1][0][19] = 32,
- [2][1][0][21] = 32,
- [2][1][0][23] = 32,
- [2][1][0][25] = 32,
- [2][1][0][27] = 32,
- [2][1][0][29] = 32,
- [2][1][0][31] = 32,
- [2][1][0][33] = 32,
- [2][1][0][35] = 32,
- [2][1][0][37] = 32,
- [2][1][0][38] = 16,
- [2][1][0][40] = 16,
- [2][1][0][42] = 16,
- [2][1][0][44] = 16,
- [2][1][0][46] = 16,
- [0][0][2][0] = 48,
- [0][0][1][0] = 24,
- [0][0][3][0] = 26,
- [0][0][5][0] = 22,
- [0][0][6][0] = 24,
- [0][0][9][0] = 24,
- [0][0][8][0] = 30,
- [0][0][11][0] = 24,
- [0][0][2][2] = 48,
- [0][0][1][2] = 24,
- [0][0][3][2] = 26,
- [0][0][5][2] = 22,
- [0][0][6][2] = 24,
- [0][0][9][2] = 24,
- [0][0][8][2] = 30,
- [0][0][11][2] = 24,
- [0][0][2][4] = 48,
- [0][0][1][4] = 24,
- [0][0][3][4] = 26,
- [0][0][5][4] = 22,
- [0][0][6][4] = 24,
- [0][0][9][4] = 24,
- [0][0][8][4] = 30,
- [0][0][11][4] = 24,
- [0][0][2][6] = 48,
- [0][0][1][6] = 24,
- [0][0][3][6] = 26,
- [0][0][5][6] = 22,
- [0][0][6][6] = 24,
- [0][0][9][6] = 24,
- [0][0][8][6] = 30,
- [0][0][11][6] = 24,
- [0][0][2][8] = 48,
- [0][0][1][8] = 24,
- [0][0][3][8] = 26,
- [0][0][5][8] = 48,
- [0][0][6][8] = 24,
- [0][0][9][8] = 24,
- [0][0][8][8] = 54,
- [0][0][11][8] = 24,
- [0][0][2][10] = 48,
- [0][0][1][10] = 24,
- [0][0][3][10] = 26,
- [0][0][5][10] = 48,
- [0][0][6][10] = 24,
- [0][0][9][10] = 24,
- [0][0][8][10] = 54,
- [0][0][11][10] = 24,
- [0][0][2][12] = 48,
- [0][0][1][12] = 24,
- [0][0][3][12] = 26,
- [0][0][5][12] = 48,
- [0][0][6][12] = 24,
- [0][0][9][12] = 24,
- [0][0][8][12] = 54,
- [0][0][11][12] = 24,
- [0][0][2][14] = 48,
- [0][0][1][14] = 24,
- [0][0][3][14] = 26,
- [0][0][5][14] = 48,
- [0][0][6][14] = 24,
- [0][0][9][14] = 24,
- [0][0][8][14] = 54,
- [0][0][11][14] = 24,
- [0][0][2][15] = 48,
- [0][0][1][15] = 24,
- [0][0][3][15] = 44,
- [0][0][5][15] = 48,
- [0][0][6][15] = 24,
- [0][0][9][15] = 24,
- [0][0][8][15] = 54,
- [0][0][11][15] = 24,
- [0][0][2][17] = 48,
- [0][0][1][17] = 24,
- [0][0][3][17] = 44,
- [0][0][5][17] = 48,
- [0][0][6][17] = 24,
- [0][0][9][17] = 24,
- [0][0][8][17] = 54,
- [0][0][11][17] = 24,
- [0][0][2][19] = 48,
- [0][0][1][19] = 24,
- [0][0][3][19] = 44,
- [0][0][5][19] = 48,
- [0][0][6][19] = 24,
- [0][0][9][19] = 24,
- [0][0][8][19] = 54,
- [0][0][11][19] = 24,
- [0][0][2][21] = 48,
- [0][0][1][21] = 24,
- [0][0][3][21] = 44,
- [0][0][5][21] = 48,
- [0][0][6][21] = 24,
- [0][0][9][21] = 24,
- [0][0][8][21] = 54,
- [0][0][11][21] = 24,
- [0][0][2][23] = 48,
- [0][0][1][23] = 24,
- [0][0][3][23] = 44,
- [0][0][5][23] = 48,
- [0][0][6][23] = 24,
- [0][0][9][23] = 24,
- [0][0][8][23] = 54,
- [0][0][11][23] = 24,
- [0][0][2][25] = 48,
- [0][0][1][25] = 24,
- [0][0][3][25] = 44,
- [0][0][5][25] = 127,
- [0][0][6][25] = 24,
- [0][0][9][25] = 127,
- [0][0][8][25] = 54,
- [0][0][11][25] = 24,
- [0][0][2][27] = 48,
- [0][0][1][27] = 24,
- [0][0][3][27] = 44,
- [0][0][5][27] = 127,
- [0][0][6][27] = 24,
- [0][0][9][27] = 127,
- [0][0][8][27] = 54,
- [0][0][11][27] = 24,
- [0][0][2][29] = 48,
- [0][0][1][29] = 24,
- [0][0][3][29] = 44,
- [0][0][5][29] = 127,
- [0][0][6][29] = 24,
- [0][0][9][29] = 127,
- [0][0][8][29] = 54,
- [0][0][11][29] = 24,
- [0][0][2][31] = 48,
- [0][0][1][31] = 24,
- [0][0][3][31] = 44,
- [0][0][5][31] = 48,
- [0][0][6][31] = 24,
- [0][0][9][31] = 24,
- [0][0][8][31] = 54,
- [0][0][11][31] = 24,
- [0][0][2][33] = 48,
- [0][0][1][33] = 24,
- [0][0][3][33] = 44,
- [0][0][5][33] = 48,
- [0][0][6][33] = 24,
- [0][0][9][33] = 24,
- [0][0][8][33] = 54,
- [0][0][11][33] = 24,
- [0][0][2][35] = 48,
- [0][0][1][35] = 24,
- [0][0][3][35] = 44,
- [0][0][5][35] = 48,
- [0][0][6][35] = 24,
- [0][0][9][35] = 24,
- [0][0][8][35] = 54,
- [0][0][11][35] = 24,
- [0][0][2][37] = 48,
- [0][0][1][37] = 127,
- [0][0][3][37] = 44,
- [0][0][5][37] = 48,
- [0][0][6][37] = 24,
- [0][0][9][37] = 48,
- [0][0][8][37] = 54,
- [0][0][11][37] = 127,
- [0][0][2][38] = 76,
- [0][0][1][38] = 28,
- [0][0][3][38] = 127,
- [0][0][5][38] = 76,
- [0][0][6][38] = 28,
- [0][0][9][38] = 76,
- [0][0][8][38] = 54,
- [0][0][11][38] = 28,
- [0][0][2][40] = 76,
- [0][0][1][40] = 28,
- [0][0][3][40] = 127,
- [0][0][5][40] = 76,
- [0][0][6][40] = 28,
- [0][0][9][40] = 76,
- [0][0][8][40] = 54,
- [0][0][11][40] = 28,
- [0][0][2][42] = 76,
- [0][0][1][42] = 28,
- [0][0][3][42] = 127,
- [0][0][5][42] = 76,
- [0][0][6][42] = 28,
- [0][0][9][42] = 76,
- [0][0][8][42] = 54,
- [0][0][11][42] = 28,
- [0][0][2][44] = 76,
- [0][0][1][44] = 28,
- [0][0][3][44] = 127,
- [0][0][5][44] = 76,
- [0][0][6][44] = 28,
- [0][0][9][44] = 76,
- [0][0][8][44] = 54,
- [0][0][11][44] = 28,
- [0][0][2][46] = 76,
- [0][0][1][46] = 28,
- [0][0][3][46] = 127,
- [0][0][5][46] = 76,
- [0][0][6][46] = 28,
- [0][0][9][46] = 76,
- [0][0][8][46] = 54,
- [0][0][11][46] = 28,
- [0][1][2][0] = 36,
- [0][1][1][0] = 12,
- [0][1][3][0] = 14,
- [0][1][5][0] = 8,
- [0][1][6][0] = 12,
- [0][1][9][0] = 12,
- [0][1][8][0] = 18,
- [0][1][11][0] = 12,
- [0][1][2][2] = 36,
- [0][1][1][2] = 12,
- [0][1][3][2] = 14,
- [0][1][5][2] = 8,
- [0][1][6][2] = 12,
- [0][1][9][2] = 12,
- [0][1][8][2] = 18,
- [0][1][11][2] = 12,
- [0][1][2][4] = 36,
- [0][1][1][4] = 12,
- [0][1][3][4] = 14,
- [0][1][5][4] = 8,
- [0][1][6][4] = 12,
- [0][1][9][4] = 12,
- [0][1][8][4] = 18,
- [0][1][11][4] = 12,
- [0][1][2][6] = 36,
- [0][1][1][6] = 12,
- [0][1][3][6] = 14,
- [0][1][5][6] = 8,
- [0][1][6][6] = 12,
- [0][1][9][6] = 12,
- [0][1][8][6] = 18,
- [0][1][11][6] = 12,
- [0][1][2][8] = 36,
- [0][1][1][8] = 12,
- [0][1][3][8] = 14,
- [0][1][5][8] = 36,
- [0][1][6][8] = 12,
- [0][1][9][8] = 12,
- [0][1][8][8] = 42,
- [0][1][11][8] = 12,
- [0][1][2][10] = 36,
- [0][1][1][10] = 12,
- [0][1][3][10] = 14,
- [0][1][5][10] = 36,
- [0][1][6][10] = 12,
- [0][1][9][10] = 12,
- [0][1][8][10] = 42,
- [0][1][11][10] = 12,
- [0][1][2][12] = 36,
- [0][1][1][12] = 12,
- [0][1][3][12] = 14,
- [0][1][5][12] = 36,
- [0][1][6][12] = 12,
- [0][1][9][12] = 12,
- [0][1][8][12] = 42,
- [0][1][11][12] = 12,
- [0][1][2][14] = 36,
- [0][1][1][14] = 12,
- [0][1][3][14] = 14,
- [0][1][5][14] = 36,
- [0][1][6][14] = 12,
- [0][1][9][14] = 12,
- [0][1][8][14] = 42,
- [0][1][11][14] = 12,
- [0][1][2][15] = 36,
- [0][1][1][15] = 12,
- [0][1][3][15] = 32,
- [0][1][5][15] = 36,
- [0][1][6][15] = 12,
- [0][1][9][15] = 12,
- [0][1][8][15] = 42,
- [0][1][11][15] = 12,
- [0][1][2][17] = 36,
- [0][1][1][17] = 12,
- [0][1][3][17] = 32,
- [0][1][5][17] = 36,
- [0][1][6][17] = 12,
- [0][1][9][17] = 12,
- [0][1][8][17] = 42,
- [0][1][11][17] = 12,
- [0][1][2][19] = 36,
- [0][1][1][19] = 12,
- [0][1][3][19] = 32,
- [0][1][5][19] = 36,
- [0][1][6][19] = 12,
- [0][1][9][19] = 12,
- [0][1][8][19] = 42,
- [0][1][11][19] = 12,
- [0][1][2][21] = 36,
- [0][1][1][21] = 12,
- [0][1][3][21] = 32,
- [0][1][5][21] = 36,
- [0][1][6][21] = 12,
- [0][1][9][21] = 12,
- [0][1][8][21] = 42,
- [0][1][11][21] = 12,
- [0][1][2][23] = 36,
- [0][1][1][23] = 12,
- [0][1][3][23] = 32,
- [0][1][5][23] = 36,
- [0][1][6][23] = 12,
- [0][1][9][23] = 12,
- [0][1][8][23] = 42,
- [0][1][11][23] = 12,
- [0][1][2][25] = 36,
- [0][1][1][25] = 12,
- [0][1][3][25] = 32,
- [0][1][5][25] = 127,
- [0][1][6][25] = 12,
- [0][1][9][25] = 127,
- [0][1][8][25] = 42,
- [0][1][11][25] = 12,
- [0][1][2][27] = 36,
- [0][1][1][27] = 12,
- [0][1][3][27] = 32,
- [0][1][5][27] = 127,
- [0][1][6][27] = 12,
- [0][1][9][27] = 127,
- [0][1][8][27] = 42,
- [0][1][11][27] = 12,
- [0][1][2][29] = 36,
- [0][1][1][29] = 12,
- [0][1][3][29] = 32,
- [0][1][5][29] = 127,
- [0][1][6][29] = 12,
- [0][1][9][29] = 127,
- [0][1][8][29] = 42,
- [0][1][11][29] = 12,
- [0][1][2][31] = 36,
- [0][1][1][31] = 12,
- [0][1][3][31] = 32,
- [0][1][5][31] = 36,
- [0][1][6][31] = 12,
- [0][1][9][31] = 12,
- [0][1][8][31] = 42,
- [0][1][11][31] = 12,
- [0][1][2][33] = 36,
- [0][1][1][33] = 12,
- [0][1][3][33] = 32,
- [0][1][5][33] = 36,
- [0][1][6][33] = 12,
- [0][1][9][33] = 12,
- [0][1][8][33] = 42,
- [0][1][11][33] = 12,
- [0][1][2][35] = 36,
- [0][1][1][35] = 12,
- [0][1][3][35] = 32,
- [0][1][5][35] = 36,
- [0][1][6][35] = 12,
- [0][1][9][35] = 12,
- [0][1][8][35] = 42,
- [0][1][11][35] = 12,
- [0][1][2][37] = 36,
- [0][1][1][37] = 127,
- [0][1][3][37] = 32,
- [0][1][5][37] = 36,
- [0][1][6][37] = 12,
- [0][1][9][37] = 36,
- [0][1][8][37] = 42,
- [0][1][11][37] = 127,
- [0][1][2][38] = 72,
- [0][1][1][38] = 16,
- [0][1][3][38] = 127,
- [0][1][5][38] = 72,
- [0][1][6][38] = 16,
- [0][1][9][38] = 76,
- [0][1][8][38] = 42,
- [0][1][11][38] = 16,
- [0][1][2][40] = 76,
- [0][1][1][40] = 16,
- [0][1][3][40] = 127,
- [0][1][5][40] = 76,
- [0][1][6][40] = 16,
- [0][1][9][40] = 76,
- [0][1][8][40] = 42,
- [0][1][11][40] = 16,
- [0][1][2][42] = 76,
- [0][1][1][42] = 16,
- [0][1][3][42] = 127,
- [0][1][5][42] = 76,
- [0][1][6][42] = 16,
- [0][1][9][42] = 76,
- [0][1][8][42] = 42,
- [0][1][11][42] = 16,
- [0][1][2][44] = 76,
- [0][1][1][44] = 16,
- [0][1][3][44] = 127,
- [0][1][5][44] = 76,
- [0][1][6][44] = 16,
- [0][1][9][44] = 76,
- [0][1][8][44] = 42,
- [0][1][11][44] = 16,
- [0][1][2][46] = 76,
- [0][1][1][46] = 16,
- [0][1][3][46] = 127,
- [0][1][5][46] = 76,
- [0][1][6][46] = 16,
- [0][1][9][46] = 76,
- [0][1][8][46] = 42,
- [0][1][11][46] = 16,
- [1][0][2][0] = 62,
- [1][0][1][0] = 36,
- [1][0][3][0] = 36,
- [1][0][5][0] = 34,
- [1][0][6][0] = 36,
- [1][0][9][0] = 36,
- [1][0][8][0] = 30,
- [1][0][11][0] = 36,
- [1][0][2][2] = 62,
- [1][0][1][2] = 36,
- [1][0][3][2] = 36,
- [1][0][5][2] = 34,
- [1][0][6][2] = 36,
- [1][0][9][2] = 36,
- [1][0][8][2] = 30,
- [1][0][11][2] = 36,
- [1][0][2][4] = 62,
- [1][0][1][4] = 36,
- [1][0][3][4] = 36,
- [1][0][5][4] = 34,
- [1][0][6][4] = 36,
- [1][0][9][4] = 36,
- [1][0][8][4] = 30,
- [1][0][11][4] = 36,
- [1][0][2][6] = 62,
- [1][0][1][6] = 36,
- [1][0][3][6] = 36,
- [1][0][5][6] = 34,
- [1][0][6][6] = 36,
- [1][0][9][6] = 36,
- [1][0][8][6] = 30,
- [1][0][11][6] = 36,
- [1][0][2][8] = 62,
- [1][0][1][8] = 36,
- [1][0][3][8] = 36,
- [1][0][5][8] = 62,
- [1][0][6][8] = 36,
- [1][0][9][8] = 36,
- [1][0][8][8] = 54,
- [1][0][11][8] = 36,
- [1][0][2][10] = 62,
- [1][0][1][10] = 36,
- [1][0][3][10] = 36,
- [1][0][5][10] = 62,
- [1][0][6][10] = 36,
- [1][0][9][10] = 36,
- [1][0][8][10] = 54,
- [1][0][11][10] = 36,
- [1][0][2][12] = 62,
- [1][0][1][12] = 36,
- [1][0][3][12] = 36,
- [1][0][5][12] = 62,
- [1][0][6][12] = 36,
- [1][0][9][12] = 36,
- [1][0][8][12] = 54,
- [1][0][11][12] = 36,
- [1][0][2][14] = 62,
- [1][0][1][14] = 36,
- [1][0][3][14] = 36,
- [1][0][5][14] = 62,
- [1][0][6][14] = 36,
- [1][0][9][14] = 36,
- [1][0][8][14] = 54,
- [1][0][11][14] = 36,
- [1][0][2][15] = 62,
- [1][0][1][15] = 36,
- [1][0][3][15] = 58,
- [1][0][5][15] = 62,
- [1][0][6][15] = 36,
- [1][0][9][15] = 36,
- [1][0][8][15] = 54,
- [1][0][11][15] = 36,
- [1][0][2][17] = 62,
- [1][0][1][17] = 36,
- [1][0][3][17] = 58,
- [1][0][5][17] = 62,
- [1][0][6][17] = 36,
- [1][0][9][17] = 36,
- [1][0][8][17] = 54,
- [1][0][11][17] = 36,
- [1][0][2][19] = 62,
- [1][0][1][19] = 36,
- [1][0][3][19] = 58,
- [1][0][5][19] = 62,
- [1][0][6][19] = 36,
- [1][0][9][19] = 36,
- [1][0][8][19] = 54,
- [1][0][11][19] = 36,
- [1][0][2][21] = 62,
- [1][0][1][21] = 36,
- [1][0][3][21] = 58,
- [1][0][5][21] = 62,
- [1][0][6][21] = 36,
- [1][0][9][21] = 36,
- [1][0][8][21] = 54,
- [1][0][11][21] = 36,
- [1][0][2][23] = 62,
- [1][0][1][23] = 36,
- [1][0][3][23] = 58,
- [1][0][5][23] = 62,
- [1][0][6][23] = 36,
- [1][0][9][23] = 36,
- [1][0][8][23] = 54,
- [1][0][11][23] = 36,
- [1][0][2][25] = 62,
- [1][0][1][25] = 36,
- [1][0][3][25] = 58,
- [1][0][5][25] = 127,
- [1][0][6][25] = 36,
- [1][0][9][25] = 127,
- [1][0][8][25] = 54,
- [1][0][11][25] = 36,
- [1][0][2][27] = 62,
- [1][0][1][27] = 36,
- [1][0][3][27] = 58,
- [1][0][5][27] = 127,
- [1][0][6][27] = 36,
- [1][0][9][27] = 127,
- [1][0][8][27] = 54,
- [1][0][11][27] = 36,
- [1][0][2][29] = 62,
- [1][0][1][29] = 36,
- [1][0][3][29] = 58,
- [1][0][5][29] = 127,
- [1][0][6][29] = 36,
- [1][0][9][29] = 127,
- [1][0][8][29] = 54,
- [1][0][11][29] = 36,
- [1][0][2][31] = 62,
- [1][0][1][31] = 36,
- [1][0][3][31] = 58,
- [1][0][5][31] = 62,
- [1][0][6][31] = 36,
- [1][0][9][31] = 36,
- [1][0][8][31] = 54,
- [1][0][11][31] = 36,
- [1][0][2][33] = 62,
- [1][0][1][33] = 36,
- [1][0][3][33] = 58,
- [1][0][5][33] = 62,
- [1][0][6][33] = 36,
- [1][0][9][33] = 36,
- [1][0][8][33] = 54,
- [1][0][11][33] = 36,
- [1][0][2][35] = 62,
- [1][0][1][35] = 36,
- [1][0][3][35] = 58,
- [1][0][5][35] = 62,
- [1][0][6][35] = 36,
- [1][0][9][35] = 36,
- [1][0][8][35] = 54,
- [1][0][11][35] = 36,
- [1][0][2][37] = 56,
- [1][0][1][37] = 62,
- [1][0][3][37] = 127,
- [1][0][5][37] = 58,
- [1][0][6][37] = 62,
- [1][0][9][37] = 36,
- [1][0][8][37] = 62,
- [1][0][11][37] = 54,
- [1][0][2][38] = 76,
- [1][0][1][38] = 28,
- [1][0][3][38] = 127,
- [1][0][5][38] = 76,
- [1][0][6][38] = 28,
- [1][0][9][38] = 76,
- [1][0][8][38] = 54,
- [1][0][11][38] = 28,
- [1][0][2][40] = 76,
- [1][0][1][40] = 28,
- [1][0][3][40] = 127,
- [1][0][5][40] = 76,
- [1][0][6][40] = 28,
- [1][0][9][40] = 76,
- [1][0][8][40] = 54,
- [1][0][11][40] = 28,
- [1][0][2][42] = 76,
- [1][0][1][42] = 28,
- [1][0][3][42] = 127,
- [1][0][5][42] = 76,
- [1][0][6][42] = 28,
- [1][0][9][42] = 76,
- [1][0][8][42] = 54,
- [1][0][11][42] = 28,
- [1][0][2][44] = 76,
- [1][0][1][44] = 28,
- [1][0][3][44] = 127,
- [1][0][5][44] = 76,
- [1][0][6][44] = 28,
- [1][0][9][44] = 76,
- [1][0][8][44] = 54,
- [1][0][11][44] = 28,
- [1][0][2][46] = 76,
- [1][0][1][46] = 28,
- [1][0][3][46] = 127,
- [1][0][5][46] = 76,
- [1][0][6][46] = 28,
- [1][0][9][46] = 76,
- [1][0][8][46] = 54,
- [1][0][11][46] = 28,
- [1][1][2][0] = 46,
- [1][1][1][0] = 22,
- [1][1][3][0] = 24,
- [1][1][5][0] = 18,
- [1][1][6][0] = 22,
- [1][1][9][0] = 22,
- [1][1][8][0] = 18,
- [1][1][11][0] = 22,
- [1][1][2][2] = 46,
- [1][1][1][2] = 22,
- [1][1][3][2] = 24,
- [1][1][5][2] = 18,
- [1][1][6][2] = 22,
- [1][1][9][2] = 22,
- [1][1][8][2] = 18,
- [1][1][11][2] = 22,
- [1][1][2][4] = 46,
- [1][1][1][4] = 22,
- [1][1][3][4] = 24,
- [1][1][5][4] = 18,
- [1][1][6][4] = 22,
- [1][1][9][4] = 22,
- [1][1][8][4] = 18,
- [1][1][11][4] = 22,
- [1][1][2][6] = 46,
- [1][1][1][6] = 22,
- [1][1][3][6] = 24,
- [1][1][5][6] = 18,
- [1][1][6][6] = 22,
- [1][1][9][6] = 22,
- [1][1][8][6] = 18,
- [1][1][11][6] = 22,
- [1][1][2][8] = 46,
- [1][1][1][8] = 22,
- [1][1][3][8] = 24,
- [1][1][5][8] = 46,
- [1][1][6][8] = 22,
- [1][1][9][8] = 22,
- [1][1][8][8] = 42,
- [1][1][11][8] = 22,
- [1][1][2][10] = 46,
- [1][1][1][10] = 22,
- [1][1][3][10] = 24,
- [1][1][5][10] = 46,
- [1][1][6][10] = 22,
- [1][1][9][10] = 22,
- [1][1][8][10] = 42,
- [1][1][11][10] = 22,
- [1][1][2][12] = 46,
- [1][1][1][12] = 22,
- [1][1][3][12] = 24,
- [1][1][5][12] = 46,
- [1][1][6][12] = 22,
- [1][1][9][12] = 22,
- [1][1][8][12] = 42,
- [1][1][11][12] = 22,
- [1][1][2][14] = 46,
- [1][1][1][14] = 22,
- [1][1][3][14] = 24,
- [1][1][5][14] = 46,
- [1][1][6][14] = 22,
- [1][1][9][14] = 22,
- [1][1][8][14] = 42,
- [1][1][11][14] = 22,
- [1][1][2][15] = 46,
- [1][1][1][15] = 22,
- [1][1][3][15] = 46,
- [1][1][5][15] = 46,
- [1][1][6][15] = 22,
- [1][1][9][15] = 22,
- [1][1][8][15] = 42,
- [1][1][11][15] = 22,
- [1][1][2][17] = 46,
- [1][1][1][17] = 22,
- [1][1][3][17] = 46,
- [1][1][5][17] = 46,
- [1][1][6][17] = 22,
- [1][1][9][17] = 22,
- [1][1][8][17] = 42,
- [1][1][11][17] = 22,
- [1][1][2][19] = 46,
- [1][1][1][19] = 22,
- [1][1][3][19] = 46,
- [1][1][5][19] = 46,
- [1][1][6][19] = 22,
- [1][1][9][19] = 22,
- [1][1][8][19] = 42,
- [1][1][11][19] = 22,
- [1][1][2][21] = 46,
- [1][1][1][21] = 22,
- [1][1][3][21] = 46,
- [1][1][5][21] = 46,
- [1][1][6][21] = 22,
- [1][1][9][21] = 22,
- [1][1][8][21] = 42,
- [1][1][11][21] = 22,
- [1][1][2][23] = 46,
- [1][1][1][23] = 22,
- [1][1][3][23] = 46,
- [1][1][5][23] = 46,
- [1][1][6][23] = 22,
- [1][1][9][23] = 22,
- [1][1][8][23] = 42,
- [1][1][11][23] = 22,
- [1][1][2][25] = 46,
- [1][1][1][25] = 22,
- [1][1][3][25] = 46,
- [1][1][5][25] = 127,
- [1][1][6][25] = 22,
- [1][1][9][25] = 127,
- [1][1][8][25] = 42,
- [1][1][11][25] = 22,
- [1][1][2][27] = 46,
- [1][1][1][27] = 22,
- [1][1][3][27] = 46,
- [1][1][5][27] = 127,
- [1][1][6][27] = 22,
- [1][1][9][27] = 127,
- [1][1][8][27] = 42,
- [1][1][11][27] = 22,
- [1][1][2][29] = 46,
- [1][1][1][29] = 22,
- [1][1][3][29] = 46,
- [1][1][5][29] = 127,
- [1][1][6][29] = 22,
- [1][1][9][29] = 127,
- [1][1][8][29] = 42,
- [1][1][11][29] = 22,
- [1][1][2][31] = 46,
- [1][1][1][31] = 22,
- [1][1][3][31] = 46,
- [1][1][5][31] = 46,
- [1][1][6][31] = 22,
- [1][1][9][31] = 22,
- [1][1][8][31] = 42,
- [1][1][11][31] = 22,
- [1][1][2][33] = 46,
- [1][1][1][33] = 22,
- [1][1][3][33] = 46,
- [1][1][5][33] = 46,
- [1][1][6][33] = 22,
- [1][1][9][33] = 22,
- [1][1][8][33] = 42,
- [1][1][11][33] = 22,
- [1][1][2][35] = 46,
- [1][1][1][35] = 22,
- [1][1][3][35] = 46,
- [1][1][5][35] = 46,
- [1][1][6][35] = 22,
- [1][1][9][35] = 22,
- [1][1][8][35] = 42,
- [1][1][11][35] = 22,
- [1][1][2][37] = 46,
- [1][1][1][37] = 127,
- [1][1][3][37] = 46,
- [1][1][5][37] = 46,
- [1][1][6][37] = 22,
- [1][1][9][37] = 50,
- [1][1][8][37] = 42,
- [1][1][11][37] = 127,
- [1][1][2][38] = 74,
- [1][1][1][38] = 16,
- [1][1][3][38] = 127,
- [1][1][5][38] = 74,
- [1][1][6][38] = 16,
- [1][1][9][38] = 76,
- [1][1][8][38] = 42,
- [1][1][11][38] = 16,
- [1][1][2][40] = 76,
- [1][1][1][40] = 16,
- [1][1][3][40] = 127,
- [1][1][5][40] = 76,
- [1][1][6][40] = 16,
- [1][1][9][40] = 76,
- [1][1][8][40] = 42,
- [1][1][11][40] = 16,
- [1][1][2][42] = 76,
- [1][1][1][42] = 16,
- [1][1][3][42] = 127,
- [1][1][5][42] = 76,
- [1][1][6][42] = 16,
- [1][1][9][42] = 76,
- [1][1][8][42] = 42,
- [1][1][11][42] = 16,
- [1][1][2][44] = 76,
- [1][1][1][44] = 16,
- [1][1][3][44] = 127,
- [1][1][5][44] = 76,
- [1][1][6][44] = 16,
- [1][1][9][44] = 76,
- [1][1][8][44] = 42,
- [1][1][11][44] = 16,
- [1][1][2][46] = 76,
- [1][1][1][46] = 16,
- [1][1][3][46] = 127,
- [1][1][5][46] = 76,
- [1][1][6][46] = 16,
- [1][1][9][46] = 76,
- [1][1][8][46] = 42,
- [1][1][11][46] = 16,
- [2][0][2][0] = 74,
- [2][0][1][0] = 46,
- [2][0][3][0] = 50,
- [2][0][5][0] = 46,
- [2][0][6][0] = 46,
- [2][0][9][0] = 46,
- [2][0][8][0] = 30,
- [2][0][11][0] = 46,
- [2][0][2][2] = 74,
- [2][0][1][2] = 46,
- [2][0][3][2] = 50,
- [2][0][5][2] = 46,
- [2][0][6][2] = 46,
- [2][0][9][2] = 46,
- [2][0][8][2] = 30,
- [2][0][11][2] = 46,
- [2][0][2][4] = 74,
- [2][0][1][4] = 46,
- [2][0][3][4] = 50,
- [2][0][5][4] = 46,
- [2][0][6][4] = 46,
- [2][0][9][4] = 46,
- [2][0][8][4] = 30,
- [2][0][11][4] = 46,
- [2][0][2][6] = 74,
- [2][0][1][6] = 46,
- [2][0][3][6] = 50,
- [2][0][5][6] = 46,
- [2][0][6][6] = 46,
- [2][0][9][6] = 46,
- [2][0][8][6] = 30,
- [2][0][11][6] = 46,
- [2][0][2][8] = 74,
- [2][0][1][8] = 46,
- [2][0][3][8] = 50,
- [2][0][5][8] = 66,
- [2][0][6][8] = 46,
- [2][0][9][8] = 46,
- [2][0][8][8] = 54,
- [2][0][11][8] = 46,
- [2][0][2][10] = 74,
- [2][0][1][10] = 46,
- [2][0][3][10] = 50,
- [2][0][5][10] = 66,
- [2][0][6][10] = 46,
- [2][0][9][10] = 46,
- [2][0][8][10] = 54,
- [2][0][11][10] = 46,
- [2][0][2][12] = 74,
- [2][0][1][12] = 46,
- [2][0][3][12] = 50,
- [2][0][5][12] = 66,
- [2][0][6][12] = 46,
- [2][0][9][12] = 46,
- [2][0][8][12] = 54,
- [2][0][11][12] = 46,
- [2][0][2][14] = 74,
- [2][0][1][14] = 46,
- [2][0][3][14] = 50,
- [2][0][5][14] = 66,
- [2][0][6][14] = 46,
- [2][0][9][14] = 46,
- [2][0][8][14] = 54,
- [2][0][11][14] = 46,
- [2][0][2][15] = 74,
- [2][0][1][15] = 46,
- [2][0][3][15] = 70,
- [2][0][5][15] = 74,
- [2][0][6][15] = 46,
- [2][0][9][15] = 46,
- [2][0][8][15] = 54,
- [2][0][11][15] = 46,
- [2][0][2][17] = 74,
- [2][0][1][17] = 46,
- [2][0][3][17] = 70,
- [2][0][5][17] = 74,
- [2][0][6][17] = 46,
- [2][0][9][17] = 46,
- [2][0][8][17] = 54,
- [2][0][11][17] = 46,
- [2][0][2][19] = 74,
- [2][0][1][19] = 46,
- [2][0][3][19] = 70,
- [2][0][5][19] = 74,
- [2][0][6][19] = 46,
- [2][0][9][19] = 46,
- [2][0][8][19] = 54,
- [2][0][11][19] = 46,
- [2][0][2][21] = 74,
- [2][0][1][21] = 46,
- [2][0][3][21] = 70,
- [2][0][5][21] = 74,
- [2][0][6][21] = 46,
- [2][0][9][21] = 46,
- [2][0][8][21] = 54,
- [2][0][11][21] = 46,
- [2][0][2][23] = 74,
- [2][0][1][23] = 46,
- [2][0][3][23] = 70,
- [2][0][5][23] = 74,
- [2][0][6][23] = 46,
- [2][0][9][23] = 46,
- [2][0][8][23] = 54,
- [2][0][11][23] = 46,
- [2][0][2][25] = 74,
- [2][0][1][25] = 46,
- [2][0][3][25] = 70,
- [2][0][5][25] = 127,
- [2][0][6][25] = 46,
- [2][0][9][25] = 127,
- [2][0][8][25] = 54,
- [2][0][11][25] = 46,
- [2][0][2][27] = 74,
- [2][0][1][27] = 46,
- [2][0][3][27] = 70,
- [2][0][5][27] = 127,
- [2][0][6][27] = 46,
- [2][0][9][27] = 127,
- [2][0][8][27] = 54,
- [2][0][11][27] = 46,
- [2][0][2][29] = 74,
- [2][0][1][29] = 46,
- [2][0][3][29] = 70,
- [2][0][5][29] = 127,
- [2][0][6][29] = 46,
- [2][0][9][29] = 127,
- [2][0][8][29] = 54,
- [2][0][11][29] = 46,
- [2][0][2][31] = 74,
- [2][0][1][31] = 46,
- [2][0][3][31] = 70,
- [2][0][5][31] = 74,
- [2][0][6][31] = 46,
- [2][0][9][31] = 46,
- [2][0][8][31] = 54,
- [2][0][11][31] = 46,
- [2][0][2][33] = 74,
- [2][0][1][33] = 46,
- [2][0][3][33] = 70,
- [2][0][5][33] = 74,
- [2][0][6][33] = 46,
- [2][0][9][33] = 46,
- [2][0][8][33] = 54,
- [2][0][11][33] = 46,
- [2][0][2][35] = 74,
- [2][0][1][35] = 46,
- [2][0][3][35] = 70,
- [2][0][5][35] = 74,
- [2][0][6][35] = 46,
- [2][0][9][35] = 46,
- [2][0][8][35] = 54,
- [2][0][11][35] = 46,
- [2][0][2][37] = 74,
- [2][0][1][37] = 127,
- [2][0][3][37] = 70,
- [2][0][5][37] = 74,
- [2][0][6][37] = 46,
- [2][0][9][37] = 74,
- [2][0][8][37] = 54,
- [2][0][11][37] = 127,
- [2][0][2][38] = 76,
- [2][0][1][38] = 28,
- [2][0][3][38] = 127,
- [2][0][5][38] = 76,
- [2][0][6][38] = 28,
- [2][0][9][38] = 76,
- [2][0][8][38] = 54,
- [2][0][11][38] = 28,
- [2][0][2][40] = 76,
- [2][0][1][40] = 28,
- [2][0][3][40] = 127,
- [2][0][5][40] = 76,
- [2][0][6][40] = 28,
- [2][0][9][40] = 76,
- [2][0][8][40] = 54,
- [2][0][11][40] = 28,
- [2][0][2][42] = 76,
- [2][0][1][42] = 28,
- [2][0][3][42] = 127,
- [2][0][5][42] = 76,
- [2][0][6][42] = 28,
- [2][0][9][42] = 76,
- [2][0][8][42] = 54,
- [2][0][11][42] = 28,
- [2][0][2][44] = 76,
- [2][0][1][44] = 28,
- [2][0][3][44] = 127,
- [2][0][5][44] = 76,
- [2][0][6][44] = 28,
- [2][0][9][44] = 76,
- [2][0][8][44] = 54,
- [2][0][11][44] = 28,
- [2][0][2][46] = 76,
- [2][0][1][46] = 28,
- [2][0][3][46] = 127,
- [2][0][5][46] = 76,
- [2][0][6][46] = 28,
- [2][0][9][46] = 76,
- [2][0][8][46] = 54,
- [2][0][11][46] = 28,
- [2][1][2][0] = 58,
- [2][1][1][0] = 32,
- [2][1][3][0] = 38,
- [2][1][5][0] = 30,
- [2][1][6][0] = 32,
- [2][1][9][0] = 32,
- [2][1][8][0] = 18,
- [2][1][11][0] = 32,
- [2][1][2][2] = 58,
- [2][1][1][2] = 32,
- [2][1][3][2] = 38,
- [2][1][5][2] = 30,
- [2][1][6][2] = 32,
- [2][1][9][2] = 32,
- [2][1][8][2] = 18,
- [2][1][11][2] = 32,
- [2][1][2][4] = 58,
- [2][1][1][4] = 32,
- [2][1][3][4] = 38,
- [2][1][5][4] = 30,
- [2][1][6][4] = 32,
- [2][1][9][4] = 32,
- [2][1][8][4] = 18,
- [2][1][11][4] = 32,
- [2][1][2][6] = 58,
- [2][1][1][6] = 32,
- [2][1][3][6] = 38,
- [2][1][5][6] = 30,
- [2][1][6][6] = 32,
- [2][1][9][6] = 32,
- [2][1][8][6] = 18,
- [2][1][11][6] = 32,
- [2][1][2][8] = 58,
- [2][1][1][8] = 32,
- [2][1][3][8] = 38,
- [2][1][5][8] = 52,
- [2][1][6][8] = 32,
- [2][1][9][8] = 32,
- [2][1][8][8] = 42,
- [2][1][11][8] = 32,
- [2][1][2][10] = 58,
- [2][1][1][10] = 32,
- [2][1][3][10] = 38,
- [2][1][5][10] = 52,
- [2][1][6][10] = 32,
- [2][1][9][10] = 32,
- [2][1][8][10] = 42,
- [2][1][11][10] = 32,
- [2][1][2][12] = 58,
- [2][1][1][12] = 32,
- [2][1][3][12] = 38,
- [2][1][5][12] = 52,
- [2][1][6][12] = 32,
- [2][1][9][12] = 32,
- [2][1][8][12] = 42,
- [2][1][11][12] = 32,
- [2][1][2][14] = 58,
- [2][1][1][14] = 32,
- [2][1][3][14] = 38,
- [2][1][5][14] = 52,
- [2][1][6][14] = 32,
- [2][1][9][14] = 32,
- [2][1][8][14] = 42,
- [2][1][11][14] = 32,
- [2][1][2][15] = 58,
- [2][1][1][15] = 32,
- [2][1][3][15] = 58,
- [2][1][5][15] = 58,
- [2][1][6][15] = 32,
- [2][1][9][15] = 32,
- [2][1][8][15] = 42,
- [2][1][11][15] = 32,
- [2][1][2][17] = 58,
- [2][1][1][17] = 32,
- [2][1][3][17] = 58,
- [2][1][5][17] = 58,
- [2][1][6][17] = 32,
- [2][1][9][17] = 32,
- [2][1][8][17] = 42,
- [2][1][11][17] = 32,
- [2][1][2][19] = 58,
- [2][1][1][19] = 32,
- [2][1][3][19] = 58,
- [2][1][5][19] = 58,
- [2][1][6][19] = 32,
- [2][1][9][19] = 32,
- [2][1][8][19] = 42,
- [2][1][11][19] = 32,
- [2][1][2][21] = 58,
- [2][1][1][21] = 32,
- [2][1][3][21] = 58,
- [2][1][5][21] = 58,
- [2][1][6][21] = 32,
- [2][1][9][21] = 32,
- [2][1][8][21] = 42,
- [2][1][11][21] = 32,
- [2][1][2][23] = 58,
- [2][1][1][23] = 32,
- [2][1][3][23] = 58,
- [2][1][5][23] = 58,
- [2][1][6][23] = 32,
- [2][1][9][23] = 32,
- [2][1][8][23] = 42,
- [2][1][11][23] = 32,
- [2][1][2][25] = 58,
- [2][1][1][25] = 32,
- [2][1][3][25] = 58,
- [2][1][5][25] = 127,
- [2][1][6][25] = 32,
- [2][1][9][25] = 127,
- [2][1][8][25] = 42,
- [2][1][11][25] = 32,
- [2][1][2][27] = 58,
- [2][1][1][27] = 32,
- [2][1][3][27] = 58,
- [2][1][5][27] = 127,
- [2][1][6][27] = 32,
- [2][1][9][27] = 127,
- [2][1][8][27] = 42,
- [2][1][11][27] = 32,
- [2][1][2][29] = 58,
- [2][1][1][29] = 32,
- [2][1][3][29] = 58,
- [2][1][5][29] = 127,
- [2][1][6][29] = 32,
- [2][1][9][29] = 127,
- [2][1][8][29] = 42,
- [2][1][11][29] = 32,
- [2][1][2][31] = 58,
- [2][1][1][31] = 32,
- [2][1][3][31] = 58,
- [2][1][5][31] = 58,
- [2][1][6][31] = 32,
- [2][1][9][31] = 32,
- [2][1][8][31] = 42,
- [2][1][11][31] = 32,
- [2][1][2][33] = 58,
- [2][1][1][33] = 32,
- [2][1][3][33] = 58,
- [2][1][5][33] = 58,
- [2][1][6][33] = 32,
- [2][1][9][33] = 32,
- [2][1][8][33] = 42,
- [2][1][11][33] = 32,
- [2][1][2][35] = 58,
- [2][1][1][35] = 32,
- [2][1][3][35] = 58,
- [2][1][5][35] = 58,
- [2][1][6][35] = 32,
- [2][1][9][35] = 32,
- [2][1][8][35] = 42,
- [2][1][11][35] = 32,
- [2][1][2][37] = 58,
- [2][1][1][37] = 127,
- [2][1][3][37] = 58,
- [2][1][5][37] = 58,
- [2][1][6][37] = 32,
- [2][1][9][37] = 62,
- [2][1][8][37] = 42,
- [2][1][11][37] = 127,
- [2][1][2][38] = 76,
- [2][1][1][38] = 16,
- [2][1][3][38] = 127,
- [2][1][5][38] = 76,
- [2][1][6][38] = 16,
- [2][1][9][38] = 76,
- [2][1][8][38] = 42,
- [2][1][11][38] = 16,
- [2][1][2][40] = 76,
- [2][1][1][40] = 16,
- [2][1][3][40] = 127,
- [2][1][5][40] = 76,
- [2][1][6][40] = 16,
- [2][1][9][40] = 76,
- [2][1][8][40] = 42,
- [2][1][11][40] = 16,
- [2][1][2][42] = 76,
- [2][1][1][42] = 16,
- [2][1][3][42] = 127,
- [2][1][5][42] = 76,
- [2][1][6][42] = 16,
- [2][1][9][42] = 76,
- [2][1][8][42] = 42,
- [2][1][11][42] = 16,
- [2][1][2][44] = 76,
- [2][1][1][44] = 16,
- [2][1][3][44] = 127,
- [2][1][5][44] = 76,
- [2][1][6][44] = 16,
- [2][1][9][44] = 76,
- [2][1][8][44] = 42,
- [2][1][11][44] = 16,
- [2][1][2][46] = 76,
- [2][1][1][46] = 16,
- [2][1][3][46] = 127,
- [2][1][5][46] = 76,
- [2][1][6][46] = 16,
- [2][1][9][46] = 76,
- [2][1][8][46] = 42,
- [2][1][11][46] = 16,
+ [0][0][RTW89_WW][0] = 22,
+ [0][0][RTW89_WW][2] = 22,
+ [0][0][RTW89_WW][4] = 22,
+ [0][0][RTW89_WW][6] = 20,
+ [0][0][RTW89_WW][8] = 24,
+ [0][0][RTW89_WW][10] = 24,
+ [0][0][RTW89_WW][12] = 24,
+ [0][0][RTW89_WW][14] = 24,
+ [0][0][RTW89_WW][15] = 24,
+ [0][0][RTW89_WW][17] = 24,
+ [0][0][RTW89_WW][19] = 24,
+ [0][0][RTW89_WW][21] = 24,
+ [0][0][RTW89_WW][23] = 24,
+ [0][0][RTW89_WW][25] = 24,
+ [0][0][RTW89_WW][27] = 24,
+ [0][0][RTW89_WW][29] = 24,
+ [0][0][RTW89_WW][31] = 24,
+ [0][0][RTW89_WW][33] = 24,
+ [0][0][RTW89_WW][35] = 24,
+ [0][0][RTW89_WW][37] = 44,
+ [0][0][RTW89_WW][38] = 28,
+ [0][0][RTW89_WW][40] = 28,
+ [0][0][RTW89_WW][42] = 28,
+ [0][0][RTW89_WW][44] = 28,
+ [0][0][RTW89_WW][46] = 28,
+ [0][1][RTW89_WW][0] = 8,
+ [0][1][RTW89_WW][2] = 8,
+ [0][1][RTW89_WW][4] = 8,
+ [0][1][RTW89_WW][6] = 4,
+ [0][1][RTW89_WW][8] = 12,
+ [0][1][RTW89_WW][10] = 12,
+ [0][1][RTW89_WW][12] = 12,
+ [0][1][RTW89_WW][14] = 12,
+ [0][1][RTW89_WW][15] = 12,
+ [0][1][RTW89_WW][17] = 12,
+ [0][1][RTW89_WW][19] = 12,
+ [0][1][RTW89_WW][21] = 12,
+ [0][1][RTW89_WW][23] = 12,
+ [0][1][RTW89_WW][25] = 12,
+ [0][1][RTW89_WW][27] = 12,
+ [0][1][RTW89_WW][29] = 12,
+ [0][1][RTW89_WW][31] = 12,
+ [0][1][RTW89_WW][33] = 12,
+ [0][1][RTW89_WW][35] = 12,
+ [0][1][RTW89_WW][37] = 32,
+ [0][1][RTW89_WW][38] = 16,
+ [0][1][RTW89_WW][40] = 16,
+ [0][1][RTW89_WW][42] = 16,
+ [0][1][RTW89_WW][44] = 16,
+ [0][1][RTW89_WW][46] = 16,
+ [1][0][RTW89_WW][0] = 30,
+ [1][0][RTW89_WW][2] = 30,
+ [1][0][RTW89_WW][4] = 30,
+ [1][0][RTW89_WW][6] = 30,
+ [1][0][RTW89_WW][8] = 36,
+ [1][0][RTW89_WW][10] = 36,
+ [1][0][RTW89_WW][12] = 36,
+ [1][0][RTW89_WW][14] = 36,
+ [1][0][RTW89_WW][15] = 36,
+ [1][0][RTW89_WW][17] = 36,
+ [1][0][RTW89_WW][19] = 36,
+ [1][0][RTW89_WW][21] = 36,
+ [1][0][RTW89_WW][23] = 36,
+ [1][0][RTW89_WW][25] = 36,
+ [1][0][RTW89_WW][27] = 36,
+ [1][0][RTW89_WW][29] = 36,
+ [1][0][RTW89_WW][31] = 36,
+ [1][0][RTW89_WW][33] = 36,
+ [1][0][RTW89_WW][35] = 36,
+ [1][0][RTW89_WW][37] = 54,
+ [1][0][RTW89_WW][38] = 28,
+ [1][0][RTW89_WW][40] = 28,
+ [1][0][RTW89_WW][42] = 28,
+ [1][0][RTW89_WW][44] = 28,
+ [1][0][RTW89_WW][46] = 28,
+ [1][1][RTW89_WW][0] = 18,
+ [1][1][RTW89_WW][2] = 18,
+ [1][1][RTW89_WW][4] = 18,
+ [1][1][RTW89_WW][6] = 16,
+ [1][1][RTW89_WW][8] = 22,
+ [1][1][RTW89_WW][10] = 22,
+ [1][1][RTW89_WW][12] = 22,
+ [1][1][RTW89_WW][14] = 22,
+ [1][1][RTW89_WW][15] = 22,
+ [1][1][RTW89_WW][17] = 22,
+ [1][1][RTW89_WW][19] = 22,
+ [1][1][RTW89_WW][21] = 22,
+ [1][1][RTW89_WW][23] = 22,
+ [1][1][RTW89_WW][25] = 22,
+ [1][1][RTW89_WW][27] = 22,
+ [1][1][RTW89_WW][29] = 22,
+ [1][1][RTW89_WW][31] = 22,
+ [1][1][RTW89_WW][33] = 22,
+ [1][1][RTW89_WW][35] = 22,
+ [1][1][RTW89_WW][37] = 42,
+ [1][1][RTW89_WW][38] = 16,
+ [1][1][RTW89_WW][40] = 16,
+ [1][1][RTW89_WW][42] = 16,
+ [1][1][RTW89_WW][44] = 16,
+ [1][1][RTW89_WW][46] = 16,
+ [2][0][RTW89_WW][0] = 30,
+ [2][0][RTW89_WW][2] = 30,
+ [2][0][RTW89_WW][4] = 30,
+ [2][0][RTW89_WW][6] = 30,
+ [2][0][RTW89_WW][8] = 46,
+ [2][0][RTW89_WW][10] = 46,
+ [2][0][RTW89_WW][12] = 46,
+ [2][0][RTW89_WW][14] = 46,
+ [2][0][RTW89_WW][15] = 46,
+ [2][0][RTW89_WW][17] = 46,
+ [2][0][RTW89_WW][19] = 46,
+ [2][0][RTW89_WW][21] = 46,
+ [2][0][RTW89_WW][23] = 46,
+ [2][0][RTW89_WW][25] = 46,
+ [2][0][RTW89_WW][27] = 46,
+ [2][0][RTW89_WW][29] = 46,
+ [2][0][RTW89_WW][31] = 46,
+ [2][0][RTW89_WW][33] = 46,
+ [2][0][RTW89_WW][35] = 46,
+ [2][0][RTW89_WW][37] = 54,
+ [2][0][RTW89_WW][38] = 28,
+ [2][0][RTW89_WW][40] = 28,
+ [2][0][RTW89_WW][42] = 28,
+ [2][0][RTW89_WW][44] = 28,
+ [2][0][RTW89_WW][46] = 28,
+ [2][1][RTW89_WW][0] = 18,
+ [2][1][RTW89_WW][2] = 18,
+ [2][1][RTW89_WW][4] = 18,
+ [2][1][RTW89_WW][6] = 18,
+ [2][1][RTW89_WW][8] = 32,
+ [2][1][RTW89_WW][10] = 32,
+ [2][1][RTW89_WW][12] = 32,
+ [2][1][RTW89_WW][14] = 32,
+ [2][1][RTW89_WW][15] = 32,
+ [2][1][RTW89_WW][17] = 32,
+ [2][1][RTW89_WW][19] = 32,
+ [2][1][RTW89_WW][21] = 32,
+ [2][1][RTW89_WW][23] = 32,
+ [2][1][RTW89_WW][25] = 32,
+ [2][1][RTW89_WW][27] = 32,
+ [2][1][RTW89_WW][29] = 32,
+ [2][1][RTW89_WW][31] = 32,
+ [2][1][RTW89_WW][33] = 32,
+ [2][1][RTW89_WW][35] = 32,
+ [2][1][RTW89_WW][37] = 42,
+ [2][1][RTW89_WW][38] = 16,
+ [2][1][RTW89_WW][40] = 16,
+ [2][1][RTW89_WW][42] = 16,
+ [2][1][RTW89_WW][44] = 16,
+ [2][1][RTW89_WW][46] = 16,
+ [0][0][RTW89_FCC][0] = 48,
+ [0][0][RTW89_ETSI][0] = 24,
+ [0][0][RTW89_MKK][0] = 26,
+ [0][0][RTW89_IC][0] = 22,
+ [0][0][RTW89_KCC][0] = 46,
+ [0][0][RTW89_ACMA][0] = 24,
+ [0][0][RTW89_CHILE][0] = 30,
+ [0][0][RTW89_UKRAINE][0] = 24,
+ [0][0][RTW89_MEXICO][0] = 48,
+ [0][0][RTW89_CN][0] = 24,
+ [0][0][RTW89_QATAR][0] = 24,
+ [0][0][RTW89_FCC][2] = 48,
+ [0][0][RTW89_ETSI][2] = 24,
+ [0][0][RTW89_MKK][2] = 26,
+ [0][0][RTW89_IC][2] = 22,
+ [0][0][RTW89_KCC][2] = 46,
+ [0][0][RTW89_ACMA][2] = 24,
+ [0][0][RTW89_CHILE][2] = 30,
+ [0][0][RTW89_UKRAINE][2] = 24,
+ [0][0][RTW89_MEXICO][2] = 48,
+ [0][0][RTW89_CN][2] = 24,
+ [0][0][RTW89_QATAR][2] = 24,
+ [0][0][RTW89_FCC][4] = 48,
+ [0][0][RTW89_ETSI][4] = 24,
+ [0][0][RTW89_MKK][4] = 26,
+ [0][0][RTW89_IC][4] = 22,
+ [0][0][RTW89_KCC][4] = 46,
+ [0][0][RTW89_ACMA][4] = 24,
+ [0][0][RTW89_CHILE][4] = 30,
+ [0][0][RTW89_UKRAINE][4] = 24,
+ [0][0][RTW89_MEXICO][4] = 48,
+ [0][0][RTW89_CN][4] = 24,
+ [0][0][RTW89_QATAR][4] = 24,
+ [0][0][RTW89_FCC][6] = 48,
+ [0][0][RTW89_ETSI][6] = 24,
+ [0][0][RTW89_MKK][6] = 26,
+ [0][0][RTW89_IC][6] = 22,
+ [0][0][RTW89_KCC][6] = 20,
+ [0][0][RTW89_ACMA][6] = 24,
+ [0][0][RTW89_CHILE][6] = 30,
+ [0][0][RTW89_UKRAINE][6] = 24,
+ [0][0][RTW89_MEXICO][6] = 48,
+ [0][0][RTW89_CN][6] = 24,
+ [0][0][RTW89_QATAR][6] = 24,
+ [0][0][RTW89_FCC][8] = 48,
+ [0][0][RTW89_ETSI][8] = 24,
+ [0][0][RTW89_MKK][8] = 26,
+ [0][0][RTW89_IC][8] = 48,
+ [0][0][RTW89_KCC][8] = 46,
+ [0][0][RTW89_ACMA][8] = 24,
+ [0][0][RTW89_CHILE][8] = 48,
+ [0][0][RTW89_UKRAINE][8] = 24,
+ [0][0][RTW89_MEXICO][8] = 48,
+ [0][0][RTW89_CN][8] = 24,
+ [0][0][RTW89_QATAR][8] = 24,
+ [0][0][RTW89_FCC][10] = 48,
+ [0][0][RTW89_ETSI][10] = 24,
+ [0][0][RTW89_MKK][10] = 26,
+ [0][0][RTW89_IC][10] = 48,
+ [0][0][RTW89_KCC][10] = 46,
+ [0][0][RTW89_ACMA][10] = 24,
+ [0][0][RTW89_CHILE][10] = 48,
+ [0][0][RTW89_UKRAINE][10] = 24,
+ [0][0][RTW89_MEXICO][10] = 48,
+ [0][0][RTW89_CN][10] = 24,
+ [0][0][RTW89_QATAR][10] = 24,
+ [0][0][RTW89_FCC][12] = 48,
+ [0][0][RTW89_ETSI][12] = 24,
+ [0][0][RTW89_MKK][12] = 26,
+ [0][0][RTW89_IC][12] = 48,
+ [0][0][RTW89_KCC][12] = 46,
+ [0][0][RTW89_ACMA][12] = 24,
+ [0][0][RTW89_CHILE][12] = 48,
+ [0][0][RTW89_UKRAINE][12] = 24,
+ [0][0][RTW89_MEXICO][12] = 48,
+ [0][0][RTW89_CN][12] = 24,
+ [0][0][RTW89_QATAR][12] = 24,
+ [0][0][RTW89_FCC][14] = 48,
+ [0][0][RTW89_ETSI][14] = 24,
+ [0][0][RTW89_MKK][14] = 26,
+ [0][0][RTW89_IC][14] = 48,
+ [0][0][RTW89_KCC][14] = 46,
+ [0][0][RTW89_ACMA][14] = 24,
+ [0][0][RTW89_CHILE][14] = 48,
+ [0][0][RTW89_UKRAINE][14] = 24,
+ [0][0][RTW89_MEXICO][14] = 48,
+ [0][0][RTW89_CN][14] = 24,
+ [0][0][RTW89_QATAR][14] = 24,
+ [0][0][RTW89_FCC][15] = 48,
+ [0][0][RTW89_ETSI][15] = 24,
+ [0][0][RTW89_MKK][15] = 44,
+ [0][0][RTW89_IC][15] = 48,
+ [0][0][RTW89_KCC][15] = 46,
+ [0][0][RTW89_ACMA][15] = 24,
+ [0][0][RTW89_CHILE][15] = 48,
+ [0][0][RTW89_UKRAINE][15] = 24,
+ [0][0][RTW89_MEXICO][15] = 48,
+ [0][0][RTW89_CN][15] = 127,
+ [0][0][RTW89_QATAR][15] = 24,
+ [0][0][RTW89_FCC][17] = 48,
+ [0][0][RTW89_ETSI][17] = 24,
+ [0][0][RTW89_MKK][17] = 44,
+ [0][0][RTW89_IC][17] = 48,
+ [0][0][RTW89_KCC][17] = 46,
+ [0][0][RTW89_ACMA][17] = 24,
+ [0][0][RTW89_CHILE][17] = 48,
+ [0][0][RTW89_UKRAINE][17] = 24,
+ [0][0][RTW89_MEXICO][17] = 48,
+ [0][0][RTW89_CN][17] = 127,
+ [0][0][RTW89_QATAR][17] = 24,
+ [0][0][RTW89_FCC][19] = 48,
+ [0][0][RTW89_ETSI][19] = 24,
+ [0][0][RTW89_MKK][19] = 44,
+ [0][0][RTW89_IC][19] = 48,
+ [0][0][RTW89_KCC][19] = 46,
+ [0][0][RTW89_ACMA][19] = 24,
+ [0][0][RTW89_CHILE][19] = 48,
+ [0][0][RTW89_UKRAINE][19] = 24,
+ [0][0][RTW89_MEXICO][19] = 48,
+ [0][0][RTW89_CN][19] = 127,
+ [0][0][RTW89_QATAR][19] = 24,
+ [0][0][RTW89_FCC][21] = 48,
+ [0][0][RTW89_ETSI][21] = 24,
+ [0][0][RTW89_MKK][21] = 44,
+ [0][0][RTW89_IC][21] = 48,
+ [0][0][RTW89_KCC][21] = 46,
+ [0][0][RTW89_ACMA][21] = 24,
+ [0][0][RTW89_CHILE][21] = 48,
+ [0][0][RTW89_UKRAINE][21] = 24,
+ [0][0][RTW89_MEXICO][21] = 48,
+ [0][0][RTW89_CN][21] = 127,
+ [0][0][RTW89_QATAR][21] = 24,
+ [0][0][RTW89_FCC][23] = 48,
+ [0][0][RTW89_ETSI][23] = 24,
+ [0][0][RTW89_MKK][23] = 44,
+ [0][0][RTW89_IC][23] = 48,
+ [0][0][RTW89_KCC][23] = 46,
+ [0][0][RTW89_ACMA][23] = 24,
+ [0][0][RTW89_CHILE][23] = 48,
+ [0][0][RTW89_UKRAINE][23] = 24,
+ [0][0][RTW89_MEXICO][23] = 48,
+ [0][0][RTW89_CN][23] = 127,
+ [0][0][RTW89_QATAR][23] = 24,
+ [0][0][RTW89_FCC][25] = 48,
+ [0][0][RTW89_ETSI][25] = 24,
+ [0][0][RTW89_MKK][25] = 44,
+ [0][0][RTW89_IC][25] = 127,
+ [0][0][RTW89_KCC][25] = 46,
+ [0][0][RTW89_ACMA][25] = 127,
+ [0][0][RTW89_CHILE][25] = 48,
+ [0][0][RTW89_UKRAINE][25] = 24,
+ [0][0][RTW89_MEXICO][25] = 48,
+ [0][0][RTW89_CN][25] = 127,
+ [0][0][RTW89_QATAR][25] = 24,
+ [0][0][RTW89_FCC][27] = 48,
+ [0][0][RTW89_ETSI][27] = 24,
+ [0][0][RTW89_MKK][27] = 44,
+ [0][0][RTW89_IC][27] = 127,
+ [0][0][RTW89_KCC][27] = 46,
+ [0][0][RTW89_ACMA][27] = 127,
+ [0][0][RTW89_CHILE][27] = 48,
+ [0][0][RTW89_UKRAINE][27] = 24,
+ [0][0][RTW89_MEXICO][27] = 48,
+ [0][0][RTW89_CN][27] = 127,
+ [0][0][RTW89_QATAR][27] = 24,
+ [0][0][RTW89_FCC][29] = 48,
+ [0][0][RTW89_ETSI][29] = 24,
+ [0][0][RTW89_MKK][29] = 44,
+ [0][0][RTW89_IC][29] = 127,
+ [0][0][RTW89_KCC][29] = 46,
+ [0][0][RTW89_ACMA][29] = 127,
+ [0][0][RTW89_CHILE][29] = 48,
+ [0][0][RTW89_UKRAINE][29] = 24,
+ [0][0][RTW89_MEXICO][29] = 48,
+ [0][0][RTW89_CN][29] = 127,
+ [0][0][RTW89_QATAR][29] = 24,
+ [0][0][RTW89_FCC][31] = 48,
+ [0][0][RTW89_ETSI][31] = 24,
+ [0][0][RTW89_MKK][31] = 44,
+ [0][0][RTW89_IC][31] = 48,
+ [0][0][RTW89_KCC][31] = 46,
+ [0][0][RTW89_ACMA][31] = 24,
+ [0][0][RTW89_CHILE][31] = 48,
+ [0][0][RTW89_UKRAINE][31] = 24,
+ [0][0][RTW89_MEXICO][31] = 48,
+ [0][0][RTW89_CN][31] = 127,
+ [0][0][RTW89_QATAR][31] = 24,
+ [0][0][RTW89_FCC][33] = 48,
+ [0][0][RTW89_ETSI][33] = 24,
+ [0][0][RTW89_MKK][33] = 44,
+ [0][0][RTW89_IC][33] = 48,
+ [0][0][RTW89_KCC][33] = 46,
+ [0][0][RTW89_ACMA][33] = 24,
+ [0][0][RTW89_CHILE][33] = 48,
+ [0][0][RTW89_UKRAINE][33] = 24,
+ [0][0][RTW89_MEXICO][33] = 48,
+ [0][0][RTW89_CN][33] = 127,
+ [0][0][RTW89_QATAR][33] = 24,
+ [0][0][RTW89_FCC][35] = 48,
+ [0][0][RTW89_ETSI][35] = 24,
+ [0][0][RTW89_MKK][35] = 44,
+ [0][0][RTW89_IC][35] = 48,
+ [0][0][RTW89_KCC][35] = 46,
+ [0][0][RTW89_ACMA][35] = 24,
+ [0][0][RTW89_CHILE][35] = 48,
+ [0][0][RTW89_UKRAINE][35] = 24,
+ [0][0][RTW89_MEXICO][35] = 48,
+ [0][0][RTW89_CN][35] = 127,
+ [0][0][RTW89_QATAR][35] = 24,
+ [0][0][RTW89_FCC][37] = 48,
+ [0][0][RTW89_ETSI][37] = 127,
+ [0][0][RTW89_MKK][37] = 44,
+ [0][0][RTW89_IC][37] = 48,
+ [0][0][RTW89_KCC][37] = 46,
+ [0][0][RTW89_ACMA][37] = 48,
+ [0][0][RTW89_CHILE][37] = 48,
+ [0][0][RTW89_UKRAINE][37] = 127,
+ [0][0][RTW89_MEXICO][37] = 48,
+ [0][0][RTW89_CN][37] = 127,
+ [0][0][RTW89_QATAR][37] = 127,
+ [0][0][RTW89_FCC][38] = 76,
+ [0][0][RTW89_ETSI][38] = 28,
+ [0][0][RTW89_MKK][38] = 127,
+ [0][0][RTW89_IC][38] = 76,
+ [0][0][RTW89_KCC][38] = 46,
+ [0][0][RTW89_ACMA][38] = 76,
+ [0][0][RTW89_CHILE][38] = 54,
+ [0][0][RTW89_UKRAINE][38] = 28,
+ [0][0][RTW89_MEXICO][38] = 76,
+ [0][0][RTW89_CN][38] = 62,
+ [0][0][RTW89_QATAR][38] = 28,
+ [0][0][RTW89_FCC][40] = 76,
+ [0][0][RTW89_ETSI][40] = 28,
+ [0][0][RTW89_MKK][40] = 127,
+ [0][0][RTW89_IC][40] = 76,
+ [0][0][RTW89_KCC][40] = 46,
+ [0][0][RTW89_ACMA][40] = 76,
+ [0][0][RTW89_CHILE][40] = 54,
+ [0][0][RTW89_UKRAINE][40] = 28,
+ [0][0][RTW89_MEXICO][40] = 76,
+ [0][0][RTW89_CN][40] = 62,
+ [0][0][RTW89_QATAR][40] = 28,
+ [0][0][RTW89_FCC][42] = 76,
+ [0][0][RTW89_ETSI][42] = 28,
+ [0][0][RTW89_MKK][42] = 127,
+ [0][0][RTW89_IC][42] = 76,
+ [0][0][RTW89_KCC][42] = 46,
+ [0][0][RTW89_ACMA][42] = 76,
+ [0][0][RTW89_CHILE][42] = 54,
+ [0][0][RTW89_UKRAINE][42] = 28,
+ [0][0][RTW89_MEXICO][42] = 76,
+ [0][0][RTW89_CN][42] = 62,
+ [0][0][RTW89_QATAR][42] = 28,
+ [0][0][RTW89_FCC][44] = 76,
+ [0][0][RTW89_ETSI][44] = 28,
+ [0][0][RTW89_MKK][44] = 127,
+ [0][0][RTW89_IC][44] = 76,
+ [0][0][RTW89_KCC][44] = 46,
+ [0][0][RTW89_ACMA][44] = 76,
+ [0][0][RTW89_CHILE][44] = 54,
+ [0][0][RTW89_UKRAINE][44] = 28,
+ [0][0][RTW89_MEXICO][44] = 76,
+ [0][0][RTW89_CN][44] = 62,
+ [0][0][RTW89_QATAR][44] = 28,
+ [0][0][RTW89_FCC][46] = 76,
+ [0][0][RTW89_ETSI][46] = 28,
+ [0][0][RTW89_MKK][46] = 127,
+ [0][0][RTW89_IC][46] = 76,
+ [0][0][RTW89_KCC][46] = 46,
+ [0][0][RTW89_ACMA][46] = 76,
+ [0][0][RTW89_CHILE][46] = 54,
+ [0][0][RTW89_UKRAINE][46] = 28,
+ [0][0][RTW89_MEXICO][46] = 76,
+ [0][0][RTW89_CN][46] = 62,
+ [0][0][RTW89_QATAR][46] = 28,
+ [0][1][RTW89_FCC][0] = 36,
+ [0][1][RTW89_ETSI][0] = 12,
+ [0][1][RTW89_MKK][0] = 14,
+ [0][1][RTW89_IC][0] = 8,
+ [0][1][RTW89_KCC][0] = 32,
+ [0][1][RTW89_ACMA][0] = 12,
+ [0][1][RTW89_CHILE][0] = 18,
+ [0][1][RTW89_UKRAINE][0] = 12,
+ [0][1][RTW89_MEXICO][0] = 36,
+ [0][1][RTW89_CN][0] = 12,
+ [0][1][RTW89_QATAR][0] = 12,
+ [0][1][RTW89_FCC][2] = 36,
+ [0][1][RTW89_ETSI][2] = 12,
+ [0][1][RTW89_MKK][2] = 14,
+ [0][1][RTW89_IC][2] = 8,
+ [0][1][RTW89_KCC][2] = 32,
+ [0][1][RTW89_ACMA][2] = 12,
+ [0][1][RTW89_CHILE][2] = 18,
+ [0][1][RTW89_UKRAINE][2] = 12,
+ [0][1][RTW89_MEXICO][2] = 36,
+ [0][1][RTW89_CN][2] = 12,
+ [0][1][RTW89_QATAR][2] = 12,
+ [0][1][RTW89_FCC][4] = 36,
+ [0][1][RTW89_ETSI][4] = 12,
+ [0][1][RTW89_MKK][4] = 14,
+ [0][1][RTW89_IC][4] = 8,
+ [0][1][RTW89_KCC][4] = 32,
+ [0][1][RTW89_ACMA][4] = 12,
+ [0][1][RTW89_CHILE][4] = 18,
+ [0][1][RTW89_UKRAINE][4] = 12,
+ [0][1][RTW89_MEXICO][4] = 36,
+ [0][1][RTW89_CN][4] = 12,
+ [0][1][RTW89_QATAR][4] = 12,
+ [0][1][RTW89_FCC][6] = 36,
+ [0][1][RTW89_ETSI][6] = 12,
+ [0][1][RTW89_MKK][6] = 14,
+ [0][1][RTW89_IC][6] = 8,
+ [0][1][RTW89_KCC][6] = 4,
+ [0][1][RTW89_ACMA][6] = 12,
+ [0][1][RTW89_CHILE][6] = 18,
+ [0][1][RTW89_UKRAINE][6] = 12,
+ [0][1][RTW89_MEXICO][6] = 36,
+ [0][1][RTW89_CN][6] = 12,
+ [0][1][RTW89_QATAR][6] = 12,
+ [0][1][RTW89_FCC][8] = 36,
+ [0][1][RTW89_ETSI][8] = 12,
+ [0][1][RTW89_MKK][8] = 14,
+ [0][1][RTW89_IC][8] = 36,
+ [0][1][RTW89_KCC][8] = 32,
+ [0][1][RTW89_ACMA][8] = 12,
+ [0][1][RTW89_CHILE][8] = 36,
+ [0][1][RTW89_UKRAINE][8] = 12,
+ [0][1][RTW89_MEXICO][8] = 36,
+ [0][1][RTW89_CN][8] = 12,
+ [0][1][RTW89_QATAR][8] = 12,
+ [0][1][RTW89_FCC][10] = 36,
+ [0][1][RTW89_ETSI][10] = 12,
+ [0][1][RTW89_MKK][10] = 14,
+ [0][1][RTW89_IC][10] = 36,
+ [0][1][RTW89_KCC][10] = 32,
+ [0][1][RTW89_ACMA][10] = 12,
+ [0][1][RTW89_CHILE][10] = 36,
+ [0][1][RTW89_UKRAINE][10] = 12,
+ [0][1][RTW89_MEXICO][10] = 36,
+ [0][1][RTW89_CN][10] = 12,
+ [0][1][RTW89_QATAR][10] = 12,
+ [0][1][RTW89_FCC][12] = 36,
+ [0][1][RTW89_ETSI][12] = 12,
+ [0][1][RTW89_MKK][12] = 14,
+ [0][1][RTW89_IC][12] = 36,
+ [0][1][RTW89_KCC][12] = 32,
+ [0][1][RTW89_ACMA][12] = 12,
+ [0][1][RTW89_CHILE][12] = 36,
+ [0][1][RTW89_UKRAINE][12] = 12,
+ [0][1][RTW89_MEXICO][12] = 36,
+ [0][1][RTW89_CN][12] = 12,
+ [0][1][RTW89_QATAR][12] = 12,
+ [0][1][RTW89_FCC][14] = 36,
+ [0][1][RTW89_ETSI][14] = 12,
+ [0][1][RTW89_MKK][14] = 14,
+ [0][1][RTW89_IC][14] = 36,
+ [0][1][RTW89_KCC][14] = 32,
+ [0][1][RTW89_ACMA][14] = 12,
+ [0][1][RTW89_CHILE][14] = 36,
+ [0][1][RTW89_UKRAINE][14] = 12,
+ [0][1][RTW89_MEXICO][14] = 36,
+ [0][1][RTW89_CN][14] = 12,
+ [0][1][RTW89_QATAR][14] = 12,
+ [0][1][RTW89_FCC][15] = 36,
+ [0][1][RTW89_ETSI][15] = 12,
+ [0][1][RTW89_MKK][15] = 32,
+ [0][1][RTW89_IC][15] = 36,
+ [0][1][RTW89_KCC][15] = 32,
+ [0][1][RTW89_ACMA][15] = 12,
+ [0][1][RTW89_CHILE][15] = 36,
+ [0][1][RTW89_UKRAINE][15] = 12,
+ [0][1][RTW89_MEXICO][15] = 36,
+ [0][1][RTW89_CN][15] = 127,
+ [0][1][RTW89_QATAR][15] = 12,
+ [0][1][RTW89_FCC][17] = 36,
+ [0][1][RTW89_ETSI][17] = 12,
+ [0][1][RTW89_MKK][17] = 32,
+ [0][1][RTW89_IC][17] = 36,
+ [0][1][RTW89_KCC][17] = 32,
+ [0][1][RTW89_ACMA][17] = 12,
+ [0][1][RTW89_CHILE][17] = 36,
+ [0][1][RTW89_UKRAINE][17] = 12,
+ [0][1][RTW89_MEXICO][17] = 36,
+ [0][1][RTW89_CN][17] = 127,
+ [0][1][RTW89_QATAR][17] = 12,
+ [0][1][RTW89_FCC][19] = 36,
+ [0][1][RTW89_ETSI][19] = 12,
+ [0][1][RTW89_MKK][19] = 32,
+ [0][1][RTW89_IC][19] = 36,
+ [0][1][RTW89_KCC][19] = 32,
+ [0][1][RTW89_ACMA][19] = 12,
+ [0][1][RTW89_CHILE][19] = 36,
+ [0][1][RTW89_UKRAINE][19] = 12,
+ [0][1][RTW89_MEXICO][19] = 36,
+ [0][1][RTW89_CN][19] = 127,
+ [0][1][RTW89_QATAR][19] = 12,
+ [0][1][RTW89_FCC][21] = 36,
+ [0][1][RTW89_ETSI][21] = 12,
+ [0][1][RTW89_MKK][21] = 32,
+ [0][1][RTW89_IC][21] = 36,
+ [0][1][RTW89_KCC][21] = 32,
+ [0][1][RTW89_ACMA][21] = 12,
+ [0][1][RTW89_CHILE][21] = 36,
+ [0][1][RTW89_UKRAINE][21] = 12,
+ [0][1][RTW89_MEXICO][21] = 36,
+ [0][1][RTW89_CN][21] = 127,
+ [0][1][RTW89_QATAR][21] = 12,
+ [0][1][RTW89_FCC][23] = 36,
+ [0][1][RTW89_ETSI][23] = 12,
+ [0][1][RTW89_MKK][23] = 32,
+ [0][1][RTW89_IC][23] = 36,
+ [0][1][RTW89_KCC][23] = 32,
+ [0][1][RTW89_ACMA][23] = 12,
+ [0][1][RTW89_CHILE][23] = 36,
+ [0][1][RTW89_UKRAINE][23] = 12,
+ [0][1][RTW89_MEXICO][23] = 36,
+ [0][1][RTW89_CN][23] = 127,
+ [0][1][RTW89_QATAR][23] = 12,
+ [0][1][RTW89_FCC][25] = 36,
+ [0][1][RTW89_ETSI][25] = 12,
+ [0][1][RTW89_MKK][25] = 32,
+ [0][1][RTW89_IC][25] = 127,
+ [0][1][RTW89_KCC][25] = 32,
+ [0][1][RTW89_ACMA][25] = 127,
+ [0][1][RTW89_CHILE][25] = 36,
+ [0][1][RTW89_UKRAINE][25] = 12,
+ [0][1][RTW89_MEXICO][25] = 36,
+ [0][1][RTW89_CN][25] = 127,
+ [0][1][RTW89_QATAR][25] = 12,
+ [0][1][RTW89_FCC][27] = 36,
+ [0][1][RTW89_ETSI][27] = 12,
+ [0][1][RTW89_MKK][27] = 32,
+ [0][1][RTW89_IC][27] = 127,
+ [0][1][RTW89_KCC][27] = 32,
+ [0][1][RTW89_ACMA][27] = 127,
+ [0][1][RTW89_CHILE][27] = 36,
+ [0][1][RTW89_UKRAINE][27] = 12,
+ [0][1][RTW89_MEXICO][27] = 36,
+ [0][1][RTW89_CN][27] = 127,
+ [0][1][RTW89_QATAR][27] = 12,
+ [0][1][RTW89_FCC][29] = 36,
+ [0][1][RTW89_ETSI][29] = 12,
+ [0][1][RTW89_MKK][29] = 32,
+ [0][1][RTW89_IC][29] = 127,
+ [0][1][RTW89_KCC][29] = 32,
+ [0][1][RTW89_ACMA][29] = 127,
+ [0][1][RTW89_CHILE][29] = 36,
+ [0][1][RTW89_UKRAINE][29] = 12,
+ [0][1][RTW89_MEXICO][29] = 36,
+ [0][1][RTW89_CN][29] = 127,
+ [0][1][RTW89_QATAR][29] = 12,
+ [0][1][RTW89_FCC][31] = 36,
+ [0][1][RTW89_ETSI][31] = 12,
+ [0][1][RTW89_MKK][31] = 32,
+ [0][1][RTW89_IC][31] = 36,
+ [0][1][RTW89_KCC][31] = 32,
+ [0][1][RTW89_ACMA][31] = 12,
+ [0][1][RTW89_CHILE][31] = 36,
+ [0][1][RTW89_UKRAINE][31] = 12,
+ [0][1][RTW89_MEXICO][31] = 36,
+ [0][1][RTW89_CN][31] = 127,
+ [0][1][RTW89_QATAR][31] = 12,
+ [0][1][RTW89_FCC][33] = 36,
+ [0][1][RTW89_ETSI][33] = 12,
+ [0][1][RTW89_MKK][33] = 32,
+ [0][1][RTW89_IC][33] = 36,
+ [0][1][RTW89_KCC][33] = 32,
+ [0][1][RTW89_ACMA][33] = 12,
+ [0][1][RTW89_CHILE][33] = 36,
+ [0][1][RTW89_UKRAINE][33] = 12,
+ [0][1][RTW89_MEXICO][33] = 36,
+ [0][1][RTW89_CN][33] = 127,
+ [0][1][RTW89_QATAR][33] = 12,
+ [0][1][RTW89_FCC][35] = 36,
+ [0][1][RTW89_ETSI][35] = 12,
+ [0][1][RTW89_MKK][35] = 32,
+ [0][1][RTW89_IC][35] = 36,
+ [0][1][RTW89_KCC][35] = 32,
+ [0][1][RTW89_ACMA][35] = 12,
+ [0][1][RTW89_CHILE][35] = 36,
+ [0][1][RTW89_UKRAINE][35] = 12,
+ [0][1][RTW89_MEXICO][35] = 36,
+ [0][1][RTW89_CN][35] = 127,
+ [0][1][RTW89_QATAR][35] = 12,
+ [0][1][RTW89_FCC][37] = 36,
+ [0][1][RTW89_ETSI][37] = 127,
+ [0][1][RTW89_MKK][37] = 32,
+ [0][1][RTW89_IC][37] = 36,
+ [0][1][RTW89_KCC][37] = 32,
+ [0][1][RTW89_ACMA][37] = 36,
+ [0][1][RTW89_CHILE][37] = 36,
+ [0][1][RTW89_UKRAINE][37] = 127,
+ [0][1][RTW89_MEXICO][37] = 36,
+ [0][1][RTW89_CN][37] = 127,
+ [0][1][RTW89_QATAR][37] = 127,
+ [0][1][RTW89_FCC][38] = 72,
+ [0][1][RTW89_ETSI][38] = 16,
+ [0][1][RTW89_MKK][38] = 127,
+ [0][1][RTW89_IC][38] = 72,
+ [0][1][RTW89_KCC][38] = 32,
+ [0][1][RTW89_ACMA][38] = 76,
+ [0][1][RTW89_CHILE][38] = 42,
+ [0][1][RTW89_UKRAINE][38] = 16,
+ [0][1][RTW89_MEXICO][38] = 72,
+ [0][1][RTW89_CN][38] = 50,
+ [0][1][RTW89_QATAR][38] = 16,
+ [0][1][RTW89_FCC][40] = 76,
+ [0][1][RTW89_ETSI][40] = 16,
+ [0][1][RTW89_MKK][40] = 127,
+ [0][1][RTW89_IC][40] = 76,
+ [0][1][RTW89_KCC][40] = 32,
+ [0][1][RTW89_ACMA][40] = 76,
+ [0][1][RTW89_CHILE][40] = 42,
+ [0][1][RTW89_UKRAINE][40] = 16,
+ [0][1][RTW89_MEXICO][40] = 76,
+ [0][1][RTW89_CN][40] = 50,
+ [0][1][RTW89_QATAR][40] = 16,
+ [0][1][RTW89_FCC][42] = 76,
+ [0][1][RTW89_ETSI][42] = 16,
+ [0][1][RTW89_MKK][42] = 127,
+ [0][1][RTW89_IC][42] = 76,
+ [0][1][RTW89_KCC][42] = 32,
+ [0][1][RTW89_ACMA][42] = 76,
+ [0][1][RTW89_CHILE][42] = 42,
+ [0][1][RTW89_UKRAINE][42] = 16,
+ [0][1][RTW89_MEXICO][42] = 76,
+ [0][1][RTW89_CN][42] = 50,
+ [0][1][RTW89_QATAR][42] = 16,
+ [0][1][RTW89_FCC][44] = 76,
+ [0][1][RTW89_ETSI][44] = 16,
+ [0][1][RTW89_MKK][44] = 127,
+ [0][1][RTW89_IC][44] = 76,
+ [0][1][RTW89_KCC][44] = 32,
+ [0][1][RTW89_ACMA][44] = 76,
+ [0][1][RTW89_CHILE][44] = 42,
+ [0][1][RTW89_UKRAINE][44] = 16,
+ [0][1][RTW89_MEXICO][44] = 76,
+ [0][1][RTW89_CN][44] = 50,
+ [0][1][RTW89_QATAR][44] = 16,
+ [0][1][RTW89_FCC][46] = 76,
+ [0][1][RTW89_ETSI][46] = 16,
+ [0][1][RTW89_MKK][46] = 127,
+ [0][1][RTW89_IC][46] = 76,
+ [0][1][RTW89_KCC][46] = 32,
+ [0][1][RTW89_ACMA][46] = 76,
+ [0][1][RTW89_CHILE][46] = 42,
+ [0][1][RTW89_UKRAINE][46] = 16,
+ [0][1][RTW89_MEXICO][46] = 76,
+ [0][1][RTW89_CN][46] = 50,
+ [0][1][RTW89_QATAR][46] = 16,
+ [1][0][RTW89_FCC][0] = 62,
+ [1][0][RTW89_ETSI][0] = 36,
+ [1][0][RTW89_MKK][0] = 36,
+ [1][0][RTW89_IC][0] = 34,
+ [1][0][RTW89_KCC][0] = 58,
+ [1][0][RTW89_ACMA][0] = 36,
+ [1][0][RTW89_CHILE][0] = 30,
+ [1][0][RTW89_UKRAINE][0] = 36,
+ [1][0][RTW89_MEXICO][0] = 62,
+ [1][0][RTW89_CN][0] = 36,
+ [1][0][RTW89_QATAR][0] = 36,
+ [1][0][RTW89_FCC][2] = 62,
+ [1][0][RTW89_ETSI][2] = 36,
+ [1][0][RTW89_MKK][2] = 36,
+ [1][0][RTW89_IC][2] = 34,
+ [1][0][RTW89_KCC][2] = 58,
+ [1][0][RTW89_ACMA][2] = 36,
+ [1][0][RTW89_CHILE][2] = 30,
+ [1][0][RTW89_UKRAINE][2] = 36,
+ [1][0][RTW89_MEXICO][2] = 62,
+ [1][0][RTW89_CN][2] = 36,
+ [1][0][RTW89_QATAR][2] = 36,
+ [1][0][RTW89_FCC][4] = 62,
+ [1][0][RTW89_ETSI][4] = 36,
+ [1][0][RTW89_MKK][4] = 36,
+ [1][0][RTW89_IC][4] = 34,
+ [1][0][RTW89_KCC][4] = 58,
+ [1][0][RTW89_ACMA][4] = 36,
+ [1][0][RTW89_CHILE][4] = 30,
+ [1][0][RTW89_UKRAINE][4] = 36,
+ [1][0][RTW89_MEXICO][4] = 62,
+ [1][0][RTW89_CN][4] = 36,
+ [1][0][RTW89_QATAR][4] = 36,
+ [1][0][RTW89_FCC][6] = 62,
+ [1][0][RTW89_ETSI][6] = 36,
+ [1][0][RTW89_MKK][6] = 36,
+ [1][0][RTW89_IC][6] = 34,
+ [1][0][RTW89_KCC][6] = 32,
+ [1][0][RTW89_ACMA][6] = 36,
+ [1][0][RTW89_CHILE][6] = 30,
+ [1][0][RTW89_UKRAINE][6] = 36,
+ [1][0][RTW89_MEXICO][6] = 62,
+ [1][0][RTW89_CN][6] = 36,
+ [1][0][RTW89_QATAR][6] = 36,
+ [1][0][RTW89_FCC][8] = 62,
+ [1][0][RTW89_ETSI][8] = 36,
+ [1][0][RTW89_MKK][8] = 36,
+ [1][0][RTW89_IC][8] = 62,
+ [1][0][RTW89_KCC][8] = 58,
+ [1][0][RTW89_ACMA][8] = 36,
+ [1][0][RTW89_CHILE][8] = 54,
+ [1][0][RTW89_UKRAINE][8] = 36,
+ [1][0][RTW89_MEXICO][8] = 62,
+ [1][0][RTW89_CN][8] = 36,
+ [1][0][RTW89_QATAR][8] = 36,
+ [1][0][RTW89_FCC][10] = 62,
+ [1][0][RTW89_ETSI][10] = 36,
+ [1][0][RTW89_MKK][10] = 36,
+ [1][0][RTW89_IC][10] = 62,
+ [1][0][RTW89_KCC][10] = 58,
+ [1][0][RTW89_ACMA][10] = 36,
+ [1][0][RTW89_CHILE][10] = 54,
+ [1][0][RTW89_UKRAINE][10] = 36,
+ [1][0][RTW89_MEXICO][10] = 62,
+ [1][0][RTW89_CN][10] = 36,
+ [1][0][RTW89_QATAR][10] = 36,
+ [1][0][RTW89_FCC][12] = 62,
+ [1][0][RTW89_ETSI][12] = 36,
+ [1][0][RTW89_MKK][12] = 36,
+ [1][0][RTW89_IC][12] = 62,
+ [1][0][RTW89_KCC][12] = 58,
+ [1][0][RTW89_ACMA][12] = 36,
+ [1][0][RTW89_CHILE][12] = 54,
+ [1][0][RTW89_UKRAINE][12] = 36,
+ [1][0][RTW89_MEXICO][12] = 62,
+ [1][0][RTW89_CN][12] = 36,
+ [1][0][RTW89_QATAR][12] = 36,
+ [1][0][RTW89_FCC][14] = 62,
+ [1][0][RTW89_ETSI][14] = 36,
+ [1][0][RTW89_MKK][14] = 36,
+ [1][0][RTW89_IC][14] = 62,
+ [1][0][RTW89_KCC][14] = 58,
+ [1][0][RTW89_ACMA][14] = 36,
+ [1][0][RTW89_CHILE][14] = 54,
+ [1][0][RTW89_UKRAINE][14] = 36,
+ [1][0][RTW89_MEXICO][14] = 62,
+ [1][0][RTW89_CN][14] = 36,
+ [1][0][RTW89_QATAR][14] = 36,
+ [1][0][RTW89_FCC][15] = 62,
+ [1][0][RTW89_ETSI][15] = 36,
+ [1][0][RTW89_MKK][15] = 58,
+ [1][0][RTW89_IC][15] = 62,
+ [1][0][RTW89_KCC][15] = 58,
+ [1][0][RTW89_ACMA][15] = 36,
+ [1][0][RTW89_CHILE][15] = 54,
+ [1][0][RTW89_UKRAINE][15] = 36,
+ [1][0][RTW89_MEXICO][15] = 62,
+ [1][0][RTW89_CN][15] = 127,
+ [1][0][RTW89_QATAR][15] = 36,
+ [1][0][RTW89_FCC][17] = 62,
+ [1][0][RTW89_ETSI][17] = 36,
+ [1][0][RTW89_MKK][17] = 58,
+ [1][0][RTW89_IC][17] = 62,
+ [1][0][RTW89_KCC][17] = 58,
+ [1][0][RTW89_ACMA][17] = 36,
+ [1][0][RTW89_CHILE][17] = 54,
+ [1][0][RTW89_UKRAINE][17] = 36,
+ [1][0][RTW89_MEXICO][17] = 62,
+ [1][0][RTW89_CN][17] = 127,
+ [1][0][RTW89_QATAR][17] = 36,
+ [1][0][RTW89_FCC][19] = 62,
+ [1][0][RTW89_ETSI][19] = 36,
+ [1][0][RTW89_MKK][19] = 58,
+ [1][0][RTW89_IC][19] = 62,
+ [1][0][RTW89_KCC][19] = 58,
+ [1][0][RTW89_ACMA][19] = 36,
+ [1][0][RTW89_CHILE][19] = 54,
+ [1][0][RTW89_UKRAINE][19] = 36,
+ [1][0][RTW89_MEXICO][19] = 62,
+ [1][0][RTW89_CN][19] = 127,
+ [1][0][RTW89_QATAR][19] = 36,
+ [1][0][RTW89_FCC][21] = 62,
+ [1][0][RTW89_ETSI][21] = 36,
+ [1][0][RTW89_MKK][21] = 58,
+ [1][0][RTW89_IC][21] = 62,
+ [1][0][RTW89_KCC][21] = 58,
+ [1][0][RTW89_ACMA][21] = 36,
+ [1][0][RTW89_CHILE][21] = 54,
+ [1][0][RTW89_UKRAINE][21] = 36,
+ [1][0][RTW89_MEXICO][21] = 62,
+ [1][0][RTW89_CN][21] = 127,
+ [1][0][RTW89_QATAR][21] = 36,
+ [1][0][RTW89_FCC][23] = 62,
+ [1][0][RTW89_ETSI][23] = 36,
+ [1][0][RTW89_MKK][23] = 58,
+ [1][0][RTW89_IC][23] = 62,
+ [1][0][RTW89_KCC][23] = 58,
+ [1][0][RTW89_ACMA][23] = 36,
+ [1][0][RTW89_CHILE][23] = 54,
+ [1][0][RTW89_UKRAINE][23] = 36,
+ [1][0][RTW89_MEXICO][23] = 62,
+ [1][0][RTW89_CN][23] = 127,
+ [1][0][RTW89_QATAR][23] = 36,
+ [1][0][RTW89_FCC][25] = 62,
+ [1][0][RTW89_ETSI][25] = 36,
+ [1][0][RTW89_MKK][25] = 58,
+ [1][0][RTW89_IC][25] = 127,
+ [1][0][RTW89_KCC][25] = 58,
+ [1][0][RTW89_ACMA][25] = 127,
+ [1][0][RTW89_CHILE][25] = 54,
+ [1][0][RTW89_UKRAINE][25] = 36,
+ [1][0][RTW89_MEXICO][25] = 62,
+ [1][0][RTW89_CN][25] = 127,
+ [1][0][RTW89_QATAR][25] = 36,
+ [1][0][RTW89_FCC][27] = 62,
+ [1][0][RTW89_ETSI][27] = 36,
+ [1][0][RTW89_MKK][27] = 58,
+ [1][0][RTW89_IC][27] = 127,
+ [1][0][RTW89_KCC][27] = 58,
+ [1][0][RTW89_ACMA][27] = 127,
+ [1][0][RTW89_CHILE][27] = 54,
+ [1][0][RTW89_UKRAINE][27] = 36,
+ [1][0][RTW89_MEXICO][27] = 62,
+ [1][0][RTW89_CN][27] = 127,
+ [1][0][RTW89_QATAR][27] = 36,
+ [1][0][RTW89_FCC][29] = 62,
+ [1][0][RTW89_ETSI][29] = 36,
+ [1][0][RTW89_MKK][29] = 58,
+ [1][0][RTW89_IC][29] = 127,
+ [1][0][RTW89_KCC][29] = 58,
+ [1][0][RTW89_ACMA][29] = 127,
+ [1][0][RTW89_CHILE][29] = 54,
+ [1][0][RTW89_UKRAINE][29] = 36,
+ [1][0][RTW89_MEXICO][29] = 62,
+ [1][0][RTW89_CN][29] = 127,
+ [1][0][RTW89_QATAR][29] = 36,
+ [1][0][RTW89_FCC][31] = 62,
+ [1][0][RTW89_ETSI][31] = 36,
+ [1][0][RTW89_MKK][31] = 58,
+ [1][0][RTW89_IC][31] = 62,
+ [1][0][RTW89_KCC][31] = 58,
+ [1][0][RTW89_ACMA][31] = 36,
+ [1][0][RTW89_CHILE][31] = 54,
+ [1][0][RTW89_UKRAINE][31] = 36,
+ [1][0][RTW89_MEXICO][31] = 62,
+ [1][0][RTW89_CN][31] = 127,
+ [1][0][RTW89_QATAR][31] = 36,
+ [1][0][RTW89_FCC][33] = 62,
+ [1][0][RTW89_ETSI][33] = 36,
+ [1][0][RTW89_MKK][33] = 58,
+ [1][0][RTW89_IC][33] = 62,
+ [1][0][RTW89_KCC][33] = 58,
+ [1][0][RTW89_ACMA][33] = 36,
+ [1][0][RTW89_CHILE][33] = 54,
+ [1][0][RTW89_UKRAINE][33] = 36,
+ [1][0][RTW89_MEXICO][33] = 62,
+ [1][0][RTW89_CN][33] = 127,
+ [1][0][RTW89_QATAR][33] = 36,
+ [1][0][RTW89_FCC][35] = 62,
+ [1][0][RTW89_ETSI][35] = 36,
+ [1][0][RTW89_MKK][35] = 58,
+ [1][0][RTW89_IC][35] = 62,
+ [1][0][RTW89_KCC][35] = 58,
+ [1][0][RTW89_ACMA][35] = 36,
+ [1][0][RTW89_CHILE][35] = 54,
+ [1][0][RTW89_UKRAINE][35] = 36,
+ [1][0][RTW89_MEXICO][35] = 62,
+ [1][0][RTW89_CN][35] = 127,
+ [1][0][RTW89_QATAR][35] = 36,
+ [1][0][RTW89_FCC][37] = 62,
+ [1][0][RTW89_ETSI][37] = 127,
+ [1][0][RTW89_MKK][37] = 58,
+ [1][0][RTW89_IC][37] = 62,
+ [1][0][RTW89_KCC][37] = 58,
+ [1][0][RTW89_ACMA][37] = 62,
+ [1][0][RTW89_CHILE][37] = 54,
+ [1][0][RTW89_UKRAINE][37] = 127,
+ [1][0][RTW89_MEXICO][37] = 62,
+ [1][0][RTW89_CN][37] = 127,
+ [1][0][RTW89_QATAR][37] = 127,
+ [1][0][RTW89_FCC][38] = 76,
+ [1][0][RTW89_ETSI][38] = 28,
+ [1][0][RTW89_MKK][38] = 127,
+ [1][0][RTW89_IC][38] = 76,
+ [1][0][RTW89_KCC][38] = 58,
+ [1][0][RTW89_ACMA][38] = 76,
+ [1][0][RTW89_CHILE][38] = 54,
+ [1][0][RTW89_UKRAINE][38] = 28,
+ [1][0][RTW89_MEXICO][38] = 76,
+ [1][0][RTW89_CN][38] = 74,
+ [1][0][RTW89_QATAR][38] = 28,
+ [1][0][RTW89_FCC][40] = 76,
+ [1][0][RTW89_ETSI][40] = 28,
+ [1][0][RTW89_MKK][40] = 127,
+ [1][0][RTW89_IC][40] = 76,
+ [1][0][RTW89_KCC][40] = 58,
+ [1][0][RTW89_ACMA][40] = 76,
+ [1][0][RTW89_CHILE][40] = 54,
+ [1][0][RTW89_UKRAINE][40] = 28,
+ [1][0][RTW89_MEXICO][40] = 76,
+ [1][0][RTW89_CN][40] = 74,
+ [1][0][RTW89_QATAR][40] = 28,
+ [1][0][RTW89_FCC][42] = 76,
+ [1][0][RTW89_ETSI][42] = 28,
+ [1][0][RTW89_MKK][42] = 127,
+ [1][0][RTW89_IC][42] = 76,
+ [1][0][RTW89_KCC][42] = 58,
+ [1][0][RTW89_ACMA][42] = 76,
+ [1][0][RTW89_CHILE][42] = 54,
+ [1][0][RTW89_UKRAINE][42] = 28,
+ [1][0][RTW89_MEXICO][42] = 76,
+ [1][0][RTW89_CN][42] = 74,
+ [1][0][RTW89_QATAR][42] = 28,
+ [1][0][RTW89_FCC][44] = 76,
+ [1][0][RTW89_ETSI][44] = 28,
+ [1][0][RTW89_MKK][44] = 127,
+ [1][0][RTW89_IC][44] = 76,
+ [1][0][RTW89_KCC][44] = 58,
+ [1][0][RTW89_ACMA][44] = 76,
+ [1][0][RTW89_CHILE][44] = 54,
+ [1][0][RTW89_UKRAINE][44] = 28,
+ [1][0][RTW89_MEXICO][44] = 76,
+ [1][0][RTW89_CN][44] = 74,
+ [1][0][RTW89_QATAR][44] = 28,
+ [1][0][RTW89_FCC][46] = 76,
+ [1][0][RTW89_ETSI][46] = 28,
+ [1][0][RTW89_MKK][46] = 127,
+ [1][0][RTW89_IC][46] = 76,
+ [1][0][RTW89_KCC][46] = 58,
+ [1][0][RTW89_ACMA][46] = 76,
+ [1][0][RTW89_CHILE][46] = 54,
+ [1][0][RTW89_UKRAINE][46] = 28,
+ [1][0][RTW89_MEXICO][46] = 76,
+ [1][0][RTW89_CN][46] = 74,
+ [1][0][RTW89_QATAR][46] = 28,
+ [1][1][RTW89_FCC][0] = 46,
+ [1][1][RTW89_ETSI][0] = 22,
+ [1][1][RTW89_MKK][0] = 24,
+ [1][1][RTW89_IC][0] = 18,
+ [1][1][RTW89_KCC][0] = 44,
+ [1][1][RTW89_ACMA][0] = 22,
+ [1][1][RTW89_CHILE][0] = 18,
+ [1][1][RTW89_UKRAINE][0] = 22,
+ [1][1][RTW89_MEXICO][0] = 46,
+ [1][1][RTW89_CN][0] = 22,
+ [1][1][RTW89_QATAR][0] = 22,
+ [1][1][RTW89_FCC][2] = 46,
+ [1][1][RTW89_ETSI][2] = 22,
+ [1][1][RTW89_MKK][2] = 24,
+ [1][1][RTW89_IC][2] = 18,
+ [1][1][RTW89_KCC][2] = 44,
+ [1][1][RTW89_ACMA][2] = 22,
+ [1][1][RTW89_CHILE][2] = 18,
+ [1][1][RTW89_UKRAINE][2] = 22,
+ [1][1][RTW89_MEXICO][2] = 46,
+ [1][1][RTW89_CN][2] = 22,
+ [1][1][RTW89_QATAR][2] = 22,
+ [1][1][RTW89_FCC][4] = 46,
+ [1][1][RTW89_ETSI][4] = 22,
+ [1][1][RTW89_MKK][4] = 24,
+ [1][1][RTW89_IC][4] = 18,
+ [1][1][RTW89_KCC][4] = 44,
+ [1][1][RTW89_ACMA][4] = 22,
+ [1][1][RTW89_CHILE][4] = 18,
+ [1][1][RTW89_UKRAINE][4] = 22,
+ [1][1][RTW89_MEXICO][4] = 46,
+ [1][1][RTW89_CN][4] = 22,
+ [1][1][RTW89_QATAR][4] = 22,
+ [1][1][RTW89_FCC][6] = 46,
+ [1][1][RTW89_ETSI][6] = 22,
+ [1][1][RTW89_MKK][6] = 24,
+ [1][1][RTW89_IC][6] = 18,
+ [1][1][RTW89_KCC][6] = 16,
+ [1][1][RTW89_ACMA][6] = 22,
+ [1][1][RTW89_CHILE][6] = 18,
+ [1][1][RTW89_UKRAINE][6] = 22,
+ [1][1][RTW89_MEXICO][6] = 46,
+ [1][1][RTW89_CN][6] = 22,
+ [1][1][RTW89_QATAR][6] = 22,
+ [1][1][RTW89_FCC][8] = 46,
+ [1][1][RTW89_ETSI][8] = 22,
+ [1][1][RTW89_MKK][8] = 24,
+ [1][1][RTW89_IC][8] = 46,
+ [1][1][RTW89_KCC][8] = 44,
+ [1][1][RTW89_ACMA][8] = 22,
+ [1][1][RTW89_CHILE][8] = 42,
+ [1][1][RTW89_UKRAINE][8] = 22,
+ [1][1][RTW89_MEXICO][8] = 46,
+ [1][1][RTW89_CN][8] = 22,
+ [1][1][RTW89_QATAR][8] = 22,
+ [1][1][RTW89_FCC][10] = 46,
+ [1][1][RTW89_ETSI][10] = 22,
+ [1][1][RTW89_MKK][10] = 24,
+ [1][1][RTW89_IC][10] = 46,
+ [1][1][RTW89_KCC][10] = 44,
+ [1][1][RTW89_ACMA][10] = 22,
+ [1][1][RTW89_CHILE][10] = 42,
+ [1][1][RTW89_UKRAINE][10] = 22,
+ [1][1][RTW89_MEXICO][10] = 46,
+ [1][1][RTW89_CN][10] = 22,
+ [1][1][RTW89_QATAR][10] = 22,
+ [1][1][RTW89_FCC][12] = 46,
+ [1][1][RTW89_ETSI][12] = 22,
+ [1][1][RTW89_MKK][12] = 24,
+ [1][1][RTW89_IC][12] = 46,
+ [1][1][RTW89_KCC][12] = 44,
+ [1][1][RTW89_ACMA][12] = 22,
+ [1][1][RTW89_CHILE][12] = 42,
+ [1][1][RTW89_UKRAINE][12] = 22,
+ [1][1][RTW89_MEXICO][12] = 46,
+ [1][1][RTW89_CN][12] = 22,
+ [1][1][RTW89_QATAR][12] = 22,
+ [1][1][RTW89_FCC][14] = 46,
+ [1][1][RTW89_ETSI][14] = 22,
+ [1][1][RTW89_MKK][14] = 24,
+ [1][1][RTW89_IC][14] = 46,
+ [1][1][RTW89_KCC][14] = 44,
+ [1][1][RTW89_ACMA][14] = 22,
+ [1][1][RTW89_CHILE][14] = 42,
+ [1][1][RTW89_UKRAINE][14] = 22,
+ [1][1][RTW89_MEXICO][14] = 46,
+ [1][1][RTW89_CN][14] = 22,
+ [1][1][RTW89_QATAR][14] = 22,
+ [1][1][RTW89_FCC][15] = 46,
+ [1][1][RTW89_ETSI][15] = 22,
+ [1][1][RTW89_MKK][15] = 46,
+ [1][1][RTW89_IC][15] = 46,
+ [1][1][RTW89_KCC][15] = 44,
+ [1][1][RTW89_ACMA][15] = 22,
+ [1][1][RTW89_CHILE][15] = 42,
+ [1][1][RTW89_UKRAINE][15] = 22,
+ [1][1][RTW89_MEXICO][15] = 46,
+ [1][1][RTW89_CN][15] = 127,
+ [1][1][RTW89_QATAR][15] = 22,
+ [1][1][RTW89_FCC][17] = 46,
+ [1][1][RTW89_ETSI][17] = 22,
+ [1][1][RTW89_MKK][17] = 46,
+ [1][1][RTW89_IC][17] = 46,
+ [1][1][RTW89_KCC][17] = 44,
+ [1][1][RTW89_ACMA][17] = 22,
+ [1][1][RTW89_CHILE][17] = 42,
+ [1][1][RTW89_UKRAINE][17] = 22,
+ [1][1][RTW89_MEXICO][17] = 46,
+ [1][1][RTW89_CN][17] = 127,
+ [1][1][RTW89_QATAR][17] = 22,
+ [1][1][RTW89_FCC][19] = 46,
+ [1][1][RTW89_ETSI][19] = 22,
+ [1][1][RTW89_MKK][19] = 46,
+ [1][1][RTW89_IC][19] = 46,
+ [1][1][RTW89_KCC][19] = 44,
+ [1][1][RTW89_ACMA][19] = 22,
+ [1][1][RTW89_CHILE][19] = 42,
+ [1][1][RTW89_UKRAINE][19] = 22,
+ [1][1][RTW89_MEXICO][19] = 46,
+ [1][1][RTW89_CN][19] = 127,
+ [1][1][RTW89_QATAR][19] = 22,
+ [1][1][RTW89_FCC][21] = 46,
+ [1][1][RTW89_ETSI][21] = 22,
+ [1][1][RTW89_MKK][21] = 46,
+ [1][1][RTW89_IC][21] = 46,
+ [1][1][RTW89_KCC][21] = 44,
+ [1][1][RTW89_ACMA][21] = 22,
+ [1][1][RTW89_CHILE][21] = 42,
+ [1][1][RTW89_UKRAINE][21] = 22,
+ [1][1][RTW89_MEXICO][21] = 46,
+ [1][1][RTW89_CN][21] = 127,
+ [1][1][RTW89_QATAR][21] = 22,
+ [1][1][RTW89_FCC][23] = 46,
+ [1][1][RTW89_ETSI][23] = 22,
+ [1][1][RTW89_MKK][23] = 46,
+ [1][1][RTW89_IC][23] = 46,
+ [1][1][RTW89_KCC][23] = 44,
+ [1][1][RTW89_ACMA][23] = 22,
+ [1][1][RTW89_CHILE][23] = 42,
+ [1][1][RTW89_UKRAINE][23] = 22,
+ [1][1][RTW89_MEXICO][23] = 46,
+ [1][1][RTW89_CN][23] = 127,
+ [1][1][RTW89_QATAR][23] = 22,
+ [1][1][RTW89_FCC][25] = 46,
+ [1][1][RTW89_ETSI][25] = 22,
+ [1][1][RTW89_MKK][25] = 46,
+ [1][1][RTW89_IC][25] = 127,
+ [1][1][RTW89_KCC][25] = 44,
+ [1][1][RTW89_ACMA][25] = 127,
+ [1][1][RTW89_CHILE][25] = 42,
+ [1][1][RTW89_UKRAINE][25] = 22,
+ [1][1][RTW89_MEXICO][25] = 46,
+ [1][1][RTW89_CN][25] = 127,
+ [1][1][RTW89_QATAR][25] = 22,
+ [1][1][RTW89_FCC][27] = 46,
+ [1][1][RTW89_ETSI][27] = 22,
+ [1][1][RTW89_MKK][27] = 46,
+ [1][1][RTW89_IC][27] = 127,
+ [1][1][RTW89_KCC][27] = 44,
+ [1][1][RTW89_ACMA][27] = 127,
+ [1][1][RTW89_CHILE][27] = 42,
+ [1][1][RTW89_UKRAINE][27] = 22,
+ [1][1][RTW89_MEXICO][27] = 46,
+ [1][1][RTW89_CN][27] = 127,
+ [1][1][RTW89_QATAR][27] = 22,
+ [1][1][RTW89_FCC][29] = 46,
+ [1][1][RTW89_ETSI][29] = 22,
+ [1][1][RTW89_MKK][29] = 46,
+ [1][1][RTW89_IC][29] = 127,
+ [1][1][RTW89_KCC][29] = 44,
+ [1][1][RTW89_ACMA][29] = 127,
+ [1][1][RTW89_CHILE][29] = 42,
+ [1][1][RTW89_UKRAINE][29] = 22,
+ [1][1][RTW89_MEXICO][29] = 46,
+ [1][1][RTW89_CN][29] = 127,
+ [1][1][RTW89_QATAR][29] = 22,
+ [1][1][RTW89_FCC][31] = 46,
+ [1][1][RTW89_ETSI][31] = 22,
+ [1][1][RTW89_MKK][31] = 46,
+ [1][1][RTW89_IC][31] = 46,
+ [1][1][RTW89_KCC][31] = 44,
+ [1][1][RTW89_ACMA][31] = 22,
+ [1][1][RTW89_CHILE][31] = 42,
+ [1][1][RTW89_UKRAINE][31] = 22,
+ [1][1][RTW89_MEXICO][31] = 46,
+ [1][1][RTW89_CN][31] = 127,
+ [1][1][RTW89_QATAR][31] = 22,
+ [1][1][RTW89_FCC][33] = 46,
+ [1][1][RTW89_ETSI][33] = 22,
+ [1][1][RTW89_MKK][33] = 46,
+ [1][1][RTW89_IC][33] = 46,
+ [1][1][RTW89_KCC][33] = 44,
+ [1][1][RTW89_ACMA][33] = 22,
+ [1][1][RTW89_CHILE][33] = 42,
+ [1][1][RTW89_UKRAINE][33] = 22,
+ [1][1][RTW89_MEXICO][33] = 46,
+ [1][1][RTW89_CN][33] = 127,
+ [1][1][RTW89_QATAR][33] = 22,
+ [1][1][RTW89_FCC][35] = 46,
+ [1][1][RTW89_ETSI][35] = 22,
+ [1][1][RTW89_MKK][35] = 46,
+ [1][1][RTW89_IC][35] = 46,
+ [1][1][RTW89_KCC][35] = 44,
+ [1][1][RTW89_ACMA][35] = 22,
+ [1][1][RTW89_CHILE][35] = 42,
+ [1][1][RTW89_UKRAINE][35] = 22,
+ [1][1][RTW89_MEXICO][35] = 46,
+ [1][1][RTW89_CN][35] = 127,
+ [1][1][RTW89_QATAR][35] = 22,
+ [1][1][RTW89_FCC][37] = 46,
+ [1][1][RTW89_ETSI][37] = 127,
+ [1][1][RTW89_MKK][37] = 46,
+ [1][1][RTW89_IC][37] = 46,
+ [1][1][RTW89_KCC][37] = 44,
+ [1][1][RTW89_ACMA][37] = 50,
+ [1][1][RTW89_CHILE][37] = 42,
+ [1][1][RTW89_UKRAINE][37] = 127,
+ [1][1][RTW89_MEXICO][37] = 46,
+ [1][1][RTW89_CN][37] = 127,
+ [1][1][RTW89_QATAR][37] = 127,
+ [1][1][RTW89_FCC][38] = 74,
+ [1][1][RTW89_ETSI][38] = 16,
+ [1][1][RTW89_MKK][38] = 127,
+ [1][1][RTW89_IC][38] = 74,
+ [1][1][RTW89_KCC][38] = 44,
+ [1][1][RTW89_ACMA][38] = 76,
+ [1][1][RTW89_CHILE][38] = 42,
+ [1][1][RTW89_UKRAINE][38] = 16,
+ [1][1][RTW89_MEXICO][38] = 74,
+ [1][1][RTW89_CN][38] = 62,
+ [1][1][RTW89_QATAR][38] = 16,
+ [1][1][RTW89_FCC][40] = 76,
+ [1][1][RTW89_ETSI][40] = 16,
+ [1][1][RTW89_MKK][40] = 127,
+ [1][1][RTW89_IC][40] = 76,
+ [1][1][RTW89_KCC][40] = 44,
+ [1][1][RTW89_ACMA][40] = 76,
+ [1][1][RTW89_CHILE][40] = 42,
+ [1][1][RTW89_UKRAINE][40] = 16,
+ [1][1][RTW89_MEXICO][40] = 76,
+ [1][1][RTW89_CN][40] = 62,
+ [1][1][RTW89_QATAR][40] = 16,
+ [1][1][RTW89_FCC][42] = 76,
+ [1][1][RTW89_ETSI][42] = 16,
+ [1][1][RTW89_MKK][42] = 127,
+ [1][1][RTW89_IC][42] = 76,
+ [1][1][RTW89_KCC][42] = 44,
+ [1][1][RTW89_ACMA][42] = 76,
+ [1][1][RTW89_CHILE][42] = 42,
+ [1][1][RTW89_UKRAINE][42] = 16,
+ [1][1][RTW89_MEXICO][42] = 76,
+ [1][1][RTW89_CN][42] = 62,
+ [1][1][RTW89_QATAR][42] = 16,
+ [1][1][RTW89_FCC][44] = 76,
+ [1][1][RTW89_ETSI][44] = 16,
+ [1][1][RTW89_MKK][44] = 127,
+ [1][1][RTW89_IC][44] = 76,
+ [1][1][RTW89_KCC][44] = 44,
+ [1][1][RTW89_ACMA][44] = 76,
+ [1][1][RTW89_CHILE][44] = 42,
+ [1][1][RTW89_UKRAINE][44] = 16,
+ [1][1][RTW89_MEXICO][44] = 76,
+ [1][1][RTW89_CN][44] = 62,
+ [1][1][RTW89_QATAR][44] = 16,
+ [1][1][RTW89_FCC][46] = 76,
+ [1][1][RTW89_ETSI][46] = 16,
+ [1][1][RTW89_MKK][46] = 127,
+ [1][1][RTW89_IC][46] = 76,
+ [1][1][RTW89_KCC][46] = 44,
+ [1][1][RTW89_ACMA][46] = 76,
+ [1][1][RTW89_CHILE][46] = 42,
+ [1][1][RTW89_UKRAINE][46] = 16,
+ [1][1][RTW89_MEXICO][46] = 76,
+ [1][1][RTW89_CN][46] = 62,
+ [1][1][RTW89_QATAR][46] = 16,
+ [2][0][RTW89_FCC][0] = 74,
+ [2][0][RTW89_ETSI][0] = 46,
+ [2][0][RTW89_MKK][0] = 50,
+ [2][0][RTW89_IC][0] = 46,
+ [2][0][RTW89_KCC][0] = 70,
+ [2][0][RTW89_ACMA][0] = 46,
+ [2][0][RTW89_CHILE][0] = 30,
+ [2][0][RTW89_UKRAINE][0] = 46,
+ [2][0][RTW89_MEXICO][0] = 62,
+ [2][0][RTW89_CN][0] = 46,
+ [2][0][RTW89_QATAR][0] = 46,
+ [2][0][RTW89_FCC][2] = 74,
+ [2][0][RTW89_ETSI][2] = 46,
+ [2][0][RTW89_MKK][2] = 50,
+ [2][0][RTW89_IC][2] = 46,
+ [2][0][RTW89_KCC][2] = 70,
+ [2][0][RTW89_ACMA][2] = 46,
+ [2][0][RTW89_CHILE][2] = 30,
+ [2][0][RTW89_UKRAINE][2] = 46,
+ [2][0][RTW89_MEXICO][2] = 62,
+ [2][0][RTW89_CN][2] = 46,
+ [2][0][RTW89_QATAR][2] = 46,
+ [2][0][RTW89_FCC][4] = 74,
+ [2][0][RTW89_ETSI][4] = 46,
+ [2][0][RTW89_MKK][4] = 50,
+ [2][0][RTW89_IC][4] = 46,
+ [2][0][RTW89_KCC][4] = 70,
+ [2][0][RTW89_ACMA][4] = 46,
+ [2][0][RTW89_CHILE][4] = 30,
+ [2][0][RTW89_UKRAINE][4] = 46,
+ [2][0][RTW89_MEXICO][4] = 62,
+ [2][0][RTW89_CN][4] = 46,
+ [2][0][RTW89_QATAR][4] = 46,
+ [2][0][RTW89_FCC][6] = 74,
+ [2][0][RTW89_ETSI][6] = 46,
+ [2][0][RTW89_MKK][6] = 50,
+ [2][0][RTW89_IC][6] = 46,
+ [2][0][RTW89_KCC][6] = 44,
+ [2][0][RTW89_ACMA][6] = 46,
+ [2][0][RTW89_CHILE][6] = 30,
+ [2][0][RTW89_UKRAINE][6] = 46,
+ [2][0][RTW89_MEXICO][6] = 62,
+ [2][0][RTW89_CN][6] = 46,
+ [2][0][RTW89_QATAR][6] = 46,
+ [2][0][RTW89_FCC][8] = 74,
+ [2][0][RTW89_ETSI][8] = 46,
+ [2][0][RTW89_MKK][8] = 50,
+ [2][0][RTW89_IC][8] = 66,
+ [2][0][RTW89_KCC][8] = 70,
+ [2][0][RTW89_ACMA][8] = 46,
+ [2][0][RTW89_CHILE][8] = 54,
+ [2][0][RTW89_UKRAINE][8] = 46,
+ [2][0][RTW89_MEXICO][8] = 74,
+ [2][0][RTW89_CN][8] = 46,
+ [2][0][RTW89_QATAR][8] = 46,
+ [2][0][RTW89_FCC][10] = 74,
+ [2][0][RTW89_ETSI][10] = 46,
+ [2][0][RTW89_MKK][10] = 50,
+ [2][0][RTW89_IC][10] = 66,
+ [2][0][RTW89_KCC][10] = 70,
+ [2][0][RTW89_ACMA][10] = 46,
+ [2][0][RTW89_CHILE][10] = 54,
+ [2][0][RTW89_UKRAINE][10] = 46,
+ [2][0][RTW89_MEXICO][10] = 74,
+ [2][0][RTW89_CN][10] = 46,
+ [2][0][RTW89_QATAR][10] = 46,
+ [2][0][RTW89_FCC][12] = 74,
+ [2][0][RTW89_ETSI][12] = 46,
+ [2][0][RTW89_MKK][12] = 50,
+ [2][0][RTW89_IC][12] = 66,
+ [2][0][RTW89_KCC][12] = 70,
+ [2][0][RTW89_ACMA][12] = 46,
+ [2][0][RTW89_CHILE][12] = 54,
+ [2][0][RTW89_UKRAINE][12] = 46,
+ [2][0][RTW89_MEXICO][12] = 74,
+ [2][0][RTW89_CN][12] = 46,
+ [2][0][RTW89_QATAR][12] = 46,
+ [2][0][RTW89_FCC][14] = 74,
+ [2][0][RTW89_ETSI][14] = 46,
+ [2][0][RTW89_MKK][14] = 50,
+ [2][0][RTW89_IC][14] = 66,
+ [2][0][RTW89_KCC][14] = 70,
+ [2][0][RTW89_ACMA][14] = 46,
+ [2][0][RTW89_CHILE][14] = 54,
+ [2][0][RTW89_UKRAINE][14] = 46,
+ [2][0][RTW89_MEXICO][14] = 74,
+ [2][0][RTW89_CN][14] = 46,
+ [2][0][RTW89_QATAR][14] = 46,
+ [2][0][RTW89_FCC][15] = 74,
+ [2][0][RTW89_ETSI][15] = 46,
+ [2][0][RTW89_MKK][15] = 70,
+ [2][0][RTW89_IC][15] = 74,
+ [2][0][RTW89_KCC][15] = 70,
+ [2][0][RTW89_ACMA][15] = 46,
+ [2][0][RTW89_CHILE][15] = 54,
+ [2][0][RTW89_UKRAINE][15] = 46,
+ [2][0][RTW89_MEXICO][15] = 74,
+ [2][0][RTW89_CN][15] = 127,
+ [2][0][RTW89_QATAR][15] = 46,
+ [2][0][RTW89_FCC][17] = 74,
+ [2][0][RTW89_ETSI][17] = 46,
+ [2][0][RTW89_MKK][17] = 70,
+ [2][0][RTW89_IC][17] = 74,
+ [2][0][RTW89_KCC][17] = 70,
+ [2][0][RTW89_ACMA][17] = 46,
+ [2][0][RTW89_CHILE][17] = 54,
+ [2][0][RTW89_UKRAINE][17] = 46,
+ [2][0][RTW89_MEXICO][17] = 74,
+ [2][0][RTW89_CN][17] = 127,
+ [2][0][RTW89_QATAR][17] = 46,
+ [2][0][RTW89_FCC][19] = 74,
+ [2][0][RTW89_ETSI][19] = 46,
+ [2][0][RTW89_MKK][19] = 70,
+ [2][0][RTW89_IC][19] = 74,
+ [2][0][RTW89_KCC][19] = 70,
+ [2][0][RTW89_ACMA][19] = 46,
+ [2][0][RTW89_CHILE][19] = 54,
+ [2][0][RTW89_UKRAINE][19] = 46,
+ [2][0][RTW89_MEXICO][19] = 74,
+ [2][0][RTW89_CN][19] = 127,
+ [2][0][RTW89_QATAR][19] = 46,
+ [2][0][RTW89_FCC][21] = 74,
+ [2][0][RTW89_ETSI][21] = 46,
+ [2][0][RTW89_MKK][21] = 70,
+ [2][0][RTW89_IC][21] = 74,
+ [2][0][RTW89_KCC][21] = 70,
+ [2][0][RTW89_ACMA][21] = 46,
+ [2][0][RTW89_CHILE][21] = 54,
+ [2][0][RTW89_UKRAINE][21] = 46,
+ [2][0][RTW89_MEXICO][21] = 74,
+ [2][0][RTW89_CN][21] = 127,
+ [2][0][RTW89_QATAR][21] = 46,
+ [2][0][RTW89_FCC][23] = 74,
+ [2][0][RTW89_ETSI][23] = 46,
+ [2][0][RTW89_MKK][23] = 70,
+ [2][0][RTW89_IC][23] = 74,
+ [2][0][RTW89_KCC][23] = 70,
+ [2][0][RTW89_ACMA][23] = 46,
+ [2][0][RTW89_CHILE][23] = 54,
+ [2][0][RTW89_UKRAINE][23] = 46,
+ [2][0][RTW89_MEXICO][23] = 74,
+ [2][0][RTW89_CN][23] = 127,
+ [2][0][RTW89_QATAR][23] = 46,
+ [2][0][RTW89_FCC][25] = 74,
+ [2][0][RTW89_ETSI][25] = 46,
+ [2][0][RTW89_MKK][25] = 70,
+ [2][0][RTW89_IC][25] = 127,
+ [2][0][RTW89_KCC][25] = 70,
+ [2][0][RTW89_ACMA][25] = 127,
+ [2][0][RTW89_CHILE][25] = 54,
+ [2][0][RTW89_UKRAINE][25] = 46,
+ [2][0][RTW89_MEXICO][25] = 74,
+ [2][0][RTW89_CN][25] = 127,
+ [2][0][RTW89_QATAR][25] = 46,
+ [2][0][RTW89_FCC][27] = 74,
+ [2][0][RTW89_ETSI][27] = 46,
+ [2][0][RTW89_MKK][27] = 70,
+ [2][0][RTW89_IC][27] = 127,
+ [2][0][RTW89_KCC][27] = 70,
+ [2][0][RTW89_ACMA][27] = 127,
+ [2][0][RTW89_CHILE][27] = 54,
+ [2][0][RTW89_UKRAINE][27] = 46,
+ [2][0][RTW89_MEXICO][27] = 74,
+ [2][0][RTW89_CN][27] = 127,
+ [2][0][RTW89_QATAR][27] = 46,
+ [2][0][RTW89_FCC][29] = 74,
+ [2][0][RTW89_ETSI][29] = 46,
+ [2][0][RTW89_MKK][29] = 70,
+ [2][0][RTW89_IC][29] = 127,
+ [2][0][RTW89_KCC][29] = 70,
+ [2][0][RTW89_ACMA][29] = 127,
+ [2][0][RTW89_CHILE][29] = 54,
+ [2][0][RTW89_UKRAINE][29] = 46,
+ [2][0][RTW89_MEXICO][29] = 74,
+ [2][0][RTW89_CN][29] = 127,
+ [2][0][RTW89_QATAR][29] = 46,
+ [2][0][RTW89_FCC][31] = 74,
+ [2][0][RTW89_ETSI][31] = 46,
+ [2][0][RTW89_MKK][31] = 70,
+ [2][0][RTW89_IC][31] = 74,
+ [2][0][RTW89_KCC][31] = 70,
+ [2][0][RTW89_ACMA][31] = 46,
+ [2][0][RTW89_CHILE][31] = 54,
+ [2][0][RTW89_UKRAINE][31] = 46,
+ [2][0][RTW89_MEXICO][31] = 74,
+ [2][0][RTW89_CN][31] = 127,
+ [2][0][RTW89_QATAR][31] = 46,
+ [2][0][RTW89_FCC][33] = 74,
+ [2][0][RTW89_ETSI][33] = 46,
+ [2][0][RTW89_MKK][33] = 70,
+ [2][0][RTW89_IC][33] = 74,
+ [2][0][RTW89_KCC][33] = 70,
+ [2][0][RTW89_ACMA][33] = 46,
+ [2][0][RTW89_CHILE][33] = 54,
+ [2][0][RTW89_UKRAINE][33] = 46,
+ [2][0][RTW89_MEXICO][33] = 74,
+ [2][0][RTW89_CN][33] = 127,
+ [2][0][RTW89_QATAR][33] = 46,
+ [2][0][RTW89_FCC][35] = 74,
+ [2][0][RTW89_ETSI][35] = 46,
+ [2][0][RTW89_MKK][35] = 70,
+ [2][0][RTW89_IC][35] = 74,
+ [2][0][RTW89_KCC][35] = 70,
+ [2][0][RTW89_ACMA][35] = 46,
+ [2][0][RTW89_CHILE][35] = 54,
+ [2][0][RTW89_UKRAINE][35] = 46,
+ [2][0][RTW89_MEXICO][35] = 74,
+ [2][0][RTW89_CN][35] = 127,
+ [2][0][RTW89_QATAR][35] = 46,
+ [2][0][RTW89_FCC][37] = 74,
+ [2][0][RTW89_ETSI][37] = 127,
+ [2][0][RTW89_MKK][37] = 70,
+ [2][0][RTW89_IC][37] = 74,
+ [2][0][RTW89_KCC][37] = 70,
+ [2][0][RTW89_ACMA][37] = 74,
+ [2][0][RTW89_CHILE][37] = 54,
+ [2][0][RTW89_UKRAINE][37] = 127,
+ [2][0][RTW89_MEXICO][37] = 74,
+ [2][0][RTW89_CN][37] = 127,
+ [2][0][RTW89_QATAR][37] = 127,
+ [2][0][RTW89_FCC][38] = 76,
+ [2][0][RTW89_ETSI][38] = 28,
+ [2][0][RTW89_MKK][38] = 127,
+ [2][0][RTW89_IC][38] = 76,
+ [2][0][RTW89_KCC][38] = 70,
+ [2][0][RTW89_ACMA][38] = 76,
+ [2][0][RTW89_CHILE][38] = 54,
+ [2][0][RTW89_UKRAINE][38] = 28,
+ [2][0][RTW89_MEXICO][38] = 76,
+ [2][0][RTW89_CN][38] = 76,
+ [2][0][RTW89_QATAR][38] = 28,
+ [2][0][RTW89_FCC][40] = 76,
+ [2][0][RTW89_ETSI][40] = 28,
+ [2][0][RTW89_MKK][40] = 127,
+ [2][0][RTW89_IC][40] = 76,
+ [2][0][RTW89_KCC][40] = 70,
+ [2][0][RTW89_ACMA][40] = 76,
+ [2][0][RTW89_CHILE][40] = 54,
+ [2][0][RTW89_UKRAINE][40] = 28,
+ [2][0][RTW89_MEXICO][40] = 76,
+ [2][0][RTW89_CN][40] = 76,
+ [2][0][RTW89_QATAR][40] = 28,
+ [2][0][RTW89_FCC][42] = 76,
+ [2][0][RTW89_ETSI][42] = 28,
+ [2][0][RTW89_MKK][42] = 127,
+ [2][0][RTW89_IC][42] = 76,
+ [2][0][RTW89_KCC][42] = 70,
+ [2][0][RTW89_ACMA][42] = 76,
+ [2][0][RTW89_CHILE][42] = 54,
+ [2][0][RTW89_UKRAINE][42] = 28,
+ [2][0][RTW89_MEXICO][42] = 76,
+ [2][0][RTW89_CN][42] = 76,
+ [2][0][RTW89_QATAR][42] = 28,
+ [2][0][RTW89_FCC][44] = 76,
+ [2][0][RTW89_ETSI][44] = 28,
+ [2][0][RTW89_MKK][44] = 127,
+ [2][0][RTW89_IC][44] = 76,
+ [2][0][RTW89_KCC][44] = 70,
+ [2][0][RTW89_ACMA][44] = 76,
+ [2][0][RTW89_CHILE][44] = 54,
+ [2][0][RTW89_UKRAINE][44] = 28,
+ [2][0][RTW89_MEXICO][44] = 76,
+ [2][0][RTW89_CN][44] = 76,
+ [2][0][RTW89_QATAR][44] = 28,
+ [2][0][RTW89_FCC][46] = 76,
+ [2][0][RTW89_ETSI][46] = 28,
+ [2][0][RTW89_MKK][46] = 127,
+ [2][0][RTW89_IC][46] = 76,
+ [2][0][RTW89_KCC][46] = 70,
+ [2][0][RTW89_ACMA][46] = 76,
+ [2][0][RTW89_CHILE][46] = 54,
+ [2][0][RTW89_UKRAINE][46] = 28,
+ [2][0][RTW89_MEXICO][46] = 76,
+ [2][0][RTW89_CN][46] = 76,
+ [2][0][RTW89_QATAR][46] = 28,
+ [2][1][RTW89_FCC][0] = 58,
+ [2][1][RTW89_ETSI][0] = 32,
+ [2][1][RTW89_MKK][0] = 38,
+ [2][1][RTW89_IC][0] = 30,
+ [2][1][RTW89_KCC][0] = 54,
+ [2][1][RTW89_ACMA][0] = 32,
+ [2][1][RTW89_CHILE][0] = 18,
+ [2][1][RTW89_UKRAINE][0] = 32,
+ [2][1][RTW89_MEXICO][0] = 50,
+ [2][1][RTW89_CN][0] = 32,
+ [2][1][RTW89_QATAR][0] = 32,
+ [2][1][RTW89_FCC][2] = 58,
+ [2][1][RTW89_ETSI][2] = 32,
+ [2][1][RTW89_MKK][2] = 38,
+ [2][1][RTW89_IC][2] = 30,
+ [2][1][RTW89_KCC][2] = 54,
+ [2][1][RTW89_ACMA][2] = 32,
+ [2][1][RTW89_CHILE][2] = 18,
+ [2][1][RTW89_UKRAINE][2] = 32,
+ [2][1][RTW89_MEXICO][2] = 50,
+ [2][1][RTW89_CN][2] = 32,
+ [2][1][RTW89_QATAR][2] = 32,
+ [2][1][RTW89_FCC][4] = 58,
+ [2][1][RTW89_ETSI][4] = 32,
+ [2][1][RTW89_MKK][4] = 38,
+ [2][1][RTW89_IC][4] = 30,
+ [2][1][RTW89_KCC][4] = 54,
+ [2][1][RTW89_ACMA][4] = 32,
+ [2][1][RTW89_CHILE][4] = 18,
+ [2][1][RTW89_UKRAINE][4] = 32,
+ [2][1][RTW89_MEXICO][4] = 50,
+ [2][1][RTW89_CN][4] = 32,
+ [2][1][RTW89_QATAR][4] = 32,
+ [2][1][RTW89_FCC][6] = 58,
+ [2][1][RTW89_ETSI][6] = 32,
+ [2][1][RTW89_MKK][6] = 38,
+ [2][1][RTW89_IC][6] = 30,
+ [2][1][RTW89_KCC][6] = 26,
+ [2][1][RTW89_ACMA][6] = 32,
+ [2][1][RTW89_CHILE][6] = 18,
+ [2][1][RTW89_UKRAINE][6] = 32,
+ [2][1][RTW89_MEXICO][6] = 50,
+ [2][1][RTW89_CN][6] = 32,
+ [2][1][RTW89_QATAR][6] = 32,
+ [2][1][RTW89_FCC][8] = 58,
+ [2][1][RTW89_ETSI][8] = 32,
+ [2][1][RTW89_MKK][8] = 38,
+ [2][1][RTW89_IC][8] = 52,
+ [2][1][RTW89_KCC][8] = 54,
+ [2][1][RTW89_ACMA][8] = 32,
+ [2][1][RTW89_CHILE][8] = 42,
+ [2][1][RTW89_UKRAINE][8] = 32,
+ [2][1][RTW89_MEXICO][8] = 58,
+ [2][1][RTW89_CN][8] = 32,
+ [2][1][RTW89_QATAR][8] = 32,
+ [2][1][RTW89_FCC][10] = 58,
+ [2][1][RTW89_ETSI][10] = 32,
+ [2][1][RTW89_MKK][10] = 38,
+ [2][1][RTW89_IC][10] = 52,
+ [2][1][RTW89_KCC][10] = 54,
+ [2][1][RTW89_ACMA][10] = 32,
+ [2][1][RTW89_CHILE][10] = 42,
+ [2][1][RTW89_UKRAINE][10] = 32,
+ [2][1][RTW89_MEXICO][10] = 58,
+ [2][1][RTW89_CN][10] = 32,
+ [2][1][RTW89_QATAR][10] = 32,
+ [2][1][RTW89_FCC][12] = 58,
+ [2][1][RTW89_ETSI][12] = 32,
+ [2][1][RTW89_MKK][12] = 38,
+ [2][1][RTW89_IC][12] = 52,
+ [2][1][RTW89_KCC][12] = 54,
+ [2][1][RTW89_ACMA][12] = 32,
+ [2][1][RTW89_CHILE][12] = 42,
+ [2][1][RTW89_UKRAINE][12] = 32,
+ [2][1][RTW89_MEXICO][12] = 58,
+ [2][1][RTW89_CN][12] = 32,
+ [2][1][RTW89_QATAR][12] = 32,
+ [2][1][RTW89_FCC][14] = 58,
+ [2][1][RTW89_ETSI][14] = 32,
+ [2][1][RTW89_MKK][14] = 38,
+ [2][1][RTW89_IC][14] = 52,
+ [2][1][RTW89_KCC][14] = 54,
+ [2][1][RTW89_ACMA][14] = 32,
+ [2][1][RTW89_CHILE][14] = 42,
+ [2][1][RTW89_UKRAINE][14] = 32,
+ [2][1][RTW89_MEXICO][14] = 58,
+ [2][1][RTW89_CN][14] = 32,
+ [2][1][RTW89_QATAR][14] = 32,
+ [2][1][RTW89_FCC][15] = 58,
+ [2][1][RTW89_ETSI][15] = 32,
+ [2][1][RTW89_MKK][15] = 58,
+ [2][1][RTW89_IC][15] = 58,
+ [2][1][RTW89_KCC][15] = 54,
+ [2][1][RTW89_ACMA][15] = 32,
+ [2][1][RTW89_CHILE][15] = 42,
+ [2][1][RTW89_UKRAINE][15] = 32,
+ [2][1][RTW89_MEXICO][15] = 58,
+ [2][1][RTW89_CN][15] = 127,
+ [2][1][RTW89_QATAR][15] = 32,
+ [2][1][RTW89_FCC][17] = 58,
+ [2][1][RTW89_ETSI][17] = 32,
+ [2][1][RTW89_MKK][17] = 58,
+ [2][1][RTW89_IC][17] = 58,
+ [2][1][RTW89_KCC][17] = 54,
+ [2][1][RTW89_ACMA][17] = 32,
+ [2][1][RTW89_CHILE][17] = 42,
+ [2][1][RTW89_UKRAINE][17] = 32,
+ [2][1][RTW89_MEXICO][17] = 58,
+ [2][1][RTW89_CN][17] = 127,
+ [2][1][RTW89_QATAR][17] = 32,
+ [2][1][RTW89_FCC][19] = 58,
+ [2][1][RTW89_ETSI][19] = 32,
+ [2][1][RTW89_MKK][19] = 58,
+ [2][1][RTW89_IC][19] = 58,
+ [2][1][RTW89_KCC][19] = 54,
+ [2][1][RTW89_ACMA][19] = 32,
+ [2][1][RTW89_CHILE][19] = 42,
+ [2][1][RTW89_UKRAINE][19] = 32,
+ [2][1][RTW89_MEXICO][19] = 58,
+ [2][1][RTW89_CN][19] = 127,
+ [2][1][RTW89_QATAR][19] = 32,
+ [2][1][RTW89_FCC][21] = 58,
+ [2][1][RTW89_ETSI][21] = 32,
+ [2][1][RTW89_MKK][21] = 58,
+ [2][1][RTW89_IC][21] = 58,
+ [2][1][RTW89_KCC][21] = 54,
+ [2][1][RTW89_ACMA][21] = 32,
+ [2][1][RTW89_CHILE][21] = 42,
+ [2][1][RTW89_UKRAINE][21] = 32,
+ [2][1][RTW89_MEXICO][21] = 58,
+ [2][1][RTW89_CN][21] = 127,
+ [2][1][RTW89_QATAR][21] = 32,
+ [2][1][RTW89_FCC][23] = 58,
+ [2][1][RTW89_ETSI][23] = 32,
+ [2][1][RTW89_MKK][23] = 58,
+ [2][1][RTW89_IC][23] = 58,
+ [2][1][RTW89_KCC][23] = 54,
+ [2][1][RTW89_ACMA][23] = 32,
+ [2][1][RTW89_CHILE][23] = 42,
+ [2][1][RTW89_UKRAINE][23] = 32,
+ [2][1][RTW89_MEXICO][23] = 58,
+ [2][1][RTW89_CN][23] = 127,
+ [2][1][RTW89_QATAR][23] = 32,
+ [2][1][RTW89_FCC][25] = 58,
+ [2][1][RTW89_ETSI][25] = 32,
+ [2][1][RTW89_MKK][25] = 58,
+ [2][1][RTW89_IC][25] = 127,
+ [2][1][RTW89_KCC][25] = 54,
+ [2][1][RTW89_ACMA][25] = 127,
+ [2][1][RTW89_CHILE][25] = 42,
+ [2][1][RTW89_UKRAINE][25] = 32,
+ [2][1][RTW89_MEXICO][25] = 58,
+ [2][1][RTW89_CN][25] = 127,
+ [2][1][RTW89_QATAR][25] = 32,
+ [2][1][RTW89_FCC][27] = 58,
+ [2][1][RTW89_ETSI][27] = 32,
+ [2][1][RTW89_MKK][27] = 58,
+ [2][1][RTW89_IC][27] = 127,
+ [2][1][RTW89_KCC][27] = 54,
+ [2][1][RTW89_ACMA][27] = 127,
+ [2][1][RTW89_CHILE][27] = 42,
+ [2][1][RTW89_UKRAINE][27] = 32,
+ [2][1][RTW89_MEXICO][27] = 58,
+ [2][1][RTW89_CN][27] = 127,
+ [2][1][RTW89_QATAR][27] = 32,
+ [2][1][RTW89_FCC][29] = 58,
+ [2][1][RTW89_ETSI][29] = 32,
+ [2][1][RTW89_MKK][29] = 58,
+ [2][1][RTW89_IC][29] = 127,
+ [2][1][RTW89_KCC][29] = 54,
+ [2][1][RTW89_ACMA][29] = 127,
+ [2][1][RTW89_CHILE][29] = 42,
+ [2][1][RTW89_UKRAINE][29] = 32,
+ [2][1][RTW89_MEXICO][29] = 58,
+ [2][1][RTW89_CN][29] = 127,
+ [2][1][RTW89_QATAR][29] = 32,
+ [2][1][RTW89_FCC][31] = 58,
+ [2][1][RTW89_ETSI][31] = 32,
+ [2][1][RTW89_MKK][31] = 58,
+ [2][1][RTW89_IC][31] = 58,
+ [2][1][RTW89_KCC][31] = 54,
+ [2][1][RTW89_ACMA][31] = 32,
+ [2][1][RTW89_CHILE][31] = 42,
+ [2][1][RTW89_UKRAINE][31] = 32,
+ [2][1][RTW89_MEXICO][31] = 58,
+ [2][1][RTW89_CN][31] = 127,
+ [2][1][RTW89_QATAR][31] = 32,
+ [2][1][RTW89_FCC][33] = 58,
+ [2][1][RTW89_ETSI][33] = 32,
+ [2][1][RTW89_MKK][33] = 58,
+ [2][1][RTW89_IC][33] = 58,
+ [2][1][RTW89_KCC][33] = 54,
+ [2][1][RTW89_ACMA][33] = 32,
+ [2][1][RTW89_CHILE][33] = 42,
+ [2][1][RTW89_UKRAINE][33] = 32,
+ [2][1][RTW89_MEXICO][33] = 58,
+ [2][1][RTW89_CN][33] = 127,
+ [2][1][RTW89_QATAR][33] = 32,
+ [2][1][RTW89_FCC][35] = 58,
+ [2][1][RTW89_ETSI][35] = 32,
+ [2][1][RTW89_MKK][35] = 58,
+ [2][1][RTW89_IC][35] = 58,
+ [2][1][RTW89_KCC][35] = 54,
+ [2][1][RTW89_ACMA][35] = 32,
+ [2][1][RTW89_CHILE][35] = 42,
+ [2][1][RTW89_UKRAINE][35] = 32,
+ [2][1][RTW89_MEXICO][35] = 58,
+ [2][1][RTW89_CN][35] = 127,
+ [2][1][RTW89_QATAR][35] = 32,
+ [2][1][RTW89_FCC][37] = 58,
+ [2][1][RTW89_ETSI][37] = 127,
+ [2][1][RTW89_MKK][37] = 58,
+ [2][1][RTW89_IC][37] = 58,
+ [2][1][RTW89_KCC][37] = 54,
+ [2][1][RTW89_ACMA][37] = 62,
+ [2][1][RTW89_CHILE][37] = 42,
+ [2][1][RTW89_UKRAINE][37] = 127,
+ [2][1][RTW89_MEXICO][37] = 58,
+ [2][1][RTW89_CN][37] = 127,
+ [2][1][RTW89_QATAR][37] = 127,
+ [2][1][RTW89_FCC][38] = 76,
+ [2][1][RTW89_ETSI][38] = 16,
+ [2][1][RTW89_MKK][38] = 127,
+ [2][1][RTW89_IC][38] = 76,
+ [2][1][RTW89_KCC][38] = 54,
+ [2][1][RTW89_ACMA][38] = 76,
+ [2][1][RTW89_CHILE][38] = 42,
+ [2][1][RTW89_UKRAINE][38] = 16,
+ [2][1][RTW89_MEXICO][38] = 76,
+ [2][1][RTW89_CN][38] = 64,
+ [2][1][RTW89_QATAR][38] = 16,
+ [2][1][RTW89_FCC][40] = 76,
+ [2][1][RTW89_ETSI][40] = 16,
+ [2][1][RTW89_MKK][40] = 127,
+ [2][1][RTW89_IC][40] = 76,
+ [2][1][RTW89_KCC][40] = 54,
+ [2][1][RTW89_ACMA][40] = 76,
+ [2][1][RTW89_CHILE][40] = 42,
+ [2][1][RTW89_UKRAINE][40] = 16,
+ [2][1][RTW89_MEXICO][40] = 76,
+ [2][1][RTW89_CN][40] = 64,
+ [2][1][RTW89_QATAR][40] = 16,
+ [2][1][RTW89_FCC][42] = 76,
+ [2][1][RTW89_ETSI][42] = 16,
+ [2][1][RTW89_MKK][42] = 127,
+ [2][1][RTW89_IC][42] = 76,
+ [2][1][RTW89_KCC][42] = 54,
+ [2][1][RTW89_ACMA][42] = 76,
+ [2][1][RTW89_CHILE][42] = 42,
+ [2][1][RTW89_UKRAINE][42] = 16,
+ [2][1][RTW89_MEXICO][42] = 76,
+ [2][1][RTW89_CN][42] = 64,
+ [2][1][RTW89_QATAR][42] = 16,
+ [2][1][RTW89_FCC][44] = 76,
+ [2][1][RTW89_ETSI][44] = 16,
+ [2][1][RTW89_MKK][44] = 127,
+ [2][1][RTW89_IC][44] = 76,
+ [2][1][RTW89_KCC][44] = 54,
+ [2][1][RTW89_ACMA][44] = 76,
+ [2][1][RTW89_CHILE][44] = 42,
+ [2][1][RTW89_UKRAINE][44] = 16,
+ [2][1][RTW89_MEXICO][44] = 76,
+ [2][1][RTW89_CN][44] = 64,
+ [2][1][RTW89_QATAR][44] = 16,
+ [2][1][RTW89_FCC][46] = 76,
+ [2][1][RTW89_ETSI][46] = 16,
+ [2][1][RTW89_MKK][46] = 127,
+ [2][1][RTW89_IC][46] = 76,
+ [2][1][RTW89_KCC][46] = 54,
+ [2][1][RTW89_ACMA][46] = 76,
+ [2][1][RTW89_CHILE][46] = 42,
+ [2][1][RTW89_UKRAINE][46] = 16,
+ [2][1][RTW89_MEXICO][46] = 76,
+ [2][1][RTW89_CN][46] = 64,
+ [2][1][RTW89_QATAR][46] = 16,
};
#define DECLARE_DIG_TABLE(name) \
diff --git a/drivers/net/wireless/realtek/rtw89/txrx.h b/drivers/net/wireless/realtek/rtw89/txrx.h
index f1e0fe3..75b1124 100644
--- a/drivers/net/wireless/realtek/rtw89/txrx.h
+++ b/drivers/net/wireless/realtek/rtw89/txrx.h
@@ -140,52 +140,56 @@
le32_get_bits((rxdesc)->dword5, GENMASK(7, 0))
#define RTW89_GET_RXINFO_USR_NUM(rpt) \
- le32_get_bits(*((__le32 *)rpt), GENMASK(3, 0))
+ le32_get_bits(*((const __le32 *)rpt), GENMASK(3, 0))
#define RTW89_GET_RXINFO_FW_DEFINE(rpt) \
- le32_get_bits(*((__le32 *)rpt), GENMASK(15, 8))
+ le32_get_bits(*((const __le32 *)rpt), GENMASK(15, 8))
#define RTW89_GET_RXINFO_LSIG_LEN(rpt) \
- le32_get_bits(*((__le32 *)rpt), GENMASK(27, 16))
+ le32_get_bits(*((const __le32 *)rpt), GENMASK(27, 16))
#define RTW89_GET_RXINFO_IS_TO_SELF(rpt) \
- le32_get_bits(*((__le32 *)rpt), BIT(28))
+ le32_get_bits(*((const __le32 *)rpt), BIT(28))
#define RTW89_GET_RXINFO_RX_CNT_VLD(rpt) \
- le32_get_bits(*((__le32 *)rpt), BIT(29))
+ le32_get_bits(*((const __le32 *)rpt), BIT(29))
#define RTW89_GET_RXINFO_LONG_RXD(rpt) \
- le32_get_bits(*((__le32 *)rpt), GENMASK(31, 30))
+ le32_get_bits(*((const __le32 *)rpt), GENMASK(31, 30))
#define RTW89_GET_RXINFO_SERVICE(rpt) \
- le32_get_bits(*((__le32 *)(rpt) + 1), GENMASK(15, 0))
+ le32_get_bits(*((const __le32 *)(rpt) + 1), GENMASK(15, 0))
#define RTW89_GET_RXINFO_PLCP_LEN(rpt) \
- le32_get_bits(*((__le32 *)(rpt) + 1), GENMASK(23, 16))
+ le32_get_bits(*((const __le32 *)(rpt) + 1), GENMASK(23, 16))
#define RTW89_GET_RXINFO_MAC_ID_VALID(rpt, usr) \
- le32_get_bits(*((__le32 *)(rpt) + (usr) + 2), BIT(0))
+ le32_get_bits(*((const __le32 *)(rpt) + (usr) + 2), BIT(0))
#define RTW89_GET_RXINFO_DATA(rpt, usr) \
- le32_get_bits(*((__le32 *)(rpt) + (usr) + 2), BIT(1))
+ le32_get_bits(*((const __le32 *)(rpt) + (usr) + 2), BIT(1))
#define RTW89_GET_RXINFO_CTRL(rpt, usr) \
- le32_get_bits(*((__le32 *)(rpt) + (usr) + 2), BIT(2))
+ le32_get_bits(*((const __le32 *)(rpt) + (usr) + 2), BIT(2))
#define RTW89_GET_RXINFO_MGMT(rpt, usr) \
- le32_get_bits(*((__le32 *)(rpt) + (usr) + 2), BIT(3))
+ le32_get_bits(*((const __le32 *)(rpt) + (usr) + 2), BIT(3))
#define RTW89_GET_RXINFO_BCM(rpt, usr) \
- le32_get_bits(*((__le32 *)(rpt) + (usr) + 2), BIT(4))
+ le32_get_bits(*((const __le32 *)(rpt) + (usr) + 2), BIT(4))
#define RTW89_GET_RXINFO_MACID(rpt, usr) \
- le32_get_bits(*((__le32 *)(rpt) + (usr) + 2), GENMASK(15, 8))
+ le32_get_bits(*((const __le32 *)(rpt) + (usr) + 2), GENMASK(15, 8))
+#define RTW89_GET_PHY_STS_IE_MAP(sts) \
+ le32_get_bits(*((const __le32 *)(sts)), GENMASK(4, 0))
#define RTW89_GET_PHY_STS_RSSI_A(sts) \
- le32_get_bits(*((__le32 *)(sts) + 1), GENMASK(7, 0))
+ le32_get_bits(*((const __le32 *)(sts) + 1), GENMASK(7, 0))
#define RTW89_GET_PHY_STS_RSSI_B(sts) \
- le32_get_bits(*((__le32 *)(sts) + 1), GENMASK(15, 8))
+ le32_get_bits(*((const __le32 *)(sts) + 1), GENMASK(15, 8))
#define RTW89_GET_PHY_STS_RSSI_C(sts) \
- le32_get_bits(*((__le32 *)(sts) + 1), GENMASK(23, 16))
+ le32_get_bits(*((const __le32 *)(sts) + 1), GENMASK(23, 16))
#define RTW89_GET_PHY_STS_RSSI_D(sts) \
- le32_get_bits(*((__le32 *)(sts) + 1), GENMASK(31, 24))
+ le32_get_bits(*((const __le32 *)(sts) + 1), GENMASK(31, 24))
#define RTW89_GET_PHY_STS_LEN(sts) \
- le32_get_bits(*((__le32 *)sts), GENMASK(15, 8))
+ le32_get_bits(*((const __le32 *)sts), GENMASK(15, 8))
#define RTW89_GET_PHY_STS_RSSI_AVG(sts) \
- le32_get_bits(*((__le32 *)sts), GENMASK(31, 24))
+ le32_get_bits(*((const __le32 *)sts), GENMASK(31, 24))
#define RTW89_GET_PHY_STS_IE_TYPE(ie) \
- le32_get_bits(*((__le32 *)ie), GENMASK(4, 0))
+ le32_get_bits(*((const __le32 *)ie), GENMASK(4, 0))
#define RTW89_GET_PHY_STS_IE_LEN(ie) \
- le32_get_bits(*((__le32 *)ie), GENMASK(11, 5))
-#define RTW89_GET_PHY_STS_IE0_CFO(ie) \
- le32_get_bits(*((__le32 *)(ie) + 1), GENMASK(31, 20))
+ le32_get_bits(*((const __le32 *)ie), GENMASK(11, 5))
+#define RTW89_GET_PHY_STS_IE01_CH_IDX(ie) \
+ le32_get_bits(*((const __le32 *)ie), GENMASK(23, 16))
+#define RTW89_GET_PHY_STS_IE01_CFO(ie) \
+ le32_get_bits(*((const __le32 *)(ie) + 1), GENMASK(31, 20))
enum rtw89_tx_channel {
RTW89_TXCH_ACH0 = 0,
@@ -251,45 +255,6 @@ enum rtw89_tx_qsel {
/* reserved */
};
-enum rtw89_phy_status_ie_type {
- RTW89_PHYSTS_IE00_CMN_CCK = 0,
- RTW89_PHYSTS_IE01_CMN_OFDM = 1,
- RTW89_PHYSTS_IE02_CMN_EXT_AX = 2,
- RTW89_PHYSTS_IE03_CMN_EXT_SEG_1 = 3,
- RTW89_PHYSTS_IE04_CMN_EXT_PATH_A = 4,
- RTW89_PHYSTS_IE05_CMN_EXT_PATH_B = 5,
- RTW89_PHYSTS_IE06_CMN_EXT_PATH_C = 6,
- RTW89_PHYSTS_IE07_CMN_EXT_PATH_D = 7,
- RTW89_PHYSTS_IE08_FTR_CH = 8,
- RTW89_PHYSTS_IE09_FTR_PLCP_0 = 9,
- RTW89_PHYSTS_IE10_FTR_PLCP_EXT = 10,
- RTW89_PHYSTS_IE11_FTR_PLCP_HISTOGRAM = 11,
- RTW89_PHYSTS_IE12_MU_EIGEN_INFO = 12,
- RTW89_PHYSTS_IE13_DL_MU_DEF = 13,
- RTW89_PHYSTS_IE14_TB_UL_CQI = 14,
- RTW89_PHYSTS_IE15_TB_UL_DEF = 15,
- RTW89_PHYSTS_IE16_RSVD16 = 16,
- RTW89_PHYSTS_IE17_TB_UL_CTRL = 17,
- RTW89_PHYSTS_IE18_DBG_OFDM_FD_CMN = 18,
- RTW89_PHYSTS_IE19_DBG_OFDM_TD_CMN = 19,
- RTW89_PHYSTS_IE20_DBG_OFDM_FD_USER_SEG_0 = 20,
- RTW89_PHYSTS_IE21_DBG_OFDM_FD_USER_SEG_1 = 21,
- RTW89_PHYSTS_IE22_DBG_OFDM_FD_USER_AGC = 22,
- RTW89_PHYSTS_IE23_RSVD23 = 23,
- RTW89_PHYSTS_IE24_DBG_OFDM_TD_PATH_A = 24,
- RTW89_PHYSTS_IE25_DBG_OFDM_TD_PATH_B = 25,
- RTW89_PHYSTS_IE26_DBG_OFDM_TD_PATH_C = 26,
- RTW89_PHYSTS_IE27_DBG_OFDM_TD_PATH_D = 27,
- RTW89_PHYSTS_IE28_DBG_CCK_PATH_A = 28,
- RTW89_PHYSTS_IE29_DBG_CCK_PATH_B = 29,
- RTW89_PHYSTS_IE30_DBG_CCK_PATH_C = 30,
- RTW89_PHYSTS_IE31_DBG_CCK_PATH_D = 31,
-
- /* keep last */
- RTW89_PHYSTS_IE_NUM,
- RTW89_PHYSTS_IE_MAX = RTW89_PHYSTS_IE_NUM - 1
-};
-
static inline u8 rtw89_core_get_qsel(struct rtw89_dev *rtwdev, u8 tid)
{
switch (tid) {
diff --git a/drivers/net/wireless/rsi/rsi_91x_mac80211.c b/drivers/net/wireless/rsi/rsi_91x_mac80211.c
index e70c1c7f..913e11f 100644
--- a/drivers/net/wireless/rsi/rsi_91x_mac80211.c
+++ b/drivers/net/wireless/rsi/rsi_91x_mac80211.c
@@ -1108,6 +1108,9 @@ static int rsi_mac80211_ampdu_action(struct ieee80211_hw *hw,
break;
}
+ if (ii >= RSI_MAX_VIFS)
+ return status;
+
mutex_lock(&common->mutex);
if (ssn != NULL)
diff --git a/drivers/net/wireless/rsi/rsi_91x_main.c b/drivers/net/wireless/rsi/rsi_91x_main.c
index f1bf71e..5d1490f 100644
--- a/drivers/net/wireless/rsi/rsi_91x_main.c
+++ b/drivers/net/wireless/rsi/rsi_91x_main.c
@@ -23,6 +23,7 @@
#include "rsi_common.h"
#include "rsi_coex.h"
#include "rsi_hal.h"
+#include "rsi_usb.h"
u32 rsi_zone_enabled = /* INFO_ZONE |
INIT_ZONE |
@@ -168,6 +169,9 @@ int rsi_read_pkt(struct rsi_common *common, u8 *rx_pkt, s32 rcv_pkt_len)
frame_desc = &rx_pkt[index];
actual_length = *(u16 *)&frame_desc[0];
offset = *(u16 *)&frame_desc[2];
+ if (!rcv_pkt_len && offset >
+ RSI_MAX_RX_USB_PKT_SIZE - FRAME_DESC_SZ)
+ goto fail;
queueno = rsi_get_queueno(frame_desc, offset);
length = rsi_get_length(frame_desc, offset);
diff --git a/drivers/net/wireless/rsi/rsi_91x_usb.c b/drivers/net/wireless/rsi/rsi_91x_usb.c
index 6821ea9..66fe386 100644
--- a/drivers/net/wireless/rsi/rsi_91x_usb.c
+++ b/drivers/net/wireless/rsi/rsi_91x_usb.c
@@ -269,8 +269,12 @@ static void rsi_rx_done_handler(struct urb *urb)
struct rsi_91x_usbdev *dev = (struct rsi_91x_usbdev *)rx_cb->data;
int status = -EINVAL;
+ if (!rx_cb->rx_skb)
+ return;
+
if (urb->status) {
dev_kfree_skb(rx_cb->rx_skb);
+ rx_cb->rx_skb = NULL;
return;
}
@@ -294,8 +298,10 @@ static void rsi_rx_done_handler(struct urb *urb)
if (rsi_rx_urb_submit(dev->priv, rx_cb->ep_num, GFP_ATOMIC))
rsi_dbg(ERR_ZONE, "%s: Failed in urb submission", __func__);
- if (status)
+ if (status) {
dev_kfree_skb(rx_cb->rx_skb);
+ rx_cb->rx_skb = NULL;
+ }
}
static void rsi_rx_urb_kill(struct rsi_hw *adapter, u8 ep_num)
@@ -324,7 +330,6 @@ static int rsi_rx_urb_submit(struct rsi_hw *adapter, u8 ep_num, gfp_t mem_flags)
struct sk_buff *skb;
u8 dword_align_bytes = 0;
-#define RSI_MAX_RX_USB_PKT_SIZE 3000
skb = dev_alloc_skb(RSI_MAX_RX_USB_PKT_SIZE);
if (!skb)
return -ENOMEM;
diff --git a/drivers/net/wireless/rsi/rsi_usb.h b/drivers/net/wireless/rsi/rsi_usb.h
index 254d19b..9618517 100644
--- a/drivers/net/wireless/rsi/rsi_usb.h
+++ b/drivers/net/wireless/rsi/rsi_usb.h
@@ -44,6 +44,8 @@
#define RSI_USB_BUF_SIZE 4096
#define RSI_USB_CTRL_BUF_SIZE 0x04
+#define RSI_MAX_RX_USB_PKT_SIZE 3000
+
struct rx_usb_ctrl_block {
u8 *data;
struct urb *rx_urb;
diff --git a/drivers/net/wireless/ti/wl1251/main.c b/drivers/net/wireless/ti/wl1251/main.c
index 136a0d3..a25a614 100644
--- a/drivers/net/wireless/ti/wl1251/main.c
+++ b/drivers/net/wireless/ti/wl1251/main.c
@@ -1520,6 +1520,12 @@ int wl1251_init_ieee80211(struct wl1251 *wl)
wl->hw->wiphy->interface_modes = BIT(NL80211_IFTYPE_STATION) |
BIT(NL80211_IFTYPE_ADHOC);
wl->hw->wiphy->max_scan_ssids = 1;
+
+ /* We set max_scan_ie_len to a random value to make wpa_supplicant scans not
+ * fail, as the driver will the ignore the extra passed IEs anyway
+ */
+ wl->hw->wiphy->max_scan_ie_len = 512;
+
wl->hw->wiphy->bands[NL80211_BAND_2GHZ] = &wl1251_band_2ghz;
wl->hw->queues = 4;
diff --git a/drivers/net/wireless/ti/wlcore/sdio.c b/drivers/net/wireless/ti/wlcore/sdio.c
index 9fd8cf2..72fc41a 100644
--- a/drivers/net/wireless/ti/wlcore/sdio.c
+++ b/drivers/net/wireless/ti/wlcore/sdio.c
@@ -26,7 +26,7 @@
#include "wl12xx_80211.h"
#include "io.h"
-static bool dump = false;
+static bool dump;
struct wl12xx_sdio_glue {
struct device *dev;
diff --git a/drivers/net/wwan/Kconfig b/drivers/net/wwan/Kconfig
index 17543be..609fd4a 100644
--- a/drivers/net/wwan/Kconfig
+++ b/drivers/net/wwan/Kconfig
@@ -16,6 +16,17 @@
if WWAN
+config WWAN_DEBUGFS
+ bool "WWAN devices debugfs interface" if EXPERT
+ depends on DEBUG_FS
+ default y
+ help
+ Enables debugfs infrastructure for the WWAN core and device drivers.
+
+ If this option is selected, then you can find the debug interface
+ elements for each WWAN device in a directory that is corresponding to
+ the device name: debugfs/wwan/wwanX.
+
config WWAN_HWSIM
tristate "Simulated WWAN device"
help
@@ -50,6 +61,19 @@
To compile this driver as a module, choose M here: the module will be
called mhi_wwan_mbim.
+config QCOM_BAM_DMUX
+ tristate "Qualcomm BAM-DMUX WWAN network driver"
+ depends on (DMA_ENGINE && PM && QCOM_SMEM_STATE) || COMPILE_TEST
+ help
+ The BAM Data Multiplexer provides access to the network data channels
+ of modems integrated into many older Qualcomm SoCs, e.g. Qualcomm
+ MSM8916 or MSM8974. The connection can be established via QMI/AT from
+ userspace with control ports available through the WWAN subsystem
+ (CONFIG_RPMSG_WWAN_CTRL) or QRTR network sockets (CONFIG_QRTR).
+
+ To compile this driver as a module, choose M here: the module will be
+ called qcom_bam_dmux.
+
config RPMSG_WWAN_CTRL
tristate "RPMSG WWAN control driver"
depends on RPMSG
@@ -72,6 +96,7 @@
tristate "IOSM Driver for Intel M.2 WWAN Device"
depends on INTEL_IOMMU
select NET_DEVLINK
+ select RELAY if WWAN_DEBUGFS
help
This driver enables Intel M.2 WWAN Device communication.
diff --git a/drivers/net/wwan/Makefile b/drivers/net/wwan/Makefile
index fe51fee..e722650 100644
--- a/drivers/net/wwan/Makefile
+++ b/drivers/net/wwan/Makefile
@@ -10,5 +10,6 @@
obj-$(CONFIG_MHI_WWAN_CTRL) += mhi_wwan_ctrl.o
obj-$(CONFIG_MHI_WWAN_MBIM) += mhi_wwan_mbim.o
+obj-$(CONFIG_QCOM_BAM_DMUX) += qcom_bam_dmux.o
obj-$(CONFIG_RPMSG_WWAN_CTRL) += rpmsg_wwan_ctrl.o
obj-$(CONFIG_IOSM) += iosm/
diff --git a/drivers/net/wwan/iosm/Makefile b/drivers/net/wwan/iosm/Makefile
index b838034..fa8d6af 100644
--- a/drivers/net/wwan/iosm/Makefile
+++ b/drivers/net/wwan/iosm/Makefile
@@ -23,4 +23,8 @@
iosm_ipc_flash.o \
iosm_ipc_coredump.o
+iosm-$(CONFIG_WWAN_DEBUGFS) += \
+ iosm_ipc_debugfs.o \
+ iosm_ipc_trace.o
+
obj-$(CONFIG_IOSM) := iosm.o
diff --git a/drivers/net/wwan/iosm/iosm_ipc_debugfs.c b/drivers/net/wwan/iosm/iosm_ipc_debugfs.c
new file mode 100644
index 0000000..f2f5775
--- /dev/null
+++ b/drivers/net/wwan/iosm/iosm_ipc_debugfs.c
@@ -0,0 +1,29 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2020-2021 Intel Corporation.
+ */
+
+#include <linux/debugfs.h>
+#include <linux/wwan.h>
+
+#include "iosm_ipc_imem.h"
+#include "iosm_ipc_trace.h"
+#include "iosm_ipc_debugfs.h"
+
+void ipc_debugfs_init(struct iosm_imem *ipc_imem)
+{
+ struct dentry *debugfs_pdev = wwan_get_debugfs_dir(ipc_imem->dev);
+
+ ipc_imem->debugfs_dir = debugfs_create_dir(KBUILD_MODNAME,
+ debugfs_pdev);
+
+ ipc_imem->trace = ipc_trace_init(ipc_imem);
+ if (!ipc_imem->trace)
+ dev_warn(ipc_imem->dev, "trace channel init failed");
+}
+
+void ipc_debugfs_deinit(struct iosm_imem *ipc_imem)
+{
+ ipc_trace_deinit(ipc_imem->trace);
+ debugfs_remove_recursive(ipc_imem->debugfs_dir);
+}
diff --git a/drivers/net/wwan/iosm/iosm_ipc_debugfs.h b/drivers/net/wwan/iosm/iosm_ipc_debugfs.h
new file mode 100644
index 0000000..8a84bfa
--- /dev/null
+++ b/drivers/net/wwan/iosm/iosm_ipc_debugfs.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: GPL-2.0-only
+ *
+ * Copyright (C) 2020-2021 Intel Corporation.
+ */
+
+#ifndef IOSM_IPC_DEBUGFS_H
+#define IOSM_IPC_DEBUGFS_H
+
+#ifdef CONFIG_WWAN_DEBUGFS
+void ipc_debugfs_init(struct iosm_imem *ipc_imem);
+void ipc_debugfs_deinit(struct iosm_imem *ipc_imem);
+#else
+static inline void ipc_debugfs_init(struct iosm_imem *ipc_imem) {}
+static inline void ipc_debugfs_deinit(struct iosm_imem *ipc_imem) {}
+#endif
+
+#endif
diff --git a/drivers/net/wwan/iosm/iosm_ipc_imem.c b/drivers/net/wwan/iosm/iosm_ipc_imem.c
index 12c03da..f9e8e0e 100644
--- a/drivers/net/wwan/iosm/iosm_ipc_imem.c
+++ b/drivers/net/wwan/iosm/iosm_ipc_imem.c
@@ -10,6 +10,8 @@
#include "iosm_ipc_flash.h"
#include "iosm_ipc_imem.h"
#include "iosm_ipc_port.h"
+#include "iosm_ipc_trace.h"
+#include "iosm_ipc_debugfs.h"
/* Check the wwan ips if it is valid with Channel as input. */
static int ipc_imem_check_wwan_ips(struct ipc_mem_channel *chnl)
@@ -132,7 +134,6 @@ static int ipc_imem_setup_cp_mux_cap_init(struct iosm_imem *ipc_imem,
* for channel alloc function.
*/
cfg->instance_id = IPC_MEM_MUX_IP_CH_IF_ID;
- cfg->nr_sessions = IPC_MEM_MUX_IP_SESSION_ENTRIES;
return 0;
}
@@ -269,9 +270,14 @@ static void ipc_imem_dl_skb_process(struct iosm_imem *ipc_imem,
switch (pipe->channel->ctype) {
case IPC_CTYPE_CTRL:
port_id = pipe->channel->channel_id;
+ ipc_pcie_addr_unmap(ipc_imem->pcie, IPC_CB(skb)->len,
+ IPC_CB(skb)->mapping,
+ IPC_CB(skb)->direction);
if (port_id == IPC_MEM_CTRL_CHL_ID_7)
ipc_imem_sys_devlink_notify_rx(ipc_imem->ipc_devlink,
skb);
+ else if (ipc_is_trace_channel(ipc_imem, port_id))
+ ipc_trace_port_rx(ipc_imem, skb);
else
wwan_port_rx(ipc_imem->ipc_port[port_id]->iosm_port,
skb);
@@ -555,6 +561,8 @@ static void ipc_imem_run_state_worker(struct work_struct *instance)
ctrl_chl_idx++;
}
+ ipc_debugfs_init(ipc_imem);
+
ipc_task_queue_send_task(ipc_imem, ipc_imem_send_mdm_rdy_cb, 0, NULL, 0,
false);
@@ -1170,6 +1178,7 @@ void ipc_imem_cleanup(struct iosm_imem *ipc_imem)
if (test_and_clear_bit(FULLY_FUNCTIONAL, &ipc_imem->flag)) {
ipc_mux_deinit(ipc_imem->mux);
+ ipc_debugfs_deinit(ipc_imem);
ipc_wwan_deinit(ipc_imem->wwan);
ipc_port_deinit(ipc_imem->ipc_port);
}
diff --git a/drivers/net/wwan/iosm/iosm_ipc_imem.h b/drivers/net/wwan/iosm/iosm_ipc_imem.h
index 6b8a837..98554e9 100644
--- a/drivers/net/wwan/iosm/iosm_ipc_imem.h
+++ b/drivers/net/wwan/iosm/iosm_ipc_imem.h
@@ -305,6 +305,7 @@ enum ipc_phase {
* @sio: IPC SIO data structure pointer
* @ipc_port: IPC PORT data structure pointer
* @pcie: IPC PCIe
+ * @trace: IPC trace data structure pointer
* @dev: Pointer to device structure
* @ipc_requested_state: Expected IPC state on CP.
* @channels: Channel list with UL/DL pipe pairs.
@@ -339,6 +340,7 @@ enum ipc_phase {
* @ev_mux_net_transmit_pending:0 means inform the IPC tasklet to pass
* @reset_det_n: Reset detect flag
* @pcie_wake_n: Pcie wake flag
+ * @debugfs_dir: Debug FS directory for driver-specific entries
*/
struct iosm_imem {
struct iosm_mmio *mmio;
@@ -348,6 +350,9 @@ struct iosm_imem {
struct iosm_mux *mux;
struct iosm_cdev *ipc_port[IPC_MEM_MAX_CHANNELS];
struct iosm_pcie *pcie;
+#ifdef CONFIG_WWAN_DEBUGFS
+ struct iosm_trace *trace;
+#endif
struct device *dev;
enum ipc_mem_device_ipc_state ipc_requested_state;
struct ipc_mem_channel channels[IPC_MEM_MAX_CHANNELS];
@@ -376,6 +381,9 @@ struct iosm_imem {
ev_mux_net_transmit_pending:1,
reset_det_n:1,
pcie_wake_n:1;
+#ifdef CONFIG_WWAN_DEBUGFS
+ struct dentry *debugfs_dir;
+#endif
};
/**
diff --git a/drivers/net/wwan/iosm/iosm_ipc_imem_ops.c b/drivers/net/wwan/iosm/iosm_ipc_imem_ops.c
index 831cdae..57304a5 100644
--- a/drivers/net/wwan/iosm/iosm_ipc_imem_ops.c
+++ b/drivers/net/wwan/iosm/iosm_ipc_imem_ops.c
@@ -176,11 +176,14 @@ static bool ipc_imem_is_channel_active(struct iosm_imem *ipc_imem,
return false;
}
-/* Release a sio link to CP. */
-void ipc_imem_sys_cdev_close(struct iosm_cdev *ipc_cdev)
+/**
+ * ipc_imem_sys_port_close - Release a sio link to CP.
+ * @ipc_imem: Imem instance.
+ * @channel: Channel instance.
+ */
+void ipc_imem_sys_port_close(struct iosm_imem *ipc_imem,
+ struct ipc_mem_channel *channel)
{
- struct iosm_imem *ipc_imem = ipc_cdev->ipc_imem;
- struct ipc_mem_channel *channel = ipc_cdev->channel;
enum ipc_phase curr_phase;
int status = 0;
u32 tail = 0;
@@ -638,6 +641,6 @@ int ipc_imem_sys_devlink_read(struct iosm_devlink *devlink, u8 *data,
memcpy(data, skb->data, skb->len);
devlink_read_fail:
- ipc_pcie_kfree_skb(devlink->pcie, skb);
+ dev_kfree_skb(skb);
return rc;
}
diff --git a/drivers/net/wwan/iosm/iosm_ipc_imem_ops.h b/drivers/net/wwan/iosm/iosm_ipc_imem_ops.h
index f0c88ac..f8afb21 100644
--- a/drivers/net/wwan/iosm/iosm_ipc_imem_ops.h
+++ b/drivers/net/wwan/iosm/iosm_ipc_imem_ops.h
@@ -43,12 +43,8 @@
*/
struct ipc_mem_channel *ipc_imem_sys_port_open(struct iosm_imem *ipc_imem,
int chl_id, int hp_id);
-
-/**
- * ipc_imem_sys_cdev_close - Release a sio link to CP.
- * @ipc_cdev: iosm sio instance.
- */
-void ipc_imem_sys_cdev_close(struct iosm_cdev *ipc_cdev);
+void ipc_imem_sys_port_close(struct iosm_imem *ipc_imem,
+ struct ipc_mem_channel *channel);
/**
* ipc_imem_sys_cdev_write - Route the uplink buffer to CP.
@@ -145,4 +141,5 @@ int ipc_imem_sys_devlink_read(struct iosm_devlink *ipc_devlink, u8 *data,
*/
int ipc_imem_sys_devlink_write(struct iosm_devlink *ipc_devlink,
unsigned char *buf, int count);
+
#endif
diff --git a/drivers/net/wwan/iosm/iosm_ipc_mmio.c b/drivers/net/wwan/iosm/iosm_ipc_mmio.c
index 09f94c1..f09e5e7 100644
--- a/drivers/net/wwan/iosm/iosm_ipc_mmio.c
+++ b/drivers/net/wwan/iosm/iosm_ipc_mmio.c
@@ -192,7 +192,7 @@ void ipc_mmio_config(struct iosm_mmio *ipc_mmio)
iowrite64(0, ipc_mmio->base + ipc_mmio->offset.ap_win_end);
iowrite64(ipc_mmio->context_info_addr,
- ipc_mmio->base + ipc_mmio->offset.context_info);
+ ipc_mmio->base + ipc_mmio->offset.context_info);
}
void ipc_mmio_set_psi_addr_and_size(struct iosm_mmio *ipc_mmio, dma_addr_t addr,
diff --git a/drivers/net/wwan/iosm/iosm_ipc_mux.c b/drivers/net/wwan/iosm/iosm_ipc_mux.c
index c1c77ce..8e66ffe 100644
--- a/drivers/net/wwan/iosm/iosm_ipc_mux.c
+++ b/drivers/net/wwan/iosm/iosm_ipc_mux.c
@@ -97,7 +97,7 @@ static bool ipc_mux_session_open(struct iosm_mux *ipc_mux,
/* Search for a free session interface id. */
if_id = le32_to_cpu(session_open->if_id);
- if (if_id < 0 || if_id >= ipc_mux->nr_sessions) {
+ if (if_id < 0 || if_id >= IPC_MEM_MUX_IP_SESSION_ENTRIES) {
dev_err(ipc_mux->dev, "invalid interface id=%d", if_id);
return false;
}
@@ -129,6 +129,7 @@ static bool ipc_mux_session_open(struct iosm_mux *ipc_mux,
/* Save and return the assigned if id. */
session_open->if_id = cpu_to_le32(if_id);
+ ipc_mux->nr_sessions++;
return true;
}
@@ -151,7 +152,7 @@ static void ipc_mux_session_close(struct iosm_mux *ipc_mux,
/* Copy the session interface id. */
if_id = le32_to_cpu(msg->if_id);
- if (if_id < 0 || if_id >= ipc_mux->nr_sessions) {
+ if (if_id < 0 || if_id >= IPC_MEM_MUX_IP_SESSION_ENTRIES) {
dev_err(ipc_mux->dev, "invalid session id %d", if_id);
return;
}
@@ -170,6 +171,7 @@ static void ipc_mux_session_close(struct iosm_mux *ipc_mux,
ipc_mux->session[if_id].flow_ctl_mask = 0;
ipc_mux_session_reset(ipc_mux, if_id);
+ ipc_mux->nr_sessions--;
}
static void ipc_mux_channel_close(struct iosm_mux *ipc_mux,
@@ -178,7 +180,7 @@ static void ipc_mux_channel_close(struct iosm_mux *ipc_mux,
int i;
/* Free pending session UL packet. */
- for (i = 0; i < ipc_mux->nr_sessions; i++)
+ for (i = 0; i < IPC_MEM_MUX_IP_SESSION_ENTRIES; i++)
if (ipc_mux->session[i].wwan)
ipc_mux_session_reset(ipc_mux, i);
@@ -244,6 +246,11 @@ static int ipc_mux_schedule(struct iosm_mux *ipc_mux, union mux_msg *msg)
/* Release an IP session. */
ipc_mux->event = MUX_E_MUX_SESSION_CLOSE;
ipc_mux_session_close(ipc_mux, &msg->session_close);
+ if (!ipc_mux->nr_sessions) {
+ ipc_mux->event = MUX_E_MUX_CHANNEL_CLOSE;
+ ipc_mux_channel_close(ipc_mux,
+ &msg->channel_close);
+ }
ret = ipc_mux->channel_id;
goto out;
@@ -281,7 +288,6 @@ struct iosm_mux *ipc_mux_init(struct ipc_mux_config *mux_cfg,
ipc_mux->protocol = mux_cfg->protocol;
ipc_mux->ul_flow = mux_cfg->ul_flow;
- ipc_mux->nr_sessions = mux_cfg->nr_sessions;
ipc_mux->instance_id = mux_cfg->instance_id;
ipc_mux->wwan_q_offset = 0;
@@ -340,7 +346,7 @@ static void ipc_mux_restart_tx_for_all_sessions(struct iosm_mux *ipc_mux)
struct mux_session *session;
int idx;
- for (idx = 0; idx < ipc_mux->nr_sessions; idx++) {
+ for (idx = 0; idx < IPC_MEM_MUX_IP_SESSION_ENTRIES; idx++) {
session = &ipc_mux->session[idx];
if (!session->wwan)
@@ -365,7 +371,7 @@ static void ipc_mux_stop_netif_for_all_sessions(struct iosm_mux *ipc_mux)
struct mux_session *session;
int idx;
- for (idx = 0; idx < ipc_mux->nr_sessions; idx++) {
+ for (idx = 0; idx < IPC_MEM_MUX_IP_SESSION_ENTRIES; idx++) {
session = &ipc_mux->session[idx];
if (!session->wwan)
@@ -387,7 +393,7 @@ void ipc_mux_check_n_restart_tx(struct iosm_mux *ipc_mux)
int ipc_mux_get_max_sessions(struct iosm_mux *ipc_mux)
{
- return ipc_mux ? ipc_mux->nr_sessions : -EFAULT;
+ return ipc_mux ? IPC_MEM_MUX_IP_SESSION_ENTRIES : -EFAULT;
}
enum ipc_mux_protocol ipc_mux_get_active_protocol(struct iosm_mux *ipc_mux)
@@ -435,9 +441,11 @@ void ipc_mux_deinit(struct iosm_mux *ipc_mux)
return;
ipc_mux_stop_netif_for_all_sessions(ipc_mux);
- channel_close = &mux_msg.channel_close;
- channel_close->event = MUX_E_MUX_CHANNEL_CLOSE;
- ipc_mux_schedule(ipc_mux, &mux_msg);
+ if (ipc_mux->state == MUX_S_ACTIVE) {
+ channel_close = &mux_msg.channel_close;
+ channel_close->event = MUX_E_MUX_CHANNEL_CLOSE;
+ ipc_mux_schedule(ipc_mux, &mux_msg);
+ }
/* Empty the ADB free list. */
free_list = &ipc_mux->ul_adb.free_list;
diff --git a/drivers/net/wwan/iosm/iosm_ipc_mux.h b/drivers/net/wwan/iosm/iosm_ipc_mux.h
index ddd2cd0..88debaa 100644
--- a/drivers/net/wwan/iosm/iosm_ipc_mux.h
+++ b/drivers/net/wwan/iosm/iosm_ipc_mux.h
@@ -278,7 +278,6 @@ struct iosm_mux {
struct ipc_mux_config {
enum ipc_mux_protocol protocol;
enum ipc_mux_ul_flow ul_flow;
- int nr_sessions;
int instance_id;
};
diff --git a/drivers/net/wwan/iosm/iosm_ipc_mux_codec.c b/drivers/net/wwan/iosm/iosm_ipc_mux_codec.c
index bdb2d32..40fb54a 100644
--- a/drivers/net/wwan/iosm/iosm_ipc_mux_codec.c
+++ b/drivers/net/wwan/iosm/iosm_ipc_mux_codec.c
@@ -175,7 +175,7 @@ static int ipc_mux_dl_dlcmds_decode_process(struct iosm_mux *ipc_mux,
switch (le32_to_cpu(cmdh->command_type)) {
case MUX_LITE_CMD_FLOW_CTL:
- if (cmdh->if_id >= ipc_mux->nr_sessions) {
+ if (cmdh->if_id >= IPC_MEM_MUX_IP_SESSION_ENTRIES) {
dev_err(ipc_mux->dev, "if_id [%d] not valid",
cmdh->if_id);
return -EINVAL; /* No session interface id. */
@@ -307,13 +307,13 @@ static void ipc_mux_dl_fcth_decode(struct iosm_mux *ipc_mux,
}
if_id = fct->if_id;
- if (if_id >= ipc_mux->nr_sessions) {
+ if (if_id >= IPC_MEM_MUX_IP_SESSION_ENTRIES) {
dev_err(ipc_mux->dev, "not supported if_id: %d", if_id);
return;
}
/* Is the session active ? */
- if_id = array_index_nospec(if_id, ipc_mux->nr_sessions);
+ if_id = array_index_nospec(if_id, IPC_MEM_MUX_IP_SESSION_ENTRIES);
wwan = ipc_mux->session[if_id].wwan;
if (!wwan) {
dev_err(ipc_mux->dev, "session Net ID is NULL");
@@ -355,13 +355,13 @@ static void ipc_mux_dl_adgh_decode(struct iosm_mux *ipc_mux,
}
if_id = adgh->if_id;
- if (if_id >= ipc_mux->nr_sessions) {
+ if (if_id >= IPC_MEM_MUX_IP_SESSION_ENTRIES) {
dev_err(ipc_mux->dev, "invalid if_id while decoding %d", if_id);
return;
}
/* Is the session active ? */
- if_id = array_index_nospec(if_id, ipc_mux->nr_sessions);
+ if_id = array_index_nospec(if_id, IPC_MEM_MUX_IP_SESSION_ENTRIES);
wwan = ipc_mux->session[if_id].wwan;
if (!wwan) {
dev_err(ipc_mux->dev, "session Net ID is NULL");
@@ -538,7 +538,7 @@ static void ipc_mux_stop_tx_for_all_sessions(struct iosm_mux *ipc_mux)
struct mux_session *session;
int idx;
- for (idx = 0; idx < ipc_mux->nr_sessions; idx++) {
+ for (idx = 0; idx < IPC_MEM_MUX_IP_SESSION_ENTRIES; idx++) {
session = &ipc_mux->session[idx];
if (!session->wwan)
@@ -563,7 +563,7 @@ static bool ipc_mux_lite_send_qlt(struct iosm_mux *ipc_mux)
qlt_size = offsetof(struct ipc_mem_lite_gen_tbl, vfl) +
MUX_QUEUE_LEVEL * sizeof(struct mux_lite_vfl);
- for (i = 0; i < ipc_mux->nr_sessions; i++) {
+ for (i = 0; i < IPC_MEM_MUX_IP_SESSION_ENTRIES; i++) {
session = &ipc_mux->session[i];
if (!session->wwan || session->flow_ctl_mask)
@@ -777,13 +777,13 @@ bool ipc_mux_ul_data_encode(struct iosm_mux *ipc_mux)
ipc_mux->adb_prep_ongoing = true;
- for (i = 0; i < ipc_mux->nr_sessions; i++) {
+ for (i = 0; i < IPC_MEM_MUX_IP_SESSION_ENTRIES; i++) {
session_id = ipc_mux->rr_next_session;
session = &ipc_mux->session[session_id];
/* Go to next handle rr_next_session overflow */
ipc_mux->rr_next_session++;
- if (ipc_mux->rr_next_session >= ipc_mux->nr_sessions)
+ if (ipc_mux->rr_next_session >= IPC_MEM_MUX_IP_SESSION_ENTRIES)
ipc_mux->rr_next_session = 0;
if (!session->wwan || session->flow_ctl_mask ||
diff --git a/drivers/net/wwan/iosm/iosm_ipc_pcie.c b/drivers/net/wwan/iosm/iosm_ipc_pcie.c
index 2fe88b8..d73894e 100644
--- a/drivers/net/wwan/iosm/iosm_ipc_pcie.c
+++ b/drivers/net/wwan/iosm/iosm_ipc_pcie.c
@@ -363,67 +363,22 @@ static int __maybe_unused ipc_pcie_resume_s2idle(struct iosm_pcie *ipc_pcie)
int __maybe_unused ipc_pcie_suspend(struct iosm_pcie *ipc_pcie)
{
- struct pci_dev *pdev;
- int ret;
-
- pdev = ipc_pcie->pci;
-
- /* Execute D3 one time. */
- if (pdev->current_state != PCI_D0) {
- dev_dbg(ipc_pcie->dev, "done for PM=%d", pdev->current_state);
- return 0;
- }
-
/* The HAL shall ask the shared memory layer whether D3 is allowed. */
ipc_imem_pm_suspend(ipc_pcie->imem);
- /* Save the PCI configuration space of a device before suspending. */
- ret = pci_save_state(pdev);
-
- if (ret) {
- dev_err(ipc_pcie->dev, "pci_save_state error=%d", ret);
- return ret;
- }
-
- /* Set the power state of a PCI device.
- * Transition a device to a new power state, using the device's PCI PM
- * registers.
- */
- ret = pci_set_power_state(pdev, PCI_D3cold);
-
- if (ret) {
- dev_err(ipc_pcie->dev, "pci_set_power_state error=%d", ret);
- return ret;
- }
-
dev_dbg(ipc_pcie->dev, "SUSPEND done");
- return ret;
+ return 0;
}
int __maybe_unused ipc_pcie_resume(struct iosm_pcie *ipc_pcie)
{
- int ret;
-
- /* Set the power state of a PCI device.
- * Transition a device to a new power state, using the device's PCI PM
- * registers.
- */
- ret = pci_set_power_state(ipc_pcie->pci, PCI_D0);
-
- if (ret) {
- dev_err(ipc_pcie->dev, "pci_set_power_state error=%d", ret);
- return ret;
- }
-
- pci_restore_state(ipc_pcie->pci);
-
/* The HAL shall inform the shared memory layer that the device is
* active.
*/
ipc_imem_pm_resume(ipc_pcie->imem);
dev_dbg(ipc_pcie->dev, "RESUME done");
- return ret;
+ return 0;
}
static int __maybe_unused ipc_pcie_suspend_cb(struct device *dev)
diff --git a/drivers/net/wwan/iosm/iosm_ipc_port.c b/drivers/net/wwan/iosm/iosm_ipc_port.c
index beb9448..b6d81c6 100644
--- a/drivers/net/wwan/iosm/iosm_ipc_port.c
+++ b/drivers/net/wwan/iosm/iosm_ipc_port.c
@@ -27,7 +27,7 @@ static void ipc_port_ctrl_stop(struct wwan_port *port)
{
struct iosm_cdev *ipc_port = wwan_port_get_drvdata(port);
- ipc_imem_sys_cdev_close(ipc_port);
+ ipc_imem_sys_port_close(ipc_port->ipc_imem, ipc_port->channel);
}
/* transfer control data to modem */
diff --git a/drivers/net/wwan/iosm/iosm_ipc_trace.c b/drivers/net/wwan/iosm/iosm_ipc_trace.c
new file mode 100644
index 0000000..eeecfa3
--- /dev/null
+++ b/drivers/net/wwan/iosm/iosm_ipc_trace.c
@@ -0,0 +1,182 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2020-2021 Intel Corporation.
+ */
+
+#include <linux/wwan.h>
+#include "iosm_ipc_trace.h"
+
+/* sub buffer size and number of sub buffer */
+#define IOSM_TRC_SUB_BUFF_SIZE 131072
+#define IOSM_TRC_N_SUB_BUFF 32
+
+#define IOSM_TRC_FILE_PERM 0600
+
+#define IOSM_TRC_DEBUGFS_TRACE "trace"
+#define IOSM_TRC_DEBUGFS_TRACE_CTRL "trace_ctrl"
+
+/**
+ * ipc_trace_port_rx - Receive trace packet from cp and write to relay buffer
+ * @ipc_imem: Pointer to iosm_imem structure
+ * @skb: Pointer to struct sk_buff
+ */
+void ipc_trace_port_rx(struct iosm_imem *ipc_imem, struct sk_buff *skb)
+{
+ struct iosm_trace *ipc_trace = ipc_imem->trace;
+
+ if (ipc_trace->ipc_rchan)
+ relay_write(ipc_trace->ipc_rchan, skb->data, skb->len);
+
+ dev_kfree_skb(skb);
+}
+
+/* Creates relay file in debugfs. */
+static struct dentry *
+ipc_trace_create_buf_file_handler(const char *filename,
+ struct dentry *parent,
+ umode_t mode,
+ struct rchan_buf *buf,
+ int *is_global)
+{
+ *is_global = 1;
+ return debugfs_create_file(filename, mode, parent, buf,
+ &relay_file_operations);
+}
+
+/* Removes relay file from debugfs. */
+static int ipc_trace_remove_buf_file_handler(struct dentry *dentry)
+{
+ debugfs_remove(dentry);
+ return 0;
+}
+
+static int ipc_trace_subbuf_start_handler(struct rchan_buf *buf, void *subbuf,
+ void *prev_subbuf,
+ size_t prev_padding)
+{
+ if (relay_buf_full(buf)) {
+ pr_err_ratelimited("Relay_buf full dropping traces");
+ return 0;
+ }
+
+ return 1;
+}
+
+/* Relay interface callbacks */
+static struct rchan_callbacks relay_callbacks = {
+ .subbuf_start = ipc_trace_subbuf_start_handler,
+ .create_buf_file = ipc_trace_create_buf_file_handler,
+ .remove_buf_file = ipc_trace_remove_buf_file_handler,
+};
+
+/* Copy the trace control mode to user buffer */
+static ssize_t ipc_trace_ctrl_file_read(struct file *filp, char __user *buffer,
+ size_t count, loff_t *ppos)
+{
+ struct iosm_trace *ipc_trace = filp->private_data;
+ char buf[16];
+ int len;
+
+ mutex_lock(&ipc_trace->trc_mutex);
+ len = snprintf(buf, sizeof(buf), "%d\n", ipc_trace->mode);
+ mutex_unlock(&ipc_trace->trc_mutex);
+
+ return simple_read_from_buffer(buffer, count, ppos, buf, len);
+}
+
+/* Open and close the trace channel depending on user input */
+static ssize_t ipc_trace_ctrl_file_write(struct file *filp,
+ const char __user *buffer,
+ size_t count, loff_t *ppos)
+{
+ struct iosm_trace *ipc_trace = filp->private_data;
+ unsigned long val;
+ int ret;
+
+ ret = kstrtoul_from_user(buffer, count, 10, &val);
+ if (ret)
+ return ret;
+
+ mutex_lock(&ipc_trace->trc_mutex);
+ if (val == TRACE_ENABLE && ipc_trace->mode != TRACE_ENABLE) {
+ ipc_trace->channel = ipc_imem_sys_port_open(ipc_trace->ipc_imem,
+ ipc_trace->chl_id,
+ IPC_HP_CDEV_OPEN);
+ if (!ipc_trace->channel) {
+ ret = -EIO;
+ goto unlock;
+ }
+ ipc_trace->mode = TRACE_ENABLE;
+ } else if (val == TRACE_DISABLE && ipc_trace->mode != TRACE_DISABLE) {
+ ipc_trace->mode = TRACE_DISABLE;
+ /* close trace channel */
+ ipc_imem_sys_port_close(ipc_trace->ipc_imem,
+ ipc_trace->channel);
+ relay_flush(ipc_trace->ipc_rchan);
+ }
+ ret = count;
+unlock:
+ mutex_unlock(&ipc_trace->trc_mutex);
+ return ret;
+}
+
+static const struct file_operations ipc_trace_fops = {
+ .open = simple_open,
+ .write = ipc_trace_ctrl_file_write,
+ .read = ipc_trace_ctrl_file_read,
+};
+
+/**
+ * ipc_trace_init - Create trace interface & debugfs entries
+ * @ipc_imem: Pointer to iosm_imem structure
+ *
+ * Returns: Pointer to trace instance on success else NULL
+ */
+struct iosm_trace *ipc_trace_init(struct iosm_imem *ipc_imem)
+{
+ struct ipc_chnl_cfg chnl_cfg = { 0 };
+ struct iosm_trace *ipc_trace;
+
+ ipc_chnl_cfg_get(&chnl_cfg, IPC_MEM_CTRL_CHL_ID_3);
+ ipc_imem_channel_init(ipc_imem, IPC_CTYPE_CTRL, chnl_cfg,
+ IRQ_MOD_OFF);
+
+ ipc_trace = kzalloc(sizeof(*ipc_trace), GFP_KERNEL);
+ if (!ipc_trace)
+ return NULL;
+
+ ipc_trace->mode = TRACE_DISABLE;
+ ipc_trace->dev = ipc_imem->dev;
+ ipc_trace->ipc_imem = ipc_imem;
+ ipc_trace->chl_id = IPC_MEM_CTRL_CHL_ID_3;
+
+ mutex_init(&ipc_trace->trc_mutex);
+
+ ipc_trace->ctrl_file = debugfs_create_file(IOSM_TRC_DEBUGFS_TRACE_CTRL,
+ IOSM_TRC_FILE_PERM,
+ ipc_imem->debugfs_dir,
+ ipc_trace, &ipc_trace_fops);
+
+ ipc_trace->ipc_rchan = relay_open(IOSM_TRC_DEBUGFS_TRACE,
+ ipc_imem->debugfs_dir,
+ IOSM_TRC_SUB_BUFF_SIZE,
+ IOSM_TRC_N_SUB_BUFF,
+ &relay_callbacks, NULL);
+
+ return ipc_trace;
+}
+
+/**
+ * ipc_trace_deinit - Closing relayfs, removing debugfs entries
+ * @ipc_trace: Pointer to the iosm_trace data struct
+ */
+void ipc_trace_deinit(struct iosm_trace *ipc_trace)
+{
+ if (!ipc_trace)
+ return;
+
+ debugfs_remove(ipc_trace->ctrl_file);
+ relay_close(ipc_trace->ipc_rchan);
+ mutex_destroy(&ipc_trace->trc_mutex);
+ kfree(ipc_trace);
+}
diff --git a/drivers/net/wwan/iosm/iosm_ipc_trace.h b/drivers/net/wwan/iosm/iosm_ipc_trace.h
new file mode 100644
index 0000000..5ebe779
--- /dev/null
+++ b/drivers/net/wwan/iosm/iosm_ipc_trace.h
@@ -0,0 +1,74 @@
+/* SPDX-License-Identifier: GPL-2.0-only
+ *
+ * Copyright (C) 2020-2021 Intel Corporation.
+ */
+
+#ifndef IOSM_IPC_TRACE_H
+#define IOSM_IPC_TRACE_H
+
+#include <linux/debugfs.h>
+#include <linux/relay.h>
+
+#include "iosm_ipc_chnl_cfg.h"
+#include "iosm_ipc_imem_ops.h"
+
+/**
+ * enum trace_ctrl_mode - State of trace channel
+ * @TRACE_DISABLE: mode for disable trace
+ * @TRACE_ENABLE: mode for enable trace
+ */
+enum trace_ctrl_mode {
+ TRACE_DISABLE = 0,
+ TRACE_ENABLE,
+};
+
+/**
+ * struct iosm_trace - Struct for trace interface
+ * @ipc_rchan: Pointer to relay channel
+ * @ctrl_file: Pointer to trace control file
+ * @ipc_imem: Imem instance
+ * @dev: Pointer to device struct
+ * @channel: Channel instance
+ * @chl_id: Channel Indentifier
+ * @trc_mutex: Mutex used for read and write mode
+ * @mode: Mode for enable and disable trace
+ */
+
+struct iosm_trace {
+ struct rchan *ipc_rchan;
+ struct dentry *ctrl_file;
+ struct iosm_imem *ipc_imem;
+ struct device *dev;
+ struct ipc_mem_channel *channel;
+ enum ipc_channel_id chl_id;
+ struct mutex trc_mutex; /* Mutex used for read and write mode */
+ enum trace_ctrl_mode mode;
+};
+
+#ifdef CONFIG_WWAN_DEBUGFS
+
+static inline bool ipc_is_trace_channel(struct iosm_imem *ipc_mem, u16 chl_id)
+{
+ return ipc_mem->trace && ipc_mem->trace->chl_id == chl_id;
+}
+
+struct iosm_trace *ipc_trace_init(struct iosm_imem *ipc_imem);
+void ipc_trace_deinit(struct iosm_trace *ipc_trace);
+void ipc_trace_port_rx(struct iosm_imem *ipc_imem, struct sk_buff *skb);
+
+#else
+
+static inline bool ipc_is_trace_channel(struct iosm_imem *ipc_mem, u16 chl_id)
+{
+ return false;
+}
+
+static inline void ipc_trace_port_rx(struct iosm_imem *ipc_imem,
+ struct sk_buff *skb)
+{
+ dev_kfree_skb(skb);
+}
+
+#endif
+
+#endif
diff --git a/drivers/net/wwan/iosm/iosm_ipc_wwan.c b/drivers/net/wwan/iosm/iosm_ipc_wwan.c
index b571d9c..2715114 100644
--- a/drivers/net/wwan/iosm/iosm_ipc_wwan.c
+++ b/drivers/net/wwan/iosm/iosm_ipc_wwan.c
@@ -8,6 +8,7 @@
#include <linux/if_link.h>
#include <linux/rtnetlink.h>
#include <linux/wwan.h>
+#include <net/pkt_sched.h>
#include "iosm_ipc_chnl_cfg.h"
#include "iosm_ipc_imem_ops.h"
@@ -159,7 +160,7 @@ static void ipc_wwan_setup(struct net_device *iosm_dev)
{
iosm_dev->header_ops = NULL;
iosm_dev->hard_header_len = 0;
- iosm_dev->priv_flags |= IFF_NO_QUEUE;
+ iosm_dev->tx_queue_len = DEFAULT_TX_QUEUE_LEN;
iosm_dev->type = ARPHRD_NONE;
iosm_dev->mtu = ETH_DATA_LEN;
diff --git a/drivers/net/wwan/iosm/iosm_ipc_wwan.h b/drivers/net/wwan/iosm/iosm_ipc_wwan.h
index 4925f22..a23e926 100644
--- a/drivers/net/wwan/iosm/iosm_ipc_wwan.h
+++ b/drivers/net/wwan/iosm/iosm_ipc_wwan.h
@@ -42,14 +42,4 @@ int ipc_wwan_receive(struct iosm_wwan *ipc_wwan, struct sk_buff *skb_arg,
*
*/
void ipc_wwan_tx_flowctrl(struct iosm_wwan *ipc_wwan, int id, bool on);
-
-/**
- * ipc_wwan_is_tx_stopped - Checks if Tx stopped for a Interface id.
- * @ipc_wwan: Pointer to wwan instance
- * @id: Ipc mux channel session id
- *
- * Return: true if stopped, false otherwise
- */
-bool ipc_wwan_is_tx_stopped(struct iosm_wwan *ipc_wwan, int id);
-
#endif
diff --git a/drivers/net/wwan/qcom_bam_dmux.c b/drivers/net/wwan/qcom_bam_dmux.c
new file mode 100644
index 0000000..5dfa2eb
--- /dev/null
+++ b/drivers/net/wwan/qcom_bam_dmux.c
@@ -0,0 +1,907 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Qualcomm BAM-DMUX WWAN network driver
+ * Copyright (c) 2020, Stephan Gerhold <stephan@gerhold.net>
+ */
+
+#include <linux/atomic.h>
+#include <linux/bitops.h>
+#include <linux/completion.h>
+#include <linux/dma-mapping.h>
+#include <linux/dmaengine.h>
+#include <linux/if_arp.h>
+#include <linux/interrupt.h>
+#include <linux/mod_devicetable.h>
+#include <linux/module.h>
+#include <linux/netdevice.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/soc/qcom/smem_state.h>
+#include <linux/spinlock.h>
+#include <linux/wait.h>
+#include <linux/workqueue.h>
+#include <net/pkt_sched.h>
+
+#define BAM_DMUX_BUFFER_SIZE SZ_2K
+#define BAM_DMUX_HDR_SIZE sizeof(struct bam_dmux_hdr)
+#define BAM_DMUX_MAX_DATA_SIZE (BAM_DMUX_BUFFER_SIZE - BAM_DMUX_HDR_SIZE)
+#define BAM_DMUX_NUM_SKB 32
+
+#define BAM_DMUX_HDR_MAGIC 0x33fc
+
+#define BAM_DMUX_AUTOSUSPEND_DELAY 1000
+#define BAM_DMUX_REMOTE_TIMEOUT msecs_to_jiffies(2000)
+
+enum {
+ BAM_DMUX_CMD_DATA,
+ BAM_DMUX_CMD_OPEN,
+ BAM_DMUX_CMD_CLOSE,
+};
+
+enum {
+ BAM_DMUX_CH_DATA_0,
+ BAM_DMUX_CH_DATA_1,
+ BAM_DMUX_CH_DATA_2,
+ BAM_DMUX_CH_DATA_3,
+ BAM_DMUX_CH_DATA_4,
+ BAM_DMUX_CH_DATA_5,
+ BAM_DMUX_CH_DATA_6,
+ BAM_DMUX_CH_DATA_7,
+ BAM_DMUX_NUM_CH
+};
+
+struct bam_dmux_hdr {
+ u16 magic;
+ u8 signal;
+ u8 cmd;
+ u8 pad;
+ u8 ch;
+ u16 len;
+};
+
+struct bam_dmux_skb_dma {
+ struct bam_dmux *dmux;
+ struct sk_buff *skb;
+ dma_addr_t addr;
+};
+
+struct bam_dmux {
+ struct device *dev;
+
+ int pc_irq;
+ bool pc_state, pc_ack_state;
+ struct qcom_smem_state *pc, *pc_ack;
+ u32 pc_mask, pc_ack_mask;
+ wait_queue_head_t pc_wait;
+ struct completion pc_ack_completion;
+
+ struct dma_chan *rx, *tx;
+ struct bam_dmux_skb_dma rx_skbs[BAM_DMUX_NUM_SKB];
+ struct bam_dmux_skb_dma tx_skbs[BAM_DMUX_NUM_SKB];
+ spinlock_t tx_lock; /* Protect tx_skbs, tx_next_skb */
+ unsigned int tx_next_skb;
+ atomic_long_t tx_deferred_skb;
+ struct work_struct tx_wakeup_work;
+
+ DECLARE_BITMAP(remote_channels, BAM_DMUX_NUM_CH);
+ struct work_struct register_netdev_work;
+ struct net_device *netdevs[BAM_DMUX_NUM_CH];
+};
+
+struct bam_dmux_netdev {
+ struct bam_dmux *dmux;
+ u8 ch;
+};
+
+static void bam_dmux_pc_vote(struct bam_dmux *dmux, bool enable)
+{
+ reinit_completion(&dmux->pc_ack_completion);
+ qcom_smem_state_update_bits(dmux->pc, dmux->pc_mask,
+ enable ? dmux->pc_mask : 0);
+}
+
+static void bam_dmux_pc_ack(struct bam_dmux *dmux)
+{
+ qcom_smem_state_update_bits(dmux->pc_ack, dmux->pc_ack_mask,
+ dmux->pc_ack_state ? 0 : dmux->pc_ack_mask);
+ dmux->pc_ack_state = !dmux->pc_ack_state;
+}
+
+static bool bam_dmux_skb_dma_map(struct bam_dmux_skb_dma *skb_dma,
+ enum dma_data_direction dir)
+{
+ struct device *dev = skb_dma->dmux->dev;
+
+ skb_dma->addr = dma_map_single(dev, skb_dma->skb->data, skb_dma->skb->len, dir);
+ if (dma_mapping_error(dev, skb_dma->addr)) {
+ dev_err(dev, "Failed to DMA map buffer\n");
+ skb_dma->addr = 0;
+ return false;
+ }
+
+ return true;
+}
+
+static void bam_dmux_skb_dma_unmap(struct bam_dmux_skb_dma *skb_dma,
+ enum dma_data_direction dir)
+{
+ dma_unmap_single(skb_dma->dmux->dev, skb_dma->addr, skb_dma->skb->len, dir);
+ skb_dma->addr = 0;
+}
+
+static void bam_dmux_tx_wake_queues(struct bam_dmux *dmux)
+{
+ int i;
+
+ dev_dbg(dmux->dev, "wake queues\n");
+
+ for (i = 0; i < BAM_DMUX_NUM_CH; ++i) {
+ struct net_device *netdev = dmux->netdevs[i];
+
+ if (netdev && netif_running(netdev))
+ netif_wake_queue(netdev);
+ }
+}
+
+static void bam_dmux_tx_stop_queues(struct bam_dmux *dmux)
+{
+ int i;
+
+ dev_dbg(dmux->dev, "stop queues\n");
+
+ for (i = 0; i < BAM_DMUX_NUM_CH; ++i) {
+ struct net_device *netdev = dmux->netdevs[i];
+
+ if (netdev)
+ netif_stop_queue(netdev);
+ }
+}
+
+static void bam_dmux_tx_done(struct bam_dmux_skb_dma *skb_dma)
+{
+ struct bam_dmux *dmux = skb_dma->dmux;
+ unsigned long flags;
+
+ pm_runtime_mark_last_busy(dmux->dev);
+ pm_runtime_put_autosuspend(dmux->dev);
+
+ if (skb_dma->addr)
+ bam_dmux_skb_dma_unmap(skb_dma, DMA_TO_DEVICE);
+
+ spin_lock_irqsave(&dmux->tx_lock, flags);
+ skb_dma->skb = NULL;
+ if (skb_dma == &dmux->tx_skbs[dmux->tx_next_skb % BAM_DMUX_NUM_SKB])
+ bam_dmux_tx_wake_queues(dmux);
+ spin_unlock_irqrestore(&dmux->tx_lock, flags);
+}
+
+static void bam_dmux_tx_callback(void *data)
+{
+ struct bam_dmux_skb_dma *skb_dma = data;
+ struct sk_buff *skb = skb_dma->skb;
+
+ bam_dmux_tx_done(skb_dma);
+ dev_consume_skb_any(skb);
+}
+
+static bool bam_dmux_skb_dma_submit_tx(struct bam_dmux_skb_dma *skb_dma)
+{
+ struct bam_dmux *dmux = skb_dma->dmux;
+ struct dma_async_tx_descriptor *desc;
+
+ desc = dmaengine_prep_slave_single(dmux->tx, skb_dma->addr,
+ skb_dma->skb->len, DMA_MEM_TO_DEV,
+ DMA_PREP_INTERRUPT);
+ if (!desc) {
+ dev_err(dmux->dev, "Failed to prepare TX DMA buffer\n");
+ return false;
+ }
+
+ desc->callback = bam_dmux_tx_callback;
+ desc->callback_param = skb_dma;
+ desc->cookie = dmaengine_submit(desc);
+ return true;
+}
+
+static struct bam_dmux_skb_dma *
+bam_dmux_tx_queue(struct bam_dmux *dmux, struct sk_buff *skb)
+{
+ struct bam_dmux_skb_dma *skb_dma;
+ unsigned long flags;
+
+ spin_lock_irqsave(&dmux->tx_lock, flags);
+
+ skb_dma = &dmux->tx_skbs[dmux->tx_next_skb % BAM_DMUX_NUM_SKB];
+ if (skb_dma->skb) {
+ bam_dmux_tx_stop_queues(dmux);
+ spin_unlock_irqrestore(&dmux->tx_lock, flags);
+ return NULL;
+ }
+ skb_dma->skb = skb;
+
+ dmux->tx_next_skb++;
+ if (dmux->tx_skbs[dmux->tx_next_skb % BAM_DMUX_NUM_SKB].skb)
+ bam_dmux_tx_stop_queues(dmux);
+
+ spin_unlock_irqrestore(&dmux->tx_lock, flags);
+ return skb_dma;
+}
+
+static int bam_dmux_send_cmd(struct bam_dmux_netdev *bndev, u8 cmd)
+{
+ struct bam_dmux *dmux = bndev->dmux;
+ struct bam_dmux_skb_dma *skb_dma;
+ struct bam_dmux_hdr *hdr;
+ struct sk_buff *skb;
+ int ret;
+
+ skb = alloc_skb(sizeof(*hdr), GFP_KERNEL);
+ if (!skb)
+ return -ENOMEM;
+
+ hdr = skb_put_zero(skb, sizeof(*hdr));
+ hdr->magic = BAM_DMUX_HDR_MAGIC;
+ hdr->cmd = cmd;
+ hdr->ch = bndev->ch;
+
+ skb_dma = bam_dmux_tx_queue(dmux, skb);
+ if (!skb_dma) {
+ ret = -EAGAIN;
+ goto free_skb;
+ }
+
+ ret = pm_runtime_get_sync(dmux->dev);
+ if (ret < 0)
+ goto tx_fail;
+
+ if (!bam_dmux_skb_dma_map(skb_dma, DMA_TO_DEVICE)) {
+ ret = -ENOMEM;
+ goto tx_fail;
+ }
+
+ if (!bam_dmux_skb_dma_submit_tx(skb_dma)) {
+ ret = -EIO;
+ goto tx_fail;
+ }
+
+ dma_async_issue_pending(dmux->tx);
+ return 0;
+
+tx_fail:
+ bam_dmux_tx_done(skb_dma);
+free_skb:
+ dev_kfree_skb(skb);
+ return ret;
+}
+
+static int bam_dmux_netdev_open(struct net_device *netdev)
+{
+ struct bam_dmux_netdev *bndev = netdev_priv(netdev);
+ int ret;
+
+ ret = bam_dmux_send_cmd(bndev, BAM_DMUX_CMD_OPEN);
+ if (ret)
+ return ret;
+
+ netif_start_queue(netdev);
+ return 0;
+}
+
+static int bam_dmux_netdev_stop(struct net_device *netdev)
+{
+ struct bam_dmux_netdev *bndev = netdev_priv(netdev);
+
+ netif_stop_queue(netdev);
+ bam_dmux_send_cmd(bndev, BAM_DMUX_CMD_CLOSE);
+ return 0;
+}
+
+static unsigned int needed_room(unsigned int avail, unsigned int needed)
+{
+ if (avail >= needed)
+ return 0;
+ return needed - avail;
+}
+
+static int bam_dmux_tx_prepare_skb(struct bam_dmux_netdev *bndev,
+ struct sk_buff *skb)
+{
+ unsigned int head = needed_room(skb_headroom(skb), BAM_DMUX_HDR_SIZE);
+ unsigned int pad = sizeof(u32) - skb->len % sizeof(u32);
+ unsigned int tail = needed_room(skb_tailroom(skb), pad);
+ struct bam_dmux_hdr *hdr;
+ int ret;
+
+ if (head || tail || skb_cloned(skb)) {
+ ret = pskb_expand_head(skb, head, tail, GFP_ATOMIC);
+ if (ret)
+ return ret;
+ }
+
+ hdr = skb_push(skb, sizeof(*hdr));
+ hdr->magic = BAM_DMUX_HDR_MAGIC;
+ hdr->signal = 0;
+ hdr->cmd = BAM_DMUX_CMD_DATA;
+ hdr->pad = pad;
+ hdr->ch = bndev->ch;
+ hdr->len = skb->len - sizeof(*hdr);
+ if (pad)
+ skb_put_zero(skb, pad);
+
+ return 0;
+}
+
+static netdev_tx_t bam_dmux_netdev_start_xmit(struct sk_buff *skb,
+ struct net_device *netdev)
+{
+ struct bam_dmux_netdev *bndev = netdev_priv(netdev);
+ struct bam_dmux *dmux = bndev->dmux;
+ struct bam_dmux_skb_dma *skb_dma;
+ int active, ret;
+
+ skb_dma = bam_dmux_tx_queue(dmux, skb);
+ if (!skb_dma)
+ return NETDEV_TX_BUSY;
+
+ active = pm_runtime_get(dmux->dev);
+ if (active < 0 && active != -EINPROGRESS)
+ goto drop;
+
+ ret = bam_dmux_tx_prepare_skb(bndev, skb);
+ if (ret)
+ goto drop;
+
+ if (!bam_dmux_skb_dma_map(skb_dma, DMA_TO_DEVICE))
+ goto drop;
+
+ if (active <= 0) {
+ /* Cannot sleep here so mark skb for wakeup handler and return */
+ if (!atomic_long_fetch_or(BIT(skb_dma - dmux->tx_skbs),
+ &dmux->tx_deferred_skb))
+ queue_pm_work(&dmux->tx_wakeup_work);
+ return NETDEV_TX_OK;
+ }
+
+ if (!bam_dmux_skb_dma_submit_tx(skb_dma))
+ goto drop;
+
+ dma_async_issue_pending(dmux->tx);
+ return NETDEV_TX_OK;
+
+drop:
+ bam_dmux_tx_done(skb_dma);
+ dev_kfree_skb_any(skb);
+ return NETDEV_TX_OK;
+}
+
+static void bam_dmux_tx_wakeup_work(struct work_struct *work)
+{
+ struct bam_dmux *dmux = container_of(work, struct bam_dmux, tx_wakeup_work);
+ unsigned long pending;
+ int ret, i;
+
+ ret = pm_runtime_resume_and_get(dmux->dev);
+ if (ret < 0) {
+ dev_err(dmux->dev, "Failed to resume: %d\n", ret);
+ return;
+ }
+
+ pending = atomic_long_xchg(&dmux->tx_deferred_skb, 0);
+ if (!pending)
+ goto out;
+
+ dev_dbg(dmux->dev, "pending skbs after wakeup: %#lx\n", pending);
+ for_each_set_bit(i, &pending, BAM_DMUX_NUM_SKB) {
+ bam_dmux_skb_dma_submit_tx(&dmux->tx_skbs[i]);
+ }
+ dma_async_issue_pending(dmux->tx);
+
+out:
+ pm_runtime_mark_last_busy(dmux->dev);
+ pm_runtime_put_autosuspend(dmux->dev);
+}
+
+static const struct net_device_ops bam_dmux_ops = {
+ .ndo_open = bam_dmux_netdev_open,
+ .ndo_stop = bam_dmux_netdev_stop,
+ .ndo_start_xmit = bam_dmux_netdev_start_xmit,
+};
+
+static const struct device_type wwan_type = {
+ .name = "wwan",
+};
+
+static void bam_dmux_netdev_setup(struct net_device *dev)
+{
+ dev->netdev_ops = &bam_dmux_ops;
+
+ dev->type = ARPHRD_RAWIP;
+ SET_NETDEV_DEVTYPE(dev, &wwan_type);
+ dev->flags = IFF_POINTOPOINT | IFF_NOARP;
+
+ dev->mtu = ETH_DATA_LEN;
+ dev->max_mtu = BAM_DMUX_MAX_DATA_SIZE;
+ dev->needed_headroom = sizeof(struct bam_dmux_hdr);
+ dev->needed_tailroom = sizeof(u32); /* word-aligned */
+ dev->tx_queue_len = DEFAULT_TX_QUEUE_LEN;
+
+ /* This perm addr will be used as interface identifier by IPv6 */
+ dev->addr_assign_type = NET_ADDR_RANDOM;
+ eth_random_addr(dev->perm_addr);
+}
+
+static void bam_dmux_register_netdev_work(struct work_struct *work)
+{
+ struct bam_dmux *dmux = container_of(work, struct bam_dmux, register_netdev_work);
+ struct bam_dmux_netdev *bndev;
+ struct net_device *netdev;
+ int ch, ret;
+
+ for_each_set_bit(ch, dmux->remote_channels, BAM_DMUX_NUM_CH) {
+ if (dmux->netdevs[ch])
+ continue;
+
+ netdev = alloc_netdev(sizeof(*bndev), "wwan%d", NET_NAME_ENUM,
+ bam_dmux_netdev_setup);
+ if (!netdev)
+ return;
+
+ SET_NETDEV_DEV(netdev, dmux->dev);
+ netdev->dev_port = ch;
+
+ bndev = netdev_priv(netdev);
+ bndev->dmux = dmux;
+ bndev->ch = ch;
+
+ ret = register_netdev(netdev);
+ if (ret) {
+ dev_err(dmux->dev, "Failed to register netdev for channel %u: %d\n",
+ ch, ret);
+ free_netdev(netdev);
+ return;
+ }
+
+ dmux->netdevs[ch] = netdev;
+ }
+}
+
+static void bam_dmux_rx_callback(void *data);
+
+static bool bam_dmux_skb_dma_submit_rx(struct bam_dmux_skb_dma *skb_dma)
+{
+ struct bam_dmux *dmux = skb_dma->dmux;
+ struct dma_async_tx_descriptor *desc;
+
+ desc = dmaengine_prep_slave_single(dmux->rx, skb_dma->addr,
+ skb_dma->skb->len, DMA_DEV_TO_MEM,
+ DMA_PREP_INTERRUPT);
+ if (!desc) {
+ dev_err(dmux->dev, "Failed to prepare RX DMA buffer\n");
+ return false;
+ }
+
+ desc->callback = bam_dmux_rx_callback;
+ desc->callback_param = skb_dma;
+ desc->cookie = dmaengine_submit(desc);
+ return true;
+}
+
+static bool bam_dmux_skb_dma_queue_rx(struct bam_dmux_skb_dma *skb_dma, gfp_t gfp)
+{
+ if (!skb_dma->skb) {
+ skb_dma->skb = __netdev_alloc_skb(NULL, BAM_DMUX_BUFFER_SIZE, gfp);
+ if (!skb_dma->skb)
+ return false;
+ skb_put(skb_dma->skb, BAM_DMUX_BUFFER_SIZE);
+ }
+
+ return bam_dmux_skb_dma_map(skb_dma, DMA_FROM_DEVICE) &&
+ bam_dmux_skb_dma_submit_rx(skb_dma);
+}
+
+static void bam_dmux_cmd_data(struct bam_dmux_skb_dma *skb_dma)
+{
+ struct bam_dmux *dmux = skb_dma->dmux;
+ struct sk_buff *skb = skb_dma->skb;
+ struct bam_dmux_hdr *hdr = (struct bam_dmux_hdr *)skb->data;
+ struct net_device *netdev = dmux->netdevs[hdr->ch];
+
+ if (!netdev || !netif_running(netdev)) {
+ dev_warn(dmux->dev, "Data for inactive channel %u\n", hdr->ch);
+ return;
+ }
+
+ if (hdr->len > BAM_DMUX_MAX_DATA_SIZE) {
+ dev_err(dmux->dev, "Data larger than buffer? (%u > %u)\n",
+ hdr->len, (u16)BAM_DMUX_MAX_DATA_SIZE);
+ return;
+ }
+
+ skb_dma->skb = NULL; /* Hand over to network stack */
+
+ skb_pull(skb, sizeof(*hdr));
+ skb_trim(skb, hdr->len);
+ skb->dev = netdev;
+
+ /* Only Raw-IP/QMAP is supported by this driver */
+ switch (skb->data[0] & 0xf0) {
+ case 0x40:
+ skb->protocol = htons(ETH_P_IP);
+ break;
+ case 0x60:
+ skb->protocol = htons(ETH_P_IPV6);
+ break;
+ default:
+ skb->protocol = htons(ETH_P_MAP);
+ break;
+ }
+
+ netif_receive_skb(skb);
+}
+
+static void bam_dmux_cmd_open(struct bam_dmux *dmux, struct bam_dmux_hdr *hdr)
+{
+ struct net_device *netdev = dmux->netdevs[hdr->ch];
+
+ dev_dbg(dmux->dev, "open channel: %u\n", hdr->ch);
+
+ if (__test_and_set_bit(hdr->ch, dmux->remote_channels)) {
+ dev_warn(dmux->dev, "Channel already open: %u\n", hdr->ch);
+ return;
+ }
+
+ if (netdev) {
+ netif_device_attach(netdev);
+ } else {
+ /* Cannot sleep here, schedule work to register the netdev */
+ schedule_work(&dmux->register_netdev_work);
+ }
+}
+
+static void bam_dmux_cmd_close(struct bam_dmux *dmux, struct bam_dmux_hdr *hdr)
+{
+ struct net_device *netdev = dmux->netdevs[hdr->ch];
+
+ dev_dbg(dmux->dev, "close channel: %u\n", hdr->ch);
+
+ if (!__test_and_clear_bit(hdr->ch, dmux->remote_channels)) {
+ dev_err(dmux->dev, "Channel not open: %u\n", hdr->ch);
+ return;
+ }
+
+ if (netdev)
+ netif_device_detach(netdev);
+}
+
+static void bam_dmux_rx_callback(void *data)
+{
+ struct bam_dmux_skb_dma *skb_dma = data;
+ struct bam_dmux *dmux = skb_dma->dmux;
+ struct sk_buff *skb = skb_dma->skb;
+ struct bam_dmux_hdr *hdr = (struct bam_dmux_hdr *)skb->data;
+
+ bam_dmux_skb_dma_unmap(skb_dma, DMA_FROM_DEVICE);
+
+ if (hdr->magic != BAM_DMUX_HDR_MAGIC) {
+ dev_err(dmux->dev, "Invalid magic in header: %#x\n", hdr->magic);
+ goto out;
+ }
+
+ if (hdr->ch >= BAM_DMUX_NUM_CH) {
+ dev_dbg(dmux->dev, "Unsupported channel: %u\n", hdr->ch);
+ goto out;
+ }
+
+ switch (hdr->cmd) {
+ case BAM_DMUX_CMD_DATA:
+ bam_dmux_cmd_data(skb_dma);
+ break;
+ case BAM_DMUX_CMD_OPEN:
+ bam_dmux_cmd_open(dmux, hdr);
+ break;
+ case BAM_DMUX_CMD_CLOSE:
+ bam_dmux_cmd_close(dmux, hdr);
+ break;
+ default:
+ dev_err(dmux->dev, "Unsupported command %u on channel %u\n",
+ hdr->cmd, hdr->ch);
+ break;
+ }
+
+out:
+ if (bam_dmux_skb_dma_queue_rx(skb_dma, GFP_ATOMIC))
+ dma_async_issue_pending(dmux->rx);
+}
+
+static bool bam_dmux_power_on(struct bam_dmux *dmux)
+{
+ struct device *dev = dmux->dev;
+ struct dma_slave_config dma_rx_conf = {
+ .direction = DMA_DEV_TO_MEM,
+ .src_maxburst = BAM_DMUX_BUFFER_SIZE,
+ };
+ int i;
+
+ dmux->rx = dma_request_chan(dev, "rx");
+ if (IS_ERR(dmux->rx)) {
+ dev_err(dev, "Failed to request RX DMA channel: %pe\n", dmux->rx);
+ dmux->rx = NULL;
+ return false;
+ }
+ dmaengine_slave_config(dmux->rx, &dma_rx_conf);
+
+ for (i = 0; i < BAM_DMUX_NUM_SKB; i++) {
+ if (!bam_dmux_skb_dma_queue_rx(&dmux->rx_skbs[i], GFP_KERNEL))
+ return false;
+ }
+ dma_async_issue_pending(dmux->rx);
+
+ return true;
+}
+
+static void bam_dmux_free_skbs(struct bam_dmux_skb_dma skbs[],
+ enum dma_data_direction dir)
+{
+ int i;
+
+ for (i = 0; i < BAM_DMUX_NUM_SKB; i++) {
+ struct bam_dmux_skb_dma *skb_dma = &skbs[i];
+
+ if (skb_dma->addr)
+ bam_dmux_skb_dma_unmap(skb_dma, dir);
+ if (skb_dma->skb) {
+ dev_kfree_skb(skb_dma->skb);
+ skb_dma->skb = NULL;
+ }
+ }
+}
+
+static void bam_dmux_power_off(struct bam_dmux *dmux)
+{
+ if (dmux->tx) {
+ dmaengine_terminate_sync(dmux->tx);
+ dma_release_channel(dmux->tx);
+ dmux->tx = NULL;
+ }
+
+ if (dmux->rx) {
+ dmaengine_terminate_sync(dmux->rx);
+ dma_release_channel(dmux->rx);
+ dmux->rx = NULL;
+ }
+
+ bam_dmux_free_skbs(dmux->rx_skbs, DMA_FROM_DEVICE);
+}
+
+static irqreturn_t bam_dmux_pc_irq(int irq, void *data)
+{
+ struct bam_dmux *dmux = data;
+ bool new_state = !dmux->pc_state;
+
+ dev_dbg(dmux->dev, "pc: %u\n", new_state);
+
+ if (new_state) {
+ if (bam_dmux_power_on(dmux))
+ bam_dmux_pc_ack(dmux);
+ else
+ bam_dmux_power_off(dmux);
+ } else {
+ bam_dmux_power_off(dmux);
+ bam_dmux_pc_ack(dmux);
+ }
+
+ dmux->pc_state = new_state;
+ wake_up_all(&dmux->pc_wait);
+
+ return IRQ_HANDLED;
+}
+
+static irqreturn_t bam_dmux_pc_ack_irq(int irq, void *data)
+{
+ struct bam_dmux *dmux = data;
+
+ dev_dbg(dmux->dev, "pc ack\n");
+ complete_all(&dmux->pc_ack_completion);
+
+ return IRQ_HANDLED;
+}
+
+static int bam_dmux_runtime_suspend(struct device *dev)
+{
+ struct bam_dmux *dmux = dev_get_drvdata(dev);
+
+ dev_dbg(dev, "runtime suspend\n");
+ bam_dmux_pc_vote(dmux, false);
+
+ return 0;
+}
+
+static int __maybe_unused bam_dmux_runtime_resume(struct device *dev)
+{
+ struct bam_dmux *dmux = dev_get_drvdata(dev);
+
+ dev_dbg(dev, "runtime resume\n");
+
+ /* Wait until previous power down was acked */
+ if (!wait_for_completion_timeout(&dmux->pc_ack_completion,
+ BAM_DMUX_REMOTE_TIMEOUT))
+ return -ETIMEDOUT;
+
+ /* Vote for power state */
+ bam_dmux_pc_vote(dmux, true);
+
+ /* Wait for ack */
+ if (!wait_for_completion_timeout(&dmux->pc_ack_completion,
+ BAM_DMUX_REMOTE_TIMEOUT)) {
+ bam_dmux_pc_vote(dmux, false);
+ return -ETIMEDOUT;
+ }
+
+ /* Wait until we're up */
+ if (!wait_event_timeout(dmux->pc_wait, dmux->pc_state,
+ BAM_DMUX_REMOTE_TIMEOUT)) {
+ bam_dmux_pc_vote(dmux, false);
+ return -ETIMEDOUT;
+ }
+
+ /* Ensure that we actually initialized successfully */
+ if (!dmux->rx) {
+ bam_dmux_pc_vote(dmux, false);
+ return -ENXIO;
+ }
+
+ /* Request TX channel if necessary */
+ if (dmux->tx)
+ return 0;
+
+ dmux->tx = dma_request_chan(dev, "tx");
+ if (IS_ERR(dmux->rx)) {
+ dev_err(dev, "Failed to request TX DMA channel: %pe\n", dmux->tx);
+ dmux->tx = NULL;
+ bam_dmux_runtime_suspend(dev);
+ return -ENXIO;
+ }
+
+ return 0;
+}
+
+static int bam_dmux_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct bam_dmux *dmux;
+ int ret, pc_ack_irq, i;
+ unsigned int bit;
+
+ dmux = devm_kzalloc(dev, sizeof(*dmux), GFP_KERNEL);
+ if (!dmux)
+ return -ENOMEM;
+
+ dmux->dev = dev;
+ platform_set_drvdata(pdev, dmux);
+
+ dmux->pc_irq = platform_get_irq_byname(pdev, "pc");
+ if (dmux->pc_irq < 0)
+ return dmux->pc_irq;
+
+ pc_ack_irq = platform_get_irq_byname(pdev, "pc-ack");
+ if (pc_ack_irq < 0)
+ return pc_ack_irq;
+
+ dmux->pc = devm_qcom_smem_state_get(dev, "pc", &bit);
+ if (IS_ERR(dmux->pc))
+ return dev_err_probe(dev, PTR_ERR(dmux->pc),
+ "Failed to get pc state\n");
+ dmux->pc_mask = BIT(bit);
+
+ dmux->pc_ack = devm_qcom_smem_state_get(dev, "pc-ack", &bit);
+ if (IS_ERR(dmux->pc_ack))
+ return dev_err_probe(dev, PTR_ERR(dmux->pc_ack),
+ "Failed to get pc-ack state\n");
+ dmux->pc_ack_mask = BIT(bit);
+
+ init_waitqueue_head(&dmux->pc_wait);
+ init_completion(&dmux->pc_ack_completion);
+ complete_all(&dmux->pc_ack_completion);
+
+ spin_lock_init(&dmux->tx_lock);
+ INIT_WORK(&dmux->tx_wakeup_work, bam_dmux_tx_wakeup_work);
+ INIT_WORK(&dmux->register_netdev_work, bam_dmux_register_netdev_work);
+
+ for (i = 0; i < BAM_DMUX_NUM_SKB; i++) {
+ dmux->rx_skbs[i].dmux = dmux;
+ dmux->tx_skbs[i].dmux = dmux;
+ }
+
+ /* Runtime PM manages our own power vote.
+ * Note that the RX path may be active even if we are runtime suspended,
+ * since it is controlled by the remote side.
+ */
+ pm_runtime_set_autosuspend_delay(dev, BAM_DMUX_AUTOSUSPEND_DELAY);
+ pm_runtime_use_autosuspend(dev);
+ pm_runtime_enable(dev);
+
+ ret = devm_request_threaded_irq(dev, pc_ack_irq, NULL, bam_dmux_pc_ack_irq,
+ IRQF_ONESHOT, NULL, dmux);
+ if (ret)
+ return ret;
+
+ ret = devm_request_threaded_irq(dev, dmux->pc_irq, NULL, bam_dmux_pc_irq,
+ IRQF_ONESHOT, NULL, dmux);
+ if (ret)
+ return ret;
+
+ ret = irq_get_irqchip_state(dmux->pc_irq, IRQCHIP_STATE_LINE_LEVEL,
+ &dmux->pc_state);
+ if (ret)
+ return ret;
+
+ /* Check if remote finished initialization before us */
+ if (dmux->pc_state) {
+ if (bam_dmux_power_on(dmux))
+ bam_dmux_pc_ack(dmux);
+ else
+ bam_dmux_power_off(dmux);
+ }
+
+ return 0;
+}
+
+static int bam_dmux_remove(struct platform_device *pdev)
+{
+ struct bam_dmux *dmux = platform_get_drvdata(pdev);
+ struct device *dev = dmux->dev;
+ LIST_HEAD(list);
+ int i;
+
+ /* Unregister network interfaces */
+ cancel_work_sync(&dmux->register_netdev_work);
+ rtnl_lock();
+ for (i = 0; i < BAM_DMUX_NUM_CH; ++i)
+ if (dmux->netdevs[i])
+ unregister_netdevice_queue(dmux->netdevs[i], &list);
+ unregister_netdevice_many(&list);
+ rtnl_unlock();
+ cancel_work_sync(&dmux->tx_wakeup_work);
+
+ /* Drop our own power vote */
+ pm_runtime_disable(dev);
+ pm_runtime_dont_use_autosuspend(dev);
+ bam_dmux_runtime_suspend(dev);
+ pm_runtime_set_suspended(dev);
+
+ /* Try to wait for remote side to drop power vote */
+ if (!wait_event_timeout(dmux->pc_wait, !dmux->rx, BAM_DMUX_REMOTE_TIMEOUT))
+ dev_err(dev, "Timed out waiting for remote side to suspend\n");
+
+ /* Make sure everything is cleaned up before we return */
+ disable_irq(dmux->pc_irq);
+ bam_dmux_power_off(dmux);
+ bam_dmux_free_skbs(dmux->tx_skbs, DMA_TO_DEVICE);
+
+ return 0;
+}
+
+static const struct dev_pm_ops bam_dmux_pm_ops = {
+ SET_RUNTIME_PM_OPS(bam_dmux_runtime_suspend, bam_dmux_runtime_resume, NULL)
+};
+
+static const struct of_device_id bam_dmux_of_match[] = {
+ { .compatible = "qcom,bam-dmux" },
+ { /* sentinel */ }
+};
+MODULE_DEVICE_TABLE(of, bam_dmux_of_match);
+
+static struct platform_driver bam_dmux_driver = {
+ .probe = bam_dmux_probe,
+ .remove = bam_dmux_remove,
+ .driver = {
+ .name = "bam-dmux",
+ .pm = &bam_dmux_pm_ops,
+ .of_match_table = bam_dmux_of_match,
+ },
+};
+module_platform_driver(bam_dmux_driver);
+
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("Qualcomm BAM-DMUX WWAN Network Driver");
+MODULE_AUTHOR("Stephan Gerhold <stephan@gerhold.net>");
diff --git a/drivers/net/wwan/wwan_core.c b/drivers/net/wwan/wwan_core.c
index d293ab6..1508dc2 100644
--- a/drivers/net/wwan/wwan_core.c
+++ b/drivers/net/wwan/wwan_core.c
@@ -3,6 +3,7 @@
#include <linux/err.h>
#include <linux/errno.h>
+#include <linux/debugfs.h>
#include <linux/fs.h>
#include <linux/init.h>
#include <linux/idr.h>
@@ -25,6 +26,7 @@ static DEFINE_IDA(minors); /* minors for WWAN port chardevs */
static DEFINE_IDA(wwan_dev_ids); /* for unique WWAN device IDs */
static struct class *wwan_class;
static int wwan_major;
+static struct dentry *wwan_debugfs_dir;
#define to_wwan_dev(d) container_of(d, struct wwan_device, dev)
#define to_wwan_port(d) container_of(d, struct wwan_port, dev)
@@ -40,6 +42,7 @@ static int wwan_major;
* @port_id: Current available port ID to pick.
* @ops: wwan device ops
* @ops_ctxt: context to pass to ops
+ * @debugfs_dir: WWAN device debugfs dir
*/
struct wwan_device {
unsigned int id;
@@ -47,6 +50,9 @@ struct wwan_device {
atomic_t port_id;
const struct wwan_ops *ops;
void *ops_ctxt;
+#ifdef CONFIG_WWAN_DEBUGFS
+ struct dentry *debugfs_dir;
+#endif
};
/**
@@ -142,6 +148,20 @@ static struct wwan_device *wwan_dev_get_by_name(const char *name)
return to_wwan_dev(dev);
}
+#ifdef CONFIG_WWAN_DEBUGFS
+struct dentry *wwan_get_debugfs_dir(struct device *parent)
+{
+ struct wwan_device *wwandev;
+
+ wwandev = wwan_dev_get_by_parent(parent);
+ if (IS_ERR(wwandev))
+ return ERR_CAST(wwandev);
+
+ return wwandev->debugfs_dir;
+}
+EXPORT_SYMBOL_GPL(wwan_get_debugfs_dir);
+#endif
+
/* This function allocates and registers a new WWAN device OR if a WWAN device
* already exist for the given parent, it gets a reference and return it.
* This function is not exported (for now), it is called indirectly via
@@ -189,6 +209,12 @@ static struct wwan_device *wwan_create_dev(struct device *parent)
goto done_unlock;
}
+#ifdef CONFIG_WWAN_DEBUGFS
+ wwandev->debugfs_dir =
+ debugfs_create_dir(kobject_name(&wwandev->dev.kobj),
+ wwan_debugfs_dir);
+#endif
+
done_unlock:
mutex_unlock(&wwan_register_lock);
@@ -218,10 +244,14 @@ static void wwan_remove_dev(struct wwan_device *wwandev)
else
ret = device_for_each_child(&wwandev->dev, NULL, is_wwan_child);
- if (!ret)
+ if (!ret) {
+#ifdef CONFIG_WWAN_DEBUGFS
+ debugfs_remove_recursive(wwandev->debugfs_dir);
+#endif
device_unregister(&wwandev->dev);
- else
+ } else {
put_device(&wwandev->dev);
+ }
mutex_unlock(&wwan_register_lock);
}
@@ -1117,6 +1147,10 @@ static int __init wwan_init(void)
goto destroy;
}
+#ifdef CONFIG_WWAN_DEBUGFS
+ wwan_debugfs_dir = debugfs_create_dir("wwan", NULL);
+#endif
+
return 0;
destroy:
@@ -1128,6 +1162,7 @@ static int __init wwan_init(void)
static void __exit wwan_exit(void)
{
+ debugfs_remove_recursive(wwan_debugfs_dir);
__unregister_chrdev(wwan_major, 0, WWAN_MAX_MINORS, "wwan_port");
rtnl_link_unregister(&wwan_rtnl_link_ops);
class_destroy(wwan_class);
diff --git a/drivers/net/xen-netfront.c b/drivers/net/xen-netfront.c
index d514d96..8b18246 100644
--- a/drivers/net/xen-netfront.c
+++ b/drivers/net/xen-netfront.c
@@ -948,7 +948,7 @@ static u32 xennet_run_xdp(struct netfront_queue *queue, struct page *pdata,
break;
default:
- bpf_warn_invalid_xdp_action(act);
+ bpf_warn_invalid_xdp_action(queue->info->netdev, prog, act);
}
return act;
diff --git a/drivers/nfc/fdp/i2c.c b/drivers/nfc/fdp/i2c.c
index f78670b..28a9e1e 100644
--- a/drivers/nfc/fdp/i2c.c
+++ b/drivers/nfc/fdp/i2c.c
@@ -205,9 +205,7 @@ static irqreturn_t fdp_nci_i2c_irq_thread_fn(int irq, void *phy_id)
r = fdp_nci_i2c_read(phy, &skb);
- if (r == -EREMOTEIO)
- return IRQ_HANDLED;
- else if (r == -ENOMEM || r == -EBADMSG)
+ if (r == -EREMOTEIO || r == -ENOMEM || r == -EBADMSG)
return IRQ_HANDLED;
if (skb != NULL)
diff --git a/drivers/nfc/st21nfca/i2c.c b/drivers/nfc/st21nfca/i2c.c
index f126ce9..a86b5ed 100644
--- a/drivers/nfc/st21nfca/i2c.c
+++ b/drivers/nfc/st21nfca/i2c.c
@@ -315,10 +315,8 @@ static int st21nfca_hci_i2c_repack(struct sk_buff *skb)
skb_pull(skb, 1);
r = check_crc(skb->data, skb->len);
- if (r != 0) {
- i = 0;
+ if (r != 0)
return -EBADMSG;
- }
/* remove headbyte */
skb_pull(skb, 1);
@@ -524,7 +522,8 @@ static int st21nfca_hci_i2c_probe(struct i2c_client *client,
phy->gpiod_ena = devm_gpiod_get(dev, "enable", GPIOD_OUT_LOW);
if (IS_ERR(phy->gpiod_ena)) {
nfc_err(dev, "Unable to get ENABLE GPIO\n");
- return PTR_ERR(phy->gpiod_ena);
+ r = PTR_ERR(phy->gpiod_ena);
+ goto out_free;
}
phy->se_status.is_ese_present =
@@ -535,7 +534,7 @@ static int st21nfca_hci_i2c_probe(struct i2c_client *client,
r = st21nfca_hci_platform_init(phy);
if (r < 0) {
nfc_err(&client->dev, "Unable to reboot st21nfca\n");
- return r;
+ goto out_free;
}
r = devm_request_threaded_irq(&client->dev, client->irq, NULL,
@@ -544,15 +543,23 @@ static int st21nfca_hci_i2c_probe(struct i2c_client *client,
ST21NFCA_HCI_DRIVER_NAME, phy);
if (r < 0) {
nfc_err(&client->dev, "Unable to register IRQ handler\n");
- return r;
+ goto out_free;
}
- return st21nfca_hci_probe(phy, &i2c_phy_ops, LLC_SHDLC_NAME,
- ST21NFCA_FRAME_HEADROOM,
- ST21NFCA_FRAME_TAILROOM,
- ST21NFCA_HCI_LLC_MAX_PAYLOAD,
- &phy->hdev,
- &phy->se_status);
+ r = st21nfca_hci_probe(phy, &i2c_phy_ops, LLC_SHDLC_NAME,
+ ST21NFCA_FRAME_HEADROOM,
+ ST21NFCA_FRAME_TAILROOM,
+ ST21NFCA_HCI_LLC_MAX_PAYLOAD,
+ &phy->hdev,
+ &phy->se_status);
+ if (r)
+ goto out_free;
+
+ return 0;
+
+out_free:
+ kfree_skb(phy->pending_skb);
+ return r;
}
static int st21nfca_hci_i2c_remove(struct i2c_client *client)
@@ -563,6 +570,8 @@ static int st21nfca_hci_i2c_remove(struct i2c_client *client)
if (phy->powered)
st21nfca_hci_i2c_disable(phy);
+ if (phy->pending_skb)
+ kfree_skb(phy->pending_skb);
return 0;
}
diff --git a/drivers/of/platform.c b/drivers/of/platform.c
index b3faf89..7933500 100644
--- a/drivers/of/platform.c
+++ b/drivers/of/platform.c
@@ -540,6 +540,10 @@ static int __init of_platform_default_populate_init(void)
of_node_put(node);
}
+ node = of_get_compatible_child(of_chosen, "simple-framebuffer");
+ of_platform_device_create(node, NULL, NULL);
+ of_node_put(node);
+
/* Populate everything else. */
of_platform_default_populate(NULL, NULL, NULL);
diff --git a/drivers/pcmcia/pcmcia_cis.c b/drivers/pcmcia/pcmcia_cis.c
index f650e19..6bc0bc2 100644
--- a/drivers/pcmcia/pcmcia_cis.c
+++ b/drivers/pcmcia/pcmcia_cis.c
@@ -386,7 +386,7 @@ size_t pcmcia_get_tuple(struct pcmcia_device *p_dev, cisdata_t code,
}
EXPORT_SYMBOL(pcmcia_get_tuple);
-
+#ifdef CONFIG_NET
/*
* pcmcia_do_get_mac() - internal helper for pcmcia_get_mac_from_cis()
*
@@ -431,3 +431,4 @@ int pcmcia_get_mac_from_cis(struct pcmcia_device *p_dev, struct net_device *dev)
}
EXPORT_SYMBOL(pcmcia_get_mac_from_cis);
+#endif /* CONFIG_NET */
diff --git a/drivers/perf/Kconfig b/drivers/perf/Kconfig
index 4374af2..e1a0c44 100644
--- a/drivers/perf/Kconfig
+++ b/drivers/perf/Kconfig
@@ -43,7 +43,7 @@
config ARM_CMN
tristate "Arm CMN-600 PMU support"
- depends on ARM64 || (COMPILE_TEST && 64BIT)
+ depends on ARM64 || COMPILE_TEST
help
Support for PMU events monitoring on the Arm CMN-600 Coherent Mesh
Network interconnect.
@@ -139,6 +139,13 @@
Support for PMU events monitoring on the ARM DMC-620 memory
controller.
+config MARVELL_CN10K_TAD_PMU
+ tristate "Marvell CN10K LLC-TAD PMU"
+ depends on ARM64 || (COMPILE_TEST && 64BIT)
+ help
+ Provides support for Last-Level cache Tag-and-data Units (LLC-TAD)
+ performance monitors on CN10K family silicons.
+
source "drivers/perf/hisilicon/Kconfig"
endmenu
diff --git a/drivers/perf/Makefile b/drivers/perf/Makefile
index 5260b11..2db5418 100644
--- a/drivers/perf/Makefile
+++ b/drivers/perf/Makefile
@@ -14,3 +14,4 @@
obj-$(CONFIG_XGENE_PMU) += xgene_pmu.o
obj-$(CONFIG_ARM_SPE_PMU) += arm_spe_pmu.o
obj-$(CONFIG_ARM_DMC620_PMU) += arm_dmc620_pmu.o
+obj-$(CONFIG_MARVELL_CN10K_TAD_PMU) += marvell_cn10k_tad_pmu.o
diff --git a/drivers/perf/arm-cmn.c b/drivers/perf/arm-cmn.c
index bc3cba5..0e48adc 100644
--- a/drivers/perf/arm-cmn.c
+++ b/drivers/perf/arm-cmn.c
@@ -5,8 +5,10 @@
#include <linux/acpi.h>
#include <linux/bitfield.h>
#include <linux/bitops.h>
+#include <linux/debugfs.h>
#include <linux/interrupt.h>
#include <linux/io.h>
+#include <linux/io-64-nonatomic-lo-hi.h>
#include <linux/kernel.h>
#include <linux/list.h>
#include <linux/module.h>
@@ -23,7 +25,10 @@
#define CMN_NI_LOGICAL_ID GENMASK_ULL(47, 32)
#define CMN_NODEID_DEVID(reg) ((reg) & 3)
+#define CMN_NODEID_EXT_DEVID(reg) ((reg) & 1)
#define CMN_NODEID_PID(reg) (((reg) >> 2) & 1)
+#define CMN_NODEID_EXT_PID(reg) (((reg) >> 1) & 3)
+#define CMN_NODEID_1x1_PID(reg) (((reg) >> 2) & 7)
#define CMN_NODEID_X(reg, bits) ((reg) >> (3 + (bits)))
#define CMN_NODEID_Y(reg, bits) (((reg) >> 3) & ((1U << (bits)) - 1))
@@ -34,20 +39,28 @@
#define CMN_CHILD_NODE_ADDR GENMASK(27, 0)
#define CMN_CHILD_NODE_EXTERNAL BIT(31)
-#define CMN_ADDR_NODE_PTR GENMASK(27, 14)
+#define CMN_MAX_DIMENSION 8
+#define CMN_MAX_XPS (CMN_MAX_DIMENSION * CMN_MAX_DIMENSION)
+#define CMN_MAX_DTMS (CMN_MAX_XPS + (CMN_MAX_DIMENSION - 1) * 4)
-#define CMN_NODE_PTR_DEVID(ptr) (((ptr) >> 2) & 3)
-#define CMN_NODE_PTR_PID(ptr) ((ptr) & 1)
-#define CMN_NODE_PTR_X(ptr, bits) ((ptr) >> (6 + (bits)))
-#define CMN_NODE_PTR_Y(ptr, bits) (((ptr) >> 6) & ((1U << (bits)) - 1))
-
-#define CMN_MAX_XPS (8 * 8)
-
-/* The CFG node has one other useful purpose */
+/* The CFG node has various info besides the discovery tree */
#define CMN_CFGM_PERIPH_ID_2 0x0010
#define CMN_CFGM_PID2_REVISION GENMASK(7, 4)
-/* PMU registers occupy the 3rd 4KB page of each node's 16KB space */
+#define CMN_CFGM_INFO_GLOBAL 0x900
+#define CMN_INFO_MULTIPLE_DTM_EN BIT_ULL(63)
+#define CMN_INFO_RSP_VC_NUM GENMASK_ULL(53, 52)
+#define CMN_INFO_DAT_VC_NUM GENMASK_ULL(51, 50)
+
+/* XPs also have some local topology info which has uses too */
+#define CMN_MXP__CONNECT_INFO_P0 0x0008
+#define CMN_MXP__CONNECT_INFO_P1 0x0010
+#define CMN_MXP__CONNECT_INFO_P2 0x0028
+#define CMN_MXP__CONNECT_INFO_P3 0x0030
+#define CMN_MXP__CONNECT_INFO_P4 0x0038
+#define CMN_MXP__CONNECT_INFO_P5 0x0040
+
+/* PMU registers occupy the 3rd 4KB page of each node's region */
#define CMN_PMU_OFFSET 0x2000
/* For most nodes, this is all there is */
@@ -57,6 +70,7 @@
/* DTMs live in the PMU space of XP registers */
#define CMN_DTM_WPn(n) (0x1A0 + (n) * 0x18)
#define CMN_DTM_WPn_CONFIG(n) (CMN_DTM_WPn(n) + 0x00)
+#define CMN_DTM_WPn_CONFIG_WP_DEV_SEL2 GENMASK_ULL(18,17)
#define CMN_DTM_WPn_CONFIG_WP_COMBINE BIT(6)
#define CMN_DTM_WPn_CONFIG_WP_EXCLUSIVE BIT(5)
#define CMN_DTM_WPn_CONFIG_WP_GRP BIT(4)
@@ -81,7 +95,11 @@
#define CMN_DTM_PMEVCNTSR 0x240
+#define CMN_DTM_UNIT_INFO 0x0910
+
#define CMN_DTM_NUM_COUNTERS 4
+/* Want more local counters? Why not replicate the whole DTM! Ugh... */
+#define CMN_DTM_OFFSET(n) ((n) * 0x200)
/* The DTC node is where the magic happens */
#define CMN_DT_DTC_CTL 0x0a00
@@ -122,11 +140,11 @@
/* Event attributes */
-#define CMN_CONFIG_TYPE GENMASK(15, 0)
-#define CMN_CONFIG_EVENTID GENMASK(23, 16)
-#define CMN_CONFIG_OCCUPID GENMASK(27, 24)
-#define CMN_CONFIG_BYNODEID BIT(31)
-#define CMN_CONFIG_NODEID GENMASK(47, 32)
+#define CMN_CONFIG_TYPE GENMASK_ULL(15, 0)
+#define CMN_CONFIG_EVENTID GENMASK_ULL(23, 16)
+#define CMN_CONFIG_OCCUPID GENMASK_ULL(27, 24)
+#define CMN_CONFIG_BYNODEID BIT_ULL(31)
+#define CMN_CONFIG_NODEID GENMASK_ULL(47, 32)
#define CMN_EVENT_TYPE(event) FIELD_GET(CMN_CONFIG_TYPE, (event)->attr.config)
#define CMN_EVENT_EVENTID(event) FIELD_GET(CMN_CONFIG_EVENTID, (event)->attr.config)
@@ -134,13 +152,13 @@
#define CMN_EVENT_BYNODEID(event) FIELD_GET(CMN_CONFIG_BYNODEID, (event)->attr.config)
#define CMN_EVENT_NODEID(event) FIELD_GET(CMN_CONFIG_NODEID, (event)->attr.config)
-#define CMN_CONFIG_WP_COMBINE GENMASK(27, 24)
-#define CMN_CONFIG_WP_DEV_SEL BIT(48)
-#define CMN_CONFIG_WP_CHN_SEL GENMASK(50, 49)
-#define CMN_CONFIG_WP_GRP BIT(52)
-#define CMN_CONFIG_WP_EXCLUSIVE BIT(53)
-#define CMN_CONFIG1_WP_VAL GENMASK(63, 0)
-#define CMN_CONFIG2_WP_MASK GENMASK(63, 0)
+#define CMN_CONFIG_WP_COMBINE GENMASK_ULL(27, 24)
+#define CMN_CONFIG_WP_DEV_SEL GENMASK_ULL(50, 48)
+#define CMN_CONFIG_WP_CHN_SEL GENMASK_ULL(55, 51)
+#define CMN_CONFIG_WP_GRP BIT_ULL(56)
+#define CMN_CONFIG_WP_EXCLUSIVE BIT_ULL(57)
+#define CMN_CONFIG1_WP_VAL GENMASK_ULL(63, 0)
+#define CMN_CONFIG2_WP_MASK GENMASK_ULL(63, 0)
#define CMN_EVENT_WP_COMBINE(event) FIELD_GET(CMN_CONFIG_WP_COMBINE, (event)->attr.config)
#define CMN_EVENT_WP_DEV_SEL(event) FIELD_GET(CMN_CONFIG_WP_DEV_SEL, (event)->attr.config)
@@ -155,7 +173,13 @@
#define CMN_WP_DOWN 2
-/* r0px probably don't exist in silicon, thankfully */
+enum cmn_model {
+ CMN_ANY = -1,
+ CMN600 = 1,
+ CI700 = 2,
+};
+
+/* CMN-600 r0px shouldn't exist in silicon, thankfully */
enum cmn_revision {
CMN600_R1P0,
CMN600_R1P1,
@@ -163,6 +187,10 @@ enum cmn_revision {
CMN600_R1P3,
CMN600_R2P0,
CMN600_R3P0,
+ CMN600_R3P1,
+ CI700_R0P0 = 0,
+ CI700_R1P0,
+ CI700_R2P0,
};
enum cmn_node_type {
@@ -174,9 +202,12 @@ enum cmn_node_type {
CMN_TYPE_HNF,
CMN_TYPE_XP,
CMN_TYPE_SBSX,
- CMN_TYPE_RNI = 0xa,
+ CMN_TYPE_MPAM_S,
+ CMN_TYPE_MPAM_NS,
+ CMN_TYPE_RNI,
CMN_TYPE_RND = 0xd,
CMN_TYPE_RNSAM = 0xf,
+ CMN_TYPE_MTSX,
CMN_TYPE_CXRA = 0x100,
CMN_TYPE_CXHA = 0x101,
CMN_TYPE_CXLA = 0x102,
@@ -189,32 +220,32 @@ struct arm_cmn_node {
u16 id, logid;
enum cmn_node_type type;
+ int dtm;
union {
- /* Device node */
+ /* DN/HN-F/CXHA */
struct {
- int to_xp;
- /* DN/HN-F/CXHA */
- unsigned int occupid_val;
- unsigned int occupid_count;
+ u8 occupid_val;
+ u8 occupid_count;
};
/* XP */
- struct {
- int dtc;
- u32 pmu_config_low;
- union {
- u8 input_sel[4];
- __le32 pmu_config_high;
- };
- s8 wp_event[4];
- };
+ u8 dtc;
};
-
union {
u8 event[4];
__le32 event_sel;
};
};
+struct arm_cmn_dtm {
+ void __iomem *base;
+ u32 pmu_config_low;
+ union {
+ u8 input_sel[4];
+ __le32 pmu_config_high;
+ };
+ s8 wp_event[4];
+};
+
struct arm_cmn_dtc {
void __iomem *base;
int irq;
@@ -231,35 +262,238 @@ struct arm_cmn_dtc {
struct arm_cmn {
struct device *dev;
void __iomem *base;
+ unsigned int state;
enum cmn_revision rev;
+ enum cmn_model model;
u8 mesh_x;
u8 mesh_y;
u16 num_xps;
u16 num_dns;
+ bool multi_dtm;
+ u8 ports_used;
+ struct {
+ unsigned int rsp_vc_num : 2;
+ unsigned int dat_vc_num : 2;
+ };
+
struct arm_cmn_node *xps;
struct arm_cmn_node *dns;
+ struct arm_cmn_dtm *dtms;
struct arm_cmn_dtc *dtc;
unsigned int num_dtcs;
int cpu;
struct hlist_node cpuhp_node;
- unsigned int state;
struct pmu pmu;
+ struct dentry *debug;
};
#define to_cmn(p) container_of(p, struct arm_cmn, pmu)
static int arm_cmn_hp_state;
+struct arm_cmn_nodeid {
+ u8 x;
+ u8 y;
+ u8 port;
+ u8 dev;
+};
+
+static int arm_cmn_xyidbits(const struct arm_cmn *cmn)
+{
+ int dim = max(cmn->mesh_x, cmn->mesh_y);
+
+ return dim > 4 ? 3 : 2;
+}
+
+static struct arm_cmn_nodeid arm_cmn_nid(const struct arm_cmn *cmn, u16 id)
+{
+ struct arm_cmn_nodeid nid;
+
+ if (cmn->num_xps == 1) {
+ nid.x = 0;
+ nid.y = 0;
+ nid.port = CMN_NODEID_1x1_PID(id);
+ nid.dev = CMN_NODEID_DEVID(id);
+ } else {
+ int bits = arm_cmn_xyidbits(cmn);
+
+ nid.x = CMN_NODEID_X(id, bits);
+ nid.y = CMN_NODEID_Y(id, bits);
+ if (cmn->ports_used & 0xc) {
+ nid.port = CMN_NODEID_EXT_PID(id);
+ nid.dev = CMN_NODEID_EXT_DEVID(id);
+ } else {
+ nid.port = CMN_NODEID_PID(id);
+ nid.dev = CMN_NODEID_DEVID(id);
+ }
+ }
+ return nid;
+}
+
+static struct arm_cmn_node *arm_cmn_node_to_xp(const struct arm_cmn *cmn,
+ const struct arm_cmn_node *dn)
+{
+ struct arm_cmn_nodeid nid = arm_cmn_nid(cmn, dn->id);
+ int xp_idx = cmn->mesh_x * nid.y + nid.x;
+
+ return cmn->xps + xp_idx;
+}
+static struct arm_cmn_node *arm_cmn_node(const struct arm_cmn *cmn,
+ enum cmn_node_type type)
+{
+ struct arm_cmn_node *dn;
+
+ for (dn = cmn->dns; dn->type; dn++)
+ if (dn->type == type)
+ return dn;
+ return NULL;
+}
+
+struct dentry *arm_cmn_debugfs;
+
+#ifdef CONFIG_DEBUG_FS
+static const char *arm_cmn_device_type(u8 type)
+{
+ switch(type) {
+ case 0x01: return " RN-I |";
+ case 0x02: return " RN-D |";
+ case 0x04: return " RN-F_B |";
+ case 0x05: return "RN-F_B_E|";
+ case 0x06: return " RN-F_A |";
+ case 0x07: return "RN-F_A_E|";
+ case 0x08: return " HN-T |";
+ case 0x09: return " HN-I |";
+ case 0x0a: return " HN-D |";
+ case 0x0c: return " SN-F |";
+ case 0x0d: return " SBSX |";
+ case 0x0e: return " HN-F |";
+ case 0x0f: return " SN-F_E |";
+ case 0x10: return " SN-F_D |";
+ case 0x11: return " CXHA |";
+ case 0x12: return " CXRA |";
+ case 0x13: return " CXRH |";
+ case 0x14: return " RN-F_D |";
+ case 0x15: return "RN-F_D_E|";
+ case 0x16: return " RN-F_C |";
+ case 0x17: return "RN-F_C_E|";
+ case 0x1c: return " MTSX |";
+ default: return " |";
+ }
+}
+
+static void arm_cmn_show_logid(struct seq_file *s, int x, int y, int p, int d)
+{
+ struct arm_cmn *cmn = s->private;
+ struct arm_cmn_node *dn;
+
+ for (dn = cmn->dns; dn->type; dn++) {
+ struct arm_cmn_nodeid nid = arm_cmn_nid(cmn, dn->id);
+
+ if (dn->type == CMN_TYPE_XP)
+ continue;
+ /* Ignore the extra components that will overlap on some ports */
+ if (dn->type < CMN_TYPE_HNI)
+ continue;
+
+ if (nid.x != x || nid.y != y || nid.port != p || nid.dev != d)
+ continue;
+
+ seq_printf(s, " #%-2d |", dn->logid);
+ return;
+ }
+ seq_puts(s, " |");
+}
+
+static int arm_cmn_map_show(struct seq_file *s, void *data)
+{
+ struct arm_cmn *cmn = s->private;
+ int x, y, p, pmax = fls(cmn->ports_used);
+
+ seq_puts(s, " X");
+ for (x = 0; x < cmn->mesh_x; x++)
+ seq_printf(s, " %d ", x);
+ seq_puts(s, "\nY P D+");
+ y = cmn->mesh_y;
+ while (y--) {
+ int xp_base = cmn->mesh_x * y;
+ u8 port[6][CMN_MAX_DIMENSION];
+
+ for (x = 0; x < cmn->mesh_x; x++)
+ seq_puts(s, "--------+");
+
+ seq_printf(s, "\n%d |", y);
+ for (x = 0; x < cmn->mesh_x; x++) {
+ struct arm_cmn_node *xp = cmn->xps + xp_base + x;
+ void __iomem *base = xp->pmu_base - CMN_PMU_OFFSET;
+
+ port[0][x] = readl_relaxed(base + CMN_MXP__CONNECT_INFO_P0);
+ port[1][x] = readl_relaxed(base + CMN_MXP__CONNECT_INFO_P1);
+ port[2][x] = readl_relaxed(base + CMN_MXP__CONNECT_INFO_P2);
+ port[3][x] = readl_relaxed(base + CMN_MXP__CONNECT_INFO_P3);
+ port[4][x] = readl_relaxed(base + CMN_MXP__CONNECT_INFO_P4);
+ port[5][x] = readl_relaxed(base + CMN_MXP__CONNECT_INFO_P5);
+ seq_printf(s, " XP #%-2d |", xp_base + x);
+ }
+
+ seq_puts(s, "\n |");
+ for (x = 0; x < cmn->mesh_x; x++) {
+ u8 dtc = cmn->xps[xp_base + x].dtc;
+
+ if (dtc & (dtc - 1))
+ seq_puts(s, " DTC ?? |");
+ else
+ seq_printf(s, " DTC %ld |", __ffs(dtc));
+ }
+ seq_puts(s, "\n |");
+ for (x = 0; x < cmn->mesh_x; x++)
+ seq_puts(s, "........|");
+
+ for (p = 0; p < pmax; p++) {
+ seq_printf(s, "\n %d |", p);
+ for (x = 0; x < cmn->mesh_x; x++)
+ seq_puts(s, arm_cmn_device_type(port[p][x]));
+ seq_puts(s, "\n 0|");
+ for (x = 0; x < cmn->mesh_x; x++)
+ arm_cmn_show_logid(s, x, y, p, 0);
+ seq_puts(s, "\n 1|");
+ for (x = 0; x < cmn->mesh_x; x++)
+ arm_cmn_show_logid(s, x, y, p, 1);
+ }
+ seq_puts(s, "\n-----+");
+ }
+ for (x = 0; x < cmn->mesh_x; x++)
+ seq_puts(s, "--------+");
+ seq_puts(s, "\n");
+ return 0;
+}
+DEFINE_SHOW_ATTRIBUTE(arm_cmn_map);
+
+static void arm_cmn_debugfs_init(struct arm_cmn *cmn, int id)
+{
+ const char *name = "map";
+
+ if (id > 0)
+ name = devm_kasprintf(cmn->dev, GFP_KERNEL, "map_%d", id);
+ if (!name)
+ return;
+
+ cmn->debug = debugfs_create_file(name, 0444, arm_cmn_debugfs, cmn, &arm_cmn_map_fops);
+}
+#else
+static void arm_cmn_debugfs_init(struct arm_cmn *cmn, int id) {}
+#endif
+
struct arm_cmn_hw_event {
struct arm_cmn_node *dn;
u64 dtm_idx[2];
unsigned int dtc_idx;
u8 dtcs_used;
u8 num_dns;
+ u8 dtm_offset;
};
#define for_each_hw_dn(hw, dn, i) \
@@ -283,6 +517,7 @@ static unsigned int arm_cmn_get_index(u64 x[], unsigned int pos)
struct arm_cmn_event_attr {
struct device_attribute attr;
+ enum cmn_model model;
enum cmn_node_type type;
u8 eventid;
u8 occupid;
@@ -294,50 +529,22 @@ struct arm_cmn_format_attr {
int config;
};
-static int arm_cmn_xyidbits(const struct arm_cmn *cmn)
-{
- return cmn->mesh_x > 4 || cmn->mesh_y > 4 ? 3 : 2;
-}
-
-static void arm_cmn_init_node_to_xp(const struct arm_cmn *cmn,
- struct arm_cmn_node *dn)
-{
- int bits = arm_cmn_xyidbits(cmn);
- int x = CMN_NODEID_X(dn->id, bits);
- int y = CMN_NODEID_Y(dn->id, bits);
- int xp_idx = cmn->mesh_x * y + x;
-
- dn->to_xp = (cmn->xps + xp_idx) - dn;
-}
-
-static struct arm_cmn_node *arm_cmn_node_to_xp(struct arm_cmn_node *dn)
-{
- return dn->type == CMN_TYPE_XP ? dn : dn + dn->to_xp;
-}
-
-static struct arm_cmn_node *arm_cmn_node(const struct arm_cmn *cmn,
- enum cmn_node_type type)
-{
- int i;
-
- for (i = 0; i < cmn->num_dns; i++)
- if (cmn->dns[i].type == type)
- return &cmn->dns[i];
- return NULL;
-}
-
-#define CMN_EVENT_ATTR(_name, _type, _eventid, _occupid) \
+#define CMN_EVENT_ATTR(_model, _name, _type, _eventid, _occupid) \
(&((struct arm_cmn_event_attr[]) {{ \
.attr = __ATTR(_name, 0444, arm_cmn_event_show, NULL), \
+ .model = _model, \
.type = _type, \
.eventid = _eventid, \
.occupid = _occupid, \
}})[0].attr.attr)
-static bool arm_cmn_is_occup_event(enum cmn_node_type type, unsigned int id)
+static bool arm_cmn_is_occup_event(enum cmn_model model,
+ enum cmn_node_type type, unsigned int id)
{
- return (type == CMN_TYPE_DVM && id == 0x05) ||
- (type == CMN_TYPE_HNF && id == 0x0f);
+ if (type == CMN_TYPE_DVM)
+ return (model == CMN600 && id == 0x05) ||
+ (model == CI700 && id == 0x0c);
+ return type == CMN_TYPE_HNF && id == 0x0f;
}
static ssize_t arm_cmn_event_show(struct device *dev,
@@ -355,7 +562,7 @@ static ssize_t arm_cmn_event_show(struct device *dev,
"type=0x%x,eventid=0x%x,wp_dev_sel=?,wp_chn_sel=?,wp_grp=?,wp_val=?,wp_mask=?\n",
eattr->type, eattr->eventid);
- if (arm_cmn_is_occup_event(eattr->type, eattr->eventid))
+ if (arm_cmn_is_occup_event(eattr->model, eattr->type, eattr->eventid))
return sysfs_emit(buf, "type=0x%x,eventid=0x%x,occupid=0x%x\n",
eattr->type, eattr->eventid, eattr->occupid);
@@ -370,60 +577,81 @@ static umode_t arm_cmn_event_attr_is_visible(struct kobject *kobj,
struct device *dev = kobj_to_dev(kobj);
struct arm_cmn *cmn = to_cmn(dev_get_drvdata(dev));
struct arm_cmn_event_attr *eattr;
- enum cmn_node_type type;
eattr = container_of(attr, typeof(*eattr), attr.attr);
- type = eattr->type;
- /* Watchpoints aren't nodes */
- if (type == CMN_TYPE_WP)
- type = CMN_TYPE_XP;
+ if (!(eattr->model & cmn->model))
+ return 0;
- /* Revision-specific differences */
- if (cmn->rev < CMN600_R1P2) {
- if (type == CMN_TYPE_HNF && eattr->eventid == 0x1b)
+ /* Watchpoints aren't nodes, so avoid confusion */
+ if (eattr->type == CMN_TYPE_WP)
+ return attr->mode;
+
+ /* Hide XP events for unused interfaces/channels */
+ if (eattr->type == CMN_TYPE_XP) {
+ unsigned int intf = (eattr->eventid >> 2) & 7;
+ unsigned int chan = eattr->eventid >> 5;
+
+ if ((intf & 4) && !(cmn->ports_used & BIT(intf & 3)))
+ return 0;
+
+ if ((chan == 5 && cmn->rsp_vc_num < 2) ||
+ (chan == 6 && cmn->dat_vc_num < 2))
return 0;
}
- if (!arm_cmn_node(cmn, type))
+ /* Revision-specific differences */
+ if (cmn->model == CMN600 && cmn->rev < CMN600_R1P2) {
+ if (eattr->type == CMN_TYPE_HNF && eattr->eventid == 0x1b)
+ return 0;
+ }
+
+ if (!arm_cmn_node(cmn, eattr->type))
return 0;
return attr->mode;
}
-#define _CMN_EVENT_DVM(_name, _event, _occup) \
- CMN_EVENT_ATTR(dn_##_name, CMN_TYPE_DVM, _event, _occup)
+#define _CMN_EVENT_DVM(_model, _name, _event, _occup) \
+ CMN_EVENT_ATTR(_model, dn_##_name, CMN_TYPE_DVM, _event, _occup)
#define CMN_EVENT_DTC(_name) \
- CMN_EVENT_ATTR(dtc_##_name, CMN_TYPE_DTC, 0, 0)
-#define _CMN_EVENT_HNF(_name, _event, _occup) \
- CMN_EVENT_ATTR(hnf_##_name, CMN_TYPE_HNF, _event, _occup)
+ CMN_EVENT_ATTR(CMN_ANY, dtc_##_name, CMN_TYPE_DTC, 0, 0)
+#define _CMN_EVENT_HNF(_model, _name, _event, _occup) \
+ CMN_EVENT_ATTR(_model, hnf_##_name, CMN_TYPE_HNF, _event, _occup)
#define CMN_EVENT_HNI(_name, _event) \
- CMN_EVENT_ATTR(hni_##_name, CMN_TYPE_HNI, _event, 0)
+ CMN_EVENT_ATTR(CMN_ANY, hni_##_name, CMN_TYPE_HNI, _event, 0)
#define __CMN_EVENT_XP(_name, _event) \
- CMN_EVENT_ATTR(mxp_##_name, CMN_TYPE_XP, _event, 0)
-#define CMN_EVENT_SBSX(_name, _event) \
- CMN_EVENT_ATTR(sbsx_##_name, CMN_TYPE_SBSX, _event, 0)
-#define CMN_EVENT_RNID(_name, _event) \
- CMN_EVENT_ATTR(rnid_##_name, CMN_TYPE_RNI, _event, 0)
+ CMN_EVENT_ATTR(CMN_ANY, mxp_##_name, CMN_TYPE_XP, _event, 0)
+#define CMN_EVENT_SBSX(_model, _name, _event) \
+ CMN_EVENT_ATTR(_model, sbsx_##_name, CMN_TYPE_SBSX, _event, 0)
+#define CMN_EVENT_RNID(_model, _name, _event) \
+ CMN_EVENT_ATTR(_model, rnid_##_name, CMN_TYPE_RNI, _event, 0)
+#define CMN_EVENT_MTSX(_name, _event) \
+ CMN_EVENT_ATTR(CMN_ANY, mtsx_##_name, CMN_TYPE_MTSX, _event, 0)
-#define CMN_EVENT_DVM(_name, _event) \
- _CMN_EVENT_DVM(_name, _event, 0)
-#define CMN_EVENT_HNF(_name, _event) \
- _CMN_EVENT_HNF(_name, _event, 0)
+#define CMN_EVENT_DVM(_model, _name, _event) \
+ _CMN_EVENT_DVM(_model, _name, _event, 0)
+#define CMN_EVENT_HNF(_model, _name, _event) \
+ _CMN_EVENT_HNF(_model, _name, _event, 0)
#define _CMN_EVENT_XP(_name, _event) \
__CMN_EVENT_XP(e_##_name, (_event) | (0 << 2)), \
__CMN_EVENT_XP(w_##_name, (_event) | (1 << 2)), \
__CMN_EVENT_XP(n_##_name, (_event) | (2 << 2)), \
__CMN_EVENT_XP(s_##_name, (_event) | (3 << 2)), \
__CMN_EVENT_XP(p0_##_name, (_event) | (4 << 2)), \
- __CMN_EVENT_XP(p1_##_name, (_event) | (5 << 2))
+ __CMN_EVENT_XP(p1_##_name, (_event) | (5 << 2)), \
+ __CMN_EVENT_XP(p2_##_name, (_event) | (6 << 2)), \
+ __CMN_EVENT_XP(p3_##_name, (_event) | (7 << 2))
/* Good thing there are only 3 fundamental XP events... */
#define CMN_EVENT_XP(_name, _event) \
_CMN_EVENT_XP(req_##_name, (_event) | (0 << 5)), \
_CMN_EVENT_XP(rsp_##_name, (_event) | (1 << 5)), \
_CMN_EVENT_XP(snp_##_name, (_event) | (2 << 5)), \
- _CMN_EVENT_XP(dat_##_name, (_event) | (3 << 5))
+ _CMN_EVENT_XP(dat_##_name, (_event) | (3 << 5)), \
+ _CMN_EVENT_XP(pub_##_name, (_event) | (4 << 5)), \
+ _CMN_EVENT_XP(rsp2_##_name, (_event) | (5 << 5)), \
+ _CMN_EVENT_XP(dat2_##_name, (_event) | (6 << 5))
static struct attribute *arm_cmn_event_attrs[] = {
@@ -434,115 +662,152 @@ static struct attribute *arm_cmn_event_attrs[] = {
* slot, but our lazy short-cut of using the DTM counter index for
* the PMU index as well happens to avoid that by construction.
*/
- CMN_EVENT_DVM(rxreq_dvmop, 0x01),
- CMN_EVENT_DVM(rxreq_dvmsync, 0x02),
- CMN_EVENT_DVM(rxreq_dvmop_vmid_filtered, 0x03),
- CMN_EVENT_DVM(rxreq_retried, 0x04),
- _CMN_EVENT_DVM(rxreq_trk_occupancy_all, 0x05, 0),
- _CMN_EVENT_DVM(rxreq_trk_occupancy_dvmop, 0x05, 1),
- _CMN_EVENT_DVM(rxreq_trk_occupancy_dvmsync, 0x05, 2),
+ CMN_EVENT_DVM(CMN600, rxreq_dvmop, 0x01),
+ CMN_EVENT_DVM(CMN600, rxreq_dvmsync, 0x02),
+ CMN_EVENT_DVM(CMN600, rxreq_dvmop_vmid_filtered, 0x03),
+ CMN_EVENT_DVM(CMN600, rxreq_retried, 0x04),
+ _CMN_EVENT_DVM(CMN600, rxreq_trk_occupancy_all, 0x05, 0),
+ _CMN_EVENT_DVM(CMN600, rxreq_trk_occupancy_dvmop, 0x05, 1),
+ _CMN_EVENT_DVM(CMN600, rxreq_trk_occupancy_dvmsync, 0x05, 2),
+ CMN_EVENT_DVM(CI700, dvmop_tlbi, 0x01),
+ CMN_EVENT_DVM(CI700, dvmop_bpi, 0x02),
+ CMN_EVENT_DVM(CI700, dvmop_pici, 0x03),
+ CMN_EVENT_DVM(CI700, dvmop_vici, 0x04),
+ CMN_EVENT_DVM(CI700, dvmsync, 0x05),
+ CMN_EVENT_DVM(CI700, vmid_filtered, 0x06),
+ CMN_EVENT_DVM(CI700, rndop_filtered, 0x07),
+ CMN_EVENT_DVM(CI700, retry, 0x08),
+ CMN_EVENT_DVM(CI700, txsnp_flitv, 0x09),
+ CMN_EVENT_DVM(CI700, txsnp_stall, 0x0a),
+ CMN_EVENT_DVM(CI700, trkfull, 0x0b),
+ _CMN_EVENT_DVM(CI700, trk_occupancy_all, 0x0c, 0),
+ _CMN_EVENT_DVM(CI700, trk_occupancy_dvmop, 0x0c, 1),
+ _CMN_EVENT_DVM(CI700, trk_occupancy_dvmsync, 0x0c, 2),
- CMN_EVENT_HNF(cache_miss, 0x01),
- CMN_EVENT_HNF(slc_sf_cache_access, 0x02),
- CMN_EVENT_HNF(cache_fill, 0x03),
- CMN_EVENT_HNF(pocq_retry, 0x04),
- CMN_EVENT_HNF(pocq_reqs_recvd, 0x05),
- CMN_EVENT_HNF(sf_hit, 0x06),
- CMN_EVENT_HNF(sf_evictions, 0x07),
- CMN_EVENT_HNF(dir_snoops_sent, 0x08),
- CMN_EVENT_HNF(brd_snoops_sent, 0x09),
- CMN_EVENT_HNF(slc_eviction, 0x0a),
- CMN_EVENT_HNF(slc_fill_invalid_way, 0x0b),
- CMN_EVENT_HNF(mc_retries, 0x0c),
- CMN_EVENT_HNF(mc_reqs, 0x0d),
- CMN_EVENT_HNF(qos_hh_retry, 0x0e),
- _CMN_EVENT_HNF(qos_pocq_occupancy_all, 0x0f, 0),
- _CMN_EVENT_HNF(qos_pocq_occupancy_read, 0x0f, 1),
- _CMN_EVENT_HNF(qos_pocq_occupancy_write, 0x0f, 2),
- _CMN_EVENT_HNF(qos_pocq_occupancy_atomic, 0x0f, 3),
- _CMN_EVENT_HNF(qos_pocq_occupancy_stash, 0x0f, 4),
- CMN_EVENT_HNF(pocq_addrhaz, 0x10),
- CMN_EVENT_HNF(pocq_atomic_addrhaz, 0x11),
- CMN_EVENT_HNF(ld_st_swp_adq_full, 0x12),
- CMN_EVENT_HNF(cmp_adq_full, 0x13),
- CMN_EVENT_HNF(txdat_stall, 0x14),
- CMN_EVENT_HNF(txrsp_stall, 0x15),
- CMN_EVENT_HNF(seq_full, 0x16),
- CMN_EVENT_HNF(seq_hit, 0x17),
- CMN_EVENT_HNF(snp_sent, 0x18),
- CMN_EVENT_HNF(sfbi_dir_snp_sent, 0x19),
- CMN_EVENT_HNF(sfbi_brd_snp_sent, 0x1a),
- CMN_EVENT_HNF(snp_sent_untrk, 0x1b),
- CMN_EVENT_HNF(intv_dirty, 0x1c),
- CMN_EVENT_HNF(stash_snp_sent, 0x1d),
- CMN_EVENT_HNF(stash_data_pull, 0x1e),
- CMN_EVENT_HNF(snp_fwded, 0x1f),
+ CMN_EVENT_HNF(CMN_ANY, cache_miss, 0x01),
+ CMN_EVENT_HNF(CMN_ANY, slc_sf_cache_access, 0x02),
+ CMN_EVENT_HNF(CMN_ANY, cache_fill, 0x03),
+ CMN_EVENT_HNF(CMN_ANY, pocq_retry, 0x04),
+ CMN_EVENT_HNF(CMN_ANY, pocq_reqs_recvd, 0x05),
+ CMN_EVENT_HNF(CMN_ANY, sf_hit, 0x06),
+ CMN_EVENT_HNF(CMN_ANY, sf_evictions, 0x07),
+ CMN_EVENT_HNF(CMN_ANY, dir_snoops_sent, 0x08),
+ CMN_EVENT_HNF(CMN_ANY, brd_snoops_sent, 0x09),
+ CMN_EVENT_HNF(CMN_ANY, slc_eviction, 0x0a),
+ CMN_EVENT_HNF(CMN_ANY, slc_fill_invalid_way, 0x0b),
+ CMN_EVENT_HNF(CMN_ANY, mc_retries, 0x0c),
+ CMN_EVENT_HNF(CMN_ANY, mc_reqs, 0x0d),
+ CMN_EVENT_HNF(CMN_ANY, qos_hh_retry, 0x0e),
+ _CMN_EVENT_HNF(CMN_ANY, qos_pocq_occupancy_all, 0x0f, 0),
+ _CMN_EVENT_HNF(CMN_ANY, qos_pocq_occupancy_read, 0x0f, 1),
+ _CMN_EVENT_HNF(CMN_ANY, qos_pocq_occupancy_write, 0x0f, 2),
+ _CMN_EVENT_HNF(CMN_ANY, qos_pocq_occupancy_atomic, 0x0f, 3),
+ _CMN_EVENT_HNF(CMN_ANY, qos_pocq_occupancy_stash, 0x0f, 4),
+ CMN_EVENT_HNF(CMN_ANY, pocq_addrhaz, 0x10),
+ CMN_EVENT_HNF(CMN_ANY, pocq_atomic_addrhaz, 0x11),
+ CMN_EVENT_HNF(CMN_ANY, ld_st_swp_adq_full, 0x12),
+ CMN_EVENT_HNF(CMN_ANY, cmp_adq_full, 0x13),
+ CMN_EVENT_HNF(CMN_ANY, txdat_stall, 0x14),
+ CMN_EVENT_HNF(CMN_ANY, txrsp_stall, 0x15),
+ CMN_EVENT_HNF(CMN_ANY, seq_full, 0x16),
+ CMN_EVENT_HNF(CMN_ANY, seq_hit, 0x17),
+ CMN_EVENT_HNF(CMN_ANY, snp_sent, 0x18),
+ CMN_EVENT_HNF(CMN_ANY, sfbi_dir_snp_sent, 0x19),
+ CMN_EVENT_HNF(CMN_ANY, sfbi_brd_snp_sent, 0x1a),
+ CMN_EVENT_HNF(CMN_ANY, snp_sent_untrk, 0x1b),
+ CMN_EVENT_HNF(CMN_ANY, intv_dirty, 0x1c),
+ CMN_EVENT_HNF(CMN_ANY, stash_snp_sent, 0x1d),
+ CMN_EVENT_HNF(CMN_ANY, stash_data_pull, 0x1e),
+ CMN_EVENT_HNF(CMN_ANY, snp_fwded, 0x1f),
+ CMN_EVENT_HNF(CI700, atomic_fwd, 0x20),
+ CMN_EVENT_HNF(CI700, mpam_hardlim, 0x21),
+ CMN_EVENT_HNF(CI700, mpam_softlim, 0x22),
- CMN_EVENT_HNI(rrt_rd_occ_cnt_ovfl, 0x20),
- CMN_EVENT_HNI(rrt_wr_occ_cnt_ovfl, 0x21),
- CMN_EVENT_HNI(rdt_rd_occ_cnt_ovfl, 0x22),
- CMN_EVENT_HNI(rdt_wr_occ_cnt_ovfl, 0x23),
- CMN_EVENT_HNI(wdb_occ_cnt_ovfl, 0x24),
- CMN_EVENT_HNI(rrt_rd_alloc, 0x25),
- CMN_EVENT_HNI(rrt_wr_alloc, 0x26),
- CMN_EVENT_HNI(rdt_rd_alloc, 0x27),
- CMN_EVENT_HNI(rdt_wr_alloc, 0x28),
- CMN_EVENT_HNI(wdb_alloc, 0x29),
- CMN_EVENT_HNI(txrsp_retryack, 0x2a),
- CMN_EVENT_HNI(arvalid_no_arready, 0x2b),
- CMN_EVENT_HNI(arready_no_arvalid, 0x2c),
- CMN_EVENT_HNI(awvalid_no_awready, 0x2d),
- CMN_EVENT_HNI(awready_no_awvalid, 0x2e),
- CMN_EVENT_HNI(wvalid_no_wready, 0x2f),
- CMN_EVENT_HNI(txdat_stall, 0x30),
- CMN_EVENT_HNI(nonpcie_serialization, 0x31),
- CMN_EVENT_HNI(pcie_serialization, 0x32),
+ CMN_EVENT_HNI(rrt_rd_occ_cnt_ovfl, 0x20),
+ CMN_EVENT_HNI(rrt_wr_occ_cnt_ovfl, 0x21),
+ CMN_EVENT_HNI(rdt_rd_occ_cnt_ovfl, 0x22),
+ CMN_EVENT_HNI(rdt_wr_occ_cnt_ovfl, 0x23),
+ CMN_EVENT_HNI(wdb_occ_cnt_ovfl, 0x24),
+ CMN_EVENT_HNI(rrt_rd_alloc, 0x25),
+ CMN_EVENT_HNI(rrt_wr_alloc, 0x26),
+ CMN_EVENT_HNI(rdt_rd_alloc, 0x27),
+ CMN_EVENT_HNI(rdt_wr_alloc, 0x28),
+ CMN_EVENT_HNI(wdb_alloc, 0x29),
+ CMN_EVENT_HNI(txrsp_retryack, 0x2a),
+ CMN_EVENT_HNI(arvalid_no_arready, 0x2b),
+ CMN_EVENT_HNI(arready_no_arvalid, 0x2c),
+ CMN_EVENT_HNI(awvalid_no_awready, 0x2d),
+ CMN_EVENT_HNI(awready_no_awvalid, 0x2e),
+ CMN_EVENT_HNI(wvalid_no_wready, 0x2f),
+ CMN_EVENT_HNI(txdat_stall, 0x30),
+ CMN_EVENT_HNI(nonpcie_serialization, 0x31),
+ CMN_EVENT_HNI(pcie_serialization, 0x32),
- CMN_EVENT_XP(txflit_valid, 0x01),
- CMN_EVENT_XP(txflit_stall, 0x02),
- CMN_EVENT_XP(partial_dat_flit, 0x03),
+ CMN_EVENT_XP(txflit_valid, 0x01),
+ CMN_EVENT_XP(txflit_stall, 0x02),
+ CMN_EVENT_XP(partial_dat_flit, 0x03),
/* We treat watchpoints as a special made-up class of XP events */
- CMN_EVENT_ATTR(watchpoint_up, CMN_TYPE_WP, 0, 0),
- CMN_EVENT_ATTR(watchpoint_down, CMN_TYPE_WP, 2, 0),
+ CMN_EVENT_ATTR(CMN_ANY, watchpoint_up, CMN_TYPE_WP, CMN_WP_UP, 0),
+ CMN_EVENT_ATTR(CMN_ANY, watchpoint_down, CMN_TYPE_WP, CMN_WP_DOWN, 0),
- CMN_EVENT_SBSX(rd_req, 0x01),
- CMN_EVENT_SBSX(wr_req, 0x02),
- CMN_EVENT_SBSX(cmo_req, 0x03),
- CMN_EVENT_SBSX(txrsp_retryack, 0x04),
- CMN_EVENT_SBSX(txdat_flitv, 0x05),
- CMN_EVENT_SBSX(txrsp_flitv, 0x06),
- CMN_EVENT_SBSX(rd_req_trkr_occ_cnt_ovfl, 0x11),
- CMN_EVENT_SBSX(wr_req_trkr_occ_cnt_ovfl, 0x12),
- CMN_EVENT_SBSX(cmo_req_trkr_occ_cnt_ovfl, 0x13),
- CMN_EVENT_SBSX(wdb_occ_cnt_ovfl, 0x14),
- CMN_EVENT_SBSX(rd_axi_trkr_occ_cnt_ovfl, 0x15),
- CMN_EVENT_SBSX(cmo_axi_trkr_occ_cnt_ovfl, 0x16),
- CMN_EVENT_SBSX(arvalid_no_arready, 0x21),
- CMN_EVENT_SBSX(awvalid_no_awready, 0x22),
- CMN_EVENT_SBSX(wvalid_no_wready, 0x23),
- CMN_EVENT_SBSX(txdat_stall, 0x24),
- CMN_EVENT_SBSX(txrsp_stall, 0x25),
+ CMN_EVENT_SBSX(CMN_ANY, rd_req, 0x01),
+ CMN_EVENT_SBSX(CMN_ANY, wr_req, 0x02),
+ CMN_EVENT_SBSX(CMN_ANY, cmo_req, 0x03),
+ CMN_EVENT_SBSX(CMN_ANY, txrsp_retryack, 0x04),
+ CMN_EVENT_SBSX(CMN_ANY, txdat_flitv, 0x05),
+ CMN_EVENT_SBSX(CMN_ANY, txrsp_flitv, 0x06),
+ CMN_EVENT_SBSX(CMN_ANY, rd_req_trkr_occ_cnt_ovfl, 0x11),
+ CMN_EVENT_SBSX(CMN_ANY, wr_req_trkr_occ_cnt_ovfl, 0x12),
+ CMN_EVENT_SBSX(CMN_ANY, cmo_req_trkr_occ_cnt_ovfl, 0x13),
+ CMN_EVENT_SBSX(CMN_ANY, wdb_occ_cnt_ovfl, 0x14),
+ CMN_EVENT_SBSX(CMN_ANY, rd_axi_trkr_occ_cnt_ovfl, 0x15),
+ CMN_EVENT_SBSX(CMN_ANY, cmo_axi_trkr_occ_cnt_ovfl, 0x16),
+ CMN_EVENT_SBSX(CI700, rdb_occ_cnt_ovfl, 0x17),
+ CMN_EVENT_SBSX(CMN_ANY, arvalid_no_arready, 0x21),
+ CMN_EVENT_SBSX(CMN_ANY, awvalid_no_awready, 0x22),
+ CMN_EVENT_SBSX(CMN_ANY, wvalid_no_wready, 0x23),
+ CMN_EVENT_SBSX(CMN_ANY, txdat_stall, 0x24),
+ CMN_EVENT_SBSX(CMN_ANY, txrsp_stall, 0x25),
- CMN_EVENT_RNID(s0_rdata_beats, 0x01),
- CMN_EVENT_RNID(s1_rdata_beats, 0x02),
- CMN_EVENT_RNID(s2_rdata_beats, 0x03),
- CMN_EVENT_RNID(rxdat_flits, 0x04),
- CMN_EVENT_RNID(txdat_flits, 0x05),
- CMN_EVENT_RNID(txreq_flits_total, 0x06),
- CMN_EVENT_RNID(txreq_flits_retried, 0x07),
- CMN_EVENT_RNID(rrt_occ_ovfl, 0x08),
- CMN_EVENT_RNID(wrt_occ_ovfl, 0x09),
- CMN_EVENT_RNID(txreq_flits_replayed, 0x0a),
- CMN_EVENT_RNID(wrcancel_sent, 0x0b),
- CMN_EVENT_RNID(s0_wdata_beats, 0x0c),
- CMN_EVENT_RNID(s1_wdata_beats, 0x0d),
- CMN_EVENT_RNID(s2_wdata_beats, 0x0e),
- CMN_EVENT_RNID(rrt_alloc, 0x0f),
- CMN_EVENT_RNID(wrt_alloc, 0x10),
- CMN_EVENT_RNID(rdb_unord, 0x11),
- CMN_EVENT_RNID(rdb_replay, 0x12),
- CMN_EVENT_RNID(rdb_hybrid, 0x13),
- CMN_EVENT_RNID(rdb_ord, 0x14),
+ CMN_EVENT_RNID(CMN_ANY, s0_rdata_beats, 0x01),
+ CMN_EVENT_RNID(CMN_ANY, s1_rdata_beats, 0x02),
+ CMN_EVENT_RNID(CMN_ANY, s2_rdata_beats, 0x03),
+ CMN_EVENT_RNID(CMN_ANY, rxdat_flits, 0x04),
+ CMN_EVENT_RNID(CMN_ANY, txdat_flits, 0x05),
+ CMN_EVENT_RNID(CMN_ANY, txreq_flits_total, 0x06),
+ CMN_EVENT_RNID(CMN_ANY, txreq_flits_retried, 0x07),
+ CMN_EVENT_RNID(CMN_ANY, rrt_occ_ovfl, 0x08),
+ CMN_EVENT_RNID(CMN_ANY, wrt_occ_ovfl, 0x09),
+ CMN_EVENT_RNID(CMN_ANY, txreq_flits_replayed, 0x0a),
+ CMN_EVENT_RNID(CMN_ANY, wrcancel_sent, 0x0b),
+ CMN_EVENT_RNID(CMN_ANY, s0_wdata_beats, 0x0c),
+ CMN_EVENT_RNID(CMN_ANY, s1_wdata_beats, 0x0d),
+ CMN_EVENT_RNID(CMN_ANY, s2_wdata_beats, 0x0e),
+ CMN_EVENT_RNID(CMN_ANY, rrt_alloc, 0x0f),
+ CMN_EVENT_RNID(CMN_ANY, wrt_alloc, 0x10),
+ CMN_EVENT_RNID(CMN600, rdb_unord, 0x11),
+ CMN_EVENT_RNID(CMN600, rdb_replay, 0x12),
+ CMN_EVENT_RNID(CMN600, rdb_hybrid, 0x13),
+ CMN_EVENT_RNID(CMN600, rdb_ord, 0x14),
+ CMN_EVENT_RNID(CI700, padb_occ_ovfl, 0x11),
+ CMN_EVENT_RNID(CI700, rpdb_occ_ovfl, 0x12),
+ CMN_EVENT_RNID(CI700, rrt_occup_ovfl_slice1, 0x13),
+ CMN_EVENT_RNID(CI700, rrt_occup_ovfl_slice2, 0x14),
+ CMN_EVENT_RNID(CI700, rrt_occup_ovfl_slice3, 0x15),
+ CMN_EVENT_RNID(CI700, wrt_throttled, 0x16),
+
+ CMN_EVENT_MTSX(tc_lookup, 0x01),
+ CMN_EVENT_MTSX(tc_fill, 0x02),
+ CMN_EVENT_MTSX(tc_miss, 0x03),
+ CMN_EVENT_MTSX(tdb_forward, 0x04),
+ CMN_EVENT_MTSX(tcq_hazard, 0x05),
+ CMN_EVENT_MTSX(tcq_rd_alloc, 0x06),
+ CMN_EVENT_MTSX(tcq_wr_alloc, 0x07),
+ CMN_EVENT_MTSX(tcq_cmo_alloc, 0x08),
+ CMN_EVENT_MTSX(axi_rd_req, 0x09),
+ CMN_EVENT_MTSX(axi_wr_req, 0x0a),
+ CMN_EVENT_MTSX(tcq_occ_cnt_ovfl, 0x0b),
+ CMN_EVENT_MTSX(tdb_occ_cnt_ovfl, 0x0c),
NULL
};
@@ -644,7 +909,8 @@ static u32 arm_cmn_wp_config(struct perf_event *event)
config = FIELD_PREP(CMN_DTM_WPn_CONFIG_WP_DEV_SEL, dev) |
FIELD_PREP(CMN_DTM_WPn_CONFIG_WP_CHN_SEL, chn) |
FIELD_PREP(CMN_DTM_WPn_CONFIG_WP_GRP, grp) |
- FIELD_PREP(CMN_DTM_WPn_CONFIG_WP_EXCLUSIVE, exc);
+ FIELD_PREP(CMN_DTM_WPn_CONFIG_WP_EXCLUSIVE, exc) |
+ FIELD_PREP(CMN_DTM_WPn_CONFIG_WP_DEV_SEL2, dev >> 1);
if (combine && !grp)
config |= CMN_DTM_WPn_CONFIG_WP_COMBINE;
@@ -679,18 +945,19 @@ static void arm_cmn_pmu_disable(struct pmu *pmu)
static u64 arm_cmn_read_dtm(struct arm_cmn *cmn, struct arm_cmn_hw_event *hw,
bool snapshot)
{
+ struct arm_cmn_dtm *dtm = NULL;
struct arm_cmn_node *dn;
- unsigned int i, offset;
- u64 count = 0;
+ unsigned int i, offset, dtm_idx;
+ u64 reg, count = 0;
offset = snapshot ? CMN_DTM_PMEVCNTSR : CMN_DTM_PMEVCNT;
for_each_hw_dn(hw, dn, i) {
- struct arm_cmn_node *xp = arm_cmn_node_to_xp(dn);
- int dtm_idx = arm_cmn_get_index(hw->dtm_idx, i);
- u64 reg = readq_relaxed(xp->pmu_base + offset);
- u16 dtm_count = reg >> (dtm_idx * 16);
-
- count += dtm_count;
+ if (dtm != &cmn->dtms[dn->dtm]) {
+ dtm = &cmn->dtms[dn->dtm] + hw->dtm_offset;
+ reg = readq_relaxed(dtm->base + offset);
+ }
+ dtm_idx = arm_cmn_get_index(hw->dtm_idx, i);
+ count += (u16)(reg >> (dtm_idx * 16));
}
return count;
}
@@ -774,8 +1041,10 @@ static void arm_cmn_event_start(struct perf_event *event, int flags)
u64 mask = CMN_EVENT_WP_MASK(event);
for_each_hw_dn(hw, dn, i) {
- writeq_relaxed(val, dn->pmu_base + CMN_DTM_WPn_VAL(wp_idx));
- writeq_relaxed(mask, dn->pmu_base + CMN_DTM_WPn_MASK(wp_idx));
+ void __iomem *base = dn->pmu_base + CMN_DTM_OFFSET(hw->dtm_offset);
+
+ writeq_relaxed(val, base + CMN_DTM_WPn_VAL(wp_idx));
+ writeq_relaxed(mask, base + CMN_DTM_WPn_MASK(wp_idx));
}
} else for_each_hw_dn(hw, dn, i) {
int dtm_idx = arm_cmn_get_index(hw->dtm_idx, i);
@@ -800,8 +1069,10 @@ static void arm_cmn_event_stop(struct perf_event *event, int flags)
int wp_idx = arm_cmn_wp_idx(event);
for_each_hw_dn(hw, dn, i) {
- writeq_relaxed(0, dn->pmu_base + CMN_DTM_WPn_MASK(wp_idx));
- writeq_relaxed(~0ULL, dn->pmu_base + CMN_DTM_WPn_VAL(wp_idx));
+ void __iomem *base = dn->pmu_base + CMN_DTM_OFFSET(hw->dtm_offset);
+
+ writeq_relaxed(0, base + CMN_DTM_WPn_MASK(wp_idx));
+ writeq_relaxed(~0ULL, base + CMN_DTM_WPn_VAL(wp_idx));
}
} else for_each_hw_dn(hw, dn, i) {
int dtm_idx = arm_cmn_get_index(hw->dtm_idx, i);
@@ -814,14 +1085,15 @@ static void arm_cmn_event_stop(struct perf_event *event, int flags)
}
struct arm_cmn_val {
- u8 dtm_count[CMN_MAX_XPS];
- u8 occupid[CMN_MAX_XPS];
- u8 wp[CMN_MAX_XPS][4];
+ u8 dtm_count[CMN_MAX_DTMS];
+ u8 occupid[CMN_MAX_DTMS];
+ u8 wp[CMN_MAX_DTMS][4];
int dtc_count;
bool cycles;
};
-static void arm_cmn_val_add_event(struct arm_cmn_val *val, struct perf_event *event)
+static void arm_cmn_val_add_event(struct arm_cmn *cmn, struct arm_cmn_val *val,
+ struct perf_event *event)
{
struct arm_cmn_hw_event *hw = to_cmn_hw(event);
struct arm_cmn_node *dn;
@@ -839,33 +1111,33 @@ static void arm_cmn_val_add_event(struct arm_cmn_val *val, struct perf_event *ev
}
val->dtc_count++;
- if (arm_cmn_is_occup_event(type, CMN_EVENT_EVENTID(event)))
+ if (arm_cmn_is_occup_event(cmn->model, type, CMN_EVENT_EVENTID(event)))
occupid = CMN_EVENT_OCCUPID(event) + 1;
else
occupid = 0;
for_each_hw_dn(hw, dn, i) {
- int wp_idx, xp = arm_cmn_node_to_xp(dn)->logid;
+ int wp_idx, dtm = dn->dtm;
- val->dtm_count[xp]++;
- val->occupid[xp] = occupid;
+ val->dtm_count[dtm]++;
+ val->occupid[dtm] = occupid;
if (type != CMN_TYPE_WP)
continue;
wp_idx = arm_cmn_wp_idx(event);
- val->wp[xp][wp_idx] = CMN_EVENT_WP_COMBINE(event) + 1;
+ val->wp[dtm][wp_idx] = CMN_EVENT_WP_COMBINE(event) + 1;
}
}
-static int arm_cmn_validate_group(struct perf_event *event)
+static int arm_cmn_validate_group(struct arm_cmn *cmn, struct perf_event *event)
{
struct arm_cmn_hw_event *hw = to_cmn_hw(event);
struct arm_cmn_node *dn;
struct perf_event *sibling, *leader = event->group_leader;
enum cmn_node_type type;
- struct arm_cmn_val val;
- int i;
+ struct arm_cmn_val *val;
+ int i, ret = -EINVAL;
u8 occupid;
if (leader == event)
@@ -874,54 +1146,61 @@ static int arm_cmn_validate_group(struct perf_event *event)
if (event->pmu != leader->pmu && !is_software_event(leader))
return -EINVAL;
- memset(&val, 0, sizeof(val));
+ val = kzalloc(sizeof(*val), GFP_KERNEL);
+ if (!val)
+ return -ENOMEM;
- arm_cmn_val_add_event(&val, leader);
+ arm_cmn_val_add_event(cmn, val, leader);
for_each_sibling_event(sibling, leader)
- arm_cmn_val_add_event(&val, sibling);
+ arm_cmn_val_add_event(cmn, val, sibling);
type = CMN_EVENT_TYPE(event);
- if (type == CMN_TYPE_DTC)
- return val.cycles ? -EINVAL : 0;
+ if (type == CMN_TYPE_DTC) {
+ ret = val->cycles ? -EINVAL : 0;
+ goto done;
+ }
- if (val.dtc_count == CMN_DT_NUM_COUNTERS)
- return -EINVAL;
+ if (val->dtc_count == CMN_DT_NUM_COUNTERS)
+ goto done;
- if (arm_cmn_is_occup_event(type, CMN_EVENT_EVENTID(event)))
+ if (arm_cmn_is_occup_event(cmn->model, type, CMN_EVENT_EVENTID(event)))
occupid = CMN_EVENT_OCCUPID(event) + 1;
else
occupid = 0;
for_each_hw_dn(hw, dn, i) {
- int wp_idx, wp_cmb, xp = arm_cmn_node_to_xp(dn)->logid;
+ int wp_idx, wp_cmb, dtm = dn->dtm;
- if (val.dtm_count[xp] == CMN_DTM_NUM_COUNTERS)
- return -EINVAL;
+ if (val->dtm_count[dtm] == CMN_DTM_NUM_COUNTERS)
+ goto done;
- if (occupid && val.occupid[xp] && occupid != val.occupid[xp])
- return -EINVAL;
+ if (occupid && val->occupid[dtm] && occupid != val->occupid[dtm])
+ goto done;
if (type != CMN_TYPE_WP)
continue;
wp_idx = arm_cmn_wp_idx(event);
- if (val.wp[xp][wp_idx])
- return -EINVAL;
+ if (val->wp[dtm][wp_idx])
+ goto done;
- wp_cmb = val.wp[xp][wp_idx ^ 1];
+ wp_cmb = val->wp[dtm][wp_idx ^ 1];
if (wp_cmb && wp_cmb != CMN_EVENT_WP_COMBINE(event) + 1)
- return -EINVAL;
+ goto done;
}
- return 0;
+ ret = 0;
+done:
+ kfree(val);
+ return ret;
}
static int arm_cmn_event_init(struct perf_event *event)
{
struct arm_cmn *cmn = to_cmn(event->pmu);
struct arm_cmn_hw_event *hw = to_cmn_hw(event);
+ struct arm_cmn_node *dn;
enum cmn_node_type type;
- unsigned int i;
bool bynodeid;
u16 nodeid, eventid;
@@ -947,38 +1226,37 @@ static int arm_cmn_event_init(struct perf_event *event)
eventid = CMN_EVENT_EVENTID(event);
if (eventid != CMN_WP_UP && eventid != CMN_WP_DOWN)
return -EINVAL;
+ /* ...but the DTM may depend on which port we're watching */
+ if (cmn->multi_dtm)
+ hw->dtm_offset = CMN_EVENT_WP_DEV_SEL(event) / 2;
}
bynodeid = CMN_EVENT_BYNODEID(event);
nodeid = CMN_EVENT_NODEID(event);
hw->dn = arm_cmn_node(cmn, type);
- for (i = hw->dn - cmn->dns; i < cmn->num_dns && cmn->dns[i].type == type; i++) {
- if (!bynodeid) {
- hw->num_dns++;
- } else if (cmn->dns[i].id != nodeid) {
+ if (!hw->dn)
+ return -EINVAL;
+ for (dn = hw->dn; dn->type == type; dn++) {
+ if (bynodeid && dn->id != nodeid) {
hw->dn++;
- } else {
- hw->num_dns = 1;
- break;
+ continue;
}
+ hw->dtcs_used |= arm_cmn_node_to_xp(cmn, dn)->dtc;
+ hw->num_dns++;
+ if (bynodeid)
+ break;
}
if (!hw->num_dns) {
- int bits = arm_cmn_xyidbits(cmn);
+ struct arm_cmn_nodeid nid = arm_cmn_nid(cmn, nodeid);
dev_dbg(cmn->dev, "invalid node 0x%x (%d,%d,%d,%d) type 0x%x\n",
- nodeid, CMN_NODEID_X(nodeid, bits), CMN_NODEID_Y(nodeid, bits),
- CMN_NODEID_PID(nodeid), CMN_NODEID_DEVID(nodeid), type);
+ nodeid, nid.x, nid.y, nid.port, nid.dev, type);
return -EINVAL;
}
- /*
- * By assuming events count in all DTC domains, we cunningly avoid
- * needing to know anything about how XPs are assigned to domains.
- */
- hw->dtcs_used = (1U << cmn->num_dtcs) - 1;
- return arm_cmn_validate_group(event);
+ return arm_cmn_validate_group(cmn, event);
}
static void arm_cmn_event_clear(struct arm_cmn *cmn, struct perf_event *event,
@@ -988,17 +1266,17 @@ static void arm_cmn_event_clear(struct arm_cmn *cmn, struct perf_event *event,
enum cmn_node_type type = CMN_EVENT_TYPE(event);
while (i--) {
- struct arm_cmn_node *xp = arm_cmn_node_to_xp(hw->dn + i);
+ struct arm_cmn_dtm *dtm = &cmn->dtms[hw->dn[i].dtm] + hw->dtm_offset;
unsigned int dtm_idx = arm_cmn_get_index(hw->dtm_idx, i);
if (type == CMN_TYPE_WP)
- hw->dn[i].wp_event[arm_cmn_wp_idx(event)] = -1;
+ dtm->wp_event[arm_cmn_wp_idx(event)] = -1;
- if (arm_cmn_is_occup_event(type, CMN_EVENT_EVENTID(event)))
+ if (arm_cmn_is_occup_event(cmn->model, type, CMN_EVENT_EVENTID(event)))
hw->dn[i].occupid_count--;
- xp->pmu_config_low &= ~CMN__PMEVCNT_PAIRED(dtm_idx);
- writel_relaxed(xp->pmu_config_low, xp->pmu_base + CMN_DTM_PMU_CONFIG);
+ dtm->pmu_config_low &= ~CMN__PMEVCNT_PAIRED(dtm_idx);
+ writel_relaxed(dtm->pmu_config_low, dtm->base + CMN_DTM_PMU_CONFIG);
}
memset(hw->dtm_idx, 0, sizeof(hw->dtm_idx));
@@ -1040,12 +1318,12 @@ static int arm_cmn_event_add(struct perf_event *event, int flags)
/* ...then the local counters to feed it. */
for_each_hw_dn(hw, dn, i) {
- struct arm_cmn_node *xp = arm_cmn_node_to_xp(dn);
+ struct arm_cmn_dtm *dtm = &cmn->dtms[dn->dtm] + hw->dtm_offset;
unsigned int dtm_idx, shift;
u64 reg;
dtm_idx = 0;
- while (xp->pmu_config_low & CMN__PMEVCNT_PAIRED(dtm_idx))
+ while (dtm->pmu_config_low & CMN__PMEVCNT_PAIRED(dtm_idx))
if (++dtm_idx == CMN_DTM_NUM_COUNTERS)
goto free_dtms;
@@ -1055,26 +1333,28 @@ static int arm_cmn_event_add(struct perf_event *event, int flags)
int tmp, wp_idx = arm_cmn_wp_idx(event);
u32 cfg = arm_cmn_wp_config(event);
- if (dn->wp_event[wp_idx] >= 0)
+ if (dtm->wp_event[wp_idx] >= 0)
goto free_dtms;
- tmp = dn->wp_event[wp_idx ^ 1];
+ tmp = dtm->wp_event[wp_idx ^ 1];
if (tmp >= 0 && CMN_EVENT_WP_COMBINE(event) !=
CMN_EVENT_WP_COMBINE(dtc->counters[tmp]))
goto free_dtms;
input_sel = CMN__PMEVCNT0_INPUT_SEL_WP + wp_idx;
- dn->wp_event[wp_idx] = dtc_idx;
- writel_relaxed(cfg, dn->pmu_base + CMN_DTM_WPn_CONFIG(wp_idx));
+ dtm->wp_event[wp_idx] = dtc_idx;
+ writel_relaxed(cfg, dtm->base + CMN_DTM_WPn_CONFIG(wp_idx));
} else {
- unsigned int port = CMN_NODEID_PID(dn->id);
- unsigned int dev = CMN_NODEID_DEVID(dn->id);
+ struct arm_cmn_nodeid nid = arm_cmn_nid(cmn, dn->id);
+
+ if (cmn->multi_dtm)
+ nid.port %= 2;
input_sel = CMN__PMEVCNT0_INPUT_SEL_DEV + dtm_idx +
- (port << 4) + (dev << 2);
+ (nid.port << 4) + (nid.dev << 2);
- if (arm_cmn_is_occup_event(type, CMN_EVENT_EVENTID(event))) {
- int occupid = CMN_EVENT_OCCUPID(event);
+ if (arm_cmn_is_occup_event(cmn->model, type, CMN_EVENT_EVENTID(event))) {
+ u8 occupid = CMN_EVENT_OCCUPID(event);
if (dn->occupid_count == 0) {
dn->occupid_val = occupid;
@@ -1089,13 +1369,13 @@ static int arm_cmn_event_add(struct perf_event *event, int flags)
arm_cmn_set_index(hw->dtm_idx, i, dtm_idx);
- xp->input_sel[dtm_idx] = input_sel;
+ dtm->input_sel[dtm_idx] = input_sel;
shift = CMN__PMEVCNTn_GLOBAL_NUM_SHIFT(dtm_idx);
- xp->pmu_config_low &= ~(CMN__PMEVCNT0_GLOBAL_NUM << shift);
- xp->pmu_config_low |= FIELD_PREP(CMN__PMEVCNT0_GLOBAL_NUM, dtc_idx) << shift;
- xp->pmu_config_low |= CMN__PMEVCNT_PAIRED(dtm_idx);
- reg = (u64)le32_to_cpu(xp->pmu_config_high) << 32 | xp->pmu_config_low;
- writeq_relaxed(reg, xp->pmu_base + CMN_DTM_PMU_CONFIG);
+ dtm->pmu_config_low &= ~(CMN__PMEVCNT0_GLOBAL_NUM << shift);
+ dtm->pmu_config_low |= FIELD_PREP(CMN__PMEVCNT0_GLOBAL_NUM, dtc_idx) << shift;
+ dtm->pmu_config_low |= CMN__PMEVCNT_PAIRED(dtm_idx);
+ reg = (u64)le32_to_cpu(dtm->pmu_config_high) << 32 | dtm->pmu_config_low;
+ writeq_relaxed(reg, dtm->base + CMN_DTM_PMU_CONFIG);
}
/* Go go go! */
@@ -1147,23 +1427,47 @@ static int arm_cmn_commit_txn(struct pmu *pmu)
return 0;
}
-static int arm_cmn_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node)
+static void arm_cmn_migrate(struct arm_cmn *cmn, unsigned int cpu)
+{
+ unsigned int i;
+
+ perf_pmu_migrate_context(&cmn->pmu, cmn->cpu, cpu);
+ for (i = 0; i < cmn->num_dtcs; i++)
+ irq_set_affinity(cmn->dtc[i].irq, cpumask_of(cpu));
+ cmn->cpu = cpu;
+}
+
+static int arm_cmn_pmu_online_cpu(unsigned int cpu, struct hlist_node *cpuhp_node)
{
struct arm_cmn *cmn;
- unsigned int i, target;
+ int node;
- cmn = hlist_entry_safe(node, struct arm_cmn, cpuhp_node);
+ cmn = hlist_entry_safe(cpuhp_node, struct arm_cmn, cpuhp_node);
+ node = dev_to_node(cmn->dev);
+ if (node != NUMA_NO_NODE && cpu_to_node(cmn->cpu) != node && cpu_to_node(cpu) == node)
+ arm_cmn_migrate(cmn, cpu);
+ return 0;
+}
+
+static int arm_cmn_pmu_offline_cpu(unsigned int cpu, struct hlist_node *cpuhp_node)
+{
+ struct arm_cmn *cmn;
+ unsigned int target;
+ int node;
+ cpumask_t mask;
+
+ cmn = hlist_entry_safe(cpuhp_node, struct arm_cmn, cpuhp_node);
if (cpu != cmn->cpu)
return 0;
- target = cpumask_any_but(cpu_online_mask, cpu);
- if (target >= nr_cpu_ids)
- return 0;
-
- perf_pmu_migrate_context(&cmn->pmu, cpu, target);
- for (i = 0; i < cmn->num_dtcs; i++)
- irq_set_affinity(cmn->dtc[i].irq, cpumask_of(target));
- cmn->cpu = target;
+ node = dev_to_node(cmn->dev);
+ if (cpumask_and(&mask, cpumask_of_node(node), cpu_online_mask) &&
+ cpumask_andnot(&mask, &mask, cpumask_of(cpu)))
+ target = cpumask_any(&mask);
+ else
+ target = cpumask_any_but(cpu_online_mask, cpu);
+ if (target < nr_cpu_ids)
+ arm_cmn_migrate(cmn, target);
return 0;
}
@@ -1231,23 +1535,22 @@ static int arm_cmn_init_irqs(struct arm_cmn *cmn)
return 0;
}
-static void arm_cmn_init_dtm(struct arm_cmn_node *xp)
+static void arm_cmn_init_dtm(struct arm_cmn_dtm *dtm, struct arm_cmn_node *xp, int idx)
{
int i;
+ dtm->base = xp->pmu_base + CMN_DTM_OFFSET(idx);
+ dtm->pmu_config_low = CMN_DTM_PMU_CONFIG_PMU_EN;
for (i = 0; i < 4; i++) {
- xp->wp_event[i] = -1;
- writeq_relaxed(0, xp->pmu_base + CMN_DTM_WPn_MASK(i));
- writeq_relaxed(~0ULL, xp->pmu_base + CMN_DTM_WPn_VAL(i));
+ dtm->wp_event[i] = -1;
+ writeq_relaxed(0, dtm->base + CMN_DTM_WPn_MASK(i));
+ writeq_relaxed(~0ULL, dtm->base + CMN_DTM_WPn_VAL(i));
}
- xp->pmu_config_low = CMN_DTM_PMU_CONFIG_PMU_EN;
- xp->dtc = -1;
}
static int arm_cmn_init_dtc(struct arm_cmn *cmn, struct arm_cmn_node *dn, int idx)
{
struct arm_cmn_dtc *dtc = cmn->dtc + idx;
- struct arm_cmn_node *xp;
dtc->base = dn->pmu_base - CMN_PMU_OFFSET;
dtc->irq = platform_get_irq(to_platform_device(cmn->dev), idx);
@@ -1258,10 +1561,6 @@ static int arm_cmn_init_dtc(struct arm_cmn *cmn, struct arm_cmn_node *dn, int id
writel_relaxed(0x1ff, dtc->base + CMN_DT_PMOVSR_CLR);
writel_relaxed(CMN_DT_PMCR_OVFL_INTR_EN, dtc->base + CMN_DT_PMCR);
- /* We do at least know that a DTC's XP must be in that DTC's domain */
- xp = arm_cmn_node_to_xp(dn);
- xp->dtc = idx;
-
return 0;
}
@@ -1278,8 +1577,9 @@ static int arm_cmn_node_cmp(const void *a, const void *b)
static int arm_cmn_init_dtcs(struct arm_cmn *cmn)
{
- struct arm_cmn_node *dn;
+ struct arm_cmn_node *dn, *xp;
int dtc_idx = 0;
+ u8 dtcs_present = (1 << cmn->num_dtcs) - 1;
cmn->dtc = devm_kcalloc(cmn->dev, cmn->num_dtcs, sizeof(cmn->dtc[0]), GFP_KERNEL);
if (!cmn->dtc)
@@ -1289,14 +1589,26 @@ static int arm_cmn_init_dtcs(struct arm_cmn *cmn)
cmn->xps = arm_cmn_node(cmn, CMN_TYPE_XP);
- for (dn = cmn->dns; dn < cmn->dns + cmn->num_dns; dn++) {
- if (dn->type != CMN_TYPE_XP)
- arm_cmn_init_node_to_xp(cmn, dn);
- else if (cmn->num_dtcs == 1)
- dn->dtc = 0;
+ for (dn = cmn->dns; dn->type; dn++) {
+ if (dn->type == CMN_TYPE_XP) {
+ dn->dtc &= dtcs_present;
+ continue;
+ }
- if (dn->type == CMN_TYPE_DTC)
- arm_cmn_init_dtc(cmn, dn, dtc_idx++);
+ xp = arm_cmn_node_to_xp(cmn, dn);
+ dn->dtm = xp->dtm;
+ if (cmn->multi_dtm)
+ dn->dtm += arm_cmn_nid(cmn, dn->id).port / 2;
+
+ if (dn->type == CMN_TYPE_DTC) {
+ int err;
+ /* We do at least know that a DTC's XP must be in that DTC's domain */
+ if (xp->dtc == 0xf)
+ xp->dtc = 1 << dtc_idx;
+ err = arm_cmn_init_dtc(cmn, dn, dtc_idx++);
+ if (err)
+ return err;
+ }
/* To the PMU, RN-Ds don't add anything over RN-Is, so smoosh them together */
if (dn->type == CMN_TYPE_RND)
@@ -1335,20 +1647,26 @@ static int arm_cmn_discover(struct arm_cmn *cmn, unsigned int rgn_offset)
{
void __iomem *cfg_region;
struct arm_cmn_node cfg, *dn;
+ struct arm_cmn_dtm *dtm;
u16 child_count, child_poff;
u32 xp_offset[CMN_MAX_XPS];
u64 reg;
int i, j;
-
- cfg_region = cmn->base + rgn_offset;
- reg = readl_relaxed(cfg_region + CMN_CFGM_PERIPH_ID_2);
- cmn->rev = FIELD_GET(CMN_CFGM_PID2_REVISION, reg);
- dev_dbg(cmn->dev, "periph_id_2 revision: %d\n", cmn->rev);
+ size_t sz;
arm_cmn_init_node_info(cmn, rgn_offset, &cfg);
if (cfg.type != CMN_TYPE_CFG)
return -ENODEV;
+ cfg_region = cmn->base + rgn_offset;
+ reg = readl_relaxed(cfg_region + CMN_CFGM_PERIPH_ID_2);
+ cmn->rev = FIELD_GET(CMN_CFGM_PID2_REVISION, reg);
+
+ reg = readq_relaxed(cfg_region + CMN_CFGM_INFO_GLOBAL);
+ cmn->multi_dtm = reg & CMN_INFO_MULTIPLE_DTM_EN;
+ cmn->rsp_vc_num = FIELD_GET(CMN_INFO_RSP_VC_NUM, reg);
+ cmn->dat_vc_num = FIELD_GET(CMN_INFO_DAT_VC_NUM, reg);
+
reg = readq_relaxed(cfg_region + CMN_CHILD_INFO);
child_count = FIELD_GET(CMN_CI_CHILD_COUNT, reg);
child_poff = FIELD_GET(CMN_CI_CHILD_PTR_OFFSET, reg);
@@ -1365,20 +1683,28 @@ static int arm_cmn_discover(struct arm_cmn *cmn, unsigned int rgn_offset)
cmn->num_dns += FIELD_GET(CMN_CI_CHILD_COUNT, reg);
}
- /* Cheeky +1 to help terminate pointer-based iteration */
- cmn->dns = devm_kcalloc(cmn->dev, cmn->num_dns + 1,
- sizeof(*cmn->dns), GFP_KERNEL);
- if (!cmn->dns)
+ /* Cheeky +1 to help terminate pointer-based iteration later */
+ dn = devm_kcalloc(cmn->dev, cmn->num_dns + 1, sizeof(*dn), GFP_KERNEL);
+ if (!dn)
+ return -ENOMEM;
+
+ /* Initial safe upper bound on DTMs for any possible mesh layout */
+ i = cmn->num_xps;
+ if (cmn->multi_dtm)
+ i += cmn->num_xps + 1;
+ dtm = devm_kcalloc(cmn->dev, i, sizeof(*dtm), GFP_KERNEL);
+ if (!dtm)
return -ENOMEM;
/* Pass 2: now we can actually populate the nodes */
- dn = cmn->dns;
+ cmn->dns = dn;
+ cmn->dtms = dtm;
for (i = 0; i < cmn->num_xps; i++) {
void __iomem *xp_region = cmn->base + xp_offset[i];
struct arm_cmn_node *xp = dn++;
+ unsigned int xp_ports = 0;
arm_cmn_init_node_info(cmn, xp_offset[i], xp);
- arm_cmn_init_dtm(xp);
/*
* Thanks to the order in which XP logical IDs seem to be
* assigned, we can handily infer the mesh X dimension by
@@ -1388,6 +1714,40 @@ static int arm_cmn_discover(struct arm_cmn *cmn, unsigned int rgn_offset)
if (xp->id == (1 << 3))
cmn->mesh_x = xp->logid;
+ if (cmn->model == CMN600)
+ xp->dtc = 0xf;
+ else
+ xp->dtc = 1 << readl_relaxed(xp_region + CMN_DTM_UNIT_INFO);
+
+ xp->dtm = dtm - cmn->dtms;
+ arm_cmn_init_dtm(dtm++, xp, 0);
+ /*
+ * Keeping track of connected ports will let us filter out
+ * unnecessary XP events easily. We can also reliably infer the
+ * "extra device ports" configuration for the node ID format
+ * from this, since in that case we will see at least one XP
+ * with port 2 connected, for the HN-D.
+ */
+ if (readq_relaxed(xp_region + CMN_MXP__CONNECT_INFO_P0))
+ xp_ports |= BIT(0);
+ if (readq_relaxed(xp_region + CMN_MXP__CONNECT_INFO_P1))
+ xp_ports |= BIT(1);
+ if (readq_relaxed(xp_region + CMN_MXP__CONNECT_INFO_P2))
+ xp_ports |= BIT(2);
+ if (readq_relaxed(xp_region + CMN_MXP__CONNECT_INFO_P3))
+ xp_ports |= BIT(3);
+ if (readq_relaxed(xp_region + CMN_MXP__CONNECT_INFO_P4))
+ xp_ports |= BIT(4);
+ if (readq_relaxed(xp_region + CMN_MXP__CONNECT_INFO_P5))
+ xp_ports |= BIT(5);
+
+ if (cmn->multi_dtm && (xp_ports & 0xc))
+ arm_cmn_init_dtm(dtm++, xp, 1);
+ if (cmn->multi_dtm && (xp_ports & 0x30))
+ arm_cmn_init_dtm(dtm++, xp, 2);
+
+ cmn->ports_used |= xp_ports;
+
reg = readq_relaxed(xp_region + CMN_CHILD_INFO);
child_count = FIELD_GET(CMN_CI_CHILD_COUNT, reg);
child_poff = FIELD_GET(CMN_CI_CHILD_PTR_OFFSET, reg);
@@ -1422,11 +1782,14 @@ static int arm_cmn_discover(struct arm_cmn *cmn, unsigned int rgn_offset)
case CMN_TYPE_SBSX:
case CMN_TYPE_RNI:
case CMN_TYPE_RND:
+ case CMN_TYPE_MTSX:
case CMN_TYPE_CXRA:
case CMN_TYPE_CXHA:
dn++;
break;
/* Nothing to see here */
+ case CMN_TYPE_MPAM_S:
+ case CMN_TYPE_MPAM_NS:
case CMN_TYPE_RNSAM:
case CMN_TYPE_CXLA:
break;
@@ -1441,6 +1804,16 @@ static int arm_cmn_discover(struct arm_cmn *cmn, unsigned int rgn_offset)
/* Correct for any nodes we skipped */
cmn->num_dns = dn - cmn->dns;
+ sz = (void *)(dn + 1) - (void *)cmn->dns;
+ dn = devm_krealloc(cmn->dev, cmn->dns, sz, GFP_KERNEL);
+ if (dn)
+ cmn->dns = dn;
+
+ sz = (void *)dtm - (void *)cmn->dtms;
+ dtm = devm_krealloc(cmn->dev, cmn->dtms, sz, GFP_KERNEL);
+ if (dtm)
+ cmn->dtms = dtm;
+
/*
* If mesh_x wasn't set during discovery then we never saw
* an XP at (0,1), thus we must have an Nx1 configuration.
@@ -1449,13 +1822,20 @@ static int arm_cmn_discover(struct arm_cmn *cmn, unsigned int rgn_offset)
cmn->mesh_x = cmn->num_xps;
cmn->mesh_y = cmn->num_xps / cmn->mesh_x;
- dev_dbg(cmn->dev, "mesh %dx%d, ID width %d\n",
- cmn->mesh_x, cmn->mesh_y, arm_cmn_xyidbits(cmn));
+ /* 1x1 config plays havoc with XP event encodings */
+ if (cmn->num_xps == 1)
+ dev_warn(cmn->dev, "1x1 config not fully supported, translate XP events manually\n");
+
+ dev_dbg(cmn->dev, "model %d, periph_id_2 revision %d\n", cmn->model, cmn->rev);
+ reg = cmn->ports_used;
+ dev_dbg(cmn->dev, "mesh %dx%d, ID width %d, ports %6pbl%s\n",
+ cmn->mesh_x, cmn->mesh_y, arm_cmn_xyidbits(cmn), ®,
+ cmn->multi_dtm ? ", multi-DTM" : "");
return 0;
}
-static int arm_cmn_acpi_probe(struct platform_device *pdev, struct arm_cmn *cmn)
+static int arm_cmn600_acpi_probe(struct platform_device *pdev, struct arm_cmn *cmn)
{
struct resource *cfg, *root;
@@ -1482,21 +1862,11 @@ static int arm_cmn_acpi_probe(struct platform_device *pdev, struct arm_cmn *cmn)
return root->start - cfg->start;
}
-static int arm_cmn_of_probe(struct platform_device *pdev, struct arm_cmn *cmn)
+static int arm_cmn600_of_probe(struct device_node *np)
{
- struct device_node *np = pdev->dev.of_node;
u32 rootnode;
- int ret;
- cmn->base = devm_platform_ioremap_resource(pdev, 0);
- if (IS_ERR(cmn->base))
- return PTR_ERR(cmn->base);
-
- ret = of_property_read_u32(np, "arm,root-node", &rootnode);
- if (ret)
- return ret;
-
- return rootnode;
+ return of_property_read_u32(np, "arm,root-node", &rootnode) ?: rootnode;
}
static int arm_cmn_probe(struct platform_device *pdev)
@@ -1504,19 +1874,26 @@ static int arm_cmn_probe(struct platform_device *pdev)
struct arm_cmn *cmn;
const char *name;
static atomic_t id;
- int err, rootnode;
+ int err, rootnode, this_id;
cmn = devm_kzalloc(&pdev->dev, sizeof(*cmn), GFP_KERNEL);
if (!cmn)
return -ENOMEM;
cmn->dev = &pdev->dev;
+ cmn->model = (unsigned long)device_get_match_data(cmn->dev);
platform_set_drvdata(pdev, cmn);
- if (has_acpi_companion(cmn->dev))
- rootnode = arm_cmn_acpi_probe(pdev, cmn);
- else
- rootnode = arm_cmn_of_probe(pdev, cmn);
+ if (cmn->model == CMN600 && has_acpi_companion(cmn->dev)) {
+ rootnode = arm_cmn600_acpi_probe(pdev, cmn);
+ } else {
+ rootnode = 0;
+ cmn->base = devm_platform_ioremap_resource(pdev, 0);
+ if (IS_ERR(cmn->base))
+ return PTR_ERR(cmn->base);
+ if (cmn->model == CMN600)
+ rootnode = arm_cmn600_of_probe(pdev->dev.of_node);
+ }
if (rootnode < 0)
return rootnode;
@@ -1532,7 +1909,7 @@ static int arm_cmn_probe(struct platform_device *pdev)
if (err)
return err;
- cmn->cpu = raw_smp_processor_id();
+ cmn->cpu = cpumask_local_spread(0, dev_to_node(cmn->dev));
cmn->pmu = (struct pmu) {
.module = THIS_MODULE,
.attr_groups = arm_cmn_attr_groups,
@@ -1551,7 +1928,8 @@ static int arm_cmn_probe(struct platform_device *pdev)
.cancel_txn = arm_cmn_end_txn,
};
- name = devm_kasprintf(cmn->dev, GFP_KERNEL, "arm_cmn_%d", atomic_fetch_inc(&id));
+ this_id = atomic_fetch_inc(&id);
+ name = devm_kasprintf(cmn->dev, GFP_KERNEL, "arm_cmn_%d", this_id);
if (!name)
return -ENOMEM;
@@ -1561,7 +1939,10 @@ static int arm_cmn_probe(struct platform_device *pdev)
err = perf_pmu_register(&cmn->pmu, name, -1);
if (err)
- cpuhp_state_remove_instance(arm_cmn_hp_state, &cmn->cpuhp_node);
+ cpuhp_state_remove_instance_nocalls(arm_cmn_hp_state, &cmn->cpuhp_node);
+ else
+ arm_cmn_debugfs_init(cmn, this_id);
+
return err;
}
@@ -1572,13 +1953,15 @@ static int arm_cmn_remove(struct platform_device *pdev)
writel_relaxed(0, cmn->dtc[0].base + CMN_DT_DTC_CTL);
perf_pmu_unregister(&cmn->pmu);
- cpuhp_state_remove_instance(arm_cmn_hp_state, &cmn->cpuhp_node);
+ cpuhp_state_remove_instance_nocalls(arm_cmn_hp_state, &cmn->cpuhp_node);
+ debugfs_remove(cmn->debug);
return 0;
}
#ifdef CONFIG_OF
static const struct of_device_id arm_cmn_of_match[] = {
- { .compatible = "arm,cmn-600", },
+ { .compatible = "arm,cmn-600", .data = (void *)CMN600 },
+ { .compatible = "arm,ci-700", .data = (void *)CI700 },
{}
};
MODULE_DEVICE_TABLE(of, arm_cmn_of_match);
@@ -1586,7 +1969,7 @@ MODULE_DEVICE_TABLE(of, arm_cmn_of_match);
#ifdef CONFIG_ACPI
static const struct acpi_device_id arm_cmn_acpi_match[] = {
- { "ARMHC600", },
+ { "ARMHC600", CMN600 },
{}
};
MODULE_DEVICE_TABLE(acpi, arm_cmn_acpi_match);
@@ -1607,15 +1990,20 @@ static int __init arm_cmn_init(void)
int ret;
ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN,
- "perf/arm/cmn:online", NULL,
+ "perf/arm/cmn:online",
+ arm_cmn_pmu_online_cpu,
arm_cmn_pmu_offline_cpu);
if (ret < 0)
return ret;
arm_cmn_hp_state = ret;
+ arm_cmn_debugfs = debugfs_create_dir("arm-cmn", NULL);
+
ret = platform_driver_register(&arm_cmn_driver);
- if (ret)
+ if (ret) {
cpuhp_remove_multi_state(arm_cmn_hp_state);
+ debugfs_remove(arm_cmn_debugfs);
+ }
return ret;
}
@@ -1623,6 +2011,7 @@ static void __exit arm_cmn_exit(void)
{
platform_driver_unregister(&arm_cmn_driver);
cpuhp_remove_multi_state(arm_cmn_hp_state);
+ debugfs_remove(arm_cmn_debugfs);
}
module_init(arm_cmn_init);
diff --git a/drivers/perf/arm_smmuv3_pmu.c b/drivers/perf/arm_smmuv3_pmu.c
index 2263488..1ae19f73 100644
--- a/drivers/perf/arm_smmuv3_pmu.c
+++ b/drivers/perf/arm_smmuv3_pmu.c
@@ -47,6 +47,7 @@
#include <linux/kernel.h>
#include <linux/list.h>
#include <linux/msi.h>
+#include <linux/of.h>
#include <linux/perf_event.h>
#include <linux/platform_device.h>
#include <linux/smp.h>
@@ -75,6 +76,10 @@
#define SMMU_PMCG_CR 0xE04
#define SMMU_PMCG_CR_ENABLE BIT(0)
#define SMMU_PMCG_IIDR 0xE08
+#define SMMU_PMCG_IIDR_PRODUCTID GENMASK(31, 20)
+#define SMMU_PMCG_IIDR_VARIANT GENMASK(19, 16)
+#define SMMU_PMCG_IIDR_REVISION GENMASK(15, 12)
+#define SMMU_PMCG_IIDR_IMPLEMENTER GENMASK(11, 0)
#define SMMU_PMCG_CEID0 0xE20
#define SMMU_PMCG_CEID1 0xE28
#define SMMU_PMCG_IRQ_CTRL 0xE50
@@ -83,6 +88,20 @@
#define SMMU_PMCG_IRQ_CFG1 0xE60
#define SMMU_PMCG_IRQ_CFG2 0xE64
+/* IMP-DEF ID registers */
+#define SMMU_PMCG_PIDR0 0xFE0
+#define SMMU_PMCG_PIDR0_PART_0 GENMASK(7, 0)
+#define SMMU_PMCG_PIDR1 0xFE4
+#define SMMU_PMCG_PIDR1_DES_0 GENMASK(7, 4)
+#define SMMU_PMCG_PIDR1_PART_1 GENMASK(3, 0)
+#define SMMU_PMCG_PIDR2 0xFE8
+#define SMMU_PMCG_PIDR2_REVISION GENMASK(7, 4)
+#define SMMU_PMCG_PIDR2_DES_1 GENMASK(2, 0)
+#define SMMU_PMCG_PIDR3 0xFEC
+#define SMMU_PMCG_PIDR3_REVAND GENMASK(7, 4)
+#define SMMU_PMCG_PIDR4 0xFD0
+#define SMMU_PMCG_PIDR4_DES_2 GENMASK(3, 0)
+
/* MSI config fields */
#define MSI_CFG0_ADDR_MASK GENMASK_ULL(51, 2)
#define MSI_CFG2_MEMATTR_DEVICE_nGnRE 0x1
@@ -754,6 +773,41 @@ static void smmu_pmu_get_acpi_options(struct smmu_pmu *smmu_pmu)
dev_notice(smmu_pmu->dev, "option mask 0x%x\n", smmu_pmu->options);
}
+static bool smmu_pmu_coresight_id_regs(struct smmu_pmu *smmu_pmu)
+{
+ return of_device_is_compatible(smmu_pmu->dev->of_node,
+ "arm,mmu-600-pmcg");
+}
+
+static void smmu_pmu_get_iidr(struct smmu_pmu *smmu_pmu)
+{
+ u32 iidr = readl_relaxed(smmu_pmu->reg_base + SMMU_PMCG_IIDR);
+
+ if (!iidr && smmu_pmu_coresight_id_regs(smmu_pmu)) {
+ u32 pidr0 = readl(smmu_pmu->reg_base + SMMU_PMCG_PIDR0);
+ u32 pidr1 = readl(smmu_pmu->reg_base + SMMU_PMCG_PIDR1);
+ u32 pidr2 = readl(smmu_pmu->reg_base + SMMU_PMCG_PIDR2);
+ u32 pidr3 = readl(smmu_pmu->reg_base + SMMU_PMCG_PIDR3);
+ u32 pidr4 = readl(smmu_pmu->reg_base + SMMU_PMCG_PIDR4);
+
+ u32 productid = FIELD_GET(SMMU_PMCG_PIDR0_PART_0, pidr0) |
+ (FIELD_GET(SMMU_PMCG_PIDR1_PART_1, pidr1) << 8);
+ u32 variant = FIELD_GET(SMMU_PMCG_PIDR2_REVISION, pidr2);
+ u32 revision = FIELD_GET(SMMU_PMCG_PIDR3_REVAND, pidr3);
+ u32 implementer =
+ FIELD_GET(SMMU_PMCG_PIDR1_DES_0, pidr1) |
+ (FIELD_GET(SMMU_PMCG_PIDR2_DES_1, pidr2) << 4) |
+ (FIELD_GET(SMMU_PMCG_PIDR4_DES_2, pidr4) << 8);
+
+ iidr = FIELD_PREP(SMMU_PMCG_IIDR_PRODUCTID, productid) |
+ FIELD_PREP(SMMU_PMCG_IIDR_VARIANT, variant) |
+ FIELD_PREP(SMMU_PMCG_IIDR_REVISION, revision) |
+ FIELD_PREP(SMMU_PMCG_IIDR_IMPLEMENTER, implementer);
+ }
+
+ smmu_pmu->iidr = iidr;
+}
+
static int smmu_pmu_probe(struct platform_device *pdev)
{
struct smmu_pmu *smmu_pmu;
@@ -825,7 +879,7 @@ static int smmu_pmu_probe(struct platform_device *pdev)
return err;
}
- smmu_pmu->iidr = readl_relaxed(smmu_pmu->reg_base + SMMU_PMCG_IIDR);
+ smmu_pmu_get_iidr(smmu_pmu);
name = devm_kasprintf(&pdev->dev, GFP_KERNEL, "smmuv3_pmcg_%llx",
(res_0->start) >> SMMU_PMCG_PA_SHIFT);
@@ -834,7 +888,8 @@ static int smmu_pmu_probe(struct platform_device *pdev)
return -EINVAL;
}
- smmu_pmu_get_acpi_options(smmu_pmu);
+ if (!dev->of_node)
+ smmu_pmu_get_acpi_options(smmu_pmu);
/* Pick one CPU to be the preferred one to use */
smmu_pmu->on_cpu = raw_smp_processor_id();
@@ -884,9 +939,18 @@ static void smmu_pmu_shutdown(struct platform_device *pdev)
smmu_pmu_disable(&smmu_pmu->pmu);
}
+#ifdef CONFIG_OF
+static const struct of_device_id smmu_pmu_of_match[] = {
+ { .compatible = "arm,smmu-v3-pmcg" },
+ {}
+};
+MODULE_DEVICE_TABLE(of, smmu_pmu_of_match);
+#endif
+
static struct platform_driver smmu_pmu_driver = {
.driver = {
.name = "arm-smmu-v3-pmcg",
+ .of_match_table = of_match_ptr(smmu_pmu_of_match),
.suppress_bind_attrs = true,
},
.probe = smmu_pmu_probe,
diff --git a/drivers/perf/hisilicon/Kconfig b/drivers/perf/hisilicon/Kconfig
index c5d1b701..5546218 100644
--- a/drivers/perf/hisilicon/Kconfig
+++ b/drivers/perf/hisilicon/Kconfig
@@ -5,3 +5,12 @@
help
Support for HiSilicon SoC L3 Cache performance monitor, Hydra Home
Agent performance monitor and DDR Controller performance monitor.
+
+config HISI_PCIE_PMU
+ tristate "HiSilicon PCIE PERF PMU"
+ depends on PCI && ARM64
+ help
+ Provide support for HiSilicon PCIe performance monitoring unit (PMU)
+ RCiEP devices.
+ Adds the PCIe PMU into perf events system for monitoring latency,
+ bandwidth etc.
diff --git a/drivers/perf/hisilicon/Makefile b/drivers/perf/hisilicon/Makefile
index 7643c9f..506ed39 100644
--- a/drivers/perf/hisilicon/Makefile
+++ b/drivers/perf/hisilicon/Makefile
@@ -2,3 +2,5 @@
obj-$(CONFIG_HISI_PMU) += hisi_uncore_pmu.o hisi_uncore_l3c_pmu.o \
hisi_uncore_hha_pmu.o hisi_uncore_ddrc_pmu.o hisi_uncore_sllc_pmu.o \
hisi_uncore_pa_pmu.o
+
+obj-$(CONFIG_HISI_PCIE_PMU) += hisi_pcie_pmu.o
diff --git a/drivers/perf/hisilicon/hisi_pcie_pmu.c b/drivers/perf/hisilicon/hisi_pcie_pmu.c
new file mode 100644
index 0000000..2177170
--- /dev/null
+++ b/drivers/perf/hisilicon/hisi_pcie_pmu.c
@@ -0,0 +1,948 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * This driver adds support for PCIe PMU RCiEP device. Related
+ * perf events are bandwidth, latency etc.
+ *
+ * Copyright (C) 2021 HiSilicon Limited
+ * Author: Qi Liu <liuqi115@huawei.com>
+ */
+#include <linux/bitfield.h>
+#include <linux/bitmap.h>
+#include <linux/bug.h>
+#include <linux/device.h>
+#include <linux/err.h>
+#include <linux/interrupt.h>
+#include <linux/irq.h>
+#include <linux/kernel.h>
+#include <linux/list.h>
+#include <linux/module.h>
+#include <linux/pci.h>
+#include <linux/perf_event.h>
+
+#define DRV_NAME "hisi_pcie_pmu"
+/* Define registers */
+#define HISI_PCIE_GLOBAL_CTRL 0x00
+#define HISI_PCIE_EVENT_CTRL 0x010
+#define HISI_PCIE_CNT 0x090
+#define HISI_PCIE_EXT_CNT 0x110
+#define HISI_PCIE_INT_STAT 0x150
+#define HISI_PCIE_INT_MASK 0x154
+#define HISI_PCIE_REG_BDF 0xfe0
+#define HISI_PCIE_REG_VERSION 0xfe4
+#define HISI_PCIE_REG_INFO 0xfe8
+
+/* Define command in HISI_PCIE_GLOBAL_CTRL */
+#define HISI_PCIE_GLOBAL_EN 0x01
+#define HISI_PCIE_GLOBAL_NONE 0
+
+/* Define command in HISI_PCIE_EVENT_CTRL */
+#define HISI_PCIE_EVENT_EN BIT_ULL(20)
+#define HISI_PCIE_RESET_CNT BIT_ULL(22)
+#define HISI_PCIE_INIT_SET BIT_ULL(34)
+#define HISI_PCIE_THR_EN BIT_ULL(26)
+#define HISI_PCIE_TARGET_EN BIT_ULL(32)
+#define HISI_PCIE_TRIG_EN BIT_ULL(52)
+
+/* Define offsets in HISI_PCIE_EVENT_CTRL */
+#define HISI_PCIE_EVENT_M GENMASK_ULL(15, 0)
+#define HISI_PCIE_THR_MODE_M GENMASK_ULL(27, 27)
+#define HISI_PCIE_THR_M GENMASK_ULL(31, 28)
+#define HISI_PCIE_TARGET_M GENMASK_ULL(52, 36)
+#define HISI_PCIE_TRIG_MODE_M GENMASK_ULL(53, 53)
+#define HISI_PCIE_TRIG_M GENMASK_ULL(59, 56)
+
+#define HISI_PCIE_MAX_COUNTERS 8
+#define HISI_PCIE_REG_STEP 8
+#define HISI_PCIE_THR_MAX_VAL 10
+#define HISI_PCIE_TRIG_MAX_VAL 10
+#define HISI_PCIE_MAX_PERIOD (GENMASK_ULL(63, 0))
+#define HISI_PCIE_INIT_VAL BIT_ULL(63)
+
+struct hisi_pcie_pmu {
+ struct perf_event *hw_events[HISI_PCIE_MAX_COUNTERS];
+ struct hlist_node node;
+ struct pci_dev *pdev;
+ struct pmu pmu;
+ void __iomem *base;
+ int irq;
+ u32 identifier;
+ /* Minimum and maximum BDF of root ports monitored by PMU */
+ u16 bdf_min;
+ u16 bdf_max;
+ int on_cpu;
+};
+
+struct hisi_pcie_reg_pair {
+ u16 lo;
+ u16 hi;
+};
+
+#define to_pcie_pmu(p) (container_of((p), struct hisi_pcie_pmu, pmu))
+#define GET_PCI_DEVFN(bdf) ((bdf) & 0xff)
+
+#define HISI_PCIE_PMU_FILTER_ATTR(_name, _config, _hi, _lo) \
+ static u64 hisi_pcie_get_##_name(struct perf_event *event) \
+ { \
+ return FIELD_GET(GENMASK(_hi, _lo), event->attr._config); \
+ } \
+
+HISI_PCIE_PMU_FILTER_ATTR(event, config, 16, 0);
+HISI_PCIE_PMU_FILTER_ATTR(thr_len, config1, 3, 0);
+HISI_PCIE_PMU_FILTER_ATTR(thr_mode, config1, 4, 4);
+HISI_PCIE_PMU_FILTER_ATTR(trig_len, config1, 8, 5);
+HISI_PCIE_PMU_FILTER_ATTR(trig_mode, config1, 9, 9);
+HISI_PCIE_PMU_FILTER_ATTR(port, config2, 15, 0);
+HISI_PCIE_PMU_FILTER_ATTR(bdf, config2, 31, 16);
+
+static ssize_t hisi_pcie_format_sysfs_show(struct device *dev, struct device_attribute *attr,
+ char *buf)
+{
+ struct dev_ext_attribute *eattr;
+
+ eattr = container_of(attr, struct dev_ext_attribute, attr);
+
+ return sysfs_emit(buf, "%s\n", (char *)eattr->var);
+}
+
+static ssize_t hisi_pcie_event_sysfs_show(struct device *dev, struct device_attribute *attr,
+ char *buf)
+{
+ struct perf_pmu_events_attr *pmu_attr =
+ container_of(attr, struct perf_pmu_events_attr, attr);
+
+ return sysfs_emit(buf, "config=0x%llx\n", pmu_attr->id);
+}
+
+#define HISI_PCIE_PMU_FORMAT_ATTR(_name, _format) \
+ (&((struct dev_ext_attribute[]){ \
+ { .attr = __ATTR(_name, 0444, hisi_pcie_format_sysfs_show, \
+ NULL), \
+ .var = (void *)_format } \
+ })[0].attr.attr)
+
+#define HISI_PCIE_PMU_EVENT_ATTR(_name, _id) \
+ PMU_EVENT_ATTR_ID(_name, hisi_pcie_event_sysfs_show, _id)
+
+static ssize_t cpumask_show(struct device *dev, struct device_attribute *attr, char *buf)
+{
+ struct hisi_pcie_pmu *pcie_pmu = to_pcie_pmu(dev_get_drvdata(dev));
+
+ return cpumap_print_to_pagebuf(true, buf, cpumask_of(pcie_pmu->on_cpu));
+}
+static DEVICE_ATTR_RO(cpumask);
+
+static ssize_t identifier_show(struct device *dev, struct device_attribute *attr, char *buf)
+{
+ struct hisi_pcie_pmu *pcie_pmu = to_pcie_pmu(dev_get_drvdata(dev));
+
+ return sysfs_emit(buf, "%#x\n", pcie_pmu->identifier);
+}
+static DEVICE_ATTR_RO(identifier);
+
+static ssize_t bus_show(struct device *dev, struct device_attribute *attr, char *buf)
+{
+ struct hisi_pcie_pmu *pcie_pmu = to_pcie_pmu(dev_get_drvdata(dev));
+
+ return sysfs_emit(buf, "%#04x\n", PCI_BUS_NUM(pcie_pmu->bdf_min));
+}
+static DEVICE_ATTR_RO(bus);
+
+static struct hisi_pcie_reg_pair
+hisi_pcie_parse_reg_value(struct hisi_pcie_pmu *pcie_pmu, u32 reg_off)
+{
+ u32 val = readl_relaxed(pcie_pmu->base + reg_off);
+ struct hisi_pcie_reg_pair regs = {
+ .lo = val,
+ .hi = val >> 16,
+ };
+
+ return regs;
+}
+
+/*
+ * Hardware counter and ext_counter work together for bandwidth, latency, bus
+ * utilization and buffer occupancy events. For example, RX memory write latency
+ * events(index = 0x0010), counter counts total delay cycles and ext_counter
+ * counts RX memory write PCIe packets number.
+ *
+ * As we don't want PMU driver to process these two data, "delay cycles" can
+ * be treated as an independent event(index = 0x0010), "RX memory write packets
+ * number" as another(index = 0x10010). BIT 16 is used to distinguish and 0-15
+ * bits are "real" event index, which can be used to set HISI_PCIE_EVENT_CTRL.
+ */
+#define EXT_COUNTER_IS_USED(idx) ((idx) & BIT(16))
+
+static u32 hisi_pcie_get_real_event(struct perf_event *event)
+{
+ return hisi_pcie_get_event(event) & GENMASK(15, 0);
+}
+
+static u32 hisi_pcie_pmu_get_offset(u32 offset, u32 idx)
+{
+ return offset + HISI_PCIE_REG_STEP * idx;
+}
+
+static u32 hisi_pcie_pmu_readl(struct hisi_pcie_pmu *pcie_pmu, u32 reg_offset,
+ u32 idx)
+{
+ u32 offset = hisi_pcie_pmu_get_offset(reg_offset, idx);
+
+ return readl_relaxed(pcie_pmu->base + offset);
+}
+
+static void hisi_pcie_pmu_writel(struct hisi_pcie_pmu *pcie_pmu, u32 reg_offset, u32 idx, u32 val)
+{
+ u32 offset = hisi_pcie_pmu_get_offset(reg_offset, idx);
+
+ writel_relaxed(val, pcie_pmu->base + offset);
+}
+
+static u64 hisi_pcie_pmu_readq(struct hisi_pcie_pmu *pcie_pmu, u32 reg_offset, u32 idx)
+{
+ u32 offset = hisi_pcie_pmu_get_offset(reg_offset, idx);
+
+ return readq_relaxed(pcie_pmu->base + offset);
+}
+
+static void hisi_pcie_pmu_writeq(struct hisi_pcie_pmu *pcie_pmu, u32 reg_offset, u32 idx, u64 val)
+{
+ u32 offset = hisi_pcie_pmu_get_offset(reg_offset, idx);
+
+ writeq_relaxed(val, pcie_pmu->base + offset);
+}
+
+static void hisi_pcie_pmu_config_filter(struct perf_event *event)
+{
+ struct hisi_pcie_pmu *pcie_pmu = to_pcie_pmu(event->pmu);
+ struct hw_perf_event *hwc = &event->hw;
+ u64 reg = HISI_PCIE_INIT_SET;
+ u64 port, trig_len, thr_len;
+
+ /* Config HISI_PCIE_EVENT_CTRL according to event. */
+ reg |= FIELD_PREP(HISI_PCIE_EVENT_M, hisi_pcie_get_real_event(event));
+
+ /* Config HISI_PCIE_EVENT_CTRL according to root port or EP device. */
+ port = hisi_pcie_get_port(event);
+ if (port)
+ reg |= FIELD_PREP(HISI_PCIE_TARGET_M, port);
+ else
+ reg |= HISI_PCIE_TARGET_EN |
+ FIELD_PREP(HISI_PCIE_TARGET_M, hisi_pcie_get_bdf(event));
+
+ /* Config HISI_PCIE_EVENT_CTRL according to trigger condition. */
+ trig_len = hisi_pcie_get_trig_len(event);
+ if (trig_len) {
+ reg |= FIELD_PREP(HISI_PCIE_TRIG_M, trig_len);
+ reg |= FIELD_PREP(HISI_PCIE_TRIG_MODE_M, hisi_pcie_get_trig_mode(event));
+ reg |= HISI_PCIE_TRIG_EN;
+ }
+
+ /* Config HISI_PCIE_EVENT_CTRL according to threshold condition. */
+ thr_len = hisi_pcie_get_thr_len(event);
+ if (thr_len) {
+ reg |= FIELD_PREP(HISI_PCIE_THR_M, thr_len);
+ reg |= FIELD_PREP(HISI_PCIE_THR_MODE_M, hisi_pcie_get_thr_mode(event));
+ reg |= HISI_PCIE_THR_EN;
+ }
+
+ hisi_pcie_pmu_writeq(pcie_pmu, HISI_PCIE_EVENT_CTRL, hwc->idx, reg);
+}
+
+static void hisi_pcie_pmu_clear_filter(struct perf_event *event)
+{
+ struct hisi_pcie_pmu *pcie_pmu = to_pcie_pmu(event->pmu);
+ struct hw_perf_event *hwc = &event->hw;
+
+ hisi_pcie_pmu_writeq(pcie_pmu, HISI_PCIE_EVENT_CTRL, hwc->idx, HISI_PCIE_INIT_SET);
+}
+
+static bool hisi_pcie_pmu_valid_requester_id(struct hisi_pcie_pmu *pcie_pmu, u32 bdf)
+{
+ struct pci_dev *root_port, *pdev;
+ u16 rp_bdf;
+
+ pdev = pci_get_domain_bus_and_slot(pci_domain_nr(pcie_pmu->pdev->bus), PCI_BUS_NUM(bdf),
+ GET_PCI_DEVFN(bdf));
+ if (!pdev)
+ return false;
+
+ root_port = pcie_find_root_port(pdev);
+ if (!root_port) {
+ pci_dev_put(pdev);
+ return false;
+ }
+
+ pci_dev_put(pdev);
+ rp_bdf = pci_dev_id(root_port);
+ return rp_bdf >= pcie_pmu->bdf_min && rp_bdf <= pcie_pmu->bdf_max;
+}
+
+static bool hisi_pcie_pmu_valid_filter(struct perf_event *event,
+ struct hisi_pcie_pmu *pcie_pmu)
+{
+ u32 requester_id = hisi_pcie_get_bdf(event);
+
+ if (hisi_pcie_get_thr_len(event) > HISI_PCIE_THR_MAX_VAL)
+ return false;
+
+ if (hisi_pcie_get_trig_len(event) > HISI_PCIE_TRIG_MAX_VAL)
+ return false;
+
+ if (requester_id) {
+ if (!hisi_pcie_pmu_valid_requester_id(pcie_pmu, requester_id))
+ return false;
+ }
+
+ return true;
+}
+
+static bool hisi_pcie_pmu_cmp_event(struct perf_event *target,
+ struct perf_event *event)
+{
+ return hisi_pcie_get_real_event(target) == hisi_pcie_get_real_event(event);
+}
+
+static bool hisi_pcie_pmu_validate_event_group(struct perf_event *event)
+{
+ struct perf_event *sibling, *leader = event->group_leader;
+ struct perf_event *event_group[HISI_PCIE_MAX_COUNTERS];
+ int counters = 1;
+ int num;
+
+ event_group[0] = leader;
+ if (!is_software_event(leader)) {
+ if (leader->pmu != event->pmu)
+ return false;
+
+ if (leader != event && !hisi_pcie_pmu_cmp_event(leader, event))
+ event_group[counters++] = event;
+ }
+
+ for_each_sibling_event(sibling, event->group_leader) {
+ if (is_software_event(sibling))
+ continue;
+
+ if (sibling->pmu != event->pmu)
+ return false;
+
+ for (num = 0; num < counters; num++) {
+ if (hisi_pcie_pmu_cmp_event(event_group[num], sibling))
+ break;
+ }
+
+ if (num == counters)
+ event_group[counters++] = sibling;
+ }
+
+ return counters <= HISI_PCIE_MAX_COUNTERS;
+}
+
+static int hisi_pcie_pmu_event_init(struct perf_event *event)
+{
+ struct hisi_pcie_pmu *pcie_pmu = to_pcie_pmu(event->pmu);
+ struct hw_perf_event *hwc = &event->hw;
+
+ event->cpu = pcie_pmu->on_cpu;
+
+ if (EXT_COUNTER_IS_USED(hisi_pcie_get_event(event)))
+ hwc->event_base = HISI_PCIE_EXT_CNT;
+ else
+ hwc->event_base = HISI_PCIE_CNT;
+
+ if (event->attr.type != event->pmu->type)
+ return -ENOENT;
+
+ /* Sampling is not supported. */
+ if (is_sampling_event(event) || event->attach_state & PERF_ATTACH_TASK)
+ return -EOPNOTSUPP;
+
+ if (!hisi_pcie_pmu_valid_filter(event, pcie_pmu))
+ return -EINVAL;
+
+ if (!hisi_pcie_pmu_validate_event_group(event))
+ return -EINVAL;
+
+ return 0;
+}
+
+static u64 hisi_pcie_pmu_read_counter(struct perf_event *event)
+{
+ struct hisi_pcie_pmu *pcie_pmu = to_pcie_pmu(event->pmu);
+ u32 idx = event->hw.idx;
+
+ return hisi_pcie_pmu_readq(pcie_pmu, event->hw.event_base, idx);
+}
+
+static int hisi_pcie_pmu_find_related_event(struct hisi_pcie_pmu *pcie_pmu,
+ struct perf_event *event)
+{
+ struct perf_event *sibling;
+ int idx;
+
+ for (idx = 0; idx < HISI_PCIE_MAX_COUNTERS; idx++) {
+ sibling = pcie_pmu->hw_events[idx];
+ if (!sibling)
+ continue;
+
+ if (!hisi_pcie_pmu_cmp_event(sibling, event))
+ continue;
+
+ /* Related events must be used in group */
+ if (sibling->group_leader == event->group_leader)
+ return idx;
+ else
+ return -EINVAL;
+ }
+
+ return idx;
+}
+
+static int hisi_pcie_pmu_get_event_idx(struct hisi_pcie_pmu *pcie_pmu)
+{
+ int idx;
+
+ for (idx = 0; idx < HISI_PCIE_MAX_COUNTERS; idx++) {
+ if (!pcie_pmu->hw_events[idx])
+ return idx;
+ }
+
+ return -EINVAL;
+}
+
+static void hisi_pcie_pmu_event_update(struct perf_event *event)
+{
+ struct hw_perf_event *hwc = &event->hw;
+ u64 new_cnt, prev_cnt, delta;
+
+ do {
+ prev_cnt = local64_read(&hwc->prev_count);
+ new_cnt = hisi_pcie_pmu_read_counter(event);
+ } while (local64_cmpxchg(&hwc->prev_count, prev_cnt,
+ new_cnt) != prev_cnt);
+
+ delta = (new_cnt - prev_cnt) & HISI_PCIE_MAX_PERIOD;
+ local64_add(delta, &event->count);
+}
+
+static void hisi_pcie_pmu_read(struct perf_event *event)
+{
+ hisi_pcie_pmu_event_update(event);
+}
+
+static void hisi_pcie_pmu_set_period(struct perf_event *event)
+{
+ struct hisi_pcie_pmu *pcie_pmu = to_pcie_pmu(event->pmu);
+ struct hw_perf_event *hwc = &event->hw;
+ int idx = hwc->idx;
+
+ local64_set(&hwc->prev_count, HISI_PCIE_INIT_VAL);
+ hisi_pcie_pmu_writeq(pcie_pmu, HISI_PCIE_CNT, idx, HISI_PCIE_INIT_VAL);
+ hisi_pcie_pmu_writeq(pcie_pmu, HISI_PCIE_EXT_CNT, idx, HISI_PCIE_INIT_VAL);
+}
+
+static void hisi_pcie_pmu_enable_counter(struct hisi_pcie_pmu *pcie_pmu, struct hw_perf_event *hwc)
+{
+ u32 idx = hwc->idx;
+ u64 val;
+
+ val = hisi_pcie_pmu_readq(pcie_pmu, HISI_PCIE_EVENT_CTRL, idx);
+ val |= HISI_PCIE_EVENT_EN;
+ hisi_pcie_pmu_writeq(pcie_pmu, HISI_PCIE_EVENT_CTRL, idx, val);
+}
+
+static void hisi_pcie_pmu_disable_counter(struct hisi_pcie_pmu *pcie_pmu, struct hw_perf_event *hwc)
+{
+ u32 idx = hwc->idx;
+ u64 val;
+
+ val = hisi_pcie_pmu_readq(pcie_pmu, HISI_PCIE_EVENT_CTRL, idx);
+ val &= ~HISI_PCIE_EVENT_EN;
+ hisi_pcie_pmu_writeq(pcie_pmu, HISI_PCIE_EVENT_CTRL, idx, val);
+}
+
+static void hisi_pcie_pmu_enable_int(struct hisi_pcie_pmu *pcie_pmu, struct hw_perf_event *hwc)
+{
+ u32 idx = hwc->idx;
+
+ hisi_pcie_pmu_writel(pcie_pmu, HISI_PCIE_INT_MASK, idx, 0);
+}
+
+static void hisi_pcie_pmu_disable_int(struct hisi_pcie_pmu *pcie_pmu, struct hw_perf_event *hwc)
+{
+ u32 idx = hwc->idx;
+
+ hisi_pcie_pmu_writel(pcie_pmu, HISI_PCIE_INT_MASK, idx, 1);
+}
+
+static void hisi_pcie_pmu_reset_counter(struct hisi_pcie_pmu *pcie_pmu, int idx)
+{
+ hisi_pcie_pmu_writeq(pcie_pmu, HISI_PCIE_EVENT_CTRL, idx, HISI_PCIE_RESET_CNT);
+ hisi_pcie_pmu_writeq(pcie_pmu, HISI_PCIE_EVENT_CTRL, idx, HISI_PCIE_INIT_SET);
+}
+
+static void hisi_pcie_pmu_start(struct perf_event *event, int flags)
+{
+ struct hisi_pcie_pmu *pcie_pmu = to_pcie_pmu(event->pmu);
+ struct hw_perf_event *hwc = &event->hw;
+ int idx = hwc->idx;
+ u64 prev_cnt;
+
+ if (WARN_ON_ONCE(!(hwc->state & PERF_HES_STOPPED)))
+ return;
+
+ WARN_ON_ONCE(!(hwc->state & PERF_HES_UPTODATE));
+ hwc->state = 0;
+
+ hisi_pcie_pmu_config_filter(event);
+ hisi_pcie_pmu_enable_counter(pcie_pmu, hwc);
+ hisi_pcie_pmu_enable_int(pcie_pmu, hwc);
+ hisi_pcie_pmu_set_period(event);
+
+ if (flags & PERF_EF_RELOAD) {
+ prev_cnt = local64_read(&hwc->prev_count);
+ hisi_pcie_pmu_writeq(pcie_pmu, hwc->event_base, idx, prev_cnt);
+ }
+
+ perf_event_update_userpage(event);
+}
+
+static void hisi_pcie_pmu_stop(struct perf_event *event, int flags)
+{
+ struct hisi_pcie_pmu *pcie_pmu = to_pcie_pmu(event->pmu);
+ struct hw_perf_event *hwc = &event->hw;
+
+ hisi_pcie_pmu_event_update(event);
+ hisi_pcie_pmu_disable_int(pcie_pmu, hwc);
+ hisi_pcie_pmu_disable_counter(pcie_pmu, hwc);
+ hisi_pcie_pmu_clear_filter(event);
+ WARN_ON_ONCE(hwc->state & PERF_HES_STOPPED);
+ hwc->state |= PERF_HES_STOPPED;
+
+ if (hwc->state & PERF_HES_UPTODATE)
+ return;
+
+ hwc->state |= PERF_HES_UPTODATE;
+}
+
+static int hisi_pcie_pmu_add(struct perf_event *event, int flags)
+{
+ struct hisi_pcie_pmu *pcie_pmu = to_pcie_pmu(event->pmu);
+ struct hw_perf_event *hwc = &event->hw;
+ int idx;
+
+ hwc->state = PERF_HES_STOPPED | PERF_HES_UPTODATE;
+
+ /* Check all working events to find a related event. */
+ idx = hisi_pcie_pmu_find_related_event(pcie_pmu, event);
+ if (idx < 0)
+ return idx;
+
+ /* Current event shares an enabled counter with the related event */
+ if (idx < HISI_PCIE_MAX_COUNTERS) {
+ hwc->idx = idx;
+ goto start_count;
+ }
+
+ idx = hisi_pcie_pmu_get_event_idx(pcie_pmu);
+ if (idx < 0)
+ return idx;
+
+ hwc->idx = idx;
+ pcie_pmu->hw_events[idx] = event;
+ /* Reset Counter to avoid previous statistic interference. */
+ hisi_pcie_pmu_reset_counter(pcie_pmu, idx);
+
+start_count:
+ if (flags & PERF_EF_START)
+ hisi_pcie_pmu_start(event, PERF_EF_RELOAD);
+
+ return 0;
+}
+
+static void hisi_pcie_pmu_del(struct perf_event *event, int flags)
+{
+ struct hisi_pcie_pmu *pcie_pmu = to_pcie_pmu(event->pmu);
+ struct hw_perf_event *hwc = &event->hw;
+
+ hisi_pcie_pmu_stop(event, PERF_EF_UPDATE);
+ pcie_pmu->hw_events[hwc->idx] = NULL;
+ perf_event_update_userpage(event);
+}
+
+static void hisi_pcie_pmu_enable(struct pmu *pmu)
+{
+ struct hisi_pcie_pmu *pcie_pmu = to_pcie_pmu(pmu);
+ int num;
+
+ for (num = 0; num < HISI_PCIE_MAX_COUNTERS; num++) {
+ if (pcie_pmu->hw_events[num])
+ break;
+ }
+
+ if (num == HISI_PCIE_MAX_COUNTERS)
+ return;
+
+ writel(HISI_PCIE_GLOBAL_EN, pcie_pmu->base + HISI_PCIE_GLOBAL_CTRL);
+}
+
+static void hisi_pcie_pmu_disable(struct pmu *pmu)
+{
+ struct hisi_pcie_pmu *pcie_pmu = to_pcie_pmu(pmu);
+
+ writel(HISI_PCIE_GLOBAL_NONE, pcie_pmu->base + HISI_PCIE_GLOBAL_CTRL);
+}
+
+static irqreturn_t hisi_pcie_pmu_irq(int irq, void *data)
+{
+ struct hisi_pcie_pmu *pcie_pmu = data;
+ irqreturn_t ret = IRQ_NONE;
+ struct perf_event *event;
+ u32 overflown;
+ int idx;
+
+ for (idx = 0; idx < HISI_PCIE_MAX_COUNTERS; idx++) {
+ overflown = hisi_pcie_pmu_readl(pcie_pmu, HISI_PCIE_INT_STAT, idx);
+ if (!overflown)
+ continue;
+
+ /* Clear status of interrupt. */
+ hisi_pcie_pmu_writel(pcie_pmu, HISI_PCIE_INT_STAT, idx, 1);
+ event = pcie_pmu->hw_events[idx];
+ if (!event)
+ continue;
+
+ hisi_pcie_pmu_event_update(event);
+ hisi_pcie_pmu_set_period(event);
+ ret = IRQ_HANDLED;
+ }
+
+ return ret;
+}
+
+static int hisi_pcie_pmu_irq_register(struct pci_dev *pdev, struct hisi_pcie_pmu *pcie_pmu)
+{
+ int irq, ret;
+
+ ret = pci_alloc_irq_vectors(pdev, 1, 1, PCI_IRQ_MSI);
+ if (ret < 0) {
+ pci_err(pdev, "Failed to enable MSI vectors: %d\n", ret);
+ return ret;
+ }
+
+ irq = pci_irq_vector(pdev, 0);
+ ret = request_irq(irq, hisi_pcie_pmu_irq, IRQF_NOBALANCING | IRQF_NO_THREAD, DRV_NAME,
+ pcie_pmu);
+ if (ret) {
+ pci_err(pdev, "Failed to register IRQ: %d\n", ret);
+ pci_free_irq_vectors(pdev);
+ return ret;
+ }
+
+ pcie_pmu->irq = irq;
+
+ return 0;
+}
+
+static void hisi_pcie_pmu_irq_unregister(struct pci_dev *pdev, struct hisi_pcie_pmu *pcie_pmu)
+{
+ free_irq(pcie_pmu->irq, pcie_pmu);
+ pci_free_irq_vectors(pdev);
+}
+
+static int hisi_pcie_pmu_online_cpu(unsigned int cpu, struct hlist_node *node)
+{
+ struct hisi_pcie_pmu *pcie_pmu = hlist_entry_safe(node, struct hisi_pcie_pmu, node);
+
+ if (pcie_pmu->on_cpu == -1) {
+ pcie_pmu->on_cpu = cpu;
+ WARN_ON(irq_set_affinity(pcie_pmu->irq, cpumask_of(cpu)));
+ }
+
+ return 0;
+}
+
+static int hisi_pcie_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node)
+{
+ struct hisi_pcie_pmu *pcie_pmu = hlist_entry_safe(node, struct hisi_pcie_pmu, node);
+ unsigned int target;
+
+ /* Nothing to do if this CPU doesn't own the PMU */
+ if (pcie_pmu->on_cpu != cpu)
+ return 0;
+
+ pcie_pmu->on_cpu = -1;
+ /* Choose a new CPU from all online cpus. */
+ target = cpumask_first(cpu_online_mask);
+ if (target >= nr_cpu_ids) {
+ pci_err(pcie_pmu->pdev, "There is no CPU to set\n");
+ return 0;
+ }
+
+ perf_pmu_migrate_context(&pcie_pmu->pmu, cpu, target);
+ /* Use this CPU for event counting */
+ pcie_pmu->on_cpu = target;
+ WARN_ON(irq_set_affinity(pcie_pmu->irq, cpumask_of(target)));
+
+ return 0;
+}
+
+static struct attribute *hisi_pcie_pmu_events_attr[] = {
+ HISI_PCIE_PMU_EVENT_ATTR(rx_mwr_latency, 0x0010),
+ HISI_PCIE_PMU_EVENT_ATTR(rx_mwr_cnt, 0x10010),
+ HISI_PCIE_PMU_EVENT_ATTR(rx_mrd_latency, 0x0210),
+ HISI_PCIE_PMU_EVENT_ATTR(rx_mrd_cnt, 0x10210),
+ HISI_PCIE_PMU_EVENT_ATTR(tx_mrd_latency, 0x0011),
+ HISI_PCIE_PMU_EVENT_ATTR(tx_mrd_cnt, 0x10011),
+ HISI_PCIE_PMU_EVENT_ATTR(rx_mrd_flux, 0x1005),
+ HISI_PCIE_PMU_EVENT_ATTR(rx_mrd_time, 0x11005),
+ HISI_PCIE_PMU_EVENT_ATTR(tx_mrd_flux, 0x2004),
+ HISI_PCIE_PMU_EVENT_ATTR(tx_mrd_time, 0x12004),
+ NULL
+};
+
+static struct attribute_group hisi_pcie_pmu_events_group = {
+ .name = "events",
+ .attrs = hisi_pcie_pmu_events_attr,
+};
+
+static struct attribute *hisi_pcie_pmu_format_attr[] = {
+ HISI_PCIE_PMU_FORMAT_ATTR(event, "config:0-16"),
+ HISI_PCIE_PMU_FORMAT_ATTR(thr_len, "config1:0-3"),
+ HISI_PCIE_PMU_FORMAT_ATTR(thr_mode, "config1:4"),
+ HISI_PCIE_PMU_FORMAT_ATTR(trig_len, "config1:5-8"),
+ HISI_PCIE_PMU_FORMAT_ATTR(trig_mode, "config1:9"),
+ HISI_PCIE_PMU_FORMAT_ATTR(port, "config2:0-15"),
+ HISI_PCIE_PMU_FORMAT_ATTR(bdf, "config2:16-31"),
+ NULL
+};
+
+static const struct attribute_group hisi_pcie_pmu_format_group = {
+ .name = "format",
+ .attrs = hisi_pcie_pmu_format_attr,
+};
+
+static struct attribute *hisi_pcie_pmu_bus_attrs[] = {
+ &dev_attr_bus.attr,
+ NULL
+};
+
+static const struct attribute_group hisi_pcie_pmu_bus_attr_group = {
+ .attrs = hisi_pcie_pmu_bus_attrs,
+};
+
+static struct attribute *hisi_pcie_pmu_cpumask_attrs[] = {
+ &dev_attr_cpumask.attr,
+ NULL
+};
+
+static const struct attribute_group hisi_pcie_pmu_cpumask_attr_group = {
+ .attrs = hisi_pcie_pmu_cpumask_attrs,
+};
+
+static struct attribute *hisi_pcie_pmu_identifier_attrs[] = {
+ &dev_attr_identifier.attr,
+ NULL
+};
+
+static const struct attribute_group hisi_pcie_pmu_identifier_attr_group = {
+ .attrs = hisi_pcie_pmu_identifier_attrs,
+};
+
+static const struct attribute_group *hisi_pcie_pmu_attr_groups[] = {
+ &hisi_pcie_pmu_events_group,
+ &hisi_pcie_pmu_format_group,
+ &hisi_pcie_pmu_bus_attr_group,
+ &hisi_pcie_pmu_cpumask_attr_group,
+ &hisi_pcie_pmu_identifier_attr_group,
+ NULL
+};
+
+static int hisi_pcie_alloc_pmu(struct pci_dev *pdev, struct hisi_pcie_pmu *pcie_pmu)
+{
+ struct hisi_pcie_reg_pair regs;
+ u16 sicl_id, core_id;
+ char *name;
+
+ regs = hisi_pcie_parse_reg_value(pcie_pmu, HISI_PCIE_REG_BDF);
+ pcie_pmu->bdf_min = regs.lo;
+ pcie_pmu->bdf_max = regs.hi;
+
+ regs = hisi_pcie_parse_reg_value(pcie_pmu, HISI_PCIE_REG_INFO);
+ sicl_id = regs.hi;
+ core_id = regs.lo;
+
+ name = devm_kasprintf(&pdev->dev, GFP_KERNEL, "hisi_pcie%u_core%u", sicl_id, core_id);
+ if (!name)
+ return -ENOMEM;
+
+ pcie_pmu->pdev = pdev;
+ pcie_pmu->on_cpu = -1;
+ pcie_pmu->identifier = readl(pcie_pmu->base + HISI_PCIE_REG_VERSION);
+ pcie_pmu->pmu = (struct pmu) {
+ .name = name,
+ .module = THIS_MODULE,
+ .event_init = hisi_pcie_pmu_event_init,
+ .pmu_enable = hisi_pcie_pmu_enable,
+ .pmu_disable = hisi_pcie_pmu_disable,
+ .add = hisi_pcie_pmu_add,
+ .del = hisi_pcie_pmu_del,
+ .start = hisi_pcie_pmu_start,
+ .stop = hisi_pcie_pmu_stop,
+ .read = hisi_pcie_pmu_read,
+ .task_ctx_nr = perf_invalid_context,
+ .attr_groups = hisi_pcie_pmu_attr_groups,
+ .capabilities = PERF_PMU_CAP_NO_EXCLUDE,
+ };
+
+ return 0;
+}
+
+static int hisi_pcie_init_pmu(struct pci_dev *pdev, struct hisi_pcie_pmu *pcie_pmu)
+{
+ int ret;
+
+ pcie_pmu->base = pci_ioremap_bar(pdev, 2);
+ if (!pcie_pmu->base) {
+ pci_err(pdev, "Ioremap failed for pcie_pmu resource\n");
+ return -ENOMEM;
+ }
+
+ ret = hisi_pcie_alloc_pmu(pdev, pcie_pmu);
+ if (ret)
+ goto err_iounmap;
+
+ ret = hisi_pcie_pmu_irq_register(pdev, pcie_pmu);
+ if (ret)
+ goto err_iounmap;
+
+ ret = cpuhp_state_add_instance(CPUHP_AP_PERF_ARM_HISI_PCIE_PMU_ONLINE, &pcie_pmu->node);
+ if (ret) {
+ pci_err(pdev, "Failed to register hotplug: %d\n", ret);
+ goto err_irq_unregister;
+ }
+
+ ret = perf_pmu_register(&pcie_pmu->pmu, pcie_pmu->pmu.name, -1);
+ if (ret) {
+ pci_err(pdev, "Failed to register PCIe PMU: %d\n", ret);
+ goto err_hotplug_unregister;
+ }
+
+ return ret;
+
+err_hotplug_unregister:
+ cpuhp_state_remove_instance_nocalls(
+ CPUHP_AP_PERF_ARM_HISI_PCIE_PMU_ONLINE, &pcie_pmu->node);
+
+err_irq_unregister:
+ hisi_pcie_pmu_irq_unregister(pdev, pcie_pmu);
+
+err_iounmap:
+ iounmap(pcie_pmu->base);
+
+ return ret;
+}
+
+static void hisi_pcie_uninit_pmu(struct pci_dev *pdev)
+{
+ struct hisi_pcie_pmu *pcie_pmu = pci_get_drvdata(pdev);
+
+ perf_pmu_unregister(&pcie_pmu->pmu);
+ cpuhp_state_remove_instance_nocalls(
+ CPUHP_AP_PERF_ARM_HISI_PCIE_PMU_ONLINE, &pcie_pmu->node);
+ hisi_pcie_pmu_irq_unregister(pdev, pcie_pmu);
+ iounmap(pcie_pmu->base);
+}
+
+static int hisi_pcie_init_dev(struct pci_dev *pdev)
+{
+ int ret;
+
+ ret = pcim_enable_device(pdev);
+ if (ret) {
+ pci_err(pdev, "Failed to enable PCI device: %d\n", ret);
+ return ret;
+ }
+
+ ret = pcim_iomap_regions(pdev, BIT(2), DRV_NAME);
+ if (ret < 0) {
+ pci_err(pdev, "Failed to request PCI mem regions: %d\n", ret);
+ return ret;
+ }
+
+ pci_set_master(pdev);
+
+ return 0;
+}
+
+static int hisi_pcie_pmu_probe(struct pci_dev *pdev, const struct pci_device_id *id)
+{
+ struct hisi_pcie_pmu *pcie_pmu;
+ int ret;
+
+ pcie_pmu = devm_kzalloc(&pdev->dev, sizeof(*pcie_pmu), GFP_KERNEL);
+ if (!pcie_pmu)
+ return -ENOMEM;
+
+ ret = hisi_pcie_init_dev(pdev);
+ if (ret)
+ return ret;
+
+ ret = hisi_pcie_init_pmu(pdev, pcie_pmu);
+ if (ret)
+ return ret;
+
+ pci_set_drvdata(pdev, pcie_pmu);
+
+ return ret;
+}
+
+static void hisi_pcie_pmu_remove(struct pci_dev *pdev)
+{
+ hisi_pcie_uninit_pmu(pdev);
+ pci_set_drvdata(pdev, NULL);
+}
+
+static const struct pci_device_id hisi_pcie_pmu_ids[] = {
+ { PCI_DEVICE(PCI_VENDOR_ID_HUAWEI, 0xa12d) },
+ { 0, }
+};
+MODULE_DEVICE_TABLE(pci, hisi_pcie_pmu_ids);
+
+static struct pci_driver hisi_pcie_pmu_driver = {
+ .name = DRV_NAME,
+ .id_table = hisi_pcie_pmu_ids,
+ .probe = hisi_pcie_pmu_probe,
+ .remove = hisi_pcie_pmu_remove,
+};
+
+static int __init hisi_pcie_module_init(void)
+{
+ int ret;
+
+ ret = cpuhp_setup_state_multi(CPUHP_AP_PERF_ARM_HISI_PCIE_PMU_ONLINE,
+ "AP_PERF_ARM_HISI_PCIE_PMU_ONLINE",
+ hisi_pcie_pmu_online_cpu,
+ hisi_pcie_pmu_offline_cpu);
+ if (ret) {
+ pr_err("Failed to setup PCIe PMU hotplug: %d\n", ret);
+ return ret;
+ }
+
+ ret = pci_register_driver(&hisi_pcie_pmu_driver);
+ if (ret)
+ cpuhp_remove_multi_state(CPUHP_AP_PERF_ARM_HISI_PCIE_PMU_ONLINE);
+
+ return ret;
+}
+module_init(hisi_pcie_module_init);
+
+static void __exit hisi_pcie_module_exit(void)
+{
+ pci_unregister_driver(&hisi_pcie_pmu_driver);
+ cpuhp_remove_multi_state(CPUHP_AP_PERF_ARM_HISI_PCIE_PMU_ONLINE);
+}
+module_exit(hisi_pcie_module_exit);
+
+MODULE_DESCRIPTION("HiSilicon PCIe PMU driver");
+MODULE_LICENSE("GPL v2");
+MODULE_AUTHOR("Qi Liu <liuqi115@huawei.com>");
diff --git a/drivers/perf/marvell_cn10k_tad_pmu.c b/drivers/perf/marvell_cn10k_tad_pmu.c
new file mode 100644
index 0000000..7f4d292
--- /dev/null
+++ b/drivers/perf/marvell_cn10k_tad_pmu.c
@@ -0,0 +1,429 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Marvell CN10K LLC-TAD perf driver
+ *
+ * Copyright (C) 2021 Marvell
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#define pr_fmt(fmt) "tad_pmu: " fmt
+
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/of_device.h>
+#include <linux/cpuhotplug.h>
+#include <linux/perf_event.h>
+#include <linux/platform_device.h>
+
+#define TAD_PFC_OFFSET 0x0
+#define TAD_PFC(counter) (TAD_PFC_OFFSET | (counter << 3))
+#define TAD_PRF_OFFSET 0x100
+#define TAD_PRF(counter) (TAD_PRF_OFFSET | (counter << 3))
+#define TAD_PRF_CNTSEL_MASK 0xFF
+#define TAD_MAX_COUNTERS 8
+
+#define to_tad_pmu(p) (container_of(p, struct tad_pmu, pmu))
+
+struct tad_region {
+ void __iomem *base;
+};
+
+struct tad_pmu {
+ struct pmu pmu;
+ struct tad_region *regions;
+ u32 region_cnt;
+ unsigned int cpu;
+ struct hlist_node node;
+ struct perf_event *events[TAD_MAX_COUNTERS];
+ DECLARE_BITMAP(counters_map, TAD_MAX_COUNTERS);
+};
+
+static int tad_pmu_cpuhp_state;
+
+static void tad_pmu_event_counter_read(struct perf_event *event)
+{
+ struct tad_pmu *tad_pmu = to_tad_pmu(event->pmu);
+ struct hw_perf_event *hwc = &event->hw;
+ u32 counter_idx = hwc->idx;
+ u64 prev, new;
+ int i;
+
+ do {
+ prev = local64_read(&hwc->prev_count);
+ for (i = 0, new = 0; i < tad_pmu->region_cnt; i++)
+ new += readq(tad_pmu->regions[i].base +
+ TAD_PFC(counter_idx));
+ } while (local64_cmpxchg(&hwc->prev_count, prev, new) != prev);
+
+ local64_add(new - prev, &event->count);
+}
+
+static void tad_pmu_event_counter_stop(struct perf_event *event, int flags)
+{
+ struct tad_pmu *tad_pmu = to_tad_pmu(event->pmu);
+ struct hw_perf_event *hwc = &event->hw;
+ u32 counter_idx = hwc->idx;
+ int i;
+
+ /* TAD()_PFC() stop counting on the write
+ * which sets TAD()_PRF()[CNTSEL] == 0
+ */
+ for (i = 0; i < tad_pmu->region_cnt; i++) {
+ writeq_relaxed(0, tad_pmu->regions[i].base +
+ TAD_PRF(counter_idx));
+ }
+
+ tad_pmu_event_counter_read(event);
+ hwc->state |= PERF_HES_STOPPED | PERF_HES_UPTODATE;
+}
+
+static void tad_pmu_event_counter_start(struct perf_event *event, int flags)
+{
+ struct tad_pmu *tad_pmu = to_tad_pmu(event->pmu);
+ struct hw_perf_event *hwc = &event->hw;
+ u32 event_idx = event->attr.config;
+ u32 counter_idx = hwc->idx;
+ u64 reg_val;
+ int i;
+
+ hwc->state = 0;
+
+ /* Typically TAD_PFC() are zeroed to start counting */
+ for (i = 0; i < tad_pmu->region_cnt; i++)
+ writeq_relaxed(0, tad_pmu->regions[i].base +
+ TAD_PFC(counter_idx));
+
+ /* TAD()_PFC() start counting on the write
+ * which sets TAD()_PRF()[CNTSEL] != 0
+ */
+ for (i = 0; i < tad_pmu->region_cnt; i++) {
+ reg_val = readq_relaxed(tad_pmu->regions[i].base +
+ TAD_PRF(counter_idx));
+ reg_val |= (event_idx & 0xFF);
+ writeq_relaxed(reg_val, tad_pmu->regions[i].base +
+ TAD_PRF(counter_idx));
+ }
+}
+
+static void tad_pmu_event_counter_del(struct perf_event *event, int flags)
+{
+ struct tad_pmu *tad_pmu = to_tad_pmu(event->pmu);
+ struct hw_perf_event *hwc = &event->hw;
+ int idx = hwc->idx;
+
+ tad_pmu_event_counter_stop(event, flags | PERF_EF_UPDATE);
+ tad_pmu->events[idx] = NULL;
+ clear_bit(idx, tad_pmu->counters_map);
+}
+
+static int tad_pmu_event_counter_add(struct perf_event *event, int flags)
+{
+ struct tad_pmu *tad_pmu = to_tad_pmu(event->pmu);
+ struct hw_perf_event *hwc = &event->hw;
+ int idx;
+
+ /* Get a free counter for this event */
+ idx = find_first_zero_bit(tad_pmu->counters_map, TAD_MAX_COUNTERS);
+ if (idx == TAD_MAX_COUNTERS)
+ return -EAGAIN;
+
+ set_bit(idx, tad_pmu->counters_map);
+
+ hwc->idx = idx;
+ hwc->state = PERF_HES_STOPPED;
+ tad_pmu->events[idx] = event;
+
+ if (flags & PERF_EF_START)
+ tad_pmu_event_counter_start(event, flags);
+
+ return 0;
+}
+
+static int tad_pmu_event_init(struct perf_event *event)
+{
+ struct tad_pmu *tad_pmu = to_tad_pmu(event->pmu);
+
+ if (!event->attr.disabled)
+ return -EINVAL;
+
+ if (event->attr.type != event->pmu->type)
+ return -ENOENT;
+
+ if (event->state != PERF_EVENT_STATE_OFF)
+ return -EINVAL;
+
+ event->cpu = tad_pmu->cpu;
+ event->hw.idx = -1;
+ event->hw.config_base = event->attr.config;
+
+ return 0;
+}
+
+static ssize_t tad_pmu_event_show(struct device *dev,
+ struct device_attribute *attr, char *page)
+{
+ struct perf_pmu_events_attr *pmu_attr;
+
+ pmu_attr = container_of(attr, struct perf_pmu_events_attr, attr);
+ return sysfs_emit(page, "event=0x%02llx\n", pmu_attr->id);
+}
+
+#define TAD_PMU_EVENT_ATTR(name, config) \
+ PMU_EVENT_ATTR_ID(name, tad_pmu_event_show, config)
+
+static struct attribute *tad_pmu_event_attrs[] = {
+ TAD_PMU_EVENT_ATTR(tad_none, 0x0),
+ TAD_PMU_EVENT_ATTR(tad_req_msh_in_any, 0x1),
+ TAD_PMU_EVENT_ATTR(tad_req_msh_in_mn, 0x2),
+ TAD_PMU_EVENT_ATTR(tad_req_msh_in_exlmn, 0x3),
+ TAD_PMU_EVENT_ATTR(tad_rsp_msh_in_any, 0x4),
+ TAD_PMU_EVENT_ATTR(tad_rsp_msh_in_mn, 0x5),
+ TAD_PMU_EVENT_ATTR(tad_rsp_msh_in_exlmn, 0x6),
+ TAD_PMU_EVENT_ATTR(tad_rsp_msh_in_dss, 0x7),
+ TAD_PMU_EVENT_ATTR(tad_rsp_msh_in_retry_dss, 0x8),
+ TAD_PMU_EVENT_ATTR(tad_dat_msh_in_any, 0x9),
+ TAD_PMU_EVENT_ATTR(tad_dat_msh_in_dss, 0xa),
+ TAD_PMU_EVENT_ATTR(tad_req_msh_out_any, 0xb),
+ TAD_PMU_EVENT_ATTR(tad_req_msh_out_dss_rd, 0xc),
+ TAD_PMU_EVENT_ATTR(tad_req_msh_out_dss_wr, 0xd),
+ TAD_PMU_EVENT_ATTR(tad_req_msh_out_evict, 0xe),
+ TAD_PMU_EVENT_ATTR(tad_rsp_msh_out_any, 0xf),
+ TAD_PMU_EVENT_ATTR(tad_rsp_msh_out_retry_exlmn, 0x10),
+ TAD_PMU_EVENT_ATTR(tad_rsp_msh_out_retry_mn, 0x11),
+ TAD_PMU_EVENT_ATTR(tad_rsp_msh_out_exlmn, 0x12),
+ TAD_PMU_EVENT_ATTR(tad_rsp_msh_out_mn, 0x13),
+ TAD_PMU_EVENT_ATTR(tad_snp_msh_out_any, 0x14),
+ TAD_PMU_EVENT_ATTR(tad_snp_msh_out_mn, 0x15),
+ TAD_PMU_EVENT_ATTR(tad_snp_msh_out_exlmn, 0x16),
+ TAD_PMU_EVENT_ATTR(tad_dat_msh_out_any, 0x17),
+ TAD_PMU_EVENT_ATTR(tad_dat_msh_out_fill, 0x18),
+ TAD_PMU_EVENT_ATTR(tad_dat_msh_out_dss, 0x19),
+ TAD_PMU_EVENT_ATTR(tad_alloc_dtg, 0x1a),
+ TAD_PMU_EVENT_ATTR(tad_alloc_ltg, 0x1b),
+ TAD_PMU_EVENT_ATTR(tad_alloc_any, 0x1c),
+ TAD_PMU_EVENT_ATTR(tad_hit_dtg, 0x1d),
+ TAD_PMU_EVENT_ATTR(tad_hit_ltg, 0x1e),
+ TAD_PMU_EVENT_ATTR(tad_hit_any, 0x1f),
+ TAD_PMU_EVENT_ATTR(tad_tag_rd, 0x20),
+ TAD_PMU_EVENT_ATTR(tad_dat_rd, 0x21),
+ TAD_PMU_EVENT_ATTR(tad_dat_rd_byp, 0x22),
+ TAD_PMU_EVENT_ATTR(tad_ifb_occ, 0x23),
+ TAD_PMU_EVENT_ATTR(tad_req_occ, 0x24),
+ NULL
+};
+
+static const struct attribute_group tad_pmu_events_attr_group = {
+ .name = "events",
+ .attrs = tad_pmu_event_attrs,
+};
+
+PMU_FORMAT_ATTR(event, "config:0-7");
+
+static struct attribute *tad_pmu_format_attrs[] = {
+ &format_attr_event.attr,
+ NULL
+};
+
+static struct attribute_group tad_pmu_format_attr_group = {
+ .name = "format",
+ .attrs = tad_pmu_format_attrs,
+};
+
+static ssize_t tad_pmu_cpumask_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct tad_pmu *tad_pmu = to_tad_pmu(dev_get_drvdata(dev));
+
+ return cpumap_print_to_pagebuf(true, buf, cpumask_of(tad_pmu->cpu));
+}
+
+static DEVICE_ATTR(cpumask, 0444, tad_pmu_cpumask_show, NULL);
+
+static struct attribute *tad_pmu_cpumask_attrs[] = {
+ &dev_attr_cpumask.attr,
+ NULL
+};
+
+static struct attribute_group tad_pmu_cpumask_attr_group = {
+ .attrs = tad_pmu_cpumask_attrs,
+};
+
+static const struct attribute_group *tad_pmu_attr_groups[] = {
+ &tad_pmu_events_attr_group,
+ &tad_pmu_format_attr_group,
+ &tad_pmu_cpumask_attr_group,
+ NULL
+};
+
+static int tad_pmu_probe(struct platform_device *pdev)
+{
+ struct device_node *node = pdev->dev.of_node;
+ struct tad_region *regions;
+ struct tad_pmu *tad_pmu;
+ struct resource *res;
+ u32 tad_pmu_page_size;
+ u32 tad_page_size;
+ u32 tad_cnt;
+ int i, ret;
+ char *name;
+
+ tad_pmu = devm_kzalloc(&pdev->dev, sizeof(*tad_pmu), GFP_KERNEL);
+ if (!tad_pmu)
+ return -ENOMEM;
+
+ platform_set_drvdata(pdev, tad_pmu);
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ if (!res) {
+ dev_err(&pdev->dev, "Mem resource not found\n");
+ return -ENODEV;
+ }
+
+ ret = of_property_read_u32(node, "marvell,tad-page-size",
+ &tad_page_size);
+ if (ret) {
+ dev_err(&pdev->dev, "Can't find tad-page-size property\n");
+ return ret;
+ }
+
+ ret = of_property_read_u32(node, "marvell,tad-pmu-page-size",
+ &tad_pmu_page_size);
+ if (ret) {
+ dev_err(&pdev->dev, "Can't find tad-pmu-page-size property\n");
+ return ret;
+ }
+
+ ret = of_property_read_u32(node, "marvell,tad-cnt", &tad_cnt);
+ if (ret) {
+ dev_err(&pdev->dev, "Can't find tad-cnt property\n");
+ return ret;
+ }
+
+ regions = devm_kcalloc(&pdev->dev, tad_cnt,
+ sizeof(*regions), GFP_KERNEL);
+ if (!regions)
+ return -ENOMEM;
+
+ /* ioremap the distributed TAD pmu regions */
+ for (i = 0; i < tad_cnt && res->start < res->end; i++) {
+ regions[i].base = devm_ioremap(&pdev->dev,
+ res->start,
+ tad_pmu_page_size);
+ if (!regions[i].base) {
+ dev_err(&pdev->dev, "TAD%d ioremap fail\n", i);
+ return -ENOMEM;
+ }
+ res->start += tad_page_size;
+ }
+
+ tad_pmu->regions = regions;
+ tad_pmu->region_cnt = tad_cnt;
+
+ tad_pmu->pmu = (struct pmu) {
+
+ .module = THIS_MODULE,
+ .attr_groups = tad_pmu_attr_groups,
+ .capabilities = PERF_PMU_CAP_NO_EXCLUDE |
+ PERF_PMU_CAP_NO_INTERRUPT,
+ .task_ctx_nr = perf_invalid_context,
+
+ .event_init = tad_pmu_event_init,
+ .add = tad_pmu_event_counter_add,
+ .del = tad_pmu_event_counter_del,
+ .start = tad_pmu_event_counter_start,
+ .stop = tad_pmu_event_counter_stop,
+ .read = tad_pmu_event_counter_read,
+ };
+
+ tad_pmu->cpu = raw_smp_processor_id();
+
+ /* Register pmu instance for cpu hotplug */
+ ret = cpuhp_state_add_instance_nocalls(tad_pmu_cpuhp_state,
+ &tad_pmu->node);
+ if (ret) {
+ dev_err(&pdev->dev, "Error %d registering hotplug\n", ret);
+ return ret;
+ }
+
+ name = "tad";
+ ret = perf_pmu_register(&tad_pmu->pmu, name, -1);
+ if (ret)
+ cpuhp_state_remove_instance_nocalls(tad_pmu_cpuhp_state,
+ &tad_pmu->node);
+
+ return ret;
+}
+
+static int tad_pmu_remove(struct platform_device *pdev)
+{
+ struct tad_pmu *pmu = platform_get_drvdata(pdev);
+
+ cpuhp_state_remove_instance_nocalls(tad_pmu_cpuhp_state,
+ &pmu->node);
+ perf_pmu_unregister(&pmu->pmu);
+
+ return 0;
+}
+
+static const struct of_device_id tad_pmu_of_match[] = {
+ { .compatible = "marvell,cn10k-tad-pmu", },
+ {},
+};
+
+static struct platform_driver tad_pmu_driver = {
+ .driver = {
+ .name = "cn10k_tad_pmu",
+ .of_match_table = of_match_ptr(tad_pmu_of_match),
+ .suppress_bind_attrs = true,
+ },
+ .probe = tad_pmu_probe,
+ .remove = tad_pmu_remove,
+};
+
+static int tad_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node)
+{
+ struct tad_pmu *pmu = hlist_entry_safe(node, struct tad_pmu, node);
+ unsigned int target;
+
+ if (cpu != pmu->cpu)
+ return 0;
+
+ target = cpumask_any_but(cpu_online_mask, cpu);
+ if (target >= nr_cpu_ids)
+ return 0;
+
+ perf_pmu_migrate_context(&pmu->pmu, cpu, target);
+ pmu->cpu = target;
+
+ return 0;
+}
+
+static int __init tad_pmu_init(void)
+{
+ int ret;
+
+ ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN,
+ "perf/cn10k/tadpmu:online",
+ NULL,
+ tad_pmu_offline_cpu);
+ if (ret < 0)
+ return ret;
+ tad_pmu_cpuhp_state = ret;
+ return platform_driver_register(&tad_pmu_driver);
+}
+
+static void __exit tad_pmu_exit(void)
+{
+ platform_driver_unregister(&tad_pmu_driver);
+ cpuhp_remove_multi_state(tad_pmu_cpuhp_state);
+}
+
+module_init(tad_pmu_init);
+module_exit(tad_pmu_exit);
+
+MODULE_DESCRIPTION("Marvell CN10K LLC-TAD Perf driver");
+MODULE_AUTHOR("Bhaskara Budiredla <bbudiredla@marvell.com>");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/phy/marvell/phy-mvebu-cp110-comphy.c b/drivers/phy/marvell/phy-mvebu-cp110-comphy.c
index bbd6f2a..34672e8 100644
--- a/drivers/phy/marvell/phy-mvebu-cp110-comphy.c
+++ b/drivers/phy/marvell/phy-mvebu-cp110-comphy.c
@@ -141,6 +141,7 @@
#define COMPHY_FW_SPEED_1250 0
#define COMPHY_FW_SPEED_3125 2
#define COMPHY_FW_SPEED_5000 3
+#define COMPHY_FW_SPEED_515625 4
#define COMPHY_FW_SPEED_103125 6
#define COMPHY_FW_PORT_OFFSET 8
#define COMPHY_FW_PORT_MASK GENMASK(11, 8)
@@ -220,6 +221,7 @@ static const struct mvebu_comphy_conf mvebu_comphy_cp110_modes[] = {
ETH_CONF(2, 0, PHY_INTERFACE_MODE_SGMII, 0x1, COMPHY_FW_MODE_SGMII),
ETH_CONF(2, 0, PHY_INTERFACE_MODE_2500BASEX, 0x1, COMPHY_FW_MODE_2500BASEX),
ETH_CONF(2, 0, PHY_INTERFACE_MODE_RXAUI, 0x1, COMPHY_FW_MODE_RXAUI),
+ ETH_CONF(2, 0, PHY_INTERFACE_MODE_5GBASER, 0x1, COMPHY_FW_MODE_XFI),
ETH_CONF(2, 0, PHY_INTERFACE_MODE_10GBASER, 0x1, COMPHY_FW_MODE_XFI),
GEN_CONF(2, 0, PHY_MODE_USB_HOST_SS, COMPHY_FW_MODE_USB3H),
GEN_CONF(2, 0, PHY_MODE_SATA, COMPHY_FW_MODE_SATA),
@@ -234,6 +236,7 @@ static const struct mvebu_comphy_conf mvebu_comphy_cp110_modes[] = {
/* lane 4 */
ETH_CONF(4, 0, PHY_INTERFACE_MODE_SGMII, 0x2, COMPHY_FW_MODE_SGMII),
ETH_CONF(4, 0, PHY_INTERFACE_MODE_2500BASEX, 0x2, COMPHY_FW_MODE_2500BASEX),
+ ETH_CONF(4, 0, PHY_INTERFACE_MODE_5GBASER, 0x2, COMPHY_FW_MODE_XFI),
ETH_CONF(4, 0, PHY_INTERFACE_MODE_10GBASER, 0x2, COMPHY_FW_MODE_XFI),
ETH_CONF(4, 0, PHY_INTERFACE_MODE_RXAUI, 0x2, COMPHY_FW_MODE_RXAUI),
GEN_CONF(4, 0, PHY_MODE_USB_DEVICE_SS, COMPHY_FW_MODE_USB3D),
@@ -241,6 +244,7 @@ static const struct mvebu_comphy_conf mvebu_comphy_cp110_modes[] = {
GEN_CONF(4, 1, PHY_MODE_PCIE, COMPHY_FW_MODE_PCIE),
ETH_CONF(4, 1, PHY_INTERFACE_MODE_SGMII, 0x1, COMPHY_FW_MODE_SGMII),
ETH_CONF(4, 1, PHY_INTERFACE_MODE_2500BASEX, -1, COMPHY_FW_MODE_2500BASEX),
+ ETH_CONF(4, 1, PHY_INTERFACE_MODE_5GBASER, -1, COMPHY_FW_MODE_XFI),
ETH_CONF(4, 1, PHY_INTERFACE_MODE_10GBASER, -1, COMPHY_FW_MODE_XFI),
/* lane 5 */
ETH_CONF(5, 1, PHY_INTERFACE_MODE_RXAUI, 0x2, COMPHY_FW_MODE_RXAUI),
@@ -790,6 +794,11 @@ static int mvebu_comphy_power_on(struct phy *phy)
lane->id);
fw_speed = COMPHY_FW_SPEED_3125;
break;
+ case PHY_INTERFACE_MODE_5GBASER:
+ dev_dbg(priv->dev, "set lane %d to 5GBASE-R mode\n",
+ lane->id);
+ fw_speed = COMPHY_FW_SPEED_515625;
+ break;
case PHY_INTERFACE_MODE_10GBASER:
dev_dbg(priv->dev, "set lane %d to 10GBASE-R mode\n",
lane->id);
diff --git a/drivers/pinctrl/Kconfig b/drivers/pinctrl/Kconfig
index 6a961d5..0d5b61e 100644
--- a/drivers/pinctrl/Kconfig
+++ b/drivers/pinctrl/Kconfig
@@ -281,6 +281,23 @@
select PINCONF
select GPIOLIB_IRQCHIP
+config PINCTRL_STARFIVE
+ tristate "Pinctrl and GPIO driver for the StarFive JH7100 SoC"
+ depends on SOC_STARFIVE || COMPILE_TEST
+ depends on OF
+ default SOC_STARFIVE
+ select GENERIC_PINCTRL_GROUPS
+ select GENERIC_PINMUX_FUNCTIONS
+ select GENERIC_PINCONF
+ select GPIOLIB
+ select GPIOLIB_IRQCHIP
+ select OF_GPIO
+ help
+ Say yes here to support pin control on the StarFive JH7100 SoC.
+ This also provides an interface to the GPIO pins not used by other
+ peripherals supporting inputs, outputs, configuring pull-up/pull-down
+ and interrupts on input changes.
+
config PINCTRL_STMFX
tristate "STMicroelectronics STMFX GPIO expander pinctrl driver"
depends on I2C
diff --git a/drivers/pinctrl/Makefile b/drivers/pinctrl/Makefile
index 5e63de2..f5bdd6b 100644
--- a/drivers/pinctrl/Makefile
+++ b/drivers/pinctrl/Makefile
@@ -39,6 +39,7 @@
obj-$(CONFIG_PINCTRL_LPC18XX) += pinctrl-lpc18xx.o
obj-$(CONFIG_PINCTRL_TB10X) += pinctrl-tb10x.o
obj-$(CONFIG_PINCTRL_ST) += pinctrl-st.o
+obj-$(CONFIG_PINCTRL_STARFIVE) += pinctrl-starfive.o
obj-$(CONFIG_PINCTRL_STMFX) += pinctrl-stmfx.o
obj-$(CONFIG_PINCTRL_ZYNQ) += pinctrl-zynq.o
obj-$(CONFIG_PINCTRL_ZYNQMP) += pinctrl-zynqmp.o
diff --git a/drivers/pinctrl/pinctrl-starfive.c b/drivers/pinctrl/pinctrl-starfive.c
new file mode 100644
index 0000000..0b91215
--- /dev/null
+++ b/drivers/pinctrl/pinctrl-starfive.c
@@ -0,0 +1,1354 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Pinctrl / GPIO driver for StarFive JH7100 SoC
+ *
+ * Copyright (C) 2020 Shanghai StarFive Technology Co., Ltd.
+ * Copyright (C) 2021 Emil Renner Berthing <kernel@esmil.dk>
+ */
+
+#include <linux/bits.h>
+#include <linux/clk.h>
+#include <linux/gpio/driver.h>
+#include <linux/io.h>
+#include <linux/mod_devicetable.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/platform_device.h>
+#include <linux/reset.h>
+#include <linux/spinlock.h>
+
+#include <linux/pinctrl/pinctrl.h>
+#include <linux/pinctrl/pinmux.h>
+
+#include <dt-bindings/pinctrl/pinctrl-starfive.h>
+
+#include "core.h"
+#include "pinctrl-utils.h"
+#include "pinmux.h"
+#include "pinconf.h"
+
+#define DRIVER_NAME "pinctrl-starfive"
+
+/*
+ * Refer to Section 12. GPIO Registers in the JH7100 data sheet:
+ * https://github.com/starfive-tech/JH7100_Docs
+ */
+#define NR_GPIOS 64
+
+/*
+ * Global enable for GPIO interrupts. If bit 0 is set to 1 the GPIO interrupts
+ * are enabled. If set to 0 the GPIO interrupts are disabled.
+ */
+#define GPIOEN 0x000
+
+/*
+ * The following 32-bit registers come in pairs, but only the offset of the
+ * first register is defined. The first controls (interrupts for) GPIO 0-31 and
+ * the second GPIO 32-63.
+ */
+
+/*
+ * Interrupt Type. If set to 1 the interrupt is edge-triggered. If set to 0 the
+ * interrupt is level-triggered.
+ */
+#define GPIOIS 0x010
+
+/*
+ * Edge-Trigger Interrupt Type. If set to 1 the interrupt gets triggered on
+ * both positive and negative edges. If set to 0 the interrupt is triggered by a
+ * single edge.
+ */
+#define GPIOIBE 0x018
+
+/*
+ * Interrupt Trigger Polarity. If set to 1 the interrupt is triggered on a
+ * rising edge (edge-triggered) or high level (level-triggered). If set to 0 the
+ * interrupt is triggered on a falling edge (edge-triggered) or low level
+ * (level-triggered).
+ */
+#define GPIOIEV 0x020
+
+/*
+ * Interrupt Mask. If set to 1 the interrupt is enabled (unmasked). If set to 0
+ * the interrupt is disabled (masked). Note that the current documentation is
+ * wrong and says the exct opposite of this.
+ */
+#define GPIOIE 0x028
+
+/*
+ * Clear Edge-Triggered Interrupts. Write a 1 to clear the edge-triggered
+ * interrupt.
+ */
+#define GPIOIC 0x030
+
+/*
+ * Edge-Triggered Interrupt Status. A 1 means the configured edge was detected.
+ */
+#define GPIORIS 0x038
+
+/*
+ * Interrupt Status after Masking. A 1 means the configured edge or level was
+ * detected and not masked.
+ */
+#define GPIOMIS 0x040
+
+/*
+ * Data Value. Dynamically reflects the value of the GPIO pin. If 1 the pin is
+ * a digital 1 and if 0 the pin is a digital 0.
+ */
+#define GPIODIN 0x048
+
+/*
+ * From the data sheet section 12.2, there are 64 32-bit output data registers
+ * and 64 output enable registers. Output data and output enable registers for
+ * a given GPIO are contiguous. Eg. GPO0_DOUT_CFG is 0x50 and GPO0_DOEN_CFG is
+ * 0x54 while GPO1_DOUT_CFG is 0x58 and GPO1_DOEN_CFG is 0x5c. The stride
+ * between GPIO registers is effectively 8, thus: GPOn_DOUT_CFG is 0x50 + 8n
+ * and GPOn_DOEN_CFG is 0x54 + 8n.
+ */
+#define GPON_DOUT_CFG 0x050
+#define GPON_DOEN_CFG 0x054
+
+/*
+ * From Section 12.3, there are 75 input signal configuration registers which
+ * are 4 bytes wide starting with GPI_CPU_JTAG_TCK_CFG at 0x250 and ending with
+ * GPI_USB_OVER_CURRENT_CFG 0x378
+ */
+#define GPI_CFG_OFFSET 0x250
+
+/*
+ * Pad Control Bits. There are 16 pad control bits for each pin located in 103
+ * 32-bit registers controlling PAD_GPIO[0] to PAD_GPIO[63] followed by
+ * PAD_FUNC_SHARE[0] to PAD_FUNC_SHARE[141]. Odd numbered pins use the upper 16
+ * bit of each register.
+ */
+#define PAD_SLEW_RATE_MASK GENMASK(11, 9)
+#define PAD_SLEW_RATE_POS 9
+#define PAD_BIAS_STRONG_PULL_UP BIT(8)
+#define PAD_INPUT_ENABLE BIT(7)
+#define PAD_INPUT_SCHMITT_ENABLE BIT(6)
+#define PAD_BIAS_DISABLE BIT(5)
+#define PAD_BIAS_PULL_DOWN BIT(4)
+#define PAD_BIAS_MASK \
+ (PAD_BIAS_STRONG_PULL_UP | \
+ PAD_BIAS_DISABLE | \
+ PAD_BIAS_PULL_DOWN)
+#define PAD_DRIVE_STRENGTH_MASK GENMASK(3, 0)
+#define PAD_DRIVE_STRENGTH_POS 0
+
+/*
+ * From Section 11, the IO_PADSHARE_SEL register can be programmed to select
+ * one of seven pre-defined multiplexed signal groups on PAD_FUNC_SHARE and
+ * PAD_GPIO pads. This is a global setting.
+ */
+#define IO_PADSHARE_SEL 0x1a0
+
+/*
+ * This just needs to be some number such that when
+ * sfp->gpio.pin_base = PAD_INVALID_GPIO then
+ * starfive_pin_to_gpio(sfp, validpin) is never a valid GPIO number.
+ * That is it should underflow and return something >= NR_GPIOS.
+ */
+#define PAD_INVALID_GPIO 0x10000
+
+/*
+ * The packed pinmux values from the device tree look like this:
+ *
+ * | 31 - 24 | 23 - 16 | 15 - 8 | 7 | 6 | 5 - 0 |
+ * | dout | doen | din | dout rev | doen rev | gpio nr |
+ *
+ * ..but the GPOn_DOUT_CFG and GPOn_DOEN_CFG registers look like this:
+ *
+ * | 31 | 30 - 8 | 7 - 0 |
+ * | dout/doen rev | unused | dout/doen |
+ */
+static unsigned int starfive_pinmux_to_gpio(u32 v)
+{
+ return v & (NR_GPIOS - 1);
+}
+
+static u32 starfive_pinmux_to_dout(u32 v)
+{
+ return ((v & BIT(7)) << (31 - 7)) | ((v >> 24) & GENMASK(7, 0));
+}
+
+static u32 starfive_pinmux_to_doen(u32 v)
+{
+ return ((v & BIT(6)) << (31 - 6)) | ((v >> 16) & GENMASK(7, 0));
+}
+
+static u32 starfive_pinmux_to_din(u32 v)
+{
+ return (v >> 8) & GENMASK(7, 0);
+}
+
+/*
+ * The maximum GPIO output current depends on the chosen drive strength:
+ *
+ * DS: 0 1 2 3 4 5 6 7
+ * mA: 14.2 21.2 28.2 35.2 42.2 49.1 56.0 62.8
+ *
+ * After rounding that is 7*DS + 14 mA
+ */
+static u32 starfive_drive_strength_to_max_mA(u16 ds)
+{
+ return 7 * ds + 14;
+}
+
+static u16 starfive_drive_strength_from_max_mA(u32 i)
+{
+ return (clamp(i, 14U, 63U) - 14) / 7;
+}
+
+struct starfive_pinctrl {
+ struct gpio_chip gc;
+ struct pinctrl_gpio_range gpios;
+ raw_spinlock_t lock;
+ void __iomem *base;
+ void __iomem *padctl;
+ struct pinctrl_dev *pctl;
+};
+
+static inline unsigned int starfive_pin_to_gpio(const struct starfive_pinctrl *sfp,
+ unsigned int pin)
+{
+ return pin - sfp->gpios.pin_base;
+}
+
+static inline unsigned int starfive_gpio_to_pin(const struct starfive_pinctrl *sfp,
+ unsigned int gpio)
+{
+ return sfp->gpios.pin_base + gpio;
+}
+
+static struct starfive_pinctrl *starfive_from_irq_data(struct irq_data *d)
+{
+ struct gpio_chip *gc = irq_data_get_irq_chip_data(d);
+
+ return container_of(gc, struct starfive_pinctrl, gc);
+}
+
+static struct starfive_pinctrl *starfive_from_irq_desc(struct irq_desc *desc)
+{
+ struct gpio_chip *gc = irq_desc_get_handler_data(desc);
+
+ return container_of(gc, struct starfive_pinctrl, gc);
+}
+
+static const struct pinctrl_pin_desc starfive_pins[] = {
+ PINCTRL_PIN(PAD_GPIO(0), "GPIO[0]"),
+ PINCTRL_PIN(PAD_GPIO(1), "GPIO[1]"),
+ PINCTRL_PIN(PAD_GPIO(2), "GPIO[2]"),
+ PINCTRL_PIN(PAD_GPIO(3), "GPIO[3]"),
+ PINCTRL_PIN(PAD_GPIO(4), "GPIO[4]"),
+ PINCTRL_PIN(PAD_GPIO(5), "GPIO[5]"),
+ PINCTRL_PIN(PAD_GPIO(6), "GPIO[6]"),
+ PINCTRL_PIN(PAD_GPIO(7), "GPIO[7]"),
+ PINCTRL_PIN(PAD_GPIO(8), "GPIO[8]"),
+ PINCTRL_PIN(PAD_GPIO(9), "GPIO[9]"),
+ PINCTRL_PIN(PAD_GPIO(10), "GPIO[10]"),
+ PINCTRL_PIN(PAD_GPIO(11), "GPIO[11]"),
+ PINCTRL_PIN(PAD_GPIO(12), "GPIO[12]"),
+ PINCTRL_PIN(PAD_GPIO(13), "GPIO[13]"),
+ PINCTRL_PIN(PAD_GPIO(14), "GPIO[14]"),
+ PINCTRL_PIN(PAD_GPIO(15), "GPIO[15]"),
+ PINCTRL_PIN(PAD_GPIO(16), "GPIO[16]"),
+ PINCTRL_PIN(PAD_GPIO(17), "GPIO[17]"),
+ PINCTRL_PIN(PAD_GPIO(18), "GPIO[18]"),
+ PINCTRL_PIN(PAD_GPIO(19), "GPIO[19]"),
+ PINCTRL_PIN(PAD_GPIO(20), "GPIO[20]"),
+ PINCTRL_PIN(PAD_GPIO(21), "GPIO[21]"),
+ PINCTRL_PIN(PAD_GPIO(22), "GPIO[22]"),
+ PINCTRL_PIN(PAD_GPIO(23), "GPIO[23]"),
+ PINCTRL_PIN(PAD_GPIO(24), "GPIO[24]"),
+ PINCTRL_PIN(PAD_GPIO(25), "GPIO[25]"),
+ PINCTRL_PIN(PAD_GPIO(26), "GPIO[26]"),
+ PINCTRL_PIN(PAD_GPIO(27), "GPIO[27]"),
+ PINCTRL_PIN(PAD_GPIO(28), "GPIO[28]"),
+ PINCTRL_PIN(PAD_GPIO(29), "GPIO[29]"),
+ PINCTRL_PIN(PAD_GPIO(30), "GPIO[30]"),
+ PINCTRL_PIN(PAD_GPIO(31), "GPIO[31]"),
+ PINCTRL_PIN(PAD_GPIO(32), "GPIO[32]"),
+ PINCTRL_PIN(PAD_GPIO(33), "GPIO[33]"),
+ PINCTRL_PIN(PAD_GPIO(34), "GPIO[34]"),
+ PINCTRL_PIN(PAD_GPIO(35), "GPIO[35]"),
+ PINCTRL_PIN(PAD_GPIO(36), "GPIO[36]"),
+ PINCTRL_PIN(PAD_GPIO(37), "GPIO[37]"),
+ PINCTRL_PIN(PAD_GPIO(38), "GPIO[38]"),
+ PINCTRL_PIN(PAD_GPIO(39), "GPIO[39]"),
+ PINCTRL_PIN(PAD_GPIO(40), "GPIO[40]"),
+ PINCTRL_PIN(PAD_GPIO(41), "GPIO[41]"),
+ PINCTRL_PIN(PAD_GPIO(42), "GPIO[42]"),
+ PINCTRL_PIN(PAD_GPIO(43), "GPIO[43]"),
+ PINCTRL_PIN(PAD_GPIO(44), "GPIO[44]"),
+ PINCTRL_PIN(PAD_GPIO(45), "GPIO[45]"),
+ PINCTRL_PIN(PAD_GPIO(46), "GPIO[46]"),
+ PINCTRL_PIN(PAD_GPIO(47), "GPIO[47]"),
+ PINCTRL_PIN(PAD_GPIO(48), "GPIO[48]"),
+ PINCTRL_PIN(PAD_GPIO(49), "GPIO[49]"),
+ PINCTRL_PIN(PAD_GPIO(50), "GPIO[50]"),
+ PINCTRL_PIN(PAD_GPIO(51), "GPIO[51]"),
+ PINCTRL_PIN(PAD_GPIO(52), "GPIO[52]"),
+ PINCTRL_PIN(PAD_GPIO(53), "GPIO[53]"),
+ PINCTRL_PIN(PAD_GPIO(54), "GPIO[54]"),
+ PINCTRL_PIN(PAD_GPIO(55), "GPIO[55]"),
+ PINCTRL_PIN(PAD_GPIO(56), "GPIO[56]"),
+ PINCTRL_PIN(PAD_GPIO(57), "GPIO[57]"),
+ PINCTRL_PIN(PAD_GPIO(58), "GPIO[58]"),
+ PINCTRL_PIN(PAD_GPIO(59), "GPIO[59]"),
+ PINCTRL_PIN(PAD_GPIO(60), "GPIO[60]"),
+ PINCTRL_PIN(PAD_GPIO(61), "GPIO[61]"),
+ PINCTRL_PIN(PAD_GPIO(62), "GPIO[62]"),
+ PINCTRL_PIN(PAD_GPIO(63), "GPIO[63]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(0), "FUNC_SHARE[0]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(1), "FUNC_SHARE[1]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(2), "FUNC_SHARE[2]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(3), "FUNC_SHARE[3]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(4), "FUNC_SHARE[4]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(5), "FUNC_SHARE[5]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(6), "FUNC_SHARE[6]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(7), "FUNC_SHARE[7]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(8), "FUNC_SHARE[8]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(9), "FUNC_SHARE[9]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(10), "FUNC_SHARE[10]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(11), "FUNC_SHARE[11]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(12), "FUNC_SHARE[12]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(13), "FUNC_SHARE[13]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(14), "FUNC_SHARE[14]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(15), "FUNC_SHARE[15]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(16), "FUNC_SHARE[16]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(17), "FUNC_SHARE[17]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(18), "FUNC_SHARE[18]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(19), "FUNC_SHARE[19]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(20), "FUNC_SHARE[20]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(21), "FUNC_SHARE[21]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(22), "FUNC_SHARE[22]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(23), "FUNC_SHARE[23]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(24), "FUNC_SHARE[24]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(25), "FUNC_SHARE[25]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(26), "FUNC_SHARE[26]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(27), "FUNC_SHARE[27]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(28), "FUNC_SHARE[28]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(29), "FUNC_SHARE[29]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(30), "FUNC_SHARE[30]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(31), "FUNC_SHARE[31]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(32), "FUNC_SHARE[32]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(33), "FUNC_SHARE[33]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(34), "FUNC_SHARE[34]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(35), "FUNC_SHARE[35]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(36), "FUNC_SHARE[36]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(37), "FUNC_SHARE[37]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(38), "FUNC_SHARE[38]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(39), "FUNC_SHARE[39]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(40), "FUNC_SHARE[40]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(41), "FUNC_SHARE[41]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(42), "FUNC_SHARE[42]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(43), "FUNC_SHARE[43]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(44), "FUNC_SHARE[44]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(45), "FUNC_SHARE[45]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(46), "FUNC_SHARE[46]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(47), "FUNC_SHARE[47]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(48), "FUNC_SHARE[48]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(49), "FUNC_SHARE[49]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(50), "FUNC_SHARE[50]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(51), "FUNC_SHARE[51]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(52), "FUNC_SHARE[52]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(53), "FUNC_SHARE[53]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(54), "FUNC_SHARE[54]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(55), "FUNC_SHARE[55]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(56), "FUNC_SHARE[56]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(57), "FUNC_SHARE[57]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(58), "FUNC_SHARE[58]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(59), "FUNC_SHARE[59]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(60), "FUNC_SHARE[60]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(61), "FUNC_SHARE[61]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(62), "FUNC_SHARE[62]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(63), "FUNC_SHARE[63]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(64), "FUNC_SHARE[64]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(65), "FUNC_SHARE[65]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(66), "FUNC_SHARE[66]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(67), "FUNC_SHARE[67]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(68), "FUNC_SHARE[68]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(69), "FUNC_SHARE[69]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(70), "FUNC_SHARE[70]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(71), "FUNC_SHARE[71]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(72), "FUNC_SHARE[72]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(73), "FUNC_SHARE[73]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(74), "FUNC_SHARE[74]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(75), "FUNC_SHARE[75]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(76), "FUNC_SHARE[76]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(77), "FUNC_SHARE[77]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(78), "FUNC_SHARE[78]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(79), "FUNC_SHARE[79]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(80), "FUNC_SHARE[80]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(81), "FUNC_SHARE[81]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(82), "FUNC_SHARE[82]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(83), "FUNC_SHARE[83]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(84), "FUNC_SHARE[84]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(85), "FUNC_SHARE[85]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(86), "FUNC_SHARE[86]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(87), "FUNC_SHARE[87]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(88), "FUNC_SHARE[88]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(89), "FUNC_SHARE[89]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(90), "FUNC_SHARE[90]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(91), "FUNC_SHARE[91]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(92), "FUNC_SHARE[92]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(93), "FUNC_SHARE[93]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(94), "FUNC_SHARE[94]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(95), "FUNC_SHARE[95]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(96), "FUNC_SHARE[96]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(97), "FUNC_SHARE[97]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(98), "FUNC_SHARE[98]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(99), "FUNC_SHARE[99]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(100), "FUNC_SHARE[100]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(101), "FUNC_SHARE[101]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(102), "FUNC_SHARE[102]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(103), "FUNC_SHARE[103]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(104), "FUNC_SHARE[104]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(105), "FUNC_SHARE[105]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(106), "FUNC_SHARE[106]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(107), "FUNC_SHARE[107]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(108), "FUNC_SHARE[108]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(109), "FUNC_SHARE[109]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(110), "FUNC_SHARE[110]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(111), "FUNC_SHARE[111]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(112), "FUNC_SHARE[112]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(113), "FUNC_SHARE[113]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(114), "FUNC_SHARE[114]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(115), "FUNC_SHARE[115]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(116), "FUNC_SHARE[116]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(117), "FUNC_SHARE[117]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(118), "FUNC_SHARE[118]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(119), "FUNC_SHARE[119]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(120), "FUNC_SHARE[120]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(121), "FUNC_SHARE[121]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(122), "FUNC_SHARE[122]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(123), "FUNC_SHARE[123]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(124), "FUNC_SHARE[124]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(125), "FUNC_SHARE[125]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(126), "FUNC_SHARE[126]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(127), "FUNC_SHARE[127]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(128), "FUNC_SHARE[128]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(129), "FUNC_SHARE[129]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(130), "FUNC_SHARE[130]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(131), "FUNC_SHARE[131]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(132), "FUNC_SHARE[132]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(133), "FUNC_SHARE[133]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(134), "FUNC_SHARE[134]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(135), "FUNC_SHARE[135]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(136), "FUNC_SHARE[136]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(137), "FUNC_SHARE[137]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(138), "FUNC_SHARE[138]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(139), "FUNC_SHARE[139]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(140), "FUNC_SHARE[140]"),
+ PINCTRL_PIN(PAD_FUNC_SHARE(141), "FUNC_SHARE[141]"),
+};
+
+#ifdef CONFIG_DEBUG_FS
+static void starfive_pin_dbg_show(struct pinctrl_dev *pctldev,
+ struct seq_file *s,
+ unsigned int pin)
+{
+ struct starfive_pinctrl *sfp = pinctrl_dev_get_drvdata(pctldev);
+ unsigned int gpio = starfive_pin_to_gpio(sfp, pin);
+ void __iomem *reg;
+ u32 dout, doen;
+
+ if (gpio >= NR_GPIOS)
+ return;
+
+ reg = sfp->base + GPON_DOUT_CFG + 8 * gpio;
+ dout = readl_relaxed(reg + 0x000);
+ doen = readl_relaxed(reg + 0x004);
+
+ seq_printf(s, "dout=%lu%s doen=%lu%s",
+ dout & GENMASK(7, 0), (dout & BIT(31)) ? "r" : "",
+ doen & GENMASK(7, 0), (doen & BIT(31)) ? "r" : "");
+}
+#else
+#define starfive_pin_dbg_show NULL
+#endif
+
+static int starfive_dt_node_to_map(struct pinctrl_dev *pctldev,
+ struct device_node *np,
+ struct pinctrl_map **maps,
+ unsigned int *num_maps)
+{
+ struct starfive_pinctrl *sfp = pinctrl_dev_get_drvdata(pctldev);
+ struct device *dev = sfp->gc.parent;
+ struct device_node *child;
+ struct pinctrl_map *map;
+ const char **pgnames;
+ const char *grpname;
+ u32 *pinmux;
+ int ngroups;
+ int *pins;
+ int nmaps;
+ int ret;
+
+ nmaps = 0;
+ ngroups = 0;
+ for_each_child_of_node(np, child) {
+ int npinmux = of_property_count_u32_elems(child, "pinmux");
+ int npins = of_property_count_u32_elems(child, "pins");
+
+ if (npinmux > 0 && npins > 0) {
+ dev_err(dev, "invalid pinctrl group %pOFn.%pOFn: both pinmux and pins set\n",
+ np, child);
+ of_node_put(child);
+ return -EINVAL;
+ }
+ if (npinmux == 0 && npins == 0) {
+ dev_err(dev, "invalid pinctrl group %pOFn.%pOFn: neither pinmux nor pins set\n",
+ np, child);
+ of_node_put(child);
+ return -EINVAL;
+ }
+
+ if (npinmux > 0)
+ nmaps += 2;
+ else
+ nmaps += 1;
+ ngroups += 1;
+ }
+
+ pgnames = devm_kcalloc(dev, ngroups, sizeof(*pgnames), GFP_KERNEL);
+ if (!pgnames)
+ return -ENOMEM;
+
+ map = kcalloc(nmaps, sizeof(*map), GFP_KERNEL);
+ if (!map)
+ return -ENOMEM;
+
+ nmaps = 0;
+ ngroups = 0;
+ for_each_child_of_node(np, child) {
+ int npins;
+ int i;
+
+ grpname = devm_kasprintf(dev, GFP_KERNEL, "%pOFn.%pOFn", np, child);
+ if (!grpname) {
+ ret = -ENOMEM;
+ goto put_child;
+ }
+
+ pgnames[ngroups++] = grpname;
+
+ if ((npins = of_property_count_u32_elems(child, "pinmux")) > 0) {
+ pins = devm_kcalloc(dev, npins, sizeof(*pins), GFP_KERNEL);
+ if (!pins) {
+ ret = -ENOMEM;
+ goto put_child;
+ }
+
+ pinmux = devm_kcalloc(dev, npins, sizeof(*pinmux), GFP_KERNEL);
+ if (!pinmux) {
+ ret = -ENOMEM;
+ goto put_child;
+ }
+
+ ret = of_property_read_u32_array(child, "pinmux", pinmux, npins);
+ if (ret)
+ goto put_child;
+
+ for (i = 0; i < npins; i++) {
+ unsigned int gpio = starfive_pinmux_to_gpio(pinmux[i]);
+
+ pins[i] = starfive_gpio_to_pin(sfp, gpio);
+ }
+
+ map[nmaps].type = PIN_MAP_TYPE_MUX_GROUP;
+ map[nmaps].data.mux.function = np->name;
+ map[nmaps].data.mux.group = grpname;
+ nmaps += 1;
+ } else if ((npins = of_property_count_u32_elems(child, "pins")) > 0) {
+ pins = devm_kcalloc(dev, npins, sizeof(*pins), GFP_KERNEL);
+ if (!pins) {
+ ret = -ENOMEM;
+ goto put_child;
+ }
+
+ pinmux = NULL;
+
+ for (i = 0; i < npins; i++) {
+ u32 v;
+
+ ret = of_property_read_u32_index(child, "pins", i, &v);
+ if (ret)
+ goto put_child;
+ pins[i] = v;
+ }
+ } else {
+ ret = -EINVAL;
+ goto put_child;
+ }
+
+ ret = pinctrl_generic_add_group(pctldev, grpname, pins, npins, pinmux);
+ if (ret < 0) {
+ dev_err(dev, "error adding group %s: %d\n", grpname, ret);
+ goto put_child;
+ }
+
+ ret = pinconf_generic_parse_dt_config(child, pctldev,
+ &map[nmaps].data.configs.configs,
+ &map[nmaps].data.configs.num_configs);
+ if (ret) {
+ dev_err(dev, "error parsing pin config of group %s: %d\n",
+ grpname, ret);
+ goto put_child;
+ }
+
+ /* don't create a map if there are no pinconf settings */
+ if (map[nmaps].data.configs.num_configs == 0)
+ continue;
+
+ map[nmaps].type = PIN_MAP_TYPE_CONFIGS_GROUP;
+ map[nmaps].data.configs.group_or_pin = grpname;
+ nmaps += 1;
+ }
+
+ ret = pinmux_generic_add_function(pctldev, np->name, pgnames, ngroups, NULL);
+ if (ret < 0) {
+ dev_err(dev, "error adding function %s: %d\n", np->name, ret);
+ goto free_map;
+ }
+
+ *maps = map;
+ *num_maps = nmaps;
+ return 0;
+
+put_child:
+ of_node_put(child);
+free_map:
+ pinctrl_utils_free_map(pctldev, map, nmaps);
+ return ret;
+}
+
+static const struct pinctrl_ops starfive_pinctrl_ops = {
+ .get_groups_count = pinctrl_generic_get_group_count,
+ .get_group_name = pinctrl_generic_get_group_name,
+ .get_group_pins = pinctrl_generic_get_group_pins,
+ .pin_dbg_show = starfive_pin_dbg_show,
+ .dt_node_to_map = starfive_dt_node_to_map,
+ .dt_free_map = pinctrl_utils_free_map,
+};
+
+static int starfive_set_mux(struct pinctrl_dev *pctldev,
+ unsigned int fsel, unsigned int gsel)
+{
+ struct starfive_pinctrl *sfp = pinctrl_dev_get_drvdata(pctldev);
+ struct device *dev = sfp->gc.parent;
+ const struct group_desc *group;
+ const u32 *pinmux;
+ unsigned int i;
+
+ group = pinctrl_generic_get_group(pctldev, gsel);
+ if (!group)
+ return -EINVAL;
+
+ pinmux = group->data;
+ for (i = 0; i < group->num_pins; i++) {
+ u32 v = pinmux[i];
+ unsigned int gpio = starfive_pinmux_to_gpio(v);
+ u32 dout = starfive_pinmux_to_dout(v);
+ u32 doen = starfive_pinmux_to_doen(v);
+ u32 din = starfive_pinmux_to_din(v);
+ void __iomem *reg_dout;
+ void __iomem *reg_doen;
+ void __iomem *reg_din;
+ unsigned long flags;
+
+ dev_dbg(dev, "GPIO%u: dout=0x%x doen=0x%x din=0x%x\n",
+ gpio, dout, doen, din);
+
+ reg_dout = sfp->base + GPON_DOUT_CFG + 8 * gpio;
+ reg_doen = sfp->base + GPON_DOEN_CFG + 8 * gpio;
+ if (din != GPI_NONE)
+ reg_din = sfp->base + GPI_CFG_OFFSET + 4 * din;
+ else
+ reg_din = NULL;
+
+ raw_spin_lock_irqsave(&sfp->lock, flags);
+ writel_relaxed(dout, reg_dout);
+ writel_relaxed(doen, reg_doen);
+ if (reg_din)
+ writel_relaxed(gpio + 2, reg_din);
+ raw_spin_unlock_irqrestore(&sfp->lock, flags);
+ }
+
+ return 0;
+}
+
+static const struct pinmux_ops starfive_pinmux_ops = {
+ .get_functions_count = pinmux_generic_get_function_count,
+ .get_function_name = pinmux_generic_get_function_name,
+ .get_function_groups = pinmux_generic_get_function_groups,
+ .set_mux = starfive_set_mux,
+ .strict = true,
+};
+
+static u16 starfive_padctl_get(struct starfive_pinctrl *sfp,
+ unsigned int pin)
+{
+ void __iomem *reg = sfp->padctl + 4 * (pin / 2);
+ int shift = 16 * (pin % 2);
+
+ return readl_relaxed(reg) >> shift;
+}
+
+static void starfive_padctl_rmw(struct starfive_pinctrl *sfp,
+ unsigned int pin,
+ u16 _mask, u16 _value)
+{
+ void __iomem *reg = sfp->padctl + 4 * (pin / 2);
+ int shift = 16 * (pin % 2);
+ u32 mask = (u32)_mask << shift;
+ u32 value = (u32)_value << shift;
+ unsigned long flags;
+
+ dev_dbg(sfp->gc.parent, "padctl_rmw(%u, 0x%03x, 0x%03x)\n", pin, _mask, _value);
+
+ raw_spin_lock_irqsave(&sfp->lock, flags);
+ value |= readl_relaxed(reg) & ~mask;
+ writel_relaxed(value, reg);
+ raw_spin_unlock_irqrestore(&sfp->lock, flags);
+}
+
+#define PIN_CONFIG_STARFIVE_STRONG_PULL_UP (PIN_CONFIG_END + 1)
+
+static const struct pinconf_generic_params starfive_pinconf_custom_params[] = {
+ { "starfive,strong-pull-up", PIN_CONFIG_STARFIVE_STRONG_PULL_UP, 1 },
+};
+
+#ifdef CONFIG_DEBUG_FS
+static const struct pin_config_item starfive_pinconf_custom_conf_items[] = {
+ PCONFDUMP(PIN_CONFIG_STARFIVE_STRONG_PULL_UP, "input bias strong pull-up", NULL, false),
+};
+
+static_assert(ARRAY_SIZE(starfive_pinconf_custom_conf_items) ==
+ ARRAY_SIZE(starfive_pinconf_custom_params));
+#else
+#define starfive_pinconf_custom_conf_items NULL
+#endif
+
+static int starfive_pinconf_get(struct pinctrl_dev *pctldev,
+ unsigned int pin, unsigned long *config)
+{
+ struct starfive_pinctrl *sfp = pinctrl_dev_get_drvdata(pctldev);
+ int param = pinconf_to_config_param(*config);
+ u16 value = starfive_padctl_get(sfp, pin);
+ bool enabled;
+ u32 arg;
+
+ switch (param) {
+ case PIN_CONFIG_BIAS_DISABLE:
+ enabled = value & PAD_BIAS_DISABLE;
+ arg = 0;
+ break;
+ case PIN_CONFIG_BIAS_PULL_DOWN:
+ enabled = value & PAD_BIAS_PULL_DOWN;
+ arg = 1;
+ break;
+ case PIN_CONFIG_BIAS_PULL_UP:
+ enabled = !(value & PAD_BIAS_MASK);
+ arg = 1;
+ break;
+ case PIN_CONFIG_DRIVE_STRENGTH:
+ enabled = value & PAD_DRIVE_STRENGTH_MASK;
+ arg = starfive_drive_strength_to_max_mA(value & PAD_DRIVE_STRENGTH_MASK);
+ break;
+ case PIN_CONFIG_INPUT_ENABLE:
+ enabled = value & PAD_INPUT_ENABLE;
+ arg = enabled;
+ break;
+ case PIN_CONFIG_INPUT_SCHMITT_ENABLE:
+ enabled = value & PAD_INPUT_SCHMITT_ENABLE;
+ arg = enabled;
+ break;
+ case PIN_CONFIG_SLEW_RATE:
+ enabled = value & PAD_SLEW_RATE_MASK;
+ arg = (value & PAD_SLEW_RATE_MASK) >> PAD_SLEW_RATE_POS;
+ break;
+ case PIN_CONFIG_STARFIVE_STRONG_PULL_UP:
+ enabled = value & PAD_BIAS_STRONG_PULL_UP;
+ arg = enabled;
+ break;
+ default:
+ return -ENOTSUPP;
+ }
+
+ *config = pinconf_to_config_packed(param, arg);
+ return enabled ? 0 : -EINVAL;
+}
+
+static int starfive_pinconf_group_get(struct pinctrl_dev *pctldev,
+ unsigned int gsel, unsigned long *config)
+{
+ const struct group_desc *group;
+
+ group = pinctrl_generic_get_group(pctldev, gsel);
+ if (!group)
+ return -EINVAL;
+
+ return starfive_pinconf_get(pctldev, group->pins[0], config);
+}
+
+static int starfive_pinconf_group_set(struct pinctrl_dev *pctldev,
+ unsigned int gsel,
+ unsigned long *configs,
+ unsigned int num_configs)
+{
+ struct starfive_pinctrl *sfp = pinctrl_dev_get_drvdata(pctldev);
+ const struct group_desc *group;
+ u16 mask, value;
+ int i;
+
+ group = pinctrl_generic_get_group(pctldev, gsel);
+ if (!group)
+ return -EINVAL;
+
+ mask = 0;
+ value = 0;
+ for (i = 0; i < num_configs; i++) {
+ int param = pinconf_to_config_param(configs[i]);
+ u32 arg = pinconf_to_config_argument(configs[i]);
+
+ switch (param) {
+ case PIN_CONFIG_BIAS_DISABLE:
+ mask |= PAD_BIAS_MASK;
+ value = (value & ~PAD_BIAS_MASK) | PAD_BIAS_DISABLE;
+ break;
+ case PIN_CONFIG_BIAS_PULL_DOWN:
+ if (arg == 0)
+ return -ENOTSUPP;
+ mask |= PAD_BIAS_MASK;
+ value = (value & ~PAD_BIAS_MASK) | PAD_BIAS_PULL_DOWN;
+ break;
+ case PIN_CONFIG_BIAS_PULL_UP:
+ if (arg == 0)
+ return -ENOTSUPP;
+ mask |= PAD_BIAS_MASK;
+ value = value & ~PAD_BIAS_MASK;
+ break;
+ case PIN_CONFIG_DRIVE_STRENGTH:
+ mask |= PAD_DRIVE_STRENGTH_MASK;
+ value = (value & ~PAD_DRIVE_STRENGTH_MASK) |
+ starfive_drive_strength_from_max_mA(arg);
+ break;
+ case PIN_CONFIG_INPUT_ENABLE:
+ mask |= PAD_INPUT_ENABLE;
+ if (arg)
+ value |= PAD_INPUT_ENABLE;
+ else
+ value &= ~PAD_INPUT_ENABLE;
+ break;
+ case PIN_CONFIG_INPUT_SCHMITT_ENABLE:
+ mask |= PAD_INPUT_SCHMITT_ENABLE;
+ if (arg)
+ value |= PAD_INPUT_SCHMITT_ENABLE;
+ else
+ value &= ~PAD_INPUT_SCHMITT_ENABLE;
+ break;
+ case PIN_CONFIG_SLEW_RATE:
+ mask |= PAD_SLEW_RATE_MASK;
+ value = (value & ~PAD_SLEW_RATE_MASK) |
+ ((arg << PAD_SLEW_RATE_POS) & PAD_SLEW_RATE_MASK);
+ break;
+ case PIN_CONFIG_STARFIVE_STRONG_PULL_UP:
+ if (arg) {
+ mask |= PAD_BIAS_MASK;
+ value = (value & ~PAD_BIAS_MASK) |
+ PAD_BIAS_STRONG_PULL_UP;
+ } else {
+ mask |= PAD_BIAS_STRONG_PULL_UP;
+ value = value & ~PAD_BIAS_STRONG_PULL_UP;
+ }
+ break;
+ default:
+ return -ENOTSUPP;
+ }
+ }
+
+ for (i = 0; i < group->num_pins; i++)
+ starfive_padctl_rmw(sfp, group->pins[i], mask, value);
+
+ return 0;
+}
+
+#ifdef CONFIG_DEBUG_FS
+static void starfive_pinconf_dbg_show(struct pinctrl_dev *pctldev,
+ struct seq_file *s, unsigned int pin)
+{
+ struct starfive_pinctrl *sfp = pinctrl_dev_get_drvdata(pctldev);
+ u16 value = starfive_padctl_get(sfp, pin);
+
+ seq_printf(s, " (0x%03x)", value);
+}
+#else
+#define starfive_pinconf_dbg_show NULL
+#endif
+
+static const struct pinconf_ops starfive_pinconf_ops = {
+ .pin_config_get = starfive_pinconf_get,
+ .pin_config_group_get = starfive_pinconf_group_get,
+ .pin_config_group_set = starfive_pinconf_group_set,
+ .pin_config_dbg_show = starfive_pinconf_dbg_show,
+ .is_generic = true,
+};
+
+static struct pinctrl_desc starfive_desc = {
+ .name = DRIVER_NAME,
+ .pins = starfive_pins,
+ .npins = ARRAY_SIZE(starfive_pins),
+ .pctlops = &starfive_pinctrl_ops,
+ .pmxops = &starfive_pinmux_ops,
+ .confops = &starfive_pinconf_ops,
+ .owner = THIS_MODULE,
+ .num_custom_params = ARRAY_SIZE(starfive_pinconf_custom_params),
+ .custom_params = starfive_pinconf_custom_params,
+ .custom_conf_items = starfive_pinconf_custom_conf_items,
+};
+
+static int starfive_gpio_request(struct gpio_chip *gc, unsigned int gpio)
+{
+ return pinctrl_gpio_request(gc->base + gpio);
+}
+
+static void starfive_gpio_free(struct gpio_chip *gc, unsigned int gpio)
+{
+ pinctrl_gpio_free(gc->base + gpio);
+}
+
+static int starfive_gpio_get_direction(struct gpio_chip *gc, unsigned int gpio)
+{
+ struct starfive_pinctrl *sfp = container_of(gc, struct starfive_pinctrl, gc);
+ void __iomem *doen = sfp->base + GPON_DOEN_CFG + 8 * gpio;
+
+ if (readl_relaxed(doen) == GPO_ENABLE)
+ return GPIO_LINE_DIRECTION_OUT;
+
+ return GPIO_LINE_DIRECTION_IN;
+}
+
+static int starfive_gpio_direction_input(struct gpio_chip *gc,
+ unsigned int gpio)
+{
+ struct starfive_pinctrl *sfp = container_of(gc, struct starfive_pinctrl, gc);
+ void __iomem *doen = sfp->base + GPON_DOEN_CFG + 8 * gpio;
+ unsigned long flags;
+
+ /* enable input and schmitt trigger */
+ starfive_padctl_rmw(sfp, starfive_gpio_to_pin(sfp, gpio),
+ PAD_INPUT_ENABLE | PAD_INPUT_SCHMITT_ENABLE,
+ PAD_INPUT_ENABLE | PAD_INPUT_SCHMITT_ENABLE);
+
+ raw_spin_lock_irqsave(&sfp->lock, flags);
+ writel_relaxed(GPO_DISABLE, doen);
+ raw_spin_unlock_irqrestore(&sfp->lock, flags);
+ return 0;
+}
+
+static int starfive_gpio_direction_output(struct gpio_chip *gc,
+ unsigned int gpio, int value)
+{
+ struct starfive_pinctrl *sfp = container_of(gc, struct starfive_pinctrl, gc);
+ void __iomem *dout = sfp->base + GPON_DOUT_CFG + 8 * gpio;
+ void __iomem *doen = sfp->base + GPON_DOEN_CFG + 8 * gpio;
+ unsigned long flags;
+
+ raw_spin_lock_irqsave(&sfp->lock, flags);
+ writel_relaxed(value, dout);
+ writel_relaxed(GPO_ENABLE, doen);
+ raw_spin_unlock_irqrestore(&sfp->lock, flags);
+
+ /* disable input, schmitt trigger and bias */
+ starfive_padctl_rmw(sfp, starfive_gpio_to_pin(sfp, gpio),
+ PAD_BIAS_MASK | PAD_INPUT_ENABLE | PAD_INPUT_SCHMITT_ENABLE,
+ PAD_BIAS_DISABLE);
+
+ return 0;
+}
+
+static int starfive_gpio_get(struct gpio_chip *gc, unsigned int gpio)
+{
+ struct starfive_pinctrl *sfp = container_of(gc, struct starfive_pinctrl, gc);
+ void __iomem *din = sfp->base + GPIODIN + 4 * (gpio / 32);
+
+ return !!(readl_relaxed(din) & BIT(gpio % 32));
+}
+
+static void starfive_gpio_set(struct gpio_chip *gc, unsigned int gpio,
+ int value)
+{
+ struct starfive_pinctrl *sfp = container_of(gc, struct starfive_pinctrl, gc);
+ void __iomem *dout = sfp->base + GPON_DOUT_CFG + 8 * gpio;
+ unsigned long flags;
+
+ raw_spin_lock_irqsave(&sfp->lock, flags);
+ writel_relaxed(value, dout);
+ raw_spin_unlock_irqrestore(&sfp->lock, flags);
+}
+
+static int starfive_gpio_set_config(struct gpio_chip *gc, unsigned int gpio,
+ unsigned long config)
+{
+ struct starfive_pinctrl *sfp = container_of(gc, struct starfive_pinctrl, gc);
+ u32 arg = pinconf_to_config_argument(config);
+ u16 value;
+ u16 mask;
+
+ switch (pinconf_to_config_param(config)) {
+ case PIN_CONFIG_BIAS_DISABLE:
+ mask = PAD_BIAS_MASK;
+ value = PAD_BIAS_DISABLE;
+ break;
+ case PIN_CONFIG_BIAS_PULL_DOWN:
+ if (arg == 0)
+ return -ENOTSUPP;
+ mask = PAD_BIAS_MASK;
+ value = PAD_BIAS_PULL_DOWN;
+ break;
+ case PIN_CONFIG_BIAS_PULL_UP:
+ if (arg == 0)
+ return -ENOTSUPP;
+ mask = PAD_BIAS_MASK;
+ value = 0;
+ break;
+ case PIN_CONFIG_DRIVE_PUSH_PULL:
+ return 0;
+ case PIN_CONFIG_INPUT_ENABLE:
+ mask = PAD_INPUT_ENABLE;
+ value = arg ? PAD_INPUT_ENABLE : 0;
+ break;
+ case PIN_CONFIG_INPUT_SCHMITT_ENABLE:
+ mask = PAD_INPUT_SCHMITT_ENABLE;
+ value = arg ? PAD_INPUT_SCHMITT_ENABLE : 0;
+ break;
+ default:
+ return -ENOTSUPP;
+ };
+
+ starfive_padctl_rmw(sfp, starfive_gpio_to_pin(sfp, gpio), mask, value);
+ return 0;
+}
+
+static int starfive_gpio_add_pin_ranges(struct gpio_chip *gc)
+{
+ struct starfive_pinctrl *sfp = container_of(gc, struct starfive_pinctrl, gc);
+
+ sfp->gpios.name = sfp->gc.label;
+ sfp->gpios.base = sfp->gc.base;
+ /*
+ * sfp->gpios.pin_base depends on the chosen signal group
+ * and is set in starfive_probe()
+ */
+ sfp->gpios.npins = NR_GPIOS;
+ sfp->gpios.gc = &sfp->gc;
+ pinctrl_add_gpio_range(sfp->pctl, &sfp->gpios);
+ return 0;
+}
+
+static void starfive_irq_ack(struct irq_data *d)
+{
+ struct starfive_pinctrl *sfp = starfive_from_irq_data(d);
+ irq_hw_number_t gpio = irqd_to_hwirq(d);
+ void __iomem *ic = sfp->base + GPIOIC + 4 * (gpio / 32);
+ u32 mask = BIT(gpio % 32);
+ unsigned long flags;
+
+ raw_spin_lock_irqsave(&sfp->lock, flags);
+ writel_relaxed(mask, ic);
+ raw_spin_unlock_irqrestore(&sfp->lock, flags);
+}
+
+static void starfive_irq_mask(struct irq_data *d)
+{
+ struct starfive_pinctrl *sfp = starfive_from_irq_data(d);
+ irq_hw_number_t gpio = irqd_to_hwirq(d);
+ void __iomem *ie = sfp->base + GPIOIE + 4 * (gpio / 32);
+ u32 mask = BIT(gpio % 32);
+ unsigned long flags;
+ u32 value;
+
+ raw_spin_lock_irqsave(&sfp->lock, flags);
+ value = readl_relaxed(ie) & ~mask;
+ writel_relaxed(value, ie);
+ raw_spin_unlock_irqrestore(&sfp->lock, flags);
+}
+
+static void starfive_irq_mask_ack(struct irq_data *d)
+{
+ struct starfive_pinctrl *sfp = starfive_from_irq_data(d);
+ irq_hw_number_t gpio = irqd_to_hwirq(d);
+ void __iomem *ie = sfp->base + GPIOIE + 4 * (gpio / 32);
+ void __iomem *ic = sfp->base + GPIOIC + 4 * (gpio / 32);
+ u32 mask = BIT(gpio % 32);
+ unsigned long flags;
+ u32 value;
+
+ raw_spin_lock_irqsave(&sfp->lock, flags);
+ value = readl_relaxed(ie) & ~mask;
+ writel_relaxed(value, ie);
+ writel_relaxed(mask, ic);
+ raw_spin_unlock_irqrestore(&sfp->lock, flags);
+}
+
+static void starfive_irq_unmask(struct irq_data *d)
+{
+ struct starfive_pinctrl *sfp = starfive_from_irq_data(d);
+ irq_hw_number_t gpio = irqd_to_hwirq(d);
+ void __iomem *ie = sfp->base + GPIOIE + 4 * (gpio / 32);
+ u32 mask = BIT(gpio % 32);
+ unsigned long flags;
+ u32 value;
+
+ raw_spin_lock_irqsave(&sfp->lock, flags);
+ value = readl_relaxed(ie) | mask;
+ writel_relaxed(value, ie);
+ raw_spin_unlock_irqrestore(&sfp->lock, flags);
+}
+
+static int starfive_irq_set_type(struct irq_data *d, unsigned int trigger)
+{
+ struct starfive_pinctrl *sfp = starfive_from_irq_data(d);
+ irq_hw_number_t gpio = irqd_to_hwirq(d);
+ void __iomem *base = sfp->base + 4 * (gpio / 32);
+ u32 mask = BIT(gpio % 32);
+ u32 irq_type, edge_both, polarity;
+ unsigned long flags;
+
+ switch (trigger) {
+ case IRQ_TYPE_EDGE_RISING:
+ irq_type = mask; /* 1: edge triggered */
+ edge_both = 0; /* 0: single edge */
+ polarity = mask; /* 1: rising edge */
+ break;
+ case IRQ_TYPE_EDGE_FALLING:
+ irq_type = mask; /* 1: edge triggered */
+ edge_both = 0; /* 0: single edge */
+ polarity = 0; /* 0: falling edge */
+ break;
+ case IRQ_TYPE_EDGE_BOTH:
+ irq_type = mask; /* 1: edge triggered */
+ edge_both = mask; /* 1: both edges */
+ polarity = 0; /* 0: ignored */
+ break;
+ case IRQ_TYPE_LEVEL_HIGH:
+ irq_type = 0; /* 0: level triggered */
+ edge_both = 0; /* 0: ignored */
+ polarity = mask; /* 1: high level */
+ break;
+ case IRQ_TYPE_LEVEL_LOW:
+ irq_type = 0; /* 0: level triggered */
+ edge_both = 0; /* 0: ignored */
+ polarity = 0; /* 0: low level */
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ if (trigger & IRQ_TYPE_EDGE_BOTH)
+ irq_set_handler_locked(d, handle_edge_irq);
+ else
+ irq_set_handler_locked(d, handle_level_irq);
+
+ raw_spin_lock_irqsave(&sfp->lock, flags);
+ irq_type |= readl_relaxed(base + GPIOIS) & ~mask;
+ writel_relaxed(irq_type, base + GPIOIS);
+ edge_both |= readl_relaxed(base + GPIOIBE) & ~mask;
+ writel_relaxed(edge_both, base + GPIOIBE);
+ polarity |= readl_relaxed(base + GPIOIEV) & ~mask;
+ writel_relaxed(polarity, base + GPIOIEV);
+ raw_spin_unlock_irqrestore(&sfp->lock, flags);
+ return 0;
+}
+
+static struct irq_chip starfive_irq_chip = {
+ .irq_ack = starfive_irq_ack,
+ .irq_mask = starfive_irq_mask,
+ .irq_mask_ack = starfive_irq_mask_ack,
+ .irq_unmask = starfive_irq_unmask,
+ .irq_set_type = starfive_irq_set_type,
+ .flags = IRQCHIP_SET_TYPE_MASKED,
+};
+
+static void starfive_gpio_irq_handler(struct irq_desc *desc)
+{
+ struct starfive_pinctrl *sfp = starfive_from_irq_desc(desc);
+ struct irq_chip *chip = irq_desc_get_chip(desc);
+ unsigned long mis;
+ unsigned int pin;
+
+ chained_irq_enter(chip, desc);
+
+ mis = readl_relaxed(sfp->base + GPIOMIS + 0);
+ for_each_set_bit(pin, &mis, 32)
+ generic_handle_domain_irq(sfp->gc.irq.domain, pin);
+
+ mis = readl_relaxed(sfp->base + GPIOMIS + 4);
+ for_each_set_bit(pin, &mis, 32)
+ generic_handle_domain_irq(sfp->gc.irq.domain, pin + 32);
+
+ chained_irq_exit(chip, desc);
+}
+
+static int starfive_gpio_init_hw(struct gpio_chip *gc)
+{
+ struct starfive_pinctrl *sfp = container_of(gc, struct starfive_pinctrl, gc);
+
+ /* mask all GPIO interrupts */
+ writel(0, sfp->base + GPIOIE + 0);
+ writel(0, sfp->base + GPIOIE + 4);
+ /* clear edge interrupt flags */
+ writel(~0U, sfp->base + GPIOIC + 0);
+ writel(~0U, sfp->base + GPIOIC + 4);
+ /* enable GPIO interrupts */
+ writel(1, sfp->base + GPIOEN);
+ return 0;
+}
+
+static void starfive_disable_clock(void *data)
+{
+ clk_disable_unprepare(data);
+}
+
+static int starfive_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct starfive_pinctrl *sfp;
+ struct reset_control *rst;
+ struct clk *clk;
+ u32 value;
+ int ret;
+
+ sfp = devm_kzalloc(dev, sizeof(*sfp), GFP_KERNEL);
+ if (!sfp)
+ return -ENOMEM;
+
+ sfp->base = devm_platform_ioremap_resource_byname(pdev, "gpio");
+ if (IS_ERR(sfp->base))
+ return PTR_ERR(sfp->base);
+
+ sfp->padctl = devm_platform_ioremap_resource_byname(pdev, "padctl");
+ if (IS_ERR(sfp->padctl))
+ return PTR_ERR(sfp->padctl);
+
+ clk = devm_clk_get(dev, NULL);
+ if (IS_ERR(clk))
+ return dev_err_probe(dev, PTR_ERR(clk), "could not get clock\n");
+
+ rst = devm_reset_control_get_exclusive(dev, NULL);
+ if (IS_ERR(rst))
+ return dev_err_probe(dev, PTR_ERR(rst), "could not get reset\n");
+
+ ret = clk_prepare_enable(clk);
+ if (ret)
+ return dev_err_probe(dev, ret, "could not enable clock\n");
+
+ ret = devm_add_action_or_reset(dev, starfive_disable_clock, clk);
+ if (ret)
+ return ret;
+
+ /*
+ * We don't want to assert reset and risk undoing pin muxing for the
+ * early boot serial console, but let's make sure the reset line is
+ * deasserted in case someone runs a really minimal bootloader.
+ */
+ ret = reset_control_deassert(rst);
+ if (ret)
+ return dev_err_probe(dev, ret, "could not deassert reset\n");
+
+ platform_set_drvdata(pdev, sfp);
+ sfp->gc.parent = dev;
+ raw_spin_lock_init(&sfp->lock);
+
+ ret = devm_pinctrl_register_and_init(dev, &starfive_desc, sfp, &sfp->pctl);
+ if (ret)
+ return dev_err_probe(dev, ret, "could not register pinctrl driver\n");
+
+ if (!of_property_read_u32(dev->of_node, "starfive,signal-group", &value)) {
+ if (value > 6)
+ return dev_err_probe(dev, -EINVAL, "invalid signal group %u\n", value);
+ writel(value, sfp->padctl + IO_PADSHARE_SEL);
+ }
+
+ value = readl(sfp->padctl + IO_PADSHARE_SEL);
+ switch (value) {
+ case 0:
+ sfp->gpios.pin_base = PAD_INVALID_GPIO;
+ goto out_pinctrl_enable;
+ case 1:
+ sfp->gpios.pin_base = PAD_GPIO(0);
+ break;
+ case 2:
+ sfp->gpios.pin_base = PAD_FUNC_SHARE(72);
+ break;
+ case 3:
+ sfp->gpios.pin_base = PAD_FUNC_SHARE(70);
+ break;
+ case 4: case 5: case 6:
+ sfp->gpios.pin_base = PAD_FUNC_SHARE(0);
+ break;
+ default:
+ return dev_err_probe(dev, -EINVAL, "invalid signal group %u\n", value);
+ }
+
+ sfp->gc.label = dev_name(dev);
+ sfp->gc.owner = THIS_MODULE;
+ sfp->gc.request = starfive_gpio_request;
+ sfp->gc.free = starfive_gpio_free;
+ sfp->gc.get_direction = starfive_gpio_get_direction;
+ sfp->gc.direction_input = starfive_gpio_direction_input;
+ sfp->gc.direction_output = starfive_gpio_direction_output;
+ sfp->gc.get = starfive_gpio_get;
+ sfp->gc.set = starfive_gpio_set;
+ sfp->gc.set_config = starfive_gpio_set_config;
+ sfp->gc.add_pin_ranges = starfive_gpio_add_pin_ranges;
+ sfp->gc.base = -1;
+ sfp->gc.ngpio = NR_GPIOS;
+
+ starfive_irq_chip.parent_device = dev;
+ starfive_irq_chip.name = sfp->gc.label;
+
+ sfp->gc.irq.chip = &starfive_irq_chip;
+ sfp->gc.irq.parent_handler = starfive_gpio_irq_handler;
+ sfp->gc.irq.num_parents = 1;
+ sfp->gc.irq.parents = devm_kcalloc(dev, sfp->gc.irq.num_parents,
+ sizeof(*sfp->gc.irq.parents), GFP_KERNEL);
+ if (!sfp->gc.irq.parents)
+ return -ENOMEM;
+ sfp->gc.irq.default_type = IRQ_TYPE_NONE;
+ sfp->gc.irq.handler = handle_bad_irq;
+ sfp->gc.irq.init_hw = starfive_gpio_init_hw;
+
+ ret = platform_get_irq(pdev, 0);
+ if (ret < 0)
+ return ret;
+ sfp->gc.irq.parents[0] = ret;
+
+ ret = devm_gpiochip_add_data(dev, &sfp->gc, sfp);
+ if (ret)
+ return dev_err_probe(dev, ret, "could not register gpiochip\n");
+
+out_pinctrl_enable:
+ return pinctrl_enable(sfp->pctl);
+}
+
+static const struct of_device_id starfive_of_match[] = {
+ { .compatible = "starfive,jh7100-pinctrl" },
+ { /* sentinel */ }
+};
+MODULE_DEVICE_TABLE(of, starfive_of_match);
+
+static struct platform_driver starfive_pinctrl_driver = {
+ .probe = starfive_probe,
+ .driver = {
+ .name = DRIVER_NAME,
+ .of_match_table = starfive_of_match,
+ },
+};
+module_platform_driver(starfive_pinctrl_driver);
+
+MODULE_DESCRIPTION("Pinctrl driver for StarFive SoCs");
+MODULE_AUTHOR("Emil Renner Berthing <kernel@esmil.dk>");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/platform/x86/Kconfig b/drivers/platform/x86/Kconfig
index 7400bc5..97e8762 100644
--- a/drivers/platform/x86/Kconfig
+++ b/drivers/platform/x86/Kconfig
@@ -517,7 +517,9 @@
depends on ACPI_VIDEO || ACPI_VIDEO = n
depends on BACKLIGHT_CLASS_DEVICE
depends on I2C
+ depends on DRM
select ACPI_PLATFORM_PROFILE
+ select DRM_PRIVACY_SCREEN
select HWMON
select NVRAM
select NEW_LEDS
diff --git a/drivers/platform/x86/intel/int3472/Makefile b/drivers/platform/x86/intel/int3472/Makefile
index 2362e04..cfec778 100644
--- a/drivers/platform/x86/intel/int3472/Makefile
+++ b/drivers/platform/x86/intel/int3472/Makefile
@@ -1,5 +1,4 @@
-obj-$(CONFIG_INTEL_SKL_INT3472) += intel_skl_int3472.o
-intel_skl_int3472-y := intel_skl_int3472_common.o \
- intel_skl_int3472_discrete.o \
- intel_skl_int3472_tps68470.o \
- intel_skl_int3472_clk_and_regulator.o
+obj-$(CONFIG_INTEL_SKL_INT3472) += intel_skl_int3472_discrete.o \
+ intel_skl_int3472_tps68470.o
+intel_skl_int3472_discrete-y := discrete.o clk_and_regulator.o common.o
+intel_skl_int3472_tps68470-y := tps68470.o tps68470_board_data.o common.o
diff --git a/drivers/platform/x86/intel/int3472/clk_and_regulator.c b/drivers/platform/x86/intel/int3472/clk_and_regulator.c
new file mode 100644
index 0000000..1cf9589
--- /dev/null
+++ b/drivers/platform/x86/intel/int3472/clk_and_regulator.c
@@ -0,0 +1,207 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Author: Dan Scally <djrscally@gmail.com> */
+
+#include <linux/acpi.h>
+#include <linux/clkdev.h>
+#include <linux/clk-provider.h>
+#include <linux/device.h>
+#include <linux/gpio/consumer.h>
+#include <linux/regulator/driver.h>
+#include <linux/slab.h>
+
+#include "common.h"
+
+/*
+ * The regulators have to have .ops to be valid, but the only ops we actually
+ * support are .enable and .disable which are handled via .ena_gpiod. Pass an
+ * empty struct to clear the check without lying about capabilities.
+ */
+static const struct regulator_ops int3472_gpio_regulator_ops;
+
+static int skl_int3472_clk_prepare(struct clk_hw *hw)
+{
+ struct int3472_gpio_clock *clk = to_int3472_clk(hw);
+
+ gpiod_set_value_cansleep(clk->ena_gpio, 1);
+ gpiod_set_value_cansleep(clk->led_gpio, 1);
+
+ return 0;
+}
+
+static void skl_int3472_clk_unprepare(struct clk_hw *hw)
+{
+ struct int3472_gpio_clock *clk = to_int3472_clk(hw);
+
+ gpiod_set_value_cansleep(clk->ena_gpio, 0);
+ gpiod_set_value_cansleep(clk->led_gpio, 0);
+}
+
+static int skl_int3472_clk_enable(struct clk_hw *hw)
+{
+ /*
+ * We're just turning a GPIO on to enable the clock, which operation
+ * has the potential to sleep. Given .enable() cannot sleep, but
+ * .prepare() can, we toggle the GPIO in .prepare() instead. Thus,
+ * nothing to do here.
+ */
+ return 0;
+}
+
+static void skl_int3472_clk_disable(struct clk_hw *hw)
+{
+ /* Likewise, nothing to do here... */
+}
+
+static unsigned int skl_int3472_get_clk_frequency(struct int3472_discrete_device *int3472)
+{
+ union acpi_object *obj;
+ unsigned int freq;
+
+ obj = skl_int3472_get_acpi_buffer(int3472->sensor, "SSDB");
+ if (IS_ERR(obj))
+ return 0; /* report rate as 0 on error */
+
+ if (obj->buffer.length < CIO2_SENSOR_SSDB_MCLKSPEED_OFFSET + sizeof(u32)) {
+ dev_err(int3472->dev, "The buffer is too small\n");
+ kfree(obj);
+ return 0;
+ }
+
+ freq = *(u32 *)(obj->buffer.pointer + CIO2_SENSOR_SSDB_MCLKSPEED_OFFSET);
+
+ kfree(obj);
+ return freq;
+}
+
+static unsigned long skl_int3472_clk_recalc_rate(struct clk_hw *hw,
+ unsigned long parent_rate)
+{
+ struct int3472_gpio_clock *clk = to_int3472_clk(hw);
+
+ return clk->frequency;
+}
+
+static const struct clk_ops skl_int3472_clock_ops = {
+ .prepare = skl_int3472_clk_prepare,
+ .unprepare = skl_int3472_clk_unprepare,
+ .enable = skl_int3472_clk_enable,
+ .disable = skl_int3472_clk_disable,
+ .recalc_rate = skl_int3472_clk_recalc_rate,
+};
+
+int skl_int3472_register_clock(struct int3472_discrete_device *int3472)
+{
+ struct clk_init_data init = {
+ .ops = &skl_int3472_clock_ops,
+ .flags = CLK_GET_RATE_NOCACHE,
+ };
+ int ret;
+
+ init.name = kasprintf(GFP_KERNEL, "%s-clk",
+ acpi_dev_name(int3472->adev));
+ if (!init.name)
+ return -ENOMEM;
+
+ int3472->clock.frequency = skl_int3472_get_clk_frequency(int3472);
+
+ int3472->clock.clk_hw.init = &init;
+ int3472->clock.clk = clk_register(&int3472->adev->dev,
+ &int3472->clock.clk_hw);
+ if (IS_ERR(int3472->clock.clk)) {
+ ret = PTR_ERR(int3472->clock.clk);
+ goto out_free_init_name;
+ }
+
+ int3472->clock.cl = clkdev_create(int3472->clock.clk, NULL,
+ int3472->sensor_name);
+ if (!int3472->clock.cl) {
+ ret = -ENOMEM;
+ goto err_unregister_clk;
+ }
+
+ kfree(init.name);
+ return 0;
+
+err_unregister_clk:
+ clk_unregister(int3472->clock.clk);
+out_free_init_name:
+ kfree(init.name);
+
+ return ret;
+}
+
+void skl_int3472_unregister_clock(struct int3472_discrete_device *int3472)
+{
+ clkdev_drop(int3472->clock.cl);
+ clk_unregister(int3472->clock.clk);
+}
+
+int skl_int3472_register_regulator(struct int3472_discrete_device *int3472,
+ struct acpi_resource_gpio *agpio)
+{
+ const struct int3472_sensor_config *sensor_config;
+ char *path = agpio->resource_source.string_ptr;
+ struct regulator_consumer_supply supply_map;
+ struct regulator_init_data init_data = { };
+ struct regulator_config cfg = { };
+ int ret;
+
+ sensor_config = int3472->sensor_config;
+ if (IS_ERR(sensor_config)) {
+ dev_err(int3472->dev, "No sensor module config\n");
+ return PTR_ERR(sensor_config);
+ }
+
+ if (!sensor_config->supply_map.supply) {
+ dev_err(int3472->dev, "No supply name defined\n");
+ return -ENODEV;
+ }
+
+ init_data.constraints.valid_ops_mask = REGULATOR_CHANGE_STATUS;
+ init_data.num_consumer_supplies = 1;
+ supply_map = sensor_config->supply_map;
+ supply_map.dev_name = int3472->sensor_name;
+ init_data.consumer_supplies = &supply_map;
+
+ snprintf(int3472->regulator.regulator_name,
+ sizeof(int3472->regulator.regulator_name), "%s-regulator",
+ acpi_dev_name(int3472->adev));
+ snprintf(int3472->regulator.supply_name,
+ GPIO_REGULATOR_SUPPLY_NAME_LENGTH, "supply-0");
+
+ int3472->regulator.rdesc = INT3472_REGULATOR(
+ int3472->regulator.regulator_name,
+ int3472->regulator.supply_name,
+ &int3472_gpio_regulator_ops);
+
+ int3472->regulator.gpio = acpi_get_and_request_gpiod(path, agpio->pin_table[0],
+ "int3472,regulator");
+ if (IS_ERR(int3472->regulator.gpio)) {
+ dev_err(int3472->dev, "Failed to get regulator GPIO line\n");
+ return PTR_ERR(int3472->regulator.gpio);
+ }
+
+ cfg.dev = &int3472->adev->dev;
+ cfg.init_data = &init_data;
+ cfg.ena_gpiod = int3472->regulator.gpio;
+
+ int3472->regulator.rdev = regulator_register(&int3472->regulator.rdesc,
+ &cfg);
+ if (IS_ERR(int3472->regulator.rdev)) {
+ ret = PTR_ERR(int3472->regulator.rdev);
+ goto err_free_gpio;
+ }
+
+ return 0;
+
+err_free_gpio:
+ gpiod_put(int3472->regulator.gpio);
+
+ return ret;
+}
+
+void skl_int3472_unregister_regulator(struct int3472_discrete_device *int3472)
+{
+ regulator_unregister(int3472->regulator.rdev);
+ gpiod_put(int3472->regulator.gpio);
+}
diff --git a/drivers/platform/x86/intel/int3472/common.c b/drivers/platform/x86/intel/int3472/common.c
new file mode 100644
index 0000000..77cf058
--- /dev/null
+++ b/drivers/platform/x86/intel/int3472/common.c
@@ -0,0 +1,82 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Author: Dan Scally <djrscally@gmail.com> */
+
+#include <linux/acpi.h>
+#include <linux/slab.h>
+
+#include "common.h"
+
+union acpi_object *skl_int3472_get_acpi_buffer(struct acpi_device *adev, char *id)
+{
+ struct acpi_buffer buffer = { ACPI_ALLOCATE_BUFFER, NULL };
+ acpi_handle handle = adev->handle;
+ union acpi_object *obj;
+ acpi_status status;
+
+ status = acpi_evaluate_object(handle, id, NULL, &buffer);
+ if (ACPI_FAILURE(status))
+ return ERR_PTR(-ENODEV);
+
+ obj = buffer.pointer;
+ if (!obj)
+ return ERR_PTR(-ENODEV);
+
+ if (obj->type != ACPI_TYPE_BUFFER) {
+ acpi_handle_err(handle, "%s object is not an ACPI buffer\n", id);
+ kfree(obj);
+ return ERR_PTR(-EINVAL);
+ }
+
+ return obj;
+}
+
+int skl_int3472_fill_cldb(struct acpi_device *adev, struct int3472_cldb *cldb)
+{
+ union acpi_object *obj;
+ int ret;
+
+ obj = skl_int3472_get_acpi_buffer(adev, "CLDB");
+ if (IS_ERR(obj))
+ return PTR_ERR(obj);
+
+ if (obj->buffer.length > sizeof(*cldb)) {
+ acpi_handle_err(adev->handle, "The CLDB buffer is too large\n");
+ ret = -EINVAL;
+ goto out_free_obj;
+ }
+
+ memcpy(cldb, obj->buffer.pointer, obj->buffer.length);
+ ret = 0;
+
+out_free_obj:
+ kfree(obj);
+ return ret;
+}
+
+/* sensor_adev_ret may be NULL, name_ret must not be NULL */
+int skl_int3472_get_sensor_adev_and_name(struct device *dev,
+ struct acpi_device **sensor_adev_ret,
+ const char **name_ret)
+{
+ struct acpi_device *adev = ACPI_COMPANION(dev);
+ struct acpi_device *sensor;
+ int ret = 0;
+
+ sensor = acpi_dev_get_first_consumer_dev(adev);
+ if (!sensor) {
+ dev_err(dev, "INT3472 seems to have no dependents.\n");
+ return -ENODEV;
+ }
+
+ *name_ret = devm_kasprintf(dev, GFP_KERNEL, I2C_DEV_NAME_FORMAT,
+ acpi_dev_name(sensor));
+ if (!*name_ret)
+ ret = -ENOMEM;
+
+ if (ret == 0 && sensor_adev_ret)
+ *sensor_adev_ret = sensor;
+ else
+ acpi_dev_put(sensor);
+
+ return ret;
+}
diff --git a/drivers/platform/x86/intel/int3472/common.h b/drivers/platform/x86/intel/int3472/common.h
new file mode 100644
index 0000000..53270d1
--- /dev/null
+++ b/drivers/platform/x86/intel/int3472/common.h
@@ -0,0 +1,122 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Author: Dan Scally <djrscally@gmail.com> */
+
+#ifndef _INTEL_SKL_INT3472_H
+#define _INTEL_SKL_INT3472_H
+
+#include <linux/clk-provider.h>
+#include <linux/gpio/machine.h>
+#include <linux/regulator/driver.h>
+#include <linux/regulator/machine.h>
+#include <linux/types.h>
+
+/* FIXME drop this once the I2C_DEV_NAME_FORMAT macro has been added to include/linux/i2c.h */
+#ifndef I2C_DEV_NAME_FORMAT
+#define I2C_DEV_NAME_FORMAT "i2c-%s"
+#endif
+
+/* PMIC GPIO Types */
+#define INT3472_GPIO_TYPE_RESET 0x00
+#define INT3472_GPIO_TYPE_POWERDOWN 0x01
+#define INT3472_GPIO_TYPE_POWER_ENABLE 0x0b
+#define INT3472_GPIO_TYPE_CLK_ENABLE 0x0c
+#define INT3472_GPIO_TYPE_PRIVACY_LED 0x0d
+
+#define INT3472_PDEV_MAX_NAME_LEN 23
+#define INT3472_MAX_SENSOR_GPIOS 3
+
+#define GPIO_REGULATOR_NAME_LENGTH 21
+#define GPIO_REGULATOR_SUPPLY_NAME_LENGTH 9
+
+#define CIO2_SENSOR_SSDB_MCLKSPEED_OFFSET 86
+
+#define INT3472_REGULATOR(_name, _supply, _ops) \
+ (const struct regulator_desc) { \
+ .name = _name, \
+ .supply_name = _supply, \
+ .type = REGULATOR_VOLTAGE, \
+ .ops = _ops, \
+ .owner = THIS_MODULE, \
+ }
+
+#define to_int3472_clk(hw) \
+ container_of(hw, struct int3472_gpio_clock, clk_hw)
+
+#define to_int3472_device(clk) \
+ container_of(clk, struct int3472_discrete_device, clock)
+
+struct acpi_device;
+struct i2c_client;
+struct platform_device;
+
+struct int3472_cldb {
+ u8 version;
+ /*
+ * control logic type
+ * 0: UNKNOWN
+ * 1: DISCRETE(CRD-D)
+ * 2: PMIC TPS68470
+ * 3: PMIC uP6641
+ */
+ u8 control_logic_type;
+ u8 control_logic_id;
+ u8 sensor_card_sku;
+ u8 reserved[28];
+};
+
+struct int3472_gpio_function_remap {
+ const char *documented;
+ const char *actual;
+};
+
+struct int3472_sensor_config {
+ const char *sensor_module_name;
+ struct regulator_consumer_supply supply_map;
+ const struct int3472_gpio_function_remap *function_maps;
+};
+
+struct int3472_discrete_device {
+ struct acpi_device *adev;
+ struct device *dev;
+ struct acpi_device *sensor;
+ const char *sensor_name;
+
+ const struct int3472_sensor_config *sensor_config;
+
+ struct int3472_gpio_regulator {
+ char regulator_name[GPIO_REGULATOR_NAME_LENGTH];
+ char supply_name[GPIO_REGULATOR_SUPPLY_NAME_LENGTH];
+ struct gpio_desc *gpio;
+ struct regulator_dev *rdev;
+ struct regulator_desc rdesc;
+ } regulator;
+
+ struct int3472_gpio_clock {
+ struct clk *clk;
+ struct clk_hw clk_hw;
+ struct clk_lookup *cl;
+ struct gpio_desc *ena_gpio;
+ struct gpio_desc *led_gpio;
+ u32 frequency;
+ } clock;
+
+ unsigned int ngpios; /* how many GPIOs have we seen */
+ unsigned int n_sensor_gpios; /* how many have we mapped to sensor */
+ struct gpiod_lookup_table gpios;
+};
+
+union acpi_object *skl_int3472_get_acpi_buffer(struct acpi_device *adev,
+ char *id);
+int skl_int3472_fill_cldb(struct acpi_device *adev, struct int3472_cldb *cldb);
+int skl_int3472_get_sensor_adev_and_name(struct device *dev,
+ struct acpi_device **sensor_adev_ret,
+ const char **name_ret);
+
+int skl_int3472_register_clock(struct int3472_discrete_device *int3472);
+void skl_int3472_unregister_clock(struct int3472_discrete_device *int3472);
+
+int skl_int3472_register_regulator(struct int3472_discrete_device *int3472,
+ struct acpi_resource_gpio *agpio);
+void skl_int3472_unregister_regulator(struct int3472_discrete_device *int3472);
+
+#endif
diff --git a/drivers/platform/x86/intel/int3472/discrete.c b/drivers/platform/x86/intel/int3472/discrete.c
new file mode 100644
index 0000000..5b514fa
--- /dev/null
+++ b/drivers/platform/x86/intel/int3472/discrete.c
@@ -0,0 +1,422 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Author: Dan Scally <djrscally@gmail.com> */
+
+#include <linux/acpi.h>
+#include <linux/clkdev.h>
+#include <linux/clk-provider.h>
+#include <linux/device.h>
+#include <linux/gpio/consumer.h>
+#include <linux/gpio/machine.h>
+#include <linux/i2c.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/overflow.h>
+#include <linux/platform_device.h>
+#include <linux/uuid.h>
+
+#include "common.h"
+
+/*
+ * 79234640-9e10-4fea-a5c1-b5aa8b19756f
+ * This _DSM GUID returns information about the GPIO lines mapped to a
+ * discrete INT3472 device. Function number 1 returns a count of the GPIO
+ * lines that are mapped. Subsequent functions return 32 bit ints encoding
+ * information about the GPIO line, including its purpose.
+ */
+static const guid_t int3472_gpio_guid =
+ GUID_INIT(0x79234640, 0x9e10, 0x4fea,
+ 0xa5, 0xc1, 0xb5, 0xaa, 0x8b, 0x19, 0x75, 0x6f);
+
+/*
+ * 822ace8f-2814-4174-a56b-5f029fe079ee
+ * This _DSM GUID returns a string from the sensor device, which acts as a
+ * module identifier.
+ */
+static const guid_t cio2_sensor_module_guid =
+ GUID_INIT(0x822ace8f, 0x2814, 0x4174,
+ 0xa5, 0x6b, 0x5f, 0x02, 0x9f, 0xe0, 0x79, 0xee);
+
+/*
+ * Here follows platform specific mapping information that we can pass to
+ * the functions mapping resources to the sensors. Where the sensors have
+ * a power enable pin defined in DSDT we need to provide a supply name so
+ * the sensor drivers can find the regulator. The device name will be derived
+ * from the sensor's ACPI device within the code. Optionally, we can provide a
+ * NULL terminated array of function name mappings to deal with any platform
+ * specific deviations from the documented behaviour of GPIOs.
+ *
+ * Map a GPIO function name to NULL to prevent the driver from mapping that
+ * GPIO at all.
+ */
+
+static const struct int3472_gpio_function_remap ov2680_gpio_function_remaps[] = {
+ { "reset", NULL },
+ { "powerdown", "reset" },
+ { }
+};
+
+static const struct int3472_sensor_config int3472_sensor_configs[] = {
+ /* Lenovo Miix 510-12ISK - OV2680, Front */
+ { "GNDF140809R", { 0 }, ov2680_gpio_function_remaps },
+ /* Lenovo Miix 510-12ISK - OV5648, Rear */
+ { "GEFF150023R", REGULATOR_SUPPLY("avdd", NULL), NULL },
+ /* Surface Go 1&2 - OV5693, Front */
+ { "YHCU", REGULATOR_SUPPLY("avdd", NULL), NULL },
+};
+
+static const struct int3472_sensor_config *
+skl_int3472_get_sensor_module_config(struct int3472_discrete_device *int3472)
+{
+ union acpi_object *obj;
+ unsigned int i;
+
+ obj = acpi_evaluate_dsm_typed(int3472->sensor->handle,
+ &cio2_sensor_module_guid, 0x00,
+ 0x01, NULL, ACPI_TYPE_STRING);
+
+ if (!obj) {
+ dev_err(int3472->dev,
+ "Failed to get sensor module string from _DSM\n");
+ return ERR_PTR(-ENODEV);
+ }
+
+ if (obj->string.type != ACPI_TYPE_STRING) {
+ dev_err(int3472->dev,
+ "Sensor _DSM returned a non-string value\n");
+
+ ACPI_FREE(obj);
+ return ERR_PTR(-EINVAL);
+ }
+
+ for (i = 0; i < ARRAY_SIZE(int3472_sensor_configs); i++) {
+ if (!strcmp(int3472_sensor_configs[i].sensor_module_name,
+ obj->string.pointer))
+ break;
+ }
+
+ ACPI_FREE(obj);
+
+ if (i >= ARRAY_SIZE(int3472_sensor_configs))
+ return ERR_PTR(-EINVAL);
+
+ return &int3472_sensor_configs[i];
+}
+
+static int skl_int3472_map_gpio_to_sensor(struct int3472_discrete_device *int3472,
+ struct acpi_resource_gpio *agpio,
+ const char *func, u32 polarity)
+{
+ const struct int3472_sensor_config *sensor_config;
+ char *path = agpio->resource_source.string_ptr;
+ struct gpiod_lookup *table_entry;
+ struct acpi_device *adev;
+ acpi_handle handle;
+ acpi_status status;
+ int ret;
+
+ if (int3472->n_sensor_gpios >= INT3472_MAX_SENSOR_GPIOS) {
+ dev_warn(int3472->dev, "Too many GPIOs mapped\n");
+ return -EINVAL;
+ }
+
+ sensor_config = int3472->sensor_config;
+ if (!IS_ERR(sensor_config) && sensor_config->function_maps) {
+ const struct int3472_gpio_function_remap *remap;
+
+ for (remap = sensor_config->function_maps; remap->documented; remap++) {
+ if (!strcmp(func, remap->documented)) {
+ func = remap->actual;
+ break;
+ }
+ }
+ }
+
+ /* Functions mapped to NULL should not be mapped to the sensor */
+ if (!func)
+ return 0;
+
+ status = acpi_get_handle(NULL, path, &handle);
+ if (ACPI_FAILURE(status))
+ return -EINVAL;
+
+ ret = acpi_bus_get_device(handle, &adev);
+ if (ret)
+ return -ENODEV;
+
+ table_entry = &int3472->gpios.table[int3472->n_sensor_gpios];
+ table_entry->key = acpi_dev_name(adev);
+ table_entry->chip_hwnum = agpio->pin_table[0];
+ table_entry->con_id = func;
+ table_entry->idx = 0;
+ table_entry->flags = polarity;
+
+ int3472->n_sensor_gpios++;
+
+ return 0;
+}
+
+static int skl_int3472_map_gpio_to_clk(struct int3472_discrete_device *int3472,
+ struct acpi_resource_gpio *agpio, u8 type)
+{
+ char *path = agpio->resource_source.string_ptr;
+ u16 pin = agpio->pin_table[0];
+ struct gpio_desc *gpio;
+
+ switch (type) {
+ case INT3472_GPIO_TYPE_CLK_ENABLE:
+ gpio = acpi_get_and_request_gpiod(path, pin, "int3472,clk-enable");
+ if (IS_ERR(gpio))
+ return (PTR_ERR(gpio));
+
+ int3472->clock.ena_gpio = gpio;
+ break;
+ case INT3472_GPIO_TYPE_PRIVACY_LED:
+ gpio = acpi_get_and_request_gpiod(path, pin, "int3472,privacy-led");
+ if (IS_ERR(gpio))
+ return (PTR_ERR(gpio));
+
+ int3472->clock.led_gpio = gpio;
+ break;
+ default:
+ dev_err(int3472->dev, "Invalid GPIO type 0x%02x for clock\n", type);
+ break;
+ }
+
+ return 0;
+}
+
+/**
+ * skl_int3472_handle_gpio_resources: Map PMIC resources to consuming sensor
+ * @ares: A pointer to a &struct acpi_resource
+ * @data: A pointer to a &struct int3472_discrete_device
+ *
+ * This function handles GPIO resources that are against an INT3472
+ * ACPI device, by checking the value of the corresponding _DSM entry.
+ * This will return a 32bit int, where the lowest byte represents the
+ * function of the GPIO pin:
+ *
+ * 0x00 Reset
+ * 0x01 Power down
+ * 0x0b Power enable
+ * 0x0c Clock enable
+ * 0x0d Privacy LED
+ *
+ * There are some known platform specific quirks where that does not quite
+ * hold up; for example where a pin with type 0x01 (Power down) is mapped to
+ * a sensor pin that performs a reset function or entries in _CRS and _DSM that
+ * do not actually correspond to a physical connection. These will be handled
+ * by the mapping sub-functions.
+ *
+ * GPIOs will either be mapped directly to the sensor device or else used
+ * to create clocks and regulators via the usual frameworks.
+ *
+ * Return:
+ * * 1 - To continue the loop
+ * * 0 - When all resources found are handled properly.
+ * * -EINVAL - If the resource is not a GPIO IO resource
+ * * -ENODEV - If the resource has no corresponding _DSM entry
+ * * -Other - Errors propagated from one of the sub-functions.
+ */
+static int skl_int3472_handle_gpio_resources(struct acpi_resource *ares,
+ void *data)
+{
+ struct int3472_discrete_device *int3472 = data;
+ struct acpi_resource_gpio *agpio;
+ union acpi_object *obj;
+ const char *err_msg;
+ int ret;
+ u8 type;
+
+ if (!acpi_gpio_get_io_resource(ares, &agpio))
+ return 1;
+
+ /*
+ * ngpios + 2 because the index of this _DSM function is 1-based and
+ * the first function is just a count.
+ */
+ obj = acpi_evaluate_dsm_typed(int3472->adev->handle,
+ &int3472_gpio_guid, 0x00,
+ int3472->ngpios + 2,
+ NULL, ACPI_TYPE_INTEGER);
+
+ if (!obj) {
+ dev_warn(int3472->dev, "No _DSM entry for GPIO pin %u\n",
+ agpio->pin_table[0]);
+ return 1;
+ }
+
+ type = obj->integer.value & 0xff;
+
+ switch (type) {
+ case INT3472_GPIO_TYPE_RESET:
+ ret = skl_int3472_map_gpio_to_sensor(int3472, agpio, "reset",
+ GPIO_ACTIVE_LOW);
+ if (ret)
+ err_msg = "Failed to map reset pin to sensor\n";
+
+ break;
+ case INT3472_GPIO_TYPE_POWERDOWN:
+ ret = skl_int3472_map_gpio_to_sensor(int3472, agpio, "powerdown",
+ GPIO_ACTIVE_LOW);
+ if (ret)
+ err_msg = "Failed to map powerdown pin to sensor\n";
+
+ break;
+ case INT3472_GPIO_TYPE_CLK_ENABLE:
+ case INT3472_GPIO_TYPE_PRIVACY_LED:
+ ret = skl_int3472_map_gpio_to_clk(int3472, agpio, type);
+ if (ret)
+ err_msg = "Failed to map GPIO to clock\n";
+
+ break;
+ case INT3472_GPIO_TYPE_POWER_ENABLE:
+ ret = skl_int3472_register_regulator(int3472, agpio);
+ if (ret)
+ err_msg = "Failed to map regulator to sensor\n";
+
+ break;
+ default:
+ dev_warn(int3472->dev,
+ "GPIO type 0x%02x unknown; the sensor may not work\n",
+ type);
+ ret = 1;
+ break;
+ }
+
+ int3472->ngpios++;
+ ACPI_FREE(obj);
+
+ if (ret < 0)
+ return dev_err_probe(int3472->dev, ret, err_msg);
+
+ return ret;
+}
+
+static int skl_int3472_parse_crs(struct int3472_discrete_device *int3472)
+{
+ LIST_HEAD(resource_list);
+ int ret;
+
+ /*
+ * No error check, because not having a sensor config is not necessarily
+ * a failure mode.
+ */
+ int3472->sensor_config = skl_int3472_get_sensor_module_config(int3472);
+
+ ret = acpi_dev_get_resources(int3472->adev, &resource_list,
+ skl_int3472_handle_gpio_resources,
+ int3472);
+ if (ret < 0)
+ return ret;
+
+ acpi_dev_free_resource_list(&resource_list);
+
+ /*
+ * If we find no clock enable GPIO pin then the privacy LED won't work.
+ * We've never seen that situation, but it's possible. Warn the user so
+ * it's clear what's happened.
+ */
+ if (int3472->clock.ena_gpio) {
+ ret = skl_int3472_register_clock(int3472);
+ if (ret)
+ return ret;
+ } else {
+ if (int3472->clock.led_gpio)
+ dev_warn(int3472->dev,
+ "No clk GPIO. The privacy LED won't work\n");
+ }
+
+ int3472->gpios.dev_id = int3472->sensor_name;
+ gpiod_add_lookup_table(&int3472->gpios);
+
+ return 0;
+}
+
+static int skl_int3472_discrete_remove(struct platform_device *pdev);
+
+static int skl_int3472_discrete_probe(struct platform_device *pdev)
+{
+ struct acpi_device *adev = ACPI_COMPANION(&pdev->dev);
+ struct int3472_discrete_device *int3472;
+ struct int3472_cldb cldb;
+ int ret;
+
+ ret = skl_int3472_fill_cldb(adev, &cldb);
+ if (ret) {
+ dev_err(&pdev->dev, "Couldn't fill CLDB structure\n");
+ return ret;
+ }
+
+ if (cldb.control_logic_type != 1) {
+ dev_err(&pdev->dev, "Unsupported control logic type %u\n",
+ cldb.control_logic_type);
+ return -EINVAL;
+ }
+
+ /* Max num GPIOs we've seen plus a terminator */
+ int3472 = devm_kzalloc(&pdev->dev, struct_size(int3472, gpios.table,
+ INT3472_MAX_SENSOR_GPIOS + 1), GFP_KERNEL);
+ if (!int3472)
+ return -ENOMEM;
+
+ int3472->adev = adev;
+ int3472->dev = &pdev->dev;
+ platform_set_drvdata(pdev, int3472);
+
+ ret = skl_int3472_get_sensor_adev_and_name(&pdev->dev, &int3472->sensor,
+ &int3472->sensor_name);
+ if (ret)
+ return ret;
+
+ /*
+ * Initialising this list means we can call gpiod_remove_lookup_table()
+ * in failure paths without issue.
+ */
+ INIT_LIST_HEAD(&int3472->gpios.list);
+
+ ret = skl_int3472_parse_crs(int3472);
+ if (ret) {
+ skl_int3472_discrete_remove(pdev);
+ return ret;
+ }
+
+ acpi_dev_clear_dependencies(adev);
+ return 0;
+}
+
+static int skl_int3472_discrete_remove(struct platform_device *pdev)
+{
+ struct int3472_discrete_device *int3472 = platform_get_drvdata(pdev);
+
+ gpiod_remove_lookup_table(&int3472->gpios);
+
+ if (int3472->clock.cl)
+ skl_int3472_unregister_clock(int3472);
+
+ gpiod_put(int3472->clock.ena_gpio);
+ gpiod_put(int3472->clock.led_gpio);
+
+ skl_int3472_unregister_regulator(int3472);
+
+ return 0;
+}
+
+static const struct acpi_device_id int3472_device_id[] = {
+ { "INT3472", 0 },
+ { }
+};
+MODULE_DEVICE_TABLE(acpi, int3472_device_id);
+
+static struct platform_driver int3472_discrete = {
+ .driver = {
+ .name = "int3472-discrete",
+ .acpi_match_table = int3472_device_id,
+ },
+ .probe = skl_int3472_discrete_probe,
+ .remove = skl_int3472_discrete_remove,
+};
+module_platform_driver(int3472_discrete);
+
+MODULE_DESCRIPTION("Intel SkyLake INT3472 ACPI Discrete Device Driver");
+MODULE_AUTHOR("Daniel Scally <djrscally@gmail.com>");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/platform/x86/intel/int3472/intel_skl_int3472_clk_and_regulator.c b/drivers/platform/x86/intel/int3472/intel_skl_int3472_clk_and_regulator.c
deleted file mode 100644
index 1700e75..0000000
--- a/drivers/platform/x86/intel/int3472/intel_skl_int3472_clk_and_regulator.c
+++ /dev/null
@@ -1,207 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/* Author: Dan Scally <djrscally@gmail.com> */
-
-#include <linux/acpi.h>
-#include <linux/clkdev.h>
-#include <linux/clk-provider.h>
-#include <linux/device.h>
-#include <linux/gpio/consumer.h>
-#include <linux/regulator/driver.h>
-#include <linux/slab.h>
-
-#include "intel_skl_int3472_common.h"
-
-/*
- * The regulators have to have .ops to be valid, but the only ops we actually
- * support are .enable and .disable which are handled via .ena_gpiod. Pass an
- * empty struct to clear the check without lying about capabilities.
- */
-static const struct regulator_ops int3472_gpio_regulator_ops;
-
-static int skl_int3472_clk_prepare(struct clk_hw *hw)
-{
- struct int3472_gpio_clock *clk = to_int3472_clk(hw);
-
- gpiod_set_value_cansleep(clk->ena_gpio, 1);
- gpiod_set_value_cansleep(clk->led_gpio, 1);
-
- return 0;
-}
-
-static void skl_int3472_clk_unprepare(struct clk_hw *hw)
-{
- struct int3472_gpio_clock *clk = to_int3472_clk(hw);
-
- gpiod_set_value_cansleep(clk->ena_gpio, 0);
- gpiod_set_value_cansleep(clk->led_gpio, 0);
-}
-
-static int skl_int3472_clk_enable(struct clk_hw *hw)
-{
- /*
- * We're just turning a GPIO on to enable the clock, which operation
- * has the potential to sleep. Given .enable() cannot sleep, but
- * .prepare() can, we toggle the GPIO in .prepare() instead. Thus,
- * nothing to do here.
- */
- return 0;
-}
-
-static void skl_int3472_clk_disable(struct clk_hw *hw)
-{
- /* Likewise, nothing to do here... */
-}
-
-static unsigned int skl_int3472_get_clk_frequency(struct int3472_discrete_device *int3472)
-{
- union acpi_object *obj;
- unsigned int freq;
-
- obj = skl_int3472_get_acpi_buffer(int3472->sensor, "SSDB");
- if (IS_ERR(obj))
- return 0; /* report rate as 0 on error */
-
- if (obj->buffer.length < CIO2_SENSOR_SSDB_MCLKSPEED_OFFSET + sizeof(u32)) {
- dev_err(int3472->dev, "The buffer is too small\n");
- kfree(obj);
- return 0;
- }
-
- freq = *(u32 *)(obj->buffer.pointer + CIO2_SENSOR_SSDB_MCLKSPEED_OFFSET);
-
- kfree(obj);
- return freq;
-}
-
-static unsigned long skl_int3472_clk_recalc_rate(struct clk_hw *hw,
- unsigned long parent_rate)
-{
- struct int3472_gpio_clock *clk = to_int3472_clk(hw);
-
- return clk->frequency;
-}
-
-static const struct clk_ops skl_int3472_clock_ops = {
- .prepare = skl_int3472_clk_prepare,
- .unprepare = skl_int3472_clk_unprepare,
- .enable = skl_int3472_clk_enable,
- .disable = skl_int3472_clk_disable,
- .recalc_rate = skl_int3472_clk_recalc_rate,
-};
-
-int skl_int3472_register_clock(struct int3472_discrete_device *int3472)
-{
- struct clk_init_data init = {
- .ops = &skl_int3472_clock_ops,
- .flags = CLK_GET_RATE_NOCACHE,
- };
- int ret;
-
- init.name = kasprintf(GFP_KERNEL, "%s-clk",
- acpi_dev_name(int3472->adev));
- if (!init.name)
- return -ENOMEM;
-
- int3472->clock.frequency = skl_int3472_get_clk_frequency(int3472);
-
- int3472->clock.clk_hw.init = &init;
- int3472->clock.clk = clk_register(&int3472->adev->dev,
- &int3472->clock.clk_hw);
- if (IS_ERR(int3472->clock.clk)) {
- ret = PTR_ERR(int3472->clock.clk);
- goto out_free_init_name;
- }
-
- int3472->clock.cl = clkdev_create(int3472->clock.clk, NULL,
- int3472->sensor_name);
- if (!int3472->clock.cl) {
- ret = -ENOMEM;
- goto err_unregister_clk;
- }
-
- kfree(init.name);
- return 0;
-
-err_unregister_clk:
- clk_unregister(int3472->clock.clk);
-out_free_init_name:
- kfree(init.name);
-
- return ret;
-}
-
-void skl_int3472_unregister_clock(struct int3472_discrete_device *int3472)
-{
- clkdev_drop(int3472->clock.cl);
- clk_unregister(int3472->clock.clk);
-}
-
-int skl_int3472_register_regulator(struct int3472_discrete_device *int3472,
- struct acpi_resource_gpio *agpio)
-{
- const struct int3472_sensor_config *sensor_config;
- char *path = agpio->resource_source.string_ptr;
- struct regulator_consumer_supply supply_map;
- struct regulator_init_data init_data = { };
- struct regulator_config cfg = { };
- int ret;
-
- sensor_config = int3472->sensor_config;
- if (IS_ERR(sensor_config)) {
- dev_err(int3472->dev, "No sensor module config\n");
- return PTR_ERR(sensor_config);
- }
-
- if (!sensor_config->supply_map.supply) {
- dev_err(int3472->dev, "No supply name defined\n");
- return -ENODEV;
- }
-
- init_data.constraints.valid_ops_mask = REGULATOR_CHANGE_STATUS;
- init_data.num_consumer_supplies = 1;
- supply_map = sensor_config->supply_map;
- supply_map.dev_name = int3472->sensor_name;
- init_data.consumer_supplies = &supply_map;
-
- snprintf(int3472->regulator.regulator_name,
- sizeof(int3472->regulator.regulator_name), "%s-regulator",
- acpi_dev_name(int3472->adev));
- snprintf(int3472->regulator.supply_name,
- GPIO_REGULATOR_SUPPLY_NAME_LENGTH, "supply-0");
-
- int3472->regulator.rdesc = INT3472_REGULATOR(
- int3472->regulator.regulator_name,
- int3472->regulator.supply_name,
- &int3472_gpio_regulator_ops);
-
- int3472->regulator.gpio = acpi_get_and_request_gpiod(path, agpio->pin_table[0],
- "int3472,regulator");
- if (IS_ERR(int3472->regulator.gpio)) {
- dev_err(int3472->dev, "Failed to get regulator GPIO line\n");
- return PTR_ERR(int3472->regulator.gpio);
- }
-
- cfg.dev = &int3472->adev->dev;
- cfg.init_data = &init_data;
- cfg.ena_gpiod = int3472->regulator.gpio;
-
- int3472->regulator.rdev = regulator_register(&int3472->regulator.rdesc,
- &cfg);
- if (IS_ERR(int3472->regulator.rdev)) {
- ret = PTR_ERR(int3472->regulator.rdev);
- goto err_free_gpio;
- }
-
- return 0;
-
-err_free_gpio:
- gpiod_put(int3472->regulator.gpio);
-
- return ret;
-}
-
-void skl_int3472_unregister_regulator(struct int3472_discrete_device *int3472)
-{
- regulator_unregister(int3472->regulator.rdev);
- gpiod_put(int3472->regulator.gpio);
-}
diff --git a/drivers/platform/x86/intel/int3472/intel_skl_int3472_common.c b/drivers/platform/x86/intel/int3472/intel_skl_int3472_common.c
deleted file mode 100644
index 497e74f..0000000
--- a/drivers/platform/x86/intel/int3472/intel_skl_int3472_common.c
+++ /dev/null
@@ -1,106 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/* Author: Dan Scally <djrscally@gmail.com> */
-
-#include <linux/acpi.h>
-#include <linux/i2c.h>
-#include <linux/platform_device.h>
-#include <linux/slab.h>
-
-#include "intel_skl_int3472_common.h"
-
-union acpi_object *skl_int3472_get_acpi_buffer(struct acpi_device *adev, char *id)
-{
- struct acpi_buffer buffer = { ACPI_ALLOCATE_BUFFER, NULL };
- acpi_handle handle = adev->handle;
- union acpi_object *obj;
- acpi_status status;
-
- status = acpi_evaluate_object(handle, id, NULL, &buffer);
- if (ACPI_FAILURE(status))
- return ERR_PTR(-ENODEV);
-
- obj = buffer.pointer;
- if (!obj)
- return ERR_PTR(-ENODEV);
-
- if (obj->type != ACPI_TYPE_BUFFER) {
- acpi_handle_err(handle, "%s object is not an ACPI buffer\n", id);
- kfree(obj);
- return ERR_PTR(-EINVAL);
- }
-
- return obj;
-}
-
-int skl_int3472_fill_cldb(struct acpi_device *adev, struct int3472_cldb *cldb)
-{
- union acpi_object *obj;
- int ret;
-
- obj = skl_int3472_get_acpi_buffer(adev, "CLDB");
- if (IS_ERR(obj))
- return PTR_ERR(obj);
-
- if (obj->buffer.length > sizeof(*cldb)) {
- acpi_handle_err(adev->handle, "The CLDB buffer is too large\n");
- ret = -EINVAL;
- goto out_free_obj;
- }
-
- memcpy(cldb, obj->buffer.pointer, obj->buffer.length);
- ret = 0;
-
-out_free_obj:
- kfree(obj);
- return ret;
-}
-
-static const struct acpi_device_id int3472_device_id[] = {
- { "INT3472", 0 },
- { }
-};
-MODULE_DEVICE_TABLE(acpi, int3472_device_id);
-
-static struct platform_driver int3472_discrete = {
- .driver = {
- .name = "int3472-discrete",
- .acpi_match_table = int3472_device_id,
- },
- .probe = skl_int3472_discrete_probe,
- .remove = skl_int3472_discrete_remove,
-};
-
-static struct i2c_driver int3472_tps68470 = {
- .driver = {
- .name = "int3472-tps68470",
- .acpi_match_table = int3472_device_id,
- },
- .probe_new = skl_int3472_tps68470_probe,
-};
-
-static int skl_int3472_init(void)
-{
- int ret;
-
- ret = platform_driver_register(&int3472_discrete);
- if (ret)
- return ret;
-
- ret = i2c_register_driver(THIS_MODULE, &int3472_tps68470);
- if (ret)
- platform_driver_unregister(&int3472_discrete);
-
- return ret;
-}
-module_init(skl_int3472_init);
-
-static void skl_int3472_exit(void)
-{
- platform_driver_unregister(&int3472_discrete);
- i2c_del_driver(&int3472_tps68470);
-}
-module_exit(skl_int3472_exit);
-
-MODULE_DESCRIPTION("Intel SkyLake INT3472 ACPI Device Driver");
-MODULE_AUTHOR("Daniel Scally <djrscally@gmail.com>");
-MODULE_LICENSE("GPL v2");
diff --git a/drivers/platform/x86/intel/int3472/intel_skl_int3472_common.h b/drivers/platform/x86/intel/int3472/intel_skl_int3472_common.h
deleted file mode 100644
index 714fde7..0000000
--- a/drivers/platform/x86/intel/int3472/intel_skl_int3472_common.h
+++ /dev/null
@@ -1,122 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-/* Author: Dan Scally <djrscally@gmail.com> */
-
-#ifndef _INTEL_SKL_INT3472_H
-#define _INTEL_SKL_INT3472_H
-
-#include <linux/clk-provider.h>
-#include <linux/gpio/machine.h>
-#include <linux/regulator/driver.h>
-#include <linux/regulator/machine.h>
-#include <linux/types.h>
-
-/* FIXME drop this once the I2C_DEV_NAME_FORMAT macro has been added to include/linux/i2c.h */
-#ifndef I2C_DEV_NAME_FORMAT
-#define I2C_DEV_NAME_FORMAT "i2c-%s"
-#endif
-
-/* PMIC GPIO Types */
-#define INT3472_GPIO_TYPE_RESET 0x00
-#define INT3472_GPIO_TYPE_POWERDOWN 0x01
-#define INT3472_GPIO_TYPE_POWER_ENABLE 0x0b
-#define INT3472_GPIO_TYPE_CLK_ENABLE 0x0c
-#define INT3472_GPIO_TYPE_PRIVACY_LED 0x0d
-
-#define INT3472_PDEV_MAX_NAME_LEN 23
-#define INT3472_MAX_SENSOR_GPIOS 3
-
-#define GPIO_REGULATOR_NAME_LENGTH 21
-#define GPIO_REGULATOR_SUPPLY_NAME_LENGTH 9
-
-#define CIO2_SENSOR_SSDB_MCLKSPEED_OFFSET 86
-
-#define INT3472_REGULATOR(_name, _supply, _ops) \
- (const struct regulator_desc) { \
- .name = _name, \
- .supply_name = _supply, \
- .type = REGULATOR_VOLTAGE, \
- .ops = _ops, \
- .owner = THIS_MODULE, \
- }
-
-#define to_int3472_clk(hw) \
- container_of(hw, struct int3472_gpio_clock, clk_hw)
-
-#define to_int3472_device(clk) \
- container_of(clk, struct int3472_discrete_device, clock)
-
-struct acpi_device;
-struct i2c_client;
-struct platform_device;
-
-struct int3472_cldb {
- u8 version;
- /*
- * control logic type
- * 0: UNKNOWN
- * 1: DISCRETE(CRD-D)
- * 2: PMIC TPS68470
- * 3: PMIC uP6641
- */
- u8 control_logic_type;
- u8 control_logic_id;
- u8 sensor_card_sku;
- u8 reserved[28];
-};
-
-struct int3472_gpio_function_remap {
- const char *documented;
- const char *actual;
-};
-
-struct int3472_sensor_config {
- const char *sensor_module_name;
- struct regulator_consumer_supply supply_map;
- const struct int3472_gpio_function_remap *function_maps;
-};
-
-struct int3472_discrete_device {
- struct acpi_device *adev;
- struct device *dev;
- struct acpi_device *sensor;
- const char *sensor_name;
-
- const struct int3472_sensor_config *sensor_config;
-
- struct int3472_gpio_regulator {
- char regulator_name[GPIO_REGULATOR_NAME_LENGTH];
- char supply_name[GPIO_REGULATOR_SUPPLY_NAME_LENGTH];
- struct gpio_desc *gpio;
- struct regulator_dev *rdev;
- struct regulator_desc rdesc;
- } regulator;
-
- struct int3472_gpio_clock {
- struct clk *clk;
- struct clk_hw clk_hw;
- struct clk_lookup *cl;
- struct gpio_desc *ena_gpio;
- struct gpio_desc *led_gpio;
- u32 frequency;
- } clock;
-
- unsigned int ngpios; /* how many GPIOs have we seen */
- unsigned int n_sensor_gpios; /* how many have we mapped to sensor */
- struct gpiod_lookup_table gpios;
-};
-
-int skl_int3472_discrete_probe(struct platform_device *pdev);
-int skl_int3472_discrete_remove(struct platform_device *pdev);
-int skl_int3472_tps68470_probe(struct i2c_client *client);
-union acpi_object *skl_int3472_get_acpi_buffer(struct acpi_device *adev,
- char *id);
-int skl_int3472_fill_cldb(struct acpi_device *adev, struct int3472_cldb *cldb);
-
-int skl_int3472_register_clock(struct int3472_discrete_device *int3472);
-void skl_int3472_unregister_clock(struct int3472_discrete_device *int3472);
-
-int skl_int3472_register_regulator(struct int3472_discrete_device *int3472,
- struct acpi_resource_gpio *agpio);
-void skl_int3472_unregister_regulator(struct int3472_discrete_device *int3472);
-
-#endif
diff --git a/drivers/platform/x86/intel/int3472/intel_skl_int3472_discrete.c b/drivers/platform/x86/intel/int3472/intel_skl_int3472_discrete.c
deleted file mode 100644
index e59d79c..0000000
--- a/drivers/platform/x86/intel/int3472/intel_skl_int3472_discrete.c
+++ /dev/null
@@ -1,413 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/* Author: Dan Scally <djrscally@gmail.com> */
-
-#include <linux/acpi.h>
-#include <linux/clkdev.h>
-#include <linux/clk-provider.h>
-#include <linux/device.h>
-#include <linux/gpio/consumer.h>
-#include <linux/gpio/machine.h>
-#include <linux/i2c.h>
-#include <linux/kernel.h>
-#include <linux/module.h>
-#include <linux/overflow.h>
-#include <linux/platform_device.h>
-#include <linux/uuid.h>
-
-#include "intel_skl_int3472_common.h"
-
-/*
- * 79234640-9e10-4fea-a5c1-b5aa8b19756f
- * This _DSM GUID returns information about the GPIO lines mapped to a
- * discrete INT3472 device. Function number 1 returns a count of the GPIO
- * lines that are mapped. Subsequent functions return 32 bit ints encoding
- * information about the GPIO line, including its purpose.
- */
-static const guid_t int3472_gpio_guid =
- GUID_INIT(0x79234640, 0x9e10, 0x4fea,
- 0xa5, 0xc1, 0xb5, 0xaa, 0x8b, 0x19, 0x75, 0x6f);
-
-/*
- * 822ace8f-2814-4174-a56b-5f029fe079ee
- * This _DSM GUID returns a string from the sensor device, which acts as a
- * module identifier.
- */
-static const guid_t cio2_sensor_module_guid =
- GUID_INIT(0x822ace8f, 0x2814, 0x4174,
- 0xa5, 0x6b, 0x5f, 0x02, 0x9f, 0xe0, 0x79, 0xee);
-
-/*
- * Here follows platform specific mapping information that we can pass to
- * the functions mapping resources to the sensors. Where the sensors have
- * a power enable pin defined in DSDT we need to provide a supply name so
- * the sensor drivers can find the regulator. The device name will be derived
- * from the sensor's ACPI device within the code. Optionally, we can provide a
- * NULL terminated array of function name mappings to deal with any platform
- * specific deviations from the documented behaviour of GPIOs.
- *
- * Map a GPIO function name to NULL to prevent the driver from mapping that
- * GPIO at all.
- */
-
-static const struct int3472_gpio_function_remap ov2680_gpio_function_remaps[] = {
- { "reset", NULL },
- { "powerdown", "reset" },
- { }
-};
-
-static const struct int3472_sensor_config int3472_sensor_configs[] = {
- /* Lenovo Miix 510-12ISK - OV2680, Front */
- { "GNDF140809R", { 0 }, ov2680_gpio_function_remaps },
- /* Lenovo Miix 510-12ISK - OV5648, Rear */
- { "GEFF150023R", REGULATOR_SUPPLY("avdd", NULL), NULL },
- /* Surface Go 1&2 - OV5693, Front */
- { "YHCU", REGULATOR_SUPPLY("avdd", NULL), NULL },
-};
-
-static const struct int3472_sensor_config *
-skl_int3472_get_sensor_module_config(struct int3472_discrete_device *int3472)
-{
- union acpi_object *obj;
- unsigned int i;
-
- obj = acpi_evaluate_dsm_typed(int3472->sensor->handle,
- &cio2_sensor_module_guid, 0x00,
- 0x01, NULL, ACPI_TYPE_STRING);
-
- if (!obj) {
- dev_err(int3472->dev,
- "Failed to get sensor module string from _DSM\n");
- return ERR_PTR(-ENODEV);
- }
-
- if (obj->string.type != ACPI_TYPE_STRING) {
- dev_err(int3472->dev,
- "Sensor _DSM returned a non-string value\n");
-
- ACPI_FREE(obj);
- return ERR_PTR(-EINVAL);
- }
-
- for (i = 0; i < ARRAY_SIZE(int3472_sensor_configs); i++) {
- if (!strcmp(int3472_sensor_configs[i].sensor_module_name,
- obj->string.pointer))
- break;
- }
-
- ACPI_FREE(obj);
-
- if (i >= ARRAY_SIZE(int3472_sensor_configs))
- return ERR_PTR(-EINVAL);
-
- return &int3472_sensor_configs[i];
-}
-
-static int skl_int3472_map_gpio_to_sensor(struct int3472_discrete_device *int3472,
- struct acpi_resource_gpio *agpio,
- const char *func, u32 polarity)
-{
- const struct int3472_sensor_config *sensor_config;
- char *path = agpio->resource_source.string_ptr;
- struct gpiod_lookup *table_entry;
- struct acpi_device *adev;
- acpi_handle handle;
- acpi_status status;
- int ret;
-
- if (int3472->n_sensor_gpios >= INT3472_MAX_SENSOR_GPIOS) {
- dev_warn(int3472->dev, "Too many GPIOs mapped\n");
- return -EINVAL;
- }
-
- sensor_config = int3472->sensor_config;
- if (!IS_ERR(sensor_config) && sensor_config->function_maps) {
- const struct int3472_gpio_function_remap *remap;
-
- for (remap = sensor_config->function_maps; remap->documented; remap++) {
- if (!strcmp(func, remap->documented)) {
- func = remap->actual;
- break;
- }
- }
- }
-
- /* Functions mapped to NULL should not be mapped to the sensor */
- if (!func)
- return 0;
-
- status = acpi_get_handle(NULL, path, &handle);
- if (ACPI_FAILURE(status))
- return -EINVAL;
-
- ret = acpi_bus_get_device(handle, &adev);
- if (ret)
- return -ENODEV;
-
- table_entry = &int3472->gpios.table[int3472->n_sensor_gpios];
- table_entry->key = acpi_dev_name(adev);
- table_entry->chip_hwnum = agpio->pin_table[0];
- table_entry->con_id = func;
- table_entry->idx = 0;
- table_entry->flags = polarity;
-
- int3472->n_sensor_gpios++;
-
- return 0;
-}
-
-static int skl_int3472_map_gpio_to_clk(struct int3472_discrete_device *int3472,
- struct acpi_resource_gpio *agpio, u8 type)
-{
- char *path = agpio->resource_source.string_ptr;
- u16 pin = agpio->pin_table[0];
- struct gpio_desc *gpio;
-
- switch (type) {
- case INT3472_GPIO_TYPE_CLK_ENABLE:
- gpio = acpi_get_and_request_gpiod(path, pin, "int3472,clk-enable");
- if (IS_ERR(gpio))
- return (PTR_ERR(gpio));
-
- int3472->clock.ena_gpio = gpio;
- break;
- case INT3472_GPIO_TYPE_PRIVACY_LED:
- gpio = acpi_get_and_request_gpiod(path, pin, "int3472,privacy-led");
- if (IS_ERR(gpio))
- return (PTR_ERR(gpio));
-
- int3472->clock.led_gpio = gpio;
- break;
- default:
- dev_err(int3472->dev, "Invalid GPIO type 0x%02x for clock\n", type);
- break;
- }
-
- return 0;
-}
-
-/**
- * skl_int3472_handle_gpio_resources: Map PMIC resources to consuming sensor
- * @ares: A pointer to a &struct acpi_resource
- * @data: A pointer to a &struct int3472_discrete_device
- *
- * This function handles GPIO resources that are against an INT3472
- * ACPI device, by checking the value of the corresponding _DSM entry.
- * This will return a 32bit int, where the lowest byte represents the
- * function of the GPIO pin:
- *
- * 0x00 Reset
- * 0x01 Power down
- * 0x0b Power enable
- * 0x0c Clock enable
- * 0x0d Privacy LED
- *
- * There are some known platform specific quirks where that does not quite
- * hold up; for example where a pin with type 0x01 (Power down) is mapped to
- * a sensor pin that performs a reset function or entries in _CRS and _DSM that
- * do not actually correspond to a physical connection. These will be handled
- * by the mapping sub-functions.
- *
- * GPIOs will either be mapped directly to the sensor device or else used
- * to create clocks and regulators via the usual frameworks.
- *
- * Return:
- * * 1 - To continue the loop
- * * 0 - When all resources found are handled properly.
- * * -EINVAL - If the resource is not a GPIO IO resource
- * * -ENODEV - If the resource has no corresponding _DSM entry
- * * -Other - Errors propagated from one of the sub-functions.
- */
-static int skl_int3472_handle_gpio_resources(struct acpi_resource *ares,
- void *data)
-{
- struct int3472_discrete_device *int3472 = data;
- struct acpi_resource_gpio *agpio;
- union acpi_object *obj;
- const char *err_msg;
- int ret;
- u8 type;
-
- if (!acpi_gpio_get_io_resource(ares, &agpio))
- return 1;
-
- /*
- * ngpios + 2 because the index of this _DSM function is 1-based and
- * the first function is just a count.
- */
- obj = acpi_evaluate_dsm_typed(int3472->adev->handle,
- &int3472_gpio_guid, 0x00,
- int3472->ngpios + 2,
- NULL, ACPI_TYPE_INTEGER);
-
- if (!obj) {
- dev_warn(int3472->dev, "No _DSM entry for GPIO pin %u\n",
- agpio->pin_table[0]);
- return 1;
- }
-
- type = obj->integer.value & 0xff;
-
- switch (type) {
- case INT3472_GPIO_TYPE_RESET:
- ret = skl_int3472_map_gpio_to_sensor(int3472, agpio, "reset",
- GPIO_ACTIVE_LOW);
- if (ret)
- err_msg = "Failed to map reset pin to sensor\n";
-
- break;
- case INT3472_GPIO_TYPE_POWERDOWN:
- ret = skl_int3472_map_gpio_to_sensor(int3472, agpio, "powerdown",
- GPIO_ACTIVE_LOW);
- if (ret)
- err_msg = "Failed to map powerdown pin to sensor\n";
-
- break;
- case INT3472_GPIO_TYPE_CLK_ENABLE:
- case INT3472_GPIO_TYPE_PRIVACY_LED:
- ret = skl_int3472_map_gpio_to_clk(int3472, agpio, type);
- if (ret)
- err_msg = "Failed to map GPIO to clock\n";
-
- break;
- case INT3472_GPIO_TYPE_POWER_ENABLE:
- ret = skl_int3472_register_regulator(int3472, agpio);
- if (ret)
- err_msg = "Failed to map regulator to sensor\n";
-
- break;
- default:
- dev_warn(int3472->dev,
- "GPIO type 0x%02x unknown; the sensor may not work\n",
- type);
- ret = 1;
- break;
- }
-
- int3472->ngpios++;
- ACPI_FREE(obj);
-
- if (ret < 0)
- return dev_err_probe(int3472->dev, ret, err_msg);
-
- return ret;
-}
-
-static int skl_int3472_parse_crs(struct int3472_discrete_device *int3472)
-{
- LIST_HEAD(resource_list);
- int ret;
-
- /*
- * No error check, because not having a sensor config is not necessarily
- * a failure mode.
- */
- int3472->sensor_config = skl_int3472_get_sensor_module_config(int3472);
-
- ret = acpi_dev_get_resources(int3472->adev, &resource_list,
- skl_int3472_handle_gpio_resources,
- int3472);
- if (ret < 0)
- return ret;
-
- acpi_dev_free_resource_list(&resource_list);
-
- /*
- * If we find no clock enable GPIO pin then the privacy LED won't work.
- * We've never seen that situation, but it's possible. Warn the user so
- * it's clear what's happened.
- */
- if (int3472->clock.ena_gpio) {
- ret = skl_int3472_register_clock(int3472);
- if (ret)
- return ret;
- } else {
- if (int3472->clock.led_gpio)
- dev_warn(int3472->dev,
- "No clk GPIO. The privacy LED won't work\n");
- }
-
- int3472->gpios.dev_id = int3472->sensor_name;
- gpiod_add_lookup_table(&int3472->gpios);
-
- return 0;
-}
-
-int skl_int3472_discrete_probe(struct platform_device *pdev)
-{
- struct acpi_device *adev = ACPI_COMPANION(&pdev->dev);
- struct int3472_discrete_device *int3472;
- struct int3472_cldb cldb;
- int ret;
-
- ret = skl_int3472_fill_cldb(adev, &cldb);
- if (ret) {
- dev_err(&pdev->dev, "Couldn't fill CLDB structure\n");
- return ret;
- }
-
- if (cldb.control_logic_type != 1) {
- dev_err(&pdev->dev, "Unsupported control logic type %u\n",
- cldb.control_logic_type);
- return -EINVAL;
- }
-
- /* Max num GPIOs we've seen plus a terminator */
- int3472 = devm_kzalloc(&pdev->dev, struct_size(int3472, gpios.table,
- INT3472_MAX_SENSOR_GPIOS + 1), GFP_KERNEL);
- if (!int3472)
- return -ENOMEM;
-
- int3472->adev = adev;
- int3472->dev = &pdev->dev;
- platform_set_drvdata(pdev, int3472);
-
- int3472->sensor = acpi_dev_get_first_consumer_dev(adev);
- if (!int3472->sensor) {
- dev_err(&pdev->dev, "INT3472 seems to have no dependents.\n");
- return -ENODEV;
- }
-
- int3472->sensor_name = devm_kasprintf(int3472->dev, GFP_KERNEL,
- I2C_DEV_NAME_FORMAT,
- acpi_dev_name(int3472->sensor));
- if (!int3472->sensor_name) {
- ret = -ENOMEM;
- goto err_put_sensor;
- }
-
- /*
- * Initialising this list means we can call gpiod_remove_lookup_table()
- * in failure paths without issue.
- */
- INIT_LIST_HEAD(&int3472->gpios.list);
-
- ret = skl_int3472_parse_crs(int3472);
- if (ret) {
- skl_int3472_discrete_remove(pdev);
- return ret;
- }
-
- return 0;
-
-err_put_sensor:
- acpi_dev_put(int3472->sensor);
-
- return ret;
-}
-
-int skl_int3472_discrete_remove(struct platform_device *pdev)
-{
- struct int3472_discrete_device *int3472 = platform_get_drvdata(pdev);
-
- gpiod_remove_lookup_table(&int3472->gpios);
-
- if (int3472->clock.cl)
- skl_int3472_unregister_clock(int3472);
-
- gpiod_put(int3472->clock.ena_gpio);
- gpiod_put(int3472->clock.led_gpio);
-
- skl_int3472_unregister_regulator(int3472);
-
- return 0;
-}
diff --git a/drivers/platform/x86/intel/int3472/intel_skl_int3472_tps68470.c b/drivers/platform/x86/intel/int3472/intel_skl_int3472_tps68470.c
deleted file mode 100644
index c05b4cf..0000000
--- a/drivers/platform/x86/intel/int3472/intel_skl_int3472_tps68470.c
+++ /dev/null
@@ -1,137 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/* Author: Dan Scally <djrscally@gmail.com> */
-
-#include <linux/i2c.h>
-#include <linux/mfd/core.h>
-#include <linux/mfd/tps68470.h>
-#include <linux/platform_device.h>
-#include <linux/regmap.h>
-
-#include "intel_skl_int3472_common.h"
-
-#define DESIGNED_FOR_CHROMEOS 1
-#define DESIGNED_FOR_WINDOWS 2
-
-static const struct mfd_cell tps68470_cros[] = {
- { .name = "tps68470-gpio" },
- { .name = "tps68470_pmic_opregion" },
-};
-
-static const struct mfd_cell tps68470_win[] = {
- { .name = "tps68470-gpio" },
- { .name = "tps68470-clk" },
- { .name = "tps68470-regulator" },
-};
-
-static const struct regmap_config tps68470_regmap_config = {
- .reg_bits = 8,
- .val_bits = 8,
- .max_register = TPS68470_REG_MAX,
-};
-
-static int tps68470_chip_init(struct device *dev, struct regmap *regmap)
-{
- unsigned int version;
- int ret;
-
- /* Force software reset */
- ret = regmap_write(regmap, TPS68470_REG_RESET, TPS68470_REG_RESET_MASK);
- if (ret)
- return ret;
-
- ret = regmap_read(regmap, TPS68470_REG_REVID, &version);
- if (ret) {
- dev_err(dev, "Failed to read revision register: %d\n", ret);
- return ret;
- }
-
- dev_info(dev, "TPS68470 REVID: 0x%02x\n", version);
-
- return 0;
-}
-
-/** skl_int3472_tps68470_calc_type: Check what platform a device is designed for
- * @adev: A pointer to a &struct acpi_device
- *
- * Check CLDB buffer against the PMIC's adev. If present, then we check
- * the value of control_logic_type field and follow one of the
- * following scenarios:
- *
- * 1. No CLDB - likely ACPI tables designed for ChromeOS. We
- * create platform devices for the GPIOs and OpRegion drivers.
- *
- * 2. CLDB, with control_logic_type = 2 - probably ACPI tables
- * made for Windows 2-in-1 platforms. Register pdevs for GPIO,
- * Clock and Regulator drivers to bind to.
- *
- * 3. Any other value in control_logic_type, we should never have
- * gotten to this point; fail probe and return.
- *
- * Return:
- * * 1 Device intended for ChromeOS
- * * 2 Device intended for Windows
- * * -EINVAL Where @adev has an object named CLDB but it does not conform to
- * our expectations
- */
-static int skl_int3472_tps68470_calc_type(struct acpi_device *adev)
-{
- struct int3472_cldb cldb = { 0 };
- int ret;
-
- /*
- * A CLDB buffer that exists, but which does not match our expectations
- * should trigger an error so we don't blindly continue.
- */
- ret = skl_int3472_fill_cldb(adev, &cldb);
- if (ret && ret != -ENODEV)
- return ret;
-
- if (ret)
- return DESIGNED_FOR_CHROMEOS;
-
- if (cldb.control_logic_type != 2)
- return -EINVAL;
-
- return DESIGNED_FOR_WINDOWS;
-}
-
-int skl_int3472_tps68470_probe(struct i2c_client *client)
-{
- struct acpi_device *adev = ACPI_COMPANION(&client->dev);
- struct regmap *regmap;
- int device_type;
- int ret;
-
- regmap = devm_regmap_init_i2c(client, &tps68470_regmap_config);
- if (IS_ERR(regmap)) {
- dev_err(&client->dev, "Failed to create regmap: %ld\n", PTR_ERR(regmap));
- return PTR_ERR(regmap);
- }
-
- i2c_set_clientdata(client, regmap);
-
- ret = tps68470_chip_init(&client->dev, regmap);
- if (ret < 0) {
- dev_err(&client->dev, "TPS68470 init error %d\n", ret);
- return ret;
- }
-
- device_type = skl_int3472_tps68470_calc_type(adev);
- switch (device_type) {
- case DESIGNED_FOR_WINDOWS:
- ret = devm_mfd_add_devices(&client->dev, PLATFORM_DEVID_NONE,
- tps68470_win, ARRAY_SIZE(tps68470_win),
- NULL, 0, NULL);
- break;
- case DESIGNED_FOR_CHROMEOS:
- ret = devm_mfd_add_devices(&client->dev, PLATFORM_DEVID_NONE,
- tps68470_cros, ARRAY_SIZE(tps68470_cros),
- NULL, 0, NULL);
- break;
- default:
- dev_err(&client->dev, "Failed to add MFD devices\n");
- return device_type;
- }
-
- return ret;
-}
diff --git a/drivers/platform/x86/intel/int3472/tps68470.c b/drivers/platform/x86/intel/int3472/tps68470.c
new file mode 100644
index 0000000..22f61b4
--- /dev/null
+++ b/drivers/platform/x86/intel/int3472/tps68470.c
@@ -0,0 +1,211 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Author: Dan Scally <djrscally@gmail.com> */
+
+#include <linux/i2c.h>
+#include <linux/kernel.h>
+#include <linux/mfd/core.h>
+#include <linux/mfd/tps68470.h>
+#include <linux/platform_device.h>
+#include <linux/platform_data/tps68470.h>
+#include <linux/regmap.h>
+#include <linux/string.h>
+
+#include "common.h"
+#include "tps68470.h"
+
+#define DESIGNED_FOR_CHROMEOS 1
+#define DESIGNED_FOR_WINDOWS 2
+
+#define TPS68470_WIN_MFD_CELL_COUNT 3
+
+static const struct mfd_cell tps68470_cros[] = {
+ { .name = "tps68470-gpio" },
+ { .name = "tps68470_pmic_opregion" },
+};
+
+static const struct regmap_config tps68470_regmap_config = {
+ .reg_bits = 8,
+ .val_bits = 8,
+ .max_register = TPS68470_REG_MAX,
+};
+
+static int tps68470_chip_init(struct device *dev, struct regmap *regmap)
+{
+ unsigned int version;
+ int ret;
+
+ /* Force software reset */
+ ret = regmap_write(regmap, TPS68470_REG_RESET, TPS68470_REG_RESET_MASK);
+ if (ret)
+ return ret;
+
+ ret = regmap_read(regmap, TPS68470_REG_REVID, &version);
+ if (ret) {
+ dev_err(dev, "Failed to read revision register: %d\n", ret);
+ return ret;
+ }
+
+ dev_info(dev, "TPS68470 REVID: 0x%02x\n", version);
+
+ return 0;
+}
+
+/** skl_int3472_tps68470_calc_type: Check what platform a device is designed for
+ * @adev: A pointer to a &struct acpi_device
+ *
+ * Check CLDB buffer against the PMIC's adev. If present, then we check
+ * the value of control_logic_type field and follow one of the
+ * following scenarios:
+ *
+ * 1. No CLDB - likely ACPI tables designed for ChromeOS. We
+ * create platform devices for the GPIOs and OpRegion drivers.
+ *
+ * 2. CLDB, with control_logic_type = 2 - probably ACPI tables
+ * made for Windows 2-in-1 platforms. Register pdevs for GPIO,
+ * Clock and Regulator drivers to bind to.
+ *
+ * 3. Any other value in control_logic_type, we should never have
+ * gotten to this point; fail probe and return.
+ *
+ * Return:
+ * * 1 Device intended for ChromeOS
+ * * 2 Device intended for Windows
+ * * -EINVAL Where @adev has an object named CLDB but it does not conform to
+ * our expectations
+ */
+static int skl_int3472_tps68470_calc_type(struct acpi_device *adev)
+{
+ struct int3472_cldb cldb = { 0 };
+ int ret;
+
+ /*
+ * A CLDB buffer that exists, but which does not match our expectations
+ * should trigger an error so we don't blindly continue.
+ */
+ ret = skl_int3472_fill_cldb(adev, &cldb);
+ if (ret && ret != -ENODEV)
+ return ret;
+
+ if (ret)
+ return DESIGNED_FOR_CHROMEOS;
+
+ if (cldb.control_logic_type != 2)
+ return -EINVAL;
+
+ return DESIGNED_FOR_WINDOWS;
+}
+
+static int skl_int3472_tps68470_probe(struct i2c_client *client)
+{
+ struct acpi_device *adev = ACPI_COMPANION(&client->dev);
+ const struct int3472_tps68470_board_data *board_data;
+ struct tps68470_clk_platform_data clk_pdata = {};
+ struct mfd_cell *cells;
+ struct regmap *regmap;
+ int device_type;
+ int ret;
+
+ ret = skl_int3472_get_sensor_adev_and_name(&client->dev, NULL,
+ &clk_pdata.consumer_dev_name);
+ if (ret)
+ return ret;
+
+ regmap = devm_regmap_init_i2c(client, &tps68470_regmap_config);
+ if (IS_ERR(regmap)) {
+ dev_err(&client->dev, "Failed to create regmap: %ld\n", PTR_ERR(regmap));
+ return PTR_ERR(regmap);
+ }
+
+ i2c_set_clientdata(client, regmap);
+
+ ret = tps68470_chip_init(&client->dev, regmap);
+ if (ret < 0) {
+ dev_err(&client->dev, "TPS68470 init error %d\n", ret);
+ return ret;
+ }
+
+ device_type = skl_int3472_tps68470_calc_type(adev);
+ switch (device_type) {
+ case DESIGNED_FOR_WINDOWS:
+ board_data = int3472_tps68470_get_board_data(dev_name(&client->dev));
+ if (!board_data)
+ return dev_err_probe(&client->dev, -ENODEV, "No board-data found for this model\n");
+
+ cells = kcalloc(TPS68470_WIN_MFD_CELL_COUNT, sizeof(*cells), GFP_KERNEL);
+ if (!cells)
+ return -ENOMEM;
+
+ /*
+ * The order of the cells matters here! The clk must be first
+ * because the regulator depends on it. The gpios must be last,
+ * acpi_gpiochip_add() calls acpi_dev_clear_dependencies() and
+ * the clk + regulators must be ready when this happens.
+ */
+ cells[0].name = "tps68470-clk";
+ cells[0].platform_data = &clk_pdata;
+ cells[0].pdata_size = sizeof(clk_pdata);
+ cells[1].name = "tps68470-regulator";
+ cells[1].platform_data = (void *)board_data->tps68470_regulator_pdata;
+ cells[1].pdata_size = sizeof(struct tps68470_regulator_platform_data);
+ cells[2].name = "tps68470-gpio";
+
+ gpiod_add_lookup_table(board_data->tps68470_gpio_lookup_table);
+
+ ret = devm_mfd_add_devices(&client->dev, PLATFORM_DEVID_NONE,
+ cells, TPS68470_WIN_MFD_CELL_COUNT,
+ NULL, 0, NULL);
+ kfree(cells);
+
+ if (ret)
+ gpiod_remove_lookup_table(board_data->tps68470_gpio_lookup_table);
+
+ break;
+ case DESIGNED_FOR_CHROMEOS:
+ ret = devm_mfd_add_devices(&client->dev, PLATFORM_DEVID_NONE,
+ tps68470_cros, ARRAY_SIZE(tps68470_cros),
+ NULL, 0, NULL);
+ break;
+ default:
+ dev_err(&client->dev, "Failed to add MFD devices\n");
+ return device_type;
+ }
+
+ /*
+ * No acpi_dev_clear_dependencies() here, since the acpi_gpiochip_add()
+ * for the GPIO cell already does this.
+ */
+
+ return ret;
+}
+
+static int skl_int3472_tps68470_remove(struct i2c_client *client)
+{
+ const struct int3472_tps68470_board_data *board_data;
+
+ board_data = int3472_tps68470_get_board_data(dev_name(&client->dev));
+ if (board_data)
+ gpiod_remove_lookup_table(board_data->tps68470_gpio_lookup_table);
+
+ return 0;
+}
+
+static const struct acpi_device_id int3472_device_id[] = {
+ { "INT3472", 0 },
+ { }
+};
+MODULE_DEVICE_TABLE(acpi, int3472_device_id);
+
+static struct i2c_driver int3472_tps68470 = {
+ .driver = {
+ .name = "int3472-tps68470",
+ .acpi_match_table = int3472_device_id,
+ },
+ .probe_new = skl_int3472_tps68470_probe,
+ .remove = skl_int3472_tps68470_remove,
+};
+module_i2c_driver(int3472_tps68470);
+
+MODULE_DESCRIPTION("Intel SkyLake INT3472 ACPI TPS68470 Device Driver");
+MODULE_AUTHOR("Daniel Scally <djrscally@gmail.com>");
+MODULE_LICENSE("GPL v2");
+MODULE_SOFTDEP("pre: clk-tps68470 tps68470-regulator");
diff --git a/drivers/platform/x86/intel/int3472/tps68470.h b/drivers/platform/x86/intel/int3472/tps68470.h
new file mode 100644
index 0000000..cfd33eb
--- /dev/null
+++ b/drivers/platform/x86/intel/int3472/tps68470.h
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * TI TPS68470 PMIC platform data definition.
+ *
+ * Copyright (c) 2021 Red Hat Inc.
+ *
+ * Red Hat authors:
+ * Hans de Goede <hdegoede@redhat.com>
+ */
+
+#ifndef _INTEL_SKL_INT3472_TPS68470_H
+#define _INTEL_SKL_INT3472_TPS68470_H
+
+struct gpiod_lookup_table;
+struct tps68470_regulator_platform_data;
+
+struct int3472_tps68470_board_data {
+ const char *dev_name;
+ struct gpiod_lookup_table *tps68470_gpio_lookup_table;
+ const struct tps68470_regulator_platform_data *tps68470_regulator_pdata;
+};
+
+const struct int3472_tps68470_board_data *int3472_tps68470_get_board_data(const char *dev_name);
+
+#endif
diff --git a/drivers/platform/x86/intel/int3472/tps68470_board_data.c b/drivers/platform/x86/intel/int3472/tps68470_board_data.c
new file mode 100644
index 0000000..faa5570
--- /dev/null
+++ b/drivers/platform/x86/intel/int3472/tps68470_board_data.c
@@ -0,0 +1,145 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * TI TPS68470 PMIC platform data definition.
+ *
+ * Copyright (c) 2021 Dan Scally <djrscally@gmail.com>
+ * Copyright (c) 2021 Red Hat Inc.
+ *
+ * Red Hat authors:
+ * Hans de Goede <hdegoede@redhat.com>
+ */
+
+#include <linux/dmi.h>
+#include <linux/gpio/machine.h>
+#include <linux/platform_data/tps68470.h>
+#include <linux/regulator/machine.h>
+#include "tps68470.h"
+
+static struct regulator_consumer_supply int347a_core_consumer_supplies[] = {
+ REGULATOR_SUPPLY("dvdd", "i2c-INT347A:00"),
+};
+
+static struct regulator_consumer_supply int347a_ana_consumer_supplies[] = {
+ REGULATOR_SUPPLY("avdd", "i2c-INT347A:00"),
+};
+
+static struct regulator_consumer_supply int347a_vcm_consumer_supplies[] = {
+ REGULATOR_SUPPLY("vdd", "i2c-INT347A:00-VCM"),
+};
+
+static struct regulator_consumer_supply int347a_vsio_consumer_supplies[] = {
+ REGULATOR_SUPPLY("dovdd", "i2c-INT347A:00"),
+ REGULATOR_SUPPLY("vsio", "i2c-INT347A:00-VCM"),
+};
+
+static const struct regulator_init_data surface_go_tps68470_core_reg_init_data = {
+ .constraints = {
+ .min_uV = 1200000,
+ .max_uV = 1200000,
+ .apply_uV = true,
+ .valid_ops_mask = REGULATOR_CHANGE_STATUS,
+ },
+ .num_consumer_supplies = ARRAY_SIZE(int347a_core_consumer_supplies),
+ .consumer_supplies = int347a_core_consumer_supplies,
+};
+
+static const struct regulator_init_data surface_go_tps68470_ana_reg_init_data = {
+ .constraints = {
+ .min_uV = 2815200,
+ .max_uV = 2815200,
+ .apply_uV = true,
+ .valid_ops_mask = REGULATOR_CHANGE_STATUS,
+ },
+ .num_consumer_supplies = ARRAY_SIZE(int347a_ana_consumer_supplies),
+ .consumer_supplies = int347a_ana_consumer_supplies,
+};
+
+static const struct regulator_init_data surface_go_tps68470_vcm_reg_init_data = {
+ .constraints = {
+ .min_uV = 2815200,
+ .max_uV = 2815200,
+ .apply_uV = true,
+ .valid_ops_mask = REGULATOR_CHANGE_STATUS,
+ },
+ .num_consumer_supplies = ARRAY_SIZE(int347a_vcm_consumer_supplies),
+ .consumer_supplies = int347a_vcm_consumer_supplies,
+};
+
+/* Ensure the always-on VIO regulator has the same voltage as VSIO */
+static const struct regulator_init_data surface_go_tps68470_vio_reg_init_data = {
+ .constraints = {
+ .min_uV = 1800600,
+ .max_uV = 1800600,
+ .apply_uV = true,
+ .always_on = true,
+ },
+};
+
+static const struct regulator_init_data surface_go_tps68470_vsio_reg_init_data = {
+ .constraints = {
+ .min_uV = 1800600,
+ .max_uV = 1800600,
+ .apply_uV = true,
+ .valid_ops_mask = REGULATOR_CHANGE_STATUS,
+ },
+ .num_consumer_supplies = ARRAY_SIZE(int347a_vsio_consumer_supplies),
+ .consumer_supplies = int347a_vsio_consumer_supplies,
+};
+
+static const struct tps68470_regulator_platform_data surface_go_tps68470_pdata = {
+ .reg_init_data = {
+ [TPS68470_CORE] = &surface_go_tps68470_core_reg_init_data,
+ [TPS68470_ANA] = &surface_go_tps68470_ana_reg_init_data,
+ [TPS68470_VCM] = &surface_go_tps68470_vcm_reg_init_data,
+ [TPS68470_VIO] = &surface_go_tps68470_vio_reg_init_data,
+ [TPS68470_VSIO] = &surface_go_tps68470_vsio_reg_init_data,
+ },
+};
+
+static struct gpiod_lookup_table surface_go_tps68470_gpios = {
+ .dev_id = "i2c-INT347A:00",
+ .table = {
+ GPIO_LOOKUP("tps68470-gpio", 9, "reset", GPIO_ACTIVE_LOW),
+ GPIO_LOOKUP("tps68470-gpio", 7, "powerdown", GPIO_ACTIVE_LOW)
+ }
+};
+
+static const struct int3472_tps68470_board_data surface_go_tps68470_board_data = {
+ .dev_name = "i2c-INT3472:05",
+ .tps68470_gpio_lookup_table = &surface_go_tps68470_gpios,
+ .tps68470_regulator_pdata = &surface_go_tps68470_pdata,
+};
+
+static const struct dmi_system_id int3472_tps68470_board_data_table[] = {
+ {
+ .matches = {
+ DMI_EXACT_MATCH(DMI_SYS_VENDOR, "Microsoft Corporation"),
+ DMI_EXACT_MATCH(DMI_PRODUCT_NAME, "Surface Go"),
+ },
+ .driver_data = (void *)&surface_go_tps68470_board_data,
+ },
+ {
+ .matches = {
+ DMI_EXACT_MATCH(DMI_SYS_VENDOR, "Microsoft Corporation"),
+ DMI_EXACT_MATCH(DMI_PRODUCT_NAME, "Surface Go 2"),
+ },
+ .driver_data = (void *)&surface_go_tps68470_board_data,
+ },
+ { }
+};
+
+const struct int3472_tps68470_board_data *int3472_tps68470_get_board_data(const char *dev_name)
+{
+ const struct int3472_tps68470_board_data *board_data;
+ const struct dmi_system_id *match;
+
+ for (match = dmi_first_match(int3472_tps68470_board_data_table);
+ match;
+ match = dmi_first_match(match + 1)) {
+ board_data = match->driver_data;
+ if (strcmp(board_data->dev_name, dev_name) == 0)
+ return board_data;
+ }
+
+ return NULL;
+}
diff --git a/drivers/platform/x86/thinkpad_acpi.c b/drivers/platform/x86/thinkpad_acpi.c
index bb1abb9..341655d 100644
--- a/drivers/platform/x86/thinkpad_acpi.c
+++ b/drivers/platform/x86/thinkpad_acpi.c
@@ -73,6 +73,7 @@
#include <linux/uaccess.h>
#include <acpi/battery.h>
#include <acpi/video.h>
+#include <drm/drm_privacy_screen_driver.h>
#include "dual_accel_detect.h"
/* ThinkPad CMOS commands */
@@ -157,6 +158,7 @@ enum tpacpi_hkey_event_t {
TP_HKEY_EV_VOL_UP = 0x1015, /* Volume up or unmute */
TP_HKEY_EV_VOL_DOWN = 0x1016, /* Volume down or unmute */
TP_HKEY_EV_VOL_MUTE = 0x1017, /* Mixer output mute */
+ TP_HKEY_EV_PRIVACYGUARD_TOGGLE = 0x130f, /* Toggle priv.guard on/off */
/* Reasons for waking up from S3/S4 */
TP_HKEY_EV_WKUP_S3_UNDOCK = 0x2304, /* undock requested, S3 */
@@ -3788,6 +3790,30 @@ static bool adaptive_keyboard_hotkey_notify_hotkey(unsigned int scancode)
}
}
+static bool hotkey_notify_extended_hotkey(const u32 hkey)
+{
+ unsigned int scancode;
+
+ switch (hkey) {
+ case TP_HKEY_EV_PRIVACYGUARD_TOGGLE:
+ tpacpi_driver_event(hkey);
+ return true;
+ }
+
+ /* Extended keycodes start at 0x300 and our offset into the map
+ * TP_ACPI_HOTKEYSCAN_EXTENDED_START. The calculated scancode
+ * will be positive, but might not be in the correct range.
+ */
+ scancode = (hkey & 0xfff) - (0x300 - TP_ACPI_HOTKEYSCAN_EXTENDED_START);
+ if (scancode >= TP_ACPI_HOTKEYSCAN_EXTENDED_START &&
+ scancode < TPACPI_HOTKEY_MAP_LEN) {
+ tpacpi_input_send_key(scancode);
+ return true;
+ }
+
+ return false;
+}
+
static bool hotkey_notify_hotkey(const u32 hkey,
bool *send_acpi_ev,
bool *ignore_acpi_ev)
@@ -3822,17 +3848,7 @@ static bool hotkey_notify_hotkey(const u32 hkey,
return adaptive_keyboard_hotkey_notify_hotkey(scancode);
case 3:
- /* Extended keycodes start at 0x300 and our offset into the map
- * TP_ACPI_HOTKEYSCAN_EXTENDED_START. The calculated scancode
- * will be positive, but might not be in the correct range.
- */
- scancode -= (0x300 - TP_ACPI_HOTKEYSCAN_EXTENDED_START);
- if (scancode >= TP_ACPI_HOTKEYSCAN_EXTENDED_START &&
- scancode < TPACPI_HOTKEY_MAP_LEN) {
- tpacpi_input_send_key(scancode);
- return true;
- }
- break;
+ return hotkey_notify_extended_hotkey(hkey);
}
return false;
@@ -9715,69 +9731,85 @@ static struct ibm_struct battery_driver_data = {
* LCD Shadow subdriver, for the Lenovo PrivacyGuard feature
*/
-static int lcdshadow_state;
+static struct drm_privacy_screen *lcdshadow_dev;
+static acpi_handle lcdshadow_get_handle;
+static acpi_handle lcdshadow_set_handle;
-static int lcdshadow_on_off(bool state)
+static int lcdshadow_set_sw_state(struct drm_privacy_screen *priv,
+ enum drm_privacy_screen_status state)
{
- acpi_handle set_shadow_handle;
int output;
- if (ACPI_FAILURE(acpi_get_handle(hkey_handle, "SSSS", &set_shadow_handle))) {
- pr_warn("Thinkpad ACPI has no %s interface.\n", "SSSS");
- return -EIO;
- }
-
- if (!acpi_evalf(set_shadow_handle, &output, NULL, "dd", (int)state))
+ if (WARN_ON(!mutex_is_locked(&priv->lock)))
return -EIO;
- lcdshadow_state = state;
+ if (!acpi_evalf(lcdshadow_set_handle, &output, NULL, "dd", (int)state))
+ return -EIO;
+
+ priv->hw_state = priv->sw_state = state;
return 0;
}
-static int lcdshadow_set(bool on)
+static void lcdshadow_get_hw_state(struct drm_privacy_screen *priv)
{
- if (lcdshadow_state < 0)
- return lcdshadow_state;
- if (lcdshadow_state == on)
- return 0;
- return lcdshadow_on_off(on);
+ int output;
+
+ if (!acpi_evalf(lcdshadow_get_handle, &output, NULL, "dd", 0))
+ return;
+
+ priv->hw_state = priv->sw_state = output & 0x1;
}
+static const struct drm_privacy_screen_ops lcdshadow_ops = {
+ .set_sw_state = lcdshadow_set_sw_state,
+ .get_hw_state = lcdshadow_get_hw_state,
+};
+
static int tpacpi_lcdshadow_init(struct ibm_init_struct *iibm)
{
- acpi_handle get_shadow_handle;
+ acpi_status status1, status2;
int output;
- if (ACPI_FAILURE(acpi_get_handle(hkey_handle, "GSSS", &get_shadow_handle))) {
- lcdshadow_state = -ENODEV;
+ status1 = acpi_get_handle(hkey_handle, "GSSS", &lcdshadow_get_handle);
+ status2 = acpi_get_handle(hkey_handle, "SSSS", &lcdshadow_set_handle);
+ if (ACPI_FAILURE(status1) || ACPI_FAILURE(status2))
return 0;
- }
- if (!acpi_evalf(get_shadow_handle, &output, NULL, "dd", 0)) {
- lcdshadow_state = -EIO;
+ if (!acpi_evalf(lcdshadow_get_handle, &output, NULL, "dd", 0))
return -EIO;
- }
- if (!(output & 0x10000)) {
- lcdshadow_state = -ENODEV;
+
+ if (!(output & 0x10000))
return 0;
- }
- lcdshadow_state = output & 0x1;
+
+ lcdshadow_dev = drm_privacy_screen_register(&tpacpi_pdev->dev,
+ &lcdshadow_ops);
+ if (IS_ERR(lcdshadow_dev))
+ return PTR_ERR(lcdshadow_dev);
return 0;
}
+static void lcdshadow_exit(void)
+{
+ drm_privacy_screen_unregister(lcdshadow_dev);
+}
+
static void lcdshadow_resume(void)
{
- if (lcdshadow_state >= 0)
- lcdshadow_on_off(lcdshadow_state);
+ if (!lcdshadow_dev)
+ return;
+
+ mutex_lock(&lcdshadow_dev->lock);
+ lcdshadow_set_sw_state(lcdshadow_dev, lcdshadow_dev->sw_state);
+ mutex_unlock(&lcdshadow_dev->lock);
}
static int lcdshadow_read(struct seq_file *m)
{
- if (lcdshadow_state < 0) {
+ if (!lcdshadow_dev) {
seq_puts(m, "status:\t\tnot supported\n");
} else {
- seq_printf(m, "status:\t\t%d\n", lcdshadow_state);
+ seq_printf(m, "status:\t\t%d\n", lcdshadow_dev->hw_state);
seq_puts(m, "commands:\t0, 1\n");
}
@@ -9789,7 +9821,7 @@ static int lcdshadow_write(char *buf)
char *cmd;
int res, state = -EINVAL;
- if (lcdshadow_state < 0)
+ if (!lcdshadow_dev)
return -ENODEV;
while ((cmd = strsep(&buf, ","))) {
@@ -9801,11 +9833,18 @@ static int lcdshadow_write(char *buf)
if (state >= 2 || state < 0)
return -EINVAL;
- return lcdshadow_set(state);
+ mutex_lock(&lcdshadow_dev->lock);
+ res = lcdshadow_set_sw_state(lcdshadow_dev, state);
+ mutex_unlock(&lcdshadow_dev->lock);
+
+ drm_privacy_screen_call_notifier_chain(lcdshadow_dev);
+
+ return res;
}
static struct ibm_struct lcdshadow_driver_data = {
.name = "lcdshadow",
+ .exit = lcdshadow_exit,
.resume = lcdshadow_resume,
.read = lcdshadow_read,
.write = lcdshadow_write,
@@ -10615,6 +10654,20 @@ static void tpacpi_driver_event(const unsigned int hkey_event)
if (!atomic_add_unless(&dytc_ignore_event, -1, 0))
dytc_profile_refresh();
}
+
+ if (lcdshadow_dev && hkey_event == TP_HKEY_EV_PRIVACYGUARD_TOGGLE) {
+ enum drm_privacy_screen_status old_hw_state;
+ bool changed;
+
+ mutex_lock(&lcdshadow_dev->lock);
+ old_hw_state = lcdshadow_dev->hw_state;
+ lcdshadow_get_hw_state(lcdshadow_dev);
+ changed = lcdshadow_dev->hw_state != old_hw_state;
+ mutex_unlock(&lcdshadow_dev->lock);
+
+ if (changed)
+ drm_privacy_screen_call_notifier_chain(lcdshadow_dev);
+ }
}
static void hotkey_driver_event(const unsigned int scancode)
diff --git a/drivers/power/reset/ltc2952-poweroff.c b/drivers/power/reset/ltc2952-poweroff.c
index fbb3443..65d9528 100644
--- a/drivers/power/reset/ltc2952-poweroff.c
+++ b/drivers/power/reset/ltc2952-poweroff.c
@@ -159,8 +159,8 @@ static void ltc2952_poweroff_kill(void)
static void ltc2952_poweroff_default(struct ltc2952_poweroff *data)
{
- data->wde_interval = 300L * 1E6L;
- data->trigger_delay = ktime_set(2, 500L*1E6L);
+ data->wde_interval = 300L * NSEC_PER_MSEC;
+ data->trigger_delay = ktime_set(2, 500L * NSEC_PER_MSEC);
hrtimer_init(&data->timer_trigger, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
data->timer_trigger.function = ltc2952_poweroff_timer_trigger;
diff --git a/drivers/power/supply/bq25890_charger.c b/drivers/power/supply/bq25890_charger.c
index 34ec186..b7eac54 100644
--- a/drivers/power/supply/bq25890_charger.c
+++ b/drivers/power/supply/bq25890_charger.c
@@ -581,12 +581,12 @@ static irqreturn_t __bq25890_handle_irq(struct bq25890_device *bq)
if (!new_state.online && bq->state.online) { /* power removed */
/* disable ADC */
- ret = bq25890_field_write(bq, F_CONV_START, 0);
+ ret = bq25890_field_write(bq, F_CONV_RATE, 0);
if (ret < 0)
goto error;
} else if (new_state.online && !bq->state.online) { /* power inserted */
/* enable ADC, to have control of charge current/voltage */
- ret = bq25890_field_write(bq, F_CONV_START, 1);
+ ret = bq25890_field_write(bq, F_CONV_RATE, 1);
if (ret < 0)
goto error;
}
diff --git a/drivers/power/supply/power_supply_core.c b/drivers/power/supply/power_supply_core.c
index fc12a4f..6093754 100644
--- a/drivers/power/supply/power_supply_core.c
+++ b/drivers/power/supply/power_supply_core.c
@@ -853,6 +853,10 @@ power_supply_find_ocv2cap_table(struct power_supply_battery_info *info,
return NULL;
for (i = 0; i < POWER_SUPPLY_OCV_TEMP_MAX; i++) {
+ /* Out of capacity tables */
+ if (!info->ocv_table[i])
+ break;
+
temp_diff = abs(info->ocv_temp[i] - temp);
if (temp_diff < best_temp_diff) {
diff --git a/drivers/ptp/ptp_ines.c b/drivers/ptp/ptp_ines.c
index 6c7c284..61f47fb9 100644
--- a/drivers/ptp/ptp_ines.c
+++ b/drivers/ptp/ptp_ines.c
@@ -338,10 +338,6 @@ static int ines_hwtstamp(struct mii_timestamper *mii_ts, struct ifreq *ifr)
if (copy_from_user(&cfg, ifr->ifr_data, sizeof(cfg)))
return -EFAULT;
- /* reserved for future extensions */
- if (cfg.flags)
- return -EINVAL;
-
switch (cfg.tx_type) {
case HWTSTAMP_TX_OFF:
ts_stat_tx = 0;
diff --git a/drivers/ptp/ptp_vclock.c b/drivers/ptp/ptp_vclock.c
index baee037..ab1d233 100644
--- a/drivers/ptp/ptp_vclock.c
+++ b/drivers/ptp/ptp_vclock.c
@@ -185,8 +185,8 @@ int ptp_get_vclocks_index(int pclock_index, int **vclock_index)
}
EXPORT_SYMBOL(ptp_get_vclocks_index);
-void ptp_convert_timestamp(struct skb_shared_hwtstamps *hwtstamps,
- int vclock_index)
+ktime_t ptp_convert_timestamp(const struct skb_shared_hwtstamps *hwtstamps,
+ int vclock_index)
{
char name[PTP_CLOCK_NAME_LEN] = "";
struct ptp_vclock *vclock;
@@ -198,12 +198,12 @@ void ptp_convert_timestamp(struct skb_shared_hwtstamps *hwtstamps,
snprintf(name, PTP_CLOCK_NAME_LEN, "ptp%d", vclock_index);
dev = class_find_device_by_name(ptp_class, name);
if (!dev)
- return;
+ return 0;
ptp = dev_get_drvdata(dev);
if (!ptp->is_virtual_clock) {
put_device(dev);
- return;
+ return 0;
}
vclock = info_to_vclock(ptp->info);
@@ -215,7 +215,7 @@ void ptp_convert_timestamp(struct skb_shared_hwtstamps *hwtstamps,
spin_unlock_irqrestore(&vclock->lock, flags);
put_device(dev);
- hwtstamps->hwtstamp = ns_to_ktime(ns);
+ return ns_to_ktime(ns);
}
EXPORT_SYMBOL(ptp_convert_timestamp);
#endif
diff --git a/drivers/pwm/core.c b/drivers/pwm/core.c
index fb04a43..93772ab 100644
--- a/drivers/pwm/core.c
+++ b/drivers/pwm/core.c
@@ -152,6 +152,32 @@ of_pwm_xlate_with_flags(struct pwm_chip *pc, const struct of_phandle_args *args)
}
EXPORT_SYMBOL_GPL(of_pwm_xlate_with_flags);
+struct pwm_device *
+of_pwm_single_xlate(struct pwm_chip *pc, const struct of_phandle_args *args)
+{
+ struct pwm_device *pwm;
+
+ if (pc->of_pwm_n_cells < 1)
+ return ERR_PTR(-EINVAL);
+
+ /* validate that one cell is specified, optionally with flags */
+ if (args->args_count != 1 && args->args_count != 2)
+ return ERR_PTR(-EINVAL);
+
+ pwm = pwm_request_from_chip(pc, 0, NULL);
+ if (IS_ERR(pwm))
+ return pwm;
+
+ pwm->args.period = args->args[0];
+ pwm->args.polarity = PWM_POLARITY_NORMAL;
+
+ if (args->args_count == 2 && args->args[2] & PWM_POLARITY_INVERTED)
+ pwm->args.polarity = PWM_POLARITY_INVERSED;
+
+ return pwm;
+}
+EXPORT_SYMBOL_GPL(of_pwm_single_xlate);
+
static void of_pwmchip_add(struct pwm_chip *chip)
{
if (!chip->dev || !chip->dev->of_node)
diff --git a/drivers/pwm/pwm-pxa.c b/drivers/pwm/pwm-pxa.c
index a9efdcf8..238ec88 100644
--- a/drivers/pwm/pwm-pxa.c
+++ b/drivers/pwm/pwm-pxa.c
@@ -148,20 +148,6 @@ static const struct platform_device_id *pxa_pwm_get_id_dt(struct device *dev)
return id ? id->data : NULL;
}
-static struct pwm_device *
-pxa_pwm_of_xlate(struct pwm_chip *pc, const struct of_phandle_args *args)
-{
- struct pwm_device *pwm;
-
- pwm = pwm_request_from_chip(pc, 0, NULL);
- if (IS_ERR(pwm))
- return pwm;
-
- pwm->args.period = args->args[0];
-
- return pwm;
-}
-
static int pwm_probe(struct platform_device *pdev)
{
const struct platform_device_id *id = platform_get_device_id(pdev);
@@ -187,7 +173,7 @@ static int pwm_probe(struct platform_device *pdev)
pc->chip.npwm = (id->driver_data & HAS_SECONDARY_PWM) ? 2 : 1;
if (IS_ENABLED(CONFIG_OF)) {
- pc->chip.of_xlate = pxa_pwm_of_xlate;
+ pc->chip.of_xlate = of_pwm_single_xlate;
pc->chip.of_pwm_n_cells = 1;
}
diff --git a/drivers/pwm/pwm-tegra.c b/drivers/pwm/pwm-tegra.c
index 11a10b5..18cf974 100644
--- a/drivers/pwm/pwm-tegra.c
+++ b/drivers/pwm/pwm-tegra.c
@@ -42,12 +42,16 @@
#include <linux/module.h>
#include <linux/of.h>
#include <linux/of_device.h>
+#include <linux/pm_opp.h>
#include <linux/pwm.h>
#include <linux/platform_device.h>
#include <linux/pinctrl/consumer.h>
+#include <linux/pm_runtime.h>
#include <linux/slab.h>
#include <linux/reset.h>
+#include <soc/tegra/common.h>
+
#define PWM_ENABLE (1 << 31)
#define PWM_DUTY_WIDTH 8
#define PWM_DUTY_SHIFT 16
@@ -145,7 +149,7 @@ static int tegra_pwm_config(struct pwm_chip *chip, struct pwm_device *pwm,
required_clk_rate =
(NSEC_PER_SEC / period_ns) << PWM_DUTY_WIDTH;
- err = clk_set_rate(pc->clk, required_clk_rate);
+ err = dev_pm_opp_set_rate(pc->dev, required_clk_rate);
if (err < 0)
return -EINVAL;
@@ -181,8 +185,8 @@ static int tegra_pwm_config(struct pwm_chip *chip, struct pwm_device *pwm,
* before writing the register. Otherwise, keep it enabled.
*/
if (!pwm_is_enabled(pwm)) {
- err = clk_prepare_enable(pc->clk);
- if (err < 0)
+ err = pm_runtime_resume_and_get(pc->dev);
+ if (err)
return err;
} else
val |= PWM_ENABLE;
@@ -193,7 +197,7 @@ static int tegra_pwm_config(struct pwm_chip *chip, struct pwm_device *pwm,
* If the PWM is not enabled, turn the clock off again to save power.
*/
if (!pwm_is_enabled(pwm))
- clk_disable_unprepare(pc->clk);
+ pm_runtime_put(pc->dev);
return 0;
}
@@ -204,8 +208,8 @@ static int tegra_pwm_enable(struct pwm_chip *chip, struct pwm_device *pwm)
int rc = 0;
u32 val;
- rc = clk_prepare_enable(pc->clk);
- if (rc < 0)
+ rc = pm_runtime_resume_and_get(pc->dev);
+ if (rc)
return rc;
val = pwm_readl(pc, pwm->hwpwm);
@@ -224,7 +228,7 @@ static void tegra_pwm_disable(struct pwm_chip *chip, struct pwm_device *pwm)
val &= ~PWM_ENABLE;
pwm_writel(pc, pwm->hwpwm, val);
- clk_disable_unprepare(pc->clk);
+ pm_runtime_put_sync(pc->dev);
}
static const struct pwm_ops tegra_pwm_ops = {
@@ -256,11 +260,20 @@ static int tegra_pwm_probe(struct platform_device *pdev)
if (IS_ERR(pwm->clk))
return PTR_ERR(pwm->clk);
+ ret = devm_tegra_core_dev_init_opp_table_common(&pdev->dev);
+ if (ret)
+ return ret;
+
+ pm_runtime_enable(&pdev->dev);
+ ret = pm_runtime_resume_and_get(&pdev->dev);
+ if (ret)
+ return ret;
+
/* Set maximum frequency of the IP */
- ret = clk_set_rate(pwm->clk, pwm->soc->max_frequency);
+ ret = dev_pm_opp_set_rate(pwm->dev, pwm->soc->max_frequency);
if (ret < 0) {
dev_err(&pdev->dev, "Failed to set max frequency: %d\n", ret);
- return ret;
+ goto put_pm;
}
/*
@@ -278,7 +291,7 @@ static int tegra_pwm_probe(struct platform_device *pdev)
if (IS_ERR(pwm->rst)) {
ret = PTR_ERR(pwm->rst);
dev_err(&pdev->dev, "Reset control is not found: %d\n", ret);
- return ret;
+ goto put_pm;
}
reset_control_deassert(pwm->rst);
@@ -291,10 +304,16 @@ static int tegra_pwm_probe(struct platform_device *pdev)
if (ret < 0) {
dev_err(&pdev->dev, "pwmchip_add() failed: %d\n", ret);
reset_control_assert(pwm->rst);
- return ret;
+ goto put_pm;
}
+ pm_runtime_put(&pdev->dev);
+
return 0;
+put_pm:
+ pm_runtime_put_sync_suspend(&pdev->dev);
+ pm_runtime_force_suspend(&pdev->dev);
+ return ret;
}
static int tegra_pwm_remove(struct platform_device *pdev)
@@ -305,20 +324,44 @@ static int tegra_pwm_remove(struct platform_device *pdev)
reset_control_assert(pc->rst);
+ pm_runtime_force_suspend(&pdev->dev);
+
return 0;
}
-#ifdef CONFIG_PM_SLEEP
-static int tegra_pwm_suspend(struct device *dev)
+static int __maybe_unused tegra_pwm_runtime_suspend(struct device *dev)
{
- return pinctrl_pm_select_sleep_state(dev);
+ struct tegra_pwm_chip *pc = dev_get_drvdata(dev);
+ int err;
+
+ clk_disable_unprepare(pc->clk);
+
+ err = pinctrl_pm_select_sleep_state(dev);
+ if (err) {
+ clk_prepare_enable(pc->clk);
+ return err;
+ }
+
+ return 0;
}
-static int tegra_pwm_resume(struct device *dev)
+static int __maybe_unused tegra_pwm_runtime_resume(struct device *dev)
{
- return pinctrl_pm_select_default_state(dev);
+ struct tegra_pwm_chip *pc = dev_get_drvdata(dev);
+ int err;
+
+ err = pinctrl_pm_select_default_state(dev);
+ if (err)
+ return err;
+
+ err = clk_prepare_enable(pc->clk);
+ if (err) {
+ pinctrl_pm_select_sleep_state(dev);
+ return err;
+ }
+
+ return 0;
}
-#endif
static const struct tegra_pwm_soc tegra20_pwm_soc = {
.num_channels = 4,
@@ -344,7 +387,10 @@ static const struct of_device_id tegra_pwm_of_match[] = {
MODULE_DEVICE_TABLE(of, tegra_pwm_of_match);
static const struct dev_pm_ops tegra_pwm_pm_ops = {
- SET_SYSTEM_SLEEP_PM_OPS(tegra_pwm_suspend, tegra_pwm_resume)
+ SET_RUNTIME_PM_OPS(tegra_pwm_runtime_suspend, tegra_pwm_runtime_resume,
+ NULL)
+ SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend,
+ pm_runtime_force_resume)
};
static struct platform_driver tegra_pwm_driver = {
diff --git a/drivers/ras/cec.c b/drivers/ras/cec.c
index d7894f1..42f2fc0 100644
--- a/drivers/ras/cec.c
+++ b/drivers/ras/cec.c
@@ -38,7 +38,7 @@
* elements entered into the array, during which, we're decaying all elements.
* If, after decay, an element gets inserted again, its generation is set to 11b
* to make sure it has higher numerical count than other, older elements and
- * thus emulate an an LRU-like behavior when deleting elements to free up space
+ * thus emulate an LRU-like behavior when deleting elements to free up space
* in the page.
*
* When an element reaches it's max count of action_threshold, we try to poison
diff --git a/drivers/reset/Kconfig b/drivers/reset/Kconfig
index 85024eb..6f8ba0d 100644
--- a/drivers/reset/Kconfig
+++ b/drivers/reset/Kconfig
@@ -224,6 +224,13 @@
This enables the reset driver for the SoCFPGA ARMv7 platforms. This
driver gets initialized early during platform init calls.
+config RESET_STARFIVE_JH7100
+ bool "StarFive JH7100 Reset Driver"
+ depends on SOC_STARFIVE || COMPILE_TEST
+ default SOC_STARFIVE
+ help
+ This enables the reset controller driver for the StarFive JH7100 SoC.
+
config RESET_SUNXI
bool "Allwinner SoCs Reset Driver" if COMPILE_TEST && !ARCH_SUNXI
default ARCH_SUNXI
diff --git a/drivers/reset/Makefile b/drivers/reset/Makefile
index 21d46d8..bd0a97be 100644
--- a/drivers/reset/Makefile
+++ b/drivers/reset/Makefile
@@ -29,6 +29,7 @@
obj-$(CONFIG_RESET_SCMI) += reset-scmi.o
obj-$(CONFIG_RESET_SIMPLE) += reset-simple.o
obj-$(CONFIG_RESET_SOCFPGA) += reset-socfpga.o
+obj-$(CONFIG_RESET_STARFIVE_JH7100) += reset-starfive-jh7100.o
obj-$(CONFIG_RESET_SUNXI) += reset-sunxi.o
obj-$(CONFIG_RESET_TI_SCI) += reset-ti-sci.o
obj-$(CONFIG_RESET_TI_SYSCON) += reset-ti-syscon.o
diff --git a/drivers/reset/reset-rzg2l-usbphy-ctrl.c b/drivers/reset/reset-rzg2l-usbphy-ctrl.c
index e0704fd..1e83150 100644
--- a/drivers/reset/reset-rzg2l-usbphy-ctrl.c
+++ b/drivers/reset/reset-rzg2l-usbphy-ctrl.c
@@ -137,7 +137,12 @@ static int rzg2l_usbphy_ctrl_probe(struct platform_device *pdev)
dev_set_drvdata(dev, priv);
pm_runtime_enable(&pdev->dev);
- pm_runtime_resume_and_get(&pdev->dev);
+ error = pm_runtime_resume_and_get(&pdev->dev);
+ if (error < 0) {
+ pm_runtime_disable(&pdev->dev);
+ reset_control_assert(priv->rstc);
+ return dev_err_probe(&pdev->dev, error, "pm_runtime_resume_and_get failed");
+ }
/* put pll and phy into reset state */
spin_lock_irqsave(&priv->lock, flags);
diff --git a/drivers/reset/reset-starfive-jh7100.c b/drivers/reset/reset-starfive-jh7100.c
new file mode 100644
index 0000000..fc44b2f
--- /dev/null
+++ b/drivers/reset/reset-starfive-jh7100.c
@@ -0,0 +1,173 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Reset driver for the StarFive JH7100 SoC
+ *
+ * Copyright (C) 2021 Emil Renner Berthing <kernel@esmil.dk>
+ */
+
+#include <linux/bitmap.h>
+#include <linux/io.h>
+#include <linux/io-64-nonatomic-lo-hi.h>
+#include <linux/iopoll.h>
+#include <linux/mod_devicetable.h>
+#include <linux/platform_device.h>
+#include <linux/reset-controller.h>
+#include <linux/spinlock.h>
+
+#include <dt-bindings/reset/starfive-jh7100.h>
+
+/* register offsets */
+#define JH7100_RESET_ASSERT0 0x00
+#define JH7100_RESET_ASSERT1 0x04
+#define JH7100_RESET_ASSERT2 0x08
+#define JH7100_RESET_ASSERT3 0x0c
+#define JH7100_RESET_STATUS0 0x10
+#define JH7100_RESET_STATUS1 0x14
+#define JH7100_RESET_STATUS2 0x18
+#define JH7100_RESET_STATUS3 0x1c
+
+/*
+ * Writing a 1 to the n'th bit of the m'th ASSERT register asserts
+ * line 32m + n, and writing a 0 deasserts the same line.
+ * Most reset lines have their status inverted so a 0 bit in the STATUS
+ * register means the line is asserted and a 1 means it's deasserted. A few
+ * lines don't though, so store the expected value of the status registers when
+ * all lines are asserted.
+ */
+static const u64 jh7100_reset_asserted[2] = {
+ /* STATUS0 */
+ BIT_ULL_MASK(JH7100_RST_U74) |
+ BIT_ULL_MASK(JH7100_RST_VP6_DRESET) |
+ BIT_ULL_MASK(JH7100_RST_VP6_BRESET) |
+ /* STATUS1 */
+ BIT_ULL_MASK(JH7100_RST_HIFI4_DRESET) |
+ BIT_ULL_MASK(JH7100_RST_HIFI4_BRESET),
+ /* STATUS2 */
+ BIT_ULL_MASK(JH7100_RST_E24) |
+ /* STATUS3 */
+ 0,
+};
+
+struct jh7100_reset {
+ struct reset_controller_dev rcdev;
+ /* protect registers against concurrent read-modify-write */
+ spinlock_t lock;
+ void __iomem *base;
+};
+
+static inline struct jh7100_reset *
+jh7100_reset_from(struct reset_controller_dev *rcdev)
+{
+ return container_of(rcdev, struct jh7100_reset, rcdev);
+}
+
+static int jh7100_reset_update(struct reset_controller_dev *rcdev,
+ unsigned long id, bool assert)
+{
+ struct jh7100_reset *data = jh7100_reset_from(rcdev);
+ unsigned long offset = BIT_ULL_WORD(id);
+ u64 mask = BIT_ULL_MASK(id);
+ void __iomem *reg_assert = data->base + JH7100_RESET_ASSERT0 + offset * sizeof(u64);
+ void __iomem *reg_status = data->base + JH7100_RESET_STATUS0 + offset * sizeof(u64);
+ u64 done = jh7100_reset_asserted[offset] & mask;
+ u64 value;
+ unsigned long flags;
+ int ret;
+
+ if (!assert)
+ done ^= mask;
+
+ spin_lock_irqsave(&data->lock, flags);
+
+ value = readq(reg_assert);
+ if (assert)
+ value |= mask;
+ else
+ value &= ~mask;
+ writeq(value, reg_assert);
+
+ /* if the associated clock is gated, deasserting might otherwise hang forever */
+ ret = readq_poll_timeout_atomic(reg_status, value, (value & mask) == done, 0, 1000);
+
+ spin_unlock_irqrestore(&data->lock, flags);
+ return ret;
+}
+
+static int jh7100_reset_assert(struct reset_controller_dev *rcdev,
+ unsigned long id)
+{
+ return jh7100_reset_update(rcdev, id, true);
+}
+
+static int jh7100_reset_deassert(struct reset_controller_dev *rcdev,
+ unsigned long id)
+{
+ return jh7100_reset_update(rcdev, id, false);
+}
+
+static int jh7100_reset_reset(struct reset_controller_dev *rcdev,
+ unsigned long id)
+{
+ int ret;
+
+ ret = jh7100_reset_assert(rcdev, id);
+ if (ret)
+ return ret;
+
+ return jh7100_reset_deassert(rcdev, id);
+}
+
+static int jh7100_reset_status(struct reset_controller_dev *rcdev,
+ unsigned long id)
+{
+ struct jh7100_reset *data = jh7100_reset_from(rcdev);
+ unsigned long offset = BIT_ULL_WORD(id);
+ u64 mask = BIT_ULL_MASK(id);
+ void __iomem *reg_status = data->base + JH7100_RESET_STATUS0 + offset * sizeof(u64);
+ u64 value = readq(reg_status);
+
+ return !((value ^ jh7100_reset_asserted[offset]) & mask);
+}
+
+static const struct reset_control_ops jh7100_reset_ops = {
+ .assert = jh7100_reset_assert,
+ .deassert = jh7100_reset_deassert,
+ .reset = jh7100_reset_reset,
+ .status = jh7100_reset_status,
+};
+
+static int __init jh7100_reset_probe(struct platform_device *pdev)
+{
+ struct jh7100_reset *data;
+
+ data = devm_kzalloc(&pdev->dev, sizeof(*data), GFP_KERNEL);
+ if (!data)
+ return -ENOMEM;
+
+ data->base = devm_platform_ioremap_resource(pdev, 0);
+ if (IS_ERR(data->base))
+ return PTR_ERR(data->base);
+
+ data->rcdev.ops = &jh7100_reset_ops;
+ data->rcdev.owner = THIS_MODULE;
+ data->rcdev.nr_resets = JH7100_RSTN_END;
+ data->rcdev.dev = &pdev->dev;
+ data->rcdev.of_node = pdev->dev.of_node;
+ spin_lock_init(&data->lock);
+
+ return devm_reset_controller_register(&pdev->dev, &data->rcdev);
+}
+
+static const struct of_device_id jh7100_reset_dt_ids[] = {
+ { .compatible = "starfive,jh7100-reset" },
+ { /* sentinel */ }
+};
+
+static struct platform_driver jh7100_reset_driver = {
+ .driver = {
+ .name = "jh7100-reset",
+ .of_match_table = jh7100_reset_dt_ids,
+ .suppress_bind_attrs = true,
+ },
+};
+builtin_platform_driver_probe(jh7100_reset_driver, jh7100_reset_probe);
diff --git a/drivers/s390/block/dasd_devmap.c b/drivers/s390/block/dasd_devmap.c
index 6043c83..811e79c 100644
--- a/drivers/s390/block/dasd_devmap.c
+++ b/drivers/s390/block/dasd_devmap.c
@@ -1824,10 +1824,11 @@ static struct attribute *paths_info_attrs[] = {
&path_fcs_attribute.attr,
NULL,
};
+ATTRIBUTE_GROUPS(paths_info);
static struct kobj_type path_attr_type = {
.release = dasd_path_release,
- .default_attrs = paths_info_attrs,
+ .default_groups = paths_info_groups,
.sysfs_ops = &kobj_sysfs_ops,
};
diff --git a/drivers/s390/char/sclp_early.c b/drivers/s390/char/sclp_early.c
index b64feab..e9943a8 100644
--- a/drivers/s390/char/sclp_early.c
+++ b/drivers/s390/char/sclp_early.c
@@ -139,7 +139,7 @@ int __init sclp_early_get_core_info(struct sclp_core_info *info)
}
sclp_fill_core_info(info, sccb);
out:
- memblock_phys_free((unsigned long)sccb, length);
+ memblock_free(sccb, length);
return rc;
}
diff --git a/drivers/s390/char/sclp_sd.c b/drivers/s390/char/sclp_sd.c
index 25c2d76..f9e164b 100644
--- a/drivers/s390/char/sclp_sd.c
+++ b/drivers/s390/char/sclp_sd.c
@@ -438,11 +438,12 @@ static struct attribute *sclp_sd_file_default_attrs[] = {
&reload_attr.attr,
NULL,
};
+ATTRIBUTE_GROUPS(sclp_sd_file_default);
static struct kobj_type sclp_sd_file_ktype = {
.sysfs_ops = &kobj_sysfs_ops,
.release = sclp_sd_file_release,
- .default_attrs = sclp_sd_file_default_attrs,
+ .default_groups = sclp_sd_file_default_groups,
};
/**
diff --git a/drivers/s390/char/vmcp.c b/drivers/s390/char/vmcp.c
index 9e06628..4cebfaa 100644
--- a/drivers/s390/char/vmcp.c
+++ b/drivers/s390/char/vmcp.c
@@ -72,7 +72,7 @@ static void vmcp_response_alloc(struct vmcp_session *session)
if (order > 2)
page = cma_alloc(vmcp_cma, nr_pages, 0, false);
if (page) {
- session->response = (char *)page_to_phys(page);
+ session->response = (char *)page_to_virt(page);
session->cma_alloc = 1;
return;
}
@@ -89,7 +89,7 @@ static void vmcp_response_free(struct vmcp_session *session)
order = get_order(session->bufsize);
nr_pages = ALIGN(session->bufsize, PAGE_SIZE) >> PAGE_SHIFT;
if (session->cma_alloc) {
- page = phys_to_page((unsigned long)session->response);
+ page = virt_to_page((unsigned long)session->response);
cma_release(vmcp_cma, page, nr_pages);
session->cma_alloc = 0;
} else {
diff --git a/drivers/s390/cio/chsc_sch.c b/drivers/s390/cio/chsc_sch.c
index 684348d..962dfa2 100644
--- a/drivers/s390/cio/chsc_sch.c
+++ b/drivers/s390/cio/chsc_sch.c
@@ -91,11 +91,6 @@ static int chsc_subchannel_probe(struct subchannel *sch)
sch->schid.ssid, sch->schid.sch_no, ret);
dev_set_drvdata(&sch->dev, NULL);
kfree(private);
- } else {
- if (dev_get_uevent_suppress(&sch->dev)) {
- dev_set_uevent_suppress(&sch->dev, 0);
- kobject_uevent(&sch->dev.kobj, KOBJ_ADD);
- }
}
return ret;
}
diff --git a/drivers/s390/cio/css.c b/drivers/s390/cio/css.c
index ce9e751..fa82933 100644
--- a/drivers/s390/cio/css.c
+++ b/drivers/s390/cio/css.c
@@ -470,16 +470,6 @@ int css_register_subchannel(struct subchannel *sch)
if (sch->st == SUBCHANNEL_TYPE_IO)
sch->dev.type = &io_subchannel_type;
- /*
- * We don't want to generate uevents for I/O subchannels that don't
- * have a working ccw device behind them since they will be
- * unregistered before they can be used anyway, so we delay the add
- * uevent until after device recognition was successful.
- * Note that we suppress the uevent for all subchannel types;
- * the subchannel driver can decide itself when it wants to inform
- * userspace of its existence.
- */
- dev_set_uevent_suppress(&sch->dev, 1);
css_update_ssd_info(sch);
/* make it known to the system */
ret = css_sch_device_register(sch);
@@ -488,15 +478,6 @@ int css_register_subchannel(struct subchannel *sch)
sch->schid.ssid, sch->schid.sch_no, ret);
return ret;
}
- if (!sch->driver) {
- /*
- * No driver matched. Generate the uevent now so that
- * a fitting driver module may be loaded based on the
- * modalias.
- */
- dev_set_uevent_suppress(&sch->dev, 0);
- kobject_uevent(&sch->dev.kobj, KOBJ_ADD);
- }
return ret;
}
diff --git a/drivers/s390/cio/device.c b/drivers/s390/cio/device.c
index 07a1761..cd938a2 100644
--- a/drivers/s390/cio/device.c
+++ b/drivers/s390/cio/device.c
@@ -838,14 +838,6 @@ static void io_subchannel_register(struct ccw_device *cdev)
adjust_init_count = 0;
goto out;
}
- /*
- * Now we know this subchannel will stay, we can throw
- * our delayed uevent.
- */
- if (dev_get_uevent_suppress(&sch->dev)) {
- dev_set_uevent_suppress(&sch->dev, 0);
- kobject_uevent(&sch->dev.kobj, KOBJ_ADD);
- }
/* make it known to the system */
ret = device_add(&cdev->dev);
if (ret) {
@@ -1036,15 +1028,9 @@ static int io_subchannel_probe(struct subchannel *sch)
"0.%x.%04x (rc=%d)\n",
sch->schid.ssid, sch->schid.sch_no, rc);
/*
- * The console subchannel already has an associated ccw_device.
- * Throw the delayed uevent for the subchannel, register
- * the ccw_device and exit.
- */
- if (dev_get_uevent_suppress(&sch->dev)) {
- /* should always be the case for the console */
- dev_set_uevent_suppress(&sch->dev, 0);
- kobject_uevent(&sch->dev.kobj, KOBJ_ADD);
- }
+ * The console subchannel already has an associated ccw_device.
+ * Register it and exit.
+ */
cdev = sch_get_cdev(sch);
rc = device_add(&cdev->dev);
if (rc) {
diff --git a/drivers/s390/cio/eadm_sch.c b/drivers/s390/cio/eadm_sch.c
index 15bdae5..8b46368 100644
--- a/drivers/s390/cio/eadm_sch.c
+++ b/drivers/s390/cio/eadm_sch.c
@@ -243,11 +243,6 @@ static int eadm_subchannel_probe(struct subchannel *sch)
spin_lock_irq(&list_lock);
list_add(&private->head, &eadm_list);
spin_unlock_irq(&list_lock);
-
- if (dev_get_uevent_suppress(&sch->dev)) {
- dev_set_uevent_suppress(&sch->dev, 0);
- kobject_uevent(&sch->dev.kobj, KOBJ_ADD);
- }
out:
return ret;
}
diff --git a/drivers/s390/cio/qdio.h b/drivers/s390/cio/qdio.h
index 99c2212d..5ea6249 100644
--- a/drivers/s390/cio/qdio.h
+++ b/drivers/s390/cio/qdio.h
@@ -236,12 +236,11 @@ struct qdio_irq {
int nr_input_qs;
int nr_output_qs;
- struct ccw1 ccw;
- struct ciw equeue;
- struct ciw aqueue;
+ struct ccw1 *ccw;
struct qdio_ssqd_desc ssqd_desc;
void (*orig_handler) (struct ccw_device *, unsigned long, struct irb *);
+ qdio_handler_t (*error_handler);
int perf_stat_enabled;
@@ -338,7 +337,7 @@ void qdio_setup_ssqd_info(struct qdio_irq *irq_ptr);
int qdio_setup_get_ssqd(struct qdio_irq *irq_ptr,
struct subchannel_id *schid,
struct qdio_ssqd_desc *data);
-int qdio_setup_irq(struct qdio_irq *irq_ptr, struct qdio_initialize *init_data);
+void qdio_setup_irq(struct qdio_irq *irq_ptr, struct qdio_initialize *init_data);
void qdio_shutdown_irq(struct qdio_irq *irq);
void qdio_print_subchannel_info(struct qdio_irq *irq_ptr);
void qdio_free_queues(struct qdio_irq *irq_ptr);
diff --git a/drivers/s390/cio/qdio_main.c b/drivers/s390/cio/qdio_main.c
index 45e810c..9cde557 100644
--- a/drivers/s390/cio/qdio_main.c
+++ b/drivers/s390/cio/qdio_main.c
@@ -10,6 +10,7 @@
#include <linux/module.h>
#include <linux/init.h>
#include <linux/kernel.h>
+#include <linux/kmemleak.h>
#include <linux/delay.h>
#include <linux/gfp.h>
#include <linux/io.h>
@@ -169,8 +170,6 @@ static int qdio_do_sqbs(struct qdio_q *q, unsigned char state, int start,
int tmp_count = count, tmp_start = start;
int nr = q->nr;
- if (!count)
- return 0;
qperf_inc(q, sqbs);
if (!q->is_input_q)
@@ -499,6 +498,31 @@ static int get_inbound_buffer_frontier(struct qdio_q *q, unsigned int start,
}
}
+int qdio_inspect_input_queue(struct ccw_device *cdev, unsigned int nr,
+ unsigned int *bufnr, unsigned int *error)
+{
+ struct qdio_irq *irq = cdev->private->qdio_data;
+ unsigned int start;
+ struct qdio_q *q;
+ int count;
+
+ if (!irq)
+ return -ENODEV;
+
+ q = irq->input_qs[nr];
+ start = q->first_to_check;
+ *error = 0;
+
+ count = get_inbound_buffer_frontier(q, start, error);
+ if (count == 0)
+ return 0;
+
+ *bufnr = start;
+ q->first_to_check = add_buf(start, count);
+ return count;
+}
+EXPORT_SYMBOL_GPL(qdio_inspect_input_queue);
+
static inline int qdio_inbound_q_done(struct qdio_q *q, unsigned int start)
{
unsigned char state = 0;
@@ -578,6 +602,31 @@ static int get_outbound_buffer_frontier(struct qdio_q *q, unsigned int start,
}
}
+int qdio_inspect_output_queue(struct ccw_device *cdev, unsigned int nr,
+ unsigned int *bufnr, unsigned int *error)
+{
+ struct qdio_irq *irq = cdev->private->qdio_data;
+ unsigned int start;
+ struct qdio_q *q;
+ int count;
+
+ if (!irq)
+ return -ENODEV;
+
+ q = irq->output_qs[nr];
+ start = q->first_to_check;
+ *error = 0;
+
+ count = get_outbound_buffer_frontier(q, start, error);
+ if (count == 0)
+ return 0;
+
+ *bufnr = start;
+ q->first_to_check = add_buf(start, count);
+ return count;
+}
+EXPORT_SYMBOL_GPL(qdio_inspect_output_queue);
+
static int qdio_kick_outbound_q(struct qdio_q *q, unsigned int count,
unsigned long aob)
{
@@ -653,24 +702,18 @@ static void qdio_handle_activate_check(struct qdio_irq *irq_ptr,
unsigned long intparm, int cstat,
int dstat)
{
- struct qdio_q *q;
+ unsigned int first_to_check = 0;
DBF_ERROR("%4x ACT CHECK", irq_ptr->schid.sch_no);
DBF_ERROR("intp :%lx", intparm);
DBF_ERROR("ds: %2x cs:%2x", dstat, cstat);
- if (irq_ptr->nr_input_qs) {
- q = irq_ptr->input_qs[0];
- } else if (irq_ptr->nr_output_qs) {
- q = irq_ptr->output_qs[0];
- } else {
- dump_stack();
- goto no_handler;
- }
+ /* zfcp wants this: */
+ if (irq_ptr->nr_input_qs)
+ first_to_check = irq_ptr->input_qs[0]->first_to_check;
- q->handler(q->irq_ptr->cdev, QDIO_ERROR_ACTIVATE,
- q->nr, q->first_to_check, 0, irq_ptr->int_parm);
-no_handler:
+ irq_ptr->error_handler(irq_ptr->cdev, QDIO_ERROR_ACTIVATE, 0,
+ first_to_check, 0, irq_ptr->int_parm);
qdio_set_state(irq_ptr, QDIO_IRQ_STATE_STOPPED);
/*
* In case of z/VM LGR (Live Guest Migration) QDIO recovery will happen.
@@ -874,6 +917,7 @@ int qdio_free(struct ccw_device *cdev)
qdio_free_queues(irq_ptr);
free_page((unsigned long) irq_ptr->qdr);
free_page(irq_ptr->chsc_page);
+ kfree(irq_ptr->ccw);
free_page((unsigned long) irq_ptr);
return 0;
}
@@ -899,11 +943,17 @@ int qdio_allocate(struct ccw_device *cdev, unsigned int no_input_qs,
no_output_qs > QDIO_MAX_QUEUES_PER_IRQ)
return -EINVAL;
- /* irq_ptr must be in GFP_DMA since it contains ccw1.cda */
- irq_ptr = (void *) get_zeroed_page(GFP_KERNEL | GFP_DMA);
+ irq_ptr = (void *) get_zeroed_page(GFP_KERNEL);
if (!irq_ptr)
return -ENOMEM;
+ irq_ptr->ccw = kmalloc(sizeof(*irq_ptr->ccw), GFP_KERNEL | GFP_DMA);
+ if (!irq_ptr->ccw)
+ goto err_ccw;
+
+ /* kmemleak doesn't scan the page-allocated irq_ptr: */
+ kmemleak_not_leak(irq_ptr->ccw);
+
irq_ptr->cdev = cdev;
mutex_init(&irq_ptr->setup_mutex);
if (qdio_allocate_dbf(irq_ptr))
@@ -941,6 +991,8 @@ int qdio_allocate(struct ccw_device *cdev, unsigned int no_input_qs,
free_page(irq_ptr->chsc_page);
err_chsc:
err_dbf:
+ kfree(irq_ptr->ccw);
+err_ccw:
free_page((unsigned long) irq_ptr);
return rc;
}
@@ -972,6 +1024,7 @@ int qdio_establish(struct ccw_device *cdev,
{
struct qdio_irq *irq_ptr = cdev->private->qdio_data;
struct subchannel_id schid;
+ struct ciw *ciw;
long timeout;
int rc;
@@ -985,8 +1038,11 @@ int qdio_establish(struct ccw_device *cdev,
init_data->no_output_qs > irq_ptr->max_output_qs)
return -EINVAL;
- if ((init_data->no_input_qs && !init_data->input_handler) ||
- (init_data->no_output_qs && !init_data->output_handler))
+ /* Needed as error_handler: */
+ if (!init_data->input_handler)
+ return -EINVAL;
+
+ if (init_data->no_output_qs && !init_data->output_handler)
return -EINVAL;
if (!init_data->input_sbal_addr_array ||
@@ -996,6 +1052,12 @@ int qdio_establish(struct ccw_device *cdev,
if (!init_data->irq_poll)
return -EINVAL;
+ ciw = ccw_device_get_ciw(cdev, CIW_TYPE_EQUEUE);
+ if (!ciw) {
+ DBF_ERROR("%4x NO EQ", schid.sch_no);
+ return -EIO;
+ }
+
mutex_lock(&irq_ptr->setup_mutex);
qdio_trace_init_data(irq_ptr, init_data);
qdio_setup_irq(irq_ptr, init_data);
@@ -1005,15 +1067,15 @@ int qdio_establish(struct ccw_device *cdev,
goto err_thinint;
/* establish q */
- irq_ptr->ccw.cmd_code = irq_ptr->equeue.cmd;
- irq_ptr->ccw.flags = CCW_FLAG_SLI;
- irq_ptr->ccw.count = irq_ptr->equeue.count;
- irq_ptr->ccw.cda = (u32) virt_to_phys(irq_ptr->qdr);
+ irq_ptr->ccw->cmd_code = ciw->cmd;
+ irq_ptr->ccw->flags = CCW_FLAG_SLI;
+ irq_ptr->ccw->count = ciw->count;
+ irq_ptr->ccw->cda = (u32) virt_to_phys(irq_ptr->qdr);
spin_lock_irq(get_ccwdev_lock(cdev));
ccw_device_set_options_mask(cdev, 0);
- rc = ccw_device_start(cdev, &irq_ptr->ccw, QDIO_DOING_ESTABLISH, 0, 0);
+ rc = ccw_device_start(cdev, irq_ptr->ccw, QDIO_DOING_ESTABLISH, 0, 0);
spin_unlock_irq(get_ccwdev_lock(cdev));
if (rc) {
DBF_ERROR("%4x est IO ERR", irq_ptr->schid.sch_no);
@@ -1065,6 +1127,7 @@ int qdio_activate(struct ccw_device *cdev)
{
struct qdio_irq *irq_ptr = cdev->private->qdio_data;
struct subchannel_id schid;
+ struct ciw *ciw;
int rc;
ccw_device_get_schid(cdev, &schid);
@@ -1073,21 +1136,27 @@ int qdio_activate(struct ccw_device *cdev)
if (!irq_ptr)
return -ENODEV;
+ ciw = ccw_device_get_ciw(cdev, CIW_TYPE_AQUEUE);
+ if (!ciw) {
+ DBF_ERROR("%4x NO AQ", schid.sch_no);
+ return -EIO;
+ }
+
mutex_lock(&irq_ptr->setup_mutex);
if (irq_ptr->state == QDIO_IRQ_STATE_INACTIVE) {
rc = -EBUSY;
goto out;
}
- irq_ptr->ccw.cmd_code = irq_ptr->aqueue.cmd;
- irq_ptr->ccw.flags = CCW_FLAG_SLI;
- irq_ptr->ccw.count = irq_ptr->aqueue.count;
- irq_ptr->ccw.cda = 0;
+ irq_ptr->ccw->cmd_code = ciw->cmd;
+ irq_ptr->ccw->flags = CCW_FLAG_SLI;
+ irq_ptr->ccw->count = ciw->count;
+ irq_ptr->ccw->cda = 0;
spin_lock_irq(get_ccwdev_lock(cdev));
ccw_device_set_options(cdev, CCWDEV_REPORT_ALL);
- rc = ccw_device_start(cdev, &irq_ptr->ccw, QDIO_DOING_ACTIVATE,
+ rc = ccw_device_start(cdev, irq_ptr->ccw, QDIO_DOING_ACTIVATE,
0, DOIO_DENY_PREFETCH);
spin_unlock_irq(get_ccwdev_lock(cdev));
if (rc) {
@@ -1144,6 +1213,35 @@ static int handle_inbound(struct qdio_q *q, int bufnr, int count)
}
/**
+ * qdio_add_bufs_to_input_queue - process buffers on an Input Queue
+ * @cdev: associated ccw_device for the qdio subchannel
+ * @q_nr: queue number
+ * @bufnr: buffer number
+ * @count: how many buffers to process
+ */
+int qdio_add_bufs_to_input_queue(struct ccw_device *cdev, unsigned int q_nr,
+ unsigned int bufnr, unsigned int count)
+{
+ struct qdio_irq *irq_ptr = cdev->private->qdio_data;
+
+ if (bufnr >= QDIO_MAX_BUFFERS_PER_Q || count > QDIO_MAX_BUFFERS_PER_Q)
+ return -EINVAL;
+
+ if (!irq_ptr)
+ return -ENODEV;
+
+ DBF_DEV_EVENT(DBF_INFO, irq_ptr, "addi b:%02x c:%02x", bufnr, count);
+
+ if (irq_ptr->state != QDIO_IRQ_STATE_ACTIVE)
+ return -EIO;
+ if (!count)
+ return 0;
+
+ return handle_inbound(irq_ptr->input_qs[q_nr], bufnr, count);
+}
+EXPORT_SYMBOL_GPL(qdio_add_bufs_to_input_queue);
+
+/**
* handle_outbound - process filled outbound buffers
* @q: queue containing the buffers
* @bufnr: first buffer to process
@@ -1184,16 +1282,16 @@ static int handle_outbound(struct qdio_q *q, unsigned int bufnr, unsigned int co
}
/**
- * do_QDIO - process input or output buffers
+ * qdio_add_bufs_to_output_queue - process buffers on an Output Queue
* @cdev: associated ccw_device for the qdio subchannel
- * @callflags: input or output and special flags from the program
* @q_nr: queue number
* @bufnr: buffer number
* @count: how many buffers to process
- * @aob: asynchronous operation block (outbound only)
+ * @aob: asynchronous operation block
*/
-int do_QDIO(struct ccw_device *cdev, unsigned int callflags,
- int q_nr, unsigned int bufnr, unsigned int count, struct qaob *aob)
+int qdio_add_bufs_to_output_queue(struct ccw_device *cdev, unsigned int q_nr,
+ unsigned int bufnr, unsigned int count,
+ struct qaob *aob)
{
struct qdio_irq *irq_ptr = cdev->private->qdio_data;
@@ -1203,20 +1301,16 @@ int do_QDIO(struct ccw_device *cdev, unsigned int callflags,
if (!irq_ptr)
return -ENODEV;
- DBF_DEV_EVENT(DBF_INFO, irq_ptr,
- "do%02x b:%02x c:%02x", callflags, bufnr, count);
+ DBF_DEV_EVENT(DBF_INFO, irq_ptr, "addo b:%02x c:%02x", bufnr, count);
if (irq_ptr->state != QDIO_IRQ_STATE_ACTIVE)
return -EIO;
if (!count)
return 0;
- if (callflags & QDIO_FLAG_SYNC_INPUT)
- return handle_inbound(irq_ptr->input_qs[q_nr], bufnr, count);
- else if (callflags & QDIO_FLAG_SYNC_OUTPUT)
- return handle_outbound(irq_ptr->output_qs[q_nr], bufnr, count, aob);
- return -EINVAL;
+
+ return handle_outbound(irq_ptr->output_qs[q_nr], bufnr, count, aob);
}
-EXPORT_SYMBOL_GPL(do_QDIO);
+EXPORT_SYMBOL_GPL(qdio_add_bufs_to_output_queue);
/**
* qdio_start_irq - enable interrupt processing for the device
@@ -1263,40 +1357,6 @@ int qdio_start_irq(struct ccw_device *cdev)
}
EXPORT_SYMBOL(qdio_start_irq);
-static int __qdio_inspect_queue(struct qdio_q *q, unsigned int *bufnr,
- unsigned int *error)
-{
- unsigned int start = q->first_to_check;
- int count;
-
- *error = 0;
- count = q->is_input_q ? get_inbound_buffer_frontier(q, start, error) :
- get_outbound_buffer_frontier(q, start, error);
- if (count == 0)
- return 0;
-
- *bufnr = start;
-
- /* for the next time */
- q->first_to_check = add_buf(start, count);
-
- return count;
-}
-
-int qdio_inspect_queue(struct ccw_device *cdev, unsigned int nr, bool is_input,
- unsigned int *bufnr, unsigned int *error)
-{
- struct qdio_irq *irq_ptr = cdev->private->qdio_data;
- struct qdio_q *q;
-
- if (!irq_ptr)
- return -ENODEV;
- q = is_input ? irq_ptr->input_qs[nr] : irq_ptr->output_qs[nr];
-
- return __qdio_inspect_queue(q, bufnr, error);
-}
-EXPORT_SYMBOL_GPL(qdio_inspect_queue);
-
/**
* qdio_stop_irq - disable interrupt processing for the device
* @cdev: associated ccw_device for the qdio subchannel
diff --git a/drivers/s390/cio/qdio_setup.c b/drivers/s390/cio/qdio_setup.c
index efbb5e5..714878e 100644
--- a/drivers/s390/cio/qdio_setup.c
+++ b/drivers/s390/cio/qdio_setup.c
@@ -351,19 +351,18 @@ static void setup_qib(struct qdio_irq *irq_ptr,
sizeof(irq_ptr->qib.parm));
}
-int qdio_setup_irq(struct qdio_irq *irq_ptr, struct qdio_initialize *init_data)
+void qdio_setup_irq(struct qdio_irq *irq_ptr, struct qdio_initialize *init_data)
{
struct ccw_device *cdev = irq_ptr->cdev;
- struct ciw *ciw;
irq_ptr->qdioac1 = 0;
- memset(&irq_ptr->ccw, 0, sizeof(irq_ptr->ccw));
memset(&irq_ptr->ssqd_desc, 0, sizeof(irq_ptr->ssqd_desc));
memset(&irq_ptr->perf_stat, 0, sizeof(irq_ptr->perf_stat));
irq_ptr->debugfs_dev = NULL;
irq_ptr->sch_token = irq_ptr->perf_stat_enabled = 0;
irq_ptr->state = QDIO_IRQ_STATE_INACTIVE;
+ irq_ptr->error_handler = init_data->input_handler;
irq_ptr->int_parm = init_data->int_parm;
irq_ptr->nr_input_qs = init_data->no_input_qs;
@@ -386,23 +385,6 @@ int qdio_setup_irq(struct qdio_irq *irq_ptr, struct qdio_initialize *init_data)
irq_ptr->orig_handler = cdev->handler;
cdev->handler = qdio_int_handler;
spin_unlock_irq(get_ccwdev_lock(cdev));
-
- /* get qdio commands */
- ciw = ccw_device_get_ciw(cdev, CIW_TYPE_EQUEUE);
- if (!ciw) {
- DBF_ERROR("%4x NO EQ", irq_ptr->schid.sch_no);
- return -EINVAL;
- }
- irq_ptr->equeue = *ciw;
-
- ciw = ccw_device_get_ciw(cdev, CIW_TYPE_AQUEUE);
- if (!ciw) {
- DBF_ERROR("%4x NO AQ", irq_ptr->schid.sch_no);
- return -EINVAL;
- }
- irq_ptr->aqueue = *ciw;
-
- return 0;
}
void qdio_shutdown_irq(struct qdio_irq *irq)
diff --git a/drivers/s390/cio/vfio_ccw_drv.c b/drivers/s390/cio/vfio_ccw_drv.c
index 04074277..ee182cf 100644
--- a/drivers/s390/cio/vfio_ccw_drv.c
+++ b/drivers/s390/cio/vfio_ccw_drv.c
@@ -244,11 +244,6 @@ static int vfio_ccw_sch_probe(struct subchannel *sch)
if (ret)
goto out_disable;
- if (dev_get_uevent_suppress(&sch->dev)) {
- dev_set_uevent_suppress(&sch->dev, 0);
- kobject_uevent(&sch->dev.kobj, KOBJ_ADD);
- }
-
VFIO_CCW_MSG_EVENT(4, "bound to subchannel %x.%x.%04x\n",
sch->schid.cssid, sch->schid.ssid,
sch->schid.sch_no);
diff --git a/drivers/s390/crypto/vfio_ap_drv.c b/drivers/s390/crypto/vfio_ap_drv.c
index 03311a4..e043ae2 100644
--- a/drivers/s390/crypto/vfio_ap_drv.c
+++ b/drivers/s390/crypto/vfio_ap_drv.c
@@ -17,6 +17,9 @@
#define VFIO_AP_ROOT_NAME "vfio_ap"
#define VFIO_AP_DEV_NAME "matrix"
+#define AP_QUEUE_ASSIGNED "assigned"
+#define AP_QUEUE_UNASSIGNED "unassigned"
+#define AP_QUEUE_IN_USE "in use"
MODULE_AUTHOR("IBM Corporation");
MODULE_DESCRIPTION("VFIO AP device driver, Copyright IBM Corp. 2018");
@@ -41,26 +44,95 @@ static struct ap_device_id ap_queue_ids[] = {
MODULE_DEVICE_TABLE(vfio_ap, ap_queue_ids);
+static struct ap_matrix_mdev *vfio_ap_mdev_for_queue(struct vfio_ap_queue *q)
+{
+ struct ap_matrix_mdev *matrix_mdev;
+ unsigned long apid = AP_QID_CARD(q->apqn);
+ unsigned long apqi = AP_QID_QUEUE(q->apqn);
+
+ list_for_each_entry(matrix_mdev, &matrix_dev->mdev_list, node) {
+ if (test_bit_inv(apid, matrix_mdev->matrix.apm) &&
+ test_bit_inv(apqi, matrix_mdev->matrix.aqm))
+ return matrix_mdev;
+ }
+
+ return NULL;
+}
+
+static ssize_t status_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ ssize_t nchars = 0;
+ struct vfio_ap_queue *q;
+ struct ap_matrix_mdev *matrix_mdev;
+ struct ap_device *apdev = to_ap_dev(dev);
+
+ mutex_lock(&matrix_dev->lock);
+ q = dev_get_drvdata(&apdev->device);
+ matrix_mdev = vfio_ap_mdev_for_queue(q);
+
+ if (matrix_mdev) {
+ if (matrix_mdev->kvm)
+ nchars = scnprintf(buf, PAGE_SIZE, "%s\n",
+ AP_QUEUE_IN_USE);
+ else
+ nchars = scnprintf(buf, PAGE_SIZE, "%s\n",
+ AP_QUEUE_ASSIGNED);
+ } else {
+ nchars = scnprintf(buf, PAGE_SIZE, "%s\n",
+ AP_QUEUE_UNASSIGNED);
+ }
+
+ mutex_unlock(&matrix_dev->lock);
+
+ return nchars;
+}
+
+static DEVICE_ATTR_RO(status);
+
+static struct attribute *vfio_queue_attrs[] = {
+ &dev_attr_status.attr,
+ NULL,
+};
+
+static const struct attribute_group vfio_queue_attr_group = {
+ .attrs = vfio_queue_attrs,
+};
+
/**
* vfio_ap_queue_dev_probe: Allocate a vfio_ap_queue structure and associate it
* with the device as driver_data.
*
* @apdev: the AP device being probed
*
- * Return: returns 0 if the probe succeeded; otherwise, returns -ENOMEM if
- * storage could not be allocated for a vfio_ap_queue object.
+ * Return: returns 0 if the probe succeeded; otherwise, returns an error if
+ * storage could not be allocated for a vfio_ap_queue object or the
+ * sysfs 'status' attribute could not be created for the queue device.
*/
static int vfio_ap_queue_dev_probe(struct ap_device *apdev)
{
+ int ret;
struct vfio_ap_queue *q;
q = kzalloc(sizeof(*q), GFP_KERNEL);
if (!q)
return -ENOMEM;
+
+ mutex_lock(&matrix_dev->lock);
dev_set_drvdata(&apdev->device, q);
q->apqn = to_ap_queue(&apdev->device)->qid;
q->saved_isc = VFIO_AP_ISC_INVALID;
- return 0;
+
+ ret = sysfs_create_group(&apdev->device.kobj, &vfio_queue_attr_group);
+ if (ret) {
+ dev_set_drvdata(&apdev->device, NULL);
+ kfree(q);
+ }
+
+ mutex_unlock(&matrix_dev->lock);
+
+ return ret;
}
/**
@@ -75,6 +147,7 @@ static void vfio_ap_queue_dev_remove(struct ap_device *apdev)
struct vfio_ap_queue *q;
mutex_lock(&matrix_dev->lock);
+ sysfs_remove_group(&apdev->device.kobj, &vfio_queue_attr_group);
q = dev_get_drvdata(&apdev->device);
vfio_ap_mdev_reset_queue(q, 1);
dev_set_drvdata(&apdev->device, NULL);
diff --git a/drivers/s390/crypto/zcrypt_api.c b/drivers/s390/crypto/zcrypt_api.c
index 4c3dcc4..9811ab8 100644
--- a/drivers/s390/crypto/zcrypt_api.c
+++ b/drivers/s390/crypto/zcrypt_api.c
@@ -878,14 +878,13 @@ static long _zcrypt_send_cprb(bool userspace, struct ap_perms *perms,
/*
* If a valid target domain is set and this domain is NOT a usage
- * domain but a control only domain, use the default domain as target.
+ * domain but a control only domain, autoselect target domain.
*/
tdom = *domain;
if (tdom < AP_DOMAINS &&
!ap_test_config_usage_domain(tdom) &&
- ap_test_config_ctrl_domain(tdom) &&
- ap_domain_index >= 0)
- tdom = ap_domain_index;
+ ap_test_config_ctrl_domain(tdom))
+ tdom = AUTOSEL_DOM;
pref_zc = NULL;
pref_zq = NULL;
diff --git a/drivers/s390/net/ctcm_dbug.h b/drivers/s390/net/ctcm_dbug.h
index 675575e..cce11da 100644
--- a/drivers/s390/net/ctcm_dbug.h
+++ b/drivers/s390/net/ctcm_dbug.h
@@ -65,6 +65,7 @@ extern struct ctcm_dbf_info ctcm_dbf[CTCM_DBF_INFOS];
int ctcm_register_dbf_views(void);
void ctcm_unregister_dbf_views(void);
+__printf(3, 4)
void ctcm_dbf_longtext(enum ctcm_dbf_names dbf_nix, int level, char *text, ...);
static inline const char *strtail(const char *s, int n)
diff --git a/drivers/s390/net/ctcm_fsms.c b/drivers/s390/net/ctcm_fsms.c
index de2423c..5db591c 100644
--- a/drivers/s390/net/ctcm_fsms.c
+++ b/drivers/s390/net/ctcm_fsms.c
@@ -1406,7 +1406,7 @@ static void ctcmpc_chx_rx(fsm_instance *fi, int event, void *arg)
if (new_skb == NULL) {
CTCM_DBF_TEXT_(MPC_ERROR, CTC_DBF_ERROR,
- "%s(%d): skb allocation failed",
+ "%s(%s): skb allocation failed",
CTCM_FUNTAIL, dev->name);
fsm_event(priv->mpcg->fsm, MPCG_EVENT_INOP, dev);
goto again;
diff --git a/drivers/s390/net/lcs.c b/drivers/s390/net/lcs.c
index 2a64797..a61d38a 100644
--- a/drivers/s390/net/lcs.c
+++ b/drivers/s390/net/lcs.c
@@ -1808,19 +1808,20 @@ lcs_get_frames_cb(struct lcs_channel *channel, struct lcs_buffer *buffer)
return;
}
/* What kind of frame is it? */
- if (lcs_hdr->type == LCS_FRAME_TYPE_CONTROL)
+ if (lcs_hdr->type == LCS_FRAME_TYPE_CONTROL) {
/* Control frame. */
lcs_get_control(card, (struct lcs_cmd *) lcs_hdr);
- else if (lcs_hdr->type == LCS_FRAME_TYPE_ENET ||
- lcs_hdr->type == LCS_FRAME_TYPE_TR ||
- lcs_hdr->type == LCS_FRAME_TYPE_FDDI)
+ } else if (lcs_hdr->type == LCS_FRAME_TYPE_ENET ||
+ lcs_hdr->type == LCS_FRAME_TYPE_TR ||
+ lcs_hdr->type == LCS_FRAME_TYPE_FDDI) {
/* Normal network packet. */
lcs_get_skb(card, (char *)(lcs_hdr + 1),
lcs_hdr->offset - offset -
sizeof(struct lcs_header));
- else
+ } else {
/* Unknown frame type. */
; // FIXME: error message ?
+ }
/* Proceed to next frame. */
offset = lcs_hdr->offset;
lcs_hdr->offset = LCS_ILLEGAL_OFFSET;
diff --git a/drivers/s390/net/qeth_core.h b/drivers/s390/net/qeth_core.h
index 20dca4c..de25d7a 100644
--- a/drivers/s390/net/qeth_core.h
+++ b/drivers/s390/net/qeth_core.h
@@ -1030,8 +1030,6 @@ static inline int qeth_send_simple_setassparms_v6(struct qeth_card *card,
data, QETH_PROT_IPV6);
}
-int qeth_get_priority_queue(struct qeth_card *card, struct sk_buff *skb);
-
extern const struct qeth_discipline qeth_l2_discipline;
extern const struct qeth_discipline qeth_l3_discipline;
extern const struct ethtool_ops qeth_ethtool_ops;
@@ -1099,6 +1097,8 @@ void qeth_get_stats64(struct net_device *dev, struct rtnl_link_stats64 *stats);
int qeth_set_real_num_tx_queues(struct qeth_card *card, unsigned int count);
u16 qeth_iqd_select_queue(struct net_device *dev, struct sk_buff *skb,
u8 cast_type, struct net_device *sb_dev);
+u16 qeth_osa_select_queue(struct net_device *dev, struct sk_buff *skb,
+ struct net_device *sb_dev);
int qeth_open(struct net_device *dev);
int qeth_stop(struct net_device *dev);
diff --git a/drivers/s390/net/qeth_core_main.c b/drivers/s390/net/qeth_core_main.c
index 26c55f6..29f0111 100644
--- a/drivers/s390/net/qeth_core_main.c
+++ b/drivers/s390/net/qeth_core_main.c
@@ -194,9 +194,6 @@ static void qeth_clear_working_pool_list(struct qeth_card *card)
&card->qdio.in_buf_pool.entry_list, list)
list_del(&pool_entry->list);
- if (!queue)
- return;
-
for (i = 0; i < ARRAY_SIZE(queue->bufs); i++)
queue->bufs[i].pool_entry = NULL;
}
@@ -275,8 +272,8 @@ int qeth_resize_buffer_pool(struct qeth_card *card, unsigned int count)
QETH_CARD_TEXT(card, 2, "realcbp");
- /* Defer until queue is allocated: */
- if (!card->qdio.in_q)
+ /* Defer until pool is allocated: */
+ if (list_empty(&pool->entry_list))
goto out;
/* Remove entries from the pool: */
@@ -355,8 +352,8 @@ static int qeth_cq_init(struct qeth_card *card)
qdio_reset_buffers(card->qdio.c_q->qdio_bufs,
QDIO_MAX_BUFFERS_PER_Q);
card->qdio.c_q->next_buf_to_init = 127;
- rc = do_QDIO(CARD_DDEV(card), QDIO_FLAG_SYNC_INPUT, 1, 0, 127,
- NULL);
+
+ rc = qdio_add_bufs_to_input_queue(CARD_DDEV(card), 1, 0, 127);
if (rc) {
QETH_CARD_TEXT_(card, 2, "1err%d", rc);
goto out;
@@ -2557,14 +2554,9 @@ static int qeth_alloc_qdio_queues(struct qeth_card *card)
QETH_QDIO_ALLOCATED) != QETH_QDIO_UNINITIALIZED)
return 0;
- QETH_CARD_TEXT(card, 2, "inq");
- card->qdio.in_q = qeth_alloc_qdio_queue();
- if (!card->qdio.in_q)
- goto out_nomem;
-
/* inbound buffer pool */
if (qeth_alloc_buffer_pool(card))
- goto out_freeinq;
+ goto out_buffer_pool;
/* outbound */
for (i = 0; i < card->qdio.no_out_queues; ++i) {
@@ -2605,10 +2597,7 @@ static int qeth_alloc_qdio_queues(struct qeth_card *card)
card->qdio.out_qs[i] = NULL;
}
qeth_free_buffer_pool(card);
-out_freeinq:
- qeth_free_qdio_queue(card->qdio.in_q);
- card->qdio.in_q = NULL;
-out_nomem:
+out_buffer_pool:
atomic_set(&card->qdio.state, QETH_QDIO_UNINITIALIZED);
return -ENOMEM;
}
@@ -2623,11 +2612,12 @@ static void qeth_free_qdio_queues(struct qeth_card *card)
qeth_free_cq(card);
for (j = 0; j < QDIO_MAX_BUFFERS_PER_Q; ++j) {
- if (card->qdio.in_q->bufs[j].rx_skb)
+ if (card->qdio.in_q->bufs[j].rx_skb) {
consume_skb(card->qdio.in_q->bufs[j].rx_skb);
+ card->qdio.in_q->bufs[j].rx_skb = NULL;
+ }
}
- qeth_free_qdio_queue(card->qdio.in_q);
- card->qdio.in_q = NULL;
+
/* inbound buffer pool */
qeth_free_buffer_pool(card);
/* free outbound qdio_qs */
@@ -2926,8 +2916,7 @@ static int qeth_init_qdio_queues(struct qeth_card *card)
}
card->qdio.in_q->next_buf_to_init = QDIO_BUFNR(rx_bufs);
- rc = do_QDIO(CARD_DDEV(card), QDIO_FLAG_SYNC_INPUT, 0, 0, rx_bufs,
- NULL);
+ rc = qdio_add_bufs_to_input_queue(CARD_DDEV(card), 0, 0, rx_bufs);
if (rc) {
QETH_CARD_TEXT_(card, 2, "1err%d", rc);
return rc;
@@ -3415,8 +3404,9 @@ static unsigned int qeth_rx_refill_queue(struct qeth_card *card,
return 0;
}
- rc = do_QDIO(CARD_DDEV(card), QDIO_FLAG_SYNC_INPUT, 0,
- queue->next_buf_to_init, count, NULL);
+ rc = qdio_add_bufs_to_input_queue(CARD_DDEV(card), 0,
+ queue->next_buf_to_init,
+ count);
if (rc) {
QETH_CARD_TEXT(card, 2, "qinberr");
}
@@ -3588,8 +3578,8 @@ static void qeth_flush_buffers(struct qeth_qdio_out_q *queue, int index,
}
QETH_TXQ_STAT_INC(queue, doorbell);
- rc = do_QDIO(CARD_DDEV(card), QDIO_FLAG_SYNC_OUTPUT, queue->queue_no,
- index, count, aob);
+ rc = qdio_add_bufs_to_output_queue(CARD_DDEV(card), queue->queue_no,
+ index, count, aob);
switch (rc) {
case 0:
@@ -3645,12 +3635,10 @@ static void qeth_check_outbound_queue(struct qeth_qdio_out_q *queue)
if ((atomic_read(&queue->used_buffers) <= QETH_LOW_WATERMARK_PACK) ||
!atomic_read(&queue->set_pci_flags_count)) {
unsigned int index, flush_cnt;
- bool q_was_packing;
spin_lock(&queue->lock);
index = queue->next_buf_to_fill;
- q_was_packing = queue->do_pack;
flush_cnt = qeth_switch_to_nonpacking_if_needed(queue);
if (!flush_cnt && !atomic_read(&queue->set_pci_flags_count))
@@ -3658,8 +3646,7 @@ static void qeth_check_outbound_queue(struct qeth_qdio_out_q *queue)
if (flush_cnt) {
qeth_flush_buffers(queue, index, flush_cnt);
- if (q_was_packing)
- QETH_TXQ_STAT_ADD(queue, bufs_pack, flush_cnt);
+ QETH_TXQ_STAT_ADD(queue, bufs_pack, flush_cnt);
}
spin_unlock(&queue->lock);
@@ -3739,8 +3726,8 @@ static void qeth_qdio_cq_handler(struct qeth_card *card, unsigned int qdio_err,
}
qeth_scrub_qdio_buffer(buffer, QDIO_MAX_ELEMENTS_PER_BUFFER);
}
- rc = do_QDIO(CARD_DDEV(card), QDIO_FLAG_SYNC_INPUT, queue,
- cq->next_buf_to_init, count, NULL);
+ rc = qdio_add_bufs_to_input_queue(CARD_DDEV(card), queue,
+ cq->next_buf_to_init, count);
if (rc) {
dev_warn(&card->gdev->dev,
"QDIO reported an error, rc=%i\n", rc);
@@ -3779,7 +3766,7 @@ static void qeth_qdio_output_handler(struct ccw_device *ccwdev,
/*
* Note: Function assumes that we have 4 outbound queues.
*/
-int qeth_get_priority_queue(struct qeth_card *card, struct sk_buff *skb)
+static int qeth_get_priority_queue(struct qeth_card *card, struct sk_buff *skb)
{
struct vlan_ethhdr *veth = vlan_eth_hdr(skb);
u8 tos;
@@ -3824,7 +3811,6 @@ int qeth_get_priority_queue(struct qeth_card *card, struct sk_buff *skb)
}
return card->qdio.default_out_queue;
}
-EXPORT_SYMBOL_GPL(qeth_get_priority_queue);
/**
* qeth_get_elements_for_frags() - find number of SBALEs for skb frags.
@@ -5585,29 +5571,9 @@ static void qeth_l3_rebuild_skb(struct qeth_card *card, struct sk_buff *skb,
#endif
static void qeth_receive_skb(struct qeth_card *card, struct sk_buff *skb,
- struct qeth_hdr *hdr, bool uses_frags)
+ bool uses_frags, bool is_cso)
{
struct napi_struct *napi = &card->napi;
- bool is_cso;
-
- switch (hdr->hdr.l2.id) {
-#if IS_ENABLED(CONFIG_QETH_L3)
- case QETH_HEADER_TYPE_LAYER3:
- qeth_l3_rebuild_skb(card, skb, hdr);
- is_cso = hdr->hdr.l3.ext_flags & QETH_HDR_EXT_CSUM_TRANSP_REQ;
- break;
-#endif
- case QETH_HEADER_TYPE_LAYER2:
- is_cso = hdr->hdr.l2.flags[1] & QETH_HDR_EXT_CSUM_TRANSP_REQ;
- break;
- default:
- /* never happens */
- if (uses_frags)
- napi_free_frags(napi);
- else
- kfree_skb(skb);
- return;
- }
if (is_cso && (card->dev->features & NETIF_F_RXCSUM)) {
skb->ip_summed = CHECKSUM_UNNECESSARY;
@@ -5664,6 +5630,7 @@ static int qeth_extract_skb(struct qeth_card *card,
struct qeth_hdr *hdr;
struct sk_buff *skb;
int skb_len = 0;
+ bool is_cso;
element = &buffer->element[*element_no];
@@ -5683,11 +5650,15 @@ static int qeth_extract_skb(struct qeth_card *card,
switch (hdr->hdr.l2.id) {
case QETH_HEADER_TYPE_LAYER2:
skb_len = hdr->hdr.l2.pkt_length;
+ is_cso = hdr->hdr.l2.flags[1] & QETH_HDR_EXT_CSUM_TRANSP_REQ;
+
linear_len = ETH_HLEN;
headroom = 0;
break;
case QETH_HEADER_TYPE_LAYER3:
skb_len = hdr->hdr.l3.length;
+ is_cso = hdr->hdr.l3.ext_flags & QETH_HDR_EXT_CSUM_TRANSP_REQ;
+
if (!IS_LAYER3(card)) {
QETH_CARD_STAT_INC(card, rx_dropped_notsupp);
goto walk_packet;
@@ -5814,7 +5785,12 @@ static int qeth_extract_skb(struct qeth_card *card,
*element_no = element - &buffer->element[0];
*__offset = offset;
- qeth_receive_skb(card, skb, hdr, uses_frags);
+#if IS_ENABLED(CONFIG_QETH_L3)
+ if (hdr->hdr.l2.id == QETH_HEADER_TYPE_LAYER3)
+ qeth_l3_rebuild_skb(card, skb, hdr);
+#endif
+
+ qeth_receive_skb(card, skb, uses_frags, is_cso);
return 0;
}
@@ -5850,10 +5826,10 @@ static unsigned int qeth_rx_poll(struct qeth_card *card, int budget)
/* Fetch completed RX buffers: */
if (!card->rx.b_count) {
card->rx.qdio_err = 0;
- card->rx.b_count = qdio_inspect_queue(CARD_DDEV(card),
- 0, true,
- &card->rx.b_index,
- &card->rx.qdio_err);
+ card->rx.b_count =
+ qdio_inspect_input_queue(CARD_DDEV(card), 0,
+ &card->rx.b_index,
+ &card->rx.qdio_err);
if (card->rx.b_count <= 0) {
card->rx.b_count = 0;
break;
@@ -5900,8 +5876,8 @@ static void qeth_cq_poll(struct qeth_card *card)
unsigned int start, error;
int completed;
- completed = qdio_inspect_queue(CARD_DDEV(card), 1, true, &start,
- &error);
+ completed = qdio_inspect_input_queue(CARD_DDEV(card), 1, &start,
+ &error);
if (completed <= 0)
return;
@@ -6038,8 +6014,8 @@ static int qeth_tx_poll(struct napi_struct *napi, int budget)
return 0;
}
- completed = qdio_inspect_queue(CARD_DDEV(card), queue_no, false,
- &start, &error);
+ completed = qdio_inspect_output_queue(CARD_DDEV(card), queue_no,
+ &start, &error);
if (completed <= 0) {
/* Ensure we see TX completion for pending work: */
if (napi_complete_done(napi, 0) &&
@@ -6447,6 +6423,12 @@ static int qeth_core_probe_device(struct ccwgroup_device *gdev)
qeth_determine_capabilities(card);
qeth_set_blkt_defaults(card);
+ card->qdio.in_q = qeth_alloc_qdio_queue();
+ if (!card->qdio.in_q) {
+ rc = -ENOMEM;
+ goto err_rx_queue;
+ }
+
card->qdio.no_out_queues = card->dev->num_tx_queues;
rc = qeth_update_from_chp_desc(card);
if (rc)
@@ -6473,6 +6455,8 @@ static int qeth_core_probe_device(struct ccwgroup_device *gdev)
err_setup_disc:
err_chp_desc:
+ qeth_free_qdio_queue(card->qdio.in_q);
+err_rx_queue:
free_netdev(card->dev);
err_card:
qeth_core_free_card(card);
@@ -6494,6 +6478,7 @@ static void qeth_core_remove_device(struct ccwgroup_device *gdev)
qeth_free_qdio_queues(card);
+ qeth_free_qdio_queue(card->qdio.in_q);
free_netdev(card->dev);
qeth_core_free_card(card);
put_device(&gdev->dev);
@@ -7089,6 +7074,18 @@ u16 qeth_iqd_select_queue(struct net_device *dev, struct sk_buff *skb,
}
EXPORT_SYMBOL_GPL(qeth_iqd_select_queue);
+u16 qeth_osa_select_queue(struct net_device *dev, struct sk_buff *skb,
+ struct net_device *sb_dev)
+{
+ struct qeth_card *card = dev->ml_priv;
+
+ if (qeth_uses_tx_prio_queueing(card))
+ return qeth_get_priority_queue(card, skb);
+
+ return netdev_pick_tx(dev, skb, sb_dev);
+}
+EXPORT_SYMBOL_GPL(qeth_osa_select_queue);
+
int qeth_open(struct net_device *dev)
{
struct qeth_card *card = dev->ml_priv;
diff --git a/drivers/s390/net/qeth_ethtool.c b/drivers/s390/net/qeth_ethtool.c
index 46d0fe0..b0b36b2 100644
--- a/drivers/s390/net/qeth_ethtool.c
+++ b/drivers/s390/net/qeth_ethtool.c
@@ -144,7 +144,9 @@ static int qeth_set_coalesce(struct net_device *dev,
}
static void qeth_get_ringparam(struct net_device *dev,
- struct ethtool_ringparam *param)
+ struct ethtool_ringparam *param,
+ struct kernel_ethtool_ringparam *kernel_param,
+ struct netlink_ext_ack *extack)
{
struct qeth_card *card = dev->ml_priv;
diff --git a/drivers/s390/net/qeth_l2_main.c b/drivers/s390/net/qeth_l2_main.c
index 0347fc1..303461d 100644
--- a/drivers/s390/net/qeth_l2_main.c
+++ b/drivers/s390/net/qeth_l2_main.c
@@ -519,19 +519,11 @@ static netdev_tx_t qeth_l2_hard_start_xmit(struct sk_buff *skb,
return NETDEV_TX_OK;
}
-static u16 qeth_l2_select_queue(struct net_device *dev, struct sk_buff *skb,
- struct net_device *sb_dev)
+static u16 qeth_l2_iqd_select_queue(struct net_device *dev, struct sk_buff *skb,
+ struct net_device *sb_dev)
{
- struct qeth_card *card = dev->ml_priv;
-
- if (IS_IQD(card))
- return qeth_iqd_select_queue(dev, skb,
- qeth_get_ether_cast_type(skb),
- sb_dev);
- if (qeth_uses_tx_prio_queueing(card))
- return qeth_get_priority_queue(card, skb);
-
- return netdev_pick_tx(dev, skb, sb_dev);
+ return qeth_iqd_select_queue(dev, skb, qeth_get_ether_cast_type(skb),
+ sb_dev);
}
static void qeth_l2_set_rx_mode(struct net_device *dev)
@@ -726,7 +718,8 @@ struct qeth_l2_br2dev_event_work {
unsigned char addr[ETH_ALEN];
};
-static const struct net_device_ops qeth_l2_netdev_ops;
+static const struct net_device_ops qeth_l2_iqd_netdev_ops;
+static const struct net_device_ops qeth_l2_osa_netdev_ops;
static bool qeth_l2_must_learn(struct net_device *netdev,
struct net_device *dstdev)
@@ -738,7 +731,8 @@ static bool qeth_l2_must_learn(struct net_device *netdev,
(priv->brport_features & BR_LEARNING_SYNC) &&
!(br_port_flag_is_set(netdev, BR_ISOLATED) &&
br_port_flag_is_set(dstdev, BR_ISOLATED)) &&
- netdev->netdev_ops == &qeth_l2_netdev_ops);
+ (netdev->netdev_ops == &qeth_l2_iqd_netdev_ops ||
+ netdev->netdev_ops == &qeth_l2_osa_netdev_ops));
}
/**
@@ -1051,13 +1045,34 @@ static int qeth_l2_bridge_setlink(struct net_device *dev, struct nlmsghdr *nlh,
return rc;
}
-static const struct net_device_ops qeth_l2_netdev_ops = {
+static const struct net_device_ops qeth_l2_iqd_netdev_ops = {
.ndo_open = qeth_open,
.ndo_stop = qeth_stop,
.ndo_get_stats64 = qeth_get_stats64,
.ndo_start_xmit = qeth_l2_hard_start_xmit,
.ndo_features_check = qeth_features_check,
- .ndo_select_queue = qeth_l2_select_queue,
+ .ndo_select_queue = qeth_l2_iqd_select_queue,
+ .ndo_validate_addr = qeth_l2_validate_addr,
+ .ndo_set_rx_mode = qeth_l2_set_rx_mode,
+ .ndo_eth_ioctl = qeth_do_ioctl,
+ .ndo_siocdevprivate = qeth_siocdevprivate,
+ .ndo_set_mac_address = qeth_l2_set_mac_address,
+ .ndo_vlan_rx_add_vid = qeth_l2_vlan_rx_add_vid,
+ .ndo_vlan_rx_kill_vid = qeth_l2_vlan_rx_kill_vid,
+ .ndo_tx_timeout = qeth_tx_timeout,
+ .ndo_fix_features = qeth_fix_features,
+ .ndo_set_features = qeth_set_features,
+ .ndo_bridge_getlink = qeth_l2_bridge_getlink,
+ .ndo_bridge_setlink = qeth_l2_bridge_setlink,
+};
+
+static const struct net_device_ops qeth_l2_osa_netdev_ops = {
+ .ndo_open = qeth_open,
+ .ndo_stop = qeth_stop,
+ .ndo_get_stats64 = qeth_get_stats64,
+ .ndo_start_xmit = qeth_l2_hard_start_xmit,
+ .ndo_features_check = qeth_features_check,
+ .ndo_select_queue = qeth_osa_select_queue,
.ndo_validate_addr = qeth_l2_validate_addr,
.ndo_set_rx_mode = qeth_l2_set_rx_mode,
.ndo_eth_ioctl = qeth_do_ioctl,
@@ -1068,14 +1083,13 @@ static const struct net_device_ops qeth_l2_netdev_ops = {
.ndo_tx_timeout = qeth_tx_timeout,
.ndo_fix_features = qeth_fix_features,
.ndo_set_features = qeth_set_features,
- .ndo_bridge_getlink = qeth_l2_bridge_getlink,
- .ndo_bridge_setlink = qeth_l2_bridge_setlink,
};
static int qeth_l2_setup_netdev(struct qeth_card *card)
{
+ card->dev->netdev_ops = IS_IQD(card) ? &qeth_l2_iqd_netdev_ops :
+ &qeth_l2_osa_netdev_ops;
card->dev->needed_headroom = sizeof(struct qeth_hdr);
- card->dev->netdev_ops = &qeth_l2_netdev_ops;
card->dev->priv_flags |= IFF_UNICAST_FLT;
if (IS_OSM(card)) {
diff --git a/drivers/s390/net/qeth_l3_main.c b/drivers/s390/net/qeth_l3_main.c
index 48a886f..9251ad2 100644
--- a/drivers/s390/net/qeth_l3_main.c
+++ b/drivers/s390/net/qeth_l3_main.c
@@ -1822,17 +1822,6 @@ static u16 qeth_l3_iqd_select_queue(struct net_device *dev, struct sk_buff *skb,
qeth_l3_get_cast_type(skb, proto), sb_dev);
}
-static u16 qeth_l3_osa_select_queue(struct net_device *dev, struct sk_buff *skb,
- struct net_device *sb_dev)
-{
- struct qeth_card *card = dev->ml_priv;
-
- if (qeth_uses_tx_prio_queueing(card))
- return qeth_get_priority_queue(card, skb);
-
- return netdev_pick_tx(dev, skb, sb_dev);
-}
-
static const struct net_device_ops qeth_l3_netdev_ops = {
.ndo_open = qeth_open,
.ndo_stop = qeth_stop,
@@ -1854,7 +1843,7 @@ static const struct net_device_ops qeth_l3_osa_netdev_ops = {
.ndo_get_stats64 = qeth_get_stats64,
.ndo_start_xmit = qeth_l3_hard_start_xmit,
.ndo_features_check = qeth_l3_osa_features_check,
- .ndo_select_queue = qeth_l3_osa_select_queue,
+ .ndo_select_queue = qeth_osa_select_queue,
.ndo_validate_addr = eth_validate_addr,
.ndo_set_rx_mode = qeth_l3_set_rx_mode,
.ndo_eth_ioctl = qeth_do_ioctl,
diff --git a/drivers/s390/scsi/zfcp_qdio.c b/drivers/s390/scsi/zfcp_qdio.c
index 6a27201..f54f506 100644
--- a/drivers/s390/scsi/zfcp_qdio.c
+++ b/drivers/s390/scsi/zfcp_qdio.c
@@ -79,7 +79,7 @@ static void zfcp_qdio_request_tasklet(struct tasklet_struct *tasklet)
unsigned int start, error;
int completed;
- completed = qdio_inspect_queue(cdev, 0, false, &start, &error);
+ completed = qdio_inspect_output_queue(cdev, 0, &start, &error);
if (completed > 0) {
if (error) {
zfcp_qdio_handler_error(qdio, "qdreqt1", error);
@@ -154,7 +154,7 @@ static void zfcp_qdio_int_resp(struct ccw_device *cdev, unsigned int qdio_err,
/*
* put SBALs back to response queue
*/
- if (do_QDIO(cdev, QDIO_FLAG_SYNC_INPUT, 0, idx, count, NULL))
+ if (qdio_add_bufs_to_input_queue(cdev, 0, idx, count))
zfcp_erp_adapter_reopen(qdio->adapter, 0, "qdires2");
}
@@ -169,7 +169,7 @@ static void zfcp_qdio_irq_tasklet(struct tasklet_struct *tasklet)
tasklet_schedule(&qdio->request_tasklet);
/* Check the Response Queue: */
- completed = qdio_inspect_queue(cdev, 0, true, &start, &error);
+ completed = qdio_inspect_input_queue(cdev, 0, &start, &error);
if (completed < 0)
return;
if (completed > 0)
@@ -326,8 +326,9 @@ int zfcp_qdio_send(struct zfcp_qdio *qdio, struct zfcp_qdio_req *q_req)
atomic_sub(sbal_number, &qdio->req_q_free);
- retval = do_QDIO(qdio->adapter->ccw_device, QDIO_FLAG_SYNC_OUTPUT, 0,
- q_req->sbal_first, sbal_number, NULL);
+ retval = qdio_add_bufs_to_output_queue(qdio->adapter->ccw_device, 0,
+ q_req->sbal_first, sbal_number,
+ NULL);
if (unlikely(retval)) {
/* Failed to submit the IO, roll back our modifications. */
@@ -395,7 +396,10 @@ void zfcp_qdio_close(struct zfcp_qdio *qdio)
if (!(atomic_read(&adapter->status) & ZFCP_STATUS_ADAPTER_QDIOUP))
return;
- /* clear QDIOUP flag, thus do_QDIO is not called during qdio_shutdown */
+ /*
+ * Clear QDIOUP flag, thus qdio_add_bufs_to_output_queue() is not called
+ * during qdio_shutdown().
+ */
spin_lock_irq(&qdio->req_q_lock);
atomic_andnot(ZFCP_STATUS_ADAPTER_QDIOUP, &adapter->status);
spin_unlock_irq(&qdio->req_q_lock);
@@ -498,8 +502,7 @@ int zfcp_qdio_open(struct zfcp_qdio *qdio)
sbale->addr = 0;
}
- if (do_QDIO(cdev, QDIO_FLAG_SYNC_INPUT, 0, 0, QDIO_MAX_BUFFERS_PER_Q,
- NULL))
+ if (qdio_add_bufs_to_input_queue(cdev, 0, 0, QDIO_MAX_BUFFERS_PER_Q))
goto failed_qdio;
/* set index of first available SBALS / number of available SBALS */
diff --git a/drivers/scsi/libiscsi.c b/drivers/scsi/libiscsi.c
index 284b939..059dae8 100644
--- a/drivers/scsi/libiscsi.c
+++ b/drivers/scsi/libiscsi.c
@@ -3100,6 +3100,8 @@ void iscsi_conn_teardown(struct iscsi_cls_conn *cls_conn)
{
struct iscsi_conn *conn = cls_conn->dd_data;
struct iscsi_session *session = conn->session;
+ char *tmp_persistent_address = conn->persistent_address;
+ char *tmp_local_ipaddr = conn->local_ipaddr;
del_timer_sync(&conn->transport_timer);
@@ -3121,8 +3123,6 @@ void iscsi_conn_teardown(struct iscsi_cls_conn *cls_conn)
spin_lock_bh(&session->frwd_lock);
free_pages((unsigned long) conn->data,
get_order(ISCSI_DEF_MAX_RECV_SEG_LEN));
- kfree(conn->persistent_address);
- kfree(conn->local_ipaddr);
/* regular RX path uses back_lock */
spin_lock_bh(&session->back_lock);
kfifo_in(&session->cmdpool.queue, (void*)&conn->login_task,
@@ -3134,6 +3134,8 @@ void iscsi_conn_teardown(struct iscsi_cls_conn *cls_conn)
mutex_unlock(&session->eh_mutex);
iscsi_destroy_conn(cls_conn);
+ kfree(tmp_persistent_address);
+ kfree(tmp_local_ipaddr);
}
EXPORT_SYMBOL_GPL(iscsi_conn_teardown);
diff --git a/drivers/scsi/lpfc/lpfc_debugfs.c b/drivers/scsi/lpfc/lpfc_debugfs.c
index bd6d459..08b2e85 100644
--- a/drivers/scsi/lpfc/lpfc_debugfs.c
+++ b/drivers/scsi/lpfc/lpfc_debugfs.c
@@ -2954,8 +2954,8 @@ lpfc_debugfs_nvmeio_trc_write(struct file *file, const char __user *buf,
char mybuf[64];
char *pbuf;
- if (nbytes > 64)
- nbytes = 64;
+ if (nbytes > 63)
+ nbytes = 63;
memset(mybuf, 0, sizeof(mybuf));
diff --git a/drivers/scsi/vmw_pvscsi.c b/drivers/scsi/vmw_pvscsi.c
index c2ba652..1f037b8 100644
--- a/drivers/scsi/vmw_pvscsi.c
+++ b/drivers/scsi/vmw_pvscsi.c
@@ -586,9 +586,12 @@ static void pvscsi_complete_request(struct pvscsi_adapter *adapter,
* Commands like INQUIRY may transfer less data than
* requested by the initiator via bufflen. Set residual
* count to make upper layer aware of the actual amount
- * of data returned.
+ * of data returned. There are cases when controller
+ * returns zero dataLen with non zero data - do not set
+ * residual count in that case.
*/
- scsi_set_resid(cmd, scsi_bufflen(cmd) - e->dataLen);
+ if (e->dataLen && (e->dataLen < scsi_bufflen(cmd)))
+ scsi_set_resid(cmd, scsi_bufflen(cmd) - e->dataLen);
cmd->result = (DID_OK << 16);
break;
diff --git a/drivers/soc/Kconfig b/drivers/soc/Kconfig
index e8a30c4..a856267 100644
--- a/drivers/soc/Kconfig
+++ b/drivers/soc/Kconfig
@@ -3,6 +3,7 @@
source "drivers/soc/actions/Kconfig"
source "drivers/soc/amlogic/Kconfig"
+source "drivers/soc/apple/Kconfig"
source "drivers/soc/aspeed/Kconfig"
source "drivers/soc/atmel/Kconfig"
source "drivers/soc/bcm/Kconfig"
diff --git a/drivers/soc/Makefile b/drivers/soc/Makefile
index a05e9fb..adb30c2 100644
--- a/drivers/soc/Makefile
+++ b/drivers/soc/Makefile
@@ -4,6 +4,7 @@
#
obj-$(CONFIG_ARCH_ACTIONS) += actions/
+obj-$(CONFIG_ARCH_APPLE) += apple/
obj-y += aspeed/
obj-$(CONFIG_ARCH_AT91) += atmel/
obj-y += bcm/
diff --git a/drivers/soc/apple/Kconfig b/drivers/soc/apple/Kconfig
new file mode 100644
index 0000000..9b8de31
--- /dev/null
+++ b/drivers/soc/apple/Kconfig
@@ -0,0 +1,22 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+if ARCH_APPLE || COMPILE_TEST
+
+menu "Apple SoC drivers"
+
+config APPLE_PMGR_PWRSTATE
+ bool "Apple SoC PMGR power state control"
+ depends on PM
+ select REGMAP
+ select MFD_SYSCON
+ select PM_GENERIC_DOMAINS
+ select RESET_CONTROLLER
+ default ARCH_APPLE
+ help
+ The PMGR block in Apple SoCs provides high-level power state
+ controls for SoC devices. This driver manages them through the
+ generic power domain framework, and also provides reset support.
+
+endmenu
+
+endif
diff --git a/drivers/soc/apple/Makefile b/drivers/soc/apple/Makefile
new file mode 100644
index 0000000..c114e84
--- /dev/null
+++ b/drivers/soc/apple/Makefile
@@ -0,0 +1,2 @@
+# SPDX-License-Identifier: GPL-2.0-only
+obj-$(CONFIG_APPLE_PMGR_PWRSTATE) += apple-pmgr-pwrstate.o
diff --git a/drivers/soc/apple/apple-pmgr-pwrstate.c b/drivers/soc/apple/apple-pmgr-pwrstate.c
new file mode 100644
index 0000000..e112228
--- /dev/null
+++ b/drivers/soc/apple/apple-pmgr-pwrstate.c
@@ -0,0 +1,324 @@
+// SPDX-License-Identifier: GPL-2.0-only OR MIT
+/*
+ * Apple SoC PMGR device power state driver
+ *
+ * Copyright The Asahi Linux Contributors
+ */
+
+#include <linux/bitops.h>
+#include <linux/bitfield.h>
+#include <linux/err.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/platform_device.h>
+#include <linux/pm_domain.h>
+#include <linux/regmap.h>
+#include <linux/mfd/syscon.h>
+#include <linux/reset-controller.h>
+#include <linux/module.h>
+
+#define APPLE_PMGR_RESET BIT(31)
+#define APPLE_PMGR_AUTO_ENABLE BIT(28)
+#define APPLE_PMGR_PS_AUTO GENMASK(27, 24)
+#define APPLE_PMGR_PS_MIN GENMASK(19, 16)
+#define APPLE_PMGR_PARENT_OFF BIT(11)
+#define APPLE_PMGR_DEV_DISABLE BIT(10)
+#define APPLE_PMGR_WAS_CLKGATED BIT(9)
+#define APPLE_PMGR_WAS_PWRGATED BIT(8)
+#define APPLE_PMGR_PS_ACTUAL GENMASK(7, 4)
+#define APPLE_PMGR_PS_TARGET GENMASK(3, 0)
+
+#define APPLE_PMGR_FLAGS (APPLE_PMGR_WAS_CLKGATED | APPLE_PMGR_WAS_PWRGATED)
+
+#define APPLE_PMGR_PS_ACTIVE 0xf
+#define APPLE_PMGR_PS_CLKGATE 0x4
+#define APPLE_PMGR_PS_PWRGATE 0x0
+
+#define APPLE_PMGR_PS_SET_TIMEOUT 100
+#define APPLE_PMGR_RESET_TIME 1
+
+struct apple_pmgr_ps {
+ struct device *dev;
+ struct generic_pm_domain genpd;
+ struct reset_controller_dev rcdev;
+ struct regmap *regmap;
+ u32 offset;
+ u32 min_state;
+};
+
+#define genpd_to_apple_pmgr_ps(_genpd) container_of(_genpd, struct apple_pmgr_ps, genpd)
+#define rcdev_to_apple_pmgr_ps(_rcdev) container_of(_rcdev, struct apple_pmgr_ps, rcdev)
+
+static int apple_pmgr_ps_set(struct generic_pm_domain *genpd, u32 pstate, bool auto_enable)
+{
+ int ret;
+ struct apple_pmgr_ps *ps = genpd_to_apple_pmgr_ps(genpd);
+ u32 reg;
+
+ ret = regmap_read(ps->regmap, ps->offset, ®);
+ if (ret < 0)
+ return ret;
+
+ /* Resets are synchronous, and only work if the device is powered and clocked. */
+ if (reg & APPLE_PMGR_RESET && pstate != APPLE_PMGR_PS_ACTIVE)
+ dev_err(ps->dev, "PS %s: powering off with RESET active\n",
+ genpd->name);
+
+ reg &= ~(APPLE_PMGR_AUTO_ENABLE | APPLE_PMGR_FLAGS | APPLE_PMGR_PS_TARGET);
+ reg |= FIELD_PREP(APPLE_PMGR_PS_TARGET, pstate);
+
+ dev_dbg(ps->dev, "PS %s: pwrstate = 0x%x: 0x%x\n", genpd->name, pstate, reg);
+
+ regmap_write(ps->regmap, ps->offset, reg);
+
+ ret = regmap_read_poll_timeout_atomic(
+ ps->regmap, ps->offset, reg,
+ (FIELD_GET(APPLE_PMGR_PS_ACTUAL, reg) == pstate), 1,
+ APPLE_PMGR_PS_SET_TIMEOUT);
+ if (ret < 0)
+ dev_err(ps->dev, "PS %s: Failed to reach power state 0x%x (now: 0x%x)\n",
+ genpd->name, pstate, reg);
+
+ if (auto_enable) {
+ /* Not all devices implement this; this is a no-op where not implemented. */
+ reg &= ~APPLE_PMGR_FLAGS;
+ reg |= APPLE_PMGR_AUTO_ENABLE;
+ regmap_write(ps->regmap, ps->offset, reg);
+ }
+
+ return ret;
+}
+
+static bool apple_pmgr_ps_is_active(struct apple_pmgr_ps *ps)
+{
+ u32 reg = 0;
+
+ regmap_read(ps->regmap, ps->offset, ®);
+ /*
+ * We consider domains as active if they are actually on, or if they have auto-PM
+ * enabled and the intended target is on.
+ */
+ return (FIELD_GET(APPLE_PMGR_PS_ACTUAL, reg) == APPLE_PMGR_PS_ACTIVE ||
+ (FIELD_GET(APPLE_PMGR_PS_TARGET, reg) == APPLE_PMGR_PS_ACTIVE &&
+ reg & APPLE_PMGR_AUTO_ENABLE));
+}
+
+static int apple_pmgr_ps_power_on(struct generic_pm_domain *genpd)
+{
+ return apple_pmgr_ps_set(genpd, APPLE_PMGR_PS_ACTIVE, true);
+}
+
+static int apple_pmgr_ps_power_off(struct generic_pm_domain *genpd)
+{
+ return apple_pmgr_ps_set(genpd, APPLE_PMGR_PS_PWRGATE, false);
+}
+
+static int apple_pmgr_reset_assert(struct reset_controller_dev *rcdev, unsigned long id)
+{
+ struct apple_pmgr_ps *ps = rcdev_to_apple_pmgr_ps(rcdev);
+
+ mutex_lock(&ps->genpd.mlock);
+
+ if (ps->genpd.status == GENPD_STATE_OFF)
+ dev_err(ps->dev, "PS 0x%x: asserting RESET while powered down\n", ps->offset);
+
+ dev_dbg(ps->dev, "PS 0x%x: assert reset\n", ps->offset);
+ /* Quiesce device before asserting reset */
+ regmap_update_bits(ps->regmap, ps->offset, APPLE_PMGR_FLAGS | APPLE_PMGR_DEV_DISABLE,
+ APPLE_PMGR_DEV_DISABLE);
+ regmap_update_bits(ps->regmap, ps->offset, APPLE_PMGR_FLAGS | APPLE_PMGR_RESET,
+ APPLE_PMGR_RESET);
+
+ mutex_unlock(&ps->genpd.mlock);
+
+ return 0;
+}
+
+static int apple_pmgr_reset_deassert(struct reset_controller_dev *rcdev, unsigned long id)
+{
+ struct apple_pmgr_ps *ps = rcdev_to_apple_pmgr_ps(rcdev);
+
+ mutex_lock(&ps->genpd.mlock);
+
+ dev_dbg(ps->dev, "PS 0x%x: deassert reset\n", ps->offset);
+ regmap_update_bits(ps->regmap, ps->offset, APPLE_PMGR_FLAGS | APPLE_PMGR_RESET, 0);
+ regmap_update_bits(ps->regmap, ps->offset, APPLE_PMGR_FLAGS | APPLE_PMGR_DEV_DISABLE, 0);
+
+ if (ps->genpd.status == GENPD_STATE_OFF)
+ dev_err(ps->dev, "PS 0x%x: RESET was deasserted while powered down\n", ps->offset);
+
+ mutex_unlock(&ps->genpd.mlock);
+
+ return 0;
+}
+
+static int apple_pmgr_reset_reset(struct reset_controller_dev *rcdev, unsigned long id)
+{
+ int ret;
+
+ ret = apple_pmgr_reset_assert(rcdev, id);
+ if (ret)
+ return ret;
+
+ usleep_range(APPLE_PMGR_RESET_TIME, 2 * APPLE_PMGR_RESET_TIME);
+
+ return apple_pmgr_reset_deassert(rcdev, id);
+}
+
+static int apple_pmgr_reset_status(struct reset_controller_dev *rcdev, unsigned long id)
+{
+ struct apple_pmgr_ps *ps = rcdev_to_apple_pmgr_ps(rcdev);
+ u32 reg = 0;
+
+ regmap_read(ps->regmap, ps->offset, ®);
+
+ return !!(reg & APPLE_PMGR_RESET);
+}
+
+const struct reset_control_ops apple_pmgr_reset_ops = {
+ .assert = apple_pmgr_reset_assert,
+ .deassert = apple_pmgr_reset_deassert,
+ .reset = apple_pmgr_reset_reset,
+ .status = apple_pmgr_reset_status,
+};
+
+static int apple_pmgr_reset_xlate(struct reset_controller_dev *rcdev,
+ const struct of_phandle_args *reset_spec)
+{
+ return 0;
+}
+
+static int apple_pmgr_ps_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct device_node *node = dev->of_node;
+ struct apple_pmgr_ps *ps;
+ struct regmap *regmap;
+ struct of_phandle_iterator it;
+ int ret;
+ const char *name;
+ bool active;
+
+ regmap = syscon_node_to_regmap(node->parent);
+ if (IS_ERR(regmap))
+ return PTR_ERR(regmap);
+
+ ps = devm_kzalloc(dev, sizeof(*ps), GFP_KERNEL);
+ if (!ps)
+ return -ENOMEM;
+
+ ps->dev = dev;
+ ps->regmap = regmap;
+
+ ret = of_property_read_string(node, "label", &name);
+ if (ret < 0) {
+ dev_err(dev, "missing label property\n");
+ return ret;
+ }
+
+ ret = of_property_read_u32(node, "reg", &ps->offset);
+ if (ret < 0) {
+ dev_err(dev, "missing reg property\n");
+ return ret;
+ }
+
+ ps->genpd.name = name;
+ ps->genpd.power_on = apple_pmgr_ps_power_on;
+ ps->genpd.power_off = apple_pmgr_ps_power_off;
+
+ ret = of_property_read_u32(node, "apple,min-state", &ps->min_state);
+ if (ret == 0 && ps->min_state <= APPLE_PMGR_PS_ACTIVE)
+ regmap_update_bits(regmap, ps->offset, APPLE_PMGR_FLAGS | APPLE_PMGR_PS_MIN,
+ FIELD_PREP(APPLE_PMGR_PS_MIN, ps->min_state));
+
+ active = apple_pmgr_ps_is_active(ps);
+ if (of_property_read_bool(node, "apple,always-on")) {
+ ps->genpd.flags |= GENPD_FLAG_ALWAYS_ON;
+ if (!active) {
+ dev_warn(dev, "always-on domain %s is not on at boot\n", name);
+ /* Turn it on so pm_genpd_init does not fail */
+ active = apple_pmgr_ps_power_on(&ps->genpd) == 0;
+ }
+ }
+
+ /* Turn on auto-PM if the domain is already on */
+ if (active)
+ regmap_update_bits(regmap, ps->offset, APPLE_PMGR_FLAGS | APPLE_PMGR_AUTO_ENABLE,
+ APPLE_PMGR_AUTO_ENABLE);
+
+ ret = pm_genpd_init(&ps->genpd, NULL, !active);
+ if (ret < 0) {
+ dev_err(dev, "pm_genpd_init failed\n");
+ return ret;
+ }
+
+ ret = of_genpd_add_provider_simple(node, &ps->genpd);
+ if (ret < 0) {
+ dev_err(dev, "of_genpd_add_provider_simple failed\n");
+ return ret;
+ }
+
+ of_for_each_phandle(&it, ret, node, "power-domains", "#power-domain-cells", -1) {
+ struct of_phandle_args parent, child;
+
+ parent.np = it.node;
+ parent.args_count = of_phandle_iterator_args(&it, parent.args, MAX_PHANDLE_ARGS);
+ child.np = node;
+ child.args_count = 0;
+ ret = of_genpd_add_subdomain(&parent, &child);
+
+ if (ret == -EPROBE_DEFER) {
+ of_node_put(parent.np);
+ goto err_remove;
+ } else if (ret < 0) {
+ dev_err(dev, "failed to add to parent domain: %d (%s -> %s)\n",
+ ret, it.node->name, node->name);
+ of_node_put(parent.np);
+ goto err_remove;
+ }
+ }
+
+ /*
+ * Do not participate in regular PM; parent power domains are handled via the
+ * genpd hierarchy.
+ */
+ pm_genpd_remove_device(dev);
+
+ ps->rcdev.owner = THIS_MODULE;
+ ps->rcdev.nr_resets = 1;
+ ps->rcdev.ops = &apple_pmgr_reset_ops;
+ ps->rcdev.of_node = dev->of_node;
+ ps->rcdev.of_reset_n_cells = 0;
+ ps->rcdev.of_xlate = apple_pmgr_reset_xlate;
+
+ ret = devm_reset_controller_register(dev, &ps->rcdev);
+ if (ret < 0)
+ goto err_remove;
+
+ return 0;
+err_remove:
+ of_genpd_del_provider(node);
+ pm_genpd_remove(&ps->genpd);
+ return ret;
+}
+
+static const struct of_device_id apple_pmgr_ps_of_match[] = {
+ { .compatible = "apple,pmgr-pwrstate" },
+ {}
+};
+
+MODULE_DEVICE_TABLE(of, apple_pmgr_ps_of_match);
+
+static struct platform_driver apple_pmgr_ps_driver = {
+ .probe = apple_pmgr_ps_probe,
+ .driver = {
+ .name = "apple-pmgr-pwrstate",
+ .of_match_table = apple_pmgr_ps_of_match,
+ },
+};
+
+MODULE_AUTHOR("Hector Martin <marcan@marcan.st>");
+MODULE_DESCRIPTION("PMGR power state driver for Apple SoCs");
+MODULE_LICENSE("GPL v2");
+
+module_platform_driver(apple_pmgr_ps_driver);
diff --git a/drivers/soc/bcm/brcmstb/pm/pm-mips.c b/drivers/soc/bcm/brcmstb/pm/pm-mips.c
index cdc3e38..4dfb5a8 100644
--- a/drivers/soc/bcm/brcmstb/pm/pm-mips.c
+++ b/drivers/soc/bcm/brcmstb/pm/pm-mips.c
@@ -405,11 +405,14 @@ static int brcmstb_pm_init(void)
i = ctrl.num_memc;
if (i >= MAX_NUM_MEMC) {
pr_warn("Too many MEMCs (max %d)\n", MAX_NUM_MEMC);
+ of_node_put(dn);
break;
}
base = brcmstb_ioremap_node(dn, 0);
- if (IS_ERR(base))
+ if (IS_ERR(base)) {
+ of_node_put(dn);
goto ddr_err;
+ }
ctrl.memcs[i].ddr_phy_base = base;
ctrl.num_memc++;
diff --git a/drivers/soc/imx/gpcv2.c b/drivers/soc/imx/gpcv2.c
index b8d52d8..3e59d47 100644
--- a/drivers/soc/imx/gpcv2.c
+++ b/drivers/soc/imx/gpcv2.c
@@ -377,7 +377,7 @@ static int imx_pgc_power_down(struct generic_pm_domain *genpd)
}
}
- pm_runtime_put(domain->dev);
+ pm_runtime_put_sync_suspend(domain->dev);
return 0;
@@ -734,6 +734,7 @@ static const struct imx_pgc_domain imx8mm_pgc_domains[] = {
.map = IMX8MM_VPUH1_A53_DOMAIN,
},
.pgc = BIT(IMX8MM_PGC_VPUH1),
+ .keep_clocks = true,
},
[IMX8MM_POWER_DOMAIN_DISPMIX] = {
@@ -840,6 +841,32 @@ static const struct imx_pgc_domain imx8mn_pgc_domains[] = {
.hskack = IMX8MN_GPUMIX_HSK_PWRDNACKN,
},
.pgc = BIT(IMX8MN_PGC_GPUMIX),
+ .keep_clocks = true,
+ },
+
+ [IMX8MN_POWER_DOMAIN_DISPMIX] = {
+ .genpd = {
+ .name = "dispmix",
+ },
+ .bits = {
+ .pxx = IMX8MN_DISPMIX_SW_Pxx_REQ,
+ .map = IMX8MN_DISPMIX_A53_DOMAIN,
+ .hskreq = IMX8MN_DISPMIX_HSK_PWRDNREQN,
+ .hskack = IMX8MN_DISPMIX_HSK_PWRDNACKN,
+ },
+ .pgc = BIT(IMX8MN_PGC_DISPMIX),
+ .keep_clocks = true,
+ },
+
+ [IMX8MN_POWER_DOMAIN_MIPI] = {
+ .genpd = {
+ .name = "mipi",
+ },
+ .bits = {
+ .pxx = IMX8MN_MIPI_SW_Pxx_REQ,
+ .map = IMX8MN_MIPI_A53_DOMAIN,
+ },
+ .pgc = BIT(IMX8MN_PGC_MIPI),
},
};
diff --git a/drivers/soc/imx/imx8m-blk-ctrl.c b/drivers/soc/imx/imx8m-blk-ctrl.c
index c2f076b..511e74f0 100644
--- a/drivers/soc/imx/imx8m-blk-ctrl.c
+++ b/drivers/soc/imx/imx8m-blk-ctrl.c
@@ -14,6 +14,7 @@
#include <linux/clk.h>
#include <dt-bindings/power/imx8mm-power.h>
+#include <dt-bindings/power/imx8mn-power.h>
#define BLK_SFT_RSTN 0x0
#define BLK_CLK_EN 0x4
@@ -517,6 +518,77 @@ static const struct imx8m_blk_ctrl_data imx8mm_disp_blk_ctl_dev_data = {
.num_domains = ARRAY_SIZE(imx8mm_disp_blk_ctl_domain_data),
};
+
+static int imx8mn_disp_power_notifier(struct notifier_block *nb,
+ unsigned long action, void *data)
+{
+ struct imx8m_blk_ctrl *bc = container_of(nb, struct imx8m_blk_ctrl,
+ power_nb);
+
+ if (action != GENPD_NOTIFY_ON && action != GENPD_NOTIFY_PRE_OFF)
+ return NOTIFY_OK;
+
+ /* Enable bus clock and deassert bus reset */
+ regmap_set_bits(bc->regmap, BLK_CLK_EN, BIT(8));
+ regmap_set_bits(bc->regmap, BLK_SFT_RSTN, BIT(8));
+
+ /*
+ * On power up we have no software backchannel to the GPC to
+ * wait for the ADB handshake to happen, so we just delay for a
+ * bit. On power down the GPC driver waits for the handshake.
+ */
+ if (action == GENPD_NOTIFY_ON)
+ udelay(5);
+
+
+ return NOTIFY_OK;
+}
+
+static const struct imx8m_blk_ctrl_domain_data imx8mn_disp_blk_ctl_domain_data[] = {
+ [IMX8MN_DISPBLK_PD_MIPI_DSI] = {
+ .name = "dispblk-mipi-dsi",
+ .clk_names = (const char *[]){ "dsi-pclk", "dsi-ref", },
+ .num_clks = 2,
+ .gpc_name = "mipi-dsi",
+ .rst_mask = BIT(0) | BIT(1),
+ .clk_mask = BIT(0) | BIT(1),
+ .mipi_phy_rst_mask = BIT(17),
+ },
+ [IMX8MN_DISPBLK_PD_MIPI_CSI] = {
+ .name = "dispblk-mipi-csi",
+ .clk_names = (const char *[]){ "csi-aclk", "csi-pclk" },
+ .num_clks = 2,
+ .gpc_name = "mipi-csi",
+ .rst_mask = BIT(2) | BIT(3),
+ .clk_mask = BIT(2) | BIT(3),
+ .mipi_phy_rst_mask = BIT(16),
+ },
+ [IMX8MN_DISPBLK_PD_LCDIF] = {
+ .name = "dispblk-lcdif",
+ .clk_names = (const char *[]){ "lcdif-axi", "lcdif-apb", "lcdif-pix", },
+ .num_clks = 3,
+ .gpc_name = "lcdif",
+ .rst_mask = BIT(4) | BIT(5),
+ .clk_mask = BIT(4) | BIT(5),
+ },
+ [IMX8MN_DISPBLK_PD_ISI] = {
+ .name = "dispblk-isi",
+ .clk_names = (const char *[]){ "disp_axi", "disp_apb", "disp_axi_root",
+ "disp_apb_root"},
+ .num_clks = 4,
+ .gpc_name = "isi",
+ .rst_mask = BIT(6) | BIT(7),
+ .clk_mask = BIT(6) | BIT(7),
+ },
+};
+
+static const struct imx8m_blk_ctrl_data imx8mn_disp_blk_ctl_dev_data = {
+ .max_reg = 0x84,
+ .power_notifier_fn = imx8mn_disp_power_notifier,
+ .domains = imx8mn_disp_blk_ctl_domain_data,
+ .num_domains = ARRAY_SIZE(imx8mn_disp_blk_ctl_domain_data),
+};
+
static const struct of_device_id imx8m_blk_ctrl_of_match[] = {
{
.compatible = "fsl,imx8mm-vpu-blk-ctrl",
@@ -524,7 +596,10 @@ static const struct of_device_id imx8m_blk_ctrl_of_match[] = {
}, {
.compatible = "fsl,imx8mm-disp-blk-ctrl",
.data = &imx8mm_disp_blk_ctl_dev_data
- } ,{
+ }, {
+ .compatible = "fsl,imx8mn-disp-blk-ctrl",
+ .data = &imx8mn_disp_blk_ctl_dev_data
+ }, {
/* Sentinel */
}
};
diff --git a/drivers/soc/qcom/cpr.c b/drivers/soc/qcom/cpr.c
index 1d818a8..e9b854e 100644
--- a/drivers/soc/qcom/cpr.c
+++ b/drivers/soc/qcom/cpr.c
@@ -1010,7 +1010,7 @@ static int cpr_interpolate(const struct corner *corner, int step_volt,
return corner->uV;
temp = f_diff * (uV_high - uV_low);
- do_div(temp, f_high - f_low);
+ temp = div64_ul(temp, f_high - f_low);
/*
* max_volt_scale has units of uV/MHz while freq values
diff --git a/drivers/soc/qcom/llcc-qcom.c b/drivers/soc/qcom/llcc-qcom.c
index 6bf2f1d..ec52f29 100644
--- a/drivers/soc/qcom/llcc-qcom.c
+++ b/drivers/soc/qcom/llcc-qcom.c
@@ -195,6 +195,28 @@ static const struct llcc_slice_config sm8250_data[] = {
{ LLCC_WRCACHE, 31, 256, 1, 1, 0xfff, 0x0, 0, 0, 0, 0, 1, 0 },
};
+static const struct llcc_slice_config sm8350_data[] = {
+ { LLCC_CPUSS, 1, 3072, 1, 1, 0xfff, 0x0, 0, 0, 0, 0, 1, 1 },
+ { LLCC_VIDSC0, 2, 512, 3, 1, 0xfff, 0x0, 0, 0, 0, 0, 1, 0 },
+ { LLCC_AUDIO, 6, 1024, 1, 1, 0xfff, 0x0, 0, 0, 0, 0, 0, 0 },
+ { LLCC_MDMHPGRW, 7, 1024, 3, 0, 0xfff, 0x0, 0, 0, 0, 0, 1, 0 },
+ { LLCC_MODHW, 9, 1024, 1, 1, 0xfff, 0x0, 0, 0, 0, 0, 1, 0 },
+ { LLCC_CMPT, 10, 3072, 1, 1, 0xfff, 0x0, 0, 0, 0, 0, 1, 0 },
+ { LLCC_GPUHTW, 11, 1024, 1, 1, 0xfff, 0x0, 0, 0, 0, 0, 1, 0 },
+ { LLCC_GPU, 12, 1024, 1, 0, 0xfff, 0x0, 0, 0, 0, 1, 1, 0 },
+ { LLCC_MMUHWT, 13, 1024, 1, 1, 0xfff, 0x0, 0, 0, 0, 0, 0, 1 },
+ { LLCC_DISP, 16, 3072, 2, 1, 0xfff, 0x0, 0, 0, 0, 0, 1, 0 },
+ { LLCC_MDMPNG, 21, 1024, 0, 1, 0xf, 0x0, 0, 0, 0, 0, 1, 0 },
+ { LLCC_AUDHW, 22, 1024, 1, 1, 0xfff, 0x0, 0, 0, 0, 0, 1, 0 },
+ { LLCC_CVP, 28, 512, 3, 1, 0xfff, 0x0, 0, 0, 0, 0, 1, 0 },
+ { LLCC_MODPE, 29, 256, 1, 1, 0xf, 0x0, 0, 0, 0, 0, 1, 0 },
+ { LLCC_APTCM, 30, 1024, 3, 1, 0x0, 0x1, 1, 0, 0, 0, 1, 0 },
+ { LLCC_WRCACHE, 31, 512, 1, 1, 0xfff, 0x0, 0, 0, 0, 0, 0, 1 },
+ { LLCC_CVPFW, 17, 512, 1, 0, 0xfff, 0x0, 0, 0, 0, 0, 1, 0 },
+ { LLCC_CPUSS1, 3, 1024, 1, 1, 0xfff, 0x0, 0, 0, 0, 0, 1, 0 },
+ { LLCC_CPUHWT, 5, 512, 1, 1, 0xfff, 0x0, 0, 0, 0, 0, 0, 1 },
+};
+
static const struct qcom_llcc_config sc7180_cfg = {
.sct_data = sc7180_data,
.size = ARRAY_SIZE(sc7180_data),
@@ -228,6 +250,11 @@ static const struct qcom_llcc_config sm8250_cfg = {
.size = ARRAY_SIZE(sm8250_data),
};
+static const struct qcom_llcc_config sm8350_cfg = {
+ .sct_data = sm8350_data,
+ .size = ARRAY_SIZE(sm8350_data),
+};
+
static struct llcc_drv_data *drv_data = (void *) -EPROBE_DEFER;
/**
@@ -644,6 +671,7 @@ static const struct of_device_id qcom_llcc_of_match[] = {
{ .compatible = "qcom,sm6350-llcc", .data = &sm6350_cfg },
{ .compatible = "qcom,sm8150-llcc", .data = &sm8150_cfg },
{ .compatible = "qcom,sm8250-llcc", .data = &sm8250_cfg },
+ { .compatible = "qcom,sm8350-llcc", .data = &sm8350_cfg },
{ }
};
diff --git a/drivers/soc/qcom/qcom_aoss.c b/drivers/soc/qcom/qcom_aoss.c
index 34acf58..cbe5e39 100644
--- a/drivers/soc/qcom/qcom_aoss.c
+++ b/drivers/soc/qcom/qcom_aoss.c
@@ -352,7 +352,7 @@ static int qmp_cdev_set_cur_state(struct thermal_cooling_device *cdev,
return ret;
}
-static struct thermal_cooling_device_ops qmp_cooling_device_ops = {
+static const struct thermal_cooling_device_ops qmp_cooling_device_ops = {
.get_max_state = qmp_cdev_get_max_state,
.get_cur_state = qmp_cdev_get_cur_state,
.set_cur_state = qmp_cdev_set_cur_state,
diff --git a/drivers/soc/qcom/qcom_stats.c b/drivers/soc/qcom/qcom_stats.c
index 131d24c..d6bfd1b 100644
--- a/drivers/soc/qcom/qcom_stats.c
+++ b/drivers/soc/qcom/qcom_stats.c
@@ -237,6 +237,15 @@ static const struct stats_config rpm_data = {
.subsystem_stats_in_smem = false,
};
+/* Older RPM firmwares have the stats at a fixed offset instead */
+static const struct stats_config rpm_data_dba0 = {
+ .stats_offset = 0xdba0,
+ .num_records = 2,
+ .appended_stats_avail = true,
+ .dynamic_offset = false,
+ .subsystem_stats_in_smem = false,
+};
+
static const struct stats_config rpmh_data = {
.stats_offset = 0x48,
.num_records = 3,
@@ -246,6 +255,10 @@ static const struct stats_config rpmh_data = {
};
static const struct of_device_id qcom_stats_table[] = {
+ { .compatible = "qcom,apq8084-rpm-stats", .data = &rpm_data_dba0 },
+ { .compatible = "qcom,msm8226-rpm-stats", .data = &rpm_data_dba0 },
+ { .compatible = "qcom,msm8916-rpm-stats", .data = &rpm_data_dba0 },
+ { .compatible = "qcom,msm8974-rpm-stats", .data = &rpm_data_dba0 },
{ .compatible = "qcom,rpm-stats", .data = &rpm_data },
{ .compatible = "qcom,rpmh-stats", .data = &rpmh_data },
{ }
diff --git a/drivers/soc/qcom/qmi_interface.c b/drivers/soc/qcom/qmi_interface.c
index 1a03eaa..c8c4c73 100644
--- a/drivers/soc/qcom/qmi_interface.c
+++ b/drivers/soc/qcom/qmi_interface.c
@@ -96,7 +96,7 @@ static void qmi_recv_del_server(struct qmi_handle *qmi,
* @node: id of the dying node
*
* Signals the client that all previously registered services on this node are
- * now gone and then calls the bye callback to allow the client client further
+ * now gone and then calls the bye callback to allow the client further
* cleaning up resources associated with this remote.
*/
static void qmi_recv_bye(struct qmi_handle *qmi,
diff --git a/drivers/soc/qcom/rpmh-rsc.c b/drivers/soc/qcom/rpmh-rsc.c
index 3a12a48..01c2f50c 100644
--- a/drivers/soc/qcom/rpmh-rsc.c
+++ b/drivers/soc/qcom/rpmh-rsc.c
@@ -691,7 +691,7 @@ static int find_slots(struct tcs_group *tcs, const struct tcs_request *msg,
* @drv: The controller.
* @msg: The data to be written to the controller.
*
- * This should only be called for for sleep/wake state, never active-only
+ * This should only be called for sleep/wake state, never active-only
* state.
*
* The caller must ensure that no other RPMH actions are happening and the
diff --git a/drivers/soc/qcom/rpmhpd.c b/drivers/soc/qcom/rpmhpd.c
index 1118345..58f1dc9 100644
--- a/drivers/soc/qcom/rpmhpd.c
+++ b/drivers/soc/qcom/rpmhpd.c
@@ -63,73 +63,134 @@ struct rpmhpd_desc {
static DEFINE_MUTEX(rpmhpd_lock);
-/* SDM845 RPMH powerdomains */
+/* RPMH powerdomains */
-static struct rpmhpd sdm845_ebi = {
+static struct rpmhpd cx_ao;
+static struct rpmhpd mx;
+static struct rpmhpd mx_ao;
+static struct rpmhpd cx = {
+ .pd = { .name = "cx", },
+ .peer = &cx_ao,
+ .res_name = "cx.lvl",
+};
+
+static struct rpmhpd cx_ao = {
+ .pd = { .name = "cx_ao", },
+ .active_only = true,
+ .peer = &cx,
+ .res_name = "cx.lvl",
+};
+
+static struct rpmhpd cx_ao_w_mx_parent;
+static struct rpmhpd cx_w_mx_parent = {
+ .pd = { .name = "cx", },
+ .peer = &cx_ao_w_mx_parent,
+ .parent = &mx.pd,
+ .res_name = "cx.lvl",
+};
+
+static struct rpmhpd cx_ao_w_mx_parent = {
+ .pd = { .name = "cx_ao", },
+ .active_only = true,
+ .peer = &cx_w_mx_parent,
+ .parent = &mx_ao.pd,
+ .res_name = "cx.lvl",
+};
+
+static struct rpmhpd ebi = {
.pd = { .name = "ebi", },
.res_name = "ebi.lvl",
};
-static struct rpmhpd sdm845_lmx = {
- .pd = { .name = "lmx", },
- .res_name = "lmx.lvl",
-};
-
-static struct rpmhpd sdm845_lcx = {
- .pd = { .name = "lcx", },
- .res_name = "lcx.lvl",
-};
-
-static struct rpmhpd sdm845_gfx = {
+static struct rpmhpd gfx = {
.pd = { .name = "gfx", },
.res_name = "gfx.lvl",
};
-static struct rpmhpd sdm845_mss = {
+static struct rpmhpd lcx = {
+ .pd = { .name = "lcx", },
+ .res_name = "lcx.lvl",
+};
+
+static struct rpmhpd lmx = {
+ .pd = { .name = "lmx", },
+ .res_name = "lmx.lvl",
+};
+
+static struct rpmhpd mmcx_ao;
+static struct rpmhpd mmcx = {
+ .pd = { .name = "mmcx", },
+ .peer = &mmcx_ao,
+ .res_name = "mmcx.lvl",
+};
+
+static struct rpmhpd mmcx_ao = {
+ .pd = { .name = "mmcx_ao", },
+ .active_only = true,
+ .peer = &mmcx,
+ .res_name = "mmcx.lvl",
+};
+
+static struct rpmhpd mmcx_ao_w_cx_parent;
+static struct rpmhpd mmcx_w_cx_parent = {
+ .pd = { .name = "mmcx", },
+ .peer = &mmcx_ao_w_cx_parent,
+ .parent = &cx.pd,
+ .res_name = "mmcx.lvl",
+};
+
+static struct rpmhpd mmcx_ao_w_cx_parent = {
+ .pd = { .name = "mmcx_ao", },
+ .active_only = true,
+ .peer = &mmcx_w_cx_parent,
+ .parent = &cx_ao.pd,
+ .res_name = "mmcx.lvl",
+};
+
+static struct rpmhpd mss = {
.pd = { .name = "mss", },
.res_name = "mss.lvl",
};
-static struct rpmhpd sdm845_mx_ao;
-static struct rpmhpd sdm845_mx = {
+static struct rpmhpd mx_ao;
+static struct rpmhpd mx = {
.pd = { .name = "mx", },
- .peer = &sdm845_mx_ao,
+ .peer = &mx_ao,
.res_name = "mx.lvl",
};
-static struct rpmhpd sdm845_mx_ao = {
+static struct rpmhpd mx_ao = {
.pd = { .name = "mx_ao", },
.active_only = true,
- .peer = &sdm845_mx,
+ .peer = &mx,
.res_name = "mx.lvl",
};
-static struct rpmhpd sdm845_cx_ao;
-static struct rpmhpd sdm845_cx = {
- .pd = { .name = "cx", },
- .peer = &sdm845_cx_ao,
- .parent = &sdm845_mx.pd,
- .res_name = "cx.lvl",
+static struct rpmhpd mxc_ao;
+static struct rpmhpd mxc = {
+ .pd = { .name = "mxc", },
+ .peer = &mxc_ao,
+ .res_name = "mxc.lvl",
};
-static struct rpmhpd sdm845_cx_ao = {
- .pd = { .name = "cx_ao", },
+static struct rpmhpd mxc_ao = {
+ .pd = { .name = "mxc_ao", },
.active_only = true,
- .peer = &sdm845_cx,
- .parent = &sdm845_mx_ao.pd,
- .res_name = "cx.lvl",
+ .peer = &mxc,
+ .res_name = "mxc.lvl",
};
+/* SDM845 RPMH powerdomains */
static struct rpmhpd *sdm845_rpmhpds[] = {
- [SDM845_EBI] = &sdm845_ebi,
- [SDM845_MX] = &sdm845_mx,
- [SDM845_MX_AO] = &sdm845_mx_ao,
- [SDM845_CX] = &sdm845_cx,
- [SDM845_CX_AO] = &sdm845_cx_ao,
- [SDM845_LMX] = &sdm845_lmx,
- [SDM845_LCX] = &sdm845_lcx,
- [SDM845_GFX] = &sdm845_gfx,
- [SDM845_MSS] = &sdm845_mss,
+ [SDM845_CX] = &cx_w_mx_parent,
+ [SDM845_CX_AO] = &cx_ao_w_mx_parent,
+ [SDM845_EBI] = &ebi,
+ [SDM845_GFX] = &gfx,
+ [SDM845_LCX] = &lcx,
+ [SDM845_LMX] = &lmx,
+ [SDM845_MSS] = &mss,
+ [SDM845_MX] = &mx,
+ [SDM845_MX_AO] = &mx_ao,
};
static const struct rpmhpd_desc sdm845_desc = {
@@ -139,9 +200,9 @@ static const struct rpmhpd_desc sdm845_desc = {
/* SDX55 RPMH powerdomains */
static struct rpmhpd *sdx55_rpmhpds[] = {
- [SDX55_MSS] = &sdm845_mss,
- [SDX55_MX] = &sdm845_mx,
- [SDX55_CX] = &sdm845_cx,
+ [SDX55_CX] = &cx_w_mx_parent,
+ [SDX55_MSS] = &mss,
+ [SDX55_MX] = &mx,
};
static const struct rpmhpd_desc sdx55_desc = {
@@ -151,12 +212,12 @@ static const struct rpmhpd_desc sdx55_desc = {
/* SM6350 RPMH powerdomains */
static struct rpmhpd *sm6350_rpmhpds[] = {
- [SM6350_CX] = &sdm845_cx,
- [SM6350_GFX] = &sdm845_gfx,
- [SM6350_LCX] = &sdm845_lcx,
- [SM6350_LMX] = &sdm845_lmx,
- [SM6350_MSS] = &sdm845_mss,
- [SM6350_MX] = &sdm845_mx,
+ [SM6350_CX] = &cx_w_mx_parent,
+ [SM6350_GFX] = &gfx,
+ [SM6350_LCX] = &lcx,
+ [SM6350_LMX] = &lmx,
+ [SM6350_MSS] = &mss,
+ [SM6350_MX] = &mx,
};
static const struct rpmhpd_desc sm6350_desc = {
@@ -165,33 +226,18 @@ static const struct rpmhpd_desc sm6350_desc = {
};
/* SM8150 RPMH powerdomains */
-
-static struct rpmhpd sm8150_mmcx_ao;
-static struct rpmhpd sm8150_mmcx = {
- .pd = { .name = "mmcx", },
- .peer = &sm8150_mmcx_ao,
- .res_name = "mmcx.lvl",
-};
-
-static struct rpmhpd sm8150_mmcx_ao = {
- .pd = { .name = "mmcx_ao", },
- .active_only = true,
- .peer = &sm8150_mmcx,
- .res_name = "mmcx.lvl",
-};
-
static struct rpmhpd *sm8150_rpmhpds[] = {
- [SM8150_MSS] = &sdm845_mss,
- [SM8150_EBI] = &sdm845_ebi,
- [SM8150_LMX] = &sdm845_lmx,
- [SM8150_LCX] = &sdm845_lcx,
- [SM8150_GFX] = &sdm845_gfx,
- [SM8150_MX] = &sdm845_mx,
- [SM8150_MX_AO] = &sdm845_mx_ao,
- [SM8150_CX] = &sdm845_cx,
- [SM8150_CX_AO] = &sdm845_cx_ao,
- [SM8150_MMCX] = &sm8150_mmcx,
- [SM8150_MMCX_AO] = &sm8150_mmcx_ao,
+ [SM8150_CX] = &cx_w_mx_parent,
+ [SM8150_CX_AO] = &cx_ao_w_mx_parent,
+ [SM8150_EBI] = &ebi,
+ [SM8150_GFX] = &gfx,
+ [SM8150_LCX] = &lcx,
+ [SM8150_LMX] = &lmx,
+ [SM8150_MMCX] = &mmcx,
+ [SM8150_MMCX_AO] = &mmcx_ao,
+ [SM8150_MSS] = &mss,
+ [SM8150_MX] = &mx,
+ [SM8150_MX_AO] = &mx_ao,
};
static const struct rpmhpd_desc sm8150_desc = {
@@ -199,17 +245,18 @@ static const struct rpmhpd_desc sm8150_desc = {
.num_pds = ARRAY_SIZE(sm8150_rpmhpds),
};
+/* SM8250 RPMH powerdomains */
static struct rpmhpd *sm8250_rpmhpds[] = {
- [SM8250_CX] = &sdm845_cx,
- [SM8250_CX_AO] = &sdm845_cx_ao,
- [SM8250_EBI] = &sdm845_ebi,
- [SM8250_GFX] = &sdm845_gfx,
- [SM8250_LCX] = &sdm845_lcx,
- [SM8250_LMX] = &sdm845_lmx,
- [SM8250_MMCX] = &sm8150_mmcx,
- [SM8250_MMCX_AO] = &sm8150_mmcx_ao,
- [SM8250_MX] = &sdm845_mx,
- [SM8250_MX_AO] = &sdm845_mx_ao,
+ [SM8250_CX] = &cx_w_mx_parent,
+ [SM8250_CX_AO] = &cx_ao_w_mx_parent,
+ [SM8250_EBI] = &ebi,
+ [SM8250_GFX] = &gfx,
+ [SM8250_LCX] = &lcx,
+ [SM8250_LMX] = &lmx,
+ [SM8250_MMCX] = &mmcx,
+ [SM8250_MMCX_AO] = &mmcx_ao,
+ [SM8250_MX] = &mx,
+ [SM8250_MX_AO] = &mx_ao,
};
static const struct rpmhpd_desc sm8250_desc = {
@@ -218,34 +265,20 @@ static const struct rpmhpd_desc sm8250_desc = {
};
/* SM8350 Power domains */
-static struct rpmhpd sm8350_mxc_ao;
-static struct rpmhpd sm8350_mxc = {
- .pd = { .name = "mxc", },
- .peer = &sm8350_mxc_ao,
- .res_name = "mxc.lvl",
-};
-
-static struct rpmhpd sm8350_mxc_ao = {
- .pd = { .name = "mxc_ao", },
- .active_only = true,
- .peer = &sm8350_mxc,
- .res_name = "mxc.lvl",
-};
-
static struct rpmhpd *sm8350_rpmhpds[] = {
- [SM8350_CX] = &sdm845_cx,
- [SM8350_CX_AO] = &sdm845_cx_ao,
- [SM8350_EBI] = &sdm845_ebi,
- [SM8350_GFX] = &sdm845_gfx,
- [SM8350_LCX] = &sdm845_lcx,
- [SM8350_LMX] = &sdm845_lmx,
- [SM8350_MMCX] = &sm8150_mmcx,
- [SM8350_MMCX_AO] = &sm8150_mmcx_ao,
- [SM8350_MX] = &sdm845_mx,
- [SM8350_MX_AO] = &sdm845_mx_ao,
- [SM8350_MXC] = &sm8350_mxc,
- [SM8350_MXC_AO] = &sm8350_mxc_ao,
- [SM8350_MSS] = &sdm845_mss,
+ [SM8350_CX] = &cx_w_mx_parent,
+ [SM8350_CX_AO] = &cx_ao_w_mx_parent,
+ [SM8350_EBI] = &ebi,
+ [SM8350_GFX] = &gfx,
+ [SM8350_LCX] = &lcx,
+ [SM8350_LMX] = &lmx,
+ [SM8350_MMCX] = &mmcx,
+ [SM8350_MMCX_AO] = &mmcx_ao,
+ [SM8350_MSS] = &mss,
+ [SM8350_MX] = &mx,
+ [SM8350_MX_AO] = &mx_ao,
+ [SM8350_MXC] = &mxc,
+ [SM8350_MXC_AO] = &mxc_ao,
};
static const struct rpmhpd_desc sm8350_desc = {
@@ -253,16 +286,38 @@ static const struct rpmhpd_desc sm8350_desc = {
.num_pds = ARRAY_SIZE(sm8350_rpmhpds),
};
+/* SM8450 RPMH powerdomains */
+static struct rpmhpd *sm8450_rpmhpds[] = {
+ [SM8450_CX] = &cx,
+ [SM8450_CX_AO] = &cx_ao,
+ [SM8450_EBI] = &ebi,
+ [SM8450_GFX] = &gfx,
+ [SM8450_LCX] = &lcx,
+ [SM8450_LMX] = &lmx,
+ [SM8450_MMCX] = &mmcx_w_cx_parent,
+ [SM8450_MMCX_AO] = &mmcx_ao_w_cx_parent,
+ [SM8450_MSS] = &mss,
+ [SM8450_MX] = &mx,
+ [SM8450_MX_AO] = &mx_ao,
+ [SM8450_MXC] = &mxc,
+ [SM8450_MXC_AO] = &mxc_ao,
+};
+
+static const struct rpmhpd_desc sm8450_desc = {
+ .rpmhpds = sm8450_rpmhpds,
+ .num_pds = ARRAY_SIZE(sm8450_rpmhpds),
+};
+
/* SC7180 RPMH powerdomains */
static struct rpmhpd *sc7180_rpmhpds[] = {
- [SC7180_CX] = &sdm845_cx,
- [SC7180_CX_AO] = &sdm845_cx_ao,
- [SC7180_GFX] = &sdm845_gfx,
- [SC7180_MX] = &sdm845_mx,
- [SC7180_MX_AO] = &sdm845_mx_ao,
- [SC7180_LMX] = &sdm845_lmx,
- [SC7180_LCX] = &sdm845_lcx,
- [SC7180_MSS] = &sdm845_mss,
+ [SC7180_CX] = &cx_w_mx_parent,
+ [SC7180_CX_AO] = &cx_ao_w_mx_parent,
+ [SC7180_GFX] = &gfx,
+ [SC7180_LCX] = &lcx,
+ [SC7180_LMX] = &lmx,
+ [SC7180_MSS] = &mss,
+ [SC7180_MX] = &mx,
+ [SC7180_MX_AO] = &mx_ao,
};
static const struct rpmhpd_desc sc7180_desc = {
@@ -272,15 +327,15 @@ static const struct rpmhpd_desc sc7180_desc = {
/* SC7280 RPMH powerdomains */
static struct rpmhpd *sc7280_rpmhpds[] = {
- [SC7280_CX] = &sdm845_cx,
- [SC7280_CX_AO] = &sdm845_cx_ao,
- [SC7280_EBI] = &sdm845_ebi,
- [SC7280_GFX] = &sdm845_gfx,
- [SC7280_MX] = &sdm845_mx,
- [SC7280_MX_AO] = &sdm845_mx_ao,
- [SC7280_LMX] = &sdm845_lmx,
- [SC7280_LCX] = &sdm845_lcx,
- [SC7280_MSS] = &sdm845_mss,
+ [SC7280_CX] = &cx,
+ [SC7280_CX_AO] = &cx_ao,
+ [SC7280_EBI] = &ebi,
+ [SC7280_GFX] = &gfx,
+ [SC7280_LCX] = &lcx,
+ [SC7280_LMX] = &lmx,
+ [SC7280_MSS] = &mss,
+ [SC7280_MX] = &mx,
+ [SC7280_MX_AO] = &mx_ao,
};
static const struct rpmhpd_desc sc7280_desc = {
@@ -290,17 +345,17 @@ static const struct rpmhpd_desc sc7280_desc = {
/* SC8180x RPMH powerdomains */
static struct rpmhpd *sc8180x_rpmhpds[] = {
- [SC8180X_CX] = &sdm845_cx,
- [SC8180X_CX_AO] = &sdm845_cx_ao,
- [SC8180X_EBI] = &sdm845_ebi,
- [SC8180X_GFX] = &sdm845_gfx,
- [SC8180X_LCX] = &sdm845_lcx,
- [SC8180X_LMX] = &sdm845_lmx,
- [SC8180X_MMCX] = &sm8150_mmcx,
- [SC8180X_MMCX_AO] = &sm8150_mmcx_ao,
- [SC8180X_MSS] = &sdm845_mss,
- [SC8180X_MX] = &sdm845_mx,
- [SC8180X_MX_AO] = &sdm845_mx_ao,
+ [SC8180X_CX] = &cx_w_mx_parent,
+ [SC8180X_CX_AO] = &cx_ao_w_mx_parent,
+ [SC8180X_EBI] = &ebi,
+ [SC8180X_GFX] = &gfx,
+ [SC8180X_LCX] = &lcx,
+ [SC8180X_LMX] = &lmx,
+ [SC8180X_MMCX] = &mmcx,
+ [SC8180X_MMCX_AO] = &mmcx_ao,
+ [SC8180X_MSS] = &mss,
+ [SC8180X_MX] = &mx,
+ [SC8180X_MX_AO] = &mx_ao,
};
static const struct rpmhpd_desc sc8180x_desc = {
@@ -318,6 +373,7 @@ static const struct of_device_id rpmhpd_match_table[] = {
{ .compatible = "qcom,sm8150-rpmhpd", .data = &sm8150_desc },
{ .compatible = "qcom,sm8250-rpmhpd", .data = &sm8250_desc },
{ .compatible = "qcom,sm8350-rpmhpd", .data = &sm8350_desc },
+ { .compatible = "qcom,sm8450-rpmhpd", .data = &sm8450_desc },
{ }
};
MODULE_DEVICE_TABLE(of, rpmhpd_match_table);
diff --git a/drivers/soc/qcom/rpmpd.c b/drivers/soc/qcom/rpmpd.c
index 4f69fb9..0a8d8d2 100644
--- a/drivers/soc/qcom/rpmpd.c
+++ b/drivers/soc/qcom/rpmpd.c
@@ -102,7 +102,6 @@ struct rpmpd {
const bool active_only;
unsigned int corner;
bool enabled;
- const char *res_name;
const int res_type;
const int res_id;
struct qcom_smd_rpm *rpm;
@@ -396,6 +395,45 @@ static const struct rpmpd_desc sm6115_desc = {
.max_state = RPM_SMD_LEVEL_TURBO_NO_CPR,
};
+/* sm6125 RPM Power domains */
+DEFINE_RPMPD_PAIR(sm6125, vddcx, vddcx_ao, RWCX, LEVEL, 0);
+DEFINE_RPMPD_VFL(sm6125, vddcx_vfl, RWCX, 0);
+
+DEFINE_RPMPD_PAIR(sm6125, vddmx, vddmx_ao, RWMX, LEVEL, 0);
+DEFINE_RPMPD_VFL(sm6125, vddmx_vfl, RWMX, 0);
+
+static struct rpmpd *sm6125_rpmpds[] = {
+ [SM6125_VDDCX] = &sm6125_vddcx,
+ [SM6125_VDDCX_AO] = &sm6125_vddcx_ao,
+ [SM6125_VDDCX_VFL] = &sm6125_vddcx_vfl,
+ [SM6125_VDDMX] = &sm6125_vddmx,
+ [SM6125_VDDMX_AO] = &sm6125_vddmx_ao,
+ [SM6125_VDDMX_VFL] = &sm6125_vddmx_vfl,
+};
+
+static const struct rpmpd_desc sm6125_desc = {
+ .rpmpds = sm6125_rpmpds,
+ .num_pds = ARRAY_SIZE(sm6125_rpmpds),
+ .max_state = RPM_SMD_LEVEL_BINNING,
+};
+
+static struct rpmpd *qcm2290_rpmpds[] = {
+ [QCM2290_VDDCX] = &sm6115_vddcx,
+ [QCM2290_VDDCX_AO] = &sm6115_vddcx_ao,
+ [QCM2290_VDDCX_VFL] = &sm6115_vddcx_vfl,
+ [QCM2290_VDDMX] = &sm6115_vddmx,
+ [QCM2290_VDDMX_AO] = &sm6115_vddmx_ao,
+ [QCM2290_VDDMX_VFL] = &sm6115_vddmx_vfl,
+ [QCM2290_VDD_LPI_CX] = &sm6115_vdd_lpi_cx,
+ [QCM2290_VDD_LPI_MX] = &sm6115_vdd_lpi_mx,
+};
+
+static const struct rpmpd_desc qcm2290_desc = {
+ .rpmpds = qcm2290_rpmpds,
+ .num_pds = ARRAY_SIZE(qcm2290_rpmpds),
+ .max_state = RPM_SMD_LEVEL_TURBO_NO_CPR,
+};
+
static const struct of_device_id rpmpd_match_table[] = {
{ .compatible = "qcom,mdm9607-rpmpd", .data = &mdm9607_desc },
{ .compatible = "qcom,msm8916-rpmpd", .data = &msm8916_desc },
@@ -405,9 +443,11 @@ static const struct of_device_id rpmpd_match_table[] = {
{ .compatible = "qcom,msm8994-rpmpd", .data = &msm8994_desc },
{ .compatible = "qcom,msm8996-rpmpd", .data = &msm8996_desc },
{ .compatible = "qcom,msm8998-rpmpd", .data = &msm8998_desc },
+ { .compatible = "qcom,qcm2290-rpmpd", .data = &qcm2290_desc },
{ .compatible = "qcom,qcs404-rpmpd", .data = &qcs404_desc },
{ .compatible = "qcom,sdm660-rpmpd", .data = &sdm660_desc },
{ .compatible = "qcom,sm6115-rpmpd", .data = &sm6115_desc },
+ { .compatible = "qcom,sm6125-rpmpd", .data = &sm6125_desc },
{ }
};
MODULE_DEVICE_TABLE(of, rpmpd_match_table);
diff --git a/drivers/soc/qcom/smem.c b/drivers/soc/qcom/smem.c
index c7e519b..e2057d8 100644
--- a/drivers/soc/qcom/smem.c
+++ b/drivers/soc/qcom/smem.c
@@ -85,7 +85,7 @@
#define SMEM_GLOBAL_HOST 0xfffe
/* Max number of processors/hosts in a system */
-#define SMEM_HOST_COUNT 14
+#define SMEM_HOST_COUNT 15
/**
* struct smem_proc_comm - proc_comm communication struct (legacy)
diff --git a/drivers/soc/qcom/socinfo.c b/drivers/soc/qcom/socinfo.c
index 9a0eb59..6dc0f39 100644
--- a/drivers/soc/qcom/socinfo.c
+++ b/drivers/soc/qcom/socinfo.c
@@ -313,8 +313,11 @@ static const struct soc_id soc_id[] = {
{ 421, "IPQ6000" },
{ 422, "IPQ6010" },
{ 425, "SC7180" },
+ { 434, "SM6350" },
{ 453, "IPQ6005" },
{ 455, "QRB5165" },
+ { 457, "SM8450" },
+ { 459, "SM7225" },
};
static const char *socinfo_machine(struct device *dev, unsigned int id)
diff --git a/drivers/soc/renesas/Kconfig b/drivers/soc/renesas/Kconfig
index ce16ef5..2cbd03d 100644
--- a/drivers/soc/renesas/Kconfig
+++ b/drivers/soc/renesas/Kconfig
@@ -235,6 +235,13 @@
This enables support for the Renesas R-Car M3-W+ SoC.
This includes different gradings like R-Car M3e and M3e-2G.
+config ARCH_R8A779F0
+ bool "ARM64 Platform support for R-Car S4-8"
+ select ARCH_RCAR_GEN3
+ select SYSC_R8A779F0
+ help
+ This enables support for the Renesas R-Car S4-8 SoC.
+
config ARCH_R8A77980
bool "ARM64 Platform support for R-Car V3H"
select ARCH_RCAR_GEN3
@@ -297,6 +304,9 @@
config SYSC_RCAR
bool "System Controller support for R-Car" if COMPILE_TEST
+config SYSC_RCAR_GEN4
+ bool "System Controller support for R-Car Gen4" if COMPILE_TEST
+
config SYSC_R8A77995
bool "System Controller support for R-Car D3" if COMPILE_TEST
select SYSC_RCAR
@@ -337,6 +347,10 @@
bool "System Controller support for R-Car M3-W+" if COMPILE_TEST
select SYSC_RCAR
+config SYSC_R8A779F0
+ bool "System Controller support for R-Car S4-8" if COMPILE_TEST
+ select SYSC_RCAR_GEN4
+
config SYSC_R8A7792
bool "System Controller support for R-Car V2H" if COMPILE_TEST
select SYSC_RCAR
@@ -351,6 +365,7 @@
config SYSC_R8A779A0
bool "System Controller support for R-Car V3U" if COMPILE_TEST
+ select SYSC_RCAR_GEN4
config SYSC_RMOBILE
bool "System Controller support for R-Mobile" if COMPILE_TEST
diff --git a/drivers/soc/renesas/Makefile b/drivers/soc/renesas/Makefile
index 9b29bed..deeb41f 100644
--- a/drivers/soc/renesas/Makefile
+++ b/drivers/soc/renesas/Makefile
@@ -25,6 +25,7 @@
obj-$(CONFIG_SYSC_R8A77990) += r8a77990-sysc.o
obj-$(CONFIG_SYSC_R8A77995) += r8a77995-sysc.o
obj-$(CONFIG_SYSC_R8A779A0) += r8a779a0-sysc.o
+obj-$(CONFIG_SYSC_R8A779F0) += r8a779f0-sysc.o
ifdef CONFIG_SMP
obj-$(CONFIG_ARCH_R9A06G032) += r9a06g032-smp.o
endif
@@ -32,4 +33,5 @@
# Family
obj-$(CONFIG_RST_RCAR) += rcar-rst.o
obj-$(CONFIG_SYSC_RCAR) += rcar-sysc.o
+obj-$(CONFIG_SYSC_RCAR_GEN4) += rcar-gen4-sysc.o
obj-$(CONFIG_SYSC_RMOBILE) += rmobile-sysc.o
diff --git a/drivers/soc/renesas/r8a779a0-sysc.c b/drivers/soc/renesas/r8a779a0-sysc.c
index 7410b9f..fdfc857 100644
--- a/drivers/soc/renesas/r8a779a0-sysc.c
+++ b/drivers/soc/renesas/r8a779a0-sysc.c
@@ -21,35 +21,9 @@
#include <dt-bindings/power/r8a779a0-sysc.h>
-/*
- * Power Domain flags
- */
-#define PD_CPU BIT(0) /* Area contains main CPU core */
-#define PD_SCU BIT(1) /* Area contains SCU and L2 cache */
-#define PD_NO_CR BIT(2) /* Area lacks PWR{ON,OFF}CR registers */
+#include "rcar-gen4-sysc.h"
-#define PD_CPU_NOCR PD_CPU | PD_NO_CR /* CPU area lacks CR */
-#define PD_ALWAYS_ON PD_NO_CR /* Always-on area */
-
-/*
- * Description of a Power Area
- */
-struct r8a779a0_sysc_area {
- const char *name;
- u8 pdr; /* PDRn */
- int parent; /* -1 if none */
- unsigned int flags; /* See PD_* */
-};
-
-/*
- * SoC-specific Power Area Description
- */
-struct r8a779a0_sysc_info {
- const struct r8a779a0_sysc_area *areas;
- unsigned int num_areas;
-};
-
-static struct r8a779a0_sysc_area r8a779a0_areas[] __initdata = {
+static struct rcar_gen4_sysc_area r8a779a0_areas[] __initdata = {
{ "always-on", R8A779A0_PD_ALWAYS_ON, -1, PD_ALWAYS_ON },
{ "a3e0", R8A779A0_PD_A3E0, R8A779A0_PD_ALWAYS_ON, PD_SCU },
{ "a3e1", R8A779A0_PD_A3E1, R8A779A0_PD_ALWAYS_ON, PD_SCU },
@@ -96,355 +70,7 @@ static struct r8a779a0_sysc_area r8a779a0_areas[] __initdata = {
{ "a1dsp1", R8A779A0_PD_A1DSP1, R8A779A0_PD_A2CN1 },
};
-static const struct r8a779a0_sysc_info r8a779a0_sysc_info __initconst = {
+const struct rcar_gen4_sysc_info r8a779a0_sysc_info __initconst = {
.areas = r8a779a0_areas,
.num_areas = ARRAY_SIZE(r8a779a0_areas),
};
-
-/* SYSC Common */
-#define SYSCSR 0x000 /* SYSC Status Register */
-#define SYSCPONSR(x) (0x800 + ((x) * 0x4)) /* Power-ON Status Register 0 */
-#define SYSCPOFFSR(x) (0x808 + ((x) * 0x4)) /* Power-OFF Status Register */
-#define SYSCISCR(x) (0x810 + ((x) * 0x4)) /* Interrupt Status/Clear Register */
-#define SYSCIER(x) (0x820 + ((x) * 0x4)) /* Interrupt Enable Register */
-#define SYSCIMR(x) (0x830 + ((x) * 0x4)) /* Interrupt Mask Register */
-
-/* Power Domain Registers */
-#define PDRSR(n) (0x1000 + ((n) * 0x40))
-#define PDRONCR(n) (0x1004 + ((n) * 0x40))
-#define PDROFFCR(n) (0x1008 + ((n) * 0x40))
-#define PDRESR(n) (0x100C + ((n) * 0x40))
-
-/* PWRON/PWROFF */
-#define PWRON_PWROFF BIT(0) /* Power-ON/OFF request */
-
-/* PDRESR */
-#define PDRESR_ERR BIT(0)
-
-/* PDRSR */
-#define PDRSR_OFF BIT(0) /* Power-OFF state */
-#define PDRSR_ON BIT(4) /* Power-ON state */
-#define PDRSR_OFF_STATE BIT(8) /* Processing Power-OFF sequence */
-#define PDRSR_ON_STATE BIT(12) /* Processing Power-ON sequence */
-
-#define SYSCSR_BUSY GENMASK(1, 0) /* All bit sets is not busy */
-
-#define SYSCSR_TIMEOUT 10000
-#define SYSCSR_DELAY_US 10
-
-#define PDRESR_RETRIES 1000
-#define PDRESR_DELAY_US 10
-
-#define SYSCISR_TIMEOUT 10000
-#define SYSCISR_DELAY_US 10
-
-#define NUM_DOMAINS_EACH_REG BITS_PER_TYPE(u32)
-
-static void __iomem *r8a779a0_sysc_base;
-static DEFINE_SPINLOCK(r8a779a0_sysc_lock); /* SMP CPUs + I/O devices */
-
-static int r8a779a0_sysc_pwr_on_off(u8 pdr, bool on)
-{
- unsigned int reg_offs;
- u32 val;
- int ret;
-
- if (on)
- reg_offs = PDRONCR(pdr);
- else
- reg_offs = PDROFFCR(pdr);
-
- /* Wait until SYSC is ready to accept a power request */
- ret = readl_poll_timeout_atomic(r8a779a0_sysc_base + SYSCSR, val,
- (val & SYSCSR_BUSY) == SYSCSR_BUSY,
- SYSCSR_DELAY_US, SYSCSR_TIMEOUT);
- if (ret < 0)
- return -EAGAIN;
-
- /* Submit power shutoff or power resume request */
- iowrite32(PWRON_PWROFF, r8a779a0_sysc_base + reg_offs);
-
- return 0;
-}
-
-static int clear_irq_flags(unsigned int reg_idx, unsigned int isr_mask)
-{
- u32 val;
- int ret;
-
- iowrite32(isr_mask, r8a779a0_sysc_base + SYSCISCR(reg_idx));
-
- ret = readl_poll_timeout_atomic(r8a779a0_sysc_base + SYSCISCR(reg_idx),
- val, !(val & isr_mask),
- SYSCISR_DELAY_US, SYSCISR_TIMEOUT);
- if (ret < 0) {
- pr_err("\n %s : Can not clear IRQ flags in SYSCISCR", __func__);
- return -EIO;
- }
-
- return 0;
-}
-
-static int r8a779a0_sysc_power(u8 pdr, bool on)
-{
- unsigned int isr_mask;
- unsigned int reg_idx, bit_idx;
- unsigned int status;
- unsigned long flags;
- int ret = 0;
- u32 val;
- int k;
-
- spin_lock_irqsave(&r8a779a0_sysc_lock, flags);
-
- reg_idx = pdr / NUM_DOMAINS_EACH_REG;
- bit_idx = pdr % NUM_DOMAINS_EACH_REG;
-
- isr_mask = BIT(bit_idx);
-
- /*
- * The interrupt source needs to be enabled, but masked, to prevent the
- * CPU from receiving it.
- */
- iowrite32(ioread32(r8a779a0_sysc_base + SYSCIER(reg_idx)) | isr_mask,
- r8a779a0_sysc_base + SYSCIER(reg_idx));
- iowrite32(ioread32(r8a779a0_sysc_base + SYSCIMR(reg_idx)) | isr_mask,
- r8a779a0_sysc_base + SYSCIMR(reg_idx));
-
- ret = clear_irq_flags(reg_idx, isr_mask);
- if (ret)
- goto out;
-
- /* Submit power shutoff or resume request until it was accepted */
- for (k = 0; k < PDRESR_RETRIES; k++) {
- ret = r8a779a0_sysc_pwr_on_off(pdr, on);
- if (ret)
- goto out;
-
- status = ioread32(r8a779a0_sysc_base + PDRESR(pdr));
- if (!(status & PDRESR_ERR))
- break;
-
- udelay(PDRESR_DELAY_US);
- }
-
- if (k == PDRESR_RETRIES) {
- ret = -EIO;
- goto out;
- }
-
- /* Wait until the power shutoff or resume request has completed * */
- ret = readl_poll_timeout_atomic(r8a779a0_sysc_base + SYSCISCR(reg_idx),
- val, (val & isr_mask),
- SYSCISR_DELAY_US, SYSCISR_TIMEOUT);
- if (ret < 0) {
- ret = -EIO;
- goto out;
- }
-
- /* Clear interrupt flags */
- ret = clear_irq_flags(reg_idx, isr_mask);
- if (ret)
- goto out;
-
- out:
- spin_unlock_irqrestore(&r8a779a0_sysc_lock, flags);
-
- pr_debug("sysc power %s domain %d: %08x -> %d\n", on ? "on" : "off",
- pdr, ioread32(r8a779a0_sysc_base + SYSCISCR(reg_idx)), ret);
- return ret;
-}
-
-static bool r8a779a0_sysc_power_is_off(u8 pdr)
-{
- unsigned int st;
-
- st = ioread32(r8a779a0_sysc_base + PDRSR(pdr));
-
- if (st & PDRSR_OFF)
- return true;
-
- return false;
-}
-
-struct r8a779a0_sysc_pd {
- struct generic_pm_domain genpd;
- u8 pdr;
- unsigned int flags;
- char name[];
-};
-
-static inline struct r8a779a0_sysc_pd *to_r8a779a0_pd(struct generic_pm_domain *d)
-{
- return container_of(d, struct r8a779a0_sysc_pd, genpd);
-}
-
-static int r8a779a0_sysc_pd_power_off(struct generic_pm_domain *genpd)
-{
- struct r8a779a0_sysc_pd *pd = to_r8a779a0_pd(genpd);
-
- pr_debug("%s: %s\n", __func__, genpd->name);
- return r8a779a0_sysc_power(pd->pdr, false);
-}
-
-static int r8a779a0_sysc_pd_power_on(struct generic_pm_domain *genpd)
-{
- struct r8a779a0_sysc_pd *pd = to_r8a779a0_pd(genpd);
-
- pr_debug("%s: %s\n", __func__, genpd->name);
- return r8a779a0_sysc_power(pd->pdr, true);
-}
-
-static int __init r8a779a0_sysc_pd_setup(struct r8a779a0_sysc_pd *pd)
-{
- struct generic_pm_domain *genpd = &pd->genpd;
- const char *name = pd->genpd.name;
- int error;
-
- if (pd->flags & PD_CPU) {
- /*
- * This domain contains a CPU core and therefore it should
- * only be turned off if the CPU is not in use.
- */
- pr_debug("PM domain %s contains %s\n", name, "CPU");
- genpd->flags |= GENPD_FLAG_ALWAYS_ON;
- } else if (pd->flags & PD_SCU) {
- /*
- * This domain contains an SCU and cache-controller, and
- * therefore it should only be turned off if the CPU cores are
- * not in use.
- */
- pr_debug("PM domain %s contains %s\n", name, "SCU");
- genpd->flags |= GENPD_FLAG_ALWAYS_ON;
- } else if (pd->flags & PD_NO_CR) {
- /*
- * This domain cannot be turned off.
- */
- genpd->flags |= GENPD_FLAG_ALWAYS_ON;
- }
-
- if (!(pd->flags & (PD_CPU | PD_SCU))) {
- /* Enable Clock Domain for I/O devices */
- genpd->flags |= GENPD_FLAG_PM_CLK | GENPD_FLAG_ACTIVE_WAKEUP;
- genpd->attach_dev = cpg_mssr_attach_dev;
- genpd->detach_dev = cpg_mssr_detach_dev;
- }
-
- genpd->power_off = r8a779a0_sysc_pd_power_off;
- genpd->power_on = r8a779a0_sysc_pd_power_on;
-
- if (pd->flags & (PD_CPU | PD_NO_CR)) {
- /* Skip CPUs (handled by SMP code) and areas without control */
- pr_debug("%s: Not touching %s\n", __func__, genpd->name);
- goto finalize;
- }
-
- if (!r8a779a0_sysc_power_is_off(pd->pdr)) {
- pr_debug("%s: %s is already powered\n", __func__, genpd->name);
- goto finalize;
- }
-
- r8a779a0_sysc_power(pd->pdr, true);
-
-finalize:
- error = pm_genpd_init(genpd, &simple_qos_governor, false);
- if (error)
- pr_err("Failed to init PM domain %s: %d\n", name, error);
-
- return error;
-}
-
-static const struct of_device_id r8a779a0_sysc_matches[] __initconst = {
- { .compatible = "renesas,r8a779a0-sysc", .data = &r8a779a0_sysc_info },
- { /* sentinel */ }
-};
-
-struct r8a779a0_pm_domains {
- struct genpd_onecell_data onecell_data;
- struct generic_pm_domain *domains[R8A779A0_PD_ALWAYS_ON + 1];
-};
-
-static struct genpd_onecell_data *r8a779a0_sysc_onecell_data;
-
-static int __init r8a779a0_sysc_pd_init(void)
-{
- const struct r8a779a0_sysc_info *info;
- const struct of_device_id *match;
- struct r8a779a0_pm_domains *domains;
- struct device_node *np;
- void __iomem *base;
- unsigned int i;
- int error;
-
- np = of_find_matching_node_and_match(NULL, r8a779a0_sysc_matches, &match);
- if (!np)
- return -ENODEV;
-
- info = match->data;
-
- base = of_iomap(np, 0);
- if (!base) {
- pr_warn("%pOF: Cannot map regs\n", np);
- error = -ENOMEM;
- goto out_put;
- }
-
- r8a779a0_sysc_base = base;
-
- domains = kzalloc(sizeof(*domains), GFP_KERNEL);
- if (!domains) {
- error = -ENOMEM;
- goto out_put;
- }
-
- domains->onecell_data.domains = domains->domains;
- domains->onecell_data.num_domains = ARRAY_SIZE(domains->domains);
- r8a779a0_sysc_onecell_data = &domains->onecell_data;
-
- for (i = 0; i < info->num_areas; i++) {
- const struct r8a779a0_sysc_area *area = &info->areas[i];
- struct r8a779a0_sysc_pd *pd;
- size_t n;
-
- if (!area->name) {
- /* Skip NULLified area */
- continue;
- }
-
- n = strlen(area->name) + 1;
- pd = kzalloc(sizeof(*pd) + n, GFP_KERNEL);
- if (!pd) {
- error = -ENOMEM;
- goto out_put;
- }
-
- memcpy(pd->name, area->name, n);
- pd->genpd.name = pd->name;
- pd->pdr = area->pdr;
- pd->flags = area->flags;
-
- error = r8a779a0_sysc_pd_setup(pd);
- if (error)
- goto out_put;
-
- domains->domains[area->pdr] = &pd->genpd;
-
- if (area->parent < 0)
- continue;
-
- error = pm_genpd_add_subdomain(domains->domains[area->parent],
- &pd->genpd);
- if (error) {
- pr_warn("Failed to add PM subdomain %s to parent %u\n",
- area->name, area->parent);
- goto out_put;
- }
- }
-
- error = of_genpd_add_provider_onecell(np, &domains->onecell_data);
-
-out_put:
- of_node_put(np);
- return error;
-}
-early_initcall(r8a779a0_sysc_pd_init);
diff --git a/drivers/soc/renesas/r8a779f0-sysc.c b/drivers/soc/renesas/r8a779f0-sysc.c
new file mode 100644
index 0000000..5602aa6
--- /dev/null
+++ b/drivers/soc/renesas/r8a779f0-sysc.c
@@ -0,0 +1,47 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Renesas R-Car S4-8 System Controller
+ *
+ * Copyright (C) 2021 Renesas Electronics Corp.
+ */
+
+#include <linux/bits.h>
+#include <linux/clk/renesas.h>
+#include <linux/delay.h>
+#include <linux/err.h>
+#include <linux/io.h>
+#include <linux/iopoll.h>
+#include <linux/kernel.h>
+#include <linux/mm.h>
+#include <linux/of_address.h>
+#include <linux/pm_domain.h>
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+#include <linux/types.h>
+
+#include <dt-bindings/power/r8a779f0-sysc.h>
+
+#include "rcar-gen4-sysc.h"
+
+static struct rcar_gen4_sysc_area r8a779f0_areas[] __initdata = {
+ { "always-on", R8A779F0_PD_ALWAYS_ON, -1, PD_ALWAYS_ON },
+ { "a3e0", R8A779F0_PD_A3E0, R8A779F0_PD_ALWAYS_ON, PD_SCU },
+ { "a3e1", R8A779F0_PD_A3E1, R8A779F0_PD_ALWAYS_ON, PD_SCU },
+ { "a2e0d0", R8A779F0_PD_A2E0D0, R8A779F0_PD_A3E0, PD_SCU },
+ { "a2e0d1", R8A779F0_PD_A2E0D1, R8A779F0_PD_A3E0, PD_SCU },
+ { "a2e1d0", R8A779F0_PD_A2E1D0, R8A779F0_PD_A3E1, PD_SCU },
+ { "a2e1d1", R8A779F0_PD_A2E1D1, R8A779F0_PD_A3E1, PD_SCU },
+ { "a1e0d0c0", R8A779F0_PD_A1E0D0C0, R8A779F0_PD_A2E0D0, PD_CPU_NOCR },
+ { "a1e0d0c1", R8A779F0_PD_A1E0D0C1, R8A779F0_PD_A2E0D0, PD_CPU_NOCR },
+ { "a1e0d1c0", R8A779F0_PD_A1E0D1C0, R8A779F0_PD_A2E0D1, PD_CPU_NOCR },
+ { "a1e0d1c1", R8A779F0_PD_A1E0D1C1, R8A779F0_PD_A2E0D1, PD_CPU_NOCR },
+ { "a1e1d0c0", R8A779F0_PD_A1E1D0C0, R8A779F0_PD_A2E1D0, PD_CPU_NOCR },
+ { "a1e1d0c1", R8A779F0_PD_A1E1D0C1, R8A779F0_PD_A2E1D0, PD_CPU_NOCR },
+ { "a1e1d1c0", R8A779F0_PD_A1E1D1C0, R8A779F0_PD_A2E1D1, PD_CPU_NOCR },
+ { "a1e1d1c1", R8A779F0_PD_A1E1D1C1, R8A779F0_PD_A2E1D1, PD_CPU_NOCR },
+};
+
+const struct rcar_gen4_sysc_info r8a779f0_sysc_info __initconst = {
+ .areas = r8a779f0_areas,
+ .num_areas = ARRAY_SIZE(r8a779f0_areas),
+};
diff --git a/drivers/soc/renesas/rcar-gen4-sysc.c b/drivers/soc/renesas/rcar-gen4-sysc.c
new file mode 100644
index 0000000..831162a
--- /dev/null
+++ b/drivers/soc/renesas/rcar-gen4-sysc.c
@@ -0,0 +1,376 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * R-Car Gen4 SYSC Power management support
+ *
+ * Copyright (C) 2021 Renesas Electronics Corp.
+ */
+
+#include <linux/bits.h>
+#include <linux/clk/renesas.h>
+#include <linux/delay.h>
+#include <linux/err.h>
+#include <linux/io.h>
+#include <linux/iopoll.h>
+#include <linux/kernel.h>
+#include <linux/mm.h>
+#include <linux/of_address.h>
+#include <linux/pm_domain.h>
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+#include <linux/types.h>
+
+#include "rcar-gen4-sysc.h"
+
+/* SYSC Common */
+#define SYSCSR 0x000 /* SYSC Status Register */
+#define SYSCPONSR(x) (0x800 + ((x) * 0x4)) /* Power-ON Status Register 0 */
+#define SYSCPOFFSR(x) (0x808 + ((x) * 0x4)) /* Power-OFF Status Register */
+#define SYSCISCR(x) (0x810 + ((x) * 0x4)) /* Interrupt Status/Clear Register */
+#define SYSCIER(x) (0x820 + ((x) * 0x4)) /* Interrupt Enable Register */
+#define SYSCIMR(x) (0x830 + ((x) * 0x4)) /* Interrupt Mask Register */
+
+/* Power Domain Registers */
+#define PDRSR(n) (0x1000 + ((n) * 0x40))
+#define PDRONCR(n) (0x1004 + ((n) * 0x40))
+#define PDROFFCR(n) (0x1008 + ((n) * 0x40))
+#define PDRESR(n) (0x100C + ((n) * 0x40))
+
+/* PWRON/PWROFF */
+#define PWRON_PWROFF BIT(0) /* Power-ON/OFF request */
+
+/* PDRESR */
+#define PDRESR_ERR BIT(0)
+
+/* PDRSR */
+#define PDRSR_OFF BIT(0) /* Power-OFF state */
+#define PDRSR_ON BIT(4) /* Power-ON state */
+#define PDRSR_OFF_STATE BIT(8) /* Processing Power-OFF sequence */
+#define PDRSR_ON_STATE BIT(12) /* Processing Power-ON sequence */
+
+#define SYSCSR_BUSY GENMASK(1, 0) /* All bit sets is not busy */
+
+#define SYSCSR_TIMEOUT 10000
+#define SYSCSR_DELAY_US 10
+
+#define PDRESR_RETRIES 1000
+#define PDRESR_DELAY_US 10
+
+#define SYSCISR_TIMEOUT 10000
+#define SYSCISR_DELAY_US 10
+
+#define RCAR_GEN4_PD_ALWAYS_ON 64
+#define NUM_DOMAINS_EACH_REG BITS_PER_TYPE(u32)
+
+static void __iomem *rcar_gen4_sysc_base;
+static DEFINE_SPINLOCK(rcar_gen4_sysc_lock); /* SMP CPUs + I/O devices */
+
+static int rcar_gen4_sysc_pwr_on_off(u8 pdr, bool on)
+{
+ unsigned int reg_offs;
+ u32 val;
+ int ret;
+
+ if (on)
+ reg_offs = PDRONCR(pdr);
+ else
+ reg_offs = PDROFFCR(pdr);
+
+ /* Wait until SYSC is ready to accept a power request */
+ ret = readl_poll_timeout_atomic(rcar_gen4_sysc_base + SYSCSR, val,
+ (val & SYSCSR_BUSY) == SYSCSR_BUSY,
+ SYSCSR_DELAY_US, SYSCSR_TIMEOUT);
+ if (ret < 0)
+ return -EAGAIN;
+
+ /* Submit power shutoff or power resume request */
+ iowrite32(PWRON_PWROFF, rcar_gen4_sysc_base + reg_offs);
+
+ return 0;
+}
+
+static int clear_irq_flags(unsigned int reg_idx, unsigned int isr_mask)
+{
+ u32 val;
+ int ret;
+
+ iowrite32(isr_mask, rcar_gen4_sysc_base + SYSCISCR(reg_idx));
+
+ ret = readl_poll_timeout_atomic(rcar_gen4_sysc_base + SYSCISCR(reg_idx),
+ val, !(val & isr_mask),
+ SYSCISR_DELAY_US, SYSCISR_TIMEOUT);
+ if (ret < 0) {
+ pr_err("\n %s : Can not clear IRQ flags in SYSCISCR", __func__);
+ return -EIO;
+ }
+
+ return 0;
+}
+
+static int rcar_gen4_sysc_power(u8 pdr, bool on)
+{
+ unsigned int isr_mask;
+ unsigned int reg_idx, bit_idx;
+ unsigned int status;
+ unsigned long flags;
+ int ret = 0;
+ u32 val;
+ int k;
+
+ spin_lock_irqsave(&rcar_gen4_sysc_lock, flags);
+
+ reg_idx = pdr / NUM_DOMAINS_EACH_REG;
+ bit_idx = pdr % NUM_DOMAINS_EACH_REG;
+
+ isr_mask = BIT(bit_idx);
+
+ /*
+ * The interrupt source needs to be enabled, but masked, to prevent the
+ * CPU from receiving it.
+ */
+ iowrite32(ioread32(rcar_gen4_sysc_base + SYSCIER(reg_idx)) | isr_mask,
+ rcar_gen4_sysc_base + SYSCIER(reg_idx));
+ iowrite32(ioread32(rcar_gen4_sysc_base + SYSCIMR(reg_idx)) | isr_mask,
+ rcar_gen4_sysc_base + SYSCIMR(reg_idx));
+
+ ret = clear_irq_flags(reg_idx, isr_mask);
+ if (ret)
+ goto out;
+
+ /* Submit power shutoff or resume request until it was accepted */
+ for (k = 0; k < PDRESR_RETRIES; k++) {
+ ret = rcar_gen4_sysc_pwr_on_off(pdr, on);
+ if (ret)
+ goto out;
+
+ status = ioread32(rcar_gen4_sysc_base + PDRESR(pdr));
+ if (!(status & PDRESR_ERR))
+ break;
+
+ udelay(PDRESR_DELAY_US);
+ }
+
+ if (k == PDRESR_RETRIES) {
+ ret = -EIO;
+ goto out;
+ }
+
+ /* Wait until the power shutoff or resume request has completed * */
+ ret = readl_poll_timeout_atomic(rcar_gen4_sysc_base + SYSCISCR(reg_idx),
+ val, (val & isr_mask),
+ SYSCISR_DELAY_US, SYSCISR_TIMEOUT);
+ if (ret < 0) {
+ ret = -EIO;
+ goto out;
+ }
+
+ /* Clear interrupt flags */
+ ret = clear_irq_flags(reg_idx, isr_mask);
+ if (ret)
+ goto out;
+
+ out:
+ spin_unlock_irqrestore(&rcar_gen4_sysc_lock, flags);
+
+ pr_debug("sysc power %s domain %d: %08x -> %d\n", on ? "on" : "off",
+ pdr, ioread32(rcar_gen4_sysc_base + SYSCISCR(reg_idx)), ret);
+ return ret;
+}
+
+static bool rcar_gen4_sysc_power_is_off(u8 pdr)
+{
+ unsigned int st;
+
+ st = ioread32(rcar_gen4_sysc_base + PDRSR(pdr));
+
+ if (st & PDRSR_OFF)
+ return true;
+
+ return false;
+}
+
+struct rcar_gen4_sysc_pd {
+ struct generic_pm_domain genpd;
+ u8 pdr;
+ unsigned int flags;
+ char name[];
+};
+
+static inline struct rcar_gen4_sysc_pd *to_rcar_gen4_pd(struct generic_pm_domain *d)
+{
+ return container_of(d, struct rcar_gen4_sysc_pd, genpd);
+}
+
+static int rcar_gen4_sysc_pd_power_off(struct generic_pm_domain *genpd)
+{
+ struct rcar_gen4_sysc_pd *pd = to_rcar_gen4_pd(genpd);
+
+ pr_debug("%s: %s\n", __func__, genpd->name);
+ return rcar_gen4_sysc_power(pd->pdr, false);
+}
+
+static int rcar_gen4_sysc_pd_power_on(struct generic_pm_domain *genpd)
+{
+ struct rcar_gen4_sysc_pd *pd = to_rcar_gen4_pd(genpd);
+
+ pr_debug("%s: %s\n", __func__, genpd->name);
+ return rcar_gen4_sysc_power(pd->pdr, true);
+}
+
+static int __init rcar_gen4_sysc_pd_setup(struct rcar_gen4_sysc_pd *pd)
+{
+ struct generic_pm_domain *genpd = &pd->genpd;
+ const char *name = pd->genpd.name;
+ int error;
+
+ if (pd->flags & PD_CPU) {
+ /*
+ * This domain contains a CPU core and therefore it should
+ * only be turned off if the CPU is not in use.
+ */
+ pr_debug("PM domain %s contains %s\n", name, "CPU");
+ genpd->flags |= GENPD_FLAG_ALWAYS_ON;
+ } else if (pd->flags & PD_SCU) {
+ /*
+ * This domain contains an SCU and cache-controller, and
+ * therefore it should only be turned off if the CPU cores are
+ * not in use.
+ */
+ pr_debug("PM domain %s contains %s\n", name, "SCU");
+ genpd->flags |= GENPD_FLAG_ALWAYS_ON;
+ } else if (pd->flags & PD_NO_CR) {
+ /*
+ * This domain cannot be turned off.
+ */
+ genpd->flags |= GENPD_FLAG_ALWAYS_ON;
+ }
+
+ if (!(pd->flags & (PD_CPU | PD_SCU))) {
+ /* Enable Clock Domain for I/O devices */
+ genpd->flags |= GENPD_FLAG_PM_CLK | GENPD_FLAG_ACTIVE_WAKEUP;
+ genpd->attach_dev = cpg_mssr_attach_dev;
+ genpd->detach_dev = cpg_mssr_detach_dev;
+ }
+
+ genpd->power_off = rcar_gen4_sysc_pd_power_off;
+ genpd->power_on = rcar_gen4_sysc_pd_power_on;
+
+ if (pd->flags & (PD_CPU | PD_NO_CR)) {
+ /* Skip CPUs (handled by SMP code) and areas without control */
+ pr_debug("%s: Not touching %s\n", __func__, genpd->name);
+ goto finalize;
+ }
+
+ if (!rcar_gen4_sysc_power_is_off(pd->pdr)) {
+ pr_debug("%s: %s is already powered\n", __func__, genpd->name);
+ goto finalize;
+ }
+
+ rcar_gen4_sysc_power(pd->pdr, true);
+
+finalize:
+ error = pm_genpd_init(genpd, &simple_qos_governor, false);
+ if (error)
+ pr_err("Failed to init PM domain %s: %d\n", name, error);
+
+ return error;
+}
+
+static const struct of_device_id rcar_gen4_sysc_matches[] __initconst = {
+#ifdef CONFIG_SYSC_R8A779A0
+ { .compatible = "renesas,r8a779a0-sysc", .data = &r8a779a0_sysc_info },
+#endif
+#ifdef CONFIG_SYSC_R8A779F0
+ { .compatible = "renesas,r8a779f0-sysc", .data = &r8a779f0_sysc_info },
+#endif
+ { /* sentinel */ }
+};
+
+struct rcar_gen4_pm_domains {
+ struct genpd_onecell_data onecell_data;
+ struct generic_pm_domain *domains[RCAR_GEN4_PD_ALWAYS_ON + 1];
+};
+
+static struct genpd_onecell_data *rcar_gen4_sysc_onecell_data;
+
+static int __init rcar_gen4_sysc_pd_init(void)
+{
+ const struct rcar_gen4_sysc_info *info;
+ const struct of_device_id *match;
+ struct rcar_gen4_pm_domains *domains;
+ struct device_node *np;
+ void __iomem *base;
+ unsigned int i;
+ int error;
+
+ np = of_find_matching_node_and_match(NULL, rcar_gen4_sysc_matches, &match);
+ if (!np)
+ return -ENODEV;
+
+ info = match->data;
+
+ base = of_iomap(np, 0);
+ if (!base) {
+ pr_warn("%pOF: Cannot map regs\n", np);
+ error = -ENOMEM;
+ goto out_put;
+ }
+
+ rcar_gen4_sysc_base = base;
+
+ domains = kzalloc(sizeof(*domains), GFP_KERNEL);
+ if (!domains) {
+ error = -ENOMEM;
+ goto out_put;
+ }
+
+ domains->onecell_data.domains = domains->domains;
+ domains->onecell_data.num_domains = ARRAY_SIZE(domains->domains);
+ rcar_gen4_sysc_onecell_data = &domains->onecell_data;
+
+ for (i = 0; i < info->num_areas; i++) {
+ const struct rcar_gen4_sysc_area *area = &info->areas[i];
+ struct rcar_gen4_sysc_pd *pd;
+ size_t n;
+
+ if (!area->name) {
+ /* Skip NULLified area */
+ continue;
+ }
+
+ n = strlen(area->name) + 1;
+ pd = kzalloc(sizeof(*pd) + n, GFP_KERNEL);
+ if (!pd) {
+ error = -ENOMEM;
+ goto out_put;
+ }
+
+ memcpy(pd->name, area->name, n);
+ pd->genpd.name = pd->name;
+ pd->pdr = area->pdr;
+ pd->flags = area->flags;
+
+ error = rcar_gen4_sysc_pd_setup(pd);
+ if (error)
+ goto out_put;
+
+ domains->domains[area->pdr] = &pd->genpd;
+
+ if (area->parent < 0)
+ continue;
+
+ error = pm_genpd_add_subdomain(domains->domains[area->parent],
+ &pd->genpd);
+ if (error) {
+ pr_warn("Failed to add PM subdomain %s to parent %u\n",
+ area->name, area->parent);
+ goto out_put;
+ }
+ }
+
+ error = of_genpd_add_provider_onecell(np, &domains->onecell_data);
+
+out_put:
+ of_node_put(np);
+ return error;
+}
+early_initcall(rcar_gen4_sysc_pd_init);
diff --git a/drivers/soc/renesas/rcar-gen4-sysc.h b/drivers/soc/renesas/rcar-gen4-sysc.h
new file mode 100644
index 0000000..0e0bd10
--- /dev/null
+++ b/drivers/soc/renesas/rcar-gen4-sysc.h
@@ -0,0 +1,43 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * R-Car Gen4 System Controller
+ *
+ * Copyright (C) 2021 Renesas Electronics Corp.
+ */
+#ifndef __SOC_RENESAS_RCAR_GEN4_SYSC_H__
+#define __SOC_RENESAS_RCAR_GEN4_SYSC_H__
+
+#include <linux/types.h>
+
+/*
+ * Power Domain flags
+ */
+#define PD_CPU BIT(0) /* Area contains main CPU core */
+#define PD_SCU BIT(1) /* Area contains SCU and L2 cache */
+#define PD_NO_CR BIT(2) /* Area lacks PWR{ON,OFF}CR registers */
+
+#define PD_CPU_NOCR (PD_CPU | PD_NO_CR) /* CPU area lacks CR */
+#define PD_ALWAYS_ON PD_NO_CR /* Always-on area */
+
+/*
+ * Description of a Power Area
+ */
+struct rcar_gen4_sysc_area {
+ const char *name;
+ u8 pdr; /* PDRn */
+ int parent; /* -1 if none */
+ unsigned int flags; /* See PD_* */
+};
+
+/*
+ * SoC-specific Power Area Description
+ */
+struct rcar_gen4_sysc_info {
+ const struct rcar_gen4_sysc_area *areas;
+ unsigned int num_areas;
+};
+
+extern const struct rcar_gen4_sysc_info r8a779a0_sysc_info;
+extern const struct rcar_gen4_sysc_info r8a779f0_sysc_info;
+
+#endif /* __SOC_RENESAS_RCAR_GEN4_SYSC_H__ */
diff --git a/drivers/soc/renesas/rcar-rst.c b/drivers/soc/renesas/rcar-rst.c
index 8a1e402..4d293eb 100644
--- a/drivers/soc/renesas/rcar-rst.c
+++ b/drivers/soc/renesas/rcar-rst.c
@@ -13,15 +13,43 @@
#define WDTRSTCR_RESET 0xA55A0002
#define WDTRSTCR 0x0054
+#define CR7BAR 0x0070
+#define CR7BAREN BIT(4)
+#define CR7BAR_MASK 0xFFFC0000
+
+static void __iomem *rcar_rst_base;
+static u32 saved_mode __initdata;
+static int (*rcar_rst_set_rproc_boot_addr_func)(u64 boot_addr);
+
static int rcar_rst_enable_wdt_reset(void __iomem *base)
{
iowrite32(WDTRSTCR_RESET, base + WDTRSTCR);
return 0;
}
+/*
+ * Most of the R-Car Gen3 SoCs have an ARM Realtime Core.
+ * Firmware boot address has to be set in CR7BAR before
+ * starting the realtime core.
+ * Boot address must be aligned on a 256k boundary.
+ */
+static int rcar_rst_set_gen3_rproc_boot_addr(u64 boot_addr)
+{
+ if (boot_addr & ~(u64)CR7BAR_MASK) {
+ pr_err("Invalid boot address got %llx\n", boot_addr);
+ return -EINVAL;
+ }
+
+ iowrite32(boot_addr, rcar_rst_base + CR7BAR);
+ iowrite32(boot_addr | CR7BAREN, rcar_rst_base + CR7BAR);
+
+ return 0;
+}
+
struct rst_config {
unsigned int modemr; /* Mode Monitoring Register Offset */
int (*configure)(void __iomem *base); /* Platform specific config */
+ int (*set_rproc_boot_addr)(u64 boot_addr);
};
static const struct rst_config rcar_rst_gen1 __initconst = {
@@ -35,9 +63,10 @@ static const struct rst_config rcar_rst_gen2 __initconst = {
static const struct rst_config rcar_rst_gen3 __initconst = {
.modemr = 0x60,
+ .set_rproc_boot_addr = rcar_rst_set_gen3_rproc_boot_addr,
};
-static const struct rst_config rcar_rst_r8a779a0 __initconst = {
+static const struct rst_config rcar_rst_gen4 __initconst = {
.modemr = 0x00, /* MODEMR0 and it has CPG related bits */
};
@@ -71,14 +100,12 @@ static const struct of_device_id rcar_rst_matches[] __initconst = {
{ .compatible = "renesas,r8a77980-rst", .data = &rcar_rst_gen3 },
{ .compatible = "renesas,r8a77990-rst", .data = &rcar_rst_gen3 },
{ .compatible = "renesas,r8a77995-rst", .data = &rcar_rst_gen3 },
- /* R-Car V3U */
- { .compatible = "renesas,r8a779a0-rst", .data = &rcar_rst_r8a779a0 },
+ /* R-Car Gen4 */
+ { .compatible = "renesas,r8a779a0-rst", .data = &rcar_rst_gen4 },
+ { .compatible = "renesas,r8a779f0-rst", .data = &rcar_rst_gen4 },
{ /* sentinel */ }
};
-static void __iomem *rcar_rst_base __initdata;
-static u32 saved_mode __initdata;
-
static int __init rcar_rst_init(void)
{
const struct of_device_id *match;
@@ -100,6 +127,8 @@ static int __init rcar_rst_init(void)
rcar_rst_base = base;
cfg = match->data;
+ rcar_rst_set_rproc_boot_addr_func = cfg->set_rproc_boot_addr;
+
saved_mode = ioread32(base + cfg->modemr);
if (cfg->configure) {
error = cfg->configure(base);
@@ -130,3 +159,12 @@ int __init rcar_rst_read_mode_pins(u32 *mode)
*mode = saved_mode;
return 0;
}
+
+int rcar_rst_set_rproc_boot_addr(u64 boot_addr)
+{
+ if (!rcar_rst_set_rproc_boot_addr_func)
+ return -EIO;
+
+ return rcar_rst_set_rproc_boot_addr_func(boot_addr);
+}
+EXPORT_SYMBOL_GPL(rcar_rst_set_rproc_boot_addr);
diff --git a/drivers/soc/renesas/renesas-soc.c b/drivers/soc/renesas/renesas-soc.c
index 7961b0b..62540ff 100644
--- a/drivers/soc/renesas/renesas-soc.c
+++ b/drivers/soc/renesas/renesas-soc.c
@@ -33,6 +33,10 @@ static const struct renesas_family fam_rcar_gen3 __initconst __maybe_unused = {
.reg = 0xfff00044, /* PRR (Product Register) */
};
+static const struct renesas_family fam_rcar_gen4 __initconst __maybe_unused = {
+ .name = "R-Car Gen4",
+};
+
static const struct renesas_family fam_rmobile __initconst __maybe_unused = {
.name = "R-Mobile",
.reg = 0xe600101c, /* CCCR (Common Chip Code Register) */
@@ -214,6 +218,11 @@ static const struct renesas_soc soc_rcar_v3u __initconst __maybe_unused = {
.id = 0x59,
};
+static const struct renesas_soc soc_rcar_s4 __initconst __maybe_unused = {
+ .family = &fam_rcar_gen4,
+ .id = 0x5a,
+};
+
static const struct renesas_soc soc_shmobile_ag5 __initconst __maybe_unused = {
.family = &fam_shmobile,
.id = 0x37,
@@ -319,6 +328,9 @@ static const struct of_device_id renesas_socs[] __initconst = {
#ifdef CONFIG_ARCH_R8A779A0
{ .compatible = "renesas,r8a779a0", .data = &soc_rcar_v3u },
#endif
+#ifdef CONFIG_ARCH_R8A779F0
+ { .compatible = "renesas,r8a779f0", .data = &soc_rcar_s4 },
+#endif
#if defined(CONFIG_ARCH_R9A07G044)
{ .compatible = "renesas,r9a07g044", .data = &soc_rz_g2l },
#endif
@@ -328,94 +340,92 @@ static const struct of_device_id renesas_socs[] __initconst = {
{ /* sentinel */ }
};
+struct renesas_id {
+ unsigned int offset;
+ u32 mask;
+};
+
+static const struct renesas_id id_bsid __initconst = {
+ .offset = 0,
+ .mask = 0xff0000,
+ /*
+ * TODO: Upper 4 bits of BSID are for chip version, but the format is
+ * not known at this time so we don't know how to specify eshi and eslo
+ */
+};
+
+static const struct renesas_id id_rzg2l __initconst = {
+ .offset = 0xa04,
+ .mask = 0xfffffff,
+};
+
+static const struct renesas_id id_prr __initconst = {
+ .offset = 0,
+ .mask = 0xff00,
+};
+
+static const struct of_device_id renesas_ids[] __initconst = {
+ { .compatible = "renesas,bsid", .data = &id_bsid },
+ { .compatible = "renesas,r9a07g044-sysc", .data = &id_rzg2l },
+ { .compatible = "renesas,prr", .data = &id_prr },
+ { /* sentinel */ }
+};
+
static int __init renesas_soc_init(void)
{
struct soc_device_attribute *soc_dev_attr;
+ unsigned int product, eshi = 0, eslo;
const struct renesas_family *family;
const struct of_device_id *match;
const struct renesas_soc *soc;
+ const struct renesas_id *id;
void __iomem *chipid = NULL;
struct soc_device *soc_dev;
struct device_node *np;
- unsigned int product, eshi = 0, eslo;
+ const char *soc_id;
match = of_match_node(renesas_socs, of_root);
if (!match)
return -ENODEV;
+ soc_id = strchr(match->compatible, ',') + 1;
soc = match->data;
family = soc->family;
- np = of_find_compatible_node(NULL, NULL, "renesas,bsid");
+ np = of_find_matching_node_and_match(NULL, renesas_ids, &match);
if (np) {
- chipid = of_iomap(np, 0);
- of_node_put(np);
-
- if (chipid) {
- product = readl(chipid);
- iounmap(chipid);
-
- if (soc->id && ((product >> 16) & 0xff) != soc->id) {
- pr_warn("SoC mismatch (product = 0x%x)\n",
- product);
- return -ENODEV;
- }
- }
-
- /*
- * TODO: Upper 4 bits of BSID are for chip version, but the
- * format is not known at this time so we don't know how to
- * specify eshi and eslo
- */
-
- goto done;
- }
-
- np = of_find_compatible_node(NULL, NULL, "renesas,r9a07g044-sysc");
- if (np) {
- chipid = of_iomap(np, 0);
- of_node_put(np);
-
- if (chipid) {
- product = readl(chipid + 0x0a04);
- iounmap(chipid);
-
- if (soc->id && (product & 0xfffffff) != soc->id) {
- pr_warn("SoC mismatch (product = 0x%x)\n",
- product);
- return -ENODEV;
- }
- }
-
- goto done;
- }
-
- /* Try PRR first, then hardcoded fallback */
- np = of_find_compatible_node(NULL, NULL, "renesas,prr");
- if (np) {
+ id = match->data;
chipid = of_iomap(np, 0);
of_node_put(np);
} else if (soc->id && family->reg) {
+ /* Try hardcoded CCCR/PRR fallback */
+ id = &id_prr;
chipid = ioremap(family->reg, 4);
}
+
if (chipid) {
- product = readl(chipid);
+ product = readl(chipid + id->offset);
iounmap(chipid);
- /* R-Car M3-W ES1.1 incorrectly identifies as ES2.0 */
- if ((product & 0x7fff) == 0x5210)
- product ^= 0x11;
- /* R-Car M3-W ES1.3 incorrectly identifies as ES2.1 */
- if ((product & 0x7fff) == 0x5211)
- product ^= 0x12;
- if (soc->id && ((product >> 8) & 0xff) != soc->id) {
+
+ if (id == &id_prr) {
+ /* R-Car M3-W ES1.1 incorrectly identifies as ES2.0 */
+ if ((product & 0x7fff) == 0x5210)
+ product ^= 0x11;
+ /* R-Car M3-W ES1.3 incorrectly identifies as ES2.1 */
+ if ((product & 0x7fff) == 0x5211)
+ product ^= 0x12;
+
+ eshi = ((product >> 4) & 0x0f) + 1;
+ eslo = product & 0xf;
+ }
+
+ if (soc->id &&
+ ((product & id->mask) >> __ffs(id->mask)) != soc->id) {
pr_warn("SoC mismatch (product = 0x%x)\n", product);
return -ENODEV;
}
- eshi = ((product >> 4) & 0x0f) + 1;
- eslo = product & 0xf;
}
-done:
soc_dev_attr = kzalloc(sizeof(*soc_dev_attr), GFP_KERNEL);
if (!soc_dev_attr)
return -ENOMEM;
@@ -425,8 +435,7 @@ static int __init renesas_soc_init(void)
of_node_put(np);
soc_dev_attr->family = kstrdup_const(family->name, GFP_KERNEL);
- soc_dev_attr->soc_id = kstrdup_const(strchr(match->compatible, ',') + 1,
- GFP_KERNEL);
+ soc_dev_attr->soc_id = kstrdup_const(soc_id, GFP_KERNEL);
if (eshi)
soc_dev_attr->revision = kasprintf(GFP_KERNEL, "ES%u.%u", eshi,
eslo);
diff --git a/drivers/soc/samsung/Kconfig b/drivers/soc/samsung/Kconfig
index e2cedef..a9f8b22 100644
--- a/drivers/soc/samsung/Kconfig
+++ b/drivers/soc/samsung/Kconfig
@@ -23,6 +23,20 @@
Support for Samsung Exynos SoC ChipID and Adaptive Supply Voltage.
This driver can also be built as module (exynos_chipid).
+config EXYNOS_USI
+ tristate "Exynos USI (Universal Serial Interface) driver"
+ default ARCH_EXYNOS && ARM64
+ depends on ARCH_EXYNOS || COMPILE_TEST
+ select MFD_SYSCON
+ help
+ Enable support for USI block. USI (Universal Serial Interface) is an
+ IP-core found in modern Samsung Exynos SoCs, like Exynos850 and
+ ExynosAutoV0. USI block can be configured to provide one of the
+ following serial protocols: UART, SPI or High Speed I2C.
+
+ This driver allows one to configure USI for desired protocol, which
+ is usually done in USI node in Device Tree.
+
config EXYNOS_PMU
bool "Exynos PMU controller driver" if COMPILE_TEST
depends on ARCH_EXYNOS || ((ARM || ARM64) && COMPILE_TEST)
diff --git a/drivers/soc/samsung/Makefile b/drivers/soc/samsung/Makefile
index 2ae4bea..9f59d19 100644
--- a/drivers/soc/samsung/Makefile
+++ b/drivers/soc/samsung/Makefile
@@ -4,6 +4,8 @@
obj-$(CONFIG_EXYNOS_CHIPID) += exynos_chipid.o
exynos_chipid-y += exynos-chipid.o exynos-asv.o
+obj-$(CONFIG_EXYNOS_USI) += exynos-usi.o
+
obj-$(CONFIG_EXYNOS_PMU) += exynos-pmu.o
obj-$(CONFIG_EXYNOS_PMU_ARM_DRIVERS) += exynos3250-pmu.o exynos4-pmu.o \
diff --git a/drivers/soc/samsung/exynos-chipid.c b/drivers/soc/samsung/exynos-chipid.c
index a28053e..2746d05 100644
--- a/drivers/soc/samsung/exynos-chipid.c
+++ b/drivers/soc/samsung/exynos-chipid.c
@@ -42,6 +42,7 @@ static const struct exynos_soc_id {
unsigned int id;
} soc_ids[] = {
/* List ordered by SoC name */
+ /* Compatible with: samsung,exynos4210-chipid */
{ "EXYNOS3250", 0xE3472000 },
{ "EXYNOS4210", 0x43200000 }, /* EVT0 revision */
{ "EXYNOS4210", 0x43210000 },
@@ -55,6 +56,8 @@ static const struct exynos_soc_id {
{ "EXYNOS5440", 0xE5440000 },
{ "EXYNOS5800", 0xE5422000 },
{ "EXYNOS7420", 0xE7420000 },
+ /* Compatible with: samsung,exynos850-chipid */
+ { "EXYNOS7885", 0xE7885000 },
{ "EXYNOS850", 0xE3830000 },
{ "EXYNOSAUTOV9", 0xAAA80000 },
};
diff --git a/drivers/soc/samsung/exynos-pmu.c b/drivers/soc/samsung/exynos-pmu.c
index a18c93a..732c86c 100644
--- a/drivers/soc/samsung/exynos-pmu.c
+++ b/drivers/soc/samsung/exynos-pmu.c
@@ -94,6 +94,8 @@ static const struct of_device_id exynos_pmu_of_device_ids[] = {
.compatible = "samsung,exynos5433-pmu",
}, {
.compatible = "samsung,exynos7-pmu",
+ }, {
+ .compatible = "samsung,exynos850-pmu",
},
{ /*sentinel*/ },
};
diff --git a/drivers/soc/samsung/exynos-usi.c b/drivers/soc/samsung/exynos-usi.c
new file mode 100644
index 0000000..1143526
--- /dev/null
+++ b/drivers/soc/samsung/exynos-usi.c
@@ -0,0 +1,285 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2021 Linaro Ltd.
+ * Author: Sam Protsenko <semen.protsenko@linaro.org>
+ *
+ * Samsung Exynos USI driver (Universal Serial Interface).
+ */
+
+#include <linux/clk.h>
+#include <linux/mfd/syscon.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/of_platform.h>
+#include <linux/platform_device.h>
+#include <linux/regmap.h>
+
+#include <dt-bindings/soc/samsung,exynos-usi.h>
+
+/* USIv2: System Register: SW_CONF register bits */
+#define USI_V2_SW_CONF_NONE 0x0
+#define USI_V2_SW_CONF_UART BIT(0)
+#define USI_V2_SW_CONF_SPI BIT(1)
+#define USI_V2_SW_CONF_I2C BIT(2)
+#define USI_V2_SW_CONF_MASK (USI_V2_SW_CONF_UART | USI_V2_SW_CONF_SPI | \
+ USI_V2_SW_CONF_I2C)
+
+/* USIv2: USI register offsets */
+#define USI_CON 0x04
+#define USI_OPTION 0x08
+
+/* USIv2: USI register bits */
+#define USI_CON_RESET BIT(0)
+#define USI_OPTION_CLKREQ_ON BIT(1)
+#define USI_OPTION_CLKSTOP_ON BIT(2)
+
+enum exynos_usi_ver {
+ USI_VER2 = 2,
+};
+
+struct exynos_usi_variant {
+ enum exynos_usi_ver ver; /* USI IP-core version */
+ unsigned int sw_conf_mask; /* SW_CONF mask for all protocols */
+ size_t min_mode; /* first index in exynos_usi_modes[] */
+ size_t max_mode; /* last index in exynos_usi_modes[] */
+ size_t num_clks; /* number of clocks to assert */
+ const char * const *clk_names; /* clock names to assert */
+};
+
+struct exynos_usi {
+ struct device *dev;
+ void __iomem *regs; /* USI register map */
+ struct clk_bulk_data *clks; /* USI clocks */
+
+ size_t mode; /* current USI SW_CONF mode index */
+ bool clkreq_on; /* always provide clock to IP */
+
+ /* System Register */
+ struct regmap *sysreg; /* System Register map */
+ unsigned int sw_conf; /* SW_CONF register offset in sysreg */
+
+ const struct exynos_usi_variant *data;
+};
+
+struct exynos_usi_mode {
+ const char *name; /* mode name */
+ unsigned int val; /* mode register value */
+};
+
+static const struct exynos_usi_mode exynos_usi_modes[] = {
+ [USI_V2_NONE] = { .name = "none", .val = USI_V2_SW_CONF_NONE },
+ [USI_V2_UART] = { .name = "uart", .val = USI_V2_SW_CONF_UART },
+ [USI_V2_SPI] = { .name = "spi", .val = USI_V2_SW_CONF_SPI },
+ [USI_V2_I2C] = { .name = "i2c", .val = USI_V2_SW_CONF_I2C },
+};
+
+static const char * const exynos850_usi_clk_names[] = { "pclk", "ipclk" };
+static const struct exynos_usi_variant exynos850_usi_data = {
+ .ver = USI_VER2,
+ .sw_conf_mask = USI_V2_SW_CONF_MASK,
+ .min_mode = USI_V2_NONE,
+ .max_mode = USI_V2_I2C,
+ .num_clks = ARRAY_SIZE(exynos850_usi_clk_names),
+ .clk_names = exynos850_usi_clk_names,
+};
+
+static const struct of_device_id exynos_usi_dt_match[] = {
+ {
+ .compatible = "samsung,exynos850-usi",
+ .data = &exynos850_usi_data,
+ },
+ { } /* sentinel */
+};
+MODULE_DEVICE_TABLE(of, exynos_usi_dt_match);
+
+/**
+ * exynos_usi_set_sw_conf - Set USI block configuration mode
+ * @usi: USI driver object
+ * @mode: Mode index
+ *
+ * Select underlying serial protocol (UART/SPI/I2C) in USI IP-core.
+ *
+ * Return: 0 on success, or negative error code on failure.
+ */
+static int exynos_usi_set_sw_conf(struct exynos_usi *usi, size_t mode)
+{
+ unsigned int val;
+ int ret;
+
+ if (mode < usi->data->min_mode || mode > usi->data->max_mode)
+ return -EINVAL;
+
+ val = exynos_usi_modes[mode].val;
+ ret = regmap_update_bits(usi->sysreg, usi->sw_conf,
+ usi->data->sw_conf_mask, val);
+ if (ret)
+ return ret;
+
+ usi->mode = mode;
+ dev_dbg(usi->dev, "protocol: %s\n", exynos_usi_modes[usi->mode].name);
+
+ return 0;
+}
+
+/**
+ * exynos_usi_enable - Initialize USI block
+ * @usi: USI driver object
+ *
+ * USI IP-core start state is "reset" (on startup and after CPU resume). This
+ * routine enables the USI block by clearing the reset flag. It also configures
+ * HWACG behavior (needed e.g. for UART Rx). It should be performed before
+ * underlying protocol becomes functional.
+ *
+ * Return: 0 on success, or negative error code on failure.
+ */
+static int exynos_usi_enable(const struct exynos_usi *usi)
+{
+ u32 val;
+ int ret;
+
+ ret = clk_bulk_prepare_enable(usi->data->num_clks, usi->clks);
+ if (ret)
+ return ret;
+
+ /* Enable USI block */
+ val = readl(usi->regs + USI_CON);
+ val &= ~USI_CON_RESET;
+ writel(val, usi->regs + USI_CON);
+ udelay(1);
+
+ /* Continuously provide the clock to USI IP w/o gating */
+ if (usi->clkreq_on) {
+ val = readl(usi->regs + USI_OPTION);
+ val &= ~USI_OPTION_CLKSTOP_ON;
+ val |= USI_OPTION_CLKREQ_ON;
+ writel(val, usi->regs + USI_OPTION);
+ }
+
+ clk_bulk_disable_unprepare(usi->data->num_clks, usi->clks);
+
+ return ret;
+}
+
+static int exynos_usi_configure(struct exynos_usi *usi)
+{
+ int ret;
+
+ ret = exynos_usi_set_sw_conf(usi, usi->mode);
+ if (ret)
+ return ret;
+
+ if (usi->data->ver == USI_VER2)
+ return exynos_usi_enable(usi);
+
+ return 0;
+}
+
+static int exynos_usi_parse_dt(struct device_node *np, struct exynos_usi *usi)
+{
+ int ret;
+ u32 mode;
+
+ ret = of_property_read_u32(np, "samsung,mode", &mode);
+ if (ret)
+ return ret;
+ if (mode < usi->data->min_mode || mode > usi->data->max_mode)
+ return -EINVAL;
+ usi->mode = mode;
+
+ usi->sysreg = syscon_regmap_lookup_by_phandle(np, "samsung,sysreg");
+ if (IS_ERR(usi->sysreg))
+ return PTR_ERR(usi->sysreg);
+
+ ret = of_property_read_u32_index(np, "samsung,sysreg", 1,
+ &usi->sw_conf);
+ if (ret)
+ return ret;
+
+ usi->clkreq_on = of_property_read_bool(np, "samsung,clkreq-on");
+
+ return 0;
+}
+
+static int exynos_usi_get_clocks(struct exynos_usi *usi)
+{
+ const size_t num = usi->data->num_clks;
+ struct device *dev = usi->dev;
+ size_t i;
+
+ if (num == 0)
+ return 0;
+
+ usi->clks = devm_kcalloc(dev, num, sizeof(*usi->clks), GFP_KERNEL);
+ if (!usi->clks)
+ return -ENOMEM;
+
+ for (i = 0; i < num; ++i)
+ usi->clks[i].id = usi->data->clk_names[i];
+
+ return devm_clk_bulk_get(dev, num, usi->clks);
+}
+
+static int exynos_usi_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct device_node *np = dev->of_node;
+ struct exynos_usi *usi;
+ int ret;
+
+ usi = devm_kzalloc(dev, sizeof(*usi), GFP_KERNEL);
+ if (!usi)
+ return -ENOMEM;
+
+ usi->dev = dev;
+ platform_set_drvdata(pdev, usi);
+
+ usi->data = of_device_get_match_data(dev);
+ if (!usi->data)
+ return -EINVAL;
+
+ ret = exynos_usi_parse_dt(np, usi);
+ if (ret)
+ return ret;
+
+ ret = exynos_usi_get_clocks(usi);
+ if (ret)
+ return ret;
+
+ if (usi->data->ver == USI_VER2) {
+ usi->regs = devm_platform_ioremap_resource(pdev, 0);
+ if (IS_ERR(usi->regs))
+ return PTR_ERR(usi->regs);
+ }
+
+ ret = exynos_usi_configure(usi);
+ if (ret)
+ return ret;
+
+ /* Make it possible to embed protocol nodes into USI np */
+ return of_platform_populate(np, NULL, NULL, dev);
+}
+
+static int __maybe_unused exynos_usi_resume_noirq(struct device *dev)
+{
+ struct exynos_usi *usi = dev_get_drvdata(dev);
+
+ return exynos_usi_configure(usi);
+}
+
+static const struct dev_pm_ops exynos_usi_pm = {
+ SET_NOIRQ_SYSTEM_SLEEP_PM_OPS(NULL, exynos_usi_resume_noirq)
+};
+
+static struct platform_driver exynos_usi_driver = {
+ .driver = {
+ .name = "exynos-usi",
+ .pm = &exynos_usi_pm,
+ .of_match_table = exynos_usi_dt_match,
+ },
+ .probe = exynos_usi_probe,
+};
+module_platform_driver(exynos_usi_driver);
+
+MODULE_DESCRIPTION("Samsung USI driver");
+MODULE_AUTHOR("Sam Protsenko <semen.protsenko@linaro.org>");
+MODULE_LICENSE("GPL");
diff --git a/drivers/soc/tegra/common.c b/drivers/soc/tegra/common.c
index cd33e99..32c346b 100644
--- a/drivers/soc/tegra/common.c
+++ b/drivers/soc/tegra/common.c
@@ -10,6 +10,7 @@
#include <linux/export.h>
#include <linux/of.h>
#include <linux/pm_opp.h>
+#include <linux/pm_runtime.h>
#include <soc/tegra/common.h>
#include <soc/tegra/fuse.h>
@@ -43,6 +44,7 @@ static int tegra_core_dev_init_opp_state(struct device *dev)
{
unsigned long rate;
struct clk *clk;
+ bool rpm_enabled;
int err;
clk = devm_clk_get(dev, NULL);
@@ -57,8 +59,31 @@ static int tegra_core_dev_init_opp_state(struct device *dev)
return -EINVAL;
}
+ /*
+ * Runtime PM of the device must be enabled in order to set up
+ * GENPD's performance properly because GENPD core checks whether
+ * device is suspended and this check doesn't work while RPM is
+ * disabled. This makes sure the OPP vote below gets cached in
+ * GENPD for the device. Instead, the vote is done the next time
+ * the device gets runtime resumed.
+ */
+ rpm_enabled = pm_runtime_enabled(dev);
+ if (!rpm_enabled)
+ pm_runtime_enable(dev);
+
+ /* should never happen in practice */
+ if (!pm_runtime_enabled(dev)) {
+ dev_WARN(dev, "failed to enable runtime PM\n");
+ pm_runtime_disable(dev);
+ return -EINVAL;
+ }
+
/* first dummy rate-setting initializes voltage vote */
err = dev_pm_opp_set_rate(dev, rate);
+
+ if (!rpm_enabled)
+ pm_runtime_disable(dev);
+
if (err) {
dev_err(dev, "failed to initialize OPP clock: %d\n", err);
return err;
@@ -111,9 +136,7 @@ int devm_tegra_core_dev_init_opp_table(struct device *dev,
*/
err = devm_pm_opp_of_add_table(dev);
if (err) {
- if (err == -ENODEV)
- dev_err_once(dev, "OPP table not found, please update device-tree\n");
- else
+ if (err != -ENODEV)
dev_err(dev, "failed to add OPP table: %d\n", err);
return err;
diff --git a/drivers/soc/tegra/fuse/fuse-tegra.c b/drivers/soc/tegra/fuse/fuse-tegra.c
index e714ed3..913103e 100644
--- a/drivers/soc/tegra/fuse/fuse-tegra.c
+++ b/drivers/soc/tegra/fuse/fuse-tegra.c
@@ -14,6 +14,7 @@
#include <linux/of_address.h>
#include <linux/platform_device.h>
#include <linux/pm_runtime.h>
+#include <linux/reset.h>
#include <linux/slab.h>
#include <linux/sys_soc.h>
@@ -181,6 +182,12 @@ static const struct nvmem_cell_info tegra_fuse_cells[] = {
},
};
+static void tegra_fuse_restore(void *base)
+{
+ fuse->clk = NULL;
+ fuse->base = base;
+}
+
static int tegra_fuse_probe(struct platform_device *pdev)
{
void __iomem *base = fuse->base;
@@ -188,13 +195,16 @@ static int tegra_fuse_probe(struct platform_device *pdev)
struct resource *res;
int err;
+ err = devm_add_action(&pdev->dev, tegra_fuse_restore, base);
+ if (err)
+ return err;
+
/* take over the memory region from the early initialization */
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
fuse->phys = res->start;
fuse->base = devm_ioremap_resource(&pdev->dev, res);
if (IS_ERR(fuse->base)) {
err = PTR_ERR(fuse->base);
- fuse->base = base;
return err;
}
@@ -204,19 +214,20 @@ static int tegra_fuse_probe(struct platform_device *pdev)
dev_err(&pdev->dev, "failed to get FUSE clock: %ld",
PTR_ERR(fuse->clk));
- fuse->base = base;
return PTR_ERR(fuse->clk);
}
platform_set_drvdata(pdev, fuse);
fuse->dev = &pdev->dev;
- pm_runtime_enable(&pdev->dev);
+ err = devm_pm_runtime_enable(&pdev->dev);
+ if (err)
+ return err;
if (fuse->soc->probe) {
err = fuse->soc->probe(fuse);
if (err < 0)
- goto restore;
+ return err;
}
memset(&nvmem, 0, sizeof(nvmem));
@@ -240,19 +251,37 @@ static int tegra_fuse_probe(struct platform_device *pdev)
err = PTR_ERR(fuse->nvmem);
dev_err(&pdev->dev, "failed to register NVMEM device: %d\n",
err);
- goto restore;
+ return err;
+ }
+
+ fuse->rst = devm_reset_control_get_optional(&pdev->dev, "fuse");
+ if (IS_ERR(fuse->rst)) {
+ err = PTR_ERR(fuse->rst);
+ dev_err(&pdev->dev, "failed to get FUSE reset: %pe\n",
+ fuse->rst);
+ return err;
+ }
+
+ /*
+ * FUSE clock is enabled at a boot time, hence this resume/suspend
+ * disables the clock besides the h/w resetting.
+ */
+ err = pm_runtime_resume_and_get(&pdev->dev);
+ if (err)
+ return err;
+
+ err = reset_control_reset(fuse->rst);
+ pm_runtime_put(&pdev->dev);
+
+ if (err < 0) {
+ dev_err(&pdev->dev, "failed to reset FUSE: %d\n", err);
+ return err;
}
/* release the early I/O memory mapping */
iounmap(base);
return 0;
-
-restore:
- fuse->clk = NULL;
- fuse->base = base;
- pm_runtime_disable(&pdev->dev);
- return err;
}
static int __maybe_unused tegra_fuse_runtime_resume(struct device *dev)
diff --git a/drivers/soc/tegra/fuse/fuse-tegra20.c b/drivers/soc/tegra/fuse/fuse-tegra20.c
index 8ec9fc5..12503f5 100644
--- a/drivers/soc/tegra/fuse/fuse-tegra20.c
+++ b/drivers/soc/tegra/fuse/fuse-tegra20.c
@@ -94,9 +94,28 @@ static bool dma_filter(struct dma_chan *chan, void *filter_param)
return of_device_is_compatible(np, "nvidia,tegra20-apbdma");
}
+static void tegra20_fuse_release_channel(void *data)
+{
+ struct tegra_fuse *fuse = data;
+
+ dma_release_channel(fuse->apbdma.chan);
+ fuse->apbdma.chan = NULL;
+}
+
+static void tegra20_fuse_free_coherent(void *data)
+{
+ struct tegra_fuse *fuse = data;
+
+ dma_free_coherent(fuse->dev, sizeof(u32), fuse->apbdma.virt,
+ fuse->apbdma.phys);
+ fuse->apbdma.virt = NULL;
+ fuse->apbdma.phys = 0x0;
+}
+
static int tegra20_fuse_probe(struct tegra_fuse *fuse)
{
dma_cap_mask_t mask;
+ int err;
dma_cap_zero(mask);
dma_cap_set(DMA_SLAVE, mask);
@@ -105,13 +124,21 @@ static int tegra20_fuse_probe(struct tegra_fuse *fuse)
if (!fuse->apbdma.chan)
return -EPROBE_DEFER;
+ err = devm_add_action_or_reset(fuse->dev, tegra20_fuse_release_channel,
+ fuse);
+ if (err)
+ return err;
+
fuse->apbdma.virt = dma_alloc_coherent(fuse->dev, sizeof(u32),
&fuse->apbdma.phys,
GFP_KERNEL);
- if (!fuse->apbdma.virt) {
- dma_release_channel(fuse->apbdma.chan);
+ if (!fuse->apbdma.virt)
return -ENOMEM;
- }
+
+ err = devm_add_action_or_reset(fuse->dev, tegra20_fuse_free_coherent,
+ fuse);
+ if (err)
+ return err;
fuse->apbdma.config.src_addr_width = DMA_SLAVE_BUSWIDTH_4_BYTES;
fuse->apbdma.config.dst_addr_width = DMA_SLAVE_BUSWIDTH_4_BYTES;
diff --git a/drivers/soc/tegra/fuse/fuse.h b/drivers/soc/tegra/fuse/fuse.h
index ecff0c0..2bb1f9d 100644
--- a/drivers/soc/tegra/fuse/fuse.h
+++ b/drivers/soc/tegra/fuse/fuse.h
@@ -43,6 +43,7 @@ struct tegra_fuse {
void __iomem *base;
phys_addr_t phys;
struct clk *clk;
+ struct reset_control *rst;
u32 (*read_early)(struct tegra_fuse *fuse, unsigned int offset);
u32 (*read)(struct tegra_fuse *fuse, unsigned int offset);
diff --git a/drivers/soc/tegra/pmc.c b/drivers/soc/tegra/pmc.c
index 575d6d5..5aceacb 100644
--- a/drivers/soc/tegra/pmc.c
+++ b/drivers/soc/tegra/pmc.c
@@ -1064,10 +1064,8 @@ int tegra_pmc_cpu_remove_clamping(unsigned int cpuid)
return tegra_powergate_remove_clamping(id);
}
-static int tegra_pmc_restart_notify(struct notifier_block *this,
- unsigned long action, void *data)
+static void tegra_pmc_program_reboot_reason(const char *cmd)
{
- const char *cmd = data;
u32 value;
value = tegra_pmc_scratch_readl(pmc, pmc->soc->regs->scratch0);
@@ -1085,6 +1083,25 @@ static int tegra_pmc_restart_notify(struct notifier_block *this,
}
tegra_pmc_scratch_writel(pmc, value, pmc->soc->regs->scratch0);
+}
+
+static int tegra_pmc_reboot_notify(struct notifier_block *this,
+ unsigned long action, void *data)
+{
+ if (action == SYS_RESTART)
+ tegra_pmc_program_reboot_reason(data);
+
+ return NOTIFY_DONE;
+}
+
+static struct notifier_block tegra_pmc_reboot_notifier = {
+ .notifier_call = tegra_pmc_reboot_notify,
+};
+
+static int tegra_pmc_restart_notify(struct notifier_block *this,
+ unsigned long action, void *data)
+{
+ u32 value;
/* reset everything but PMC_SCRATCH0 and PMC_RST_STATUS */
value = tegra_pmc_readl(pmc, PMC_CNTRL);
@@ -1353,7 +1370,7 @@ static int tegra_pmc_core_pd_add(struct tegra_pmc *pmc, struct device_node *np)
if (!genpd)
return -ENOMEM;
- genpd->name = np->name;
+ genpd->name = "core";
genpd->set_performance_state = tegra_pmc_core_pd_set_performance_state;
genpd->opp_to_performance_state = tegra_pmc_core_pd_opp_to_performance_state;
@@ -2890,6 +2907,14 @@ static int tegra_pmc_probe(struct platform_device *pdev)
goto cleanup_sysfs;
}
+ err = devm_register_reboot_notifier(&pdev->dev,
+ &tegra_pmc_reboot_notifier);
+ if (err) {
+ dev_err(&pdev->dev, "unable to register reboot notifier, %d\n",
+ err);
+ goto cleanup_debugfs;
+ }
+
err = register_restart_handler(&tegra_pmc_restart_handler);
if (err) {
dev_err(&pdev->dev, "unable to register restart handler, %d\n",
@@ -2963,7 +2988,7 @@ static SIMPLE_DEV_PM_OPS(tegra_pmc_pm_ops, tegra_pmc_suspend, tegra_pmc_resume);
static const char * const tegra20_powergates[] = {
[TEGRA_POWERGATE_CPU] = "cpu",
- [TEGRA_POWERGATE_3D] = "3d",
+ [TEGRA_POWERGATE_3D] = "td",
[TEGRA_POWERGATE_VENC] = "venc",
[TEGRA_POWERGATE_VDEC] = "vdec",
[TEGRA_POWERGATE_PCIE] = "pcie",
@@ -3071,7 +3096,7 @@ static const struct tegra_pmc_soc tegra20_pmc_soc = {
static const char * const tegra30_powergates[] = {
[TEGRA_POWERGATE_CPU] = "cpu0",
- [TEGRA_POWERGATE_3D] = "3d0",
+ [TEGRA_POWERGATE_3D] = "td",
[TEGRA_POWERGATE_VENC] = "venc",
[TEGRA_POWERGATE_VDEC] = "vdec",
[TEGRA_POWERGATE_PCIE] = "pcie",
@@ -3083,7 +3108,7 @@ static const char * const tegra30_powergates[] = {
[TEGRA_POWERGATE_CPU2] = "cpu2",
[TEGRA_POWERGATE_CPU3] = "cpu3",
[TEGRA_POWERGATE_CELP] = "celp",
- [TEGRA_POWERGATE_3D1] = "3d1",
+ [TEGRA_POWERGATE_3D1] = "td2",
};
static const u8 tegra30_cpu_powergates[] = {
@@ -3132,7 +3157,7 @@ static const struct tegra_pmc_soc tegra30_pmc_soc = {
static const char * const tegra114_powergates[] = {
[TEGRA_POWERGATE_CPU] = "crail",
- [TEGRA_POWERGATE_3D] = "3d",
+ [TEGRA_POWERGATE_3D] = "td",
[TEGRA_POWERGATE_VENC] = "venc",
[TEGRA_POWERGATE_VDEC] = "vdec",
[TEGRA_POWERGATE_MPE] = "mpe",
diff --git a/drivers/soc/tegra/regulators-tegra20.c b/drivers/soc/tegra/regulators-tegra20.c
index b8ce9fd..6a2f90a 100644
--- a/drivers/soc/tegra/regulators-tegra20.c
+++ b/drivers/soc/tegra/regulators-tegra20.c
@@ -16,7 +16,9 @@
#include <linux/regulator/coupler.h>
#include <linux/regulator/driver.h>
#include <linux/regulator/machine.h>
+#include <linux/suspend.h>
+#include <soc/tegra/fuse.h>
#include <soc/tegra/pmc.h>
struct tegra_regulator_coupler {
@@ -25,9 +27,12 @@ struct tegra_regulator_coupler {
struct regulator_dev *cpu_rdev;
struct regulator_dev *rtc_rdev;
struct notifier_block reboot_notifier;
+ struct notifier_block suspend_notifier;
int core_min_uV, cpu_min_uV;
bool sys_reboot_mode_req;
bool sys_reboot_mode;
+ bool sys_suspend_mode_req;
+ bool sys_suspend_mode;
};
static inline struct tegra_regulator_coupler *
@@ -105,6 +110,28 @@ static int tegra20_core_rtc_max_spread(struct regulator_dev *core_rdev,
return 150000;
}
+static int tegra20_cpu_nominal_uV(void)
+{
+ switch (tegra_sku_info.soc_speedo_id) {
+ case 0:
+ return 1100000;
+ case 1:
+ return 1025000;
+ default:
+ return 1125000;
+ }
+}
+
+static int tegra20_core_nominal_uV(void)
+{
+ switch (tegra_sku_info.soc_speedo_id) {
+ default:
+ return 1225000;
+ case 2:
+ return 1300000;
+ }
+}
+
static int tegra20_core_rtc_update(struct tegra_regulator_coupler *tegra,
struct regulator_dev *core_rdev,
struct regulator_dev *rtc_rdev,
@@ -144,6 +171,11 @@ static int tegra20_core_rtc_update(struct tegra_regulator_coupler *tegra,
if (err)
return err;
+ /* prepare voltage level for suspend */
+ if (tegra->sys_suspend_mode)
+ core_min_uV = clamp(tegra20_core_nominal_uV(),
+ core_min_uV, core_max_uV);
+
core_uV = regulator_get_voltage_rdev(core_rdev);
if (core_uV < 0)
return core_uV;
@@ -279,6 +311,11 @@ static int tegra20_cpu_voltage_update(struct tegra_regulator_coupler *tegra,
if (tegra->sys_reboot_mode)
cpu_min_uV = max(cpu_min_uV, tegra->cpu_min_uV);
+ /* prepare voltage level for suspend */
+ if (tegra->sys_suspend_mode)
+ cpu_min_uV = clamp(tegra20_cpu_nominal_uV(),
+ cpu_min_uV, cpu_max_uV);
+
if (cpu_min_uV > cpu_uV) {
err = tegra20_core_rtc_update(tegra, core_rdev, rtc_rdev,
cpu_uV, cpu_min_uV);
@@ -320,6 +357,7 @@ static int tegra20_regulator_balance_voltage(struct regulator_coupler *coupler,
}
tegra->sys_reboot_mode = READ_ONCE(tegra->sys_reboot_mode_req);
+ tegra->sys_suspend_mode = READ_ONCE(tegra->sys_suspend_mode_req);
if (rdev == cpu_rdev)
return tegra20_cpu_voltage_update(tegra, cpu_rdev,
@@ -334,6 +372,63 @@ static int tegra20_regulator_balance_voltage(struct regulator_coupler *coupler,
return -EPERM;
}
+static int tegra20_regulator_prepare_suspend(struct tegra_regulator_coupler *tegra,
+ bool sys_suspend_mode)
+{
+ int err;
+
+ if (!tegra->core_rdev || !tegra->rtc_rdev || !tegra->cpu_rdev)
+ return 0;
+
+ /*
+ * All power domains are enabled early during resume from suspend
+ * by GENPD core. Domains like VENC may require a higher voltage
+ * when enabled during resume from suspend. This also prepares
+ * hardware for resuming from LP0.
+ */
+
+ WRITE_ONCE(tegra->sys_suspend_mode_req, sys_suspend_mode);
+
+ err = regulator_sync_voltage_rdev(tegra->cpu_rdev);
+ if (err)
+ return err;
+
+ err = regulator_sync_voltage_rdev(tegra->core_rdev);
+ if (err)
+ return err;
+
+ return 0;
+}
+
+static int tegra20_regulator_suspend(struct notifier_block *notifier,
+ unsigned long mode, void *arg)
+{
+ struct tegra_regulator_coupler *tegra;
+ int ret = 0;
+
+ tegra = container_of(notifier, struct tegra_regulator_coupler,
+ suspend_notifier);
+
+ switch (mode) {
+ case PM_HIBERNATION_PREPARE:
+ case PM_RESTORE_PREPARE:
+ case PM_SUSPEND_PREPARE:
+ ret = tegra20_regulator_prepare_suspend(tegra, true);
+ break;
+
+ case PM_POST_HIBERNATION:
+ case PM_POST_RESTORE:
+ case PM_POST_SUSPEND:
+ ret = tegra20_regulator_prepare_suspend(tegra, false);
+ break;
+ }
+
+ if (ret)
+ pr_err("failed to prepare regulators: %d\n", ret);
+
+ return notifier_from_errno(ret);
+}
+
static int tegra20_regulator_prepare_reboot(struct tegra_regulator_coupler *tegra,
bool sys_reboot_mode)
{
@@ -444,6 +539,7 @@ static struct tegra_regulator_coupler tegra20_coupler = {
.balance_voltage = tegra20_regulator_balance_voltage,
},
.reboot_notifier.notifier_call = tegra20_regulator_reboot,
+ .suspend_notifier.notifier_call = tegra20_regulator_suspend,
};
static int __init tegra_regulator_coupler_init(void)
@@ -456,6 +552,9 @@ static int __init tegra_regulator_coupler_init(void)
err = register_reboot_notifier(&tegra20_coupler.reboot_notifier);
WARN_ON(err);
+ err = register_pm_notifier(&tegra20_coupler.suspend_notifier);
+ WARN_ON(err);
+
return regulator_coupler_register(&tegra20_coupler.coupler);
}
arch_initcall(tegra_regulator_coupler_init);
diff --git a/drivers/soc/tegra/regulators-tegra30.c b/drivers/soc/tegra/regulators-tegra30.c
index e74bbc9c..8fd43c6 100644
--- a/drivers/soc/tegra/regulators-tegra30.c
+++ b/drivers/soc/tegra/regulators-tegra30.c
@@ -16,6 +16,7 @@
#include <linux/regulator/coupler.h>
#include <linux/regulator/driver.h>
#include <linux/regulator/machine.h>
+#include <linux/suspend.h>
#include <soc/tegra/fuse.h>
#include <soc/tegra/pmc.h>
@@ -25,9 +26,12 @@ struct tegra_regulator_coupler {
struct regulator_dev *core_rdev;
struct regulator_dev *cpu_rdev;
struct notifier_block reboot_notifier;
+ struct notifier_block suspend_notifier;
int core_min_uV, cpu_min_uV;
bool sys_reboot_mode_req;
bool sys_reboot_mode;
+ bool sys_suspend_mode_req;
+ bool sys_suspend_mode;
};
static inline struct tegra_regulator_coupler *
@@ -113,6 +117,52 @@ static int tegra30_core_cpu_limit(int cpu_uV)
return -EINVAL;
}
+static int tegra30_cpu_nominal_uV(void)
+{
+ switch (tegra_sku_info.cpu_speedo_id) {
+ case 10 ... 11:
+ return 850000;
+
+ case 9:
+ return 912000;
+
+ case 1 ... 3:
+ case 7 ... 8:
+ return 1050000;
+
+ default:
+ return 1125000;
+
+ case 4 ... 6:
+ case 12 ... 13:
+ return 1237000;
+ }
+}
+
+static int tegra30_core_nominal_uV(void)
+{
+ switch (tegra_sku_info.soc_speedo_id) {
+ case 0:
+ return 1200000;
+
+ case 1:
+ if (tegra_sku_info.cpu_speedo_id != 7 &&
+ tegra_sku_info.cpu_speedo_id != 8)
+ return 1200000;
+
+ fallthrough;
+
+ case 2:
+ if (tegra_sku_info.cpu_speedo_id != 13)
+ return 1300000;
+
+ return 1350000;
+
+ default:
+ return 1250000;
+ }
+}
+
static int tegra30_voltage_update(struct tegra_regulator_coupler *tegra,
struct regulator_dev *cpu_rdev,
struct regulator_dev *core_rdev)
@@ -168,6 +218,11 @@ static int tegra30_voltage_update(struct tegra_regulator_coupler *tegra,
if (err)
return err;
+ /* prepare voltage level for suspend */
+ if (tegra->sys_suspend_mode)
+ core_min_uV = clamp(tegra30_core_nominal_uV(),
+ core_min_uV, core_max_uV);
+
core_uV = regulator_get_voltage_rdev(core_rdev);
if (core_uV < 0)
return core_uV;
@@ -223,6 +278,11 @@ static int tegra30_voltage_update(struct tegra_regulator_coupler *tegra,
if (tegra->sys_reboot_mode)
cpu_min_uV = max(cpu_min_uV, tegra->cpu_min_uV);
+ /* prepare voltage level for suspend */
+ if (tegra->sys_suspend_mode)
+ cpu_min_uV = clamp(tegra30_cpu_nominal_uV(),
+ cpu_min_uV, cpu_max_uV);
+
if (core_min_limited_uV > core_uV) {
pr_err("core voltage constraint violated: %d %d %d\n",
core_uV, core_min_limited_uV, cpu_uV);
@@ -292,10 +352,68 @@ static int tegra30_regulator_balance_voltage(struct regulator_coupler *coupler,
}
tegra->sys_reboot_mode = READ_ONCE(tegra->sys_reboot_mode_req);
+ tegra->sys_suspend_mode = READ_ONCE(tegra->sys_suspend_mode_req);
return tegra30_voltage_update(tegra, cpu_rdev, core_rdev);
}
+static int tegra30_regulator_prepare_suspend(struct tegra_regulator_coupler *tegra,
+ bool sys_suspend_mode)
+{
+ int err;
+
+ if (!tegra->core_rdev || !tegra->cpu_rdev)
+ return 0;
+
+ /*
+ * All power domains are enabled early during resume from suspend
+ * by GENPD core. Domains like VENC may require a higher voltage
+ * when enabled during resume from suspend. This also prepares
+ * hardware for resuming from LP0.
+ */
+
+ WRITE_ONCE(tegra->sys_suspend_mode_req, sys_suspend_mode);
+
+ err = regulator_sync_voltage_rdev(tegra->cpu_rdev);
+ if (err)
+ return err;
+
+ err = regulator_sync_voltage_rdev(tegra->core_rdev);
+ if (err)
+ return err;
+
+ return 0;
+}
+
+static int tegra30_regulator_suspend(struct notifier_block *notifier,
+ unsigned long mode, void *arg)
+{
+ struct tegra_regulator_coupler *tegra;
+ int ret = 0;
+
+ tegra = container_of(notifier, struct tegra_regulator_coupler,
+ suspend_notifier);
+
+ switch (mode) {
+ case PM_HIBERNATION_PREPARE:
+ case PM_RESTORE_PREPARE:
+ case PM_SUSPEND_PREPARE:
+ ret = tegra30_regulator_prepare_suspend(tegra, true);
+ break;
+
+ case PM_POST_HIBERNATION:
+ case PM_POST_RESTORE:
+ case PM_POST_SUSPEND:
+ ret = tegra30_regulator_prepare_suspend(tegra, false);
+ break;
+ }
+
+ if (ret)
+ pr_err("failed to prepare regulators: %d\n", ret);
+
+ return notifier_from_errno(ret);
+}
+
static int tegra30_regulator_prepare_reboot(struct tegra_regulator_coupler *tegra,
bool sys_reboot_mode)
{
@@ -395,6 +513,7 @@ static struct tegra_regulator_coupler tegra30_coupler = {
.balance_voltage = tegra30_regulator_balance_voltage,
},
.reboot_notifier.notifier_call = tegra30_regulator_reboot,
+ .suspend_notifier.notifier_call = tegra30_regulator_suspend,
};
static int __init tegra_regulator_coupler_init(void)
@@ -407,6 +526,9 @@ static int __init tegra_regulator_coupler_init(void)
err = register_reboot_notifier(&tegra30_coupler.reboot_notifier);
WARN_ON(err);
+ err = register_pm_notifier(&tegra30_coupler.suspend_notifier);
+ WARN_ON(err);
+
return regulator_coupler_register(&tegra30_coupler.coupler);
}
arch_initcall(tegra_regulator_coupler_init);
diff --git a/drivers/soc/ti/k3-socinfo.c b/drivers/soc/ti/k3-socinfo.c
index fd91129..b6b2150 100644
--- a/drivers/soc/ti/k3-socinfo.c
+++ b/drivers/soc/ti/k3-socinfo.c
@@ -40,7 +40,8 @@ static const struct k3_soc_id {
{ 0xBB5A, "AM65X" },
{ 0xBB64, "J721E" },
{ 0xBB6D, "J7200" },
- { 0xBB38, "AM64X" }
+ { 0xBB38, "AM64X" },
+ { 0xBB75, "J721S2"},
};
static int
diff --git a/drivers/soc/ti/knav_dma.c b/drivers/soc/ti/knav_dma.c
index 591d14e..700d8ee 100644
--- a/drivers/soc/ti/knav_dma.c
+++ b/drivers/soc/ti/knav_dma.c
@@ -646,31 +646,31 @@ static int dma_init(struct device_node *cloud, struct device_node *dma_node)
}
dma->reg_global = pktdma_get_regs(dma, node, 0, &size);
- if (!dma->reg_global)
- return -ENODEV;
+ if (IS_ERR(dma->reg_global))
+ return PTR_ERR(dma->reg_global);
if (size < sizeof(struct reg_global)) {
dev_err(kdev->dev, "bad size %pa for global regs\n", &size);
return -ENODEV;
}
dma->reg_tx_chan = pktdma_get_regs(dma, node, 1, &size);
- if (!dma->reg_tx_chan)
- return -ENODEV;
+ if (IS_ERR(dma->reg_tx_chan))
+ return PTR_ERR(dma->reg_tx_chan);
max_tx_chan = size / sizeof(struct reg_chan);
dma->reg_rx_chan = pktdma_get_regs(dma, node, 2, &size);
- if (!dma->reg_rx_chan)
- return -ENODEV;
+ if (IS_ERR(dma->reg_rx_chan))
+ return PTR_ERR(dma->reg_rx_chan);
max_rx_chan = size / sizeof(struct reg_chan);
dma->reg_tx_sched = pktdma_get_regs(dma, node, 3, &size);
- if (!dma->reg_tx_sched)
- return -ENODEV;
+ if (IS_ERR(dma->reg_tx_sched))
+ return PTR_ERR(dma->reg_tx_sched);
max_tx_sched = size / sizeof(struct reg_tx_sched);
dma->reg_rx_flow = pktdma_get_regs(dma, node, 4, &size);
- if (!dma->reg_rx_flow)
- return -ENODEV;
+ if (IS_ERR(dma->reg_rx_flow))
+ return PTR_ERR(dma->reg_rx_flow);
max_rx_flow = size / sizeof(struct reg_rx_flow);
dma->rx_priority = DMA_PRIO_DEFAULT;
diff --git a/drivers/soc/ti/pruss.c b/drivers/soc/ti/pruss.c
index 49da387..b367793 100644
--- a/drivers/soc/ti/pruss.c
+++ b/drivers/soc/ti/pruss.c
@@ -129,7 +129,7 @@ static int pruss_clk_init(struct pruss *pruss, struct device_node *cfg_node)
clks_np = of_get_child_by_name(cfg_node, "clocks");
if (!clks_np) {
- dev_err(dev, "%pOF is missing its 'clocks' node\n", clks_np);
+ dev_err(dev, "%pOF is missing its 'clocks' node\n", cfg_node);
return -ENODEV;
}
diff --git a/drivers/soc/xilinx/zynqmp_pm_domains.c b/drivers/soc/xilinx/zynqmp_pm_domains.c
index 226d343..fcce243 100644
--- a/drivers/soc/xilinx/zynqmp_pm_domains.c
+++ b/drivers/soc/xilinx/zynqmp_pm_domains.c
@@ -20,8 +20,6 @@
#include <linux/firmware/xlnx-zynqmp.h>
#define ZYNQMP_NUM_DOMAINS (100)
-/* Flag stating if PM nodes mapped to the PM domain has been requested */
-#define ZYNQMP_PM_DOMAIN_REQUESTED BIT(0)
static int min_capability;
@@ -29,14 +27,17 @@ static int min_capability;
* struct zynqmp_pm_domain - Wrapper around struct generic_pm_domain
* @gpd: Generic power domain
* @node_id: PM node ID corresponding to device inside PM domain
- * @flags: ZynqMP PM domain flags
+ * @requested: The PM node mapped to the PM domain has been requested
*/
struct zynqmp_pm_domain {
struct generic_pm_domain gpd;
u32 node_id;
- u8 flags;
+ bool requested;
};
+#define to_zynqmp_pm_domain(pm_domain) \
+ container_of(pm_domain, struct zynqmp_pm_domain, gpd)
+
/**
* zynqmp_gpd_is_active_wakeup_path() - Check if device is in wakeup source
* path
@@ -71,21 +72,23 @@ static int zynqmp_gpd_is_active_wakeup_path(struct device *dev, void *not_used)
*/
static int zynqmp_gpd_power_on(struct generic_pm_domain *domain)
{
+ struct zynqmp_pm_domain *pd = to_zynqmp_pm_domain(domain);
int ret;
- struct zynqmp_pm_domain *pd;
- pd = container_of(domain, struct zynqmp_pm_domain, gpd);
ret = zynqmp_pm_set_requirement(pd->node_id,
ZYNQMP_PM_CAPABILITY_ACCESS,
ZYNQMP_PM_MAX_QOS,
ZYNQMP_PM_REQUEST_ACK_BLOCKING);
if (ret) {
- pr_err("%s() %s set requirement for node %d failed: %d\n",
- __func__, domain->name, pd->node_id, ret);
+ dev_err(&domain->dev,
+ "failed to set requirement to 0x%x for PM node id %d: %d\n",
+ ZYNQMP_PM_CAPABILITY_ACCESS, pd->node_id, ret);
return ret;
}
- pr_debug("%s() Powered on %s domain\n", __func__, domain->name);
+ dev_dbg(&domain->dev, "set requirement to 0x%x for PM node id %d\n",
+ ZYNQMP_PM_CAPABILITY_ACCESS, pd->node_id);
+
return 0;
}
@@ -100,18 +103,16 @@ static int zynqmp_gpd_power_on(struct generic_pm_domain *domain)
*/
static int zynqmp_gpd_power_off(struct generic_pm_domain *domain)
{
+ struct zynqmp_pm_domain *pd = to_zynqmp_pm_domain(domain);
int ret;
struct pm_domain_data *pdd, *tmp;
- struct zynqmp_pm_domain *pd;
u32 capabilities = min_capability;
bool may_wakeup;
- pd = container_of(domain, struct zynqmp_pm_domain, gpd);
-
/* If domain is already released there is nothing to be done */
- if (!(pd->flags & ZYNQMP_PM_DOMAIN_REQUESTED)) {
- pr_debug("%s() %s domain is already released\n",
- __func__, domain->name);
+ if (!pd->requested) {
+ dev_dbg(&domain->dev, "PM node id %d is already released\n",
+ pd->node_id);
return 0;
}
@@ -128,17 +129,16 @@ static int zynqmp_gpd_power_off(struct generic_pm_domain *domain)
ret = zynqmp_pm_set_requirement(pd->node_id, capabilities, 0,
ZYNQMP_PM_REQUEST_ACK_NO);
- /**
- * If powering down of any node inside this domain fails,
- * report and return the error
- */
if (ret) {
- pr_err("%s() %s set requirement for node %d failed: %d\n",
- __func__, domain->name, pd->node_id, ret);
+ dev_err(&domain->dev,
+ "failed to set requirement to 0x%x for PM node id %d: %d\n",
+ capabilities, pd->node_id, ret);
return ret;
}
- pr_debug("%s() Powered off %s domain\n", __func__, domain->name);
+ dev_dbg(&domain->dev, "set requirement to 0x%x for PM node id %d\n",
+ capabilities, pd->node_id);
+
return 0;
}
@@ -152,10 +152,14 @@ static int zynqmp_gpd_power_off(struct generic_pm_domain *domain)
static int zynqmp_gpd_attach_dev(struct generic_pm_domain *domain,
struct device *dev)
{
+ struct zynqmp_pm_domain *pd = to_zynqmp_pm_domain(domain);
+ struct device_link *link;
int ret;
- struct zynqmp_pm_domain *pd;
- pd = container_of(domain, struct zynqmp_pm_domain, gpd);
+ link = device_link_add(dev, &domain->dev, DL_FLAG_SYNC_STATE_ONLY);
+ if (!link)
+ dev_dbg(&domain->dev, "failed to create device link for %s\n",
+ dev_name(dev));
/* If this is not the first device to attach there is nothing to do */
if (domain->device_count)
@@ -163,17 +167,17 @@ static int zynqmp_gpd_attach_dev(struct generic_pm_domain *domain,
ret = zynqmp_pm_request_node(pd->node_id, 0, 0,
ZYNQMP_PM_REQUEST_ACK_BLOCKING);
- /* If requesting a node fails print and return the error */
if (ret) {
- pr_err("%s() %s request failed for node %d: %d\n",
- __func__, domain->name, pd->node_id, ret);
+ dev_err(&domain->dev, "%s request failed for node %d: %d\n",
+ domain->name, pd->node_id, ret);
return ret;
}
- pd->flags |= ZYNQMP_PM_DOMAIN_REQUESTED;
+ pd->requested = true;
- pr_debug("%s() %s attached to %s domain\n", __func__,
- dev_name(dev), domain->name);
+ dev_dbg(&domain->dev, "%s requested PM node id %d\n",
+ dev_name(dev), pd->node_id);
+
return 0;
}
@@ -185,27 +189,24 @@ static int zynqmp_gpd_attach_dev(struct generic_pm_domain *domain,
static void zynqmp_gpd_detach_dev(struct generic_pm_domain *domain,
struct device *dev)
{
+ struct zynqmp_pm_domain *pd = to_zynqmp_pm_domain(domain);
int ret;
- struct zynqmp_pm_domain *pd;
-
- pd = container_of(domain, struct zynqmp_pm_domain, gpd);
/* If this is not the last device to detach there is nothing to do */
if (domain->device_count)
return;
ret = zynqmp_pm_release_node(pd->node_id);
- /* If releasing a node fails print the error and return */
if (ret) {
- pr_err("%s() %s release failed for node %d: %d\n",
- __func__, domain->name, pd->node_id, ret);
+ dev_err(&domain->dev, "failed to release PM node id %d: %d\n",
+ pd->node_id, ret);
return;
}
- pd->flags &= ~ZYNQMP_PM_DOMAIN_REQUESTED;
+ pd->requested = false;
- pr_debug("%s() %s detached from %s domain\n", __func__,
- dev_name(dev), domain->name);
+ dev_dbg(&domain->dev, "%s released PM node id %d\n",
+ dev_name(dev), pd->node_id);
}
static struct generic_pm_domain *zynqmp_gpd_xlate
@@ -215,7 +216,7 @@ static struct generic_pm_domain *zynqmp_gpd_xlate
unsigned int i, idx = genpdspec->args[0];
struct zynqmp_pm_domain *pd;
- pd = container_of(genpd_data->domains[0], struct zynqmp_pm_domain, gpd);
+ pd = to_zynqmp_pm_domain(genpd_data->domains[0]);
if (genpdspec->args_count != 1)
return ERR_PTR(-EINVAL);
@@ -299,9 +300,19 @@ static int zynqmp_gpd_remove(struct platform_device *pdev)
return 0;
}
+static void zynqmp_gpd_sync_state(struct device *dev)
+{
+ int ret;
+
+ ret = zynqmp_pm_init_finalize();
+ if (ret)
+ dev_warn(dev, "failed to release power management to firmware\n");
+}
+
static struct platform_driver zynqmp_power_domain_driver = {
.driver = {
.name = "zynqmp_power_controller",
+ .sync_state = zynqmp_gpd_sync_state,
},
.probe = zynqmp_gpd_probe,
.remove = zynqmp_gpd_remove,
diff --git a/drivers/soc/xilinx/zynqmp_power.c b/drivers/soc/xilinx/zynqmp_power.c
index c556623..f8c3019 100644
--- a/drivers/soc/xilinx/zynqmp_power.c
+++ b/drivers/soc/xilinx/zynqmp_power.c
@@ -178,7 +178,6 @@ static int zynqmp_pm_probe(struct platform_device *pdev)
u32 pm_api_version;
struct mbox_client *client;
- zynqmp_pm_init_finalize();
zynqmp_pm_get_api_version(&pm_api_version);
/* Check PM API version number */
diff --git a/drivers/spi/spi-rpc-if.c b/drivers/spi/spi-rpc-if.c
index 83796a4..fe82f35 100644
--- a/drivers/spi/spi-rpc-if.c
+++ b/drivers/spi/spi-rpc-if.c
@@ -156,7 +156,9 @@ static int rpcif_spi_probe(struct platform_device *pdev)
ctlr->mode_bits = SPI_CPOL | SPI_CPHA | SPI_TX_QUAD | SPI_RX_QUAD;
ctlr->flags = SPI_CONTROLLER_HALF_DUPLEX;
- rpcif_hw_init(rpc, false);
+ error = rpcif_hw_init(rpc, false);
+ if (error)
+ return error;
error = spi_register_controller(ctlr);
if (error) {
diff --git a/drivers/spi/spi-tegra20-slink.c b/drivers/spi/spi-tegra20-slink.c
index e8204e1..2a03739 100644
--- a/drivers/spi/spi-tegra20-slink.c
+++ b/drivers/spi/spi-tegra20-slink.c
@@ -18,12 +18,15 @@
#include <linux/kthread.h>
#include <linux/module.h>
#include <linux/platform_device.h>
+#include <linux/pm_opp.h>
#include <linux/pm_runtime.h>
#include <linux/of.h>
#include <linux/of_device.h>
#include <linux/reset.h>
#include <linux/spi/spi.h>
+#include <soc/tegra/common.h>
+
#define SLINK_COMMAND 0x000
#define SLINK_BIT_LENGTH(x) (((x) & 0x1f) << 0)
#define SLINK_WORD_SIZE(x) (((x) & 0x1f) << 5)
@@ -680,7 +683,7 @@ static int tegra_slink_start_transfer_one(struct spi_device *spi,
bits_per_word = t->bits_per_word;
speed = t->speed_hz;
if (speed != tspi->cur_speed) {
- clk_set_rate(tspi->clk, speed * 4);
+ dev_pm_opp_set_rate(tspi->dev, speed * 4);
tspi->cur_speed = speed;
}
@@ -1066,6 +1069,10 @@ static int tegra_slink_probe(struct platform_device *pdev)
goto exit_free_master;
}
+ ret = devm_tegra_core_dev_init_opp_table_common(&pdev->dev);
+ if (ret)
+ goto exit_free_master;
+
tspi->max_buf_size = SLINK_FIFO_DEPTH << 2;
tspi->dma_buf_size = DEFAULT_SPI_DMA_BUF_LEN;
diff --git a/drivers/staging/media/Kconfig b/drivers/staging/media/Kconfig
index e3aaae9..b81cfa7 100644
--- a/drivers/staging/media/Kconfig
+++ b/drivers/staging/media/Kconfig
@@ -26,6 +26,8 @@
source "drivers/staging/media/imx/Kconfig"
+source "drivers/staging/media/max96712/Kconfig"
+
source "drivers/staging/media/meson/vdec/Kconfig"
source "drivers/staging/media/omap4iss/Kconfig"
diff --git a/drivers/staging/media/Makefile b/drivers/staging/media/Makefile
index 5b5afc5..7e2c86e 100644
--- a/drivers/staging/media/Makefile
+++ b/drivers/staging/media/Makefile
@@ -1,6 +1,7 @@
# SPDX-License-Identifier: GPL-2.0
obj-$(CONFIG_INTEL_ATOMISP) += atomisp/
obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx/
+obj-$(CONFIG_VIDEO_MAX96712) += max96712/
obj-$(CONFIG_VIDEO_MESON_VDEC) += meson/vdec/
obj-$(CONFIG_VIDEO_OMAP4) += omap4iss/
obj-$(CONFIG_VIDEO_ROCKCHIP_VDEC) += rkvdec/
diff --git a/drivers/staging/media/atomisp/Makefile b/drivers/staging/media/atomisp/Makefile
index 606b775..2485d7b 100644
--- a/drivers/staging/media/atomisp/Makefile
+++ b/drivers/staging/media/atomisp/Makefile
@@ -27,20 +27,16 @@
pci/sh_css_firmware.o \
pci/sh_css_host_data.o \
pci/sh_css_hrt.o \
- pci/sh_css_metadata.o \
pci/sh_css_metrics.o \
pci/sh_css_mipi.o \
pci/sh_css_mmu.o \
- pci/sh_css_morph.o \
pci/sh_css.o \
pci/sh_css_param_dvs.o \
pci/sh_css_param_shading.o \
pci/sh_css_params.o \
pci/sh_css_properties.o \
- pci/sh_css_shading.o \
pci/sh_css_sp.o \
pci/sh_css_stream_format.o \
- pci/sh_css_stream.o \
pci/sh_css_version.o \
pci/base/circbuf/src/circbuf.o \
pci/base/refcount/src/refcount.o \
@@ -53,6 +49,9 @@
pci/hmm/hmm.o \
pci/hmm/hmm_reserved_pool.o \
pci/ia_css_device_access.o \
+ pci/ia_css_isp_configs.o \
+ pci/ia_css_isp_states.o \
+ pci/ia_css_isp_params.o \
pci/isp/kernels/aa/aa_2/ia_css_aa2.host.o \
pci/isp/kernels/anr/anr_1.0/ia_css_anr.host.o \
pci/isp/kernels/anr/anr_2/ia_css_anr2.host.o \
@@ -154,21 +153,13 @@
pci/hive_isp_css_common/host/timed_ctrl.o \
pci/hive_isp_css_common/host/vmem.o \
pci/hive_isp_css_shared/host/tag.o \
- pci/system_local.o \
-
-obj-byt = \
- pci/css_2400_system/hive/ia_css_isp_configs.o \
- pci/css_2400_system/hive/ia_css_isp_params.o \
- pci/css_2400_system/hive/ia_css_isp_states.o \
+ pci/system_local.o
# These will be needed when clean merge CHT support nicely into the driver
# Keep them here handy for when we get to that point
#
obj-cht = \
- pci/css_2401_system/hive/ia_css_isp_configs.o \
- pci/css_2401_system/hive/ia_css_isp_params.o \
- pci/css_2401_system/hive/ia_css_isp_states.o \
pci/css_2401_system/host/csi_rx.o \
pci/css_2401_system/host/ibuf_ctrl.o \
pci/css_2401_system/host/isys_dma.o \
@@ -306,10 +297,8 @@
-I$(atomisp)/pci/runtime/rmgr/interface/ \
-I$(atomisp)/pci/runtime/spctrl/interface/ \
-I$(atomisp)/pci/runtime/tagger/interface/ \
- -I$(atomisp)/pci/css_2400_system/hive/ \
-I$(atomisp)/pci/css_2401_system/ \
-I$(atomisp)/pci/css_2401_system/host/ \
- -I$(atomisp)/pci/css_2401_system/hive/ \
-I$(atomisp)/pci/css_2401_system/hrt/
DEFINES := -DHRT_HW -DHRT_ISP_CSS_CUSTOM_HOST -DHRT_USE_VIR_ADDRS -D__HOST__
@@ -324,10 +313,7 @@
atomisp-objs += \
$(obj-cht) \
pci/runtime/isys/src/ibuf_ctrl_rmgr.o
-DEFINES += -DISP2401 -DISP2401_NEW_INPUT_SYSTEM -DSYSTEM_hive_isp_css_2401_system
-else
-atomisp-objs += $(obj-byt)
-DEFINES += -DISP2400 -DSYSTEM_hive_isp_css_2400_system
+DEFINES += -DISP2401
endif
ccflags-y += $(INCLUDES) $(DEFINES) -fno-common
diff --git a/drivers/staging/media/atomisp/TODO b/drivers/staging/media/atomisp/TODO
index 2d1ef9e..43b8420 100644
--- a/drivers/staging/media/atomisp/TODO
+++ b/drivers/staging/media/atomisp/TODO
@@ -1,92 +1,161 @@
+For both Cherrytrail (CHT) and Baytrail (BHT) the driver
+requires the "candrpv_0415_20150521_0458" firmware version.
+It should be noticed that the firmware file is different,
+depending on the ISP model, so they're stored with different
+names:
+
+- for BHT: /lib/firmware/shisp_2400b0_v21.bin
+
+ Warning: The driver was not tested yet for BHT.
+
+- for CHT: /lib/firmware/shisp_2401a0_v21.bin
+
+ https://github.com/intel-aero/meta-intel-aero-base/blob/master/recipes-kernel/linux/linux-yocto/shisp_2401a0_v21.bin
+
NOTE:
=====
-While the driver probes the hardware and reports itself as a
-V4L2 driver, there are still some issues preventing it to
-stream (at least it doesn't with the standard V4L2 applications.
-Didn't test yet with some custom-made app for this driver).
-Solving the related bugs and issues preventing it to work is
-needed (items 6 and 7 from the list below).
+This driver currently doesn't work with most V4L2 applications,
+as there are still some issues with regards to implementing
+certain APIs at the standard way.
+
+Also, currently only USERPTR streaming mode is working.
+
+In order to test, it is needed to know what's the sensor's
+resolution. This can be checked with:
+
+$ v4l2-ctl --get-fmt-video
+ Format Video Capture:
+ Width/Height : 1600/1200
+ ...
+
+It is known to work with:
+
+- v4l2grab at contrib/test directory at https://git.linuxtv.org/v4l-utils.git/
+
+ The resolution should not be bigger than the max resolution
+ supported by the sensor, or it will fail. So, if the sensor
+ reports:
+
+ The driver can be tested with:
+
+ v4l2grab -f YUYV -x 1600 -y 1200 -d /dev/video2 -u
+
+- NVT at https://github.com/intel/nvt
+
+ $ ./v4l2n -o testimage_@.raw \
+ --device /dev/video2 \
+ --input 0 \
+ --exposure=30000,30000,30000,30000 \
+ --parm type=1,capturemode=CI_MODE_PREVIEW \
+ --fmt type=1,width=1600,height=1200,pixelformat=YUYV \
+ --reqbufs count=2,memory=USERPTR \
+ --parameters=wb_config.r=32768,wb_config.gr=21043,wb_config.gb=21043,wb_config.b=30863 \
+ --capture=20
+
+ As the output is in raw format, images need to be converted with:
+
+ $ for i in $(seq 0 19); do
+ name="testimage_$(printf "%03i" $i)"
+ ./raw2pnm -x$WIDTH -y$HEIGHT -f$FORMAT $name.raw $name.pnm
+ rm $name.raw
+ done
TODO
====
-1. The atomisp doesn't rely at the usual i2c stuff to discover the
- sensors. Instead, it calls a function from atomisp_gmin_platform.c.
- There are some hacks added there for it to wait for sensors to be
- probed (with a timeout of 2 seconds or so).
- This should be converted to the usual way, using V4L2 async subdev
- framework to wait for cameras to be probed;
+1. Fix support for MMAP streaming mode. This is required for most
+ V4L2 applications;
-2. Use ACPI _DSM table - DONE!
+2. Implement and/or fix V4L2 ioctls in order to allow a normal app to
+ use it;
-3. Switch the driver to use pm_runtime stuff. Right now, it probes the
- existing PMIC code and sensors call it directly.
+3. Ensure that the driver will pass v4l2-compliance tests;
-4. There's a problem at the sensor drivers: when trying to set a video
- format, the atomisp main driver calls the sensor drivers with the
- sensor turned off. This causes them to fail.
+4. Get manufacturer's authorization to redistribute the binaries for
+ the firmware files;
- The only exception is the atomisp-ov2880, which has a hack inside it
- to turn it on when VIDIOC_S_FMT is called.
+5. remove VIDEO_ATOMISP_ISP2401, making the driver to auto-detect the
+ register address differences between ISP2400 and ISP2401;
- The right fix seems to power on the sensor when a video device is
- opened (or at the first VIDIOC_ ioctl - except for VIDIOC_QUERYCAP),
- powering it down at close() syscall.
+6. Cleanup the driver code, removing the abstraction layers inside it;
- Such kind of control would need to be done inside the atomisp driver,
- not at the sensors code.
+7. The atomisp doesn't rely at the usual i2c stuff to discover the
+ sensors. Instead, it calls a function from atomisp_gmin_platform.c.
+ There are some hacks added there for it to wait for sensors to be
+ probed (with a timeout of 2 seconds or so). This should be converted
+ to the usual way, using V4L2 async subdev framework to wait for
+ cameras to be probed;
-5. There are several issues related to memory management, causing
- crashes. The atomisp splits the memory management on three separate
- regions:
+8. Switch to standard V4L2 sub-device API for sensor and lens. In
+ particular, the user space API needs to support V4L2 controls as
+ defined in the V4L2 spec and references to atomisp must be removed from
+ these drivers.
+
+9. Use LED flash API for flash LED drivers such as LM3554 (which already
+ has a LED class driver).
+
+10. Migrate the sensor drivers out of staging or re-using existing
+ drivers;
+
+11. Switch the driver to use pm_runtime stuff. Right now, it probes the
+ existing PMIC code and sensors call it directly.
+
+12. There's a problem on sensor drivers: when trying to set a video
+ format, the atomisp main driver calls the sensor drivers with the
+ sensor turned off. This causes them to fail.
+
+ This was fixed at atomisp-ov2880, which has a hack inside it
+ to turn it on when VIDIOC_S_FMT is called, but this has to be
+ cheked on other drivers as well.
+
+ The right fix seems to power on the sensor when a video device is
+ opened (or at the first VIDIOC_ ioctl - except for VIDIOC_QUERYCAP),
+ powering it down at close() syscall.
+
+ Such kind of control would need to be done inside the atomisp driver,
+ not at the sensors code.
+
+13. There are several issues related to memory management, that can
+ cause crashes and/or memory leaks. The atomisp splits the memory
+ management on three separate regions:
- dynamic pool;
- reserved pool;
- generic pool
- The code implementing it is at:
+ The code implementing it is at:
drivers/staging/media/atomisp/pci/hmm/
- It also has a separate code for managing DMA buffers at:
+ It also has a separate code for managing DMA buffers at:
drivers/staging/media/atomisp/pci/mmu/
- The code there is really dirty, ugly and probably wrong. I fixed
- one bug there already, but the best would be to just trash it and use
- something else. Maybe the code from the newer intel driver could
- serve as a model:
+ The code there is really dirty, ugly and probably wrong. I fixed
+ one bug there already, but the best would be to just trash it and use
+ something else. Maybe the code from the newer intel driver could
+ serve as a model:
drivers/staging/media/ipu3/ipu3-mmu.c
- But converting it to use something like that is painful and may
- cause some breakages.
+ But converting it to use something like that is painful and may
+ cause some breakages.
-6. There is some issues at the frame receive logic, causing the
- DQBUF ioctls to fail.
+14. The file structure needs to get tidied up to resemble a normal Linux
+ driver.
-7. A single AtomISP driver needs to be implemented to support both
- Baytrail (BYT) and Cherrytail (CHT) platforms at the same time.
- The current driver is a mechanical and hand combined merge of the
- two using several runtime macros, plus some ifdef ISP2401 to select the
- CHT version. Yet, there are some ISP-specific headers that change the
- driver's behavior during compile time.
+15. Lots of the midlayer glue. Unused code and abstraction needs removing.
-8. The file structure needs to get tidied up to resemble a normal Linux
- driver.
-
-9. Lots of the midlayer glue. unused code and abstraction needs removing.
-
-10. The AtomISP driver includes some special IOCTLS (ATOMISP_IOC_XXXX_XXXX)
+16. The AtomISP driver includes some special IOCTLS (ATOMISP_IOC_XXXX_XXXX)
and controls that require some cleanup. Some of those code may have
been removed during the cleanups. They could be needed in order to
- properly support 3A algorithms
+ properly support 3A algorithms.
Such IOCTL interface needs more documentation. The better would
be to use something close to the interface used by the IPU3 IMGU driver.
-11. The ISP code has some dependencies of the exact FW version.
+17. The ISP code has some dependencies of the exact FW version.
The version defined in pci/sh_css_firmware.c:
BYT (isp2400): "irci_stable_candrpv_0415_20150521_0458"
@@ -106,24 +175,16 @@
there are any specific things that can be done to fold in support for
multiple firmware versions.
-12. Switch to standard V4L2 sub-device API for sensor and lens. In
- particular, the user space API needs to support V4L2 controls as
- defined in the V4L2 spec and references to atomisp must be removed from
- these drivers.
-13. Use LED flash API for flash LED drivers such as LM3554 (which already
- has a LED class driver).
+18. Switch from videobuf1 to videobuf2. Videobuf1 is being removed!
-14. Switch from videobuf1 to videobuf2. Videobuf1 is being removed!
-
-15. Correct Coding Style. Please refrain sending coding style patches
+19. Correct Coding Style. Please refrain sending coding style patches
for this driver until the other work is done, as there will be a lot
of code churn until this driver becomes functional again.
-16. Fix private ioctls to not need a compat_ioctl handler for running
- 32-bit tasks. The compat code has been removed because of bugs,
- and should not be needed for modern drivers. Fixing this properly
- unfortunately means an incompatible ABI change.
+20. Remove the logic which sets up pipelines inside it, moving it to
+ libcamera and implement MC support.
+
Limitations
===========
diff --git a/drivers/staging/media/atomisp/i2c/atomisp-gc0310.c b/drivers/staging/media/atomisp/i2c/atomisp-gc0310.c
index 687888d..cbc8b1d 100644
--- a/drivers/staging/media/atomisp/i2c/atomisp-gc0310.c
+++ b/drivers/staging/media/atomisp/i2c/atomisp-gc0310.c
@@ -266,7 +266,7 @@ static int gc0310_g_bin_factor_x(struct v4l2_subdev *sd, s32 *val)
{
struct gc0310_device *dev = to_gc0310_sensor(sd);
- *val = gc0310_res[dev->fmt_idx].bin_factor_x;
+ *val = dev->res->bin_factor_x;
return 0;
}
@@ -275,7 +275,7 @@ static int gc0310_g_bin_factor_y(struct v4l2_subdev *sd, s32 *val)
{
struct gc0310_device *dev = to_gc0310_sensor(sd);
- *val = gc0310_res[dev->fmt_idx].bin_factor_y;
+ *val = dev->res->bin_factor_y;
return 0;
}
@@ -878,76 +878,6 @@ static int gc0310_s_power(struct v4l2_subdev *sd, int on)
return gc0310_init(sd);
}
-/*
- * distance - calculate the distance
- * @res: resolution
- * @w: width
- * @h: height
- *
- * Get the gap between resolution and w/h.
- * res->width/height smaller than w/h wouldn't be considered.
- * Returns the value of gap or -1 if fail.
- */
-#define LARGEST_ALLOWED_RATIO_MISMATCH 800
-static int distance(struct gc0310_resolution *res, u32 w, u32 h)
-{
- unsigned int w_ratio = (res->width << 13) / w;
- unsigned int h_ratio;
- int match;
-
- if (h == 0)
- return -1;
- h_ratio = (res->height << 13) / h;
- if (h_ratio == 0)
- return -1;
- match = abs(((w_ratio << 13) / h_ratio) - 8192);
-
- if ((w_ratio < 8192) || (h_ratio < 8192) ||
- (match > LARGEST_ALLOWED_RATIO_MISMATCH))
- return -1;
-
- return w_ratio + h_ratio;
-}
-
-/* Return the nearest higher resolution index */
-static int nearest_resolution_index(int w, int h)
-{
- int i;
- int idx = -1;
- int dist;
- int min_dist = INT_MAX;
- struct gc0310_resolution *tmp_res = NULL;
-
- for (i = 0; i < N_RES; i++) {
- tmp_res = &gc0310_res[i];
- dist = distance(tmp_res, w, h);
- if (dist == -1)
- continue;
- if (dist < min_dist) {
- min_dist = dist;
- idx = i;
- }
- }
-
- return idx;
-}
-
-static int get_resolution_index(int w, int h)
-{
- int i;
-
- for (i = 0; i < N_RES; i++) {
- if (w != gc0310_res[i].width)
- continue;
- if (h != gc0310_res[i].height)
- continue;
-
- return i;
- }
-
- return -1;
-}
-
/* TODO: remove it. */
static int startup(struct v4l2_subdev *sd)
{
@@ -955,7 +885,7 @@ static int startup(struct v4l2_subdev *sd)
struct i2c_client *client = v4l2_get_subdevdata(sd);
int ret = 0;
- ret = gc0310_write_reg_array(client, gc0310_res[dev->fmt_idx].regs);
+ ret = gc0310_write_reg_array(client, dev->res->regs);
if (ret) {
dev_err(&client->dev, "gc0310 write register err.\n");
return ret;
@@ -972,8 +902,8 @@ static int gc0310_set_fmt(struct v4l2_subdev *sd,
struct gc0310_device *dev = to_gc0310_sensor(sd);
struct i2c_client *client = v4l2_get_subdevdata(sd);
struct camera_mipi_info *gc0310_info = NULL;
+ struct gc0310_resolution *res;
int ret = 0;
- int idx = 0;
if (format->pad)
return -EINVAL;
@@ -987,15 +917,16 @@ static int gc0310_set_fmt(struct v4l2_subdev *sd,
mutex_lock(&dev->input_lock);
- idx = nearest_resolution_index(fmt->width, fmt->height);
- if (idx == -1) {
- /* return the largest resolution */
- fmt->width = gc0310_res[N_RES - 1].width;
- fmt->height = gc0310_res[N_RES - 1].height;
- } else {
- fmt->width = gc0310_res[idx].width;
- fmt->height = gc0310_res[idx].height;
- }
+ res = v4l2_find_nearest_size(gc0310_res_preview,
+ ARRAY_SIZE(gc0310_res_preview), width,
+ height, fmt->width, fmt->height);
+ if (!res)
+ res = &gc0310_res_preview[N_RES - 1];
+
+ fmt->width = res->width;
+ fmt->height = res->height;
+ dev->res = res;
+
fmt->code = MEDIA_BUS_FMT_SGRBG8_1X8;
if (format->which == V4L2_SUBDEV_FORMAT_TRY) {
@@ -1004,23 +935,15 @@ static int gc0310_set_fmt(struct v4l2_subdev *sd,
return 0;
}
- dev->fmt_idx = get_resolution_index(fmt->width, fmt->height);
- if (dev->fmt_idx == -1) {
- dev_err(&client->dev, "get resolution fail\n");
- mutex_unlock(&dev->input_lock);
- return -EINVAL;
- }
-
dev_dbg(&client->dev, "%s: before gc0310_write_reg_array %s\n",
- __func__, gc0310_res[dev->fmt_idx].desc);
+ __func__, dev->res->desc);
ret = startup(sd);
if (ret) {
dev_err(&client->dev, "gc0310 startup err\n");
goto err;
}
- ret = gc0310_get_intg_factor(client, gc0310_info,
- &gc0310_res[dev->fmt_idx]);
+ ret = gc0310_get_intg_factor(client, gc0310_info, dev->res);
if (ret) {
dev_err(&client->dev, "failed to get integration_factor\n");
goto err;
@@ -1044,8 +967,8 @@ static int gc0310_get_fmt(struct v4l2_subdev *sd,
if (!fmt)
return -EINVAL;
- fmt->width = gc0310_res[dev->fmt_idx].width;
- fmt->height = gc0310_res[dev->fmt_idx].height;
+ fmt->width = dev->res->width;
+ fmt->height = dev->res->height;
fmt->code = MEDIA_BUS_FMT_SGRBG8_1X8;
return 0;
@@ -1199,7 +1122,7 @@ static int gc0310_g_frame_interval(struct v4l2_subdev *sd,
struct gc0310_device *dev = to_gc0310_sensor(sd);
interval->interval.numerator = 1;
- interval->interval.denominator = gc0310_res[dev->fmt_idx].fps;
+ interval->interval.denominator = dev->res->fps;
return 0;
}
@@ -1237,7 +1160,7 @@ static int gc0310_g_skip_frames(struct v4l2_subdev *sd, u32 *frames)
struct gc0310_device *dev = to_gc0310_sensor(sd);
mutex_lock(&dev->input_lock);
- *frames = gc0310_res[dev->fmt_idx].skip_frames;
+ *frames = dev->res->skip_frames;
mutex_unlock(&dev->input_lock);
return 0;
@@ -1301,7 +1224,7 @@ static int gc0310_probe(struct i2c_client *client)
mutex_init(&dev->input_lock);
- dev->fmt_idx = 0;
+ dev->res = &gc0310_res_preview[0];
v4l2_i2c_subdev_init(&dev->sd, client, &gc0310_ops);
pdata = gmin_camera_platform_data(&dev->sd,
diff --git a/drivers/staging/media/atomisp/i2c/atomisp-gc2235.c b/drivers/staging/media/atomisp/i2c/atomisp-gc2235.c
index 4d76959..0e6b2e6 100644
--- a/drivers/staging/media/atomisp/i2c/atomisp-gc2235.c
+++ b/drivers/staging/media/atomisp/i2c/atomisp-gc2235.c
@@ -570,14 +570,16 @@ static int power_ctrl(struct v4l2_subdev *sd, bool flag)
static int gpio_ctrl(struct v4l2_subdev *sd, bool flag)
{
struct gc2235_device *dev = to_gc2235_sensor(sd);
- int ret = -1;
+ int ret;
if (!dev || !dev->platform_data)
return -ENODEV;
- ret |= dev->platform_data->gpio1_ctrl(sd, !flag);
+ ret = dev->platform_data->gpio1_ctrl(sd, !flag);
usleep_range(60, 90);
- return dev->platform_data->gpio0_ctrl(sd, flag);
+ ret |= dev->platform_data->gpio0_ctrl(sd, flag);
+
+ return ret;
}
static int power_up(struct v4l2_subdev *sd)
@@ -670,76 +672,6 @@ static int gc2235_s_power(struct v4l2_subdev *sd, int on)
return ret;
}
-/*
- * distance - calculate the distance
- * @res: resolution
- * @w: width
- * @h: height
- *
- * Get the gap between resolution and w/h.
- * res->width/height smaller than w/h wouldn't be considered.
- * Returns the value of gap or -1 if fail.
- */
-#define LARGEST_ALLOWED_RATIO_MISMATCH 800
-static int distance(struct gc2235_resolution *res, u32 w, u32 h)
-{
- unsigned int w_ratio = (res->width << 13) / w;
- unsigned int h_ratio;
- int match;
-
- if (h == 0)
- return -1;
- h_ratio = (res->height << 13) / h;
- if (h_ratio == 0)
- return -1;
- match = abs(((w_ratio << 13) / h_ratio) - 8192);
-
- if ((w_ratio < 8192) || (h_ratio < 8192) ||
- (match > LARGEST_ALLOWED_RATIO_MISMATCH))
- return -1;
-
- return w_ratio + h_ratio;
-}
-
-/* Return the nearest higher resolution index */
-static int nearest_resolution_index(int w, int h)
-{
- int i;
- int idx = -1;
- int dist;
- int min_dist = INT_MAX;
- struct gc2235_resolution *tmp_res = NULL;
-
- for (i = 0; i < N_RES; i++) {
- tmp_res = &gc2235_res[i];
- dist = distance(tmp_res, w, h);
- if (dist == -1)
- continue;
- if (dist < min_dist) {
- min_dist = dist;
- idx = i;
- }
- }
-
- return idx;
-}
-
-static int get_resolution_index(int w, int h)
-{
- int i;
-
- for (i = 0; i < N_RES; i++) {
- if (w != gc2235_res[i].width)
- continue;
- if (h != gc2235_res[i].height)
- continue;
-
- return i;
- }
-
- return -1;
-}
-
static int startup(struct v4l2_subdev *sd)
{
struct gc2235_device *dev = to_gc2235_sensor(sd);
@@ -758,7 +690,7 @@ static int startup(struct v4l2_subdev *sd)
gc2235_write_reg_array(client, gc2235_init_settings);
}
- ret = gc2235_write_reg_array(client, gc2235_res[dev->fmt_idx].regs);
+ ret = gc2235_write_reg_array(client, dev->res->regs);
if (ret) {
dev_err(&client->dev, "gc2235 write register err.\n");
return ret;
@@ -776,8 +708,8 @@ static int gc2235_set_fmt(struct v4l2_subdev *sd,
struct gc2235_device *dev = to_gc2235_sensor(sd);
struct i2c_client *client = v4l2_get_subdevdata(sd);
struct camera_mipi_info *gc2235_info = NULL;
+ struct gc2235_resolution *res;
int ret = 0;
- int idx;
gc2235_info = v4l2_get_subdev_hostdata(sd);
if (!gc2235_info)
@@ -786,16 +718,18 @@ static int gc2235_set_fmt(struct v4l2_subdev *sd,
return -EINVAL;
if (!fmt)
return -EINVAL;
+
mutex_lock(&dev->input_lock);
- idx = nearest_resolution_index(fmt->width, fmt->height);
- if (idx == -1) {
- /* return the largest resolution */
- fmt->width = gc2235_res[N_RES - 1].width;
- fmt->height = gc2235_res[N_RES - 1].height;
- } else {
- fmt->width = gc2235_res[idx].width;
- fmt->height = gc2235_res[idx].height;
- }
+ res = v4l2_find_nearest_size(gc2235_res_preview,
+ ARRAY_SIZE(gc2235_res_preview), width,
+ height, fmt->width, fmt->height);
+ if (!res)
+ res = &gc2235_res_preview[N_RES - 1];
+
+ fmt->width = res->width;
+ fmt->height = res->height;
+ dev->res = res;
+
fmt->code = MEDIA_BUS_FMT_SGRBG10_1X10;
if (format->which == V4L2_SUBDEV_FORMAT_TRY) {
sd_state->pads->try_fmt = *fmt;
@@ -803,13 +737,6 @@ static int gc2235_set_fmt(struct v4l2_subdev *sd,
return 0;
}
- dev->fmt_idx = get_resolution_index(fmt->width, fmt->height);
- if (dev->fmt_idx == -1) {
- dev_err(&client->dev, "get resolution fail\n");
- mutex_unlock(&dev->input_lock);
- return -EINVAL;
- }
-
ret = startup(sd);
if (ret) {
dev_err(&client->dev, "gc2235 startup err\n");
@@ -817,7 +744,7 @@ static int gc2235_set_fmt(struct v4l2_subdev *sd,
}
ret = gc2235_get_intg_factor(client, gc2235_info,
- &gc2235_res[dev->fmt_idx]);
+ dev->res);
if (ret)
dev_err(&client->dev, "failed to get integration_factor\n");
@@ -839,8 +766,8 @@ static int gc2235_get_fmt(struct v4l2_subdev *sd,
if (!fmt)
return -EINVAL;
- fmt->width = gc2235_res[dev->fmt_idx].width;
- fmt->height = gc2235_res[dev->fmt_idx].height;
+ fmt->width = dev->res->width;
+ fmt->height = dev->res->height;
fmt->code = MEDIA_BUS_FMT_SGRBG10_1X10;
return 0;
@@ -953,7 +880,7 @@ static int gc2235_g_frame_interval(struct v4l2_subdev *sd,
struct gc2235_device *dev = to_gc2235_sensor(sd);
interval->interval.numerator = 1;
- interval->interval.denominator = gc2235_res[dev->fmt_idx].fps;
+ interval->interval.denominator = dev->res->fps;
return 0;
}
@@ -991,7 +918,7 @@ static int gc2235_g_skip_frames(struct v4l2_subdev *sd, u32 *frames)
struct gc2235_device *dev = to_gc2235_sensor(sd);
mutex_lock(&dev->input_lock);
- *frames = gc2235_res[dev->fmt_idx].skip_frames;
+ *frames = dev->res->skip_frames;
mutex_unlock(&dev->input_lock);
return 0;
@@ -1055,7 +982,7 @@ static int gc2235_probe(struct i2c_client *client)
mutex_init(&dev->input_lock);
- dev->fmt_idx = 0;
+ dev->res = &gc2235_res_preview[0];
v4l2_i2c_subdev_init(&dev->sd, client, &gc2235_ops);
gcpdev = gmin_camera_platform_data(&dev->sd,
diff --git a/drivers/staging/media/atomisp/i2c/atomisp-mt9m114.c b/drivers/staging/media/atomisp/i2c/atomisp-mt9m114.c
index 49f4090..00d6842 100644
--- a/drivers/staging/media/atomisp/i2c/atomisp-mt9m114.c
+++ b/drivers/staging/media/atomisp/i2c/atomisp-mt9m114.c
@@ -579,107 +579,6 @@ static int mt9m114_s_power(struct v4l2_subdev *sd, int power)
return mt9m114_init_common(sd);
}
-/*
- * distance - calculate the distance
- * @res: resolution
- * @w: width
- * @h: height
- *
- * Get the gap between resolution and w/h.
- * res->width/height smaller than w/h wouldn't be considered.
- * Returns the value of gap or -1 if fail.
- */
-#define LARGEST_ALLOWED_RATIO_MISMATCH 600
-static int distance(struct mt9m114_res_struct const *res, u32 w, u32 h)
-{
- unsigned int w_ratio;
- unsigned int h_ratio;
- int match;
-
- if (w == 0)
- return -1;
- w_ratio = (res->width << 13) / w;
- if (h == 0)
- return -1;
- h_ratio = (res->height << 13) / h;
- if (h_ratio == 0)
- return -1;
- match = abs(((w_ratio << 13) / h_ratio) - 8192);
-
- if ((w_ratio < 8192) || (h_ratio < 8192) ||
- (match > LARGEST_ALLOWED_RATIO_MISMATCH))
- return -1;
-
- return w_ratio + h_ratio;
-}
-
-/* Return the nearest higher resolution index */
-static int nearest_resolution_index(int w, int h)
-{
- int i;
- int idx = -1;
- int dist;
- int min_dist = INT_MAX;
- const struct mt9m114_res_struct *tmp_res = NULL;
-
- for (i = 0; i < ARRAY_SIZE(mt9m114_res); i++) {
- tmp_res = &mt9m114_res[i];
- dist = distance(tmp_res, w, h);
- if (dist == -1)
- continue;
- if (dist < min_dist) {
- min_dist = dist;
- idx = i;
- }
- }
-
- return idx;
-}
-
-static int mt9m114_try_res(u32 *w, u32 *h)
-{
- int idx = 0;
-
- if ((*w > MT9M114_RES_960P_SIZE_H)
- || (*h > MT9M114_RES_960P_SIZE_V)) {
- *w = MT9M114_RES_960P_SIZE_H;
- *h = MT9M114_RES_960P_SIZE_V;
- } else {
- idx = nearest_resolution_index(*w, *h);
-
- /*
- * nearest_resolution_index() doesn't return smaller
- * resolutions. If it fails, it means the requested
- * resolution is higher than wecan support. Fallback
- * to highest possible resolution in this case.
- */
- if (idx == -1)
- idx = ARRAY_SIZE(mt9m114_res) - 1;
-
- *w = mt9m114_res[idx].width;
- *h = mt9m114_res[idx].height;
- }
-
- return 0;
-}
-
-static struct mt9m114_res_struct *mt9m114_to_res(u32 w, u32 h)
-{
- int index;
-
- for (index = 0; index < N_RES; index++) {
- if ((mt9m114_res[index].width == w) &&
- (mt9m114_res[index].height == h))
- break;
- }
-
- /* No mode found */
- if (index >= N_RES)
- return NULL;
-
- return &mt9m114_res[index];
-}
-
static int mt9m114_res2size(struct v4l2_subdev *sd, int *h_size, int *v_size)
{
struct mt9m114_device *dev = to_mt9m114_sensor(sd);
@@ -829,7 +728,7 @@ static int mt9m114_set_fmt(struct v4l2_subdev *sd,
struct v4l2_mbus_framefmt *fmt = &format->format;
struct i2c_client *c = v4l2_get_subdevdata(sd);
struct mt9m114_device *dev = to_mt9m114_sensor(sd);
- struct mt9m114_res_struct *res_index;
+ struct mt9m114_res_struct *res;
u32 width = fmt->width;
u32 height = fmt->height;
struct camera_mipi_info *mt9m114_info = NULL;
@@ -845,20 +744,21 @@ static int mt9m114_set_fmt(struct v4l2_subdev *sd,
if (!mt9m114_info)
return -EINVAL;
- mt9m114_try_res(&width, &height);
+ res = v4l2_find_nearest_size(mt9m114_res,
+ ARRAY_SIZE(mt9m114_res), width,
+ height, fmt->width, fmt->height);
+ if (!res)
+ res = &mt9m114_res[N_RES - 1];
+
+ fmt->width = res->width;
+ fmt->height = res->height;
+
if (format->which == V4L2_SUBDEV_FORMAT_TRY) {
sd_state->pads->try_fmt = *fmt;
return 0;
}
- res_index = mt9m114_to_res(width, height);
- /* Sanity check */
- if (unlikely(!res_index)) {
- WARN_ON(1);
- return -EINVAL;
- }
-
- switch (res_index->res) {
+ switch (res->res) {
case MT9M114_RES_736P:
ret = mt9m114_write_reg_array(c, mt9m114_736P_init, NO_POLLING);
ret += misensor_rmw_reg(c, MISENSOR_16BIT, MISENSOR_READ_MODE,
@@ -876,7 +776,7 @@ static int mt9m114_set_fmt(struct v4l2_subdev *sd,
MISENSOR_R_MODE_MASK, MISENSOR_NORMAL_SET);
break;
default:
- v4l2_err(sd, "set resolution: %d failed!\n", res_index->res);
+ v4l2_err(sd, "set resolution: %d failed!\n", res->res);
return -EINVAL;
}
@@ -890,7 +790,7 @@ static int mt9m114_set_fmt(struct v4l2_subdev *sd,
if (mt9m114_set_suspend(sd))
return -EINVAL;
- if (dev->res != res_index->res) {
+ if (dev->res != res->res) {
int index;
/* Switch to different size */
@@ -922,7 +822,7 @@ static int mt9m114_set_fmt(struct v4l2_subdev *sd,
}
}
ret = mt9m114_get_intg_factor(c, mt9m114_info,
- &mt9m114_res[res_index->res]);
+ &mt9m114_res[res->res]);
if (ret) {
dev_err(&c->dev, "failed to get integration_factor\n");
return -EINVAL;
@@ -931,7 +831,7 @@ static int mt9m114_set_fmt(struct v4l2_subdev *sd,
* mt9m114 - we don't poll for context switch
* because it does not happen with streaming disabled.
*/
- dev->res = res_index->res;
+ dev->res = res->res;
fmt->width = width;
fmt->height = height;
diff --git a/drivers/staging/media/atomisp/i2c/atomisp-ov2680.c b/drivers/staging/media/atomisp/i2c/atomisp-ov2680.c
index 2111e4a..4ba99c6 100644
--- a/drivers/staging/media/atomisp/i2c/atomisp-ov2680.c
+++ b/drivers/staging/media/atomisp/i2c/atomisp-ov2680.c
@@ -48,7 +48,7 @@ static enum atomisp_bayer_order ov2680_bayer_order_mapping[] = {
/* i2c read/write stuff */
static int ov2680_read_reg(struct i2c_client *client,
- int len, u16 reg, u16 *val)
+ int len, u16 reg, u32 *val)
{
struct i2c_msg msgs[2];
u8 addr_buf[2] = { reg >> 8, reg & 0xff };
@@ -86,7 +86,7 @@ static int ov2680_write_reg(struct i2c_client *client, unsigned int len,
int ret;
if (len == 2)
- put_unaligned_be16(val << (8 * (4 - len)), buf + 2);
+ put_unaligned_be16(val, buf + 2);
else if (len == 1)
buf[2] = val;
else
@@ -147,7 +147,7 @@ static int ov2680_g_bin_factor_x(struct v4l2_subdev *sd, s32 *val)
struct i2c_client *client = v4l2_get_subdevdata(sd);
dev_dbg(&client->dev, "++++ov2680_g_bin_factor_x\n");
- *val = ov2680_res[dev->fmt_idx].bin_factor_x;
+ *val = dev->res->bin_factor_x;
return 0;
}
@@ -157,7 +157,7 @@ static int ov2680_g_bin_factor_y(struct v4l2_subdev *sd, s32 *val)
struct ov2680_device *dev = to_ov2680_sensor(sd);
struct i2c_client *client = v4l2_get_subdevdata(sd);
- *val = ov2680_res[dev->fmt_idx].bin_factor_y;
+ *val = dev->res->bin_factor_y;
dev_dbg(&client->dev, "++++ov2680_g_bin_factor_y\n");
return 0;
}
@@ -166,11 +166,9 @@ static int ov2680_get_intg_factor(struct i2c_client *client,
struct camera_mipi_info *info,
const struct ov2680_resolution *res)
{
- struct v4l2_subdev *sd = i2c_get_clientdata(client);
- struct ov2680_device *dev = to_ov2680_sensor(sd);
struct atomisp_sensor_mode_data *buf = &info->data;
unsigned int pix_clk_freq_hz;
- u16 reg_val;
+ u32 reg_val;
int ret;
dev_dbg(&client->dev, "++++ov2680_get_intg_factor\n");
@@ -180,7 +178,6 @@ static int ov2680_get_intg_factor(struct i2c_client *client,
/* pixel clock */
pix_clk_freq_hz = res->pix_clk_freq * 1000000;
- dev->vt_pix_clk_freq_mhz = pix_clk_freq_hz;
buf->vt_pix_clk_freq_mhz = pix_clk_freq_hz;
/* get integration time */
@@ -254,7 +251,7 @@ static long __ov2680_set_exposure(struct v4l2_subdev *sd, int coarse_itg,
"+++++++__ov2680_set_exposure coarse_itg %d, gain %d, digitgain %d++\n",
coarse_itg, gain, digitgain);
- vts = ov2680_res[dev->fmt_idx].lines_per_frame;
+ vts = dev->res->lines_per_frame;
/* group hold */
ret = ov2680_write_reg(client, 1,
@@ -359,10 +356,17 @@ static int ov2680_set_exposure(struct v4l2_subdev *sd, int exposure,
int gain, int digitgain)
{
struct ov2680_device *dev = to_ov2680_sensor(sd);
- int ret;
+ int ret = 0;
mutex_lock(&dev->input_lock);
- ret = __ov2680_set_exposure(sd, exposure, gain, digitgain);
+
+ dev->exposure = exposure;
+ dev->gain = gain;
+ dev->digitgain = digitgain;
+
+ if (dev->power_on)
+ ret = __ov2680_set_exposure(sd, exposure, gain, digitgain);
+
mutex_unlock(&dev->input_lock);
return ret;
@@ -383,7 +387,6 @@ static long ov2680_s_exposure(struct v4l2_subdev *sd,
return -EINVAL;
}
- // EXPOSURE CONTROL DISABLED FOR INITIAL CHECKIN, TUNING DOESN'T WORK
return ov2680_set_exposure(sd, coarse_itg, analog_gain, digital_gain);
}
@@ -406,56 +409,25 @@ static long ov2680_ioctl(struct v4l2_subdev *sd, unsigned int cmd, void *arg)
static int ov2680_q_exposure(struct v4l2_subdev *sd, s32 *value)
{
struct i2c_client *client = v4l2_get_subdevdata(sd);
- u16 reg_v, reg_v2;
+ u32 reg_val;
int ret;
/* get exposure */
- ret = ov2680_read_reg(client, 1,
- OV2680_EXPOSURE_L,
- ®_v);
+ ret = ov2680_read_reg(client, 3, OV2680_EXPOSURE_H, ®_val);
if (ret)
- goto err;
+ return ret;
- ret = ov2680_read_reg(client, 1,
- OV2680_EXPOSURE_M,
- ®_v2);
- if (ret)
- goto err;
-
- reg_v += reg_v2 << 8;
- ret = ov2680_read_reg(client, 1,
- OV2680_EXPOSURE_H,
- ®_v2);
- if (ret)
- goto err;
-
- *value = reg_v + (((u32)reg_v2 << 16));
-err:
- return ret;
-}
-
-static u32 ov2680_translate_bayer_order(enum atomisp_bayer_order code)
-{
- switch (code) {
- case atomisp_bayer_order_rggb:
- return MEDIA_BUS_FMT_SRGGB10_1X10;
- case atomisp_bayer_order_grbg:
- return MEDIA_BUS_FMT_SGRBG10_1X10;
- case atomisp_bayer_order_bggr:
- return MEDIA_BUS_FMT_SBGGR10_1X10;
- case atomisp_bayer_order_gbrg:
- return MEDIA_BUS_FMT_SGBRG10_1X10;
- }
+ /* Lower four bits are not part of the exposure val (always 0) */
+ *value = reg_val >> 4;
return 0;
}
static int ov2680_v_flip(struct v4l2_subdev *sd, s32 value)
{
- struct ov2680_device *dev = to_ov2680_sensor(sd);
struct camera_mipi_info *ov2680_info = NULL;
struct i2c_client *client = v4l2_get_subdevdata(sd);
int ret;
- u16 val;
+ u32 val;
u8 index;
dev_dbg(&client->dev, "@%s: value:%d\n", __func__, value);
@@ -476,19 +448,16 @@ static int ov2680_v_flip(struct v4l2_subdev *sd, s32 value)
ov2680_info = v4l2_get_subdev_hostdata(sd);
if (ov2680_info) {
ov2680_info->raw_bayer_order = ov2680_bayer_order_mapping[index];
- dev->format.code = ov2680_translate_bayer_order(
- ov2680_info->raw_bayer_order);
}
return ret;
}
static int ov2680_h_flip(struct v4l2_subdev *sd, s32 value)
{
- struct ov2680_device *dev = to_ov2680_sensor(sd);
struct camera_mipi_info *ov2680_info = NULL;
struct i2c_client *client = v4l2_get_subdevdata(sd);
int ret;
- u16 val;
+ u32 val;
u8 index;
dev_dbg(&client->dev, "@%s: value:%d\n", __func__, value);
@@ -510,8 +479,6 @@ static int ov2680_h_flip(struct v4l2_subdev *sd, s32 value)
ov2680_info = v4l2_get_subdev_hostdata(sd);
if (ov2680_info) {
ov2680_info->raw_bayer_order = ov2680_bayer_order_mapping[index];
- dev->format.code = ov2680_translate_bayer_order(
- ov2680_info->raw_bayer_order);
}
return ret;
}
@@ -677,25 +644,6 @@ static int ov2680_init_registers(struct v4l2_subdev *sd)
return ret;
}
-static int ov2680_init(struct v4l2_subdev *sd)
-{
- struct ov2680_device *dev = to_ov2680_sensor(sd);
-
- int ret;
-
- mutex_lock(&dev->input_lock);
-
- /* restore settings */
- ov2680_res = ov2680_res_preview;
- N_RES = N_RES_PREVIEW;
-
- ret = ov2680_init_registers(sd);
-
- mutex_unlock(&dev->input_lock);
-
- return ret;
-}
-
static int power_ctrl(struct v4l2_subdev *sd, bool flag)
{
int ret = 0;
@@ -760,6 +708,9 @@ static int power_up(struct v4l2_subdev *sd)
return -ENODEV;
}
+ if (dev->power_on)
+ return 0; /* Already on */
+
/* power control */
ret = power_ctrl(sd, 1);
if (ret)
@@ -784,8 +735,19 @@ static int power_up(struct v4l2_subdev *sd)
/* according to DS, 20ms is needed between PWDN and i2c access */
msleep(20);
+ ret = ov2680_init_registers(sd);
+ if (ret)
+ goto fail_init_registers;
+
+ ret = __ov2680_set_exposure(sd, dev->exposure, dev->gain, dev->digitgain);
+ if (ret)
+ goto fail_init_registers;
+
+ dev->power_on = true;
return 0;
+fail_init_registers:
+ dev->platform_data->flisclk_ctrl(sd, 0);
fail_clk:
gpio_ctrl(sd, 0);
fail_power:
@@ -809,6 +771,9 @@ static int power_down(struct v4l2_subdev *sd)
return -ENODEV;
}
+ if (!dev->power_on)
+ return 0; /* Already off */
+
ret = dev->platform_data->flisclk_ctrl(sd, 0);
if (ret)
dev_err(&client->dev, "flisclk failed\n");
@@ -823,96 +788,33 @@ static int power_down(struct v4l2_subdev *sd)
/* power control */
ret = power_ctrl(sd, 0);
- if (ret)
+ if (ret) {
dev_err(&client->dev, "vprog failed.\n");
+ return ret;
+ }
- return ret;
+ dev->power_on = false;
+ return 0;
}
static int ov2680_s_power(struct v4l2_subdev *sd, int on)
{
+ struct ov2680_device *dev = to_ov2680_sensor(sd);
int ret;
+ mutex_lock(&dev->input_lock);
+
if (on == 0) {
ret = power_down(sd);
} else {
ret = power_up(sd);
- if (!ret)
- return ov2680_init(sd);
}
+
+ mutex_unlock(&dev->input_lock);
+
return ret;
}
-/*
- * distance - calculate the distance
- * @res: resolution
- * @w: width
- * @h: height
- *
- * Get the gap between resolution and w/h.
- * res->width/height smaller than w/h wouldn't be considered.
- * Returns the value of gap or -1 if fail.
- */
-#define LARGEST_ALLOWED_RATIO_MISMATCH 600
-static int distance(struct ov2680_resolution *res, u32 w, u32 h)
-{
- unsigned int w_ratio = (res->width << 13) / w;
- unsigned int h_ratio;
- int match;
-
- if (h == 0)
- return -1;
- h_ratio = (res->height << 13) / h;
- if (h_ratio == 0)
- return -1;
- match = abs(((w_ratio << 13) / h_ratio) - 8192);
-
- if ((w_ratio < 8192) || (h_ratio < 8192) ||
- (match > LARGEST_ALLOWED_RATIO_MISMATCH))
- return -1;
-
- return w_ratio + h_ratio;
-}
-
-/* Return the nearest higher resolution index */
-static int nearest_resolution_index(int w, int h)
-{
- int i;
- int idx = -1;
- int dist;
- int min_dist = INT_MAX;
- struct ov2680_resolution *tmp_res = NULL;
-
- for (i = 0; i < N_RES; i++) {
- tmp_res = &ov2680_res[i];
- dist = distance(tmp_res, w, h);
- if (dist == -1)
- continue;
- if (dist < min_dist) {
- min_dist = dist;
- idx = i;
- }
- }
-
- return idx;
-}
-
-static int get_resolution_index(int w, int h)
-{
- int i;
-
- for (i = 0; i < N_RES; i++) {
- if (w != ov2680_res[i].width)
- continue;
- if (h != ov2680_res[i].height)
- continue;
-
- return i;
- }
-
- return -1;
-}
-
static int ov2680_set_fmt(struct v4l2_subdev *sd,
struct v4l2_subdev_state *sd_state,
struct v4l2_subdev_format *format)
@@ -921,8 +823,8 @@ static int ov2680_set_fmt(struct v4l2_subdev *sd,
struct ov2680_device *dev = to_ov2680_sensor(sd);
struct i2c_client *client = v4l2_get_subdevdata(sd);
struct camera_mipi_info *ov2680_info = NULL;
- int ret = 0;
- int idx = 0;
+ struct ov2680_resolution *res;
+ int vts, ret = 0;
dev_dbg(&client->dev, "%s: %s: pad: %d, fmt: %p\n",
__func__,
@@ -940,41 +842,44 @@ static int ov2680_set_fmt(struct v4l2_subdev *sd,
return -EINVAL;
mutex_lock(&dev->input_lock);
- idx = nearest_resolution_index(fmt->width, fmt->height);
- if (idx == -1) {
- /* return the largest resolution */
- fmt->width = ov2680_res[N_RES - 1].width;
- fmt->height = ov2680_res[N_RES - 1].height;
- } else {
- fmt->width = ov2680_res[idx].width;
- fmt->height = ov2680_res[idx].height;
- }
+
+ res = v4l2_find_nearest_size(ov2680_res_preview,
+ ARRAY_SIZE(ov2680_res_preview), width,
+ height, fmt->width, fmt->height);
+ if (!res)
+ res = &ov2680_res_preview[N_RES_PREVIEW - 1];
+
+ fmt->width = res->width;
+ fmt->height = res->height;
+
fmt->code = MEDIA_BUS_FMT_SBGGR10_1X10;
if (format->which == V4L2_SUBDEV_FORMAT_TRY) {
sd_state->pads->try_fmt = *fmt;
mutex_unlock(&dev->input_lock);
return 0;
}
- dev->fmt_idx = get_resolution_index(fmt->width, fmt->height);
- dev_dbg(&client->dev, "%s: Resolution index: %d\n",
- __func__, dev->fmt_idx);
- if (dev->fmt_idx == -1) {
- dev_err(&client->dev, "get resolution fail\n");
- mutex_unlock(&dev->input_lock);
- return -EINVAL;
- }
- dev_dbg(&client->dev, "%s: i=%d, w=%d, h=%d\n",
- __func__, dev->fmt_idx, fmt->width, fmt->height);
- // IS IT NEEDED?
+ dev_dbg(&client->dev, "%s: %dx%d\n",
+ __func__, fmt->width, fmt->height);
+
+ /* s_power has not been called yet for std v4l2 clients (camorama) */
power_up(sd);
- ret = ov2680_write_reg_array(client, ov2680_res[dev->fmt_idx].regs);
+ ret = ov2680_write_reg_array(client, dev->res->regs);
if (ret)
dev_err(&client->dev,
"ov2680 write resolution register err: %d\n", ret);
- ret = ov2680_get_intg_factor(client, ov2680_info,
- &ov2680_res[dev->fmt_idx]);
+ vts = dev->res->lines_per_frame;
+
+ /* If necessary increase the VTS to match exposure + MARGIN */
+ if (dev->exposure > vts - OV2680_INTEGRATION_TIME_MARGIN)
+ vts = dev->exposure + OV2680_INTEGRATION_TIME_MARGIN;
+
+ ret = ov2680_write_reg(client, 2, OV2680_TIMING_VTS_H, vts);
+ if (ret)
+ dev_err(&client->dev, "ov2680 write vts err: %d\n", ret);
+
+ ret = ov2680_get_intg_factor(client, ov2680_info, res);
if (ret) {
dev_err(&client->dev, "failed to get integration factor\n");
goto err;
@@ -989,8 +894,6 @@ static int ov2680_set_fmt(struct v4l2_subdev *sd,
if (v_flag)
ov2680_v_flip(sd, v_flag);
- v4l2_info(client, "\n%s idx %d\n", __func__, dev->fmt_idx);
-
/*
* ret = startup(sd);
* if (ret)
@@ -1014,8 +917,8 @@ static int ov2680_get_fmt(struct v4l2_subdev *sd,
if (!fmt)
return -EINVAL;
- fmt->width = ov2680_res[dev->fmt_idx].width;
- fmt->height = ov2680_res[dev->fmt_idx].height;
+ fmt->width = dev->res->width;
+ fmt->height = dev->res->height;
fmt->code = MEDIA_BUS_FMT_SBGGR10_1X10;
return 0;
@@ -1024,7 +927,7 @@ static int ov2680_get_fmt(struct v4l2_subdev *sd,
static int ov2680_detect(struct i2c_client *client)
{
struct i2c_adapter *adapter = client->adapter;
- u16 high, low;
+ u32 high, low;
int ret;
u16 id;
u8 revision;
@@ -1072,11 +975,6 @@ static int ov2680_s_stream(struct v4l2_subdev *sd, int enable)
ret = ov2680_write_reg(client, 1, OV2680_SW_STREAM,
enable ? OV2680_START_STREAMING :
OV2680_STOP_STREAMING);
-#if 0
- /* restore settings */
- ov2680_res = ov2680_res_preview;
- N_RES = N_RES_PREVIEW;
-#endif
//otp valid at stream on state
//if(!dev->otp_data)
@@ -1101,16 +999,6 @@ static int ov2680_s_config(struct v4l2_subdev *sd,
(struct camera_sensor_platform_data *)platform_data;
mutex_lock(&dev->input_lock);
- /*
- * power off the module, then power on it in future
- * as first power on by board may not fulfill the
- * power on sequqence needed by the module
- */
- ret = power_down(sd);
- if (ret) {
- dev_err(&client->dev, "ov2680 power-off err.\n");
- goto fail_power_off;
- }
ret = power_up(sd);
if (ret) {
@@ -1144,7 +1032,6 @@ static int ov2680_s_config(struct v4l2_subdev *sd,
fail_power_on:
power_down(sd);
dev_err(&client->dev, "sensor power-gating failed\n");
-fail_power_off:
mutex_unlock(&dev->input_lock);
return ret;
}
@@ -1155,7 +1042,7 @@ static int ov2680_g_frame_interval(struct v4l2_subdev *sd,
struct ov2680_device *dev = to_ov2680_sensor(sd);
interval->interval.numerator = 1;
- interval->interval.denominator = ov2680_res[dev->fmt_idx].fps;
+ interval->interval.denominator = dev->res->fps;
return 0;
}
@@ -1177,13 +1064,33 @@ static int ov2680_enum_frame_size(struct v4l2_subdev *sd,
{
int index = fse->index;
- if (index >= N_RES)
+ if (index >= N_RES_PREVIEW)
return -EINVAL;
- fse->min_width = ov2680_res[index].width;
- fse->min_height = ov2680_res[index].height;
- fse->max_width = ov2680_res[index].width;
- fse->max_height = ov2680_res[index].height;
+ fse->min_width = ov2680_res_preview[index].width;
+ fse->min_height = ov2680_res_preview[index].height;
+ fse->max_width = ov2680_res_preview[index].width;
+ fse->max_height = ov2680_res_preview[index].height;
+
+ return 0;
+}
+
+static int ov2680_enum_frame_interval(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *sd_state,
+ struct v4l2_subdev_frame_interval_enum *fie)
+{
+ struct v4l2_fract fract;
+
+ if (fie->index >= N_RES_PREVIEW ||
+ fie->width > ov2680_res_preview[0].width ||
+ fie->height > ov2680_res_preview[0].height ||
+ fie->which > V4L2_SUBDEV_FORMAT_ACTIVE)
+ return -EINVAL;
+
+ fract.denominator = ov2680_res_preview[fie->index].fps;
+ fract.numerator = 1;
+
+ fie->interval = fract;
return 0;
}
@@ -1193,7 +1100,7 @@ static int ov2680_g_skip_frames(struct v4l2_subdev *sd, u32 *frames)
struct ov2680_device *dev = to_ov2680_sensor(sd);
mutex_lock(&dev->input_lock);
- *frames = ov2680_res[dev->fmt_idx].skip_frames;
+ *frames = dev->res->skip_frames;
mutex_unlock(&dev->input_lock);
return 0;
@@ -1216,6 +1123,7 @@ static const struct v4l2_subdev_core_ops ov2680_core_ops = {
static const struct v4l2_subdev_pad_ops ov2680_pad_ops = {
.enum_mbus_code = ov2680_enum_mbus_code,
.enum_frame_size = ov2680_enum_frame_size,
+ .enum_frame_interval = ov2680_enum_frame_interval,
.get_fmt = ov2680_get_fmt,
.set_fmt = ov2680_set_fmt,
};
@@ -1257,7 +1165,9 @@ static int ov2680_probe(struct i2c_client *client)
mutex_init(&dev->input_lock);
- dev->fmt_idx = 0;
+ dev->res = &ov2680_res_preview[0];
+ dev->exposure = dev->res->lines_per_frame - OV2680_INTEGRATION_TIME_MARGIN;
+ dev->gain = 250; /* 0-2047 */
v4l2_i2c_subdev_init(&dev->sd, client, &ov2680_ops);
pdata = gmin_camera_platform_data(&dev->sd,
@@ -1278,7 +1188,6 @@ static int ov2680_probe(struct i2c_client *client)
dev->sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
dev->pad.flags = MEDIA_PAD_FL_SOURCE;
- dev->format.code = MEDIA_BUS_FMT_SBGGR10_1X10;
dev->sd.entity.function = MEDIA_ENT_F_CAM_SENSOR;
ret =
v4l2_ctrl_handler_init(&dev->ctrl_handler,
diff --git a/drivers/staging/media/atomisp/i2c/atomisp-ov2722.c b/drivers/staging/media/atomisp/i2c/atomisp-ov2722.c
index 90d0871..da98094 100644
--- a/drivers/staging/media/atomisp/i2c/atomisp-ov2722.c
+++ b/drivers/staging/media/atomisp/i2c/atomisp-ov2722.c
@@ -557,7 +557,7 @@ static int ov2722_g_volatile_ctrl(struct v4l2_ctrl *ctrl)
ret = ov2722_g_fnumber_range(&dev->sd, &ctrl->val);
break;
case V4L2_CID_LINK_FREQ:
- val = ov2722_res[dev->fmt_idx].mipi_freq;
+ val = dev->res->mipi_freq;
if (val == 0)
return -EINVAL;
@@ -782,76 +782,6 @@ static int ov2722_s_power(struct v4l2_subdev *sd, int on)
return ret;
}
-/*
- * distance - calculate the distance
- * @res: resolution
- * @w: width
- * @h: height
- *
- * Get the gap between resolution and w/h.
- * res->width/height smaller than w/h wouldn't be considered.
- * Returns the value of gap or -1 if fail.
- */
-#define LARGEST_ALLOWED_RATIO_MISMATCH 800
-static int distance(struct ov2722_resolution *res, u32 w, u32 h)
-{
- unsigned int w_ratio = (res->width << 13) / w;
- unsigned int h_ratio;
- int match;
-
- if (h == 0)
- return -1;
- h_ratio = (res->height << 13) / h;
- if (h_ratio == 0)
- return -1;
- match = abs(((w_ratio << 13) / h_ratio) - 8192);
-
- if ((w_ratio < 8192) || (h_ratio < 8192) ||
- (match > LARGEST_ALLOWED_RATIO_MISMATCH))
- return -1;
-
- return w_ratio + h_ratio;
-}
-
-/* Return the nearest higher resolution index */
-static int nearest_resolution_index(int w, int h)
-{
- int i;
- int idx = -1;
- int dist;
- int min_dist = INT_MAX;
- struct ov2722_resolution *tmp_res = NULL;
-
- for (i = 0; i < N_RES; i++) {
- tmp_res = &ov2722_res[i];
- dist = distance(tmp_res, w, h);
- if (dist == -1)
- continue;
- if (dist < min_dist) {
- min_dist = dist;
- idx = i;
- }
- }
-
- return idx;
-}
-
-static int get_resolution_index(int w, int h)
-{
- int i;
-
- for (i = 0; i < N_RES; i++) {
- if (w != ov2722_res[i].width)
- continue;
- if (h != ov2722_res[i].height)
- continue;
-
- return i;
- }
-
- return -1;
-}
-
/* TODO: remove it. */
static int startup(struct v4l2_subdev *sd)
{
@@ -866,7 +796,7 @@ static int startup(struct v4l2_subdev *sd)
return ret;
}
- ret = ov2722_write_reg_array(client, ov2722_res[dev->fmt_idx].regs);
+ ret = ov2722_write_reg_array(client, dev->res->regs);
if (ret) {
dev_err(&client->dev, "ov2722 write register err.\n");
return ret;
@@ -882,9 +812,9 @@ static int ov2722_set_fmt(struct v4l2_subdev *sd,
struct v4l2_mbus_framefmt *fmt = &format->format;
struct ov2722_device *dev = to_ov2722_sensor(sd);
struct i2c_client *client = v4l2_get_subdevdata(sd);
+ struct ov2722_resolution *res;
struct camera_mipi_info *ov2722_info = NULL;
int ret = 0;
- int idx;
if (format->pad)
return -EINVAL;
@@ -895,15 +825,16 @@ static int ov2722_set_fmt(struct v4l2_subdev *sd,
return -EINVAL;
mutex_lock(&dev->input_lock);
- idx = nearest_resolution_index(fmt->width, fmt->height);
- if (idx == -1) {
- /* return the largest resolution */
- fmt->width = ov2722_res[N_RES - 1].width;
- fmt->height = ov2722_res[N_RES - 1].height;
- } else {
- fmt->width = ov2722_res[idx].width;
- fmt->height = ov2722_res[idx].height;
- }
+ res = v4l2_find_nearest_size(ov2722_res_preview,
+ ARRAY_SIZE(ov2722_res_preview), width,
+ height, fmt->width, fmt->height);
+ if (!res)
+ res = &ov2722_res_preview[N_RES - 1];
+
+ fmt->width = res->width;
+ fmt->height = res->height;
+ dev->res = res;
+
fmt->code = MEDIA_BUS_FMT_SGRBG10_1X10;
if (format->which == V4L2_SUBDEV_FORMAT_TRY) {
sd_state->pads->try_fmt = *fmt;
@@ -911,15 +842,9 @@ static int ov2722_set_fmt(struct v4l2_subdev *sd,
return 0;
}
- dev->fmt_idx = get_resolution_index(fmt->width, fmt->height);
- if (dev->fmt_idx == -1) {
- dev_err(&client->dev, "get resolution fail\n");
- mutex_unlock(&dev->input_lock);
- return -EINVAL;
- }
- dev->pixels_per_line = ov2722_res[dev->fmt_idx].pixels_per_line;
- dev->lines_per_frame = ov2722_res[dev->fmt_idx].lines_per_frame;
+ dev->pixels_per_line = dev->res->pixels_per_line;
+ dev->lines_per_frame = dev->res->lines_per_frame;
ret = startup(sd);
if (ret) {
@@ -950,8 +875,7 @@ static int ov2722_set_fmt(struct v4l2_subdev *sd,
}
}
- ret = ov2722_get_intg_factor(client, ov2722_info,
- &ov2722_res[dev->fmt_idx]);
+ ret = ov2722_get_intg_factor(client, ov2722_info, dev->res);
if (ret)
dev_err(&client->dev, "failed to get integration_factor\n");
@@ -972,8 +896,8 @@ static int ov2722_get_fmt(struct v4l2_subdev *sd,
if (!fmt)
return -EINVAL;
- fmt->width = ov2722_res[dev->fmt_idx].width;
- fmt->height = ov2722_res[dev->fmt_idx].height;
+ fmt->width = dev->res->width;
+ fmt->height = dev->res->height;
fmt->code = MEDIA_BUS_FMT_SBGGR10_1X10;
return 0;
@@ -1098,7 +1022,7 @@ static int ov2722_g_frame_interval(struct v4l2_subdev *sd,
struct ov2722_device *dev = to_ov2722_sensor(sd);
interval->interval.numerator = 1;
- interval->interval.denominator = ov2722_res[dev->fmt_idx].fps;
+ interval->interval.denominator = dev->res->fps;
return 0;
}
@@ -1136,7 +1060,7 @@ static int ov2722_g_skip_frames(struct v4l2_subdev *sd, u32 *frames)
struct ov2722_device *dev = to_ov2722_sensor(sd);
mutex_lock(&dev->input_lock);
- *frames = ov2722_res[dev->fmt_idx].skip_frames;
+ *frames = dev->res->skip_frames;
mutex_unlock(&dev->input_lock);
return 0;
@@ -1220,7 +1144,7 @@ static int ov2722_probe(struct i2c_client *client)
mutex_init(&dev->input_lock);
- dev->fmt_idx = 0;
+ dev->res = &ov2722_res_preview[0];
v4l2_i2c_subdev_init(&dev->sd, client, &ov2722_ops);
ovpdev = gmin_camera_platform_data(&dev->sd,
diff --git a/drivers/staging/media/atomisp/i2c/gc0310.h b/drivers/staging/media/atomisp/i2c/gc0310.h
index 2fe3de1..db643eb 100644
--- a/drivers/staging/media/atomisp/i2c/gc0310.h
+++ b/drivers/staging/media/atomisp/i2c/gc0310.h
@@ -150,8 +150,7 @@ struct gc0310_device {
struct camera_sensor_platform_data *platform_data;
int vt_pix_clk_freq_mhz;
- int fmt_idx;
- u8 res;
+ struct gc0310_resolution *res;
u8 type;
};
diff --git a/drivers/staging/media/atomisp/i2c/gc2235.h b/drivers/staging/media/atomisp/i2c/gc2235.h
index ba5db12..806be5d 100644
--- a/drivers/staging/media/atomisp/i2c/gc2235.h
+++ b/drivers/staging/media/atomisp/i2c/gc2235.h
@@ -158,11 +158,10 @@ struct gc2235_device {
struct v4l2_mbus_framefmt format;
struct mutex input_lock;
struct v4l2_ctrl_handler ctrl_handler;
+ struct gc2235_resolution *res;
struct camera_sensor_platform_data *platform_data;
int vt_pix_clk_freq_mhz;
- int fmt_idx;
- u8 res;
u8 type;
};
diff --git a/drivers/staging/media/atomisp/i2c/ov2680.h b/drivers/staging/media/atomisp/i2c/ov2680.h
index 874115f..4e35119 100644
--- a/drivers/staging/media/atomisp/i2c/ov2680.h
+++ b/drivers/staging/media/atomisp/i2c/ov2680.h
@@ -141,7 +141,6 @@ struct regval_list {
};
struct ov2680_resolution {
- u8 *desc;
const struct ov2680_reg *regs;
int res;
int width;
@@ -154,7 +153,6 @@ struct ov2680_resolution {
u8 bin_factor_x;
u8 bin_factor_y;
u8 bin_mode;
- bool used;
};
struct ov2680_format {
@@ -169,15 +167,14 @@ struct ov2680_format {
struct ov2680_device {
struct v4l2_subdev sd;
struct media_pad pad;
- struct v4l2_mbus_framefmt format;
struct mutex input_lock;
struct v4l2_ctrl_handler ctrl_handler;
+ struct ov2680_resolution *res;
struct camera_sensor_platform_data *platform_data;
- int vt_pix_clk_freq_mhz;
- int fmt_idx;
- int run_mode;
- u8 res;
- u8 type;
+ bool power_on;
+ u16 exposure;
+ u16 gain;
+ u16 digitgain;
};
/**
@@ -283,14 +280,11 @@ static struct ov2680_reg const ov2680_global_setting[] = {
{}
};
-#if 0 /* None of the definitions below are used currently */
/*
* 176x144 30fps VBlanking 1lane 10Bit (binning)
*/
static struct ov2680_reg const ov2680_QCIF_30fps[] = {
{0x3086, 0x01},
- {0x3501, 0x24},
- {0x3502, 0x40},
{0x370a, 0x23},
{0x3801, 0xa0},
{0x3802, 0x00},
@@ -305,8 +299,6 @@ static struct ov2680_reg const ov2680_QCIF_30fps[] = {
{0x380b, 0xa0},
{0x380c, 0x06},
{0x380d, 0xb0},
- {0x380e, 0x02},
- {0x380f, 0x84},
{0x3810, 0x00},
{0x3811, 0x04},
{0x3812, 0x00},
@@ -334,8 +326,6 @@ static struct ov2680_reg const ov2680_QCIF_30fps[] = {
*/
static struct ov2680_reg const ov2680_CIF_30fps[] = {
{0x3086, 0x01},
- {0x3501, 0x24},
- {0x3502, 0x40},
{0x370a, 0x23},
{0x3801, 0xa0},
{0x3802, 0x00},
@@ -350,8 +340,6 @@ static struct ov2680_reg const ov2680_CIF_30fps[] = {
{0x380b, 0x30},
{0x380c, 0x06},
{0x380d, 0xb0},
- {0x380e, 0x02},
- {0x380f, 0x84},
{0x3810, 0x00},
{0x3811, 0x04},
{0x3812, 0x00},
@@ -377,8 +365,6 @@ static struct ov2680_reg const ov2680_CIF_30fps[] = {
*/
static struct ov2680_reg const ov2680_QVGA_30fps[] = {
{0x3086, 0x01},
- {0x3501, 0x24},
- {0x3502, 0x40},
{0x370a, 0x23},
{0x3801, 0xa0},
{0x3802, 0x00},
@@ -393,8 +379,6 @@ static struct ov2680_reg const ov2680_QVGA_30fps[] = {
{0x380b, 0x00},
{0x380c, 0x06},
{0x380d, 0xb0},
- {0x380e, 0x02},
- {0x380f, 0x84},
{0x3810, 0x00},
{0x3811, 0x04},
{0x3812, 0x00},
@@ -420,8 +404,6 @@ static struct ov2680_reg const ov2680_QVGA_30fps[] = {
*/
static struct ov2680_reg const ov2680_656x496_30fps[] = {
{0x3086, 0x01},
- {0x3501, 0x24},
- {0x3502, 0x40},
{0x370a, 0x23},
{0x3801, 0xa0},
{0x3802, 0x00},
@@ -436,8 +418,6 @@ static struct ov2680_reg const ov2680_656x496_30fps[] = {
{0x380b, 0xf0},
{0x380c, 0x06},
{0x380d, 0xb0},
- {0x380e, 0x02},
- {0x380f, 0x84},
{0x3810, 0x00},
{0x3811, 0x04},
{0x3812, 0x00},
@@ -459,12 +439,10 @@ static struct ov2680_reg const ov2680_656x496_30fps[] = {
};
/*
- * 800x600 30fps VBlanking 1lane 10Bit (binning)
+ * 720x592 30fps VBlanking 1lane 10Bit (binning)
*/
static struct ov2680_reg const ov2680_720x592_30fps[] = {
{0x3086, 0x01},
- {0x3501, 0x26},
- {0x3502, 0x40},
{0x370a, 0x23},
{0x3801, 0x00}, // X_ADDR_START;
{0x3802, 0x00},
@@ -479,8 +457,6 @@ static struct ov2680_reg const ov2680_720x592_30fps[] = {
{0x380b, 0x50}, // Y_OUTPUT_SIZE;
{0x380c, 0x06},
{0x380d, 0xac}, // HTS;
- {0x380e, 0x02},
- {0x380f, 0x84}, // VTS;
{0x3810, 0x00},
{0x3811, 0x00},
{0x3812, 0x00},
@@ -508,8 +484,6 @@ static struct ov2680_reg const ov2680_720x592_30fps[] = {
*/
static struct ov2680_reg const ov2680_800x600_30fps[] = {
{0x3086, 0x01},
- {0x3501, 0x26},
- {0x3502, 0x40},
{0x370a, 0x23},
{0x3801, 0x00},
{0x3802, 0x00},
@@ -524,8 +498,6 @@ static struct ov2680_reg const ov2680_800x600_30fps[] = {
{0x380b, 0x58},
{0x380c, 0x06},
{0x380d, 0xac},
- {0x380e, 0x02},
- {0x380f, 0x84},
{0x3810, 0x00},
{0x3811, 0x00},
{0x3812, 0x00},
@@ -551,8 +523,6 @@ static struct ov2680_reg const ov2680_800x600_30fps[] = {
*/
static struct ov2680_reg const ov2680_720p_30fps[] = {
{0x3086, 0x00},
- {0x3501, 0x48},
- {0x3502, 0xe0},
{0x370a, 0x21},
{0x3801, 0xa0},
{0x3802, 0x00},
@@ -567,8 +537,6 @@ static struct ov2680_reg const ov2680_720p_30fps[] = {
{0x380b, 0xe0},
{0x380c, 0x06},
{0x380d, 0xa8},
- {0x380e, 0x05},
- {0x380f, 0x0e},
{0x3810, 0x00},
{0x3811, 0x08},
{0x3812, 0x00},
@@ -594,8 +562,6 @@ static struct ov2680_reg const ov2680_720p_30fps[] = {
*/
static struct ov2680_reg const ov2680_1296x976_30fps[] = {
{0x3086, 0x00},
- {0x3501, 0x48},
- {0x3502, 0xe0},
{0x370a, 0x21},
{0x3801, 0xa0},
{0x3802, 0x00},
@@ -610,8 +576,6 @@ static struct ov2680_reg const ov2680_1296x976_30fps[] = {
{0x380b, 0xd0},
{0x380c, 0x06},
{0x380d, 0xa8},
- {0x380e, 0x05},
- {0x380f, 0x0e},
{0x3810, 0x00},
{0x3811, 0x08},
{0x3812, 0x00},
@@ -637,8 +601,6 @@ static struct ov2680_reg const ov2680_1296x976_30fps[] = {
*/
static struct ov2680_reg const ov2680_1456x1096_30fps[] = {
{0x3086, 0x00},
- {0x3501, 0x48},
- {0x3502, 0xe0},
{0x370a, 0x21},
{0x3801, 0x90},
{0x3802, 0x00},
@@ -653,8 +615,6 @@ static struct ov2680_reg const ov2680_1456x1096_30fps[] = {
{0x380b, 0x48},
{0x380c, 0x06},
{0x380d, 0xa8},
- {0x380e, 0x05},
- {0x380f, 0x0e},
{0x3810, 0x00},
{0x3811, 0x08},
{0x3812, 0x00},
@@ -674,7 +634,6 @@ static struct ov2680_reg const ov2680_1456x1096_30fps[] = {
// {0x5090, 0x0c},
{}
};
-#endif
/*
*1616x916 30fps VBlanking 1lane 10bit
@@ -682,8 +641,6 @@ static struct ov2680_reg const ov2680_1456x1096_30fps[] = {
static struct ov2680_reg const ov2680_1616x916_30fps[] = {
{0x3086, 0x00},
- {0x3501, 0x48},
- {0x3502, 0xe0},
{0x370a, 0x21},
{0x3801, 0x00},
{0x3802, 0x00},
@@ -698,8 +655,6 @@ static struct ov2680_reg const ov2680_1616x916_30fps[] = {
{0x380b, 0x94},
{0x380c, 0x06},
{0x380d, 0xa8},
- {0x380e, 0x05},
- {0x380f, 0x0e},
{0x3810, 0x00},
{0x3811, 0x00},
{0x3812, 0x00},
@@ -721,13 +676,10 @@ static struct ov2680_reg const ov2680_1616x916_30fps[] = {
};
/*
- * 1612x1212 30fps VBlanking 1lane 10Bit
+ * 1616x1082 30fps VBlanking 1lane 10Bit
*/
-#if 0
static struct ov2680_reg const ov2680_1616x1082_30fps[] = {
{0x3086, 0x00},
- {0x3501, 0x48},
- {0x3502, 0xe0},
{0x370a, 0x21},
{0x3801, 0x00},
{0x3802, 0x00},
@@ -742,8 +694,6 @@ static struct ov2680_reg const ov2680_1616x1082_30fps[] = {
{0x380b, 0x3a},
{0x380c, 0x06},
{0x380d, 0xa8},
- {0x380e, 0x05},
- {0x380f, 0x0e},
{0x3810, 0x00},
{0x3811, 0x00},
{0x3812, 0x00},
@@ -763,14 +713,12 @@ static struct ov2680_reg const ov2680_1616x1082_30fps[] = {
{0x5081, 0x41},
{}
};
-#endif
+
/*
* 1616x1216 30fps VBlanking 1lane 10Bit
*/
static struct ov2680_reg const ov2680_1616x1216_30fps[] = {
{0x3086, 0x00},
- {0x3501, 0x48},
- {0x3502, 0xe0},
{0x370a, 0x21},
{0x3801, 0x00},
{0x3802, 0x00},
@@ -785,8 +733,6 @@ static struct ov2680_reg const ov2680_1616x1216_30fps[] = {
{0x380b, 0xc0},//c0},
{0x380c, 0x06},
{0x380d, 0xa8},
- {0x380e, 0x05},
- {0x380f, 0x0e},
{0x3810, 0x00},
{0x3811, 0x00},
{0x3812, 0x00},
@@ -809,12 +755,10 @@ static struct ov2680_reg const ov2680_1616x1216_30fps[] = {
static struct ov2680_resolution ov2680_res_preview[] = {
{
- .desc = "ov2680_1616x1216_30fps",
.width = 1616,
.height = 1216,
.pix_clk_freq = 66,
.fps = 30,
- .used = 0,
.pixels_per_line = 1698,//1704,
.lines_per_frame = 1294,
.bin_factor_x = 0,
@@ -824,12 +768,23 @@ static struct ov2680_resolution ov2680_res_preview[] = {
.regs = ov2680_1616x1216_30fps,
},
{
- .desc = "ov2680_1616x916_30fps",
+ .width = 1616,
+ .height = 1082,
+ .pix_clk_freq = 66,
+ .fps = 30,
+ .pixels_per_line = 1698,//1704,
+ .lines_per_frame = 1294,
+ .bin_factor_x = 0,
+ .bin_factor_y = 0,
+ .bin_mode = 0,
+ .skip_frames = 3,
+ .regs = ov2680_1616x1082_30fps,
+ },
+ {
.width = 1616,
.height = 916,
.fps = 30,
.pix_clk_freq = 66,
- .used = 0,
.pixels_per_line = 1698,//1704,
.lines_per_frame = 1294,
.bin_factor_x = 0,
@@ -838,11 +793,125 @@ static struct ov2680_resolution ov2680_res_preview[] = {
.skip_frames = 3,
.regs = ov2680_1616x916_30fps,
},
+ {
+ .width = 1456,
+ .height = 1096,
+ .fps = 30,
+ .pix_clk_freq = 66,
+ .pixels_per_line = 1698,//1704,
+ .lines_per_frame = 1294,
+ .bin_factor_x = 0,
+ .bin_factor_y = 0,
+ .bin_mode = 0,
+ .skip_frames = 3,
+ .regs = ov2680_1456x1096_30fps,
+ },
+ {
+ .width = 1296,
+ .height = 976,
+ .fps = 30,
+ .pix_clk_freq = 66,
+ .pixels_per_line = 1698,//1704,
+ .lines_per_frame = 1294,
+ .bin_factor_x = 0,
+ .bin_factor_y = 0,
+ .bin_mode = 0,
+ .skip_frames = 3,
+ .regs = ov2680_1296x976_30fps,
+ },
+ {
+ .width = 1280,
+ .height = 720,
+ .fps = 60,
+ .pix_clk_freq = 66,
+ .pixels_per_line = 1698,//1704,
+ .lines_per_frame = 1294,
+ .bin_factor_x = 0,
+ .bin_factor_y = 0,
+ .bin_mode = 0,
+ .skip_frames = 3,
+ .regs = ov2680_720p_30fps,
+ },
+ {
+ .width = 800,
+ .height = 600,
+ .fps = 60,
+ .pix_clk_freq = 66,
+ .pixels_per_line = 1698,//1704,
+ .lines_per_frame = 1294,
+ .bin_factor_x = 0,
+ .bin_factor_y = 0,
+ .bin_mode = 0,
+ .skip_frames = 3,
+ .regs = ov2680_800x600_30fps,
+ },
+ {
+ .width = 720,
+ .height = 592,
+ .fps = 60,
+ .pix_clk_freq = 66,
+ .pixels_per_line = 1698,//1704,
+ .lines_per_frame = 1294,
+ .bin_factor_x = 0,
+ .bin_factor_y = 0,
+ .bin_mode = 0,
+ .skip_frames = 3,
+ .regs = ov2680_720x592_30fps,
+ },
+ {
+ .width = 656,
+ .height = 496,
+ .fps = 60,
+ .pix_clk_freq = 66,
+ .pixels_per_line = 1698,//1704,
+ .lines_per_frame = 1294,
+ .bin_factor_x = 0,
+ .bin_factor_y = 0,
+ .bin_mode = 0,
+ .skip_frames = 3,
+ .regs = ov2680_656x496_30fps,
+ },
+ {
+ .width = 336,
+ .height = 256,
+ .fps = 60,
+ .pix_clk_freq = 66,
+ .pixels_per_line = 1698,//1704,
+ .lines_per_frame = 1294,
+ .bin_factor_x = 0,
+ .bin_factor_y = 0,
+ .bin_mode = 0,
+ .skip_frames = 3,
+ .regs = ov2680_QVGA_30fps,
+ },
+ {
+ .width = 352,
+ .height = 288,
+ .fps = 60,
+ .pix_clk_freq = 66,
+ .pixels_per_line = 1698,//1704,
+ .lines_per_frame = 1294,
+ .bin_factor_x = 0,
+ .bin_factor_y = 0,
+ .bin_mode = 0,
+ .skip_frames = 3,
+ .regs = ov2680_CIF_30fps,
+ },
+ {
+ .width = 176,
+ .height = 144,
+ .fps = 60,
+ .pix_clk_freq = 66,
+ .pixels_per_line = 1698,//1704,
+ .lines_per_frame = 1294,
+ .bin_factor_x = 0,
+ .bin_factor_y = 0,
+ .bin_mode = 0,
+ .skip_frames = 3,
+ .regs = ov2680_QCIF_30fps,
+ },
};
#define N_RES_PREVIEW (ARRAY_SIZE(ov2680_res_preview))
-static struct ov2680_resolution *ov2680_res = ov2680_res_preview;
-static unsigned long N_RES = N_RES_PREVIEW;
-
#endif
diff --git a/drivers/staging/media/atomisp/i2c/ov2722.h b/drivers/staging/media/atomisp/i2c/ov2722.h
index 7b0debb..d6e2510 100644
--- a/drivers/staging/media/atomisp/i2c/ov2722.h
+++ b/drivers/staging/media/atomisp/i2c/ov2722.h
@@ -201,14 +201,13 @@ struct ov2722_device {
struct media_pad pad;
struct v4l2_mbus_framefmt format;
struct mutex input_lock;
+ struct ov2722_resolution *res;
struct camera_sensor_platform_data *platform_data;
int vt_pix_clk_freq_mhz;
- int fmt_idx;
int run_mode;
u16 pixels_per_line;
u16 lines_per_frame;
- u8 res;
u8 type;
struct v4l2_ctrl_handler ctrl_handler;
diff --git a/drivers/staging/media/atomisp/pci/atomisp_cmd.c b/drivers/staging/media/atomisp/pci/atomisp_cmd.c
index 366161c..97d5a52 100644
--- a/drivers/staging/media/atomisp/pci/atomisp_cmd.c
+++ b/drivers/staging/media/atomisp/pci/atomisp_cmd.c
@@ -857,7 +857,8 @@ static struct atomisp_video_pipe *__atomisp_get_pipe(
} else if (asd->run_mode->val == ATOMISP_RUN_MODE_VIDEO) {
/* For online video or SDV video pipe. */
if (css_pipe_id == IA_CSS_PIPE_ID_VIDEO ||
- css_pipe_id == IA_CSS_PIPE_ID_COPY) {
+ css_pipe_id == IA_CSS_PIPE_ID_COPY ||
+ css_pipe_id == IA_CSS_PIPE_ID_YUVPP) {
if (buf_type == IA_CSS_BUFFER_TYPE_OUTPUT_FRAME)
return &asd->video_out_video_capture;
return &asd->video_out_preview;
@@ -865,7 +866,8 @@ static struct atomisp_video_pipe *__atomisp_get_pipe(
} else if (asd->run_mode->val == ATOMISP_RUN_MODE_PREVIEW) {
/* For online preview or ZSL preview pipe. */
if (css_pipe_id == IA_CSS_PIPE_ID_PREVIEW ||
- css_pipe_id == IA_CSS_PIPE_ID_COPY)
+ css_pipe_id == IA_CSS_PIPE_ID_COPY ||
+ css_pipe_id == IA_CSS_PIPE_ID_YUVPP)
return &asd->video_out_preview;
}
/* For capture pipe. */
@@ -1046,13 +1048,8 @@ void atomisp_buf_done(struct atomisp_sub_device *asd, int error,
asd->pending_capture_request--;
- if (IS_ISP2401)
- asd->re_trigger_capture = false;
-
dev_dbg(isp->dev, "Trigger capture again for new buffer. err=%d\n",
err);
- } else if (IS_ISP2401) {
- asd->re_trigger_capture = true;
}
break;
case IA_CSS_BUFFER_TYPE_OUTPUT_FRAME:
@@ -1474,7 +1471,7 @@ void atomisp_wdt_work(struct work_struct *work)
wdt_work);
int i;
unsigned int pipe_wdt_cnt[MAX_STREAM_NUM][4] = { {0} };
- bool css_recover = false;
+ bool css_recover = true;
rt_mutex_lock(&isp->mutex);
if (!atomisp_streaming_count(isp)) {
@@ -1487,12 +1484,7 @@ void atomisp_wdt_work(struct work_struct *work)
dev_err(isp->dev, "timeout %d of %d\n",
atomic_read(&isp->wdt_count) + 1,
ATOMISP_ISP_MAX_TIMEOUT_COUNT);
-
- if (atomic_inc_return(&isp->wdt_count) < ATOMISP_ISP_MAX_TIMEOUT_COUNT)
- css_recover = true;
} else {
- css_recover = true;
-
for (i = 0; i < isp->num_of_streams; i++) {
struct atomisp_sub_device *asd = &isp->asd[i];
@@ -1715,6 +1707,12 @@ void atomisp_wdt_refresh_pipe(struct atomisp_video_pipe *pipe,
{
unsigned long next;
+ if (!pipe->asd) {
+ dev_err(pipe->isp->dev, "%s(): asd is NULL, device is %s\n",
+ __func__, pipe->vdev.name);
+ return;
+ }
+
if (delay != ATOMISP_WDT_KEEP_CURRENT_DELAY)
pipe->wdt_duration = delay;
@@ -1777,6 +1775,12 @@ void atomisp_wdt_refresh(struct atomisp_sub_device *asd, unsigned int delay)
/* ISP2401 */
void atomisp_wdt_stop_pipe(struct atomisp_video_pipe *pipe, bool sync)
{
+ if (!pipe->asd) {
+ dev_err(pipe->isp->dev, "%s(): asd is NULL, device is %s\n",
+ __func__, pipe->vdev.name);
+ return;
+ }
+
if (!atomisp_is_wdt_running(pipe))
return;
@@ -1869,7 +1873,7 @@ irqreturn_t atomisp_isr_thread(int irq, void *isp_ptr)
/*
* The standard CSS2.0 API tells the following calling sequence of
* dequeue ready buffers:
- * while (ia_css_dequeue_event(...)) {
+ * while (ia_css_dequeue_psys_event(...)) {
* switch (event.type) {
* ...
* ia_css_pipe_dequeue_buffer()
@@ -1920,22 +1924,6 @@ irqreturn_t atomisp_isr_thread(int irq, void *isp_ptr)
}
/*
- * utils for buffer allocation/free
- */
-
-int atomisp_get_frame_pgnr(struct atomisp_device *isp,
- const struct ia_css_frame *frame, u32 *p_pgnr)
-{
- if (!frame) {
- dev_err(isp->dev, "%s: NULL frame pointer ERROR.\n", __func__);
- return -EINVAL;
- }
-
- *p_pgnr = DIV_ROUND_UP(frame->data_bytes, PAGE_SIZE);
- return 0;
-}
-
-/*
* Get internal fmt according to V4L2 fmt
*/
static enum ia_css_frame_format
@@ -1968,9 +1956,11 @@ v4l2_fmt_to_sh_fmt(u32 fmt)
return IA_CSS_FRAME_FORMAT_RGBA888;
case V4L2_PIX_FMT_RGB565:
return IA_CSS_FRAME_FORMAT_RGB565;
+#if 0
case V4L2_PIX_FMT_JPEG:
case V4L2_PIX_FMT_CUSTOM_M10MO_RAW:
return IA_CSS_FRAME_FORMAT_BINARY_8;
+#endif
case V4L2_PIX_FMT_SBGGR16:
case V4L2_PIX_FMT_SBGGR10:
case V4L2_PIX_FMT_SGBRG10:
@@ -2022,7 +2012,7 @@ static int raw_output_format_match_input(u32 input, u32 output)
return -EINVAL;
}
-static u32 get_pixel_depth(u32 pixelformat)
+u32 atomisp_get_pixel_depth(u32 pixelformat)
{
switch (pixelformat) {
case V4L2_PIX_FMT_YUV420:
@@ -3619,18 +3609,10 @@ int atomisp_cp_lsc_table(struct atomisp_sub_device *asd,
}
/* Shading table size per color */
- if (!IS_ISP2401) {
- if (st->width > ISP2400_SH_CSS_MAX_SCTBL_WIDTH_PER_COLOR ||
- st->height > ISP2400_SH_CSS_MAX_SCTBL_HEIGHT_PER_COLOR) {
- dev_err(asd->isp->dev, "shading table w/h validate failed!");
- return -EINVAL;
- }
- } else {
- if (st->width > ISP2401_SH_CSS_MAX_SCTBL_WIDTH_PER_COLOR ||
- st->height > ISP2401_SH_CSS_MAX_SCTBL_HEIGHT_PER_COLOR) {
- dev_err(asd->isp->dev, "shading table w/h validate failed!");
- return -EINVAL;
- }
+ if (st->width > SH_CSS_MAX_SCTBL_WIDTH_PER_COLOR ||
+ st->height > SH_CSS_MAX_SCTBL_HEIGHT_PER_COLOR) {
+ dev_err(asd->isp->dev, "shading table w/h validate failed!");
+ return -EINVAL;
}
shading_table = atomisp_css_shading_table_alloc(st->width, st->height);
@@ -4109,6 +4091,12 @@ void atomisp_handle_parameter_and_buffer(struct atomisp_video_pipe *pipe)
unsigned long irqflags;
bool need_to_enqueue_buffer = false;
+ if (!asd) {
+ dev_err(pipe->isp->dev, "%s(): asd is NULL, device is %s\n",
+ __func__, pipe->vdev.name);
+ return;
+ }
+
if (atomisp_is_vf_pipe(pipe))
return;
@@ -4196,6 +4184,12 @@ int atomisp_set_parameters(struct video_device *vdev,
struct atomisp_css_params *css_param = &asd->params.css_param;
int ret;
+ if (!asd) {
+ dev_err(pipe->isp->dev, "%s(): asd is NULL, device is %s\n",
+ __func__, vdev->name);
+ return -EINVAL;
+ }
+
if (!asd->stream_env[ATOMISP_INPUT_STREAM_GENERAL].stream) {
dev_err(asd->isp->dev, "%s: internal error!\n", __func__);
return -EINVAL;
@@ -4792,15 +4786,6 @@ int atomisp_get_sensor_mode_data(struct atomisp_sub_device *asd,
return 0;
}
-int atomisp_get_fmt(struct video_device *vdev, struct v4l2_format *f)
-{
- struct atomisp_video_pipe *pipe = atomisp_to_video_pipe(vdev);
-
- f->fmt.pix = pipe->pix;
-
- return 0;
-}
-
static void __atomisp_update_stream_env(struct atomisp_sub_device *asd,
u16 stream_index, struct atomisp_input_stream_info *stream_info)
{
@@ -4857,6 +4842,12 @@ int atomisp_try_fmt(struct video_device *vdev, struct v4l2_pix_format *f,
int source_pad = atomisp_subdev_source_pad(vdev);
int ret;
+ if (!asd) {
+ dev_err(isp->dev, "%s(): asd is NULL, device is %s\n",
+ __func__, vdev->name);
+ return -EINVAL;
+ }
+
if (!isp->inputs[asd->input_curr].camera)
return -EINVAL;
@@ -4943,7 +4934,7 @@ atomisp_try_fmt_file(struct atomisp_device *isp, struct v4l2_format *f)
return -EINVAL;
}
- depth = get_pixel_depth(pixelformat);
+ depth = atomisp_get_pixel_depth(pixelformat);
if (field == V4L2_FIELD_ANY) {
field = V4L2_FIELD_NONE;
@@ -5133,11 +5124,11 @@ static int css_input_resolution_changed(struct atomisp_sub_device *asd,
dev_dbg(asd->isp->dev, "css_input_resolution_changed to %ux%u\n",
ffmt->width, ffmt->height);
-#if defined(ISP2401_NEW_INPUT_SYSTEM)
- atomisp_css_input_set_two_pixels_per_clock(asd, false);
-#else
- atomisp_css_input_set_two_pixels_per_clock(asd, true);
-#endif
+ if (IS_ISP2401)
+ atomisp_css_input_set_two_pixels_per_clock(asd, false);
+ else
+ atomisp_css_input_set_two_pixels_per_clock(asd, true);
+
if (asd->continuous_mode->val) {
/* Note for all checks: ffmt includes pad_w+pad_h */
if (asd->run_mode->val == ATOMISP_RUN_MODE_VIDEO ||
@@ -5194,10 +5185,17 @@ static int atomisp_set_fmt_to_isp(struct video_device *vdev,
int (*configure_pp_input)(struct atomisp_sub_device *asd,
unsigned int width, unsigned int height) =
configure_pp_input_nop;
- u16 stream_index = atomisp_source_pad_to_stream_id(asd, source_pad);
+ u16 stream_index;
const struct atomisp_in_fmt_conv *fc;
int ret, i;
+ if (!asd) {
+ dev_err(isp->dev, "%s(): asd is NULL, device is %s\n",
+ __func__, vdev->name);
+ return -EINVAL;
+ }
+ stream_index = atomisp_source_pad_to_stream_id(asd, source_pad);
+
v4l2_fh_init(&fh.vfh, vdev);
isp_sink_crop = atomisp_subdev_get_rect(
@@ -5417,9 +5415,9 @@ static int atomisp_set_fmt_to_isp(struct video_device *vdev,
else
ret = get_frame_info(asd, output_info);
if (ret) {
- dev_err(isp->dev, "get_frame_info %ux%u (padded to %u)\n",
- pix->width, pix->height, pix->bytesperline);
- return -EINVAL;
+ dev_err(isp->dev, "__get_frame_info %ux%u (padded to %u) returned %d\n",
+ pix->width, pix->height, pix->bytesperline, ret);
+ return ret;
}
atomisp_update_grid_info(asd, pipe_id, source_pad);
@@ -5465,9 +5463,14 @@ static void atomisp_get_dis_envelop(struct atomisp_sub_device *asd,
static void atomisp_check_copy_mode(struct atomisp_sub_device *asd,
int source_pad, struct v4l2_pix_format *f)
{
-#if defined(ISP2401_NEW_INPUT_SYSTEM)
struct v4l2_mbus_framefmt *sink, *src;
+ if (!IS_ISP2401) {
+ /* Only used for the new input system */
+ asd->copy_mode = false;
+ return;
+ }
+
sink = atomisp_subdev_get_ffmt(&asd->subdev, NULL,
V4L2_SUBDEV_FORMAT_ACTIVE, ATOMISP_SUBDEV_PAD_SINK);
src = atomisp_subdev_get_ffmt(&asd->subdev, NULL,
@@ -5481,8 +5484,6 @@ static void atomisp_check_copy_mode(struct atomisp_sub_device *asd,
sensor[asd->sensor_curr].stream_num > 1)))
asd->copy_mode = true;
else
-#endif
- /* Only used for the new input system */
asd->copy_mode = false;
dev_dbg(asd->isp->dev, "copy_mode: %d\n", asd->copy_mode);
@@ -5493,7 +5494,8 @@ static int atomisp_set_fmt_to_snr(struct video_device *vdev,
unsigned int padding_w, unsigned int padding_h,
unsigned int dvs_env_w, unsigned int dvs_env_h)
{
- struct atomisp_sub_device *asd = atomisp_to_video_pipe(vdev)->asd;
+ struct atomisp_video_pipe *pipe = atomisp_to_video_pipe(vdev);
+ struct atomisp_sub_device *asd = pipe->asd;
const struct atomisp_format_bridge *format;
struct v4l2_subdev_pad_config pad_cfg;
struct v4l2_subdev_state pad_state = {
@@ -5504,7 +5506,7 @@ static int atomisp_set_fmt_to_snr(struct video_device *vdev,
};
struct v4l2_mbus_framefmt *ffmt = &vformat.format;
struct v4l2_mbus_framefmt *req_ffmt;
- struct atomisp_device *isp = asd->isp;
+ struct atomisp_device *isp;
struct atomisp_input_stream_info *stream_info =
(struct atomisp_input_stream_info *)ffmt->reserved;
u16 stream_index = ATOMISP_INPUT_STREAM_GENERAL;
@@ -5512,6 +5514,14 @@ static int atomisp_set_fmt_to_snr(struct video_device *vdev,
struct v4l2_subdev_fh fh;
int ret;
+ if (!asd) {
+ dev_err(pipe->isp->dev, "%s(): asd is NULL, device is %s\n",
+ __func__, vdev->name);
+ return -EINVAL;
+ }
+
+ isp = asd->isp;
+
v4l2_fh_init(&fh.vfh, vdev);
stream_index = atomisp_source_pad_to_stream_id(asd, source_pad);
@@ -5540,6 +5550,10 @@ static int atomisp_set_fmt_to_snr(struct video_device *vdev,
pad, set_fmt, &pad_state, &vformat);
if (ret)
return ret;
+
+ dev_dbg(isp->dev, "video dis: sensor width: %d, height: %d\n",
+ ffmt->width, ffmt->height);
+
if (ffmt->width < req_ffmt->width ||
ffmt->height < req_ffmt->height) {
req_ffmt->height -= dvs_env_h;
@@ -5550,8 +5564,6 @@ static int atomisp_set_fmt_to_snr(struct video_device *vdev,
asd->params.video_dis_en = false;
}
}
- dev_dbg(isp->dev, "sensor width: %d, height: %d\n",
- ffmt->width, ffmt->height);
vformat.which = V4L2_SUBDEV_FORMAT_ACTIVE;
ret = v4l2_subdev_call(isp->inputs[asd->input_curr].camera, pad,
set_fmt, NULL, &vformat);
@@ -5590,18 +5602,28 @@ int atomisp_set_fmt(struct video_device *vdev, struct v4l2_format *f)
const struct atomisp_format_bridge *format_bridge;
const struct atomisp_format_bridge *snr_format_bridge;
struct ia_css_frame_info output_info, raw_output_info;
- struct v4l2_pix_format snr_fmt = f->fmt.pix;
- struct v4l2_pix_format backup_fmt = snr_fmt, s_fmt;
+ struct v4l2_pix_format snr_fmt;
+ struct v4l2_pix_format backup_fmt, s_fmt;
unsigned int dvs_env_w = 0, dvs_env_h = 0;
unsigned int padding_w = pad_w, padding_h = pad_h;
bool res_overflow = false, crop_needs_override = false;
struct v4l2_mbus_framefmt *isp_sink_fmt;
struct v4l2_mbus_framefmt isp_source_fmt = {0};
+ struct v4l2_subdev_format vformat = {
+ .which = V4L2_SUBDEV_FORMAT_ACTIVE,
+ };
+ struct v4l2_mbus_framefmt *ffmt = &vformat.format;
struct v4l2_rect isp_sink_crop;
u16 source_pad = atomisp_subdev_source_pad(vdev);
struct v4l2_subdev_fh fh;
int ret;
+ if (!asd) {
+ dev_err(isp->dev, "%s(): asd is NULL, device is %s\n",
+ __func__, vdev->name);
+ return -EINVAL;
+ }
+
if (source_pad >= ATOMISP_SUBDEV_PADS_NUM)
return -EINVAL;
@@ -5621,9 +5643,38 @@ int atomisp_set_fmt(struct video_device *vdev, struct v4l2_format *f)
if (!format_bridge)
return -EINVAL;
+ /* Currently, raw formats are broken!!! */
+
+ if (format_bridge->sh_fmt == IA_CSS_FRAME_FORMAT_RAW) {
+ f->fmt.pix.pixelformat = V4L2_PIX_FMT_YUV420;
+
+ format_bridge = atomisp_get_format_bridge(f->fmt.pix.pixelformat);
+ if (!format_bridge)
+ return -EINVAL;
+ }
pipe->sh_fmt = format_bridge->sh_fmt;
pipe->pix.pixelformat = f->fmt.pix.pixelformat;
+ /* Ensure that the resolution is equal or below the maximum supported */
+
+ vformat.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+ v4l2_fill_mbus_format(ffmt, &f->fmt.pix, format_bridge->mbus_code);
+ ffmt->height += padding_h;
+ ffmt->width += padding_w;
+
+ ret = v4l2_subdev_call(isp->inputs[asd->input_curr].camera, pad,
+ set_fmt, NULL, &vformat);
+ if (ret)
+ return ret;
+
+ f->fmt.pix.width = ffmt->width - padding_w;
+ f->fmt.pix.height = ffmt->height - padding_h;
+
+ snr_fmt = f->fmt.pix;
+ backup_fmt = snr_fmt;
+
+ /**********************************************************************/
+
if (source_pad == ATOMISP_SUBDEV_PAD_SOURCE_VF ||
(source_pad == ATOMISP_SUBDEV_PAD_SOURCE_PREVIEW
&& asd->run_mode->val == ATOMISP_RUN_MODE_VIDEO)) {
@@ -5945,7 +5996,7 @@ int atomisp_set_fmt(struct video_device *vdev, struct v4l2_format *f)
* which appears to be related by a hardware
* performance limitation. It's unclear why this
* particular code triggers the issue. */
- if (!IS_ISP2401 || crop_needs_override) {
+ if (crop_needs_override) {
if (isp_sink_crop.width * main_compose.height >
isp_sink_crop.height * main_compose.width) {
sink_crop.height = isp_sink_crop.height;
@@ -5987,6 +6038,14 @@ int atomisp_set_fmt(struct video_device *vdev, struct v4l2_format *f)
pipe->pix.width = f->fmt.pix.width;
pipe->pix.height = f->fmt.pix.height;
pipe->pix.pixelformat = f->fmt.pix.pixelformat;
+ /*
+ * FIXME: do we need to setup this differently, depending on the
+ * sensor or the pipeline?
+ */
+ pipe->pix.colorspace = V4L2_COLORSPACE_REC709;
+ pipe->pix.ycbcr_enc = V4L2_YCBCR_ENC_709;
+ pipe->pix.xfer_func = V4L2_XFER_FUNC_709;
+
if (format_bridge->planar) {
pipe->pix.bytesperline = output_info.padded_width;
pipe->pix.sizeimage = PAGE_ALIGN(f->fmt.pix.height *
@@ -6021,6 +6080,15 @@ int atomisp_set_fmt(struct video_device *vdev, struct v4l2_format *f)
else
isp->need_gfx_throttle = true;
+ /* Report the needed sizes */
+ f->fmt.pix.sizeimage = pipe->pix.sizeimage;
+ f->fmt.pix.bytesperline = pipe->pix.bytesperline;
+
+ dev_dbg(isp->dev, "%s: %dx%d, image size: %d, %d bytes per line\n",
+ __func__,
+ f->fmt.pix.width, f->fmt.pix.height,
+ f->fmt.pix.sizeimage, f->fmt.pix.bytesperline);
+
return 0;
}
@@ -6034,6 +6102,12 @@ int atomisp_set_fmt_file(struct video_device *vdev, struct v4l2_format *f)
struct v4l2_subdev_fh fh;
int ret;
+ if (!asd) {
+ dev_err(isp->dev, "%s(): asd is NULL, device is %s\n",
+ __func__, vdev->name);
+ return -EINVAL;
+ }
+
v4l2_fh_init(&fh.vfh, vdev);
dev_dbg(isp->dev, "setting fmt %ux%u 0x%x for file inject\n",
@@ -6092,15 +6166,9 @@ int atomisp_set_shading_table(struct atomisp_sub_device *asd,
}
/* Shading table size per color */
- if (!IS_ISP2401) {
- if (user_shading_table->width > ISP2400_SH_CSS_MAX_SCTBL_WIDTH_PER_COLOR ||
- user_shading_table->height > ISP2400_SH_CSS_MAX_SCTBL_HEIGHT_PER_COLOR)
- return -EINVAL;
- } else {
- if (user_shading_table->width > ISP2401_SH_CSS_MAX_SCTBL_WIDTH_PER_COLOR ||
- user_shading_table->height > ISP2401_SH_CSS_MAX_SCTBL_HEIGHT_PER_COLOR)
- return -EINVAL;
- }
+ if (user_shading_table->width > SH_CSS_MAX_SCTBL_WIDTH_PER_COLOR ||
+ user_shading_table->height > SH_CSS_MAX_SCTBL_HEIGHT_PER_COLOR)
+ return -EINVAL;
shading_table = atomisp_css_shading_table_alloc(
user_shading_table->width, user_shading_table->height);
@@ -6359,6 +6427,12 @@ bool atomisp_is_vf_pipe(struct atomisp_video_pipe *pipe)
{
struct atomisp_sub_device *asd = pipe->asd;
+ if (!asd) {
+ dev_err(pipe->isp->dev, "%s(): asd is NULL, device is %s\n",
+ __func__, pipe->vdev.name);
+ return false;
+ }
+
if (pipe == &asd->video_out_vf)
return true;
@@ -6572,6 +6646,12 @@ static int atomisp_get_pipe_id(struct atomisp_video_pipe *pipe)
{
struct atomisp_sub_device *asd = pipe->asd;
+ if (!asd) {
+ dev_err(pipe->isp->dev, "%s(): asd is NULL, device is %s\n",
+ __func__, pipe->vdev.name);
+ return -EINVAL;
+ }
+
if (ATOMISP_USE_YUVPP(asd)) {
return IA_CSS_PIPE_ID_YUVPP;
} else if (asd->vfpp->val == ATOMISP_VFPP_DISABLE_SCALER) {
@@ -6609,6 +6689,12 @@ int atomisp_get_invalid_frame_num(struct video_device *vdev,
struct ia_css_pipe_info p_info;
int ret;
+ if (!asd) {
+ dev_err(pipe->isp->dev, "%s(): asd is NULL, device is %s\n",
+ __func__, vdev->name);
+ return -EINVAL;
+ }
+
if (asd->isp->inputs[asd->input_curr].camera_caps->
sensor[asd->sensor_curr].stream_num > 1) {
/* External ISP */
diff --git a/drivers/staging/media/atomisp/pci/atomisp_cmd.h b/drivers/staging/media/atomisp/pci/atomisp_cmd.h
index e8bdd26..ebc7294 100644
--- a/drivers/staging/media/atomisp/pci/atomisp_cmd.h
+++ b/drivers/staging/media/atomisp/pci/atomisp_cmd.h
@@ -74,8 +74,6 @@ irqreturn_t atomisp_isr_thread(int irq, void *isp_ptr);
const struct atomisp_format_bridge *get_atomisp_format_bridge_from_mbus(
u32 mbus_code);
bool atomisp_is_mbuscode_raw(uint32_t code);
-int atomisp_get_frame_pgnr(struct atomisp_device *isp,
- const struct ia_css_frame *frame, u32 *p_pgnr);
void atomisp_delayed_init_work(struct work_struct *work);
/* Get internal fmt according to V4L2 fmt */
@@ -266,8 +264,6 @@ int atomisp_compare_grid(struct atomisp_sub_device *asd,
int atomisp_get_sensor_mode_data(struct atomisp_sub_device *asd,
struct atomisp_sensor_mode_data *config);
-int atomisp_get_fmt(struct video_device *vdev, struct v4l2_format *f);
-
/* This function looks up the closest available resolution. */
int atomisp_try_fmt(struct video_device *vdev, struct v4l2_pix_format *f,
bool *res_overflow);
@@ -341,6 +337,8 @@ enum atomisp_metadata_type
atomisp_get_metadata_type(struct atomisp_sub_device *asd,
enum ia_css_pipe_id pipe_id);
+u32 atomisp_get_pixel_depth(u32 pixelformat);
+
/* Function for HAL to inject a fake event to wake up poll thread */
int atomisp_inject_a_fake_event(struct atomisp_sub_device *asd, int *event);
diff --git a/drivers/staging/media/atomisp/pci/atomisp_compat.h b/drivers/staging/media/atomisp/pci/atomisp_compat.h
index c16eaf3..64c1bf0 100644
--- a/drivers/staging/media/atomisp/pci/atomisp_compat.h
+++ b/drivers/staging/media/atomisp/pci/atomisp_compat.h
@@ -133,8 +133,6 @@ void atomisp_css_get_dis_statistics(struct atomisp_sub_device *asd,
struct atomisp_css_buffer *isp_css_buffer,
struct ia_css_isp_dvs_statistics_map *dvs_map);
-int atomisp_css_dequeue_event(struct atomisp_css_event *current_event);
-
void atomisp_css_temp_pipe_to_pipe_id(struct atomisp_sub_device *asd,
struct atomisp_css_event *current_event);
diff --git a/drivers/staging/media/atomisp/pci/atomisp_compat_css20.c b/drivers/staging/media/atomisp/pci/atomisp_compat_css20.c
index 99a632f..1173be0 100644
--- a/drivers/staging/media/atomisp/pci/atomisp_compat_css20.c
+++ b/drivers/staging/media/atomisp/pci/atomisp_compat_css20.c
@@ -611,15 +611,6 @@ static void __apply_additional_pipe_config(
if (stream_env->pipe_configs[pipe_id].
default_capture_config.mode == IA_CSS_CAPTURE_MODE_RAW)
stream_env->pipe_configs[pipe_id].enable_dz = false;
-
- if (IS_ISP2401) {
- /* the isp default to use ISP2.2 and the camera hal will
- * control whether use isp2.7 */
- if (asd->select_isp_version->val == ATOMISP_CSS_ISP_PIPE_VERSION_2_7)
- stream_env->pipe_configs[pipe_id].isp_pipe_version = SH_CSS_ISP_PIPE_VERSION_2_7;
- else
- stream_env->pipe_configs[pipe_id].isp_pipe_version = SH_CSS_ISP_PIPE_VERSION_2_2;
- }
break;
case IA_CSS_PIPE_ID_VIDEO:
/* enable reduced pipe to have binary
@@ -972,7 +963,7 @@ int atomisp_css_irq_translate(struct atomisp_device *isp,
void atomisp_css_rx_get_irq_info(enum mipi_port_id port,
unsigned int *infos)
{
-#ifndef ISP2401_NEW_INPUT_SYSTEM
+#ifndef IS_ISP2401
ia_css_isys_rx_get_irq_info(port, infos);
#else
*infos = 0;
@@ -982,7 +973,7 @@ void atomisp_css_rx_get_irq_info(enum mipi_port_id port,
void atomisp_css_rx_clear_irq_info(enum mipi_port_id port,
unsigned int infos)
{
-#ifndef ISP2401_NEW_INPUT_SYSTEM
+#ifndef IS_ISP2401
ia_css_isys_rx_clear_irq_info(port, infos);
#endif
}
@@ -1633,14 +1624,6 @@ void atomisp_css_get_dis_statistics(struct atomisp_sub_device *asd,
}
}
-int atomisp_css_dequeue_event(struct atomisp_css_event *current_event)
-{
- if (ia_css_dequeue_event(¤t_event->event))
- return -EINVAL;
-
- return 0;
-}
-
void atomisp_css_temp_pipe_to_pipe_id(struct atomisp_sub_device *asd,
struct atomisp_css_event *current_event)
{
@@ -2657,42 +2640,51 @@ static int __get_frame_info(struct atomisp_sub_device *asd,
if (__destroy_pipes(asd, true))
dev_warn(isp->dev, "destroy pipe failed.\n");
- if (__create_pipes(asd))
+ if (__create_pipes(asd)) {
+ dev_err(isp->dev, "can't create pipes\n");
return -EINVAL;
-
- if (__create_streams(asd))
- goto stream_err;
-
- ret = ia_css_pipe_get_info(
- asd->stream_env[stream_index]
- .pipes[pipe_id], &p_info);
- if (!ret) {
- switch (type) {
- case ATOMISP_CSS_VF_FRAME:
- *info = p_info.vf_output_info[0];
- dev_dbg(isp->dev, "getting vf frame info.\n");
- break;
- case ATOMISP_CSS_SECOND_VF_FRAME:
- *info = p_info.vf_output_info[1];
- dev_dbg(isp->dev, "getting second vf frame info.\n");
- break;
- case ATOMISP_CSS_OUTPUT_FRAME:
- *info = p_info.output_info[0];
- dev_dbg(isp->dev, "getting main frame info.\n");
- break;
- case ATOMISP_CSS_SECOND_OUTPUT_FRAME:
- *info = p_info.output_info[1];
- dev_dbg(isp->dev, "getting second main frame info.\n");
- break;
- case ATOMISP_CSS_RAW_FRAME:
- *info = p_info.raw_output_info;
- dev_dbg(isp->dev, "getting raw frame info.\n");
- }
- dev_dbg(isp->dev, "get frame info: w=%d, h=%d, num_invalid_frames %d.\n",
- info->res.width, info->res.height, p_info.num_invalid_frames);
- return 0;
}
+ if (__create_streams(asd)) {
+ dev_err(isp->dev, "can't create streams\n");
+ goto stream_err;
+ }
+
+ ret = ia_css_pipe_get_info(asd->stream_env[stream_index].pipes[pipe_id],
+ &p_info);
+ if (ret) {
+ dev_err(isp->dev, "can't get info from pipe\n");
+ goto stream_err;
+ }
+
+ switch (type) {
+ case ATOMISP_CSS_VF_FRAME:
+ *info = p_info.vf_output_info[0];
+ dev_dbg(isp->dev, "getting vf frame info.\n");
+ break;
+ case ATOMISP_CSS_SECOND_VF_FRAME:
+ *info = p_info.vf_output_info[1];
+ dev_dbg(isp->dev, "getting second vf frame info.\n");
+ break;
+ case ATOMISP_CSS_OUTPUT_FRAME:
+ *info = p_info.output_info[0];
+ dev_dbg(isp->dev, "getting main frame info.\n");
+ break;
+ case ATOMISP_CSS_SECOND_OUTPUT_FRAME:
+ *info = p_info.output_info[1];
+ dev_dbg(isp->dev, "getting second main frame info.\n");
+ break;
+ default:
+ case ATOMISP_CSS_RAW_FRAME:
+ *info = p_info.raw_output_info;
+ dev_dbg(isp->dev, "getting raw frame info.\n");
+ break;
+ }
+ dev_dbg(isp->dev, "get frame info: w=%d, h=%d, num_invalid_frames %d.\n",
+ info->res.width, info->res.height, p_info.num_invalid_frames);
+
+ return 0;
+
stream_err:
__destroy_pipes(asd, true);
return -EINVAL;
@@ -4111,7 +4103,7 @@ int atomisp_css_isr_thread(struct atomisp_device *isp,
bool reset_wdt_timer[MAX_STREAM_NUM] = {false};
int i;
- while (!atomisp_css_dequeue_event(¤t_event)) {
+ while (!ia_css_dequeue_psys_event(¤t_event.event)) {
if (current_event.event.type ==
IA_CSS_EVENT_TYPE_FW_ASSERT) {
/*
diff --git a/drivers/staging/media/atomisp/pci/atomisp_fops.c b/drivers/staging/media/atomisp/pci/atomisp_fops.c
index f82bf08..be6a74d 100644
--- a/drivers/staging/media/atomisp/pci/atomisp_fops.c
+++ b/drivers/staging/media/atomisp/pci/atomisp_fops.c
@@ -464,13 +464,11 @@ int atomisp_qbuffers_to_css(struct atomisp_sub_device *asd)
css_capture_pipe_id = IA_CSS_PIPE_ID_CAPTURE;
}
-#ifdef ISP2401_NEW_INPUT_SYSTEM
- if (asd->copy_mode) {
+ if (IS_ISP2401 && asd->copy_mode) {
css_capture_pipe_id = IA_CSS_PIPE_ID_COPY;
css_preview_pipe_id = IA_CSS_PIPE_ID_COPY;
css_video_pipe_id = IA_CSS_PIPE_ID_COPY;
}
-#endif
if (asd->yuvpp_mode) {
capture_pipe = &asd->video_out_capture;
@@ -772,6 +770,30 @@ static int atomisp_open(struct file *file)
dev_dbg(isp->dev, "open device %s\n", vdev->name);
+ /*
+ * Ensure that if we are still loading we block. Once the loading
+ * is over we can proceed. We can't blindly hold the lock until
+ * that occurs as if the load fails we'll deadlock the unload
+ */
+ rt_mutex_lock(&isp->loading);
+ /*
+ * FIXME: revisit this with a better check once the code structure
+ * is cleaned up a bit more
+ */
+ ret = v4l2_fh_open(file);
+ if (ret) {
+ dev_err(isp->dev,
+ "%s: v4l2_fh_open() returned error %d\n",
+ __func__, ret);
+ rt_mutex_unlock(&isp->loading);
+ return ret;
+ }
+ if (!isp->ready) {
+ rt_mutex_unlock(&isp->loading);
+ return -ENXIO;
+ }
+ rt_mutex_unlock(&isp->loading);
+
rt_mutex_lock(&isp->mutex);
acc_node = !strcmp(vdev->name, "ATOMISP ISP ACC");
@@ -877,6 +899,11 @@ static int atomisp_open(struct file *file)
else
pipe->users++;
rt_mutex_unlock(&isp->mutex);
+
+ /* Ensure that a mode is set */
+ if (!acc_node)
+ v4l2_ctrl_s_ctrl(asd->run_mode, pipe->default_run_mode);
+
return 0;
css_error:
@@ -1025,7 +1052,7 @@ static int atomisp_release(struct file *file)
rt_mutex_unlock(&isp->mutex);
mutex_unlock(&isp->streamoff_mutex);
- return 0;
+ return v4l2_fh_release(file);
}
/*
@@ -1067,7 +1094,7 @@ static int frame_mmap(struct atomisp_device *isp,
host_virt = vma->vm_start;
isp_virt = frame->data;
- atomisp_get_frame_pgnr(isp, frame, &pgnr);
+ pgnr = DIV_ROUND_UP(frame->data_bytes, PAGE_SIZE);
if (do_isp_mm_remap(isp, vma, isp_virt, host_virt, pgnr))
return -EAGAIN;
@@ -1171,6 +1198,12 @@ static int atomisp_mmap(struct file *file, struct vm_area_struct *vma)
u32 origin_size, new_size;
int ret;
+ if (!asd) {
+ dev_err(isp->dev, "%s(): asd is NULL, device is %s\n",
+ __func__, vdev->name);
+ return -EINVAL;
+ }
+
if (!(vma->vm_flags & (VM_WRITE | VM_READ)))
return -EACCES;
diff --git a/drivers/staging/media/atomisp/pci/atomisp_gmin_platform.c b/drivers/staging/media/atomisp/pci/atomisp_gmin_platform.c
index d8c9e31..1cc5810 100644
--- a/drivers/staging/media/atomisp/pci/atomisp_gmin_platform.c
+++ b/drivers/staging/media/atomisp/pci/atomisp_gmin_platform.c
@@ -321,6 +321,18 @@ static struct gmin_cfg_var i8880_vars[] = {
{},
};
+/*
+ * Surface 3 does not describe CsiPort/CsiLanes in both DSDT and EFI.
+ */
+static struct gmin_cfg_var surface3_vars[] = {
+ {"APTA0330:00_CsiPort", "0"},
+ {"APTA0330:00_CsiLanes", "2"},
+
+ {"OVTI8835:00_CsiPort", "1"},
+ {"OVTI8835:00_CsiLanes", "4"},
+ {},
+};
+
static const struct dmi_system_id gmin_vars[] = {
{
.ident = "BYT-T FFD8",
@@ -358,6 +370,13 @@ static const struct dmi_system_id gmin_vars[] = {
},
.driver_data = i8880_vars,
},
+ {
+ .ident = "Surface 3",
+ .matches = {
+ DMI_MATCH(DMI_BOARD_NAME, "Surface 3"),
+ },
+ .driver_data = surface3_vars,
+ },
{}
};
@@ -481,7 +500,7 @@ static int atomisp_get_acpi_power(struct device *dev)
static u8 gmin_get_pmic_id_and_addr(struct device *dev)
{
- struct i2c_client *power;
+ struct i2c_client *power = NULL;
static u8 pmic_i2c_addr;
if (pmic_id)
diff --git a/drivers/staging/media/atomisp/pci/atomisp_internal.h b/drivers/staging/media/atomisp/pci/atomisp_internal.h
index c01db10..f71ab1e 100644
--- a/drivers/staging/media/atomisp/pci/atomisp_internal.h
+++ b/drivers/staging/media/atomisp/pci/atomisp_internal.h
@@ -247,6 +247,13 @@ struct atomisp_device {
* structures and css API calls. */
struct rt_mutex mutex;
/*
+ * This mutex ensures that we don't allow an open to succeed while
+ * the initialization process is incomplete
+ */
+ struct rt_mutex loading;
+ /* Set once the ISP is ready to allow opens */
+ bool ready;
+ /*
* Serialise streamoff: mutex is dropped during streamoff to
* cancel the watchdog queue. MUST be acquired BEFORE
* "mutex".
diff --git a/drivers/staging/media/atomisp/pci/atomisp_ioctl.c b/drivers/staging/media/atomisp/pci/atomisp_ioctl.c
index c8a6256..8fd470e 100644
--- a/drivers/staging/media/atomisp/pci/atomisp_ioctl.c
+++ b/drivers/staging/media/atomisp/pci/atomisp_ioctl.c
@@ -492,23 +492,22 @@ const struct atomisp_format_bridge atomisp_output_fmts[] = {
.mbus_code = MEDIA_BUS_FMT_BGR565_2X8_LE,
.sh_fmt = IA_CSS_FRAME_FORMAT_RGB565,
.description = "16 RGB 5-6-5"
+#if 0
}, {
.pixelformat = V4L2_PIX_FMT_JPEG,
.depth = 8,
.mbus_code = MEDIA_BUS_FMT_JPEG_1X8,
.sh_fmt = IA_CSS_FRAME_FORMAT_BINARY_8,
.description = "JPEG"
- },
-#if 0
- {
+ }, {
/* This is a custom format being used by M10MO to send the RAW data */
.pixelformat = V4L2_PIX_FMT_CUSTOM_M10MO_RAW,
.depth = 8,
.mbus_code = V4L2_MBUS_FMT_CUSTOM_M10MO_RAW,
.sh_fmt = IA_CSS_FRAME_FORMAT_BINARY_8,
.description = "Custom RAW for M10MO"
- },
#endif
+ },
};
const struct atomisp_format_bridge *
@@ -646,6 +645,12 @@ static int atomisp_g_input(struct file *file, void *fh, unsigned int *input)
struct atomisp_device *isp = video_get_drvdata(vdev);
struct atomisp_sub_device *asd = atomisp_to_video_pipe(vdev)->asd;
+ if (!asd) {
+ dev_err(isp->dev, "%s(): asd is NULL, device is %s\n",
+ __func__, vdev->name);
+ return -EINVAL;
+ }
+
rt_mutex_lock(&isp->mutex);
*input = asd->input_curr;
rt_mutex_unlock(&isp->mutex);
@@ -665,6 +670,12 @@ static int atomisp_s_input(struct file *file, void *fh, unsigned int input)
struct v4l2_subdev *motor;
int ret;
+ if (!asd) {
+ dev_err(isp->dev, "%s(): asd is NULL, device is %s\n",
+ __func__, vdev->name);
+ return -EINVAL;
+ }
+
rt_mutex_lock(&isp->mutex);
if (input >= ATOM_ISP_MAX_INPUTS || input >= isp->input_cnt) {
dev_dbg(isp->dev, "input_cnt: %d\n", isp->input_cnt);
@@ -755,24 +766,91 @@ static int atomisp_s_input(struct file *file, void *fh, unsigned int input)
return ret;
}
+static int atomisp_enum_framesizes(struct file *file, void *priv,
+ struct v4l2_frmsizeenum *fsize)
+{
+ struct video_device *vdev = video_devdata(file);
+ struct atomisp_device *isp = video_get_drvdata(vdev);
+ struct atomisp_sub_device *asd = atomisp_to_video_pipe(vdev)->asd;
+ struct v4l2_subdev_frame_size_enum fse = {
+ .index = fsize->index,
+ .which = V4L2_SUBDEV_FORMAT_ACTIVE,
+ };
+ int ret;
+
+ ret = v4l2_subdev_call(isp->inputs[asd->input_curr].camera,
+ pad, enum_frame_size, NULL, &fse);
+ if (ret)
+ return ret;
+
+ fsize->type = V4L2_FRMSIZE_TYPE_DISCRETE;
+ fsize->discrete.width = fse.max_width - pad_w;
+ fsize->discrete.height = fse.max_height - pad_h;
+
+ return 0;
+}
+
+static int atomisp_enum_frameintervals(struct file *file, void *priv,
+ struct v4l2_frmivalenum *fival)
+{
+ struct video_device *vdev = video_devdata(file);
+ struct atomisp_device *isp = video_get_drvdata(vdev);
+ struct atomisp_sub_device *asd = atomisp_to_video_pipe(vdev)->asd;
+ struct v4l2_subdev_frame_interval_enum fie = {
+ .code = atomisp_in_fmt_conv[0].code,
+ .index = fival->index,
+ .width = fival->width,
+ .height = fival->height,
+ .which = V4L2_SUBDEV_FORMAT_ACTIVE,
+ };
+ int ret;
+
+ ret = v4l2_subdev_call(isp->inputs[asd->input_curr].camera,
+ pad, enum_frame_interval, NULL,
+ &fie);
+ if (ret)
+ return ret;
+
+ fival->type = V4L2_FRMIVAL_TYPE_DISCRETE;
+ fival->discrete = fie.interval;
+
+ return ret;
+}
+
static int atomisp_enum_fmt_cap(struct file *file, void *fh,
struct v4l2_fmtdesc *f)
{
struct video_device *vdev = video_devdata(file);
struct atomisp_device *isp = video_get_drvdata(vdev);
struct atomisp_sub_device *asd = atomisp_to_video_pipe(vdev)->asd;
- struct v4l2_subdev_mbus_code_enum code = { 0 };
+ struct v4l2_subdev_mbus_code_enum code = {
+ .which = V4L2_SUBDEV_FORMAT_ACTIVE,
+ };
+ const struct atomisp_format_bridge *format;
+ struct v4l2_subdev *camera;
unsigned int i, fi = 0;
int rval;
+ if (!asd) {
+ dev_err(isp->dev, "%s(): asd is NULL, device is %s\n",
+ __func__, vdev->name);
+ return -EINVAL;
+ }
+
+ camera = isp->inputs[asd->input_curr].camera;
+ if(!camera) {
+ dev_err(isp->dev, "%s(): camera is NULL, device is %s\n",
+ __func__, vdev->name);
+ return -EINVAL;
+ }
+
rt_mutex_lock(&isp->mutex);
- rval = v4l2_subdev_call(isp->inputs[asd->input_curr].camera, pad,
- enum_mbus_code, NULL, &code);
+
+ rval = v4l2_subdev_call(camera, pad, enum_mbus_code, NULL, &code);
if (rval == -ENOIOCTLCMD) {
dev_warn(isp->dev,
- "enum_mbus_code pad op not supported. Please fix your sensor driver!\n");
- // rval = v4l2_subdev_call(isp->inputs[asd->input_curr].camera,
- // video, enum_mbus_fmt, 0, &code.code);
+ "enum_mbus_code pad op not supported by %s. Please fix your sensor driver!\n",
+ camera->name);
}
rt_mutex_unlock(&isp->mutex);
@@ -780,15 +858,15 @@ static int atomisp_enum_fmt_cap(struct file *file, void *fh,
return rval;
for (i = 0; i < ARRAY_SIZE(atomisp_output_fmts); i++) {
- const struct atomisp_format_bridge *format =
- &atomisp_output_fmts[i];
+ format = &atomisp_output_fmts[i];
/*
* Is the atomisp-supported format is valid for the
* sensor (configuration)? If not, skip it.
+ *
+ * FIXME: fix the pipeline to allow sensor format too.
*/
- if (format->sh_fmt == IA_CSS_FRAME_FORMAT_RAW
- && format->mbus_code != code.code)
+ if (format->sh_fmt == IA_CSS_FRAME_FORMAT_RAW)
continue;
/* Found a match. Now let's pick f->index'th one. */
@@ -806,20 +884,6 @@ static int atomisp_enum_fmt_cap(struct file *file, void *fh,
return -EINVAL;
}
-static int atomisp_g_fmt_cap(struct file *file, void *fh,
- struct v4l2_format *f)
-{
- struct video_device *vdev = video_devdata(file);
- struct atomisp_device *isp = video_get_drvdata(vdev);
-
- int ret;
-
- rt_mutex_lock(&isp->mutex);
- ret = atomisp_get_fmt(vdev, f);
- rt_mutex_unlock(&isp->mutex);
- return ret;
-}
-
static int atomisp_g_fmt_file(struct file *file, void *fh,
struct v4l2_format *f)
{
@@ -834,6 +898,72 @@ static int atomisp_g_fmt_file(struct file *file, void *fh,
return 0;
}
+static int atomisp_adjust_fmt(struct v4l2_format *f)
+{
+ const struct atomisp_format_bridge *format_bridge;
+ u32 padded_width;
+
+ format_bridge = atomisp_get_format_bridge(f->fmt.pix.pixelformat);
+
+ padded_width = f->fmt.pix.width + pad_w;
+
+ if (format_bridge->planar) {
+ f->fmt.pix.bytesperline = padded_width;
+ f->fmt.pix.sizeimage = PAGE_ALIGN(f->fmt.pix.height *
+ DIV_ROUND_UP(format_bridge->depth *
+ padded_width, 8));
+ } else {
+ f->fmt.pix.bytesperline = DIV_ROUND_UP(format_bridge->depth *
+ padded_width, 8);
+ f->fmt.pix.sizeimage = PAGE_ALIGN(f->fmt.pix.height * f->fmt.pix.bytesperline);
+ }
+
+ if (f->fmt.pix.field == V4L2_FIELD_ANY)
+ f->fmt.pix.field = V4L2_FIELD_NONE;
+
+ format_bridge = atomisp_get_format_bridge(f->fmt.pix.pixelformat);
+ if (!format_bridge)
+ return -EINVAL;
+
+ /* Currently, raw formats are broken!!! */
+ if (format_bridge->sh_fmt == IA_CSS_FRAME_FORMAT_RAW) {
+ f->fmt.pix.pixelformat = V4L2_PIX_FMT_YUV420;
+
+ format_bridge = atomisp_get_format_bridge(f->fmt.pix.pixelformat);
+ if (!format_bridge)
+ return -EINVAL;
+ }
+
+ padded_width = f->fmt.pix.width + pad_w;
+
+ if (format_bridge->planar) {
+ f->fmt.pix.bytesperline = padded_width;
+ f->fmt.pix.sizeimage = PAGE_ALIGN(f->fmt.pix.height *
+ DIV_ROUND_UP(format_bridge->depth *
+ padded_width, 8));
+ } else {
+ f->fmt.pix.bytesperline = DIV_ROUND_UP(format_bridge->depth *
+ padded_width, 8);
+ f->fmt.pix.sizeimage = PAGE_ALIGN(f->fmt.pix.height * f->fmt.pix.bytesperline);
+ }
+
+ if (f->fmt.pix.field == V4L2_FIELD_ANY)
+ f->fmt.pix.field = V4L2_FIELD_NONE;
+
+ /*
+ * FIXME: do we need to setup this differently, depending on the
+ * sensor or the pipeline?
+ */
+ f->fmt.pix.colorspace = V4L2_COLORSPACE_REC709;
+ f->fmt.pix.ycbcr_enc = V4L2_YCBCR_ENC_709;
+ f->fmt.pix.xfer_func = V4L2_XFER_FUNC_709;
+
+ f->fmt.pix.width -= pad_w;
+ f->fmt.pix.height -= pad_h;
+
+ return 0;
+}
+
/* This function looks up the closest available resolution. */
static int atomisp_try_fmt_cap(struct file *file, void *fh,
struct v4l2_format *f)
@@ -845,7 +975,35 @@ static int atomisp_try_fmt_cap(struct file *file, void *fh,
rt_mutex_lock(&isp->mutex);
ret = atomisp_try_fmt(vdev, &f->fmt.pix, NULL);
rt_mutex_unlock(&isp->mutex);
- return ret;
+
+ if (ret)
+ return ret;
+
+ return atomisp_adjust_fmt(f);
+}
+
+static int atomisp_g_fmt_cap(struct file *file, void *fh,
+ struct v4l2_format *f)
+{
+ struct video_device *vdev = video_devdata(file);
+ struct atomisp_device *isp = video_get_drvdata(vdev);
+ struct atomisp_video_pipe *pipe;
+
+ rt_mutex_lock(&isp->mutex);
+ pipe = atomisp_to_video_pipe(vdev);
+ rt_mutex_unlock(&isp->mutex);
+
+ f->fmt.pix = pipe->pix;
+
+ /* If s_fmt was issued, just return whatever is was previouly set */
+ if (f->fmt.pix.sizeimage)
+ return 0;
+
+ f->fmt.pix.pixelformat = V4L2_PIX_FMT_YUYV;
+ f->fmt.pix.width = 10000;
+ f->fmt.pix.height = 10000;
+
+ return atomisp_try_fmt_cap(file, fh, f);
}
static int atomisp_s_fmt_cap(struct file *file, void *fh,
@@ -1024,9 +1182,16 @@ int __atomisp_reqbufs(struct file *file, void *fh,
struct ia_css_frame *frame;
struct videobuf_vmalloc_memory *vm_mem;
u16 source_pad = atomisp_subdev_source_pad(vdev);
- u16 stream_id = atomisp_source_pad_to_stream_id(asd, source_pad);
+ u16 stream_id;
int ret = 0, i = 0;
+ if (!asd) {
+ dev_err(pipe->isp->dev, "%s(): asd is NULL, device is %s\n",
+ __func__, vdev->name);
+ return -EINVAL;
+ }
+ stream_id = atomisp_source_pad_to_stream_id(asd, source_pad);
+
if (req->count == 0) {
mutex_lock(&pipe->capq.vb_lock);
if (!list_empty(&pipe->capq.stream))
@@ -1154,6 +1319,12 @@ static int atomisp_qbuf(struct file *file, void *fh, struct v4l2_buffer *buf)
u32 pgnr;
int ret = 0;
+ if (!asd) {
+ dev_err(isp->dev, "%s(): asd is NULL, device is %s\n",
+ __func__, vdev->name);
+ return -EINVAL;
+ }
+
rt_mutex_lock(&isp->mutex);
if (isp->isp_fatal_error) {
ret = -EIO;
@@ -1287,25 +1458,8 @@ static int atomisp_qbuf(struct file *file, void *fh, struct v4l2_buffer *buf)
pipe->capq.streaming &&
!asd->enable_raw_buffer_lock->val &&
asd->params.offline_parm.num_captures == 1) {
- if (!IS_ISP2401) {
asd->pending_capture_request++;
dev_dbg(isp->dev, "Add one pending capture request.\n");
- } else {
- if (asd->re_trigger_capture) {
- ret = atomisp_css_offline_capture_configure(asd,
- asd->params.offline_parm.num_captures,
- asd->params.offline_parm.skip_frames,
- asd->params.offline_parm.offset);
- asd->re_trigger_capture = false;
- dev_dbg(isp->dev, "%s Trigger capture again ret=%d\n",
- __func__, ret);
-
- } else {
- asd->pending_capture_request++;
- asd->re_trigger_capture = false;
- dev_dbg(isp->dev, "Add one pending capture request.\n");
- }
- }
}
rt_mutex_unlock(&isp->mutex);
@@ -1389,6 +1543,12 @@ static int atomisp_dqbuf(struct file *file, void *fh, struct v4l2_buffer *buf)
struct atomisp_device *isp = video_get_drvdata(vdev);
int ret = 0;
+ if (!asd) {
+ dev_err(isp->dev, "%s(): asd is NULL, device is %s\n",
+ __func__, vdev->name);
+ return -EINVAL;
+ }
+
rt_mutex_lock(&isp->mutex);
if (isp->isp_fatal_error) {
@@ -1640,6 +1800,12 @@ static int atomisp_streamon(struct file *file, void *fh,
int ret = 0;
unsigned long irqflags;
+ if (!asd) {
+ dev_err(isp->dev, "%s(): asd is NULL, device is %s\n",
+ __func__, vdev->name);
+ return -EINVAL;
+ }
+
dev_dbg(isp->dev, "Start stream on pad %d for asd%d\n",
atomisp_subdev_source_pad(vdev), asd->index);
@@ -1686,8 +1852,6 @@ static int atomisp_streamon(struct file *file, void *fh,
/* Reset pending capture request count. */
asd->pending_capture_request = 0;
- if (IS_ISP2401)
- asd->re_trigger_capture = false;
if ((atomisp_subdev_streaming_count(asd) > sensor_start_stream) &&
(!isp->inputs[asd->input_curr].camera_caps->multi_stream_ctrl)) {
@@ -1901,6 +2065,12 @@ int __atomisp_streamoff(struct file *file, void *fh, enum v4l2_buf_type type)
unsigned long flags;
bool first_streamoff = false;
+ if (!asd) {
+ dev_err(isp->dev, "%s(): asd is NULL, device is %s\n",
+ __func__, vdev->name);
+ return -EINVAL;
+ }
+
dev_dbg(isp->dev, "Stop stream on pad %d for asd%d\n",
atomisp_subdev_source_pad(vdev), asd->index);
@@ -2150,6 +2320,12 @@ static int atomisp_g_ctrl(struct file *file, void *fh,
struct atomisp_device *isp = video_get_drvdata(vdev);
int i, ret = -EINVAL;
+ if (!asd) {
+ dev_err(isp->dev, "%s(): asd is NULL, device is %s\n",
+ __func__, vdev->name);
+ return -EINVAL;
+ }
+
for (i = 0; i < ctrls_num; i++) {
if (ci_v4l2_controls[i].id == control->id) {
ret = 0;
@@ -2229,6 +2405,12 @@ static int atomisp_s_ctrl(struct file *file, void *fh,
struct atomisp_device *isp = video_get_drvdata(vdev);
int i, ret = -EINVAL;
+ if (!asd) {
+ dev_err(isp->dev, "%s(): asd is NULL, device is %s\n",
+ __func__, vdev->name);
+ return -EINVAL;
+ }
+
for (i = 0; i < ctrls_num; i++) {
if (ci_v4l2_controls[i].id == control->id) {
ret = 0;
@@ -2310,6 +2492,12 @@ static int atomisp_queryctl(struct file *file, void *fh,
struct atomisp_sub_device *asd = atomisp_to_video_pipe(vdev)->asd;
struct atomisp_device *isp = video_get_drvdata(vdev);
+ if (!asd) {
+ dev_err(isp->dev, "%s(): asd is NULL, device is %s\n",
+ __func__, vdev->name);
+ return -EINVAL;
+ }
+
switch (qc->id) {
case V4L2_CID_FOCUS_ABSOLUTE:
case V4L2_CID_FOCUS_RELATIVE:
@@ -2355,6 +2543,12 @@ static int atomisp_camera_g_ext_ctrls(struct file *file, void *fh,
int i;
int ret = 0;
+ if (!asd) {
+ dev_err(isp->dev, "%s(): asd is NULL, device is %s\n",
+ __func__, vdev->name);
+ return -EINVAL;
+ }
+
if (!IS_ISP2401)
motor = isp->inputs[asd->input_curr].motor;
else
@@ -2466,6 +2660,12 @@ static int atomisp_camera_s_ext_ctrls(struct file *file, void *fh,
int i;
int ret = 0;
+ if (!asd) {
+ dev_err(isp->dev, "%s(): asd is NULL, device is %s\n",
+ __func__, vdev->name);
+ return -EINVAL;
+ }
+
if (!IS_ISP2401)
motor = isp->inputs[asd->input_curr].motor;
else
@@ -2591,6 +2791,12 @@ static int atomisp_g_parm(struct file *file, void *fh,
struct atomisp_sub_device *asd = atomisp_to_video_pipe(vdev)->asd;
struct atomisp_device *isp = video_get_drvdata(vdev);
+ if (!asd) {
+ dev_err(isp->dev, "%s(): asd is NULL, device is %s\n",
+ __func__, vdev->name);
+ return -EINVAL;
+ }
+
if (parm->type != V4L2_BUF_TYPE_VIDEO_CAPTURE) {
dev_err(isp->dev, "unsupported v4l2 buf type\n");
return -EINVAL;
@@ -2613,6 +2819,12 @@ static int atomisp_s_parm(struct file *file, void *fh,
int rval;
int fps;
+ if (!asd) {
+ dev_err(isp->dev, "%s(): asd is NULL, device is %s\n",
+ __func__, vdev->name);
+ return -EINVAL;
+ }
+
if (parm->type != V4L2_BUF_TYPE_VIDEO_CAPTURE) {
dev_err(isp->dev, "unsupported v4l2 buf type\n");
return -EINVAL;
@@ -3035,6 +3247,8 @@ const struct v4l2_ioctl_ops atomisp_ioctl_ops = {
.vidioc_g_ctrl = atomisp_g_ctrl,
.vidioc_s_ext_ctrls = atomisp_s_ext_ctrls,
.vidioc_g_ext_ctrls = atomisp_g_ext_ctrls,
+ .vidioc_enum_framesizes = atomisp_enum_framesizes,
+ .vidioc_enum_frameintervals = atomisp_enum_frameintervals,
.vidioc_enum_fmt_vid_cap = atomisp_enum_fmt_cap,
.vidioc_try_fmt_vid_cap = atomisp_try_fmt_cap,
.vidioc_g_fmt_vid_cap = atomisp_g_fmt_cap,
diff --git a/drivers/staging/media/atomisp/pci/atomisp_subdev.c b/drivers/staging/media/atomisp/pci/atomisp_subdev.c
index 12f22ad..1807cfa 100644
--- a/drivers/staging/media/atomisp/pci/atomisp_subdev.c
+++ b/drivers/staging/media/atomisp/pci/atomisp_subdev.c
@@ -1058,24 +1058,6 @@ static const struct v4l2_ctrl_config ctrl_depth_mode = {
.def = 0,
};
-/*
- * Control for selectting ISP version
- *
- * When enabled, that means ISP version will be used ISP2.7. when disable, the
- * isp will default to use ISP2.2.
- * Note: Make sure set this configuration before creating stream.
- */
-static const struct v4l2_ctrl_config ctrl_select_isp_version = {
- .ops = &ctrl_ops,
- .id = V4L2_CID_ATOMISP_SELECT_ISP_VERSION,
- .type = V4L2_CTRL_TYPE_BOOLEAN,
- .name = "Select Isp version",
- .min = 0,
- .max = 1,
- .step = 1,
- .def = 0,
-};
-
static void atomisp_init_subdev_pipe(struct atomisp_sub_device *asd,
struct atomisp_video_pipe *pipe, enum v4l2_buf_type buf_type)
{
@@ -1164,23 +1146,28 @@ static int isp_subdev_init_entities(struct atomisp_sub_device *asd)
atomisp_init_acc_pipe(asd, &asd->video_acc);
- ret = atomisp_video_init(&asd->video_in, "MEMORY");
+ ret = atomisp_video_init(&asd->video_in, "MEMORY",
+ ATOMISP_RUN_MODE_SDV);
if (ret < 0)
return ret;
- ret = atomisp_video_init(&asd->video_out_capture, "CAPTURE");
+ ret = atomisp_video_init(&asd->video_out_capture, "CAPTURE",
+ ATOMISP_RUN_MODE_STILL_CAPTURE);
if (ret < 0)
return ret;
- ret = atomisp_video_init(&asd->video_out_vf, "VIEWFINDER");
+ ret = atomisp_video_init(&asd->video_out_vf, "VIEWFINDER",
+ ATOMISP_RUN_MODE_CONTINUOUS_CAPTURE);
if (ret < 0)
return ret;
- ret = atomisp_video_init(&asd->video_out_preview, "PREVIEW");
+ ret = atomisp_video_init(&asd->video_out_preview, "PREVIEW",
+ ATOMISP_RUN_MODE_PREVIEW);
if (ret < 0)
return ret;
- ret = atomisp_video_init(&asd->video_out_video_capture, "VIDEO");
+ ret = atomisp_video_init(&asd->video_out_video_capture, "VIDEO",
+ ATOMISP_RUN_MODE_VIDEO);
if (ret < 0)
return ret;
@@ -1218,11 +1205,6 @@ static int isp_subdev_init_entities(struct atomisp_sub_device *asd)
v4l2_ctrl_new_custom(&asd->ctrl_handler,
&ctrl_disable_dz,
NULL);
- if (IS_ISP2401) {
- asd->select_isp_version = v4l2_ctrl_new_custom(&asd->ctrl_handler,
- &ctrl_select_isp_version,
- NULL);
- }
/* Make controls visible on subdev as well. */
asd->subdev.ctrl_handler = &asd->ctrl_handler;
@@ -1351,6 +1333,14 @@ int atomisp_subdev_register_entities(struct atomisp_sub_device *asd,
if (ret < 0)
goto error;
+ asd->video_out_preview.vdev.v4l2_dev = vdev;
+ asd->video_out_preview.vdev.device_caps = device_caps |
+ V4L2_CAP_VIDEO_OUTPUT;
+ ret = video_register_device(&asd->video_out_preview.vdev,
+ VFL_TYPE_VIDEO, -1);
+ if (ret < 0)
+ goto error;
+
asd->video_out_capture.vdev.v4l2_dev = vdev;
asd->video_out_capture.vdev.device_caps = device_caps |
V4L2_CAP_VIDEO_OUTPUT;
@@ -1366,13 +1356,7 @@ int atomisp_subdev_register_entities(struct atomisp_sub_device *asd,
VFL_TYPE_VIDEO, -1);
if (ret < 0)
goto error;
- asd->video_out_preview.vdev.v4l2_dev = vdev;
- asd->video_out_preview.vdev.device_caps = device_caps |
- V4L2_CAP_VIDEO_OUTPUT;
- ret = video_register_device(&asd->video_out_preview.vdev,
- VFL_TYPE_VIDEO, -1);
- if (ret < 0)
- goto error;
+
asd->video_out_video_capture.vdev.v4l2_dev = vdev;
asd->video_out_video_capture.vdev.device_caps = device_caps |
V4L2_CAP_VIDEO_OUTPUT;
diff --git a/drivers/staging/media/atomisp/pci/atomisp_subdev.h b/drivers/staging/media/atomisp/pci/atomisp_subdev.h
index d6fcfab..7d731f1 100644
--- a/drivers/staging/media/atomisp/pci/atomisp_subdev.h
+++ b/drivers/staging/media/atomisp/pci/atomisp_subdev.h
@@ -81,6 +81,9 @@ struct atomisp_video_pipe {
/* the link list to store per_frame parameters */
struct list_head per_frame_params;
+ /* Store here the initial run mode */
+ unsigned int default_run_mode;
+
unsigned int buffers_in_css;
/* irq_lock is used to protect video buffer state change operations and
@@ -316,7 +319,6 @@ struct atomisp_sub_device {
/* ISP2401 */
struct v4l2_ctrl *ion_dev_fd;
- struct v4l2_ctrl *select_isp_version;
struct v4l2_ctrl *disable_dz;
diff --git a/drivers/staging/media/atomisp/pci/atomisp_v4l2.c b/drivers/staging/media/atomisp/pci/atomisp_v4l2.c
index 1e324f1..1b24089 100644
--- a/drivers/staging/media/atomisp/pci/atomisp_v4l2.c
+++ b/drivers/staging/media/atomisp/pci/atomisp_v4l2.c
@@ -447,7 +447,8 @@ const struct atomisp_dfs_config dfs_config_cht_soc = {
.dfs_table_size = ARRAY_SIZE(dfs_rules_cht_soc),
};
-int atomisp_video_init(struct atomisp_video_pipe *video, const char *name)
+int atomisp_video_init(struct atomisp_video_pipe *video, const char *name,
+ unsigned int run_mode)
{
int ret;
const char *direction;
@@ -478,6 +479,7 @@ int atomisp_video_init(struct atomisp_video_pipe *video, const char *name)
"ATOMISP ISP %s %s", name, direction);
video->vdev.release = video_device_release_empty;
video_set_drvdata(&video->vdev, video->isp);
+ video->default_run_mode = run_mode;
return 0;
}
@@ -711,15 +713,15 @@ static int atomisp_mrfld_power(struct atomisp_device *isp, bool enable)
dev_dbg(isp->dev, "IUNIT power-%s.\n", enable ? "on" : "off");
- /*WA:Enable DVFS*/
+ /* WA for P-Unit, if DVFS enabled, ISP timeout observed */
if (IS_CHT && enable)
- punit_ddr_dvfs_enable(true);
+ punit_ddr_dvfs_enable(false);
/*
* FIXME:WA for ECS28A, with this sleep, CTS
* android.hardware.camera2.cts.CameraDeviceTest#testCameraDeviceAbort
* PASS, no impact on other platforms
- */
+ */
if (IS_BYT && enable)
msleep(10);
@@ -727,7 +729,7 @@ static int atomisp_mrfld_power(struct atomisp_device *isp, bool enable)
iosf_mbi_modify(BT_MBI_UNIT_PMC, MBI_REG_READ, MRFLD_ISPSSPM0,
val, MRFLD_ISPSSPM0_ISPSSC_MASK);
- /*WA:Enable DVFS*/
+ /* WA:Enable DVFS */
if (IS_CHT && !enable)
punit_ddr_dvfs_enable(true);
@@ -1182,6 +1184,7 @@ static void atomisp_unregister_entities(struct atomisp_device *isp)
v4l2_device_unregister(&isp->v4l2_dev);
media_device_unregister(&isp->media_dev);
+ media_device_cleanup(&isp->media_dev);
}
static int atomisp_register_entities(struct atomisp_device *isp)
@@ -1566,6 +1569,7 @@ static int atomisp_pci_probe(struct pci_dev *pdev, const struct pci_device_id *i
dev_dbg(&pdev->dev, "atomisp mmio base: %p\n", isp->base);
rt_mutex_init(&isp->mutex);
+ rt_mutex_init(&isp->loading);
mutex_init(&isp->streamoff_mutex);
spin_lock_init(&isp->lock);
@@ -1633,12 +1637,7 @@ static int atomisp_pci_probe(struct pci_dev *pdev, const struct pci_device_id *i
pdev->d3cold_delay = 0;
break;
case ATOMISP_PCI_DEVICE_SOC_ANN:
- isp->media_dev.hw_revision = (
-#ifdef ISP2401_NEW_INPUT_SYSTEM
- ATOMISP_HW_REVISION_ISP2401
-#else
- ATOMISP_HW_REVISION_ISP2401_LEGACY
-#endif
+ isp->media_dev.hw_revision = ( ATOMISP_HW_REVISION_ISP2401
<< ATOMISP_HW_REVISION_SHIFT);
isp->media_dev.hw_revision |= pdev->revision < 2 ?
ATOMISP_HW_STEPPING_A0 : ATOMISP_HW_STEPPING_B0;
@@ -1646,12 +1645,7 @@ static int atomisp_pci_probe(struct pci_dev *pdev, const struct pci_device_id *i
isp->hpll_freq = HPLL_FREQ_1600MHZ;
break;
case ATOMISP_PCI_DEVICE_SOC_CHT:
- isp->media_dev.hw_revision = (
-#ifdef ISP2401_NEW_INPUT_SYSTEM
- ATOMISP_HW_REVISION_ISP2401
-#else
- ATOMISP_HW_REVISION_ISP2401_LEGACY
-#endif
+ isp->media_dev.hw_revision = ( ATOMISP_HW_REVISION_ISP2401
<< ATOMISP_HW_REVISION_SHIFT);
isp->media_dev.hw_revision |= pdev->revision < 2 ?
ATOMISP_HW_STEPPING_A0 : ATOMISP_HW_STEPPING_B0;
@@ -1748,6 +1742,8 @@ static int atomisp_pci_probe(struct pci_dev *pdev, const struct pci_device_id *i
pci_write_config_dword(pdev, MRFLD_PCI_CSI_AFE_TRIM_CONTROL, csi_afe_trim);
}
+ rt_mutex_lock(&isp->loading);
+
err = atomisp_initialize_modules(isp);
if (err < 0) {
dev_err(&pdev->dev, "atomisp_initialize_modules (%d)\n", err);
@@ -1805,6 +1801,8 @@ static int atomisp_pci_probe(struct pci_dev *pdev, const struct pci_device_id *i
release_firmware(isp->firmware);
isp->firmware = NULL;
isp->css_env.isp_css_fw.data = NULL;
+ isp->ready = true;
+ rt_mutex_unlock(&isp->loading);
atomisp_drvfs_init(isp);
@@ -1824,6 +1822,7 @@ static int atomisp_pci_probe(struct pci_dev *pdev, const struct pci_device_id *i
register_entities_fail:
atomisp_uninitialize_modules(isp);
initialize_modules_fail:
+ rt_mutex_unlock(&isp->loading);
cpu_latency_qos_remove_request(&isp->pm_qos);
atomisp_msi_irq_uninit(isp);
pci_free_irq_vectors(pdev);
diff --git a/drivers/staging/media/atomisp/pci/atomisp_v4l2.h b/drivers/staging/media/atomisp/pci/atomisp_v4l2.h
index 81bb356..72611b8 100644
--- a/drivers/staging/media/atomisp/pci/atomisp_v4l2.h
+++ b/drivers/staging/media/atomisp/pci/atomisp_v4l2.h
@@ -27,7 +27,8 @@ struct v4l2_device;
struct atomisp_device;
struct firmware;
-int atomisp_video_init(struct atomisp_video_pipe *video, const char *name);
+int atomisp_video_init(struct atomisp_video_pipe *video, const char *name,
+ unsigned int run_mode);
void atomisp_acc_init(struct atomisp_acc_pipe *video, const char *name);
void atomisp_video_unregister(struct atomisp_video_pipe *video);
void atomisp_acc_unregister(struct atomisp_acc_pipe *video);
diff --git a/drivers/staging/media/atomisp/pci/camera/pipe/src/pipe_binarydesc.c b/drivers/staging/media/atomisp/pci/camera/pipe/src/pipe_binarydesc.c
index f20c9b02..7dd0e4a 100644
--- a/drivers/staging/media/atomisp/pci/camera/pipe/src/pipe_binarydesc.c
+++ b/drivers/staging/media/atomisp/pci/camera/pipe/src/pipe_binarydesc.c
@@ -58,7 +58,6 @@ static void pipe_binarydesc_get_offline(
descr->enable_dz = true;
descr->enable_xnr = false;
descr->enable_dpc = false;
- descr->enable_luma_only = false;
descr->enable_tnr = false;
descr->enable_capture_pp_bli = false;
descr->enable_fractional_ds = false;
@@ -390,8 +389,6 @@ int ia_css_pipe_get_video_binarydesc(
pipe->extra_config.enable_fractional_ds;
video_descr->enable_dpc =
pipe->config.enable_dpc;
- video_descr->enable_luma_only =
- pipe->config.enable_luma_only;
video_descr->enable_tnr =
pipe->config.enable_tnr;
@@ -574,11 +571,9 @@ void ia_css_pipe_get_primary_binarydesc(
in_info->res = pipe->config.input_effective_res;
in_info->padded_width = in_info->res.width;
-#if !defined(HAS_NO_PACKED_RAW_PIXELS)
if (pipe->stream->config.pack_raw_pixels)
in_info->format = IA_CSS_FRAME_FORMAT_RAW_PACKED;
else
-#endif
in_info->format = IA_CSS_FRAME_FORMAT_RAW;
in_info->raw_bit_depth = ia_css_pipe_util_pipe_input_format_bpp(pipe);
@@ -600,24 +595,15 @@ void ia_css_pipe_get_primary_binarydesc(
prim_descr->isp_pipe_version = pipe->config.isp_pipe_version;
prim_descr->enable_fractional_ds =
pipe->extra_config.enable_fractional_ds;
- prim_descr->enable_luma_only =
- pipe->config.enable_luma_only;
/* We have both striped and non-striped primary binaries,
* if continuous viewfinder is required, then we must select
* a striped one. Otherwise we prefer to use a non-striped
* since it has better performance. */
if (pipe_version == IA_CSS_PIPE_VERSION_2_6_1)
prim_descr->striped = false;
- else if (!IS_ISP2401) {
+ else
prim_descr->striped = prim_descr->continuous &&
(!pipe->stream->stop_copy_preview || !pipe->stream->disable_cont_vf);
- } else {
- prim_descr->striped = prim_descr->continuous && !pipe->stream->disable_cont_vf;
-
- if ((pipe->config.default_capture_config.enable_xnr != 0) &&
- (pipe->extra_config.enable_dvs_6axis == true))
- prim_descr->enable_xnr = true;
- }
}
IA_CSS_LEAVE_PRIVATE("");
}
@@ -849,14 +835,7 @@ void ia_css_pipe_get_ldc_binarydesc(
assert(out_info);
IA_CSS_ENTER_PRIVATE("");
- if (!IS_ISP2401) {
- *in_info = *out_info;
- } else {
- if (pipe->out_yuv_ds_input_info.res.width)
- *in_info = pipe->out_yuv_ds_input_info;
- else
- *in_info = *out_info;
- }
+ *in_info = *out_info;
in_info->format = IA_CSS_FRAME_FORMAT_YUV420;
in_info->raw_bit_depth = 0;
diff --git a/drivers/staging/media/atomisp/pci/css_2400_system/hive/ia_css_isp_configs.c b/drivers/staging/media/atomisp/pci/css_2400_system/hive/ia_css_isp_configs.c
deleted file mode 100644
index 1a021ae..0000000
--- a/drivers/staging/media/atomisp/pci/css_2400_system/hive/ia_css_isp_configs.c
+++ /dev/null
@@ -1,386 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/*
- * Support for Intel Camera Imaging ISP subsystem.
- * Copyright (c) 2015, Intel Corporation.
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms and conditions of the GNU General Public License,
- * version 2, as published by the Free Software Foundation.
- *
- * This program is distributed in the hope it will be useful, but WITHOUT
- * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
- * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
- * more details.
- */
-
-/* Generated code: do not edit or commmit. */
-
-#define IA_CSS_INCLUDE_CONFIGURATIONS
-#include "ia_css_pipeline.h"
-#include "ia_css_isp_configs.h"
-#include "ia_css_debug.h"
-#include "assert_support.h"
-
-/* Code generated by genparam/genconfig.c:gen_configure_function() */
-
-void
-ia_css_configure_iterator(
- const struct ia_css_binary *binary,
- const struct ia_css_iterator_configuration *config_dmem)
-{
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_iterator() enter:\n");
-
- {
- unsigned int offset = 0;
- unsigned int size = 0;
-
- if (binary->info->mem_offsets.offsets.config) {
- size = binary->info->mem_offsets.offsets.config->dmem.iterator.size;
- offset = binary->info->mem_offsets.offsets.config->dmem.iterator.offset;
- }
- if (size) {
- ia_css_iterator_config((struct sh_css_isp_iterator_isp_config *)
- &binary->mem_params.params[IA_CSS_PARAM_CLASS_CONFIG][IA_CSS_ISP_DMEM].address[offset],
- config_dmem, size);
- }
- }
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_iterator() leave:\n");
-}
-
-/* Code generated by genparam/genconfig.c:gen_configure_function() */
-
-void
-ia_css_configure_copy_output(
- const struct ia_css_binary *binary,
- const struct ia_css_copy_output_configuration *config_dmem)
-{
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_copy_output() enter:\n");
-
- {
- unsigned int offset = 0;
- unsigned int size = 0;
-
- if (binary->info->mem_offsets.offsets.config) {
- size = binary->info->mem_offsets.offsets.config->dmem.copy_output.size;
- offset = binary->info->mem_offsets.offsets.config->dmem.copy_output.offset;
- }
- if (size) {
- ia_css_copy_output_config((struct sh_css_isp_copy_output_isp_config *)
- &binary->mem_params.params[IA_CSS_PARAM_CLASS_CONFIG][IA_CSS_ISP_DMEM].address[offset],
- config_dmem, size);
- }
- }
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_copy_output() leave:\n");
-}
-
-/* Code generated by genparam/genconfig.c:gen_configure_function() */
-
-void
-ia_css_configure_crop(
- const struct ia_css_binary *binary,
- const struct ia_css_crop_configuration *config_dmem)
-{
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_crop() enter:\n");
-
- {
- unsigned int offset = 0;
- unsigned int size = 0;
-
- if (binary->info->mem_offsets.offsets.config) {
- size = binary->info->mem_offsets.offsets.config->dmem.crop.size;
- offset = binary->info->mem_offsets.offsets.config->dmem.crop.offset;
- }
- if (size) {
- ia_css_crop_config((struct sh_css_isp_crop_isp_config *)
- &binary->mem_params.params[IA_CSS_PARAM_CLASS_CONFIG][IA_CSS_ISP_DMEM].address[offset],
- config_dmem, size);
- }
- }
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_crop() leave:\n");
-}
-
-/* Code generated by genparam/genconfig.c:gen_configure_function() */
-
-void
-ia_css_configure_fpn(
- const struct ia_css_binary *binary,
- const struct ia_css_fpn_configuration *config_dmem)
-{
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_fpn() enter:\n");
-
- {
- unsigned int offset = 0;
- unsigned int size = 0;
-
- if (binary->info->mem_offsets.offsets.config) {
- size = binary->info->mem_offsets.offsets.config->dmem.fpn.size;
- offset = binary->info->mem_offsets.offsets.config->dmem.fpn.offset;
- }
- if (size) {
- ia_css_fpn_config((struct sh_css_isp_fpn_isp_config *)
- &binary->mem_params.params[IA_CSS_PARAM_CLASS_CONFIG][IA_CSS_ISP_DMEM].address[offset],
- config_dmem, size);
- }
- }
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_fpn() leave:\n");
-}
-
-/* Code generated by genparam/genconfig.c:gen_configure_function() */
-
-void
-ia_css_configure_dvs(
- const struct ia_css_binary *binary,
- const struct ia_css_dvs_configuration *config_dmem)
-{
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_dvs() enter:\n");
-
- {
- unsigned int offset = 0;
- unsigned int size = 0;
-
- if (binary->info->mem_offsets.offsets.config) {
- size = binary->info->mem_offsets.offsets.config->dmem.dvs.size;
- offset = binary->info->mem_offsets.offsets.config->dmem.dvs.offset;
- }
- if (size) {
- ia_css_dvs_config((struct sh_css_isp_dvs_isp_config *)
- &binary->mem_params.params[IA_CSS_PARAM_CLASS_CONFIG][IA_CSS_ISP_DMEM].address[offset],
- config_dmem, size);
- }
- }
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_dvs() leave:\n");
-}
-
-/* Code generated by genparam/genconfig.c:gen_configure_function() */
-
-void
-ia_css_configure_qplane(
- const struct ia_css_binary *binary,
- const struct ia_css_qplane_configuration *config_dmem)
-{
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_qplane() enter:\n");
-
- {
- unsigned int offset = 0;
- unsigned int size = 0;
-
- if (binary->info->mem_offsets.offsets.config) {
- size = binary->info->mem_offsets.offsets.config->dmem.qplane.size;
- offset = binary->info->mem_offsets.offsets.config->dmem.qplane.offset;
- }
- if (size) {
- ia_css_qplane_config((struct sh_css_isp_qplane_isp_config *)
- &binary->mem_params.params[IA_CSS_PARAM_CLASS_CONFIG][IA_CSS_ISP_DMEM].address[offset],
- config_dmem, size);
- }
- }
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_qplane() leave:\n");
-}
-
-/* Code generated by genparam/genconfig.c:gen_configure_function() */
-
-void
-ia_css_configure_output0(
- const struct ia_css_binary *binary,
- const struct ia_css_output0_configuration *config_dmem)
-{
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_output0() enter:\n");
-
- {
- unsigned int offset = 0;
- unsigned int size = 0;
-
- if (binary->info->mem_offsets.offsets.config) {
- size = binary->info->mem_offsets.offsets.config->dmem.output0.size;
- offset = binary->info->mem_offsets.offsets.config->dmem.output0.offset;
- }
- if (size) {
- ia_css_output0_config((struct sh_css_isp_output_isp_config *)
- &binary->mem_params.params[IA_CSS_PARAM_CLASS_CONFIG][IA_CSS_ISP_DMEM].address[offset],
- config_dmem, size);
- }
- }
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_output0() leave:\n");
-}
-
-/* Code generated by genparam/genconfig.c:gen_configure_function() */
-
-void
-ia_css_configure_output1(
- const struct ia_css_binary *binary,
- const struct ia_css_output1_configuration *config_dmem)
-{
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_output1() enter:\n");
-
- {
- unsigned int offset = 0;
- unsigned int size = 0;
-
- if (binary->info->mem_offsets.offsets.config) {
- size = binary->info->mem_offsets.offsets.config->dmem.output1.size;
- offset = binary->info->mem_offsets.offsets.config->dmem.output1.offset;
- }
- if (size) {
- ia_css_output1_config((struct sh_css_isp_output_isp_config *)
- &binary->mem_params.params[IA_CSS_PARAM_CLASS_CONFIG][IA_CSS_ISP_DMEM].address[offset],
- config_dmem, size);
- }
- }
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_output1() leave:\n");
-}
-
-/* Code generated by genparam/genconfig.c:gen_configure_function() */
-
-void
-ia_css_configure_output(
- const struct ia_css_binary *binary,
- const struct ia_css_output_configuration *config_dmem)
-{
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_output() enter:\n");
-
- {
- unsigned int offset = 0;
- unsigned int size = 0;
-
- if (binary->info->mem_offsets.offsets.config) {
- size = binary->info->mem_offsets.offsets.config->dmem.output.size;
- offset = binary->info->mem_offsets.offsets.config->dmem.output.offset;
- }
- if (size) {
- ia_css_output_config((struct sh_css_isp_output_isp_config *)
- &binary->mem_params.params[IA_CSS_PARAM_CLASS_CONFIG][IA_CSS_ISP_DMEM].address[offset],
- config_dmem, size);
- }
- }
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_output() leave:\n");
-}
-
-/* Code generated by genparam/genconfig.c:gen_configure_function() */
-
-void
-ia_css_configure_raw(
- const struct ia_css_binary *binary,
- const struct ia_css_raw_configuration *config_dmem)
-{
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_raw() enter:\n");
-
- {
- unsigned int offset = 0;
- unsigned int size = 0;
-
- if (binary->info->mem_offsets.offsets.config) {
- size = binary->info->mem_offsets.offsets.config->dmem.raw.size;
- offset = binary->info->mem_offsets.offsets.config->dmem.raw.offset;
- }
- if (size) {
- ia_css_raw_config((struct sh_css_isp_raw_isp_config *)
- &binary->mem_params.params[IA_CSS_PARAM_CLASS_CONFIG][IA_CSS_ISP_DMEM].address[offset],
- config_dmem, size);
- }
- }
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_raw() leave:\n");
-}
-
-/* Code generated by genparam/genconfig.c:gen_configure_function() */
-
-void
-ia_css_configure_tnr(
- const struct ia_css_binary *binary,
- const struct ia_css_tnr_configuration *config_dmem)
-{
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_tnr() enter:\n");
-
- {
- unsigned int offset = 0;
- unsigned int size = 0;
-
- if (binary->info->mem_offsets.offsets.config) {
- size = binary->info->mem_offsets.offsets.config->dmem.tnr.size;
- offset = binary->info->mem_offsets.offsets.config->dmem.tnr.offset;
- }
- if (size) {
- ia_css_tnr_config((struct sh_css_isp_tnr_isp_config *)
- &binary->mem_params.params[IA_CSS_PARAM_CLASS_CONFIG][IA_CSS_ISP_DMEM].address[offset],
- config_dmem, size);
- }
- }
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_tnr() leave:\n");
-}
-
-/* Code generated by genparam/genconfig.c:gen_configure_function() */
-
-void
-ia_css_configure_ref(
- const struct ia_css_binary *binary,
- const struct ia_css_ref_configuration *config_dmem)
-{
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_ref() enter:\n");
-
- {
- unsigned int offset = 0;
- unsigned int size = 0;
-
- if (binary->info->mem_offsets.offsets.config) {
- size = binary->info->mem_offsets.offsets.config->dmem.ref.size;
- offset = binary->info->mem_offsets.offsets.config->dmem.ref.offset;
- }
- if (size) {
- ia_css_ref_config((struct sh_css_isp_ref_isp_config *)
- &binary->mem_params.params[IA_CSS_PARAM_CLASS_CONFIG][IA_CSS_ISP_DMEM].address[offset],
- config_dmem, size);
- }
- }
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_ref() leave:\n");
-}
-
-/* Code generated by genparam/genconfig.c:gen_configure_function() */
-
-void
-ia_css_configure_vf(
- const struct ia_css_binary *binary,
- const struct ia_css_vf_configuration *config_dmem)
-{
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_vf() enter:\n");
-
- {
- unsigned int offset = 0;
- unsigned int size = 0;
-
- if (binary->info->mem_offsets.offsets.config) {
- size = binary->info->mem_offsets.offsets.config->dmem.vf.size;
- offset = binary->info->mem_offsets.offsets.config->dmem.vf.offset;
- }
- if (size) {
- ia_css_vf_config((struct sh_css_isp_vf_isp_config *)
- &binary->mem_params.params[IA_CSS_PARAM_CLASS_CONFIG][IA_CSS_ISP_DMEM].address[offset],
- config_dmem, size);
- }
- }
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_vf() leave:\n");
-}
diff --git a/drivers/staging/media/atomisp/pci/css_2400_system/hive/ia_css_isp_params.c b/drivers/staging/media/atomisp/pci/css_2400_system/hive/ia_css_isp_params.c
deleted file mode 100644
index b786247..0000000
--- a/drivers/staging/media/atomisp/pci/css_2400_system/hive/ia_css_isp_params.c
+++ /dev/null
@@ -1,3420 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/*
- * Support for Intel Camera Imaging ISP subsystem.
- * Copyright (c) 2015, Intel Corporation.
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms and conditions of the GNU General Public License,
- * version 2, as published by the Free Software Foundation.
- *
- * This program is distributed in the hope it will be useful, but WITHOUT
- * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
- * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
- * more details.
- */
-#define IA_CSS_INCLUDE_PARAMETERS
-#include "sh_css_params.h"
-#include "isp/kernels/aa/aa_2/ia_css_aa2.host.h"
-#include "isp/kernels/anr/anr_1.0/ia_css_anr.host.h"
-#include "isp/kernels/anr/anr_2/ia_css_anr2.host.h"
-#include "isp/kernels/bh/bh_2/ia_css_bh.host.h"
-#include "isp/kernels/bnr/bnr_1.0/ia_css_bnr.host.h"
-#include "isp/kernels/bnr/bnr2_2/ia_css_bnr2_2.host.h"
-#include "isp/kernels/cnr/cnr_2/ia_css_cnr2.host.h"
-#include "isp/kernels/crop/crop_1.0/ia_css_crop.host.h"
-#include "isp/kernels/csc/csc_1.0/ia_css_csc.host.h"
-#include "isp/kernels/ctc/ctc_1.0/ia_css_ctc.host.h"
-#include "isp/kernels/ctc/ctc1_5/ia_css_ctc1_5.host.h"
-#include "isp/kernels/ctc/ctc2/ia_css_ctc2.host.h"
-#include "isp/kernels/de/de_1.0/ia_css_de.host.h"
-#include "isp/kernels/de/de_2/ia_css_de2.host.h"
-#include "isp/kernels/dp/dp_1.0/ia_css_dp.host.h"
-#include "isp/kernels/fixedbds/fixedbds_1.0/ia_css_fixedbds_param.h"
-#include "isp/kernels/fpn/fpn_1.0/ia_css_fpn.host.h"
-#include "isp/kernels/gc/gc_1.0/ia_css_gc.host.h"
-#include "isp/kernels/gc/gc_2/ia_css_gc2.host.h"
-#include "isp/kernels/macc/macc_1.0/ia_css_macc.host.h"
-#include "isp/kernels/macc/macc1_5/ia_css_macc1_5.host.h"
-#include "isp/kernels/ob/ob_1.0/ia_css_ob.host.h"
-#include "isp/kernels/ob/ob2/ia_css_ob2.host.h"
-#include "isp/kernels/output/output_1.0/ia_css_output.host.h"
-#include "isp/kernels/raw_aa_binning/raw_aa_binning_1.0/ia_css_raa.host.h"
-#include "isp/kernels/s3a/s3a_1.0/ia_css_s3a.host.h"
-#include "isp/kernels/sc/sc_1.0/ia_css_sc.host.h"
-#include "isp/kernels/sdis/sdis_1.0/ia_css_sdis.host.h"
-#include "isp/kernels/sdis/sdis_2/ia_css_sdis2.host.h"
-#include "isp/kernels/tnr/tnr_1.0/ia_css_tnr.host.h"
-#include "isp/kernels/uds/uds_1.0/ia_css_uds_param.h"
-#include "isp/kernels/wb/wb_1.0/ia_css_wb.host.h"
-#include "isp/kernels/xnr/xnr_1.0/ia_css_xnr.host.h"
-#include "isp/kernels/xnr/xnr_3.0/ia_css_xnr3.host.h"
-#include "isp/kernels/ynr/ynr_1.0/ia_css_ynr.host.h"
-#include "isp/kernels/ynr/ynr_2/ia_css_ynr2.host.h"
-#include "isp/kernels/fc/fc_1.0/ia_css_formats.host.h"
-#include "isp/kernels/tdf/tdf_1.0/ia_css_tdf.host.h"
-#include "isp/kernels/dpc2/ia_css_dpc2.host.h"
-#include "isp/kernels/eed1_8/ia_css_eed1_8.host.h"
-#include "isp/kernels/bnlm/ia_css_bnlm.host.h"
-#include "isp/kernels/conversion/conversion_1.0/ia_css_conversion.host.h"
-/* Generated code: do not edit or commmit. */
-
-#include "ia_css_pipeline.h"
-#include "ia_css_isp_params.h"
-#include "ia_css_debug.h"
-#include "assert_support.h"
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_aa(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.aa.size;
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.aa.offset;
-
- if (size) {
- struct sh_css_isp_aa_params *t = (struct sh_css_isp_aa_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset];
- t->strength = params->aa_config.strength;
- }
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_anr(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.anr.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.anr.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_anr() enter:\n");
-
- ia_css_anr_encode((struct sh_css_isp_anr_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->anr_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_anr() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_anr2(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->vmem.anr2.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->vmem.anr2.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_anr2() enter:\n");
-
- ia_css_anr2_vmem_encode((struct ia_css_isp_anr2_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_VMEM].address[offset],
- ¶ms->anr_thres,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_VMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_anr2() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_bh(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.bh.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.bh.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_bh() enter:\n");
-
- ia_css_bh_encode((struct sh_css_isp_bh_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->s3a_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_bh() leave:\n");
- }
- }
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->hmem0.bh.size;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_bh() enter:\n");
-
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_HMEM0] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_bh() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_cnr(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.cnr.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.cnr.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_cnr() enter:\n");
-
- ia_css_cnr_encode((struct sh_css_isp_cnr_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->cnr_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_cnr() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_crop(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.crop.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.crop.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_crop() enter:\n");
-
- ia_css_crop_encode((struct sh_css_isp_crop_isp_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->crop_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_crop() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_csc(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.csc.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.csc.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_csc() enter:\n");
-
- ia_css_csc_encode((struct sh_css_isp_csc_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->cc_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_csc() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_dp(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.dp.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.dp.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_dp() enter:\n");
-
- ia_css_dp_encode((struct sh_css_isp_dp_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->dp_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_dp() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_bnr(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.bnr.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.bnr.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_bnr() enter:\n");
-
- ia_css_bnr_encode((struct sh_css_isp_bnr_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->nr_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_bnr() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_de(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.de.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.de.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_de() enter:\n");
-
- ia_css_de_encode((struct sh_css_isp_de_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->de_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_de() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_ecd(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.ecd.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.ecd.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_ecd() enter:\n");
-
- ia_css_ecd_encode((struct sh_css_isp_ecd_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->ecd_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_ecd() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_formats(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.formats.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.formats.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_formats() enter:\n");
-
- ia_css_formats_encode((struct sh_css_isp_formats_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->formats_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_formats() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_fpn(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.fpn.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.fpn.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_fpn() enter:\n");
-
- ia_css_fpn_encode((struct sh_css_isp_fpn_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->fpn_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_fpn() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_gc(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.gc.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.gc.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_gc() enter:\n");
-
- ia_css_gc_encode((struct sh_css_isp_gc_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->gc_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_gc() leave:\n");
- }
- }
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->vamem1.gc.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->vamem1.gc.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_gc() enter:\n");
-
- ia_css_gc_vamem_encode((struct sh_css_isp_gc_vamem_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_VAMEM1].address[offset],
- ¶ms->gc_table,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_VAMEM1] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_gc() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_ce(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.ce.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.ce.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_ce() enter:\n");
-
- ia_css_ce_encode((struct sh_css_isp_ce_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->ce_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_ce() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_yuv2rgb(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.yuv2rgb.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.yuv2rgb.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_yuv2rgb() enter:\n");
-
- ia_css_yuv2rgb_encode((struct sh_css_isp_csc_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->yuv2rgb_cc_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_yuv2rgb() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_rgb2yuv(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.rgb2yuv.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.rgb2yuv.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_rgb2yuv() enter:\n");
-
- ia_css_rgb2yuv_encode((struct sh_css_isp_csc_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->rgb2yuv_cc_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_rgb2yuv() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_r_gamma(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->vamem0.r_gamma.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->vamem0.r_gamma.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_r_gamma() enter:\n");
-
- ia_css_r_gamma_vamem_encode((struct sh_css_isp_rgb_gamma_vamem_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_VAMEM0].address[offset],
- ¶ms->r_gamma_table,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_VAMEM0] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_r_gamma() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_g_gamma(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->vamem1.g_gamma.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->vamem1.g_gamma.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_g_gamma() enter:\n");
-
- ia_css_g_gamma_vamem_encode((struct sh_css_isp_rgb_gamma_vamem_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_VAMEM1].address[offset],
- ¶ms->g_gamma_table,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_VAMEM1] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_g_gamma() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_b_gamma(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->vamem2.b_gamma.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->vamem2.b_gamma.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_b_gamma() enter:\n");
-
- ia_css_b_gamma_vamem_encode((struct sh_css_isp_rgb_gamma_vamem_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_VAMEM2].address[offset],
- ¶ms->b_gamma_table,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_VAMEM2] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_b_gamma() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_uds(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.uds.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.uds.offset;
-
- if (size) {
- struct sh_css_sp_uds_params *p;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_uds() enter:\n");
-
- p = (struct sh_css_sp_uds_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset];
- p->crop_pos = params->uds_config.crop_pos;
- p->uds = params->uds_config.uds;
-
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_uds() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_raa(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.raa.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.raa.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_raa() enter:\n");
-
- ia_css_raa_encode((struct sh_css_isp_aa_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->raa_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_raa() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_s3a(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.s3a.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.s3a.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_s3a() enter:\n");
-
- ia_css_s3a_encode((struct sh_css_isp_s3a_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->s3a_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_s3a() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_ob(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.ob.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.ob.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_ob() enter:\n");
-
- ia_css_ob_encode((struct sh_css_isp_ob_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->ob_config,
- ¶ms->stream_configs.ob, size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_ob() leave:\n");
- }
- }
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->vmem.ob.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->vmem.ob.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_ob() enter:\n");
-
- ia_css_ob_vmem_encode((struct sh_css_isp_ob_vmem_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_VMEM].address[offset],
- ¶ms->ob_config,
- ¶ms->stream_configs.ob, size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_VMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_ob() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_output(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.output.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.output.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_output() enter:\n");
-
- ia_css_output_encode((struct sh_css_isp_output_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->output_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_output() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_sc(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.sc.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.sc.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_sc() enter:\n");
-
- ia_css_sc_encode((struct sh_css_isp_sc_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->sc_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_sc() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_bds(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.bds.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.bds.offset;
-
- if (size) {
- struct sh_css_isp_bds_params *p;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_bds() enter:\n");
-
- p = (struct sh_css_isp_bds_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset];
- p->baf_strength = params->bds_config.strength;
-
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_bds() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_tnr(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.tnr.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.tnr.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_tnr() enter:\n");
-
- ia_css_tnr_encode((struct sh_css_isp_tnr_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->tnr_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_tnr() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_macc(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.macc.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.macc.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_macc() enter:\n");
-
- ia_css_macc_encode((struct sh_css_isp_macc_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->macc_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_macc() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_sdis_horicoef(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->vmem.sdis_horicoef.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->vmem.sdis_horicoef.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_sdis_horicoef() enter:\n");
-
- ia_css_sdis_horicoef_vmem_encode((struct sh_css_isp_sdis_hori_coef_tbl *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_VMEM].address[offset],
- ¶ms->dvs_coefs,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_VMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_sdis_horicoef() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_sdis_vertcoef(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->vmem.sdis_vertcoef.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->vmem.sdis_vertcoef.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_sdis_vertcoef() enter:\n");
-
- ia_css_sdis_vertcoef_vmem_encode((struct sh_css_isp_sdis_vert_coef_tbl *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_VMEM].address[offset],
- ¶ms->dvs_coefs,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_VMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_sdis_vertcoef() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_sdis_horiproj(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.sdis_horiproj.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.sdis_horiproj.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_sdis_horiproj() enter:\n");
-
- ia_css_sdis_horiproj_encode((struct sh_css_isp_sdis_hori_proj_tbl *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->dvs_coefs,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_sdis_horiproj() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_sdis_vertproj(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.sdis_vertproj.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.sdis_vertproj.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_sdis_vertproj() enter:\n");
-
- ia_css_sdis_vertproj_encode((struct sh_css_isp_sdis_vert_proj_tbl *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->dvs_coefs,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_sdis_vertproj() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_sdis2_horicoef(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->vmem.sdis2_horicoef.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->vmem.sdis2_horicoef.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_sdis2_horicoef() enter:\n");
-
- ia_css_sdis2_horicoef_vmem_encode((struct sh_css_isp_sdis_hori_coef_tbl *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_VMEM].address[offset],
- ¶ms->dvs2_coefs,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_VMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_sdis2_horicoef() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_sdis2_vertcoef(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->vmem.sdis2_vertcoef.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->vmem.sdis2_vertcoef.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_sdis2_vertcoef() enter:\n");
-
- ia_css_sdis2_vertcoef_vmem_encode((struct sh_css_isp_sdis_vert_coef_tbl *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_VMEM].address[offset],
- ¶ms->dvs2_coefs,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_VMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_sdis2_vertcoef() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_sdis2_horiproj(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.sdis2_horiproj.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.sdis2_horiproj.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_sdis2_horiproj() enter:\n");
-
- ia_css_sdis2_horiproj_encode((struct sh_css_isp_sdis_hori_proj_tbl *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->dvs2_coefs,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_sdis2_horiproj() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_sdis2_vertproj(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.sdis2_vertproj.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.sdis2_vertproj.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_sdis2_vertproj() enter:\n");
-
- ia_css_sdis2_vertproj_encode((struct sh_css_isp_sdis_vert_proj_tbl *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->dvs2_coefs,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_sdis2_vertproj() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_wb(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.wb.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.wb.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_wb() enter:\n");
-
- ia_css_wb_encode((struct sh_css_isp_wb_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->wb_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_wb() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_nr(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.nr.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.nr.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_nr() enter:\n");
-
- ia_css_nr_encode((struct sh_css_isp_ynr_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->nr_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_nr() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_yee(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.yee.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.yee.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_yee() enter:\n");
-
- ia_css_yee_encode((struct sh_css_isp_yee_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->yee_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_yee() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_ynr(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.ynr.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.ynr.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_ynr() enter:\n");
-
- ia_css_ynr_encode((struct sh_css_isp_yee2_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->ynr_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_ynr() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_fc(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.fc.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.fc.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_fc() enter:\n");
-
- ia_css_fc_encode((struct sh_css_isp_fc_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->fc_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_fc() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_ctc(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.ctc.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.ctc.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_ctc() enter:\n");
-
- ia_css_ctc_encode((struct sh_css_isp_ctc_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->ctc_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_ctc() leave:\n");
- }
- }
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->vamem0.ctc.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->vamem0.ctc.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_ctc() enter:\n");
-
- ia_css_ctc_vamem_encode((struct sh_css_isp_ctc_vamem_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_VAMEM0].address[offset],
- ¶ms->ctc_table,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_VAMEM0] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_ctc() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_xnr_table(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->vamem1.xnr_table.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->vamem1.xnr_table.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_xnr_table() enter:\n");
-
- ia_css_xnr_table_vamem_encode((struct sh_css_isp_xnr_vamem_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_VAMEM1].address[offset],
- ¶ms->xnr_table,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_VAMEM1] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_xnr_table() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_xnr(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.xnr.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.xnr.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_xnr() enter:\n");
-
- ia_css_xnr_encode((struct sh_css_isp_xnr_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->xnr_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_xnr() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_xnr3(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.xnr3.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.xnr3.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_xnr3() enter:\n");
-
- ia_css_xnr3_encode((struct sh_css_isp_xnr3_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->xnr3_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_xnr3() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_param_process_table() */
-
-void (*ia_css_kernel_process_param[IA_CSS_NUM_PARAMETER_IDS])(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params) = {
- ia_css_process_aa,
- ia_css_process_anr,
- ia_css_process_anr2,
- ia_css_process_bh,
- ia_css_process_cnr,
- ia_css_process_crop,
- ia_css_process_csc,
- ia_css_process_dp,
- ia_css_process_bnr,
- ia_css_process_de,
- ia_css_process_ecd,
- ia_css_process_formats,
- ia_css_process_fpn,
- ia_css_process_gc,
- ia_css_process_ce,
- ia_css_process_yuv2rgb,
- ia_css_process_rgb2yuv,
- ia_css_process_r_gamma,
- ia_css_process_g_gamma,
- ia_css_process_b_gamma,
- ia_css_process_uds,
- ia_css_process_raa,
- ia_css_process_s3a,
- ia_css_process_ob,
- ia_css_process_output,
- ia_css_process_sc,
- ia_css_process_bds,
- ia_css_process_tnr,
- ia_css_process_macc,
- ia_css_process_sdis_horicoef,
- ia_css_process_sdis_vertcoef,
- ia_css_process_sdis_horiproj,
- ia_css_process_sdis_vertproj,
- ia_css_process_sdis2_horicoef,
- ia_css_process_sdis2_vertcoef,
- ia_css_process_sdis2_horiproj,
- ia_css_process_sdis2_vertproj,
- ia_css_process_wb,
- ia_css_process_nr,
- ia_css_process_yee,
- ia_css_process_ynr,
- ia_css_process_fc,
- ia_css_process_ctc,
- ia_css_process_xnr_table,
- ia_css_process_xnr,
- ia_css_process_xnr3,
-};
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_dp_config(const struct ia_css_isp_parameters *params,
- struct ia_css_dp_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_dp_config() enter: config=%p\n",
- config);
-
- *config = params->dp_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_dp_config() leave\n");
- ia_css_dp_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_dp_config(struct ia_css_isp_parameters *params,
- const struct ia_css_dp_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_dp_config() enter:\n");
- ia_css_dp_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->dp_config = *config;
- params->config_changed[IA_CSS_DP_ID] = true;
- params->config_changed[IA_CSS_DP_ID] = true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_dp_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_wb_config(const struct ia_css_isp_parameters *params,
- struct ia_css_wb_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_wb_config() enter: config=%p\n",
- config);
-
- *config = params->wb_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_wb_config() leave\n");
- ia_css_wb_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_wb_config(struct ia_css_isp_parameters *params,
- const struct ia_css_wb_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_wb_config() enter:\n");
- ia_css_wb_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->wb_config = *config;
- params->config_changed[IA_CSS_WB_ID] = true;
- params->config_changed[IA_CSS_WB_ID] = true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_wb_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_tnr_config(const struct ia_css_isp_parameters *params,
- struct ia_css_tnr_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_tnr_config() enter: config=%p\n",
- config);
-
- *config = params->tnr_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_tnr_config() leave\n");
- ia_css_tnr_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_tnr_config(struct ia_css_isp_parameters *params,
- const struct ia_css_tnr_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_tnr_config() enter:\n");
- ia_css_tnr_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->tnr_config = *config;
- params->config_changed[IA_CSS_TNR_ID] = true;
- params->config_changed[IA_CSS_TNR_ID] = true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_tnr_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_ob_config(const struct ia_css_isp_parameters *params,
- struct ia_css_ob_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_ob_config() enter: config=%p\n",
- config);
-
- *config = params->ob_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_ob_config() leave\n");
- ia_css_ob_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_ob_config(struct ia_css_isp_parameters *params,
- const struct ia_css_ob_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_ob_config() enter:\n");
- ia_css_ob_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->ob_config = *config;
- params->config_changed[IA_CSS_OB_ID] = true;
- params->config_changed[IA_CSS_OB_ID] = true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_ob_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_de_config(const struct ia_css_isp_parameters *params,
- struct ia_css_de_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_de_config() enter: config=%p\n",
- config);
-
- *config = params->de_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_de_config() leave\n");
- ia_css_de_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_de_config(struct ia_css_isp_parameters *params,
- const struct ia_css_de_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_de_config() enter:\n");
- ia_css_de_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->de_config = *config;
- params->config_changed[IA_CSS_DE_ID] = true;
- params->config_changed[IA_CSS_DE_ID] = true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_de_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_anr_config(const struct ia_css_isp_parameters *params,
- struct ia_css_anr_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_anr_config() enter: config=%p\n",
- config);
-
- *config = params->anr_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_anr_config() leave\n");
- ia_css_anr_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_anr_config(struct ia_css_isp_parameters *params,
- const struct ia_css_anr_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_anr_config() enter:\n");
- ia_css_anr_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->anr_config = *config;
- params->config_changed[IA_CSS_ANR_ID] = true;
- params->config_changed[IA_CSS_ANR_ID] = true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_anr_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_anr2_config(const struct ia_css_isp_parameters *params,
- struct ia_css_anr_thres *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_anr2_config() enter: config=%p\n",
- config);
-
- *config = params->anr_thres;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_anr2_config() leave\n");
- ia_css_anr2_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_anr2_config(struct ia_css_isp_parameters *params,
- const struct ia_css_anr_thres *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_anr2_config() enter:\n");
- ia_css_anr2_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->anr_thres = *config;
- params->config_changed[IA_CSS_ANR2_ID] = true;
- params->config_changed[IA_CSS_ANR2_ID] = true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_anr2_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_ce_config(const struct ia_css_isp_parameters *params,
- struct ia_css_ce_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_ce_config() enter: config=%p\n",
- config);
-
- *config = params->ce_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_ce_config() leave\n");
- ia_css_ce_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_ce_config(struct ia_css_isp_parameters *params,
- const struct ia_css_ce_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_ce_config() enter:\n");
- ia_css_ce_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->ce_config = *config;
- params->config_changed[IA_CSS_CE_ID] = true;
- params->config_changed[IA_CSS_CE_ID] = true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_ce_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_ecd_config(const struct ia_css_isp_parameters *params,
- struct ia_css_ecd_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_ecd_config() enter: config=%p\n",
- config);
-
- *config = params->ecd_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_ecd_config() leave\n");
- ia_css_ecd_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_ecd_config(struct ia_css_isp_parameters *params,
- const struct ia_css_ecd_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_ecd_config() enter:\n");
- ia_css_ecd_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->ecd_config = *config;
- params->config_changed[IA_CSS_ECD_ID] = true;
- params->config_changed[IA_CSS_ECD_ID] = true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_ecd_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_ynr_config(const struct ia_css_isp_parameters *params,
- struct ia_css_ynr_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_ynr_config() enter: config=%p\n",
- config);
-
- *config = params->ynr_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_ynr_config() leave\n");
- ia_css_ynr_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_ynr_config(struct ia_css_isp_parameters *params,
- const struct ia_css_ynr_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_ynr_config() enter:\n");
- ia_css_ynr_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->ynr_config = *config;
- params->config_changed[IA_CSS_YNR_ID] = true;
- params->config_changed[IA_CSS_YNR_ID] = true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_ynr_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_fc_config(const struct ia_css_isp_parameters *params,
- struct ia_css_fc_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_fc_config() enter: config=%p\n",
- config);
-
- *config = params->fc_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_fc_config() leave\n");
- ia_css_fc_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_fc_config(struct ia_css_isp_parameters *params,
- const struct ia_css_fc_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_fc_config() enter:\n");
- ia_css_fc_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->fc_config = *config;
- params->config_changed[IA_CSS_FC_ID] = true;
- params->config_changed[IA_CSS_FC_ID] = true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_fc_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_cnr_config(const struct ia_css_isp_parameters *params,
- struct ia_css_cnr_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_cnr_config() enter: config=%p\n",
- config);
-
- *config = params->cnr_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_cnr_config() leave\n");
- ia_css_cnr_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_cnr_config(struct ia_css_isp_parameters *params,
- const struct ia_css_cnr_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_cnr_config() enter:\n");
- ia_css_cnr_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->cnr_config = *config;
- params->config_changed[IA_CSS_CNR_ID] = true;
- params->config_changed[IA_CSS_CNR_ID] = true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_cnr_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_macc_config(const struct ia_css_isp_parameters *params,
- struct ia_css_macc_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_macc_config() enter: config=%p\n",
- config);
-
- *config = params->macc_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_macc_config() leave\n");
- ia_css_macc_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_macc_config(struct ia_css_isp_parameters *params,
- const struct ia_css_macc_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_macc_config() enter:\n");
- ia_css_macc_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->macc_config = *config;
- params->config_changed[IA_CSS_MACC_ID] = true;
- params->config_changed[IA_CSS_MACC_ID] = true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_macc_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_ctc_config(const struct ia_css_isp_parameters *params,
- struct ia_css_ctc_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_ctc_config() enter: config=%p\n",
- config);
-
- *config = params->ctc_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_ctc_config() leave\n");
- ia_css_ctc_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_ctc_config(struct ia_css_isp_parameters *params,
- const struct ia_css_ctc_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_ctc_config() enter:\n");
- ia_css_ctc_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->ctc_config = *config;
- params->config_changed[IA_CSS_CTC_ID] = true;
- params->config_changed[IA_CSS_CTC_ID] = true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_ctc_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_aa_config(const struct ia_css_isp_parameters *params,
- struct ia_css_aa_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_aa_config() enter: config=%p\n",
- config);
-
- *config = params->aa_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_aa_config() leave\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_aa_config(struct ia_css_isp_parameters *params,
- const struct ia_css_aa_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_aa_config() enter:\n");
- params->aa_config = *config;
- params->config_changed[IA_CSS_AA_ID] = true;
- params->config_changed[IA_CSS_AA_ID] = true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_aa_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_yuv2rgb_config(const struct ia_css_isp_parameters *params,
- struct ia_css_cc_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_yuv2rgb_config() enter: config=%p\n",
- config);
-
- *config = params->yuv2rgb_cc_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_yuv2rgb_config() leave\n");
- ia_css_yuv2rgb_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_yuv2rgb_config(struct ia_css_isp_parameters *params,
- const struct ia_css_cc_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_yuv2rgb_config() enter:\n");
- ia_css_yuv2rgb_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->yuv2rgb_cc_config = *config;
- params->config_changed[IA_CSS_YUV2RGB_ID] = true;
- params->config_changed[IA_CSS_YUV2RGB_ID] = true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_yuv2rgb_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_rgb2yuv_config(const struct ia_css_isp_parameters *params,
- struct ia_css_cc_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_rgb2yuv_config() enter: config=%p\n",
- config);
-
- *config = params->rgb2yuv_cc_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_rgb2yuv_config() leave\n");
- ia_css_rgb2yuv_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_rgb2yuv_config(struct ia_css_isp_parameters *params,
- const struct ia_css_cc_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_rgb2yuv_config() enter:\n");
- ia_css_rgb2yuv_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->rgb2yuv_cc_config = *config;
- params->config_changed[IA_CSS_RGB2YUV_ID] = true;
- params->config_changed[IA_CSS_RGB2YUV_ID] = true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_rgb2yuv_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_csc_config(const struct ia_css_isp_parameters *params,
- struct ia_css_cc_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_csc_config() enter: config=%p\n",
- config);
-
- *config = params->cc_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_csc_config() leave\n");
- ia_css_csc_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_csc_config(struct ia_css_isp_parameters *params,
- const struct ia_css_cc_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_csc_config() enter:\n");
- ia_css_csc_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->cc_config = *config;
- params->config_changed[IA_CSS_CSC_ID] = true;
- params->config_changed[IA_CSS_CSC_ID] = true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_csc_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_nr_config(const struct ia_css_isp_parameters *params,
- struct ia_css_nr_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_nr_config() enter: config=%p\n",
- config);
-
- *config = params->nr_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_nr_config() leave\n");
- ia_css_nr_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_nr_config(struct ia_css_isp_parameters *params,
- const struct ia_css_nr_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_nr_config() enter:\n");
- ia_css_nr_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->nr_config = *config;
- params->config_changed[IA_CSS_BNR_ID] = true;
- params->config_changed[IA_CSS_NR_ID] = true;
- params->config_changed[IA_CSS_NR_ID] = true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_nr_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_gc_config(const struct ia_css_isp_parameters *params,
- struct ia_css_gc_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_gc_config() enter: config=%p\n",
- config);
-
- *config = params->gc_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_gc_config() leave\n");
- ia_css_gc_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_gc_config(struct ia_css_isp_parameters *params,
- const struct ia_css_gc_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_gc_config() enter:\n");
- ia_css_gc_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->gc_config = *config;
- params->config_changed[IA_CSS_GC_ID] = true;
- params->config_changed[IA_CSS_GC_ID] = true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_gc_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_sdis_horicoef_config(const struct ia_css_isp_parameters *params,
- struct ia_css_dvs_coefficients *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_sdis_horicoef_config() enter: config=%p\n",
- config);
-
- *config = params->dvs_coefs;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_sdis_horicoef_config() leave\n");
- ia_css_sdis_horicoef_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_sdis_horicoef_config(struct ia_css_isp_parameters *params,
- const struct ia_css_dvs_coefficients *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE,
- "ia_css_set_sdis_horicoef_config() enter:\n");
- ia_css_sdis_horicoef_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->dvs_coefs = *config;
- params->config_changed[IA_CSS_SDIS_HORICOEF_ID] = true;
- params->config_changed[IA_CSS_SDIS_VERTCOEF_ID] = true;
- params->config_changed[IA_CSS_SDIS_HORIPROJ_ID] = true;
- params->config_changed[IA_CSS_SDIS_VERTPROJ_ID] = true;
- params->config_changed[IA_CSS_SDIS_HORICOEF_ID] = true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_sdis_horicoef_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_sdis_vertcoef_config(const struct ia_css_isp_parameters *params,
- struct ia_css_dvs_coefficients *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_sdis_vertcoef_config() enter: config=%p\n",
- config);
-
- *config = params->dvs_coefs;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_sdis_vertcoef_config() leave\n");
- ia_css_sdis_vertcoef_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_sdis_vertcoef_config(struct ia_css_isp_parameters *params,
- const struct ia_css_dvs_coefficients *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE,
- "ia_css_set_sdis_vertcoef_config() enter:\n");
- ia_css_sdis_vertcoef_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->dvs_coefs = *config;
- params->config_changed[IA_CSS_SDIS_HORICOEF_ID] = true;
- params->config_changed[IA_CSS_SDIS_VERTCOEF_ID] = true;
- params->config_changed[IA_CSS_SDIS_HORIPROJ_ID] = true;
- params->config_changed[IA_CSS_SDIS_VERTPROJ_ID] = true;
- params->config_changed[IA_CSS_SDIS_VERTCOEF_ID] = true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_sdis_vertcoef_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_sdis_horiproj_config(const struct ia_css_isp_parameters *params,
- struct ia_css_dvs_coefficients *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_sdis_horiproj_config() enter: config=%p\n",
- config);
-
- *config = params->dvs_coefs;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_sdis_horiproj_config() leave\n");
- ia_css_sdis_horiproj_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_sdis_horiproj_config(struct ia_css_isp_parameters *params,
- const struct ia_css_dvs_coefficients *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE,
- "ia_css_set_sdis_horiproj_config() enter:\n");
- ia_css_sdis_horiproj_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->dvs_coefs = *config;
- params->config_changed[IA_CSS_SDIS_HORICOEF_ID] = true;
- params->config_changed[IA_CSS_SDIS_VERTCOEF_ID] = true;
- params->config_changed[IA_CSS_SDIS_HORIPROJ_ID] = true;
- params->config_changed[IA_CSS_SDIS_VERTPROJ_ID] = true;
- params->config_changed[IA_CSS_SDIS_HORIPROJ_ID] = true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_sdis_horiproj_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_sdis_vertproj_config(const struct ia_css_isp_parameters *params,
- struct ia_css_dvs_coefficients *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_sdis_vertproj_config() enter: config=%p\n",
- config);
-
- *config = params->dvs_coefs;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_sdis_vertproj_config() leave\n");
- ia_css_sdis_vertproj_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_sdis_vertproj_config(struct ia_css_isp_parameters *params,
- const struct ia_css_dvs_coefficients *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE,
- "ia_css_set_sdis_vertproj_config() enter:\n");
- ia_css_sdis_vertproj_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->dvs_coefs = *config;
- params->config_changed[IA_CSS_SDIS_HORICOEF_ID] = true;
- params->config_changed[IA_CSS_SDIS_VERTCOEF_ID] = true;
- params->config_changed[IA_CSS_SDIS_HORIPROJ_ID] = true;
- params->config_changed[IA_CSS_SDIS_VERTPROJ_ID] = true;
- params->config_changed[IA_CSS_SDIS_VERTPROJ_ID] = true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_sdis_vertproj_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_sdis2_horicoef_config(const struct ia_css_isp_parameters *params,
- struct ia_css_dvs2_coefficients *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_sdis2_horicoef_config() enter: config=%p\n",
- config);
-
- *config = params->dvs2_coefs;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_sdis2_horicoef_config() leave\n");
- ia_css_sdis2_horicoef_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_sdis2_horicoef_config(struct ia_css_isp_parameters *params,
- const struct ia_css_dvs2_coefficients *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE,
- "ia_css_set_sdis2_horicoef_config() enter:\n");
- ia_css_sdis2_horicoef_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->dvs2_coefs = *config;
- params->config_changed[IA_CSS_SDIS2_HORICOEF_ID] = true;
- params->config_changed[IA_CSS_SDIS2_VERTCOEF_ID] = true;
- params->config_changed[IA_CSS_SDIS2_HORIPROJ_ID] = true;
- params->config_changed[IA_CSS_SDIS2_VERTPROJ_ID] = true;
- params->config_changed[IA_CSS_SDIS2_HORICOEF_ID] = true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_sdis2_horicoef_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_sdis2_vertcoef_config(const struct ia_css_isp_parameters *params,
- struct ia_css_dvs2_coefficients *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_sdis2_vertcoef_config() enter: config=%p\n",
- config);
-
- *config = params->dvs2_coefs;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_sdis2_vertcoef_config() leave\n");
- ia_css_sdis2_vertcoef_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_sdis2_vertcoef_config(struct ia_css_isp_parameters *params,
- const struct ia_css_dvs2_coefficients *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE,
- "ia_css_set_sdis2_vertcoef_config() enter:\n");
- ia_css_sdis2_vertcoef_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->dvs2_coefs = *config;
- params->config_changed[IA_CSS_SDIS2_HORICOEF_ID] = true;
- params->config_changed[IA_CSS_SDIS2_VERTCOEF_ID] = true;
- params->config_changed[IA_CSS_SDIS2_HORIPROJ_ID] = true;
- params->config_changed[IA_CSS_SDIS2_VERTPROJ_ID] = true;
- params->config_changed[IA_CSS_SDIS2_VERTCOEF_ID] = true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_sdis2_vertcoef_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_sdis2_horiproj_config(const struct ia_css_isp_parameters *params,
- struct ia_css_dvs2_coefficients *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_sdis2_horiproj_config() enter: config=%p\n",
- config);
-
- *config = params->dvs2_coefs;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_sdis2_horiproj_config() leave\n");
- ia_css_sdis2_horiproj_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_sdis2_horiproj_config(struct ia_css_isp_parameters *params,
- const struct ia_css_dvs2_coefficients *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE,
- "ia_css_set_sdis2_horiproj_config() enter:\n");
- ia_css_sdis2_horiproj_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->dvs2_coefs = *config;
- params->config_changed[IA_CSS_SDIS2_HORICOEF_ID] = true;
- params->config_changed[IA_CSS_SDIS2_VERTCOEF_ID] = true;
- params->config_changed[IA_CSS_SDIS2_HORIPROJ_ID] = true;
- params->config_changed[IA_CSS_SDIS2_VERTPROJ_ID] = true;
- params->config_changed[IA_CSS_SDIS2_HORIPROJ_ID] = true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_sdis2_horiproj_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_sdis2_vertproj_config(const struct ia_css_isp_parameters *params,
- struct ia_css_dvs2_coefficients *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_sdis2_vertproj_config() enter: config=%p\n",
- config);
-
- *config = params->dvs2_coefs;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_sdis2_vertproj_config() leave\n");
- ia_css_sdis2_vertproj_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_sdis2_vertproj_config(struct ia_css_isp_parameters *params,
- const struct ia_css_dvs2_coefficients *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE,
- "ia_css_set_sdis2_vertproj_config() enter:\n");
- ia_css_sdis2_vertproj_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->dvs2_coefs = *config;
- params->config_changed[IA_CSS_SDIS2_HORICOEF_ID] = true;
- params->config_changed[IA_CSS_SDIS2_VERTCOEF_ID] = true;
- params->config_changed[IA_CSS_SDIS2_HORIPROJ_ID] = true;
- params->config_changed[IA_CSS_SDIS2_VERTPROJ_ID] = true;
- params->config_changed[IA_CSS_SDIS2_VERTPROJ_ID] = true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_sdis2_vertproj_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_r_gamma_config(const struct ia_css_isp_parameters *params,
- struct ia_css_rgb_gamma_table *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_r_gamma_config() enter: config=%p\n",
- config);
-
- *config = params->r_gamma_table;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_r_gamma_config() leave\n");
- ia_css_r_gamma_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_r_gamma_config(struct ia_css_isp_parameters *params,
- const struct ia_css_rgb_gamma_table *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_r_gamma_config() enter:\n");
- ia_css_r_gamma_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->r_gamma_table = *config;
- params->config_changed[IA_CSS_R_GAMMA_ID] = true;
- params->config_changed[IA_CSS_R_GAMMA_ID] = true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_r_gamma_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_g_gamma_config(const struct ia_css_isp_parameters *params,
- struct ia_css_rgb_gamma_table *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_g_gamma_config() enter: config=%p\n",
- config);
-
- *config = params->g_gamma_table;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_g_gamma_config() leave\n");
- ia_css_g_gamma_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_g_gamma_config(struct ia_css_isp_parameters *params,
- const struct ia_css_rgb_gamma_table *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_g_gamma_config() enter:\n");
- ia_css_g_gamma_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->g_gamma_table = *config;
- params->config_changed[IA_CSS_G_GAMMA_ID] = true;
- params->config_changed[IA_CSS_G_GAMMA_ID] = true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_g_gamma_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_b_gamma_config(const struct ia_css_isp_parameters *params,
- struct ia_css_rgb_gamma_table *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_b_gamma_config() enter: config=%p\n",
- config);
-
- *config = params->b_gamma_table;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_b_gamma_config() leave\n");
- ia_css_b_gamma_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_b_gamma_config(struct ia_css_isp_parameters *params,
- const struct ia_css_rgb_gamma_table *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_b_gamma_config() enter:\n");
- ia_css_b_gamma_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->b_gamma_table = *config;
- params->config_changed[IA_CSS_B_GAMMA_ID] = true;
- params->config_changed[IA_CSS_B_GAMMA_ID] = true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_b_gamma_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_xnr_table_config(const struct ia_css_isp_parameters *params,
- struct ia_css_xnr_table *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_xnr_table_config() enter: config=%p\n",
- config);
-
- *config = params->xnr_table;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_xnr_table_config() leave\n");
- ia_css_xnr_table_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_xnr_table_config(struct ia_css_isp_parameters *params,
- const struct ia_css_xnr_table *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE,
- "ia_css_set_xnr_table_config() enter:\n");
- ia_css_xnr_table_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->xnr_table = *config;
- params->config_changed[IA_CSS_XNR_TABLE_ID] = true;
- params->config_changed[IA_CSS_XNR_TABLE_ID] = true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_xnr_table_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_formats_config(const struct ia_css_isp_parameters *params,
- struct ia_css_formats_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_formats_config() enter: config=%p\n",
- config);
-
- *config = params->formats_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_formats_config() leave\n");
- ia_css_formats_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_formats_config(struct ia_css_isp_parameters *params,
- const struct ia_css_formats_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_formats_config() enter:\n");
- ia_css_formats_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->formats_config = *config;
- params->config_changed[IA_CSS_FORMATS_ID] = true;
- params->config_changed[IA_CSS_FORMATS_ID] = true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_formats_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_xnr_config(const struct ia_css_isp_parameters *params,
- struct ia_css_xnr_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_xnr_config() enter: config=%p\n",
- config);
-
- *config = params->xnr_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_xnr_config() leave\n");
- ia_css_xnr_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_xnr_config(struct ia_css_isp_parameters *params,
- const struct ia_css_xnr_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_xnr_config() enter:\n");
- ia_css_xnr_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->xnr_config = *config;
- params->config_changed[IA_CSS_XNR_ID] = true;
- params->config_changed[IA_CSS_XNR_ID] = true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_xnr_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_xnr3_config(const struct ia_css_isp_parameters *params,
- struct ia_css_xnr3_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_xnr3_config() enter: config=%p\n",
- config);
-
- *config = params->xnr3_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_xnr3_config() leave\n");
- ia_css_xnr3_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_xnr3_config(struct ia_css_isp_parameters *params,
- const struct ia_css_xnr3_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_xnr3_config() enter:\n");
- ia_css_xnr3_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->xnr3_config = *config;
- params->config_changed[IA_CSS_XNR3_ID] = true;
- params->config_changed[IA_CSS_XNR3_ID] = true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_xnr3_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_s3a_config(const struct ia_css_isp_parameters *params,
- struct ia_css_3a_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_s3a_config() enter: config=%p\n",
- config);
-
- *config = params->s3a_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_s3a_config() leave\n");
- ia_css_s3a_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_s3a_config(struct ia_css_isp_parameters *params,
- const struct ia_css_3a_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_s3a_config() enter:\n");
- ia_css_s3a_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->s3a_config = *config;
- params->config_changed[IA_CSS_BH_ID] = true;
- params->config_changed[IA_CSS_S3A_ID] = true;
- params->config_changed[IA_CSS_S3A_ID] = true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_s3a_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_output_config(const struct ia_css_isp_parameters *params,
- struct ia_css_output_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_output_config() enter: config=%p\n",
- config);
-
- *config = params->output_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_output_config() leave\n");
- ia_css_output_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_output_config(struct ia_css_isp_parameters *params,
- const struct ia_css_output_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_output_config() enter:\n");
- ia_css_output_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->output_config = *config;
- params->config_changed[IA_CSS_OUTPUT_ID] = true;
- params->config_changed[IA_CSS_OUTPUT_ID] = true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_output_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_global_access_function() */
-
-void
-ia_css_get_configs(struct ia_css_isp_parameters *params,
- const struct ia_css_isp_config *config)
-{
- ia_css_get_dp_config(params, config->dp_config);
- ia_css_get_wb_config(params, config->wb_config);
- ia_css_get_tnr_config(params, config->tnr_config);
- ia_css_get_ob_config(params, config->ob_config);
- ia_css_get_de_config(params, config->de_config);
- ia_css_get_anr_config(params, config->anr_config);
- ia_css_get_anr2_config(params, config->anr_thres);
- ia_css_get_ce_config(params, config->ce_config);
- ia_css_get_ecd_config(params, config->ecd_config);
- ia_css_get_ynr_config(params, config->ynr_config);
- ia_css_get_fc_config(params, config->fc_config);
- ia_css_get_cnr_config(params, config->cnr_config);
- ia_css_get_macc_config(params, config->macc_config);
- ia_css_get_ctc_config(params, config->ctc_config);
- ia_css_get_aa_config(params, config->aa_config);
- ia_css_get_yuv2rgb_config(params, config->yuv2rgb_cc_config);
- ia_css_get_rgb2yuv_config(params, config->rgb2yuv_cc_config);
- ia_css_get_csc_config(params, config->cc_config);
- ia_css_get_nr_config(params, config->nr_config);
- ia_css_get_gc_config(params, config->gc_config);
- ia_css_get_sdis_horicoef_config(params, config->dvs_coefs);
- ia_css_get_sdis_vertcoef_config(params, config->dvs_coefs);
- ia_css_get_sdis_horiproj_config(params, config->dvs_coefs);
- ia_css_get_sdis_vertproj_config(params, config->dvs_coefs);
- ia_css_get_sdis2_horicoef_config(params, config->dvs2_coefs);
- ia_css_get_sdis2_vertcoef_config(params, config->dvs2_coefs);
- ia_css_get_sdis2_horiproj_config(params, config->dvs2_coefs);
- ia_css_get_sdis2_vertproj_config(params, config->dvs2_coefs);
- ia_css_get_r_gamma_config(params, config->r_gamma_table);
- ia_css_get_g_gamma_config(params, config->g_gamma_table);
- ia_css_get_b_gamma_config(params, config->b_gamma_table);
- ia_css_get_xnr_table_config(params, config->xnr_table);
- ia_css_get_formats_config(params, config->formats_config);
- ia_css_get_xnr_config(params, config->xnr_config);
- ia_css_get_xnr3_config(params, config->xnr3_config);
- ia_css_get_s3a_config(params, config->s3a_config);
- ia_css_get_output_config(params, config->output_config);
-}
-
-/* Code generated by genparam/gencode.c:gen_global_access_function() */
-
-void
-ia_css_set_configs(struct ia_css_isp_parameters *params,
- const struct ia_css_isp_config *config)
-{
- ia_css_set_dp_config(params, config->dp_config);
- ia_css_set_wb_config(params, config->wb_config);
- ia_css_set_tnr_config(params, config->tnr_config);
- ia_css_set_ob_config(params, config->ob_config);
- ia_css_set_de_config(params, config->de_config);
- ia_css_set_anr_config(params, config->anr_config);
- ia_css_set_anr2_config(params, config->anr_thres);
- ia_css_set_ce_config(params, config->ce_config);
- ia_css_set_ecd_config(params, config->ecd_config);
- ia_css_set_ynr_config(params, config->ynr_config);
- ia_css_set_fc_config(params, config->fc_config);
- ia_css_set_cnr_config(params, config->cnr_config);
- ia_css_set_macc_config(params, config->macc_config);
- ia_css_set_ctc_config(params, config->ctc_config);
- ia_css_set_aa_config(params, config->aa_config);
- ia_css_set_yuv2rgb_config(params, config->yuv2rgb_cc_config);
- ia_css_set_rgb2yuv_config(params, config->rgb2yuv_cc_config);
- ia_css_set_csc_config(params, config->cc_config);
- ia_css_set_nr_config(params, config->nr_config);
- ia_css_set_gc_config(params, config->gc_config);
- ia_css_set_sdis_horicoef_config(params, config->dvs_coefs);
- ia_css_set_sdis_vertcoef_config(params, config->dvs_coefs);
- ia_css_set_sdis_horiproj_config(params, config->dvs_coefs);
- ia_css_set_sdis_vertproj_config(params, config->dvs_coefs);
- ia_css_set_sdis2_horicoef_config(params, config->dvs2_coefs);
- ia_css_set_sdis2_vertcoef_config(params, config->dvs2_coefs);
- ia_css_set_sdis2_horiproj_config(params, config->dvs2_coefs);
- ia_css_set_sdis2_vertproj_config(params, config->dvs2_coefs);
- ia_css_set_r_gamma_config(params, config->r_gamma_table);
- ia_css_set_g_gamma_config(params, config->g_gamma_table);
- ia_css_set_b_gamma_config(params, config->b_gamma_table);
- ia_css_set_xnr_table_config(params, config->xnr_table);
- ia_css_set_formats_config(params, config->formats_config);
- ia_css_set_xnr_config(params, config->xnr_config);
- ia_css_set_xnr3_config(params, config->xnr3_config);
- ia_css_set_s3a_config(params, config->s3a_config);
- ia_css_set_output_config(params, config->output_config);
-}
diff --git a/drivers/staging/media/atomisp/pci/css_2401_system/hive/ia_css_isp_configs.c b/drivers/staging/media/atomisp/pci/css_2401_system/hive/ia_css_isp_configs.c
deleted file mode 100644
index 1a021ae..0000000
--- a/drivers/staging/media/atomisp/pci/css_2401_system/hive/ia_css_isp_configs.c
+++ /dev/null
@@ -1,386 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/*
- * Support for Intel Camera Imaging ISP subsystem.
- * Copyright (c) 2015, Intel Corporation.
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms and conditions of the GNU General Public License,
- * version 2, as published by the Free Software Foundation.
- *
- * This program is distributed in the hope it will be useful, but WITHOUT
- * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
- * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
- * more details.
- */
-
-/* Generated code: do not edit or commmit. */
-
-#define IA_CSS_INCLUDE_CONFIGURATIONS
-#include "ia_css_pipeline.h"
-#include "ia_css_isp_configs.h"
-#include "ia_css_debug.h"
-#include "assert_support.h"
-
-/* Code generated by genparam/genconfig.c:gen_configure_function() */
-
-void
-ia_css_configure_iterator(
- const struct ia_css_binary *binary,
- const struct ia_css_iterator_configuration *config_dmem)
-{
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_iterator() enter:\n");
-
- {
- unsigned int offset = 0;
- unsigned int size = 0;
-
- if (binary->info->mem_offsets.offsets.config) {
- size = binary->info->mem_offsets.offsets.config->dmem.iterator.size;
- offset = binary->info->mem_offsets.offsets.config->dmem.iterator.offset;
- }
- if (size) {
- ia_css_iterator_config((struct sh_css_isp_iterator_isp_config *)
- &binary->mem_params.params[IA_CSS_PARAM_CLASS_CONFIG][IA_CSS_ISP_DMEM].address[offset],
- config_dmem, size);
- }
- }
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_iterator() leave:\n");
-}
-
-/* Code generated by genparam/genconfig.c:gen_configure_function() */
-
-void
-ia_css_configure_copy_output(
- const struct ia_css_binary *binary,
- const struct ia_css_copy_output_configuration *config_dmem)
-{
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_copy_output() enter:\n");
-
- {
- unsigned int offset = 0;
- unsigned int size = 0;
-
- if (binary->info->mem_offsets.offsets.config) {
- size = binary->info->mem_offsets.offsets.config->dmem.copy_output.size;
- offset = binary->info->mem_offsets.offsets.config->dmem.copy_output.offset;
- }
- if (size) {
- ia_css_copy_output_config((struct sh_css_isp_copy_output_isp_config *)
- &binary->mem_params.params[IA_CSS_PARAM_CLASS_CONFIG][IA_CSS_ISP_DMEM].address[offset],
- config_dmem, size);
- }
- }
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_copy_output() leave:\n");
-}
-
-/* Code generated by genparam/genconfig.c:gen_configure_function() */
-
-void
-ia_css_configure_crop(
- const struct ia_css_binary *binary,
- const struct ia_css_crop_configuration *config_dmem)
-{
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_crop() enter:\n");
-
- {
- unsigned int offset = 0;
- unsigned int size = 0;
-
- if (binary->info->mem_offsets.offsets.config) {
- size = binary->info->mem_offsets.offsets.config->dmem.crop.size;
- offset = binary->info->mem_offsets.offsets.config->dmem.crop.offset;
- }
- if (size) {
- ia_css_crop_config((struct sh_css_isp_crop_isp_config *)
- &binary->mem_params.params[IA_CSS_PARAM_CLASS_CONFIG][IA_CSS_ISP_DMEM].address[offset],
- config_dmem, size);
- }
- }
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_crop() leave:\n");
-}
-
-/* Code generated by genparam/genconfig.c:gen_configure_function() */
-
-void
-ia_css_configure_fpn(
- const struct ia_css_binary *binary,
- const struct ia_css_fpn_configuration *config_dmem)
-{
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_fpn() enter:\n");
-
- {
- unsigned int offset = 0;
- unsigned int size = 0;
-
- if (binary->info->mem_offsets.offsets.config) {
- size = binary->info->mem_offsets.offsets.config->dmem.fpn.size;
- offset = binary->info->mem_offsets.offsets.config->dmem.fpn.offset;
- }
- if (size) {
- ia_css_fpn_config((struct sh_css_isp_fpn_isp_config *)
- &binary->mem_params.params[IA_CSS_PARAM_CLASS_CONFIG][IA_CSS_ISP_DMEM].address[offset],
- config_dmem, size);
- }
- }
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_fpn() leave:\n");
-}
-
-/* Code generated by genparam/genconfig.c:gen_configure_function() */
-
-void
-ia_css_configure_dvs(
- const struct ia_css_binary *binary,
- const struct ia_css_dvs_configuration *config_dmem)
-{
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_dvs() enter:\n");
-
- {
- unsigned int offset = 0;
- unsigned int size = 0;
-
- if (binary->info->mem_offsets.offsets.config) {
- size = binary->info->mem_offsets.offsets.config->dmem.dvs.size;
- offset = binary->info->mem_offsets.offsets.config->dmem.dvs.offset;
- }
- if (size) {
- ia_css_dvs_config((struct sh_css_isp_dvs_isp_config *)
- &binary->mem_params.params[IA_CSS_PARAM_CLASS_CONFIG][IA_CSS_ISP_DMEM].address[offset],
- config_dmem, size);
- }
- }
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_dvs() leave:\n");
-}
-
-/* Code generated by genparam/genconfig.c:gen_configure_function() */
-
-void
-ia_css_configure_qplane(
- const struct ia_css_binary *binary,
- const struct ia_css_qplane_configuration *config_dmem)
-{
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_qplane() enter:\n");
-
- {
- unsigned int offset = 0;
- unsigned int size = 0;
-
- if (binary->info->mem_offsets.offsets.config) {
- size = binary->info->mem_offsets.offsets.config->dmem.qplane.size;
- offset = binary->info->mem_offsets.offsets.config->dmem.qplane.offset;
- }
- if (size) {
- ia_css_qplane_config((struct sh_css_isp_qplane_isp_config *)
- &binary->mem_params.params[IA_CSS_PARAM_CLASS_CONFIG][IA_CSS_ISP_DMEM].address[offset],
- config_dmem, size);
- }
- }
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_qplane() leave:\n");
-}
-
-/* Code generated by genparam/genconfig.c:gen_configure_function() */
-
-void
-ia_css_configure_output0(
- const struct ia_css_binary *binary,
- const struct ia_css_output0_configuration *config_dmem)
-{
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_output0() enter:\n");
-
- {
- unsigned int offset = 0;
- unsigned int size = 0;
-
- if (binary->info->mem_offsets.offsets.config) {
- size = binary->info->mem_offsets.offsets.config->dmem.output0.size;
- offset = binary->info->mem_offsets.offsets.config->dmem.output0.offset;
- }
- if (size) {
- ia_css_output0_config((struct sh_css_isp_output_isp_config *)
- &binary->mem_params.params[IA_CSS_PARAM_CLASS_CONFIG][IA_CSS_ISP_DMEM].address[offset],
- config_dmem, size);
- }
- }
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_output0() leave:\n");
-}
-
-/* Code generated by genparam/genconfig.c:gen_configure_function() */
-
-void
-ia_css_configure_output1(
- const struct ia_css_binary *binary,
- const struct ia_css_output1_configuration *config_dmem)
-{
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_output1() enter:\n");
-
- {
- unsigned int offset = 0;
- unsigned int size = 0;
-
- if (binary->info->mem_offsets.offsets.config) {
- size = binary->info->mem_offsets.offsets.config->dmem.output1.size;
- offset = binary->info->mem_offsets.offsets.config->dmem.output1.offset;
- }
- if (size) {
- ia_css_output1_config((struct sh_css_isp_output_isp_config *)
- &binary->mem_params.params[IA_CSS_PARAM_CLASS_CONFIG][IA_CSS_ISP_DMEM].address[offset],
- config_dmem, size);
- }
- }
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_output1() leave:\n");
-}
-
-/* Code generated by genparam/genconfig.c:gen_configure_function() */
-
-void
-ia_css_configure_output(
- const struct ia_css_binary *binary,
- const struct ia_css_output_configuration *config_dmem)
-{
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_output() enter:\n");
-
- {
- unsigned int offset = 0;
- unsigned int size = 0;
-
- if (binary->info->mem_offsets.offsets.config) {
- size = binary->info->mem_offsets.offsets.config->dmem.output.size;
- offset = binary->info->mem_offsets.offsets.config->dmem.output.offset;
- }
- if (size) {
- ia_css_output_config((struct sh_css_isp_output_isp_config *)
- &binary->mem_params.params[IA_CSS_PARAM_CLASS_CONFIG][IA_CSS_ISP_DMEM].address[offset],
- config_dmem, size);
- }
- }
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_output() leave:\n");
-}
-
-/* Code generated by genparam/genconfig.c:gen_configure_function() */
-
-void
-ia_css_configure_raw(
- const struct ia_css_binary *binary,
- const struct ia_css_raw_configuration *config_dmem)
-{
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_raw() enter:\n");
-
- {
- unsigned int offset = 0;
- unsigned int size = 0;
-
- if (binary->info->mem_offsets.offsets.config) {
- size = binary->info->mem_offsets.offsets.config->dmem.raw.size;
- offset = binary->info->mem_offsets.offsets.config->dmem.raw.offset;
- }
- if (size) {
- ia_css_raw_config((struct sh_css_isp_raw_isp_config *)
- &binary->mem_params.params[IA_CSS_PARAM_CLASS_CONFIG][IA_CSS_ISP_DMEM].address[offset],
- config_dmem, size);
- }
- }
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_raw() leave:\n");
-}
-
-/* Code generated by genparam/genconfig.c:gen_configure_function() */
-
-void
-ia_css_configure_tnr(
- const struct ia_css_binary *binary,
- const struct ia_css_tnr_configuration *config_dmem)
-{
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_tnr() enter:\n");
-
- {
- unsigned int offset = 0;
- unsigned int size = 0;
-
- if (binary->info->mem_offsets.offsets.config) {
- size = binary->info->mem_offsets.offsets.config->dmem.tnr.size;
- offset = binary->info->mem_offsets.offsets.config->dmem.tnr.offset;
- }
- if (size) {
- ia_css_tnr_config((struct sh_css_isp_tnr_isp_config *)
- &binary->mem_params.params[IA_CSS_PARAM_CLASS_CONFIG][IA_CSS_ISP_DMEM].address[offset],
- config_dmem, size);
- }
- }
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_tnr() leave:\n");
-}
-
-/* Code generated by genparam/genconfig.c:gen_configure_function() */
-
-void
-ia_css_configure_ref(
- const struct ia_css_binary *binary,
- const struct ia_css_ref_configuration *config_dmem)
-{
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_ref() enter:\n");
-
- {
- unsigned int offset = 0;
- unsigned int size = 0;
-
- if (binary->info->mem_offsets.offsets.config) {
- size = binary->info->mem_offsets.offsets.config->dmem.ref.size;
- offset = binary->info->mem_offsets.offsets.config->dmem.ref.offset;
- }
- if (size) {
- ia_css_ref_config((struct sh_css_isp_ref_isp_config *)
- &binary->mem_params.params[IA_CSS_PARAM_CLASS_CONFIG][IA_CSS_ISP_DMEM].address[offset],
- config_dmem, size);
- }
- }
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_ref() leave:\n");
-}
-
-/* Code generated by genparam/genconfig.c:gen_configure_function() */
-
-void
-ia_css_configure_vf(
- const struct ia_css_binary *binary,
- const struct ia_css_vf_configuration *config_dmem)
-{
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_vf() enter:\n");
-
- {
- unsigned int offset = 0;
- unsigned int size = 0;
-
- if (binary->info->mem_offsets.offsets.config) {
- size = binary->info->mem_offsets.offsets.config->dmem.vf.size;
- offset = binary->info->mem_offsets.offsets.config->dmem.vf.offset;
- }
- if (size) {
- ia_css_vf_config((struct sh_css_isp_vf_isp_config *)
- &binary->mem_params.params[IA_CSS_PARAM_CLASS_CONFIG][IA_CSS_ISP_DMEM].address[offset],
- config_dmem, size);
- }
- }
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_vf() leave:\n");
-}
diff --git a/drivers/staging/media/atomisp/pci/css_2401_system/hive/ia_css_isp_params.c b/drivers/staging/media/atomisp/pci/css_2401_system/hive/ia_css_isp_params.c
deleted file mode 100644
index d9c672d..0000000
--- a/drivers/staging/media/atomisp/pci/css_2401_system/hive/ia_css_isp_params.c
+++ /dev/null
@@ -1,3367 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/*
- * Support for Intel Camera Imaging ISP subsystem.
- * Copyright (c) 2015, Intel Corporation.
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms and conditions of the GNU General Public License,
- * version 2, as published by the Free Software Foundation.
- *
- * This program is distributed in the hope it will be useful, but WITHOUT
- * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
- * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
- * more details.
- */
-
-#define IA_CSS_INCLUDE_PARAMETERS
-#include "sh_css_params.h"
-#include "isp/kernels/aa/aa_2/ia_css_aa2.host.h"
-#include "isp/kernels/anr/anr_1.0/ia_css_anr.host.h"
-#include "isp/kernels/anr/anr_2/ia_css_anr2.host.h"
-#include "isp/kernels/bh/bh_2/ia_css_bh.host.h"
-#include "isp/kernels/bnr/bnr_1.0/ia_css_bnr.host.h"
-#include "isp/kernels/bnr/bnr2_2/ia_css_bnr2_2.host.h"
-#include "isp/kernels/cnr/cnr_2/ia_css_cnr2.host.h"
-#include "isp/kernels/crop/crop_1.0/ia_css_crop.host.h"
-#include "isp/kernels/csc/csc_1.0/ia_css_csc.host.h"
-#include "isp/kernels/ctc/ctc_1.0/ia_css_ctc.host.h"
-#include "isp/kernels/ctc/ctc1_5/ia_css_ctc1_5.host.h"
-#include "isp/kernels/ctc/ctc2/ia_css_ctc2.host.h"
-#include "isp/kernels/de/de_1.0/ia_css_de.host.h"
-#include "isp/kernels/de/de_2/ia_css_de2.host.h"
-#include "isp/kernels/dp/dp_1.0/ia_css_dp.host.h"
-#include "isp/kernels/fixedbds/fixedbds_1.0/ia_css_fixedbds_param.h"
-#include "isp/kernels/fpn/fpn_1.0/ia_css_fpn.host.h"
-#include "isp/kernels/gc/gc_1.0/ia_css_gc.host.h"
-#include "isp/kernels/gc/gc_2/ia_css_gc2.host.h"
-#include "isp/kernels/macc/macc_1.0/ia_css_macc.host.h"
-#include "isp/kernels/macc/macc1_5/ia_css_macc1_5.host.h"
-#include "isp/kernels/ob/ob_1.0/ia_css_ob.host.h"
-#include "isp/kernels/ob/ob2/ia_css_ob2.host.h"
-#include "isp/kernels/output/output_1.0/ia_css_output.host.h"
-#include "isp/kernels/raw_aa_binning/raw_aa_binning_1.0/ia_css_raa.host.h"
-#include "isp/kernels/s3a/s3a_1.0/ia_css_s3a.host.h"
-#include "isp/kernels/sc/sc_1.0/ia_css_sc.host.h"
-#include "isp/kernels/sdis/sdis_1.0/ia_css_sdis.host.h"
-#include "isp/kernels/sdis/sdis_2/ia_css_sdis2.host.h"
-#include "isp/kernels/tnr/tnr_1.0/ia_css_tnr.host.h"
-#include "isp/kernels/uds/uds_1.0/ia_css_uds_param.h"
-#include "isp/kernels/wb/wb_1.0/ia_css_wb.host.h"
-#include "isp/kernels/xnr/xnr_1.0/ia_css_xnr.host.h"
-#include "isp/kernels/xnr/xnr_3.0/ia_css_xnr3.host.h"
-#include "isp/kernels/ynr/ynr_1.0/ia_css_ynr.host.h"
-#include "isp/kernels/ynr/ynr_2/ia_css_ynr2.host.h"
-#include "isp/kernels/fc/fc_1.0/ia_css_formats.host.h"
-#include "isp/kernels/tdf/tdf_1.0/ia_css_tdf.host.h"
-#include "isp/kernels/dpc2/ia_css_dpc2.host.h"
-#include "isp/kernels/eed1_8/ia_css_eed1_8.host.h"
-#include "isp/kernels/bnlm/ia_css_bnlm.host.h"
-#include "isp/kernels/conversion/conversion_1.0/ia_css_conversion.host.h"
-/* Generated code: do not edit or commmit. */
-
-#include "ia_css_pipeline.h"
-#include "ia_css_isp_params.h"
-#include "ia_css_debug.h"
-#include "assert_support.h"
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_aa(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.aa.size;
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.aa.offset;
-
- if (size) {
- struct sh_css_isp_aa_params *t = (struct sh_css_isp_aa_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset];
- t->strength = params->aa_config.strength;
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_anr(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.anr.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.anr.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_anr() enter:\n");
-
- ia_css_anr_encode((struct sh_css_isp_anr_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->anr_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_anr() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_anr2(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->vmem.anr2.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->vmem.anr2.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_anr2() enter:\n");
-
- ia_css_anr2_vmem_encode((struct ia_css_isp_anr2_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_VMEM].address[offset],
- ¶ms->anr_thres,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_VMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_anr2() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_bh(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.bh.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.bh.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_bh() enter:\n");
-
- ia_css_bh_encode((struct sh_css_isp_bh_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->s3a_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_bh() leave:\n");
- }
- }
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->hmem0.bh.size;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_bh() enter:\n");
-
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_HMEM0] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_bh() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_cnr(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.cnr.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.cnr.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_cnr() enter:\n");
-
- ia_css_cnr_encode((struct sh_css_isp_cnr_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->cnr_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_cnr() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_crop(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.crop.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.crop.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_crop() enter:\n");
-
- ia_css_crop_encode((struct sh_css_isp_crop_isp_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->crop_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_crop() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_csc(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.csc.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.csc.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_csc() enter:\n");
-
- ia_css_csc_encode((struct sh_css_isp_csc_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->cc_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_csc() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_dp(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.dp.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.dp.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_dp() enter:\n");
-
- ia_css_dp_encode((struct sh_css_isp_dp_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->dp_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_dp() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_bnr(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.bnr.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.bnr.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_bnr() enter:\n");
-
- ia_css_bnr_encode((struct sh_css_isp_bnr_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->nr_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_bnr() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_de(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.de.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.de.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_de() enter:\n");
-
- ia_css_de_encode((struct sh_css_isp_de_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->de_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_de() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_ecd(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.ecd.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.ecd.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_ecd() enter:\n");
-
- ia_css_ecd_encode((struct sh_css_isp_ecd_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->ecd_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_ecd() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_formats(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.formats.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.formats.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_formats() enter:\n");
-
- ia_css_formats_encode((struct sh_css_isp_formats_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->formats_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_formats() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_fpn(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.fpn.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.fpn.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_fpn() enter:\n");
-
- ia_css_fpn_encode((struct sh_css_isp_fpn_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->fpn_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_fpn() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_gc(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.gc.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.gc.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_gc() enter:\n");
-
- ia_css_gc_encode((struct sh_css_isp_gc_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->gc_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_gc() leave:\n");
- }
- }
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->vamem1.gc.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->vamem1.gc.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_gc() enter:\n");
-
- ia_css_gc_vamem_encode((struct sh_css_isp_gc_vamem_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_VAMEM1].address[offset],
- ¶ms->gc_table,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_VAMEM1] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_gc() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_ce(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.ce.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.ce.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_ce() enter:\n");
-
- ia_css_ce_encode((struct sh_css_isp_ce_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->ce_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_ce() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_yuv2rgb(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.yuv2rgb.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.yuv2rgb.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_yuv2rgb() enter:\n");
-
- ia_css_yuv2rgb_encode((struct sh_css_isp_csc_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->yuv2rgb_cc_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_yuv2rgb() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_rgb2yuv(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.rgb2yuv.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.rgb2yuv.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_rgb2yuv() enter:\n");
-
- ia_css_rgb2yuv_encode((struct sh_css_isp_csc_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->rgb2yuv_cc_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_rgb2yuv() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_r_gamma(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->vamem0.r_gamma.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->vamem0.r_gamma.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_r_gamma() enter:\n");
-
- ia_css_r_gamma_vamem_encode((struct sh_css_isp_rgb_gamma_vamem_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_VAMEM0].address[offset],
- ¶ms->r_gamma_table,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_VAMEM0] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_r_gamma() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_g_gamma(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->vamem1.g_gamma.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->vamem1.g_gamma.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_g_gamma() enter:\n");
-
- ia_css_g_gamma_vamem_encode((struct sh_css_isp_rgb_gamma_vamem_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_VAMEM1].address[offset],
- ¶ms->g_gamma_table,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_VAMEM1] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_g_gamma() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_b_gamma(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->vamem2.b_gamma.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->vamem2.b_gamma.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_b_gamma() enter:\n");
-
- ia_css_b_gamma_vamem_encode((struct sh_css_isp_rgb_gamma_vamem_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_VAMEM2].address[offset],
- ¶ms->b_gamma_table,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_VAMEM2] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_b_gamma() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_uds(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.uds.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.uds.offset;
-
- if (size) {
- struct sh_css_sp_uds_params *p;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_uds() enter:\n");
-
- p = (struct sh_css_sp_uds_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset];
- p->crop_pos = params->uds_config.crop_pos;
- p->uds = params->uds_config.uds;
-
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_uds() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_raa(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.raa.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.raa.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_raa() enter:\n");
-
- ia_css_raa_encode((struct sh_css_isp_aa_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->raa_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_raa() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_s3a(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.s3a.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.s3a.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_s3a() enter:\n");
-
- ia_css_s3a_encode((struct sh_css_isp_s3a_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->s3a_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_s3a() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_ob(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.ob.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.ob.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_ob() enter:\n");
-
- ia_css_ob_encode((struct sh_css_isp_ob_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->ob_config,
- ¶ms->stream_configs.ob, size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_ob() leave:\n");
- }
- }
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->vmem.ob.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->vmem.ob.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_ob() enter:\n");
-
- ia_css_ob_vmem_encode((struct sh_css_isp_ob_vmem_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_VMEM].address[offset],
- ¶ms->ob_config,
- ¶ms->stream_configs.ob, size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_VMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_ob() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_output(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.output.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.output.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_output() enter:\n");
-
- ia_css_output_encode((struct sh_css_isp_output_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->output_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_output() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_sc(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.sc.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.sc.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_sc() enter:\n");
-
- ia_css_sc_encode((struct sh_css_isp_sc_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->sc_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_sc() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_bds(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.bds.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.bds.offset;
-
- if (size) {
- struct sh_css_isp_bds_params *p;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_bds() enter:\n");
-
- p = (struct sh_css_isp_bds_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset];
- p->baf_strength = params->bds_config.strength;
-
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_bds() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_tnr(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.tnr.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.tnr.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_tnr() enter:\n");
-
- ia_css_tnr_encode((struct sh_css_isp_tnr_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->tnr_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_tnr() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_macc(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.macc.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.macc.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_macc() enter:\n");
-
- ia_css_macc_encode((struct sh_css_isp_macc_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->macc_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_macc() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_sdis_horicoef(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->vmem.sdis_horicoef.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->vmem.sdis_horicoef.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_sdis_horicoef() enter:\n");
-
- ia_css_sdis_horicoef_vmem_encode((struct sh_css_isp_sdis_hori_coef_tbl *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_VMEM].address[offset],
- ¶ms->dvs_coefs,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_VMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_sdis_horicoef() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_sdis_vertcoef(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->vmem.sdis_vertcoef.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->vmem.sdis_vertcoef.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_sdis_vertcoef() enter:\n");
-
- ia_css_sdis_vertcoef_vmem_encode((struct sh_css_isp_sdis_vert_coef_tbl *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_VMEM].address[offset],
- ¶ms->dvs_coefs,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_VMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_sdis_vertcoef() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_sdis_horiproj(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.sdis_horiproj.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.sdis_horiproj.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_sdis_horiproj() enter:\n");
-
- ia_css_sdis_horiproj_encode((struct sh_css_isp_sdis_hori_proj_tbl *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->dvs_coefs,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_sdis_horiproj() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_sdis_vertproj(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.sdis_vertproj.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.sdis_vertproj.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_sdis_vertproj() enter:\n");
-
- ia_css_sdis_vertproj_encode((struct sh_css_isp_sdis_vert_proj_tbl *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->dvs_coefs,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_sdis_vertproj() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_sdis2_horicoef(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->vmem.sdis2_horicoef.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->vmem.sdis2_horicoef.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_sdis2_horicoef() enter:\n");
-
- ia_css_sdis2_horicoef_vmem_encode((struct sh_css_isp_sdis_hori_coef_tbl *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_VMEM].address[offset],
- ¶ms->dvs2_coefs,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_VMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_sdis2_horicoef() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_sdis2_vertcoef(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->vmem.sdis2_vertcoef.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->vmem.sdis2_vertcoef.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_sdis2_vertcoef() enter:\n");
-
- ia_css_sdis2_vertcoef_vmem_encode((struct sh_css_isp_sdis_vert_coef_tbl *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_VMEM].address[offset],
- ¶ms->dvs2_coefs,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_VMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_sdis2_vertcoef() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_sdis2_horiproj(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.sdis2_horiproj.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.sdis2_horiproj.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_sdis2_horiproj() enter:\n");
-
- ia_css_sdis2_horiproj_encode((struct sh_css_isp_sdis_hori_proj_tbl *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->dvs2_coefs,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_sdis2_horiproj() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_sdis2_vertproj(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.sdis2_vertproj.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.sdis2_vertproj.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_sdis2_vertproj() enter:\n");
-
- ia_css_sdis2_vertproj_encode((struct sh_css_isp_sdis_vert_proj_tbl *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->dvs2_coefs,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_sdis2_vertproj() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_wb(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.wb.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.wb.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_wb() enter:\n");
-
- ia_css_wb_encode((struct sh_css_isp_wb_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->wb_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_wb() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_nr(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.nr.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.nr.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_nr() enter:\n");
-
- ia_css_nr_encode((struct sh_css_isp_ynr_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->nr_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_nr() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_yee(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.yee.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.yee.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_yee() enter:\n");
-
- ia_css_yee_encode((struct sh_css_isp_yee_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->yee_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_yee() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_ynr(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.ynr.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.ynr.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_ynr() enter:\n");
-
- ia_css_ynr_encode((struct sh_css_isp_yee2_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->ynr_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_ynr() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_fc(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.fc.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.fc.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_fc() enter:\n");
-
- ia_css_fc_encode((struct sh_css_isp_fc_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->fc_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_fc() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_ctc(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.ctc.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.ctc.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_ctc() enter:\n");
-
- ia_css_ctc_encode((struct sh_css_isp_ctc_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->ctc_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_ctc() leave:\n");
- }
- }
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->vamem0.ctc.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->vamem0.ctc.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_ctc() enter:\n");
-
- ia_css_ctc_vamem_encode((struct sh_css_isp_ctc_vamem_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_VAMEM0].address[offset],
- ¶ms->ctc_table,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_VAMEM0] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_ctc() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_xnr_table(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->vamem1.xnr_table.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->vamem1.xnr_table.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_xnr_table() enter:\n");
-
- ia_css_xnr_table_vamem_encode((struct sh_css_isp_xnr_vamem_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_VAMEM1].address[offset],
- ¶ms->xnr_table,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_VAMEM1] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_xnr_table() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_xnr(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.xnr.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.xnr.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_xnr() enter:\n");
-
- ia_css_xnr_encode((struct sh_css_isp_xnr_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->xnr_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_xnr() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_process_function() */
-
-static void
-ia_css_process_xnr3(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params)
-{
- assert(params);
-
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.xnr3.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.xnr3.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_xnr3() enter:\n");
-
- ia_css_xnr3_encode((struct sh_css_isp_xnr3_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->xnr3_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_xnr3() leave:\n");
- }
- }
- {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->vmem.xnr3.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->vmem.xnr3.offset;
-
- if (size) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_xnr3() enter:\n");
-
- ia_css_xnr3_vmem_encode((struct sh_css_isp_xnr3_vmem_params *)
- &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_VMEM].address[offset],
- ¶ms->xnr3_config,
- size);
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_VMEM] =
- true;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_process_xnr3() leave:\n");
- }
- }
-}
-
-/* Code generated by genparam/gencode.c:gen_param_process_table() */
-
-void (*ia_css_kernel_process_param[IA_CSS_NUM_PARAMETER_IDS])(
- unsigned int pipe_id,
- const struct ia_css_pipeline_stage *stage,
- struct ia_css_isp_parameters *params) = {
- ia_css_process_aa,
- ia_css_process_anr,
- ia_css_process_anr2,
- ia_css_process_bh,
- ia_css_process_cnr,
- ia_css_process_crop,
- ia_css_process_csc,
- ia_css_process_dp,
- ia_css_process_bnr,
- ia_css_process_de,
- ia_css_process_ecd,
- ia_css_process_formats,
- ia_css_process_fpn,
- ia_css_process_gc,
- ia_css_process_ce,
- ia_css_process_yuv2rgb,
- ia_css_process_rgb2yuv,
- ia_css_process_r_gamma,
- ia_css_process_g_gamma,
- ia_css_process_b_gamma,
- ia_css_process_uds,
- ia_css_process_raa,
- ia_css_process_s3a,
- ia_css_process_ob,
- ia_css_process_output,
- ia_css_process_sc,
- ia_css_process_bds,
- ia_css_process_tnr,
- ia_css_process_macc,
- ia_css_process_sdis_horicoef,
- ia_css_process_sdis_vertcoef,
- ia_css_process_sdis_horiproj,
- ia_css_process_sdis_vertproj,
- ia_css_process_sdis2_horicoef,
- ia_css_process_sdis2_vertcoef,
- ia_css_process_sdis2_horiproj,
- ia_css_process_sdis2_vertproj,
- ia_css_process_wb,
- ia_css_process_nr,
- ia_css_process_yee,
- ia_css_process_ynr,
- ia_css_process_fc,
- ia_css_process_ctc,
- ia_css_process_xnr_table,
- ia_css_process_xnr,
- ia_css_process_xnr3,
-};
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_dp_config(const struct ia_css_isp_parameters *params,
- struct ia_css_dp_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_dp_config() enter: config=%p\n",
- config);
-
- *config = params->dp_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_dp_config() leave\n");
- ia_css_dp_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_dp_config(struct ia_css_isp_parameters *params,
- const struct ia_css_dp_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_dp_config() enter:\n");
- ia_css_dp_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->dp_config = *config;
- params->config_changed[IA_CSS_DP_ID] = true;
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_dp_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_wb_config(const struct ia_css_isp_parameters *params,
- struct ia_css_wb_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_wb_config() enter: config=%p\n",
- config);
-
- *config = params->wb_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_wb_config() leave\n");
- ia_css_wb_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_wb_config(struct ia_css_isp_parameters *params,
- const struct ia_css_wb_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_wb_config() enter:\n");
- ia_css_wb_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->wb_config = *config;
- params->config_changed[IA_CSS_WB_ID] = true;
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_wb_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_tnr_config(const struct ia_css_isp_parameters *params,
- struct ia_css_tnr_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_tnr_config() enter: config=%p\n",
- config);
-
- *config = params->tnr_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_tnr_config() leave\n");
- ia_css_tnr_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_tnr_config(struct ia_css_isp_parameters *params,
- const struct ia_css_tnr_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_tnr_config() enter:\n");
- ia_css_tnr_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->tnr_config = *config;
- params->config_changed[IA_CSS_TNR_ID] = true;
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_tnr_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_ob_config(const struct ia_css_isp_parameters *params,
- struct ia_css_ob_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_ob_config() enter: config=%p\n",
- config);
-
- *config = params->ob_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_ob_config() leave\n");
- ia_css_ob_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_ob_config(struct ia_css_isp_parameters *params,
- const struct ia_css_ob_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_ob_config() enter:\n");
- ia_css_ob_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->ob_config = *config;
- params->config_changed[IA_CSS_OB_ID] = true;
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_ob_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_de_config(const struct ia_css_isp_parameters *params,
- struct ia_css_de_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_de_config() enter: config=%p\n",
- config);
-
- *config = params->de_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_de_config() leave\n");
- ia_css_de_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_de_config(struct ia_css_isp_parameters *params,
- const struct ia_css_de_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_de_config() enter:\n");
- ia_css_de_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->de_config = *config;
- params->config_changed[IA_CSS_DE_ID] = true;
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_de_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_anr_config(const struct ia_css_isp_parameters *params,
- struct ia_css_anr_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_anr_config() enter: config=%p\n",
- config);
-
- *config = params->anr_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_anr_config() leave\n");
- ia_css_anr_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_anr_config(struct ia_css_isp_parameters *params,
- const struct ia_css_anr_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_anr_config() enter:\n");
- ia_css_anr_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->anr_config = *config;
- params->config_changed[IA_CSS_ANR_ID] = true;
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_anr_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_anr2_config(const struct ia_css_isp_parameters *params,
- struct ia_css_anr_thres *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_anr2_config() enter: config=%p\n",
- config);
-
- *config = params->anr_thres;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_anr2_config() leave\n");
- ia_css_anr2_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_anr2_config(struct ia_css_isp_parameters *params,
- const struct ia_css_anr_thres *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_anr2_config() enter:\n");
- ia_css_anr2_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->anr_thres = *config;
- params->config_changed[IA_CSS_ANR2_ID] = true;
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_anr2_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_ce_config(const struct ia_css_isp_parameters *params,
- struct ia_css_ce_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_ce_config() enter: config=%p\n",
- config);
-
- *config = params->ce_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_ce_config() leave\n");
- ia_css_ce_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_ce_config(struct ia_css_isp_parameters *params,
- const struct ia_css_ce_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_ce_config() enter:\n");
- ia_css_ce_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->ce_config = *config;
- params->config_changed[IA_CSS_CE_ID] = true;
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_ce_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_ecd_config(const struct ia_css_isp_parameters *params,
- struct ia_css_ecd_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_ecd_config() enter: config=%p\n",
- config);
-
- *config = params->ecd_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_ecd_config() leave\n");
- ia_css_ecd_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_ecd_config(struct ia_css_isp_parameters *params,
- const struct ia_css_ecd_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_ecd_config() enter:\n");
- ia_css_ecd_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->ecd_config = *config;
- params->config_changed[IA_CSS_ECD_ID] = true;
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_ecd_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_ynr_config(const struct ia_css_isp_parameters *params,
- struct ia_css_ynr_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_ynr_config() enter: config=%p\n",
- config);
-
- *config = params->ynr_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_ynr_config() leave\n");
- ia_css_ynr_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_ynr_config(struct ia_css_isp_parameters *params,
- const struct ia_css_ynr_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_ynr_config() enter:\n");
- ia_css_ynr_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->ynr_config = *config;
- params->config_changed[IA_CSS_YNR_ID] = true;
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_ynr_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_fc_config(const struct ia_css_isp_parameters *params,
- struct ia_css_fc_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_fc_config() enter: config=%p\n",
- config);
-
- *config = params->fc_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_fc_config() leave\n");
- ia_css_fc_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_fc_config(struct ia_css_isp_parameters *params,
- const struct ia_css_fc_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_fc_config() enter:\n");
- ia_css_fc_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->fc_config = *config;
- params->config_changed[IA_CSS_FC_ID] = true;
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_fc_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_cnr_config(const struct ia_css_isp_parameters *params,
- struct ia_css_cnr_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_cnr_config() enter: config=%p\n",
- config);
-
- *config = params->cnr_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_cnr_config() leave\n");
- ia_css_cnr_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_cnr_config(struct ia_css_isp_parameters *params,
- const struct ia_css_cnr_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_cnr_config() enter:\n");
- ia_css_cnr_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->cnr_config = *config;
- params->config_changed[IA_CSS_CNR_ID] = true;
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_cnr_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_macc_config(const struct ia_css_isp_parameters *params,
- struct ia_css_macc_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_macc_config() enter: config=%p\n",
- config);
-
- *config = params->macc_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_macc_config() leave\n");
- ia_css_macc_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_macc_config(struct ia_css_isp_parameters *params,
- const struct ia_css_macc_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_macc_config() enter:\n");
- ia_css_macc_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->macc_config = *config;
- params->config_changed[IA_CSS_MACC_ID] = true;
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_macc_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_ctc_config(const struct ia_css_isp_parameters *params,
- struct ia_css_ctc_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_ctc_config() enter: config=%p\n",
- config);
-
- *config = params->ctc_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_ctc_config() leave\n");
- ia_css_ctc_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_ctc_config(struct ia_css_isp_parameters *params,
- const struct ia_css_ctc_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_ctc_config() enter:\n");
- ia_css_ctc_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->ctc_config = *config;
- params->config_changed[IA_CSS_CTC_ID] = true;
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_ctc_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_aa_config(const struct ia_css_isp_parameters *params,
- struct ia_css_aa_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_aa_config() enter: config=%p\n",
- config);
-
- *config = params->aa_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_aa_config() leave\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_aa_config(struct ia_css_isp_parameters *params,
- const struct ia_css_aa_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_aa_config() enter:\n");
- params->aa_config = *config;
- params->config_changed[IA_CSS_AA_ID] = true;
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_aa_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_yuv2rgb_config(const struct ia_css_isp_parameters *params,
- struct ia_css_cc_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_yuv2rgb_config() enter: config=%p\n",
- config);
-
- *config = params->yuv2rgb_cc_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_yuv2rgb_config() leave\n");
- ia_css_yuv2rgb_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_yuv2rgb_config(struct ia_css_isp_parameters *params,
- const struct ia_css_cc_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_yuv2rgb_config() enter:\n");
- ia_css_yuv2rgb_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->yuv2rgb_cc_config = *config;
- params->config_changed[IA_CSS_YUV2RGB_ID] = true;
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_yuv2rgb_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_rgb2yuv_config(const struct ia_css_isp_parameters *params,
- struct ia_css_cc_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_rgb2yuv_config() enter: config=%p\n",
- config);
-
- *config = params->rgb2yuv_cc_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_rgb2yuv_config() leave\n");
- ia_css_rgb2yuv_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_rgb2yuv_config(struct ia_css_isp_parameters *params,
- const struct ia_css_cc_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_rgb2yuv_config() enter:\n");
- ia_css_rgb2yuv_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->rgb2yuv_cc_config = *config;
- params->config_changed[IA_CSS_RGB2YUV_ID] = true;
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_rgb2yuv_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_csc_config(const struct ia_css_isp_parameters *params,
- struct ia_css_cc_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_csc_config() enter: config=%p\n",
- config);
-
- *config = params->cc_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_csc_config() leave\n");
- ia_css_csc_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_csc_config(struct ia_css_isp_parameters *params,
- const struct ia_css_cc_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_csc_config() enter:\n");
- ia_css_csc_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->cc_config = *config;
- params->config_changed[IA_CSS_CSC_ID] = true;
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_csc_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_nr_config(const struct ia_css_isp_parameters *params,
- struct ia_css_nr_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_nr_config() enter: config=%p\n",
- config);
-
- *config = params->nr_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_nr_config() leave\n");
- ia_css_nr_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_nr_config(struct ia_css_isp_parameters *params,
- const struct ia_css_nr_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_nr_config() enter:\n");
- ia_css_nr_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->nr_config = *config;
- params->config_changed[IA_CSS_BNR_ID] = true;
- params->config_changed[IA_CSS_NR_ID] = true;
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_nr_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_gc_config(const struct ia_css_isp_parameters *params,
- struct ia_css_gc_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_gc_config() enter: config=%p\n",
- config);
-
- *config = params->gc_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_gc_config() leave\n");
- ia_css_gc_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_gc_config(struct ia_css_isp_parameters *params,
- const struct ia_css_gc_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_gc_config() enter:\n");
- ia_css_gc_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->gc_config = *config;
- params->config_changed[IA_CSS_GC_ID] = true;
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_gc_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_sdis_horicoef_config(const struct ia_css_isp_parameters *params,
- struct ia_css_dvs_coefficients *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_sdis_horicoef_config() enter: config=%p\n",
- config);
-
- *config = params->dvs_coefs;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_sdis_horicoef_config() leave\n");
- ia_css_sdis_horicoef_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_sdis_horicoef_config(struct ia_css_isp_parameters *params,
- const struct ia_css_dvs_coefficients *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE,
- "ia_css_set_sdis_horicoef_config() enter:\n");
- ia_css_sdis_horicoef_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->dvs_coefs = *config;
- params->config_changed[IA_CSS_SDIS_HORICOEF_ID] = true;
- params->config_changed[IA_CSS_SDIS_VERTCOEF_ID] = true;
- params->config_changed[IA_CSS_SDIS_HORIPROJ_ID] = true;
- params->config_changed[IA_CSS_SDIS_VERTPROJ_ID] = true;
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_sdis_horicoef_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_sdis_vertcoef_config(const struct ia_css_isp_parameters *params,
- struct ia_css_dvs_coefficients *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_sdis_vertcoef_config() enter: config=%p\n",
- config);
-
- *config = params->dvs_coefs;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_sdis_vertcoef_config() leave\n");
- ia_css_sdis_vertcoef_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_sdis_vertcoef_config(struct ia_css_isp_parameters *params,
- const struct ia_css_dvs_coefficients *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE,
- "ia_css_set_sdis_vertcoef_config() enter:\n");
- ia_css_sdis_vertcoef_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->dvs_coefs = *config;
- params->config_changed[IA_CSS_SDIS_HORICOEF_ID] = true;
- params->config_changed[IA_CSS_SDIS_VERTCOEF_ID] = true;
- params->config_changed[IA_CSS_SDIS_HORIPROJ_ID] = true;
- params->config_changed[IA_CSS_SDIS_VERTPROJ_ID] = true;
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_sdis_vertcoef_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_sdis_horiproj_config(const struct ia_css_isp_parameters *params,
- struct ia_css_dvs_coefficients *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_sdis_horiproj_config() enter: config=%p\n",
- config);
-
- *config = params->dvs_coefs;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_sdis_horiproj_config() leave\n");
- ia_css_sdis_horiproj_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_sdis_horiproj_config(struct ia_css_isp_parameters *params,
- const struct ia_css_dvs_coefficients *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE,
- "ia_css_set_sdis_horiproj_config() enter:\n");
- ia_css_sdis_horiproj_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->dvs_coefs = *config;
- params->config_changed[IA_CSS_SDIS_HORICOEF_ID] = true;
- params->config_changed[IA_CSS_SDIS_VERTCOEF_ID] = true;
- params->config_changed[IA_CSS_SDIS_HORIPROJ_ID] = true;
- params->config_changed[IA_CSS_SDIS_VERTPROJ_ID] = true;
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_sdis_horiproj_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_sdis_vertproj_config(const struct ia_css_isp_parameters *params,
- struct ia_css_dvs_coefficients *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_sdis_vertproj_config() enter: config=%p\n",
- config);
-
- *config = params->dvs_coefs;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_sdis_vertproj_config() leave\n");
- ia_css_sdis_vertproj_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_sdis_vertproj_config(struct ia_css_isp_parameters *params,
- const struct ia_css_dvs_coefficients *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE,
- "ia_css_set_sdis_vertproj_config() enter:\n");
- ia_css_sdis_vertproj_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->dvs_coefs = *config;
- params->config_changed[IA_CSS_SDIS_HORICOEF_ID] = true;
- params->config_changed[IA_CSS_SDIS_VERTCOEF_ID] = true;
- params->config_changed[IA_CSS_SDIS_HORIPROJ_ID] = true;
- params->config_changed[IA_CSS_SDIS_VERTPROJ_ID] = true;
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_sdis_vertproj_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_sdis2_horicoef_config(const struct ia_css_isp_parameters *params,
- struct ia_css_dvs2_coefficients *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_sdis2_horicoef_config() enter: config=%p\n",
- config);
-
- *config = params->dvs2_coefs;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_sdis2_horicoef_config() leave\n");
- ia_css_sdis2_horicoef_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_sdis2_horicoef_config(struct ia_css_isp_parameters *params,
- const struct ia_css_dvs2_coefficients *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE,
- "ia_css_set_sdis2_horicoef_config() enter:\n");
- ia_css_sdis2_horicoef_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->dvs2_coefs = *config;
- params->config_changed[IA_CSS_SDIS2_HORICOEF_ID] = true;
- params->config_changed[IA_CSS_SDIS2_VERTCOEF_ID] = true;
- params->config_changed[IA_CSS_SDIS2_HORIPROJ_ID] = true;
- params->config_changed[IA_CSS_SDIS2_VERTPROJ_ID] = true;
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_sdis2_horicoef_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_sdis2_vertcoef_config(const struct ia_css_isp_parameters *params,
- struct ia_css_dvs2_coefficients *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_sdis2_vertcoef_config() enter: config=%p\n",
- config);
-
- *config = params->dvs2_coefs;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_sdis2_vertcoef_config() leave\n");
- ia_css_sdis2_vertcoef_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_sdis2_vertcoef_config(struct ia_css_isp_parameters *params,
- const struct ia_css_dvs2_coefficients *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE,
- "ia_css_set_sdis2_vertcoef_config() enter:\n");
- ia_css_sdis2_vertcoef_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->dvs2_coefs = *config;
- params->config_changed[IA_CSS_SDIS2_HORICOEF_ID] = true;
- params->config_changed[IA_CSS_SDIS2_VERTCOEF_ID] = true;
- params->config_changed[IA_CSS_SDIS2_HORIPROJ_ID] = true;
- params->config_changed[IA_CSS_SDIS2_VERTPROJ_ID] = true;
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_sdis2_vertcoef_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_sdis2_horiproj_config(const struct ia_css_isp_parameters *params,
- struct ia_css_dvs2_coefficients *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_sdis2_horiproj_config() enter: config=%p\n",
- config);
-
- *config = params->dvs2_coefs;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_sdis2_horiproj_config() leave\n");
- ia_css_sdis2_horiproj_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_sdis2_horiproj_config(struct ia_css_isp_parameters *params,
- const struct ia_css_dvs2_coefficients *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE,
- "ia_css_set_sdis2_horiproj_config() enter:\n");
- ia_css_sdis2_horiproj_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->dvs2_coefs = *config;
- params->config_changed[IA_CSS_SDIS2_HORICOEF_ID] = true;
- params->config_changed[IA_CSS_SDIS2_VERTCOEF_ID] = true;
- params->config_changed[IA_CSS_SDIS2_HORIPROJ_ID] = true;
- params->config_changed[IA_CSS_SDIS2_VERTPROJ_ID] = true;
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_sdis2_horiproj_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_sdis2_vertproj_config(const struct ia_css_isp_parameters *params,
- struct ia_css_dvs2_coefficients *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_sdis2_vertproj_config() enter: config=%p\n",
- config);
-
- *config = params->dvs2_coefs;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_sdis2_vertproj_config() leave\n");
- ia_css_sdis2_vertproj_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_sdis2_vertproj_config(struct ia_css_isp_parameters *params,
- const struct ia_css_dvs2_coefficients *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE,
- "ia_css_set_sdis2_vertproj_config() enter:\n");
- ia_css_sdis2_vertproj_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->dvs2_coefs = *config;
- params->config_changed[IA_CSS_SDIS2_HORICOEF_ID] = true;
- params->config_changed[IA_CSS_SDIS2_VERTCOEF_ID] = true;
- params->config_changed[IA_CSS_SDIS2_HORIPROJ_ID] = true;
- params->config_changed[IA_CSS_SDIS2_VERTPROJ_ID] = true;
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_sdis2_vertproj_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_r_gamma_config(const struct ia_css_isp_parameters *params,
- struct ia_css_rgb_gamma_table *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_r_gamma_config() enter: config=%p\n",
- config);
-
- *config = params->r_gamma_table;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_r_gamma_config() leave\n");
- ia_css_r_gamma_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_r_gamma_config(struct ia_css_isp_parameters *params,
- const struct ia_css_rgb_gamma_table *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_r_gamma_config() enter:\n");
- ia_css_r_gamma_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->r_gamma_table = *config;
- params->config_changed[IA_CSS_R_GAMMA_ID] = true;
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_r_gamma_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_g_gamma_config(const struct ia_css_isp_parameters *params,
- struct ia_css_rgb_gamma_table *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_g_gamma_config() enter: config=%p\n",
- config);
-
- *config = params->g_gamma_table;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_g_gamma_config() leave\n");
- ia_css_g_gamma_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_g_gamma_config(struct ia_css_isp_parameters *params,
- const struct ia_css_rgb_gamma_table *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_g_gamma_config() enter:\n");
- ia_css_g_gamma_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->g_gamma_table = *config;
- params->config_changed[IA_CSS_G_GAMMA_ID] = true;
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_g_gamma_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_b_gamma_config(const struct ia_css_isp_parameters *params,
- struct ia_css_rgb_gamma_table *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_b_gamma_config() enter: config=%p\n",
- config);
-
- *config = params->b_gamma_table;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_b_gamma_config() leave\n");
- ia_css_b_gamma_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_b_gamma_config(struct ia_css_isp_parameters *params,
- const struct ia_css_rgb_gamma_table *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_b_gamma_config() enter:\n");
- ia_css_b_gamma_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->b_gamma_table = *config;
- params->config_changed[IA_CSS_B_GAMMA_ID] = true;
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_b_gamma_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_xnr_table_config(const struct ia_css_isp_parameters *params,
- struct ia_css_xnr_table *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_xnr_table_config() enter: config=%p\n",
- config);
-
- *config = params->xnr_table;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_xnr_table_config() leave\n");
- ia_css_xnr_table_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_xnr_table_config(struct ia_css_isp_parameters *params,
- const struct ia_css_xnr_table *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE,
- "ia_css_set_xnr_table_config() enter:\n");
- ia_css_xnr_table_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->xnr_table = *config;
- params->config_changed[IA_CSS_XNR_TABLE_ID] = true;
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_xnr_table_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_formats_config(const struct ia_css_isp_parameters *params,
- struct ia_css_formats_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_formats_config() enter: config=%p\n",
- config);
-
- *config = params->formats_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_formats_config() leave\n");
- ia_css_formats_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_formats_config(struct ia_css_isp_parameters *params,
- const struct ia_css_formats_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_formats_config() enter:\n");
- ia_css_formats_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->formats_config = *config;
- params->config_changed[IA_CSS_FORMATS_ID] = true;
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_formats_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_xnr_config(const struct ia_css_isp_parameters *params,
- struct ia_css_xnr_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_xnr_config() enter: config=%p\n",
- config);
-
- *config = params->xnr_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_xnr_config() leave\n");
- ia_css_xnr_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_xnr_config(struct ia_css_isp_parameters *params,
- const struct ia_css_xnr_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_xnr_config() enter:\n");
- ia_css_xnr_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->xnr_config = *config;
- params->config_changed[IA_CSS_XNR_ID] = true;
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_xnr_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_xnr3_config(const struct ia_css_isp_parameters *params,
- struct ia_css_xnr3_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_xnr3_config() enter: config=%p\n",
- config);
-
- *config = params->xnr3_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_xnr3_config() leave\n");
- ia_css_xnr3_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_xnr3_config(struct ia_css_isp_parameters *params,
- const struct ia_css_xnr3_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_xnr3_config() enter:\n");
- ia_css_xnr3_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->xnr3_config = *config;
- params->config_changed[IA_CSS_XNR3_ID] = true;
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_xnr3_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_s3a_config(const struct ia_css_isp_parameters *params,
- struct ia_css_3a_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_s3a_config() enter: config=%p\n",
- config);
-
- *config = params->s3a_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_s3a_config() leave\n");
- ia_css_s3a_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_s3a_config(struct ia_css_isp_parameters *params,
- const struct ia_css_3a_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_s3a_config() enter:\n");
- ia_css_s3a_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->s3a_config = *config;
- params->config_changed[IA_CSS_BH_ID] = true;
- params->config_changed[IA_CSS_S3A_ID] = true;
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_s3a_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_get_function() */
-
-static void
-ia_css_get_output_config(const struct ia_css_isp_parameters *params,
- struct ia_css_output_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_output_config() enter: config=%p\n",
- config);
-
- *config = params->output_config;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_get_output_config() leave\n");
- ia_css_output_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
-}
-
-/* Code generated by genparam/gencode.c:gen_set_function() */
-
-void
-ia_css_set_output_config(struct ia_css_isp_parameters *params,
- const struct ia_css_output_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_output_config() enter:\n");
- ia_css_output_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
- params->output_config = *config;
- params->config_changed[IA_CSS_OUTPUT_ID] = true;
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_set_output_config() leave: return_void\n");
-}
-
-/* Code generated by genparam/gencode.c:gen_global_access_function() */
-
-void
-ia_css_get_configs(struct ia_css_isp_parameters *params,
- const struct ia_css_isp_config *config)
-{
- ia_css_get_dp_config(params, config->dp_config);
- ia_css_get_wb_config(params, config->wb_config);
- ia_css_get_tnr_config(params, config->tnr_config);
- ia_css_get_ob_config(params, config->ob_config);
- ia_css_get_de_config(params, config->de_config);
- ia_css_get_anr_config(params, config->anr_config);
- ia_css_get_anr2_config(params, config->anr_thres);
- ia_css_get_ce_config(params, config->ce_config);
- ia_css_get_ecd_config(params, config->ecd_config);
- ia_css_get_ynr_config(params, config->ynr_config);
- ia_css_get_fc_config(params, config->fc_config);
- ia_css_get_cnr_config(params, config->cnr_config);
- ia_css_get_macc_config(params, config->macc_config);
- ia_css_get_ctc_config(params, config->ctc_config);
- ia_css_get_aa_config(params, config->aa_config);
- ia_css_get_yuv2rgb_config(params, config->yuv2rgb_cc_config);
- ia_css_get_rgb2yuv_config(params, config->rgb2yuv_cc_config);
- ia_css_get_csc_config(params, config->cc_config);
- ia_css_get_nr_config(params, config->nr_config);
- ia_css_get_gc_config(params, config->gc_config);
- ia_css_get_sdis_horicoef_config(params, config->dvs_coefs);
- ia_css_get_sdis_vertcoef_config(params, config->dvs_coefs);
- ia_css_get_sdis_horiproj_config(params, config->dvs_coefs);
- ia_css_get_sdis_vertproj_config(params, config->dvs_coefs);
- ia_css_get_sdis2_horicoef_config(params, config->dvs2_coefs);
- ia_css_get_sdis2_vertcoef_config(params, config->dvs2_coefs);
- ia_css_get_sdis2_horiproj_config(params, config->dvs2_coefs);
- ia_css_get_sdis2_vertproj_config(params, config->dvs2_coefs);
- ia_css_get_r_gamma_config(params, config->r_gamma_table);
- ia_css_get_g_gamma_config(params, config->g_gamma_table);
- ia_css_get_b_gamma_config(params, config->b_gamma_table);
- ia_css_get_xnr_table_config(params, config->xnr_table);
- ia_css_get_formats_config(params, config->formats_config);
- ia_css_get_xnr_config(params, config->xnr_config);
- ia_css_get_xnr3_config(params, config->xnr3_config);
- ia_css_get_s3a_config(params, config->s3a_config);
- ia_css_get_output_config(params, config->output_config);
-}
-
-/* Code generated by genparam/gencode.c:gen_global_access_function() */
-
-void
-ia_css_set_configs(struct ia_css_isp_parameters *params,
- const struct ia_css_isp_config *config)
-{
- ia_css_set_dp_config(params, config->dp_config);
- ia_css_set_wb_config(params, config->wb_config);
- ia_css_set_tnr_config(params, config->tnr_config);
- ia_css_set_ob_config(params, config->ob_config);
- ia_css_set_de_config(params, config->de_config);
- ia_css_set_anr_config(params, config->anr_config);
- ia_css_set_anr2_config(params, config->anr_thres);
- ia_css_set_ce_config(params, config->ce_config);
- ia_css_set_ecd_config(params, config->ecd_config);
- ia_css_set_ynr_config(params, config->ynr_config);
- ia_css_set_fc_config(params, config->fc_config);
- ia_css_set_cnr_config(params, config->cnr_config);
- ia_css_set_macc_config(params, config->macc_config);
- ia_css_set_ctc_config(params, config->ctc_config);
- ia_css_set_aa_config(params, config->aa_config);
- ia_css_set_yuv2rgb_config(params, config->yuv2rgb_cc_config);
- ia_css_set_rgb2yuv_config(params, config->rgb2yuv_cc_config);
- ia_css_set_csc_config(params, config->cc_config);
- ia_css_set_nr_config(params, config->nr_config);
- ia_css_set_gc_config(params, config->gc_config);
- ia_css_set_sdis_horicoef_config(params, config->dvs_coefs);
- ia_css_set_sdis_vertcoef_config(params, config->dvs_coefs);
- ia_css_set_sdis_horiproj_config(params, config->dvs_coefs);
- ia_css_set_sdis_vertproj_config(params, config->dvs_coefs);
- ia_css_set_sdis2_horicoef_config(params, config->dvs2_coefs);
- ia_css_set_sdis2_vertcoef_config(params, config->dvs2_coefs);
- ia_css_set_sdis2_horiproj_config(params, config->dvs2_coefs);
- ia_css_set_sdis2_vertproj_config(params, config->dvs2_coefs);
- ia_css_set_r_gamma_config(params, config->r_gamma_table);
- ia_css_set_g_gamma_config(params, config->g_gamma_table);
- ia_css_set_b_gamma_config(params, config->b_gamma_table);
- ia_css_set_xnr_table_config(params, config->xnr_table);
- ia_css_set_formats_config(params, config->formats_config);
- ia_css_set_xnr_config(params, config->xnr_config);
- ia_css_set_xnr3_config(params, config->xnr3_config);
- ia_css_set_s3a_config(params, config->s3a_config);
- ia_css_set_output_config(params, config->output_config);
-}
diff --git a/drivers/staging/media/atomisp/pci/css_2401_system/hive/ia_css_isp_states.c b/drivers/staging/media/atomisp/pci/css_2401_system/hive/ia_css_isp_states.c
deleted file mode 100644
index 514ffe0..0000000
--- a/drivers/staging/media/atomisp/pci/css_2401_system/hive/ia_css_isp_states.c
+++ /dev/null
@@ -1,224 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/*
- * Support for Intel Camera Imaging ISP subsystem.
- * Copyright (c) 2015, Intel Corporation.
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms and conditions of the GNU General Public License,
- * version 2, as published by the Free Software Foundation.
- *
- * This program is distributed in the hope it will be useful, but WITHOUT
- * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
- * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
- * more details.
- */
-
-/* Generated code: do not edit or commmit. */
-
-#include "ia_css_pipeline.h"
-#include "ia_css_isp_states.h"
-#include "ia_css_debug.h"
-#include "assert_support.h"
-
-/* Code generated by genparam/genstate.c:gen_init_function() */
-
-static void
-ia_css_initialize_aa_state(
- const struct ia_css_binary *binary)
-{
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_initialize_aa_state() enter:\n");
-
- {
- unsigned int size = binary->info->mem_offsets.offsets.state->vmem.aa.size;
- unsigned int offset = binary->info->mem_offsets.offsets.state->vmem.aa.offset;
-
- if (size)
- memset(&binary->mem_params.params[IA_CSS_PARAM_CLASS_STATE][IA_CSS_ISP_VMEM].address[offset],
- 0, size);
- }
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_initialize_aa_state() leave:\n");
-}
-
-/* Code generated by genparam/genstate.c:gen_init_function() */
-
-static void
-ia_css_initialize_cnr_state(
- const struct ia_css_binary *binary)
-{
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_initialize_cnr_state() enter:\n");
-
- {
- unsigned int size = binary->info->mem_offsets.offsets.state->vmem.cnr.size;
-
- unsigned int offset = binary->info->mem_offsets.offsets.state->vmem.cnr.offset;
-
- if (size) {
- ia_css_init_cnr_state(
- &binary->mem_params.params[IA_CSS_PARAM_CLASS_STATE][IA_CSS_ISP_VMEM].address[offset],
- size);
- }
- }
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_initialize_cnr_state() leave:\n");
-}
-
-/* Code generated by genparam/genstate.c:gen_init_function() */
-
-static void
-ia_css_initialize_cnr2_state(
- const struct ia_css_binary *binary)
-{
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_initialize_cnr2_state() enter:\n");
-
- {
- unsigned int size = binary->info->mem_offsets.offsets.state->vmem.cnr2.size;
-
- unsigned int offset = binary->info->mem_offsets.offsets.state->vmem.cnr2.offset;
-
- if (size) {
- ia_css_init_cnr2_state(
- &binary->mem_params.params[IA_CSS_PARAM_CLASS_STATE][IA_CSS_ISP_VMEM].address[offset],
- size);
- }
- }
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_initialize_cnr2_state() leave:\n");
-}
-
-/* Code generated by genparam/genstate.c:gen_init_function() */
-
-static void
-ia_css_initialize_dp_state(
- const struct ia_css_binary *binary)
-{
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_initialize_dp_state() enter:\n");
-
- {
- unsigned int size = binary->info->mem_offsets.offsets.state->vmem.dp.size;
-
- unsigned int offset = binary->info->mem_offsets.offsets.state->vmem.dp.offset;
-
- if (size) {
- ia_css_init_dp_state(
- &binary->mem_params.params[IA_CSS_PARAM_CLASS_STATE][IA_CSS_ISP_VMEM].address[offset],
- size);
- }
- }
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_initialize_dp_state() leave:\n");
-}
-
-/* Code generated by genparam/genstate.c:gen_init_function() */
-
-static void
-ia_css_initialize_de_state(
- const struct ia_css_binary *binary)
-{
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_initialize_de_state() enter:\n");
-
- {
- unsigned int size = binary->info->mem_offsets.offsets.state->vmem.de.size;
-
- unsigned int offset = binary->info->mem_offsets.offsets.state->vmem.de.offset;
-
- if (size) {
- ia_css_init_de_state(
- &binary->mem_params.params[IA_CSS_PARAM_CLASS_STATE][IA_CSS_ISP_VMEM].address[offset],
- size);
- }
- }
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_initialize_de_state() leave:\n");
-}
-
-/* Code generated by genparam/genstate.c:gen_init_function() */
-
-static void
-ia_css_initialize_tnr_state(
- const struct ia_css_binary *binary)
-{
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_initialize_tnr_state() enter:\n");
-
- {
- unsigned int size = binary->info->mem_offsets.offsets.state->dmem.tnr.size;
-
- unsigned int offset = binary->info->mem_offsets.offsets.state->dmem.tnr.offset;
-
- if (size) {
- ia_css_init_tnr_state((struct sh_css_isp_tnr_dmem_state *)
- &binary->mem_params.params[IA_CSS_PARAM_CLASS_STATE][IA_CSS_ISP_DMEM].address[offset],
- size);
- }
- }
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_initialize_tnr_state() leave:\n");
-}
-
-/* Code generated by genparam/genstate.c:gen_init_function() */
-
-static void
-ia_css_initialize_ref_state(
- const struct ia_css_binary *binary)
-{
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_initialize_ref_state() enter:\n");
-
- {
- unsigned int size = binary->info->mem_offsets.offsets.state->dmem.ref.size;
-
- unsigned int offset = binary->info->mem_offsets.offsets.state->dmem.ref.offset;
-
- if (size) {
- ia_css_init_ref_state((struct sh_css_isp_ref_dmem_state *)
- &binary->mem_params.params[IA_CSS_PARAM_CLASS_STATE][IA_CSS_ISP_DMEM].address[offset],
- size);
- }
- }
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_initialize_ref_state() leave:\n");
-}
-
-/* Code generated by genparam/genstate.c:gen_init_function() */
-
-static void
-ia_css_initialize_ynr_state(
- const struct ia_css_binary *binary)
-{
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_initialize_ynr_state() enter:\n");
-
- {
- unsigned int size = binary->info->mem_offsets.offsets.state->vmem.ynr.size;
-
- unsigned int offset = binary->info->mem_offsets.offsets.state->vmem.ynr.offset;
-
- if (size) {
- ia_css_init_ynr_state(
- &binary->mem_params.params[IA_CSS_PARAM_CLASS_STATE][IA_CSS_ISP_VMEM].address[offset],
- size);
- }
- }
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_initialize_ynr_state() leave:\n");
-}
-
-/* Code generated by genparam/genstate.c:gen_state_init_table() */
-
-void (*ia_css_kernel_init_state[IA_CSS_NUM_STATE_IDS])(
- const struct ia_css_binary *binary) = {
- ia_css_initialize_aa_state,
- ia_css_initialize_cnr_state,
- ia_css_initialize_cnr2_state,
- ia_css_initialize_dp_state,
- ia_css_initialize_de_state,
- ia_css_initialize_tnr_state,
- ia_css_initialize_ref_state,
- ia_css_initialize_ynr_state,
-};
diff --git a/drivers/staging/media/atomisp/pci/hive_isp_css_common/host/gdc.c b/drivers/staging/media/atomisp/pci/hive_isp_css_common/host/gdc.c
index 8ed1cff..25e082d 100644
--- a/drivers/staging/media/atomisp/pci/hive_isp_css_common/host/gdc.c
+++ b/drivers/staging/media/atomisp/pci/hive_isp_css_common/host/gdc.c
@@ -28,10 +28,6 @@ static inline void gdc_reg_store(
const unsigned int reg,
const hrt_data value);
-static inline hrt_data gdc_reg_load(
- const gdc_ID_t ID,
- const unsigned int reg);
-
#ifndef __INLINE_GDC__
#include "gdc_private.h"
#endif /* __INLINE_GDC__ */
@@ -117,10 +113,3 @@ static inline void gdc_reg_store(
ia_css_device_store_uint32(GDC_BASE[ID] + reg * sizeof(hrt_data), value);
return;
}
-
-static inline hrt_data gdc_reg_load(
- const gdc_ID_t ID,
- const unsigned int reg)
-{
- return ia_css_device_load_uint32(GDC_BASE[ID] + reg * sizeof(hrt_data));
-}
diff --git a/drivers/staging/media/atomisp/pci/ia_css_acc_types.h b/drivers/staging/media/atomisp/pci/ia_css_acc_types.h
index 36583ab..d0ce2f8 100644
--- a/drivers/staging/media/atomisp/pci/ia_css_acc_types.h
+++ b/drivers/staging/media/atomisp/pci/ia_css_acc_types.h
@@ -222,11 +222,6 @@ struct ia_css_binary_info {
struct ia_css_isp_param_isp_segments mem_initializers;
/* MW: Packing (related) bools in an integer ?? */
struct {
- /* ISP2401 */
- u8 luma_only;
- u8 input_yuv;
- u8 input_raw;
-
u8 reduced_pipe;
u8 vf_veceven;
u8 dis;
diff --git a/drivers/staging/media/atomisp/pci/ia_css_event_public.h b/drivers/staging/media/atomisp/pci/ia_css_event_public.h
index 08ea801..76219d7 100644
--- a/drivers/staging/media/atomisp/pci/ia_css_event_public.h
+++ b/drivers/staging/media/atomisp/pci/ia_css_event_public.h
@@ -157,21 +157,6 @@ struct ia_css_event {
int
ia_css_dequeue_psys_event(struct ia_css_event *event);
-/* @brief Dequeue an event from the CSS system.
- *
- * @param[out] event Pointer to the event struct which will be filled by
- * this function if an event is available.
- * @return -ENODATA if no events are
- * available or
- * 0 otherwise.
- *
- * deprecated{Use ia_css_dequeue_psys_event instead}.
- * Unless the isys event queue is explicitly enabled, this function will
- * dequeue both isys (EOF) and psys events (all others).
- */
-int
-ia_css_dequeue_event(struct ia_css_event *event);
-
/* @brief Dequeue an ISYS event from the CSS system.
*
* @param[out] event Pointer to the event struct which will be filled by
@@ -182,7 +167,7 @@ ia_css_dequeue_event(struct ia_css_event *event);
*
* This function dequeues an event from the ISYS event queue. The queue is
* between host and the CSS system.
- * Unlike the ia_css_dequeue_event() function, this function can be called
+ * Unlike the ia_css_dequeue_psys_event() function, this function can be called
* directly from an interrupt service routine (ISR) and it is safe to call
* this function in parallel with other CSS API functions (but only one
* call to this function should be in flight at any point in time).
diff --git a/drivers/staging/media/atomisp/pci/ia_css_isp_configs.c b/drivers/staging/media/atomisp/pci/ia_css_isp_configs.c
new file mode 100644
index 0000000..d28a76a
--- /dev/null
+++ b/drivers/staging/media/atomisp/pci/ia_css_isp_configs.c
@@ -0,0 +1,321 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Support for Intel Camera Imaging ISP subsystem.
+ * Copyright (c) 2015, Intel Corporation.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ */
+
+/* Generated code: do not edit or commmit. */
+
+#define IA_CSS_INCLUDE_CONFIGURATIONS
+#include "ia_css_pipeline.h"
+#include "ia_css_isp_configs.h"
+#include "ia_css_debug.h"
+#include "assert_support.h"
+
+int ia_css_configure_iterator(const struct ia_css_binary *binary,
+ const struct ia_css_iterator_configuration *config_dmem)
+{
+ unsigned int offset = 0;
+ unsigned int size = 0;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "%s:\n", __func__);
+
+ if (!binary->info->mem_offsets.offsets.config)
+ return 0;
+
+ size = binary->info->mem_offsets.offsets.config->dmem.iterator.size;
+ if (!size)
+ return 0;
+
+ offset = binary->info->mem_offsets.offsets.config->dmem.iterator.offset;
+
+ ia_css_iterator_config((struct sh_css_isp_iterator_isp_config *)
+ &binary->mem_params.params[IA_CSS_PARAM_CLASS_CONFIG][IA_CSS_ISP_DMEM].address[offset],
+ config_dmem, size);
+ return 0;
+}
+
+int ia_css_configure_copy_output(const struct ia_css_binary *binary,
+ const struct ia_css_copy_output_configuration *config_dmem)
+{
+ unsigned int offset = 0;
+ unsigned int size = 0;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "%s:\n", __func__);
+
+ if (!binary->info->mem_offsets.offsets.config)
+ return 0;
+
+ size = binary->info->mem_offsets.offsets.config->dmem.copy_output.size;
+ if (!size)
+ return 0;
+
+ offset = binary->info->mem_offsets.offsets.config->dmem.copy_output.offset;
+
+ ia_css_copy_output_config((struct sh_css_isp_copy_output_isp_config *)
+ &binary->mem_params.params[IA_CSS_PARAM_CLASS_CONFIG][IA_CSS_ISP_DMEM].address[offset],
+ config_dmem, size);
+ return 0;
+}
+
+/* Code generated by genparam/genconfig.c:gen_configure_function() */
+
+int ia_css_configure_crop(const struct ia_css_binary *binary,
+ const struct ia_css_crop_configuration *config_dmem)
+{
+ unsigned int offset = 0;
+ unsigned int size = 0;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "%s:\n", __func__);
+
+ if (!binary->info->mem_offsets.offsets.config)
+ return 0;
+
+ size = binary->info->mem_offsets.offsets.config->dmem.crop.size;
+ if (!size)
+ return 0;
+
+ offset = binary->info->mem_offsets.offsets.config->dmem.crop.offset;
+
+ ia_css_crop_config((struct sh_css_isp_crop_isp_config *)
+ &binary->mem_params.params[IA_CSS_PARAM_CLASS_CONFIG][IA_CSS_ISP_DMEM].address[offset],
+ config_dmem, size);
+ return 0;
+}
+
+int ia_css_configure_fpn(const struct ia_css_binary *binary,
+ const struct ia_css_fpn_configuration *config_dmem)
+{
+ unsigned int offset = 0;
+ unsigned int size = 0;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "%s:\n", __func__);
+
+ if (!binary->info->mem_offsets.offsets.config)
+ return 0;
+
+ size = binary->info->mem_offsets.offsets.config->dmem.fpn.size;
+ if (!size)
+ return 0;
+
+ offset = binary->info->mem_offsets.offsets.config->dmem.fpn.offset;
+ ia_css_fpn_config((struct sh_css_isp_fpn_isp_config *)
+ &binary->mem_params.params[IA_CSS_PARAM_CLASS_CONFIG][IA_CSS_ISP_DMEM].address[offset],
+ config_dmem, size);
+ return 0;
+}
+
+int ia_css_configure_dvs(const struct ia_css_binary *binary,
+ const struct ia_css_dvs_configuration *config_dmem)
+{
+ unsigned int offset = 0;
+ unsigned int size = 0;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "%s:\n", __func__);
+
+ if (!binary->info->mem_offsets.offsets.config)
+ return 0;
+
+ size = binary->info->mem_offsets.offsets.config->dmem.dvs.size;
+ if (!size)
+ return 0;
+
+ offset = binary->info->mem_offsets.offsets.config->dmem.dvs.offset;
+ ia_css_dvs_config((struct sh_css_isp_dvs_isp_config *)
+ &binary->mem_params.params[IA_CSS_PARAM_CLASS_CONFIG][IA_CSS_ISP_DMEM].address[offset],
+ config_dmem, size);
+ return 0;
+}
+
+int ia_css_configure_qplane(const struct ia_css_binary *binary,
+ const struct ia_css_qplane_configuration *config_dmem)
+{
+ unsigned int offset = 0;
+ unsigned int size = 0;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "%s:\n", __func__);
+
+ if (!binary->info->mem_offsets.offsets.config)
+ return 0;
+
+ size = binary->info->mem_offsets.offsets.config->dmem.qplane.size;
+ if (!size)
+ return 0;
+
+ offset = binary->info->mem_offsets.offsets.config->dmem.qplane.offset;
+ ia_css_qplane_config((struct sh_css_isp_qplane_isp_config *)
+ &binary->mem_params.params[IA_CSS_PARAM_CLASS_CONFIG][IA_CSS_ISP_DMEM].address[offset],
+ config_dmem, size);
+
+ return 0;
+}
+
+int ia_css_configure_output0(const struct ia_css_binary *binary,
+ const struct ia_css_output0_configuration *config_dmem)
+{
+ unsigned int offset = 0;
+ unsigned int size = 0;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "%s:\n", __func__);
+
+ if (!binary->info->mem_offsets.offsets.config)
+ return 0;
+
+ size = binary->info->mem_offsets.offsets.config->dmem.output0.size;
+ if (!size)
+ return 0;
+
+ offset = binary->info->mem_offsets.offsets.config->dmem.output0.offset;
+
+ ia_css_output0_config((struct sh_css_isp_output_isp_config *)
+ &binary->mem_params.params[IA_CSS_PARAM_CLASS_CONFIG][IA_CSS_ISP_DMEM].address[offset],
+ config_dmem, size);
+ return 0;
+}
+
+int ia_css_configure_output1(const struct ia_css_binary *binary,
+ const struct ia_css_output1_configuration *config_dmem)
+{
+ unsigned int offset = 0;
+ unsigned int size = 0;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "%s:\n", __func__);
+
+ if (!binary->info->mem_offsets.offsets.config)
+ return 0;
+
+ size = binary->info->mem_offsets.offsets.config->dmem.output1.size;
+ if (!size)
+ return 0;
+
+ offset = binary->info->mem_offsets.offsets.config->dmem.output1.offset;
+
+ ia_css_output1_config((struct sh_css_isp_output_isp_config *)
+ &binary->mem_params.params[IA_CSS_PARAM_CLASS_CONFIG][IA_CSS_ISP_DMEM].address[offset],
+ config_dmem, size);
+ return 0;
+}
+
+int ia_css_configure_output(const struct ia_css_binary *binary,
+ const struct ia_css_output_configuration *config_dmem)
+{
+ unsigned int offset = 0;
+ unsigned int size = 0;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "%s:\n", __func__);
+
+ if (!binary->info->mem_offsets.offsets.config)
+ return 0;
+
+ size = binary->info->mem_offsets.offsets.config->dmem.output.size;
+ if (!size)
+ return 0;
+
+ offset = binary->info->mem_offsets.offsets.config->dmem.output.offset;
+
+ ia_css_output_config((struct sh_css_isp_output_isp_config *)
+ &binary->mem_params.params[IA_CSS_PARAM_CLASS_CONFIG][IA_CSS_ISP_DMEM].address[offset],
+ config_dmem, size);
+ return 0;
+}
+
+int ia_css_configure_raw(const struct ia_css_binary *binary,
+ const struct ia_css_raw_configuration *config_dmem)
+{
+ unsigned int offset = 0;
+ unsigned int size = 0;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "%s:\n", __func__);
+
+ if (!binary->info->mem_offsets.offsets.config)
+ return 0;
+
+ size = binary->info->mem_offsets.offsets.config->dmem.raw.size;
+ if (!size)
+ return 0;
+
+ offset = binary->info->mem_offsets.offsets.config->dmem.raw.offset;
+
+ ia_css_raw_config((struct sh_css_isp_raw_isp_config *)
+ &binary->mem_params.params[IA_CSS_PARAM_CLASS_CONFIG][IA_CSS_ISP_DMEM].address[offset],
+ config_dmem, size);
+ return 0;
+}
+
+int ia_css_configure_tnr(const struct ia_css_binary *binary,
+ const struct ia_css_tnr_configuration *config_dmem)
+{
+ unsigned int offset = 0;
+ unsigned int size = 0;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "%s:\n", __func__);
+
+ if (!binary->info->mem_offsets.offsets.config)
+ return 0;
+
+ size = binary->info->mem_offsets.offsets.config->dmem.tnr.size;
+ if (!size)
+ return 0;
+
+ offset = binary->info->mem_offsets.offsets.config->dmem.tnr.offset;
+
+ ia_css_tnr_config((struct sh_css_isp_tnr_isp_config *)
+ &binary->mem_params.params[IA_CSS_PARAM_CLASS_CONFIG][IA_CSS_ISP_DMEM].address[offset],
+ config_dmem, size);
+ return 0;
+}
+
+int ia_css_configure_ref(const struct ia_css_binary *binary,
+ const struct ia_css_ref_configuration *config_dmem)
+{
+ unsigned int offset = 0;
+ unsigned int size = 0;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "%s:\n", __func__);
+
+ if (!binary->info->mem_offsets.offsets.config)
+ return 0;
+
+ size = binary->info->mem_offsets.offsets.config->dmem.ref.size;
+ if (!size)
+ return 0;
+
+ offset = binary->info->mem_offsets.offsets.config->dmem.ref.offset;
+
+ ia_css_ref_config((struct sh_css_isp_ref_isp_config *)
+ &binary->mem_params.params[IA_CSS_PARAM_CLASS_CONFIG][IA_CSS_ISP_DMEM].address[offset],
+ config_dmem, size);
+ return 0;
+}
+
+int ia_css_configure_vf(const struct ia_css_binary *binary,
+ const struct ia_css_vf_configuration *config_dmem)
+{
+ unsigned int offset = 0;
+ unsigned int size = 0;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "%s:\n", __func__);
+
+ if (!binary->info->mem_offsets.offsets.config)
+ return 0;
+
+ size = binary->info->mem_offsets.offsets.config->dmem.vf.size;
+ if (!size)
+ return 0;
+
+ offset = binary->info->mem_offsets.offsets.config->dmem.vf.offset;
+
+ ia_css_vf_config((struct sh_css_isp_vf_isp_config *)
+ &binary->mem_params.params[IA_CSS_PARAM_CLASS_CONFIG][IA_CSS_ISP_DMEM].address[offset],
+ config_dmem, size);
+ return 0;
+}
diff --git a/drivers/staging/media/atomisp/pci/ia_css_isp_configs.h b/drivers/staging/media/atomisp/pci/ia_css_isp_configs.h
index 1abb2fd..fffcfc8 100644
--- a/drivers/staging/media/atomisp/pci/ia_css_isp_configs.h
+++ b/drivers/staging/media/atomisp/pci/ia_css_isp_configs.h
@@ -23,22 +23,15 @@
#include "isp/kernels/raw/raw_1.0/ia_css_raw.host.h"
#include "isp/kernels/ref/ref_1.0/ia_css_ref.host.h"
#include "isp/kernels/s3a/s3a_1.0/ia_css_s3a.host.h"
-
-/* ISP2401 */
-#include "isp/kernels/sc/sc_1.0/ia_css_sc.host.h"
-
#include "isp/kernels/tnr/tnr_1.0/ia_css_tnr.host.h"
#include "isp/kernels/vf/vf_1.0/ia_css_vf.host.h"
#include "isp/kernels/iterator/iterator_1.0/ia_css_iterator.host.h"
#include "isp/kernels/copy_output/copy_output_1.0/ia_css_copy_output.host.h"
-#endif /* IA_CSS_INCLUDE_CONFIGURATIONS */
-/* Generated code: do not edit or commmit. */
+#endif
#ifndef _IA_CSS_ISP_CONFIG_H
#define _IA_CSS_ISP_CONFIG_H
-/* Code generated by genparam/gencode.c:gen_param_enum() */
-
enum ia_css_configuration_ids {
IA_CSS_ITERATOR_CONFIG_ID,
IA_CSS_COPY_OUTPUT_CONFIG_ID,
@@ -60,8 +53,6 @@ enum ia_css_configuration_ids {
IA_CSS_NUM_CONFIGURATION_IDS
};
-/* Code generated by genparam/gencode.c:gen_param_offsets() */
-
struct ia_css_config_memory_offsets {
struct {
struct ia_css_isp_parameter iterator;
@@ -73,10 +64,6 @@ struct ia_css_config_memory_offsets {
struct ia_css_isp_parameter output0;
struct ia_css_isp_parameter output1;
struct ia_css_isp_parameter output;
-
- /* ISP2401 */
- struct ia_css_isp_parameter sc;
-
struct ia_css_isp_parameter raw;
struct ia_css_isp_parameter tnr;
struct ia_css_isp_parameter ref;
@@ -88,96 +75,44 @@ struct ia_css_config_memory_offsets {
#include "ia_css_stream.h" /* struct ia_css_stream */
#include "ia_css_binary.h" /* struct ia_css_binary */
-/* Code generated by genparam/genconfig.c:gen_configure_function() */
-void
-ia_css_configure_iterator(
- const struct ia_css_binary *binary,
- const struct ia_css_iterator_configuration *config_dmem);
+int ia_css_configure_iterator(const struct ia_css_binary *binary,
+ const struct ia_css_iterator_configuration *config_dmem);
-/* Code generated by genparam/genconfig.c:gen_configure_function() */
+int ia_css_configure_copy_output(const struct ia_css_binary *binary,
+ const struct ia_css_copy_output_configuration *config_dmem);
-void
-ia_css_configure_copy_output(
- const struct ia_css_binary *binary,
- const struct ia_css_copy_output_configuration *config_dmem);
+int ia_css_configure_crop(const struct ia_css_binary *binary,
+ const struct ia_css_crop_configuration *config_dmem);
-/* Code generated by genparam/genconfig.c:gen_configure_function() */
+int ia_css_configure_fpn(const struct ia_css_binary *binary,
+ const struct ia_css_fpn_configuration *config_dmem);
-void
-ia_css_configure_crop(
- const struct ia_css_binary *binary,
- const struct ia_css_crop_configuration *config_dmem);
+int ia_css_configure_dvs(const struct ia_css_binary *binary,
+ const struct ia_css_dvs_configuration *config_dmem);
-/* Code generated by genparam/genconfig.c:gen_configure_function() */
+int ia_css_configure_qplane(const struct ia_css_binary *binary,
+ const struct ia_css_qplane_configuration *config_dmem);
+int ia_css_configure_output0(const struct ia_css_binary *binary,
+ const struct ia_css_output0_configuration *config_dmem);
-void
-ia_css_configure_fpn(
- const struct ia_css_binary *binary,
- const struct ia_css_fpn_configuration *config_dmem);
+int ia_css_configure_output1(const struct ia_css_binary *binary,
+ const struct ia_css_output1_configuration *config_dmem);
-/* Code generated by genparam/genconfig.c:gen_configure_function() */
+int ia_css_configure_output(const struct ia_css_binary *binary,
+ const struct ia_css_output_configuration *config_dmem);
-void
-ia_css_configure_dvs(
- const struct ia_css_binary *binary,
- const struct ia_css_dvs_configuration *config_dmem);
+int ia_css_configure_raw(const struct ia_css_binary *binary,
+ const struct ia_css_raw_configuration *config_dmem);
-/* Code generated by genparam/genconfig.c:gen_configure_function() */
+int ia_css_configure_tnr(const struct ia_css_binary *binary,
+ const struct ia_css_tnr_configuration *config_dmem);
-void
-ia_css_configure_qplane(
- const struct ia_css_binary *binary,
- const struct ia_css_qplane_configuration *config_dmem);
+int ia_css_configure_ref(const struct ia_css_binary *binary,
+ const struct ia_css_ref_configuration *config_dmem);
-/* Code generated by genparam/genconfig.c:gen_configure_function() */
-
-void
-ia_css_configure_output0(
- const struct ia_css_binary *binary,
- const struct ia_css_output0_configuration *config_dmem);
-
-/* Code generated by genparam/genconfig.c:gen_configure_function() */
-
-void
-ia_css_configure_output1(
- const struct ia_css_binary *binary,
- const struct ia_css_output1_configuration *config_dmem);
-
-/* Code generated by genparam/genconfig.c:gen_configure_function() */
-
-void
-ia_css_configure_output(
- const struct ia_css_binary *binary,
- const struct ia_css_output_configuration *config_dmem);
-
-/* Code generated by genparam/genconfig.c:gen_configure_function() */
-
-void
-ia_css_configure_raw(
- const struct ia_css_binary *binary,
- const struct ia_css_raw_configuration *config_dmem);
-
-/* Code generated by genparam/genconfig.c:gen_configure_function() */
-
-void
-ia_css_configure_tnr(
- const struct ia_css_binary *binary,
- const struct ia_css_tnr_configuration *config_dmem);
-
-/* Code generated by genparam/genconfig.c:gen_configure_function() */
-
-void
-ia_css_configure_ref(
- const struct ia_css_binary *binary,
- const struct ia_css_ref_configuration *config_dmem);
-
-/* Code generated by genparam/genconfig.c:gen_configure_function() */
-
-void
-ia_css_configure_vf(
- const struct ia_css_binary *binary,
- const struct ia_css_vf_configuration *config_dmem);
+int ia_css_configure_vf(const struct ia_css_binary *binary,
+ const struct ia_css_vf_configuration *config_dmem);
#endif /* IA_CSS_INCLUDE_CONFIGURATION */
diff --git a/drivers/staging/media/atomisp/pci/ia_css_isp_params.c b/drivers/staging/media/atomisp/pci/ia_css_isp_params.c
new file mode 100644
index 0000000..503ac65
--- /dev/null
+++ b/drivers/staging/media/atomisp/pci/ia_css_isp_params.c
@@ -0,0 +1,3344 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Support for Intel Camera Imaging ISP subsystem.
+ * Copyright (c) 2015, Intel Corporation.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ */
+
+#define IA_CSS_INCLUDE_PARAMETERS
+#include "sh_css_params.h"
+#include "isp/kernels/aa/aa_2/ia_css_aa2.host.h"
+#include "isp/kernels/anr/anr_1.0/ia_css_anr.host.h"
+#include "isp/kernels/anr/anr_2/ia_css_anr2.host.h"
+#include "isp/kernels/bh/bh_2/ia_css_bh.host.h"
+#include "isp/kernels/bnr/bnr_1.0/ia_css_bnr.host.h"
+#include "isp/kernels/bnr/bnr2_2/ia_css_bnr2_2.host.h"
+#include "isp/kernels/cnr/cnr_2/ia_css_cnr2.host.h"
+#include "isp/kernels/crop/crop_1.0/ia_css_crop.host.h"
+#include "isp/kernels/csc/csc_1.0/ia_css_csc.host.h"
+#include "isp/kernels/ctc/ctc_1.0/ia_css_ctc.host.h"
+#include "isp/kernels/ctc/ctc1_5/ia_css_ctc1_5.host.h"
+#include "isp/kernels/ctc/ctc2/ia_css_ctc2.host.h"
+#include "isp/kernels/de/de_1.0/ia_css_de.host.h"
+#include "isp/kernels/de/de_2/ia_css_de2.host.h"
+#include "isp/kernels/dp/dp_1.0/ia_css_dp.host.h"
+#include "isp/kernels/fixedbds/fixedbds_1.0/ia_css_fixedbds_param.h"
+#include "isp/kernels/fpn/fpn_1.0/ia_css_fpn.host.h"
+#include "isp/kernels/gc/gc_1.0/ia_css_gc.host.h"
+#include "isp/kernels/gc/gc_2/ia_css_gc2.host.h"
+#include "isp/kernels/macc/macc_1.0/ia_css_macc.host.h"
+#include "isp/kernels/macc/macc1_5/ia_css_macc1_5.host.h"
+#include "isp/kernels/ob/ob_1.0/ia_css_ob.host.h"
+#include "isp/kernels/ob/ob2/ia_css_ob2.host.h"
+#include "isp/kernels/output/output_1.0/ia_css_output.host.h"
+#include "isp/kernels/raw_aa_binning/raw_aa_binning_1.0/ia_css_raa.host.h"
+#include "isp/kernels/s3a/s3a_1.0/ia_css_s3a.host.h"
+#include "isp/kernels/sc/sc_1.0/ia_css_sc.host.h"
+#include "isp/kernels/sdis/sdis_1.0/ia_css_sdis.host.h"
+#include "isp/kernels/sdis/sdis_2/ia_css_sdis2.host.h"
+#include "isp/kernels/tnr/tnr_1.0/ia_css_tnr.host.h"
+#include "isp/kernels/uds/uds_1.0/ia_css_uds_param.h"
+#include "isp/kernels/wb/wb_1.0/ia_css_wb.host.h"
+#include "isp/kernels/xnr/xnr_1.0/ia_css_xnr.host.h"
+#include "isp/kernels/xnr/xnr_3.0/ia_css_xnr3.host.h"
+#include "isp/kernels/ynr/ynr_1.0/ia_css_ynr.host.h"
+#include "isp/kernels/ynr/ynr_2/ia_css_ynr2.host.h"
+#include "isp/kernels/fc/fc_1.0/ia_css_formats.host.h"
+#include "isp/kernels/tdf/tdf_1.0/ia_css_tdf.host.h"
+#include "isp/kernels/dpc2/ia_css_dpc2.host.h"
+#include "isp/kernels/eed1_8/ia_css_eed1_8.host.h"
+#include "isp/kernels/bnlm/ia_css_bnlm.host.h"
+#include "isp/kernels/conversion/conversion_1.0/ia_css_conversion.host.h"
+/* Generated code: do not edit or commmit. */
+
+#include "ia_css_pipeline.h"
+#include "ia_css_isp_params.h"
+#include "ia_css_debug.h"
+#include "assert_support.h"
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_aa(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->dmem.aa.size;
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->dmem.aa.offset;
+
+ if (size) {
+ struct sh_css_isp_aa_params *t = (struct sh_css_isp_aa_params *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset];
+ t->strength = params->aa_config.strength;
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_anr(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->dmem.anr.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->dmem.anr.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_anr() enter:\n");
+
+ ia_css_anr_encode((struct sh_css_isp_anr_params *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
+ ¶ms->anr_config,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_anr() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_anr2(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->vmem.anr2.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->vmem.anr2.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_anr2() enter:\n");
+
+ ia_css_anr2_vmem_encode((struct ia_css_isp_anr2_params *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_VMEM].address[offset],
+ ¶ms->anr_thres,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_VMEM] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_anr2() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_bh(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->dmem.bh.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->dmem.bh.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_bh() enter:\n");
+
+ ia_css_bh_encode((struct sh_css_isp_bh_params *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
+ ¶ms->s3a_config,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_bh() leave:\n");
+ }
+ }
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->hmem0.bh.size;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_bh() enter:\n");
+
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_HMEM0] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_bh() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_cnr(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->dmem.cnr.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->dmem.cnr.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_cnr() enter:\n");
+
+ ia_css_cnr_encode((struct sh_css_isp_cnr_params *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
+ ¶ms->cnr_config,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_cnr() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_crop(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->dmem.crop.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->dmem.crop.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_crop() enter:\n");
+
+ ia_css_crop_encode((struct sh_css_isp_crop_isp_params *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
+ ¶ms->crop_config,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_crop() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_csc(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->dmem.csc.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->dmem.csc.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_csc() enter:\n");
+
+ ia_css_csc_encode((struct sh_css_isp_csc_params *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
+ ¶ms->cc_config,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_csc() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_dp(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->dmem.dp.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->dmem.dp.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_dp() enter:\n");
+
+ ia_css_dp_encode((struct sh_css_isp_dp_params *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
+ ¶ms->dp_config,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_dp() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_bnr(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->dmem.bnr.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->dmem.bnr.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_bnr() enter:\n");
+
+ ia_css_bnr_encode((struct sh_css_isp_bnr_params *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
+ ¶ms->nr_config,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_bnr() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_de(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->dmem.de.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->dmem.de.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_de() enter:\n");
+
+ ia_css_de_encode((struct sh_css_isp_de_params *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
+ ¶ms->de_config,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_de() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_ecd(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->dmem.ecd.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->dmem.ecd.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_ecd() enter:\n");
+
+ ia_css_ecd_encode((struct sh_css_isp_ecd_params *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
+ ¶ms->ecd_config,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_ecd() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_formats(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->dmem.formats.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->dmem.formats.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_formats() enter:\n");
+
+ ia_css_formats_encode((struct sh_css_isp_formats_params *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
+ ¶ms->formats_config,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_formats() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_fpn(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->dmem.fpn.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->dmem.fpn.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_fpn() enter:\n");
+
+ ia_css_fpn_encode((struct sh_css_isp_fpn_params *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
+ ¶ms->fpn_config,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_fpn() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_gc(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->dmem.gc.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->dmem.gc.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_gc() enter:\n");
+
+ ia_css_gc_encode((struct sh_css_isp_gc_params *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
+ ¶ms->gc_config,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_gc() leave:\n");
+ }
+ }
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->vamem1.gc.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->vamem1.gc.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_gc() enter:\n");
+
+ ia_css_gc_vamem_encode((struct sh_css_isp_gc_vamem_params *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_VAMEM1].address[offset],
+ ¶ms->gc_table,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_VAMEM1] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_gc() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_ce(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->dmem.ce.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->dmem.ce.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_ce() enter:\n");
+
+ ia_css_ce_encode((struct sh_css_isp_ce_params *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
+ ¶ms->ce_config,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_ce() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_yuv2rgb(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->dmem.yuv2rgb.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->dmem.yuv2rgb.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_yuv2rgb() enter:\n");
+
+ ia_css_yuv2rgb_encode((struct sh_css_isp_csc_params *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
+ ¶ms->yuv2rgb_cc_config,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_yuv2rgb() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_rgb2yuv(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->dmem.rgb2yuv.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->dmem.rgb2yuv.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_rgb2yuv() enter:\n");
+
+ ia_css_rgb2yuv_encode((struct sh_css_isp_csc_params *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
+ ¶ms->rgb2yuv_cc_config,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_rgb2yuv() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_r_gamma(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->vamem0.r_gamma.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->vamem0.r_gamma.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_r_gamma() enter:\n");
+
+ ia_css_r_gamma_vamem_encode((struct sh_css_isp_rgb_gamma_vamem_params *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_VAMEM0].address[offset],
+ ¶ms->r_gamma_table,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_VAMEM0] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_r_gamma() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_g_gamma(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->vamem1.g_gamma.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->vamem1.g_gamma.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_g_gamma() enter:\n");
+
+ ia_css_g_gamma_vamem_encode((struct sh_css_isp_rgb_gamma_vamem_params *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_VAMEM1].address[offset],
+ ¶ms->g_gamma_table,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_VAMEM1] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_g_gamma() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_b_gamma(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->vamem2.b_gamma.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->vamem2.b_gamma.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_b_gamma() enter:\n");
+
+ ia_css_b_gamma_vamem_encode((struct sh_css_isp_rgb_gamma_vamem_params *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_VAMEM2].address[offset],
+ ¶ms->b_gamma_table,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_VAMEM2] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_b_gamma() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_uds(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->dmem.uds.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->dmem.uds.offset;
+
+ if (size) {
+ struct sh_css_sp_uds_params *p;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_uds() enter:\n");
+
+ p = (struct sh_css_sp_uds_params *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset];
+ p->crop_pos = params->uds_config.crop_pos;
+ p->uds = params->uds_config.uds;
+
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_uds() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_raa(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->dmem.raa.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->dmem.raa.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_raa() enter:\n");
+
+ ia_css_raa_encode((struct sh_css_isp_aa_params *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
+ ¶ms->raa_config,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_raa() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_s3a(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->dmem.s3a.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->dmem.s3a.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_s3a() enter:\n");
+
+ ia_css_s3a_encode((struct sh_css_isp_s3a_params *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
+ ¶ms->s3a_config,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_s3a() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_ob(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->dmem.ob.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->dmem.ob.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_ob() enter:\n");
+
+ ia_css_ob_encode((struct sh_css_isp_ob_params *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
+ ¶ms->ob_config,
+ ¶ms->stream_configs.ob, size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_ob() leave:\n");
+ }
+ }
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->vmem.ob.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->vmem.ob.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_ob() enter:\n");
+
+ ia_css_ob_vmem_encode((struct sh_css_isp_ob_vmem_params *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_VMEM].address[offset],
+ ¶ms->ob_config,
+ ¶ms->stream_configs.ob, size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_VMEM] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_ob() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_output(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->dmem.output.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->dmem.output.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_output() enter:\n");
+
+ ia_css_output_encode((struct sh_css_isp_output_params *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
+ ¶ms->output_config,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_output() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_sc(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->dmem.sc.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->dmem.sc.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_sc() enter:\n");
+
+ ia_css_sc_encode((struct sh_css_isp_sc_params *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
+ ¶ms->sc_config,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_sc() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_bds(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->dmem.bds.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->dmem.bds.offset;
+
+ if (size) {
+ struct sh_css_isp_bds_params *p;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_bds() enter:\n");
+
+ p = (struct sh_css_isp_bds_params *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset];
+ p->baf_strength = params->bds_config.strength;
+
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_bds() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_tnr(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->dmem.tnr.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->dmem.tnr.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_tnr() enter:\n");
+
+ ia_css_tnr_encode((struct sh_css_isp_tnr_params *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
+ ¶ms->tnr_config,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_tnr() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_macc(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->dmem.macc.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->dmem.macc.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_macc() enter:\n");
+
+ ia_css_macc_encode((struct sh_css_isp_macc_params *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
+ ¶ms->macc_config,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_macc() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_sdis_horicoef(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->vmem.sdis_horicoef.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->vmem.sdis_horicoef.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_sdis_horicoef() enter:\n");
+
+ ia_css_sdis_horicoef_vmem_encode((struct sh_css_isp_sdis_hori_coef_tbl *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_VMEM].address[offset],
+ ¶ms->dvs_coefs,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_VMEM] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_sdis_horicoef() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_sdis_vertcoef(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->vmem.sdis_vertcoef.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->vmem.sdis_vertcoef.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_sdis_vertcoef() enter:\n");
+
+ ia_css_sdis_vertcoef_vmem_encode((struct sh_css_isp_sdis_vert_coef_tbl *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_VMEM].address[offset],
+ ¶ms->dvs_coefs,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_VMEM] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_sdis_vertcoef() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_sdis_horiproj(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->dmem.sdis_horiproj.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->dmem.sdis_horiproj.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_sdis_horiproj() enter:\n");
+
+ ia_css_sdis_horiproj_encode((struct sh_css_isp_sdis_hori_proj_tbl *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
+ ¶ms->dvs_coefs,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_sdis_horiproj() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_sdis_vertproj(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->dmem.sdis_vertproj.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->dmem.sdis_vertproj.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_sdis_vertproj() enter:\n");
+
+ ia_css_sdis_vertproj_encode((struct sh_css_isp_sdis_vert_proj_tbl *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
+ ¶ms->dvs_coefs,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_sdis_vertproj() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_sdis2_horicoef(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->vmem.sdis2_horicoef.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->vmem.sdis2_horicoef.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_sdis2_horicoef() enter:\n");
+
+ ia_css_sdis2_horicoef_vmem_encode((struct sh_css_isp_sdis_hori_coef_tbl *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_VMEM].address[offset],
+ ¶ms->dvs2_coefs,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_VMEM] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_sdis2_horicoef() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_sdis2_vertcoef(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->vmem.sdis2_vertcoef.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->vmem.sdis2_vertcoef.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_sdis2_vertcoef() enter:\n");
+
+ ia_css_sdis2_vertcoef_vmem_encode((struct sh_css_isp_sdis_vert_coef_tbl *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_VMEM].address[offset],
+ ¶ms->dvs2_coefs,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_VMEM] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_sdis2_vertcoef() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_sdis2_horiproj(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->dmem.sdis2_horiproj.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->dmem.sdis2_horiproj.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_sdis2_horiproj() enter:\n");
+
+ ia_css_sdis2_horiproj_encode((struct sh_css_isp_sdis_hori_proj_tbl *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
+ ¶ms->dvs2_coefs,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_sdis2_horiproj() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_sdis2_vertproj(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->dmem.sdis2_vertproj.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->dmem.sdis2_vertproj.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_sdis2_vertproj() enter:\n");
+
+ ia_css_sdis2_vertproj_encode((struct sh_css_isp_sdis_vert_proj_tbl *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
+ ¶ms->dvs2_coefs,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_sdis2_vertproj() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_wb(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->dmem.wb.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->dmem.wb.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_wb() enter:\n");
+
+ ia_css_wb_encode((struct sh_css_isp_wb_params *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
+ ¶ms->wb_config,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_wb() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_nr(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->dmem.nr.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->dmem.nr.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_nr() enter:\n");
+
+ ia_css_nr_encode((struct sh_css_isp_ynr_params *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
+ ¶ms->nr_config,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_nr() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_yee(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->dmem.yee.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->dmem.yee.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_yee() enter:\n");
+
+ ia_css_yee_encode((struct sh_css_isp_yee_params *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
+ ¶ms->yee_config,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_yee() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_ynr(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->dmem.ynr.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->dmem.ynr.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_ynr() enter:\n");
+
+ ia_css_ynr_encode((struct sh_css_isp_yee2_params *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
+ ¶ms->ynr_config,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_ynr() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_fc(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->dmem.fc.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->dmem.fc.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_fc() enter:\n");
+
+ ia_css_fc_encode((struct sh_css_isp_fc_params *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
+ ¶ms->fc_config,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE, "ia_css_process_fc() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_ctc(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->dmem.ctc.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->dmem.ctc.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_ctc() enter:\n");
+
+ ia_css_ctc_encode((struct sh_css_isp_ctc_params *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
+ ¶ms->ctc_config,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_ctc() leave:\n");
+ }
+ }
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->vamem0.ctc.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->vamem0.ctc.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_ctc() enter:\n");
+
+ ia_css_ctc_vamem_encode((struct sh_css_isp_ctc_vamem_params *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_VAMEM0].address[offset],
+ ¶ms->ctc_table,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_VAMEM0] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_ctc() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_xnr_table(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->vamem1.xnr_table.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->vamem1.xnr_table.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_xnr_table() enter:\n");
+
+ ia_css_xnr_table_vamem_encode((struct sh_css_isp_xnr_vamem_params *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_VAMEM1].address[offset],
+ ¶ms->xnr_table,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_VAMEM1] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_xnr_table() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_xnr(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->dmem.xnr.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->dmem.xnr.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_xnr() enter:\n");
+
+ ia_css_xnr_encode((struct sh_css_isp_xnr_params *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
+ ¶ms->xnr_config,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_xnr() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_process_function() */
+
+static void
+ia_css_process_xnr3(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params)
+{
+ assert(params);
+
+ {
+ unsigned int size =
+ stage->binary->info->mem_offsets.offsets.param->dmem.xnr3.size;
+
+ unsigned int offset =
+ stage->binary->info->mem_offsets.offsets.param->dmem.xnr3.offset;
+
+ if (size) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_xnr3() enter:\n");
+
+ ia_css_xnr3_encode((struct sh_css_isp_xnr3_params *)
+ &stage->binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
+ ¶ms->xnr3_config,
+ size);
+ params->isp_params_changed = true;
+ params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
+ true;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_process_xnr3() leave:\n");
+ }
+ }
+}
+
+/* Code generated by genparam/gencode.c:gen_param_process_table() */
+
+void (*ia_css_kernel_process_param[IA_CSS_NUM_PARAMETER_IDS])(
+ unsigned int pipe_id,
+ const struct ia_css_pipeline_stage *stage,
+ struct ia_css_isp_parameters *params) = {
+ ia_css_process_aa,
+ ia_css_process_anr,
+ ia_css_process_anr2,
+ ia_css_process_bh,
+ ia_css_process_cnr,
+ ia_css_process_crop,
+ ia_css_process_csc,
+ ia_css_process_dp,
+ ia_css_process_bnr,
+ ia_css_process_de,
+ ia_css_process_ecd,
+ ia_css_process_formats,
+ ia_css_process_fpn,
+ ia_css_process_gc,
+ ia_css_process_ce,
+ ia_css_process_yuv2rgb,
+ ia_css_process_rgb2yuv,
+ ia_css_process_r_gamma,
+ ia_css_process_g_gamma,
+ ia_css_process_b_gamma,
+ ia_css_process_uds,
+ ia_css_process_raa,
+ ia_css_process_s3a,
+ ia_css_process_ob,
+ ia_css_process_output,
+ ia_css_process_sc,
+ ia_css_process_bds,
+ ia_css_process_tnr,
+ ia_css_process_macc,
+ ia_css_process_sdis_horicoef,
+ ia_css_process_sdis_vertcoef,
+ ia_css_process_sdis_horiproj,
+ ia_css_process_sdis_vertproj,
+ ia_css_process_sdis2_horicoef,
+ ia_css_process_sdis2_vertcoef,
+ ia_css_process_sdis2_horiproj,
+ ia_css_process_sdis2_vertproj,
+ ia_css_process_wb,
+ ia_css_process_nr,
+ ia_css_process_yee,
+ ia_css_process_ynr,
+ ia_css_process_fc,
+ ia_css_process_ctc,
+ ia_css_process_xnr_table,
+ ia_css_process_xnr,
+ ia_css_process_xnr3,
+};
+
+/* Code generated by genparam/gencode.c:gen_get_function() */
+
+static void
+ia_css_get_dp_config(const struct ia_css_isp_parameters *params,
+ struct ia_css_dp_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_dp_config() enter: config=%p\n",
+ config);
+
+ *config = params->dp_config;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_dp_config() leave\n");
+ ia_css_dp_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+}
+
+/* Code generated by genparam/gencode.c:gen_set_function() */
+
+void
+ia_css_set_dp_config(struct ia_css_isp_parameters *params,
+ const struct ia_css_dp_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_dp_config() enter:\n");
+ ia_css_dp_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+ params->dp_config = *config;
+ params->config_changed[IA_CSS_DP_ID] = true;
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_set_dp_config() leave: return_void\n");
+}
+
+/* Code generated by genparam/gencode.c:gen_get_function() */
+
+static void
+ia_css_get_wb_config(const struct ia_css_isp_parameters *params,
+ struct ia_css_wb_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_wb_config() enter: config=%p\n",
+ config);
+
+ *config = params->wb_config;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_wb_config() leave\n");
+ ia_css_wb_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+}
+
+/* Code generated by genparam/gencode.c:gen_set_function() */
+
+void
+ia_css_set_wb_config(struct ia_css_isp_parameters *params,
+ const struct ia_css_wb_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_wb_config() enter:\n");
+ ia_css_wb_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+ params->wb_config = *config;
+ params->config_changed[IA_CSS_WB_ID] = true;
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_set_wb_config() leave: return_void\n");
+}
+
+/* Code generated by genparam/gencode.c:gen_get_function() */
+
+static void
+ia_css_get_tnr_config(const struct ia_css_isp_parameters *params,
+ struct ia_css_tnr_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_tnr_config() enter: config=%p\n",
+ config);
+
+ *config = params->tnr_config;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_tnr_config() leave\n");
+ ia_css_tnr_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+}
+
+/* Code generated by genparam/gencode.c:gen_set_function() */
+
+void
+ia_css_set_tnr_config(struct ia_css_isp_parameters *params,
+ const struct ia_css_tnr_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_tnr_config() enter:\n");
+ ia_css_tnr_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+ params->tnr_config = *config;
+ params->config_changed[IA_CSS_TNR_ID] = true;
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_set_tnr_config() leave: return_void\n");
+}
+
+/* Code generated by genparam/gencode.c:gen_get_function() */
+
+static void
+ia_css_get_ob_config(const struct ia_css_isp_parameters *params,
+ struct ia_css_ob_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_ob_config() enter: config=%p\n",
+ config);
+
+ *config = params->ob_config;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_ob_config() leave\n");
+ ia_css_ob_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+}
+
+/* Code generated by genparam/gencode.c:gen_set_function() */
+
+void
+ia_css_set_ob_config(struct ia_css_isp_parameters *params,
+ const struct ia_css_ob_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_ob_config() enter:\n");
+ ia_css_ob_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+ params->ob_config = *config;
+ params->config_changed[IA_CSS_OB_ID] = true;
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_set_ob_config() leave: return_void\n");
+}
+
+/* Code generated by genparam/gencode.c:gen_get_function() */
+
+static void
+ia_css_get_de_config(const struct ia_css_isp_parameters *params,
+ struct ia_css_de_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_de_config() enter: config=%p\n",
+ config);
+
+ *config = params->de_config;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_de_config() leave\n");
+ ia_css_de_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+}
+
+/* Code generated by genparam/gencode.c:gen_set_function() */
+
+void
+ia_css_set_de_config(struct ia_css_isp_parameters *params,
+ const struct ia_css_de_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_de_config() enter:\n");
+ ia_css_de_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+ params->de_config = *config;
+ params->config_changed[IA_CSS_DE_ID] = true;
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_set_de_config() leave: return_void\n");
+}
+
+/* Code generated by genparam/gencode.c:gen_get_function() */
+
+static void
+ia_css_get_anr_config(const struct ia_css_isp_parameters *params,
+ struct ia_css_anr_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_anr_config() enter: config=%p\n",
+ config);
+
+ *config = params->anr_config;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_anr_config() leave\n");
+ ia_css_anr_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+}
+
+/* Code generated by genparam/gencode.c:gen_set_function() */
+
+void
+ia_css_set_anr_config(struct ia_css_isp_parameters *params,
+ const struct ia_css_anr_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_anr_config() enter:\n");
+ ia_css_anr_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+ params->anr_config = *config;
+ params->config_changed[IA_CSS_ANR_ID] = true;
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_set_anr_config() leave: return_void\n");
+}
+
+/* Code generated by genparam/gencode.c:gen_get_function() */
+
+static void
+ia_css_get_anr2_config(const struct ia_css_isp_parameters *params,
+ struct ia_css_anr_thres *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_anr2_config() enter: config=%p\n",
+ config);
+
+ *config = params->anr_thres;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_anr2_config() leave\n");
+ ia_css_anr2_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+}
+
+/* Code generated by genparam/gencode.c:gen_set_function() */
+
+void
+ia_css_set_anr2_config(struct ia_css_isp_parameters *params,
+ const struct ia_css_anr_thres *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_anr2_config() enter:\n");
+ ia_css_anr2_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+ params->anr_thres = *config;
+ params->config_changed[IA_CSS_ANR2_ID] = true;
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_set_anr2_config() leave: return_void\n");
+}
+
+/* Code generated by genparam/gencode.c:gen_get_function() */
+
+static void
+ia_css_get_ce_config(const struct ia_css_isp_parameters *params,
+ struct ia_css_ce_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_ce_config() enter: config=%p\n",
+ config);
+
+ *config = params->ce_config;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_ce_config() leave\n");
+ ia_css_ce_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+}
+
+/* Code generated by genparam/gencode.c:gen_set_function() */
+
+void
+ia_css_set_ce_config(struct ia_css_isp_parameters *params,
+ const struct ia_css_ce_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_ce_config() enter:\n");
+ ia_css_ce_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+ params->ce_config = *config;
+ params->config_changed[IA_CSS_CE_ID] = true;
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_set_ce_config() leave: return_void\n");
+}
+
+/* Code generated by genparam/gencode.c:gen_get_function() */
+
+static void
+ia_css_get_ecd_config(const struct ia_css_isp_parameters *params,
+ struct ia_css_ecd_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_ecd_config() enter: config=%p\n",
+ config);
+
+ *config = params->ecd_config;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_ecd_config() leave\n");
+ ia_css_ecd_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+}
+
+/* Code generated by genparam/gencode.c:gen_set_function() */
+
+void
+ia_css_set_ecd_config(struct ia_css_isp_parameters *params,
+ const struct ia_css_ecd_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_ecd_config() enter:\n");
+ ia_css_ecd_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+ params->ecd_config = *config;
+ params->config_changed[IA_CSS_ECD_ID] = true;
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_set_ecd_config() leave: return_void\n");
+}
+
+/* Code generated by genparam/gencode.c:gen_get_function() */
+
+static void
+ia_css_get_ynr_config(const struct ia_css_isp_parameters *params,
+ struct ia_css_ynr_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_ynr_config() enter: config=%p\n",
+ config);
+
+ *config = params->ynr_config;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_ynr_config() leave\n");
+ ia_css_ynr_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+}
+
+/* Code generated by genparam/gencode.c:gen_set_function() */
+
+void
+ia_css_set_ynr_config(struct ia_css_isp_parameters *params,
+ const struct ia_css_ynr_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_ynr_config() enter:\n");
+ ia_css_ynr_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+ params->ynr_config = *config;
+ params->config_changed[IA_CSS_YNR_ID] = true;
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_set_ynr_config() leave: return_void\n");
+}
+
+/* Code generated by genparam/gencode.c:gen_get_function() */
+
+static void
+ia_css_get_fc_config(const struct ia_css_isp_parameters *params,
+ struct ia_css_fc_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_fc_config() enter: config=%p\n",
+ config);
+
+ *config = params->fc_config;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_fc_config() leave\n");
+ ia_css_fc_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+}
+
+/* Code generated by genparam/gencode.c:gen_set_function() */
+
+void
+ia_css_set_fc_config(struct ia_css_isp_parameters *params,
+ const struct ia_css_fc_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_fc_config() enter:\n");
+ ia_css_fc_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+ params->fc_config = *config;
+ params->config_changed[IA_CSS_FC_ID] = true;
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_set_fc_config() leave: return_void\n");
+}
+
+/* Code generated by genparam/gencode.c:gen_get_function() */
+
+static void
+ia_css_get_cnr_config(const struct ia_css_isp_parameters *params,
+ struct ia_css_cnr_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_cnr_config() enter: config=%p\n",
+ config);
+
+ *config = params->cnr_config;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_cnr_config() leave\n");
+ ia_css_cnr_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+}
+
+/* Code generated by genparam/gencode.c:gen_set_function() */
+
+void
+ia_css_set_cnr_config(struct ia_css_isp_parameters *params,
+ const struct ia_css_cnr_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_cnr_config() enter:\n");
+ ia_css_cnr_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+ params->cnr_config = *config;
+ params->config_changed[IA_CSS_CNR_ID] = true;
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_set_cnr_config() leave: return_void\n");
+}
+
+/* Code generated by genparam/gencode.c:gen_get_function() */
+
+static void
+ia_css_get_macc_config(const struct ia_css_isp_parameters *params,
+ struct ia_css_macc_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_macc_config() enter: config=%p\n",
+ config);
+
+ *config = params->macc_config;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_macc_config() leave\n");
+ ia_css_macc_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+}
+
+/* Code generated by genparam/gencode.c:gen_set_function() */
+
+void
+ia_css_set_macc_config(struct ia_css_isp_parameters *params,
+ const struct ia_css_macc_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_macc_config() enter:\n");
+ ia_css_macc_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+ params->macc_config = *config;
+ params->config_changed[IA_CSS_MACC_ID] = true;
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_set_macc_config() leave: return_void\n");
+}
+
+/* Code generated by genparam/gencode.c:gen_get_function() */
+
+static void
+ia_css_get_ctc_config(const struct ia_css_isp_parameters *params,
+ struct ia_css_ctc_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_ctc_config() enter: config=%p\n",
+ config);
+
+ *config = params->ctc_config;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_ctc_config() leave\n");
+ ia_css_ctc_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+}
+
+/* Code generated by genparam/gencode.c:gen_set_function() */
+
+void
+ia_css_set_ctc_config(struct ia_css_isp_parameters *params,
+ const struct ia_css_ctc_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_ctc_config() enter:\n");
+ ia_css_ctc_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+ params->ctc_config = *config;
+ params->config_changed[IA_CSS_CTC_ID] = true;
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_set_ctc_config() leave: return_void\n");
+}
+
+/* Code generated by genparam/gencode.c:gen_get_function() */
+
+static void
+ia_css_get_aa_config(const struct ia_css_isp_parameters *params,
+ struct ia_css_aa_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_aa_config() enter: config=%p\n",
+ config);
+
+ *config = params->aa_config;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_aa_config() leave\n");
+}
+
+/* Code generated by genparam/gencode.c:gen_set_function() */
+
+void
+ia_css_set_aa_config(struct ia_css_isp_parameters *params,
+ const struct ia_css_aa_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_aa_config() enter:\n");
+ params->aa_config = *config;
+ params->config_changed[IA_CSS_AA_ID] = true;
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_set_aa_config() leave: return_void\n");
+}
+
+/* Code generated by genparam/gencode.c:gen_get_function() */
+
+static void
+ia_css_get_yuv2rgb_config(const struct ia_css_isp_parameters *params,
+ struct ia_css_cc_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_yuv2rgb_config() enter: config=%p\n",
+ config);
+
+ *config = params->yuv2rgb_cc_config;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_yuv2rgb_config() leave\n");
+ ia_css_yuv2rgb_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+}
+
+/* Code generated by genparam/gencode.c:gen_set_function() */
+
+void
+ia_css_set_yuv2rgb_config(struct ia_css_isp_parameters *params,
+ const struct ia_css_cc_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_yuv2rgb_config() enter:\n");
+ ia_css_yuv2rgb_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+ params->yuv2rgb_cc_config = *config;
+ params->config_changed[IA_CSS_YUV2RGB_ID] = true;
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_set_yuv2rgb_config() leave: return_void\n");
+}
+
+/* Code generated by genparam/gencode.c:gen_get_function() */
+
+static void
+ia_css_get_rgb2yuv_config(const struct ia_css_isp_parameters *params,
+ struct ia_css_cc_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_rgb2yuv_config() enter: config=%p\n",
+ config);
+
+ *config = params->rgb2yuv_cc_config;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_rgb2yuv_config() leave\n");
+ ia_css_rgb2yuv_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+}
+
+/* Code generated by genparam/gencode.c:gen_set_function() */
+
+void
+ia_css_set_rgb2yuv_config(struct ia_css_isp_parameters *params,
+ const struct ia_css_cc_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_rgb2yuv_config() enter:\n");
+ ia_css_rgb2yuv_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+ params->rgb2yuv_cc_config = *config;
+ params->config_changed[IA_CSS_RGB2YUV_ID] = true;
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_set_rgb2yuv_config() leave: return_void\n");
+}
+
+/* Code generated by genparam/gencode.c:gen_get_function() */
+
+static void
+ia_css_get_csc_config(const struct ia_css_isp_parameters *params,
+ struct ia_css_cc_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_csc_config() enter: config=%p\n",
+ config);
+
+ *config = params->cc_config;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_csc_config() leave\n");
+ ia_css_csc_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+}
+
+/* Code generated by genparam/gencode.c:gen_set_function() */
+
+void
+ia_css_set_csc_config(struct ia_css_isp_parameters *params,
+ const struct ia_css_cc_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_csc_config() enter:\n");
+ ia_css_csc_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+ params->cc_config = *config;
+ params->config_changed[IA_CSS_CSC_ID] = true;
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_set_csc_config() leave: return_void\n");
+}
+
+/* Code generated by genparam/gencode.c:gen_get_function() */
+
+static void
+ia_css_get_nr_config(const struct ia_css_isp_parameters *params,
+ struct ia_css_nr_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_nr_config() enter: config=%p\n",
+ config);
+
+ *config = params->nr_config;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_nr_config() leave\n");
+ ia_css_nr_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+}
+
+/* Code generated by genparam/gencode.c:gen_set_function() */
+
+void
+ia_css_set_nr_config(struct ia_css_isp_parameters *params,
+ const struct ia_css_nr_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_nr_config() enter:\n");
+ ia_css_nr_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+ params->nr_config = *config;
+ params->config_changed[IA_CSS_BNR_ID] = true;
+ params->config_changed[IA_CSS_NR_ID] = true;
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_set_nr_config() leave: return_void\n");
+}
+
+/* Code generated by genparam/gencode.c:gen_get_function() */
+
+static void
+ia_css_get_gc_config(const struct ia_css_isp_parameters *params,
+ struct ia_css_gc_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_gc_config() enter: config=%p\n",
+ config);
+
+ *config = params->gc_config;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_gc_config() leave\n");
+ ia_css_gc_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+}
+
+/* Code generated by genparam/gencode.c:gen_set_function() */
+
+void
+ia_css_set_gc_config(struct ia_css_isp_parameters *params,
+ const struct ia_css_gc_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_gc_config() enter:\n");
+ ia_css_gc_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+ params->gc_config = *config;
+ params->config_changed[IA_CSS_GC_ID] = true;
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_set_gc_config() leave: return_void\n");
+}
+
+/* Code generated by genparam/gencode.c:gen_get_function() */
+
+static void
+ia_css_get_sdis_horicoef_config(const struct ia_css_isp_parameters *params,
+ struct ia_css_dvs_coefficients *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_sdis_horicoef_config() enter: config=%p\n",
+ config);
+
+ *config = params->dvs_coefs;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_sdis_horicoef_config() leave\n");
+ ia_css_sdis_horicoef_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+}
+
+/* Code generated by genparam/gencode.c:gen_set_function() */
+
+void
+ia_css_set_sdis_horicoef_config(struct ia_css_isp_parameters *params,
+ const struct ia_css_dvs_coefficients *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE,
+ "ia_css_set_sdis_horicoef_config() enter:\n");
+ ia_css_sdis_horicoef_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+ params->dvs_coefs = *config;
+ params->config_changed[IA_CSS_SDIS_HORICOEF_ID] = true;
+ params->config_changed[IA_CSS_SDIS_VERTCOEF_ID] = true;
+ params->config_changed[IA_CSS_SDIS_HORIPROJ_ID] = true;
+ params->config_changed[IA_CSS_SDIS_VERTPROJ_ID] = true;
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_set_sdis_horicoef_config() leave: return_void\n");
+}
+
+/* Code generated by genparam/gencode.c:gen_get_function() */
+
+static void
+ia_css_get_sdis_vertcoef_config(const struct ia_css_isp_parameters *params,
+ struct ia_css_dvs_coefficients *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_sdis_vertcoef_config() enter: config=%p\n",
+ config);
+
+ *config = params->dvs_coefs;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_sdis_vertcoef_config() leave\n");
+ ia_css_sdis_vertcoef_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+}
+
+/* Code generated by genparam/gencode.c:gen_set_function() */
+
+void
+ia_css_set_sdis_vertcoef_config(struct ia_css_isp_parameters *params,
+ const struct ia_css_dvs_coefficients *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE,
+ "ia_css_set_sdis_vertcoef_config() enter:\n");
+ ia_css_sdis_vertcoef_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+ params->dvs_coefs = *config;
+ params->config_changed[IA_CSS_SDIS_HORICOEF_ID] = true;
+ params->config_changed[IA_CSS_SDIS_VERTCOEF_ID] = true;
+ params->config_changed[IA_CSS_SDIS_HORIPROJ_ID] = true;
+ params->config_changed[IA_CSS_SDIS_VERTPROJ_ID] = true;
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_set_sdis_vertcoef_config() leave: return_void\n");
+}
+
+/* Code generated by genparam/gencode.c:gen_get_function() */
+
+static void
+ia_css_get_sdis_horiproj_config(const struct ia_css_isp_parameters *params,
+ struct ia_css_dvs_coefficients *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_sdis_horiproj_config() enter: config=%p\n",
+ config);
+
+ *config = params->dvs_coefs;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_sdis_horiproj_config() leave\n");
+ ia_css_sdis_horiproj_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+}
+
+/* Code generated by genparam/gencode.c:gen_set_function() */
+
+void
+ia_css_set_sdis_horiproj_config(struct ia_css_isp_parameters *params,
+ const struct ia_css_dvs_coefficients *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE,
+ "ia_css_set_sdis_horiproj_config() enter:\n");
+ ia_css_sdis_horiproj_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+ params->dvs_coefs = *config;
+ params->config_changed[IA_CSS_SDIS_HORICOEF_ID] = true;
+ params->config_changed[IA_CSS_SDIS_VERTCOEF_ID] = true;
+ params->config_changed[IA_CSS_SDIS_HORIPROJ_ID] = true;
+ params->config_changed[IA_CSS_SDIS_VERTPROJ_ID] = true;
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_set_sdis_horiproj_config() leave: return_void\n");
+}
+
+/* Code generated by genparam/gencode.c:gen_get_function() */
+
+static void
+ia_css_get_sdis_vertproj_config(const struct ia_css_isp_parameters *params,
+ struct ia_css_dvs_coefficients *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_sdis_vertproj_config() enter: config=%p\n",
+ config);
+
+ *config = params->dvs_coefs;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_sdis_vertproj_config() leave\n");
+ ia_css_sdis_vertproj_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+}
+
+/* Code generated by genparam/gencode.c:gen_set_function() */
+
+void
+ia_css_set_sdis_vertproj_config(struct ia_css_isp_parameters *params,
+ const struct ia_css_dvs_coefficients *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE,
+ "ia_css_set_sdis_vertproj_config() enter:\n");
+ ia_css_sdis_vertproj_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+ params->dvs_coefs = *config;
+ params->config_changed[IA_CSS_SDIS_HORICOEF_ID] = true;
+ params->config_changed[IA_CSS_SDIS_VERTCOEF_ID] = true;
+ params->config_changed[IA_CSS_SDIS_HORIPROJ_ID] = true;
+ params->config_changed[IA_CSS_SDIS_VERTPROJ_ID] = true;
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_set_sdis_vertproj_config() leave: return_void\n");
+}
+
+/* Code generated by genparam/gencode.c:gen_get_function() */
+
+static void
+ia_css_get_sdis2_horicoef_config(const struct ia_css_isp_parameters *params,
+ struct ia_css_dvs2_coefficients *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_sdis2_horicoef_config() enter: config=%p\n",
+ config);
+
+ *config = params->dvs2_coefs;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_sdis2_horicoef_config() leave\n");
+ ia_css_sdis2_horicoef_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+}
+
+/* Code generated by genparam/gencode.c:gen_set_function() */
+
+void
+ia_css_set_sdis2_horicoef_config(struct ia_css_isp_parameters *params,
+ const struct ia_css_dvs2_coefficients *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE,
+ "ia_css_set_sdis2_horicoef_config() enter:\n");
+ ia_css_sdis2_horicoef_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+ params->dvs2_coefs = *config;
+ params->config_changed[IA_CSS_SDIS2_HORICOEF_ID] = true;
+ params->config_changed[IA_CSS_SDIS2_VERTCOEF_ID] = true;
+ params->config_changed[IA_CSS_SDIS2_HORIPROJ_ID] = true;
+ params->config_changed[IA_CSS_SDIS2_VERTPROJ_ID] = true;
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_set_sdis2_horicoef_config() leave: return_void\n");
+}
+
+/* Code generated by genparam/gencode.c:gen_get_function() */
+
+static void
+ia_css_get_sdis2_vertcoef_config(const struct ia_css_isp_parameters *params,
+ struct ia_css_dvs2_coefficients *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_sdis2_vertcoef_config() enter: config=%p\n",
+ config);
+
+ *config = params->dvs2_coefs;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_sdis2_vertcoef_config() leave\n");
+ ia_css_sdis2_vertcoef_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+}
+
+/* Code generated by genparam/gencode.c:gen_set_function() */
+
+void
+ia_css_set_sdis2_vertcoef_config(struct ia_css_isp_parameters *params,
+ const struct ia_css_dvs2_coefficients *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE,
+ "ia_css_set_sdis2_vertcoef_config() enter:\n");
+ ia_css_sdis2_vertcoef_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+ params->dvs2_coefs = *config;
+ params->config_changed[IA_CSS_SDIS2_HORICOEF_ID] = true;
+ params->config_changed[IA_CSS_SDIS2_VERTCOEF_ID] = true;
+ params->config_changed[IA_CSS_SDIS2_HORIPROJ_ID] = true;
+ params->config_changed[IA_CSS_SDIS2_VERTPROJ_ID] = true;
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_set_sdis2_vertcoef_config() leave: return_void\n");
+}
+
+/* Code generated by genparam/gencode.c:gen_get_function() */
+
+static void
+ia_css_get_sdis2_horiproj_config(const struct ia_css_isp_parameters *params,
+ struct ia_css_dvs2_coefficients *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_sdis2_horiproj_config() enter: config=%p\n",
+ config);
+
+ *config = params->dvs2_coefs;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_sdis2_horiproj_config() leave\n");
+ ia_css_sdis2_horiproj_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+}
+
+/* Code generated by genparam/gencode.c:gen_set_function() */
+
+void
+ia_css_set_sdis2_horiproj_config(struct ia_css_isp_parameters *params,
+ const struct ia_css_dvs2_coefficients *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE,
+ "ia_css_set_sdis2_horiproj_config() enter:\n");
+ ia_css_sdis2_horiproj_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+ params->dvs2_coefs = *config;
+ params->config_changed[IA_CSS_SDIS2_HORICOEF_ID] = true;
+ params->config_changed[IA_CSS_SDIS2_VERTCOEF_ID] = true;
+ params->config_changed[IA_CSS_SDIS2_HORIPROJ_ID] = true;
+ params->config_changed[IA_CSS_SDIS2_VERTPROJ_ID] = true;
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_set_sdis2_horiproj_config() leave: return_void\n");
+}
+
+/* Code generated by genparam/gencode.c:gen_get_function() */
+
+static void
+ia_css_get_sdis2_vertproj_config(const struct ia_css_isp_parameters *params,
+ struct ia_css_dvs2_coefficients *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_sdis2_vertproj_config() enter: config=%p\n",
+ config);
+
+ *config = params->dvs2_coefs;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_sdis2_vertproj_config() leave\n");
+ ia_css_sdis2_vertproj_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+}
+
+/* Code generated by genparam/gencode.c:gen_set_function() */
+
+void
+ia_css_set_sdis2_vertproj_config(struct ia_css_isp_parameters *params,
+ const struct ia_css_dvs2_coefficients *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE,
+ "ia_css_set_sdis2_vertproj_config() enter:\n");
+ ia_css_sdis2_vertproj_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+ params->dvs2_coefs = *config;
+ params->config_changed[IA_CSS_SDIS2_HORICOEF_ID] = true;
+ params->config_changed[IA_CSS_SDIS2_VERTCOEF_ID] = true;
+ params->config_changed[IA_CSS_SDIS2_HORIPROJ_ID] = true;
+ params->config_changed[IA_CSS_SDIS2_VERTPROJ_ID] = true;
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_set_sdis2_vertproj_config() leave: return_void\n");
+}
+
+/* Code generated by genparam/gencode.c:gen_get_function() */
+
+static void
+ia_css_get_r_gamma_config(const struct ia_css_isp_parameters *params,
+ struct ia_css_rgb_gamma_table *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_r_gamma_config() enter: config=%p\n",
+ config);
+
+ *config = params->r_gamma_table;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_r_gamma_config() leave\n");
+ ia_css_r_gamma_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+}
+
+/* Code generated by genparam/gencode.c:gen_set_function() */
+
+void
+ia_css_set_r_gamma_config(struct ia_css_isp_parameters *params,
+ const struct ia_css_rgb_gamma_table *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_r_gamma_config() enter:\n");
+ ia_css_r_gamma_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+ params->r_gamma_table = *config;
+ params->config_changed[IA_CSS_R_GAMMA_ID] = true;
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_set_r_gamma_config() leave: return_void\n");
+}
+
+/* Code generated by genparam/gencode.c:gen_get_function() */
+
+static void
+ia_css_get_g_gamma_config(const struct ia_css_isp_parameters *params,
+ struct ia_css_rgb_gamma_table *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_g_gamma_config() enter: config=%p\n",
+ config);
+
+ *config = params->g_gamma_table;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_g_gamma_config() leave\n");
+ ia_css_g_gamma_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+}
+
+/* Code generated by genparam/gencode.c:gen_set_function() */
+
+void
+ia_css_set_g_gamma_config(struct ia_css_isp_parameters *params,
+ const struct ia_css_rgb_gamma_table *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_g_gamma_config() enter:\n");
+ ia_css_g_gamma_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+ params->g_gamma_table = *config;
+ params->config_changed[IA_CSS_G_GAMMA_ID] = true;
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_set_g_gamma_config() leave: return_void\n");
+}
+
+/* Code generated by genparam/gencode.c:gen_get_function() */
+
+static void
+ia_css_get_b_gamma_config(const struct ia_css_isp_parameters *params,
+ struct ia_css_rgb_gamma_table *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_b_gamma_config() enter: config=%p\n",
+ config);
+
+ *config = params->b_gamma_table;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_b_gamma_config() leave\n");
+ ia_css_b_gamma_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+}
+
+/* Code generated by genparam/gencode.c:gen_set_function() */
+
+void
+ia_css_set_b_gamma_config(struct ia_css_isp_parameters *params,
+ const struct ia_css_rgb_gamma_table *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_b_gamma_config() enter:\n");
+ ia_css_b_gamma_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+ params->b_gamma_table = *config;
+ params->config_changed[IA_CSS_B_GAMMA_ID] = true;
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_set_b_gamma_config() leave: return_void\n");
+}
+
+/* Code generated by genparam/gencode.c:gen_get_function() */
+
+static void
+ia_css_get_xnr_table_config(const struct ia_css_isp_parameters *params,
+ struct ia_css_xnr_table *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_xnr_table_config() enter: config=%p\n",
+ config);
+
+ *config = params->xnr_table;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_xnr_table_config() leave\n");
+ ia_css_xnr_table_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+}
+
+/* Code generated by genparam/gencode.c:gen_set_function() */
+
+void
+ia_css_set_xnr_table_config(struct ia_css_isp_parameters *params,
+ const struct ia_css_xnr_table *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE,
+ "ia_css_set_xnr_table_config() enter:\n");
+ ia_css_xnr_table_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+ params->xnr_table = *config;
+ params->config_changed[IA_CSS_XNR_TABLE_ID] = true;
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_set_xnr_table_config() leave: return_void\n");
+}
+
+/* Code generated by genparam/gencode.c:gen_get_function() */
+
+static void
+ia_css_get_formats_config(const struct ia_css_isp_parameters *params,
+ struct ia_css_formats_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_formats_config() enter: config=%p\n",
+ config);
+
+ *config = params->formats_config;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_formats_config() leave\n");
+ ia_css_formats_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+}
+
+/* Code generated by genparam/gencode.c:gen_set_function() */
+
+void
+ia_css_set_formats_config(struct ia_css_isp_parameters *params,
+ const struct ia_css_formats_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_formats_config() enter:\n");
+ ia_css_formats_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+ params->formats_config = *config;
+ params->config_changed[IA_CSS_FORMATS_ID] = true;
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_set_formats_config() leave: return_void\n");
+}
+
+/* Code generated by genparam/gencode.c:gen_get_function() */
+
+static void
+ia_css_get_xnr_config(const struct ia_css_isp_parameters *params,
+ struct ia_css_xnr_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_xnr_config() enter: config=%p\n",
+ config);
+
+ *config = params->xnr_config;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_xnr_config() leave\n");
+ ia_css_xnr_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+}
+
+/* Code generated by genparam/gencode.c:gen_set_function() */
+
+void
+ia_css_set_xnr_config(struct ia_css_isp_parameters *params,
+ const struct ia_css_xnr_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_xnr_config() enter:\n");
+ ia_css_xnr_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+ params->xnr_config = *config;
+ params->config_changed[IA_CSS_XNR_ID] = true;
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_set_xnr_config() leave: return_void\n");
+}
+
+/* Code generated by genparam/gencode.c:gen_get_function() */
+
+static void
+ia_css_get_xnr3_config(const struct ia_css_isp_parameters *params,
+ struct ia_css_xnr3_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_xnr3_config() enter: config=%p\n",
+ config);
+
+ *config = params->xnr3_config;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_xnr3_config() leave\n");
+ ia_css_xnr3_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+}
+
+/* Code generated by genparam/gencode.c:gen_set_function() */
+
+void
+ia_css_set_xnr3_config(struct ia_css_isp_parameters *params,
+ const struct ia_css_xnr3_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_xnr3_config() enter:\n");
+ ia_css_xnr3_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+ params->xnr3_config = *config;
+ params->config_changed[IA_CSS_XNR3_ID] = true;
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_set_xnr3_config() leave: return_void\n");
+}
+
+/* Code generated by genparam/gencode.c:gen_get_function() */
+
+static void
+ia_css_get_s3a_config(const struct ia_css_isp_parameters *params,
+ struct ia_css_3a_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_s3a_config() enter: config=%p\n",
+ config);
+
+ *config = params->s3a_config;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_s3a_config() leave\n");
+ ia_css_s3a_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+}
+
+/* Code generated by genparam/gencode.c:gen_set_function() */
+
+void
+ia_css_set_s3a_config(struct ia_css_isp_parameters *params,
+ const struct ia_css_3a_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_s3a_config() enter:\n");
+ ia_css_s3a_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+ params->s3a_config = *config;
+ params->config_changed[IA_CSS_BH_ID] = true;
+ params->config_changed[IA_CSS_S3A_ID] = true;
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_set_s3a_config() leave: return_void\n");
+}
+
+/* Code generated by genparam/gencode.c:gen_get_function() */
+
+static void
+ia_css_get_output_config(const struct ia_css_isp_parameters *params,
+ struct ia_css_output_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_output_config() enter: config=%p\n",
+ config);
+
+ *config = params->output_config;
+
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_get_output_config() leave\n");
+ ia_css_output_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+}
+
+/* Code generated by genparam/gencode.c:gen_set_function() */
+
+void
+ia_css_set_output_config(struct ia_css_isp_parameters *params,
+ const struct ia_css_output_config *config)
+{
+ if (!config)
+ return;
+
+ assert(params);
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_set_output_config() enter:\n");
+ ia_css_output_debug_dtrace(config, IA_CSS_DEBUG_TRACE);
+ params->output_config = *config;
+ params->config_changed[IA_CSS_OUTPUT_ID] = true;
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "ia_css_set_output_config() leave: return_void\n");
+}
+
+/* Code generated by genparam/gencode.c:gen_global_access_function() */
+
+void
+ia_css_get_configs(struct ia_css_isp_parameters *params,
+ const struct ia_css_isp_config *config)
+{
+ ia_css_get_dp_config(params, config->dp_config);
+ ia_css_get_wb_config(params, config->wb_config);
+ ia_css_get_tnr_config(params, config->tnr_config);
+ ia_css_get_ob_config(params, config->ob_config);
+ ia_css_get_de_config(params, config->de_config);
+ ia_css_get_anr_config(params, config->anr_config);
+ ia_css_get_anr2_config(params, config->anr_thres);
+ ia_css_get_ce_config(params, config->ce_config);
+ ia_css_get_ecd_config(params, config->ecd_config);
+ ia_css_get_ynr_config(params, config->ynr_config);
+ ia_css_get_fc_config(params, config->fc_config);
+ ia_css_get_cnr_config(params, config->cnr_config);
+ ia_css_get_macc_config(params, config->macc_config);
+ ia_css_get_ctc_config(params, config->ctc_config);
+ ia_css_get_aa_config(params, config->aa_config);
+ ia_css_get_yuv2rgb_config(params, config->yuv2rgb_cc_config);
+ ia_css_get_rgb2yuv_config(params, config->rgb2yuv_cc_config);
+ ia_css_get_csc_config(params, config->cc_config);
+ ia_css_get_nr_config(params, config->nr_config);
+ ia_css_get_gc_config(params, config->gc_config);
+ ia_css_get_sdis_horicoef_config(params, config->dvs_coefs);
+ ia_css_get_sdis_vertcoef_config(params, config->dvs_coefs);
+ ia_css_get_sdis_horiproj_config(params, config->dvs_coefs);
+ ia_css_get_sdis_vertproj_config(params, config->dvs_coefs);
+ ia_css_get_sdis2_horicoef_config(params, config->dvs2_coefs);
+ ia_css_get_sdis2_vertcoef_config(params, config->dvs2_coefs);
+ ia_css_get_sdis2_horiproj_config(params, config->dvs2_coefs);
+ ia_css_get_sdis2_vertproj_config(params, config->dvs2_coefs);
+ ia_css_get_r_gamma_config(params, config->r_gamma_table);
+ ia_css_get_g_gamma_config(params, config->g_gamma_table);
+ ia_css_get_b_gamma_config(params, config->b_gamma_table);
+ ia_css_get_xnr_table_config(params, config->xnr_table);
+ ia_css_get_formats_config(params, config->formats_config);
+ ia_css_get_xnr_config(params, config->xnr_config);
+ ia_css_get_xnr3_config(params, config->xnr3_config);
+ ia_css_get_s3a_config(params, config->s3a_config);
+ ia_css_get_output_config(params, config->output_config);
+}
+
+/* Code generated by genparam/gencode.c:gen_global_access_function() */
+
+void
+ia_css_set_configs(struct ia_css_isp_parameters *params,
+ const struct ia_css_isp_config *config)
+{
+ ia_css_set_dp_config(params, config->dp_config);
+ ia_css_set_wb_config(params, config->wb_config);
+ ia_css_set_tnr_config(params, config->tnr_config);
+ ia_css_set_ob_config(params, config->ob_config);
+ ia_css_set_de_config(params, config->de_config);
+ ia_css_set_anr_config(params, config->anr_config);
+ ia_css_set_anr2_config(params, config->anr_thres);
+ ia_css_set_ce_config(params, config->ce_config);
+ ia_css_set_ecd_config(params, config->ecd_config);
+ ia_css_set_ynr_config(params, config->ynr_config);
+ ia_css_set_fc_config(params, config->fc_config);
+ ia_css_set_cnr_config(params, config->cnr_config);
+ ia_css_set_macc_config(params, config->macc_config);
+ ia_css_set_ctc_config(params, config->ctc_config);
+ ia_css_set_aa_config(params, config->aa_config);
+ ia_css_set_yuv2rgb_config(params, config->yuv2rgb_cc_config);
+ ia_css_set_rgb2yuv_config(params, config->rgb2yuv_cc_config);
+ ia_css_set_csc_config(params, config->cc_config);
+ ia_css_set_nr_config(params, config->nr_config);
+ ia_css_set_gc_config(params, config->gc_config);
+ ia_css_set_sdis_horicoef_config(params, config->dvs_coefs);
+ ia_css_set_sdis_vertcoef_config(params, config->dvs_coefs);
+ ia_css_set_sdis_horiproj_config(params, config->dvs_coefs);
+ ia_css_set_sdis_vertproj_config(params, config->dvs_coefs);
+ ia_css_set_sdis2_horicoef_config(params, config->dvs2_coefs);
+ ia_css_set_sdis2_vertcoef_config(params, config->dvs2_coefs);
+ ia_css_set_sdis2_horiproj_config(params, config->dvs2_coefs);
+ ia_css_set_sdis2_vertproj_config(params, config->dvs2_coefs);
+ ia_css_set_r_gamma_config(params, config->r_gamma_table);
+ ia_css_set_g_gamma_config(params, config->g_gamma_table);
+ ia_css_set_b_gamma_config(params, config->b_gamma_table);
+ ia_css_set_xnr_table_config(params, config->xnr_table);
+ ia_css_set_formats_config(params, config->formats_config);
+ ia_css_set_xnr_config(params, config->xnr_config);
+ ia_css_set_xnr3_config(params, config->xnr3_config);
+ ia_css_set_s3a_config(params, config->s3a_config);
+ ia_css_set_output_config(params, config->output_config);
+}
diff --git a/drivers/staging/media/atomisp/pci/ia_css_isp_params.h b/drivers/staging/media/atomisp/pci/ia_css_isp_params.h
index 6e3082b..c2de689 100644
--- a/drivers/staging/media/atomisp/pci/ia_css_isp_params.h
+++ b/drivers/staging/media/atomisp/pci/ia_css_isp_params.h
@@ -121,9 +121,6 @@ struct ia_css_memory_offsets {
struct ia_css_isp_parameter sdis_vertcoef;
struct ia_css_isp_parameter sdis2_horicoef;
struct ia_css_isp_parameter sdis2_vertcoef;
-
- /* ISP2401 */
- struct ia_css_isp_parameter xnr3;
} vmem;
struct {
struct ia_css_isp_parameter bh;
diff --git a/drivers/staging/media/atomisp/pci/css_2400_system/hive/ia_css_isp_states.c b/drivers/staging/media/atomisp/pci/ia_css_isp_states.c
similarity index 100%
rename from drivers/staging/media/atomisp/pci/css_2400_system/hive/ia_css_isp_states.c
rename to drivers/staging/media/atomisp/pci/ia_css_isp_states.c
diff --git a/drivers/staging/media/atomisp/pci/ia_css_pipe.h b/drivers/staging/media/atomisp/pci/ia_css_pipe.h
index bb0abf9..fb58535 100644
--- a/drivers/staging/media/atomisp/pci/ia_css_pipe.h
+++ b/drivers/staging/media/atomisp/pci/ia_css_pipe.h
@@ -33,7 +33,7 @@ struct ia_css_preview_settings {
/* 2401 only for these two - do we in fact use them for anything real */
struct ia_css_frame *delay_frames[MAX_NUM_VIDEO_DELAY_FRAMES];
- struct ia_css_frame *tnr_frames[NUM_TNR_FRAMES];
+ struct ia_css_frame *tnr_frames[NUM_VIDEO_TNR_FRAMES];
struct ia_css_pipe *copy_pipe;
struct ia_css_pipe *capture_pipe;
@@ -81,7 +81,7 @@ struct ia_css_video_settings {
struct ia_css_binary vf_pp_binary;
struct ia_css_binary *yuv_scaler_binary;
struct ia_css_frame *delay_frames[MAX_NUM_VIDEO_DELAY_FRAMES];
- struct ia_css_frame *tnr_frames[NUM_TNR_FRAMES];
+ struct ia_css_frame *tnr_frames[NUM_VIDEO_TNR_FRAMES];
struct ia_css_frame *vf_pp_in_frame;
struct ia_css_pipe *copy_pipe;
struct ia_css_pipe *capture_pipe;
diff --git a/drivers/staging/media/atomisp/pci/ia_css_pipe_public.h b/drivers/staging/media/atomisp/pci/ia_css_pipe_public.h
index 4affd21..7352cbf 100644
--- a/drivers/staging/media/atomisp/pci/ia_css_pipe_public.h
+++ b/drivers/staging/media/atomisp/pci/ia_css_pipe_public.h
@@ -123,9 +123,6 @@ struct ia_css_pipe_config {
processing stages. */
/* ISP2401 */
- bool enable_luma_only;
- /** Enabling of monochrome mode for a pipeline. If enabled only luma processing
- will be done. */
bool enable_tnr;
/** Enabling of TNR (temporal noise reduction). This is only applicable to video
pipes. Non video-pipes should always set this parameter to false. */
@@ -482,29 +479,6 @@ ia_css_pipe_get_qos_ext_state(struct ia_css_pipe *pipe,
u32 fw_handle,
bool *enable);
-/* ISP2401 */
-/* @brief Update mapped CSS and ISP arguments for QoS pipe during SP runtime.
- * @param[in] pipe Pipe handle.
- * @param[in] fw_handle Extension firmware Handle (ia_css_fw_info.handle).
- * @param[in] css_seg Parameter memory descriptors for CSS segments.
- * @param[in] isp_seg Parameter memory descriptors for ISP segments.
- *
- * @return
- * 0 : Success
- * -EINVAL : Invalid Parameters
- * -EBUSY : Inactive QOS Pipe
- * (No active stream with this pipe)
- *
- * \deprecated{This interface is used to temporarily support a late-developed,
- * specific use-case on a specific IPU2 platform. It will not be supported or
- * maintained on IPU3 or further.}
- */
-int
-ia_css_pipe_update_qos_ext_mapped_arg(struct ia_css_pipe *pipe,
- u32 fw_handle,
- struct ia_css_isp_param_css_segments *css_seg,
- struct ia_css_isp_param_isp_segments *isp_seg);
-
/* @brief Get selected configuration settings
* @param[in] pipe The pipe.
* @param[out] config Configuration settings.
diff --git a/drivers/staging/media/atomisp/pci/ia_css_stream.h b/drivers/staging/media/atomisp/pci/ia_css_stream.h
index 70b0378..cf84758 100644
--- a/drivers/staging/media/atomisp/pci/ia_css_stream.h
+++ b/drivers/staging/media/atomisp/pci/ia_css_stream.h
@@ -18,9 +18,7 @@
#include <type_support.h>
#include <system_local.h>
-#if !defined(ISP2401)
#include <input_system.h>
-#endif
#include "ia_css_types.h"
#include "ia_css_stream_public.h"
@@ -30,9 +28,7 @@
struct ia_css_stream {
struct ia_css_stream_config config;
struct ia_css_stream_info info;
-#if !defined(ISP2401)
rx_cfg_t csi_rx_config;
-#endif
bool reconfigure_css_rx;
struct ia_css_pipe *last_pipe;
int num_pipes;
diff --git a/drivers/staging/media/atomisp/pci/ia_css_stream_public.h b/drivers/staging/media/atomisp/pci/ia_css_stream_public.h
index 83846e4..47846ec 100644
--- a/drivers/staging/media/atomisp/pci/ia_css_stream_public.h
+++ b/drivers/staging/media/atomisp/pci/ia_css_stream_public.h
@@ -102,12 +102,10 @@ struct ia_css_stream_config {
isys_config[IA_CSS_STREAM_MAX_ISYS_STREAM_PER_CH];
struct ia_css_stream_input_config input_config;
- /* Currently, Android and Windows platforms interpret the binning_factor parameter
- * differently. In Android, the binning factor is expressed in the form
- * 2^N * 2^N, whereas in Windows platform, the binning factor is N*N
- * To use the Windows method of specification, the caller has to define
- * macro USE_WINDOWS_BINNING_FACTOR. This is for backward compatibility only
- * and will be deprecated. In the future,all platforms will use the N*N method
+ /*
+ * Currently, Linux and Windows platforms interpret the binning_factor
+ * parameter differently. In Linux, the binning factor is expressed
+ * in the form 2^N * 2^N
*/
/* ISP2401 */
unsigned int sensor_binning_factor; /** Binning factor used by sensor
@@ -202,15 +200,6 @@ int
ia_css_stream_get_info(const struct ia_css_stream *stream,
struct ia_css_stream_info *stream_info);
-/* @brief load (rebuild) a stream that was unloaded.
- * @param[in] stream The stream
- * @return 0 or the error code
- *
- * Rebuild a stream, including allocating structs, setting configuration and
- * building the required pipes.
- */
-int
-ia_css_stream_load(struct ia_css_stream *stream);
/* @brief Starts the stream.
* @param[in] stream The stream.
diff --git a/drivers/staging/media/atomisp/pci/input_system_local.h b/drivers/staging/media/atomisp/pci/input_system_local.h
index b33aa28..357987d 100644
--- a/drivers/staging/media/atomisp/pci/input_system_local.h
+++ b/drivers/staging/media/atomisp/pci/input_system_local.h
@@ -4,6 +4,140 @@
* (c) 2020 Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
*/
+#include "type_support.h"
+#include "input_system_global.h"
+
+typedef enum {
+ INPUT_SYSTEM_PORT_A = 0,
+ INPUT_SYSTEM_PORT_B,
+ INPUT_SYSTEM_PORT_C,
+ N_INPUT_SYSTEM_PORTS
+} input_system_csi_port_t;
+
+typedef struct ctrl_unit_cfg_s ctrl_unit_cfg_t;
+typedef struct input_system_network_cfg_s input_system_network_cfg_t;
+typedef struct target_cfg2400_s target_cfg2400_t;
+typedef struct channel_cfg_s channel_cfg_t;
+typedef struct backend_channel_cfg_s backend_channel_cfg_t;
+typedef struct input_system_cfg2400_s input_system_cfg2400_t;
+typedef struct mipi_port_state_s mipi_port_state_t;
+typedef struct rx_channel_state_s rx_channel_state_t;
+typedef struct input_switch_cfg_channel_s input_switch_cfg_channel_t;
+typedef struct input_switch_cfg_s input_switch_cfg_t;
+
+struct ctrl_unit_cfg_s {
+ isp2400_ib_buffer_t buffer_mipi[N_CAPTURE_UNIT_ID];
+ isp2400_ib_buffer_t buffer_acquire[N_ACQUISITION_UNIT_ID];
+};
+
+struct input_system_network_cfg_s {
+ input_system_connection_t multicast_cfg[N_CAPTURE_UNIT_ID];
+ input_system_multiplex_t mux_cfg;
+ ctrl_unit_cfg_t ctrl_unit_cfg[N_CTRL_UNIT_ID];
+};
+
+typedef struct {
+// TBD.
+ u32 dummy_parameter;
+} target_isp_cfg_t;
+
+typedef struct {
+// TBD.
+ u32 dummy_parameter;
+} target_sp_cfg_t;
+
+typedef struct {
+// TBD.
+ u32 dummy_parameter;
+} target_strm2mem_cfg_t;
+
+struct input_switch_cfg_channel_s {
+ u32 hsync_data_reg[2];
+ u32 vsync_data_reg;
+};
+
+struct backend_channel_cfg_s {
+ u32 fmt_control_word_1; // Format config.
+ u32 fmt_control_word_2;
+ u32 no_side_band;
+};
+
+typedef union {
+ csi_cfg_t csi_cfg;
+ tpg_cfg_t tpg_cfg;
+ prbs_cfg_t prbs_cfg;
+ gpfifo_cfg_t gpfifo_cfg;
+} source_cfg_t;
+
+struct input_switch_cfg_s {
+ u32 hsync_data_reg[N_RX_CHANNEL_ID * 2];
+ u32 vsync_data_reg;
+};
+
+/*
+ * In 2300 ports can be configured independently and stream
+ * formats need to be specified. In 2400, there are only 8
+ * supported configurations but the HW is fused to support
+ * only a single one.
+ *
+ * In 2300 the compressed format types are programmed by the
+ * user. In 2400 all stream formats are encoded on the stream.
+ *
+ * Use the enum to check validity of a user configuration
+ */
+typedef enum {
+ MONO_4L_1L_0L = 0,
+ MONO_3L_1L_0L,
+ MONO_2L_1L_0L,
+ MONO_1L_1L_0L,
+ STEREO_2L_1L_2L,
+ STEREO_3L_1L_1L,
+ STEREO_2L_1L_1L,
+ STEREO_1L_1L_1L,
+ N_RX_MODE
+} rx_mode_t;
+
+#define UNCOMPRESSED_BITS_PER_PIXEL_10 10
+#define UNCOMPRESSED_BITS_PER_PIXEL_12 12
+#define COMPRESSED_BITS_PER_PIXEL_6 6
+#define COMPRESSED_BITS_PER_PIXEL_7 7
+#define COMPRESSED_BITS_PER_PIXEL_8 8
+enum mipi_compressor {
+ MIPI_COMPRESSOR_NONE = 0,
+ MIPI_COMPRESSOR_10_6_10,
+ MIPI_COMPRESSOR_10_7_10,
+ MIPI_COMPRESSOR_10_8_10,
+ MIPI_COMPRESSOR_12_6_12,
+ MIPI_COMPRESSOR_12_7_12,
+ MIPI_COMPRESSOR_12_8_12,
+ N_MIPI_COMPRESSOR_METHODS
+};
+
+typedef enum mipi_compressor mipi_compressor_t;
+
+typedef enum {
+ MIPI_PREDICTOR_NONE = 0,
+ MIPI_PREDICTOR_TYPE1,
+ MIPI_PREDICTOR_TYPE2,
+ N_MIPI_PREDICTOR_TYPES
+} mipi_predictor_t;
+
+typedef struct rx_cfg_s rx_cfg_t;
+
+/*
+ * Applied per port
+ */
+struct rx_cfg_s {
+ rx_mode_t mode; /* The HW config */
+ enum mipi_port_id port; /* The port ID to apply the control on */
+ unsigned int timeout;
+ unsigned int initcount;
+ unsigned int synccount;
+ unsigned int rxcount;
+ mipi_predictor_t comp; /* Just for backward compatibility */
+ bool is_two_ppc;
+};
+
#ifdef ISP2401
# include "isp2401_input_system_local.h"
#else
diff --git a/drivers/staging/media/atomisp/pci/isp/kernels/bh/bh_2/ia_css_bh.host.c b/drivers/staging/media/atomisp/pci/isp/kernels/bh/bh_2/ia_css_bh.host.c
index 82aa69b..2091f00 100644
--- a/drivers/staging/media/atomisp/pci/isp/kernels/bh/bh_2/ia_css_bh.host.c
+++ b/drivers/staging/media/atomisp/pci/isp/kernels/bh/bh_2/ia_css_bh.host.c
@@ -13,7 +13,6 @@
* more details.
*/
-#if !defined(HAS_NO_HMEM)
#include "ia_css_types.h"
#include "sh_css_internal.h"
@@ -63,4 +62,3 @@ ia_css_bh_encode(
uDIGIT_FITTING(from->ae_y_coef_b, 16, SH_CSS_AE_YCOEF_SHIFT);
}
-#endif
diff --git a/drivers/staging/media/atomisp/pci/isp/kernels/copy_output/copy_output_1.0/ia_css_copy_output.host.c b/drivers/staging/media/atomisp/pci/isp/kernels/copy_output/copy_output_1.0/ia_css_copy_output.host.c
index 5d34f32..cc415c7 100644
--- a/drivers/staging/media/atomisp/pci/isp/kernels/copy_output/copy_output_1.0/ia_css_copy_output.host.c
+++ b/drivers/staging/media/atomisp/pci/isp/kernels/copy_output/copy_output_1.0/ia_css_copy_output.host.c
@@ -34,14 +34,12 @@ ia_css_copy_output_config(
to->enable = from->enable;
}
-void
-ia_css_copy_output_configure(
- const struct ia_css_binary *binary,
- bool enable)
+int ia_css_copy_output_configure(const struct ia_css_binary *binary,
+ bool enable)
{
struct ia_css_copy_output_configuration config = default_config;
config.enable = enable;
- ia_css_configure_copy_output(binary, &config);
+ return ia_css_configure_copy_output(binary, &config);
}
diff --git a/drivers/staging/media/atomisp/pci/isp/kernels/copy_output/copy_output_1.0/ia_css_copy_output.host.h b/drivers/staging/media/atomisp/pci/isp/kernels/copy_output/copy_output_1.0/ia_css_copy_output.host.h
index 615cb67..44e3e45 100644
--- a/drivers/staging/media/atomisp/pci/isp/kernels/copy_output/copy_output_1.0/ia_css_copy_output.host.h
+++ b/drivers/staging/media/atomisp/pci/isp/kernels/copy_output/copy_output_1.0/ia_css_copy_output.host.h
@@ -27,9 +27,7 @@ ia_css_copy_output_config(
const struct ia_css_copy_output_configuration *from,
unsigned int size);
-void
-ia_css_copy_output_configure(
- const struct ia_css_binary *binary,
- bool enable);
+int ia_css_copy_output_configure(const struct ia_css_binary *binary,
+ bool enable);
#endif /* __IA_CSS_COPY_OUTPUT_HOST_H */
diff --git a/drivers/staging/media/atomisp/pci/isp/kernels/crop/crop_1.0/ia_css_crop.host.c b/drivers/staging/media/atomisp/pci/isp/kernels/crop/crop_1.0/ia_css_crop.host.c
index 3891206..8c1d50f 100644
--- a/drivers/staging/media/atomisp/pci/isp/kernels/crop/crop_1.0/ia_css_crop.host.c
+++ b/drivers/staging/media/atomisp/pci/isp/kernels/crop/crop_1.0/ia_css_crop.host.c
@@ -36,30 +36,32 @@ ia_css_crop_encode(
to->crop_pos = from->crop_pos;
}
-void
-ia_css_crop_config(
- struct sh_css_isp_crop_isp_config *to,
- const struct ia_css_crop_configuration *from,
- unsigned int size)
+int ia_css_crop_config(struct sh_css_isp_crop_isp_config *to,
+ const struct ia_css_crop_configuration *from,
+ unsigned int size)
{
unsigned int elems_a = ISP_VEC_NELEMS;
+ int ret;
- (void)size;
- ia_css_dma_configure_from_info(&to->port_b, from->info);
+ ret = ia_css_dma_configure_from_info(&to->port_b, from->info);
+ if (ret)
+ return ret;
+
to->width_a_over_b = elems_a / to->port_b.elems;
/* Assume divisiblity here, may need to generalize to fixed point. */
- assert(elems_a % to->port_b.elems == 0);
+ if (elems_a % to->port_b.elems != 0)
+ return -EINVAL;
+
+ return 0;
}
-void
-ia_css_crop_configure(
- const struct ia_css_binary *binary,
- const struct ia_css_frame_info *info)
+int ia_css_crop_configure(const struct ia_css_binary *binary,
+ const struct ia_css_frame_info *info)
{
struct ia_css_crop_configuration config = default_config;
config.info = info;
- ia_css_configure_crop(binary, &config);
+ return ia_css_configure_crop(binary, &config);
}
diff --git a/drivers/staging/media/atomisp/pci/isp/kernels/crop/crop_1.0/ia_css_crop.host.h b/drivers/staging/media/atomisp/pci/isp/kernels/crop/crop_1.0/ia_css_crop.host.h
index 21a259d..e700149 100644
--- a/drivers/staging/media/atomisp/pci/isp/kernels/crop/crop_1.0/ia_css_crop.host.h
+++ b/drivers/staging/media/atomisp/pci/isp/kernels/crop/crop_1.0/ia_css_crop.host.h
@@ -28,15 +28,11 @@ ia_css_crop_encode(
const struct ia_css_crop_config *from,
unsigned int size);
-void
-ia_css_crop_config(
- struct sh_css_isp_crop_isp_config *to,
- const struct ia_css_crop_configuration *from,
- unsigned int size);
+int ia_css_crop_config(struct sh_css_isp_crop_isp_config *to,
+ const struct ia_css_crop_configuration *from,
+ unsigned int size);
-void
-ia_css_crop_configure(
- const struct ia_css_binary *binary,
- const struct ia_css_frame_info *from);
+int ia_css_crop_configure(const struct ia_css_binary *binary,
+ const struct ia_css_frame_info *from);
#endif /* __IA_CSS_CROP_HOST_H */
diff --git a/drivers/staging/media/atomisp/pci/isp/kernels/dvs/dvs_1.0/ia_css_dvs.host.c b/drivers/staging/media/atomisp/pci/isp/kernels/dvs/dvs_1.0/ia_css_dvs.host.c
index 67f5540..07ce5b4 100644
--- a/drivers/staging/media/atomisp/pci/isp/kernels/dvs/dvs_1.0/ia_css_dvs.host.c
+++ b/drivers/staging/media/atomisp/pci/isp/kernels/dvs/dvs_1.0/ia_css_dvs.host.c
@@ -46,16 +46,14 @@ ia_css_dvs_config(
DVS_NUM_BLOCKS_Y(from->info->res.height);
}
-void
-ia_css_dvs_configure(
- const struct ia_css_binary *binary,
- const struct ia_css_frame_info *info)
+int ia_css_dvs_configure(const struct ia_css_binary *binary,
+ const struct ia_css_frame_info *info)
{
struct ia_css_dvs_configuration config = default_config;
config.info = info;
- ia_css_configure_dvs(binary, &config);
+ return ia_css_configure_dvs(binary, &config);
}
static void
diff --git a/drivers/staging/media/atomisp/pci/isp/kernels/dvs/dvs_1.0/ia_css_dvs.host.h b/drivers/staging/media/atomisp/pci/isp/kernels/dvs/dvs_1.0/ia_css_dvs.host.h
index f9bc17e..332aa54 100644
--- a/drivers/staging/media/atomisp/pci/isp/kernels/dvs/dvs_1.0/ia_css_dvs.host.h
+++ b/drivers/staging/media/atomisp/pci/isp/kernels/dvs/dvs_1.0/ia_css_dvs.host.h
@@ -35,10 +35,8 @@ ia_css_dvs_config(
const struct ia_css_dvs_configuration *from,
unsigned int size);
-void
-ia_css_dvs_configure(
- const struct ia_css_binary *binary,
- const struct ia_css_frame_info *from);
+int ia_css_dvs_configure(const struct ia_css_binary *binary,
+ const struct ia_css_frame_info *from);
void
convert_dvs_6axis_config(
diff --git a/drivers/staging/media/atomisp/pci/isp/kernels/fpn/fpn_1.0/ia_css_fpn.host.c b/drivers/staging/media/atomisp/pci/isp/kernels/fpn/fpn_1.0/ia_css_fpn.host.c
index 47b5c79..57b5e11 100644
--- a/drivers/staging/media/atomisp/pci/isp/kernels/fpn/fpn_1.0/ia_css_fpn.host.c
+++ b/drivers/staging/media/atomisp/pci/isp/kernels/fpn/fpn_1.0/ia_css_fpn.host.c
@@ -51,26 +51,28 @@ ia_css_fpn_dump(
"fpn_enabled", fpn->enabled);
}
-void
-ia_css_fpn_config(
- struct sh_css_isp_fpn_isp_config *to,
- const struct ia_css_fpn_configuration *from,
- unsigned int size)
+int ia_css_fpn_config(struct sh_css_isp_fpn_isp_config *to,
+ const struct ia_css_fpn_configuration *from,
+ unsigned int size)
{
unsigned int elems_a = ISP_VEC_NELEMS;
+ int ret;
- (void)size;
- ia_css_dma_configure_from_info(&to->port_b, from->info);
+ ret = ia_css_dma_configure_from_info(&to->port_b, from->info);
+ if (ret)
+ return ret;
+
to->width_a_over_b = elems_a / to->port_b.elems;
/* Assume divisiblity here, may need to generalize to fixed point. */
- assert(elems_a % to->port_b.elems == 0);
+ if (elems_a % to->port_b.elems != 0)
+ return -EINVAL;
+
+ return 0;
}
-void
-ia_css_fpn_configure(
- const struct ia_css_binary *binary,
- const struct ia_css_frame_info *info)
+int ia_css_fpn_configure(const struct ia_css_binary *binary,
+ const struct ia_css_frame_info *info)
{
struct ia_css_frame_info my_info = IA_CSS_BINARY_DEFAULT_FRAME_INFO;
const struct ia_css_fpn_configuration config = {
@@ -85,5 +87,5 @@ ia_css_fpn_configure(
my_info.raw_bayer_order = info->raw_bayer_order;
my_info.crop_info = info->crop_info;
- ia_css_configure_fpn(binary, &config);
+ return ia_css_configure_fpn(binary, &config);
}
diff --git a/drivers/staging/media/atomisp/pci/isp/kernels/fpn/fpn_1.0/ia_css_fpn.host.h b/drivers/staging/media/atomisp/pci/isp/kernels/fpn/fpn_1.0/ia_css_fpn.host.h
index 12187d2..bd341fa 100644
--- a/drivers/staging/media/atomisp/pci/isp/kernels/fpn/fpn_1.0/ia_css_fpn.host.h
+++ b/drivers/staging/media/atomisp/pci/isp/kernels/fpn/fpn_1.0/ia_css_fpn.host.h
@@ -31,15 +31,11 @@ ia_css_fpn_dump(
const struct sh_css_isp_fpn_params *fpn,
unsigned int level);
-void
-ia_css_fpn_config(
- struct sh_css_isp_fpn_isp_config *to,
- const struct ia_css_fpn_configuration *from,
- unsigned int size);
+int ia_css_fpn_config(struct sh_css_isp_fpn_isp_config *to,
+ const struct ia_css_fpn_configuration *from,
+ unsigned int size);
-void
-ia_css_fpn_configure(
- const struct ia_css_binary *binary,
- const struct ia_css_frame_info *from);
+int ia_css_fpn_configure(const struct ia_css_binary *binary,
+ const struct ia_css_frame_info *from);
#endif /* __IA_CSS_FPN_HOST_H */
diff --git a/drivers/staging/media/atomisp/pci/isp/kernels/hdr/ia_css_hdr.host.c b/drivers/staging/media/atomisp/pci/isp/kernels/hdr/ia_css_hdr.host.c
index 698550c..85a02b6 100644
--- a/drivers/staging/media/atomisp/pci/isp/kernels/hdr/ia_css_hdr.host.c
+++ b/drivers/staging/media/atomisp/pci/isp/kernels/hdr/ia_css_hdr.host.c
@@ -1,6 +1,5 @@
// SPDX-License-Identifier: GPL-2.0
/* Release Version: irci_stable_candrpv_0415_20150521_0458 */
-/* Release Version: irci_ecr-master_20150911_0724 */
/*
* Support for Intel Camera Imaging ISP subsystem.
* Copyright (c) 2015, Intel Corporation.
diff --git a/drivers/staging/media/atomisp/pci/isp/kernels/hdr/ia_css_hdr.host.h b/drivers/staging/media/atomisp/pci/isp/kernels/hdr/ia_css_hdr.host.h
index 04599ab..83277b6 100644
--- a/drivers/staging/media/atomisp/pci/isp/kernels/hdr/ia_css_hdr.host.h
+++ b/drivers/staging/media/atomisp/pci/isp/kernels/hdr/ia_css_hdr.host.h
@@ -1,6 +1,5 @@
/* SPDX-License-Identifier: GPL-2.0 */
/* Release Version: irci_stable_candrpv_0415_20150521_0458 */
-/* Release Version: irci_ecr-master_20150911_0724 */
/*
* Support for Intel Camera Imaging ISP subsystem.
* Copyright (c) 2015, Intel Corporation.
diff --git a/drivers/staging/media/atomisp/pci/isp/kernels/hdr/ia_css_hdr_param.h b/drivers/staging/media/atomisp/pci/isp/kernels/hdr/ia_css_hdr_param.h
index 97a89fd..998c6d8 100644
--- a/drivers/staging/media/atomisp/pci/isp/kernels/hdr/ia_css_hdr_param.h
+++ b/drivers/staging/media/atomisp/pci/isp/kernels/hdr/ia_css_hdr_param.h
@@ -1,6 +1,5 @@
/* SPDX-License-Identifier: GPL-2.0 */
/* Release Version: irci_stable_candrpv_0415_20150521_0458 */
-/* Release Version: irci_ecr-master_20150911_0724 */
/*
* Support for Intel Camera Imaging ISP subsystem.
* Copyright (c) 2015, Intel Corporation.
diff --git a/drivers/staging/media/atomisp/pci/isp/kernels/hdr/ia_css_hdr_types.h b/drivers/staging/media/atomisp/pci/isp/kernels/hdr/ia_css_hdr_types.h
index 1b40908..175c301 100644
--- a/drivers/staging/media/atomisp/pci/isp/kernels/hdr/ia_css_hdr_types.h
+++ b/drivers/staging/media/atomisp/pci/isp/kernels/hdr/ia_css_hdr_types.h
@@ -1,6 +1,5 @@
/* SPDX-License-Identifier: GPL-2.0 */
/* Release Version: irci_stable_candrpv_0415_20150521_0458 */
-/* Release Version: irci_ecr-master_20150911_0724 */
/*
* Support for Intel Camera Imaging ISP subsystem.
* Copyright (c) 2015, Intel Corporation.
diff --git a/drivers/staging/media/atomisp/pci/isp/kernels/ipu2_io_ls/bayer_io_ls/ia_css_bayer_io.host.c b/drivers/staging/media/atomisp/pci/isp/kernels/ipu2_io_ls/bayer_io_ls/ia_css_bayer_io.host.c
index ea80551..c7d8855 100644
--- a/drivers/staging/media/atomisp/pci/isp/kernels/ipu2_io_ls/bayer_io_ls/ia_css_bayer_io.host.c
+++ b/drivers/staging/media/atomisp/pci/isp/kernels/ipu2_io_ls/bayer_io_ls/ia_css_bayer_io.host.c
@@ -22,10 +22,8 @@
#include "ia_css_isp_params.h"
#include "ia_css_frame.h"
-void
-ia_css_bayer_io_config(
- const struct ia_css_binary *binary,
- const struct sh_css_binary_args *args)
+int ia_css_bayer_io_config(const struct ia_css_binary *binary,
+ const struct sh_css_binary_args *args)
{
const struct ia_css_frame *in_frame = args->in_frame;
const struct ia_css_frame **out_frames = (const struct ia_css_frame **)
@@ -38,6 +36,7 @@ ia_css_bayer_io_config(
ddr_bits_per_element);
unsigned int size_get = 0, size_put = 0;
unsigned int offset = 0;
+ int ret;
if (binary->info->mem_offsets.offsets.param) {
size_get = binary->info->mem_offsets.offsets.param->dmem.get.size;
@@ -53,7 +52,9 @@ ia_css_bayer_io_config(
"ia_css_bayer_io_config() get part enter:\n");
#endif
- ia_css_dma_configure_from_info(&config, in_frame_info);
+ ret = ia_css_dma_configure_from_info(&config, in_frame_info);
+ if (ret)
+ return ret;
// The base_address of the input frame will be set in the ISP
to->width = in_frame_info->res.width;
to->height = in_frame_info->res.height;
@@ -79,7 +80,9 @@ ia_css_bayer_io_config(
"ia_css_bayer_io_config() put part enter:\n");
#endif
- ia_css_dma_configure_from_info(&config, &out_frames[0]->info);
+ ret = ia_css_dma_configure_from_info(&config, &out_frames[0]->info);
+ if (ret)
+ return ret;
to->base_address = out_frames[0]->data;
to->width = out_frames[0]->info.res.width;
to->height = out_frames[0]->info.res.height;
@@ -91,4 +94,5 @@ ia_css_bayer_io_config(
"ia_css_bayer_io_config() put part leave:\n");
#endif
}
+ return 0;
}
diff --git a/drivers/staging/media/atomisp/pci/isp/kernels/ipu2_io_ls/bayer_io_ls/ia_css_bayer_io.host.h b/drivers/staging/media/atomisp/pci/isp/kernels/ipu2_io_ls/bayer_io_ls/ia_css_bayer_io.host.h
index 635ccb1..9c7e5a1 100644
--- a/drivers/staging/media/atomisp/pci/isp/kernels/ipu2_io_ls/bayer_io_ls/ia_css_bayer_io.host.h
+++ b/drivers/staging/media/atomisp/pci/isp/kernels/ipu2_io_ls/bayer_io_ls/ia_css_bayer_io.host.h
@@ -21,9 +21,7 @@
#include "ia_css_binary.h"
#include "sh_css_internal.h"
-void
-ia_css_bayer_io_config(
- const struct ia_css_binary *binary,
- const struct sh_css_binary_args *args);
+int ia_css_bayer_io_config(const struct ia_css_binary *binary,
+ const struct sh_css_binary_args *args);
#endif /*__BAYER_IO_HOST_H */
diff --git a/drivers/staging/media/atomisp/pci/isp/kernels/ipu2_io_ls/yuv444_io_ls/ia_css_yuv444_io.host.c b/drivers/staging/media/atomisp/pci/isp/kernels/ipu2_io_ls/yuv444_io_ls/ia_css_yuv444_io.host.c
index f8bd207..7d2ef6e 100644
--- a/drivers/staging/media/atomisp/pci/isp/kernels/ipu2_io_ls/yuv444_io_ls/ia_css_yuv444_io.host.c
+++ b/drivers/staging/media/atomisp/pci/isp/kernels/ipu2_io_ls/yuv444_io_ls/ia_css_yuv444_io.host.c
@@ -22,10 +22,8 @@ more details.
#include "ia_css_isp_params.h"
#include "ia_css_frame.h"
-void
-ia_css_yuv444_io_config(
- const struct ia_css_binary *binary,
- const struct sh_css_binary_args *args)
+int ia_css_yuv444_io_config(const struct ia_css_binary *binary,
+ const struct sh_css_binary_args *args)
{
const struct ia_css_frame *in_frame = args->in_frame;
const struct ia_css_frame **out_frames = (const struct ia_css_frame **)
@@ -38,6 +36,7 @@ ia_css_yuv444_io_config(
ddr_bits_per_element);
unsigned int size_get = 0, size_put = 0;
unsigned int offset = 0;
+ int ret;
if (binary->info->mem_offsets.offsets.param) {
size_get = binary->info->mem_offsets.offsets.param->dmem.get.size;
@@ -53,7 +52,10 @@ ia_css_yuv444_io_config(
"ia_css_yuv444_io_config() get part enter:\n");
#endif
- ia_css_dma_configure_from_info(&config, in_frame_info);
+ ret = ia_css_dma_configure_from_info(&config, in_frame_info);
+ if (ret)
+ return ret;
+
// The base_address of the input frame will be set in the ISP
to->width = in_frame_info->res.width;
to->height = in_frame_info->res.height;
@@ -79,7 +81,10 @@ ia_css_yuv444_io_config(
"ia_css_yuv444_io_config() put part enter:\n");
#endif
- ia_css_dma_configure_from_info(&config, &out_frames[0]->info);
+ ret = ia_css_dma_configure_from_info(&config, &out_frames[0]->info);
+ if (ret)
+ return ret;
+
to->base_address = out_frames[0]->data;
to->width = out_frames[0]->info.res.width;
to->height = out_frames[0]->info.res.height;
@@ -91,4 +96,5 @@ ia_css_yuv444_io_config(
"ia_css_yuv444_io_config() put part leave:\n");
#endif
}
+ return 0;
}
diff --git a/drivers/staging/media/atomisp/pci/isp/kernels/ipu2_io_ls/yuv444_io_ls/ia_css_yuv444_io.host.h b/drivers/staging/media/atomisp/pci/isp/kernels/ipu2_io_ls/yuv444_io_ls/ia_css_yuv444_io.host.h
index e7cfd38..13e50590 100644
--- a/drivers/staging/media/atomisp/pci/isp/kernels/ipu2_io_ls/yuv444_io_ls/ia_css_yuv444_io.host.h
+++ b/drivers/staging/media/atomisp/pci/isp/kernels/ipu2_io_ls/yuv444_io_ls/ia_css_yuv444_io.host.h
@@ -21,9 +21,7 @@ more details.
#include "ia_css_binary.h"
#include "sh_css_internal.h"
-void
-ia_css_yuv444_io_config(
- const struct ia_css_binary *binary,
- const struct sh_css_binary_args *args);
+int ia_css_yuv444_io_config(const struct ia_css_binary *binary,
+ const struct sh_css_binary_args *args);
#endif /*__YUV44_IO_HOST_H */
diff --git a/drivers/staging/media/atomisp/pci/isp/kernels/iterator/iterator_1.0/ia_css_iterator.host.c b/drivers/staging/media/atomisp/pci/isp/kernels/iterator/iterator_1.0/ia_css_iterator.host.c
index 6d8a35a..5f186fb 100644
--- a/drivers/staging/media/atomisp/pci/isp/kernels/iterator/iterator_1.0/ia_css_iterator.host.c
+++ b/drivers/staging/media/atomisp/pci/isp/kernels/iterator/iterator_1.0/ia_css_iterator.host.c
@@ -38,10 +38,9 @@ ia_css_iterator_config(
ia_css_resolution_to_sp_resolution(&to->dvs_envelope, from->dvs_envelope);
}
-int
-ia_css_iterator_configure(
- const struct ia_css_binary *binary,
- const struct ia_css_frame_info *in_info) {
+int ia_css_iterator_configure(const struct ia_css_binary *binary,
+ const struct ia_css_frame_info *in_info)
+{
struct ia_css_frame_info my_info = IA_CSS_BINARY_DEFAULT_FRAME_INFO;
struct ia_css_iterator_configuration config = default_config;
@@ -75,7 +74,5 @@ ia_css_iterator_configure(
my_info.res.height <<= binary->vf_downscale_log2;
}
- ia_css_configure_iterator(binary, &config);
-
- return 0;
+ return ia_css_configure_iterator(binary, &config);
}
diff --git a/drivers/staging/media/atomisp/pci/isp/kernels/output/output_1.0/ia_css_output.host.c b/drivers/staging/media/atomisp/pci/isp/kernels/output/output_1.0/ia_css_output.host.c
index c8e074f..be9e4ef 100644
--- a/drivers/staging/media/atomisp/pci/isp/kernels/output/output_1.0/ia_css_output.host.c
+++ b/drivers/staging/media/atomisp/pci/isp/kernels/output/output_1.0/ia_css_output.host.c
@@ -52,49 +52,45 @@ ia_css_output_encode(
to->enable_vflip = from->enable_vflip;
}
-void
-ia_css_output_config(
- struct sh_css_isp_output_isp_config *to,
- const struct ia_css_output_configuration *from,
- unsigned int size)
+int ia_css_output_config(struct sh_css_isp_output_isp_config *to,
+ const struct ia_css_output_configuration *from,
+ unsigned int size)
{
unsigned int elems_a = ISP_VEC_NELEMS;
+ int ret;
- (void)size;
- ia_css_dma_configure_from_info(&to->port_b, from->info);
+ ret = ia_css_dma_configure_from_info(&to->port_b, from->info);
+ if (ret)
+ return ret;
+
to->width_a_over_b = elems_a / to->port_b.elems;
to->height = from->info ? from->info->res.height : 0;
to->enable = from->info != NULL;
ia_css_frame_info_to_frame_sp_info(&to->info, from->info);
/* Assume divisiblity here, may need to generalize to fixed point. */
- assert(elems_a % to->port_b.elems == 0);
+ if (elems_a % to->port_b.elems != 0)
+ return -EINVAL;
+
+ return 0;
}
-void
-ia_css_output0_config(
- struct sh_css_isp_output_isp_config *to,
- const struct ia_css_output0_configuration *from,
- unsigned int size)
+int ia_css_output0_config(struct sh_css_isp_output_isp_config *to,
+ const struct ia_css_output0_configuration *from,
+ unsigned int size)
{
- ia_css_output_config(
- to, (const struct ia_css_output_configuration *)from, size);
+ return ia_css_output_config(to, (const struct ia_css_output_configuration *)from, size);
}
-void
-ia_css_output1_config(
- struct sh_css_isp_output_isp_config *to,
- const struct ia_css_output1_configuration *from,
- unsigned int size)
+int ia_css_output1_config(struct sh_css_isp_output_isp_config *to,
+ const struct ia_css_output1_configuration *from,
+ unsigned int size)
{
- ia_css_output_config(
- to, (const struct ia_css_output_configuration *)from, size);
+ return ia_css_output_config(to, (const struct ia_css_output_configuration *)from, size);
}
-void
-ia_css_output_configure(
- const struct ia_css_binary *binary,
- const struct ia_css_frame_info *info)
+int ia_css_output_configure(const struct ia_css_binary *binary,
+ const struct ia_css_frame_info *info)
{
if (info) {
struct ia_css_output_configuration config =
@@ -102,14 +98,13 @@ ia_css_output_configure(
config.info = info;
- ia_css_configure_output(binary, &config);
+ return ia_css_configure_output(binary, &config);
}
+ return 0;
}
-void
-ia_css_output0_configure(
- const struct ia_css_binary *binary,
- const struct ia_css_frame_info *info)
+int ia_css_output0_configure(const struct ia_css_binary *binary,
+ const struct ia_css_frame_info *info)
{
if (info) {
struct ia_css_output0_configuration config =
@@ -117,14 +112,13 @@ ia_css_output0_configure(
config.info = info;
- ia_css_configure_output0(binary, &config);
+ return ia_css_configure_output0(binary, &config);
}
+ return 0;
}
-void
-ia_css_output1_configure(
- const struct ia_css_binary *binary,
- const struct ia_css_frame_info *info)
+int ia_css_output1_configure(const struct ia_css_binary *binary,
+ const struct ia_css_frame_info *info)
{
if (info) {
struct ia_css_output1_configuration config =
@@ -132,8 +126,9 @@ ia_css_output1_configure(
config.info = info;
- ia_css_configure_output1(binary, &config);
+ return ia_css_configure_output1(binary, &config);
}
+ return 0;
}
void
diff --git a/drivers/staging/media/atomisp/pci/isp/kernels/output/output_1.0/ia_css_output.host.h b/drivers/staging/media/atomisp/pci/isp/kernels/output/output_1.0/ia_css_output.host.h
index 1f5a224..c8523e9 100644
--- a/drivers/staging/media/atomisp/pci/isp/kernels/output/output_1.0/ia_css_output.host.h
+++ b/drivers/staging/media/atomisp/pci/isp/kernels/output/output_1.0/ia_css_output.host.h
@@ -30,38 +30,26 @@ ia_css_output_encode(
const struct ia_css_output_config *from,
unsigned int size);
-void
-ia_css_output_config(
- struct sh_css_isp_output_isp_config *to,
- const struct ia_css_output_configuration *from,
- unsigned int size);
+int ia_css_output_config(struct sh_css_isp_output_isp_config *to,
+ const struct ia_css_output_configuration *from,
+ unsigned int size);
-void
-ia_css_output0_config(
- struct sh_css_isp_output_isp_config *to,
- const struct ia_css_output0_configuration *from,
- unsigned int size);
+int ia_css_output0_config(struct sh_css_isp_output_isp_config *to,
+ const struct ia_css_output0_configuration *from,
+ unsigned int size);
-void
-ia_css_output1_config(
- struct sh_css_isp_output_isp_config *to,
- const struct ia_css_output1_configuration *from,
- unsigned int size);
+int ia_css_output1_config(struct sh_css_isp_output_isp_config *to,
+ const struct ia_css_output1_configuration *from,
+ unsigned int size);
-void
-ia_css_output_configure(
- const struct ia_css_binary *binary,
- const struct ia_css_frame_info *from);
+int ia_css_output_configure(const struct ia_css_binary *binary,
+ const struct ia_css_frame_info *from);
-void
-ia_css_output0_configure(
- const struct ia_css_binary *binary,
- const struct ia_css_frame_info *from);
+int ia_css_output0_configure(const struct ia_css_binary *binary,
+ const struct ia_css_frame_info *from);
-void
-ia_css_output1_configure(
- const struct ia_css_binary *binary,
- const struct ia_css_frame_info *from);
+int ia_css_output1_configure(const struct ia_css_binary *binary,
+ const struct ia_css_frame_info *from);
void
ia_css_output_dump(
diff --git a/drivers/staging/media/atomisp/pci/isp/kernels/qplane/qplane_2/ia_css_qplane.host.c b/drivers/staging/media/atomisp/pci/isp/kernels/qplane/qplane_2/ia_css_qplane.host.c
index 1603fd4..9fd4435 100644
--- a/drivers/staging/media/atomisp/pci/isp/kernels/qplane/qplane_2/ia_css_qplane.host.c
+++ b/drivers/staging/media/atomisp/pci/isp/kernels/qplane/qplane_2/ia_css_qplane.host.c
@@ -28,35 +28,37 @@ static const struct ia_css_qplane_configuration default_config = {
.pipe = (struct sh_css_sp_pipeline *)NULL,
};
-void
-ia_css_qplane_config(
- struct sh_css_isp_qplane_isp_config *to,
- const struct ia_css_qplane_configuration *from,
- unsigned int size)
+int ia_css_qplane_config(struct sh_css_isp_qplane_isp_config *to,
+ const struct ia_css_qplane_configuration *from,
+ unsigned int size)
{
unsigned int elems_a = ISP_VEC_NELEMS;
+ int ret;
- (void)size;
- ia_css_dma_configure_from_info(&to->port_b, from->info);
+ ret = ia_css_dma_configure_from_info(&to->port_b, from->info);
+ if (ret)
+ return ret;
+
to->width_a_over_b = elems_a / to->port_b.elems;
/* Assume divisiblity here, may need to generalize to fixed point. */
- assert(elems_a % to->port_b.elems == 0);
+ if (elems_a % to->port_b.elems != 0)
+ return -EINVAL;
to->inout_port_config = from->pipe->inout_port_config;
to->format = from->info->format;
+
+ return 0;
}
-void
-ia_css_qplane_configure(
- const struct sh_css_sp_pipeline *pipe,
- const struct ia_css_binary *binary,
- const struct ia_css_frame_info *info)
+int ia_css_qplane_configure(const struct sh_css_sp_pipeline *pipe,
+ const struct ia_css_binary *binary,
+ const struct ia_css_frame_info *info)
{
struct ia_css_qplane_configuration config = default_config;
config.pipe = pipe;
config.info = info;
- ia_css_configure_qplane(binary, &config);
+ return ia_css_configure_qplane(binary, &config);
}
diff --git a/drivers/staging/media/atomisp/pci/isp/kernels/qplane/qplane_2/ia_css_qplane.host.h b/drivers/staging/media/atomisp/pci/isp/kernels/qplane/qplane_2/ia_css_qplane.host.h
index 8d94095..b3f8fa3 100644
--- a/drivers/staging/media/atomisp/pci/isp/kernels/qplane/qplane_2/ia_css_qplane.host.h
+++ b/drivers/staging/media/atomisp/pci/isp/kernels/qplane/qplane_2/ia_css_qplane.host.h
@@ -29,16 +29,12 @@
#include "ia_css_qplane_types.h"
#include "ia_css_qplane_param.h"
-void
-ia_css_qplane_config(
- struct sh_css_isp_qplane_isp_config *to,
- const struct ia_css_qplane_configuration *from,
- unsigned int size);
+int ia_css_qplane_config(struct sh_css_isp_qplane_isp_config *to,
+ const struct ia_css_qplane_configuration *from,
+ unsigned int size);
-void
-ia_css_qplane_configure(
- const struct sh_css_sp_pipeline *pipe,
- const struct ia_css_binary *binary,
- const struct ia_css_frame_info *from);
+int ia_css_qplane_configure(const struct sh_css_sp_pipeline *pipe,
+ const struct ia_css_binary *binary,
+ const struct ia_css_frame_info *from);
#endif /* __IA_CSS_QPLANE_HOST_H */
diff --git a/drivers/staging/media/atomisp/pci/isp/kernels/raw/raw_1.0/ia_css_raw.host.c b/drivers/staging/media/atomisp/pci/isp/kernels/raw/raw_1.0/ia_css_raw.host.c
index c505c94..646d6e39 100644
--- a/drivers/staging/media/atomisp/pci/isp/kernels/raw/raw_1.0/ia_css_raw.host.c
+++ b/drivers/staging/media/atomisp/pci/isp/kernels/raw/raw_1.0/ia_css_raw.host.c
@@ -29,12 +29,6 @@ static const struct ia_css_raw_configuration default_config = {
.pipe = (struct sh_css_sp_pipeline *)NULL,
};
-static inline unsigned
-sh_css_elems_bytes_from_info(unsigned int raw_bit_depth)
-{
- return CEIL_DIV(raw_bit_depth, 8);
-}
-
/* MW: These areMIPI / ISYS properties, not camera function properties */
static enum sh_stream_format
css2isp_stream_format(enum atomisp_input_format from) {
@@ -70,17 +64,15 @@ css2isp_stream_format(enum atomisp_input_format from) {
}
}
-void
-ia_css_raw_config(
- struct sh_css_isp_raw_isp_config *to,
- const struct ia_css_raw_configuration *from,
- unsigned int size)
+int ia_css_raw_config(struct sh_css_isp_raw_isp_config *to,
+ const struct ia_css_raw_configuration *from,
+ unsigned int size)
{
unsigned int elems_a = ISP_VEC_NELEMS;
const struct ia_css_frame_info *in_info = from->in_info;
const struct ia_css_frame_info *internal_info = from->internal_info;
+ int ret;
- (void)size;
#if !defined(ISP2401)
/* 2401 input system uses input width width */
in_info = internal_info;
@@ -92,7 +84,9 @@ ia_css_raw_config(
in_info = internal_info;
#endif
- ia_css_dma_configure_from_info(&to->port_b, in_info);
+ ret = ia_css_dma_configure_from_info(&to->port_b, in_info);
+ if (ret)
+ return ret;
/* Assume divisiblity here, may need to generalize to fixed point. */
assert((in_info->format == IA_CSS_FRAME_FORMAT_RAW_PACKED) ||
@@ -110,16 +104,16 @@ ia_css_raw_config(
to->start_line = in_info->crop_info.start_line;
to->enable_left_padding = from->enable_left_padding;
#endif
+
+ return 0;
}
-void
-ia_css_raw_configure(
- const struct sh_css_sp_pipeline *pipe,
- const struct ia_css_binary *binary,
- const struct ia_css_frame_info *in_info,
- const struct ia_css_frame_info *internal_info,
- bool two_ppc,
- bool deinterleaved)
+int ia_css_raw_configure(const struct sh_css_sp_pipeline *pipe,
+ const struct ia_css_binary *binary,
+ const struct ia_css_frame_info *in_info,
+ const struct ia_css_frame_info *internal_info,
+ bool two_ppc,
+ bool deinterleaved)
{
u8 enable_left_padding = (uint8_t)((binary->left_padding) ? 1 : 0);
struct ia_css_raw_configuration config = default_config;
@@ -132,5 +126,5 @@ ia_css_raw_configure(
config.deinterleaved = deinterleaved;
config.enable_left_padding = enable_left_padding;
- ia_css_configure_raw(binary, &config);
+ return ia_css_configure_raw(binary, &config);
}
diff --git a/drivers/staging/media/atomisp/pci/isp/kernels/raw/raw_1.0/ia_css_raw.host.h b/drivers/staging/media/atomisp/pci/isp/kernels/raw/raw_1.0/ia_css_raw.host.h
index 3469284..23da51a 100644
--- a/drivers/staging/media/atomisp/pci/isp/kernels/raw/raw_1.0/ia_css_raw.host.h
+++ b/drivers/staging/media/atomisp/pci/isp/kernels/raw/raw_1.0/ia_css_raw.host.h
@@ -21,19 +21,15 @@
#include "ia_css_raw_types.h"
#include "ia_css_raw_param.h"
-void
-ia_css_raw_config(
- struct sh_css_isp_raw_isp_config *to,
- const struct ia_css_raw_configuration *from,
- unsigned int size);
+int ia_css_raw_config(struct sh_css_isp_raw_isp_config *to,
+ const struct ia_css_raw_configuration *from,
+ unsigned int size);
-void
-ia_css_raw_configure(
- const struct sh_css_sp_pipeline *pipe,
- const struct ia_css_binary *binary,
- const struct ia_css_frame_info *in_info,
- const struct ia_css_frame_info *internal_info,
- bool two_ppc,
- bool deinterleaved);
+int ia_css_raw_configure(const struct sh_css_sp_pipeline *pipe,
+ const struct ia_css_binary *binary,
+ const struct ia_css_frame_info *in_info,
+ const struct ia_css_frame_info *internal_info,
+ bool two_ppc,
+ bool deinterleaved);
#endif /* __IA_CSS_RAW_HOST_H */
diff --git a/drivers/staging/media/atomisp/pci/isp/kernels/raw_aa_binning/raw_aa_binning_1.0/ia_css_raa.host.c b/drivers/staging/media/atomisp/pci/isp/kernels/raw_aa_binning/raw_aa_binning_1.0/ia_css_raa.host.c
index 29c707e..9b756da 100644
--- a/drivers/staging/media/atomisp/pci/isp/kernels/raw_aa_binning/raw_aa_binning_1.0/ia_css_raa.host.c
+++ b/drivers/staging/media/atomisp/pci/isp/kernels/raw_aa_binning/raw_aa_binning_1.0/ia_css_raa.host.c
@@ -13,7 +13,6 @@
* more details.
*/
-#if !defined(HAS_NO_HMEM)
#include "ia_css_types.h"
#include "sh_css_internal.h"
@@ -32,4 +31,3 @@ ia_css_raa_encode(
(void)from;
}
-#endif
diff --git a/drivers/staging/media/atomisp/pci/isp/kernels/ref/ref_1.0/ia_css_ref.host.c b/drivers/staging/media/atomisp/pci/isp/kernels/ref/ref_1.0/ia_css_ref.host.c
index 061558f..08ed916 100644
--- a/drivers/staging/media/atomisp/pci/isp/kernels/ref/ref_1.0/ia_css_ref.host.c
+++ b/drivers/staging/media/atomisp/pci/isp/kernels/ref/ref_1.0/ia_css_ref.host.c
@@ -22,16 +22,17 @@
#include "isp.h"
#include "ia_css_ref.host.h"
-void
-ia_css_ref_config(
- struct sh_css_isp_ref_isp_config *to,
- const struct ia_css_ref_configuration *from,
- unsigned int size)
+int ia_css_ref_config(struct sh_css_isp_ref_isp_config *to,
+ const struct ia_css_ref_configuration *from,
+ unsigned int size)
{
unsigned int elems_a = ISP_VEC_NELEMS, i;
+ int ret;
if (from->ref_frames[0]) {
- ia_css_dma_configure_from_info(&to->port_b, &from->ref_frames[0]->info);
+ ret = ia_css_dma_configure_from_info(&to->port_b, &from->ref_frames[0]->info);
+ if (ret)
+ return ret;
to->width_a_over_b = elems_a / to->port_b.elems;
to->dvs_frame_delay = from->dvs_frame_delay;
} else {
@@ -52,22 +53,25 @@ ia_css_ref_config(
}
/* Assume divisiblity here, may need to generalize to fixed point. */
- assert(elems_a % to->port_b.elems == 0);
+ if (elems_a % to->port_b.elems != 0)
+ return -EINVAL;
+
+ return 0;
}
-void
-ia_css_ref_configure(
- const struct ia_css_binary *binary,
- const struct ia_css_frame * const *ref_frames,
- const uint32_t dvs_frame_delay)
+int ia_css_ref_configure(const struct ia_css_binary *binary,
+ const struct ia_css_frame * const *ref_frames,
+ const uint32_t dvs_frame_delay)
{
struct ia_css_ref_configuration config;
unsigned int i;
for (i = 0; i < MAX_NUM_VIDEO_DELAY_FRAMES; i++)
config.ref_frames[i] = ref_frames[i];
+
config.dvs_frame_delay = dvs_frame_delay;
- ia_css_configure_ref(binary, &config);
+
+ return ia_css_configure_ref(binary, &config);
}
void
diff --git a/drivers/staging/media/atomisp/pci/isp/kernels/ref/ref_1.0/ia_css_ref.host.h b/drivers/staging/media/atomisp/pci/isp/kernels/ref/ref_1.0/ia_css_ref.host.h
index 3ce590b..388cd4c 100644
--- a/drivers/staging/media/atomisp/pci/isp/kernels/ref/ref_1.0/ia_css_ref.host.h
+++ b/drivers/staging/media/atomisp/pci/isp/kernels/ref/ref_1.0/ia_css_ref.host.h
@@ -23,17 +23,13 @@
#include "ia_css_ref_param.h"
#include "ia_css_ref_state.h"
-void
-ia_css_ref_config(
- struct sh_css_isp_ref_isp_config *to,
- const struct ia_css_ref_configuration *from,
- unsigned int size);
+int ia_css_ref_config(struct sh_css_isp_ref_isp_config *to,
+ const struct ia_css_ref_configuration *from,
+ unsigned int size);
-void
-ia_css_ref_configure(
- const struct ia_css_binary *binary,
- const struct ia_css_frame * const *ref_frames,
- const uint32_t dvs_frame_delay);
+int ia_css_ref_configure(const struct ia_css_binary *binary,
+ const struct ia_css_frame * const *ref_frames,
+ const uint32_t dvs_frame_delay);
void
ia_css_init_ref_state(
diff --git a/drivers/staging/media/atomisp/pci/isp/kernels/s3a/s3a_1.0/ia_css_s3a.host.c b/drivers/staging/media/atomisp/pci/isp/kernels/s3a/s3a_1.0/ia_css_s3a.host.c
index ba52c80..bd7b89d 100644
--- a/drivers/staging/media/atomisp/pci/isp/kernels/s3a/s3a_1.0/ia_css_s3a.host.c
+++ b/drivers/staging/media/atomisp/pci/isp/kernels/s3a/s3a_1.0/ia_css_s3a.host.c
@@ -227,10 +227,6 @@ ia_css_s3a_hmem_decode(
struct ia_css_3a_statistics *host_stats,
const struct ia_css_bh_table *hmem_buf)
{
-#if defined(HAS_NO_HMEM)
- (void)host_stats;
- (void)hmem_buf;
-#else
struct ia_css_3a_rgby_output *out_ptr;
int i;
@@ -291,7 +287,6 @@ ia_css_s3a_hmem_decode(
out_ptr[0].g -= diff;
out_ptr[0].b -= diff;
out_ptr[0].y -= diff;
-#endif
}
void
diff --git a/drivers/staging/media/atomisp/pci/isp/kernels/sc/sc_1.0/ia_css_sc.host.c b/drivers/staging/media/atomisp/pci/isp/kernels/sc/sc_1.0/ia_css_sc.host.c
index f3fb4b9..6974b34 100644
--- a/drivers/staging/media/atomisp/pci/isp/kernels/sc/sc_1.0/ia_css_sc.host.c
+++ b/drivers/staging/media/atomisp/pci/isp/kernels/sc/sc_1.0/ia_css_sc.host.c
@@ -23,35 +23,6 @@
#include "ia_css_sc.host.h"
-/* Code generated by genparam/genconfig.c:gen_configure_function() */
-
-/* ISP2401 */
-static void
-ia_css_configure_sc(
- const struct ia_css_binary *binary,
- const struct ia_css_sc_configuration *config_dmem)
-{
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_sc() enter:\n");
-
- {
- unsigned int offset = 0;
- unsigned int size = 0;
-
- if (binary->info->mem_offsets.offsets.config) {
- size = binary->info->mem_offsets.offsets.config->dmem.sc.size;
- offset = binary->info->mem_offsets.offsets.config->dmem.sc.offset;
- }
- if (size) {
- ia_css_sc_config((struct sh_css_isp_sc_isp_config *)
- &binary->mem_params.params[IA_CSS_PARAM_CLASS_CONFIG][IA_CSS_ISP_DMEM].address[offset],
- config_dmem, size);
- }
- }
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "ia_css_configure_sc() leave:\n");
-}
-
void
ia_css_sc_encode(
struct sh_css_isp_sc_params *to,
@@ -73,45 +44,6 @@ ia_css_sc_dump(
"sc_gain_shift", sc->gain_shift);
}
-/* ISP2401 */
-void
-ia_css_sc_config(
- struct sh_css_isp_sc_isp_config *to,
- const struct ia_css_sc_configuration *from,
- unsigned int size)
-{
- u32 internal_org_x_bqs = from->internal_frame_origin_x_bqs_on_sctbl;
- u32 internal_org_y_bqs = from->internal_frame_origin_y_bqs_on_sctbl;
- u32 slice, rest, i;
-
- (void)size;
-
- /* The internal_frame_origin_x_bqs_on_sctbl is separated to 8 times of slice_vec. */
- rest = internal_org_x_bqs;
- for (i = 0; i < SH_CSS_SC_INTERPED_GAIN_HOR_SLICE_TIMES; i++) {
- slice = min(rest, ((uint32_t)ISP_SLICE_NELEMS));
- rest = rest - slice;
- to->interped_gain_hor_slice_bqs[i] = slice;
- }
-
- to->internal_frame_origin_y_bqs_on_sctbl = internal_org_y_bqs;
-}
-
-/* ISP2401 */
-void
-ia_css_sc_configure(
- const struct ia_css_binary *binary,
- u32 internal_frame_origin_x_bqs_on_sctbl,
- uint32_t internal_frame_origin_y_bqs_on_sctbl)
-{
- const struct ia_css_sc_configuration config = {
- internal_frame_origin_x_bqs_on_sctbl,
- internal_frame_origin_y_bqs_on_sctbl
- };
-
- ia_css_configure_sc(binary, &config);
-}
-
/* ------ deprecated(bz675) : from ------ */
/* It looks like @parameter{} (in *.pipe) is used to generate the process/get/set functions,
for parameters which should be used in the isp kernels.
diff --git a/drivers/staging/media/atomisp/pci/isp/kernels/sc/sc_1.0/ia_css_sc.host.h b/drivers/staging/media/atomisp/pci/isp/kernels/sc/sc_1.0/ia_css_sc.host.h
index f1eb568..d103103 100644
--- a/drivers/staging/media/atomisp/pci/isp/kernels/sc/sc_1.0/ia_css_sc.host.h
+++ b/drivers/staging/media/atomisp/pci/isp/kernels/sc/sc_1.0/ia_css_sc.host.h
@@ -32,39 +32,6 @@ ia_css_sc_dump(
const struct sh_css_isp_sc_params *sc,
unsigned int level);
-/* @brief Configure the shading correction.
- * @param[out] to Parameters used in the shading correction kernel in the isp.
- * @param[in] from Parameters passed from the host.
- * @param[in] size Size of the sh_css_isp_sc_isp_config structure.
- *
- * This function passes the parameters for the shading correction from the host to the isp.
- */
-/* ISP2401 */
-void
-ia_css_sc_config(
- struct sh_css_isp_sc_isp_config *to,
- const struct ia_css_sc_configuration *from,
- unsigned int size);
-
-/* @brief Configure the shading correction.
- * @param[in] binary The binary, which has the shading correction.
- * @param[in] internal_frame_origin_x_bqs_on_sctbl
- * X coordinate (in bqs) of the origin of the internal frame on the shading table.
- * @param[in] internal_frame_origin_y_bqs_on_sctbl
- * Y coordinate (in bqs) of the origin of the internal frame on the shading table.
- *
- * This function calls the ia_css_configure_sc() function.
- * (The ia_css_configure_sc() function is automatically generated in ia_css_isp.configs.c.)
- * The ia_css_configure_sc() function calls the ia_css_sc_config() function
- * to pass the parameters for the shading correction from the host to the isp.
- */
-/* ISP2401 */
-void
-ia_css_sc_configure(
- const struct ia_css_binary *binary,
- u32 internal_frame_origin_x_bqs_on_sctbl,
- uint32_t internal_frame_origin_y_bqs_on_sctbl);
-
/* ------ deprecated(bz675) : from ------ */
void
sh_css_get_shading_settings(const struct ia_css_isp_parameters *params,
diff --git a/drivers/staging/media/atomisp/pci/isp/kernels/sc/sc_1.0/ia_css_sc_types.h b/drivers/staging/media/atomisp/pci/isp/kernels/sc/sc_1.0/ia_css_sc_types.h
index aae5345..1d70f6b 100644
--- a/drivers/staging/media/atomisp/pci/isp/kernels/sc/sc_1.0/ia_css_sc_types.h
+++ b/drivers/staging/media/atomisp/pci/isp/kernels/sc/sc_1.0/ia_css_sc_types.h
@@ -118,18 +118,4 @@ struct ia_css_shading_settings {
/* ------ deprecated(bz675) : to ------ */
-/* Shading Correction configuration.
- *
- * NOTE: The shading table size is larger than or equal to the internal frame size.
- */
-/* ISP2401 */
-struct ia_css_sc_configuration {
- u32 internal_frame_origin_x_bqs_on_sctbl; /** Origin X (in bqs) of internal frame on shading table. */
- u32 internal_frame_origin_y_bqs_on_sctbl; /** Origin Y (in bqs) of internal frame on shading table. */
- /** NOTE: bqs = size in BQ(Bayer Quad) unit.
- 1BQ means {Gr,R,B,Gb}(2x2 pixels).
- Horizontal 1 bqs corresponds to horizontal 2 pixels.
- Vertical 1 bqs corresponds to vertical 2 pixels. */
-};
-
#endif /* __IA_CSS_SC_TYPES_H */
diff --git a/drivers/staging/media/atomisp/pci/isp/kernels/tnr/tnr_1.0/ia_css_tnr.host.c b/drivers/staging/media/atomisp/pci/isp/kernels/tnr/tnr_1.0/ia_css_tnr.host.c
index ac80e6c..53050c0 100644
--- a/drivers/staging/media/atomisp/pci/isp/kernels/tnr/tnr_1.0/ia_css_tnr.host.c
+++ b/drivers/staging/media/atomisp/pci/isp/kernels/tnr/tnr_1.0/ia_css_tnr.host.c
@@ -71,40 +71,41 @@ ia_css_tnr_debug_dtrace(
config->threshold_y, config->threshold_uv);
}
-void
-ia_css_tnr_config(
- struct sh_css_isp_tnr_isp_config *to,
- const struct ia_css_tnr_configuration *from,
- unsigned int size)
+int ia_css_tnr_config(struct sh_css_isp_tnr_isp_config *to,
+ const struct ia_css_tnr_configuration *from,
+ unsigned int size)
{
unsigned int elems_a = ISP_VEC_NELEMS;
unsigned int i;
+ int ret;
- (void)size;
- ia_css_dma_configure_from_info(&to->port_b, &from->tnr_frames[0]->info);
+ ret = ia_css_dma_configure_from_info(&to->port_b, &from->tnr_frames[0]->info);
+ if (ret)
+ return ret;
to->width_a_over_b = elems_a / to->port_b.elems;
to->frame_height = from->tnr_frames[0]->info.res.height;
- for (i = 0; i < NUM_TNR_FRAMES; i++) {
+ for (i = 0; i < NUM_VIDEO_TNR_FRAMES; i++) {
to->tnr_frame_addr[i] = from->tnr_frames[i]->data +
from->tnr_frames[i]->planes.yuyv.offset;
}
/* Assume divisiblity here, may need to generalize to fixed point. */
- assert(elems_a % to->port_b.elems == 0);
+ if (elems_a % to->port_b.elems != 0)
+ return -EINVAL;
+
+ return 0;
}
-void
-ia_css_tnr_configure(
- const struct ia_css_binary *binary,
- const struct ia_css_frame * const *frames)
+int ia_css_tnr_configure(const struct ia_css_binary *binary,
+ const struct ia_css_frame * const *frames)
{
struct ia_css_tnr_configuration config;
unsigned int i;
- for (i = 0; i < NUM_TNR_FRAMES; i++)
+ for (i = 0; i < NUM_VIDEO_TNR_FRAMES; i++)
config.tnr_frames[i] = frames[i];
- ia_css_configure_tnr(binary, &config);
+ return ia_css_configure_tnr(binary, &config);
}
void
@@ -114,7 +115,7 @@ ia_css_init_tnr_state(
{
(void)size;
- assert(NUM_TNR_FRAMES >= 2);
+ assert(NUM_VIDEO_TNR_FRAMES >= 2);
assert(sizeof(*state) == size);
state->tnr_in_buf_idx = 0;
state->tnr_out_buf_idx = 1;
diff --git a/drivers/staging/media/atomisp/pci/isp/kernels/tnr/tnr_1.0/ia_css_tnr.host.h b/drivers/staging/media/atomisp/pci/isp/kernels/tnr/tnr_1.0/ia_css_tnr.host.h
index 90d6e6b..acf9205 100644
--- a/drivers/staging/media/atomisp/pci/isp/kernels/tnr/tnr_1.0/ia_css_tnr.host.h
+++ b/drivers/staging/media/atomisp/pci/isp/kernels/tnr/tnr_1.0/ia_css_tnr.host.h
@@ -39,16 +39,12 @@ ia_css_tnr_debug_dtrace(
const struct ia_css_tnr_config *config,
unsigned int level);
-void
-ia_css_tnr_config(
- struct sh_css_isp_tnr_isp_config *to,
- const struct ia_css_tnr_configuration *from,
- unsigned int size);
+int ia_css_tnr_config(struct sh_css_isp_tnr_isp_config *to,
+ const struct ia_css_tnr_configuration *from,
+ unsigned int size);
-void
-ia_css_tnr_configure(
- const struct ia_css_binary *binary,
- const struct ia_css_frame * const *frames);
+int ia_css_tnr_configure(const struct ia_css_binary *binary,
+ const struct ia_css_frame * const *frames);
void
ia_css_init_tnr_state(
diff --git a/drivers/staging/media/atomisp/pci/isp/kernels/tnr/tnr_1.0/ia_css_tnr_param.h b/drivers/staging/media/atomisp/pci/isp/kernels/tnr/tnr_1.0/ia_css_tnr_param.h
index 60a2542..551dd5c 100644
--- a/drivers/staging/media/atomisp/pci/isp/kernels/tnr/tnr_1.0/ia_css_tnr_param.h
+++ b/drivers/staging/media/atomisp/pci/isp/kernels/tnr/tnr_1.0/ia_css_tnr_param.h
@@ -28,14 +28,14 @@ struct sh_css_isp_tnr_params {
};
struct ia_css_tnr_configuration {
- const struct ia_css_frame *tnr_frames[NUM_TNR_FRAMES];
+ const struct ia_css_frame *tnr_frames[NUM_VIDEO_TNR_FRAMES];
};
struct sh_css_isp_tnr_isp_config {
u32 width_a_over_b;
u32 frame_height;
struct dma_port_config port_b;
- ia_css_ptr tnr_frame_addr[NUM_TNR_FRAMES];
+ ia_css_ptr tnr_frame_addr[NUM_VIDEO_TNR_FRAMES];
};
#endif /* __IA_CSS_TNR_PARAM_H */
diff --git a/drivers/staging/media/atomisp/pci/isp/kernels/vf/vf_1.0/ia_css_vf.host.c b/drivers/staging/media/atomisp/pci/isp/kernels/vf/vf_1.0/ia_css_vf.host.c
index dd36709..aecdcbe 100644
--- a/drivers/staging/media/atomisp/pci/isp/kernels/vf/vf_1.0/ia_css_vf.host.c
+++ b/drivers/staging/media/atomisp/pci/isp/kernels/vf/vf_1.0/ia_css_vf.host.c
@@ -26,26 +26,28 @@
#include "isp.h"
-void
-ia_css_vf_config(
- struct sh_css_isp_vf_isp_config *to,
- const struct ia_css_vf_configuration *from,
- unsigned int size)
+int ia_css_vf_config(struct sh_css_isp_vf_isp_config *to,
+ const struct ia_css_vf_configuration *from,
+ unsigned int size)
{
unsigned int elems_a = ISP_VEC_NELEMS;
+ int ret;
- (void)size;
to->vf_downscale_bits = from->vf_downscale_bits;
to->enable = from->info != NULL;
if (from->info) {
ia_css_frame_info_to_frame_sp_info(&to->info, from->info);
- ia_css_dma_configure_from_info(&to->dma.port_b, from->info);
+ ret = ia_css_dma_configure_from_info(&to->dma.port_b, from->info);
+ if (ret)
+ return ret;
to->dma.width_a_over_b = elems_a / to->dma.port_b.elems;
/* Assume divisiblity here, may need to generalize to fixed point. */
- assert(elems_a % to->dma.port_b.elems == 0);
+ if (elems_a % to->dma.port_b.elems != 0)
+ return -EINVAL;
}
+ return 0;
}
/* compute the log2 of the downscale factor needed to get closest
@@ -120,12 +122,11 @@ configure_dma(
config->info = vf_info;
}
-int
-ia_css_vf_configure(
- const struct ia_css_binary *binary,
- const struct ia_css_frame_info *out_info,
- struct ia_css_frame_info *vf_info,
- unsigned int *downscale_log2) {
+int ia_css_vf_configure(const struct ia_css_binary *binary,
+ const struct ia_css_frame_info *out_info,
+ struct ia_css_frame_info *vf_info,
+ unsigned int *downscale_log2)
+{
int err;
struct ia_css_vf_configuration config;
const struct ia_css_binary_info *info = &binary->info->sp;
@@ -138,7 +139,6 @@ ia_css_vf_configure(
if (vf_info)
vf_info->raw_bit_depth = info->dma.vfdec_bits_per_pixel;
- ia_css_configure_vf(binary, &config);
- return 0;
+ return ia_css_configure_vf(binary, &config);
}
diff --git a/drivers/staging/media/atomisp/pci/isp/kernels/vf/vf_1.0/ia_css_vf.host.h b/drivers/staging/media/atomisp/pci/isp/kernels/vf/vf_1.0/ia_css_vf.host.h
index 0e8de03..d6b45d3 100644
--- a/drivers/staging/media/atomisp/pci/isp/kernels/vf/vf_1.0/ia_css_vf.host.h
+++ b/drivers/staging/media/atomisp/pci/isp/kernels/vf/vf_1.0/ia_css_vf.host.h
@@ -32,11 +32,9 @@ sh_css_vf_downscale_log2(
const struct ia_css_frame_info *vf_info,
unsigned int *downscale_log2);
-void
-ia_css_vf_config(
- struct sh_css_isp_vf_isp_config *to,
- const struct ia_css_vf_configuration *from,
- unsigned int size);
+int ia_css_vf_config(struct sh_css_isp_vf_isp_config *to,
+ const struct ia_css_vf_configuration *from,
+ unsigned int size);
int
ia_css_vf_configure(
diff --git a/drivers/staging/media/atomisp/pci/isp/modes/interface/isp_const.h b/drivers/staging/media/atomisp/pci/isp/modes/interface/isp_const.h
index 11e439d..bfe4f59 100644
--- a/drivers/staging/media/atomisp/pci/isp/modes/interface/isp_const.h
+++ b/drivers/staging/media/atomisp/pci/isp/modes/interface/isp_const.h
@@ -36,17 +36,7 @@ more details.
#define BITS8_ELEMENTS_PER_XMEM_ADDR CEIL_DIV(XMEM_WIDTH_BITS, 8)
#define BITS16_ELEMENTS_PER_XMEM_ADDR CEIL_DIV(XMEM_WIDTH_BITS, 16)
-#if ISP_VEC_NELEMS == 64
#define ISP_NWAY_LOG2 6
-#elif ISP_VEC_NELEMS == 32
-#define ISP_NWAY_LOG2 5
-#elif ISP_VEC_NELEMS == 16
-#define ISP_NWAY_LOG2 4
-#elif ISP_VEC_NELEMS == 8
-#define ISP_NWAY_LOG2 3
-#else
-#error "isp_const.h ISP_VEC_NELEMS must be one of {8, 16, 32, 64}"
-#endif
/* *****************************
* ISP input/output buffer sizes
@@ -164,9 +154,9 @@ more details.
/* [isp vmem] table size[vectors] per line per color (GR,R,B,GB),
multiples of NWAY */
#define ISP2400_SCTBL_VECTORS_PER_LINE_PER_COLOR \
- CEIL_DIV(ISP2400_SH_CSS_MAX_SCTBL_WIDTH_PER_COLOR, ISP_VEC_NELEMS)
+ CEIL_DIV(SH_CSS_MAX_SCTBL_WIDTH_PER_COLOR, ISP_VEC_NELEMS)
#define ISP2401_SCTBL_VECTORS_PER_LINE_PER_COLOR \
- CEIL_DIV(ISP2401_SH_CSS_MAX_SCTBL_WIDTH_PER_COLOR, ISP_VEC_NELEMS)
+ CEIL_DIV(SH_CSS_MAX_SCTBL_WIDTH_PER_COLOR, ISP_VEC_NELEMS)
/* [isp vmem] table size[vectors] per line for 4colors (GR,R,B,GB),
multiples of NWAY */
#define SCTBL_VECTORS_PER_LINE \
diff --git a/drivers/staging/media/atomisp/pci/isp2400_input_system_local.h b/drivers/staging/media/atomisp/pci/isp2400_input_system_local.h
index 2614b89..6880c9b 100644
--- a/drivers/staging/media/atomisp/pci/isp2400_input_system_local.h
+++ b/drivers/staging/media/atomisp/pci/isp2400_input_system_local.h
@@ -16,10 +16,6 @@
#ifndef __INPUT_SYSTEM_LOCAL_H_INCLUDED__
#define __INPUT_SYSTEM_LOCAL_H_INCLUDED__
-#include <type_support.h>
-
-#include "input_system_global.h"
-
#include "input_system_defs.h" /* HIVE_ISYS_GPREG_MULTICAST_A_IDX,... */
/*
@@ -33,54 +29,7 @@
#include "isp_acquisition_defs.h"
#include "input_system_ctrl_defs.h"
-typedef enum {
- INPUT_SYSTEM_PORT_A = 0,
- INPUT_SYSTEM_PORT_B,
- INPUT_SYSTEM_PORT_C,
- N_INPUT_SYSTEM_PORTS
-} input_system_csi_port_t;
-
-typedef struct ctrl_unit_cfg_s ctrl_unit_cfg_t;
-typedef struct input_system_network_cfg_s input_system_network_cfg_t;
-typedef struct target_cfg2400_s target_cfg2400_t;
-typedef struct channel_cfg_s channel_cfg_t;
-typedef struct backend_channel_cfg_s backend_channel_cfg_t;
typedef struct input_system_cfg2400_s input_system_cfg2400_t;
-typedef struct mipi_port_state_s mipi_port_state_t;
-typedef struct rx_channel_state_s rx_channel_state_t;
-typedef struct input_switch_cfg_channel_s input_switch_cfg_channel_t;
-typedef struct input_switch_cfg_s input_switch_cfg_t;
-
-struct ctrl_unit_cfg_s {
- isp2400_ib_buffer_t buffer_mipi[N_CAPTURE_UNIT_ID];
- isp2400_ib_buffer_t buffer_acquire[N_ACQUISITION_UNIT_ID];
-};
-
-struct input_system_network_cfg_s {
- input_system_connection_t multicast_cfg[N_CAPTURE_UNIT_ID];
- input_system_multiplex_t mux_cfg;
- ctrl_unit_cfg_t ctrl_unit_cfg[N_CTRL_UNIT_ID];
-};
-
-typedef struct {
-// TBD.
- u32 dummy_parameter;
-} target_isp_cfg_t;
-
-typedef struct {
-// TBD.
- u32 dummy_parameter;
-} target_sp_cfg_t;
-
-typedef struct {
-// TBD.
- u32 dummy_parameter;
-} target_strm2mem_cfg_t;
-
-struct input_switch_cfg_channel_s {
- u32 hsync_data_reg[2];
- u32 vsync_data_reg;
-};
struct target_cfg2400_s {
input_switch_cfg_channel_t input_switch_channel_cfg;
@@ -89,24 +38,6 @@ struct target_cfg2400_s {
target_strm2mem_cfg_t target_strm2mem_cfg;
};
-struct backend_channel_cfg_s {
- u32 fmt_control_word_1; // Format config.
- u32 fmt_control_word_2;
- u32 no_side_band;
-};
-
-typedef union {
- csi_cfg_t csi_cfg;
- tpg_cfg_t tpg_cfg;
- prbs_cfg_t prbs_cfg;
- gpfifo_cfg_t gpfifo_cfg;
-} source_cfg_t;
-
-struct input_switch_cfg_s {
- u32 hsync_data_reg[N_RX_CHANNEL_ID * 2];
- u32 vsync_data_reg;
-};
-
// Configuration of a channel.
struct channel_cfg_s {
u32 ch_id;
@@ -238,47 +169,6 @@ typedef struct capture_unit_state_s capture_unit_state_t;
typedef struct acquisition_unit_state_s acquisition_unit_state_t;
typedef struct ctrl_unit_state_s ctrl_unit_state_t;
-/*
- * In 2300 ports can be configured independently and stream
- * formats need to be specified. In 2400, there are only 8
- * supported configurations but the HW is fused to support
- * only a single one.
- *
- * In 2300 the compressed format types are programmed by the
- * user. In 2400 all stream formats are encoded on the stream.
- *
- * Use the enum to check validity of a user configuration
- */
-typedef enum {
- MONO_4L_1L_0L = 0,
- MONO_3L_1L_0L,
- MONO_2L_1L_0L,
- MONO_1L_1L_0L,
- STEREO_2L_1L_2L,
- STEREO_3L_1L_1L,
- STEREO_2L_1L_1L,
- STEREO_1L_1L_1L,
- N_RX_MODE
-} rx_mode_t;
-
-typedef enum {
- MIPI_PREDICTOR_NONE = 0,
- MIPI_PREDICTOR_TYPE1,
- MIPI_PREDICTOR_TYPE2,
- N_MIPI_PREDICTOR_TYPES
-} mipi_predictor_t;
-
-typedef enum {
- MIPI_COMPRESSOR_NONE = 0,
- MIPI_COMPRESSOR_10_6_10,
- MIPI_COMPRESSOR_10_7_10,
- MIPI_COMPRESSOR_10_8_10,
- MIPI_COMPRESSOR_12_6_12,
- MIPI_COMPRESSOR_12_7_12,
- MIPI_COMPRESSOR_12_8_12,
- N_MIPI_COMPRESSOR_METHODS
-} mipi_compressor_t;
-
typedef enum {
MIPI_FORMAT_RGB888 = 0,
MIPI_FORMAT_RGB555,
@@ -339,22 +229,6 @@ typedef enum {
RX_IRQ_INFO_ERR_LINE_SYNC = 1UL << _HRT_CSS_RECEIVER_IRQ_ERR_LINE_SYNC_BIT,
} rx_irq_info_t;
-typedef struct rx_cfg_s rx_cfg_t;
-
-/*
- * Applied per port
- */
-struct rx_cfg_s {
- rx_mode_t mode; /* The HW config */
- enum mipi_port_id port; /* The port ID to apply the control on */
- unsigned int timeout;
- unsigned int initcount;
- unsigned int synccount;
- unsigned int rxcount;
- mipi_predictor_t comp; /* Just for backward compatibility */
- bool is_two_ppc;
-};
-
/* NOTE: The base has already an offset of 0x0100 */
static const hrt_address __maybe_unused MIPI_PORT_OFFSET[N_MIPI_PORT_ID] = {
0x00000000UL,
diff --git a/drivers/staging/media/atomisp/pci/isp2401_input_system_global.h b/drivers/staging/media/atomisp/pci/isp2401_input_system_global.h
index f387738..e3c8606 100644
--- a/drivers/staging/media/atomisp/pci/isp2401_input_system_global.h
+++ b/drivers/staging/media/atomisp/pci/isp2401_input_system_global.h
@@ -44,11 +44,6 @@ typedef enum {
N_INPUT_SYSTEM_SOURCE_TYPE
} input_system_source_type_t;
-typedef enum {
- INPUT_SYSTEM_POLL_ON_WAIT_FOR_FRAME,
- INPUT_SYSTEM_POLL_ON_CAPTURE_REQUEST,
-} input_system_polling_mode_t;
-
typedef struct input_system_channel_s input_system_channel_t;
struct input_system_channel_s {
stream2mmio_ID_t stream2mmio_id;
@@ -111,9 +106,6 @@ struct isp2401_input_system_cfg_s {
input_system_source_type_t mode;
- /* ISP2401 */
- input_system_polling_mode_t polling_mode;
-
bool online;
bool raw_packed;
s8 linked_isys_stream_id;
@@ -165,10 +157,6 @@ struct virtual_input_system_stream_s {
u8 online;
s8 linked_isys_stream_id;
u8 valid;
-
- /* ISP2401 */
- input_system_polling_mode_t polling_mode;
- s32 subscr_index;
};
typedef struct virtual_input_system_stream_cfg_s
diff --git a/drivers/staging/media/atomisp/pci/isp2401_input_system_local.h b/drivers/staging/media/atomisp/pci/isp2401_input_system_local.h
index 2402609..74bfa10 100644
--- a/drivers/staging/media/atomisp/pci/isp2401_input_system_local.h
+++ b/drivers/staging/media/atomisp/pci/isp2401_input_system_local.h
@@ -16,9 +16,6 @@
#ifndef __INPUT_SYSTEM_LOCAL_H_INCLUDED__
#define __INPUT_SYSTEM_LOCAL_H_INCLUDED__
-#include "type_support.h"
-#include "input_system_global.h"
-
#include "csi_rx.h"
#include "pixelgen.h"
#include "isys_stream2mmio.h"
@@ -69,29 +66,6 @@ typedef enum {
/* The number of stores for compressed format types */
#define N_MIPI_COMPRESSOR_CONTEXT (N_RX_CHANNEL_ID * N_MIPI_FORMAT_CUSTOM)
-#define UNCOMPRESSED_BITS_PER_PIXEL_10 10
-#define UNCOMPRESSED_BITS_PER_PIXEL_12 12
-#define COMPRESSED_BITS_PER_PIXEL_6 6
-#define COMPRESSED_BITS_PER_PIXEL_7 7
-#define COMPRESSED_BITS_PER_PIXEL_8 8
-enum mipi_compressor {
- MIPI_COMPRESSOR_NONE = 0,
- MIPI_COMPRESSOR_10_6_10,
- MIPI_COMPRESSOR_10_7_10,
- MIPI_COMPRESSOR_10_8_10,
- MIPI_COMPRESSOR_12_6_12,
- MIPI_COMPRESSOR_12_7_12,
- MIPI_COMPRESSOR_12_8_12,
- N_MIPI_COMPRESSOR_METHODS
-};
-
-typedef enum {
- MIPI_PREDICTOR_NONE = 0,
- MIPI_PREDICTOR_TYPE1,
- MIPI_PREDICTOR_TYPE2,
- N_MIPI_PREDICTOR_TYPES
-} mipi_predictor_t;
-
typedef struct input_system_state_s input_system_state_t;
struct input_system_state_s {
ibuf_ctrl_state_t ibuf_ctrl_state[N_IBUF_CTRL_ID];
diff --git a/drivers/staging/media/atomisp/pci/runtime/binary/interface/ia_css_binary.h b/drivers/staging/media/atomisp/pci/runtime/binary/interface/ia_css_binary.h
index b44099d..9935ac8 100644
--- a/drivers/staging/media/atomisp/pci/runtime/binary/interface/ia_css_binary.h
+++ b/drivers/staging/media/atomisp/pci/runtime/binary/interface/ia_css_binary.h
@@ -94,7 +94,6 @@ struct ia_css_binary_descr {
bool enable_dpc;
/* ISP2401 */
- bool enable_luma_only;
bool enable_tnr;
bool enable_capture_pp_bli;
@@ -131,8 +130,6 @@ struct ia_css_binary {
int sctbl_width_per_color;
int sctbl_aligned_width_per_color;
int sctbl_height;
- int sctbl_legacy_width_per_color;
- int sctbl_legacy_height;
struct ia_css_sdis_info dis;
struct ia_css_resolution dvs_envelope;
bool online;
diff --git a/drivers/staging/media/atomisp/pci/runtime/binary/src/binary.c b/drivers/staging/media/atomisp/pci/runtime/binary/src/binary.c
index 060d387..406ed5f 100644
--- a/drivers/staging/media/atomisp/pci/runtime/binary/src/binary.c
+++ b/drivers/staging/media/atomisp/pci/runtime/binary/src/binary.c
@@ -108,7 +108,6 @@ ia_css_binary_internal_res(const struct ia_css_frame_info *in_info,
binary_dvs_env.height);
}
-/* ISP2400 */
/* Computation results of the origin coordinate of bayer on the shading table. */
struct sh_css_shading_table_bayer_origin_compute_results {
u32 bayer_scale_hor_ratio_in; /* Horizontal ratio (in) of bayer scaling. */
@@ -119,23 +118,7 @@ struct sh_css_shading_table_bayer_origin_compute_results {
u32 sc_bayer_origin_y_bqs_on_shading_table; /* Y coordinate (in bqs) of bayer origin on shading table. */
};
-/* ISP2401 */
-/* Requirements for the shading correction. */
-struct sh_css_binary_sc_requirements {
- /* Bayer scaling factor, for the scaling which is applied before shading correction. */
- u32 bayer_scale_hor_ratio_in; /* Horizontal ratio (in) of scaling applied BEFORE shading correction. */
- u32 bayer_scale_hor_ratio_out; /* Horizontal ratio (out) of scaling applied BEFORE shading correction. */
- u32 bayer_scale_ver_ratio_in; /* Vertical ratio (in) of scaling applied BEFORE shading correction. */
- u32 bayer_scale_ver_ratio_out; /* Vertical ratio (out) of scaling applied BEFORE shading correction. */
-
- /* ISP internal frame is composed of the real sensor data and the padding data. */
- u32 sensor_data_origin_x_bqs_on_internal; /* X origin (in bqs) of sensor data on internal frame
- at shading correction. */
- u32 sensor_data_origin_y_bqs_on_internal; /* Y origin (in bqs) of sensor data on internal frame
- at shading correction. */
-};
-
-/* ISP2400: Get the requirements for the shading correction. */
+/* Get the requirements for the shading correction. */
static int
ia_css_binary_compute_shading_table_bayer_origin(
const struct ia_css_binary *binary, /* [in] */
@@ -261,227 +244,12 @@ ia_css_binary_compute_shading_table_bayer_origin(
return err;
}
-/* ISP2401: Get the requirements for the shading correction. */
-static int
-sh_css_binary_get_sc_requirements(const struct ia_css_binary *binary, /* [in] */
- unsigned int required_bds_factor, /* [in] */
- const struct ia_css_stream_config *stream_config, /* [in] */
- struct sh_css_binary_sc_requirements *scr) /* [out] */
-{
- int err;
-
- /* Numerator and denominator of the fixed bayer downscaling factor. (numerator >= denominator) */
- unsigned int bds_num, bds_den;
-
- /* Horizontal/Vertical ratio of bayer scaling between input area and output area. */
- unsigned int bs_hor_ratio_in, bs_hor_ratio_out, bs_ver_ratio_in, bs_ver_ratio_out;
-
- /* Left padding set by InputFormatter. */
- unsigned int left_padding_bqs;
-
- /* Flags corresponding to NEED_BDS_FACTOR_2_00/NEED_BDS_FACTOR_1_50/NEED_BDS_FACTOR_1_25 macros
- * defined in isp kernels. */
- unsigned int need_bds_factor_2_00, need_bds_factor_1_50, need_bds_factor_1_25;
-
- /* Left padding adjusted inside the isp kernels. */
- unsigned int left_padding_adjusted_bqs;
-
- /* Top padding padded inside the isp kernel for bayer downscaling binaries. */
- unsigned int top_padding_bqs;
-
- /* Bayer downscaling factor 1.0 by fixed-point. */
- int bds_frac_acc = FRAC_ACC; /* FRAC_ACC is defined in ia_css_fixedbds_param.h. */
-
- /* Right/Down shift amount caused by filters applied BEFORE shading corrertion. */
- unsigned int right_shift_bqs_before_bs; /* right shift before bayer scaling */
- unsigned int right_shift_bqs_after_bs; /* right shift after bayer scaling */
- unsigned int down_shift_bqs_before_bs; /* down shift before bayer scaling */
- unsigned int down_shift_bqs_after_bs; /* down shift after bayer scaling */
-
- /* Origin of the real sensor data area on the internal frame at shading correction. */
- unsigned int sensor_data_origin_x_bqs_on_internal;
- unsigned int sensor_data_origin_y_bqs_on_internal;
-
- unsigned int bs_frac = bds_frac_acc; /* scaling factor 1.0 in fixed point */
- unsigned int bs_out, bs_in; /* scaling ratio in fixed point */
-
- IA_CSS_ENTER_PRIVATE("binary=%p, required_bds_factor=%d, stream_config=%p",
- binary, required_bds_factor, stream_config);
-
- /* Get the numerator and denominator of the required bayer downscaling factor. */
- err = sh_css_bds_factor_get_numerator_denominator(required_bds_factor,
- &bds_num, &bds_den);
- if (err) {
- IA_CSS_LEAVE_ERR_PRIVATE(err);
- return err;
- }
-
- IA_CSS_LOG("bds_num=%d, bds_den=%d", bds_num, bds_den);
-
- /* Set the horizontal/vertical ratio of bayer scaling between input area and output area. */
- bs_hor_ratio_in = bds_num;
- bs_hor_ratio_out = bds_den;
- bs_ver_ratio_in = bds_num;
- bs_ver_ratio_out = bds_den;
-
- /* Set the left padding set by InputFormatter. (ia_css_ifmtr_configure() in ifmtr.c) */
- if (stream_config->left_padding == -1)
- left_padding_bqs = _ISP_BQS(binary->left_padding);
- else
- left_padding_bqs = (unsigned int)((int)ISP_VEC_NELEMS - _ISP_BQS(stream_config->left_padding));
-
- IA_CSS_LOG("stream.left_padding=%d, binary.left_padding=%d, left_padding_bqs=%d",
- stream_config->left_padding, binary->left_padding,
- left_padding_bqs);
-
- /* Set the left padding adjusted inside the isp kernels.
- * When the bds_factor isn't 1.00, the left padding size is adjusted inside the isp,
- * before bayer downscaling. (scaled_hor_plane_index(), raw_compute_hphase() in raw.isp.c)
- */
- need_bds_factor_2_00 = ((binary->info->sp.bds.supported_bds_factors &
- (PACK_BDS_FACTOR(SH_CSS_BDS_FACTOR_2_00) |
- PACK_BDS_FACTOR(SH_CSS_BDS_FACTOR_2_50) |
- PACK_BDS_FACTOR(SH_CSS_BDS_FACTOR_3_00) |
- PACK_BDS_FACTOR(SH_CSS_BDS_FACTOR_4_00) |
- PACK_BDS_FACTOR(SH_CSS_BDS_FACTOR_4_50) |
- PACK_BDS_FACTOR(SH_CSS_BDS_FACTOR_5_00) |
- PACK_BDS_FACTOR(SH_CSS_BDS_FACTOR_6_00) |
- PACK_BDS_FACTOR(SH_CSS_BDS_FACTOR_8_00))) != 0);
-
- need_bds_factor_1_50 = ((binary->info->sp.bds.supported_bds_factors &
- (PACK_BDS_FACTOR(SH_CSS_BDS_FACTOR_1_50) |
- PACK_BDS_FACTOR(SH_CSS_BDS_FACTOR_2_25) |
- PACK_BDS_FACTOR(SH_CSS_BDS_FACTOR_3_00) |
- PACK_BDS_FACTOR(SH_CSS_BDS_FACTOR_4_50) |
- PACK_BDS_FACTOR(SH_CSS_BDS_FACTOR_6_00))) != 0);
-
- need_bds_factor_1_25 = ((binary->info->sp.bds.supported_bds_factors &
- (PACK_BDS_FACTOR(SH_CSS_BDS_FACTOR_1_25) |
- PACK_BDS_FACTOR(SH_CSS_BDS_FACTOR_2_50) |
- PACK_BDS_FACTOR(SH_CSS_BDS_FACTOR_5_00))) != 0);
-
- if (binary->info->sp.pipeline.left_cropping > 0 &&
- (need_bds_factor_2_00 || need_bds_factor_1_50 || need_bds_factor_1_25)) {
- /*
- * downscale 2.0 -> first_vec_adjusted_bqs = 128
- * downscale 1.5 -> first_vec_adjusted_bqs = 96
- * downscale 1.25 -> first_vec_adjusted_bqs = 80
- */
- unsigned int first_vec_adjusted_bqs = ISP_VEC_NELEMS * bs_hor_ratio_in / bs_hor_ratio_out;
- left_padding_adjusted_bqs = first_vec_adjusted_bqs
- - _ISP_BQS(binary->info->sp.pipeline.left_cropping);
- } else {
- left_padding_adjusted_bqs = left_padding_bqs;
- }
-
- IA_CSS_LOG("supported_bds_factors=%d, need_bds_factor:2_00=%d, 1_50=%d, 1_25=%d",
- binary->info->sp.bds.supported_bds_factors,
- need_bds_factor_2_00, need_bds_factor_1_50,
- need_bds_factor_1_25);
- IA_CSS_LOG("left_cropping=%d, left_padding_adjusted_bqs=%d",
- binary->info->sp.pipeline.left_cropping,
- left_padding_adjusted_bqs);
-
- /* Set the top padding padded inside the isp kernel for bayer downscaling binaries.
- * When the bds_factor isn't 1.00, the top padding is padded inside the isp
- * before bayer downscaling, because the top cropping size (input margin) is not enough.
- * (calculate_input_line(), raw_compute_vphase(), dma_read_raw() in raw.isp.c)
- * NOTE: In dma_read_raw(), the factor passed to raw_compute_vphase() is got by get_bds_factor_for_dma_read().
- * This factor is BDS_FPVAL_100/BDS_FPVAL_125/BDS_FPVAL_150/BDS_FPVAL_200.
- */
- top_padding_bqs = 0;
- if (binary->info->sp.pipeline.top_cropping > 0 &&
- (required_bds_factor == SH_CSS_BDS_FACTOR_1_25 ||
- required_bds_factor == SH_CSS_BDS_FACTOR_1_50 ||
- required_bds_factor == SH_CSS_BDS_FACTOR_2_00)) {
- /* Calculation from calculate_input_line() and raw_compute_vphase() in raw.isp.c. */
- int top_cropping_bqs = _ISP_BQS(binary->info->sp.pipeline.top_cropping);
- /* top cropping (in bqs) */
- int factor = bds_num * bds_frac_acc /
- bds_den; /* downscaling factor by fixed-point */
- int top_padding_bqsxfrac_acc = (top_cropping_bqs * factor - top_cropping_bqs *
- bds_frac_acc)
- + (2 * bds_frac_acc - factor); /* top padding by fixed-point (in bqs) */
-
- top_padding_bqs = (unsigned int)((top_padding_bqsxfrac_acc + bds_frac_acc / 2 -
- 1) / bds_frac_acc);
- }
-
- IA_CSS_LOG("top_cropping=%d, top_padding_bqs=%d",
- binary->info->sp.pipeline.top_cropping, top_padding_bqs);
-
- /* Set the right/down shift amount caused by filters applied BEFORE bayer scaling,
- * which scaling is applied BEFORE shading corrertion.
- *
- * When the bds_factor isn't 1.00, 3x3 anti-alias filter is applied to each color plane(Gr/R/B/Gb)
- * before bayer downscaling.
- * This filter shifts each color plane (Gr/R/B/Gb) to right/down directions by 1 pixel.
- */
- right_shift_bqs_before_bs = 0;
- down_shift_bqs_before_bs = 0;
-
- if (need_bds_factor_2_00 || need_bds_factor_1_50 || need_bds_factor_1_25) {
- right_shift_bqs_before_bs = 1;
- down_shift_bqs_before_bs = 1;
- }
-
- IA_CSS_LOG("right_shift_bqs_before_bs=%d, down_shift_bqs_before_bs=%d",
- right_shift_bqs_before_bs, down_shift_bqs_before_bs);
-
- /* Set the right/down shift amount caused by filters applied AFTER bayer scaling,
- * which scaling is applied BEFORE shading corrertion.
- *
- * When DPC&BNR is processed between bayer scaling and shading correction,
- * DPC&BNR moves each color plane (Gr/R/B/Gb) to right/down directions by 1 pixel.
- */
- right_shift_bqs_after_bs = 0;
- down_shift_bqs_after_bs = 0;
-
- /* if DPC&BNR is enabled in the binary */
- if (binary->info->mem_offsets.offsets.param->dmem.dp.size != 0) {
- right_shift_bqs_after_bs = 1;
- down_shift_bqs_after_bs = 1;
- }
-
- IA_CSS_LOG("right_shift_bqs_after_bs=%d, down_shift_bqs_after_bs=%d",
- right_shift_bqs_after_bs, down_shift_bqs_after_bs);
-
- bs_out = bs_hor_ratio_out * bs_frac;
- bs_in = bs_hor_ratio_in * bs_frac;
- sensor_data_origin_x_bqs_on_internal =
- ((left_padding_adjusted_bqs + right_shift_bqs_before_bs) * bs_out + bs_in / 2) / bs_in
- + right_shift_bqs_after_bs; /* "+ bs_in/2": rounding */
-
- bs_out = bs_ver_ratio_out * bs_frac;
- bs_in = bs_ver_ratio_in * bs_frac;
- sensor_data_origin_y_bqs_on_internal =
- ((top_padding_bqs + down_shift_bqs_before_bs) * bs_out + bs_in / 2) / bs_in
- + down_shift_bqs_after_bs; /* "+ bs_in/2": rounding */
-
- scr->bayer_scale_hor_ratio_in = (uint32_t)bs_hor_ratio_in;
- scr->bayer_scale_hor_ratio_out = (uint32_t)bs_hor_ratio_out;
- scr->bayer_scale_ver_ratio_in = (uint32_t)bs_ver_ratio_in;
- scr->bayer_scale_ver_ratio_out = (uint32_t)bs_ver_ratio_out;
- scr->sensor_data_origin_x_bqs_on_internal = (uint32_t)sensor_data_origin_x_bqs_on_internal;
- scr->sensor_data_origin_y_bqs_on_internal = (uint32_t)sensor_data_origin_y_bqs_on_internal;
-
- IA_CSS_LOG("sc_requirements: %d, %d, %d, %d, %d, %d",
- scr->bayer_scale_hor_ratio_in,
- scr->bayer_scale_hor_ratio_out,
- scr->bayer_scale_ver_ratio_in, scr->bayer_scale_ver_ratio_out,
- scr->sensor_data_origin_x_bqs_on_internal,
- scr->sensor_data_origin_y_bqs_on_internal);
-
- IA_CSS_LEAVE_ERR_PRIVATE(err);
- return err;
-}
-
/* Get the shading information of Shading Correction Type 1. */
static int
-isp2400_binary_get_shading_info_type_1(const struct ia_css_binary *binary, /* [in] */
- unsigned int required_bds_factor, /* [in] */
- const struct ia_css_stream_config *stream_config, /* [in] */
- struct ia_css_shading_info *info) /* [out] */
+binary_get_shading_info_type_1(const struct ia_css_binary *binary, /* [in] */
+ unsigned int required_bds_factor, /* [in] */
+ const struct ia_css_stream_config *stream_config, /* [in] */
+ struct ia_css_shading_info *info) /* [out] */
{
int err;
struct sh_css_shading_table_bayer_origin_compute_results res;
@@ -522,173 +290,6 @@ isp2400_binary_get_shading_info_type_1(const struct ia_css_binary *binary, /* [i
return err;
}
-/* Get the shading information of Shading Correction Type 1. */
-static int
-isp2401_binary_get_shading_info_type_1(const struct ia_css_binary *binary, /* [in] */
- unsigned int required_bds_factor, /* [in] */
- const struct ia_css_stream_config *stream_config, /* [in] */
- struct ia_css_shading_info *shading_info, /* [out] */
- struct ia_css_pipe_config *pipe_config) /* [out] */
-{
- int err;
- struct sh_css_binary_sc_requirements scr;
-
- u32 in_width_bqs, in_height_bqs, internal_width_bqs, internal_height_bqs;
- u32 num_hor_grids, num_ver_grids, bqs_per_grid_cell, tbl_width_bqs, tbl_height_bqs;
- u32 sensor_org_x_bqs_on_internal, sensor_org_y_bqs_on_internal, sensor_width_bqs, sensor_height_bqs;
- u32 sensor_center_x_bqs_on_internal, sensor_center_y_bqs_on_internal;
- u32 left, right, upper, lower;
- u32 adjust_left, adjust_right, adjust_upper, adjust_lower, adjust_width_bqs, adjust_height_bqs;
- u32 internal_org_x_bqs_on_tbl, internal_org_y_bqs_on_tbl;
- u32 sensor_org_x_bqs_on_tbl, sensor_org_y_bqs_on_tbl;
-
- assert(binary);
- assert(stream_config);
- assert(shading_info);
- assert(pipe_config);
-
- IA_CSS_ENTER_PRIVATE("binary=%p, required_bds_factor=%d, stream_config=%p",
- binary, required_bds_factor, stream_config);
-
- /* Initialize by default values. */
- *shading_info = DEFAULT_SHADING_INFO_TYPE_1;
-
- err = sh_css_binary_get_sc_requirements(binary, required_bds_factor, stream_config, &scr);
- if (err) {
- IA_CSS_LEAVE_ERR_PRIVATE(err);
- return err;
- }
-
- IA_CSS_LOG("binary: id=%d, sctbl=%dx%d, deci=%d",
- binary->info->sp.id, binary->sctbl_width_per_color, binary->sctbl_height, binary->deci_factor_log2);
- IA_CSS_LOG("binary: in=%dx%d, in_padded_w=%d, int=%dx%d, int_padded_w=%d, out=%dx%d, out_padded_w=%d",
- binary->in_frame_info.res.width, binary->in_frame_info.res.height, binary->in_frame_info.padded_width,
- binary->internal_frame_info.res.width, binary->internal_frame_info.res.height,
- binary->internal_frame_info.padded_width,
- binary->out_frame_info[0].res.width, binary->out_frame_info[0].res.height,
- binary->out_frame_info[0].padded_width);
-
- /* Set the input size from sensor, which includes left/top crop size. */
- in_width_bqs = _ISP_BQS(binary->in_frame_info.res.width);
- in_height_bqs = _ISP_BQS(binary->in_frame_info.res.height);
-
- /*
- * Frame size internally used in ISP, including sensor data and padding.
- * This is the frame size, to which the shading correction is applied.
- */
- internal_width_bqs = _ISP_BQS(binary->internal_frame_info.res.width);
- internal_height_bqs = _ISP_BQS(binary->internal_frame_info.res.height);
-
- /* Shading table. */
- num_hor_grids = binary->sctbl_width_per_color;
- num_ver_grids = binary->sctbl_height;
- bqs_per_grid_cell = (1 << binary->deci_factor_log2);
- tbl_width_bqs = (num_hor_grids - 1) * bqs_per_grid_cell;
- tbl_height_bqs = (num_ver_grids - 1) * bqs_per_grid_cell;
-
- IA_CSS_LOG("tbl_width_bqs=%d, tbl_height_bqs=%d", tbl_width_bqs, tbl_height_bqs);
-
- /*
- * Real sensor data area on the internal frame at shading correction.
- * Filters and scaling are applied to the internal frame before
- * shading correction, depending on the binary.
- */
- sensor_org_x_bqs_on_internal = scr.sensor_data_origin_x_bqs_on_internal;
- sensor_org_y_bqs_on_internal = scr.sensor_data_origin_y_bqs_on_internal;
- {
- unsigned int bs_frac = 8; /* scaling factor 1.0 in fixed point (8 == FRAC_ACC macro in ISP) */
- unsigned int bs_out, bs_in; /* scaling ratio in fixed point */
-
- bs_out = scr.bayer_scale_hor_ratio_out * bs_frac;
- bs_in = scr.bayer_scale_hor_ratio_in * bs_frac;
- sensor_width_bqs = (in_width_bqs * bs_out + bs_in / 2) / bs_in; /* "+ bs_in/2": rounding */
-
- bs_out = scr.bayer_scale_ver_ratio_out * bs_frac;
- bs_in = scr.bayer_scale_ver_ratio_in * bs_frac;
- sensor_height_bqs = (in_height_bqs * bs_out + bs_in / 2) / bs_in; /* "+ bs_in/2": rounding */
- }
-
- /* Center of the sensor data on the internal frame at shading correction. */
- sensor_center_x_bqs_on_internal = sensor_org_x_bqs_on_internal + sensor_width_bqs / 2;
- sensor_center_y_bqs_on_internal = sensor_org_y_bqs_on_internal + sensor_height_bqs / 2;
-
- /* Size of left/right/upper/lower sides of the sensor center on the internal frame. */
- left = sensor_center_x_bqs_on_internal;
- right = internal_width_bqs - sensor_center_x_bqs_on_internal;
- upper = sensor_center_y_bqs_on_internal;
- lower = internal_height_bqs - sensor_center_y_bqs_on_internal;
-
- /* Align the size of left/right/upper/lower sides to a multiple of the grid cell size. */
- adjust_left = CEIL_MUL(left, bqs_per_grid_cell);
- adjust_right = CEIL_MUL(right, bqs_per_grid_cell);
- adjust_upper = CEIL_MUL(upper, bqs_per_grid_cell);
- adjust_lower = CEIL_MUL(lower, bqs_per_grid_cell);
-
- /* Shading table should cover the adjusted frame size. */
- adjust_width_bqs = adjust_left + adjust_right;
- adjust_height_bqs = adjust_upper + adjust_lower;
-
- IA_CSS_LOG("adjust_width_bqs=%d, adjust_height_bqs=%d", adjust_width_bqs, adjust_height_bqs);
-
- if (adjust_width_bqs > tbl_width_bqs || adjust_height_bqs > tbl_height_bqs) {
- IA_CSS_LEAVE_ERR_PRIVATE(-EINVAL);
- return -EINVAL;
- }
-
- /* Origin of the internal frame on the shading table. */
- internal_org_x_bqs_on_tbl = adjust_left - left;
- internal_org_y_bqs_on_tbl = adjust_upper - upper;
-
- /* Origin of the real sensor data area on the shading table. */
- sensor_org_x_bqs_on_tbl = internal_org_x_bqs_on_tbl + sensor_org_x_bqs_on_internal;
- sensor_org_y_bqs_on_tbl = internal_org_y_bqs_on_tbl + sensor_org_y_bqs_on_internal;
-
- /* The shading information necessary as API is stored in the shading_info. */
- shading_info->info.type_1.num_hor_grids = num_hor_grids;
- shading_info->info.type_1.num_ver_grids = num_ver_grids;
- shading_info->info.type_1.bqs_per_grid_cell = bqs_per_grid_cell;
-
- shading_info->info.type_1.bayer_scale_hor_ratio_in = scr.bayer_scale_hor_ratio_in;
- shading_info->info.type_1.bayer_scale_hor_ratio_out = scr.bayer_scale_hor_ratio_out;
- shading_info->info.type_1.bayer_scale_ver_ratio_in = scr.bayer_scale_ver_ratio_in;
- shading_info->info.type_1.bayer_scale_ver_ratio_out = scr.bayer_scale_ver_ratio_out;
-
- shading_info->info.type_1.isp_input_sensor_data_res_bqs.width = in_width_bqs;
- shading_info->info.type_1.isp_input_sensor_data_res_bqs.height = in_height_bqs;
-
- shading_info->info.type_1.sensor_data_res_bqs.width = sensor_width_bqs;
- shading_info->info.type_1.sensor_data_res_bqs.height = sensor_height_bqs;
-
- shading_info->info.type_1.sensor_data_origin_bqs_on_sctbl.x = (int32_t)sensor_org_x_bqs_on_tbl;
- shading_info->info.type_1.sensor_data_origin_bqs_on_sctbl.y = (int32_t)sensor_org_y_bqs_on_tbl;
-
- /* The shading information related to ISP (but, not necessary as API) is stored in the pipe_config. */
- pipe_config->internal_frame_origin_bqs_on_sctbl.x = (int32_t)internal_org_x_bqs_on_tbl;
- pipe_config->internal_frame_origin_bqs_on_sctbl.y = (int32_t)internal_org_y_bqs_on_tbl;
-
- IA_CSS_LOG("shading_info: grids=%dx%d, cell=%d, scale=%d,%d,%d,%d, input=%dx%d, data=%dx%d, origin=(%d,%d)",
- shading_info->info.type_1.num_hor_grids,
- shading_info->info.type_1.num_ver_grids,
- shading_info->info.type_1.bqs_per_grid_cell,
- shading_info->info.type_1.bayer_scale_hor_ratio_in,
- shading_info->info.type_1.bayer_scale_hor_ratio_out,
- shading_info->info.type_1.bayer_scale_ver_ratio_in,
- shading_info->info.type_1.bayer_scale_ver_ratio_out,
- shading_info->info.type_1.isp_input_sensor_data_res_bqs.width,
- shading_info->info.type_1.isp_input_sensor_data_res_bqs.height,
- shading_info->info.type_1.sensor_data_res_bqs.width,
- shading_info->info.type_1.sensor_data_res_bqs.height,
- shading_info->info.type_1.sensor_data_origin_bqs_on_sctbl.x,
- shading_info->info.type_1.sensor_data_origin_bqs_on_sctbl.y);
-
- IA_CSS_LOG("pipe_config: origin=(%d,%d)",
- pipe_config->internal_frame_origin_bqs_on_sctbl.x,
- pipe_config->internal_frame_origin_bqs_on_sctbl.y);
-
- IA_CSS_LEAVE_ERR_PRIVATE(err);
- return err;
-}
-
int
ia_css_binary_get_shading_info(const struct ia_css_binary *binary, /* [in] */
@@ -706,24 +307,13 @@ ia_css_binary_get_shading_info(const struct ia_css_binary *binary, /* [in] */
IA_CSS_ENTER_PRIVATE("binary=%p, type=%d, required_bds_factor=%d, stream_config=%p",
binary, type, required_bds_factor, stream_config);
- if (type != IA_CSS_SHADING_CORRECTION_TYPE_1) {
- err = -ENOTSUPP;
-
- IA_CSS_LEAVE_ERR_PRIVATE(err);
- return err;
- }
-
- if (!IS_ISP2401)
- err = isp2400_binary_get_shading_info_type_1(binary,
- required_bds_factor,
- stream_config,
- shading_info);
+ if (type == IA_CSS_SHADING_CORRECTION_TYPE_1)
+ err = binary_get_shading_info_type_1(binary,
+ required_bds_factor,
+ stream_config,
+ shading_info);
else
- err = isp2401_binary_get_shading_info_type_1(binary,
- required_bds_factor,
- stream_config,
- shading_info,
- pipe_config);
+ err = -ENOTSUPP;
IA_CSS_LEAVE_ERR_PRIVATE(err);
return err;
@@ -805,11 +395,7 @@ ia_css_binary_3a_grid_info(const struct ia_css_binary *binary,
s3a_info->deci_factor_log2 = binary->deci_factor_log2;
s3a_info->elem_bit_depth = SH_CSS_BAYER_BITS;
s3a_info->use_dmem = binary->info->sp.s3a.s3atbl_use_dmem;
-#if defined(HAS_NO_HMEM)
- s3a_info->has_histogram = 1;
-#else
s3a_info->has_histogram = 0;
-#endif
IA_CSS_LEAVE_ERR_PRIVATE(err);
return err;
}
@@ -965,15 +551,9 @@ binary_grid_deci_factor_log2(int width, int height)
/* 3A/Shading decimation factor spcification (at August 2008)
* ------------------------------------------------------------------
* [Image Width (BQ)] [Decimation Factor (BQ)] [Resulting grid cells]
- #ifndef ISP2401
* 1280 ?c 32 40 ?c
* 640 ?c 1279 16 40 ?c 80
* ?c 639 8 ?c 80
- #else
- * from 1280 32 from 40
- * from 640 to 1279 16 from 40 to 80
- * to 639 8 to 80
- #endif
* ------------------------------------------------------------------
*/
/* Maximum and minimum decimation factor by the specification */
@@ -1335,26 +915,14 @@ ia_css_binary_fill_info(const struct ia_css_binary_xinfo *xinfo,
if (info->enable.sc)
{
- if (!IS_ISP2401) {
- binary->sctbl_width_per_color = _ISP2400_SCTBL_WIDTH_PER_COLOR(sc_3a_dis_padded_width, s3a_log_deci);
- binary->sctbl_aligned_width_per_color = ISP2400_SH_CSS_MAX_SCTBL_ALIGNED_WIDTH_PER_COLOR;
- binary->sctbl_height = _ISP2400_SCTBL_HEIGHT(sc_3a_dis_height, s3a_log_deci);
- } else {
- binary->sctbl_width_per_color = _ISP2401_SCTBL_WIDTH_PER_COLOR(isp_internal_width, s3a_log_deci);
- binary->sctbl_aligned_width_per_color = ISP2401_SH_CSS_MAX_SCTBL_ALIGNED_WIDTH_PER_COLOR;
- binary->sctbl_height = _ISP2401_SCTBL_HEIGHT(isp_internal_height, s3a_log_deci);
- binary->sctbl_legacy_width_per_color = _ISP_SCTBL_LEGACY_WIDTH_PER_COLOR(sc_3a_dis_padded_width, s3a_log_deci);
- binary->sctbl_legacy_height = _ISP_SCTBL_LEGACY_HEIGHT(sc_3a_dis_height, s3a_log_deci);
- }
+ binary->sctbl_width_per_color = _ISP_SCTBL_WIDTH_PER_COLOR(sc_3a_dis_padded_width, s3a_log_deci);
+ binary->sctbl_aligned_width_per_color = SH_CSS_MAX_SCTBL_ALIGNED_WIDTH_PER_COLOR;
+ binary->sctbl_height = _ISP_SCTBL_HEIGHT(sc_3a_dis_height, s3a_log_deci);
} else
{
binary->sctbl_width_per_color = 0;
binary->sctbl_aligned_width_per_color = 0;
binary->sctbl_height = 0;
- if (IS_ISP2401) {
- binary->sctbl_legacy_width_per_color = 0;
- binary->sctbl_legacy_height = 0;
- }
}
ia_css_sdis_init_info(&binary->dis,
sc_3a_dis_width,
@@ -1383,20 +951,13 @@ static int __ia_css_binary_find(struct ia_css_binary_descr *descr,
*req_vf_info;
struct ia_css_binary_xinfo *xcandidate;
-#ifndef ISP2401
bool need_ds, need_dz, need_dvs, need_xnr, need_dpc;
-#else
- bool need_ds, need_dz, need_dvs, need_xnr, need_dpc, need_tnr;
-#endif
bool striped;
bool enable_yuv_ds;
bool enable_high_speed;
bool enable_dvs_6axis;
bool enable_reduced_pipe;
bool enable_capture_pp_bli;
-#ifdef ISP2401
- bool enable_luma_only;
-#endif
int err = -EINVAL;
bool continuous;
unsigned int isp_pipe_version;
@@ -1418,41 +979,26 @@ static int __ia_css_binary_find(struct ia_css_binary_descr *descr,
stream_format = descr->stream_format;
req_in_info = descr->in_info;
req_bds_out_info = descr->bds_out_info;
- for (i = 0; i < IA_CSS_BINARY_MAX_OUTPUT_PORTS; i++)
- {
+ for (i = 0; i < IA_CSS_BINARY_MAX_OUTPUT_PORTS; i++) {
req_out_info[i] = descr->out_info[i];
if (req_out_info[i] && (req_out_info[i]->res.width != 0))
req_bin_out_info = req_out_info[i];
}
if (!req_bin_out_info)
return -EINVAL;
-#ifndef ISP2401
req_vf_info = descr->vf_info;
-#else
-
- if ((descr->vf_info) && (descr->vf_info->res.width == 0))
- /* width==0 means that there is no vf pin (e.g. in SkyCam preview case) */
- req_vf_info = NULL;
- else
- req_vf_info = descr->vf_info;
-#endif
need_xnr = descr->enable_xnr;
need_ds = descr->enable_fractional_ds;
need_dz = false;
need_dvs = false;
need_dpc = descr->enable_dpc;
-#ifdef ISP2401
- need_tnr = descr->enable_tnr;
-#endif
+
enable_yuv_ds = descr->enable_yuv_ds;
enable_high_speed = descr->enable_high_speed;
enable_dvs_6axis = descr->enable_dvs_6axis;
enable_reduced_pipe = descr->enable_reduced_pipe;
enable_capture_pp_bli = descr->enable_capture_pp_bli;
-#ifdef ISP2401
- enable_luma_only = descr->enable_luma_only;
-#endif
continuous = descr->continuous;
striped = descr->striped;
isp_pipe_version = descr->isp_pipe_version;
@@ -1462,8 +1008,7 @@ static int __ia_css_binary_find(struct ia_css_binary_descr *descr,
internal_res.width = 0;
internal_res.height = 0;
- if (mode == IA_CSS_BINARY_MODE_VIDEO)
- {
+ if (mode == IA_CSS_BINARY_MODE_VIDEO) {
dvs_env = descr->dvs_env;
need_dz = descr->enable_dz;
/* Video is the only mode that has a nodz variant. */
@@ -1472,8 +1017,7 @@ static int __ia_css_binary_find(struct ia_css_binary_descr *descr,
/* print a map of the binary file */
ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "BINARY INFO:\n");
- for (i = 0; i < IA_CSS_BINARY_NUM_MODES; i++)
- {
+ for (i = 0; i < IA_CSS_BINARY_NUM_MODES; i++) {
xcandidate = binary_infos[i];
if (xcandidate) {
ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "%d:\n", i);
@@ -1488,8 +1032,7 @@ static int __ia_css_binary_find(struct ia_css_binary_descr *descr,
/* printf("sh_css_binary_find: pipe version %d\n", isp_pipe_version); */
for (xcandidate = binary_infos[mode]; xcandidate;
- xcandidate = xcandidate->next)
- {
+ xcandidate = xcandidate->next) {
struct ia_css_binary_info *candidate = &xcandidate->sp;
/* printf("sh_css_binary_find: evaluating candidate:
* %d\n",candidate->id); */
@@ -1747,24 +1290,6 @@ static int __ia_css_binary_find(struct ia_css_binary_descr *descr,
continue;
}
-#ifdef ISP2401
- if (candidate->enable.luma_only != enable_luma_only) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE,
- "ia_css_binary_find() [%d] continue: %d != %d\n",
- __LINE__, candidate->enable.luma_only,
- descr->enable_luma_only);
- continue;
- }
-
- if (!candidate->enable.tnr && need_tnr) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE,
- "ia_css_binary_find() [%d] continue: !%d && %d\n",
- __LINE__, candidate->enable.tnr,
- descr->enable_tnr);
- continue;
- }
-
-#endif
/* reconfigure any variable properties of the binary */
err = ia_css_binary_fill_info(xcandidate, online, two_ppc,
stream_format, req_in_info,
diff --git a/drivers/staging/media/atomisp/pci/runtime/debug/src/ia_css_debug.c b/drivers/staging/media/atomisp/pci/runtime/debug/src/ia_css_debug.c
index 05ce0f7..f462387 100644
--- a/drivers/staging/media/atomisp/pci/runtime/debug/src/ia_css_debug.c
+++ b/drivers/staging/media/atomisp/pci/runtime/debug/src/ia_css_debug.c
@@ -2821,7 +2821,7 @@ ia_css_debug_pipe_graph_dump_stage(
"in", true);
}
- for (i = 0; i < NUM_TNR_FRAMES; i++) {
+ for (i = 0; i < NUM_VIDEO_TNR_FRAMES; i++) {
if (stage->args.tnr_frames[i]) {
ia_css_debug_pipe_graph_dump_frame(
stage->args.tnr_frames[i], id,
diff --git a/drivers/staging/media/atomisp/pci/runtime/frame/interface/ia_css_frame.h b/drivers/staging/media/atomisp/pci/runtime/frame/interface/ia_css_frame.h
index 31f01e0..c756a13 100644
--- a/drivers/staging/media/atomisp/pci/runtime/frame/interface/ia_css_frame.h
+++ b/drivers/staging/media/atomisp/pci/runtime/frame/interface/ia_css_frame.h
@@ -138,27 +138,7 @@ bool ia_css_frame_is_same_type(
* @param[in] info The frame info
* @return
*/
-void ia_css_dma_configure_from_info(
- struct dma_port_config *config,
- const struct ia_css_frame_info *info);
-
-/* ISP2401 */
-/* @brief Finds the cropping resolution
- * This function finds the maximum cropping resolution in an input image keeping
- * the aspect ratio for the given output resolution.Calculates the coordinates
- * for cropping from the center and returns the starting pixel location of the
- * region in the input image. Also returns the dimension of the cropping
- * resolution.
- *
- * @param
- * @param[in] in_res Resolution of input image
- * @param[in] out_res Resolution of output image
- * @param[out] crop_res Crop resolution of input image
- * @return Returns 0 or -EINVAL on error
- */
-int
-ia_css_frame_find_crop_resolution(const struct ia_css_resolution *in_res,
- const struct ia_css_resolution *out_res,
- struct ia_css_resolution *crop_res);
+int ia_css_dma_configure_from_info(struct dma_port_config *config,
+ const struct ia_css_frame_info *info);
#endif /* __IA_CSS_FRAME_H__ */
diff --git a/drivers/staging/media/atomisp/pci/runtime/frame/src/frame.c b/drivers/staging/media/atomisp/pci/runtime/frame/src/frame.c
index 10c4907..a3aae63 100644
--- a/drivers/staging/media/atomisp/pci/runtime/frame/src/frame.c
+++ b/drivers/staging/media/atomisp/pci/runtime/frame/src/frame.c
@@ -168,26 +168,24 @@ int ia_css_frame_map(struct ia_css_frame **frame,
if (err)
return err;
- if (!err) {
- if (pgnr < ((PAGE_ALIGN(me->data_bytes)) >> PAGE_SHIFT)) {
- dev_err(atomisp_dev,
- "user space memory size is less than the expected size..\n");
- err = -ENOMEM;
- goto error;
- } else if (pgnr > ((PAGE_ALIGN(me->data_bytes)) >> PAGE_SHIFT)) {
- dev_err(atomisp_dev,
- "user space memory size is large than the expected size..\n");
- err = -ENOMEM;
- goto error;
- }
-
- me->data = hmm_alloc(me->data_bytes, HMM_BO_USER, 0, data,
- attribute & ATOMISP_MAP_FLAG_CACHED);
-
- if (me->data == mmgr_NULL)
- err = -EINVAL;
+ if (pgnr < ((PAGE_ALIGN(me->data_bytes)) >> PAGE_SHIFT)) {
+ dev_err(atomisp_dev,
+ "user space memory size is less than the expected size..\n");
+ err = -ENOMEM;
+ goto error;
+ } else if (pgnr > ((PAGE_ALIGN(me->data_bytes)) >> PAGE_SHIFT)) {
+ dev_err(atomisp_dev,
+ "user space memory size is large than the expected size..\n");
+ err = -ENOMEM;
+ goto error;
}
+ me->data = hmm_alloc(me->data_bytes, HMM_BO_USER, 0, data,
+ attribute & ATOMISP_MAP_FLAG_CACHED);
+
+ if (me->data == mmgr_NULL)
+ err = -EINVAL;
+
error:
if (err) {
kvfree(me);
@@ -594,10 +592,8 @@ bool ia_css_frame_is_same_type(const struct ia_css_frame *frame_a,
return is_equal;
}
-void
-ia_css_dma_configure_from_info(
- struct dma_port_config *config,
- const struct ia_css_frame_info *info)
+int ia_css_dma_configure_from_info(struct dma_port_config *config,
+ const struct ia_css_frame_info *info)
{
unsigned int is_raw_packed = info->format == IA_CSS_FRAME_FORMAT_RAW_PACKED;
unsigned int bits_per_pixel = is_raw_packed ? info->raw_bit_depth :
@@ -610,7 +606,13 @@ ia_css_dma_configure_from_info(
config->elems = (uint8_t)elems_b;
config->width = (uint16_t)info->res.width;
config->crop = 0;
- assert(config->width <= info->padded_width);
+
+ if (config->width > info->padded_width) {
+ dev_err(atomisp_dev, "internal error: padded_width is too small!\n");
+ return -EINVAL;
+ }
+
+ return 0;
}
/**************************************************************************
@@ -931,73 +933,3 @@ void ia_css_resolution_to_sp_resolution(
to->width = (uint16_t)from->width;
to->height = (uint16_t)from->height;
}
-
-/* ISP2401 */
-int
-ia_css_frame_find_crop_resolution(const struct ia_css_resolution *in_res,
- const struct ia_css_resolution *out_res,
- struct ia_css_resolution *crop_res) {
- u32 wd_even_ceil, ht_even_ceil;
- u32 in_ratio, out_ratio;
-
- if ((!in_res) || (!out_res) || (!crop_res))
- return -EINVAL;
-
- IA_CSS_ENTER_PRIVATE("in(%ux%u) -> out(%ux%u)", in_res->width,
- in_res->height, out_res->width, out_res->height);
-
- if ((in_res->width == 0)
- || (in_res->height == 0)
- || (out_res->width == 0)
- || (out_res->height == 0))
- return -EINVAL;
-
- if ((out_res->width > in_res->width) ||
- (out_res->height > in_res->height))
- return -EINVAL;
-
- /* If aspect ratio (width/height) of out_res is higher than the aspect
- * ratio of the in_res, then we crop vertically, otherwise we crop
- * horizontally.
- */
- in_ratio = in_res->width * out_res->height;
- out_ratio = out_res->width * in_res->height;
-
- if (in_ratio == out_ratio)
- {
- crop_res->width = in_res->width;
- crop_res->height = in_res->height;
- } else if (out_ratio > in_ratio)
- {
- crop_res->width = in_res->width;
- crop_res->height = ROUND_DIV(out_res->height * crop_res->width,
- out_res->width);
- } else
- {
- crop_res->height = in_res->height;
- crop_res->width = ROUND_DIV(out_res->width * crop_res->height,
- out_res->height);
- }
-
- /* Round new (cropped) width and height to an even number.
- * binarydesc_calculate_bds_factor is such that we should consider as
- * much of the input as possible. This is different only when we end up
- * with an odd number in the last step. So, we take the next even number
- * if it falls within the input, otherwise take the previous even no.
- */
- wd_even_ceil = EVEN_CEIL(crop_res->width);
- ht_even_ceil = EVEN_CEIL(crop_res->height);
- if ((wd_even_ceil > in_res->width) || (ht_even_ceil > in_res->height))
- {
- crop_res->width = EVEN_FLOOR(crop_res->width);
- crop_res->height = EVEN_FLOOR(crop_res->height);
- } else
- {
- crop_res->width = wd_even_ceil;
- crop_res->height = ht_even_ceil;
- }
-
- IA_CSS_LEAVE_PRIVATE("in(%ux%u) -> out(%ux%u)", crop_res->width,
- crop_res->height, out_res->width, out_res->height);
- return 0;
-}
diff --git a/drivers/staging/media/atomisp/pci/runtime/isys/src/virtual_isys.c b/drivers/staging/media/atomisp/pci/runtime/isys/src/virtual_isys.c
index 82f3c19..8fc7746 100644
--- a/drivers/staging/media/atomisp/pci/runtime/isys/src/virtual_isys.c
+++ b/drivers/staging/media/atomisp/pci/runtime/isys/src/virtual_isys.c
@@ -189,17 +189,6 @@ ia_css_isys_error_t ia_css_isys_stream_create(
return false;
}
-#ifdef ISP2401
- /*
- * Early polling is required for timestamp accuracy in certain cause.
- * The ISYS HW polling is started on
- * ia_css_isys_stream_capture_indication() instead of
- * ia_css_pipeline_sp_wait_for_isys_stream_N() as isp processing of
- * capture takes longer than getting an ISYS frame
- */
- isys_stream->polling_mode = isys_stream_descr->polling_mode;
-
-#endif
/* create metadata channel */
if (isys_stream_descr->metadata.enable) {
rc = create_input_system_channel(isys_stream_descr, true,
diff --git a/drivers/staging/media/atomisp/pci/runtime/pipeline/src/pipeline.c b/drivers/staging/media/atomisp/pci/runtime/pipeline/src/pipeline.c
index d03957d..dfc5024 100644
--- a/drivers/staging/media/atomisp/pci/runtime/pipeline/src/pipeline.c
+++ b/drivers/staging/media/atomisp/pci/runtime/pipeline/src/pipeline.c
@@ -140,8 +140,7 @@ void ia_css_pipeline_start(enum ia_css_pipe_id pipe_id,
false, false, false, true, SH_CSS_BDS_FACTOR_1_00,
SH_CSS_PIPE_CONFIG_OVRD_NO_OVRD,
IA_CSS_INPUT_MODE_MEMORY, NULL, NULL,
- (enum mipi_port_id)0,
- NULL, NULL);
+ (enum mipi_port_id)0);
ia_css_pipeline_get_sp_thread_id(pipe_num, &thread_id);
if (!sh_css_sp_is_running()) {
diff --git a/drivers/staging/media/atomisp/pci/sh_css.c b/drivers/staging/media/atomisp/pci/sh_css.c
index c4b35cb..1d605e5 100644
--- a/drivers/staging/media/atomisp/pci/sh_css.c
+++ b/drivers/staging/media/atomisp/pci/sh_css.c
@@ -20,6 +20,8 @@
#include "hmm.h"
+#include "atomisp_internal.h"
+
#include "ia_css.h"
#include "sh_css_hrt.h" /* only for file 2 MIPI */
#include "ia_css_buffer.h"
@@ -62,9 +64,9 @@
#include "ia_css_mmu_private.h" /* sh_css_mmu_set_page_table_base_index() */
#include "gdc_device.h" /* HRT_GDC_N */
#include "dma.h" /* dma_set_max_burst_size() */
-#include "irq.h" /* virq */
-#include "sp.h" /* cnd_sp_irq_enable() */
-#include "isp.h" /* cnd_isp_irq_enable, ISP_VEC_NELEMS */
+#include "irq.h" /* virq */
+#include "sp.h" /* cnd_sp_irq_enable() */
+#include "isp.h" /* cnd_isp_irq_enable, ISP_VEC_NELEMS */
#include "gp_device.h" /* gp_device_reg_store() */
#define __INLINE_GPIO__
#include "gpio.h"
@@ -74,15 +76,6 @@
#define SH_CSS_VIDEO_BUFFER_ALIGNMENT 0
-#if WITH_PC_MONITORING
-#define MULTIPLE_SAMPLES 1
-#define NOF_SAMPLES 60
-#include "linux/kthread.h"
-#include "linux/sched.h"
-#include "linux/delay.h"
-#include "sh_css_metrics.h"
-static int thread_alive;
-#endif /* WITH_PC_MONITORING */
#include "ia_css_spctrl.h"
#include "ia_css_version_data.h"
@@ -97,8 +90,9 @@ static int thread_alive;
/* Size of Refcount List */
#define REFCOUNT_SIZE 1000
-/* for JPEG, we don't know the length of the image upfront,
- * but since we support sensor upto 16MP, we take this as
+/*
+ * for JPEG, we don't know the length of the image upfront,
+ * but since we support sensor up to 16MP, we take this as
* upper limit.
*/
#define JPEG_BYTES (16 * 1024 * 1024)
@@ -110,9 +104,10 @@ struct sh_css my_css;
int __printf(1, 0) (*sh_css_printf)(const char *fmt, va_list args) = NULL;
-/* modes of work: stream_create and stream_destroy will update the save/restore data
- only when in working mode, not suspend/resume
-*/
+/*
+ * modes of work: stream_create and stream_destroy will update the save/restore
+ * data only when in working mode, not suspend/resume
+ */
enum ia_sh_css_modes {
sh_css_mode_none = 0,
sh_css_mode_working,
@@ -120,40 +115,55 @@ enum ia_sh_css_modes {
sh_css_mode_resume
};
-/* a stream seed, to save and restore the stream data.
- the stream seed contains all the data required to "grow" the seed again after it was closed.
+/**
+ * struct sh_css_stream_seed - a stream seed, to save and restore the
+ * stream data.
+ *
+ * @orig_stream: pointer to restore the original handle
+ * @stream: handle, used as ID too.
+ * @stream_config: stream config struct
+ * @num_pipes: number of pipes
+ * @pipes: pipe handles
+ * @orig_pipes: pointer to restore original handle
+ * @pipe_config: pipe config structs
+ *
+ * the stream seed contains all the data required to "grow" the seed again
+ * after it was closed.
*/
struct sh_css_stream_seed {
- struct ia_css_stream
- **orig_stream; /* pointer to restore the original handle */
- struct ia_css_stream *stream; /* handle, used as ID too.*/
- struct ia_css_stream_config stream_config; /* stream config struct */
+ struct ia_css_stream **orig_stream;
+ struct ia_css_stream *stream;
+ struct ia_css_stream_config stream_config;
int num_pipes;
- struct ia_css_pipe *pipes[IA_CSS_PIPE_ID_NUM]; /* pipe handles */
- struct ia_css_pipe
- **orig_pipes[IA_CSS_PIPE_ID_NUM]; /* pointer to restore original handle */
- struct ia_css_pipe_config
- pipe_config[IA_CSS_PIPE_ID_NUM]; /* pipe config structs */
+ struct ia_css_pipe *pipes[IA_CSS_PIPE_ID_NUM];
+ struct ia_css_pipe **orig_pipes[IA_CSS_PIPE_ID_NUM];
+ struct ia_css_pipe_config pipe_config[IA_CSS_PIPE_ID_NUM];
};
#define MAX_ACTIVE_STREAMS 5
-/* A global struct for save/restore to hold all the data that should sustain power-down:
- MMU base, IRQ type, env for routines, binary loaded FW and the stream seeds.
-*/
+/*
+ * A global struct for save/restore to hold all the data that should
+ * sustain power-down: MMU base, IRQ type, env for routines, binary loaded FW
+ * and the stream seeds.
+ */
struct sh_css_save {
enum ia_sh_css_modes mode;
- u32 mmu_base; /* the last mmu_base */
+ u32 mmu_base; /* the last mmu_base */
enum ia_css_irq_type irq_type;
struct sh_css_stream_seed stream_seeds[MAX_ACTIVE_STREAMS];
- struct ia_css_fw *loaded_fw; /* fw struct previously loaded */
- struct ia_css_env driver_env; /* driver-supplied env copy */
+ struct ia_css_fw *loaded_fw; /* fw struct previously loaded */
+ struct ia_css_env driver_env; /* driver-supplied env copy */
};
static bool my_css_save_initialized; /* if my_css_save was initialized */
static struct sh_css_save my_css_save;
-/* pqiao NOTICE: this is for css internal buffer recycling when stopping pipeline,
- this array is temporary and will be replaced by resource manager*/
+/*
+ * pqiao NOTICE: this is for css internal buffer recycling when stopping
+ * pipeline,
+ * this array is temporary and will be replaced by resource manager
+ */
+
/* Taking the biggest Size for number of Elements */
#define MAX_HMM_BUFFER_NUM \
(SH_CSS_MAX_NUM_QUEUES * (IA_CSS_NUM_ELEMS_SP2HOST_BUFFER_QUEUE + 2))
@@ -181,27 +191,6 @@ allocate_delay_frames(struct ia_css_pipe *pipe);
static int
sh_css_pipe_start(struct ia_css_stream *stream);
-/* ISP 2401 */
-/*
- * @brief Stop all "ia_css_pipe" instances in the target
- * "ia_css_stream" instance.
- *
- * @param[in] stream Point to the target "ia_css_stream" instance.
- *
- * @return
- * - 0, if the "stop" requests have been successfully sent out.
- * - CSS error code, otherwise.
- *
- *
- * NOTE
- * This API sends the "stop" requests to the "ia_css_pipe"
- * instances in the same "ia_css_stream" instance. It will
- * return without waiting for all "ia_css_pipe" instatnces
- * being stopped.
- */
-static int
-sh_css_pipes_stop(struct ia_css_stream *stream);
-
/*
* @brief Check if all "ia_css_pipe" instances in the target
* "ia_css_stream" instance have stopped.
@@ -213,9 +202,6 @@ sh_css_pipes_stop(struct ia_css_stream *stream);
* instance have ben stopped.
* - false, otherwise.
*/
-/* ISP 2401 */
-static bool
-sh_css_pipes_have_stopped(struct ia_css_stream *stream);
/* ISP 2401 */
static int
@@ -224,9 +210,6 @@ ia_css_pipe_check_format(struct ia_css_pipe *pipe,
/* ISP 2401 */
static int
-check_pipe_resolutions(const struct ia_css_pipe *pipe);
-
-static int
ia_css_pipe_load_extension(struct ia_css_pipe *pipe,
struct ia_css_fw_info *firmware);
@@ -384,10 +367,6 @@ ia_css_get_acc_configs(
struct ia_css_pipe *pipe,
struct ia_css_isp_config *config);
-#if CONFIG_ON_FRAME_ENQUEUE()
-static int set_config_on_frame_enqueue(struct ia_css_frame_info
- *info, struct frame_data_wrapper *frame);
-#endif
#ifdef ISP2401
static unsigned int get_crop_lines_for_bayer_order(const struct
@@ -396,17 +375,7 @@ static unsigned int get_crop_columns_for_bayer_order(const struct
ia_css_stream_config *config);
static void get_pipe_extra_pixel(struct ia_css_pipe *pipe,
unsigned int *extra_row, unsigned int *extra_column);
-static int
-aspect_ratio_crop_init(struct ia_css_stream *curr_stream,
- struct ia_css_pipe *pipes[],
- bool *do_crop_status);
-static bool
-aspect_ratio_crop_check(bool enabled, struct ia_css_pipe *curr_pipe);
-
-static int
-aspect_ratio_crop(struct ia_css_pipe *curr_pipe,
- struct ia_css_resolution *effective_res);
#endif
static void
@@ -447,9 +416,10 @@ static enum ia_css_frame_format yuv422_copy_formats[] = {
IA_CSS_FRAME_FORMAT_YUYV
};
-/* Verify whether the selected output format is can be produced
+/*
+ * Verify whether the selected output format is can be produced
* by the copy binary given the stream format.
- * */
+ */
static int
verify_copy_out_frame_format(struct ia_css_pipe *pipe)
{
@@ -522,6 +492,7 @@ ia_css_stream_input_format_bits_per_pixel(struct ia_css_stream *stream)
return bpp;
}
+/* TODO: move define to proper file in tools */
#define GP_ISEL_TPG_MODE 0x90058
#if !defined(ISP2401)
@@ -573,12 +544,8 @@ sh_css_config_input_network(struct ia_css_stream *stream)
vblank_cycles = vblank_lines * (width + hblank_cycles);
sh_css_sp_configure_sync_gen(width, height, hblank_cycles,
vblank_cycles);
- if (!IS_ISP2401) {
- if (pipe->stream->config.mode == IA_CSS_INPUT_MODE_TPG) {
- /* TODO: move define to proper file in tools */
- ia_css_device_store_uint32(GP_ISEL_TPG_MODE, 0);
- }
- }
+ if (pipe->stream->config.mode == IA_CSS_INPUT_MODE_TPG)
+ ia_css_device_store_uint32(GP_ISEL_TPG_MODE, 0);
}
ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
"sh_css_config_input_network() leave:\n");
@@ -689,13 +656,13 @@ static unsigned int csi2_protocol_calculate_max_subpixels_per_line(
/*
* The frame format layout is shown below.
*
- * Line 0: Pixel Pixel ... Pixel
- * Line 1: Pixel Pixel ... Pixel
- * Line 2: Pixel Pixel ... Pixel
- * Line 3: Pixel Pixel ... Pixel
+ * Line 0: Pixel ... Pixel
+ * Line 1: Pixel ... Pixel
+ * Line 2: Pixel ... Pixel
+ * Line 3: Pixel ... Pixel
* ...
- * Line (n-2): Pixel Pixel ... Pixel
- * Line (n-1): Pixel Pixel ... Pixel
+ * Line (n-2): Pixel ... Pixel
+ * Line (n-1): Pixel ... Pixel
*
* In this frame format, the even-line is
* as wide as the odd-line.
@@ -906,8 +873,10 @@ static bool sh_css_translate_stream_cfg_to_input_system_input_port_attr(
isys_stream_descr->metadata.lines_per_frame =
stream_cfg->metadata_config.resolution.height;
#ifdef ISP2401
- /* For new input system, number of str2mmio requests must be even.
- * So we round up number of metadata lines to be even. */
+ /*
+ * For new input system, number of str2mmio requests must be even.
+ * So we round up number of metadata lines to be even.
+ */
if (isys_stream_descr->metadata.lines_per_frame > 0)
isys_stream_descr->metadata.lines_per_frame +=
(isys_stream_descr->metadata.lines_per_frame & 1);
@@ -1003,22 +972,10 @@ static bool sh_css_translate_stream_cfg_to_isys_stream_descr(
isys_stream_descr->raw_packed = stream_cfg->pack_raw_pixels;
isys_stream_descr->linked_isys_stream_id = (int8_t)
stream_cfg->isys_config[isys_stream_idx].linked_isys_stream_id;
- /*
- * Early polling is required for timestamp accuracy in certain case.
- * The ISYS HW polling is started on
- * ia_css_isys_stream_capture_indication() instead of
- * ia_css_pipeline_sp_wait_for_isys_stream_N() as isp processing of
- * capture takes longer than getting an ISYS frame
- *
- * Only 2401 relevant ??
- */
-#if 0 // FIXME: NOT USED on Yocto Aero
- isys_stream_descr->polling_mode
- = early_polling ? INPUT_SYSTEM_POLL_ON_CAPTURE_REQUEST
- : INPUT_SYSTEM_POLL_ON_WAIT_FOR_FRAME;
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "sh_css_translate_stream_cfg_to_isys_stream_descr() leave:\n");
-#endif
+
+ if (IS_ISP2401)
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "sh_css_translate_stream_cfg_to_isys_stream_descr() leave:\n");
return rc;
}
@@ -1065,17 +1022,6 @@ sh_css_config_input_network(struct ia_css_stream *stream)
pipe = stream->last_pipe->pipe_settings.video.copy_pipe;
} else {
pipe = stream->last_pipe;
- if (stream->last_pipe->config.mode == IA_CSS_PIPE_MODE_CAPTURE) {
- /*
- * We need to poll the ISYS HW in capture_indication itself
- * for "non-continuous" capture usecase for getting accurate
- * isys frame capture timestamps.
- * This is because the capturepipe propcessing takes longer
- * to execute than the input system frame capture.
- * 2401 specific
- */
- early_polling = true;
- }
}
if (!pipe)
@@ -1086,7 +1032,8 @@ sh_css_config_input_network(struct ia_css_stream *stream)
binary = pipe->pipeline.stages->binary;
if (binary) {
- /* this was being done in ifmtr in 2400.
+ /*
+ * this was being done in ifmtr in 2400.
* online and cont bypass the init_in_frameinfo_memory_defaults
* so need to do it here
*/
@@ -1259,89 +1206,6 @@ static inline int stream_unregister_with_csi_rx(
}
#endif
-#if WITH_PC_MONITORING
-static struct task_struct *my_kthread; /* Handle for the monitoring thread */
-static int sh_binary_running; /* Enable sampling in the thread */
-
-static void print_pc_histo(char *core_name, struct sh_css_pc_histogram *hist)
-{
- unsigned int i;
- unsigned int cnt_run = 0;
- unsigned int cnt_stall = 0;
-
- if (!hist)
- return;
-
- sh_css_print("%s histogram length = %d\n", core_name, hist->length);
- sh_css_print("%s PC\turn\tstall\n", core_name);
-
- for (i = 0; i < hist->length; i++) {
- if ((hist->run[i] == 0) && (hist->run[i] == hist->stall[i]))
- continue;
- sh_css_print("%s %d\t%d\t%d\n",
- core_name, i, hist->run[i], hist->stall[i]);
- cnt_run += hist->run[i];
- cnt_stall += hist->stall[i];
- }
-
- sh_css_print(" Statistics for %s, cnt_run = %d, cnt_stall = %d, hist->length = %d\n",
- core_name, cnt_run, cnt_stall, hist->length);
-}
-
-static void print_pc_histogram(void)
-{
- struct ia_css_binary_metrics *metrics;
-
- for (metrics = sh_css_metrics.binary_metrics;
- metrics;
- metrics = metrics->next) {
- if (metrics->mode == IA_CSS_BINARY_MODE_PREVIEW ||
- metrics->mode == IA_CSS_BINARY_MODE_VF_PP) {
- sh_css_print("pc_histogram for binary %d is SKIPPED\n",
- metrics->id);
- continue;
- }
-
- sh_css_print(" pc_histogram for binary %d\n", metrics->id);
- print_pc_histo(" ISP", &metrics->isp_histogram);
- print_pc_histo(" SP", &metrics->sp_histogram);
- sh_css_print("print_pc_histogram() done for binary->id = %d, done.\n",
- metrics->id);
- }
-
- sh_css_print("PC_MONITORING:print_pc_histogram() -- DONE\n");
-}
-
-static int pc_monitoring(void *data)
-{
- int i = 0;
-
- (void)data;
- while (true) {
- if (sh_binary_running) {
- sh_css_metrics_sample_pcs();
-#if MULTIPLE_SAMPLES
- for (i = 0; i < NOF_SAMPLES; i++)
- sh_css_metrics_sample_pcs();
-#endif
- }
- usleep_range(10, 50);
- }
- return 0;
-}
-
-static void spying_thread_create(void)
-{
- my_kthread = kthread_run(pc_monitoring, NULL, "sh_pc_monitor");
- sh_css_metrics_enable_pc_histogram(1);
-}
-
-static void input_frame_info(struct ia_css_frame_info frame_info)
-{
- sh_css_print("SH_CSS:input_frame_info() -- frame->info.res.width = %d, frame->info.res.height = %d, format = %d\n",
- frame_info.res.width, frame_info.res.height, frame_info.format);
-}
-#endif /* WITH_PC_MONITORING */
static void
start_binary(struct ia_css_pipe *pipe,
@@ -1353,15 +1217,6 @@ start_binary(struct ia_css_pipe *pipe,
if (binary)
sh_css_metrics_start_binary(&binary->metrics);
-#if WITH_PC_MONITORING
- sh_css_print("PC_MONITORING: %s() -- binary id = %d , enable_dvs_envelope = %d\n",
- __func__, binary->info->sp.id,
- binary->info->sp.enable.dvs_envelope);
- input_frame_info(binary->in_frame_info);
-
- if (binary && binary->info->sp.pipeline.mode == IA_CSS_BINARY_MODE_VIDEO)
- sh_binary_running = true;
-#endif
#if !defined(ISP2401)
if (pipe->stream->reconfigure_css_rx) {
@@ -1406,7 +1261,7 @@ void sh_css_binary_args_reset(struct sh_css_binary_args *args)
{
unsigned int i;
- for (i = 0; i < NUM_TNR_FRAMES; i++)
+ for (i = 0; i < NUM_VIDEO_TNR_FRAMES; i++)
args->tnr_frames[i] = NULL;
for (i = 0; i < MAX_NUM_VIDEO_DELAY_FRAMES; i++)
args->delay_frames[i] = NULL;
@@ -1424,20 +1279,11 @@ static void start_pipe(
enum sh_css_pipe_config_override copy_ovrd,
enum ia_css_input_mode input_mode)
{
- const struct ia_css_coordinate *coord = NULL;
- const struct ia_css_isp_parameters *params = NULL;
-
-
IA_CSS_ENTER_PRIVATE("me = %p, copy_ovrd = %d, input_mode = %d",
me, copy_ovrd, input_mode);
assert(me); /* all callers are in this file and call with non null argument */
- if (!IS_ISP2401) {
- coord = &me->config.internal_frame_origin_bqs_on_sctbl;
- params = me->stream->isp_params_configs;
- }
-
sh_css_sp_init_pipeline(&me->pipeline,
me->mode,
(uint8_t)ia_css_pipe_get_pipe_num(me),
@@ -1452,9 +1298,7 @@ static void start_pipe(
&me->stream->info.metadata_info
, (input_mode == IA_CSS_INPUT_MODE_MEMORY) ?
(enum mipi_port_id)0 :
- me->stream->config.source.port.port,
- coord,
- params);
+ me->stream->config.source.port.port);
if (me->config.mode != IA_CSS_PIPE_MODE_COPY) {
struct ia_css_pipeline_stage *stage;
@@ -1565,17 +1409,18 @@ ia_css_reset_defaults(struct sh_css *css)
/* Reset everything to zero */
memset(&default_css, 0, sizeof(default_css));
- /* Initialize the non zero values*/
+ /* Initialize the non zero values */
default_css.check_system_idle = true;
default_css.num_cont_raw_frames = NUM_CONTINUOUS_FRAMES;
- /* All should be 0: but memset does it already.
+ /*
+ * All should be 0: but memset does it already.
* default_css.num_mipi_frames[N_CSI_PORTS] = 0;
*/
default_css.irq_type = IA_CSS_IRQ_TYPE_EDGE;
- /*Set the defaults to the output */
+ /* Set the defaults to the output */
*css = default_css;
}
@@ -1638,6 +1483,7 @@ ia_css_init(struct device *dev, const struct ia_css_env *env,
* the SIZE_OF_XXX macro of the corresponding struct. If they are not
* equal, functionality will break.
*/
+
/* Check struct sh_css_ddr_address_map */
COMPILATION_ERROR_IF(sizeof(struct sh_css_ddr_address_map) != SIZE_OF_SH_CSS_DDR_ADDRESS_MAP_STRUCT);
/* Check struct host_sp_queues */
@@ -1707,8 +1553,11 @@ ia_css_init(struct device *dev, const struct ia_css_env *env,
mipi_init();
#ifndef ISP2401
- /* In case this has been programmed already, update internal
- data structure ... DEPRECATED */
+ /*
+ * In case this has been programmed already, update internal
+ * data structure ...
+ * DEPRECATED
+ */
my_css.page_table_base_index = mmu_get_page_table_base_index(MMU0_ID);
#endif
@@ -1746,9 +1595,8 @@ ia_css_init(struct device *dev, const struct ia_css_env *env,
return err;
}
fw_explicitly_loaded = false;
-#ifndef ISP2401
+
my_css_save.loaded_fw = (struct ia_css_fw *)fw;
-#endif
}
if (!sh_css_setup_spctrl_config(&sh_css_sp_fw, SP_PROG_NAME, &spctrl_cfg))
return -EINVAL;
@@ -1759,24 +1607,17 @@ ia_css_init(struct device *dev, const struct ia_css_env *env,
return err;
}
-#if WITH_PC_MONITORING
- if (!thread_alive) {
- thread_alive++;
- sh_css_print("PC_MONITORING: %s() -- create thread DISABLED\n",
- __func__);
- spying_thread_create();
- }
-#endif
if (!sh_css_hrt_system_is_idle()) {
IA_CSS_LEAVE_ERR(-EBUSY);
return -EBUSY;
}
- /* can be called here, queuing works, but:
- - when sp is started later, it will wipe queued items
- so for now we leave it for later and make sure
- updates are not called to frequently.
- sh_css_init_buffer_queues();
- */
+ /*
+ * can be called here, queuing works, but:
+ * - when sp is started later, it will wipe queued items
+ * so for now we leave it for later and make sure
+ * updates are not called to frequently.
+ * sh_css_init_buffer_queues();
+ */
#if defined(ISP2401)
gp_device_reg_store(GP_DEVICE0_ID, _REG_GP_SWITCH_ISYS2401_ADDR, 1);
@@ -1817,10 +1658,12 @@ sh_css_flush(struct ia_css_acc_fw *fw)
my_css.flush(fw);
}
-/* Mapping sp threads. Currently, this is done when a stream is created and
+/*
+ * Mapping sp threads. Currently, this is done when a stream is created and
* pipelines are ready to be converted to sp pipelines. Be careful if you are
* doing it from stream_create since we could run out of sp threads due to
- * allocation on inactive pipelines. */
+ * allocation on inactive pipelines.
+ */
static int
map_sp_threads(struct ia_css_stream *stream, bool map)
{
@@ -1884,8 +1727,10 @@ map_sp_threads(struct ia_css_stream *stream, bool map)
return err;
}
-/* creates a host pipeline skeleton for all pipes in a stream. Called during
- * stream_create. */
+/*
+ * creates a host pipeline skeleton for all pipes in a stream. Called during
+ * stream_create.
+ */
static int
create_host_pipeline_structure(struct ia_css_stream *stream)
{
@@ -1984,8 +1829,10 @@ create_host_pipeline_structure(struct ia_css_stream *stream)
return err;
}
-/* creates a host pipeline for all pipes in a stream. Called during
- * stream_start. */
+/*
+ * creates a host pipeline for all pipes in a stream. Called during
+ * stream_start.
+ */
static int
create_host_pipeline(struct ia_css_stream *stream)
{
@@ -2005,15 +1852,25 @@ create_host_pipeline(struct ia_css_stream *stream)
main_pipe = stream->last_pipe;
pipe_id = main_pipe->mode;
- /* No continuous frame allocation for capture pipe. It uses the
- * "main" pipe's frames. */
+ /*
+ * No continuous frame allocation for capture pipe. It uses the
+ * "main" pipe's frames.
+ */
if ((pipe_id == IA_CSS_PIPE_ID_PREVIEW) ||
(pipe_id == IA_CSS_PIPE_ID_VIDEO)) {
- /* About pipe_id == IA_CSS_PIPE_ID_PREVIEW && stream->config.mode != IA_CSS_INPUT_MODE_MEMORY:
- * The original condition pipe_id == IA_CSS_PIPE_ID_PREVIEW is too strong. E.g. in SkyCam (with memory
- * based input frames) there is no continuous mode and thus no need for allocated continuous frames
- * This is not only for SkyCam but for all preview cases that use DDR based input frames. For this
- * reason the stream->config.mode != IA_CSS_INPUT_MODE_MEMORY has beed added.
+ /*
+ * About
+ * pipe_id == IA_CSS_PIPE_ID_PREVIEW &&
+ * stream->config.mode != IA_CSS_INPUT_MODE_MEMORY:
+ *
+ * The original condition pipe_id == IA_CSS_PIPE_ID_PREVIEW is
+ * too strong. E.g. in SkyCam (with memory based input frames)
+ * there is no continuous mode and thus no need for allocated
+ * continuous frames.
+ * This is not only for SkyCam but for all preview cases that
+ * use DDR based input frames. For this reason the
+ * stream->config.mode != IA_CSS_INPUT_MODE_MEMORY has beed
+ * added.
*/
if (stream->config.continuous ||
(pipe_id == IA_CSS_PIPE_ID_PREVIEW &&
@@ -2223,7 +2080,7 @@ pipe_generate_pipe_num(const struct ia_css_pipe *pipe,
/* Assign a new pipe_num .... search for empty place */
for (i = 0; i < IA_CSS_PIPELINE_NUM_MAX; i++) {
if (!my_css.all_pipes[i]) {
- /*position is reserved */
+ /* position is reserved */
my_css.all_pipes[i] = (struct ia_css_pipe *)pipe;
pipe_num = i;
break;
@@ -2340,8 +2197,10 @@ ia_css_pipe_destroy(struct ia_css_pipe *pipe)
switch (pipe->config.mode) {
case IA_CSS_PIPE_MODE_PREVIEW:
- /* need to take into account that this function is also called
- on the internal copy pipe */
+ /*
+ * need to take into account that this function is also called
+ * on the internal copy pipe
+ */
if (pipe->mode == IA_CSS_PIPE_ID_PREVIEW) {
ia_css_frame_free_multiple(NUM_CONTINUOUS_FRAMES,
pipe->continuous_frames);
@@ -2369,10 +2228,10 @@ ia_css_pipe_destroy(struct ia_css_pipe *pipe)
}
}
#ifndef ISP2401
- ia_css_frame_free_multiple(NUM_TNR_FRAMES,
+ ia_css_frame_free_multiple(NUM_VIDEO_TNR_FRAMES,
pipe->pipe_settings.video.tnr_frames);
#else
- ia_css_frame_free_multiple(NUM_TNR_FRAMES,
+ ia_css_frame_free_multiple(NUM_VIDEO_TNR_FRAMES,
pipe->pipe_settings.video.tnr_frames);
#endif
ia_css_frame_free_multiple(MAX_NUM_VIDEO_DELAY_FRAMES,
@@ -2391,8 +2250,10 @@ ia_css_pipe_destroy(struct ia_css_pipe *pipe)
break;
}
- sh_css_params_free_gdc_lut(pipe->scaler_pp_lut);
- pipe->scaler_pp_lut = mmgr_NULL;
+ if (pipe->scaler_pp_lut != mmgr_NULL) {
+ hmm_free(pipe->scaler_pp_lut);
+ pipe->scaler_pp_lut = mmgr_NULL;
+ }
my_css.active_pipes[ia_css_pipe_get_pipe_num(pipe)] = NULL;
sh_css_pipe_free_shading_table(pipe);
@@ -2413,15 +2274,13 @@ void
ia_css_uninit(void)
{
ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_uninit() enter: void\n");
-#if WITH_PC_MONITORING
- sh_css_print("PC_MONITORING: %s() -- started\n", __func__);
- print_pc_histogram();
-#endif
sh_css_params_free_default_gdc_lut();
/* TODO: JB: implement decent check and handling of freeing mipi frames */
- //assert(ref_count_mipi_allocation == 0); //mipi frames are not freed
+ if (!mipi_is_free())
+ dev_warn(atomisp_dev, "mipi frames are not freed.\n");
+
/* cleanup generic data */
sh_css_params_uninit();
ia_css_refcount_uninit();
@@ -2465,16 +2324,13 @@ int ia_css_irq_translate(
if (status == hrt_isp_css_irq_status_error)
return -EINVAL;
-#if WITH_PC_MONITORING
- sh_css_print("PC_MONITORING: %s() irq = %d, sh_binary_running set to 0\n",
- __func__, irq);
- sh_binary_running = 0;
-#endif
switch (irq) {
case virq_sp:
- /* When SP goes to idle, info is available in the
- * event queue. */
+ /*
+ * When SP goes to idle, info is available in the
+ * event queue.
+ */
infos |= IA_CSS_IRQ_INFO_EVENTS_READY;
break;
case virq_isp:
@@ -2582,8 +2438,10 @@ sh_css_get_sw_interrupt_value(unsigned int irq)
return irq_value;
}
-/* configure and load the copy binary, the next binary is used to
- determine whether the copy binary needs to do left padding. */
+/*
+ * configure and load the copy binary, the next binary is used to
+ * determine whether the copy binary needs to do left padding.
+ */
static int load_copy_binary(
struct ia_css_pipe *pipe,
struct ia_css_binary *copy_binary,
@@ -2670,13 +2528,11 @@ alloc_continuous_frames(struct ia_css_pipe *pipe, bool init_time)
ref_info.padded_width = CEIL_MUL(ref_info.res.width, 2 * ISP_VEC_NELEMS);
#endif
-#if !defined(HAS_NO_PACKED_RAW_PIXELS)
if (pipe->stream->config.pack_raw_pixels) {
ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
"alloc_continuous_frames() IA_CSS_FRAME_FORMAT_RAW_PACKED\n");
ref_info.format = IA_CSS_FRAME_FORMAT_RAW_PACKED;
} else
-#endif
{
ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
"alloc_continuous_frames() IA_CSS_FRAME_FORMAT_RAW\n");
@@ -2780,7 +2636,8 @@ load_preview_binaries(struct ia_css_pipe *pipe)
if (err)
return err;
- /* Note: the current selection of vf_pp binary and
+ /*
+ * Note: the current selection of vf_pp binary and
* parameterization of the preview binary contains a few pieces
* of hardcoded knowledge. This needs to be cleaned up such that
* the binary selection becomes more generic.
@@ -2793,7 +2650,7 @@ load_preview_binaries(struct ia_css_pipe *pipe)
* The decision if the vf_pp binary is needed for YUV downscaling is
* made after the preview binary selection, since some preview binaries
* can perform the requested YUV downscaling.
- * */
+ */
need_vf_pp = pipe->config.enable_dz;
need_vf_pp |= pipe_out_info->format != IA_CSS_FRAME_FORMAT_YUV_LINE &&
!(pipe_out_info->format == IA_CSS_FRAME_FORMAT_NV12 ||
@@ -2805,11 +2662,12 @@ load_preview_binaries(struct ia_css_pipe *pipe)
prev_vf_info = pipe->vf_yuv_ds_input_info;
else
prev_vf_info = *pipe_out_info;
- /* If vf_pp is needed, then preview must output yuv_line.
+ /*
+ * If vf_pp is needed, then preview must output yuv_line.
* The exception is when vf_pp is manually disabled, that is only
* used in combination with a pipeline extension that requires
* yuv_line as input.
- * */
+ */
if (need_vf_pp)
ia_css_frame_info_set_format(&prev_vf_info,
IA_CSS_FRAME_FORMAT_YUV_LINE);
@@ -2827,22 +2685,12 @@ load_preview_binaries(struct ia_css_pipe *pipe)
if (err)
return err;
- if (IS_ISP2401) {
- /* The delay latency determines the number of invalid frames after
- * a stream is started. */
- pipe->num_invalid_frames = pipe->dvs_frame_delay;
- pipe->info.num_invalid_frames = pipe->num_invalid_frames;
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE,
- "load_preview_binaries() num_invalid_frames=%d dvs_frame_delay=%d\n",
- pipe->num_invalid_frames, pipe->dvs_frame_delay);
- }
-
/* The vf_pp binary is needed when (further) YUV downscaling is required */
need_vf_pp |= mycs->preview_binary.out_frame_info[0].res.width != pipe_out_info->res.width;
need_vf_pp |= mycs->preview_binary.out_frame_info[0].res.height != pipe_out_info->res.height;
- /* When vf_pp is needed, then the output format of the selected
+ /*
+ * When vf_pp is needed, then the output format of the selected
* preview binary must be yuv_line. If this is not the case,
* then the preview binary selection is done again.
*/
@@ -2886,12 +2734,14 @@ load_preview_binaries(struct ia_css_pipe *pipe)
}
#ifdef ISP2401
- /* When the input system is 2401, only the Direct Sensor Mode
+ /*
+ * When the input system is 2401, only the Direct Sensor Mode
* Offline Preview uses the ISP copy binary.
*/
need_isp_copy_binary = !online && sensor;
#else
- /* About pipe->stream->config.mode == IA_CSS_INPUT_MODE_MEMORY:
+ /*
+ * About pipe->stream->config.mode == IA_CSS_INPUT_MODE_MEMORY:
* This is typical the case with SkyCam (which has no input system) but it also applies to all cases
* where the driver chooses for memory based input frames. In these cases, a copy binary (which typical
* copies sensor data to DDR) does not have much use.
@@ -3245,8 +3095,10 @@ get_crop_columns_for_bayer_order(const struct ia_css_stream_config *config)
return 0;
}
-/* This function is to get the sum of all extra pixels in addition to the effective
- * input, it includes dvs envelop and filter run-in */
+/*
+ * This function is to get the sum of all extra pixels in addition to the effective
+ * input, it includes dvs envelop and filter run-in
+ */
static void get_pipe_extra_pixel(struct ia_css_pipe *pipe,
unsigned int *extra_row, unsigned int *extra_column)
{
@@ -3255,7 +3107,8 @@ static void get_pipe_extra_pixel(struct ia_css_pipe *pipe,
unsigned int i;
struct ia_css_resolution dvs_env = pipe->config.dvs_envelope;
- /* The dvs envelope info may not be correctly sent down via pipe config
+ /*
+ * The dvs envelope info may not be correctly sent down via pipe config
* The check is made and the correct value is populated in the binary info
* Use this value when computing crop, else excess lines may get trimmed
*/
@@ -3349,7 +3202,8 @@ ia_css_get_crop_offsets(
* 2. Require the special support for the online use cases.
*/
- /* ISP expects GRBG bayer order, we skip one line and/or one row
+ /*
+ * ISP expects GRBG bayer order, we skip one line and/or one row
* to correct in case the input bayer order is different.
*/
column += get_crop_columns_for_bayer_order(&pipe->stream->config);
@@ -3467,7 +3321,8 @@ static int create_host_video_pipeline(struct ia_css_pipe *pipe)
me->dvs_frame_delay = pipe->dvs_frame_delay;
#ifdef ISP2401
- /* When the input system is 2401, always enable 'in_frameinfo_memory'
+ /*
+ * When the input system is 2401, always enable 'in_frameinfo_memory'
* except for the following: online or continuous
*/
need_in_frameinfo_memory = !(pipe->stream->config.online ||
@@ -3523,7 +3378,8 @@ static int create_host_video_pipeline(struct ia_css_pipe *pipe)
in_frame = me->stages->args.out_frame[0];
} else if (pipe->stream->config.continuous) {
#ifdef ISP2401
- /* When continuous is enabled, configure in_frame with the
+ /*
+ * When continuous is enabled, configure in_frame with the
* last pipe, which is the copy pipe.
*/
in_frame = pipe->stream->last_pipe->continuous_frames[0];
@@ -3535,8 +3391,10 @@ static int create_host_video_pipeline(struct ia_css_pipe *pipe)
ia_css_pipe_util_set_output_frames(out_frames, 0,
need_yuv_pp ? NULL : out_frame);
- /* when the video binary supports a second output pin,
- it can directly produce the vf_frame. */
+ /*
+ * when the video binary supports a second output pin,
+ * it can directly produce the vf_frame.
+ */
if (need_vf_pp) {
ia_css_pipe_get_generic_stage_desc(&stage_desc, video_binary,
out_frames, in_frame, NULL);
@@ -3568,7 +3426,7 @@ static int create_host_video_pipeline(struct ia_css_pipe *pipe)
if (video_stage) {
int frm;
- for (frm = 0; frm < NUM_TNR_FRAMES; frm++) {
+ for (frm = 0; frm < NUM_VIDEO_TNR_FRAMES; frm++) {
video_stage->args.tnr_frames[frm] =
pipe->pipe_settings.video.tnr_frames[frm];
}
@@ -3708,7 +3566,8 @@ create_host_preview_pipeline(struct ia_css_pipe *pipe)
ia_css_pipeline_clean(me);
#ifdef ISP2401
- /* When the input system is 2401, always enable 'in_frameinfo_memory'
+ /*
+ * When the input system is 2401, always enable 'in_frameinfo_memory'
* except for the following:
* - Direct Sensor Mode Online Preview
* - Buffered Sensor Mode Online Preview
@@ -3757,7 +3616,8 @@ create_host_preview_pipeline(struct ia_css_pipe *pipe)
in_frame = me->stages->args.out_frame[0];
} else if (pipe->stream->config.continuous) {
#ifdef ISP2401
- /* When continuous is enabled, configure in_frame with the
+ /*
+ * When continuous is enabled, configure in_frame with the
* last pipe, which is the copy pipe.
*/
if (continuous || !online)
@@ -3837,8 +3697,7 @@ preview_start(struct ia_css_pipe *pipe)
struct ia_css_pipe *acc_pipe;
enum sh_css_pipe_config_override copy_ovrd;
enum ia_css_input_mode preview_pipe_input_mode;
- const struct ia_css_coordinate *coord = NULL;
- const struct ia_css_isp_parameters *params = NULL;
+ unsigned int thread_id;
IA_CSS_ENTER_PRIVATE("pipe = %p", pipe);
if ((!pipe) || (!pipe->stream) || (pipe->mode != IA_CSS_PIPE_ID_PREVIEW)) {
@@ -3862,22 +3721,13 @@ preview_start(struct ia_css_pipe *pipe)
}
send_raw_frames(pipe);
- {
- unsigned int thread_id;
+ ia_css_pipeline_get_sp_thread_id(ia_css_pipe_get_pipe_num(pipe), &thread_id);
+ copy_ovrd = 1 << thread_id;
- ia_css_pipeline_get_sp_thread_id(ia_css_pipe_get_pipe_num(pipe), &thread_id);
- copy_ovrd = 1 << thread_id;
-
- if (pipe->stream->cont_capt) {
- ia_css_pipeline_get_sp_thread_id(ia_css_pipe_get_pipe_num(capture_pipe),
- &thread_id);
- copy_ovrd |= 1 << thread_id;
- }
- }
-
- if (IS_ISP2401) {
- coord = &pipe->config.internal_frame_origin_bqs_on_sctbl;
- params = pipe->stream->isp_params_configs;
+ if (pipe->stream->cont_capt) {
+ ia_css_pipeline_get_sp_thread_id(ia_css_pipe_get_pipe_num(capture_pipe),
+ &thread_id);
+ copy_ovrd |= 1 << thread_id;
}
/* Construct and load the copy pipe */
@@ -3892,12 +3742,12 @@ preview_start(struct ia_css_pipe *pipe)
pipe->stream->config.mode,
&pipe->stream->config.metadata_config,
&pipe->stream->info.metadata_info,
- pipe->stream->config.source.port.port,
- coord,
- params);
+ pipe->stream->config.source.port.port);
- /* make the preview pipe start with mem mode input, copy handles
- the actual mode */
+ /*
+ * make the preview pipe start with mem mode input, copy handles
+ * the actual mode
+ */
preview_pipe_input_mode = IA_CSS_INPUT_MODE_MEMORY;
}
@@ -3915,9 +3765,7 @@ preview_start(struct ia_css_pipe *pipe)
IA_CSS_INPUT_MODE_MEMORY,
&pipe->stream->config.metadata_config,
&pipe->stream->info.metadata_info,
- (enum mipi_port_id)0,
- coord,
- params);
+ (enum mipi_port_id)0);
}
if (acc_pipe) {
@@ -3933,9 +3781,7 @@ preview_start(struct ia_css_pipe *pipe)
IA_CSS_INPUT_MODE_MEMORY,
NULL,
NULL,
- (enum mipi_port_id)0,
- coord,
- params);
+ (enum mipi_port_id)0);
}
start_pipe(pipe, copy_ovrd, preview_pipe_input_mode);
@@ -3968,38 +3814,7 @@ ia_css_pipe_enqueue_buffer(struct ia_css_pipe *pipe,
}
buf_type = buffer->type;
- /* following code will be enabled when IA_CSS_BUFFER_TYPE_SEC_OUTPUT_FRAME
- is removed */
-#if 0
- if (buf_type == IA_CSS_BUFFER_TYPE_OUTPUT_FRAME) {
- bool found_pipe = false;
- for (i = 0; i < IA_CSS_PIPE_MAX_OUTPUT_STAGE; i++) {
- if ((buffer->data.frame->info.res.width == pipe->output_info[i].res.width) &&
- (buffer->data.frame->info.res.height == pipe->output_info[i].res.height)) {
- buf_type += i;
- found_pipe = true;
- break;
- }
- }
- if (!found_pipe)
- return -EINVAL;
- }
- if (buf_type == IA_CSS_BUFFER_TYPE_VF_OUTPUT_FRAME) {
- bool found_pipe = false;
-
- for (i = 0; i < IA_CSS_PIPE_MAX_OUTPUT_STAGE; i++) {
- if ((buffer->data.frame->info.res.width == pipe->vf_output_info[i].res.width) &&
- (buffer->data.frame->info.res.height == pipe->vf_output_info[i].res.height)) {
- buf_type += i;
- found_pipe = true;
- break;
- }
- }
- if (!found_pipe)
- return -EINVAL;
- }
-#endif
pipe_id = pipe->mode;
IA_CSS_LOG("pipe_id=%d, buf_type=%d", pipe_id, buf_type);
@@ -4086,15 +3901,6 @@ ia_css_pipe_enqueue_buffer(struct ia_css_pipe *pipe,
"ia_css_pipe_enqueue_buffer() buf_type=%d, data(DDR address)=0x%x\n",
buf_type, buffer->data.frame->data);
-#if CONFIG_ON_FRAME_ENQUEUE()
- return_err = set_config_on_frame_enqueue(
- &buffer->data.frame->info,
- &ddr_buffer.payload.frame);
- if (return_err) {
- IA_CSS_LEAVE_ERR(return_err);
- return return_err;
- }
-#endif
}
/* start of test for using rmgr for acq/rel memory */
@@ -4124,8 +3930,10 @@ ia_css_pipe_enqueue_buffer(struct ia_css_pipe *pipe,
}
for (stage = pipeline->stages; stage; stage = stage->next) {
- /* The SP will read the params
- after it got empty 3a and dis */
+ /*
+ * The SP will read the params after it got
+ * empty 3a and dis
+ */
if (STATS_ENABLED(stage)) {
/* there is a stage that needs it */
return_err = ia_css_bufq_enqueue_buffer(thread_id,
@@ -4142,14 +3950,12 @@ ia_css_pipe_enqueue_buffer(struct ia_css_pipe *pipe,
return_err = ia_css_bufq_enqueue_buffer(thread_id,
queue_id,
(uint32_t)h_vbuf->vptr);
-#if defined(SH_CSS_ENABLE_PER_FRAME_PARAMS)
if (!return_err &&
buf_type == IA_CSS_BUFFER_TYPE_OUTPUT_FRAME) {
IA_CSS_LOG("pfp: enqueued OF %d to q %d thread %d",
ddr_buffer.payload.frame.frame_data,
queue_id, thread_id);
}
-#endif
}
if (!return_err) {
@@ -4191,7 +3997,7 @@ ia_css_pipe_enqueue_buffer(struct ia_css_pipe *pipe,
/*
* TODO: Free up the hmm memory space.
- */
+ */
int
ia_css_pipe_dequeue_buffer(struct ia_css_pipe *pipe,
struct ia_css_buffer *buffer)
@@ -4258,7 +4064,8 @@ ia_css_pipe_dequeue_buffer(struct ia_css_pipe *pipe,
hmm_buffer_record = sh_css_hmm_buffer_record_validate(
ddr_buffer_addr, buf_type);
if (hmm_buffer_record) {
- /* valid hmm_buffer_record found. Save the kernel_ptr
+ /*
+ * valid hmm_buffer_record found. Save the kernel_ptr
* for validation after performing hmm_load. The
* vbuf handle and buffer_record can be released.
*/
@@ -4276,7 +4083,8 @@ ia_css_pipe_dequeue_buffer(struct ia_css_pipe *pipe,
&ddr_buffer,
sizeof(struct sh_css_hmm_buffer));
- /* if the kernel_ptr is 0 or an invalid, return an error.
+ /*
+ * if the kernel_ptr is 0 or an invalid, return an error.
* do not access the buffer via the kernal_ptr.
*/
if ((ddr_buffer.kernel_ptr == 0) ||
@@ -4290,8 +4098,11 @@ ia_css_pipe_dequeue_buffer(struct ia_css_pipe *pipe,
}
if (ddr_buffer.kernel_ptr != 0) {
- /* buffer->exp_id : all instances to be removed later once the driver change
- * is completed. See patch #5758 for reference */
+ /*
+ * buffer->exp_id : all instances to be removed later
+ * once the driver change is completed. See patch #5758
+ * for reference
+ */
buffer->exp_id = 0;
buffer->driver_cookie = ddr_buffer.cookie_ptr;
buffer->timing_data = ddr_buffer.timing_data;
@@ -4307,8 +4118,10 @@ ia_css_pipe_dequeue_buffer(struct ia_css_pipe *pipe,
case IA_CSS_BUFFER_TYPE_SEC_OUTPUT_FRAME:
if (pipe && pipe->stop_requested) {
#if !defined(ISP2401)
- /* free mipi frames only for old input system
- * for 2401 it is done in ia_css_stream_destroy call
+ /*
+ * free mipi frames only for old input
+ * system for 2401 it is done in
+ * ia_css_stream_destroy call
*/
return_err = free_mipi_frames(pipe);
if (return_err) {
@@ -4345,12 +4158,10 @@ ia_css_pipe_dequeue_buffer(struct ia_css_pipe *pipe,
sh_css_sp_get_binary_copy_size();
#endif
}
-#if defined(SH_CSS_ENABLE_PER_FRAME_PARAMS)
if (buf_type == IA_CSS_BUFFER_TYPE_OUTPUT_FRAME) {
IA_CSS_LOG("pfp: dequeued OF %d with config id %d thread %d",
frame->data, frame->isp_config_id, thread_id);
}
-#endif
ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE,
"ia_css_pipe_dequeue_buffer() buf_type=%d, data(DDR address)=0x%x\n",
@@ -4419,45 +4230,41 @@ ia_css_pipe_dequeue_buffer(struct ia_css_pipe *pipe,
* 4) "enum ia_css_event_type convert_event_sp_to_host_domain" (sh_css.c)
*/
static enum ia_css_event_type convert_event_sp_to_host_domain[] = {
- IA_CSS_EVENT_TYPE_OUTPUT_FRAME_DONE, /** Output frame ready. */
- IA_CSS_EVENT_TYPE_SECOND_OUTPUT_FRAME_DONE, /** Second output frame ready. */
- IA_CSS_EVENT_TYPE_VF_OUTPUT_FRAME_DONE, /** Viewfinder Output frame ready. */
- IA_CSS_EVENT_TYPE_SECOND_VF_OUTPUT_FRAME_DONE, /** Second viewfinder Output frame ready. */
- IA_CSS_EVENT_TYPE_3A_STATISTICS_DONE, /** Indication that 3A statistics are available. */
- IA_CSS_EVENT_TYPE_DIS_STATISTICS_DONE, /** Indication that DIS statistics are available. */
- IA_CSS_EVENT_TYPE_PIPELINE_DONE, /** Pipeline Done event, sent after last pipeline stage. */
- IA_CSS_EVENT_TYPE_FRAME_TAGGED, /** Frame tagged. */
- IA_CSS_EVENT_TYPE_INPUT_FRAME_DONE, /** Input frame ready. */
- IA_CSS_EVENT_TYPE_METADATA_DONE, /** Metadata ready. */
- IA_CSS_EVENT_TYPE_LACE_STATISTICS_DONE, /** Indication that LACE statistics are available. */
- IA_CSS_EVENT_TYPE_ACC_STAGE_COMPLETE, /** Extension stage executed. */
- IA_CSS_EVENT_TYPE_TIMER, /** Timing measurement data. */
- IA_CSS_EVENT_TYPE_PORT_EOF, /** End Of Frame event, sent when in buffered sensor mode. */
- IA_CSS_EVENT_TYPE_FW_WARNING, /** Performance warning encountered by FW */
- IA_CSS_EVENT_TYPE_FW_ASSERT, /** Assertion hit by FW */
+ IA_CSS_EVENT_TYPE_OUTPUT_FRAME_DONE, /* Output frame ready. */
+ IA_CSS_EVENT_TYPE_SECOND_OUTPUT_FRAME_DONE, /* Second output frame ready. */
+ IA_CSS_EVENT_TYPE_VF_OUTPUT_FRAME_DONE, /* Viewfinder Output frame ready. */
+ IA_CSS_EVENT_TYPE_SECOND_VF_OUTPUT_FRAME_DONE, /* Second viewfinder Output frame ready. */
+ IA_CSS_EVENT_TYPE_3A_STATISTICS_DONE, /* Indication that 3A statistics are available. */
+ IA_CSS_EVENT_TYPE_DIS_STATISTICS_DONE, /* Indication that DIS statistics are available. */
+ IA_CSS_EVENT_TYPE_PIPELINE_DONE, /* Pipeline Done event, sent after last pipeline stage. */
+ IA_CSS_EVENT_TYPE_FRAME_TAGGED, /* Frame tagged. */
+ IA_CSS_EVENT_TYPE_INPUT_FRAME_DONE, /* Input frame ready. */
+ IA_CSS_EVENT_TYPE_METADATA_DONE, /* Metadata ready. */
+ IA_CSS_EVENT_TYPE_LACE_STATISTICS_DONE, /* Indication that LACE statistics are available. */
+ IA_CSS_EVENT_TYPE_ACC_STAGE_COMPLETE, /* Extension stage executed. */
+ IA_CSS_EVENT_TYPE_TIMER, /* Timing measurement data. */
+ IA_CSS_EVENT_TYPE_PORT_EOF, /* End Of Frame event, sent when in buffered sensor mode. */
+ IA_CSS_EVENT_TYPE_FW_WARNING, /* Performance warning encountered by FW */
+ IA_CSS_EVENT_TYPE_FW_ASSERT, /* Assertion hit by FW */
0, /* error if sp passes SH_CSS_SP_EVENT_NR_OF_TYPES as a valid event. */
};
int
-ia_css_dequeue_event(struct ia_css_event *event)
-{
- return ia_css_dequeue_psys_event(event);
-}
-
-int
ia_css_dequeue_psys_event(struct ia_css_event *event)
{
enum ia_css_pipe_id pipe_id = 0;
u8 payload[4] = {0, 0, 0, 0};
int ret_err;
- /*TODO:
+ /*
+ * TODO:
* a) use generic decoding function , same as the one used by sp.
* b) group decode and dequeue into eventQueue module
*
* We skip the IA_CSS_ENTER logging call
* to avoid flooding the logs when the host application
- * uses polling. */
+ * uses polling.
+ */
if (!event)
return -EINVAL;
@@ -4476,9 +4283,11 @@ ia_css_dequeue_psys_event(struct ia_css_event *event)
ia_css_bufq_enqueue_psys_event(
IA_CSS_PSYS_SW_EVENT_EVENT_DEQUEUED, 0, 0, 0);
- /* Events are decoded into 4 bytes of payload, the first byte
+ /*
+ * Events are decoded into 4 bytes of payload, the first byte
* contains the sp event type. This is converted to a host enum.
- * TODO: can this enum conversion be eliminated */
+ * TODO: can this enum conversion be eliminated
+ */
event->type = convert_event_sp_to_host_domain[payload[0]];
/* Some sane default values since not all events use all fields. */
event->pipe = NULL;
@@ -4491,7 +4300,10 @@ ia_css_dequeue_psys_event(struct ia_css_event *event)
event->timer_subcode = 0;
if (event->type == IA_CSS_EVENT_TYPE_TIMER) {
- /* timer event ??? get the 2nd event and decode the data into the event struct */
+ /*
+ * timer event ??? get the 2nd event and decode the data
+ * into the event struct
+ */
u32 tmp_data;
/* 1st event: LSB 16-bit timer data and code */
event->timer_data = ((payload[1] & 0xFF) | ((payload[3] & 0xFF) << 8));
@@ -4500,8 +4312,10 @@ ia_css_dequeue_psys_event(struct ia_css_event *event)
ret_err = ia_css_bufq_dequeue_psys_event(payload);
if (ret_err) {
/* no 2nd event ??? an error */
- /* Putting IA_CSS_ERROR is resulting in failures in
- * Merrifield smoke testing */
+ /*
+ * Putting IA_CSS_ERROR is resulting in failures in
+ * Merrifield smoke testing
+ */
IA_CSS_WARNING("Timer: Error de-queuing the 2nd TIMER event!!!\n");
return ret_err;
}
@@ -4515,11 +4329,15 @@ ia_css_dequeue_psys_event(struct ia_css_event *event)
event->timer_data |= (tmp_data << 16);
event->timer_subcode = payload[2];
} else {
- /* It's a non timer event. So clear first half of the timer event data.
- * If the second part of the TIMER event is not received, we discard
- * the first half of the timer data and process the non timer event without
- * affecting the flow. So the non timer event falls through
- * the code. */
+ /*
+ * It's a non timer event. So clear first half of the
+ * timer event data.
+ * If the second part of the TIMER event is not
+ * received, we discard the first half of the timer
+ * data and process the non timer event without
+ * affecting the flow. So the non timer event falls
+ * through the code.
+ */
event->timer_data = 0;
event->timer_code = 0;
event->timer_subcode = 0;
@@ -4540,9 +4358,11 @@ ia_css_dequeue_psys_event(struct ia_css_event *event)
event->fw_assert_line_no = (payload[2] << 8) + payload[3];
/* payload[2] is line_no>>8, payload[3] is line_no&0xff */
} else if (event->type != IA_CSS_EVENT_TYPE_TIMER) {
- /* pipe related events.
+ /*
+ * pipe related events.
* payload[1] contains the pipe_num,
- * payload[2] contains the pipe_id. These are different. */
+ * payload[2] contains the pipe_id. These are different.
+ */
event->pipe = find_pipe_by_num(payload[1]);
pipe_id = (enum ia_css_pipe_id)payload[2];
/* Check to see if pipe still exists */
@@ -4594,9 +4414,11 @@ ia_css_dequeue_isys_event(struct ia_css_event *event)
u8 payload[4] = {0, 0, 0, 0};
int err = 0;
- /* We skip the IA_CSS_ENTER logging call
+ /*
+ * We skip the IA_CSS_ENTER logging call
* to avoid flooding the logs when the host application
- * uses polling. */
+ * uses polling.
+ */
if (!event)
return -EINVAL;
@@ -4720,7 +4542,8 @@ sh_css_pipe_start(struct ia_css_stream *stream)
return err;
}
- /* Force ISP parameter calculation after a mode change
+ /*
+ * Force ISP parameter calculation after a mode change
* Acceleration API examples pass NULL for stream but they
* don't use ISP parameters anyway. So this should be okay.
* The SP binary (jpeg) copy does not use any parameters.
@@ -4889,185 +4712,6 @@ ia_css_stream_get_buffer_depth(struct ia_css_stream *stream,
return 0;
}
-/*
- * @brief Stop all "ia_css_pipe" instances in the target
- * "ia_css_stream" instance.
- *
- * Refer to "Local prototypes" for more info.
- */
-/* ISP2401 */
-static int
-sh_css_pipes_stop(struct ia_css_stream *stream)
-{
- int err = 0;
- struct ia_css_pipe *main_pipe;
- enum ia_css_pipe_id main_pipe_id;
- int i;
-
- if (!stream) {
- IA_CSS_LOG("stream does NOT exist!");
- err = -EINVAL;
- goto ERR;
- }
-
- main_pipe = stream->last_pipe;
- if (!main_pipe) {
- IA_CSS_LOG("main_pipe does NOT exist!");
- err = -EINVAL;
- goto ERR;
- }
-
- main_pipe_id = main_pipe->mode;
- IA_CSS_ENTER_PRIVATE("main_pipe_id=%d", main_pipe_id);
-
- /*
- * Stop all "ia_css_pipe" instances in this target
- * "ia_css_stream" instance.
- */
- for (i = 0; i < stream->num_pipes; i++) {
- /* send the "stop" request to the "ia_css_pipe" instance */
- IA_CSS_LOG("Send the stop-request to the pipe: pipe_id=%d",
- stream->pipes[i]->pipeline.pipe_id);
- err = ia_css_pipeline_request_stop(&stream->pipes[i]->pipeline);
-
- /*
- * Exit this loop if "ia_css_pipeline_request_stop()"
- * returns the error code.
- *
- * The error code would be generated in the following
- * two cases:
- * (1) The Scalar Processor has already been stopped.
- * (2) The "Host->SP" event queue is full.
- *
- * As the convention of using CSS API 2.0/2.1, such CSS
- * error code would be propogated from the CSS-internal
- * API returned value to the CSS API returned value. Then
- * the CSS driver should capture these error code and
- * handle it in the driver exception handling mechanism.
- */
- if (err)
- goto ERR;
- }
-
- /*
- * In the CSS firmware use scenario "Continuous Preview"
- * as well as "Continuous Video", the "ia_css_pipe" instance
- * "Copy Pipe" is activated. This "Copy Pipe" is private to
- * the CSS firmware so that it is not listed in the target
- * "ia_css_stream" instance.
- *
- * We need to stop this "Copy Pipe", as well.
- */
- if (main_pipe->stream->config.continuous) {
- struct ia_css_pipe *copy_pipe = NULL;
-
- /* get the reference to "Copy Pipe" */
- if (main_pipe_id == IA_CSS_PIPE_ID_PREVIEW)
- copy_pipe = main_pipe->pipe_settings.preview.copy_pipe;
- else if (main_pipe_id == IA_CSS_PIPE_ID_VIDEO)
- copy_pipe = main_pipe->pipe_settings.video.copy_pipe;
-
- /* return the error code if "Copy Pipe" does NOT exist */
- if (!copy_pipe) {
- IA_CSS_LOG("Copy Pipe does NOT exist!");
- err = -EINVAL;
- goto ERR;
- }
-
- /* send the "stop" request to "Copy Pipe" */
- IA_CSS_LOG("Send the stop-request to the pipe: pipe_id=%d",
- copy_pipe->pipeline.pipe_id);
- err = ia_css_pipeline_request_stop(©_pipe->pipeline);
- }
-
-ERR:
- IA_CSS_LEAVE_ERR_PRIVATE(err);
- return err;
-}
-
-/*
- * @brief Check if all "ia_css_pipe" instances in the target
- * "ia_css_stream" instance have stopped.
- *
- * Refer to "Local prototypes" for more info.
- */
-/* ISP2401 */
-static bool
-sh_css_pipes_have_stopped(struct ia_css_stream *stream)
-{
- bool rval = true;
-
- struct ia_css_pipe *main_pipe;
- enum ia_css_pipe_id main_pipe_id;
-
- int i;
-
- if (!stream) {
- IA_CSS_LOG("stream does NOT exist!");
- rval = false;
- goto RET;
- }
-
- main_pipe = stream->last_pipe;
-
- if (!main_pipe) {
- IA_CSS_LOG("main_pipe does NOT exist!");
- rval = false;
- goto RET;
- }
-
- main_pipe_id = main_pipe->mode;
- IA_CSS_ENTER_PRIVATE("main_pipe_id=%d", main_pipe_id);
-
- /*
- * Check if every "ia_css_pipe" instance in this target
- * "ia_css_stream" instance has stopped.
- */
- for (i = 0; i < stream->num_pipes; i++) {
- rval = rval && ia_css_pipeline_has_stopped(&stream->pipes[i]->pipeline);
- IA_CSS_LOG("Pipe has stopped: pipe_id=%d, stopped=%d",
- stream->pipes[i]->pipeline.pipe_id,
- rval);
- }
-
- /*
- * In the CSS firmware use scenario "Continuous Preview"
- * as well as "Continuous Video", the "ia_css_pipe" instance
- * "Copy Pipe" is activated. This "Copy Pipe" is private to
- * the CSS firmware so that it is not listed in the target
- * "ia_css_stream" instance.
- *
- * We need to check if this "Copy Pipe" has stopped, as well.
- */
- if (main_pipe->stream->config.continuous) {
- struct ia_css_pipe *copy_pipe = NULL;
-
- /* get the reference to "Copy Pipe" */
- if (main_pipe_id == IA_CSS_PIPE_ID_PREVIEW)
- copy_pipe = main_pipe->pipe_settings.preview.copy_pipe;
- else if (main_pipe_id == IA_CSS_PIPE_ID_VIDEO)
- copy_pipe = main_pipe->pipe_settings.video.copy_pipe;
-
- /* return if "Copy Pipe" does NOT exist */
- if (!copy_pipe) {
- IA_CSS_LOG("Copy Pipe does NOT exist!");
-
- rval = false;
- goto RET;
- }
-
- /* check if "Copy Pipe" has stopped or not */
- rval = rval && ia_css_pipeline_has_stopped(©_pipe->pipeline);
- IA_CSS_LOG("Pipe has stopped: pipe_id=%d, stopped=%d",
- copy_pipe->pipeline.pipe_id,
- rval);
- }
-
-RET:
- IA_CSS_LEAVE_PRIVATE("rval=%d", rval);
- return rval;
-}
-
#if !defined(ISP2401)
unsigned int
sh_css_get_mipi_sizes_for_check(const unsigned int port, const unsigned int idx)
@@ -5137,11 +4781,13 @@ sh_css_pipe_get_shading_info(struct ia_css_pipe *pipe,
(const struct ia_css_stream_config *)&pipe->stream->config,
shading_info, pipe_config);
- /* Other function calls can be added here when other shading correction types will be added
- * in the future.
+ /*
+ * Other function calls can be added here when other shading
+ * correction types will be added in the future.
*/
} else {
- /* When the pipe does not have a binary which has the shading
+ /*
+ * When the pipe does not have a binary which has the shading
* correction, this function does not need to fill the shading
* information. It is not a error case, and then
* this function should return 0.
@@ -5180,6 +4826,8 @@ sh_css_pipe_get_grid_info(struct ia_css_pipe *pipe,
ia_css_binary_dvs_stat_grid_info(binary, info, pipe);
} else {
memset(&info->dvs_grid, 0, sizeof(info->dvs_grid));
+ memset(&info->dvs_grid.dvs_stat_grid_info, 0,
+ sizeof(info->dvs_grid.dvs_stat_grid_info));
}
if (binary) {
@@ -5254,7 +4902,8 @@ static int load_video_binaries(struct ia_css_pipe *pipe)
IA_CSS_ENTER_PRIVATE("");
assert(pipe);
assert(pipe->mode == IA_CSS_PIPE_ID_VIDEO);
- /* we only test the video_binary because offline video doesn't need a
+ /*
+ * we only test the video_binary because offline video doesn't need a
* vf_pp binary and online does not (always use) the copy_binary.
* All are always reset at the same time anyway.
*/
@@ -5367,7 +5016,8 @@ static int load_video_binaries(struct ia_css_pipe *pipe)
if (err)
return err;
- /* In the case where video_vf_info is not NULL, this allows
+ /*
+ * In the case where video_vf_info is not NULL, this allows
* us to find a potential video library with desired vf format.
* If success, no vf_pp binary is needed.
* If failed, we will look up video binary with YUV_LINE vf format
@@ -5382,17 +5032,23 @@ static int load_video_binaries(struct ia_css_pipe *pipe)
else
return err;
} else if (video_vf_info) {
- /* The first video binary lookup is successful, but we may
- * still need vf_pp binary based on additiona check */
+ /*
+ * The first video binary lookup is successful, but we
+ * may still need vf_pp binary based on additional check
+ */
num_output_pins = mycs->video_binary.info->num_output_pins;
vf_ds_log2 = mycs->video_binary.vf_downscale_log2;
- /* If the binary has dual output pins, we need vf_pp if the resolution
- * is different. */
+ /*
+ * If the binary has dual output pins, we need vf_pp
+ * if the resolution is different.
+ */
need_vf_pp |= ((num_output_pins == 2) && vf_res_different_than_output);
- /* If the binary has single output pin, we need vf_pp if additional
- * scaling is needed for vf */
+ /*
+ * If the binary has single output pin, we need vf_pp
+ * if additional scaling is needed for vf
+ */
need_vf_pp |= ((num_output_pins == 1) &&
((video_vf_info->res.width << vf_ds_log2 != pipe_out_info->res.width) ||
(video_vf_info->res.height << vf_ds_log2 != pipe_out_info->res.height)));
@@ -5422,19 +5078,25 @@ static int load_video_binaries(struct ia_css_pipe *pipe)
}
}
- /* If a video binary does not use a ref_frame, we set the frame delay
- * to 0. This is the case for the 1-stage low-power video binary. */
+ /*
+ * If a video binary does not use a ref_frame, we set the frame delay
+ * to 0. This is the case for the 1-stage low-power video binary.
+ */
if (!mycs->video_binary.info->sp.enable.ref_frame)
pipe->dvs_frame_delay = 0;
- /* The delay latency determines the number of invalid frames after
- * a stream is started. */
+ /*
+ * The delay latency determines the number of invalid frames after
+ * a stream is started.
+ */
pipe->num_invalid_frames = pipe->dvs_frame_delay;
pipe->info.num_invalid_frames = pipe->num_invalid_frames;
- /* Viewfinder frames also decrement num_invalid_frames. If the pipe
+ /*
+ * Viewfinder frames also decrement num_invalid_frames. If the pipe
* outputs a viewfinder output, then we need double the number of
- * invalid frames */
+ * invalid frames
+ */
if (video_vf_info)
pipe->num_invalid_frames *= 2;
@@ -5446,7 +5108,8 @@ static int load_video_binaries(struct ia_css_pipe *pipe)
#if !defined(ISP2401)
/* Copy */
if (!online && !continuous) {
- /* TODO: what exactly needs doing, prepend the copy binary to
+ /*
+ * TODO: what exactly needs doing, prepend the copy binary to
* video base this only on !online?
*/
err = load_copy_binary(pipe,
@@ -5459,7 +5122,6 @@ static int load_video_binaries(struct ia_css_pipe *pipe)
(void)continuous;
#endif
-#if !defined(HAS_OUTPUT_SYSTEM)
if (pipe->enable_viewfinder[IA_CSS_PIPE_OUTPUT_STAGE_0] && need_vf_pp) {
struct ia_css_binary_descr vf_pp_descr;
@@ -5469,8 +5131,11 @@ static int load_video_binaries(struct ia_css_pipe *pipe)
&mycs->video_binary.vf_frame_info,
pipe_vf_out_info);
} else {
- /* output from main binary is not yuv line. currently this is
- * possible only when bci is enabled on vfpp output */
+ /*
+ * output from main binary is not yuv line. currently
+ * this is possible only when bci is enabled on vfpp
+ * output
+ */
assert(pipe->config.enable_vfpp_bci);
ia_css_pipe_get_yuvscaler_binarydesc(pipe, &vf_pp_descr,
&mycs->video_binary.vf_frame_info,
@@ -5482,7 +5147,6 @@ static int load_video_binaries(struct ia_css_pipe *pipe)
if (err)
return err;
}
-#endif
err = allocate_delay_frames(pipe);
@@ -5490,35 +5154,10 @@ static int load_video_binaries(struct ia_css_pipe *pipe)
return err;
if (mycs->video_binary.info->sp.enable.block_output) {
- unsigned int tnr_width;
- unsigned int tnr_height;
-
tnr_info = mycs->video_binary.out_frame_info[0];
- if (IS_ISP2401) {
- /* Select resolution for TNR. If
- * output_system_in_resolution(GDC_out_resolution) is
- * being used, then select that as it will also be in resolution for
- * TNR. At present, it only make sense for Skycam */
- if (pipe->config.output_system_in_res.width &&
- pipe->config.output_system_in_res.height) {
- tnr_width = pipe->config.output_system_in_res.width;
- tnr_height = pipe->config.output_system_in_res.height;
- } else {
- tnr_width = tnr_info.res.width;
- tnr_height = tnr_info.res.height;
- }
-
- /* Make tnr reference buffers output block width(in pix) align */
- tnr_info.res.width = CEIL_MUL(tnr_width,
- (mycs->video_binary.info->sp.block.block_width * ISP_NWAY));
- tnr_info.padded_width = tnr_info.res.width;
- } else {
- tnr_height = tnr_info.res.height;
- }
-
/* Make tnr reference buffers output block height align */
- tnr_info.res.height = CEIL_MUL(tnr_height,
+ tnr_info.res.height = CEIL_MUL(tnr_info.res.height,
mycs->video_binary.info->sp.block.output_block_height);
} else {
tnr_info = mycs->video_binary.internal_frame_info;
@@ -5526,7 +5165,7 @@ static int load_video_binaries(struct ia_css_pipe *pipe)
tnr_info.format = IA_CSS_FRAME_FORMAT_YUV_LINE;
tnr_info.raw_bit_depth = SH_CSS_TNR_BIT_DEPTH;
- for (i = 0; i < NUM_TNR_FRAMES; i++) {
+ for (i = 0; i < NUM_VIDEO_TNR_FRAMES; i++) {
if (mycs->tnr_frames[i]) {
ia_css_frame_free(mycs->tnr_frames[i]);
mycs->tnr_frames[i] = NULL;
@@ -5574,9 +5213,7 @@ static int video_start(struct ia_css_pipe *pipe)
struct ia_css_pipe *copy_pipe, *capture_pipe;
enum sh_css_pipe_config_override copy_ovrd;
enum ia_css_input_mode video_pipe_input_mode;
-
- const struct ia_css_coordinate *coord = NULL;
- const struct ia_css_isp_parameters *params = NULL;
+ unsigned int thread_id;
IA_CSS_ENTER_PRIVATE("pipe = %p", pipe);
if ((!pipe) || (pipe->mode != IA_CSS_PIPE_ID_VIDEO)) {
@@ -5598,22 +5235,14 @@ static int video_start(struct ia_css_pipe *pipe)
return err;
send_raw_frames(pipe);
- {
- unsigned int thread_id;
- ia_css_pipeline_get_sp_thread_id(ia_css_pipe_get_pipe_num(pipe), &thread_id);
- copy_ovrd = 1 << thread_id;
+ ia_css_pipeline_get_sp_thread_id(ia_css_pipe_get_pipe_num(pipe), &thread_id);
+ copy_ovrd = 1 << thread_id;
- if (pipe->stream->cont_capt) {
- ia_css_pipeline_get_sp_thread_id(ia_css_pipe_get_pipe_num(capture_pipe),
- &thread_id);
- copy_ovrd |= 1 << thread_id;
- }
- }
-
- if (IS_ISP2401) {
- coord = &pipe->config.internal_frame_origin_bqs_on_sctbl;
- params = pipe->stream->isp_params_configs;
+ if (pipe->stream->cont_capt) {
+ ia_css_pipeline_get_sp_thread_id(ia_css_pipe_get_pipe_num(capture_pipe),
+ &thread_id);
+ copy_ovrd |= 1 << thread_id;
}
/* Construct and load the copy pipe */
@@ -5628,12 +5257,12 @@ static int video_start(struct ia_css_pipe *pipe)
pipe->stream->config.mode,
&pipe->stream->config.metadata_config,
&pipe->stream->info.metadata_info,
- pipe->stream->config.source.port.port,
- coord,
- params);
+ pipe->stream->config.source.port.port);
- /* make the video pipe start with mem mode input, copy handles
- the actual mode */
+ /*
+ * make the video pipe start with mem mode input, copy handles
+ * the actual mode
+ */
video_pipe_input_mode = IA_CSS_INPUT_MODE_MEMORY;
}
@@ -5651,9 +5280,7 @@ static int video_start(struct ia_css_pipe *pipe)
IA_CSS_INPUT_MODE_MEMORY,
&pipe->stream->config.metadata_config,
&pipe->stream->info.metadata_info,
- (enum mipi_port_id)0,
- coord,
- params);
+ (enum mipi_port_id)0);
}
start_pipe(pipe, copy_ovrd, video_pipe_input_mode);
@@ -5763,12 +5390,6 @@ static bool need_capture_pp(
assert(pipe);
assert(pipe->mode == IA_CSS_PIPE_ID_CAPTURE);
- if (IS_ISP2401) {
- /* ldc and capture_pp are not supported in the same pipeline */
- if (need_capt_ldc(pipe))
- return false;
- }
-
/* determine whether we need to use the capture_pp binary.
* This is needed for:
* 1. XNR or
@@ -5891,9 +5512,11 @@ static int load_primary_binaries(
}
need_pp = need_capture_pp(pipe);
- /* we use the vf output info to get the primary/capture_pp binary
- configured for vf_veceven. It will select the closest downscaling
- factor. */
+ /*
+ * we use the vf output info to get the primary/capture_pp binary
+ * configured for vf_veceven. It will select the closest downscaling
+ * factor.
+ */
vf_info = *pipe_vf_out_info;
/*
@@ -5905,13 +5528,15 @@ static int load_primary_binaries(
* required. This should not be considered as a clean solution.
* Proper investigation should be done to come up with the clean
* solution.
- * */
+ */
ia_css_frame_info_set_format(&vf_info, IA_CSS_FRAME_FORMAT_YUV_LINE);
- /* TODO: All this yuv_scaler and capturepp calculation logic
+ /*
+ * TODO: All this yuv_scaler and capturepp calculation logic
* can be shared later. Capture_pp is also a yuv_scale binary
* with extra XNR funcionality. Therefore, it can be made as the
- * first step of the cascade. */
+ * first step of the cascade.
+ */
capt_pp_out_info = pipe->out_yuv_ds_input_info;
capt_pp_out_info.format = IA_CSS_FRAME_FORMAT_YUV420;
capt_pp_out_info.res.width /= MAX_PREFERRED_YUV_DS_PER_STEP;
@@ -5973,33 +5598,13 @@ static int load_primary_binaries(
/* TODO Do we disable ldc for skycam */
need_ldc = need_capt_ldc(pipe);
- if (IS_ISP2401 && need_ldc) {
- /* ldc and capt_pp are not supported in the same pipeline */
- struct ia_css_binary_descr capt_ldc_descr;
-
- ia_css_pipe_get_ldc_binarydesc(pipe,
- &capt_ldc_descr, &prim_out_info,
- &capt_pp_out_info);
-
- err = ia_css_binary_find(&capt_ldc_descr,
- &mycs->capture_ldc_binary);
- if (err) {
- IA_CSS_LEAVE_ERR_PRIVATE(err);
- return err;
- }
- need_pp = false;
- need_ldc = false;
- }
/* we build up the pipeline starting at the end */
/* Capture post-processing */
if (need_pp) {
struct ia_css_binary_descr capture_pp_descr;
- if (!IS_ISP2401)
- capt_pp_in_info = need_ldc ? &capt_ldc_out_info : &prim_out_info;
- else
- capt_pp_in_info = &prim_out_info;
+ capt_pp_in_info = need_ldc ? &capt_ldc_out_info : &prim_out_info;
ia_css_pipe_get_capturepp_binarydesc(pipe,
&capture_pp_descr,
@@ -6057,15 +5662,15 @@ static int load_primary_binaries(
vf_pp_in_info = &mycs->primary_binary[mycs->num_primary_stage - 1].vf_frame_info;
/*
- * WARNING: The #if def flag has been added below as a
- * temporary solution to solve the problem of enabling the
- * view finder in a single binary in a capture flow. The
- * vf-pp stage has been removed for Skycam in the solution
- * provided. The vf-pp stage should be re-introduced when
- * required. Thisshould not be considered as a clean solution.
- * Proper * investigation should be done to come up with the clean
- * solution.
- * */
+ * WARNING: The #if def flag has been added below as a
+ * temporary solution to solve the problem of enabling the
+ * view finder in a single binary in a capture flow. The
+ * vf-pp stage has been removed for Skycam in the solution
+ * provided. The vf-pp stage should be re-introduced when
+ * required. Thisshould not be considered as a clean solution.
+ * Proper * investigation should be done to come up with the clean
+ * solution.
+ */
if (pipe->enable_viewfinder[IA_CSS_PIPE_OUTPUT_STAGE_0]) {
struct ia_css_binary_descr vf_pp_descr;
@@ -6083,9 +5688,10 @@ static int load_primary_binaries(
return err;
#ifdef ISP2401
- /* When the input system is 2401, only the Direct Sensor Mode
- * Offline Capture uses the ISP copy binary.
- */
+ /*
+ * When the input system is 2401, only the Direct Sensor Mode
+ * Offline Capture uses the ISP copy binary.
+ */
need_isp_copy_binary = !online && sensor;
#else
need_isp_copy_binary = !online && !continuous && !memory;
@@ -6139,17 +5745,19 @@ allocate_delay_frames(struct ia_css_pipe *pipe)
struct ia_css_video_settings *mycs_video = &pipe->pipe_settings.video;
ref_info = mycs_video->video_binary.internal_frame_info;
- /*The ref frame expects
- * 1. Y plane
- * 2. UV plane with line interleaving, like below
- * UUUUUU(width/2 times) VVVVVVVV..(width/2 times)
- *
- * This format is not YUV420(which has Y, U and V planes).
- * Its closer to NV12, except that the UV plane has UV
- * interleaving, like UVUVUVUVUVUVUVUVU...
- *
- * TODO: make this ref_frame format as a separate frame format
- */
+
+ /*
+ * The ref frame expects
+ * 1. Y plane
+ * 2. UV plane with line interleaving, like below
+ * UUUUUU(width/2 times) VVVVVVVV..(width/2 times)
+ *
+ * This format is not YUV420(which has Y, U and V planes).
+ * Its closer to NV12, except that the UV plane has UV
+ * interleaving, like UVUVUVUVUVUVUVUVU...
+ *
+ * TODO: make this ref_frame format as a separate frame format
+ */
ref_info.format = IA_CSS_FRAME_FORMAT_NV12;
delay_frames = mycs_video->delay_frames;
}
@@ -6158,17 +5766,19 @@ allocate_delay_frames(struct ia_css_pipe *pipe)
struct ia_css_preview_settings *mycs_preview = &pipe->pipe_settings.preview;
ref_info = mycs_preview->preview_binary.internal_frame_info;
- /*The ref frame expects
- * 1. Y plane
- * 2. UV plane with line interleaving, like below
- * UUUUUU(width/2 times) VVVVVVVV..(width/2 times)
- *
- * This format is not YUV420(which has Y, U and V planes).
- * Its closer to NV12, except that the UV plane has UV
- * interleaving, like UVUVUVUVUVUVUVUVU...
- *
- * TODO: make this ref_frame format as a separate frame format
- */
+
+ /*
+ * The ref frame expects
+ * 1. Y plane
+ * 2. UV plane with line interleaving, like below
+ * UUUUUU(width/2 times) VVVVVVVV..(width/2 times)
+ *
+ * This format is not YUV420(which has Y, U and V planes).
+ * Its closer to NV12, except that the UV plane has UV
+ * interleaving, like UVUVUVUVUVUVUVUVU...
+ *
+ * TODO: make this ref_frame format as a separate frame format
+ */
ref_info.format = IA_CSS_FRAME_FORMAT_NV12;
delay_frames = mycs_preview->delay_frames;
}
@@ -6625,9 +6235,11 @@ need_yuv_scaler_stage(const struct ia_css_pipe *pipe)
return false;
}
-/* TODO: it is temporarily created from ia_css_pipe_create_cas_scaler_desc */
-/* which has some hard-coded knowledge which prevents reuse of the function. */
-/* Later, merge this with ia_css_pipe_create_cas_scaler_desc */
+/*
+ * TODO: it is temporarily created from ia_css_pipe_create_cas_scaler_desc
+ * which has some hard-coded knowledge which prevents reuse of the function.
+ * Later, merge this with ia_css_pipe_create_cas_scaler_desc
+ */
static int ia_css_pipe_create_cas_scaler_desc_single_output(
struct ia_css_frame_info *cas_scaler_in_info,
struct ia_css_frame_info *cas_scaler_out_info,
@@ -7024,22 +6636,22 @@ load_yuvpp_binaries(struct ia_css_pipe *pipe)
#if defined(ISP2401)
/*
- * NOTES
- * - Why does the "yuvpp" pipe needs "isp_copy_binary" (i.e. ISP Copy) when
- * its input is "ATOMISP_INPUT_FORMAT_YUV422_8"?
- *
- * In most use cases, the first stage in the "yuvpp" pipe is the "yuv_scale_
- * binary". However, the "yuv_scale_binary" does NOT support the input-frame
- * format as "IA_CSS_STREAM _FORMAT_YUV422_8".
- *
- * Hence, the "isp_copy_binary" is required to be present in front of the "yuv
- * _scale_binary". It would translate the input-frame to the frame formats that
- * are supported by the "yuv_scale_binary".
- *
- * Please refer to "FrameWork/css/isp/pipes/capture_pp/capture_pp_1.0/capture_
- * pp_defs.h" for the list of input-frame formats that are supported by the
- * "yuv_scale_binary".
- */
+ * NOTES
+ * - Why does the "yuvpp" pipe needs "isp_copy_binary" (i.e. ISP Copy) when
+ * its input is "ATOMISP_INPUT_FORMAT_YUV422_8"?
+ *
+ * In most use cases, the first stage in the "yuvpp" pipe is the "yuv_scale_
+ * binary". However, the "yuv_scale_binary" does NOT support the input-frame
+ * format as "IA_CSS_STREAM _FORMAT_YUV422_8".
+ *
+ * Hence, the "isp_copy_binary" is required to be present in front of the "yuv
+ * _scale_binary". It would translate the input-frame to the frame formats that
+ * are supported by the "yuv_scale_binary".
+ *
+ * Please refer to "FrameWork/css/isp/pipes/capture_pp/capture_pp_1.0/capture_
+ * pp_defs.h" for the list of input-frame formats that are supported by the
+ * "yuv_scale_binary".
+ */
need_isp_copy_binary =
(pipe->stream->config.input_config.format == ATOMISP_INPUT_FORMAT_YUV422_8);
#else /* !ISP2401 */
@@ -7055,23 +6667,23 @@ load_yuvpp_binaries(struct ia_css_pipe *pipe)
goto ERR;
/*
- * NOTES
- * - Why is "pipe->pipe_settings.capture.copy_binary.online" specified?
- *
- * In some use cases, the first stage in the "yuvpp" pipe is the
- * "isp_copy_binary". The "isp_copy_binary" is designed to process
- * the input from either the system DDR or from the IPU internal VMEM.
- * So it provides the flag "online" to specify where its input is from,
- * i.e.:
- *
- * (1) "online <= true", the input is from the IPU internal VMEM.
- * (2) "online <= false", the input is from the system DDR.
- *
- * In other use cases, the first stage in the "yuvpp" pipe is the
- * "yuv_scale_binary". "The "yuv_scale_binary" is designed to process the
- * input ONLY from the system DDR. So it does not provide the flag "online"
- * to specify where its input is from.
- */
+ * NOTES
+ * - Why is "pipe->pipe_settings.capture.copy_binary.online" specified?
+ *
+ * In some use cases, the first stage in the "yuvpp" pipe is the
+ * "isp_copy_binary". The "isp_copy_binary" is designed to process
+ * the input from either the system DDR or from the IPU internal VMEM.
+ * So it provides the flag "online" to specify where its input is from,
+ * i.e.:
+ *
+ * (1) "online <= true", the input is from the IPU internal VMEM.
+ * (2) "online <= false", the input is from the system DDR.
+ *
+ * In other use cases, the first stage in the "yuvpp" pipe is the
+ * "yuv_scale_binary". "The "yuv_scale_binary" is designed to process the
+ * input ONLY from the system DDR. So it does not provide the flag "online"
+ * to specify where its input is from.
+ */
pipe->pipe_settings.capture.copy_binary.online = pipe->stream->config.online;
}
@@ -7162,6 +6774,7 @@ static int yuvpp_start(struct ia_css_pipe *pipe)
int err = 0;
enum sh_css_pipe_config_override copy_ovrd;
enum ia_css_input_mode yuvpp_pipe_input_mode;
+ unsigned int thread_id;
IA_CSS_ENTER_PRIVATE("pipe = %p", pipe);
if ((!pipe) || (pipe->mode != IA_CSS_PIPE_ID_YUVPP)) {
@@ -7181,12 +6794,8 @@ static int yuvpp_start(struct ia_css_pipe *pipe)
return err;
}
- {
- unsigned int thread_id;
-
- ia_css_pipeline_get_sp_thread_id(ia_css_pipe_get_pipe_num(pipe), &thread_id);
- copy_ovrd = 1 << thread_id;
- }
+ ia_css_pipeline_get_sp_thread_id(ia_css_pipe_get_pipe_num(pipe), &thread_id);
+ copy_ovrd = 1 << thread_id;
start_pipe(pipe, copy_ovrd, yuvpp_pipe_input_mode);
@@ -7264,8 +6873,11 @@ sh_css_pipe_load_binaries(struct ia_css_pipe *pipe)
}
if (err) {
if (sh_css_pipe_unload_binaries(pipe)) {
- /* currently css does not support multiple error returns in a single function,
- * using -EINVAL in this case */
+ /*
+ * currently css does not support multiple error
+ * returns in a single function, using -EINVAL in
+ * this case
+ */
err = -EINVAL;
}
}
@@ -7316,12 +6928,13 @@ create_host_yuvpp_pipeline(struct ia_css_pipe *pipe)
num_output_stage = pipe->pipe_settings.yuvpp.num_output;
#ifdef ISP2401
- /* When the input system is 2401, always enable 'in_frameinfo_memory'
- * except for the following:
- * - Direct Sensor Mode Online Capture
- * - Direct Sensor Mode Continuous Capture
- * - Buffered Sensor Mode Continuous Capture
- */
+ /*
+ * When the input system is 2401, always enable 'in_frameinfo_memory'
+ * except for the following:
+ * - Direct Sensor Mode Online Capture
+ * - Direct Sensor Mode Continuous Capture
+ * - Buffered Sensor Mode Continuous Capture
+ */
sensor = pipe->stream->config.mode == IA_CSS_INPUT_MODE_SENSOR;
buffered_sensor = pipe->stream->config.mode == IA_CSS_INPUT_MODE_BUFFERED_SENSOR;
online = pipe->stream->config.online;
@@ -7332,19 +6945,23 @@ create_host_yuvpp_pipeline(struct ia_css_pipe *pipe)
/* Construct in_frame info (only in case we have dynamic input */
need_in_frameinfo_memory = pipe->stream->config.mode == IA_CSS_INPUT_MODE_MEMORY;
#endif
- /* the input frame can come from:
- * a) memory: connect yuvscaler to me->in_frame
- * b) sensor, via copy binary: connect yuvscaler to copy binary later on */
+ /*
+ * the input frame can come from:
+ *
+ * a) memory: connect yuvscaler to me->in_frame
+ * b) sensor, via copy binary: connect yuvscaler to copy binary later
+ * on
+ */
if (need_in_frameinfo_memory) {
/* TODO: improve for different input formats. */
/*
- * "pipe->stream->config.input_config.format" represents the sensor output
- * frame format, e.g. YUV422 8-bit.
- *
- * "in_frame_format" represents the imaging pipe's input frame format, e.g.
- * Bayer-Quad RAW.
- */
+ * "pipe->stream->config.input_config.format" represents the sensor output
+ * frame format, e.g. YUV422 8-bit.
+ *
+ * "in_frame_format" represents the imaging pipe's input frame format, e.g.
+ * Bayer-Quad RAW.
+ */
int in_frame_format;
if (pipe->stream->config.input_config.format ==
@@ -7353,22 +6970,22 @@ create_host_yuvpp_pipeline(struct ia_css_pipe *pipe)
} else if (pipe->stream->config.input_config.format ==
ATOMISP_INPUT_FORMAT_YUV422_8) {
/*
- * When the sensor output frame format is "ATOMISP_INPUT_FORMAT_YUV422_8",
- * the "isp_copy_var" binary is selected as the first stage in the yuvpp
- * pipe.
- *
- * For the "isp_copy_var" binary, it reads the YUV422-8 pixels from
- * the frame buffer (at DDR) to the frame-line buffer (at VMEM).
- *
- * By now, the "isp_copy_var" binary does NOT provide a separated
- * frame-line buffer to store the YUV422-8 pixels. Instead, it stores
- * the YUV422-8 pixels in the frame-line buffer which is designed to
- * store the Bayer-Quad RAW pixels.
- *
- * To direct the "isp_copy_var" binary reading from the RAW frame-line
- * buffer, its input frame format must be specified as "IA_CSS_FRAME_
- * FORMAT_RAW".
- */
+ * When the sensor output frame format is "ATOMISP_INPUT_FORMAT_YUV422_8",
+ * the "isp_copy_var" binary is selected as the first stage in the yuvpp
+ * pipe.
+ *
+ * For the "isp_copy_var" binary, it reads the YUV422-8 pixels from
+ * the frame buffer (at DDR) to the frame-line buffer (at VMEM).
+ *
+ * By now, the "isp_copy_var" binary does NOT provide a separated
+ * frame-line buffer to store the YUV422-8 pixels. Instead, it stores
+ * the YUV422-8 pixels in the frame-line buffer which is designed to
+ * store the Bayer-Quad RAW pixels.
+ *
+ * To direct the "isp_copy_var" binary reading from the RAW frame-line
+ * buffer, its input frame format must be specified as "IA_CSS_FRAME_
+ * FORMAT_RAW".
+ */
in_frame_format = IA_CSS_FRAME_FORMAT_RAW;
} else {
in_frame_format = IA_CSS_FRAME_FORMAT_NV12;
@@ -7667,13 +7284,14 @@ create_host_regular_capture_pipeline(struct ia_css_pipe *pipe)
ia_css_pipe_util_create_output_frames(out_frames);
#ifdef ISP2401
- /* When the input system is 2401, always enable 'in_frameinfo_memory'
- * except for the following:
- * - Direct Sensor Mode Online Capture
- * - Direct Sensor Mode Online Capture
- * - Direct Sensor Mode Continuous Capture
- * - Buffered Sensor Mode Continuous Capture
- */
+ /*
+ * When the input system is 2401, always enable 'in_frameinfo_memory'
+ * except for the following:
+ * - Direct Sensor Mode Online Capture
+ * - Direct Sensor Mode Online Capture
+ * - Direct Sensor Mode Continuous Capture
+ * - Buffered Sensor Mode Continuous Capture
+ */
sensor = (pipe->stream->config.mode == IA_CSS_INPUT_MODE_SENSOR);
buffered_sensor = (pipe->stream->config.mode == IA_CSS_INPUT_MODE_BUFFERED_SENSOR);
online = pipe->stream->config.online;
@@ -7806,15 +7424,15 @@ create_host_regular_capture_pipeline(struct ia_css_pipe *pipe)
local_out_frame = NULL;
ia_css_pipe_util_set_output_frames(out_frames, 0, local_out_frame);
/*
- * WARNING: The #if def flag has been added below as a
- * temporary solution to solve the problem of enabling the
- * view finder in a single binary in a capture flow. The
- * vf-pp stage has been removed from Skycam in the solution
- * provided. The vf-pp stage should be re-introduced when
- * required. This * should not be considered as a clean solution.
- * Proper investigation should be done to come up with the clean
- * solution.
- * */
+ * WARNING: The #if def flag has been added below as a
+ * temporary solution to solve the problem of enabling the
+ * view finder in a single binary in a capture flow. The
+ * vf-pp stage has been removed from Skycam in the solution
+ * provided. The vf-pp stage should be re-introduced when
+ * required. This * should not be considered as a clean solution.
+ * Proper investigation should be done to come up with the clean
+ * solution.
+ */
ia_css_pipe_get_generic_stage_desc(&stage_desc,
primary_binary[i],
out_frames,
@@ -7828,8 +7446,7 @@ create_host_regular_capture_pipeline(struct ia_css_pipe *pipe)
return err;
}
}
- /* If we use copy iso primary,
- the input must be yuv iso raw */
+ /* If we use copy iso primary, the input must be yuv iso raw */
current_stage->args.copy_vf =
primary_binary[0]->info->sp.pipeline.mode ==
IA_CSS_BINARY_MODE_COPY;
@@ -7888,7 +7505,6 @@ create_host_regular_capture_pipeline(struct ia_css_pipe *pipe)
}
}
-#ifndef ISP2401
if (need_pp && current_stage) {
struct ia_css_frame *local_in_frame = NULL;
@@ -7908,20 +7524,6 @@ create_host_regular_capture_pipeline(struct ia_css_pipe *pipe)
}
err = add_capture_pp_stage(pipe, me, local_in_frame,
need_yuv_pp ? NULL : out_frame,
-#else
- /* ldc and capture_pp not supported in same pipeline */
- if (need_ldc && current_stage) {
- in_frame = current_stage->args.out_frame[0];
- ia_css_pipe_util_set_output_frames(out_frames, 0, out_frame);
- ia_css_pipe_get_generic_stage_desc(&stage_desc, capture_ldc_binary,
- out_frames, in_frame, NULL);
- err = ia_css_pipeline_create_and_add_stage(me, &stage_desc,
- NULL);
- } else if (need_pp && current_stage) {
- in_frame = current_stage->args.out_frame[0];
- err = add_capture_pp_stage(pipe, me, in_frame,
- need_yuv_pp ? NULL : out_frame,
-#endif
capture_pp_binary,
¤t_stage);
if (err) {
@@ -7954,14 +7556,14 @@ create_host_regular_capture_pipeline(struct ia_css_pipe *pipe)
}
/*
- * WARNING: The #if def flag has been added below as a
- * temporary solution to solve the problem of enabling the
- * view finder in a single binary in a capture flow. The vf-pp
- * stage has been removed from Skycam in the solution provided.
- * The vf-pp stage should be re-introduced when required. This
- * should not be considered as a clean solution. Proper
- * investigation should be done to come up with the clean solution.
- * */
+ * WARNING: The #if def flag has been added below as a
+ * temporary solution to solve the problem of enabling the
+ * view finder in a single binary in a capture flow. The vf-pp
+ * stage has been removed from Skycam in the solution provided.
+ * The vf-pp stage should be re-introduced when required. This
+ * should not be considered as a clean solution. Proper
+ * investigation should be done to come up with the clean solution.
+ */
if (mode != IA_CSS_CAPTURE_MODE_RAW &&
mode != IA_CSS_CAPTURE_MODE_BAYER &&
current_stage && vf_frame) {
@@ -8005,6 +7607,7 @@ create_host_capture_pipeline(struct ia_css_pipe *pipe)
static int capture_start(struct ia_css_pipe *pipe)
{
struct ia_css_pipeline *me;
+ unsigned int thread_id;
int err = 0;
enum sh_css_pipe_config_override copy_ovrd;
@@ -8034,7 +7637,7 @@ static int capture_start(struct ia_css_pipe *pipe)
IA_CSS_LEAVE_ERR_PRIVATE(err);
return err;
}
-#elif defined(ISP2401)
+#else
if (pipe->config.mode != IA_CSS_PIPE_MODE_COPY) {
err = send_mipi_frames(pipe);
if (err) {
@@ -8042,23 +7645,19 @@ static int capture_start(struct ia_css_pipe *pipe)
return err;
}
}
-
#endif
- {
- unsigned int thread_id;
+ ia_css_pipeline_get_sp_thread_id(ia_css_pipe_get_pipe_num(pipe), &thread_id);
+ copy_ovrd = 1 << thread_id;
- ia_css_pipeline_get_sp_thread_id(ia_css_pipe_get_pipe_num(pipe), &thread_id);
- copy_ovrd = 1 << thread_id;
- }
start_pipe(pipe, copy_ovrd, pipe->stream->config.mode);
#if !defined(ISP2401)
/*
- * old isys: for IA_CSS_PIPE_MODE_COPY pipe, isys rx has to be configured,
- * which is currently done in start_binary(); but COPY pipe contains no binary,
- * and does not call start_binary(); so we need to configure the rx here.
- */
+ * old isys: for IA_CSS_PIPE_MODE_COPY pipe, isys rx has to be configured,
+ * which is currently done in start_binary(); but COPY pipe contains no binary,
+ * and does not call start_binary(); so we need to configure the rx here.
+ */
if (pipe->config.mode == IA_CSS_PIPE_MODE_COPY &&
pipe->stream->reconfigure_css_rx) {
ia_css_isys_rx_configure(&pipe->stream->csi_rx_config,
@@ -8174,7 +7773,8 @@ append_firmware(struct ia_css_fw_info **l, struct ia_css_fw_info *firmware)
while (*l)
l = &(*l)->next;
*l = firmware;
- /*firmware->next = NULL;*/ /* when multiple acc extensions are loaded, 'next' can be not NULL */
+ /* when multiple acc extensions are loaded, 'next' can be not NULL */
+ /*firmware->next = NULL;*/
IA_CSS_LEAVE_PRIVATE("");
}
@@ -8360,9 +7960,9 @@ sh_css_pipeline_add_acc_stage(struct ia_css_pipeline *pipeline,
}
/*
- * @brief Tag a specific frame in continuous capture.
- * Refer to "sh_css_internal.h" for details.
- */
+ * @brief Tag a specific frame in continuous capture.
+ * Refer to "sh_css_internal.h" for details.
+ */
int ia_css_stream_capture_frame(struct ia_css_stream *stream,
unsigned int exp_id)
{
@@ -8389,10 +7989,12 @@ int ia_css_stream_capture_frame(struct ia_css_stream *stream,
sh_css_create_tag_descr(0, 0, 0, exp_id, &tag_descr);
/* Encode the tag descriptor into a 32-bit value */
encoded_tag_descr = sh_css_encode_tag_descr(&tag_descr);
- /* Enqueue the encoded tag to the host2sp queue.
- * Note: The pipe and stage IDs for tag_cmd queue are hard-coded to 0
- * on both host and the SP side.
- * It is mainly because it is enough to have only one tag_cmd queue */
+ /*
+ * Enqueue the encoded tag to the host2sp queue.
+ * Note: The pipe and stage IDs for tag_cmd queue are hard-coded to 0
+ * on both host and the SP side.
+ * It is mainly because it is enough to have only one tag_cmd queue
+ */
err = ia_css_bufq_enqueue_tag_cmd(encoded_tag_descr);
IA_CSS_LEAVE_ERR(err);
@@ -8400,9 +8002,9 @@ int ia_css_stream_capture_frame(struct ia_css_stream *stream,
}
/*
- * @brief Configure the continuous capture.
- * Refer to "sh_css_internal.h" for details.
- */
+ * @brief Configure the continuous capture.
+ * Refer to "sh_css_internal.h" for details.
+ */
int ia_css_stream_capture(struct ia_css_stream *stream, int num_captures,
unsigned int skip, int offset)
{
@@ -8438,10 +8040,12 @@ int ia_css_stream_capture(struct ia_css_stream *stream, int num_captures,
return -EBUSY;
}
- /* Enqueue the encoded tag to the host2sp queue.
- * Note: The pipe and stage IDs for tag_cmd queue are hard-coded to 0
- * on both host and the SP side.
- * It is mainly because it is enough to have only one tag_cmd queue */
+ /*
+ * Enqueue the encoded tag to the host2sp queue.
+ * Note: The pipe and stage IDs for tag_cmd queue are hard-coded to 0
+ * on both host and the SP side.
+ * It is mainly because it is enough to have only one tag_cmd queue
+ */
return_err = ia_css_bufq_enqueue_tag_cmd((uint32_t)encoded_tag_descr);
ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE,
@@ -8486,9 +8090,7 @@ sh_css_init_host_sp_control_vars(void)
unsigned int HIVE_ADDR_host_sp_queues_initialized;
unsigned int HIVE_ADDR_sp_sleep_mode;
unsigned int HIVE_ADDR_ia_css_dmaproxy_sp_invalidate_tlb;
-#ifndef ISP2401
unsigned int HIVE_ADDR_sp_stop_copy_preview;
-#endif
unsigned int HIVE_ADDR_host_sp_com;
unsigned int o = offsetof(struct host_sp_communication, host2sp_command)
/ sizeof(int);
@@ -8505,20 +8107,9 @@ sh_css_init_host_sp_control_vars(void)
fw->info.sp.host_sp_queues_initialized;
HIVE_ADDR_sp_sleep_mode = fw->info.sp.sleep_mode;
HIVE_ADDR_ia_css_dmaproxy_sp_invalidate_tlb = fw->info.sp.invalidate_tlb;
-#ifndef ISP2401
HIVE_ADDR_sp_stop_copy_preview = fw->info.sp.stop_copy_preview;
-#endif
HIVE_ADDR_host_sp_com = fw->info.sp.host_sp_com;
- (void)HIVE_ADDR_ia_css_ispctrl_sp_isp_started; /* Suppres warnings in CRUN */
-
- (void)HIVE_ADDR_sp_sleep_mode;
- (void)HIVE_ADDR_ia_css_dmaproxy_sp_invalidate_tlb;
-#ifndef ISP2401
- (void)HIVE_ADDR_sp_stop_copy_preview;
-#endif
- (void)HIVE_ADDR_host_sp_com;
-
sp_dmem_store_uint32(SP0_ID,
(unsigned int)sp_address_of(ia_css_ispctrl_sp_isp_started),
(uint32_t)(0));
@@ -8532,11 +8123,9 @@ sh_css_init_host_sp_control_vars(void)
sp_dmem_store_uint32(SP0_ID,
(unsigned int)sp_address_of(ia_css_dmaproxy_sp_invalidate_tlb),
(uint32_t)(false));
-#ifndef ISP2401
sp_dmem_store_uint32(SP0_ID,
(unsigned int)sp_address_of(sp_stop_copy_preview),
my_css.stop_copy_preview ? (uint32_t)(1) : (uint32_t)(0));
-#endif
store_sp_array_uint(host_sp_com, o, host2sp_cmd_ready);
for (i = 0; i < N_CSI_PORTS; i++) {
@@ -8586,9 +8175,11 @@ void ia_css_stream_config_defaults(struct ia_css_stream_config *stream_config)
stream_config->online = true;
stream_config->left_padding = -1;
stream_config->pixels_per_clock = 1;
- /* temporary default value for backwards compatibility.
- * This field used to be hardcoded within CSS but this has now
- * been moved to the stream_config struct. */
+ /*
+ * temporary default value for backwards compatibility.
+ * This field used to be hardcoded within CSS but this has now
+ * been moved to the stream_config struct.
+ */
stream_config->source.port.rxcount = 0x04040404;
}
@@ -8602,7 +8193,7 @@ ia_css_acc_pipe_create(struct ia_css_pipe *pipe)
return -EINVAL;
}
- /* There is not meaning for num_execs = 0 semantically. Run atleast once. */
+ /* There is not meaning for num_execs = 0 semantically. Run at least once. */
if (pipe->config.acc_num_execs == 0)
pipe->config.acc_num_execs = 1;
@@ -8673,9 +8264,11 @@ ia_css_pipe_create_extra(const struct ia_css_pipe_config *config,
ia_css_pipe_extra_config_defaults(&internal_pipe->extra_config);
if (config->mode == IA_CSS_PIPE_MODE_ACC) {
- /* Temporary hack to migrate acceleration to CSS 2.0.
- * In the future the code for all pipe types should be
- * unified. */
+ /*
+ * Temporary hack to migrate acceleration to CSS 2.0.
+ * In the future the code for all pipe types should be
+ * unified.
+ */
*pipe = internal_pipe;
if (!internal_pipe->config.acc_extension &&
internal_pipe->config.num_acc_stages ==
@@ -8687,21 +8280,23 @@ ia_css_pipe_create_extra(const struct ia_css_pipe_config *config,
return ia_css_acc_pipe_create(internal_pipe);
}
- /* Use config value when dvs_frame_delay setting equal to 2, otherwise always 1 by default */
+ /*
+ * Use config value when dvs_frame_delay setting equal to 2,
+ * otherwise always 1 by default
+ */
if (internal_pipe->config.dvs_frame_delay == IA_CSS_FRAME_DELAY_2)
internal_pipe->dvs_frame_delay = 2;
else
internal_pipe->dvs_frame_delay = 1;
- /* we still keep enable_raw_binning for backward compatibility, for any new
- fractional bayer downscaling, we should use bayer_ds_out_res. if both are
- specified, bayer_ds_out_res will take precedence.if none is specified, we
- set bayer_ds_out_res equal to IF output resolution(IF may do cropping on
- sensor output) or use default decimation factor 1. */
- if (internal_pipe->extra_config.enable_raw_binning &&
- internal_pipe->config.bayer_ds_out_res.width) {
- /* fill some code here, if no code is needed, please remove it during integration */
- }
+ /*
+ * we still keep enable_raw_binning for backward compatibility,
+ * for any new fractional bayer downscaling, we should use
+ * bayer_ds_out_res. if both are specified, bayer_ds_out_res will
+ * take precedence.if none is specified, we set bayer_ds_out_res
+ * equal to IF output resolution(IF may do cropping on sensor output)
+ * or use default decimation factor 1.
+ */
/* YUV downscaling */
if ((internal_pipe->config.vf_pp_in_res.width ||
@@ -8905,8 +8500,10 @@ ia_css_stream_configure_rx(struct ia_css_stream *stream)
if (config->compression.type == IA_CSS_CSI2_COMPRESSION_TYPE_NONE)
stream->csi_rx_config.comp = MIPI_PREDICTOR_NONE;
else
- /* not implemented yet, requires extension of the rx_cfg_t
- * struct */
+ /*
+ * not implemented yet, requires extension of the rx_cfg_t
+ * struct
+ */
return -EINVAL;
stream->csi_rx_config.is_two_ppc = (stream->config.pixels_per_clock == 2);
@@ -8993,56 +8590,15 @@ metadata_info_init(const struct ia_css_metadata_config *mdc,
return -EINVAL;
md->resolution = mdc->resolution;
- /* We round up the stride to a multiple of the width
- * of the port going to DDR, this is a HW requirements (DMA). */
+ /*
+ * We round up the stride to a multiple of the width
+ * of the port going to DDR, this is a HW requirements (DMA).
+ */
md->stride = CEIL_MUL(mdc->resolution.width, HIVE_ISP_DDR_WORD_BYTES);
md->size = mdc->resolution.height * md->stride;
return 0;
}
-/* ISP2401 */
-static int check_pipe_resolutions(const struct ia_css_pipe *pipe)
-{
- int err = 0;
-
- IA_CSS_ENTER_PRIVATE("");
-
- if (!pipe || !pipe->stream) {
- IA_CSS_ERROR("null arguments");
- err = -EINVAL;
- goto EXIT;
- }
-
- if (ia_css_util_check_res(pipe->config.input_effective_res.width,
- pipe->config.input_effective_res.height) != 0) {
- IA_CSS_ERROR("effective resolution not supported");
- err = -EINVAL;
- goto EXIT;
- }
- if (!ia_css_util_resolution_is_zero(
- pipe->stream->config.input_config.input_res)) {
- if (!ia_css_util_res_leq(pipe->config.input_effective_res,
- pipe->stream->config.input_config.input_res)) {
- IA_CSS_ERROR("effective resolution is larger than input resolution");
- err = -EINVAL;
- goto EXIT;
- }
- }
- if (!ia_css_util_resolution_is_even(pipe->config.output_info[0].res)) {
- IA_CSS_ERROR("output resolution must be even");
- err = -EINVAL;
- goto EXIT;
- }
- if (!ia_css_util_resolution_is_even(pipe->config.vf_output_info[0].res)) {
- IA_CSS_ERROR("VF resolution must be even");
- err = -EINVAL;
- goto EXIT;
- }
-EXIT:
- IA_CSS_LEAVE_ERR_PRIVATE(err);
- return err;
-}
-
int
ia_css_stream_create(const struct ia_css_stream_config *stream_config,
int num_pipes,
@@ -9057,9 +8613,6 @@ ia_css_stream_create(const struct ia_css_stream_config *stream_config,
int err = -EINVAL;
struct ia_css_metadata_info md_info;
struct ia_css_resolution effective_res;
-#ifdef ISP2401
- bool aspect_ratio_crop_enabled = false;
-#endif
IA_CSS_ENTER("num_pipes=%d", num_pipes);
ia_css_debug_dump_stream_config(stream_config, num_pipes);
@@ -9236,14 +8789,6 @@ ia_css_stream_create(const struct ia_css_stream_config *stream_config,
IA_CSS_LOG("mode sensor/default");
}
-#ifdef ISP2401
- err = aspect_ratio_crop_init(curr_stream, pipes,
- &aspect_ratio_crop_enabled);
- if (err) {
- IA_CSS_LEAVE_ERR(err);
- goto ERR;
- }
-#endif
for (i = 0; i < num_pipes; i++) {
struct ia_css_resolution effective_res;
@@ -9256,22 +8801,6 @@ ia_css_stream_create(const struct ia_css_stream_config *stream_config,
if (effective_res.height == 0 || effective_res.width == 0) {
effective_res = curr_pipe->stream->config.input_config.effective_res;
-#if defined(ISP2401)
- /* The aspect ratio cropping is currently only
- * supported on the new input system. */
- if (aspect_ratio_crop_check(aspect_ratio_crop_enabled, curr_pipe)) {
- struct ia_css_resolution crop_res;
-
- err = aspect_ratio_crop(curr_pipe, &crop_res);
- if (!err) {
- effective_res = crop_res;
- } else {
- /* in case of error fallback to default
- * effective resolution from driver. */
- IA_CSS_LOG("aspect_ratio_crop() failed with err(%d)", err);
- }
- }
-#endif
curr_pipe->config.input_effective_res = effective_res;
}
IA_CSS_LOG("effective_res=%dx%d",
@@ -9279,17 +8808,6 @@ ia_css_stream_create(const struct ia_css_stream_config *stream_config,
effective_res.height);
}
- if (IS_ISP2401) {
- for (i = 0; i < num_pipes; i++) {
- if (pipes[i]->config.mode != IA_CSS_PIPE_MODE_ACC &&
- pipes[i]->config.mode != IA_CSS_PIPE_MODE_COPY) {
- err = check_pipe_resolutions(pipes[i]);
- if (err)
- goto ERR;
- }
- }
- }
-
err = ia_css_stream_isp_parameters_init(curr_stream);
if (err)
goto ERR;
@@ -9327,9 +8845,7 @@ ia_css_stream_create(const struct ia_css_stream_config *stream_config,
if (num_pipes >= 2) {
curr_stream->cont_capt = true;
curr_stream->disable_cont_vf = curr_stream->config.disable_cont_viewfinder;
-
- if (!IS_ISP2401)
- curr_stream->stop_copy_preview = my_css.stop_copy_preview;
+ curr_stream->stop_copy_preview = my_css.stop_copy_preview;
}
/* Create copy pipe here, since it may not be exposed to the driver */
@@ -9387,16 +8903,15 @@ ia_css_stream_create(const struct ia_css_stream_config *stream_config,
/* set current stream */
curr_pipe->stream = curr_stream;
- if (!IS_ISP2401) {
- /* take over effective info */
+ /* take over effective info */
- effective_res = curr_pipe->config.input_effective_res;
- err = ia_css_util_check_res(
- effective_res.width,
- effective_res.height);
- if (err)
- goto ERR;
- }
+ effective_res = curr_pipe->config.input_effective_res;
+ err = ia_css_util_check_res(
+ effective_res.width,
+ effective_res.height);
+ if (err)
+ goto ERR;
+
/* sensor binning per pipe */
if (sensor_binning_changed)
sh_css_pipe_free_shading_table(curr_pipe);
@@ -9421,9 +8936,6 @@ ia_css_stream_create(const struct ia_css_stream_config *stream_config,
goto ERR;
}
- if (IS_ISP2401)
- pipe_info->output_system_in_res_info = curr_pipe->config.output_system_in_res;
-
if (!spcopyonly) {
if (!IS_ISP2401)
err = sh_css_pipe_get_shading_info(curr_pipe,
@@ -9523,8 +9035,6 @@ ia_css_stream_destroy(struct ia_css_stream *stream)
if ((stream->last_pipe) &&
ia_css_pipeline_is_mapped(stream->last_pipe->pipe_num)) {
#if defined(ISP2401)
- bool free_mpi;
-
for (i = 0; i < stream->num_pipes; i++) {
struct ia_css_pipe *entry = stream->pipes[i];
unsigned int sp_thread_id;
@@ -9548,19 +9058,16 @@ ia_css_stream_destroy(struct ia_css_stream *stream)
}
}
}
- free_mpi = stream->config.mode == IA_CSS_INPUT_MODE_BUFFERED_SENSOR;
- if (IS_ISP2401) {
- free_mpi |= stream->config.mode == IA_CSS_INPUT_MODE_TPG;
- free_mpi |= stream->config.mode == IA_CSS_INPUT_MODE_PRBS;
- }
-
- if (free_mpi) {
+ if (stream->config.mode == IA_CSS_INPUT_MODE_BUFFERED_SENSOR) {
for (i = 0; i < stream->num_pipes; i++) {
struct ia_css_pipe *entry = stream->pipes[i];
- /* free any mipi frames that are remaining:
- * some test stream create-destroy cycles do not generate output frames
- * and the mipi buffer is not freed in the deque function
- */
+ /*
+ * free any mipi frames that are remaining:
+ * some test stream create-destroy cycles do
+ * not generate output frames
+ * and the mipi buffer is not freed in the
+ * deque function
+ */
if (entry)
free_mipi_frames(entry);
}
@@ -9638,58 +9145,6 @@ ia_css_stream_get_info(const struct ia_css_stream *stream,
return 0;
}
-/*
- * Rebuild a stream, including allocating structs, setting configuration and
- * building the required pipes.
- * The data is taken from the css_save struct updated upon stream creation.
- * The stream handle is used to identify the correct entry in the css_save struct
- */
-int
-ia_css_stream_load(struct ia_css_stream *stream)
-{
- int i, j, err;
-
- if (IS_ISP2401) {
- /* TODO remove function - DEPRECATED */
- (void)stream;
- return -ENOTSUPP;
- }
-
- assert(stream);
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_stream_load() enter,\n");
- for (i = 0; i < MAX_ACTIVE_STREAMS; i++) {
- if (my_css_save.stream_seeds[i].stream != stream)
- continue;
-
- for (j = 0; j < my_css_save.stream_seeds[i].num_pipes; j++) {
- int k;
-
- err = ia_css_pipe_create(&my_css_save.stream_seeds[i].pipe_config[j],
- &my_css_save.stream_seeds[i].pipes[j]);
- if (!err)
- continue;
-
- for (k = 0; k < j; k++)
- ia_css_pipe_destroy(my_css_save.stream_seeds[i].pipes[k]);
- return err;
- }
- err = ia_css_stream_create(&my_css_save.stream_seeds[i].stream_config,
- my_css_save.stream_seeds[i].num_pipes,
- my_css_save.stream_seeds[i].pipes,
- &my_css_save.stream_seeds[i].stream);
- if (!err)
- break;
-
- ia_css_stream_destroy(stream);
- for (j = 0; j < my_css_save.stream_seeds[i].num_pipes; j++)
- ia_css_pipe_destroy(my_css_save.stream_seeds[i].pipes[j]);
- return err;
- }
-
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "ia_css_stream_load() exit,\n");
- return 0;
-}
-
int
ia_css_stream_start(struct ia_css_stream *stream)
{
@@ -9763,16 +9218,14 @@ ia_css_stream_stop(struct ia_css_stream *stream)
}
#endif
- if (!IS_ISP2401)
- err = ia_css_pipeline_request_stop(&stream->last_pipe->pipeline);
- else
- err = sh_css_pipes_stop(stream);
-
+ err = ia_css_pipeline_request_stop(&stream->last_pipe->pipeline);
if (err)
return err;
- /* Ideally, unmapping should happen after pipeline_stop, but current
- * semantics do not allow that. */
+ /*
+ * Ideally, unmapping should happen after pipeline_stop, but current
+ * semantics do not allow that.
+ */
/* err = map_sp_threads(stream, false); */
return err;
@@ -9785,19 +9238,16 @@ ia_css_stream_has_stopped(struct ia_css_stream *stream)
assert(stream);
- if (!IS_ISP2401)
- stopped = ia_css_pipeline_has_stopped(&stream->last_pipe->pipeline);
- else
- stopped = sh_css_pipes_have_stopped(stream);
+ stopped = ia_css_pipeline_has_stopped(&stream->last_pipe->pipeline);
return stopped;
}
/* ISP2400 */
/*
- * Destroy the stream and all the pipes related to it.
- * The stream handle is used to identify the correct entry in the css_save struct
- */
+ * Destroy the stream and all the pipes related to it.
+ * The stream handle is used to identify the correct entry in the css_save struct
+ */
int
ia_css_stream_unload(struct ia_css_stream *stream)
{
@@ -10060,11 +9510,12 @@ ia_css_pipe_get_pipe_num(const struct ia_css_pipe *pipe)
{
assert(pipe);
- /* KW was not sure this function was not returning a value
- that was out of range; so added an assert, and, for the
- case when asserts are not enabled, clip to the largest
- value; pipe_num is unsigned so the value cannot be too small
- */
+ /*
+ * KW was not sure this function was not returning a value
+ * that was out of range; so added an assert, and, for the
+ * case when asserts are not enabled, clip to the largest
+ * value; pipe_num is unsigned so the value cannot be too small
+ */
assert(pipe->pipe_num < IA_CSS_PIPELINE_NUM_MAX);
if (pipe->pipe_num >= IA_CSS_PIPELINE_NUM_MAX)
@@ -10119,10 +9570,10 @@ ia_css_start_sp(void)
}
/*
- * Time to wait SP for termincate. Only condition when this can happen
- * is a fatal hw failure, but we must be able to detect this and emit
- * a proper error trace.
- */
+ * Time to wait SP for termincate. Only condition when this can happen
+ * is a fatal hw failure, but we must be able to detect this and emit
+ * a proper error trace.
+ */
#define SP_SHUTDOWN_TIMEOUT_US 200000
int
@@ -10142,14 +9593,10 @@ ia_css_stop_sp(void)
}
/* For now, stop whole SP */
- if (!IS_ISP2401) {
- sh_css_write_host2sp_command(host2sp_cmd_terminate);
- } else {
- if (!sh_css_write_host2sp_command(host2sp_cmd_terminate)) {
- IA_CSS_ERROR("Call to 'sh-css_write_host2sp_command()' failed");
- ia_css_debug_dump_sp_sw_debug_info();
- ia_css_debug_dump_debug_info(NULL);
- }
+ if (!sh_css_write_host2sp_command(host2sp_cmd_terminate)) {
+ IA_CSS_ERROR("Call to 'sh-css_write_host2sp_command()' failed");
+ ia_css_debug_dump_sp_sw_debug_info();
+ ia_css_debug_dump_debug_info(NULL);
}
sh_css_sp_set_sp_running(false);
@@ -10245,9 +9692,7 @@ void ia_css_pipe_map_queue(struct ia_css_pipe *pipe, bool map)
ia_css_queue_map(thread_id, IA_CSS_BUFFER_TYPE_OUTPUT_FRAME, map);
ia_css_queue_map(thread_id, IA_CSS_BUFFER_TYPE_PARAMETER_SET, map);
ia_css_queue_map(thread_id, IA_CSS_BUFFER_TYPE_PER_FRAME_PARAMETER_SET, map);
-#if defined SH_CSS_ENABLE_METADATA
ia_css_queue_map(thread_id, IA_CSS_BUFFER_TYPE_METADATA, map);
-#endif
if (pipe->pipe_settings.preview.preview_binary.info &&
pipe->pipe_settings.preview.preview_binary.info->sp.enable.s3a)
ia_css_queue_map(thread_id, IA_CSS_BUFFER_TYPE_3A_STATISTICS, map);
@@ -10260,9 +9705,7 @@ void ia_css_pipe_map_queue(struct ia_css_pipe *pipe, bool map)
ia_css_queue_map(thread_id, IA_CSS_BUFFER_TYPE_VF_OUTPUT_FRAME, map);
ia_css_queue_map(thread_id, IA_CSS_BUFFER_TYPE_PARAMETER_SET, map);
ia_css_queue_map(thread_id, IA_CSS_BUFFER_TYPE_PER_FRAME_PARAMETER_SET, map);
-#if defined SH_CSS_ENABLE_METADATA
ia_css_queue_map(thread_id, IA_CSS_BUFFER_TYPE_METADATA, map);
-#endif
if (pipe->config.default_capture_config.mode == IA_CSS_CAPTURE_MODE_PRIMARY) {
for (i = 0; i < pipe->pipe_settings.capture.num_primary_stage; i++) {
if (pipe->pipe_settings.capture.primary_binary[i].info &&
@@ -10287,9 +9730,7 @@ void ia_css_pipe_map_queue(struct ia_css_pipe *pipe, bool map)
ia_css_queue_map(thread_id, IA_CSS_BUFFER_TYPE_VF_OUTPUT_FRAME, map);
ia_css_queue_map(thread_id, IA_CSS_BUFFER_TYPE_PARAMETER_SET, map);
ia_css_queue_map(thread_id, IA_CSS_BUFFER_TYPE_PER_FRAME_PARAMETER_SET, map);
-#if defined SH_CSS_ENABLE_METADATA
ia_css_queue_map(thread_id, IA_CSS_BUFFER_TYPE_METADATA, map);
-#endif
if (pipe->pipe_settings.video.video_binary.info &&
pipe->pipe_settings.video.video_binary.info->sp.enable.s3a)
ia_css_queue_map(thread_id, IA_CSS_BUFFER_TYPE_3A_STATISTICS, map);
@@ -10302,18 +9743,14 @@ void ia_css_pipe_map_queue(struct ia_css_pipe *pipe, bool map)
ia_css_queue_map(thread_id, IA_CSS_BUFFER_TYPE_INPUT_FRAME, map);
if (!pipe->stream->config.continuous)
ia_css_queue_map(thread_id, IA_CSS_BUFFER_TYPE_OUTPUT_FRAME, map);
-#if defined SH_CSS_ENABLE_METADATA
ia_css_queue_map(thread_id, IA_CSS_BUFFER_TYPE_METADATA, map);
-#endif
} else if (pipe->mode == IA_CSS_PIPE_ID_ACC) {
if (need_input_queue)
ia_css_queue_map(thread_id, IA_CSS_BUFFER_TYPE_INPUT_FRAME, map);
ia_css_queue_map(thread_id, IA_CSS_BUFFER_TYPE_OUTPUT_FRAME, map);
ia_css_queue_map(thread_id, IA_CSS_BUFFER_TYPE_PARAMETER_SET, map);
ia_css_queue_map(thread_id, IA_CSS_BUFFER_TYPE_PER_FRAME_PARAMETER_SET, map);
-#if defined SH_CSS_ENABLE_METADATA
ia_css_queue_map(thread_id, IA_CSS_BUFFER_TYPE_METADATA, map);
-#endif
} else if (pipe->mode == IA_CSS_PIPE_ID_YUVPP) {
unsigned int idx;
@@ -10325,38 +9762,11 @@ void ia_css_pipe_map_queue(struct ia_css_pipe *pipe, bool map)
if (need_input_queue)
ia_css_queue_map(thread_id, IA_CSS_BUFFER_TYPE_INPUT_FRAME, map);
ia_css_queue_map(thread_id, IA_CSS_BUFFER_TYPE_PARAMETER_SET, map);
-#if defined SH_CSS_ENABLE_METADATA
ia_css_queue_map(thread_id, IA_CSS_BUFFER_TYPE_METADATA, map);
-#endif
}
IA_CSS_LEAVE("");
}
-#if CONFIG_ON_FRAME_ENQUEUE()
-static int set_config_on_frame_enqueue(struct ia_css_frame_info
- *info, struct frame_data_wrapper *frame)
-{
- frame->config_on_frame_enqueue.padded_width = 0;
-
- /* currently we support configuration on frame enqueue only on YUV formats */
- /* on other formats the padded_width is zeroed for no configuration override */
- switch (info->format) {
- case IA_CSS_FRAME_FORMAT_YUV420:
- case IA_CSS_FRAME_FORMAT_NV12:
- if (info->padded_width > info->res.width)
- frame->config_on_frame_enqueue.padded_width = info->padded_width;
- else if ((info->padded_width < info->res.width) && (info->padded_width > 0))
- return -EINVAL;
-
- /* nothing to do if width == padded width or padded width is zeroed (the same) */
- break;
- default:
- break;
- }
-
- return 0;
-}
-#endif
int
ia_css_unlock_raw_frame(struct ia_css_stream *stream, uint32_t exp_id)
@@ -10365,8 +9775,10 @@ ia_css_unlock_raw_frame(struct ia_css_stream *stream, uint32_t exp_id)
IA_CSS_ENTER("");
- /* Only continuous streams have a tagger to which we can send the
- * unlock message. */
+ /*
+ * Only continuous streams have a tagger to which we can send the
+ * unlock message.
+ */
if (!stream || !stream->config.continuous) {
IA_CSS_ERROR("invalid stream pointer");
return -EINVAL;
@@ -10378,8 +9790,10 @@ ia_css_unlock_raw_frame(struct ia_css_stream *stream, uint32_t exp_id)
return -EINVAL;
}
- /* Send the event. Since we verified that the exp_id is valid,
- * we can safely assign it to an 8-bit argument here. */
+ /*
+ * Send the event. Since we verified that the exp_id is valid,
+ * we can safely assign it to an 8-bit argument here.
+ */
ret = ia_css_bufq_enqueue_psys_event(
IA_CSS_PSYS_SW_EVENT_UNLOCK_RAW_BUFFER, exp_id, 0, 0);
@@ -10387,9 +9801,10 @@ ia_css_unlock_raw_frame(struct ia_css_stream *stream, uint32_t exp_id)
return ret;
}
-/* @brief Set the state (Enable or Disable) of the Extension stage in the
- * given pipe.
- */
+/*
+ * @brief Set the state (Enable or Disable) of the Extension stage in the
+ * given pipe.
+ */
int
ia_css_pipe_set_qos_ext_state(struct ia_css_pipe *pipe, uint32_t fw_handle,
bool enable)
@@ -10433,9 +9848,10 @@ ia_css_pipe_set_qos_ext_state(struct ia_css_pipe *pipe, uint32_t fw_handle,
return err;
}
-/* @brief Get the state (Enable or Disable) of the Extension stage in the
- * given pipe.
- */
+/*
+ * @brief Get the state (Enable or Disable) of the Extension stage in the
+ * given pipe.
+ */
int
ia_css_pipe_get_qos_ext_state(struct ia_css_pipe *pipe, uint32_t fw_handle,
bool *enable)
@@ -10471,215 +9887,6 @@ ia_css_pipe_get_qos_ext_state(struct ia_css_pipe *pipe, uint32_t fw_handle,
return err;
}
-/* ISP2401 */
-int
-ia_css_pipe_update_qos_ext_mapped_arg(struct ia_css_pipe *pipe,
- u32 fw_handle,
- struct ia_css_isp_param_css_segments *css_seg,
- struct ia_css_isp_param_isp_segments *isp_seg)
-{
- unsigned int HIVE_ADDR_sp_group;
- static struct sh_css_sp_group sp_group;
- static struct sh_css_sp_stage sp_stage;
- static struct sh_css_isp_stage isp_stage;
- const struct ia_css_fw_info *fw;
- unsigned int thread_id;
- struct ia_css_pipeline_stage *stage;
- int err = 0;
- int stage_num = 0;
- enum ia_css_isp_memories mem;
- bool enabled;
-
- IA_CSS_ENTER("");
-
- fw = &sh_css_sp_fw;
-
- /* Parameter Check */
- if (!pipe || !pipe->stream) {
- IA_CSS_ERROR("Invalid Pipe.");
- err = -EINVAL;
- } else if (!(pipe->config.acc_extension)) {
- IA_CSS_ERROR("Invalid Pipe (No Extension Firmware).");
- err = -EINVAL;
- } else if (!sh_css_sp_is_running()) {
- IA_CSS_ERROR("Leaving: queue unavailable.");
- err = -EBUSY;
- } else {
- /* Query the thread_id and stage_num corresponding to the Extension firmware */
- ia_css_pipeline_get_sp_thread_id(ia_css_pipe_get_pipe_num(pipe), &thread_id);
- err = ia_css_pipeline_get_stage_from_fw(&pipe->pipeline, fw_handle, &stage);
- if (!err) {
- /* Get the Extension State */
- enabled = (SH_CSS_QOS_STAGE_IS_ENABLED(&sh_css_sp_group.pipe[thread_id],
- stage->stage_num)) ? true : false;
- /* Update mapped arg only when extension stage is not enabled */
- if (enabled) {
- IA_CSS_ERROR("Leaving: cannot update when stage is enabled.");
- err = -EBUSY;
- } else {
- stage_num = stage->stage_num;
-
- HIVE_ADDR_sp_group = fw->info.sp.group;
- sp_dmem_load(SP0_ID,
- (unsigned int)sp_address_of(sp_group),
- &sp_group,
- sizeof(struct sh_css_sp_group));
- hmm_load(sp_group.pipe[thread_id].sp_stage_addr[stage_num],
- &sp_stage, sizeof(struct sh_css_sp_stage));
-
- hmm_load(sp_stage.isp_stage_addr,
- &isp_stage, sizeof(struct sh_css_isp_stage));
-
- for (mem = 0; mem < N_IA_CSS_ISP_MEMORIES; mem++) {
- isp_stage.mem_initializers.params[IA_CSS_PARAM_CLASS_PARAM][mem].address =
- css_seg->params[IA_CSS_PARAM_CLASS_PARAM][mem].address;
- isp_stage.mem_initializers.params[IA_CSS_PARAM_CLASS_PARAM][mem].size =
- css_seg->params[IA_CSS_PARAM_CLASS_PARAM][mem].size;
- isp_stage.binary_info.mem_initializers.params[IA_CSS_PARAM_CLASS_PARAM][mem].address
- =
- isp_seg->params[IA_CSS_PARAM_CLASS_PARAM][mem].address;
- isp_stage.binary_info.mem_initializers.params[IA_CSS_PARAM_CLASS_PARAM][mem].size
- =
- isp_seg->params[IA_CSS_PARAM_CLASS_PARAM][mem].size;
- }
-
- hmm_store(sp_stage.isp_stage_addr,
- &isp_stage,
- sizeof(struct sh_css_isp_stage));
- }
- }
- }
- IA_CSS_LEAVE("err:%d handle:%u", err, fw_handle);
- return err;
-}
-
-#ifdef ISP2401
-static int
-aspect_ratio_crop_init(struct ia_css_stream *curr_stream,
- struct ia_css_pipe *pipes[],
- bool *do_crop_status)
-{
- int err = 0;
- int i;
- struct ia_css_pipe *curr_pipe;
- u32 pipe_mask = 0;
-
- if ((!curr_stream) ||
- (curr_stream->num_pipes == 0) ||
- (!pipes) ||
- (!do_crop_status)) {
- err = -EINVAL;
- IA_CSS_LEAVE_ERR(err);
- return err;
- }
-
- for (i = 0; i < curr_stream->num_pipes; i++) {
- curr_pipe = pipes[i];
- pipe_mask |= (1 << curr_pipe->config.mode);
- }
-
- *do_crop_status =
- (((pipe_mask & (1 << IA_CSS_PIPE_MODE_PREVIEW)) ||
- (pipe_mask & (1 << IA_CSS_PIPE_MODE_VIDEO))) &&
- (pipe_mask & (1 << IA_CSS_PIPE_MODE_CAPTURE)) &&
- curr_stream->config.continuous);
- return 0;
-}
-
-static bool
-aspect_ratio_crop_check(bool enabled, struct ia_css_pipe *curr_pipe)
-{
- bool status = false;
-
- if ((curr_pipe) && enabled) {
- if ((curr_pipe->config.mode == IA_CSS_PIPE_MODE_PREVIEW) ||
- (curr_pipe->config.mode == IA_CSS_PIPE_MODE_VIDEO) ||
- (curr_pipe->config.mode == IA_CSS_PIPE_MODE_CAPTURE))
- status = true;
- }
-
- return status;
-}
-
-static int
-aspect_ratio_crop(struct ia_css_pipe *curr_pipe,
- struct ia_css_resolution *effective_res)
-{
- int err = 0;
- struct ia_css_resolution crop_res;
- struct ia_css_resolution *in_res = NULL;
- struct ia_css_resolution *out_res = NULL;
- bool use_bds_output_info = false;
- bool use_vf_pp_in_res = false;
- bool use_capt_pp_in_res = false;
-
- if ((!curr_pipe) ||
- (!effective_res)) {
- err = -EINVAL;
- IA_CSS_LEAVE_ERR(err);
- return err;
- }
-
- if ((curr_pipe->config.mode != IA_CSS_PIPE_MODE_PREVIEW) &&
- (curr_pipe->config.mode != IA_CSS_PIPE_MODE_VIDEO) &&
- (curr_pipe->config.mode != IA_CSS_PIPE_MODE_CAPTURE)) {
- err = -EINVAL;
- IA_CSS_LEAVE_ERR(err);
- return err;
- }
-
- use_bds_output_info =
- ((curr_pipe->bds_output_info.res.width != 0) &&
- (curr_pipe->bds_output_info.res.height != 0));
-
- use_vf_pp_in_res =
- ((curr_pipe->config.vf_pp_in_res.width != 0) &&
- (curr_pipe->config.vf_pp_in_res.height != 0));
-
- use_capt_pp_in_res =
- ((curr_pipe->config.capt_pp_in_res.width != 0) &&
- (curr_pipe->config.capt_pp_in_res.height != 0));
-
- in_res = &curr_pipe->stream->config.input_config.effective_res;
- out_res = &curr_pipe->output_info[0].res;
-
- switch (curr_pipe->config.mode) {
- case IA_CSS_PIPE_MODE_PREVIEW:
- if (use_bds_output_info)
- out_res = &curr_pipe->bds_output_info.res;
- else if (use_vf_pp_in_res)
- out_res = &curr_pipe->config.vf_pp_in_res;
- break;
- case IA_CSS_PIPE_MODE_VIDEO:
- if (use_bds_output_info)
- out_res = &curr_pipe->bds_output_info.res;
- break;
- case IA_CSS_PIPE_MODE_CAPTURE:
- if (use_capt_pp_in_res)
- out_res = &curr_pipe->config.capt_pp_in_res;
- break;
- case IA_CSS_PIPE_MODE_ACC:
- case IA_CSS_PIPE_MODE_COPY:
- case IA_CSS_PIPE_MODE_YUVPP:
- default:
- IA_CSS_ERROR("aspect ratio cropping invalid args: mode[%d]\n",
- curr_pipe->config.mode);
- assert(0);
- break;
- }
-
- err = ia_css_frame_find_crop_resolution(in_res, out_res, &crop_res);
- if (!err)
- *effective_res = crop_res;
- else
- /* in case of error fallback to default
- * effective resolution from driver. */
- IA_CSS_LOG("ia_css_frame_find_crop_resolution() failed with err(%d)", err);
-
- return err;
-}
-#endif
-
static void
sh_css_hmm_buffer_record_init(void)
{
diff --git a/drivers/staging/media/atomisp/pci/sh_css_defs.h b/drivers/staging/media/atomisp/pci/sh_css_defs.h
index 30a84a5..7eb10b2 100644
--- a/drivers/staging/media/atomisp/pci/sh_css_defs.h
+++ b/drivers/staging/media/atomisp/pci/sh_css_defs.h
@@ -117,13 +117,8 @@ RGB[0,8191],coef[-8192,8191] -> RGB[0,8191]
#define SH_CSS_NUM_INPUT_BUF_LINES 4
/* Left cropping only applicable for sufficiently large nway */
-#if ISP_VEC_NELEMS == 16
-#define SH_CSS_MAX_LEFT_CROPPING 0
-#define SH_CSS_MAX_TOP_CROPPING 0
-#else
#define SH_CSS_MAX_LEFT_CROPPING 12
#define SH_CSS_MAX_TOP_CROPPING 12
-#endif
#define SH_CSS_SP_MAX_WIDTH 1280
@@ -137,13 +132,8 @@ RGB[0,8191],coef[-8192,8191] -> RGB[0,8191]
#define SH_CSS_MIN_DVS_ENVELOPE 12U
/* The FPGA system (vec_nelems == 16) only supports upto 5MP */
-#if ISP_VEC_NELEMS == 16
-#define SH_CSS_MAX_SENSOR_WIDTH 2560
-#define SH_CSS_MAX_SENSOR_HEIGHT 1920
-#else
#define SH_CSS_MAX_SENSOR_WIDTH 4608
#define SH_CSS_MAX_SENSOR_HEIGHT 3450
-#endif
/* Limited to reduce vmem pressure */
#if ISP_VMEM_DEPTH >= 3072
@@ -178,50 +168,20 @@ RGB[0,8191],coef[-8192,8191] -> RGB[0,8191]
#define SH_CSS_MORPH_TABLE_ELEMS_PER_DDR_WORD \
(HIVE_ISP_DDR_WORD_BYTES / SH_CSS_MORPH_TABLE_ELEM_BYTES)
-#define ISP2400_SH_CSS_MAX_SCTBL_WIDTH_PER_COLOR (SH_CSS_MAX_BQ_GRID_WIDTH + 1)
-#define ISP2400_SH_CSS_MAX_SCTBL_HEIGHT_PER_COLOR (SH_CSS_MAX_BQ_GRID_HEIGHT + 1)
+#define SH_CSS_MAX_SCTBL_WIDTH_PER_COLOR (SH_CSS_MAX_BQ_GRID_WIDTH + 1)
+#define SH_CSS_MAX_SCTBL_HEIGHT_PER_COLOR (SH_CSS_MAX_BQ_GRID_HEIGHT + 1)
-#define ISP2400_SH_CSS_MAX_SCTBL_ALIGNED_WIDTH_PER_COLOR \
- CEIL_MUL(ISP2400_SH_CSS_MAX_SCTBL_WIDTH_PER_COLOR, ISP_VEC_NELEMS)
-
-/* TODO: I will move macros of "*_SCTBL_*" to SC kernel.
- "+ 2" should be "+ SH_CSS_SCTBL_CENTERING_MARGIN + SH_CSS_SCTBL_LAST_GRID_COUNT". (michie, Sep/23/2014) */
-#define ISP2401_SH_CSS_MAX_SCTBL_WIDTH_PER_COLOR (SH_CSS_MAX_BQ_GRID_WIDTH + 2)
-#define ISP2401_SH_CSS_MAX_SCTBL_HEIGHT_PER_COLOR (SH_CSS_MAX_BQ_GRID_HEIGHT + 2)
-
-#define ISP2401_SH_CSS_MAX_SCTBL_ALIGNED_WIDTH_PER_COLOR \
- CEIL_MUL(ISP2400_SH_CSS_MAX_SCTBL_WIDTH_PER_COLOR, ISP_VEC_NELEMS)
+#define SH_CSS_MAX_SCTBL_ALIGNED_WIDTH_PER_COLOR \
+ CEIL_MUL(SH_CSS_MAX_SCTBL_WIDTH_PER_COLOR, ISP_VEC_NELEMS)
/* Each line of this table is aligned to the maximum line width. */
#define SH_CSS_MAX_S3ATBL_WIDTH SH_CSS_MAX_BQ_GRID_WIDTH
/* Video mode specific DVS define */
/* The video binary supports a delay of 1 or 2 frames */
-#define VIDEO_FRAME_DELAY 2
+#define MAX_DVS_FRAME_DELAY 2
/* +1 because DVS reads the previous and writes the current frame concurrently */
-#define MAX_NUM_VIDEO_DELAY_FRAMES (VIDEO_FRAME_DELAY + 1)
-
-/* Preview mode specific DVS define. */
-/* In preview we only need GDC functionality (and not the DVS functionality) */
-/* The minimum number of DVS frames you need is 2, one were GDC reads from and another where GDC writes into */
-#define NUM_PREVIEW_DVS_FRAMES (2)
-
-/* TNR is no longer exclusive to video, SkyCam preview has TNR too (same kernel as video).
- * All uses the generic define NUM_TNR_FRAMES. The define NUM_VIDEO_TNR_FRAMES has been deprecated.
- *
- * Notes
- * 1) The value depends on the used TNR kernel and is not something that depends on the mode
- * and it is not something you just could choice.
- * 2) For the luma only pipeline a version that supports two different sets of TNR reference frames
- * is being used.
- *.
- */
-#define NUM_VALID_TNR_REF_FRAMES (1) /* At least one valid TNR reference frame is required */
-#define NUM_TNR_FRAMES_PER_REF_BUF_SET (2)
-/* In luma-only mode alternate illuminated frames are supported, that requires two double buffers */
-#define NUM_TNR_REF_BUF_SETS (1)
-
-#define NUM_TNR_FRAMES (NUM_TNR_FRAMES_PER_REF_BUF_SET * NUM_TNR_REF_BUF_SETS)
+#define MAX_NUM_VIDEO_DELAY_FRAMES (MAX_DVS_FRAME_DELAY + 1)
#define NUM_VIDEO_TNR_FRAMES 2
@@ -250,11 +210,11 @@ RGB[0,8191],coef[-8192,8191] -> RGB[0,8191]
CEIL_MUL(_ISP_MORPH_TABLE_WIDTH(width), \
SH_CSS_MORPH_TABLE_ELEMS_PER_DDR_WORD)
-#define _ISP2400_SCTBL_WIDTH_PER_COLOR(input_width, deci_factor_log2) \
+#define _ISP_SCTBL_WIDTH_PER_COLOR(input_width, deci_factor_log2) \
(ISP_BQ_GRID_WIDTH(input_width, deci_factor_log2) + 1)
-#define _ISP2400_SCTBL_HEIGHT(input_height, deci_factor_log2) \
+#define _ISP_SCTBL_HEIGHT(input_height, deci_factor_log2) \
(ISP_BQ_GRID_HEIGHT(input_height, deci_factor_log2) + 1)
-#define _ISP2400_SCTBL_ALIGNED_WIDTH_PER_COLOR(input_width, deci_factor_log2) \
+#define _ISP_SCTBL_ALIGNED_WIDTH_PER_COLOR(input_width, deci_factor_log2) \
CEIL_MUL(_ISP_SCTBL_WIDTH_PER_COLOR(input_width, deci_factor_log2), \
ISP_VEC_NELEMS)
diff --git a/drivers/staging/media/atomisp/pci/sh_css_firmware.c b/drivers/staging/media/atomisp/pci/sh_css_firmware.c
index e1a16a5..9414964 100644
--- a/drivers/staging/media/atomisp/pci/sh_css_firmware.c
+++ b/drivers/staging/media/atomisp/pci/sh_css_firmware.c
@@ -56,8 +56,7 @@ static struct firmware_header *firmware_header;
* which will be replaced with the actual RELEASE_VERSION
* during package generation. Please do not modify
*/
-static const char *isp2400_release_version = STR(irci_stable_candrpv_0415_20150521_0458);
-static const char *isp2401_release_version = STR(irci_ecr - master_20150911_0724);
+static const char *release_version = STR(irci_stable_candrpv_0415_20150521_0458);
#define MAX_FW_REL_VER_NAME 300
static char FW_rel_ver_name[MAX_FW_REL_VER_NAME] = "---";
@@ -190,13 +189,6 @@ sh_css_check_firmware_version(struct device *dev, const char *fw_data)
{
struct sh_css_fw_bi_file_h *file_header;
- const char *release_version;
-
- if (!IS_ISP2401)
- release_version = isp2400_release_version;
- else
- release_version = isp2401_release_version;
-
firmware_header = (struct firmware_header *)fw_data;
file_header = &firmware_header->file_header;
@@ -232,12 +224,6 @@ sh_css_load_firmware(struct device *dev, const char *fw_data,
struct ia_css_fw_info *binaries;
struct sh_css_fw_bi_file_h *file_header;
int ret;
- const char *release_version;
-
- if (!IS_ISP2401)
- release_version = isp2400_release_version;
- else
- release_version = isp2401_release_version;
firmware_header = (struct firmware_header *)fw_data;
file_header = &firmware_header->file_header;
diff --git a/drivers/staging/media/atomisp/pci/sh_css_firmware.h b/drivers/staging/media/atomisp/pci/sh_css_firmware.h
index 66cd38f..a73ce70 100644
--- a/drivers/staging/media/atomisp/pci/sh_css_firmware.h
+++ b/drivers/staging/media/atomisp/pci/sh_css_firmware.h
@@ -29,9 +29,6 @@ struct sh_css_fw_bi_file_h {
};
extern struct ia_css_fw_info sh_css_sp_fw;
-#if defined(HAS_BL)
-extern struct ia_css_fw_info sh_css_bl_fw;
-#endif /* HAS_BL */
extern struct ia_css_blob_descr *sh_css_blob_info;
extern unsigned int sh_css_num_binaries;
diff --git a/drivers/staging/media/atomisp/pci/sh_css_internal.h b/drivers/staging/media/atomisp/pci/sh_css_internal.h
index 496faa7..435b3ce 100644
--- a/drivers/staging/media/atomisp/pci/sh_css_internal.h
+++ b/drivers/staging/media/atomisp/pci/sh_css_internal.h
@@ -86,12 +86,6 @@
#define SH_CSS_MAX_IF_CONFIGS 3 /* Must match with IA_CSS_NR_OF_CONFIGS (not defined yet).*/
#define SH_CSS_IF_CONFIG_NOT_NEEDED 0xFF
-#define SH_CSS_ENABLE_METADATA
-
-#if defined(SH_CSS_ENABLE_METADATA) && !defined(ISP2401)
-#define SH_CSS_ENABLE_METADATA_THREAD
-#endif
-
/*
* SH_CSS_MAX_SP_THREADS:
* sp threads visible to host with connected communication queues
@@ -101,7 +95,7 @@
* these threads can't be used as image pipe
*/
-#if defined(SH_CSS_ENABLE_METADATA_THREAD)
+#if !defined(ISP2401)
#define SH_CSS_SP_INTERNAL_METADATA_THREAD 1
#else
#define SH_CSS_SP_INTERNAL_METADATA_THREAD 0
@@ -276,7 +270,7 @@ struct sh_css_binary_args {
struct ia_css_frame *in_frame; /* input frame */
const struct ia_css_frame
*delay_frames[MAX_NUM_VIDEO_DELAY_FRAMES]; /* reference input frame */
- const struct ia_css_frame *tnr_frames[NUM_TNR_FRAMES]; /* tnr frames */
+ const struct ia_css_frame *tnr_frames[NUM_VIDEO_TNR_FRAMES]; /* tnr frames */
struct ia_css_frame
*out_frame[IA_CSS_BINARY_MAX_OUTPUT_PORTS]; /* output frame */
struct ia_css_frame *out_vf_frame; /* viewfinder output frame */
@@ -526,7 +520,6 @@ struct sh_css_sp_pipeline {
this struct; needs cleanup */
s32 num_execs; /* number of times to run if this is
an acceleration pipe. */
-#if defined(SH_CSS_ENABLE_METADATA)
struct {
u32 format; /* Metadata format in hrt format */
u32 width; /* Width of a line */
@@ -535,10 +528,7 @@ struct sh_css_sp_pipeline {
u32 size; /* Total size (in bytes) */
ia_css_ptr cont_buf; /* Address of continuous buffer */
} metadata;
-#endif
-#if defined(SH_CSS_ENABLE_PER_FRAME_PARAMS)
u32 output_frame_queue_id;
-#endif
union {
struct {
u32 bytes_available;
@@ -551,14 +541,6 @@ struct sh_css_sp_pipeline {
u32 raw_bit_depth;
} raw;
} copy;
-
-/* ISP2401 */
-
- /* Parameters passed to Shading Correction kernel. */
- struct {
- u32 internal_frame_origin_x_bqs_on_sctbl; /* Origin X (bqs) of internal frame on shading table */
- u32 internal_frame_origin_y_bqs_on_sctbl; /* Origin Y (bqs) of internal frame on shading table */
- } shading;
};
/*
@@ -580,9 +562,7 @@ struct ia_css_frames_sp {
struct ia_css_frame_sp_info internal_frame_info;
struct ia_css_buffer_sp s3a_buf;
struct ia_css_buffer_sp dvs_buf;
-#if defined SH_CSS_ENABLE_METADATA
struct ia_css_buffer_sp metadata_buf;
-#endif
};
/* Information for a single pipeline stage for an ISP */
@@ -695,8 +675,6 @@ struct sh_css_sp_output {
unsigned int sw_interrupt_value[SH_CSS_NUM_SDW_IRQS];
};
-#define CONFIG_ON_FRAME_ENQUEUE() 0
-
/**
* @brief Data structure for the circular buffer.
* The circular buffer is empty if "start == end". The
@@ -734,9 +712,6 @@ struct sh_css_hmm_buffer {
u32 exp_id;
u32 isp_parameters_id; /** Unique ID to track which config was
actually applied to a particular frame */
-#if CONFIG_ON_FRAME_ENQUEUE()
- struct sh_css_config_on_frame_enqueue config_on_frame_enqueue;
-#endif
} frame;
ia_css_ptr ddr_ptrs;
} payload;
@@ -752,16 +727,9 @@ struct sh_css_hmm_buffer {
clock_value_t isys_eof_clock_tick;
};
-#if CONFIG_ON_FRAME_ENQUEUE()
-#define SIZE_OF_FRAME_STRUCT \
- (SIZE_OF_HRT_VADDRESS + \
- (3 * sizeof(uint32_t)) + \
- sizeof(uint32_t))
-#else
#define SIZE_OF_FRAME_STRUCT \
(SIZE_OF_HRT_VADDRESS + \
(3 * sizeof(uint32_t)))
-#endif
#define SIZE_OF_PAYLOAD_UNION \
(MAX(MAX(MAX(MAX( \
diff --git a/drivers/staging/media/atomisp/pci/sh_css_metadata.c b/drivers/staging/media/atomisp/pci/sh_css_metadata.c
deleted file mode 100644
index 04a4b7d..0000000
--- a/drivers/staging/media/atomisp/pci/sh_css_metadata.c
+++ /dev/null
@@ -1,17 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/*
- * Support for Intel Camera Imaging ISP subsystem.
- * Copyright (c) 2015, Intel Corporation.
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms and conditions of the GNU General Public License,
- * version 2, as published by the Free Software Foundation.
- *
- * This program is distributed in the hope it will be useful, but WITHOUT
- * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
- * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
- * more details.
- */
-
-/* This file will contain the code to implement the functions declared in ia_css_metadata.h
- and associated helper functions */
diff --git a/drivers/staging/media/atomisp/pci/sh_css_metrics.c b/drivers/staging/media/atomisp/pci/sh_css_metrics.c
index 9744bbe..8ded6cd 100644
--- a/drivers/staging/media/atomisp/pci/sh_css_metrics.c
+++ b/drivers/staging/media/atomisp/pci/sh_css_metrics.c
@@ -123,40 +123,15 @@ sh_css_metrics_sample_pcs(void)
unsigned int pc;
unsigned int msink;
-#if SUSPEND
- unsigned int sc = 0;
- unsigned int stopped_sc = 0;
- unsigned int resume_sc = 0;
-#endif
-#if MULTIPLE_PCS
- int i;
- unsigned int pc_tab[NOF_PCS];
-
- for (i = 0; i < NOF_PCS; i++)
- pc_tab[i] = 0;
-#endif
if (!pc_histogram_enabled)
return;
if (isp_histogram) {
-#if SUSPEND
- /* STOP the ISP */
- isp_ctrl_store(ISP0_ID, ISP_SC_REG, STOP_MASK);
-#endif
msink = isp_ctrl_load(ISP0_ID, ISP_CTRL_SINK_REG);
-#if MULTIPLE_PCS
- for (i = 0; i < NOF_PCS; i++)
- pc_tab[i] = isp_ctrl_load(ISP0_ID, ISP_PC_REG);
-#else
pc = isp_ctrl_load(ISP0_ID, ISP_PC_REG);
-#endif
-#if SUSPEND
- /* RESUME the ISP */
- isp_ctrl_store(ISP0_ID, ISP_SC_REG, RESUME_MASK);
-#endif
isp_histogram->msink[pc] &= msink;
stall = (msink != 0x7FF);
diff --git a/drivers/staging/media/atomisp/pci/sh_css_mipi.c b/drivers/staging/media/atomisp/pci/sh_css_mipi.c
index 75489f7..0acf754 100644
--- a/drivers/staging/media/atomisp/pci/sh_css_mipi.c
+++ b/drivers/staging/media/atomisp/pci/sh_css_mipi.c
@@ -33,47 +33,6 @@
static u32
ref_count_mipi_allocation[N_CSI_PORTS]; /* Initialized in mipi_init */
-/*
- * Check if a source port or TPG/PRBS ID is valid
- */
-static bool ia_css_mipi_is_source_port_valid(struct ia_css_pipe *pipe,
- unsigned int *pport)
-{
- bool ret = true;
- unsigned int port = 0;
- unsigned int max_ports = 0;
-
- switch (pipe->stream->config.mode) {
- case IA_CSS_INPUT_MODE_BUFFERED_SENSOR:
- port = (unsigned int)pipe->stream->config.source.port.port;
- max_ports = N_CSI_PORTS;
- break;
- case IA_CSS_INPUT_MODE_TPG:
- port = (unsigned int)pipe->stream->config.source.tpg.id;
- max_ports = N_CSS_TPG_IDS;
- break;
- case IA_CSS_INPUT_MODE_PRBS:
- port = (unsigned int)pipe->stream->config.source.prbs.id;
- max_ports = N_CSS_PRBS_IDS;
- break;
- default:
- assert(false);
- ret = false;
- break;
- }
-
- if (ret) {
- assert(port < max_ports);
-
- if (port >= max_ports)
- ret = false;
- }
-
- *pport = port;
-
- return ret;
-}
-
/* Assumptions:
* - A line is multiple of 4 bytes = 1 word.
* - Each frame has SOF and EOF (each 1 word).
@@ -133,15 +92,11 @@ ia_css_mipi_frame_calculate_size(const unsigned int width,
break;
case ATOMISP_INPUT_FORMAT_YUV420_10: /* odd 4p, 5B, 40bits, even 4p, 10B, 80bits */
case ATOMISP_INPUT_FORMAT_RAW_10: /* 4p, 5B, 40bits */
-#if !defined(HAS_NO_PACKED_RAW_PIXELS)
/* The changes will be reverted as soon as RAW
* Buffers are deployed by the 2401 Input System
* in the non-continuous use scenario.
*/
bits_per_pixel = 10;
-#else
- bits_per_pixel = 16;
-#endif
break;
case ATOMISP_INPUT_FORMAT_YUV420_8_LEGACY: /* 2p, 3B, 24bits */
case ATOMISP_INPUT_FORMAT_RAW_12: /* 2p, 3B, 24bits */
@@ -231,6 +186,10 @@ ia_css_mipi_frame_calculate_size(const unsigned int width,
return err;
}
+/*
+ * Check if a source port or TPG/PRBS ID is valid
+ */
+
#if !defined(ISP2401)
int
ia_css_mipi_frame_enable_check_on_size(const enum mipi_port_id port,
@@ -265,16 +224,31 @@ mipi_init(void)
ref_count_mipi_allocation[i] = 0;
}
-int
-calculate_mipi_buff_size(
- struct ia_css_stream_config *stream_cfg,
- unsigned int *size_mem_words)
+bool mipi_is_free(void)
{
-#if !defined(ISP2401)
- int err = -EINVAL;
- (void)stream_cfg;
- (void)size_mem_words;
-#else
+ unsigned int i;
+
+ for (i = 0; i < N_CSI_PORTS; i++)
+ if (ref_count_mipi_allocation[i])
+ return false;
+
+ return true;
+}
+
+#if defined(ISP2401)
+/*
+ * @brief Calculate the required MIPI buffer sizes.
+ * Based on the stream configuration, calculate the
+ * required MIPI buffer sizes (in DDR words).
+ *
+ * @param[in] stream_cfg Point to the target stream configuration
+ * @param[out] size_mem_words MIPI buffer size in DDR words.
+ *
+ * @return
+ */
+static int calculate_mipi_buff_size(struct ia_css_stream_config *stream_cfg,
+ unsigned int *size_mem_words)
+{
unsigned int width;
unsigned int height;
enum atomisp_input_format format;
@@ -366,26 +340,9 @@ calculate_mipi_buff_size(
*size_mem_words = mem_words_per_buff;
IA_CSS_LEAVE_ERR(err);
-#endif
return err;
}
-
-static bool buffers_needed(struct ia_css_pipe *pipe)
-{
- if (!IS_ISP2401) {
- if (pipe->stream->config.mode == IA_CSS_INPUT_MODE_BUFFERED_SENSOR)
- return false;
- else
- return true;
- }
-
- if (pipe->stream->config.mode == IA_CSS_INPUT_MODE_BUFFERED_SENSOR ||
- pipe->stream->config.mode == IA_CSS_INPUT_MODE_TPG ||
- pipe->stream->config.mode == IA_CSS_INPUT_MODE_PRBS)
- return false;
-
- return true;
-}
+#endif
int
allocate_mipi_frames(struct ia_css_pipe *pipe,
@@ -415,43 +372,25 @@ allocate_mipi_frames(struct ia_css_pipe *pipe,
}
#endif
-
- if (!buffers_needed(pipe)) {
+ if (pipe->stream->config.mode != IA_CSS_INPUT_MODE_BUFFERED_SENSOR) {
ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
"allocate_mipi_frames(%p) exit: no buffers needed for pipe mode.\n",
pipe);
return 0; /* AM TODO: Check */
}
- if (!IS_ISP2401)
- port = (unsigned int)pipe->stream->config.source.port.port;
- else
- err = ia_css_mipi_is_source_port_valid(pipe, &port);
-
- assert(port < N_CSI_PORTS);
-
- if (port >= N_CSI_PORTS || err) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "allocate_mipi_frames(%p) exit: error: port is not correct (port=%d).\n",
- pipe, port);
+ port = (unsigned int)pipe->stream->config.source.port.port;
+ if (port >= N_CSI_PORTS) {
+ IA_CSS_ERROR("allocate_mipi_frames(%p) exit: port is not correct (port=%d).",
+ pipe, port);
return -EINVAL;
}
#ifdef ISP2401
- err = calculate_mipi_buff_size(
- &pipe->stream->config,
- &my_css.mipi_frame_size[port]);
-#endif
-
-#if !defined(ISP2401)
- if (ref_count_mipi_allocation[port] != 0) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "allocate_mipi_frames(%p) exit: already allocated for this port (port=%d).\n",
- pipe, port);
- return 0;
- }
-#else
- /* 2401 system allows multiple streams to use same physical port. This is not
+ err = calculate_mipi_buff_size(&pipe->stream->config,
+ &my_css.mipi_frame_size[port]);
+ /*
+ * 2401 system allows multiple streams to use same physical port. This is not
* true for 2400 system. Currently 2401 uses MIPI buffers as a temporary solution.
* TODO AM: Once that is changed (removed) this code should be removed as well.
* In that case only 2400 related code should remain.
@@ -463,6 +402,13 @@ allocate_mipi_frames(struct ia_css_pipe *pipe,
pipe, port);
return 0;
}
+#else
+ if (ref_count_mipi_allocation[port] != 0) {
+ ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
+ "allocate_mipi_frames(%p) exit: already allocated for this port (port=%d).\n",
+ pipe, port);
+ return 0;
+ }
#endif
ref_count_mipi_allocation[port]++;
@@ -494,9 +440,8 @@ allocate_mipi_frames(struct ia_css_pipe *pipe,
my_css.mipi_frames[port][j] = NULL;
}
}
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "allocate_mipi_frames(%p, %d) exit: error: allocation failed.\n",
- pipe, port);
+ IA_CSS_ERROR("allocate_mipi_frames(%p, %d) exit: allocation failed.",
+ pipe, port);
return err;
}
}
@@ -539,30 +484,22 @@ free_mipi_frames(struct ia_css_pipe *pipe)
if (pipe) {
assert(pipe->stream);
if ((!pipe) || (!pipe->stream)) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "free_mipi_frames(%p) exit: error: pipe or stream is null.\n",
- pipe);
+ IA_CSS_ERROR("free_mipi_frames(%p) exit: pipe or stream is null.",
+ pipe);
return -EINVAL;
}
- if (!buffers_needed(pipe)) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "free_mipi_frames(%p) exit: error: wrong mode.\n",
- pipe);
+ if (pipe->stream->config.mode != IA_CSS_INPUT_MODE_BUFFERED_SENSOR) {
+ IA_CSS_ERROR("free_mipi_frames(%p) exit: wrong mode.",
+ pipe);
return err;
}
- if (!IS_ISP2401)
- port = (unsigned int)pipe->stream->config.source.port.port;
- else
- err = ia_css_mipi_is_source_port_valid(pipe, &port);
+ port = (unsigned int)pipe->stream->config.source.port.port;
- assert(port < N_CSI_PORTS);
-
- if (port >= N_CSI_PORTS || err) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "free_mipi_frames(%p, %d) exit: error: pipe port is not correct.\n",
- pipe, port);
+ if (port >= N_CSI_PORTS) {
+ IA_CSS_ERROR("free_mipi_frames(%p, %d) exit: pipe port is not correct.",
+ pipe, port);
return err;
}
@@ -570,9 +507,8 @@ free_mipi_frames(struct ia_css_pipe *pipe)
#if !defined(ISP2401)
assert(ref_count_mipi_allocation[port] == 1);
if (ref_count_mipi_allocation[port] != 1) {
- ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE_PRIVATE,
- "free_mipi_frames(%p) exit: error: wrong ref_count (ref_count=%d).\n",
- pipe, ref_count_mipi_allocation[port]);
+ IA_CSS_ERROR("free_mipi_frames(%p) exit: wrong ref_count (ref_count=%d).",
+ pipe, ref_count_mipi_allocation[port]);
return err;
}
#endif
@@ -640,11 +576,7 @@ send_mipi_frames(struct ia_css_pipe *pipe)
{
int err = -EINVAL;
unsigned int i;
-#ifndef ISP2401
unsigned int port;
-#else
- unsigned int port = 0;
-#endif
IA_CSS_ENTER_PRIVATE("pipe=%p", pipe);
@@ -657,21 +589,16 @@ send_mipi_frames(struct ia_css_pipe *pipe)
/* multi stream video needs mipi buffers */
/* nothing to be done in other cases. */
- if (!buffers_needed(pipe)) {
+ if (pipe->stream->config.mode != IA_CSS_INPUT_MODE_BUFFERED_SENSOR) {
IA_CSS_LOG("nothing to be done for this mode");
return 0;
/* TODO: AM: maybe this should be returning an error. */
}
- if (!IS_ISP2401)
- port = (unsigned int)pipe->stream->config.source.port.port;
- else
- err = ia_css_mipi_is_source_port_valid(pipe, &port);
+ port = (unsigned int)pipe->stream->config.source.port.port;
- assert(port < N_CSI_PORTS);
-
- if (port >= N_CSI_PORTS || err) {
- IA_CSS_ERROR("send_mipi_frames(%p) exit: invalid port specified (port=%d).\n",
+ if (port >= N_CSI_PORTS) {
+ IA_CSS_ERROR("send_mipi_frames(%p) exit: invalid port specified (port=%d).",
pipe, port);
return err;
}
diff --git a/drivers/staging/media/atomisp/pci/sh_css_mipi.h b/drivers/staging/media/atomisp/pci/sh_css_mipi.h
index 52f08a1..e6c86d0 100644
--- a/drivers/staging/media/atomisp/pci/sh_css_mipi.h
+++ b/drivers/staging/media/atomisp/pci/sh_css_mipi.h
@@ -23,6 +23,8 @@
void
mipi_init(void);
+bool mipi_is_free(void);
+
int
allocate_mipi_frames(struct ia_css_pipe *pipe, struct ia_css_stream_info *info);
@@ -32,19 +34,4 @@ free_mipi_frames(struct ia_css_pipe *pipe);
int
send_mipi_frames(struct ia_css_pipe *pipe);
-/**
- * @brief Calculate the required MIPI buffer sizes.
- * Based on the stream configuration, calculate the
- * required MIPI buffer sizes (in DDR words).
- *
- * @param[in] stream_cfg Point to the target stream configuration
- * @param[out] size_mem_words MIPI buffer size in DDR words.
- *
- * @return
- */
-int
-calculate_mipi_buff_size(
- struct ia_css_stream_config *stream_cfg,
- unsigned int *size_mem_words);
-
#endif /* __SH_CSS_MIPI_H */
diff --git a/drivers/staging/media/atomisp/pci/sh_css_morph.c b/drivers/staging/media/atomisp/pci/sh_css_morph.c
deleted file mode 100644
index edd1da9..0000000
--- a/drivers/staging/media/atomisp/pci/sh_css_morph.c
+++ /dev/null
@@ -1,17 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/*
- * Support for Intel Camera Imaging ISP subsystem.
- * Copyright (c) 2015, Intel Corporation.
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms and conditions of the GNU General Public License,
- * version 2, as published by the Free Software Foundation.
- *
- * This program is distributed in the hope it will be useful, but WITHOUT
- * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
- * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
- * more details.
- */
-
-/* This file will contain the code to implement the functions declared in ia_css_morph.h
- and associated helper functions */
diff --git a/drivers/staging/media/atomisp/pci/sh_css_param_shading.c b/drivers/staging/media/atomisp/pci/sh_css_param_shading.c
index 69cc4e4..41a4c91 100644
--- a/drivers/staging/media/atomisp/pci/sh_css_param_shading.c
+++ b/drivers/staging/media/atomisp/pci/sh_css_param_shading.c
@@ -118,8 +118,10 @@ crop_and_interpolate(unsigned int cropped_width,
int ty, src_y0, src_y1;
unsigned int sy0, sy1, dy0, dy1, divy;
- /* calculate target point and make sure it falls within
- the table */
+ /*
+ * calculate target point and make sure it falls within
+ * the table
+ */
ty = out_start_row + i * out_cell_size;
/* calculate closest source points in shading table and
@@ -166,19 +168,24 @@ crop_and_interpolate(unsigned int cropped_width,
src_x1 = clamp(src_x1, 0, (int)table_width - 1);
tx = min(clamp(tx, 0, (int)sensor_width - 1),
(int)table_cell_w);
- /* calculate closest source points for distance
- computation */
+ /*
+ * calculate closest source points for distance
+ * computation
+ */
sx0 = min(src_x0 * in_cell_size, sensor_width - 1);
sx1 = min(src_x1 * in_cell_size, sensor_width - 1);
- /* calculate distances between source and target
- pixels */
+ /*
+ * calculate distances between source and target
+ * pixels
+ */
dx0 = tx - sx0;
dx1 = sx1 - tx;
divx = sx1 - sx0;
/* if we're at the edge, we just use the closest
- point still in the grid. We make up for the divider
- in this case by setting the distance to
- out_cell_size, since it's actually 0. */
+ * point still in the grid. We make up for the divider
+ * in this case by setting the distance to
+ * out_cell_size, since it's actually 0.
+ */
if (divx == 0) {
dx0 = 1;
divx = 1;
@@ -242,8 +249,8 @@ prepare_shading_table(const struct ia_css_shading_table *in_table,
if (!in_table) {
sh_css_params_shading_id_table_generate(target_table,
- binary->sctbl_legacy_width_per_color,
- binary->sctbl_legacy_height);
+ binary->sctbl_width_per_color,
+ binary->sctbl_height);
return;
}
@@ -271,43 +278,33 @@ prepare_shading_table(const struct ia_css_shading_table *in_table,
bds_denominator -
binary->info->sp.pipeline.top_cropping;
-#if !defined(USE_WINDOWS_BINNING_FACTOR)
- /* @deprecated{This part of the code will be replaced by the code
- * in the #else section below to make the calculation same across
- * all platforms.
- * Android and Windows platforms interpret the binning_factor parameter
- * differently. In Android, the binning factor is expressed in the form
- * 2^N * 2^N, whereas in Windows platform, the binning factor is N*N}
+ /*
+ * We take into account the binning done by the sensor. We do this
+ * by cropping the non-binned part of the shading table and then
+ * increasing the size of a grid cell with this same binning factor.
*/
-
- /* We take into account the binning done by the sensor. We do this
- by cropping the non-binned part of the shading table and then
- increasing the size of a grid cell with this same binning factor. */
input_width <<= sensor_binning;
input_height <<= sensor_binning;
- /* We also scale the padding by the same binning factor. This will
- make it much easier later on to calculate the padding of the
- shading table. */
+ /*
+ * We also scale the padding by the same binning factor. This will
+ * make it much easier later on to calculate the padding of the
+ * shading table.
+ */
left_padding <<= sensor_binning;
right_padding <<= sensor_binning;
top_padding <<= sensor_binning;
-#else
- input_width *= sensor_binning;
- input_height *= sensor_binning;
- left_padding *= sensor_binning;
- right_padding *= sensor_binning;
- top_padding *= sensor_binning;
-#endif /*USE_WINDOWS_BINNING_FACTOR*/
- /* during simulation, the used resolution can exceed the sensor
- resolution, so we clip it. */
+ /*
+ * during simulation, the used resolution can exceed the sensor
+ * resolution, so we clip it.
+ */
input_width = min(input_width, in_table->sensor_width);
input_height = min(input_height, in_table->sensor_height);
/* This prepare_shading_table() function is called only in legacy API (not in new API).
Then, the legacy shading table width and height should be used. */
- table_width = binary->sctbl_legacy_width_per_color;
- table_height = binary->sctbl_legacy_height;
+ table_width = binary->sctbl_width_per_color;
+ table_height = binary->sctbl_height;
result = ia_css_shading_table_alloc(table_width, table_height);
if (!result) {
@@ -318,8 +315,10 @@ prepare_shading_table(const struct ia_css_shading_table *in_table,
result->sensor_height = in_table->sensor_height;
result->fraction_bits = in_table->fraction_bits;
- /* now we crop the original shading table and then interpolate to the
- requested resolution and decimation factor. */
+ /*
+ * now we crop the original shading table and then interpolate to the
+ * requested resolution and decimation factor.
+ */
for (i = 0; i < IA_CSS_SC_NUM_COLORS; i++) {
crop_and_interpolate(input_width, input_height,
left_padding, right_padding, top_padding,
@@ -376,9 +375,10 @@ ia_css_shading_table_free(struct ia_css_shading_table *table)
if (!table)
return;
- /* We only output logging when the table is not NULL, otherwise
+ /*
+ * We only output logging when the table is not NULL, otherwise
* logs will give the impression that a table was freed.
- * */
+ */
IA_CSS_ENTER("");
for (i = 0; i < IA_CSS_SC_NUM_COLORS; i++) {
diff --git a/drivers/staging/media/atomisp/pci/sh_css_params.c b/drivers/staging/media/atomisp/pci/sh_css_params.c
index dbd3bfe..09f87c2 100644
--- a/drivers/staging/media/atomisp/pci/sh_css_params.c
+++ b/drivers/staging/media/atomisp/pci/sh_css_params.c
@@ -16,12 +16,10 @@
#include "gdc_device.h" /* gdc_lut_store(), ... */
#include "isp.h" /* ISP_VEC_ELEMBITS */
#include "vamem.h"
-#if !defined(HAS_NO_HMEM)
#ifndef __INLINE_HMEM__
#define __INLINE_HMEM__
#endif
#include "hmem.h"
-#endif /* !defined(HAS_NO_HMEM) */
#define IA_CSS_INCLUDE_PARAMETERS
#define IA_CSS_INCLUDE_ACC_PARAMETERS
@@ -96,9 +94,6 @@
#include "xnr/xnr_3.0/ia_css_xnr3.host.h"
-#if defined(HAS_OUTPUT_SYSTEM)
-#include <components/output_system/sc_output_system_1.0/host/output_system.host.h>
-#endif
#include "sh_css_frac.h"
#include "ia_css_bufq.h"
@@ -107,15 +102,10 @@
(sizeof(char) * (binary)->in_frame_info.res.height * \
(binary)->in_frame_info.padded_width)
-#define ISP2400_SCTBL_BYTES(binary) \
+#define SCTBL_BYTES(binary) \
(sizeof(unsigned short) * (binary)->sctbl_height * \
(binary)->sctbl_aligned_width_per_color * IA_CSS_SC_NUM_COLORS)
-#define ISP2401_SCTBL_BYTES(binary) \
- (sizeof(unsigned short) * max((binary)->sctbl_height, (binary)->sctbl_legacy_height) * \
- /* height should be the larger height between new api and legacy api */ \
- (binary)->sctbl_aligned_width_per_color * IA_CSS_SC_NUM_COLORS)
-
#define MORPH_PLANE_BYTES(binary) \
(SH_CSS_MORPH_TABLE_ELEM_BYTES * (binary)->morph_tbl_aligned_width * \
(binary)->morph_tbl_height)
@@ -734,13 +724,11 @@ sh_css_set_global_isp_config_on_pipe(
const struct ia_css_isp_config *config,
struct ia_css_pipe *pipe);
-#if defined(SH_CSS_ENABLE_PER_FRAME_PARAMS)
static int
sh_css_set_per_frame_isp_config_on_pipe(
struct ia_css_stream *stream,
const struct ia_css_isp_config *config,
struct ia_css_pipe *pipe);
-#endif
static int
sh_css_update_uds_and_crop_info_based_on_zoom_region(
@@ -1031,16 +1019,6 @@ sh_css_params_set_binning_factor(struct ia_css_stream *stream,
}
static void
-sh_css_update_shading_table_status(struct ia_css_pipe *pipe,
- struct ia_css_isp_parameters *params)
-{
- if (params && pipe && (pipe->pipe_num != params->sc_table_last_pipe_num)) {
- params->sc_table_dirty = true;
- params->sc_table_last_pipe_num = pipe->pipe_num;
- }
-}
-
-static void
sh_css_set_shading_table(struct ia_css_stream *stream,
struct ia_css_isp_parameters *params,
const struct ia_css_shading_table *table)
@@ -1053,10 +1031,9 @@ sh_css_set_shading_table(struct ia_css_stream *stream,
if (!table->enable)
table = NULL;
- if ((table != params->sc_table) || params->sc_table_dirty) {
+ if (table != params->sc_table) {
params->sc_table = table;
params->sc_table_changed = true;
- params->sc_table_dirty = false;
/* Not very clean, this goes to sh_css.c to invalidate the
* shading table for all pipes. Should replaced by a loop
* and a pipe-specific call.
@@ -1510,10 +1487,8 @@ ia_css_translate_3a_statistics(
ia_css_s3a_vmem_decode(host_stats, isp_stats->vmem_stats_hi,
isp_stats->vmem_stats_lo);
}
-#if !defined(HAS_NO_HMEM)
IA_CSS_LOG("3A: HMEM");
ia_css_s3a_hmem_decode(host_stats, isp_stats->hmem_stats);
-#endif
IA_CSS_LEAVE("void");
}
@@ -1616,57 +1591,6 @@ ia_css_set_param_exceptions(const struct ia_css_pipe *pipe,
params->dp_config.r = params->wb_config.r;
params->dp_config.b = params->wb_config.b;
params->dp_config.gb = params->wb_config.gb;
-
- if (IS_ISP2401) {
- assert(pipe);
- assert(pipe->mode < IA_CSS_PIPE_ID_NUM);
-
- if (pipe->mode < IA_CSS_PIPE_ID_NUM) {
- params->pipe_dp_config[pipe->mode].gr = params->wb_config.gr;
- params->pipe_dp_config[pipe->mode].r = params->wb_config.r;
- params->pipe_dp_config[pipe->mode].b = params->wb_config.b;
- params->pipe_dp_config[pipe->mode].gb = params->wb_config.gb;
- }
- }
-}
-
-/* ISP2401 */
-static void
-sh_css_set_dp_config(const struct ia_css_pipe *pipe,
- struct ia_css_isp_parameters *params,
- const struct ia_css_dp_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- assert(pipe);
- assert(pipe->mode < IA_CSS_PIPE_ID_NUM);
-
- IA_CSS_ENTER_PRIVATE("config=%p", config);
- ia_css_dp_debug_dtrace(config, IA_CSS_DEBUG_TRACE_PRIVATE);
- if (pipe->mode < IA_CSS_PIPE_ID_NUM) {
- params->pipe_dp_config[pipe->mode] = *config;
- params->pipe_dpc_config_changed[pipe->mode] = true;
- }
- IA_CSS_LEAVE_PRIVATE("void");
-}
-
-static void
-sh_css_get_dp_config(const struct ia_css_pipe *pipe,
- const struct ia_css_isp_parameters *params,
- struct ia_css_dp_config *config)
-{
- if (!config)
- return;
-
- assert(params);
- assert(pipe);
- IA_CSS_ENTER_PRIVATE("config=%p", config);
-
- *config = params->pipe_dp_config[pipe->mode];
-
- IA_CSS_LEAVE_PRIVATE("void");
}
static void
@@ -1740,9 +1664,7 @@ sh_css_set_pipe_dvs_6axis_config(const struct ia_css_pipe *pipe,
copy_dvs_6axis_table(params->pipe_dvs_6axis_config[pipe->mode], dvs_config);
-#if !defined(HAS_NO_DVS_6AXIS_CONFIG_UPDATE)
params->pipe_dvs_6axis_config_changed[pipe->mode] = true;
-#endif
IA_CSS_LEAVE_PRIVATE("void");
}
@@ -1908,11 +1830,9 @@ ia_css_stream_set_isp_config_on_pipe(
IA_CSS_ENTER("stream=%p, config=%p, pipe=%p", stream, config, pipe);
-#if defined(SH_CSS_ENABLE_PER_FRAME_PARAMS)
if (config->output_frame)
err = sh_css_set_per_frame_isp_config_on_pipe(stream, config, pipe);
else
-#endif
err = sh_css_set_global_isp_config_on_pipe(stream->pipes[0], config, pipe);
IA_CSS_LEAVE_ERR(err);
@@ -1933,11 +1853,9 @@ ia_css_pipe_set_isp_config(struct ia_css_pipe *pipe,
ia_css_debug_dtrace(IA_CSS_DEBUG_TRACE, "config=%p\n", config);
-#if defined(SH_CSS_ENABLE_PER_FRAME_PARAMS)
if (config->output_frame)
err = sh_css_set_per_frame_isp_config_on_pipe(pipe->stream, config, pipe);
else
-#endif
err = sh_css_set_global_isp_config_on_pipe(pipe, config, pipe_in);
IA_CSS_LEAVE_ERR(err);
return err;
@@ -1972,7 +1890,6 @@ sh_css_set_global_isp_config_on_pipe(
return err;
}
-#if defined(SH_CSS_ENABLE_PER_FRAME_PARAMS)
static int
sh_css_set_per_frame_isp_config_on_pipe(
struct ia_css_stream *stream,
@@ -2042,7 +1959,6 @@ sh_css_set_per_frame_isp_config_on_pipe(
IA_CSS_LEAVE_ERR_PRIVATE(err);
return err;
}
-#endif
static int
sh_css_init_isp_params_from_config(struct ia_css_pipe *pipe,
@@ -2067,7 +1983,6 @@ sh_css_init_isp_params_from_config(struct ia_css_pipe *pipe,
sh_css_set_pipe_dvs_6axis_config(pipe, params, config->dvs_6axis_config);
sh_css_set_dz_config(params, config->dz_config);
sh_css_set_motion_vector(params, config->motion_vector);
- sh_css_update_shading_table_status(pipe_in, params);
sh_css_set_shading_table(pipe->stream, params, config->shading_table);
sh_css_set_morph_table(params, config->morph_table);
sh_css_set_macc_table(params, config->macc_table);
@@ -2083,16 +1998,6 @@ sh_css_init_isp_params_from_config(struct ia_css_pipe *pipe,
params->output_frame = config->output_frame;
params->isp_parameters_id = config->isp_config_id;
- /* Currently we do not offer CSS interface to set different
- * configurations for DPC, i.e. depending on DPC being enabled
- * before (NORM+OBC) or after. The folllowing code to set the
- * DPC configuration should be updated when this interface is made
- * available */
- if (IS_ISP2401) {
- sh_css_set_dp_config(pipe, params, config->dp_config);
- ia_css_set_param_exceptions(pipe, params);
- }
-
if (0 ==
sh_css_select_dp_10bpp_config(pipe, &is_dp_10bpp)) {
/* return an error when both DPC and BDS is enabled by the
@@ -2107,8 +2012,7 @@ sh_css_init_isp_params_from_config(struct ia_css_pipe *pipe,
goto exit;
}
- if (!IS_ISP2401)
- ia_css_set_param_exceptions(pipe, params);
+ ia_css_set_param_exceptions(pipe, params);
exit:
IA_CSS_LEAVE_ERR_PRIVATE(err);
@@ -2143,7 +2047,6 @@ ia_css_pipe_get_isp_config(struct ia_css_pipe *pipe,
sh_css_get_ee_config(params, config->ee_config);
sh_css_get_baa_config(params, config->baa_config);
sh_css_get_pipe_dvs_6axis_config(pipe, params, config->dvs_6axis_config);
- sh_css_get_dp_config(pipe, params, config->dp_config);
sh_css_get_macc_table(params, config->macc_table);
sh_css_get_gamma_table(params, config->gamma_table);
sh_css_get_ctc_table(params, config->ctc_table);
@@ -2250,9 +2153,7 @@ ia_css_isp_3a_statistics_allocate(const struct ia_css_3a_grid_info *grid)
me->vmem_size = ISP_S3ATBL_HI_LO_STRIDE_BYTES *
grid->aligned_height;
}
-#if !defined(HAS_NO_HMEM)
me->hmem_size = sizeof_hmem(HMEM0_ID);
-#endif
/* All subsections need to be aligned to the system bus width */
me->dmem_size = CEIL_MUL(me->dmem_size, HIVE_ISP_DDR_WORD_BYTES);
@@ -2431,7 +2332,7 @@ sh_css_create_isp_params(struct ia_css_stream *stream,
unsigned int i;
struct sh_css_ddr_address_map *ddr_ptrs;
struct sh_css_ddr_address_map_size *ddr_ptrs_size;
- int err = 0;
+ int err;
size_t params_size;
struct ia_css_isp_parameters *params =
kvmalloc(sizeof(struct ia_css_isp_parameters), GFP_KERNEL);
@@ -2473,7 +2374,11 @@ sh_css_create_isp_params(struct ia_css_stream *stream,
succ &= (ddr_ptrs->macc_tbl != mmgr_NULL);
*isp_params_out = params;
- return err;
+
+ if (!succ)
+ return -ENOMEM;
+
+ return 0;
}
static bool
@@ -2521,29 +2426,7 @@ sh_css_init_isp_params_from_global(struct ia_css_stream *stream,
ia_css_set_ob_config(params, &default_ob_config);
ia_css_set_dp_config(params, &default_dp_config);
- if (!IS_ISP2401) {
- ia_css_set_param_exceptions(pipe_in, params);
- } else {
- for (i = 0; i < stream->num_pipes; i++) {
- if (sh_css_select_dp_10bpp_config(stream->pipes[i],
- &is_dp_10bpp) == 0) {
- /* set the return value as false if both DPC and
- * BDS is enabled by the user. But we do not return
- * the value immediately to enable internal firmware
- * feature testing. */
- if (is_dp_10bpp) {
- sh_css_set_dp_config(stream->pipes[i], params, &default_dp_10bpp_config);
- } else {
- sh_css_set_dp_config(stream->pipes[i], params, &default_dp_config);
- }
- } else {
- retval = false;
- goto exit;
- }
-
- ia_css_set_param_exceptions(stream->pipes[i], params);
- }
- }
+ ia_css_set_param_exceptions(pipe_in, params);
ia_css_set_de_config(params, &default_de_config);
ia_css_set_gc_config(params, &default_gc_config);
@@ -2580,8 +2463,6 @@ sh_css_init_isp_params_from_global(struct ia_css_stream *stream,
params->sc_table = NULL;
params->sc_table_changed = true;
- params->sc_table_dirty = false;
- params->sc_table_last_pipe_num = 0;
ia_css_sdis2_clear_coefficients(¶ms->dvs2_coefs);
params->dvs2_coef_table_changed = true;
@@ -2639,29 +2520,15 @@ sh_css_init_isp_params_from_global(struct ia_css_stream *stream,
* BDS is enabled by the user. But we do not return
* the value immediately to enable internal firmware
* feature testing. */
-
- if (is_dp_10bpp) {
- retval = false;
- /* FIXME: should it ignore this error? */
- }
+ retval = !is_dp_10bpp;
+ /* FIXME: should it ignore this error? */
} else {
retval = false;
goto exit;
}
- if (IS_ISP2401) {
- if (stream->pipes[i]->mode < IA_CSS_PIPE_ID_NUM) {
- sh_css_set_dp_config(stream->pipes[i], params,
- &stream_params->pipe_dp_config[stream->pipes[i]->mode]);
- ia_css_set_param_exceptions(stream->pipes[i], params);
- } else {
- retval = false;
- goto exit;
- }
- }
}
- if (!IS_ISP2401)
- ia_css_set_param_exceptions(pipe_in, params);
+ ia_css_set_param_exceptions(pipe_in, params);
params->fpn_config.data = stream_params->fpn_config.data;
params->config_changed[IA_CSS_FPN_ID] =
@@ -2672,13 +2539,10 @@ sh_css_init_isp_params_from_global(struct ia_css_stream *stream,
sh_css_set_morph_table(params, stream_params->morph_table);
if (stream_params->sc_table) {
- sh_css_update_shading_table_status(pipe_in, params);
sh_css_set_shading_table(stream, params, stream_params->sc_table);
} else {
params->sc_table = NULL;
params->sc_table_changed = true;
- params->sc_table_dirty = false;
- params->sc_table_last_pipe_num = 0;
}
/* Only IA_CSS_PIPE_ID_VIDEO & IA_CSS_PIPE_ID_CAPTURE will support dvs_6axis_config*/
@@ -2771,18 +2635,6 @@ static void host_lut_store(const void *lut)
gdc_lut_store((gdc_ID_t)i, (const int (*)[HRT_GDC_N]) lut);
}
-/* Note that allocation is in ipu address space. */
-inline ia_css_ptr sh_css_params_alloc_gdc_lut(void)
-{
- return hmm_alloc(sizeof(zoom_table), HMM_BO_PRIVATE, 0, NULL, 0);
-}
-
-inline void sh_css_params_free_gdc_lut(ia_css_ptr addr)
-{
- if (addr != mmgr_NULL)
- hmm_free(addr);
-}
-
int ia_css_pipe_set_bci_scaler_lut(struct ia_css_pipe *pipe,
const void *lut)
{
@@ -2809,14 +2661,13 @@ int ia_css_pipe_set_bci_scaler_lut(struct ia_css_pipe *pipe,
}
/* Free any existing tables. */
- sh_css_params_free_gdc_lut(pipe->scaler_pp_lut);
- pipe->scaler_pp_lut = mmgr_NULL;
+ if (pipe->scaler_pp_lut != mmgr_NULL) {
+ hmm_free(pipe->scaler_pp_lut);
+ pipe->scaler_pp_lut = mmgr_NULL;
+ }
if (!stream_started) {
- if (!IS_ISP2401)
- pipe->scaler_pp_lut = hmm_alloc(sizeof(zoom_table), HMM_BO_PRIVATE, 0, NULL, 0);
- else
- pipe->scaler_pp_lut = sh_css_params_alloc_gdc_lut();
+ pipe->scaler_pp_lut = hmm_alloc(sizeof(zoom_table), HMM_BO_PRIVATE, 0, NULL, 0);
if (pipe->scaler_pp_lut == mmgr_NULL) {
ia_css_debug_dtrace(IA_CSS_DEBUG_ERROR,
@@ -2858,10 +2709,7 @@ int sh_css_params_map_and_store_default_gdc_lut(void)
host_lut_store((void *)zoom_table);
- if (!IS_ISP2401)
- default_gdc_lut = hmm_alloc(sizeof(zoom_table), HMM_BO_PRIVATE, 0, NULL, 0);
- else
- default_gdc_lut = sh_css_params_alloc_gdc_lut();
+ default_gdc_lut = hmm_alloc(sizeof(zoom_table), HMM_BO_PRIVATE, 0, NULL, 0);
if (default_gdc_lut == mmgr_NULL)
return -ENOMEM;
@@ -2879,8 +2727,10 @@ void sh_css_params_free_default_gdc_lut(void)
{
IA_CSS_ENTER_PRIVATE("void");
- sh_css_params_free_gdc_lut(default_gdc_lut);
- default_gdc_lut = mmgr_NULL;
+ if (default_gdc_lut != mmgr_NULL) {
+ hmm_free(default_gdc_lut);
+ default_gdc_lut = mmgr_NULL;
+ }
IA_CSS_LEAVE_PRIVATE("void");
}
@@ -3252,15 +3102,10 @@ sh_css_param_update_isp_params(struct ia_css_pipe *curr_pipe,
isp_pipe_version = ia_css_pipe_get_isp_pipe_version(pipe);
ia_css_pipeline_get_sp_thread_id(pipe_num, &thread_id);
-#if defined(SH_CSS_ENABLE_PER_FRAME_PARAMS)
ia_css_query_internal_queue_id(params->output_frame
? IA_CSS_BUFFER_TYPE_PER_FRAME_PARAMETER_SET
: IA_CSS_BUFFER_TYPE_PARAMETER_SET,
thread_id, &queue_id);
-#else
- ia_css_query_internal_queue_id(IA_CSS_BUFFER_TYPE_PARAMETER_SET, thread_id,
- &queue_id);
-#endif
if (!sh_css_sp_is_running()) {
/* SP is not running. The queues are not valid */
err = -EBUSY;
@@ -3356,12 +3201,10 @@ sh_css_param_update_isp_params(struct ia_css_pipe *curr_pipe,
err = ia_css_bufq_enqueue_buffer(thread_id, queue_id, (uint32_t)cpy);
if (err) {
free_ia_css_isp_parameter_set_info(cpy);
-#if defined(SH_CSS_ENABLE_PER_FRAME_PARAMS)
IA_CSS_LOG("pfp: FAILED to add config id %d for OF %d to q %d on thread %d",
isp_params_info.isp_parameters_id,
isp_params_info.output_frame_ptr,
queue_id, thread_id);
-#endif
break;
} else {
/* TMP: check discrepancy between nr of enqueued
@@ -3383,12 +3226,10 @@ sh_css_param_update_isp_params(struct ia_css_pipe *curr_pipe,
(uint8_t)thread_id,
(uint8_t)queue_id,
0);
-#if defined(SH_CSS_ENABLE_PER_FRAME_PARAMS)
IA_CSS_LOG("pfp: added config id %d for OF %d to q %d on thread %d",
isp_params_info.isp_parameters_id,
isp_params_info.output_frame_ptr,
queue_id, thread_id);
-#endif
}
/* clean-up old copy */
ia_css_dequeue_param_buffers(/*pipe_num*/);
@@ -3466,18 +3307,12 @@ sh_css_params_write_to_ddr_internal(
if (binary->info->sp.enable.sc) {
u32 enable_conv;
- size_t bytes;
-
- if (!IS_ISP2401)
- bytes = ISP2400_SCTBL_BYTES(binary);
- else
- bytes = ISP2401_SCTBL_BYTES(binary);
enable_conv = params->shading_settings.enable_shading_table_conversion;
buff_realloced = reallocate_buffer(&ddr_map->sc_tbl,
&ddr_map_size->sc_tbl,
- bytes,
+ SCTBL_BYTES(binary),
params->sc_table_changed,
&err);
if (err) {
@@ -3562,31 +3397,9 @@ sh_css_params_write_to_ddr_internal(
}
}
- /* DPC configuration is made pipe specific to allow flexibility in positioning of the
- * DPC kernel. The code below sets the pipe specific configuration to
- * individual binaries. */
- if (IS_ISP2401 &&
- params->pipe_dpc_config_changed[pipe_id] && binary->info->sp.enable.dpc) {
- unsigned int size =
- stage->binary->info->mem_offsets.offsets.param->dmem.dp.size;
-
- unsigned int offset =
- stage->binary->info->mem_offsets.offsets.param->dmem.dp.offset;
-
- if (size) {
- ia_css_dp_encode((struct sh_css_isp_dp_params *)
- &binary->mem_params.params[IA_CSS_PARAM_CLASS_PARAM][IA_CSS_ISP_DMEM].address[offset],
- ¶ms->pipe_dp_config[pipe_id], size);
-
- params->isp_params_changed = true;
- params->isp_mem_params_changed[pipe_id][stage->stage_num][IA_CSS_ISP_DMEM] =
- true;
- }
- }
-
if (params->config_changed[IA_CSS_MACC_ID] && binary->info->sp.enable.macc) {
unsigned int i, j, idx;
- unsigned int idx_map[] = {
+ static const unsigned int idx_map[] = {
0, 1, 3, 2, 6, 7, 5, 4, 12, 13, 15, 14, 10, 11, 9, 8
};
@@ -3665,13 +3478,7 @@ sh_css_params_write_to_ddr_internal(
if (!params->pipe_dvs_6axis_config[pipe_id]) {
struct ia_css_resolution dvs_offset = {0};
- if (!IS_ISP2401) {
- dvs_offset.width = (PIX_SHIFT_FILTER_RUN_IN_X + binary->dvs_envelope.width) / 2;
- } else {
- if (binary->dvs_envelope.width || binary->dvs_envelope.height) {
- dvs_offset.width = (PIX_SHIFT_FILTER_RUN_IN_X + binary->dvs_envelope.width) / 2;
- }
- }
+ dvs_offset.width = (PIX_SHIFT_FILTER_RUN_IN_X + binary->dvs_envelope.width) / 2;
dvs_offset.height = (PIX_SHIFT_FILTER_RUN_IN_Y + binary->dvs_envelope.height) / 2;
params->pipe_dvs_6axis_config[pipe_id] =
@@ -4335,12 +4142,8 @@ ia_css_3a_statistics_allocate(const struct ia_css_3a_grid_info *grid)
me->data = kvmalloc(grid_size * sizeof(*me->data), GFP_KERNEL);
if (!me->data)
goto err;
-#if !defined(HAS_NO_HMEM)
/* No weighted histogram, no structure, treat the histogram data as a byte dump in a byte array */
me->rgby_data = kvmalloc(sizeof_hmem(HMEM0_ID), GFP_KERNEL);
-#else
- me->rgby_data = NULL;
-#endif
IA_CSS_LEAVE("return=%p", me);
return me;
diff --git a/drivers/staging/media/atomisp/pci/sh_css_params.h b/drivers/staging/media/atomisp/pci/sh_css_params.h
index 62a7b6a..bbca19d 100644
--- a/drivers/staging/media/atomisp/pci/sh_css_params.h
+++ b/drivers/staging/media/atomisp/pci/sh_css_params.h
@@ -121,8 +121,6 @@ struct ia_css_isp_parameters {
bool dvs2_coef_table_changed;
bool morph_table_changed;
bool sc_table_changed;
- bool sc_table_dirty;
- unsigned int sc_table_last_pipe_num;
bool anr_thres_changed;
/* ---- deprecated: replaced with pipe_dvs_6axis_config_changed ---- */
bool dvs_6axis_config_changed;
@@ -168,12 +166,6 @@ ia_css_params_alloc_convert_sctbl(
struct ia_css_isp_config *
sh_css_pipe_isp_config_get(struct ia_css_pipe *pipe);
-/* ipu address allocation/free for gdc lut */
-ia_css_ptr
-sh_css_params_alloc_gdc_lut(void);
-void
-sh_css_params_free_gdc_lut(ia_css_ptr addr);
-
int
sh_css_params_map_and_store_default_gdc_lut(void);
diff --git a/drivers/staging/media/atomisp/pci/sh_css_shading.c b/drivers/staging/media/atomisp/pci/sh_css_shading.c
deleted file mode 100644
index 462caf9..0000000
--- a/drivers/staging/media/atomisp/pci/sh_css_shading.c
+++ /dev/null
@@ -1,17 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/*
- * Support for Intel Camera Imaging ISP subsystem.
- * Copyright (c) 2015, Intel Corporation.
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms and conditions of the GNU General Public License,
- * version 2, as published by the Free Software Foundation.
- *
- * This program is distributed in the hope it will be useful, but WITHOUT
- * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
- * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
- * more details.
- */
-
-/* This file will contain the code to implement the functions declared in ia_css_shading.h
- and associated helper functions */
diff --git a/drivers/staging/media/atomisp/pci/sh_css_sp.c b/drivers/staging/media/atomisp/pci/sh_css_sp.c
index a73e8ca..615500a 100644
--- a/drivers/staging/media/atomisp/pci/sh_css_sp.c
+++ b/drivers/staging/media/atomisp/pci/sh_css_sp.c
@@ -330,9 +330,7 @@ sh_css_sp_start_isys_copy(struct ia_css_frame *out_frame,
unsigned int thread_id;
u8 stage_num = 0;
struct sh_css_sp_pipeline *pipe;
-#if defined SH_CSS_ENABLE_METADATA
enum sh_css_queue_id queue_id;
-#endif
assert(out_frame);
@@ -372,7 +370,6 @@ sh_css_sp_start_isys_copy(struct ia_css_frame *out_frame,
set_output_frame_buffer(out_frame, 0);
-#if defined SH_CSS_ENABLE_METADATA
if (pipe->metadata.height > 0) {
ia_css_query_internal_queue_id(IA_CSS_BUFFER_TYPE_METADATA, thread_id,
&queue_id);
@@ -380,7 +377,6 @@ sh_css_sp_start_isys_copy(struct ia_css_frame *out_frame,
queue_id, mmgr_EXCEPTION,
IA_CSS_BUFFER_TYPE_METADATA);
}
-#endif
ia_css_debug_pipe_graph_dump_sp_raw_copy(out_frame);
}
@@ -812,29 +808,47 @@ is_sp_stage(struct ia_css_pipeline_stage *stage)
return stage->sp_func != IA_CSS_PIPELINE_NO_FUNC;
}
-static int
-configure_isp_from_args(
- const struct sh_css_sp_pipeline *pipeline,
- const struct ia_css_binary *binary,
- const struct sh_css_binary_args *args,
- bool two_ppc,
- bool deinterleaved)
+static int configure_isp_from_args(const struct sh_css_sp_pipeline *pipeline,
+ const struct ia_css_binary *binary,
+ const struct sh_css_binary_args *args,
+ bool two_ppc,
+ bool deinterleaved)
{
- ia_css_fpn_configure(binary, &binary->in_frame_info);
- ia_css_crop_configure(binary, &args->delay_frames[0]->info);
- ia_css_qplane_configure(pipeline, binary, &binary->in_frame_info);
- ia_css_output0_configure(binary, &args->out_frame[0]->info);
- ia_css_output1_configure(binary, &args->out_vf_frame->info);
- ia_css_copy_output_configure(binary, args->copy_output);
- ia_css_output0_configure(binary, &args->out_frame[0]->info);
-#ifdef ISP2401
- ia_css_sc_configure(binary, pipeline->shading.internal_frame_origin_x_bqs_on_sctbl,
- pipeline->shading.internal_frame_origin_y_bqs_on_sctbl);
-#endif
- ia_css_iterator_configure(binary, &args->in_frame->info);
- ia_css_dvs_configure(binary, &args->out_frame[0]->info);
- ia_css_output_configure(binary, &args->out_frame[0]->info);
- ia_css_raw_configure(pipeline, binary, &args->in_frame->info, &binary->in_frame_info, two_ppc, deinterleaved);
+ int ret;
+
+ ret = ia_css_fpn_configure(binary, &binary->in_frame_info);
+ if (ret)
+ return ret;
+ ret = ia_css_crop_configure(binary, &args->delay_frames[0]->info);
+ if (ret)
+ return ret;
+ ret = ia_css_qplane_configure(pipeline, binary, &binary->in_frame_info);
+ if (ret)
+ return ret;
+ ret = ia_css_output0_configure(binary, &args->out_frame[0]->info);
+ if (ret)
+ return ret;
+ ret = ia_css_output1_configure(binary, &args->out_vf_frame->info);
+ if (ret)
+ return ret;
+ ret = ia_css_copy_output_configure(binary, args->copy_output);
+ if (ret)
+ return ret;
+ ret = ia_css_output0_configure(binary, &args->out_frame[0]->info);
+ if (ret)
+ return ret;
+ ret = ia_css_iterator_configure(binary, &args->in_frame->info);
+ if (ret)
+ return ret;
+ ret = ia_css_dvs_configure(binary, &args->out_frame[0]->info);
+ if (ret)
+ return ret;
+ ret = ia_css_output_configure(binary, &args->out_frame[0]->info);
+ if (ret)
+ return ret;
+ ret = ia_css_raw_configure(pipeline, binary, &args->in_frame->info, &binary->in_frame_info, two_ppc, deinterleaved);
+ if (ret)
+ return ret;
/*
* FIXME: args->delay_frames can be NULL here
@@ -846,10 +860,13 @@ configure_isp_from_args(
* without crashing, but the pipeline should likely be built without
* adding it at the first place (or there are a hidden bug somewhere)
*/
- ia_css_ref_configure(binary, args->delay_frames, pipeline->dvs_frame_delay);
- ia_css_tnr_configure(binary, args->tnr_frames);
- ia_css_bayer_io_config(binary, args);
- return 0;
+ ret = ia_css_ref_configure(binary, args->delay_frames, pipeline->dvs_frame_delay);
+ if (ret)
+ return ret;
+ ret = ia_css_tnr_configure(binary, args->tnr_frames);
+ if (ret)
+ return ret;
+ return ia_css_bayer_io_config(binary, args);
}
static void
@@ -883,9 +900,7 @@ initialize_stage_frames(struct ia_css_frames_sp *frames)
initialize_frame_buffer_attribute(&frames->out_vf.buf_attr);
initialize_frame_buffer_attribute(&frames->s3a_buf);
initialize_frame_buffer_attribute(&frames->dvs_buf);
-#if defined SH_CSS_ENABLE_METADATA
initialize_frame_buffer_attribute(&frames->metadata_buf);
-#endif
}
static int
@@ -1011,24 +1026,20 @@ sh_css_sp_init_stage(struct ia_css_binary *binary,
mmgr_EXCEPTION,
IA_CSS_BUFFER_TYPE_DIS_STATISTICS);
}
-#if defined SH_CSS_ENABLE_METADATA
ia_css_query_internal_queue_id(IA_CSS_BUFFER_TYPE_METADATA, thread_id, &queue_id);
sh_css_copy_buffer_attr_to_spbuffer(&sh_css_sp_stage.frames.metadata_buf, queue_id, mmgr_EXCEPTION, IA_CSS_BUFFER_TYPE_METADATA);
-#endif
if (err)
return err;
#ifdef ISP2401
- if (stage == 0) {
- pipe = find_pipe_by_num(sh_css_sp_group.pipe[thread_id].pipe_num);
- if (!pipe)
- return -EINVAL;
+ pipe = find_pipe_by_num(sh_css_sp_group.pipe[thread_id].pipe_num);
+ if (!pipe)
+ return -EINVAL;
- if (args->in_frame)
- ia_css_get_crop_offsets(pipe, &args->in_frame->info);
- else
- ia_css_get_crop_offsets(pipe, &binary->in_frame_info);
- }
+ if (args->in_frame)
+ ia_css_get_crop_offsets(pipe, &args->in_frame->info);
+ else
+ ia_css_get_crop_offsets(pipe, &binary->in_frame_info);
#else
(void)pipe; /*avoid build warning*/
#endif
@@ -1196,12 +1207,7 @@ sh_css_sp_init_pipeline(struct ia_css_pipeline *me,
enum ia_css_input_mode input_mode,
const struct ia_css_metadata_config *md_config,
const struct ia_css_metadata_info *md_info,
- const enum mipi_port_id port_id,
- const struct ia_css_coordinate
- *internal_frame_origin_bqs_on_sctbl, /* Origin of internal frame
- positioned on shading table at shading correction in ISP. */
- const struct ia_css_isp_parameters *params
- )
+ const enum mipi_port_id port_id)
{
/* Get first stage */
struct ia_css_pipeline_stage *stage = NULL;
@@ -1288,7 +1294,6 @@ sh_css_sp_init_pipeline(struct ia_css_pipeline *me,
}
sh_css_sp_group.pipe[thread_id].scaler_pp_lut = sh_css_pipe_get_pp_gdc_lut(pipe);
-#if defined(SH_CSS_ENABLE_METADATA)
if (md_info && md_info->size > 0) {
sh_css_sp_group.pipe[thread_id].metadata.width = md_info->resolution.width;
sh_css_sp_group.pipe[thread_id].metadata.height = md_info->resolution.height;
@@ -1298,37 +1303,13 @@ sh_css_sp_init_pipeline(struct ia_css_pipeline *me,
md_config->data_type, MIPI_PREDICTOR_NONE,
&sh_css_sp_group.pipe[thread_id].metadata.format);
}
-#else
- (void)md_config;
- (void)md_info;
-#endif
-#if defined(SH_CSS_ENABLE_PER_FRAME_PARAMS)
sh_css_sp_group.pipe[thread_id].output_frame_queue_id = (uint32_t)SH_CSS_INVALID_QUEUE_ID;
if (pipe_id != IA_CSS_PIPE_ID_COPY) {
ia_css_query_internal_queue_id(IA_CSS_BUFFER_TYPE_OUTPUT_FRAME, thread_id,
(enum sh_css_queue_id *)(
&sh_css_sp_group.pipe[thread_id].output_frame_queue_id));
}
-#endif
-
- if (IS_ISP2401) {
- /* For the shading correction type 1 (the legacy shading table conversion in css is not used),
- * the parameters are passed to the isp for the shading table centering.
- */
- if (internal_frame_origin_bqs_on_sctbl &&
- params && params->shading_settings.enable_shading_table_conversion == 0) {
- sh_css_sp_group.pipe[thread_id].shading.internal_frame_origin_x_bqs_on_sctbl
- = (uint32_t)internal_frame_origin_bqs_on_sctbl->x;
- sh_css_sp_group.pipe[thread_id].shading.internal_frame_origin_y_bqs_on_sctbl
- = (uint32_t)internal_frame_origin_bqs_on_sctbl->y;
- } else {
- sh_css_sp_group.pipe[thread_id].shading.internal_frame_origin_x_bqs_on_sctbl =
- 0;
- sh_css_sp_group.pipe[thread_id].shading.internal_frame_origin_y_bqs_on_sctbl =
- 0;
- }
- }
IA_CSS_LOG("pipe_id %d port_config %08x",
pipe_id, sh_css_sp_group.pipe[thread_id].inout_port_config);
diff --git a/drivers/staging/media/atomisp/pci/sh_css_sp.h b/drivers/staging/media/atomisp/pci/sh_css_sp.h
index 832eed7..f69a79b 100644
--- a/drivers/staging/media/atomisp/pci/sh_css_sp.h
+++ b/drivers/staging/media/atomisp/pci/sh_css_sp.h
@@ -66,12 +66,7 @@ sh_css_sp_init_pipeline(struct ia_css_pipeline *me,
enum ia_css_input_mode input_mode,
const struct ia_css_metadata_config *md_config,
const struct ia_css_metadata_info *md_info,
- const enum mipi_port_id port_id,
- const struct ia_css_coordinate
- *internal_frame_origin_bqs_on_sctbl, /* Origin of internal frame
- positioned on shading table at shading correction in ISP. */
- const struct ia_css_isp_parameters *params
- );
+ const enum mipi_port_id port_id);
void
sh_css_sp_uninit_pipeline(unsigned int pipe_num);
diff --git a/drivers/staging/media/atomisp/pci/sh_css_stream.c b/drivers/staging/media/atomisp/pci/sh_css_stream.c
deleted file mode 100644
index a768ce9..0000000
--- a/drivers/staging/media/atomisp/pci/sh_css_stream.c
+++ /dev/null
@@ -1,17 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/*
- * Support for Intel Camera Imaging ISP subsystem.
- * Copyright (c) 2015, Intel Corporation.
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms and conditions of the GNU General Public License,
- * version 2, as published by the Free Software Foundation.
- *
- * This program is distributed in the hope it will be useful, but WITHOUT
- * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
- * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
- * more details.
- */
-
-/* This file will contain the code to implement the functions declared in ia_css_stream.h
- and associated helper functions */
diff --git a/drivers/staging/media/atomisp/pci/system_global.h b/drivers/staging/media/atomisp/pci/system_global.h
index 9b22b8c1..060b924 100644
--- a/drivers/staging/media/atomisp/pci/system_global.h
+++ b/drivers/staging/media/atomisp/pci/system_global.h
@@ -25,9 +25,6 @@
* N.B. the 3 input formatters are of 2 different classess
*/
-/* per-frame parameter handling support */
-#define SH_CSS_ENABLE_PER_FRAME_PARAMS
-
#define DMA_DDR_TO_VAMEM_WORKAROUND
#define DMA_DDR_TO_HMEM_WORKAROUND
diff --git a/drivers/staging/media/hantro/Kconfig b/drivers/staging/media/hantro/Kconfig
index 20b1f6d..3c5d833 100644
--- a/drivers/staging/media/hantro/Kconfig
+++ b/drivers/staging/media/hantro/Kconfig
@@ -1,7 +1,7 @@
# SPDX-License-Identifier: GPL-2.0
config VIDEO_HANTRO
tristate "Hantro VPU driver"
- depends on ARCH_MXC || ARCH_ROCKCHIP || ARCH_AT91 || COMPILE_TEST
+ depends on ARCH_MXC || ARCH_ROCKCHIP || ARCH_AT91 || ARCH_SUNXI || COMPILE_TEST
depends on VIDEO_DEV && VIDEO_V4L2
select MEDIA_CONTROLLER
select MEDIA_CONTROLLER_REQUEST_API
@@ -9,6 +9,7 @@
select VIDEOBUF2_VMALLOC
select V4L2_MEM2MEM_DEV
select V4L2_H264
+ select V4L2_VP9
help
Support for the Hantro IP based Video Processing Units present on
Rockchip and NXP i.MX8M SoCs, which accelerate video and image
@@ -39,3 +40,11 @@
default y
help
Enable support for RK3288, RK3328, and RK3399 SoCs.
+
+config VIDEO_HANTRO_SUNXI
+ bool "Hantro VPU Allwinner support"
+ depends on VIDEO_HANTRO
+ depends on ARCH_SUNXI || COMPILE_TEST
+ default y
+ help
+ Enable support for H6 SoC.
diff --git a/drivers/staging/media/hantro/Makefile b/drivers/staging/media/hantro/Makefile
index 9003683..ebd5ede 100644
--- a/drivers/staging/media/hantro/Makefile
+++ b/drivers/staging/media/hantro/Makefile
@@ -10,8 +10,10 @@
hantro_g1.o \
hantro_g1_h264_dec.o \
hantro_g1_mpeg2_dec.o \
- hantro_g2_hevc_dec.o \
hantro_g1_vp8_dec.o \
+ hantro_g2.o \
+ hantro_g2_hevc_dec.o \
+ hantro_g2_vp9_dec.o \
rockchip_vpu2_hw_jpeg_enc.o \
rockchip_vpu2_hw_h264_dec.o \
rockchip_vpu2_hw_mpeg2_dec.o \
@@ -20,7 +22,8 @@
hantro_h264.o \
hantro_hevc.o \
hantro_mpeg2.o \
- hantro_vp8.o
+ hantro_vp8.o \
+ hantro_vp9.o
hantro-vpu-$(CONFIG_VIDEO_HANTRO_IMX8M) += \
imx8m_vpu_hw.o
@@ -30,3 +33,6 @@
hantro-vpu-$(CONFIG_VIDEO_HANTRO_ROCKCHIP) += \
rockchip_vpu_hw.o
+
+hantro-vpu-$(CONFIG_VIDEO_HANTRO_SUNXI) += \
+ sunxi_vpu_hw.o
diff --git a/drivers/staging/media/hantro/hantro.h b/drivers/staging/media/hantro/hantro.h
index c2e2dca..06d0f35 100644
--- a/drivers/staging/media/hantro/hantro.h
+++ b/drivers/staging/media/hantro/hantro.h
@@ -16,6 +16,7 @@
#include <linux/videodev2.h>
#include <linux/wait.h>
#include <linux/clk.h>
+#include <linux/reset.h>
#include <media/v4l2-ctrls.h>
#include <media/v4l2-device.h>
@@ -28,6 +29,7 @@
struct hantro_ctx;
struct hantro_codec_ops;
+struct hantro_postproc_ops;
#define HANTRO_JPEG_ENCODER BIT(0)
#define HANTRO_ENCODERS 0x0000ffff
@@ -35,6 +37,7 @@ struct hantro_codec_ops;
#define HANTRO_VP8_DECODER BIT(17)
#define HANTRO_H264_DECODER BIT(18)
#define HANTRO_HEVC_DECODER BIT(19)
+#define HANTRO_VP9_DECODER BIT(20)
#define HANTRO_DECODERS 0xffff0000
/**
@@ -59,6 +62,7 @@ struct hantro_irq {
* @num_dec_fmts: Number of decoder formats.
* @postproc_fmts: Post-processor formats.
* @num_postproc_fmts: Number of post-processor formats.
+ * @postproc_ops: Post-processor ops.
* @codec: Supported codecs
* @codec_ops: Codec ops.
* @init: Initialize hardware, optional.
@@ -69,7 +73,9 @@ struct hantro_irq {
* @num_clocks: number of clocks in the array
* @reg_names: array of register range names
* @num_regs: number of register range names in the array
- * @postproc_regs: &struct hantro_postproc_regs pointer
+ * @double_buffer: core needs double buffering
+ * @legacy_regs: core uses legacy register set
+ * @late_postproc: postproc must be set up at the end of the job
*/
struct hantro_variant {
unsigned int enc_offset;
@@ -80,6 +86,7 @@ struct hantro_variant {
unsigned int num_dec_fmts;
const struct hantro_fmt *postproc_fmts;
unsigned int num_postproc_fmts;
+ const struct hantro_postproc_ops *postproc_ops;
unsigned int codec;
const struct hantro_codec_ops *codec_ops;
int (*init)(struct hantro_dev *vpu);
@@ -90,7 +97,9 @@ struct hantro_variant {
int num_clocks;
const char * const *reg_names;
int num_regs;
- const struct hantro_postproc_regs *postproc_regs;
+ unsigned int double_buffer : 1;
+ unsigned int legacy_regs : 1;
+ unsigned int late_postproc : 1;
};
/**
@@ -101,6 +110,7 @@ struct hantro_variant {
* @HANTRO_MODE_MPEG2_DEC: MPEG-2 decoder.
* @HANTRO_MODE_VP8_DEC: VP8 decoder.
* @HANTRO_MODE_HEVC_DEC: HEVC decoder.
+ * @HANTRO_MODE_VP9_DEC: VP9 decoder.
*/
enum hantro_codec_mode {
HANTRO_MODE_NONE = -1,
@@ -109,6 +119,7 @@ enum hantro_codec_mode {
HANTRO_MODE_MPEG2_DEC,
HANTRO_MODE_VP8_DEC,
HANTRO_MODE_HEVC_DEC,
+ HANTRO_MODE_VP9_DEC,
};
/*
@@ -167,6 +178,7 @@ hantro_vdev_to_func(struct video_device *vdev)
* @dev: Pointer to device for convenient logging using
* dev_ macros.
* @clocks: Array of clock handles.
+ * @resets: Array of reset handles.
* @reg_bases: Mapped addresses of VPU registers.
* @enc_base: Mapped address of VPU encoder register for convenience.
* @dec_base: Mapped address of VPU decoder register for convenience.
@@ -186,6 +198,7 @@ struct hantro_dev {
struct platform_device *pdev;
struct device *dev;
struct clk_bulk_data *clocks;
+ struct reset_control *resets;
void __iomem **reg_bases;
void __iomem *enc_base;
void __iomem *dec_base;
@@ -222,6 +235,7 @@ struct hantro_dev {
* @mpeg2_dec: MPEG-2-decoding context.
* @vp8_dec: VP8-decoding context.
* @hevc_dec: HEVC-decoding context.
+ * @vp9_dec: VP9-decoding context.
*/
struct hantro_ctx {
struct hantro_dev *dev;
@@ -249,6 +263,7 @@ struct hantro_ctx {
struct hantro_mpeg2_dec_hw_ctx mpeg2_dec;
struct hantro_vp8_dec_hw_ctx vp8_dec;
struct hantro_hevc_dec_hw_ctx hevc_dec;
+ struct hantro_vp9_dec_hw_ctx vp9_dec;
};
};
@@ -262,6 +277,7 @@ struct hantro_ctx {
* @max_depth: Maximum depth, for bitstream formats
* @enc_fmt: Format identifier for encoder registers.
* @frmsize: Supported range of frame sizes (only for bitstream formats).
+ * @postprocessed: Indicates if this format needs the post-processor.
*/
struct hantro_fmt {
char *name;
@@ -271,6 +287,7 @@ struct hantro_fmt {
int max_depth;
enum hantro_enc_fmt enc_fmt;
struct v4l2_frmsize_stepwise frmsize;
+ bool postprocessed;
};
struct hantro_reg {
@@ -296,6 +313,22 @@ struct hantro_postproc_regs {
struct hantro_reg display_width;
};
+struct hantro_vp9_decoded_buffer_info {
+ /* Info needed when the decoded frame serves as a reference frame. */
+ unsigned short width;
+ unsigned short height;
+ u32 bit_depth : 4;
+};
+
+struct hantro_decoded_buffer {
+ /* Must be the first field in this struct. */
+ struct v4l2_m2m_buffer base;
+
+ union {
+ struct hantro_vp9_decoded_buffer_info vp9;
+ };
+};
+
/* Logging helpers */
/**
@@ -366,6 +399,13 @@ static inline void vdpu_write(struct hantro_dev *vpu, u32 val, u32 reg)
writel(val, vpu->dec_base + reg);
}
+static inline void hantro_write_addr(struct hantro_dev *vpu,
+ unsigned long offset,
+ dma_addr_t addr)
+{
+ vdpu_write(vpu, addr & 0xffffffff, offset);
+}
+
static inline u32 vdpu_read(struct hantro_dev *vpu, u32 reg)
{
u32 val = readl(vpu->dec_base + reg);
@@ -426,6 +466,12 @@ hantro_get_dec_buf_addr(struct hantro_ctx *ctx, struct vb2_buffer *vb)
return vb2_dma_contig_plane_dma_addr(vb, 0);
}
+static inline struct hantro_decoded_buffer *
+vb2_to_hantro_decoded_buf(struct vb2_buffer *buf)
+{
+ return container_of(buf, struct hantro_decoded_buffer, base.vb.vb2_buf);
+}
+
void hantro_postproc_disable(struct hantro_ctx *ctx);
void hantro_postproc_enable(struct hantro_ctx *ctx);
void hantro_postproc_free(struct hantro_ctx *ctx);
diff --git a/drivers/staging/media/hantro/hantro_drv.c b/drivers/staging/media/hantro/hantro_drv.c
index fb82b92..6a51f39 100644
--- a/drivers/staging/media/hantro/hantro_drv.c
+++ b/drivers/staging/media/hantro/hantro_drv.c
@@ -130,7 +130,7 @@ void hantro_start_prepare_run(struct hantro_ctx *ctx)
v4l2_ctrl_request_setup(src_buf->vb2_buf.req_obj.req,
&ctx->ctrl_handler);
- if (!ctx->is_encoder) {
+ if (!ctx->is_encoder && !ctx->dev->variant->late_postproc) {
if (hantro_needs_postproc(ctx, ctx->vpu_dst_fmt))
hantro_postproc_enable(ctx);
else
@@ -142,6 +142,13 @@ void hantro_end_prepare_run(struct hantro_ctx *ctx)
{
struct vb2_v4l2_buffer *src_buf;
+ if (!ctx->is_encoder && ctx->dev->variant->late_postproc) {
+ if (hantro_needs_postproc(ctx, ctx->vpu_dst_fmt))
+ hantro_postproc_enable(ctx);
+ else
+ hantro_postproc_disable(ctx);
+ }
+
src_buf = hantro_get_src_buf(ctx);
v4l2_ctrl_request_complete(src_buf->vb2_buf.req_obj.req,
&ctx->ctrl_handler);
@@ -232,7 +239,7 @@ queue_init(void *priv, struct vb2_queue *src_vq, struct vb2_queue *dst_vq)
dst_vq->io_modes = VB2_MMAP | VB2_DMABUF;
dst_vq->drv_priv = ctx;
dst_vq->ops = &hantro_queue_ops;
- dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
+ dst_vq->buf_struct_size = sizeof(struct hantro_decoded_buffer);
dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
dst_vq->lock = &ctx->dev->vpu_mutex;
dst_vq->dev = ctx->dev->v4l2_dev.dev;
@@ -263,6 +270,12 @@ static int hantro_try_ctrl(struct v4l2_ctrl *ctrl)
if (sps->bit_depth_luma_minus8 != 0)
/* Only 8-bit is supported */
return -EINVAL;
+ } else if (ctrl->id == V4L2_CID_STATELESS_VP9_FRAME) {
+ const struct v4l2_ctrl_vp9_frame *dec_params = ctrl->p_new.p_vp9_frame;
+
+ /* We only support profile 0 */
+ if (dec_params->profile != 0)
+ return -EINVAL;
}
return 0;
}
@@ -461,6 +474,16 @@ static const struct hantro_ctrl controls[] = {
.step = 1,
.ops = &hantro_hevc_ctrl_ops,
},
+ }, {
+ .codec = HANTRO_VP9_DECODER,
+ .cfg = {
+ .id = V4L2_CID_STATELESS_VP9_FRAME,
+ },
+ }, {
+ .codec = HANTRO_VP9_DECODER,
+ .cfg = {
+ .id = V4L2_CID_STATELESS_VP9_COMPRESSED_HDR,
+ },
},
};
@@ -598,6 +621,9 @@ static const struct of_device_id of_hantro_match[] = {
#ifdef CONFIG_VIDEO_HANTRO_SAMA5D4
{ .compatible = "microchip,sama5d4-vdec", .data = &sama5d4_vdec_variant, },
#endif
+#ifdef CONFIG_VIDEO_HANTRO_SUNXI
+ { .compatible = "allwinner,sun50i-h6-vpu-g2", .data = &sunxi_vpu_variant, },
+#endif
{ /* sentinel */ }
};
MODULE_DEVICE_TABLE(of, of_hantro_match);
@@ -889,6 +915,10 @@ static int hantro_probe(struct platform_device *pdev)
return PTR_ERR(vpu->clocks[0].clk);
}
+ vpu->resets = devm_reset_control_array_get(&pdev->dev, false, true);
+ if (IS_ERR(vpu->resets))
+ return PTR_ERR(vpu->resets);
+
num_bases = vpu->variant->num_regs ?: 1;
vpu->reg_bases = devm_kcalloc(&pdev->dev, num_bases,
sizeof(*vpu->reg_bases), GFP_KERNEL);
@@ -907,6 +937,11 @@ static int hantro_probe(struct platform_device *pdev)
vpu->enc_base = vpu->reg_bases[0] + vpu->variant->enc_offset;
vpu->dec_base = vpu->reg_bases[0] + vpu->variant->dec_offset;
+ /**
+ * TODO: Eventually allow taking advantage of full 64-bit address space.
+ * Until then we assume the MSB portion of buffers' base addresses is
+ * always 0 due to this masking operation.
+ */
ret = dma_set_coherent_mask(vpu->dev, DMA_BIT_MASK(32));
if (ret) {
dev_err(vpu->dev, "Could not set DMA coherent mask.\n");
@@ -957,10 +992,16 @@ static int hantro_probe(struct platform_device *pdev)
pm_runtime_use_autosuspend(vpu->dev);
pm_runtime_enable(vpu->dev);
+ ret = reset_control_deassert(vpu->resets);
+ if (ret) {
+ dev_err(&pdev->dev, "Failed to deassert resets\n");
+ goto err_pm_disable;
+ }
+
ret = clk_bulk_prepare(vpu->variant->num_clocks, vpu->clocks);
if (ret) {
dev_err(&pdev->dev, "Failed to prepare clocks\n");
- return ret;
+ goto err_rst_assert;
}
ret = v4l2_device_register(&pdev->dev, &vpu->v4l2_dev);
@@ -1016,6 +1057,9 @@ static int hantro_probe(struct platform_device *pdev)
v4l2_device_unregister(&vpu->v4l2_dev);
err_clk_unprepare:
clk_bulk_unprepare(vpu->variant->num_clocks, vpu->clocks);
+err_rst_assert:
+ reset_control_assert(vpu->resets);
+err_pm_disable:
pm_runtime_dont_use_autosuspend(vpu->dev);
pm_runtime_disable(vpu->dev);
return ret;
@@ -1034,6 +1078,7 @@ static int hantro_remove(struct platform_device *pdev)
v4l2_m2m_release(vpu->m2m_dev);
v4l2_device_unregister(&vpu->v4l2_dev);
clk_bulk_unprepare(vpu->variant->num_clocks, vpu->clocks);
+ reset_control_assert(vpu->resets);
pm_runtime_dont_use_autosuspend(vpu->dev);
pm_runtime_disable(vpu->dev);
return 0;
diff --git a/drivers/staging/media/hantro/hantro_g2.c b/drivers/staging/media/hantro/hantro_g2.c
new file mode 100644
index 0000000..ee5f14c
--- /dev/null
+++ b/drivers/staging/media/hantro/hantro_g2.c
@@ -0,0 +1,44 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Hantro VPU codec driver
+ *
+ * Copyright (C) 2021 Collabora Ltd, Andrzej Pietrasiewicz <andrzej.p@collabora.com>
+ */
+
+#include "hantro_hw.h"
+#include "hantro_g2_regs.h"
+
+void hantro_g2_check_idle(struct hantro_dev *vpu)
+{
+ int i;
+
+ for (i = 0; i < 3; i++) {
+ u32 status;
+
+ /* Make sure the VPU is idle */
+ status = vdpu_read(vpu, G2_REG_INTERRUPT);
+ if (status & G2_REG_INTERRUPT_DEC_E) {
+ dev_warn(vpu->dev, "device still running, aborting");
+ status |= G2_REG_INTERRUPT_DEC_ABORT_E | G2_REG_INTERRUPT_DEC_IRQ_DIS;
+ vdpu_write(vpu, status, G2_REG_INTERRUPT);
+ }
+ }
+}
+
+irqreturn_t hantro_g2_irq(int irq, void *dev_id)
+{
+ struct hantro_dev *vpu = dev_id;
+ enum vb2_buffer_state state;
+ u32 status;
+
+ status = vdpu_read(vpu, G2_REG_INTERRUPT);
+ state = (status & G2_REG_INTERRUPT_DEC_RDY_INT) ?
+ VB2_BUF_STATE_DONE : VB2_BUF_STATE_ERROR;
+
+ vdpu_write(vpu, 0, G2_REG_INTERRUPT);
+ vdpu_write(vpu, G2_REG_CONFIG_DEC_CLK_GATE_E, G2_REG_CONFIG);
+
+ hantro_irq_done(vpu, state);
+
+ return IRQ_HANDLED;
+}
diff --git a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
index 76a9211..99d8ea7 100644
--- a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
+++ b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
@@ -8,20 +8,6 @@
#include "hantro_hw.h"
#include "hantro_g2_regs.h"
-#define HEVC_DEC_MODE 0xC
-
-#define BUS_WIDTH_32 0
-#define BUS_WIDTH_64 1
-#define BUS_WIDTH_128 2
-#define BUS_WIDTH_256 3
-
-static inline void hantro_write_addr(struct hantro_dev *vpu,
- unsigned long offset,
- dma_addr_t addr)
-{
- vdpu_write(vpu, addr & 0xffffffff, offset);
-}
-
static void prepare_tile_info_buffer(struct hantro_ctx *ctx)
{
struct hantro_dev *vpu = ctx->dev;
@@ -368,6 +354,8 @@ static int set_ref(struct hantro_ctx *ctx)
const struct v4l2_hevc_dpb_entry *dpb = decode_params->dpb;
dma_addr_t luma_addr, chroma_addr, mv_addr = 0;
struct hantro_dev *vpu = ctx->dev;
+ struct vb2_v4l2_buffer *vb2_dst;
+ struct hantro_decoded_buffer *dst;
size_t cr_offset = hantro_hevc_chroma_offset(sps);
size_t mv_offset = hantro_hevc_motion_vectors_offset(sps);
u32 max_ref_frames;
@@ -448,32 +436,37 @@ static int set_ref(struct hantro_ctx *ctx)
if (dpb[i].rps == V4L2_HEVC_DPB_ENTRY_RPS_LT_CURR)
dpb_longterm_e |= BIT(V4L2_HEVC_DPB_ENTRIES_NUM_MAX - 1 - i);
- hantro_write_addr(vpu, G2_REG_ADDR_REF(i), luma_addr);
- hantro_write_addr(vpu, G2_REG_CHR_REF(i), chroma_addr);
- hantro_write_addr(vpu, G2_REG_DMV_REF(i), mv_addr);
+ hantro_write_addr(vpu, G2_REF_LUMA_ADDR(i), luma_addr);
+ hantro_write_addr(vpu, G2_REF_CHROMA_ADDR(i), chroma_addr);
+ hantro_write_addr(vpu, G2_REF_MV_ADDR(i), mv_addr);
}
- luma_addr = hantro_hevc_get_ref_buf(ctx, decode_params->pic_order_cnt_val);
+ vb2_dst = hantro_get_dst_buf(ctx);
+ dst = vb2_to_hantro_decoded_buf(&vb2_dst->vb2_buf);
+ luma_addr = hantro_get_dec_buf_addr(ctx, &dst->base.vb.vb2_buf);
if (!luma_addr)
return -ENOMEM;
+ if (hantro_hevc_add_ref_buf(ctx, decode_params->pic_order_cnt_val, luma_addr))
+ return -EINVAL;
+
chroma_addr = luma_addr + cr_offset;
mv_addr = luma_addr + mv_offset;
- hantro_write_addr(vpu, G2_REG_ADDR_REF(i), luma_addr);
- hantro_write_addr(vpu, G2_REG_CHR_REF(i), chroma_addr);
- hantro_write_addr(vpu, G2_REG_DMV_REF(i++), mv_addr);
+ hantro_write_addr(vpu, G2_REF_LUMA_ADDR(i), luma_addr);
+ hantro_write_addr(vpu, G2_REF_CHROMA_ADDR(i), chroma_addr);
+ hantro_write_addr(vpu, G2_REF_MV_ADDR(i++), mv_addr);
- hantro_write_addr(vpu, G2_ADDR_DST, luma_addr);
- hantro_write_addr(vpu, G2_ADDR_DST_CHR, chroma_addr);
- hantro_write_addr(vpu, G2_ADDR_DST_MV, mv_addr);
+ hantro_write_addr(vpu, G2_OUT_LUMA_ADDR, luma_addr);
+ hantro_write_addr(vpu, G2_OUT_CHROMA_ADDR, chroma_addr);
+ hantro_write_addr(vpu, G2_OUT_MV_ADDR, mv_addr);
hantro_hevc_ref_remove_unused(ctx);
for (; i < V4L2_HEVC_DPB_ENTRIES_NUM_MAX; i++) {
- hantro_write_addr(vpu, G2_REG_ADDR_REF(i), 0);
- hantro_write_addr(vpu, G2_REG_CHR_REF(i), 0);
- hantro_write_addr(vpu, G2_REG_DMV_REF(i), 0);
+ hantro_write_addr(vpu, G2_REF_LUMA_ADDR(i), 0);
+ hantro_write_addr(vpu, G2_REF_CHROMA_ADDR(i), 0);
+ hantro_write_addr(vpu, G2_REF_MV_ADDR(i), 0);
}
hantro_reg_write(vpu, &g2_refer_lterm_e, dpb_longterm_e);
@@ -483,37 +476,28 @@ static int set_ref(struct hantro_ctx *ctx)
static void set_buffers(struct hantro_ctx *ctx)
{
- struct vb2_v4l2_buffer *src_buf, *dst_buf;
+ struct vb2_v4l2_buffer *src_buf;
struct hantro_dev *vpu = ctx->dev;
- const struct hantro_hevc_dec_ctrls *ctrls = &ctx->hevc_dec.ctrls;
- const struct v4l2_ctrl_hevc_sps *sps = ctrls->sps;
- size_t cr_offset = hantro_hevc_chroma_offset(sps);
- dma_addr_t src_dma, dst_dma;
+ dma_addr_t src_dma;
u32 src_len, src_buf_len;
src_buf = hantro_get_src_buf(ctx);
- dst_buf = hantro_get_dst_buf(ctx);
/* Source (stream) buffer. */
src_dma = vb2_dma_contig_plane_dma_addr(&src_buf->vb2_buf, 0);
src_len = vb2_get_plane_payload(&src_buf->vb2_buf, 0);
src_buf_len = vb2_plane_size(&src_buf->vb2_buf, 0);
- hantro_write_addr(vpu, G2_ADDR_STR, src_dma);
+ hantro_write_addr(vpu, G2_STREAM_ADDR, src_dma);
hantro_reg_write(vpu, &g2_stream_len, src_len);
hantro_reg_write(vpu, &g2_strm_buffer_len, src_buf_len);
hantro_reg_write(vpu, &g2_strm_start_offset, 0);
hantro_reg_write(vpu, &g2_write_mvs_e, 1);
- /* Destination (decoded frame) buffer. */
- dst_dma = hantro_get_dec_buf_addr(ctx, &dst_buf->vb2_buf);
-
- hantro_write_addr(vpu, G2_RASTER_SCAN, dst_dma);
- hantro_write_addr(vpu, G2_RASTER_SCAN_CHR, dst_dma + cr_offset);
- hantro_write_addr(vpu, G2_ADDR_TILE_SIZE, ctx->hevc_dec.tile_sizes.dma);
- hantro_write_addr(vpu, G2_TILE_FILTER, ctx->hevc_dec.tile_filter.dma);
- hantro_write_addr(vpu, G2_TILE_SAO, ctx->hevc_dec.tile_sao.dma);
- hantro_write_addr(vpu, G2_TILE_BSD, ctx->hevc_dec.tile_bsd.dma);
+ hantro_write_addr(vpu, G2_TILE_SIZES_ADDR, ctx->hevc_dec.tile_sizes.dma);
+ hantro_write_addr(vpu, G2_TILE_FILTER_ADDR, ctx->hevc_dec.tile_filter.dma);
+ hantro_write_addr(vpu, G2_TILE_SAO_ADDR, ctx->hevc_dec.tile_sao.dma);
+ hantro_write_addr(vpu, G2_TILE_BSD_ADDR, ctx->hevc_dec.tile_bsd.dma);
}
static void prepare_scaling_list_buffer(struct hantro_ctx *ctx)
@@ -563,24 +547,7 @@ static void prepare_scaling_list_buffer(struct hantro_ctx *ctx)
for (k = 0; k < 8; k++)
*p++ = sc->scaling_list_32x32[i][8 * k + j];
- hantro_write_addr(vpu, HEVC_SCALING_LIST, ctx->hevc_dec.scaling_lists.dma);
-}
-
-static void hantro_g2_check_idle(struct hantro_dev *vpu)
-{
- int i;
-
- for (i = 0; i < 3; i++) {
- u32 status;
-
- /* Make sure the VPU is idle */
- status = vdpu_read(vpu, G2_REG_INTERRUPT);
- if (status & G2_REG_INTERRUPT_DEC_E) {
- dev_warn(vpu->dev, "device still running, aborting");
- status |= G2_REG_INTERRUPT_DEC_ABORT_E | G2_REG_INTERRUPT_DEC_IRQ_DIS;
- vdpu_write(vpu, status, G2_REG_INTERRUPT);
- }
- }
+ hantro_write_addr(vpu, G2_HEVC_SCALING_LIST_ADDR, ctx->hevc_dec.scaling_lists.dma);
}
int hantro_g2_hevc_dec_run(struct hantro_ctx *ctx)
@@ -619,9 +586,6 @@ int hantro_g2_hevc_dec_run(struct hantro_ctx *ctx)
/* Don't compress buffers */
hantro_reg_write(vpu, &g2_ref_compress_bypass, 1);
- /* use NV12 as output format */
- hantro_reg_write(vpu, &g2_out_rs_e, 1);
-
/* Bus width and max burst */
hantro_reg_write(vpu, &g2_buswidth, BUS_WIDTH_128);
hantro_reg_write(vpu, &g2_max_burst, 16);
diff --git a/drivers/staging/media/hantro/hantro_g2_regs.h b/drivers/staging/media/hantro/hantro_g2_regs.h
index bb22fa9..b7c6f98 100644
--- a/drivers/staging/media/hantro/hantro_g2_regs.h
+++ b/drivers/staging/media/hantro/hantro_g2_regs.h
@@ -27,8 +27,22 @@
#define G2_REG_INTERRUPT_DEC_IRQ_DIS BIT(4)
#define G2_REG_INTERRUPT_DEC_E BIT(0)
+#define HEVC_DEC_MODE 0xc
+#define VP9_DEC_MODE 0xd
+
+#define BUS_WIDTH_32 0
+#define BUS_WIDTH_64 1
+#define BUS_WIDTH_128 2
+#define BUS_WIDTH_256 3
+
#define g2_strm_swap G2_DEC_REG(2, 28, 0xf)
+#define g2_strm_swap_old G2_DEC_REG(2, 27, 0x1f)
+#define g2_pic_swap G2_DEC_REG(2, 22, 0x1f)
#define g2_dirmv_swap G2_DEC_REG(2, 20, 0xf)
+#define g2_dirmv_swap_old G2_DEC_REG(2, 17, 0x1f)
+#define g2_tab0_swap_old G2_DEC_REG(2, 12, 0x1f)
+#define g2_tab1_swap_old G2_DEC_REG(2, 7, 0x1f)
+#define g2_tab2_swap_old G2_DEC_REG(2, 2, 0x1f)
#define g2_mode G2_DEC_REG(3, 27, 0x1f)
#define g2_compress_swap G2_DEC_REG(3, 20, 0xf)
@@ -37,11 +51,14 @@
#define g2_out_dis G2_DEC_REG(3, 15, 0x1)
#define g2_out_filtering_dis G2_DEC_REG(3, 14, 0x1)
#define g2_write_mvs_e G2_DEC_REG(3, 12, 0x1)
+#define g2_tab3_swap_old G2_DEC_REG(3, 7, 0x1f)
+#define g2_rscan_swap G2_DEC_REG(3, 2, 0x1f)
#define g2_pic_width_in_cbs G2_DEC_REG(4, 19, 0x1fff)
#define g2_pic_height_in_cbs G2_DEC_REG(4, 6, 0x1fff)
#define g2_num_ref_frames G2_DEC_REG(4, 0, 0x1f)
+#define g2_start_bit G2_DEC_REG(5, 25, 0x7f)
#define g2_scaling_list_e G2_DEC_REG(5, 24, 0x1)
#define g2_cb_qp_offset G2_DEC_REG(5, 19, 0x1f)
#define g2_cr_qp_offset G2_DEC_REG(5, 14, 0x1f)
@@ -49,6 +66,7 @@
#define g2_tempor_mvp_e G2_DEC_REG(5, 11, 0x1)
#define g2_max_cu_qpd_depth G2_DEC_REG(5, 5, 0x3f)
#define g2_cu_qpd_e G2_DEC_REG(5, 4, 0x1)
+#define g2_pix_shift G2_DEC_REG(5, 0, 0xf)
#define g2_stream_len G2_DEC_REG(6, 0, 0xffffffff)
@@ -71,24 +89,40 @@
#define g2_const_intra_e G2_DEC_REG(8, 31, 0x1)
#define g2_filt_ctrl_pres G2_DEC_REG(8, 30, 0x1)
+#define g2_bit_depth_y G2_DEC_REG(8, 21, 0xf)
+#define g2_bit_depth_c G2_DEC_REG(8, 17, 0xf)
#define g2_idr_pic_e G2_DEC_REG(8, 16, 0x1)
#define g2_bit_depth_pcm_y G2_DEC_REG(8, 12, 0xf)
#define g2_bit_depth_pcm_c G2_DEC_REG(8, 8, 0xf)
#define g2_bit_depth_y_minus8 G2_DEC_REG(8, 6, 0x3)
#define g2_bit_depth_c_minus8 G2_DEC_REG(8, 4, 0x3)
+#define g2_rs_out_bit_depth G2_DEC_REG(8, 4, 0xf)
#define g2_output_8_bits G2_DEC_REG(8, 3, 0x1)
+#define g2_output_format G2_DEC_REG(8, 0, 0x7)
+#define g2_pp_pix_shift G2_DEC_REG(8, 0, 0xf)
#define g2_refidx1_active G2_DEC_REG(9, 19, 0x1f)
#define g2_refidx0_active G2_DEC_REG(9, 14, 0x1f)
#define g2_hdr_skip_length G2_DEC_REG(9, 0, 0x3fff)
#define g2_start_code_e G2_DEC_REG(10, 31, 0x1)
+#define g2_init_qp_old G2_DEC_REG(10, 25, 0x3f)
#define g2_init_qp G2_DEC_REG(10, 24, 0x3f)
+#define g2_num_tile_cols_old G2_DEC_REG(10, 20, 0x1f)
#define g2_num_tile_cols G2_DEC_REG(10, 19, 0x1f)
+#define g2_num_tile_rows_old G2_DEC_REG(10, 15, 0x1f)
#define g2_num_tile_rows G2_DEC_REG(10, 14, 0x1f)
#define g2_tile_e G2_DEC_REG(10, 1, 0x1)
#define g2_entropy_sync_e G2_DEC_REG(10, 0, 0x1)
+#define vp9_transform_mode G2_DEC_REG(11, 27, 0x7)
+#define vp9_filt_sharpness G2_DEC_REG(11, 21, 0x7)
+#define vp9_mcomp_filt_type G2_DEC_REG(11, 8, 0x7)
+#define vp9_high_prec_mv_e G2_DEC_REG(11, 7, 0x1)
+#define vp9_comp_pred_mode G2_DEC_REG(11, 4, 0x3)
+#define vp9_gref_sign_bias G2_DEC_REG(11, 2, 0x1)
+#define vp9_aref_sign_bias G2_DEC_REG(11, 0, 0x1)
+
#define g2_refer_lterm_e G2_DEC_REG(12, 16, 0xffff)
#define g2_min_cb_size G2_DEC_REG(12, 13, 0x7)
#define g2_max_cb_size G2_DEC_REG(12, 10, 0x7)
@@ -147,6 +181,50 @@
#define g2_partial_ctb_y G2_DEC_REG(20, 30, 0x1)
#define g2_pic_width_4x4 G2_DEC_REG(20, 16, 0xfff)
#define g2_pic_height_4x4 G2_DEC_REG(20, 0, 0xfff)
+
+#define vp9_qp_delta_y_dc G2_DEC_REG(13, 23, 0x3f)
+#define vp9_qp_delta_ch_dc G2_DEC_REG(13, 17, 0x3f)
+#define vp9_qp_delta_ch_ac G2_DEC_REG(13, 11, 0x3f)
+#define vp9_last_sign_bias G2_DEC_REG(13, 10, 0x1)
+#define vp9_lossless_e G2_DEC_REG(13, 9, 0x1)
+#define vp9_comp_pred_var_ref1 G2_DEC_REG(13, 7, 0x3)
+#define vp9_comp_pred_var_ref0 G2_DEC_REG(13, 5, 0x3)
+#define vp9_comp_pred_fixed_ref G2_DEC_REG(13, 3, 0x3)
+#define vp9_segment_temp_upd_e G2_DEC_REG(13, 2, 0x1)
+#define vp9_segment_upd_e G2_DEC_REG(13, 1, 0x1)
+#define vp9_segment_e G2_DEC_REG(13, 0, 0x1)
+
+#define vp9_filt_level G2_DEC_REG(14, 18, 0x3f)
+#define vp9_refpic_seg0 G2_DEC_REG(14, 15, 0x7)
+#define vp9_skip_seg0 G2_DEC_REG(14, 14, 0x1)
+#define vp9_filt_level_seg0 G2_DEC_REG(14, 8, 0x3f)
+#define vp9_quant_seg0 G2_DEC_REG(14, 0, 0xff)
+
+#define vp9_refpic_seg1 G2_DEC_REG(15, 15, 0x7)
+#define vp9_skip_seg1 G2_DEC_REG(15, 14, 0x1)
+#define vp9_filt_level_seg1 G2_DEC_REG(15, 8, 0x3f)
+#define vp9_quant_seg1 G2_DEC_REG(15, 0, 0xff)
+
+#define vp9_refpic_seg2 G2_DEC_REG(16, 15, 0x7)
+#define vp9_skip_seg2 G2_DEC_REG(16, 14, 0x1)
+#define vp9_filt_level_seg2 G2_DEC_REG(16, 8, 0x3f)
+#define vp9_quant_seg2 G2_DEC_REG(16, 0, 0xff)
+
+#define vp9_refpic_seg3 G2_DEC_REG(17, 15, 0x7)
+#define vp9_skip_seg3 G2_DEC_REG(17, 14, 0x1)
+#define vp9_filt_level_seg3 G2_DEC_REG(17, 8, 0x3f)
+#define vp9_quant_seg3 G2_DEC_REG(17, 0, 0xff)
+
+#define vp9_refpic_seg4 G2_DEC_REG(18, 15, 0x7)
+#define vp9_skip_seg4 G2_DEC_REG(18, 14, 0x1)
+#define vp9_filt_level_seg4 G2_DEC_REG(18, 8, 0x3f)
+#define vp9_quant_seg4 G2_DEC_REG(18, 0, 0xff)
+
+#define vp9_refpic_seg5 G2_DEC_REG(19, 15, 0x7)
+#define vp9_skip_seg5 G2_DEC_REG(19, 14, 0x1)
+#define vp9_filt_level_seg5 G2_DEC_REG(19, 8, 0x3f)
+#define vp9_quant_seg5 G2_DEC_REG(19, 0, 0xff)
+
#define hevc_cur_poc_00 G2_DEC_REG(46, 24, 0xff)
#define hevc_cur_poc_01 G2_DEC_REG(46, 16, 0xff)
#define hevc_cur_poc_02 G2_DEC_REG(46, 8, 0xff)
@@ -167,9 +245,48 @@
#define hevc_cur_poc_14 G2_DEC_REG(49, 8, 0xff)
#define hevc_cur_poc_15 G2_DEC_REG(49, 0, 0xff)
+#define vp9_refpic_seg6 G2_DEC_REG(31, 15, 0x7)
+#define vp9_skip_seg6 G2_DEC_REG(31, 14, 0x1)
+#define vp9_filt_level_seg6 G2_DEC_REG(31, 8, 0x3f)
+#define vp9_quant_seg6 G2_DEC_REG(31, 0, 0xff)
+
+#define vp9_refpic_seg7 G2_DEC_REG(32, 15, 0x7)
+#define vp9_skip_seg7 G2_DEC_REG(32, 14, 0x1)
+#define vp9_filt_level_seg7 G2_DEC_REG(32, 8, 0x3f)
+#define vp9_quant_seg7 G2_DEC_REG(32, 0, 0xff)
+
+#define vp9_lref_width G2_DEC_REG(33, 16, 0xffff)
+#define vp9_lref_height G2_DEC_REG(33, 0, 0xffff)
+
+#define vp9_gref_width G2_DEC_REG(34, 16, 0xffff)
+#define vp9_gref_height G2_DEC_REG(34, 0, 0xffff)
+
+#define vp9_aref_width G2_DEC_REG(35, 16, 0xffff)
+#define vp9_aref_height G2_DEC_REG(35, 0, 0xffff)
+
+#define vp9_lref_hor_scale G2_DEC_REG(36, 16, 0xffff)
+#define vp9_lref_ver_scale G2_DEC_REG(36, 0, 0xffff)
+
+#define vp9_gref_hor_scale G2_DEC_REG(37, 16, 0xffff)
+#define vp9_gref_ver_scale G2_DEC_REG(37, 0, 0xffff)
+
+#define vp9_aref_hor_scale G2_DEC_REG(38, 16, 0xffff)
+#define vp9_aref_ver_scale G2_DEC_REG(38, 0, 0xffff)
+
+#define vp9_filt_ref_adj_0 G2_DEC_REG(46, 24, 0x7f)
+#define vp9_filt_ref_adj_1 G2_DEC_REG(46, 16, 0x7f)
+#define vp9_filt_ref_adj_2 G2_DEC_REG(46, 8, 0x7f)
+#define vp9_filt_ref_adj_3 G2_DEC_REG(46, 0, 0x7f)
+
+#define vp9_filt_mb_adj_0 G2_DEC_REG(47, 24, 0x7f)
+#define vp9_filt_mb_adj_1 G2_DEC_REG(47, 16, 0x7f)
+#define vp9_filt_mb_adj_2 G2_DEC_REG(47, 8, 0x7f)
+#define vp9_filt_mb_adj_3 G2_DEC_REG(47, 0, 0x7f)
+
#define g2_apf_threshold G2_DEC_REG(55, 0, 0xffff)
#define g2_clk_gate_e G2_DEC_REG(58, 16, 0x1)
+#define g2_double_buffer_e G2_DEC_REG(58, 15, 0x1)
#define g2_buswidth G2_DEC_REG(58, 8, 0x7)
#define g2_max_burst G2_DEC_REG(58, 0, 0xff)
@@ -177,20 +294,24 @@
#define G2_REG_CONFIG_DEC_CLK_GATE_E BIT(16)
#define G2_REG_CONFIG_DEC_CLK_GATE_IDLE_E BIT(17)
-#define G2_ADDR_DST (G2_SWREG(65))
-#define G2_REG_ADDR_REF(i) (G2_SWREG(67) + ((i) * 0x8))
-#define G2_ADDR_DST_CHR (G2_SWREG(99))
-#define G2_REG_CHR_REF(i) (G2_SWREG(101) + ((i) * 0x8))
-#define G2_ADDR_DST_MV (G2_SWREG(133))
-#define G2_REG_DMV_REF(i) (G2_SWREG(135) + ((i) * 0x8))
-#define G2_ADDR_TILE_SIZE (G2_SWREG(167))
-#define G2_ADDR_STR (G2_SWREG(169))
-#define HEVC_SCALING_LIST (G2_SWREG(171))
-#define G2_RASTER_SCAN (G2_SWREG(175))
-#define G2_RASTER_SCAN_CHR (G2_SWREG(177))
-#define G2_TILE_FILTER (G2_SWREG(179))
-#define G2_TILE_SAO (G2_SWREG(181))
-#define G2_TILE_BSD (G2_SWREG(183))
+#define G2_OUT_LUMA_ADDR (G2_SWREG(65))
+#define G2_REF_LUMA_ADDR(i) (G2_SWREG(67) + ((i) * 0x8))
+#define G2_VP9_SEGMENT_WRITE_ADDR (G2_SWREG(79))
+#define G2_VP9_SEGMENT_READ_ADDR (G2_SWREG(81))
+#define G2_OUT_CHROMA_ADDR (G2_SWREG(99))
+#define G2_REF_CHROMA_ADDR(i) (G2_SWREG(101) + ((i) * 0x8))
+#define G2_OUT_MV_ADDR (G2_SWREG(133))
+#define G2_REF_MV_ADDR(i) (G2_SWREG(135) + ((i) * 0x8))
+#define G2_TILE_SIZES_ADDR (G2_SWREG(167))
+#define G2_STREAM_ADDR (G2_SWREG(169))
+#define G2_HEVC_SCALING_LIST_ADDR (G2_SWREG(171))
+#define G2_VP9_CTX_COUNT_ADDR (G2_SWREG(171))
+#define G2_VP9_PROBS_ADDR (G2_SWREG(173))
+#define G2_RS_OUT_LUMA_ADDR (G2_SWREG(175))
+#define G2_RS_OUT_CHROMA_ADDR (G2_SWREG(177))
+#define G2_TILE_FILTER_ADDR (G2_SWREG(179))
+#define G2_TILE_SAO_ADDR (G2_SWREG(181))
+#define G2_TILE_BSD_ADDR (G2_SWREG(183))
#define g2_strm_buffer_len G2_DEC_REG(258, 0, 0xffffffff)
#define g2_strm_start_offset G2_DEC_REG(259, 0, 0xffffffff)
diff --git a/drivers/staging/media/hantro/hantro_g2_vp9_dec.c b/drivers/staging/media/hantro/hantro_g2_vp9_dec.c
new file mode 100644
index 0000000..91c21b6
--- /dev/null
+++ b/drivers/staging/media/hantro/hantro_g2_vp9_dec.c
@@ -0,0 +1,1022 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Hantro VP9 codec driver
+ *
+ * Copyright (C) 2021 Collabora Ltd.
+ */
+#include "media/videobuf2-core.h"
+#include "media/videobuf2-dma-contig.h"
+#include "media/videobuf2-v4l2.h"
+#include <linux/kernel.h>
+#include <linux/vmalloc.h>
+#include <media/v4l2-mem2mem.h>
+#include <media/v4l2-vp9.h>
+
+#include "hantro.h"
+#include "hantro_vp9.h"
+#include "hantro_g2_regs.h"
+
+#define G2_ALIGN 16
+
+enum hantro_ref_frames {
+ INTRA_FRAME = 0,
+ LAST_FRAME = 1,
+ GOLDEN_FRAME = 2,
+ ALTREF_FRAME = 3,
+ MAX_REF_FRAMES = 4
+};
+
+static int start_prepare_run(struct hantro_ctx *ctx, const struct v4l2_ctrl_vp9_frame **dec_params)
+{
+ const struct v4l2_ctrl_vp9_compressed_hdr *prob_updates;
+ struct hantro_vp9_dec_hw_ctx *vp9_ctx = &ctx->vp9_dec;
+ struct v4l2_ctrl *ctrl;
+ unsigned int fctx_idx;
+
+ /* v4l2-specific stuff */
+ hantro_start_prepare_run(ctx);
+
+ ctrl = v4l2_ctrl_find(&ctx->ctrl_handler, V4L2_CID_STATELESS_VP9_FRAME);
+ if (WARN_ON(!ctrl))
+ return -EINVAL;
+ *dec_params = ctrl->p_cur.p;
+
+ ctrl = v4l2_ctrl_find(&ctx->ctrl_handler, V4L2_CID_STATELESS_VP9_COMPRESSED_HDR);
+ if (WARN_ON(!ctrl))
+ return -EINVAL;
+ prob_updates = ctrl->p_cur.p;
+ vp9_ctx->cur.tx_mode = prob_updates->tx_mode;
+
+ /*
+ * vp9 stuff
+ *
+ * by this point the userspace has done all parts of 6.2 uncompressed_header()
+ * except this fragment:
+ * if ( FrameIsIntra || error_resilient_mode ) {
+ * setup_past_independence ( )
+ * if ( frame_type == KEY_FRAME || error_resilient_mode == 1 ||
+ * reset_frame_context == 3 ) {
+ * for ( i = 0; i < 4; i ++ ) {
+ * save_probs( i )
+ * }
+ * } else if ( reset_frame_context == 2 ) {
+ * save_probs( frame_context_idx )
+ * }
+ * frame_context_idx = 0
+ * }
+ */
+ fctx_idx = v4l2_vp9_reset_frame_ctx(*dec_params, vp9_ctx->frame_context);
+ vp9_ctx->cur.frame_context_idx = fctx_idx;
+
+ /* 6.1 frame(sz): load_probs() and load_probs2() */
+ vp9_ctx->probability_tables = vp9_ctx->frame_context[fctx_idx];
+
+ /*
+ * The userspace has also performed 6.3 compressed_header(), but handling the
+ * probs in a special way. All probs which need updating, except MV-related,
+ * have been read from the bitstream and translated through inv_map_table[],
+ * but no 6.3.6 inv_recenter_nonneg(v, m) has been performed. The values passed
+ * by userspace are either translated values (there are no 0 values in
+ * inv_map_table[]), or zero to indicate no update. All MV-related probs which need
+ * updating have been read from the bitstream and (mv_prob << 1) | 1 has been
+ * performed. The values passed by userspace are either new values
+ * to replace old ones (the above mentioned shift and bitwise or never result in
+ * a zero) or zero to indicate no update.
+ * fw_update_probs() performs actual probs updates or leaves probs as-is
+ * for values for which a zero was passed from userspace.
+ */
+ v4l2_vp9_fw_update_probs(&vp9_ctx->probability_tables, prob_updates, *dec_params);
+
+ return 0;
+}
+
+static size_t chroma_offset(const struct hantro_ctx *ctx,
+ const struct v4l2_ctrl_vp9_frame *dec_params)
+{
+ int bytes_per_pixel = dec_params->bit_depth == 8 ? 1 : 2;
+
+ return ctx->src_fmt.width * ctx->src_fmt.height * bytes_per_pixel;
+}
+
+static size_t mv_offset(const struct hantro_ctx *ctx,
+ const struct v4l2_ctrl_vp9_frame *dec_params)
+{
+ size_t cr_offset = chroma_offset(ctx, dec_params);
+
+ return ALIGN((cr_offset * 3) / 2, G2_ALIGN);
+}
+
+static struct hantro_decoded_buffer *
+get_ref_buf(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *dst, u64 timestamp)
+{
+ struct v4l2_m2m_ctx *m2m_ctx = ctx->fh.m2m_ctx;
+ struct vb2_queue *cap_q = &m2m_ctx->cap_q_ctx.q;
+ int buf_idx;
+
+ /*
+ * If a ref is unused or invalid, address of current destination
+ * buffer is returned.
+ */
+ buf_idx = vb2_find_timestamp(cap_q, timestamp, 0);
+ if (buf_idx < 0)
+ return vb2_to_hantro_decoded_buf(&dst->vb2_buf);
+
+ return vb2_to_hantro_decoded_buf(vb2_get_buffer(cap_q, buf_idx));
+}
+
+static void update_dec_buf_info(struct hantro_decoded_buffer *buf,
+ const struct v4l2_ctrl_vp9_frame *dec_params)
+{
+ buf->vp9.width = dec_params->frame_width_minus_1 + 1;
+ buf->vp9.height = dec_params->frame_height_minus_1 + 1;
+ buf->vp9.bit_depth = dec_params->bit_depth;
+}
+
+static void update_ctx_cur_info(struct hantro_vp9_dec_hw_ctx *vp9_ctx,
+ struct hantro_decoded_buffer *buf,
+ const struct v4l2_ctrl_vp9_frame *dec_params)
+{
+ vp9_ctx->cur.valid = true;
+ vp9_ctx->cur.reference_mode = dec_params->reference_mode;
+ vp9_ctx->cur.interpolation_filter = dec_params->interpolation_filter;
+ vp9_ctx->cur.flags = dec_params->flags;
+ vp9_ctx->cur.timestamp = buf->base.vb.vb2_buf.timestamp;
+}
+
+static void config_output(struct hantro_ctx *ctx,
+ struct hantro_decoded_buffer *dst,
+ const struct v4l2_ctrl_vp9_frame *dec_params)
+{
+ dma_addr_t luma_addr, chroma_addr, mv_addr;
+
+ hantro_reg_write(ctx->dev, &g2_out_dis, 0);
+ if (!ctx->dev->variant->legacy_regs)
+ hantro_reg_write(ctx->dev, &g2_output_format, 0);
+
+ luma_addr = hantro_get_dec_buf_addr(ctx, &dst->base.vb.vb2_buf);
+ hantro_write_addr(ctx->dev, G2_OUT_LUMA_ADDR, luma_addr);
+
+ chroma_addr = luma_addr + chroma_offset(ctx, dec_params);
+ hantro_write_addr(ctx->dev, G2_OUT_CHROMA_ADDR, chroma_addr);
+
+ mv_addr = luma_addr + mv_offset(ctx, dec_params);
+ hantro_write_addr(ctx->dev, G2_OUT_MV_ADDR, mv_addr);
+}
+
+struct hantro_vp9_ref_reg {
+ const struct hantro_reg width;
+ const struct hantro_reg height;
+ const struct hantro_reg hor_scale;
+ const struct hantro_reg ver_scale;
+ u32 y_base;
+ u32 c_base;
+};
+
+static void config_ref(struct hantro_ctx *ctx,
+ struct hantro_decoded_buffer *dst,
+ const struct hantro_vp9_ref_reg *ref_reg,
+ const struct v4l2_ctrl_vp9_frame *dec_params,
+ u64 ref_ts)
+{
+ struct hantro_decoded_buffer *buf;
+ dma_addr_t luma_addr, chroma_addr;
+ u32 refw, refh;
+
+ buf = get_ref_buf(ctx, &dst->base.vb, ref_ts);
+ refw = buf->vp9.width;
+ refh = buf->vp9.height;
+
+ hantro_reg_write(ctx->dev, &ref_reg->width, refw);
+ hantro_reg_write(ctx->dev, &ref_reg->height, refh);
+
+ hantro_reg_write(ctx->dev, &ref_reg->hor_scale, (refw << 14) / dst->vp9.width);
+ hantro_reg_write(ctx->dev, &ref_reg->ver_scale, (refh << 14) / dst->vp9.height);
+
+ luma_addr = hantro_get_dec_buf_addr(ctx, &buf->base.vb.vb2_buf);
+ hantro_write_addr(ctx->dev, ref_reg->y_base, luma_addr);
+
+ chroma_addr = luma_addr + chroma_offset(ctx, dec_params);
+ hantro_write_addr(ctx->dev, ref_reg->c_base, chroma_addr);
+}
+
+static void config_ref_registers(struct hantro_ctx *ctx,
+ const struct v4l2_ctrl_vp9_frame *dec_params,
+ struct hantro_decoded_buffer *dst,
+ struct hantro_decoded_buffer *mv_ref)
+{
+ static const struct hantro_vp9_ref_reg ref_regs[] = {
+ {
+ /* Last */
+ .width = vp9_lref_width,
+ .height = vp9_lref_height,
+ .hor_scale = vp9_lref_hor_scale,
+ .ver_scale = vp9_lref_ver_scale,
+ .y_base = G2_REF_LUMA_ADDR(0),
+ .c_base = G2_REF_CHROMA_ADDR(0),
+ }, {
+ /* Golden */
+ .width = vp9_gref_width,
+ .height = vp9_gref_height,
+ .hor_scale = vp9_gref_hor_scale,
+ .ver_scale = vp9_gref_ver_scale,
+ .y_base = G2_REF_LUMA_ADDR(4),
+ .c_base = G2_REF_CHROMA_ADDR(4),
+ }, {
+ /* Altref */
+ .width = vp9_aref_width,
+ .height = vp9_aref_height,
+ .hor_scale = vp9_aref_hor_scale,
+ .ver_scale = vp9_aref_ver_scale,
+ .y_base = G2_REF_LUMA_ADDR(5),
+ .c_base = G2_REF_CHROMA_ADDR(5),
+ },
+ };
+ dma_addr_t mv_addr;
+
+ config_ref(ctx, dst, &ref_regs[0], dec_params, dec_params->last_frame_ts);
+ config_ref(ctx, dst, &ref_regs[1], dec_params, dec_params->golden_frame_ts);
+ config_ref(ctx, dst, &ref_regs[2], dec_params, dec_params->alt_frame_ts);
+
+ mv_addr = hantro_get_dec_buf_addr(ctx, &mv_ref->base.vb.vb2_buf) +
+ mv_offset(ctx, dec_params);
+ hantro_write_addr(ctx->dev, G2_REF_MV_ADDR(0), mv_addr);
+
+ hantro_reg_write(ctx->dev, &vp9_last_sign_bias,
+ dec_params->ref_frame_sign_bias & V4L2_VP9_SIGN_BIAS_LAST ? 1 : 0);
+
+ hantro_reg_write(ctx->dev, &vp9_gref_sign_bias,
+ dec_params->ref_frame_sign_bias & V4L2_VP9_SIGN_BIAS_GOLDEN ? 1 : 0);
+
+ hantro_reg_write(ctx->dev, &vp9_aref_sign_bias,
+ dec_params->ref_frame_sign_bias & V4L2_VP9_SIGN_BIAS_ALT ? 1 : 0);
+}
+
+static void recompute_tile_info(unsigned short *tile_info, unsigned int tiles, unsigned int sbs)
+{
+ int i;
+ unsigned int accumulated = 0;
+ unsigned int next_accumulated;
+
+ for (i = 1; i <= tiles; ++i) {
+ next_accumulated = i * sbs / tiles;
+ *tile_info++ = next_accumulated - accumulated;
+ accumulated = next_accumulated;
+ }
+}
+
+static void
+recompute_tile_rc_info(struct hantro_ctx *ctx,
+ unsigned int tile_r, unsigned int tile_c,
+ unsigned int sbs_r, unsigned int sbs_c)
+{
+ struct hantro_vp9_dec_hw_ctx *vp9_ctx = &ctx->vp9_dec;
+
+ recompute_tile_info(vp9_ctx->tile_r_info, tile_r, sbs_r);
+ recompute_tile_info(vp9_ctx->tile_c_info, tile_c, sbs_c);
+
+ vp9_ctx->last_tile_r = tile_r;
+ vp9_ctx->last_tile_c = tile_c;
+ vp9_ctx->last_sbs_r = sbs_r;
+ vp9_ctx->last_sbs_c = sbs_c;
+}
+
+static inline unsigned int first_tile_row(unsigned int tile_r, unsigned int sbs_r)
+{
+ if (tile_r == sbs_r + 1)
+ return 1;
+
+ if (tile_r == sbs_r + 2)
+ return 2;
+
+ return 0;
+}
+
+static void
+fill_tile_info(struct hantro_ctx *ctx,
+ unsigned int tile_r, unsigned int tile_c,
+ unsigned int sbs_r, unsigned int sbs_c,
+ unsigned short *tile_mem)
+{
+ struct hantro_vp9_dec_hw_ctx *vp9_ctx = &ctx->vp9_dec;
+ unsigned int i, j;
+ bool first = true;
+
+ for (i = first_tile_row(tile_r, sbs_r); i < tile_r; ++i) {
+ unsigned short r_info = vp9_ctx->tile_r_info[i];
+
+ if (first) {
+ if (i > 0)
+ r_info += vp9_ctx->tile_r_info[0];
+ if (i == 2)
+ r_info += vp9_ctx->tile_r_info[1];
+ first = false;
+ }
+ for (j = 0; j < tile_c; ++j) {
+ *tile_mem++ = vp9_ctx->tile_c_info[j];
+ *tile_mem++ = r_info;
+ }
+ }
+}
+
+static void
+config_tiles(struct hantro_ctx *ctx,
+ const struct v4l2_ctrl_vp9_frame *dec_params,
+ struct hantro_decoded_buffer *dst)
+{
+ struct hantro_vp9_dec_hw_ctx *vp9_ctx = &ctx->vp9_dec;
+ struct hantro_aux_buf *misc = &vp9_ctx->misc;
+ struct hantro_aux_buf *tile_edge = &vp9_ctx->tile_edge;
+ dma_addr_t addr;
+ unsigned short *tile_mem;
+ unsigned int rows, cols;
+
+ addr = misc->dma + vp9_ctx->tile_info_offset;
+ hantro_write_addr(ctx->dev, G2_TILE_SIZES_ADDR, addr);
+
+ tile_mem = misc->cpu + vp9_ctx->tile_info_offset;
+ if (dec_params->tile_cols_log2 || dec_params->tile_rows_log2) {
+ unsigned int tile_r = (1 << dec_params->tile_rows_log2);
+ unsigned int tile_c = (1 << dec_params->tile_cols_log2);
+ unsigned int sbs_r = hantro_vp9_num_sbs(dst->vp9.height);
+ unsigned int sbs_c = hantro_vp9_num_sbs(dst->vp9.width);
+
+ if (tile_r != vp9_ctx->last_tile_r || tile_c != vp9_ctx->last_tile_c ||
+ sbs_r != vp9_ctx->last_sbs_r || sbs_c != vp9_ctx->last_sbs_c)
+ recompute_tile_rc_info(ctx, tile_r, tile_c, sbs_r, sbs_c);
+
+ fill_tile_info(ctx, tile_r, tile_c, sbs_r, sbs_c, tile_mem);
+
+ cols = tile_c;
+ rows = tile_r;
+ hantro_reg_write(ctx->dev, &g2_tile_e, 1);
+ } else {
+ tile_mem[0] = hantro_vp9_num_sbs(dst->vp9.width);
+ tile_mem[1] = hantro_vp9_num_sbs(dst->vp9.height);
+
+ cols = 1;
+ rows = 1;
+ hantro_reg_write(ctx->dev, &g2_tile_e, 0);
+ }
+
+ if (ctx->dev->variant->legacy_regs) {
+ hantro_reg_write(ctx->dev, &g2_num_tile_cols_old, cols);
+ hantro_reg_write(ctx->dev, &g2_num_tile_rows_old, rows);
+ } else {
+ hantro_reg_write(ctx->dev, &g2_num_tile_cols, cols);
+ hantro_reg_write(ctx->dev, &g2_num_tile_rows, rows);
+ }
+
+ /* provide aux buffers even if no tiles are used */
+ addr = tile_edge->dma;
+ hantro_write_addr(ctx->dev, G2_TILE_FILTER_ADDR, addr);
+
+ addr = tile_edge->dma + vp9_ctx->bsd_ctrl_offset;
+ hantro_write_addr(ctx->dev, G2_TILE_BSD_ADDR, addr);
+}
+
+static void
+update_feat_and_flag(struct hantro_vp9_dec_hw_ctx *vp9_ctx,
+ const struct v4l2_vp9_segmentation *seg,
+ unsigned int feature,
+ unsigned int segid)
+{
+ u8 mask = V4L2_VP9_SEGMENT_FEATURE_ENABLED(feature);
+
+ vp9_ctx->feature_data[segid][feature] = seg->feature_data[segid][feature];
+ vp9_ctx->feature_enabled[segid] &= ~mask;
+ vp9_ctx->feature_enabled[segid] |= (seg->feature_enabled[segid] & mask);
+}
+
+static inline s16 clip3(s16 x, s16 y, s16 z)
+{
+ return (z < x) ? x : (z > y) ? y : z;
+}
+
+static s16 feat_val_clip3(s16 feat_val, s16 feature_data, bool absolute, u8 clip)
+{
+ if (absolute)
+ return feature_data;
+
+ return clip3(0, 255, feat_val + feature_data);
+}
+
+static void config_segment(struct hantro_ctx *ctx, const struct v4l2_ctrl_vp9_frame *dec_params)
+{
+ struct hantro_vp9_dec_hw_ctx *vp9_ctx = &ctx->vp9_dec;
+ const struct v4l2_vp9_segmentation *seg;
+ s16 feat_val;
+ unsigned char feat_id;
+ unsigned int segid;
+ bool segment_enabled, absolute, update_data;
+
+ static const struct hantro_reg seg_regs[8][V4L2_VP9_SEG_LVL_MAX] = {
+ { vp9_quant_seg0, vp9_filt_level_seg0, vp9_refpic_seg0, vp9_skip_seg0 },
+ { vp9_quant_seg1, vp9_filt_level_seg1, vp9_refpic_seg1, vp9_skip_seg1 },
+ { vp9_quant_seg2, vp9_filt_level_seg2, vp9_refpic_seg2, vp9_skip_seg2 },
+ { vp9_quant_seg3, vp9_filt_level_seg3, vp9_refpic_seg3, vp9_skip_seg3 },
+ { vp9_quant_seg4, vp9_filt_level_seg4, vp9_refpic_seg4, vp9_skip_seg4 },
+ { vp9_quant_seg5, vp9_filt_level_seg5, vp9_refpic_seg5, vp9_skip_seg5 },
+ { vp9_quant_seg6, vp9_filt_level_seg6, vp9_refpic_seg6, vp9_skip_seg6 },
+ { vp9_quant_seg7, vp9_filt_level_seg7, vp9_refpic_seg7, vp9_skip_seg7 },
+ };
+
+ segment_enabled = !!(dec_params->seg.flags & V4L2_VP9_SEGMENTATION_FLAG_ENABLED);
+ hantro_reg_write(ctx->dev, &vp9_segment_e, segment_enabled);
+ hantro_reg_write(ctx->dev, &vp9_segment_upd_e,
+ !!(dec_params->seg.flags & V4L2_VP9_SEGMENTATION_FLAG_UPDATE_MAP));
+ hantro_reg_write(ctx->dev, &vp9_segment_temp_upd_e,
+ !!(dec_params->seg.flags & V4L2_VP9_SEGMENTATION_FLAG_TEMPORAL_UPDATE));
+
+ seg = &dec_params->seg;
+ absolute = !!(seg->flags & V4L2_VP9_SEGMENTATION_FLAG_ABS_OR_DELTA_UPDATE);
+ update_data = !!(seg->flags & V4L2_VP9_SEGMENTATION_FLAG_UPDATE_DATA);
+
+ for (segid = 0; segid < 8; ++segid) {
+ /* Quantizer segment feature */
+ feat_id = V4L2_VP9_SEG_LVL_ALT_Q;
+ feat_val = dec_params->quant.base_q_idx;
+ if (segment_enabled) {
+ if (update_data)
+ update_feat_and_flag(vp9_ctx, seg, feat_id, segid);
+ if (v4l2_vp9_seg_feat_enabled(vp9_ctx->feature_enabled, feat_id, segid))
+ feat_val = feat_val_clip3(feat_val,
+ vp9_ctx->feature_data[segid][feat_id],
+ absolute, 255);
+ }
+ hantro_reg_write(ctx->dev, &seg_regs[segid][feat_id], feat_val);
+
+ /* Loop filter segment feature */
+ feat_id = V4L2_VP9_SEG_LVL_ALT_L;
+ feat_val = dec_params->lf.level;
+ if (segment_enabled) {
+ if (update_data)
+ update_feat_and_flag(vp9_ctx, seg, feat_id, segid);
+ if (v4l2_vp9_seg_feat_enabled(vp9_ctx->feature_enabled, feat_id, segid))
+ feat_val = feat_val_clip3(feat_val,
+ vp9_ctx->feature_data[segid][feat_id],
+ absolute, 63);
+ }
+ hantro_reg_write(ctx->dev, &seg_regs[segid][feat_id], feat_val);
+
+ /* Reference frame segment feature */
+ feat_id = V4L2_VP9_SEG_LVL_REF_FRAME;
+ feat_val = 0;
+ if (segment_enabled) {
+ if (update_data)
+ update_feat_and_flag(vp9_ctx, seg, feat_id, segid);
+ if (!(dec_params->flags & V4L2_VP9_FRAME_FLAG_KEY_FRAME) &&
+ v4l2_vp9_seg_feat_enabled(vp9_ctx->feature_enabled, feat_id, segid))
+ feat_val = vp9_ctx->feature_data[segid][feat_id] + 1;
+ }
+ hantro_reg_write(ctx->dev, &seg_regs[segid][feat_id], feat_val);
+
+ /* Skip segment feature */
+ feat_id = V4L2_VP9_SEG_LVL_SKIP;
+ feat_val = 0;
+ if (segment_enabled) {
+ if (update_data)
+ update_feat_and_flag(vp9_ctx, seg, feat_id, segid);
+ feat_val = v4l2_vp9_seg_feat_enabled(vp9_ctx->feature_enabled,
+ feat_id, segid) ? 1 : 0;
+ }
+ hantro_reg_write(ctx->dev, &seg_regs[segid][feat_id], feat_val);
+ }
+}
+
+static void config_loop_filter(struct hantro_ctx *ctx, const struct v4l2_ctrl_vp9_frame *dec_params)
+{
+ bool d = dec_params->lf.flags & V4L2_VP9_LOOP_FILTER_FLAG_DELTA_ENABLED;
+
+ hantro_reg_write(ctx->dev, &vp9_filt_level, dec_params->lf.level);
+ hantro_reg_write(ctx->dev, &g2_out_filtering_dis, dec_params->lf.level == 0);
+ hantro_reg_write(ctx->dev, &vp9_filt_sharpness, dec_params->lf.sharpness);
+
+ hantro_reg_write(ctx->dev, &vp9_filt_ref_adj_0, d ? dec_params->lf.ref_deltas[0] : 0);
+ hantro_reg_write(ctx->dev, &vp9_filt_ref_adj_1, d ? dec_params->lf.ref_deltas[1] : 0);
+ hantro_reg_write(ctx->dev, &vp9_filt_ref_adj_2, d ? dec_params->lf.ref_deltas[2] : 0);
+ hantro_reg_write(ctx->dev, &vp9_filt_ref_adj_3, d ? dec_params->lf.ref_deltas[3] : 0);
+ hantro_reg_write(ctx->dev, &vp9_filt_mb_adj_0, d ? dec_params->lf.mode_deltas[0] : 0);
+ hantro_reg_write(ctx->dev, &vp9_filt_mb_adj_1, d ? dec_params->lf.mode_deltas[1] : 0);
+}
+
+static void config_picture_dimensions(struct hantro_ctx *ctx, struct hantro_decoded_buffer *dst)
+{
+ u32 pic_w_4x4, pic_h_4x4;
+
+ hantro_reg_write(ctx->dev, &g2_pic_width_in_cbs, (dst->vp9.width + 7) / 8);
+ hantro_reg_write(ctx->dev, &g2_pic_height_in_cbs, (dst->vp9.height + 7) / 8);
+ pic_w_4x4 = roundup(dst->vp9.width, 8) >> 2;
+ pic_h_4x4 = roundup(dst->vp9.height, 8) >> 2;
+ hantro_reg_write(ctx->dev, &g2_pic_width_4x4, pic_w_4x4);
+ hantro_reg_write(ctx->dev, &g2_pic_height_4x4, pic_h_4x4);
+}
+
+static void
+config_bit_depth(struct hantro_ctx *ctx, const struct v4l2_ctrl_vp9_frame *dec_params)
+{
+ if (ctx->dev->variant->legacy_regs) {
+ u8 pp_shift = 0;
+
+ hantro_reg_write(ctx->dev, &g2_bit_depth_y, dec_params->bit_depth);
+ hantro_reg_write(ctx->dev, &g2_bit_depth_c, dec_params->bit_depth);
+ hantro_reg_write(ctx->dev, &g2_rs_out_bit_depth, dec_params->bit_depth);
+
+ if (dec_params->bit_depth > 8)
+ pp_shift = 16 - dec_params->bit_depth;
+
+ hantro_reg_write(ctx->dev, &g2_pp_pix_shift, pp_shift);
+ hantro_reg_write(ctx->dev, &g2_pix_shift, 0);
+ } else {
+ hantro_reg_write(ctx->dev, &g2_bit_depth_y_minus8, dec_params->bit_depth - 8);
+ hantro_reg_write(ctx->dev, &g2_bit_depth_c_minus8, dec_params->bit_depth - 8);
+ }
+}
+
+static inline bool is_lossless(const struct v4l2_vp9_quantization *quant)
+{
+ return quant->base_q_idx == 0 && quant->delta_q_uv_ac == 0 &&
+ quant->delta_q_uv_dc == 0 && quant->delta_q_y_dc == 0;
+}
+
+static void
+config_quant(struct hantro_ctx *ctx, const struct v4l2_ctrl_vp9_frame *dec_params)
+{
+ hantro_reg_write(ctx->dev, &vp9_qp_delta_y_dc, dec_params->quant.delta_q_y_dc);
+ hantro_reg_write(ctx->dev, &vp9_qp_delta_ch_dc, dec_params->quant.delta_q_uv_dc);
+ hantro_reg_write(ctx->dev, &vp9_qp_delta_ch_ac, dec_params->quant.delta_q_uv_ac);
+ hantro_reg_write(ctx->dev, &vp9_lossless_e, is_lossless(&dec_params->quant));
+}
+
+static u32
+hantro_interp_filter_from_v4l2(unsigned int interpolation_filter)
+{
+ switch (interpolation_filter) {
+ case V4L2_VP9_INTERP_FILTER_EIGHTTAP:
+ return 0x1;
+ case V4L2_VP9_INTERP_FILTER_EIGHTTAP_SMOOTH:
+ return 0;
+ case V4L2_VP9_INTERP_FILTER_EIGHTTAP_SHARP:
+ return 0x2;
+ case V4L2_VP9_INTERP_FILTER_BILINEAR:
+ return 0x3;
+ case V4L2_VP9_INTERP_FILTER_SWITCHABLE:
+ return 0x4;
+ }
+
+ return 0;
+}
+
+static void
+config_others(struct hantro_ctx *ctx, const struct v4l2_ctrl_vp9_frame *dec_params,
+ bool intra_only, bool resolution_change)
+{
+ struct hantro_vp9_dec_hw_ctx *vp9_ctx = &ctx->vp9_dec;
+
+ hantro_reg_write(ctx->dev, &g2_idr_pic_e, intra_only);
+
+ hantro_reg_write(ctx->dev, &vp9_transform_mode, vp9_ctx->cur.tx_mode);
+
+ hantro_reg_write(ctx->dev, &vp9_mcomp_filt_type, intra_only ?
+ 0 : hantro_interp_filter_from_v4l2(dec_params->interpolation_filter));
+
+ hantro_reg_write(ctx->dev, &vp9_high_prec_mv_e,
+ !!(dec_params->flags & V4L2_VP9_FRAME_FLAG_ALLOW_HIGH_PREC_MV));
+
+ hantro_reg_write(ctx->dev, &vp9_comp_pred_mode, dec_params->reference_mode);
+
+ hantro_reg_write(ctx->dev, &g2_tempor_mvp_e,
+ !(dec_params->flags & V4L2_VP9_FRAME_FLAG_ERROR_RESILIENT) &&
+ !(dec_params->flags & V4L2_VP9_FRAME_FLAG_KEY_FRAME) &&
+ !(vp9_ctx->last.flags & V4L2_VP9_FRAME_FLAG_KEY_FRAME) &&
+ !(dec_params->flags & V4L2_VP9_FRAME_FLAG_INTRA_ONLY) &&
+ !resolution_change &&
+ vp9_ctx->last.flags & V4L2_VP9_FRAME_FLAG_SHOW_FRAME
+ );
+
+ hantro_reg_write(ctx->dev, &g2_write_mvs_e,
+ !(dec_params->flags & V4L2_VP9_FRAME_FLAG_KEY_FRAME));
+}
+
+static void
+config_compound_reference(struct hantro_ctx *ctx,
+ const struct v4l2_ctrl_vp9_frame *dec_params)
+{
+ u32 comp_fixed_ref, comp_var_ref[2];
+ bool last_ref_frame_sign_bias;
+ bool golden_ref_frame_sign_bias;
+ bool alt_ref_frame_sign_bias;
+ bool comp_ref_allowed = 0;
+
+ comp_fixed_ref = 0;
+ comp_var_ref[0] = 0;
+ comp_var_ref[1] = 0;
+
+ last_ref_frame_sign_bias = dec_params->ref_frame_sign_bias & V4L2_VP9_SIGN_BIAS_LAST;
+ golden_ref_frame_sign_bias = dec_params->ref_frame_sign_bias & V4L2_VP9_SIGN_BIAS_GOLDEN;
+ alt_ref_frame_sign_bias = dec_params->ref_frame_sign_bias & V4L2_VP9_SIGN_BIAS_ALT;
+
+ /* 6.3.12 Frame reference mode syntax */
+ comp_ref_allowed |= golden_ref_frame_sign_bias != last_ref_frame_sign_bias;
+ comp_ref_allowed |= alt_ref_frame_sign_bias != last_ref_frame_sign_bias;
+
+ if (comp_ref_allowed) {
+ if (last_ref_frame_sign_bias ==
+ golden_ref_frame_sign_bias) {
+ comp_fixed_ref = ALTREF_FRAME;
+ comp_var_ref[0] = LAST_FRAME;
+ comp_var_ref[1] = GOLDEN_FRAME;
+ } else if (last_ref_frame_sign_bias ==
+ alt_ref_frame_sign_bias) {
+ comp_fixed_ref = GOLDEN_FRAME;
+ comp_var_ref[0] = LAST_FRAME;
+ comp_var_ref[1] = ALTREF_FRAME;
+ } else {
+ comp_fixed_ref = LAST_FRAME;
+ comp_var_ref[0] = GOLDEN_FRAME;
+ comp_var_ref[1] = ALTREF_FRAME;
+ }
+ }
+
+ hantro_reg_write(ctx->dev, &vp9_comp_pred_fixed_ref, comp_fixed_ref);
+ hantro_reg_write(ctx->dev, &vp9_comp_pred_var_ref0, comp_var_ref[0]);
+ hantro_reg_write(ctx->dev, &vp9_comp_pred_var_ref1, comp_var_ref[1]);
+}
+
+#define INNER_LOOP \
+do { \
+ for (m = 0; m < ARRAY_SIZE(adaptive->coef[0][0][0][0]); ++m) { \
+ memcpy(adaptive->coef[i][j][k][l][m], \
+ probs->coef[i][j][k][l][m], \
+ sizeof(probs->coef[i][j][k][l][m])); \
+ \
+ adaptive->coef[i][j][k][l][m][3] = 0; \
+ } \
+} while (0)
+
+static void config_probs(struct hantro_ctx *ctx, const struct v4l2_ctrl_vp9_frame *dec_params)
+{
+ struct hantro_vp9_dec_hw_ctx *vp9_ctx = &ctx->vp9_dec;
+ struct hantro_aux_buf *misc = &vp9_ctx->misc;
+ struct hantro_g2_all_probs *all_probs = misc->cpu;
+ struct hantro_g2_probs *adaptive;
+ struct hantro_g2_mv_probs *mv;
+ const struct v4l2_vp9_segmentation *seg = &dec_params->seg;
+ const struct v4l2_vp9_frame_context *probs = &vp9_ctx->probability_tables;
+ int i, j, k, l, m;
+
+ for (i = 0; i < ARRAY_SIZE(all_probs->kf_y_mode_prob); ++i)
+ for (j = 0; j < ARRAY_SIZE(all_probs->kf_y_mode_prob[0]); ++j) {
+ memcpy(all_probs->kf_y_mode_prob[i][j],
+ v4l2_vp9_kf_y_mode_prob[i][j],
+ ARRAY_SIZE(all_probs->kf_y_mode_prob[i][j]));
+
+ all_probs->kf_y_mode_prob_tail[i][j][0] =
+ v4l2_vp9_kf_y_mode_prob[i][j][8];
+ }
+
+ memcpy(all_probs->mb_segment_tree_probs, seg->tree_probs,
+ sizeof(all_probs->mb_segment_tree_probs));
+
+ memcpy(all_probs->segment_pred_probs, seg->pred_probs,
+ sizeof(all_probs->segment_pred_probs));
+
+ for (i = 0; i < ARRAY_SIZE(all_probs->kf_uv_mode_prob); ++i) {
+ memcpy(all_probs->kf_uv_mode_prob[i], v4l2_vp9_kf_uv_mode_prob[i],
+ ARRAY_SIZE(all_probs->kf_uv_mode_prob[i]));
+
+ all_probs->kf_uv_mode_prob_tail[i][0] = v4l2_vp9_kf_uv_mode_prob[i][8];
+ }
+
+ adaptive = &all_probs->probs;
+
+ for (i = 0; i < ARRAY_SIZE(adaptive->inter_mode); ++i) {
+ memcpy(adaptive->inter_mode[i], probs->inter_mode[i],
+ ARRAY_SIZE(probs->inter_mode[i]));
+
+ adaptive->inter_mode[i][3] = 0;
+ }
+
+ memcpy(adaptive->is_inter, probs->is_inter, sizeof(adaptive->is_inter));
+
+ for (i = 0; i < ARRAY_SIZE(adaptive->uv_mode); ++i) {
+ memcpy(adaptive->uv_mode[i], probs->uv_mode[i],
+ sizeof(adaptive->uv_mode[i]));
+ adaptive->uv_mode_tail[i][0] = probs->uv_mode[i][8];
+ }
+
+ memcpy(adaptive->tx8, probs->tx8, sizeof(adaptive->tx8));
+ memcpy(adaptive->tx16, probs->tx16, sizeof(adaptive->tx16));
+ memcpy(adaptive->tx32, probs->tx32, sizeof(adaptive->tx32));
+
+ for (i = 0; i < ARRAY_SIZE(adaptive->y_mode); ++i) {
+ memcpy(adaptive->y_mode[i], probs->y_mode[i],
+ ARRAY_SIZE(adaptive->y_mode[i]));
+
+ adaptive->y_mode_tail[i][0] = probs->y_mode[i][8];
+ }
+
+ for (i = 0; i < ARRAY_SIZE(adaptive->partition[0]); ++i) {
+ memcpy(adaptive->partition[0][i], v4l2_vp9_kf_partition_probs[i],
+ sizeof(v4l2_vp9_kf_partition_probs[i]));
+
+ adaptive->partition[0][i][3] = 0;
+ }
+
+ for (i = 0; i < ARRAY_SIZE(adaptive->partition[1]); ++i) {
+ memcpy(adaptive->partition[1][i], probs->partition[i],
+ sizeof(probs->partition[i]));
+
+ adaptive->partition[1][i][3] = 0;
+ }
+
+ memcpy(adaptive->interp_filter, probs->interp_filter,
+ sizeof(adaptive->interp_filter));
+
+ memcpy(adaptive->comp_mode, probs->comp_mode, sizeof(adaptive->comp_mode));
+
+ memcpy(adaptive->skip, probs->skip, sizeof(adaptive->skip));
+
+ mv = &adaptive->mv;
+
+ memcpy(mv->joint, probs->mv.joint, sizeof(mv->joint));
+ memcpy(mv->sign, probs->mv.sign, sizeof(mv->sign));
+ memcpy(mv->class0_bit, probs->mv.class0_bit, sizeof(mv->class0_bit));
+ memcpy(mv->fr, probs->mv.fr, sizeof(mv->fr));
+ memcpy(mv->class0_hp, probs->mv.class0_hp, sizeof(mv->class0_hp));
+ memcpy(mv->hp, probs->mv.hp, sizeof(mv->hp));
+ memcpy(mv->classes, probs->mv.classes, sizeof(mv->classes));
+ memcpy(mv->class0_fr, probs->mv.class0_fr, sizeof(mv->class0_fr));
+ memcpy(mv->bits, probs->mv.bits, sizeof(mv->bits));
+
+ memcpy(adaptive->single_ref, probs->single_ref, sizeof(adaptive->single_ref));
+
+ memcpy(adaptive->comp_ref, probs->comp_ref, sizeof(adaptive->comp_ref));
+
+ for (i = 0; i < ARRAY_SIZE(adaptive->coef); ++i)
+ for (j = 0; j < ARRAY_SIZE(adaptive->coef[0]); ++j)
+ for (k = 0; k < ARRAY_SIZE(adaptive->coef[0][0]); ++k)
+ for (l = 0; l < ARRAY_SIZE(adaptive->coef[0][0][0]); ++l)
+ INNER_LOOP;
+
+ hantro_write_addr(ctx->dev, G2_VP9_PROBS_ADDR, misc->dma);
+}
+
+static void config_counts(struct hantro_ctx *ctx)
+{
+ struct hantro_vp9_dec_hw_ctx *vp9_dec = &ctx->vp9_dec;
+ struct hantro_aux_buf *misc = &vp9_dec->misc;
+ dma_addr_t addr = misc->dma + vp9_dec->ctx_counters_offset;
+
+ hantro_write_addr(ctx->dev, G2_VP9_CTX_COUNT_ADDR, addr);
+}
+
+static void config_seg_map(struct hantro_ctx *ctx,
+ const struct v4l2_ctrl_vp9_frame *dec_params,
+ bool intra_only, bool update_map)
+{
+ struct hantro_vp9_dec_hw_ctx *vp9_ctx = &ctx->vp9_dec;
+ struct hantro_aux_buf *segment_map = &vp9_ctx->segment_map;
+ dma_addr_t addr;
+
+ if (intra_only ||
+ (dec_params->flags & V4L2_VP9_FRAME_FLAG_ERROR_RESILIENT)) {
+ memset(segment_map->cpu, 0, segment_map->size);
+ memset(vp9_ctx->feature_data, 0, sizeof(vp9_ctx->feature_data));
+ memset(vp9_ctx->feature_enabled, 0, sizeof(vp9_ctx->feature_enabled));
+ }
+
+ addr = segment_map->dma + vp9_ctx->active_segment * vp9_ctx->segment_map_size;
+ hantro_write_addr(ctx->dev, G2_VP9_SEGMENT_READ_ADDR, addr);
+
+ addr = segment_map->dma + (1 - vp9_ctx->active_segment) * vp9_ctx->segment_map_size;
+ hantro_write_addr(ctx->dev, G2_VP9_SEGMENT_WRITE_ADDR, addr);
+
+ if (update_map)
+ vp9_ctx->active_segment = 1 - vp9_ctx->active_segment;
+}
+
+static void
+config_source(struct hantro_ctx *ctx, const struct v4l2_ctrl_vp9_frame *dec_params,
+ struct vb2_v4l2_buffer *vb2_src)
+{
+ dma_addr_t stream_base, tmp_addr;
+ unsigned int headres_size;
+ u32 src_len, start_bit, src_buf_len;
+
+ headres_size = dec_params->uncompressed_header_size
+ + dec_params->compressed_header_size;
+
+ stream_base = vb2_dma_contig_plane_dma_addr(&vb2_src->vb2_buf, 0);
+
+ tmp_addr = stream_base + headres_size;
+ if (ctx->dev->variant->legacy_regs)
+ hantro_write_addr(ctx->dev, G2_STREAM_ADDR, (tmp_addr & ~0xf));
+ else
+ hantro_write_addr(ctx->dev, G2_STREAM_ADDR, stream_base);
+
+ start_bit = (tmp_addr & 0xf) * 8;
+ hantro_reg_write(ctx->dev, &g2_start_bit, start_bit);
+
+ src_len = vb2_get_plane_payload(&vb2_src->vb2_buf, 0);
+ src_len += start_bit / 8 - headres_size;
+ hantro_reg_write(ctx->dev, &g2_stream_len, src_len);
+
+ if (!ctx->dev->variant->legacy_regs) {
+ tmp_addr &= ~0xf;
+ hantro_reg_write(ctx->dev, &g2_strm_start_offset, tmp_addr - stream_base);
+ src_buf_len = vb2_plane_size(&vb2_src->vb2_buf, 0);
+ hantro_reg_write(ctx->dev, &g2_strm_buffer_len, src_buf_len);
+ }
+}
+
+static void
+config_registers(struct hantro_ctx *ctx, const struct v4l2_ctrl_vp9_frame *dec_params,
+ struct vb2_v4l2_buffer *vb2_src, struct vb2_v4l2_buffer *vb2_dst)
+{
+ struct hantro_decoded_buffer *dst, *last, *mv_ref;
+ struct hantro_vp9_dec_hw_ctx *vp9_ctx = &ctx->vp9_dec;
+ const struct v4l2_vp9_segmentation *seg;
+ bool intra_only, resolution_change;
+
+ /* vp9 stuff */
+ dst = vb2_to_hantro_decoded_buf(&vb2_dst->vb2_buf);
+
+ if (vp9_ctx->last.valid)
+ last = get_ref_buf(ctx, &dst->base.vb, vp9_ctx->last.timestamp);
+ else
+ last = dst;
+
+ update_dec_buf_info(dst, dec_params);
+ update_ctx_cur_info(vp9_ctx, dst, dec_params);
+ seg = &dec_params->seg;
+
+ intra_only = !!(dec_params->flags &
+ (V4L2_VP9_FRAME_FLAG_KEY_FRAME |
+ V4L2_VP9_FRAME_FLAG_INTRA_ONLY));
+
+ if (!intra_only &&
+ !(dec_params->flags & V4L2_VP9_FRAME_FLAG_ERROR_RESILIENT) &&
+ vp9_ctx->last.valid)
+ mv_ref = last;
+ else
+ mv_ref = dst;
+
+ resolution_change = dst->vp9.width != last->vp9.width ||
+ dst->vp9.height != last->vp9.height;
+
+ /* configure basic registers */
+ hantro_reg_write(ctx->dev, &g2_mode, VP9_DEC_MODE);
+ if (!ctx->dev->variant->legacy_regs) {
+ hantro_reg_write(ctx->dev, &g2_strm_swap, 0xf);
+ hantro_reg_write(ctx->dev, &g2_dirmv_swap, 0xf);
+ hantro_reg_write(ctx->dev, &g2_compress_swap, 0xf);
+ hantro_reg_write(ctx->dev, &g2_ref_compress_bypass, 1);
+ } else {
+ hantro_reg_write(ctx->dev, &g2_strm_swap_old, 0x1f);
+ hantro_reg_write(ctx->dev, &g2_pic_swap, 0x10);
+ hantro_reg_write(ctx->dev, &g2_dirmv_swap_old, 0x10);
+ hantro_reg_write(ctx->dev, &g2_tab0_swap_old, 0x10);
+ hantro_reg_write(ctx->dev, &g2_tab1_swap_old, 0x10);
+ hantro_reg_write(ctx->dev, &g2_tab2_swap_old, 0x10);
+ hantro_reg_write(ctx->dev, &g2_tab3_swap_old, 0x10);
+ hantro_reg_write(ctx->dev, &g2_rscan_swap, 0x10);
+ }
+ hantro_reg_write(ctx->dev, &g2_buswidth, BUS_WIDTH_128);
+ hantro_reg_write(ctx->dev, &g2_max_burst, 16);
+ hantro_reg_write(ctx->dev, &g2_apf_threshold, 8);
+ hantro_reg_write(ctx->dev, &g2_clk_gate_e, 1);
+ hantro_reg_write(ctx->dev, &g2_max_cb_size, 6);
+ hantro_reg_write(ctx->dev, &g2_min_cb_size, 3);
+ if (ctx->dev->variant->double_buffer)
+ hantro_reg_write(ctx->dev, &g2_double_buffer_e, 1);
+
+ config_output(ctx, dst, dec_params);
+
+ if (!intra_only)
+ config_ref_registers(ctx, dec_params, dst, mv_ref);
+
+ config_tiles(ctx, dec_params, dst);
+ config_segment(ctx, dec_params);
+ config_loop_filter(ctx, dec_params);
+ config_picture_dimensions(ctx, dst);
+ config_bit_depth(ctx, dec_params);
+ config_quant(ctx, dec_params);
+ config_others(ctx, dec_params, intra_only, resolution_change);
+ config_compound_reference(ctx, dec_params);
+ config_probs(ctx, dec_params);
+ config_counts(ctx);
+ config_seg_map(ctx, dec_params, intra_only,
+ seg->flags & V4L2_VP9_SEGMENTATION_FLAG_UPDATE_MAP);
+ config_source(ctx, dec_params, vb2_src);
+}
+
+int hantro_g2_vp9_dec_run(struct hantro_ctx *ctx)
+{
+ const struct v4l2_ctrl_vp9_frame *decode_params;
+ struct vb2_v4l2_buffer *src;
+ struct vb2_v4l2_buffer *dst;
+ int ret;
+
+ hantro_g2_check_idle(ctx->dev);
+
+ ret = start_prepare_run(ctx, &decode_params);
+ if (ret) {
+ hantro_end_prepare_run(ctx);
+ return ret;
+ }
+
+ src = hantro_get_src_buf(ctx);
+ dst = hantro_get_dst_buf(ctx);
+
+ config_registers(ctx, decode_params, src, dst);
+
+ hantro_end_prepare_run(ctx);
+
+ vdpu_write(ctx->dev, G2_REG_INTERRUPT_DEC_E, G2_REG_INTERRUPT);
+
+ return 0;
+}
+
+#define copy_tx_and_skip(p1, p2) \
+do { \
+ memcpy((p1)->tx8, (p2)->tx8, sizeof((p1)->tx8)); \
+ memcpy((p1)->tx16, (p2)->tx16, sizeof((p1)->tx16)); \
+ memcpy((p1)->tx32, (p2)->tx32, sizeof((p1)->tx32)); \
+ memcpy((p1)->skip, (p2)->skip, sizeof((p1)->skip)); \
+} while (0)
+
+void hantro_g2_vp9_dec_done(struct hantro_ctx *ctx)
+{
+ struct hantro_vp9_dec_hw_ctx *vp9_ctx = &ctx->vp9_dec;
+ unsigned int fctx_idx;
+
+ if (!(vp9_ctx->cur.flags & V4L2_VP9_FRAME_FLAG_REFRESH_FRAME_CTX))
+ goto out_update_last;
+
+ fctx_idx = vp9_ctx->cur.frame_context_idx;
+
+ if (!(vp9_ctx->cur.flags & V4L2_VP9_FRAME_FLAG_PARALLEL_DEC_MODE)) {
+ /* error_resilient_mode == 0 && frame_parallel_decoding_mode == 0 */
+ struct v4l2_vp9_frame_context *probs = &vp9_ctx->probability_tables;
+ bool frame_is_intra = vp9_ctx->cur.flags &
+ (V4L2_VP9_FRAME_FLAG_KEY_FRAME | V4L2_VP9_FRAME_FLAG_INTRA_ONLY);
+ struct tx_and_skip {
+ u8 tx8[2][1];
+ u8 tx16[2][2];
+ u8 tx32[2][3];
+ u8 skip[3];
+ } _tx_skip, *tx_skip = &_tx_skip;
+ struct v4l2_vp9_frame_symbol_counts *counts;
+ struct symbol_counts *hantro_cnts;
+ u32 tx16p[2][4];
+ int i;
+
+ /* buffer the forward-updated TX and skip probs */
+ if (frame_is_intra)
+ copy_tx_and_skip(tx_skip, probs);
+
+ /* 6.1.2 refresh_probs(): load_probs() and load_probs2() */
+ *probs = vp9_ctx->frame_context[fctx_idx];
+
+ /* if FrameIsIntra then undo the effect of load_probs2() */
+ if (frame_is_intra)
+ copy_tx_and_skip(probs, tx_skip);
+
+ counts = &vp9_ctx->cnts;
+ hantro_cnts = vp9_ctx->misc.cpu + vp9_ctx->ctx_counters_offset;
+ for (i = 0; i < ARRAY_SIZE(tx16p); ++i) {
+ memcpy(tx16p[i],
+ hantro_cnts->tx16x16_count[i],
+ sizeof(hantro_cnts->tx16x16_count[0]));
+ tx16p[i][3] = 0;
+ }
+ counts->tx16p = &tx16p;
+
+ v4l2_vp9_adapt_coef_probs(probs, counts,
+ !vp9_ctx->last.valid ||
+ vp9_ctx->last.flags & V4L2_VP9_FRAME_FLAG_KEY_FRAME,
+ frame_is_intra);
+
+ if (!frame_is_intra) {
+ /* load_probs2() already done */
+ u32 mv_mode[7][4];
+
+ for (i = 0; i < ARRAY_SIZE(mv_mode); ++i) {
+ mv_mode[i][0] = hantro_cnts->inter_mode_counts[i][1][0];
+ mv_mode[i][1] = hantro_cnts->inter_mode_counts[i][2][0];
+ mv_mode[i][2] = hantro_cnts->inter_mode_counts[i][0][0];
+ mv_mode[i][3] = hantro_cnts->inter_mode_counts[i][2][1];
+ }
+ counts->mv_mode = &mv_mode;
+ v4l2_vp9_adapt_noncoef_probs(&vp9_ctx->probability_tables, counts,
+ vp9_ctx->cur.reference_mode,
+ vp9_ctx->cur.interpolation_filter,
+ vp9_ctx->cur.tx_mode, vp9_ctx->cur.flags);
+ }
+ }
+
+ vp9_ctx->frame_context[fctx_idx] = vp9_ctx->probability_tables;
+
+out_update_last:
+ vp9_ctx->last = vp9_ctx->cur;
+}
diff --git a/drivers/staging/media/hantro/hantro_h1_jpeg_enc.c b/drivers/staging/media/hantro/hantro_h1_jpeg_enc.c
index 56cf261..1450013 100644
--- a/drivers/staging/media/hantro/hantro_h1_jpeg_enc.c
+++ b/drivers/staging/media/hantro/hantro_h1_jpeg_enc.c
@@ -113,9 +113,8 @@ int hantro_h1_jpeg_enc_run(struct hantro_ctx *ctx)
hantro_h1_set_src_img_ctrl(vpu, ctx);
hantro_h1_jpeg_enc_set_buffers(vpu, ctx, &src_buf->vb2_buf);
- hantro_h1_jpeg_enc_set_qtable(vpu,
- hantro_jpeg_get_qtable(0),
- hantro_jpeg_get_qtable(1));
+ hantro_h1_jpeg_enc_set_qtable(vpu, jpeg_ctx.hw_luma_qtable,
+ jpeg_ctx.hw_chroma_qtable);
reg = H1_REG_AXI_CTRL_OUTPUT_SWAP16
| H1_REG_AXI_CTRL_INPUT_SWAP16
@@ -140,7 +139,7 @@ int hantro_h1_jpeg_enc_run(struct hantro_ctx *ctx)
return 0;
}
-void hantro_jpeg_enc_done(struct hantro_ctx *ctx)
+void hantro_h1_jpeg_enc_done(struct hantro_ctx *ctx)
{
struct hantro_dev *vpu = ctx->dev;
u32 bytesused = vepu_read(vpu, H1_REG_STR_BUF_LIMIT) / 8;
diff --git a/drivers/staging/media/hantro/hantro_hevc.c b/drivers/staging/media/hantro/hantro_hevc.c
index ee03123..b49a41d 100644
--- a/drivers/staging/media/hantro/hantro_hevc.c
+++ b/drivers/staging/media/hantro/hantro_hevc.c
@@ -44,47 +44,6 @@ size_t hantro_hevc_motion_vectors_offset(const struct v4l2_ctrl_hevc_sps *sps)
return ALIGN((cr_offset * 3) / 2, G2_ALIGN);
}
-static size_t hantro_hevc_mv_size(const struct v4l2_ctrl_hevc_sps *sps)
-{
- u32 min_cb_log2_size_y = sps->log2_min_luma_coding_block_size_minus3 + 3;
- u32 ctb_log2_size_y = min_cb_log2_size_y + sps->log2_diff_max_min_luma_coding_block_size;
- u32 pic_width_in_ctbs_y = (sps->pic_width_in_luma_samples + (1 << ctb_log2_size_y) - 1)
- >> ctb_log2_size_y;
- u32 pic_height_in_ctbs_y = (sps->pic_height_in_luma_samples + (1 << ctb_log2_size_y) - 1)
- >> ctb_log2_size_y;
- size_t mv_size;
-
- mv_size = pic_width_in_ctbs_y * pic_height_in_ctbs_y *
- (1 << (2 * (ctb_log2_size_y - 4))) * 16;
-
- vpu_debug(4, "%dx%d (CTBs) %zu MV bytes\n",
- pic_width_in_ctbs_y, pic_height_in_ctbs_y, mv_size);
-
- return mv_size;
-}
-
-static size_t hantro_hevc_ref_size(struct hantro_ctx *ctx)
-{
- const struct hantro_hevc_dec_ctrls *ctrls = &ctx->hevc_dec.ctrls;
- const struct v4l2_ctrl_hevc_sps *sps = ctrls->sps;
-
- return hantro_hevc_motion_vectors_offset(sps) + hantro_hevc_mv_size(sps);
-}
-
-static void hantro_hevc_ref_free(struct hantro_ctx *ctx)
-{
- struct hantro_hevc_dec_hw_ctx *hevc_dec = &ctx->hevc_dec;
- struct hantro_dev *vpu = ctx->dev;
- int i;
-
- for (i = 0; i < NUM_REF_PICTURES; i++) {
- if (hevc_dec->ref_bufs[i].cpu)
- dma_free_coherent(vpu->dev, hevc_dec->ref_bufs[i].size,
- hevc_dec->ref_bufs[i].cpu,
- hevc_dec->ref_bufs[i].dma);
- }
-}
-
static void hantro_hevc_ref_init(struct hantro_ctx *ctx)
{
struct hantro_hevc_dec_hw_ctx *hevc_dec = &ctx->hevc_dec;
@@ -108,37 +67,25 @@ dma_addr_t hantro_hevc_get_ref_buf(struct hantro_ctx *ctx,
}
}
- /* Allocate a new reference buffer */
+ return 0;
+}
+
+int hantro_hevc_add_ref_buf(struct hantro_ctx *ctx, int poc, dma_addr_t addr)
+{
+ struct hantro_hevc_dec_hw_ctx *hevc_dec = &ctx->hevc_dec;
+ int i;
+
+ /* Add a new reference buffer */
for (i = 0; i < NUM_REF_PICTURES; i++) {
if (hevc_dec->ref_bufs_poc[i] == UNUSED_REF) {
- if (!hevc_dec->ref_bufs[i].cpu) {
- struct hantro_dev *vpu = ctx->dev;
-
- /*
- * Allocate the space needed for the raw data +
- * motion vector data. Optimizations could be to
- * allocate raw data in non coherent memory and only
- * clear the motion vector data.
- */
- hevc_dec->ref_bufs[i].cpu =
- dma_alloc_coherent(vpu->dev,
- hantro_hevc_ref_size(ctx),
- &hevc_dec->ref_bufs[i].dma,
- GFP_KERNEL);
- if (!hevc_dec->ref_bufs[i].cpu)
- return 0;
-
- hevc_dec->ref_bufs[i].size = hantro_hevc_ref_size(ctx);
- }
hevc_dec->ref_bufs_used |= 1 << i;
- memset(hevc_dec->ref_bufs[i].cpu, 0, hantro_hevc_ref_size(ctx));
hevc_dec->ref_bufs_poc[i] = poc;
-
- return hevc_dec->ref_bufs[i].dma;
+ hevc_dec->ref_bufs[i].dma = addr;
+ return 0;
}
}
- return 0;
+ return -EINVAL;
}
void hantro_hevc_ref_remove_unused(struct hantro_ctx *ctx)
@@ -314,8 +261,6 @@ void hantro_hevc_dec_exit(struct hantro_ctx *ctx)
hevc_dec->tile_bsd.cpu,
hevc_dec->tile_bsd.dma);
hevc_dec->tile_bsd.cpu = NULL;
-
- hantro_hevc_ref_free(ctx);
}
int hantro_hevc_dec_init(struct hantro_ctx *ctx)
diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
index 267a6d3..4a19ae8 100644
--- a/drivers/staging/media/hantro/hantro_hw.h
+++ b/drivers/staging/media/hantro/hantro_hw.h
@@ -12,6 +12,7 @@
#include <linux/interrupt.h>
#include <linux/v4l2-controls.h>
#include <media/v4l2-ctrls.h>
+#include <media/v4l2-vp9.h>
#include <media/videobuf2-core.h>
#define DEC_8190_ALIGN_MASK 0x07U
@@ -166,6 +167,82 @@ struct hantro_vp8_dec_hw_ctx {
};
/**
+ * struct hantro_vp9_frame_info
+ *
+ * @valid: frame info valid flag
+ * @frame_context_idx: index of frame context
+ * @reference_mode: inter prediction type
+ * @tx_mode: transform mode
+ * @interpolation_filter: filter selection for inter prediction
+ * @flags: frame flags
+ * @timestamp: frame timestamp
+ */
+struct hantro_vp9_frame_info {
+ u32 valid : 1;
+ u32 frame_context_idx : 2;
+ u32 reference_mode : 2;
+ u32 tx_mode : 3;
+ u32 interpolation_filter : 3;
+ u32 flags;
+ u64 timestamp;
+};
+
+#define MAX_SB_COLS 64
+#define MAX_SB_ROWS 34
+
+/**
+ * struct hantro_vp9_dec_hw_ctx
+ *
+ * @tile_edge: auxiliary DMA buffer for tile edge processing
+ * @segment_map: auxiliary DMA buffer for segment map
+ * @misc: auxiliary DMA buffer for tile info, probabilities and hw counters
+ * @cnts: vp9 library struct for abstracting hw counters access
+ * @probability_tables: VP9 probability tables implied by the spec
+ * @frame_context: VP9 frame contexts
+ * @cur: current frame information
+ * @last: last frame information
+ * @bsd_ctrl_offset: bsd offset into tile_edge
+ * @segment_map_size: size of segment map
+ * @ctx_counters_offset: hw counters offset into misc
+ * @tile_info_offset: tile info offset into misc
+ * @tile_r_info: per-tile information array
+ * @tile_c_info: per-tile information array
+ * @last_tile_r: last number of tile rows
+ * @last_tile_c: last number of tile cols
+ * @last_sbs_r: last number of superblock rows
+ * @last_sbs_c: last number of superblock cols
+ * @active_segment: number of active segment (alternating between 0 and 1)
+ * @feature_enabled: segmentation feature enabled flags
+ * @feature_data: segmentation feature data
+ */
+struct hantro_vp9_dec_hw_ctx {
+ struct hantro_aux_buf tile_edge;
+ struct hantro_aux_buf segment_map;
+ struct hantro_aux_buf misc;
+ struct v4l2_vp9_frame_symbol_counts cnts;
+ struct v4l2_vp9_frame_context probability_tables;
+ struct v4l2_vp9_frame_context frame_context[4];
+ struct hantro_vp9_frame_info cur;
+ struct hantro_vp9_frame_info last;
+
+ unsigned int bsd_ctrl_offset;
+ unsigned int segment_map_size;
+ unsigned int ctx_counters_offset;
+ unsigned int tile_info_offset;
+
+ unsigned short tile_r_info[MAX_SB_ROWS];
+ unsigned short tile_c_info[MAX_SB_COLS];
+ unsigned int last_tile_r;
+ unsigned int last_tile_c;
+ unsigned int last_sbs_r;
+ unsigned int last_sbs_c;
+
+ unsigned int active_segment;
+ u8 feature_enabled[8];
+ s16 feature_data[8][4];
+};
+
+/**
* struct hantro_postproc_ctx
*
* @dec_q: References buffers, in decoder format.
@@ -175,6 +252,17 @@ struct hantro_postproc_ctx {
};
/**
+ * struct hantro_postproc_ops - post-processor operations
+ *
+ * @enable: Enable the post-processor block. Optional.
+ * @disable: Disable the post-processor block. Optional.
+ */
+struct hantro_postproc_ops {
+ void (*enable)(struct hantro_ctx *ctx);
+ void (*disable)(struct hantro_ctx *ctx);
+};
+
+/**
* struct hantro_codec_ops - codec mode specific operations
*
* @init: If needed, can be used for initialization.
@@ -220,8 +308,10 @@ extern const struct hantro_variant rk3288_vpu_variant;
extern const struct hantro_variant rk3328_vpu_variant;
extern const struct hantro_variant rk3399_vpu_variant;
extern const struct hantro_variant sama5d4_vdec_variant;
+extern const struct hantro_variant sunxi_vpu_variant;
-extern const struct hantro_postproc_regs hantro_g1_postproc_regs;
+extern const struct hantro_postproc_ops hantro_g1_postproc_ops;
+extern const struct hantro_postproc_ops hantro_g2_postproc_ops;
extern const u32 hantro_vp8_dec_mc_filter[8][6];
@@ -239,7 +329,8 @@ int hantro_h1_jpeg_enc_run(struct hantro_ctx *ctx);
int rockchip_vpu2_jpeg_enc_run(struct hantro_ctx *ctx);
int hantro_jpeg_enc_init(struct hantro_ctx *ctx);
void hantro_jpeg_enc_exit(struct hantro_ctx *ctx);
-void hantro_jpeg_enc_done(struct hantro_ctx *ctx);
+void hantro_h1_jpeg_enc_done(struct hantro_ctx *ctx);
+void rockchip_vpu2_jpeg_enc_done(struct hantro_ctx *ctx);
dma_addr_t hantro_h264_get_ref_buf(struct hantro_ctx *ctx,
unsigned int dpb_idx);
@@ -256,10 +347,29 @@ void hantro_hevc_dec_exit(struct hantro_ctx *ctx);
int hantro_g2_hevc_dec_run(struct hantro_ctx *ctx);
int hantro_hevc_dec_prepare_run(struct hantro_ctx *ctx);
dma_addr_t hantro_hevc_get_ref_buf(struct hantro_ctx *ctx, int poc);
+int hantro_hevc_add_ref_buf(struct hantro_ctx *ctx, int poc, dma_addr_t addr);
void hantro_hevc_ref_remove_unused(struct hantro_ctx *ctx);
size_t hantro_hevc_chroma_offset(const struct v4l2_ctrl_hevc_sps *sps);
size_t hantro_hevc_motion_vectors_offset(const struct v4l2_ctrl_hevc_sps *sps);
+static inline unsigned short hantro_vp9_num_sbs(unsigned short dimension)
+{
+ return (dimension + 63) / 64;
+}
+
+static inline size_t
+hantro_vp9_mv_size(unsigned int width, unsigned int height)
+{
+ int num_ctbs;
+
+ /*
+ * There can be up to (CTBs x 64) number of blocks,
+ * and the motion vector for each block needs 16 bytes.
+ */
+ num_ctbs = hantro_vp9_num_sbs(width) * hantro_vp9_num_sbs(height);
+ return (num_ctbs * 64) * 16;
+}
+
static inline size_t
hantro_h264_mv_size(unsigned int width, unsigned int height)
{
@@ -287,6 +397,16 @@ hantro_h264_mv_size(unsigned int width, unsigned int height)
return 64 * MB_WIDTH(width) * MB_WIDTH(height) + 32;
}
+static inline size_t
+hantro_hevc_mv_size(unsigned int width, unsigned int height)
+{
+ /*
+ * A CTB can be 64x64, 32x32 or 16x16.
+ * Allocated memory for the "worse" case: 16x16
+ */
+ return width * height / 16;
+}
+
int hantro_g1_mpeg2_dec_run(struct hantro_ctx *ctx);
int rockchip_vpu2_mpeg2_dec_run(struct hantro_ctx *ctx);
void hantro_mpeg2_dec_copy_qtable(u8 *qtable,
@@ -301,4 +421,11 @@ void hantro_vp8_dec_exit(struct hantro_ctx *ctx);
void hantro_vp8_prob_update(struct hantro_ctx *ctx,
const struct v4l2_ctrl_vp8_frame *hdr);
+int hantro_g2_vp9_dec_run(struct hantro_ctx *ctx);
+void hantro_g2_vp9_dec_done(struct hantro_ctx *ctx);
+int hantro_vp9_dec_init(struct hantro_ctx *ctx);
+void hantro_vp9_dec_exit(struct hantro_ctx *ctx);
+void hantro_g2_check_idle(struct hantro_dev *vpu);
+irqreturn_t hantro_g2_irq(int irq, void *dev_id);
+
#endif /* HANTRO_HW_H_ */
diff --git a/drivers/staging/media/hantro/hantro_jpeg.c b/drivers/staging/media/hantro/hantro_jpeg.c
index 36c140fc..df62fbd 100644
--- a/drivers/staging/media/hantro/hantro_jpeg.c
+++ b/drivers/staging/media/hantro/hantro_jpeg.c
@@ -36,8 +36,6 @@ static const unsigned char luma_q_table[] = {
0x48, 0x5c, 0x5f, 0x62, 0x70, 0x64, 0x67, 0x63
};
-static unsigned char luma_q_table_reordered[ARRAY_SIZE(luma_q_table)];
-
static const unsigned char chroma_q_table[] = {
0x11, 0x12, 0x18, 0x2f, 0x63, 0x63, 0x63, 0x63,
0x12, 0x15, 0x1a, 0x42, 0x63, 0x63, 0x63, 0x63,
@@ -49,8 +47,6 @@ static const unsigned char chroma_q_table[] = {
0x63, 0x63, 0x63, 0x63, 0x63, 0x63, 0x63, 0x63
};
-static unsigned char chroma_q_table_reordered[ARRAY_SIZE(chroma_q_table)];
-
static const unsigned char zigzag[64] = {
0, 1, 8, 16, 9, 2, 3, 10,
17, 24, 32, 25, 18, 11, 4, 5,
@@ -277,7 +273,7 @@ jpeg_scale_quant_table(unsigned char *file_q_tab,
}
}
-static void jpeg_set_quality(unsigned char *buffer, int quality)
+static void jpeg_set_quality(struct hantro_jpeg_ctx *ctx)
{
int scale;
@@ -285,24 +281,15 @@ static void jpeg_set_quality(unsigned char *buffer, int quality)
* Non-linear scaling factor:
* [5,50] -> [1000..100], [51,100] -> [98..0]
*/
- if (quality < 50)
- scale = 5000 / quality;
+ if (ctx->quality < 50)
+ scale = 5000 / ctx->quality;
else
- scale = 200 - 2 * quality;
+ scale = 200 - 2 * ctx->quality;
- jpeg_scale_quant_table(buffer + LUMA_QUANT_OFF,
- luma_q_table_reordered,
- luma_q_table, scale);
- jpeg_scale_quant_table(buffer + CHROMA_QUANT_OFF,
- chroma_q_table_reordered,
- chroma_q_table, scale);
-}
-
-unsigned char *hantro_jpeg_get_qtable(int index)
-{
- if (index == 0)
- return luma_q_table_reordered;
- return chroma_q_table_reordered;
+ jpeg_scale_quant_table(ctx->buffer + LUMA_QUANT_OFF,
+ ctx->hw_luma_qtable, luma_q_table, scale);
+ jpeg_scale_quant_table(ctx->buffer + CHROMA_QUANT_OFF,
+ ctx->hw_chroma_qtable, chroma_q_table, scale);
}
void hantro_jpeg_header_assemble(struct hantro_jpeg_ctx *ctx)
@@ -324,7 +311,7 @@ void hantro_jpeg_header_assemble(struct hantro_jpeg_ctx *ctx)
memcpy(buf + HUFF_CHROMA_AC_OFF, chroma_ac_table,
sizeof(chroma_ac_table));
- jpeg_set_quality(buf, ctx->quality);
+ jpeg_set_quality(ctx);
}
int hantro_jpeg_enc_init(struct hantro_ctx *ctx)
diff --git a/drivers/staging/media/hantro/hantro_jpeg.h b/drivers/staging/media/hantro/hantro_jpeg.h
index 9474a00..035ab25 100644
--- a/drivers/staging/media/hantro/hantro_jpeg.h
+++ b/drivers/staging/media/hantro/hantro_jpeg.h
@@ -1,13 +1,15 @@
/* SPDX-License-Identifier: GPL-2.0+ */
#define JPEG_HEADER_SIZE 601
+#define JPEG_QUANT_SIZE 64
struct hantro_jpeg_ctx {
int width;
int height;
int quality;
unsigned char *buffer;
+ unsigned char hw_luma_qtable[JPEG_QUANT_SIZE];
+ unsigned char hw_chroma_qtable[JPEG_QUANT_SIZE];
};
-unsigned char *hantro_jpeg_get_qtable(int index);
void hantro_jpeg_header_assemble(struct hantro_jpeg_ctx *ctx);
diff --git a/drivers/staging/media/hantro/hantro_postproc.c b/drivers/staging/media/hantro/hantro_postproc.c
index ed8916c..248abe5 100644
--- a/drivers/staging/media/hantro/hantro_postproc.c
+++ b/drivers/staging/media/hantro/hantro_postproc.c
@@ -11,18 +11,19 @@
#include "hantro.h"
#include "hantro_hw.h"
#include "hantro_g1_regs.h"
+#include "hantro_g2_regs.h"
#define HANTRO_PP_REG_WRITE(vpu, reg_name, val) \
{ \
hantro_reg_write(vpu, \
- &(vpu)->variant->postproc_regs->reg_name, \
+ &hantro_g1_postproc_regs.reg_name, \
val); \
}
#define HANTRO_PP_REG_WRITE_S(vpu, reg_name, val) \
{ \
hantro_reg_write_s(vpu, \
- &(vpu)->variant->postproc_regs->reg_name, \
+ &hantro_g1_postproc_regs.reg_name, \
val); \
}
@@ -33,7 +34,7 @@
#define VPU_PP_OUT_RGB 0x0
#define VPU_PP_OUT_YUYV 0x3
-const struct hantro_postproc_regs hantro_g1_postproc_regs = {
+static const struct hantro_postproc_regs hantro_g1_postproc_regs = {
.pipeline_en = {G1_REG_PP_INTERRUPT, 1, 0x1},
.max_burst = {G1_REG_PP_DEV_CONFIG, 0, 0x1f},
.clk_gate = {G1_REG_PP_DEV_CONFIG, 1, 0x1},
@@ -53,27 +54,18 @@ const struct hantro_postproc_regs hantro_g1_postproc_regs = {
bool hantro_needs_postproc(const struct hantro_ctx *ctx,
const struct hantro_fmt *fmt)
{
- struct hantro_dev *vpu = ctx->dev;
-
if (ctx->is_encoder)
return false;
-
- if (!vpu->variant->postproc_fmts)
- return false;
-
- return fmt->fourcc != V4L2_PIX_FMT_NV12;
+ return fmt->postprocessed;
}
-void hantro_postproc_enable(struct hantro_ctx *ctx)
+static void hantro_postproc_g1_enable(struct hantro_ctx *ctx)
{
struct hantro_dev *vpu = ctx->dev;
struct vb2_v4l2_buffer *dst_buf;
u32 src_pp_fmt, dst_pp_fmt;
dma_addr_t dst_dma;
- if (!vpu->variant->postproc_regs)
- return;
-
/* Turn on pipeline mode. Must be done first. */
HANTRO_PP_REG_WRITE_S(vpu, pipeline_en, 0x1);
@@ -108,6 +100,21 @@ void hantro_postproc_enable(struct hantro_ctx *ctx)
HANTRO_PP_REG_WRITE(vpu, display_width, ctx->dst_fmt.width);
}
+static void hantro_postproc_g2_enable(struct hantro_ctx *ctx)
+{
+ struct hantro_dev *vpu = ctx->dev;
+ struct vb2_v4l2_buffer *dst_buf;
+ size_t chroma_offset = ctx->dst_fmt.width * ctx->dst_fmt.height;
+ dma_addr_t dst_dma;
+
+ dst_buf = hantro_get_dst_buf(ctx);
+ dst_dma = vb2_dma_contig_plane_dma_addr(&dst_buf->vb2_buf, 0);
+
+ hantro_write_addr(vpu, G2_RS_OUT_LUMA_ADDR, dst_dma);
+ hantro_write_addr(vpu, G2_RS_OUT_CHROMA_ADDR, dst_dma + chroma_offset);
+ hantro_reg_write(vpu, &g2_out_rs_e, 1);
+}
+
void hantro_postproc_free(struct hantro_ctx *ctx)
{
struct hantro_dev *vpu = ctx->dev;
@@ -132,9 +139,16 @@ int hantro_postproc_alloc(struct hantro_ctx *ctx)
unsigned int num_buffers = cap_queue->num_buffers;
unsigned int i, buf_size;
- buf_size = ctx->dst_fmt.plane_fmt[0].sizeimage +
- hantro_h264_mv_size(ctx->dst_fmt.width,
- ctx->dst_fmt.height);
+ buf_size = ctx->dst_fmt.plane_fmt[0].sizeimage;
+ if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_H264_SLICE)
+ buf_size += hantro_h264_mv_size(ctx->dst_fmt.width,
+ ctx->dst_fmt.height);
+ else if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_VP9_FRAME)
+ buf_size += hantro_vp9_mv_size(ctx->dst_fmt.width,
+ ctx->dst_fmt.height);
+ else if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_HEVC_SLICE)
+ buf_size += hantro_hevc_mv_size(ctx->dst_fmt.width,
+ ctx->dst_fmt.height);
for (i = 0; i < num_buffers; ++i) {
struct hantro_aux_buf *priv = &ctx->postproc.dec_q[i];
@@ -153,12 +167,42 @@ int hantro_postproc_alloc(struct hantro_ctx *ctx)
return 0;
}
+static void hantro_postproc_g1_disable(struct hantro_ctx *ctx)
+{
+ struct hantro_dev *vpu = ctx->dev;
+
+ HANTRO_PP_REG_WRITE_S(vpu, pipeline_en, 0x0);
+}
+
+static void hantro_postproc_g2_disable(struct hantro_ctx *ctx)
+{
+ struct hantro_dev *vpu = ctx->dev;
+
+ hantro_reg_write(vpu, &g2_out_rs_e, 0);
+}
+
void hantro_postproc_disable(struct hantro_ctx *ctx)
{
struct hantro_dev *vpu = ctx->dev;
- if (!vpu->variant->postproc_regs)
- return;
-
- HANTRO_PP_REG_WRITE_S(vpu, pipeline_en, 0x0);
+ if (vpu->variant->postproc_ops && vpu->variant->postproc_ops->disable)
+ vpu->variant->postproc_ops->disable(ctx);
}
+
+void hantro_postproc_enable(struct hantro_ctx *ctx)
+{
+ struct hantro_dev *vpu = ctx->dev;
+
+ if (vpu->variant->postproc_ops && vpu->variant->postproc_ops->enable)
+ vpu->variant->postproc_ops->enable(ctx);
+}
+
+const struct hantro_postproc_ops hantro_g1_postproc_ops = {
+ .enable = hantro_postproc_g1_enable,
+ .disable = hantro_postproc_g1_disable,
+};
+
+const struct hantro_postproc_ops hantro_g2_postproc_ops = {
+ .enable = hantro_postproc_g2_enable,
+ .disable = hantro_postproc_g2_disable,
+};
diff --git a/drivers/staging/media/hantro/hantro_v4l2.c b/drivers/staging/media/hantro/hantro_v4l2.c
index bcb0bdf..e595905 100644
--- a/drivers/staging/media/hantro/hantro_v4l2.c
+++ b/drivers/staging/media/hantro/hantro_v4l2.c
@@ -23,8 +23,6 @@
#include <media/v4l2-ctrls.h>
#include <media/v4l2-event.h>
#include <media/v4l2-mem2mem.h>
-#include <media/videobuf2-core.h>
-#include <media/videobuf2-dma-sg.h>
#include "hantro.h"
#include "hantro_hw.h"
@@ -285,6 +283,16 @@ static int hantro_try_fmt(const struct hantro_ctx *ctx,
pix_mp->plane_fmt[0].sizeimage +=
hantro_h264_mv_size(pix_mp->width,
pix_mp->height);
+ else if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_VP9_FRAME &&
+ !hantro_needs_postproc(ctx, fmt))
+ pix_mp->plane_fmt[0].sizeimage +=
+ hantro_vp9_mv_size(pix_mp->width,
+ pix_mp->height);
+ else if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_HEVC_SLICE &&
+ !hantro_needs_postproc(ctx, fmt))
+ pix_mp->plane_fmt[0].sizeimage +=
+ hantro_hevc_mv_size(pix_mp->width,
+ pix_mp->height);
} else if (!pix_mp->plane_fmt[0].sizeimage) {
/*
* For coded formats the application can specify
@@ -393,6 +401,7 @@ hantro_update_requires_request(struct hantro_ctx *ctx, u32 fourcc)
case V4L2_PIX_FMT_VP8_FRAME:
case V4L2_PIX_FMT_H264_SLICE:
case V4L2_PIX_FMT_HEVC_SLICE:
+ case V4L2_PIX_FMT_VP9_FRAME:
ctx->fh.m2m_ctx->out_q_ctx.q.requires_requests = true;
break;
default:
diff --git a/drivers/staging/media/hantro/hantro_vp9.c b/drivers/staging/media/hantro/hantro_vp9.c
new file mode 100644
index 0000000..566cd37
--- /dev/null
+++ b/drivers/staging/media/hantro/hantro_vp9.c
@@ -0,0 +1,240 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Hantro VP9 codec driver
+ *
+ * Copyright (C) 2021 Collabora Ltd.
+ */
+
+#include <linux/types.h>
+#include <media/v4l2-mem2mem.h>
+
+#include "hantro.h"
+#include "hantro_hw.h"
+#include "hantro_vp9.h"
+
+#define POW2(x) (1 << (x))
+
+#define MAX_LOG2_TILE_COLUMNS 6
+#define MAX_NUM_TILE_COLS POW2(MAX_LOG2_TILE_COLUMNS)
+#define MAX_TILE_COLS 20
+#define MAX_TILE_ROWS 22
+
+static size_t hantro_vp9_tile_filter_size(unsigned int height)
+{
+ u32 h, height32, size;
+
+ h = roundup(height, 8);
+
+ height32 = roundup(h, 64);
+ size = 24 * height32 * (MAX_NUM_TILE_COLS - 1); /* luma: 8, chroma: 8 + 8 */
+
+ return size;
+}
+
+static size_t hantro_vp9_bsd_control_size(unsigned int height)
+{
+ u32 h, height32;
+
+ h = roundup(height, 8);
+ height32 = roundup(h, 64);
+
+ return 16 * (height32 / 4) * (MAX_NUM_TILE_COLS - 1);
+}
+
+static size_t hantro_vp9_segment_map_size(unsigned int width, unsigned int height)
+{
+ u32 w, h;
+ int num_ctbs;
+
+ w = roundup(width, 8);
+ h = roundup(height, 8);
+ num_ctbs = ((w + 63) / 64) * ((h + 63) / 64);
+
+ return num_ctbs * 32;
+}
+
+static inline size_t hantro_vp9_prob_tab_size(void)
+{
+ return roundup(sizeof(struct hantro_g2_all_probs), 16);
+}
+
+static inline size_t hantro_vp9_count_tab_size(void)
+{
+ return roundup(sizeof(struct symbol_counts), 16);
+}
+
+static inline size_t hantro_vp9_tile_info_size(void)
+{
+ return roundup((MAX_TILE_COLS * MAX_TILE_ROWS * 4 * sizeof(u16) + 15 + 16) & ~0xf, 16);
+}
+
+static void *get_coeffs_arr(struct symbol_counts *cnts, int i, int j, int k, int l, int m)
+{
+ if (i == 0)
+ return &cnts->count_coeffs[j][k][l][m];
+
+ if (i == 1)
+ return &cnts->count_coeffs8x8[j][k][l][m];
+
+ if (i == 2)
+ return &cnts->count_coeffs16x16[j][k][l][m];
+
+ if (i == 3)
+ return &cnts->count_coeffs32x32[j][k][l][m];
+
+ return NULL;
+}
+
+static void *get_eobs1(struct symbol_counts *cnts, int i, int j, int k, int l, int m)
+{
+ if (i == 0)
+ return &cnts->count_coeffs[j][k][l][m][3];
+
+ if (i == 1)
+ return &cnts->count_coeffs8x8[j][k][l][m][3];
+
+ if (i == 2)
+ return &cnts->count_coeffs16x16[j][k][l][m][3];
+
+ if (i == 3)
+ return &cnts->count_coeffs32x32[j][k][l][m][3];
+
+ return NULL;
+}
+
+#define INNER_LOOP \
+ do { \
+ for (m = 0; m < ARRAY_SIZE(vp9_ctx->cnts.coeff[i][0][0][0]); ++m) { \
+ vp9_ctx->cnts.coeff[i][j][k][l][m] = \
+ get_coeffs_arr(cnts, i, j, k, l, m); \
+ vp9_ctx->cnts.eob[i][j][k][l][m][0] = \
+ &cnts->count_eobs[i][j][k][l][m]; \
+ vp9_ctx->cnts.eob[i][j][k][l][m][1] = \
+ get_eobs1(cnts, i, j, k, l, m); \
+ } \
+ } while (0)
+
+static void init_v4l2_vp9_count_tbl(struct hantro_ctx *ctx)
+{
+ struct hantro_vp9_dec_hw_ctx *vp9_ctx = &ctx->vp9_dec;
+ struct symbol_counts *cnts = vp9_ctx->misc.cpu + vp9_ctx->ctx_counters_offset;
+ int i, j, k, l, m;
+
+ vp9_ctx->cnts.partition = &cnts->partition_counts;
+ vp9_ctx->cnts.skip = &cnts->mbskip_count;
+ vp9_ctx->cnts.intra_inter = &cnts->intra_inter_count;
+ vp9_ctx->cnts.tx32p = &cnts->tx32x32_count;
+ /*
+ * g2 hardware uses tx16x16_count[2][3], while the api
+ * expects tx16p[2][4], so this must be explicitly copied
+ * into vp9_ctx->cnts.tx16p when passing the data to the
+ * vp9 library function
+ */
+ vp9_ctx->cnts.tx8p = &cnts->tx8x8_count;
+
+ vp9_ctx->cnts.y_mode = &cnts->sb_ymode_counts;
+ vp9_ctx->cnts.uv_mode = &cnts->uv_mode_counts;
+ vp9_ctx->cnts.comp = &cnts->comp_inter_count;
+ vp9_ctx->cnts.comp_ref = &cnts->comp_ref_count;
+ vp9_ctx->cnts.single_ref = &cnts->single_ref_count;
+ vp9_ctx->cnts.filter = &cnts->switchable_interp_counts;
+ vp9_ctx->cnts.mv_joint = &cnts->mv_counts.joints;
+ vp9_ctx->cnts.sign = &cnts->mv_counts.sign;
+ vp9_ctx->cnts.classes = &cnts->mv_counts.classes;
+ vp9_ctx->cnts.class0 = &cnts->mv_counts.class0;
+ vp9_ctx->cnts.bits = &cnts->mv_counts.bits;
+ vp9_ctx->cnts.class0_fp = &cnts->mv_counts.class0_fp;
+ vp9_ctx->cnts.fp = &cnts->mv_counts.fp;
+ vp9_ctx->cnts.class0_hp = &cnts->mv_counts.class0_hp;
+ vp9_ctx->cnts.hp = &cnts->mv_counts.hp;
+
+ for (i = 0; i < ARRAY_SIZE(vp9_ctx->cnts.coeff); ++i)
+ for (j = 0; j < ARRAY_SIZE(vp9_ctx->cnts.coeff[i]); ++j)
+ for (k = 0; k < ARRAY_SIZE(vp9_ctx->cnts.coeff[i][0]); ++k)
+ for (l = 0; l < ARRAY_SIZE(vp9_ctx->cnts.coeff[i][0][0]); ++l)
+ INNER_LOOP;
+}
+
+int hantro_vp9_dec_init(struct hantro_ctx *ctx)
+{
+ struct hantro_dev *vpu = ctx->dev;
+ const struct hantro_variant *variant = vpu->variant;
+ struct hantro_vp9_dec_hw_ctx *vp9_dec = &ctx->vp9_dec;
+ struct hantro_aux_buf *tile_edge = &vp9_dec->tile_edge;
+ struct hantro_aux_buf *segment_map = &vp9_dec->segment_map;
+ struct hantro_aux_buf *misc = &vp9_dec->misc;
+ u32 i, max_width, max_height, size;
+
+ if (variant->num_dec_fmts < 1)
+ return -EINVAL;
+
+ for (i = 0; i < variant->num_dec_fmts; ++i)
+ if (variant->dec_fmts[i].fourcc == V4L2_PIX_FMT_VP9_FRAME)
+ break;
+
+ if (i == variant->num_dec_fmts)
+ return -EINVAL;
+
+ max_width = vpu->variant->dec_fmts[i].frmsize.max_width;
+ max_height = vpu->variant->dec_fmts[i].frmsize.max_height;
+
+ size = hantro_vp9_tile_filter_size(max_height);
+ vp9_dec->bsd_ctrl_offset = size;
+ size += hantro_vp9_bsd_control_size(max_height);
+
+ tile_edge->cpu = dma_alloc_coherent(vpu->dev, size, &tile_edge->dma, GFP_KERNEL);
+ if (!tile_edge->cpu)
+ return -ENOMEM;
+
+ tile_edge->size = size;
+ memset(tile_edge->cpu, 0, size);
+
+ size = hantro_vp9_segment_map_size(max_width, max_height);
+ vp9_dec->segment_map_size = size;
+ size *= 2; /* we need two areas of this size, used alternately */
+
+ segment_map->cpu = dma_alloc_coherent(vpu->dev, size, &segment_map->dma, GFP_KERNEL);
+ if (!segment_map->cpu)
+ goto err_segment_map;
+
+ segment_map->size = size;
+ memset(segment_map->cpu, 0, size);
+
+ size = hantro_vp9_prob_tab_size();
+ vp9_dec->ctx_counters_offset = size;
+ size += hantro_vp9_count_tab_size();
+ vp9_dec->tile_info_offset = size;
+ size += hantro_vp9_tile_info_size();
+
+ misc->cpu = dma_alloc_coherent(vpu->dev, size, &misc->dma, GFP_KERNEL);
+ if (!misc->cpu)
+ goto err_misc;
+
+ misc->size = size;
+ memset(misc->cpu, 0, size);
+
+ init_v4l2_vp9_count_tbl(ctx);
+
+ return 0;
+
+err_misc:
+ dma_free_coherent(vpu->dev, segment_map->size, segment_map->cpu, segment_map->dma);
+
+err_segment_map:
+ dma_free_coherent(vpu->dev, tile_edge->size, tile_edge->cpu, tile_edge->dma);
+
+ return -ENOMEM;
+}
+
+void hantro_vp9_dec_exit(struct hantro_ctx *ctx)
+{
+ struct hantro_dev *vpu = ctx->dev;
+ struct hantro_vp9_dec_hw_ctx *vp9_dec = &ctx->vp9_dec;
+ struct hantro_aux_buf *tile_edge = &vp9_dec->tile_edge;
+ struct hantro_aux_buf *segment_map = &vp9_dec->segment_map;
+ struct hantro_aux_buf *misc = &vp9_dec->misc;
+
+ dma_free_coherent(vpu->dev, misc->size, misc->cpu, misc->dma);
+ dma_free_coherent(vpu->dev, segment_map->size, segment_map->cpu, segment_map->dma);
+ dma_free_coherent(vpu->dev, tile_edge->size, tile_edge->cpu, tile_edge->dma);
+}
diff --git a/drivers/staging/media/hantro/hantro_vp9.h b/drivers/staging/media/hantro/hantro_vp9.h
new file mode 100644
index 0000000..26b6927
--- /dev/null
+++ b/drivers/staging/media/hantro/hantro_vp9.h
@@ -0,0 +1,102 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Hantro VP9 codec driver
+ *
+ * Copyright (C) 2021 Collabora Ltd.
+ */
+
+struct hantro_g2_mv_probs {
+ u8 joint[3];
+ u8 sign[2];
+ u8 class0_bit[2][1];
+ u8 fr[2][3];
+ u8 class0_hp[2];
+ u8 hp[2];
+ u8 classes[2][10];
+ u8 class0_fr[2][2][3];
+ u8 bits[2][10];
+};
+
+struct hantro_g2_probs {
+ u8 inter_mode[7][4];
+ u8 is_inter[4];
+ u8 uv_mode[10][8];
+ u8 tx8[2][1];
+ u8 tx16[2][2];
+ u8 tx32[2][3];
+ u8 y_mode_tail[4][1];
+ u8 y_mode[4][8];
+ u8 partition[2][16][4]; /* [keyframe][][], [inter][][] */
+ u8 uv_mode_tail[10][1];
+ u8 interp_filter[4][2];
+ u8 comp_mode[5];
+ u8 skip[3];
+
+ u8 pad1[1];
+
+ struct hantro_g2_mv_probs mv;
+
+ u8 single_ref[5][2];
+ u8 comp_ref[5];
+
+ u8 pad2[17];
+
+ u8 coef[4][2][2][6][6][4];
+};
+
+struct hantro_g2_all_probs {
+ u8 kf_y_mode_prob[10][10][8];
+
+ u8 kf_y_mode_prob_tail[10][10][1];
+ u8 ref_pred_probs[3];
+ u8 mb_segment_tree_probs[7];
+ u8 segment_pred_probs[3];
+ u8 ref_scores[4];
+ u8 prob_comppred[2];
+
+ u8 pad1[9];
+
+ u8 kf_uv_mode_prob[10][8];
+ u8 kf_uv_mode_prob_tail[10][1];
+
+ u8 pad2[6];
+
+ struct hantro_g2_probs probs;
+};
+
+struct mv_counts {
+ u32 joints[4];
+ u32 sign[2][2];
+ u32 classes[2][11];
+ u32 class0[2][2];
+ u32 bits[2][10][2];
+ u32 class0_fp[2][2][4];
+ u32 fp[2][4];
+ u32 class0_hp[2][2];
+ u32 hp[2][2];
+};
+
+struct symbol_counts {
+ u32 inter_mode_counts[7][3][2];
+ u32 sb_ymode_counts[4][10];
+ u32 uv_mode_counts[10][10];
+ u32 partition_counts[16][4];
+ u32 switchable_interp_counts[4][3];
+ u32 intra_inter_count[4][2];
+ u32 comp_inter_count[5][2];
+ u32 single_ref_count[5][2][2];
+ u32 comp_ref_count[5][2];
+ u32 tx32x32_count[2][4];
+ u32 tx16x16_count[2][3];
+ u32 tx8x8_count[2][2];
+ u32 mbskip_count[3][2];
+
+ struct mv_counts mv_counts;
+
+ u32 count_coeffs[2][2][6][6][4];
+ u32 count_coeffs8x8[2][2][6][6][4];
+ u32 count_coeffs16x16[2][2][6][6][4];
+ u32 count_coeffs32x32[2][2][6][6][4];
+
+ u32 count_eobs[4][2][2][6][6];
+};
diff --git a/drivers/staging/media/hantro/imx8m_vpu_hw.c b/drivers/staging/media/hantro/imx8m_vpu_hw.c
index ea919bf..f5991b8 100644
--- a/drivers/staging/media/hantro/imx8m_vpu_hw.c
+++ b/drivers/staging/media/hantro/imx8m_vpu_hw.c
@@ -82,6 +82,7 @@ static const struct hantro_fmt imx8m_vpu_postproc_fmts[] = {
{
.fourcc = V4L2_PIX_FMT_YUYV,
.codec_mode = HANTRO_MODE_NONE,
+ .postprocessed = true,
},
};
@@ -131,10 +132,18 @@ static const struct hantro_fmt imx8m_vpu_dec_fmts[] = {
},
};
-static const struct hantro_fmt imx8m_vpu_g2_dec_fmts[] = {
+static const struct hantro_fmt imx8m_vpu_g2_postproc_fmts[] = {
{
.fourcc = V4L2_PIX_FMT_NV12,
.codec_mode = HANTRO_MODE_NONE,
+ .postprocessed = true,
+ },
+};
+
+static const struct hantro_fmt imx8m_vpu_g2_dec_fmts[] = {
+ {
+ .fourcc = V4L2_PIX_FMT_NV12_4L4,
+ .codec_mode = HANTRO_MODE_NONE,
},
{
.fourcc = V4L2_PIX_FMT_HEVC_SLICE,
@@ -149,6 +158,19 @@ static const struct hantro_fmt imx8m_vpu_g2_dec_fmts[] = {
.step_height = MB_DIM,
},
},
+ {
+ .fourcc = V4L2_PIX_FMT_VP9_FRAME,
+ .codec_mode = HANTRO_MODE_VP9_DEC,
+ .max_depth = 2,
+ .frmsize = {
+ .min_width = 48,
+ .max_width = 3840,
+ .step_width = MB_DIM,
+ .min_height = 48,
+ .max_height = 2160,
+ .step_height = MB_DIM,
+ },
+ },
};
static irqreturn_t imx8m_vpu_g1_irq(int irq, void *dev_id)
@@ -169,24 +191,6 @@ static irqreturn_t imx8m_vpu_g1_irq(int irq, void *dev_id)
return IRQ_HANDLED;
}
-static irqreturn_t imx8m_vpu_g2_irq(int irq, void *dev_id)
-{
- struct hantro_dev *vpu = dev_id;
- enum vb2_buffer_state state;
- u32 status;
-
- status = vdpu_read(vpu, G2_REG_INTERRUPT);
- state = (status & G2_REG_INTERRUPT_DEC_RDY_INT) ?
- VB2_BUF_STATE_DONE : VB2_BUF_STATE_ERROR;
-
- vdpu_write(vpu, 0, G2_REG_INTERRUPT);
- vdpu_write(vpu, G2_REG_CONFIG_DEC_CLK_GATE_E, G2_REG_CONFIG);
-
- hantro_irq_done(vpu, state);
-
- return IRQ_HANDLED;
-}
-
static int imx8mq_vpu_hw_init(struct hantro_dev *vpu)
{
vpu->ctrl_base = vpu->reg_bases[vpu->variant->num_regs - 1];
@@ -240,6 +244,13 @@ static const struct hantro_codec_ops imx8mq_vpu_g2_codec_ops[] = {
.init = hantro_hevc_dec_init,
.exit = hantro_hevc_dec_exit,
},
+ [HANTRO_MODE_VP9_DEC] = {
+ .run = hantro_g2_vp9_dec_run,
+ .done = hantro_g2_vp9_dec_done,
+ .reset = imx8m_vpu_g2_reset,
+ .init = hantro_vp9_dec_init,
+ .exit = hantro_vp9_dec_exit,
+ },
};
/*
@@ -251,7 +262,7 @@ static const struct hantro_irq imx8mq_irqs[] = {
};
static const struct hantro_irq imx8mq_g2_irqs[] = {
- { "g2", imx8m_vpu_g2_irq },
+ { "g2", hantro_g2_irq },
};
static const char * const imx8mq_clk_names[] = { "g1", "g2", "bus" };
@@ -262,7 +273,7 @@ const struct hantro_variant imx8mq_vpu_variant = {
.num_dec_fmts = ARRAY_SIZE(imx8m_vpu_dec_fmts),
.postproc_fmts = imx8m_vpu_postproc_fmts,
.num_postproc_fmts = ARRAY_SIZE(imx8m_vpu_postproc_fmts),
- .postproc_regs = &hantro_g1_postproc_regs,
+ .postproc_ops = &hantro_g1_postproc_ops,
.codec = HANTRO_MPEG2_DECODER | HANTRO_VP8_DECODER |
HANTRO_H264_DECODER,
.codec_ops = imx8mq_vpu_codec_ops,
@@ -280,7 +291,10 @@ const struct hantro_variant imx8mq_vpu_g2_variant = {
.dec_offset = 0x0,
.dec_fmts = imx8m_vpu_g2_dec_fmts,
.num_dec_fmts = ARRAY_SIZE(imx8m_vpu_g2_dec_fmts),
- .codec = HANTRO_HEVC_DECODER,
+ .postproc_fmts = imx8m_vpu_g2_postproc_fmts,
+ .num_postproc_fmts = ARRAY_SIZE(imx8m_vpu_g2_postproc_fmts),
+ .postproc_ops = &hantro_g2_postproc_ops,
+ .codec = HANTRO_HEVC_DECODER | HANTRO_VP9_DECODER,
.codec_ops = imx8mq_vpu_g2_codec_ops,
.init = imx8mq_vpu_hw_init,
.runtime_resume = imx8mq_runtime_resume,
diff --git a/drivers/staging/media/hantro/rockchip_vpu2_hw_jpeg_enc.c b/drivers/staging/media/hantro/rockchip_vpu2_hw_jpeg_enc.c
index 991213c..4df16f5 100644
--- a/drivers/staging/media/hantro/rockchip_vpu2_hw_jpeg_enc.c
+++ b/drivers/staging/media/hantro/rockchip_vpu2_hw_jpeg_enc.c
@@ -143,9 +143,8 @@ int rockchip_vpu2_jpeg_enc_run(struct hantro_ctx *ctx)
rockchip_vpu2_set_src_img_ctrl(vpu, ctx);
rockchip_vpu2_jpeg_enc_set_buffers(vpu, ctx, &src_buf->vb2_buf);
- rockchip_vpu2_jpeg_enc_set_qtable(vpu,
- hantro_jpeg_get_qtable(0),
- hantro_jpeg_get_qtable(1));
+ rockchip_vpu2_jpeg_enc_set_qtable(vpu, jpeg_ctx.hw_luma_qtable,
+ jpeg_ctx.hw_chroma_qtable);
reg = VEPU_REG_OUTPUT_SWAP32
| VEPU_REG_OUTPUT_SWAP16
@@ -171,3 +170,20 @@ int rockchip_vpu2_jpeg_enc_run(struct hantro_ctx *ctx)
return 0;
}
+
+void rockchip_vpu2_jpeg_enc_done(struct hantro_ctx *ctx)
+{
+ struct hantro_dev *vpu = ctx->dev;
+ u32 bytesused = vepu_read(vpu, VEPU_REG_STR_BUF_LIMIT) / 8;
+ struct vb2_v4l2_buffer *dst_buf = hantro_get_dst_buf(ctx);
+
+ /*
+ * TODO: Rework the JPEG encoder to eliminate the need
+ * for a bounce buffer.
+ */
+ memcpy(vb2_plane_vaddr(&dst_buf->vb2_buf, 0) +
+ ctx->vpu_dst_fmt->header_size,
+ ctx->jpeg_enc.bounce_buffer.cpu, bytesused);
+ vb2_set_plane_payload(&dst_buf->vb2_buf, 0,
+ ctx->vpu_dst_fmt->header_size + bytesused);
+}
diff --git a/drivers/staging/media/hantro/rockchip_vpu_hw.c b/drivers/staging/media/hantro/rockchip_vpu_hw.c
index d4f5295..c203b60 100644
--- a/drivers/staging/media/hantro/rockchip_vpu_hw.c
+++ b/drivers/staging/media/hantro/rockchip_vpu_hw.c
@@ -62,6 +62,7 @@ static const struct hantro_fmt rockchip_vpu1_postproc_fmts[] = {
{
.fourcc = V4L2_PIX_FMT_YUYV,
.codec_mode = HANTRO_MODE_NONE,
+ .postprocessed = true,
},
};
@@ -343,7 +344,7 @@ static const struct hantro_codec_ops rk3066_vpu_codec_ops[] = {
.run = hantro_h1_jpeg_enc_run,
.reset = rockchip_vpu1_enc_reset,
.init = hantro_jpeg_enc_init,
- .done = hantro_jpeg_enc_done,
+ .done = hantro_h1_jpeg_enc_done,
.exit = hantro_jpeg_enc_exit,
},
[HANTRO_MODE_H264_DEC] = {
@@ -371,7 +372,7 @@ static const struct hantro_codec_ops rk3288_vpu_codec_ops[] = {
.run = hantro_h1_jpeg_enc_run,
.reset = rockchip_vpu1_enc_reset,
.init = hantro_jpeg_enc_init,
- .done = hantro_jpeg_enc_done,
+ .done = hantro_h1_jpeg_enc_done,
.exit = hantro_jpeg_enc_exit,
},
[HANTRO_MODE_H264_DEC] = {
@@ -399,6 +400,7 @@ static const struct hantro_codec_ops rk3399_vpu_codec_ops[] = {
.run = rockchip_vpu2_jpeg_enc_run,
.reset = rockchip_vpu2_enc_reset,
.init = hantro_jpeg_enc_init,
+ .done = rockchip_vpu2_jpeg_enc_done,
.exit = hantro_jpeg_enc_exit,
},
[HANTRO_MODE_H264_DEC] = {
@@ -460,7 +462,7 @@ const struct hantro_variant rk3036_vpu_variant = {
.num_dec_fmts = ARRAY_SIZE(rk3066_vpu_dec_fmts),
.postproc_fmts = rockchip_vpu1_postproc_fmts,
.num_postproc_fmts = ARRAY_SIZE(rockchip_vpu1_postproc_fmts),
- .postproc_regs = &hantro_g1_postproc_regs,
+ .postproc_ops = &hantro_g1_postproc_ops,
.codec = HANTRO_MPEG2_DECODER | HANTRO_VP8_DECODER |
HANTRO_H264_DECODER,
.codec_ops = rk3036_vpu_codec_ops,
@@ -485,7 +487,7 @@ const struct hantro_variant rk3066_vpu_variant = {
.num_dec_fmts = ARRAY_SIZE(rk3066_vpu_dec_fmts),
.postproc_fmts = rockchip_vpu1_postproc_fmts,
.num_postproc_fmts = ARRAY_SIZE(rockchip_vpu1_postproc_fmts),
- .postproc_regs = &hantro_g1_postproc_regs,
+ .postproc_ops = &hantro_g1_postproc_ops,
.codec = HANTRO_JPEG_ENCODER | HANTRO_MPEG2_DECODER |
HANTRO_VP8_DECODER | HANTRO_H264_DECODER,
.codec_ops = rk3066_vpu_codec_ops,
@@ -505,7 +507,7 @@ const struct hantro_variant rk3288_vpu_variant = {
.num_dec_fmts = ARRAY_SIZE(rk3288_vpu_dec_fmts),
.postproc_fmts = rockchip_vpu1_postproc_fmts,
.num_postproc_fmts = ARRAY_SIZE(rockchip_vpu1_postproc_fmts),
- .postproc_regs = &hantro_g1_postproc_regs,
+ .postproc_ops = &hantro_g1_postproc_ops,
.codec = HANTRO_JPEG_ENCODER | HANTRO_MPEG2_DECODER |
HANTRO_VP8_DECODER | HANTRO_H264_DECODER,
.codec_ops = rk3288_vpu_codec_ops,
diff --git a/drivers/staging/media/hantro/sama5d4_vdec_hw.c b/drivers/staging/media/hantro/sama5d4_vdec_hw.c
index 9c3b8cd..b2fc1c5 100644
--- a/drivers/staging/media/hantro/sama5d4_vdec_hw.c
+++ b/drivers/staging/media/hantro/sama5d4_vdec_hw.c
@@ -15,6 +15,7 @@ static const struct hantro_fmt sama5d4_vdec_postproc_fmts[] = {
{
.fourcc = V4L2_PIX_FMT_YUYV,
.codec_mode = HANTRO_MODE_NONE,
+ .postprocessed = true,
},
};
@@ -100,7 +101,7 @@ const struct hantro_variant sama5d4_vdec_variant = {
.num_dec_fmts = ARRAY_SIZE(sama5d4_vdec_fmts),
.postproc_fmts = sama5d4_vdec_postproc_fmts,
.num_postproc_fmts = ARRAY_SIZE(sama5d4_vdec_postproc_fmts),
- .postproc_regs = &hantro_g1_postproc_regs,
+ .postproc_ops = &hantro_g1_postproc_ops,
.codec = HANTRO_MPEG2_DECODER | HANTRO_VP8_DECODER |
HANTRO_H264_DECODER,
.codec_ops = sama5d4_vdec_codec_ops,
diff --git a/drivers/staging/media/hantro/sunxi_vpu_hw.c b/drivers/staging/media/hantro/sunxi_vpu_hw.c
new file mode 100644
index 0000000..9063340
--- /dev/null
+++ b/drivers/staging/media/hantro/sunxi_vpu_hw.c
@@ -0,0 +1,86 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Allwinner Hantro G2 VPU codec driver
+ *
+ * Copyright (C) 2021 Jernej Skrabec <jernej.skrabec@gmail.com>
+ */
+
+#include <linux/clk.h>
+
+#include "hantro.h"
+
+static const struct hantro_fmt sunxi_vpu_postproc_fmts[] = {
+ {
+ .fourcc = V4L2_PIX_FMT_NV12,
+ .codec_mode = HANTRO_MODE_NONE,
+ .postprocessed = true,
+ },
+};
+
+static const struct hantro_fmt sunxi_vpu_dec_fmts[] = {
+ {
+ .fourcc = V4L2_PIX_FMT_NV12_4L4,
+ .codec_mode = HANTRO_MODE_NONE,
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_VP9_FRAME,
+ .codec_mode = HANTRO_MODE_VP9_DEC,
+ .max_depth = 2,
+ .frmsize = {
+ .min_width = 48,
+ .max_width = 3840,
+ .step_width = MB_DIM,
+ .min_height = 48,
+ .max_height = 2160,
+ .step_height = MB_DIM,
+ },
+ },
+};
+
+static int sunxi_vpu_hw_init(struct hantro_dev *vpu)
+{
+ clk_set_rate(vpu->clocks[0].clk, 300000000);
+
+ return 0;
+}
+
+static void sunxi_vpu_reset(struct hantro_ctx *ctx)
+{
+ struct hantro_dev *vpu = ctx->dev;
+
+ reset_control_reset(vpu->resets);
+}
+
+static const struct hantro_codec_ops sunxi_vpu_codec_ops[] = {
+ [HANTRO_MODE_VP9_DEC] = {
+ .run = hantro_g2_vp9_dec_run,
+ .done = hantro_g2_vp9_dec_done,
+ .reset = sunxi_vpu_reset,
+ .init = hantro_vp9_dec_init,
+ .exit = hantro_vp9_dec_exit,
+ },
+};
+
+static const struct hantro_irq sunxi_irqs[] = {
+ { NULL, hantro_g2_irq },
+};
+
+static const char * const sunxi_clk_names[] = { "mod", "bus" };
+
+const struct hantro_variant sunxi_vpu_variant = {
+ .dec_fmts = sunxi_vpu_dec_fmts,
+ .num_dec_fmts = ARRAY_SIZE(sunxi_vpu_dec_fmts),
+ .postproc_fmts = sunxi_vpu_postproc_fmts,
+ .num_postproc_fmts = ARRAY_SIZE(sunxi_vpu_postproc_fmts),
+ .postproc_ops = &hantro_g2_postproc_ops,
+ .codec = HANTRO_VP9_DECODER,
+ .codec_ops = sunxi_vpu_codec_ops,
+ .init = sunxi_vpu_hw_init,
+ .irqs = sunxi_irqs,
+ .num_irqs = ARRAY_SIZE(sunxi_irqs),
+ .clk_names = sunxi_clk_names,
+ .num_clocks = ARRAY_SIZE(sunxi_clk_names),
+ .double_buffer = 1,
+ .legacy_regs = 1,
+ .late_postproc = 1,
+};
diff --git a/drivers/staging/media/imx/imx-media-csc-scaler.c b/drivers/staging/media/imx/imx-media-csc-scaler.c
index eb6da9b..1fd39a2 100644
--- a/drivers/staging/media/imx/imx-media-csc-scaler.c
+++ b/drivers/staging/media/imx/imx-media-csc-scaler.c
@@ -820,7 +820,7 @@ static const struct v4l2_file_operations ipu_csc_scaler_fops = {
.mmap = v4l2_m2m_fop_mmap,
};
-static struct v4l2_m2m_ops m2m_ops = {
+static const struct v4l2_m2m_ops m2m_ops = {
.device_run = device_run,
.job_abort = job_abort,
};
diff --git a/drivers/staging/media/imx/imx-media-csi.c b/drivers/staging/media/imx/imx-media-csi.c
index 1caa100..bd7f156 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -150,7 +150,7 @@ static inline bool requires_passthrough(struct v4l2_fwnode_endpoint *ep,
const struct imx_media_pixfmt *incc)
{
if (ep->bus_type == V4L2_MBUS_BT656) // including BT.1120
- return 0;
+ return false;
return incc->bayer || is_parallel_16bit_bus(ep) ||
(is_parallel_bus(ep) &&
diff --git a/drivers/staging/media/imx/imx-media-utils.c b/drivers/staging/media/imx/imx-media-utils.c
index 6f90acf..94bc866 100644
--- a/drivers/staging/media/imx/imx-media-utils.c
+++ b/drivers/staging/media/imx/imx-media-utils.c
@@ -569,48 +569,6 @@ int imx_media_mbus_fmt_to_pix_fmt(struct v4l2_pix_format *pix,
}
EXPORT_SYMBOL_GPL(imx_media_mbus_fmt_to_pix_fmt);
-int imx_media_mbus_fmt_to_ipu_image(struct ipu_image *image,
- const struct v4l2_mbus_framefmt *mbus)
-{
- int ret;
-
- memset(image, 0, sizeof(*image));
-
- ret = imx_media_mbus_fmt_to_pix_fmt(&image->pix, mbus, NULL);
- if (ret)
- return ret;
-
- image->rect.width = mbus->width;
- image->rect.height = mbus->height;
-
- return 0;
-}
-EXPORT_SYMBOL_GPL(imx_media_mbus_fmt_to_ipu_image);
-
-int imx_media_ipu_image_to_mbus_fmt(struct v4l2_mbus_framefmt *mbus,
- const struct ipu_image *image)
-{
- const struct imx_media_pixfmt *fmt;
-
- fmt = imx_media_find_pixel_format(image->pix.pixelformat,
- PIXFMT_SEL_ANY);
- if (!fmt || !fmt->codes || !fmt->codes[0])
- return -EINVAL;
-
- memset(mbus, 0, sizeof(*mbus));
- mbus->width = image->pix.width;
- mbus->height = image->pix.height;
- mbus->code = fmt->codes[0];
- mbus->field = image->pix.field;
- mbus->colorspace = image->pix.colorspace;
- mbus->xfer_func = image->pix.xfer_func;
- mbus->ycbcr_enc = image->pix.ycbcr_enc;
- mbus->quantization = image->pix.quantization;
-
- return 0;
-}
-EXPORT_SYMBOL_GPL(imx_media_ipu_image_to_mbus_fmt);
-
void imx_media_free_dma_buf(struct device *dev,
struct imx_media_dma_buf *buf)
{
diff --git a/drivers/staging/media/imx/imx-media.h b/drivers/staging/media/imx/imx-media.h
index 6740e79..f263fc3 100644
--- a/drivers/staging/media/imx/imx-media.h
+++ b/drivers/staging/media/imx/imx-media.h
@@ -199,10 +199,6 @@ void imx_media_try_colorimetry(struct v4l2_mbus_framefmt *tryfmt,
int imx_media_mbus_fmt_to_pix_fmt(struct v4l2_pix_format *pix,
const struct v4l2_mbus_framefmt *mbus,
const struct imx_media_pixfmt *cc);
-int imx_media_mbus_fmt_to_ipu_image(struct ipu_image *image,
- const struct v4l2_mbus_framefmt *mbus);
-int imx_media_ipu_image_to_mbus_fmt(struct v4l2_mbus_framefmt *mbus,
- const struct ipu_image *image);
void imx_media_grp_id_to_sd_name(char *sd_name, int sz,
u32 grp_id, int ipu_id);
struct v4l2_subdev *
diff --git a/drivers/staging/media/imx/imx6-mipi-csi2.c b/drivers/staging/media/imx/imx6-mipi-csi2.c
index a0941fc29..558b256 100644
--- a/drivers/staging/media/imx/imx6-mipi-csi2.c
+++ b/drivers/staging/media/imx/imx6-mipi-csi2.c
@@ -382,13 +382,17 @@ static int csi2_start(struct csi2_dev *csi2)
csi2_enable(csi2, true);
/* Step 5 */
+ ret = v4l2_subdev_call(csi2->src_sd, video, pre_streamon,
+ V4L2_SUBDEV_PRE_STREAMON_FL_MANUAL_LP);
+ if (ret && ret != -ENOIOCTLCMD)
+ goto err_assert_reset;
csi2_dphy_wait_stopstate(csi2, lanes);
/* Step 6 */
ret = v4l2_subdev_call(csi2->src_sd, video, s_stream, 1);
ret = (ret && ret != -ENOIOCTLCMD) ? ret : 0;
if (ret)
- goto err_assert_reset;
+ goto err_stop_lp11;
/* Step 7 */
ret = csi2_dphy_wait_clock_lane(csi2);
@@ -399,6 +403,8 @@ static int csi2_start(struct csi2_dev *csi2)
err_stop_upstream:
v4l2_subdev_call(csi2->src_sd, video, s_stream, 0);
+err_stop_lp11:
+ v4l2_subdev_call(csi2->src_sd, video, post_streamoff);
err_assert_reset:
csi2_enable(csi2, false);
err_disable_clk:
@@ -410,6 +416,7 @@ static void csi2_stop(struct csi2_dev *csi2)
{
/* stop upstream */
v4l2_subdev_call(csi2->src_sd, video, s_stream, 0);
+ v4l2_subdev_call(csi2->src_sd, video, post_streamoff);
csi2_enable(csi2, false);
clk_disable_unprepare(csi2->pix_clk);
diff --git a/drivers/staging/media/ipu3/include/uapi/intel-ipu3.h b/drivers/staging/media/ipu3/include/uapi/intel-ipu3.h
index 585f559..dbdd015 100644
--- a/drivers/staging/media/ipu3/include/uapi/intel-ipu3.h
+++ b/drivers/staging/media/ipu3/include/uapi/intel-ipu3.h
@@ -34,11 +34,17 @@
* struct ipu3_uapi_grid_config - Grid plane config
*
* @width: Grid horizontal dimensions, in number of grid blocks(cells).
+ * For AWB, the range is (16, 80).
+ * For AF/AE, the range is (16, 32).
* @height: Grid vertical dimensions, in number of grid cells.
+ * For AWB, the range is (16, 60).
+ * For AF/AE, the range is (16, 24).
* @block_width_log2: Log2 of the width of each cell in pixels.
- * for (2^3, 2^4, 2^5, 2^6, 2^7), values [3, 7].
+ * For AWB, the range is [3, 6].
+ * For AF/AE, the range is [3, 7].
* @block_height_log2: Log2 of the height of each cell in pixels.
- * for (2^3, 2^4, 2^5, 2^6, 2^7), values [3, 7].
+ * For AWB, the range is [3, 6].
+ * For AF/AE, the range is [3, 7].
* @height_per_slice: The number of blocks in vertical axis per slice.
* Default 2.
* @x_start: X value of top left corner of Region of Interest(ROI).
@@ -61,17 +67,39 @@ struct ipu3_uapi_grid_config {
__u16 y_end;
} __packed;
+/**
+ * struct ipu3_uapi_awb_set_item - Memory layout for each cell in AWB
+ *
+ * @Gr_avg: Green average for red lines in the cell.
+ * @R_avg: Red average in the cell.
+ * @B_avg: Blue average in the cell.
+ * @Gb_avg: Green average for blue lines in the cell.
+ * @sat_ratio: Percentage of pixels over the thresholds specified in
+ * ipu3_uapi_awb_config_s, coded from 0 to 255.
+ * @padding0: Unused byte for padding.
+ * @padding1: Unused byte for padding.
+ * @padding2: Unused byte for padding.
+ */
+struct ipu3_uapi_awb_set_item {
+ __u8 Gr_avg;
+ __u8 R_avg;
+ __u8 B_avg;
+ __u8 Gb_avg;
+ __u8 sat_ratio;
+ __u8 padding0;
+ __u8 padding1;
+ __u8 padding2;
+} __attribute__((packed));
+
/*
* The grid based data is divided into "slices" called set, each slice of setX
* refers to ipu3_uapi_grid_config width * height_per_slice.
*/
#define IPU3_UAPI_AWB_MAX_SETS 60
/* Based on grid size 80 * 60 and cell size 16 x 16 */
-#define IPU3_UAPI_AWB_SET_SIZE 1280
-#define IPU3_UAPI_AWB_MD_ITEM_SIZE 8
+#define IPU3_UAPI_AWB_SET_SIZE 160
#define IPU3_UAPI_AWB_SPARE_FOR_BUBBLES \
- (IPU3_UAPI_MAX_BUBBLE_SIZE * IPU3_UAPI_MAX_STRIPES * \
- IPU3_UAPI_AWB_MD_ITEM_SIZE)
+ (IPU3_UAPI_MAX_BUBBLE_SIZE * IPU3_UAPI_MAX_STRIPES)
#define IPU3_UAPI_AWB_MAX_BUFFER_SIZE \
(IPU3_UAPI_AWB_MAX_SETS * \
(IPU3_UAPI_AWB_SET_SIZE + IPU3_UAPI_AWB_SPARE_FOR_BUBBLES))
@@ -83,7 +111,7 @@ struct ipu3_uapi_grid_config {
* the average values for each color channel.
*/
struct ipu3_uapi_awb_raw_buffer {
- __u8 meta_data[IPU3_UAPI_AWB_MAX_BUFFER_SIZE]
+ struct ipu3_uapi_awb_set_item meta_data[IPU3_UAPI_AWB_MAX_BUFFER_SIZE]
__attribute__((aligned(32)));
} __packed;
diff --git a/drivers/staging/media/ipu3/ipu3-css-params.c b/drivers/staging/media/ipu3/ipu3-css-params.c
index e9d6bd9..d9e3c37 100644
--- a/drivers/staging/media/ipu3/ipu3-css-params.c
+++ b/drivers/staging/media/ipu3/ipu3-css-params.c
@@ -771,7 +771,6 @@ static int imgu_css_osys_calc_frame_and_stripe_params(
*/
{
unsigned int i;
- int pin_scale = 0;
/*Input resolution */
stripe_params[s].input_width = stripe_input_width_y;
@@ -791,8 +790,6 @@ static int imgu_css_osys_calc_frame_and_stripe_params(
reso.pin_height[i];
stripe_params[s].output_offset[i] =
stripe_offset_out_y;
-
- pin_scale += frame_params[i].scaled;
} else {
/* Unscaled pin */
stripe_params[s].output_width[i] =
diff --git a/drivers/staging/media/max96712/Kconfig b/drivers/staging/media/max96712/Kconfig
new file mode 100644
index 0000000..acde14f
--- /dev/null
+++ b/drivers/staging/media/max96712/Kconfig
@@ -0,0 +1,14 @@
+# SPDX-License-Identifier: GPL-2.0
+config VIDEO_MAX96712
+ tristate "Maxim MAX96712 Quad GMSL2 Deserializer support"
+ depends on I2C
+ depends on OF_GPIO
+ depends on VIDEO_V4L2
+ select V4L2_FWNODE
+ select VIDEO_V4L2_SUBDEV_API
+ select MEDIA_CONTROLLER
+ help
+ This driver supports the Maxim MAX96712 Quad GMSL2 Deserializer.
+
+ To compile this driver as a module, choose M here: the
+ module will be called max96712.
diff --git a/drivers/staging/media/max96712/Makefile b/drivers/staging/media/max96712/Makefile
new file mode 100644
index 0000000..70c1974
--- /dev/null
+++ b/drivers/staging/media/max96712/Makefile
@@ -0,0 +1,2 @@
+# SPDX-License-Identifier: GPL-2.0
+obj-$(CONFIG_VIDEO_MAX96712) += max96712.o
diff --git a/drivers/staging/media/max96712/max96712.c b/drivers/staging/media/max96712/max96712.c
new file mode 100644
index 0000000..9bc72d9
--- /dev/null
+++ b/drivers/staging/media/max96712/max96712.c
@@ -0,0 +1,440 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Maxim MAX9286 Quad GMSL2 Deserializer Driver
+ *
+ * Copyright (C) 2021 Renesas Electronics Corporation
+ * Copyright (C) 2021 Niklas Söderlund
+ */
+
+#include <linux/delay.h>
+#include <linux/i2c.h>
+#include <linux/module.h>
+#include <linux/of_graph.h>
+#include <linux/regmap.h>
+
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-fwnode.h>
+#include <media/v4l2-subdev.h>
+
+#define MAX96712_ID 0x20
+
+#define MAX96712_DPLL_FREQ 1000
+
+enum max96712_pattern {
+ MAX96712_PATTERN_CHECKERBOARD = 0,
+ MAX96712_PATTERN_GRADIENT,
+};
+
+struct max96712_priv {
+ struct i2c_client *client;
+ struct regmap *regmap;
+ struct gpio_desc *gpiod_pwdn;
+
+ struct v4l2_fwnode_bus_mipi_csi2 mipi;
+
+ struct v4l2_subdev sd;
+ struct v4l2_ctrl_handler ctrl_handler;
+ struct media_pad pads[1];
+
+ enum max96712_pattern pattern;
+};
+
+static int max96712_read(struct max96712_priv *priv, int reg)
+{
+ int ret, val;
+
+ ret = regmap_read(priv->regmap, reg, &val);
+ if (ret) {
+ dev_err(&priv->client->dev, "read 0x%04x failed\n", reg);
+ return ret;
+ }
+
+ return val;
+}
+
+static int max96712_write(struct max96712_priv *priv, unsigned int reg, u8 val)
+{
+ int ret;
+
+ ret = regmap_write(priv->regmap, reg, val);
+ if (ret)
+ dev_err(&priv->client->dev, "write 0x%04x failed\n", reg);
+
+ return ret;
+}
+
+static int max96712_update_bits(struct max96712_priv *priv, unsigned int reg,
+ u8 mask, u8 val)
+{
+ int ret;
+
+ ret = regmap_update_bits(priv->regmap, reg, mask, val);
+ if (ret)
+ dev_err(&priv->client->dev, "update 0x%04x failed\n", reg);
+
+ return ret;
+}
+
+static int max96712_write_bulk(struct max96712_priv *priv, unsigned int reg,
+ const void *val, size_t val_count)
+{
+ int ret;
+
+ ret = regmap_bulk_write(priv->regmap, reg, val, val_count);
+ if (ret)
+ dev_err(&priv->client->dev, "bulk write 0x%04x failed\n", reg);
+
+ return ret;
+}
+
+static int max96712_write_bulk_value(struct max96712_priv *priv,
+ unsigned int reg, unsigned int val,
+ size_t val_count)
+{
+ unsigned int i;
+ u8 values[4];
+
+ for (i = 1; i <= val_count; i++)
+ values[i - 1] = (val >> ((val_count - i) * 8)) & 0xff;
+
+ return max96712_write_bulk(priv, reg, &values, val_count);
+}
+
+static void max96712_reset(struct max96712_priv *priv)
+{
+ max96712_update_bits(priv, 0x13, 0x40, 0x40);
+ msleep(20);
+}
+
+static void max96712_mipi_enable(struct max96712_priv *priv, bool enable)
+{
+ if (enable) {
+ max96712_update_bits(priv, 0x40b, 0x02, 0x02);
+ max96712_update_bits(priv, 0x8a0, 0x80, 0x80);
+ } else {
+ max96712_update_bits(priv, 0x8a0, 0x80, 0x00);
+ max96712_update_bits(priv, 0x40b, 0x02, 0x00);
+ }
+}
+
+static void max96712_mipi_configure(struct max96712_priv *priv)
+{
+ unsigned int i;
+ u8 phy5 = 0;
+
+ max96712_mipi_enable(priv, false);
+
+ /* Select 2x4 mode. */
+ max96712_write(priv, 0x8a0, 0x04);
+
+ /* Configure a 4-lane DPHY using PHY0 and PHY1. */
+ /* TODO: Add support for 2-lane and 1-lane configurations. */
+ /* TODO: Add support CPHY mode. */
+ max96712_write(priv, 0x94a, 0xc0);
+
+ /* Configure lane mapping for PHY0 and PHY1. */
+ /* TODO: Add support for lane swapping. */
+ max96712_write(priv, 0x8a3, 0xe4);
+
+ /* Configure lane polarity for PHY0 and PHY1. */
+ for (i = 0; i < priv->mipi.num_data_lanes + 1; i++)
+ if (priv->mipi.lane_polarities[i])
+ phy5 |= BIT(i == 0 ? 5 : i < 3 ? i - 1 : i);
+ max96712_write(priv, 0x8a5, phy5);
+
+ /* Set link frequency for PHY0 and PHY1. */
+ max96712_update_bits(priv, 0x415, 0x3f,
+ ((MAX96712_DPLL_FREQ / 100) & 0x1f) | BIT(5));
+ max96712_update_bits(priv, 0x418, 0x3f,
+ ((MAX96712_DPLL_FREQ / 100) & 0x1f) | BIT(5));
+
+ /* Enable PHY0 and PHY1 */
+ max96712_update_bits(priv, 0x8a2, 0xf0, 0x30);
+}
+
+static void max96712_pattern_enable(struct max96712_priv *priv, bool enable)
+{
+ const u32 h_active = 1920;
+ const u32 h_fp = 88;
+ const u32 h_sw = 44;
+ const u32 h_bp = 148;
+ const u32 h_tot = h_active + h_fp + h_sw + h_bp;
+
+ const u32 v_active = 1080;
+ const u32 v_fp = 4;
+ const u32 v_sw = 5;
+ const u32 v_bp = 36;
+ const u32 v_tot = v_active + v_fp + v_sw + v_bp;
+
+ if (!enable) {
+ max96712_write(priv, 0x1051, 0x00);
+ return;
+ }
+
+ /* PCLK 75MHz. */
+ max96712_write(priv, 0x0009, 0x01);
+
+ /* Configure Video Timing Generator for 1920x1080 @ 30 fps. */
+ max96712_write_bulk_value(priv, 0x1052, 0, 3);
+ max96712_write_bulk_value(priv, 0x1055, v_sw * h_tot, 3);
+ max96712_write_bulk_value(priv, 0x1058,
+ (v_active + v_fp + + v_bp) * h_tot, 3);
+ max96712_write_bulk_value(priv, 0x105b, 0, 3);
+ max96712_write_bulk_value(priv, 0x105e, h_sw, 2);
+ max96712_write_bulk_value(priv, 0x1060, h_active + h_fp + h_bp, 2);
+ max96712_write_bulk_value(priv, 0x1062, v_tot, 2);
+ max96712_write_bulk_value(priv, 0x1064,
+ h_tot * (v_sw + v_bp) + (h_sw + h_bp), 3);
+ max96712_write_bulk_value(priv, 0x1067, h_active, 2);
+ max96712_write_bulk_value(priv, 0x1069, h_fp + h_sw + h_bp, 2);
+ max96712_write_bulk_value(priv, 0x106b, v_active, 2);
+
+ /* Generate VS, HS and DE in free-running mode. */
+ max96712_write(priv, 0x1050, 0xfb);
+
+ /* Configure Video Pattern Generator. */
+ if (priv->pattern == MAX96712_PATTERN_CHECKERBOARD) {
+ /* Set checkerboard pattern size. */
+ max96712_write(priv, 0x1074, 0x3c);
+ max96712_write(priv, 0x1075, 0x3c);
+ max96712_write(priv, 0x1076, 0x3c);
+
+ /* Set checkerboard pattern colors. */
+ max96712_write_bulk_value(priv, 0x106e, 0xfecc00, 3);
+ max96712_write_bulk_value(priv, 0x1071, 0x006aa7, 3);
+
+ /* Generate checkerboard pattern. */
+ max96712_write(priv, 0x1051, 0x10);
+ } else {
+ /* Set gradient increment. */
+ max96712_write(priv, 0x106d, 0x10);
+
+ /* Generate gradient pattern. */
+ max96712_write(priv, 0x1051, 0x20);
+ }
+}
+
+static int max96712_s_stream(struct v4l2_subdev *sd, int enable)
+{
+ struct max96712_priv *priv = v4l2_get_subdevdata(sd);
+
+ if (enable) {
+ max96712_pattern_enable(priv, true);
+ max96712_mipi_enable(priv, true);
+ } else {
+ max96712_mipi_enable(priv, false);
+ max96712_pattern_enable(priv, false);
+ }
+
+ return 0;
+}
+
+static const struct v4l2_subdev_video_ops max96712_video_ops = {
+ .s_stream = max96712_s_stream,
+};
+
+static int max96712_get_pad_format(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *sd_state,
+ struct v4l2_subdev_format *format)
+{
+ format->format.width = 1920;
+ format->format.height = 1080;
+ format->format.code = MEDIA_BUS_FMT_RGB888_1X24;
+ format->format.field = V4L2_FIELD_NONE;
+
+ return 0;
+}
+
+static const struct v4l2_subdev_pad_ops max96712_pad_ops = {
+ .get_fmt = max96712_get_pad_format,
+ .set_fmt = max96712_get_pad_format,
+};
+
+static const struct v4l2_subdev_ops max96712_subdev_ops = {
+ .video = &max96712_video_ops,
+ .pad = &max96712_pad_ops,
+};
+
+static const char * const max96712_test_pattern[] = {
+ "Checkerboard",
+ "Gradient",
+};
+
+static int max96712_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+ struct max96712_priv *priv =
+ container_of(ctrl->handler, struct max96712_priv, ctrl_handler);
+
+ switch (ctrl->id) {
+ case V4L2_CID_TEST_PATTERN:
+ priv->pattern = ctrl->val ?
+ MAX96712_PATTERN_GRADIENT :
+ MAX96712_PATTERN_CHECKERBOARD;
+ break;
+ }
+ return 0;
+}
+
+static const struct v4l2_ctrl_ops max96712_ctrl_ops = {
+ .s_ctrl = max96712_s_ctrl,
+};
+
+static int max96712_v4l2_register(struct max96712_priv *priv)
+{
+ long pixel_rate;
+ int ret;
+
+ v4l2_i2c_subdev_init(&priv->sd, priv->client, &max96712_subdev_ops);
+ priv->sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
+ priv->sd.entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
+
+ v4l2_ctrl_handler_init(&priv->ctrl_handler, 2);
+
+ /*
+ * TODO: Once V4L2_CID_LINK_FREQ is changed from a menu control to an
+ * INT64 control it should be used here instead of V4L2_CID_PIXEL_RATE.
+ */
+ pixel_rate = MAX96712_DPLL_FREQ / priv->mipi.num_data_lanes * 1000000;
+ v4l2_ctrl_new_std(&priv->ctrl_handler, NULL, V4L2_CID_PIXEL_RATE,
+ pixel_rate, pixel_rate, 1, pixel_rate);
+
+ v4l2_ctrl_new_std_menu_items(&priv->ctrl_handler, &max96712_ctrl_ops,
+ V4L2_CID_TEST_PATTERN,
+ ARRAY_SIZE(max96712_test_pattern) - 1,
+ 0, 0, max96712_test_pattern);
+
+ priv->sd.ctrl_handler = &priv->ctrl_handler;
+ ret = priv->ctrl_handler.error;
+ if (ret)
+ goto error;
+
+ priv->pads[0].flags = MEDIA_PAD_FL_SOURCE;
+ ret = media_entity_pads_init(&priv->sd.entity, 1, priv->pads);
+ if (ret)
+ goto error;
+
+ v4l2_set_subdevdata(&priv->sd, priv);
+
+ ret = v4l2_async_register_subdev(&priv->sd);
+ if (ret < 0) {
+ dev_err(&priv->client->dev, "Unable to register subdevice\n");
+ goto error;
+ }
+
+ return 0;
+error:
+ v4l2_ctrl_handler_free(&priv->ctrl_handler);
+
+ return ret;
+}
+
+static int max96712_parse_dt(struct max96712_priv *priv)
+{
+ struct fwnode_handle *ep;
+ struct v4l2_fwnode_endpoint v4l2_ep = {
+ .bus_type = V4L2_MBUS_CSI2_DPHY
+ };
+ int ret;
+
+ ep = fwnode_graph_get_endpoint_by_id(dev_fwnode(&priv->client->dev), 4,
+ 0, 0);
+ if (!ep) {
+ dev_err(&priv->client->dev, "Not connected to subdevice\n");
+ return -EINVAL;
+ }
+
+ ret = v4l2_fwnode_endpoint_parse(ep, &v4l2_ep);
+ fwnode_handle_put(ep);
+ if (ret) {
+ dev_err(&priv->client->dev, "Could not parse v4l2 endpoint\n");
+ return -EINVAL;
+ }
+
+ if (v4l2_ep.bus.mipi_csi2.num_data_lanes != 4) {
+ dev_err(&priv->client->dev, "Only 4 data lanes supported\n");
+ return -EINVAL;
+ }
+
+ priv->mipi = v4l2_ep.bus.mipi_csi2;
+
+ return 0;
+}
+
+static const struct regmap_config max96712_i2c_regmap = {
+ .reg_bits = 16,
+ .val_bits = 8,
+ .max_register = 0x1f00,
+};
+
+static int max96712_probe(struct i2c_client *client)
+{
+ struct max96712_priv *priv;
+ int ret;
+
+ priv = devm_kzalloc(&client->dev, sizeof(*priv), GFP_KERNEL);
+ if (!priv)
+ return -ENOMEM;
+
+ priv->client = client;
+ i2c_set_clientdata(client, priv);
+
+ priv->regmap = devm_regmap_init_i2c(client, &max96712_i2c_regmap);
+ if (IS_ERR(priv->regmap))
+ return PTR_ERR(priv->regmap);
+
+ priv->gpiod_pwdn = devm_gpiod_get_optional(&client->dev, "enable",
+ GPIOD_OUT_HIGH);
+ if (IS_ERR(priv->gpiod_pwdn))
+ return PTR_ERR(priv->gpiod_pwdn);
+
+ gpiod_set_consumer_name(priv->gpiod_pwdn, "max96712-pwdn");
+ gpiod_set_value_cansleep(priv->gpiod_pwdn, 1);
+
+ if (priv->gpiod_pwdn)
+ usleep_range(4000, 5000);
+
+ if (max96712_read(priv, 0x4a) != MAX96712_ID)
+ return -ENODEV;
+
+ max96712_reset(priv);
+
+ ret = max96712_parse_dt(priv);
+ if (ret)
+ return ret;
+
+ max96712_mipi_configure(priv);
+
+ return max96712_v4l2_register(priv);
+}
+
+static int max96712_remove(struct i2c_client *client)
+{
+ struct max96712_priv *priv = i2c_get_clientdata(client);
+
+ v4l2_async_unregister_subdev(&priv->sd);
+
+ gpiod_set_value_cansleep(priv->gpiod_pwdn, 0);
+
+ return 0;
+}
+
+static const struct of_device_id max96712_of_table[] = {
+ { .compatible = "maxim,max96712" },
+ { /* sentinel */ },
+};
+MODULE_DEVICE_TABLE(of, max96712_of_table);
+
+static struct i2c_driver max96712_i2c_driver = {
+ .driver = {
+ .name = "max96712",
+ .of_match_table = of_match_ptr(max96712_of_table),
+ },
+ .probe_new = max96712_probe,
+ .remove = max96712_remove,
+};
+
+module_i2c_driver(max96712_i2c_driver);
+
+MODULE_DESCRIPTION("Maxim MAX96712 Quad GMSL2 Deserializer Driver");
+MODULE_AUTHOR("Niklas Söderlund <niklas.soderlund@ragnatech.se>");
+MODULE_LICENSE("GPL");
diff --git a/drivers/staging/media/meson/vdec/vdec_helpers.c b/drivers/staging/media/meson/vdec/vdec_helpers.c
index b9125c2..203d7af 100644
--- a/drivers/staging/media/meson/vdec/vdec_helpers.c
+++ b/drivers/staging/media/meson/vdec/vdec_helpers.c
@@ -276,13 +276,13 @@ static void dst_buf_done(struct amvdec_session *sess,
switch (sess->pixfmt_cap) {
case V4L2_PIX_FMT_NV12M:
- vbuf->vb2_buf.planes[0].bytesused = output_size;
- vbuf->vb2_buf.planes[1].bytesused = output_size / 2;
+ vb2_set_plane_payload(&vbuf->vb2_buf, 0, output_size);
+ vb2_set_plane_payload(&vbuf->vb2_buf, 1, output_size / 2);
break;
case V4L2_PIX_FMT_YUV420M:
- vbuf->vb2_buf.planes[0].bytesused = output_size;
- vbuf->vb2_buf.planes[1].bytesused = output_size / 4;
- vbuf->vb2_buf.planes[2].bytesused = output_size / 4;
+ vb2_set_plane_payload(&vbuf->vb2_buf, 0, output_size);
+ vb2_set_plane_payload(&vbuf->vb2_buf, 1, output_size / 4);
+ vb2_set_plane_payload(&vbuf->vb2_buf, 2, output_size / 4);
break;
}
diff --git a/drivers/staging/media/rkvdec/Kconfig b/drivers/staging/media/rkvdec/Kconfig
index c02199b..dc7292f 100644
--- a/drivers/staging/media/rkvdec/Kconfig
+++ b/drivers/staging/media/rkvdec/Kconfig
@@ -9,6 +9,7 @@
select VIDEOBUF2_VMALLOC
select V4L2_MEM2MEM_DEV
select V4L2_H264
+ select V4L2_VP9
help
Support for the Rockchip Video Decoder IP present on Rockchip SoCs,
which accelerates video decoding.
diff --git a/drivers/staging/media/rkvdec/Makefile b/drivers/staging/media/rkvdec/Makefile
index c08fed0..cb86b42 100644
--- a/drivers/staging/media/rkvdec/Makefile
+++ b/drivers/staging/media/rkvdec/Makefile
@@ -1,3 +1,3 @@
obj-$(CONFIG_VIDEO_ROCKCHIP_VDEC) += rockchip-vdec.o
-rockchip-vdec-y += rkvdec.o rkvdec-h264.o
+rockchip-vdec-y += rkvdec.o rkvdec-h264.o rkvdec-vp9.o
diff --git a/drivers/staging/media/rkvdec/rkvdec-vp9.c b/drivers/staging/media/rkvdec/rkvdec-vp9.c
new file mode 100644
index 0000000..311a126
--- /dev/null
+++ b/drivers/staging/media/rkvdec/rkvdec-vp9.c
@@ -0,0 +1,1072 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Rockchip Video Decoder VP9 backend
+ *
+ * Copyright (C) 2019 Collabora, Ltd.
+ * Boris Brezillon <boris.brezillon@collabora.com>
+ * Copyright (C) 2021 Collabora, Ltd.
+ * Andrzej Pietrasiewicz <andrzej.p@collabora.com>
+ *
+ * Copyright (C) 2016 Rockchip Electronics Co., Ltd.
+ * Alpha Lin <Alpha.Lin@rock-chips.com>
+ */
+
+/*
+ * For following the vp9 spec please start reading this driver
+ * code from rkvdec_vp9_run() followed by rkvdec_vp9_done().
+ */
+
+#include <linux/kernel.h>
+#include <linux/vmalloc.h>
+#include <media/v4l2-mem2mem.h>
+#include <media/v4l2-vp9.h>
+
+#include "rkvdec.h"
+#include "rkvdec-regs.h"
+
+#define RKVDEC_VP9_PROBE_SIZE 4864
+#define RKVDEC_VP9_COUNT_SIZE 13232
+#define RKVDEC_VP9_MAX_SEGMAP_SIZE 73728
+
+struct rkvdec_vp9_intra_mode_probs {
+ u8 y_mode[105];
+ u8 uv_mode[23];
+};
+
+struct rkvdec_vp9_intra_only_frame_probs {
+ u8 coef_intra[4][2][128];
+ struct rkvdec_vp9_intra_mode_probs intra_mode[10];
+};
+
+struct rkvdec_vp9_inter_frame_probs {
+ u8 y_mode[4][9];
+ u8 comp_mode[5];
+ u8 comp_ref[5];
+ u8 single_ref[5][2];
+ u8 inter_mode[7][3];
+ u8 interp_filter[4][2];
+ u8 padding0[11];
+ u8 coef[2][4][2][128];
+ u8 uv_mode_0_2[3][9];
+ u8 padding1[5];
+ u8 uv_mode_3_5[3][9];
+ u8 padding2[5];
+ u8 uv_mode_6_8[3][9];
+ u8 padding3[5];
+ u8 uv_mode_9[9];
+ u8 padding4[7];
+ u8 padding5[16];
+ struct {
+ u8 joint[3];
+ u8 sign[2];
+ u8 classes[2][10];
+ u8 class0_bit[2];
+ u8 bits[2][10];
+ u8 class0_fr[2][2][3];
+ u8 fr[2][3];
+ u8 class0_hp[2];
+ u8 hp[2];
+ } mv;
+};
+
+struct rkvdec_vp9_probs {
+ u8 partition[16][3];
+ u8 pred[3];
+ u8 tree[7];
+ u8 skip[3];
+ u8 tx32[2][3];
+ u8 tx16[2][2];
+ u8 tx8[2][1];
+ u8 is_inter[4];
+ /* 128 bit alignment */
+ u8 padding0[3];
+ union {
+ struct rkvdec_vp9_inter_frame_probs inter;
+ struct rkvdec_vp9_intra_only_frame_probs intra_only;
+ };
+};
+
+/* Data structure describing auxiliary buffer format. */
+struct rkvdec_vp9_priv_tbl {
+ struct rkvdec_vp9_probs probs;
+ u8 segmap[2][RKVDEC_VP9_MAX_SEGMAP_SIZE];
+};
+
+struct rkvdec_vp9_refs_counts {
+ u32 eob[2];
+ u32 coeff[3];
+};
+
+struct rkvdec_vp9_inter_frame_symbol_counts {
+ u32 partition[16][4];
+ u32 skip[3][2];
+ u32 inter[4][2];
+ u32 tx32p[2][4];
+ u32 tx16p[2][4];
+ u32 tx8p[2][2];
+ u32 y_mode[4][10];
+ u32 uv_mode[10][10];
+ u32 comp[5][2];
+ u32 comp_ref[5][2];
+ u32 single_ref[5][2][2];
+ u32 mv_mode[7][4];
+ u32 filter[4][3];
+ u32 mv_joint[4];
+ u32 sign[2][2];
+ /* add 1 element for align */
+ u32 classes[2][11 + 1];
+ u32 class0[2][2];
+ u32 bits[2][10][2];
+ u32 class0_fp[2][2][4];
+ u32 fp[2][4];
+ u32 class0_hp[2][2];
+ u32 hp[2][2];
+ struct rkvdec_vp9_refs_counts ref_cnt[2][4][2][6][6];
+};
+
+struct rkvdec_vp9_intra_frame_symbol_counts {
+ u32 partition[4][4][4];
+ u32 skip[3][2];
+ u32 intra[4][2];
+ u32 tx32p[2][4];
+ u32 tx16p[2][4];
+ u32 tx8p[2][2];
+ struct rkvdec_vp9_refs_counts ref_cnt[2][4][2][6][6];
+};
+
+struct rkvdec_vp9_run {
+ struct rkvdec_run base;
+ const struct v4l2_ctrl_vp9_frame *decode_params;
+};
+
+struct rkvdec_vp9_frame_info {
+ u32 valid : 1;
+ u32 segmapid : 1;
+ u32 frame_context_idx : 2;
+ u32 reference_mode : 2;
+ u32 tx_mode : 3;
+ u32 interpolation_filter : 3;
+ u32 flags;
+ u64 timestamp;
+ struct v4l2_vp9_segmentation seg;
+ struct v4l2_vp9_loop_filter lf;
+};
+
+struct rkvdec_vp9_ctx {
+ struct rkvdec_aux_buf priv_tbl;
+ struct rkvdec_aux_buf count_tbl;
+ struct v4l2_vp9_frame_symbol_counts inter_cnts;
+ struct v4l2_vp9_frame_symbol_counts intra_cnts;
+ struct v4l2_vp9_frame_context probability_tables;
+ struct v4l2_vp9_frame_context frame_context[4];
+ struct rkvdec_vp9_frame_info cur;
+ struct rkvdec_vp9_frame_info last;
+};
+
+static void write_coeff_plane(const u8 coef[6][6][3], u8 *coeff_plane)
+{
+ unsigned int idx = 0, byte_count = 0;
+ int k, m, n;
+ u8 p;
+
+ for (k = 0; k < 6; k++) {
+ for (m = 0; m < 6; m++) {
+ for (n = 0; n < 3; n++) {
+ p = coef[k][m][n];
+ coeff_plane[idx++] = p;
+ byte_count++;
+ if (byte_count == 27) {
+ idx += 5;
+ byte_count = 0;
+ }
+ }
+ }
+ }
+}
+
+static void init_intra_only_probs(struct rkvdec_ctx *ctx,
+ const struct rkvdec_vp9_run *run)
+{
+ struct rkvdec_vp9_ctx *vp9_ctx = ctx->priv;
+ struct rkvdec_vp9_priv_tbl *tbl = vp9_ctx->priv_tbl.cpu;
+ struct rkvdec_vp9_intra_only_frame_probs *rkprobs;
+ const struct v4l2_vp9_frame_context *probs;
+ unsigned int i, j, k;
+
+ rkprobs = &tbl->probs.intra_only;
+ probs = &vp9_ctx->probability_tables;
+
+ /*
+ * intra only 149 x 128 bits ,aligned to 152 x 128 bits coeff related
+ * prob 64 x 128 bits
+ */
+ for (i = 0; i < ARRAY_SIZE(probs->coef); i++) {
+ for (j = 0; j < ARRAY_SIZE(probs->coef[0]); j++)
+ write_coeff_plane(probs->coef[i][j][0],
+ rkprobs->coef_intra[i][j]);
+ }
+
+ /* intra mode prob 80 x 128 bits */
+ for (i = 0; i < ARRAY_SIZE(v4l2_vp9_kf_y_mode_prob); i++) {
+ unsigned int byte_count = 0;
+ int idx = 0;
+
+ /* vp9_kf_y_mode_prob */
+ for (j = 0; j < ARRAY_SIZE(v4l2_vp9_kf_y_mode_prob[0]); j++) {
+ for (k = 0; k < ARRAY_SIZE(v4l2_vp9_kf_y_mode_prob[0][0]);
+ k++) {
+ u8 val = v4l2_vp9_kf_y_mode_prob[i][j][k];
+
+ rkprobs->intra_mode[i].y_mode[idx++] = val;
+ byte_count++;
+ if (byte_count == 27) {
+ byte_count = 0;
+ idx += 5;
+ }
+ }
+ }
+
+ }
+
+ for (i = 0; i < sizeof(v4l2_vp9_kf_uv_mode_prob); ++i) {
+ const u8 *ptr = (const u8 *)v4l2_vp9_kf_uv_mode_prob;
+
+ rkprobs->intra_mode[i / 23].uv_mode[i % 23] = ptr[i];
+ }
+}
+
+static void init_inter_probs(struct rkvdec_ctx *ctx,
+ const struct rkvdec_vp9_run *run)
+{
+ struct rkvdec_vp9_ctx *vp9_ctx = ctx->priv;
+ struct rkvdec_vp9_priv_tbl *tbl = vp9_ctx->priv_tbl.cpu;
+ struct rkvdec_vp9_inter_frame_probs *rkprobs;
+ const struct v4l2_vp9_frame_context *probs;
+ unsigned int i, j, k;
+
+ rkprobs = &tbl->probs.inter;
+ probs = &vp9_ctx->probability_tables;
+
+ /*
+ * inter probs
+ * 151 x 128 bits, aligned to 152 x 128 bits
+ * inter only
+ * intra_y_mode & inter_block info 6 x 128 bits
+ */
+
+ memcpy(rkprobs->y_mode, probs->y_mode, sizeof(rkprobs->y_mode));
+ memcpy(rkprobs->comp_mode, probs->comp_mode,
+ sizeof(rkprobs->comp_mode));
+ memcpy(rkprobs->comp_ref, probs->comp_ref,
+ sizeof(rkprobs->comp_ref));
+ memcpy(rkprobs->single_ref, probs->single_ref,
+ sizeof(rkprobs->single_ref));
+ memcpy(rkprobs->inter_mode, probs->inter_mode,
+ sizeof(rkprobs->inter_mode));
+ memcpy(rkprobs->interp_filter, probs->interp_filter,
+ sizeof(rkprobs->interp_filter));
+
+ /* 128 x 128 bits coeff related */
+ for (i = 0; i < ARRAY_SIZE(probs->coef); i++) {
+ for (j = 0; j < ARRAY_SIZE(probs->coef[0]); j++) {
+ for (k = 0; k < ARRAY_SIZE(probs->coef[0][0]); k++)
+ write_coeff_plane(probs->coef[i][j][k],
+ rkprobs->coef[k][i][j]);
+ }
+ }
+
+ /* intra uv mode 6 x 128 */
+ memcpy(rkprobs->uv_mode_0_2, &probs->uv_mode[0],
+ sizeof(rkprobs->uv_mode_0_2));
+ memcpy(rkprobs->uv_mode_3_5, &probs->uv_mode[3],
+ sizeof(rkprobs->uv_mode_3_5));
+ memcpy(rkprobs->uv_mode_6_8, &probs->uv_mode[6],
+ sizeof(rkprobs->uv_mode_6_8));
+ memcpy(rkprobs->uv_mode_9, &probs->uv_mode[9],
+ sizeof(rkprobs->uv_mode_9));
+
+ /* mv related 6 x 128 */
+ memcpy(rkprobs->mv.joint, probs->mv.joint,
+ sizeof(rkprobs->mv.joint));
+ memcpy(rkprobs->mv.sign, probs->mv.sign,
+ sizeof(rkprobs->mv.sign));
+ memcpy(rkprobs->mv.classes, probs->mv.classes,
+ sizeof(rkprobs->mv.classes));
+ memcpy(rkprobs->mv.class0_bit, probs->mv.class0_bit,
+ sizeof(rkprobs->mv.class0_bit));
+ memcpy(rkprobs->mv.bits, probs->mv.bits,
+ sizeof(rkprobs->mv.bits));
+ memcpy(rkprobs->mv.class0_fr, probs->mv.class0_fr,
+ sizeof(rkprobs->mv.class0_fr));
+ memcpy(rkprobs->mv.fr, probs->mv.fr,
+ sizeof(rkprobs->mv.fr));
+ memcpy(rkprobs->mv.class0_hp, probs->mv.class0_hp,
+ sizeof(rkprobs->mv.class0_hp));
+ memcpy(rkprobs->mv.hp, probs->mv.hp,
+ sizeof(rkprobs->mv.hp));
+}
+
+static void init_probs(struct rkvdec_ctx *ctx,
+ const struct rkvdec_vp9_run *run)
+{
+ const struct v4l2_ctrl_vp9_frame *dec_params;
+ struct rkvdec_vp9_ctx *vp9_ctx = ctx->priv;
+ struct rkvdec_vp9_priv_tbl *tbl = vp9_ctx->priv_tbl.cpu;
+ struct rkvdec_vp9_probs *rkprobs = &tbl->probs;
+ const struct v4l2_vp9_segmentation *seg;
+ const struct v4l2_vp9_frame_context *probs;
+ bool intra_only;
+
+ dec_params = run->decode_params;
+ probs = &vp9_ctx->probability_tables;
+ seg = &dec_params->seg;
+
+ memset(rkprobs, 0, sizeof(*rkprobs));
+
+ intra_only = !!(dec_params->flags &
+ (V4L2_VP9_FRAME_FLAG_KEY_FRAME |
+ V4L2_VP9_FRAME_FLAG_INTRA_ONLY));
+
+ /* sb info 5 x 128 bit */
+ memcpy(rkprobs->partition,
+ intra_only ? v4l2_vp9_kf_partition_probs : probs->partition,
+ sizeof(rkprobs->partition));
+
+ memcpy(rkprobs->pred, seg->pred_probs, sizeof(rkprobs->pred));
+ memcpy(rkprobs->tree, seg->tree_probs, sizeof(rkprobs->tree));
+ memcpy(rkprobs->skip, probs->skip, sizeof(rkprobs->skip));
+ memcpy(rkprobs->tx32, probs->tx32, sizeof(rkprobs->tx32));
+ memcpy(rkprobs->tx16, probs->tx16, sizeof(rkprobs->tx16));
+ memcpy(rkprobs->tx8, probs->tx8, sizeof(rkprobs->tx8));
+ memcpy(rkprobs->is_inter, probs->is_inter, sizeof(rkprobs->is_inter));
+
+ if (intra_only)
+ init_intra_only_probs(ctx, run);
+ else
+ init_inter_probs(ctx, run);
+}
+
+struct rkvdec_vp9_ref_reg {
+ u32 reg_frm_size;
+ u32 reg_hor_stride;
+ u32 reg_y_stride;
+ u32 reg_yuv_stride;
+ u32 reg_ref_base;
+};
+
+static struct rkvdec_vp9_ref_reg ref_regs[] = {
+ {
+ .reg_frm_size = RKVDEC_REG_VP9_FRAME_SIZE(0),
+ .reg_hor_stride = RKVDEC_VP9_HOR_VIRSTRIDE(0),
+ .reg_y_stride = RKVDEC_VP9_LAST_FRAME_YSTRIDE,
+ .reg_yuv_stride = RKVDEC_VP9_LAST_FRAME_YUVSTRIDE,
+ .reg_ref_base = RKVDEC_REG_VP9_LAST_FRAME_BASE,
+ },
+ {
+ .reg_frm_size = RKVDEC_REG_VP9_FRAME_SIZE(1),
+ .reg_hor_stride = RKVDEC_VP9_HOR_VIRSTRIDE(1),
+ .reg_y_stride = RKVDEC_VP9_GOLDEN_FRAME_YSTRIDE,
+ .reg_yuv_stride = 0,
+ .reg_ref_base = RKVDEC_REG_VP9_GOLDEN_FRAME_BASE,
+ },
+ {
+ .reg_frm_size = RKVDEC_REG_VP9_FRAME_SIZE(2),
+ .reg_hor_stride = RKVDEC_VP9_HOR_VIRSTRIDE(2),
+ .reg_y_stride = RKVDEC_VP9_ALTREF_FRAME_YSTRIDE,
+ .reg_yuv_stride = 0,
+ .reg_ref_base = RKVDEC_REG_VP9_ALTREF_FRAME_BASE,
+ }
+};
+
+static struct rkvdec_decoded_buffer *
+get_ref_buf(struct rkvdec_ctx *ctx, struct vb2_v4l2_buffer *dst, u64 timestamp)
+{
+ struct v4l2_m2m_ctx *m2m_ctx = ctx->fh.m2m_ctx;
+ struct vb2_queue *cap_q = &m2m_ctx->cap_q_ctx.q;
+ int buf_idx;
+
+ /*
+ * If a ref is unused or invalid, address of current destination
+ * buffer is returned.
+ */
+ buf_idx = vb2_find_timestamp(cap_q, timestamp, 0);
+ if (buf_idx < 0)
+ return vb2_to_rkvdec_decoded_buf(&dst->vb2_buf);
+
+ return vb2_to_rkvdec_decoded_buf(vb2_get_buffer(cap_q, buf_idx));
+}
+
+static dma_addr_t get_mv_base_addr(struct rkvdec_decoded_buffer *buf)
+{
+ unsigned int aligned_pitch, aligned_height, yuv_len;
+
+ aligned_height = round_up(buf->vp9.height, 64);
+ aligned_pitch = round_up(buf->vp9.width * buf->vp9.bit_depth, 512) / 8;
+ yuv_len = (aligned_height * aligned_pitch * 3) / 2;
+
+ return vb2_dma_contig_plane_dma_addr(&buf->base.vb.vb2_buf, 0) +
+ yuv_len;
+}
+
+static void config_ref_registers(struct rkvdec_ctx *ctx,
+ const struct rkvdec_vp9_run *run,
+ struct rkvdec_decoded_buffer *ref_buf,
+ struct rkvdec_vp9_ref_reg *ref_reg)
+{
+ unsigned int aligned_pitch, aligned_height, y_len, yuv_len;
+ struct rkvdec_dev *rkvdec = ctx->dev;
+
+ aligned_height = round_up(ref_buf->vp9.height, 64);
+ writel_relaxed(RKVDEC_VP9_FRAMEWIDTH(ref_buf->vp9.width) |
+ RKVDEC_VP9_FRAMEHEIGHT(ref_buf->vp9.height),
+ rkvdec->regs + ref_reg->reg_frm_size);
+
+ writel_relaxed(vb2_dma_contig_plane_dma_addr(&ref_buf->base.vb.vb2_buf, 0),
+ rkvdec->regs + ref_reg->reg_ref_base);
+
+ if (&ref_buf->base.vb == run->base.bufs.dst)
+ return;
+
+ aligned_pitch = round_up(ref_buf->vp9.width * ref_buf->vp9.bit_depth, 512) / 8;
+ y_len = aligned_height * aligned_pitch;
+ yuv_len = (y_len * 3) / 2;
+
+ writel_relaxed(RKVDEC_HOR_Y_VIRSTRIDE(aligned_pitch / 16) |
+ RKVDEC_HOR_UV_VIRSTRIDE(aligned_pitch / 16),
+ rkvdec->regs + ref_reg->reg_hor_stride);
+ writel_relaxed(RKVDEC_VP9_REF_YSTRIDE(y_len / 16),
+ rkvdec->regs + ref_reg->reg_y_stride);
+
+ if (!ref_reg->reg_yuv_stride)
+ return;
+
+ writel_relaxed(RKVDEC_VP9_REF_YUVSTRIDE(yuv_len / 16),
+ rkvdec->regs + ref_reg->reg_yuv_stride);
+}
+
+static void config_seg_registers(struct rkvdec_ctx *ctx, unsigned int segid)
+{
+ struct rkvdec_vp9_ctx *vp9_ctx = ctx->priv;
+ const struct v4l2_vp9_segmentation *seg;
+ struct rkvdec_dev *rkvdec = ctx->dev;
+ s16 feature_val;
+ int feature_id;
+ u32 val = 0;
+
+ seg = vp9_ctx->last.valid ? &vp9_ctx->last.seg : &vp9_ctx->cur.seg;
+ feature_id = V4L2_VP9_SEG_LVL_ALT_Q;
+ if (v4l2_vp9_seg_feat_enabled(seg->feature_enabled, feature_id, segid)) {
+ feature_val = seg->feature_data[segid][feature_id];
+ val |= RKVDEC_SEGID_FRAME_QP_DELTA_EN(1) |
+ RKVDEC_SEGID_FRAME_QP_DELTA(feature_val);
+ }
+
+ feature_id = V4L2_VP9_SEG_LVL_ALT_L;
+ if (v4l2_vp9_seg_feat_enabled(seg->feature_enabled, feature_id, segid)) {
+ feature_val = seg->feature_data[segid][feature_id];
+ val |= RKVDEC_SEGID_FRAME_LOOPFILTER_VALUE_EN(1) |
+ RKVDEC_SEGID_FRAME_LOOPFILTER_VALUE(feature_val);
+ }
+
+ feature_id = V4L2_VP9_SEG_LVL_REF_FRAME;
+ if (v4l2_vp9_seg_feat_enabled(seg->feature_enabled, feature_id, segid)) {
+ feature_val = seg->feature_data[segid][feature_id];
+ val |= RKVDEC_SEGID_REFERINFO_EN(1) |
+ RKVDEC_SEGID_REFERINFO(feature_val);
+ }
+
+ feature_id = V4L2_VP9_SEG_LVL_SKIP;
+ if (v4l2_vp9_seg_feat_enabled(seg->feature_enabled, feature_id, segid))
+ val |= RKVDEC_SEGID_FRAME_SKIP_EN(1);
+
+ if (!segid &&
+ (seg->flags & V4L2_VP9_SEGMENTATION_FLAG_ABS_OR_DELTA_UPDATE))
+ val |= RKVDEC_SEGID_ABS_DELTA(1);
+
+ writel_relaxed(val, rkvdec->regs + RKVDEC_VP9_SEGID_GRP(segid));
+}
+
+static void update_dec_buf_info(struct rkvdec_decoded_buffer *buf,
+ const struct v4l2_ctrl_vp9_frame *dec_params)
+{
+ buf->vp9.width = dec_params->frame_width_minus_1 + 1;
+ buf->vp9.height = dec_params->frame_height_minus_1 + 1;
+ buf->vp9.bit_depth = dec_params->bit_depth;
+}
+
+static void update_ctx_cur_info(struct rkvdec_vp9_ctx *vp9_ctx,
+ struct rkvdec_decoded_buffer *buf,
+ const struct v4l2_ctrl_vp9_frame *dec_params)
+{
+ vp9_ctx->cur.valid = true;
+ vp9_ctx->cur.reference_mode = dec_params->reference_mode;
+ vp9_ctx->cur.interpolation_filter = dec_params->interpolation_filter;
+ vp9_ctx->cur.flags = dec_params->flags;
+ vp9_ctx->cur.timestamp = buf->base.vb.vb2_buf.timestamp;
+ vp9_ctx->cur.seg = dec_params->seg;
+ vp9_ctx->cur.lf = dec_params->lf;
+}
+
+static void update_ctx_last_info(struct rkvdec_vp9_ctx *vp9_ctx)
+{
+ vp9_ctx->last = vp9_ctx->cur;
+}
+
+static void config_registers(struct rkvdec_ctx *ctx,
+ const struct rkvdec_vp9_run *run)
+{
+ unsigned int y_len, uv_len, yuv_len, bit_depth, aligned_height, aligned_pitch, stream_len;
+ const struct v4l2_ctrl_vp9_frame *dec_params;
+ struct rkvdec_decoded_buffer *ref_bufs[3];
+ struct rkvdec_decoded_buffer *dst, *last, *mv_ref;
+ struct rkvdec_vp9_ctx *vp9_ctx = ctx->priv;
+ u32 val, last_frame_info = 0;
+ const struct v4l2_vp9_segmentation *seg;
+ struct rkvdec_dev *rkvdec = ctx->dev;
+ dma_addr_t addr;
+ bool intra_only;
+ unsigned int i;
+
+ dec_params = run->decode_params;
+ dst = vb2_to_rkvdec_decoded_buf(&run->base.bufs.dst->vb2_buf);
+ ref_bufs[0] = get_ref_buf(ctx, &dst->base.vb, dec_params->last_frame_ts);
+ ref_bufs[1] = get_ref_buf(ctx, &dst->base.vb, dec_params->golden_frame_ts);
+ ref_bufs[2] = get_ref_buf(ctx, &dst->base.vb, dec_params->alt_frame_ts);
+
+ if (vp9_ctx->last.valid)
+ last = get_ref_buf(ctx, &dst->base.vb, vp9_ctx->last.timestamp);
+ else
+ last = dst;
+
+ update_dec_buf_info(dst, dec_params);
+ update_ctx_cur_info(vp9_ctx, dst, dec_params);
+ seg = &dec_params->seg;
+
+ intra_only = !!(dec_params->flags &
+ (V4L2_VP9_FRAME_FLAG_KEY_FRAME |
+ V4L2_VP9_FRAME_FLAG_INTRA_ONLY));
+
+ writel_relaxed(RKVDEC_MODE(RKVDEC_MODE_VP9),
+ rkvdec->regs + RKVDEC_REG_SYSCTRL);
+
+ bit_depth = dec_params->bit_depth;
+ aligned_height = round_up(ctx->decoded_fmt.fmt.pix_mp.height, 64);
+
+ aligned_pitch = round_up(ctx->decoded_fmt.fmt.pix_mp.width *
+ bit_depth,
+ 512) / 8;
+ y_len = aligned_height * aligned_pitch;
+ uv_len = y_len / 2;
+ yuv_len = y_len + uv_len;
+
+ writel_relaxed(RKVDEC_Y_HOR_VIRSTRIDE(aligned_pitch / 16) |
+ RKVDEC_UV_HOR_VIRSTRIDE(aligned_pitch / 16),
+ rkvdec->regs + RKVDEC_REG_PICPAR);
+ writel_relaxed(RKVDEC_Y_VIRSTRIDE(y_len / 16),
+ rkvdec->regs + RKVDEC_REG_Y_VIRSTRIDE);
+ writel_relaxed(RKVDEC_YUV_VIRSTRIDE(yuv_len / 16),
+ rkvdec->regs + RKVDEC_REG_YUV_VIRSTRIDE);
+
+ stream_len = vb2_get_plane_payload(&run->base.bufs.src->vb2_buf, 0);
+ writel_relaxed(RKVDEC_STRM_LEN(stream_len),
+ rkvdec->regs + RKVDEC_REG_STRM_LEN);
+
+ /*
+ * Reset count buffer, because decoder only output intra related syntax
+ * counts when decoding intra frame, but update entropy need to update
+ * all the probabilities.
+ */
+ if (intra_only)
+ memset(vp9_ctx->count_tbl.cpu, 0, vp9_ctx->count_tbl.size);
+
+ vp9_ctx->cur.segmapid = vp9_ctx->last.segmapid;
+ if (!intra_only &&
+ !(dec_params->flags & V4L2_VP9_FRAME_FLAG_ERROR_RESILIENT) &&
+ (!(seg->flags & V4L2_VP9_SEGMENTATION_FLAG_ENABLED) ||
+ (seg->flags & V4L2_VP9_SEGMENTATION_FLAG_UPDATE_MAP)))
+ vp9_ctx->cur.segmapid++;
+
+ for (i = 0; i < ARRAY_SIZE(ref_bufs); i++)
+ config_ref_registers(ctx, run, ref_bufs[i], &ref_regs[i]);
+
+ for (i = 0; i < 8; i++)
+ config_seg_registers(ctx, i);
+
+ writel_relaxed(RKVDEC_VP9_TX_MODE(vp9_ctx->cur.tx_mode) |
+ RKVDEC_VP9_FRAME_REF_MODE(dec_params->reference_mode),
+ rkvdec->regs + RKVDEC_VP9_CPRHEADER_CONFIG);
+
+ if (!intra_only) {
+ const struct v4l2_vp9_loop_filter *lf;
+ s8 delta;
+
+ if (vp9_ctx->last.valid)
+ lf = &vp9_ctx->last.lf;
+ else
+ lf = &vp9_ctx->cur.lf;
+
+ val = 0;
+ for (i = 0; i < ARRAY_SIZE(lf->ref_deltas); i++) {
+ delta = lf->ref_deltas[i];
+ val |= RKVDEC_REF_DELTAS_LASTFRAME(i, delta);
+ }
+
+ writel_relaxed(val,
+ rkvdec->regs + RKVDEC_VP9_REF_DELTAS_LASTFRAME);
+
+ for (i = 0; i < ARRAY_SIZE(lf->mode_deltas); i++) {
+ delta = lf->mode_deltas[i];
+ last_frame_info |= RKVDEC_MODE_DELTAS_LASTFRAME(i,
+ delta);
+ }
+ }
+
+ if (vp9_ctx->last.valid && !intra_only &&
+ vp9_ctx->last.seg.flags & V4L2_VP9_SEGMENTATION_FLAG_ENABLED)
+ last_frame_info |= RKVDEC_SEG_EN_LASTFRAME;
+
+ if (vp9_ctx->last.valid &&
+ vp9_ctx->last.flags & V4L2_VP9_FRAME_FLAG_SHOW_FRAME)
+ last_frame_info |= RKVDEC_LAST_SHOW_FRAME;
+
+ if (vp9_ctx->last.valid &&
+ vp9_ctx->last.flags &
+ (V4L2_VP9_FRAME_FLAG_KEY_FRAME | V4L2_VP9_FRAME_FLAG_INTRA_ONLY))
+ last_frame_info |= RKVDEC_LAST_INTRA_ONLY;
+
+ if (vp9_ctx->last.valid &&
+ last->vp9.width == dst->vp9.width &&
+ last->vp9.height == dst->vp9.height)
+ last_frame_info |= RKVDEC_LAST_WIDHHEIGHT_EQCUR;
+
+ writel_relaxed(last_frame_info,
+ rkvdec->regs + RKVDEC_VP9_INFO_LASTFRAME);
+
+ writel_relaxed(stream_len - dec_params->compressed_header_size -
+ dec_params->uncompressed_header_size,
+ rkvdec->regs + RKVDEC_VP9_LASTTILE_SIZE);
+
+ for (i = 0; !intra_only && i < ARRAY_SIZE(ref_bufs); i++) {
+ unsigned int refw = ref_bufs[i]->vp9.width;
+ unsigned int refh = ref_bufs[i]->vp9.height;
+ u32 hscale, vscale;
+
+ hscale = (refw << 14) / dst->vp9.width;
+ vscale = (refh << 14) / dst->vp9.height;
+ writel_relaxed(RKVDEC_VP9_REF_HOR_SCALE(hscale) |
+ RKVDEC_VP9_REF_VER_SCALE(vscale),
+ rkvdec->regs + RKVDEC_VP9_REF_SCALE(i));
+ }
+
+ addr = vb2_dma_contig_plane_dma_addr(&dst->base.vb.vb2_buf, 0);
+ writel_relaxed(addr, rkvdec->regs + RKVDEC_REG_DECOUT_BASE);
+ addr = vb2_dma_contig_plane_dma_addr(&run->base.bufs.src->vb2_buf, 0);
+ writel_relaxed(addr, rkvdec->regs + RKVDEC_REG_STRM_RLC_BASE);
+ writel_relaxed(vp9_ctx->priv_tbl.dma +
+ offsetof(struct rkvdec_vp9_priv_tbl, probs),
+ rkvdec->regs + RKVDEC_REG_CABACTBL_PROB_BASE);
+ writel_relaxed(vp9_ctx->count_tbl.dma,
+ rkvdec->regs + RKVDEC_REG_VP9COUNT_BASE);
+
+ writel_relaxed(vp9_ctx->priv_tbl.dma +
+ offsetof(struct rkvdec_vp9_priv_tbl, segmap) +
+ (RKVDEC_VP9_MAX_SEGMAP_SIZE * vp9_ctx->cur.segmapid),
+ rkvdec->regs + RKVDEC_REG_VP9_SEGIDCUR_BASE);
+ writel_relaxed(vp9_ctx->priv_tbl.dma +
+ offsetof(struct rkvdec_vp9_priv_tbl, segmap) +
+ (RKVDEC_VP9_MAX_SEGMAP_SIZE * (!vp9_ctx->cur.segmapid)),
+ rkvdec->regs + RKVDEC_REG_VP9_SEGIDLAST_BASE);
+
+ if (!intra_only &&
+ !(dec_params->flags & V4L2_VP9_FRAME_FLAG_ERROR_RESILIENT) &&
+ vp9_ctx->last.valid)
+ mv_ref = last;
+ else
+ mv_ref = dst;
+
+ writel_relaxed(get_mv_base_addr(mv_ref),
+ rkvdec->regs + RKVDEC_VP9_REF_COLMV_BASE);
+
+ writel_relaxed(ctx->decoded_fmt.fmt.pix_mp.width |
+ (ctx->decoded_fmt.fmt.pix_mp.height << 16),
+ rkvdec->regs + RKVDEC_REG_PERFORMANCE_CYCLE);
+}
+
+static int validate_dec_params(struct rkvdec_ctx *ctx,
+ const struct v4l2_ctrl_vp9_frame *dec_params)
+{
+ unsigned int aligned_width, aligned_height;
+
+ /* We only support profile 0. */
+ if (dec_params->profile != 0) {
+ dev_err(ctx->dev->dev, "unsupported profile %d\n",
+ dec_params->profile);
+ return -EINVAL;
+ }
+
+ aligned_width = round_up(dec_params->frame_width_minus_1 + 1, 64);
+ aligned_height = round_up(dec_params->frame_height_minus_1 + 1, 64);
+
+ /*
+ * Userspace should update the capture/decoded format when the
+ * resolution changes.
+ */
+ if (aligned_width != ctx->decoded_fmt.fmt.pix_mp.width ||
+ aligned_height != ctx->decoded_fmt.fmt.pix_mp.height) {
+ dev_err(ctx->dev->dev,
+ "unexpected bitstream resolution %dx%d\n",
+ dec_params->frame_width_minus_1 + 1,
+ dec_params->frame_height_minus_1 + 1);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int rkvdec_vp9_run_preamble(struct rkvdec_ctx *ctx,
+ struct rkvdec_vp9_run *run)
+{
+ const struct v4l2_ctrl_vp9_frame *dec_params;
+ const struct v4l2_ctrl_vp9_compressed_hdr *prob_updates;
+ struct rkvdec_vp9_ctx *vp9_ctx = ctx->priv;
+ struct v4l2_ctrl *ctrl;
+ unsigned int fctx_idx;
+ int ret;
+
+ /* v4l2-specific stuff */
+ rkvdec_run_preamble(ctx, &run->base);
+
+ ctrl = v4l2_ctrl_find(&ctx->ctrl_hdl,
+ V4L2_CID_STATELESS_VP9_FRAME);
+ if (WARN_ON(!ctrl))
+ return -EINVAL;
+ dec_params = ctrl->p_cur.p;
+
+ ret = validate_dec_params(ctx, dec_params);
+ if (ret)
+ return ret;
+
+ run->decode_params = dec_params;
+
+ ctrl = v4l2_ctrl_find(&ctx->ctrl_hdl, V4L2_CID_STATELESS_VP9_COMPRESSED_HDR);
+ if (WARN_ON(!ctrl))
+ return -EINVAL;
+ prob_updates = ctrl->p_cur.p;
+ vp9_ctx->cur.tx_mode = prob_updates->tx_mode;
+
+ /*
+ * vp9 stuff
+ *
+ * by this point the userspace has done all parts of 6.2 uncompressed_header()
+ * except this fragment:
+ * if ( FrameIsIntra || error_resilient_mode ) {
+ * setup_past_independence ( )
+ * if ( frame_type == KEY_FRAME || error_resilient_mode == 1 ||
+ * reset_frame_context == 3 ) {
+ * for ( i = 0; i < 4; i ++ ) {
+ * save_probs( i )
+ * }
+ * } else if ( reset_frame_context == 2 ) {
+ * save_probs( frame_context_idx )
+ * }
+ * frame_context_idx = 0
+ * }
+ */
+ fctx_idx = v4l2_vp9_reset_frame_ctx(dec_params, vp9_ctx->frame_context);
+ vp9_ctx->cur.frame_context_idx = fctx_idx;
+
+ /* 6.1 frame(sz): load_probs() and load_probs2() */
+ vp9_ctx->probability_tables = vp9_ctx->frame_context[fctx_idx];
+
+ /*
+ * The userspace has also performed 6.3 compressed_header(), but handling the
+ * probs in a special way. All probs which need updating, except MV-related,
+ * have been read from the bitstream and translated through inv_map_table[],
+ * but no 6.3.6 inv_recenter_nonneg(v, m) has been performed. The values passed
+ * by userspace are either translated values (there are no 0 values in
+ * inv_map_table[]), or zero to indicate no update. All MV-related probs which need
+ * updating have been read from the bitstream and (mv_prob << 1) | 1 has been
+ * performed. The values passed by userspace are either new values
+ * to replace old ones (the above mentioned shift and bitwise or never result in
+ * a zero) or zero to indicate no update.
+ * fw_update_probs() performs actual probs updates or leaves probs as-is
+ * for values for which a zero was passed from userspace.
+ */
+ v4l2_vp9_fw_update_probs(&vp9_ctx->probability_tables, prob_updates, dec_params);
+
+ return 0;
+}
+
+static int rkvdec_vp9_run(struct rkvdec_ctx *ctx)
+{
+ struct rkvdec_dev *rkvdec = ctx->dev;
+ struct rkvdec_vp9_run run = { };
+ int ret;
+
+ ret = rkvdec_vp9_run_preamble(ctx, &run);
+ if (ret) {
+ rkvdec_run_postamble(ctx, &run.base);
+ return ret;
+ }
+
+ /* Prepare probs. */
+ init_probs(ctx, &run);
+
+ /* Configure hardware registers. */
+ config_registers(ctx, &run);
+
+ rkvdec_run_postamble(ctx, &run.base);
+
+ schedule_delayed_work(&rkvdec->watchdog_work, msecs_to_jiffies(2000));
+
+ writel(1, rkvdec->regs + RKVDEC_REG_PREF_LUMA_CACHE_COMMAND);
+ writel(1, rkvdec->regs + RKVDEC_REG_PREF_CHR_CACHE_COMMAND);
+
+ writel(0xe, rkvdec->regs + RKVDEC_REG_STRMD_ERR_EN);
+ /* Start decoding! */
+ writel(RKVDEC_INTERRUPT_DEC_E | RKVDEC_CONFIG_DEC_CLK_GATE_E |
+ RKVDEC_TIMEOUT_E | RKVDEC_BUF_EMPTY_E,
+ rkvdec->regs + RKVDEC_REG_INTERRUPT);
+
+ return 0;
+}
+
+#define copy_tx_and_skip(p1, p2) \
+do { \
+ memcpy((p1)->tx8, (p2)->tx8, sizeof((p1)->tx8)); \
+ memcpy((p1)->tx16, (p2)->tx16, sizeof((p1)->tx16)); \
+ memcpy((p1)->tx32, (p2)->tx32, sizeof((p1)->tx32)); \
+ memcpy((p1)->skip, (p2)->skip, sizeof((p1)->skip)); \
+} while (0)
+
+static void rkvdec_vp9_done(struct rkvdec_ctx *ctx,
+ struct vb2_v4l2_buffer *src_buf,
+ struct vb2_v4l2_buffer *dst_buf,
+ enum vb2_buffer_state result)
+{
+ struct rkvdec_vp9_ctx *vp9_ctx = ctx->priv;
+ unsigned int fctx_idx;
+
+ /* v4l2-specific stuff */
+ if (result == VB2_BUF_STATE_ERROR)
+ goto out_update_last;
+
+ /*
+ * vp9 stuff
+ *
+ * 6.1.2 refresh_probs()
+ *
+ * In the spec a complementary condition goes last in 6.1.2 refresh_probs(),
+ * but it makes no sense to perform all the activities from the first "if"
+ * there if we actually are not refreshing the frame context. On top of that,
+ * because of 6.2 uncompressed_header() whenever error_resilient_mode == 1,
+ * refresh_frame_context == 0. Consequently, if we don't jump to out_update_last
+ * it means error_resilient_mode must be 0.
+ */
+ if (!(vp9_ctx->cur.flags & V4L2_VP9_FRAME_FLAG_REFRESH_FRAME_CTX))
+ goto out_update_last;
+
+ fctx_idx = vp9_ctx->cur.frame_context_idx;
+
+ if (!(vp9_ctx->cur.flags & V4L2_VP9_FRAME_FLAG_PARALLEL_DEC_MODE)) {
+ /* error_resilient_mode == 0 && frame_parallel_decoding_mode == 0 */
+ struct v4l2_vp9_frame_context *probs = &vp9_ctx->probability_tables;
+ bool frame_is_intra = vp9_ctx->cur.flags &
+ (V4L2_VP9_FRAME_FLAG_KEY_FRAME | V4L2_VP9_FRAME_FLAG_INTRA_ONLY);
+ struct tx_and_skip {
+ u8 tx8[2][1];
+ u8 tx16[2][2];
+ u8 tx32[2][3];
+ u8 skip[3];
+ } _tx_skip, *tx_skip = &_tx_skip;
+ struct v4l2_vp9_frame_symbol_counts *counts;
+
+ /* buffer the forward-updated TX and skip probs */
+ if (frame_is_intra)
+ copy_tx_and_skip(tx_skip, probs);
+
+ /* 6.1.2 refresh_probs(): load_probs() and load_probs2() */
+ *probs = vp9_ctx->frame_context[fctx_idx];
+
+ /* if FrameIsIntra then undo the effect of load_probs2() */
+ if (frame_is_intra)
+ copy_tx_and_skip(probs, tx_skip);
+
+ counts = frame_is_intra ? &vp9_ctx->intra_cnts : &vp9_ctx->inter_cnts;
+ v4l2_vp9_adapt_coef_probs(probs, counts,
+ !vp9_ctx->last.valid ||
+ vp9_ctx->last.flags & V4L2_VP9_FRAME_FLAG_KEY_FRAME,
+ frame_is_intra);
+ if (!frame_is_intra) {
+ const struct rkvdec_vp9_inter_frame_symbol_counts *inter_cnts;
+ u32 classes[2][11];
+ int i;
+
+ inter_cnts = vp9_ctx->count_tbl.cpu;
+ for (i = 0; i < ARRAY_SIZE(classes); ++i)
+ memcpy(classes[i], inter_cnts->classes[i], sizeof(classes[0]));
+ counts->classes = &classes;
+
+ /* load_probs2() already done */
+ v4l2_vp9_adapt_noncoef_probs(&vp9_ctx->probability_tables, counts,
+ vp9_ctx->cur.reference_mode,
+ vp9_ctx->cur.interpolation_filter,
+ vp9_ctx->cur.tx_mode, vp9_ctx->cur.flags);
+ }
+ }
+
+ /* 6.1.2 refresh_probs(): save_probs(fctx_idx) */
+ vp9_ctx->frame_context[fctx_idx] = vp9_ctx->probability_tables;
+
+out_update_last:
+ update_ctx_last_info(vp9_ctx);
+}
+
+static void rkvdec_init_v4l2_vp9_count_tbl(struct rkvdec_ctx *ctx)
+{
+ struct rkvdec_vp9_ctx *vp9_ctx = ctx->priv;
+ struct rkvdec_vp9_intra_frame_symbol_counts *intra_cnts = vp9_ctx->count_tbl.cpu;
+ struct rkvdec_vp9_inter_frame_symbol_counts *inter_cnts = vp9_ctx->count_tbl.cpu;
+ int i, j, k, l, m;
+
+ vp9_ctx->inter_cnts.partition = &inter_cnts->partition;
+ vp9_ctx->inter_cnts.skip = &inter_cnts->skip;
+ vp9_ctx->inter_cnts.intra_inter = &inter_cnts->inter;
+ vp9_ctx->inter_cnts.tx32p = &inter_cnts->tx32p;
+ vp9_ctx->inter_cnts.tx16p = &inter_cnts->tx16p;
+ vp9_ctx->inter_cnts.tx8p = &inter_cnts->tx8p;
+
+ vp9_ctx->intra_cnts.partition = (u32 (*)[16][4])(&intra_cnts->partition);
+ vp9_ctx->intra_cnts.skip = &intra_cnts->skip;
+ vp9_ctx->intra_cnts.intra_inter = &intra_cnts->intra;
+ vp9_ctx->intra_cnts.tx32p = &intra_cnts->tx32p;
+ vp9_ctx->intra_cnts.tx16p = &intra_cnts->tx16p;
+ vp9_ctx->intra_cnts.tx8p = &intra_cnts->tx8p;
+
+ vp9_ctx->inter_cnts.y_mode = &inter_cnts->y_mode;
+ vp9_ctx->inter_cnts.uv_mode = &inter_cnts->uv_mode;
+ vp9_ctx->inter_cnts.comp = &inter_cnts->comp;
+ vp9_ctx->inter_cnts.comp_ref = &inter_cnts->comp_ref;
+ vp9_ctx->inter_cnts.single_ref = &inter_cnts->single_ref;
+ vp9_ctx->inter_cnts.mv_mode = &inter_cnts->mv_mode;
+ vp9_ctx->inter_cnts.filter = &inter_cnts->filter;
+ vp9_ctx->inter_cnts.mv_joint = &inter_cnts->mv_joint;
+ vp9_ctx->inter_cnts.sign = &inter_cnts->sign;
+ /*
+ * rk hardware actually uses "u32 classes[2][11 + 1];"
+ * instead of "u32 classes[2][11];", so this must be explicitly
+ * copied into vp9_ctx->classes when passing the data to the
+ * vp9 library function
+ */
+ vp9_ctx->inter_cnts.class0 = &inter_cnts->class0;
+ vp9_ctx->inter_cnts.bits = &inter_cnts->bits;
+ vp9_ctx->inter_cnts.class0_fp = &inter_cnts->class0_fp;
+ vp9_ctx->inter_cnts.fp = &inter_cnts->fp;
+ vp9_ctx->inter_cnts.class0_hp = &inter_cnts->class0_hp;
+ vp9_ctx->inter_cnts.hp = &inter_cnts->hp;
+
+#define INNERMOST_LOOP \
+ do { \
+ for (m = 0; m < ARRAY_SIZE(vp9_ctx->inter_cnts.coeff[0][0][0][0]); ++m) {\
+ vp9_ctx->inter_cnts.coeff[i][j][k][l][m] = \
+ &inter_cnts->ref_cnt[k][i][j][l][m].coeff; \
+ vp9_ctx->inter_cnts.eob[i][j][k][l][m][0] = \
+ &inter_cnts->ref_cnt[k][i][j][l][m].eob[0]; \
+ vp9_ctx->inter_cnts.eob[i][j][k][l][m][1] = \
+ &inter_cnts->ref_cnt[k][i][j][l][m].eob[1]; \
+ \
+ vp9_ctx->intra_cnts.coeff[i][j][k][l][m] = \
+ &intra_cnts->ref_cnt[k][i][j][l][m].coeff; \
+ vp9_ctx->intra_cnts.eob[i][j][k][l][m][0] = \
+ &intra_cnts->ref_cnt[k][i][j][l][m].eob[0]; \
+ vp9_ctx->intra_cnts.eob[i][j][k][l][m][1] = \
+ &intra_cnts->ref_cnt[k][i][j][l][m].eob[1]; \
+ } \
+ } while (0)
+
+ for (i = 0; i < ARRAY_SIZE(vp9_ctx->inter_cnts.coeff); ++i)
+ for (j = 0; j < ARRAY_SIZE(vp9_ctx->inter_cnts.coeff[0]); ++j)
+ for (k = 0; k < ARRAY_SIZE(vp9_ctx->inter_cnts.coeff[0][0]); ++k)
+ for (l = 0; l < ARRAY_SIZE(vp9_ctx->inter_cnts.coeff[0][0][0]); ++l)
+ INNERMOST_LOOP;
+#undef INNERMOST_LOOP
+}
+
+static int rkvdec_vp9_start(struct rkvdec_ctx *ctx)
+{
+ struct rkvdec_dev *rkvdec = ctx->dev;
+ struct rkvdec_vp9_priv_tbl *priv_tbl;
+ struct rkvdec_vp9_ctx *vp9_ctx;
+ unsigned char *count_tbl;
+ int ret;
+
+ vp9_ctx = kzalloc(sizeof(*vp9_ctx), GFP_KERNEL);
+ if (!vp9_ctx)
+ return -ENOMEM;
+
+ ctx->priv = vp9_ctx;
+
+ priv_tbl = dma_alloc_coherent(rkvdec->dev, sizeof(*priv_tbl),
+ &vp9_ctx->priv_tbl.dma, GFP_KERNEL);
+ if (!priv_tbl) {
+ ret = -ENOMEM;
+ goto err_free_ctx;
+ }
+
+ vp9_ctx->priv_tbl.size = sizeof(*priv_tbl);
+ vp9_ctx->priv_tbl.cpu = priv_tbl;
+ memset(priv_tbl, 0, sizeof(*priv_tbl));
+
+ count_tbl = dma_alloc_coherent(rkvdec->dev, RKVDEC_VP9_COUNT_SIZE,
+ &vp9_ctx->count_tbl.dma, GFP_KERNEL);
+ if (!count_tbl) {
+ ret = -ENOMEM;
+ goto err_free_priv_tbl;
+ }
+
+ vp9_ctx->count_tbl.size = RKVDEC_VP9_COUNT_SIZE;
+ vp9_ctx->count_tbl.cpu = count_tbl;
+ memset(count_tbl, 0, sizeof(*count_tbl));
+ rkvdec_init_v4l2_vp9_count_tbl(ctx);
+
+ return 0;
+
+err_free_priv_tbl:
+ dma_free_coherent(rkvdec->dev, vp9_ctx->priv_tbl.size,
+ vp9_ctx->priv_tbl.cpu, vp9_ctx->priv_tbl.dma);
+
+err_free_ctx:
+ kfree(vp9_ctx);
+ return ret;
+}
+
+static void rkvdec_vp9_stop(struct rkvdec_ctx *ctx)
+{
+ struct rkvdec_vp9_ctx *vp9_ctx = ctx->priv;
+ struct rkvdec_dev *rkvdec = ctx->dev;
+
+ dma_free_coherent(rkvdec->dev, vp9_ctx->count_tbl.size,
+ vp9_ctx->count_tbl.cpu, vp9_ctx->count_tbl.dma);
+ dma_free_coherent(rkvdec->dev, vp9_ctx->priv_tbl.size,
+ vp9_ctx->priv_tbl.cpu, vp9_ctx->priv_tbl.dma);
+ kfree(vp9_ctx);
+}
+
+static int rkvdec_vp9_adjust_fmt(struct rkvdec_ctx *ctx,
+ struct v4l2_format *f)
+{
+ struct v4l2_pix_format_mplane *fmt = &f->fmt.pix_mp;
+
+ fmt->num_planes = 1;
+ if (!fmt->plane_fmt[0].sizeimage)
+ fmt->plane_fmt[0].sizeimage = fmt->width * fmt->height * 2;
+ return 0;
+}
+
+const struct rkvdec_coded_fmt_ops rkvdec_vp9_fmt_ops = {
+ .adjust_fmt = rkvdec_vp9_adjust_fmt,
+ .start = rkvdec_vp9_start,
+ .stop = rkvdec_vp9_stop,
+ .run = rkvdec_vp9_run,
+ .done = rkvdec_vp9_done,
+};
diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c
index 4fd4a29..c0cf348 100644
--- a/drivers/staging/media/rkvdec/rkvdec.c
+++ b/drivers/staging/media/rkvdec/rkvdec.c
@@ -99,10 +99,30 @@ static const struct rkvdec_ctrls rkvdec_h264_ctrls = {
.num_ctrls = ARRAY_SIZE(rkvdec_h264_ctrl_descs),
};
-static const u32 rkvdec_h264_decoded_fmts[] = {
+static const u32 rkvdec_h264_vp9_decoded_fmts[] = {
V4L2_PIX_FMT_NV12,
};
+static const struct rkvdec_ctrl_desc rkvdec_vp9_ctrl_descs[] = {
+ {
+ .cfg.id = V4L2_CID_STATELESS_VP9_FRAME,
+ },
+ {
+ .cfg.id = V4L2_CID_STATELESS_VP9_COMPRESSED_HDR,
+ },
+ {
+ .cfg.id = V4L2_CID_MPEG_VIDEO_VP9_PROFILE,
+ .cfg.min = V4L2_MPEG_VIDEO_VP9_PROFILE_0,
+ .cfg.max = V4L2_MPEG_VIDEO_VP9_PROFILE_0,
+ .cfg.def = V4L2_MPEG_VIDEO_VP9_PROFILE_0,
+ },
+};
+
+static const struct rkvdec_ctrls rkvdec_vp9_ctrls = {
+ .ctrls = rkvdec_vp9_ctrl_descs,
+ .num_ctrls = ARRAY_SIZE(rkvdec_vp9_ctrl_descs),
+};
+
static const struct rkvdec_coded_fmt_desc rkvdec_coded_fmts[] = {
{
.fourcc = V4L2_PIX_FMT_H264_SLICE,
@@ -116,8 +136,23 @@ static const struct rkvdec_coded_fmt_desc rkvdec_coded_fmts[] = {
},
.ctrls = &rkvdec_h264_ctrls,
.ops = &rkvdec_h264_fmt_ops,
- .num_decoded_fmts = ARRAY_SIZE(rkvdec_h264_decoded_fmts),
- .decoded_fmts = rkvdec_h264_decoded_fmts,
+ .num_decoded_fmts = ARRAY_SIZE(rkvdec_h264_vp9_decoded_fmts),
+ .decoded_fmts = rkvdec_h264_vp9_decoded_fmts,
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_VP9_FRAME,
+ .frmsize = {
+ .min_width = 64,
+ .max_width = 4096,
+ .step_width = 64,
+ .min_height = 64,
+ .max_height = 2304,
+ .step_height = 64,
+ },
+ .ctrls = &rkvdec_vp9_ctrls,
+ .ops = &rkvdec_vp9_fmt_ops,
+ .num_decoded_fmts = ARRAY_SIZE(rkvdec_h264_vp9_decoded_fmts),
+ .decoded_fmts = rkvdec_h264_vp9_decoded_fmts,
}
};
@@ -677,7 +712,7 @@ static void rkvdec_device_run(void *priv)
rkvdec_job_finish(ctx, VB2_BUF_STATE_ERROR);
}
-static struct v4l2_m2m_ops rkvdec_m2m_ops = {
+static const struct v4l2_m2m_ops rkvdec_m2m_ops = {
.device_run = rkvdec_device_run,
};
diff --git a/drivers/staging/media/rkvdec/rkvdec.h b/drivers/staging/media/rkvdec/rkvdec.h
index 52ac387..2f4ea17 100644
--- a/drivers/staging/media/rkvdec/rkvdec.h
+++ b/drivers/staging/media/rkvdec/rkvdec.h
@@ -42,14 +42,18 @@ struct rkvdec_run {
struct rkvdec_vp9_decoded_buffer_info {
/* Info needed when the decoded frame serves as a reference frame. */
- u16 width;
- u16 height;
- u32 bit_depth : 4;
+ unsigned short width;
+ unsigned short height;
+ unsigned int bit_depth : 4;
};
struct rkvdec_decoded_buffer {
/* Must be the first field in this struct. */
struct v4l2_m2m_buffer base;
+
+ union {
+ struct rkvdec_vp9_decoded_buffer_info vp9;
+ };
};
static inline struct rkvdec_decoded_buffer *
@@ -116,4 +120,6 @@ void rkvdec_run_preamble(struct rkvdec_ctx *ctx, struct rkvdec_run *run);
void rkvdec_run_postamble(struct rkvdec_ctx *ctx, struct rkvdec_run *run);
extern const struct rkvdec_coded_fmt_ops rkvdec_h264_fmt_ops;
+extern const struct rkvdec_coded_fmt_ops rkvdec_vp9_fmt_ops;
+
#endif /* RKVDEC_H_ */
diff --git a/drivers/staging/media/sunxi/cedrus/cedrus.c b/drivers/staging/media/sunxi/cedrus/cedrus.c
index c76fc97..4a4b714 100644
--- a/drivers/staging/media/sunxi/cedrus/cedrus.c
+++ b/drivers/staging/media/sunxi/cedrus/cedrus.c
@@ -580,6 +580,14 @@ static const struct cedrus_variant sun8i_r40_cedrus_variant = {
.mod_rate = 297000000,
};
+static const struct cedrus_variant sun20i_d1_cedrus_variant = {
+ .capabilities = CEDRUS_CAPABILITY_UNTILED |
+ CEDRUS_CAPABILITY_MPEG2_DEC |
+ CEDRUS_CAPABILITY_H264_DEC |
+ CEDRUS_CAPABILITY_H265_DEC,
+ .mod_rate = 432000000,
+};
+
static const struct cedrus_variant sun50i_a64_cedrus_variant = {
.capabilities = CEDRUS_CAPABILITY_UNTILED |
CEDRUS_CAPABILITY_MPEG2_DEC |
@@ -638,6 +646,10 @@ static const struct of_device_id cedrus_dt_match[] = {
.data = &sun8i_r40_cedrus_variant,
},
{
+ .compatible = "allwinner,sun20i-d1-video-engine",
+ .data = &sun20i_d1_cedrus_variant,
+ },
+ {
.compatible = "allwinner,sun50i-a64-video-engine",
.data = &sun50i_a64_cedrus_variant,
},
diff --git a/drivers/staging/media/tegra-vde/vde.c b/drivers/staging/media/tegra-vde/vde.c
index ed4c125..a8f1a02 100644
--- a/drivers/staging/media/tegra-vde/vde.c
+++ b/drivers/staging/media/tegra-vde/vde.c
@@ -20,6 +20,7 @@
#include <linux/slab.h>
#include <linux/uaccess.h>
+#include <soc/tegra/common.h>
#include <soc/tegra/pmc.h>
#include "uapi.h"
@@ -84,6 +85,96 @@ static int tegra_vde_wait_mbe(struct tegra_vde *vde)
(tmp >= 0x10), 1, 100);
}
+static int tegra_vde_alloc_bo(struct tegra_vde *vde,
+ struct tegra_vde_bo **ret_bo,
+ enum dma_data_direction dma_dir,
+ size_t size)
+{
+ struct device *dev = vde->miscdev.parent;
+ struct tegra_vde_bo *bo;
+ int err;
+
+ bo = kzalloc(sizeof(*bo), GFP_KERNEL);
+ if (!bo)
+ return -ENOMEM;
+
+ bo->vde = vde;
+ bo->size = size;
+ bo->dma_dir = dma_dir;
+ bo->dma_attrs = DMA_ATTR_WRITE_COMBINE |
+ DMA_ATTR_NO_KERNEL_MAPPING;
+
+ if (!vde->domain)
+ bo->dma_attrs |= DMA_ATTR_FORCE_CONTIGUOUS;
+
+ bo->dma_cookie = dma_alloc_attrs(dev, bo->size, &bo->dma_handle,
+ GFP_KERNEL, bo->dma_attrs);
+ if (!bo->dma_cookie) {
+ dev_err(dev, "Failed to allocate DMA buffer of size: %zu\n",
+ bo->size);
+ err = -ENOMEM;
+ goto free_bo;
+ }
+
+ err = dma_get_sgtable_attrs(dev, &bo->sgt, bo->dma_cookie,
+ bo->dma_handle, bo->size, bo->dma_attrs);
+ if (err) {
+ dev_err(dev, "Failed to get DMA buffer SG table: %d\n", err);
+ goto free_attrs;
+ }
+
+ err = dma_map_sgtable(dev, &bo->sgt, bo->dma_dir, bo->dma_attrs);
+ if (err) {
+ dev_err(dev, "Failed to map DMA buffer SG table: %d\n", err);
+ goto free_table;
+ }
+
+ if (vde->domain) {
+ err = tegra_vde_iommu_map(vde, &bo->sgt, &bo->iova, bo->size);
+ if (err) {
+ dev_err(dev, "Failed to map DMA buffer IOVA: %d\n", err);
+ goto unmap_sgtable;
+ }
+
+ bo->dma_addr = iova_dma_addr(&vde->iova, bo->iova);
+ } else {
+ bo->dma_addr = sg_dma_address(bo->sgt.sgl);
+ }
+
+ *ret_bo = bo;
+
+ return 0;
+
+unmap_sgtable:
+ dma_unmap_sgtable(dev, &bo->sgt, bo->dma_dir, bo->dma_attrs);
+free_table:
+ sg_free_table(&bo->sgt);
+free_attrs:
+ dma_free_attrs(dev, bo->size, bo->dma_cookie, bo->dma_handle,
+ bo->dma_attrs);
+free_bo:
+ kfree(bo);
+
+ return err;
+}
+
+static void tegra_vde_free_bo(struct tegra_vde_bo *bo)
+{
+ struct tegra_vde *vde = bo->vde;
+ struct device *dev = vde->miscdev.parent;
+
+ if (vde->domain)
+ tegra_vde_iommu_unmap(vde, bo->iova);
+
+ dma_unmap_sgtable(dev, &bo->sgt, bo->dma_dir, bo->dma_attrs);
+
+ sg_free_table(&bo->sgt);
+
+ dma_free_attrs(dev, bo->size, bo->dma_cookie, bo->dma_handle,
+ bo->dma_attrs);
+ kfree(bo);
+}
+
static int tegra_vde_setup_mbe_frame_idx(struct tegra_vde *vde,
unsigned int refs_nb,
bool setup_refs)
@@ -249,7 +340,7 @@ static void tegra_vde_setup_iram_tables(struct tegra_vde *vde,
value |= frame->frame_num;
} else {
aux_addr = 0x6ADEAD00;
- value = 0;
+ value = 0x3f;
}
tegra_vde_setup_iram_entry(vde, 0, i, value, aux_addr);
@@ -424,6 +515,9 @@ static int tegra_vde_setup_hw_context(struct tegra_vde *vde,
tegra_vde_writel(vde, bitstream_data_addr, vde->sxe, 0x6C);
+ if (vde->soc->supports_ref_pic_marking)
+ tegra_vde_writel(vde, vde->secure_bo->dma_addr, vde->sxe, 0x7c);
+
value = 0x10000005;
value |= ctx->pic_width_in_mbs << 11;
value |= ctx->pic_height_in_mbs << 3;
@@ -920,13 +1014,17 @@ static __maybe_unused int tegra_vde_runtime_suspend(struct device *dev)
struct tegra_vde *vde = dev_get_drvdata(dev);
int err;
- err = tegra_powergate_power_off(TEGRA_POWERGATE_VDEC);
- if (err) {
- dev_err(dev, "Failed to power down HW: %d\n", err);
- return err;
+ if (!dev->pm_domain) {
+ err = tegra_powergate_power_off(TEGRA_POWERGATE_VDEC);
+ if (err) {
+ dev_err(dev, "Failed to power down HW: %d\n", err);
+ return err;
+ }
}
clk_disable_unprepare(vde->clk);
+ reset_control_release(vde->rst);
+ reset_control_release(vde->rst_mc);
return 0;
}
@@ -936,14 +1034,45 @@ static __maybe_unused int tegra_vde_runtime_resume(struct device *dev)
struct tegra_vde *vde = dev_get_drvdata(dev);
int err;
- err = tegra_powergate_sequence_power_up(TEGRA_POWERGATE_VDEC,
- vde->clk, vde->rst);
+ err = reset_control_acquire(vde->rst_mc);
if (err) {
- dev_err(dev, "Failed to power up HW : %d\n", err);
+ dev_err(dev, "Failed to acquire mc reset: %d\n", err);
return err;
}
+ err = reset_control_acquire(vde->rst);
+ if (err) {
+ dev_err(dev, "Failed to acquire reset: %d\n", err);
+ goto release_mc_reset;
+ }
+
+ if (!dev->pm_domain) {
+ err = tegra_powergate_sequence_power_up(TEGRA_POWERGATE_VDEC,
+ vde->clk, vde->rst);
+ if (err) {
+ dev_err(dev, "Failed to power up HW : %d\n", err);
+ goto release_reset;
+ }
+ } else {
+ /*
+ * tegra_powergate_sequence_power_up() leaves clocks enabled,
+ * while GENPD not.
+ */
+ err = clk_prepare_enable(vde->clk);
+ if (err) {
+ dev_err(dev, "Failed to enable clock: %d\n", err);
+ goto release_reset;
+ }
+ }
+
return 0;
+
+release_reset:
+ reset_control_release(vde->rst);
+release_mc_reset:
+ reset_control_release(vde->rst_mc);
+
+ return err;
}
static int tegra_vde_probe(struct platform_device *pdev)
@@ -958,6 +1087,8 @@ static int tegra_vde_probe(struct platform_device *pdev)
platform_set_drvdata(pdev, vde);
+ vde->soc = of_device_get_match_data(&pdev->dev);
+
vde->sxe = devm_platform_ioremap_resource_byname(pdev, "sxe");
if (IS_ERR(vde->sxe))
return PTR_ERR(vde->sxe);
@@ -1001,14 +1132,14 @@ static int tegra_vde_probe(struct platform_device *pdev)
return err;
}
- vde->rst = devm_reset_control_get(dev, NULL);
+ vde->rst = devm_reset_control_get_exclusive_released(dev, NULL);
if (IS_ERR(vde->rst)) {
err = PTR_ERR(vde->rst);
dev_err(dev, "Could not get VDE reset %d\n", err);
return err;
}
- vde->rst_mc = devm_reset_control_get_optional(dev, "mc");
+ vde->rst_mc = devm_reset_control_get_optional_exclusive_released(dev, "mc");
if (IS_ERR(vde->rst_mc)) {
err = PTR_ERR(vde->rst_mc);
dev_err(dev, "Could not get MC reset %d\n", err);
@@ -1026,6 +1157,12 @@ static int tegra_vde_probe(struct platform_device *pdev)
return err;
}
+ err = devm_tegra_core_dev_init_opp_table_common(dev);
+ if (err) {
+ dev_err(dev, "Could initialize OPP table %d\n", err);
+ return err;
+ }
+
vde->iram_pool = of_gen_pool_get(dev->of_node, "iram", 0);
if (!vde->iram_pool) {
dev_err(dev, "Could not get IRAM pool\n");
@@ -1056,12 +1193,6 @@ static int tegra_vde_probe(struct platform_device *pdev)
goto err_gen_free;
}
- err = misc_register(&vde->miscdev);
- if (err) {
- dev_err(dev, "Failed to register misc device: %d\n", err);
- goto err_deinit_iommu;
- }
-
pm_runtime_enable(dev);
pm_runtime_use_autosuspend(dev);
pm_runtime_set_autosuspend_delay(dev, 300);
@@ -1077,15 +1208,26 @@ static int tegra_vde_probe(struct platform_device *pdev)
pm_runtime_put(dev);
+ err = tegra_vde_alloc_bo(vde, &vde->secure_bo, DMA_FROM_DEVICE, 4096);
+ if (err) {
+ dev_err(dev, "Failed to allocate secure BO: %d\n", err);
+ goto err_pm_runtime;
+ }
+
+ err = misc_register(&vde->miscdev);
+ if (err) {
+ dev_err(dev, "Failed to register misc device: %d\n", err);
+ goto err_free_secure_bo;
+ }
+
return 0;
+err_free_secure_bo:
+ tegra_vde_free_bo(vde->secure_bo);
err_pm_runtime:
- misc_deregister(&vde->miscdev);
-
pm_runtime_dont_use_autosuspend(dev);
pm_runtime_disable(dev);
-err_deinit_iommu:
tegra_vde_iommu_deinit(vde);
err_gen_free:
@@ -1100,6 +1242,10 @@ static int tegra_vde_remove(struct platform_device *pdev)
struct tegra_vde *vde = platform_get_drvdata(pdev);
struct device *dev = &pdev->dev;
+ misc_deregister(&vde->miscdev);
+
+ tegra_vde_free_bo(vde->secure_bo);
+
/*
* As it increments RPM usage_count even on errors, we don't need to
* check the returned code here.
@@ -1116,8 +1262,6 @@ static int tegra_vde_remove(struct platform_device *pdev)
pm_runtime_put_noidle(dev);
clk_disable_unprepare(vde->clk);
- misc_deregister(&vde->miscdev);
-
tegra_vde_dmabuf_cache_unmap_all(vde);
tegra_vde_iommu_deinit(vde);
@@ -1133,8 +1277,7 @@ static void tegra_vde_shutdown(struct platform_device *pdev)
* On some devices bootloader isn't ready to a power-gated VDE on
* a warm-reboot, machine will hang in that case.
*/
- if (pm_runtime_status_suspended(&pdev->dev))
- tegra_vde_runtime_resume(&pdev->dev);
+ pm_runtime_get_sync(&pdev->dev);
}
static __maybe_unused int tegra_vde_pm_suspend(struct device *dev)
@@ -1173,8 +1316,27 @@ static const struct dev_pm_ops tegra_vde_pm_ops = {
tegra_vde_pm_resume)
};
+static const struct tegra_vde_soc tegra124_vde_soc = {
+ .supports_ref_pic_marking = true,
+};
+
+static const struct tegra_vde_soc tegra114_vde_soc = {
+ .supports_ref_pic_marking = true,
+};
+
+static const struct tegra_vde_soc tegra30_vde_soc = {
+ .supports_ref_pic_marking = false,
+};
+
+static const struct tegra_vde_soc tegra20_vde_soc = {
+ .supports_ref_pic_marking = false,
+};
+
static const struct of_device_id tegra_vde_of_match[] = {
- { .compatible = "nvidia,tegra20-vde", },
+ { .compatible = "nvidia,tegra124-vde", .data = &tegra124_vde_soc },
+ { .compatible = "nvidia,tegra114-vde", .data = &tegra114_vde_soc },
+ { .compatible = "nvidia,tegra30-vde", .data = &tegra30_vde_soc },
+ { .compatible = "nvidia,tegra20-vde", .data = &tegra20_vde_soc },
{ },
};
MODULE_DEVICE_TABLE(of, tegra_vde_of_match);
diff --git a/drivers/staging/media/tegra-vde/vde.h b/drivers/staging/media/tegra-vde/vde.h
index 5561291..bbd42b8 100644
--- a/drivers/staging/media/tegra-vde/vde.h
+++ b/drivers/staging/media/tegra-vde/vde.h
@@ -24,6 +24,22 @@ struct iommu_domain;
struct reset_control;
struct dma_buf_attachment;
+struct tegra_vde_soc {
+ bool supports_ref_pic_marking;
+};
+
+struct tegra_vde_bo {
+ struct iova *iova;
+ struct sg_table sgt;
+ struct tegra_vde *vde;
+ enum dma_data_direction dma_dir;
+ unsigned long dma_attrs;
+ dma_addr_t dma_handle;
+ dma_addr_t dma_addr;
+ void *dma_cookie;
+ size_t size;
+};
+
struct tegra_vde {
void __iomem *sxe;
void __iomem *bsev;
@@ -48,6 +64,8 @@ struct tegra_vde {
struct iova_domain iova;
struct iova *iova_resv_static_addresses;
struct iova *iova_resv_last_page;
+ const struct tegra_vde_soc *soc;
+ struct tegra_vde_bo *secure_bo;
dma_addr_t iram_lists_addr;
u32 *iram;
};
diff --git a/drivers/staging/rtl8192e/rtllib_softmac.c b/drivers/staging/rtl8192e/rtllib_softmac.c
index d2726d0..aabbea4 100644
--- a/drivers/staging/rtl8192e/rtllib_softmac.c
+++ b/drivers/staging/rtl8192e/rtllib_softmac.c
@@ -2515,7 +2515,7 @@ void rtllib_stop_all_queues(struct rtllib_device *ieee)
unsigned int i;
for (i = 0; i < ieee->dev->num_tx_queues; i++)
- netdev_get_tx_queue(ieee->dev, i)->trans_start = jiffies;
+ txq_trans_cond_update(netdev_get_tx_queue(ieee->dev, i));
netif_tx_stop_all_queues(ieee->dev);
}
diff --git a/drivers/tee/optee/Makefile b/drivers/tee/optee/Makefile
index 66b8a17..a6eff38 100644
--- a/drivers/tee/optee/Makefile
+++ b/drivers/tee/optee/Makefile
@@ -2,6 +2,7 @@
obj-$(CONFIG_OPTEE) += optee.o
optee-objs += core.o
optee-objs += call.o
+optee-objs += notif.o
optee-objs += rpc.o
optee-objs += supp.o
optee-objs += device.o
diff --git a/drivers/tee/optee/core.c b/drivers/tee/optee/core.c
index 2a66a52..1ca3208 100644
--- a/drivers/tee/optee/core.c
+++ b/drivers/tee/optee/core.c
@@ -157,6 +157,7 @@ void optee_remove_common(struct optee *optee)
/* Unregister OP-TEE specific client devices on TEE bus */
optee_unregister_devices();
+ optee_notif_uninit(optee);
/*
* The two devices have to be unregistered before we can free the
* other resources.
@@ -165,7 +166,6 @@ void optee_remove_common(struct optee *optee)
tee_device_unregister(optee->teedev);
tee_shm_pool_free(optee->pool);
- optee_wait_queue_exit(&optee->wait_queue);
optee_supp_uninit(&optee->supp);
mutex_destroy(&optee->call_queue.mutex);
}
diff --git a/drivers/tee/optee/ffa_abi.c b/drivers/tee/optee/ffa_abi.c
index d8c8683..20a1b1a 100644
--- a/drivers/tee/optee/ffa_abi.c
+++ b/drivers/tee/optee/ffa_abi.c
@@ -855,9 +855,13 @@ static int optee_ffa_probe(struct ffa_device *ffa_dev)
mutex_init(&optee->ffa.mutex);
mutex_init(&optee->call_queue.mutex);
INIT_LIST_HEAD(&optee->call_queue.waiters);
- optee_wait_queue_init(&optee->wait_queue);
optee_supp_init(&optee->supp);
ffa_dev_set_drvdata(ffa_dev, optee);
+ rc = optee_notif_init(optee, OPTEE_DEFAULT_MAX_NOTIF_VALUE);
+ if (rc) {
+ optee_ffa_remove(ffa_dev);
+ return rc;
+ }
rc = optee_enumerate_devices(PTA_CMD_GET_DEVICES);
if (rc) {
diff --git a/drivers/tee/optee/notif.c b/drivers/tee/optee/notif.c
new file mode 100644
index 0000000..a28fa03
--- /dev/null
+++ b/drivers/tee/optee/notif.c
@@ -0,0 +1,125 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2015-2021, Linaro Limited
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/arm-smccc.h>
+#include <linux/errno.h>
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+#include <linux/tee_drv.h>
+#include "optee_private.h"
+
+struct notif_entry {
+ struct list_head link;
+ struct completion c;
+ u_int key;
+};
+
+static bool have_key(struct optee *optee, u_int key)
+{
+ struct notif_entry *entry;
+
+ list_for_each_entry(entry, &optee->notif.db, link)
+ if (entry->key == key)
+ return true;
+
+ return false;
+}
+
+int optee_notif_wait(struct optee *optee, u_int key)
+{
+ unsigned long flags;
+ struct notif_entry *entry;
+ int rc = 0;
+
+ if (key > optee->notif.max_key)
+ return -EINVAL;
+
+ entry = kmalloc(sizeof(*entry), GFP_KERNEL);
+ if (!entry)
+ return -ENOMEM;
+ init_completion(&entry->c);
+ entry->key = key;
+
+ spin_lock_irqsave(&optee->notif.lock, flags);
+
+ /*
+ * If the bit is already set it means that the key has already
+ * been posted and we must not wait.
+ */
+ if (test_bit(key, optee->notif.bitmap)) {
+ clear_bit(key, optee->notif.bitmap);
+ goto out;
+ }
+
+ /*
+ * Check if someone is already waiting for this key. If there is
+ * it's a programming error.
+ */
+ if (have_key(optee, key)) {
+ rc = -EBUSY;
+ goto out;
+ }
+
+ list_add_tail(&entry->link, &optee->notif.db);
+
+ /*
+ * Unlock temporarily and wait for completion.
+ */
+ spin_unlock_irqrestore(&optee->notif.lock, flags);
+ wait_for_completion(&entry->c);
+ spin_lock_irqsave(&optee->notif.lock, flags);
+
+ list_del(&entry->link);
+out:
+ spin_unlock_irqrestore(&optee->notif.lock, flags);
+
+ kfree(entry);
+
+ return rc;
+}
+
+int optee_notif_send(struct optee *optee, u_int key)
+{
+ unsigned long flags;
+ struct notif_entry *entry;
+
+ if (key > optee->notif.max_key)
+ return -EINVAL;
+
+ spin_lock_irqsave(&optee->notif.lock, flags);
+
+ list_for_each_entry(entry, &optee->notif.db, link)
+ if (entry->key == key) {
+ complete(&entry->c);
+ goto out;
+ }
+
+ /* Only set the bit in case there where nobody waiting */
+ set_bit(key, optee->notif.bitmap);
+out:
+ spin_unlock_irqrestore(&optee->notif.lock, flags);
+
+ return 0;
+}
+
+int optee_notif_init(struct optee *optee, u_int max_key)
+{
+ spin_lock_init(&optee->notif.lock);
+ INIT_LIST_HEAD(&optee->notif.db);
+ optee->notif.bitmap = bitmap_zalloc(max_key, GFP_KERNEL);
+ if (!optee->notif.bitmap)
+ return -ENOMEM;
+
+ optee->notif.max_key = max_key;
+
+ return 0;
+}
+
+void optee_notif_uninit(struct optee *optee)
+{
+ kfree(optee->notif.bitmap);
+}
diff --git a/drivers/tee/optee/optee_msg.h b/drivers/tee/optee/optee_msg.h
index 2422e18..70e9cc2 100644
--- a/drivers/tee/optee/optee_msg.h
+++ b/drivers/tee/optee/optee_msg.h
@@ -318,6 +318,13 @@ struct optee_msg_arg {
* [in] param[0].u.rmem.shm_ref holds shared memory reference
* [in] param[0].u.rmem.offs 0
* [in] param[0].u.rmem.size 0
+ *
+ * OPTEE_MSG_CMD_DO_BOTTOM_HALF does the scheduled bottom half processing
+ * of a driver.
+ *
+ * OPTEE_MSG_CMD_STOP_ASYNC_NOTIF informs secure world that from now is
+ * normal world unable to process asynchronous notifications. Typically
+ * used when the driver is shut down.
*/
#define OPTEE_MSG_CMD_OPEN_SESSION 0
#define OPTEE_MSG_CMD_INVOKE_COMMAND 1
@@ -325,6 +332,8 @@ struct optee_msg_arg {
#define OPTEE_MSG_CMD_CANCEL 3
#define OPTEE_MSG_CMD_REGISTER_SHM 4
#define OPTEE_MSG_CMD_UNREGISTER_SHM 5
+#define OPTEE_MSG_CMD_DO_BOTTOM_HALF 6
+#define OPTEE_MSG_CMD_STOP_ASYNC_NOTIF 7
#define OPTEE_MSG_FUNCID_CALL_WITH_ARG 0x0004
#endif /* _OPTEE_MSG_H */
diff --git a/drivers/tee/optee/optee_private.h b/drivers/tee/optee/optee_private.h
index 6660e05..46f74ab 100644
--- a/drivers/tee/optee/optee_private.h
+++ b/drivers/tee/optee/optee_private.h
@@ -28,6 +28,13 @@
#define TEEC_ORIGIN_COMMS 0x00000002
+/*
+ * This value should be larger than the number threads in secure world to
+ * meet the need from secure world. The number of threads in secure world
+ * are usually not even close to 255 so we should be safe for now.
+ */
+#define OPTEE_DEFAULT_MAX_NOTIF_VALUE 255
+
typedef void (optee_invoke_fn)(unsigned long, unsigned long, unsigned long,
unsigned long, unsigned long, unsigned long,
unsigned long, unsigned long,
@@ -44,10 +51,13 @@ struct optee_call_queue {
struct list_head waiters;
};
-struct optee_wait_queue {
- /* Serializes access to this struct */
- struct mutex mu;
+struct optee_notif {
+ u_int max_key;
+ struct tee_context *ctx;
+ /* Serializes access to the elements below in this struct */
+ spinlock_t lock;
struct list_head db;
+ u_long *bitmap;
};
/**
@@ -79,6 +89,7 @@ struct optee_smc {
optee_invoke_fn *invoke_fn;
void *memremaped_shm;
u32 sec_caps;
+ unsigned int notif_irq;
};
/**
@@ -129,8 +140,7 @@ struct optee_ops {
* @smc: specific to SMC ABI
* @ffa: specific to FF-A ABI
* @call_queue: queue of threads waiting to call @invoke_fn
- * @wait_queue: queue of threads from secure world waiting for a
- * secure world sync object
+ * @notif: notification synchronization struct
* @supp: supplicant synchronization struct for RPC to supplicant
* @pool: shared memory pool
* @rpc_arg_count: If > 0 number of RPC parameters to make room for
@@ -147,7 +157,7 @@ struct optee {
struct optee_ffa ffa;
};
struct optee_call_queue call_queue;
- struct optee_wait_queue wait_queue;
+ struct optee_notif notif;
struct optee_supp supp;
struct tee_shm_pool *pool;
unsigned int rpc_arg_count;
@@ -185,8 +195,10 @@ struct optee_call_ctx {
size_t num_entries;
};
-void optee_wait_queue_init(struct optee_wait_queue *wq);
-void optee_wait_queue_exit(struct optee_wait_queue *wq);
+int optee_notif_init(struct optee *optee, u_int max_key);
+void optee_notif_uninit(struct optee *optee);
+int optee_notif_wait(struct optee *optee, u_int key);
+int optee_notif_send(struct optee *optee, u_int key);
u32 optee_supp_thrd_req(struct tee_context *ctx, u32 func, size_t num_params,
struct tee_param *param);
diff --git a/drivers/tee/optee/optee_rpc_cmd.h b/drivers/tee/optee/optee_rpc_cmd.h
index b827514..f3f06e0 100644
--- a/drivers/tee/optee/optee_rpc_cmd.h
+++ b/drivers/tee/optee/optee_rpc_cmd.h
@@ -28,24 +28,27 @@
#define OPTEE_RPC_CMD_GET_TIME 3
/*
- * Wait queue primitive, helper for secure world to implement a wait queue.
+ * Notification from/to secure world.
*
- * If secure world needs to wait for a secure world mutex it issues a sleep
- * request instead of spinning in secure world. Conversely is a wakeup
- * request issued when a secure world mutex with a thread waiting thread is
- * unlocked.
+ * If secure world needs to wait for something, for instance a mutex, it
+ * does a notification wait request instead of spinning in secure world.
+ * Conversely can a synchronous notification can be sent when a secure
+ * world mutex with a thread waiting thread is unlocked.
*
- * Waiting on a key
- * [in] value[0].a OPTEE_RPC_WAIT_QUEUE_SLEEP
- * [in] value[0].b Wait key
+ * This interface can also be used to wait for a asynchronous notification
+ * which instead is sent via a non-secure interrupt.
*
- * Waking up a key
- * [in] value[0].a OPTEE_RPC_WAIT_QUEUE_WAKEUP
- * [in] value[0].b Wakeup key
+ * Waiting on notification
+ * [in] value[0].a OPTEE_RPC_NOTIFICATION_WAIT
+ * [in] value[0].b notification value
+ *
+ * Sending a synchronous notification
+ * [in] value[0].a OPTEE_RPC_NOTIFICATION_SEND
+ * [in] value[0].b notification value
*/
-#define OPTEE_RPC_CMD_WAIT_QUEUE 4
-#define OPTEE_RPC_WAIT_QUEUE_SLEEP 0
-#define OPTEE_RPC_WAIT_QUEUE_WAKEUP 1
+#define OPTEE_RPC_CMD_NOTIFICATION 4
+#define OPTEE_RPC_NOTIFICATION_WAIT 0
+#define OPTEE_RPC_NOTIFICATION_SEND 1
/*
* Suspend execution
diff --git a/drivers/tee/optee/optee_smc.h b/drivers/tee/optee/optee_smc.h
index 80eb763..d44a6ae 100644
--- a/drivers/tee/optee/optee_smc.h
+++ b/drivers/tee/optee/optee_smc.h
@@ -107,6 +107,12 @@ struct optee_smc_call_get_os_revision_result {
/*
* Call with struct optee_msg_arg as argument
*
+ * When calling this function normal world has a few responsibilities:
+ * 1. It must be able to handle eventual RPCs
+ * 2. Non-secure interrupts should not be masked
+ * 3. If asynchronous notifications has been negotiated successfully, then
+ * asynchronous notifications should be unmasked during this call.
+ *
* Call register usage:
* a0 SMC Function ID, OPTEE_SMC*CALL_WITH_ARG
* a1 Upper 32 bits of a 64-bit physical pointer to a struct optee_msg_arg
@@ -195,7 +201,8 @@ struct optee_smc_get_shm_config_result {
* Normal return register usage:
* a0 OPTEE_SMC_RETURN_OK
* a1 bitfield of secure world capabilities OPTEE_SMC_SEC_CAP_*
- * a2-7 Preserved
+ * a2 The maximum secure world notification number
+ * a3-7 Preserved
*
* Error return register usage:
* a0 OPTEE_SMC_RETURN_ENOTAVAIL, can't use the capabilities from normal world
@@ -218,6 +225,8 @@ struct optee_smc_get_shm_config_result {
#define OPTEE_SMC_SEC_CAP_VIRTUALIZATION BIT(3)
/* Secure world supports Shared Memory with a NULL reference */
#define OPTEE_SMC_SEC_CAP_MEMREF_NULL BIT(4)
+/* Secure world supports asynchronous notification of normal world */
+#define OPTEE_SMC_SEC_CAP_ASYNC_NOTIF BIT(5)
#define OPTEE_SMC_FUNCID_EXCHANGE_CAPABILITIES 9
#define OPTEE_SMC_EXCHANGE_CAPABILITIES \
@@ -226,8 +235,8 @@ struct optee_smc_get_shm_config_result {
struct optee_smc_exchange_capabilities_result {
unsigned long status;
unsigned long capabilities;
+ unsigned long max_notif_value;
unsigned long reserved0;
- unsigned long reserved1;
};
/*
@@ -320,6 +329,68 @@ struct optee_smc_disable_shm_cache_result {
OPTEE_SMC_FAST_CALL_VAL(OPTEE_SMC_FUNCID_GET_THREAD_COUNT)
/*
+ * Inform OP-TEE that normal world is able to receive asynchronous
+ * notifications.
+ *
+ * Call requests usage:
+ * a0 SMC Function ID, OPTEE_SMC_ENABLE_ASYNC_NOTIF
+ * a1-6 Not used
+ * a7 Hypervisor Client ID register
+ *
+ * Normal return register usage:
+ * a0 OPTEE_SMC_RETURN_OK
+ * a1-7 Preserved
+ *
+ * Not supported return register usage:
+ * a0 OPTEE_SMC_RETURN_ENOTAVAIL
+ * a1-7 Preserved
+ */
+#define OPTEE_SMC_FUNCID_ENABLE_ASYNC_NOTIF 16
+#define OPTEE_SMC_ENABLE_ASYNC_NOTIF \
+ OPTEE_SMC_FAST_CALL_VAL(OPTEE_SMC_FUNCID_ENABLE_ASYNC_NOTIF)
+
+/*
+ * Retrieve a value of notifications pending since the last call of this
+ * function.
+ *
+ * OP-TEE keeps a record of all posted values. When an interrupt is
+ * received which indicates that there are posted values this function
+ * should be called until all pended values have been retrieved. When a
+ * value is retrieved, it's cleared from the record in secure world.
+ *
+ * Call requests usage:
+ * a0 SMC Function ID, OPTEE_SMC_GET_ASYNC_NOTIF_VALUE
+ * a1-6 Not used
+ * a7 Hypervisor Client ID register
+ *
+ * Normal return register usage:
+ * a0 OPTEE_SMC_RETURN_OK
+ * a1 value
+ * a2 Bit[0]: OPTEE_SMC_ASYNC_NOTIF_VALUE_VALID if the value in a1 is
+ * valid, else 0 if no values where pending
+ * a2 Bit[1]: OPTEE_SMC_ASYNC_NOTIF_VALUE_PENDING if another value is
+ * pending, else 0.
+ * Bit[31:2]: MBZ
+ * a3-7 Preserved
+ *
+ * Not supported return register usage:
+ * a0 OPTEE_SMC_RETURN_ENOTAVAIL
+ * a1-7 Preserved
+ */
+#define OPTEE_SMC_ASYNC_NOTIF_VALUE_VALID BIT(0)
+#define OPTEE_SMC_ASYNC_NOTIF_VALUE_PENDING BIT(1)
+
+/*
+ * Notification that OP-TEE expects a yielding call to do some bottom half
+ * work in a driver.
+ */
+#define OPTEE_SMC_ASYNC_NOTIF_VALUE_DO_BOTTOM_HALF 0
+
+#define OPTEE_SMC_FUNCID_GET_ASYNC_NOTIF_VALUE 17
+#define OPTEE_SMC_GET_ASYNC_NOTIF_VALUE \
+ OPTEE_SMC_FAST_CALL_VAL(OPTEE_SMC_FUNCID_GET_ASYNC_NOTIF_VALUE)
+
+/*
* Resume from RPC (for example after processing a foreign interrupt)
*
* Call register usage:
diff --git a/drivers/tee/optee/rpc.c b/drivers/tee/optee/rpc.c
index cd642e3..e69bc63 100644
--- a/drivers/tee/optee/rpc.c
+++ b/drivers/tee/optee/rpc.c
@@ -12,23 +12,6 @@
#include "optee_private.h"
#include "optee_rpc_cmd.h"
-struct wq_entry {
- struct list_head link;
- struct completion c;
- u32 key;
-};
-
-void optee_wait_queue_init(struct optee_wait_queue *priv)
-{
- mutex_init(&priv->mu);
- INIT_LIST_HEAD(&priv->db);
-}
-
-void optee_wait_queue_exit(struct optee_wait_queue *priv)
-{
- mutex_destroy(&priv->mu);
-}
-
static void handle_rpc_func_cmd_get_time(struct optee_msg_arg *arg)
{
struct timespec64 ts;
@@ -144,48 +127,6 @@ static void handle_rpc_func_cmd_i2c_transfer(struct tee_context *ctx,
}
#endif
-static struct wq_entry *wq_entry_get(struct optee_wait_queue *wq, u32 key)
-{
- struct wq_entry *w;
-
- mutex_lock(&wq->mu);
-
- list_for_each_entry(w, &wq->db, link)
- if (w->key == key)
- goto out;
-
- w = kmalloc(sizeof(*w), GFP_KERNEL);
- if (w) {
- init_completion(&w->c);
- w->key = key;
- list_add_tail(&w->link, &wq->db);
- }
-out:
- mutex_unlock(&wq->mu);
- return w;
-}
-
-static void wq_sleep(struct optee_wait_queue *wq, u32 key)
-{
- struct wq_entry *w = wq_entry_get(wq, key);
-
- if (w) {
- wait_for_completion(&w->c);
- mutex_lock(&wq->mu);
- list_del(&w->link);
- mutex_unlock(&wq->mu);
- kfree(w);
- }
-}
-
-static void wq_wakeup(struct optee_wait_queue *wq, u32 key)
-{
- struct wq_entry *w = wq_entry_get(wq, key);
-
- if (w)
- complete(&w->c);
-}
-
static void handle_rpc_func_cmd_wq(struct optee *optee,
struct optee_msg_arg *arg)
{
@@ -197,11 +138,13 @@ static void handle_rpc_func_cmd_wq(struct optee *optee,
goto bad;
switch (arg->params[0].u.value.a) {
- case OPTEE_RPC_WAIT_QUEUE_SLEEP:
- wq_sleep(&optee->wait_queue, arg->params[0].u.value.b);
+ case OPTEE_RPC_NOTIFICATION_WAIT:
+ if (optee_notif_wait(optee, arg->params[0].u.value.b))
+ goto bad;
break;
- case OPTEE_RPC_WAIT_QUEUE_WAKEUP:
- wq_wakeup(&optee->wait_queue, arg->params[0].u.value.b);
+ case OPTEE_RPC_NOTIFICATION_SEND:
+ if (optee_notif_send(optee, arg->params[0].u.value.b))
+ goto bad;
break;
default:
goto bad;
@@ -319,7 +262,7 @@ void optee_rpc_cmd(struct tee_context *ctx, struct optee *optee,
case OPTEE_RPC_CMD_GET_TIME:
handle_rpc_func_cmd_get_time(arg);
break;
- case OPTEE_RPC_CMD_WAIT_QUEUE:
+ case OPTEE_RPC_CMD_NOTIFICATION:
handle_rpc_func_cmd_wq(optee, arg);
break;
case OPTEE_RPC_CMD_SUSPEND:
diff --git a/drivers/tee/optee/smc_abi.c b/drivers/tee/optee/smc_abi.c
index cf2e329..449d6a7 100644
--- a/drivers/tee/optee/smc_abi.c
+++ b/drivers/tee/optee/smc_abi.c
@@ -8,13 +8,16 @@
#include <linux/arm-smccc.h>
#include <linux/errno.h>
+#include <linux/interrupt.h>
#include <linux/io.h>
-#include <linux/sched.h>
+#include <linux/irqdomain.h>
#include <linux/mm.h>
#include <linux/module.h>
#include <linux/of.h>
+#include <linux/of_irq.h>
#include <linux/of_platform.h>
#include <linux/platform_device.h>
+#include <linux/sched.h>
#include <linux/slab.h>
#include <linux/string.h>
#include <linux/tee_drv.h>
@@ -35,7 +38,8 @@
* 2. Low level support functions to register shared memory in secure world
* 3. Dynamic shared memory pool based on alloc_pages()
* 4. Do a normal scheduled call into secure world
- * 5. Driver initialization.
+ * 5. Asynchronous notification
+ * 6. Driver initialization.
*/
#define OPTEE_SHM_NUM_PRIV_PAGES CONFIG_OPTEE_SHM_NUM_PRIV_PAGES
@@ -877,10 +881,137 @@ static int optee_smc_do_call_with_arg(struct tee_context *ctx,
return rc;
}
+static int simple_call_with_arg(struct tee_context *ctx, u32 cmd)
+{
+ struct optee_msg_arg *msg_arg;
+ struct tee_shm *shm;
+
+ shm = optee_get_msg_arg(ctx, 0, &msg_arg);
+ if (IS_ERR(shm))
+ return PTR_ERR(shm);
+
+ msg_arg->cmd = cmd;
+ optee_smc_do_call_with_arg(ctx, shm);
+
+ tee_shm_free(shm);
+ return 0;
+}
+
+static int optee_smc_do_bottom_half(struct tee_context *ctx)
+{
+ return simple_call_with_arg(ctx, OPTEE_MSG_CMD_DO_BOTTOM_HALF);
+}
+
+static int optee_smc_stop_async_notif(struct tee_context *ctx)
+{
+ return simple_call_with_arg(ctx, OPTEE_MSG_CMD_STOP_ASYNC_NOTIF);
+}
+
/*
- * 5. Driver initialization
+ * 5. Asynchronous notification
+ */
+
+static u32 get_async_notif_value(optee_invoke_fn *invoke_fn, bool *value_valid,
+ bool *value_pending)
+{
+ struct arm_smccc_res res;
+
+ invoke_fn(OPTEE_SMC_GET_ASYNC_NOTIF_VALUE, 0, 0, 0, 0, 0, 0, 0, &res);
+
+ if (res.a0)
+ return 0;
+ *value_valid = (res.a2 & OPTEE_SMC_ASYNC_NOTIF_VALUE_VALID);
+ *value_pending = (res.a2 & OPTEE_SMC_ASYNC_NOTIF_VALUE_PENDING);
+ return res.a1;
+}
+
+static irqreturn_t notif_irq_handler(int irq, void *dev_id)
+{
+ struct optee *optee = dev_id;
+ bool do_bottom_half = false;
+ bool value_valid;
+ bool value_pending;
+ u32 value;
+
+ do {
+ value = get_async_notif_value(optee->smc.invoke_fn,
+ &value_valid, &value_pending);
+ if (!value_valid)
+ break;
+
+ if (value == OPTEE_SMC_ASYNC_NOTIF_VALUE_DO_BOTTOM_HALF)
+ do_bottom_half = true;
+ else
+ optee_notif_send(optee, value);
+ } while (value_pending);
+
+ if (do_bottom_half)
+ return IRQ_WAKE_THREAD;
+ return IRQ_HANDLED;
+}
+
+static irqreturn_t notif_irq_thread_fn(int irq, void *dev_id)
+{
+ struct optee *optee = dev_id;
+
+ optee_smc_do_bottom_half(optee->notif.ctx);
+
+ return IRQ_HANDLED;
+}
+
+static int optee_smc_notif_init_irq(struct optee *optee, u_int irq)
+{
+ struct tee_context *ctx;
+ int rc;
+
+ ctx = teedev_open(optee->teedev);
+ if (IS_ERR(ctx))
+ return PTR_ERR(ctx);
+
+ optee->notif.ctx = ctx;
+ rc = request_threaded_irq(irq, notif_irq_handler,
+ notif_irq_thread_fn,
+ 0, "optee_notification", optee);
+ if (rc)
+ goto err_close_ctx;
+
+ optee->smc.notif_irq = irq;
+
+ return 0;
+
+err_close_ctx:
+ teedev_close_context(optee->notif.ctx);
+ optee->notif.ctx = NULL;
+
+ return rc;
+}
+
+static void optee_smc_notif_uninit_irq(struct optee *optee)
+{
+ if (optee->notif.ctx) {
+ optee_smc_stop_async_notif(optee->notif.ctx);
+ if (optee->smc.notif_irq) {
+ free_irq(optee->smc.notif_irq, optee);
+ irq_dispose_mapping(optee->smc.notif_irq);
+ }
+
+ /*
+ * The thread normally working with optee->notif.ctx was
+ * stopped with free_irq() above.
+ *
+ * Note we're not using teedev_close_context() or
+ * tee_client_close_context() since we have already called
+ * tee_device_put() while initializing to avoid a circular
+ * reference counting.
+ */
+ teedev_close_context(optee->notif.ctx);
+ }
+}
+
+/*
+ * 6. Driver initialization
*
- * During driver inititialization is secure world probed to find out which
+ * During driver initialization is secure world probed to find out which
* features it supports so the driver can be initialized with a matching
* configuration. This involves for instance support for dynamic shared
* memory instead of a static memory carvout.
@@ -952,6 +1083,17 @@ static const struct optee_ops optee_ops = {
.from_msg_param = optee_from_msg_param,
};
+static int enable_async_notif(optee_invoke_fn *invoke_fn)
+{
+ struct arm_smccc_res res;
+
+ invoke_fn(OPTEE_SMC_ENABLE_ASYNC_NOTIF, 0, 0, 0, 0, 0, 0, 0, &res);
+
+ if (res.a0)
+ return -EINVAL;
+ return 0;
+}
+
static bool optee_msg_api_uid_is_optee_api(optee_invoke_fn *invoke_fn)
{
struct arm_smccc_res res;
@@ -1001,7 +1143,7 @@ static bool optee_msg_api_revision_is_compatible(optee_invoke_fn *invoke_fn)
}
static bool optee_msg_exchange_capabilities(optee_invoke_fn *invoke_fn,
- u32 *sec_caps)
+ u32 *sec_caps, u32 *max_notif_value)
{
union {
struct arm_smccc_res smccc;
@@ -1024,6 +1166,11 @@ static bool optee_msg_exchange_capabilities(optee_invoke_fn *invoke_fn,
return false;
*sec_caps = res.result.capabilities;
+ if (*sec_caps & OPTEE_SMC_SEC_CAP_ASYNC_NOTIF)
+ *max_notif_value = res.result.max_notif_value;
+ else
+ *max_notif_value = OPTEE_DEFAULT_MAX_NOTIF_VALUE;
+
return true;
}
@@ -1188,6 +1335,8 @@ static int optee_smc_remove(struct platform_device *pdev)
*/
optee_disable_shm_cache(optee);
+ optee_smc_notif_uninit_irq(optee);
+
optee_remove_common(optee);
if (optee->smc.memremaped_shm)
@@ -1217,6 +1366,7 @@ static int optee_probe(struct platform_device *pdev)
struct optee *optee = NULL;
void *memremaped_shm = NULL;
struct tee_device *teedev;
+ u32 max_notif_value;
u32 sec_caps;
int rc;
@@ -1236,7 +1386,8 @@ static int optee_probe(struct platform_device *pdev)
return -EINVAL;
}
- if (!optee_msg_exchange_capabilities(invoke_fn, &sec_caps)) {
+ if (!optee_msg_exchange_capabilities(invoke_fn, &sec_caps,
+ &max_notif_value)) {
pr_warn("capabilities mismatch\n");
return -EINVAL;
}
@@ -1259,7 +1410,7 @@ static int optee_probe(struct platform_device *pdev)
optee = kzalloc(sizeof(*optee), GFP_KERNEL);
if (!optee) {
rc = -ENOMEM;
- goto err;
+ goto err_free_pool;
}
optee->ops = &optee_ops;
@@ -1269,32 +1420,55 @@ static int optee_probe(struct platform_device *pdev)
teedev = tee_device_alloc(&optee_clnt_desc, NULL, pool, optee);
if (IS_ERR(teedev)) {
rc = PTR_ERR(teedev);
- goto err;
+ goto err_free_optee;
}
optee->teedev = teedev;
teedev = tee_device_alloc(&optee_supp_desc, NULL, pool, optee);
if (IS_ERR(teedev)) {
rc = PTR_ERR(teedev);
- goto err;
+ goto err_unreg_teedev;
}
optee->supp_teedev = teedev;
rc = tee_device_register(optee->teedev);
if (rc)
- goto err;
+ goto err_unreg_supp_teedev;
rc = tee_device_register(optee->supp_teedev);
if (rc)
- goto err;
+ goto err_unreg_supp_teedev;
mutex_init(&optee->call_queue.mutex);
INIT_LIST_HEAD(&optee->call_queue.waiters);
- optee_wait_queue_init(&optee->wait_queue);
optee_supp_init(&optee->supp);
optee->smc.memremaped_shm = memremaped_shm;
optee->pool = pool;
+ platform_set_drvdata(pdev, optee);
+ rc = optee_notif_init(optee, max_notif_value);
+ if (rc)
+ goto err_supp_uninit;
+
+ if (sec_caps & OPTEE_SMC_SEC_CAP_ASYNC_NOTIF) {
+ unsigned int irq;
+
+ rc = platform_get_irq(pdev, 0);
+ if (rc < 0) {
+ pr_err("platform_get_irq: ret %d\n", rc);
+ goto err_notif_uninit;
+ }
+ irq = rc;
+
+ rc = optee_smc_notif_init_irq(optee, irq);
+ if (rc) {
+ irq_dispose_mapping(irq);
+ goto err_notif_uninit;
+ }
+ enable_async_notif(optee->smc.invoke_fn);
+ pr_info("Asynchronous notifications enabled\n");
+ }
+
/*
* Ensure that there are no pre-existing shm objects before enabling
* the shm cache so that there's no chance of receiving an invalid
@@ -1309,29 +1483,30 @@ static int optee_probe(struct platform_device *pdev)
if (optee->smc.sec_caps & OPTEE_SMC_SEC_CAP_DYNAMIC_SHM)
pr_info("dynamic shared memory is enabled\n");
- platform_set_drvdata(pdev, optee);
-
rc = optee_enumerate_devices(PTA_CMD_GET_DEVICES);
- if (rc) {
- optee_smc_remove(pdev);
- return rc;
- }
+ if (rc)
+ goto err_disable_shm_cache;
pr_info("initialized driver\n");
return 0;
-err:
- if (optee) {
- /*
- * tee_device_unregister() is safe to call even if the
- * devices hasn't been registered with
- * tee_device_register() yet.
- */
- tee_device_unregister(optee->supp_teedev);
- tee_device_unregister(optee->teedev);
- kfree(optee);
- }
- if (pool)
- tee_shm_pool_free(pool);
+
+err_disable_shm_cache:
+ optee_disable_shm_cache(optee);
+ optee_smc_notif_uninit_irq(optee);
+ optee_unregister_devices();
+err_notif_uninit:
+ optee_notif_uninit(optee);
+err_supp_uninit:
+ optee_supp_uninit(&optee->supp);
+ mutex_destroy(&optee->call_queue.mutex);
+err_unreg_supp_teedev:
+ tee_device_unregister(optee->supp_teedev);
+err_unreg_teedev:
+ tee_device_unregister(optee->teedev);
+err_free_optee:
+ kfree(optee);
+err_free_pool:
+ tee_shm_pool_free(pool);
if (memremaped_shm)
memunmap(memremaped_shm);
return rc;
diff --git a/drivers/tee/tee_core.c b/drivers/tee/tee_core.c
index 2b37bc4..3fc426d 100644
--- a/drivers/tee/tee_core.c
+++ b/drivers/tee/tee_core.c
@@ -43,7 +43,7 @@ static DEFINE_SPINLOCK(driver_lock);
static struct class *tee_class;
static dev_t tee_devt;
-static struct tee_context *teedev_open(struct tee_device *teedev)
+struct tee_context *teedev_open(struct tee_device *teedev)
{
int rc;
struct tee_context *ctx;
@@ -70,6 +70,7 @@ static struct tee_context *teedev_open(struct tee_device *teedev)
return ERR_PTR(rc);
}
+EXPORT_SYMBOL_GPL(teedev_open);
void teedev_ctx_get(struct tee_context *ctx)
{
@@ -96,11 +97,14 @@ void teedev_ctx_put(struct tee_context *ctx)
kref_put(&ctx->refcount, teedev_ctx_release);
}
-static void teedev_close_context(struct tee_context *ctx)
+void teedev_close_context(struct tee_context *ctx)
{
- tee_device_put(ctx->teedev);
+ struct tee_device *teedev = ctx->teedev;
+
teedev_ctx_put(ctx);
+ tee_device_put(teedev);
}
+EXPORT_SYMBOL_GPL(teedev_close_context);
static int tee_open(struct inode *inode, struct file *filp)
{
diff --git a/drivers/tty/serial/8250/8250_dw.c b/drivers/tty/serial/8250/8250_dw.c
index 53f57c3..1769808 100644
--- a/drivers/tty/serial/8250/8250_dw.c
+++ b/drivers/tty/serial/8250/8250_dw.c
@@ -414,6 +414,8 @@ static void dw8250_quirks(struct uart_port *p, struct dw8250_data *data)
if (of_device_is_compatible(np, "marvell,armada-38x-uart"))
p->serial_out = dw8250_serial_out38x;
+ if (of_device_is_compatible(np, "starfive,jh7100-uart"))
+ p->set_termios = dw8250_do_set_termios;
} else if (acpi_dev_present("APMC0D08", NULL, -1)) {
p->iotype = UPIO_MEM32;
@@ -696,6 +698,7 @@ static const struct of_device_id dw8250_of_match[] = {
{ .compatible = "cavium,octeon-3860-uart" },
{ .compatible = "marvell,armada-38x-uart" },
{ .compatible = "renesas,rzn1-uart" },
+ { .compatible = "starfive,jh7100-uart" },
{ /* Sentinel */ }
};
MODULE_DEVICE_TABLE(of, dw8250_of_match);
diff --git a/drivers/usb/chipidea/ci_hdrc_tegra.c b/drivers/usb/chipidea/ci_hdrc_tegra.c
index 6036114..a72a947 100644
--- a/drivers/usb/chipidea/ci_hdrc_tegra.c
+++ b/drivers/usb/chipidea/ci_hdrc_tegra.c
@@ -7,6 +7,7 @@
#include <linux/io.h>
#include <linux/module.h>
#include <linux/of_device.h>
+#include <linux/pm_runtime.h>
#include <linux/reset.h>
#include <linux/usb.h>
@@ -15,6 +16,8 @@
#include <linux/usb/of.h>
#include <linux/usb/phy.h>
+#include <soc/tegra/common.h>
+
#include "../host/ehci.h"
#include "ci.h"
@@ -278,6 +281,8 @@ static int tegra_usb_probe(struct platform_device *pdev)
if (!usb)
return -ENOMEM;
+ platform_set_drvdata(pdev, usb);
+
soc = of_device_get_match_data(&pdev->dev);
if (!soc) {
dev_err(&pdev->dev, "failed to match OF data\n");
@@ -296,11 +301,14 @@ static int tegra_usb_probe(struct platform_device *pdev)
return err;
}
- err = clk_prepare_enable(usb->clk);
- if (err < 0) {
- dev_err(&pdev->dev, "failed to enable clock: %d\n", err);
+ err = devm_tegra_core_dev_init_opp_table_common(&pdev->dev);
+ if (err)
return err;
- }
+
+ pm_runtime_enable(&pdev->dev);
+ err = pm_runtime_resume_and_get(&pdev->dev);
+ if (err)
+ return err;
if (device_property_present(&pdev->dev, "nvidia,needs-double-reset"))
usb->needs_double_reset = true;
@@ -320,8 +328,6 @@ static int tegra_usb_probe(struct platform_device *pdev)
if (err)
goto fail_power_off;
- platform_set_drvdata(pdev, usb);
-
/* setup and register ChipIdea HDRC device */
usb->soc = soc;
usb->data.name = "tegra-usb";
@@ -350,7 +356,9 @@ static int tegra_usb_probe(struct platform_device *pdev)
phy_shutdown:
usb_phy_shutdown(usb->phy);
fail_power_off:
- clk_disable_unprepare(usb->clk);
+ pm_runtime_put_sync_suspend(&pdev->dev);
+ pm_runtime_force_suspend(&pdev->dev);
+
return err;
}
@@ -360,15 +368,46 @@ static int tegra_usb_remove(struct platform_device *pdev)
ci_hdrc_remove_device(usb->dev);
usb_phy_shutdown(usb->phy);
+
+ pm_runtime_put_sync_suspend(&pdev->dev);
+ pm_runtime_force_suspend(&pdev->dev);
+
+ return 0;
+}
+
+static int __maybe_unused tegra_usb_runtime_resume(struct device *dev)
+{
+ struct tegra_usb *usb = dev_get_drvdata(dev);
+ int err;
+
+ err = clk_prepare_enable(usb->clk);
+ if (err < 0) {
+ dev_err(dev, "failed to enable clock: %d\n", err);
+ return err;
+ }
+
+ return 0;
+}
+
+static int __maybe_unused tegra_usb_runtime_suspend(struct device *dev)
+{
+ struct tegra_usb *usb = dev_get_drvdata(dev);
+
clk_disable_unprepare(usb->clk);
return 0;
}
+static const struct dev_pm_ops tegra_usb_pm = {
+ SET_RUNTIME_PM_OPS(tegra_usb_runtime_suspend, tegra_usb_runtime_resume,
+ NULL)
+};
+
static struct platform_driver tegra_usb_driver = {
.driver = {
.name = "tegra-usb",
.of_match_table = tegra_usb_of_match,
+ .pm = &tegra_usb_pm,
},
.probe = tegra_usb_probe,
.remove = tegra_usb_remove,
diff --git a/drivers/usb/gadget/function/f_fs.c b/drivers/usb/gadget/function/f_fs.c
index e20c19a..a7e069b 100644
--- a/drivers/usb/gadget/function/f_fs.c
+++ b/drivers/usb/gadget/function/f_fs.c
@@ -1773,11 +1773,15 @@ static void ffs_data_clear(struct ffs_data *ffs)
BUG_ON(ffs->gadget);
- if (ffs->epfiles)
+ if (ffs->epfiles) {
ffs_epfiles_destroy(ffs->epfiles, ffs->eps_count);
+ ffs->epfiles = NULL;
+ }
- if (ffs->ffs_eventfd)
+ if (ffs->ffs_eventfd) {
eventfd_ctx_put(ffs->ffs_eventfd);
+ ffs->ffs_eventfd = NULL;
+ }
kfree(ffs->raw_descs_data);
kfree(ffs->raw_strings);
@@ -1790,7 +1794,6 @@ static void ffs_data_reset(struct ffs_data *ffs)
ffs_data_clear(ffs);
- ffs->epfiles = NULL;
ffs->raw_descs_data = NULL;
ffs->raw_descs = NULL;
ffs->raw_strings = NULL;
diff --git a/drivers/usb/host/xhci-pci.c b/drivers/usb/host/xhci-pci.c
index 3af0178..5c35197 100644
--- a/drivers/usb/host/xhci-pci.c
+++ b/drivers/usb/host/xhci-pci.c
@@ -123,7 +123,6 @@ static void xhci_pci_quirks(struct device *dev, struct xhci_hcd *xhci)
/* Look for vendor-specific quirks */
if (pdev->vendor == PCI_VENDOR_ID_FRESCO_LOGIC &&
(pdev->device == PCI_DEVICE_ID_FRESCO_LOGIC_PDK ||
- pdev->device == PCI_DEVICE_ID_FRESCO_LOGIC_FL1100 ||
pdev->device == PCI_DEVICE_ID_FRESCO_LOGIC_FL1400)) {
if (pdev->device == PCI_DEVICE_ID_FRESCO_LOGIC_PDK &&
pdev->revision == 0x0) {
@@ -158,6 +157,10 @@ static void xhci_pci_quirks(struct device *dev, struct xhci_hcd *xhci)
pdev->device == PCI_DEVICE_ID_FRESCO_LOGIC_FL1009)
xhci->quirks |= XHCI_BROKEN_STREAMS;
+ if (pdev->vendor == PCI_VENDOR_ID_FRESCO_LOGIC &&
+ pdev->device == PCI_DEVICE_ID_FRESCO_LOGIC_FL1100)
+ xhci->quirks |= XHCI_TRUST_TX_LENGTH;
+
if (pdev->vendor == PCI_VENDOR_ID_NEC)
xhci->quirks |= XHCI_NEC_HOST;
diff --git a/drivers/usb/mtu3/mtu3_gadget.c b/drivers/usb/mtu3/mtu3_gadget.c
index a9a65b4..9977600 100644
--- a/drivers/usb/mtu3/mtu3_gadget.c
+++ b/drivers/usb/mtu3/mtu3_gadget.c
@@ -77,7 +77,7 @@ static int mtu3_ep_enable(struct mtu3_ep *mep)
if (usb_endpoint_xfer_int(desc) ||
usb_endpoint_xfer_isoc(desc)) {
interval = desc->bInterval;
- interval = clamp_val(interval, 1, 16) - 1;
+ interval = clamp_val(interval, 1, 16);
if (usb_endpoint_xfer_isoc(desc) && comp_desc)
mult = comp_desc->bmAttributes;
}
@@ -89,10 +89,17 @@ static int mtu3_ep_enable(struct mtu3_ep *mep)
if (usb_endpoint_xfer_isoc(desc) ||
usb_endpoint_xfer_int(desc)) {
interval = desc->bInterval;
- interval = clamp_val(interval, 1, 16) - 1;
+ interval = clamp_val(interval, 1, 16);
mult = usb_endpoint_maxp_mult(desc) - 1;
}
break;
+ case USB_SPEED_FULL:
+ if (usb_endpoint_xfer_isoc(desc))
+ interval = clamp_val(desc->bInterval, 1, 16);
+ else if (usb_endpoint_xfer_int(desc))
+ interval = clamp_val(desc->bInterval, 1, 255);
+
+ break;
default:
break; /*others are ignored */
}
@@ -235,6 +242,7 @@ struct usb_request *mtu3_alloc_request(struct usb_ep *ep, gfp_t gfp_flags)
mreq->request.dma = DMA_ADDR_INVALID;
mreq->epnum = mep->epnum;
mreq->mep = mep;
+ INIT_LIST_HEAD(&mreq->list);
trace_mtu3_alloc_request(mreq);
return &mreq->request;
diff --git a/drivers/usb/mtu3/mtu3_qmu.c b/drivers/usb/mtu3/mtu3_qmu.c
index 3f414f9..2ea3157 100644
--- a/drivers/usb/mtu3/mtu3_qmu.c
+++ b/drivers/usb/mtu3/mtu3_qmu.c
@@ -273,6 +273,8 @@ static int mtu3_prepare_tx_gpd(struct mtu3_ep *mep, struct mtu3_request *mreq)
gpd->dw3_info |= cpu_to_le32(GPD_EXT_FLAG_ZLP);
}
+ /* prevent reorder, make sure GPD's HWO is set last */
+ mb();
gpd->dw0_info |= cpu_to_le32(GPD_FLAGS_IOC | GPD_FLAGS_HWO);
mreq->gpd = gpd;
@@ -306,6 +308,8 @@ static int mtu3_prepare_rx_gpd(struct mtu3_ep *mep, struct mtu3_request *mreq)
gpd->next_gpd = cpu_to_le32(lower_32_bits(enq_dma));
ext_addr |= GPD_EXT_NGP(mtu, upper_32_bits(enq_dma));
gpd->dw3_info = cpu_to_le32(ext_addr);
+ /* prevent reorder, make sure GPD's HWO is set last */
+ mb();
gpd->dw0_info |= cpu_to_le32(GPD_FLAGS_IOC | GPD_FLAGS_HWO);
mreq->gpd = gpd;
@@ -445,7 +449,8 @@ static void qmu_tx_zlp_error_handler(struct mtu3 *mtu, u8 epnum)
return;
}
mtu3_setbits(mbase, MU3D_EP_TXCR0(mep->epnum), TX_TXPKTRDY);
-
+ /* prevent reorder, make sure GPD's HWO is set last */
+ mb();
/* by pass the current GDP */
gpd_current->dw0_info |= cpu_to_le32(GPD_FLAGS_BPS | GPD_FLAGS_HWO);
diff --git a/drivers/usb/typec/ucsi/ucsi.c b/drivers/usb/typec/ucsi/ucsi.c
index 6aa2838..08561bf 100644
--- a/drivers/usb/typec/ucsi/ucsi.c
+++ b/drivers/usb/typec/ucsi/ucsi.c
@@ -1150,7 +1150,9 @@ static int ucsi_register_port(struct ucsi *ucsi, int index)
ret = 0;
}
- if (UCSI_CONSTAT_PWR_OPMODE(con->status.flags) == UCSI_CONSTAT_PWR_OPMODE_PD) {
+ if (con->partner &&
+ UCSI_CONSTAT_PWR_OPMODE(con->status.flags) ==
+ UCSI_CONSTAT_PWR_OPMODE_PD) {
ucsi_get_src_pdos(con);
ucsi_check_altmodes(con);
}
diff --git a/drivers/video/console/vgacon.c b/drivers/video/console/vgacon.c
index 9a49ea6..576612f 100644
--- a/drivers/video/console/vgacon.c
+++ b/drivers/video/console/vgacon.c
@@ -97,30 +97,9 @@ static int vga_video_font_height;
static int vga_scan_lines __read_mostly;
static unsigned int vga_rolled_over; /* last vc_origin offset before wrap */
-static bool vgacon_text_mode_force;
static bool vga_hardscroll_enabled;
static bool vga_hardscroll_user_enable = true;
-bool vgacon_text_force(void)
-{
- return vgacon_text_mode_force;
-}
-EXPORT_SYMBOL(vgacon_text_force);
-
-static int __init text_mode(char *str)
-{
- vgacon_text_mode_force = true;
-
- pr_warn("You have booted with nomodeset. This means your GPU drivers are DISABLED\n");
- pr_warn("Any video related functionality will be severely degraded, and you may not even be able to suspend the system properly\n");
- pr_warn("Unless you actually understand what nomodeset does, you should reboot without enabling it\n");
-
- return 1;
-}
-
-/* force text mode - used by kernel modesetting */
-__setup("nomodeset", text_mode);
-
static int __init no_scroll(char *str)
{
/*
diff --git a/drivers/video/fbdev/core/fbmem.c b/drivers/video/fbdev/core/fbmem.c
index 826175ad..0fa7ede 100644
--- a/drivers/video/fbdev/core/fbmem.c
+++ b/drivers/video/fbdev/core/fbmem.c
@@ -1763,6 +1763,53 @@ int remove_conflicting_framebuffers(struct apertures_struct *a,
EXPORT_SYMBOL(remove_conflicting_framebuffers);
/**
+ * is_firmware_framebuffer - detect if firmware-configured framebuffer matches
+ * @a: memory range, users of which are to be checked
+ *
+ * This function checks framebuffer devices (initialized by firmware/bootloader)
+ * which use memory range described by @a. If @a matchesm the function returns
+ * true, otherwise false.
+ */
+bool is_firmware_framebuffer(struct apertures_struct *a)
+{
+ bool do_free = false;
+ bool found = false;
+ int i;
+
+ if (!a) {
+ a = alloc_apertures(1);
+ if (!a)
+ return false;
+
+ a->ranges[0].base = 0;
+ a->ranges[0].size = ~0;
+ do_free = true;
+ }
+
+ mutex_lock(®istration_lock);
+ /* check all firmware fbs and kick off if the base addr overlaps */
+ for_each_registered_fb(i) {
+ struct apertures_struct *gen_aper;
+
+ if (!(registered_fb[i]->flags & FBINFO_MISC_FIRMWARE))
+ continue;
+
+ gen_aper = registered_fb[i]->apertures;
+ if (fb_do_apertures_overlap(gen_aper, a)) {
+ found = true;
+ break;
+ }
+ }
+ mutex_unlock(®istration_lock);
+
+ if (do_free)
+ kfree(a);
+
+ return found;
+}
+EXPORT_SYMBOL(is_firmware_framebuffer);
+
+/**
* remove_conflicting_pci_framebuffers - remove firmware-configured framebuffers for PCI devices
* @pdev: PCI device
* @name: requesting driver name
diff --git a/drivers/video/fbdev/core/fbsysfs.c b/drivers/video/fbdev/core/fbsysfs.c
index 65dae05..2689294 100644
--- a/drivers/video/fbdev/core/fbsysfs.c
+++ b/drivers/video/fbdev/core/fbsysfs.c
@@ -230,7 +230,7 @@ static ssize_t show_bpp(struct device *device, struct device_attribute *attr,
char *buf)
{
struct fb_info *fb_info = dev_get_drvdata(device);
- return snprintf(buf, PAGE_SIZE, "%d\n", fb_info->var.bits_per_pixel);
+ return sysfs_emit(buf, "%d\n", fb_info->var.bits_per_pixel);
}
static ssize_t store_rotate(struct device *device,
@@ -257,7 +257,7 @@ static ssize_t show_rotate(struct device *device,
{
struct fb_info *fb_info = dev_get_drvdata(device);
- return snprintf(buf, PAGE_SIZE, "%d\n", fb_info->var.rotate);
+ return sysfs_emit(buf, "%d\n", fb_info->var.rotate);
}
static ssize_t store_virtual(struct device *device,
@@ -285,7 +285,7 @@ static ssize_t show_virtual(struct device *device,
struct device_attribute *attr, char *buf)
{
struct fb_info *fb_info = dev_get_drvdata(device);
- return snprintf(buf, PAGE_SIZE, "%d,%d\n", fb_info->var.xres_virtual,
+ return sysfs_emit(buf, "%d,%d\n", fb_info->var.xres_virtual,
fb_info->var.yres_virtual);
}
@@ -293,7 +293,7 @@ static ssize_t show_stride(struct device *device,
struct device_attribute *attr, char *buf)
{
struct fb_info *fb_info = dev_get_drvdata(device);
- return snprintf(buf, PAGE_SIZE, "%d\n", fb_info->fix.line_length);
+ return sysfs_emit(buf, "%d\n", fb_info->fix.line_length);
}
static ssize_t store_blank(struct device *device,
@@ -381,7 +381,7 @@ static ssize_t show_pan(struct device *device,
struct device_attribute *attr, char *buf)
{
struct fb_info *fb_info = dev_get_drvdata(device);
- return snprintf(buf, PAGE_SIZE, "%d,%d\n", fb_info->var.xoffset,
+ return sysfs_emit(buf, "%d,%d\n", fb_info->var.xoffset,
fb_info->var.yoffset);
}
@@ -390,7 +390,7 @@ static ssize_t show_name(struct device *device,
{
struct fb_info *fb_info = dev_get_drvdata(device);
- return snprintf(buf, PAGE_SIZE, "%s\n", fb_info->fix.id);
+ return sysfs_emit(buf, "%s\n", fb_info->fix.id);
}
static ssize_t store_fbstate(struct device *device,
@@ -418,7 +418,7 @@ static ssize_t show_fbstate(struct device *device,
struct device_attribute *attr, char *buf)
{
struct fb_info *fb_info = dev_get_drvdata(device);
- return snprintf(buf, PAGE_SIZE, "%d\n", fb_info->state);
+ return sysfs_emit(buf, "%d\n", fb_info->state);
}
#if IS_ENABLED(CONFIG_FB_BACKLIGHT)
diff --git a/drivers/video/fbdev/omap/omapfb_main.c b/drivers/video/fbdev/omap/omapfb_main.c
index 3d090d2..b495c09 100644
--- a/drivers/video/fbdev/omap/omapfb_main.c
+++ b/drivers/video/fbdev/omap/omapfb_main.c
@@ -1555,6 +1555,7 @@ static void omapfb_free_resources(struct omapfb_device *fbdev, int state)
case 1:
dev_set_drvdata(fbdev->dev, NULL);
kfree(fbdev);
+ break;
case 0:
/* nothing to free */
break;
diff --git a/drivers/video/fbdev/omap2/omapfb/dss/display-sysfs.c b/drivers/video/fbdev/omap2/omapfb/dss/display-sysfs.c
index 6dbe265..8f355d1 100644
--- a/drivers/video/fbdev/omap2/omapfb/dss/display-sysfs.c
+++ b/drivers/video/fbdev/omap2/omapfb/dss/display-sysfs.c
@@ -19,14 +19,14 @@
static ssize_t display_name_show(struct omap_dss_device *dssdev, char *buf)
{
- return snprintf(buf, PAGE_SIZE, "%s\n",
+ return sysfs_emit(buf, "%s\n",
dssdev->name ?
dssdev->name : "");
}
static ssize_t display_enabled_show(struct omap_dss_device *dssdev, char *buf)
{
- return snprintf(buf, PAGE_SIZE, "%d\n",
+ return sysfs_emit(buf, "%d\n",
omapdss_device_is_enabled(dssdev));
}
@@ -59,7 +59,7 @@ static ssize_t display_enabled_store(struct omap_dss_device *dssdev,
static ssize_t display_tear_show(struct omap_dss_device *dssdev, char *buf)
{
- return snprintf(buf, PAGE_SIZE, "%d\n",
+ return sysfs_emit(buf, "%d\n",
dssdev->driver->get_te ?
dssdev->driver->get_te(dssdev) : 0);
}
@@ -93,7 +93,7 @@ static ssize_t display_timings_show(struct omap_dss_device *dssdev, char *buf)
dssdev->driver->get_timings(dssdev, &t);
- return snprintf(buf, PAGE_SIZE, "%u,%u/%u/%u/%u,%u/%u/%u/%u\n",
+ return sysfs_emit(buf, "%u,%u/%u/%u/%u,%u/%u/%u/%u\n",
t.pixelclock,
t.x_res, t.hfp, t.hbp, t.hsw,
t.y_res, t.vfp, t.vbp, t.vsw);
@@ -143,7 +143,7 @@ static ssize_t display_rotate_show(struct omap_dss_device *dssdev, char *buf)
if (!dssdev->driver->get_rotate)
return -ENOENT;
rotate = dssdev->driver->get_rotate(dssdev);
- return snprintf(buf, PAGE_SIZE, "%u\n", rotate);
+ return sysfs_emit(buf, "%u\n", rotate);
}
static ssize_t display_rotate_store(struct omap_dss_device *dssdev,
@@ -171,7 +171,7 @@ static ssize_t display_mirror_show(struct omap_dss_device *dssdev, char *buf)
if (!dssdev->driver->get_mirror)
return -ENOENT;
mirror = dssdev->driver->get_mirror(dssdev);
- return snprintf(buf, PAGE_SIZE, "%u\n", mirror);
+ return sysfs_emit(buf, "%u\n", mirror);
}
static ssize_t display_mirror_store(struct omap_dss_device *dssdev,
@@ -203,7 +203,7 @@ static ssize_t display_wss_show(struct omap_dss_device *dssdev, char *buf)
wss = dssdev->driver->get_wss(dssdev);
- return snprintf(buf, PAGE_SIZE, "0x%05x\n", wss);
+ return sysfs_emit(buf, "0x%05x\n", wss);
}
static ssize_t display_wss_store(struct omap_dss_device *dssdev,
diff --git a/drivers/video/fbdev/omap2/omapfb/dss/manager-sysfs.c b/drivers/video/fbdev/omap2/omapfb/dss/manager-sysfs.c
index b52cc1a..3ffb1fe 100644
--- a/drivers/video/fbdev/omap2/omapfb/dss/manager-sysfs.c
+++ b/drivers/video/fbdev/omap2/omapfb/dss/manager-sysfs.c
@@ -22,14 +22,14 @@
static ssize_t manager_name_show(struct omap_overlay_manager *mgr, char *buf)
{
- return snprintf(buf, PAGE_SIZE, "%s\n", mgr->name);
+ return sysfs_emit(buf, "%s\n", mgr->name);
}
static ssize_t manager_display_show(struct omap_overlay_manager *mgr, char *buf)
{
struct omap_dss_device *dssdev = mgr->get_device(mgr);
- return snprintf(buf, PAGE_SIZE, "%s\n", dssdev ?
+ return sysfs_emit(buf, "%s\n", dssdev ?
dssdev->name : "<none>");
}
@@ -120,7 +120,7 @@ static ssize_t manager_default_color_show(struct omap_overlay_manager *mgr,
mgr->get_manager_info(mgr, &info);
- return snprintf(buf, PAGE_SIZE, "%#x\n", info.default_color);
+ return sysfs_emit(buf, "%#x\n", info.default_color);
}
static ssize_t manager_default_color_store(struct omap_overlay_manager *mgr,
@@ -165,7 +165,7 @@ static ssize_t manager_trans_key_type_show(struct omap_overlay_manager *mgr,
key_type = info.trans_key_type;
BUG_ON(key_type >= ARRAY_SIZE(trans_key_type_str));
- return snprintf(buf, PAGE_SIZE, "%s\n", trans_key_type_str[key_type]);
+ return sysfs_emit(buf, "%s\n", trans_key_type_str[key_type]);
}
static ssize_t manager_trans_key_type_store(struct omap_overlay_manager *mgr,
@@ -200,7 +200,7 @@ static ssize_t manager_trans_key_value_show(struct omap_overlay_manager *mgr,
mgr->get_manager_info(mgr, &info);
- return snprintf(buf, PAGE_SIZE, "%#x\n", info.trans_key);
+ return sysfs_emit(buf, "%#x\n", info.trans_key);
}
static ssize_t manager_trans_key_value_store(struct omap_overlay_manager *mgr,
@@ -236,7 +236,7 @@ static ssize_t manager_trans_key_enabled_show(struct omap_overlay_manager *mgr,
mgr->get_manager_info(mgr, &info);
- return snprintf(buf, PAGE_SIZE, "%d\n", info.trans_enabled);
+ return sysfs_emit(buf, "%d\n", info.trans_enabled);
}
static ssize_t manager_trans_key_enabled_store(struct omap_overlay_manager *mgr,
@@ -275,7 +275,7 @@ static ssize_t manager_alpha_blending_enabled_show(
mgr->get_manager_info(mgr, &info);
- return snprintf(buf, PAGE_SIZE, "%d\n",
+ return sysfs_emit(buf, "%d\n",
info.partial_alpha_enabled);
}
@@ -316,7 +316,7 @@ static ssize_t manager_cpr_enable_show(struct omap_overlay_manager *mgr,
mgr->get_manager_info(mgr, &info);
- return snprintf(buf, PAGE_SIZE, "%d\n", info.cpr_enable);
+ return sysfs_emit(buf, "%d\n", info.cpr_enable);
}
static ssize_t manager_cpr_enable_store(struct omap_overlay_manager *mgr,
@@ -358,7 +358,7 @@ static ssize_t manager_cpr_coef_show(struct omap_overlay_manager *mgr,
mgr->get_manager_info(mgr, &info);
- return snprintf(buf, PAGE_SIZE,
+ return sysfs_emit(buf,
"%d %d %d %d %d %d %d %d %d\n",
info.cpr_coefs.rr,
info.cpr_coefs.rg,
diff --git a/drivers/video/fbdev/omap2/omapfb/dss/overlay-sysfs.c b/drivers/video/fbdev/omap2/omapfb/dss/overlay-sysfs.c
index 36acf36..421dcb7 100644
--- a/drivers/video/fbdev/omap2/omapfb/dss/overlay-sysfs.c
+++ b/drivers/video/fbdev/omap2/omapfb/dss/overlay-sysfs.c
@@ -22,12 +22,12 @@
static ssize_t overlay_name_show(struct omap_overlay *ovl, char *buf)
{
- return snprintf(buf, PAGE_SIZE, "%s\n", ovl->name);
+ return sysfs_emit(buf, "%s\n", ovl->name);
}
static ssize_t overlay_manager_show(struct omap_overlay *ovl, char *buf)
{
- return snprintf(buf, PAGE_SIZE, "%s\n",
+ return sysfs_emit(buf, "%s\n",
ovl->manager ? ovl->manager->name : "<none>");
}
@@ -108,7 +108,7 @@ static ssize_t overlay_input_size_show(struct omap_overlay *ovl, char *buf)
ovl->get_overlay_info(ovl, &info);
- return snprintf(buf, PAGE_SIZE, "%d,%d\n",
+ return sysfs_emit(buf, "%d,%d\n",
info.width, info.height);
}
@@ -118,7 +118,7 @@ static ssize_t overlay_screen_width_show(struct omap_overlay *ovl, char *buf)
ovl->get_overlay_info(ovl, &info);
- return snprintf(buf, PAGE_SIZE, "%d\n", info.screen_width);
+ return sysfs_emit(buf, "%d\n", info.screen_width);
}
static ssize_t overlay_position_show(struct omap_overlay *ovl, char *buf)
@@ -127,7 +127,7 @@ static ssize_t overlay_position_show(struct omap_overlay *ovl, char *buf)
ovl->get_overlay_info(ovl, &info);
- return snprintf(buf, PAGE_SIZE, "%d,%d\n",
+ return sysfs_emit(buf, "%d,%d\n",
info.pos_x, info.pos_y);
}
@@ -166,7 +166,7 @@ static ssize_t overlay_output_size_show(struct omap_overlay *ovl, char *buf)
ovl->get_overlay_info(ovl, &info);
- return snprintf(buf, PAGE_SIZE, "%d,%d\n",
+ return sysfs_emit(buf, "%d,%d\n",
info.out_width, info.out_height);
}
@@ -201,7 +201,7 @@ static ssize_t overlay_output_size_store(struct omap_overlay *ovl,
static ssize_t overlay_enabled_show(struct omap_overlay *ovl, char *buf)
{
- return snprintf(buf, PAGE_SIZE, "%d\n", ovl->is_enabled(ovl));
+ return sysfs_emit(buf, "%d\n", ovl->is_enabled(ovl));
}
static ssize_t overlay_enabled_store(struct omap_overlay *ovl, const char *buf,
@@ -231,7 +231,7 @@ static ssize_t overlay_global_alpha_show(struct omap_overlay *ovl, char *buf)
ovl->get_overlay_info(ovl, &info);
- return snprintf(buf, PAGE_SIZE, "%d\n",
+ return sysfs_emit(buf, "%d\n",
info.global_alpha);
}
@@ -273,7 +273,7 @@ static ssize_t overlay_pre_mult_alpha_show(struct omap_overlay *ovl,
ovl->get_overlay_info(ovl, &info);
- return snprintf(buf, PAGE_SIZE, "%d\n",
+ return sysfs_emit(buf, "%d\n",
info.pre_mult_alpha);
}
@@ -314,7 +314,7 @@ static ssize_t overlay_zorder_show(struct omap_overlay *ovl, char *buf)
ovl->get_overlay_info(ovl, &info);
- return snprintf(buf, PAGE_SIZE, "%d\n", info.zorder);
+ return sysfs_emit(buf, "%d\n", info.zorder);
}
static ssize_t overlay_zorder_store(struct omap_overlay *ovl,
diff --git a/drivers/video/fbdev/omap2/omapfb/omapfb-sysfs.c b/drivers/video/fbdev/omap2/omapfb/omapfb-sysfs.c
index 2d39dbf..06dc41a 100644
--- a/drivers/video/fbdev/omap2/omapfb/omapfb-sysfs.c
+++ b/drivers/video/fbdev/omap2/omapfb/omapfb-sysfs.c
@@ -29,7 +29,7 @@ static ssize_t show_rotate_type(struct device *dev,
struct fb_info *fbi = dev_get_drvdata(dev);
struct omapfb_info *ofbi = FB2OFB(fbi);
- return snprintf(buf, PAGE_SIZE, "%d\n", ofbi->rotation_type);
+ return sysfs_emit(buf, "%d\n", ofbi->rotation_type);
}
static ssize_t store_rotate_type(struct device *dev,
@@ -83,7 +83,7 @@ static ssize_t show_mirror(struct device *dev,
struct fb_info *fbi = dev_get_drvdata(dev);
struct omapfb_info *ofbi = FB2OFB(fbi);
- return snprintf(buf, PAGE_SIZE, "%d\n", ofbi->mirror);
+ return sysfs_emit(buf, "%d\n", ofbi->mirror);
}
static ssize_t store_mirror(struct device *dev,
@@ -415,7 +415,7 @@ static ssize_t show_size(struct device *dev,
struct fb_info *fbi = dev_get_drvdata(dev);
struct omapfb_info *ofbi = FB2OFB(fbi);
- return snprintf(buf, PAGE_SIZE, "%lu\n", ofbi->region->size);
+ return sysfs_emit(buf, "%lu\n", ofbi->region->size);
}
static ssize_t store_size(struct device *dev, struct device_attribute *attr,
@@ -492,7 +492,7 @@ static ssize_t show_phys(struct device *dev,
struct fb_info *fbi = dev_get_drvdata(dev);
struct omapfb_info *ofbi = FB2OFB(fbi);
- return snprintf(buf, PAGE_SIZE, "%0x\n", ofbi->region->paddr);
+ return sysfs_emit(buf, "%0x\n", ofbi->region->paddr);
}
static ssize_t show_virt(struct device *dev,
@@ -501,7 +501,7 @@ static ssize_t show_virt(struct device *dev,
struct fb_info *fbi = dev_get_drvdata(dev);
struct omapfb_info *ofbi = FB2OFB(fbi);
- return snprintf(buf, PAGE_SIZE, "%p\n", ofbi->region->vaddr);
+ return sysfs_emit(buf, "%p\n", ofbi->region->vaddr);
}
static ssize_t show_upd_mode(struct device *dev,
@@ -516,7 +516,7 @@ static ssize_t show_upd_mode(struct device *dev,
if (r)
return r;
- return snprintf(buf, PAGE_SIZE, "%u\n", (unsigned)mode);
+ return sysfs_emit(buf, "%u\n", (unsigned int)mode);
}
static ssize_t store_upd_mode(struct device *dev, struct device_attribute *attr,
diff --git a/drivers/video/fbdev/simplefb.c b/drivers/video/fbdev/simplefb.c
index b63074f..5754188 100644
--- a/drivers/video/fbdev/simplefb.c
+++ b/drivers/video/fbdev/simplefb.c
@@ -541,26 +541,7 @@ static struct platform_driver simplefb_driver = {
.remove = simplefb_remove,
};
-static int __init simplefb_init(void)
-{
- int ret;
- struct device_node *np;
-
- ret = platform_driver_register(&simplefb_driver);
- if (ret)
- return ret;
-
- if (IS_ENABLED(CONFIG_OF_ADDRESS) && of_chosen) {
- for_each_child_of_node(of_chosen, np) {
- if (of_device_is_compatible(np, "simple-framebuffer"))
- of_platform_device_create(np, NULL, NULL);
- }
- }
-
- return 0;
-}
-
-fs_initcall(simplefb_init);
+module_platform_driver(simplefb_driver);
MODULE_AUTHOR("Stephen Warren <swarren@wwwdotorg.org>");
MODULE_DESCRIPTION("Simple framebuffer driver");
diff --git a/drivers/virt/nitro_enclaves/ne_misc_dev.c b/drivers/virt/nitro_enclaves/ne_misc_dev.c
index 8939612..6894ccb 100644
--- a/drivers/virt/nitro_enclaves/ne_misc_dev.c
+++ b/drivers/virt/nitro_enclaves/ne_misc_dev.c
@@ -886,8 +886,9 @@ static int ne_set_user_memory_region_ioctl(struct ne_enclave *ne_enclave,
goto put_pages;
}
- gup_rc = get_user_pages(mem_region.userspace_addr + memory_size, 1, FOLL_GET,
- ne_mem_region->pages + i, NULL);
+ gup_rc = get_user_pages_unlocked(mem_region.userspace_addr + memory_size, 1,
+ ne_mem_region->pages + i, FOLL_GET);
+
if (gup_rc < 0) {
rc = gup_rc;
diff --git a/fs/namespace.c b/fs/namespace.c
index 659a8f3..b696543 100644
--- a/fs/namespace.c
+++ b/fs/namespace.c
@@ -4263,12 +4263,11 @@ SYSCALL_DEFINE5(mount_setattr, int, dfd, const char __user *, path,
return err;
err = user_path_at(dfd, path, kattr.lookup_flags, &target);
- if (err)
- return err;
-
- err = do_mount_setattr(&target, &kattr);
+ if (!err) {
+ err = do_mount_setattr(&target, &kattr);
+ path_put(&target);
+ }
finish_mount_kattr(&kattr);
- path_put(&target);
return err;
}
diff --git a/fs/nfs/dir.c b/fs/nfs/dir.c
index 731d310..3477936 100644
--- a/fs/nfs/dir.c
+++ b/fs/nfs/dir.c
@@ -18,6 +18,7 @@
* 6 Jun 1999 Cache readdir lookups in the page cache. -DaveM
*/
+#include <linux/compat.h>
#include <linux/module.h>
#include <linux/time.h>
#include <linux/errno.h>
diff --git a/fs/nfs/fs_context.c b/fs/nfs/fs_context.c
index 0d444a9..ea17fa1 100644
--- a/fs/nfs/fs_context.c
+++ b/fs/nfs/fs_context.c
@@ -10,6 +10,7 @@
* Split from fs/nfs/super.c by David Howells <dhowells@redhat.com>
*/
+#include <linux/compat.h>
#include <linux/module.h>
#include <linux/fs.h>
#include <linux/fs_context.h>
diff --git a/fs/proc/proc_net.c b/fs/proc/proc_net.c
index 15c2e55..39b823a 100644
--- a/fs/proc/proc_net.c
+++ b/fs/proc/proc_net.c
@@ -61,15 +61,27 @@ static int seq_open_net(struct inode *inode, struct file *file)
}
#ifdef CONFIG_NET_NS
p->net = net;
+ netns_tracker_alloc(net, &p->ns_tracker, GFP_KERNEL);
#endif
return 0;
}
+static void seq_file_net_put_net(struct seq_file *seq)
+{
+#ifdef CONFIG_NET_NS
+ struct seq_net_private *priv = seq->private;
+
+ put_net_track(priv->net, &priv->ns_tracker);
+#else
+ put_net(&init_net);
+#endif
+}
+
static int seq_release_net(struct inode *ino, struct file *f)
{
struct seq_file *seq = f->private_data;
- put_net(seq_file_net(seq));
+ seq_file_net_put_net(seq);
seq_release_private(ino, f);
return 0;
}
@@ -87,7 +99,8 @@ int bpf_iter_init_seq_net(void *priv_data, struct bpf_iter_aux_info *aux)
#ifdef CONFIG_NET_NS
struct seq_net_private *p = priv_data;
- p->net = get_net(current->nsproxy->net_ns);
+ p->net = get_net_track(current->nsproxy->net_ns, &p->ns_tracker,
+ GFP_KERNEL);
#endif
return 0;
}
@@ -97,7 +110,7 @@ void bpf_iter_fini_seq_net(void *priv_data)
#ifdef CONFIG_NET_NS
struct seq_net_private *p = priv_data;
- put_net(p->net);
+ put_net_track(p->net, &p->ns_tracker);
#endif
}
diff --git a/fs/pstore/ftrace.c b/fs/pstore/ftrace.c
index 5939595..776cae2 100644
--- a/fs/pstore/ftrace.c
+++ b/fs/pstore/ftrace.c
@@ -64,20 +64,12 @@ static struct ftrace_ops pstore_ftrace_ops __read_mostly = {
static DEFINE_MUTEX(pstore_ftrace_lock);
static bool pstore_ftrace_enabled;
-static ssize_t pstore_ftrace_knob_write(struct file *f, const char __user *buf,
- size_t count, loff_t *ppos)
+static int pstore_set_ftrace_enabled(bool on)
{
- u8 on;
ssize_t ret;
- ret = kstrtou8_from_user(buf, count, 2, &on);
- if (ret)
- return ret;
-
- mutex_lock(&pstore_ftrace_lock);
-
- if (!on ^ pstore_ftrace_enabled)
- goto out;
+ if (on == pstore_ftrace_enabled)
+ return 0;
if (on) {
ftrace_ops_set_global_filter(&pstore_ftrace_ops);
@@ -89,15 +81,30 @@ static ssize_t pstore_ftrace_knob_write(struct file *f, const char __user *buf,
if (ret) {
pr_err("%s: unable to %sregister ftrace ops: %zd\n",
__func__, on ? "" : "un", ret);
- goto err;
+ } else {
+ pstore_ftrace_enabled = on;
}
- pstore_ftrace_enabled = on;
-out:
- ret = count;
-err:
+ return ret;
+}
+
+static ssize_t pstore_ftrace_knob_write(struct file *f, const char __user *buf,
+ size_t count, loff_t *ppos)
+{
+ u8 on;
+ ssize_t ret;
+
+ ret = kstrtou8_from_user(buf, count, 2, &on);
+ if (ret)
+ return ret;
+
+ mutex_lock(&pstore_ftrace_lock);
+ ret = pstore_set_ftrace_enabled(on);
mutex_unlock(&pstore_ftrace_lock);
+ if (ret == 0)
+ ret = count;
+
return ret;
}
@@ -117,6 +124,11 @@ static const struct file_operations pstore_knob_fops = {
static struct dentry *pstore_ftrace_dir;
+static bool record_ftrace;
+module_param(record_ftrace, bool, 0400);
+MODULE_PARM_DESC(record_ftrace,
+ "enable ftrace recording immediately (default: off)");
+
void pstore_register_ftrace(void)
{
if (!psinfo->write)
@@ -124,6 +136,8 @@ void pstore_register_ftrace(void)
pstore_ftrace_dir = debugfs_create_dir("pstore", NULL);
+ pstore_set_ftrace_enabled(record_ftrace);
+
debugfs_create_file("record_ftrace", 0600, pstore_ftrace_dir, NULL,
&pstore_knob_fops);
}
diff --git a/fs/select.c b/fs/select.c
index 945896d..02cd8cb 100644
--- a/fs/select.c
+++ b/fs/select.c
@@ -15,6 +15,7 @@
* of fds to overcome nfds < 16390 descriptors limit (Tigran Aivazian).
*/
+#include <linux/compat.h>
#include <linux/kernel.h>
#include <linux/sched/signal.h>
#include <linux/sched/rt.h>
diff --git a/fs/xfs/xfs_ioctl.c b/fs/xfs/xfs_ioctl.c
index 174cd89..bc85e04 100644
--- a/fs/xfs/xfs_ioctl.c
+++ b/fs/xfs/xfs_ioctl.c
@@ -687,7 +687,8 @@ xfs_ioc_space(
if (bf->l_start > XFS_ISIZE(ip)) {
error = xfs_alloc_file_space(ip, XFS_ISIZE(ip),
- bf->l_start - XFS_ISIZE(ip), 0);
+ bf->l_start - XFS_ISIZE(ip),
+ XFS_BMAPI_PREALLOC);
if (error)
goto out_unlock;
}
diff --git a/include/acpi/acpi_bus.h b/include/acpi/acpi_bus.h
index 04b9ace..44a3082 100644
--- a/include/acpi/acpi_bus.h
+++ b/include/acpi/acpi_bus.h
@@ -202,7 +202,8 @@ struct acpi_device_flags {
u32 coherent_dma:1;
u32 cca_seen:1;
u32 enumeration_by_parent:1;
- u32 reserved:19;
+ u32 honor_deps:1;
+ u32 reserved:18;
};
/* File System */
@@ -285,6 +286,7 @@ struct acpi_dep_data {
struct list_head node;
acpi_handle supplier;
acpi_handle consumer;
+ bool honor_dep;
};
/* Performance Management */
@@ -716,6 +718,7 @@ static inline bool acpi_device_can_poweroff(struct acpi_device *adev)
bool acpi_dev_hid_uid_match(struct acpi_device *adev, const char *hid2, const char *uid2);
void acpi_dev_clear_dependencies(struct acpi_device *supplier);
+bool acpi_dev_ready_for_enumeration(const struct acpi_device *device);
struct acpi_device *acpi_dev_get_first_consumer_dev(struct acpi_device *supplier);
struct acpi_device *
acpi_dev_get_next_match_dev(struct acpi_device *adev, const char *hid, const char *uid, s64 hrv);
diff --git a/include/asm-generic/barrier.h b/include/asm-generic/barrier.h
index 640f094..4c2c1b8 100644
--- a/include/asm-generic/barrier.h
+++ b/include/asm-generic/barrier.h
@@ -251,5 +251,16 @@ do { \
#define pmem_wmb() wmb()
#endif
+/*
+ * ioremap_wc() maps I/O memory as memory with write-combining attributes. For
+ * this kind of memory accesses, the CPU may wait for prior accesses to be
+ * merged with subsequent ones. In some situation, such wait is bad for the
+ * performance. io_stop_wc() can be used to prevent the merging of
+ * write-combining memory accesses before this macro with those after it.
+ */
+#ifndef io_stop_wc
+#define io_stop_wc do { } while (0)
+#endif
+
#endif /* !__ASSEMBLY__ */
#endif /* __ASM_GENERIC_BARRIER_H */
diff --git a/include/asm-generic/error-injection.h b/include/asm-generic/error-injection.h
index 7ddd9dc..fbca56b 100644
--- a/include/asm-generic/error-injection.h
+++ b/include/asm-generic/error-injection.h
@@ -20,7 +20,7 @@ struct pt_regs;
#ifdef CONFIG_FUNCTION_ERROR_INJECTION
/*
- * Whitelist ganerating macro. Specify functions which can be
+ * Whitelist generating macro. Specify functions which can be
* error-injectable using this macro.
*/
#define ALLOW_ERROR_INJECTION(fname, _etype) \
@@ -29,7 +29,7 @@ static struct error_injection_entry __used \
_eil_addr_##fname = { \
.addr = (unsigned long)fname, \
.etype = EI_ETYPE_##_etype, \
- };
+ }
void override_function_with_return(struct pt_regs *regs);
#else
diff --git a/include/asm-generic/sections.h b/include/asm-generic/sections.h
index 1dfadb2..76a0f16 100644
--- a/include/asm-generic/sections.h
+++ b/include/asm-generic/sections.h
@@ -130,18 +130,24 @@ static inline bool init_section_intersects(void *virt, size_t size)
/**
* is_kernel_core_data - checks if the pointer address is located in the
- * .data section
+ * .data or .bss section
*
* @addr: address to check
*
- * Returns: true if the address is located in .data, false otherwise.
+ * Returns: true if the address is located in .data or .bss, false otherwise.
* Note: On some archs it may return true for core RODATA, and false
* for others. But will always be true for core RW data.
*/
static inline bool is_kernel_core_data(unsigned long addr)
{
- return addr >= (unsigned long)_sdata &&
- addr < (unsigned long)_edata;
+ if (addr >= (unsigned long)_sdata && addr < (unsigned long)_edata)
+ return true;
+
+ if (addr >= (unsigned long)__bss_start &&
+ addr < (unsigned long)__bss_stop)
+ return true;
+
+ return false;
}
/**
diff --git a/include/crypto/chacha.h b/include/crypto/chacha.h
index dabaee6..b3ea73b8 100644
--- a/include/crypto/chacha.h
+++ b/include/crypto/chacha.h
@@ -47,12 +47,19 @@ static inline void hchacha_block(const u32 *state, u32 *out, int nrounds)
hchacha_block_generic(state, out, nrounds);
}
+enum chacha_constants { /* expand 32-byte k */
+ CHACHA_CONSTANT_EXPA = 0x61707865U,
+ CHACHA_CONSTANT_ND_3 = 0x3320646eU,
+ CHACHA_CONSTANT_2_BY = 0x79622d32U,
+ CHACHA_CONSTANT_TE_K = 0x6b206574U
+};
+
static inline void chacha_init_consts(u32 *state)
{
- state[0] = 0x61707865; /* "expa" */
- state[1] = 0x3320646e; /* "nd 3" */
- state[2] = 0x79622d32; /* "2-by" */
- state[3] = 0x6b206574; /* "te k" */
+ state[0] = CHACHA_CONSTANT_EXPA;
+ state[1] = CHACHA_CONSTANT_ND_3;
+ state[2] = CHACHA_CONSTANT_2_BY;
+ state[3] = CHACHA_CONSTANT_TE_K;
}
void chacha_init_arch(u32 *state, const u32 *key, const u8 *iv);
diff --git a/include/crypto/internal/blake2s.h b/include/crypto/internal/blake2s.h
index 8e50d48..d39cfa0 100644
--- a/include/crypto/internal/blake2s.h
+++ b/include/crypto/internal/blake2s.h
@@ -11,11 +11,11 @@
#include <crypto/internal/hash.h>
#include <linux/string.h>
-void blake2s_compress_generic(struct blake2s_state *state,const u8 *block,
+void blake2s_compress_generic(struct blake2s_state *state, const u8 *block,
size_t nblocks, const u32 inc);
-void blake2s_compress_arch(struct blake2s_state *state,const u8 *block,
- size_t nblocks, const u32 inc);
+void blake2s_compress(struct blake2s_state *state, const u8 *block,
+ size_t nblocks, const u32 inc);
bool blake2s_selftest(void);
diff --git a/include/drm/drm_connector.h b/include/drm/drm_connector.h
index 379746d32..b501d0b 100644
--- a/include/drm/drm_connector.h
+++ b/include/drm/drm_connector.h
@@ -27,6 +27,7 @@
#include <linux/llist.h>
#include <linux/ctype.h>
#include <linux/hdmi.h>
+#include <linux/notifier.h>
#include <drm/drm_mode_object.h>
#include <drm/drm_util.h>
@@ -40,6 +41,7 @@ struct drm_encoder;
struct drm_property;
struct drm_property_blob;
struct drm_printer;
+struct drm_privacy_screen;
struct edid;
struct i2c_adapter;
@@ -320,6 +322,30 @@ struct drm_monitor_range_info {
u8 max_vfreq;
};
+/**
+ * enum drm_privacy_screen_status - privacy screen status
+ *
+ * This enum is used to track and control the state of the integrated privacy
+ * screen present on some display panels, via the "privacy-screen sw-state"
+ * and "privacy-screen hw-state" properties. Note the _LOCKED enum values
+ * are only valid for the "privacy-screen hw-state" property.
+ *
+ * @PRIVACY_SCREEN_DISABLED:
+ * The privacy-screen on the panel is disabled
+ * @PRIVACY_SCREEN_ENABLED:
+ * The privacy-screen on the panel is enabled
+ * @PRIVACY_SCREEN_DISABLED_LOCKED:
+ * The privacy-screen on the panel is disabled and locked (cannot be changed)
+ * @PRIVACY_SCREEN_ENABLED_LOCKED:
+ * The privacy-screen on the panel is enabled and locked (cannot be changed)
+ */
+enum drm_privacy_screen_status {
+ PRIVACY_SCREEN_DISABLED = 0,
+ PRIVACY_SCREEN_ENABLED,
+ PRIVACY_SCREEN_DISABLED_LOCKED,
+ PRIVACY_SCREEN_ENABLED_LOCKED,
+};
+
/*
* This is a consolidated colorimetry list supported by HDMI and
* DP protocol standard. The respective connectors will register
@@ -794,6 +820,12 @@ struct drm_connector_state {
u8 max_bpc;
/**
+ * @privacy_screen_sw_state: See :ref:`Standard Connector
+ * Properties<standard_connector_properties>`
+ */
+ enum drm_privacy_screen_status privacy_screen_sw_state;
+
+ /**
* @hdr_output_metadata:
* DRM blob property for HDR output metadata
*/
@@ -1421,6 +1453,24 @@ struct drm_connector {
*/
struct drm_property *max_bpc_property;
+ /** @privacy_screen: drm_privacy_screen for this connector, or NULL. */
+ struct drm_privacy_screen *privacy_screen;
+
+ /** @privacy_screen_notifier: privacy-screen notifier_block */
+ struct notifier_block privacy_screen_notifier;
+
+ /**
+ * @privacy_screen_sw_state_property: Optional atomic property for the
+ * connector to control the integrated privacy screen.
+ */
+ struct drm_property *privacy_screen_sw_state_property;
+
+ /**
+ * @privacy_screen_hw_state_property: Optional atomic property for the
+ * connector to report the actual integrated privacy screen state.
+ */
+ struct drm_property *privacy_screen_hw_state_property;
+
#define DRM_CONNECTOR_POLL_HPD (1 << 0)
#define DRM_CONNECTOR_POLL_CONNECT (1 << 1)
#define DRM_CONNECTOR_POLL_DISCONNECT (1 << 2)
@@ -1744,6 +1794,11 @@ int drm_connector_set_panel_orientation_with_quirk(
int width, int height);
int drm_connector_attach_max_bpc_property(struct drm_connector *connector,
int min, int max);
+void drm_connector_create_privacy_screen_properties(struct drm_connector *conn);
+void drm_connector_attach_privacy_screen_properties(struct drm_connector *conn);
+void drm_connector_attach_privacy_screen_provider(
+ struct drm_connector *connector, struct drm_privacy_screen *priv);
+void drm_connector_update_privacy_screen(const struct drm_connector_state *connector_state);
/**
* struct drm_tile_group - Tile group metadata
diff --git a/include/drm/drm_device.h b/include/drm/drm_device.h
index 604b1d1..9923c7a 100644
--- a/include/drm/drm_device.h
+++ b/include/drm/drm_device.h
@@ -6,16 +6,13 @@
#include <linux/mutex.h>
#include <linux/idr.h>
-#include <drm/drm_hashtab.h>
+#include <drm/drm_legacy.h>
#include <drm/drm_mode_config.h>
struct drm_driver;
struct drm_minor;
struct drm_master;
-struct drm_device_dma;
struct drm_vblank_crtc;
-struct drm_sg_mem;
-struct drm_local_map;
struct drm_vma_offset_manager;
struct drm_vram_mm;
struct drm_fb_helper;
diff --git a/include/drm/drm_dp_helper.h b/include/drm/drm_dp_helper.h
index b52df4d..30359e4 100644
--- a/include/drm/drm_dp_helper.h
+++ b/include/drm/drm_dp_helper.h
@@ -1114,8 +1114,15 @@ struct drm_panel;
# define DP_UHBR20 (1 << 1)
# define DP_UHBR13_5 (1 << 2)
-#define DP_128B132B_TRAINING_AUX_RD_INTERVAL 0x2216 /* 2.0 */
-# define DP_128B132B_TRAINING_AUX_RD_INTERVAL_MASK 0x7f
+#define DP_128B132B_TRAINING_AUX_RD_INTERVAL 0x2216 /* 2.0 */
+# define DP_128B132B_TRAINING_AUX_RD_INTERVAL_MASK 0x7f
+# define DP_128B132B_TRAINING_AUX_RD_INTERVAL_400_US 0x00
+# define DP_128B132B_TRAINING_AUX_RD_INTERVAL_4_MS 0x01
+# define DP_128B132B_TRAINING_AUX_RD_INTERVAL_8_MS 0x02
+# define DP_128B132B_TRAINING_AUX_RD_INTERVAL_12_MS 0x03
+# define DP_128B132B_TRAINING_AUX_RD_INTERVAL_16_MS 0x04
+# define DP_128B132B_TRAINING_AUX_RD_INTERVAL_32_MS 0x05
+# define DP_128B132B_TRAINING_AUX_RD_INTERVAL_64_MS 0x06
#define DP_TEST_264BIT_CUSTOM_PATTERN_7_0 0x2230
#define DP_TEST_264BIT_CUSTOM_PATTERN_263_256 0x2250
@@ -1389,6 +1396,11 @@ enum drm_dp_phy {
# define DP_VOLTAGE_SWING_LEVEL_3_SUPPORTED BIT(0)
# define DP_PRE_EMPHASIS_LEVEL_3_SUPPORTED BIT(1)
+#define DP_128B132B_TRAINING_AUX_RD_INTERVAL_PHY_REPEATER1 0xf0022 /* 2.0 */
+#define DP_128B132B_TRAINING_AUX_RD_INTERVAL_PHY_REPEATER(dp_phy) \
+ DP_LTTPR_REG(dp_phy, DP_128B132B_TRAINING_AUX_RD_INTERVAL_PHY_REPEATER1)
+/* see DP_128B132B_TRAINING_AUX_RD_INTERVAL for values */
+
#define DP_LANE0_1_STATUS_PHY_REPEATER1 0xf0030 /* 1.3 */
#define DP_LANE0_1_STATUS_PHY_REPEATER(dp_phy) \
DP_LTTPR_REG(dp_phy, DP_LANE0_1_STATUS_PHY_REPEATER1)
@@ -1527,6 +1539,11 @@ u8 drm_dp_get_adjust_request_post_cursor(const u8 link_status[DP_LINK_STATUS_SIZ
#define DP_LTTPR_COMMON_CAP_SIZE 8
#define DP_LTTPR_PHY_CAP_SIZE 3
+int drm_dp_read_clock_recovery_delay(struct drm_dp_aux *aux, const u8 dpcd[DP_RECEIVER_CAP_SIZE],
+ enum drm_dp_phy dp_phy, bool uhbr);
+int drm_dp_read_channel_eq_delay(struct drm_dp_aux *aux, const u8 dpcd[DP_RECEIVER_CAP_SIZE],
+ enum drm_dp_phy dp_phy, bool uhbr);
+
void drm_dp_link_train_clock_recovery_delay(const struct drm_dp_aux *aux,
const u8 dpcd[DP_RECEIVER_CAP_SIZE]);
void drm_dp_lttpr_link_train_clock_recovery_delay(void);
@@ -1767,6 +1784,13 @@ drm_dp_tps3_supported(const u8 dpcd[DP_RECEIVER_CAP_SIZE])
}
static inline bool
+drm_dp_max_downspread(const u8 dpcd[DP_RECEIVER_CAP_SIZE])
+{
+ return dpcd[DP_DPCD_REV] >= 0x11 ||
+ dpcd[DP_MAX_DOWNSPREAD] & DP_MAX_DOWNSPREAD_0_5;
+}
+
+static inline bool
drm_dp_tps4_supported(const u8 dpcd[DP_RECEIVER_CAP_SIZE])
{
return dpcd[DP_DPCD_REV] >= 0x14 &&
@@ -1851,7 +1875,7 @@ drm_dp_sink_can_do_video_without_timing_msa(const u8 dpcd[DP_RECEIVER_CAP_SIZE])
*
* Note that currently this function will return %false for panels which support various DPCD
* backlight features but which require the brightness be set through PWM, and don't support setting
- * the brightness level via the DPCD. This is a TODO.
+ * the brightness level via the DPCD.
*
* Returns: %True if @edp_dpcd indicates that VESA backlight controls are supported, %false
* otherwise
@@ -1859,8 +1883,7 @@ drm_dp_sink_can_do_video_without_timing_msa(const u8 dpcd[DP_RECEIVER_CAP_SIZE])
static inline bool
drm_edp_backlight_supported(const u8 edp_dpcd[EDP_DISPLAY_CTL_CAP_SIZE])
{
- return (edp_dpcd[1] & DP_EDP_TCON_BACKLIGHT_ADJUSTMENT_CAP) &&
- (edp_dpcd[2] & DP_EDP_BACKLIGHT_BRIGHTNESS_AUX_SET_CAP);
+ return !!(edp_dpcd[1] & DP_EDP_TCON_BACKLIGHT_ADJUSTMENT_CAP);
}
/*
@@ -2221,6 +2244,7 @@ drm_dp_has_quirk(const struct drm_dp_desc *desc, enum drm_dp_quirk quirk)
* @max: The maximum backlight level that may be set
* @lsb_reg_used: Do we also write values to the DP_EDP_BACKLIGHT_BRIGHTNESS_LSB register?
* @aux_enable: Does the panel support the AUX enable cap?
+ * @aux_set: Does the panel support setting the brightness through AUX?
*
* This structure contains various data about an eDP backlight, which can be populated by using
* drm_edp_backlight_init().
@@ -2232,6 +2256,7 @@ struct drm_edp_backlight_info {
bool lsb_reg_used : 1;
bool aux_enable : 1;
+ bool aux_set : 1;
};
int
diff --git a/include/drm/drm_drv.h b/include/drm/drm_drv.h
index 0cd9595..f6159ac 100644
--- a/include/drm/drm_drv.h
+++ b/include/drm/drm_drv.h
@@ -291,8 +291,9 @@ struct drm_driver {
/**
* @gem_create_object: constructor for gem objects
*
- * Hook for allocating the GEM object struct, for use by the CMA and
- * SHMEM GEM helpers.
+ * Hook for allocating the GEM object struct, for use by the CMA
+ * and SHMEM GEM helpers. Returns a GEM object on success, or an
+ * ERR_PTR()-encoded error code otherwise.
*/
struct drm_gem_object *(*gem_create_object)(struct drm_device *dev,
size_t size);
@@ -345,11 +346,14 @@ struct drm_driver {
* mmap hook for GEM drivers, used to implement dma-buf mmap in the
* PRIME helpers.
*
- * FIXME: There's way too much duplication going on here, and also moved
- * to &drm_gem_object_funcs.
+ * This hook only exists for historical reasons. Drivers must use
+ * drm_gem_prime_mmap() to implement it.
+ *
+ * FIXME: Convert all drivers to implement mmap in struct
+ * &drm_gem_object_funcs and inline drm_gem_prime_mmap() into
+ * its callers. This hook should be removed afterwards.
*/
- int (*gem_prime_mmap)(struct drm_gem_object *obj,
- struct vm_area_struct *vma);
+ int (*gem_prime_mmap)(struct drm_gem_object *obj, struct vm_area_struct *vma);
/**
* @dumb_create:
@@ -598,5 +602,6 @@ static inline bool drm_drv_uses_atomic_modeset(struct drm_device *dev)
int drm_dev_set_unique(struct drm_device *dev, const char *name);
+extern bool drm_firmware_drivers_only(void);
#endif
diff --git a/include/drm/drm_format_helper.h b/include/drm/drm_format_helper.h
index e86925c..b30ed5d 100644
--- a/include/drm/drm_format_helper.h
+++ b/include/drm/drm_format_helper.h
@@ -6,38 +6,41 @@
#ifndef __LINUX_DRM_FORMAT_HELPER_H
#define __LINUX_DRM_FORMAT_HELPER_H
+struct drm_format_info;
struct drm_framebuffer;
struct drm_rect;
-void drm_fb_memcpy(void *dst, void *vaddr, struct drm_framebuffer *fb,
- struct drm_rect *clip);
-void drm_fb_memcpy_dstclip(void __iomem *dst, unsigned int dst_pitch, void *vaddr,
- struct drm_framebuffer *fb,
- struct drm_rect *clip);
-void drm_fb_swab(void *dst, void *src, struct drm_framebuffer *fb,
- struct drm_rect *clip, bool cached);
-void drm_fb_xrgb8888_to_rgb332(void *dst, void *vaddr, struct drm_framebuffer *fb,
- struct drm_rect *clip);
-void drm_fb_xrgb8888_to_rgb565(void *dst, void *vaddr,
- struct drm_framebuffer *fb,
- struct drm_rect *clip, bool swab);
-void drm_fb_xrgb8888_to_rgb565_dstclip(void __iomem *dst, unsigned int dst_pitch,
- void *vaddr, struct drm_framebuffer *fb,
- struct drm_rect *clip, bool swab);
-void drm_fb_xrgb8888_to_rgb888(void *dst, void *src, struct drm_framebuffer *fb,
- struct drm_rect *clip);
-void drm_fb_xrgb8888_to_rgb888_dstclip(void __iomem *dst, unsigned int dst_pitch,
- void *vaddr, struct drm_framebuffer *fb,
- struct drm_rect *clip);
-void drm_fb_xrgb8888_to_gray8(u8 *dst, void *vaddr, struct drm_framebuffer *fb,
- struct drm_rect *clip);
+unsigned int drm_fb_clip_offset(unsigned int pitch, const struct drm_format_info *format,
+ const struct drm_rect *clip);
-int drm_fb_blit_rect_dstclip(void __iomem *dst, unsigned int dst_pitch,
- uint32_t dst_format, void *vmap,
- struct drm_framebuffer *fb,
- struct drm_rect *rect);
-int drm_fb_blit_dstclip(void __iomem *dst, unsigned int dst_pitch,
- uint32_t dst_format, void *vmap,
- struct drm_framebuffer *fb);
+void drm_fb_memcpy(void *dst, unsigned int dst_pitch, const void *vaddr,
+ const struct drm_framebuffer *fb, const struct drm_rect *clip);
+void drm_fb_memcpy_toio(void __iomem *dst, unsigned int dst_pitch, const void *vaddr,
+ const struct drm_framebuffer *fb, const struct drm_rect *clip);
+void drm_fb_swab(void *dst, unsigned int dst_pitch, const void *src,
+ const struct drm_framebuffer *fb, const struct drm_rect *clip,
+ bool cached);
+void drm_fb_xrgb8888_to_rgb332(void *dst, unsigned int dst_pitch, const void *vaddr,
+ const struct drm_framebuffer *fb, const struct drm_rect *clip);
+void drm_fb_xrgb8888_to_rgb565(void *dst, unsigned int dst_pitch, const void *vaddr,
+ const struct drm_framebuffer *fb, const struct drm_rect *clip,
+ bool swab);
+void drm_fb_xrgb8888_to_rgb565_toio(void __iomem *dst, unsigned int dst_pitch,
+ const void *vaddr, const struct drm_framebuffer *fb,
+ const struct drm_rect *clip, bool swab);
+void drm_fb_xrgb8888_to_rgb888(void *dst, unsigned int dst_pitch, const void *src,
+ const struct drm_framebuffer *fb, const struct drm_rect *clip);
+void drm_fb_xrgb8888_to_rgb888_toio(void __iomem *dst, unsigned int dst_pitch,
+ const void *vaddr, const struct drm_framebuffer *fb,
+ const struct drm_rect *clip);
+void drm_fb_xrgb8888_to_xrgb2101010_toio(void __iomem *dst, unsigned int dst_pitch,
+ const void *vaddr, const struct drm_framebuffer *fb,
+ const struct drm_rect *clip);
+void drm_fb_xrgb8888_to_gray8(void *dst, unsigned int dst_pitch, const void *vaddr,
+ const struct drm_framebuffer *fb, const struct drm_rect *clip);
+
+int drm_fb_blit_toio(void __iomem *dst, unsigned int dst_pitch, uint32_t dst_format,
+ const void *vmap, const struct drm_framebuffer *fb,
+ const struct drm_rect *rect);
#endif /* __LINUX_DRM_FORMAT_HELPER_H */
diff --git a/include/drm/drm_gem_atomic_helper.h b/include/drm/drm_gem_atomic_helper.h
index 48222a1..0b1e2dd 100644
--- a/include/drm/drm_gem_atomic_helper.h
+++ b/include/drm/drm_gem_atomic_helper.h
@@ -23,6 +23,24 @@ int drm_gem_simple_display_pipe_prepare_fb(struct drm_simple_display_pipe *pipe,
*/
/**
+ * DRM_SHADOW_PLANE_MAX_WIDTH - Maximum width of a plane's shadow buffer in pixels
+ *
+ * For drivers with shadow planes, the maximum width of the framebuffer is
+ * usually independent from hardware limitations. Drivers can initialize struct
+ * drm_mode_config.max_width from DRM_SHADOW_PLANE_MAX_WIDTH.
+ */
+#define DRM_SHADOW_PLANE_MAX_WIDTH (4096u)
+
+/**
+ * DRM_SHADOW_PLANE_MAX_HEIGHT - Maximum height of a plane's shadow buffer in scanlines
+ *
+ * For drivers with shadow planes, the maximum height of the framebuffer is
+ * usually independent from hardware limitations. Drivers can initialize struct
+ * drm_mode_config.max_height from DRM_SHADOW_PLANE_MAX_HEIGHT.
+ */
+#define DRM_SHADOW_PLANE_MAX_HEIGHT (4096u)
+
+/**
* struct drm_shadow_plane_state - plane state for planes with shadow buffers
*
* For planes that use a shadow buffer, struct drm_shadow_plane_state
diff --git a/include/drm/drm_gem_cma_helper.h b/include/drm/drm_gem_cma_helper.h
index cd13508..adb507a 100644
--- a/include/drm/drm_gem_cma_helper.h
+++ b/include/drm/drm_gem_cma_helper.h
@@ -32,42 +32,108 @@ struct drm_gem_cma_object {
#define to_drm_gem_cma_obj(gem_obj) \
container_of(gem_obj, struct drm_gem_cma_object, base)
-#ifndef CONFIG_MMU
-#define DRM_GEM_CMA_UNMAPPED_AREA_FOPS \
- .get_unmapped_area = drm_gem_cma_get_unmapped_area,
-#else
-#define DRM_GEM_CMA_UNMAPPED_AREA_FOPS
-#endif
+struct drm_gem_cma_object *drm_gem_cma_create(struct drm_device *drm,
+ size_t size);
+void drm_gem_cma_free(struct drm_gem_cma_object *cma_obj);
+void drm_gem_cma_print_info(const struct drm_gem_cma_object *cma_obj,
+ struct drm_printer *p, unsigned int indent);
+struct sg_table *drm_gem_cma_get_sg_table(struct drm_gem_cma_object *cma_obj);
+int drm_gem_cma_vmap(struct drm_gem_cma_object *cma_obj, struct dma_buf_map *map);
+int drm_gem_cma_mmap(struct drm_gem_cma_object *cma_obj, struct vm_area_struct *vma);
+
+extern const struct vm_operations_struct drm_gem_cma_vm_ops;
+
+/*
+ * GEM object functions
+ */
/**
- * DEFINE_DRM_GEM_CMA_FOPS() - macro to generate file operations for CMA drivers
- * @name: name for the generated structure
+ * drm_gem_cma_object_free - GEM object function for drm_gem_cma_free()
+ * @obj: GEM object to free
*
- * This macro autogenerates a suitable &struct file_operations for CMA based
- * drivers, which can be assigned to &drm_driver.fops. Note that this structure
- * cannot be shared between drivers, because it contains a reference to the
- * current module using THIS_MODULE.
- *
- * Note that the declaration is already marked as static - if you need a
- * non-static version of this you're probably doing it wrong and will break the
- * THIS_MODULE reference by accident.
+ * This function wraps drm_gem_cma_free_object(). Drivers that employ the CMA helpers
+ * should use it as their &drm_gem_object_funcs.free handler.
*/
-#define DEFINE_DRM_GEM_CMA_FOPS(name) \
- static const struct file_operations name = {\
- .owner = THIS_MODULE,\
- .open = drm_open,\
- .release = drm_release,\
- .unlocked_ioctl = drm_ioctl,\
- .compat_ioctl = drm_compat_ioctl,\
- .poll = drm_poll,\
- .read = drm_read,\
- .llseek = noop_llseek,\
- .mmap = drm_gem_mmap,\
- DRM_GEM_CMA_UNMAPPED_AREA_FOPS \
- }
+static inline void drm_gem_cma_object_free(struct drm_gem_object *obj)
+{
+ struct drm_gem_cma_object *cma_obj = to_drm_gem_cma_obj(obj);
-/* free GEM object */
-void drm_gem_cma_free_object(struct drm_gem_object *gem_obj);
+ drm_gem_cma_free(cma_obj);
+}
+
+/**
+ * drm_gem_cma_object_print_info() - Print &drm_gem_cma_object info for debugfs
+ * @p: DRM printer
+ * @indent: Tab indentation level
+ * @obj: GEM object
+ *
+ * This function wraps drm_gem_cma_print_info(). Drivers that employ the CMA helpers
+ * should use this function as their &drm_gem_object_funcs.print_info handler.
+ */
+static inline void drm_gem_cma_object_print_info(struct drm_printer *p, unsigned int indent,
+ const struct drm_gem_object *obj)
+{
+ const struct drm_gem_cma_object *cma_obj = to_drm_gem_cma_obj(obj);
+
+ drm_gem_cma_print_info(cma_obj, p, indent);
+}
+
+/**
+ * drm_gem_cma_object_get_sg_table - GEM object function for drm_gem_cma_get_sg_table()
+ * @obj: GEM object
+ *
+ * This function wraps drm_gem_cma_get_sg_table(). Drivers that employ the CMA helpers should
+ * use it as their &drm_gem_object_funcs.get_sg_table handler.
+ *
+ * Returns:
+ * A pointer to the scatter/gather table of pinned pages or NULL on failure.
+ */
+static inline struct sg_table *drm_gem_cma_object_get_sg_table(struct drm_gem_object *obj)
+{
+ struct drm_gem_cma_object *cma_obj = to_drm_gem_cma_obj(obj);
+
+ return drm_gem_cma_get_sg_table(cma_obj);
+}
+
+/*
+ * drm_gem_cma_object_vmap - GEM object function for drm_gem_cma_vmap()
+ * @obj: GEM object
+ * @map: Returns the kernel virtual address of the CMA GEM object's backing store.
+ *
+ * This function wraps drm_gem_cma_vmap(). Drivers that employ the CMA helpers should
+ * use it as their &drm_gem_object_funcs.vmap handler.
+ *
+ * Returns:
+ * 0 on success or a negative error code on failure.
+ */
+static inline int drm_gem_cma_object_vmap(struct drm_gem_object *obj, struct dma_buf_map *map)
+{
+ struct drm_gem_cma_object *cma_obj = to_drm_gem_cma_obj(obj);
+
+ return drm_gem_cma_vmap(cma_obj, map);
+}
+
+/**
+ * drm_gem_cma_object_mmap - GEM object function for drm_gem_cma_mmap()
+ * @obj: GEM object
+ * @vma: VMA for the area to be mapped
+ *
+ * This function wraps drm_gem_cma_mmap(). Drivers that employ the cma helpers should
+ * use it as their &drm_gem_object_funcs.mmap handler.
+ *
+ * Returns:
+ * 0 on success or a negative error code on failure.
+ */
+static inline int drm_gem_cma_object_mmap(struct drm_gem_object *obj, struct vm_area_struct *vma)
+{
+ struct drm_gem_cma_object *cma_obj = to_drm_gem_cma_obj(obj);
+
+ return drm_gem_cma_mmap(cma_obj, vma);
+}
+
+/*
+ * Driver ops
+ */
/* create memory region for DRM framebuffer */
int drm_gem_cma_dumb_create_internal(struct drm_file *file_priv,
@@ -79,30 +145,10 @@ int drm_gem_cma_dumb_create(struct drm_file *file_priv,
struct drm_device *drm,
struct drm_mode_create_dumb *args);
-/* allocate physical memory */
-struct drm_gem_cma_object *drm_gem_cma_create(struct drm_device *drm,
- size_t size);
-
-extern const struct vm_operations_struct drm_gem_cma_vm_ops;
-
-#ifndef CONFIG_MMU
-unsigned long drm_gem_cma_get_unmapped_area(struct file *filp,
- unsigned long addr,
- unsigned long len,
- unsigned long pgoff,
- unsigned long flags);
-#endif
-
-void drm_gem_cma_print_info(struct drm_printer *p, unsigned int indent,
- const struct drm_gem_object *obj);
-
-struct sg_table *drm_gem_cma_get_sg_table(struct drm_gem_object *obj);
struct drm_gem_object *
drm_gem_cma_prime_import_sg_table(struct drm_device *dev,
struct dma_buf_attachment *attach,
struct sg_table *sgt);
-int drm_gem_cma_vmap(struct drm_gem_object *obj, struct dma_buf_map *map);
-int drm_gem_cma_mmap(struct drm_gem_object *obj, struct vm_area_struct *vma);
/**
* DRM_GEM_CMA_DRIVER_OPS_WITH_DUMB_CREATE - CMA GEM driver operations
@@ -185,4 +231,47 @@ drm_gem_cma_prime_import_sg_table_vmap(struct drm_device *drm,
struct dma_buf_attachment *attach,
struct sg_table *sgt);
+/*
+ * File ops
+ */
+
+#ifndef CONFIG_MMU
+unsigned long drm_gem_cma_get_unmapped_area(struct file *filp,
+ unsigned long addr,
+ unsigned long len,
+ unsigned long pgoff,
+ unsigned long flags);
+#define DRM_GEM_CMA_UNMAPPED_AREA_FOPS \
+ .get_unmapped_area = drm_gem_cma_get_unmapped_area,
+#else
+#define DRM_GEM_CMA_UNMAPPED_AREA_FOPS
+#endif
+
+/**
+ * DEFINE_DRM_GEM_CMA_FOPS() - macro to generate file operations for CMA drivers
+ * @name: name for the generated structure
+ *
+ * This macro autogenerates a suitable &struct file_operations for CMA based
+ * drivers, which can be assigned to &drm_driver.fops. Note that this structure
+ * cannot be shared between drivers, because it contains a reference to the
+ * current module using THIS_MODULE.
+ *
+ * Note that the declaration is already marked as static - if you need a
+ * non-static version of this you're probably doing it wrong and will break the
+ * THIS_MODULE reference by accident.
+ */
+#define DEFINE_DRM_GEM_CMA_FOPS(name) \
+ static const struct file_operations name = {\
+ .owner = THIS_MODULE,\
+ .open = drm_open,\
+ .release = drm_release,\
+ .unlocked_ioctl = drm_ioctl,\
+ .compat_ioctl = drm_compat_ioctl,\
+ .poll = drm_poll,\
+ .read = drm_read,\
+ .llseek = noop_llseek,\
+ .mmap = drm_gem_mmap,\
+ DRM_GEM_CMA_UNMAPPED_AREA_FOPS \
+ }
+
#endif /* __DRM_GEM_CMA_HELPER_H__ */
diff --git a/include/drm/drm_gem_shmem_helper.h b/include/drm/drm_gem_shmem_helper.h
index 434328d..311d66c 100644
--- a/include/drm/drm_gem_shmem_helper.h
+++ b/include/drm/drm_gem_shmem_helper.h
@@ -107,16 +107,17 @@ struct drm_gem_shmem_object {
container_of(obj, struct drm_gem_shmem_object, base)
struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev, size_t size);
-void drm_gem_shmem_free_object(struct drm_gem_object *obj);
+void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem);
int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem);
void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem);
-int drm_gem_shmem_pin(struct drm_gem_object *obj);
-void drm_gem_shmem_unpin(struct drm_gem_object *obj);
-int drm_gem_shmem_vmap(struct drm_gem_object *obj, struct dma_buf_map *map);
-void drm_gem_shmem_vunmap(struct drm_gem_object *obj, struct dma_buf_map *map);
+int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem);
+void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem);
+int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem, struct dma_buf_map *map);
+void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem, struct dma_buf_map *map);
+int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct *vma);
-int drm_gem_shmem_madvise(struct drm_gem_object *obj, int madv);
+int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv);
static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
{
@@ -125,29 +126,156 @@ static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem
!shmem->base.dma_buf && !shmem->base.import_attach;
}
-void drm_gem_shmem_purge_locked(struct drm_gem_object *obj);
-bool drm_gem_shmem_purge(struct drm_gem_object *obj);
+void drm_gem_shmem_purge_locked(struct drm_gem_shmem_object *shmem);
+bool drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
-struct drm_gem_shmem_object *
-drm_gem_shmem_create_with_handle(struct drm_file *file_priv,
- struct drm_device *dev, size_t size,
- uint32_t *handle);
+struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem);
+struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem);
-int drm_gem_shmem_dumb_create(struct drm_file *file, struct drm_device *dev,
- struct drm_mode_create_dumb *args);
+void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
+ struct drm_printer *p, unsigned int indent);
-int drm_gem_shmem_mmap(struct drm_gem_object *obj, struct vm_area_struct *vma);
+/*
+ * GEM object functions
+ */
-void drm_gem_shmem_print_info(struct drm_printer *p, unsigned int indent,
- const struct drm_gem_object *obj);
+/**
+ * drm_gem_shmem_object_free - GEM object function for drm_gem_shmem_free()
+ * @obj: GEM object to free
+ *
+ * This function wraps drm_gem_shmem_free(). Drivers that employ the shmem helpers
+ * should use it as their &drm_gem_object_funcs.free handler.
+ */
+static inline void drm_gem_shmem_object_free(struct drm_gem_object *obj)
+{
+ struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
-struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_object *obj);
+ drm_gem_shmem_free(shmem);
+}
+
+/**
+ * drm_gem_shmem_object_print_info() - Print &drm_gem_shmem_object info for debugfs
+ * @p: DRM printer
+ * @indent: Tab indentation level
+ * @obj: GEM object
+ *
+ * This function wraps drm_gem_shmem_print_info(). Drivers that employ the shmem helpers should
+ * use this function as their &drm_gem_object_funcs.print_info handler.
+ */
+static inline void drm_gem_shmem_object_print_info(struct drm_printer *p, unsigned int indent,
+ const struct drm_gem_object *obj)
+{
+ const struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
+
+ drm_gem_shmem_print_info(shmem, p, indent);
+}
+
+/**
+ * drm_gem_shmem_object_pin - GEM object function for drm_gem_shmem_pin()
+ * @obj: GEM object
+ *
+ * This function wraps drm_gem_shmem_pin(). Drivers that employ the shmem helpers should
+ * use it as their &drm_gem_object_funcs.pin handler.
+ */
+static inline int drm_gem_shmem_object_pin(struct drm_gem_object *obj)
+{
+ struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
+
+ return drm_gem_shmem_pin(shmem);
+}
+
+/**
+ * drm_gem_shmem_object_unpin - GEM object function for drm_gem_shmem_unpin()
+ * @obj: GEM object
+ *
+ * This function wraps drm_gem_shmem_unpin(). Drivers that employ the shmem helpers should
+ * use it as their &drm_gem_object_funcs.unpin handler.
+ */
+static inline void drm_gem_shmem_object_unpin(struct drm_gem_object *obj)
+{
+ struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
+
+ drm_gem_shmem_unpin(shmem);
+}
+
+/**
+ * drm_gem_shmem_object_get_sg_table - GEM object function for drm_gem_shmem_get_sg_table()
+ * @obj: GEM object
+ *
+ * This function wraps drm_gem_shmem_get_sg_table(). Drivers that employ the shmem helpers should
+ * use it as their &drm_gem_object_funcs.get_sg_table handler.
+ *
+ * Returns:
+ * A pointer to the scatter/gather table of pinned pages or NULL on failure.
+ */
+static inline struct sg_table *drm_gem_shmem_object_get_sg_table(struct drm_gem_object *obj)
+{
+ struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
+
+ return drm_gem_shmem_get_sg_table(shmem);
+}
+
+/*
+ * drm_gem_shmem_object_vmap - GEM object function for drm_gem_shmem_vmap()
+ * @obj: GEM object
+ * @map: Returns the kernel virtual address of the SHMEM GEM object's backing store.
+ *
+ * This function wraps drm_gem_shmem_vmap(). Drivers that employ the shmem helpers should
+ * use it as their &drm_gem_object_funcs.vmap handler.
+ *
+ * Returns:
+ * 0 on success or a negative error code on failure.
+ */
+static inline int drm_gem_shmem_object_vmap(struct drm_gem_object *obj, struct dma_buf_map *map)
+{
+ struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
+
+ return drm_gem_shmem_vmap(shmem, map);
+}
+
+/*
+ * drm_gem_shmem_object_vunmap - GEM object function for drm_gem_shmem_vunmap()
+ * @obj: GEM object
+ * @map: Kernel virtual address where the SHMEM GEM object was mapped
+ *
+ * This function wraps drm_gem_shmem_vunmap(). Drivers that employ the shmem helpers should
+ * use it as their &drm_gem_object_funcs.vunmap handler.
+ */
+static inline void drm_gem_shmem_object_vunmap(struct drm_gem_object *obj, struct dma_buf_map *map)
+{
+ struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
+
+ drm_gem_shmem_vunmap(shmem, map);
+}
+
+/**
+ * drm_gem_shmem_object_mmap - GEM object function for drm_gem_shmem_mmap()
+ * @obj: GEM object
+ * @vma: VMA for the area to be mapped
+ *
+ * This function wraps drm_gem_shmem_mmap(). Drivers that employ the shmem helpers should
+ * use it as their &drm_gem_object_funcs.mmap handler.
+ *
+ * Returns:
+ * 0 on success or a negative error code on failure.
+ */
+static inline int drm_gem_shmem_object_mmap(struct drm_gem_object *obj, struct vm_area_struct *vma)
+{
+ struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
+
+ return drm_gem_shmem_mmap(shmem, vma);
+}
+
+/*
+ * Driver ops
+ */
+
struct drm_gem_object *
drm_gem_shmem_prime_import_sg_table(struct drm_device *dev,
struct dma_buf_attachment *attach,
struct sg_table *sgt);
-
-struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_object *obj);
+int drm_gem_shmem_dumb_create(struct drm_file *file, struct drm_device *dev,
+ struct drm_mode_create_dumb *args);
/**
* DRM_GEM_SHMEM_DRIVER_OPS - Default shmem GEM operations
diff --git a/include/drm/drm_gem_ttm_helper.h b/include/drm/drm_gem_ttm_helper.h
index c1aa02b..78040f6 100644
--- a/include/drm/drm_gem_ttm_helper.h
+++ b/include/drm/drm_gem_ttm_helper.h
@@ -3,7 +3,7 @@
#ifndef DRM_GEM_TTM_HELPER_H
#define DRM_GEM_TTM_HELPER_H
-#include <linux/kernel.h>
+#include <linux/container_of.h>
#include <drm/drm_device.h>
#include <drm/drm_gem.h>
diff --git a/include/drm/drm_gem_vram_helper.h b/include/drm/drm_gem_vram_helper.h
index d3cf06c..b4ce27a 100644
--- a/include/drm/drm_gem_vram_helper.h
+++ b/include/drm/drm_gem_vram_helper.h
@@ -11,8 +11,8 @@
#include <drm/ttm/ttm_bo_api.h>
#include <drm/ttm/ttm_bo_driver.h>
+#include <linux/container_of.h>
#include <linux/dma-buf-map.h>
-#include <linux/kernel.h> /* for container_of() */
struct drm_mode_create_dumb;
struct drm_plane;
diff --git a/include/drm/drm_hashtab.h b/include/drm/drm_hashtab.h
deleted file mode 100644
index bb95ff0..0000000
--- a/include/drm/drm_hashtab.h
+++ /dev/null
@@ -1,79 +0,0 @@
-/**************************************************************************
- *
- * Copyright 2006 Tungsten Graphics, Inc., Bismack, ND. USA.
- * All Rights Reserved.
- *
- * Permission is hereby granted, free of charge, to any person obtaining a
- * copy of this software and associated documentation files (the
- * "Software"), to deal in the Software without restriction, including
- * without limitation the rights to use, copy, modify, merge, publish,
- * distribute, sub license, and/or sell copies of the Software, and to
- * permit persons to whom the Software is furnished to do so, subject to
- * the following conditions:
- *
- * The above copyright notice and this permission notice (including the
- * next paragraph) shall be included in all copies or substantial portions
- * of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. IN NO EVENT SHALL
- * THE COPYRIGHT HOLDERS, AUTHORS AND/OR ITS SUPPLIERS BE LIABLE FOR ANY CLAIM,
- * DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
- * OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
- * USE OR OTHER DEALINGS IN THE SOFTWARE.
- *
- *
- **************************************************************************/
-/*
- * Simple open hash tab implementation.
- *
- * Authors:
- * Thomas Hellström <thomas-at-tungstengraphics-dot-com>
- */
-
-#ifndef DRM_HASHTAB_H
-#define DRM_HASHTAB_H
-
-#include <linux/list.h>
-
-#define drm_hash_entry(_ptr, _type, _member) container_of(_ptr, _type, _member)
-
-struct drm_hash_item {
- struct hlist_node head;
- unsigned long key;
-};
-
-struct drm_open_hash {
- struct hlist_head *table;
- u8 order;
-};
-
-int drm_ht_create(struct drm_open_hash *ht, unsigned int order);
-int drm_ht_insert_item(struct drm_open_hash *ht, struct drm_hash_item *item);
-int drm_ht_just_insert_please(struct drm_open_hash *ht, struct drm_hash_item *item,
- unsigned long seed, int bits, int shift,
- unsigned long add);
-int drm_ht_find_item(struct drm_open_hash *ht, unsigned long key, struct drm_hash_item **item);
-
-void drm_ht_verbose_list(struct drm_open_hash *ht, unsigned long key);
-int drm_ht_remove_key(struct drm_open_hash *ht, unsigned long key);
-int drm_ht_remove_item(struct drm_open_hash *ht, struct drm_hash_item *item);
-void drm_ht_remove(struct drm_open_hash *ht);
-
-/*
- * RCU-safe interface
- *
- * The user of this API needs to make sure that two or more instances of the
- * hash table manipulation functions are never run simultaneously.
- * The lookup function drm_ht_find_item_rcu may, however, run simultaneously
- * with any of the manipulation functions as long as it's called from within
- * an RCU read-locked section.
- */
-#define drm_ht_insert_item_rcu drm_ht_insert_item
-#define drm_ht_just_insert_please_rcu drm_ht_just_insert_please
-#define drm_ht_remove_key_rcu drm_ht_remove_key
-#define drm_ht_remove_item_rcu drm_ht_remove_item
-#define drm_ht_find_item_rcu drm_ht_find_item
-
-#endif
diff --git a/include/drm/drm_legacy.h b/include/drm/drm_legacy.h
index 58dc8d8..0fc8541 100644
--- a/include/drm/drm_legacy.h
+++ b/include/drm/drm_legacy.h
@@ -37,7 +37,6 @@
#include <drm/drm.h>
#include <drm/drm_auth.h>
-#include <drm/drm_hashtab.h>
struct drm_device;
struct drm_driver;
@@ -51,6 +50,20 @@ struct pci_driver;
* you're doing it terribly wrong.
*/
+/*
+ * Hash-table Support
+ */
+
+struct drm_hash_item {
+ struct hlist_node head;
+ unsigned long key;
+};
+
+struct drm_open_hash {
+ struct hlist_head *table;
+ u8 order;
+};
+
/**
* DMA buffer.
*/
diff --git a/include/drm/drm_mm.h b/include/drm/drm_mm.h
index 9b4292f..ac33ba1 100644
--- a/include/drm/drm_mm.h
+++ b/include/drm/drm_mm.h
@@ -39,13 +39,15 @@
*/
#include <linux/bug.h>
#include <linux/rbtree.h>
-#include <linux/kernel.h>
+#include <linux/limits.h>
#include <linux/mm_types.h>
#include <linux/list.h>
#include <linux/spinlock.h>
#ifdef CONFIG_DRM_DEBUG_MM
#include <linux/stackdepot.h>
#endif
+#include <linux/types.h>
+
#include <drm/drm_print.h>
#ifdef CONFIG_DRM_DEBUG_MM
diff --git a/include/drm/drm_mode_config.h b/include/drm/drm_mode_config.h
index 48b7de8..91ca575 100644
--- a/include/drm/drm_mode_config.h
+++ b/include/drm/drm_mode_config.h
@@ -359,6 +359,19 @@ struct drm_mode_config_funcs {
* Core mode resource tracking structure. All CRTC, encoders, and connectors
* enumerated by the driver are added here, as are global properties. Some
* global restrictions are also here, e.g. dimension restrictions.
+ *
+ * Framebuffer sizes refer to the virtual screen that can be displayed by
+ * the CRTC. This can be different from the physical resolution programmed.
+ * The minimum width and height, stored in @min_width and @min_height,
+ * describe the smallest size of the framebuffer. It correlates to the
+ * minimum programmable resolution.
+ * The maximum width, stored in @max_width, is typically limited by the
+ * maximum pitch between two adjacent scanlines. The maximum height, stored
+ * in @max_height, is usually only limited by the amount of addressable video
+ * memory. For hardware that has no real maximum, drivers should pick a
+ * reasonable default.
+ *
+ * See also @DRM_SHADOW_PLANE_MAX_WIDTH and @DRM_SHADOW_PLANE_MAX_HEIGHT.
*/
struct drm_mode_config {
/**
diff --git a/include/drm/drm_of.h b/include/drm/drm_of.h
index b9b093a..99f79ac 100644
--- a/include/drm/drm_of.h
+++ b/include/drm/drm_of.h
@@ -49,6 +49,7 @@ int drm_of_find_panel_or_bridge(const struct device_node *np,
struct drm_bridge **bridge);
int drm_of_lvds_get_dual_link_pixel_order(const struct device_node *port1,
const struct device_node *port2);
+int drm_of_lvds_get_data_mapping(const struct device_node *port);
#else
static inline uint32_t drm_of_crtc_port_mask(struct drm_device *dev,
struct device_node *port)
@@ -98,6 +99,12 @@ drm_of_lvds_get_dual_link_pixel_order(const struct device_node *port1,
{
return -EINVAL;
}
+
+static inline int
+drm_of_lvds_get_data_mapping(const struct device_node *port)
+{
+ return -EINVAL;
+}
#endif
/*
diff --git a/include/drm/drm_privacy_screen_consumer.h b/include/drm/drm_privacy_screen_consumer.h
new file mode 100644
index 0000000..7f66a90
--- /dev/null
+++ b/include/drm/drm_privacy_screen_consumer.h
@@ -0,0 +1,65 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright (C) 2020 Red Hat, Inc.
+ *
+ * Authors:
+ * Hans de Goede <hdegoede@redhat.com>
+ */
+
+#ifndef __DRM_PRIVACY_SCREEN_CONSUMER_H__
+#define __DRM_PRIVACY_SCREEN_CONSUMER_H__
+
+#include <linux/device.h>
+#include <drm/drm_connector.h>
+
+struct drm_privacy_screen;
+
+#if IS_ENABLED(CONFIG_DRM_PRIVACY_SCREEN)
+struct drm_privacy_screen *drm_privacy_screen_get(struct device *dev,
+ const char *con_id);
+void drm_privacy_screen_put(struct drm_privacy_screen *priv);
+
+int drm_privacy_screen_set_sw_state(struct drm_privacy_screen *priv,
+ enum drm_privacy_screen_status sw_state);
+void drm_privacy_screen_get_state(struct drm_privacy_screen *priv,
+ enum drm_privacy_screen_status *sw_state_ret,
+ enum drm_privacy_screen_status *hw_state_ret);
+
+int drm_privacy_screen_register_notifier(struct drm_privacy_screen *priv,
+ struct notifier_block *nb);
+int drm_privacy_screen_unregister_notifier(struct drm_privacy_screen *priv,
+ struct notifier_block *nb);
+#else
+static inline struct drm_privacy_screen *drm_privacy_screen_get(struct device *dev,
+ const char *con_id)
+{
+ return ERR_PTR(-ENODEV);
+}
+static inline void drm_privacy_screen_put(struct drm_privacy_screen *priv)
+{
+}
+static inline int drm_privacy_screen_set_sw_state(struct drm_privacy_screen *priv,
+ enum drm_privacy_screen_status sw_state)
+{
+ return -ENODEV;
+}
+static inline void drm_privacy_screen_get_state(struct drm_privacy_screen *priv,
+ enum drm_privacy_screen_status *sw_state_ret,
+ enum drm_privacy_screen_status *hw_state_ret)
+{
+ *sw_state_ret = PRIVACY_SCREEN_DISABLED;
+ *hw_state_ret = PRIVACY_SCREEN_DISABLED;
+}
+static inline int drm_privacy_screen_register_notifier(struct drm_privacy_screen *priv,
+ struct notifier_block *nb)
+{
+ return -ENODEV;
+}
+static inline int drm_privacy_screen_unregister_notifier(struct drm_privacy_screen *priv,
+ struct notifier_block *nb)
+{
+ return -ENODEV;
+}
+#endif
+
+#endif
diff --git a/include/drm/drm_privacy_screen_driver.h b/include/drm/drm_privacy_screen_driver.h
new file mode 100644
index 0000000..24591b6
--- /dev/null
+++ b/include/drm/drm_privacy_screen_driver.h
@@ -0,0 +1,84 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright (C) 2020 Red Hat, Inc.
+ *
+ * Authors:
+ * Hans de Goede <hdegoede@redhat.com>
+ */
+
+#ifndef __DRM_PRIVACY_SCREEN_DRIVER_H__
+#define __DRM_PRIVACY_SCREEN_DRIVER_H__
+
+#include <linux/device.h>
+#include <linux/list.h>
+#include <linux/mutex.h>
+#include <drm/drm_connector.h>
+
+struct drm_privacy_screen;
+
+/**
+ * struct drm_privacy_screen_ops - drm_privacy_screen operations
+ *
+ * Defines the operations which the privacy-screen class code may call.
+ * These functions should be implemented by the privacy-screen driver.
+ */
+struct drm_privacy_screen_ops {
+ /**
+ * @set_sw_state: Called to request a change of the privacy-screen
+ * state. The privacy-screen class code contains a check to avoid this
+ * getting called when the hw_state reports the state is locked.
+ * It is the driver's responsibility to update sw_state and hw_state.
+ * This is always called with the drm_privacy_screen's lock held.
+ */
+ int (*set_sw_state)(struct drm_privacy_screen *priv,
+ enum drm_privacy_screen_status sw_state);
+ /**
+ * @get_hw_state: Called to request that the driver gets the current
+ * privacy-screen state from the hardware and then updates sw_state and
+ * hw_state accordingly. This will be called by the core just before
+ * the privacy-screen is registered in sysfs.
+ */
+ void (*get_hw_state)(struct drm_privacy_screen *priv);
+};
+
+/**
+ * struct drm_privacy_screen - central privacy-screen structure
+ *
+ * Central privacy-screen structure, this contains the struct device used
+ * to register the screen in sysfs, the screen's state, ops, etc.
+ */
+struct drm_privacy_screen {
+ /** @dev: device used to register the privacy-screen in sysfs. */
+ struct device dev;
+ /** @lock: mutex protection all fields in this struct. */
+ struct mutex lock;
+ /** @list: privacy-screen devices list list-entry. */
+ struct list_head list;
+ /** @notifier_head: privacy-screen notifier head. */
+ struct blocking_notifier_head notifier_head;
+ /**
+ * @ops: &struct drm_privacy_screen_ops for this privacy-screen.
+ * This is NULL if the driver has unregistered the privacy-screen.
+ */
+ const struct drm_privacy_screen_ops *ops;
+ /**
+ * @sw_state: The privacy-screen's software state, see
+ * :ref:`Standard Connector Properties<standard_connector_properties>`
+ * for more info.
+ */
+ enum drm_privacy_screen_status sw_state;
+ /**
+ * @hw_state: The privacy-screen's hardware state, see
+ * :ref:`Standard Connector Properties<standard_connector_properties>`
+ * for more info.
+ */
+ enum drm_privacy_screen_status hw_state;
+};
+
+struct drm_privacy_screen *drm_privacy_screen_register(
+ struct device *parent, const struct drm_privacy_screen_ops *ops);
+void drm_privacy_screen_unregister(struct drm_privacy_screen *priv);
+
+void drm_privacy_screen_call_notifier_chain(struct drm_privacy_screen *priv);
+
+#endif
diff --git a/include/drm/drm_privacy_screen_machine.h b/include/drm/drm_privacy_screen_machine.h
new file mode 100644
index 0000000..02e5371
--- /dev/null
+++ b/include/drm/drm_privacy_screen_machine.h
@@ -0,0 +1,46 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright (C) 2020 Red Hat, Inc.
+ *
+ * Authors:
+ * Hans de Goede <hdegoede@redhat.com>
+ */
+
+#ifndef __DRM_PRIVACY_SCREEN_MACHINE_H__
+#define __DRM_PRIVACY_SCREEN_MACHINE_H__
+
+#include <linux/list.h>
+
+/**
+ * struct drm_privacy_screen_lookup - static privacy-screen lookup list entry
+ *
+ * Used for the static lookup-list for mapping privacy-screen consumer
+ * dev-connector pairs to a privacy-screen provider.
+ */
+struct drm_privacy_screen_lookup {
+ /** @list: Lookup list list-entry. */
+ struct list_head list;
+ /** @dev_id: Consumer device name or NULL to match all devices. */
+ const char *dev_id;
+ /** @con_id: Consumer connector name or NULL to match all connectors. */
+ const char *con_id;
+ /** @provider: dev_name() of the privacy_screen provider. */
+ const char *provider;
+};
+
+void drm_privacy_screen_lookup_add(struct drm_privacy_screen_lookup *lookup);
+void drm_privacy_screen_lookup_remove(struct drm_privacy_screen_lookup *lookup);
+
+#if IS_ENABLED(CONFIG_DRM_PRIVACY_SCREEN) && IS_ENABLED(CONFIG_X86)
+void drm_privacy_screen_lookup_init(void);
+void drm_privacy_screen_lookup_exit(void);
+#else
+static inline void drm_privacy_screen_lookup_init(void)
+{
+}
+static inline void drm_privacy_screen_lookup_exit(void)
+{
+}
+#endif
+
+#endif
diff --git a/include/drm/drm_probe_helper.h b/include/drm/drm_probe_helper.h
index 04c57564..48300aa 100644
--- a/include/drm/drm_probe_helper.h
+++ b/include/drm/drm_probe_helper.h
@@ -20,6 +20,7 @@ void drm_kms_helper_poll_fini(struct drm_device *dev);
bool drm_helper_hpd_irq_event(struct drm_device *dev);
bool drm_connector_helper_hpd_irq_event(struct drm_connector *connector);
void drm_kms_helper_hotplug_event(struct drm_device *dev);
+void drm_kms_helper_connector_hotplug_event(struct drm_connector *connector);
void drm_kms_helper_poll_disable(struct drm_device *dev);
void drm_kms_helper_poll_enable(struct drm_device *dev);
diff --git a/include/drm/drm_sysfs.h b/include/drm/drm_sysfs.h
index d454ef6..6273cac 100644
--- a/include/drm/drm_sysfs.h
+++ b/include/drm/drm_sysfs.h
@@ -11,6 +11,7 @@ int drm_class_device_register(struct device *dev);
void drm_class_device_unregister(struct device *dev);
void drm_sysfs_hotplug_event(struct drm_device *dev);
+void drm_sysfs_connector_hotplug_event(struct drm_connector *connector);
void drm_sysfs_connector_status_event(struct drm_connector *connector,
struct drm_property *property);
#endif
diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h
index f011e4c..bbc22fa 100644
--- a/include/drm/gpu_scheduler.h
+++ b/include/drm/gpu_scheduler.h
@@ -28,6 +28,7 @@
#include <linux/dma-fence.h>
#include <linux/completion.h>
#include <linux/xarray.h>
+#include <linux/irq_work.h>
#define MAX_WAIT_SCHED_ENTITY_Q_EMPTY msecs_to_jiffies(1000)
@@ -286,7 +287,16 @@ struct drm_sched_job {
struct list_head list;
struct drm_gpu_scheduler *sched;
struct drm_sched_fence *s_fence;
- struct dma_fence_cb finish_cb;
+
+ /*
+ * work is used only after finish_cb has been used and will not be
+ * accessed anymore.
+ */
+ union {
+ struct dma_fence_cb finish_cb;
+ struct irq_work work;
+ };
+
uint64_t id;
atomic_t karma;
enum drm_sched_priority s_priority;
diff --git a/include/drm/i915_pciids.h b/include/drm/i915_pciids.h
index c00ac54..baf3d1d 100644
--- a/include/drm/i915_pciids.h
+++ b/include/drm/i915_pciids.h
@@ -666,4 +666,13 @@
INTEL_VGA_DEVICE(0x46C2, info), \
INTEL_VGA_DEVICE(0x46C3, info)
+/* RPL-S */
+#define INTEL_RPLS_IDS(info) \
+ INTEL_VGA_DEVICE(0xA780, info), \
+ INTEL_VGA_DEVICE(0xA781, info), \
+ INTEL_VGA_DEVICE(0xA782, info), \
+ INTEL_VGA_DEVICE(0xA783, info), \
+ INTEL_VGA_DEVICE(0xA788, info), \
+ INTEL_VGA_DEVICE(0xA789, info)
+
#endif /* _I915_PCIIDS_H */
diff --git a/include/drm/intel-gtt.h b/include/drm/intel-gtt.h
index abfefaa..67530bf 100644
--- a/include/drm/intel-gtt.h
+++ b/include/drm/intel-gtt.h
@@ -4,9 +4,11 @@
#ifndef _DRM_INTEL_GTT_H
#define _DRM_INTEL_GTT_H
-#include <linux/agp_backend.h>
-#include <linux/intel-iommu.h>
-#include <linux/kernel.h>
+#include <linux/types.h>
+
+struct agp_bridge_data;
+struct pci_dev;
+struct sg_table;
void intel_gtt_get(u64 *gtt_total,
phys_addr_t *mappable_base,
diff --git a/include/drm/ttm/ttm_bo_api.h b/include/drm/ttm/ttm_bo_api.h
index cd785cf..c17b2df 100644
--- a/include/drm/ttm/ttm_bo_api.h
+++ b/include/drm/ttm/ttm_bo_api.h
@@ -32,7 +32,6 @@
#define _TTM_BO_API_H_
#include <drm/drm_gem.h>
-#include <drm/drm_hashtab.h>
#include <drm/drm_vma_manager.h>
#include <linux/kref.h>
#include <linux/list.h>
diff --git a/include/drm/ttm/ttm_placement.h b/include/drm/ttm/ttm_placement.h
index 76d1b911..8074d0f 100644
--- a/include/drm/ttm/ttm_placement.h
+++ b/include/drm/ttm/ttm_placement.h
@@ -35,6 +35,17 @@
/*
* Memory regions for data placement.
+ *
+ * Buffers placed in TTM_PL_SYSTEM are considered under TTMs control and can
+ * be swapped out whenever TTMs thinks it is a good idea.
+ * In cases where drivers would like to use TTM_PL_SYSTEM as a valid
+ * placement they need to be able to handle the issues that arise due to the
+ * above manually.
+ *
+ * For BO's which reside in system memory but for which the accelerator
+ * requires direct access (i.e. their usage needs to be synchronized
+ * between the CPU and accelerator via fences) a new, driver private
+ * placement that can handle such scenarios is a good idea.
*/
#define TTM_PL_SYSTEM 0
diff --git a/include/dt-bindings/clock/qcom,gcc-sdx65.h b/include/dt-bindings/clock/qcom,gcc-sdx65.h
new file mode 100644
index 0000000..75ecc92
--- /dev/null
+++ b/include/dt-bindings/clock/qcom,gcc-sdx65.h
@@ -0,0 +1,122 @@
+/* SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) */
+/*
+ * Copyright (c) 2021, Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _DT_BINDINGS_CLK_QCOM_GCC_SDX65_H
+#define _DT_BINDINGS_CLK_QCOM_GCC_SDX65_H
+
+/* GCC clocks */
+#define GPLL0 0
+#define GPLL0_OUT_EVEN 1
+#define GCC_AHB_PCIE_LINK_CLK 2
+#define GCC_BLSP1_AHB_CLK 3
+#define GCC_BLSP1_QUP1_I2C_APPS_CLK 4
+#define GCC_BLSP1_QUP1_I2C_APPS_CLK_SRC 5
+#define GCC_BLSP1_QUP1_SPI_APPS_CLK 6
+#define GCC_BLSP1_QUP1_SPI_APPS_CLK_SRC 7
+#define GCC_BLSP1_QUP2_I2C_APPS_CLK 8
+#define GCC_BLSP1_QUP2_I2C_APPS_CLK_SRC 9
+#define GCC_BLSP1_QUP2_SPI_APPS_CLK 10
+#define GCC_BLSP1_QUP2_SPI_APPS_CLK_SRC 11
+#define GCC_BLSP1_QUP3_I2C_APPS_CLK 12
+#define GCC_BLSP1_QUP3_I2C_APPS_CLK_SRC 13
+#define GCC_BLSP1_QUP3_SPI_APPS_CLK 14
+#define GCC_BLSP1_QUP3_SPI_APPS_CLK_SRC 15
+#define GCC_BLSP1_QUP4_I2C_APPS_CLK 16
+#define GCC_BLSP1_QUP4_I2C_APPS_CLK_SRC 17
+#define GCC_BLSP1_QUP4_SPI_APPS_CLK 18
+#define GCC_BLSP1_QUP4_SPI_APPS_CLK_SRC 19
+#define GCC_BLSP1_SLEEP_CLK 20
+#define GCC_BLSP1_UART1_APPS_CLK 21
+#define GCC_BLSP1_UART1_APPS_CLK_SRC 22
+#define GCC_BLSP1_UART2_APPS_CLK 23
+#define GCC_BLSP1_UART2_APPS_CLK_SRC 24
+#define GCC_BLSP1_UART3_APPS_CLK 25
+#define GCC_BLSP1_UART3_APPS_CLK_SRC 26
+#define GCC_BLSP1_UART4_APPS_CLK 27
+#define GCC_BLSP1_UART4_APPS_CLK_SRC 28
+#define GCC_BOOT_ROM_AHB_CLK 29
+#define GCC_CPUSS_AHB_CLK 30
+#define GCC_CPUSS_AHB_CLK_SRC 31
+#define GCC_CPUSS_AHB_POSTDIV_CLK_SRC 32
+#define GCC_CPUSS_GNOC_CLK 33
+#define GCC_GP1_CLK 34
+#define GCC_GP1_CLK_SRC 35
+#define GCC_GP2_CLK 36
+#define GCC_GP2_CLK_SRC 37
+#define GCC_GP3_CLK 38
+#define GCC_GP3_CLK_SRC 39
+#define GCC_PCIE_0_CLKREF_EN 40
+#define GCC_PCIE_AUX_CLK 41
+#define GCC_PCIE_AUX_CLK_SRC 42
+#define GCC_PCIE_AUX_PHY_CLK_SRC 43
+#define GCC_PCIE_CFG_AHB_CLK 44
+#define GCC_PCIE_MSTR_AXI_CLK 45
+#define GCC_PCIE_PIPE_CLK 46
+#define GCC_PCIE_PIPE_CLK_SRC 47
+#define GCC_PCIE_RCHNG_PHY_CLK 48
+#define GCC_PCIE_RCHNG_PHY_CLK_SRC 49
+#define GCC_PCIE_SLEEP_CLK 50
+#define GCC_PCIE_SLV_AXI_CLK 51
+#define GCC_PCIE_SLV_Q2A_AXI_CLK 52
+#define GCC_PDM2_CLK 53
+#define GCC_PDM2_CLK_SRC 54
+#define GCC_PDM_AHB_CLK 55
+#define GCC_PDM_XO4_CLK 56
+#define GCC_RX1_USB2_CLKREF_EN 57
+#define GCC_SDCC1_AHB_CLK 58
+#define GCC_SDCC1_APPS_CLK 59
+#define GCC_SDCC1_APPS_CLK_SRC 60
+#define GCC_SPMI_FETCHER_AHB_CLK 61
+#define GCC_SPMI_FETCHER_CLK 62
+#define GCC_SPMI_FETCHER_CLK_SRC 63
+#define GCC_SYS_NOC_CPUSS_AHB_CLK 64
+#define GCC_USB30_MASTER_CLK 65
+#define GCC_USB30_MASTER_CLK_SRC 66
+#define GCC_USB30_MOCK_UTMI_CLK 67
+#define GCC_USB30_MOCK_UTMI_CLK_SRC 68
+#define GCC_USB30_MOCK_UTMI_POSTDIV_CLK_SRC 69
+#define GCC_USB30_MSTR_AXI_CLK 70
+#define GCC_USB30_SLEEP_CLK 71
+#define GCC_USB30_SLV_AHB_CLK 72
+#define GCC_USB3_PHY_AUX_CLK 73
+#define GCC_USB3_PHY_AUX_CLK_SRC 74
+#define GCC_USB3_PHY_PIPE_CLK 75
+#define GCC_USB3_PHY_PIPE_CLK_SRC 76
+#define GCC_USB3_PRIM_CLKREF_EN 77
+#define GCC_USB_PHY_CFG_AHB2PHY_CLK 78
+#define GCC_XO_DIV4_CLK 79
+#define GCC_XO_PCIE_LINK_CLK 80
+
+/* GCC resets */
+#define GCC_BLSP1_QUP1_BCR 0
+#define GCC_BLSP1_QUP2_BCR 1
+#define GCC_BLSP1_QUP3_BCR 2
+#define GCC_BLSP1_QUP4_BCR 3
+#define GCC_BLSP1_UART1_BCR 4
+#define GCC_BLSP1_UART2_BCR 5
+#define GCC_BLSP1_UART3_BCR 6
+#define GCC_BLSP1_UART4_BCR 7
+#define GCC_PCIE_BCR 8
+#define GCC_PCIE_LINK_DOWN_BCR 9
+#define GCC_PCIE_NOCSR_COM_PHY_BCR 10
+#define GCC_PCIE_PHY_BCR 11
+#define GCC_PCIE_PHY_CFG_AHB_BCR 12
+#define GCC_PCIE_PHY_COM_BCR 13
+#define GCC_PCIE_PHY_NOCSR_COM_PHY_BCR 14
+#define GCC_PDM_BCR 15
+#define GCC_QUSB2PHY_BCR 16
+#define GCC_SDCC1_BCR 17
+#define GCC_SPMI_FETCHER_BCR 18
+#define GCC_TCSR_PCIE_BCR 19
+#define GCC_USB30_BCR 20
+#define GCC_USB3_PHY_BCR 21
+#define GCC_USB3PHY_PHY_BCR 22
+#define GCC_USB_PHY_CFG_AHB2PHY_BCR 23
+
+/* GCC power domains */
+#define USB30_GDSC 0
+#define PCIE_GDSC 1
+
+#endif
diff --git a/include/dt-bindings/clock/qcom,gcc-sm8450.h b/include/dt-bindings/clock/qcom,gcc-sm8450.h
new file mode 100644
index 0000000..cf14693
--- /dev/null
+++ b/include/dt-bindings/clock/qcom,gcc-sm8450.h
@@ -0,0 +1,244 @@
+/* SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) */
+/*
+ * Copyright (c) 2020, The Linux Foundation. All rights reserved.
+ * Copyright (c) 2021, Linaro Limited
+ */
+
+#ifndef _DT_BINDINGS_CLK_QCOM_GCC_SM8450_H
+#define _DT_BINDINGS_CLK_QCOM_GCC_SM8450_H
+
+/* GCC HW clocks */
+#define CORE_BI_PLL_TEST_SE 0
+#define PCIE_0_PIPE_CLK 1
+#define PCIE_1_PHY_AUX_CLK 2
+#define PCIE_1_PIPE_CLK 3
+#define UFS_PHY_RX_SYMBOL_0_CLK 4
+#define UFS_PHY_RX_SYMBOL_1_CLK 5
+#define UFS_PHY_TX_SYMBOL_0_CLK 6
+#define USB3_PHY_WRAPPER_GCC_USB30_PIPE_CLK 7
+
+/* GCC clocks */
+#define GCC_AGGRE_NOC_PCIE_0_AXI_CLK 8
+#define GCC_AGGRE_NOC_PCIE_1_AXI_CLK 9
+#define GCC_AGGRE_UFS_PHY_AXI_CLK 10
+#define GCC_AGGRE_UFS_PHY_AXI_HW_CTL_CLK 11
+#define GCC_AGGRE_USB3_PRIM_AXI_CLK 12
+#define GCC_ANOC_PCIE_PWRCTL_CLK 13
+#define GCC_BOOT_ROM_AHB_CLK 14
+#define GCC_CAMERA_AHB_CLK 15
+#define GCC_CAMERA_HF_AXI_CLK 16
+#define GCC_CAMERA_SF_AXI_CLK 17
+#define GCC_CAMERA_XO_CLK 18
+#define GCC_CFG_NOC_PCIE_ANOC_AHB_CLK 19
+#define GCC_CFG_NOC_USB3_PRIM_AXI_CLK 20
+#define GCC_CPUSS_AHB_CLK 21
+#define GCC_CPUSS_AHB_CLK_SRC 22
+#define GCC_CPUSS_AHB_POSTDIV_CLK_SRC 23
+#define GCC_CPUSS_CONFIG_NOC_SF_CLK 24
+#define GCC_DDRSS_GPU_AXI_CLK 25
+#define GCC_DDRSS_PCIE_SF_TBU_CLK 26
+#define GCC_DISP_AHB_CLK 27
+#define GCC_DISP_HF_AXI_CLK 28
+#define GCC_DISP_SF_AXI_CLK 29
+#define GCC_DISP_XO_CLK 30
+#define GCC_EUSB3_0_CLKREF_EN 31
+#define GCC_GP1_CLK 32
+#define GCC_GP1_CLK_SRC 33
+#define GCC_GP2_CLK 34
+#define GCC_GP2_CLK_SRC 35
+#define GCC_GP3_CLK 36
+#define GCC_GP3_CLK_SRC 37
+#define GCC_GPLL0 38
+#define GCC_GPLL0_OUT_EVEN 39
+#define GCC_GPLL4 40
+#define GCC_GPLL9 41
+#define GCC_GPU_CFG_AHB_CLK 42
+#define GCC_GPU_GPLL0_CLK_SRC 43
+#define GCC_GPU_GPLL0_DIV_CLK_SRC 44
+#define GCC_GPU_MEMNOC_GFX_CLK 45
+#define GCC_GPU_SNOC_DVM_GFX_CLK 46
+#define GCC_PCIE_0_AUX_CLK 47
+#define GCC_PCIE_0_AUX_CLK_SRC 48
+#define GCC_PCIE_0_CFG_AHB_CLK 49
+#define GCC_PCIE_0_CLKREF_EN 50
+#define GCC_PCIE_0_MSTR_AXI_CLK 51
+#define GCC_PCIE_0_PHY_RCHNG_CLK 52
+#define GCC_PCIE_0_PHY_RCHNG_CLK_SRC 53
+#define GCC_PCIE_0_PIPE_CLK 54
+#define GCC_PCIE_0_PIPE_CLK_SRC 55
+#define GCC_PCIE_0_SLV_AXI_CLK 56
+#define GCC_PCIE_0_SLV_Q2A_AXI_CLK 57
+#define GCC_PCIE_1_AUX_CLK 58
+#define GCC_PCIE_1_AUX_CLK_SRC 59
+#define GCC_PCIE_1_CFG_AHB_CLK 60
+#define GCC_PCIE_1_CLKREF_EN 61
+#define GCC_PCIE_1_MSTR_AXI_CLK 62
+#define GCC_PCIE_1_PHY_AUX_CLK 63
+#define GCC_PCIE_1_PHY_AUX_CLK_SRC 64
+#define GCC_PCIE_1_PHY_RCHNG_CLK 65
+#define GCC_PCIE_1_PHY_RCHNG_CLK_SRC 66
+#define GCC_PCIE_1_PIPE_CLK 67
+#define GCC_PCIE_1_PIPE_CLK_SRC 68
+#define GCC_PCIE_1_SLV_AXI_CLK 69
+#define GCC_PCIE_1_SLV_Q2A_AXI_CLK 70
+#define GCC_PDM2_CLK 71
+#define GCC_PDM2_CLK_SRC 72
+#define GCC_PDM_AHB_CLK 73
+#define GCC_PDM_XO4_CLK 74
+#define GCC_QMIP_CAMERA_NRT_AHB_CLK 75
+#define GCC_QMIP_CAMERA_RT_AHB_CLK 76
+#define GCC_QMIP_DISP_AHB_CLK 77
+#define GCC_QMIP_GPU_AHB_CLK 78
+#define GCC_QMIP_PCIE_AHB_CLK 79
+#define GCC_QMIP_VIDEO_CV_CPU_AHB_CLK 80
+#define GCC_QMIP_VIDEO_CVP_AHB_CLK 81
+#define GCC_QMIP_VIDEO_V_CPU_AHB_CLK 82
+#define GCC_QMIP_VIDEO_VCODEC_AHB_CLK 83
+#define GCC_QUPV3_WRAP0_CORE_2X_CLK 84
+#define GCC_QUPV3_WRAP0_CORE_CLK 85
+#define GCC_QUPV3_WRAP0_S0_CLK 86
+#define GCC_QUPV3_WRAP0_S0_CLK_SRC 87
+#define GCC_QUPV3_WRAP0_S1_CLK 88
+#define GCC_QUPV3_WRAP0_S1_CLK_SRC 89
+#define GCC_QUPV3_WRAP0_S2_CLK 90
+#define GCC_QUPV3_WRAP0_S2_CLK_SRC 91
+#define GCC_QUPV3_WRAP0_S3_CLK 92
+#define GCC_QUPV3_WRAP0_S3_CLK_SRC 93
+#define GCC_QUPV3_WRAP0_S4_CLK 94
+#define GCC_QUPV3_WRAP0_S4_CLK_SRC 95
+#define GCC_QUPV3_WRAP0_S5_CLK 96
+#define GCC_QUPV3_WRAP0_S5_CLK_SRC 97
+#define GCC_QUPV3_WRAP0_S6_CLK 98
+#define GCC_QUPV3_WRAP0_S6_CLK_SRC 99
+#define GCC_QUPV3_WRAP0_S7_CLK 100
+#define GCC_QUPV3_WRAP0_S7_CLK_SRC 101
+#define GCC_QUPV3_WRAP1_CORE_2X_CLK 102
+#define GCC_QUPV3_WRAP1_CORE_CLK 103
+#define GCC_QUPV3_WRAP1_S0_CLK 104
+#define GCC_QUPV3_WRAP1_S0_CLK_SRC 105
+#define GCC_QUPV3_WRAP1_S1_CLK 106
+#define GCC_QUPV3_WRAP1_S1_CLK_SRC 107
+#define GCC_QUPV3_WRAP1_S2_CLK 108
+#define GCC_QUPV3_WRAP1_S2_CLK_SRC 109
+#define GCC_QUPV3_WRAP1_S3_CLK 110
+#define GCC_QUPV3_WRAP1_S3_CLK_SRC 111
+#define GCC_QUPV3_WRAP1_S4_CLK 112
+#define GCC_QUPV3_WRAP1_S4_CLK_SRC 113
+#define GCC_QUPV3_WRAP1_S5_CLK 114
+#define GCC_QUPV3_WRAP1_S5_CLK_SRC 115
+#define GCC_QUPV3_WRAP1_S6_CLK 116
+#define GCC_QUPV3_WRAP1_S6_CLK_SRC 117
+#define GCC_QUPV3_WRAP2_CORE_2X_CLK 118
+#define GCC_QUPV3_WRAP2_CORE_CLK 119
+#define GCC_QUPV3_WRAP2_S0_CLK 120
+#define GCC_QUPV3_WRAP2_S0_CLK_SRC 121
+#define GCC_QUPV3_WRAP2_S1_CLK 122
+#define GCC_QUPV3_WRAP2_S1_CLK_SRC 123
+#define GCC_QUPV3_WRAP2_S2_CLK 124
+#define GCC_QUPV3_WRAP2_S2_CLK_SRC 125
+#define GCC_QUPV3_WRAP2_S3_CLK 126
+#define GCC_QUPV3_WRAP2_S3_CLK_SRC 127
+#define GCC_QUPV3_WRAP2_S4_CLK 128
+#define GCC_QUPV3_WRAP2_S4_CLK_SRC 129
+#define GCC_QUPV3_WRAP2_S5_CLK 130
+#define GCC_QUPV3_WRAP2_S5_CLK_SRC 131
+#define GCC_QUPV3_WRAP2_S6_CLK 132
+#define GCC_QUPV3_WRAP2_S6_CLK_SRC 133
+#define GCC_QUPV3_WRAP_0_M_AHB_CLK 134
+#define GCC_QUPV3_WRAP_0_S_AHB_CLK 135
+#define GCC_QUPV3_WRAP_1_M_AHB_CLK 136
+#define GCC_QUPV3_WRAP_1_S_AHB_CLK 137
+#define GCC_QUPV3_WRAP_2_M_AHB_CLK 138
+#define GCC_QUPV3_WRAP_2_S_AHB_CLK 139
+#define GCC_SDCC2_AHB_CLK 140
+#define GCC_SDCC2_APPS_CLK 141
+#define GCC_SDCC2_APPS_CLK_SRC 142
+#define GCC_SDCC2_AT_CLK 143
+#define GCC_SDCC4_AHB_CLK 144
+#define GCC_SDCC4_APPS_CLK 145
+#define GCC_SDCC4_APPS_CLK_SRC 146
+#define GCC_SDCC4_AT_CLK 147
+#define GCC_SYS_NOC_CPUSS_AHB_CLK 148
+#define GCC_UFS_0_CLKREF_EN 149
+#define GCC_UFS_PHY_AHB_CLK 150
+#define GCC_UFS_PHY_AXI_CLK 151
+#define GCC_UFS_PHY_AXI_CLK_SRC 152
+#define GCC_UFS_PHY_AXI_HW_CTL_CLK 153
+#define GCC_UFS_PHY_ICE_CORE_CLK 154
+#define GCC_UFS_PHY_ICE_CORE_CLK_SRC 155
+#define GCC_UFS_PHY_ICE_CORE_HW_CTL_CLK 156
+#define GCC_UFS_PHY_PHY_AUX_CLK 157
+#define GCC_UFS_PHY_PHY_AUX_CLK_SRC 158
+#define GCC_UFS_PHY_PHY_AUX_HW_CTL_CLK 159
+#define GCC_UFS_PHY_RX_SYMBOL_0_CLK 160
+#define GCC_UFS_PHY_RX_SYMBOL_0_CLK_SRC 161
+#define GCC_UFS_PHY_RX_SYMBOL_1_CLK 162
+#define GCC_UFS_PHY_RX_SYMBOL_1_CLK_SRC 163
+#define GCC_UFS_PHY_TX_SYMBOL_0_CLK 164
+#define GCC_UFS_PHY_TX_SYMBOL_0_CLK_SRC 165
+#define GCC_UFS_PHY_UNIPRO_CORE_CLK 166
+#define GCC_UFS_PHY_UNIPRO_CORE_CLK_SRC 167
+#define GCC_UFS_PHY_UNIPRO_CORE_HW_CTL_CLK 168
+#define GCC_USB30_PRIM_MASTER_CLK 169
+#define GCC_USB30_PRIM_MASTER_CLK_SRC 170
+#define GCC_USB30_PRIM_MOCK_UTMI_CLK 171
+#define GCC_USB30_PRIM_MOCK_UTMI_CLK_SRC 172
+#define GCC_USB30_PRIM_MOCK_UTMI_POSTDIV_CLK_SRC 173
+#define GCC_USB30_PRIM_SLEEP_CLK 174
+#define GCC_USB3_0_CLKREF_EN 175
+#define GCC_USB3_PRIM_PHY_AUX_CLK 176
+#define GCC_USB3_PRIM_PHY_AUX_CLK_SRC 177
+#define GCC_USB3_PRIM_PHY_COM_AUX_CLK 178
+#define GCC_USB3_PRIM_PHY_PIPE_CLK 179
+#define GCC_USB3_PRIM_PHY_PIPE_CLK_SRC 180
+#define GCC_VIDEO_AHB_CLK 181
+#define GCC_VIDEO_AXI0_CLK 182
+#define GCC_VIDEO_AXI1_CLK 183
+#define GCC_VIDEO_XO_CLK 184
+
+/* GCC resets */
+#define GCC_CAMERA_BCR 0
+#define GCC_DISPLAY_BCR 1
+#define GCC_GPU_BCR 2
+#define GCC_MMSS_BCR 3
+#define GCC_PCIE_0_BCR 4
+#define GCC_PCIE_0_LINK_DOWN_BCR 5
+#define GCC_PCIE_0_NOCSR_COM_PHY_BCR 6
+#define GCC_PCIE_0_PHY_BCR 7
+#define GCC_PCIE_0_PHY_NOCSR_COM_PHY_BCR 8
+#define GCC_PCIE_1_BCR 9
+#define GCC_PCIE_1_LINK_DOWN_BCR 10
+#define GCC_PCIE_1_NOCSR_COM_PHY_BCR 11
+#define GCC_PCIE_1_PHY_BCR 12
+#define GCC_PCIE_1_PHY_NOCSR_COM_PHY_BCR 13
+#define GCC_PCIE_PHY_BCR 14
+#define GCC_PCIE_PHY_CFG_AHB_BCR 15
+#define GCC_PCIE_PHY_COM_BCR 16
+#define GCC_PDM_BCR 17
+#define GCC_QUPV3_WRAPPER_0_BCR 18
+#define GCC_QUPV3_WRAPPER_1_BCR 19
+#define GCC_QUPV3_WRAPPER_2_BCR 20
+#define GCC_QUSB2PHY_PRIM_BCR 21
+#define GCC_QUSB2PHY_SEC_BCR 22
+#define GCC_SDCC2_BCR 23
+#define GCC_SDCC4_BCR 24
+#define GCC_UFS_PHY_BCR 25
+#define GCC_USB30_PRIM_BCR 26
+#define GCC_USB3_DP_PHY_PRIM_BCR 27
+#define GCC_USB3_DP_PHY_SEC_BCR 28
+#define GCC_USB3_PHY_PRIM_BCR 29
+#define GCC_USB3_PHY_SEC_BCR 30
+#define GCC_USB3PHY_PHY_PRIM_BCR 31
+#define GCC_USB3PHY_PHY_SEC_BCR 32
+#define GCC_USB_PHY_CFG_AHB2PHY_BCR 33
+#define GCC_VIDEO_AXI0_CLK_ARES 34
+#define GCC_VIDEO_AXI1_CLK_ARES 35
+#define GCC_VIDEO_BCR 36
+
+/* GCC power domains */
+#define PCIE_0_GDSC 0
+#define PCIE_1_GDSC 1
+#define UFS_PHY_GDSC 2
+#define USB30_PRIM_GDSC 3
+
+#endif
diff --git a/include/dt-bindings/clock/r8a779f0-cpg-mssr.h b/include/dt-bindings/clock/r8a779f0-cpg-mssr.h
new file mode 100644
index 0000000..f2ae1c6a
--- /dev/null
+++ b/include/dt-bindings/clock/r8a779f0-cpg-mssr.h
@@ -0,0 +1,64 @@
+/* SPDX-License-Identifier: (GPL-2.0 or MIT) */
+/*
+ * Copyright (C) 2021 Renesas Electronics Corp.
+ */
+#ifndef __DT_BINDINGS_CLOCK_R8A779F0_CPG_MSSR_H__
+#define __DT_BINDINGS_CLOCK_R8A779F0_CPG_MSSR_H__
+
+#include <dt-bindings/clock/renesas-cpg-mssr.h>
+
+/* r8a779f0 CPG Core Clocks */
+
+#define R8A779F0_CLK_ZX 0
+#define R8A779F0_CLK_ZS 1
+#define R8A779F0_CLK_ZT 2
+#define R8A779F0_CLK_ZTR 3
+#define R8A779F0_CLK_S0D2 4
+#define R8A779F0_CLK_S0D3 5
+#define R8A779F0_CLK_S0D4 6
+#define R8A779F0_CLK_S0D2_MM 7
+#define R8A779F0_CLK_S0D3_MM 8
+#define R8A779F0_CLK_S0D4_MM 9
+#define R8A779F0_CLK_S0D2_RT 10
+#define R8A779F0_CLK_S0D3_RT 11
+#define R8A779F0_CLK_S0D4_RT 12
+#define R8A779F0_CLK_S0D6_RT 13
+#define R8A779F0_CLK_S0D3_PER 14
+#define R8A779F0_CLK_S0D6_PER 15
+#define R8A779F0_CLK_S0D12_PER 16
+#define R8A779F0_CLK_S0D24_PER 17
+#define R8A779F0_CLK_S0D2_HSC 18
+#define R8A779F0_CLK_S0D3_HSC 19
+#define R8A779F0_CLK_S0D4_HSC 20
+#define R8A779F0_CLK_S0D6_HSC 21
+#define R8A779F0_CLK_S0D12_HSC 22
+#define R8A779F0_CLK_S0D2_CC 23
+#define R8A779F0_CLK_CL 24
+#define R8A779F0_CLK_CL16M 25
+#define R8A779F0_CLK_CL16M_MM 26
+#define R8A779F0_CLK_CL16M_RT 27
+#define R8A779F0_CLK_CL16M_PER 28
+#define R8A779F0_CLK_CL16M_HSC 29
+#define R8A779F0_CLK_Z0 30
+#define R8A779F0_CLK_Z1 31
+#define R8A779F0_CLK_ZB3 32
+#define R8A779F0_CLK_ZB3D2 33
+#define R8A779F0_CLK_ZB3D4 34
+#define R8A779F0_CLK_SD0H 35
+#define R8A779F0_CLK_SD0 36
+#define R8A779F0_CLK_RPC 37
+#define R8A779F0_CLK_RPCD2 38
+#define R8A779F0_CLK_MSO 39
+#define R8A779F0_CLK_SASYNCRT 40
+#define R8A779F0_CLK_SASYNCPERD1 41
+#define R8A779F0_CLK_SASYNCPERD2 42
+#define R8A779F0_CLK_SASYNCPERD4 43
+#define R8A779F0_CLK_DBGSOC_HSC 44
+#define R8A779F0_CLK_RSW2 45
+#define R8A779F0_CLK_OSC 46
+#define R8A779F0_CLK_ZR 47
+#define R8A779F0_CLK_CPEX 48
+#define R8A779F0_CLK_CBFUSA 49
+#define R8A779F0_CLK_R 50
+
+#endif /* __DT_BINDINGS_CLOCK_R8A779F0_CPG_MSSR_H__ */
diff --git a/include/dt-bindings/clock/starfive-jh7100.h b/include/dt-bindings/clock/starfive-jh7100.h
new file mode 100644
index 0000000..aa0863b
--- /dev/null
+++ b/include/dt-bindings/clock/starfive-jh7100.h
@@ -0,0 +1,202 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/*
+ * Copyright (C) 2021 Ahmad Fatoum, Pengutronix
+ */
+
+#ifndef __DT_BINDINGS_CLOCK_STARFIVE_JH7100_H__
+#define __DT_BINDINGS_CLOCK_STARFIVE_JH7100_H__
+
+#define JH7100_CLK_CPUNDBUS_ROOT 0
+#define JH7100_CLK_DLA_ROOT 1
+#define JH7100_CLK_DSP_ROOT 2
+#define JH7100_CLK_GMACUSB_ROOT 3
+#define JH7100_CLK_PERH0_ROOT 4
+#define JH7100_CLK_PERH1_ROOT 5
+#define JH7100_CLK_VIN_ROOT 6
+#define JH7100_CLK_VOUT_ROOT 7
+#define JH7100_CLK_AUDIO_ROOT 8
+#define JH7100_CLK_CDECHIFI4_ROOT 9
+#define JH7100_CLK_CDEC_ROOT 10
+#define JH7100_CLK_VOUTBUS_ROOT 11
+#define JH7100_CLK_CPUNBUS_ROOT_DIV 12
+#define JH7100_CLK_DSP_ROOT_DIV 13
+#define JH7100_CLK_PERH0_SRC 14
+#define JH7100_CLK_PERH1_SRC 15
+#define JH7100_CLK_PLL0_TESTOUT 16
+#define JH7100_CLK_PLL1_TESTOUT 17
+#define JH7100_CLK_PLL2_TESTOUT 18
+#define JH7100_CLK_PLL2_REF 19
+#define JH7100_CLK_CPU_CORE 20
+#define JH7100_CLK_CPU_AXI 21
+#define JH7100_CLK_AHB_BUS 22
+#define JH7100_CLK_APB1_BUS 23
+#define JH7100_CLK_APB2_BUS 24
+#define JH7100_CLK_DOM3AHB_BUS 25
+#define JH7100_CLK_DOM7AHB_BUS 26
+#define JH7100_CLK_U74_CORE0 27
+#define JH7100_CLK_U74_CORE1 28
+#define JH7100_CLK_U74_AXI 29
+#define JH7100_CLK_U74RTC_TOGGLE 30
+#define JH7100_CLK_SGDMA2P_AXI 31
+#define JH7100_CLK_DMA2PNOC_AXI 32
+#define JH7100_CLK_SGDMA2P_AHB 33
+#define JH7100_CLK_DLA_BUS 34
+#define JH7100_CLK_DLA_AXI 35
+#define JH7100_CLK_DLANOC_AXI 36
+#define JH7100_CLK_DLA_APB 37
+#define JH7100_CLK_VP6_CORE 38
+#define JH7100_CLK_VP6BUS_SRC 39
+#define JH7100_CLK_VP6_AXI 40
+#define JH7100_CLK_VCDECBUS_SRC 41
+#define JH7100_CLK_VDEC_BUS 42
+#define JH7100_CLK_VDEC_AXI 43
+#define JH7100_CLK_VDECBRG_MAIN 44
+#define JH7100_CLK_VDEC_BCLK 45
+#define JH7100_CLK_VDEC_CCLK 46
+#define JH7100_CLK_VDEC_APB 47
+#define JH7100_CLK_JPEG_AXI 48
+#define JH7100_CLK_JPEG_CCLK 49
+#define JH7100_CLK_JPEG_APB 50
+#define JH7100_CLK_GC300_2X 51
+#define JH7100_CLK_GC300_AHB 52
+#define JH7100_CLK_JPCGC300_AXIBUS 53
+#define JH7100_CLK_GC300_AXI 54
+#define JH7100_CLK_JPCGC300_MAIN 55
+#define JH7100_CLK_VENC_BUS 56
+#define JH7100_CLK_VENC_AXI 57
+#define JH7100_CLK_VENCBRG_MAIN 58
+#define JH7100_CLK_VENC_BCLK 59
+#define JH7100_CLK_VENC_CCLK 60
+#define JH7100_CLK_VENC_APB 61
+#define JH7100_CLK_DDRPLL_DIV2 62
+#define JH7100_CLK_DDRPLL_DIV4 63
+#define JH7100_CLK_DDRPLL_DIV8 64
+#define JH7100_CLK_DDROSC_DIV2 65
+#define JH7100_CLK_DDRC0 66
+#define JH7100_CLK_DDRC1 67
+#define JH7100_CLK_DDRPHY_APB 68
+#define JH7100_CLK_NOC_ROB 69
+#define JH7100_CLK_NOC_COG 70
+#define JH7100_CLK_NNE_AHB 71
+#define JH7100_CLK_NNEBUS_SRC1 72
+#define JH7100_CLK_NNE_BUS 73
+#define JH7100_CLK_NNE_AXI 74
+#define JH7100_CLK_NNENOC_AXI 75
+#define JH7100_CLK_DLASLV_AXI 76
+#define JH7100_CLK_DSPX2C_AXI 77
+#define JH7100_CLK_HIFI4_SRC 78
+#define JH7100_CLK_HIFI4_COREFREE 79
+#define JH7100_CLK_HIFI4_CORE 80
+#define JH7100_CLK_HIFI4_BUS 81
+#define JH7100_CLK_HIFI4_AXI 82
+#define JH7100_CLK_HIFI4NOC_AXI 83
+#define JH7100_CLK_SGDMA1P_BUS 84
+#define JH7100_CLK_SGDMA1P_AXI 85
+#define JH7100_CLK_DMA1P_AXI 86
+#define JH7100_CLK_X2C_AXI 87
+#define JH7100_CLK_USB_BUS 88
+#define JH7100_CLK_USB_AXI 89
+#define JH7100_CLK_USBNOC_AXI 90
+#define JH7100_CLK_USBPHY_ROOTDIV 91
+#define JH7100_CLK_USBPHY_125M 92
+#define JH7100_CLK_USBPHY_PLLDIV25M 93
+#define JH7100_CLK_USBPHY_25M 94
+#define JH7100_CLK_AUDIO_DIV 95
+#define JH7100_CLK_AUDIO_SRC 96
+#define JH7100_CLK_AUDIO_12288 97
+#define JH7100_CLK_VIN_SRC 98
+#define JH7100_CLK_ISP0_BUS 99
+#define JH7100_CLK_ISP0_AXI 100
+#define JH7100_CLK_ISP0NOC_AXI 101
+#define JH7100_CLK_ISPSLV_AXI 102
+#define JH7100_CLK_ISP1_BUS 103
+#define JH7100_CLK_ISP1_AXI 104
+#define JH7100_CLK_ISP1NOC_AXI 105
+#define JH7100_CLK_VIN_BUS 106
+#define JH7100_CLK_VIN_AXI 107
+#define JH7100_CLK_VINNOC_AXI 108
+#define JH7100_CLK_VOUT_SRC 109
+#define JH7100_CLK_DISPBUS_SRC 110
+#define JH7100_CLK_DISP_BUS 111
+#define JH7100_CLK_DISP_AXI 112
+#define JH7100_CLK_DISPNOC_AXI 113
+#define JH7100_CLK_SDIO0_AHB 114
+#define JH7100_CLK_SDIO0_CCLKINT 115
+#define JH7100_CLK_SDIO0_CCLKINT_INV 116
+#define JH7100_CLK_SDIO1_AHB 117
+#define JH7100_CLK_SDIO1_CCLKINT 118
+#define JH7100_CLK_SDIO1_CCLKINT_INV 119
+#define JH7100_CLK_GMAC_AHB 120
+#define JH7100_CLK_GMAC_ROOT_DIV 121
+#define JH7100_CLK_GMAC_PTP_REF 122
+#define JH7100_CLK_GMAC_GTX 123
+#define JH7100_CLK_GMAC_RMII_TX 124
+#define JH7100_CLK_GMAC_RMII_RX 125
+#define JH7100_CLK_GMAC_TX 126
+#define JH7100_CLK_GMAC_TX_INV 127
+#define JH7100_CLK_GMAC_RX_PRE 128
+#define JH7100_CLK_GMAC_RX_INV 129
+#define JH7100_CLK_GMAC_RMII 130
+#define JH7100_CLK_GMAC_TOPHYREF 131
+#define JH7100_CLK_SPI2AHB_AHB 132
+#define JH7100_CLK_SPI2AHB_CORE 133
+#define JH7100_CLK_EZMASTER_AHB 134
+#define JH7100_CLK_E24_AHB 135
+#define JH7100_CLK_E24RTC_TOGGLE 136
+#define JH7100_CLK_QSPI_AHB 137
+#define JH7100_CLK_QSPI_APB 138
+#define JH7100_CLK_QSPI_REF 139
+#define JH7100_CLK_SEC_AHB 140
+#define JH7100_CLK_AES 141
+#define JH7100_CLK_SHA 142
+#define JH7100_CLK_PKA 143
+#define JH7100_CLK_TRNG_APB 144
+#define JH7100_CLK_OTP_APB 145
+#define JH7100_CLK_UART0_APB 146
+#define JH7100_CLK_UART0_CORE 147
+#define JH7100_CLK_UART1_APB 148
+#define JH7100_CLK_UART1_CORE 149
+#define JH7100_CLK_SPI0_APB 150
+#define JH7100_CLK_SPI0_CORE 151
+#define JH7100_CLK_SPI1_APB 152
+#define JH7100_CLK_SPI1_CORE 153
+#define JH7100_CLK_I2C0_APB 154
+#define JH7100_CLK_I2C0_CORE 155
+#define JH7100_CLK_I2C1_APB 156
+#define JH7100_CLK_I2C1_CORE 157
+#define JH7100_CLK_GPIO_APB 158
+#define JH7100_CLK_UART2_APB 159
+#define JH7100_CLK_UART2_CORE 160
+#define JH7100_CLK_UART3_APB 161
+#define JH7100_CLK_UART3_CORE 162
+#define JH7100_CLK_SPI2_APB 163
+#define JH7100_CLK_SPI2_CORE 164
+#define JH7100_CLK_SPI3_APB 165
+#define JH7100_CLK_SPI3_CORE 166
+#define JH7100_CLK_I2C2_APB 167
+#define JH7100_CLK_I2C2_CORE 168
+#define JH7100_CLK_I2C3_APB 169
+#define JH7100_CLK_I2C3_CORE 170
+#define JH7100_CLK_WDTIMER_APB 171
+#define JH7100_CLK_WDT_CORE 172
+#define JH7100_CLK_TIMER0_CORE 173
+#define JH7100_CLK_TIMER1_CORE 174
+#define JH7100_CLK_TIMER2_CORE 175
+#define JH7100_CLK_TIMER3_CORE 176
+#define JH7100_CLK_TIMER4_CORE 177
+#define JH7100_CLK_TIMER5_CORE 178
+#define JH7100_CLK_TIMER6_CORE 179
+#define JH7100_CLK_VP6INTC_APB 180
+#define JH7100_CLK_PWM_APB 181
+#define JH7100_CLK_MSI_APB 182
+#define JH7100_CLK_TEMP_APB 183
+#define JH7100_CLK_TEMP_SENSE 184
+#define JH7100_CLK_SYSERR_APB 185
+
+#define JH7100_CLK_PLL0_OUT 186
+#define JH7100_CLK_PLL1_OUT 187
+#define JH7100_CLK_PLL2_OUT 188
+
+#define JH7100_CLK_END 189
+
+#endif /* __DT_BINDINGS_CLOCK_STARFIVE_JH7100_H__ */
diff --git a/include/dt-bindings/clock/sun50i-a64-ccu.h b/include/dt-bindings/clock/sun50i-a64-ccu.h
index 318eb15..1758921 100644
--- a/include/dt-bindings/clock/sun50i-a64-ccu.h
+++ b/include/dt-bindings/clock/sun50i-a64-ccu.h
@@ -113,7 +113,7 @@
#define CLK_USB_OHCI0 91
#define CLK_USB_OHCI1 93
-
+#define CLK_DRAM 94
#define CLK_DRAM_VE 95
#define CLK_DRAM_CSI 96
#define CLK_DRAM_DEINTERLACE 97
diff --git a/include/dt-bindings/clock/sun8i-h3-ccu.h b/include/dt-bindings/clock/sun8i-h3-ccu.h
index 30d2d15..5d4ada2 100644
--- a/include/dt-bindings/clock/sun8i-h3-ccu.h
+++ b/include/dt-bindings/clock/sun8i-h3-ccu.h
@@ -126,7 +126,7 @@
#define CLK_USB_OHCI1 93
#define CLK_USB_OHCI2 94
#define CLK_USB_OHCI3 95
-
+#define CLK_DRAM 96
#define CLK_DRAM_VE 97
#define CLK_DRAM_CSI 98
#define CLK_DRAM_DEINTERLACE 99
diff --git a/include/dt-bindings/clock/tegra234-clock.h b/include/dt-bindings/clock/tegra234-clock.h
index 2c82072..8d7e66e 100644
--- a/include/dt-bindings/clock/tegra234-clock.h
+++ b/include/dt-bindings/clock/tegra234-clock.h
@@ -4,11 +4,31 @@
#ifndef DT_BINDINGS_CLOCK_TEGRA234_CLOCK_H
#define DT_BINDINGS_CLOCK_TEGRA234_CLOCK_H
+/**
+ * @file
+ * @defgroup bpmp_clock_ids Clock ID's
+ * @{
+ */
+/**
+ * @brief controls the EMC clock frequency.
+ * @details Doing a clk_set_rate on this clock will select the
+ * appropriate clock source, program the source rate and execute a
+ * specific sequence to switch to the new clock source for both memory
+ * controllers. This can be used to control the balance between memory
+ * throughput and memory controller power.
+ */
+#define TEGRA234_CLK_EMC 31U
/** @brief output of gate CLK_ENB_FUSE */
-#define TEGRA234_CLK_FUSE 40
+#define TEGRA234_CLK_FUSE 40U
/** @brief output of mux controlled by CLK_RST_CONTROLLER_CLK_SOURCE_SDMMC4 */
-#define TEGRA234_CLK_SDMMC4 123
+#define TEGRA234_CLK_SDMMC4 123U
/** @brief output of mux controlled by CLK_RST_CONTROLLER_CLK_SOURCE_UARTA */
-#define TEGRA234_CLK_UARTA 155
+#define TEGRA234_CLK_UARTA 155U
+/** @brief CLK_RST_CONTROLLER_CLK_SOURCE_SDMMC_LEGACY_TM switch divider output */
+#define TEGRA234_CLK_SDMMC_LEGACY_TM 219U
+/** @brief PLL controlled by CLK_RST_CONTROLLER_PLLC4_BASE */
+#define TEGRA234_CLK_PLLC4 237U
+/** @brief 32K input clock provided by PMIC */
+#define TEGRA234_CLK_CLK_32K 289U
#endif
diff --git a/include/dt-bindings/memory/tegra234-mc.h b/include/dt-bindings/memory/tegra234-mc.h
new file mode 100644
index 0000000..2662f70
--- /dev/null
+++ b/include/dt-bindings/memory/tegra234-mc.h
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: (GPL-2.0 OR MIT) */
+
+#ifndef DT_BINDINGS_MEMORY_TEGRA234_MC_H
+#define DT_BINDINGS_MEMORY_TEGRA234_MC_H
+
+/* special clients */
+#define TEGRA234_SID_INVALID 0x00
+#define TEGRA234_SID_PASSTHROUGH 0x7f
+
+
+/* NISO1 stream IDs */
+#define TEGRA234_SID_SDMMC4 0x02
+#define TEGRA234_SID_BPMP 0x10
+
+/*
+ * memory client IDs
+ */
+
+/* sdmmcd memory read client */
+#define TEGRA234_MEMORY_CLIENT_SDMMCRAB 0x63
+/* sdmmcd memory write client */
+#define TEGRA234_MEMORY_CLIENT_SDMMCWAB 0x67
+/* BPMP read client */
+#define TEGRA234_MEMORY_CLIENT_BPMPR 0x93
+/* BPMP write client */
+#define TEGRA234_MEMORY_CLIENT_BPMPW 0x94
+/* BPMPDMA read client */
+#define TEGRA234_MEMORY_CLIENT_BPMPDMAR 0x95
+/* BPMPDMA write client */
+#define TEGRA234_MEMORY_CLIENT_BPMPDMAW 0x96
+
+#endif
diff --git a/include/dt-bindings/pinctrl/k3.h b/include/dt-bindings/pinctrl/k3.h
index e085f10..63e038e 100644
--- a/include/dt-bindings/pinctrl/k3.h
+++ b/include/dt-bindings/pinctrl/k3.h
@@ -38,4 +38,7 @@
#define AM64X_IOPAD(pa, val, muxmode) (((pa) & 0x1fff)) ((val) | (muxmode))
#define AM64X_MCU_IOPAD(pa, val, muxmode) (((pa) & 0x1fff)) ((val) | (muxmode))
+#define J721S2_IOPAD(pa, val, muxmode) (((pa) & 0x1fff)) ((val) | (muxmode))
+#define J721S2_WKUP_IOPAD(pa, val, muxmode) (((pa) & 0x1fff)) ((val) | (muxmode))
+
#endif
diff --git a/include/dt-bindings/pinctrl/pinctrl-starfive.h b/include/dt-bindings/pinctrl/pinctrl-starfive.h
new file mode 100644
index 0000000..de4f75c
--- /dev/null
+++ b/include/dt-bindings/pinctrl/pinctrl-starfive.h
@@ -0,0 +1,275 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/*
+ * Copyright (C) 2021 Emil Renner Berthing <kernel@esmil.dk>
+ */
+
+#ifndef __DT_BINDINGS_PINCTRL_STARFIVE_H__
+#define __DT_BINDINGS_PINCTRL_STARFIVE_H__
+
+#define PAD_GPIO_OFFSET 0
+#define PAD_FUNC_SHARE_OFFSET 64
+#define PAD_GPIO(x) (PAD_GPIO_OFFSET + (x))
+#define PAD_FUNC_SHARE(x) (PAD_FUNC_SHARE_OFFSET + (x))
+
+/*
+ * GPIOMUX bits:
+ * | 31 - 24 | 23 - 16 | 15 - 8 | 7 | 6 | 5 - 0 |
+ * | dout | doen | din | dout rev | doen rev | gpio nr |
+ *
+ * dout: output signal
+ * doen: output enable signal
+ * din: optional input signal, 0xff = none
+ * dout rev: output signal reverse bit
+ * doen rev: output enable signal reverse bit
+ * gpio nr: gpio number, 0 - 63
+ */
+#define GPIOMUX(n, dout, doen, din) ( \
+ (((dout) & 0x80000000) >> (31 - 7)) | (((dout) & 0xff) << 24) | \
+ (((doen) & 0x80000000) >> (31 - 6)) | (((doen) & 0xff) << 16) | \
+ (((din) & 0xff) << 8) | \
+ ((n) & 0x3f))
+
+#define GPO_REVERSE 0x80000000
+
+#define GPO_LOW 0
+#define GPO_HIGH 1
+#define GPO_ENABLE 0
+#define GPO_DISABLE 1
+#define GPO_CLK_GMAC_PAPHYREF 2
+#define GPO_JTAG_TDO 3
+#define GPO_JTAG_TDO_OEN 4
+#define GPO_DMIC_CLK_OUT 5
+#define GPO_DSP_JTDOEN_PAD 6
+#define GPO_DSP_JTDO_PAD 7
+#define GPO_I2C0_PAD_SCK_OE 8
+#define GPO_I2C0_PAD_SCK_OEN (GPO_I2C0_PAD_SCK_OE | GPO_REVERSE)
+#define GPO_I2C0_PAD_SDA_OE 9
+#define GPO_I2C0_PAD_SDA_OEN (GPO_I2C0_PAD_SDA_OE | GPO_REVERSE)
+#define GPO_I2C1_PAD_SCK_OE 10
+#define GPO_I2C1_PAD_SCK_OEN (GPO_I2C1_PAD_SCK_OE | GPO_REVERSE)
+#define GPO_I2C1_PAD_SDA_OE 11
+#define GPO_I2C1_PAD_SDA_OEN (GPO_I2C1_PAD_SDA_OE | GPO_REVERSE)
+#define GPO_I2C2_PAD_SCK_OE 12
+#define GPO_I2C2_PAD_SCK_OEN (GPO_I2C2_PAD_SCK_OE | GPO_REVERSE)
+#define GPO_I2C2_PAD_SDA_OE 13
+#define GPO_I2C2_PAD_SDA_OEN (GPO_I2C2_PAD_SDA_OE | GPO_REVERSE)
+#define GPO_I2C3_PAD_SCK_OE 14
+#define GPO_I2C3_PAD_SCK_OEN (GPO_I2C3_PAD_SCK_OE | GPO_REVERSE)
+#define GPO_I2C3_PAD_SDA_OE 15
+#define GPO_I2C3_PAD_SDA_OEN (GPO_I2C3_PAD_SDA_OE | GPO_REVERSE)
+#define GPO_I2SRX_BCLK_OUT 16
+#define GPO_I2SRX_BCLK_OUT_OEN 17
+#define GPO_I2SRX_LRCK_OUT 18
+#define GPO_I2SRX_LRCK_OUT_OEN 19
+#define GPO_I2SRX_MCLK_OUT 20
+#define GPO_I2STX_BCLK_OUT 21
+#define GPO_I2STX_BCLK_OUT_OEN 22
+#define GPO_I2STX_LRCK_OUT 23
+#define GPO_I2STX_LRCK_OUT_OEN 24
+#define GPO_I2STX_MCLK_OUT 25
+#define GPO_I2STX_SDOUT0 26
+#define GPO_I2STX_SDOUT1 27
+#define GPO_LCD_PAD_CSM_N 28
+#define GPO_PWM_PAD_OE_N_BIT0 29
+#define GPO_PWM_PAD_OE_N_BIT1 30
+#define GPO_PWM_PAD_OE_N_BIT2 31
+#define GPO_PWM_PAD_OE_N_BIT3 32
+#define GPO_PWM_PAD_OE_N_BIT4 33
+#define GPO_PWM_PAD_OE_N_BIT5 34
+#define GPO_PWM_PAD_OE_N_BIT6 35
+#define GPO_PWM_PAD_OE_N_BIT7 36
+#define GPO_PWM_PAD_OUT_BIT0 37
+#define GPO_PWM_PAD_OUT_BIT1 38
+#define GPO_PWM_PAD_OUT_BIT2 39
+#define GPO_PWM_PAD_OUT_BIT3 40
+#define GPO_PWM_PAD_OUT_BIT4 41
+#define GPO_PWM_PAD_OUT_BIT5 42
+#define GPO_PWM_PAD_OUT_BIT6 43
+#define GPO_PWM_PAD_OUT_BIT7 44
+#define GPO_PWMDAC_LEFT_OUT 45
+#define GPO_PWMDAC_RIGHT_OUT 46
+#define GPO_QSPI_CSN1_OUT 47
+#define GPO_QSPI_CSN2_OUT 48
+#define GPO_QSPI_CSN3_OUT 49
+#define GPO_REGISTER23_SCFG_CMSENSOR_RST0 50
+#define GPO_REGISTER23_SCFG_CMSENSOR_RST1 51
+#define GPO_REGISTER32_SCFG_GMAC_PHY_RSTN 52
+#define GPO_SDIO0_PAD_CARD_POWER_EN 53
+#define GPO_SDIO0_PAD_CCLK_OUT 54
+#define GPO_SDIO0_PAD_CCMD_OE 55
+#define GPO_SDIO0_PAD_CCMD_OEN (GPO_SDIO0_PAD_CCMD_OE | GPO_REVERSE)
+#define GPO_SDIO0_PAD_CCMD_OUT 56
+#define GPO_SDIO0_PAD_CDATA_OE_BIT0 57
+#define GPO_SDIO0_PAD_CDATA_OEN_BIT0 (GPO_SDIO0_PAD_CDATA_OE_BIT0 | GPO_REVERSE)
+#define GPO_SDIO0_PAD_CDATA_OE_BIT1 58
+#define GPO_SDIO0_PAD_CDATA_OEN_BIT1 (GPO_SDIO0_PAD_CDATA_OE_BIT1 | GPO_REVERSE)
+#define GPO_SDIO0_PAD_CDATA_OE_BIT2 59
+#define GPO_SDIO0_PAD_CDATA_OEN_BIT2 (GPO_SDIO0_PAD_CDATA_OE_BIT2 | GPO_REVERSE)
+#define GPO_SDIO0_PAD_CDATA_OE_BIT3 60
+#define GPO_SDIO0_PAD_CDATA_OEN_BIT3 (GPO_SDIO0_PAD_CDATA_OE_BIT3 | GPO_REVERSE)
+#define GPO_SDIO0_PAD_CDATA_OE_BIT4 61
+#define GPO_SDIO0_PAD_CDATA_OEN_BIT4 (GPO_SDIO0_PAD_CDATA_OE_BIT4 | GPO_REVERSE)
+#define GPO_SDIO0_PAD_CDATA_OE_BIT5 62
+#define GPO_SDIO0_PAD_CDATA_OEN_BIT5 (GPO_SDIO0_PAD_CDATA_OE_BIT5 | GPO_REVERSE)
+#define GPO_SDIO0_PAD_CDATA_OE_BIT6 63
+#define GPO_SDIO0_PAD_CDATA_OEN_BIT6 (GPO_SDIO0_PAD_CDATA_OE_BIT6 | GPO_REVERSE)
+#define GPO_SDIO0_PAD_CDATA_OE_BIT7 64
+#define GPO_SDIO0_PAD_CDATA_OEN_BIT7 (GPO_SDIO0_PAD_CDATA_OE_BIT7 | GPO_REVERSE)
+#define GPO_SDIO0_PAD_CDATA_OUT_BIT0 65
+#define GPO_SDIO0_PAD_CDATA_OUT_BIT1 66
+#define GPO_SDIO0_PAD_CDATA_OUT_BIT2 67
+#define GPO_SDIO0_PAD_CDATA_OUT_BIT3 68
+#define GPO_SDIO0_PAD_CDATA_OUT_BIT4 69
+#define GPO_SDIO0_PAD_CDATA_OUT_BIT5 70
+#define GPO_SDIO0_PAD_CDATA_OUT_BIT6 71
+#define GPO_SDIO0_PAD_CDATA_OUT_BIT7 72
+#define GPO_SDIO0_PAD_RST_N 73
+#define GPO_SDIO1_PAD_CARD_POWER_EN 74
+#define GPO_SDIO1_PAD_CCLK_OUT 75
+#define GPO_SDIO1_PAD_CCMD_OE 76
+#define GPO_SDIO1_PAD_CCMD_OEN (GPO_SDIO1_PAD_CCMD_OE | GPO_REVERSE)
+#define GPO_SDIO1_PAD_CCMD_OUT 77
+#define GPO_SDIO1_PAD_CDATA_OE_BIT0 78
+#define GPO_SDIO1_PAD_CDATA_OEN_BIT0 (GPO_SDIO1_PAD_CDATA_OE_BIT0 | GPO_REVERSE)
+#define GPO_SDIO1_PAD_CDATA_OE_BIT1 79
+#define GPO_SDIO1_PAD_CDATA_OEN_BIT1 (GPO_SDIO1_PAD_CDATA_OE_BIT1 | GPO_REVERSE)
+#define GPO_SDIO1_PAD_CDATA_OE_BIT2 80
+#define GPO_SDIO1_PAD_CDATA_OEN_BIT2 (GPO_SDIO1_PAD_CDATA_OE_BIT2 | GPO_REVERSE)
+#define GPO_SDIO1_PAD_CDATA_OE_BIT3 81
+#define GPO_SDIO1_PAD_CDATA_OEN_BIT3 (GPO_SDIO1_PAD_CDATA_OE_BIT3 | GPO_REVERSE)
+#define GPO_SDIO1_PAD_CDATA_OE_BIT4 82
+#define GPO_SDIO1_PAD_CDATA_OEN_BIT4 (GPO_SDIO1_PAD_CDATA_OE_BIT4 | GPO_REVERSE)
+#define GPO_SDIO1_PAD_CDATA_OE_BIT5 83
+#define GPO_SDIO1_PAD_CDATA_OEN_BIT5 (GPO_SDIO1_PAD_CDATA_OE_BIT5 | GPO_REVERSE)
+#define GPO_SDIO1_PAD_CDATA_OE_BIT6 84
+#define GPO_SDIO1_PAD_CDATA_OEN_BIT6 (GPO_SDIO1_PAD_CDATA_OE_BIT6 | GPO_REVERSE)
+#define GPO_SDIO1_PAD_CDATA_OE_BIT7 85
+#define GPO_SDIO1_PAD_CDATA_OEN_BIT7 (GPO_SDIO1_PAD_CDATA_OE_BIT7 | GPO_REVERSE)
+#define GPO_SDIO1_PAD_CDATA_OUT_BIT0 86
+#define GPO_SDIO1_PAD_CDATA_OUT_BIT1 87
+#define GPO_SDIO1_PAD_CDATA_OUT_BIT2 88
+#define GPO_SDIO1_PAD_CDATA_OUT_BIT3 89
+#define GPO_SDIO1_PAD_CDATA_OUT_BIT4 90
+#define GPO_SDIO1_PAD_CDATA_OUT_BIT5 91
+#define GPO_SDIO1_PAD_CDATA_OUT_BIT6 92
+#define GPO_SDIO1_PAD_CDATA_OUT_BIT7 93
+#define GPO_SDIO1_PAD_RST_N 94
+#define GPO_SPDIF_TX_SDOUT 95
+#define GPO_SPDIF_TX_SDOUT_OEN 96
+#define GPO_SPI0_PAD_OE_N 97
+#define GPO_SPI0_PAD_SCK_OUT 98
+#define GPO_SPI0_PAD_SS_0_N 99
+#define GPO_SPI0_PAD_SS_1_N 100
+#define GPO_SPI0_PAD_TXD 101
+#define GPO_SPI1_PAD_OE_N 102
+#define GPO_SPI1_PAD_SCK_OUT 103
+#define GPO_SPI1_PAD_SS_0_N 104
+#define GPO_SPI1_PAD_SS_1_N 105
+#define GPO_SPI1_PAD_TXD 106
+#define GPO_SPI2_PAD_OE_N 107
+#define GPO_SPI2_PAD_SCK_OUT 108
+#define GPO_SPI2_PAD_SS_0_N 109
+#define GPO_SPI2_PAD_SS_1_N 110
+#define GPO_SPI2_PAD_TXD 111
+#define GPO_SPI2AHB_PAD_OE_N_BIT0 112
+#define GPO_SPI2AHB_PAD_OE_N_BIT1 113
+#define GPO_SPI2AHB_PAD_OE_N_BIT2 114
+#define GPO_SPI2AHB_PAD_OE_N_BIT3 115
+#define GPO_SPI2AHB_PAD_TXD_BIT0 116
+#define GPO_SPI2AHB_PAD_TXD_BIT1 117
+#define GPO_SPI2AHB_PAD_TXD_BIT2 118
+#define GPO_SPI2AHB_PAD_TXD_BIT3 119
+#define GPO_SPI3_PAD_OE_N 120
+#define GPO_SPI3_PAD_SCK_OUT 121
+#define GPO_SPI3_PAD_SS_0_N 122
+#define GPO_SPI3_PAD_SS_1_N 123
+#define GPO_SPI3_PAD_TXD 124
+#define GPO_UART0_PAD_DTRN 125
+#define GPO_UART0_PAD_RTSN 126
+#define GPO_UART0_PAD_SOUT 127
+#define GPO_UART1_PAD_SOUT 128
+#define GPO_UART2_PAD_DTR_N 129
+#define GPO_UART2_PAD_RTS_N 130
+#define GPO_UART2_PAD_SOUT 131
+#define GPO_UART3_PAD_SOUT 132
+#define GPO_USB_DRV_BUS 133
+
+#define GPI_CPU_JTAG_TCK 0
+#define GPI_CPU_JTAG_TDI 1
+#define GPI_CPU_JTAG_TMS 2
+#define GPI_CPU_JTAG_TRST 3
+#define GPI_DMIC_SDIN_BIT0 4
+#define GPI_DMIC_SDIN_BIT1 5
+#define GPI_DSP_JTCK_PAD 6
+#define GPI_DSP_JTDI_PAD 7
+#define GPI_DSP_JTMS_PAD 8
+#define GPI_DSP_TRST_PAD 9
+#define GPI_I2C0_PAD_SCK_IN 10
+#define GPI_I2C0_PAD_SDA_IN 11
+#define GPI_I2C1_PAD_SCK_IN 12
+#define GPI_I2C1_PAD_SDA_IN 13
+#define GPI_I2C2_PAD_SCK_IN 14
+#define GPI_I2C2_PAD_SDA_IN 15
+#define GPI_I2C3_PAD_SCK_IN 16
+#define GPI_I2C3_PAD_SDA_IN 17
+#define GPI_I2SRX_BCLK_IN 18
+#define GPI_I2SRX_LRCK_IN 19
+#define GPI_I2SRX_SDIN_BIT0 20
+#define GPI_I2SRX_SDIN_BIT1 21
+#define GPI_I2SRX_SDIN_BIT2 22
+#define GPI_I2STX_BCLK_IN 23
+#define GPI_I2STX_LRCK_IN 24
+#define GPI_SDIO0_PAD_CARD_DETECT_N 25
+#define GPI_SDIO0_PAD_CARD_WRITE_PRT 26
+#define GPI_SDIO0_PAD_CCMD_IN 27
+#define GPI_SDIO0_PAD_CDATA_IN_BIT0 28
+#define GPI_SDIO0_PAD_CDATA_IN_BIT1 29
+#define GPI_SDIO0_PAD_CDATA_IN_BIT2 30
+#define GPI_SDIO0_PAD_CDATA_IN_BIT3 31
+#define GPI_SDIO0_PAD_CDATA_IN_BIT4 32
+#define GPI_SDIO0_PAD_CDATA_IN_BIT5 33
+#define GPI_SDIO0_PAD_CDATA_IN_BIT6 34
+#define GPI_SDIO0_PAD_CDATA_IN_BIT7 35
+#define GPI_SDIO1_PAD_CARD_DETECT_N 36
+#define GPI_SDIO1_PAD_CARD_WRITE_PRT 37
+#define GPI_SDIO1_PAD_CCMD_IN 38
+#define GPI_SDIO1_PAD_CDATA_IN_BIT0 39
+#define GPI_SDIO1_PAD_CDATA_IN_BIT1 40
+#define GPI_SDIO1_PAD_CDATA_IN_BIT2 41
+#define GPI_SDIO1_PAD_CDATA_IN_BIT3 42
+#define GPI_SDIO1_PAD_CDATA_IN_BIT4 43
+#define GPI_SDIO1_PAD_CDATA_IN_BIT5 44
+#define GPI_SDIO1_PAD_CDATA_IN_BIT6 45
+#define GPI_SDIO1_PAD_CDATA_IN_BIT7 46
+#define GPI_SPDIF_RX_SDIN 47
+#define GPI_SPI0_PAD_RXD 48
+#define GPI_SPI0_PAD_SS_IN_N 49
+#define GPI_SPI1_PAD_RXD 50
+#define GPI_SPI1_PAD_SS_IN_N 51
+#define GPI_SPI2_PAD_RXD 52
+#define GPI_SPI2_PAD_SS_IN_N 53
+#define GPI_SPI2AHB_PAD_RXD_BIT0 54
+#define GPI_SPI2AHB_PAD_RXD_BIT1 55
+#define GPI_SPI2AHB_PAD_RXD_BIT2 56
+#define GPI_SPI2AHB_PAD_RXD_BIT3 57
+#define GPI_SPI2AHB_PAD_SS_N 58
+#define GPI_SPI2AHB_SLV_SCLKIN 59
+#define GPI_SPI3_PAD_RXD 60
+#define GPI_SPI3_PAD_SS_IN_N 61
+#define GPI_UART0_PAD_CTSN 62
+#define GPI_UART0_PAD_DCDN 63
+#define GPI_UART0_PAD_DSRN 64
+#define GPI_UART0_PAD_RIN 65
+#define GPI_UART0_PAD_SIN 66
+#define GPI_UART1_PAD_SIN 67
+#define GPI_UART2_PAD_CTS_N 68
+#define GPI_UART2_PAD_DCD_N 69
+#define GPI_UART2_PAD_DSR_N 70
+#define GPI_UART2_PAD_RI_N 71
+#define GPI_UART2_PAD_SIN 72
+#define GPI_UART3_PAD_SIN 73
+#define GPI_USB_OVER_CURRENT 74
+
+#define GPI_NONE 0xff
+
+#endif /* __DT_BINDINGS_PINCTRL_STARFIVE_H__ */
diff --git a/include/dt-bindings/pinctrl/samsung.h b/include/dt-bindings/pinctrl/samsung.h
index b183250..9509706 100644
--- a/include/dt-bindings/pinctrl/samsung.h
+++ b/include/dt-bindings/pinctrl/samsung.h
@@ -36,7 +36,10 @@
#define EXYNOS5260_PIN_DRV_LV4 2
#define EXYNOS5260_PIN_DRV_LV6 3
-/* Drive strengths for Exynos5410, Exynos542x and Exynos5800 */
+/*
+ * Drive strengths for Exynos5410, Exynos542x, Exynos5800 and Exynos850 (except
+ * GPIO_HSI block)
+ */
#define EXYNOS5420_PIN_DRV_LV1 0
#define EXYNOS5420_PIN_DRV_LV2 1
#define EXYNOS5420_PIN_DRV_LV3 2
@@ -56,6 +59,14 @@
#define EXYNOS5433_PIN_DRV_SLOW_SR5 0xc
#define EXYNOS5433_PIN_DRV_SLOW_SR6 0xf
+/* Drive strengths for Exynos850 GPIO_HSI block */
+#define EXYNOS850_HSI_PIN_DRV_LV1 0 /* 1x */
+#define EXYNOS850_HSI_PIN_DRV_LV1_5 1 /* 1.5x */
+#define EXYNOS850_HSI_PIN_DRV_LV2 2 /* 2x */
+#define EXYNOS850_HSI_PIN_DRV_LV2_5 3 /* 2.5x */
+#define EXYNOS850_HSI_PIN_DRV_LV3 4 /* 3x */
+#define EXYNOS850_HSI_PIN_DRV_LV4 5 /* 4x */
+
#define EXYNOS_PIN_FUNC_INPUT 0
#define EXYNOS_PIN_FUNC_OUTPUT 1
#define EXYNOS_PIN_FUNC_2 2
diff --git a/include/dt-bindings/power/imx8mn-power.h b/include/dt-bindings/power/imx8mn-power.h
index 102ee85..eedd0e5 100644
--- a/include/dt-bindings/power/imx8mn-power.h
+++ b/include/dt-bindings/power/imx8mn-power.h
@@ -12,4 +12,9 @@
#define IMX8MN_POWER_DOMAIN_DISPMIX 3
#define IMX8MN_POWER_DOMAIN_MIPI 4
+#define IMX8MN_DISPBLK_PD_MIPI_DSI 0
+#define IMX8MN_DISPBLK_PD_MIPI_CSI 1
+#define IMX8MN_DISPBLK_PD_LCDIF 2
+#define IMX8MN_DISPBLK_PD_ISI 3
+
#endif
diff --git a/include/dt-bindings/power/imx8ulp-power.h b/include/dt-bindings/power/imx8ulp-power.h
new file mode 100644
index 0000000..a556b2e
--- /dev/null
+++ b/include/dt-bindings/power/imx8ulp-power.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: (GPL-2.0 OR MIT) */
+/*
+ * Copyright 2021 NXP
+ */
+
+#ifndef __DT_BINDINGS_IMX8ULP_POWER_H__
+#define __DT_BINDINGS_IMX8ULP_POWER_H__
+
+#define IMX8ULP_PD_DMA1 0
+#define IMX8ULP_PD_FLEXSPI2 1
+#define IMX8ULP_PD_USB0 2
+#define IMX8ULP_PD_USDHC0 3
+#define IMX8ULP_PD_USDHC1 4
+#define IMX8ULP_PD_USDHC2_USB1 5
+#define IMX8ULP_PD_DCNANO 6
+#define IMX8ULP_PD_EPDC 7
+#define IMX8ULP_PD_DMA2 8
+#define IMX8ULP_PD_GPU2D 9
+#define IMX8ULP_PD_GPU3D 10
+#define IMX8ULP_PD_HIFI4 11
+#define IMX8ULP_PD_ISI 12
+#define IMX8ULP_PD_MIPI_CSI 13
+#define IMX8ULP_PD_MIPI_DSI 14
+#define IMX8ULP_PD_PXP 15
+
+#endif
diff --git a/include/dt-bindings/power/qcom-rpmpd.h b/include/dt-bindings/power/qcom-rpmpd.h
index 960f797..edfc1ff 100644
--- a/include/dt-bindings/power/qcom-rpmpd.h
+++ b/include/dt-bindings/power/qcom-rpmpd.h
@@ -68,6 +68,21 @@
#define SM8350_MXC_AO 11
#define SM8350_MSS 12
+/* SM8450 Power Domain Indexes */
+#define SM8450_CX 0
+#define SM8450_CX_AO 1
+#define SM8450_EBI 2
+#define SM8450_GFX 3
+#define SM8450_LCX 4
+#define SM8450_LMX 5
+#define SM8450_MMCX 6
+#define SM8450_MMCX_AO 7
+#define SM8450_MX 8
+#define SM8450_MX_AO 9
+#define SM8450_MXC 10
+#define SM8450_MXC_AO 11
+#define SM8450_MSS 12
+
/* SC7180 Power Domain Indexes */
#define SC7180_CX 0
#define SC7180_CX_AO 1
@@ -219,6 +234,24 @@
#define SM6115_VDD_LPI_CX 6
#define SM6115_VDD_LPI_MX 7
+/* SM6125 Power Domains */
+#define SM6125_VDDCX 0
+#define SM6125_VDDCX_AO 1
+#define SM6125_VDDCX_VFL 2
+#define SM6125_VDDMX 3
+#define SM6125_VDDMX_AO 4
+#define SM6125_VDDMX_VFL 5
+
+/* QCM2290 Power Domains */
+#define QCM2290_VDDCX 0
+#define QCM2290_VDDCX_AO 1
+#define QCM2290_VDDCX_VFL 2
+#define QCM2290_VDDMX 3
+#define QCM2290_VDDMX_AO 4
+#define QCM2290_VDDMX_VFL 5
+#define QCM2290_VDD_LPI_CX 6
+#define QCM2290_VDD_LPI_MX 7
+
/* RPM SMD Power Domain performance levels */
#define RPM_SMD_LEVEL_RETENTION 16
#define RPM_SMD_LEVEL_RETENTION_PLUS 32
diff --git a/include/dt-bindings/power/r8a779f0-sysc.h b/include/dt-bindings/power/r8a779f0-sysc.h
new file mode 100644
index 0000000..0ec8ad7
--- /dev/null
+++ b/include/dt-bindings/power/r8a779f0-sysc.h
@@ -0,0 +1,30 @@
+/* SPDX-License-Identifier: (GPL-2.0 or MIT) */
+/*
+ * Copyright (C) 2021 Renesas Electronics Corp.
+ */
+#ifndef __DT_BINDINGS_POWER_R8A779F0_SYSC_H__
+#define __DT_BINDINGS_POWER_R8A779F0_SYSC_H__
+
+/*
+ * These power domain indices match the Power Domain Register Numbers (PDR)
+ */
+
+#define R8A779F0_PD_A1E0D0C0 0
+#define R8A779F0_PD_A1E0D0C1 1
+#define R8A779F0_PD_A1E0D1C0 2
+#define R8A779F0_PD_A1E0D1C1 3
+#define R8A779F0_PD_A1E1D0C0 4
+#define R8A779F0_PD_A1E1D0C1 5
+#define R8A779F0_PD_A1E1D1C0 6
+#define R8A779F0_PD_A1E1D1C1 7
+#define R8A779F0_PD_A2E0D0 16
+#define R8A779F0_PD_A2E0D1 17
+#define R8A779F0_PD_A2E1D0 18
+#define R8A779F0_PD_A2E1D1 19
+#define R8A779F0_PD_A3E0 20
+#define R8A779F0_PD_A3E1 21
+
+/* Always-on power area */
+#define R8A779F0_PD_ALWAYS_ON 64
+
+#endif /* __DT_BINDINGS_POWER_R8A779A0_SYSC_H__*/
diff --git a/include/dt-bindings/reset/starfive-jh7100.h b/include/dt-bindings/reset/starfive-jh7100.h
new file mode 100644
index 0000000..540e192
--- /dev/null
+++ b/include/dt-bindings/reset/starfive-jh7100.h
@@ -0,0 +1,126 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/*
+ * Copyright (C) 2021 Ahmad Fatoum, Pengutronix
+ */
+
+#ifndef __DT_BINDINGS_RESET_STARFIVE_JH7100_H__
+#define __DT_BINDINGS_RESET_STARFIVE_JH7100_H__
+
+#define JH7100_RSTN_DOM3AHB_BUS 0
+#define JH7100_RSTN_DOM7AHB_BUS 1
+#define JH7100_RST_U74 2
+#define JH7100_RSTN_U74_AXI 3
+#define JH7100_RSTN_SGDMA2P_AHB 4
+#define JH7100_RSTN_SGDMA2P_AXI 5
+#define JH7100_RSTN_DMA2PNOC_AXI 6
+#define JH7100_RSTN_DLA_AXI 7
+#define JH7100_RSTN_DLANOC_AXI 8
+#define JH7100_RSTN_DLA_APB 9
+#define JH7100_RST_VP6_DRESET 10
+#define JH7100_RST_VP6_BRESET 11
+#define JH7100_RSTN_VP6_AXI 12
+#define JH7100_RSTN_VDECBRG_MAIN 13
+#define JH7100_RSTN_VDEC_AXI 14
+#define JH7100_RSTN_VDEC_BCLK 15
+#define JH7100_RSTN_VDEC_CCLK 16
+#define JH7100_RSTN_VDEC_APB 17
+#define JH7100_RSTN_JPEG_AXI 18
+#define JH7100_RSTN_JPEG_CCLK 19
+#define JH7100_RSTN_JPEG_APB 20
+#define JH7100_RSTN_JPCGC300_MAIN 21
+#define JH7100_RSTN_GC300_2X 22
+#define JH7100_RSTN_GC300_AXI 23
+#define JH7100_RSTN_GC300_AHB 24
+#define JH7100_RSTN_VENC_AXI 25
+#define JH7100_RSTN_VENCBRG_MAIN 26
+#define JH7100_RSTN_VENC_BCLK 27
+#define JH7100_RSTN_VENC_CCLK 28
+#define JH7100_RSTN_VENC_APB 29
+#define JH7100_RSTN_DDRPHY_APB 30
+#define JH7100_RSTN_NOC_ROB 31
+#define JH7100_RSTN_NOC_COG 32
+#define JH7100_RSTN_HIFI4_AXI 33
+#define JH7100_RSTN_HIFI4NOC_AXI 34
+#define JH7100_RST_HIFI4_DRESET 35
+#define JH7100_RST_HIFI4_BRESET 36
+#define JH7100_RSTN_USB_AXI 37
+#define JH7100_RSTN_USBNOC_AXI 38
+#define JH7100_RSTN_SGDMA1P_AXI 39
+#define JH7100_RSTN_DMA1P_AXI 40
+#define JH7100_RSTN_X2C_AXI 41
+#define JH7100_RSTN_NNE_AHB 42
+#define JH7100_RSTN_NNE_AXI 43
+#define JH7100_RSTN_NNENOC_AXI 44
+#define JH7100_RSTN_DLASLV_AXI 45
+#define JH7100_RSTN_DSPX2C_AXI 46
+#define JH7100_RSTN_VIN_SRC 47
+#define JH7100_RSTN_ISPSLV_AXI 48
+#define JH7100_RSTN_VIN_AXI 49
+#define JH7100_RSTN_VINNOC_AXI 50
+#define JH7100_RSTN_ISP0_AXI 51
+#define JH7100_RSTN_ISP0NOC_AXI 52
+#define JH7100_RSTN_ISP1_AXI 53
+#define JH7100_RSTN_ISP1NOC_AXI 54
+#define JH7100_RSTN_VOUT_SRC 55
+#define JH7100_RSTN_DISP_AXI 56
+#define JH7100_RSTN_DISPNOC_AXI 57
+#define JH7100_RSTN_SDIO0_AHB 58
+#define JH7100_RSTN_SDIO1_AHB 59
+#define JH7100_RSTN_GMAC_AHB 60
+#define JH7100_RSTN_SPI2AHB_AHB 61
+#define JH7100_RSTN_SPI2AHB_CORE 62
+#define JH7100_RSTN_EZMASTER_AHB 63
+#define JH7100_RST_E24 64
+#define JH7100_RSTN_QSPI_AHB 65
+#define JH7100_RSTN_QSPI_CORE 66
+#define JH7100_RSTN_QSPI_APB 67
+#define JH7100_RSTN_SEC_AHB 68
+#define JH7100_RSTN_AES 69
+#define JH7100_RSTN_PKA 70
+#define JH7100_RSTN_SHA 71
+#define JH7100_RSTN_TRNG_APB 72
+#define JH7100_RSTN_OTP_APB 73
+#define JH7100_RSTN_UART0_APB 74
+#define JH7100_RSTN_UART0_CORE 75
+#define JH7100_RSTN_UART1_APB 76
+#define JH7100_RSTN_UART1_CORE 77
+#define JH7100_RSTN_SPI0_APB 78
+#define JH7100_RSTN_SPI0_CORE 79
+#define JH7100_RSTN_SPI1_APB 80
+#define JH7100_RSTN_SPI1_CORE 81
+#define JH7100_RSTN_I2C0_APB 82
+#define JH7100_RSTN_I2C0_CORE 83
+#define JH7100_RSTN_I2C1_APB 84
+#define JH7100_RSTN_I2C1_CORE 85
+#define JH7100_RSTN_GPIO_APB 86
+#define JH7100_RSTN_UART2_APB 87
+#define JH7100_RSTN_UART2_CORE 88
+#define JH7100_RSTN_UART3_APB 89
+#define JH7100_RSTN_UART3_CORE 90
+#define JH7100_RSTN_SPI2_APB 91
+#define JH7100_RSTN_SPI2_CORE 92
+#define JH7100_RSTN_SPI3_APB 93
+#define JH7100_RSTN_SPI3_CORE 94
+#define JH7100_RSTN_I2C2_APB 95
+#define JH7100_RSTN_I2C2_CORE 96
+#define JH7100_RSTN_I2C3_APB 97
+#define JH7100_RSTN_I2C3_CORE 98
+#define JH7100_RSTN_WDTIMER_APB 99
+#define JH7100_RSTN_WDT 100
+#define JH7100_RSTN_TIMER0 101
+#define JH7100_RSTN_TIMER1 102
+#define JH7100_RSTN_TIMER2 103
+#define JH7100_RSTN_TIMER3 104
+#define JH7100_RSTN_TIMER4 105
+#define JH7100_RSTN_TIMER5 106
+#define JH7100_RSTN_TIMER6 107
+#define JH7100_RSTN_VP6INTC_APB 108
+#define JH7100_RSTN_PWM_APB 109
+#define JH7100_RSTN_MSI_APB 110
+#define JH7100_RSTN_TEMP_APB 111
+#define JH7100_RSTN_TEMP_SENSE 112
+#define JH7100_RSTN_SYSERR_APB 113
+
+#define JH7100_RSTN_END 114
+
+#endif /* __DT_BINDINGS_RESET_STARFIVE_JH7100_H__ */
diff --git a/include/dt-bindings/reset/tegra234-reset.h b/include/dt-bindings/reset/tegra234-reset.h
index b3c63be..50e13bc 100644
--- a/include/dt-bindings/reset/tegra234-reset.h
+++ b/include/dt-bindings/reset/tegra234-reset.h
@@ -4,7 +4,15 @@
#ifndef DT_BINDINGS_RESET_TEGRA234_RESET_H
#define DT_BINDINGS_RESET_TEGRA234_RESET_H
-#define TEGRA234_RESET_SDMMC4 85
-#define TEGRA234_RESET_UARTA 100
+/**
+ * @file
+ * @defgroup bpmp_reset_ids Reset ID's
+ * @brief Identifiers for Resets controllable by firmware
+ * @{
+ */
+#define TEGRA234_RESET_SDMMC4 85U
+#define TEGRA234_RESET_UARTA 100U
+
+/** @} */
#endif
diff --git a/include/dt-bindings/soc/samsung,exynos-usi.h b/include/dt-bindings/soc/samsung,exynos-usi.h
new file mode 100644
index 0000000..a01af16
--- /dev/null
+++ b/include/dt-bindings/soc/samsung,exynos-usi.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) */
+/*
+ * Copyright (c) 2021 Linaro Ltd.
+ * Author: Sam Protsenko <semen.protsenko@linaro.org>
+ *
+ * Device Tree bindings for Samsung Exynos USI (Universal Serial Interface).
+ */
+
+#ifndef __DT_BINDINGS_SAMSUNG_EXYNOS_USI_H
+#define __DT_BINDINGS_SAMSUNG_EXYNOS_USI_H
+
+#define USI_V2_NONE 0
+#define USI_V2_UART 1
+#define USI_V2_SPI 2
+#define USI_V2_I2C 3
+
+#endif /* __DT_BINDINGS_SAMSUNG_EXYNOS_USI_H */
diff --git a/include/linux/atalk.h b/include/linux/atalk.h
index f6034ba..a55bfc6 100644
--- a/include/linux/atalk.h
+++ b/include/linux/atalk.h
@@ -113,7 +113,7 @@ extern int aarp_proto_init(void);
/* Inter module exports */
/* Give a device find its atif control structure */
-#if IS_ENABLED(CONFIG_IRDA) || IS_ENABLED(CONFIG_ATALK)
+#if IS_ENABLED(CONFIG_ATALK)
static inline struct atalk_iface *atalk_find_dev(struct net_device *dev)
{
return dev->atalk_ptr;
diff --git a/include/linux/avf/virtchnl.h b/include/linux/avf/virtchnl.h
index b30a1bc..2ce27e8 100644
--- a/include/linux/avf/virtchnl.h
+++ b/include/linux/avf/virtchnl.h
@@ -141,6 +141,13 @@ enum virtchnl_ops {
VIRTCHNL_OP_DEL_RSS_CFG = 46,
VIRTCHNL_OP_ADD_FDIR_FILTER = 47,
VIRTCHNL_OP_DEL_FDIR_FILTER = 48,
+ VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS = 51,
+ VIRTCHNL_OP_ADD_VLAN_V2 = 52,
+ VIRTCHNL_OP_DEL_VLAN_V2 = 53,
+ VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 = 54,
+ VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 = 55,
+ VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 = 56,
+ VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2 = 57,
VIRTCHNL_OP_MAX,
};
@@ -246,6 +253,7 @@ VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vsi_resource);
#define VIRTCHNL_VF_OFFLOAD_REQ_QUEUES BIT(6)
/* used to negotiate communicating link speeds in Mbps */
#define VIRTCHNL_VF_CAP_ADV_LINK_SPEED BIT(7)
+#define VIRTCHNL_VF_OFFLOAD_VLAN_V2 BIT(15)
#define VIRTCHNL_VF_OFFLOAD_VLAN BIT(16)
#define VIRTCHNL_VF_OFFLOAD_RX_POLLING BIT(17)
#define VIRTCHNL_VF_OFFLOAD_RSS_PCTYPE_V2 BIT(18)
@@ -475,6 +483,351 @@ struct virtchnl_vlan_filter_list {
VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_vlan_filter_list);
+/* This enum is used for all of the VIRTCHNL_VF_OFFLOAD_VLAN_V2_CAPS related
+ * structures and opcodes.
+ *
+ * VIRTCHNL_VLAN_UNSUPPORTED - This field is not supported and if a VF driver
+ * populates it the PF should return VIRTCHNL_STATUS_ERR_NOT_SUPPORTED.
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 - This field supports 0x8100 ethertype.
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 - This field supports 0x88A8 ethertype.
+ * VIRTCHNL_VLAN_ETHERTYPE_9100 - This field supports 0x9100 ethertype.
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_AND - Used when multiple ethertypes can be supported
+ * by the PF concurrently. For example, if the PF can support
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 AND VIRTCHNL_VLAN_ETHERTYPE_88A8 filters it
+ * would OR the following bits:
+ *
+ * VIRTHCNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * The VF would interpret this as VLAN filtering can be supported on both 0x8100
+ * and 0x88A8 VLAN ethertypes.
+ *
+ * VIRTCHNL_ETHERTYPE_XOR - Used when only a single ethertype can be supported
+ * by the PF concurrently. For example if the PF can support
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 XOR VIRTCHNL_VLAN_ETHERTYPE_88A8 stripping
+ * offload it would OR the following bits:
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * The VF would interpret this as VLAN stripping can be supported on either
+ * 0x8100 or 0x88a8 VLAN ethertypes. So when requesting VLAN stripping via
+ * VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 the specified ethertype will override
+ * the previously set value.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG1 - Used to tell the VF to insert and/or
+ * strip the VLAN tag using the L2TAG1 field of the Tx/Rx descriptors.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 - Used to tell the VF to insert hardware
+ * offloaded VLAN tags using the L2TAG2 field of the Tx descriptor.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 - Used to tell the VF to strip hardware
+ * offloaded VLAN tags using the L2TAG2_2 field of the Rx descriptor.
+ *
+ * VIRTCHNL_VLAN_PRIO - This field supports VLAN priority bits. This is used for
+ * VLAN filtering if the underlying PF supports it.
+ *
+ * VIRTCHNL_VLAN_TOGGLE_ALLOWED - This field is used to say whether a
+ * certain VLAN capability can be toggled. For example if the underlying PF/CP
+ * allows the VF to toggle VLAN filtering, stripping, and/or insertion it should
+ * set this bit along with the supported ethertypes.
+ */
+enum virtchnl_vlan_support {
+ VIRTCHNL_VLAN_UNSUPPORTED = 0,
+ VIRTCHNL_VLAN_ETHERTYPE_8100 = BIT(0),
+ VIRTCHNL_VLAN_ETHERTYPE_88A8 = BIT(1),
+ VIRTCHNL_VLAN_ETHERTYPE_9100 = BIT(2),
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG1 = BIT(8),
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 = BIT(9),
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2_2 = BIT(10),
+ VIRTCHNL_VLAN_PRIO = BIT(24),
+ VIRTCHNL_VLAN_FILTER_MASK = BIT(28),
+ VIRTCHNL_VLAN_ETHERTYPE_AND = BIT(29),
+ VIRTCHNL_VLAN_ETHERTYPE_XOR = BIT(30),
+ VIRTCHNL_VLAN_TOGGLE = BIT(31),
+};
+
+/* This structure is used as part of the VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS
+ * for filtering, insertion, and stripping capabilities.
+ *
+ * If only outer capabilities are supported (for filtering, insertion, and/or
+ * stripping) then this refers to the outer most or single VLAN from the VF's
+ * perspective.
+ *
+ * If only inner capabilities are supported (for filtering, insertion, and/or
+ * stripping) then this refers to the outer most or single VLAN from the VF's
+ * perspective. Functionally this is the same as if only outer capabilities are
+ * supported. The VF driver is just forced to use the inner fields when
+ * adding/deleting filters and enabling/disabling offloads (if supported).
+ *
+ * If both outer and inner capabilities are supported (for filtering, insertion,
+ * and/or stripping) then outer refers to the outer most or single VLAN and
+ * inner refers to the second VLAN, if it exists, in the packet.
+ *
+ * There is no support for tunneled VLAN offloads, so outer or inner are never
+ * referring to a tunneled packet from the VF's perspective.
+ */
+struct virtchnl_vlan_supported_caps {
+ u32 outer;
+ u32 inner;
+};
+
+/* The PF populates these fields based on the supported VLAN filtering. If a
+ * field is VIRTCHNL_VLAN_UNSUPPORTED then it's not supported and the PF will
+ * reject any VIRTCHNL_OP_ADD_VLAN_V2 or VIRTCHNL_OP_DEL_VLAN_V2 messages using
+ * the unsupported fields.
+ *
+ * Also, a VF is only allowed to toggle its VLAN filtering setting if the
+ * VIRTCHNL_VLAN_TOGGLE bit is set.
+ *
+ * The ethertype(s) specified in the ethertype_init field are the ethertypes
+ * enabled for VLAN filtering. VLAN filtering in this case refers to the outer
+ * most VLAN from the VF's perspective. If both inner and outer filtering are
+ * allowed then ethertype_init only refers to the outer most VLAN as only
+ * VLAN ethertype supported for inner VLAN filtering is
+ * VIRTCHNL_VLAN_ETHERTYPE_8100. By default, inner VLAN filtering is disabled
+ * when both inner and outer filtering are allowed.
+ *
+ * The max_filters field tells the VF how many VLAN filters it's allowed to have
+ * at any one time. If it exceeds this amount and tries to add another filter,
+ * then the request will be rejected by the PF. To prevent failures, the VF
+ * should keep track of how many VLAN filters it has added and not attempt to
+ * add more than max_filters.
+ */
+struct virtchnl_vlan_filtering_caps {
+ struct virtchnl_vlan_supported_caps filtering_support;
+ u32 ethertype_init;
+ u16 max_filters;
+ u8 pad[2];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vlan_filtering_caps);
+
+/* This enum is used for the virtchnl_vlan_offload_caps structure to specify
+ * if the PF supports a different ethertype for stripping and insertion.
+ *
+ * VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION - The ethertype(s) specified
+ * for stripping affect the ethertype(s) specified for insertion and visa versa
+ * as well. If the VF tries to configure VLAN stripping via
+ * VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 with VIRTCHNL_VLAN_ETHERTYPE_8100 then
+ * that will be the ethertype for both stripping and insertion.
+ *
+ * VIRTCHNL_ETHERTYPE_MATCH_NOT_REQUIRED - The ethertype(s) specified for
+ * stripping do not affect the ethertype(s) specified for insertion and visa
+ * versa.
+ */
+enum virtchnl_vlan_ethertype_match {
+ VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION = 0,
+ VIRTCHNL_ETHERTYPE_MATCH_NOT_REQUIRED = 1,
+};
+
+/* The PF populates these fields based on the supported VLAN offloads. If a
+ * field is VIRTCHNL_VLAN_UNSUPPORTED then it's not supported and the PF will
+ * reject any VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 or
+ * VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 messages using the unsupported fields.
+ *
+ * Also, a VF is only allowed to toggle its VLAN offload setting if the
+ * VIRTCHNL_VLAN_TOGGLE_ALLOWED bit is set.
+ *
+ * The VF driver needs to be aware of how the tags are stripped by hardware and
+ * inserted by the VF driver based on the level of offload support. The PF will
+ * populate these fields based on where the VLAN tags are expected to be
+ * offloaded via the VIRTHCNL_VLAN_TAG_LOCATION_* bits. The VF will need to
+ * interpret these fields. See the definition of the
+ * VIRTCHNL_VLAN_TAG_LOCATION_* bits above the virtchnl_vlan_support
+ * enumeration.
+ */
+struct virtchnl_vlan_offload_caps {
+ struct virtchnl_vlan_supported_caps stripping_support;
+ struct virtchnl_vlan_supported_caps insertion_support;
+ u32 ethertype_init;
+ u8 ethertype_match;
+ u8 pad[3];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_vlan_offload_caps);
+
+/* VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS
+ * VF sends this message to determine its VLAN capabilities.
+ *
+ * PF will mark which capabilities it supports based on hardware support and
+ * current configuration. For example, if a port VLAN is configured the PF will
+ * not allow outer VLAN filtering, stripping, or insertion to be configured so
+ * it will block these features from the VF.
+ *
+ * The VF will need to cross reference its capabilities with the PFs
+ * capabilities in the response message from the PF to determine the VLAN
+ * support.
+ */
+struct virtchnl_vlan_caps {
+ struct virtchnl_vlan_filtering_caps filtering;
+ struct virtchnl_vlan_offload_caps offloads;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_vlan_caps);
+
+struct virtchnl_vlan {
+ u16 tci; /* tci[15:13] = PCP and tci[11:0] = VID */
+ u16 tci_mask; /* only valid if VIRTCHNL_VLAN_FILTER_MASK set in
+ * filtering caps
+ */
+ u16 tpid; /* 0x8100, 0x88a8, etc. and only type(s) set in
+ * filtering caps. Note that tpid here does not refer to
+ * VIRTCHNL_VLAN_ETHERTYPE_*, but it refers to the
+ * actual 2-byte VLAN TPID
+ */
+ u8 pad[2];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_vlan);
+
+struct virtchnl_vlan_filter {
+ struct virtchnl_vlan inner;
+ struct virtchnl_vlan outer;
+ u8 pad[16];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(32, virtchnl_vlan_filter);
+
+/* VIRTCHNL_OP_ADD_VLAN_V2
+ * VIRTCHNL_OP_DEL_VLAN_V2
+ *
+ * VF sends these messages to add/del one or more VLAN tag filters for Rx
+ * traffic.
+ *
+ * The PF attempts to add the filters and returns status.
+ *
+ * The VF should only ever attempt to add/del virtchnl_vlan_filter(s) using the
+ * supported fields negotiated via VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS.
+ */
+struct virtchnl_vlan_filter_list_v2 {
+ u16 vport_id;
+ u16 num_elements;
+ u8 pad[4];
+ struct virtchnl_vlan_filter filters[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_vlan_filter_list_v2);
+
+/* VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2
+ * VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2
+ *
+ * VF sends this message to enable or disable VLAN stripping or insertion. It
+ * also needs to specify an ethertype. The VF knows which VLAN ethertypes are
+ * allowed and whether or not it's allowed to enable/disable the specific
+ * offload via the VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS message. The VF needs to
+ * parse the virtchnl_vlan_caps.offloads fields to determine which offload
+ * messages are allowed.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.offloads in the
+ * following manner the VF will be allowed to enable and/or disable 0x8100 inner
+ * VLAN insertion and/or stripping via the opcodes listed above. Inner in this
+ * case means the outer most or single VLAN from the VF's perspective. This is
+ * because no outer offloads are supported. See the comments above the
+ * virtchnl_vlan_supported_caps structure for more details.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.inner =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.inner =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * In order to enable inner (again note that in this case inner is the outer
+ * most or single VLAN from the VF's perspective) VLAN stripping for 0x8100
+ * VLANs, the VF would populate the virtchnl_vlan_setting structure in the
+ * following manner and send the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 message.
+ *
+ * virtchnl_vlan_setting.inner_ethertype_setting =
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * The reason that VLAN TPID(s) are not being used for the
+ * outer_ethertype_setting and inner_ethertype_setting fields is because it's
+ * possible a device could support VLAN insertion and/or stripping offload on
+ * multiple ethertypes concurrently, so this method allows a VF to request
+ * multiple ethertypes in one message using the virtchnl_vlan_support
+ * enumeration.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.offloads in the
+ * following manner the VF will be allowed to enable 0x8100 and 0x88a8 outer
+ * VLAN insertion and stripping simultaneously. The
+ * virtchnl_vlan_caps.offloads.ethertype_match field will also have to be
+ * populated based on what the PF can support.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * In order to enable outer VLAN stripping for 0x8100 and 0x88a8 VLANs, the VF
+ * would populate the virthcnl_vlan_offload_structure in the following manner
+ * and send the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 message.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting =
+ * VIRTHCNL_VLAN_ETHERTYPE_8100 |
+ * VIRTHCNL_VLAN_ETHERTYPE_88A8;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * There is also the case where a PF and the underlying hardware can support
+ * VLAN offloads on multiple ethertypes, but not concurrently. For example, if
+ * the PF populates the virtchnl_vlan_caps.offloads in the following manner the
+ * VF will be allowed to enable and/or disable 0x8100 XOR 0x88a8 outer VLAN
+ * offloads. The ethertypes must match for stripping and insertion.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * virtchnl_vlan_caps.offloads.ethertype_match =
+ * VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION;
+ *
+ * In order to enable outer VLAN stripping for 0x88a8 VLANs, the VF would
+ * populate the virtchnl_vlan_setting structure in the following manner and send
+ * the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2. Also, this will change the
+ * ethertype for VLAN insertion if it's enabled. So, for completeness, a
+ * VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 with the same ethertype should be sent.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting = VIRTHCNL_VLAN_ETHERTYPE_88A8;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ */
+struct virtchnl_vlan_setting {
+ u32 outer_ethertype_setting;
+ u32 inner_ethertype_setting;
+ u16 vport_id;
+ u8 pad[6];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vlan_setting);
+
/* VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE
* VF sends VSI id and flags.
* PF returns status code in retval.
@@ -1156,6 +1509,30 @@ virtchnl_vc_validate_vf_msg(struct virtchnl_version_info *ver, u32 v_opcode,
case VIRTCHNL_OP_DEL_FDIR_FILTER:
valid_len = sizeof(struct virtchnl_fdir_del);
break;
+ case VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS:
+ break;
+ case VIRTCHNL_OP_ADD_VLAN_V2:
+ case VIRTCHNL_OP_DEL_VLAN_V2:
+ valid_len = sizeof(struct virtchnl_vlan_filter_list_v2);
+ if (msglen >= valid_len) {
+ struct virtchnl_vlan_filter_list_v2 *vfl =
+ (struct virtchnl_vlan_filter_list_v2 *)msg;
+
+ valid_len += (vfl->num_elements - 1) *
+ sizeof(struct virtchnl_vlan_filter);
+
+ if (vfl->num_elements == 0) {
+ err_msg_format = true;
+ break;
+ }
+ }
+ break;
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2:
+ case VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2:
+ valid_len = sizeof(struct virtchnl_vlan_setting);
+ break;
/* These are always errors coming from the VF. */
case VIRTCHNL_OP_EVENT:
case VIRTCHNL_OP_UNKNOWN:
diff --git a/include/linux/bootmem_info.h b/include/linux/bootmem_info.h
index 2bc8b1f..cc35d01 100644
--- a/include/linux/bootmem_info.h
+++ b/include/linux/bootmem_info.h
@@ -30,7 +30,7 @@ void put_page_bootmem(struct page *page);
*/
static inline void free_bootmem_page(struct page *page)
{
- unsigned long magic = (unsigned long)page->freelist;
+ unsigned long magic = page->index;
/*
* The reserve_bootmem_region sets the reserved flag on bootmem
diff --git a/include/linux/bpf-cgroup-defs.h b/include/linux/bpf-cgroup-defs.h
new file mode 100644
index 0000000..695d122
--- /dev/null
+++ b/include/linux/bpf-cgroup-defs.h
@@ -0,0 +1,70 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _BPF_CGROUP_DEFS_H
+#define _BPF_CGROUP_DEFS_H
+
+#ifdef CONFIG_CGROUP_BPF
+
+#include <linux/list.h>
+#include <linux/percpu-refcount.h>
+#include <linux/workqueue.h>
+
+struct bpf_prog_array;
+
+enum cgroup_bpf_attach_type {
+ CGROUP_BPF_ATTACH_TYPE_INVALID = -1,
+ CGROUP_INET_INGRESS = 0,
+ CGROUP_INET_EGRESS,
+ CGROUP_INET_SOCK_CREATE,
+ CGROUP_SOCK_OPS,
+ CGROUP_DEVICE,
+ CGROUP_INET4_BIND,
+ CGROUP_INET6_BIND,
+ CGROUP_INET4_CONNECT,
+ CGROUP_INET6_CONNECT,
+ CGROUP_INET4_POST_BIND,
+ CGROUP_INET6_POST_BIND,
+ CGROUP_UDP4_SENDMSG,
+ CGROUP_UDP6_SENDMSG,
+ CGROUP_SYSCTL,
+ CGROUP_UDP4_RECVMSG,
+ CGROUP_UDP6_RECVMSG,
+ CGROUP_GETSOCKOPT,
+ CGROUP_SETSOCKOPT,
+ CGROUP_INET4_GETPEERNAME,
+ CGROUP_INET6_GETPEERNAME,
+ CGROUP_INET4_GETSOCKNAME,
+ CGROUP_INET6_GETSOCKNAME,
+ CGROUP_INET_SOCK_RELEASE,
+ MAX_CGROUP_BPF_ATTACH_TYPE
+};
+
+struct cgroup_bpf {
+ /* array of effective progs in this cgroup */
+ struct bpf_prog_array __rcu *effective[MAX_CGROUP_BPF_ATTACH_TYPE];
+
+ /* attached progs to this cgroup and attach flags
+ * when flags == 0 or BPF_F_ALLOW_OVERRIDE the progs list will
+ * have either zero or one element
+ * when BPF_F_ALLOW_MULTI the list can have up to BPF_CGROUP_MAX_PROGS
+ */
+ struct list_head progs[MAX_CGROUP_BPF_ATTACH_TYPE];
+ u32 flags[MAX_CGROUP_BPF_ATTACH_TYPE];
+
+ /* list of cgroup shared storages */
+ struct list_head storages;
+
+ /* temp storage for effective prog array used by prog_attach/detach */
+ struct bpf_prog_array *inactive;
+
+ /* reference counter used to detach bpf programs after cgroup removal */
+ struct percpu_ref refcnt;
+
+ /* cgroup_bpf is released using a work queue */
+ struct work_struct release_work;
+};
+
+#else /* CONFIG_CGROUP_BPF */
+struct cgroup_bpf {};
+#endif /* CONFIG_CGROUP_BPF */
+
+#endif
diff --git a/include/linux/bpf-cgroup.h b/include/linux/bpf-cgroup.h
index 11820a4..b525d8c 100644
--- a/include/linux/bpf-cgroup.h
+++ b/include/linux/bpf-cgroup.h
@@ -3,10 +3,10 @@
#define _BPF_CGROUP_H
#include <linux/bpf.h>
+#include <linux/bpf-cgroup-defs.h>
#include <linux/errno.h>
#include <linux/jump_label.h>
#include <linux/percpu.h>
-#include <linux/percpu-refcount.h>
#include <linux/rbtree.h>
#include <uapi/linux/bpf.h>
@@ -23,33 +23,6 @@ struct ctl_table_header;
struct task_struct;
#ifdef CONFIG_CGROUP_BPF
-enum cgroup_bpf_attach_type {
- CGROUP_BPF_ATTACH_TYPE_INVALID = -1,
- CGROUP_INET_INGRESS = 0,
- CGROUP_INET_EGRESS,
- CGROUP_INET_SOCK_CREATE,
- CGROUP_SOCK_OPS,
- CGROUP_DEVICE,
- CGROUP_INET4_BIND,
- CGROUP_INET6_BIND,
- CGROUP_INET4_CONNECT,
- CGROUP_INET6_CONNECT,
- CGROUP_INET4_POST_BIND,
- CGROUP_INET6_POST_BIND,
- CGROUP_UDP4_SENDMSG,
- CGROUP_UDP6_SENDMSG,
- CGROUP_SYSCTL,
- CGROUP_UDP4_RECVMSG,
- CGROUP_UDP6_RECVMSG,
- CGROUP_GETSOCKOPT,
- CGROUP_SETSOCKOPT,
- CGROUP_INET4_GETPEERNAME,
- CGROUP_INET6_GETPEERNAME,
- CGROUP_INET4_GETSOCKNAME,
- CGROUP_INET6_GETSOCKNAME,
- CGROUP_INET_SOCK_RELEASE,
- MAX_CGROUP_BPF_ATTACH_TYPE
-};
#define CGROUP_ATYPE(type) \
case BPF_##type: return type
@@ -127,33 +100,6 @@ struct bpf_prog_list {
struct bpf_cgroup_storage *storage[MAX_BPF_CGROUP_STORAGE_TYPE];
};
-struct bpf_prog_array;
-
-struct cgroup_bpf {
- /* array of effective progs in this cgroup */
- struct bpf_prog_array __rcu *effective[MAX_CGROUP_BPF_ATTACH_TYPE];
-
- /* attached progs to this cgroup and attach flags
- * when flags == 0 or BPF_F_ALLOW_OVERRIDE the progs list will
- * have either zero or one element
- * when BPF_F_ALLOW_MULTI the list can have up to BPF_CGROUP_MAX_PROGS
- */
- struct list_head progs[MAX_CGROUP_BPF_ATTACH_TYPE];
- u32 flags[MAX_CGROUP_BPF_ATTACH_TYPE];
-
- /* list of cgroup shared storages */
- struct list_head storages;
-
- /* temp storage for effective prog array used by prog_attach/detach */
- struct bpf_prog_array *inactive;
-
- /* reference counter used to detach bpf programs after cgroup removal */
- struct percpu_ref refcnt;
-
- /* cgroup_bpf is released using a work queue */
- struct work_struct release_work;
-};
-
int cgroup_bpf_inherit(struct cgroup *cgrp);
void cgroup_bpf_offline(struct cgroup *cgrp);
@@ -451,7 +397,6 @@ int cgroup_bpf_prog_query(const union bpf_attr *attr,
union bpf_attr __user *uattr);
#else
-struct cgroup_bpf {};
static inline int cgroup_bpf_inherit(struct cgroup *cgrp) { return 0; }
static inline void cgroup_bpf_offline(struct cgroup *cgrp) {}
diff --git a/include/linux/bpf-netns.h b/include/linux/bpf-netns.h
index 722f799..413cfa5 100644
--- a/include/linux/bpf-netns.h
+++ b/include/linux/bpf-netns.h
@@ -3,15 +3,9 @@
#define _BPF_NETNS_H
#include <linux/mutex.h>
+#include <net/netns/bpf.h>
#include <uapi/linux/bpf.h>
-enum netns_bpf_attach_type {
- NETNS_BPF_INVALID = -1,
- NETNS_BPF_FLOW_DISSECTOR = 0,
- NETNS_BPF_SK_LOOKUP,
- MAX_NETNS_BPF_ATTACH_TYPE
-};
-
static inline enum netns_bpf_attach_type
to_netns_bpf_attach_type(enum bpf_attach_type attach_type)
{
diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index 755f38e..6e947cd 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -297,6 +297,34 @@ bool bpf_map_meta_equal(const struct bpf_map *meta0,
extern const struct bpf_map_ops bpf_map_offload_ops;
+/* bpf_type_flag contains a set of flags that are applicable to the values of
+ * arg_type, ret_type and reg_type. For example, a pointer value may be null,
+ * or a memory is read-only. We classify types into two categories: base types
+ * and extended types. Extended types are base types combined with a type flag.
+ *
+ * Currently there are no more than 32 base types in arg_type, ret_type and
+ * reg_types.
+ */
+#define BPF_BASE_TYPE_BITS 8
+
+enum bpf_type_flag {
+ /* PTR may be NULL. */
+ PTR_MAYBE_NULL = BIT(0 + BPF_BASE_TYPE_BITS),
+
+ /* MEM is read-only. When applied on bpf_arg, it indicates the arg is
+ * compatible with both mutable and immutable memory.
+ */
+ MEM_RDONLY = BIT(1 + BPF_BASE_TYPE_BITS),
+
+ __BPF_TYPE_LAST_FLAG = MEM_RDONLY,
+};
+
+/* Max number of base types. */
+#define BPF_BASE_TYPE_LIMIT (1UL << BPF_BASE_TYPE_BITS)
+
+/* Max number of all types. */
+#define BPF_TYPE_LIMIT (__BPF_TYPE_LAST_FLAG | (__BPF_TYPE_LAST_FLAG - 1))
+
/* function argument constraints */
enum bpf_arg_type {
ARG_DONTCARE = 0, /* unused argument in helper function */
@@ -308,13 +336,11 @@ enum bpf_arg_type {
ARG_PTR_TO_MAP_KEY, /* pointer to stack used as map key */
ARG_PTR_TO_MAP_VALUE, /* pointer to stack used as map value */
ARG_PTR_TO_UNINIT_MAP_VALUE, /* pointer to valid memory used to store a map value */
- ARG_PTR_TO_MAP_VALUE_OR_NULL, /* pointer to stack used as map value or NULL */
/* the following constraints used to prototype bpf_memcmp() and other
* functions that access data on eBPF program stack
*/
ARG_PTR_TO_MEM, /* pointer to valid memory (stack, packet, map value) */
- ARG_PTR_TO_MEM_OR_NULL, /* pointer to valid memory or NULL */
ARG_PTR_TO_UNINIT_MEM, /* pointer to memory does not need to be initialized,
* helper function must fill all bytes or clear
* them in error case.
@@ -324,42 +350,65 @@ enum bpf_arg_type {
ARG_CONST_SIZE_OR_ZERO, /* number of bytes accessed from memory or 0 */
ARG_PTR_TO_CTX, /* pointer to context */
- ARG_PTR_TO_CTX_OR_NULL, /* pointer to context or NULL */
ARG_ANYTHING, /* any (initialized) argument is ok */
ARG_PTR_TO_SPIN_LOCK, /* pointer to bpf_spin_lock */
ARG_PTR_TO_SOCK_COMMON, /* pointer to sock_common */
ARG_PTR_TO_INT, /* pointer to int */
ARG_PTR_TO_LONG, /* pointer to long */
ARG_PTR_TO_SOCKET, /* pointer to bpf_sock (fullsock) */
- ARG_PTR_TO_SOCKET_OR_NULL, /* pointer to bpf_sock (fullsock) or NULL */
ARG_PTR_TO_BTF_ID, /* pointer to in-kernel struct */
ARG_PTR_TO_ALLOC_MEM, /* pointer to dynamically allocated memory */
- ARG_PTR_TO_ALLOC_MEM_OR_NULL, /* pointer to dynamically allocated memory or NULL */
ARG_CONST_ALLOC_SIZE_OR_ZERO, /* number of allocated bytes requested */
ARG_PTR_TO_BTF_ID_SOCK_COMMON, /* pointer to in-kernel sock_common or bpf-mirrored bpf_sock */
ARG_PTR_TO_PERCPU_BTF_ID, /* pointer to in-kernel percpu type */
ARG_PTR_TO_FUNC, /* pointer to a bpf program function */
- ARG_PTR_TO_STACK_OR_NULL, /* pointer to stack or NULL */
+ ARG_PTR_TO_STACK, /* pointer to stack */
ARG_PTR_TO_CONST_STR, /* pointer to a null terminated read-only string */
ARG_PTR_TO_TIMER, /* pointer to bpf_timer */
__BPF_ARG_TYPE_MAX,
+
+ /* Extended arg_types. */
+ ARG_PTR_TO_MAP_VALUE_OR_NULL = PTR_MAYBE_NULL | ARG_PTR_TO_MAP_VALUE,
+ ARG_PTR_TO_MEM_OR_NULL = PTR_MAYBE_NULL | ARG_PTR_TO_MEM,
+ ARG_PTR_TO_CTX_OR_NULL = PTR_MAYBE_NULL | ARG_PTR_TO_CTX,
+ ARG_PTR_TO_SOCKET_OR_NULL = PTR_MAYBE_NULL | ARG_PTR_TO_SOCKET,
+ ARG_PTR_TO_ALLOC_MEM_OR_NULL = PTR_MAYBE_NULL | ARG_PTR_TO_ALLOC_MEM,
+ ARG_PTR_TO_STACK_OR_NULL = PTR_MAYBE_NULL | ARG_PTR_TO_STACK,
+
+ /* This must be the last entry. Its purpose is to ensure the enum is
+ * wide enough to hold the higher bits reserved for bpf_type_flag.
+ */
+ __BPF_ARG_TYPE_LIMIT = BPF_TYPE_LIMIT,
};
+static_assert(__BPF_ARG_TYPE_MAX <= BPF_BASE_TYPE_LIMIT);
/* type of values returned from helper functions */
enum bpf_return_type {
RET_INTEGER, /* function returns integer */
RET_VOID, /* function doesn't return anything */
RET_PTR_TO_MAP_VALUE, /* returns a pointer to map elem value */
- RET_PTR_TO_MAP_VALUE_OR_NULL, /* returns a pointer to map elem value or NULL */
- RET_PTR_TO_SOCKET_OR_NULL, /* returns a pointer to a socket or NULL */
- RET_PTR_TO_TCP_SOCK_OR_NULL, /* returns a pointer to a tcp_sock or NULL */
- RET_PTR_TO_SOCK_COMMON_OR_NULL, /* returns a pointer to a sock_common or NULL */
- RET_PTR_TO_ALLOC_MEM_OR_NULL, /* returns a pointer to dynamically allocated memory or NULL */
- RET_PTR_TO_BTF_ID_OR_NULL, /* returns a pointer to a btf_id or NULL */
- RET_PTR_TO_MEM_OR_BTF_ID_OR_NULL, /* returns a pointer to a valid memory or a btf_id or NULL */
+ RET_PTR_TO_SOCKET, /* returns a pointer to a socket */
+ RET_PTR_TO_TCP_SOCK, /* returns a pointer to a tcp_sock */
+ RET_PTR_TO_SOCK_COMMON, /* returns a pointer to a sock_common */
+ RET_PTR_TO_ALLOC_MEM, /* returns a pointer to dynamically allocated memory */
RET_PTR_TO_MEM_OR_BTF_ID, /* returns a pointer to a valid memory or a btf_id */
RET_PTR_TO_BTF_ID, /* returns a pointer to a btf_id */
+ __BPF_RET_TYPE_MAX,
+
+ /* Extended ret_types. */
+ RET_PTR_TO_MAP_VALUE_OR_NULL = PTR_MAYBE_NULL | RET_PTR_TO_MAP_VALUE,
+ RET_PTR_TO_SOCKET_OR_NULL = PTR_MAYBE_NULL | RET_PTR_TO_SOCKET,
+ RET_PTR_TO_TCP_SOCK_OR_NULL = PTR_MAYBE_NULL | RET_PTR_TO_TCP_SOCK,
+ RET_PTR_TO_SOCK_COMMON_OR_NULL = PTR_MAYBE_NULL | RET_PTR_TO_SOCK_COMMON,
+ RET_PTR_TO_ALLOC_MEM_OR_NULL = PTR_MAYBE_NULL | RET_PTR_TO_ALLOC_MEM,
+ RET_PTR_TO_BTF_ID_OR_NULL = PTR_MAYBE_NULL | RET_PTR_TO_BTF_ID,
+
+ /* This must be the last entry. Its purpose is to ensure the enum is
+ * wide enough to hold the higher bits reserved for bpf_type_flag.
+ */
+ __BPF_RET_TYPE_LIMIT = BPF_TYPE_LIMIT,
};
+static_assert(__BPF_RET_TYPE_MAX <= BPF_BASE_TYPE_LIMIT);
/* eBPF function prototype used by verifier to allow BPF_CALLs from eBPF programs
* to in-kernel helper functions and for adjusting imm32 field in BPF_CALL
@@ -421,18 +470,15 @@ enum bpf_reg_type {
PTR_TO_CTX, /* reg points to bpf_context */
CONST_PTR_TO_MAP, /* reg points to struct bpf_map */
PTR_TO_MAP_VALUE, /* reg points to map element value */
- PTR_TO_MAP_VALUE_OR_NULL,/* points to map elem value or NULL */
+ PTR_TO_MAP_KEY, /* reg points to a map element key */
PTR_TO_STACK, /* reg == frame_pointer + offset */
PTR_TO_PACKET_META, /* skb->data - meta_len */
PTR_TO_PACKET, /* reg points to skb->data */
PTR_TO_PACKET_END, /* skb->data + headlen */
PTR_TO_FLOW_KEYS, /* reg points to bpf_flow_keys */
PTR_TO_SOCKET, /* reg points to struct bpf_sock */
- PTR_TO_SOCKET_OR_NULL, /* reg points to struct bpf_sock or NULL */
PTR_TO_SOCK_COMMON, /* reg points to sock_common */
- PTR_TO_SOCK_COMMON_OR_NULL, /* reg points to sock_common or NULL */
PTR_TO_TCP_SOCK, /* reg points to struct tcp_sock */
- PTR_TO_TCP_SOCK_OR_NULL, /* reg points to struct tcp_sock or NULL */
PTR_TO_TP_BUFFER, /* reg points to a writable raw tp's buffer */
PTR_TO_XDP_SOCK, /* reg points to struct xdp_sock */
/* PTR_TO_BTF_ID points to a kernel struct that does not need
@@ -450,18 +496,25 @@ enum bpf_reg_type {
* been checked for null. Used primarily to inform the verifier
* an explicit null check is required for this struct.
*/
- PTR_TO_BTF_ID_OR_NULL,
PTR_TO_MEM, /* reg points to valid memory region */
- PTR_TO_MEM_OR_NULL, /* reg points to valid memory region or NULL */
- PTR_TO_RDONLY_BUF, /* reg points to a readonly buffer */
- PTR_TO_RDONLY_BUF_OR_NULL, /* reg points to a readonly buffer or NULL */
- PTR_TO_RDWR_BUF, /* reg points to a read/write buffer */
- PTR_TO_RDWR_BUF_OR_NULL, /* reg points to a read/write buffer or NULL */
+ PTR_TO_BUF, /* reg points to a read/write buffer */
PTR_TO_PERCPU_BTF_ID, /* reg points to a percpu kernel variable */
PTR_TO_FUNC, /* reg points to a bpf program function */
- PTR_TO_MAP_KEY, /* reg points to a map element key */
__BPF_REG_TYPE_MAX,
+
+ /* Extended reg_types. */
+ PTR_TO_MAP_VALUE_OR_NULL = PTR_MAYBE_NULL | PTR_TO_MAP_VALUE,
+ PTR_TO_SOCKET_OR_NULL = PTR_MAYBE_NULL | PTR_TO_SOCKET,
+ PTR_TO_SOCK_COMMON_OR_NULL = PTR_MAYBE_NULL | PTR_TO_SOCK_COMMON,
+ PTR_TO_TCP_SOCK_OR_NULL = PTR_MAYBE_NULL | PTR_TO_TCP_SOCK,
+ PTR_TO_BTF_ID_OR_NULL = PTR_MAYBE_NULL | PTR_TO_BTF_ID,
+
+ /* This must be the last entry. Its purpose is to ensure the enum is
+ * wide enough to hold the higher bits reserved for bpf_type_flag.
+ */
+ __BPF_REG_TYPE_LIMIT = BPF_TYPE_LIMIT,
};
+static_assert(__BPF_REG_TYPE_MAX <= BPF_BASE_TYPE_LIMIT);
/* The information passed from prog-specific *_is_valid_access
* back to the verifier.
@@ -777,6 +830,7 @@ void bpf_ksym_add(struct bpf_ksym *ksym);
void bpf_ksym_del(struct bpf_ksym *ksym);
int bpf_jit_charge_modmem(u32 pages);
void bpf_jit_uncharge_modmem(u32 pages);
+bool bpf_prog_has_trampoline(const struct bpf_prog *prog);
#else
static inline int bpf_trampoline_link_prog(struct bpf_prog *prog,
struct bpf_trampoline *tr)
@@ -805,6 +859,10 @@ static inline bool is_bpf_image_address(unsigned long address)
{
return false;
}
+static inline bool bpf_prog_has_trampoline(const struct bpf_prog *prog)
+{
+ return false;
+}
#endif
struct bpf_func_info_aux {
@@ -1082,7 +1140,7 @@ struct bpf_array {
};
#define BPF_COMPLEXITY_LIMIT_INSNS 1000000 /* yes. 1M insns */
-#define MAX_TAIL_CALL_CNT 32
+#define MAX_TAIL_CALL_CNT 33
#define BPF_F_ACCESS_MASK (BPF_F_RDONLY | \
BPF_F_RDONLY_PROG | \
@@ -1611,17 +1669,17 @@ void bpf_patch_call_args(struct bpf_insn *insn, u32 stack_depth);
struct btf *bpf_get_btf_vmlinux(void);
/* Map specifics */
-struct xdp_buff;
+struct xdp_frame;
struct sk_buff;
struct bpf_dtab_netdev;
struct bpf_cpu_map_entry;
void __dev_flush(void);
-int dev_xdp_enqueue(struct net_device *dev, struct xdp_buff *xdp,
+int dev_xdp_enqueue(struct net_device *dev, struct xdp_frame *xdpf,
struct net_device *dev_rx);
-int dev_map_enqueue(struct bpf_dtab_netdev *dst, struct xdp_buff *xdp,
+int dev_map_enqueue(struct bpf_dtab_netdev *dst, struct xdp_frame *xdpf,
struct net_device *dev_rx);
-int dev_map_enqueue_multi(struct xdp_buff *xdp, struct net_device *dev_rx,
+int dev_map_enqueue_multi(struct xdp_frame *xdpf, struct net_device *dev_rx,
struct bpf_map *map, bool exclude_ingress);
int dev_map_generic_redirect(struct bpf_dtab_netdev *dst, struct sk_buff *skb,
struct bpf_prog *xdp_prog);
@@ -1630,7 +1688,7 @@ int dev_map_redirect_multi(struct net_device *dev, struct sk_buff *skb,
bool exclude_ingress);
void __cpu_map_flush(void);
-int cpu_map_enqueue(struct bpf_cpu_map_entry *rcpu, struct xdp_buff *xdp,
+int cpu_map_enqueue(struct bpf_cpu_map_entry *rcpu, struct xdp_frame *xdpf,
struct net_device *dev_rx);
int cpu_map_generic_redirect(struct bpf_cpu_map_entry *rcpu,
struct sk_buff *skb);
@@ -1722,6 +1780,14 @@ bool bpf_prog_has_kfunc_call(const struct bpf_prog *prog);
const struct btf_func_model *
bpf_jit_find_kfunc_model(const struct bpf_prog *prog,
const struct bpf_insn *insn);
+struct bpf_core_ctx {
+ struct bpf_verifier_log *log;
+ const struct btf *btf;
+};
+
+int bpf_core_apply(struct bpf_core_ctx *ctx, const struct bpf_core_relo *relo,
+ int relo_idx, void *insn);
+
#else /* !CONFIG_BPF_SYSCALL */
static inline struct bpf_prog *bpf_prog_get(u32 ufd)
{
@@ -1800,26 +1866,26 @@ static inline void __dev_flush(void)
{
}
-struct xdp_buff;
+struct xdp_frame;
struct bpf_dtab_netdev;
struct bpf_cpu_map_entry;
static inline
-int dev_xdp_enqueue(struct net_device *dev, struct xdp_buff *xdp,
+int dev_xdp_enqueue(struct net_device *dev, struct xdp_frame *xdpf,
struct net_device *dev_rx)
{
return 0;
}
static inline
-int dev_map_enqueue(struct bpf_dtab_netdev *dst, struct xdp_buff *xdp,
+int dev_map_enqueue(struct bpf_dtab_netdev *dst, struct xdp_frame *xdpf,
struct net_device *dev_rx)
{
return 0;
}
static inline
-int dev_map_enqueue_multi(struct xdp_buff *xdp, struct net_device *dev_rx,
+int dev_map_enqueue_multi(struct xdp_frame *xdpf, struct net_device *dev_rx,
struct bpf_map *map, bool exclude_ingress)
{
return 0;
@@ -1847,7 +1913,7 @@ static inline void __cpu_map_flush(void)
}
static inline int cpu_map_enqueue(struct bpf_cpu_map_entry *rcpu,
- struct xdp_buff *xdp,
+ struct xdp_frame *xdpf,
struct net_device *dev_rx)
{
return 0;
@@ -2153,6 +2219,9 @@ extern const struct bpf_func_proto bpf_btf_find_by_name_kind_proto;
extern const struct bpf_func_proto bpf_sk_setsockopt_proto;
extern const struct bpf_func_proto bpf_sk_getsockopt_proto;
extern const struct bpf_func_proto bpf_kallsyms_lookup_name_proto;
+extern const struct bpf_func_proto bpf_find_vma_proto;
+extern const struct bpf_func_proto bpf_loop_proto;
+extern const struct bpf_func_proto bpf_strncmp_proto;
const struct bpf_func_proto *tracing_prog_func_proto(
enum bpf_func_id func_id, const struct bpf_prog *prog);
diff --git a/include/linux/bpf_local_storage.h b/include/linux/bpf_local_storage.h
index 24496bc..37b3906 100644
--- a/include/linux/bpf_local_storage.h
+++ b/include/linux/bpf_local_storage.h
@@ -8,6 +8,7 @@
#define _BPF_LOCAL_STORAGE_H
#include <linux/bpf.h>
+#include <linux/filter.h>
#include <linux/rculist.h>
#include <linux/list.h>
#include <linux/hash.h>
@@ -16,6 +17,9 @@
#define BPF_LOCAL_STORAGE_CACHE_SIZE 16
+#define bpf_rcu_lock_held() \
+ (rcu_read_lock_held() || rcu_read_lock_trace_held() || \
+ rcu_read_lock_bh_held())
struct bpf_local_storage_map_bucket {
struct hlist_head list;
raw_spinlock_t lock;
@@ -161,4 +165,6 @@ struct bpf_local_storage_data *
bpf_local_storage_update(void *owner, struct bpf_local_storage_map *smap,
void *value, u64 map_flags);
+void bpf_local_storage_free_rcu(struct rcu_head *rcu);
+
#endif /* _BPF_LOCAL_STORAGE_H */
diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
index c8a78e8..143401d 100644
--- a/include/linux/bpf_verifier.h
+++ b/include/linux/bpf_verifier.h
@@ -18,6 +18,8 @@
* that converting umax_value to int cannot overflow.
*/
#define BPF_MAX_VAR_SIZ (1 << 29)
+/* size of type_str_buf in bpf_verifier. */
+#define TYPE_STR_BUF_LEN 64
/* Liveness marks, used for registers and spilled-regs (in stack slots).
* Read marks propagate upwards until they find a write mark; they record that
@@ -388,6 +390,8 @@ static inline bool bpf_verifier_log_full(const struct bpf_verifier_log *log)
#define BPF_LOG_LEVEL (BPF_LOG_LEVEL1 | BPF_LOG_LEVEL2)
#define BPF_LOG_MASK (BPF_LOG_LEVEL | BPF_LOG_STATS)
#define BPF_LOG_KERNEL (BPF_LOG_MASK + 1) /* kernel internal flag */
+#define BPF_LOG_MIN_ALIGNMENT 8U
+#define BPF_LOG_ALIGNMENT 40U
static inline bool bpf_verifier_log_needed(const struct bpf_verifier_log *log)
{
@@ -396,6 +400,13 @@ static inline bool bpf_verifier_log_needed(const struct bpf_verifier_log *log)
log->level == BPF_LOG_KERNEL);
}
+static inline bool
+bpf_verifier_log_attr_valid(const struct bpf_verifier_log *log)
+{
+ return log->len_total >= 128 && log->len_total <= UINT_MAX >> 2 &&
+ log->level && log->ubuf && !(log->level & ~BPF_LOG_MASK);
+}
+
#define BPF_MAX_SUBPROGS 256
struct bpf_subprog_info {
@@ -467,6 +478,16 @@ struct bpf_verifier_env {
/* longest register parentage chain walked for liveness marking */
u32 longest_mark_read_walk;
bpfptr_t fd_array;
+
+ /* bit mask to keep track of whether a register has been accessed
+ * since the last time the function state was printed
+ */
+ u32 scratched_regs;
+ /* Same as scratched_regs but for stack slots */
+ u64 scratched_stack_slots;
+ u32 prev_log_len, prev_insn_print_len;
+ /* buffer used in reg_type_str() to generate reg_type string */
+ char type_str_buf[TYPE_STR_BUF_LEN];
};
__printf(2, 0) void bpf_verifier_vlog(struct bpf_verifier_log *log,
@@ -529,5 +550,18 @@ int bpf_check_attach_target(struct bpf_verifier_log *log,
struct bpf_attach_target_info *tgt_info);
void bpf_free_kfunc_btf_tab(struct bpf_kfunc_btf_tab *tab);
+#define BPF_BASE_TYPE_MASK GENMASK(BPF_BASE_TYPE_BITS - 1, 0)
+
+/* extract base type from bpf_{arg, return, reg}_type. */
+static inline u32 base_type(u32 type)
+{
+ return type & BPF_BASE_TYPE_MASK;
+}
+
+/* extract flags from an extended type. See bpf_type_flag in bpf.h. */
+static inline u32 type_flag(u32 type)
+{
+ return type & ~BPF_BASE_TYPE_MASK;
+}
#endif /* _LINUX_BPF_VERIFIER_H */
diff --git a/include/linux/btf.h b/include/linux/btf.h
index 0e1b628..0c74348 100644
--- a/include/linux/btf.h
+++ b/include/linux/btf.h
@@ -144,6 +144,53 @@ static inline bool btf_type_is_enum(const struct btf_type *t)
return BTF_INFO_KIND(t->info) == BTF_KIND_ENUM;
}
+static inline bool str_is_empty(const char *s)
+{
+ return !s || !s[0];
+}
+
+static inline u16 btf_kind(const struct btf_type *t)
+{
+ return BTF_INFO_KIND(t->info);
+}
+
+static inline bool btf_is_enum(const struct btf_type *t)
+{
+ return btf_kind(t) == BTF_KIND_ENUM;
+}
+
+static inline bool btf_is_composite(const struct btf_type *t)
+{
+ u16 kind = btf_kind(t);
+
+ return kind == BTF_KIND_STRUCT || kind == BTF_KIND_UNION;
+}
+
+static inline bool btf_is_array(const struct btf_type *t)
+{
+ return btf_kind(t) == BTF_KIND_ARRAY;
+}
+
+static inline bool btf_is_int(const struct btf_type *t)
+{
+ return btf_kind(t) == BTF_KIND_INT;
+}
+
+static inline bool btf_is_ptr(const struct btf_type *t)
+{
+ return btf_kind(t) == BTF_KIND_PTR;
+}
+
+static inline u8 btf_int_offset(const struct btf_type *t)
+{
+ return BTF_INT_OFFSET(*(u32 *)(t + 1));
+}
+
+static inline u8 btf_int_encoding(const struct btf_type *t)
+{
+ return BTF_INT_ENCODING(*(u32 *)(t + 1));
+}
+
static inline bool btf_type_is_scalar(const struct btf_type *t)
{
return btf_type_is_int(t) || btf_type_is_enum(t);
@@ -184,6 +231,11 @@ static inline u16 btf_type_vlen(const struct btf_type *t)
return BTF_INFO_VLEN(t->info);
}
+static inline u16 btf_vlen(const struct btf_type *t)
+{
+ return btf_type_vlen(t);
+}
+
static inline u16 btf_func_linkage(const struct btf_type *t)
{
return BTF_INFO_VLEN(t->info);
@@ -194,25 +246,54 @@ static inline bool btf_type_kflag(const struct btf_type *t)
return BTF_INFO_KFLAG(t->info);
}
-static inline u32 btf_member_bit_offset(const struct btf_type *struct_type,
- const struct btf_member *member)
+static inline u32 __btf_member_bit_offset(const struct btf_type *struct_type,
+ const struct btf_member *member)
{
return btf_type_kflag(struct_type) ? BTF_MEMBER_BIT_OFFSET(member->offset)
: member->offset;
}
-static inline u32 btf_member_bitfield_size(const struct btf_type *struct_type,
- const struct btf_member *member)
+static inline u32 __btf_member_bitfield_size(const struct btf_type *struct_type,
+ const struct btf_member *member)
{
return btf_type_kflag(struct_type) ? BTF_MEMBER_BITFIELD_SIZE(member->offset)
: 0;
}
+static inline struct btf_member *btf_members(const struct btf_type *t)
+{
+ return (struct btf_member *)(t + 1);
+}
+
+static inline u32 btf_member_bit_offset(const struct btf_type *t, u32 member_idx)
+{
+ const struct btf_member *m = btf_members(t) + member_idx;
+
+ return __btf_member_bit_offset(t, m);
+}
+
+static inline u32 btf_member_bitfield_size(const struct btf_type *t, u32 member_idx)
+{
+ const struct btf_member *m = btf_members(t) + member_idx;
+
+ return __btf_member_bitfield_size(t, m);
+}
+
static inline const struct btf_member *btf_type_member(const struct btf_type *t)
{
return (const struct btf_member *)(t + 1);
}
+static inline struct btf_array *btf_array(const struct btf_type *t)
+{
+ return (struct btf_array *)(t + 1);
+}
+
+static inline struct btf_enum *btf_enum(const struct btf_type *t)
+{
+ return (struct btf_enum *)(t + 1);
+}
+
static inline const struct btf_var_secinfo *btf_type_var_secinfo(
const struct btf_type *t)
{
diff --git a/include/linux/btf_ids.h b/include/linux/btf_ids.h
index 47d9abf..919c0fd 100644
--- a/include/linux/btf_ids.h
+++ b/include/linux/btf_ids.h
@@ -73,7 +73,7 @@ asm( \
__BTF_ID_LIST(name, local) \
extern u32 name[];
-#define BTF_ID_LIST_GLOBAL(name) \
+#define BTF_ID_LIST_GLOBAL(name, n) \
__BTF_ID_LIST(name, globl)
/* The BTF_ID_LIST_SINGLE macro defines a BTF_ID_LIST with
@@ -83,7 +83,7 @@ __BTF_ID_LIST(name, globl)
BTF_ID_LIST(name) \
BTF_ID(prefix, typename)
#define BTF_ID_LIST_GLOBAL_SINGLE(name, prefix, typename) \
- BTF_ID_LIST_GLOBAL(name) \
+ BTF_ID_LIST_GLOBAL(name, 1) \
BTF_ID(prefix, typename)
/*
@@ -149,7 +149,7 @@ extern struct btf_id_set name;
#define BTF_ID_LIST(name) static u32 name[5];
#define BTF_ID(prefix, name)
#define BTF_ID_UNUSED
-#define BTF_ID_LIST_GLOBAL(name) u32 name[1];
+#define BTF_ID_LIST_GLOBAL(name, n) u32 name[n];
#define BTF_ID_LIST_SINGLE(name, prefix, typename) static u32 name[1];
#define BTF_ID_LIST_GLOBAL_SINGLE(name, prefix, typename) u32 name[1];
#define BTF_SET_START(name) static struct btf_id_set name = { 0 };
@@ -189,6 +189,18 @@ MAX_BTF_SOCK_TYPE,
extern u32 btf_sock_ids[];
#endif
-extern u32 btf_task_struct_ids[];
+#define BTF_TRACING_TYPE_xxx \
+ BTF_TRACING_TYPE(BTF_TRACING_TYPE_TASK, task_struct) \
+ BTF_TRACING_TYPE(BTF_TRACING_TYPE_FILE, file) \
+ BTF_TRACING_TYPE(BTF_TRACING_TYPE_VMA, vm_area_struct)
+
+enum {
+#define BTF_TRACING_TYPE(name, type) name,
+BTF_TRACING_TYPE_xxx
+#undef BTF_TRACING_TYPE
+MAX_BTF_TRACING_TYPE,
+};
+
+extern u32 btf_tracing_ids[];
#endif
diff --git a/include/linux/can/bittiming.h b/include/linux/can/bittiming.h
index 20b50ba..a81652d 100644
--- a/include/linux/can/bittiming.h
+++ b/include/linux/can/bittiming.h
@@ -12,13 +12,6 @@
#define CAN_SYNC_SEG 1
-/* Kilobits and Megabits per second */
-#define CAN_KBPS 1000UL
-#define CAN_MBPS 1000000UL
-
-/* Megahertz */
-#define CAN_MHZ 1000000UL
-
#define CAN_CTRLMODE_TDC_MASK \
(CAN_CTRLMODE_TDC_AUTO | CAN_CTRLMODE_TDC_MANUAL)
diff --git a/include/linux/can/dev.h b/include/linux/can/dev.h
index 45f19d9..c2ea47f 100644
--- a/include/linux/can/dev.h
+++ b/include/linux/can/dev.h
@@ -64,12 +64,14 @@ struct can_priv {
struct gpio_desc *termination_gpio;
u16 termination_gpio_ohms[CAN_TERMINATION_GPIO_MAX];
+ unsigned int echo_skb_max;
+ struct sk_buff **echo_skb;
+
enum can_state state;
/* CAN controller features - see include/uapi/linux/can/netlink.h */
u32 ctrlmode; /* current options setting */
u32 ctrlmode_supported; /* options that can be modified by netlink */
- u32 ctrlmode_static; /* static enabled options for driver/hardware */
int restart_ms;
struct delayed_work restart_work;
@@ -84,9 +86,6 @@ struct can_priv {
struct can_berr_counter *bec);
int (*do_get_auto_tdcv)(const struct net_device *dev, u32 *tdcv);
- unsigned int echo_skb_max;
- struct sk_buff **echo_skb;
-
#ifdef CONFIG_CAN_LEDS
struct led_trigger *tx_led_trig;
char tx_led_trig_name[CAN_LED_NAME_SZ];
@@ -132,18 +131,29 @@ static inline s32 can_get_relative_tdco(const struct can_priv *priv)
}
/* helper to define static CAN controller features at device creation time */
-static inline void can_set_static_ctrlmode(struct net_device *dev,
- u32 static_mode)
+static inline int __must_check can_set_static_ctrlmode(struct net_device *dev,
+ u32 static_mode)
{
struct can_priv *priv = netdev_priv(dev);
/* alloc_candev() succeeded => netdev_priv() is valid at this point */
+ if (priv->ctrlmode_supported & static_mode) {
+ netdev_warn(dev,
+ "Controller features can not be supported and static at the same time\n");
+ return -EINVAL;
+ }
priv->ctrlmode = static_mode;
- priv->ctrlmode_static = static_mode;
/* override MTU which was set by default in can_setup()? */
if (static_mode & CAN_CTRLMODE_FD)
dev->mtu = CANFD_MTU;
+
+ return 0;
+}
+
+static inline u32 can_get_static_ctrlmode(struct can_priv *priv)
+{
+ return priv->ctrlmode & ~priv->ctrlmode_supported;
}
void can_setup(struct net_device *dev);
diff --git a/include/linux/can/skb.h b/include/linux/can/skb.h
index d311bc3..fdb22b0 100644
--- a/include/linux/can/skb.h
+++ b/include/linux/can/skb.h
@@ -21,8 +21,9 @@ int can_put_echo_skb(struct sk_buff *skb, struct net_device *dev,
unsigned int idx, unsigned int frame_len);
struct sk_buff *__can_get_echo_skb(struct net_device *dev, unsigned int idx,
u8 *len_ptr, unsigned int *frame_len_ptr);
-unsigned int can_get_echo_skb(struct net_device *dev, unsigned int idx,
- unsigned int *frame_len_ptr);
+unsigned int __must_check can_get_echo_skb(struct net_device *dev,
+ unsigned int idx,
+ unsigned int *frame_len_ptr);
void can_free_echo_skb(struct net_device *dev, unsigned int idx,
unsigned int *frame_len_ptr);
struct sk_buff *alloc_can_skb(struct net_device *dev, struct can_frame **cf);
diff --git a/include/linux/cc_platform.h b/include/linux/cc_platform.h
index a075b70..efd8205 100644
--- a/include/linux/cc_platform.h
+++ b/include/linux/cc_platform.h
@@ -61,6 +61,17 @@ enum cc_attr {
* Examples include SEV-ES.
*/
CC_ATTR_GUEST_STATE_ENCRYPT,
+
+ /**
+ * @CC_ATTR_GUEST_UNROLL_STRING_IO: String I/O is implemented with
+ * IN/OUT instructions
+ *
+ * The platform/OS is running as a guest/virtual machine and uses
+ * IN/OUT instructions in place of string I/O.
+ *
+ * Examples include TDX guest & SEV.
+ */
+ CC_ATTR_GUEST_UNROLL_STRING_IO,
};
#ifdef CONFIG_ARCH_HAS_CC_PLATFORM
diff --git a/include/linux/cgroup-defs.h b/include/linux/cgroup-defs.h
index db2e147..411684c 100644
--- a/include/linux/cgroup-defs.h
+++ b/include/linux/cgroup-defs.h
@@ -19,7 +19,7 @@
#include <linux/percpu-rwsem.h>
#include <linux/u64_stats_sync.h>
#include <linux/workqueue.h>
-#include <linux/bpf-cgroup.h>
+#include <linux/bpf-cgroup-defs.h>
#include <linux/psi_types.h>
#ifdef CONFIG_CGROUPS
diff --git a/include/linux/console.h b/include/linux/console.h
index a97f277..7cd758a 100644
--- a/include/linux/console.h
+++ b/include/linux/console.h
@@ -219,12 +219,6 @@ extern atomic_t ignore_console_lock_warning;
#define VESA_HSYNC_SUSPEND 2
#define VESA_POWERDOWN 3
-#ifdef CONFIG_VGA_CONSOLE
-extern bool vgacon_text_force(void);
-#else
-static inline bool vgacon_text_force(void) { return false; }
-#endif
-
extern void console_init(void);
/* For deferred console takeover */
diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
index 773c837..411a428 100644
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -225,6 +225,7 @@ enum cpuhp_state {
CPUHP_AP_PERF_ARM_HISI_L3_ONLINE,
CPUHP_AP_PERF_ARM_HISI_PA_ONLINE,
CPUHP_AP_PERF_ARM_HISI_SLLC_ONLINE,
+ CPUHP_AP_PERF_ARM_HISI_PCIE_PMU_ONLINE,
CPUHP_AP_PERF_ARM_L2X0_ONLINE,
CPUHP_AP_PERF_ARM_QCOM_L2_ONLINE,
CPUHP_AP_PERF_ARM_QCOM_L3_ONLINE,
diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h
index 42a323a..7ab5007 100644
--- a/include/linux/dma-buf.h
+++ b/include/linux/dma-buf.h
@@ -433,8 +433,8 @@ struct dma_buf {
/** @poll: for userspace poll support */
wait_queue_head_t poll;
- /** @cb_excl: for userspace poll support */
- /** @cb_shared: for userspace poll support */
+ /** @cb_in: for userspace poll support */
+ /** @cb_out: for userspace poll support */
struct dma_buf_poll_cb_t {
struct dma_fence_cb cb;
wait_queue_head_t *poll;
diff --git a/include/linux/dma-fence.h b/include/linux/dma-fence.h
index a706b7b..1ea6917 100644
--- a/include/linux/dma-fence.h
+++ b/include/linux/dma-fence.h
@@ -264,6 +264,7 @@ void dma_fence_init(struct dma_fence *fence, const struct dma_fence_ops *ops,
void dma_fence_release(struct kref *kref);
void dma_fence_free(struct dma_fence *fence);
+void dma_fence_describe(struct dma_fence *fence, struct seq_file *seq);
/**
* dma_fence_put - decreases refcount of the fence
diff --git a/include/linux/dma-resv.h b/include/linux/dma-resv.h
index dbd235a..eebf043 100644
--- a/include/linux/dma-resv.h
+++ b/include/linux/dma-resv.h
@@ -441,32 +441,6 @@ dma_resv_excl_fence(struct dma_resv *obj)
}
/**
- * dma_resv_get_excl_unlocked - get the reservation object's
- * exclusive fence, without lock held.
- * @obj: the reservation object
- *
- * If there is an exclusive fence, this atomically increments it's
- * reference count and returns it.
- *
- * RETURNS
- * The exclusive fence or NULL if none
- */
-static inline struct dma_fence *
-dma_resv_get_excl_unlocked(struct dma_resv *obj)
-{
- struct dma_fence *fence;
-
- if (!rcu_access_pointer(obj->fence_excl))
- return NULL;
-
- rcu_read_lock();
- fence = dma_fence_get_rcu_safe(&obj->fence_excl);
- rcu_read_unlock();
-
- return fence;
-}
-
-/**
* dma_resv_shared_list - get the reservation object's shared fence list
* @obj: the reservation object
*
@@ -490,5 +464,6 @@ int dma_resv_copy_fences(struct dma_resv *dst, struct dma_resv *src);
long dma_resv_wait_timeout(struct dma_resv *obj, bool wait_all, bool intr,
unsigned long timeout);
bool dma_resv_test_signaled(struct dma_resv *obj, bool test_all);
+void dma_resv_describe(struct dma_resv *obj, struct seq_file *seq);
#endif /* _LINUX_RESERVATION_H */
diff --git a/include/linux/dsa/8021q.h b/include/linux/dsa/8021q.h
index 254b165..939a1be 100644
--- a/include/linux/dsa/8021q.h
+++ b/include/linux/dsa/8021q.h
@@ -7,6 +7,7 @@
#include <linux/refcount.h>
#include <linux/types.h>
+#include <net/dsa.h>
struct dsa_switch;
struct dsa_port;
@@ -37,14 +38,12 @@ struct sk_buff *dsa_8021q_xmit(struct sk_buff *skb, struct net_device *netdev,
void dsa_8021q_rcv(struct sk_buff *skb, int *source_port, int *switch_id);
int dsa_tag_8021q_bridge_tx_fwd_offload(struct dsa_switch *ds, int port,
- struct net_device *br,
- int bridge_num);
+ struct dsa_bridge bridge);
void dsa_tag_8021q_bridge_tx_fwd_unoffload(struct dsa_switch *ds, int port,
- struct net_device *br,
- int bridge_num);
+ struct dsa_bridge bridge);
-u16 dsa_8021q_bridge_tx_fwd_offload_vid(int bridge_num);
+u16 dsa_8021q_bridge_tx_fwd_offload_vid(unsigned int bridge_num);
u16 dsa_tag_8021q_tx_vid(const struct dsa_port *dp);
diff --git a/include/linux/dsa/loop.h b/include/linux/dsa/loop.h
index 5a3470b..b8fef35 100644
--- a/include/linux/dsa/loop.h
+++ b/include/linux/dsa/loop.h
@@ -2,6 +2,7 @@
#ifndef DSA_LOOP_H
#define DSA_LOOP_H
+#include <linux/if_vlan.h>
#include <linux/types.h>
#include <linux/ethtool.h>
#include <net/dsa.h>
diff --git a/include/linux/dsa/ocelot.h b/include/linux/dsa/ocelot.h
index 7ee708a..dca2969 100644
--- a/include/linux/dsa/ocelot.h
+++ b/include/linux/dsa/ocelot.h
@@ -8,6 +8,7 @@
#include <linux/kthread.h>
#include <linux/packing.h>
#include <linux/skbuff.h>
+#include <net/dsa.h>
struct ocelot_skb_cb {
struct sk_buff *clone;
@@ -168,11 +169,18 @@ struct felix_deferred_xmit_work {
struct kthread_work work;
};
-struct felix_port {
+struct ocelot_8021q_tagger_data {
void (*xmit_work_fn)(struct kthread_work *work);
- struct kthread_worker *xmit_worker;
};
+static inline struct ocelot_8021q_tagger_data *
+ocelot_8021q_tagger_data(struct dsa_switch *ds)
+{
+ BUG_ON(ds->dst->tag_ops->proto != DSA_TAG_PROTO_OCELOT_8021Q);
+
+ return ds->tagger_data;
+}
+
static inline void ocelot_xfh_get_rew_val(void *extraction, u64 *rew_val)
{
packing(extraction, rew_val, 116, 85, OCELOT_TAG_LEN, UNPACK, 0);
diff --git a/include/linux/dsa/sja1105.h b/include/linux/dsa/sja1105.h
index e6c78be..159e431 100644
--- a/include/linux/dsa/sja1105.h
+++ b/include/linux/dsa/sja1105.h
@@ -35,23 +35,26 @@
#define SJA1105_META_SMAC 0x222222222222ull
#define SJA1105_META_DMAC 0x0180C200000Eull
-#define SJA1105_HWTS_RX_EN 0
+enum sja1110_meta_tstamp {
+ SJA1110_META_TSTAMP_TX = 0,
+ SJA1110_META_TSTAMP_RX = 1,
+};
-/* Global tagger data: each struct sja1105_port has a reference to
- * the structure defined in struct sja1105_private.
- */
+struct sja1105_deferred_xmit_work {
+ struct dsa_port *dp;
+ struct sk_buff *skb;
+ struct kthread_work work;
+};
+
+/* Global tagger data */
struct sja1105_tagger_data {
- struct sk_buff *stampable_skb;
- /* Protects concurrent access to the meta state machine
- * from taggers running on multiple ports on SMP systems
- */
- spinlock_t meta_lock;
- unsigned long state;
- u8 ts_id;
- /* Used on SJA1110 where meta frames are generated only for
- * 2-step TX timestamps
- */
- struct sk_buff_head skb_txtstamp_queue;
+ /* Tagger to switch */
+ void (*xmit_work_fn)(struct kthread_work *work);
+ void (*meta_tstamp_handler)(struct dsa_switch *ds, int port, u8 ts_id,
+ enum sja1110_meta_tstamp dir, u64 tstamp);
+ /* Switch to tagger */
+ bool (*rxtstamp_get_state)(struct dsa_switch *ds);
+ void (*rxtstamp_set_state)(struct dsa_switch *ds, bool on);
};
struct sja1105_skb_cb {
@@ -64,32 +67,13 @@ struct sja1105_skb_cb {
#define SJA1105_SKB_CB(skb) \
((struct sja1105_skb_cb *)((skb)->cb))
-struct sja1105_port {
- struct kthread_worker *xmit_worker;
- struct kthread_work xmit_work;
- struct sk_buff_head xmit_queue;
- struct sja1105_tagger_data *data;
- bool hwts_tx_en;
-};
-
-/* Timestamps are in units of 8 ns clock ticks (equivalent to
- * a fixed 125 MHz clock).
- */
-#define SJA1105_TICK_NS 8
-
-static inline s64 ns_to_sja1105_ticks(s64 ns)
+static inline struct sja1105_tagger_data *
+sja1105_tagger_data(struct dsa_switch *ds)
{
- return ns / SJA1105_TICK_NS;
-}
+ BUG_ON(ds->dst->tag_ops->proto != DSA_TAG_PROTO_SJA1105 &&
+ ds->dst->tag_ops->proto != DSA_TAG_PROTO_SJA1110);
-static inline s64 sja1105_ticks_to_ns(s64 ticks)
-{
- return ticks * SJA1105_TICK_NS;
-}
-
-static inline bool dsa_port_is_sja1105(struct dsa_port *dp)
-{
- return true;
+ return ds->tagger_data;
}
#endif /* _NET_DSA_SJA1105_H */
diff --git a/include/linux/edac.h b/include/linux/edac.h
index 4207d06..e730b34 100644
--- a/include/linux/edac.h
+++ b/include/linux/edac.h
@@ -182,6 +182,8 @@ static inline char *mc_event_error_type(const unsigned int err_type)
* @MEM_LRDDR4: Load-Reduced DDR4 memory.
* @MEM_LPDDR4: Low-Power DDR4 memory.
* @MEM_DDR5: Unbuffered DDR5 RAM
+ * @MEM_RDDR5: Registered DDR5 RAM
+ * @MEM_LRDDR5: Load-Reduced DDR5 memory.
* @MEM_NVDIMM: Non-volatile RAM
* @MEM_WIO2: Wide I/O 2.
* @MEM_HBM2: High bandwidth Memory Gen 2.
@@ -211,6 +213,8 @@ enum mem_type {
MEM_LRDDR4,
MEM_LPDDR4,
MEM_DDR5,
+ MEM_RDDR5,
+ MEM_LRDDR5,
MEM_NVDIMM,
MEM_WIO2,
MEM_HBM2,
@@ -239,6 +243,8 @@ enum mem_type {
#define MEM_FLAG_LRDDR4 BIT(MEM_LRDDR4)
#define MEM_FLAG_LPDDR4 BIT(MEM_LPDDR4)
#define MEM_FLAG_DDR5 BIT(MEM_DDR5)
+#define MEM_FLAG_RDDR5 BIT(MEM_RDDR5)
+#define MEM_FLAG_LRDDR5 BIT(MEM_LRDDR5)
#define MEM_FLAG_NVDIMM BIT(MEM_NVDIMM)
#define MEM_FLAG_WIO2 BIT(MEM_WIO2)
#define MEM_FLAG_HBM2 BIT(MEM_HBM2)
diff --git a/include/linux/efi.h b/include/linux/efi.h
index dbd39b2..ef8dbc0 100644
--- a/include/linux/efi.h
+++ b/include/linux/efi.h
@@ -1283,4 +1283,10 @@ static inline struct efi_mokvar_table_entry *efi_mokvar_entry_find(
}
#endif
+#ifdef CONFIG_SYSFB
+extern void efifb_setup_from_dmi(struct screen_info *si, const char *opt);
+#else
+static inline void efifb_setup_from_dmi(struct screen_info *si, const char *opt) { }
+#endif
+
#endif /* _LINUX_EFI_H */
diff --git a/include/linux/entry-kvm.h b/include/linux/entry-kvm.h
index 0d7865a..07c878d 100644
--- a/include/linux/entry-kvm.h
+++ b/include/linux/entry-kvm.h
@@ -75,7 +75,7 @@ static inline void xfer_to_guest_mode_prepare(void)
*/
static inline bool __xfer_to_guest_mode_work_pending(void)
{
- unsigned long ti_work = READ_ONCE(current_thread_info()->flags);
+ unsigned long ti_work = read_thread_flags();
return !!(ti_work & XFER_TO_GUEST_MODE_WORK);
}
diff --git a/include/linux/ethtool.h b/include/linux/ethtool.h
index 845a0ff..a26f37a 100644
--- a/include/linux/ethtool.h
+++ b/include/linux/ethtool.h
@@ -67,6 +67,22 @@ enum {
ETH_RSS_HASH_FUNCS_COUNT
};
+/**
+ * struct kernel_ethtool_ringparam - RX/TX ring configuration
+ * @rx_buf_len: Current length of buffers on the rx ring.
+ */
+struct kernel_ethtool_ringparam {
+ u32 rx_buf_len;
+};
+
+/**
+ * enum ethtool_supported_ring_param - indicator caps for setting ring params
+ * @ETHTOOL_RING_USE_RX_BUF_LEN: capture for setting rx_buf_len
+ */
+enum ethtool_supported_ring_param {
+ ETHTOOL_RING_USE_RX_BUF_LEN = BIT(0),
+};
+
#define __ETH_RSS_HASH_BIT(bit) ((u32)1 << (bit))
#define __ETH_RSS_HASH(name) __ETH_RSS_HASH_BIT(ETH_RSS_HASH_##name##_BIT)
@@ -432,6 +448,7 @@ struct ethtool_module_power_mode_params {
* @cap_link_lanes_supported: indicates if the driver supports lanes
* parameter.
* @supported_coalesce_params: supported types of interrupt coalescing.
+ * @supported_ring_params: supported ring params.
* @get_drvinfo: Report driver/device information. Should only set the
* @driver, @version, @fw_version and @bus_info fields. If not
* implemented, the @driver and @bus_info fields will be filled in
@@ -613,6 +630,7 @@ struct ethtool_module_power_mode_params {
struct ethtool_ops {
u32 cap_link_lanes_supported:1;
u32 supported_coalesce_params;
+ u32 supported_ring_params;
void (*get_drvinfo)(struct net_device *, struct ethtool_drvinfo *);
int (*get_regs_len)(struct net_device *);
void (*get_regs)(struct net_device *, struct ethtool_regs *, void *);
@@ -638,9 +656,13 @@ struct ethtool_ops {
struct kernel_ethtool_coalesce *,
struct netlink_ext_ack *);
void (*get_ringparam)(struct net_device *,
- struct ethtool_ringparam *);
+ struct ethtool_ringparam *,
+ struct kernel_ethtool_ringparam *,
+ struct netlink_ext_ack *);
int (*set_ringparam)(struct net_device *,
- struct ethtool_ringparam *);
+ struct ethtool_ringparam *,
+ struct kernel_ethtool_ringparam *,
+ struct netlink_ext_ack *);
void (*get_pause_stats)(struct net_device *dev,
struct ethtool_pause_stats *pause_stats);
void (*get_pauseparam)(struct net_device *,
diff --git a/include/linux/fb.h b/include/linux/fb.h
index 6f3db99..3da9584 100644
--- a/include/linux/fb.h
+++ b/include/linux/fb.h
@@ -610,6 +610,7 @@ extern int remove_conflicting_pci_framebuffers(struct pci_dev *pdev,
const char *name);
extern int remove_conflicting_framebuffers(struct apertures_struct *a,
const char *name, bool primary);
+extern bool is_firmware_framebuffer(struct apertures_struct *a);
extern int fb_prepare_logo(struct fb_info *fb_info, int rotate);
extern int fb_show_logo(struct fb_info *fb_info, int rotate);
extern char* fb_get_buffer_offset(struct fb_info *info, struct fb_pixmap *buf, u32 size);
diff --git a/include/linux/filter.h b/include/linux/filter.h
index 7f1e88e..71fa57b 100644
--- a/include/linux/filter.h
+++ b/include/linux/filter.h
@@ -1019,6 +1019,10 @@ int xdp_do_generic_redirect(struct net_device *dev, struct sk_buff *skb,
int xdp_do_redirect(struct net_device *dev,
struct xdp_buff *xdp,
struct bpf_prog *prog);
+int xdp_do_redirect_frame(struct net_device *dev,
+ struct xdp_buff *xdp,
+ struct xdp_frame *xdpf,
+ struct bpf_prog *prog);
void xdp_do_flush(void);
/* The xdp_do_flush_map() helper has been renamed to drop the _map suffix, as
@@ -1027,7 +1031,7 @@ void xdp_do_flush(void);
*/
#define xdp_do_flush_map xdp_do_flush
-void bpf_warn_invalid_xdp_action(u32 act);
+void bpf_warn_invalid_xdp_action(struct net_device *dev, struct bpf_prog *prog, u32 act);
#ifdef CONFIG_INET
struct sock *bpf_run_sk_reuseport(struct sock_reuseport *reuse, struct sock *sk,
@@ -1371,6 +1375,7 @@ struct bpf_sk_lookup_kern {
const struct in6_addr *daddr;
} v6;
struct sock *selected_sk;
+ u32 ingress_ifindex;
bool no_reuseport;
};
@@ -1433,7 +1438,7 @@ extern struct static_key_false bpf_sk_lookup_enabled;
static inline bool bpf_sk_lookup_run_v4(struct net *net, int protocol,
const __be32 saddr, const __be16 sport,
const __be32 daddr, const u16 dport,
- struct sock **psk)
+ const int ifindex, struct sock **psk)
{
struct bpf_prog_array *run_array;
struct sock *selected_sk = NULL;
@@ -1449,6 +1454,7 @@ static inline bool bpf_sk_lookup_run_v4(struct net *net, int protocol,
.v4.daddr = daddr,
.sport = sport,
.dport = dport,
+ .ingress_ifindex = ifindex,
};
u32 act;
@@ -1471,7 +1477,7 @@ static inline bool bpf_sk_lookup_run_v6(struct net *net, int protocol,
const __be16 sport,
const struct in6_addr *daddr,
const u16 dport,
- struct sock **psk)
+ const int ifindex, struct sock **psk)
{
struct bpf_prog_array *run_array;
struct sock *selected_sk = NULL;
@@ -1487,6 +1493,7 @@ static inline bool bpf_sk_lookup_run_v6(struct net *net, int protocol,
.v6.daddr = daddr,
.sport = sport,
.dport = dport,
+ .ingress_ifindex = ifindex,
};
u32 act;
diff --git a/include/linux/host1x.h b/include/linux/host1x.h
index 7bccf58..e8dc5bc 100644
--- a/include/linux/host1x.h
+++ b/include/linux/host1x.h
@@ -7,6 +7,8 @@
#define __LINUX_HOST1X_H
#include <linux/device.h>
+#include <linux/dma-direction.h>
+#include <linux/spinlock.h>
#include <linux/types.h>
enum host1x_class {
@@ -15,6 +17,8 @@ enum host1x_class {
HOST1X_CLASS_GR2D_SB = 0x52,
HOST1X_CLASS_VIC = 0x5D,
HOST1X_CLASS_GR3D = 0x60,
+ HOST1X_CLASS_NVDEC = 0xF0,
+ HOST1X_CLASS_NVDEC1 = 0xF5,
};
struct host1x;
@@ -24,6 +28,28 @@ struct iommu_group;
u64 host1x_get_dma_mask(struct host1x *host1x);
/**
+ * struct host1x_bo_cache - host1x buffer object cache
+ * @mappings: list of mappings
+ * @lock: synchronizes accesses to the list of mappings
+ */
+struct host1x_bo_cache {
+ struct list_head mappings;
+ struct mutex lock;
+};
+
+static inline void host1x_bo_cache_init(struct host1x_bo_cache *cache)
+{
+ INIT_LIST_HEAD(&cache->mappings);
+ mutex_init(&cache->lock);
+}
+
+static inline void host1x_bo_cache_destroy(struct host1x_bo_cache *cache)
+{
+ /* XXX warn if not empty? */
+ mutex_destroy(&cache->lock);
+}
+
+/**
* struct host1x_client_ops - host1x client operations
* @early_init: host1x client early initialization code
* @init: host1x client initialization code
@@ -73,6 +99,8 @@ struct host1x_client {
struct host1x_client *parent;
unsigned int usecount;
struct mutex lock;
+
+ struct host1x_bo_cache cache;
};
/*
@@ -82,23 +110,48 @@ struct host1x_client {
struct host1x_bo;
struct sg_table;
+struct host1x_bo_mapping {
+ struct kref ref;
+ struct dma_buf_attachment *attach;
+ enum dma_data_direction direction;
+ struct list_head list;
+ struct host1x_bo *bo;
+ struct sg_table *sgt;
+ unsigned int chunks;
+ struct device *dev;
+ dma_addr_t phys;
+ size_t size;
+
+ struct host1x_bo_cache *cache;
+ struct list_head entry;
+};
+
+static inline struct host1x_bo_mapping *to_host1x_bo_mapping(struct kref *ref)
+{
+ return container_of(ref, struct host1x_bo_mapping, ref);
+}
+
struct host1x_bo_ops {
struct host1x_bo *(*get)(struct host1x_bo *bo);
void (*put)(struct host1x_bo *bo);
- struct sg_table *(*pin)(struct device *dev, struct host1x_bo *bo,
- dma_addr_t *phys);
- void (*unpin)(struct device *dev, struct sg_table *sgt);
+ struct host1x_bo_mapping *(*pin)(struct device *dev, struct host1x_bo *bo,
+ enum dma_data_direction dir);
+ void (*unpin)(struct host1x_bo_mapping *map);
void *(*mmap)(struct host1x_bo *bo);
void (*munmap)(struct host1x_bo *bo, void *addr);
};
struct host1x_bo {
const struct host1x_bo_ops *ops;
+ struct list_head mappings;
+ spinlock_t lock;
};
static inline void host1x_bo_init(struct host1x_bo *bo,
const struct host1x_bo_ops *ops)
{
+ INIT_LIST_HEAD(&bo->mappings);
+ spin_lock_init(&bo->lock);
bo->ops = ops;
}
@@ -112,18 +165,10 @@ static inline void host1x_bo_put(struct host1x_bo *bo)
bo->ops->put(bo);
}
-static inline struct sg_table *host1x_bo_pin(struct device *dev,
- struct host1x_bo *bo,
- dma_addr_t *phys)
-{
- return bo->ops->pin(dev, bo, phys);
-}
-
-static inline void host1x_bo_unpin(struct device *dev, struct host1x_bo *bo,
- struct sg_table *sgt)
-{
- bo->ops->unpin(dev, sgt);
-}
+struct host1x_bo_mapping *host1x_bo_pin(struct device *dev, struct host1x_bo *bo,
+ enum dma_data_direction dir,
+ struct host1x_bo_cache *cache);
+void host1x_bo_unpin(struct host1x_bo_mapping *map);
static inline void *host1x_bo_mmap(struct host1x_bo *bo)
{
@@ -181,6 +226,7 @@ struct host1x_job;
struct host1x_channel *host1x_channel_request(struct host1x_client *client);
struct host1x_channel *host1x_channel_get(struct host1x_channel *channel);
+void host1x_channel_stop(struct host1x_channel *channel);
void host1x_channel_put(struct host1x_channel *channel);
int host1x_job_submit(struct host1x_job *job);
diff --git a/include/linux/i2c.h b/include/linux/i2c.h
index 16119ac..7d4f52c 100644
--- a/include/linux/i2c.h
+++ b/include/linux/i2c.h
@@ -1025,8 +1025,9 @@ bool i2c_acpi_get_i2c_resource(struct acpi_resource *ares,
struct acpi_resource_i2c_serialbus **i2c);
int i2c_acpi_client_count(struct acpi_device *adev);
u32 i2c_acpi_find_bus_speed(struct device *dev);
-struct i2c_client *i2c_acpi_new_device(struct device *dev, int index,
- struct i2c_board_info *info);
+struct i2c_client *i2c_acpi_new_device_by_fwnode(struct fwnode_handle *fwnode,
+ int index,
+ struct i2c_board_info *info);
struct i2c_adapter *i2c_acpi_find_adapter_by_handle(acpi_handle handle);
bool i2c_acpi_waive_d0_probe(struct device *dev);
#else
@@ -1043,8 +1044,9 @@ static inline u32 i2c_acpi_find_bus_speed(struct device *dev)
{
return 0;
}
-static inline struct i2c_client *i2c_acpi_new_device(struct device *dev,
- int index, struct i2c_board_info *info)
+static inline struct i2c_client *i2c_acpi_new_device_by_fwnode(
+ struct fwnode_handle *fwnode, int index,
+ struct i2c_board_info *info)
{
return ERR_PTR(-ENODEV);
}
@@ -1058,4 +1060,11 @@ static inline bool i2c_acpi_waive_d0_probe(struct device *dev)
}
#endif /* CONFIG_ACPI */
+static inline struct i2c_client *i2c_acpi_new_device(struct device *dev,
+ int index,
+ struct i2c_board_info *info)
+{
+ return i2c_acpi_new_device_by_fwnode(dev_fwnode(dev), index, info);
+}
+
#endif /* _LINUX_I2C_H */
diff --git a/include/linux/ieee80211.h b/include/linux/ieee80211.h
index 11d7af2..559b6c6 100644
--- a/include/linux/ieee80211.h
+++ b/include/linux/ieee80211.h
@@ -2258,11 +2258,12 @@ enum ieee80211_client_reg_power {
#define IEEE80211_HE_PHY_CAP9_RX_1024_QAM_LESS_THAN_242_TONE_RU 0x08
#define IEEE80211_HE_PHY_CAP9_RX_FULL_BW_SU_USING_MU_WITH_COMP_SIGB 0x10
#define IEEE80211_HE_PHY_CAP9_RX_FULL_BW_SU_USING_MU_WITH_NON_COMP_SIGB 0x20
-#define IEEE80211_HE_PHY_CAP9_NOMIMAL_PKT_PADDING_0US 0x00
-#define IEEE80211_HE_PHY_CAP9_NOMIMAL_PKT_PADDING_8US 0x40
-#define IEEE80211_HE_PHY_CAP9_NOMIMAL_PKT_PADDING_16US 0x80
-#define IEEE80211_HE_PHY_CAP9_NOMIMAL_PKT_PADDING_RESERVED 0xc0
-#define IEEE80211_HE_PHY_CAP9_NOMIMAL_PKT_PADDING_MASK 0xc0
+#define IEEE80211_HE_PHY_CAP9_NOMINAL_PKT_PADDING_0US 0x0
+#define IEEE80211_HE_PHY_CAP9_NOMINAL_PKT_PADDING_8US 0x1
+#define IEEE80211_HE_PHY_CAP9_NOMINAL_PKT_PADDING_16US 0x2
+#define IEEE80211_HE_PHY_CAP9_NOMINAL_PKT_PADDING_RESERVED 0x3
+#define IEEE80211_HE_PHY_CAP9_NOMINAL_PKT_PADDING_POS 6
+#define IEEE80211_HE_PHY_CAP9_NOMINAL_PKT_PADDING_MASK 0xc0
#define IEEE80211_HE_PHY_CAP10_HE_MU_M1RU_MAX_LTF 0x01
diff --git a/include/linux/if_eql.h b/include/linux/if_eql.h
index d984694..d75601d 100644
--- a/include/linux/if_eql.h
+++ b/include/linux/if_eql.h
@@ -26,6 +26,7 @@
typedef struct slave {
struct list_head list;
struct net_device *dev;
+ netdevice_tracker dev_tracker;
long priority;
long priority_bps;
long priority_Bps;
diff --git a/include/linux/if_vlan.h b/include/linux/if_vlan.h
index 41a5183..8420fe5 100644
--- a/include/linux/if_vlan.h
+++ b/include/linux/if_vlan.h
@@ -162,6 +162,7 @@ struct netpoll;
* @vlan_id: VLAN identifier
* @flags: device flags
* @real_dev: underlying netdevice
+ * @dev_tracker: refcount tracker for @real_dev reference
* @real_dev_addr: address of underlying netdevice
* @dent: proc dir entry
* @vlan_pcpu_stats: ptr to percpu rx stats
@@ -177,6 +178,8 @@ struct vlan_dev_priv {
u16 flags;
struct net_device *real_dev;
+ netdevice_tracker dev_tracker;
+
unsigned char real_dev_addr[ETH_ALEN];
struct proc_dir_entry *dent;
diff --git a/include/linux/inetdevice.h b/include/linux/inetdevice.h
index 518b484..674aeea 100644
--- a/include/linux/inetdevice.h
+++ b/include/linux/inetdevice.h
@@ -24,6 +24,8 @@ struct ipv4_devconf {
struct in_device {
struct net_device *dev;
+ netdevice_tracker dev_tracker;
+
refcount_t refcnt;
int dead;
struct in_ifaddr __rcu *ifa_list;/* IP ifaddr chain */
diff --git a/include/linux/ipv6.h b/include/linux/ipv6.h
index 20c1f96..a59d25f 100644
--- a/include/linux/ipv6.h
+++ b/include/linux/ipv6.h
@@ -133,6 +133,7 @@ struct inet6_skb_parm {
__u16 dsthao;
#endif
__u16 frag_max_size;
+ __u16 srhoff;
#define IP6SKB_XFRM_TRANSFORMED 1
#define IP6SKB_FORWARDED 2
@@ -142,6 +143,7 @@ struct inet6_skb_parm {
#define IP6SKB_HOPBYHOP 32
#define IP6SKB_L3SLAVE 64
#define IP6SKB_JUMBOGRAM 128
+#define IP6SKB_SEG6 256
};
#if defined(CONFIG_NET_L3_MASTER_DEV)
diff --git a/include/linux/kasan.h b/include/linux/kasan.h
index d8783b6..fb78108 100644
--- a/include/linux/kasan.h
+++ b/include/linux/kasan.h
@@ -9,6 +9,7 @@
struct kmem_cache;
struct page;
+struct slab;
struct vm_struct;
struct task_struct;
@@ -193,11 +194,11 @@ static __always_inline size_t kasan_metadata_size(struct kmem_cache *cache)
return 0;
}
-void __kasan_poison_slab(struct page *page);
-static __always_inline void kasan_poison_slab(struct page *page)
+void __kasan_poison_slab(struct slab *slab);
+static __always_inline void kasan_poison_slab(struct slab *slab)
{
if (kasan_enabled())
- __kasan_poison_slab(page);
+ __kasan_poison_slab(slab);
}
void __kasan_unpoison_object_data(struct kmem_cache *cache, void *object);
@@ -322,7 +323,7 @@ static inline void kasan_cache_create(struct kmem_cache *cache,
slab_flags_t *flags) {}
static inline void kasan_cache_create_kmalloc(struct kmem_cache *cache) {}
static inline size_t kasan_metadata_size(struct kmem_cache *cache) { return 0; }
-static inline void kasan_poison_slab(struct page *page) {}
+static inline void kasan_poison_slab(struct slab *slab) {}
static inline void kasan_unpoison_object_data(struct kmem_cache *cache,
void *object) {}
static inline void kasan_poison_object_data(struct kmem_cache *cache,
diff --git a/include/linux/mdio.h b/include/linux/mdio.h
index 9f3587a..ecac96d 100644
--- a/include/linux/mdio.h
+++ b/include/linux/mdio.h
@@ -7,6 +7,7 @@
#define __LINUX_MDIO_H__
#include <uapi/linux/mdio.h>
+#include <linux/bitfield.h>
#include <linux/mod_devicetable.h>
/* Or MII_ADDR_C45 into regnum for read/write on mii_bus to enable the 21 bit
@@ -14,6 +15,7 @@
*/
#define MII_ADDR_C45 (1<<30)
#define MII_DEVADDR_C45_SHIFT 16
+#define MII_DEVADDR_C45_MASK GENMASK(20, 16)
#define MII_REGADDR_C45_MASK GENMASK(15, 0)
struct gpio_desc;
@@ -381,6 +383,16 @@ static inline u32 mdiobus_c45_addr(int devad, u16 regnum)
return MII_ADDR_C45 | devad << MII_DEVADDR_C45_SHIFT | regnum;
}
+static inline u16 mdiobus_c45_regad(u32 regnum)
+{
+ return FIELD_GET(MII_REGADDR_C45_MASK, regnum);
+}
+
+static inline u16 mdiobus_c45_devad(u32 regnum)
+{
+ return FIELD_GET(MII_DEVADDR_C45_MASK, regnum);
+}
+
static inline int __mdiobus_c45_read(struct mii_bus *bus, int prtad, int devad,
u16 regnum)
{
diff --git a/include/linux/mdio/mdio-mscc-miim.h b/include/linux/mdio/mdio-mscc-miim.h
new file mode 100644
index 0000000..5b4ed2c3
--- /dev/null
+++ b/include/linux/mdio/mdio-mscc-miim.h
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: (GPL-2.0 OR MIT) */
+/*
+ * Driver for the MDIO interface of Microsemi network switches.
+ *
+ * Author: Colin Foster <colin.foster@in-advantage.com>
+ * Copyright (C) 2021 Innovative Advantage
+ */
+#ifndef MDIO_MSCC_MIIM_H
+#define MDIO_MSCC_MIIM_H
+
+#include <linux/device.h>
+#include <linux/phy.h>
+#include <linux/regmap.h>
+
+int mscc_miim_setup(struct device *device, struct mii_bus **bus,
+ const char *name, struct regmap *mii_regmap,
+ int status_offset);
+
+#endif
diff --git a/include/linux/mei_cl_bus.h b/include/linux/mei_cl_bus.h
index c6786c1..df1fab4 100644
--- a/include/linux/mei_cl_bus.h
+++ b/include/linux/mei_cl_bus.h
@@ -117,4 +117,7 @@ int mei_cldev_enable(struct mei_cl_device *cldev);
int mei_cldev_disable(struct mei_cl_device *cldev);
bool mei_cldev_enabled(const struct mei_cl_device *cldev);
+void *mei_cldev_dma_map(struct mei_cl_device *cldev, u8 buffer_id, size_t size);
+int mei_cldev_dma_unmap(struct mei_cl_device *cldev);
+
#endif /* _LINUX_MEI_CL_BUS_H */
diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index 0c5c403..e34112f 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -536,45 +536,6 @@ static inline bool folio_memcg_kmem(struct folio *folio)
return folio->memcg_data & MEMCG_DATA_KMEM;
}
-/*
- * page_objcgs - get the object cgroups vector associated with a page
- * @page: a pointer to the page struct
- *
- * Returns a pointer to the object cgroups vector associated with the page,
- * or NULL. This function assumes that the page is known to have an
- * associated object cgroups vector. It's not safe to call this function
- * against pages, which might have an associated memory cgroup: e.g.
- * kernel stack pages.
- */
-static inline struct obj_cgroup **page_objcgs(struct page *page)
-{
- unsigned long memcg_data = READ_ONCE(page->memcg_data);
-
- VM_BUG_ON_PAGE(memcg_data && !(memcg_data & MEMCG_DATA_OBJCGS), page);
- VM_BUG_ON_PAGE(memcg_data & MEMCG_DATA_KMEM, page);
-
- return (struct obj_cgroup **)(memcg_data & ~MEMCG_DATA_FLAGS_MASK);
-}
-
-/*
- * page_objcgs_check - get the object cgroups vector associated with a page
- * @page: a pointer to the page struct
- *
- * Returns a pointer to the object cgroups vector associated with the page,
- * or NULL. This function is safe to use if the page can be directly associated
- * with a memory cgroup.
- */
-static inline struct obj_cgroup **page_objcgs_check(struct page *page)
-{
- unsigned long memcg_data = READ_ONCE(page->memcg_data);
-
- if (!memcg_data || !(memcg_data & MEMCG_DATA_OBJCGS))
- return NULL;
-
- VM_BUG_ON_PAGE(memcg_data & MEMCG_DATA_KMEM, page);
-
- return (struct obj_cgroup **)(memcg_data & ~MEMCG_DATA_FLAGS_MASK);
-}
#else
static inline bool folio_memcg_kmem(struct folio *folio)
@@ -582,15 +543,6 @@ static inline bool folio_memcg_kmem(struct folio *folio)
return false;
}
-static inline struct obj_cgroup **page_objcgs(struct page *page)
-{
- return NULL;
-}
-
-static inline struct obj_cgroup **page_objcgs_check(struct page *page)
-{
- return NULL;
-}
#endif
static inline bool PageMemcgKmem(struct page *page)
diff --git a/include/linux/mlx5/device.h b/include/linux/mlx5/device.h
index 9c25edf..604b85d 100644
--- a/include/linux/mlx5/device.h
+++ b/include/linux/mlx5/device.h
@@ -1117,6 +1117,7 @@ enum {
MLX5_MATCH_MISC_PARAMETERS_2 = 1 << 3,
MLX5_MATCH_MISC_PARAMETERS_3 = 1 << 4,
MLX5_MATCH_MISC_PARAMETERS_4 = 1 << 5,
+ MLX5_MATCH_MISC_PARAMETERS_5 = 1 << 6,
};
enum {
diff --git a/include/linux/mlx5/driver.h b/include/linux/mlx5/driver.h
index a623ec6..78655d8 100644
--- a/include/linux/mlx5/driver.h
+++ b/include/linux/mlx5/driver.h
@@ -478,6 +478,10 @@ struct mlx5_fc_stats {
unsigned long next_query;
unsigned long sampling_interval; /* jiffies */
u32 *bulk_query_out;
+ int bulk_query_len;
+ size_t num_counters;
+ bool bulk_query_alloc_failed;
+ unsigned long next_bulk_query_alloc;
struct mlx5_fc_pool fc_pool;
};
diff --git a/include/linux/mlx5/eq.h b/include/linux/mlx5/eq.h
index ea3ff5a..3705a38 100644
--- a/include/linux/mlx5/eq.h
+++ b/include/linux/mlx5/eq.h
@@ -9,13 +9,13 @@
#define MLX5_NUM_SPARE_EQE (0x80)
struct mlx5_eq;
+struct mlx5_irq;
struct mlx5_core_dev;
struct mlx5_eq_param {
- u8 irq_index;
int nent;
u64 mask[4];
- cpumask_var_t affinity;
+ struct mlx5_irq *irq;
};
struct mlx5_eq *
diff --git a/include/linux/mlx5/fs.h b/include/linux/mlx5/fs.h
index cd2d4c5..b1aad14 100644
--- a/include/linux/mlx5/fs.h
+++ b/include/linux/mlx5/fs.h
@@ -73,6 +73,7 @@ enum mlx5_flow_namespace_type {
MLX5_FLOW_NAMESPACE_KERNEL,
MLX5_FLOW_NAMESPACE_LEFTOVERS,
MLX5_FLOW_NAMESPACE_ANCHOR,
+ MLX5_FLOW_NAMESPACE_FDB_BYPASS,
MLX5_FLOW_NAMESPACE_FDB,
MLX5_FLOW_NAMESPACE_ESW_EGRESS,
MLX5_FLOW_NAMESPACE_ESW_INGRESS,
diff --git a/include/linux/mlx5/mlx5_ifc.h b/include/linux/mlx5/mlx5_ifc.h
index fbaab44..598ac3b 100644
--- a/include/linux/mlx5/mlx5_ifc.h
+++ b/include/linux/mlx5/mlx5_ifc.h
@@ -372,7 +372,8 @@ struct mlx5_ifc_flow_table_fields_supported_bits {
u8 reserved_at_37[0x9];
u8 geneve_tlv_option_0_data[0x1];
- u8 reserved_at_41[0x4];
+ u8 geneve_tlv_option_0_exist[0x1];
+ u8 reserved_at_42[0x3];
u8 outer_first_mpls_over_udp[0x4];
u8 outer_first_mpls_over_gre[0x4];
u8 inner_first_mpls[0x4];
@@ -551,7 +552,8 @@ struct mlx5_ifc_fte_match_set_misc_bits {
u8 bth_opcode[0x8];
u8 geneve_vni[0x18];
- u8 reserved_at_d8[0x7];
+ u8 reserved_at_d8[0x6];
+ u8 geneve_tlv_option_0_exist[0x1];
u8 geneve_oam[0x1];
u8 reserved_at_e0[0xc];
@@ -670,6 +672,26 @@ struct mlx5_ifc_fte_match_set_misc4_bits {
u8 reserved_at_100[0x100];
};
+struct mlx5_ifc_fte_match_set_misc5_bits {
+ u8 macsec_tag_0[0x20];
+
+ u8 macsec_tag_1[0x20];
+
+ u8 macsec_tag_2[0x20];
+
+ u8 macsec_tag_3[0x20];
+
+ u8 tunnel_header_0[0x20];
+
+ u8 tunnel_header_1[0x20];
+
+ u8 tunnel_header_2[0x20];
+
+ u8 tunnel_header_3[0x20];
+
+ u8 reserved_at_100[0x100];
+};
+
struct mlx5_ifc_cmd_pas_bits {
u8 pa_h[0x20];
@@ -811,7 +833,7 @@ struct mlx5_ifc_flow_table_eswitch_cap_bits {
u8 fdb_to_vport_reg_c_id[0x8];
u8 reserved_at_8[0xd];
u8 fdb_modify_header_fwd_to_table[0x1];
- u8 reserved_at_16[0x1];
+ u8 fdb_ipv4_ttl_modify[0x1];
u8 flow_source[0x1];
u8 reserved_at_18[0x2];
u8 multi_fdb_encap[0x1];
@@ -1291,7 +1313,7 @@ enum {
enum {
MLX5_FLEX_PARSER_GENEVE_ENABLED = 1 << 3,
MLX5_FLEX_PARSER_MPLS_OVER_GRE_ENABLED = 1 << 4,
- mlx5_FLEX_PARSER_MPLS_OVER_UDP_ENABLED = 1 << 5,
+ MLX5_FLEX_PARSER_MPLS_OVER_UDP_ENABLED = 1 << 5,
MLX5_FLEX_PARSER_VXLAN_GPE_ENABLED = 1 << 7,
MLX5_FLEX_PARSER_ICMP_V4_ENABLED = 1 << 8,
MLX5_FLEX_PARSER_ICMP_V6_ENABLED = 1 << 9,
@@ -1621,7 +1643,7 @@ struct mlx5_ifc_cmd_hca_cap_bits {
u8 ext_stride_num_range[0x1];
u8 roce_rw_supported[0x1];
- u8 reserved_at_3a2[0x1];
+ u8 log_max_current_uc_list_wr_supported[0x1];
u8 log_max_stride_sz_rq[0x5];
u8 reserved_at_3a8[0x3];
u8 log_min_stride_sz_rq[0x5];
@@ -1839,7 +1861,9 @@ struct mlx5_ifc_fte_match_param_bits {
struct mlx5_ifc_fte_match_set_misc4_bits misc_parameters_4;
- u8 reserved_at_c00[0x400];
+ struct mlx5_ifc_fte_match_set_misc5_bits misc_parameters_5;
+
+ u8 reserved_at_e00[0x200];
};
enum {
@@ -5977,6 +6001,7 @@ enum {
MLX5_QUERY_FLOW_GROUP_IN_MATCH_CRITERIA_ENABLE_MISC_PARAMETERS_2 = 0x3,
MLX5_QUERY_FLOW_GROUP_IN_MATCH_CRITERIA_ENABLE_MISC_PARAMETERS_3 = 0x4,
MLX5_QUERY_FLOW_GROUP_IN_MATCH_CRITERIA_ENABLE_MISC_PARAMETERS_4 = 0x5,
+ MLX5_QUERY_FLOW_GROUP_IN_MATCH_CRITERIA_ENABLE_MISC_PARAMETERS_5 = 0x6,
};
struct mlx5_ifc_query_flow_group_out_bits {
diff --git a/include/linux/mm.h b/include/linux/mm.h
index a7e4a9e..de10394 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -863,6 +863,13 @@ static inline struct page *virt_to_head_page(const void *x)
return compound_head(page);
}
+static inline struct folio *virt_to_folio(const void *x)
+{
+ struct page *page = virt_to_page(x);
+
+ return page_folio(page);
+}
+
void __put_page(struct page *page);
void put_pages_list(struct list_head *pages);
@@ -1753,6 +1760,11 @@ void page_address_init(void);
#define page_address_init() do { } while(0)
#endif
+static inline void *folio_address(const struct folio *folio)
+{
+ return page_address(&folio->page);
+}
+
extern void *page_rmapping(struct page *page);
extern struct anon_vma *page_anon_vma(struct page *page);
extern pgoff_t __page_file_index(struct page *page);
@@ -3231,6 +3243,19 @@ extern void shake_page(struct page *p);
extern atomic_long_t num_poisoned_pages __read_mostly;
extern int soft_offline_page(unsigned long pfn, int flags);
+#ifndef arch_memory_failure
+static inline int arch_memory_failure(unsigned long pfn, int flags)
+{
+ return -ENXIO;
+}
+#endif
+
+#ifndef arch_is_platform_page
+static inline bool arch_is_platform_page(u64 paddr)
+{
+ return false;
+}
+#endif
/*
* Error handlers for various types of pages.
diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index c3a6e62..1ae3537 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -56,11 +56,11 @@ struct mem_cgroup;
* in each subpage, but you may need to restore some of their values
* afterwards.
*
- * SLUB uses cmpxchg_double() to atomically update its freelist and
- * counters. That requires that freelist & counters be adjacent and
- * double-word aligned. We align all struct pages to double-word
- * boundaries, and ensure that 'freelist' is aligned within the
- * struct.
+ * SLUB uses cmpxchg_double() to atomically update its freelist and counters.
+ * That requires that freelist & counters in struct slab be adjacent and
+ * double-word aligned. Because struct slab currently just reinterprets the
+ * bits of struct page, we align all struct pages to double-word boundaries,
+ * and ensure that 'freelist' is aligned within struct slab.
*/
#ifdef CONFIG_HAVE_ALIGNED_STRUCT_PAGE
#define _struct_page_alignment __aligned(2 * sizeof(unsigned long))
diff --git a/include/linux/mmc/sdio_ids.h b/include/linux/mmc/sdio_ids.h
index a85c9f0..53f0efa 100644
--- a/include/linux/mmc/sdio_ids.h
+++ b/include/linux/mmc/sdio_ids.h
@@ -105,6 +105,7 @@
#define SDIO_VENDOR_ID_MEDIATEK 0x037a
#define SDIO_DEVICE_ID_MEDIATEK_MT7663 0x7663
#define SDIO_DEVICE_ID_MEDIATEK_MT7668 0x7668
+#define SDIO_DEVICE_ID_MEDIATEK_MT7961 0x7961
#define SDIO_VENDOR_ID_MICROCHIP_WILC 0x0296
#define SDIO_DEVICE_ID_MICROCHIP_WILC1000 0x5347
diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
index 58e744b..936dc0b 100644
--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h
@@ -277,6 +277,7 @@ enum vmscan_throttle_state {
VMSCAN_THROTTLE_WRITEBACK,
VMSCAN_THROTTLE_ISOLATED,
VMSCAN_THROTTLE_NOPROGRESS,
+ VMSCAN_THROTTLE_CONGESTED,
NR_VMSCAN_THROTTLE,
};
diff --git a/include/linux/mroute_base.h b/include/linux/mroute_base.h
index 8071148..e05ee9f 100644
--- a/include/linux/mroute_base.h
+++ b/include/linux/mroute_base.h
@@ -12,6 +12,7 @@
/**
* struct vif_device - interface representor for multicast routing
* @dev: network device being used
+ * @dev_tracker: refcount tracker for @dev reference
* @bytes_in: statistic; bytes ingressing
* @bytes_out: statistic; bytes egresing
* @pkt_in: statistic; packets ingressing
@@ -26,6 +27,7 @@
*/
struct vif_device {
struct net_device *dev;
+ netdevice_tracker dev_tracker;
unsigned long bytes_in, bytes_out;
unsigned long pkt_in, pkt_out;
unsigned long rate_limit;
diff --git a/include/linux/net/intel/iidc.h b/include/linux/net/intel/iidc.h
index e32f671..1289593 100644
--- a/include/linux/net/intel/iidc.h
+++ b/include/linux/net/intel/iidc.h
@@ -26,6 +26,11 @@ enum iidc_reset_type {
IIDC_GLOBR,
};
+enum iidc_rdma_protocol {
+ IIDC_RDMA_PROTOCOL_IWARP = BIT(0),
+ IIDC_RDMA_PROTOCOL_ROCEV2 = BIT(1),
+};
+
#define IIDC_MAX_USER_PRIORITY 8
/* Struct to hold per RDMA Qset info */
@@ -70,8 +75,6 @@ int ice_rdma_request_reset(struct ice_pf *pf, enum iidc_reset_type reset_type);
int ice_rdma_update_vsi_filter(struct ice_pf *pf, u16 vsi_id, bool enable);
void ice_get_qos_params(struct ice_pf *pf, struct iidc_qos_params *qos);
-#define IIDC_RDMA_ROCE_NAME "roce"
-
/* Structure representing auxiliary driver tailored information about the core
* PCI dev, each auxiliary driver using the IIDC interface will have an
* instance of this struct dedicated to it.
diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index 6aadcc0..3213c72 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -48,6 +48,7 @@
#include <uapi/linux/pkt_cls.h>
#include <linux/hashtable.h>
#include <linux/rbtree.h>
+#include <net/net_trackers.h>
struct netpoll_info;
struct device;
@@ -299,7 +300,6 @@ enum netdev_state_t {
__LINK_STATE_TESTING,
};
-
struct gro_list {
struct list_head list;
int count;
@@ -579,6 +579,8 @@ struct netdev_queue {
* read-mostly part
*/
struct net_device *dev;
+ netdevice_tracker dev_tracker;
+
struct Qdisc __rcu *qdisc;
struct Qdisc *qdisc_sleeping;
#ifdef CONFIG_SYSFS
@@ -592,7 +594,7 @@ struct netdev_queue {
* Number of TX timeouts for this queue
* (/sys/class/net/DEV/Q/trans_timeout)
*/
- unsigned long trans_timeout;
+ atomic_long_t trans_timeout;
/* Subordinate device that the queue has been assigned to */
struct net_device *sb_dev;
@@ -734,6 +736,8 @@ struct netdev_rx_queue {
#endif
struct kobject kobj;
struct net_device *dev;
+ netdevice_tracker dev_tracker;
+
#ifdef CONFIG_XDP_SOCKETS
struct xsk_buff_pool *pool;
#endif
@@ -916,6 +920,7 @@ enum tc_setup_type {
TC_SETUP_QDISC_TBF,
TC_SETUP_QDISC_FIFO,
TC_SETUP_QDISC_HTB,
+ TC_SETUP_ACT,
};
/* These structures hold the attributes of bpf state that are being passed
@@ -1297,11 +1302,6 @@ struct netdev_net_notifier {
* TX queue.
* int (*ndo_get_iflink)(const struct net_device *dev);
* Called to get the iflink value of this device.
- * void (*ndo_change_proto_down)(struct net_device *dev,
- * bool proto_down);
- * This function is used to pass protocol port error state information
- * to the switch driver. The switch driver can react to the proto_down
- * by doing a phys down on the associated switch port.
* int (*ndo_fill_metadata_dst)(struct net_device *dev, struct sk_buff *skb);
* This function is used to get egress tunnel information for given skb.
* This is useful for retrieving outer tunnel header parameters while
@@ -1542,8 +1542,6 @@ struct net_device_ops {
int queue_index,
u32 maxrate);
int (*ndo_get_iflink)(const struct net_device *dev);
- int (*ndo_change_proto_down)(struct net_device *dev,
- bool proto_down);
int (*ndo_fill_metadata_dst)(struct net_device *dev,
struct sk_buff *skb);
void (*ndo_set_rx_headroom)(struct net_device *dev,
@@ -1612,6 +1610,7 @@ struct net_device_ops {
* @IFF_LIVE_RENAME_OK: rename is allowed while device is up and running
* @IFF_TX_SKB_NO_LINEAR: device/driver is capable of xmitting frames with
* skb_headlen(skb) == 0 (data starts from frag0)
+ * @IFF_CHANGE_PROTO_DOWN: device supports setting carrier via IFLA_PROTO_DOWN
*/
enum netdev_priv_flags {
IFF_802_1Q_VLAN = 1<<0,
@@ -1646,6 +1645,7 @@ enum netdev_priv_flags {
IFF_L3MDEV_RX_HANDLER = 1<<29,
IFF_LIVE_RENAME_OK = 1<<30,
IFF_TX_SKB_NO_LINEAR = 1<<31,
+ IFF_CHANGE_PROTO_DOWN = BIT_ULL(32),
};
#define IFF_802_1Q_VLAN IFF_802_1Q_VLAN
@@ -1870,6 +1870,7 @@ enum netdev_ml_priv_type {
* @proto_down_reason: reason a netdev interface is held down
* @pcpu_refcnt: Number of references to this device
* @dev_refcnt: Number of references to this device
+ * @refcnt_tracker: Tracker directory for tracked references to this device
* @todo_list: Delayed register/unregister
* @link_watch_list: XXX: need comments on this one
*
@@ -1941,6 +1942,12 @@ enum netdev_ml_priv_type {
* dev->addr_list_lock.
* @unlink_list: As netif_addr_lock() can be called recursively,
* keep a list of interfaces to be deleted.
+ * @gro_max_size: Maximum size of aggregated packet in generic
+ * receive offload (GRO)
+ *
+ * @dev_addr_shadow: Copy of @dev_addr to catch direct writes.
+ * @linkwatch_dev_tracker: refcount tracker used by linkwatch.
+ * @watchdog_dev_tracker: refcount tracker used by watchdog.
*
* FIXME: cleanup struct net_device such that network protocol info
* moves out.
@@ -1980,7 +1987,7 @@ struct net_device {
/* Read-mostly cache-line for fast-path access */
unsigned int flags;
- unsigned int priv_flags;
+ unsigned long long priv_flags;
const struct net_device_ops *netdev_ops;
int ifindex;
unsigned short gflags;
@@ -2093,7 +2100,7 @@ struct net_device {
#if IS_ENABLED(CONFIG_TIPC)
struct tipc_bearer __rcu *tipc_ptr;
#endif
-#if IS_ENABLED(CONFIG_IRDA) || IS_ENABLED(CONFIG_ATALK)
+#if IS_ENABLED(CONFIG_ATALK)
void *atalk_ptr;
#endif
struct in_device __rcu *ip_ptr;
@@ -2117,7 +2124,7 @@ struct net_device {
* Cache lines mostly used on receive path (including eth_type_trans())
*/
/* Interface address info used in eth_type_trans() */
- unsigned char *dev_addr;
+ const unsigned char *dev_addr;
struct netdev_rx_queue *_rx;
unsigned int num_rx_queues;
@@ -2126,6 +2133,8 @@ struct net_device {
struct bpf_prog __rcu *xdp_prog;
unsigned long gro_flush_timeout;
int napi_defer_hard_irqs;
+#define GRO_MAX_SIZE 65536
+ unsigned int gro_max_size;
rx_handler_func_t __rcu *rx_handler;
void __rcu *rx_handler_data;
@@ -2181,6 +2190,7 @@ struct net_device {
#else
refcount_t dev_refcnt;
#endif
+ struct ref_tracker_dir refcnt_tracker;
struct list_head link_watch_list;
@@ -2268,6 +2278,10 @@ struct net_device {
/* protected by rtnl_lock */
struct bpf_xdp_entity xdp_state[__MAX_XDP_MODE];
+
+ u8 dev_addr_shadow[MAX_ADDR_LEN];
+ netdevice_tracker linkwatch_dev_tracker;
+ netdevice_tracker watchdog_dev_tracker;
};
#define to_net_dev(d) container_of(d, struct net_device, dev)
@@ -2520,113 +2534,11 @@ static inline void netif_napi_del(struct napi_struct *napi)
synchronize_net();
}
-struct napi_gro_cb {
- /* Virtual address of skb_shinfo(skb)->frags[0].page + offset. */
- void *frag0;
-
- /* Length of frag0. */
- unsigned int frag0_len;
-
- /* This indicates where we are processing relative to skb->data. */
- int data_offset;
-
- /* This is non-zero if the packet cannot be merged with the new skb. */
- u16 flush;
-
- /* Save the IP ID here and check when we get to the transport layer */
- u16 flush_id;
-
- /* Number of segments aggregated. */
- u16 count;
-
- /* Start offset for remote checksum offload */
- u16 gro_remcsum_start;
-
- /* jiffies when first packet was created/queued */
- unsigned long age;
-
- /* Used in ipv6_gro_receive() and foo-over-udp */
- u16 proto;
-
- /* This is non-zero if the packet may be of the same flow. */
- u8 same_flow:1;
-
- /* Used in tunnel GRO receive */
- u8 encap_mark:1;
-
- /* GRO checksum is valid */
- u8 csum_valid:1;
-
- /* Number of checksums via CHECKSUM_UNNECESSARY */
- u8 csum_cnt:3;
-
- /* Free the skb? */
- u8 free:2;
-#define NAPI_GRO_FREE 1
-#define NAPI_GRO_FREE_STOLEN_HEAD 2
-
- /* Used in foo-over-udp, set in udp[46]_gro_receive */
- u8 is_ipv6:1;
-
- /* Used in GRE, set in fou/gue_gro_receive */
- u8 is_fou:1;
-
- /* Used to determine if flush_id can be ignored */
- u8 is_atomic:1;
-
- /* Number of gro_receive callbacks this packet already went through */
- u8 recursion_counter:4;
-
- /* GRO is done by frag_list pointer chaining. */
- u8 is_flist:1;
-
- /* used to support CHECKSUM_COMPLETE for tunneling protocols */
- __wsum csum;
-
- /* used in skb_gro_receive() slow path */
- struct sk_buff *last;
-};
-
-#define NAPI_GRO_CB(skb) ((struct napi_gro_cb *)(skb)->cb)
-
-#define GRO_RECURSION_LIMIT 15
-static inline int gro_recursion_inc_test(struct sk_buff *skb)
-{
- return ++NAPI_GRO_CB(skb)->recursion_counter == GRO_RECURSION_LIMIT;
-}
-
-typedef struct sk_buff *(*gro_receive_t)(struct list_head *, struct sk_buff *);
-static inline struct sk_buff *call_gro_receive(gro_receive_t cb,
- struct list_head *head,
- struct sk_buff *skb)
-{
- if (unlikely(gro_recursion_inc_test(skb))) {
- NAPI_GRO_CB(skb)->flush |= 1;
- return NULL;
- }
-
- return cb(head, skb);
-}
-
-typedef struct sk_buff *(*gro_receive_sk_t)(struct sock *, struct list_head *,
- struct sk_buff *);
-static inline struct sk_buff *call_gro_receive_sk(gro_receive_sk_t cb,
- struct sock *sk,
- struct list_head *head,
- struct sk_buff *skb)
-{
- if (unlikely(gro_recursion_inc_test(skb))) {
- NAPI_GRO_CB(skb)->flush |= 1;
- return NULL;
- }
-
- return cb(sk, head, skb);
-}
-
struct packet_type {
__be16 type; /* This is really htons(ether_type). */
bool ignore_outgoing;
struct net_device *dev; /* NULL is wildcarded here */
+ netdevice_tracker dev_tracker;
int (*func) (struct sk_buff *,
struct net_device *,
struct packet_type *,
@@ -3005,254 +2917,7 @@ struct net_device *dev_get_by_index_rcu(struct net *net, int ifindex);
struct net_device *dev_get_by_napi_id(unsigned int napi_id);
int netdev_get_name(struct net *net, char *name, int ifindex);
int dev_restart(struct net_device *dev);
-int skb_gro_receive(struct sk_buff *p, struct sk_buff *skb);
-int skb_gro_receive_list(struct sk_buff *p, struct sk_buff *skb);
-static inline unsigned int skb_gro_offset(const struct sk_buff *skb)
-{
- return NAPI_GRO_CB(skb)->data_offset;
-}
-
-static inline unsigned int skb_gro_len(const struct sk_buff *skb)
-{
- return skb->len - NAPI_GRO_CB(skb)->data_offset;
-}
-
-static inline void skb_gro_pull(struct sk_buff *skb, unsigned int len)
-{
- NAPI_GRO_CB(skb)->data_offset += len;
-}
-
-static inline void *skb_gro_header_fast(struct sk_buff *skb,
- unsigned int offset)
-{
- return NAPI_GRO_CB(skb)->frag0 + offset;
-}
-
-static inline int skb_gro_header_hard(struct sk_buff *skb, unsigned int hlen)
-{
- return NAPI_GRO_CB(skb)->frag0_len < hlen;
-}
-
-static inline void skb_gro_frag0_invalidate(struct sk_buff *skb)
-{
- NAPI_GRO_CB(skb)->frag0 = NULL;
- NAPI_GRO_CB(skb)->frag0_len = 0;
-}
-
-static inline void *skb_gro_header_slow(struct sk_buff *skb, unsigned int hlen,
- unsigned int offset)
-{
- if (!pskb_may_pull(skb, hlen))
- return NULL;
-
- skb_gro_frag0_invalidate(skb);
- return skb->data + offset;
-}
-
-static inline void *skb_gro_network_header(struct sk_buff *skb)
-{
- return (NAPI_GRO_CB(skb)->frag0 ?: skb->data) +
- skb_network_offset(skb);
-}
-
-static inline void skb_gro_postpull_rcsum(struct sk_buff *skb,
- const void *start, unsigned int len)
-{
- if (NAPI_GRO_CB(skb)->csum_valid)
- NAPI_GRO_CB(skb)->csum = csum_sub(NAPI_GRO_CB(skb)->csum,
- csum_partial(start, len, 0));
-}
-
-/* GRO checksum functions. These are logical equivalents of the normal
- * checksum functions (in skbuff.h) except that they operate on the GRO
- * offsets and fields in sk_buff.
- */
-
-__sum16 __skb_gro_checksum_complete(struct sk_buff *skb);
-
-static inline bool skb_at_gro_remcsum_start(struct sk_buff *skb)
-{
- return (NAPI_GRO_CB(skb)->gro_remcsum_start == skb_gro_offset(skb));
-}
-
-static inline bool __skb_gro_checksum_validate_needed(struct sk_buff *skb,
- bool zero_okay,
- __sum16 check)
-{
- return ((skb->ip_summed != CHECKSUM_PARTIAL ||
- skb_checksum_start_offset(skb) <
- skb_gro_offset(skb)) &&
- !skb_at_gro_remcsum_start(skb) &&
- NAPI_GRO_CB(skb)->csum_cnt == 0 &&
- (!zero_okay || check));
-}
-
-static inline __sum16 __skb_gro_checksum_validate_complete(struct sk_buff *skb,
- __wsum psum)
-{
- if (NAPI_GRO_CB(skb)->csum_valid &&
- !csum_fold(csum_add(psum, NAPI_GRO_CB(skb)->csum)))
- return 0;
-
- NAPI_GRO_CB(skb)->csum = psum;
-
- return __skb_gro_checksum_complete(skb);
-}
-
-static inline void skb_gro_incr_csum_unnecessary(struct sk_buff *skb)
-{
- if (NAPI_GRO_CB(skb)->csum_cnt > 0) {
- /* Consume a checksum from CHECKSUM_UNNECESSARY */
- NAPI_GRO_CB(skb)->csum_cnt--;
- } else {
- /* Update skb for CHECKSUM_UNNECESSARY and csum_level when we
- * verified a new top level checksum or an encapsulated one
- * during GRO. This saves work if we fallback to normal path.
- */
- __skb_incr_checksum_unnecessary(skb);
- }
-}
-
-#define __skb_gro_checksum_validate(skb, proto, zero_okay, check, \
- compute_pseudo) \
-({ \
- __sum16 __ret = 0; \
- if (__skb_gro_checksum_validate_needed(skb, zero_okay, check)) \
- __ret = __skb_gro_checksum_validate_complete(skb, \
- compute_pseudo(skb, proto)); \
- if (!__ret) \
- skb_gro_incr_csum_unnecessary(skb); \
- __ret; \
-})
-
-#define skb_gro_checksum_validate(skb, proto, compute_pseudo) \
- __skb_gro_checksum_validate(skb, proto, false, 0, compute_pseudo)
-
-#define skb_gro_checksum_validate_zero_check(skb, proto, check, \
- compute_pseudo) \
- __skb_gro_checksum_validate(skb, proto, true, check, compute_pseudo)
-
-#define skb_gro_checksum_simple_validate(skb) \
- __skb_gro_checksum_validate(skb, 0, false, 0, null_compute_pseudo)
-
-static inline bool __skb_gro_checksum_convert_check(struct sk_buff *skb)
-{
- return (NAPI_GRO_CB(skb)->csum_cnt == 0 &&
- !NAPI_GRO_CB(skb)->csum_valid);
-}
-
-static inline void __skb_gro_checksum_convert(struct sk_buff *skb,
- __wsum pseudo)
-{
- NAPI_GRO_CB(skb)->csum = ~pseudo;
- NAPI_GRO_CB(skb)->csum_valid = 1;
-}
-
-#define skb_gro_checksum_try_convert(skb, proto, compute_pseudo) \
-do { \
- if (__skb_gro_checksum_convert_check(skb)) \
- __skb_gro_checksum_convert(skb, \
- compute_pseudo(skb, proto)); \
-} while (0)
-
-struct gro_remcsum {
- int offset;
- __wsum delta;
-};
-
-static inline void skb_gro_remcsum_init(struct gro_remcsum *grc)
-{
- grc->offset = 0;
- grc->delta = 0;
-}
-
-static inline void *skb_gro_remcsum_process(struct sk_buff *skb, void *ptr,
- unsigned int off, size_t hdrlen,
- int start, int offset,
- struct gro_remcsum *grc,
- bool nopartial)
-{
- __wsum delta;
- size_t plen = hdrlen + max_t(size_t, offset + sizeof(u16), start);
-
- BUG_ON(!NAPI_GRO_CB(skb)->csum_valid);
-
- if (!nopartial) {
- NAPI_GRO_CB(skb)->gro_remcsum_start = off + hdrlen + start;
- return ptr;
- }
-
- ptr = skb_gro_header_fast(skb, off);
- if (skb_gro_header_hard(skb, off + plen)) {
- ptr = skb_gro_header_slow(skb, off + plen, off);
- if (!ptr)
- return NULL;
- }
-
- delta = remcsum_adjust(ptr + hdrlen, NAPI_GRO_CB(skb)->csum,
- start, offset);
-
- /* Adjust skb->csum since we changed the packet */
- NAPI_GRO_CB(skb)->csum = csum_add(NAPI_GRO_CB(skb)->csum, delta);
-
- grc->offset = off + hdrlen + offset;
- grc->delta = delta;
-
- return ptr;
-}
-
-static inline void skb_gro_remcsum_cleanup(struct sk_buff *skb,
- struct gro_remcsum *grc)
-{
- void *ptr;
- size_t plen = grc->offset + sizeof(u16);
-
- if (!grc->delta)
- return;
-
- ptr = skb_gro_header_fast(skb, grc->offset);
- if (skb_gro_header_hard(skb, grc->offset + sizeof(u16))) {
- ptr = skb_gro_header_slow(skb, plen, grc->offset);
- if (!ptr)
- return;
- }
-
- remcsum_unadjust((__sum16 *)ptr, grc->delta);
-}
-
-#ifdef CONFIG_XFRM_OFFLOAD
-static inline void skb_gro_flush_final(struct sk_buff *skb, struct sk_buff *pp, int flush)
-{
- if (PTR_ERR(pp) != -EINPROGRESS)
- NAPI_GRO_CB(skb)->flush |= flush;
-}
-static inline void skb_gro_flush_final_remcsum(struct sk_buff *skb,
- struct sk_buff *pp,
- int flush,
- struct gro_remcsum *grc)
-{
- if (PTR_ERR(pp) != -EINPROGRESS) {
- NAPI_GRO_CB(skb)->flush |= flush;
- skb_gro_remcsum_cleanup(skb, grc);
- skb->remcsum_offload = 0;
- }
-}
-#else
-static inline void skb_gro_flush_final(struct sk_buff *skb, struct sk_buff *pp, int flush)
-{
- NAPI_GRO_CB(skb)->flush |= flush;
-}
-static inline void skb_gro_flush_final_remcsum(struct sk_buff *skb,
- struct sk_buff *pp,
- int flush,
- struct gro_remcsum *grc)
-{
- NAPI_GRO_CB(skb)->flush |= flush;
- skb_gro_remcsum_cleanup(skb, grc);
- skb->remcsum_offload = 0;
-}
-#endif
static inline int dev_hard_header(struct sk_buff *skb, struct net_device *dev,
unsigned short type,
@@ -4007,6 +3672,7 @@ int netif_rx_ni(struct sk_buff *skb);
int netif_rx_any_context(struct sk_buff *skb);
int netif_receive_skb(struct sk_buff *skb);
int netif_receive_skb_core(struct sk_buff *skb);
+void netif_receive_skb_list_internal(struct list_head *head);
void netif_receive_skb_list(struct list_head *head);
gro_result_t napi_gro_receive(struct napi_struct *napi, struct sk_buff *skb);
void napi_gro_flush(struct napi_struct *napi, bool flush_old);
@@ -4080,7 +3746,6 @@ int dev_get_port_parent_id(struct net_device *dev,
struct netdev_phys_item_id *ppid, bool recurse);
bool netdev_port_same_parent_id(struct net_device *a, struct net_device *b);
int dev_change_proto_down(struct net_device *dev, bool proto_down);
-int dev_change_proto_down_generic(struct net_device *dev, bool proto_down);
void dev_change_proto_down_reason(struct net_device *dev, unsigned long mask,
u32 value);
struct sk_buff *validate_xmit_skb_list(struct sk_buff *skb, struct net_device *dev, bool *again);
@@ -4156,6 +3821,7 @@ void netdev_run_todo(void);
* @dev: network device
*
* Release reference to device to allow it to be freed.
+ * Try using dev_put_track() instead.
*/
static inline void dev_put(struct net_device *dev)
{
@@ -4173,6 +3839,7 @@ static inline void dev_put(struct net_device *dev)
* @dev: network device
*
* Hold reference to device to keep it from being freed.
+ * Try using dev_hold_track() instead.
*/
static inline void dev_hold(struct net_device *dev)
{
@@ -4185,6 +3852,55 @@ static inline void dev_hold(struct net_device *dev)
}
}
+static inline void netdev_tracker_alloc(struct net_device *dev,
+ netdevice_tracker *tracker, gfp_t gfp)
+{
+#ifdef CONFIG_NET_DEV_REFCNT_TRACKER
+ ref_tracker_alloc(&dev->refcnt_tracker, tracker, gfp);
+#endif
+}
+
+static inline void netdev_tracker_free(struct net_device *dev,
+ netdevice_tracker *tracker)
+{
+#ifdef CONFIG_NET_DEV_REFCNT_TRACKER
+ ref_tracker_free(&dev->refcnt_tracker, tracker);
+#endif
+}
+
+static inline void dev_hold_track(struct net_device *dev,
+ netdevice_tracker *tracker, gfp_t gfp)
+{
+ if (dev) {
+ dev_hold(dev);
+ netdev_tracker_alloc(dev, tracker, gfp);
+ }
+}
+
+static inline void dev_put_track(struct net_device *dev,
+ netdevice_tracker *tracker)
+{
+ if (dev) {
+ netdev_tracker_free(dev, tracker);
+ dev_put(dev);
+ }
+}
+
+static inline void dev_replace_track(struct net_device *odev,
+ struct net_device *ndev,
+ netdevice_tracker *tracker,
+ gfp_t gfp)
+{
+ if (odev)
+ netdev_tracker_free(odev, tracker);
+
+ dev_hold(ndev);
+ dev_put(odev);
+
+ if (ndev)
+ netdev_tracker_alloc(ndev, tracker, gfp);
+}
+
/* Carrier loss detection, dial on demand. The functions netif_carrier_on
* and _off may be called from IRQ context, but it is caller
* who is responsible for serialization of these calls.
@@ -4451,10 +4167,21 @@ static inline void __netif_tx_unlock_bh(struct netdev_queue *txq)
spin_unlock_bh(&txq->_xmit_lock);
}
+/*
+ * txq->trans_start can be read locklessly from dev_watchdog()
+ */
static inline void txq_trans_update(struct netdev_queue *txq)
{
if (txq->xmit_lock_owner != -1)
- txq->trans_start = jiffies;
+ WRITE_ONCE(txq->trans_start, jiffies);
+}
+
+static inline void txq_trans_cond_update(struct netdev_queue *txq)
+{
+ unsigned long now = jiffies;
+
+ if (READ_ONCE(txq->trans_start) != now)
+ WRITE_ONCE(txq->trans_start, now);
}
/* legacy drivers only, netdev_start_xmit() sets txq->trans_start */
@@ -4462,8 +4189,7 @@ static inline void netif_trans_update(struct net_device *dev)
{
struct netdev_queue *txq = netdev_get_tx_queue(dev, 0);
- if (txq->trans_start != jiffies)
- txq->trans_start = jiffies;
+ txq_trans_cond_update(txq);
}
/**
@@ -4472,27 +4198,7 @@ static inline void netif_trans_update(struct net_device *dev)
*
* Get network device transmit lock
*/
-static inline void netif_tx_lock(struct net_device *dev)
-{
- unsigned int i;
- int cpu;
-
- spin_lock(&dev->tx_global_lock);
- cpu = smp_processor_id();
- for (i = 0; i < dev->num_tx_queues; i++) {
- struct netdev_queue *txq = netdev_get_tx_queue(dev, i);
-
- /* We are the only thread of execution doing a
- * freeze, but we have to grab the _xmit_lock in
- * order to synchronize with threads which are in
- * the ->hard_start_xmit() handler and already
- * checked the frozen bit.
- */
- __netif_tx_lock(txq, cpu);
- set_bit(__QUEUE_STATE_FROZEN, &txq->state);
- __netif_tx_unlock(txq);
- }
-}
+void netif_tx_lock(struct net_device *dev);
static inline void netif_tx_lock_bh(struct net_device *dev)
{
@@ -4500,22 +4206,7 @@ static inline void netif_tx_lock_bh(struct net_device *dev)
netif_tx_lock(dev);
}
-static inline void netif_tx_unlock(struct net_device *dev)
-{
- unsigned int i;
-
- for (i = 0; i < dev->num_tx_queues; i++) {
- struct netdev_queue *txq = netdev_get_tx_queue(dev, i);
-
- /* No need to grab the _xmit_lock here. If the
- * queue is not stopped for another reason, we
- * force a schedule.
- */
- clear_bit(__QUEUE_STATE_FROZEN, &txq->state);
- netif_schedule_queue(txq);
- }
- spin_unlock(&dev->tx_global_lock);
-}
+void netif_tx_unlock(struct net_device *dev);
static inline void netif_tx_unlock_bh(struct net_device *dev)
{
@@ -4649,10 +4340,13 @@ void __hw_addr_unsync_dev(struct netdev_hw_addr_list *list,
void __hw_addr_init(struct netdev_hw_addr_list *list);
/* Functions used for device addresses handling */
+void dev_addr_mod(struct net_device *dev, unsigned int offset,
+ const void *addr, size_t len);
+
static inline void
__dev_addr_set(struct net_device *dev, const void *addr, size_t len)
{
- memcpy(dev->dev_addr, addr, len);
+ dev_addr_mod(dev, 0, addr, len);
}
static inline void dev_addr_set(struct net_device *dev, const u8 *addr)
@@ -4660,19 +4354,13 @@ static inline void dev_addr_set(struct net_device *dev, const u8 *addr)
__dev_addr_set(dev, addr, dev->addr_len);
}
-static inline void
-dev_addr_mod(struct net_device *dev, unsigned int offset,
- const void *addr, size_t len)
-{
- memcpy(&dev->dev_addr[offset], addr, len);
-}
-
int dev_addr_add(struct net_device *dev, const unsigned char *addr,
unsigned char addr_type);
int dev_addr_del(struct net_device *dev, const unsigned char *addr,
unsigned char addr_type);
void dev_addr_flush(struct net_device *dev);
int dev_addr_init(struct net_device *dev);
+void dev_addr_check(struct net_device *dev);
/* Functions used for unicast addresses handling */
int dev_uc_add(struct net_device *dev, const unsigned char *addr);
@@ -5111,7 +4799,22 @@ static inline bool netif_needs_gso(struct sk_buff *skb,
static inline void netif_set_gso_max_size(struct net_device *dev,
unsigned int size)
{
- dev->gso_max_size = size;
+ /* dev->gso_max_size is read locklessly from sk_setup_caps() */
+ WRITE_ONCE(dev->gso_max_size, size);
+}
+
+static inline void netif_set_gso_max_segs(struct net_device *dev,
+ unsigned int segs)
+{
+ /* dev->gso_max_segs is read locklessly from sk_setup_caps() */
+ WRITE_ONCE(dev->gso_max_segs, segs);
+}
+
+static inline void netif_set_gro_max_size(struct net_device *dev,
+ unsigned int size)
+{
+ /* This pairs with the READ_ONCE() in skb_gro_receive() */
+ WRITE_ONCE(dev->gro_max_size, size);
}
static inline void skb_gso_error_unwind(struct sk_buff *skb, __be16 protocol,
@@ -5298,7 +5001,7 @@ void netdev_info(const struct net_device *dev, const char *format, ...);
#define netdev_level_once(level, dev, fmt, ...) \
do { \
- static bool __print_once __read_mostly; \
+ static bool __section(".data.once") __print_once; \
\
if (!__print_once) { \
__print_once = true; \
diff --git a/include/linux/netfilter.h b/include/linux/netfilter.h
index 3fda1a5..15e71bf 100644
--- a/include/linux/netfilter.h
+++ b/include/linux/netfilter.h
@@ -381,13 +381,13 @@ struct nf_nat_hook {
enum ip_conntrack_dir dir);
};
-extern struct nf_nat_hook __rcu *nf_nat_hook;
+extern const struct nf_nat_hook __rcu *nf_nat_hook;
static inline void
nf_nat_decode_session(struct sk_buff *skb, struct flowi *fl, u_int8_t family)
{
#if IS_ENABLED(CONFIG_NF_NAT)
- struct nf_nat_hook *nat_hook;
+ const struct nf_nat_hook *nat_hook;
rcu_read_lock();
nat_hook = rcu_dereference(nf_nat_hook);
@@ -440,7 +440,6 @@ nf_nat_decode_session(struct sk_buff *skb, struct flowi *fl, u_int8_t family)
#if IS_ENABLED(CONFIG_NF_CONNTRACK)
#include <linux/netfilter/nf_conntrack_zones_common.h>
-extern void (*ip_ct_attach)(struct sk_buff *, const struct sk_buff *) __rcu;
void nf_ct_attach(struct sk_buff *, const struct sk_buff *);
struct nf_conntrack_tuple;
bool nf_ct_get_tuple_skb(struct nf_conntrack_tuple *dst_tuple,
@@ -463,8 +462,9 @@ struct nf_ct_hook {
void (*destroy)(struct nf_conntrack *);
bool (*get_tuple_skb)(struct nf_conntrack_tuple *,
const struct sk_buff *);
+ void (*attach)(struct sk_buff *nskb, const struct sk_buff *skb);
};
-extern struct nf_ct_hook __rcu *nf_ct_hook;
+extern const struct nf_ct_hook __rcu *nf_ct_hook;
struct nlattr;
@@ -479,7 +479,7 @@ struct nfnl_ct_hook {
void (*seq_adjust)(struct sk_buff *skb, struct nf_conn *ct,
enum ip_conntrack_info ctinfo, s32 off);
};
-extern struct nfnl_ct_hook __rcu *nfnl_ct_hook;
+extern const struct nfnl_ct_hook __rcu *nfnl_ct_hook;
/**
* nf_skb_duplicated - TEE target has sent a packet
diff --git a/include/linux/netfilter/nf_conntrack_common.h b/include/linux/netfilter/nf_conntrack_common.h
index 700ea07..2770db2 100644
--- a/include/linux/netfilter/nf_conntrack_common.h
+++ b/include/linux/netfilter/nf_conntrack_common.h
@@ -2,7 +2,7 @@
#ifndef _NF_CONNTRACK_COMMON_H
#define _NF_CONNTRACK_COMMON_H
-#include <linux/atomic.h>
+#include <linux/refcount.h>
#include <uapi/linux/netfilter/nf_conntrack_common.h>
struct ip_conntrack_stat {
@@ -25,19 +25,21 @@ struct ip_conntrack_stat {
#define NFCT_PTRMASK ~(NFCT_INFOMASK)
struct nf_conntrack {
- atomic_t use;
+ refcount_t use;
};
void nf_conntrack_destroy(struct nf_conntrack *nfct);
+
+/* like nf_ct_put, but without module dependency on nf_conntrack */
static inline void nf_conntrack_put(struct nf_conntrack *nfct)
{
- if (nfct && atomic_dec_and_test(&nfct->use))
+ if (nfct && refcount_dec_and_test(&nfct->use))
nf_conntrack_destroy(nfct);
}
static inline void nf_conntrack_get(struct nf_conntrack *nfct)
{
if (nfct)
- atomic_inc(&nfct->use);
+ refcount_inc(&nfct->use);
}
#endif /* _NF_CONNTRACK_COMMON_H */
diff --git a/include/linux/netfilter_netdev.h b/include/linux/netfilter_netdev.h
index b71b57a..b4dd96e 100644
--- a/include/linux/netfilter_netdev.h
+++ b/include/linux/netfilter_netdev.h
@@ -94,7 +94,7 @@ static inline struct sk_buff *nf_hook_egress(struct sk_buff *skb, int *rc,
return skb;
#endif
- e = rcu_dereference(dev->nf_hooks_egress);
+ e = rcu_dereference_check(dev->nf_hooks_egress, rcu_read_lock_bh_held());
if (!e)
return skb;
diff --git a/include/linux/netpoll.h b/include/linux/netpoll.h
index e6a2d72e..bd19c4b 100644
--- a/include/linux/netpoll.h
+++ b/include/linux/netpoll.h
@@ -24,6 +24,7 @@ union inet_addr {
struct netpoll {
struct net_device *dev;
+ netdevice_tracker dev_tracker;
char dev_name[IFNAMSIZ];
const char *name;
diff --git a/include/linux/numa.h b/include/linux/numa.h
index cb44cfe..59df211 100644
--- a/include/linux/numa.h
+++ b/include/linux/numa.h
@@ -58,4 +58,8 @@ static inline int phys_to_target_node(u64 start)
}
#endif
+#ifdef CONFIG_HAVE_ARCH_NODE_DEV_GROUP
+extern const struct attribute_group arch_node_dev_group;
+#endif
+
#endif /* _LINUX_NUMA_H */
diff --git a/include/linux/once.h b/include/linux/once.h
index d361fb1..f545230 100644
--- a/include/linux/once.h
+++ b/include/linux/once.h
@@ -38,7 +38,7 @@ void __do_once_done(bool *done, struct static_key_true *once_key,
#define DO_ONCE(func, ...) \
({ \
bool ___ret = false; \
- static bool ___done = false; \
+ static bool __section(".data.once") ___done = false; \
static DEFINE_STATIC_KEY_TRUE(___once_key); \
if (static_branch_unlikely(&___once_key)) { \
unsigned long ___flags; \
diff --git a/include/linux/pcs-lynx.h b/include/linux/pcs-lynx.h
index a6440d6..5712cc2 100644
--- a/include/linux/pcs-lynx.h
+++ b/include/linux/pcs-lynx.h
@@ -9,13 +9,10 @@
#include <linux/mdio.h>
#include <linux/phylink.h>
-struct lynx_pcs {
- struct phylink_pcs pcs;
- struct mdio_device *mdio;
-};
+struct mdio_device *lynx_get_mdio_device(struct phylink_pcs *pcs);
-struct lynx_pcs *lynx_pcs_create(struct mdio_device *mdio);
+struct phylink_pcs *lynx_pcs_create(struct mdio_device *mdio);
-void lynx_pcs_destroy(struct lynx_pcs *pcs);
+void lynx_pcs_destroy(struct phylink_pcs *pcs);
#endif /* __LINUX_PCS_LYNX_H */
diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h
index 0dcfd26..474eaec 100644
--- a/include/linux/perf_event.h
+++ b/include/linux/perf_event.h
@@ -130,6 +130,15 @@ struct hw_perf_event_extra {
};
/**
+ * hw_perf_event::flag values
+ *
+ * PERF_EVENT_FLAG_ARCH bits are reserved for architecture-specific
+ * usage.
+ */
+#define PERF_EVENT_FLAG_ARCH 0x0000ffff
+#define PERF_EVENT_FLAG_USER_READ_CNT 0x80000000
+
+/**
* struct hw_perf_event - performance event hardware details:
*/
struct hw_perf_event {
@@ -611,6 +620,7 @@ struct swevent_hlist {
#define PERF_ATTACH_SCHED_CB 0x20
#define PERF_ATTACH_CHILD 0x40
+struct bpf_prog;
struct perf_cgroup;
struct perf_buffer;
@@ -822,6 +832,7 @@ struct perf_event_context {
int nr_events;
int nr_active;
+ int nr_user;
int is_active;
int nr_stat;
int nr_freq;
diff --git a/include/linux/phy.h b/include/linux/phy.h
index cbf03a5..6de8d7a 100644
--- a/include/linux/phy.h
+++ b/include/linux/phy.h
@@ -99,7 +99,7 @@ extern const int phy_10gbit_features_array[1];
* @PHY_INTERFACE_MODE_RGMII_RXID: RGMII with Internal RX delay
* @PHY_INTERFACE_MODE_RGMII_TXID: RGMII with Internal RX delay
* @PHY_INTERFACE_MODE_RTBI: Reduced TBI
- * @PHY_INTERFACE_MODE_SMII: ??? MII
+ * @PHY_INTERFACE_MODE_SMII: Serial MII
* @PHY_INTERFACE_MODE_XGMII: 10 gigabit media-independent interface
* @PHY_INTERFACE_MODE_XLGMII:40 gigabit media-independent interface
* @PHY_INTERFACE_MODE_MOCA: Multimedia over Coax
diff --git a/include/linux/phylink.h b/include/linux/phylink.h
index f037470..713a0c9 100644
--- a/include/linux/phylink.h
+++ b/include/linux/phylink.h
@@ -20,6 +20,29 @@ enum {
MLO_AN_PHY = 0, /* Conventional PHY */
MLO_AN_FIXED, /* Fixed-link mode */
MLO_AN_INBAND, /* In-band protocol */
+
+ MAC_SYM_PAUSE = BIT(0),
+ MAC_ASYM_PAUSE = BIT(1),
+ MAC_10HD = BIT(2),
+ MAC_10FD = BIT(3),
+ MAC_10 = MAC_10HD | MAC_10FD,
+ MAC_100HD = BIT(4),
+ MAC_100FD = BIT(5),
+ MAC_100 = MAC_100HD | MAC_100FD,
+ MAC_1000HD = BIT(6),
+ MAC_1000FD = BIT(7),
+ MAC_1000 = MAC_1000HD | MAC_1000FD,
+ MAC_2500FD = BIT(8),
+ MAC_5000FD = BIT(9),
+ MAC_10000FD = BIT(10),
+ MAC_20000FD = BIT(11),
+ MAC_25000FD = BIT(12),
+ MAC_40000FD = BIT(13),
+ MAC_50000FD = BIT(14),
+ MAC_56000FD = BIT(15),
+ MAC_100000FD = BIT(16),
+ MAC_200000FD = BIT(17),
+ MAC_400000FD = BIT(18),
};
static inline bool phylink_autoneg_inband(unsigned int mode)
@@ -61,6 +84,8 @@ enum phylink_op_type {
* struct phylink_config - PHYLINK configuration structure
* @dev: a pointer to a struct device associated with the MAC
* @type: operation type of PHYLINK instance
+ * @legacy_pre_march2020: driver has not been updated for March 2020 updates
+ * (See commit 7cceb599d15d ("net: phylink: avoid mac_config calls")
* @pcs_poll: MAC PCS cannot provide link change interrupt
* @poll_fixed_state: if true, starts link_poll,
* if MAC link is at %MLO_AN_FIXED mode.
@@ -69,21 +94,25 @@ enum phylink_op_type {
* if MAC link is at %MLO_AN_FIXED mode.
* @supported_interfaces: bitmap describing which PHY_INTERFACE_MODE_xxx
* are supported by the MAC/PCS.
+ * @mac_capabilities: MAC pause/speed/duplex capabilities.
*/
struct phylink_config {
struct device *dev;
enum phylink_op_type type;
+ bool legacy_pre_march2020;
bool pcs_poll;
bool poll_fixed_state;
bool ovr_an_inband;
void (*get_fixed_state)(struct phylink_config *config,
struct phylink_link_state *state);
DECLARE_PHY_INTERFACE_MASK(supported_interfaces);
+ unsigned long mac_capabilities;
};
/**
* struct phylink_mac_ops - MAC operations structure.
* @validate: Validate and update the link configuration.
+ * @mac_select_pcs: Select a PCS for the interface mode.
* @mac_pcs_get_state: Read the current link state from the hardware.
* @mac_prepare: prepare for a major reconfiguration of the interface.
* @mac_config: configure the MAC for the selected mode and state.
@@ -98,6 +127,8 @@ struct phylink_mac_ops {
void (*validate)(struct phylink_config *config,
unsigned long *supported,
struct phylink_link_state *state);
+ struct phylink_pcs *(*mac_select_pcs)(struct phylink_config *config,
+ phy_interface_t interface);
void (*mac_pcs_get_state)(struct phylink_config *config,
struct phylink_link_state *state);
int (*mac_prepare)(struct phylink_config *config, unsigned int mode,
@@ -150,6 +181,21 @@ struct phylink_mac_ops {
*/
void validate(struct phylink_config *config, unsigned long *supported,
struct phylink_link_state *state);
+/**
+ * mac_select_pcs: Select a PCS for the interface mode.
+ * @config: a pointer to a &struct phylink_config.
+ * @interface: PHY interface mode for PCS
+ *
+ * Return the &struct phylink_pcs for the specified interface mode, or
+ * NULL if none is required, or an error pointer on error.
+ *
+ * This must not modify any state. It is used to query which PCS should
+ * be used. Phylink will use this during validation to ensure that the
+ * configuration is valid, and when setting a configuration to internally
+ * set the PCS that will be used.
+ */
+struct phylink_pcs *mac_select_pcs(struct phylink_config *config,
+ phy_interface_t interface);
/**
* mac_pcs_get_state() - Read the current inband link state from the hardware
@@ -162,6 +208,10 @@ void validate(struct phylink_config *config, unsigned long *supported,
* negotiation completion state in @state->an_complete, and link up state
* in @state->link. If possible, @state->lp_advertising should also be
* populated.
+ *
+ * Note: This is a legacy method. This function will not be called unless
+ * legacy_pre_march2020 is set in &struct phylink_config and there is no
+ * PCS attached.
*/
void mac_pcs_get_state(struct phylink_config *config,
struct phylink_link_state *state);
@@ -202,6 +252,15 @@ int mac_prepare(struct phylink_config *config, unsigned int mode,
* guaranteed to be correct, and so any mac_config() implementation must
* never reference these fields.
*
+ * Note: For legacy March 2020 drivers (drivers with legacy_pre_march2020 set
+ * in their &phylnk_config and which don't have a PCS), this function will be
+ * called on each link up event, and to also change the in-band advert. For
+ * non-legacy drivers, it will only be called to reconfigure the MAC for a
+ * "major" change in e.g. interface mode. It will not be called for changes
+ * in speed, duplex or pause modes or to change the in-band advertisement.
+ * In any case, it is strongly preferred that speed, duplex and pause settings
+ * are handled in the mac_link_up() method and not in this method.
+ *
* (this requires a rewrite - please refer to mac_link_up() for situations
* where the PCS and MAC are not tightly integrated.)
*
@@ -286,6 +345,10 @@ int mac_finish(struct phylink_config *config, unsigned int mode,
/**
* mac_an_restart() - restart 802.3z BaseX autonegotiation
* @config: a pointer to a &struct phylink_config.
+ *
+ * Note: This is a legacy method. This function will not be called unless
+ * legacy_pre_march2020 is set in &struct phylink_config and there is no
+ * PCS attached.
*/
void mac_an_restart(struct phylink_config *config);
@@ -353,6 +416,7 @@ struct phylink_pcs {
/**
* struct phylink_pcs_ops - MAC PCS operations structure.
+ * @pcs_validate: validate the link configuration.
* @pcs_get_state: read the current MAC PCS link state from the hardware.
* @pcs_config: configure the MAC PCS for the selected mode and state.
* @pcs_an_restart: restart 802.3z BaseX autonegotiation.
@@ -360,6 +424,8 @@ struct phylink_pcs {
* (where necessary).
*/
struct phylink_pcs_ops {
+ int (*pcs_validate)(struct phylink_pcs *pcs, unsigned long *supported,
+ const struct phylink_link_state *state);
void (*pcs_get_state)(struct phylink_pcs *pcs,
struct phylink_link_state *state);
int (*pcs_config)(struct phylink_pcs *pcs, unsigned int mode,
@@ -373,6 +439,23 @@ struct phylink_pcs_ops {
#if 0 /* For kernel-doc purposes only. */
/**
+ * pcs_validate() - validate the link configuration.
+ * @pcs: a pointer to a &struct phylink_pcs.
+ * @supported: ethtool bitmask for supported link modes.
+ * @state: a const pointer to a &struct phylink_link_state.
+ *
+ * Validate the interface mode, and advertising's autoneg bit, removing any
+ * media ethtool link modes that would not be supportable from the supported
+ * mask. Phylink will propagate the changes to the advertising mask. See the
+ * &struct phylink_mac_ops validate() method.
+ *
+ * Returns -EINVAL if the interface mode/autoneg mode is not supported.
+ * Returns non-zero positive if the link state can be supported.
+ */
+int pcs_validate(struct phylink_pcs *pcs, unsigned long *supported,
+ const struct phylink_link_state *state);
+
+/**
* pcs_get_state() - Read the current inband link state from the hardware
* @pcs: a pointer to a &struct phylink_pcs.
* @state: a pointer to a &struct phylink_link_state.
@@ -442,6 +525,12 @@ void pcs_link_up(struct phylink_pcs *pcs, unsigned int mode,
phy_interface_t interface, int speed, int duplex);
#endif
+void phylink_get_linkmodes(unsigned long *linkmodes, phy_interface_t interface,
+ unsigned long mac_capabilities);
+void phylink_generic_validate(struct phylink_config *config,
+ unsigned long *supported,
+ struct phylink_link_state *state);
+
struct phylink *phylink_create(struct phylink_config *, struct fwnode_handle *,
phy_interface_t iface,
const struct phylink_mac_ops *mac_ops);
@@ -496,11 +585,12 @@ void phylink_set_port_modes(unsigned long *bits);
void phylink_set_10g_modes(unsigned long *mask);
void phylink_helper_basex_speed(struct phylink_link_state *state);
+void phylink_mii_c22_pcs_decode_state(struct phylink_link_state *state,
+ u16 bmsr, u16 lpa);
void phylink_mii_c22_pcs_get_state(struct mdio_device *pcs,
struct phylink_link_state *state);
-int phylink_mii_c22_pcs_set_advertisement(struct mdio_device *pcs,
- phy_interface_t interface,
- const unsigned long *advertising);
+int phylink_mii_c22_pcs_encode_advertisement(phy_interface_t interface,
+ const unsigned long *advertising);
int phylink_mii_c22_pcs_config(struct mdio_device *pcs, unsigned int mode,
phy_interface_t interface,
const unsigned long *advertising);
diff --git a/include/linux/platform_data/pata_ixp4xx_cf.h b/include/linux/platform_data/pata_ixp4xx_cf.h
deleted file mode 100644
index e60fa41..0000000
--- a/include/linux/platform_data/pata_ixp4xx_cf.h
+++ /dev/null
@@ -1,21 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#ifndef __PLATFORM_DATA_PATA_IXP4XX_H
-#define __PLATFORM_DATA_PATA_IXP4XX_H
-
-#include <linux/types.h>
-
-/*
- * This structure provide a means for the board setup code
- * to give information to th pata_ixp4xx driver. It is
- * passed as platform_data.
- */
-struct ixp4xx_pata_data {
- volatile u32 *cs0_cfg;
- volatile u32 *cs1_cfg;
- unsigned long cs0_bits;
- unsigned long cs1_bits;
- void __iomem *cmd;
- void __iomem *ctl;
-};
-
-#endif
diff --git a/include/linux/platform_data/tps68470.h b/include/linux/platform_data/tps68470.h
new file mode 100644
index 0000000..126d082
--- /dev/null
+++ b/include/linux/platform_data/tps68470.h
@@ -0,0 +1,35 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * TI TPS68470 PMIC platform data definition.
+ *
+ * Copyright (c) 2021 Red Hat Inc.
+ *
+ * Red Hat authors:
+ * Hans de Goede <hdegoede@redhat.com>
+ */
+#ifndef __PDATA_TPS68470_H
+#define __PDATA_TPS68470_H
+
+enum tps68470_regulators {
+ TPS68470_CORE,
+ TPS68470_ANA,
+ TPS68470_VCM,
+ TPS68470_VIO,
+ TPS68470_VSIO,
+ TPS68470_AUX1,
+ TPS68470_AUX2,
+ TPS68470_NUM_REGULATORS
+};
+
+struct regulator_init_data;
+
+struct tps68470_regulator_platform_data {
+ const struct regulator_init_data *reg_init_data[TPS68470_NUM_REGULATORS];
+};
+
+struct tps68470_clk_platform_data {
+ const char *consumer_dev_name;
+ const char *consumer_con_id;
+};
+
+#endif
diff --git a/include/linux/ptp_clock_kernel.h b/include/linux/ptp_clock_kernel.h
index 2e55650..554454c 100644
--- a/include/linux/ptp_clock_kernel.h
+++ b/include/linux/ptp_clock_kernel.h
@@ -351,15 +351,17 @@ int ptp_get_vclocks_index(int pclock_index, int **vclock_index);
*
* @hwtstamps: skb_shared_hwtstamps structure pointer
* @vclock_index: phc index of ptp vclock.
+ *
+ * Returns converted timestamp, or 0 on error.
*/
-void ptp_convert_timestamp(struct skb_shared_hwtstamps *hwtstamps,
- int vclock_index);
+ktime_t ptp_convert_timestamp(const struct skb_shared_hwtstamps *hwtstamps,
+ int vclock_index);
#else
static inline int ptp_get_vclocks_index(int pclock_index, int **vclock_index)
{ return 0; }
-static inline void ptp_convert_timestamp(struct skb_shared_hwtstamps *hwtstamps,
- int vclock_index)
-{ }
+static inline ktime_t ptp_convert_timestamp(const struct skb_shared_hwtstamps *hwtstamps,
+ int vclock_index)
+{ return 0; }
#endif
diff --git a/include/linux/pwm.h b/include/linux/pwm.h
index e6dac95..9771a07 100644
--- a/include/linux/pwm.h
+++ b/include/linux/pwm.h
@@ -414,6 +414,8 @@ struct pwm_device *pwm_request_from_chip(struct pwm_chip *chip,
struct pwm_device *of_pwm_xlate_with_flags(struct pwm_chip *pc,
const struct of_phandle_args *args);
+struct pwm_device *of_pwm_single_xlate(struct pwm_chip *pc,
+ const struct of_phandle_args *args);
struct pwm_device *pwm_get(struct device *dev, const char *con_id);
struct pwm_device *of_pwm_get(struct device *dev, struct device_node *np,
diff --git a/include/linux/qed/qed_if.h b/include/linux/qed/qed_if.h
index 0dae7fc..6dc4943 100644
--- a/include/linux/qed/qed_if.h
+++ b/include/linux/qed/qed_if.h
@@ -652,6 +652,7 @@ struct qed_dev_info {
bool wol_support;
bool smart_an;
+ bool esl;
/* MBI version */
u32 mbi_version;
@@ -807,6 +808,12 @@ struct qed_devlink {
struct devlink_health_reporter *fw_reporter;
};
+struct qed_sb_info_dbg {
+ u32 igu_prod;
+ u32 igu_cons;
+ u16 pi[PIS_PER_SB];
+};
+
struct qed_common_cb_ops {
void (*arfs_filter_op)(void *dev, void *fltr, u8 fw_rc);
void (*link_update)(void *dev, struct qed_link_output *link);
@@ -1194,6 +1201,13 @@ struct qed_common_ops {
struct devlink* (*devlink_register)(struct qed_dev *cdev);
void (*devlink_unregister)(struct devlink *devlink);
+
+ __printf(2, 3) void (*mfw_report)(struct qed_dev *cdev, char *fmt, ...);
+
+ int (*get_sb_info)(struct qed_dev *cdev, struct qed_sb_info *sb,
+ u16 qid, struct qed_sb_info_dbg *sb_dbg);
+
+ int (*get_esl_status)(struct qed_dev *cdev, bool *esl_active);
};
#define MASK_FIELD(_name, _value) \
diff --git a/include/linux/random.h b/include/linux/random.h
index f45b8be..c45b269 100644
--- a/include/linux/random.h
+++ b/include/linux/random.h
@@ -35,7 +35,7 @@ static inline void add_latent_entropy(void) {}
extern void add_input_randomness(unsigned int type, unsigned int code,
unsigned int value) __latent_entropy;
-extern void add_interrupt_randomness(int irq, int irq_flags) __latent_entropy;
+extern void add_interrupt_randomness(int irq) __latent_entropy;
extern void get_random_bytes(void *buf, int nbytes);
extern int wait_for_random_bytes(void);
diff --git a/include/linux/ref_tracker.h b/include/linux/ref_tracker.h
new file mode 100644
index 0000000..c11c9db
--- /dev/null
+++ b/include/linux/ref_tracker.h
@@ -0,0 +1,73 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+#ifndef _LINUX_REF_TRACKER_H
+#define _LINUX_REF_TRACKER_H
+#include <linux/refcount.h>
+#include <linux/types.h>
+#include <linux/spinlock.h>
+
+struct ref_tracker;
+
+struct ref_tracker_dir {
+#ifdef CONFIG_REF_TRACKER
+ spinlock_t lock;
+ unsigned int quarantine_avail;
+ refcount_t untracked;
+ struct list_head list; /* List of active trackers */
+ struct list_head quarantine; /* List of dead trackers */
+#endif
+};
+
+#ifdef CONFIG_REF_TRACKER
+static inline void ref_tracker_dir_init(struct ref_tracker_dir *dir,
+ unsigned int quarantine_count)
+{
+ INIT_LIST_HEAD(&dir->list);
+ INIT_LIST_HEAD(&dir->quarantine);
+ spin_lock_init(&dir->lock);
+ dir->quarantine_avail = quarantine_count;
+ refcount_set(&dir->untracked, 1);
+}
+
+void ref_tracker_dir_exit(struct ref_tracker_dir *dir);
+
+void ref_tracker_dir_print(struct ref_tracker_dir *dir,
+ unsigned int display_limit);
+
+int ref_tracker_alloc(struct ref_tracker_dir *dir,
+ struct ref_tracker **trackerp, gfp_t gfp);
+
+int ref_tracker_free(struct ref_tracker_dir *dir,
+ struct ref_tracker **trackerp);
+
+#else /* CONFIG_REF_TRACKER */
+
+static inline void ref_tracker_dir_init(struct ref_tracker_dir *dir,
+ unsigned int quarantine_count)
+{
+}
+
+static inline void ref_tracker_dir_exit(struct ref_tracker_dir *dir)
+{
+}
+
+static inline void ref_tracker_dir_print(struct ref_tracker_dir *dir,
+ unsigned int display_limit)
+{
+}
+
+static inline int ref_tracker_alloc(struct ref_tracker_dir *dir,
+ struct ref_tracker **trackerp,
+ gfp_t gfp)
+{
+ return 0;
+}
+
+static inline int ref_tracker_free(struct ref_tracker_dir *dir,
+ struct ref_tracker **trackerp)
+{
+ return 0;
+}
+
+#endif
+
+#endif /* _LINUX_REF_TRACKER_H */
diff --git a/include/linux/regmap.h b/include/linux/regmap.h
index e3c9a25..22652e5 100644
--- a/include/linux/regmap.h
+++ b/include/linux/regmap.h
@@ -290,6 +290,11 @@ typedef void (*regmap_unlock)(void *);
* read operation on a bus such as SPI, I2C, etc. Most of the
* devices do not need this.
* @reg_write: Same as above for writing.
+ * @reg_update_bits: Optional callback that if filled will be used to perform
+ * all the update_bits(rmw) operation. Should only be provided
+ * if the function require special handling with lock and reg
+ * handling and the operation cannot be represented as a simple
+ * update_bits operation on a bus such as SPI, I2C, etc.
* @fast_io: Register IO is fast. Use a spinlock instead of a mutex
* to perform locking. This field is ignored if custom lock/unlock
* functions are used (see fields lock/unlock of struct regmap_config).
@@ -372,6 +377,8 @@ struct regmap_config {
int (*reg_read)(void *context, unsigned int reg, unsigned int *val);
int (*reg_write)(void *context, unsigned int reg, unsigned int val);
+ int (*reg_update_bits)(void *context, unsigned int reg,
+ unsigned int mask, unsigned int val);
bool fast_io;
diff --git a/include/linux/rfkill.h b/include/linux/rfkill.h
index 231e06b..c35f396 100644
--- a/include/linux/rfkill.h
+++ b/include/linux/rfkill.h
@@ -230,6 +230,13 @@ void rfkill_set_states(struct rfkill *rfkill, bool sw, bool hw);
bool rfkill_blocked(struct rfkill *rfkill);
/**
+ * rfkill_soft_blocked - Query soft rfkill block state
+ *
+ * @rfkill: rfkill struct to query
+ */
+bool rfkill_soft_blocked(struct rfkill *rfkill);
+
+/**
* rfkill_find_type - Helper for finding rfkill type by name
* @name: the name of the type
*
diff --git a/include/linux/seq_file_net.h b/include/linux/seq_file_net.h
index 0fdbe1d..b97912f 100644
--- a/include/linux/seq_file_net.h
+++ b/include/linux/seq_file_net.h
@@ -9,7 +9,8 @@ extern struct net init_net;
struct seq_net_private {
#ifdef CONFIG_NET_NS
- struct net *net;
+ struct net *net;
+ netns_tracker ns_tracker;
#endif
};
diff --git a/include/linux/siphash.h b/include/linux/siphash.h
index 0cda618..cce8a9a 100644
--- a/include/linux/siphash.h
+++ b/include/linux/siphash.h
@@ -21,6 +21,8 @@ typedef struct {
u64 key[2];
} siphash_key_t;
+#define siphash_aligned_key_t siphash_key_t __aligned(16)
+
static inline bool siphash_key_is_zero(const siphash_key_t *key)
{
return !(key->key[0] | key->key[1]);
diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
index 4507d77..bf11e1f 100644
--- a/include/linux/skbuff.h
+++ b/include/linux/skbuff.h
@@ -36,6 +36,7 @@
#include <linux/splice.h>
#include <linux/in6.h>
#include <linux/if_packet.h>
+#include <linux/llist.h>
#include <net/flow.h>
#include <net/page_pool.h>
#if IS_ENABLED(CONFIG_NF_CONNTRACK)
@@ -287,14 +288,18 @@ struct tc_skb_ext {
__u32 chain;
__u16 mru;
__u16 zone;
- bool post_ct;
+ u8 post_ct:1;
+ u8 post_ct_snat:1;
+ u8 post_ct_dnat:1;
};
#endif
struct sk_buff_head {
- /* These two members must be first. */
- struct sk_buff *next;
- struct sk_buff *prev;
+ /* These two members must be first to match sk_buff. */
+ struct_group_tagged(sk_buff_list, list,
+ struct sk_buff *next;
+ struct sk_buff *prev;
+ );
__u32 qlen;
spinlock_t lock;
@@ -302,6 +307,22 @@ struct sk_buff_head {
struct sk_buff;
+/* The reason of skb drop, which is used in kfree_skb_reason().
+ * en...maybe they should be splited by group?
+ *
+ * Each item here should also be in 'TRACE_SKB_DROP_REASON', which is
+ * used to translate the reason to string.
+ */
+enum skb_drop_reason {
+ SKB_DROP_REASON_NOT_SPECIFIED,
+ SKB_DROP_REASON_NO_SOCKET,
+ SKB_DROP_REASON_PKT_TOO_SMALL,
+ SKB_DROP_REASON_TCP_CSUM,
+ SKB_DROP_REASON_TCP_FILTER,
+ SKB_DROP_REASON_UDP_CSUM,
+ SKB_DROP_REASON_MAX,
+};
+
/* To allow 64K frame to be packed as single skb without frag_list we
* require 64K/PAGE_SIZE pages plus 1 additional page to allow for
* buffers which do not start on a page boundary.
@@ -627,6 +648,7 @@ typedef unsigned char *sk_buff_data_t;
* for retransmit timer
* @rbnode: RB tree node, alternative to next/prev for netem/tcp
* @list: queue head
+ * @ll_node: anchor in an llist (eg socket defer_list)
* @sk: Socket we are owned by
* @ip_defrag_offset: (aka @sk) alternate use of @sk, used in
* fragmentation management
@@ -729,7 +751,7 @@ typedef unsigned char *sk_buff_data_t;
struct sk_buff {
union {
struct {
- /* These two members must be first. */
+ /* These two members must be first to match sk_buff_head. */
struct sk_buff *next;
struct sk_buff *prev;
@@ -744,6 +766,7 @@ struct sk_buff {
};
struct rb_node rbnode; /* used in netem, ip4 defrag, and tcp stack */
struct list_head list;
+ struct llist_node ll_node;
};
union {
@@ -793,7 +816,7 @@ struct sk_buff {
#else
#define CLONED_MASK 1
#endif
-#define CLONED_OFFSET() offsetof(struct sk_buff, __cloned_offset)
+#define CLONED_OFFSET offsetof(struct sk_buff, __cloned_offset)
/* private: */
__u8 __cloned_offset[0];
@@ -809,25 +832,15 @@ struct sk_buff {
__u8 active_extensions;
#endif
- /* fields enclosed in headers_start/headers_end are copied
+ /* Fields enclosed in headers group are copied
* using a single memcpy() in __copy_skb_header()
*/
- /* private: */
- __u32 headers_start[0];
- /* public: */
-
-/* if you move pkt_type around you also must adapt those constants */
-#ifdef __BIG_ENDIAN_BITFIELD
-#define PKT_TYPE_MAX (7 << 5)
-#else
-#define PKT_TYPE_MAX 7
-#endif
-#define PKT_TYPE_OFFSET() offsetof(struct sk_buff, __pkt_type_offset)
+ struct_group(headers,
/* private: */
__u8 __pkt_type_offset[0];
/* public: */
- __u8 pkt_type:3;
+ __u8 pkt_type:3; /* see PKT_TYPE_MAX */
__u8 ignore_df:1;
__u8 nf_trace:1;
__u8 ip_summed:2;
@@ -843,16 +856,10 @@ struct sk_buff {
__u8 encap_hdr_csum:1;
__u8 csum_valid:1;
-#ifdef __BIG_ENDIAN_BITFIELD
-#define PKT_VLAN_PRESENT_BIT 7
-#else
-#define PKT_VLAN_PRESENT_BIT 0
-#endif
-#define PKT_VLAN_PRESENT_OFFSET() offsetof(struct sk_buff, __pkt_vlan_present_offset)
/* private: */
__u8 __pkt_vlan_present_offset[0];
/* public: */
- __u8 vlan_present:1;
+ __u8 vlan_present:1; /* See PKT_VLAN_PRESENT_BIT */
__u8 csum_complete_sw:1;
__u8 csum_level:2;
__u8 csum_not_inet:1;
@@ -933,9 +940,7 @@ struct sk_buff {
u64 kcov_handle;
#endif
- /* private: */
- __u32 headers_end[0];
- /* public: */
+ ); /* end headers group */
/* These elements must be at the end, see alloc_skb() for details. */
sk_buff_data_t tail;
@@ -951,6 +956,22 @@ struct sk_buff {
#endif
};
+/* if you move pkt_type around you also must adapt those constants */
+#ifdef __BIG_ENDIAN_BITFIELD
+#define PKT_TYPE_MAX (7 << 5)
+#else
+#define PKT_TYPE_MAX 7
+#endif
+#define PKT_TYPE_OFFSET offsetof(struct sk_buff, __pkt_type_offset)
+
+/* if you move pkt_vlan_present around you also must adapt these constants */
+#ifdef __BIG_ENDIAN_BITFIELD
+#define PKT_VLAN_PRESENT_BIT 7
+#else
+#define PKT_VLAN_PRESENT_BIT 0
+#endif
+#define PKT_VLAN_PRESENT_OFFSET offsetof(struct sk_buff, __pkt_vlan_present_offset)
+
#ifdef __KERNEL__
/*
* Handling routines are only of interest to the kernel
@@ -1082,8 +1103,18 @@ static inline bool skb_unref(struct sk_buff *skb)
return true;
}
+void kfree_skb_reason(struct sk_buff *skb, enum skb_drop_reason reason);
+
+/**
+ * kfree_skb - free an sk_buff with 'NOT_SPECIFIED' reason
+ * @skb: buffer to free
+ */
+static inline void kfree_skb(struct sk_buff *skb)
+{
+ kfree_skb_reason(skb, SKB_DROP_REASON_NOT_SPECIFIED);
+}
+
void skb_release_head_state(struct sk_buff *skb);
-void kfree_skb(struct sk_buff *skb);
void kfree_skb_list(struct sk_buff *segs);
void skb_dump(const char *level, const struct sk_buff *skb, bool full_pkt);
void skb_tx_error(struct sk_buff *skb);
@@ -1976,8 +2007,8 @@ static inline void __skb_insert(struct sk_buff *newsk,
*/
WRITE_ONCE(newsk->next, next);
WRITE_ONCE(newsk->prev, prev);
- WRITE_ONCE(next->prev, newsk);
- WRITE_ONCE(prev->next, newsk);
+ WRITE_ONCE(((struct sk_buff_list *)next)->prev, newsk);
+ WRITE_ONCE(((struct sk_buff_list *)prev)->next, newsk);
WRITE_ONCE(list->qlen, list->qlen + 1);
}
@@ -2073,7 +2104,7 @@ static inline void __skb_queue_after(struct sk_buff_head *list,
struct sk_buff *prev,
struct sk_buff *newsk)
{
- __skb_insert(newsk, prev, prev->next, list);
+ __skb_insert(newsk, prev, ((struct sk_buff_list *)prev)->next, list);
}
void skb_append(struct sk_buff *old, struct sk_buff *newsk,
@@ -2083,7 +2114,7 @@ static inline void __skb_queue_before(struct sk_buff_head *list,
struct sk_buff *next,
struct sk_buff *newsk)
{
- __skb_insert(newsk, next->prev, next, list);
+ __skb_insert(newsk, ((struct sk_buff_list *)next)->prev, next, list);
}
/**
@@ -2373,6 +2404,8 @@ static inline void *skb_pull_inline(struct sk_buff *skb, unsigned int len)
return unlikely(len > skb->len) ? NULL : __skb_pull(skb, len);
}
+void *skb_pull_data(struct sk_buff *skb, size_t len);
+
void *__pskb_pull_tail(struct sk_buff *skb, int delta);
static inline void *__pskb_pull(struct sk_buff *skb, unsigned int len)
@@ -3485,7 +3518,12 @@ __skb_postpull_rcsum(struct sk_buff *skb, const void *start, unsigned int len,
static inline void skb_postpull_rcsum(struct sk_buff *skb,
const void *start, unsigned int len)
{
- __skb_postpull_rcsum(skb, start, len, 0);
+ if (skb->ip_summed == CHECKSUM_COMPLETE)
+ skb->csum = wsum_negate(csum_partial(start, len,
+ wsum_negate(skb->csum)));
+ else if (skb->ip_summed == CHECKSUM_PARTIAL &&
+ skb_checksum_start_offset(skb) < 0)
+ skb->ip_summed = CHECKSUM_NONE;
}
static __always_inline void
diff --git a/include/linux/skmsg.h b/include/linux/skmsg.h
index 584d94b..18a717f 100644
--- a/include/linux/skmsg.h
+++ b/include/linux/skmsg.h
@@ -507,12 +507,6 @@ static inline bool sk_psock_strp_enabled(struct sk_psock *psock)
return !!psock->saved_data_ready;
}
-static inline bool sk_is_tcp(const struct sock *sk)
-{
- return sk->sk_type == SOCK_STREAM &&
- sk->sk_protocol == IPPROTO_TCP;
-}
-
static inline bool sk_is_udp(const struct sock *sk)
{
return sk->sk_type == SOCK_DGRAM &&
diff --git a/include/linux/slab.h b/include/linux/slab.h
index 1810451..367366f 100644
--- a/include/linux/slab.h
+++ b/include/linux/slab.h
@@ -189,14 +189,6 @@ bool kmem_valid_obj(void *object);
void kmem_dump_obj(void *object);
#endif
-#ifdef CONFIG_HAVE_HARDENED_USERCOPY_ALLOCATOR
-void __check_heap_object(const void *ptr, unsigned long n, struct page *page,
- bool to_user);
-#else
-static inline void __check_heap_object(const void *ptr, unsigned long n,
- struct page *page, bool to_user) { }
-#endif
-
/*
* Some archs want to perform DMA into kmalloc caches and need a guaranteed
* alignment larger than the alignment of a 64-bit integer.
diff --git a/include/linux/slab_def.h b/include/linux/slab_def.h
index 3aa5e1e..e24c9af 100644
--- a/include/linux/slab_def.h
+++ b/include/linux/slab_def.h
@@ -87,11 +87,11 @@ struct kmem_cache {
struct kmem_cache_node *node[MAX_NUMNODES];
};
-static inline void *nearest_obj(struct kmem_cache *cache, struct page *page,
+static inline void *nearest_obj(struct kmem_cache *cache, const struct slab *slab,
void *x)
{
- void *object = x - (x - page->s_mem) % cache->size;
- void *last_object = page->s_mem + (cache->num - 1) * cache->size;
+ void *object = x - (x - slab->s_mem) % cache->size;
+ void *last_object = slab->s_mem + (cache->num - 1) * cache->size;
if (unlikely(object > last_object))
return last_object;
@@ -106,16 +106,16 @@ static inline void *nearest_obj(struct kmem_cache *cache, struct page *page,
* reciprocal_divide(offset, cache->reciprocal_buffer_size)
*/
static inline unsigned int obj_to_index(const struct kmem_cache *cache,
- const struct page *page, void *obj)
+ const struct slab *slab, void *obj)
{
- u32 offset = (obj - page->s_mem);
+ u32 offset = (obj - slab->s_mem);
return reciprocal_divide(offset, cache->reciprocal_buffer_size);
}
-static inline int objs_per_slab_page(const struct kmem_cache *cache,
- const struct page *page)
+static inline int objs_per_slab(const struct kmem_cache *cache,
+ const struct slab *slab)
{
- if (is_kfence_address(page_address(page)))
+ if (is_kfence_address(slab_address(slab)))
return 1;
return cache->num;
}
diff --git a/include/linux/slub_def.h b/include/linux/slub_def.h
index 0fa751b..33c5c0e 100644
--- a/include/linux/slub_def.h
+++ b/include/linux/slub_def.h
@@ -48,9 +48,9 @@ enum stat_item {
struct kmem_cache_cpu {
void **freelist; /* Pointer to next available object */
unsigned long tid; /* Globally unique transaction id */
- struct page *page; /* The slab from which we are allocating */
+ struct slab *slab; /* The slab from which we are allocating */
#ifdef CONFIG_SLUB_CPU_PARTIAL
- struct page *partial; /* Partially allocated frozen slabs */
+ struct slab *partial; /* Partially allocated frozen slabs */
#endif
local_lock_t lock; /* Protects the fields above */
#ifdef CONFIG_SLUB_STATS
@@ -99,8 +99,8 @@ struct kmem_cache {
#ifdef CONFIG_SLUB_CPU_PARTIAL
/* Number of per cpu partial objects to keep around */
unsigned int cpu_partial;
- /* Number of per cpu partial pages to keep around */
- unsigned int cpu_partial_pages;
+ /* Number of per cpu partial slabs to keep around */
+ unsigned int cpu_partial_slabs;
#endif
struct kmem_cache_order_objects oo;
@@ -156,16 +156,13 @@ static inline void sysfs_slab_release(struct kmem_cache *s)
}
#endif
-void object_err(struct kmem_cache *s, struct page *page,
- u8 *object, char *reason);
-
void *fixup_red_left(struct kmem_cache *s, void *p);
-static inline void *nearest_obj(struct kmem_cache *cache, struct page *page,
+static inline void *nearest_obj(struct kmem_cache *cache, const struct slab *slab,
void *x) {
- void *object = x - (x - page_address(page)) % cache->size;
- void *last_object = page_address(page) +
- (page->objects - 1) * cache->size;
+ void *object = x - (x - slab_address(slab)) % cache->size;
+ void *last_object = slab_address(slab) +
+ (slab->objects - 1) * cache->size;
void *result = (unlikely(object > last_object)) ? last_object : object;
result = fixup_red_left(cache, result);
@@ -181,16 +178,16 @@ static inline unsigned int __obj_to_index(const struct kmem_cache *cache,
}
static inline unsigned int obj_to_index(const struct kmem_cache *cache,
- const struct page *page, void *obj)
+ const struct slab *slab, void *obj)
{
if (is_kfence_address(obj))
return 0;
- return __obj_to_index(cache, page_address(page), obj);
+ return __obj_to_index(cache, slab_address(slab), obj);
}
-static inline int objs_per_slab_page(const struct kmem_cache *cache,
- const struct page *page)
+static inline int objs_per_slab(const struct kmem_cache *cache,
+ const struct slab *slab)
{
- return page->objects;
+ return slab->objects;
}
#endif /* _LINUX_SLUB_DEF_H */
diff --git a/include/linux/soc/qcom/llcc-qcom.h b/include/linux/soc/qcom/llcc-qcom.h
index 437c9df..9e8fd92 100644
--- a/include/linux/soc/qcom/llcc-qcom.h
+++ b/include/linux/soc/qcom/llcc-qcom.h
@@ -33,6 +33,9 @@
#define LLCC_MODPE 29
#define LLCC_APTCM 30
#define LLCC_WRCACHE 31
+#define LLCC_CVPFW 32
+#define LLCC_CPUSS1 33
+#define LLCC_CPUHWT 36
/**
* struct llcc_slice_desc - Cache slice descriptor
diff --git a/include/linux/soc/renesas/rcar-rst.h b/include/linux/soc/renesas/rcar-rst.h
index 7899a5b..1f1fe8b 100644
--- a/include/linux/soc/renesas/rcar-rst.h
+++ b/include/linux/soc/renesas/rcar-rst.h
@@ -4,8 +4,10 @@
#ifdef CONFIG_RST_RCAR
int rcar_rst_read_mode_pins(u32 *mode);
+int rcar_rst_set_rproc_boot_addr(u64 boot_addr);
#else
static inline int rcar_rst_read_mode_pins(u32 *mode) { return -ENODEV; }
+static inline int rcar_rst_set_rproc_boot_addr(u64 boot_addr) { return -ENODEV; }
#endif
#endif /* __LINUX_SOC_RENESAS_RCAR_RST_H__ */
diff --git a/include/linux/stacktrace.h b/include/linux/stacktrace.h
index bef1588..9745588 100644
--- a/include/linux/stacktrace.h
+++ b/include/linux/stacktrace.h
@@ -8,22 +8,6 @@
struct task_struct;
struct pt_regs;
-#ifdef CONFIG_STACKTRACE
-void stack_trace_print(const unsigned long *trace, unsigned int nr_entries,
- int spaces);
-int stack_trace_snprint(char *buf, size_t size, const unsigned long *entries,
- unsigned int nr_entries, int spaces);
-unsigned int stack_trace_save(unsigned long *store, unsigned int size,
- unsigned int skipnr);
-unsigned int stack_trace_save_tsk(struct task_struct *task,
- unsigned long *store, unsigned int size,
- unsigned int skipnr);
-unsigned int stack_trace_save_regs(struct pt_regs *regs, unsigned long *store,
- unsigned int size, unsigned int skipnr);
-unsigned int stack_trace_save_user(unsigned long *store, unsigned int size);
-unsigned int filter_irq_stacks(unsigned long *entries, unsigned int nr_entries);
-
-/* Internal interfaces. Do not use in generic code */
#ifdef CONFIG_ARCH_STACKWALK
/**
@@ -76,8 +60,25 @@ int arch_stack_walk_reliable(stack_trace_consume_fn consume_entry, void *cookie,
void arch_stack_walk_user(stack_trace_consume_fn consume_entry, void *cookie,
const struct pt_regs *regs);
+#endif /* CONFIG_ARCH_STACKWALK */
-#else /* CONFIG_ARCH_STACKWALK */
+#ifdef CONFIG_STACKTRACE
+void stack_trace_print(const unsigned long *trace, unsigned int nr_entries,
+ int spaces);
+int stack_trace_snprint(char *buf, size_t size, const unsigned long *entries,
+ unsigned int nr_entries, int spaces);
+unsigned int stack_trace_save(unsigned long *store, unsigned int size,
+ unsigned int skipnr);
+unsigned int stack_trace_save_tsk(struct task_struct *task,
+ unsigned long *store, unsigned int size,
+ unsigned int skipnr);
+unsigned int stack_trace_save_regs(struct pt_regs *regs, unsigned long *store,
+ unsigned int size, unsigned int skipnr);
+unsigned int stack_trace_save_user(unsigned long *store, unsigned int size);
+unsigned int filter_irq_stacks(unsigned long *entries, unsigned int nr_entries);
+
+#ifndef CONFIG_ARCH_STACKWALK
+/* Internal interfaces. Do not use in generic code */
struct stack_trace {
unsigned int nr_entries, max_entries;
unsigned long *entries;
diff --git a/include/linux/stmmac.h b/include/linux/stmmac.h
index a6f03b3..24eea1b 100644
--- a/include/linux/stmmac.h
+++ b/include/linux/stmmac.h
@@ -233,6 +233,7 @@ struct plat_stmmacenet_data {
int (*clks_config)(void *priv, bool enabled);
int (*crosststamp)(ktime_t *device, struct system_counterval_t *system,
void *ctx);
+ void (*dump_debug_regs)(void *priv);
void *bsp_priv;
struct clk *stmmac_clk;
struct clk *pclk;
@@ -241,6 +242,7 @@ struct plat_stmmacenet_data {
unsigned int clk_ref_rate;
unsigned int mult_fact_100ns;
s32 ptp_max_adj;
+ u32 cdc_error_adj;
struct reset_control *stmmac_rst;
struct reset_control *stmmac_ahb_rst;
struct stmmac_axi *axi;
diff --git a/include/linux/tcp.h b/include/linux/tcp.h
index 48d8a36..78b91bb 100644
--- a/include/linux/tcp.h
+++ b/include/linux/tcp.h
@@ -512,11 +512,13 @@ static inline u16 tcp_mss_clamp(const struct tcp_sock *tp, u16 mss)
int tcp_skb_shift(struct sk_buff *to, struct sk_buff *from, int pcount,
int shiftlen);
+void __tcp_sock_set_cork(struct sock *sk, bool on);
void tcp_sock_set_cork(struct sock *sk, bool on);
int tcp_sock_set_keepcnt(struct sock *sk, int val);
int tcp_sock_set_keepidle_locked(struct sock *sk, int val);
int tcp_sock_set_keepidle(struct sock *sk, int val);
int tcp_sock_set_keepintvl(struct sock *sk, int val);
+void __tcp_sock_set_nodelay(struct sock *sk, bool on);
void tcp_sock_set_nodelay(struct sock *sk);
void tcp_sock_set_quickack(struct sock *sk, int val);
int tcp_sock_set_syncnt(struct sock *sk, int val);
diff --git a/include/linux/tee_drv.h b/include/linux/tee_drv.h
index cf59996..5e1533e 100644
--- a/include/linux/tee_drv.h
+++ b/include/linux/tee_drv.h
@@ -587,4 +587,18 @@ struct tee_client_driver {
#define to_tee_client_driver(d) \
container_of(d, struct tee_client_driver, driver)
+/**
+ * teedev_open() - Open a struct tee_device
+ * @teedev: Device to open
+ *
+ * @return a pointer to struct tee_context on success or an ERR_PTR on failure.
+ */
+struct tee_context *teedev_open(struct tee_device *teedev);
+
+/**
+ * teedev_close_context() - closes a struct tee_context
+ * @ctx: The struct tee_context to close
+ */
+void teedev_close_context(struct tee_context *ctx);
+
#endif /*__TEE_DRV_H*/
diff --git a/include/linux/thread_info.h b/include/linux/thread_info.h
index ad0c4e0..73a6f34 100644
--- a/include/linux/thread_info.h
+++ b/include/linux/thread_info.h
@@ -118,6 +118,15 @@ static inline int test_ti_thread_flag(struct thread_info *ti, int flag)
return test_bit(flag, (unsigned long *)&ti->flags);
}
+/*
+ * This may be used in noinstr code, and needs to be __always_inline to prevent
+ * inadvertent instrumentation.
+ */
+static __always_inline unsigned long read_ti_thread_flags(struct thread_info *ti)
+{
+ return READ_ONCE(ti->flags);
+}
+
#define set_thread_flag(flag) \
set_ti_thread_flag(current_thread_info(), flag)
#define clear_thread_flag(flag) \
@@ -130,6 +139,11 @@ static inline int test_ti_thread_flag(struct thread_info *ti, int flag)
test_and_clear_ti_thread_flag(current_thread_info(), flag)
#define test_thread_flag(flag) \
test_ti_thread_flag(current_thread_info(), flag)
+#define read_thread_flags() \
+ read_ti_thread_flags(current_thread_info())
+
+#define read_task_thread_flags(t) \
+ read_ti_thread_flags(task_thread_info(t))
#ifdef CONFIG_GENERIC_ENTRY
#define set_syscall_work(fl) \
diff --git a/include/linux/u64_stats_sync.h b/include/linux/u64_stats_sync.h
index e8ec116..6ad4e90 100644
--- a/include/linux/u64_stats_sync.h
+++ b/include/linux/u64_stats_sync.h
@@ -66,7 +66,7 @@
#include <linux/seqlock.h>
struct u64_stats_sync {
-#if BITS_PER_LONG==32 && defined(CONFIG_SMP)
+#if BITS_PER_LONG == 32 && (defined(CONFIG_SMP) || defined(CONFIG_PREEMPT_RT))
seqcount_t seq;
#endif
};
@@ -125,7 +125,7 @@ static inline void u64_stats_inc(u64_stats_t *p)
}
#endif
-#if BITS_PER_LONG == 32 && defined(CONFIG_SMP)
+#if BITS_PER_LONG == 32 && (defined(CONFIG_SMP) || defined(CONFIG_PREEMPT_RT))
#define u64_stats_init(syncp) seqcount_init(&(syncp)->seq)
#else
static inline void u64_stats_init(struct u64_stats_sync *syncp)
@@ -135,15 +135,19 @@ static inline void u64_stats_init(struct u64_stats_sync *syncp)
static inline void u64_stats_update_begin(struct u64_stats_sync *syncp)
{
-#if BITS_PER_LONG==32 && defined(CONFIG_SMP)
+#if BITS_PER_LONG == 32 && (defined(CONFIG_SMP) || defined(CONFIG_PREEMPT_RT))
+ if (IS_ENABLED(CONFIG_PREEMPT_RT))
+ preempt_disable();
write_seqcount_begin(&syncp->seq);
#endif
}
static inline void u64_stats_update_end(struct u64_stats_sync *syncp)
{
-#if BITS_PER_LONG==32 && defined(CONFIG_SMP)
+#if BITS_PER_LONG == 32 && (defined(CONFIG_SMP) || defined(CONFIG_PREEMPT_RT))
write_seqcount_end(&syncp->seq);
+ if (IS_ENABLED(CONFIG_PREEMPT_RT))
+ preempt_enable();
#endif
}
@@ -152,8 +156,11 @@ u64_stats_update_begin_irqsave(struct u64_stats_sync *syncp)
{
unsigned long flags = 0;
-#if BITS_PER_LONG==32 && defined(CONFIG_SMP)
- local_irq_save(flags);
+#if BITS_PER_LONG == 32 && (defined(CONFIG_SMP) || defined(CONFIG_PREEMPT_RT))
+ if (IS_ENABLED(CONFIG_PREEMPT_RT))
+ preempt_disable();
+ else
+ local_irq_save(flags);
write_seqcount_begin(&syncp->seq);
#endif
return flags;
@@ -163,15 +170,18 @@ static inline void
u64_stats_update_end_irqrestore(struct u64_stats_sync *syncp,
unsigned long flags)
{
-#if BITS_PER_LONG==32 && defined(CONFIG_SMP)
+#if BITS_PER_LONG == 32 && (defined(CONFIG_SMP) || defined(CONFIG_PREEMPT_RT))
write_seqcount_end(&syncp->seq);
- local_irq_restore(flags);
+ if (IS_ENABLED(CONFIG_PREEMPT_RT))
+ preempt_enable();
+ else
+ local_irq_restore(flags);
#endif
}
static inline unsigned int __u64_stats_fetch_begin(const struct u64_stats_sync *syncp)
{
-#if BITS_PER_LONG==32 && defined(CONFIG_SMP)
+#if BITS_PER_LONG == 32 && (defined(CONFIG_SMP) || defined(CONFIG_PREEMPT_RT))
return read_seqcount_begin(&syncp->seq);
#else
return 0;
@@ -180,7 +190,7 @@ static inline unsigned int __u64_stats_fetch_begin(const struct u64_stats_sync *
static inline unsigned int u64_stats_fetch_begin(const struct u64_stats_sync *syncp)
{
-#if BITS_PER_LONG==32 && !defined(CONFIG_SMP)
+#if BITS_PER_LONG == 32 && (!defined(CONFIG_SMP) && !defined(CONFIG_PREEMPT_RT))
preempt_disable();
#endif
return __u64_stats_fetch_begin(syncp);
@@ -189,7 +199,7 @@ static inline unsigned int u64_stats_fetch_begin(const struct u64_stats_sync *sy
static inline bool __u64_stats_fetch_retry(const struct u64_stats_sync *syncp,
unsigned int start)
{
-#if BITS_PER_LONG==32 && defined(CONFIG_SMP)
+#if BITS_PER_LONG == 32 && (defined(CONFIG_SMP) || defined(CONFIG_PREEMPT_RT))
return read_seqcount_retry(&syncp->seq, start);
#else
return false;
@@ -199,7 +209,7 @@ static inline bool __u64_stats_fetch_retry(const struct u64_stats_sync *syncp,
static inline bool u64_stats_fetch_retry(const struct u64_stats_sync *syncp,
unsigned int start)
{
-#if BITS_PER_LONG==32 && !defined(CONFIG_SMP)
+#if BITS_PER_LONG == 32 && (!defined(CONFIG_SMP) && !defined(CONFIG_PREEMPT_RT))
preempt_enable();
#endif
return __u64_stats_fetch_retry(syncp, start);
@@ -213,7 +223,9 @@ static inline bool u64_stats_fetch_retry(const struct u64_stats_sync *syncp,
*/
static inline unsigned int u64_stats_fetch_begin_irq(const struct u64_stats_sync *syncp)
{
-#if BITS_PER_LONG==32 && !defined(CONFIG_SMP)
+#if BITS_PER_LONG == 32 && defined(CONFIG_PREEMPT_RT)
+ preempt_disable();
+#elif BITS_PER_LONG == 32 && !defined(CONFIG_SMP)
local_irq_disable();
#endif
return __u64_stats_fetch_begin(syncp);
@@ -222,7 +234,9 @@ static inline unsigned int u64_stats_fetch_begin_irq(const struct u64_stats_sync
static inline bool u64_stats_fetch_retry_irq(const struct u64_stats_sync *syncp,
unsigned int start)
{
-#if BITS_PER_LONG==32 && !defined(CONFIG_SMP)
+#if BITS_PER_LONG == 32 && defined(CONFIG_PREEMPT_RT)
+ preempt_enable();
+#elif BITS_PER_LONG == 32 && !defined(CONFIG_SMP)
local_irq_enable();
#endif
return __u64_stats_fetch_retry(syncp, start);
diff --git a/include/linux/wwan.h b/include/linux/wwan.h
index 9fac819..afb3334 100644
--- a/include/linux/wwan.h
+++ b/include/linux/wwan.h
@@ -4,12 +4,9 @@
#ifndef __WWAN_H
#define __WWAN_H
-#include <linux/device.h>
-#include <linux/kernel.h>
#include <linux/poll.h>
-#include <linux/skbuff.h>
-#include <linux/netlink.h>
#include <linux/netdevice.h>
+#include <linux/types.h>
/**
* enum wwan_port_type - WWAN port types
@@ -37,6 +34,10 @@ enum wwan_port_type {
WWAN_PORT_UNKNOWN,
};
+struct device;
+struct file;
+struct netlink_ext_ack;
+struct sk_buff;
struct wwan_port;
/** struct wwan_port_ops - The WWAN port operations
@@ -171,4 +172,13 @@ int wwan_register_ops(struct device *parent, const struct wwan_ops *ops,
void wwan_unregister_ops(struct device *parent);
+#ifdef CONFIG_WWAN_DEBUGFS
+struct dentry *wwan_get_debugfs_dir(struct device *parent);
+#else
+static inline struct dentry *wwan_get_debugfs_dir(struct device *parent)
+{
+ return ERR_PTR(-ENODEV);
+}
+#endif
+
#endif /* __WWAN_H */
diff --git a/include/media/cec.h b/include/media/cec.h
index 208c961..77346f7 100644
--- a/include/media/cec.h
+++ b/include/media/cec.h
@@ -26,13 +26,17 @@
* @dev: cec device
* @cdev: cec character device
* @minor: device node minor number
+ * @lock: lock to serialize open/release and registration
* @registered: the device was correctly registered
* @unregistered: the device was unregistered
+ * @lock_fhs: lock to control access to @fhs
* @fhs: the list of open filehandles (cec_fh)
- * @lock: lock to control access to this structure
*
* This structure represents a cec-related device node.
*
+ * To add or remove filehandles from @fhs the @lock must be taken first,
+ * followed by @lock_fhs. It is safe to access @fhs if either lock is held.
+ *
* The @parent is a physical device. It must be set by core or device drivers
* before registering the node.
*/
@@ -43,10 +47,13 @@ struct cec_devnode {
/* device info */
int minor;
+ /* serialize open/release and registration */
+ struct mutex lock;
bool registered;
bool unregistered;
+ /* protect access to fhs */
+ struct mutex lock_fhs;
struct list_head fhs;
- struct mutex lock;
};
struct cec_adapter;
diff --git a/include/media/dmxdev.h b/include/media/dmxdev.h
index baafa3b8..63219a6 100644
--- a/include/media/dmxdev.h
+++ b/include/media/dmxdev.h
@@ -21,7 +21,6 @@
#include <linux/types.h>
#include <linux/spinlock.h>
-#include <linux/kernel.h>
#include <linux/time.h>
#include <linux/timer.h>
#include <linux/wait.h>
diff --git a/include/media/dvb_frontend.h b/include/media/dvb_frontend.h
index 0d76fa4..e7c4487 100644
--- a/include/media/dvb_frontend.h
+++ b/include/media/dvb_frontend.h
@@ -364,6 +364,10 @@ struct dvb_frontend_internal_info {
* allocated by the driver.
* @init: callback function used to initialize the tuner device.
* @sleep: callback function used to put the tuner to sleep.
+ * @suspend: callback function used to inform that the Kernel will
+ * suspend.
+ * @resume: callback function used to inform that the Kernel is
+ * resuming from suspend.
* @write: callback function used by some demod legacy drivers to
* allow other drivers to write data into their registers.
* Should not be used on new drivers.
@@ -443,6 +447,8 @@ struct dvb_frontend_ops {
int (*init)(struct dvb_frontend* fe);
int (*sleep)(struct dvb_frontend* fe);
+ int (*suspend)(struct dvb_frontend *fe);
+ int (*resume)(struct dvb_frontend *fe);
int (*write)(struct dvb_frontend* fe, const u8 buf[], int len);
@@ -755,7 +761,8 @@ void dvb_frontend_detach(struct dvb_frontend *fe);
* &dvb_frontend_ops.tuner_ops.suspend\(\) is available, it calls it. Otherwise,
* it will call &dvb_frontend_ops.tuner_ops.sleep\(\), if available.
*
- * It will also call &dvb_frontend_ops.sleep\(\) to put the demod to suspend.
+ * It will also call &dvb_frontend_ops.suspend\(\) to put the demod to suspend,
+ * if available. Otherwise it will call &dvb_frontend_ops.sleep\(\).
*
* The drivers should also call dvb_frontend_suspend\(\) as part of their
* handler for the &device_driver.suspend\(\).
@@ -769,7 +776,9 @@ int dvb_frontend_suspend(struct dvb_frontend *fe);
*
* This function resumes the usual operation of the tuner after resume.
*
- * In order to resume the frontend, it calls the demod &dvb_frontend_ops.init\(\).
+ * In order to resume the frontend, it calls the demod
+ * &dvb_frontend_ops.resume\(\) if available. Otherwise it calls demod
+ * &dvb_frontend_ops.init\(\).
*
* If &dvb_frontend_ops.tuner_ops.resume\(\) is available, It, it calls it.
* Otherwise,t will call &dvb_frontend_ops.tuner_ops.init\(\), if available.
diff --git a/include/media/rc-core.h b/include/media/rc-core.h
index 8c5b797..ab9d3b7 100644
--- a/include/media/rc-core.h
+++ b/include/media/rc-core.h
@@ -59,7 +59,6 @@ enum rc_filter_type {
* @rc: rcdev for this lirc chardev
* @carrier_low: when setting the carrier range, first the low end must be
* set with an ioctl and then the high end with another ioctl
- * @send_timeout_reports: report timeouts in lirc raw IR.
* @rawir: queue for incoming raw IR
* @scancodes: queue for incoming decoded scancodes
* @wait_poll: poll struct for lirc device
@@ -72,7 +71,6 @@ struct lirc_fh {
struct list_head list;
struct rc_dev *rc;
int carrier_low;
- bool send_timeout_reports;
DECLARE_KFIFO_PTR(rawir, unsigned int);
DECLARE_KFIFO_PTR(scancodes, struct lirc_scancode);
wait_queue_head_t wait_poll;
diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
index 575b59f..b3ce438 100644
--- a/include/media/v4l2-ctrls.h
+++ b/include/media/v4l2-ctrls.h
@@ -50,6 +50,8 @@ struct video_device;
* @p_h264_decode_params: Pointer to a struct v4l2_ctrl_h264_decode_params.
* @p_h264_pred_weights: Pointer to a struct v4l2_ctrl_h264_pred_weights.
* @p_vp8_frame: Pointer to a VP8 frame params structure.
+ * @p_vp9_compressed_hdr_probs: Pointer to a VP9 frame compressed header probs structure.
+ * @p_vp9_frame: Pointer to a VP9 frame params structure.
* @p_hevc_sps: Pointer to an HEVC sequence parameter set structure.
* @p_hevc_pps: Pointer to an HEVC picture parameter set structure.
* @p_hevc_slice_params: Pointer to an HEVC slice parameters structure.
@@ -80,6 +82,8 @@ union v4l2_ctrl_ptr {
struct v4l2_ctrl_hevc_sps *p_hevc_sps;
struct v4l2_ctrl_hevc_pps *p_hevc_pps;
struct v4l2_ctrl_hevc_slice_params *p_hevc_slice_params;
+ struct v4l2_ctrl_vp9_compressed_hdr *p_vp9_compressed_hdr_probs;
+ struct v4l2_ctrl_vp9_frame *p_vp9_frame;
struct v4l2_ctrl_hdr10_cll_info *p_hdr10_cll;
struct v4l2_ctrl_hdr10_mastering_display *p_hdr10_mastering;
struct v4l2_area *p_area;
diff --git a/include/media/v4l2-mem2mem.h b/include/media/v4l2-mem2mem.h
index 5a91b54..fdbd525 100644
--- a/include/media/v4l2-mem2mem.h
+++ b/include/media/v4l2-mem2mem.h
@@ -495,6 +495,11 @@ __poll_t v4l2_m2m_poll(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
int v4l2_m2m_mmap(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
struct vm_area_struct *vma);
+#ifndef CONFIG_MMU
+unsigned long v4l2_m2m_get_unmapped_area(struct file *file, unsigned long addr,
+ unsigned long len, unsigned long pgoff,
+ unsigned long flags);
+#endif
/**
* v4l2_m2m_init() - initialize per-driver m2m data
*
diff --git a/include/media/v4l2-vp9.h b/include/media/v4l2-vp9.h
new file mode 100644
index 0000000..05478ad
--- /dev/null
+++ b/include/media/v4l2-vp9.h
@@ -0,0 +1,233 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Helper functions for vp9 codecs.
+ *
+ * Copyright (c) 2021 Collabora, Ltd.
+ *
+ * Author: Andrzej Pietrasiewicz <andrzej.p@collabora.com>
+ */
+
+#ifndef _MEDIA_V4L2_VP9_H
+#define _MEDIA_V4L2_VP9_H
+
+#include <media/v4l2-ctrls.h>
+
+/**
+ * struct v4l2_vp9_frame_mv_context - motion vector-related probabilities
+ *
+ * @joint: motion vector joint probabilities.
+ * @sign: motion vector sign probabilities.
+ * @classes: motion vector class probabilities.
+ * @class0_bit: motion vector class0 bit probabilities.
+ * @bits: motion vector bits probabilities.
+ * @class0_fr: motion vector class0 fractional bit probabilities.
+ * @fr: motion vector fractional bit probabilities.
+ * @class0_hp: motion vector class0 high precision fractional bit probabilities.
+ * @hp: motion vector high precision fractional bit probabilities.
+ *
+ * A member of v4l2_vp9_frame_context.
+ */
+struct v4l2_vp9_frame_mv_context {
+ u8 joint[3];
+ u8 sign[2];
+ u8 classes[2][10];
+ u8 class0_bit[2];
+ u8 bits[2][10];
+ u8 class0_fr[2][2][3];
+ u8 fr[2][3];
+ u8 class0_hp[2];
+ u8 hp[2];
+};
+
+/**
+ * struct v4l2_vp9_frame_context - frame probabilities, including motion-vector related
+ *
+ * @tx8: TX 8x8 probabilities.
+ * @tx16: TX 16x16 probabilities.
+ * @tx32: TX 32x32 probabilities.
+ * @coef: coefficient probabilities.
+ * @skip: skip probabilities.
+ * @inter_mode: inter mode probabilities.
+ * @interp_filter: interpolation filter probabilities.
+ * @is_inter: is inter-block probabilities.
+ * @comp_mode: compound prediction mode probabilities.
+ * @single_ref: single ref probabilities.
+ * @comp_ref: compound ref probabilities.
+ * @y_mode: Y prediction mode probabilities.
+ * @uv_mode: UV prediction mode probabilities.
+ * @partition: partition probabilities.
+ * @mv: motion vector probabilities.
+ *
+ * Drivers which need to keep track of frame context(s) can use this struct.
+ * The members correspond to probability tables, which are specified only implicitly in the
+ * vp9 spec. Section 10.5 "Default probability tables" contains all the types of involved
+ * tables, i.e. the actual tables are of the same kind, and when they are reset (which is
+ * mandated by the spec sometimes) they are overwritten with values from the default tables.
+ */
+struct v4l2_vp9_frame_context {
+ u8 tx8[2][1];
+ u8 tx16[2][2];
+ u8 tx32[2][3];
+ u8 coef[4][2][2][6][6][3];
+ u8 skip[3];
+ u8 inter_mode[7][3];
+ u8 interp_filter[4][2];
+ u8 is_inter[4];
+ u8 comp_mode[5];
+ u8 single_ref[5][2];
+ u8 comp_ref[5];
+ u8 y_mode[4][9];
+ u8 uv_mode[10][9];
+ u8 partition[16][3];
+
+ struct v4l2_vp9_frame_mv_context mv;
+};
+
+/**
+ * struct v4l2_vp9_frame_symbol_counts - pointers to arrays of symbol counts
+ *
+ * @partition: partition counts.
+ * @skip: skip counts.
+ * @intra_inter: is inter-block counts.
+ * @tx32p: TX32 counts.
+ * @tx16p: TX16 counts.
+ * @tx8p: TX8 counts.
+ * @y_mode: Y prediction mode counts.
+ * @uv_mode: UV prediction mode counts.
+ * @comp: compound prediction mode counts.
+ * @comp_ref: compound ref counts.
+ * @single_ref: single ref counts.
+ * @mv_mode: inter mode counts.
+ * @filter: interpolation filter counts.
+ * @mv_joint: motion vector joint counts.
+ * @sign: motion vector sign counts.
+ * @classes: motion vector class counts.
+ * @class0: motion vector class0 bit counts.
+ * @bits: motion vector bits counts.
+ * @class0_fp: motion vector class0 fractional bit counts.
+ * @fp: motion vector fractional bit counts.
+ * @class0_hp: motion vector class0 high precision fractional bit counts.
+ * @hp: motion vector high precision fractional bit counts.
+ * @coeff: coefficient counts.
+ * @eob: eob counts
+ *
+ * The fields correspond to what is specified in section 8.3 "Clear counts process" of the spec.
+ * Different pieces of hardware can report the counts in different order, so we cannot rely on
+ * simply overlaying a struct on a relevant block of memory. Instead we provide pointers to
+ * arrays or array of pointers to arrays in case of coeff, or array of pointers for eob.
+ */
+struct v4l2_vp9_frame_symbol_counts {
+ u32 (*partition)[16][4];
+ u32 (*skip)[3][2];
+ u32 (*intra_inter)[4][2];
+ u32 (*tx32p)[2][4];
+ u32 (*tx16p)[2][4];
+ u32 (*tx8p)[2][2];
+ u32 (*y_mode)[4][10];
+ u32 (*uv_mode)[10][10];
+ u32 (*comp)[5][2];
+ u32 (*comp_ref)[5][2];
+ u32 (*single_ref)[5][2][2];
+ u32 (*mv_mode)[7][4];
+ u32 (*filter)[4][3];
+ u32 (*mv_joint)[4];
+ u32 (*sign)[2][2];
+ u32 (*classes)[2][11];
+ u32 (*class0)[2][2];
+ u32 (*bits)[2][10][2];
+ u32 (*class0_fp)[2][2][4];
+ u32 (*fp)[2][4];
+ u32 (*class0_hp)[2][2];
+ u32 (*hp)[2][2];
+ u32 (*coeff[4][2][2][6][6])[3];
+ u32 *eob[4][2][2][6][6][2];
+};
+
+extern const u8 v4l2_vp9_kf_y_mode_prob[10][10][9]; /* Section 10.4 of the spec */
+extern const u8 v4l2_vp9_kf_partition_probs[16][3]; /* Section 10.4 of the spec */
+extern const u8 v4l2_vp9_kf_uv_mode_prob[10][9]; /* Section 10.4 of the spec */
+extern const struct v4l2_vp9_frame_context v4l2_vp9_default_probs; /* Section 10.5 of the spec */
+
+/**
+ * v4l2_vp9_fw_update_probs() - Perform forward update of vp9 probabilities
+ *
+ * @probs: current probabilities values
+ * @deltas: delta values from compressed header
+ * @dec_params: vp9 frame decoding parameters
+ *
+ * This function performs forward updates of probabilities for the vp9 boolean decoder.
+ * The frame header can contain a directive to update the probabilities (deltas), if so, then
+ * the deltas are provided in the header, too. The userspace parses those and passes the said
+ * deltas struct to the kernel.
+ */
+void v4l2_vp9_fw_update_probs(struct v4l2_vp9_frame_context *probs,
+ const struct v4l2_ctrl_vp9_compressed_hdr *deltas,
+ const struct v4l2_ctrl_vp9_frame *dec_params);
+
+/**
+ * v4l2_vp9_reset_frame_ctx() - Reset appropriate frame context
+ *
+ * @dec_params: vp9 frame decoding parameters
+ * @frame_context: array of the 4 frame contexts
+ *
+ * This function resets appropriate frame contexts, based on what's in dec_params.
+ *
+ * Returns the frame context index after the update, which might be reset to zero if
+ * mandated by the spec.
+ */
+u8 v4l2_vp9_reset_frame_ctx(const struct v4l2_ctrl_vp9_frame *dec_params,
+ struct v4l2_vp9_frame_context *frame_context);
+
+/**
+ * v4l2_vp9_adapt_coef_probs() - Perform backward update of vp9 coefficients probabilities
+ *
+ * @probs: current probabilities values
+ * @counts: values of symbol counts after the current frame has been decoded
+ * @use_128: flag to request that 128 is used as update factor if true, otherwise 112 is used
+ * @frame_is_intra: flag indicating that FrameIsIntra is true
+ *
+ * This function performs backward updates of coefficients probabilities for the vp9 boolean
+ * decoder. After a frame has been decoded the counts of how many times a given symbol has
+ * occurred are known and are used to update the probability of each symbol.
+ */
+void v4l2_vp9_adapt_coef_probs(struct v4l2_vp9_frame_context *probs,
+ struct v4l2_vp9_frame_symbol_counts *counts,
+ bool use_128,
+ bool frame_is_intra);
+
+/**
+ * v4l2_vp9_adapt_noncoef_probs() - Perform backward update of vp9 non-coefficients probabilities
+ *
+ * @probs: current probabilities values
+ * @counts: values of symbol counts after the current frame has been decoded
+ * @reference_mode: specifies the type of inter prediction to be used. See
+ * &v4l2_vp9_reference_mode for more details
+ * @interpolation_filter: specifies the filter selection used for performing inter prediction.
+ * See &v4l2_vp9_interpolation_filter for more details
+ * @tx_mode: specifies the TX mode. See &v4l2_vp9_tx_mode for more details
+ * @flags: combination of V4L2_VP9_FRAME_FLAG_* flags
+ *
+ * This function performs backward updates of non-coefficients probabilities for the vp9 boolean
+ * decoder. After a frame has been decoded the counts of how many times a given symbol has
+ * occurred are known and are used to update the probability of each symbol.
+ */
+void v4l2_vp9_adapt_noncoef_probs(struct v4l2_vp9_frame_context *probs,
+ struct v4l2_vp9_frame_symbol_counts *counts,
+ u8 reference_mode, u8 interpolation_filter, u8 tx_mode,
+ u32 flags);
+
+/**
+ * v4l2_vp9_seg_feat_enabled() - Check if a segmentation feature is enabled
+ *
+ * @feature_enabled: array of 8-bit flags (for all segments)
+ * @feature: id of the feature to check
+ * @segid: id of the segment to look up
+ *
+ * This function returns true if a given feature is active in a given segment.
+ */
+bool
+v4l2_vp9_seg_feat_enabled(const u8 *feature_enabled,
+ unsigned int feature,
+ unsigned int segid);
+
+#endif /* _MEDIA_V4L2_VP9_H */
diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index 2467284..5468b63 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -1155,8 +1155,15 @@ static inline void *vb2_get_drv_priv(struct vb2_queue *q)
static inline void vb2_set_plane_payload(struct vb2_buffer *vb,
unsigned int plane_no, unsigned long size)
{
- if (plane_no < vb->num_planes)
+ /*
+ * size must never be larger than the buffer length, so
+ * warn and clamp to the buffer length if that's the case.
+ */
+ if (plane_no < vb->num_planes) {
+ if (WARN_ON_ONCE(size > vb->planes[plane_no].length))
+ size = vb->planes[plane_no].length;
vb->planes[plane_no].bytesused = size;
+ }
}
/**
diff --git a/include/memory/renesas-rpc-if.h b/include/memory/renesas-rpc-if.h
index 77c694a1..7c93f51 100644
--- a/include/memory/renesas-rpc-if.h
+++ b/include/memory/renesas-rpc-if.h
@@ -57,6 +57,11 @@ struct rpcif_op {
} data;
};
+enum rpcif_type {
+ RPCIF_RCAR_GEN3,
+ RPCIF_RZ_G2L,
+};
+
struct rpcif {
struct device *dev;
void __iomem *base;
@@ -64,6 +69,7 @@ struct rpcif {
struct regmap *regmap;
struct reset_control *rstc;
size_t size;
+ enum rpcif_type type;
enum rpcif_data_dir dir;
u8 bus_size;
void *buffer;
@@ -78,7 +84,7 @@ struct rpcif {
};
int rpcif_sw_init(struct rpcif *rpc, struct device *dev);
-void rpcif_hw_init(struct rpcif *rpc, bool hyperflash);
+int rpcif_hw_init(struct rpcif *rpc, bool hyperflash);
void rpcif_prepare(struct rpcif *rpc, const struct rpcif_op *op, u64 *offs,
size_t *len);
int rpcif_manual_xfer(struct rpcif *rpc);
diff --git a/include/net/act_api.h b/include/net/act_api.h
index b5b624c..3049cb6 100644
--- a/include/net/act_api.h
+++ b/include/net/act_api.h
@@ -7,6 +7,7 @@
*/
#include <linux/refcount.h>
+#include <net/flow_offload.h>
#include <net/sch_generic.h>
#include <net/pkt_sched.h>
#include <net/net_namespace.h>
@@ -44,6 +45,7 @@ struct tc_action {
u8 hw_stats;
u8 used_hw_stats;
bool used_hw_stats_valid;
+ u32 in_hw_count;
};
#define tcf_index common.tcfa_index
#define tcf_refcnt common.tcfa_refcnt
@@ -88,6 +90,16 @@ static inline void tcf_tm_dump(struct tcf_t *dtm, const struct tcf_t *stm)
dtm->expires = jiffies_to_clock_t(stm->expires);
}
+static inline enum flow_action_hw_stats tc_act_hw_stats(u8 hw_stats)
+{
+ if (WARN_ON_ONCE(hw_stats > TCA_ACT_HW_STATS_ANY))
+ return FLOW_ACTION_HW_STATS_DONT_CARE;
+ else if (!hw_stats)
+ return FLOW_ACTION_HW_STATS_DISABLED;
+
+ return hw_stats;
+}
+
#ifdef CONFIG_NET_CLS_ACT
#define ACT_P_CREATED 1
@@ -121,6 +133,8 @@ struct tc_action_ops {
struct psample_group *
(*get_psample_group)(const struct tc_action *a,
tc_action_priv_destructor *destructor);
+ int (*offload_act_setup)(struct tc_action *act, void *entry_data,
+ u32 *index_inc, bool bind);
};
struct tc_action_net {
@@ -189,7 +203,7 @@ int tcf_action_exec(struct sk_buff *skb, struct tc_action **actions,
int tcf_action_init(struct net *net, struct tcf_proto *tp, struct nlattr *nla,
struct nlattr *est,
struct tc_action *actions[], int init_res[], size_t *attr_size,
- u32 flags, struct netlink_ext_ack *extack);
+ u32 flags, u32 fl_flags, struct netlink_ext_ack *extack);
struct tc_action_ops *tc_action_load_ops(struct nlattr *nla, bool police,
bool rtnl_held,
struct netlink_ext_ack *extack);
@@ -240,6 +254,9 @@ void tcf_action_update_stats(struct tc_action *a, u64 bytes, u64 packets,
u64 drops, bool hw);
int tcf_action_copy_stats(struct sk_buff *, struct tc_action *, int);
+int tcf_action_update_hw_stats(struct tc_action *action);
+int tcf_action_reoffload_cb(flow_indr_block_bind_cb_t *cb,
+ void *cb_priv, bool add);
int tcf_action_check_ctrlact(int action, struct tcf_proto *tp,
struct tcf_chain **handle,
struct netlink_ext_ack *newchain);
@@ -251,6 +268,14 @@ DECLARE_STATIC_KEY_FALSE(tcf_frag_xmit_count);
#endif
int tcf_dev_queue_xmit(struct sk_buff *skb, int (*xmit)(struct sk_buff *skb));
+
+#else /* !CONFIG_NET_CLS_ACT */
+
+static inline int tcf_action_reoffload_cb(flow_indr_block_bind_cb_t *cb,
+ void *cb_priv, bool add) {
+ return 0;
+}
+
#endif /* CONFIG_NET_CLS_ACT */
static inline void tcf_action_stats_update(struct tc_action *a, u64 bytes,
diff --git a/include/net/af_unix.h b/include/net/af_unix.h
index 7d142e8..a7ef624 100644
--- a/include/net/af_unix.h
+++ b/include/net/af_unix.h
@@ -20,13 +20,12 @@ struct sock *unix_peer_get(struct sock *sk);
#define UNIX_HASH_BITS 8
extern unsigned int unix_tot_inflight;
-extern spinlock_t unix_table_lock;
+extern spinlock_t unix_table_locks[2 * UNIX_HASH_SIZE];
extern struct hlist_head unix_socket_table[2 * UNIX_HASH_SIZE];
struct unix_address {
refcount_t refcnt;
int len;
- unsigned int hash;
struct sockaddr_un name[];
};
diff --git a/include/net/arp.h b/include/net/arp.h
index 4950191..031374a 100644
--- a/include/net/arp.h
+++ b/include/net/arp.h
@@ -53,13 +53,7 @@ static inline void __ipv4_confirm_neigh(struct net_device *dev, u32 key)
rcu_read_lock_bh();
n = __ipv4_neigh_lookup_noref(dev, key);
- if (n) {
- unsigned long now = jiffies;
-
- /* avoid dirtying neighbour */
- if (READ_ONCE(n->confirmed) != now)
- WRITE_ONCE(n->confirmed, now);
- }
+ neigh_confirm(n);
rcu_read_unlock_bh();
}
diff --git a/include/net/ax25.h b/include/net/ax25.h
index 03d409d..526e495 100644
--- a/include/net/ax25.h
+++ b/include/net/ax25.h
@@ -229,7 +229,10 @@ struct ctl_table;
typedef struct ax25_dev {
struct ax25_dev *next;
+
struct net_device *dev;
+ netdevice_tracker dev_tracker;
+
struct net_device *forward;
struct ctl_table_header *sysheader;
int values[AX25_MAX_VALUES];
diff --git a/include/net/bareudp.h b/include/net/bareudp.h
index dc65a0d..17610c8 100644
--- a/include/net/bareudp.h
+++ b/include/net/bareudp.h
@@ -3,21 +3,10 @@
#ifndef __NET_BAREUDP_H
#define __NET_BAREUDP_H
+#include <linux/netdevice.h>
#include <linux/types.h>
-#include <linux/skbuff.h>
#include <net/rtnetlink.h>
-struct bareudp_conf {
- __be16 ethertype;
- __be16 port;
- u16 sport_min;
- bool multi_proto_mode;
-};
-
-struct net_device *bareudp_dev_create(struct net *net, const char *name,
- u8 name_assign_type,
- struct bareudp_conf *info);
-
static inline bool netif_is_bareudp(const struct net_device *dev)
{
return dev->rtnl_link_ops &&
diff --git a/include/net/bluetooth/bluetooth.h b/include/net/bluetooth/bluetooth.h
index 3271870..4b3d0b1 100644
--- a/include/net/bluetooth/bluetooth.h
+++ b/include/net/bluetooth/bluetooth.h
@@ -380,6 +380,7 @@ typedef void (*hci_req_complete_skb_t)(struct hci_dev *hdev, u8 status,
#define HCI_REQ_SKB BIT(1)
struct hci_ctrl {
+ struct sock *sk;
u16 opcode;
u8 req_flags;
u8 req_event;
@@ -389,6 +390,11 @@ struct hci_ctrl {
};
};
+struct mgmt_ctrl {
+ struct hci_dev *hdev;
+ u16 opcode;
+};
+
struct bt_skb_cb {
u8 pkt_type;
u8 force_active;
@@ -398,6 +404,7 @@ struct bt_skb_cb {
struct l2cap_ctrl l2cap;
struct sco_ctrl sco;
struct hci_ctrl hci;
+ struct mgmt_ctrl mgmt;
};
};
#define bt_cb(skb) ((struct bt_skb_cb *)((skb)->cb))
@@ -405,6 +412,8 @@ struct bt_skb_cb {
#define hci_skb_pkt_type(skb) bt_cb((skb))->pkt_type
#define hci_skb_expect(skb) bt_cb((skb))->expect
#define hci_skb_opcode(skb) bt_cb((skb))->hci.opcode
+#define hci_skb_event(skb) bt_cb((skb))->hci.req_event
+#define hci_skb_sk(skb) bt_cb((skb))->hci.sk
static inline struct sk_buff *bt_skb_alloc(unsigned int len, gfp_t how)
{
diff --git a/include/net/bluetooth/hci.h b/include/net/bluetooth/hci.h
index 63065bc..35c073d 100644
--- a/include/net/bluetooth/hci.h
+++ b/include/net/bluetooth/hci.h
@@ -246,6 +246,15 @@ enum {
* HCI after resume.
*/
HCI_QUIRK_NO_SUSPEND_NOTIFIER,
+
+ /*
+ * When this quirk is set, LE tx power is not queried on startup
+ * and the min/max tx power values default to HCI_TX_POWER_INVALID.
+ *
+ * This quirk can be set before hci_register_dev is called or
+ * during the hdev->setup vendor callback.
+ */
+ HCI_QUIRK_BROKEN_READ_TRANSMIT_POWER,
};
/* HCI device flags */
@@ -332,6 +341,7 @@ enum {
HCI_FORCE_NO_MITM,
HCI_QUALITY_REPORT,
HCI_OFFLOAD_CODECS_ENABLED,
+ HCI_LE_SIMULTANEOUS_ROLES,
__HCI_NUM_FLAGS,
};
@@ -566,6 +576,7 @@ enum {
#define HCI_ERROR_INVALID_LL_PARAMS 0x1e
#define HCI_ERROR_UNSPECIFIED 0x1f
#define HCI_ERROR_ADVERTISING_TIMEOUT 0x3c
+#define HCI_ERROR_CANCELLED_BY_HOST 0x44
/* Flow control modes */
#define HCI_FLOW_CTL_MODE_PACKET_BASED 0x00
@@ -1046,8 +1057,8 @@ struct hci_cp_read_stored_link_key {
} __packed;
struct hci_rp_read_stored_link_key {
__u8 status;
- __u8 max_keys;
- __u8 num_keys;
+ __le16 max_keys;
+ __le16 num_keys;
} __packed;
#define HCI_OP_DELETE_STORED_LINK_KEY 0x0c12
@@ -1057,7 +1068,7 @@ struct hci_cp_delete_stored_link_key {
} __packed;
struct hci_rp_delete_stored_link_key {
__u8 status;
- __u8 num_keys;
+ __le16 num_keys;
} __packed;
#define HCI_MAX_NAME_LENGTH 248
@@ -1930,6 +1941,16 @@ struct hci_rp_le_read_transmit_power {
__s8 max_le_tx_power;
} __packed;
+#define HCI_NETWORK_PRIVACY 0x00
+#define HCI_DEVICE_PRIVACY 0x01
+
+#define HCI_OP_LE_SET_PRIVACY_MODE 0x204e
+struct hci_cp_le_set_privacy_mode {
+ __u8 bdaddr_type;
+ bdaddr_t bdaddr;
+ __u8 mode;
+} __packed;
+
#define HCI_OP_LE_READ_BUFFER_SIZE_V2 0x2060
struct hci_rp_le_read_buffer_size_v2 {
__u8 status;
@@ -2011,6 +2032,10 @@ struct hci_cp_le_reject_cis {
} __packed;
/* ---- HCI Events ---- */
+struct hci_ev_status {
+ __u8 status;
+} __packed;
+
#define HCI_EV_INQUIRY_COMPLETE 0x01
#define HCI_EV_INQUIRY_RESULT 0x02
@@ -2023,6 +2048,11 @@ struct inquiry_info {
__le16 clock_offset;
} __packed;
+struct hci_ev_inquiry_result {
+ __u8 num;
+ struct inquiry_info info[];
+};
+
#define HCI_EV_CONN_COMPLETE 0x03
struct hci_ev_conn_complete {
__u8 status;
@@ -2134,7 +2164,7 @@ struct hci_comp_pkts_info {
} __packed;
struct hci_ev_num_comp_pkts {
- __u8 num_hndl;
+ __u8 num;
struct hci_comp_pkts_info handles[];
} __packed;
@@ -2184,7 +2214,7 @@ struct hci_ev_pscan_rep_mode {
} __packed;
#define HCI_EV_INQUIRY_RESULT_WITH_RSSI 0x22
-struct inquiry_info_with_rssi {
+struct inquiry_info_rssi {
bdaddr_t bdaddr;
__u8 pscan_rep_mode;
__u8 pscan_period_mode;
@@ -2192,7 +2222,7 @@ struct inquiry_info_with_rssi {
__le16 clock_offset;
__s8 rssi;
} __packed;
-struct inquiry_info_with_rssi_and_pscan_mode {
+struct inquiry_info_rssi_pscan {
bdaddr_t bdaddr;
__u8 pscan_rep_mode;
__u8 pscan_period_mode;
@@ -2201,6 +2231,10 @@ struct inquiry_info_with_rssi_and_pscan_mode {
__le16 clock_offset;
__s8 rssi;
} __packed;
+struct hci_ev_inquiry_result_rssi {
+ __u8 num;
+ __u8 data[];
+} __packed;
#define HCI_EV_REMOTE_EXT_FEATURES 0x23
struct hci_ev_remote_ext_features {
@@ -2255,6 +2289,11 @@ struct extended_inquiry_info {
__u8 data[240];
} __packed;
+struct hci_ev_ext_inquiry_result {
+ __u8 num;
+ struct extended_inquiry_info info[];
+} __packed;
+
#define HCI_EV_KEY_REFRESH_COMPLETE 0x30
struct hci_ev_key_refresh_complete {
__u8 status;
@@ -2422,13 +2461,18 @@ struct hci_ev_le_conn_complete {
#define HCI_EV_LE_ADVERTISING_REPORT 0x02
struct hci_ev_le_advertising_info {
- __u8 evt_type;
+ __u8 type;
__u8 bdaddr_type;
bdaddr_t bdaddr;
__u8 length;
__u8 data[];
} __packed;
+struct hci_ev_le_advertising_report {
+ __u8 num;
+ struct hci_ev_le_advertising_info info[];
+} __packed;
+
#define HCI_EV_LE_CONN_UPDATE_COMPLETE 0x03
struct hci_ev_le_conn_update_complete {
__u8 status;
@@ -2472,7 +2516,7 @@ struct hci_ev_le_data_len_change {
#define HCI_EV_LE_DIRECT_ADV_REPORT 0x0B
struct hci_ev_le_direct_adv_info {
- __u8 evt_type;
+ __u8 type;
__u8 bdaddr_type;
bdaddr_t bdaddr;
__u8 direct_addr_type;
@@ -2480,6 +2524,11 @@ struct hci_ev_le_direct_adv_info {
__s8 rssi;
} __packed;
+struct hci_ev_le_direct_adv_report {
+ __u8 num;
+ struct hci_ev_le_direct_adv_info info[];
+} __packed;
+
#define HCI_EV_LE_PHY_UPDATE_COMPLETE 0x0c
struct hci_ev_le_phy_update_complete {
__u8 status;
@@ -2489,8 +2538,8 @@ struct hci_ev_le_phy_update_complete {
} __packed;
#define HCI_EV_LE_EXT_ADV_REPORT 0x0d
-struct hci_ev_le_ext_adv_report {
- __le16 evt_type;
+struct hci_ev_le_ext_adv_info {
+ __le16 type;
__u8 bdaddr_type;
bdaddr_t bdaddr;
__u8 primary_phy;
@@ -2498,11 +2547,16 @@ struct hci_ev_le_ext_adv_report {
__u8 sid;
__u8 tx_power;
__s8 rssi;
- __le16 interval;
- __u8 direct_addr_type;
+ __le16 interval;
+ __u8 direct_addr_type;
bdaddr_t direct_addr;
- __u8 length;
- __u8 data[];
+ __u8 length;
+ __u8 data[];
+} __packed;
+
+struct hci_ev_le_ext_adv_report {
+ __u8 num;
+ struct hci_ev_le_ext_adv_info info[];
} __packed;
#define HCI_EV_LE_ENHANCED_CONN_COMPLETE 0x0a
diff --git a/include/net/bluetooth/hci_core.h b/include/net/bluetooth/hci_core.h
index dd8840e..586f69d 100644
--- a/include/net/bluetooth/hci_core.h
+++ b/include/net/bluetooth/hci_core.h
@@ -30,6 +30,7 @@
#include <linux/rculist.h>
#include <net/bluetooth/hci.h>
+#include <net/bluetooth/hci_sync.h>
#include <net/bluetooth/hci_sock.h>
/* HCI priority */
@@ -87,6 +88,7 @@ struct discovery_state {
u8 (*uuids)[16];
unsigned long scan_start;
unsigned long scan_duration;
+ unsigned long name_resolve_timeout;
};
#define SUSPEND_NOTIFIER_TIMEOUT msecs_to_jiffies(2000) /* 2 seconds */
@@ -150,23 +152,23 @@ struct bdaddr_list_with_irk {
u8 local_irk[16];
};
+enum hci_conn_flags {
+ HCI_CONN_FLAG_REMOTE_WAKEUP,
+ HCI_CONN_FLAG_DEVICE_PRIVACY,
+
+ __HCI_CONN_NUM_FLAGS,
+};
+
+/* Make sure number of flags doesn't exceed sizeof(current_flags) */
+static_assert(__HCI_CONN_NUM_FLAGS < 32);
+
struct bdaddr_list_with_flags {
struct list_head list;
bdaddr_t bdaddr;
u8 bdaddr_type;
- u32 current_flags;
+ DECLARE_BITMAP(flags, __HCI_CONN_NUM_FLAGS);
};
-enum hci_conn_flags {
- HCI_CONN_FLAG_REMOTE_WAKEUP,
- HCI_CONN_FLAG_MAX
-};
-
-#define hci_conn_test_flag(nr, flags) ((flags) & (1U << nr))
-
-/* Make sure number of flags doesn't exceed sizeof(current_flags) */
-static_assert(HCI_CONN_FLAG_MAX < 32);
-
struct bt_uuid {
struct list_head list;
u8 uuid[16];
@@ -351,8 +353,8 @@ struct hci_dev {
__u16 lmp_subver;
__u16 voice_setting;
__u8 num_iac;
- __u8 stored_max_keys;
- __u8 stored_num_keys;
+ __u16 stored_max_keys;
+ __u16 stored_num_keys;
__u8 io_capability;
__s8 inq_tx_power;
__u8 err_data_reporting;
@@ -475,6 +477,10 @@ struct hci_dev {
struct work_struct power_on;
struct delayed_work power_off;
struct work_struct error_reset;
+ struct work_struct cmd_sync_work;
+ struct list_head cmd_sync_work_list;
+ struct mutex cmd_sync_work_lock;
+ struct work_struct cmd_sync_cancel_work;
__u16 discov_timeout;
struct delayed_work discov_off;
@@ -489,10 +495,7 @@ struct hci_dev {
struct work_struct tx_work;
struct work_struct discov_update;
- struct work_struct bg_scan_update;
struct work_struct scan_update;
- struct work_struct connectable_update;
- struct work_struct discoverable_update;
struct delayed_work le_scan_disable;
struct delayed_work le_scan_restart;
@@ -519,7 +522,6 @@ struct hci_dev {
bool advertising_paused;
struct notifier_block suspend_notifier;
- struct work_struct suspend_prepare;
enum suspended_state suspend_state_next;
enum suspended_state suspend_state;
bool scanning_paused;
@@ -528,9 +530,6 @@ struct hci_dev {
bdaddr_t wake_addr;
u8 wake_addr_type;
- wait_queue_head_t suspend_wait_q;
- DECLARE_BITMAP(suspend_tasks, __SUSPEND_NUM_TASKS);
-
struct hci_conn_hash conn_hash;
struct list_head mgmt_pending;
@@ -562,6 +561,7 @@ struct hci_dev {
struct rfkill *rfkill;
DECLARE_BITMAP(dev_flags, __HCI_NUM_FLAGS);
+ DECLARE_BITMAP(conn_flags, __HCI_CONN_NUM_FLAGS);
__s8 adv_tx_power;
__u8 adv_data[HCI_MAX_EXT_AD_LENGTH];
@@ -603,6 +603,7 @@ struct hci_dev {
#if IS_ENABLED(CONFIG_BT_AOSPEXT)
bool aosp_capable;
+ bool aosp_quality_report;
#endif
int (*open)(struct hci_dev *hdev);
@@ -756,7 +757,8 @@ struct hci_conn_params {
struct hci_conn *conn;
bool explicit_connect;
- u32 current_flags;
+ DECLARE_BITMAP(flags, __HCI_CONN_NUM_FLAGS);
+ u8 privacy_mode;
};
extern struct list_head hci_dev_list;
@@ -781,6 +783,12 @@ extern struct mutex hci_cb_list_lock;
hci_dev_clear_flag(hdev, HCI_QUALITY_REPORT); \
} while (0)
+#define hci_dev_le_state_simultaneous(hdev) \
+ (test_bit(HCI_QUIRK_VALID_LE_STATES, &hdev->quirks) && \
+ (hdev->le_states[4] & 0x08) && /* Central */ \
+ (hdev->le_states[4] & 0x40) && /* Peripheral */ \
+ (hdev->le_states[3] & 0x10)) /* Simultaneous */
+
/* ----- HCI interface to upper protocols ----- */
int l2cap_connect_ind(struct hci_dev *hdev, bdaddr_t *bdaddr);
int l2cap_disconn_ind(struct hci_conn *hcon);
@@ -1119,8 +1127,7 @@ struct hci_conn *hci_connect_le_scan(struct hci_dev *hdev, bdaddr_t *dst,
enum conn_reasons conn_reason);
struct hci_conn *hci_connect_le(struct hci_dev *hdev, bdaddr_t *dst,
u8 dst_type, bool dst_resolved, u8 sec_level,
- u16 conn_timeout, u8 role,
- bdaddr_t *direct_rpa);
+ u16 conn_timeout, u8 role);
struct hci_conn *hci_connect_acl(struct hci_dev *hdev, bdaddr_t *dst,
u8 sec_level, u8 auth_type,
enum conn_reasons conn_reason);
@@ -1461,8 +1468,14 @@ void hci_conn_del_sysfs(struct hci_conn *conn);
#define scan_coded(dev) (((dev)->le_tx_def_phys & HCI_LE_SET_PHY_CODED) || \
((dev)->le_rx_def_phys & HCI_LE_SET_PHY_CODED))
+#define ll_privacy_capable(dev) ((dev)->le_features[0] & HCI_LE_LL_PRIVACY)
+
/* Use LL Privacy based address resolution if supported */
-#define use_ll_privacy(dev) ((dev)->le_features[0] & HCI_LE_LL_PRIVACY)
+#define use_ll_privacy(dev) (ll_privacy_capable(dev) && \
+ hci_dev_test_flag(dev, HCI_ENABLE_LL_PRIVACY))
+
+#define privacy_mode_capable(dev) (use_ll_privacy(dev) && \
+ (hdev->commands[39] & 0x04))
/* Use enhanced synchronous connection if command is supported */
#define enhanced_sco_capable(dev) ((dev)->commands[29] & 0x08)
@@ -1690,10 +1703,6 @@ static inline int hci_check_conn_params(u16 min, u16 max, u16 latency,
int hci_register_cb(struct hci_cb *hcb);
int hci_unregister_cb(struct hci_cb *hcb);
-struct sk_buff *__hci_cmd_sync(struct hci_dev *hdev, u16 opcode, u32 plen,
- const void *param, u32 timeout);
-struct sk_buff *__hci_cmd_sync_ev(struct hci_dev *hdev, u16 opcode, u32 plen,
- const void *param, u8 event, u32 timeout);
int __hci_cmd_send(struct hci_dev *hdev, u16 opcode, u32 plen,
const void *param);
@@ -1704,9 +1713,6 @@ void hci_send_sco(struct hci_conn *conn, struct sk_buff *skb);
void *hci_sent_cmd_data(struct hci_dev *hdev, __u16 opcode);
-struct sk_buff *hci_cmd_sync(struct hci_dev *hdev, u16 opcode, u32 plen,
- const void *param, u32 timeout);
-
u32 hci_conn_get_phy(struct hci_conn *conn);
/* ----- HCI Sockets ----- */
@@ -1765,6 +1771,8 @@ void hci_mgmt_chan_unregister(struct hci_mgmt_chan *c);
#define DISCOV_LE_FAST_ADV_INT_MIN 0x00A0 /* 100 msec */
#define DISCOV_LE_FAST_ADV_INT_MAX 0x00F0 /* 150 msec */
+#define NAME_RESOLVE_DURATION msecs_to_jiffies(10240) /* 10.24 sec */
+
void mgmt_fill_version_info(void *ver);
int mgmt_new_settings(struct hci_dev *hdev);
void mgmt_index_added(struct hci_dev *hdev);
@@ -1806,7 +1814,6 @@ int mgmt_user_passkey_notify(struct hci_dev *hdev, bdaddr_t *bdaddr,
u8 entered);
void mgmt_auth_failed(struct hci_conn *conn, u8 status);
void mgmt_auth_enable_complete(struct hci_dev *hdev, u8 status);
-void mgmt_ssp_enable_complete(struct hci_dev *hdev, u8 enable, u8 status);
void mgmt_set_class_of_dev_complete(struct hci_dev *hdev, u8 *dev_class,
u8 status);
void mgmt_set_local_name_complete(struct hci_dev *hdev, u8 *name, u8 status);
@@ -1831,8 +1838,6 @@ void mgmt_new_conn_param(struct hci_dev *hdev, bdaddr_t *bdaddr,
u16 max_interval, u16 latency, u16 timeout);
void mgmt_smp_complete(struct hci_conn *conn, bool complete);
bool mgmt_get_connectable(struct hci_dev *hdev);
-void mgmt_set_connectable_complete(struct hci_dev *hdev, u8 status);
-void mgmt_set_discoverable_complete(struct hci_dev *hdev, u8 status);
u8 mgmt_get_adv_discov_flags(struct hci_dev *hdev);
void mgmt_advertising_added(struct sock *sk, struct hci_dev *hdev,
u8 instance);
diff --git a/include/net/bluetooth/hci_sync.h b/include/net/bluetooth/hci_sync.h
new file mode 100644
index 0000000..2492e3b
--- /dev/null
+++ b/include/net/bluetooth/hci_sync.h
@@ -0,0 +1,108 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * BlueZ - Bluetooth protocol stack for Linux
+ *
+ * Copyright (C) 2021 Intel Corporation
+ */
+
+typedef int (*hci_cmd_sync_work_func_t)(struct hci_dev *hdev, void *data);
+typedef void (*hci_cmd_sync_work_destroy_t)(struct hci_dev *hdev, void *data,
+ int err);
+
+struct hci_cmd_sync_work_entry {
+ struct list_head list;
+ hci_cmd_sync_work_func_t func;
+ void *data;
+ hci_cmd_sync_work_destroy_t destroy;
+};
+
+/* Function with sync suffix shall not be called with hdev->lock held as they
+ * wait the command to complete and in the meantime an event could be received
+ * which could attempt to acquire hdev->lock causing a deadlock.
+ */
+struct sk_buff *__hci_cmd_sync(struct hci_dev *hdev, u16 opcode, u32 plen,
+ const void *param, u32 timeout);
+struct sk_buff *hci_cmd_sync(struct hci_dev *hdev, u16 opcode, u32 plen,
+ const void *param, u32 timeout);
+struct sk_buff *__hci_cmd_sync_ev(struct hci_dev *hdev, u16 opcode, u32 plen,
+ const void *param, u8 event, u32 timeout);
+struct sk_buff *__hci_cmd_sync_sk(struct hci_dev *hdev, u16 opcode, u32 plen,
+ const void *param, u8 event, u32 timeout,
+ struct sock *sk);
+int __hci_cmd_sync_status(struct hci_dev *hdev, u16 opcode, u32 plen,
+ const void *param, u32 timeout);
+int __hci_cmd_sync_status_sk(struct hci_dev *hdev, u16 opcode, u32 plen,
+ const void *param, u8 event, u32 timeout,
+ struct sock *sk);
+
+void hci_cmd_sync_init(struct hci_dev *hdev);
+void hci_cmd_sync_clear(struct hci_dev *hdev);
+void hci_cmd_sync_cancel(struct hci_dev *hdev, int err);
+void __hci_cmd_sync_cancel(struct hci_dev *hdev, int err);
+
+int hci_cmd_sync_queue(struct hci_dev *hdev, hci_cmd_sync_work_func_t func,
+ void *data, hci_cmd_sync_work_destroy_t destroy);
+
+int hci_update_eir_sync(struct hci_dev *hdev);
+int hci_update_class_sync(struct hci_dev *hdev);
+
+int hci_update_eir_sync(struct hci_dev *hdev);
+int hci_update_class_sync(struct hci_dev *hdev);
+int hci_update_name_sync(struct hci_dev *hdev);
+int hci_write_ssp_mode_sync(struct hci_dev *hdev, u8 mode);
+
+int hci_update_random_address_sync(struct hci_dev *hdev, bool require_privacy,
+ bool rpa, u8 *own_addr_type);
+
+int hci_update_scan_rsp_data_sync(struct hci_dev *hdev, u8 instance);
+int hci_update_adv_data_sync(struct hci_dev *hdev, u8 instance);
+int hci_schedule_adv_instance_sync(struct hci_dev *hdev, u8 instance,
+ bool force);
+
+int hci_setup_ext_adv_instance_sync(struct hci_dev *hdev, u8 instance);
+int hci_start_ext_adv_sync(struct hci_dev *hdev, u8 instance);
+int hci_enable_ext_advertising_sync(struct hci_dev *hdev, u8 instance);
+int hci_enable_advertising_sync(struct hci_dev *hdev);
+int hci_enable_advertising(struct hci_dev *hdev);
+
+int hci_remove_advertising_sync(struct hci_dev *hdev, struct sock *sk,
+ u8 instance, bool force);
+int hci_disable_advertising_sync(struct hci_dev *hdev);
+
+int hci_update_passive_scan_sync(struct hci_dev *hdev);
+int hci_update_passive_scan(struct hci_dev *hdev);
+int hci_read_rssi_sync(struct hci_dev *hdev, __le16 handle);
+int hci_read_tx_power_sync(struct hci_dev *hdev, __le16 handle, u8 type);
+int hci_write_sc_support_sync(struct hci_dev *hdev, u8 val);
+int hci_read_clock_sync(struct hci_dev *hdev, struct hci_cp_read_clock *cp);
+
+int hci_write_fast_connectable_sync(struct hci_dev *hdev, bool enable);
+int hci_update_scan_sync(struct hci_dev *hdev);
+
+int hci_write_le_host_supported_sync(struct hci_dev *hdev, u8 le, u8 simul);
+int hci_remove_ext_adv_instance_sync(struct hci_dev *hdev, u8 instance,
+ struct sock *sk);
+struct sk_buff *hci_read_local_oob_data_sync(struct hci_dev *hdev, bool ext,
+ struct sock *sk);
+
+int hci_reset_sync(struct hci_dev *hdev);
+int hci_dev_open_sync(struct hci_dev *hdev);
+int hci_dev_close_sync(struct hci_dev *hdev);
+
+int hci_powered_update_sync(struct hci_dev *hdev);
+int hci_set_powered_sync(struct hci_dev *hdev, u8 val);
+
+int hci_update_discoverable_sync(struct hci_dev *hdev);
+int hci_update_discoverable(struct hci_dev *hdev);
+
+int hci_update_connectable_sync(struct hci_dev *hdev);
+
+int hci_start_discovery_sync(struct hci_dev *hdev);
+int hci_stop_discovery_sync(struct hci_dev *hdev);
+
+int hci_suspend_sync(struct hci_dev *hdev);
+int hci_resume_sync(struct hci_dev *hdev);
+
+struct hci_conn;
+
+int hci_le_create_conn_sync(struct hci_dev *hdev, struct hci_conn *conn);
diff --git a/include/net/bluetooth/mgmt.h b/include/net/bluetooth/mgmt.h
index 23a0524..107b25d 100644
--- a/include/net/bluetooth/mgmt.h
+++ b/include/net/bluetooth/mgmt.h
@@ -936,10 +936,11 @@ struct mgmt_ev_auth_failed {
__u8 status;
} __packed;
-#define MGMT_DEV_FOUND_CONFIRM_NAME 0x01
-#define MGMT_DEV_FOUND_LEGACY_PAIRING 0x02
-#define MGMT_DEV_FOUND_NOT_CONNECTABLE 0x04
-#define MGMT_DEV_FOUND_INITIATED_CONN 0x08
+#define MGMT_DEV_FOUND_CONFIRM_NAME 0x01
+#define MGMT_DEV_FOUND_LEGACY_PAIRING 0x02
+#define MGMT_DEV_FOUND_NOT_CONNECTABLE 0x04
+#define MGMT_DEV_FOUND_INITIATED_CONN 0x08
+#define MGMT_DEV_FOUND_NAME_REQUEST_FAILED 0x10
#define MGMT_EV_DEVICE_FOUND 0x0012
struct mgmt_ev_device_found {
diff --git a/include/net/bond_options.h b/include/net/bond_options.h
index e64833a..dd75c07 100644
--- a/include/net/bond_options.h
+++ b/include/net/bond_options.h
@@ -65,6 +65,7 @@ enum {
BOND_OPT_NUM_PEER_NOTIF_ALIAS,
BOND_OPT_PEER_NOTIF_DELAY,
BOND_OPT_LACP_ACTIVE,
+ BOND_OPT_MISSED_MAX,
BOND_OPT_LAST
};
diff --git a/include/net/bonding.h b/include/net/bonding.h
index 15e083e..f6ae3a4 100644
--- a/include/net/bonding.h
+++ b/include/net/bonding.h
@@ -121,6 +121,7 @@ struct bond_params {
int xmit_policy;
int miimon;
u8 num_peer_notif;
+ u8 missed_max;
int arp_interval;
int arp_validate;
int arp_all_targets;
diff --git a/include/net/cfg80211.h b/include/net/cfg80211.h
index 423f97b..d19e48f 100644
--- a/include/net/cfg80211.h
+++ b/include/net/cfg80211.h
@@ -1188,17 +1188,6 @@ struct cfg80211_unsol_bcast_probe_resp {
};
/**
- * enum cfg80211_ap_settings_flags - AP settings flags
- *
- * Used by cfg80211_ap_settings
- *
- * @AP_SETTINGS_EXTERNAL_AUTH_SUPPORT: AP supports external authentication
- */
-enum cfg80211_ap_settings_flags {
- AP_SETTINGS_EXTERNAL_AUTH_SUPPORT = BIT(0),
-};
-
-/**
* struct cfg80211_ap_settings - AP configuration
*
* Used to configure an AP interface.
@@ -4072,6 +4061,15 @@ struct mgmt_frame_regs {
* @set_fils_aad: Set FILS AAD data to the AP driver so that the driver can use
* those to decrypt (Re)Association Request and encrypt (Re)Association
* Response frame.
+ *
+ * @set_radar_background: Configure dedicated offchannel chain available for
+ * radar/CAC detection on some hw. This chain can't be used to transmit
+ * or receive frames and it is bounded to a running wdev.
+ * Background radar/CAC detection allows to avoid the CAC downtime
+ * switching to a different channel during CAC detection on the selected
+ * radar channel.
+ * The caller is expected to set chandef pointer to NULL in order to
+ * disable background CAC/radar detection.
*/
struct cfg80211_ops {
int (*suspend)(struct wiphy *wiphy, struct cfg80211_wowlan *wow);
@@ -4404,6 +4402,8 @@ struct cfg80211_ops {
struct cfg80211_color_change_settings *params);
int (*set_fils_aad)(struct wiphy *wiphy, struct net_device *dev,
struct cfg80211_fils_aad *fils_aad);
+ int (*set_radar_background)(struct wiphy *wiphy,
+ struct cfg80211_chan_def *chandef);
};
/*
@@ -6375,17 +6375,6 @@ static inline void cfg80211_gen_new_bssid(const u8 *bssid, u8 max_bssid,
}
/**
- * cfg80211_get_ies_channel_number - returns the channel number from ies
- * @ie: IEs
- * @ielen: length of IEs
- * @band: enum nl80211_band of the channel
- *
- * Returns the channel number, or -1 if none could be determined.
- */
-int cfg80211_get_ies_channel_number(const u8 *ie, size_t ielen,
- enum nl80211_band band);
-
-/**
* cfg80211_is_element_inherited - returns if element ID should be inherited
* @element: element to check
* @non_inherit_element: non inheritance element
@@ -6421,6 +6410,19 @@ enum cfg80211_bss_frame_type {
};
/**
+ * cfg80211_get_ies_channel_number - returns the channel number from ies
+ * @ie: IEs
+ * @ielen: length of IEs
+ * @band: enum nl80211_band of the channel
+ * @ftype: frame type
+ *
+ * Returns the channel number, or -1 if none could be determined.
+ */
+int cfg80211_get_ies_channel_number(const u8 *ie, size_t ielen,
+ enum nl80211_band band,
+ enum cfg80211_bss_frame_type ftype);
+
+/**
* cfg80211_inform_bss_data - inform cfg80211 of a new BSS
*
* @wiphy: the wiphy reporting the BSS
@@ -7594,15 +7596,33 @@ void cfg80211_cqm_txe_notify(struct net_device *dev, const u8 *peer,
void cfg80211_cqm_beacon_loss_notify(struct net_device *dev, gfp_t gfp);
/**
- * cfg80211_radar_event - radar detection event
+ * __cfg80211_radar_event - radar detection event
* @wiphy: the wiphy
* @chandef: chandef for the current channel
+ * @offchan: the radar has been detected on the offchannel chain
* @gfp: context flags
*
* This function is called when a radar is detected on the current chanenl.
*/
-void cfg80211_radar_event(struct wiphy *wiphy,
- struct cfg80211_chan_def *chandef, gfp_t gfp);
+void __cfg80211_radar_event(struct wiphy *wiphy,
+ struct cfg80211_chan_def *chandef,
+ bool offchan, gfp_t gfp);
+
+static inline void
+cfg80211_radar_event(struct wiphy *wiphy,
+ struct cfg80211_chan_def *chandef,
+ gfp_t gfp)
+{
+ __cfg80211_radar_event(wiphy, chandef, false, gfp);
+}
+
+static inline void
+cfg80211_background_radar_event(struct wiphy *wiphy,
+ struct cfg80211_chan_def *chandef,
+ gfp_t gfp)
+{
+ __cfg80211_radar_event(wiphy, chandef, true, gfp);
+}
/**
* cfg80211_sta_opmode_change_notify - STA's ht/vht operation mode change event
@@ -7633,6 +7653,14 @@ void cfg80211_cac_event(struct net_device *netdev,
const struct cfg80211_chan_def *chandef,
enum nl80211_radar_event event, gfp_t gfp);
+/**
+ * cfg80211_background_cac_abort - Channel Availability Check offchan abort event
+ * @wiphy: the wiphy
+ *
+ * This function is called by the driver when a Channel Availability Check
+ * (CAC) is aborted by a offchannel dedicated chain.
+ */
+void cfg80211_background_cac_abort(struct wiphy *wiphy);
/**
* cfg80211_gtk_rekey_notify - notify userspace about driver rekeying
@@ -8250,6 +8278,18 @@ bool cfg80211_iftype_allowed(struct wiphy *wiphy, enum nl80211_iftype iftype,
bool is_4addr, u8 check_swif);
+/**
+ * cfg80211_assoc_comeback - notification of association that was
+ * temporarly rejected with a comeback
+ * @netdev: network device
+ * @bss: the bss entry with which association is in progress.
+ * @timeout: timeout interval value TUs.
+ *
+ * this function may sleep. the caller must hold the corresponding wdev's mutex.
+ */
+void cfg80211_assoc_comeback(struct net_device *netdev,
+ struct cfg80211_bss *bss, u32 timeout);
+
/* Logging, debugging and troubleshooting/diagnostic helpers. */
/* wiphy_printk helpers, similar to dev_printk */
diff --git a/include/net/checksum.h b/include/net/checksum.h
index 5b96d5b..5218041 100644
--- a/include/net/checksum.h
+++ b/include/net/checksum.h
@@ -180,4 +180,8 @@ static inline void remcsum_unadjust(__sum16 *psum, __wsum delta)
*psum = csum_fold(csum_sub(delta, (__force __wsum)*psum));
}
+static inline __wsum wsum_negate(__wsum val)
+{
+ return (__force __wsum)-((__force u32)val);
+}
#endif
diff --git a/include/net/codel.h b/include/net/codel.h
index a6c9e34..5fed2f1 100644
--- a/include/net/codel.h
+++ b/include/net/codel.h
@@ -44,8 +44,6 @@
#include <linux/types.h>
#include <linux/ktime.h>
#include <linux/skbuff.h>
-#include <net/pkt_sched.h>
-#include <net/inet_ecn.h>
/* Controlling Queue Delay (CoDel) algorithm
* =========================================
diff --git a/include/net/codel_impl.h b/include/net/codel_impl.h
index 137d40d..78a27ac 100644
--- a/include/net/codel_impl.h
+++ b/include/net/codel_impl.h
@@ -49,6 +49,8 @@
* Implemented on linux by Dave Taht and Eric Dumazet
*/
+#include <net/inet_ecn.h>
+
static void codel_params_init(struct codel_params *params)
{
params->interval = MS2TIME(100);
diff --git a/include/net/codel_qdisc.h b/include/net/codel_qdisc.h
index 098630f..58b6d0e 100644
--- a/include/net/codel_qdisc.h
+++ b/include/net/codel_qdisc.h
@@ -49,6 +49,8 @@
* Implemented on linux by Dave Taht and Eric Dumazet
*/
+#include <net/pkt_sched.h>
+
/* Qdiscs using codel plugin must use codel_skb_cb in their own cb[] */
struct codel_skb_cb {
codel_time_t enqueue_time;
diff --git a/include/net/devlink.h b/include/net/devlink.h
index aab3d00..8d5349d 100644
--- a/include/net/devlink.h
+++ b/include/net/devlink.h
@@ -361,33 +361,6 @@ devlink_resource_size_params_init(struct devlink_resource_size_params *size_para
typedef u64 devlink_resource_occ_get_t(void *priv);
-/**
- * struct devlink_resource - devlink resource
- * @name: name of the resource
- * @id: id, per devlink instance
- * @size: size of the resource
- * @size_new: updated size of the resource, reload is needed
- * @size_valid: valid in case the total size of the resource is valid
- * including its children
- * @parent: parent resource
- * @size_params: size parameters
- * @list: parent list
- * @resource_list: list of child resources
- */
-struct devlink_resource {
- const char *name;
- u64 id;
- u64 size;
- u64 size_new;
- bool size_valid;
- struct devlink_resource *parent;
- struct devlink_resource_size_params size_params;
- struct list_head list;
- struct list_head resource_list;
- devlink_resource_occ_get_t *occ_get;
- void *occ_get_priv;
-};
-
#define DEVLINK_RESOURCE_ID_PARENT_TOP 0
#define DEVLINK_RESOURCE_GENERIC_NAME_PORTS "physical_ports"
@@ -485,6 +458,9 @@ enum devlink_param_generic_id {
DEVLINK_PARAM_GENERIC_ID_ENABLE_ETH,
DEVLINK_PARAM_GENERIC_ID_ENABLE_RDMA,
DEVLINK_PARAM_GENERIC_ID_ENABLE_VNET,
+ DEVLINK_PARAM_GENERIC_ID_ENABLE_IWARP,
+ DEVLINK_PARAM_GENERIC_ID_IO_EQ_SIZE,
+ DEVLINK_PARAM_GENERIC_ID_EVENT_EQ_SIZE,
/* add new param generic ids above here*/
__DEVLINK_PARAM_GENERIC_ID_MAX,
@@ -534,6 +510,15 @@ enum devlink_param_generic_id {
#define DEVLINK_PARAM_GENERIC_ENABLE_VNET_NAME "enable_vnet"
#define DEVLINK_PARAM_GENERIC_ENABLE_VNET_TYPE DEVLINK_PARAM_TYPE_BOOL
+#define DEVLINK_PARAM_GENERIC_ENABLE_IWARP_NAME "enable_iwarp"
+#define DEVLINK_PARAM_GENERIC_ENABLE_IWARP_TYPE DEVLINK_PARAM_TYPE_BOOL
+
+#define DEVLINK_PARAM_GENERIC_IO_EQ_SIZE_NAME "io_eq_size"
+#define DEVLINK_PARAM_GENERIC_IO_EQ_SIZE_TYPE DEVLINK_PARAM_TYPE_U32
+
+#define DEVLINK_PARAM_GENERIC_EVENT_EQ_SIZE_NAME "event_eq_size"
+#define DEVLINK_PARAM_GENERIC_EVENT_EQ_SIZE_TYPE DEVLINK_PARAM_TYPE_U32
+
#define DEVLINK_PARAM_GENERIC(_id, _cmodes, _get, _set, _validate) \
{ \
.id = DEVLINK_PARAM_GENERIC_ID_##_id, \
@@ -687,13 +672,17 @@ struct devlink_health_reporter_ops {
* @trap_name: Trap name.
* @trap_group_name: Trap group name.
* @input_dev: Input netdevice.
+ * @dev_tracker: refcount tracker for @input_dev.
* @fa_cookie: Flow action user cookie.
* @trap_type: Trap type.
*/
struct devlink_trap_metadata {
const char *trap_name;
const char *trap_group_name;
+
struct net_device *input_dev;
+ netdevice_tracker dev_tracker;
+
const struct flow_action_cookie *fa_cookie;
enum devlink_trap_type trap_type;
};
@@ -1567,8 +1556,7 @@ int devlink_resource_register(struct devlink *devlink,
u64 resource_id,
u64 parent_resource_id,
const struct devlink_resource_size_params *size_params);
-void devlink_resources_unregister(struct devlink *devlink,
- struct devlink_resource *resource);
+void devlink_resources_unregister(struct devlink *devlink);
int devlink_resource_size_get(struct devlink *devlink,
u64 resource_id,
u64 *p_resource_size);
diff --git a/include/net/dsa.h b/include/net/dsa.h
index eff5c44..57b3e4e 100644
--- a/include/net/dsa.h
+++ b/include/net/dsa.h
@@ -88,6 +88,8 @@ struct dsa_device_ops {
struct sk_buff *(*rcv)(struct sk_buff *skb, struct net_device *dev);
void (*flow_dissect)(const struct sk_buff *skb, __be16 *proto,
int *offset);
+ int (*connect)(struct dsa_switch *ds);
+ void (*disconnect)(struct dsa_switch *ds);
unsigned int needed_headroom;
unsigned int needed_tailroom;
const char *name;
@@ -117,6 +119,9 @@ struct dsa_netdevice_ops {
struct dsa_switch_tree {
struct list_head list;
+ /* List of switch ports */
+ struct list_head ports;
+
/* Notifier chain for switch-wide events */
struct raw_notifier_head nh;
@@ -126,8 +131,10 @@ struct dsa_switch_tree {
/* Number of switches attached to this tree */
struct kref refcount;
- /* Has this tree been applied to the hardware? */
- bool setup;
+ /* Maps offloaded LAG netdevs to a zero-based linear ID for
+ * drivers that need it.
+ */
+ struct net_device **lags;
/* Tagging protocol operations */
const struct dsa_device_ops *tag_ops;
@@ -137,22 +144,19 @@ struct dsa_switch_tree {
*/
enum dsa_tag_protocol default_proto;
+ /* Has this tree been applied to the hardware? */
+ bool setup;
+
/*
* Configuration data for the platform device that owns
* this dsa switch tree instance.
*/
struct dsa_platform_data *pd;
- /* List of switch ports */
- struct list_head ports;
-
/* List of DSA links composing the routing table */
struct list_head rtable;
- /* Maps offloaded LAG netdevs to a zero-based linear ID for
- * drivers that need it.
- */
- struct net_device **lags;
+ /* Length of "lags" array */
unsigned int lags_len;
/* Track the largest switch index within a tree */
@@ -219,6 +223,12 @@ struct dsa_mall_tc_entry {
};
};
+struct dsa_bridge {
+ struct net_device *dev;
+ unsigned int num;
+ bool tx_fwd_offload;
+ refcount_t refcount;
+};
struct dsa_port {
/* A CPU port is physically connected to a master device.
@@ -238,6 +248,10 @@ struct dsa_port {
struct dsa_switch_tree *dst;
struct sk_buff *(*rcv)(struct sk_buff *skb, struct net_device *dev);
+ struct dsa_switch *ds;
+
+ unsigned int index;
+
enum {
DSA_PORT_TYPE_UNUSED = 0,
DSA_PORT_TYPE_CPU,
@@ -245,36 +259,40 @@ struct dsa_port {
DSA_PORT_TYPE_USER,
} type;
- struct dsa_switch *ds;
- unsigned int index;
const char *name;
struct dsa_port *cpu_dp;
u8 mac[ETH_ALEN];
+
+ u8 stp_state;
+
+ /* Warning: the following bit fields are not atomic, and updating them
+ * can only be done from code paths where concurrency is not possible
+ * (probe time or under rtnl_lock).
+ */
+ u8 vlan_filtering:1;
+
+ /* Managed by DSA on user ports and by drivers on CPU and DSA ports */
+ u8 learning:1;
+
+ u8 lag_tx_enabled:1;
+
+ u8 devlink_port_setup:1;
+
+ u8 setup:1;
+
struct device_node *dn;
unsigned int ageing_time;
- bool vlan_filtering;
- /* Managed by DSA on user ports and by drivers on CPU and DSA ports */
- bool learning;
- u8 stp_state;
- struct net_device *bridge_dev;
- int bridge_num;
+
+ struct dsa_bridge *bridge;
struct devlink_port devlink_port;
- bool devlink_port_setup;
struct phylink *pl;
struct phylink_config pl_config;
struct net_device *lag_dev;
- bool lag_tx_enabled;
struct net_device *hsr_dev;
struct list_head list;
/*
- * Give the switch driver somewhere to hang its per-port private data
- * structures (accessible from the tagger).
- */
- void *priv;
-
- /*
* Original copy of the master netdev ethtool_ops
*/
const struct ethtool_ops *orig_ethtool_ops;
@@ -290,8 +308,6 @@ struct dsa_port {
struct mutex addr_lists_lock;
struct list_head fdbs;
struct list_head mdbs;
-
- bool setup;
};
/* TODO: ideally DSA ports would have a single dp->link_dp member,
@@ -313,8 +329,6 @@ struct dsa_mac_addr {
};
struct dsa_switch {
- bool setup;
-
struct device *dev;
/*
@@ -323,6 +337,57 @@ struct dsa_switch {
struct dsa_switch_tree *dst;
unsigned int index;
+ /* Warning: the following bit fields are not atomic, and updating them
+ * can only be done from code paths where concurrency is not possible
+ * (probe time or under rtnl_lock).
+ */
+ u32 setup:1;
+
+ /* Disallow bridge core from requesting different VLAN awareness
+ * settings on ports if not hardware-supported
+ */
+ u32 vlan_filtering_is_global:1;
+
+ /* Keep VLAN filtering enabled on ports not offloading any upper */
+ u32 needs_standalone_vlan_filtering:1;
+
+ /* Pass .port_vlan_add and .port_vlan_del to drivers even for bridges
+ * that have vlan_filtering=0. All drivers should ideally set this (and
+ * then the option would get removed), but it is unknown whether this
+ * would break things or not.
+ */
+ u32 configure_vlan_while_not_filtering:1;
+
+ /* If the switch driver always programs the CPU port as egress tagged
+ * despite the VLAN configuration indicating otherwise, then setting
+ * @untag_bridge_pvid will force the DSA receive path to pop the
+ * bridge's default_pvid VLAN tagged frames to offer a consistent
+ * behavior between a vlan_filtering=0 and vlan_filtering=1 bridge
+ * device.
+ */
+ u32 untag_bridge_pvid:1;
+
+ /* Let DSA manage the FDB entries towards the
+ * CPU, based on the software bridge database.
+ */
+ u32 assisted_learning_on_cpu_port:1;
+
+ /* In case vlan_filtering_is_global is set, the VLAN awareness state
+ * should be retrieved from here and not from the per-port settings.
+ */
+ u32 vlan_filtering:1;
+
+ /* MAC PCS does not provide link state change interrupt, and requires
+ * polling. Flag passed on to PHYLINK.
+ */
+ u32 pcs_poll:1;
+
+ /* For switches that only have the MRU configurable. To ensure the
+ * configured MTU is not exceeded, normalization of MRU on all bridged
+ * interfaces is needed.
+ */
+ u32 mtu_enforcement_ingress:1;
+
/* Listener for switch fabric events */
struct notifier_block nb;
@@ -332,6 +397,8 @@ struct dsa_switch {
*/
void *priv;
+ void *tagger_data;
+
/*
* Configuration data for this switch.
*/
@@ -361,50 +428,6 @@ struct dsa_switch {
/* Number of switch port queues */
unsigned int num_tx_queues;
- /* Disallow bridge core from requesting different VLAN awareness
- * settings on ports if not hardware-supported
- */
- bool vlan_filtering_is_global;
-
- /* Keep VLAN filtering enabled on ports not offloading any upper. */
- bool needs_standalone_vlan_filtering;
-
- /* Pass .port_vlan_add and .port_vlan_del to drivers even for bridges
- * that have vlan_filtering=0. All drivers should ideally set this (and
- * then the option would get removed), but it is unknown whether this
- * would break things or not.
- */
- bool configure_vlan_while_not_filtering;
-
- /* If the switch driver always programs the CPU port as egress tagged
- * despite the VLAN configuration indicating otherwise, then setting
- * @untag_bridge_pvid will force the DSA receive path to pop the bridge's
- * default_pvid VLAN tagged frames to offer a consistent behavior
- * between a vlan_filtering=0 and vlan_filtering=1 bridge device.
- */
- bool untag_bridge_pvid;
-
- /* Let DSA manage the FDB entries towards the CPU, based on the
- * software bridge database.
- */
- bool assisted_learning_on_cpu_port;
-
- /* In case vlan_filtering_is_global is set, the VLAN awareness state
- * should be retrieved from here and not from the per-port settings.
- */
- bool vlan_filtering;
-
- /* MAC PCS does not provide link state change interrupt, and requires
- * polling. Flag passed on to PHYLINK.
- */
- bool pcs_poll;
-
- /* For switches that only have the MRU configurable. To ensure the
- * configured MTU is not exceeded, normalization of MRU on all bridged
- * interfaces is needed.
- */
- bool mtu_enforcement_ingress;
-
/* Drivers that benefit from having an ID associated with each
* offloaded LAG should set this to the maximum number of
* supported IDs. DSA will then maintain a mapping of _at
@@ -413,14 +436,14 @@ struct dsa_switch {
*/
unsigned int num_lag_ids;
- /* Drivers that support bridge forwarding offload should set this to
- * the maximum number of bridges spanning the same switch tree (or all
- * trees, in the case of cross-tree bridging support) that can be
- * offloaded.
+ /* Drivers that support bridge forwarding offload or FDB isolation
+ * should set this to the maximum number of bridges spanning the same
+ * switch tree (or all trees, in the case of cross-tree bridging
+ * support) that can be offloaded.
*/
- unsigned int num_fwd_offloading_bridges;
+ unsigned int max_num_bridges;
- size_t num_ports;
+ unsigned int num_ports;
};
static inline struct dsa_port *dsa_to_port(struct dsa_switch *ds, int p)
@@ -588,7 +611,7 @@ static inline bool dsa_port_is_vlan_filtering(const struct dsa_port *dp)
static inline
struct net_device *dsa_port_to_bridge_port(const struct dsa_port *dp)
{
- if (!dp->bridge_dev)
+ if (!dp->bridge)
return NULL;
if (dp->lag_dev)
@@ -599,6 +622,76 @@ struct net_device *dsa_port_to_bridge_port(const struct dsa_port *dp)
return dp->slave;
}
+static inline struct net_device *
+dsa_port_bridge_dev_get(const struct dsa_port *dp)
+{
+ return dp->bridge ? dp->bridge->dev : NULL;
+}
+
+static inline unsigned int dsa_port_bridge_num_get(struct dsa_port *dp)
+{
+ return dp->bridge ? dp->bridge->num : 0;
+}
+
+static inline bool dsa_port_bridge_same(const struct dsa_port *a,
+ const struct dsa_port *b)
+{
+ struct net_device *br_a = dsa_port_bridge_dev_get(a);
+ struct net_device *br_b = dsa_port_bridge_dev_get(b);
+
+ /* Standalone ports are not in the same bridge with one another */
+ return (!br_a || !br_b) ? false : (br_a == br_b);
+}
+
+static inline bool dsa_port_offloads_bridge_port(struct dsa_port *dp,
+ const struct net_device *dev)
+{
+ return dsa_port_to_bridge_port(dp) == dev;
+}
+
+static inline bool
+dsa_port_offloads_bridge_dev(struct dsa_port *dp,
+ const struct net_device *bridge_dev)
+{
+ /* DSA ports connected to a bridge, and event was emitted
+ * for the bridge.
+ */
+ return dsa_port_bridge_dev_get(dp) == bridge_dev;
+}
+
+static inline bool dsa_port_offloads_bridge(struct dsa_port *dp,
+ const struct dsa_bridge *bridge)
+{
+ return dsa_port_bridge_dev_get(dp) == bridge->dev;
+}
+
+/* Returns true if any port of this tree offloads the given net_device */
+static inline bool dsa_tree_offloads_bridge_port(struct dsa_switch_tree *dst,
+ const struct net_device *dev)
+{
+ struct dsa_port *dp;
+
+ list_for_each_entry(dp, &dst->ports, list)
+ if (dsa_port_offloads_bridge_port(dp, dev))
+ return true;
+
+ return false;
+}
+
+/* Returns true if any port of this tree offloads the given bridge */
+static inline bool
+dsa_tree_offloads_bridge_dev(struct dsa_switch_tree *dst,
+ const struct net_device *bridge_dev)
+{
+ struct dsa_port *dp;
+
+ list_for_each_entry(dp, &dst->ports, list)
+ if (dsa_port_offloads_bridge_dev(dp, bridge_dev))
+ return true;
+
+ return false;
+}
+
typedef int dsa_fdb_dump_cb_t(const unsigned char *addr, u16 vid,
bool is_static, void *data);
struct dsa_switch_ops {
@@ -614,6 +707,13 @@ struct dsa_switch_ops {
enum dsa_tag_protocol mprot);
int (*change_tag_protocol)(struct dsa_switch *ds, int port,
enum dsa_tag_protocol proto);
+ /*
+ * Method for switch drivers to connect to the tagging protocol driver
+ * in current use. The switch driver can provide handlers for certain
+ * types of packets for switch management.
+ */
+ int (*connect_tag_protocol)(struct dsa_switch *ds,
+ enum dsa_tag_protocol proto);
/* Optional switch-wide initialization and destruction methods */
int (*setup)(struct dsa_switch *ds);
@@ -645,8 +745,8 @@ struct dsa_switch_ops {
/*
* PHYLINK integration
*/
- void (*phylink_get_interfaces)(struct dsa_switch *ds, int port,
- unsigned long *supported_interfaces);
+ void (*phylink_get_caps)(struct dsa_switch *ds, int port,
+ struct phylink_config *config);
void (*phylink_validate)(struct dsa_switch *ds, int port,
unsigned long *supported,
struct phylink_link_state *state);
@@ -748,17 +848,10 @@ struct dsa_switch_ops {
*/
int (*set_ageing_time)(struct dsa_switch *ds, unsigned int msecs);
int (*port_bridge_join)(struct dsa_switch *ds, int port,
- struct net_device *bridge);
+ struct dsa_bridge bridge,
+ bool *tx_fwd_offload);
void (*port_bridge_leave)(struct dsa_switch *ds, int port,
- struct net_device *bridge);
- /* Called right after .port_bridge_join() */
- int (*port_bridge_tx_fwd_offload)(struct dsa_switch *ds, int port,
- struct net_device *bridge,
- int bridge_num);
- /* Called right before .port_bridge_leave() */
- void (*port_bridge_tx_fwd_unoffload)(struct dsa_switch *ds, int port,
- struct net_device *bridge,
- int bridge_num);
+ struct dsa_bridge bridge);
void (*port_stp_state_set)(struct dsa_switch *ds, int port,
u8 state);
void (*port_fast_age)(struct dsa_switch *ds, int port);
@@ -830,10 +923,10 @@ struct dsa_switch_ops {
*/
int (*crosschip_bridge_join)(struct dsa_switch *ds, int tree_index,
int sw_index, int port,
- struct net_device *br);
+ struct dsa_bridge bridge);
void (*crosschip_bridge_leave)(struct dsa_switch *ds, int tree_index,
int sw_index, int port,
- struct net_device *br);
+ struct dsa_bridge bridge);
int (*crosschip_lag_change)(struct dsa_switch *ds, int sw_index,
int port);
int (*crosschip_lag_join)(struct dsa_switch *ds, int sw_index,
diff --git a/include/net/dst.h b/include/net/dst.h
index a057319..6aa252c 100644
--- a/include/net/dst.h
+++ b/include/net/dst.h
@@ -77,6 +77,7 @@ struct dst_entry {
#ifndef CONFIG_64BIT
atomic_t __refcnt; /* 32-bit offset 64 */
#endif
+ netdevice_tracker dev_tracker;
};
struct dst_metrics {
diff --git a/include/net/failover.h b/include/net/failover.h
index bb15438..f2b42b4 100644
--- a/include/net/failover.h
+++ b/include/net/failover.h
@@ -25,6 +25,7 @@ struct failover_ops {
struct failover {
struct list_head list;
struct net_device __rcu *failover_dev;
+ netdevice_tracker dev_tracker;
struct failover_ops __rcu *ops;
};
diff --git a/include/net/fib_rules.h b/include/net/fib_rules.h
index bd07484a..82da359 100644
--- a/include/net/fib_rules.h
+++ b/include/net/fib_rules.h
@@ -91,7 +91,6 @@ struct fib_rules_ops {
void (*flush_cache)(struct fib_rules_ops *ops);
int nlgroup;
- const struct nla_policy *policy;
struct list_head rules_list;
struct module *owner;
struct net *fro_net;
@@ -103,26 +102,6 @@ struct fib_rule_notifier_info {
struct fib_rule *rule;
};
-#define FRA_GENERIC_POLICY \
- [FRA_UNSPEC] = { .strict_start_type = FRA_DPORT_RANGE + 1 }, \
- [FRA_IIFNAME] = { .type = NLA_STRING, .len = IFNAMSIZ - 1 }, \
- [FRA_OIFNAME] = { .type = NLA_STRING, .len = IFNAMSIZ - 1 }, \
- [FRA_PRIORITY] = { .type = NLA_U32 }, \
- [FRA_FWMARK] = { .type = NLA_U32 }, \
- [FRA_TUN_ID] = { .type = NLA_U64 }, \
- [FRA_FWMASK] = { .type = NLA_U32 }, \
- [FRA_TABLE] = { .type = NLA_U32 }, \
- [FRA_SUPPRESS_PREFIXLEN] = { .type = NLA_U32 }, \
- [FRA_SUPPRESS_IFGROUP] = { .type = NLA_U32 }, \
- [FRA_GOTO] = { .type = NLA_U32 }, \
- [FRA_L3MDEV] = { .type = NLA_U8 }, \
- [FRA_UID_RANGE] = { .len = sizeof(struct fib_rule_uid_range) }, \
- [FRA_PROTOCOL] = { .type = NLA_U8 }, \
- [FRA_IP_PROTO] = { .type = NLA_U8 }, \
- [FRA_SPORT_RANGE] = { .len = sizeof(struct fib_rule_port_range) }, \
- [FRA_DPORT_RANGE] = { .len = sizeof(struct fib_rule_port_range) }
-
-
static inline void fib_rule_get(struct fib_rule *rule)
{
refcount_inc(&rule->refcnt);
diff --git a/include/net/flow_offload.h b/include/net/flow_offload.h
index 3961461..5b8c54e 100644
--- a/include/net/flow_offload.h
+++ b/include/net/flow_offload.h
@@ -197,6 +197,7 @@ void flow_action_cookie_destroy(struct flow_action_cookie *cookie);
struct flow_action_entry {
enum flow_action_id id;
+ u32 hw_index;
enum flow_action_hw_stats hw_stats;
action_destr destructor;
void *destructor_priv;
@@ -232,7 +233,6 @@ struct flow_action_entry {
bool truncate;
} sample;
struct { /* FLOW_ACTION_POLICE */
- u32 index;
u32 burst;
u64 rate_bytes_ps;
u64 burst_pkt;
@@ -267,7 +267,6 @@ struct flow_action_entry {
u8 ttl;
} mpls_mangle;
struct {
- u32 index;
s32 prio;
u64 basetime;
u64 cycletime;
@@ -552,6 +551,23 @@ struct flow_cls_offload {
u32 classid;
};
+enum offload_act_command {
+ FLOW_ACT_REPLACE,
+ FLOW_ACT_DESTROY,
+ FLOW_ACT_STATS,
+};
+
+struct flow_offload_action {
+ struct netlink_ext_ack *extack; /* NULL in FLOW_ACT_STATS process*/
+ enum offload_act_command command;
+ enum flow_action_id id;
+ u32 index;
+ struct flow_stats stats;
+ struct flow_action action;
+};
+
+struct flow_offload_action *offload_action_alloc(unsigned int num_actions);
+
static inline struct flow_rule *
flow_cls_offload_flow_rule(struct flow_cls_offload *flow_cmd)
{
diff --git a/include/net/gro.h b/include/net/gro.h
index 01edaf3..8f75802 100644
--- a/include/net/gro.h
+++ b/include/net/gro.h
@@ -4,9 +4,368 @@
#define _NET_IPV6_GRO_H
#include <linux/indirect_call_wrapper.h>
+#include <linux/ip.h>
+#include <linux/ipv6.h>
+#include <net/ip6_checksum.h>
+#include <linux/skbuff.h>
+#include <net/udp.h>
-struct list_head;
-struct sk_buff;
+struct napi_gro_cb {
+ /* Virtual address of skb_shinfo(skb)->frags[0].page + offset. */
+ void *frag0;
+
+ /* Length of frag0. */
+ unsigned int frag0_len;
+
+ /* This indicates where we are processing relative to skb->data. */
+ int data_offset;
+
+ /* This is non-zero if the packet cannot be merged with the new skb. */
+ u16 flush;
+
+ /* Save the IP ID here and check when we get to the transport layer */
+ u16 flush_id;
+
+ /* Number of segments aggregated. */
+ u16 count;
+
+ /* Start offset for remote checksum offload */
+ u16 gro_remcsum_start;
+
+ /* jiffies when first packet was created/queued */
+ unsigned long age;
+
+ /* Used in ipv6_gro_receive() and foo-over-udp */
+ u16 proto;
+
+ /* This is non-zero if the packet may be of the same flow. */
+ u8 same_flow:1;
+
+ /* Used in tunnel GRO receive */
+ u8 encap_mark:1;
+
+ /* GRO checksum is valid */
+ u8 csum_valid:1;
+
+ /* Number of checksums via CHECKSUM_UNNECESSARY */
+ u8 csum_cnt:3;
+
+ /* Free the skb? */
+ u8 free:2;
+#define NAPI_GRO_FREE 1
+#define NAPI_GRO_FREE_STOLEN_HEAD 2
+
+ /* Used in foo-over-udp, set in udp[46]_gro_receive */
+ u8 is_ipv6:1;
+
+ /* Used in GRE, set in fou/gue_gro_receive */
+ u8 is_fou:1;
+
+ /* Used to determine if flush_id can be ignored */
+ u8 is_atomic:1;
+
+ /* Number of gro_receive callbacks this packet already went through */
+ u8 recursion_counter:4;
+
+ /* GRO is done by frag_list pointer chaining. */
+ u8 is_flist:1;
+
+ /* used to support CHECKSUM_COMPLETE for tunneling protocols */
+ __wsum csum;
+
+ /* used in skb_gro_receive() slow path */
+ struct sk_buff *last;
+};
+
+#define NAPI_GRO_CB(skb) ((struct napi_gro_cb *)(skb)->cb)
+
+#define GRO_RECURSION_LIMIT 15
+static inline int gro_recursion_inc_test(struct sk_buff *skb)
+{
+ return ++NAPI_GRO_CB(skb)->recursion_counter == GRO_RECURSION_LIMIT;
+}
+
+typedef struct sk_buff *(*gro_receive_t)(struct list_head *, struct sk_buff *);
+static inline struct sk_buff *call_gro_receive(gro_receive_t cb,
+ struct list_head *head,
+ struct sk_buff *skb)
+{
+ if (unlikely(gro_recursion_inc_test(skb))) {
+ NAPI_GRO_CB(skb)->flush |= 1;
+ return NULL;
+ }
+
+ return cb(head, skb);
+}
+
+typedef struct sk_buff *(*gro_receive_sk_t)(struct sock *, struct list_head *,
+ struct sk_buff *);
+static inline struct sk_buff *call_gro_receive_sk(gro_receive_sk_t cb,
+ struct sock *sk,
+ struct list_head *head,
+ struct sk_buff *skb)
+{
+ if (unlikely(gro_recursion_inc_test(skb))) {
+ NAPI_GRO_CB(skb)->flush |= 1;
+ return NULL;
+ }
+
+ return cb(sk, head, skb);
+}
+
+static inline unsigned int skb_gro_offset(const struct sk_buff *skb)
+{
+ return NAPI_GRO_CB(skb)->data_offset;
+}
+
+static inline unsigned int skb_gro_len(const struct sk_buff *skb)
+{
+ return skb->len - NAPI_GRO_CB(skb)->data_offset;
+}
+
+static inline void skb_gro_pull(struct sk_buff *skb, unsigned int len)
+{
+ NAPI_GRO_CB(skb)->data_offset += len;
+}
+
+static inline void *skb_gro_header_fast(struct sk_buff *skb,
+ unsigned int offset)
+{
+ return NAPI_GRO_CB(skb)->frag0 + offset;
+}
+
+static inline int skb_gro_header_hard(struct sk_buff *skb, unsigned int hlen)
+{
+ return NAPI_GRO_CB(skb)->frag0_len < hlen;
+}
+
+static inline void skb_gro_frag0_invalidate(struct sk_buff *skb)
+{
+ NAPI_GRO_CB(skb)->frag0 = NULL;
+ NAPI_GRO_CB(skb)->frag0_len = 0;
+}
+
+static inline void *skb_gro_header_slow(struct sk_buff *skb, unsigned int hlen,
+ unsigned int offset)
+{
+ if (!pskb_may_pull(skb, hlen))
+ return NULL;
+
+ skb_gro_frag0_invalidate(skb);
+ return skb->data + offset;
+}
+
+static inline void *skb_gro_network_header(struct sk_buff *skb)
+{
+ return (NAPI_GRO_CB(skb)->frag0 ?: skb->data) +
+ skb_network_offset(skb);
+}
+
+static inline __wsum inet_gro_compute_pseudo(struct sk_buff *skb, int proto)
+{
+ const struct iphdr *iph = skb_gro_network_header(skb);
+
+ return csum_tcpudp_nofold(iph->saddr, iph->daddr,
+ skb_gro_len(skb), proto, 0);
+}
+
+static inline void skb_gro_postpull_rcsum(struct sk_buff *skb,
+ const void *start, unsigned int len)
+{
+ if (NAPI_GRO_CB(skb)->csum_valid)
+ NAPI_GRO_CB(skb)->csum = wsum_negate(csum_partial(start, len,
+ wsum_negate(NAPI_GRO_CB(skb)->csum)));
+}
+
+/* GRO checksum functions. These are logical equivalents of the normal
+ * checksum functions (in skbuff.h) except that they operate on the GRO
+ * offsets and fields in sk_buff.
+ */
+
+__sum16 __skb_gro_checksum_complete(struct sk_buff *skb);
+
+static inline bool skb_at_gro_remcsum_start(struct sk_buff *skb)
+{
+ return (NAPI_GRO_CB(skb)->gro_remcsum_start == skb_gro_offset(skb));
+}
+
+static inline bool __skb_gro_checksum_validate_needed(struct sk_buff *skb,
+ bool zero_okay,
+ __sum16 check)
+{
+ return ((skb->ip_summed != CHECKSUM_PARTIAL ||
+ skb_checksum_start_offset(skb) <
+ skb_gro_offset(skb)) &&
+ !skb_at_gro_remcsum_start(skb) &&
+ NAPI_GRO_CB(skb)->csum_cnt == 0 &&
+ (!zero_okay || check));
+}
+
+static inline __sum16 __skb_gro_checksum_validate_complete(struct sk_buff *skb,
+ __wsum psum)
+{
+ if (NAPI_GRO_CB(skb)->csum_valid &&
+ !csum_fold(csum_add(psum, NAPI_GRO_CB(skb)->csum)))
+ return 0;
+
+ NAPI_GRO_CB(skb)->csum = psum;
+
+ return __skb_gro_checksum_complete(skb);
+}
+
+static inline void skb_gro_incr_csum_unnecessary(struct sk_buff *skb)
+{
+ if (NAPI_GRO_CB(skb)->csum_cnt > 0) {
+ /* Consume a checksum from CHECKSUM_UNNECESSARY */
+ NAPI_GRO_CB(skb)->csum_cnt--;
+ } else {
+ /* Update skb for CHECKSUM_UNNECESSARY and csum_level when we
+ * verified a new top level checksum or an encapsulated one
+ * during GRO. This saves work if we fallback to normal path.
+ */
+ __skb_incr_checksum_unnecessary(skb);
+ }
+}
+
+#define __skb_gro_checksum_validate(skb, proto, zero_okay, check, \
+ compute_pseudo) \
+({ \
+ __sum16 __ret = 0; \
+ if (__skb_gro_checksum_validate_needed(skb, zero_okay, check)) \
+ __ret = __skb_gro_checksum_validate_complete(skb, \
+ compute_pseudo(skb, proto)); \
+ if (!__ret) \
+ skb_gro_incr_csum_unnecessary(skb); \
+ __ret; \
+})
+
+#define skb_gro_checksum_validate(skb, proto, compute_pseudo) \
+ __skb_gro_checksum_validate(skb, proto, false, 0, compute_pseudo)
+
+#define skb_gro_checksum_validate_zero_check(skb, proto, check, \
+ compute_pseudo) \
+ __skb_gro_checksum_validate(skb, proto, true, check, compute_pseudo)
+
+#define skb_gro_checksum_simple_validate(skb) \
+ __skb_gro_checksum_validate(skb, 0, false, 0, null_compute_pseudo)
+
+static inline bool __skb_gro_checksum_convert_check(struct sk_buff *skb)
+{
+ return (NAPI_GRO_CB(skb)->csum_cnt == 0 &&
+ !NAPI_GRO_CB(skb)->csum_valid);
+}
+
+static inline void __skb_gro_checksum_convert(struct sk_buff *skb,
+ __wsum pseudo)
+{
+ NAPI_GRO_CB(skb)->csum = ~pseudo;
+ NAPI_GRO_CB(skb)->csum_valid = 1;
+}
+
+#define skb_gro_checksum_try_convert(skb, proto, compute_pseudo) \
+do { \
+ if (__skb_gro_checksum_convert_check(skb)) \
+ __skb_gro_checksum_convert(skb, \
+ compute_pseudo(skb, proto)); \
+} while (0)
+
+struct gro_remcsum {
+ int offset;
+ __wsum delta;
+};
+
+static inline void skb_gro_remcsum_init(struct gro_remcsum *grc)
+{
+ grc->offset = 0;
+ grc->delta = 0;
+}
+
+static inline void *skb_gro_remcsum_process(struct sk_buff *skb, void *ptr,
+ unsigned int off, size_t hdrlen,
+ int start, int offset,
+ struct gro_remcsum *grc,
+ bool nopartial)
+{
+ __wsum delta;
+ size_t plen = hdrlen + max_t(size_t, offset + sizeof(u16), start);
+
+ BUG_ON(!NAPI_GRO_CB(skb)->csum_valid);
+
+ if (!nopartial) {
+ NAPI_GRO_CB(skb)->gro_remcsum_start = off + hdrlen + start;
+ return ptr;
+ }
+
+ ptr = skb_gro_header_fast(skb, off);
+ if (skb_gro_header_hard(skb, off + plen)) {
+ ptr = skb_gro_header_slow(skb, off + plen, off);
+ if (!ptr)
+ return NULL;
+ }
+
+ delta = remcsum_adjust(ptr + hdrlen, NAPI_GRO_CB(skb)->csum,
+ start, offset);
+
+ /* Adjust skb->csum since we changed the packet */
+ NAPI_GRO_CB(skb)->csum = csum_add(NAPI_GRO_CB(skb)->csum, delta);
+
+ grc->offset = off + hdrlen + offset;
+ grc->delta = delta;
+
+ return ptr;
+}
+
+static inline void skb_gro_remcsum_cleanup(struct sk_buff *skb,
+ struct gro_remcsum *grc)
+{
+ void *ptr;
+ size_t plen = grc->offset + sizeof(u16);
+
+ if (!grc->delta)
+ return;
+
+ ptr = skb_gro_header_fast(skb, grc->offset);
+ if (skb_gro_header_hard(skb, grc->offset + sizeof(u16))) {
+ ptr = skb_gro_header_slow(skb, plen, grc->offset);
+ if (!ptr)
+ return;
+ }
+
+ remcsum_unadjust((__sum16 *)ptr, grc->delta);
+}
+
+#ifdef CONFIG_XFRM_OFFLOAD
+static inline void skb_gro_flush_final(struct sk_buff *skb, struct sk_buff *pp, int flush)
+{
+ if (PTR_ERR(pp) != -EINPROGRESS)
+ NAPI_GRO_CB(skb)->flush |= flush;
+}
+static inline void skb_gro_flush_final_remcsum(struct sk_buff *skb,
+ struct sk_buff *pp,
+ int flush,
+ struct gro_remcsum *grc)
+{
+ if (PTR_ERR(pp) != -EINPROGRESS) {
+ NAPI_GRO_CB(skb)->flush |= flush;
+ skb_gro_remcsum_cleanup(skb, grc);
+ skb->remcsum_offload = 0;
+ }
+}
+#else
+static inline void skb_gro_flush_final(struct sk_buff *skb, struct sk_buff *pp, int flush)
+{
+ NAPI_GRO_CB(skb)->flush |= flush;
+}
+static inline void skb_gro_flush_final_remcsum(struct sk_buff *skb,
+ struct sk_buff *pp,
+ int flush,
+ struct gro_remcsum *grc)
+{
+ NAPI_GRO_CB(skb)->flush |= flush;
+ skb_gro_remcsum_cleanup(skb, grc);
+ skb->remcsum_offload = 0;
+}
+#endif
INDIRECT_CALLABLE_DECLARE(struct sk_buff *ipv6_gro_receive(struct list_head *,
struct sk_buff *));
@@ -15,6 +374,14 @@ INDIRECT_CALLABLE_DECLARE(struct sk_buff *inet_gro_receive(struct list_head *,
struct sk_buff *));
INDIRECT_CALLABLE_DECLARE(int inet_gro_complete(struct sk_buff *, int));
+INDIRECT_CALLABLE_DECLARE(struct sk_buff *udp4_gro_receive(struct list_head *,
+ struct sk_buff *));
+INDIRECT_CALLABLE_DECLARE(int udp4_gro_complete(struct sk_buff *, int));
+
+INDIRECT_CALLABLE_DECLARE(struct sk_buff *udp6_gro_receive(struct list_head *,
+ struct sk_buff *));
+INDIRECT_CALLABLE_DECLARE(int udp6_gro_complete(struct sk_buff *, int));
+
#define indirect_call_gro_receive_inet(cb, f2, f1, head, skb) \
({ \
unlikely(gro_recursion_inc_test(skb)) ? \
@@ -22,4 +389,54 @@ INDIRECT_CALLABLE_DECLARE(int inet_gro_complete(struct sk_buff *, int));
INDIRECT_CALL_INET(cb, f2, f1, head, skb); \
})
+struct sk_buff *udp_gro_receive(struct list_head *head, struct sk_buff *skb,
+ struct udphdr *uh, struct sock *sk);
+int udp_gro_complete(struct sk_buff *skb, int nhoff, udp_lookup_t lookup);
+
+static inline struct udphdr *udp_gro_udphdr(struct sk_buff *skb)
+{
+ struct udphdr *uh;
+ unsigned int hlen, off;
+
+ off = skb_gro_offset(skb);
+ hlen = off + sizeof(*uh);
+ uh = skb_gro_header_fast(skb, off);
+ if (skb_gro_header_hard(skb, hlen))
+ uh = skb_gro_header_slow(skb, hlen, off);
+
+ return uh;
+}
+
+static inline __wsum ip6_gro_compute_pseudo(struct sk_buff *skb, int proto)
+{
+ const struct ipv6hdr *iph = skb_gro_network_header(skb);
+
+ return ~csum_unfold(csum_ipv6_magic(&iph->saddr, &iph->daddr,
+ skb_gro_len(skb), proto, 0));
+}
+
+int skb_gro_receive(struct sk_buff *p, struct sk_buff *skb);
+
+/* Pass the currently batched GRO_NORMAL SKBs up to the stack. */
+static inline void gro_normal_list(struct napi_struct *napi)
+{
+ if (!napi->rx_count)
+ return;
+ netif_receive_skb_list_internal(&napi->rx_list);
+ INIT_LIST_HEAD(&napi->rx_list);
+ napi->rx_count = 0;
+}
+
+/* Queue one GRO_NORMAL SKB up for list processing. If batch size exceeded,
+ * pass the whole batch up to the stack.
+ */
+static inline void gro_normal_one(struct napi_struct *napi, struct sk_buff *skb, int segs)
+{
+ list_add_tail(&skb->list, &napi->rx_list);
+ napi->rx_count += segs;
+ if (napi->rx_count >= gro_normal_batch)
+ gro_normal_list(napi);
+}
+
+
#endif /* _NET_IPV6_GRO_H */
diff --git a/include/net/if_inet6.h b/include/net/if_inet6.h
index 653e7d0..f026cf0 100644
--- a/include/net/if_inet6.h
+++ b/include/net/if_inet6.h
@@ -160,6 +160,7 @@ struct ipv6_devstat {
struct inet6_dev {
struct net_device *dev;
+ netdevice_tracker dev_tracker;
struct list_head addr_list;
diff --git a/include/net/inet_connection_sock.h b/include/net/inet_connection_sock.h
index fa6a872..4ad47d9 100644
--- a/include/net/inet_connection_sock.h
+++ b/include/net/inet_connection_sock.h
@@ -304,7 +304,7 @@ static inline __poll_t inet_csk_listen_poll(const struct sock *sk)
(EPOLLIN | EPOLLRDNORM) : 0;
}
-int inet_csk_listen_start(struct sock *sk, int backlog);
+int inet_csk_listen_start(struct sock *sk);
void inet_csk_listen_stop(struct sock *sk);
void inet_csk_addr2sockaddr(struct sock *sk, struct sockaddr *uaddr);
diff --git a/include/net/inet_sock.h b/include/net/inet_sock.h
index 9e1111f..234d70a 100644
--- a/include/net/inet_sock.h
+++ b/include/net/inet_sock.h
@@ -372,4 +372,16 @@ static inline bool inet_can_nonlocal_bind(struct net *net,
inet->freebind || inet->transparent;
}
+static inline bool inet_addr_valid_or_nonlocal(struct net *net,
+ struct inet_sock *inet,
+ __be32 addr,
+ int addr_type)
+{
+ return inet_can_nonlocal_bind(net, inet) ||
+ addr == htonl(INADDR_ANY) ||
+ addr_type == RTN_LOCAL ||
+ addr_type == RTN_MULTICAST ||
+ addr_type == RTN_BROADCAST;
+}
+
#endif /* _INET_SOCK_H */
diff --git a/include/net/ip.h b/include/net/ip.h
index b71e885..81e23a1 100644
--- a/include/net/ip.h
+++ b/include/net/ip.h
@@ -568,14 +568,6 @@ static inline void iph_to_flow_copy_v4addrs(struct flow_keys *flow,
flow->control.addr_type = FLOW_DISSECTOR_KEY_IPV4_ADDRS;
}
-static inline __wsum inet_gro_compute_pseudo(struct sk_buff *skb, int proto)
-{
- const struct iphdr *iph = skb_gro_network_header(skb);
-
- return csum_tcpudp_nofold(iph->saddr, iph->daddr,
- skb_gro_len(skb), proto, 0);
-}
-
/*
* Map a multicast IP onto multicast MAC for type ethernet.
*/
@@ -791,5 +783,6 @@ int ip_sock_set_mtu_discover(struct sock *sk, int val);
void ip_sock_set_pktinfo(struct sock *sk);
void ip_sock_set_recverr(struct sock *sk);
void ip_sock_set_tos(struct sock *sk, int val);
+void __ip_sock_set_tos(struct sock *sk, int val);
#endif /* _IP_H */
diff --git a/include/net/ip6_checksum.h b/include/net/ip6_checksum.h
index b3f4eaa..c8a96b8 100644
--- a/include/net/ip6_checksum.h
+++ b/include/net/ip6_checksum.h
@@ -43,14 +43,6 @@ static inline __wsum ip6_compute_pseudo(struct sk_buff *skb, int proto)
skb->len, proto, 0));
}
-static inline __wsum ip6_gro_compute_pseudo(struct sk_buff *skb, int proto)
-{
- const struct ipv6hdr *iph = skb_gro_network_header(skb);
-
- return ~csum_unfold(csum_ipv6_magic(&iph->saddr, &iph->daddr,
- skb_gro_len(skb), proto, 0));
-}
-
static __inline__ __sum16 tcp_v6_check(int len,
const struct in6_addr *saddr,
const struct in6_addr *daddr,
@@ -65,15 +57,9 @@ static inline void __tcp_v6_send_check(struct sk_buff *skb,
{
struct tcphdr *th = tcp_hdr(skb);
- if (skb->ip_summed == CHECKSUM_PARTIAL) {
- th->check = ~tcp_v6_check(skb->len, saddr, daddr, 0);
- skb->csum_start = skb_transport_header(skb) - skb->head;
- skb->csum_offset = offsetof(struct tcphdr, check);
- } else {
- th->check = tcp_v6_check(skb->len, saddr, daddr,
- csum_partial(th, th->doff << 2,
- skb->csum));
- }
+ th->check = ~tcp_v6_check(skb->len, saddr, daddr, 0);
+ skb->csum_start = skb_transport_header(skb) - skb->head;
+ skb->csum_offset = offsetof(struct tcphdr, check);
}
static inline void tcp_v6_gso_csum_prep(struct sk_buff *skb)
diff --git a/include/net/ip6_fib.h b/include/net/ip6_fib.h
index 83b8070..a9a4ccc 100644
--- a/include/net/ip6_fib.h
+++ b/include/net/ip6_fib.h
@@ -20,6 +20,7 @@
#include <net/inetpeer.h>
#include <net/fib_notifier.h>
#include <linux/indirect_call_wrapper.h>
+#include <uapi/linux/bpf.h>
#ifdef CONFIG_IPV6_MULTIPLE_TABLES
#define FIB6_TABLE_HASHSZ 256
diff --git a/include/net/ip6_route.h b/include/net/ip6_route.h
index 5efd0b7..ca2d6b6 100644
--- a/include/net/ip6_route.h
+++ b/include/net/ip6_route.h
@@ -263,19 +263,19 @@ static inline bool ipv6_anycast_destination(const struct dst_entry *dst,
int ip6_fragment(struct net *net, struct sock *sk, struct sk_buff *skb,
int (*output)(struct net *, struct sock *, struct sk_buff *));
-static inline unsigned int ip6_skb_dst_mtu(struct sk_buff *skb)
+static inline unsigned int ip6_skb_dst_mtu(const struct sk_buff *skb)
{
+ const struct ipv6_pinfo *np = skb->sk && !dev_recursion_level() ?
+ inet6_sk(skb->sk) : NULL;
+ const struct dst_entry *dst = skb_dst(skb);
unsigned int mtu;
- struct ipv6_pinfo *np = skb->sk && !dev_recursion_level() ?
- inet6_sk(skb->sk) : NULL;
-
if (np && np->pmtudisc >= IPV6_PMTUDISC_PROBE) {
- mtu = READ_ONCE(skb_dst(skb)->dev->mtu);
- mtu -= lwtunnel_headroom(skb_dst(skb)->lwtstate, mtu);
- } else
- mtu = dst_mtu(skb_dst(skb));
-
+ mtu = READ_ONCE(dst->dev->mtu);
+ mtu -= lwtunnel_headroom(dst->lwtstate, mtu);
+ } else {
+ mtu = dst_mtu(dst);
+ }
return mtu;
}
diff --git a/include/net/ip6_tunnel.h b/include/net/ip6_tunnel.h
index 028eaea..a38c4f1 100644
--- a/include/net/ip6_tunnel.h
+++ b/include/net/ip6_tunnel.h
@@ -46,6 +46,7 @@ struct __ip6_tnl_parm {
struct ip6_tnl {
struct ip6_tnl __rcu *next; /* next tunnel in list */
struct net_device *dev; /* virtual device associated with tunnel */
+ netdevice_tracker dev_tracker;
struct net *net; /* netns for packet i/o */
struct __ip6_tnl_parm parms; /* tunnel configuration parameters */
struct flowi fl; /* flowi template for xmit */
diff --git a/include/net/ip_fib.h b/include/net/ip_fib.h
index 3417ba2..c4297704 100644
--- a/include/net/ip_fib.h
+++ b/include/net/ip_fib.h
@@ -79,6 +79,7 @@ struct fnhe_hash_bucket {
struct fib_nh_common {
struct net_device *nhc_dev;
+ netdevice_tracker nhc_dev_tracker;
int nhc_oif;
unsigned char nhc_scope;
u8 nhc_family;
@@ -111,6 +112,7 @@ struct fib_nh {
int nh_saddr_genid;
#define fib_nh_family nh_common.nhc_family
#define fib_nh_dev nh_common.nhc_dev
+#define fib_nh_dev_tracker nh_common.nhc_dev_tracker
#define fib_nh_oif nh_common.nhc_oif
#define fib_nh_flags nh_common.nhc_flags
#define fib_nh_lws nh_common.nhc_lwtstate
diff --git a/include/net/ip_tunnels.h b/include/net/ip_tunnels.h
index bc3b13e..0219fe9 100644
--- a/include/net/ip_tunnels.h
+++ b/include/net/ip_tunnels.h
@@ -104,7 +104,10 @@ struct metadata_dst;
struct ip_tunnel {
struct ip_tunnel __rcu *next;
struct hlist_node hash_node;
+
struct net_device *dev;
+ netdevice_tracker dev_tracker;
+
struct net *net; /* netns for packet i/o */
unsigned long err_time; /* Time when the last ICMP error
diff --git a/include/net/ipv6.h b/include/net/ipv6.h
index c19bf51..3afcb12 100644
--- a/include/net/ipv6.h
+++ b/include/net/ipv6.h
@@ -21,6 +21,8 @@
#include <net/snmp.h>
#include <net/netns/hash.h>
+struct ip_tunnel_info;
+
#define SIN6_LEN_RFC2133 24
#define IPV6_MAXPLEN 65535
@@ -345,9 +347,9 @@ struct ipcm6_cookie {
struct sockcm_cookie sockc;
__s16 hlimit;
__s16 tclass;
+ __u16 gso_size;
__s8 dontfrag;
struct ipv6_txoptions *opt;
- __u16 gso_size;
};
static inline void ipcm6_init(struct ipcm6_cookie *ipc6)
diff --git a/include/net/iucv/af_iucv.h b/include/net/iucv/af_iucv.h
index ff06246..df85d19 100644
--- a/include/net/iucv/af_iucv.h
+++ b/include/net/iucv/af_iucv.h
@@ -112,10 +112,12 @@ enum iucv_tx_notify {
struct iucv_sock {
struct sock sk;
- char src_user_id[8];
- char src_name[8];
- char dst_user_id[8];
- char dst_name[8];
+ struct_group(init,
+ char src_user_id[8];
+ char src_name[8];
+ char dst_user_id[8];
+ char dst_name[8];
+ );
struct list_head accept_q;
spinlock_t accept_q_lock;
struct sock *parent;
diff --git a/include/net/llc_conn.h b/include/net/llc_conn.h
index ea985aa..2c1ea34 100644
--- a/include/net/llc_conn.h
+++ b/include/net/llc_conn.h
@@ -38,6 +38,7 @@ struct llc_sock {
struct llc_addr laddr; /* lsap/mac pair */
struct llc_addr daddr; /* dsap/mac pair */
struct net_device *dev; /* device to send to remote */
+ netdevice_tracker dev_tracker;
u32 copied_seq; /* head of yet unread data */
u8 retry_count; /* number of retries */
u8 ack_must_be_send;
diff --git a/include/net/mac80211.h b/include/net/mac80211.h
index dd757f0..c50221d 100644
--- a/include/net/mac80211.h
+++ b/include/net/mac80211.h
@@ -1205,12 +1205,7 @@ ieee80211_tx_info_clear_status(struct ieee80211_tx_info *info)
/* clear the rate counts */
for (i = 0; i < IEEE80211_TX_MAX_RATES; i++)
info->status.rates[i].count = 0;
-
- BUILD_BUG_ON(
- offsetof(struct ieee80211_tx_info, status.ack_signal) != 20);
- memset(&info->status.ampdu_ack_len, 0,
- sizeof(struct ieee80211_tx_info) -
- offsetof(struct ieee80211_tx_info, status.ampdu_ack_len));
+ memset_after(&info->status, 0, rates);
}
@@ -3944,6 +3939,16 @@ struct ieee80211_prep_tx_info {
* twt structure.
* @twt_teardown_request: Update the hw with TWT teardown request received
* from the peer.
+ * @set_radar_background: Configure dedicated offchannel chain available for
+ * radar/CAC detection on some hw. This chain can't be used to transmit
+ * or receive frames and it is bounded to a running wdev.
+ * Background radar/CAC detection allows to avoid the CAC downtime
+ * switching to a different channel during CAC detection on the selected
+ * radar channel.
+ * The caller is expected to set chandef pointer to NULL in order to
+ * disable background CAC/radar detection.
+ * @net_fill_forward_path: Called from .ndo_fill_forward_path in order to
+ * resolve a path for hardware flow offloading
*/
struct ieee80211_ops {
void (*tx)(struct ieee80211_hw *hw,
@@ -4272,6 +4277,13 @@ struct ieee80211_ops {
struct ieee80211_twt_setup *twt);
void (*twt_teardown_request)(struct ieee80211_hw *hw,
struct ieee80211_sta *sta, u8 flowid);
+ int (*set_radar_background)(struct ieee80211_hw *hw,
+ struct cfg80211_chan_def *chandef);
+ int (*net_fill_forward_path)(struct ieee80211_hw *hw,
+ struct ieee80211_vif *vif,
+ struct ieee80211_sta *sta,
+ struct net_device_path_ctx *ctx,
+ struct net_device_path *path);
};
/**
@@ -5602,6 +5614,9 @@ void ieee80211_iterate_active_interfaces_atomic(struct ieee80211_hw *hw,
* This function iterates over the interfaces associated with a given
* hardware that are currently active and calls the callback for them.
* This version can only be used while holding the wiphy mutex.
+ * The driver must not call this with a lock held that it can also take in
+ * response to callbacks from mac80211, and it must not call this within
+ * callbacks made by mac80211 - both would result in deadlocks.
*
* @hw: the hardware struct of which the interfaces should be iterated over
* @iter_flags: iteration flags, see &enum ieee80211_interface_iteration_flags
@@ -5616,6 +5631,24 @@ void ieee80211_iterate_active_interfaces_mtx(struct ieee80211_hw *hw,
void *data);
/**
+ * ieee80211_iterate_stations - iterate stations
+ *
+ * This function iterates over all stations associated with a given
+ * hardware that are currently uploaded to the driver and calls the callback
+ * function for them.
+ * This function allows the iterator function to sleep, when the iterator
+ * function is atomic @ieee80211_iterate_stations_atomic can be used.
+ *
+ * @hw: the hardware struct of which the interfaces should be iterated over
+ * @iterator: the iterator function to call, cannot sleep
+ * @data: first argument of the iterator function
+ */
+void ieee80211_iterate_stations(struct ieee80211_hw *hw,
+ void (*iterator)(void *data,
+ struct ieee80211_sta *sta),
+ void *data);
+
+/**
* ieee80211_iterate_stations_atomic - iterate stations
*
* This function iterates over all stations associated with a given
@@ -6064,6 +6097,18 @@ void ieee80211_radar_detected(struct ieee80211_hw *hw);
void ieee80211_chswitch_done(struct ieee80211_vif *vif, bool success);
/**
+ * ieee80211_channel_switch_disconnect - disconnect due to channel switch error
+ * @vif &struct ieee80211_vif pointer from the add_interface callback.
+ * @block_tx: if %true, do not send deauth frame.
+ *
+ * Instruct mac80211 to disconnect due to a channel switch error. The channel
+ * switch can request to block the tx and so, we need to make sure we do not send
+ * a deauth frame in this case.
+ */
+void ieee80211_channel_switch_disconnect(struct ieee80211_vif *vif,
+ bool block_tx);
+
+/**
* ieee80211_request_smps - request SM PS transition
* @vif: &struct ieee80211_vif pointer from the add_interface callback.
* @smps_mode: new SM PS mode
diff --git a/include/net/ndisc.h b/include/net/ndisc.h
index 04341d8..53cb8de 100644
--- a/include/net/ndisc.h
+++ b/include/net/ndisc.h
@@ -411,13 +411,7 @@ static inline void __ipv6_confirm_neigh(struct net_device *dev,
rcu_read_lock_bh();
n = __ipv6_neigh_lookup_noref(dev, pkey);
- if (n) {
- unsigned long now = jiffies;
-
- /* avoid dirtying neighbour */
- if (READ_ONCE(n->confirmed) != now)
- WRITE_ONCE(n->confirmed, now);
- }
+ neigh_confirm(n);
rcu_read_unlock_bh();
}
@@ -428,13 +422,7 @@ static inline void __ipv6_confirm_neigh_stub(struct net_device *dev,
rcu_read_lock_bh();
n = __ipv6_neigh_lookup_noref_stub(dev, pkey);
- if (n) {
- unsigned long now = jiffies;
-
- /* avoid dirtying neighbour */
- if (READ_ONCE(n->confirmed) != now)
- WRITE_ONCE(n->confirmed, now);
- }
+ neigh_confirm(n);
rcu_read_unlock_bh();
}
diff --git a/include/net/neighbour.h b/include/net/neighbour.h
index 38a0c1d..937389e 100644
--- a/include/net/neighbour.h
+++ b/include/net/neighbour.h
@@ -70,6 +70,7 @@ enum {
struct neigh_parms {
possible_net_t net;
struct net_device *dev;
+ netdevice_tracker dev_tracker;
struct list_head list;
int (*neigh_setup)(struct neighbour *);
struct neigh_table *tbl;
@@ -158,6 +159,7 @@ struct neighbour {
struct list_head managed_list;
struct rcu_head rcu;
struct net_device *dev;
+ netdevice_tracker dev_tracker;
u8 primary_key[0];
} __randomize_layout;
@@ -173,6 +175,7 @@ struct pneigh_entry {
struct pneigh_entry *next;
possible_net_t net;
struct net_device *dev;
+ netdevice_tracker dev_tracker;
u32 flags;
u8 protocol;
u8 key[];
@@ -321,6 +324,17 @@ static inline struct neighbour *__neigh_lookup_noref(struct neigh_table *tbl,
return ___neigh_lookup_noref(tbl, tbl->key_eq, tbl->hash, pkey, dev);
}
+static inline void neigh_confirm(struct neighbour *n)
+{
+ if (n) {
+ unsigned long now = jiffies;
+
+ /* avoid dirtying neighbour */
+ if (READ_ONCE(n->confirmed) != now)
+ WRITE_ONCE(n->confirmed, now);
+ }
+}
+
void neigh_table_init(int index, struct neigh_table *tbl);
int neigh_table_clear(int index, struct neigh_table *tbl);
struct neighbour *neigh_lookup(struct neigh_table *tbl, const void *pkey,
diff --git a/include/net/net_namespace.h b/include/net/net_namespace.h
index bb5fa59..5b61c46 100644
--- a/include/net/net_namespace.h
+++ b/include/net/net_namespace.h
@@ -34,6 +34,7 @@
#include <net/netns/smc.h>
#include <net/netns/bpf.h>
#include <net/netns/mctp.h>
+#include <net/net_trackers.h>
#include <linux/ns_common.h>
#include <linux/idr.h>
#include <linux/skbuff.h>
@@ -87,6 +88,7 @@ struct net {
struct idr netns_ids;
struct ns_common ns;
+ struct ref_tracker_dir refcnt_tracker;
struct list_head dev_base_head;
struct proc_dir_entry *proc_net;
@@ -240,6 +242,7 @@ void ipx_unregister_sysctl(void);
#ifdef CONFIG_NET_NS
void __put_net(struct net *net);
+/* Try using get_net_track() instead */
static inline struct net *get_net(struct net *net)
{
refcount_inc(&net->ns.count);
@@ -258,6 +261,7 @@ static inline struct net *maybe_get_net(struct net *net)
return net;
}
+/* Try using put_net_track() instead */
static inline void put_net(struct net *net)
{
if (refcount_dec_and_test(&net->ns.count))
@@ -308,6 +312,36 @@ static inline int check_net(const struct net *net)
#endif
+static inline void netns_tracker_alloc(struct net *net,
+ netns_tracker *tracker, gfp_t gfp)
+{
+#ifdef CONFIG_NET_NS_REFCNT_TRACKER
+ ref_tracker_alloc(&net->refcnt_tracker, tracker, gfp);
+#endif
+}
+
+static inline void netns_tracker_free(struct net *net,
+ netns_tracker *tracker)
+{
+#ifdef CONFIG_NET_NS_REFCNT_TRACKER
+ ref_tracker_free(&net->refcnt_tracker, tracker);
+#endif
+}
+
+static inline struct net *get_net_track(struct net *net,
+ netns_tracker *tracker, gfp_t gfp)
+{
+ get_net(net);
+ netns_tracker_alloc(net, tracker, gfp);
+ return net;
+}
+
+static inline void put_net_track(struct net *net, netns_tracker *tracker)
+{
+ netns_tracker_free(net, tracker);
+ put_net(net);
+}
+
typedef struct {
#ifdef CONFIG_NET_NS
struct net *net;
diff --git a/include/net/net_trackers.h b/include/net/net_trackers.h
new file mode 100644
index 0000000..d94c76c
--- /dev/null
+++ b/include/net/net_trackers.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __NET_NET_TRACKERS_H
+#define __NET_NET_TRACKERS_H
+#include <linux/ref_tracker.h>
+
+#ifdef CONFIG_NET_DEV_REFCNT_TRACKER
+typedef struct ref_tracker *netdevice_tracker;
+#else
+typedef struct {} netdevice_tracker;
+#endif
+
+#ifdef CONFIG_NET_NS_REFCNT_TRACKER
+typedef struct ref_tracker *netns_tracker;
+#else
+typedef struct {} netns_tracker;
+#endif
+
+#endif /* __NET_NET_TRACKERS_H */
diff --git a/include/net/netfilter/nf_conntrack.h b/include/net/netfilter/nf_conntrack.h
index d24b0a3..8731d5b 100644
--- a/include/net/netfilter/nf_conntrack.h
+++ b/include/net/netfilter/nf_conntrack.h
@@ -76,6 +76,8 @@ struct nf_conn {
* Hint, SKB address this struct and refcnt via skb->_nfct and
* helpers nf_conntrack_get() and nf_conntrack_put().
* Helper nf_ct_put() equals nf_conntrack_put() by dec refcnt,
+ * except that the latter uses internal indirection and does not
+ * result in a conntrack module dependency.
* beware nf_ct_get() is different and don't inc refcnt.
*/
struct nf_conntrack ct_general;
@@ -95,6 +97,7 @@ struct nf_conn {
unsigned long status;
u16 cpu;
+ u16 local_origin:1;
possible_net_t ct_net;
#if IS_ENABLED(CONFIG_NF_NAT)
@@ -169,11 +172,13 @@ nf_ct_get(const struct sk_buff *skb, enum ip_conntrack_info *ctinfo)
return (struct nf_conn *)(nfct & NFCT_PTRMASK);
}
+void nf_ct_destroy(struct nf_conntrack *nfct);
+
/* decrement reference count on a conntrack */
static inline void nf_ct_put(struct nf_conn *ct)
{
- WARN_ON(!ct);
- nf_conntrack_put(&ct->ct_general);
+ if (ct && refcount_dec_and_test(&ct->ct_general.use))
+ nf_ct_destroy(&ct->ct_general);
}
/* Protocol module loading */
@@ -278,7 +283,7 @@ static inline unsigned long nf_ct_expires(const struct nf_conn *ct)
{
s32 timeout = READ_ONCE(ct->timeout) - nfct_time_stamp;
- return timeout > 0 ? timeout : 0;
+ return max(timeout, 0);
}
static inline bool nf_ct_is_expired(const struct nf_conn *ct)
diff --git a/include/net/netfilter/nf_conntrack_act_ct.h b/include/net/netfilter/nf_conntrack_act_ct.h
new file mode 100644
index 0000000..078d3c5
--- /dev/null
+++ b/include/net/netfilter/nf_conntrack_act_ct.h
@@ -0,0 +1,50 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#ifndef _NF_CONNTRACK_ACT_CT_H
+#define _NF_CONNTRACK_ACT_CT_H
+
+#include <net/netfilter/nf_conntrack.h>
+#include <linux/netfilter/nf_conntrack_common.h>
+#include <net/netfilter/nf_conntrack_extend.h>
+
+struct nf_conn_act_ct_ext {
+ int ifindex[IP_CT_DIR_MAX];
+};
+
+static inline struct nf_conn_act_ct_ext *nf_conn_act_ct_ext_find(const struct nf_conn *ct)
+{
+#if IS_ENABLED(CONFIG_NET_ACT_CT)
+ return nf_ct_ext_find(ct, NF_CT_EXT_ACT_CT);
+#else
+ return NULL;
+#endif
+}
+
+static inline struct nf_conn_act_ct_ext *nf_conn_act_ct_ext_add(struct nf_conn *ct)
+{
+#if IS_ENABLED(CONFIG_NET_ACT_CT)
+ struct nf_conn_act_ct_ext *act_ct = nf_ct_ext_find(ct, NF_CT_EXT_ACT_CT);
+
+ if (act_ct)
+ return act_ct;
+
+ act_ct = nf_ct_ext_add(ct, NF_CT_EXT_ACT_CT, GFP_ATOMIC);
+ return act_ct;
+#else
+ return NULL;
+#endif
+}
+
+static inline void nf_conn_act_ct_ext_fill(struct sk_buff *skb, struct nf_conn *ct,
+ enum ip_conntrack_info ctinfo)
+{
+#if IS_ENABLED(CONFIG_NET_ACT_CT)
+ struct nf_conn_act_ct_ext *act_ct_ext;
+
+ act_ct_ext = nf_conn_act_ct_ext_find(ct);
+ if (dev_net(skb->dev) == &init_net && act_ct_ext)
+ act_ct_ext->ifindex[CTINFO2DIR(ctinfo)] = skb->dev->ifindex;
+#endif
+}
+
+#endif /* _NF_CONNTRACK_ACT_CT_H */
diff --git a/include/net/netfilter/nf_conntrack_extend.h b/include/net/netfilter/nf_conntrack_extend.h
index e1e5883..c7515d8 100644
--- a/include/net/netfilter/nf_conntrack_extend.h
+++ b/include/net/netfilter/nf_conntrack_extend.h
@@ -28,6 +28,9 @@ enum nf_ct_ext_id {
#if IS_ENABLED(CONFIG_NETFILTER_SYNPROXY)
NF_CT_EXT_SYNPROXY,
#endif
+#if IS_ENABLED(CONFIG_NET_ACT_CT)
+ NF_CT_EXT_ACT_CT,
+#endif
NF_CT_EXT_NUM,
};
@@ -40,6 +43,7 @@ enum nf_ct_ext_id {
#define NF_CT_EXT_TIMEOUT_TYPE struct nf_conn_timeout
#define NF_CT_EXT_LABELS_TYPE struct nf_conn_labels
#define NF_CT_EXT_SYNPROXY_TYPE struct nf_conn_synproxy
+#define NF_CT_EXT_ACT_CT_TYPE struct nf_conn_act_ct_ext
/* Extensions: optional stuff which isn't permanently in struct. */
struct nf_ct_ext {
diff --git a/include/net/netfilter/nf_tables.h b/include/net/netfilter/nf_tables.h
index a0d9e0b..eaf55da 100644
--- a/include/net/netfilter/nf_tables.h
+++ b/include/net/netfilter/nf_tables.h
@@ -105,6 +105,8 @@ struct nft_data {
};
} __attribute__((aligned(__alignof__(u64))));
+#define NFT_REG32_NUM 20
+
/**
* struct nft_regs - nf_tables register set
*
@@ -115,11 +117,21 @@ struct nft_data {
*/
struct nft_regs {
union {
- u32 data[20];
+ u32 data[NFT_REG32_NUM];
struct nft_verdict verdict;
};
};
+struct nft_regs_track {
+ struct {
+ const struct nft_expr *selector;
+ const struct nft_expr *bitwise;
+ } regs[NFT_REG32_NUM];
+
+ const struct nft_expr *cur;
+ const struct nft_expr *last;
+};
+
/* Store/load an u8, u16 or u64 integer to/from the u32 data register.
*
* Note, when using concatenations, register allocation happens at 32-bit
@@ -346,6 +358,8 @@ int nft_expr_clone(struct nft_expr *dst, struct nft_expr *src);
void nft_expr_destroy(const struct nft_ctx *ctx, struct nft_expr *expr);
int nft_expr_dump(struct sk_buff *skb, unsigned int attr,
const struct nft_expr *expr);
+bool nft_expr_reduce_bitwise(struct nft_regs_track *track,
+ const struct nft_expr *expr);
struct nft_set_ext;
@@ -884,6 +898,8 @@ struct nft_expr_ops {
int (*validate)(const struct nft_ctx *ctx,
const struct nft_expr *expr,
const struct nft_data **data);
+ bool (*reduce)(struct nft_regs_track *track,
+ const struct nft_expr *expr);
bool (*gc)(struct net *net,
const struct nft_expr *expr);
int (*offload)(struct nft_offload_ctx *ctx,
@@ -974,6 +990,20 @@ static inline void nft_set_elem_update_expr(const struct nft_set_ext *ext,
#define NFT_CHAIN_POLICY_UNSET U8_MAX
+struct nft_rule_dp {
+ u64 is_last:1,
+ dlen:12,
+ handle:42; /* for tracing */
+ unsigned char data[]
+ __attribute__((aligned(__alignof__(struct nft_expr))));
+};
+
+struct nft_rule_blob {
+ unsigned long size;
+ unsigned char data[]
+ __attribute__((aligned(__alignof__(struct nft_rule_dp))));
+};
+
/**
* struct nft_chain - nf_tables chain
*
@@ -987,8 +1017,8 @@ static inline void nft_set_elem_update_expr(const struct nft_set_ext *ext,
* @name: name of the chain
*/
struct nft_chain {
- struct nft_rule *__rcu *rules_gen_0;
- struct nft_rule *__rcu *rules_gen_1;
+ struct nft_rule_blob __rcu *blob_gen_0;
+ struct nft_rule_blob __rcu *blob_gen_1;
struct list_head rules;
struct list_head list;
struct rhlist_head rhlhead;
@@ -1003,7 +1033,7 @@ struct nft_chain {
u8 *udata;
/* Only used during control plane commit phase: */
- struct nft_rule **rules_next;
+ struct nft_rule_blob *blob_next;
};
int nft_chain_validate(const struct nft_ctx *ctx, const struct nft_chain *chain);
@@ -1321,7 +1351,7 @@ struct nft_traceinfo {
const struct nft_pktinfo *pkt;
const struct nft_base_chain *basechain;
const struct nft_chain *chain;
- const struct nft_rule *rule;
+ const struct nft_rule_dp *rule;
const struct nft_verdict *verdict;
enum nft_trace_types type;
bool packet_dumped;
diff --git a/include/net/netfilter/nf_tables_core.h b/include/net/netfilter/nf_tables_core.h
index 0fa5a6d..b6fb1fd 100644
--- a/include/net/netfilter/nf_tables_core.h
+++ b/include/net/netfilter/nf_tables_core.h
@@ -7,6 +7,7 @@
extern struct nft_expr_type nft_imm_type;
extern struct nft_expr_type nft_cmp_type;
+extern struct nft_expr_type nft_counter_type;
extern struct nft_expr_type nft_lookup_type;
extern struct nft_expr_type nft_bitwise_type;
extern struct nft_expr_type nft_byteorder_type;
@@ -21,6 +22,7 @@ extern struct nft_expr_type nft_last_type;
#ifdef CONFIG_NETWORK_SECMARK
extern struct nft_object_type nft_secmark_obj_type;
#endif
+extern struct nft_object_type nft_counter_obj_type;
int nf_tables_core_module_init(void);
void nf_tables_core_module_exit(void);
@@ -120,6 +122,8 @@ bool nft_pipapo_lookup(const struct net *net, const struct nft_set *set,
bool nft_pipapo_avx2_lookup(const struct net *net, const struct nft_set *set,
const u32 *key, const struct nft_set_ext **ext);
+void nft_counter_init_seqcount(void);
+
struct nft_expr;
struct nft_regs;
struct nft_pktinfo;
@@ -143,4 +147,6 @@ void nft_dynset_eval(const struct nft_expr *expr,
struct nft_regs *regs, const struct nft_pktinfo *pkt);
void nft_rt_get_eval(const struct nft_expr *expr,
struct nft_regs *regs, const struct nft_pktinfo *pkt);
+void nft_counter_eval(const struct nft_expr *expr, struct nft_regs *regs,
+ const struct nft_pktinfo *pkt);
#endif /* _NET_NF_TABLES_CORE_H */
diff --git a/include/net/netns/bpf.h b/include/net/netns/bpf.h
index 0ca6a1b..2c01a27 100644
--- a/include/net/netns/bpf.h
+++ b/include/net/netns/bpf.h
@@ -6,11 +6,18 @@
#ifndef __NETNS_BPF_H__
#define __NETNS_BPF_H__
-#include <linux/bpf-netns.h>
+#include <linux/list.h>
struct bpf_prog;
struct bpf_prog_array;
+enum netns_bpf_attach_type {
+ NETNS_BPF_INVALID = -1,
+ NETNS_BPF_FLOW_DISSECTOR = 0,
+ NETNS_BPF_SK_LOOKUP,
+ MAX_NETNS_BPF_ATTACH_TYPE
+};
+
struct netns_bpf {
/* Array of programs to run compiled from progs or links */
struct bpf_prog_array __rcu *run_array[MAX_NETNS_BPF_ATTACH_TYPE];
diff --git a/include/net/netns/core.h b/include/net/netns/core.h
index 36c2d99..552bc25 100644
--- a/include/net/netns/core.h
+++ b/include/net/netns/core.h
@@ -12,7 +12,6 @@ struct netns_core {
int sysctl_somaxconn;
#ifdef CONFIG_PROC_FS
- int __percpu *sock_inuse;
struct prot_inuse __percpu *prot_inuse;
#endif
};
diff --git a/include/net/netns/ipv4.h b/include/net/netns/ipv4.h
index 6c5b2ef..7855764 100644
--- a/include/net/netns/ipv4.h
+++ b/include/net/netns/ipv4.h
@@ -85,6 +85,9 @@ struct netns_ipv4 {
int sysctl_icmp_ratelimit;
int sysctl_icmp_ratemask;
+ u32 ip_rt_min_pmtu;
+ int ip_rt_mtu_expires;
+
struct local_ports ip_local_ports;
u8 sysctl_tcp_ecn;
diff --git a/include/net/page_pool.h b/include/net/page_pool.h
index a408240..79a8055 100644
--- a/include/net/page_pool.h
+++ b/include/net/page_pool.h
@@ -80,6 +80,8 @@ struct page_pool_params {
enum dma_data_direction dma_dir; /* DMA mapping direction */
unsigned int max_len; /* max DMA sync memory size */
unsigned int offset; /* DMA addr offset */
+ void (*init_callback)(struct page *page, void *arg);
+ void *init_arg;
};
struct page_pool {
@@ -94,6 +96,7 @@ struct page_pool {
unsigned int frag_offset;
struct page *frag_page;
long frag_users;
+ u32 xdp_mem_id;
/*
* Data structure for allocation side
@@ -168,9 +171,12 @@ bool page_pool_return_skb_page(struct page *page);
struct page_pool *page_pool_create(const struct page_pool_params *params);
+struct xdp_mem_info;
+
#ifdef CONFIG_PAGE_POOL
void page_pool_destroy(struct page_pool *pool);
-void page_pool_use_xdp_mem(struct page_pool *pool, void (*disconnect)(void *));
+void page_pool_use_xdp_mem(struct page_pool *pool, void (*disconnect)(void *),
+ struct xdp_mem_info *mem);
void page_pool_release_page(struct page_pool *pool, struct page *page);
void page_pool_put_page_bulk(struct page_pool *pool, void **data,
int count);
@@ -180,7 +186,8 @@ static inline void page_pool_destroy(struct page_pool *pool)
}
static inline void page_pool_use_xdp_mem(struct page_pool *pool,
- void (*disconnect)(void *))
+ void (*disconnect)(void *),
+ struct xdp_mem_info *mem)
{
}
static inline void page_pool_release_page(struct page_pool *pool,
diff --git a/include/net/pkt_cls.h b/include/net/pkt_cls.h
index 193f88e..ebef45e 100644
--- a/include/net/pkt_cls.h
+++ b/include/net/pkt_cls.h
@@ -202,7 +202,8 @@ struct tcf_exts {
__u32 type; /* for backward compat(TCA_OLD_COMPAT) */
int nr_actions;
struct tc_action **actions;
- struct net *net;
+ struct net *net;
+ netns_tracker ns_tracker;
#endif
/* Map to export classifier specific extension TLV types to the
* generic extensions API. Unsupported extensions must be set to 0.
@@ -218,6 +219,7 @@ static inline int tcf_exts_init(struct tcf_exts *exts, struct net *net,
exts->type = 0;
exts->nr_actions = 0;
exts->net = net;
+ netns_tracker_alloc(net, &exts->ns_tracker, GFP_KERNEL);
exts->actions = kcalloc(TCA_ACT_MAX_PRIO, sizeof(struct tc_action *),
GFP_KERNEL);
if (!exts->actions)
@@ -236,6 +238,8 @@ static inline bool tcf_exts_get_net(struct tcf_exts *exts)
{
#ifdef CONFIG_NET_CLS_ACT
exts->net = maybe_get_net(exts->net);
+ if (exts->net)
+ netns_tracker_alloc(exts->net, &exts->ns_tracker, GFP_KERNEL);
return exts->net != NULL;
#else
return true;
@@ -246,7 +250,7 @@ static inline void tcf_exts_put_net(struct tcf_exts *exts)
{
#ifdef CONFIG_NET_CLS_ACT
if (exts->net)
- put_net(exts->net);
+ put_net_track(exts->net, &exts->ns_tracker);
#endif
}
@@ -258,26 +262,31 @@ static inline void tcf_exts_put_net(struct tcf_exts *exts)
for (; 0; (void)(i), (void)(a), (void)(exts))
#endif
+#define tcf_act_for_each_action(i, a, actions) \
+ for (i = 0; i < TCA_ACT_MAX_PRIO && ((a) = actions[i]); i++)
+
static inline void
-tcf_exts_stats_update(const struct tcf_exts *exts,
- u64 bytes, u64 packets, u64 drops, u64 lastuse,
- u8 used_hw_stats, bool used_hw_stats_valid)
+tcf_exts_hw_stats_update(const struct tcf_exts *exts,
+ u64 bytes, u64 packets, u64 drops, u64 lastuse,
+ u8 used_hw_stats, bool used_hw_stats_valid)
{
#ifdef CONFIG_NET_CLS_ACT
int i;
- preempt_disable();
-
for (i = 0; i < exts->nr_actions; i++) {
struct tc_action *a = exts->actions[i];
- tcf_action_stats_update(a, bytes, packets, drops,
- lastuse, true);
- a->used_hw_stats = used_hw_stats;
- a->used_hw_stats_valid = used_hw_stats_valid;
- }
+ /* if stats from hw, just skip */
+ if (tcf_action_update_hw_stats(a)) {
+ preempt_disable();
+ tcf_action_stats_update(a, bytes, packets, drops,
+ lastuse, true);
+ preempt_enable();
- preempt_enable();
+ a->used_hw_stats = used_hw_stats;
+ a->used_hw_stats_valid = used_hw_stats_valid;
+ }
+ }
#endif
}
@@ -321,6 +330,9 @@ int tcf_exts_validate(struct net *net, struct tcf_proto *tp,
struct nlattr **tb, struct nlattr *rate_tlv,
struct tcf_exts *exts, u32 flags,
struct netlink_ext_ack *extack);
+int tcf_exts_validate_ex(struct net *net, struct tcf_proto *tp, struct nlattr **tb,
+ struct nlattr *rate_tlv, struct tcf_exts *exts,
+ u32 flags, u32 fl_flags, struct netlink_ext_ack *extack);
void tcf_exts_destroy(struct tcf_exts *exts);
void tcf_exts_change(struct tcf_exts *dst, struct tcf_exts *src);
int tcf_exts_dump(struct sk_buff *skb, struct tcf_exts *exts);
@@ -532,9 +544,11 @@ tcf_match_indev(struct sk_buff *skb, int ifindex)
return ifindex == skb->skb_iif;
}
-int tc_setup_flow_action(struct flow_action *flow_action,
- const struct tcf_exts *exts);
-void tc_cleanup_flow_action(struct flow_action *flow_action);
+int tc_setup_offload_action(struct flow_action *flow_action,
+ const struct tcf_exts *exts);
+void tc_cleanup_offload_action(struct flow_action *flow_action);
+int tc_setup_action(struct flow_action *flow_action,
+ struct tc_action *actions[]);
int tc_setup_cb_call(struct tcf_block *block, enum tc_setup_type type,
void *type_data, bool err_stop, bool rtnl_held);
diff --git a/include/net/pkt_sched.h b/include/net/pkt_sched.h
index 9e71691..9e7b21c 100644
--- a/include/net/pkt_sched.h
+++ b/include/net/pkt_sched.h
@@ -197,7 +197,9 @@ struct tc_skb_cb {
struct qdisc_skb_cb qdisc_cb;
u16 mru;
- bool post_ct;
+ u8 post_ct:1;
+ u8 post_ct_snat:1;
+ u8 post_ct_dnat:1;
u16 zone; /* Only valid if post_ct = true */
};
diff --git a/include/net/route.h b/include/net/route.h
index 2e6c0e1..4c858dc 100644
--- a/include/net/route.h
+++ b/include/net/route.h
@@ -43,6 +43,7 @@
#define RT_CONN_FLAGS(sk) (RT_TOS(inet_sk(sk)->tos) | sock_flag(sk, SOCK_LOCALROUTE))
#define RT_CONN_FLAGS_TOS(sk,tos) (RT_TOS(tos) | sock_flag(sk, SOCK_LOCALROUTE))
+struct ip_tunnel_info;
struct fib_nh;
struct fib_info;
struct uncached_list;
diff --git a/include/net/sch_generic.h b/include/net/sch_generic.h
index c70e6d2..c11dbac 100644
--- a/include/net/sch_generic.h
+++ b/include/net/sch_generic.h
@@ -125,7 +125,7 @@ struct Qdisc {
spinlock_t seqlock;
struct rcu_head rcu;
-
+ netdevice_tracker dev_tracker;
/* private data */
long privdata[] ____cacheline_aligned;
};
diff --git a/include/net/sctp/sctp.h b/include/net/sctp/sctp.h
index 189fdb9..bf3716f 100644
--- a/include/net/sctp/sctp.h
+++ b/include/net/sctp/sctp.h
@@ -105,19 +105,18 @@ extern struct percpu_counter sctp_sockets_allocated;
int sctp_asconf_mgmt(struct sctp_sock *, struct sctp_sockaddr_entry *);
struct sk_buff *sctp_skb_recv_datagram(struct sock *, int, int, int *);
+typedef int (*sctp_callback_t)(struct sctp_endpoint *, struct sctp_transport *, void *);
void sctp_transport_walk_start(struct rhashtable_iter *iter);
void sctp_transport_walk_stop(struct rhashtable_iter *iter);
struct sctp_transport *sctp_transport_get_next(struct net *net,
struct rhashtable_iter *iter);
struct sctp_transport *sctp_transport_get_idx(struct net *net,
struct rhashtable_iter *iter, int pos);
-int sctp_transport_lookup_process(int (*cb)(struct sctp_transport *, void *),
- struct net *net,
+int sctp_transport_lookup_process(sctp_callback_t cb, struct net *net,
const union sctp_addr *laddr,
const union sctp_addr *paddr, void *p);
-int sctp_for_each_transport(int (*cb)(struct sctp_transport *, void *),
- int (*cb_done)(struct sctp_transport *, void *),
- struct net *net, int *pos, void *p);
+int sctp_transport_traverse_process(sctp_callback_t cb, sctp_callback_t cb_done,
+ struct net *net, int *pos, void *p);
int sctp_for_each_endpoint(int (*cb)(struct sctp_endpoint *, void *), void *p);
int sctp_get_sctp_info(struct sock *sk, struct sctp_association *asoc,
struct sctp_info *info);
@@ -510,8 +509,8 @@ static inline int sctp_ep_hashfn(struct net *net, __u16 lport)
return (net_hash_mix(net) + lport) & (sctp_ep_hashsize - 1);
}
-#define sctp_for_each_hentry(epb, head) \
- hlist_for_each_entry(epb, head, node)
+#define sctp_for_each_hentry(ep, head) \
+ hlist_for_each_entry(ep, head, node)
/* Is a socket of this style? */
#define sctp_style(sk, style) __sctp_style((sk), (SCTP_SOCKET_##style))
diff --git a/include/net/sctp/structs.h b/include/net/sctp/structs.h
index 899c29c..350f250 100644
--- a/include/net/sctp/structs.h
+++ b/include/net/sctp/structs.h
@@ -984,12 +984,10 @@ struct sctp_transport {
} cacc;
struct {
- __u32 last_rtx_chunks;
__u16 pmtu;
__u16 probe_size;
__u16 probe_high;
- __u8 probe_count:3;
- __u8 raise_count:5;
+ __u8 probe_count;
__u8 state;
} pl; /* plpmtud related */
@@ -1011,6 +1009,7 @@ void sctp_transport_reset_t3_rtx(struct sctp_transport *);
void sctp_transport_reset_hb_timer(struct sctp_transport *);
void sctp_transport_reset_reconf_timer(struct sctp_transport *transport);
void sctp_transport_reset_probe_timer(struct sctp_transport *transport);
+void sctp_transport_reset_raise_timer(struct sctp_transport *transport);
int sctp_transport_hold(struct sctp_transport *);
void sctp_transport_put(struct sctp_transport *);
void sctp_transport_update_rto(struct sctp_transport *, __u32);
@@ -1025,7 +1024,7 @@ bool sctp_transport_update_pmtu(struct sctp_transport *t, u32 pmtu);
void sctp_transport_immediate_rtx(struct sctp_transport *);
void sctp_transport_dst_release(struct sctp_transport *t);
void sctp_transport_dst_confirm(struct sctp_transport *t);
-bool sctp_transport_pl_send(struct sctp_transport *t);
+void sctp_transport_pl_send(struct sctp_transport *t);
bool sctp_transport_pl_recv(struct sctp_transport *t);
@@ -1244,10 +1243,6 @@ enum sctp_endpoint_type {
*/
struct sctp_ep_common {
- /* Fields to help us manage our entries in the hash tables. */
- struct hlist_node node;
- int hashent;
-
/* Runtime type information. What kind of endpoint is this? */
enum sctp_endpoint_type type;
@@ -1299,6 +1294,10 @@ struct sctp_endpoint {
/* Common substructure for endpoint and association. */
struct sctp_ep_common base;
+ /* Fields to help us manage our entries in the hash tables. */
+ struct hlist_node node;
+ int hashent;
+
/* Associations: A list of current associations and mappings
* to the data consumers for each association. This
* may be in the form of a hash table or other
@@ -1355,6 +1354,7 @@ struct sctp_endpoint {
reconf_enable:1;
__u8 strreset_enable;
+ struct rcu_head rcu;
};
/* Recover the outter endpoint structure. */
@@ -1370,7 +1370,7 @@ static inline struct sctp_endpoint *sctp_ep(struct sctp_ep_common *base)
struct sctp_endpoint *sctp_endpoint_new(struct sock *, gfp_t);
void sctp_endpoint_free(struct sctp_endpoint *);
void sctp_endpoint_put(struct sctp_endpoint *);
-void sctp_endpoint_hold(struct sctp_endpoint *);
+int sctp_endpoint_hold(struct sctp_endpoint *ep);
void sctp_endpoint_add_asoc(struct sctp_endpoint *, struct sctp_association *);
struct sctp_association *sctp_endpoint_lookup_assoc(
const struct sctp_endpoint *ep,
diff --git a/include/net/seg6.h b/include/net/seg6.h
index 9d19c15..af668f1 100644
--- a/include/net/seg6.h
+++ b/include/net/seg6.h
@@ -58,9 +58,30 @@ extern int seg6_local_init(void);
extern void seg6_local_exit(void);
extern bool seg6_validate_srh(struct ipv6_sr_hdr *srh, int len, bool reduced);
+extern struct ipv6_sr_hdr *seg6_get_srh(struct sk_buff *skb, int flags);
+extern void seg6_icmp_srh(struct sk_buff *skb, struct inet6_skb_parm *opt);
extern int seg6_do_srh_encap(struct sk_buff *skb, struct ipv6_sr_hdr *osrh,
int proto);
extern int seg6_do_srh_inline(struct sk_buff *skb, struct ipv6_sr_hdr *osrh);
extern int seg6_lookup_nexthop(struct sk_buff *skb, struct in6_addr *nhaddr,
u32 tbl_id);
+
+/* If the packet which invoked an ICMP error contains an SRH return
+ * the true destination address from within the SRH, otherwise use the
+ * destination address in the IP header.
+ */
+static inline const struct in6_addr *seg6_get_daddr(struct sk_buff *skb,
+ struct inet6_skb_parm *opt)
+{
+ struct ipv6_sr_hdr *srh;
+
+ if (opt->flags & IP6SKB_SEG6) {
+ srh = (struct ipv6_sr_hdr *)(skb->data + opt->srhoff);
+ return &srh->segments[0];
+ }
+
+ return NULL;
+}
+
+
#endif
diff --git a/include/net/sock.h b/include/net/sock.h
index d47e965..ff9b508 100644
--- a/include/net/sock.h
+++ b/include/net/sock.h
@@ -56,13 +56,13 @@
#include <linux/wait.h>
#include <linux/cgroup-defs.h>
#include <linux/rbtree.h>
-#include <linux/filter.h>
#include <linux/rculist_nulls.h>
#include <linux/poll.h>
#include <linux/sockptr.h>
#include <linux/indirect_call_wrapper.h>
#include <linux/atomic.h>
#include <linux/refcount.h>
+#include <linux/llist.h>
#include <net/dst.h>
#include <net/checksum.h>
#include <net/tcp_states.h>
@@ -248,6 +248,7 @@ struct sock_common {
};
struct bpf_local_storage;
+struct sk_filter;
/**
* struct sock - network layer representation of sockets
@@ -284,15 +285,14 @@ struct bpf_local_storage;
* @sk_no_check_tx: %SO_NO_CHECK setting, set checksum in TX packets
* @sk_no_check_rx: allow zero checksum in RX packets
* @sk_route_caps: route capabilities (e.g. %NETIF_F_TSO)
- * @sk_route_nocaps: forbidden route capabilities (e.g NETIF_F_GSO_MASK)
- * @sk_route_forced_caps: static, forced route capabilities
- * (set in tcp_init_sock())
+ * @sk_gso_disabled: if set, NETIF_F_GSO_MASK is forbidden.
* @sk_gso_type: GSO type (e.g. %SKB_GSO_TCPV4)
* @sk_gso_max_size: Maximum GSO segment size to build
* @sk_gso_max_segs: Maximum number of GSO segments
* @sk_pacing_shift: scaling factor for TCP Small Queues
* @sk_lingertime: %SO_LINGER l_linger setting
* @sk_backlog: always used with the per-socket spinlock held
+ * @defer_list: head of llist storing skbs to be freed
* @sk_callback_lock: used with the callbacks in the end of this struct
* @sk_error_queue: rarely used
* @sk_prot_creator: sk_prot of original sock creator (see ipv6_setsockopt,
@@ -350,6 +350,7 @@ struct bpf_local_storage;
* @sk_txtime_deadline_mode: set deadline mode for SO_TXTIME
* @sk_txtime_report_errors: set report errors mode for SO_TXTIME
* @sk_txtime_unused: unused txtime flags
+ * @ns_tracker: tracker for netns reference
*/
struct sock {
/*
@@ -391,6 +392,11 @@ struct sock {
#define sk_flags __sk_common.skc_flags
#define sk_rxhash __sk_common.skc_rxhash
+ /* early demux fields */
+ struct dst_entry __rcu *sk_rx_dst;
+ int sk_rx_dst_ifindex;
+ u32 sk_rx_dst_cookie;
+
socket_lock_t sk_lock;
atomic_t sk_drops;
int sk_rcvlowat;
@@ -410,6 +416,8 @@ struct sock {
struct sk_buff *head;
struct sk_buff *tail;
} sk_backlog;
+ struct llist_head defer_list;
+
#define sk_rmem_alloc sk_backlog.rmem_alloc
int sk_forward_alloc;
@@ -431,9 +439,6 @@ struct sock {
#ifdef CONFIG_XFRM
struct xfrm_policy __rcu *sk_policy[2];
#endif
- struct dst_entry __rcu *sk_rx_dst;
- int sk_rx_dst_ifindex;
- u32 sk_rx_dst_cookie;
struct dst_entry __rcu *sk_dst_cache;
atomic_t sk_omem_alloc;
@@ -460,8 +465,6 @@ struct sock {
unsigned long sk_max_pacing_rate;
struct page_frag sk_frag;
netdev_features_t sk_route_caps;
- netdev_features_t sk_route_nocaps;
- netdev_features_t sk_route_forced_caps;
int sk_gso_type;
unsigned int sk_gso_max_size;
gfp_t sk_allocation;
@@ -471,7 +474,7 @@ struct sock {
* Because of non atomicity rules, all
* changes are protected by socket lock.
*/
- u8 sk_padding : 1,
+ u8 sk_gso_disabled : 1,
sk_kern_sock : 1,
sk_no_check_tx : 1,
sk_no_check_rx : 1,
@@ -493,6 +496,7 @@ struct sock {
u16 sk_busy_poll_budget;
#endif
spinlock_t sk_peer_lock;
+ int sk_bind_phc;
struct pid *sk_peer_pid;
const struct cred *sk_peer_cred;
@@ -502,7 +506,6 @@ struct sock {
seqlock_t sk_stamp_seq;
#endif
u16 sk_tsflags;
- int sk_bind_phc;
u8 sk_shutdown;
u32 sk_tskey;
atomic_t sk_zckey;
@@ -536,6 +539,7 @@ struct sock {
struct bpf_local_storage __rcu *sk_bpf_storage;
#endif
struct rcu_head sk_rcu;
+ netns_tracker ns_tracker;
};
enum sk_pacing {
@@ -1022,12 +1026,18 @@ static inline __must_check int sk_add_backlog(struct sock *sk, struct sk_buff *s
int __sk_backlog_rcv(struct sock *sk, struct sk_buff *skb);
+INDIRECT_CALLABLE_DECLARE(int tcp_v4_do_rcv(struct sock *sk, struct sk_buff *skb));
+INDIRECT_CALLABLE_DECLARE(int tcp_v6_do_rcv(struct sock *sk, struct sk_buff *skb));
+
static inline int sk_backlog_rcv(struct sock *sk, struct sk_buff *skb)
{
if (sk_memalloc_socks() && skb_pfmemalloc(skb))
return __sk_backlog_rcv(sk, skb);
- return sk->sk_backlog_rcv(sk, skb);
+ return INDIRECT_CALL_INET(sk->sk_backlog_rcv,
+ tcp_v6_do_rcv,
+ tcp_v4_do_rcv,
+ sk, skb);
}
static inline void sk_incoming_cpu_update(struct sock *sk)
@@ -1199,6 +1209,7 @@ struct proto {
void (*unhash)(struct sock *sk);
void (*rehash)(struct sock *sk);
int (*get_port)(struct sock *sk, unsigned short snum);
+ void (*put_port)(struct sock *sk);
#ifdef CONFIG_BPF_SYSCALL
int (*psock_update_sk_prot)(struct sock *sk,
struct sk_psock *psock,
@@ -1210,7 +1221,9 @@ struct proto {
unsigned int inuse_idx;
#endif
+#if IS_ENABLED(CONFIG_MPTCP)
int (*forward_alloc_get)(const struct sock *sk);
+#endif
bool (*stream_memory_free)(const struct sock *sk, int wake);
bool (*sock_is_readable)(struct sock *sk);
@@ -1299,10 +1312,11 @@ INDIRECT_CALLABLE_DECLARE(bool tcp_stream_memory_free(const struct sock *sk, int
static inline int sk_forward_alloc_get(const struct sock *sk)
{
- if (!sk->sk_prot->forward_alloc_get)
- return sk->sk_forward_alloc;
-
- return sk->sk_prot->forward_alloc_get(sk);
+#if IS_ENABLED(CONFIG_MPTCP)
+ if (sk->sk_prot->forward_alloc_get)
+ return sk->sk_prot->forward_alloc_get(sk);
+#endif
+ return sk->sk_forward_alloc;
}
static inline bool __sk_stream_memory_free(const struct sock *sk, int wake)
@@ -1419,13 +1433,32 @@ proto_memory_pressure(struct proto *prot)
#ifdef CONFIG_PROC_FS
-/* Called with local bh disabled */
-void sock_prot_inuse_add(struct net *net, struct proto *prot, int inc);
+#define PROTO_INUSE_NR 64 /* should be enough for the first time */
+struct prot_inuse {
+ int all;
+ int val[PROTO_INUSE_NR];
+};
+
+static inline void sock_prot_inuse_add(const struct net *net,
+ const struct proto *prot, int val)
+{
+ this_cpu_add(net->core.prot_inuse->val[prot->inuse_idx], val);
+}
+
+static inline void sock_inuse_add(const struct net *net, int val)
+{
+ this_cpu_add(net->core.prot_inuse->all, val);
+}
+
int sock_prot_inuse_get(struct net *net, struct proto *proto);
int sock_inuse_get(struct net *net);
#else
-static inline void sock_prot_inuse_add(struct net *net, struct proto *prot,
- int inc)
+static inline void sock_prot_inuse_add(const struct net *net,
+ const struct proto *prot, int val)
+{
+}
+
+static inline void sock_inuse_add(const struct net *net, int val)
{
}
#endif
@@ -1603,16 +1636,6 @@ static inline void sk_mem_uncharge(struct sock *sk, int size)
__sk_mem_reclaim(sk, SK_RECLAIM_CHUNK);
}
-static inline void sock_release_ownership(struct sock *sk)
-{
- if (sk->sk_lock.owned) {
- sk->sk_lock.owned = 0;
-
- /* The sk_lock has mutex_unlock() semantics: */
- mutex_release(&sk->sk_lock.dep_map, _RET_IP_);
- }
-}
-
/*
* Macro so as to not evaluate some arguments when
* lockdep is not enabled.
@@ -1739,12 +1762,23 @@ static inline bool sock_owned_by_user_nocheck(const struct sock *sk)
return sk->sk_lock.owned;
}
+static inline void sock_release_ownership(struct sock *sk)
+{
+ if (sock_owned_by_user_nocheck(sk)) {
+ sk->sk_lock.owned = 0;
+
+ /* The sk_lock has mutex_unlock() semantics: */
+ mutex_release(&sk->sk_lock.dep_map, _RET_IP_);
+ }
+}
+
/* no reclassification while locks are held */
static inline bool sock_allow_reclassification(const struct sock *csk)
{
struct sock *sk = (struct sock *)csk;
- return !sk->sk_lock.owned && !spin_is_locked(&sk->sk_lock.slock);
+ return !sock_owned_by_user_nocheck(sk) &&
+ !spin_is_locked(&sk->sk_lock.slock);
}
struct sock *sk_alloc(struct net *net, int family, gfp_t priority,
@@ -2118,13 +2152,10 @@ static inline void sock_confirm_neigh(struct sk_buff *skb, struct neighbour *n)
{
if (skb_get_dst_pending_confirm(skb)) {
struct sock *sk = skb->sk;
- unsigned long now = jiffies;
- /* avoid dirtying neighbour */
- if (READ_ONCE(n->confirmed) != now)
- WRITE_ONCE(n->confirmed, now);
if (sk && READ_ONCE(sk->sk_dst_pending_confirm))
WRITE_ONCE(sk->sk_dst_pending_confirm, 0);
+ neigh_confirm(n);
}
}
@@ -2137,10 +2168,10 @@ static inline bool sk_can_gso(const struct sock *sk)
void sk_setup_caps(struct sock *sk, struct dst_entry *dst);
-static inline void sk_nocaps_add(struct sock *sk, netdev_features_t flags)
+static inline void sk_gso_disable(struct sock *sk)
{
- sk->sk_route_nocaps |= flags;
- sk->sk_route_caps &= ~flags;
+ sk->sk_gso_disabled = 1;
+ sk->sk_route_caps &= ~NETIF_F_GSO_MASK;
}
static inline int skb_do_copy_data_nocache(struct sock *sk, struct sk_buff *skb,
@@ -2654,6 +2685,11 @@ static inline void skb_setup_tx_timestamp(struct sk_buff *skb, __u16 tsflags)
&skb_shinfo(skb)->tskey);
}
+static inline bool sk_is_tcp(const struct sock *sk)
+{
+ return sk->sk_type == SOCK_STREAM && sk->sk_protocol == IPPROTO_TCP;
+}
+
/**
* sk_eat_skb - Release a skb if it is no longer needed
* @sk: socket to eat this skb from
diff --git a/include/net/tc_act/tc_gate.h b/include/net/tc_act/tc_gate.h
index 8bc6be8..c8fa11e 100644
--- a/include/net/tc_act/tc_gate.h
+++ b/include/net/tc_act/tc_gate.h
@@ -60,11 +60,6 @@ static inline bool is_tcf_gate(const struct tc_action *a)
return false;
}
-static inline u32 tcf_gate_index(const struct tc_action *a)
-{
- return a->tcfa_index;
-}
-
static inline s32 tcf_gate_prio(const struct tc_action *a)
{
s32 tcfg_prio;
diff --git a/include/net/tc_act/tc_mirred.h b/include/net/tc_act/tc_mirred.h
index 1cace4c..32ce8ea 100644
--- a/include/net/tc_act/tc_mirred.h
+++ b/include/net/tc_act/tc_mirred.h
@@ -10,6 +10,7 @@ struct tcf_mirred {
int tcfm_eaction;
bool tcfm_mac_header_xmit;
struct net_device __rcu *tcfm_dev;
+ netdevice_tracker tcfm_dev_tracker;
struct list_head tcfm_list;
};
#define to_mirred(a) ((struct tcf_mirred *)a)
diff --git a/include/net/tcp.h b/include/net/tcp.h
index 4da22b4..44e442b 100644
--- a/include/net/tcp.h
+++ b/include/net/tcp.h
@@ -1368,6 +1368,16 @@ static inline bool tcp_checksum_complete(struct sk_buff *skb)
}
bool tcp_add_backlog(struct sock *sk, struct sk_buff *skb);
+
+void __sk_defer_free_flush(struct sock *sk);
+
+static inline void sk_defer_free_flush(struct sock *sk)
+{
+ if (llist_empty(&sk->defer_list))
+ return;
+ __sk_defer_free_flush(sk);
+}
+
int tcp_filter(struct sock *sk, struct sk_buff *skb);
void tcp_set_state(struct sock *sk, int state);
void tcp_done(struct sock *sk);
@@ -2172,9 +2182,13 @@ static inline void tcp_segs_in(struct tcp_sock *tp, const struct sk_buff *skb)
u16 segs_in;
segs_in = max_t(u16, 1, skb_shinfo(skb)->gso_segs);
- tp->segs_in += segs_in;
+
+ /* We update these fields while other threads might
+ * read them from tcp_get_info()
+ */
+ WRITE_ONCE(tp->segs_in, tp->segs_in + segs_in);
if (skb->len > tcp_hdrlen(skb))
- tp->data_segs_in += segs_in;
+ WRITE_ONCE(tp->data_segs_in, tp->data_segs_in + segs_in);
}
/*
diff --git a/include/net/udp.h b/include/net/udp.h
index 909ecf4..f1c2a88 100644
--- a/include/net/udp.h
+++ b/include/net/udp.h
@@ -167,36 +167,12 @@ static inline void udp_csum_pull_header(struct sk_buff *skb)
typedef struct sock *(*udp_lookup_t)(const struct sk_buff *skb, __be16 sport,
__be16 dport);
-INDIRECT_CALLABLE_DECLARE(struct sk_buff *udp4_gro_receive(struct list_head *,
- struct sk_buff *));
-INDIRECT_CALLABLE_DECLARE(int udp4_gro_complete(struct sk_buff *, int));
-INDIRECT_CALLABLE_DECLARE(struct sk_buff *udp6_gro_receive(struct list_head *,
- struct sk_buff *));
-INDIRECT_CALLABLE_DECLARE(int udp6_gro_complete(struct sk_buff *, int));
INDIRECT_CALLABLE_DECLARE(void udp_v6_early_demux(struct sk_buff *));
INDIRECT_CALLABLE_DECLARE(int udpv6_rcv(struct sk_buff *));
-struct sk_buff *udp_gro_receive(struct list_head *head, struct sk_buff *skb,
- struct udphdr *uh, struct sock *sk);
-int udp_gro_complete(struct sk_buff *skb, int nhoff, udp_lookup_t lookup);
-
struct sk_buff *__udp_gso_segment(struct sk_buff *gso_skb,
netdev_features_t features, bool is_ipv6);
-static inline struct udphdr *udp_gro_udphdr(struct sk_buff *skb)
-{
- struct udphdr *uh;
- unsigned int hlen, off;
-
- off = skb_gro_offset(skb);
- hlen = off + sizeof(*uh);
- uh = skb_gro_header_fast(skb, off);
- if (skb_gro_header_hard(skb, hlen))
- uh = skb_gro_header_slow(skb, hlen, off);
-
- return uh;
-}
-
/* hash routines shared between UDPv4/6 and UDP-Litev4/6 */
static inline int udp_lib_hash(struct sock *sk)
{
diff --git a/include/net/vxlan.h b/include/net/vxlan.h
index 08537aa..5a934be 100644
--- a/include/net/vxlan.h
+++ b/include/net/vxlan.h
@@ -10,6 +10,7 @@
#include <net/nexthop.h>
#define IANA_VXLAN_UDP_PORT 4789
+#define IANA_VXLAN_GPE_UDP_PORT 4790
/* VXLAN protocol (RFC 7348) header:
* +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
diff --git a/include/net/xdp.h b/include/net/xdp.h
index 447f9b1..8f0812e 100644
--- a/include/net/xdp.h
+++ b/include/net/xdp.h
@@ -260,6 +260,9 @@ bool xdp_rxq_info_is_reg(struct xdp_rxq_info *xdp_rxq);
int xdp_rxq_info_reg_mem_model(struct xdp_rxq_info *xdp_rxq,
enum xdp_mem_type type, void *allocator);
void xdp_rxq_info_unreg_mem_model(struct xdp_rxq_info *xdp_rxq);
+int xdp_reg_mem_model(struct xdp_mem_info *mem,
+ enum xdp_mem_type type, void *allocator);
+void xdp_unreg_mem_model(struct xdp_mem_info *mem);
/* Drivers not supporting XDP metadata can use this helper, which
* rejects any room expansion for metadata as a result.
diff --git a/include/net/xdp_priv.h b/include/net/xdp_priv.h
index a9d5b76..a2d58b1 100644
--- a/include/net/xdp_priv.h
+++ b/include/net/xdp_priv.h
@@ -10,7 +10,6 @@ struct xdp_mem_allocator {
union {
void *allocator;
struct page_pool *page_pool;
- struct zero_copy_allocator *zc_alloc;
};
struct rhash_head node;
struct rcu_head rcu;
diff --git a/include/net/xdp_sock.h b/include/net/xdp_sock.h
index fff069d..3057e1a 100644
--- a/include/net/xdp_sock.h
+++ b/include/net/xdp_sock.h
@@ -6,6 +6,7 @@
#ifndef _LINUX_XDP_SOCK_H
#define _LINUX_XDP_SOCK_H
+#include <linux/bpf.h>
#include <linux/workqueue.h>
#include <linux/if_xdp.h>
#include <linux/mutex.h>
diff --git a/include/net/xfrm.h b/include/net/xfrm.h
index 2308210..fdb41e8 100644
--- a/include/net/xfrm.h
+++ b/include/net/xfrm.h
@@ -128,6 +128,7 @@ struct xfrm_state_walk {
struct xfrm_state_offload {
struct net_device *dev;
+ netdevice_tracker dev_tracker;
struct net_device *real_dev;
unsigned long offload_handle;
unsigned int num_exthdrs;
@@ -200,6 +201,11 @@ struct xfrm_state {
struct xfrm_algo_aead *aead;
const char *geniv;
+ /* mapping change rate limiting */
+ __be16 new_mapping_sport;
+ u32 new_mapping; /* seconds */
+ u32 mapping_maxage; /* seconds for input SA */
+
/* Data for encapsulator */
struct xfrm_encap_tmpl *encap;
struct sock __rcu *encap_sk;
@@ -1162,7 +1168,7 @@ static inline int xfrm_route_forward(struct sk_buff *skb, unsigned short family)
{
struct net *net = dev_net(skb->dev);
- if (xfrm_default_allow(net, XFRM_POLICY_FWD))
+ if (xfrm_default_allow(net, XFRM_POLICY_OUT))
return !net->xfrm.policy_count[XFRM_POLICY_OUT] ||
(skb_dst(skb)->flags & DST_NOXFRM) ||
__xfrm_route_forward(skb, family);
@@ -1913,7 +1919,7 @@ static inline void xfrm_dev_state_free(struct xfrm_state *x)
if (dev->xfrmdev_ops->xdo_dev_state_free)
dev->xfrmdev_ops->xdo_dev_state_free(x);
xso->dev = NULL;
- dev_put(dev);
+ dev_put_track(dev, &xso->dev_tracker);
}
}
#else
diff --git a/include/soc/mscc/ocelot.h b/include/soc/mscc/ocelot.h
index fef3a36..5c3a359 100644
--- a/include/soc/mscc/ocelot.h
+++ b/include/soc/mscc/ocelot.h
@@ -118,6 +118,7 @@ enum ocelot_target {
S2,
HSIO,
PTP,
+ FDMA,
GCB,
DEV_GMII,
TARGET_MAX,
@@ -555,12 +556,26 @@ struct ocelot_ops {
u16 (*wm_enc)(u16 value);
u16 (*wm_dec)(u16 value);
void (*wm_stat)(u32 val, u32 *inuse, u32 *maxuse);
+ void (*psfp_init)(struct ocelot *ocelot);
+ int (*psfp_filter_add)(struct ocelot *ocelot, int port,
+ struct flow_cls_offload *f);
+ int (*psfp_filter_del)(struct ocelot *ocelot, struct flow_cls_offload *f);
+ int (*psfp_stats_get)(struct ocelot *ocelot, struct flow_cls_offload *f,
+ struct flow_stats *stats);
+ void (*cut_through_fwd)(struct ocelot *ocelot);
+};
+
+struct ocelot_vcap_policer {
+ struct list_head pol_list;
+ u16 base;
+ u16 max;
+ u16 base2;
+ u16 max2;
};
struct ocelot_vcap_block {
struct list_head rules;
int count;
- int pol_lpr;
};
struct ocelot_bridge_vlan {
@@ -581,6 +596,12 @@ enum ocelot_port_tag_config {
OCELOT_PORT_TAG_TRUNK = 3,
};
+struct ocelot_psfp_list {
+ struct list_head stream_list;
+ struct list_head sfi_list;
+ struct list_head sgi_list;
+};
+
enum ocelot_sb {
OCELOT_SB_BUF,
OCELOT_SB_REF,
@@ -593,6 +614,19 @@ enum ocelot_sb_pool {
OCELOT_SB_POOL_NUM,
};
+/* MAC table entry types.
+ * ENTRYTYPE_NORMAL is subject to aging.
+ * ENTRYTYPE_LOCKED is not subject to aging.
+ * ENTRYTYPE_MACv4 is not subject to aging. For IPv4 multicast.
+ * ENTRYTYPE_MACv6 is not subject to aging. For IPv6 multicast.
+ */
+enum macaccess_entry_type {
+ ENTRYTYPE_NORMAL = 0,
+ ENTRYTYPE_LOCKED,
+ ENTRYTYPE_MACv4,
+ ENTRYTYPE_MACv6,
+};
+
#define OCELOT_QUIRK_PCS_PERFORMS_RATE_ADAPTATION BIT(0)
#define OCELOT_QUIRK_QSGMII_PORTS_MUST_BE_UP BIT(1)
@@ -623,6 +657,8 @@ struct ocelot_port {
struct net_device *bridge;
u8 stp_state;
+
+ int speed;
};
struct ocelot {
@@ -667,8 +703,11 @@ struct ocelot {
struct list_head dummy_rules;
struct ocelot_vcap_block block[3];
+ struct ocelot_vcap_policer vcap_pol;
struct vcap_props *vcap;
+ struct ocelot_psfp_list psfp;
+
/* Workqueue to check statistics for overflow with its lock */
struct mutex stats_lock;
u64 *stats;
@@ -677,6 +716,8 @@ struct ocelot {
/* Lock for serializing access to the MAC table */
struct mutex mact_lock;
+ /* Lock for serializing forwarding domain changes */
+ struct mutex fwd_domain_lock;
struct workqueue_struct *owq;
@@ -692,6 +733,8 @@ struct ocelot {
/* Protects the PTP clock */
spinlock_t ptp_clock_lock;
struct ptp_pin_desc ptp_pins[OCELOT_PTP_PINS_NUM];
+
+ struct ocelot_fdma *fdma;
};
struct ocelot_policer {
@@ -754,8 +797,11 @@ void __ocelot_target_write_ix(struct ocelot *ocelot, enum ocelot_target target,
bool ocelot_can_inject(struct ocelot *ocelot, int grp);
void ocelot_port_inject_frame(struct ocelot *ocelot, int port, int grp,
u32 rew_op, struct sk_buff *skb);
+void ocelot_ifh_port_set(void *ifh, int port, u32 rew_op, u32 vlan_tag);
int ocelot_xtr_poll_frame(struct ocelot *ocelot, int grp, struct sk_buff **skb);
void ocelot_drain_cpu_queue(struct ocelot *ocelot, int grp);
+void ocelot_ptp_rx_timestamp(struct ocelot *ocelot, struct sk_buff *skb,
+ u64 timestamp);
/* Hardware initialization */
int ocelot_regfields_init(struct ocelot *ocelot,
@@ -776,7 +822,9 @@ void ocelot_set_ageing_time(struct ocelot *ocelot, unsigned int msecs);
int ocelot_port_vlan_filtering(struct ocelot *ocelot, int port, bool enabled,
struct netlink_ext_ack *extack);
void ocelot_bridge_stp_state_set(struct ocelot *ocelot, int port, u8 state);
-void ocelot_apply_bridge_fwd_mask(struct ocelot *ocelot);
+u32 ocelot_get_dsa_8021q_cpu_mask(struct ocelot *ocelot);
+u32 ocelot_get_bridge_fwd_mask(struct ocelot *ocelot, int src_port);
+void ocelot_apply_bridge_fwd_mask(struct ocelot *ocelot, bool joining);
int ocelot_port_pre_bridge_flags(struct ocelot *ocelot, int port,
struct switchdev_brport_flags val);
void ocelot_port_bridge_flags(struct ocelot *ocelot, int port,
@@ -785,6 +833,7 @@ void ocelot_port_bridge_join(struct ocelot *ocelot, int port,
struct net_device *bridge);
void ocelot_port_bridge_leave(struct ocelot *ocelot, int port,
struct net_device *bridge);
+int ocelot_mact_flush(struct ocelot *ocelot, int port);
int ocelot_fdb_dump(struct ocelot *ocelot, int port,
dsa_fdb_dump_cb_t *cb, void *data);
int ocelot_fdb_add(struct ocelot *ocelot, int port,
@@ -870,6 +919,19 @@ void ocelot_phylink_mac_link_up(struct ocelot *ocelot, int port,
bool tx_pause, bool rx_pause,
unsigned long quirks);
+int ocelot_mact_lookup(struct ocelot *ocelot, int *dst_idx,
+ const unsigned char mac[ETH_ALEN],
+ unsigned int vid, enum macaccess_entry_type *type);
+int ocelot_mact_learn_streamdata(struct ocelot *ocelot, int dst_idx,
+ const unsigned char mac[ETH_ALEN],
+ unsigned int vid,
+ enum macaccess_entry_type type,
+ int sfid, int ssid);
+
+int ocelot_vcap_policer_add(struct ocelot *ocelot, u32 pol_ix,
+ struct ocelot_policer *pol);
+int ocelot_vcap_policer_del(struct ocelot *ocelot, u32 pol_ix);
+
#if IS_ENABLED(CONFIG_BRIDGE_MRP)
int ocelot_mrp_add(struct ocelot *ocelot, int port,
const struct switchdev_obj_mrp *mrp);
diff --git a/include/soc/mscc/ocelot_ana.h b/include/soc/mscc/ocelot_ana.h
index 1669481..67e0ae0 100644
--- a/include/soc/mscc/ocelot_ana.h
+++ b/include/soc/mscc/ocelot_ana.h
@@ -227,6 +227,11 @@
#define ANA_TABLES_SFIDACCESS_SFID_TBL_CMD(x) ((x) & GENMASK(1, 0))
#define ANA_TABLES_SFIDACCESS_SFID_TBL_CMD_M GENMASK(1, 0)
+#define SFIDACCESS_CMD_IDLE 0
+#define SFIDACCESS_CMD_READ 1
+#define SFIDACCESS_CMD_WRITE 2
+#define SFIDACCESS_CMD_INIT 3
+
#define ANA_TABLES_SFIDTIDX_SGID_VALID BIT(26)
#define ANA_TABLES_SFIDTIDX_SGID(x) (((x) << 18) & GENMASK(25, 18))
#define ANA_TABLES_SFIDTIDX_SGID_M GENMASK(25, 18)
@@ -255,6 +260,11 @@
#define ANA_SG_CONFIG_REG_3_INIT_IPS(x) (((x) << 21) & GENMASK(24, 21))
#define ANA_SG_CONFIG_REG_3_INIT_IPS_M GENMASK(24, 21)
#define ANA_SG_CONFIG_REG_3_INIT_IPS_X(x) (((x) & GENMASK(24, 21)) >> 21)
+#define ANA_SG_CONFIG_REG_3_IPV_VALID BIT(24)
+#define ANA_SG_CONFIG_REG_3_IPV_INVALID(x) (((x) << 24) & GENMASK(24, 24))
+#define ANA_SG_CONFIG_REG_3_INIT_IPV(x) (((x) << 21) & GENMASK(23, 21))
+#define ANA_SG_CONFIG_REG_3_INIT_IPV_M GENMASK(23, 21)
+#define ANA_SG_CONFIG_REG_3_INIT_IPV_X(x) (((x) & GENMASK(23, 21)) >> 21)
#define ANA_SG_CONFIG_REG_3_INIT_GATE_STATE BIT(25)
#define ANA_SG_GCL_GS_CONFIG_RSZ 0x4
diff --git a/include/soc/mscc/ocelot_vcap.h b/include/soc/mscc/ocelot_vcap.h
index 4d1dfa1..709cbc1 100644
--- a/include/soc/mscc/ocelot_vcap.h
+++ b/include/soc/mscc/ocelot_vcap.h
@@ -656,6 +656,7 @@ enum ocelot_vcap_filter_type {
OCELOT_VCAP_FILTER_DUMMY,
OCELOT_VCAP_FILTER_PAG,
OCELOT_VCAP_FILTER_OFFLOAD,
+ OCELOT_PSFP_FILTER_OFFLOAD,
};
struct ocelot_vcap_id {
diff --git a/include/soc/mscc/vsc7514_regs.h b/include/soc/mscc/vsc7514_regs.h
new file mode 100644
index 0000000..ceee26c9
--- /dev/null
+++ b/include/soc/mscc/vsc7514_regs.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: (GPL-2.0 OR MIT) */
+/*
+ * Microsemi Ocelot Switch driver
+ *
+ * Copyright (c) 2021 Innovative Advantage Inc.
+ */
+
+#ifndef VSC7514_REGS_H
+#define VSC7514_REGS_H
+
+#include <soc/mscc/ocelot_vcap.h>
+
+extern const u32 vsc7514_ana_regmap[];
+extern const u32 vsc7514_qs_regmap[];
+extern const u32 vsc7514_qsys_regmap[];
+extern const u32 vsc7514_rew_regmap[];
+extern const u32 vsc7514_sys_regmap[];
+extern const u32 vsc7514_vcap_regmap[];
+extern const u32 vsc7514_ptp_regmap[];
+extern const u32 vsc7514_dev_gmii_regmap[];
+
+extern const struct vcap_field vsc7514_vcap_es0_keys[];
+extern const struct vcap_field vsc7514_vcap_es0_actions[];
+extern const struct vcap_field vsc7514_vcap_is1_keys[];
+extern const struct vcap_field vsc7514_vcap_is1_actions[];
+extern const struct vcap_field vsc7514_vcap_is2_keys[];
+extern const struct vcap_field vsc7514_vcap_is2_actions[];
+
+#endif
diff --git a/include/soc/tegra/common.h b/include/soc/tegra/common.h
index af41ad8..8ec1ac0 100644
--- a/include/soc/tegra/common.h
+++ b/include/soc/tegra/common.h
@@ -39,4 +39,19 @@ devm_tegra_core_dev_init_opp_table(struct device *dev,
}
#endif
+static inline int
+devm_tegra_core_dev_init_opp_table_common(struct device *dev)
+{
+ struct tegra_core_opp_params opp_params = {};
+ int err;
+
+ opp_params.init_state = true;
+
+ err = devm_tegra_core_dev_init_opp_table(dev, &opp_params);
+ if (err != -ENODEV)
+ return err;
+
+ return 0;
+}
+
#endif /* __SOC_TEGRA_COMMON_H__ */
diff --git a/include/trace/events/skb.h b/include/trace/events/skb.h
index 9e92f22..3e042ca 100644
--- a/include/trace/events/skb.h
+++ b/include/trace/events/skb.h
@@ -9,29 +9,56 @@
#include <linux/netdevice.h>
#include <linux/tracepoint.h>
+#define TRACE_SKB_DROP_REASON \
+ EM(SKB_DROP_REASON_NOT_SPECIFIED, NOT_SPECIFIED) \
+ EM(SKB_DROP_REASON_NO_SOCKET, NO_SOCKET) \
+ EM(SKB_DROP_REASON_PKT_TOO_SMALL, PKT_TOO_SMALL) \
+ EM(SKB_DROP_REASON_TCP_CSUM, TCP_CSUM) \
+ EM(SKB_DROP_REASON_TCP_FILTER, TCP_FILTER) \
+ EM(SKB_DROP_REASON_UDP_CSUM, UDP_CSUM) \
+ EMe(SKB_DROP_REASON_MAX, MAX)
+
+#undef EM
+#undef EMe
+
+#define EM(a, b) TRACE_DEFINE_ENUM(a);
+#define EMe(a, b) TRACE_DEFINE_ENUM(a);
+
+TRACE_SKB_DROP_REASON
+
+#undef EM
+#undef EMe
+#define EM(a, b) { a, #b },
+#define EMe(a, b) { a, #b }
+
/*
* Tracepoint for free an sk_buff:
*/
TRACE_EVENT(kfree_skb,
- TP_PROTO(struct sk_buff *skb, void *location),
+ TP_PROTO(struct sk_buff *skb, void *location,
+ enum skb_drop_reason reason),
- TP_ARGS(skb, location),
+ TP_ARGS(skb, location, reason),
TP_STRUCT__entry(
- __field( void *, skbaddr )
- __field( void *, location )
- __field( unsigned short, protocol )
+ __field(void *, skbaddr)
+ __field(void *, location)
+ __field(unsigned short, protocol)
+ __field(enum skb_drop_reason, reason)
),
TP_fast_assign(
__entry->skbaddr = skb;
__entry->location = location;
__entry->protocol = ntohs(skb->protocol);
+ __entry->reason = reason;
),
- TP_printk("skbaddr=%p protocol=%u location=%p",
- __entry->skbaddr, __entry->protocol, __entry->location)
+ TP_printk("skbaddr=%p protocol=%u location=%p reason: %s",
+ __entry->skbaddr, __entry->protocol, __entry->location,
+ __print_symbolic(__entry->reason,
+ TRACE_SKB_DROP_REASON))
);
TRACE_EVENT(consume_skb,
diff --git a/include/trace/events/vmscan.h b/include/trace/events/vmscan.h
index f25a614..ca2e900 100644
--- a/include/trace/events/vmscan.h
+++ b/include/trace/events/vmscan.h
@@ -30,12 +30,14 @@
#define _VMSCAN_THROTTLE_WRITEBACK (1 << VMSCAN_THROTTLE_WRITEBACK)
#define _VMSCAN_THROTTLE_ISOLATED (1 << VMSCAN_THROTTLE_ISOLATED)
#define _VMSCAN_THROTTLE_NOPROGRESS (1 << VMSCAN_THROTTLE_NOPROGRESS)
+#define _VMSCAN_THROTTLE_CONGESTED (1 << VMSCAN_THROTTLE_CONGESTED)
#define show_throttle_flags(flags) \
(flags) ? __print_flags(flags, "|", \
{_VMSCAN_THROTTLE_WRITEBACK, "VMSCAN_THROTTLE_WRITEBACK"}, \
{_VMSCAN_THROTTLE_ISOLATED, "VMSCAN_THROTTLE_ISOLATED"}, \
- {_VMSCAN_THROTTLE_NOPROGRESS, "VMSCAN_THROTTLE_NOPROGRESS"} \
+ {_VMSCAN_THROTTLE_NOPROGRESS, "VMSCAN_THROTTLE_NOPROGRESS"}, \
+ {_VMSCAN_THROTTLE_CONGESTED, "VMSCAN_THROTTLE_CONGESTED"} \
) : "VMSCAN_THROTTLE_NONE"
diff --git a/include/uapi/drm/amdgpu_drm.h b/include/uapi/drm/amdgpu_drm.h
index 26e45fc..0b94ec7 100644
--- a/include/uapi/drm/amdgpu_drm.h
+++ b/include/uapi/drm/amdgpu_drm.h
@@ -80,7 +80,7 @@ extern "C" {
*
* %AMDGPU_GEM_DOMAIN_GTT GPU accessible system memory, mapped into the
* GPU's virtual address space via gart. Gart memory linearizes non-contiguous
- * pages of system memory, allows GPU access system memory in a linezrized
+ * pages of system memory, allows GPU access system memory in a linearized
* fashion.
*
* %AMDGPU_GEM_DOMAIN_VRAM Local video memory. For APUs, it is memory
diff --git a/include/uapi/drm/drm.h b/include/uapi/drm/drm.h
index 3b810b5..6428085 100644
--- a/include/uapi/drm/drm.h
+++ b/include/uapi/drm/drm.h
@@ -1096,6 +1096,24 @@ extern "C" {
#define DRM_IOCTL_SYNCOBJ_TRANSFER DRM_IOWR(0xCC, struct drm_syncobj_transfer)
#define DRM_IOCTL_SYNCOBJ_TIMELINE_SIGNAL DRM_IOWR(0xCD, struct drm_syncobj_timeline_array)
+/**
+ * DRM_IOCTL_MODE_GETFB2 - Get framebuffer metadata.
+ *
+ * This queries metadata about a framebuffer. User-space fills
+ * &drm_mode_fb_cmd2.fb_id as the input, and the kernels fills the rest of the
+ * struct as the output.
+ *
+ * If the client is DRM master or has &CAP_SYS_ADMIN, &drm_mode_fb_cmd2.handles
+ * will be filled with GEM buffer handles. Planes are valid until one has a
+ * zero handle -- this can be used to compute the number of planes.
+ *
+ * Otherwise, &drm_mode_fb_cmd2.handles will be zeroed and planes are valid
+ * until one has a zero &drm_mode_fb_cmd2.pitches.
+ *
+ * If the framebuffer has a format modifier, &DRM_MODE_FB_MODIFIERS will be set
+ * in &drm_mode_fb_cmd2.flags and &drm_mode_fb_cmd2.modifier will contain the
+ * modifier. Otherwise, user-space must ignore &drm_mode_fb_cmd2.modifier.
+ */
#define DRM_IOCTL_MODE_GETFB2 DRM_IOWR(0xCE, struct drm_mode_fb_cmd2)
/*
diff --git a/include/uapi/drm/drm_fourcc.h b/include/uapi/drm/drm_fourcc.h
index 7f652c9..fc0c145 100644
--- a/include/uapi/drm/drm_fourcc.h
+++ b/include/uapi/drm/drm_fourcc.h
@@ -314,6 +314,13 @@ extern "C" {
*/
#define DRM_FORMAT_P016 fourcc_code('P', '0', '1', '6') /* 2x2 subsampled Cr:Cb plane 16 bits per channel */
+/* 2 plane YCbCr420.
+ * 3 10 bit components and 2 padding bits packed into 4 bytes.
+ * index 0 = Y plane, [31:0] x:Y2:Y1:Y0 2:10:10:10 little endian
+ * index 1 = Cr:Cb plane, [63:0] x:Cr2:Cb2:Cr1:x:Cb1:Cr0:Cb0 [2:10:10:10:2:10:10:10] little endian
+ */
+#define DRM_FORMAT_P030 fourcc_code('P', '0', '3', '0') /* 2x2 subsampled Cr:Cb plane 10 bits per channel packed */
+
/* 3 plane non-subsampled (444) YCbCr
* 16 bits per component, but only 10 bits are used and 6 bits are padded
* index 0: Y plane, [15:0] Y:x [10:6] little endian
@@ -854,6 +861,10 @@ drm_fourcc_canonicalize_nvidia_format_mod(__u64 modifier)
* and UV. Some SAND-using hardware stores UV in a separate tiled
* image from Y to reduce the column height, which is not supported
* with these modifiers.
+ *
+ * The DRM_FORMAT_MOD_BROADCOM_SAND128_COL_HEIGHT modifier is also
+ * supported for DRM_FORMAT_P030 where the columns remain as 128 bytes
+ * wide, but as this is a 10 bpp format that translates to 96 pixels.
*/
#define DRM_FORMAT_MOD_BROADCOM_SAND32_COL_HEIGHT(v) \
diff --git a/include/uapi/drm/vmwgfx_drm.h b/include/uapi/drm/vmwgfx_drm.h
index 9078775..8277644 100644
--- a/include/uapi/drm/vmwgfx_drm.h
+++ b/include/uapi/drm/vmwgfx_drm.h
@@ -110,6 +110,7 @@ extern "C" {
#define DRM_VMW_PARAM_HW_CAPS2 13
#define DRM_VMW_PARAM_SM4_1 14
#define DRM_VMW_PARAM_SM5 15
+#define DRM_VMW_PARAM_GL43 16
/**
* enum drm_vmw_handle_type - handle type for ref ioctls
diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index ba5af15..b0383d3 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -1342,8 +1342,10 @@ union bpf_attr {
/* or valid module BTF object fd or 0 to attach to vmlinux */
__u32 attach_btf_obj_fd;
};
- __u32 :32; /* pad */
+ __u32 core_relo_cnt; /* number of bpf_core_relo */
__aligned_u64 fd_array; /* array of FDs */
+ __aligned_u64 core_relos;
+ __u32 core_relo_rec_size; /* sizeof(struct bpf_core_relo) */
};
struct { /* anonymous struct used by BPF_OBJ_* commands */
@@ -1744,7 +1746,7 @@ union bpf_attr {
* if the maximum number of tail calls has been reached for this
* chain of programs. This limit is defined in the kernel by the
* macro **MAX_TAIL_CALL_CNT** (not accessible to user space),
- * which is currently set to 32.
+ * which is currently set to 33.
* Return
* 0 on success, or a negative error in case of failure.
*
@@ -4938,6 +4940,84 @@ union bpf_attr {
* **-ENOENT** if symbol is not found.
*
* **-EPERM** if caller does not have permission to obtain kernel address.
+ *
+ * long bpf_find_vma(struct task_struct *task, u64 addr, void *callback_fn, void *callback_ctx, u64 flags)
+ * Description
+ * Find vma of *task* that contains *addr*, call *callback_fn*
+ * function with *task*, *vma*, and *callback_ctx*.
+ * The *callback_fn* should be a static function and
+ * the *callback_ctx* should be a pointer to the stack.
+ * The *flags* is used to control certain aspects of the helper.
+ * Currently, the *flags* must be 0.
+ *
+ * The expected callback signature is
+ *
+ * long (\*callback_fn)(struct task_struct \*task, struct vm_area_struct \*vma, void \*callback_ctx);
+ *
+ * Return
+ * 0 on success.
+ * **-ENOENT** if *task->mm* is NULL, or no vma contains *addr*.
+ * **-EBUSY** if failed to try lock mmap_lock.
+ * **-EINVAL** for invalid **flags**.
+ *
+ * long bpf_loop(u32 nr_loops, void *callback_fn, void *callback_ctx, u64 flags)
+ * Description
+ * For **nr_loops**, call **callback_fn** function
+ * with **callback_ctx** as the context parameter.
+ * The **callback_fn** should be a static function and
+ * the **callback_ctx** should be a pointer to the stack.
+ * The **flags** is used to control certain aspects of the helper.
+ * Currently, the **flags** must be 0. Currently, nr_loops is
+ * limited to 1 << 23 (~8 million) loops.
+ *
+ * long (\*callback_fn)(u32 index, void \*ctx);
+ *
+ * where **index** is the current index in the loop. The index
+ * is zero-indexed.
+ *
+ * If **callback_fn** returns 0, the helper will continue to the next
+ * loop. If return value is 1, the helper will skip the rest of
+ * the loops and return. Other return values are not used now,
+ * and will be rejected by the verifier.
+ *
+ * Return
+ * The number of loops performed, **-EINVAL** for invalid **flags**,
+ * **-E2BIG** if **nr_loops** exceeds the maximum number of loops.
+ *
+ * long bpf_strncmp(const char *s1, u32 s1_sz, const char *s2)
+ * Description
+ * Do strncmp() between **s1** and **s2**. **s1** doesn't need
+ * to be null-terminated and **s1_sz** is the maximum storage
+ * size of **s1**. **s2** must be a read-only string.
+ * Return
+ * An integer less than, equal to, or greater than zero
+ * if the first **s1_sz** bytes of **s1** is found to be
+ * less than, to match, or be greater than **s2**.
+ *
+ * long bpf_get_func_arg(void *ctx, u32 n, u64 *value)
+ * Description
+ * Get **n**-th argument (zero based) of the traced function (for tracing programs)
+ * returned in **value**.
+ *
+ * Return
+ * 0 on success.
+ * **-EINVAL** if n >= arguments count of traced function.
+ *
+ * long bpf_get_func_ret(void *ctx, u64 *value)
+ * Description
+ * Get return value of the traced function (for tracing programs)
+ * in **value**.
+ *
+ * Return
+ * 0 on success.
+ * **-EOPNOTSUPP** for tracing programs other than BPF_TRACE_FEXIT or BPF_MODIFY_RETURN.
+ *
+ * long bpf_get_func_arg_cnt(void *ctx)
+ * Description
+ * Get number of arguments of the traced function (for tracing programs).
+ *
+ * Return
+ * The number of arguments of the traced function.
*/
#define __BPF_FUNC_MAPPER(FN) \
FN(unspec), \
@@ -5120,6 +5200,12 @@ union bpf_attr {
FN(trace_vprintk), \
FN(skc_to_unix_sock), \
FN(kallsyms_lookup_name), \
+ FN(find_vma), \
+ FN(loop), \
+ FN(strncmp), \
+ FN(get_func_arg), \
+ FN(get_func_ret), \
+ FN(get_func_arg_cnt), \
/* */
/* integer value in 'imm' field of BPF_CALL instruction selects which helper
@@ -6296,6 +6382,7 @@ struct bpf_sk_lookup {
__u32 local_ip4; /* Network byte order */
__u32 local_ip6[4]; /* Network byte order */
__u32 local_port; /* Host byte order */
+ __u32 ingress_ifindex; /* The arriving interface. Determined by inet_iif. */
};
/*
@@ -6328,4 +6415,78 @@ enum {
BTF_F_ZERO = (1ULL << 3),
};
+/* bpf_core_relo_kind encodes which aspect of captured field/type/enum value
+ * has to be adjusted by relocations. It is emitted by llvm and passed to
+ * libbpf and later to the kernel.
+ */
+enum bpf_core_relo_kind {
+ BPF_CORE_FIELD_BYTE_OFFSET = 0, /* field byte offset */
+ BPF_CORE_FIELD_BYTE_SIZE = 1, /* field size in bytes */
+ BPF_CORE_FIELD_EXISTS = 2, /* field existence in target kernel */
+ BPF_CORE_FIELD_SIGNED = 3, /* field signedness (0 - unsigned, 1 - signed) */
+ BPF_CORE_FIELD_LSHIFT_U64 = 4, /* bitfield-specific left bitshift */
+ BPF_CORE_FIELD_RSHIFT_U64 = 5, /* bitfield-specific right bitshift */
+ BPF_CORE_TYPE_ID_LOCAL = 6, /* type ID in local BPF object */
+ BPF_CORE_TYPE_ID_TARGET = 7, /* type ID in target kernel */
+ BPF_CORE_TYPE_EXISTS = 8, /* type existence in target kernel */
+ BPF_CORE_TYPE_SIZE = 9, /* type size in bytes */
+ BPF_CORE_ENUMVAL_EXISTS = 10, /* enum value existence in target kernel */
+ BPF_CORE_ENUMVAL_VALUE = 11, /* enum value integer value */
+};
+
+/*
+ * "struct bpf_core_relo" is used to pass relocation data form LLVM to libbpf
+ * and from libbpf to the kernel.
+ *
+ * CO-RE relocation captures the following data:
+ * - insn_off - instruction offset (in bytes) within a BPF program that needs
+ * its insn->imm field to be relocated with actual field info;
+ * - type_id - BTF type ID of the "root" (containing) entity of a relocatable
+ * type or field;
+ * - access_str_off - offset into corresponding .BTF string section. String
+ * interpretation depends on specific relocation kind:
+ * - for field-based relocations, string encodes an accessed field using
+ * a sequence of field and array indices, separated by colon (:). It's
+ * conceptually very close to LLVM's getelementptr ([0]) instruction's
+ * arguments for identifying offset to a field.
+ * - for type-based relocations, strings is expected to be just "0";
+ * - for enum value-based relocations, string contains an index of enum
+ * value within its enum type;
+ * - kind - one of enum bpf_core_relo_kind;
+ *
+ * Example:
+ * struct sample {
+ * int a;
+ * struct {
+ * int b[10];
+ * };
+ * };
+ *
+ * struct sample *s = ...;
+ * int *x = &s->a; // encoded as "0:0" (a is field #0)
+ * int *y = &s->b[5]; // encoded as "0:1:0:5" (anon struct is field #1,
+ * // b is field #0 inside anon struct, accessing elem #5)
+ * int *z = &s[10]->b; // encoded as "10:1" (ptr is used as an array)
+ *
+ * type_id for all relocs in this example will capture BTF type id of
+ * `struct sample`.
+ *
+ * Such relocation is emitted when using __builtin_preserve_access_index()
+ * Clang built-in, passing expression that captures field address, e.g.:
+ *
+ * bpf_probe_read(&dst, sizeof(dst),
+ * __builtin_preserve_access_index(&src->a.b.c));
+ *
+ * In this case Clang will emit field relocation recording necessary data to
+ * be able to find offset of embedded `a.b.c` field within `src` struct.
+ *
+ * [0] https://llvm.org/docs/LangRef.html#getelementptr-instruction
+ */
+struct bpf_core_relo {
+ __u32 insn_off;
+ __u32 type_id;
+ __u32 access_str_off;
+ enum bpf_core_relo_kind kind;
+};
+
#endif /* _UAPI__LINUX_BPF_H__ */
diff --git a/include/uapi/linux/btf.h b/include/uapi/linux/btf.h
index deb12f7..b0d8fea 100644
--- a/include/uapi/linux/btf.h
+++ b/include/uapi/linux/btf.h
@@ -43,7 +43,7 @@ struct btf_type {
* "size" tells the size of the type it is describing.
*
* "type" is used by PTR, TYPEDEF, VOLATILE, CONST, RESTRICT,
- * FUNC, FUNC_PROTO, VAR and DECL_TAG.
+ * FUNC, FUNC_PROTO, VAR, DECL_TAG and TYPE_TAG.
* "type" is a type_id referring to another type.
*/
union {
@@ -75,6 +75,7 @@ enum {
BTF_KIND_DATASEC = 15, /* Section */
BTF_KIND_FLOAT = 16, /* Floating point */
BTF_KIND_DECL_TAG = 17, /* Decl Tag */
+ BTF_KIND_TYPE_TAG = 18, /* Type Tag */
NR_BTF_KINDS,
BTF_KIND_MAX = NR_BTF_KINDS - 1,
diff --git a/include/uapi/linux/can/netlink.h b/include/uapi/linux/can/netlink.h
index 75b85c6..02ec32d 100644
--- a/include/uapi/linux/can/netlink.h
+++ b/include/uapi/linux/can/netlink.h
@@ -137,6 +137,7 @@ enum {
IFLA_CAN_DATA_BITRATE_CONST,
IFLA_CAN_BITRATE_MAX,
IFLA_CAN_TDC,
+ IFLA_CAN_CTRLMODE_EXT,
/* add new constants above here */
__IFLA_CAN_MAX,
@@ -166,6 +167,18 @@ enum {
IFLA_CAN_TDC_MAX = __IFLA_CAN_TDC - 1
};
+/*
+ * IFLA_CAN_CTRLMODE_EXT nest: controller mode extended parameters
+ */
+enum {
+ IFLA_CAN_CTRLMODE_UNSPEC,
+ IFLA_CAN_CTRLMODE_SUPPORTED, /* u32 */
+
+ /* add new constants above here */
+ __IFLA_CAN_CTRLMODE,
+ IFLA_CAN_CTRLMODE_MAX = __IFLA_CAN_CTRLMODE - 1
+};
+
/* u16 termination range: 1..65535 Ohms */
#define CAN_TERMINATION_DISABLED 0
diff --git a/include/uapi/linux/ethtool.h b/include/uapi/linux/ethtool.h
index a2223b6..7bc4b8d 100644
--- a/include/uapi/linux/ethtool.h
+++ b/include/uapi/linux/ethtool.h
@@ -231,6 +231,7 @@ enum tunable_id {
ETHTOOL_RX_COPYBREAK,
ETHTOOL_TX_COPYBREAK,
ETHTOOL_PFC_PREVENTION_TOUT, /* timeout in msecs */
+ ETHTOOL_TX_COPYBREAK_BUF_SIZE,
/*
* Add your fresh new tunable attribute above and remember to update
* tunable_strings[] in net/ethtool/common.c
diff --git a/include/uapi/linux/ethtool_netlink.h b/include/uapi/linux/ethtool_netlink.h
index 999777d..cca6e47 100644
--- a/include/uapi/linux/ethtool_netlink.h
+++ b/include/uapi/linux/ethtool_netlink.h
@@ -329,6 +329,7 @@ enum {
ETHTOOL_A_RINGS_RX_MINI, /* u32 */
ETHTOOL_A_RINGS_RX_JUMBO, /* u32 */
ETHTOOL_A_RINGS_TX, /* u32 */
+ ETHTOOL_A_RINGS_RX_BUF_LEN, /* u32 */
/* add new constants above here */
__ETHTOOL_A_RINGS_CNT,
diff --git a/include/uapi/linux/if_link.h b/include/uapi/linux/if_link.h
index eebd389..6218f93 100644
--- a/include/uapi/linux/if_link.h
+++ b/include/uapi/linux/if_link.h
@@ -347,6 +347,7 @@ enum {
*/
IFLA_PARENT_DEV_NAME,
IFLA_PARENT_DEV_BUS_NAME,
+ IFLA_GRO_MAX_SIZE,
__IFLA_MAX
};
@@ -858,6 +859,7 @@ enum {
IFLA_BOND_TLB_DYNAMIC_LB,
IFLA_BOND_PEER_NOTIF_DELAY,
IFLA_BOND_AD_LACP_ACTIVE,
+ IFLA_BOND_MISSED_MAX,
__IFLA_BOND_MAX,
};
diff --git a/include/uapi/linux/kfd_sysfs.h b/include/uapi/linux/kfd_sysfs.h
new file mode 100644
index 0000000..e1fb78b
--- /dev/null
+++ b/include/uapi/linux/kfd_sysfs.h
@@ -0,0 +1,108 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT WITH Linux-syscall-note */
+/*
+ * Copyright 2021 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+
+#ifndef KFD_SYSFS_H_INCLUDED
+#define KFD_SYSFS_H_INCLUDED
+
+/* Capability bits in node properties */
+#define HSA_CAP_HOT_PLUGGABLE 0x00000001
+#define HSA_CAP_ATS_PRESENT 0x00000002
+#define HSA_CAP_SHARED_WITH_GRAPHICS 0x00000004
+#define HSA_CAP_QUEUE_SIZE_POW2 0x00000008
+#define HSA_CAP_QUEUE_SIZE_32BIT 0x00000010
+#define HSA_CAP_QUEUE_IDLE_EVENT 0x00000020
+#define HSA_CAP_VA_LIMIT 0x00000040
+#define HSA_CAP_WATCH_POINTS_SUPPORTED 0x00000080
+#define HSA_CAP_WATCH_POINTS_TOTALBITS_MASK 0x00000f00
+#define HSA_CAP_WATCH_POINTS_TOTALBITS_SHIFT 8
+#define HSA_CAP_DOORBELL_TYPE_TOTALBITS_MASK 0x00003000
+#define HSA_CAP_DOORBELL_TYPE_TOTALBITS_SHIFT 12
+
+#define HSA_CAP_DOORBELL_TYPE_PRE_1_0 0x0
+#define HSA_CAP_DOORBELL_TYPE_1_0 0x1
+#define HSA_CAP_DOORBELL_TYPE_2_0 0x2
+#define HSA_CAP_AQL_QUEUE_DOUBLE_MAP 0x00004000
+
+/* Old buggy user mode depends on this being 0 */
+#define HSA_CAP_RESERVED_WAS_SRAM_EDCSUPPORTED 0x00080000
+
+#define HSA_CAP_MEM_EDCSUPPORTED 0x00100000
+#define HSA_CAP_RASEVENTNOTIFY 0x00200000
+#define HSA_CAP_ASIC_REVISION_MASK 0x03c00000
+#define HSA_CAP_ASIC_REVISION_SHIFT 22
+#define HSA_CAP_SRAM_EDCSUPPORTED 0x04000000
+#define HSA_CAP_SVMAPI_SUPPORTED 0x08000000
+#define HSA_CAP_FLAGS_COHERENTHOSTACCESS 0x10000000
+#define HSA_CAP_RESERVED 0xe00f8000
+
+/* Heap types in memory properties */
+#define HSA_MEM_HEAP_TYPE_SYSTEM 0
+#define HSA_MEM_HEAP_TYPE_FB_PUBLIC 1
+#define HSA_MEM_HEAP_TYPE_FB_PRIVATE 2
+#define HSA_MEM_HEAP_TYPE_GPU_GDS 3
+#define HSA_MEM_HEAP_TYPE_GPU_LDS 4
+#define HSA_MEM_HEAP_TYPE_GPU_SCRATCH 5
+
+/* Flag bits in memory properties */
+#define HSA_MEM_FLAGS_HOT_PLUGGABLE 0x00000001
+#define HSA_MEM_FLAGS_NON_VOLATILE 0x00000002
+#define HSA_MEM_FLAGS_RESERVED 0xfffffffc
+
+/* Cache types in cache properties */
+#define HSA_CACHE_TYPE_DATA 0x00000001
+#define HSA_CACHE_TYPE_INSTRUCTION 0x00000002
+#define HSA_CACHE_TYPE_CPU 0x00000004
+#define HSA_CACHE_TYPE_HSACU 0x00000008
+#define HSA_CACHE_TYPE_RESERVED 0xfffffff0
+
+/* Link types in IO link properties (matches CRAT link types) */
+#define HSA_IOLINK_TYPE_UNDEFINED 0
+#define HSA_IOLINK_TYPE_HYPERTRANSPORT 1
+#define HSA_IOLINK_TYPE_PCIEXPRESS 2
+#define HSA_IOLINK_TYPE_AMBA 3
+#define HSA_IOLINK_TYPE_MIPI 4
+#define HSA_IOLINK_TYPE_QPI_1_1 5
+#define HSA_IOLINK_TYPE_RESERVED1 6
+#define HSA_IOLINK_TYPE_RESERVED2 7
+#define HSA_IOLINK_TYPE_RAPID_IO 8
+#define HSA_IOLINK_TYPE_INFINIBAND 9
+#define HSA_IOLINK_TYPE_RESERVED3 10
+#define HSA_IOLINK_TYPE_XGMI 11
+#define HSA_IOLINK_TYPE_XGOP 12
+#define HSA_IOLINK_TYPE_GZ 13
+#define HSA_IOLINK_TYPE_ETHERNET_RDMA 14
+#define HSA_IOLINK_TYPE_RDMA_OTHER 15
+#define HSA_IOLINK_TYPE_OTHER 16
+
+/* Flag bits in IO link properties (matches CRAT flags, excluding the
+ * bi-directional flag, which is not offially part of the CRAT spec, and
+ * only used internally in KFD)
+ */
+#define HSA_IOLINK_FLAGS_ENABLED (1 << 0)
+#define HSA_IOLINK_FLAGS_NON_COHERENT (1 << 1)
+#define HSA_IOLINK_FLAGS_NO_ATOMICS_32_BIT (1 << 2)
+#define HSA_IOLINK_FLAGS_NO_ATOMICS_64_BIT (1 << 3)
+#define HSA_IOLINK_FLAGS_NO_PEER_TO_PEER_DMA (1 << 4)
+#define HSA_IOLINK_FLAGS_RESERVED 0xffffffe0
+
+#endif
diff --git a/include/uapi/linux/net_tstamp.h b/include/uapi/linux/net_tstamp.h
index fcc61c7..55501e5 100644
--- a/include/uapi/linux/net_tstamp.h
+++ b/include/uapi/linux/net_tstamp.h
@@ -62,7 +62,7 @@ struct so_timestamping {
/**
* struct hwtstamp_config - %SIOCGHWTSTAMP and %SIOCSHWTSTAMP parameter
*
- * @flags: no flags defined right now, must be zero for %SIOCSHWTSTAMP
+ * @flags: one of HWTSTAMP_FLAG_*
* @tx_type: one of HWTSTAMP_TX_*
* @rx_filter: one of HWTSTAMP_FILTER_*
*
@@ -78,6 +78,21 @@ struct hwtstamp_config {
int rx_filter;
};
+/* possible values for hwtstamp_config->flags */
+enum hwtstamp_flags {
+ /*
+ * With this flag, the user could get bond active interface's
+ * PHC index. Note this PHC index is not stable as when there
+ * is a failover, the bond active interface will be changed, so
+ * will be the PHC index.
+ */
+ HWTSTAMP_FLAG_BONDED_PHC_INDEX = (1<<0),
+#define HWTSTAMP_FLAG_BONDED_PHC_INDEX HWTSTAMP_FLAG_BONDED_PHC_INDEX
+
+ HWTSTAMP_FLAG_LAST = HWTSTAMP_FLAG_BONDED_PHC_INDEX,
+ HWTSTAMP_FLAG_MASK = (HWTSTAMP_FLAG_LAST - 1) | HWTSTAMP_FLAG_LAST
+};
+
/* possible values for hwtstamp_config->tx_type */
enum hwtstamp_tx_types {
/*
diff --git a/include/uapi/linux/nfc.h b/include/uapi/linux/nfc.h
index f6e3c8c..4fa4e97 100644
--- a/include/uapi/linux/nfc.h
+++ b/include/uapi/linux/nfc.h
@@ -263,7 +263,7 @@ enum nfc_sdp_attr {
#define NFC_SE_ENABLED 0x1
struct sockaddr_nfc {
- sa_family_t sa_family;
+ __kernel_sa_family_t sa_family;
__u32 dev_idx;
__u32 target_idx;
__u32 nfc_protocol;
@@ -271,14 +271,14 @@ struct sockaddr_nfc {
#define NFC_LLCP_MAX_SERVICE_NAME 63
struct sockaddr_nfc_llcp {
- sa_family_t sa_family;
+ __kernel_sa_family_t sa_family;
__u32 dev_idx;
__u32 target_idx;
__u32 nfc_protocol;
__u8 dsap; /* Destination SAP, if known */
__u8 ssap; /* Source SAP to be bound to */
char service_name[NFC_LLCP_MAX_SERVICE_NAME]; /* Service name URI */;
- size_t service_name_len;
+ __kernel_size_t service_name_len;
};
/* NFC socket protocols */
diff --git a/include/uapi/linux/nl80211.h b/include/uapi/linux/nl80211.h
index 61cab81..195a238 100644
--- a/include/uapi/linux/nl80211.h
+++ b/include/uapi/linux/nl80211.h
@@ -1232,6 +1232,11 @@
* &NL80211_ATTR_FILS_NONCES - for FILS Nonces
* (STA Nonce 16 bytes followed by AP Nonce 16 bytes)
*
+ * @NL80211_CMD_ASSOC_COMEBACK: notification about an association
+ * temporal rejection with comeback. The event includes %NL80211_ATTR_MAC
+ * to describe the BSSID address of the AP and %NL80211_ATTR_TIMEOUT to
+ * specify the timeout value.
+ *
* @NL80211_CMD_MAX: highest used command number
* @__NL80211_CMD_AFTER_LAST: internal use
*/
@@ -1474,6 +1479,8 @@ enum nl80211_commands {
NL80211_CMD_SET_FILS_AAD,
+ NL80211_CMD_ASSOC_COMEBACK,
+
/* add new commands above here */
/* used to define NL80211_CMD_MAX below */
@@ -2470,7 +2477,9 @@ enum nl80211_commands {
* space supports external authentication. This attribute shall be used
* with %NL80211_CMD_CONNECT and %NL80211_CMD_START_AP request. The driver
* may offload authentication processing to user space if this capability
- * is indicated in the respective requests from the user space.
+ * is indicated in the respective requests from the user space. (This flag
+ * attribute deprecated for %NL80211_CMD_START_AP, use
+ * %NL80211_ATTR_AP_SETTINGS_FLAGS)
*
* @NL80211_ATTR_NSS: Station's New/updated RX_NSS value notified using this
* u8 attribute. This is used with %NL80211_CMD_STA_OPMODE_CHANGED.
@@ -2639,6 +2648,17 @@ enum nl80211_commands {
* Mandatory parameter for the transmitting interface to enable MBSSID.
* Optional for the non-transmitting interfaces.
*
+ * @NL80211_ATTR_RADAR_BACKGROUND: Configure dedicated offchannel chain
+ * available for radar/CAC detection on some hw. This chain can't be used
+ * to transmit or receive frames and it is bounded to a running wdev.
+ * Background radar/CAC detection allows to avoid the CAC downtime
+ * switching on a different channel during CAC detection on the selected
+ * radar channel.
+ *
+ * @NL80211_ATTR_AP_SETTINGS_FLAGS: u32 attribute contains ap settings flags,
+ * enumerated in &enum nl80211_ap_settings_flags. This attribute shall be
+ * used with %NL80211_CMD_START_AP request.
+ *
* @NUM_NL80211_ATTR: total number of nl80211_attrs available
* @NL80211_ATTR_MAX: highest attribute number currently defined
* @__NL80211_ATTR_AFTER_LAST: internal use
@@ -3145,6 +3165,10 @@ enum nl80211_attrs {
NL80211_ATTR_MBSSID_CONFIG,
NL80211_ATTR_MBSSID_ELEMS,
+ NL80211_ATTR_RADAR_BACKGROUND,
+
+ NL80211_ATTR_AP_SETTINGS_FLAGS,
+
/* add attributes here, update the policy in nl80211.c */
__NL80211_ATTR_AFTER_LAST,
@@ -3738,13 +3762,12 @@ enum nl80211_mpath_info {
* capabilities IE
* @NL80211_BAND_IFTYPE_ATTR_HE_CAP_PPE: HE PPE thresholds information as
* defined in HE capabilities IE
- * @NL80211_BAND_IFTYPE_ATTR_MAX: highest band HE capability attribute currently
- * defined
* @NL80211_BAND_IFTYPE_ATTR_HE_6GHZ_CAPA: HE 6GHz band capabilities (__le16),
* given for all 6 GHz band channels
* @NL80211_BAND_IFTYPE_ATTR_VENDOR_ELEMS: vendor element capabilities that are
* advertised on this band/for this iftype (binary)
* @__NL80211_BAND_IFTYPE_ATTR_AFTER_LAST: internal use
+ * @NL80211_BAND_IFTYPE_ATTR_MAX: highest band attribute currently defined
*/
enum nl80211_band_iftype_attr {
__NL80211_BAND_IFTYPE_ATTR_INVALID,
@@ -5555,7 +5578,7 @@ enum nl80211_iface_limit_attrs {
* => allows 8 of AP/GO that can have BI gcd >= min gcd
*
* numbers = [ #{STA} <= 2 ], channels = 2, max = 2
- * => allows two STAs on different channels
+ * => allows two STAs on the same or on different channels
*
* numbers = [ #{STA} <= 1, #{P2P-client,P2P-GO} <= 3 ], max = 4
* => allows a STA plus three P2P interfaces
@@ -5600,7 +5623,7 @@ enum nl80211_if_combination_attrs {
* @NL80211_PLINK_ESTAB: mesh peer link is established
* @NL80211_PLINK_HOLDING: mesh peer link is being closed or cancelled
* @NL80211_PLINK_BLOCKED: all frames transmitted from this mesh
- * plink are discarded
+ * plink are discarded, except for authentication frames
* @NUM_NL80211_PLINK_STATES: number of peer link states
* @MAX_NL80211_PLINK_STATES: highest numerical value of plink states
*/
@@ -5737,13 +5760,15 @@ enum nl80211_tdls_operation {
NL80211_TDLS_DISABLE_LINK,
};
-/*
+/**
* enum nl80211_ap_sme_features - device-integrated AP features
- * Reserved for future use, no bits are defined in
- * NL80211_ATTR_DEVICE_AP_SME yet.
-enum nl80211_ap_sme_features {
-};
+ * @NL80211_AP_SME_SA_QUERY_OFFLOAD: SA Query procedures offloaded to driver
+ * when user space indicates support for SA Query procedures offload during
+ * "start ap" with %NL80211_AP_SETTINGS_SA_QUERY_OFFLOAD_SUPPORT.
*/
+enum nl80211_ap_sme_features {
+ NL80211_AP_SME_SA_QUERY_OFFLOAD = 1 << 0,
+};
/**
* enum nl80211_feature_flags - device/driver features
@@ -6051,6 +6076,9 @@ enum nl80211_feature_flags {
* frames. Userspace has to share FILS AAD details to the driver by using
* @NL80211_CMD_SET_FILS_AAD.
*
+ * @NL80211_EXT_FEATURE_RADAR_BACKGROUND: Device supports background radar/CAC
+ * detection.
+ *
* @NUM_NL80211_EXT_FEATURES: number of extended features.
* @MAX_NL80211_EXT_FEATURES: highest extended feature index.
*/
@@ -6117,6 +6145,7 @@ enum nl80211_ext_feature_index {
NL80211_EXT_FEATURE_PROT_RANGE_NEGO_AND_MEASURE,
NL80211_EXT_FEATURE_BSS_COLOR,
NL80211_EXT_FEATURE_FILS_CRYPTO_OFFLOAD,
+ NL80211_EXT_FEATURE_RADAR_BACKGROUND,
/* add new features before the definition below */
NUM_NL80211_EXT_FEATURES,
@@ -7462,4 +7491,20 @@ enum nl80211_mbssid_config_attributes {
NL80211_MBSSID_CONFIG_ATTR_MAX = __NL80211_MBSSID_CONFIG_ATTR_LAST - 1,
};
+/**
+ * enum nl80211_ap_settings_flags - AP settings flags
+ *
+ * @NL80211_AP_SETTINGS_EXTERNAL_AUTH_SUPPORT: AP supports external
+ * authentication.
+ * @NL80211_AP_SETTINGS_SA_QUERY_OFFLOAD_SUPPORT: Userspace supports SA Query
+ * procedures offload to driver. If driver advertises
+ * %NL80211_AP_SME_SA_QUERY_OFFLOAD in AP SME features, userspace shall
+ * ignore SA Query procedures and validations when this flag is set by
+ * userspace.
+ */
+enum nl80211_ap_settings_flags {
+ NL80211_AP_SETTINGS_EXTERNAL_AUTH_SUPPORT = 1 << 0,
+ NL80211_AP_SETTINGS_SA_QUERY_OFFLOAD_SUPPORT = 1 << 1,
+};
+
#endif /* __LINUX_NL80211_H */
diff --git a/include/uapi/linux/pfkeyv2.h b/include/uapi/linux/pfkeyv2.h
index d65b117..8abae1f 100644
--- a/include/uapi/linux/pfkeyv2.h
+++ b/include/uapi/linux/pfkeyv2.h
@@ -309,6 +309,7 @@ struct sadb_x_filter {
#define SADB_X_AALG_SHA2_512HMAC 7
#define SADB_X_AALG_RIPEMD160HMAC 8
#define SADB_X_AALG_AES_XCBC_MAC 9
+#define SADB_X_AALG_SM3_256HMAC 10
#define SADB_X_AALG_NULL 251 /* kame */
#define SADB_AALG_MAX 251
@@ -329,6 +330,7 @@ struct sadb_x_filter {
#define SADB_X_EALG_AES_GCM_ICV16 20
#define SADB_X_EALG_CAMELLIACBC 22
#define SADB_X_EALG_NULL_AES_GMAC 23
+#define SADB_X_EALG_SM4CBC 24
#define SADB_EALG_MAX 253 /* last EALG */
/* private allocations should use 249-255 (RFC2407) */
#define SADB_X_EALG_SERPENTCBC 252 /* draft-ietf-ipsec-ciph-aes-cbc-00 */
diff --git a/include/uapi/linux/pkt_cls.h b/include/uapi/linux/pkt_cls.h
index 6836ccb..ee38b35 100644
--- a/include/uapi/linux/pkt_cls.h
+++ b/include/uapi/linux/pkt_cls.h
@@ -19,13 +19,16 @@ enum {
TCA_ACT_FLAGS,
TCA_ACT_HW_STATS,
TCA_ACT_USED_HW_STATS,
+ TCA_ACT_IN_HW_COUNT,
__TCA_ACT_MAX
};
/* See other TCA_ACT_FLAGS_ * flags in include/net/act_api.h. */
-#define TCA_ACT_FLAGS_NO_PERCPU_STATS 1 /* Don't use percpu allocator for
- * actions stats.
- */
+#define TCA_ACT_FLAGS_NO_PERCPU_STATS (1 << 0) /* Don't use percpu allocator for
+ * actions stats.
+ */
+#define TCA_ACT_FLAGS_SKIP_HW (1 << 1) /* don't offload action to HW */
+#define TCA_ACT_FLAGS_SKIP_SW (1 << 2) /* don't use action in SW */
/* tca HW stats type
* When user does not pass the attribute, he does not care.
diff --git a/include/uapi/linux/rtnetlink.h b/include/uapi/linux/rtnetlink.h
index 5888492..93d934c 100644
--- a/include/uapi/linux/rtnetlink.h
+++ b/include/uapi/linux/rtnetlink.h
@@ -754,6 +754,8 @@ enum rtnetlink_groups {
#define RTNLGRP_NEXTHOP RTNLGRP_NEXTHOP
RTNLGRP_BRVLAN,
#define RTNLGRP_BRVLAN RTNLGRP_BRVLAN
+ RTNLGRP_MCTP_IFADDR,
+#define RTNLGRP_MCTP_IFADDR RTNLGRP_MCTP_IFADDR
__RTNLGRP_MAX
};
#define RTNLGRP_MAX (__RTNLGRP_MAX - 1)
diff --git a/include/uapi/linux/smc.h b/include/uapi/linux/smc.h
index 20f33b2..6c2874f 100644
--- a/include/uapi/linux/smc.h
+++ b/include/uapi/linux/smc.h
@@ -119,6 +119,8 @@ enum {
SMC_NLA_LGR_R_CONNS_NUM, /* u32 */
SMC_NLA_LGR_R_V2_COMMON, /* nest */
SMC_NLA_LGR_R_V2, /* nest */
+ SMC_NLA_LGR_R_NET_COOKIE, /* u64 */
+ SMC_NLA_LGR_R_PAD, /* flag */
__SMC_NLA_LGR_R_MAX,
SMC_NLA_LGR_R_MAX = __SMC_NLA_LGR_R_MAX - 1
};
diff --git a/include/uapi/linux/smc_diag.h b/include/uapi/linux/smc_diag.h
index 8cb3a6f..c7008d8 100644
--- a/include/uapi/linux/smc_diag.h
+++ b/include/uapi/linux/smc_diag.h
@@ -84,11 +84,12 @@ struct smc_diag_conninfo {
/* SMC_DIAG_LINKINFO */
struct smc_diag_linkinfo {
- __u8 link_id; /* link identifier */
- __u8 ibname[IB_DEVICE_NAME_MAX]; /* name of the RDMA device */
- __u8 ibport; /* RDMA device port number */
- __u8 gid[40]; /* local GID */
- __u8 peer_gid[40]; /* peer GID */
+ __u8 link_id; /* link identifier */
+ __u8 ibname[IB_DEVICE_NAME_MAX]; /* name of the RDMA device */
+ __u8 ibport; /* RDMA device port number */
+ __u8 gid[40]; /* local GID */
+ __u8 peer_gid[40]; /* peer GID */
+ __aligned_u64 net_cookie; /* RDMA device net namespace */
};
struct smc_diag_lgrinfo {
diff --git a/include/uapi/linux/tty.h b/include/uapi/linux/tty.h
index 376cccf3..a58deb3 100644
--- a/include/uapi/linux/tty.h
+++ b/include/uapi/linux/tty.h
@@ -38,5 +38,6 @@
#define N_NCI 25 /* NFC NCI UART */
#define N_SPEAKUP 26 /* Speakup communication with synths */
#define N_NULL 27 /* Null ldisc used for error handling */
+#define N_MCTP 28 /* MCTP-over-serial */
#endif /* _UAPI_LINUX_TTY_H */
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index 5fea5fe..c8e0f84 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -128,6 +128,7 @@ enum v4l2_colorfx {
V4L2_COLORFX_SOLARIZATION = 13,
V4L2_COLORFX_ANTIQUE = 14,
V4L2_COLORFX_SET_CBCR = 15,
+ V4L2_COLORFX_SET_RGB = 16,
};
#define V4L2_CID_AUTOBRIGHTNESS (V4L2_CID_BASE+32)
#define V4L2_CID_BAND_STOP_FILTER (V4L2_CID_BASE+33)
@@ -145,9 +146,10 @@ enum v4l2_colorfx {
#define V4L2_CID_ALPHA_COMPONENT (V4L2_CID_BASE+41)
#define V4L2_CID_COLORFX_CBCR (V4L2_CID_BASE+42)
+#define V4L2_CID_COLORFX_RGB (V4L2_CID_BASE+43)
/* last CID + 1 */
-#define V4L2_CID_LASTP1 (V4L2_CID_BASE+43)
+#define V4L2_CID_LASTP1 (V4L2_CID_BASE+44)
/* USER-class private control IDs */
@@ -2016,6 +2018,290 @@ struct v4l2_ctrl_hdr10_mastering_display {
__u32 min_display_mastering_luminance;
};
+/* Stateless VP9 controls */
+
+#define V4L2_VP9_LOOP_FILTER_FLAG_DELTA_ENABLED 0x1
+#define V4L2_VP9_LOOP_FILTER_FLAG_DELTA_UPDATE 0x2
+
+/**
+ * struct v4l2_vp9_loop_filter - VP9 loop filter parameters
+ *
+ * @ref_deltas: contains the adjustment needed for the filter level based on the
+ * chosen reference frame. If this syntax element is not present in the bitstream,
+ * users should pass its last value.
+ * @mode_deltas: contains the adjustment needed for the filter level based on the
+ * chosen mode. If this syntax element is not present in the bitstream, users should
+ * pass its last value.
+ * @level: indicates the loop filter strength.
+ * @sharpness: indicates the sharpness level.
+ * @flags: combination of V4L2_VP9_LOOP_FILTER_FLAG_{} flags.
+ * @reserved: padding field. Should be zeroed by applications.
+ *
+ * This structure contains all loop filter related parameters. See sections
+ * '7.2.8 Loop filter semantics' of the VP9 specification for more details.
+ */
+struct v4l2_vp9_loop_filter {
+ __s8 ref_deltas[4];
+ __s8 mode_deltas[2];
+ __u8 level;
+ __u8 sharpness;
+ __u8 flags;
+ __u8 reserved[7];
+};
+
+/**
+ * struct v4l2_vp9_quantization - VP9 quantization parameters
+ *
+ * @base_q_idx: indicates the base frame qindex.
+ * @delta_q_y_dc: indicates the Y DC quantizer relative to base_q_idx.
+ * @delta_q_uv_dc: indicates the UV DC quantizer relative to base_q_idx.
+ * @delta_q_uv_ac: indicates the UV AC quantizer relative to base_q_idx.
+ * @reserved: padding field. Should be zeroed by applications.
+ *
+ * Encodes the quantization parameters. See section '7.2.9 Quantization params
+ * syntax' of the VP9 specification for more details.
+ */
+struct v4l2_vp9_quantization {
+ __u8 base_q_idx;
+ __s8 delta_q_y_dc;
+ __s8 delta_q_uv_dc;
+ __s8 delta_q_uv_ac;
+ __u8 reserved[4];
+};
+
+#define V4L2_VP9_SEGMENTATION_FLAG_ENABLED 0x01
+#define V4L2_VP9_SEGMENTATION_FLAG_UPDATE_MAP 0x02
+#define V4L2_VP9_SEGMENTATION_FLAG_TEMPORAL_UPDATE 0x04
+#define V4L2_VP9_SEGMENTATION_FLAG_UPDATE_DATA 0x08
+#define V4L2_VP9_SEGMENTATION_FLAG_ABS_OR_DELTA_UPDATE 0x10
+
+#define V4L2_VP9_SEG_LVL_ALT_Q 0
+#define V4L2_VP9_SEG_LVL_ALT_L 1
+#define V4L2_VP9_SEG_LVL_REF_FRAME 2
+#define V4L2_VP9_SEG_LVL_SKIP 3
+#define V4L2_VP9_SEG_LVL_MAX 4
+
+#define V4L2_VP9_SEGMENT_FEATURE_ENABLED(id) (1 << (id))
+#define V4L2_VP9_SEGMENT_FEATURE_ENABLED_MASK 0xf
+
+/**
+ * struct v4l2_vp9_segmentation - VP9 segmentation parameters
+ *
+ * @feature_data: data attached to each feature. Data entry is only valid if
+ * the feature is enabled. The array shall be indexed with segment number as
+ * the first dimension (0..7) and one of V4L2_VP9_SEG_{} as the second dimension.
+ * @feature_enabled: bitmask defining which features are enabled in each segment.
+ * The value for each segment is a combination of V4L2_VP9_SEGMENT_FEATURE_ENABLED(id)
+ * values where id is one of V4L2_VP9_SEG_LVL_{}.
+ * @tree_probs: specifies the probability values to be used when decoding a
+ * Segment-ID. See '5.15. Segmentation map' section of the VP9 specification
+ * for more details.
+ * @pred_probs: specifies the probability values to be used when decoding a
+ * Predicted-Segment-ID. See '6.4.14. Get segment id syntax' section of :ref:`vp9`
+ * for more details.
+ * @flags: combination of V4L2_VP9_SEGMENTATION_FLAG_{} flags.
+ * @reserved: padding field. Should be zeroed by applications.
+ *
+ * Encodes the quantization parameters. See section '7.2.10 Segmentation params syntax' of
+ * the VP9 specification for more details.
+ */
+struct v4l2_vp9_segmentation {
+ __s16 feature_data[8][4];
+ __u8 feature_enabled[8];
+ __u8 tree_probs[7];
+ __u8 pred_probs[3];
+ __u8 flags;
+ __u8 reserved[5];
+};
+
+#define V4L2_VP9_FRAME_FLAG_KEY_FRAME 0x001
+#define V4L2_VP9_FRAME_FLAG_SHOW_FRAME 0x002
+#define V4L2_VP9_FRAME_FLAG_ERROR_RESILIENT 0x004
+#define V4L2_VP9_FRAME_FLAG_INTRA_ONLY 0x008
+#define V4L2_VP9_FRAME_FLAG_ALLOW_HIGH_PREC_MV 0x010
+#define V4L2_VP9_FRAME_FLAG_REFRESH_FRAME_CTX 0x020
+#define V4L2_VP9_FRAME_FLAG_PARALLEL_DEC_MODE 0x040
+#define V4L2_VP9_FRAME_FLAG_X_SUBSAMPLING 0x080
+#define V4L2_VP9_FRAME_FLAG_Y_SUBSAMPLING 0x100
+#define V4L2_VP9_FRAME_FLAG_COLOR_RANGE_FULL_SWING 0x200
+
+#define V4L2_VP9_SIGN_BIAS_LAST 0x1
+#define V4L2_VP9_SIGN_BIAS_GOLDEN 0x2
+#define V4L2_VP9_SIGN_BIAS_ALT 0x4
+
+#define V4L2_VP9_RESET_FRAME_CTX_NONE 0
+#define V4L2_VP9_RESET_FRAME_CTX_SPEC 1
+#define V4L2_VP9_RESET_FRAME_CTX_ALL 2
+
+#define V4L2_VP9_INTERP_FILTER_EIGHTTAP 0
+#define V4L2_VP9_INTERP_FILTER_EIGHTTAP_SMOOTH 1
+#define V4L2_VP9_INTERP_FILTER_EIGHTTAP_SHARP 2
+#define V4L2_VP9_INTERP_FILTER_BILINEAR 3
+#define V4L2_VP9_INTERP_FILTER_SWITCHABLE 4
+
+#define V4L2_VP9_REFERENCE_MODE_SINGLE_REFERENCE 0
+#define V4L2_VP9_REFERENCE_MODE_COMPOUND_REFERENCE 1
+#define V4L2_VP9_REFERENCE_MODE_SELECT 2
+
+#define V4L2_VP9_PROFILE_MAX 3
+
+#define V4L2_CID_STATELESS_VP9_FRAME (V4L2_CID_CODEC_STATELESS_BASE + 300)
+/**
+ * struct v4l2_ctrl_vp9_frame - VP9 frame decoding control
+ *
+ * @lf: loop filter parameters. See &v4l2_vp9_loop_filter for more details.
+ * @quant: quantization parameters. See &v4l2_vp9_quantization for more details.
+ * @seg: segmentation parameters. See &v4l2_vp9_segmentation for more details.
+ * @flags: combination of V4L2_VP9_FRAME_FLAG_{} flags.
+ * @compressed_header_size: compressed header size in bytes.
+ * @uncompressed_header_size: uncompressed header size in bytes.
+ * @frame_width_minus_1: add 1 to it and you'll get the frame width expressed in pixels.
+ * @frame_height_minus_1: add 1 to it and you'll get the frame height expressed in pixels.
+ * @render_width_minus_1: add 1 to it and you'll get the expected render width expressed in
+ * pixels. This is not used during the decoding process but might be used by HW scalers
+ * to prepare a frame that's ready for scanout.
+ * @render_height_minus_1: add 1 to it and you'll get the expected render height expressed in
+ * pixels. This is not used during the decoding process but might be used by HW scalers
+ * to prepare a frame that's ready for scanout.
+ * @last_frame_ts: "last" reference buffer timestamp.
+ * The timestamp refers to the timestamp field in struct v4l2_buffer.
+ * Use v4l2_timeval_to_ns() to convert the struct timeval to a __u64.
+ * @golden_frame_ts: "golden" reference buffer timestamp.
+ * The timestamp refers to the timestamp field in struct v4l2_buffer.
+ * Use v4l2_timeval_to_ns() to convert the struct timeval to a __u64.
+ * @alt_frame_ts: "alt" reference buffer timestamp.
+ * The timestamp refers to the timestamp field in struct v4l2_buffer.
+ * Use v4l2_timeval_to_ns() to convert the struct timeval to a __u64.
+ * @ref_frame_sign_bias: a bitfield specifying whether the sign bias is set for a given
+ * reference frame. Either of V4L2_VP9_SIGN_BIAS_{}.
+ * @reset_frame_context: specifies whether the frame context should be reset to default values.
+ * Either of V4L2_VP9_RESET_FRAME_CTX_{}.
+ * @frame_context_idx: frame context that should be used/updated.
+ * @profile: VP9 profile. Can be 0, 1, 2 or 3.
+ * @bit_depth: bits per components. Can be 8, 10 or 12. Note that not all profiles support
+ * 10 and/or 12 bits depths.
+ * @interpolation_filter: specifies the filter selection used for performing inter prediction.
+ * Set to one of V4L2_VP9_INTERP_FILTER_{}.
+ * @tile_cols_log2: specifies the base 2 logarithm of the width of each tile (where the width
+ * is measured in units of 8x8 blocks). Shall be less than or equal to 6.
+ * @tile_rows_log2: specifies the base 2 logarithm of the height of each tile (where the height
+ * is measured in units of 8x8 blocks).
+ * @reference_mode: specifies the type of inter prediction to be used.
+ * Set to one of V4L2_VP9_REFERENCE_MODE_{}.
+ * @reserved: padding field. Should be zeroed by applications.
+ */
+struct v4l2_ctrl_vp9_frame {
+ struct v4l2_vp9_loop_filter lf;
+ struct v4l2_vp9_quantization quant;
+ struct v4l2_vp9_segmentation seg;
+ __u32 flags;
+ __u16 compressed_header_size;
+ __u16 uncompressed_header_size;
+ __u16 frame_width_minus_1;
+ __u16 frame_height_minus_1;
+ __u16 render_width_minus_1;
+ __u16 render_height_minus_1;
+ __u64 last_frame_ts;
+ __u64 golden_frame_ts;
+ __u64 alt_frame_ts;
+ __u8 ref_frame_sign_bias;
+ __u8 reset_frame_context;
+ __u8 frame_context_idx;
+ __u8 profile;
+ __u8 bit_depth;
+ __u8 interpolation_filter;
+ __u8 tile_cols_log2;
+ __u8 tile_rows_log2;
+ __u8 reference_mode;
+ __u8 reserved[7];
+};
+
+#define V4L2_VP9_NUM_FRAME_CTX 4
+
+/**
+ * struct v4l2_vp9_mv_probs - VP9 Motion vector probability updates
+ * @joint: motion vector joint probability updates.
+ * @sign: motion vector sign probability updates.
+ * @classes: motion vector class probability updates.
+ * @class0_bit: motion vector class0 bit probability updates.
+ * @bits: motion vector bits probability updates.
+ * @class0_fr: motion vector class0 fractional bit probability updates.
+ * @fr: motion vector fractional bit probability updates.
+ * @class0_hp: motion vector class0 high precision fractional bit probability updates.
+ * @hp: motion vector high precision fractional bit probability updates.
+ *
+ * This structure contains new values of motion vector probabilities.
+ * A value of zero in an array element means there is no update of the relevant probability.
+ * See `struct v4l2_vp9_prob_updates` for details.
+ */
+struct v4l2_vp9_mv_probs {
+ __u8 joint[3];
+ __u8 sign[2];
+ __u8 classes[2][10];
+ __u8 class0_bit[2];
+ __u8 bits[2][10];
+ __u8 class0_fr[2][2][3];
+ __u8 fr[2][3];
+ __u8 class0_hp[2];
+ __u8 hp[2];
+};
+
+#define V4L2_CID_STATELESS_VP9_COMPRESSED_HDR (V4L2_CID_CODEC_STATELESS_BASE + 301)
+
+#define V4L2_VP9_TX_MODE_ONLY_4X4 0
+#define V4L2_VP9_TX_MODE_ALLOW_8X8 1
+#define V4L2_VP9_TX_MODE_ALLOW_16X16 2
+#define V4L2_VP9_TX_MODE_ALLOW_32X32 3
+#define V4L2_VP9_TX_MODE_SELECT 4
+
+/**
+ * struct v4l2_ctrl_vp9_compressed_hdr - VP9 probability updates control
+ * @tx_mode: specifies the TX mode. Set to one of V4L2_VP9_TX_MODE_{}.
+ * @tx8: TX 8x8 probability updates.
+ * @tx16: TX 16x16 probability updates.
+ * @tx32: TX 32x32 probability updates.
+ * @coef: coefficient probability updates.
+ * @skip: skip probability updates.
+ * @inter_mode: inter mode probability updates.
+ * @interp_filter: interpolation filter probability updates.
+ * @is_inter: is inter-block probability updates.
+ * @comp_mode: compound prediction mode probability updates.
+ * @single_ref: single ref probability updates.
+ * @comp_ref: compound ref probability updates.
+ * @y_mode: Y prediction mode probability updates.
+ * @uv_mode: UV prediction mode probability updates.
+ * @partition: partition probability updates.
+ * @mv: motion vector probability updates.
+ *
+ * This structure holds the probabilities update as parsed in the compressed
+ * header (Spec 6.3). These values represent the value of probability update after
+ * being translated with inv_map_table[] (see 6.3.5). A value of zero in an array element
+ * means that there is no update of the relevant probability.
+ *
+ * This control is optional and needs to be used when dealing with the hardware which is
+ * not capable of parsing the compressed header itself. Only drivers which need it will
+ * implement it.
+ */
+struct v4l2_ctrl_vp9_compressed_hdr {
+ __u8 tx_mode;
+ __u8 tx8[2][1];
+ __u8 tx16[2][2];
+ __u8 tx32[2][3];
+ __u8 coef[4][2][2][6][6][3];
+ __u8 skip[3];
+ __u8 inter_mode[7][3];
+ __u8 interp_filter[4][2];
+ __u8 is_inter[4];
+ __u8 comp_mode[5];
+ __u8 single_ref[5][2];
+ __u8 comp_ref[5];
+ __u8 y_mode[4][9];
+ __u8 uv_mode[10][9];
+ __u8 partition[16][3];
+
+ struct v4l2_vp9_mv_probs mv;
+};
+
/* MPEG-compression definitions kept for backwards compatibility */
#ifndef __KERNEL__
#define V4L2_CTRL_CLASS_MPEG V4L2_CTRL_CLASS_CODEC
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index f118fe7..df8b9c4 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -703,6 +703,7 @@ struct v4l2_pix_format {
#define V4L2_PIX_FMT_VP8 v4l2_fourcc('V', 'P', '8', '0') /* VP8 */
#define V4L2_PIX_FMT_VP8_FRAME v4l2_fourcc('V', 'P', '8', 'F') /* VP8 parsed frame */
#define V4L2_PIX_FMT_VP9 v4l2_fourcc('V', 'P', '9', '0') /* VP9 */
+#define V4L2_PIX_FMT_VP9_FRAME v4l2_fourcc('V', 'P', '9', 'F') /* VP9 parsed frame */
#define V4L2_PIX_FMT_HEVC v4l2_fourcc('H', 'E', 'V', 'C') /* HEVC aka H.265 */
#define V4L2_PIX_FMT_FWHT v4l2_fourcc('F', 'W', 'H', 'T') /* Fast Walsh Hadamard Transform (vicodec) */
#define V4L2_PIX_FMT_FWHT_STATELESS v4l2_fourcc('S', 'F', 'W', 'H') /* Stateless FWHT (vicodec) */
@@ -1759,6 +1760,8 @@ struct v4l2_ext_control {
struct v4l2_ctrl_mpeg2_sequence __user *p_mpeg2_sequence;
struct v4l2_ctrl_mpeg2_picture __user *p_mpeg2_picture;
struct v4l2_ctrl_mpeg2_quantisation __user *p_mpeg2_quantisation;
+ struct v4l2_ctrl_vp9_compressed_hdr __user *p_vp9_compressed_hdr_probs;
+ struct v4l2_ctrl_vp9_frame __user *p_vp9_frame;
void __user *ptr;
};
} __attribute__ ((packed));
@@ -1823,6 +1826,9 @@ enum v4l2_ctrl_type {
V4L2_CTRL_TYPE_MPEG2_QUANTISATION = 0x0250,
V4L2_CTRL_TYPE_MPEG2_SEQUENCE = 0x0251,
V4L2_CTRL_TYPE_MPEG2_PICTURE = 0x0252,
+
+ V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR = 0x0260,
+ V4L2_CTRL_TYPE_VP9_FRAME = 0x0261,
};
/* Used in the VIDIOC_QUERYCTRL ioctl for querying controls */
diff --git a/include/uapi/linux/xfrm.h b/include/uapi/linux/xfrm.h
index eda0426..4e29d78 100644
--- a/include/uapi/linux/xfrm.h
+++ b/include/uapi/linux/xfrm.h
@@ -313,6 +313,7 @@ enum xfrm_attr_type_t {
XFRMA_SET_MARK, /* __u32 */
XFRMA_SET_MARK_MASK, /* __u32 */
XFRMA_IF_ID, /* __u32 */
+ XFRMA_MTIMER_THRESH, /* __u32 in seconds for input SA */
__XFRMA_MAX
#define XFRMA_OUTPUT_MARK XFRMA_SET_MARK /* Compatibility */
diff --git a/include/uapi/rdma/mlx5_user_ioctl_cmds.h b/include/uapi/rdma/mlx5_user_ioctl_cmds.h
index ca23728..e539c84 100644
--- a/include/uapi/rdma/mlx5_user_ioctl_cmds.h
+++ b/include/uapi/rdma/mlx5_user_ioctl_cmds.h
@@ -252,7 +252,7 @@ enum mlx5_ib_device_query_context_attrs {
MLX5_IB_ATTR_QUERY_CONTEXT_RESP_UCTX = (1U << UVERBS_ID_NS_SHIFT),
};
-#define MLX5_IB_DW_MATCH_PARAM 0x90
+#define MLX5_IB_DW_MATCH_PARAM 0xA0
struct mlx5_ib_match_params {
__u32 match_params[MLX5_IB_DW_MATCH_PARAM];
diff --git a/init/Kconfig b/init/Kconfig
index 4b7bac1..03e1b86 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -1933,6 +1933,7 @@
config SLAB_MERGE_DEFAULT
bool "Allow slab caches to be merged"
default y
+ depends on SLAB || SLUB
help
For reduced kernel memory fragmentation, slab caches can be
merged when they share the same size and other characteristics.
diff --git a/kernel/bpf/Makefile b/kernel/bpf/Makefile
index cf6ca33..c1a9be6 100644
--- a/kernel/bpf/Makefile
+++ b/kernel/bpf/Makefile
@@ -36,3 +36,7 @@
obj-${CONFIG_BPF_LSM} += bpf_lsm.o
endif
obj-$(CONFIG_BPF_PRELOAD) += preload/
+
+obj-$(CONFIG_BPF_SYSCALL) += relo_core.o
+$(obj)/relo_core.o: $(srctree)/tools/lib/bpf/relo_core.c FORCE
+ $(call if_changed_rule,cc_o_c)
diff --git a/kernel/bpf/bloom_filter.c b/kernel/bpf/bloom_filter.c
index 277a05e..b141a13 100644
--- a/kernel/bpf/bloom_filter.c
+++ b/kernel/bpf/bloom_filter.c
@@ -82,6 +82,11 @@ static int bloom_map_delete_elem(struct bpf_map *map, void *value)
return -EOPNOTSUPP;
}
+static int bloom_map_get_next_key(struct bpf_map *map, void *key, void *next_key)
+{
+ return -EOPNOTSUPP;
+}
+
static struct bpf_map *bloom_map_alloc(union bpf_attr *attr)
{
u32 bitset_bytes, bitset_mask, nr_hash_funcs, nr_bits;
@@ -192,6 +197,7 @@ const struct bpf_map_ops bloom_filter_map_ops = {
.map_meta_equal = bpf_map_meta_equal,
.map_alloc = bloom_map_alloc,
.map_free = bloom_map_free,
+ .map_get_next_key = bloom_map_get_next_key,
.map_push_elem = bloom_map_push_elem,
.map_peek_elem = bloom_map_peek_elem,
.map_pop_elem = bloom_map_pop_elem,
diff --git a/kernel/bpf/bpf_inode_storage.c b/kernel/bpf/bpf_inode_storage.c
index 96ceed0..e29d9e3 100644
--- a/kernel/bpf/bpf_inode_storage.c
+++ b/kernel/bpf/bpf_inode_storage.c
@@ -17,6 +17,7 @@
#include <linux/bpf_lsm.h>
#include <linux/btf_ids.h>
#include <linux/fdtable.h>
+#include <linux/rcupdate_trace.h>
DEFINE_BPF_STORAGE_CACHE(inode_cache);
@@ -44,7 +45,8 @@ static struct bpf_local_storage_data *inode_storage_lookup(struct inode *inode,
if (!bsb)
return NULL;
- inode_storage = rcu_dereference(bsb->storage);
+ inode_storage =
+ rcu_dereference_check(bsb->storage, bpf_rcu_lock_held());
if (!inode_storage)
return NULL;
@@ -172,6 +174,7 @@ BPF_CALL_4(bpf_inode_storage_get, struct bpf_map *, map, struct inode *, inode,
{
struct bpf_local_storage_data *sdata;
+ WARN_ON_ONCE(!bpf_rcu_lock_held());
if (flags & ~(BPF_LOCAL_STORAGE_GET_F_CREATE))
return (unsigned long)NULL;
@@ -204,6 +207,7 @@ BPF_CALL_4(bpf_inode_storage_get, struct bpf_map *, map, struct inode *, inode,
BPF_CALL_2(bpf_inode_storage_delete,
struct bpf_map *, map, struct inode *, inode)
{
+ WARN_ON_ONCE(!bpf_rcu_lock_held());
if (!inode)
return -EINVAL;
diff --git a/kernel/bpf/bpf_iter.c b/kernel/bpf/bpf_iter.c
index b2ee450..b7aef5b 100644
--- a/kernel/bpf/bpf_iter.c
+++ b/kernel/bpf/bpf_iter.c
@@ -714,3 +714,38 @@ const struct bpf_func_proto bpf_for_each_map_elem_proto = {
.arg3_type = ARG_PTR_TO_STACK_OR_NULL,
.arg4_type = ARG_ANYTHING,
};
+
+/* maximum number of loops */
+#define MAX_LOOPS BIT(23)
+
+BPF_CALL_4(bpf_loop, u32, nr_loops, void *, callback_fn, void *, callback_ctx,
+ u64, flags)
+{
+ bpf_callback_t callback = (bpf_callback_t)callback_fn;
+ u64 ret;
+ u32 i;
+
+ if (flags)
+ return -EINVAL;
+ if (nr_loops > MAX_LOOPS)
+ return -E2BIG;
+
+ for (i = 0; i < nr_loops; i++) {
+ ret = callback((u64)i, (u64)(long)callback_ctx, 0, 0, 0);
+ /* return value: 0 - continue, 1 - stop and return */
+ if (ret)
+ return i + 1;
+ }
+
+ return i;
+}
+
+const struct bpf_func_proto bpf_loop_proto = {
+ .func = bpf_loop,
+ .gpl_only = false,
+ .ret_type = RET_INTEGER,
+ .arg1_type = ARG_ANYTHING,
+ .arg2_type = ARG_PTR_TO_FUNC,
+ .arg3_type = ARG_PTR_TO_STACK_OR_NULL,
+ .arg4_type = ARG_ANYTHING,
+};
diff --git a/kernel/bpf/bpf_local_storage.c b/kernel/bpf/bpf_local_storage.c
index b305270..71de2a8 100644
--- a/kernel/bpf/bpf_local_storage.c
+++ b/kernel/bpf/bpf_local_storage.c
@@ -11,6 +11,9 @@
#include <net/sock.h>
#include <uapi/linux/sock_diag.h>
#include <uapi/linux/btf.h>
+#include <linux/rcupdate.h>
+#include <linux/rcupdate_trace.h>
+#include <linux/rcupdate_wait.h>
#define BPF_LOCAL_STORAGE_CREATE_FLAG_MASK (BPF_F_NO_PREALLOC | BPF_F_CLONE)
@@ -81,6 +84,22 @@ bpf_selem_alloc(struct bpf_local_storage_map *smap, void *owner,
return NULL;
}
+void bpf_local_storage_free_rcu(struct rcu_head *rcu)
+{
+ struct bpf_local_storage *local_storage;
+
+ local_storage = container_of(rcu, struct bpf_local_storage, rcu);
+ kfree_rcu(local_storage, rcu);
+}
+
+static void bpf_selem_free_rcu(struct rcu_head *rcu)
+{
+ struct bpf_local_storage_elem *selem;
+
+ selem = container_of(rcu, struct bpf_local_storage_elem, rcu);
+ kfree_rcu(selem, rcu);
+}
+
/* local_storage->lock must be held and selem->local_storage == local_storage.
* The caller must ensure selem->smap is still valid to be
* dereferenced for its smap->elem_size and smap->cache_idx.
@@ -93,7 +112,7 @@ bool bpf_selem_unlink_storage_nolock(struct bpf_local_storage *local_storage,
bool free_local_storage;
void *owner;
- smap = rcu_dereference(SDATA(selem)->smap);
+ smap = rcu_dereference_check(SDATA(selem)->smap, bpf_rcu_lock_held());
owner = local_storage->owner;
/* All uncharging on the owner must be done first.
@@ -118,12 +137,12 @@ bool bpf_selem_unlink_storage_nolock(struct bpf_local_storage *local_storage,
*
* Although the unlock will be done under
* rcu_read_lock(), it is more intutivie to
- * read if kfree_rcu(local_storage, rcu) is done
+ * read if the freeing of the storage is done
* after the raw_spin_unlock_bh(&local_storage->lock).
*
* Hence, a "bool free_local_storage" is returned
- * to the caller which then calls the kfree_rcu()
- * after unlock.
+ * to the caller which then calls then frees the storage after
+ * all the RCU grace periods have expired.
*/
}
hlist_del_init_rcu(&selem->snode);
@@ -131,8 +150,7 @@ bool bpf_selem_unlink_storage_nolock(struct bpf_local_storage *local_storage,
SDATA(selem))
RCU_INIT_POINTER(local_storage->cache[smap->cache_idx], NULL);
- kfree_rcu(selem, rcu);
-
+ call_rcu_tasks_trace(&selem->rcu, bpf_selem_free_rcu);
return free_local_storage;
}
@@ -146,7 +164,8 @@ static void __bpf_selem_unlink_storage(struct bpf_local_storage_elem *selem)
/* selem has already been unlinked from sk */
return;
- local_storage = rcu_dereference(selem->local_storage);
+ local_storage = rcu_dereference_check(selem->local_storage,
+ bpf_rcu_lock_held());
raw_spin_lock_irqsave(&local_storage->lock, flags);
if (likely(selem_linked_to_storage(selem)))
free_local_storage = bpf_selem_unlink_storage_nolock(
@@ -154,7 +173,8 @@ static void __bpf_selem_unlink_storage(struct bpf_local_storage_elem *selem)
raw_spin_unlock_irqrestore(&local_storage->lock, flags);
if (free_local_storage)
- kfree_rcu(local_storage, rcu);
+ call_rcu_tasks_trace(&local_storage->rcu,
+ bpf_local_storage_free_rcu);
}
void bpf_selem_link_storage_nolock(struct bpf_local_storage *local_storage,
@@ -174,7 +194,7 @@ void bpf_selem_unlink_map(struct bpf_local_storage_elem *selem)
/* selem has already be unlinked from smap */
return;
- smap = rcu_dereference(SDATA(selem)->smap);
+ smap = rcu_dereference_check(SDATA(selem)->smap, bpf_rcu_lock_held());
b = select_bucket(smap, selem);
raw_spin_lock_irqsave(&b->lock, flags);
if (likely(selem_linked_to_map(selem)))
@@ -213,12 +233,14 @@ bpf_local_storage_lookup(struct bpf_local_storage *local_storage,
struct bpf_local_storage_elem *selem;
/* Fast path (cache hit) */
- sdata = rcu_dereference(local_storage->cache[smap->cache_idx]);
+ sdata = rcu_dereference_check(local_storage->cache[smap->cache_idx],
+ bpf_rcu_lock_held());
if (sdata && rcu_access_pointer(sdata->smap) == smap)
return sdata;
/* Slow path (cache miss) */
- hlist_for_each_entry_rcu(selem, &local_storage->list, snode)
+ hlist_for_each_entry_rcu(selem, &local_storage->list, snode,
+ rcu_read_lock_trace_held())
if (rcu_access_pointer(SDATA(selem)->smap) == smap)
break;
@@ -306,7 +328,8 @@ int bpf_local_storage_alloc(void *owner,
* bucket->list, first_selem can be freed immediately
* (instead of kfree_rcu) because
* bpf_local_storage_map_free() does a
- * synchronize_rcu() before walking the bucket->list.
+ * synchronize_rcu_mult (waiting for both sleepable and
+ * normal programs) before walking the bucket->list.
* Hence, no one is accessing selem from the
* bucket->list under rcu_read_lock().
*/
@@ -342,7 +365,8 @@ bpf_local_storage_update(void *owner, struct bpf_local_storage_map *smap,
!map_value_has_spin_lock(&smap->map)))
return ERR_PTR(-EINVAL);
- local_storage = rcu_dereference(*owner_storage(smap, owner));
+ local_storage = rcu_dereference_check(*owner_storage(smap, owner),
+ bpf_rcu_lock_held());
if (!local_storage || hlist_empty(&local_storage->list)) {
/* Very first elem for the owner */
err = check_flags(NULL, map_flags);
diff --git a/kernel/bpf/bpf_struct_ops.c b/kernel/bpf/bpf_struct_ops.c
index 8ecfe47..21069db 100644
--- a/kernel/bpf/bpf_struct_ops.c
+++ b/kernel/bpf/bpf_struct_ops.c
@@ -165,7 +165,7 @@ void bpf_struct_ops_init(struct btf *btf, struct bpf_verifier_log *log)
break;
}
- if (btf_member_bitfield_size(t, member)) {
+ if (__btf_member_bitfield_size(t, member)) {
pr_warn("bit field member %s in struct %s is not supported\n",
mname, st_ops->name);
break;
@@ -296,7 +296,7 @@ static int check_zero_holes(const struct btf_type *t, void *data)
const struct btf_type *mtype;
for_each_member(i, t, member) {
- moff = btf_member_bit_offset(t, member) / 8;
+ moff = __btf_member_bit_offset(t, member) / 8;
if (moff > prev_mend &&
memchr_inv(data + prev_mend, 0, moff - prev_mend))
return -EINVAL;
@@ -387,7 +387,7 @@ static int bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key,
struct bpf_prog *prog;
u32 moff;
- moff = btf_member_bit_offset(t, member) / 8;
+ moff = __btf_member_bit_offset(t, member) / 8;
ptype = btf_type_resolve_ptr(btf_vmlinux, member->type, NULL);
if (ptype == module_type) {
if (*(void **)(udata + moff))
diff --git a/kernel/bpf/bpf_task_storage.c b/kernel/bpf/bpf_task_storage.c
index ebfa8bc..5da7bed 100644
--- a/kernel/bpf/bpf_task_storage.c
+++ b/kernel/bpf/bpf_task_storage.c
@@ -17,6 +17,7 @@
#include <uapi/linux/btf.h>
#include <linux/btf_ids.h>
#include <linux/fdtable.h>
+#include <linux/rcupdate_trace.h>
DEFINE_BPF_STORAGE_CACHE(task_cache);
@@ -59,7 +60,8 @@ task_storage_lookup(struct task_struct *task, struct bpf_map *map,
struct bpf_local_storage *task_storage;
struct bpf_local_storage_map *smap;
- task_storage = rcu_dereference(task->bpf_storage);
+ task_storage =
+ rcu_dereference_check(task->bpf_storage, bpf_rcu_lock_held());
if (!task_storage)
return NULL;
@@ -229,6 +231,7 @@ BPF_CALL_4(bpf_task_storage_get, struct bpf_map *, map, struct task_struct *,
{
struct bpf_local_storage_data *sdata;
+ WARN_ON_ONCE(!bpf_rcu_lock_held());
if (flags & ~(BPF_LOCAL_STORAGE_GET_F_CREATE))
return (unsigned long)NULL;
@@ -260,6 +263,7 @@ BPF_CALL_2(bpf_task_storage_delete, struct bpf_map *, map, struct task_struct *,
{
int ret;
+ WARN_ON_ONCE(!bpf_rcu_lock_held());
if (!task)
return -EINVAL;
@@ -323,7 +327,7 @@ const struct bpf_func_proto bpf_task_storage_get_proto = {
.ret_type = RET_PTR_TO_MAP_VALUE_OR_NULL,
.arg1_type = ARG_CONST_MAP_PTR,
.arg2_type = ARG_PTR_TO_BTF_ID,
- .arg2_btf_id = &btf_task_struct_ids[0],
+ .arg2_btf_id = &btf_tracing_ids[BTF_TRACING_TYPE_TASK],
.arg3_type = ARG_PTR_TO_MAP_VALUE_OR_NULL,
.arg4_type = ARG_ANYTHING,
};
@@ -334,5 +338,5 @@ const struct bpf_func_proto bpf_task_storage_delete_proto = {
.ret_type = RET_INTEGER,
.arg1_type = ARG_CONST_MAP_PTR,
.arg2_type = ARG_PTR_TO_BTF_ID,
- .arg2_btf_id = &btf_task_struct_ids[0],
+ .arg2_btf_id = &btf_tracing_ids[BTF_TRACING_TYPE_TASK],
};
diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index 9bdb037..33bb8ae 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -25,6 +25,7 @@
#include <linux/kobject.h>
#include <linux/sysfs.h>
#include <net/sock.h>
+#include "../tools/lib/bpf/relo_core.h"
/* BTF (BPF Type Format) is the meta data format which describes
* the data types of BPF program/map. Hence, it basically focus
@@ -282,6 +283,7 @@ static const char * const btf_kind_str[NR_BTF_KINDS] = {
[BTF_KIND_DATASEC] = "DATASEC",
[BTF_KIND_FLOAT] = "FLOAT",
[BTF_KIND_DECL_TAG] = "DECL_TAG",
+ [BTF_KIND_TYPE_TAG] = "TYPE_TAG",
};
const char *btf_type_str(const struct btf_type *t)
@@ -418,6 +420,7 @@ static bool btf_type_is_modifier(const struct btf_type *t)
case BTF_KIND_VOLATILE:
case BTF_KIND_CONST:
case BTF_KIND_RESTRICT:
+ case BTF_KIND_TYPE_TAG:
return true;
}
@@ -834,7 +837,7 @@ static const char *btf_show_name(struct btf_show *show)
const char *ptr_suffix = &ptr_suffixes[strlen(ptr_suffixes)];
const char *name = NULL, *prefix = "", *parens = "";
const struct btf_member *m = show->state.member;
- const struct btf_type *t = show->state.type;
+ const struct btf_type *t;
const struct btf_array *array;
u32 id = show->state.type_id;
const char *member = NULL;
@@ -1737,6 +1740,7 @@ __btf_resolve_size(const struct btf *btf, const struct btf_type *type,
case BTF_KIND_VOLATILE:
case BTF_KIND_CONST:
case BTF_KIND_RESTRICT:
+ case BTF_KIND_TYPE_TAG:
id = type->type;
type = btf_type_by_id(btf, type->type);
break;
@@ -2345,6 +2349,8 @@ static int btf_ref_type_check_meta(struct btf_verifier_env *env,
const struct btf_type *t,
u32 meta_left)
{
+ const char *value;
+
if (btf_type_vlen(t)) {
btf_verifier_log_type(env, t, "vlen != 0");
return -EINVAL;
@@ -2360,7 +2366,7 @@ static int btf_ref_type_check_meta(struct btf_verifier_env *env,
return -EINVAL;
}
- /* typedef type must have a valid name, and other ref types,
+ /* typedef/type_tag type must have a valid name, and other ref types,
* volatile, const, restrict, should have a null name.
*/
if (BTF_INFO_KIND(t->info) == BTF_KIND_TYPEDEF) {
@@ -2369,6 +2375,12 @@ static int btf_ref_type_check_meta(struct btf_verifier_env *env,
btf_verifier_log_type(env, t, "Invalid name");
return -EINVAL;
}
+ } else if (BTF_INFO_KIND(t->info) == BTF_KIND_TYPE_TAG) {
+ value = btf_name_by_offset(env->btf, t->name_off);
+ if (!value || !value[0]) {
+ btf_verifier_log_type(env, t, "Invalid name");
+ return -EINVAL;
+ }
} else {
if (t->name_off) {
btf_verifier_log_type(env, t, "Invalid name");
@@ -2958,7 +2970,7 @@ static s32 btf_struct_check_meta(struct btf_verifier_env *env,
return -EINVAL;
}
- offset = btf_member_bit_offset(t, member);
+ offset = __btf_member_bit_offset(t, member);
if (is_union && offset) {
btf_verifier_log_member(env, t, member,
"Invalid member bits_offset");
@@ -3083,7 +3095,7 @@ static int btf_find_struct_field(const struct btf *btf, const struct btf_type *t
if (off != -ENOENT)
/* only one such field is allowed */
return -E2BIG;
- off = btf_member_bit_offset(t, member);
+ off = __btf_member_bit_offset(t, member);
if (off % 8)
/* valid C code cannot generate such BTF */
return -EINVAL;
@@ -3173,8 +3185,8 @@ static void __btf_struct_show(const struct btf *btf, const struct btf_type *t,
btf_show_start_member(show, member);
- member_offset = btf_member_bit_offset(t, member);
- bitfield_size = btf_member_bitfield_size(t, member);
+ member_offset = __btf_member_bit_offset(t, member);
+ bitfield_size = __btf_member_bitfield_size(t, member);
bytes_offset = BITS_ROUNDDOWN_BYTES(member_offset);
bits8_offset = BITS_PER_BYTE_MASKED(member_offset);
if (bitfield_size) {
@@ -4059,6 +4071,7 @@ static const struct btf_kind_operations * const kind_ops[NR_BTF_KINDS] = {
[BTF_KIND_DATASEC] = &datasec_ops,
[BTF_KIND_FLOAT] = &float_ops,
[BTF_KIND_DECL_TAG] = &decl_tag_ops,
+ [BTF_KIND_TYPE_TAG] = &modifier_ops,
};
static s32 btf_check_meta(struct btf_verifier_env *env,
@@ -4460,8 +4473,7 @@ static struct btf *btf_parse(bpfptr_t btf_data, u32 btf_data_size,
log->len_total = log_size;
/* log attributes have to be sane */
- if (log->len_total < 128 || log->len_total > UINT_MAX >> 8 ||
- !log->level || !log->ubuf) {
+ if (!bpf_verifier_log_attr_valid(log)) {
err = -EINVAL;
goto errout;
}
@@ -4814,7 +4826,7 @@ struct btf *bpf_prog_get_target_btf(const struct bpf_prog *prog)
return prog->aux->attach_btf;
}
-static bool is_string_ptr(struct btf *btf, const struct btf_type *t)
+static bool is_int_ptr(struct btf *btf, const struct btf_type *t)
{
/* t comes in already as a pointer */
t = btf_type_by_id(btf, t->type);
@@ -4823,8 +4835,7 @@ static bool is_string_ptr(struct btf *btf, const struct btf_type *t)
if (BTF_INFO_KIND(t->info) == BTF_KIND_CONST)
t = btf_type_by_id(btf, t->type);
- /* char, signed char, unsigned char */
- return btf_type_is_int(t) && t->size == 1;
+ return btf_type_is_int(t);
}
bool btf_ctx_access(int off, int size, enum bpf_access_type type,
@@ -4929,10 +4940,12 @@ bool btf_ctx_access(int off, int size, enum bpf_access_type type,
/* check for PTR_TO_RDONLY_BUF_OR_NULL or PTR_TO_RDWR_BUF_OR_NULL */
for (i = 0; i < prog->aux->ctx_arg_info_size; i++) {
const struct bpf_ctx_arg_aux *ctx_arg_info = &prog->aux->ctx_arg_info[i];
+ u32 type, flag;
- if (ctx_arg_info->offset == off &&
- (ctx_arg_info->reg_type == PTR_TO_RDONLY_BUF_OR_NULL ||
- ctx_arg_info->reg_type == PTR_TO_RDWR_BUF_OR_NULL)) {
+ type = base_type(ctx_arg_info->reg_type);
+ flag = type_flag(ctx_arg_info->reg_type);
+ if (ctx_arg_info->offset == off && type == PTR_TO_BUF &&
+ (flag & PTR_MAYBE_NULL)) {
info->reg_type = ctx_arg_info->reg_type;
return true;
}
@@ -4945,7 +4958,7 @@ bool btf_ctx_access(int off, int size, enum bpf_access_type type,
*/
return true;
- if (is_string_ptr(btf, t))
+ if (is_int_ptr(btf, t))
return true;
/* this is a pointer to another type */
@@ -5048,7 +5061,7 @@ static int btf_struct_walk(struct bpf_verifier_log *log, const struct btf *btf,
if (array_elem->nelems != 0)
goto error;
- moff = btf_member_bit_offset(t, member) / 8;
+ moff = __btf_member_bit_offset(t, member) / 8;
if (off < moff)
goto error;
@@ -5071,14 +5084,14 @@ static int btf_struct_walk(struct bpf_verifier_log *log, const struct btf *btf,
for_each_member(i, t, member) {
/* offset of the field in bytes */
- moff = btf_member_bit_offset(t, member) / 8;
+ moff = __btf_member_bit_offset(t, member) / 8;
if (off + size <= moff)
/* won't find anything, field is already too far */
break;
- if (btf_member_bitfield_size(t, member)) {
- u32 end_bit = btf_member_bit_offset(t, member) +
- btf_member_bitfield_size(t, member);
+ if (__btf_member_bitfield_size(t, member)) {
+ u32 end_bit = __btf_member_bit_offset(t, member) +
+ __btf_member_bitfield_size(t, member);
/* off <= moff instead of off == moff because clang
* does not generate a BTF member for anonymous
@@ -5563,12 +5576,53 @@ static u32 *reg2btf_ids[__BPF_REG_TYPE_MAX] = {
#endif
};
+/* Returns true if struct is composed of scalars, 4 levels of nesting allowed */
+static bool __btf_type_is_scalar_struct(struct bpf_verifier_log *log,
+ const struct btf *btf,
+ const struct btf_type *t, int rec)
+{
+ const struct btf_type *member_type;
+ const struct btf_member *member;
+ u32 i;
+
+ if (!btf_type_is_struct(t))
+ return false;
+
+ for_each_member(i, t, member) {
+ const struct btf_array *array;
+
+ member_type = btf_type_skip_modifiers(btf, member->type, NULL);
+ if (btf_type_is_struct(member_type)) {
+ if (rec >= 3) {
+ bpf_log(log, "max struct nesting depth exceeded\n");
+ return false;
+ }
+ if (!__btf_type_is_scalar_struct(log, btf, member_type, rec + 1))
+ return false;
+ continue;
+ }
+ if (btf_type_is_array(member_type)) {
+ array = btf_type_array(member_type);
+ if (!array->nelems)
+ return false;
+ member_type = btf_type_skip_modifiers(btf, array->type, NULL);
+ if (!btf_type_is_scalar(member_type))
+ return false;
+ continue;
+ }
+ if (!btf_type_is_scalar(member_type))
+ return false;
+ }
+ return true;
+}
+
static int btf_check_func_arg_match(struct bpf_verifier_env *env,
const struct btf *btf, u32 func_id,
struct bpf_reg_state *regs,
bool ptr_to_mem_ok)
{
struct bpf_verifier_log *log = &env->log;
+ bool is_kfunc = btf_is_kernel(btf);
const char *func_name, *ref_tname;
const struct btf_type *t, *ref_t;
const struct btf_param *args;
@@ -5621,7 +5675,20 @@ static int btf_check_func_arg_match(struct bpf_verifier_env *env,
ref_t = btf_type_skip_modifiers(btf, t->type, &ref_id);
ref_tname = btf_name_by_offset(btf, ref_t->name_off);
- if (btf_is_kernel(btf)) {
+ if (btf_get_prog_ctx_type(log, btf, t,
+ env->prog->type, i)) {
+ /* If function expects ctx type in BTF check that caller
+ * is passing PTR_TO_CTX.
+ */
+ if (reg->type != PTR_TO_CTX) {
+ bpf_log(log,
+ "arg#%d expected pointer to ctx, but got %s\n",
+ i, btf_type_str(t));
+ return -EINVAL;
+ }
+ if (check_ctx_reg(env, reg, regno))
+ return -EINVAL;
+ } else if (is_kfunc && (reg->type == PTR_TO_BTF_ID || reg2btf_ids[reg->type])) {
const struct btf_type *reg_ref_t;
const struct btf *reg_btf;
const char *reg_ref_tname;
@@ -5637,14 +5704,9 @@ static int btf_check_func_arg_match(struct bpf_verifier_env *env,
if (reg->type == PTR_TO_BTF_ID) {
reg_btf = reg->btf;
reg_ref_id = reg->btf_id;
- } else if (reg2btf_ids[reg->type]) {
+ } else {
reg_btf = btf_vmlinux;
reg_ref_id = *reg2btf_ids[reg->type];
- } else {
- bpf_log(log, "kernel function %s args#%d expected pointer to %s %s but R%d is not a pointer to btf_id\n",
- func_name, i,
- btf_type_str(ref_t), ref_tname, regno);
- return -EINVAL;
}
reg_ref_t = btf_type_skip_modifiers(reg_btf, reg_ref_id,
@@ -5660,23 +5722,24 @@ static int btf_check_func_arg_match(struct bpf_verifier_env *env,
reg_ref_tname);
return -EINVAL;
}
- } else if (btf_get_prog_ctx_type(log, btf, t,
- env->prog->type, i)) {
- /* If function expects ctx type in BTF check that caller
- * is passing PTR_TO_CTX.
- */
- if (reg->type != PTR_TO_CTX) {
- bpf_log(log,
- "arg#%d expected pointer to ctx, but got %s\n",
- i, btf_type_str(t));
- return -EINVAL;
- }
- if (check_ctx_reg(env, reg, regno))
- return -EINVAL;
} else if (ptr_to_mem_ok) {
const struct btf_type *resolve_ret;
u32 type_size;
+ if (is_kfunc) {
+ /* Permit pointer to mem, but only when argument
+ * type is pointer to scalar, or struct composed
+ * (recursively) of scalars.
+ */
+ if (!btf_type_is_scalar(ref_t) &&
+ !__btf_type_is_scalar_struct(log, btf, ref_t, 0)) {
+ bpf_log(log,
+ "arg#%d pointer type %s %s must point to scalar or struct with scalar\n",
+ i, btf_type_str(ref_t), ref_tname);
+ return -EINVAL;
+ }
+ }
+
resolve_ret = btf_resolve_size(btf, ref_t, &type_size);
if (IS_ERR(resolve_ret)) {
bpf_log(log,
@@ -5689,6 +5752,8 @@ static int btf_check_func_arg_match(struct bpf_verifier_env *env,
if (check_mem_reg(env, reg, regno, type_size))
return -EINVAL;
} else {
+ bpf_log(log, "reg type unsupported for arg#%d %sfunction %s#%d\n", i,
+ is_kfunc ? "kernel " : "", func_name, func_id);
return -EINVAL;
}
}
@@ -5738,7 +5803,7 @@ int btf_check_kfunc_arg_match(struct bpf_verifier_env *env,
const struct btf *btf, u32 func_id,
struct bpf_reg_state *regs)
{
- return btf_check_func_arg_match(env, btf, func_id, regs, false);
+ return btf_check_func_arg_match(env, btf, func_id, regs, true);
}
/* Convert BTF of a function into bpf_reg_state if possible
@@ -5846,7 +5911,7 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog,
return -EINVAL;
}
- reg->type = PTR_TO_MEM_OR_NULL;
+ reg->type = PTR_TO_MEM | PTR_MAYBE_NULL;
reg->id = ++env->id_gen;
continue;
@@ -6157,6 +6222,8 @@ btf_module_read(struct file *file, struct kobject *kobj,
return len;
}
+static void purge_cand_cache(struct btf *btf);
+
static int btf_module_notify(struct notifier_block *nb, unsigned long op,
void *module)
{
@@ -6191,6 +6258,7 @@ static int btf_module_notify(struct notifier_block *nb, unsigned long op,
goto out;
}
+ purge_cand_cache(NULL);
mutex_lock(&btf_module_mutex);
btf_mod->module = module;
btf_mod->btf = btf;
@@ -6233,6 +6301,7 @@ static int btf_module_notify(struct notifier_block *nb, unsigned long op,
list_del(&btf_mod->list);
if (btf_mod->sysfs_attr)
sysfs_remove_bin_file(btf_kobj, btf_mod->sysfs_attr);
+ purge_cand_cache(btf_mod->btf);
btf_put(btf_mod->btf);
kfree(btf_mod->sysfs_attr);
kfree(btf_mod);
@@ -6336,13 +6405,16 @@ const struct bpf_func_proto bpf_btf_find_by_name_kind_proto = {
.func = bpf_btf_find_by_name_kind,
.gpl_only = false,
.ret_type = RET_INTEGER,
- .arg1_type = ARG_PTR_TO_MEM,
+ .arg1_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg2_type = ARG_CONST_SIZE,
.arg3_type = ARG_ANYTHING,
.arg4_type = ARG_ANYTHING,
};
-BTF_ID_LIST_GLOBAL_SINGLE(btf_task_struct_ids, struct, task_struct)
+BTF_ID_LIST_GLOBAL(btf_tracing_ids, MAX_BTF_TRACING_TYPE)
+#define BTF_TRACING_TYPE(name, type) BTF_ID(struct, type)
+BTF_TRACING_TYPE_xxx
+#undef BTF_TRACING_TYPE
/* BTF ID set registration API for modules */
@@ -6391,3 +6463,384 @@ DEFINE_KFUNC_BTF_ID_LIST(bpf_tcp_ca_kfunc_list);
DEFINE_KFUNC_BTF_ID_LIST(prog_test_kfunc_list);
#endif
+
+int bpf_core_types_are_compat(const struct btf *local_btf, __u32 local_id,
+ const struct btf *targ_btf, __u32 targ_id)
+{
+ return -EOPNOTSUPP;
+}
+
+static bool bpf_core_is_flavor_sep(const char *s)
+{
+ /* check X___Y name pattern, where X and Y are not underscores */
+ return s[0] != '_' && /* X */
+ s[1] == '_' && s[2] == '_' && s[3] == '_' && /* ___ */
+ s[4] != '_'; /* Y */
+}
+
+size_t bpf_core_essential_name_len(const char *name)
+{
+ size_t n = strlen(name);
+ int i;
+
+ for (i = n - 5; i >= 0; i--) {
+ if (bpf_core_is_flavor_sep(name + i))
+ return i + 1;
+ }
+ return n;
+}
+
+struct bpf_cand_cache {
+ const char *name;
+ u32 name_len;
+ u16 kind;
+ u16 cnt;
+ struct {
+ const struct btf *btf;
+ u32 id;
+ } cands[];
+};
+
+static void bpf_free_cands(struct bpf_cand_cache *cands)
+{
+ if (!cands->cnt)
+ /* empty candidate array was allocated on stack */
+ return;
+ kfree(cands);
+}
+
+static void bpf_free_cands_from_cache(struct bpf_cand_cache *cands)
+{
+ kfree(cands->name);
+ kfree(cands);
+}
+
+#define VMLINUX_CAND_CACHE_SIZE 31
+static struct bpf_cand_cache *vmlinux_cand_cache[VMLINUX_CAND_CACHE_SIZE];
+
+#define MODULE_CAND_CACHE_SIZE 31
+static struct bpf_cand_cache *module_cand_cache[MODULE_CAND_CACHE_SIZE];
+
+static DEFINE_MUTEX(cand_cache_mutex);
+
+static void __print_cand_cache(struct bpf_verifier_log *log,
+ struct bpf_cand_cache **cache,
+ int cache_size)
+{
+ struct bpf_cand_cache *cc;
+ int i, j;
+
+ for (i = 0; i < cache_size; i++) {
+ cc = cache[i];
+ if (!cc)
+ continue;
+ bpf_log(log, "[%d]%s(", i, cc->name);
+ for (j = 0; j < cc->cnt; j++) {
+ bpf_log(log, "%d", cc->cands[j].id);
+ if (j < cc->cnt - 1)
+ bpf_log(log, " ");
+ }
+ bpf_log(log, "), ");
+ }
+}
+
+static void print_cand_cache(struct bpf_verifier_log *log)
+{
+ mutex_lock(&cand_cache_mutex);
+ bpf_log(log, "vmlinux_cand_cache:");
+ __print_cand_cache(log, vmlinux_cand_cache, VMLINUX_CAND_CACHE_SIZE);
+ bpf_log(log, "\nmodule_cand_cache:");
+ __print_cand_cache(log, module_cand_cache, MODULE_CAND_CACHE_SIZE);
+ bpf_log(log, "\n");
+ mutex_unlock(&cand_cache_mutex);
+}
+
+static u32 hash_cands(struct bpf_cand_cache *cands)
+{
+ return jhash(cands->name, cands->name_len, 0);
+}
+
+static struct bpf_cand_cache *check_cand_cache(struct bpf_cand_cache *cands,
+ struct bpf_cand_cache **cache,
+ int cache_size)
+{
+ struct bpf_cand_cache *cc = cache[hash_cands(cands) % cache_size];
+
+ if (cc && cc->name_len == cands->name_len &&
+ !strncmp(cc->name, cands->name, cands->name_len))
+ return cc;
+ return NULL;
+}
+
+static size_t sizeof_cands(int cnt)
+{
+ return offsetof(struct bpf_cand_cache, cands[cnt]);
+}
+
+static struct bpf_cand_cache *populate_cand_cache(struct bpf_cand_cache *cands,
+ struct bpf_cand_cache **cache,
+ int cache_size)
+{
+ struct bpf_cand_cache **cc = &cache[hash_cands(cands) % cache_size], *new_cands;
+
+ if (*cc) {
+ bpf_free_cands_from_cache(*cc);
+ *cc = NULL;
+ }
+ new_cands = kmemdup(cands, sizeof_cands(cands->cnt), GFP_KERNEL);
+ if (!new_cands) {
+ bpf_free_cands(cands);
+ return ERR_PTR(-ENOMEM);
+ }
+ /* strdup the name, since it will stay in cache.
+ * the cands->name points to strings in prog's BTF and the prog can be unloaded.
+ */
+ new_cands->name = kmemdup_nul(cands->name, cands->name_len, GFP_KERNEL);
+ bpf_free_cands(cands);
+ if (!new_cands->name) {
+ kfree(new_cands);
+ return ERR_PTR(-ENOMEM);
+ }
+ *cc = new_cands;
+ return new_cands;
+}
+
+#ifdef CONFIG_DEBUG_INFO_BTF_MODULES
+static void __purge_cand_cache(struct btf *btf, struct bpf_cand_cache **cache,
+ int cache_size)
+{
+ struct bpf_cand_cache *cc;
+ int i, j;
+
+ for (i = 0; i < cache_size; i++) {
+ cc = cache[i];
+ if (!cc)
+ continue;
+ if (!btf) {
+ /* when new module is loaded purge all of module_cand_cache,
+ * since new module might have candidates with the name
+ * that matches cached cands.
+ */
+ bpf_free_cands_from_cache(cc);
+ cache[i] = NULL;
+ continue;
+ }
+ /* when module is unloaded purge cache entries
+ * that match module's btf
+ */
+ for (j = 0; j < cc->cnt; j++)
+ if (cc->cands[j].btf == btf) {
+ bpf_free_cands_from_cache(cc);
+ cache[i] = NULL;
+ break;
+ }
+ }
+
+}
+
+static void purge_cand_cache(struct btf *btf)
+{
+ mutex_lock(&cand_cache_mutex);
+ __purge_cand_cache(btf, module_cand_cache, MODULE_CAND_CACHE_SIZE);
+ mutex_unlock(&cand_cache_mutex);
+}
+#endif
+
+static struct bpf_cand_cache *
+bpf_core_add_cands(struct bpf_cand_cache *cands, const struct btf *targ_btf,
+ int targ_start_id)
+{
+ struct bpf_cand_cache *new_cands;
+ const struct btf_type *t;
+ const char *targ_name;
+ size_t targ_essent_len;
+ int n, i;
+
+ n = btf_nr_types(targ_btf);
+ for (i = targ_start_id; i < n; i++) {
+ t = btf_type_by_id(targ_btf, i);
+ if (btf_kind(t) != cands->kind)
+ continue;
+
+ targ_name = btf_name_by_offset(targ_btf, t->name_off);
+ if (!targ_name)
+ continue;
+
+ /* the resched point is before strncmp to make sure that search
+ * for non-existing name will have a chance to schedule().
+ */
+ cond_resched();
+
+ if (strncmp(cands->name, targ_name, cands->name_len) != 0)
+ continue;
+
+ targ_essent_len = bpf_core_essential_name_len(targ_name);
+ if (targ_essent_len != cands->name_len)
+ continue;
+
+ /* most of the time there is only one candidate for a given kind+name pair */
+ new_cands = kmalloc(sizeof_cands(cands->cnt + 1), GFP_KERNEL);
+ if (!new_cands) {
+ bpf_free_cands(cands);
+ return ERR_PTR(-ENOMEM);
+ }
+
+ memcpy(new_cands, cands, sizeof_cands(cands->cnt));
+ bpf_free_cands(cands);
+ cands = new_cands;
+ cands->cands[cands->cnt].btf = targ_btf;
+ cands->cands[cands->cnt].id = i;
+ cands->cnt++;
+ }
+ return cands;
+}
+
+static struct bpf_cand_cache *
+bpf_core_find_cands(struct bpf_core_ctx *ctx, u32 local_type_id)
+{
+ struct bpf_cand_cache *cands, *cc, local_cand = {};
+ const struct btf *local_btf = ctx->btf;
+ const struct btf_type *local_type;
+ const struct btf *main_btf;
+ size_t local_essent_len;
+ struct btf *mod_btf;
+ const char *name;
+ int id;
+
+ main_btf = bpf_get_btf_vmlinux();
+ if (IS_ERR(main_btf))
+ return ERR_CAST(main_btf);
+
+ local_type = btf_type_by_id(local_btf, local_type_id);
+ if (!local_type)
+ return ERR_PTR(-EINVAL);
+
+ name = btf_name_by_offset(local_btf, local_type->name_off);
+ if (str_is_empty(name))
+ return ERR_PTR(-EINVAL);
+ local_essent_len = bpf_core_essential_name_len(name);
+
+ cands = &local_cand;
+ cands->name = name;
+ cands->kind = btf_kind(local_type);
+ cands->name_len = local_essent_len;
+
+ cc = check_cand_cache(cands, vmlinux_cand_cache, VMLINUX_CAND_CACHE_SIZE);
+ /* cands is a pointer to stack here */
+ if (cc) {
+ if (cc->cnt)
+ return cc;
+ goto check_modules;
+ }
+
+ /* Attempt to find target candidates in vmlinux BTF first */
+ cands = bpf_core_add_cands(cands, main_btf, 1);
+ if (IS_ERR(cands))
+ return ERR_CAST(cands);
+
+ /* cands is a pointer to kmalloced memory here if cands->cnt > 0 */
+
+ /* populate cache even when cands->cnt == 0 */
+ cc = populate_cand_cache(cands, vmlinux_cand_cache, VMLINUX_CAND_CACHE_SIZE);
+ if (IS_ERR(cc))
+ return ERR_CAST(cc);
+
+ /* if vmlinux BTF has any candidate, don't go for module BTFs */
+ if (cc->cnt)
+ return cc;
+
+check_modules:
+ /* cands is a pointer to stack here and cands->cnt == 0 */
+ cc = check_cand_cache(cands, module_cand_cache, MODULE_CAND_CACHE_SIZE);
+ if (cc)
+ /* if cache has it return it even if cc->cnt == 0 */
+ return cc;
+
+ /* If candidate is not found in vmlinux's BTF then search in module's BTFs */
+ spin_lock_bh(&btf_idr_lock);
+ idr_for_each_entry(&btf_idr, mod_btf, id) {
+ if (!btf_is_module(mod_btf))
+ continue;
+ /* linear search could be slow hence unlock/lock
+ * the IDR to avoiding holding it for too long
+ */
+ btf_get(mod_btf);
+ spin_unlock_bh(&btf_idr_lock);
+ cands = bpf_core_add_cands(cands, mod_btf, btf_nr_types(main_btf));
+ if (IS_ERR(cands)) {
+ btf_put(mod_btf);
+ return ERR_CAST(cands);
+ }
+ spin_lock_bh(&btf_idr_lock);
+ btf_put(mod_btf);
+ }
+ spin_unlock_bh(&btf_idr_lock);
+ /* cands is a pointer to kmalloced memory here if cands->cnt > 0
+ * or pointer to stack if cands->cnd == 0.
+ * Copy it into the cache even when cands->cnt == 0 and
+ * return the result.
+ */
+ return populate_cand_cache(cands, module_cand_cache, MODULE_CAND_CACHE_SIZE);
+}
+
+int bpf_core_apply(struct bpf_core_ctx *ctx, const struct bpf_core_relo *relo,
+ int relo_idx, void *insn)
+{
+ bool need_cands = relo->kind != BPF_CORE_TYPE_ID_LOCAL;
+ struct bpf_core_cand_list cands = {};
+ struct bpf_core_spec *specs;
+ int err;
+
+ /* ~4k of temp memory necessary to convert LLVM spec like "0:1:0:5"
+ * into arrays of btf_ids of struct fields and array indices.
+ */
+ specs = kcalloc(3, sizeof(*specs), GFP_KERNEL);
+ if (!specs)
+ return -ENOMEM;
+
+ if (need_cands) {
+ struct bpf_cand_cache *cc;
+ int i;
+
+ mutex_lock(&cand_cache_mutex);
+ cc = bpf_core_find_cands(ctx, relo->type_id);
+ if (IS_ERR(cc)) {
+ bpf_log(ctx->log, "target candidate search failed for %d\n",
+ relo->type_id);
+ err = PTR_ERR(cc);
+ goto out;
+ }
+ if (cc->cnt) {
+ cands.cands = kcalloc(cc->cnt, sizeof(*cands.cands), GFP_KERNEL);
+ if (!cands.cands) {
+ err = -ENOMEM;
+ goto out;
+ }
+ }
+ for (i = 0; i < cc->cnt; i++) {
+ bpf_log(ctx->log,
+ "CO-RE relocating %s %s: found target candidate [%d]\n",
+ btf_kind_str[cc->kind], cc->name, cc->cands[i].id);
+ cands.cands[i].btf = cc->cands[i].btf;
+ cands.cands[i].id = cc->cands[i].id;
+ }
+ cands.len = cc->cnt;
+ /* cand_cache_mutex needs to span the cache lookup and
+ * copy of btf pointer into bpf_core_cand_list,
+ * since module can be unloaded while bpf_core_apply_relo_insn
+ * is working with module's btf.
+ */
+ }
+
+ err = bpf_core_apply_relo_insn((void *)ctx->log, insn, relo->insn_off / 8,
+ relo, relo_idx, ctx->btf, &cands, specs);
+out:
+ kfree(specs);
+ if (need_cands) {
+ kfree(cands.cands);
+ mutex_unlock(&cand_cache_mutex);
+ if (ctx->log->level & BPF_LOG_LEVEL2)
+ print_cand_cache(ctx->log);
+ }
+ return err;
+}
diff --git a/kernel/bpf/cgroup.c b/kernel/bpf/cgroup.c
index 43eb350..514b468 100644
--- a/kernel/bpf/cgroup.c
+++ b/kernel/bpf/cgroup.c
@@ -1789,7 +1789,7 @@ static const struct bpf_func_proto bpf_sysctl_set_new_value_proto = {
.gpl_only = false,
.ret_type = RET_INTEGER,
.arg1_type = ARG_PTR_TO_CTX,
- .arg2_type = ARG_PTR_TO_MEM,
+ .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg3_type = ARG_CONST_SIZE,
};
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index 2405e39..de3e5bc 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -1574,7 +1574,8 @@ static u64 ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn)
if (unlikely(index >= array->map.max_entries))
goto out;
- if (unlikely(tail_call_cnt > MAX_TAIL_CALL_CNT))
+
+ if (unlikely(tail_call_cnt >= MAX_TAIL_CALL_CNT))
goto out;
tail_call_cnt++;
@@ -1891,7 +1892,7 @@ static void bpf_prog_select_func(struct bpf_prog *fp)
/**
* bpf_prog_select_runtime - select exec runtime for BPF program
- * @fp: bpf_prog populated with internal BPF program
+ * @fp: bpf_prog populated with BPF program
* @err: pointer to error variable
*
* Try to JIT eBPF program, if JIT is not available, use interpreter.
@@ -2300,7 +2301,6 @@ static void bpf_prog_free_deferred(struct work_struct *work)
}
}
-/* Free internal BPF program */
void bpf_prog_free(struct bpf_prog *fp)
{
struct bpf_prog_aux *aux = fp->aux;
diff --git a/kernel/bpf/cpumap.c b/kernel/bpf/cpumap.c
index 585b2b7..b3e6b94 100644
--- a/kernel/bpf/cpumap.c
+++ b/kernel/bpf/cpumap.c
@@ -195,7 +195,7 @@ static void cpu_map_bpf_prog_run_skb(struct bpf_cpu_map_entry *rcpu,
}
return;
default:
- bpf_warn_invalid_xdp_action(act);
+ bpf_warn_invalid_xdp_action(NULL, rcpu->prog, act);
fallthrough;
case XDP_ABORTED:
trace_xdp_exception(skb->dev, rcpu->prog, act);
@@ -254,7 +254,7 @@ static int cpu_map_bpf_prog_run_xdp(struct bpf_cpu_map_entry *rcpu,
}
break;
default:
- bpf_warn_invalid_xdp_action(act);
+ bpf_warn_invalid_xdp_action(NULL, rcpu->prog, act);
fallthrough;
case XDP_DROP:
xdp_return_frame(xdpf);
@@ -746,15 +746,9 @@ static void bq_enqueue(struct bpf_cpu_map_entry *rcpu, struct xdp_frame *xdpf)
list_add(&bq->flush_node, flush_list);
}
-int cpu_map_enqueue(struct bpf_cpu_map_entry *rcpu, struct xdp_buff *xdp,
+int cpu_map_enqueue(struct bpf_cpu_map_entry *rcpu, struct xdp_frame *xdpf,
struct net_device *dev_rx)
{
- struct xdp_frame *xdpf;
-
- xdpf = xdp_convert_buff_to_frame(xdp);
- if (unlikely(!xdpf))
- return -EOVERFLOW;
-
/* Info needed when constructing SKB on remote CPU */
xdpf->dev_rx = dev_rx;
diff --git a/kernel/bpf/devmap.c b/kernel/bpf/devmap.c
index f02d045..fe019db 100644
--- a/kernel/bpf/devmap.c
+++ b/kernel/bpf/devmap.c
@@ -348,7 +348,7 @@ static int dev_map_bpf_prog_run(struct bpf_prog *xdp_prog,
frames[nframes++] = xdpf;
break;
default:
- bpf_warn_invalid_xdp_action(act);
+ bpf_warn_invalid_xdp_action(NULL, xdp_prog, act);
fallthrough;
case XDP_ABORTED:
trace_xdp_exception(dev, xdp_prog, act);
@@ -467,24 +467,19 @@ static void bq_enqueue(struct net_device *dev, struct xdp_frame *xdpf,
bq->q[bq->count++] = xdpf;
}
-static inline int __xdp_enqueue(struct net_device *dev, struct xdp_buff *xdp,
+static inline int __xdp_enqueue(struct net_device *dev, struct xdp_frame *xdpf,
struct net_device *dev_rx,
struct bpf_prog *xdp_prog)
{
- struct xdp_frame *xdpf;
int err;
if (!dev->netdev_ops->ndo_xdp_xmit)
return -EOPNOTSUPP;
- err = xdp_ok_fwd_dev(dev, xdp->data_end - xdp->data);
+ err = xdp_ok_fwd_dev(dev, xdpf->len);
if (unlikely(err))
return err;
- xdpf = xdp_convert_buff_to_frame(xdp);
- if (unlikely(!xdpf))
- return -EOVERFLOW;
-
bq_enqueue(dev, xdpf, dev_rx, xdp_prog);
return 0;
}
@@ -507,7 +502,7 @@ static u32 dev_map_bpf_prog_run_skb(struct sk_buff *skb, struct bpf_dtab_netdev
__skb_push(skb, skb->mac_len);
break;
default:
- bpf_warn_invalid_xdp_action(act);
+ bpf_warn_invalid_xdp_action(NULL, dst->xdp_prog, act);
fallthrough;
case XDP_ABORTED:
trace_xdp_exception(dst->dev, dst->xdp_prog, act);
@@ -520,27 +515,27 @@ static u32 dev_map_bpf_prog_run_skb(struct sk_buff *skb, struct bpf_dtab_netdev
return act;
}
-int dev_xdp_enqueue(struct net_device *dev, struct xdp_buff *xdp,
+int dev_xdp_enqueue(struct net_device *dev, struct xdp_frame *xdpf,
struct net_device *dev_rx)
{
- return __xdp_enqueue(dev, xdp, dev_rx, NULL);
+ return __xdp_enqueue(dev, xdpf, dev_rx, NULL);
}
-int dev_map_enqueue(struct bpf_dtab_netdev *dst, struct xdp_buff *xdp,
+int dev_map_enqueue(struct bpf_dtab_netdev *dst, struct xdp_frame *xdpf,
struct net_device *dev_rx)
{
struct net_device *dev = dst->dev;
- return __xdp_enqueue(dev, xdp, dev_rx, dst->xdp_prog);
+ return __xdp_enqueue(dev, xdpf, dev_rx, dst->xdp_prog);
}
-static bool is_valid_dst(struct bpf_dtab_netdev *obj, struct xdp_buff *xdp)
+static bool is_valid_dst(struct bpf_dtab_netdev *obj, struct xdp_frame *xdpf)
{
if (!obj ||
!obj->dev->netdev_ops->ndo_xdp_xmit)
return false;
- if (xdp_ok_fwd_dev(obj->dev, xdp->data_end - xdp->data))
+ if (xdp_ok_fwd_dev(obj->dev, xdpf->len))
return false;
return true;
@@ -586,14 +581,13 @@ static int get_upper_ifindexes(struct net_device *dev, int *indexes)
return n;
}
-int dev_map_enqueue_multi(struct xdp_buff *xdp, struct net_device *dev_rx,
+int dev_map_enqueue_multi(struct xdp_frame *xdpf, struct net_device *dev_rx,
struct bpf_map *map, bool exclude_ingress)
{
struct bpf_dtab *dtab = container_of(map, struct bpf_dtab, map);
struct bpf_dtab_netdev *dst, *last_dst = NULL;
int excluded_devices[1+MAX_NEST_DEV];
struct hlist_head *head;
- struct xdp_frame *xdpf;
int num_excluded = 0;
unsigned int i;
int err;
@@ -603,15 +597,11 @@ int dev_map_enqueue_multi(struct xdp_buff *xdp, struct net_device *dev_rx,
excluded_devices[num_excluded++] = dev_rx->ifindex;
}
- xdpf = xdp_convert_buff_to_frame(xdp);
- if (unlikely(!xdpf))
- return -EOVERFLOW;
-
if (map->map_type == BPF_MAP_TYPE_DEVMAP) {
for (i = 0; i < map->max_entries; i++) {
dst = rcu_dereference_check(dtab->netdev_map[i],
rcu_read_lock_bh_held());
- if (!is_valid_dst(dst, xdp))
+ if (!is_valid_dst(dst, xdpf))
continue;
if (is_ifindex_excluded(excluded_devices, num_excluded, dst->dev->ifindex))
@@ -634,7 +624,7 @@ int dev_map_enqueue_multi(struct xdp_buff *xdp, struct net_device *dev_rx,
head = dev_map_index_hash(dtab, i);
hlist_for_each_entry_rcu(dst, head, index_hlist,
lockdep_is_held(&dtab->index_lock)) {
- if (!is_valid_dst(dst, xdp))
+ if (!is_valid_dst(dst, xdpf))
continue;
if (is_ifindex_excluded(excluded_devices, num_excluded,
diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c
index 649f076..01cfdf4 100644
--- a/kernel/bpf/helpers.c
+++ b/kernel/bpf/helpers.c
@@ -2,6 +2,7 @@
/* Copyright (c) 2011-2014 PLUMgrid, http://plumgrid.com
*/
#include <linux/bpf.h>
+#include <linux/bpf-cgroup.h>
#include <linux/rcupdate.h>
#include <linux/random.h>
#include <linux/smp.h>
@@ -530,7 +531,7 @@ const struct bpf_func_proto bpf_strtol_proto = {
.func = bpf_strtol,
.gpl_only = false,
.ret_type = RET_INTEGER,
- .arg1_type = ARG_PTR_TO_MEM,
+ .arg1_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg2_type = ARG_CONST_SIZE,
.arg3_type = ARG_ANYTHING,
.arg4_type = ARG_PTR_TO_LONG,
@@ -558,13 +559,27 @@ const struct bpf_func_proto bpf_strtoul_proto = {
.func = bpf_strtoul,
.gpl_only = false,
.ret_type = RET_INTEGER,
- .arg1_type = ARG_PTR_TO_MEM,
+ .arg1_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg2_type = ARG_CONST_SIZE,
.arg3_type = ARG_ANYTHING,
.arg4_type = ARG_PTR_TO_LONG,
};
#endif
+BPF_CALL_3(bpf_strncmp, const char *, s1, u32, s1_sz, const char *, s2)
+{
+ return strncmp(s1, s2, s1_sz);
+}
+
+const struct bpf_func_proto bpf_strncmp_proto = {
+ .func = bpf_strncmp,
+ .gpl_only = false,
+ .ret_type = RET_INTEGER,
+ .arg1_type = ARG_PTR_TO_MEM,
+ .arg2_type = ARG_CONST_SIZE,
+ .arg3_type = ARG_PTR_TO_CONST_STR,
+};
+
BPF_CALL_4(bpf_get_ns_current_pid_tgid, u64, dev, u64, ino,
struct bpf_pidns_info *, nsdata, u32, size)
{
@@ -630,7 +645,7 @@ const struct bpf_func_proto bpf_event_output_data_proto = {
.arg1_type = ARG_PTR_TO_CTX,
.arg2_type = ARG_CONST_MAP_PTR,
.arg3_type = ARG_ANYTHING,
- .arg4_type = ARG_PTR_TO_MEM,
+ .arg4_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg5_type = ARG_CONST_SIZE_OR_ZERO,
};
@@ -667,7 +682,7 @@ BPF_CALL_2(bpf_per_cpu_ptr, const void *, ptr, u32, cpu)
const struct bpf_func_proto bpf_per_cpu_ptr_proto = {
.func = bpf_per_cpu_ptr,
.gpl_only = false,
- .ret_type = RET_PTR_TO_MEM_OR_BTF_ID_OR_NULL,
+ .ret_type = RET_PTR_TO_MEM_OR_BTF_ID | PTR_MAYBE_NULL | MEM_RDONLY,
.arg1_type = ARG_PTR_TO_PERCPU_BTF_ID,
.arg2_type = ARG_ANYTHING,
};
@@ -680,7 +695,7 @@ BPF_CALL_1(bpf_this_cpu_ptr, const void *, percpu_ptr)
const struct bpf_func_proto bpf_this_cpu_ptr_proto = {
.func = bpf_this_cpu_ptr,
.gpl_only = false,
- .ret_type = RET_PTR_TO_MEM_OR_BTF_ID,
+ .ret_type = RET_PTR_TO_MEM_OR_BTF_ID | MEM_RDONLY,
.arg1_type = ARG_PTR_TO_PERCPU_BTF_ID,
};
@@ -1011,7 +1026,7 @@ const struct bpf_func_proto bpf_snprintf_proto = {
.arg1_type = ARG_PTR_TO_MEM_OR_NULL,
.arg2_type = ARG_CONST_SIZE_OR_ZERO,
.arg3_type = ARG_PTR_TO_CONST_STR,
- .arg4_type = ARG_PTR_TO_MEM_OR_NULL,
+ .arg4_type = ARG_PTR_TO_MEM | PTR_MAYBE_NULL | MEM_RDONLY,
.arg5_type = ARG_CONST_SIZE_OR_ZERO,
};
@@ -1376,6 +1391,10 @@ bpf_base_func_proto(enum bpf_func_id func_id)
return &bpf_ringbuf_query_proto;
case BPF_FUNC_for_each_map_elem:
return &bpf_for_each_map_elem_proto;
+ case BPF_FUNC_loop:
+ return &bpf_loop_proto;
+ case BPF_FUNC_strncmp:
+ return &bpf_strncmp_proto;
default:
break;
}
diff --git a/kernel/bpf/local_storage.c b/kernel/bpf/local_storage.c
index 035e9e3..23f7f9d 100644
--- a/kernel/bpf/local_storage.c
+++ b/kernel/bpf/local_storage.c
@@ -163,8 +163,7 @@ static int cgroup_storage_update_elem(struct bpf_map *map, void *key,
return 0;
}
- new = bpf_map_kmalloc_node(map, sizeof(struct bpf_storage_buffer) +
- map->value_size,
+ new = bpf_map_kmalloc_node(map, struct_size(new, data, map->value_size),
__GFP_ZERO | GFP_ATOMIC | __GFP_NOWARN,
map->numa_node);
if (!new)
diff --git a/kernel/bpf/lpm_trie.c b/kernel/bpf/lpm_trie.c
index 423549d..5763cc7 100644
--- a/kernel/bpf/lpm_trie.c
+++ b/kernel/bpf/lpm_trie.c
@@ -412,7 +412,7 @@ static int trie_update_elem(struct bpf_map *map,
rcu_assign_pointer(im_node->child[1], node);
}
- /* Finally, assign the intermediate node to the determined spot */
+ /* Finally, assign the intermediate node to the determined slot */
rcu_assign_pointer(*slot, im_node);
out:
diff --git a/kernel/bpf/map_iter.c b/kernel/bpf/map_iter.c
index 6a9542a..b0fa190 100644
--- a/kernel/bpf/map_iter.c
+++ b/kernel/bpf/map_iter.c
@@ -174,9 +174,9 @@ static const struct bpf_iter_reg bpf_map_elem_reg_info = {
.ctx_arg_info_size = 2,
.ctx_arg_info = {
{ offsetof(struct bpf_iter__bpf_map_elem, key),
- PTR_TO_RDONLY_BUF_OR_NULL },
+ PTR_TO_BUF | PTR_MAYBE_NULL | MEM_RDONLY },
{ offsetof(struct bpf_iter__bpf_map_elem, value),
- PTR_TO_RDWR_BUF_OR_NULL },
+ PTR_TO_BUF | PTR_MAYBE_NULL },
},
};
diff --git a/kernel/bpf/mmap_unlock_work.h b/kernel/bpf/mmap_unlock_work.h
new file mode 100644
index 0000000..5d18d7d
--- /dev/null
+++ b/kernel/bpf/mmap_unlock_work.h
@@ -0,0 +1,65 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright (c) 2021 Facebook
+ */
+
+#ifndef __MMAP_UNLOCK_WORK_H__
+#define __MMAP_UNLOCK_WORK_H__
+#include <linux/irq_work.h>
+
+/* irq_work to run mmap_read_unlock() in irq_work */
+struct mmap_unlock_irq_work {
+ struct irq_work irq_work;
+ struct mm_struct *mm;
+};
+
+DECLARE_PER_CPU(struct mmap_unlock_irq_work, mmap_unlock_work);
+
+/*
+ * We cannot do mmap_read_unlock() when the irq is disabled, because of
+ * risk to deadlock with rq_lock. To look up vma when the irqs are
+ * disabled, we need to run mmap_read_unlock() in irq_work. We use a
+ * percpu variable to do the irq_work. If the irq_work is already used
+ * by another lookup, we fall over.
+ */
+static inline bool bpf_mmap_unlock_get_irq_work(struct mmap_unlock_irq_work **work_ptr)
+{
+ struct mmap_unlock_irq_work *work = NULL;
+ bool irq_work_busy = false;
+
+ if (irqs_disabled()) {
+ if (!IS_ENABLED(CONFIG_PREEMPT_RT)) {
+ work = this_cpu_ptr(&mmap_unlock_work);
+ if (irq_work_is_busy(&work->irq_work)) {
+ /* cannot queue more up_read, fallback */
+ irq_work_busy = true;
+ }
+ } else {
+ /*
+ * PREEMPT_RT does not allow to trylock mmap sem in
+ * interrupt disabled context. Force the fallback code.
+ */
+ irq_work_busy = true;
+ }
+ }
+
+ *work_ptr = work;
+ return irq_work_busy;
+}
+
+static inline void bpf_mmap_unlock_mm(struct mmap_unlock_irq_work *work, struct mm_struct *mm)
+{
+ if (!work) {
+ mmap_read_unlock(mm);
+ } else {
+ work->mm = mm;
+
+ /* The lock will be released once we're out of interrupt
+ * context. Tell lockdep that we've released it now so
+ * it doesn't complain that we forgot to release it.
+ */
+ rwsem_release(&mm->mmap_lock.dep_map, _RET_IP_);
+ irq_work_queue(&work->irq_work);
+ }
+}
+
+#endif /* __MMAP_UNLOCK_WORK_H__ */
diff --git a/kernel/bpf/net_namespace.c b/kernel/bpf/net_namespace.c
index 542f275..868cc2c 100644
--- a/kernel/bpf/net_namespace.c
+++ b/kernel/bpf/net_namespace.c
@@ -1,6 +1,7 @@
// SPDX-License-Identifier: GPL-2.0
#include <linux/bpf.h>
+#include <linux/bpf-netns.h>
#include <linux/filter.h>
#include <net/net_namespace.h>
diff --git a/kernel/bpf/reuseport_array.c b/kernel/bpf/reuseport_array.c
index 93a5539..556a769 100644
--- a/kernel/bpf/reuseport_array.c
+++ b/kernel/bpf/reuseport_array.c
@@ -152,16 +152,12 @@ static struct bpf_map *reuseport_array_alloc(union bpf_attr *attr)
{
int numa_node = bpf_map_attr_numa_node(attr);
struct reuseport_array *array;
- u64 array_size;
if (!bpf_capable())
return ERR_PTR(-EPERM);
- array_size = sizeof(*array);
- array_size += (u64)attr->max_entries * sizeof(struct sock *);
-
/* allocate all map elements and zero-initialize them */
- array = bpf_map_area_alloc(array_size, numa_node);
+ array = bpf_map_area_alloc(struct_size(array, ptrs, attr->max_entries), numa_node);
if (!array)
return ERR_PTR(-ENOMEM);
diff --git a/kernel/bpf/ringbuf.c b/kernel/bpf/ringbuf.c
index 9e0c10c..638d7fd 100644
--- a/kernel/bpf/ringbuf.c
+++ b/kernel/bpf/ringbuf.c
@@ -444,7 +444,7 @@ const struct bpf_func_proto bpf_ringbuf_output_proto = {
.func = bpf_ringbuf_output,
.ret_type = RET_INTEGER,
.arg1_type = ARG_CONST_MAP_PTR,
- .arg2_type = ARG_PTR_TO_MEM,
+ .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg3_type = ARG_CONST_SIZE_OR_ZERO,
.arg4_type = ARG_ANYTHING,
};
diff --git a/kernel/bpf/stackmap.c b/kernel/bpf/stackmap.c
index 6e75bbe..49e5672 100644
--- a/kernel/bpf/stackmap.c
+++ b/kernel/bpf/stackmap.c
@@ -7,10 +7,10 @@
#include <linux/kernel.h>
#include <linux/stacktrace.h>
#include <linux/perf_event.h>
-#include <linux/irq_work.h>
#include <linux/btf_ids.h>
#include <linux/buildid.h>
#include "percpu_freelist.h"
+#include "mmap_unlock_work.h"
#define STACK_CREATE_FLAG_MASK \
(BPF_F_NUMA_NODE | BPF_F_RDONLY | BPF_F_WRONLY | \
@@ -31,25 +31,6 @@ struct bpf_stack_map {
struct stack_map_bucket *buckets[];
};
-/* irq_work to run up_read() for build_id lookup in nmi context */
-struct stack_map_irq_work {
- struct irq_work irq_work;
- struct mm_struct *mm;
-};
-
-static void do_up_read(struct irq_work *entry)
-{
- struct stack_map_irq_work *work;
-
- if (WARN_ON_ONCE(IS_ENABLED(CONFIG_PREEMPT_RT)))
- return;
-
- work = container_of(entry, struct stack_map_irq_work, irq_work);
- mmap_read_unlock_non_owner(work->mm);
-}
-
-static DEFINE_PER_CPU(struct stack_map_irq_work, up_read_work);
-
static inline bool stack_map_use_build_id(struct bpf_map *map)
{
return (map->map_flags & BPF_F_STACK_BUILD_ID);
@@ -149,35 +130,13 @@ static void stack_map_get_build_id_offset(struct bpf_stack_build_id *id_offs,
u64 *ips, u32 trace_nr, bool user)
{
int i;
+ struct mmap_unlock_irq_work *work = NULL;
+ bool irq_work_busy = bpf_mmap_unlock_get_irq_work(&work);
struct vm_area_struct *vma;
- bool irq_work_busy = false;
- struct stack_map_irq_work *work = NULL;
- if (irqs_disabled()) {
- if (!IS_ENABLED(CONFIG_PREEMPT_RT)) {
- work = this_cpu_ptr(&up_read_work);
- if (irq_work_is_busy(&work->irq_work)) {
- /* cannot queue more up_read, fallback */
- irq_work_busy = true;
- }
- } else {
- /*
- * PREEMPT_RT does not allow to trylock mmap sem in
- * interrupt disabled context. Force the fallback code.
- */
- irq_work_busy = true;
- }
- }
-
- /*
- * We cannot do up_read() when the irq is disabled, because of
- * risk to deadlock with rq_lock. To do build_id lookup when the
- * irqs are disabled, we need to run up_read() in irq_work. We use
- * a percpu variable to do the irq_work. If the irq_work is
- * already used by another lookup, we fall back to report ips.
- *
- * Same fallback is used for kernel stack (!user) on a stackmap
- * with build_id.
+ /* If the irq_work is in use, fall back to report ips. Same
+ * fallback is used for kernel stack (!user) on a stackmap with
+ * build_id.
*/
if (!user || !current || !current->mm || irq_work_busy ||
!mmap_read_trylock(current->mm)) {
@@ -203,19 +162,7 @@ static void stack_map_get_build_id_offset(struct bpf_stack_build_id *id_offs,
- vma->vm_start;
id_offs[i].status = BPF_STACK_BUILD_ID_VALID;
}
-
- if (!work) {
- mmap_read_unlock(current->mm);
- } else {
- work->mm = current->mm;
-
- /* The lock will be released once we're out of interrupt
- * context. Tell lockdep that we've released it now so
- * it doesn't complain that we forgot to release it.
- */
- rwsem_release(¤t->mm->mmap_lock.dep_map, _RET_IP_);
- irq_work_queue(&work->irq_work);
- }
+ bpf_mmap_unlock_mm(work, current->mm);
}
static struct perf_callchain_entry *
@@ -542,7 +489,7 @@ const struct bpf_func_proto bpf_get_task_stack_proto = {
.gpl_only = false,
.ret_type = RET_INTEGER,
.arg1_type = ARG_PTR_TO_BTF_ID,
- .arg1_btf_id = &btf_task_struct_ids[0],
+ .arg1_btf_id = &btf_tracing_ids[BTF_TRACING_TYPE_TASK],
.arg2_type = ARG_PTR_TO_UNINIT_MEM,
.arg3_type = ARG_CONST_SIZE_OR_ZERO,
.arg4_type = ARG_ANYTHING,
@@ -719,16 +666,3 @@ const struct bpf_map_ops stack_trace_map_ops = {
.map_btf_name = "bpf_stack_map",
.map_btf_id = &stack_trace_map_btf_id,
};
-
-static int __init stack_map_init(void)
-{
- int cpu;
- struct stack_map_irq_work *work;
-
- for_each_possible_cpu(cpu) {
- work = per_cpu_ptr(&up_read_work, cpu);
- init_irq_work(&work->irq_work, do_up_read);
- }
- return 0;
-}
-subsys_initcall(stack_map_init);
diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
index 1033ee8..fa4505f 100644
--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -2,6 +2,7 @@
/* Copyright (c) 2011-2014 PLUMgrid, http://plumgrid.com
*/
#include <linux/bpf.h>
+#include <linux/bpf-cgroup.h>
#include <linux/bpf_trace.h>
#include <linux/bpf_lirc.h>
#include <linux/bpf_verifier.h>
@@ -2198,7 +2199,7 @@ static bool is_perfmon_prog_type(enum bpf_prog_type prog_type)
}
/* last field in 'union bpf_attr' used by this command */
-#define BPF_PROG_LOAD_LAST_FIELD fd_array
+#define BPF_PROG_LOAD_LAST_FIELD core_relo_rec_size
static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr)
{
@@ -4772,7 +4773,7 @@ static const struct bpf_func_proto bpf_sys_bpf_proto = {
.gpl_only = false,
.ret_type = RET_INTEGER,
.arg1_type = ARG_ANYTHING,
- .arg2_type = ARG_PTR_TO_MEM,
+ .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg3_type = ARG_CONST_SIZE,
};
@@ -4819,7 +4820,7 @@ const struct bpf_func_proto bpf_kallsyms_lookup_name_proto = {
.gpl_only = false,
.ret_type = RET_INTEGER,
.arg1_type = ARG_PTR_TO_MEM,
- .arg2_type = ARG_CONST_SIZE,
+ .arg2_type = ARG_CONST_SIZE_OR_ZERO,
.arg3_type = ARG_ANYTHING,
.arg4_type = ARG_PTR_TO_LONG,
};
diff --git a/kernel/bpf/task_iter.c b/kernel/bpf/task_iter.c
index b48750b..d946961 100644
--- a/kernel/bpf/task_iter.c
+++ b/kernel/bpf/task_iter.c
@@ -8,6 +8,7 @@
#include <linux/fdtable.h>
#include <linux/filter.h>
#include <linux/btf_ids.h>
+#include "mmap_unlock_work.h"
struct bpf_iter_seq_task_common {
struct pid_namespace *ns;
@@ -524,10 +525,6 @@ static const struct seq_operations task_vma_seq_ops = {
.show = task_vma_seq_show,
};
-BTF_ID_LIST(btf_task_file_ids)
-BTF_ID(struct, file)
-BTF_ID(struct, vm_area_struct)
-
static const struct bpf_iter_seq_info task_seq_info = {
.seq_ops = &task_seq_ops,
.init_seq_private = init_seq_pidns,
@@ -586,23 +583,88 @@ static struct bpf_iter_reg task_vma_reg_info = {
.seq_info = &task_vma_seq_info,
};
+BPF_CALL_5(bpf_find_vma, struct task_struct *, task, u64, start,
+ bpf_callback_t, callback_fn, void *, callback_ctx, u64, flags)
+{
+ struct mmap_unlock_irq_work *work = NULL;
+ struct vm_area_struct *vma;
+ bool irq_work_busy = false;
+ struct mm_struct *mm;
+ int ret = -ENOENT;
+
+ if (flags)
+ return -EINVAL;
+
+ if (!task)
+ return -ENOENT;
+
+ mm = task->mm;
+ if (!mm)
+ return -ENOENT;
+
+ irq_work_busy = bpf_mmap_unlock_get_irq_work(&work);
+
+ if (irq_work_busy || !mmap_read_trylock(mm))
+ return -EBUSY;
+
+ vma = find_vma(mm, start);
+
+ if (vma && vma->vm_start <= start && vma->vm_end > start) {
+ callback_fn((u64)(long)task, (u64)(long)vma,
+ (u64)(long)callback_ctx, 0, 0);
+ ret = 0;
+ }
+ bpf_mmap_unlock_mm(work, mm);
+ return ret;
+}
+
+const struct bpf_func_proto bpf_find_vma_proto = {
+ .func = bpf_find_vma,
+ .ret_type = RET_INTEGER,
+ .arg1_type = ARG_PTR_TO_BTF_ID,
+ .arg1_btf_id = &btf_tracing_ids[BTF_TRACING_TYPE_TASK],
+ .arg2_type = ARG_ANYTHING,
+ .arg3_type = ARG_PTR_TO_FUNC,
+ .arg4_type = ARG_PTR_TO_STACK_OR_NULL,
+ .arg5_type = ARG_ANYTHING,
+};
+
+DEFINE_PER_CPU(struct mmap_unlock_irq_work, mmap_unlock_work);
+
+static void do_mmap_read_unlock(struct irq_work *entry)
+{
+ struct mmap_unlock_irq_work *work;
+
+ if (WARN_ON_ONCE(IS_ENABLED(CONFIG_PREEMPT_RT)))
+ return;
+
+ work = container_of(entry, struct mmap_unlock_irq_work, irq_work);
+ mmap_read_unlock_non_owner(work->mm);
+}
+
static int __init task_iter_init(void)
{
- int ret;
+ struct mmap_unlock_irq_work *work;
+ int ret, cpu;
- task_reg_info.ctx_arg_info[0].btf_id = btf_task_struct_ids[0];
+ for_each_possible_cpu(cpu) {
+ work = per_cpu_ptr(&mmap_unlock_work, cpu);
+ init_irq_work(&work->irq_work, do_mmap_read_unlock);
+ }
+
+ task_reg_info.ctx_arg_info[0].btf_id = btf_tracing_ids[BTF_TRACING_TYPE_TASK];
ret = bpf_iter_reg_target(&task_reg_info);
if (ret)
return ret;
- task_file_reg_info.ctx_arg_info[0].btf_id = btf_task_struct_ids[0];
- task_file_reg_info.ctx_arg_info[1].btf_id = btf_task_file_ids[0];
+ task_file_reg_info.ctx_arg_info[0].btf_id = btf_tracing_ids[BTF_TRACING_TYPE_TASK];
+ task_file_reg_info.ctx_arg_info[1].btf_id = btf_tracing_ids[BTF_TRACING_TYPE_FILE];
ret = bpf_iter_reg_target(&task_file_reg_info);
if (ret)
return ret;
- task_vma_reg_info.ctx_arg_info[0].btf_id = btf_task_struct_ids[0];
- task_vma_reg_info.ctx_arg_info[1].btf_id = btf_task_file_ids[1];
+ task_vma_reg_info.ctx_arg_info[0].btf_id = btf_tracing_ids[BTF_TRACING_TYPE_TASK];
+ task_vma_reg_info.ctx_arg_info[1].btf_id = btf_tracing_ids[BTF_TRACING_TYPE_VMA];
return bpf_iter_reg_target(&task_vma_reg_info);
}
late_initcall(task_iter_init);
diff --git a/kernel/bpf/trampoline.c b/kernel/bpf/trampoline.c
index e98de5e..4b6974a 100644
--- a/kernel/bpf/trampoline.c
+++ b/kernel/bpf/trampoline.c
@@ -27,6 +27,14 @@ static struct hlist_head trampoline_table[TRAMPOLINE_TABLE_SIZE];
/* serializes access to trampoline_table */
static DEFINE_MUTEX(trampoline_mutex);
+bool bpf_prog_has_trampoline(const struct bpf_prog *prog)
+{
+ enum bpf_attach_type eatype = prog->expected_attach_type;
+
+ return eatype == BPF_TRACE_FENTRY || eatype == BPF_TRACE_FEXIT ||
+ eatype == BPF_MODIFY_RETURN;
+}
+
void *bpf_jit_alloc_exec_page(void)
{
void *image;
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index b532f10..bfb4538 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -4,6 +4,7 @@
* Copyright (c) 2018 Covalent IO, Inc. http://covalent.io
*/
#include <uapi/linux/btf.h>
+#include <linux/bpf-cgroup.h>
#include <linux/kernel.h>
#include <linux/types.h>
#include <linux/slab.h>
@@ -293,13 +294,15 @@ void bpf_verifier_vlog(struct bpf_verifier_log *log, const char *fmt,
WARN_ONCE(n >= BPF_VERIFIER_TMP_LOG_SIZE - 1,
"verifier log line truncated - local buffer too short\n");
- n = min(log->len_total - log->len_used - 1, n);
- log->kbuf[n] = '\0';
-
if (log->level == BPF_LOG_KERNEL) {
- pr_err("BPF:%s\n", log->kbuf);
+ bool newline = n > 0 && log->kbuf[n - 1] == '\n';
+
+ pr_err("BPF: %s%s", log->kbuf, newline ? "" : "\n");
return;
}
+
+ n = min(log->len_total - log->len_used - 1, n);
+ log->kbuf[n] = '\0';
if (!copy_to_user(log->ubuf + log->len_used, log->kbuf, n + 1))
log->len_used += n;
else
@@ -439,18 +442,6 @@ static bool reg_type_not_null(enum bpf_reg_type type)
type == PTR_TO_SOCK_COMMON;
}
-static bool reg_type_may_be_null(enum bpf_reg_type type)
-{
- return type == PTR_TO_MAP_VALUE_OR_NULL ||
- type == PTR_TO_SOCKET_OR_NULL ||
- type == PTR_TO_SOCK_COMMON_OR_NULL ||
- type == PTR_TO_TCP_SOCK_OR_NULL ||
- type == PTR_TO_BTF_ID_OR_NULL ||
- type == PTR_TO_MEM_OR_NULL ||
- type == PTR_TO_RDONLY_BUF_OR_NULL ||
- type == PTR_TO_RDWR_BUF_OR_NULL;
-}
-
static bool reg_may_point_to_spin_lock(const struct bpf_reg_state *reg)
{
return reg->type == PTR_TO_MAP_VALUE &&
@@ -459,12 +450,14 @@ static bool reg_may_point_to_spin_lock(const struct bpf_reg_state *reg)
static bool reg_type_may_be_refcounted_or_null(enum bpf_reg_type type)
{
- return type == PTR_TO_SOCKET ||
- type == PTR_TO_SOCKET_OR_NULL ||
- type == PTR_TO_TCP_SOCK ||
- type == PTR_TO_TCP_SOCK_OR_NULL ||
- type == PTR_TO_MEM ||
- type == PTR_TO_MEM_OR_NULL;
+ return base_type(type) == PTR_TO_SOCKET ||
+ base_type(type) == PTR_TO_TCP_SOCK ||
+ base_type(type) == PTR_TO_MEM;
+}
+
+static bool type_is_rdonly_mem(u32 type)
+{
+ return type & MEM_RDONLY;
}
static bool arg_type_may_be_refcounted(enum bpf_arg_type type)
@@ -472,14 +465,9 @@ static bool arg_type_may_be_refcounted(enum bpf_arg_type type)
return type == ARG_PTR_TO_SOCK_COMMON;
}
-static bool arg_type_may_be_null(enum bpf_arg_type type)
+static bool type_may_be_null(u32 type)
{
- return type == ARG_PTR_TO_MAP_VALUE_OR_NULL ||
- type == ARG_PTR_TO_MEM_OR_NULL ||
- type == ARG_PTR_TO_CTX_OR_NULL ||
- type == ARG_PTR_TO_SOCKET_OR_NULL ||
- type == ARG_PTR_TO_ALLOC_MEM_OR_NULL ||
- type == ARG_PTR_TO_STACK_OR_NULL;
+ return type & PTR_MAYBE_NULL;
}
/* Determine whether the function releases some resources allocated by another
@@ -539,39 +527,54 @@ static bool is_cmpxchg_insn(const struct bpf_insn *insn)
insn->imm == BPF_CMPXCHG;
}
-/* string representation of 'enum bpf_reg_type' */
-static const char * const reg_type_str[] = {
- [NOT_INIT] = "?",
- [SCALAR_VALUE] = "inv",
- [PTR_TO_CTX] = "ctx",
- [CONST_PTR_TO_MAP] = "map_ptr",
- [PTR_TO_MAP_VALUE] = "map_value",
- [PTR_TO_MAP_VALUE_OR_NULL] = "map_value_or_null",
- [PTR_TO_STACK] = "fp",
- [PTR_TO_PACKET] = "pkt",
- [PTR_TO_PACKET_META] = "pkt_meta",
- [PTR_TO_PACKET_END] = "pkt_end",
- [PTR_TO_FLOW_KEYS] = "flow_keys",
- [PTR_TO_SOCKET] = "sock",
- [PTR_TO_SOCKET_OR_NULL] = "sock_or_null",
- [PTR_TO_SOCK_COMMON] = "sock_common",
- [PTR_TO_SOCK_COMMON_OR_NULL] = "sock_common_or_null",
- [PTR_TO_TCP_SOCK] = "tcp_sock",
- [PTR_TO_TCP_SOCK_OR_NULL] = "tcp_sock_or_null",
- [PTR_TO_TP_BUFFER] = "tp_buffer",
- [PTR_TO_XDP_SOCK] = "xdp_sock",
- [PTR_TO_BTF_ID] = "ptr_",
- [PTR_TO_BTF_ID_OR_NULL] = "ptr_or_null_",
- [PTR_TO_PERCPU_BTF_ID] = "percpu_ptr_",
- [PTR_TO_MEM] = "mem",
- [PTR_TO_MEM_OR_NULL] = "mem_or_null",
- [PTR_TO_RDONLY_BUF] = "rdonly_buf",
- [PTR_TO_RDONLY_BUF_OR_NULL] = "rdonly_buf_or_null",
- [PTR_TO_RDWR_BUF] = "rdwr_buf",
- [PTR_TO_RDWR_BUF_OR_NULL] = "rdwr_buf_or_null",
- [PTR_TO_FUNC] = "func",
- [PTR_TO_MAP_KEY] = "map_key",
-};
+/* string representation of 'enum bpf_reg_type'
+ *
+ * Note that reg_type_str() can not appear more than once in a single verbose()
+ * statement.
+ */
+static const char *reg_type_str(struct bpf_verifier_env *env,
+ enum bpf_reg_type type)
+{
+ char postfix[16] = {0}, prefix[16] = {0};
+ static const char * const str[] = {
+ [NOT_INIT] = "?",
+ [SCALAR_VALUE] = "inv",
+ [PTR_TO_CTX] = "ctx",
+ [CONST_PTR_TO_MAP] = "map_ptr",
+ [PTR_TO_MAP_VALUE] = "map_value",
+ [PTR_TO_STACK] = "fp",
+ [PTR_TO_PACKET] = "pkt",
+ [PTR_TO_PACKET_META] = "pkt_meta",
+ [PTR_TO_PACKET_END] = "pkt_end",
+ [PTR_TO_FLOW_KEYS] = "flow_keys",
+ [PTR_TO_SOCKET] = "sock",
+ [PTR_TO_SOCK_COMMON] = "sock_common",
+ [PTR_TO_TCP_SOCK] = "tcp_sock",
+ [PTR_TO_TP_BUFFER] = "tp_buffer",
+ [PTR_TO_XDP_SOCK] = "xdp_sock",
+ [PTR_TO_BTF_ID] = "ptr_",
+ [PTR_TO_PERCPU_BTF_ID] = "percpu_ptr_",
+ [PTR_TO_MEM] = "mem",
+ [PTR_TO_BUF] = "buf",
+ [PTR_TO_FUNC] = "func",
+ [PTR_TO_MAP_KEY] = "map_key",
+ };
+
+ if (type & PTR_MAYBE_NULL) {
+ if (base_type(type) == PTR_TO_BTF_ID ||
+ base_type(type) == PTR_TO_PERCPU_BTF_ID)
+ strncpy(postfix, "or_null_", 16);
+ else
+ strncpy(postfix, "_or_null", 16);
+ }
+
+ if (type & MEM_RDONLY)
+ strncpy(prefix, "rdonly_", 16);
+
+ snprintf(env->type_str_buf, TYPE_STR_BUF_LEN, "%s%s%s",
+ prefix, str[base_type(type)], postfix);
+ return env->type_str_buf;
+}
static char slot_type_char[] = {
[STACK_INVALID] = '?',
@@ -606,6 +609,44 @@ static const char *kernel_type_name(const struct btf* btf, u32 id)
return btf_name_by_offset(btf, btf_type_by_id(btf, id)->name_off);
}
+static void mark_reg_scratched(struct bpf_verifier_env *env, u32 regno)
+{
+ env->scratched_regs |= 1U << regno;
+}
+
+static void mark_stack_slot_scratched(struct bpf_verifier_env *env, u32 spi)
+{
+ env->scratched_stack_slots |= 1UL << spi;
+}
+
+static bool reg_scratched(const struct bpf_verifier_env *env, u32 regno)
+{
+ return (env->scratched_regs >> regno) & 1;
+}
+
+static bool stack_slot_scratched(const struct bpf_verifier_env *env, u64 regno)
+{
+ return (env->scratched_stack_slots >> regno) & 1;
+}
+
+static bool verifier_state_scratched(const struct bpf_verifier_env *env)
+{
+ return env->scratched_regs || env->scratched_stack_slots;
+}
+
+static void mark_verifier_state_clean(struct bpf_verifier_env *env)
+{
+ env->scratched_regs = 0U;
+ env->scratched_stack_slots = 0UL;
+}
+
+/* Used for printing the entire verifier state. */
+static void mark_verifier_state_scratched(struct bpf_verifier_env *env)
+{
+ env->scratched_regs = ~0U;
+ env->scratched_stack_slots = ~0UL;
+}
+
/* The reg state of a pointer or a bounded scalar was saved when
* it was spilled to the stack.
*/
@@ -621,7 +662,8 @@ static void scrub_spilled_slot(u8 *stype)
}
static void print_verifier_state(struct bpf_verifier_env *env,
- const struct bpf_func_state *state)
+ const struct bpf_func_state *state,
+ bool print_all)
{
const struct bpf_reg_state *reg;
enum bpf_reg_type t;
@@ -634,9 +676,11 @@ static void print_verifier_state(struct bpf_verifier_env *env,
t = reg->type;
if (t == NOT_INIT)
continue;
+ if (!print_all && !reg_scratched(env, i))
+ continue;
verbose(env, " R%d", i);
print_liveness(env, reg->live);
- verbose(env, "=%s", reg_type_str[t]);
+ verbose(env, "=%s", reg_type_str(env, t));
if (t == SCALAR_VALUE && reg->precise)
verbose(env, "P");
if ((t == SCALAR_VALUE || t == PTR_TO_STACK) &&
@@ -644,9 +688,8 @@ static void print_verifier_state(struct bpf_verifier_env *env,
/* reg->off should be 0 for SCALAR_VALUE */
verbose(env, "%lld", reg->var_off.value + reg->off);
} else {
- if (t == PTR_TO_BTF_ID ||
- t == PTR_TO_BTF_ID_OR_NULL ||
- t == PTR_TO_PERCPU_BTF_ID)
+ if (base_type(t) == PTR_TO_BTF_ID ||
+ base_type(t) == PTR_TO_PERCPU_BTF_ID)
verbose(env, "%s", kernel_type_name(reg->btf, reg->btf_id));
verbose(env, "(id=%d", reg->id);
if (reg_type_may_be_refcounted_or_null(t))
@@ -655,10 +698,9 @@ static void print_verifier_state(struct bpf_verifier_env *env,
verbose(env, ",off=%d", reg->off);
if (type_is_pkt_pointer(t))
verbose(env, ",r=%d", reg->range);
- else if (t == CONST_PTR_TO_MAP ||
- t == PTR_TO_MAP_KEY ||
- t == PTR_TO_MAP_VALUE ||
- t == PTR_TO_MAP_VALUE_OR_NULL)
+ else if (base_type(t) == CONST_PTR_TO_MAP ||
+ base_type(t) == PTR_TO_MAP_KEY ||
+ base_type(t) == PTR_TO_MAP_VALUE)
verbose(env, ",ks=%d,vs=%d",
reg->map_ptr->key_size,
reg->map_ptr->value_size);
@@ -723,12 +765,14 @@ static void print_verifier_state(struct bpf_verifier_env *env,
types_buf[BPF_REG_SIZE] = 0;
if (!valid)
continue;
+ if (!print_all && !stack_slot_scratched(env, i))
+ continue;
verbose(env, " fp%d", (-i - 1) * BPF_REG_SIZE);
print_liveness(env, state->stack[i].spilled_ptr.live);
if (is_spilled_reg(&state->stack[i])) {
reg = &state->stack[i].spilled_ptr;
t = reg->type;
- verbose(env, "=%s", reg_type_str[t]);
+ verbose(env, "=%s", reg_type_str(env, t));
if (t == SCALAR_VALUE && reg->precise)
verbose(env, "P");
if (t == SCALAR_VALUE && tnum_is_const(reg->var_off))
@@ -748,6 +792,26 @@ static void print_verifier_state(struct bpf_verifier_env *env,
if (state->in_async_callback_fn)
verbose(env, " async_cb");
verbose(env, "\n");
+ mark_verifier_state_clean(env);
+}
+
+static inline u32 vlog_alignment(u32 pos)
+{
+ return round_up(max(pos + BPF_LOG_MIN_ALIGNMENT / 2, BPF_LOG_ALIGNMENT),
+ BPF_LOG_MIN_ALIGNMENT) - pos - 1;
+}
+
+static void print_insn_state(struct bpf_verifier_env *env,
+ const struct bpf_func_state *state)
+{
+ if (env->prev_log_len && env->prev_log_len == env->log.len_used) {
+ /* remove new line character */
+ bpf_vlog_reset(&env->log, env->prev_log_len - 1);
+ verbose(env, "%*c;", vlog_alignment(env->prev_insn_print_len), ' ');
+ } else {
+ verbose(env, "%d:", env->insn_idx);
+ }
+ print_verifier_state(env, state, false);
}
/* copy array src of length n * size bytes to dst. dst is reallocated if it's too
@@ -1141,8 +1205,7 @@ static void mark_reg_known_zero(struct bpf_verifier_env *env,
static void mark_ptr_not_null_reg(struct bpf_reg_state *reg)
{
- switch (reg->type) {
- case PTR_TO_MAP_VALUE_OR_NULL: {
+ if (base_type(reg->type) == PTR_TO_MAP_VALUE) {
const struct bpf_map *map = reg->map_ptr;
if (map->inner_map_meta) {
@@ -1161,32 +1224,10 @@ static void mark_ptr_not_null_reg(struct bpf_reg_state *reg)
} else {
reg->type = PTR_TO_MAP_VALUE;
}
- break;
+ return;
}
- case PTR_TO_SOCKET_OR_NULL:
- reg->type = PTR_TO_SOCKET;
- break;
- case PTR_TO_SOCK_COMMON_OR_NULL:
- reg->type = PTR_TO_SOCK_COMMON;
- break;
- case PTR_TO_TCP_SOCK_OR_NULL:
- reg->type = PTR_TO_TCP_SOCK;
- break;
- case PTR_TO_BTF_ID_OR_NULL:
- reg->type = PTR_TO_BTF_ID;
- break;
- case PTR_TO_MEM_OR_NULL:
- reg->type = PTR_TO_MEM;
- break;
- case PTR_TO_RDONLY_BUF_OR_NULL:
- reg->type = PTR_TO_RDONLY_BUF;
- break;
- case PTR_TO_RDWR_BUF_OR_NULL:
- reg->type = PTR_TO_RDWR_BUF;
- break;
- default:
- WARN_ONCE(1, "unknown nullable register type");
- }
+
+ reg->type &= ~PTR_MAYBE_NULL;
}
static bool reg_is_pkt_pointer(const struct bpf_reg_state *reg)
@@ -1544,6 +1585,7 @@ static void init_func_state(struct bpf_verifier_env *env,
state->frameno = frameno;
state->subprogno = subprogno;
init_reg_state(env, state);
+ mark_verifier_state_scratched(env);
}
/* Similar to push_stack(), but for async callbacks */
@@ -2047,7 +2089,7 @@ static int mark_reg_read(struct bpf_verifier_env *env,
break;
if (parent->live & REG_LIVE_DONE) {
verbose(env, "verifier BUG type %s var_off %lld off %d\n",
- reg_type_str[parent->type],
+ reg_type_str(env, parent->type),
parent->var_off.value, parent->off);
return -EFAULT;
}
@@ -2231,6 +2273,8 @@ static int check_reg_arg(struct bpf_verifier_env *env, u32 regno,
return -EINVAL;
}
+ mark_reg_scratched(env, regno);
+
reg = ®s[regno];
rw64 = is_reg64(env, insn, regno, reg, t);
if (t == SRC_OP) {
@@ -2335,7 +2379,7 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx,
if (insn->code == 0)
return 0;
- if (env->log.level & BPF_LOG_LEVEL) {
+ if (env->log.level & BPF_LOG_LEVEL2) {
verbose(env, "regs=%x stack=%llx before ", *reg_mask, *stack_mask);
verbose(env, "%d: ", idx);
print_bpf_insn(&cbs, insn, env->allow_ptr_leaks);
@@ -2589,7 +2633,7 @@ static int __mark_chain_precision(struct bpf_verifier_env *env, int regno,
DECLARE_BITMAP(mask, 64);
u32 history = st->jmp_history_cnt;
- if (env->log.level & BPF_LOG_LEVEL)
+ if (env->log.level & BPF_LOG_LEVEL2)
verbose(env, "last_idx %d first_idx %d\n", last_idx, first_idx);
for (i = last_idx;;) {
if (skip_first) {
@@ -2676,11 +2720,11 @@ static int __mark_chain_precision(struct bpf_verifier_env *env, int regno,
new_marks = true;
reg->precise = true;
}
- if (env->log.level & BPF_LOG_LEVEL) {
- print_verifier_state(env, func);
- verbose(env, "parent %s regs=%x stack=%llx marks\n",
+ if (env->log.level & BPF_LOG_LEVEL2) {
+ verbose(env, "parent %s regs=%x stack=%llx marks:",
new_marks ? "didn't have" : "already had",
reg_mask, stack_mask);
+ print_verifier_state(env, func, true);
}
if (!reg_mask && !stack_mask)
@@ -2706,9 +2750,8 @@ static int mark_chain_precision_stack(struct bpf_verifier_env *env, int spi)
static bool is_spillable_regtype(enum bpf_reg_type type)
{
- switch (type) {
+ switch (base_type(type)) {
case PTR_TO_MAP_VALUE:
- case PTR_TO_MAP_VALUE_OR_NULL:
case PTR_TO_STACK:
case PTR_TO_CTX:
case PTR_TO_PACKET:
@@ -2717,21 +2760,13 @@ static bool is_spillable_regtype(enum bpf_reg_type type)
case PTR_TO_FLOW_KEYS:
case CONST_PTR_TO_MAP:
case PTR_TO_SOCKET:
- case PTR_TO_SOCKET_OR_NULL:
case PTR_TO_SOCK_COMMON:
- case PTR_TO_SOCK_COMMON_OR_NULL:
case PTR_TO_TCP_SOCK:
- case PTR_TO_TCP_SOCK_OR_NULL:
case PTR_TO_XDP_SOCK:
case PTR_TO_BTF_ID:
- case PTR_TO_BTF_ID_OR_NULL:
- case PTR_TO_RDONLY_BUF:
- case PTR_TO_RDONLY_BUF_OR_NULL:
- case PTR_TO_RDWR_BUF:
- case PTR_TO_RDWR_BUF_OR_NULL:
+ case PTR_TO_BUF:
case PTR_TO_PERCPU_BTF_ID:
case PTR_TO_MEM:
- case PTR_TO_MEM_OR_NULL:
case PTR_TO_FUNC:
case PTR_TO_MAP_KEY:
return true;
@@ -2836,6 +2871,7 @@ static int check_stack_write_fixed_off(struct bpf_verifier_env *env,
env->insn_aux_data[insn_idx].sanitize_stack_spill = true;
}
+ mark_stack_slot_scratched(env, spi);
if (reg && !(off % BPF_REG_SIZE) && register_is_bounded(reg) &&
!register_is_null(reg) && env->bpf_capable) {
if (dst_reg != BPF_REG_FP) {
@@ -2957,6 +2993,7 @@ static int check_stack_write_var_off(struct bpf_verifier_env *env,
slot = -i - 1;
spi = slot / BPF_REG_SIZE;
stype = &state->stack[spi].slot_type[slot % BPF_REG_SIZE];
+ mark_stack_slot_scratched(env, spi);
if (!env->allow_ptr_leaks
&& *stype != NOT_INIT
@@ -3373,11 +3410,8 @@ static int check_mem_region_access(struct bpf_verifier_env *env, u32 regno,
/* We may have adjusted the register pointing to memory region, so we
* need to try adding each of min_value and max_value to off
* to make sure our theoretical access will be safe.
- */
- if (env->log.level & BPF_LOG_LEVEL)
- print_verifier_state(env, state);
-
- /* The minimum value is only important with signed
+ *
+ * The minimum value is only important with signed
* comparisons where we can't assume the floor of a
* value is 0. If we are using signed variables for our
* index'es we need to make sure that whatever we use
@@ -3572,7 +3606,7 @@ static int check_ctx_access(struct bpf_verifier_env *env, int insn_idx, int off,
*/
*reg_type = info.reg_type;
- if (*reg_type == PTR_TO_BTF_ID || *reg_type == PTR_TO_BTF_ID_OR_NULL) {
+ if (base_type(*reg_type) == PTR_TO_BTF_ID) {
*btf = info.btf;
*btf_id = info.btf_id;
} else {
@@ -3640,7 +3674,7 @@ static int check_sock_access(struct bpf_verifier_env *env, int insn_idx,
}
verbose(env, "R%d invalid %s access off=%d size=%d\n",
- regno, reg_type_str[reg->type], off, size);
+ regno, reg_type_str(env, reg->type), off, size);
return -EACCES;
}
@@ -4367,15 +4401,30 @@ static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, u32 regn
mark_reg_unknown(env, regs, value_regno);
}
}
- } else if (reg->type == PTR_TO_MEM) {
+ } else if (base_type(reg->type) == PTR_TO_MEM) {
+ bool rdonly_mem = type_is_rdonly_mem(reg->type);
+
+ if (type_may_be_null(reg->type)) {
+ verbose(env, "R%d invalid mem access '%s'\n", regno,
+ reg_type_str(env, reg->type));
+ return -EACCES;
+ }
+
+ if (t == BPF_WRITE && rdonly_mem) {
+ verbose(env, "R%d cannot write into %s\n",
+ regno, reg_type_str(env, reg->type));
+ return -EACCES;
+ }
+
if (t == BPF_WRITE && value_regno >= 0 &&
is_pointer_value(env, value_regno)) {
verbose(env, "R%d leaks addr into mem\n", value_regno);
return -EACCES;
}
+
err = check_mem_region_access(env, regno, off, size,
reg->mem_size, false);
- if (!err && t == BPF_READ && value_regno >= 0)
+ if (!err && value_regno >= 0 && (t == BPF_READ || rdonly_mem))
mark_reg_unknown(env, regs, value_regno);
} else if (reg->type == PTR_TO_CTX) {
enum bpf_reg_type reg_type = SCALAR_VALUE;
@@ -4405,7 +4454,7 @@ static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, u32 regn
} else {
mark_reg_known_zero(env, regs,
value_regno);
- if (reg_type_may_be_null(reg_type))
+ if (type_may_be_null(reg_type))
regs[value_regno].id = ++env->id_gen;
/* A load of ctx field could have different
* actual load size with the one encoded in the
@@ -4413,8 +4462,7 @@ static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, u32 regn
* a sub-register.
*/
regs[value_regno].subreg_def = DEF_NOT_SUBREG;
- if (reg_type == PTR_TO_BTF_ID ||
- reg_type == PTR_TO_BTF_ID_OR_NULL) {
+ if (base_type(reg_type) == PTR_TO_BTF_ID) {
regs[value_regno].btf = btf;
regs[value_regno].btf_id = btf_id;
}
@@ -4467,7 +4515,7 @@ static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, u32 regn
} else if (type_is_sk_pointer(reg->type)) {
if (t == BPF_WRITE) {
verbose(env, "R%d cannot write into %s\n",
- regno, reg_type_str[reg->type]);
+ regno, reg_type_str(env, reg->type));
return -EACCES;
}
err = check_sock_access(env, insn_idx, regno, off, size, t);
@@ -4483,26 +4531,32 @@ static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, u32 regn
} else if (reg->type == CONST_PTR_TO_MAP) {
err = check_ptr_to_map_access(env, regs, regno, off, size, t,
value_regno);
- } else if (reg->type == PTR_TO_RDONLY_BUF) {
- if (t == BPF_WRITE) {
- verbose(env, "R%d cannot write into %s\n",
- regno, reg_type_str[reg->type]);
- return -EACCES;
+ } else if (base_type(reg->type) == PTR_TO_BUF) {
+ bool rdonly_mem = type_is_rdonly_mem(reg->type);
+ const char *buf_info;
+ u32 *max_access;
+
+ if (rdonly_mem) {
+ if (t == BPF_WRITE) {
+ verbose(env, "R%d cannot write into %s\n",
+ regno, reg_type_str(env, reg->type));
+ return -EACCES;
+ }
+ buf_info = "rdonly";
+ max_access = &env->prog->aux->max_rdonly_access;
+ } else {
+ buf_info = "rdwr";
+ max_access = &env->prog->aux->max_rdwr_access;
}
+
err = check_buffer_access(env, reg, regno, off, size, false,
- "rdonly",
- &env->prog->aux->max_rdonly_access);
- if (!err && value_regno >= 0)
- mark_reg_unknown(env, regs, value_regno);
- } else if (reg->type == PTR_TO_RDWR_BUF) {
- err = check_buffer_access(env, reg, regno, off, size, false,
- "rdwr",
- &env->prog->aux->max_rdwr_access);
- if (!err && t == BPF_READ && value_regno >= 0)
+ buf_info, max_access);
+
+ if (!err && value_regno >= 0 && (rdonly_mem || t == BPF_READ))
mark_reg_unknown(env, regs, value_regno);
} else {
verbose(env, "R%d invalid mem access '%s'\n", regno,
- reg_type_str[reg->type]);
+ reg_type_str(env, reg->type));
return -EACCES;
}
@@ -4576,7 +4630,7 @@ static int check_atomic(struct bpf_verifier_env *env, int insn_idx, struct bpf_i
is_sk_reg(env, insn->dst_reg)) {
verbose(env, "BPF_ATOMIC stores into R%d %s is not allowed\n",
insn->dst_reg,
- reg_type_str[reg_state(env, insn->dst_reg)->type]);
+ reg_type_str(env, reg_state(env, insn->dst_reg)->type));
return -EACCES;
}
@@ -4759,8 +4813,10 @@ static int check_helper_mem_access(struct bpf_verifier_env *env, int regno,
struct bpf_call_arg_meta *meta)
{
struct bpf_reg_state *regs = cur_regs(env), *reg = ®s[regno];
+ const char *buf_info;
+ u32 *max_access;
- switch (reg->type) {
+ switch (base_type(reg->type)) {
case PTR_TO_PACKET:
case PTR_TO_PACKET_META:
return check_packet_access(env, regno, reg->off, access_size,
@@ -4779,18 +4835,20 @@ static int check_helper_mem_access(struct bpf_verifier_env *env, int regno,
return check_mem_region_access(env, regno, reg->off,
access_size, reg->mem_size,
zero_size_allowed);
- case PTR_TO_RDONLY_BUF:
- if (meta && meta->raw_mode)
- return -EACCES;
+ case PTR_TO_BUF:
+ if (type_is_rdonly_mem(reg->type)) {
+ if (meta && meta->raw_mode)
+ return -EACCES;
+
+ buf_info = "rdonly";
+ max_access = &env->prog->aux->max_rdonly_access;
+ } else {
+ buf_info = "rdwr";
+ max_access = &env->prog->aux->max_rdwr_access;
+ }
return check_buffer_access(env, reg, regno, reg->off,
access_size, zero_size_allowed,
- "rdonly",
- &env->prog->aux->max_rdonly_access);
- case PTR_TO_RDWR_BUF:
- return check_buffer_access(env, reg, regno, reg->off,
- access_size, zero_size_allowed,
- "rdwr",
- &env->prog->aux->max_rdwr_access);
+ buf_info, max_access);
case PTR_TO_STACK:
return check_stack_range_initialized(
env,
@@ -4802,9 +4860,9 @@ static int check_helper_mem_access(struct bpf_verifier_env *env, int regno,
register_is_null(reg))
return 0;
- verbose(env, "R%d type=%s expected=%s\n", regno,
- reg_type_str[reg->type],
- reg_type_str[PTR_TO_STACK]);
+ verbose(env, "R%d type=%s ", regno,
+ reg_type_str(env, reg->type));
+ verbose(env, "expected=%s\n", reg_type_str(env, PTR_TO_STACK));
return -EACCES;
}
}
@@ -4815,7 +4873,7 @@ int check_mem_reg(struct bpf_verifier_env *env, struct bpf_reg_state *reg,
if (register_is_null(reg))
return 0;
- if (reg_type_may_be_null(reg->type)) {
+ if (type_may_be_null(reg->type)) {
/* Assuming that the register contains a value check if the memory
* access is safe. Temporarily save and restore the register's state as
* the conversion shouldn't be visible to a caller.
@@ -4963,9 +5021,8 @@ static int process_timer_func(struct bpf_verifier_env *env, int regno,
static bool arg_type_is_mem_ptr(enum bpf_arg_type type)
{
- return type == ARG_PTR_TO_MEM ||
- type == ARG_PTR_TO_MEM_OR_NULL ||
- type == ARG_PTR_TO_UNINIT_MEM;
+ return base_type(type) == ARG_PTR_TO_MEM ||
+ base_type(type) == ARG_PTR_TO_UNINIT_MEM;
}
static bool arg_type_is_mem_size(enum bpf_arg_type type)
@@ -5070,8 +5127,7 @@ static const struct bpf_reg_types mem_types = {
PTR_TO_MAP_KEY,
PTR_TO_MAP_VALUE,
PTR_TO_MEM,
- PTR_TO_RDONLY_BUF,
- PTR_TO_RDWR_BUF,
+ PTR_TO_BUF,
},
};
@@ -5102,31 +5158,26 @@ static const struct bpf_reg_types *compatible_reg_types[__BPF_ARG_TYPE_MAX] = {
[ARG_PTR_TO_MAP_KEY] = &map_key_value_types,
[ARG_PTR_TO_MAP_VALUE] = &map_key_value_types,
[ARG_PTR_TO_UNINIT_MAP_VALUE] = &map_key_value_types,
- [ARG_PTR_TO_MAP_VALUE_OR_NULL] = &map_key_value_types,
[ARG_CONST_SIZE] = &scalar_types,
[ARG_CONST_SIZE_OR_ZERO] = &scalar_types,
[ARG_CONST_ALLOC_SIZE_OR_ZERO] = &scalar_types,
[ARG_CONST_MAP_PTR] = &const_map_ptr_types,
[ARG_PTR_TO_CTX] = &context_types,
- [ARG_PTR_TO_CTX_OR_NULL] = &context_types,
[ARG_PTR_TO_SOCK_COMMON] = &sock_types,
#ifdef CONFIG_NET
[ARG_PTR_TO_BTF_ID_SOCK_COMMON] = &btf_id_sock_common_types,
#endif
[ARG_PTR_TO_SOCKET] = &fullsock_types,
- [ARG_PTR_TO_SOCKET_OR_NULL] = &fullsock_types,
[ARG_PTR_TO_BTF_ID] = &btf_ptr_types,
[ARG_PTR_TO_SPIN_LOCK] = &spin_lock_types,
[ARG_PTR_TO_MEM] = &mem_types,
- [ARG_PTR_TO_MEM_OR_NULL] = &mem_types,
[ARG_PTR_TO_UNINIT_MEM] = &mem_types,
[ARG_PTR_TO_ALLOC_MEM] = &alloc_mem_types,
- [ARG_PTR_TO_ALLOC_MEM_OR_NULL] = &alloc_mem_types,
[ARG_PTR_TO_INT] = &int_ptr_types,
[ARG_PTR_TO_LONG] = &int_ptr_types,
[ARG_PTR_TO_PERCPU_BTF_ID] = &percpu_btf_ptr_types,
[ARG_PTR_TO_FUNC] = &func_ptr_types,
- [ARG_PTR_TO_STACK_OR_NULL] = &stack_ptr_types,
+ [ARG_PTR_TO_STACK] = &stack_ptr_types,
[ARG_PTR_TO_CONST_STR] = &const_str_ptr_types,
[ARG_PTR_TO_TIMER] = &timer_types,
};
@@ -5140,12 +5191,27 @@ static int check_reg_type(struct bpf_verifier_env *env, u32 regno,
const struct bpf_reg_types *compatible;
int i, j;
- compatible = compatible_reg_types[arg_type];
+ compatible = compatible_reg_types[base_type(arg_type)];
if (!compatible) {
verbose(env, "verifier internal error: unsupported arg type %d\n", arg_type);
return -EFAULT;
}
+ /* ARG_PTR_TO_MEM + RDONLY is compatible with PTR_TO_MEM and PTR_TO_MEM + RDONLY,
+ * but ARG_PTR_TO_MEM is compatible only with PTR_TO_MEM and NOT with PTR_TO_MEM + RDONLY
+ *
+ * Same for MAYBE_NULL:
+ *
+ * ARG_PTR_TO_MEM + MAYBE_NULL is compatible with PTR_TO_MEM and PTR_TO_MEM + MAYBE_NULL,
+ * but ARG_PTR_TO_MEM is compatible only with PTR_TO_MEM but NOT with PTR_TO_MEM + MAYBE_NULL
+ *
+ * Therefore we fold these flags depending on the arg_type before comparison.
+ */
+ if (arg_type & MEM_RDONLY)
+ type &= ~MEM_RDONLY;
+ if (arg_type & PTR_MAYBE_NULL)
+ type &= ~PTR_MAYBE_NULL;
+
for (i = 0; i < ARRAY_SIZE(compatible->types); i++) {
expected = compatible->types[i];
if (expected == NOT_INIT)
@@ -5155,14 +5221,14 @@ static int check_reg_type(struct bpf_verifier_env *env, u32 regno,
goto found;
}
- verbose(env, "R%d type=%s expected=", regno, reg_type_str[type]);
+ verbose(env, "R%d type=%s expected=", regno, reg_type_str(env, reg->type));
for (j = 0; j + 1 < i; j++)
- verbose(env, "%s, ", reg_type_str[compatible->types[j]]);
- verbose(env, "%s\n", reg_type_str[compatible->types[j]]);
+ verbose(env, "%s, ", reg_type_str(env, compatible->types[j]));
+ verbose(env, "%s\n", reg_type_str(env, compatible->types[j]));
return -EACCES;
found:
- if (type == PTR_TO_BTF_ID) {
+ if (reg->type == PTR_TO_BTF_ID) {
if (!arg_btf_id) {
if (!compatible->btf_id) {
verbose(env, "verifier internal error: missing arg compatible BTF ID\n");
@@ -5221,15 +5287,14 @@ static int check_func_arg(struct bpf_verifier_env *env, u32 arg,
return -EACCES;
}
- if (arg_type == ARG_PTR_TO_MAP_VALUE ||
- arg_type == ARG_PTR_TO_UNINIT_MAP_VALUE ||
- arg_type == ARG_PTR_TO_MAP_VALUE_OR_NULL) {
+ if (base_type(arg_type) == ARG_PTR_TO_MAP_VALUE ||
+ base_type(arg_type) == ARG_PTR_TO_UNINIT_MAP_VALUE) {
err = resolve_map_arg_type(env, meta, &arg_type);
if (err)
return err;
}
- if (register_is_null(reg) && arg_type_may_be_null(arg_type))
+ if (register_is_null(reg) && type_may_be_null(arg_type))
/* A NULL register has a SCALAR_VALUE type, so skip
* type checking.
*/
@@ -5298,10 +5363,11 @@ static int check_func_arg(struct bpf_verifier_env *env, u32 arg,
err = check_helper_mem_access(env, regno,
meta->map_ptr->key_size, false,
NULL);
- } else if (arg_type == ARG_PTR_TO_MAP_VALUE ||
- (arg_type == ARG_PTR_TO_MAP_VALUE_OR_NULL &&
- !register_is_null(reg)) ||
- arg_type == ARG_PTR_TO_UNINIT_MAP_VALUE) {
+ } else if (base_type(arg_type) == ARG_PTR_TO_MAP_VALUE ||
+ base_type(arg_type) == ARG_PTR_TO_UNINIT_MAP_VALUE) {
+ if (type_may_be_null(arg_type) && register_is_null(reg))
+ return 0;
+
/* bpf_map_xxx(..., map_ptr, ..., value) call:
* check [value, value + map->value_size) validity
*/
@@ -5965,6 +6031,7 @@ static int __check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn
}
if (insn->code == (BPF_JMP | BPF_CALL) &&
+ insn->src_reg == 0 &&
insn->imm == BPF_FUNC_timer_set_callback) {
struct bpf_verifier_state *async_cb;
@@ -6023,9 +6090,9 @@ static int __check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn
if (env->log.level & BPF_LOG_LEVEL) {
verbose(env, "caller:\n");
- print_verifier_state(env, caller);
+ print_verifier_state(env, caller, true);
verbose(env, "callee:\n");
- print_verifier_state(env, callee);
+ print_verifier_state(env, callee, true);
}
return 0;
}
@@ -6116,6 +6183,27 @@ static int set_map_elem_callback_state(struct bpf_verifier_env *env,
return 0;
}
+static int set_loop_callback_state(struct bpf_verifier_env *env,
+ struct bpf_func_state *caller,
+ struct bpf_func_state *callee,
+ int insn_idx)
+{
+ /* bpf_loop(u32 nr_loops, void *callback_fn, void *callback_ctx,
+ * u64 flags);
+ * callback_fn(u32 index, void *callback_ctx);
+ */
+ callee->regs[BPF_REG_1].type = SCALAR_VALUE;
+ callee->regs[BPF_REG_2] = caller->regs[BPF_REG_3];
+
+ /* unused */
+ __mark_reg_not_init(env, &callee->regs[BPF_REG_3]);
+ __mark_reg_not_init(env, &callee->regs[BPF_REG_4]);
+ __mark_reg_not_init(env, &callee->regs[BPF_REG_5]);
+
+ callee->in_callback_fn = true;
+ return 0;
+}
+
static int set_timer_callback_state(struct bpf_verifier_env *env,
struct bpf_func_state *caller,
struct bpf_func_state *callee,
@@ -6145,6 +6233,33 @@ static int set_timer_callback_state(struct bpf_verifier_env *env,
return 0;
}
+static int set_find_vma_callback_state(struct bpf_verifier_env *env,
+ struct bpf_func_state *caller,
+ struct bpf_func_state *callee,
+ int insn_idx)
+{
+ /* bpf_find_vma(struct task_struct *task, u64 addr,
+ * void *callback_fn, void *callback_ctx, u64 flags)
+ * (callback_fn)(struct task_struct *task,
+ * struct vm_area_struct *vma, void *callback_ctx);
+ */
+ callee->regs[BPF_REG_1] = caller->regs[BPF_REG_1];
+
+ callee->regs[BPF_REG_2].type = PTR_TO_BTF_ID;
+ __mark_reg_known_zero(&callee->regs[BPF_REG_2]);
+ callee->regs[BPF_REG_2].btf = btf_vmlinux;
+ callee->regs[BPF_REG_2].btf_id = btf_tracing_ids[BTF_TRACING_TYPE_VMA],
+
+ /* pointer to stack or null */
+ callee->regs[BPF_REG_3] = caller->regs[BPF_REG_4];
+
+ /* unused */
+ __mark_reg_not_init(env, &callee->regs[BPF_REG_4]);
+ __mark_reg_not_init(env, &callee->regs[BPF_REG_5]);
+ callee->in_callback_fn = true;
+ return 0;
+}
+
static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx)
{
struct bpf_verifier_state *state = env->cur_state;
@@ -6192,9 +6307,9 @@ static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx)
*insn_idx = callee->callsite + 1;
if (env->log.level & BPF_LOG_LEVEL) {
verbose(env, "returning from callee:\n");
- print_verifier_state(env, callee);
+ print_verifier_state(env, callee, true);
verbose(env, "to caller at %d:\n", *insn_idx);
- print_verifier_state(env, caller);
+ print_verifier_state(env, caller, true);
}
/* clear everything in the callee */
free_func_state(callee);
@@ -6360,13 +6475,11 @@ static int check_bpf_snprintf_call(struct bpf_verifier_env *env,
static int check_get_func_ip(struct bpf_verifier_env *env)
{
- enum bpf_attach_type eatype = env->prog->expected_attach_type;
enum bpf_prog_type type = resolve_prog_type(env->prog);
int func_id = BPF_FUNC_get_func_ip;
if (type == BPF_PROG_TYPE_TRACING) {
- if (eatype != BPF_TRACE_FENTRY && eatype != BPF_TRACE_FEXIT &&
- eatype != BPF_MODIFY_RETURN) {
+ if (!bpf_prog_has_trampoline(env->prog)) {
verbose(env, "func %s#%d supported only for fentry/fexit/fmod_ret programs\n",
func_id_name(func_id), func_id);
return -ENOTSUPP;
@@ -6385,6 +6498,8 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn
int *insn_idx_p)
{
const struct bpf_func_proto *fn = NULL;
+ enum bpf_return_type ret_type;
+ enum bpf_type_flag ret_flag;
struct bpf_reg_state *regs;
struct bpf_call_arg_meta meta;
int insn_idx = *insn_idx_p;
@@ -6462,13 +6577,7 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn
return err;
}
- if (func_id == BPF_FUNC_tail_call) {
- err = check_reference_leak(env);
- if (err) {
- verbose(env, "tail_call would lead to reference leak\n");
- return err;
- }
- } else if (is_release_function(func_id)) {
+ if (is_release_function(func_id)) {
err = release_reference(env, meta.ref_obj_id);
if (err) {
verbose(env, "func %s#%d reference has not been acquired before\n",
@@ -6479,34 +6588,46 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn
regs = cur_regs(env);
- /* check that flags argument in get_local_storage(map, flags) is 0,
- * this is required because get_local_storage() can't return an error.
- */
- if (func_id == BPF_FUNC_get_local_storage &&
- !register_is_null(®s[BPF_REG_2])) {
- verbose(env, "get_local_storage() doesn't support non-zero flags\n");
- return -EINVAL;
- }
-
- if (func_id == BPF_FUNC_for_each_map_elem) {
+ switch (func_id) {
+ case BPF_FUNC_tail_call:
+ err = check_reference_leak(env);
+ if (err) {
+ verbose(env, "tail_call would lead to reference leak\n");
+ return err;
+ }
+ break;
+ case BPF_FUNC_get_local_storage:
+ /* check that flags argument in get_local_storage(map, flags) is 0,
+ * this is required because get_local_storage() can't return an error.
+ */
+ if (!register_is_null(®s[BPF_REG_2])) {
+ verbose(env, "get_local_storage() doesn't support non-zero flags\n");
+ return -EINVAL;
+ }
+ break;
+ case BPF_FUNC_for_each_map_elem:
err = __check_func_call(env, insn, insn_idx_p, meta.subprogno,
set_map_elem_callback_state);
- if (err < 0)
- return -EINVAL;
- }
-
- if (func_id == BPF_FUNC_timer_set_callback) {
+ break;
+ case BPF_FUNC_timer_set_callback:
err = __check_func_call(env, insn, insn_idx_p, meta.subprogno,
set_timer_callback_state);
- if (err < 0)
- return -EINVAL;
+ break;
+ case BPF_FUNC_find_vma:
+ err = __check_func_call(env, insn, insn_idx_p, meta.subprogno,
+ set_find_vma_callback_state);
+ break;
+ case BPF_FUNC_snprintf:
+ err = check_bpf_snprintf_call(env, regs);
+ break;
+ case BPF_FUNC_loop:
+ err = __check_func_call(env, insn, insn_idx_p, meta.subprogno,
+ set_loop_callback_state);
+ break;
}
- if (func_id == BPF_FUNC_snprintf) {
- err = check_bpf_snprintf_call(env, regs);
- if (err < 0)
- return err;
- }
+ if (err)
+ return err;
/* reset caller saved regs */
for (i = 0; i < CALLER_SAVED_REGS; i++) {
@@ -6518,13 +6639,14 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn
regs[BPF_REG_0].subreg_def = DEF_NOT_SUBREG;
/* update return register (already marked as written above) */
- if (fn->ret_type == RET_INTEGER) {
+ ret_type = fn->ret_type;
+ ret_flag = type_flag(fn->ret_type);
+ if (ret_type == RET_INTEGER) {
/* sets type to SCALAR_VALUE */
mark_reg_unknown(env, regs, BPF_REG_0);
- } else if (fn->ret_type == RET_VOID) {
+ } else if (ret_type == RET_VOID) {
regs[BPF_REG_0].type = NOT_INIT;
- } else if (fn->ret_type == RET_PTR_TO_MAP_VALUE_OR_NULL ||
- fn->ret_type == RET_PTR_TO_MAP_VALUE) {
+ } else if (base_type(ret_type) == RET_PTR_TO_MAP_VALUE) {
/* There is no offset yet applied, variable or fixed */
mark_reg_known_zero(env, regs, BPF_REG_0);
/* remember map_ptr, so that check_map_access()
@@ -6538,28 +6660,25 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn
}
regs[BPF_REG_0].map_ptr = meta.map_ptr;
regs[BPF_REG_0].map_uid = meta.map_uid;
- if (fn->ret_type == RET_PTR_TO_MAP_VALUE) {
- regs[BPF_REG_0].type = PTR_TO_MAP_VALUE;
- if (map_value_has_spin_lock(meta.map_ptr))
- regs[BPF_REG_0].id = ++env->id_gen;
- } else {
- regs[BPF_REG_0].type = PTR_TO_MAP_VALUE_OR_NULL;
+ regs[BPF_REG_0].type = PTR_TO_MAP_VALUE | ret_flag;
+ if (!type_may_be_null(ret_type) &&
+ map_value_has_spin_lock(meta.map_ptr)) {
+ regs[BPF_REG_0].id = ++env->id_gen;
}
- } else if (fn->ret_type == RET_PTR_TO_SOCKET_OR_NULL) {
+ } else if (base_type(ret_type) == RET_PTR_TO_SOCKET) {
mark_reg_known_zero(env, regs, BPF_REG_0);
- regs[BPF_REG_0].type = PTR_TO_SOCKET_OR_NULL;
- } else if (fn->ret_type == RET_PTR_TO_SOCK_COMMON_OR_NULL) {
+ regs[BPF_REG_0].type = PTR_TO_SOCKET | ret_flag;
+ } else if (base_type(ret_type) == RET_PTR_TO_SOCK_COMMON) {
mark_reg_known_zero(env, regs, BPF_REG_0);
- regs[BPF_REG_0].type = PTR_TO_SOCK_COMMON_OR_NULL;
- } else if (fn->ret_type == RET_PTR_TO_TCP_SOCK_OR_NULL) {
+ regs[BPF_REG_0].type = PTR_TO_SOCK_COMMON | ret_flag;
+ } else if (base_type(ret_type) == RET_PTR_TO_TCP_SOCK) {
mark_reg_known_zero(env, regs, BPF_REG_0);
- regs[BPF_REG_0].type = PTR_TO_TCP_SOCK_OR_NULL;
- } else if (fn->ret_type == RET_PTR_TO_ALLOC_MEM_OR_NULL) {
+ regs[BPF_REG_0].type = PTR_TO_TCP_SOCK | ret_flag;
+ } else if (base_type(ret_type) == RET_PTR_TO_ALLOC_MEM) {
mark_reg_known_zero(env, regs, BPF_REG_0);
- regs[BPF_REG_0].type = PTR_TO_MEM_OR_NULL;
+ regs[BPF_REG_0].type = PTR_TO_MEM | ret_flag;
regs[BPF_REG_0].mem_size = meta.mem_size;
- } else if (fn->ret_type == RET_PTR_TO_MEM_OR_BTF_ID_OR_NULL ||
- fn->ret_type == RET_PTR_TO_MEM_OR_BTF_ID) {
+ } else if (base_type(ret_type) == RET_PTR_TO_MEM_OR_BTF_ID) {
const struct btf_type *t;
mark_reg_known_zero(env, regs, BPF_REG_0);
@@ -6577,29 +6696,30 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn
tname, PTR_ERR(ret));
return -EINVAL;
}
- regs[BPF_REG_0].type =
- fn->ret_type == RET_PTR_TO_MEM_OR_BTF_ID ?
- PTR_TO_MEM : PTR_TO_MEM_OR_NULL;
+ regs[BPF_REG_0].type = PTR_TO_MEM | ret_flag;
regs[BPF_REG_0].mem_size = tsize;
} else {
- regs[BPF_REG_0].type =
- fn->ret_type == RET_PTR_TO_MEM_OR_BTF_ID ?
- PTR_TO_BTF_ID : PTR_TO_BTF_ID_OR_NULL;
+ /* MEM_RDONLY may be carried from ret_flag, but it
+ * doesn't apply on PTR_TO_BTF_ID. Fold it, otherwise
+ * it will confuse the check of PTR_TO_BTF_ID in
+ * check_mem_access().
+ */
+ ret_flag &= ~MEM_RDONLY;
+
+ regs[BPF_REG_0].type = PTR_TO_BTF_ID | ret_flag;
regs[BPF_REG_0].btf = meta.ret_btf;
regs[BPF_REG_0].btf_id = meta.ret_btf_id;
}
- } else if (fn->ret_type == RET_PTR_TO_BTF_ID_OR_NULL ||
- fn->ret_type == RET_PTR_TO_BTF_ID) {
+ } else if (base_type(ret_type) == RET_PTR_TO_BTF_ID) {
int ret_btf_id;
mark_reg_known_zero(env, regs, BPF_REG_0);
- regs[BPF_REG_0].type = fn->ret_type == RET_PTR_TO_BTF_ID ?
- PTR_TO_BTF_ID :
- PTR_TO_BTF_ID_OR_NULL;
+ regs[BPF_REG_0].type = PTR_TO_BTF_ID | ret_flag;
ret_btf_id = *fn->ret_btf_id;
if (ret_btf_id == 0) {
- verbose(env, "invalid return type %d of func %s#%d\n",
- fn->ret_type, func_id_name(func_id), func_id);
+ verbose(env, "invalid return type %u of func %s#%d\n",
+ base_type(ret_type), func_id_name(func_id),
+ func_id);
return -EINVAL;
}
/* current BPF helper definitions are only coming from
@@ -6608,12 +6728,12 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn
regs[BPF_REG_0].btf = btf_vmlinux;
regs[BPF_REG_0].btf_id = ret_btf_id;
} else {
- verbose(env, "unknown return type %d of func %s#%d\n",
- fn->ret_type, func_id_name(func_id), func_id);
+ verbose(env, "unknown return type %u of func %s#%d\n",
+ base_type(ret_type), func_id_name(func_id), func_id);
return -EINVAL;
}
- if (reg_type_may_be_null(regs[BPF_REG_0].type))
+ if (type_may_be_null(regs[BPF_REG_0].type))
regs[BPF_REG_0].id = ++env->id_gen;
if (is_ptr_cast_function(func_id)) {
@@ -6822,25 +6942,25 @@ static bool check_reg_sane_offset(struct bpf_verifier_env *env,
if (known && (val >= BPF_MAX_VAR_OFF || val <= -BPF_MAX_VAR_OFF)) {
verbose(env, "math between %s pointer and %lld is not allowed\n",
- reg_type_str[type], val);
+ reg_type_str(env, type), val);
return false;
}
if (reg->off >= BPF_MAX_VAR_OFF || reg->off <= -BPF_MAX_VAR_OFF) {
verbose(env, "%s pointer offset %d is not allowed\n",
- reg_type_str[type], reg->off);
+ reg_type_str(env, type), reg->off);
return false;
}
if (smin == S64_MIN) {
verbose(env, "math between %s pointer and register with unbounded min value is not allowed\n",
- reg_type_str[type]);
+ reg_type_str(env, type));
return false;
}
if (smin >= BPF_MAX_VAR_OFF || smin <= -BPF_MAX_VAR_OFF) {
verbose(env, "value %lld makes %s pointer be out of bounds\n",
- smin, reg_type_str[type]);
+ smin, reg_type_str(env, type));
return false;
}
@@ -7217,11 +7337,13 @@ static int adjust_ptr_min_max_vals(struct bpf_verifier_env *env,
return -EACCES;
}
- switch (ptr_reg->type) {
- case PTR_TO_MAP_VALUE_OR_NULL:
+ if (ptr_reg->type & PTR_MAYBE_NULL) {
verbose(env, "R%d pointer arithmetic on %s prohibited, null-check it first\n",
- dst, reg_type_str[ptr_reg->type]);
+ dst, reg_type_str(env, ptr_reg->type));
return -EACCES;
+ }
+
+ switch (base_type(ptr_reg->type)) {
case CONST_PTR_TO_MAP:
/* smin_val represents the known value */
if (known && smin_val == 0 && opcode == BPF_ADD)
@@ -7229,14 +7351,11 @@ static int adjust_ptr_min_max_vals(struct bpf_verifier_env *env,
fallthrough;
case PTR_TO_PACKET_END:
case PTR_TO_SOCKET:
- case PTR_TO_SOCKET_OR_NULL:
case PTR_TO_SOCK_COMMON:
- case PTR_TO_SOCK_COMMON_OR_NULL:
case PTR_TO_TCP_SOCK:
- case PTR_TO_TCP_SOCK_OR_NULL:
case PTR_TO_XDP_SOCK:
verbose(env, "R%d pointer arithmetic on %s prohibited\n",
- dst, reg_type_str[ptr_reg->type]);
+ dst, reg_type_str(env, ptr_reg->type));
return -EACCES;
default:
break;
@@ -8209,12 +8328,12 @@ static int adjust_reg_min_max_vals(struct bpf_verifier_env *env,
/* Got here implies adding two SCALAR_VALUEs */
if (WARN_ON_ONCE(ptr_reg)) {
- print_verifier_state(env, state);
+ print_verifier_state(env, state, true);
verbose(env, "verifier internal error: unexpected ptr_reg\n");
return -EINVAL;
}
if (WARN_ON(!src_reg)) {
- print_verifier_state(env, state);
+ print_verifier_state(env, state, true);
verbose(env, "verifier internal error: no src_reg\n");
return -EINVAL;
}
@@ -8959,17 +9078,17 @@ static void mark_ptr_or_null_reg(struct bpf_func_state *state,
struct bpf_reg_state *reg, u32 id,
bool is_null)
{
- if (reg_type_may_be_null(reg->type) && reg->id == id &&
+ if (type_may_be_null(reg->type) && reg->id == id &&
!WARN_ON_ONCE(!reg->id)) {
- /* Old offset (both fixed and variable parts) should
- * have been known-zero, because we don't allow pointer
- * arithmetic on pointers that might be NULL.
- */
if (WARN_ON_ONCE(reg->smin_value || reg->smax_value ||
!tnum_equals_const(reg->var_off, 0) ||
reg->off)) {
- __mark_reg_known_zero(reg);
- reg->off = 0;
+ /* Old offset (both fixed and variable parts) should
+ * have been known-zero, because we don't allow pointer
+ * arithmetic on pointers that might be NULL. If we
+ * see this happening, don't convert the register.
+ */
+ return;
}
if (is_null) {
reg->type = SCALAR_VALUE;
@@ -9337,7 +9456,7 @@ static int check_cond_jmp_op(struct bpf_verifier_env *env,
*/
if (!is_jmp32 && BPF_SRC(insn->code) == BPF_K &&
insn->imm == 0 && (opcode == BPF_JEQ || opcode == BPF_JNE) &&
- reg_type_may_be_null(dst_reg->type)) {
+ type_may_be_null(dst_reg->type)) {
/* Mark all identical registers in each branch as either
* safe or unknown depending R == 0 or R != 0 conditional.
*/
@@ -9353,7 +9472,7 @@ static int check_cond_jmp_op(struct bpf_verifier_env *env,
return -EACCES;
}
if (env->log.level & BPF_LOG_LEVEL)
- print_verifier_state(env, this_branch->frame[this_branch->curframe]);
+ print_insn_state(env, this_branch->frame[this_branch->curframe]);
return 0;
}
@@ -9392,7 +9511,7 @@ static int check_ld_imm(struct bpf_verifier_env *env, struct bpf_insn *insn)
mark_reg_known_zero(env, regs, insn->dst_reg);
dst_reg->type = aux->btf_var.reg_type;
- switch (dst_reg->type) {
+ switch (base_type(dst_reg->type)) {
case PTR_TO_MEM:
dst_reg->mem_size = aux->btf_var.mem_size;
break;
@@ -9591,7 +9710,7 @@ static int check_return_code(struct bpf_verifier_env *env)
/* enforce return zero from async callbacks like timer */
if (reg->type != SCALAR_VALUE) {
verbose(env, "In async callback the register R0 is not a known value (%s)\n",
- reg_type_str[reg->type]);
+ reg_type_str(env, reg->type));
return -EINVAL;
}
@@ -9605,7 +9724,7 @@ static int check_return_code(struct bpf_verifier_env *env)
if (is_subprog) {
if (reg->type != SCALAR_VALUE) {
verbose(env, "At subprogram exit the register R0 is not a scalar value (%s)\n",
- reg_type_str[reg->type]);
+ reg_type_str(env, reg->type));
return -EINVAL;
}
return 0;
@@ -9669,7 +9788,7 @@ static int check_return_code(struct bpf_verifier_env *env)
if (reg->type != SCALAR_VALUE) {
verbose(env, "At program exit the register R0 is not a known value (%s)\n",
- reg_type_str[reg->type]);
+ reg_type_str(env, reg->type));
return -EINVAL;
}
@@ -10252,6 +10371,78 @@ static int check_btf_line(struct bpf_verifier_env *env,
return err;
}
+#define MIN_CORE_RELO_SIZE sizeof(struct bpf_core_relo)
+#define MAX_CORE_RELO_SIZE MAX_FUNCINFO_REC_SIZE
+
+static int check_core_relo(struct bpf_verifier_env *env,
+ const union bpf_attr *attr,
+ bpfptr_t uattr)
+{
+ u32 i, nr_core_relo, ncopy, expected_size, rec_size;
+ struct bpf_core_relo core_relo = {};
+ struct bpf_prog *prog = env->prog;
+ const struct btf *btf = prog->aux->btf;
+ struct bpf_core_ctx ctx = {
+ .log = &env->log,
+ .btf = btf,
+ };
+ bpfptr_t u_core_relo;
+ int err;
+
+ nr_core_relo = attr->core_relo_cnt;
+ if (!nr_core_relo)
+ return 0;
+ if (nr_core_relo > INT_MAX / sizeof(struct bpf_core_relo))
+ return -EINVAL;
+
+ rec_size = attr->core_relo_rec_size;
+ if (rec_size < MIN_CORE_RELO_SIZE ||
+ rec_size > MAX_CORE_RELO_SIZE ||
+ rec_size % sizeof(u32))
+ return -EINVAL;
+
+ u_core_relo = make_bpfptr(attr->core_relos, uattr.is_kernel);
+ expected_size = sizeof(struct bpf_core_relo);
+ ncopy = min_t(u32, expected_size, rec_size);
+
+ /* Unlike func_info and line_info, copy and apply each CO-RE
+ * relocation record one at a time.
+ */
+ for (i = 0; i < nr_core_relo; i++) {
+ /* future proofing when sizeof(bpf_core_relo) changes */
+ err = bpf_check_uarg_tail_zero(u_core_relo, expected_size, rec_size);
+ if (err) {
+ if (err == -E2BIG) {
+ verbose(env, "nonzero tailing record in core_relo");
+ if (copy_to_bpfptr_offset(uattr,
+ offsetof(union bpf_attr, core_relo_rec_size),
+ &expected_size, sizeof(expected_size)))
+ err = -EFAULT;
+ }
+ break;
+ }
+
+ if (copy_from_bpfptr(&core_relo, u_core_relo, ncopy)) {
+ err = -EFAULT;
+ break;
+ }
+
+ if (core_relo.insn_off % 8 || core_relo.insn_off / 8 >= prog->len) {
+ verbose(env, "Invalid core_relo[%u].insn_off:%u prog->len:%u\n",
+ i, core_relo.insn_off, prog->len);
+ err = -EINVAL;
+ break;
+ }
+
+ err = bpf_core_apply(&ctx, &core_relo, i,
+ &prog->insnsi[core_relo.insn_off / 8]);
+ if (err)
+ break;
+ bpfptr_add(&u_core_relo, rec_size);
+ }
+ return err;
+}
+
static int check_btf_info(struct bpf_verifier_env *env,
const union bpf_attr *attr,
bpfptr_t uattr)
@@ -10282,6 +10473,10 @@ static int check_btf_info(struct bpf_verifier_env *env,
if (err)
return err;
+ err = check_core_relo(env, attr, uattr);
+ if (err)
+ return err;
+
return 0;
}
@@ -10450,7 +10645,7 @@ static bool regsafe(struct bpf_verifier_env *env, struct bpf_reg_state *rold,
return true;
if (rcur->type == NOT_INIT)
return false;
- switch (rold->type) {
+ switch (base_type(rold->type)) {
case SCALAR_VALUE:
if (env->explore_alu_limits)
return false;
@@ -10472,6 +10667,22 @@ static bool regsafe(struct bpf_verifier_env *env, struct bpf_reg_state *rold,
}
case PTR_TO_MAP_KEY:
case PTR_TO_MAP_VALUE:
+ /* a PTR_TO_MAP_VALUE could be safe to use as a
+ * PTR_TO_MAP_VALUE_OR_NULL into the same map.
+ * However, if the old PTR_TO_MAP_VALUE_OR_NULL then got NULL-
+ * checked, doing so could have affected others with the same
+ * id, and we can't check for that because we lost the id when
+ * we converted to a PTR_TO_MAP_VALUE.
+ */
+ if (type_may_be_null(rold->type)) {
+ if (!type_may_be_null(rcur->type))
+ return false;
+ if (memcmp(rold, rcur, offsetof(struct bpf_reg_state, id)))
+ return false;
+ /* Check our ids match any regs they're supposed to */
+ return check_ids(rold->id, rcur->id, idmap);
+ }
+
/* If the new min/max/var_off satisfy the old ones and
* everything else matches, we are OK.
* 'id' is not compared, since it's only used for maps with
@@ -10483,20 +10694,6 @@ static bool regsafe(struct bpf_verifier_env *env, struct bpf_reg_state *rold,
return memcmp(rold, rcur, offsetof(struct bpf_reg_state, id)) == 0 &&
range_within(rold, rcur) &&
tnum_in(rold->var_off, rcur->var_off);
- case PTR_TO_MAP_VALUE_OR_NULL:
- /* a PTR_TO_MAP_VALUE could be safe to use as a
- * PTR_TO_MAP_VALUE_OR_NULL into the same map.
- * However, if the old PTR_TO_MAP_VALUE_OR_NULL then got NULL-
- * checked, doing so could have affected others with the same
- * id, and we can't check for that because we lost the id when
- * we converted to a PTR_TO_MAP_VALUE.
- */
- if (rcur->type != PTR_TO_MAP_VALUE_OR_NULL)
- return false;
- if (memcmp(rold, rcur, offsetof(struct bpf_reg_state, id)))
- return false;
- /* Check our ids match any regs they're supposed to */
- return check_ids(rold->id, rcur->id, idmap);
case PTR_TO_PACKET_META:
case PTR_TO_PACKET:
if (rcur->type != rold->type)
@@ -10525,11 +10722,8 @@ static bool regsafe(struct bpf_verifier_env *env, struct bpf_reg_state *rold,
case PTR_TO_PACKET_END:
case PTR_TO_FLOW_KEYS:
case PTR_TO_SOCKET:
- case PTR_TO_SOCKET_OR_NULL:
case PTR_TO_SOCK_COMMON:
- case PTR_TO_SOCK_COMMON_OR_NULL:
case PTR_TO_TCP_SOCK:
- case PTR_TO_TCP_SOCK_OR_NULL:
case PTR_TO_XDP_SOCK:
/* Only valid matches are exact, which memcmp() above
* would have accepted
@@ -11055,17 +11249,13 @@ static int is_state_visited(struct bpf_verifier_env *env, int insn_idx)
/* Return true if it's OK to have the same insn return a different type. */
static bool reg_type_mismatch_ok(enum bpf_reg_type type)
{
- switch (type) {
+ switch (base_type(type)) {
case PTR_TO_CTX:
case PTR_TO_SOCKET:
- case PTR_TO_SOCKET_OR_NULL:
case PTR_TO_SOCK_COMMON:
- case PTR_TO_SOCK_COMMON_OR_NULL:
case PTR_TO_TCP_SOCK:
- case PTR_TO_TCP_SOCK_OR_NULL:
case PTR_TO_XDP_SOCK:
case PTR_TO_BTF_ID:
- case PTR_TO_BTF_ID_OR_NULL:
return false;
default:
return true;
@@ -11145,16 +11335,12 @@ static int do_check(struct bpf_verifier_env *env)
if (need_resched())
cond_resched();
- if (env->log.level & BPF_LOG_LEVEL2 ||
- (env->log.level & BPF_LOG_LEVEL && do_print_state)) {
- if (env->log.level & BPF_LOG_LEVEL2)
- verbose(env, "%d:", env->insn_idx);
- else
- verbose(env, "\nfrom %d to %d%s:",
- env->prev_insn_idx, env->insn_idx,
- env->cur_state->speculative ?
- " (speculative execution)" : "");
- print_verifier_state(env, state->frame[state->curframe]);
+ if (env->log.level & BPF_LOG_LEVEL2 && do_print_state) {
+ verbose(env, "\nfrom %d to %d%s:",
+ env->prev_insn_idx, env->insn_idx,
+ env->cur_state->speculative ?
+ " (speculative execution)" : "");
+ print_verifier_state(env, state->frame[state->curframe], true);
do_print_state = false;
}
@@ -11165,9 +11351,15 @@ static int do_check(struct bpf_verifier_env *env)
.private_data = env,
};
+ if (verifier_state_scratched(env))
+ print_insn_state(env, state->frame[state->curframe]);
+
verbose_linfo(env, env->insn_idx, "; ");
+ env->prev_log_len = env->log.len_used;
verbose(env, "%d: ", env->insn_idx);
print_bpf_insn(&cbs, insn, env->allow_ptr_leaks);
+ env->prev_insn_print_len = env->log.len_used - env->prev_log_len;
+ env->prev_log_len = env->log.len_used;
}
if (bpf_prog_is_dev_bound(env->prog->aux)) {
@@ -11289,7 +11481,7 @@ static int do_check(struct bpf_verifier_env *env)
if (is_ctx_reg(env, insn->dst_reg)) {
verbose(env, "BPF_ST stores into R%d %s is not allowed\n",
insn->dst_reg,
- reg_type_str[reg_state(env, insn->dst_reg)->type]);
+ reg_type_str(env, reg_state(env, insn->dst_reg)->type));
return -EACCES;
}
@@ -11376,6 +11568,7 @@ static int do_check(struct bpf_verifier_env *env)
if (err)
return err;
process_bpf_exit:
+ mark_verifier_state_scratched(env);
update_branch_counts(env, env->cur_state);
err = pop_stack(env, &prev_insn_idx,
&env->insn_idx, pop_log);
@@ -11541,7 +11734,7 @@ static int check_pseudo_btf_id(struct bpf_verifier_env *env,
err = -EINVAL;
goto err_put;
}
- aux->btf_var.reg_type = PTR_TO_MEM;
+ aux->btf_var.reg_type = PTR_TO_MEM | MEM_RDONLY;
aux->btf_var.mem_size = tsize;
} else {
aux->btf_var.reg_type = PTR_TO_BTF_ID;
@@ -11701,6 +11894,9 @@ static int check_map_prog_compatibility(struct bpf_verifier_env *env,
}
break;
case BPF_MAP_TYPE_RINGBUF:
+ case BPF_MAP_TYPE_INODE_STORAGE:
+ case BPF_MAP_TYPE_SK_STORAGE:
+ case BPF_MAP_TYPE_TASK_STORAGE:
break;
default:
verbose(env,
@@ -12884,6 +13080,7 @@ static int fixup_kfunc_call(struct bpf_verifier_env *env,
static int do_misc_fixups(struct bpf_verifier_env *env)
{
struct bpf_prog *prog = env->prog;
+ enum bpf_attach_type eatype = prog->expected_attach_type;
bool expect_blinding = bpf_jit_blinding_enabled(prog);
enum bpf_prog_type prog_type = resolve_prog_type(prog);
struct bpf_insn *insn = prog->insnsi;
@@ -13254,11 +13451,79 @@ static int do_misc_fixups(struct bpf_verifier_env *env)
continue;
}
+ /* Implement bpf_get_func_arg inline. */
+ if (prog_type == BPF_PROG_TYPE_TRACING &&
+ insn->imm == BPF_FUNC_get_func_arg) {
+ /* Load nr_args from ctx - 8 */
+ insn_buf[0] = BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_1, -8);
+ insn_buf[1] = BPF_JMP32_REG(BPF_JGE, BPF_REG_2, BPF_REG_0, 6);
+ insn_buf[2] = BPF_ALU64_IMM(BPF_LSH, BPF_REG_2, 3);
+ insn_buf[3] = BPF_ALU64_REG(BPF_ADD, BPF_REG_2, BPF_REG_1);
+ insn_buf[4] = BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_2, 0);
+ insn_buf[5] = BPF_STX_MEM(BPF_DW, BPF_REG_3, BPF_REG_0, 0);
+ insn_buf[6] = BPF_MOV64_IMM(BPF_REG_0, 0);
+ insn_buf[7] = BPF_JMP_A(1);
+ insn_buf[8] = BPF_MOV64_IMM(BPF_REG_0, -EINVAL);
+ cnt = 9;
+
+ new_prog = bpf_patch_insn_data(env, i + delta, insn_buf, cnt);
+ if (!new_prog)
+ return -ENOMEM;
+
+ delta += cnt - 1;
+ env->prog = prog = new_prog;
+ insn = new_prog->insnsi + i + delta;
+ continue;
+ }
+
+ /* Implement bpf_get_func_ret inline. */
+ if (prog_type == BPF_PROG_TYPE_TRACING &&
+ insn->imm == BPF_FUNC_get_func_ret) {
+ if (eatype == BPF_TRACE_FEXIT ||
+ eatype == BPF_MODIFY_RETURN) {
+ /* Load nr_args from ctx - 8 */
+ insn_buf[0] = BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_1, -8);
+ insn_buf[1] = BPF_ALU64_IMM(BPF_LSH, BPF_REG_0, 3);
+ insn_buf[2] = BPF_ALU64_REG(BPF_ADD, BPF_REG_0, BPF_REG_1);
+ insn_buf[3] = BPF_LDX_MEM(BPF_DW, BPF_REG_3, BPF_REG_0, 0);
+ insn_buf[4] = BPF_STX_MEM(BPF_DW, BPF_REG_2, BPF_REG_3, 0);
+ insn_buf[5] = BPF_MOV64_IMM(BPF_REG_0, 0);
+ cnt = 6;
+ } else {
+ insn_buf[0] = BPF_MOV64_IMM(BPF_REG_0, -EOPNOTSUPP);
+ cnt = 1;
+ }
+
+ new_prog = bpf_patch_insn_data(env, i + delta, insn_buf, cnt);
+ if (!new_prog)
+ return -ENOMEM;
+
+ delta += cnt - 1;
+ env->prog = prog = new_prog;
+ insn = new_prog->insnsi + i + delta;
+ continue;
+ }
+
+ /* Implement get_func_arg_cnt inline. */
+ if (prog_type == BPF_PROG_TYPE_TRACING &&
+ insn->imm == BPF_FUNC_get_func_arg_cnt) {
+ /* Load nr_args from ctx - 8 */
+ insn_buf[0] = BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_1, -8);
+
+ new_prog = bpf_patch_insn_data(env, i + delta, insn_buf, 1);
+ if (!new_prog)
+ return -ENOMEM;
+
+ env->prog = prog = new_prog;
+ insn = new_prog->insnsi + i + delta;
+ continue;
+ }
+
/* Implement bpf_get_func_ip inline. */
if (prog_type == BPF_PROG_TYPE_TRACING &&
insn->imm == BPF_FUNC_get_func_ip) {
- /* Load IP address from ctx - 8 */
- insn_buf[0] = BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_1, -8);
+ /* Load IP address from ctx - 16 */
+ insn_buf[0] = BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_1, -16);
new_prog = bpf_patch_insn_data(env, i + delta, insn_buf, 1);
if (!new_prog)
@@ -13372,7 +13637,7 @@ static int do_check_common(struct bpf_verifier_env *env, int subprog)
mark_reg_known_zero(env, regs, i);
else if (regs[i].type == SCALAR_VALUE)
mark_reg_unknown(env, regs, i);
- else if (regs[i].type == PTR_TO_MEM_OR_NULL) {
+ else if (base_type(regs[i].type) == PTR_TO_MEM) {
const u32 mem_size = regs[i].mem_size;
mark_reg_known_zero(env, regs, i);
@@ -13960,13 +14225,15 @@ int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr)
log->ubuf = (char __user *) (unsigned long) attr->log_buf;
log->len_total = attr->log_size;
- ret = -EINVAL;
/* log attributes have to be sane */
- if (log->len_total < 128 || log->len_total > UINT_MAX >> 2 ||
- !log->level || !log->ubuf || log->level & ~BPF_LOG_MASK)
+ if (!bpf_verifier_log_attr_valid(log)) {
+ ret = -EINVAL;
goto err_unlock;
+ }
}
+ mark_verifier_state_clean(env);
+
if (IS_ERR(btf_vmlinux)) {
/* Either gcc or pahole or kernel are broken. */
verbose(env, "in-kernel BTF is malformed\n");
diff --git a/kernel/cgroup/cgroup-internal.h b/kernel/cgroup/cgroup-internal.h
index bfbeabc..6e36e85 100644
--- a/kernel/cgroup/cgroup-internal.h
+++ b/kernel/cgroup/cgroup-internal.h
@@ -65,6 +65,25 @@ static inline struct cgroup_fs_context *cgroup_fc2context(struct fs_context *fc)
return container_of(kfc, struct cgroup_fs_context, kfc);
}
+struct cgroup_pidlist;
+
+struct cgroup_file_ctx {
+ struct cgroup_namespace *ns;
+
+ struct {
+ void *trigger;
+ } psi;
+
+ struct {
+ bool started;
+ struct css_task_iter iter;
+ } procs;
+
+ struct {
+ struct cgroup_pidlist *pidlist;
+ } procs1;
+};
+
/*
* A cgroup can be associated with multiple css_sets as different tasks may
* belong to different cgroups on different hierarchies. In the other
diff --git a/kernel/cgroup/cgroup-v1.c b/kernel/cgroup/cgroup-v1.c
index 81c9e06..41e0837 100644
--- a/kernel/cgroup/cgroup-v1.c
+++ b/kernel/cgroup/cgroup-v1.c
@@ -394,6 +394,7 @@ static void *cgroup_pidlist_start(struct seq_file *s, loff_t *pos)
* next pid to display, if any
*/
struct kernfs_open_file *of = s->private;
+ struct cgroup_file_ctx *ctx = of->priv;
struct cgroup *cgrp = seq_css(s)->cgroup;
struct cgroup_pidlist *l;
enum cgroup_filetype type = seq_cft(s)->private;
@@ -403,25 +404,24 @@ static void *cgroup_pidlist_start(struct seq_file *s, loff_t *pos)
mutex_lock(&cgrp->pidlist_mutex);
/*
- * !NULL @of->priv indicates that this isn't the first start()
- * after open. If the matching pidlist is around, we can use that.
- * Look for it. Note that @of->priv can't be used directly. It
- * could already have been destroyed.
+ * !NULL @ctx->procs1.pidlist indicates that this isn't the first
+ * start() after open. If the matching pidlist is around, we can use
+ * that. Look for it. Note that @ctx->procs1.pidlist can't be used
+ * directly. It could already have been destroyed.
*/
- if (of->priv)
- of->priv = cgroup_pidlist_find(cgrp, type);
+ if (ctx->procs1.pidlist)
+ ctx->procs1.pidlist = cgroup_pidlist_find(cgrp, type);
/*
* Either this is the first start() after open or the matching
* pidlist has been destroyed inbetween. Create a new one.
*/
- if (!of->priv) {
- ret = pidlist_array_load(cgrp, type,
- (struct cgroup_pidlist **)&of->priv);
+ if (!ctx->procs1.pidlist) {
+ ret = pidlist_array_load(cgrp, type, &ctx->procs1.pidlist);
if (ret)
return ERR_PTR(ret);
}
- l = of->priv;
+ l = ctx->procs1.pidlist;
if (pid) {
int end = l->length;
@@ -449,7 +449,8 @@ static void *cgroup_pidlist_start(struct seq_file *s, loff_t *pos)
static void cgroup_pidlist_stop(struct seq_file *s, void *v)
{
struct kernfs_open_file *of = s->private;
- struct cgroup_pidlist *l = of->priv;
+ struct cgroup_file_ctx *ctx = of->priv;
+ struct cgroup_pidlist *l = ctx->procs1.pidlist;
if (l)
mod_delayed_work(cgroup_pidlist_destroy_wq, &l->destroy_dwork,
@@ -460,7 +461,8 @@ static void cgroup_pidlist_stop(struct seq_file *s, void *v)
static void *cgroup_pidlist_next(struct seq_file *s, void *v, loff_t *pos)
{
struct kernfs_open_file *of = s->private;
- struct cgroup_pidlist *l = of->priv;
+ struct cgroup_file_ctx *ctx = of->priv;
+ struct cgroup_pidlist *l = ctx->procs1.pidlist;
pid_t *p = v;
pid_t *end = l->list + l->length;
/*
@@ -504,10 +506,11 @@ static ssize_t __cgroup1_procs_write(struct kernfs_open_file *of,
goto out_unlock;
/*
- * Even if we're attaching all tasks in the thread group, we only
- * need to check permissions on one of them.
+ * Even if we're attaching all tasks in the thread group, we only need
+ * to check permissions on one of them. Check permissions using the
+ * credentials from file open to protect against inherited fd attacks.
*/
- cred = current_cred();
+ cred = of->file->f_cred;
tcred = get_task_cred(task);
if (!uid_eq(cred->euid, GLOBAL_ROOT_UID) &&
!uid_eq(cred->euid, tcred->uid) &&
diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c
index 919194d..581ca5a 100644
--- a/kernel/cgroup/cgroup.c
+++ b/kernel/cgroup/cgroup.c
@@ -30,6 +30,7 @@
#include "cgroup-internal.h"
+#include <linux/bpf-cgroup.h>
#include <linux/cred.h>
#include <linux/errno.h>
#include <linux/init_task.h>
@@ -3630,6 +3631,7 @@ static int cgroup_cpu_pressure_show(struct seq_file *seq, void *v)
static ssize_t cgroup_pressure_write(struct kernfs_open_file *of, char *buf,
size_t nbytes, enum psi_res res)
{
+ struct cgroup_file_ctx *ctx = of->priv;
struct psi_trigger *new;
struct cgroup *cgrp;
struct psi_group *psi;
@@ -3648,7 +3650,7 @@ static ssize_t cgroup_pressure_write(struct kernfs_open_file *of, char *buf,
return PTR_ERR(new);
}
- psi_trigger_replace(&of->priv, new);
+ psi_trigger_replace(&ctx->psi.trigger, new);
cgroup_put(cgrp);
@@ -3679,12 +3681,16 @@ static ssize_t cgroup_cpu_pressure_write(struct kernfs_open_file *of,
static __poll_t cgroup_pressure_poll(struct kernfs_open_file *of,
poll_table *pt)
{
- return psi_trigger_poll(&of->priv, of->file, pt);
+ struct cgroup_file_ctx *ctx = of->priv;
+
+ return psi_trigger_poll(&ctx->psi.trigger, of->file, pt);
}
static void cgroup_pressure_release(struct kernfs_open_file *of)
{
- psi_trigger_replace(&of->priv, NULL);
+ struct cgroup_file_ctx *ctx = of->priv;
+
+ psi_trigger_replace(&ctx->psi.trigger, NULL);
}
bool cgroup_psi_enabled(void)
@@ -3811,24 +3817,43 @@ static ssize_t cgroup_kill_write(struct kernfs_open_file *of, char *buf,
static int cgroup_file_open(struct kernfs_open_file *of)
{
struct cftype *cft = of_cft(of);
+ struct cgroup_file_ctx *ctx;
+ int ret;
- if (cft->open)
- return cft->open(of);
- return 0;
+ ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
+ if (!ctx)
+ return -ENOMEM;
+
+ ctx->ns = current->nsproxy->cgroup_ns;
+ get_cgroup_ns(ctx->ns);
+ of->priv = ctx;
+
+ if (!cft->open)
+ return 0;
+
+ ret = cft->open(of);
+ if (ret) {
+ put_cgroup_ns(ctx->ns);
+ kfree(ctx);
+ }
+ return ret;
}
static void cgroup_file_release(struct kernfs_open_file *of)
{
struct cftype *cft = of_cft(of);
+ struct cgroup_file_ctx *ctx = of->priv;
if (cft->release)
cft->release(of);
+ put_cgroup_ns(ctx->ns);
+ kfree(ctx);
}
static ssize_t cgroup_file_write(struct kernfs_open_file *of, char *buf,
size_t nbytes, loff_t off)
{
- struct cgroup_namespace *ns = current->nsproxy->cgroup_ns;
+ struct cgroup_file_ctx *ctx = of->priv;
struct cgroup *cgrp = of->kn->parent->priv;
struct cftype *cft = of_cft(of);
struct cgroup_subsys_state *css;
@@ -3845,7 +3870,7 @@ static ssize_t cgroup_file_write(struct kernfs_open_file *of, char *buf,
*/
if ((cgrp->root->flags & CGRP_ROOT_NS_DELEGATE) &&
!(cft->flags & CFTYPE_NS_DELEGATABLE) &&
- ns != &init_cgroup_ns && ns->root_cset->dfl_cgrp == cgrp)
+ ctx->ns != &init_cgroup_ns && ctx->ns->root_cset->dfl_cgrp == cgrp)
return -EPERM;
if (cft->write)
@@ -4751,21 +4776,21 @@ void css_task_iter_end(struct css_task_iter *it)
static void cgroup_procs_release(struct kernfs_open_file *of)
{
- if (of->priv) {
- css_task_iter_end(of->priv);
- kfree(of->priv);
- }
+ struct cgroup_file_ctx *ctx = of->priv;
+
+ if (ctx->procs.started)
+ css_task_iter_end(&ctx->procs.iter);
}
static void *cgroup_procs_next(struct seq_file *s, void *v, loff_t *pos)
{
struct kernfs_open_file *of = s->private;
- struct css_task_iter *it = of->priv;
+ struct cgroup_file_ctx *ctx = of->priv;
if (pos)
(*pos)++;
- return css_task_iter_next(it);
+ return css_task_iter_next(&ctx->procs.iter);
}
static void *__cgroup_procs_start(struct seq_file *s, loff_t *pos,
@@ -4773,21 +4798,18 @@ static void *__cgroup_procs_start(struct seq_file *s, loff_t *pos,
{
struct kernfs_open_file *of = s->private;
struct cgroup *cgrp = seq_css(s)->cgroup;
- struct css_task_iter *it = of->priv;
+ struct cgroup_file_ctx *ctx = of->priv;
+ struct css_task_iter *it = &ctx->procs.iter;
/*
* When a seq_file is seeked, it's always traversed sequentially
* from position 0, so we can simply keep iterating on !0 *pos.
*/
- if (!it) {
+ if (!ctx->procs.started) {
if (WARN_ON_ONCE((*pos)))
return ERR_PTR(-EINVAL);
-
- it = kzalloc(sizeof(*it), GFP_KERNEL);
- if (!it)
- return ERR_PTR(-ENOMEM);
- of->priv = it;
css_task_iter_start(&cgrp->self, iter_flags, it);
+ ctx->procs.started = true;
} else if (!(*pos)) {
css_task_iter_end(it);
css_task_iter_start(&cgrp->self, iter_flags, it);
@@ -4838,9 +4860,9 @@ static int cgroup_may_write(const struct cgroup *cgrp, struct super_block *sb)
static int cgroup_procs_write_permission(struct cgroup *src_cgrp,
struct cgroup *dst_cgrp,
- struct super_block *sb)
+ struct super_block *sb,
+ struct cgroup_namespace *ns)
{
- struct cgroup_namespace *ns = current->nsproxy->cgroup_ns;
struct cgroup *com_cgrp = src_cgrp;
int ret;
@@ -4869,11 +4891,12 @@ static int cgroup_procs_write_permission(struct cgroup *src_cgrp,
static int cgroup_attach_permissions(struct cgroup *src_cgrp,
struct cgroup *dst_cgrp,
- struct super_block *sb, bool threadgroup)
+ struct super_block *sb, bool threadgroup,
+ struct cgroup_namespace *ns)
{
int ret = 0;
- ret = cgroup_procs_write_permission(src_cgrp, dst_cgrp, sb);
+ ret = cgroup_procs_write_permission(src_cgrp, dst_cgrp, sb, ns);
if (ret)
return ret;
@@ -4890,8 +4913,10 @@ static int cgroup_attach_permissions(struct cgroup *src_cgrp,
static ssize_t __cgroup_procs_write(struct kernfs_open_file *of, char *buf,
bool threadgroup)
{
+ struct cgroup_file_ctx *ctx = of->priv;
struct cgroup *src_cgrp, *dst_cgrp;
struct task_struct *task;
+ const struct cred *saved_cred;
ssize_t ret;
bool locked;
@@ -4909,9 +4934,16 @@ static ssize_t __cgroup_procs_write(struct kernfs_open_file *of, char *buf,
src_cgrp = task_cgroup_from_root(task, &cgrp_dfl_root);
spin_unlock_irq(&css_set_lock);
- /* process and thread migrations follow same delegation rule */
+ /*
+ * Process and thread migrations follow same delegation rule. Check
+ * permissions using the credentials from file open to protect against
+ * inherited fd attacks.
+ */
+ saved_cred = override_creds(of->file->f_cred);
ret = cgroup_attach_permissions(src_cgrp, dst_cgrp,
- of->file->f_path.dentry->d_sb, threadgroup);
+ of->file->f_path.dentry->d_sb,
+ threadgroup, ctx->ns);
+ revert_creds(saved_cred);
if (ret)
goto out_finish;
@@ -6130,7 +6162,8 @@ static int cgroup_css_set_fork(struct kernel_clone_args *kargs)
goto err;
ret = cgroup_attach_permissions(cset->dfl_cgrp, dst_cgrp, sb,
- !(kargs->flags & CLONE_THREAD));
+ !(kargs->flags & CLONE_THREAD),
+ current->nsproxy->cgroup_ns);
if (ret)
goto err;
diff --git a/kernel/entry/common.c b/kernel/entry/common.c
index d5a61d5..bad7136 100644
--- a/kernel/entry/common.c
+++ b/kernel/entry/common.c
@@ -187,7 +187,7 @@ static unsigned long exit_to_user_mode_loop(struct pt_regs *regs,
/* Check if any of the above work has queued a deferred wakeup */
tick_nohz_user_enter_prepare();
- ti_work = READ_ONCE(current_thread_info()->flags);
+ ti_work = read_thread_flags();
}
/* Return the latest work state for arch_exit_to_user_mode() */
@@ -196,7 +196,7 @@ static unsigned long exit_to_user_mode_loop(struct pt_regs *regs,
static void exit_to_user_mode_prepare(struct pt_regs *regs)
{
- unsigned long ti_work = READ_ONCE(current_thread_info()->flags);
+ unsigned long ti_work = read_thread_flags();
lockdep_assert_irqs_disabled();
diff --git a/kernel/entry/kvm.c b/kernel/entry/kvm.c
index 49972ee..96d476e 100644
--- a/kernel/entry/kvm.c
+++ b/kernel/entry/kvm.c
@@ -26,7 +26,7 @@ static int xfer_to_guest_mode_work(struct kvm_vcpu *vcpu, unsigned long ti_work)
if (ret)
return ret;
- ti_work = READ_ONCE(current_thread_info()->flags);
+ ti_work = read_thread_flags();
} while (ti_work & XFER_TO_GUEST_MODE_WORK || need_resched());
return 0;
}
@@ -43,7 +43,7 @@ int xfer_to_guest_mode_handle_work(struct kvm_vcpu *vcpu)
* disabled in the inner loop before going into guest mode. No need
* to disable interrupts here.
*/
- ti_work = READ_ONCE(current_thread_info()->flags);
+ ti_work = read_thread_flags();
if (!(ti_work & XFER_TO_GUEST_MODE_WORK))
return 0;
diff --git a/kernel/events/core.c b/kernel/events/core.c
index 30d94f6..1362b9b 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -1808,6 +1808,8 @@ list_add_event(struct perf_event *event, struct perf_event_context *ctx)
list_add_rcu(&event->event_entry, &ctx->event_list);
ctx->nr_events++;
+ if (event->hw.flags & PERF_EVENT_FLAG_USER_READ_CNT)
+ ctx->nr_user++;
if (event->attr.inherit_stat)
ctx->nr_stat++;
@@ -1999,6 +2001,8 @@ list_del_event(struct perf_event *event, struct perf_event_context *ctx)
event->attach_state &= ~PERF_ATTACH_CONTEXT;
ctx->nr_events--;
+ if (event->hw.flags & PERF_EVENT_FLAG_USER_READ_CNT)
+ ctx->nr_user--;
if (event->attr.inherit_stat)
ctx->nr_stat--;
diff --git a/kernel/irq/chip.c b/kernel/irq/chip.c
index f895265..c093246 100644
--- a/kernel/irq/chip.c
+++ b/kernel/irq/chip.c
@@ -575,8 +575,6 @@ EXPORT_SYMBOL_GPL(handle_simple_irq);
*/
void handle_untracked_irq(struct irq_desc *desc)
{
- unsigned int flags = 0;
-
raw_spin_lock(&desc->lock);
if (!irq_may_run(desc))
@@ -593,7 +591,7 @@ void handle_untracked_irq(struct irq_desc *desc)
irqd_set(&desc->irq_data, IRQD_IRQ_INPROGRESS);
raw_spin_unlock(&desc->lock);
- __handle_irq_event_percpu(desc, &flags);
+ __handle_irq_event_percpu(desc);
raw_spin_lock(&desc->lock);
irqd_clear(&desc->irq_data, IRQD_IRQ_INPROGRESS);
diff --git a/kernel/irq/handle.c b/kernel/irq/handle.c
index 2718200..9489f93 100644
--- a/kernel/irq/handle.c
+++ b/kernel/irq/handle.c
@@ -136,7 +136,7 @@ void __irq_wake_thread(struct irq_desc *desc, struct irqaction *action)
wake_up_process(action->thread);
}
-irqreturn_t __handle_irq_event_percpu(struct irq_desc *desc, unsigned int *flags)
+irqreturn_t __handle_irq_event_percpu(struct irq_desc *desc)
{
irqreturn_t retval = IRQ_NONE;
unsigned int irq = desc->irq_data.irq;
@@ -174,10 +174,6 @@ irqreturn_t __handle_irq_event_percpu(struct irq_desc *desc, unsigned int *flags
}
__irq_wake_thread(desc, action);
-
- fallthrough; /* to add to randomness */
- case IRQ_HANDLED:
- *flags |= action->flags;
break;
default:
@@ -193,11 +189,10 @@ irqreturn_t __handle_irq_event_percpu(struct irq_desc *desc, unsigned int *flags
irqreturn_t handle_irq_event_percpu(struct irq_desc *desc)
{
irqreturn_t retval;
- unsigned int flags = 0;
- retval = __handle_irq_event_percpu(desc, &flags);
+ retval = __handle_irq_event_percpu(desc);
- add_interrupt_randomness(desc->irq_data.irq, flags);
+ add_interrupt_randomness(desc->irq_data.irq);
if (!irq_settings_no_debug(desc))
note_interrupt(desc, retval);
diff --git a/kernel/irq/internals.h b/kernel/irq/internals.h
index 5436352..99cbdf5 100644
--- a/kernel/irq/internals.h
+++ b/kernel/irq/internals.h
@@ -103,7 +103,7 @@ extern int __irq_get_irqchip_state(struct irq_data *data,
extern void init_kstat_irqs(struct irq_desc *desc, int node, int nr);
-irqreturn_t __handle_irq_event_percpu(struct irq_desc *desc, unsigned int *flags);
+irqreturn_t __handle_irq_event_percpu(struct irq_desc *desc);
irqreturn_t handle_irq_event_percpu(struct irq_desc *desc);
irqreturn_t handle_irq_event(struct irq_desc *desc);
diff --git a/kernel/kcsan/Makefile b/kernel/kcsan/Makefile
index c2bb07f..e893b0e 100644
--- a/kernel/kcsan/Makefile
+++ b/kernel/kcsan/Makefile
@@ -8,6 +8,7 @@
CFLAGS_REMOVE_report.o = $(CC_FLAGS_FTRACE)
CFLAGS_core.o := $(call cc-option,-fno-conserve-stack) \
+ $(call cc-option,-mno-outline-atomics) \
-fno-stack-protector -DDISABLE_BRANCH_PROFILING
obj-y := core.o debugfs.o report.o
diff --git a/kernel/notifier.c b/kernel/notifier.c
index b8251dc..ba005eb 100644
--- a/kernel/notifier.c
+++ b/kernel/notifier.c
@@ -20,12 +20,13 @@ BLOCKING_NOTIFIER_HEAD(reboot_notifier_list);
*/
static int notifier_chain_register(struct notifier_block **nl,
- struct notifier_block *n)
+ struct notifier_block *n)
{
while ((*nl) != NULL) {
if (unlikely((*nl) == n)) {
- WARN(1, "double register detected");
- return 0;
+ WARN(1, "notifier callback %ps already registered",
+ n->notifier_call);
+ return -EEXIST;
}
if (n->priority > (*nl)->priority)
break;
@@ -134,7 +135,7 @@ static int notifier_call_chain_robust(struct notifier_block **nl,
*
* Adds a notifier to an atomic notifier chain.
*
- * Currently always returns zero.
+ * Returns 0 on success, %-EEXIST on error.
*/
int atomic_notifier_chain_register(struct atomic_notifier_head *nh,
struct notifier_block *n)
@@ -216,7 +217,7 @@ NOKPROBE_SYMBOL(atomic_notifier_call_chain);
* Adds a notifier to a blocking notifier chain.
* Must be called in process context.
*
- * Currently always returns zero.
+ * Returns 0 on success, %-EEXIST on error.
*/
int blocking_notifier_chain_register(struct blocking_notifier_head *nh,
struct notifier_block *n)
@@ -335,7 +336,7 @@ EXPORT_SYMBOL_GPL(blocking_notifier_call_chain);
* Adds a notifier to a raw notifier chain.
* All locking must be provided by the caller.
*
- * Currently always returns zero.
+ * Returns 0 on success, %-EEXIST on error.
*/
int raw_notifier_chain_register(struct raw_notifier_head *nh,
struct notifier_block *n)
@@ -406,7 +407,7 @@ EXPORT_SYMBOL_GPL(raw_notifier_call_chain);
* Adds a notifier to an SRCU notifier chain.
* Must be called in process context.
*
- * Currently always returns zero.
+ * Returns 0 on success, %-EEXIST on error.
*/
int srcu_notifier_chain_register(struct srcu_notifier_head *nh,
struct notifier_block *n)
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 7756310..bb5e7fd 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -8520,7 +8520,7 @@ void sched_show_task(struct task_struct *p)
rcu_read_unlock();
pr_cont(" stack:%5lu pid:%5d ppid:%6d flags:0x%08lx\n",
free, task_pid_nr(p), ppid,
- (unsigned long)task_thread_info(p)->flags);
+ read_task_thread_flags(p));
print_worker_info(KERN_INFO, p);
print_stop_info(KERN_INFO, p);
diff --git a/kernel/sysctl.c b/kernel/sysctl.c
index 083be6a..d7ed1df 100644
--- a/kernel/sysctl.c
+++ b/kernel/sysctl.c
@@ -33,6 +33,7 @@
#include <linux/security.h>
#include <linux/ctype.h>
#include <linux/kmemleak.h>
+#include <linux/filter.h>
#include <linux/fs.h>
#include <linux/init.h>
#include <linux/kernel.h>
diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c
index ae97550..21aa306 100644
--- a/kernel/trace/bpf_trace.c
+++ b/kernel/trace/bpf_trace.c
@@ -345,7 +345,7 @@ static const struct bpf_func_proto bpf_probe_write_user_proto = {
.gpl_only = true,
.ret_type = RET_INTEGER,
.arg1_type = ARG_ANYTHING,
- .arg2_type = ARG_PTR_TO_MEM,
+ .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg3_type = ARG_CONST_SIZE,
};
@@ -394,7 +394,7 @@ static const struct bpf_func_proto bpf_trace_printk_proto = {
.func = bpf_trace_printk,
.gpl_only = true,
.ret_type = RET_INTEGER,
- .arg1_type = ARG_PTR_TO_MEM,
+ .arg1_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg2_type = ARG_CONST_SIZE,
};
@@ -450,9 +450,9 @@ static const struct bpf_func_proto bpf_trace_vprintk_proto = {
.func = bpf_trace_vprintk,
.gpl_only = true,
.ret_type = RET_INTEGER,
- .arg1_type = ARG_PTR_TO_MEM,
+ .arg1_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg2_type = ARG_CONST_SIZE,
- .arg3_type = ARG_PTR_TO_MEM_OR_NULL,
+ .arg3_type = ARG_PTR_TO_MEM | PTR_MAYBE_NULL | MEM_RDONLY,
.arg4_type = ARG_CONST_SIZE_OR_ZERO,
};
@@ -492,9 +492,9 @@ static const struct bpf_func_proto bpf_seq_printf_proto = {
.ret_type = RET_INTEGER,
.arg1_type = ARG_PTR_TO_BTF_ID,
.arg1_btf_id = &btf_seq_file_ids[0],
- .arg2_type = ARG_PTR_TO_MEM,
+ .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg3_type = ARG_CONST_SIZE,
- .arg4_type = ARG_PTR_TO_MEM_OR_NULL,
+ .arg4_type = ARG_PTR_TO_MEM | PTR_MAYBE_NULL | MEM_RDONLY,
.arg5_type = ARG_CONST_SIZE_OR_ZERO,
};
@@ -509,7 +509,7 @@ static const struct bpf_func_proto bpf_seq_write_proto = {
.ret_type = RET_INTEGER,
.arg1_type = ARG_PTR_TO_BTF_ID,
.arg1_btf_id = &btf_seq_file_ids[0],
- .arg2_type = ARG_PTR_TO_MEM,
+ .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg3_type = ARG_CONST_SIZE_OR_ZERO,
};
@@ -533,7 +533,7 @@ static const struct bpf_func_proto bpf_seq_printf_btf_proto = {
.ret_type = RET_INTEGER,
.arg1_type = ARG_PTR_TO_BTF_ID,
.arg1_btf_id = &btf_seq_file_ids[0],
- .arg2_type = ARG_PTR_TO_MEM,
+ .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg3_type = ARG_CONST_SIZE_OR_ZERO,
.arg4_type = ARG_ANYTHING,
};
@@ -694,7 +694,7 @@ static const struct bpf_func_proto bpf_perf_event_output_proto = {
.arg1_type = ARG_PTR_TO_CTX,
.arg2_type = ARG_CONST_MAP_PTR,
.arg3_type = ARG_ANYTHING,
- .arg4_type = ARG_PTR_TO_MEM,
+ .arg4_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg5_type = ARG_CONST_SIZE_OR_ZERO,
};
@@ -764,7 +764,7 @@ const struct bpf_func_proto bpf_get_current_task_btf_proto = {
.func = bpf_get_current_task_btf,
.gpl_only = true,
.ret_type = RET_PTR_TO_BTF_ID,
- .ret_btf_id = &btf_task_struct_ids[0],
+ .ret_btf_id = &btf_tracing_ids[BTF_TRACING_TYPE_TASK],
};
BPF_CALL_1(bpf_task_pt_regs, struct task_struct *, task)
@@ -779,7 +779,7 @@ const struct bpf_func_proto bpf_task_pt_regs_proto = {
.func = bpf_task_pt_regs,
.gpl_only = true,
.arg1_type = ARG_PTR_TO_BTF_ID,
- .arg1_btf_id = &btf_task_struct_ids[0],
+ .arg1_btf_id = &btf_tracing_ids[BTF_TRACING_TYPE_TASK],
.ret_type = RET_PTR_TO_BTF_ID,
.ret_btf_id = &bpf_task_pt_regs_ids[0],
};
@@ -1004,7 +1004,7 @@ const struct bpf_func_proto bpf_snprintf_btf_proto = {
.ret_type = RET_INTEGER,
.arg1_type = ARG_PTR_TO_MEM,
.arg2_type = ARG_CONST_SIZE,
- .arg3_type = ARG_PTR_TO_MEM,
+ .arg3_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg4_type = ARG_CONST_SIZE,
.arg5_type = ARG_ANYTHING,
};
@@ -1012,7 +1012,7 @@ const struct bpf_func_proto bpf_snprintf_btf_proto = {
BPF_CALL_1(bpf_get_func_ip_tracing, void *, ctx)
{
/* This helper call is inlined by verifier. */
- return ((u64 *)ctx)[-1];
+ return ((u64 *)ctx)[-2];
}
static const struct bpf_func_proto bpf_get_func_ip_proto_tracing = {
@@ -1091,6 +1091,53 @@ static const struct bpf_func_proto bpf_get_branch_snapshot_proto = {
.arg2_type = ARG_CONST_SIZE_OR_ZERO,
};
+BPF_CALL_3(get_func_arg, void *, ctx, u32, n, u64 *, value)
+{
+ /* This helper call is inlined by verifier. */
+ u64 nr_args = ((u64 *)ctx)[-1];
+
+ if ((u64) n >= nr_args)
+ return -EINVAL;
+ *value = ((u64 *)ctx)[n];
+ return 0;
+}
+
+static const struct bpf_func_proto bpf_get_func_arg_proto = {
+ .func = get_func_arg,
+ .ret_type = RET_INTEGER,
+ .arg1_type = ARG_PTR_TO_CTX,
+ .arg2_type = ARG_ANYTHING,
+ .arg3_type = ARG_PTR_TO_LONG,
+};
+
+BPF_CALL_2(get_func_ret, void *, ctx, u64 *, value)
+{
+ /* This helper call is inlined by verifier. */
+ u64 nr_args = ((u64 *)ctx)[-1];
+
+ *value = ((u64 *)ctx)[nr_args];
+ return 0;
+}
+
+static const struct bpf_func_proto bpf_get_func_ret_proto = {
+ .func = get_func_ret,
+ .ret_type = RET_INTEGER,
+ .arg1_type = ARG_PTR_TO_CTX,
+ .arg2_type = ARG_PTR_TO_LONG,
+};
+
+BPF_CALL_1(get_func_arg_cnt, void *, ctx)
+{
+ /* This helper call is inlined by verifier. */
+ return ((u64 *)ctx)[-1];
+}
+
+static const struct bpf_func_proto bpf_get_func_arg_cnt_proto = {
+ .func = get_func_arg_cnt,
+ .ret_type = RET_INTEGER,
+ .arg1_type = ARG_PTR_TO_CTX,
+};
+
static const struct bpf_func_proto *
bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
{
@@ -1206,6 +1253,8 @@ bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
return &bpf_get_func_ip_proto_tracing;
case BPF_FUNC_get_branch_snapshot:
return &bpf_get_branch_snapshot_proto;
+ case BPF_FUNC_find_vma:
+ return &bpf_find_vma_proto;
case BPF_FUNC_trace_vprintk:
return bpf_get_trace_vprintk_proto();
default:
@@ -1285,7 +1334,7 @@ static const struct bpf_func_proto bpf_perf_event_output_proto_tp = {
.arg1_type = ARG_PTR_TO_CTX,
.arg2_type = ARG_CONST_MAP_PTR,
.arg3_type = ARG_ANYTHING,
- .arg4_type = ARG_PTR_TO_MEM,
+ .arg4_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg5_type = ARG_CONST_SIZE_OR_ZERO,
};
@@ -1400,9 +1449,6 @@ static const struct bpf_func_proto bpf_perf_prog_read_value_proto = {
BPF_CALL_4(bpf_read_branch_records, struct bpf_perf_event_data_kern *, ctx,
void *, buf, u32, size, u64, flags)
{
-#ifndef CONFIG_X86
- return -ENOENT;
-#else
static const u32 br_entry_size = sizeof(struct perf_branch_entry);
struct perf_branch_stack *br_stack = ctx->data->br_stack;
u32 to_copy;
@@ -1411,7 +1457,7 @@ BPF_CALL_4(bpf_read_branch_records, struct bpf_perf_event_data_kern *, ctx,
return -EINVAL;
if (unlikely(!br_stack))
- return -EINVAL;
+ return -ENOENT;
if (flags & BPF_F_GET_BRANCH_RECORDS_SIZE)
return br_stack->nr * br_entry_size;
@@ -1423,7 +1469,6 @@ BPF_CALL_4(bpf_read_branch_records, struct bpf_perf_event_data_kern *, ctx,
memcpy(buf, br_stack->entries, to_copy);
return to_copy;
-#endif
}
static const struct bpf_func_proto bpf_read_branch_records_proto = {
@@ -1511,7 +1556,7 @@ static const struct bpf_func_proto bpf_perf_event_output_proto_raw_tp = {
.arg1_type = ARG_PTR_TO_CTX,
.arg2_type = ARG_CONST_MAP_PTR,
.arg3_type = ARG_ANYTHING,
- .arg4_type = ARG_PTR_TO_MEM,
+ .arg4_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg5_type = ARG_CONST_SIZE_OR_ZERO,
};
@@ -1565,7 +1610,7 @@ static const struct bpf_func_proto bpf_get_stack_proto_raw_tp = {
.gpl_only = true,
.ret_type = RET_INTEGER,
.arg1_type = ARG_PTR_TO_CTX,
- .arg2_type = ARG_PTR_TO_MEM,
+ .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg3_type = ARG_CONST_SIZE_OR_ZERO,
.arg4_type = ARG_ANYTHING,
};
@@ -1631,6 +1676,12 @@ tracing_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
NULL;
case BPF_FUNC_d_path:
return &bpf_d_path_proto;
+ case BPF_FUNC_get_func_arg:
+ return bpf_prog_has_trampoline(prog) ? &bpf_get_func_arg_proto : NULL;
+ case BPF_FUNC_get_func_ret:
+ return bpf_prog_has_trampoline(prog) ? &bpf_get_func_ret_proto : NULL;
+ case BPF_FUNC_get_func_arg_cnt:
+ return bpf_prog_has_trampoline(prog) ? &bpf_get_func_arg_cnt_proto : NULL;
default:
fn = raw_tp_prog_func_proto(func_id, prog);
if (!fn && prog->expected_attach_type == BPF_TRACE_ITER)
diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
index 88de94d..78ea542 100644
--- a/kernel/trace/trace.c
+++ b/kernel/trace/trace.c
@@ -3207,7 +3207,7 @@ struct trace_buffer_struct {
char buffer[4][TRACE_BUF_SIZE];
};
-static struct trace_buffer_struct *trace_percpu_buffer;
+static struct trace_buffer_struct __percpu *trace_percpu_buffer;
/*
* This allows for lockless recording. If we're nested too deeply, then
@@ -3217,7 +3217,7 @@ static char *get_trace_buf(void)
{
struct trace_buffer_struct *buffer = this_cpu_ptr(trace_percpu_buffer);
- if (!buffer || buffer->nesting >= 4)
+ if (!trace_percpu_buffer || buffer->nesting >= 4)
return NULL;
buffer->nesting++;
@@ -3236,7 +3236,7 @@ static void put_trace_buf(void)
static int alloc_percpu_trace_buffer(void)
{
- struct trace_buffer_struct *buffers;
+ struct trace_buffer_struct __percpu *buffers;
if (trace_percpu_buffer)
return 0;
diff --git a/kernel/trace/trace_kprobe.c b/kernel/trace/trace_kprobe.c
index 33272a7..4e1257f 100644
--- a/kernel/trace/trace_kprobe.c
+++ b/kernel/trace/trace_kprobe.c
@@ -7,6 +7,7 @@
*/
#define pr_fmt(fmt) "trace_kprobe: " fmt
+#include <linux/bpf-cgroup.h>
#include <linux/security.h>
#include <linux/module.h>
#include <linux/uaccess.h>
diff --git a/kernel/trace/trace_uprobe.c b/kernel/trace/trace_uprobe.c
index f5f0039..4f35514a 100644
--- a/kernel/trace/trace_uprobe.c
+++ b/kernel/trace/trace_uprobe.c
@@ -7,6 +7,7 @@
*/
#define pr_fmt(fmt) "trace_uprobe: " fmt
+#include <linux/bpf-cgroup.h>
#include <linux/security.h>
#include <linux/ctype.h>
#include <linux/module.h>
diff --git a/lib/Kconfig b/lib/Kconfig
index 5e7165e..655b0e43 100644
--- a/lib/Kconfig
+++ b/lib/Kconfig
@@ -680,6 +680,11 @@
Select the hash size as a power of 2 for the stackdepot hash table.
Choose a lower value to reduce the memory impact.
+config REF_TRACKER
+ bool
+ depends on STACKTRACE_SUPPORT
+ select STACKDEPOT
+
config SBITMAP
bool
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index 5e14e32..c77fe36 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -599,6 +599,11 @@
Say Y here if you need to enable miscellaneous debug code that should
be under a more specific debug option but isn't.
+menu "Networking Debugging"
+
+source "net/Kconfig.debug"
+
+endmenu # "Networking Debugging"
menu "Memory Debugging"
@@ -2107,6 +2112,16 @@
Say N if you are unsure.
+config TEST_REF_TRACKER
+ tristate "Self test for reference tracker"
+ depends on DEBUG_KERNEL && STACKTRACE_SUPPORT
+ select REF_TRACKER
+ help
+ This option provides a kernel module performing tests
+ using reference tracker infrastructure.
+
+ Say N if you are unsure.
+
config RBTREE_TEST
tristate "Red-Black tree test"
depends on DEBUG_KERNEL
diff --git a/lib/Makefile b/lib/Makefile
index 364c23f..b213a7b 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -101,7 +101,7 @@
obj-$(CONFIG_TEST_HMM) += test_hmm.o
obj-$(CONFIG_TEST_FREE_PAGES) += test_free_pages.o
obj-$(CONFIG_KPROBES_SANITY_TEST) += test_kprobes.o
-
+obj-$(CONFIG_TEST_REF_TRACKER) += test_ref_tracker.o
#
# CFLAGS for compiling floating point code inside the kernel. x86/Makefile turns
# off the generation of FPU/SSE* instructions for kernel proper but FPU_FLAGS
@@ -270,6 +270,8 @@
KASAN_SANITIZE_stackdepot.o := n
KCOV_INSTRUMENT_stackdepot.o := n
+obj-$(CONFIG_REF_TRACKER) += ref_tracker.o
+
libfdt_files = fdt.o fdt_ro.o fdt_wip.o fdt_rw.o fdt_sw.o fdt_strerror.o \
fdt_empty_tree.o fdt_addresses.o
$(foreach file, $(libfdt_files), \
diff --git a/lib/crypto/Kconfig b/lib/crypto/Kconfig
index 545ccbd..8620f38 100644
--- a/lib/crypto/Kconfig
+++ b/lib/crypto/Kconfig
@@ -1,7 +1,5 @@
# SPDX-License-Identifier: GPL-2.0
-comment "Crypto library routines"
-
config CRYPTO_LIB_AES
tristate
@@ -9,14 +7,14 @@
tristate
config CRYPTO_ARCH_HAVE_LIB_BLAKE2S
- tristate
+ bool
help
Declares whether the architecture provides an arch-specific
accelerated implementation of the Blake2s library interface,
either builtin or as a module.
config CRYPTO_LIB_BLAKE2S_GENERIC
- tristate
+ def_bool !CRYPTO_ARCH_HAVE_LIB_BLAKE2S
help
This symbol can be depended upon by arch implementations of the
Blake2s library interface that require the generic code as a
@@ -24,15 +22,6 @@
implementation is enabled, this implementation serves the users
of CRYPTO_LIB_BLAKE2S.
-config CRYPTO_LIB_BLAKE2S
- tristate "BLAKE2s hash function library"
- depends on CRYPTO_ARCH_HAVE_LIB_BLAKE2S || !CRYPTO_ARCH_HAVE_LIB_BLAKE2S
- select CRYPTO_LIB_BLAKE2S_GENERIC if CRYPTO_ARCH_HAVE_LIB_BLAKE2S=n
- help
- Enable the Blake2s library interface. This interface may be fulfilled
- by either the generic implementation or an arch-specific one, if one
- is available and enabled.
-
config CRYPTO_ARCH_HAVE_LIB_CHACHA
tristate
help
@@ -51,7 +40,7 @@
of CRYPTO_LIB_CHACHA.
config CRYPTO_LIB_CHACHA
- tristate "ChaCha library interface"
+ tristate
depends on CRYPTO_ARCH_HAVE_LIB_CHACHA || !CRYPTO_ARCH_HAVE_LIB_CHACHA
select CRYPTO_LIB_CHACHA_GENERIC if CRYPTO_ARCH_HAVE_LIB_CHACHA=n
help
@@ -76,7 +65,7 @@
of CRYPTO_LIB_CURVE25519.
config CRYPTO_LIB_CURVE25519
- tristate "Curve25519 scalar multiplication library"
+ tristate
depends on CRYPTO_ARCH_HAVE_LIB_CURVE25519 || !CRYPTO_ARCH_HAVE_LIB_CURVE25519
select CRYPTO_LIB_CURVE25519_GENERIC if CRYPTO_ARCH_HAVE_LIB_CURVE25519=n
help
@@ -111,7 +100,7 @@
of CRYPTO_LIB_POLY1305.
config CRYPTO_LIB_POLY1305
- tristate "Poly1305 library interface"
+ tristate
depends on CRYPTO_ARCH_HAVE_LIB_POLY1305 || !CRYPTO_ARCH_HAVE_LIB_POLY1305
select CRYPTO_LIB_POLY1305_GENERIC if CRYPTO_ARCH_HAVE_LIB_POLY1305=n
help
@@ -120,7 +109,7 @@
is available and enabled.
config CRYPTO_LIB_CHACHA20POLY1305
- tristate "ChaCha20-Poly1305 AEAD support (8-byte nonce library version)"
+ tristate
depends on CRYPTO_ARCH_HAVE_LIB_CHACHA || !CRYPTO_ARCH_HAVE_LIB_CHACHA
depends on CRYPTO_ARCH_HAVE_LIB_POLY1305 || !CRYPTO_ARCH_HAVE_LIB_POLY1305
select CRYPTO_LIB_CHACHA
diff --git a/lib/crypto/Makefile b/lib/crypto/Makefile
index 73205ed..ed43a41 100644
--- a/lib/crypto/Makefile
+++ b/lib/crypto/Makefile
@@ -10,11 +10,10 @@
obj-$(CONFIG_CRYPTO_LIB_ARC4) += libarc4.o
libarc4-y := arc4.o
-obj-$(CONFIG_CRYPTO_LIB_BLAKE2S_GENERIC) += libblake2s-generic.o
-libblake2s-generic-y += blake2s-generic.o
-
-obj-$(CONFIG_CRYPTO_LIB_BLAKE2S) += libblake2s.o
-libblake2s-y += blake2s.o
+# blake2s is used by the /dev/random driver which is always builtin
+obj-y += libblake2s.o
+libblake2s-y := blake2s.o
+libblake2s-$(CONFIG_CRYPTO_LIB_BLAKE2S_GENERIC) += blake2s-generic.o
obj-$(CONFIG_CRYPTO_LIB_CHACHA20POLY1305) += libchacha20poly1305.o
libchacha20poly1305-y += chacha20poly1305.o
diff --git a/lib/crypto/blake2s-generic.c b/lib/crypto/blake2s-generic.c
index 04ff8df..75ccb3e 100644
--- a/lib/crypto/blake2s-generic.c
+++ b/lib/crypto/blake2s-generic.c
@@ -37,7 +37,11 @@ static inline void blake2s_increment_counter(struct blake2s_state *state,
state->t[1] += (state->t[0] < inc);
}
-void blake2s_compress_generic(struct blake2s_state *state,const u8 *block,
+void blake2s_compress(struct blake2s_state *state, const u8 *block,
+ size_t nblocks, const u32 inc)
+ __weak __alias(blake2s_compress_generic);
+
+void blake2s_compress_generic(struct blake2s_state *state, const u8 *block,
size_t nblocks, const u32 inc)
{
u32 m[16];
diff --git a/lib/crypto/blake2s.c b/lib/crypto/blake2s.c
index 4055aa5..93f2ae0 100644
--- a/lib/crypto/blake2s.c
+++ b/lib/crypto/blake2s.c
@@ -16,12 +16,6 @@
#include <linux/init.h>
#include <linux/bug.h>
-#if IS_ENABLED(CONFIG_CRYPTO_ARCH_HAVE_LIB_BLAKE2S)
-# define blake2s_compress blake2s_compress_arch
-#else
-# define blake2s_compress blake2s_compress_generic
-#endif
-
void blake2s_update(struct blake2s_state *state, const u8 *in, size_t inlen)
{
__blake2s_update(state, in, inlen, blake2s_compress);
diff --git a/lib/kunit/test.c b/lib/kunit/test.c
index 3bd741e..c7ed4aa 100644
--- a/lib/kunit/test.c
+++ b/lib/kunit/test.c
@@ -504,25 +504,28 @@ int kunit_run_tests(struct kunit_suite *suite)
struct kunit_result_stats param_stats = { 0 };
test_case->status = KUNIT_SKIPPED;
- if (test_case->generate_params) {
+ if (!test_case->generate_params) {
+ /* Non-parameterised test. */
+ kunit_run_case_catch_errors(suite, test_case, &test);
+ kunit_update_stats(¶m_stats, test.status);
+ } else {
/* Get initial param. */
param_desc[0] = '\0';
test.param_value = test_case->generate_params(NULL, param_desc);
- }
+ kunit_log(KERN_INFO, &test, KUNIT_SUBTEST_INDENT KUNIT_SUBTEST_INDENT
+ "# Subtest: %s", test_case->name);
- do {
- kunit_run_case_catch_errors(suite, test_case, &test);
+ while (test.param_value) {
+ kunit_run_case_catch_errors(suite, test_case, &test);
- if (test_case->generate_params) {
if (param_desc[0] == '\0') {
snprintf(param_desc, sizeof(param_desc),
"param-%d", test.param_index);
}
kunit_log(KERN_INFO, &test,
- KUNIT_SUBTEST_INDENT
- "# %s: %s %d - %s",
- test_case->name,
+ KUNIT_SUBTEST_INDENT KUNIT_SUBTEST_INDENT
+ "%s %d - %s",
kunit_status_to_ok_not_ok(test.status),
test.param_index + 1, param_desc);
@@ -530,11 +533,11 @@ int kunit_run_tests(struct kunit_suite *suite)
param_desc[0] = '\0';
test.param_value = test_case->generate_params(test.param_value, param_desc);
test.param_index++;
+
+ kunit_update_stats(¶m_stats, test.status);
}
+ }
- kunit_update_stats(¶m_stats, test.status);
-
- } while (test.param_value);
kunit_print_test_stats(&test, param_stats);
diff --git a/lib/objagg.c b/lib/objagg.c
index 5e1676c..1e24862 100644
--- a/lib/objagg.c
+++ b/lib/objagg.c
@@ -781,7 +781,6 @@ static struct objagg_tmp_graph *objagg_tmp_graph_create(struct objagg *objagg)
struct objagg_tmp_node *node;
struct objagg_tmp_node *pnode;
struct objagg_obj *objagg_obj;
- size_t alloc_size;
int i, j;
graph = kzalloc(sizeof(*graph), GFP_KERNEL);
@@ -793,9 +792,7 @@ static struct objagg_tmp_graph *objagg_tmp_graph_create(struct objagg *objagg)
goto err_nodes_alloc;
graph->nodes_count = nodes_count;
- alloc_size = BITS_TO_LONGS(nodes_count * nodes_count) *
- sizeof(unsigned long);
- graph->edges = kzalloc(alloc_size, GFP_KERNEL);
+ graph->edges = bitmap_zalloc(nodes_count * nodes_count, GFP_KERNEL);
if (!graph->edges)
goto err_edges_alloc;
@@ -833,7 +830,7 @@ static struct objagg_tmp_graph *objagg_tmp_graph_create(struct objagg *objagg)
static void objagg_tmp_graph_destroy(struct objagg_tmp_graph *graph)
{
- kfree(graph->edges);
+ bitmap_free(graph->edges);
kfree(graph->nodes);
kfree(graph);
}
diff --git a/lib/ref_tracker.c b/lib/ref_tracker.c
new file mode 100644
index 0000000..0ae2e66
--- /dev/null
+++ b/lib/ref_tracker.c
@@ -0,0 +1,140 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+#include <linux/export.h>
+#include <linux/ref_tracker.h>
+#include <linux/slab.h>
+#include <linux/stacktrace.h>
+#include <linux/stackdepot.h>
+
+#define REF_TRACKER_STACK_ENTRIES 16
+
+struct ref_tracker {
+ struct list_head head; /* anchor into dir->list or dir->quarantine */
+ bool dead;
+ depot_stack_handle_t alloc_stack_handle;
+ depot_stack_handle_t free_stack_handle;
+};
+
+void ref_tracker_dir_exit(struct ref_tracker_dir *dir)
+{
+ struct ref_tracker *tracker, *n;
+ unsigned long flags;
+ bool leak = false;
+
+ spin_lock_irqsave(&dir->lock, flags);
+ list_for_each_entry_safe(tracker, n, &dir->quarantine, head) {
+ list_del(&tracker->head);
+ kfree(tracker);
+ dir->quarantine_avail++;
+ }
+ list_for_each_entry_safe(tracker, n, &dir->list, head) {
+ pr_err("leaked reference.\n");
+ if (tracker->alloc_stack_handle)
+ stack_depot_print(tracker->alloc_stack_handle);
+ leak = true;
+ list_del(&tracker->head);
+ kfree(tracker);
+ }
+ spin_unlock_irqrestore(&dir->lock, flags);
+ WARN_ON_ONCE(leak);
+ WARN_ON_ONCE(refcount_read(&dir->untracked) != 1);
+}
+EXPORT_SYMBOL(ref_tracker_dir_exit);
+
+void ref_tracker_dir_print(struct ref_tracker_dir *dir,
+ unsigned int display_limit)
+{
+ struct ref_tracker *tracker;
+ unsigned long flags;
+ unsigned int i = 0;
+
+ spin_lock_irqsave(&dir->lock, flags);
+ list_for_each_entry(tracker, &dir->list, head) {
+ if (i < display_limit) {
+ pr_err("leaked reference.\n");
+ if (tracker->alloc_stack_handle)
+ stack_depot_print(tracker->alloc_stack_handle);
+ i++;
+ } else {
+ break;
+ }
+ }
+ spin_unlock_irqrestore(&dir->lock, flags);
+}
+EXPORT_SYMBOL(ref_tracker_dir_print);
+
+int ref_tracker_alloc(struct ref_tracker_dir *dir,
+ struct ref_tracker **trackerp,
+ gfp_t gfp)
+{
+ unsigned long entries[REF_TRACKER_STACK_ENTRIES];
+ struct ref_tracker *tracker;
+ unsigned int nr_entries;
+ unsigned long flags;
+
+ *trackerp = tracker = kzalloc(sizeof(*tracker), gfp | __GFP_NOFAIL);
+ if (unlikely(!tracker)) {
+ pr_err_once("memory allocation failure, unreliable refcount tracker.\n");
+ refcount_inc(&dir->untracked);
+ return -ENOMEM;
+ }
+ nr_entries = stack_trace_save(entries, ARRAY_SIZE(entries), 1);
+ nr_entries = filter_irq_stacks(entries, nr_entries);
+ tracker->alloc_stack_handle = stack_depot_save(entries, nr_entries, gfp);
+
+ spin_lock_irqsave(&dir->lock, flags);
+ list_add(&tracker->head, &dir->list);
+ spin_unlock_irqrestore(&dir->lock, flags);
+ return 0;
+}
+EXPORT_SYMBOL_GPL(ref_tracker_alloc);
+
+int ref_tracker_free(struct ref_tracker_dir *dir,
+ struct ref_tracker **trackerp)
+{
+ unsigned long entries[REF_TRACKER_STACK_ENTRIES];
+ struct ref_tracker *tracker = *trackerp;
+ depot_stack_handle_t stack_handle;
+ unsigned int nr_entries;
+ unsigned long flags;
+
+ if (!tracker) {
+ refcount_dec(&dir->untracked);
+ return -EEXIST;
+ }
+ nr_entries = stack_trace_save(entries, ARRAY_SIZE(entries), 1);
+ nr_entries = filter_irq_stacks(entries, nr_entries);
+ stack_handle = stack_depot_save(entries, nr_entries, GFP_ATOMIC);
+
+ spin_lock_irqsave(&dir->lock, flags);
+ if (tracker->dead) {
+ pr_err("reference already released.\n");
+ if (tracker->alloc_stack_handle) {
+ pr_err("allocated in:\n");
+ stack_depot_print(tracker->alloc_stack_handle);
+ }
+ if (tracker->free_stack_handle) {
+ pr_err("freed in:\n");
+ stack_depot_print(tracker->free_stack_handle);
+ }
+ spin_unlock_irqrestore(&dir->lock, flags);
+ WARN_ON_ONCE(1);
+ return -EINVAL;
+ }
+ tracker->dead = true;
+
+ tracker->free_stack_handle = stack_handle;
+
+ list_move_tail(&tracker->head, &dir->quarantine);
+ if (!dir->quarantine_avail) {
+ tracker = list_first_entry(&dir->quarantine, struct ref_tracker, head);
+ list_del(&tracker->head);
+ } else {
+ dir->quarantine_avail--;
+ tracker = NULL;
+ }
+ spin_unlock_irqrestore(&dir->lock, flags);
+
+ kfree(tracker);
+ return 0;
+}
+EXPORT_SYMBOL_GPL(ref_tracker_free);
diff --git a/lib/test_bpf.c b/lib/test_bpf.c
index adae395..0c5cb2d 100644
--- a/lib/test_bpf.c
+++ b/lib/test_bpf.c
@@ -14683,7 +14683,7 @@ static struct tail_call_test tail_call_tests[] = {
BPF_EXIT_INSN(),
},
.flags = FLAG_NEED_STATE | FLAG_RESULT_IN_STATE,
- .result = (MAX_TAIL_CALL_CNT + 1 + 1) * MAX_TESTRUNS,
+ .result = (MAX_TAIL_CALL_CNT + 1) * MAX_TESTRUNS,
},
{
"Tail call count preserved across function calls",
@@ -14705,7 +14705,7 @@ static struct tail_call_test tail_call_tests[] = {
},
.stack_depth = 8,
.flags = FLAG_NEED_STATE | FLAG_RESULT_IN_STATE,
- .result = (MAX_TAIL_CALL_CNT + 1 + 1) * MAX_TESTRUNS,
+ .result = (MAX_TAIL_CALL_CNT + 1) * MAX_TESTRUNS,
},
{
"Tail call error path, NULL target",
diff --git a/lib/test_ref_tracker.c b/lib/test_ref_tracker.c
new file mode 100644
index 0000000..19d7dec
--- /dev/null
+++ b/lib/test_ref_tracker.c
@@ -0,0 +1,115 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Referrence tracker self test.
+ *
+ * Copyright (c) 2021 Eric Dumazet <edumazet@google.com>
+ */
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/delay.h>
+#include <linux/ref_tracker.h>
+#include <linux/slab.h>
+#include <linux/timer.h>
+
+static struct ref_tracker_dir ref_dir;
+static struct ref_tracker *tracker[20];
+
+#define TRT_ALLOC(X) static noinline void \
+ alloctest_ref_tracker_alloc##X(struct ref_tracker_dir *dir, \
+ struct ref_tracker **trackerp) \
+ { \
+ ref_tracker_alloc(dir, trackerp, GFP_KERNEL); \
+ }
+
+TRT_ALLOC(1)
+TRT_ALLOC(2)
+TRT_ALLOC(3)
+TRT_ALLOC(4)
+TRT_ALLOC(5)
+TRT_ALLOC(6)
+TRT_ALLOC(7)
+TRT_ALLOC(8)
+TRT_ALLOC(9)
+TRT_ALLOC(10)
+TRT_ALLOC(11)
+TRT_ALLOC(12)
+TRT_ALLOC(13)
+TRT_ALLOC(14)
+TRT_ALLOC(15)
+TRT_ALLOC(16)
+TRT_ALLOC(17)
+TRT_ALLOC(18)
+TRT_ALLOC(19)
+
+#undef TRT_ALLOC
+
+static noinline void
+alloctest_ref_tracker_free(struct ref_tracker_dir *dir,
+ struct ref_tracker **trackerp)
+{
+ ref_tracker_free(dir, trackerp);
+}
+
+
+static struct timer_list test_ref_tracker_timer;
+static atomic_t test_ref_timer_done = ATOMIC_INIT(0);
+
+static void test_ref_tracker_timer_func(struct timer_list *t)
+{
+ ref_tracker_alloc(&ref_dir, &tracker[0], GFP_ATOMIC);
+ atomic_set(&test_ref_timer_done, 1);
+}
+
+static int __init test_ref_tracker_init(void)
+{
+ int i;
+
+ ref_tracker_dir_init(&ref_dir, 100);
+
+ timer_setup(&test_ref_tracker_timer, test_ref_tracker_timer_func, 0);
+ mod_timer(&test_ref_tracker_timer, jiffies + 1);
+
+ alloctest_ref_tracker_alloc1(&ref_dir, &tracker[1]);
+ alloctest_ref_tracker_alloc2(&ref_dir, &tracker[2]);
+ alloctest_ref_tracker_alloc3(&ref_dir, &tracker[3]);
+ alloctest_ref_tracker_alloc4(&ref_dir, &tracker[4]);
+ alloctest_ref_tracker_alloc5(&ref_dir, &tracker[5]);
+ alloctest_ref_tracker_alloc6(&ref_dir, &tracker[6]);
+ alloctest_ref_tracker_alloc7(&ref_dir, &tracker[7]);
+ alloctest_ref_tracker_alloc8(&ref_dir, &tracker[8]);
+ alloctest_ref_tracker_alloc9(&ref_dir, &tracker[9]);
+ alloctest_ref_tracker_alloc10(&ref_dir, &tracker[10]);
+ alloctest_ref_tracker_alloc11(&ref_dir, &tracker[11]);
+ alloctest_ref_tracker_alloc12(&ref_dir, &tracker[12]);
+ alloctest_ref_tracker_alloc13(&ref_dir, &tracker[13]);
+ alloctest_ref_tracker_alloc14(&ref_dir, &tracker[14]);
+ alloctest_ref_tracker_alloc15(&ref_dir, &tracker[15]);
+ alloctest_ref_tracker_alloc16(&ref_dir, &tracker[16]);
+ alloctest_ref_tracker_alloc17(&ref_dir, &tracker[17]);
+ alloctest_ref_tracker_alloc18(&ref_dir, &tracker[18]);
+ alloctest_ref_tracker_alloc19(&ref_dir, &tracker[19]);
+
+ /* free all trackers but first 0 and 1. */
+ for (i = 2; i < ARRAY_SIZE(tracker); i++)
+ alloctest_ref_tracker_free(&ref_dir, &tracker[i]);
+
+ /* Attempt to free an already freed tracker. */
+ alloctest_ref_tracker_free(&ref_dir, &tracker[2]);
+
+ while (!atomic_read(&test_ref_timer_done))
+ msleep(1);
+
+ /* This should warn about tracker[0] & tracker[1] being not freed. */
+ ref_tracker_dir_exit(&ref_dir);
+
+ return 0;
+}
+
+static void __exit test_ref_tracker_exit(void)
+{
+}
+
+module_init(test_ref_tracker_init);
+module_exit(test_ref_tracker_exit);
+
+MODULE_LICENSE("GPL v2");
diff --git a/mm/bootmem_info.c b/mm/bootmem_info.c
index f03f42f..f18a631 100644
--- a/mm/bootmem_info.c
+++ b/mm/bootmem_info.c
@@ -15,7 +15,7 @@
void get_page_bootmem(unsigned long info, struct page *page, unsigned long type)
{
- page->freelist = (void *)type;
+ page->index = type;
SetPagePrivate(page);
set_page_private(page, info);
page_ref_inc(page);
@@ -23,14 +23,13 @@ void get_page_bootmem(unsigned long info, struct page *page, unsigned long type)
void put_page_bootmem(struct page *page)
{
- unsigned long type;
+ unsigned long type = page->index;
- type = (unsigned long) page->freelist;
BUG_ON(type < MEMORY_HOTPLUG_MIN_BOOTMEM_TYPE ||
type > MEMORY_HOTPLUG_MAX_BOOTMEM_TYPE);
if (page_ref_dec_return(page) == 1) {
- page->freelist = NULL;
+ page->index = 0;
ClearPagePrivate(page);
set_page_private(page, 0);
INIT_LIST_HEAD(&page->lru);
diff --git a/mm/damon/dbgfs.c b/mm/damon/dbgfs.c
index 4fbd729..ad65436 100644
--- a/mm/damon/dbgfs.c
+++ b/mm/damon/dbgfs.c
@@ -353,6 +353,7 @@ static ssize_t dbgfs_target_ids_write(struct file *file,
const char __user *buf, size_t count, loff_t *ppos)
{
struct damon_ctx *ctx = file->private_data;
+ struct damon_target *t, *next_t;
bool id_is_pid = true;
char *kbuf, *nrs;
unsigned long *targets;
@@ -397,8 +398,12 @@ static ssize_t dbgfs_target_ids_write(struct file *file,
goto unlock_out;
}
- /* remove targets with previously-set primitive */
- damon_set_targets(ctx, NULL, 0);
+ /* remove previously set targets */
+ damon_for_each_target_safe(t, next_t, ctx) {
+ if (targetid_is_pid(ctx))
+ put_pid((struct pid *)t->id);
+ damon_destroy_target(t);
+ }
/* Configure the context for the address space type */
if (id_is_pid)
diff --git a/mm/kasan/common.c b/mm/kasan/common.c
index 8428da2..7c06db7 100644
--- a/mm/kasan/common.c
+++ b/mm/kasan/common.c
@@ -247,8 +247,9 @@ struct kasan_free_meta *kasan_get_free_meta(struct kmem_cache *cache,
}
#endif
-void __kasan_poison_slab(struct page *page)
+void __kasan_poison_slab(struct slab *slab)
{
+ struct page *page = slab_page(slab);
unsigned long i;
for (i = 0; i < compound_nr(page); i++)
@@ -298,7 +299,7 @@ static inline u8 assign_tag(struct kmem_cache *cache,
/* For caches that either have a constructor or SLAB_TYPESAFE_BY_RCU: */
#ifdef CONFIG_SLAB
/* For SLAB assign tags based on the object index in the freelist. */
- return (u8)obj_to_index(cache, virt_to_head_page(object), (void *)object);
+ return (u8)obj_to_index(cache, virt_to_slab(object), (void *)object);
#else
/*
* For SLUB assign a random tag during slab creation, otherwise reuse
@@ -341,7 +342,7 @@ static inline bool ____kasan_slab_free(struct kmem_cache *cache, void *object,
if (is_kfence_address(object))
return false;
- if (unlikely(nearest_obj(cache, virt_to_head_page(object), object) !=
+ if (unlikely(nearest_obj(cache, virt_to_slab(object), object) !=
object)) {
kasan_report_invalid_free(tagged_object, ip);
return true;
@@ -401,9 +402,9 @@ void __kasan_kfree_large(void *ptr, unsigned long ip)
void __kasan_slab_free_mempool(void *ptr, unsigned long ip)
{
- struct page *page;
+ struct folio *folio;
- page = virt_to_head_page(ptr);
+ folio = virt_to_folio(ptr);
/*
* Even though this function is only called for kmem_cache_alloc and
@@ -411,12 +412,14 @@ void __kasan_slab_free_mempool(void *ptr, unsigned long ip)
* !PageSlab() when the size provided to kmalloc is larger than
* KMALLOC_MAX_SIZE, and kmalloc falls back onto page_alloc.
*/
- if (unlikely(!PageSlab(page))) {
+ if (unlikely(!folio_test_slab(folio))) {
if (____kasan_kfree_large(ptr, ip))
return;
- kasan_poison(ptr, page_size(page), KASAN_FREE_PAGE, false);
+ kasan_poison(ptr, folio_size(folio), KASAN_FREE_PAGE, false);
} else {
- ____kasan_slab_free(page->slab_cache, ptr, ip, false, false);
+ struct slab *slab = folio_slab(folio);
+
+ ____kasan_slab_free(slab->slab_cache, ptr, ip, false, false);
}
}
@@ -560,7 +563,7 @@ void * __must_check __kasan_kmalloc_large(const void *ptr, size_t size,
void * __must_check __kasan_krealloc(const void *object, size_t size, gfp_t flags)
{
- struct page *page;
+ struct slab *slab;
if (unlikely(object == ZERO_SIZE_PTR))
return (void *)object;
@@ -572,13 +575,13 @@ void * __must_check __kasan_krealloc(const void *object, size_t size, gfp_t flag
*/
kasan_unpoison(object, size, false);
- page = virt_to_head_page(object);
+ slab = virt_to_slab(object);
/* Piggy-back on kmalloc() instrumentation to poison the redzone. */
- if (unlikely(!PageSlab(page)))
+ if (unlikely(!slab))
return __kasan_kmalloc_large(object, size, flags);
else
- return ____kasan_kmalloc(page->slab_cache, object, size, flags);
+ return ____kasan_kmalloc(slab->slab_cache, object, size, flags);
}
bool __kasan_check_byte(const void *address, unsigned long ip)
diff --git a/mm/kasan/generic.c b/mm/kasan/generic.c
index 84a038b..a25ad40 100644
--- a/mm/kasan/generic.c
+++ b/mm/kasan/generic.c
@@ -330,16 +330,16 @@ DEFINE_ASAN_SET_SHADOW(f8);
static void __kasan_record_aux_stack(void *addr, bool can_alloc)
{
- struct page *page = kasan_addr_to_page(addr);
+ struct slab *slab = kasan_addr_to_slab(addr);
struct kmem_cache *cache;
struct kasan_alloc_meta *alloc_meta;
void *object;
- if (is_kfence_address(addr) || !(page && PageSlab(page)))
+ if (is_kfence_address(addr) || !slab)
return;
- cache = page->slab_cache;
- object = nearest_obj(cache, page, addr);
+ cache = slab->slab_cache;
+ object = nearest_obj(cache, slab, addr);
alloc_meta = kasan_get_alloc_meta(cache, object);
if (!alloc_meta)
return;
diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h
index aebd8df..c17fa8d 100644
--- a/mm/kasan/kasan.h
+++ b/mm/kasan/kasan.h
@@ -265,6 +265,7 @@ bool kasan_report(unsigned long addr, size_t size,
void kasan_report_invalid_free(void *object, unsigned long ip);
struct page *kasan_addr_to_page(const void *addr);
+struct slab *kasan_addr_to_slab(const void *addr);
depot_stack_handle_t kasan_save_stack(gfp_t flags, bool can_alloc);
void kasan_set_track(struct kasan_track *track, gfp_t flags);
diff --git a/mm/kasan/quarantine.c b/mm/kasan/quarantine.c
index d8ccff4..587da89 100644
--- a/mm/kasan/quarantine.c
+++ b/mm/kasan/quarantine.c
@@ -117,7 +117,7 @@ static unsigned long quarantine_batch_size;
static struct kmem_cache *qlink_to_cache(struct qlist_node *qlink)
{
- return virt_to_head_page(qlink)->slab_cache;
+ return virt_to_slab(qlink)->slab_cache;
}
static void *qlink_to_object(struct qlist_node *qlink, struct kmem_cache *cache)
diff --git a/mm/kasan/report.c b/mm/kasan/report.c
index 0bc10f4..3ad9624 100644
--- a/mm/kasan/report.c
+++ b/mm/kasan/report.c
@@ -150,6 +150,14 @@ struct page *kasan_addr_to_page(const void *addr)
return NULL;
}
+struct slab *kasan_addr_to_slab(const void *addr)
+{
+ if ((addr >= (void *)PAGE_OFFSET) &&
+ (addr < high_memory))
+ return virt_to_slab(addr);
+ return NULL;
+}
+
static void describe_object_addr(struct kmem_cache *cache, void *object,
const void *addr)
{
@@ -248,8 +256,9 @@ static void print_address_description(void *addr, u8 tag)
pr_err("\n");
if (page && PageSlab(page)) {
- struct kmem_cache *cache = page->slab_cache;
- void *object = nearest_obj(cache, page, addr);
+ struct slab *slab = page_slab(page);
+ struct kmem_cache *cache = slab->slab_cache;
+ void *object = nearest_obj(cache, slab, addr);
describe_object(cache, object, addr, tag);
}
diff --git a/mm/kasan/report_tags.c b/mm/kasan/report_tags.c
index 8a319fc..1b41de8 100644
--- a/mm/kasan/report_tags.c
+++ b/mm/kasan/report_tags.c
@@ -12,7 +12,7 @@ const char *kasan_get_bug_type(struct kasan_access_info *info)
#ifdef CONFIG_KASAN_TAGS_IDENTIFY
struct kasan_alloc_meta *alloc_meta;
struct kmem_cache *cache;
- struct page *page;
+ struct slab *slab;
const void *addr;
void *object;
u8 tag;
@@ -20,10 +20,10 @@ const char *kasan_get_bug_type(struct kasan_access_info *info)
tag = get_tag(info->access_addr);
addr = kasan_reset_tag(info->access_addr);
- page = kasan_addr_to_page(addr);
- if (page && PageSlab(page)) {
- cache = page->slab_cache;
- object = nearest_obj(cache, page, (void *)addr);
+ slab = kasan_addr_to_slab(addr);
+ if (slab) {
+ cache = slab->slab_cache;
+ object = nearest_obj(cache, slab, (void *)addr);
alloc_meta = kasan_get_alloc_meta(cache, object);
if (alloc_meta) {
diff --git a/mm/kfence/core.c b/mm/kfence/core.c
index a19154a..5ad40e3 100644
--- a/mm/kfence/core.c
+++ b/mm/kfence/core.c
@@ -360,7 +360,7 @@ static void *kfence_guarded_alloc(struct kmem_cache *cache, size_t size, gfp_t g
{
struct kfence_metadata *meta = NULL;
unsigned long flags;
- struct page *page;
+ struct slab *slab;
void *addr;
/* Try to obtain a free object. */
@@ -424,13 +424,14 @@ static void *kfence_guarded_alloc(struct kmem_cache *cache, size_t size, gfp_t g
alloc_covered_add(alloc_stack_hash, 1);
- /* Set required struct page fields. */
- page = virt_to_page(meta->addr);
- page->slab_cache = cache;
- if (IS_ENABLED(CONFIG_SLUB))
- page->objects = 1;
- if (IS_ENABLED(CONFIG_SLAB))
- page->s_mem = addr;
+ /* Set required slab fields. */
+ slab = virt_to_slab((void *)meta->addr);
+ slab->slab_cache = cache;
+#if defined(CONFIG_SLUB)
+ slab->objects = 1;
+#elif defined(CONFIG_SLAB)
+ slab->s_mem = addr;
+#endif
/* Memory initialization. */
for_each_canary(meta, set_canary_byte);
diff --git a/mm/kfence/kfence_test.c b/mm/kfence/kfence_test.c
index 695030c..a22b1af 100644
--- a/mm/kfence/kfence_test.c
+++ b/mm/kfence/kfence_test.c
@@ -282,7 +282,7 @@ static void *test_alloc(struct kunit *test, size_t size, gfp_t gfp, enum allocat
alloc = kmalloc(size, gfp);
if (is_kfence_address(alloc)) {
- struct page *page = virt_to_head_page(alloc);
+ struct slab *slab = virt_to_slab(alloc);
struct kmem_cache *s = test_cache ?:
kmalloc_caches[kmalloc_type(GFP_KERNEL)][__kmalloc_index(size, false)];
@@ -291,8 +291,8 @@ static void *test_alloc(struct kunit *test, size_t size, gfp_t gfp, enum allocat
* even for KFENCE objects; these are required so that
* memcg accounting works correctly.
*/
- KUNIT_EXPECT_EQ(test, obj_to_index(s, page, alloc), 0U);
- KUNIT_EXPECT_EQ(test, objs_per_slab_page(s, page), 1);
+ KUNIT_EXPECT_EQ(test, obj_to_index(s, slab, alloc), 0U);
+ KUNIT_EXPECT_EQ(test, objs_per_slab(s, slab), 1);
if (policy == ALLOCATE_ANY)
return alloc;
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 2ed5f2a..4a7b3eb 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -2816,31 +2816,31 @@ static inline void mod_objcg_mlstate(struct obj_cgroup *objcg,
rcu_read_unlock();
}
-int memcg_alloc_page_obj_cgroups(struct page *page, struct kmem_cache *s,
- gfp_t gfp, bool new_page)
+int memcg_alloc_slab_cgroups(struct slab *slab, struct kmem_cache *s,
+ gfp_t gfp, bool new_slab)
{
- unsigned int objects = objs_per_slab_page(s, page);
+ unsigned int objects = objs_per_slab(s, slab);
unsigned long memcg_data;
void *vec;
gfp &= ~OBJCGS_CLEAR_MASK;
vec = kcalloc_node(objects, sizeof(struct obj_cgroup *), gfp,
- page_to_nid(page));
+ slab_nid(slab));
if (!vec)
return -ENOMEM;
memcg_data = (unsigned long) vec | MEMCG_DATA_OBJCGS;
- if (new_page) {
+ if (new_slab) {
/*
- * If the slab page is brand new and nobody can yet access
- * it's memcg_data, no synchronization is required and
- * memcg_data can be simply assigned.
+ * If the slab is brand new and nobody can yet access its
+ * memcg_data, no synchronization is required and memcg_data can
+ * be simply assigned.
*/
- page->memcg_data = memcg_data;
- } else if (cmpxchg(&page->memcg_data, 0, memcg_data)) {
+ slab->memcg_data = memcg_data;
+ } else if (cmpxchg(&slab->memcg_data, 0, memcg_data)) {
/*
- * If the slab page is already in use, somebody can allocate
- * and assign obj_cgroups in parallel. In this case the existing
+ * If the slab is already in use, somebody can allocate and
+ * assign obj_cgroups in parallel. In this case the existing
* objcg vector should be reused.
*/
kfree(vec);
@@ -2865,38 +2865,43 @@ int memcg_alloc_page_obj_cgroups(struct page *page, struct kmem_cache *s,
*/
struct mem_cgroup *mem_cgroup_from_obj(void *p)
{
- struct page *page;
+ struct folio *folio;
if (mem_cgroup_disabled())
return NULL;
- page = virt_to_head_page(p);
+ folio = virt_to_folio(p);
/*
* Slab objects are accounted individually, not per-page.
* Memcg membership data for each individual object is saved in
- * the page->obj_cgroups.
+ * slab->memcg_data.
*/
- if (page_objcgs_check(page)) {
- struct obj_cgroup *objcg;
+ if (folio_test_slab(folio)) {
+ struct obj_cgroup **objcgs;
+ struct slab *slab;
unsigned int off;
- off = obj_to_index(page->slab_cache, page, p);
- objcg = page_objcgs(page)[off];
- if (objcg)
- return obj_cgroup_memcg(objcg);
+ slab = folio_slab(folio);
+ objcgs = slab_objcgs(slab);
+ if (!objcgs)
+ return NULL;
+
+ off = obj_to_index(slab->slab_cache, slab, p);
+ if (objcgs[off])
+ return obj_cgroup_memcg(objcgs[off]);
return NULL;
}
/*
- * page_memcg_check() is used here, because page_has_obj_cgroups()
- * check above could fail because the object cgroups vector wasn't set
- * at that moment, but it can be set concurrently.
+ * page_memcg_check() is used here, because in theory we can encounter
+ * a folio where the slab flag has been cleared already, but
+ * slab->memcg_data has not been freed yet
* page_memcg_check(page) will guarantee that a proper memory
* cgroup pointer or NULL will be returned.
*/
- return page_memcg_check(page);
+ return page_memcg_check(folio_page(folio, 0));
}
__always_inline struct obj_cgroup *get_obj_cgroup_from_current(void)
diff --git a/mm/memory-failure.c b/mm/memory-failure.c
index 3a27446..f1c389f 100644
--- a/mm/memory-failure.c
+++ b/mm/memory-failure.c
@@ -1646,21 +1646,28 @@ int memory_failure(unsigned long pfn, int flags)
if (!sysctl_memory_failure_recovery)
panic("Memory failure on page %lx", pfn);
+ mutex_lock(&mf_mutex);
+
p = pfn_to_online_page(pfn);
if (!p) {
+ res = arch_memory_failure(pfn, flags);
+ if (res == 0)
+ goto unlock_mutex;
+
if (pfn_valid(pfn)) {
pgmap = get_dev_pagemap(pfn, NULL);
- if (pgmap)
- return memory_failure_dev_pagemap(pfn, flags,
- pgmap);
+ if (pgmap) {
+ res = memory_failure_dev_pagemap(pfn, flags,
+ pgmap);
+ goto unlock_mutex;
+ }
}
pr_err("Memory failure: %#lx: memory outside kernel control\n",
pfn);
- return -ENXIO;
+ res = -ENXIO;
+ goto unlock_mutex;
}
- mutex_lock(&mf_mutex);
-
try_again:
if (PageHuge(p)) {
res = memory_failure_hugetlb(pfn, flags);
diff --git a/mm/slab.c b/mm/slab.c
index ca4822f..ddf5737 100644
--- a/mm/slab.c
+++ b/mm/slab.c
@@ -218,7 +218,7 @@ static void cache_reap(struct work_struct *unused);
static inline void fixup_objfreelist_debug(struct kmem_cache *cachep,
void **list);
static inline void fixup_slab_list(struct kmem_cache *cachep,
- struct kmem_cache_node *n, struct page *page,
+ struct kmem_cache_node *n, struct slab *slab,
void **list);
static int slab_early_init = 1;
@@ -372,10 +372,10 @@ static void **dbg_userword(struct kmem_cache *cachep, void *objp)
static int slab_max_order = SLAB_MAX_ORDER_LO;
static bool slab_max_order_set __initdata;
-static inline void *index_to_obj(struct kmem_cache *cache, struct page *page,
- unsigned int idx)
+static inline void *index_to_obj(struct kmem_cache *cache,
+ const struct slab *slab, unsigned int idx)
{
- return page->s_mem + cache->size * idx;
+ return slab->s_mem + cache->size * idx;
}
#define BOOT_CPUCACHE_ENTRIES 1
@@ -550,17 +550,17 @@ static struct array_cache *alloc_arraycache(int node, int entries,
}
static noinline void cache_free_pfmemalloc(struct kmem_cache *cachep,
- struct page *page, void *objp)
+ struct slab *slab, void *objp)
{
struct kmem_cache_node *n;
- int page_node;
+ int slab_node;
LIST_HEAD(list);
- page_node = page_to_nid(page);
- n = get_node(cachep, page_node);
+ slab_node = slab_nid(slab);
+ n = get_node(cachep, slab_node);
spin_lock(&n->list_lock);
- free_block(cachep, &objp, 1, page_node, &list);
+ free_block(cachep, &objp, 1, slab_node, &list);
spin_unlock(&n->list_lock);
slabs_destroy(cachep, &list);
@@ -761,7 +761,7 @@ static void drain_alien_cache(struct kmem_cache *cachep,
}
static int __cache_free_alien(struct kmem_cache *cachep, void *objp,
- int node, int page_node)
+ int node, int slab_node)
{
struct kmem_cache_node *n;
struct alien_cache *alien = NULL;
@@ -770,21 +770,21 @@ static int __cache_free_alien(struct kmem_cache *cachep, void *objp,
n = get_node(cachep, node);
STATS_INC_NODEFREES(cachep);
- if (n->alien && n->alien[page_node]) {
- alien = n->alien[page_node];
+ if (n->alien && n->alien[slab_node]) {
+ alien = n->alien[slab_node];
ac = &alien->ac;
spin_lock(&alien->lock);
if (unlikely(ac->avail == ac->limit)) {
STATS_INC_ACOVERFLOW(cachep);
- __drain_alien_cache(cachep, ac, page_node, &list);
+ __drain_alien_cache(cachep, ac, slab_node, &list);
}
__free_one(ac, objp);
spin_unlock(&alien->lock);
slabs_destroy(cachep, &list);
} else {
- n = get_node(cachep, page_node);
+ n = get_node(cachep, slab_node);
spin_lock(&n->list_lock);
- free_block(cachep, &objp, 1, page_node, &list);
+ free_block(cachep, &objp, 1, slab_node, &list);
spin_unlock(&n->list_lock);
slabs_destroy(cachep, &list);
}
@@ -793,16 +793,16 @@ static int __cache_free_alien(struct kmem_cache *cachep, void *objp,
static inline int cache_free_alien(struct kmem_cache *cachep, void *objp)
{
- int page_node = page_to_nid(virt_to_page(objp));
+ int slab_node = slab_nid(virt_to_slab(objp));
int node = numa_mem_id();
/*
* Make sure we are not freeing a object from another node to the array
* cache on this cpu.
*/
- if (likely(node == page_node))
+ if (likely(node == slab_node))
return 0;
- return __cache_free_alien(cachep, objp, node, page_node);
+ return __cache_free_alien(cachep, objp, node, slab_node);
}
/*
@@ -1367,57 +1367,60 @@ slab_out_of_memory(struct kmem_cache *cachep, gfp_t gfpflags, int nodeid)
* did not request dmaable memory, we might get it, but that
* would be relatively rare and ignorable.
*/
-static struct page *kmem_getpages(struct kmem_cache *cachep, gfp_t flags,
+static struct slab *kmem_getpages(struct kmem_cache *cachep, gfp_t flags,
int nodeid)
{
- struct page *page;
+ struct folio *folio;
+ struct slab *slab;
flags |= cachep->allocflags;
- page = __alloc_pages_node(nodeid, flags, cachep->gfporder);
- if (!page) {
+ folio = (struct folio *) __alloc_pages_node(nodeid, flags, cachep->gfporder);
+ if (!folio) {
slab_out_of_memory(cachep, flags, nodeid);
return NULL;
}
- account_slab_page(page, cachep->gfporder, cachep, flags);
- __SetPageSlab(page);
- /* Record if ALLOC_NO_WATERMARKS was set when allocating the slab */
- if (sk_memalloc_socks() && page_is_pfmemalloc(page))
- SetPageSlabPfmemalloc(page);
+ slab = folio_slab(folio);
- return page;
+ account_slab(slab, cachep->gfporder, cachep, flags);
+ __folio_set_slab(folio);
+ /* Record if ALLOC_NO_WATERMARKS was set when allocating the slab */
+ if (sk_memalloc_socks() && page_is_pfmemalloc(folio_page(folio, 0)))
+ slab_set_pfmemalloc(slab);
+
+ return slab;
}
/*
* Interface to system's page release.
*/
-static void kmem_freepages(struct kmem_cache *cachep, struct page *page)
+static void kmem_freepages(struct kmem_cache *cachep, struct slab *slab)
{
int order = cachep->gfporder;
+ struct folio *folio = slab_folio(slab);
- BUG_ON(!PageSlab(page));
- __ClearPageSlabPfmemalloc(page);
- __ClearPageSlab(page);
- page_mapcount_reset(page);
- /* In union with page->mapping where page allocator expects NULL */
- page->slab_cache = NULL;
+ BUG_ON(!folio_test_slab(folio));
+ __slab_clear_pfmemalloc(slab);
+ __folio_clear_slab(folio);
+ page_mapcount_reset(folio_page(folio, 0));
+ folio->mapping = NULL;
if (current->reclaim_state)
current->reclaim_state->reclaimed_slab += 1 << order;
- unaccount_slab_page(page, order, cachep);
- __free_pages(page, order);
+ unaccount_slab(slab, order, cachep);
+ __free_pages(folio_page(folio, 0), order);
}
static void kmem_rcu_free(struct rcu_head *head)
{
struct kmem_cache *cachep;
- struct page *page;
+ struct slab *slab;
- page = container_of(head, struct page, rcu_head);
- cachep = page->slab_cache;
+ slab = container_of(head, struct slab, rcu_head);
+ cachep = slab->slab_cache;
- kmem_freepages(cachep, page);
+ kmem_freepages(cachep, slab);
}
#if DEBUG
@@ -1553,18 +1556,18 @@ static void check_poison_obj(struct kmem_cache *cachep, void *objp)
/* Print some data about the neighboring objects, if they
* exist:
*/
- struct page *page = virt_to_head_page(objp);
+ struct slab *slab = virt_to_slab(objp);
unsigned int objnr;
- objnr = obj_to_index(cachep, page, objp);
+ objnr = obj_to_index(cachep, slab, objp);
if (objnr) {
- objp = index_to_obj(cachep, page, objnr - 1);
+ objp = index_to_obj(cachep, slab, objnr - 1);
realobj = (char *)objp + obj_offset(cachep);
pr_err("Prev obj: start=%px, len=%d\n", realobj, size);
print_objinfo(cachep, objp, 2);
}
if (objnr + 1 < cachep->num) {
- objp = index_to_obj(cachep, page, objnr + 1);
+ objp = index_to_obj(cachep, slab, objnr + 1);
realobj = (char *)objp + obj_offset(cachep);
pr_err("Next obj: start=%px, len=%d\n", realobj, size);
print_objinfo(cachep, objp, 2);
@@ -1575,17 +1578,17 @@ static void check_poison_obj(struct kmem_cache *cachep, void *objp)
#if DEBUG
static void slab_destroy_debugcheck(struct kmem_cache *cachep,
- struct page *page)
+ struct slab *slab)
{
int i;
if (OBJFREELIST_SLAB(cachep) && cachep->flags & SLAB_POISON) {
- poison_obj(cachep, page->freelist - obj_offset(cachep),
+ poison_obj(cachep, slab->freelist - obj_offset(cachep),
POISON_FREE);
}
for (i = 0; i < cachep->num; i++) {
- void *objp = index_to_obj(cachep, page, i);
+ void *objp = index_to_obj(cachep, slab, i);
if (cachep->flags & SLAB_POISON) {
check_poison_obj(cachep, objp);
@@ -1601,7 +1604,7 @@ static void slab_destroy_debugcheck(struct kmem_cache *cachep,
}
#else
static void slab_destroy_debugcheck(struct kmem_cache *cachep,
- struct page *page)
+ struct slab *slab)
{
}
#endif
@@ -1609,22 +1612,22 @@ static void slab_destroy_debugcheck(struct kmem_cache *cachep,
/**
* slab_destroy - destroy and release all objects in a slab
* @cachep: cache pointer being destroyed
- * @page: page pointer being destroyed
+ * @slab: slab being destroyed
*
- * Destroy all the objs in a slab page, and release the mem back to the system.
- * Before calling the slab page must have been unlinked from the cache. The
+ * Destroy all the objs in a slab, and release the mem back to the system.
+ * Before calling the slab must have been unlinked from the cache. The
* kmem_cache_node ->list_lock is not held/needed.
*/
-static void slab_destroy(struct kmem_cache *cachep, struct page *page)
+static void slab_destroy(struct kmem_cache *cachep, struct slab *slab)
{
void *freelist;
- freelist = page->freelist;
- slab_destroy_debugcheck(cachep, page);
+ freelist = slab->freelist;
+ slab_destroy_debugcheck(cachep, slab);
if (unlikely(cachep->flags & SLAB_TYPESAFE_BY_RCU))
- call_rcu(&page->rcu_head, kmem_rcu_free);
+ call_rcu(&slab->rcu_head, kmem_rcu_free);
else
- kmem_freepages(cachep, page);
+ kmem_freepages(cachep, slab);
/*
* From now on, we don't use freelist
@@ -1640,11 +1643,11 @@ static void slab_destroy(struct kmem_cache *cachep, struct page *page)
*/
static void slabs_destroy(struct kmem_cache *cachep, struct list_head *list)
{
- struct page *page, *n;
+ struct slab *slab, *n;
- list_for_each_entry_safe(page, n, list, slab_list) {
- list_del(&page->slab_list);
- slab_destroy(cachep, page);
+ list_for_each_entry_safe(slab, n, list, slab_list) {
+ list_del(&slab->slab_list);
+ slab_destroy(cachep, slab);
}
}
@@ -2194,7 +2197,7 @@ static int drain_freelist(struct kmem_cache *cache,
{
struct list_head *p;
int nr_freed;
- struct page *page;
+ struct slab *slab;
nr_freed = 0;
while (nr_freed < tofree && !list_empty(&n->slabs_free)) {
@@ -2206,8 +2209,8 @@ static int drain_freelist(struct kmem_cache *cache,
goto out;
}
- page = list_entry(p, struct page, slab_list);
- list_del(&page->slab_list);
+ slab = list_entry(p, struct slab, slab_list);
+ list_del(&slab->slab_list);
n->free_slabs--;
n->total_slabs--;
/*
@@ -2216,7 +2219,7 @@ static int drain_freelist(struct kmem_cache *cache,
*/
n->free_objects -= cache->num;
spin_unlock_irq(&n->list_lock);
- slab_destroy(cache, page);
+ slab_destroy(cache, slab);
nr_freed++;
}
out:
@@ -2291,14 +2294,14 @@ void __kmem_cache_release(struct kmem_cache *cachep)
* which are all initialized during kmem_cache_init().
*/
static void *alloc_slabmgmt(struct kmem_cache *cachep,
- struct page *page, int colour_off,
+ struct slab *slab, int colour_off,
gfp_t local_flags, int nodeid)
{
void *freelist;
- void *addr = page_address(page);
+ void *addr = slab_address(slab);
- page->s_mem = addr + colour_off;
- page->active = 0;
+ slab->s_mem = addr + colour_off;
+ slab->active = 0;
if (OBJFREELIST_SLAB(cachep))
freelist = NULL;
@@ -2315,24 +2318,24 @@ static void *alloc_slabmgmt(struct kmem_cache *cachep,
return freelist;
}
-static inline freelist_idx_t get_free_obj(struct page *page, unsigned int idx)
+static inline freelist_idx_t get_free_obj(struct slab *slab, unsigned int idx)
{
- return ((freelist_idx_t *)page->freelist)[idx];
+ return ((freelist_idx_t *) slab->freelist)[idx];
}
-static inline void set_free_obj(struct page *page,
+static inline void set_free_obj(struct slab *slab,
unsigned int idx, freelist_idx_t val)
{
- ((freelist_idx_t *)(page->freelist))[idx] = val;
+ ((freelist_idx_t *)(slab->freelist))[idx] = val;
}
-static void cache_init_objs_debug(struct kmem_cache *cachep, struct page *page)
+static void cache_init_objs_debug(struct kmem_cache *cachep, struct slab *slab)
{
#if DEBUG
int i;
for (i = 0; i < cachep->num; i++) {
- void *objp = index_to_obj(cachep, page, i);
+ void *objp = index_to_obj(cachep, slab, i);
if (cachep->flags & SLAB_STORE_USER)
*dbg_userword(cachep, objp) = NULL;
@@ -2416,17 +2419,17 @@ static freelist_idx_t next_random_slot(union freelist_init_state *state)
}
/* Swap two freelist entries */
-static void swap_free_obj(struct page *page, unsigned int a, unsigned int b)
+static void swap_free_obj(struct slab *slab, unsigned int a, unsigned int b)
{
- swap(((freelist_idx_t *)page->freelist)[a],
- ((freelist_idx_t *)page->freelist)[b]);
+ swap(((freelist_idx_t *) slab->freelist)[a],
+ ((freelist_idx_t *) slab->freelist)[b]);
}
/*
* Shuffle the freelist initialization state based on pre-computed lists.
* return true if the list was successfully shuffled, false otherwise.
*/
-static bool shuffle_freelist(struct kmem_cache *cachep, struct page *page)
+static bool shuffle_freelist(struct kmem_cache *cachep, struct slab *slab)
{
unsigned int objfreelist = 0, i, rand, count = cachep->num;
union freelist_init_state state;
@@ -2443,7 +2446,7 @@ static bool shuffle_freelist(struct kmem_cache *cachep, struct page *page)
objfreelist = count - 1;
else
objfreelist = next_random_slot(&state);
- page->freelist = index_to_obj(cachep, page, objfreelist) +
+ slab->freelist = index_to_obj(cachep, slab, objfreelist) +
obj_offset(cachep);
count--;
}
@@ -2454,51 +2457,51 @@ static bool shuffle_freelist(struct kmem_cache *cachep, struct page *page)
*/
if (!precomputed) {
for (i = 0; i < count; i++)
- set_free_obj(page, i, i);
+ set_free_obj(slab, i, i);
/* Fisher-Yates shuffle */
for (i = count - 1; i > 0; i--) {
rand = prandom_u32_state(&state.rnd_state);
rand %= (i + 1);
- swap_free_obj(page, i, rand);
+ swap_free_obj(slab, i, rand);
}
} else {
for (i = 0; i < count; i++)
- set_free_obj(page, i, next_random_slot(&state));
+ set_free_obj(slab, i, next_random_slot(&state));
}
if (OBJFREELIST_SLAB(cachep))
- set_free_obj(page, cachep->num - 1, objfreelist);
+ set_free_obj(slab, cachep->num - 1, objfreelist);
return true;
}
#else
static inline bool shuffle_freelist(struct kmem_cache *cachep,
- struct page *page)
+ struct slab *slab)
{
return false;
}
#endif /* CONFIG_SLAB_FREELIST_RANDOM */
static void cache_init_objs(struct kmem_cache *cachep,
- struct page *page)
+ struct slab *slab)
{
int i;
void *objp;
bool shuffled;
- cache_init_objs_debug(cachep, page);
+ cache_init_objs_debug(cachep, slab);
/* Try to randomize the freelist if enabled */
- shuffled = shuffle_freelist(cachep, page);
+ shuffled = shuffle_freelist(cachep, slab);
if (!shuffled && OBJFREELIST_SLAB(cachep)) {
- page->freelist = index_to_obj(cachep, page, cachep->num - 1) +
+ slab->freelist = index_to_obj(cachep, slab, cachep->num - 1) +
obj_offset(cachep);
}
for (i = 0; i < cachep->num; i++) {
- objp = index_to_obj(cachep, page, i);
+ objp = index_to_obj(cachep, slab, i);
objp = kasan_init_slab_obj(cachep, objp);
/* constructor could break poison info */
@@ -2509,68 +2512,56 @@ static void cache_init_objs(struct kmem_cache *cachep,
}
if (!shuffled)
- set_free_obj(page, i, i);
+ set_free_obj(slab, i, i);
}
}
-static void *slab_get_obj(struct kmem_cache *cachep, struct page *page)
+static void *slab_get_obj(struct kmem_cache *cachep, struct slab *slab)
{
void *objp;
- objp = index_to_obj(cachep, page, get_free_obj(page, page->active));
- page->active++;
+ objp = index_to_obj(cachep, slab, get_free_obj(slab, slab->active));
+ slab->active++;
return objp;
}
static void slab_put_obj(struct kmem_cache *cachep,
- struct page *page, void *objp)
+ struct slab *slab, void *objp)
{
- unsigned int objnr = obj_to_index(cachep, page, objp);
+ unsigned int objnr = obj_to_index(cachep, slab, objp);
#if DEBUG
unsigned int i;
/* Verify double free bug */
- for (i = page->active; i < cachep->num; i++) {
- if (get_free_obj(page, i) == objnr) {
+ for (i = slab->active; i < cachep->num; i++) {
+ if (get_free_obj(slab, i) == objnr) {
pr_err("slab: double free detected in cache '%s', objp %px\n",
cachep->name, objp);
BUG();
}
}
#endif
- page->active--;
- if (!page->freelist)
- page->freelist = objp + obj_offset(cachep);
+ slab->active--;
+ if (!slab->freelist)
+ slab->freelist = objp + obj_offset(cachep);
- set_free_obj(page, page->active, objnr);
-}
-
-/*
- * Map pages beginning at addr to the given cache and slab. This is required
- * for the slab allocator to be able to lookup the cache and slab of a
- * virtual address for kfree, ksize, and slab debugging.
- */
-static void slab_map_pages(struct kmem_cache *cache, struct page *page,
- void *freelist)
-{
- page->slab_cache = cache;
- page->freelist = freelist;
+ set_free_obj(slab, slab->active, objnr);
}
/*
* Grow (by 1) the number of slabs within a cache. This is called by
* kmem_cache_alloc() when there are no active objs left in a cache.
*/
-static struct page *cache_grow_begin(struct kmem_cache *cachep,
+static struct slab *cache_grow_begin(struct kmem_cache *cachep,
gfp_t flags, int nodeid)
{
void *freelist;
size_t offset;
gfp_t local_flags;
- int page_node;
+ int slab_node;
struct kmem_cache_node *n;
- struct page *page;
+ struct slab *slab;
/*
* Be lazy and only check for valid flags here, keeping it out of the
@@ -2590,12 +2581,12 @@ static struct page *cache_grow_begin(struct kmem_cache *cachep,
* Get mem for the objs. Attempt to allocate a physical page from
* 'nodeid'.
*/
- page = kmem_getpages(cachep, local_flags, nodeid);
- if (!page)
+ slab = kmem_getpages(cachep, local_flags, nodeid);
+ if (!slab)
goto failed;
- page_node = page_to_nid(page);
- n = get_node(cachep, page_node);
+ slab_node = slab_nid(slab);
+ n = get_node(cachep, slab_node);
/* Get colour for the slab, and cal the next value. */
n->colour_next++;
@@ -2613,54 +2604,55 @@ static struct page *cache_grow_begin(struct kmem_cache *cachep,
* page_address() in the latter returns a non-tagged pointer,
* as it should be for slab pages.
*/
- kasan_poison_slab(page);
+ kasan_poison_slab(slab);
/* Get slab management. */
- freelist = alloc_slabmgmt(cachep, page, offset,
- local_flags & ~GFP_CONSTRAINT_MASK, page_node);
+ freelist = alloc_slabmgmt(cachep, slab, offset,
+ local_flags & ~GFP_CONSTRAINT_MASK, slab_node);
if (OFF_SLAB(cachep) && !freelist)
goto opps1;
- slab_map_pages(cachep, page, freelist);
+ slab->slab_cache = cachep;
+ slab->freelist = freelist;
- cache_init_objs(cachep, page);
+ cache_init_objs(cachep, slab);
if (gfpflags_allow_blocking(local_flags))
local_irq_disable();
- return page;
+ return slab;
opps1:
- kmem_freepages(cachep, page);
+ kmem_freepages(cachep, slab);
failed:
if (gfpflags_allow_blocking(local_flags))
local_irq_disable();
return NULL;
}
-static void cache_grow_end(struct kmem_cache *cachep, struct page *page)
+static void cache_grow_end(struct kmem_cache *cachep, struct slab *slab)
{
struct kmem_cache_node *n;
void *list = NULL;
check_irq_off();
- if (!page)
+ if (!slab)
return;
- INIT_LIST_HEAD(&page->slab_list);
- n = get_node(cachep, page_to_nid(page));
+ INIT_LIST_HEAD(&slab->slab_list);
+ n = get_node(cachep, slab_nid(slab));
spin_lock(&n->list_lock);
n->total_slabs++;
- if (!page->active) {
- list_add_tail(&page->slab_list, &n->slabs_free);
+ if (!slab->active) {
+ list_add_tail(&slab->slab_list, &n->slabs_free);
n->free_slabs++;
} else
- fixup_slab_list(cachep, n, page, &list);
+ fixup_slab_list(cachep, n, slab, &list);
STATS_INC_GROWN(cachep);
- n->free_objects += cachep->num - page->active;
+ n->free_objects += cachep->num - slab->active;
spin_unlock(&n->list_lock);
fixup_objfreelist_debug(cachep, &list);
@@ -2708,13 +2700,13 @@ static void *cache_free_debugcheck(struct kmem_cache *cachep, void *objp,
unsigned long caller)
{
unsigned int objnr;
- struct page *page;
+ struct slab *slab;
BUG_ON(virt_to_cache(objp) != cachep);
objp -= obj_offset(cachep);
kfree_debugcheck(objp);
- page = virt_to_head_page(objp);
+ slab = virt_to_slab(objp);
if (cachep->flags & SLAB_RED_ZONE) {
verify_redzone_free(cachep, objp);
@@ -2724,10 +2716,10 @@ static void *cache_free_debugcheck(struct kmem_cache *cachep, void *objp,
if (cachep->flags & SLAB_STORE_USER)
*dbg_userword(cachep, objp) = (void *)caller;
- objnr = obj_to_index(cachep, page, objp);
+ objnr = obj_to_index(cachep, slab, objp);
BUG_ON(objnr >= cachep->num);
- BUG_ON(objp != index_to_obj(cachep, page, objnr));
+ BUG_ON(objp != index_to_obj(cachep, slab, objnr));
if (cachep->flags & SLAB_POISON) {
poison_obj(cachep, objp, POISON_FREE);
@@ -2757,97 +2749,97 @@ static inline void fixup_objfreelist_debug(struct kmem_cache *cachep,
}
static inline void fixup_slab_list(struct kmem_cache *cachep,
- struct kmem_cache_node *n, struct page *page,
+ struct kmem_cache_node *n, struct slab *slab,
void **list)
{
/* move slabp to correct slabp list: */
- list_del(&page->slab_list);
- if (page->active == cachep->num) {
- list_add(&page->slab_list, &n->slabs_full);
+ list_del(&slab->slab_list);
+ if (slab->active == cachep->num) {
+ list_add(&slab->slab_list, &n->slabs_full);
if (OBJFREELIST_SLAB(cachep)) {
#if DEBUG
/* Poisoning will be done without holding the lock */
if (cachep->flags & SLAB_POISON) {
- void **objp = page->freelist;
+ void **objp = slab->freelist;
*objp = *list;
*list = objp;
}
#endif
- page->freelist = NULL;
+ slab->freelist = NULL;
}
} else
- list_add(&page->slab_list, &n->slabs_partial);
+ list_add(&slab->slab_list, &n->slabs_partial);
}
/* Try to find non-pfmemalloc slab if needed */
-static noinline struct page *get_valid_first_slab(struct kmem_cache_node *n,
- struct page *page, bool pfmemalloc)
+static noinline struct slab *get_valid_first_slab(struct kmem_cache_node *n,
+ struct slab *slab, bool pfmemalloc)
{
- if (!page)
+ if (!slab)
return NULL;
if (pfmemalloc)
- return page;
+ return slab;
- if (!PageSlabPfmemalloc(page))
- return page;
+ if (!slab_test_pfmemalloc(slab))
+ return slab;
/* No need to keep pfmemalloc slab if we have enough free objects */
if (n->free_objects > n->free_limit) {
- ClearPageSlabPfmemalloc(page);
- return page;
+ slab_clear_pfmemalloc(slab);
+ return slab;
}
/* Move pfmemalloc slab to the end of list to speed up next search */
- list_del(&page->slab_list);
- if (!page->active) {
- list_add_tail(&page->slab_list, &n->slabs_free);
+ list_del(&slab->slab_list);
+ if (!slab->active) {
+ list_add_tail(&slab->slab_list, &n->slabs_free);
n->free_slabs++;
} else
- list_add_tail(&page->slab_list, &n->slabs_partial);
+ list_add_tail(&slab->slab_list, &n->slabs_partial);
- list_for_each_entry(page, &n->slabs_partial, slab_list) {
- if (!PageSlabPfmemalloc(page))
- return page;
+ list_for_each_entry(slab, &n->slabs_partial, slab_list) {
+ if (!slab_test_pfmemalloc(slab))
+ return slab;
}
n->free_touched = 1;
- list_for_each_entry(page, &n->slabs_free, slab_list) {
- if (!PageSlabPfmemalloc(page)) {
+ list_for_each_entry(slab, &n->slabs_free, slab_list) {
+ if (!slab_test_pfmemalloc(slab)) {
n->free_slabs--;
- return page;
+ return slab;
}
}
return NULL;
}
-static struct page *get_first_slab(struct kmem_cache_node *n, bool pfmemalloc)
+static struct slab *get_first_slab(struct kmem_cache_node *n, bool pfmemalloc)
{
- struct page *page;
+ struct slab *slab;
assert_spin_locked(&n->list_lock);
- page = list_first_entry_or_null(&n->slabs_partial, struct page,
+ slab = list_first_entry_or_null(&n->slabs_partial, struct slab,
slab_list);
- if (!page) {
+ if (!slab) {
n->free_touched = 1;
- page = list_first_entry_or_null(&n->slabs_free, struct page,
+ slab = list_first_entry_or_null(&n->slabs_free, struct slab,
slab_list);
- if (page)
+ if (slab)
n->free_slabs--;
}
if (sk_memalloc_socks())
- page = get_valid_first_slab(n, page, pfmemalloc);
+ slab = get_valid_first_slab(n, slab, pfmemalloc);
- return page;
+ return slab;
}
static noinline void *cache_alloc_pfmemalloc(struct kmem_cache *cachep,
struct kmem_cache_node *n, gfp_t flags)
{
- struct page *page;
+ struct slab *slab;
void *obj;
void *list = NULL;
@@ -2855,16 +2847,16 @@ static noinline void *cache_alloc_pfmemalloc(struct kmem_cache *cachep,
return NULL;
spin_lock(&n->list_lock);
- page = get_first_slab(n, true);
- if (!page) {
+ slab = get_first_slab(n, true);
+ if (!slab) {
spin_unlock(&n->list_lock);
return NULL;
}
- obj = slab_get_obj(cachep, page);
+ obj = slab_get_obj(cachep, slab);
n->free_objects--;
- fixup_slab_list(cachep, n, page, &list);
+ fixup_slab_list(cachep, n, slab, &list);
spin_unlock(&n->list_lock);
fixup_objfreelist_debug(cachep, &list);
@@ -2877,20 +2869,20 @@ static noinline void *cache_alloc_pfmemalloc(struct kmem_cache *cachep,
* or cache_grow_end() for new slab
*/
static __always_inline int alloc_block(struct kmem_cache *cachep,
- struct array_cache *ac, struct page *page, int batchcount)
+ struct array_cache *ac, struct slab *slab, int batchcount)
{
/*
* There must be at least one object available for
* allocation.
*/
- BUG_ON(page->active >= cachep->num);
+ BUG_ON(slab->active >= cachep->num);
- while (page->active < cachep->num && batchcount--) {
+ while (slab->active < cachep->num && batchcount--) {
STATS_INC_ALLOCED(cachep);
STATS_INC_ACTIVE(cachep);
STATS_SET_HIGH(cachep);
- ac->entry[ac->avail++] = slab_get_obj(cachep, page);
+ ac->entry[ac->avail++] = slab_get_obj(cachep, slab);
}
return batchcount;
@@ -2903,7 +2895,7 @@ static void *cache_alloc_refill(struct kmem_cache *cachep, gfp_t flags)
struct array_cache *ac, *shared;
int node;
void *list = NULL;
- struct page *page;
+ struct slab *slab;
check_irq_off();
node = numa_mem_id();
@@ -2936,14 +2928,14 @@ static void *cache_alloc_refill(struct kmem_cache *cachep, gfp_t flags)
while (batchcount > 0) {
/* Get slab alloc is to come from. */
- page = get_first_slab(n, false);
- if (!page)
+ slab = get_first_slab(n, false);
+ if (!slab)
goto must_grow;
check_spinlock_acquired(cachep);
- batchcount = alloc_block(cachep, ac, page, batchcount);
- fixup_slab_list(cachep, n, page, &list);
+ batchcount = alloc_block(cachep, ac, slab, batchcount);
+ fixup_slab_list(cachep, n, slab, &list);
}
must_grow:
@@ -2962,16 +2954,16 @@ static void *cache_alloc_refill(struct kmem_cache *cachep, gfp_t flags)
return obj;
}
- page = cache_grow_begin(cachep, gfp_exact_node(flags), node);
+ slab = cache_grow_begin(cachep, gfp_exact_node(flags), node);
/*
* cache_grow_begin() can reenable interrupts,
* then ac could change.
*/
ac = cpu_cache_get(cachep);
- if (!ac->avail && page)
- alloc_block(cachep, ac, page, batchcount);
- cache_grow_end(cachep, page);
+ if (!ac->avail && slab)
+ alloc_block(cachep, ac, slab, batchcount);
+ cache_grow_end(cachep, slab);
if (!ac->avail)
return NULL;
@@ -3101,7 +3093,7 @@ static void *fallback_alloc(struct kmem_cache *cache, gfp_t flags)
struct zone *zone;
enum zone_type highest_zoneidx = gfp_zone(flags);
void *obj = NULL;
- struct page *page;
+ struct slab *slab;
int nid;
unsigned int cpuset_mems_cookie;
@@ -3137,10 +3129,10 @@ static void *fallback_alloc(struct kmem_cache *cache, gfp_t flags)
* We may trigger various forms of reclaim on the allowed
* set and go into memory reserves if necessary.
*/
- page = cache_grow_begin(cache, flags, numa_mem_id());
- cache_grow_end(cache, page);
- if (page) {
- nid = page_to_nid(page);
+ slab = cache_grow_begin(cache, flags, numa_mem_id());
+ cache_grow_end(cache, slab);
+ if (slab) {
+ nid = slab_nid(slab);
obj = ____cache_alloc_node(cache,
gfp_exact_node(flags), nid);
@@ -3164,7 +3156,7 @@ static void *fallback_alloc(struct kmem_cache *cache, gfp_t flags)
static void *____cache_alloc_node(struct kmem_cache *cachep, gfp_t flags,
int nodeid)
{
- struct page *page;
+ struct slab *slab;
struct kmem_cache_node *n;
void *obj = NULL;
void *list = NULL;
@@ -3175,8 +3167,8 @@ static void *____cache_alloc_node(struct kmem_cache *cachep, gfp_t flags,
check_irq_off();
spin_lock(&n->list_lock);
- page = get_first_slab(n, false);
- if (!page)
+ slab = get_first_slab(n, false);
+ if (!slab)
goto must_grow;
check_spinlock_acquired_node(cachep, nodeid);
@@ -3185,12 +3177,12 @@ static void *____cache_alloc_node(struct kmem_cache *cachep, gfp_t flags,
STATS_INC_ACTIVE(cachep);
STATS_SET_HIGH(cachep);
- BUG_ON(page->active == cachep->num);
+ BUG_ON(slab->active == cachep->num);
- obj = slab_get_obj(cachep, page);
+ obj = slab_get_obj(cachep, slab);
n->free_objects--;
- fixup_slab_list(cachep, n, page, &list);
+ fixup_slab_list(cachep, n, slab, &list);
spin_unlock(&n->list_lock);
fixup_objfreelist_debug(cachep, &list);
@@ -3198,12 +3190,12 @@ static void *____cache_alloc_node(struct kmem_cache *cachep, gfp_t flags,
must_grow:
spin_unlock(&n->list_lock);
- page = cache_grow_begin(cachep, gfp_exact_node(flags), nodeid);
- if (page) {
+ slab = cache_grow_begin(cachep, gfp_exact_node(flags), nodeid);
+ if (slab) {
/* This slab isn't counted yet so don't update free_objects */
- obj = slab_get_obj(cachep, page);
+ obj = slab_get_obj(cachep, slab);
}
- cache_grow_end(cachep, page);
+ cache_grow_end(cachep, slab);
return obj ? obj : fallback_alloc(cachep, flags);
}
@@ -3333,40 +3325,40 @@ static void free_block(struct kmem_cache *cachep, void **objpp,
{
int i;
struct kmem_cache_node *n = get_node(cachep, node);
- struct page *page;
+ struct slab *slab;
n->free_objects += nr_objects;
for (i = 0; i < nr_objects; i++) {
void *objp;
- struct page *page;
+ struct slab *slab;
objp = objpp[i];
- page = virt_to_head_page(objp);
- list_del(&page->slab_list);
+ slab = virt_to_slab(objp);
+ list_del(&slab->slab_list);
check_spinlock_acquired_node(cachep, node);
- slab_put_obj(cachep, page, objp);
+ slab_put_obj(cachep, slab, objp);
STATS_DEC_ACTIVE(cachep);
/* fixup slab chains */
- if (page->active == 0) {
- list_add(&page->slab_list, &n->slabs_free);
+ if (slab->active == 0) {
+ list_add(&slab->slab_list, &n->slabs_free);
n->free_slabs++;
} else {
/* Unconditionally move a slab to the end of the
* partial list on free - maximum time for the
* other objects to be freed, too.
*/
- list_add_tail(&page->slab_list, &n->slabs_partial);
+ list_add_tail(&slab->slab_list, &n->slabs_partial);
}
}
while (n->free_objects > n->free_limit && !list_empty(&n->slabs_free)) {
n->free_objects -= cachep->num;
- page = list_last_entry(&n->slabs_free, struct page, slab_list);
- list_move(&page->slab_list, list);
+ slab = list_last_entry(&n->slabs_free, struct slab, slab_list);
+ list_move(&slab->slab_list, list);
n->free_slabs--;
n->total_slabs--;
}
@@ -3402,10 +3394,10 @@ static void cache_flusharray(struct kmem_cache *cachep, struct array_cache *ac)
#if STATS
{
int i = 0;
- struct page *page;
+ struct slab *slab;
- list_for_each_entry(page, &n->slabs_free, slab_list) {
- BUG_ON(page->active);
+ list_for_each_entry(slab, &n->slabs_free, slab_list) {
+ BUG_ON(slab->active);
i++;
}
@@ -3481,10 +3473,10 @@ void ___cache_free(struct kmem_cache *cachep, void *objp,
}
if (sk_memalloc_socks()) {
- struct page *page = virt_to_head_page(objp);
+ struct slab *slab = virt_to_slab(objp);
- if (unlikely(PageSlabPfmemalloc(page))) {
- cache_free_pfmemalloc(cachep, page, objp);
+ if (unlikely(slab_test_pfmemalloc(slab))) {
+ cache_free_pfmemalloc(cachep, slab, objp);
return;
}
}
@@ -3657,21 +3649,21 @@ EXPORT_SYMBOL(__kmalloc_node_track_caller);
#endif /* CONFIG_NUMA */
#ifdef CONFIG_PRINTK
-void kmem_obj_info(struct kmem_obj_info *kpp, void *object, struct page *page)
+void kmem_obj_info(struct kmem_obj_info *kpp, void *object, struct slab *slab)
{
struct kmem_cache *cachep;
unsigned int objnr;
void *objp;
kpp->kp_ptr = object;
- kpp->kp_page = page;
- cachep = page->slab_cache;
+ kpp->kp_slab = slab;
+ cachep = slab->slab_cache;
kpp->kp_slab_cache = cachep;
objp = object - obj_offset(cachep);
kpp->kp_data_offset = obj_offset(cachep);
- page = virt_to_head_page(objp);
- objnr = obj_to_index(cachep, page, objp);
- objp = index_to_obj(cachep, page, objnr);
+ slab = virt_to_slab(objp);
+ objnr = obj_to_index(cachep, slab, objp);
+ objp = index_to_obj(cachep, slab, objnr);
kpp->kp_objp = objp;
if (DEBUG && cachep->flags & SLAB_STORE_USER)
kpp->kp_ret = *dbg_userword(cachep, objp);
@@ -4177,8 +4169,8 @@ ssize_t slabinfo_write(struct file *file, const char __user *buffer,
* Returns NULL if check passes, otherwise const char * to name of cache
* to indicate an error.
*/
-void __check_heap_object(const void *ptr, unsigned long n, struct page *page,
- bool to_user)
+void __check_heap_object(const void *ptr, unsigned long n,
+ const struct slab *slab, bool to_user)
{
struct kmem_cache *cachep;
unsigned int objnr;
@@ -4187,15 +4179,15 @@ void __check_heap_object(const void *ptr, unsigned long n, struct page *page,
ptr = kasan_reset_tag(ptr);
/* Find and validate object. */
- cachep = page->slab_cache;
- objnr = obj_to_index(cachep, page, (void *)ptr);
+ cachep = slab->slab_cache;
+ objnr = obj_to_index(cachep, slab, (void *)ptr);
BUG_ON(objnr >= cachep->num);
/* Find offset within object. */
if (is_kfence_address(ptr))
offset = ptr - kfence_object_start(ptr);
else
- offset = ptr - index_to_obj(cachep, page, objnr) - obj_offset(cachep);
+ offset = ptr - index_to_obj(cachep, slab, objnr) - obj_offset(cachep);
/* Allow address range falling entirely within usercopy region. */
if (offset >= cachep->useroffset &&
diff --git a/mm/slab.h b/mm/slab.h
index 56ad7ee..95b9a74 100644
--- a/mm/slab.h
+++ b/mm/slab.h
@@ -5,6 +5,197 @@
* Internal slab definitions
*/
+/* Reuses the bits in struct page */
+struct slab {
+ unsigned long __page_flags;
+
+#if defined(CONFIG_SLAB)
+
+ union {
+ struct list_head slab_list;
+ struct rcu_head rcu_head;
+ };
+ struct kmem_cache *slab_cache;
+ void *freelist; /* array of free object indexes */
+ void *s_mem; /* first object */
+ unsigned int active;
+
+#elif defined(CONFIG_SLUB)
+
+ union {
+ struct list_head slab_list;
+ struct rcu_head rcu_head;
+#ifdef CONFIG_SLUB_CPU_PARTIAL
+ struct {
+ struct slab *next;
+ int slabs; /* Nr of slabs left */
+ };
+#endif
+ };
+ struct kmem_cache *slab_cache;
+ /* Double-word boundary */
+ void *freelist; /* first free object */
+ union {
+ unsigned long counters;
+ struct {
+ unsigned inuse:16;
+ unsigned objects:15;
+ unsigned frozen:1;
+ };
+ };
+ unsigned int __unused;
+
+#elif defined(CONFIG_SLOB)
+
+ struct list_head slab_list;
+ void *__unused_1;
+ void *freelist; /* first free block */
+ long units;
+ unsigned int __unused_2;
+
+#else
+#error "Unexpected slab allocator configured"
+#endif
+
+ atomic_t __page_refcount;
+#ifdef CONFIG_MEMCG
+ unsigned long memcg_data;
+#endif
+};
+
+#define SLAB_MATCH(pg, sl) \
+ static_assert(offsetof(struct page, pg) == offsetof(struct slab, sl))
+SLAB_MATCH(flags, __page_flags);
+SLAB_MATCH(compound_head, slab_list); /* Ensure bit 0 is clear */
+SLAB_MATCH(slab_list, slab_list);
+#ifndef CONFIG_SLOB
+SLAB_MATCH(rcu_head, rcu_head);
+SLAB_MATCH(slab_cache, slab_cache);
+#endif
+#ifdef CONFIG_SLAB
+SLAB_MATCH(s_mem, s_mem);
+SLAB_MATCH(active, active);
+#endif
+SLAB_MATCH(_refcount, __page_refcount);
+#ifdef CONFIG_MEMCG
+SLAB_MATCH(memcg_data, memcg_data);
+#endif
+#undef SLAB_MATCH
+static_assert(sizeof(struct slab) <= sizeof(struct page));
+
+/**
+ * folio_slab - Converts from folio to slab.
+ * @folio: The folio.
+ *
+ * Currently struct slab is a different representation of a folio where
+ * folio_test_slab() is true.
+ *
+ * Return: The slab which contains this folio.
+ */
+#define folio_slab(folio) (_Generic((folio), \
+ const struct folio *: (const struct slab *)(folio), \
+ struct folio *: (struct slab *)(folio)))
+
+/**
+ * slab_folio - The folio allocated for a slab
+ * @slab: The slab.
+ *
+ * Slabs are allocated as folios that contain the individual objects and are
+ * using some fields in the first struct page of the folio - those fields are
+ * now accessed by struct slab. It is occasionally necessary to convert back to
+ * a folio in order to communicate with the rest of the mm. Please use this
+ * helper function instead of casting yourself, as the implementation may change
+ * in the future.
+ */
+#define slab_folio(s) (_Generic((s), \
+ const struct slab *: (const struct folio *)s, \
+ struct slab *: (struct folio *)s))
+
+/**
+ * page_slab - Converts from first struct page to slab.
+ * @p: The first (either head of compound or single) page of slab.
+ *
+ * A temporary wrapper to convert struct page to struct slab in situations where
+ * we know the page is the compound head, or single order-0 page.
+ *
+ * Long-term ideally everything would work with struct slab directly or go
+ * through folio to struct slab.
+ *
+ * Return: The slab which contains this page
+ */
+#define page_slab(p) (_Generic((p), \
+ const struct page *: (const struct slab *)(p), \
+ struct page *: (struct slab *)(p)))
+
+/**
+ * slab_page - The first struct page allocated for a slab
+ * @slab: The slab.
+ *
+ * A convenience wrapper for converting slab to the first struct page of the
+ * underlying folio, to communicate with code not yet converted to folio or
+ * struct slab.
+ */
+#define slab_page(s) folio_page(slab_folio(s), 0)
+
+/*
+ * If network-based swap is enabled, sl*b must keep track of whether pages
+ * were allocated from pfmemalloc reserves.
+ */
+static inline bool slab_test_pfmemalloc(const struct slab *slab)
+{
+ return folio_test_active((struct folio *)slab_folio(slab));
+}
+
+static inline void slab_set_pfmemalloc(struct slab *slab)
+{
+ folio_set_active(slab_folio(slab));
+}
+
+static inline void slab_clear_pfmemalloc(struct slab *slab)
+{
+ folio_clear_active(slab_folio(slab));
+}
+
+static inline void __slab_clear_pfmemalloc(struct slab *slab)
+{
+ __folio_clear_active(slab_folio(slab));
+}
+
+static inline void *slab_address(const struct slab *slab)
+{
+ return folio_address(slab_folio(slab));
+}
+
+static inline int slab_nid(const struct slab *slab)
+{
+ return folio_nid(slab_folio(slab));
+}
+
+static inline pg_data_t *slab_pgdat(const struct slab *slab)
+{
+ return folio_pgdat(slab_folio(slab));
+}
+
+static inline struct slab *virt_to_slab(const void *addr)
+{
+ struct folio *folio = virt_to_folio(addr);
+
+ if (!folio_test_slab(folio))
+ return NULL;
+
+ return folio_slab(folio);
+}
+
+static inline int slab_order(const struct slab *slab)
+{
+ return folio_order((struct folio *)slab_folio(slab));
+}
+
+static inline size_t slab_size(const struct slab *slab)
+{
+ return PAGE_SIZE << slab_order(slab);
+}
+
#ifdef CONFIG_SLOB
/*
* Common fields provided in kmem_cache by all slab allocators
@@ -245,15 +436,33 @@ static inline bool kmem_cache_debug_flags(struct kmem_cache *s, slab_flags_t fla
}
#ifdef CONFIG_MEMCG_KMEM
-int memcg_alloc_page_obj_cgroups(struct page *page, struct kmem_cache *s,
- gfp_t gfp, bool new_page);
+/*
+ * slab_objcgs - get the object cgroups vector associated with a slab
+ * @slab: a pointer to the slab struct
+ *
+ * Returns a pointer to the object cgroups vector associated with the slab,
+ * or NULL if no such vector has been associated yet.
+ */
+static inline struct obj_cgroup **slab_objcgs(struct slab *slab)
+{
+ unsigned long memcg_data = READ_ONCE(slab->memcg_data);
+
+ VM_BUG_ON_PAGE(memcg_data && !(memcg_data & MEMCG_DATA_OBJCGS),
+ slab_page(slab));
+ VM_BUG_ON_PAGE(memcg_data & MEMCG_DATA_KMEM, slab_page(slab));
+
+ return (struct obj_cgroup **)(memcg_data & ~MEMCG_DATA_FLAGS_MASK);
+}
+
+int memcg_alloc_slab_cgroups(struct slab *slab, struct kmem_cache *s,
+ gfp_t gfp, bool new_slab);
void mod_objcg_state(struct obj_cgroup *objcg, struct pglist_data *pgdat,
enum node_stat_item idx, int nr);
-static inline void memcg_free_page_obj_cgroups(struct page *page)
+static inline void memcg_free_slab_cgroups(struct slab *slab)
{
- kfree(page_objcgs(page));
- page->memcg_data = 0;
+ kfree(slab_objcgs(slab));
+ slab->memcg_data = 0;
}
static inline size_t obj_full_size(struct kmem_cache *s)
@@ -298,7 +507,7 @@ static inline void memcg_slab_post_alloc_hook(struct kmem_cache *s,
gfp_t flags, size_t size,
void **p)
{
- struct page *page;
+ struct slab *slab;
unsigned long off;
size_t i;
@@ -307,19 +516,19 @@ static inline void memcg_slab_post_alloc_hook(struct kmem_cache *s,
for (i = 0; i < size; i++) {
if (likely(p[i])) {
- page = virt_to_head_page(p[i]);
+ slab = virt_to_slab(p[i]);
- if (!page_objcgs(page) &&
- memcg_alloc_page_obj_cgroups(page, s, flags,
+ if (!slab_objcgs(slab) &&
+ memcg_alloc_slab_cgroups(slab, s, flags,
false)) {
obj_cgroup_uncharge(objcg, obj_full_size(s));
continue;
}
- off = obj_to_index(s, page, p[i]);
+ off = obj_to_index(s, slab, p[i]);
obj_cgroup_get(objcg);
- page_objcgs(page)[off] = objcg;
- mod_objcg_state(objcg, page_pgdat(page),
+ slab_objcgs(slab)[off] = objcg;
+ mod_objcg_state(objcg, slab_pgdat(slab),
cache_vmstat_idx(s), obj_full_size(s));
} else {
obj_cgroup_uncharge(objcg, obj_full_size(s));
@@ -334,7 +543,7 @@ static inline void memcg_slab_free_hook(struct kmem_cache *s_orig,
struct kmem_cache *s;
struct obj_cgroup **objcgs;
struct obj_cgroup *objcg;
- struct page *page;
+ struct slab *slab;
unsigned int off;
int i;
@@ -345,43 +554,52 @@ static inline void memcg_slab_free_hook(struct kmem_cache *s_orig,
if (unlikely(!p[i]))
continue;
- page = virt_to_head_page(p[i]);
- objcgs = page_objcgs_check(page);
+ slab = virt_to_slab(p[i]);
+ /* we could be given a kmalloc_large() object, skip those */
+ if (!slab)
+ continue;
+
+ objcgs = slab_objcgs(slab);
if (!objcgs)
continue;
if (!s_orig)
- s = page->slab_cache;
+ s = slab->slab_cache;
else
s = s_orig;
- off = obj_to_index(s, page, p[i]);
+ off = obj_to_index(s, slab, p[i]);
objcg = objcgs[off];
if (!objcg)
continue;
objcgs[off] = NULL;
obj_cgroup_uncharge(objcg, obj_full_size(s));
- mod_objcg_state(objcg, page_pgdat(page), cache_vmstat_idx(s),
+ mod_objcg_state(objcg, slab_pgdat(slab), cache_vmstat_idx(s),
-obj_full_size(s));
obj_cgroup_put(objcg);
}
}
#else /* CONFIG_MEMCG_KMEM */
+static inline struct obj_cgroup **slab_objcgs(struct slab *slab)
+{
+ return NULL;
+}
+
static inline struct mem_cgroup *memcg_from_slab_obj(void *ptr)
{
return NULL;
}
-static inline int memcg_alloc_page_obj_cgroups(struct page *page,
+static inline int memcg_alloc_slab_cgroups(struct slab *slab,
struct kmem_cache *s, gfp_t gfp,
- bool new_page)
+ bool new_slab)
{
return 0;
}
-static inline void memcg_free_page_obj_cgroups(struct page *page)
+static inline void memcg_free_slab_cgroups(struct slab *slab)
{
}
@@ -405,35 +623,35 @@ static inline void memcg_slab_free_hook(struct kmem_cache *s,
}
#endif /* CONFIG_MEMCG_KMEM */
+#ifndef CONFIG_SLOB
static inline struct kmem_cache *virt_to_cache(const void *obj)
{
- struct page *page;
+ struct slab *slab;
- page = virt_to_head_page(obj);
- if (WARN_ONCE(!PageSlab(page), "%s: Object is not a Slab page!\n",
+ slab = virt_to_slab(obj);
+ if (WARN_ONCE(!slab, "%s: Object is not a Slab page!\n",
__func__))
return NULL;
- return page->slab_cache;
+ return slab->slab_cache;
}
-static __always_inline void account_slab_page(struct page *page, int order,
- struct kmem_cache *s,
- gfp_t gfp)
+static __always_inline void account_slab(struct slab *slab, int order,
+ struct kmem_cache *s, gfp_t gfp)
{
if (memcg_kmem_enabled() && (s->flags & SLAB_ACCOUNT))
- memcg_alloc_page_obj_cgroups(page, s, gfp, true);
+ memcg_alloc_slab_cgroups(slab, s, gfp, true);
- mod_node_page_state(page_pgdat(page), cache_vmstat_idx(s),
+ mod_node_page_state(slab_pgdat(slab), cache_vmstat_idx(s),
PAGE_SIZE << order);
}
-static __always_inline void unaccount_slab_page(struct page *page, int order,
- struct kmem_cache *s)
+static __always_inline void unaccount_slab(struct slab *slab, int order,
+ struct kmem_cache *s)
{
if (memcg_kmem_enabled())
- memcg_free_page_obj_cgroups(page);
+ memcg_free_slab_cgroups(slab);
- mod_node_page_state(page_pgdat(page), cache_vmstat_idx(s),
+ mod_node_page_state(slab_pgdat(slab), cache_vmstat_idx(s),
-(PAGE_SIZE << order));
}
@@ -452,6 +670,7 @@ static inline struct kmem_cache *cache_from_obj(struct kmem_cache *s, void *x)
print_tracking(cachep, x);
return cachep;
}
+#endif /* CONFIG_SLOB */
static inline size_t slab_ksize(const struct kmem_cache *s)
{
@@ -635,7 +854,7 @@ static inline void debugfs_slab_release(struct kmem_cache *s) { }
#define KS_ADDRS_COUNT 16
struct kmem_obj_info {
void *kp_ptr;
- struct page *kp_page;
+ struct slab *kp_slab;
void *kp_objp;
unsigned long kp_data_offset;
struct kmem_cache *kp_slab_cache;
@@ -643,7 +862,18 @@ struct kmem_obj_info {
void *kp_stack[KS_ADDRS_COUNT];
void *kp_free_stack[KS_ADDRS_COUNT];
};
-void kmem_obj_info(struct kmem_obj_info *kpp, void *object, struct page *page);
+void kmem_obj_info(struct kmem_obj_info *kpp, void *object, struct slab *slab);
+#endif
+
+#ifdef CONFIG_HAVE_HARDENED_USERCOPY_ALLOCATOR
+void __check_heap_object(const void *ptr, unsigned long n,
+ const struct slab *slab, bool to_user);
+#else
+static inline
+void __check_heap_object(const void *ptr, unsigned long n,
+ const struct slab *slab, bool to_user)
+{
+}
#endif
#endif /* MM_SLAB_H */
diff --git a/mm/slab_common.c b/mm/slab_common.c
index e5d080a..dc15566 100644
--- a/mm/slab_common.c
+++ b/mm/slab_common.c
@@ -550,13 +550,13 @@ bool slab_is_available(void)
*/
bool kmem_valid_obj(void *object)
{
- struct page *page;
+ struct folio *folio;
/* Some arches consider ZERO_SIZE_PTR to be a valid address. */
if (object < (void *)PAGE_SIZE || !virt_addr_valid(object))
return false;
- page = virt_to_head_page(object);
- return PageSlab(page);
+ folio = virt_to_folio(object);
+ return folio_test_slab(folio);
}
EXPORT_SYMBOL_GPL(kmem_valid_obj);
@@ -579,18 +579,18 @@ void kmem_dump_obj(void *object)
{
char *cp = IS_ENABLED(CONFIG_MMU) ? "" : "/vmalloc";
int i;
- struct page *page;
+ struct slab *slab;
unsigned long ptroffset;
struct kmem_obj_info kp = { };
if (WARN_ON_ONCE(!virt_addr_valid(object)))
return;
- page = virt_to_head_page(object);
- if (WARN_ON_ONCE(!PageSlab(page))) {
+ slab = virt_to_slab(object);
+ if (WARN_ON_ONCE(!slab)) {
pr_cont(" non-slab memory.\n");
return;
}
- kmem_obj_info(&kp, object, page);
+ kmem_obj_info(&kp, object, slab);
if (kp.kp_slab_cache)
pr_cont(" slab%s %s", cp, kp.kp_slab_cache->name);
else
diff --git a/mm/slob.c b/mm/slob.c
index 03deee1..60c5842 100644
--- a/mm/slob.c
+++ b/mm/slob.c
@@ -30,7 +30,7 @@
* If kmalloc is asked for objects of PAGE_SIZE or larger, it calls
* alloc_pages() directly, allocating compound pages so the page order
* does not have to be separately tracked.
- * These objects are detected in kfree() because PageSlab()
+ * These objects are detected in kfree() because folio_test_slab()
* is false for them.
*
* SLAB is emulated on top of SLOB by simply calling constructors and
@@ -105,21 +105,21 @@ static LIST_HEAD(free_slob_large);
/*
* slob_page_free: true for pages on free_slob_pages list.
*/
-static inline int slob_page_free(struct page *sp)
+static inline int slob_page_free(struct slab *slab)
{
- return PageSlobFree(sp);
+ return PageSlobFree(slab_page(slab));
}
-static void set_slob_page_free(struct page *sp, struct list_head *list)
+static void set_slob_page_free(struct slab *slab, struct list_head *list)
{
- list_add(&sp->slab_list, list);
- __SetPageSlobFree(sp);
+ list_add(&slab->slab_list, list);
+ __SetPageSlobFree(slab_page(slab));
}
-static inline void clear_slob_page_free(struct page *sp)
+static inline void clear_slob_page_free(struct slab *slab)
{
- list_del(&sp->slab_list);
- __ClearPageSlobFree(sp);
+ list_del(&slab->slab_list);
+ __ClearPageSlobFree(slab_page(slab));
}
#define SLOB_UNIT sizeof(slob_t)
@@ -234,7 +234,7 @@ static void slob_free_pages(void *b, int order)
* freelist, in this case @page_removed_from_list will be set to
* true (set to false otherwise).
*/
-static void *slob_page_alloc(struct page *sp, size_t size, int align,
+static void *slob_page_alloc(struct slab *sp, size_t size, int align,
int align_offset, bool *page_removed_from_list)
{
slob_t *prev, *cur, *aligned = NULL;
@@ -301,7 +301,8 @@ static void *slob_page_alloc(struct page *sp, size_t size, int align,
static void *slob_alloc(size_t size, gfp_t gfp, int align, int node,
int align_offset)
{
- struct page *sp;
+ struct folio *folio;
+ struct slab *sp;
struct list_head *slob_list;
slob_t *b = NULL;
unsigned long flags;
@@ -323,7 +324,7 @@ static void *slob_alloc(size_t size, gfp_t gfp, int align, int node,
* If there's a node specification, search for a partial
* page with a matching node id in the freelist.
*/
- if (node != NUMA_NO_NODE && page_to_nid(sp) != node)
+ if (node != NUMA_NO_NODE && slab_nid(sp) != node)
continue;
#endif
/* Enough room on this page? */
@@ -358,8 +359,9 @@ static void *slob_alloc(size_t size, gfp_t gfp, int align, int node,
b = slob_new_pages(gfp & ~__GFP_ZERO, 0, node);
if (!b)
return NULL;
- sp = virt_to_page(b);
- __SetPageSlab(sp);
+ folio = virt_to_folio(b);
+ __folio_set_slab(folio);
+ sp = folio_slab(folio);
spin_lock_irqsave(&slob_lock, flags);
sp->units = SLOB_UNITS(PAGE_SIZE);
@@ -381,7 +383,7 @@ static void *slob_alloc(size_t size, gfp_t gfp, int align, int node,
*/
static void slob_free(void *block, int size)
{
- struct page *sp;
+ struct slab *sp;
slob_t *prev, *next, *b = (slob_t *)block;
slobidx_t units;
unsigned long flags;
@@ -391,7 +393,7 @@ static void slob_free(void *block, int size)
return;
BUG_ON(!size);
- sp = virt_to_page(block);
+ sp = virt_to_slab(block);
units = SLOB_UNITS(size);
spin_lock_irqsave(&slob_lock, flags);
@@ -401,8 +403,7 @@ static void slob_free(void *block, int size)
if (slob_page_free(sp))
clear_slob_page_free(sp);
spin_unlock_irqrestore(&slob_lock, flags);
- __ClearPageSlab(sp);
- page_mapcount_reset(sp);
+ __folio_clear_slab(slab_folio(sp));
slob_free_pages(b, 0);
return;
}
@@ -462,10 +463,10 @@ static void slob_free(void *block, int size)
}
#ifdef CONFIG_PRINTK
-void kmem_obj_info(struct kmem_obj_info *kpp, void *object, struct page *page)
+void kmem_obj_info(struct kmem_obj_info *kpp, void *object, struct slab *slab)
{
kpp->kp_ptr = object;
- kpp->kp_page = page;
+ kpp->kp_slab = slab;
}
#endif
@@ -544,7 +545,7 @@ EXPORT_SYMBOL(__kmalloc_node_track_caller);
void kfree(const void *block)
{
- struct page *sp;
+ struct folio *sp;
trace_kfree(_RET_IP_, block);
@@ -552,16 +553,17 @@ void kfree(const void *block)
return;
kmemleak_free(block);
- sp = virt_to_page(block);
- if (PageSlab(sp)) {
+ sp = virt_to_folio(block);
+ if (folio_test_slab(sp)) {
int align = max_t(size_t, ARCH_KMALLOC_MINALIGN, ARCH_SLAB_MINALIGN);
unsigned int *m = (unsigned int *)(block - align);
slob_free(m, *m + align);
} else {
- unsigned int order = compound_order(sp);
- mod_node_page_state(page_pgdat(sp), NR_SLAB_UNRECLAIMABLE_B,
+ unsigned int order = folio_order(sp);
+
+ mod_node_page_state(folio_pgdat(sp), NR_SLAB_UNRECLAIMABLE_B,
-(PAGE_SIZE << order));
- __free_pages(sp, order);
+ __free_pages(folio_page(sp, 0), order);
}
}
@@ -570,7 +572,7 @@ EXPORT_SYMBOL(kfree);
/* can't use ksize for kmem_cache_alloc memory, only kmalloc */
size_t __ksize(const void *block)
{
- struct page *sp;
+ struct folio *folio;
int align;
unsigned int *m;
@@ -578,9 +580,9 @@ size_t __ksize(const void *block)
if (unlikely(block == ZERO_SIZE_PTR))
return 0;
- sp = virt_to_page(block);
- if (unlikely(!PageSlab(sp)))
- return page_size(sp);
+ folio = virt_to_folio(block);
+ if (unlikely(!folio_test_slab(folio)))
+ return folio_size(folio);
align = max_t(size_t, ARCH_KMALLOC_MINALIGN, ARCH_SLAB_MINALIGN);
m = (unsigned int *)(block - align);
diff --git a/mm/slub.c b/mm/slub.c
index abe7db5..2614740 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -48,7 +48,7 @@
* 1. slab_mutex (Global Mutex)
* 2. node->list_lock (Spinlock)
* 3. kmem_cache->cpu_slab->lock (Local lock)
- * 4. slab_lock(page) (Only on some arches or for debugging)
+ * 4. slab_lock(slab) (Only on some arches or for debugging)
* 5. object_map_lock (Only for debugging)
*
* slab_mutex
@@ -64,19 +64,19 @@
*
* The slab_lock is only used for debugging and on arches that do not
* have the ability to do a cmpxchg_double. It only protects:
- * A. page->freelist -> List of object free in a page
- * B. page->inuse -> Number of objects in use
- * C. page->objects -> Number of objects in page
- * D. page->frozen -> frozen state
+ * A. slab->freelist -> List of free objects in a slab
+ * B. slab->inuse -> Number of objects in use
+ * C. slab->objects -> Number of objects in slab
+ * D. slab->frozen -> frozen state
*
* Frozen slabs
*
* If a slab is frozen then it is exempt from list management. It is not
* on any list except per cpu partial list. The processor that froze the
- * slab is the one who can perform list operations on the page. Other
+ * slab is the one who can perform list operations on the slab. Other
* processors may put objects onto the freelist but the processor that
* froze the slab is the only one that can retrieve the objects from the
- * page's freelist.
+ * slab's freelist.
*
* list_lock
*
@@ -135,7 +135,7 @@
* minimal so we rely on the page allocators per cpu caches for
* fast frees and allocs.
*
- * page->frozen The slab is frozen and exempt from list processing.
+ * slab->frozen The slab is frozen and exempt from list processing.
* This means that the slab is dedicated to a purpose
* such as satisfying allocations for a specific
* processor. Objects may be freed in the slab while
@@ -250,7 +250,7 @@ static inline bool kmem_cache_has_cpu_partial(struct kmem_cache *s)
#define OO_SHIFT 16
#define OO_MASK ((1 << OO_SHIFT) - 1)
-#define MAX_OBJS_PER_PAGE 32767 /* since page.objects is u15 */
+#define MAX_OBJS_PER_PAGE 32767 /* since slab.objects is u15 */
/* Internal SLUB flags */
/* Poison object */
@@ -417,18 +417,18 @@ static inline unsigned int oo_objects(struct kmem_cache_order_objects x)
#ifdef CONFIG_SLUB_CPU_PARTIAL
static void slub_set_cpu_partial(struct kmem_cache *s, unsigned int nr_objects)
{
- unsigned int nr_pages;
+ unsigned int nr_slabs;
s->cpu_partial = nr_objects;
/*
* We take the number of objects but actually limit the number of
- * pages on the per cpu partial list, in order to limit excessive
- * growth of the list. For simplicity we assume that the pages will
+ * slabs on the per cpu partial list, in order to limit excessive
+ * growth of the list. For simplicity we assume that the slabs will
* be half-full.
*/
- nr_pages = DIV_ROUND_UP(nr_objects * 2, oo_objects(s->oo));
- s->cpu_partial_pages = nr_pages;
+ nr_slabs = DIV_ROUND_UP(nr_objects * 2, oo_objects(s->oo));
+ s->cpu_partial_slabs = nr_slabs;
}
#else
static inline void
@@ -440,28 +440,32 @@ slub_set_cpu_partial(struct kmem_cache *s, unsigned int nr_objects)
/*
* Per slab locking using the pagelock
*/
-static __always_inline void __slab_lock(struct page *page)
+static __always_inline void __slab_lock(struct slab *slab)
{
+ struct page *page = slab_page(slab);
+
VM_BUG_ON_PAGE(PageTail(page), page);
bit_spin_lock(PG_locked, &page->flags);
}
-static __always_inline void __slab_unlock(struct page *page)
+static __always_inline void __slab_unlock(struct slab *slab)
{
+ struct page *page = slab_page(slab);
+
VM_BUG_ON_PAGE(PageTail(page), page);
__bit_spin_unlock(PG_locked, &page->flags);
}
-static __always_inline void slab_lock(struct page *page, unsigned long *flags)
+static __always_inline void slab_lock(struct slab *slab, unsigned long *flags)
{
if (IS_ENABLED(CONFIG_PREEMPT_RT))
local_irq_save(*flags);
- __slab_lock(page);
+ __slab_lock(slab);
}
-static __always_inline void slab_unlock(struct page *page, unsigned long *flags)
+static __always_inline void slab_unlock(struct slab *slab, unsigned long *flags)
{
- __slab_unlock(page);
+ __slab_unlock(slab);
if (IS_ENABLED(CONFIG_PREEMPT_RT))
local_irq_restore(*flags);
}
@@ -471,7 +475,7 @@ static __always_inline void slab_unlock(struct page *page, unsigned long *flags)
* by an _irqsave() lock variant. Except on PREEMPT_RT where locks are different
* so we disable interrupts as part of slab_[un]lock().
*/
-static inline bool __cmpxchg_double_slab(struct kmem_cache *s, struct page *page,
+static inline bool __cmpxchg_double_slab(struct kmem_cache *s, struct slab *slab,
void *freelist_old, unsigned long counters_old,
void *freelist_new, unsigned long counters_new,
const char *n)
@@ -481,7 +485,7 @@ static inline bool __cmpxchg_double_slab(struct kmem_cache *s, struct page *page
#if defined(CONFIG_HAVE_CMPXCHG_DOUBLE) && \
defined(CONFIG_HAVE_ALIGNED_STRUCT_PAGE)
if (s->flags & __CMPXCHG_DOUBLE) {
- if (cmpxchg_double(&page->freelist, &page->counters,
+ if (cmpxchg_double(&slab->freelist, &slab->counters,
freelist_old, counters_old,
freelist_new, counters_new))
return true;
@@ -491,15 +495,15 @@ static inline bool __cmpxchg_double_slab(struct kmem_cache *s, struct page *page
/* init to 0 to prevent spurious warnings */
unsigned long flags = 0;
- slab_lock(page, &flags);
- if (page->freelist == freelist_old &&
- page->counters == counters_old) {
- page->freelist = freelist_new;
- page->counters = counters_new;
- slab_unlock(page, &flags);
+ slab_lock(slab, &flags);
+ if (slab->freelist == freelist_old &&
+ slab->counters == counters_old) {
+ slab->freelist = freelist_new;
+ slab->counters = counters_new;
+ slab_unlock(slab, &flags);
return true;
}
- slab_unlock(page, &flags);
+ slab_unlock(slab, &flags);
}
cpu_relax();
@@ -512,7 +516,7 @@ static inline bool __cmpxchg_double_slab(struct kmem_cache *s, struct page *page
return false;
}
-static inline bool cmpxchg_double_slab(struct kmem_cache *s, struct page *page,
+static inline bool cmpxchg_double_slab(struct kmem_cache *s, struct slab *slab,
void *freelist_old, unsigned long counters_old,
void *freelist_new, unsigned long counters_new,
const char *n)
@@ -520,7 +524,7 @@ static inline bool cmpxchg_double_slab(struct kmem_cache *s, struct page *page,
#if defined(CONFIG_HAVE_CMPXCHG_DOUBLE) && \
defined(CONFIG_HAVE_ALIGNED_STRUCT_PAGE)
if (s->flags & __CMPXCHG_DOUBLE) {
- if (cmpxchg_double(&page->freelist, &page->counters,
+ if (cmpxchg_double(&slab->freelist, &slab->counters,
freelist_old, counters_old,
freelist_new, counters_new))
return true;
@@ -530,16 +534,16 @@ static inline bool cmpxchg_double_slab(struct kmem_cache *s, struct page *page,
unsigned long flags;
local_irq_save(flags);
- __slab_lock(page);
- if (page->freelist == freelist_old &&
- page->counters == counters_old) {
- page->freelist = freelist_new;
- page->counters = counters_new;
- __slab_unlock(page);
+ __slab_lock(slab);
+ if (slab->freelist == freelist_old &&
+ slab->counters == counters_old) {
+ slab->freelist = freelist_new;
+ slab->counters = counters_new;
+ __slab_unlock(slab);
local_irq_restore(flags);
return true;
}
- __slab_unlock(page);
+ __slab_unlock(slab);
local_irq_restore(flags);
}
@@ -558,14 +562,14 @@ static unsigned long object_map[BITS_TO_LONGS(MAX_OBJS_PER_PAGE)];
static DEFINE_RAW_SPINLOCK(object_map_lock);
static void __fill_map(unsigned long *obj_map, struct kmem_cache *s,
- struct page *page)
+ struct slab *slab)
{
- void *addr = page_address(page);
+ void *addr = slab_address(slab);
void *p;
- bitmap_zero(obj_map, page->objects);
+ bitmap_zero(obj_map, slab->objects);
- for (p = page->freelist; p; p = get_freepointer(s, p))
+ for (p = slab->freelist; p; p = get_freepointer(s, p))
set_bit(__obj_to_index(s, addr, p), obj_map);
}
@@ -590,19 +594,19 @@ static inline bool slab_add_kunit_errors(void) { return false; }
#endif
/*
- * Determine a map of object in use on a page.
+ * Determine a map of objects in use in a slab.
*
- * Node listlock must be held to guarantee that the page does
+ * Node listlock must be held to guarantee that the slab does
* not vanish from under us.
*/
-static unsigned long *get_map(struct kmem_cache *s, struct page *page)
+static unsigned long *get_map(struct kmem_cache *s, struct slab *slab)
__acquires(&object_map_lock)
{
VM_BUG_ON(!irqs_disabled());
raw_spin_lock(&object_map_lock);
- __fill_map(object_map, s, page);
+ __fill_map(object_map, s, slab);
return object_map;
}
@@ -663,17 +667,17 @@ static inline void metadata_access_disable(void)
/* Verify that a pointer has an address that is valid within a slab page */
static inline int check_valid_pointer(struct kmem_cache *s,
- struct page *page, void *object)
+ struct slab *slab, void *object)
{
void *base;
if (!object)
return 1;
- base = page_address(page);
+ base = slab_address(slab);
object = kasan_reset_tag(object);
object = restore_red_left(s, object);
- if (object < base || object >= base + page->objects * s->size ||
+ if (object < base || object >= base + slab->objects * s->size ||
(object - base) % s->size) {
return 0;
}
@@ -784,12 +788,13 @@ void print_tracking(struct kmem_cache *s, void *object)
print_track("Freed", get_track(s, object, TRACK_FREE), pr_time);
}
-static void print_page_info(struct page *page)
+static void print_slab_info(const struct slab *slab)
{
- pr_err("Slab 0x%p objects=%u used=%u fp=0x%p flags=%pGp\n",
- page, page->objects, page->inuse, page->freelist,
- &page->flags);
+ struct folio *folio = (struct folio *)slab_folio(slab);
+ pr_err("Slab 0x%p objects=%u used=%u fp=0x%p flags=%pGp\n",
+ slab, slab->objects, slab->inuse, slab->freelist,
+ folio_flags(folio, 0));
}
static void slab_bug(struct kmem_cache *s, char *fmt, ...)
@@ -822,28 +827,14 @@ static void slab_fix(struct kmem_cache *s, char *fmt, ...)
va_end(args);
}
-static bool freelist_corrupted(struct kmem_cache *s, struct page *page,
- void **freelist, void *nextfree)
-{
- if ((s->flags & SLAB_CONSISTENCY_CHECKS) &&
- !check_valid_pointer(s, page, nextfree) && freelist) {
- object_err(s, page, *freelist, "Freechain corrupt");
- *freelist = NULL;
- slab_fix(s, "Isolate corrupted freechain");
- return true;
- }
-
- return false;
-}
-
-static void print_trailer(struct kmem_cache *s, struct page *page, u8 *p)
+static void print_trailer(struct kmem_cache *s, struct slab *slab, u8 *p)
{
unsigned int off; /* Offset of last byte */
- u8 *addr = page_address(page);
+ u8 *addr = slab_address(slab);
print_tracking(s, p);
- print_page_info(page);
+ print_slab_info(slab);
pr_err("Object 0x%p @offset=%tu fp=0x%p\n\n",
p, p - addr, get_freepointer(s, p));
@@ -875,18 +866,32 @@ static void print_trailer(struct kmem_cache *s, struct page *page, u8 *p)
dump_stack();
}
-void object_err(struct kmem_cache *s, struct page *page,
+static void object_err(struct kmem_cache *s, struct slab *slab,
u8 *object, char *reason)
{
if (slab_add_kunit_errors())
return;
slab_bug(s, "%s", reason);
- print_trailer(s, page, object);
+ print_trailer(s, slab, object);
add_taint(TAINT_BAD_PAGE, LOCKDEP_NOW_UNRELIABLE);
}
-static __printf(3, 4) void slab_err(struct kmem_cache *s, struct page *page,
+static bool freelist_corrupted(struct kmem_cache *s, struct slab *slab,
+ void **freelist, void *nextfree)
+{
+ if ((s->flags & SLAB_CONSISTENCY_CHECKS) &&
+ !check_valid_pointer(s, slab, nextfree) && freelist) {
+ object_err(s, slab, *freelist, "Freechain corrupt");
+ *freelist = NULL;
+ slab_fix(s, "Isolate corrupted freechain");
+ return true;
+ }
+
+ return false;
+}
+
+static __printf(3, 4) void slab_err(struct kmem_cache *s, struct slab *slab,
const char *fmt, ...)
{
va_list args;
@@ -899,7 +904,7 @@ static __printf(3, 4) void slab_err(struct kmem_cache *s, struct page *page,
vsnprintf(buf, sizeof(buf), fmt, args);
va_end(args);
slab_bug(s, "%s", buf);
- print_page_info(page);
+ print_slab_info(slab);
dump_stack();
add_taint(TAINT_BAD_PAGE, LOCKDEP_NOW_UNRELIABLE);
}
@@ -927,13 +932,13 @@ static void restore_bytes(struct kmem_cache *s, char *message, u8 data,
memset(from, data, to - from);
}
-static int check_bytes_and_report(struct kmem_cache *s, struct page *page,
+static int check_bytes_and_report(struct kmem_cache *s, struct slab *slab,
u8 *object, char *what,
u8 *start, unsigned int value, unsigned int bytes)
{
u8 *fault;
u8 *end;
- u8 *addr = page_address(page);
+ u8 *addr = slab_address(slab);
metadata_access_enable();
fault = memchr_inv(kasan_reset_tag(start), value, bytes);
@@ -952,7 +957,7 @@ static int check_bytes_and_report(struct kmem_cache *s, struct page *page,
pr_err("0x%p-0x%p @offset=%tu. First byte 0x%x instead of 0x%x\n",
fault, end - 1, fault - addr,
fault[0], value);
- print_trailer(s, page, object);
+ print_trailer(s, slab, object);
add_taint(TAINT_BAD_PAGE, LOCKDEP_NOW_UNRELIABLE);
skip_bug_print:
@@ -998,7 +1003,7 @@ static int check_bytes_and_report(struct kmem_cache *s, struct page *page,
* may be used with merged slabcaches.
*/
-static int check_pad_bytes(struct kmem_cache *s, struct page *page, u8 *p)
+static int check_pad_bytes(struct kmem_cache *s, struct slab *slab, u8 *p)
{
unsigned long off = get_info_end(s); /* The end of info */
@@ -1011,12 +1016,12 @@ static int check_pad_bytes(struct kmem_cache *s, struct page *page, u8 *p)
if (size_from_object(s) == off)
return 1;
- return check_bytes_and_report(s, page, p, "Object padding",
+ return check_bytes_and_report(s, slab, p, "Object padding",
p + off, POISON_INUSE, size_from_object(s) - off);
}
/* Check the pad bytes at the end of a slab page */
-static int slab_pad_check(struct kmem_cache *s, struct page *page)
+static int slab_pad_check(struct kmem_cache *s, struct slab *slab)
{
u8 *start;
u8 *fault;
@@ -1028,8 +1033,8 @@ static int slab_pad_check(struct kmem_cache *s, struct page *page)
if (!(s->flags & SLAB_POISON))
return 1;
- start = page_address(page);
- length = page_size(page);
+ start = slab_address(slab);
+ length = slab_size(slab);
end = start + length;
remainder = length % s->size;
if (!remainder)
@@ -1044,7 +1049,7 @@ static int slab_pad_check(struct kmem_cache *s, struct page *page)
while (end > fault && end[-1] == POISON_INUSE)
end--;
- slab_err(s, page, "Padding overwritten. 0x%p-0x%p @offset=%tu",
+ slab_err(s, slab, "Padding overwritten. 0x%p-0x%p @offset=%tu",
fault, end - 1, fault - start);
print_section(KERN_ERR, "Padding ", pad, remainder);
@@ -1052,23 +1057,23 @@ static int slab_pad_check(struct kmem_cache *s, struct page *page)
return 0;
}
-static int check_object(struct kmem_cache *s, struct page *page,
+static int check_object(struct kmem_cache *s, struct slab *slab,
void *object, u8 val)
{
u8 *p = object;
u8 *endobject = object + s->object_size;
if (s->flags & SLAB_RED_ZONE) {
- if (!check_bytes_and_report(s, page, object, "Left Redzone",
+ if (!check_bytes_and_report(s, slab, object, "Left Redzone",
object - s->red_left_pad, val, s->red_left_pad))
return 0;
- if (!check_bytes_and_report(s, page, object, "Right Redzone",
+ if (!check_bytes_and_report(s, slab, object, "Right Redzone",
endobject, val, s->inuse - s->object_size))
return 0;
} else {
if ((s->flags & SLAB_POISON) && s->object_size < s->inuse) {
- check_bytes_and_report(s, page, p, "Alignment padding",
+ check_bytes_and_report(s, slab, p, "Alignment padding",
endobject, POISON_INUSE,
s->inuse - s->object_size);
}
@@ -1076,15 +1081,15 @@ static int check_object(struct kmem_cache *s, struct page *page,
if (s->flags & SLAB_POISON) {
if (val != SLUB_RED_ACTIVE && (s->flags & __OBJECT_POISON) &&
- (!check_bytes_and_report(s, page, p, "Poison", p,
+ (!check_bytes_and_report(s, slab, p, "Poison", p,
POISON_FREE, s->object_size - 1) ||
- !check_bytes_and_report(s, page, p, "End Poison",
+ !check_bytes_and_report(s, slab, p, "End Poison",
p + s->object_size - 1, POISON_END, 1)))
return 0;
/*
* check_pad_bytes cleans up on its own.
*/
- check_pad_bytes(s, page, p);
+ check_pad_bytes(s, slab, p);
}
if (!freeptr_outside_object(s) && val == SLUB_RED_ACTIVE)
@@ -1095,8 +1100,8 @@ static int check_object(struct kmem_cache *s, struct page *page,
return 1;
/* Check free pointer validity */
- if (!check_valid_pointer(s, page, get_freepointer(s, p))) {
- object_err(s, page, p, "Freepointer corrupt");
+ if (!check_valid_pointer(s, slab, get_freepointer(s, p))) {
+ object_err(s, slab, p, "Freepointer corrupt");
/*
* No choice but to zap it and thus lose the remainder
* of the free objects in this slab. May cause
@@ -1108,55 +1113,55 @@ static int check_object(struct kmem_cache *s, struct page *page,
return 1;
}
-static int check_slab(struct kmem_cache *s, struct page *page)
+static int check_slab(struct kmem_cache *s, struct slab *slab)
{
int maxobj;
- if (!PageSlab(page)) {
- slab_err(s, page, "Not a valid slab page");
+ if (!folio_test_slab(slab_folio(slab))) {
+ slab_err(s, slab, "Not a valid slab page");
return 0;
}
- maxobj = order_objects(compound_order(page), s->size);
- if (page->objects > maxobj) {
- slab_err(s, page, "objects %u > max %u",
- page->objects, maxobj);
+ maxobj = order_objects(slab_order(slab), s->size);
+ if (slab->objects > maxobj) {
+ slab_err(s, slab, "objects %u > max %u",
+ slab->objects, maxobj);
return 0;
}
- if (page->inuse > page->objects) {
- slab_err(s, page, "inuse %u > max %u",
- page->inuse, page->objects);
+ if (slab->inuse > slab->objects) {
+ slab_err(s, slab, "inuse %u > max %u",
+ slab->inuse, slab->objects);
return 0;
}
/* Slab_pad_check fixes things up after itself */
- slab_pad_check(s, page);
+ slab_pad_check(s, slab);
return 1;
}
/*
- * Determine if a certain object on a page is on the freelist. Must hold the
+ * Determine if a certain object in a slab is on the freelist. Must hold the
* slab lock to guarantee that the chains are in a consistent state.
*/
-static int on_freelist(struct kmem_cache *s, struct page *page, void *search)
+static int on_freelist(struct kmem_cache *s, struct slab *slab, void *search)
{
int nr = 0;
void *fp;
void *object = NULL;
int max_objects;
- fp = page->freelist;
- while (fp && nr <= page->objects) {
+ fp = slab->freelist;
+ while (fp && nr <= slab->objects) {
if (fp == search)
return 1;
- if (!check_valid_pointer(s, page, fp)) {
+ if (!check_valid_pointer(s, slab, fp)) {
if (object) {
- object_err(s, page, object,
+ object_err(s, slab, object,
"Freechain corrupt");
set_freepointer(s, object, NULL);
} else {
- slab_err(s, page, "Freepointer corrupt");
- page->freelist = NULL;
- page->inuse = page->objects;
+ slab_err(s, slab, "Freepointer corrupt");
+ slab->freelist = NULL;
+ slab->inuse = slab->objects;
slab_fix(s, "Freelist cleared");
return 0;
}
@@ -1167,34 +1172,34 @@ static int on_freelist(struct kmem_cache *s, struct page *page, void *search)
nr++;
}
- max_objects = order_objects(compound_order(page), s->size);
+ max_objects = order_objects(slab_order(slab), s->size);
if (max_objects > MAX_OBJS_PER_PAGE)
max_objects = MAX_OBJS_PER_PAGE;
- if (page->objects != max_objects) {
- slab_err(s, page, "Wrong number of objects. Found %d but should be %d",
- page->objects, max_objects);
- page->objects = max_objects;
+ if (slab->objects != max_objects) {
+ slab_err(s, slab, "Wrong number of objects. Found %d but should be %d",
+ slab->objects, max_objects);
+ slab->objects = max_objects;
slab_fix(s, "Number of objects adjusted");
}
- if (page->inuse != page->objects - nr) {
- slab_err(s, page, "Wrong object count. Counter is %d but counted were %d",
- page->inuse, page->objects - nr);
- page->inuse = page->objects - nr;
+ if (slab->inuse != slab->objects - nr) {
+ slab_err(s, slab, "Wrong object count. Counter is %d but counted were %d",
+ slab->inuse, slab->objects - nr);
+ slab->inuse = slab->objects - nr;
slab_fix(s, "Object count adjusted");
}
return search == NULL;
}
-static void trace(struct kmem_cache *s, struct page *page, void *object,
+static void trace(struct kmem_cache *s, struct slab *slab, void *object,
int alloc)
{
if (s->flags & SLAB_TRACE) {
pr_info("TRACE %s %s 0x%p inuse=%d fp=0x%p\n",
s->name,
alloc ? "alloc" : "free",
- object, page->inuse,
- page->freelist);
+ object, slab->inuse,
+ slab->freelist);
if (!alloc)
print_section(KERN_INFO, "Object ", (void *)object,
@@ -1208,22 +1213,22 @@ static void trace(struct kmem_cache *s, struct page *page, void *object,
* Tracking of fully allocated slabs for debugging purposes.
*/
static void add_full(struct kmem_cache *s,
- struct kmem_cache_node *n, struct page *page)
+ struct kmem_cache_node *n, struct slab *slab)
{
if (!(s->flags & SLAB_STORE_USER))
return;
lockdep_assert_held(&n->list_lock);
- list_add(&page->slab_list, &n->full);
+ list_add(&slab->slab_list, &n->full);
}
-static void remove_full(struct kmem_cache *s, struct kmem_cache_node *n, struct page *page)
+static void remove_full(struct kmem_cache *s, struct kmem_cache_node *n, struct slab *slab)
{
if (!(s->flags & SLAB_STORE_USER))
return;
lockdep_assert_held(&n->list_lock);
- list_del(&page->slab_list);
+ list_del(&slab->slab_list);
}
/* Tracking of the number of slabs for debugging purposes */
@@ -1263,7 +1268,7 @@ static inline void dec_slabs_node(struct kmem_cache *s, int node, int objects)
}
/* Object debug checks for alloc/free paths */
-static void setup_object_debug(struct kmem_cache *s, struct page *page,
+static void setup_object_debug(struct kmem_cache *s, struct slab *slab,
void *object)
{
if (!kmem_cache_debug_flags(s, SLAB_STORE_USER|SLAB_RED_ZONE|__OBJECT_POISON))
@@ -1274,89 +1279,89 @@ static void setup_object_debug(struct kmem_cache *s, struct page *page,
}
static
-void setup_page_debug(struct kmem_cache *s, struct page *page, void *addr)
+void setup_slab_debug(struct kmem_cache *s, struct slab *slab, void *addr)
{
if (!kmem_cache_debug_flags(s, SLAB_POISON))
return;
metadata_access_enable();
- memset(kasan_reset_tag(addr), POISON_INUSE, page_size(page));
+ memset(kasan_reset_tag(addr), POISON_INUSE, slab_size(slab));
metadata_access_disable();
}
static inline int alloc_consistency_checks(struct kmem_cache *s,
- struct page *page, void *object)
+ struct slab *slab, void *object)
{
- if (!check_slab(s, page))
+ if (!check_slab(s, slab))
return 0;
- if (!check_valid_pointer(s, page, object)) {
- object_err(s, page, object, "Freelist Pointer check fails");
+ if (!check_valid_pointer(s, slab, object)) {
+ object_err(s, slab, object, "Freelist Pointer check fails");
return 0;
}
- if (!check_object(s, page, object, SLUB_RED_INACTIVE))
+ if (!check_object(s, slab, object, SLUB_RED_INACTIVE))
return 0;
return 1;
}
static noinline int alloc_debug_processing(struct kmem_cache *s,
- struct page *page,
+ struct slab *slab,
void *object, unsigned long addr)
{
if (s->flags & SLAB_CONSISTENCY_CHECKS) {
- if (!alloc_consistency_checks(s, page, object))
+ if (!alloc_consistency_checks(s, slab, object))
goto bad;
}
/* Success perform special debug activities for allocs */
if (s->flags & SLAB_STORE_USER)
set_track(s, object, TRACK_ALLOC, addr);
- trace(s, page, object, 1);
+ trace(s, slab, object, 1);
init_object(s, object, SLUB_RED_ACTIVE);
return 1;
bad:
- if (PageSlab(page)) {
+ if (folio_test_slab(slab_folio(slab))) {
/*
* If this is a slab page then lets do the best we can
* to avoid issues in the future. Marking all objects
* as used avoids touching the remaining objects.
*/
slab_fix(s, "Marking all objects used");
- page->inuse = page->objects;
- page->freelist = NULL;
+ slab->inuse = slab->objects;
+ slab->freelist = NULL;
}
return 0;
}
static inline int free_consistency_checks(struct kmem_cache *s,
- struct page *page, void *object, unsigned long addr)
+ struct slab *slab, void *object, unsigned long addr)
{
- if (!check_valid_pointer(s, page, object)) {
- slab_err(s, page, "Invalid object pointer 0x%p", object);
+ if (!check_valid_pointer(s, slab, object)) {
+ slab_err(s, slab, "Invalid object pointer 0x%p", object);
return 0;
}
- if (on_freelist(s, page, object)) {
- object_err(s, page, object, "Object already free");
+ if (on_freelist(s, slab, object)) {
+ object_err(s, slab, object, "Object already free");
return 0;
}
- if (!check_object(s, page, object, SLUB_RED_ACTIVE))
+ if (!check_object(s, slab, object, SLUB_RED_ACTIVE))
return 0;
- if (unlikely(s != page->slab_cache)) {
- if (!PageSlab(page)) {
- slab_err(s, page, "Attempt to free object(0x%p) outside of slab",
+ if (unlikely(s != slab->slab_cache)) {
+ if (!folio_test_slab(slab_folio(slab))) {
+ slab_err(s, slab, "Attempt to free object(0x%p) outside of slab",
object);
- } else if (!page->slab_cache) {
+ } else if (!slab->slab_cache) {
pr_err("SLUB <none>: no slab for object 0x%p.\n",
object);
dump_stack();
} else
- object_err(s, page, object,
+ object_err(s, slab, object,
"page slab pointer corrupt.");
return 0;
}
@@ -1365,21 +1370,21 @@ static inline int free_consistency_checks(struct kmem_cache *s,
/* Supports checking bulk free of a constructed freelist */
static noinline int free_debug_processing(
- struct kmem_cache *s, struct page *page,
+ struct kmem_cache *s, struct slab *slab,
void *head, void *tail, int bulk_cnt,
unsigned long addr)
{
- struct kmem_cache_node *n = get_node(s, page_to_nid(page));
+ struct kmem_cache_node *n = get_node(s, slab_nid(slab));
void *object = head;
int cnt = 0;
unsigned long flags, flags2;
int ret = 0;
spin_lock_irqsave(&n->list_lock, flags);
- slab_lock(page, &flags2);
+ slab_lock(slab, &flags2);
if (s->flags & SLAB_CONSISTENCY_CHECKS) {
- if (!check_slab(s, page))
+ if (!check_slab(s, slab))
goto out;
}
@@ -1387,13 +1392,13 @@ static noinline int free_debug_processing(
cnt++;
if (s->flags & SLAB_CONSISTENCY_CHECKS) {
- if (!free_consistency_checks(s, page, object, addr))
+ if (!free_consistency_checks(s, slab, object, addr))
goto out;
}
if (s->flags & SLAB_STORE_USER)
set_track(s, object, TRACK_FREE, addr);
- trace(s, page, object, 0);
+ trace(s, slab, object, 0);
/* Freepointer not overwritten by init_object(), SLAB_POISON moved it */
init_object(s, object, SLUB_RED_INACTIVE);
@@ -1406,10 +1411,10 @@ static noinline int free_debug_processing(
out:
if (cnt != bulk_cnt)
- slab_err(s, page, "Bulk freelist count(%d) invalid(%d)\n",
+ slab_err(s, slab, "Bulk freelist count(%d) invalid(%d)\n",
bulk_cnt, cnt);
- slab_unlock(page, &flags2);
+ slab_unlock(slab, &flags2);
spin_unlock_irqrestore(&n->list_lock, flags);
if (!ret)
slab_fix(s, "Object at 0x%p not freed", object);
@@ -1624,26 +1629,26 @@ slab_flags_t kmem_cache_flags(unsigned int object_size,
}
#else /* !CONFIG_SLUB_DEBUG */
static inline void setup_object_debug(struct kmem_cache *s,
- struct page *page, void *object) {}
+ struct slab *slab, void *object) {}
static inline
-void setup_page_debug(struct kmem_cache *s, struct page *page, void *addr) {}
+void setup_slab_debug(struct kmem_cache *s, struct slab *slab, void *addr) {}
static inline int alloc_debug_processing(struct kmem_cache *s,
- struct page *page, void *object, unsigned long addr) { return 0; }
+ struct slab *slab, void *object, unsigned long addr) { return 0; }
static inline int free_debug_processing(
- struct kmem_cache *s, struct page *page,
+ struct kmem_cache *s, struct slab *slab,
void *head, void *tail, int bulk_cnt,
unsigned long addr) { return 0; }
-static inline int slab_pad_check(struct kmem_cache *s, struct page *page)
+static inline int slab_pad_check(struct kmem_cache *s, struct slab *slab)
{ return 1; }
-static inline int check_object(struct kmem_cache *s, struct page *page,
+static inline int check_object(struct kmem_cache *s, struct slab *slab,
void *object, u8 val) { return 1; }
static inline void add_full(struct kmem_cache *s, struct kmem_cache_node *n,
- struct page *page) {}
+ struct slab *slab) {}
static inline void remove_full(struct kmem_cache *s, struct kmem_cache_node *n,
- struct page *page) {}
+ struct slab *slab) {}
slab_flags_t kmem_cache_flags(unsigned int object_size,
slab_flags_t flags, const char *name)
{
@@ -1662,7 +1667,7 @@ static inline void inc_slabs_node(struct kmem_cache *s, int node,
static inline void dec_slabs_node(struct kmem_cache *s, int node,
int objects) {}
-static bool freelist_corrupted(struct kmem_cache *s, struct page *page,
+static bool freelist_corrupted(struct kmem_cache *s, struct slab *slab,
void **freelist, void *nextfree)
{
return false;
@@ -1767,10 +1772,10 @@ static inline bool slab_free_freelist_hook(struct kmem_cache *s,
return *head != NULL;
}
-static void *setup_object(struct kmem_cache *s, struct page *page,
+static void *setup_object(struct kmem_cache *s, struct slab *slab,
void *object)
{
- setup_object_debug(s, page, object);
+ setup_object_debug(s, slab, object);
object = kasan_init_slab_obj(s, object);
if (unlikely(s->ctor)) {
kasan_unpoison_object_data(s, object);
@@ -1783,18 +1788,27 @@ static void *setup_object(struct kmem_cache *s, struct page *page,
/*
* Slab allocation and freeing
*/
-static inline struct page *alloc_slab_page(struct kmem_cache *s,
+static inline struct slab *alloc_slab_page(struct kmem_cache *s,
gfp_t flags, int node, struct kmem_cache_order_objects oo)
{
- struct page *page;
+ struct folio *folio;
+ struct slab *slab;
unsigned int order = oo_order(oo);
if (node == NUMA_NO_NODE)
- page = alloc_pages(flags, order);
+ folio = (struct folio *)alloc_pages(flags, order);
else
- page = __alloc_pages_node(node, flags, order);
+ folio = (struct folio *)__alloc_pages_node(node, flags, order);
- return page;
+ if (!folio)
+ return NULL;
+
+ slab = folio_slab(folio);
+ __folio_set_slab(folio);
+ if (page_is_pfmemalloc(folio_page(folio, 0)))
+ slab_set_pfmemalloc(slab);
+
+ return slab;
}
#ifdef CONFIG_SLAB_FREELIST_RANDOM
@@ -1839,7 +1853,7 @@ static void __init init_freelist_randomization(void)
}
/* Get the next entry on the pre-computed freelist randomized */
-static void *next_freelist_entry(struct kmem_cache *s, struct page *page,
+static void *next_freelist_entry(struct kmem_cache *s, struct slab *slab,
unsigned long *pos, void *start,
unsigned long page_limit,
unsigned long freelist_count)
@@ -1861,32 +1875,32 @@ static void *next_freelist_entry(struct kmem_cache *s, struct page *page,
}
/* Shuffle the single linked freelist based on a random pre-computed sequence */
-static bool shuffle_freelist(struct kmem_cache *s, struct page *page)
+static bool shuffle_freelist(struct kmem_cache *s, struct slab *slab)
{
void *start;
void *cur;
void *next;
unsigned long idx, pos, page_limit, freelist_count;
- if (page->objects < 2 || !s->random_seq)
+ if (slab->objects < 2 || !s->random_seq)
return false;
freelist_count = oo_objects(s->oo);
pos = get_random_int() % freelist_count;
- page_limit = page->objects * s->size;
- start = fixup_red_left(s, page_address(page));
+ page_limit = slab->objects * s->size;
+ start = fixup_red_left(s, slab_address(slab));
/* First entry is used as the base of the freelist */
- cur = next_freelist_entry(s, page, &pos, start, page_limit,
+ cur = next_freelist_entry(s, slab, &pos, start, page_limit,
freelist_count);
- cur = setup_object(s, page, cur);
- page->freelist = cur;
+ cur = setup_object(s, slab, cur);
+ slab->freelist = cur;
- for (idx = 1; idx < page->objects; idx++) {
- next = next_freelist_entry(s, page, &pos, start, page_limit,
+ for (idx = 1; idx < slab->objects; idx++) {
+ next = next_freelist_entry(s, slab, &pos, start, page_limit,
freelist_count);
- next = setup_object(s, page, next);
+ next = setup_object(s, slab, next);
set_freepointer(s, cur, next);
cur = next;
}
@@ -1900,15 +1914,15 @@ static inline int init_cache_random_seq(struct kmem_cache *s)
return 0;
}
static inline void init_freelist_randomization(void) { }
-static inline bool shuffle_freelist(struct kmem_cache *s, struct page *page)
+static inline bool shuffle_freelist(struct kmem_cache *s, struct slab *slab)
{
return false;
}
#endif /* CONFIG_SLAB_FREELIST_RANDOM */
-static struct page *allocate_slab(struct kmem_cache *s, gfp_t flags, int node)
+static struct slab *allocate_slab(struct kmem_cache *s, gfp_t flags, int node)
{
- struct page *page;
+ struct slab *slab;
struct kmem_cache_order_objects oo = s->oo;
gfp_t alloc_gfp;
void *start, *p, *next;
@@ -1927,63 +1941,60 @@ static struct page *allocate_slab(struct kmem_cache *s, gfp_t flags, int node)
if ((alloc_gfp & __GFP_DIRECT_RECLAIM) && oo_order(oo) > oo_order(s->min))
alloc_gfp = (alloc_gfp | __GFP_NOMEMALLOC) & ~(__GFP_RECLAIM|__GFP_NOFAIL);
- page = alloc_slab_page(s, alloc_gfp, node, oo);
- if (unlikely(!page)) {
+ slab = alloc_slab_page(s, alloc_gfp, node, oo);
+ if (unlikely(!slab)) {
oo = s->min;
alloc_gfp = flags;
/*
* Allocation may have failed due to fragmentation.
* Try a lower order alloc if possible
*/
- page = alloc_slab_page(s, alloc_gfp, node, oo);
- if (unlikely(!page))
+ slab = alloc_slab_page(s, alloc_gfp, node, oo);
+ if (unlikely(!slab))
goto out;
stat(s, ORDER_FALLBACK);
}
- page->objects = oo_objects(oo);
+ slab->objects = oo_objects(oo);
- account_slab_page(page, oo_order(oo), s, flags);
+ account_slab(slab, oo_order(oo), s, flags);
- page->slab_cache = s;
- __SetPageSlab(page);
- if (page_is_pfmemalloc(page))
- SetPageSlabPfmemalloc(page);
+ slab->slab_cache = s;
- kasan_poison_slab(page);
+ kasan_poison_slab(slab);
- start = page_address(page);
+ start = slab_address(slab);
- setup_page_debug(s, page, start);
+ setup_slab_debug(s, slab, start);
- shuffle = shuffle_freelist(s, page);
+ shuffle = shuffle_freelist(s, slab);
if (!shuffle) {
start = fixup_red_left(s, start);
- start = setup_object(s, page, start);
- page->freelist = start;
- for (idx = 0, p = start; idx < page->objects - 1; idx++) {
+ start = setup_object(s, slab, start);
+ slab->freelist = start;
+ for (idx = 0, p = start; idx < slab->objects - 1; idx++) {
next = p + s->size;
- next = setup_object(s, page, next);
+ next = setup_object(s, slab, next);
set_freepointer(s, p, next);
p = next;
}
set_freepointer(s, p, NULL);
}
- page->inuse = page->objects;
- page->frozen = 1;
+ slab->inuse = slab->objects;
+ slab->frozen = 1;
out:
- if (!page)
+ if (!slab)
return NULL;
- inc_slabs_node(s, page_to_nid(page), page->objects);
+ inc_slabs_node(s, slab_nid(slab), slab->objects);
- return page;
+ return slab;
}
-static struct page *new_slab(struct kmem_cache *s, gfp_t flags, int node)
+static struct slab *new_slab(struct kmem_cache *s, gfp_t flags, int node)
{
if (unlikely(flags & GFP_SLAB_BUG_MASK))
flags = kmalloc_fix_flags(flags);
@@ -1994,76 +2005,75 @@ static struct page *new_slab(struct kmem_cache *s, gfp_t flags, int node)
flags & (GFP_RECLAIM_MASK | GFP_CONSTRAINT_MASK), node);
}
-static void __free_slab(struct kmem_cache *s, struct page *page)
+static void __free_slab(struct kmem_cache *s, struct slab *slab)
{
- int order = compound_order(page);
+ struct folio *folio = slab_folio(slab);
+ int order = folio_order(folio);
int pages = 1 << order;
if (kmem_cache_debug_flags(s, SLAB_CONSISTENCY_CHECKS)) {
void *p;
- slab_pad_check(s, page);
- for_each_object(p, s, page_address(page),
- page->objects)
- check_object(s, page, p, SLUB_RED_INACTIVE);
+ slab_pad_check(s, slab);
+ for_each_object(p, s, slab_address(slab), slab->objects)
+ check_object(s, slab, p, SLUB_RED_INACTIVE);
}
- __ClearPageSlabPfmemalloc(page);
- __ClearPageSlab(page);
- /* In union with page->mapping where page allocator expects NULL */
- page->slab_cache = NULL;
+ __slab_clear_pfmemalloc(slab);
+ __folio_clear_slab(folio);
+ folio->mapping = NULL;
if (current->reclaim_state)
current->reclaim_state->reclaimed_slab += pages;
- unaccount_slab_page(page, order, s);
- __free_pages(page, order);
+ unaccount_slab(slab, order, s);
+ __free_pages(folio_page(folio, 0), order);
}
static void rcu_free_slab(struct rcu_head *h)
{
- struct page *page = container_of(h, struct page, rcu_head);
+ struct slab *slab = container_of(h, struct slab, rcu_head);
- __free_slab(page->slab_cache, page);
+ __free_slab(slab->slab_cache, slab);
}
-static void free_slab(struct kmem_cache *s, struct page *page)
+static void free_slab(struct kmem_cache *s, struct slab *slab)
{
if (unlikely(s->flags & SLAB_TYPESAFE_BY_RCU)) {
- call_rcu(&page->rcu_head, rcu_free_slab);
+ call_rcu(&slab->rcu_head, rcu_free_slab);
} else
- __free_slab(s, page);
+ __free_slab(s, slab);
}
-static void discard_slab(struct kmem_cache *s, struct page *page)
+static void discard_slab(struct kmem_cache *s, struct slab *slab)
{
- dec_slabs_node(s, page_to_nid(page), page->objects);
- free_slab(s, page);
+ dec_slabs_node(s, slab_nid(slab), slab->objects);
+ free_slab(s, slab);
}
/*
* Management of partially allocated slabs.
*/
static inline void
-__add_partial(struct kmem_cache_node *n, struct page *page, int tail)
+__add_partial(struct kmem_cache_node *n, struct slab *slab, int tail)
{
n->nr_partial++;
if (tail == DEACTIVATE_TO_TAIL)
- list_add_tail(&page->slab_list, &n->partial);
+ list_add_tail(&slab->slab_list, &n->partial);
else
- list_add(&page->slab_list, &n->partial);
+ list_add(&slab->slab_list, &n->partial);
}
static inline void add_partial(struct kmem_cache_node *n,
- struct page *page, int tail)
+ struct slab *slab, int tail)
{
lockdep_assert_held(&n->list_lock);
- __add_partial(n, page, tail);
+ __add_partial(n, slab, tail);
}
static inline void remove_partial(struct kmem_cache_node *n,
- struct page *page)
+ struct slab *slab)
{
lockdep_assert_held(&n->list_lock);
- list_del(&page->slab_list);
+ list_del(&slab->slab_list);
n->nr_partial--;
}
@@ -2074,12 +2084,12 @@ static inline void remove_partial(struct kmem_cache_node *n,
* Returns a list of objects or NULL if it fails.
*/
static inline void *acquire_slab(struct kmem_cache *s,
- struct kmem_cache_node *n, struct page *page,
+ struct kmem_cache_node *n, struct slab *slab,
int mode)
{
void *freelist;
unsigned long counters;
- struct page new;
+ struct slab new;
lockdep_assert_held(&n->list_lock);
@@ -2088,11 +2098,11 @@ static inline void *acquire_slab(struct kmem_cache *s,
* The old freelist is the list of objects for the
* per cpu allocation list.
*/
- freelist = page->freelist;
- counters = page->counters;
+ freelist = slab->freelist;
+ counters = slab->counters;
new.counters = counters;
if (mode) {
- new.inuse = page->objects;
+ new.inuse = slab->objects;
new.freelist = NULL;
} else {
new.freelist = freelist;
@@ -2101,35 +2111,35 @@ static inline void *acquire_slab(struct kmem_cache *s,
VM_BUG_ON(new.frozen);
new.frozen = 1;
- if (!__cmpxchg_double_slab(s, page,
+ if (!__cmpxchg_double_slab(s, slab,
freelist, counters,
new.freelist, new.counters,
"acquire_slab"))
return NULL;
- remove_partial(n, page);
+ remove_partial(n, slab);
WARN_ON(!freelist);
return freelist;
}
#ifdef CONFIG_SLUB_CPU_PARTIAL
-static void put_cpu_partial(struct kmem_cache *s, struct page *page, int drain);
+static void put_cpu_partial(struct kmem_cache *s, struct slab *slab, int drain);
#else
-static inline void put_cpu_partial(struct kmem_cache *s, struct page *page,
+static inline void put_cpu_partial(struct kmem_cache *s, struct slab *slab,
int drain) { }
#endif
-static inline bool pfmemalloc_match(struct page *page, gfp_t gfpflags);
+static inline bool pfmemalloc_match(struct slab *slab, gfp_t gfpflags);
/*
* Try to allocate a partial slab from a specific node.
*/
static void *get_partial_node(struct kmem_cache *s, struct kmem_cache_node *n,
- struct page **ret_page, gfp_t gfpflags)
+ struct slab **ret_slab, gfp_t gfpflags)
{
- struct page *page, *page2;
+ struct slab *slab, *slab2;
void *object = NULL;
unsigned long flags;
- unsigned int partial_pages = 0;
+ unsigned int partial_slabs = 0;
/*
* Racy check. If we mistakenly see no partial slabs then we
@@ -2141,28 +2151,28 @@ static void *get_partial_node(struct kmem_cache *s, struct kmem_cache_node *n,
return NULL;
spin_lock_irqsave(&n->list_lock, flags);
- list_for_each_entry_safe(page, page2, &n->partial, slab_list) {
+ list_for_each_entry_safe(slab, slab2, &n->partial, slab_list) {
void *t;
- if (!pfmemalloc_match(page, gfpflags))
+ if (!pfmemalloc_match(slab, gfpflags))
continue;
- t = acquire_slab(s, n, page, object == NULL);
+ t = acquire_slab(s, n, slab, object == NULL);
if (!t)
break;
if (!object) {
- *ret_page = page;
+ *ret_slab = slab;
stat(s, ALLOC_FROM_PARTIAL);
object = t;
} else {
- put_cpu_partial(s, page, 0);
+ put_cpu_partial(s, slab, 0);
stat(s, CPU_PARTIAL_NODE);
- partial_pages++;
+ partial_slabs++;
}
#ifdef CONFIG_SLUB_CPU_PARTIAL
if (!kmem_cache_has_cpu_partial(s)
- || partial_pages > s->cpu_partial_pages / 2)
+ || partial_slabs > s->cpu_partial_slabs / 2)
break;
#else
break;
@@ -2174,10 +2184,10 @@ static void *get_partial_node(struct kmem_cache *s, struct kmem_cache_node *n,
}
/*
- * Get a page from somewhere. Search in increasing NUMA distances.
+ * Get a slab from somewhere. Search in increasing NUMA distances.
*/
static void *get_any_partial(struct kmem_cache *s, gfp_t flags,
- struct page **ret_page)
+ struct slab **ret_slab)
{
#ifdef CONFIG_NUMA
struct zonelist *zonelist;
@@ -2219,7 +2229,7 @@ static void *get_any_partial(struct kmem_cache *s, gfp_t flags,
if (n && cpuset_zone_allowed(zone, flags) &&
n->nr_partial > s->min_partial) {
- object = get_partial_node(s, n, ret_page, flags);
+ object = get_partial_node(s, n, ret_slab, flags);
if (object) {
/*
* Don't check read_mems_allowed_retry()
@@ -2238,10 +2248,10 @@ static void *get_any_partial(struct kmem_cache *s, gfp_t flags,
}
/*
- * Get a partial page, lock it and return it.
+ * Get a partial slab, lock it and return it.
*/
static void *get_partial(struct kmem_cache *s, gfp_t flags, int node,
- struct page **ret_page)
+ struct slab **ret_slab)
{
void *object;
int searchnode = node;
@@ -2249,11 +2259,11 @@ static void *get_partial(struct kmem_cache *s, gfp_t flags, int node,
if (node == NUMA_NO_NODE)
searchnode = numa_mem_id();
- object = get_partial_node(s, get_node(s, searchnode), ret_page, flags);
+ object = get_partial_node(s, get_node(s, searchnode), ret_slab, flags);
if (object || node != NUMA_NO_NODE)
return object;
- return get_any_partial(s, flags, ret_page);
+ return get_any_partial(s, flags, ret_slab);
}
#ifdef CONFIG_PREEMPTION
@@ -2330,25 +2340,25 @@ static void init_kmem_cache_cpus(struct kmem_cache *s)
}
/*
- * Finishes removing the cpu slab. Merges cpu's freelist with page's freelist,
+ * Finishes removing the cpu slab. Merges cpu's freelist with slab's freelist,
* unfreezes the slabs and puts it on the proper list.
* Assumes the slab has been already safely taken away from kmem_cache_cpu
* by the caller.
*/
-static void deactivate_slab(struct kmem_cache *s, struct page *page,
+static void deactivate_slab(struct kmem_cache *s, struct slab *slab,
void *freelist)
{
enum slab_modes { M_NONE, M_PARTIAL, M_FULL, M_FREE };
- struct kmem_cache_node *n = get_node(s, page_to_nid(page));
+ struct kmem_cache_node *n = get_node(s, slab_nid(slab));
int lock = 0, free_delta = 0;
enum slab_modes l = M_NONE, m = M_NONE;
void *nextfree, *freelist_iter, *freelist_tail;
int tail = DEACTIVATE_TO_HEAD;
unsigned long flags = 0;
- struct page new;
- struct page old;
+ struct slab new;
+ struct slab old;
- if (page->freelist) {
+ if (slab->freelist) {
stat(s, DEACTIVATE_REMOTE_FREES);
tail = DEACTIVATE_TO_TAIL;
}
@@ -2367,7 +2377,7 @@ static void deactivate_slab(struct kmem_cache *s, struct page *page,
* 'freelist_iter' is already corrupted. So isolate all objects
* starting at 'freelist_iter' by skipping them.
*/
- if (freelist_corrupted(s, page, &freelist_iter, nextfree))
+ if (freelist_corrupted(s, slab, &freelist_iter, nextfree))
break;
freelist_tail = freelist_iter;
@@ -2377,25 +2387,25 @@ static void deactivate_slab(struct kmem_cache *s, struct page *page,
}
/*
- * Stage two: Unfreeze the page while splicing the per-cpu
- * freelist to the head of page's freelist.
+ * Stage two: Unfreeze the slab while splicing the per-cpu
+ * freelist to the head of slab's freelist.
*
- * Ensure that the page is unfrozen while the list presence
+ * Ensure that the slab is unfrozen while the list presence
* reflects the actual number of objects during unfreeze.
*
* We setup the list membership and then perform a cmpxchg
- * with the count. If there is a mismatch then the page
- * is not unfrozen but the page is on the wrong list.
+ * with the count. If there is a mismatch then the slab
+ * is not unfrozen but the slab is on the wrong list.
*
* Then we restart the process which may have to remove
- * the page from the list that we just put it on again
+ * the slab from the list that we just put it on again
* because the number of objects in the slab may have
* changed.
*/
redo:
- old.freelist = READ_ONCE(page->freelist);
- old.counters = READ_ONCE(page->counters);
+ old.freelist = READ_ONCE(slab->freelist);
+ old.counters = READ_ONCE(slab->counters);
VM_BUG_ON(!old.frozen);
/* Determine target state of the slab */
@@ -2416,9 +2426,8 @@ static void deactivate_slab(struct kmem_cache *s, struct page *page,
if (!lock) {
lock = 1;
/*
- * Taking the spinlock removes the possibility
- * that acquire_slab() will see a slab page that
- * is frozen
+ * Taking the spinlock removes the possibility that
+ * acquire_slab() will see a slab that is frozen
*/
spin_lock_irqsave(&n->list_lock, flags);
}
@@ -2437,18 +2446,18 @@ static void deactivate_slab(struct kmem_cache *s, struct page *page,
if (l != m) {
if (l == M_PARTIAL)
- remove_partial(n, page);
+ remove_partial(n, slab);
else if (l == M_FULL)
- remove_full(s, n, page);
+ remove_full(s, n, slab);
if (m == M_PARTIAL)
- add_partial(n, page, tail);
+ add_partial(n, slab, tail);
else if (m == M_FULL)
- add_full(s, n, page);
+ add_full(s, n, slab);
}
l = m;
- if (!cmpxchg_double_slab(s, page,
+ if (!cmpxchg_double_slab(s, slab,
old.freelist, old.counters,
new.freelist, new.counters,
"unfreezing slab"))
@@ -2463,26 +2472,26 @@ static void deactivate_slab(struct kmem_cache *s, struct page *page,
stat(s, DEACTIVATE_FULL);
else if (m == M_FREE) {
stat(s, DEACTIVATE_EMPTY);
- discard_slab(s, page);
+ discard_slab(s, slab);
stat(s, FREE_SLAB);
}
}
#ifdef CONFIG_SLUB_CPU_PARTIAL
-static void __unfreeze_partials(struct kmem_cache *s, struct page *partial_page)
+static void __unfreeze_partials(struct kmem_cache *s, struct slab *partial_slab)
{
struct kmem_cache_node *n = NULL, *n2 = NULL;
- struct page *page, *discard_page = NULL;
+ struct slab *slab, *slab_to_discard = NULL;
unsigned long flags = 0;
- while (partial_page) {
- struct page new;
- struct page old;
+ while (partial_slab) {
+ struct slab new;
+ struct slab old;
- page = partial_page;
- partial_page = page->next;
+ slab = partial_slab;
+ partial_slab = slab->next;
- n2 = get_node(s, page_to_nid(page));
+ n2 = get_node(s, slab_nid(slab));
if (n != n2) {
if (n)
spin_unlock_irqrestore(&n->list_lock, flags);
@@ -2493,8 +2502,8 @@ static void __unfreeze_partials(struct kmem_cache *s, struct page *partial_page)
do {
- old.freelist = page->freelist;
- old.counters = page->counters;
+ old.freelist = slab->freelist;
+ old.counters = slab->counters;
VM_BUG_ON(!old.frozen);
new.counters = old.counters;
@@ -2502,16 +2511,16 @@ static void __unfreeze_partials(struct kmem_cache *s, struct page *partial_page)
new.frozen = 0;
- } while (!__cmpxchg_double_slab(s, page,
+ } while (!__cmpxchg_double_slab(s, slab,
old.freelist, old.counters,
new.freelist, new.counters,
"unfreezing slab"));
if (unlikely(!new.inuse && n->nr_partial >= s->min_partial)) {
- page->next = discard_page;
- discard_page = page;
+ slab->next = slab_to_discard;
+ slab_to_discard = slab;
} else {
- add_partial(n, page, DEACTIVATE_TO_TAIL);
+ add_partial(n, slab, DEACTIVATE_TO_TAIL);
stat(s, FREE_ADD_PARTIAL);
}
}
@@ -2519,12 +2528,12 @@ static void __unfreeze_partials(struct kmem_cache *s, struct page *partial_page)
if (n)
spin_unlock_irqrestore(&n->list_lock, flags);
- while (discard_page) {
- page = discard_page;
- discard_page = discard_page->next;
+ while (slab_to_discard) {
+ slab = slab_to_discard;
+ slab_to_discard = slab_to_discard->next;
stat(s, DEACTIVATE_EMPTY);
- discard_slab(s, page);
+ discard_slab(s, slab);
stat(s, FREE_SLAB);
}
}
@@ -2534,73 +2543,73 @@ static void __unfreeze_partials(struct kmem_cache *s, struct page *partial_page)
*/
static void unfreeze_partials(struct kmem_cache *s)
{
- struct page *partial_page;
+ struct slab *partial_slab;
unsigned long flags;
local_lock_irqsave(&s->cpu_slab->lock, flags);
- partial_page = this_cpu_read(s->cpu_slab->partial);
+ partial_slab = this_cpu_read(s->cpu_slab->partial);
this_cpu_write(s->cpu_slab->partial, NULL);
local_unlock_irqrestore(&s->cpu_slab->lock, flags);
- if (partial_page)
- __unfreeze_partials(s, partial_page);
+ if (partial_slab)
+ __unfreeze_partials(s, partial_slab);
}
static void unfreeze_partials_cpu(struct kmem_cache *s,
struct kmem_cache_cpu *c)
{
- struct page *partial_page;
+ struct slab *partial_slab;
- partial_page = slub_percpu_partial(c);
+ partial_slab = slub_percpu_partial(c);
c->partial = NULL;
- if (partial_page)
- __unfreeze_partials(s, partial_page);
+ if (partial_slab)
+ __unfreeze_partials(s, partial_slab);
}
/*
- * Put a page that was just frozen (in __slab_free|get_partial_node) into a
- * partial page slot if available.
+ * Put a slab that was just frozen (in __slab_free|get_partial_node) into a
+ * partial slab slot if available.
*
* If we did not find a slot then simply move all the partials to the
* per node partial list.
*/
-static void put_cpu_partial(struct kmem_cache *s, struct page *page, int drain)
+static void put_cpu_partial(struct kmem_cache *s, struct slab *slab, int drain)
{
- struct page *oldpage;
- struct page *page_to_unfreeze = NULL;
+ struct slab *oldslab;
+ struct slab *slab_to_unfreeze = NULL;
unsigned long flags;
- int pages = 0;
+ int slabs = 0;
local_lock_irqsave(&s->cpu_slab->lock, flags);
- oldpage = this_cpu_read(s->cpu_slab->partial);
+ oldslab = this_cpu_read(s->cpu_slab->partial);
- if (oldpage) {
- if (drain && oldpage->pages >= s->cpu_partial_pages) {
+ if (oldslab) {
+ if (drain && oldslab->slabs >= s->cpu_partial_slabs) {
/*
* Partial array is full. Move the existing set to the
* per node partial list. Postpone the actual unfreezing
* outside of the critical section.
*/
- page_to_unfreeze = oldpage;
- oldpage = NULL;
+ slab_to_unfreeze = oldslab;
+ oldslab = NULL;
} else {
- pages = oldpage->pages;
+ slabs = oldslab->slabs;
}
}
- pages++;
+ slabs++;
- page->pages = pages;
- page->next = oldpage;
+ slab->slabs = slabs;
+ slab->next = oldslab;
- this_cpu_write(s->cpu_slab->partial, page);
+ this_cpu_write(s->cpu_slab->partial, slab);
local_unlock_irqrestore(&s->cpu_slab->lock, flags);
- if (page_to_unfreeze) {
- __unfreeze_partials(s, page_to_unfreeze);
+ if (slab_to_unfreeze) {
+ __unfreeze_partials(s, slab_to_unfreeze);
stat(s, CPU_PARTIAL_DRAIN);
}
}
@@ -2616,22 +2625,22 @@ static inline void unfreeze_partials_cpu(struct kmem_cache *s,
static inline void flush_slab(struct kmem_cache *s, struct kmem_cache_cpu *c)
{
unsigned long flags;
- struct page *page;
+ struct slab *slab;
void *freelist;
local_lock_irqsave(&s->cpu_slab->lock, flags);
- page = c->page;
+ slab = c->slab;
freelist = c->freelist;
- c->page = NULL;
+ c->slab = NULL;
c->freelist = NULL;
c->tid = next_tid(c->tid);
local_unlock_irqrestore(&s->cpu_slab->lock, flags);
- if (page) {
- deactivate_slab(s, page, freelist);
+ if (slab) {
+ deactivate_slab(s, slab, freelist);
stat(s, CPUSLAB_FLUSH);
}
}
@@ -2640,14 +2649,14 @@ static inline void __flush_cpu_slab(struct kmem_cache *s, int cpu)
{
struct kmem_cache_cpu *c = per_cpu_ptr(s->cpu_slab, cpu);
void *freelist = c->freelist;
- struct page *page = c->page;
+ struct slab *slab = c->slab;
- c->page = NULL;
+ c->slab = NULL;
c->freelist = NULL;
c->tid = next_tid(c->tid);
- if (page) {
- deactivate_slab(s, page, freelist);
+ if (slab) {
+ deactivate_slab(s, slab, freelist);
stat(s, CPUSLAB_FLUSH);
}
@@ -2676,7 +2685,7 @@ static void flush_cpu_slab(struct work_struct *w)
s = sfw->s;
c = this_cpu_ptr(s->cpu_slab);
- if (c->page)
+ if (c->slab)
flush_slab(s, c);
unfreeze_partials(s);
@@ -2686,7 +2695,7 @@ static bool has_cpu_slab(int cpu, struct kmem_cache *s)
{
struct kmem_cache_cpu *c = per_cpu_ptr(s->cpu_slab, cpu);
- return c->page || slub_percpu_partial(c);
+ return c->slab || slub_percpu_partial(c);
}
static DEFINE_MUTEX(flush_lock);
@@ -2748,19 +2757,19 @@ static int slub_cpu_dead(unsigned int cpu)
* Check if the objects in a per cpu structure fit numa
* locality expectations.
*/
-static inline int node_match(struct page *page, int node)
+static inline int node_match(struct slab *slab, int node)
{
#ifdef CONFIG_NUMA
- if (node != NUMA_NO_NODE && page_to_nid(page) != node)
+ if (node != NUMA_NO_NODE && slab_nid(slab) != node)
return 0;
#endif
return 1;
}
#ifdef CONFIG_SLUB_DEBUG
-static int count_free(struct page *page)
+static int count_free(struct slab *slab)
{
- return page->objects - page->inuse;
+ return slab->objects - slab->inuse;
}
static inline unsigned long node_nr_objs(struct kmem_cache_node *n)
@@ -2771,15 +2780,15 @@ static inline unsigned long node_nr_objs(struct kmem_cache_node *n)
#if defined(CONFIG_SLUB_DEBUG) || defined(CONFIG_SYSFS)
static unsigned long count_partial(struct kmem_cache_node *n,
- int (*get_count)(struct page *))
+ int (*get_count)(struct slab *))
{
unsigned long flags;
unsigned long x = 0;
- struct page *page;
+ struct slab *slab;
spin_lock_irqsave(&n->list_lock, flags);
- list_for_each_entry(page, &n->partial, slab_list)
- x += get_count(page);
+ list_for_each_entry(slab, &n->partial, slab_list)
+ x += get_count(slab);
spin_unlock_irqrestore(&n->list_lock, flags);
return x;
}
@@ -2822,54 +2831,41 @@ slab_out_of_memory(struct kmem_cache *s, gfp_t gfpflags, int nid)
#endif
}
-static inline bool pfmemalloc_match(struct page *page, gfp_t gfpflags)
+static inline bool pfmemalloc_match(struct slab *slab, gfp_t gfpflags)
{
- if (unlikely(PageSlabPfmemalloc(page)))
+ if (unlikely(slab_test_pfmemalloc(slab)))
return gfp_pfmemalloc_allowed(gfpflags);
return true;
}
/*
- * A variant of pfmemalloc_match() that tests page flags without asserting
- * PageSlab. Intended for opportunistic checks before taking a lock and
- * rechecking that nobody else freed the page under us.
- */
-static inline bool pfmemalloc_match_unsafe(struct page *page, gfp_t gfpflags)
-{
- if (unlikely(__PageSlabPfmemalloc(page)))
- return gfp_pfmemalloc_allowed(gfpflags);
-
- return true;
-}
-
-/*
- * Check the page->freelist of a page and either transfer the freelist to the
- * per cpu freelist or deactivate the page.
+ * Check the slab->freelist and either transfer the freelist to the
+ * per cpu freelist or deactivate the slab.
*
- * The page is still frozen if the return value is not NULL.
+ * The slab is still frozen if the return value is not NULL.
*
- * If this function returns NULL then the page has been unfrozen.
+ * If this function returns NULL then the slab has been unfrozen.
*/
-static inline void *get_freelist(struct kmem_cache *s, struct page *page)
+static inline void *get_freelist(struct kmem_cache *s, struct slab *slab)
{
- struct page new;
+ struct slab new;
unsigned long counters;
void *freelist;
lockdep_assert_held(this_cpu_ptr(&s->cpu_slab->lock));
do {
- freelist = page->freelist;
- counters = page->counters;
+ freelist = slab->freelist;
+ counters = slab->counters;
new.counters = counters;
VM_BUG_ON(!new.frozen);
- new.inuse = page->objects;
+ new.inuse = slab->objects;
new.frozen = freelist != NULL;
- } while (!__cmpxchg_double_slab(s, page,
+ } while (!__cmpxchg_double_slab(s, slab,
freelist, counters,
NULL, new.counters,
"get_freelist"));
@@ -2900,15 +2896,15 @@ static void *___slab_alloc(struct kmem_cache *s, gfp_t gfpflags, int node,
unsigned long addr, struct kmem_cache_cpu *c)
{
void *freelist;
- struct page *page;
+ struct slab *slab;
unsigned long flags;
stat(s, ALLOC_SLOWPATH);
-reread_page:
+reread_slab:
- page = READ_ONCE(c->page);
- if (!page) {
+ slab = READ_ONCE(c->slab);
+ if (!slab) {
/*
* if the node is not online or has no normal memory, just
* ignore the node constraint
@@ -2920,7 +2916,7 @@ static void *___slab_alloc(struct kmem_cache *s, gfp_t gfpflags, int node,
}
redo:
- if (unlikely(!node_match(page, node))) {
+ if (unlikely(!node_match(slab, node))) {
/*
* same as above but node_match() being false already
* implies node != NUMA_NO_NODE
@@ -2939,23 +2935,23 @@ static void *___slab_alloc(struct kmem_cache *s, gfp_t gfpflags, int node,
* PFMEMALLOC but right now, we are losing the pfmemalloc
* information when the page leaves the per-cpu allocator
*/
- if (unlikely(!pfmemalloc_match_unsafe(page, gfpflags)))
+ if (unlikely(!pfmemalloc_match(slab, gfpflags)))
goto deactivate_slab;
- /* must check again c->page in case we got preempted and it changed */
+ /* must check again c->slab in case we got preempted and it changed */
local_lock_irqsave(&s->cpu_slab->lock, flags);
- if (unlikely(page != c->page)) {
+ if (unlikely(slab != c->slab)) {
local_unlock_irqrestore(&s->cpu_slab->lock, flags);
- goto reread_page;
+ goto reread_slab;
}
freelist = c->freelist;
if (freelist)
goto load_freelist;
- freelist = get_freelist(s, page);
+ freelist = get_freelist(s, slab);
if (!freelist) {
- c->page = NULL;
+ c->slab = NULL;
local_unlock_irqrestore(&s->cpu_slab->lock, flags);
stat(s, DEACTIVATE_BYPASS);
goto new_slab;
@@ -2969,10 +2965,10 @@ static void *___slab_alloc(struct kmem_cache *s, gfp_t gfpflags, int node,
/*
* freelist is pointing to the list of objects to be used.
- * page is pointing to the page from which the objects are obtained.
- * That page must be frozen for per cpu allocations to work.
+ * slab is pointing to the slab from which the objects are obtained.
+ * That slab must be frozen for per cpu allocations to work.
*/
- VM_BUG_ON(!c->page->frozen);
+ VM_BUG_ON(!c->slab->frozen);
c->freelist = get_freepointer(s, freelist);
c->tid = next_tid(c->tid);
local_unlock_irqrestore(&s->cpu_slab->lock, flags);
@@ -2981,23 +2977,23 @@ static void *___slab_alloc(struct kmem_cache *s, gfp_t gfpflags, int node,
deactivate_slab:
local_lock_irqsave(&s->cpu_slab->lock, flags);
- if (page != c->page) {
+ if (slab != c->slab) {
local_unlock_irqrestore(&s->cpu_slab->lock, flags);
- goto reread_page;
+ goto reread_slab;
}
freelist = c->freelist;
- c->page = NULL;
+ c->slab = NULL;
c->freelist = NULL;
local_unlock_irqrestore(&s->cpu_slab->lock, flags);
- deactivate_slab(s, page, freelist);
+ deactivate_slab(s, slab, freelist);
new_slab:
if (slub_percpu_partial(c)) {
local_lock_irqsave(&s->cpu_slab->lock, flags);
- if (unlikely(c->page)) {
+ if (unlikely(c->slab)) {
local_unlock_irqrestore(&s->cpu_slab->lock, flags);
- goto reread_page;
+ goto reread_slab;
}
if (unlikely(!slub_percpu_partial(c))) {
local_unlock_irqrestore(&s->cpu_slab->lock, flags);
@@ -3005,8 +3001,8 @@ static void *___slab_alloc(struct kmem_cache *s, gfp_t gfpflags, int node,
goto new_objects;
}
- page = c->page = slub_percpu_partial(c);
- slub_set_percpu_partial(c, page);
+ slab = c->slab = slub_percpu_partial(c);
+ slub_set_percpu_partial(c, slab);
local_unlock_irqrestore(&s->cpu_slab->lock, flags);
stat(s, CPU_PARTIAL_ALLOC);
goto redo;
@@ -3014,32 +3010,32 @@ static void *___slab_alloc(struct kmem_cache *s, gfp_t gfpflags, int node,
new_objects:
- freelist = get_partial(s, gfpflags, node, &page);
+ freelist = get_partial(s, gfpflags, node, &slab);
if (freelist)
- goto check_new_page;
+ goto check_new_slab;
slub_put_cpu_ptr(s->cpu_slab);
- page = new_slab(s, gfpflags, node);
+ slab = new_slab(s, gfpflags, node);
c = slub_get_cpu_ptr(s->cpu_slab);
- if (unlikely(!page)) {
+ if (unlikely(!slab)) {
slab_out_of_memory(s, gfpflags, node);
return NULL;
}
/*
- * No other reference to the page yet so we can
+ * No other reference to the slab yet so we can
* muck around with it freely without cmpxchg
*/
- freelist = page->freelist;
- page->freelist = NULL;
+ freelist = slab->freelist;
+ slab->freelist = NULL;
stat(s, ALLOC_SLAB);
-check_new_page:
+check_new_slab:
if (kmem_cache_debug(s)) {
- if (!alloc_debug_processing(s, page, freelist, addr)) {
+ if (!alloc_debug_processing(s, slab, freelist, addr)) {
/* Slab failed checks. Next slab needed */
goto new_slab;
} else {
@@ -3051,39 +3047,39 @@ static void *___slab_alloc(struct kmem_cache *s, gfp_t gfpflags, int node,
}
}
- if (unlikely(!pfmemalloc_match(page, gfpflags)))
+ if (unlikely(!pfmemalloc_match(slab, gfpflags)))
/*
* For !pfmemalloc_match() case we don't load freelist so that
* we don't make further mismatched allocations easier.
*/
goto return_single;
-retry_load_page:
+retry_load_slab:
local_lock_irqsave(&s->cpu_slab->lock, flags);
- if (unlikely(c->page)) {
+ if (unlikely(c->slab)) {
void *flush_freelist = c->freelist;
- struct page *flush_page = c->page;
+ struct slab *flush_slab = c->slab;
- c->page = NULL;
+ c->slab = NULL;
c->freelist = NULL;
c->tid = next_tid(c->tid);
local_unlock_irqrestore(&s->cpu_slab->lock, flags);
- deactivate_slab(s, flush_page, flush_freelist);
+ deactivate_slab(s, flush_slab, flush_freelist);
stat(s, CPUSLAB_FLUSH);
- goto retry_load_page;
+ goto retry_load_slab;
}
- c->page = page;
+ c->slab = slab;
goto load_freelist;
return_single:
- deactivate_slab(s, page, get_freepointer(s, freelist));
+ deactivate_slab(s, slab, get_freepointer(s, freelist));
return freelist;
}
@@ -3140,7 +3136,7 @@ static __always_inline void *slab_alloc_node(struct kmem_cache *s,
{
void *object;
struct kmem_cache_cpu *c;
- struct page *page;
+ struct slab *slab;
unsigned long tid;
struct obj_cgroup *objcg = NULL;
bool init = false;
@@ -3172,9 +3168,9 @@ static __always_inline void *slab_alloc_node(struct kmem_cache *s,
/*
* Irqless object alloc/free algorithm used here depends on sequence
* of fetching cpu_slab's data. tid should be fetched before anything
- * on c to guarantee that object and page associated with previous tid
+ * on c to guarantee that object and slab associated with previous tid
* won't be used with current tid. If we fetch tid first, object and
- * page could be one associated with next tid and our alloc/free
+ * slab could be one associated with next tid and our alloc/free
* request will be failed. In this case, we will retry. So, no problem.
*/
barrier();
@@ -3187,7 +3183,7 @@ static __always_inline void *slab_alloc_node(struct kmem_cache *s,
*/
object = c->freelist;
- page = c->page;
+ slab = c->slab;
/*
* We cannot use the lockless fastpath on PREEMPT_RT because if a
* slowpath has taken the local_lock_irqsave(), it is not protected
@@ -3196,7 +3192,7 @@ static __always_inline void *slab_alloc_node(struct kmem_cache *s,
* there is a suitable cpu freelist.
*/
if (IS_ENABLED(CONFIG_PREEMPT_RT) ||
- unlikely(!object || !page || !node_match(page, node))) {
+ unlikely(!object || !slab || !node_match(slab, node))) {
object = __slab_alloc(s, gfpflags, node, addr, c);
} else {
void *next_object = get_freepointer_safe(s, object);
@@ -3298,17 +3294,17 @@ EXPORT_SYMBOL(kmem_cache_alloc_node_trace);
* have a longer lifetime than the cpu slabs in most processing loads.
*
* So we still attempt to reduce cache line usage. Just take the slab
- * lock and free the item. If there is no additional partial page
+ * lock and free the item. If there is no additional partial slab
* handling required then we can return immediately.
*/
-static void __slab_free(struct kmem_cache *s, struct page *page,
+static void __slab_free(struct kmem_cache *s, struct slab *slab,
void *head, void *tail, int cnt,
unsigned long addr)
{
void *prior;
int was_frozen;
- struct page new;
+ struct slab new;
unsigned long counters;
struct kmem_cache_node *n = NULL;
unsigned long flags;
@@ -3319,7 +3315,7 @@ static void __slab_free(struct kmem_cache *s, struct page *page,
return;
if (kmem_cache_debug(s) &&
- !free_debug_processing(s, page, head, tail, cnt, addr))
+ !free_debug_processing(s, slab, head, tail, cnt, addr))
return;
do {
@@ -3327,8 +3323,8 @@ static void __slab_free(struct kmem_cache *s, struct page *page,
spin_unlock_irqrestore(&n->list_lock, flags);
n = NULL;
}
- prior = page->freelist;
- counters = page->counters;
+ prior = slab->freelist;
+ counters = slab->counters;
set_freepointer(s, tail, prior);
new.counters = counters;
was_frozen = new.frozen;
@@ -3347,7 +3343,7 @@ static void __slab_free(struct kmem_cache *s, struct page *page,
} else { /* Needs to be taken off a list */
- n = get_node(s, page_to_nid(page));
+ n = get_node(s, slab_nid(slab));
/*
* Speculatively acquire the list_lock.
* If the cmpxchg does not succeed then we may
@@ -3361,7 +3357,7 @@ static void __slab_free(struct kmem_cache *s, struct page *page,
}
}
- } while (!cmpxchg_double_slab(s, page,
+ } while (!cmpxchg_double_slab(s, slab,
prior, counters,
head, new.counters,
"__slab_free"));
@@ -3376,10 +3372,10 @@ static void __slab_free(struct kmem_cache *s, struct page *page,
stat(s, FREE_FROZEN);
} else if (new.frozen) {
/*
- * If we just froze the page then put it onto the
+ * If we just froze the slab then put it onto the
* per cpu partial list.
*/
- put_cpu_partial(s, page, 1);
+ put_cpu_partial(s, slab, 1);
stat(s, CPU_PARTIAL_FREE);
}
@@ -3394,8 +3390,8 @@ static void __slab_free(struct kmem_cache *s, struct page *page,
* then add it.
*/
if (!kmem_cache_has_cpu_partial(s) && unlikely(!prior)) {
- remove_full(s, n, page);
- add_partial(n, page, DEACTIVATE_TO_TAIL);
+ remove_full(s, n, slab);
+ add_partial(n, slab, DEACTIVATE_TO_TAIL);
stat(s, FREE_ADD_PARTIAL);
}
spin_unlock_irqrestore(&n->list_lock, flags);
@@ -3406,16 +3402,16 @@ static void __slab_free(struct kmem_cache *s, struct page *page,
/*
* Slab on the partial list.
*/
- remove_partial(n, page);
+ remove_partial(n, slab);
stat(s, FREE_REMOVE_PARTIAL);
} else {
/* Slab must be on the full list */
- remove_full(s, n, page);
+ remove_full(s, n, slab);
}
spin_unlock_irqrestore(&n->list_lock, flags);
stat(s, FREE_SLAB);
- discard_slab(s, page);
+ discard_slab(s, slab);
}
/*
@@ -3430,11 +3426,11 @@ static void __slab_free(struct kmem_cache *s, struct page *page,
* with all sorts of special processing.
*
* Bulk free of a freelist with several objects (all pointing to the
- * same page) possible by specifying head and tail ptr, plus objects
+ * same slab) possible by specifying head and tail ptr, plus objects
* count (cnt). Bulk free indicated by tail pointer being set.
*/
static __always_inline void do_slab_free(struct kmem_cache *s,
- struct page *page, void *head, void *tail,
+ struct slab *slab, void *head, void *tail,
int cnt, unsigned long addr)
{
void *tail_obj = tail ? : head;
@@ -3457,7 +3453,7 @@ static __always_inline void do_slab_free(struct kmem_cache *s,
/* Same with comment on barrier() in slab_alloc_node() */
barrier();
- if (likely(page == c->page)) {
+ if (likely(slab == c->slab)) {
#ifndef CONFIG_PREEMPT_RT
void **freelist = READ_ONCE(c->freelist);
@@ -3483,7 +3479,7 @@ static __always_inline void do_slab_free(struct kmem_cache *s,
local_lock(&s->cpu_slab->lock);
c = this_cpu_ptr(s->cpu_slab);
- if (unlikely(page != c->page)) {
+ if (unlikely(slab != c->slab)) {
local_unlock(&s->cpu_slab->lock);
goto redo;
}
@@ -3498,11 +3494,11 @@ static __always_inline void do_slab_free(struct kmem_cache *s,
#endif
stat(s, FREE_FASTPATH);
} else
- __slab_free(s, page, head, tail_obj, cnt, addr);
+ __slab_free(s, slab, head, tail_obj, cnt, addr);
}
-static __always_inline void slab_free(struct kmem_cache *s, struct page *page,
+static __always_inline void slab_free(struct kmem_cache *s, struct slab *slab,
void *head, void *tail, int cnt,
unsigned long addr)
{
@@ -3511,13 +3507,13 @@ static __always_inline void slab_free(struct kmem_cache *s, struct page *page,
* to remove objects, whose reuse must be delayed.
*/
if (slab_free_freelist_hook(s, &head, &tail, &cnt))
- do_slab_free(s, page, head, tail, cnt, addr);
+ do_slab_free(s, slab, head, tail, cnt, addr);
}
#ifdef CONFIG_KASAN_GENERIC
void ___cache_free(struct kmem_cache *cache, void *x, unsigned long addr)
{
- do_slab_free(cache, virt_to_head_page(x), x, NULL, 1, addr);
+ do_slab_free(cache, virt_to_slab(x), x, NULL, 1, addr);
}
#endif
@@ -3527,35 +3523,36 @@ void kmem_cache_free(struct kmem_cache *s, void *x)
if (!s)
return;
trace_kmem_cache_free(_RET_IP_, x, s->name);
- slab_free(s, virt_to_head_page(x), x, NULL, 1, _RET_IP_);
+ slab_free(s, virt_to_slab(x), x, NULL, 1, _RET_IP_);
}
EXPORT_SYMBOL(kmem_cache_free);
struct detached_freelist {
- struct page *page;
+ struct slab *slab;
void *tail;
void *freelist;
int cnt;
struct kmem_cache *s;
};
-static inline void free_nonslab_page(struct page *page, void *object)
+static inline void free_large_kmalloc(struct folio *folio, void *object)
{
- unsigned int order = compound_order(page);
+ unsigned int order = folio_order(folio);
- if (WARN_ON_ONCE(!PageCompound(page)))
+ if (WARN_ON_ONCE(order == 0))
pr_warn_once("object pointer: 0x%p\n", object);
kfree_hook(object);
- mod_lruvec_page_state(page, NR_SLAB_UNRECLAIMABLE_B, -(PAGE_SIZE << order));
- __free_pages(page, order);
+ mod_lruvec_page_state(folio_page(folio, 0), NR_SLAB_UNRECLAIMABLE_B,
+ -(PAGE_SIZE << order));
+ __free_pages(folio_page(folio, 0), order);
}
/*
* This function progressively scans the array with free objects (with
* a limited look ahead) and extract objects belonging to the same
- * page. It builds a detached freelist directly within the given
- * page/objects. This can happen without any need for
+ * slab. It builds a detached freelist directly within the given
+ * slab/objects. This can happen without any need for
* synchronization, because the objects are owned by running process.
* The freelist is build up as a single linked list in the objects.
* The idea is, that this detached freelist can then be bulk
@@ -3570,10 +3567,11 @@ int build_detached_freelist(struct kmem_cache *s, size_t size,
size_t first_skipped_index = 0;
int lookahead = 3;
void *object;
- struct page *page;
+ struct folio *folio;
+ struct slab *slab;
/* Always re-init detached_freelist */
- df->page = NULL;
+ df->slab = NULL;
do {
object = p[--size];
@@ -3583,17 +3581,19 @@ int build_detached_freelist(struct kmem_cache *s, size_t size,
if (!object)
return 0;
- page = virt_to_head_page(object);
+ folio = virt_to_folio(object);
if (!s) {
/* Handle kalloc'ed objects */
- if (unlikely(!PageSlab(page))) {
- free_nonslab_page(page, object);
+ if (unlikely(!folio_test_slab(folio))) {
+ free_large_kmalloc(folio, object);
p[size] = NULL; /* mark object processed */
return size;
}
/* Derive kmem_cache from object */
- df->s = page->slab_cache;
+ slab = folio_slab(folio);
+ df->s = slab->slab_cache;
} else {
+ slab = folio_slab(folio);
df->s = cache_from_obj(s, object); /* Support for memcg */
}
@@ -3605,7 +3605,7 @@ int build_detached_freelist(struct kmem_cache *s, size_t size,
}
/* Start new detached freelist */
- df->page = page;
+ df->slab = slab;
set_freepointer(df->s, object, NULL);
df->tail = object;
df->freelist = object;
@@ -3617,8 +3617,8 @@ int build_detached_freelist(struct kmem_cache *s, size_t size,
if (!object)
continue; /* Skip processed objects */
- /* df->page is always set at this point */
- if (df->page == virt_to_head_page(object)) {
+ /* df->slab is always set at this point */
+ if (df->slab == virt_to_slab(object)) {
/* Opportunity build freelist */
set_freepointer(df->s, object, df->freelist);
df->freelist = object;
@@ -3650,10 +3650,10 @@ void kmem_cache_free_bulk(struct kmem_cache *s, size_t size, void **p)
struct detached_freelist df;
size = build_detached_freelist(s, size, p, &df);
- if (!df.page)
+ if (!df.slab)
continue;
- slab_free(df.s, df.page, df.freelist, df.tail, df.cnt, _RET_IP_);
+ slab_free(df.s, df.slab, df.freelist, df.tail, df.cnt, _RET_IP_);
} while (likely(size));
}
EXPORT_SYMBOL(kmem_cache_free_bulk);
@@ -3787,7 +3787,7 @@ static unsigned int slub_min_objects;
* requested a higher minimum order then we start with that one instead of
* the smallest order which will fit the object.
*/
-static inline unsigned int slab_order(unsigned int size,
+static inline unsigned int calc_slab_order(unsigned int size,
unsigned int min_objects, unsigned int max_order,
unsigned int fract_leftover)
{
@@ -3851,7 +3851,7 @@ static inline int calculate_order(unsigned int size)
fraction = 16;
while (fraction >= 4) {
- order = slab_order(size, min_objects,
+ order = calc_slab_order(size, min_objects,
slub_max_order, fraction);
if (order <= slub_max_order)
return order;
@@ -3864,14 +3864,14 @@ static inline int calculate_order(unsigned int size)
* We were unable to place multiple objects in a slab. Now
* lets see if we can place a single object there.
*/
- order = slab_order(size, 1, slub_max_order, 1);
+ order = calc_slab_order(size, 1, slub_max_order, 1);
if (order <= slub_max_order)
return order;
/*
* Doh this slab cannot be placed using slub_max_order.
*/
- order = slab_order(size, 1, MAX_ORDER, 1);
+ order = calc_slab_order(size, 1, MAX_ORDER, 1);
if (order < MAX_ORDER)
return order;
return -ENOSYS;
@@ -3923,38 +3923,38 @@ static struct kmem_cache *kmem_cache_node;
*/
static void early_kmem_cache_node_alloc(int node)
{
- struct page *page;
+ struct slab *slab;
struct kmem_cache_node *n;
BUG_ON(kmem_cache_node->size < sizeof(struct kmem_cache_node));
- page = new_slab(kmem_cache_node, GFP_NOWAIT, node);
+ slab = new_slab(kmem_cache_node, GFP_NOWAIT, node);
- BUG_ON(!page);
- if (page_to_nid(page) != node) {
+ BUG_ON(!slab);
+ if (slab_nid(slab) != node) {
pr_err("SLUB: Unable to allocate memory from node %d\n", node);
pr_err("SLUB: Allocating a useless per node structure in order to be able to continue\n");
}
- n = page->freelist;
+ n = slab->freelist;
BUG_ON(!n);
#ifdef CONFIG_SLUB_DEBUG
init_object(kmem_cache_node, n, SLUB_RED_ACTIVE);
init_tracking(kmem_cache_node, n);
#endif
n = kasan_slab_alloc(kmem_cache_node, n, GFP_KERNEL, false);
- page->freelist = get_freepointer(kmem_cache_node, n);
- page->inuse = 1;
- page->frozen = 0;
+ slab->freelist = get_freepointer(kmem_cache_node, n);
+ slab->inuse = 1;
+ slab->frozen = 0;
kmem_cache_node->node[node] = n;
init_kmem_cache_node(n);
- inc_slabs_node(kmem_cache_node, node, page->objects);
+ inc_slabs_node(kmem_cache_node, node, slab->objects);
/*
* No locks need to be taken here as it has just been
* initialized and there is no concurrent access.
*/
- __add_partial(n, page, DEACTIVATE_TO_HEAD);
+ __add_partial(n, slab, DEACTIVATE_TO_HEAD);
}
static void free_kmem_cache_nodes(struct kmem_cache *s)
@@ -4212,7 +4212,7 @@ static int kmem_cache_open(struct kmem_cache *s, slab_flags_t flags)
#endif
/*
- * The larger the object size is, the more pages we want on the partial
+ * The larger the object size is, the more slabs we want on the partial
* list to avoid pounding the page allocator excessively.
*/
set_min_partial(s, ilog2(s->size) / 2);
@@ -4240,20 +4240,20 @@ static int kmem_cache_open(struct kmem_cache *s, slab_flags_t flags)
return -EINVAL;
}
-static void list_slab_objects(struct kmem_cache *s, struct page *page,
+static void list_slab_objects(struct kmem_cache *s, struct slab *slab,
const char *text)
{
#ifdef CONFIG_SLUB_DEBUG
- void *addr = page_address(page);
+ void *addr = slab_address(slab);
unsigned long flags;
unsigned long *map;
void *p;
- slab_err(s, page, text, s->name);
- slab_lock(page, &flags);
+ slab_err(s, slab, text, s->name);
+ slab_lock(slab, &flags);
- map = get_map(s, page);
- for_each_object(p, s, addr, page->objects) {
+ map = get_map(s, slab);
+ for_each_object(p, s, addr, slab->objects) {
if (!test_bit(__obj_to_index(s, addr, p), map)) {
pr_err("Object 0x%p @offset=%tu\n", p, p - addr);
@@ -4261,7 +4261,7 @@ static void list_slab_objects(struct kmem_cache *s, struct page *page,
}
}
put_map(map);
- slab_unlock(page, &flags);
+ slab_unlock(slab, &flags);
#endif
}
@@ -4273,23 +4273,23 @@ static void list_slab_objects(struct kmem_cache *s, struct page *page,
static void free_partial(struct kmem_cache *s, struct kmem_cache_node *n)
{
LIST_HEAD(discard);
- struct page *page, *h;
+ struct slab *slab, *h;
BUG_ON(irqs_disabled());
spin_lock_irq(&n->list_lock);
- list_for_each_entry_safe(page, h, &n->partial, slab_list) {
- if (!page->inuse) {
- remove_partial(n, page);
- list_add(&page->slab_list, &discard);
+ list_for_each_entry_safe(slab, h, &n->partial, slab_list) {
+ if (!slab->inuse) {
+ remove_partial(n, slab);
+ list_add(&slab->slab_list, &discard);
} else {
- list_slab_objects(s, page,
+ list_slab_objects(s, slab,
"Objects remaining in %s on __kmem_cache_shutdown()");
}
}
spin_unlock_irq(&n->list_lock);
- list_for_each_entry_safe(page, h, &discard, slab_list)
- discard_slab(s, page);
+ list_for_each_entry_safe(slab, h, &discard, slab_list)
+ discard_slab(s, slab);
}
bool __kmem_cache_empty(struct kmem_cache *s)
@@ -4322,31 +4322,32 @@ int __kmem_cache_shutdown(struct kmem_cache *s)
}
#ifdef CONFIG_PRINTK
-void kmem_obj_info(struct kmem_obj_info *kpp, void *object, struct page *page)
+void kmem_obj_info(struct kmem_obj_info *kpp, void *object, struct slab *slab)
{
void *base;
int __maybe_unused i;
unsigned int objnr;
void *objp;
void *objp0;
- struct kmem_cache *s = page->slab_cache;
+ struct kmem_cache *s = slab->slab_cache;
struct track __maybe_unused *trackp;
kpp->kp_ptr = object;
- kpp->kp_page = page;
+ kpp->kp_slab = slab;
kpp->kp_slab_cache = s;
- base = page_address(page);
+ base = slab_address(slab);
objp0 = kasan_reset_tag(object);
#ifdef CONFIG_SLUB_DEBUG
objp = restore_red_left(s, objp0);
#else
objp = objp0;
#endif
- objnr = obj_to_index(s, page, objp);
+ objnr = obj_to_index(s, slab, objp);
kpp->kp_data_offset = (unsigned long)((char *)objp0 - (char *)objp);
objp = base + s->size * objnr;
kpp->kp_objp = objp;
- if (WARN_ON_ONCE(objp < base || objp >= base + page->objects * s->size || (objp - base) % s->size) ||
+ if (WARN_ON_ONCE(objp < base || objp >= base + slab->objects * s->size
+ || (objp - base) % s->size) ||
!(s->flags & SLAB_STORE_USER))
return;
#ifdef CONFIG_SLUB_DEBUG
@@ -4484,8 +4485,8 @@ EXPORT_SYMBOL(__kmalloc_node);
* Returns NULL if check passes, otherwise const char * to name of cache
* to indicate an error.
*/
-void __check_heap_object(const void *ptr, unsigned long n, struct page *page,
- bool to_user)
+void __check_heap_object(const void *ptr, unsigned long n,
+ const struct slab *slab, bool to_user)
{
struct kmem_cache *s;
unsigned int offset;
@@ -4494,10 +4495,10 @@ void __check_heap_object(const void *ptr, unsigned long n, struct page *page,
ptr = kasan_reset_tag(ptr);
/* Find object and usable object size. */
- s = page->slab_cache;
+ s = slab->slab_cache;
/* Reject impossible pointers. */
- if (ptr < page_address(page))
+ if (ptr < slab_address(slab))
usercopy_abort("SLUB object not in SLUB page?!", NULL,
to_user, 0, n);
@@ -4505,7 +4506,7 @@ void __check_heap_object(const void *ptr, unsigned long n, struct page *page,
if (is_kfence)
offset = ptr - kfence_object_start(ptr);
else
- offset = (ptr - page_address(page)) % s->size;
+ offset = (ptr - slab_address(slab)) % s->size;
/* Adjust for redzone and reject if within the redzone. */
if (!is_kfence && kmem_cache_debug_flags(s, SLAB_RED_ZONE)) {
@@ -4527,25 +4528,24 @@ void __check_heap_object(const void *ptr, unsigned long n, struct page *page,
size_t __ksize(const void *object)
{
- struct page *page;
+ struct folio *folio;
if (unlikely(object == ZERO_SIZE_PTR))
return 0;
- page = virt_to_head_page(object);
+ folio = virt_to_folio(object);
- if (unlikely(!PageSlab(page))) {
- WARN_ON(!PageCompound(page));
- return page_size(page);
- }
+ if (unlikely(!folio_test_slab(folio)))
+ return folio_size(folio);
- return slab_ksize(page->slab_cache);
+ return slab_ksize(folio_slab(folio)->slab_cache);
}
EXPORT_SYMBOL(__ksize);
void kfree(const void *x)
{
- struct page *page;
+ struct folio *folio;
+ struct slab *slab;
void *object = (void *)x;
trace_kfree(_RET_IP_, x);
@@ -4553,12 +4553,13 @@ void kfree(const void *x)
if (unlikely(ZERO_OR_NULL_PTR(x)))
return;
- page = virt_to_head_page(x);
- if (unlikely(!PageSlab(page))) {
- free_nonslab_page(page, object);
+ folio = virt_to_folio(x);
+ if (unlikely(!folio_test_slab(folio))) {
+ free_large_kmalloc(folio, object);
return;
}
- slab_free(page->slab_cache, page, object, NULL, 1, _RET_IP_);
+ slab = folio_slab(folio);
+ slab_free(slab->slab_cache, slab, object, NULL, 1, _RET_IP_);
}
EXPORT_SYMBOL(kfree);
@@ -4578,8 +4579,8 @@ static int __kmem_cache_do_shrink(struct kmem_cache *s)
int node;
int i;
struct kmem_cache_node *n;
- struct page *page;
- struct page *t;
+ struct slab *slab;
+ struct slab *t;
struct list_head discard;
struct list_head promote[SHRINK_PROMOTE_MAX];
unsigned long flags;
@@ -4596,22 +4597,22 @@ static int __kmem_cache_do_shrink(struct kmem_cache *s)
* Build lists of slabs to discard or promote.
*
* Note that concurrent frees may occur while we hold the
- * list_lock. page->inuse here is the upper limit.
+ * list_lock. slab->inuse here is the upper limit.
*/
- list_for_each_entry_safe(page, t, &n->partial, slab_list) {
- int free = page->objects - page->inuse;
+ list_for_each_entry_safe(slab, t, &n->partial, slab_list) {
+ int free = slab->objects - slab->inuse;
- /* Do not reread page->inuse */
+ /* Do not reread slab->inuse */
barrier();
/* We do not keep full slabs on the list */
BUG_ON(free <= 0);
- if (free == page->objects) {
- list_move(&page->slab_list, &discard);
+ if (free == slab->objects) {
+ list_move(&slab->slab_list, &discard);
n->nr_partial--;
} else if (free <= SHRINK_PROMOTE_MAX)
- list_move(&page->slab_list, promote + free - 1);
+ list_move(&slab->slab_list, promote + free - 1);
}
/*
@@ -4624,8 +4625,8 @@ static int __kmem_cache_do_shrink(struct kmem_cache *s)
spin_unlock_irqrestore(&n->list_lock, flags);
/* Release empty slabs */
- list_for_each_entry_safe(page, t, &discard, slab_list)
- discard_slab(s, page);
+ list_for_each_entry_safe(slab, t, &discard, slab_list)
+ discard_slab(s, slab);
if (slabs_node(s, node))
ret = 1;
@@ -4786,7 +4787,7 @@ static struct kmem_cache * __init bootstrap(struct kmem_cache *static_cache)
*/
__flush_cpu_slab(s, smp_processor_id());
for_each_kmem_cache_node(s, node, n) {
- struct page *p;
+ struct slab *p;
list_for_each_entry(p, &n->partial, slab_list)
p->slab_cache = s;
@@ -4964,54 +4965,54 @@ EXPORT_SYMBOL(__kmalloc_node_track_caller);
#endif
#ifdef CONFIG_SYSFS
-static int count_inuse(struct page *page)
+static int count_inuse(struct slab *slab)
{
- return page->inuse;
+ return slab->inuse;
}
-static int count_total(struct page *page)
+static int count_total(struct slab *slab)
{
- return page->objects;
+ return slab->objects;
}
#endif
#ifdef CONFIG_SLUB_DEBUG
-static void validate_slab(struct kmem_cache *s, struct page *page,
+static void validate_slab(struct kmem_cache *s, struct slab *slab,
unsigned long *obj_map)
{
void *p;
- void *addr = page_address(page);
+ void *addr = slab_address(slab);
unsigned long flags;
- slab_lock(page, &flags);
+ slab_lock(slab, &flags);
- if (!check_slab(s, page) || !on_freelist(s, page, NULL))
+ if (!check_slab(s, slab) || !on_freelist(s, slab, NULL))
goto unlock;
/* Now we know that a valid freelist exists */
- __fill_map(obj_map, s, page);
- for_each_object(p, s, addr, page->objects) {
+ __fill_map(obj_map, s, slab);
+ for_each_object(p, s, addr, slab->objects) {
u8 val = test_bit(__obj_to_index(s, addr, p), obj_map) ?
SLUB_RED_INACTIVE : SLUB_RED_ACTIVE;
- if (!check_object(s, page, p, val))
+ if (!check_object(s, slab, p, val))
break;
}
unlock:
- slab_unlock(page, &flags);
+ slab_unlock(slab, &flags);
}
static int validate_slab_node(struct kmem_cache *s,
struct kmem_cache_node *n, unsigned long *obj_map)
{
unsigned long count = 0;
- struct page *page;
+ struct slab *slab;
unsigned long flags;
spin_lock_irqsave(&n->list_lock, flags);
- list_for_each_entry(page, &n->partial, slab_list) {
- validate_slab(s, page, obj_map);
+ list_for_each_entry(slab, &n->partial, slab_list) {
+ validate_slab(s, slab, obj_map);
count++;
}
if (count != n->nr_partial) {
@@ -5023,8 +5024,8 @@ static int validate_slab_node(struct kmem_cache *s,
if (!(s->flags & SLAB_STORE_USER))
goto out;
- list_for_each_entry(page, &n->full, slab_list) {
- validate_slab(s, page, obj_map);
+ list_for_each_entry(slab, &n->full, slab_list) {
+ validate_slab(s, slab, obj_map);
count++;
}
if (count != atomic_long_read(&n->nr_slabs)) {
@@ -5190,15 +5191,15 @@ static int add_location(struct loc_track *t, struct kmem_cache *s,
}
static void process_slab(struct loc_track *t, struct kmem_cache *s,
- struct page *page, enum track_item alloc,
+ struct slab *slab, enum track_item alloc,
unsigned long *obj_map)
{
- void *addr = page_address(page);
+ void *addr = slab_address(slab);
void *p;
- __fill_map(obj_map, s, page);
+ __fill_map(obj_map, s, slab);
- for_each_object(p, s, addr, page->objects)
+ for_each_object(p, s, addr, slab->objects)
if (!test_bit(__obj_to_index(s, addr, p), obj_map))
add_location(t, s, get_track(s, p, alloc));
}
@@ -5240,35 +5241,37 @@ static ssize_t show_slab_objects(struct kmem_cache *s,
struct kmem_cache_cpu *c = per_cpu_ptr(s->cpu_slab,
cpu);
int node;
- struct page *page;
+ struct slab *slab;
- page = READ_ONCE(c->page);
- if (!page)
+ slab = READ_ONCE(c->slab);
+ if (!slab)
continue;
- node = page_to_nid(page);
+ node = slab_nid(slab);
if (flags & SO_TOTAL)
- x = page->objects;
+ x = slab->objects;
else if (flags & SO_OBJECTS)
- x = page->inuse;
+ x = slab->inuse;
else
x = 1;
total += x;
nodes[node] += x;
- page = slub_percpu_partial_read_once(c);
- if (page) {
- node = page_to_nid(page);
+#ifdef CONFIG_SLUB_CPU_PARTIAL
+ slab = slub_percpu_partial_read_once(c);
+ if (slab) {
+ node = slab_nid(slab);
if (flags & SO_TOTAL)
WARN_ON_ONCE(1);
else if (flags & SO_OBJECTS)
WARN_ON_ONCE(1);
else
- x = page->pages;
+ x = slab->slabs;
total += x;
nodes[node] += x;
}
+#endif
}
}
@@ -5467,33 +5470,35 @@ SLAB_ATTR_RO(objects_partial);
static ssize_t slabs_cpu_partial_show(struct kmem_cache *s, char *buf)
{
int objects = 0;
- int pages = 0;
- int cpu;
+ int slabs = 0;
+ int cpu __maybe_unused;
int len = 0;
+#ifdef CONFIG_SLUB_CPU_PARTIAL
for_each_online_cpu(cpu) {
- struct page *page;
+ struct slab *slab;
- page = slub_percpu_partial(per_cpu_ptr(s->cpu_slab, cpu));
+ slab = slub_percpu_partial(per_cpu_ptr(s->cpu_slab, cpu));
- if (page)
- pages += page->pages;
+ if (slab)
+ slabs += slab->slabs;
}
+#endif
- /* Approximate half-full pages , see slub_set_cpu_partial() */
- objects = (pages * oo_objects(s->oo)) / 2;
- len += sysfs_emit_at(buf, len, "%d(%d)", objects, pages);
+ /* Approximate half-full slabs, see slub_set_cpu_partial() */
+ objects = (slabs * oo_objects(s->oo)) / 2;
+ len += sysfs_emit_at(buf, len, "%d(%d)", objects, slabs);
-#ifdef CONFIG_SMP
+#if defined(CONFIG_SLUB_CPU_PARTIAL) && defined(CONFIG_SMP)
for_each_online_cpu(cpu) {
- struct page *page;
+ struct slab *slab;
- page = slub_percpu_partial(per_cpu_ptr(s->cpu_slab, cpu));
- if (page) {
- pages = READ_ONCE(page->pages);
- objects = (pages * oo_objects(s->oo)) / 2;
+ slab = slub_percpu_partial(per_cpu_ptr(s->cpu_slab, cpu));
+ if (slab) {
+ slabs = READ_ONCE(slab->slabs);
+ objects = (slabs * oo_objects(s->oo)) / 2;
len += sysfs_emit_at(buf, len, " C%d=%d(%d)",
- cpu, objects, pages);
+ cpu, objects, slabs);
}
}
#endif
@@ -6161,16 +6166,16 @@ static int slab_debug_trace_open(struct inode *inode, struct file *filep)
for_each_kmem_cache_node(s, node, n) {
unsigned long flags;
- struct page *page;
+ struct slab *slab;
if (!atomic_long_read(&n->nr_slabs))
continue;
spin_lock_irqsave(&n->list_lock, flags);
- list_for_each_entry(page, &n->partial, slab_list)
- process_slab(t, s, page, alloc, obj_map);
- list_for_each_entry(page, &n->full, slab_list)
- process_slab(t, s, page, alloc, obj_map);
+ list_for_each_entry(slab, &n->partial, slab_list)
+ process_slab(t, s, slab, alloc, obj_map);
+ list_for_each_entry(slab, &n->full, slab_list)
+ process_slab(t, s, slab, alloc, obj_map);
spin_unlock_irqrestore(&n->list_lock, flags);
}
diff --git a/mm/sparse.c b/mm/sparse.c
index e5c84b0..d21c6e5 100644
--- a/mm/sparse.c
+++ b/mm/sparse.c
@@ -722,7 +722,7 @@ static void free_map_bootmem(struct page *memmap)
>> PAGE_SHIFT;
for (i = 0; i < nr_pages; i++, page++) {
- magic = (unsigned long) page->freelist;
+ magic = page->index;
BUG_ON(magic == NODE_INFO);
diff --git a/mm/usercopy.c b/mm/usercopy.c
index b3de3c4..d0d2681 100644
--- a/mm/usercopy.c
+++ b/mm/usercopy.c
@@ -20,6 +20,7 @@
#include <linux/atomic.h>
#include <linux/jump_label.h>
#include <asm/sections.h>
+#include "slab.h"
/*
* Checks if a given pointer and length is contained by the current
@@ -223,7 +224,7 @@ static inline void check_page_span(const void *ptr, unsigned long n,
static inline void check_heap_object(const void *ptr, unsigned long n,
bool to_user)
{
- struct page *page;
+ struct folio *folio;
if (!virt_addr_valid(ptr))
return;
@@ -231,16 +232,16 @@ static inline void check_heap_object(const void *ptr, unsigned long n,
/*
* When CONFIG_HIGHMEM=y, kmap_to_page() will give either the
* highmem page or fallback to virt_to_page(). The following
- * is effectively a highmem-aware virt_to_head_page().
+ * is effectively a highmem-aware virt_to_slab().
*/
- page = compound_head(kmap_to_page((void *)ptr));
+ folio = page_folio(kmap_to_page((void *)ptr));
- if (PageSlab(page)) {
+ if (folio_test_slab(folio)) {
/* Check slab allocator for flags and size. */
- __check_heap_object(ptr, n, page, to_user);
+ __check_heap_object(ptr, n, folio_slab(folio), to_user);
} else {
/* Verify object does not incorrectly span multiple pages. */
- check_page_span(ptr, n, page, to_user);
+ check_page_span(ptr, n, folio_page(folio, 0), to_user);
}
}
diff --git a/mm/vmscan.c b/mm/vmscan.c
index fb95846..700434d 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -1021,6 +1021,39 @@ static void handle_write_error(struct address_space *mapping,
unlock_page(page);
}
+static bool skip_throttle_noprogress(pg_data_t *pgdat)
+{
+ int reclaimable = 0, write_pending = 0;
+ int i;
+
+ /*
+ * If kswapd is disabled, reschedule if necessary but do not
+ * throttle as the system is likely near OOM.
+ */
+ if (pgdat->kswapd_failures >= MAX_RECLAIM_RETRIES)
+ return true;
+
+ /*
+ * If there are a lot of dirty/writeback pages then do not
+ * throttle as throttling will occur when the pages cycle
+ * towards the end of the LRU if still under writeback.
+ */
+ for (i = 0; i < MAX_NR_ZONES; i++) {
+ struct zone *zone = pgdat->node_zones + i;
+
+ if (!populated_zone(zone))
+ continue;
+
+ reclaimable += zone_reclaimable_pages(zone);
+ write_pending += zone_page_state_snapshot(zone,
+ NR_ZONE_WRITE_PENDING);
+ }
+ if (2 * write_pending <= reclaimable)
+ return true;
+
+ return false;
+}
+
void reclaim_throttle(pg_data_t *pgdat, enum vmscan_throttle_state reason)
{
wait_queue_head_t *wqh = &pgdat->reclaim_wait[reason];
@@ -1056,8 +1089,16 @@ void reclaim_throttle(pg_data_t *pgdat, enum vmscan_throttle_state reason)
}
break;
+ case VMSCAN_THROTTLE_CONGESTED:
+ fallthrough;
case VMSCAN_THROTTLE_NOPROGRESS:
- timeout = HZ/2;
+ if (skip_throttle_noprogress(pgdat)) {
+ cond_resched();
+ return;
+ }
+
+ timeout = 1;
+
break;
case VMSCAN_THROTTLE_ISOLATED:
timeout = HZ/50;
@@ -3321,7 +3362,7 @@ static void shrink_node(pg_data_t *pgdat, struct scan_control *sc)
if (!current_is_kswapd() && current_may_throttle() &&
!sc->hibernation_mode &&
test_bit(LRUVEC_CONGESTED, &target_lruvec->flags))
- reclaim_throttle(pgdat, VMSCAN_THROTTLE_WRITEBACK);
+ reclaim_throttle(pgdat, VMSCAN_THROTTLE_CONGESTED);
if (should_continue_reclaim(pgdat, sc->nr_reclaimed - nr_reclaimed,
sc))
@@ -3386,16 +3427,16 @@ static void consider_reclaim_throttle(pg_data_t *pgdat, struct scan_control *sc)
}
/*
- * Do not throttle kswapd on NOPROGRESS as it will throttle on
- * VMSCAN_THROTTLE_WRITEBACK if there are too many pages under
- * writeback and marked for immediate reclaim at the tail of
- * the LRU.
+ * Do not throttle kswapd or cgroup reclaim on NOPROGRESS as it will
+ * throttle on VMSCAN_THROTTLE_WRITEBACK if there are too many pages
+ * under writeback and marked for immediate reclaim at the tail of the
+ * LRU.
*/
- if (current_is_kswapd())
+ if (current_is_kswapd() || cgroup_reclaim(sc))
return;
/* Throttle if making no progress at high prioities. */
- if (sc->priority < DEF_PRIORITY - 2)
+ if (sc->priority == 1 && !sc->nr_reclaimed)
reclaim_throttle(pgdat, VMSCAN_THROTTLE_NOPROGRESS);
}
@@ -3415,6 +3456,7 @@ static void shrink_zones(struct zonelist *zonelist, struct scan_control *sc)
unsigned long nr_soft_scanned;
gfp_t orig_mask;
pg_data_t *last_pgdat = NULL;
+ pg_data_t *first_pgdat = NULL;
/*
* If the number of buffer_heads in the machine exceeds the maximum
@@ -3478,14 +3520,19 @@ static void shrink_zones(struct zonelist *zonelist, struct scan_control *sc)
/* need some check for avoid more shrink_zone() */
}
+ if (!first_pgdat)
+ first_pgdat = zone->zone_pgdat;
+
/* See comment about same check for global reclaim above */
if (zone->zone_pgdat == last_pgdat)
continue;
last_pgdat = zone->zone_pgdat;
shrink_node(zone->zone_pgdat, sc);
- consider_reclaim_throttle(zone->zone_pgdat, sc);
}
+ if (first_pgdat)
+ consider_reclaim_throttle(first_pgdat, sc);
+
/*
* Restore to original mask to avoid the impact on the caller if we
* promoted it to __GFP_HIGHMEM.
diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index b897ce3..0d3b659 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -17,10 +17,10 @@
*
* Usage of struct page fields:
* page->private: points to zspage
- * page->freelist(index): links together all component pages of a zspage
+ * page->index: links together all component pages of a zspage
* For the huge page, this is always 0, so we use this field
* to store handle.
- * page->units: first object offset in a subpage of zspage
+ * page->page_type: first object offset in a subpage of zspage
*
* Usage of struct page flags:
* PG_private: identifies the first component page
@@ -489,12 +489,12 @@ static inline struct page *get_first_page(struct zspage *zspage)
static inline int get_first_obj_offset(struct page *page)
{
- return page->units;
+ return page->page_type;
}
static inline void set_first_obj_offset(struct page *page, int offset)
{
- page->units = offset;
+ page->page_type = offset;
}
static inline unsigned int get_freeobj(struct zspage *zspage)
@@ -827,7 +827,7 @@ static struct page *get_next_page(struct page *page)
if (unlikely(PageHugeObject(page)))
return NULL;
- return page->freelist;
+ return (struct page *)page->index;
}
/**
@@ -901,7 +901,7 @@ static void reset_page(struct page *page)
set_page_private(page, 0);
page_mapcount_reset(page);
ClearPageHugeObject(page);
- page->freelist = NULL;
+ page->index = 0;
}
static int trylock_zspage(struct zspage *zspage)
@@ -1027,7 +1027,7 @@ static void create_page_chain(struct size_class *class, struct zspage *zspage,
/*
* Allocate individual pages and link them together as:
- * 1. all pages are linked together using page->freelist
+ * 1. all pages are linked together using page->index
* 2. each sub-page point to zspage using page->private
*
* we set PG_private to identify the first page (i.e. no other sub-page
@@ -1036,7 +1036,7 @@ static void create_page_chain(struct size_class *class, struct zspage *zspage,
for (i = 0; i < nr_pages; i++) {
page = pages[i];
set_page_private(page, (unsigned long)zspage);
- page->freelist = NULL;
+ page->index = 0;
if (i == 0) {
zspage->first_page = page;
SetPagePrivate(page);
@@ -1044,7 +1044,7 @@ static void create_page_chain(struct size_class *class, struct zspage *zspage,
class->pages_per_zspage == 1))
SetPageHugeObject(page);
} else {
- prev_page->freelist = page;
+ prev_page->index = (unsigned long)page;
}
prev_page = page;
}
diff --git a/net/802/hippi.c b/net/802/hippi.c
index 887e73d5..1997b7d 100644
--- a/net/802/hippi.c
+++ b/net/802/hippi.c
@@ -65,7 +65,7 @@ static int hippi_header(struct sk_buff *skb, struct net_device *dev,
hip->le.src_addr_type = 2; /* 12 bit SC address */
memcpy(hip->le.src_switch_addr, dev->dev_addr + 3, 3);
- memset(&hip->le.reserved, 0, 16);
+ memset_startat(&hip->le, 0, reserved);
hip->snap.dsap = HIPPI_EXTENDED_SAP;
hip->snap.ssap = HIPPI_EXTENDED_SAP;
diff --git a/net/8021q/vlan.c b/net/8021q/vlan.c
index abaa5d9..788076b 100644
--- a/net/8021q/vlan.c
+++ b/net/8021q/vlan.c
@@ -319,8 +319,8 @@ static void vlan_transfer_features(struct net_device *dev,
{
struct vlan_dev_priv *vlan = vlan_dev_priv(vlandev);
- vlandev->gso_max_size = dev->gso_max_size;
- vlandev->gso_max_segs = dev->gso_max_segs;
+ netif_set_gso_max_size(vlandev, dev->gso_max_size);
+ netif_set_gso_max_segs(vlandev, dev->gso_max_segs);
if (vlan_hw_offload_capable(dev->features, vlan->vlan_proto))
vlandev->hard_header_len = dev->hard_header_len;
diff --git a/net/8021q/vlan_core.c b/net/8021q/vlan_core.c
index 59bc13b..acf8c79 100644
--- a/net/8021q/vlan_core.c
+++ b/net/8021q/vlan_core.c
@@ -476,10 +476,9 @@ static struct sk_buff *vlan_gro_receive(struct list_head *head,
type = vhdr->h_vlan_encapsulated_proto;
- rcu_read_lock();
ptype = gro_find_receive_by_type(type);
if (!ptype)
- goto out_unlock;
+ goto out;
flush = 0;
@@ -501,8 +500,6 @@ static struct sk_buff *vlan_gro_receive(struct list_head *head,
ipv6_gro_receive, inet_gro_receive,
head, skb);
-out_unlock:
- rcu_read_unlock();
out:
skb_gro_flush_final(skb, pp, flush);
@@ -516,14 +513,12 @@ static int vlan_gro_complete(struct sk_buff *skb, int nhoff)
struct packet_offload *ptype;
int err = -ENOENT;
- rcu_read_lock();
ptype = gro_find_complete_by_type(type);
if (ptype)
err = INDIRECT_CALL_INET(ptype->callbacks.gro_complete,
ipv6_gro_complete, inet_gro_complete,
skb, nhoff + sizeof(*vhdr));
- rcu_read_unlock();
return err;
}
diff --git a/net/8021q/vlan_dev.c b/net/8021q/vlan_dev.c
index a54535c..26d031a 100644
--- a/net/8021q/vlan_dev.c
+++ b/net/8021q/vlan_dev.c
@@ -573,8 +573,8 @@ static int vlan_dev_init(struct net_device *dev)
NETIF_F_ALL_FCOE;
dev->features |= dev->hw_features | NETIF_F_LLTX;
- dev->gso_max_size = real_dev->gso_max_size;
- dev->gso_max_segs = real_dev->gso_max_segs;
+ netif_set_gso_max_size(dev, real_dev->gso_max_size);
+ netif_set_gso_max_segs(dev, real_dev->gso_max_segs);
if (dev->features & NETIF_F_VLAN_FEATURES)
netdev_warn(real_dev, "VLAN features are set incorrectly. Q-in-Q configurations may not work correctly.\n");
@@ -616,7 +616,7 @@ static int vlan_dev_init(struct net_device *dev)
return -ENOMEM;
/* Get vlan's reference to real_dev */
- dev_hold(real_dev);
+ dev_hold_track(real_dev, &vlan->dev_tracker, GFP_KERNEL);
return 0;
}
@@ -848,7 +848,7 @@ static void vlan_dev_free(struct net_device *dev)
vlan->vlan_pcpu_stats = NULL;
/* Get rid of the vlan's reference to real_dev */
- dev_put(vlan->real_dev);
+ dev_put_track(vlan->real_dev, &vlan->dev_tracker);
}
void vlan_setup(struct net_device *dev)
diff --git a/net/8021q/vlanproc.c b/net/8021q/vlanproc.c
index ec87dea..08bf6c8 100644
--- a/net/8021q/vlanproc.c
+++ b/net/8021q/vlanproc.c
@@ -252,7 +252,7 @@ static int vlandev_seq_show(struct seq_file *seq, void *offset)
stats = dev_get_stats(vlandev, &temp);
seq_printf(seq,
- "%s VID: %d REORDER_HDR: %i dev->priv_flags: %hx\n",
+ "%s VID: %d REORDER_HDR: %i dev->priv_flags: %llx\n",
vlandev->name, vlan->vlan_id,
(int)(vlan->flags & 1), vlandev->priv_flags);
diff --git a/net/Kconfig b/net/Kconfig
index 074472d..8a1f9d0 100644
--- a/net/Kconfig
+++ b/net/Kconfig
@@ -455,4 +455,9 @@
netlink. It provides better extensibility and some new features,
e.g. notification messages.
+config NETDEV_ADDR_LIST_TEST
+ tristate "Unit tests for device address list"
+ default KUNIT_ALL_TESTS
+ depends on KUNIT
+
endif # if NET
diff --git a/net/Kconfig.debug b/net/Kconfig.debug
new file mode 100644
index 0000000..2f50611
--- /dev/null
+++ b/net/Kconfig.debug
@@ -0,0 +1,19 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+config NET_DEV_REFCNT_TRACKER
+ bool "Enable net device refcount tracking"
+ depends on DEBUG_KERNEL && STACKTRACE_SUPPORT
+ select REF_TRACKER
+ default n
+ help
+ Enable debugging feature to track device references.
+ This adds memory and cpu costs.
+
+config NET_NS_REFCNT_TRACKER
+ bool "Enable networking namespace refcount tracking"
+ depends on DEBUG_KERNEL && STACKTRACE_SUPPORT
+ select REF_TRACKER
+ default n
+ help
+ Enable debugging feature to track netns references.
+ This adds memory and cpu costs.
diff --git a/net/ax25/af_ax25.c b/net/ax25/af_ax25.c
index cfca99e..02f43f3 100644
--- a/net/ax25/af_ax25.c
+++ b/net/ax25/af_ax25.c
@@ -536,7 +536,7 @@ static int ax25_setsockopt(struct socket *sock, int level, int optname,
ax25_cb *ax25;
struct net_device *dev;
char devname[IFNAMSIZ];
- unsigned long opt;
+ unsigned int opt;
int res = 0;
if (level != SOL_AX25)
@@ -568,7 +568,7 @@ static int ax25_setsockopt(struct socket *sock, int level, int optname,
break;
case AX25_T1:
- if (opt < 1 || opt > ULONG_MAX / HZ) {
+ if (opt < 1 || opt > UINT_MAX / HZ) {
res = -EINVAL;
break;
}
@@ -577,7 +577,7 @@ static int ax25_setsockopt(struct socket *sock, int level, int optname,
break;
case AX25_T2:
- if (opt < 1 || opt > ULONG_MAX / HZ) {
+ if (opt < 1 || opt > UINT_MAX / HZ) {
res = -EINVAL;
break;
}
@@ -593,7 +593,7 @@ static int ax25_setsockopt(struct socket *sock, int level, int optname,
break;
case AX25_T3:
- if (opt < 1 || opt > ULONG_MAX / HZ) {
+ if (opt < 1 || opt > UINT_MAX / HZ) {
res = -EINVAL;
break;
}
@@ -601,7 +601,7 @@ static int ax25_setsockopt(struct socket *sock, int level, int optname,
break;
case AX25_IDLE:
- if (opt > ULONG_MAX / (60 * HZ)) {
+ if (opt > UINT_MAX / (60 * HZ)) {
res = -EINVAL;
break;
}
diff --git a/net/ax25/ax25_dev.c b/net/ax25/ax25_dev.c
index d0a043a..256fadb 100644
--- a/net/ax25/ax25_dev.c
+++ b/net/ax25/ax25_dev.c
@@ -58,7 +58,7 @@ void ax25_dev_device_up(struct net_device *dev)
dev->ax25_ptr = ax25_dev;
ax25_dev->dev = dev;
- dev_hold(dev);
+ dev_hold_track(dev, &ax25_dev->dev_tracker, GFP_ATOMIC);
ax25_dev->forward = NULL;
ax25_dev->values[AX25_VALUES_IPDEFMODE] = AX25_DEF_IPDEFMODE;
@@ -114,7 +114,7 @@ void ax25_dev_device_down(struct net_device *dev)
ax25_dev_list = s->next;
spin_unlock_bh(&ax25_dev_lock);
dev->ax25_ptr = NULL;
- dev_put(dev);
+ dev_put_track(dev, &ax25_dev->dev_tracker);
kfree(ax25_dev);
return;
}
@@ -124,7 +124,7 @@ void ax25_dev_device_down(struct net_device *dev)
s->next = ax25_dev->next;
spin_unlock_bh(&ax25_dev_lock);
dev->ax25_ptr = NULL;
- dev_put(dev);
+ dev_put_track(dev, &ax25_dev->dev_tracker);
kfree(ax25_dev);
return;
}
@@ -188,7 +188,7 @@ void __exit ax25_dev_free(void)
ax25_dev = ax25_dev_list;
while (ax25_dev != NULL) {
s = ax25_dev;
- dev_put(ax25_dev->dev);
+ dev_put_track(ax25_dev->dev, &ax25_dev->dev_tracker);
ax25_dev = ax25_dev->next;
kfree(s);
}
diff --git a/net/batman-adv/main.h b/net/batman-adv/main.h
index 058b8f2..494d1eb 100644
--- a/net/batman-adv/main.h
+++ b/net/batman-adv/main.h
@@ -13,7 +13,7 @@
#define BATADV_DRIVER_DEVICE "batman-adv"
#ifndef BATADV_SOURCE_VERSION
-#define BATADV_SOURCE_VERSION "2021.3"
+#define BATADV_SOURCE_VERSION "2022.0"
#endif
/* B.A.T.M.A.N. parameters */
diff --git a/net/batman-adv/multicast.c b/net/batman-adv/multicast.c
index 433901d..f4004cf 100644
--- a/net/batman-adv/multicast.c
+++ b/net/batman-adv/multicast.c
@@ -1339,6 +1339,7 @@ batadv_mcast_forw_rtr_node_get(struct batadv_priv *bat_priv,
* @bat_priv: the bat priv with all the soft interface information
* @skb: The multicast packet to check
* @orig: an originator to be set to forward the skb to
+ * @is_routable: stores whether the destination is routable
*
* Return: the forwarding mode as enum batadv_forw_mode and in case of
* BATADV_FORW_SINGLE set the orig to the single originator the skb
@@ -1346,17 +1347,16 @@ batadv_mcast_forw_rtr_node_get(struct batadv_priv *bat_priv,
*/
enum batadv_forw_mode
batadv_mcast_forw_mode(struct batadv_priv *bat_priv, struct sk_buff *skb,
- struct batadv_orig_node **orig)
+ struct batadv_orig_node **orig, int *is_routable)
{
int ret, tt_count, ip_count, unsnoop_count, total_count;
bool is_unsnoopable = false;
unsigned int mcast_fanout;
struct ethhdr *ethhdr;
- int is_routable = 0;
int rtr_count = 0;
ret = batadv_mcast_forw_mode_check(bat_priv, skb, &is_unsnoopable,
- &is_routable);
+ is_routable);
if (ret == -ENOMEM)
return BATADV_FORW_NONE;
else if (ret < 0)
@@ -1369,7 +1369,7 @@ batadv_mcast_forw_mode(struct batadv_priv *bat_priv, struct sk_buff *skb,
ip_count = batadv_mcast_forw_want_all_ip_count(bat_priv, ethhdr);
unsnoop_count = !is_unsnoopable ? 0 :
atomic_read(&bat_priv->mcast.num_want_all_unsnoopables);
- rtr_count = batadv_mcast_forw_rtr_count(bat_priv, is_routable);
+ rtr_count = batadv_mcast_forw_rtr_count(bat_priv, *is_routable);
total_count = tt_count + ip_count + unsnoop_count + rtr_count;
@@ -1689,6 +1689,7 @@ batadv_mcast_forw_want_rtr(struct batadv_priv *bat_priv,
* @bat_priv: the bat priv with all the soft interface information
* @skb: the multicast packet to transmit
* @vid: the vlan identifier
+ * @is_routable: stores whether the destination is routable
*
* Sends copies of a frame with multicast destination to any node that signaled
* interest in it, that is either via the translation table or the according
@@ -1701,7 +1702,7 @@ batadv_mcast_forw_want_rtr(struct batadv_priv *bat_priv,
* is neither IPv4 nor IPv6. NET_XMIT_SUCCESS otherwise.
*/
int batadv_mcast_forw_send(struct batadv_priv *bat_priv, struct sk_buff *skb,
- unsigned short vid)
+ unsigned short vid, int is_routable)
{
int ret;
@@ -1717,12 +1718,16 @@ int batadv_mcast_forw_send(struct batadv_priv *bat_priv, struct sk_buff *skb,
return ret;
}
+ if (!is_routable)
+ goto skip_mc_router;
+
ret = batadv_mcast_forw_want_rtr(bat_priv, skb, vid);
if (ret != NET_XMIT_SUCCESS) {
kfree_skb(skb);
return ret;
}
+skip_mc_router:
consume_skb(skb);
return ret;
}
diff --git a/net/batman-adv/multicast.h b/net/batman-adv/multicast.h
index 9fee5da..8aec818 100644
--- a/net/batman-adv/multicast.h
+++ b/net/batman-adv/multicast.h
@@ -43,7 +43,8 @@ enum batadv_forw_mode {
enum batadv_forw_mode
batadv_mcast_forw_mode(struct batadv_priv *bat_priv, struct sk_buff *skb,
- struct batadv_orig_node **mcast_single_orig);
+ struct batadv_orig_node **mcast_single_orig,
+ int *is_routable);
int batadv_mcast_forw_send_orig(struct batadv_priv *bat_priv,
struct sk_buff *skb,
@@ -51,7 +52,7 @@ int batadv_mcast_forw_send_orig(struct batadv_priv *bat_priv,
struct batadv_orig_node *orig_node);
int batadv_mcast_forw_send(struct batadv_priv *bat_priv, struct sk_buff *skb,
- unsigned short vid);
+ unsigned short vid, int is_routable);
void batadv_mcast_init(struct batadv_priv *bat_priv);
@@ -68,7 +69,8 @@ void batadv_mcast_purge_orig(struct batadv_orig_node *orig_node);
static inline enum batadv_forw_mode
batadv_mcast_forw_mode(struct batadv_priv *bat_priv, struct sk_buff *skb,
- struct batadv_orig_node **mcast_single_orig)
+ struct batadv_orig_node **mcast_single_orig,
+ int *is_routable)
{
return BATADV_FORW_ALL;
}
@@ -85,7 +87,7 @@ batadv_mcast_forw_send_orig(struct batadv_priv *bat_priv,
static inline int
batadv_mcast_forw_send(struct batadv_priv *bat_priv, struct sk_buff *skb,
- unsigned short vid)
+ unsigned short vid, int is_routable)
{
kfree_skb(skb);
return NET_XMIT_DROP;
diff --git a/net/batman-adv/netlink.c b/net/batman-adv/netlink.c
index 2927628..00875e1 100644
--- a/net/batman-adv/netlink.c
+++ b/net/batman-adv/netlink.c
@@ -1368,21 +1368,21 @@ static const struct genl_small_ops batadv_netlink_ops[] = {
{
.cmd = BATADV_CMD_TP_METER,
.validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP,
- .flags = GENL_ADMIN_PERM,
+ .flags = GENL_UNS_ADMIN_PERM,
.doit = batadv_netlink_tp_meter_start,
.internal_flags = BATADV_FLAG_NEED_MESH,
},
{
.cmd = BATADV_CMD_TP_METER_CANCEL,
.validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP,
- .flags = GENL_ADMIN_PERM,
+ .flags = GENL_UNS_ADMIN_PERM,
.doit = batadv_netlink_tp_meter_cancel,
.internal_flags = BATADV_FLAG_NEED_MESH,
},
{
.cmd = BATADV_CMD_GET_ROUTING_ALGOS,
.validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP,
- .flags = GENL_ADMIN_PERM,
+ .flags = GENL_UNS_ADMIN_PERM,
.dumpit = batadv_algo_dump,
},
{
@@ -1397,68 +1397,68 @@ static const struct genl_small_ops batadv_netlink_ops[] = {
{
.cmd = BATADV_CMD_GET_TRANSTABLE_LOCAL,
.validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP,
- .flags = GENL_ADMIN_PERM,
+ .flags = GENL_UNS_ADMIN_PERM,
.dumpit = batadv_tt_local_dump,
},
{
.cmd = BATADV_CMD_GET_TRANSTABLE_GLOBAL,
.validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP,
- .flags = GENL_ADMIN_PERM,
+ .flags = GENL_UNS_ADMIN_PERM,
.dumpit = batadv_tt_global_dump,
},
{
.cmd = BATADV_CMD_GET_ORIGINATORS,
.validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP,
- .flags = GENL_ADMIN_PERM,
+ .flags = GENL_UNS_ADMIN_PERM,
.dumpit = batadv_orig_dump,
},
{
.cmd = BATADV_CMD_GET_NEIGHBORS,
.validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP,
- .flags = GENL_ADMIN_PERM,
+ .flags = GENL_UNS_ADMIN_PERM,
.dumpit = batadv_hardif_neigh_dump,
},
{
.cmd = BATADV_CMD_GET_GATEWAYS,
.validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP,
- .flags = GENL_ADMIN_PERM,
+ .flags = GENL_UNS_ADMIN_PERM,
.dumpit = batadv_gw_dump,
},
{
.cmd = BATADV_CMD_GET_BLA_CLAIM,
.validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP,
- .flags = GENL_ADMIN_PERM,
+ .flags = GENL_UNS_ADMIN_PERM,
.dumpit = batadv_bla_claim_dump,
},
{
.cmd = BATADV_CMD_GET_BLA_BACKBONE,
.validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP,
- .flags = GENL_ADMIN_PERM,
+ .flags = GENL_UNS_ADMIN_PERM,
.dumpit = batadv_bla_backbone_dump,
},
{
.cmd = BATADV_CMD_GET_DAT_CACHE,
.validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP,
- .flags = GENL_ADMIN_PERM,
+ .flags = GENL_UNS_ADMIN_PERM,
.dumpit = batadv_dat_cache_dump,
},
{
.cmd = BATADV_CMD_GET_MCAST_FLAGS,
.validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP,
- .flags = GENL_ADMIN_PERM,
+ .flags = GENL_UNS_ADMIN_PERM,
.dumpit = batadv_mcast_flags_dump,
},
{
.cmd = BATADV_CMD_SET_MESH,
.validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP,
- .flags = GENL_ADMIN_PERM,
+ .flags = GENL_UNS_ADMIN_PERM,
.doit = batadv_netlink_set_mesh,
.internal_flags = BATADV_FLAG_NEED_MESH,
},
{
.cmd = BATADV_CMD_SET_HARDIF,
.validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP,
- .flags = GENL_ADMIN_PERM,
+ .flags = GENL_UNS_ADMIN_PERM,
.doit = batadv_netlink_set_hardif,
.internal_flags = BATADV_FLAG_NEED_MESH |
BATADV_FLAG_NEED_HARDIF,
@@ -1474,7 +1474,7 @@ static const struct genl_small_ops batadv_netlink_ops[] = {
{
.cmd = BATADV_CMD_SET_VLAN,
.validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP,
- .flags = GENL_ADMIN_PERM,
+ .flags = GENL_UNS_ADMIN_PERM,
.doit = batadv_netlink_set_vlan,
.internal_flags = BATADV_FLAG_NEED_MESH |
BATADV_FLAG_NEED_VLAN,
diff --git a/net/batman-adv/network-coding.c b/net/batman-adv/network-coding.c
index 0a7f1d3..974d726 100644
--- a/net/batman-adv/network-coding.c
+++ b/net/batman-adv/network-coding.c
@@ -58,13 +58,9 @@ static int batadv_nc_recv_coded_packet(struct sk_buff *skb,
*/
int __init batadv_nc_init(void)
{
- int ret;
-
/* Register our packet type */
- ret = batadv_recv_handler_register(BATADV_CODED,
- batadv_nc_recv_coded_packet);
-
- return ret;
+ return batadv_recv_handler_register(BATADV_CODED,
+ batadv_nc_recv_coded_packet);
}
/**
diff --git a/net/batman-adv/soft-interface.c b/net/batman-adv/soft-interface.c
index 7ee0933..2dbbe6c 100644
--- a/net/batman-adv/soft-interface.c
+++ b/net/batman-adv/soft-interface.c
@@ -198,6 +198,7 @@ static netdev_tx_t batadv_interface_tx(struct sk_buff *skb,
int gw_mode;
enum batadv_forw_mode forw_mode = BATADV_FORW_SINGLE;
struct batadv_orig_node *mcast_single_orig = NULL;
+ int mcast_is_routable = 0;
int network_offset = ETH_HLEN;
__be16 proto;
@@ -300,7 +301,8 @@ static netdev_tx_t batadv_interface_tx(struct sk_buff *skb,
send:
if (do_bcast && !is_broadcast_ether_addr(ethhdr->h_dest)) {
forw_mode = batadv_mcast_forw_mode(bat_priv, skb,
- &mcast_single_orig);
+ &mcast_single_orig,
+ &mcast_is_routable);
if (forw_mode == BATADV_FORW_NONE)
goto dropped;
@@ -359,7 +361,8 @@ static netdev_tx_t batadv_interface_tx(struct sk_buff *skb,
ret = batadv_mcast_forw_send_orig(bat_priv, skb, vid,
mcast_single_orig);
} else if (forw_mode == BATADV_FORW_SOME) {
- ret = batadv_mcast_forw_send(bat_priv, skb, vid);
+ ret = batadv_mcast_forw_send(bat_priv, skb, vid,
+ mcast_is_routable);
} else {
if (batadv_dat_snoop_outgoing_arp_request(bat_priv,
skb))
diff --git a/net/bluetooth/Makefile b/net/bluetooth/Makefile
index 291770f..a52bba8 100644
--- a/net/bluetooth/Makefile
+++ b/net/bluetooth/Makefile
@@ -15,7 +15,7 @@
bluetooth-y := af_bluetooth.o hci_core.o hci_conn.o hci_event.o mgmt.o \
hci_sock.o hci_sysfs.o l2cap_core.o l2cap_sock.o smp.o lib.o \
ecdh_helper.o hci_request.o mgmt_util.o mgmt_config.o hci_codec.o \
- eir.o
+ eir.o hci_sync.o
bluetooth-$(CONFIG_BT_BREDR) += sco.o
bluetooth-$(CONFIG_BT_HS) += a2mp.o amp.o
diff --git a/net/bluetooth/aosp.c b/net/bluetooth/aosp.c
index a1b7762..432ae3a 100644
--- a/net/bluetooth/aosp.c
+++ b/net/bluetooth/aosp.c
@@ -8,9 +8,43 @@
#include "aosp.h"
+/* Command complete parameters of LE_Get_Vendor_Capabilities_Command
+ * The parameters grow over time. The base version that declares the
+ * version_supported field is v0.95. Refer to
+ * https://cs.android.com/android/platform/superproject/+/master:system/
+ * bt/gd/hci/controller.cc;l=452?q=le_get_vendor_capabilities_handler
+ */
+struct aosp_rp_le_get_vendor_capa {
+ /* v0.95: 15 octets */
+ __u8 status;
+ __u8 max_advt_instances;
+ __u8 offloaded_resolution_of_private_address;
+ __le16 total_scan_results_storage;
+ __u8 max_irk_list_sz;
+ __u8 filtering_support;
+ __u8 max_filter;
+ __u8 activity_energy_info_support;
+ __le16 version_supported;
+ __le16 total_num_of_advt_tracked;
+ __u8 extended_scan_support;
+ __u8 debug_logging_supported;
+ /* v0.96: 16 octets */
+ __u8 le_address_generation_offloading_support;
+ /* v0.98: 21 octets */
+ __le32 a2dp_source_offload_capability_mask;
+ __u8 bluetooth_quality_report_support;
+ /* v1.00: 25 octets */
+ __le32 dynamic_audio_buffer_support;
+} __packed;
+
+#define VENDOR_CAPA_BASE_SIZE 15
+#define VENDOR_CAPA_0_98_SIZE 21
+
void aosp_do_open(struct hci_dev *hdev)
{
struct sk_buff *skb;
+ struct aosp_rp_le_get_vendor_capa *rp;
+ u16 version_supported;
if (!hdev->aosp_capable)
return;
@@ -20,9 +54,54 @@ void aosp_do_open(struct hci_dev *hdev)
/* LE Get Vendor Capabilities Command */
skb = __hci_cmd_sync(hdev, hci_opcode_pack(0x3f, 0x153), 0, NULL,
HCI_CMD_TIMEOUT);
- if (IS_ERR(skb))
+ if (IS_ERR(skb)) {
+ bt_dev_err(hdev, "AOSP get vendor capabilities (%ld)",
+ PTR_ERR(skb));
return;
+ }
+ /* A basic length check */
+ if (skb->len < VENDOR_CAPA_BASE_SIZE)
+ goto length_error;
+
+ rp = (struct aosp_rp_le_get_vendor_capa *)skb->data;
+
+ version_supported = le16_to_cpu(rp->version_supported);
+ /* AOSP displays the verion number like v0.98, v1.00, etc. */
+ bt_dev_info(hdev, "AOSP extensions version v%u.%02u",
+ version_supported >> 8, version_supported & 0xff);
+
+ /* Do not support very old versions. */
+ if (version_supported < 95) {
+ bt_dev_warn(hdev, "AOSP capabilities version %u too old",
+ version_supported);
+ goto done;
+ }
+
+ if (version_supported < 98) {
+ bt_dev_warn(hdev, "AOSP quality report is not supported");
+ goto done;
+ }
+
+ if (skb->len < VENDOR_CAPA_0_98_SIZE)
+ goto length_error;
+
+ /* The bluetooth_quality_report_support is defined at version
+ * v0.98. Refer to
+ * https://cs.android.com/android/platform/superproject/+/
+ * master:system/bt/gd/hci/controller.cc;l=477
+ */
+ if (rp->bluetooth_quality_report_support) {
+ hdev->aosp_quality_report = true;
+ bt_dev_info(hdev, "AOSP quality report is supported");
+ }
+
+ goto done;
+
+length_error:
+ bt_dev_err(hdev, "AOSP capabilities length %d too short", skb->len);
+
+done:
kfree_skb(skb);
}
@@ -33,3 +112,90 @@ void aosp_do_close(struct hci_dev *hdev)
bt_dev_dbg(hdev, "Cleanup of AOSP extension");
}
+
+/* BQR command */
+#define BQR_OPCODE hci_opcode_pack(0x3f, 0x015e)
+
+/* BQR report action */
+#define REPORT_ACTION_ADD 0x00
+#define REPORT_ACTION_DELETE 0x01
+#define REPORT_ACTION_CLEAR 0x02
+
+/* BQR event masks */
+#define QUALITY_MONITORING BIT(0)
+#define APPRAOCHING_LSTO BIT(1)
+#define A2DP_AUDIO_CHOPPY BIT(2)
+#define SCO_VOICE_CHOPPY BIT(3)
+
+#define DEFAULT_BQR_EVENT_MASK (QUALITY_MONITORING | APPRAOCHING_LSTO | \
+ A2DP_AUDIO_CHOPPY | SCO_VOICE_CHOPPY)
+
+/* Reporting at milliseconds so as not to stress the controller too much.
+ * Range: 0 ~ 65535 ms
+ */
+#define DEFALUT_REPORT_INTERVAL_MS 5000
+
+struct aosp_bqr_cp {
+ __u8 report_action;
+ __u32 event_mask;
+ __u16 min_report_interval;
+} __packed;
+
+static int enable_quality_report(struct hci_dev *hdev)
+{
+ struct sk_buff *skb;
+ struct aosp_bqr_cp cp;
+
+ cp.report_action = REPORT_ACTION_ADD;
+ cp.event_mask = DEFAULT_BQR_EVENT_MASK;
+ cp.min_report_interval = DEFALUT_REPORT_INTERVAL_MS;
+
+ skb = __hci_cmd_sync(hdev, BQR_OPCODE, sizeof(cp), &cp,
+ HCI_CMD_TIMEOUT);
+ if (IS_ERR(skb)) {
+ bt_dev_err(hdev, "Enabling Android BQR failed (%ld)",
+ PTR_ERR(skb));
+ return PTR_ERR(skb);
+ }
+
+ kfree_skb(skb);
+ return 0;
+}
+
+static int disable_quality_report(struct hci_dev *hdev)
+{
+ struct sk_buff *skb;
+ struct aosp_bqr_cp cp = { 0 };
+
+ cp.report_action = REPORT_ACTION_CLEAR;
+
+ skb = __hci_cmd_sync(hdev, BQR_OPCODE, sizeof(cp), &cp,
+ HCI_CMD_TIMEOUT);
+ if (IS_ERR(skb)) {
+ bt_dev_err(hdev, "Disabling Android BQR failed (%ld)",
+ PTR_ERR(skb));
+ return PTR_ERR(skb);
+ }
+
+ kfree_skb(skb);
+ return 0;
+}
+
+bool aosp_has_quality_report(struct hci_dev *hdev)
+{
+ return hdev->aosp_quality_report;
+}
+
+int aosp_set_quality_report(struct hci_dev *hdev, bool enable)
+{
+ if (!aosp_has_quality_report(hdev))
+ return -EOPNOTSUPP;
+
+ bt_dev_dbg(hdev, "quality report enable %d", enable);
+
+ /* Enable or disable the quality report feature. */
+ if (enable)
+ return enable_quality_report(hdev);
+ else
+ return disable_quality_report(hdev);
+}
diff --git a/net/bluetooth/aosp.h b/net/bluetooth/aosp.h
index 328fc6d..2fd8886 100644
--- a/net/bluetooth/aosp.h
+++ b/net/bluetooth/aosp.h
@@ -8,9 +8,22 @@
void aosp_do_open(struct hci_dev *hdev);
void aosp_do_close(struct hci_dev *hdev);
+bool aosp_has_quality_report(struct hci_dev *hdev);
+int aosp_set_quality_report(struct hci_dev *hdev, bool enable);
+
#else
static inline void aosp_do_open(struct hci_dev *hdev) {}
static inline void aosp_do_close(struct hci_dev *hdev) {}
+static inline bool aosp_has_quality_report(struct hci_dev *hdev)
+{
+ return false;
+}
+
+static inline int aosp_set_quality_report(struct hci_dev *hdev, bool enable)
+{
+ return -EOPNOTSUPP;
+}
+
#endif
diff --git a/net/bluetooth/bnep/sock.c b/net/bluetooth/bnep/sock.c
index d515571..57d509d 100644
--- a/net/bluetooth/bnep/sock.c
+++ b/net/bluetooth/bnep/sock.c
@@ -24,6 +24,7 @@
SOFTWARE IS DISCLAIMED.
*/
+#include <linux/compat.h>
#include <linux/export.h>
#include <linux/file.h>
diff --git a/net/bluetooth/cmtp/core.c b/net/bluetooth/cmtp/core.c
index 0a2d78e..83eb84e 100644
--- a/net/bluetooth/cmtp/core.c
+++ b/net/bluetooth/cmtp/core.c
@@ -501,9 +501,7 @@ static int __init cmtp_init(void)
{
BT_INFO("CMTP (CAPI Emulation) ver %s", VERSION);
- cmtp_init_sockets();
-
- return 0;
+ return cmtp_init_sockets();
}
static void __exit cmtp_exit(void)
diff --git a/net/bluetooth/eir.h b/net/bluetooth/eir.h
index 724662f..05e2e91 100644
--- a/net/bluetooth/eir.h
+++ b/net/bluetooth/eir.h
@@ -5,6 +5,8 @@
* Copyright (C) 2021 Intel Corporation
*/
+#include <asm/unaligned.h>
+
void eir_create(struct hci_dev *hdev, u8 *data);
u8 eir_create_adv_data(struct hci_dev *hdev, u8 instance, u8 *ptr);
diff --git a/net/bluetooth/hci_codec.c b/net/bluetooth/hci_codec.c
index f0421d0..3820153 100644
--- a/net/bluetooth/hci_codec.c
+++ b/net/bluetooth/hci_codec.c
@@ -25,9 +25,11 @@ static int hci_codec_list_add(struct list_head *list,
}
entry->transport = sent->transport;
entry->len = len;
- entry->num_caps = rp->num_caps;
- if (rp->num_caps)
+ entry->num_caps = 0;
+ if (rp) {
+ entry->num_caps = rp->num_caps;
memcpy(entry->caps, caps, len);
+ }
list_add(&entry->list, list);
return 0;
@@ -58,6 +60,18 @@ static void hci_read_codec_capabilities(struct hci_dev *hdev, __u8 transport,
__u32 len;
cmd->transport = i;
+
+ /* If Read_Codec_Capabilities command is not supported
+ * then just add codec to the list without caps
+ */
+ if (!(hdev->commands[45] & 0x08)) {
+ hci_dev_lock(hdev);
+ hci_codec_list_add(&hdev->local_codecs, cmd,
+ NULL, NULL, 0);
+ hci_dev_unlock(hdev);
+ continue;
+ }
+
skb = __hci_cmd_sync(hdev, HCI_OP_READ_LOCAL_CODEC_CAPS,
sizeof(*cmd), cmd,
HCI_CMD_TIMEOUT);
diff --git a/net/bluetooth/hci_conn.c b/net/bluetooth/hci_conn.c
index bd669c9..04ebe90 100644
--- a/net/bluetooth/hci_conn.c
+++ b/net/bluetooth/hci_conn.c
@@ -108,7 +108,7 @@ static void hci_connect_le_scan_cleanup(struct hci_conn *conn)
break;
}
- hci_update_background_scan(hdev);
+ hci_update_passive_scan(hdev);
}
static void hci_conn_cleanup(struct hci_conn *conn)
@@ -900,288 +900,56 @@ void hci_le_conn_failed(struct hci_conn *conn, u8 status)
hci_conn_del(conn);
- /* The suspend notifier is waiting for all devices to disconnect and an
- * LE connect cancel will result in an hci_le_conn_failed. Once the last
- * connection is deleted, we should also wake the suspend queue to
- * complete suspend operations.
- */
- if (list_empty(&hdev->conn_hash.list) &&
- test_and_clear_bit(SUSPEND_DISCONNECTING, hdev->suspend_tasks)) {
- wake_up(&hdev->suspend_wait_q);
- }
-
/* Since we may have temporarily stopped the background scanning in
* favor of connection establishment, we should restart it.
*/
- hci_update_background_scan(hdev);
+ hci_update_passive_scan(hdev);
- /* Re-enable advertising in case this was a failed connection
+ /* Enable advertising in case this was a failed connection
* attempt as a peripheral.
*/
- hci_req_reenable_advertising(hdev);
+ hci_enable_advertising(hdev);
}
-static void create_le_conn_complete(struct hci_dev *hdev, u8 status, u16 opcode)
+static void create_le_conn_complete(struct hci_dev *hdev, void *data, int err)
{
- struct hci_conn *conn;
+ struct hci_conn *conn = data;
hci_dev_lock(hdev);
- conn = hci_lookup_le_connect(hdev);
-
- if (hdev->adv_instance_cnt)
- hci_req_resume_adv_instances(hdev);
-
- if (!status) {
+ if (!err) {
hci_connect_le_scan_cleanup(conn);
goto done;
}
- bt_dev_err(hdev, "request failed to create LE connection: "
- "status 0x%2.2x", status);
+ bt_dev_err(hdev, "request failed to create LE connection: err %d", err);
if (!conn)
goto done;
- hci_le_conn_failed(conn, status);
+ hci_le_conn_failed(conn, err);
done:
hci_dev_unlock(hdev);
}
-static bool conn_use_rpa(struct hci_conn *conn)
+static int hci_connect_le_sync(struct hci_dev *hdev, void *data)
{
- struct hci_dev *hdev = conn->hdev;
+ struct hci_conn *conn = data;
- return hci_dev_test_flag(hdev, HCI_PRIVACY);
-}
+ bt_dev_dbg(hdev, "conn %p", conn);
-static void set_ext_conn_params(struct hci_conn *conn,
- struct hci_cp_le_ext_conn_param *p)
-{
- struct hci_dev *hdev = conn->hdev;
-
- memset(p, 0, sizeof(*p));
-
- p->scan_interval = cpu_to_le16(hdev->le_scan_int_connect);
- p->scan_window = cpu_to_le16(hdev->le_scan_window_connect);
- p->conn_interval_min = cpu_to_le16(conn->le_conn_min_interval);
- p->conn_interval_max = cpu_to_le16(conn->le_conn_max_interval);
- p->conn_latency = cpu_to_le16(conn->le_conn_latency);
- p->supervision_timeout = cpu_to_le16(conn->le_supv_timeout);
- p->min_ce_len = cpu_to_le16(0x0000);
- p->max_ce_len = cpu_to_le16(0x0000);
-}
-
-static void hci_req_add_le_create_conn(struct hci_request *req,
- struct hci_conn *conn,
- bdaddr_t *direct_rpa)
-{
- struct hci_dev *hdev = conn->hdev;
- u8 own_addr_type;
-
- /* If direct address was provided we use it instead of current
- * address.
- */
- if (direct_rpa) {
- if (bacmp(&req->hdev->random_addr, direct_rpa))
- hci_req_add(req, HCI_OP_LE_SET_RANDOM_ADDR, 6,
- direct_rpa);
-
- /* direct address is always RPA */
- own_addr_type = ADDR_LE_DEV_RANDOM;
- } else {
- /* Update random address, but set require_privacy to false so
- * that we never connect with an non-resolvable address.
- */
- if (hci_update_random_address(req, false, conn_use_rpa(conn),
- &own_addr_type))
- return;
- }
-
- if (use_ext_conn(hdev)) {
- struct hci_cp_le_ext_create_conn *cp;
- struct hci_cp_le_ext_conn_param *p;
- u8 data[sizeof(*cp) + sizeof(*p) * 3];
- u32 plen;
-
- cp = (void *) data;
- p = (void *) cp->data;
-
- memset(cp, 0, sizeof(*cp));
-
- bacpy(&cp->peer_addr, &conn->dst);
- cp->peer_addr_type = conn->dst_type;
- cp->own_addr_type = own_addr_type;
-
- plen = sizeof(*cp);
-
- if (scan_1m(hdev)) {
- cp->phys |= LE_SCAN_PHY_1M;
- set_ext_conn_params(conn, p);
-
- p++;
- plen += sizeof(*p);
- }
-
- if (scan_2m(hdev)) {
- cp->phys |= LE_SCAN_PHY_2M;
- set_ext_conn_params(conn, p);
-
- p++;
- plen += sizeof(*p);
- }
-
- if (scan_coded(hdev)) {
- cp->phys |= LE_SCAN_PHY_CODED;
- set_ext_conn_params(conn, p);
-
- plen += sizeof(*p);
- }
-
- hci_req_add(req, HCI_OP_LE_EXT_CREATE_CONN, plen, data);
-
- } else {
- struct hci_cp_le_create_conn cp;
-
- memset(&cp, 0, sizeof(cp));
-
- cp.scan_interval = cpu_to_le16(hdev->le_scan_int_connect);
- cp.scan_window = cpu_to_le16(hdev->le_scan_window_connect);
-
- bacpy(&cp.peer_addr, &conn->dst);
- cp.peer_addr_type = conn->dst_type;
- cp.own_address_type = own_addr_type;
- cp.conn_interval_min = cpu_to_le16(conn->le_conn_min_interval);
- cp.conn_interval_max = cpu_to_le16(conn->le_conn_max_interval);
- cp.conn_latency = cpu_to_le16(conn->le_conn_latency);
- cp.supervision_timeout = cpu_to_le16(conn->le_supv_timeout);
- cp.min_ce_len = cpu_to_le16(0x0000);
- cp.max_ce_len = cpu_to_le16(0x0000);
-
- hci_req_add(req, HCI_OP_LE_CREATE_CONN, sizeof(cp), &cp);
- }
-
- conn->state = BT_CONNECT;
- clear_bit(HCI_CONN_SCANNING, &conn->flags);
-}
-
-static void hci_req_directed_advertising(struct hci_request *req,
- struct hci_conn *conn)
-{
- struct hci_dev *hdev = req->hdev;
- u8 own_addr_type;
- u8 enable;
-
- if (ext_adv_capable(hdev)) {
- struct hci_cp_le_set_ext_adv_params cp;
- bdaddr_t random_addr;
-
- /* Set require_privacy to false so that the remote device has a
- * chance of identifying us.
- */
- if (hci_get_random_address(hdev, false, conn_use_rpa(conn), NULL,
- &own_addr_type, &random_addr) < 0)
- return;
-
- memset(&cp, 0, sizeof(cp));
-
- cp.evt_properties = cpu_to_le16(LE_LEGACY_ADV_DIRECT_IND);
- cp.own_addr_type = own_addr_type;
- cp.channel_map = hdev->le_adv_channel_map;
- cp.tx_power = HCI_TX_POWER_INVALID;
- cp.primary_phy = HCI_ADV_PHY_1M;
- cp.secondary_phy = HCI_ADV_PHY_1M;
- cp.handle = 0; /* Use instance 0 for directed adv */
- cp.own_addr_type = own_addr_type;
- cp.peer_addr_type = conn->dst_type;
- bacpy(&cp.peer_addr, &conn->dst);
-
- /* As per Core Spec 5.2 Vol 2, PART E, Sec 7.8.53, for
- * advertising_event_property LE_LEGACY_ADV_DIRECT_IND
- * does not supports advertising data when the advertising set already
- * contains some, the controller shall return erroc code 'Invalid
- * HCI Command Parameters(0x12).
- * So it is required to remove adv set for handle 0x00. since we use
- * instance 0 for directed adv.
- */
- __hci_req_remove_ext_adv_instance(req, cp.handle);
-
- hci_req_add(req, HCI_OP_LE_SET_EXT_ADV_PARAMS, sizeof(cp), &cp);
-
- if (own_addr_type == ADDR_LE_DEV_RANDOM &&
- bacmp(&random_addr, BDADDR_ANY) &&
- bacmp(&random_addr, &hdev->random_addr)) {
- struct hci_cp_le_set_adv_set_rand_addr cp;
-
- memset(&cp, 0, sizeof(cp));
-
- cp.handle = 0;
- bacpy(&cp.bdaddr, &random_addr);
-
- hci_req_add(req,
- HCI_OP_LE_SET_ADV_SET_RAND_ADDR,
- sizeof(cp), &cp);
- }
-
- __hci_req_enable_ext_advertising(req, 0x00);
- } else {
- struct hci_cp_le_set_adv_param cp;
-
- /* Clear the HCI_LE_ADV bit temporarily so that the
- * hci_update_random_address knows that it's safe to go ahead
- * and write a new random address. The flag will be set back on
- * as soon as the SET_ADV_ENABLE HCI command completes.
- */
- hci_dev_clear_flag(hdev, HCI_LE_ADV);
-
- /* Set require_privacy to false so that the remote device has a
- * chance of identifying us.
- */
- if (hci_update_random_address(req, false, conn_use_rpa(conn),
- &own_addr_type) < 0)
- return;
-
- memset(&cp, 0, sizeof(cp));
-
- /* Some controllers might reject command if intervals are not
- * within range for undirected advertising.
- * BCM20702A0 is known to be affected by this.
- */
- cp.min_interval = cpu_to_le16(0x0020);
- cp.max_interval = cpu_to_le16(0x0020);
-
- cp.type = LE_ADV_DIRECT_IND;
- cp.own_address_type = own_addr_type;
- cp.direct_addr_type = conn->dst_type;
- bacpy(&cp.direct_addr, &conn->dst);
- cp.channel_map = hdev->le_adv_channel_map;
-
- hci_req_add(req, HCI_OP_LE_SET_ADV_PARAM, sizeof(cp), &cp);
-
- enable = 0x01;
- hci_req_add(req, HCI_OP_LE_SET_ADV_ENABLE, sizeof(enable),
- &enable);
- }
-
- conn->state = BT_CONNECT;
+ return hci_le_create_conn_sync(hdev, conn);
}
struct hci_conn *hci_connect_le(struct hci_dev *hdev, bdaddr_t *dst,
u8 dst_type, bool dst_resolved, u8 sec_level,
- u16 conn_timeout, u8 role, bdaddr_t *direct_rpa)
+ u16 conn_timeout, u8 role)
{
- struct hci_conn_params *params;
struct hci_conn *conn;
struct smp_irk *irk;
- struct hci_request req;
int err;
- /* This ensures that during disable le_scan address resolution
- * will not be disabled if it is followed by le_create_conn
- */
- bool rpa_le_conn = true;
-
/* Let's make sure that le is enabled.*/
if (!hci_dev_test_flag(hdev, HCI_LE_ENABLED)) {
if (lmp_le_capable(hdev))
@@ -1240,68 +1008,13 @@ struct hci_conn *hci_connect_le(struct hci_dev *hdev, bdaddr_t *dst,
conn->sec_level = BT_SECURITY_LOW;
conn->conn_timeout = conn_timeout;
- hci_req_init(&req, hdev);
+ conn->state = BT_CONNECT;
+ clear_bit(HCI_CONN_SCANNING, &conn->flags);
- /* Disable advertising if we're active. For central role
- * connections most controllers will refuse to connect if
- * advertising is enabled, and for peripheral role connections we
- * anyway have to disable it in order to start directed
- * advertising. Any registered advertisements will be
- * re-enabled after the connection attempt is finished.
- */
- if (hci_dev_test_flag(hdev, HCI_LE_ADV))
- __hci_req_pause_adv_instances(&req);
-
- /* If requested to connect as peripheral use directed advertising */
- if (conn->role == HCI_ROLE_SLAVE) {
- /* If we're active scanning most controllers are unable
- * to initiate advertising. Simply reject the attempt.
- */
- if (hci_dev_test_flag(hdev, HCI_LE_SCAN) &&
- hdev->le_scan_type == LE_SCAN_ACTIVE) {
- hci_req_purge(&req);
- hci_conn_del(conn);
- return ERR_PTR(-EBUSY);
- }
-
- hci_req_directed_advertising(&req, conn);
- goto create_conn;
- }
-
- params = hci_conn_params_lookup(hdev, &conn->dst, conn->dst_type);
- if (params) {
- conn->le_conn_min_interval = params->conn_min_interval;
- conn->le_conn_max_interval = params->conn_max_interval;
- conn->le_conn_latency = params->conn_latency;
- conn->le_supv_timeout = params->supervision_timeout;
- } else {
- conn->le_conn_min_interval = hdev->le_conn_min_interval;
- conn->le_conn_max_interval = hdev->le_conn_max_interval;
- conn->le_conn_latency = hdev->le_conn_latency;
- conn->le_supv_timeout = hdev->le_supv_timeout;
- }
-
- /* If controller is scanning, we stop it since some controllers are
- * not able to scan and connect at the same time. Also set the
- * HCI_LE_SCAN_INTERRUPTED flag so that the command complete
- * handler for scan disabling knows to set the correct discovery
- * state.
- */
- if (hci_dev_test_flag(hdev, HCI_LE_SCAN)) {
- hci_req_add_le_scan_disable(&req, rpa_le_conn);
- hci_dev_set_flag(hdev, HCI_LE_SCAN_INTERRUPTED);
- }
-
- hci_req_add_le_create_conn(&req, conn, direct_rpa);
-
-create_conn:
- err = hci_req_run(&req, create_le_conn_complete);
+ err = hci_cmd_sync_queue(hdev, hci_connect_le_sync, conn,
+ create_le_conn_complete);
if (err) {
hci_conn_del(conn);
-
- if (hdev->adv_instance_cnt)
- hci_req_resume_adv_instances(hdev);
-
return ERR_PTR(err);
}
@@ -1411,7 +1124,7 @@ struct hci_conn *hci_connect_le_scan(struct hci_dev *hdev, bdaddr_t *dst,
conn->conn_timeout = conn_timeout;
conn->conn_reason = conn_reason;
- hci_update_background_scan(hdev);
+ hci_update_passive_scan(hdev);
done:
hci_conn_hold(conn);
diff --git a/net/bluetooth/hci_core.c b/net/bluetooth/hci_core.c
index 8d33aa6..2b7bd36 100644
--- a/net/bluetooth/hci_core.c
+++ b/net/bluetooth/hci_core.c
@@ -62,824 +62,6 @@ DEFINE_MUTEX(hci_cb_list_lock);
/* HCI ID Numbering */
static DEFINE_IDA(hci_index_ida);
-static int hci_reset_req(struct hci_request *req, unsigned long opt)
-{
- BT_DBG("%s %ld", req->hdev->name, opt);
-
- /* Reset device */
- set_bit(HCI_RESET, &req->hdev->flags);
- hci_req_add(req, HCI_OP_RESET, 0, NULL);
- return 0;
-}
-
-static void bredr_init(struct hci_request *req)
-{
- req->hdev->flow_ctl_mode = HCI_FLOW_CTL_MODE_PACKET_BASED;
-
- /* Read Local Supported Features */
- hci_req_add(req, HCI_OP_READ_LOCAL_FEATURES, 0, NULL);
-
- /* Read Local Version */
- hci_req_add(req, HCI_OP_READ_LOCAL_VERSION, 0, NULL);
-
- /* Read BD Address */
- hci_req_add(req, HCI_OP_READ_BD_ADDR, 0, NULL);
-}
-
-static void amp_init1(struct hci_request *req)
-{
- req->hdev->flow_ctl_mode = HCI_FLOW_CTL_MODE_BLOCK_BASED;
-
- /* Read Local Version */
- hci_req_add(req, HCI_OP_READ_LOCAL_VERSION, 0, NULL);
-
- /* Read Local Supported Commands */
- hci_req_add(req, HCI_OP_READ_LOCAL_COMMANDS, 0, NULL);
-
- /* Read Local AMP Info */
- hci_req_add(req, HCI_OP_READ_LOCAL_AMP_INFO, 0, NULL);
-
- /* Read Data Blk size */
- hci_req_add(req, HCI_OP_READ_DATA_BLOCK_SIZE, 0, NULL);
-
- /* Read Flow Control Mode */
- hci_req_add(req, HCI_OP_READ_FLOW_CONTROL_MODE, 0, NULL);
-
- /* Read Location Data */
- hci_req_add(req, HCI_OP_READ_LOCATION_DATA, 0, NULL);
-}
-
-static int amp_init2(struct hci_request *req)
-{
- /* Read Local Supported Features. Not all AMP controllers
- * support this so it's placed conditionally in the second
- * stage init.
- */
- if (req->hdev->commands[14] & 0x20)
- hci_req_add(req, HCI_OP_READ_LOCAL_FEATURES, 0, NULL);
-
- return 0;
-}
-
-static int hci_init1_req(struct hci_request *req, unsigned long opt)
-{
- struct hci_dev *hdev = req->hdev;
-
- BT_DBG("%s %ld", hdev->name, opt);
-
- /* Reset */
- if (!test_bit(HCI_QUIRK_RESET_ON_CLOSE, &hdev->quirks))
- hci_reset_req(req, 0);
-
- switch (hdev->dev_type) {
- case HCI_PRIMARY:
- bredr_init(req);
- break;
- case HCI_AMP:
- amp_init1(req);
- break;
- default:
- bt_dev_err(hdev, "Unknown device type %d", hdev->dev_type);
- break;
- }
-
- return 0;
-}
-
-static void bredr_setup(struct hci_request *req)
-{
- __le16 param;
- __u8 flt_type;
-
- /* Read Buffer Size (ACL mtu, max pkt, etc.) */
- hci_req_add(req, HCI_OP_READ_BUFFER_SIZE, 0, NULL);
-
- /* Read Class of Device */
- hci_req_add(req, HCI_OP_READ_CLASS_OF_DEV, 0, NULL);
-
- /* Read Local Name */
- hci_req_add(req, HCI_OP_READ_LOCAL_NAME, 0, NULL);
-
- /* Read Voice Setting */
- hci_req_add(req, HCI_OP_READ_VOICE_SETTING, 0, NULL);
-
- /* Read Number of Supported IAC */
- hci_req_add(req, HCI_OP_READ_NUM_SUPPORTED_IAC, 0, NULL);
-
- /* Read Current IAC LAP */
- hci_req_add(req, HCI_OP_READ_CURRENT_IAC_LAP, 0, NULL);
-
- /* Clear Event Filters */
- flt_type = HCI_FLT_CLEAR_ALL;
- hci_req_add(req, HCI_OP_SET_EVENT_FLT, 1, &flt_type);
-
- /* Connection accept timeout ~20 secs */
- param = cpu_to_le16(0x7d00);
- hci_req_add(req, HCI_OP_WRITE_CA_TIMEOUT, 2, ¶m);
-}
-
-static void le_setup(struct hci_request *req)
-{
- struct hci_dev *hdev = req->hdev;
-
- /* Read LE Buffer Size */
- hci_req_add(req, HCI_OP_LE_READ_BUFFER_SIZE, 0, NULL);
-
- /* Read LE Local Supported Features */
- hci_req_add(req, HCI_OP_LE_READ_LOCAL_FEATURES, 0, NULL);
-
- /* Read LE Supported States */
- hci_req_add(req, HCI_OP_LE_READ_SUPPORTED_STATES, 0, NULL);
-
- /* LE-only controllers have LE implicitly enabled */
- if (!lmp_bredr_capable(hdev))
- hci_dev_set_flag(hdev, HCI_LE_ENABLED);
-}
-
-static void hci_setup_event_mask(struct hci_request *req)
-{
- struct hci_dev *hdev = req->hdev;
-
- /* The second byte is 0xff instead of 0x9f (two reserved bits
- * disabled) since a Broadcom 1.2 dongle doesn't respond to the
- * command otherwise.
- */
- u8 events[8] = { 0xff, 0xff, 0xfb, 0xff, 0x00, 0x00, 0x00, 0x00 };
-
- /* CSR 1.1 dongles does not accept any bitfield so don't try to set
- * any event mask for pre 1.2 devices.
- */
- if (hdev->hci_ver < BLUETOOTH_VER_1_2)
- return;
-
- if (lmp_bredr_capable(hdev)) {
- events[4] |= 0x01; /* Flow Specification Complete */
- } else {
- /* Use a different default for LE-only devices */
- memset(events, 0, sizeof(events));
- events[1] |= 0x20; /* Command Complete */
- events[1] |= 0x40; /* Command Status */
- events[1] |= 0x80; /* Hardware Error */
-
- /* If the controller supports the Disconnect command, enable
- * the corresponding event. In addition enable packet flow
- * control related events.
- */
- if (hdev->commands[0] & 0x20) {
- events[0] |= 0x10; /* Disconnection Complete */
- events[2] |= 0x04; /* Number of Completed Packets */
- events[3] |= 0x02; /* Data Buffer Overflow */
- }
-
- /* If the controller supports the Read Remote Version
- * Information command, enable the corresponding event.
- */
- if (hdev->commands[2] & 0x80)
- events[1] |= 0x08; /* Read Remote Version Information
- * Complete
- */
-
- if (hdev->le_features[0] & HCI_LE_ENCRYPTION) {
- events[0] |= 0x80; /* Encryption Change */
- events[5] |= 0x80; /* Encryption Key Refresh Complete */
- }
- }
-
- if (lmp_inq_rssi_capable(hdev) ||
- test_bit(HCI_QUIRK_FIXUP_INQUIRY_MODE, &hdev->quirks))
- events[4] |= 0x02; /* Inquiry Result with RSSI */
-
- if (lmp_ext_feat_capable(hdev))
- events[4] |= 0x04; /* Read Remote Extended Features Complete */
-
- if (lmp_esco_capable(hdev)) {
- events[5] |= 0x08; /* Synchronous Connection Complete */
- events[5] |= 0x10; /* Synchronous Connection Changed */
- }
-
- if (lmp_sniffsubr_capable(hdev))
- events[5] |= 0x20; /* Sniff Subrating */
-
- if (lmp_pause_enc_capable(hdev))
- events[5] |= 0x80; /* Encryption Key Refresh Complete */
-
- if (lmp_ext_inq_capable(hdev))
- events[5] |= 0x40; /* Extended Inquiry Result */
-
- if (lmp_no_flush_capable(hdev))
- events[7] |= 0x01; /* Enhanced Flush Complete */
-
- if (lmp_lsto_capable(hdev))
- events[6] |= 0x80; /* Link Supervision Timeout Changed */
-
- if (lmp_ssp_capable(hdev)) {
- events[6] |= 0x01; /* IO Capability Request */
- events[6] |= 0x02; /* IO Capability Response */
- events[6] |= 0x04; /* User Confirmation Request */
- events[6] |= 0x08; /* User Passkey Request */
- events[6] |= 0x10; /* Remote OOB Data Request */
- events[6] |= 0x20; /* Simple Pairing Complete */
- events[7] |= 0x04; /* User Passkey Notification */
- events[7] |= 0x08; /* Keypress Notification */
- events[7] |= 0x10; /* Remote Host Supported
- * Features Notification
- */
- }
-
- if (lmp_le_capable(hdev))
- events[7] |= 0x20; /* LE Meta-Event */
-
- hci_req_add(req, HCI_OP_SET_EVENT_MASK, sizeof(events), events);
-}
-
-static int hci_init2_req(struct hci_request *req, unsigned long opt)
-{
- struct hci_dev *hdev = req->hdev;
-
- if (hdev->dev_type == HCI_AMP)
- return amp_init2(req);
-
- if (lmp_bredr_capable(hdev))
- bredr_setup(req);
- else
- hci_dev_clear_flag(hdev, HCI_BREDR_ENABLED);
-
- if (lmp_le_capable(hdev))
- le_setup(req);
-
- /* All Bluetooth 1.2 and later controllers should support the
- * HCI command for reading the local supported commands.
- *
- * Unfortunately some controllers indicate Bluetooth 1.2 support,
- * but do not have support for this command. If that is the case,
- * the driver can quirk the behavior and skip reading the local
- * supported commands.
- */
- if (hdev->hci_ver > BLUETOOTH_VER_1_1 &&
- !test_bit(HCI_QUIRK_BROKEN_LOCAL_COMMANDS, &hdev->quirks))
- hci_req_add(req, HCI_OP_READ_LOCAL_COMMANDS, 0, NULL);
-
- if (lmp_ssp_capable(hdev)) {
- /* When SSP is available, then the host features page
- * should also be available as well. However some
- * controllers list the max_page as 0 as long as SSP
- * has not been enabled. To achieve proper debugging
- * output, force the minimum max_page to 1 at least.
- */
- hdev->max_page = 0x01;
-
- if (hci_dev_test_flag(hdev, HCI_SSP_ENABLED)) {
- u8 mode = 0x01;
-
- hci_req_add(req, HCI_OP_WRITE_SSP_MODE,
- sizeof(mode), &mode);
- } else {
- struct hci_cp_write_eir cp;
-
- memset(hdev->eir, 0, sizeof(hdev->eir));
- memset(&cp, 0, sizeof(cp));
-
- hci_req_add(req, HCI_OP_WRITE_EIR, sizeof(cp), &cp);
- }
- }
-
- if (lmp_inq_rssi_capable(hdev) ||
- test_bit(HCI_QUIRK_FIXUP_INQUIRY_MODE, &hdev->quirks)) {
- u8 mode;
-
- /* If Extended Inquiry Result events are supported, then
- * they are clearly preferred over Inquiry Result with RSSI
- * events.
- */
- mode = lmp_ext_inq_capable(hdev) ? 0x02 : 0x01;
-
- hci_req_add(req, HCI_OP_WRITE_INQUIRY_MODE, 1, &mode);
- }
-
- if (lmp_inq_tx_pwr_capable(hdev))
- hci_req_add(req, HCI_OP_READ_INQ_RSP_TX_POWER, 0, NULL);
-
- if (lmp_ext_feat_capable(hdev)) {
- struct hci_cp_read_local_ext_features cp;
-
- cp.page = 0x01;
- hci_req_add(req, HCI_OP_READ_LOCAL_EXT_FEATURES,
- sizeof(cp), &cp);
- }
-
- if (hci_dev_test_flag(hdev, HCI_LINK_SECURITY)) {
- u8 enable = 1;
- hci_req_add(req, HCI_OP_WRITE_AUTH_ENABLE, sizeof(enable),
- &enable);
- }
-
- return 0;
-}
-
-static void hci_setup_link_policy(struct hci_request *req)
-{
- struct hci_dev *hdev = req->hdev;
- struct hci_cp_write_def_link_policy cp;
- u16 link_policy = 0;
-
- if (lmp_rswitch_capable(hdev))
- link_policy |= HCI_LP_RSWITCH;
- if (lmp_hold_capable(hdev))
- link_policy |= HCI_LP_HOLD;
- if (lmp_sniff_capable(hdev))
- link_policy |= HCI_LP_SNIFF;
- if (lmp_park_capable(hdev))
- link_policy |= HCI_LP_PARK;
-
- cp.policy = cpu_to_le16(link_policy);
- hci_req_add(req, HCI_OP_WRITE_DEF_LINK_POLICY, sizeof(cp), &cp);
-}
-
-static void hci_set_le_support(struct hci_request *req)
-{
- struct hci_dev *hdev = req->hdev;
- struct hci_cp_write_le_host_supported cp;
-
- /* LE-only devices do not support explicit enablement */
- if (!lmp_bredr_capable(hdev))
- return;
-
- memset(&cp, 0, sizeof(cp));
-
- if (hci_dev_test_flag(hdev, HCI_LE_ENABLED)) {
- cp.le = 0x01;
- cp.simul = 0x00;
- }
-
- if (cp.le != lmp_host_le_capable(hdev))
- hci_req_add(req, HCI_OP_WRITE_LE_HOST_SUPPORTED, sizeof(cp),
- &cp);
-}
-
-static void hci_set_event_mask_page_2(struct hci_request *req)
-{
- struct hci_dev *hdev = req->hdev;
- u8 events[8] = { 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 };
- bool changed = false;
-
- /* If Connectionless Peripheral Broadcast central role is supported
- * enable all necessary events for it.
- */
- if (lmp_cpb_central_capable(hdev)) {
- events[1] |= 0x40; /* Triggered Clock Capture */
- events[1] |= 0x80; /* Synchronization Train Complete */
- events[2] |= 0x10; /* Peripheral Page Response Timeout */
- events[2] |= 0x20; /* CPB Channel Map Change */
- changed = true;
- }
-
- /* If Connectionless Peripheral Broadcast peripheral role is supported
- * enable all necessary events for it.
- */
- if (lmp_cpb_peripheral_capable(hdev)) {
- events[2] |= 0x01; /* Synchronization Train Received */
- events[2] |= 0x02; /* CPB Receive */
- events[2] |= 0x04; /* CPB Timeout */
- events[2] |= 0x08; /* Truncated Page Complete */
- changed = true;
- }
-
- /* Enable Authenticated Payload Timeout Expired event if supported */
- if (lmp_ping_capable(hdev) || hdev->le_features[0] & HCI_LE_PING) {
- events[2] |= 0x80;
- changed = true;
- }
-
- /* Some Broadcom based controllers indicate support for Set Event
- * Mask Page 2 command, but then actually do not support it. Since
- * the default value is all bits set to zero, the command is only
- * required if the event mask has to be changed. In case no change
- * to the event mask is needed, skip this command.
- */
- if (changed)
- hci_req_add(req, HCI_OP_SET_EVENT_MASK_PAGE_2,
- sizeof(events), events);
-}
-
-static int hci_init3_req(struct hci_request *req, unsigned long opt)
-{
- struct hci_dev *hdev = req->hdev;
- u8 p;
-
- hci_setup_event_mask(req);
-
- if (hdev->commands[6] & 0x20 &&
- !test_bit(HCI_QUIRK_BROKEN_STORED_LINK_KEY, &hdev->quirks)) {
- struct hci_cp_read_stored_link_key cp;
-
- bacpy(&cp.bdaddr, BDADDR_ANY);
- cp.read_all = 0x01;
- hci_req_add(req, HCI_OP_READ_STORED_LINK_KEY, sizeof(cp), &cp);
- }
-
- if (hdev->commands[5] & 0x10)
- hci_setup_link_policy(req);
-
- if (hdev->commands[8] & 0x01)
- hci_req_add(req, HCI_OP_READ_PAGE_SCAN_ACTIVITY, 0, NULL);
-
- if (hdev->commands[18] & 0x04 &&
- !test_bit(HCI_QUIRK_BROKEN_ERR_DATA_REPORTING, &hdev->quirks))
- hci_req_add(req, HCI_OP_READ_DEF_ERR_DATA_REPORTING, 0, NULL);
-
- /* Some older Broadcom based Bluetooth 1.2 controllers do not
- * support the Read Page Scan Type command. Check support for
- * this command in the bit mask of supported commands.
- */
- if (hdev->commands[13] & 0x01)
- hci_req_add(req, HCI_OP_READ_PAGE_SCAN_TYPE, 0, NULL);
-
- if (lmp_le_capable(hdev)) {
- u8 events[8];
-
- memset(events, 0, sizeof(events));
-
- if (hdev->le_features[0] & HCI_LE_ENCRYPTION)
- events[0] |= 0x10; /* LE Long Term Key Request */
-
- /* If controller supports the Connection Parameters Request
- * Link Layer Procedure, enable the corresponding event.
- */
- if (hdev->le_features[0] & HCI_LE_CONN_PARAM_REQ_PROC)
- events[0] |= 0x20; /* LE Remote Connection
- * Parameter Request
- */
-
- /* If the controller supports the Data Length Extension
- * feature, enable the corresponding event.
- */
- if (hdev->le_features[0] & HCI_LE_DATA_LEN_EXT)
- events[0] |= 0x40; /* LE Data Length Change */
-
- /* If the controller supports LL Privacy feature, enable
- * the corresponding event.
- */
- if (hdev->le_features[0] & HCI_LE_LL_PRIVACY)
- events[1] |= 0x02; /* LE Enhanced Connection
- * Complete
- */
-
- /* If the controller supports Extended Scanner Filter
- * Policies, enable the corresponding event.
- */
- if (hdev->le_features[0] & HCI_LE_EXT_SCAN_POLICY)
- events[1] |= 0x04; /* LE Direct Advertising
- * Report
- */
-
- /* If the controller supports Channel Selection Algorithm #2
- * feature, enable the corresponding event.
- */
- if (hdev->le_features[1] & HCI_LE_CHAN_SEL_ALG2)
- events[2] |= 0x08; /* LE Channel Selection
- * Algorithm
- */
-
- /* If the controller supports the LE Set Scan Enable command,
- * enable the corresponding advertising report event.
- */
- if (hdev->commands[26] & 0x08)
- events[0] |= 0x02; /* LE Advertising Report */
-
- /* If the controller supports the LE Create Connection
- * command, enable the corresponding event.
- */
- if (hdev->commands[26] & 0x10)
- events[0] |= 0x01; /* LE Connection Complete */
-
- /* If the controller supports the LE Connection Update
- * command, enable the corresponding event.
- */
- if (hdev->commands[27] & 0x04)
- events[0] |= 0x04; /* LE Connection Update
- * Complete
- */
-
- /* If the controller supports the LE Read Remote Used Features
- * command, enable the corresponding event.
- */
- if (hdev->commands[27] & 0x20)
- events[0] |= 0x08; /* LE Read Remote Used
- * Features Complete
- */
-
- /* If the controller supports the LE Read Local P-256
- * Public Key command, enable the corresponding event.
- */
- if (hdev->commands[34] & 0x02)
- events[0] |= 0x80; /* LE Read Local P-256
- * Public Key Complete
- */
-
- /* If the controller supports the LE Generate DHKey
- * command, enable the corresponding event.
- */
- if (hdev->commands[34] & 0x04)
- events[1] |= 0x01; /* LE Generate DHKey Complete */
-
- /* If the controller supports the LE Set Default PHY or
- * LE Set PHY commands, enable the corresponding event.
- */
- if (hdev->commands[35] & (0x20 | 0x40))
- events[1] |= 0x08; /* LE PHY Update Complete */
-
- /* If the controller supports LE Set Extended Scan Parameters
- * and LE Set Extended Scan Enable commands, enable the
- * corresponding event.
- */
- if (use_ext_scan(hdev))
- events[1] |= 0x10; /* LE Extended Advertising
- * Report
- */
-
- /* If the controller supports the LE Extended Advertising
- * command, enable the corresponding event.
- */
- if (ext_adv_capable(hdev))
- events[2] |= 0x02; /* LE Advertising Set
- * Terminated
- */
-
- hci_req_add(req, HCI_OP_LE_SET_EVENT_MASK, sizeof(events),
- events);
-
- /* Read LE Advertising Channel TX Power */
- if ((hdev->commands[25] & 0x40) && !ext_adv_capable(hdev)) {
- /* HCI TS spec forbids mixing of legacy and extended
- * advertising commands wherein READ_ADV_TX_POWER is
- * also included. So do not call it if extended adv
- * is supported otherwise controller will return
- * COMMAND_DISALLOWED for extended commands.
- */
- hci_req_add(req, HCI_OP_LE_READ_ADV_TX_POWER, 0, NULL);
- }
-
- if (hdev->commands[38] & 0x80) {
- /* Read LE Min/Max Tx Power*/
- hci_req_add(req, HCI_OP_LE_READ_TRANSMIT_POWER,
- 0, NULL);
- }
-
- if (hdev->commands[26] & 0x40) {
- /* Read LE Accept List Size */
- hci_req_add(req, HCI_OP_LE_READ_ACCEPT_LIST_SIZE,
- 0, NULL);
- }
-
- if (hdev->commands[26] & 0x80) {
- /* Clear LE Accept List */
- hci_req_add(req, HCI_OP_LE_CLEAR_ACCEPT_LIST, 0, NULL);
- }
-
- if (hdev->commands[34] & 0x40) {
- /* Read LE Resolving List Size */
- hci_req_add(req, HCI_OP_LE_READ_RESOLV_LIST_SIZE,
- 0, NULL);
- }
-
- if (hdev->commands[34] & 0x20) {
- /* Clear LE Resolving List */
- hci_req_add(req, HCI_OP_LE_CLEAR_RESOLV_LIST, 0, NULL);
- }
-
- if (hdev->commands[35] & 0x04) {
- __le16 rpa_timeout = cpu_to_le16(hdev->rpa_timeout);
-
- /* Set RPA timeout */
- hci_req_add(req, HCI_OP_LE_SET_RPA_TIMEOUT, 2,
- &rpa_timeout);
- }
-
- if (hdev->le_features[0] & HCI_LE_DATA_LEN_EXT) {
- /* Read LE Maximum Data Length */
- hci_req_add(req, HCI_OP_LE_READ_MAX_DATA_LEN, 0, NULL);
-
- /* Read LE Suggested Default Data Length */
- hci_req_add(req, HCI_OP_LE_READ_DEF_DATA_LEN, 0, NULL);
- }
-
- if (ext_adv_capable(hdev)) {
- /* Read LE Number of Supported Advertising Sets */
- hci_req_add(req, HCI_OP_LE_READ_NUM_SUPPORTED_ADV_SETS,
- 0, NULL);
- }
-
- hci_set_le_support(req);
- }
-
- /* Read features beyond page 1 if available */
- for (p = 2; p < HCI_MAX_PAGES && p <= hdev->max_page; p++) {
- struct hci_cp_read_local_ext_features cp;
-
- cp.page = p;
- hci_req_add(req, HCI_OP_READ_LOCAL_EXT_FEATURES,
- sizeof(cp), &cp);
- }
-
- return 0;
-}
-
-static int hci_init4_req(struct hci_request *req, unsigned long opt)
-{
- struct hci_dev *hdev = req->hdev;
-
- /* Some Broadcom based Bluetooth controllers do not support the
- * Delete Stored Link Key command. They are clearly indicating its
- * absence in the bit mask of supported commands.
- *
- * Check the supported commands and only if the command is marked
- * as supported send it. If not supported assume that the controller
- * does not have actual support for stored link keys which makes this
- * command redundant anyway.
- *
- * Some controllers indicate that they support handling deleting
- * stored link keys, but they don't. The quirk lets a driver
- * just disable this command.
- */
- if (hdev->commands[6] & 0x80 &&
- !test_bit(HCI_QUIRK_BROKEN_STORED_LINK_KEY, &hdev->quirks)) {
- struct hci_cp_delete_stored_link_key cp;
-
- bacpy(&cp.bdaddr, BDADDR_ANY);
- cp.delete_all = 0x01;
- hci_req_add(req, HCI_OP_DELETE_STORED_LINK_KEY,
- sizeof(cp), &cp);
- }
-
- /* Set event mask page 2 if the HCI command for it is supported */
- if (hdev->commands[22] & 0x04)
- hci_set_event_mask_page_2(req);
-
- /* Read local pairing options if the HCI command is supported */
- if (hdev->commands[41] & 0x08)
- hci_req_add(req, HCI_OP_READ_LOCAL_PAIRING_OPTS, 0, NULL);
-
- /* Get MWS transport configuration if the HCI command is supported */
- if (hdev->commands[30] & 0x08)
- hci_req_add(req, HCI_OP_GET_MWS_TRANSPORT_CONFIG, 0, NULL);
-
- /* Check for Synchronization Train support */
- if (lmp_sync_train_capable(hdev))
- hci_req_add(req, HCI_OP_READ_SYNC_TRAIN_PARAMS, 0, NULL);
-
- /* Enable Secure Connections if supported and configured */
- if (hci_dev_test_flag(hdev, HCI_SSP_ENABLED) &&
- bredr_sc_enabled(hdev)) {
- u8 support = 0x01;
-
- hci_req_add(req, HCI_OP_WRITE_SC_SUPPORT,
- sizeof(support), &support);
- }
-
- /* Set erroneous data reporting if supported to the wideband speech
- * setting value
- */
- if (hdev->commands[18] & 0x08 &&
- !test_bit(HCI_QUIRK_BROKEN_ERR_DATA_REPORTING, &hdev->quirks)) {
- bool enabled = hci_dev_test_flag(hdev,
- HCI_WIDEBAND_SPEECH_ENABLED);
-
- if (enabled !=
- (hdev->err_data_reporting == ERR_DATA_REPORTING_ENABLED)) {
- struct hci_cp_write_def_err_data_reporting cp;
-
- cp.err_data_reporting = enabled ?
- ERR_DATA_REPORTING_ENABLED :
- ERR_DATA_REPORTING_DISABLED;
-
- hci_req_add(req, HCI_OP_WRITE_DEF_ERR_DATA_REPORTING,
- sizeof(cp), &cp);
- }
- }
-
- /* Set Suggested Default Data Length to maximum if supported */
- if (hdev->le_features[0] & HCI_LE_DATA_LEN_EXT) {
- struct hci_cp_le_write_def_data_len cp;
-
- cp.tx_len = cpu_to_le16(hdev->le_max_tx_len);
- cp.tx_time = cpu_to_le16(hdev->le_max_tx_time);
- hci_req_add(req, HCI_OP_LE_WRITE_DEF_DATA_LEN, sizeof(cp), &cp);
- }
-
- /* Set Default PHY parameters if command is supported */
- if (hdev->commands[35] & 0x20) {
- struct hci_cp_le_set_default_phy cp;
-
- cp.all_phys = 0x00;
- cp.tx_phys = hdev->le_tx_def_phys;
- cp.rx_phys = hdev->le_rx_def_phys;
-
- hci_req_add(req, HCI_OP_LE_SET_DEFAULT_PHY, sizeof(cp), &cp);
- }
-
- return 0;
-}
-
-static int __hci_init(struct hci_dev *hdev)
-{
- int err;
-
- err = __hci_req_sync(hdev, hci_init1_req, 0, HCI_INIT_TIMEOUT, NULL);
- if (err < 0)
- return err;
-
- if (hci_dev_test_flag(hdev, HCI_SETUP))
- hci_debugfs_create_basic(hdev);
-
- err = __hci_req_sync(hdev, hci_init2_req, 0, HCI_INIT_TIMEOUT, NULL);
- if (err < 0)
- return err;
-
- /* HCI_PRIMARY covers both single-mode LE, BR/EDR and dual-mode
- * BR/EDR/LE type controllers. AMP controllers only need the
- * first two stages of init.
- */
- if (hdev->dev_type != HCI_PRIMARY)
- return 0;
-
- err = __hci_req_sync(hdev, hci_init3_req, 0, HCI_INIT_TIMEOUT, NULL);
- if (err < 0)
- return err;
-
- err = __hci_req_sync(hdev, hci_init4_req, 0, HCI_INIT_TIMEOUT, NULL);
- if (err < 0)
- return err;
-
- /* Read local codec list if the HCI command is supported */
- if (hdev->commands[45] & 0x04)
- hci_read_supported_codecs_v2(hdev);
- else if (hdev->commands[29] & 0x20)
- hci_read_supported_codecs(hdev);
-
- /* This function is only called when the controller is actually in
- * configured state. When the controller is marked as unconfigured,
- * this initialization procedure is not run.
- *
- * It means that it is possible that a controller runs through its
- * setup phase and then discovers missing settings. If that is the
- * case, then this function will not be called. It then will only
- * be called during the config phase.
- *
- * So only when in setup phase or config phase, create the debugfs
- * entries and register the SMP channels.
- */
- if (!hci_dev_test_flag(hdev, HCI_SETUP) &&
- !hci_dev_test_flag(hdev, HCI_CONFIG))
- return 0;
-
- hci_debugfs_create_common(hdev);
-
- if (lmp_bredr_capable(hdev))
- hci_debugfs_create_bredr(hdev);
-
- if (lmp_le_capable(hdev))
- hci_debugfs_create_le(hdev);
-
- return 0;
-}
-
-static int hci_init0_req(struct hci_request *req, unsigned long opt)
-{
- struct hci_dev *hdev = req->hdev;
-
- BT_DBG("%s %ld", hdev->name, opt);
-
- /* Reset */
- if (!test_bit(HCI_QUIRK_RESET_ON_CLOSE, &hdev->quirks))
- hci_reset_req(req, 0);
-
- /* Read Local Version */
- hci_req_add(req, HCI_OP_READ_LOCAL_VERSION, 0, NULL);
-
- /* Read BD Address */
- if (hdev->set_bdaddr)
- hci_req_add(req, HCI_OP_READ_BD_ADDR, 0, NULL);
-
- return 0;
-}
-
-static int __hci_unconf_init(struct hci_dev *hdev)
-{
- int err;
-
- if (test_bit(HCI_QUIRK_RAW_DEVICE, &hdev->quirks))
- return 0;
-
- err = __hci_req_sync(hdev, hci_init0_req, 0, HCI_INIT_TIMEOUT, NULL);
- if (err < 0)
- return err;
-
- if (hci_dev_test_flag(hdev, HCI_SETUP))
- hci_debugfs_create_basic(hdev);
-
- return 0;
-}
-
static int hci_scan_req(struct hci_request *req, unsigned long opt)
{
__u8 scan = opt;
@@ -975,7 +157,7 @@ void hci_discovery_set_state(struct hci_dev *hdev, int state)
switch (state) {
case DISCOVERY_STOPPED:
- hci_update_background_scan(hdev);
+ hci_update_passive_scan(hdev);
if (old_state != DISCOVERY_STARTING)
mgmt_discovering(hdev, 0);
@@ -1289,32 +471,6 @@ int hci_inquiry(void __user *arg)
return err;
}
-/**
- * hci_dev_get_bd_addr_from_property - Get the Bluetooth Device Address
- * (BD_ADDR) for a HCI device from
- * a firmware node property.
- * @hdev: The HCI device
- *
- * Search the firmware node for 'local-bd-address'.
- *
- * All-zero BD addresses are rejected, because those could be properties
- * that exist in the firmware tables, but were not updated by the firmware. For
- * example, the DTS could define 'local-bd-address', with zero BD addresses.
- */
-static void hci_dev_get_bd_addr_from_property(struct hci_dev *hdev)
-{
- struct fwnode_handle *fwnode = dev_fwnode(hdev->dev.parent);
- bdaddr_t ba;
- int ret;
-
- ret = fwnode_property_read_u8_array(fwnode, "local-bd-address",
- (u8 *)&ba, sizeof(ba));
- if (ret < 0 || !bacmp(&ba, BDADDR_ANY))
- return;
-
- bacpy(&hdev->public_addr, &ba);
-}
-
static int hci_dev_do_open(struct hci_dev *hdev)
{
int ret = 0;
@@ -1323,205 +479,8 @@ static int hci_dev_do_open(struct hci_dev *hdev)
hci_req_sync_lock(hdev);
- if (hci_dev_test_flag(hdev, HCI_UNREGISTER)) {
- ret = -ENODEV;
- goto done;
- }
+ ret = hci_dev_open_sync(hdev);
- if (!hci_dev_test_flag(hdev, HCI_SETUP) &&
- !hci_dev_test_flag(hdev, HCI_CONFIG)) {
- /* Check for rfkill but allow the HCI setup stage to
- * proceed (which in itself doesn't cause any RF activity).
- */
- if (hci_dev_test_flag(hdev, HCI_RFKILLED)) {
- ret = -ERFKILL;
- goto done;
- }
-
- /* Check for valid public address or a configured static
- * random address, but let the HCI setup proceed to
- * be able to determine if there is a public address
- * or not.
- *
- * In case of user channel usage, it is not important
- * if a public address or static random address is
- * available.
- *
- * This check is only valid for BR/EDR controllers
- * since AMP controllers do not have an address.
- */
- if (!hci_dev_test_flag(hdev, HCI_USER_CHANNEL) &&
- hdev->dev_type == HCI_PRIMARY &&
- !bacmp(&hdev->bdaddr, BDADDR_ANY) &&
- !bacmp(&hdev->static_addr, BDADDR_ANY)) {
- ret = -EADDRNOTAVAIL;
- goto done;
- }
- }
-
- if (test_bit(HCI_UP, &hdev->flags)) {
- ret = -EALREADY;
- goto done;
- }
-
- if (hdev->open(hdev)) {
- ret = -EIO;
- goto done;
- }
-
- set_bit(HCI_RUNNING, &hdev->flags);
- hci_sock_dev_event(hdev, HCI_DEV_OPEN);
-
- atomic_set(&hdev->cmd_cnt, 1);
- set_bit(HCI_INIT, &hdev->flags);
-
- if (hci_dev_test_flag(hdev, HCI_SETUP) ||
- test_bit(HCI_QUIRK_NON_PERSISTENT_SETUP, &hdev->quirks)) {
- bool invalid_bdaddr;
-
- hci_sock_dev_event(hdev, HCI_DEV_SETUP);
-
- if (hdev->setup)
- ret = hdev->setup(hdev);
-
- /* The transport driver can set the quirk to mark the
- * BD_ADDR invalid before creating the HCI device or in
- * its setup callback.
- */
- invalid_bdaddr = test_bit(HCI_QUIRK_INVALID_BDADDR,
- &hdev->quirks);
-
- if (ret)
- goto setup_failed;
-
- if (test_bit(HCI_QUIRK_USE_BDADDR_PROPERTY, &hdev->quirks)) {
- if (!bacmp(&hdev->public_addr, BDADDR_ANY))
- hci_dev_get_bd_addr_from_property(hdev);
-
- if (bacmp(&hdev->public_addr, BDADDR_ANY) &&
- hdev->set_bdaddr) {
- ret = hdev->set_bdaddr(hdev,
- &hdev->public_addr);
-
- /* If setting of the BD_ADDR from the device
- * property succeeds, then treat the address
- * as valid even if the invalid BD_ADDR
- * quirk indicates otherwise.
- */
- if (!ret)
- invalid_bdaddr = false;
- }
- }
-
-setup_failed:
- /* The transport driver can set these quirks before
- * creating the HCI device or in its setup callback.
- *
- * For the invalid BD_ADDR quirk it is possible that
- * it becomes a valid address if the bootloader does
- * provide it (see above).
- *
- * In case any of them is set, the controller has to
- * start up as unconfigured.
- */
- if (test_bit(HCI_QUIRK_EXTERNAL_CONFIG, &hdev->quirks) ||
- invalid_bdaddr)
- hci_dev_set_flag(hdev, HCI_UNCONFIGURED);
-
- /* For an unconfigured controller it is required to
- * read at least the version information provided by
- * the Read Local Version Information command.
- *
- * If the set_bdaddr driver callback is provided, then
- * also the original Bluetooth public device address
- * will be read using the Read BD Address command.
- */
- if (hci_dev_test_flag(hdev, HCI_UNCONFIGURED))
- ret = __hci_unconf_init(hdev);
- }
-
- if (hci_dev_test_flag(hdev, HCI_CONFIG)) {
- /* If public address change is configured, ensure that
- * the address gets programmed. If the driver does not
- * support changing the public address, fail the power
- * on procedure.
- */
- if (bacmp(&hdev->public_addr, BDADDR_ANY) &&
- hdev->set_bdaddr)
- ret = hdev->set_bdaddr(hdev, &hdev->public_addr);
- else
- ret = -EADDRNOTAVAIL;
- }
-
- if (!ret) {
- if (!hci_dev_test_flag(hdev, HCI_UNCONFIGURED) &&
- !hci_dev_test_flag(hdev, HCI_USER_CHANNEL)) {
- ret = __hci_init(hdev);
- if (!ret && hdev->post_init)
- ret = hdev->post_init(hdev);
- }
- }
-
- /* If the HCI Reset command is clearing all diagnostic settings,
- * then they need to be reprogrammed after the init procedure
- * completed.
- */
- if (test_bit(HCI_QUIRK_NON_PERSISTENT_DIAG, &hdev->quirks) &&
- !hci_dev_test_flag(hdev, HCI_USER_CHANNEL) &&
- hci_dev_test_flag(hdev, HCI_VENDOR_DIAG) && hdev->set_diag)
- ret = hdev->set_diag(hdev, true);
-
- msft_do_open(hdev);
- aosp_do_open(hdev);
-
- clear_bit(HCI_INIT, &hdev->flags);
-
- if (!ret) {
- hci_dev_hold(hdev);
- hci_dev_set_flag(hdev, HCI_RPA_EXPIRED);
- hci_adv_instances_set_rpa_expired(hdev, true);
- set_bit(HCI_UP, &hdev->flags);
- hci_sock_dev_event(hdev, HCI_DEV_UP);
- hci_leds_update_powered(hdev, true);
- if (!hci_dev_test_flag(hdev, HCI_SETUP) &&
- !hci_dev_test_flag(hdev, HCI_CONFIG) &&
- !hci_dev_test_flag(hdev, HCI_UNCONFIGURED) &&
- !hci_dev_test_flag(hdev, HCI_USER_CHANNEL) &&
- hci_dev_test_flag(hdev, HCI_MGMT) &&
- hdev->dev_type == HCI_PRIMARY) {
- ret = __hci_req_hci_power_on(hdev);
- mgmt_power_on(hdev, ret);
- }
- } else {
- /* Init failed, cleanup */
- flush_work(&hdev->tx_work);
-
- /* Since hci_rx_work() is possible to awake new cmd_work
- * it should be flushed first to avoid unexpected call of
- * hci_cmd_work()
- */
- flush_work(&hdev->rx_work);
- flush_work(&hdev->cmd_work);
-
- skb_queue_purge(&hdev->cmd_q);
- skb_queue_purge(&hdev->rx_q);
-
- if (hdev->flush)
- hdev->flush(hdev);
-
- if (hdev->sent_cmd) {
- kfree_skb(hdev->sent_cmd);
- hdev->sent_cmd = NULL;
- }
-
- clear_bit(HCI_RUNNING, &hdev->flags);
- hci_sock_dev_event(hdev, HCI_DEV_CLOSE);
-
- hdev->close(hdev);
- hdev->flags &= BIT(HCI_RAW);
- }
-
-done:
hci_req_sync_unlock(hdev);
return ret;
}
@@ -1583,155 +542,18 @@ int hci_dev_open(__u16 dev)
return err;
}
-/* This function requires the caller holds hdev->lock */
-static void hci_pend_le_actions_clear(struct hci_dev *hdev)
-{
- struct hci_conn_params *p;
-
- list_for_each_entry(p, &hdev->le_conn_params, list) {
- if (p->conn) {
- hci_conn_drop(p->conn);
- hci_conn_put(p->conn);
- p->conn = NULL;
- }
- list_del_init(&p->action);
- }
-
- BT_DBG("All LE pending actions cleared");
-}
-
int hci_dev_do_close(struct hci_dev *hdev)
{
- bool auto_off;
- int err = 0;
+ int err;
BT_DBG("%s %p", hdev->name, hdev);
- cancel_delayed_work(&hdev->power_off);
- cancel_delayed_work(&hdev->ncmd_timer);
-
- hci_request_cancel_all(hdev);
hci_req_sync_lock(hdev);
- if (!hci_dev_test_flag(hdev, HCI_UNREGISTER) &&
- !hci_dev_test_flag(hdev, HCI_USER_CHANNEL) &&
- test_bit(HCI_UP, &hdev->flags)) {
- /* Execute vendor specific shutdown routine */
- if (hdev->shutdown)
- err = hdev->shutdown(hdev);
- }
-
- if (!test_and_clear_bit(HCI_UP, &hdev->flags)) {
- cancel_delayed_work_sync(&hdev->cmd_timer);
- hci_req_sync_unlock(hdev);
- return err;
- }
-
- hci_leds_update_powered(hdev, false);
-
- /* Flush RX and TX works */
- flush_work(&hdev->tx_work);
- flush_work(&hdev->rx_work);
-
- if (hdev->discov_timeout > 0) {
- hdev->discov_timeout = 0;
- hci_dev_clear_flag(hdev, HCI_DISCOVERABLE);
- hci_dev_clear_flag(hdev, HCI_LIMITED_DISCOVERABLE);
- }
-
- if (hci_dev_test_and_clear_flag(hdev, HCI_SERVICE_CACHE))
- cancel_delayed_work(&hdev->service_cache);
-
- if (hci_dev_test_flag(hdev, HCI_MGMT)) {
- struct adv_info *adv_instance;
-
- cancel_delayed_work_sync(&hdev->rpa_expired);
-
- list_for_each_entry(adv_instance, &hdev->adv_instances, list)
- cancel_delayed_work_sync(&adv_instance->rpa_expired_cb);
- }
-
- /* Avoid potential lockdep warnings from the *_flush() calls by
- * ensuring the workqueue is empty up front.
- */
- drain_workqueue(hdev->workqueue);
-
- hci_dev_lock(hdev);
-
- hci_discovery_set_state(hdev, DISCOVERY_STOPPED);
-
- auto_off = hci_dev_test_and_clear_flag(hdev, HCI_AUTO_OFF);
-
- if (!auto_off && hdev->dev_type == HCI_PRIMARY &&
- !hci_dev_test_flag(hdev, HCI_USER_CHANNEL) &&
- hci_dev_test_flag(hdev, HCI_MGMT))
- __mgmt_power_off(hdev);
-
- hci_inquiry_cache_flush(hdev);
- hci_pend_le_actions_clear(hdev);
- hci_conn_hash_flush(hdev);
- hci_dev_unlock(hdev);
-
- smp_unregister(hdev);
-
- hci_sock_dev_event(hdev, HCI_DEV_DOWN);
-
- aosp_do_close(hdev);
- msft_do_close(hdev);
-
- if (hdev->flush)
- hdev->flush(hdev);
-
- /* Reset device */
- skb_queue_purge(&hdev->cmd_q);
- atomic_set(&hdev->cmd_cnt, 1);
- if (test_bit(HCI_QUIRK_RESET_ON_CLOSE, &hdev->quirks) &&
- !auto_off && !hci_dev_test_flag(hdev, HCI_UNCONFIGURED)) {
- set_bit(HCI_INIT, &hdev->flags);
- __hci_req_sync(hdev, hci_reset_req, 0, HCI_CMD_TIMEOUT, NULL);
- clear_bit(HCI_INIT, &hdev->flags);
- }
-
- /* flush cmd work */
- flush_work(&hdev->cmd_work);
-
- /* Drop queues */
- skb_queue_purge(&hdev->rx_q);
- skb_queue_purge(&hdev->cmd_q);
- skb_queue_purge(&hdev->raw_q);
-
- /* Drop last sent command */
- if (hdev->sent_cmd) {
- cancel_delayed_work_sync(&hdev->cmd_timer);
- kfree_skb(hdev->sent_cmd);
- hdev->sent_cmd = NULL;
- }
-
- clear_bit(HCI_RUNNING, &hdev->flags);
- hci_sock_dev_event(hdev, HCI_DEV_CLOSE);
-
- if (test_and_clear_bit(SUSPEND_POWERING_DOWN, hdev->suspend_tasks))
- wake_up(&hdev->suspend_wait_q);
-
- /* After this point our queues are empty
- * and no tasks are scheduled. */
- hdev->close(hdev);
-
- /* Clear flags */
- hdev->flags &= BIT(HCI_RAW);
- hci_dev_clear_volatile_flags(hdev);
-
- /* Controller radio is available but is currently powered down */
- hdev->amp_status = AMP_STATUS_POWERED_DOWN;
-
- memset(hdev->eir, 0, sizeof(hdev->eir));
- memset(hdev->dev_class, 0, sizeof(hdev->dev_class));
- bacpy(&hdev->random_addr, BDADDR_ANY);
- hci_codec_list_clear(&hdev->local_codecs);
+ err = hci_dev_close_sync(hdev);
hci_req_sync_unlock(hdev);
- hci_dev_put(hdev);
return err;
}
@@ -1787,7 +609,7 @@ static int hci_dev_do_reset(struct hci_dev *hdev)
atomic_set(&hdev->cmd_cnt, 1);
hdev->acl_cnt = 0; hdev->sco_cnt = 0; hdev->le_cnt = 0;
- ret = __hci_req_sync(hdev, hci_reset_req, 0, HCI_INIT_TIMEOUT, NULL);
+ ret = hci_reset_sync(hdev);
hci_req_sync_unlock(hdev);
return ret;
@@ -1850,7 +672,7 @@ int hci_dev_reset_stat(__u16 dev)
return ret;
}
-static void hci_update_scan_state(struct hci_dev *hdev, u8 scan)
+static void hci_update_passive_scan_state(struct hci_dev *hdev, u8 scan)
{
bool conn_changed, discov_changed;
@@ -1951,7 +773,7 @@ int hci_dev_cmd(unsigned int cmd, void __user *arg)
* get correctly modified as this was a non-mgmt change.
*/
if (!err)
- hci_update_scan_state(hdev, dr.dev_opt);
+ hci_update_passive_scan_state(hdev, dr.dev_opt);
break;
case HCISETLINKPOL:
@@ -2133,9 +955,7 @@ static void hci_power_on(struct work_struct *work)
hci_dev_test_flag(hdev, HCI_MGMT) &&
hci_dev_test_and_clear_flag(hdev, HCI_AUTO_OFF)) {
cancel_delayed_work(&hdev->power_off);
- hci_req_sync_lock(hdev);
- err = __hci_req_hci_power_on(hdev);
- hci_req_sync_unlock(hdev);
+ err = hci_powered_update_sync(hdev);
mgmt_power_on(hdev, err);
return;
}
@@ -3096,7 +1916,7 @@ bool hci_add_adv_monitor(struct hci_dev *hdev, struct adv_monitor *monitor,
switch (hci_get_adv_monitor_offload_ext(hdev)) {
case HCI_ADV_MONITOR_EXT_NONE:
- hci_update_background_scan(hdev);
+ hci_update_passive_scan(hdev);
bt_dev_dbg(hdev, "%s add monitor status %d", hdev->name, *err);
/* Message was not forwarded to controller - not an error */
return false;
@@ -3160,7 +1980,7 @@ bool hci_remove_single_adv_monitor(struct hci_dev *hdev, u16 handle, int *err)
pending = hci_remove_adv_monitor(hdev, monitor, handle, err);
if (!*err && !pending)
- hci_update_background_scan(hdev);
+ hci_update_passive_scan(hdev);
bt_dev_dbg(hdev, "%s remove monitor handle %d, status %d, %spending",
hdev->name, handle, *err, pending ? "" : "not ");
@@ -3192,7 +2012,7 @@ bool hci_remove_all_adv_monitor(struct hci_dev *hdev, int *err)
}
if (update)
- hci_update_background_scan(hdev);
+ hci_update_passive_scan(hdev);
bt_dev_dbg(hdev, "%s remove all monitors status %d, %spending",
hdev->name, *err, pending ? "" : "not ");
@@ -3333,7 +2153,7 @@ int hci_bdaddr_list_add_with_flags(struct list_head *list, bdaddr_t *bdaddr,
bacpy(&entry->bdaddr, bdaddr);
entry->bdaddr_type = type;
- entry->current_flags = flags;
+ bitmap_from_u64(entry->flags, flags);
list_add(&entry->list, list);
@@ -3486,7 +2306,7 @@ void hci_conn_params_del(struct hci_dev *hdev, bdaddr_t *addr, u8 addr_type)
hci_conn_params_free(params);
- hci_update_background_scan(hdev);
+ hci_update_passive_scan(hdev);
BT_DBG("addr %pMR (type %u)", addr, addr_type);
}
@@ -3554,61 +2374,6 @@ void hci_copy_identity_address(struct hci_dev *hdev, bdaddr_t *bdaddr,
}
}
-static void hci_suspend_clear_tasks(struct hci_dev *hdev)
-{
- int i;
-
- for (i = 0; i < __SUSPEND_NUM_TASKS; i++)
- clear_bit(i, hdev->suspend_tasks);
-
- wake_up(&hdev->suspend_wait_q);
-}
-
-static int hci_suspend_wait_event(struct hci_dev *hdev)
-{
-#define WAKE_COND \
- (find_first_bit(hdev->suspend_tasks, __SUSPEND_NUM_TASKS) == \
- __SUSPEND_NUM_TASKS)
-
- int i;
- int ret = wait_event_timeout(hdev->suspend_wait_q,
- WAKE_COND, SUSPEND_NOTIFIER_TIMEOUT);
-
- if (ret == 0) {
- bt_dev_err(hdev, "Timed out waiting for suspend events");
- for (i = 0; i < __SUSPEND_NUM_TASKS; ++i) {
- if (test_bit(i, hdev->suspend_tasks))
- bt_dev_err(hdev, "Suspend timeout bit: %d", i);
- clear_bit(i, hdev->suspend_tasks);
- }
-
- ret = -ETIMEDOUT;
- } else {
- ret = 0;
- }
-
- return ret;
-}
-
-static void hci_prepare_suspend(struct work_struct *work)
-{
- struct hci_dev *hdev =
- container_of(work, struct hci_dev, suspend_prepare);
-
- hci_dev_lock(hdev);
- hci_req_prepare_suspend(hdev, hdev->suspend_state_next);
- hci_dev_unlock(hdev);
-}
-
-static int hci_change_suspend_state(struct hci_dev *hdev,
- enum suspended_state next)
-{
- hdev->suspend_state_next = next;
- set_bit(SUSPEND_PREPARE_NOTIFIER, hdev->suspend_tasks);
- queue_work(hdev->req_workqueue, &hdev->suspend_prepare);
- return hci_suspend_wait_event(hdev);
-}
-
static void hci_clear_wake_reason(struct hci_dev *hdev)
{
hci_dev_lock(hdev);
@@ -3745,7 +2510,8 @@ struct hci_dev *hci_alloc_dev_priv(int sizeof_priv)
INIT_WORK(&hdev->tx_work, hci_tx_work);
INIT_WORK(&hdev->power_on, hci_power_on);
INIT_WORK(&hdev->error_reset, hci_error_reset);
- INIT_WORK(&hdev->suspend_prepare, hci_prepare_suspend);
+
+ hci_cmd_sync_init(hdev);
INIT_DELAYED_WORK(&hdev->power_off, hci_power_off);
@@ -3754,7 +2520,6 @@ struct hci_dev *hci_alloc_dev_priv(int sizeof_priv)
skb_queue_head_init(&hdev->raw_q);
init_waitqueue_head(&hdev->req_wait_q);
- init_waitqueue_head(&hdev->suspend_wait_q);
INIT_DELAYED_WORK(&hdev->cmd_timer, hci_cmd_timeout);
INIT_DELAYED_WORK(&hdev->ncmd_timer, hci_ncmd_timeout);
@@ -3864,6 +2629,12 @@ int hci_register_dev(struct hci_dev *hdev)
if (test_bit(HCI_QUIRK_RAW_DEVICE, &hdev->quirks))
hci_dev_set_flag(hdev, HCI_UNCONFIGURED);
+ /* Mark Remote Wakeup connection flag as supported if driver has wakeup
+ * callback.
+ */
+ if (hdev->wakeup)
+ set_bit(HCI_CONN_FLAG_REMOTE_WAKEUP, hdev->conn_flags);
+
hci_sock_dev_event(hdev, HCI_DEV_REG);
hci_dev_hold(hdev);
@@ -3882,6 +2653,7 @@ int hci_register_dev(struct hci_dev *hdev)
return id;
err_wqueue:
+ debugfs_remove_recursive(hdev->debugfs);
destroy_workqueue(hdev->workqueue);
destroy_workqueue(hdev->req_workqueue);
err:
@@ -3904,11 +2676,10 @@ void hci_unregister_dev(struct hci_dev *hdev)
cancel_work_sync(&hdev->power_on);
- if (!test_bit(HCI_QUIRK_NO_SUSPEND_NOTIFIER, &hdev->quirks)) {
- hci_suspend_clear_tasks(hdev);
+ hci_cmd_sync_clear(hdev);
+
+ if (!test_bit(HCI_QUIRK_NO_SUSPEND_NOTIFIER, &hdev->quirks))
unregister_pm_notifier(&hdev->suspend_notifier);
- cancel_work_sync(&hdev->suspend_prepare);
- }
msft_unregister(hdev);
@@ -3975,7 +2746,6 @@ EXPORT_SYMBOL(hci_release_dev);
int hci_suspend_dev(struct hci_dev *hdev)
{
int ret;
- u8 state = BT_RUNNING;
bt_dev_dbg(hdev, "");
@@ -3984,40 +2754,17 @@ int hci_suspend_dev(struct hci_dev *hdev)
hci_dev_test_flag(hdev, HCI_UNREGISTER))
return 0;
- /* If powering down, wait for completion. */
- if (mgmt_powering_down(hdev)) {
- set_bit(SUSPEND_POWERING_DOWN, hdev->suspend_tasks);
- ret = hci_suspend_wait_event(hdev);
- if (ret)
- goto done;
- }
+ /* If powering down don't attempt to suspend */
+ if (mgmt_powering_down(hdev))
+ return 0;
- /* Suspend consists of two actions:
- * - First, disconnect everything and make the controller not
- * connectable (disabling scanning)
- * - Second, program event filter/accept list and enable scan
- */
- ret = hci_change_suspend_state(hdev, BT_SUSPEND_DISCONNECT);
- if (ret)
- goto clear;
+ hci_req_sync_lock(hdev);
+ ret = hci_suspend_sync(hdev);
+ hci_req_sync_unlock(hdev);
- state = BT_SUSPEND_DISCONNECT;
-
- /* Only configure accept list if device may wakeup. */
- if (hdev->wakeup && hdev->wakeup(hdev)) {
- ret = hci_change_suspend_state(hdev, BT_SUSPEND_CONFIGURE_WAKE);
- if (!ret)
- state = BT_SUSPEND_CONFIGURE_WAKE;
- }
-
-clear:
hci_clear_wake_reason(hdev);
- mgmt_suspending(hdev, state);
+ mgmt_suspending(hdev, hdev->suspend_state);
-done:
- /* We always allow suspend even if suspend preparation failed and
- * attempt to recover in resume.
- */
hci_sock_dev_event(hdev, HCI_DEV_SUSPEND);
return ret;
}
@@ -4039,10 +2786,12 @@ int hci_resume_dev(struct hci_dev *hdev)
if (mgmt_powering_down(hdev))
return 0;
- ret = hci_change_suspend_state(hdev, BT_RUNNING);
+ hci_req_sync_lock(hdev);
+ ret = hci_resume_sync(hdev);
+ hci_req_sync_unlock(hdev);
mgmt_resuming(hdev, hdev->wake_reason, &hdev->wake_addr,
- hdev->wake_addr_type);
+ hdev->wake_addr_type);
hci_sock_dev_event(hdev, HCI_DEV_RESUME);
return ret;
@@ -4163,7 +2912,7 @@ int hci_unregister_cb(struct hci_cb *cb)
}
EXPORT_SYMBOL(hci_unregister_cb);
-static void hci_send_frame(struct hci_dev *hdev, struct sk_buff *skb)
+static int hci_send_frame(struct hci_dev *hdev, struct sk_buff *skb)
{
int err;
@@ -4186,14 +2935,17 @@ static void hci_send_frame(struct hci_dev *hdev, struct sk_buff *skb)
if (!test_bit(HCI_RUNNING, &hdev->flags)) {
kfree_skb(skb);
- return;
+ return -EINVAL;
}
err = hdev->send(hdev, skb);
if (err < 0) {
bt_dev_err(hdev, "sending frame failed (%d)", err);
kfree_skb(skb);
+ return err;
}
+
+ return 0;
}
/* Send HCI command */
@@ -4270,25 +3022,6 @@ void *hci_sent_cmd_data(struct hci_dev *hdev, __u16 opcode)
return hdev->sent_cmd->data + HCI_COMMAND_HDR_SIZE;
}
-/* Send HCI command and wait for command complete event */
-struct sk_buff *hci_cmd_sync(struct hci_dev *hdev, u16 opcode, u32 plen,
- const void *param, u32 timeout)
-{
- struct sk_buff *skb;
-
- if (!test_bit(HCI_UP, &hdev->flags))
- return ERR_PTR(-ENETDOWN);
-
- bt_dev_dbg(hdev, "opcode 0x%4.4x plen %d", opcode, plen);
-
- hci_req_sync_lock(hdev);
- skb = __hci_cmd_sync(hdev, opcode, plen, param, timeout);
- hci_req_sync_unlock(hdev);
-
- return skb;
-}
-EXPORT_SYMBOL(hci_cmd_sync);
-
/* Send ACL data */
static void hci_add_acl_hdr(struct sk_buff *skb, __u16 handle, __u16 flags)
{
@@ -5119,10 +3852,15 @@ static void hci_cmd_work(struct work_struct *work)
hdev->sent_cmd = skb_clone(skb, GFP_KERNEL);
if (hdev->sent_cmd) {
+ int res;
if (hci_req_status_pend(hdev))
hci_dev_set_flag(hdev, HCI_CMD_PENDING);
atomic_dec(&hdev->cmd_cnt);
- hci_send_frame(hdev, skb);
+
+ res = hci_send_frame(hdev, skb);
+ if (res < 0)
+ __hci_cmd_sync_cancel(hdev, -res);
+
if (test_bit(HCI_RESET, &hdev->flags))
cancel_delayed_work(&hdev->cmd_timer);
else
diff --git a/net/bluetooth/hci_event.c b/net/bluetooth/hci_event.c
index 7d0db1c..fc30f4c 100644
--- a/net/bluetooth/hci_event.c
+++ b/net/bluetooth/hci_event.c
@@ -45,12 +45,48 @@
/* Handle HCI Event packets */
-static void hci_cc_inquiry_cancel(struct hci_dev *hdev, struct sk_buff *skb,
- u8 *new_status)
+static void *hci_ev_skb_pull(struct hci_dev *hdev, struct sk_buff *skb,
+ u8 ev, size_t len)
{
- __u8 status = *((__u8 *) skb->data);
+ void *data;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ data = skb_pull_data(skb, len);
+ if (!data)
+ bt_dev_err(hdev, "Malformed Event: 0x%2.2x", ev);
+
+ return data;
+}
+
+static void *hci_cc_skb_pull(struct hci_dev *hdev, struct sk_buff *skb,
+ u16 op, size_t len)
+{
+ void *data;
+
+ data = skb_pull_data(skb, len);
+ if (!data)
+ bt_dev_err(hdev, "Malformed Command Complete: 0x%4.4x", op);
+
+ return data;
+}
+
+static void *hci_le_ev_skb_pull(struct hci_dev *hdev, struct sk_buff *skb,
+ u8 ev, size_t len)
+{
+ void *data;
+
+ data = skb_pull_data(skb, len);
+ if (!data)
+ bt_dev_err(hdev, "Malformed LE Event: 0x%2.2x", ev);
+
+ return data;
+}
+
+static u8 hci_cc_inquiry_cancel(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
+{
+ struct hci_ev_status *rp = data;
+
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
/* It is possible that we receive Inquiry Complete event right
* before we receive Inquiry Cancel Command Complete event, in
@@ -59,15 +95,13 @@ static void hci_cc_inquiry_cancel(struct hci_dev *hdev, struct sk_buff *skb,
* we actually achieve what Inquiry Cancel wants to achieve,
* which is to end the last Inquiry session.
*/
- if (status == 0x0c && !test_bit(HCI_INQUIRY, &hdev->flags)) {
+ if (rp->status == 0x0c && !test_bit(HCI_INQUIRY, &hdev->flags)) {
bt_dev_warn(hdev, "Ignoring error of Inquiry Cancel command");
- status = 0x00;
+ rp->status = 0x00;
}
- *new_status = status;
-
- if (status)
- return;
+ if (rp->status)
+ return rp->status;
clear_bit(HCI_INQUIRY, &hdev->flags);
smp_mb__after_atomic(); /* wake_up_bit advises about this barrier */
@@ -83,49 +117,62 @@ static void hci_cc_inquiry_cancel(struct hci_dev *hdev, struct sk_buff *skb,
hci_dev_unlock(hdev);
hci_conn_check_pending(hdev);
+
+ return rp->status;
}
-static void hci_cc_periodic_inq(struct hci_dev *hdev, struct sk_buff *skb)
+static u8 hci_cc_periodic_inq(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- __u8 status = *((__u8 *) skb->data);
+ struct hci_ev_status *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
- if (status)
- return;
+ if (rp->status)
+ return rp->status;
hci_dev_set_flag(hdev, HCI_PERIODIC_INQ);
+
+ return rp->status;
}
-static void hci_cc_exit_periodic_inq(struct hci_dev *hdev, struct sk_buff *skb)
+static u8 hci_cc_exit_periodic_inq(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- __u8 status = *((__u8 *) skb->data);
+ struct hci_ev_status *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
- if (status)
- return;
+ if (rp->status)
+ return rp->status;
hci_dev_clear_flag(hdev, HCI_PERIODIC_INQ);
hci_conn_check_pending(hdev);
+
+ return rp->status;
}
-static void hci_cc_remote_name_req_cancel(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_remote_name_req_cancel(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- BT_DBG("%s", hdev->name);
+ struct hci_ev_status *rp = data;
+
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
+
+ return rp->status;
}
-static void hci_cc_role_discovery(struct hci_dev *hdev, struct sk_buff *skb)
+static u8 hci_cc_role_discovery(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_role_discovery *rp = (void *) skb->data;
+ struct hci_rp_role_discovery *rp = data;
struct hci_conn *conn;
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
if (rp->status)
- return;
+ return rp->status;
hci_dev_lock(hdev);
@@ -134,17 +181,20 @@ static void hci_cc_role_discovery(struct hci_dev *hdev, struct sk_buff *skb)
conn->role = rp->role;
hci_dev_unlock(hdev);
+
+ return rp->status;
}
-static void hci_cc_read_link_policy(struct hci_dev *hdev, struct sk_buff *skb)
+static u8 hci_cc_read_link_policy(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_read_link_policy *rp = (void *) skb->data;
+ struct hci_rp_read_link_policy *rp = data;
struct hci_conn *conn;
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
if (rp->status)
- return;
+ return rp->status;
hci_dev_lock(hdev);
@@ -153,22 +203,25 @@ static void hci_cc_read_link_policy(struct hci_dev *hdev, struct sk_buff *skb)
conn->link_policy = __le16_to_cpu(rp->policy);
hci_dev_unlock(hdev);
+
+ return rp->status;
}
-static void hci_cc_write_link_policy(struct hci_dev *hdev, struct sk_buff *skb)
+static u8 hci_cc_write_link_policy(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_write_link_policy *rp = (void *) skb->data;
+ struct hci_rp_write_link_policy *rp = data;
struct hci_conn *conn;
void *sent;
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
if (rp->status)
- return;
+ return rp->status;
sent = hci_sent_cmd_data(hdev, HCI_OP_WRITE_LINK_POLICY);
if (!sent)
- return;
+ return rp->status;
hci_dev_lock(hdev);
@@ -177,49 +230,55 @@ static void hci_cc_write_link_policy(struct hci_dev *hdev, struct sk_buff *skb)
conn->link_policy = get_unaligned_le16(sent + 2);
hci_dev_unlock(hdev);
+
+ return rp->status;
}
-static void hci_cc_read_def_link_policy(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_read_def_link_policy(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_read_def_link_policy *rp = (void *) skb->data;
+ struct hci_rp_read_def_link_policy *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
if (rp->status)
- return;
+ return rp->status;
hdev->link_policy = __le16_to_cpu(rp->policy);
+
+ return rp->status;
}
-static void hci_cc_write_def_link_policy(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_write_def_link_policy(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- __u8 status = *((__u8 *) skb->data);
+ struct hci_ev_status *rp = data;
void *sent;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
- if (status)
- return;
+ if (rp->status)
+ return rp->status;
sent = hci_sent_cmd_data(hdev, HCI_OP_WRITE_DEF_LINK_POLICY);
if (!sent)
- return;
+ return rp->status;
hdev->link_policy = get_unaligned_le16(sent);
+
+ return rp->status;
}
-static void hci_cc_reset(struct hci_dev *hdev, struct sk_buff *skb)
+static u8 hci_cc_reset(struct hci_dev *hdev, void *data, struct sk_buff *skb)
{
- __u8 status = *((__u8 *) skb->data);
+ struct hci_ev_status *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
clear_bit(HCI_RESET, &hdev->flags);
- if (status)
- return;
+ if (rp->status)
+ return rp->status;
/* Reset all non-persistent flags */
hci_dev_clear_volatile_flags(hdev);
@@ -241,91 +300,104 @@ static void hci_cc_reset(struct hci_dev *hdev, struct sk_buff *skb)
hci_bdaddr_list_clear(&hdev->le_accept_list);
hci_bdaddr_list_clear(&hdev->le_resolv_list);
+
+ return rp->status;
}
-static void hci_cc_read_stored_link_key(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_read_stored_link_key(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_read_stored_link_key *rp = (void *)skb->data;
+ struct hci_rp_read_stored_link_key *rp = data;
struct hci_cp_read_stored_link_key *sent;
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
sent = hci_sent_cmd_data(hdev, HCI_OP_READ_STORED_LINK_KEY);
if (!sent)
- return;
+ return rp->status;
if (!rp->status && sent->read_all == 0x01) {
- hdev->stored_max_keys = rp->max_keys;
- hdev->stored_num_keys = rp->num_keys;
+ hdev->stored_max_keys = le16_to_cpu(rp->max_keys);
+ hdev->stored_num_keys = le16_to_cpu(rp->num_keys);
}
+
+ return rp->status;
}
-static void hci_cc_delete_stored_link_key(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_delete_stored_link_key(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_delete_stored_link_key *rp = (void *)skb->data;
+ struct hci_rp_delete_stored_link_key *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
if (rp->status)
- return;
+ return rp->status;
if (rp->num_keys <= hdev->stored_num_keys)
- hdev->stored_num_keys -= rp->num_keys;
+ hdev->stored_num_keys -= le16_to_cpu(rp->num_keys);
else
hdev->stored_num_keys = 0;
+
+ return rp->status;
}
-static void hci_cc_write_local_name(struct hci_dev *hdev, struct sk_buff *skb)
+static u8 hci_cc_write_local_name(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- __u8 status = *((__u8 *) skb->data);
+ struct hci_ev_status *rp = data;
void *sent;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
sent = hci_sent_cmd_data(hdev, HCI_OP_WRITE_LOCAL_NAME);
if (!sent)
- return;
+ return rp->status;
hci_dev_lock(hdev);
if (hci_dev_test_flag(hdev, HCI_MGMT))
- mgmt_set_local_name_complete(hdev, sent, status);
- else if (!status)
+ mgmt_set_local_name_complete(hdev, sent, rp->status);
+ else if (!rp->status)
memcpy(hdev->dev_name, sent, HCI_MAX_NAME_LENGTH);
hci_dev_unlock(hdev);
+
+ return rp->status;
}
-static void hci_cc_read_local_name(struct hci_dev *hdev, struct sk_buff *skb)
+static u8 hci_cc_read_local_name(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_read_local_name *rp = (void *) skb->data;
+ struct hci_rp_read_local_name *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
if (rp->status)
- return;
+ return rp->status;
if (hci_dev_test_flag(hdev, HCI_SETUP) ||
hci_dev_test_flag(hdev, HCI_CONFIG))
memcpy(hdev->dev_name, rp->name, HCI_MAX_NAME_LENGTH);
+
+ return rp->status;
}
-static void hci_cc_write_auth_enable(struct hci_dev *hdev, struct sk_buff *skb)
+static u8 hci_cc_write_auth_enable(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- __u8 status = *((__u8 *) skb->data);
+ struct hci_ev_status *rp = data;
void *sent;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
sent = hci_sent_cmd_data(hdev, HCI_OP_WRITE_AUTH_ENABLE);
if (!sent)
- return;
+ return rp->status;
hci_dev_lock(hdev);
- if (!status) {
+ if (!rp->status) {
__u8 param = *((__u8 *) sent);
if (param == AUTH_ENABLED)
@@ -335,25 +407,28 @@ static void hci_cc_write_auth_enable(struct hci_dev *hdev, struct sk_buff *skb)
}
if (hci_dev_test_flag(hdev, HCI_MGMT))
- mgmt_auth_enable_complete(hdev, status);
+ mgmt_auth_enable_complete(hdev, rp->status);
hci_dev_unlock(hdev);
+
+ return rp->status;
}
-static void hci_cc_write_encrypt_mode(struct hci_dev *hdev, struct sk_buff *skb)
+static u8 hci_cc_write_encrypt_mode(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- __u8 status = *((__u8 *) skb->data);
+ struct hci_ev_status *rp = data;
__u8 param;
void *sent;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
- if (status)
- return;
+ if (rp->status)
+ return rp->status;
sent = hci_sent_cmd_data(hdev, HCI_OP_WRITE_ENCRYPT_MODE);
if (!sent)
- return;
+ return rp->status;
param = *((__u8 *) sent);
@@ -361,25 +436,28 @@ static void hci_cc_write_encrypt_mode(struct hci_dev *hdev, struct sk_buff *skb)
set_bit(HCI_ENCRYPT, &hdev->flags);
else
clear_bit(HCI_ENCRYPT, &hdev->flags);
+
+ return rp->status;
}
-static void hci_cc_write_scan_enable(struct hci_dev *hdev, struct sk_buff *skb)
+static u8 hci_cc_write_scan_enable(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- __u8 status = *((__u8 *) skb->data);
+ struct hci_ev_status *rp = data;
__u8 param;
void *sent;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
sent = hci_sent_cmd_data(hdev, HCI_OP_WRITE_SCAN_ENABLE);
if (!sent)
- return;
+ return rp->status;
param = *((__u8 *) sent);
hci_dev_lock(hdev);
- if (status) {
+ if (rp->status) {
hdev->discov_timeout = 0;
goto done;
}
@@ -396,22 +474,25 @@ static void hci_cc_write_scan_enable(struct hci_dev *hdev, struct sk_buff *skb)
done:
hci_dev_unlock(hdev);
+
+ return rp->status;
}
-static void hci_cc_set_event_filter(struct hci_dev *hdev, struct sk_buff *skb)
+static u8 hci_cc_set_event_filter(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- __u8 status = *((__u8 *)skb->data);
+ struct hci_ev_status *rp = data;
struct hci_cp_set_event_filter *cp;
void *sent;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
- if (status)
- return;
+ if (rp->status)
+ return rp->status;
sent = hci_sent_cmd_data(hdev, HCI_OP_SET_EVENT_FLT);
if (!sent)
- return;
+ return rp->status;
cp = (struct hci_cp_set_event_filter *)sent;
@@ -419,135 +500,149 @@ static void hci_cc_set_event_filter(struct hci_dev *hdev, struct sk_buff *skb)
hci_dev_clear_flag(hdev, HCI_EVENT_FILTER_CONFIGURED);
else
hci_dev_set_flag(hdev, HCI_EVENT_FILTER_CONFIGURED);
+
+ return rp->status;
}
-static void hci_cc_read_class_of_dev(struct hci_dev *hdev, struct sk_buff *skb)
+static u8 hci_cc_read_class_of_dev(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_read_class_of_dev *rp = (void *) skb->data;
+ struct hci_rp_read_class_of_dev *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
if (rp->status)
- return;
+ return rp->status;
memcpy(hdev->dev_class, rp->dev_class, 3);
- BT_DBG("%s class 0x%.2x%.2x%.2x", hdev->name,
- hdev->dev_class[2], hdev->dev_class[1], hdev->dev_class[0]);
+ bt_dev_dbg(hdev, "class 0x%.2x%.2x%.2x", hdev->dev_class[2],
+ hdev->dev_class[1], hdev->dev_class[0]);
+
+ return rp->status;
}
-static void hci_cc_write_class_of_dev(struct hci_dev *hdev, struct sk_buff *skb)
+static u8 hci_cc_write_class_of_dev(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- __u8 status = *((__u8 *) skb->data);
+ struct hci_ev_status *rp = data;
void *sent;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
sent = hci_sent_cmd_data(hdev, HCI_OP_WRITE_CLASS_OF_DEV);
if (!sent)
- return;
+ return rp->status;
hci_dev_lock(hdev);
- if (status == 0)
+ if (!rp->status)
memcpy(hdev->dev_class, sent, 3);
if (hci_dev_test_flag(hdev, HCI_MGMT))
- mgmt_set_class_of_dev_complete(hdev, sent, status);
+ mgmt_set_class_of_dev_complete(hdev, sent, rp->status);
hci_dev_unlock(hdev);
+
+ return rp->status;
}
-static void hci_cc_read_voice_setting(struct hci_dev *hdev, struct sk_buff *skb)
+static u8 hci_cc_read_voice_setting(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_read_voice_setting *rp = (void *) skb->data;
+ struct hci_rp_read_voice_setting *rp = data;
__u16 setting;
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
if (rp->status)
- return;
+ return rp->status;
setting = __le16_to_cpu(rp->voice_setting);
if (hdev->voice_setting == setting)
- return;
+ return rp->status;
hdev->voice_setting = setting;
- BT_DBG("%s voice setting 0x%4.4x", hdev->name, setting);
+ bt_dev_dbg(hdev, "voice setting 0x%4.4x", setting);
if (hdev->notify)
hdev->notify(hdev, HCI_NOTIFY_VOICE_SETTING);
+
+ return rp->status;
}
-static void hci_cc_write_voice_setting(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_write_voice_setting(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- __u8 status = *((__u8 *) skb->data);
+ struct hci_ev_status *rp = data;
__u16 setting;
void *sent;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
- if (status)
- return;
+ if (rp->status)
+ return rp->status;
sent = hci_sent_cmd_data(hdev, HCI_OP_WRITE_VOICE_SETTING);
if (!sent)
- return;
+ return rp->status;
setting = get_unaligned_le16(sent);
if (hdev->voice_setting == setting)
- return;
+ return rp->status;
hdev->voice_setting = setting;
- BT_DBG("%s voice setting 0x%4.4x", hdev->name, setting);
+ bt_dev_dbg(hdev, "voice setting 0x%4.4x", setting);
if (hdev->notify)
hdev->notify(hdev, HCI_NOTIFY_VOICE_SETTING);
+
+ return rp->status;
}
-static void hci_cc_read_num_supported_iac(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_read_num_supported_iac(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_read_num_supported_iac *rp = (void *) skb->data;
+ struct hci_rp_read_num_supported_iac *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
if (rp->status)
- return;
+ return rp->status;
hdev->num_iac = rp->num_iac;
- BT_DBG("%s num iac %d", hdev->name, hdev->num_iac);
+ bt_dev_dbg(hdev, "num iac %d", hdev->num_iac);
+
+ return rp->status;
}
-static void hci_cc_write_ssp_mode(struct hci_dev *hdev, struct sk_buff *skb)
+static u8 hci_cc_write_ssp_mode(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- __u8 status = *((__u8 *) skb->data);
+ struct hci_ev_status *rp = data;
struct hci_cp_write_ssp_mode *sent;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
sent = hci_sent_cmd_data(hdev, HCI_OP_WRITE_SSP_MODE);
if (!sent)
- return;
+ return rp->status;
hci_dev_lock(hdev);
- if (!status) {
+ if (!rp->status) {
if (sent->mode)
hdev->features[1][0] |= LMP_HOST_SSP;
else
hdev->features[1][0] &= ~LMP_HOST_SSP;
}
- if (hci_dev_test_flag(hdev, HCI_MGMT))
- mgmt_ssp_enable_complete(hdev, sent->mode, status);
- else if (!status) {
+ if (!rp->status) {
if (sent->mode)
hci_dev_set_flag(hdev, HCI_SSP_ENABLED);
else
@@ -555,29 +650,32 @@ static void hci_cc_write_ssp_mode(struct hci_dev *hdev, struct sk_buff *skb)
}
hci_dev_unlock(hdev);
+
+ return rp->status;
}
-static void hci_cc_write_sc_support(struct hci_dev *hdev, struct sk_buff *skb)
+static u8 hci_cc_write_sc_support(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- u8 status = *((u8 *) skb->data);
+ struct hci_ev_status *rp = data;
struct hci_cp_write_sc_support *sent;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
sent = hci_sent_cmd_data(hdev, HCI_OP_WRITE_SC_SUPPORT);
if (!sent)
- return;
+ return rp->status;
hci_dev_lock(hdev);
- if (!status) {
+ if (!rp->status) {
if (sent->support)
hdev->features[1][0] |= LMP_HOST_SC;
else
hdev->features[1][0] &= ~LMP_HOST_SC;
}
- if (!hci_dev_test_flag(hdev, HCI_MGMT) && !status) {
+ if (!hci_dev_test_flag(hdev, HCI_MGMT) && !rp->status) {
if (sent->support)
hci_dev_set_flag(hdev, HCI_SC_ENABLED);
else
@@ -585,16 +683,19 @@ static void hci_cc_write_sc_support(struct hci_dev *hdev, struct sk_buff *skb)
}
hci_dev_unlock(hdev);
+
+ return rp->status;
}
-static void hci_cc_read_local_version(struct hci_dev *hdev, struct sk_buff *skb)
+static u8 hci_cc_read_local_version(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_read_local_version *rp = (void *) skb->data;
+ struct hci_rp_read_local_version *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
if (rp->status)
- return;
+ return rp->status;
if (hci_dev_test_flag(hdev, HCI_SETUP) ||
hci_dev_test_flag(hdev, HCI_CONFIG)) {
@@ -604,33 +705,37 @@ static void hci_cc_read_local_version(struct hci_dev *hdev, struct sk_buff *skb)
hdev->manufacturer = __le16_to_cpu(rp->manufacturer);
hdev->lmp_subver = __le16_to_cpu(rp->lmp_subver);
}
+
+ return rp->status;
}
-static void hci_cc_read_local_commands(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_read_local_commands(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_read_local_commands *rp = (void *) skb->data;
+ struct hci_rp_read_local_commands *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
if (rp->status)
- return;
+ return rp->status;
if (hci_dev_test_flag(hdev, HCI_SETUP) ||
hci_dev_test_flag(hdev, HCI_CONFIG))
memcpy(hdev->commands, rp->commands, sizeof(hdev->commands));
+
+ return rp->status;
}
-static void hci_cc_read_auth_payload_timeout(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_read_auth_payload_timeout(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_read_auth_payload_to *rp = (void *)skb->data;
+ struct hci_rp_read_auth_payload_to *rp = data;
struct hci_conn *conn;
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
if (rp->status)
- return;
+ return rp->status;
hci_dev_lock(hdev);
@@ -639,23 +744,25 @@ static void hci_cc_read_auth_payload_timeout(struct hci_dev *hdev,
conn->auth_payload_timeout = __le16_to_cpu(rp->timeout);
hci_dev_unlock(hdev);
+
+ return rp->status;
}
-static void hci_cc_write_auth_payload_timeout(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_write_auth_payload_timeout(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_write_auth_payload_to *rp = (void *)skb->data;
+ struct hci_rp_write_auth_payload_to *rp = data;
struct hci_conn *conn;
void *sent;
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
if (rp->status)
- return;
+ return rp->status;
sent = hci_sent_cmd_data(hdev, HCI_OP_WRITE_AUTH_PAYLOAD_TO);
if (!sent)
- return;
+ return rp->status;
hci_dev_lock(hdev);
@@ -664,17 +771,19 @@ static void hci_cc_write_auth_payload_timeout(struct hci_dev *hdev,
conn->auth_payload_timeout = get_unaligned_le16(sent + 2);
hci_dev_unlock(hdev);
+
+ return rp->status;
}
-static void hci_cc_read_local_features(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_read_local_features(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_read_local_features *rp = (void *) skb->data;
+ struct hci_rp_read_local_features *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
if (rp->status)
- return;
+ return rp->status;
memcpy(hdev->features, rp->features, 8);
@@ -714,46 +823,53 @@ static void hci_cc_read_local_features(struct hci_dev *hdev,
if (hdev->features[0][5] & LMP_EDR_3S_ESCO)
hdev->esco_type |= (ESCO_2EV5 | ESCO_3EV5);
+
+ return rp->status;
}
-static void hci_cc_read_local_ext_features(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_read_local_ext_features(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_read_local_ext_features *rp = (void *) skb->data;
+ struct hci_rp_read_local_ext_features *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
if (rp->status)
- return;
+ return rp->status;
if (hdev->max_page < rp->max_page)
hdev->max_page = rp->max_page;
if (rp->page < HCI_MAX_PAGES)
memcpy(hdev->features[rp->page], rp->features, 8);
+
+ return rp->status;
}
-static void hci_cc_read_flow_control_mode(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_read_flow_control_mode(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_read_flow_control_mode *rp = (void *) skb->data;
+ struct hci_rp_read_flow_control_mode *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
if (rp->status)
- return;
+ return rp->status;
hdev->flow_ctl_mode = rp->mode;
+
+ return rp->status;
}
-static void hci_cc_read_buffer_size(struct hci_dev *hdev, struct sk_buff *skb)
+static u8 hci_cc_read_buffer_size(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_read_buffer_size *rp = (void *) skb->data;
+ struct hci_rp_read_buffer_size *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
if (rp->status)
- return;
+ return rp->status;
hdev->acl_mtu = __le16_to_cpu(rp->acl_mtu);
hdev->sco_mtu = rp->sco_mtu;
@@ -770,115 +886,130 @@ static void hci_cc_read_buffer_size(struct hci_dev *hdev, struct sk_buff *skb)
BT_DBG("%s acl mtu %d:%d sco mtu %d:%d", hdev->name, hdev->acl_mtu,
hdev->acl_pkts, hdev->sco_mtu, hdev->sco_pkts);
+
+ return rp->status;
}
-static void hci_cc_read_bd_addr(struct hci_dev *hdev, struct sk_buff *skb)
+static u8 hci_cc_read_bd_addr(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_read_bd_addr *rp = (void *) skb->data;
+ struct hci_rp_read_bd_addr *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
if (rp->status)
- return;
+ return rp->status;
if (test_bit(HCI_INIT, &hdev->flags))
bacpy(&hdev->bdaddr, &rp->bdaddr);
if (hci_dev_test_flag(hdev, HCI_SETUP))
bacpy(&hdev->setup_addr, &rp->bdaddr);
+
+ return rp->status;
}
-static void hci_cc_read_local_pairing_opts(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_read_local_pairing_opts(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_read_local_pairing_opts *rp = (void *) skb->data;
+ struct hci_rp_read_local_pairing_opts *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
if (rp->status)
- return;
+ return rp->status;
if (hci_dev_test_flag(hdev, HCI_SETUP) ||
hci_dev_test_flag(hdev, HCI_CONFIG)) {
hdev->pairing_opts = rp->pairing_opts;
hdev->max_enc_key_size = rp->max_key_size;
}
+
+ return rp->status;
}
-static void hci_cc_read_page_scan_activity(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_read_page_scan_activity(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_read_page_scan_activity *rp = (void *) skb->data;
+ struct hci_rp_read_page_scan_activity *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
if (rp->status)
- return;
+ return rp->status;
if (test_bit(HCI_INIT, &hdev->flags)) {
hdev->page_scan_interval = __le16_to_cpu(rp->interval);
hdev->page_scan_window = __le16_to_cpu(rp->window);
}
+
+ return rp->status;
}
-static void hci_cc_write_page_scan_activity(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_write_page_scan_activity(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- u8 status = *((u8 *) skb->data);
+ struct hci_ev_status *rp = data;
struct hci_cp_write_page_scan_activity *sent;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
- if (status)
- return;
+ if (rp->status)
+ return rp->status;
sent = hci_sent_cmd_data(hdev, HCI_OP_WRITE_PAGE_SCAN_ACTIVITY);
if (!sent)
- return;
+ return rp->status;
hdev->page_scan_interval = __le16_to_cpu(sent->interval);
hdev->page_scan_window = __le16_to_cpu(sent->window);
+
+ return rp->status;
}
-static void hci_cc_read_page_scan_type(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_read_page_scan_type(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_read_page_scan_type *rp = (void *) skb->data;
+ struct hci_rp_read_page_scan_type *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
if (rp->status)
- return;
+ return rp->status;
if (test_bit(HCI_INIT, &hdev->flags))
hdev->page_scan_type = rp->type;
+
+ return rp->status;
}
-static void hci_cc_write_page_scan_type(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_write_page_scan_type(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- u8 status = *((u8 *) skb->data);
+ struct hci_ev_status *rp = data;
u8 *type;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
- if (status)
- return;
+ if (rp->status)
+ return rp->status;
type = hci_sent_cmd_data(hdev, HCI_OP_WRITE_PAGE_SCAN_TYPE);
if (type)
hdev->page_scan_type = *type;
+
+ return rp->status;
}
-static void hci_cc_read_data_block_size(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_read_data_block_size(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_read_data_block_size *rp = (void *) skb->data;
+ struct hci_rp_read_data_block_size *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
if (rp->status)
- return;
+ return rp->status;
hdev->block_mtu = __le16_to_cpu(rp->max_acl_len);
hdev->block_len = __le16_to_cpu(rp->block_len);
@@ -888,21 +1019,21 @@ static void hci_cc_read_data_block_size(struct hci_dev *hdev,
BT_DBG("%s blk mtu %d cnt %d len %d", hdev->name, hdev->block_mtu,
hdev->block_cnt, hdev->block_len);
+
+ return rp->status;
}
-static void hci_cc_read_clock(struct hci_dev *hdev, struct sk_buff *skb)
+static u8 hci_cc_read_clock(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_read_clock *rp = (void *) skb->data;
+ struct hci_rp_read_clock *rp = data;
struct hci_cp_read_clock *cp;
struct hci_conn *conn;
- BT_DBG("%s", hdev->name);
-
- if (skb->len < sizeof(*rp))
- return;
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
if (rp->status)
- return;
+ return rp->status;
hci_dev_lock(hdev);
@@ -923,17 +1054,18 @@ static void hci_cc_read_clock(struct hci_dev *hdev, struct sk_buff *skb)
unlock:
hci_dev_unlock(hdev);
+ return rp->status;
}
-static void hci_cc_read_local_amp_info(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_read_local_amp_info(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_read_local_amp_info *rp = (void *) skb->data;
+ struct hci_rp_read_local_amp_info *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
if (rp->status)
- return;
+ return rp->status;
hdev->amp_status = rp->amp_status;
hdev->amp_total_bw = __le32_to_cpu(rp->total_bw);
@@ -945,59 +1077,68 @@ static void hci_cc_read_local_amp_info(struct hci_dev *hdev,
hdev->amp_assoc_size = __le16_to_cpu(rp->max_assoc_size);
hdev->amp_be_flush_to = __le32_to_cpu(rp->be_flush_to);
hdev->amp_max_flush_to = __le32_to_cpu(rp->max_flush_to);
+
+ return rp->status;
}
-static void hci_cc_read_inq_rsp_tx_power(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_read_inq_rsp_tx_power(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_read_inq_rsp_tx_power *rp = (void *) skb->data;
+ struct hci_rp_read_inq_rsp_tx_power *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
if (rp->status)
- return;
+ return rp->status;
hdev->inq_tx_power = rp->tx_power;
+
+ return rp->status;
}
-static void hci_cc_read_def_err_data_reporting(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_read_def_err_data_reporting(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_read_def_err_data_reporting *rp = (void *)skb->data;
+ struct hci_rp_read_def_err_data_reporting *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
if (rp->status)
- return;
+ return rp->status;
hdev->err_data_reporting = rp->err_data_reporting;
+
+ return rp->status;
}
-static void hci_cc_write_def_err_data_reporting(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_write_def_err_data_reporting(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- __u8 status = *((__u8 *)skb->data);
+ struct hci_ev_status *rp = data;
struct hci_cp_write_def_err_data_reporting *cp;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
- if (status)
- return;
+ if (rp->status)
+ return rp->status;
cp = hci_sent_cmd_data(hdev, HCI_OP_WRITE_DEF_ERR_DATA_REPORTING);
if (!cp)
- return;
+ return rp->status;
hdev->err_data_reporting = cp->err_data_reporting;
+
+ return rp->status;
}
-static void hci_cc_pin_code_reply(struct hci_dev *hdev, struct sk_buff *skb)
+static u8 hci_cc_pin_code_reply(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_pin_code_reply *rp = (void *) skb->data;
+ struct hci_rp_pin_code_reply *rp = data;
struct hci_cp_pin_code_reply *cp;
struct hci_conn *conn;
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
hci_dev_lock(hdev);
@@ -1017,13 +1158,15 @@ static void hci_cc_pin_code_reply(struct hci_dev *hdev, struct sk_buff *skb)
unlock:
hci_dev_unlock(hdev);
+ return rp->status;
}
-static void hci_cc_pin_code_neg_reply(struct hci_dev *hdev, struct sk_buff *skb)
+static u8 hci_cc_pin_code_neg_reply(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_pin_code_neg_reply *rp = (void *) skb->data;
+ struct hci_rp_pin_code_neg_reply *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
hci_dev_lock(hdev);
@@ -1032,17 +1175,19 @@ static void hci_cc_pin_code_neg_reply(struct hci_dev *hdev, struct sk_buff *skb)
rp->status);
hci_dev_unlock(hdev);
+
+ return rp->status;
}
-static void hci_cc_le_read_buffer_size(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_le_read_buffer_size(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_le_read_buffer_size *rp = (void *) skb->data;
+ struct hci_rp_le_read_buffer_size *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
if (rp->status)
- return;
+ return rp->status;
hdev->le_mtu = __le16_to_cpu(rp->le_mtu);
hdev->le_pkts = rp->le_max_pkt;
@@ -1050,39 +1195,46 @@ static void hci_cc_le_read_buffer_size(struct hci_dev *hdev,
hdev->le_cnt = hdev->le_pkts;
BT_DBG("%s le mtu %d:%d", hdev->name, hdev->le_mtu, hdev->le_pkts);
+
+ return rp->status;
}
-static void hci_cc_le_read_local_features(struct hci_dev *hdev,
- struct sk_buff *skb)
-{
- struct hci_rp_le_read_local_features *rp = (void *) skb->data;
-
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
-
- if (rp->status)
- return;
-
- memcpy(hdev->le_features, rp->features, 8);
-}
-
-static void hci_cc_le_read_adv_tx_power(struct hci_dev *hdev,
+static u8 hci_cc_le_read_local_features(struct hci_dev *hdev, void *data,
struct sk_buff *skb)
{
- struct hci_rp_le_read_adv_tx_power *rp = (void *) skb->data;
+ struct hci_rp_le_read_local_features *rp = data;
BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
if (rp->status)
- return;
+ return rp->status;
- hdev->adv_tx_power = rp->tx_power;
+ memcpy(hdev->le_features, rp->features, 8);
+
+ return rp->status;
}
-static void hci_cc_user_confirm_reply(struct hci_dev *hdev, struct sk_buff *skb)
+static u8 hci_cc_le_read_adv_tx_power(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_user_confirm_reply *rp = (void *) skb->data;
+ struct hci_rp_le_read_adv_tx_power *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
+
+ if (rp->status)
+ return rp->status;
+
+ hdev->adv_tx_power = rp->tx_power;
+
+ return rp->status;
+}
+
+static u8 hci_cc_user_confirm_reply(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
+{
+ struct hci_rp_user_confirm_reply *rp = data;
+
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
hci_dev_lock(hdev);
@@ -1091,14 +1243,16 @@ static void hci_cc_user_confirm_reply(struct hci_dev *hdev, struct sk_buff *skb)
rp->status);
hci_dev_unlock(hdev);
+
+ return rp->status;
}
-static void hci_cc_user_confirm_neg_reply(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_user_confirm_neg_reply(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_user_confirm_reply *rp = (void *) skb->data;
+ struct hci_rp_user_confirm_reply *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
hci_dev_lock(hdev);
@@ -1107,13 +1261,16 @@ static void hci_cc_user_confirm_neg_reply(struct hci_dev *hdev,
ACL_LINK, 0, rp->status);
hci_dev_unlock(hdev);
+
+ return rp->status;
}
-static void hci_cc_user_passkey_reply(struct hci_dev *hdev, struct sk_buff *skb)
+static u8 hci_cc_user_passkey_reply(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_user_confirm_reply *rp = (void *) skb->data;
+ struct hci_rp_user_confirm_reply *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
hci_dev_lock(hdev);
@@ -1122,14 +1279,16 @@ static void hci_cc_user_passkey_reply(struct hci_dev *hdev, struct sk_buff *skb)
0, rp->status);
hci_dev_unlock(hdev);
+
+ return rp->status;
}
-static void hci_cc_user_passkey_neg_reply(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_user_passkey_neg_reply(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_user_confirm_reply *rp = (void *) skb->data;
+ struct hci_rp_user_confirm_reply *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
hci_dev_lock(hdev);
@@ -1138,37 +1297,44 @@ static void hci_cc_user_passkey_neg_reply(struct hci_dev *hdev,
ACL_LINK, 0, rp->status);
hci_dev_unlock(hdev);
+
+ return rp->status;
}
-static void hci_cc_read_local_oob_data(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_read_local_oob_data(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_read_local_oob_data *rp = (void *) skb->data;
+ struct hci_rp_read_local_oob_data *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
+
+ return rp->status;
}
-static void hci_cc_read_local_oob_ext_data(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_read_local_oob_ext_data(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_read_local_oob_ext_data *rp = (void *) skb->data;
+ struct hci_rp_read_local_oob_ext_data *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
+
+ return rp->status;
}
-static void hci_cc_le_set_random_addr(struct hci_dev *hdev, struct sk_buff *skb)
+static u8 hci_cc_le_set_random_addr(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- __u8 status = *((__u8 *) skb->data);
+ struct hci_ev_status *rp = data;
bdaddr_t *sent;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
- if (status)
- return;
+ if (rp->status)
+ return rp->status;
sent = hci_sent_cmd_data(hdev, HCI_OP_LE_SET_RANDOM_ADDR);
if (!sent)
- return;
+ return rp->status;
hci_dev_lock(hdev);
@@ -1181,21 +1347,24 @@ static void hci_cc_le_set_random_addr(struct hci_dev *hdev, struct sk_buff *skb)
}
hci_dev_unlock(hdev);
+
+ return rp->status;
}
-static void hci_cc_le_set_default_phy(struct hci_dev *hdev, struct sk_buff *skb)
+static u8 hci_cc_le_set_default_phy(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- __u8 status = *((__u8 *) skb->data);
+ struct hci_ev_status *rp = data;
struct hci_cp_le_set_default_phy *cp;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
- if (status)
- return;
+ if (rp->status)
+ return rp->status;
cp = hci_sent_cmd_data(hdev, HCI_OP_LE_SET_DEFAULT_PHY);
if (!cp)
- return;
+ return rp->status;
hci_dev_lock(hdev);
@@ -1203,17 +1372,21 @@ static void hci_cc_le_set_default_phy(struct hci_dev *hdev, struct sk_buff *skb)
hdev->le_rx_def_phys = cp->rx_phys;
hci_dev_unlock(hdev);
+
+ return rp->status;
}
-static void hci_cc_le_set_adv_set_random_addr(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_le_set_adv_set_random_addr(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- __u8 status = *((__u8 *) skb->data);
+ struct hci_ev_status *rp = data;
struct hci_cp_le_set_adv_set_rand_addr *cp;
struct adv_info *adv;
- if (status)
- return;
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
+
+ if (rp->status)
+ return rp->status;
cp = hci_sent_cmd_data(hdev, HCI_OP_LE_SET_ADV_SET_RAND_ADDR);
/* Update only in case the adv instance since handle 0x00 shall be using
@@ -1221,7 +1394,7 @@ static void hci_cc_le_set_adv_set_random_addr(struct hci_dev *hdev,
* non-extended adverting.
*/
if (!cp || !cp->handle)
- return;
+ return rp->status;
hci_dev_lock(hdev);
@@ -1237,34 +1410,126 @@ static void hci_cc_le_set_adv_set_random_addr(struct hci_dev *hdev,
}
hci_dev_unlock(hdev);
+
+ return rp->status;
}
-static void hci_cc_le_read_transmit_power(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_le_remove_adv_set(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_le_read_transmit_power *rp = (void *)skb->data;
+ struct hci_ev_status *rp = data;
+ u8 *instance;
+ int err;
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
if (rp->status)
- return;
+ return rp->status;
+
+ instance = hci_sent_cmd_data(hdev, HCI_OP_LE_REMOVE_ADV_SET);
+ if (!instance)
+ return rp->status;
+
+ hci_dev_lock(hdev);
+
+ err = hci_remove_adv_instance(hdev, *instance);
+ if (!err)
+ mgmt_advertising_removed(hci_skb_sk(hdev->sent_cmd), hdev,
+ *instance);
+
+ hci_dev_unlock(hdev);
+
+ return rp->status;
+}
+
+static u8 hci_cc_le_clear_adv_sets(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
+{
+ struct hci_ev_status *rp = data;
+ struct adv_info *adv, *n;
+ int err;
+
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
+
+ if (rp->status)
+ return rp->status;
+
+ if (!hci_sent_cmd_data(hdev, HCI_OP_LE_CLEAR_ADV_SETS))
+ return rp->status;
+
+ hci_dev_lock(hdev);
+
+ list_for_each_entry_safe(adv, n, &hdev->adv_instances, list) {
+ u8 instance = adv->instance;
+
+ err = hci_remove_adv_instance(hdev, instance);
+ if (!err)
+ mgmt_advertising_removed(hci_skb_sk(hdev->sent_cmd),
+ hdev, instance);
+ }
+
+ hci_dev_unlock(hdev);
+
+ return rp->status;
+}
+
+static u8 hci_cc_le_read_transmit_power(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
+{
+ struct hci_rp_le_read_transmit_power *rp = data;
+
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
+
+ if (rp->status)
+ return rp->status;
hdev->min_le_tx_power = rp->min_le_tx_power;
hdev->max_le_tx_power = rp->max_le_tx_power;
+
+ return rp->status;
}
-static void hci_cc_le_set_adv_enable(struct hci_dev *hdev, struct sk_buff *skb)
+static u8 hci_cc_le_set_privacy_mode(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- __u8 *sent, status = *((__u8 *) skb->data);
+ struct hci_ev_status *rp = data;
+ struct hci_cp_le_set_privacy_mode *cp;
+ struct hci_conn_params *params;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
- if (status)
- return;
+ if (rp->status)
+ return rp->status;
+
+ cp = hci_sent_cmd_data(hdev, HCI_OP_LE_SET_PRIVACY_MODE);
+ if (!cp)
+ return rp->status;
+
+ hci_dev_lock(hdev);
+
+ params = hci_conn_params_lookup(hdev, &cp->bdaddr, cp->bdaddr_type);
+ if (params)
+ params->privacy_mode = cp->mode;
+
+ hci_dev_unlock(hdev);
+
+ return rp->status;
+}
+
+static u8 hci_cc_le_set_adv_enable(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
+{
+ struct hci_ev_status *rp = data;
+ __u8 *sent;
+
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
+
+ if (rp->status)
+ return rp->status;
sent = hci_sent_cmd_data(hdev, HCI_OP_LE_SET_ADV_ENABLE);
if (!sent)
- return;
+ return rp->status;
hci_dev_lock(hdev);
@@ -1286,24 +1551,26 @@ static void hci_cc_le_set_adv_enable(struct hci_dev *hdev, struct sk_buff *skb)
}
hci_dev_unlock(hdev);
+
+ return rp->status;
}
-static void hci_cc_le_set_ext_adv_enable(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_le_set_ext_adv_enable(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
struct hci_cp_le_set_ext_adv_enable *cp;
struct hci_cp_ext_adv_set *set;
- __u8 status = *((__u8 *) skb->data);
struct adv_info *adv = NULL, *n;
+ struct hci_ev_status *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
- if (status)
- return;
+ if (rp->status)
+ return rp->status;
cp = hci_sent_cmd_data(hdev, HCI_OP_LE_SET_EXT_ADV_ENABLE);
if (!cp)
- return;
+ return rp->status;
set = (void *)cp->data;
@@ -1326,8 +1593,10 @@ static void hci_cc_le_set_ext_adv_enable(struct hci_dev *hdev,
&conn->le_conn_timeout,
conn->conn_timeout);
} else {
- if (adv) {
- adv->enabled = false;
+ if (cp->num_of_sets) {
+ if (adv)
+ adv->enabled = false;
+
/* If just one instance was disabled check if there are
* any other instance enabled before clearing HCI_LE_ADV
*/
@@ -1348,44 +1617,48 @@ static void hci_cc_le_set_ext_adv_enable(struct hci_dev *hdev,
unlock:
hci_dev_unlock(hdev);
+ return rp->status;
}
-static void hci_cc_le_set_scan_param(struct hci_dev *hdev, struct sk_buff *skb)
+static u8 hci_cc_le_set_scan_param(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
struct hci_cp_le_set_scan_param *cp;
- __u8 status = *((__u8 *) skb->data);
+ struct hci_ev_status *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
- if (status)
- return;
+ if (rp->status)
+ return rp->status;
cp = hci_sent_cmd_data(hdev, HCI_OP_LE_SET_SCAN_PARAM);
if (!cp)
- return;
+ return rp->status;
hci_dev_lock(hdev);
hdev->le_scan_type = cp->type;
hci_dev_unlock(hdev);
+
+ return rp->status;
}
-static void hci_cc_le_set_ext_scan_param(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_le_set_ext_scan_param(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
struct hci_cp_le_set_ext_scan_params *cp;
- __u8 status = *((__u8 *) skb->data);
+ struct hci_ev_status *rp = data;
struct hci_cp_le_scan_phy_params *phy_param;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
- if (status)
- return;
+ if (rp->status)
+ return rp->status;
cp = hci_sent_cmd_data(hdev, HCI_OP_LE_SET_EXT_SCAN_PARAMS);
if (!cp)
- return;
+ return rp->status;
phy_param = (void *)cp->data;
@@ -1394,6 +1667,8 @@ static void hci_cc_le_set_ext_scan_param(struct hci_dev *hdev,
hdev->le_scan_type = phy_param->type;
hci_dev_unlock(hdev);
+
+ return rp->status;
}
static bool has_pending_adv_report(struct hci_dev *hdev)
@@ -1463,16 +1738,10 @@ static void le_set_scan_enable_complete(struct hci_dev *hdev, u8 enable)
/* The HCI_LE_SCAN_INTERRUPTED flag indicates that we
* interrupted scanning due to a connect request. Mark
- * therefore discovery as stopped. If this was not
- * because of a connect request advertising might have
- * been disabled because of active scanning, so
- * re-enable it again if necessary.
+ * therefore discovery as stopped.
*/
if (hci_dev_test_and_clear_flag(hdev, HCI_LE_SCAN_INTERRUPTED))
hci_discovery_set_state(hdev, DISCOVERY_STOPPED);
- else if (!hci_dev_test_flag(hdev, HCI_LE_ADV) &&
- hdev->discovery.state == DISCOVERY_FINDING)
- hci_req_reenable_advertising(hdev);
break;
@@ -1485,244 +1754,273 @@ static void le_set_scan_enable_complete(struct hci_dev *hdev, u8 enable)
hci_dev_unlock(hdev);
}
-static void hci_cc_le_set_scan_enable(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_le_set_scan_enable(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
struct hci_cp_le_set_scan_enable *cp;
- __u8 status = *((__u8 *) skb->data);
+ struct hci_ev_status *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
- if (status)
- return;
+ if (rp->status)
+ return rp->status;
cp = hci_sent_cmd_data(hdev, HCI_OP_LE_SET_SCAN_ENABLE);
if (!cp)
- return;
+ return rp->status;
le_set_scan_enable_complete(hdev, cp->enable);
+
+ return rp->status;
}
-static void hci_cc_le_set_ext_scan_enable(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_le_set_ext_scan_enable(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
struct hci_cp_le_set_ext_scan_enable *cp;
- __u8 status = *((__u8 *) skb->data);
+ struct hci_ev_status *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
- if (status)
- return;
+ if (rp->status)
+ return rp->status;
cp = hci_sent_cmd_data(hdev, HCI_OP_LE_SET_EXT_SCAN_ENABLE);
if (!cp)
- return;
+ return rp->status;
le_set_scan_enable_complete(hdev, cp->enable);
+
+ return rp->status;
}
-static void hci_cc_le_read_num_adv_sets(struct hci_dev *hdev,
+static u8 hci_cc_le_read_num_adv_sets(struct hci_dev *hdev, void *data,
struct sk_buff *skb)
{
- struct hci_rp_le_read_num_supported_adv_sets *rp = (void *) skb->data;
+ struct hci_rp_le_read_num_supported_adv_sets *rp = data;
- BT_DBG("%s status 0x%2.2x No of Adv sets %u", hdev->name, rp->status,
- rp->num_of_sets);
+ bt_dev_dbg(hdev, "status 0x%2.2x No of Adv sets %u", rp->status,
+ rp->num_of_sets);
if (rp->status)
- return;
+ return rp->status;
hdev->le_num_of_adv_sets = rp->num_of_sets;
+
+ return rp->status;
}
-static void hci_cc_le_read_accept_list_size(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_le_read_accept_list_size(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_le_read_accept_list_size *rp = (void *)skb->data;
+ struct hci_rp_le_read_accept_list_size *rp = data;
- BT_DBG("%s status 0x%2.2x size %u", hdev->name, rp->status, rp->size);
+ bt_dev_dbg(hdev, "status 0x%2.2x size %u", rp->status, rp->size);
if (rp->status)
- return;
+ return rp->status;
hdev->le_accept_list_size = rp->size;
+
+ return rp->status;
}
-static void hci_cc_le_clear_accept_list(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_le_clear_accept_list(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- __u8 status = *((__u8 *) skb->data);
+ struct hci_ev_status *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
- if (status)
- return;
+ if (rp->status)
+ return rp->status;
hci_bdaddr_list_clear(&hdev->le_accept_list);
+
+ return rp->status;
}
-static void hci_cc_le_add_to_accept_list(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_le_add_to_accept_list(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
struct hci_cp_le_add_to_accept_list *sent;
- __u8 status = *((__u8 *) skb->data);
+ struct hci_ev_status *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
- if (status)
- return;
+ if (rp->status)
+ return rp->status;
sent = hci_sent_cmd_data(hdev, HCI_OP_LE_ADD_TO_ACCEPT_LIST);
if (!sent)
- return;
+ return rp->status;
hci_bdaddr_list_add(&hdev->le_accept_list, &sent->bdaddr,
sent->bdaddr_type);
+
+ return rp->status;
}
-static void hci_cc_le_del_from_accept_list(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_le_del_from_accept_list(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
struct hci_cp_le_del_from_accept_list *sent;
- __u8 status = *((__u8 *) skb->data);
+ struct hci_ev_status *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
- if (status)
- return;
+ if (rp->status)
+ return rp->status;
sent = hci_sent_cmd_data(hdev, HCI_OP_LE_DEL_FROM_ACCEPT_LIST);
if (!sent)
- return;
+ return rp->status;
hci_bdaddr_list_del(&hdev->le_accept_list, &sent->bdaddr,
sent->bdaddr_type);
+
+ return rp->status;
}
-static void hci_cc_le_read_supported_states(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_le_read_supported_states(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_le_read_supported_states *rp = (void *) skb->data;
+ struct hci_rp_le_read_supported_states *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
if (rp->status)
- return;
+ return rp->status;
memcpy(hdev->le_states, rp->le_states, 8);
+
+ return rp->status;
}
-static void hci_cc_le_read_def_data_len(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_le_read_def_data_len(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_le_read_def_data_len *rp = (void *) skb->data;
+ struct hci_rp_le_read_def_data_len *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
if (rp->status)
- return;
+ return rp->status;
hdev->le_def_tx_len = le16_to_cpu(rp->tx_len);
hdev->le_def_tx_time = le16_to_cpu(rp->tx_time);
+
+ return rp->status;
}
-static void hci_cc_le_write_def_data_len(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_le_write_def_data_len(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
struct hci_cp_le_write_def_data_len *sent;
- __u8 status = *((__u8 *) skb->data);
+ struct hci_ev_status *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
- if (status)
- return;
+ if (rp->status)
+ return rp->status;
sent = hci_sent_cmd_data(hdev, HCI_OP_LE_WRITE_DEF_DATA_LEN);
if (!sent)
- return;
+ return rp->status;
hdev->le_def_tx_len = le16_to_cpu(sent->tx_len);
hdev->le_def_tx_time = le16_to_cpu(sent->tx_time);
+
+ return rp->status;
}
-static void hci_cc_le_add_to_resolv_list(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_le_add_to_resolv_list(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
struct hci_cp_le_add_to_resolv_list *sent;
- __u8 status = *((__u8 *) skb->data);
+ struct hci_ev_status *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
- if (status)
- return;
+ if (rp->status)
+ return rp->status;
sent = hci_sent_cmd_data(hdev, HCI_OP_LE_ADD_TO_RESOLV_LIST);
if (!sent)
- return;
+ return rp->status;
hci_bdaddr_list_add_with_irk(&hdev->le_resolv_list, &sent->bdaddr,
sent->bdaddr_type, sent->peer_irk,
sent->local_irk);
+
+ return rp->status;
}
-static void hci_cc_le_del_from_resolv_list(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_le_del_from_resolv_list(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
struct hci_cp_le_del_from_resolv_list *sent;
- __u8 status = *((__u8 *) skb->data);
+ struct hci_ev_status *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
- if (status)
- return;
+ if (rp->status)
+ return rp->status;
sent = hci_sent_cmd_data(hdev, HCI_OP_LE_DEL_FROM_RESOLV_LIST);
if (!sent)
- return;
+ return rp->status;
hci_bdaddr_list_del_with_irk(&hdev->le_resolv_list, &sent->bdaddr,
sent->bdaddr_type);
+
+ return rp->status;
}
-static void hci_cc_le_clear_resolv_list(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_le_clear_resolv_list(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- __u8 status = *((__u8 *) skb->data);
+ struct hci_ev_status *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
-
- if (status)
- return;
-
- hci_bdaddr_list_clear(&hdev->le_resolv_list);
-}
-
-static void hci_cc_le_read_resolv_list_size(struct hci_dev *hdev,
- struct sk_buff *skb)
-{
- struct hci_rp_le_read_resolv_list_size *rp = (void *) skb->data;
-
- BT_DBG("%s status 0x%2.2x size %u", hdev->name, rp->status, rp->size);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
if (rp->status)
- return;
+ return rp->status;
- hdev->le_resolv_list_size = rp->size;
+ hci_bdaddr_list_clear(&hdev->le_resolv_list);
+
+ return rp->status;
}
-static void hci_cc_le_set_addr_resolution_enable(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_le_read_resolv_list_size(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- __u8 *sent, status = *((__u8 *) skb->data);
+ struct hci_rp_le_read_resolv_list_size *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x size %u", rp->status, rp->size);
- if (status)
- return;
+ if (rp->status)
+ return rp->status;
+
+ hdev->le_resolv_list_size = rp->size;
+
+ return rp->status;
+}
+
+static u8 hci_cc_le_set_addr_resolution_enable(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
+{
+ struct hci_ev_status *rp = data;
+ __u8 *sent;
+
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
+
+ if (rp->status)
+ return rp->status;
sent = hci_sent_cmd_data(hdev, HCI_OP_LE_SET_ADDR_RESOLV_ENABLE);
if (!sent)
- return;
+ return rp->status;
hci_dev_lock(hdev);
@@ -1732,38 +2030,42 @@ static void hci_cc_le_set_addr_resolution_enable(struct hci_dev *hdev,
hci_dev_clear_flag(hdev, HCI_LL_RPA_RESOLUTION);
hci_dev_unlock(hdev);
+
+ return rp->status;
}
-static void hci_cc_le_read_max_data_len(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_le_read_max_data_len(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_le_read_max_data_len *rp = (void *) skb->data;
+ struct hci_rp_le_read_max_data_len *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
if (rp->status)
- return;
+ return rp->status;
hdev->le_max_tx_len = le16_to_cpu(rp->tx_len);
hdev->le_max_tx_time = le16_to_cpu(rp->tx_time);
hdev->le_max_rx_len = le16_to_cpu(rp->rx_len);
hdev->le_max_rx_time = le16_to_cpu(rp->rx_time);
+
+ return rp->status;
}
-static void hci_cc_write_le_host_supported(struct hci_dev *hdev,
- struct sk_buff *skb)
+static u8 hci_cc_write_le_host_supported(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
struct hci_cp_write_le_host_supported *sent;
- __u8 status = *((__u8 *) skb->data);
+ struct hci_ev_status *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
- if (status)
- return;
+ if (rp->status)
+ return rp->status;
sent = hci_sent_cmd_data(hdev, HCI_OP_WRITE_LE_HOST_SUPPORTED);
if (!sent)
- return;
+ return rp->status;
hci_dev_lock(hdev);
@@ -1782,41 +2084,47 @@ static void hci_cc_write_le_host_supported(struct hci_dev *hdev,
hdev->features[1][0] &= ~LMP_HOST_LE_BREDR;
hci_dev_unlock(hdev);
+
+ return rp->status;
}
-static void hci_cc_set_adv_param(struct hci_dev *hdev, struct sk_buff *skb)
+static u8 hci_cc_set_adv_param(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
struct hci_cp_le_set_adv_param *cp;
- u8 status = *((u8 *) skb->data);
+ struct hci_ev_status *rp = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
- if (status)
- return;
+ if (rp->status)
+ return rp->status;
cp = hci_sent_cmd_data(hdev, HCI_OP_LE_SET_ADV_PARAM);
if (!cp)
- return;
+ return rp->status;
hci_dev_lock(hdev);
hdev->adv_addr_type = cp->own_address_type;
hci_dev_unlock(hdev);
+
+ return rp->status;
}
-static void hci_cc_set_ext_adv_param(struct hci_dev *hdev, struct sk_buff *skb)
+static u8 hci_cc_set_ext_adv_param(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_le_set_ext_adv_params *rp = (void *) skb->data;
+ struct hci_rp_le_set_ext_adv_params *rp = data;
struct hci_cp_le_set_ext_adv_params *cp;
struct adv_info *adv_instance;
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
if (rp->status)
- return;
+ return rp->status;
cp = hci_sent_cmd_data(hdev, HCI_OP_LE_SET_EXT_ADV_PARAMS);
if (!cp)
- return;
+ return rp->status;
hci_dev_lock(hdev);
hdev->adv_addr_type = cp->own_addr_type;
@@ -1832,17 +2140,20 @@ static void hci_cc_set_ext_adv_param(struct hci_dev *hdev, struct sk_buff *skb)
hci_req_update_adv_data(hdev, cp->handle);
hci_dev_unlock(hdev);
+
+ return rp->status;
}
-static void hci_cc_read_rssi(struct hci_dev *hdev, struct sk_buff *skb)
+static u8 hci_cc_read_rssi(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_rp_read_rssi *rp = (void *) skb->data;
+ struct hci_rp_read_rssi *rp = data;
struct hci_conn *conn;
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
if (rp->status)
- return;
+ return rp->status;
hci_dev_lock(hdev);
@@ -1851,22 +2162,25 @@ static void hci_cc_read_rssi(struct hci_dev *hdev, struct sk_buff *skb)
conn->rssi = rp->rssi;
hci_dev_unlock(hdev);
+
+ return rp->status;
}
-static void hci_cc_read_tx_power(struct hci_dev *hdev, struct sk_buff *skb)
+static u8 hci_cc_read_tx_power(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
struct hci_cp_read_tx_power *sent;
- struct hci_rp_read_tx_power *rp = (void *) skb->data;
+ struct hci_rp_read_tx_power *rp = data;
struct hci_conn *conn;
- BT_DBG("%s status 0x%2.2x", hdev->name, rp->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
if (rp->status)
- return;
+ return rp->status;
sent = hci_sent_cmd_data(hdev, HCI_OP_READ_TX_POWER);
if (!sent)
- return;
+ return rp->status;
hci_dev_lock(hdev);
@@ -1885,26 +2199,30 @@ static void hci_cc_read_tx_power(struct hci_dev *hdev, struct sk_buff *skb)
unlock:
hci_dev_unlock(hdev);
+ return rp->status;
}
-static void hci_cc_write_ssp_debug_mode(struct hci_dev *hdev, struct sk_buff *skb)
+static u8 hci_cc_write_ssp_debug_mode(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- u8 status = *((u8 *) skb->data);
+ struct hci_ev_status *rp = data;
u8 *mode;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", rp->status);
- if (status)
- return;
+ if (rp->status)
+ return rp->status;
mode = hci_sent_cmd_data(hdev, HCI_OP_WRITE_SSP_DEBUG_MODE);
if (mode)
hdev->ssp_debug_mode = *mode;
+
+ return rp->status;
}
static void hci_cs_inquiry(struct hci_dev *hdev, __u8 status)
{
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", status);
if (status) {
hci_conn_check_pending(hdev);
@@ -1919,7 +2237,7 @@ static void hci_cs_create_conn(struct hci_dev *hdev, __u8 status)
struct hci_cp_create_conn *cp;
struct hci_conn *conn;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", status);
cp = hci_sent_cmd_data(hdev, HCI_OP_CREATE_CONN);
if (!cp)
@@ -1929,7 +2247,7 @@ static void hci_cs_create_conn(struct hci_dev *hdev, __u8 status)
conn = hci_conn_hash_lookup_ba(hdev, ACL_LINK, &cp->bdaddr);
- BT_DBG("%s bdaddr %pMR hcon %p", hdev->name, &cp->bdaddr, conn);
+ bt_dev_dbg(hdev, "bdaddr %pMR hcon %p", &cp->bdaddr, conn);
if (status) {
if (conn && conn->state == BT_CONNECT) {
@@ -1958,7 +2276,7 @@ static void hci_cs_add_sco(struct hci_dev *hdev, __u8 status)
struct hci_conn *acl, *sco;
__u16 handle;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", status);
if (!status)
return;
@@ -1969,7 +2287,7 @@ static void hci_cs_add_sco(struct hci_dev *hdev, __u8 status)
handle = __le16_to_cpu(cp->handle);
- BT_DBG("%s handle 0x%4.4x", hdev->name, handle);
+ bt_dev_dbg(hdev, "handle 0x%4.4x", handle);
hci_dev_lock(hdev);
@@ -1992,7 +2310,7 @@ static void hci_cs_auth_requested(struct hci_dev *hdev, __u8 status)
struct hci_cp_auth_requested *cp;
struct hci_conn *conn;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", status);
if (!status)
return;
@@ -2019,7 +2337,7 @@ static void hci_cs_set_conn_encrypt(struct hci_dev *hdev, __u8 status)
struct hci_cp_set_conn_encrypt *cp;
struct hci_conn *conn;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", status);
if (!status)
return;
@@ -2086,6 +2404,12 @@ static bool hci_resolve_next_name(struct hci_dev *hdev)
if (list_empty(&discov->resolve))
return false;
+ /* We should stop if we already spent too much time resolving names. */
+ if (time_after(jiffies, discov->name_resolve_timeout)) {
+ bt_dev_warn_ratelimited(hdev, "Name resolve takes too long.");
+ return false;
+ }
+
e = hci_inquiry_cache_lookup_resolve(hdev, BDADDR_ANY, NAME_NEEDED);
if (!e)
return false;
@@ -2132,13 +2456,10 @@ static void hci_check_pending_name(struct hci_dev *hdev, struct hci_conn *conn,
return;
list_del(&e->list);
- if (name) {
- e->name_state = NAME_KNOWN;
- mgmt_remote_name(hdev, bdaddr, ACL_LINK, 0x00,
- e->data.rssi, name, name_len);
- } else {
- e->name_state = NAME_NOT_KNOWN;
- }
+
+ e->name_state = name ? NAME_KNOWN : NAME_NOT_KNOWN;
+ mgmt_remote_name(hdev, bdaddr, ACL_LINK, 0x00, e->data.rssi,
+ name, name_len);
if (hci_resolve_next_name(hdev))
return;
@@ -2152,7 +2473,7 @@ static void hci_cs_remote_name_req(struct hci_dev *hdev, __u8 status)
struct hci_cp_remote_name_req *cp;
struct hci_conn *conn;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", status);
/* If successful wait for the name req complete event before
* checking for the need to do authentication */
@@ -2195,7 +2516,7 @@ static void hci_cs_read_remote_features(struct hci_dev *hdev, __u8 status)
struct hci_cp_read_remote_features *cp;
struct hci_conn *conn;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", status);
if (!status)
return;
@@ -2222,7 +2543,7 @@ static void hci_cs_read_remote_ext_features(struct hci_dev *hdev, __u8 status)
struct hci_cp_read_remote_ext_features *cp;
struct hci_conn *conn;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", status);
if (!status)
return;
@@ -2250,7 +2571,7 @@ static void hci_cs_setup_sync_conn(struct hci_dev *hdev, __u8 status)
struct hci_conn *acl, *sco;
__u16 handle;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", status);
if (!status)
return;
@@ -2261,7 +2582,7 @@ static void hci_cs_setup_sync_conn(struct hci_dev *hdev, __u8 status)
handle = __le16_to_cpu(cp->handle);
- BT_DBG("%s handle 0x%4.4x", hdev->name, handle);
+ bt_dev_dbg(hdev, "handle 0x%4.4x", handle);
hci_dev_lock(hdev);
@@ -2319,7 +2640,7 @@ static void hci_cs_sniff_mode(struct hci_dev *hdev, __u8 status)
struct hci_cp_sniff_mode *cp;
struct hci_conn *conn;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", status);
if (!status)
return;
@@ -2346,7 +2667,7 @@ static void hci_cs_exit_sniff_mode(struct hci_dev *hdev, __u8 status)
struct hci_cp_exit_sniff_mode *cp;
struct hci_conn *conn;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", status);
if (!status)
return;
@@ -2371,9 +2692,16 @@ static void hci_cs_exit_sniff_mode(struct hci_dev *hdev, __u8 status)
static void hci_cs_disconnect(struct hci_dev *hdev, u8 status)
{
struct hci_cp_disconnect *cp;
+ struct hci_conn_params *params;
struct hci_conn *conn;
+ bool mgmt_conn;
- if (!status)
+ bt_dev_dbg(hdev, "status 0x%2.2x", status);
+
+ /* Wait for HCI_EV_DISCONN_COMPLETE if status 0x00 and not suspended
+ * otherwise cleanup the connection immediately.
+ */
+ if (!status && !hdev->suspended)
return;
cp = hci_sent_cmd_data(hdev, HCI_OP_DISCONNECT);
@@ -2383,23 +2711,60 @@ static void hci_cs_disconnect(struct hci_dev *hdev, u8 status)
hci_dev_lock(hdev);
conn = hci_conn_hash_lookup_handle(hdev, __le16_to_cpu(cp->handle));
- if (conn) {
+ if (!conn)
+ goto unlock;
+
+ if (status) {
mgmt_disconnect_failed(hdev, &conn->dst, conn->type,
conn->dst_type, status);
if (conn->type == LE_LINK && conn->role == HCI_ROLE_SLAVE) {
hdev->cur_adv_instance = conn->adv_instance;
- hci_req_reenable_advertising(hdev);
+ hci_enable_advertising(hdev);
}
- /* If the disconnection failed for any reason, the upper layer
- * does not retry to disconnect in current implementation.
- * Hence, we need to do some basic cleanup here and re-enable
- * advertising if necessary.
- */
- hci_conn_del(conn);
+ goto done;
}
+ mgmt_conn = test_and_clear_bit(HCI_CONN_MGMT_CONNECTED, &conn->flags);
+
+ if (conn->type == ACL_LINK) {
+ if (test_bit(HCI_CONN_FLUSH_KEY, &conn->flags))
+ hci_remove_link_key(hdev, &conn->dst);
+ }
+
+ params = hci_conn_params_lookup(hdev, &conn->dst, conn->dst_type);
+ if (params) {
+ switch (params->auto_connect) {
+ case HCI_AUTO_CONN_LINK_LOSS:
+ if (cp->reason != HCI_ERROR_CONNECTION_TIMEOUT)
+ break;
+ fallthrough;
+
+ case HCI_AUTO_CONN_DIRECT:
+ case HCI_AUTO_CONN_ALWAYS:
+ list_del_init(¶ms->action);
+ list_add(¶ms->action, &hdev->pend_le_conns);
+ break;
+
+ default:
+ break;
+ }
+ }
+
+ mgmt_device_disconnected(hdev, &conn->dst, conn->type, conn->dst_type,
+ cp->reason, mgmt_conn);
+
+ hci_disconn_cfm(conn, cp->reason);
+
+done:
+ /* If the disconnection failed for any reason, the upper layer
+ * does not retry to disconnect in current implementation.
+ * Hence, we need to do some basic cleanup here and re-enable
+ * advertising if necessary.
+ */
+ hci_conn_del(conn);
+unlock:
hci_dev_unlock(hdev);
}
@@ -2466,7 +2831,7 @@ static void hci_cs_le_create_conn(struct hci_dev *hdev, u8 status)
{
struct hci_cp_le_create_conn *cp;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", status);
/* All connection failure handling is taken care of by the
* hci_le_conn_failed function which is triggered by the HCI
@@ -2491,7 +2856,7 @@ static void hci_cs_le_ext_create_conn(struct hci_dev *hdev, u8 status)
{
struct hci_cp_le_ext_create_conn *cp;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", status);
/* All connection failure handling is taken care of by the
* hci_le_conn_failed function which is triggered by the HCI
@@ -2517,7 +2882,7 @@ static void hci_cs_le_read_remote_features(struct hci_dev *hdev, u8 status)
struct hci_cp_le_read_remote_features *cp;
struct hci_conn *conn;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", status);
if (!status)
return;
@@ -2544,7 +2909,7 @@ static void hci_cs_le_start_enc(struct hci_dev *hdev, u8 status)
struct hci_cp_le_start_enc *cp;
struct hci_conn *conn;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", status);
if (!status)
return;
@@ -2592,13 +2957,14 @@ static void hci_cs_switch_role(struct hci_dev *hdev, u8 status)
hci_dev_unlock(hdev);
}
-static void hci_inquiry_complete_evt(struct hci_dev *hdev, struct sk_buff *skb)
+static void hci_inquiry_complete_evt(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- __u8 status = *((__u8 *) skb->data);
+ struct hci_ev_status *ev = data;
struct discovery_state *discov = &hdev->discovery;
struct inquiry_entry *e;
- BT_DBG("%s status 0x%2.2x", hdev->name, status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", ev->status);
hci_conn_check_pending(hdev);
@@ -2634,6 +3000,7 @@ static void hci_inquiry_complete_evt(struct hci_dev *hdev, struct sk_buff *skb)
if (e && hci_resolve_name(hdev, e) == 0) {
e->name_state = NAME_PENDING;
hci_discovery_set_state(hdev, DISCOVERY_RESOLVING);
+ discov->name_resolve_timeout = jiffies + NAME_RESOLVE_DURATION;
} else {
/* When BR/EDR inquiry is active and no LE scanning is in
* progress, then change discovery state to indicate completion.
@@ -2651,15 +3018,20 @@ static void hci_inquiry_complete_evt(struct hci_dev *hdev, struct sk_buff *skb)
hci_dev_unlock(hdev);
}
-static void hci_inquiry_result_evt(struct hci_dev *hdev, struct sk_buff *skb)
+static void hci_inquiry_result_evt(struct hci_dev *hdev, void *edata,
+ struct sk_buff *skb)
{
+ struct hci_ev_inquiry_result *ev = edata;
struct inquiry_data data;
- struct inquiry_info *info = (void *) (skb->data + 1);
- int num_rsp = *((__u8 *) skb->data);
+ int i;
- BT_DBG("%s num_rsp %d", hdev->name, num_rsp);
+ if (!hci_ev_skb_pull(hdev, skb, HCI_EV_INQUIRY_RESULT,
+ flex_array_size(ev, info, ev->num)))
+ return;
- if (!num_rsp || skb->len < num_rsp * sizeof(*info) + 1)
+ bt_dev_dbg(hdev, "num %d", ev->num);
+
+ if (!ev->num)
return;
if (hci_dev_test_flag(hdev, HCI_PERIODIC_INQ))
@@ -2667,7 +3039,8 @@ static void hci_inquiry_result_evt(struct hci_dev *hdev, struct sk_buff *skb)
hci_dev_lock(hdev);
- for (; num_rsp; num_rsp--, info++) {
+ for (i = 0; i < ev->num; i++) {
+ struct inquiry_info *info = &ev->info[i];
u32 flags;
bacpy(&data.bdaddr, &info->bdaddr);
@@ -2689,12 +3062,13 @@ static void hci_inquiry_result_evt(struct hci_dev *hdev, struct sk_buff *skb)
hci_dev_unlock(hdev);
}
-static void hci_conn_complete_evt(struct hci_dev *hdev, struct sk_buff *skb)
+static void hci_conn_complete_evt(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_ev_conn_complete *ev = (void *) skb->data;
+ struct hci_ev_conn_complete *ev = data;
struct hci_conn *conn;
- BT_DBG("%s", hdev->name);
+ bt_dev_dbg(hdev, "status 0x%2.2x", ev->status);
hci_dev_lock(hdev);
@@ -2813,16 +3187,16 @@ static void hci_reject_conn(struct hci_dev *hdev, bdaddr_t *bdaddr)
hci_send_cmd(hdev, HCI_OP_REJECT_CONN_REQ, sizeof(cp), &cp);
}
-static void hci_conn_request_evt(struct hci_dev *hdev, struct sk_buff *skb)
+static void hci_conn_request_evt(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_ev_conn_request *ev = (void *) skb->data;
+ struct hci_ev_conn_request *ev = data;
int mask = hdev->link_mode;
struct inquiry_entry *ie;
struct hci_conn *conn;
__u8 flags = 0;
- BT_DBG("%s bdaddr %pMR type 0x%x", hdev->name, &ev->bdaddr,
- ev->link_type);
+ bt_dev_dbg(hdev, "bdaddr %pMR type 0x%x", &ev->bdaddr, ev->link_type);
mask |= hci_proto_connect_ind(hdev, &ev->bdaddr, ev->link_type,
&flags);
@@ -2924,15 +3298,16 @@ static u8 hci_to_mgmt_reason(u8 err)
}
}
-static void hci_disconn_complete_evt(struct hci_dev *hdev, struct sk_buff *skb)
+static void hci_disconn_complete_evt(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_ev_disconn_complete *ev = (void *) skb->data;
+ struct hci_ev_disconn_complete *ev = data;
u8 reason;
struct hci_conn_params *params;
struct hci_conn *conn;
bool mgmt_connected;
- BT_DBG("%s status 0x%2.2x", hdev->name, ev->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", ev->status);
hci_dev_lock(hdev);
@@ -2977,7 +3352,7 @@ static void hci_disconn_complete_evt(struct hci_dev *hdev, struct sk_buff *skb)
case HCI_AUTO_CONN_ALWAYS:
list_del_init(¶ms->action);
list_add(¶ms->action, &hdev->pend_le_conns);
- hci_update_background_scan(hdev);
+ hci_update_passive_scan(hdev);
break;
default:
@@ -2987,14 +3362,6 @@ static void hci_disconn_complete_evt(struct hci_dev *hdev, struct sk_buff *skb)
hci_disconn_cfm(conn, ev->reason);
- /* The suspend notifier is waiting for all devices to disconnect so
- * clear the bit from pending tasks and inform the wait queue.
- */
- if (list_empty(&hdev->conn_hash.list) &&
- test_and_clear_bit(SUSPEND_DISCONNECTING, hdev->suspend_tasks)) {
- wake_up(&hdev->suspend_wait_q);
- }
-
/* Re-enable advertising if necessary, since it might
* have been disabled by the connection. From the
* HCI_LE_Set_Advertise_Enable command description in
@@ -3007,7 +3374,7 @@ static void hci_disconn_complete_evt(struct hci_dev *hdev, struct sk_buff *skb)
*/
if (conn->type == LE_LINK && conn->role == HCI_ROLE_SLAVE) {
hdev->cur_adv_instance = conn->adv_instance;
- hci_req_reenable_advertising(hdev);
+ hci_enable_advertising(hdev);
}
hci_conn_del(conn);
@@ -3016,12 +3383,13 @@ static void hci_disconn_complete_evt(struct hci_dev *hdev, struct sk_buff *skb)
hci_dev_unlock(hdev);
}
-static void hci_auth_complete_evt(struct hci_dev *hdev, struct sk_buff *skb)
+static void hci_auth_complete_evt(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_ev_auth_complete *ev = (void *) skb->data;
+ struct hci_ev_auth_complete *ev = data;
struct hci_conn *conn;
- BT_DBG("%s status 0x%2.2x", hdev->name, ev->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", ev->status);
hci_dev_lock(hdev);
@@ -3086,12 +3454,13 @@ static void hci_auth_complete_evt(struct hci_dev *hdev, struct sk_buff *skb)
hci_dev_unlock(hdev);
}
-static void hci_remote_name_evt(struct hci_dev *hdev, struct sk_buff *skb)
+static void hci_remote_name_evt(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_ev_remote_name *ev = (void *) skb->data;
+ struct hci_ev_remote_name *ev = data;
struct hci_conn *conn;
- BT_DBG("%s", hdev->name);
+ bt_dev_dbg(hdev, "status 0x%2.2x", ev->status);
hci_conn_check_pending(hdev);
@@ -3169,12 +3538,13 @@ static void read_enc_key_size_complete(struct hci_dev *hdev, u8 status,
hci_dev_unlock(hdev);
}
-static void hci_encrypt_change_evt(struct hci_dev *hdev, struct sk_buff *skb)
+static void hci_encrypt_change_evt(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_ev_encrypt_change *ev = (void *) skb->data;
+ struct hci_ev_encrypt_change *ev = data;
struct hci_conn *conn;
- BT_DBG("%s status 0x%2.2x", hdev->name, ev->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", ev->status);
hci_dev_lock(hdev);
@@ -3283,13 +3653,13 @@ static void hci_encrypt_change_evt(struct hci_dev *hdev, struct sk_buff *skb)
hci_dev_unlock(hdev);
}
-static void hci_change_link_key_complete_evt(struct hci_dev *hdev,
+static void hci_change_link_key_complete_evt(struct hci_dev *hdev, void *data,
struct sk_buff *skb)
{
- struct hci_ev_change_link_key_complete *ev = (void *) skb->data;
+ struct hci_ev_change_link_key_complete *ev = data;
struct hci_conn *conn;
- BT_DBG("%s status 0x%2.2x", hdev->name, ev->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", ev->status);
hci_dev_lock(hdev);
@@ -3306,13 +3676,13 @@ static void hci_change_link_key_complete_evt(struct hci_dev *hdev,
hci_dev_unlock(hdev);
}
-static void hci_remote_features_evt(struct hci_dev *hdev,
+static void hci_remote_features_evt(struct hci_dev *hdev, void *data,
struct sk_buff *skb)
{
- struct hci_ev_remote_features *ev = (void *) skb->data;
+ struct hci_ev_remote_features *ev = data;
struct hci_conn *conn;
- BT_DBG("%s status 0x%2.2x", hdev->name, ev->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", ev->status);
hci_dev_lock(hdev);
@@ -3370,366 +3740,227 @@ static inline void handle_cmd_cnt_and_timer(struct hci_dev *hdev, u8 ncmd)
}
}
-static void hci_cmd_complete_evt(struct hci_dev *hdev, struct sk_buff *skb,
- u16 *opcode, u8 *status,
+#define HCI_CC_VL(_op, _func, _min, _max) \
+{ \
+ .op = _op, \
+ .func = _func, \
+ .min_len = _min, \
+ .max_len = _max, \
+}
+
+#define HCI_CC(_op, _func, _len) \
+ HCI_CC_VL(_op, _func, _len, _len)
+
+#define HCI_CC_STATUS(_op, _func) \
+ HCI_CC(_op, _func, sizeof(struct hci_ev_status))
+
+static const struct hci_cc {
+ u16 op;
+ u8 (*func)(struct hci_dev *hdev, void *data, struct sk_buff *skb);
+ u16 min_len;
+ u16 max_len;
+} hci_cc_table[] = {
+ HCI_CC_STATUS(HCI_OP_INQUIRY_CANCEL, hci_cc_inquiry_cancel),
+ HCI_CC_STATUS(HCI_OP_PERIODIC_INQ, hci_cc_periodic_inq),
+ HCI_CC_STATUS(HCI_OP_EXIT_PERIODIC_INQ, hci_cc_exit_periodic_inq),
+ HCI_CC_STATUS(HCI_OP_REMOTE_NAME_REQ_CANCEL,
+ hci_cc_remote_name_req_cancel),
+ HCI_CC(HCI_OP_ROLE_DISCOVERY, hci_cc_role_discovery,
+ sizeof(struct hci_rp_role_discovery)),
+ HCI_CC(HCI_OP_READ_LINK_POLICY, hci_cc_read_link_policy,
+ sizeof(struct hci_rp_read_link_policy)),
+ HCI_CC(HCI_OP_WRITE_LINK_POLICY, hci_cc_write_link_policy,
+ sizeof(struct hci_rp_write_link_policy)),
+ HCI_CC(HCI_OP_READ_DEF_LINK_POLICY, hci_cc_read_def_link_policy,
+ sizeof(struct hci_rp_read_def_link_policy)),
+ HCI_CC_STATUS(HCI_OP_WRITE_DEF_LINK_POLICY,
+ hci_cc_write_def_link_policy),
+ HCI_CC_STATUS(HCI_OP_RESET, hci_cc_reset),
+ HCI_CC(HCI_OP_READ_STORED_LINK_KEY, hci_cc_read_stored_link_key,
+ sizeof(struct hci_rp_read_stored_link_key)),
+ HCI_CC(HCI_OP_DELETE_STORED_LINK_KEY, hci_cc_delete_stored_link_key,
+ sizeof(struct hci_rp_delete_stored_link_key)),
+ HCI_CC_STATUS(HCI_OP_WRITE_LOCAL_NAME, hci_cc_write_local_name),
+ HCI_CC(HCI_OP_READ_LOCAL_NAME, hci_cc_read_local_name,
+ sizeof(struct hci_rp_read_local_name)),
+ HCI_CC_STATUS(HCI_OP_WRITE_AUTH_ENABLE, hci_cc_write_auth_enable),
+ HCI_CC_STATUS(HCI_OP_WRITE_ENCRYPT_MODE, hci_cc_write_encrypt_mode),
+ HCI_CC_STATUS(HCI_OP_WRITE_SCAN_ENABLE, hci_cc_write_scan_enable),
+ HCI_CC_STATUS(HCI_OP_SET_EVENT_FLT, hci_cc_set_event_filter),
+ HCI_CC(HCI_OP_READ_CLASS_OF_DEV, hci_cc_read_class_of_dev,
+ sizeof(struct hci_rp_read_class_of_dev)),
+ HCI_CC_STATUS(HCI_OP_WRITE_CLASS_OF_DEV, hci_cc_write_class_of_dev),
+ HCI_CC(HCI_OP_READ_VOICE_SETTING, hci_cc_read_voice_setting,
+ sizeof(struct hci_rp_read_voice_setting)),
+ HCI_CC_STATUS(HCI_OP_WRITE_VOICE_SETTING, hci_cc_write_voice_setting),
+ HCI_CC(HCI_OP_READ_NUM_SUPPORTED_IAC, hci_cc_read_num_supported_iac,
+ sizeof(struct hci_rp_read_num_supported_iac)),
+ HCI_CC_STATUS(HCI_OP_WRITE_SSP_MODE, hci_cc_write_ssp_mode),
+ HCI_CC_STATUS(HCI_OP_WRITE_SC_SUPPORT, hci_cc_write_sc_support),
+ HCI_CC(HCI_OP_READ_AUTH_PAYLOAD_TO, hci_cc_read_auth_payload_timeout,
+ sizeof(struct hci_rp_read_auth_payload_to)),
+ HCI_CC(HCI_OP_WRITE_AUTH_PAYLOAD_TO, hci_cc_write_auth_payload_timeout,
+ sizeof(struct hci_rp_write_auth_payload_to)),
+ HCI_CC(HCI_OP_READ_LOCAL_VERSION, hci_cc_read_local_version,
+ sizeof(struct hci_rp_read_local_version)),
+ HCI_CC(HCI_OP_READ_LOCAL_COMMANDS, hci_cc_read_local_commands,
+ sizeof(struct hci_rp_read_local_commands)),
+ HCI_CC(HCI_OP_READ_LOCAL_FEATURES, hci_cc_read_local_features,
+ sizeof(struct hci_rp_read_local_features)),
+ HCI_CC(HCI_OP_READ_LOCAL_EXT_FEATURES, hci_cc_read_local_ext_features,
+ sizeof(struct hci_rp_read_local_ext_features)),
+ HCI_CC(HCI_OP_READ_BUFFER_SIZE, hci_cc_read_buffer_size,
+ sizeof(struct hci_rp_read_buffer_size)),
+ HCI_CC(HCI_OP_READ_BD_ADDR, hci_cc_read_bd_addr,
+ sizeof(struct hci_rp_read_bd_addr)),
+ HCI_CC(HCI_OP_READ_LOCAL_PAIRING_OPTS, hci_cc_read_local_pairing_opts,
+ sizeof(struct hci_rp_read_local_pairing_opts)),
+ HCI_CC(HCI_OP_READ_PAGE_SCAN_ACTIVITY, hci_cc_read_page_scan_activity,
+ sizeof(struct hci_rp_read_page_scan_activity)),
+ HCI_CC_STATUS(HCI_OP_WRITE_PAGE_SCAN_ACTIVITY,
+ hci_cc_write_page_scan_activity),
+ HCI_CC(HCI_OP_READ_PAGE_SCAN_TYPE, hci_cc_read_page_scan_type,
+ sizeof(struct hci_rp_read_page_scan_type)),
+ HCI_CC_STATUS(HCI_OP_WRITE_PAGE_SCAN_TYPE, hci_cc_write_page_scan_type),
+ HCI_CC(HCI_OP_READ_DATA_BLOCK_SIZE, hci_cc_read_data_block_size,
+ sizeof(struct hci_rp_read_data_block_size)),
+ HCI_CC(HCI_OP_READ_FLOW_CONTROL_MODE, hci_cc_read_flow_control_mode,
+ sizeof(struct hci_rp_read_flow_control_mode)),
+ HCI_CC(HCI_OP_READ_LOCAL_AMP_INFO, hci_cc_read_local_amp_info,
+ sizeof(struct hci_rp_read_local_amp_info)),
+ HCI_CC(HCI_OP_READ_CLOCK, hci_cc_read_clock,
+ sizeof(struct hci_rp_read_clock)),
+ HCI_CC(HCI_OP_READ_INQ_RSP_TX_POWER, hci_cc_read_inq_rsp_tx_power,
+ sizeof(struct hci_rp_read_inq_rsp_tx_power)),
+ HCI_CC(HCI_OP_READ_DEF_ERR_DATA_REPORTING,
+ hci_cc_read_def_err_data_reporting,
+ sizeof(struct hci_rp_read_def_err_data_reporting)),
+ HCI_CC_STATUS(HCI_OP_WRITE_DEF_ERR_DATA_REPORTING,
+ hci_cc_write_def_err_data_reporting),
+ HCI_CC(HCI_OP_PIN_CODE_REPLY, hci_cc_pin_code_reply,
+ sizeof(struct hci_rp_pin_code_reply)),
+ HCI_CC(HCI_OP_PIN_CODE_NEG_REPLY, hci_cc_pin_code_neg_reply,
+ sizeof(struct hci_rp_pin_code_neg_reply)),
+ HCI_CC(HCI_OP_READ_LOCAL_OOB_DATA, hci_cc_read_local_oob_data,
+ sizeof(struct hci_rp_read_local_oob_data)),
+ HCI_CC(HCI_OP_READ_LOCAL_OOB_EXT_DATA, hci_cc_read_local_oob_ext_data,
+ sizeof(struct hci_rp_read_local_oob_ext_data)),
+ HCI_CC(HCI_OP_LE_READ_BUFFER_SIZE, hci_cc_le_read_buffer_size,
+ sizeof(struct hci_rp_le_read_buffer_size)),
+ HCI_CC(HCI_OP_LE_READ_LOCAL_FEATURES, hci_cc_le_read_local_features,
+ sizeof(struct hci_rp_le_read_local_features)),
+ HCI_CC(HCI_OP_LE_READ_ADV_TX_POWER, hci_cc_le_read_adv_tx_power,
+ sizeof(struct hci_rp_le_read_adv_tx_power)),
+ HCI_CC(HCI_OP_USER_CONFIRM_REPLY, hci_cc_user_confirm_reply,
+ sizeof(struct hci_rp_user_confirm_reply)),
+ HCI_CC(HCI_OP_USER_CONFIRM_NEG_REPLY, hci_cc_user_confirm_neg_reply,
+ sizeof(struct hci_rp_user_confirm_reply)),
+ HCI_CC(HCI_OP_USER_PASSKEY_REPLY, hci_cc_user_passkey_reply,
+ sizeof(struct hci_rp_user_confirm_reply)),
+ HCI_CC(HCI_OP_USER_PASSKEY_NEG_REPLY, hci_cc_user_passkey_neg_reply,
+ sizeof(struct hci_rp_user_confirm_reply)),
+ HCI_CC_STATUS(HCI_OP_LE_SET_RANDOM_ADDR, hci_cc_le_set_random_addr),
+ HCI_CC_STATUS(HCI_OP_LE_SET_ADV_ENABLE, hci_cc_le_set_adv_enable),
+ HCI_CC_STATUS(HCI_OP_LE_SET_SCAN_PARAM, hci_cc_le_set_scan_param),
+ HCI_CC_STATUS(HCI_OP_LE_SET_SCAN_ENABLE, hci_cc_le_set_scan_enable),
+ HCI_CC(HCI_OP_LE_READ_ACCEPT_LIST_SIZE,
+ hci_cc_le_read_accept_list_size,
+ sizeof(struct hci_rp_le_read_accept_list_size)),
+ HCI_CC_STATUS(HCI_OP_LE_CLEAR_ACCEPT_LIST, hci_cc_le_clear_accept_list),
+ HCI_CC_STATUS(HCI_OP_LE_ADD_TO_ACCEPT_LIST,
+ hci_cc_le_add_to_accept_list),
+ HCI_CC_STATUS(HCI_OP_LE_DEL_FROM_ACCEPT_LIST,
+ hci_cc_le_del_from_accept_list),
+ HCI_CC(HCI_OP_LE_READ_SUPPORTED_STATES, hci_cc_le_read_supported_states,
+ sizeof(struct hci_rp_le_read_supported_states)),
+ HCI_CC(HCI_OP_LE_READ_DEF_DATA_LEN, hci_cc_le_read_def_data_len,
+ sizeof(struct hci_rp_le_read_def_data_len)),
+ HCI_CC_STATUS(HCI_OP_LE_WRITE_DEF_DATA_LEN,
+ hci_cc_le_write_def_data_len),
+ HCI_CC_STATUS(HCI_OP_LE_ADD_TO_RESOLV_LIST,
+ hci_cc_le_add_to_resolv_list),
+ HCI_CC_STATUS(HCI_OP_LE_DEL_FROM_RESOLV_LIST,
+ hci_cc_le_del_from_resolv_list),
+ HCI_CC_STATUS(HCI_OP_LE_CLEAR_RESOLV_LIST,
+ hci_cc_le_clear_resolv_list),
+ HCI_CC(HCI_OP_LE_READ_RESOLV_LIST_SIZE, hci_cc_le_read_resolv_list_size,
+ sizeof(struct hci_rp_le_read_resolv_list_size)),
+ HCI_CC_STATUS(HCI_OP_LE_SET_ADDR_RESOLV_ENABLE,
+ hci_cc_le_set_addr_resolution_enable),
+ HCI_CC(HCI_OP_LE_READ_MAX_DATA_LEN, hci_cc_le_read_max_data_len,
+ sizeof(struct hci_rp_le_read_max_data_len)),
+ HCI_CC_STATUS(HCI_OP_WRITE_LE_HOST_SUPPORTED,
+ hci_cc_write_le_host_supported),
+ HCI_CC_STATUS(HCI_OP_LE_SET_ADV_PARAM, hci_cc_set_adv_param),
+ HCI_CC(HCI_OP_READ_RSSI, hci_cc_read_rssi,
+ sizeof(struct hci_rp_read_rssi)),
+ HCI_CC(HCI_OP_READ_TX_POWER, hci_cc_read_tx_power,
+ sizeof(struct hci_rp_read_tx_power)),
+ HCI_CC_STATUS(HCI_OP_WRITE_SSP_DEBUG_MODE, hci_cc_write_ssp_debug_mode),
+ HCI_CC_STATUS(HCI_OP_LE_SET_EXT_SCAN_PARAMS,
+ hci_cc_le_set_ext_scan_param),
+ HCI_CC_STATUS(HCI_OP_LE_SET_EXT_SCAN_ENABLE,
+ hci_cc_le_set_ext_scan_enable),
+ HCI_CC_STATUS(HCI_OP_LE_SET_DEFAULT_PHY, hci_cc_le_set_default_phy),
+ HCI_CC(HCI_OP_LE_READ_NUM_SUPPORTED_ADV_SETS,
+ hci_cc_le_read_num_adv_sets,
+ sizeof(struct hci_rp_le_read_num_supported_adv_sets)),
+ HCI_CC(HCI_OP_LE_SET_EXT_ADV_PARAMS, hci_cc_set_ext_adv_param,
+ sizeof(struct hci_rp_le_set_ext_adv_params)),
+ HCI_CC_STATUS(HCI_OP_LE_SET_EXT_ADV_ENABLE,
+ hci_cc_le_set_ext_adv_enable),
+ HCI_CC_STATUS(HCI_OP_LE_SET_ADV_SET_RAND_ADDR,
+ hci_cc_le_set_adv_set_random_addr),
+ HCI_CC_STATUS(HCI_OP_LE_REMOVE_ADV_SET, hci_cc_le_remove_adv_set),
+ HCI_CC_STATUS(HCI_OP_LE_CLEAR_ADV_SETS, hci_cc_le_clear_adv_sets),
+ HCI_CC(HCI_OP_LE_READ_TRANSMIT_POWER, hci_cc_le_read_transmit_power,
+ sizeof(struct hci_rp_le_read_transmit_power)),
+ HCI_CC_STATUS(HCI_OP_LE_SET_PRIVACY_MODE, hci_cc_le_set_privacy_mode)
+};
+
+static u8 hci_cc_func(struct hci_dev *hdev, const struct hci_cc *cc,
+ struct sk_buff *skb)
+{
+ void *data;
+
+ if (skb->len < cc->min_len) {
+ bt_dev_err(hdev, "unexpected cc 0x%4.4x length: %u < %u",
+ cc->op, skb->len, cc->min_len);
+ return HCI_ERROR_UNSPECIFIED;
+ }
+
+ /* Just warn if the length is over max_len size it still be possible to
+ * partially parse the cc so leave to callback to decide if that is
+ * acceptable.
+ */
+ if (skb->len > cc->max_len)
+ bt_dev_warn(hdev, "unexpected cc 0x%4.4x length: %u > %u",
+ cc->op, skb->len, cc->max_len);
+
+ data = hci_cc_skb_pull(hdev, skb, cc->op, cc->min_len);
+ if (!data)
+ return HCI_ERROR_UNSPECIFIED;
+
+ return cc->func(hdev, data, skb);
+}
+
+static void hci_cmd_complete_evt(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb, u16 *opcode, u8 *status,
hci_req_complete_t *req_complete,
hci_req_complete_skb_t *req_complete_skb)
{
- struct hci_ev_cmd_complete *ev = (void *) skb->data;
+ struct hci_ev_cmd_complete *ev = data;
+ int i;
*opcode = __le16_to_cpu(ev->opcode);
- *status = skb->data[sizeof(*ev)];
- skb_pull(skb, sizeof(*ev));
+ bt_dev_dbg(hdev, "opcode 0x%4.4x", *opcode);
- switch (*opcode) {
- case HCI_OP_INQUIRY_CANCEL:
- hci_cc_inquiry_cancel(hdev, skb, status);
- break;
-
- case HCI_OP_PERIODIC_INQ:
- hci_cc_periodic_inq(hdev, skb);
- break;
-
- case HCI_OP_EXIT_PERIODIC_INQ:
- hci_cc_exit_periodic_inq(hdev, skb);
- break;
-
- case HCI_OP_REMOTE_NAME_REQ_CANCEL:
- hci_cc_remote_name_req_cancel(hdev, skb);
- break;
-
- case HCI_OP_ROLE_DISCOVERY:
- hci_cc_role_discovery(hdev, skb);
- break;
-
- case HCI_OP_READ_LINK_POLICY:
- hci_cc_read_link_policy(hdev, skb);
- break;
-
- case HCI_OP_WRITE_LINK_POLICY:
- hci_cc_write_link_policy(hdev, skb);
- break;
-
- case HCI_OP_READ_DEF_LINK_POLICY:
- hci_cc_read_def_link_policy(hdev, skb);
- break;
-
- case HCI_OP_WRITE_DEF_LINK_POLICY:
- hci_cc_write_def_link_policy(hdev, skb);
- break;
-
- case HCI_OP_RESET:
- hci_cc_reset(hdev, skb);
- break;
-
- case HCI_OP_READ_STORED_LINK_KEY:
- hci_cc_read_stored_link_key(hdev, skb);
- break;
-
- case HCI_OP_DELETE_STORED_LINK_KEY:
- hci_cc_delete_stored_link_key(hdev, skb);
- break;
-
- case HCI_OP_WRITE_LOCAL_NAME:
- hci_cc_write_local_name(hdev, skb);
- break;
-
- case HCI_OP_READ_LOCAL_NAME:
- hci_cc_read_local_name(hdev, skb);
- break;
-
- case HCI_OP_WRITE_AUTH_ENABLE:
- hci_cc_write_auth_enable(hdev, skb);
- break;
-
- case HCI_OP_WRITE_ENCRYPT_MODE:
- hci_cc_write_encrypt_mode(hdev, skb);
- break;
-
- case HCI_OP_WRITE_SCAN_ENABLE:
- hci_cc_write_scan_enable(hdev, skb);
- break;
-
- case HCI_OP_SET_EVENT_FLT:
- hci_cc_set_event_filter(hdev, skb);
- break;
-
- case HCI_OP_READ_CLASS_OF_DEV:
- hci_cc_read_class_of_dev(hdev, skb);
- break;
-
- case HCI_OP_WRITE_CLASS_OF_DEV:
- hci_cc_write_class_of_dev(hdev, skb);
- break;
-
- case HCI_OP_READ_VOICE_SETTING:
- hci_cc_read_voice_setting(hdev, skb);
- break;
-
- case HCI_OP_WRITE_VOICE_SETTING:
- hci_cc_write_voice_setting(hdev, skb);
- break;
-
- case HCI_OP_READ_NUM_SUPPORTED_IAC:
- hci_cc_read_num_supported_iac(hdev, skb);
- break;
-
- case HCI_OP_WRITE_SSP_MODE:
- hci_cc_write_ssp_mode(hdev, skb);
- break;
-
- case HCI_OP_WRITE_SC_SUPPORT:
- hci_cc_write_sc_support(hdev, skb);
- break;
-
- case HCI_OP_READ_AUTH_PAYLOAD_TO:
- hci_cc_read_auth_payload_timeout(hdev, skb);
- break;
-
- case HCI_OP_WRITE_AUTH_PAYLOAD_TO:
- hci_cc_write_auth_payload_timeout(hdev, skb);
- break;
-
- case HCI_OP_READ_LOCAL_VERSION:
- hci_cc_read_local_version(hdev, skb);
- break;
-
- case HCI_OP_READ_LOCAL_COMMANDS:
- hci_cc_read_local_commands(hdev, skb);
- break;
-
- case HCI_OP_READ_LOCAL_FEATURES:
- hci_cc_read_local_features(hdev, skb);
- break;
-
- case HCI_OP_READ_LOCAL_EXT_FEATURES:
- hci_cc_read_local_ext_features(hdev, skb);
- break;
-
- case HCI_OP_READ_BUFFER_SIZE:
- hci_cc_read_buffer_size(hdev, skb);
- break;
-
- case HCI_OP_READ_BD_ADDR:
- hci_cc_read_bd_addr(hdev, skb);
- break;
-
- case HCI_OP_READ_LOCAL_PAIRING_OPTS:
- hci_cc_read_local_pairing_opts(hdev, skb);
- break;
-
- case HCI_OP_READ_PAGE_SCAN_ACTIVITY:
- hci_cc_read_page_scan_activity(hdev, skb);
- break;
-
- case HCI_OP_WRITE_PAGE_SCAN_ACTIVITY:
- hci_cc_write_page_scan_activity(hdev, skb);
- break;
-
- case HCI_OP_READ_PAGE_SCAN_TYPE:
- hci_cc_read_page_scan_type(hdev, skb);
- break;
-
- case HCI_OP_WRITE_PAGE_SCAN_TYPE:
- hci_cc_write_page_scan_type(hdev, skb);
- break;
-
- case HCI_OP_READ_DATA_BLOCK_SIZE:
- hci_cc_read_data_block_size(hdev, skb);
- break;
-
- case HCI_OP_READ_FLOW_CONTROL_MODE:
- hci_cc_read_flow_control_mode(hdev, skb);
- break;
-
- case HCI_OP_READ_LOCAL_AMP_INFO:
- hci_cc_read_local_amp_info(hdev, skb);
- break;
-
- case HCI_OP_READ_CLOCK:
- hci_cc_read_clock(hdev, skb);
- break;
-
- case HCI_OP_READ_INQ_RSP_TX_POWER:
- hci_cc_read_inq_rsp_tx_power(hdev, skb);
- break;
-
- case HCI_OP_READ_DEF_ERR_DATA_REPORTING:
- hci_cc_read_def_err_data_reporting(hdev, skb);
- break;
-
- case HCI_OP_WRITE_DEF_ERR_DATA_REPORTING:
- hci_cc_write_def_err_data_reporting(hdev, skb);
- break;
-
- case HCI_OP_PIN_CODE_REPLY:
- hci_cc_pin_code_reply(hdev, skb);
- break;
-
- case HCI_OP_PIN_CODE_NEG_REPLY:
- hci_cc_pin_code_neg_reply(hdev, skb);
- break;
-
- case HCI_OP_READ_LOCAL_OOB_DATA:
- hci_cc_read_local_oob_data(hdev, skb);
- break;
-
- case HCI_OP_READ_LOCAL_OOB_EXT_DATA:
- hci_cc_read_local_oob_ext_data(hdev, skb);
- break;
-
- case HCI_OP_LE_READ_BUFFER_SIZE:
- hci_cc_le_read_buffer_size(hdev, skb);
- break;
-
- case HCI_OP_LE_READ_LOCAL_FEATURES:
- hci_cc_le_read_local_features(hdev, skb);
- break;
-
- case HCI_OP_LE_READ_ADV_TX_POWER:
- hci_cc_le_read_adv_tx_power(hdev, skb);
- break;
-
- case HCI_OP_USER_CONFIRM_REPLY:
- hci_cc_user_confirm_reply(hdev, skb);
- break;
-
- case HCI_OP_USER_CONFIRM_NEG_REPLY:
- hci_cc_user_confirm_neg_reply(hdev, skb);
- break;
-
- case HCI_OP_USER_PASSKEY_REPLY:
- hci_cc_user_passkey_reply(hdev, skb);
- break;
-
- case HCI_OP_USER_PASSKEY_NEG_REPLY:
- hci_cc_user_passkey_neg_reply(hdev, skb);
- break;
-
- case HCI_OP_LE_SET_RANDOM_ADDR:
- hci_cc_le_set_random_addr(hdev, skb);
- break;
-
- case HCI_OP_LE_SET_ADV_ENABLE:
- hci_cc_le_set_adv_enable(hdev, skb);
- break;
-
- case HCI_OP_LE_SET_SCAN_PARAM:
- hci_cc_le_set_scan_param(hdev, skb);
- break;
-
- case HCI_OP_LE_SET_SCAN_ENABLE:
- hci_cc_le_set_scan_enable(hdev, skb);
- break;
-
- case HCI_OP_LE_READ_ACCEPT_LIST_SIZE:
- hci_cc_le_read_accept_list_size(hdev, skb);
- break;
-
- case HCI_OP_LE_CLEAR_ACCEPT_LIST:
- hci_cc_le_clear_accept_list(hdev, skb);
- break;
-
- case HCI_OP_LE_ADD_TO_ACCEPT_LIST:
- hci_cc_le_add_to_accept_list(hdev, skb);
- break;
-
- case HCI_OP_LE_DEL_FROM_ACCEPT_LIST:
- hci_cc_le_del_from_accept_list(hdev, skb);
- break;
-
- case HCI_OP_LE_READ_SUPPORTED_STATES:
- hci_cc_le_read_supported_states(hdev, skb);
- break;
-
- case HCI_OP_LE_READ_DEF_DATA_LEN:
- hci_cc_le_read_def_data_len(hdev, skb);
- break;
-
- case HCI_OP_LE_WRITE_DEF_DATA_LEN:
- hci_cc_le_write_def_data_len(hdev, skb);
- break;
-
- case HCI_OP_LE_ADD_TO_RESOLV_LIST:
- hci_cc_le_add_to_resolv_list(hdev, skb);
- break;
-
- case HCI_OP_LE_DEL_FROM_RESOLV_LIST:
- hci_cc_le_del_from_resolv_list(hdev, skb);
- break;
-
- case HCI_OP_LE_CLEAR_RESOLV_LIST:
- hci_cc_le_clear_resolv_list(hdev, skb);
- break;
-
- case HCI_OP_LE_READ_RESOLV_LIST_SIZE:
- hci_cc_le_read_resolv_list_size(hdev, skb);
- break;
-
- case HCI_OP_LE_SET_ADDR_RESOLV_ENABLE:
- hci_cc_le_set_addr_resolution_enable(hdev, skb);
- break;
-
- case HCI_OP_LE_READ_MAX_DATA_LEN:
- hci_cc_le_read_max_data_len(hdev, skb);
- break;
-
- case HCI_OP_WRITE_LE_HOST_SUPPORTED:
- hci_cc_write_le_host_supported(hdev, skb);
- break;
-
- case HCI_OP_LE_SET_ADV_PARAM:
- hci_cc_set_adv_param(hdev, skb);
- break;
-
- case HCI_OP_READ_RSSI:
- hci_cc_read_rssi(hdev, skb);
- break;
-
- case HCI_OP_READ_TX_POWER:
- hci_cc_read_tx_power(hdev, skb);
- break;
-
- case HCI_OP_WRITE_SSP_DEBUG_MODE:
- hci_cc_write_ssp_debug_mode(hdev, skb);
- break;
-
- case HCI_OP_LE_SET_EXT_SCAN_PARAMS:
- hci_cc_le_set_ext_scan_param(hdev, skb);
- break;
-
- case HCI_OP_LE_SET_EXT_SCAN_ENABLE:
- hci_cc_le_set_ext_scan_enable(hdev, skb);
- break;
-
- case HCI_OP_LE_SET_DEFAULT_PHY:
- hci_cc_le_set_default_phy(hdev, skb);
- break;
-
- case HCI_OP_LE_READ_NUM_SUPPORTED_ADV_SETS:
- hci_cc_le_read_num_adv_sets(hdev, skb);
- break;
-
- case HCI_OP_LE_SET_EXT_ADV_PARAMS:
- hci_cc_set_ext_adv_param(hdev, skb);
- break;
-
- case HCI_OP_LE_SET_EXT_ADV_ENABLE:
- hci_cc_le_set_ext_adv_enable(hdev, skb);
- break;
-
- case HCI_OP_LE_SET_ADV_SET_RAND_ADDR:
- hci_cc_le_set_adv_set_random_addr(hdev, skb);
- break;
-
- case HCI_OP_LE_READ_TRANSMIT_POWER:
- hci_cc_le_read_transmit_power(hdev, skb);
- break;
-
- default:
- BT_DBG("%s opcode 0x%4.4x", hdev->name, *opcode);
- break;
+ for (i = 0; i < ARRAY_SIZE(hci_cc_table); i++) {
+ if (hci_cc_table[i].op == *opcode) {
+ *status = hci_cc_func(hdev, &hci_cc_table[i], skb);
+ break;
+ }
}
handle_cmd_cnt_and_timer(hdev, ev->ncmd);
@@ -3747,94 +3978,56 @@ static void hci_cmd_complete_evt(struct hci_dev *hdev, struct sk_buff *skb,
queue_work(hdev->workqueue, &hdev->cmd_work);
}
-static void hci_cmd_status_evt(struct hci_dev *hdev, struct sk_buff *skb,
- u16 *opcode, u8 *status,
+#define HCI_CS(_op, _func) \
+{ \
+ .op = _op, \
+ .func = _func, \
+}
+
+static const struct hci_cs {
+ u16 op;
+ void (*func)(struct hci_dev *hdev, __u8 status);
+} hci_cs_table[] = {
+ HCI_CS(HCI_OP_INQUIRY, hci_cs_inquiry),
+ HCI_CS(HCI_OP_CREATE_CONN, hci_cs_create_conn),
+ HCI_CS(HCI_OP_DISCONNECT, hci_cs_disconnect),
+ HCI_CS(HCI_OP_ADD_SCO, hci_cs_add_sco),
+ HCI_CS(HCI_OP_AUTH_REQUESTED, hci_cs_auth_requested),
+ HCI_CS(HCI_OP_SET_CONN_ENCRYPT, hci_cs_set_conn_encrypt),
+ HCI_CS(HCI_OP_REMOTE_NAME_REQ, hci_cs_remote_name_req),
+ HCI_CS(HCI_OP_READ_REMOTE_FEATURES, hci_cs_read_remote_features),
+ HCI_CS(HCI_OP_READ_REMOTE_EXT_FEATURES,
+ hci_cs_read_remote_ext_features),
+ HCI_CS(HCI_OP_SETUP_SYNC_CONN, hci_cs_setup_sync_conn),
+ HCI_CS(HCI_OP_ENHANCED_SETUP_SYNC_CONN,
+ hci_cs_enhanced_setup_sync_conn),
+ HCI_CS(HCI_OP_SNIFF_MODE, hci_cs_sniff_mode),
+ HCI_CS(HCI_OP_EXIT_SNIFF_MODE, hci_cs_exit_sniff_mode),
+ HCI_CS(HCI_OP_SWITCH_ROLE, hci_cs_switch_role),
+ HCI_CS(HCI_OP_LE_CREATE_CONN, hci_cs_le_create_conn),
+ HCI_CS(HCI_OP_LE_READ_REMOTE_FEATURES, hci_cs_le_read_remote_features),
+ HCI_CS(HCI_OP_LE_START_ENC, hci_cs_le_start_enc),
+ HCI_CS(HCI_OP_LE_EXT_CREATE_CONN, hci_cs_le_ext_create_conn)
+};
+
+static void hci_cmd_status_evt(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb, u16 *opcode, u8 *status,
hci_req_complete_t *req_complete,
hci_req_complete_skb_t *req_complete_skb)
{
- struct hci_ev_cmd_status *ev = (void *) skb->data;
-
- skb_pull(skb, sizeof(*ev));
+ struct hci_ev_cmd_status *ev = data;
+ int i;
*opcode = __le16_to_cpu(ev->opcode);
*status = ev->status;
- switch (*opcode) {
- case HCI_OP_INQUIRY:
- hci_cs_inquiry(hdev, ev->status);
- break;
+ bt_dev_dbg(hdev, "opcode 0x%4.4x", *opcode);
- case HCI_OP_CREATE_CONN:
- hci_cs_create_conn(hdev, ev->status);
- break;
-
- case HCI_OP_DISCONNECT:
- hci_cs_disconnect(hdev, ev->status);
- break;
-
- case HCI_OP_ADD_SCO:
- hci_cs_add_sco(hdev, ev->status);
- break;
-
- case HCI_OP_AUTH_REQUESTED:
- hci_cs_auth_requested(hdev, ev->status);
- break;
-
- case HCI_OP_SET_CONN_ENCRYPT:
- hci_cs_set_conn_encrypt(hdev, ev->status);
- break;
-
- case HCI_OP_REMOTE_NAME_REQ:
- hci_cs_remote_name_req(hdev, ev->status);
- break;
-
- case HCI_OP_READ_REMOTE_FEATURES:
- hci_cs_read_remote_features(hdev, ev->status);
- break;
-
- case HCI_OP_READ_REMOTE_EXT_FEATURES:
- hci_cs_read_remote_ext_features(hdev, ev->status);
- break;
-
- case HCI_OP_SETUP_SYNC_CONN:
- hci_cs_setup_sync_conn(hdev, ev->status);
- break;
-
- case HCI_OP_ENHANCED_SETUP_SYNC_CONN:
- hci_cs_enhanced_setup_sync_conn(hdev, ev->status);
- break;
-
- case HCI_OP_SNIFF_MODE:
- hci_cs_sniff_mode(hdev, ev->status);
- break;
-
- case HCI_OP_EXIT_SNIFF_MODE:
- hci_cs_exit_sniff_mode(hdev, ev->status);
- break;
-
- case HCI_OP_SWITCH_ROLE:
- hci_cs_switch_role(hdev, ev->status);
- break;
-
- case HCI_OP_LE_CREATE_CONN:
- hci_cs_le_create_conn(hdev, ev->status);
- break;
-
- case HCI_OP_LE_READ_REMOTE_FEATURES:
- hci_cs_le_read_remote_features(hdev, ev->status);
- break;
-
- case HCI_OP_LE_START_ENC:
- hci_cs_le_start_enc(hdev, ev->status);
- break;
-
- case HCI_OP_LE_EXT_CREATE_CONN:
- hci_cs_le_ext_create_conn(hdev, ev->status);
- break;
-
- default:
- BT_DBG("%s opcode 0x%4.4x", hdev->name, *opcode);
- break;
+ for (i = 0; i < ARRAY_SIZE(hci_cs_table); i++) {
+ if (hci_cs_table[i].op == *opcode) {
+ hci_cs_table[i].func(hdev, ev->status);
+ break;
+ }
}
handle_cmd_cnt_and_timer(hdev, ev->ncmd);
@@ -3845,36 +4038,39 @@ static void hci_cmd_status_evt(struct hci_dev *hdev, struct sk_buff *skb,
* (since for this kind of commands there will not be a command
* complete event).
*/
- if (ev->status ||
- (hdev->sent_cmd && !bt_cb(hdev->sent_cmd)->hci.req_event))
+ if (ev->status || (hdev->sent_cmd && !hci_skb_event(hdev->sent_cmd))) {
hci_req_cmd_complete(hdev, *opcode, ev->status, req_complete,
req_complete_skb);
-
- if (hci_dev_test_flag(hdev, HCI_CMD_PENDING)) {
- bt_dev_err(hdev,
- "unexpected event for opcode 0x%4.4x", *opcode);
- return;
+ if (hci_dev_test_flag(hdev, HCI_CMD_PENDING)) {
+ bt_dev_err(hdev, "unexpected event for opcode 0x%4.4x",
+ *opcode);
+ return;
+ }
}
if (atomic_read(&hdev->cmd_cnt) && !skb_queue_empty(&hdev->cmd_q))
queue_work(hdev->workqueue, &hdev->cmd_work);
}
-static void hci_hardware_error_evt(struct hci_dev *hdev, struct sk_buff *skb)
+static void hci_hardware_error_evt(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_ev_hardware_error *ev = (void *) skb->data;
+ struct hci_ev_hardware_error *ev = data;
+
+ bt_dev_dbg(hdev, "code 0x%2.2x", ev->code);
hdev->hw_error_code = ev->code;
queue_work(hdev->req_workqueue, &hdev->error_reset);
}
-static void hci_role_change_evt(struct hci_dev *hdev, struct sk_buff *skb)
+static void hci_role_change_evt(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_ev_role_change *ev = (void *) skb->data;
+ struct hci_ev_role_change *ev = data;
struct hci_conn *conn;
- BT_DBG("%s status 0x%2.2x", hdev->name, ev->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", ev->status);
hci_dev_lock(hdev);
@@ -3891,25 +4087,24 @@ static void hci_role_change_evt(struct hci_dev *hdev, struct sk_buff *skb)
hci_dev_unlock(hdev);
}
-static void hci_num_comp_pkts_evt(struct hci_dev *hdev, struct sk_buff *skb)
+static void hci_num_comp_pkts_evt(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_ev_num_comp_pkts *ev = (void *) skb->data;
+ struct hci_ev_num_comp_pkts *ev = data;
int i;
+ if (!hci_ev_skb_pull(hdev, skb, HCI_EV_NUM_COMP_PKTS,
+ flex_array_size(ev, handles, ev->num)))
+ return;
+
if (hdev->flow_ctl_mode != HCI_FLOW_CTL_MODE_PACKET_BASED) {
bt_dev_err(hdev, "wrong event for mode %d", hdev->flow_ctl_mode);
return;
}
- if (skb->len < sizeof(*ev) ||
- skb->len < struct_size(ev, handles, ev->num_hndl)) {
- BT_DBG("%s bad parameters", hdev->name);
- return;
- }
+ bt_dev_dbg(hdev, "num %d", ev->num);
- BT_DBG("%s num_hndl %d", hdev->name, ev->num_hndl);
-
- for (i = 0; i < ev->num_hndl; i++) {
+ for (i = 0; i < ev->num; i++) {
struct hci_comp_pkts_info *info = &ev->handles[i];
struct hci_conn *conn;
__u16 handle, count;
@@ -3979,24 +4174,24 @@ static struct hci_conn *__hci_conn_lookup_handle(struct hci_dev *hdev,
return NULL;
}
-static void hci_num_comp_blocks_evt(struct hci_dev *hdev, struct sk_buff *skb)
+static void hci_num_comp_blocks_evt(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_ev_num_comp_blocks *ev = (void *) skb->data;
+ struct hci_ev_num_comp_blocks *ev = data;
int i;
+ if (!hci_ev_skb_pull(hdev, skb, HCI_EV_NUM_COMP_BLOCKS,
+ flex_array_size(ev, handles, ev->num_hndl)))
+ return;
+
if (hdev->flow_ctl_mode != HCI_FLOW_CTL_MODE_BLOCK_BASED) {
- bt_dev_err(hdev, "wrong event for mode %d", hdev->flow_ctl_mode);
+ bt_dev_err(hdev, "wrong event for mode %d",
+ hdev->flow_ctl_mode);
return;
}
- if (skb->len < sizeof(*ev) ||
- skb->len < struct_size(ev, handles, ev->num_hndl)) {
- BT_DBG("%s bad parameters", hdev->name);
- return;
- }
-
- BT_DBG("%s num_blocks %d num_hndl %d", hdev->name, ev->num_blocks,
- ev->num_hndl);
+ bt_dev_dbg(hdev, "num_blocks %d num_hndl %d", ev->num_blocks,
+ ev->num_hndl);
for (i = 0; i < ev->num_hndl; i++) {
struct hci_comp_blocks_info *info = &ev->handles[i];
@@ -4030,12 +4225,13 @@ static void hci_num_comp_blocks_evt(struct hci_dev *hdev, struct sk_buff *skb)
queue_work(hdev->workqueue, &hdev->tx_work);
}
-static void hci_mode_change_evt(struct hci_dev *hdev, struct sk_buff *skb)
+static void hci_mode_change_evt(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_ev_mode_change *ev = (void *) skb->data;
+ struct hci_ev_mode_change *ev = data;
struct hci_conn *conn;
- BT_DBG("%s status 0x%2.2x", hdev->name, ev->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", ev->status);
hci_dev_lock(hdev);
@@ -4058,12 +4254,13 @@ static void hci_mode_change_evt(struct hci_dev *hdev, struct sk_buff *skb)
hci_dev_unlock(hdev);
}
-static void hci_pin_code_request_evt(struct hci_dev *hdev, struct sk_buff *skb)
+static void hci_pin_code_request_evt(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_ev_pin_code_req *ev = (void *) skb->data;
+ struct hci_ev_pin_code_req *ev = data;
struct hci_conn *conn;
- BT_DBG("%s", hdev->name);
+ bt_dev_dbg(hdev, "");
hci_dev_lock(hdev);
@@ -4128,14 +4325,15 @@ static void conn_set_key(struct hci_conn *conn, u8 key_type, u8 pin_len)
}
}
-static void hci_link_key_request_evt(struct hci_dev *hdev, struct sk_buff *skb)
+static void hci_link_key_request_evt(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_ev_link_key_req *ev = (void *) skb->data;
+ struct hci_ev_link_key_req *ev = data;
struct hci_cp_link_key_reply cp;
struct hci_conn *conn;
struct link_key *key;
- BT_DBG("%s", hdev->name);
+ bt_dev_dbg(hdev, "");
if (!hci_dev_test_flag(hdev, HCI_MGMT))
return;
@@ -4144,13 +4342,11 @@ static void hci_link_key_request_evt(struct hci_dev *hdev, struct sk_buff *skb)
key = hci_find_link_key(hdev, &ev->bdaddr);
if (!key) {
- BT_DBG("%s link key not found for %pMR", hdev->name,
- &ev->bdaddr);
+ bt_dev_dbg(hdev, "link key not found for %pMR", &ev->bdaddr);
goto not_found;
}
- BT_DBG("%s found key type %u for %pMR", hdev->name, key->type,
- &ev->bdaddr);
+ bt_dev_dbg(hdev, "found key type %u for %pMR", key->type, &ev->bdaddr);
conn = hci_conn_hash_lookup_ba(hdev, ACL_LINK, &ev->bdaddr);
if (conn) {
@@ -4159,15 +4355,14 @@ static void hci_link_key_request_evt(struct hci_dev *hdev, struct sk_buff *skb)
if ((key->type == HCI_LK_UNAUTH_COMBINATION_P192 ||
key->type == HCI_LK_UNAUTH_COMBINATION_P256) &&
conn->auth_type != 0xff && (conn->auth_type & 0x01)) {
- BT_DBG("%s ignoring unauthenticated key", hdev->name);
+ bt_dev_dbg(hdev, "ignoring unauthenticated key");
goto not_found;
}
if (key->type == HCI_LK_COMBINATION && key->pin_len < 16 &&
(conn->pending_sec_level == BT_SECURITY_HIGH ||
conn->pending_sec_level == BT_SECURITY_FIPS)) {
- BT_DBG("%s ignoring key unauthenticated for high security",
- hdev->name);
+ bt_dev_dbg(hdev, "ignoring key unauthenticated for high security");
goto not_found;
}
@@ -4188,15 +4383,16 @@ static void hci_link_key_request_evt(struct hci_dev *hdev, struct sk_buff *skb)
hci_dev_unlock(hdev);
}
-static void hci_link_key_notify_evt(struct hci_dev *hdev, struct sk_buff *skb)
+static void hci_link_key_notify_evt(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_ev_link_key_notify *ev = (void *) skb->data;
+ struct hci_ev_link_key_notify *ev = data;
struct hci_conn *conn;
struct link_key *key;
bool persistent;
u8 pin_len = 0;
- BT_DBG("%s", hdev->name);
+ bt_dev_dbg(hdev, "");
hci_dev_lock(hdev);
@@ -4248,12 +4444,13 @@ static void hci_link_key_notify_evt(struct hci_dev *hdev, struct sk_buff *skb)
hci_dev_unlock(hdev);
}
-static void hci_clock_offset_evt(struct hci_dev *hdev, struct sk_buff *skb)
+static void hci_clock_offset_evt(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_ev_clock_offset *ev = (void *) skb->data;
+ struct hci_ev_clock_offset *ev = data;
struct hci_conn *conn;
- BT_DBG("%s status 0x%2.2x", hdev->name, ev->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", ev->status);
hci_dev_lock(hdev);
@@ -4271,12 +4468,13 @@ static void hci_clock_offset_evt(struct hci_dev *hdev, struct sk_buff *skb)
hci_dev_unlock(hdev);
}
-static void hci_pkt_type_change_evt(struct hci_dev *hdev, struct sk_buff *skb)
+static void hci_pkt_type_change_evt(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_ev_pkt_type_change *ev = (void *) skb->data;
+ struct hci_ev_pkt_type_change *ev = data;
struct hci_conn *conn;
- BT_DBG("%s status 0x%2.2x", hdev->name, ev->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", ev->status);
hci_dev_lock(hdev);
@@ -4287,12 +4485,13 @@ static void hci_pkt_type_change_evt(struct hci_dev *hdev, struct sk_buff *skb)
hci_dev_unlock(hdev);
}
-static void hci_pscan_rep_mode_evt(struct hci_dev *hdev, struct sk_buff *skb)
+static void hci_pscan_rep_mode_evt(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_ev_pscan_rep_mode *ev = (void *) skb->data;
+ struct hci_ev_pscan_rep_mode *ev = data;
struct inquiry_entry *ie;
- BT_DBG("%s", hdev->name);
+ bt_dev_dbg(hdev, "");
hci_dev_lock(hdev);
@@ -4305,15 +4504,16 @@ static void hci_pscan_rep_mode_evt(struct hci_dev *hdev, struct sk_buff *skb)
hci_dev_unlock(hdev);
}
-static void hci_inquiry_result_with_rssi_evt(struct hci_dev *hdev,
+static void hci_inquiry_result_with_rssi_evt(struct hci_dev *hdev, void *edata,
struct sk_buff *skb)
{
+ struct hci_ev_inquiry_result_rssi *ev = edata;
struct inquiry_data data;
- int num_rsp = *((__u8 *) skb->data);
+ int i;
- BT_DBG("%s num_rsp %d", hdev->name, num_rsp);
+ bt_dev_dbg(hdev, "num_rsp %d", ev->num);
- if (!num_rsp)
+ if (!ev->num)
return;
if (hci_dev_test_flag(hdev, HCI_PERIODIC_INQ))
@@ -4321,16 +4521,22 @@ static void hci_inquiry_result_with_rssi_evt(struct hci_dev *hdev,
hci_dev_lock(hdev);
- if ((skb->len - 1) / num_rsp != sizeof(struct inquiry_info_with_rssi)) {
- struct inquiry_info_with_rssi_and_pscan_mode *info;
- info = (void *) (skb->data + 1);
+ if (skb->len == array_size(ev->num,
+ sizeof(struct inquiry_info_rssi_pscan))) {
+ struct inquiry_info_rssi_pscan *info;
- if (skb->len < num_rsp * sizeof(*info) + 1)
- goto unlock;
-
- for (; num_rsp; num_rsp--, info++) {
+ for (i = 0; i < ev->num; i++) {
u32 flags;
+ info = hci_ev_skb_pull(hdev, skb,
+ HCI_EV_INQUIRY_RESULT_WITH_RSSI,
+ sizeof(*info));
+ if (!info) {
+ bt_dev_err(hdev, "Malformed HCI Event: 0x%2.2x",
+ HCI_EV_INQUIRY_RESULT_WITH_RSSI);
+ return;
+ }
+
bacpy(&data.bdaddr, &info->bdaddr);
data.pscan_rep_mode = info->pscan_rep_mode;
data.pscan_period_mode = info->pscan_period_mode;
@@ -4346,15 +4552,22 @@ static void hci_inquiry_result_with_rssi_evt(struct hci_dev *hdev,
info->dev_class, info->rssi,
flags, NULL, 0, NULL, 0);
}
- } else {
- struct inquiry_info_with_rssi *info = (void *) (skb->data + 1);
+ } else if (skb->len == array_size(ev->num,
+ sizeof(struct inquiry_info_rssi))) {
+ struct inquiry_info_rssi *info;
- if (skb->len < num_rsp * sizeof(*info) + 1)
- goto unlock;
-
- for (; num_rsp; num_rsp--, info++) {
+ for (i = 0; i < ev->num; i++) {
u32 flags;
+ info = hci_ev_skb_pull(hdev, skb,
+ HCI_EV_INQUIRY_RESULT_WITH_RSSI,
+ sizeof(*info));
+ if (!info) {
+ bt_dev_err(hdev, "Malformed HCI Event: 0x%2.2x",
+ HCI_EV_INQUIRY_RESULT_WITH_RSSI);
+ return;
+ }
+
bacpy(&data.bdaddr, &info->bdaddr);
data.pscan_rep_mode = info->pscan_rep_mode;
data.pscan_period_mode = info->pscan_period_mode;
@@ -4370,19 +4583,21 @@ static void hci_inquiry_result_with_rssi_evt(struct hci_dev *hdev,
info->dev_class, info->rssi,
flags, NULL, 0, NULL, 0);
}
+ } else {
+ bt_dev_err(hdev, "Malformed HCI Event: 0x%2.2x",
+ HCI_EV_INQUIRY_RESULT_WITH_RSSI);
}
-unlock:
hci_dev_unlock(hdev);
}
-static void hci_remote_ext_features_evt(struct hci_dev *hdev,
+static void hci_remote_ext_features_evt(struct hci_dev *hdev, void *data,
struct sk_buff *skb)
{
- struct hci_ev_remote_ext_features *ev = (void *) skb->data;
+ struct hci_ev_remote_ext_features *ev = data;
struct hci_conn *conn;
- BT_DBG("%s", hdev->name);
+ bt_dev_dbg(hdev, "status 0x%2.2x", ev->status);
hci_dev_lock(hdev);
@@ -4440,14 +4655,13 @@ static void hci_remote_ext_features_evt(struct hci_dev *hdev,
hci_dev_unlock(hdev);
}
-static void hci_sync_conn_complete_evt(struct hci_dev *hdev,
+static void hci_sync_conn_complete_evt(struct hci_dev *hdev, void *data,
struct sk_buff *skb)
{
- struct hci_ev_sync_conn_complete *ev = (void *) skb->data;
+ struct hci_ev_sync_conn_complete *ev = data;
struct hci_conn *conn;
- unsigned int notify_evt;
- BT_DBG("%s status 0x%2.2x", hdev->name, ev->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", ev->status);
hci_dev_lock(hdev);
@@ -4517,22 +4731,18 @@ static void hci_sync_conn_complete_evt(struct hci_dev *hdev,
}
bt_dev_dbg(hdev, "SCO connected with air mode: %02x", ev->air_mode);
-
- switch (ev->air_mode) {
- case 0x02:
- notify_evt = HCI_NOTIFY_ENABLE_SCO_CVSD;
- break;
- case 0x03:
- notify_evt = HCI_NOTIFY_ENABLE_SCO_TRANSP;
- break;
- }
-
/* Notify only in case of SCO over HCI transport data path which
* is zero and non-zero value shall be non-HCI transport data path
*/
- if (conn->codec.data_path == 0) {
- if (hdev->notify)
- hdev->notify(hdev, notify_evt);
+ if (conn->codec.data_path == 0 && hdev->notify) {
+ switch (ev->air_mode) {
+ case 0x02:
+ hdev->notify(hdev, HCI_NOTIFY_ENABLE_SCO_CVSD);
+ break;
+ case 0x03:
+ hdev->notify(hdev, HCI_NOTIFY_ENABLE_SCO_TRANSP);
+ break;
+ }
}
hci_connect_cfm(conn, ev->status);
@@ -4560,17 +4770,21 @@ static inline size_t eir_get_length(u8 *eir, size_t eir_len)
return eir_len;
}
-static void hci_extended_inquiry_result_evt(struct hci_dev *hdev,
+static void hci_extended_inquiry_result_evt(struct hci_dev *hdev, void *edata,
struct sk_buff *skb)
{
+ struct hci_ev_ext_inquiry_result *ev = edata;
struct inquiry_data data;
- struct extended_inquiry_info *info = (void *) (skb->data + 1);
- int num_rsp = *((__u8 *) skb->data);
size_t eir_len;
+ int i;
- BT_DBG("%s num_rsp %d", hdev->name, num_rsp);
+ if (!hci_ev_skb_pull(hdev, skb, HCI_EV_EXTENDED_INQUIRY_RESULT,
+ flex_array_size(ev, info, ev->num)))
+ return;
- if (!num_rsp || skb->len < num_rsp * sizeof(*info) + 1)
+ bt_dev_dbg(hdev, "num %d", ev->num);
+
+ if (!ev->num)
return;
if (hci_dev_test_flag(hdev, HCI_PERIODIC_INQ))
@@ -4578,7 +4792,8 @@ static void hci_extended_inquiry_result_evt(struct hci_dev *hdev,
hci_dev_lock(hdev);
- for (; num_rsp; num_rsp--, info++) {
+ for (i = 0; i < ev->num; i++) {
+ struct extended_inquiry_info *info = &ev->info[i];
u32 flags;
bool name_known;
@@ -4610,14 +4825,14 @@ static void hci_extended_inquiry_result_evt(struct hci_dev *hdev,
hci_dev_unlock(hdev);
}
-static void hci_key_refresh_complete_evt(struct hci_dev *hdev,
+static void hci_key_refresh_complete_evt(struct hci_dev *hdev, void *data,
struct sk_buff *skb)
{
- struct hci_ev_key_refresh_complete *ev = (void *) skb->data;
+ struct hci_ev_key_refresh_complete *ev = data;
struct hci_conn *conn;
- BT_DBG("%s status 0x%2.2x handle 0x%4.4x", hdev->name, ev->status,
- __le16_to_cpu(ev->handle));
+ bt_dev_dbg(hdev, "status 0x%2.2x handle 0x%4.4x", ev->status,
+ __le16_to_cpu(ev->handle));
hci_dev_lock(hdev);
@@ -4720,12 +4935,13 @@ static u8 bredr_oob_data_present(struct hci_conn *conn)
return 0x01;
}
-static void hci_io_capa_request_evt(struct hci_dev *hdev, struct sk_buff *skb)
+static void hci_io_capa_request_evt(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_ev_io_capa_request *ev = (void *) skb->data;
+ struct hci_ev_io_capa_request *ev = data;
struct hci_conn *conn;
- BT_DBG("%s", hdev->name);
+ bt_dev_dbg(hdev, "");
hci_dev_lock(hdev);
@@ -4789,12 +5005,13 @@ static void hci_io_capa_request_evt(struct hci_dev *hdev, struct sk_buff *skb)
hci_dev_unlock(hdev);
}
-static void hci_io_capa_reply_evt(struct hci_dev *hdev, struct sk_buff *skb)
+static void hci_io_capa_reply_evt(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_ev_io_capa_reply *ev = (void *) skb->data;
+ struct hci_ev_io_capa_reply *ev = data;
struct hci_conn *conn;
- BT_DBG("%s", hdev->name);
+ bt_dev_dbg(hdev, "");
hci_dev_lock(hdev);
@@ -4809,14 +5026,14 @@ static void hci_io_capa_reply_evt(struct hci_dev *hdev, struct sk_buff *skb)
hci_dev_unlock(hdev);
}
-static void hci_user_confirm_request_evt(struct hci_dev *hdev,
+static void hci_user_confirm_request_evt(struct hci_dev *hdev, void *data,
struct sk_buff *skb)
{
- struct hci_ev_user_confirm_req *ev = (void *) skb->data;
+ struct hci_ev_user_confirm_req *ev = data;
int loc_mitm, rem_mitm, confirm_hint = 0;
struct hci_conn *conn;
- BT_DBG("%s", hdev->name);
+ bt_dev_dbg(hdev, "");
hci_dev_lock(hdev);
@@ -4837,7 +5054,7 @@ static void hci_user_confirm_request_evt(struct hci_dev *hdev,
*/
if (conn->pending_sec_level > BT_SECURITY_MEDIUM &&
conn->remote_cap == HCI_IO_NO_INPUT_OUTPUT) {
- BT_DBG("Rejecting request: remote device can't provide MITM");
+ bt_dev_dbg(hdev, "Rejecting request: remote device can't provide MITM");
hci_send_cmd(hdev, HCI_OP_USER_CONFIRM_NEG_REPLY,
sizeof(ev->bdaddr), &ev->bdaddr);
goto unlock;
@@ -4856,7 +5073,7 @@ static void hci_user_confirm_request_evt(struct hci_dev *hdev,
if (!test_bit(HCI_CONN_AUTH_PEND, &conn->flags) &&
conn->io_capability != HCI_IO_NO_INPUT_OUTPUT &&
(loc_mitm || rem_mitm)) {
- BT_DBG("Confirming auto-accept as acceptor");
+ bt_dev_dbg(hdev, "Confirming auto-accept as acceptor");
confirm_hint = 1;
goto confirm;
}
@@ -4894,24 +5111,24 @@ static void hci_user_confirm_request_evt(struct hci_dev *hdev,
hci_dev_unlock(hdev);
}
-static void hci_user_passkey_request_evt(struct hci_dev *hdev,
+static void hci_user_passkey_request_evt(struct hci_dev *hdev, void *data,
struct sk_buff *skb)
{
- struct hci_ev_user_passkey_req *ev = (void *) skb->data;
+ struct hci_ev_user_passkey_req *ev = data;
- BT_DBG("%s", hdev->name);
+ bt_dev_dbg(hdev, "");
if (hci_dev_test_flag(hdev, HCI_MGMT))
mgmt_user_passkey_request(hdev, &ev->bdaddr, ACL_LINK, 0);
}
-static void hci_user_passkey_notify_evt(struct hci_dev *hdev,
+static void hci_user_passkey_notify_evt(struct hci_dev *hdev, void *data,
struct sk_buff *skb)
{
- struct hci_ev_user_passkey_notify *ev = (void *) skb->data;
+ struct hci_ev_user_passkey_notify *ev = data;
struct hci_conn *conn;
- BT_DBG("%s", hdev->name);
+ bt_dev_dbg(hdev, "");
conn = hci_conn_hash_lookup_ba(hdev, ACL_LINK, &ev->bdaddr);
if (!conn)
@@ -4926,12 +5143,13 @@ static void hci_user_passkey_notify_evt(struct hci_dev *hdev,
conn->passkey_entered);
}
-static void hci_keypress_notify_evt(struct hci_dev *hdev, struct sk_buff *skb)
+static void hci_keypress_notify_evt(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_ev_keypress_notify *ev = (void *) skb->data;
+ struct hci_ev_keypress_notify *ev = data;
struct hci_conn *conn;
- BT_DBG("%s", hdev->name);
+ bt_dev_dbg(hdev, "");
conn = hci_conn_hash_lookup_ba(hdev, ACL_LINK, &ev->bdaddr);
if (!conn)
@@ -4964,13 +5182,13 @@ static void hci_keypress_notify_evt(struct hci_dev *hdev, struct sk_buff *skb)
conn->passkey_entered);
}
-static void hci_simple_pair_complete_evt(struct hci_dev *hdev,
+static void hci_simple_pair_complete_evt(struct hci_dev *hdev, void *data,
struct sk_buff *skb)
{
- struct hci_ev_simple_pair_complete *ev = (void *) skb->data;
+ struct hci_ev_simple_pair_complete *ev = data;
struct hci_conn *conn;
- BT_DBG("%s", hdev->name);
+ bt_dev_dbg(hdev, "");
hci_dev_lock(hdev);
@@ -4995,14 +5213,14 @@ static void hci_simple_pair_complete_evt(struct hci_dev *hdev,
hci_dev_unlock(hdev);
}
-static void hci_remote_host_features_evt(struct hci_dev *hdev,
+static void hci_remote_host_features_evt(struct hci_dev *hdev, void *data,
struct sk_buff *skb)
{
- struct hci_ev_remote_host_features *ev = (void *) skb->data;
+ struct hci_ev_remote_host_features *ev = data;
struct inquiry_entry *ie;
struct hci_conn *conn;
- BT_DBG("%s", hdev->name);
+ bt_dev_dbg(hdev, "");
hci_dev_lock(hdev);
@@ -5017,13 +5235,13 @@ static void hci_remote_host_features_evt(struct hci_dev *hdev,
hci_dev_unlock(hdev);
}
-static void hci_remote_oob_data_request_evt(struct hci_dev *hdev,
+static void hci_remote_oob_data_request_evt(struct hci_dev *hdev, void *edata,
struct sk_buff *skb)
{
- struct hci_ev_remote_oob_data_request *ev = (void *) skb->data;
+ struct hci_ev_remote_oob_data_request *ev = edata;
struct oob_data *data;
- BT_DBG("%s", hdev->name);
+ bt_dev_dbg(hdev, "");
hci_dev_lock(hdev);
@@ -5072,14 +5290,13 @@ static void hci_remote_oob_data_request_evt(struct hci_dev *hdev,
}
#if IS_ENABLED(CONFIG_BT_HS)
-static void hci_chan_selected_evt(struct hci_dev *hdev, struct sk_buff *skb)
+static void hci_chan_selected_evt(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_ev_channel_selected *ev = (void *)skb->data;
+ struct hci_ev_channel_selected *ev = data;
struct hci_conn *hcon;
- BT_DBG("%s handle 0x%2.2x", hdev->name, ev->phy_handle);
-
- skb_pull(skb, sizeof(*ev));
+ bt_dev_dbg(hdev, "handle 0x%2.2x", ev->phy_handle);
hcon = hci_conn_hash_lookup_handle(hdev, ev->phy_handle);
if (!hcon)
@@ -5088,14 +5305,14 @@ static void hci_chan_selected_evt(struct hci_dev *hdev, struct sk_buff *skb)
amp_read_loc_assoc_final_data(hdev, hcon);
}
-static void hci_phy_link_complete_evt(struct hci_dev *hdev,
+static void hci_phy_link_complete_evt(struct hci_dev *hdev, void *data,
struct sk_buff *skb)
{
- struct hci_ev_phy_link_complete *ev = (void *) skb->data;
+ struct hci_ev_phy_link_complete *ev = data;
struct hci_conn *hcon, *bredr_hcon;
- BT_DBG("%s handle 0x%2.2x status 0x%2.2x", hdev->name, ev->phy_handle,
- ev->status);
+ bt_dev_dbg(hdev, "handle 0x%2.2x status 0x%2.2x", ev->phy_handle,
+ ev->status);
hci_dev_lock(hdev);
@@ -5129,16 +5346,16 @@ static void hci_phy_link_complete_evt(struct hci_dev *hdev,
hci_dev_unlock(hdev);
}
-static void hci_loglink_complete_evt(struct hci_dev *hdev, struct sk_buff *skb)
+static void hci_loglink_complete_evt(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_ev_logical_link_complete *ev = (void *) skb->data;
+ struct hci_ev_logical_link_complete *ev = data;
struct hci_conn *hcon;
struct hci_chan *hchan;
struct amp_mgr *mgr;
- BT_DBG("%s log_handle 0x%4.4x phy_handle 0x%2.2x status 0x%2.2x",
- hdev->name, le16_to_cpu(ev->handle), ev->phy_handle,
- ev->status);
+ bt_dev_dbg(hdev, "log_handle 0x%4.4x phy_handle 0x%2.2x status 0x%2.2x",
+ le16_to_cpu(ev->handle), ev->phy_handle, ev->status);
hcon = hci_conn_hash_lookup_handle(hdev, ev->phy_handle);
if (!hcon)
@@ -5168,14 +5385,14 @@ static void hci_loglink_complete_evt(struct hci_dev *hdev, struct sk_buff *skb)
}
}
-static void hci_disconn_loglink_complete_evt(struct hci_dev *hdev,
+static void hci_disconn_loglink_complete_evt(struct hci_dev *hdev, void *data,
struct sk_buff *skb)
{
- struct hci_ev_disconn_logical_link_complete *ev = (void *) skb->data;
+ struct hci_ev_disconn_logical_link_complete *ev = data;
struct hci_chan *hchan;
- BT_DBG("%s log handle 0x%4.4x status 0x%2.2x", hdev->name,
- le16_to_cpu(ev->handle), ev->status);
+ bt_dev_dbg(hdev, "handle 0x%4.4x status 0x%2.2x",
+ le16_to_cpu(ev->handle), ev->status);
if (ev->status)
return;
@@ -5192,13 +5409,13 @@ static void hci_disconn_loglink_complete_evt(struct hci_dev *hdev,
hci_dev_unlock(hdev);
}
-static void hci_disconn_phylink_complete_evt(struct hci_dev *hdev,
+static void hci_disconn_phylink_complete_evt(struct hci_dev *hdev, void *data,
struct sk_buff *skb)
{
- struct hci_ev_disconn_phy_link_complete *ev = (void *) skb->data;
+ struct hci_ev_disconn_phy_link_complete *ev = data;
struct hci_conn *hcon;
- BT_DBG("%s status 0x%2.2x", hdev->name, ev->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", ev->status);
if (ev->status)
return;
@@ -5412,15 +5629,16 @@ static void le_conn_complete_evt(struct hci_dev *hdev, u8 status,
}
unlock:
- hci_update_background_scan(hdev);
+ hci_update_passive_scan(hdev);
hci_dev_unlock(hdev);
}
-static void hci_le_conn_complete_evt(struct hci_dev *hdev, struct sk_buff *skb)
+static void hci_le_conn_complete_evt(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_ev_le_conn_complete *ev = (void *) skb->data;
+ struct hci_ev_le_conn_complete *ev = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, ev->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", ev->status);
le_conn_complete_evt(hdev, ev->status, &ev->bdaddr, ev->bdaddr_type,
NULL, ev->role, le16_to_cpu(ev->handle),
@@ -5429,35 +5647,43 @@ static void hci_le_conn_complete_evt(struct hci_dev *hdev, struct sk_buff *skb)
le16_to_cpu(ev->supervision_timeout));
}
-static void hci_le_enh_conn_complete_evt(struct hci_dev *hdev,
+static void hci_le_enh_conn_complete_evt(struct hci_dev *hdev, void *data,
struct sk_buff *skb)
{
- struct hci_ev_le_enh_conn_complete *ev = (void *) skb->data;
+ struct hci_ev_le_enh_conn_complete *ev = data;
- BT_DBG("%s status 0x%2.2x", hdev->name, ev->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", ev->status);
le_conn_complete_evt(hdev, ev->status, &ev->bdaddr, ev->bdaddr_type,
&ev->local_rpa, ev->role, le16_to_cpu(ev->handle),
le16_to_cpu(ev->interval),
le16_to_cpu(ev->latency),
le16_to_cpu(ev->supervision_timeout));
-
- if (use_ll_privacy(hdev) &&
- hci_dev_test_flag(hdev, HCI_ENABLE_LL_PRIVACY) &&
- hci_dev_test_flag(hdev, HCI_LL_RPA_RESOLUTION))
- hci_req_disable_address_resolution(hdev);
}
-static void hci_le_ext_adv_term_evt(struct hci_dev *hdev, struct sk_buff *skb)
+static void hci_le_ext_adv_term_evt(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_evt_le_ext_adv_set_term *ev = (void *) skb->data;
+ struct hci_evt_le_ext_adv_set_term *ev = data;
struct hci_conn *conn;
- struct adv_info *adv;
+ struct adv_info *adv, *n;
- BT_DBG("%s status 0x%2.2x", hdev->name, ev->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", ev->status);
adv = hci_find_adv_instance(hdev, ev->handle);
+ /* The Bluetooth Core 5.3 specification clearly states that this event
+ * shall not be sent when the Host disables the advertising set. So in
+ * case of HCI_ERROR_CANCELLED_BY_HOST, just ignore the event.
+ *
+ * When the Host disables an advertising set, all cleanup is done via
+ * its command callback and not needed to be duplicated here.
+ */
+ if (ev->status == HCI_ERROR_CANCELLED_BY_HOST) {
+ bt_dev_warn_ratelimited(hdev, "Unexpected advertising set terminated event");
+ return;
+ }
+
if (ev->status) {
if (!adv)
return;
@@ -5466,6 +5692,13 @@ static void hci_le_ext_adv_term_evt(struct hci_dev *hdev, struct sk_buff *skb)
hci_remove_adv_instance(hdev, ev->handle);
mgmt_advertising_removed(NULL, hdev, ev->handle);
+ list_for_each_entry_safe(adv, n, &hdev->adv_instances, list) {
+ if (adv->enabled)
+ return;
+ }
+
+ /* We are no longer advertising, clear HCI_LE_ADV */
+ hci_dev_clear_flag(hdev, HCI_LE_ADV);
return;
}
@@ -5493,13 +5726,13 @@ static void hci_le_ext_adv_term_evt(struct hci_dev *hdev, struct sk_buff *skb)
}
}
-static void hci_le_conn_update_complete_evt(struct hci_dev *hdev,
+static void hci_le_conn_update_complete_evt(struct hci_dev *hdev, void *data,
struct sk_buff *skb)
{
- struct hci_ev_le_conn_update_complete *ev = (void *) skb->data;
+ struct hci_ev_le_conn_update_complete *ev = data;
struct hci_conn *conn;
- BT_DBG("%s status 0x%2.2x", hdev->name, ev->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", ev->status);
if (ev->status)
return;
@@ -5520,7 +5753,7 @@ static void hci_le_conn_update_complete_evt(struct hci_dev *hdev,
static struct hci_conn *check_pending_le_conn(struct hci_dev *hdev,
bdaddr_t *addr,
u8 addr_type, bool addr_resolved,
- u8 adv_type, bdaddr_t *direct_rpa)
+ u8 adv_type)
{
struct hci_conn *conn;
struct hci_conn_params *params;
@@ -5529,8 +5762,9 @@ static struct hci_conn *check_pending_le_conn(struct hci_dev *hdev,
if (adv_type != LE_ADV_IND && adv_type != LE_ADV_DIRECT_IND)
return NULL;
- /* Ignore if the device is blocked */
- if (hci_bdaddr_list_lookup(&hdev->reject_list, addr, addr_type))
+ /* Ignore if the device is blocked or hdev is suspended */
+ if (hci_bdaddr_list_lookup(&hdev->reject_list, addr, addr_type) ||
+ hdev->suspended)
return NULL;
/* Most controller will fail if we try to create new connections
@@ -5574,7 +5808,7 @@ static struct hci_conn *check_pending_le_conn(struct hci_dev *hdev,
conn = hci_connect_le(hdev, addr, addr_type, addr_resolved,
BT_SECURITY_LOW, hdev->def_le_autoconnect_timeout,
- HCI_ROLE_MASTER, direct_rpa);
+ HCI_ROLE_MASTER);
if (!IS_ERR(conn)) {
/* If HCI_AUTO_CONN_EXPLICIT is set, conn is already owned
* by higher layer that tried to connect, if no then
@@ -5697,7 +5931,7 @@ static void process_adv_report(struct hci_dev *hdev, u8 type, bdaddr_t *bdaddr,
* for advertising reports) and is already verified to be RPA above.
*/
conn = check_pending_le_conn(hdev, bdaddr, bdaddr_type, bdaddr_resolved,
- type, direct_addr);
+ type);
if (!ext_adv && conn && type == LE_ADV_IND && len <= HCI_MAX_AD_LENGTH) {
/* Store report for later inclusion by
* mgmt_device_connected
@@ -5814,27 +6048,38 @@ static void process_adv_report(struct hci_dev *hdev, u8 type, bdaddr_t *bdaddr,
clear_pending_adv_report(hdev);
}
-static void hci_le_adv_report_evt(struct hci_dev *hdev, struct sk_buff *skb)
+static void hci_le_adv_report_evt(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- u8 num_reports = skb->data[0];
- void *ptr = &skb->data[1];
+ struct hci_ev_le_advertising_report *ev = data;
+
+ if (!ev->num)
+ return;
hci_dev_lock(hdev);
- while (num_reports--) {
- struct hci_ev_le_advertising_info *ev = ptr;
+ while (ev->num--) {
+ struct hci_ev_le_advertising_info *info;
s8 rssi;
- if (ev->length <= HCI_MAX_AD_LENGTH) {
- rssi = ev->data[ev->length];
- process_adv_report(hdev, ev->evt_type, &ev->bdaddr,
- ev->bdaddr_type, NULL, 0, rssi,
- ev->data, ev->length, false);
+ info = hci_le_ev_skb_pull(hdev, skb,
+ HCI_EV_LE_ADVERTISING_REPORT,
+ sizeof(*info));
+ if (!info)
+ break;
+
+ if (!hci_le_ev_skb_pull(hdev, skb, HCI_EV_LE_ADVERTISING_REPORT,
+ info->length + 1))
+ break;
+
+ if (info->length <= HCI_MAX_AD_LENGTH) {
+ rssi = info->data[info->length];
+ process_adv_report(hdev, info->type, &info->bdaddr,
+ info->bdaddr_type, NULL, 0, rssi,
+ info->data, info->length, false);
} else {
bt_dev_err(hdev, "Dropping invalid advertising data");
}
-
- ptr += sizeof(*ev) + ev->length + 1;
}
hci_dev_unlock(hdev);
@@ -5884,40 +6129,50 @@ static u8 ext_evt_type_to_legacy(struct hci_dev *hdev, u16 evt_type)
return LE_ADV_INVALID;
}
-static void hci_le_ext_adv_report_evt(struct hci_dev *hdev, struct sk_buff *skb)
+static void hci_le_ext_adv_report_evt(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- u8 num_reports = skb->data[0];
- void *ptr = &skb->data[1];
+ struct hci_ev_le_ext_adv_report *ev = data;
+
+ if (!ev->num)
+ return;
hci_dev_lock(hdev);
- while (num_reports--) {
- struct hci_ev_le_ext_adv_report *ev = ptr;
+ while (ev->num--) {
+ struct hci_ev_le_ext_adv_info *info;
u8 legacy_evt_type;
u16 evt_type;
- evt_type = __le16_to_cpu(ev->evt_type);
+ info = hci_le_ev_skb_pull(hdev, skb, HCI_EV_LE_EXT_ADV_REPORT,
+ sizeof(*info));
+ if (!info)
+ break;
+
+ if (!hci_le_ev_skb_pull(hdev, skb, HCI_EV_LE_EXT_ADV_REPORT,
+ info->length))
+ break;
+
+ evt_type = __le16_to_cpu(info->type);
legacy_evt_type = ext_evt_type_to_legacy(hdev, evt_type);
if (legacy_evt_type != LE_ADV_INVALID) {
- process_adv_report(hdev, legacy_evt_type, &ev->bdaddr,
- ev->bdaddr_type, NULL, 0, ev->rssi,
- ev->data, ev->length,
+ process_adv_report(hdev, legacy_evt_type, &info->bdaddr,
+ info->bdaddr_type, NULL, 0,
+ info->rssi, info->data, info->length,
!(evt_type & LE_EXT_ADV_LEGACY_PDU));
}
-
- ptr += sizeof(*ev) + ev->length;
}
hci_dev_unlock(hdev);
}
-static void hci_le_remote_feat_complete_evt(struct hci_dev *hdev,
+static void hci_le_remote_feat_complete_evt(struct hci_dev *hdev, void *data,
struct sk_buff *skb)
{
- struct hci_ev_le_remote_feat_complete *ev = (void *)skb->data;
+ struct hci_ev_le_remote_feat_complete *ev = data;
struct hci_conn *conn;
- BT_DBG("%s status 0x%2.2x", hdev->name, ev->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", ev->status);
hci_dev_lock(hdev);
@@ -5953,15 +6208,16 @@ static void hci_le_remote_feat_complete_evt(struct hci_dev *hdev,
hci_dev_unlock(hdev);
}
-static void hci_le_ltk_request_evt(struct hci_dev *hdev, struct sk_buff *skb)
+static void hci_le_ltk_request_evt(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_ev_le_ltk_req *ev = (void *) skb->data;
+ struct hci_ev_le_ltk_req *ev = data;
struct hci_cp_le_ltk_reply cp;
struct hci_cp_le_ltk_neg_reply neg;
struct hci_conn *conn;
struct smp_ltk *ltk;
- BT_DBG("%s handle 0x%4.4x", hdev->name, __le16_to_cpu(ev->handle));
+ bt_dev_dbg(hdev, "handle 0x%4.4x", __le16_to_cpu(ev->handle));
hci_dev_lock(hdev);
@@ -6029,14 +6285,16 @@ static void send_conn_param_neg_reply(struct hci_dev *hdev, u16 handle,
&cp);
}
-static void hci_le_remote_conn_param_req_evt(struct hci_dev *hdev,
+static void hci_le_remote_conn_param_req_evt(struct hci_dev *hdev, void *data,
struct sk_buff *skb)
{
- struct hci_ev_le_remote_conn_param_req *ev = (void *) skb->data;
+ struct hci_ev_le_remote_conn_param_req *ev = data;
struct hci_cp_le_conn_param_req_reply cp;
struct hci_conn *hcon;
u16 handle, min, max, latency, timeout;
+ bt_dev_dbg(hdev, "handle 0x%4.4x", __le16_to_cpu(ev->handle));
+
handle = le16_to_cpu(ev->handle);
min = le16_to_cpu(ev->interval_min);
max = le16_to_cpu(ev->interval_max);
@@ -6087,32 +6345,40 @@ static void hci_le_remote_conn_param_req_evt(struct hci_dev *hdev,
hci_send_cmd(hdev, HCI_OP_LE_CONN_PARAM_REQ_REPLY, sizeof(cp), &cp);
}
-static void hci_le_direct_adv_report_evt(struct hci_dev *hdev,
+static void hci_le_direct_adv_report_evt(struct hci_dev *hdev, void *data,
struct sk_buff *skb)
{
- u8 num_reports = skb->data[0];
- struct hci_ev_le_direct_adv_info *ev = (void *)&skb->data[1];
+ struct hci_ev_le_direct_adv_report *ev = data;
+ int i;
- if (!num_reports || skb->len < num_reports * sizeof(*ev) + 1)
+ if (!hci_le_ev_skb_pull(hdev, skb, HCI_EV_LE_DIRECT_ADV_REPORT,
+ flex_array_size(ev, info, ev->num)))
+ return;
+
+ if (!ev->num)
return;
hci_dev_lock(hdev);
- for (; num_reports; num_reports--, ev++)
- process_adv_report(hdev, ev->evt_type, &ev->bdaddr,
- ev->bdaddr_type, &ev->direct_addr,
- ev->direct_addr_type, ev->rssi, NULL, 0,
+ for (i = 0; i < ev->num; i++) {
+ struct hci_ev_le_direct_adv_info *info = &ev->info[i];
+
+ process_adv_report(hdev, info->type, &info->bdaddr,
+ info->bdaddr_type, &info->direct_addr,
+ info->direct_addr_type, info->rssi, NULL, 0,
false);
+ }
hci_dev_unlock(hdev);
}
-static void hci_le_phy_update_evt(struct hci_dev *hdev, struct sk_buff *skb)
+static void hci_le_phy_update_evt(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb)
{
- struct hci_ev_le_phy_update_complete *ev = (void *) skb->data;
+ struct hci_ev_le_phy_update_complete *ev = data;
struct hci_conn *conn;
- BT_DBG("%s status 0x%2.2x", hdev->name, ev->status);
+ bt_dev_dbg(hdev, "status 0x%2.2x", ev->status);
if (ev->status)
return;
@@ -6130,60 +6396,113 @@ static void hci_le_phy_update_evt(struct hci_dev *hdev, struct sk_buff *skb)
hci_dev_unlock(hdev);
}
-static void hci_le_meta_evt(struct hci_dev *hdev, struct sk_buff *skb)
+#define HCI_LE_EV_VL(_op, _func, _min_len, _max_len) \
+[_op] = { \
+ .func = _func, \
+ .min_len = _min_len, \
+ .max_len = _max_len, \
+}
+
+#define HCI_LE_EV(_op, _func, _len) \
+ HCI_LE_EV_VL(_op, _func, _len, _len)
+
+#define HCI_LE_EV_STATUS(_op, _func) \
+ HCI_LE_EV(_op, _func, sizeof(struct hci_ev_status))
+
+/* Entries in this table shall have their position according to the subevent
+ * opcode they handle so the use of the macros above is recommend since it does
+ * attempt to initialize at its proper index using Designated Initializers that
+ * way events without a callback function can be ommited.
+ */
+static const struct hci_le_ev {
+ void (*func)(struct hci_dev *hdev, void *data, struct sk_buff *skb);
+ u16 min_len;
+ u16 max_len;
+} hci_le_ev_table[U8_MAX + 1] = {
+ /* [0x01 = HCI_EV_LE_CONN_COMPLETE] */
+ HCI_LE_EV(HCI_EV_LE_CONN_COMPLETE, hci_le_conn_complete_evt,
+ sizeof(struct hci_ev_le_conn_complete)),
+ /* [0x02 = HCI_EV_LE_ADVERTISING_REPORT] */
+ HCI_LE_EV_VL(HCI_EV_LE_ADVERTISING_REPORT, hci_le_adv_report_evt,
+ sizeof(struct hci_ev_le_advertising_report),
+ HCI_MAX_EVENT_SIZE),
+ /* [0x03 = HCI_EV_LE_CONN_UPDATE_COMPLETE] */
+ HCI_LE_EV(HCI_EV_LE_CONN_UPDATE_COMPLETE,
+ hci_le_conn_update_complete_evt,
+ sizeof(struct hci_ev_le_conn_update_complete)),
+ /* [0x04 = HCI_EV_LE_REMOTE_FEAT_COMPLETE] */
+ HCI_LE_EV(HCI_EV_LE_REMOTE_FEAT_COMPLETE,
+ hci_le_remote_feat_complete_evt,
+ sizeof(struct hci_ev_le_remote_feat_complete)),
+ /* [0x05 = HCI_EV_LE_LTK_REQ] */
+ HCI_LE_EV(HCI_EV_LE_LTK_REQ, hci_le_ltk_request_evt,
+ sizeof(struct hci_ev_le_ltk_req)),
+ /* [0x06 = HCI_EV_LE_REMOTE_CONN_PARAM_REQ] */
+ HCI_LE_EV(HCI_EV_LE_REMOTE_CONN_PARAM_REQ,
+ hci_le_remote_conn_param_req_evt,
+ sizeof(struct hci_ev_le_remote_conn_param_req)),
+ /* [0x0a = HCI_EV_LE_ENHANCED_CONN_COMPLETE] */
+ HCI_LE_EV(HCI_EV_LE_ENHANCED_CONN_COMPLETE,
+ hci_le_enh_conn_complete_evt,
+ sizeof(struct hci_ev_le_enh_conn_complete)),
+ /* [0x0b = HCI_EV_LE_DIRECT_ADV_REPORT] */
+ HCI_LE_EV_VL(HCI_EV_LE_DIRECT_ADV_REPORT, hci_le_direct_adv_report_evt,
+ sizeof(struct hci_ev_le_direct_adv_report),
+ HCI_MAX_EVENT_SIZE),
+ /* [0x0c = HCI_EV_LE_PHY_UPDATE_COMPLETE] */
+ HCI_LE_EV(HCI_EV_LE_PHY_UPDATE_COMPLETE, hci_le_phy_update_evt,
+ sizeof(struct hci_ev_le_phy_update_complete)),
+ /* [0x0d = HCI_EV_LE_EXT_ADV_REPORT] */
+ HCI_LE_EV_VL(HCI_EV_LE_EXT_ADV_REPORT, hci_le_ext_adv_report_evt,
+ sizeof(struct hci_ev_le_ext_adv_report),
+ HCI_MAX_EVENT_SIZE),
+ /* [0x12 = HCI_EV_LE_EXT_ADV_SET_TERM] */
+ HCI_LE_EV(HCI_EV_LE_EXT_ADV_SET_TERM, hci_le_ext_adv_term_evt,
+ sizeof(struct hci_evt_le_ext_adv_set_term)),
+};
+
+static void hci_le_meta_evt(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb, u16 *opcode, u8 *status,
+ hci_req_complete_t *req_complete,
+ hci_req_complete_skb_t *req_complete_skb)
{
- struct hci_ev_le_meta *le_ev = (void *) skb->data;
+ struct hci_ev_le_meta *ev = data;
+ const struct hci_le_ev *subev;
- skb_pull(skb, sizeof(*le_ev));
+ bt_dev_dbg(hdev, "subevent 0x%2.2x", ev->subevent);
- switch (le_ev->subevent) {
- case HCI_EV_LE_CONN_COMPLETE:
- hci_le_conn_complete_evt(hdev, skb);
- break;
-
- case HCI_EV_LE_CONN_UPDATE_COMPLETE:
- hci_le_conn_update_complete_evt(hdev, skb);
- break;
-
- case HCI_EV_LE_ADVERTISING_REPORT:
- hci_le_adv_report_evt(hdev, skb);
- break;
-
- case HCI_EV_LE_REMOTE_FEAT_COMPLETE:
- hci_le_remote_feat_complete_evt(hdev, skb);
- break;
-
- case HCI_EV_LE_LTK_REQ:
- hci_le_ltk_request_evt(hdev, skb);
- break;
-
- case HCI_EV_LE_REMOTE_CONN_PARAM_REQ:
- hci_le_remote_conn_param_req_evt(hdev, skb);
- break;
-
- case HCI_EV_LE_DIRECT_ADV_REPORT:
- hci_le_direct_adv_report_evt(hdev, skb);
- break;
-
- case HCI_EV_LE_PHY_UPDATE_COMPLETE:
- hci_le_phy_update_evt(hdev, skb);
- break;
-
- case HCI_EV_LE_EXT_ADV_REPORT:
- hci_le_ext_adv_report_evt(hdev, skb);
- break;
-
- case HCI_EV_LE_ENHANCED_CONN_COMPLETE:
- hci_le_enh_conn_complete_evt(hdev, skb);
- break;
-
- case HCI_EV_LE_EXT_ADV_SET_TERM:
- hci_le_ext_adv_term_evt(hdev, skb);
- break;
-
- default:
- break;
+ /* Only match event if command OGF is for LE */
+ if (hdev->sent_cmd &&
+ hci_opcode_ogf(hci_skb_opcode(hdev->sent_cmd)) == 0x08 &&
+ hci_skb_event(hdev->sent_cmd) == ev->subevent) {
+ *opcode = hci_skb_opcode(hdev->sent_cmd);
+ hci_req_cmd_complete(hdev, *opcode, 0x00, req_complete,
+ req_complete_skb);
}
+
+ subev = &hci_le_ev_table[ev->subevent];
+ if (!subev->func)
+ return;
+
+ if (skb->len < subev->min_len) {
+ bt_dev_err(hdev, "unexpected subevent 0x%2.2x length: %u < %u",
+ ev->subevent, skb->len, subev->min_len);
+ return;
+ }
+
+ /* Just warn if the length is over max_len size it still be
+ * possible to partially parse the event so leave to callback to
+ * decide if that is acceptable.
+ */
+ if (skb->len > subev->max_len)
+ bt_dev_warn(hdev, "unexpected subevent 0x%2.2x length: %u > %u",
+ ev->subevent, skb->len, subev->max_len);
+
+ data = hci_le_ev_skb_pull(hdev, skb, ev->subevent, subev->min_len);
+ if (!data)
+ return;
+
+ subev->func(hdev, data, skb);
}
static bool hci_get_cmd_complete(struct hci_dev *hdev, u16 opcode,
@@ -6195,13 +6514,9 @@ static bool hci_get_cmd_complete(struct hci_dev *hdev, u16 opcode,
if (!skb)
return false;
- if (skb->len < sizeof(*hdr)) {
- bt_dev_err(hdev, "too short HCI event");
+ hdr = hci_ev_skb_pull(hdev, skb, event, sizeof(*hdr));
+ if (!hdr)
return false;
- }
-
- hdr = (void *) skb->data;
- skb_pull(skb, HCI_EVENT_HDR_SIZE);
if (event) {
if (hdr->evt != event)
@@ -6221,13 +6536,9 @@ static bool hci_get_cmd_complete(struct hci_dev *hdev, u16 opcode,
return false;
}
- if (skb->len < sizeof(*ev)) {
- bt_dev_err(hdev, "too short cmd_complete event");
+ ev = hci_cc_skb_pull(hdev, skb, opcode, sizeof(*ev));
+ if (!ev)
return false;
- }
-
- ev = (void *) skb->data;
- skb_pull(skb, sizeof(*ev));
if (opcode != __le16_to_cpu(ev->opcode)) {
BT_DBG("opcode doesn't match (0x%2.2x != 0x%2.2x)", opcode,
@@ -6243,7 +6554,7 @@ static void hci_store_wake_reason(struct hci_dev *hdev, u8 event,
{
struct hci_ev_le_advertising_info *adv;
struct hci_ev_le_direct_adv_info *direct_adv;
- struct hci_ev_le_ext_adv_report *ext_adv;
+ struct hci_ev_le_ext_adv_info *ext_adv;
const struct hci_ev_conn_complete *conn_complete = (void *)skb->data;
const struct hci_ev_conn_request *conn_request = (void *)skb->data;
@@ -6307,25 +6618,252 @@ static void hci_store_wake_reason(struct hci_dev *hdev, u8 event,
hci_dev_unlock(hdev);
}
+#define HCI_EV_VL(_op, _func, _min_len, _max_len) \
+[_op] = { \
+ .req = false, \
+ .func = _func, \
+ .min_len = _min_len, \
+ .max_len = _max_len, \
+}
+
+#define HCI_EV(_op, _func, _len) \
+ HCI_EV_VL(_op, _func, _len, _len)
+
+#define HCI_EV_STATUS(_op, _func) \
+ HCI_EV(_op, _func, sizeof(struct hci_ev_status))
+
+#define HCI_EV_REQ_VL(_op, _func, _min_len, _max_len) \
+[_op] = { \
+ .req = true, \
+ .func_req = _func, \
+ .min_len = _min_len, \
+ .max_len = _max_len, \
+}
+
+#define HCI_EV_REQ(_op, _func, _len) \
+ HCI_EV_REQ_VL(_op, _func, _len, _len)
+
+/* Entries in this table shall have their position according to the event opcode
+ * they handle so the use of the macros above is recommend since it does attempt
+ * to initialize at its proper index using Designated Initializers that way
+ * events without a callback function don't have entered.
+ */
+static const struct hci_ev {
+ bool req;
+ union {
+ void (*func)(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb);
+ void (*func_req)(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb, u16 *opcode, u8 *status,
+ hci_req_complete_t *req_complete,
+ hci_req_complete_skb_t *req_complete_skb);
+ };
+ u16 min_len;
+ u16 max_len;
+} hci_ev_table[U8_MAX + 1] = {
+ /* [0x01 = HCI_EV_INQUIRY_COMPLETE] */
+ HCI_EV_STATUS(HCI_EV_INQUIRY_COMPLETE, hci_inquiry_complete_evt),
+ /* [0x02 = HCI_EV_INQUIRY_RESULT] */
+ HCI_EV_VL(HCI_EV_INQUIRY_RESULT, hci_inquiry_result_evt,
+ sizeof(struct hci_ev_inquiry_result), HCI_MAX_EVENT_SIZE),
+ /* [0x03 = HCI_EV_CONN_COMPLETE] */
+ HCI_EV(HCI_EV_CONN_COMPLETE, hci_conn_complete_evt,
+ sizeof(struct hci_ev_conn_complete)),
+ /* [0x04 = HCI_EV_CONN_REQUEST] */
+ HCI_EV(HCI_EV_CONN_REQUEST, hci_conn_request_evt,
+ sizeof(struct hci_ev_conn_request)),
+ /* [0x05 = HCI_EV_DISCONN_COMPLETE] */
+ HCI_EV(HCI_EV_DISCONN_COMPLETE, hci_disconn_complete_evt,
+ sizeof(struct hci_ev_disconn_complete)),
+ /* [0x06 = HCI_EV_AUTH_COMPLETE] */
+ HCI_EV(HCI_EV_AUTH_COMPLETE, hci_auth_complete_evt,
+ sizeof(struct hci_ev_auth_complete)),
+ /* [0x07 = HCI_EV_REMOTE_NAME] */
+ HCI_EV(HCI_EV_REMOTE_NAME, hci_remote_name_evt,
+ sizeof(struct hci_ev_remote_name)),
+ /* [0x08 = HCI_EV_ENCRYPT_CHANGE] */
+ HCI_EV(HCI_EV_ENCRYPT_CHANGE, hci_encrypt_change_evt,
+ sizeof(struct hci_ev_encrypt_change)),
+ /* [0x09 = HCI_EV_CHANGE_LINK_KEY_COMPLETE] */
+ HCI_EV(HCI_EV_CHANGE_LINK_KEY_COMPLETE,
+ hci_change_link_key_complete_evt,
+ sizeof(struct hci_ev_change_link_key_complete)),
+ /* [0x0b = HCI_EV_REMOTE_FEATURES] */
+ HCI_EV(HCI_EV_REMOTE_FEATURES, hci_remote_features_evt,
+ sizeof(struct hci_ev_remote_features)),
+ /* [0x0e = HCI_EV_CMD_COMPLETE] */
+ HCI_EV_REQ_VL(HCI_EV_CMD_COMPLETE, hci_cmd_complete_evt,
+ sizeof(struct hci_ev_cmd_complete), HCI_MAX_EVENT_SIZE),
+ /* [0x0f = HCI_EV_CMD_STATUS] */
+ HCI_EV_REQ(HCI_EV_CMD_STATUS, hci_cmd_status_evt,
+ sizeof(struct hci_ev_cmd_status)),
+ /* [0x10 = HCI_EV_CMD_STATUS] */
+ HCI_EV(HCI_EV_HARDWARE_ERROR, hci_hardware_error_evt,
+ sizeof(struct hci_ev_hardware_error)),
+ /* [0x12 = HCI_EV_ROLE_CHANGE] */
+ HCI_EV(HCI_EV_ROLE_CHANGE, hci_role_change_evt,
+ sizeof(struct hci_ev_role_change)),
+ /* [0x13 = HCI_EV_NUM_COMP_PKTS] */
+ HCI_EV_VL(HCI_EV_NUM_COMP_PKTS, hci_num_comp_pkts_evt,
+ sizeof(struct hci_ev_num_comp_pkts), HCI_MAX_EVENT_SIZE),
+ /* [0x14 = HCI_EV_MODE_CHANGE] */
+ HCI_EV(HCI_EV_MODE_CHANGE, hci_mode_change_evt,
+ sizeof(struct hci_ev_mode_change)),
+ /* [0x16 = HCI_EV_PIN_CODE_REQ] */
+ HCI_EV(HCI_EV_PIN_CODE_REQ, hci_pin_code_request_evt,
+ sizeof(struct hci_ev_pin_code_req)),
+ /* [0x17 = HCI_EV_LINK_KEY_REQ] */
+ HCI_EV(HCI_EV_LINK_KEY_REQ, hci_link_key_request_evt,
+ sizeof(struct hci_ev_link_key_req)),
+ /* [0x18 = HCI_EV_LINK_KEY_NOTIFY] */
+ HCI_EV(HCI_EV_LINK_KEY_NOTIFY, hci_link_key_notify_evt,
+ sizeof(struct hci_ev_link_key_notify)),
+ /* [0x1c = HCI_EV_CLOCK_OFFSET] */
+ HCI_EV(HCI_EV_CLOCK_OFFSET, hci_clock_offset_evt,
+ sizeof(struct hci_ev_clock_offset)),
+ /* [0x1d = HCI_EV_PKT_TYPE_CHANGE] */
+ HCI_EV(HCI_EV_PKT_TYPE_CHANGE, hci_pkt_type_change_evt,
+ sizeof(struct hci_ev_pkt_type_change)),
+ /* [0x20 = HCI_EV_PSCAN_REP_MODE] */
+ HCI_EV(HCI_EV_PSCAN_REP_MODE, hci_pscan_rep_mode_evt,
+ sizeof(struct hci_ev_pscan_rep_mode)),
+ /* [0x22 = HCI_EV_INQUIRY_RESULT_WITH_RSSI] */
+ HCI_EV_VL(HCI_EV_INQUIRY_RESULT_WITH_RSSI,
+ hci_inquiry_result_with_rssi_evt,
+ sizeof(struct hci_ev_inquiry_result_rssi),
+ HCI_MAX_EVENT_SIZE),
+ /* [0x23 = HCI_EV_REMOTE_EXT_FEATURES] */
+ HCI_EV(HCI_EV_REMOTE_EXT_FEATURES, hci_remote_ext_features_evt,
+ sizeof(struct hci_ev_remote_ext_features)),
+ /* [0x2c = HCI_EV_SYNC_CONN_COMPLETE] */
+ HCI_EV(HCI_EV_SYNC_CONN_COMPLETE, hci_sync_conn_complete_evt,
+ sizeof(struct hci_ev_sync_conn_complete)),
+ /* [0x2d = HCI_EV_EXTENDED_INQUIRY_RESULT] */
+ HCI_EV_VL(HCI_EV_EXTENDED_INQUIRY_RESULT,
+ hci_extended_inquiry_result_evt,
+ sizeof(struct hci_ev_ext_inquiry_result), HCI_MAX_EVENT_SIZE),
+ /* [0x30 = HCI_EV_KEY_REFRESH_COMPLETE] */
+ HCI_EV(HCI_EV_KEY_REFRESH_COMPLETE, hci_key_refresh_complete_evt,
+ sizeof(struct hci_ev_key_refresh_complete)),
+ /* [0x31 = HCI_EV_IO_CAPA_REQUEST] */
+ HCI_EV(HCI_EV_IO_CAPA_REQUEST, hci_io_capa_request_evt,
+ sizeof(struct hci_ev_io_capa_request)),
+ /* [0x32 = HCI_EV_IO_CAPA_REPLY] */
+ HCI_EV(HCI_EV_IO_CAPA_REPLY, hci_io_capa_reply_evt,
+ sizeof(struct hci_ev_io_capa_reply)),
+ /* [0x33 = HCI_EV_USER_CONFIRM_REQUEST] */
+ HCI_EV(HCI_EV_USER_CONFIRM_REQUEST, hci_user_confirm_request_evt,
+ sizeof(struct hci_ev_user_confirm_req)),
+ /* [0x34 = HCI_EV_USER_PASSKEY_REQUEST] */
+ HCI_EV(HCI_EV_USER_PASSKEY_REQUEST, hci_user_passkey_request_evt,
+ sizeof(struct hci_ev_user_passkey_req)),
+ /* [0x35 = HCI_EV_REMOTE_OOB_DATA_REQUEST] */
+ HCI_EV(HCI_EV_REMOTE_OOB_DATA_REQUEST, hci_remote_oob_data_request_evt,
+ sizeof(struct hci_ev_remote_oob_data_request)),
+ /* [0x36 = HCI_EV_SIMPLE_PAIR_COMPLETE] */
+ HCI_EV(HCI_EV_SIMPLE_PAIR_COMPLETE, hci_simple_pair_complete_evt,
+ sizeof(struct hci_ev_simple_pair_complete)),
+ /* [0x3b = HCI_EV_USER_PASSKEY_NOTIFY] */
+ HCI_EV(HCI_EV_USER_PASSKEY_NOTIFY, hci_user_passkey_notify_evt,
+ sizeof(struct hci_ev_user_passkey_notify)),
+ /* [0x3c = HCI_EV_KEYPRESS_NOTIFY] */
+ HCI_EV(HCI_EV_KEYPRESS_NOTIFY, hci_keypress_notify_evt,
+ sizeof(struct hci_ev_keypress_notify)),
+ /* [0x3d = HCI_EV_REMOTE_HOST_FEATURES] */
+ HCI_EV(HCI_EV_REMOTE_HOST_FEATURES, hci_remote_host_features_evt,
+ sizeof(struct hci_ev_remote_host_features)),
+ /* [0x3e = HCI_EV_LE_META] */
+ HCI_EV_REQ_VL(HCI_EV_LE_META, hci_le_meta_evt,
+ sizeof(struct hci_ev_le_meta), HCI_MAX_EVENT_SIZE),
+#if IS_ENABLED(CONFIG_BT_HS)
+ /* [0x40 = HCI_EV_PHY_LINK_COMPLETE] */
+ HCI_EV(HCI_EV_PHY_LINK_COMPLETE, hci_phy_link_complete_evt,
+ sizeof(struct hci_ev_phy_link_complete)),
+ /* [0x41 = HCI_EV_CHANNEL_SELECTED] */
+ HCI_EV(HCI_EV_CHANNEL_SELECTED, hci_chan_selected_evt,
+ sizeof(struct hci_ev_channel_selected)),
+ /* [0x42 = HCI_EV_DISCONN_PHY_LINK_COMPLETE] */
+ HCI_EV(HCI_EV_DISCONN_LOGICAL_LINK_COMPLETE,
+ hci_disconn_loglink_complete_evt,
+ sizeof(struct hci_ev_disconn_logical_link_complete)),
+ /* [0x45 = HCI_EV_LOGICAL_LINK_COMPLETE] */
+ HCI_EV(HCI_EV_LOGICAL_LINK_COMPLETE, hci_loglink_complete_evt,
+ sizeof(struct hci_ev_logical_link_complete)),
+ /* [0x46 = HCI_EV_DISCONN_LOGICAL_LINK_COMPLETE] */
+ HCI_EV(HCI_EV_DISCONN_PHY_LINK_COMPLETE,
+ hci_disconn_phylink_complete_evt,
+ sizeof(struct hci_ev_disconn_phy_link_complete)),
+#endif
+ /* [0x48 = HCI_EV_NUM_COMP_BLOCKS] */
+ HCI_EV(HCI_EV_NUM_COMP_BLOCKS, hci_num_comp_blocks_evt,
+ sizeof(struct hci_ev_num_comp_blocks)),
+ /* [0xff = HCI_EV_VENDOR] */
+ HCI_EV(HCI_EV_VENDOR, msft_vendor_evt, 0),
+};
+
+static void hci_event_func(struct hci_dev *hdev, u8 event, struct sk_buff *skb,
+ u16 *opcode, u8 *status,
+ hci_req_complete_t *req_complete,
+ hci_req_complete_skb_t *req_complete_skb)
+{
+ const struct hci_ev *ev = &hci_ev_table[event];
+ void *data;
+
+ if (!ev->func)
+ return;
+
+ if (skb->len < ev->min_len) {
+ bt_dev_err(hdev, "unexpected event 0x%2.2x length: %u < %u",
+ event, skb->len, ev->min_len);
+ return;
+ }
+
+ /* Just warn if the length is over max_len size it still be
+ * possible to partially parse the event so leave to callback to
+ * decide if that is acceptable.
+ */
+ if (skb->len > ev->max_len)
+ bt_dev_warn(hdev, "unexpected event 0x%2.2x length: %u > %u",
+ event, skb->len, ev->max_len);
+
+ data = hci_ev_skb_pull(hdev, skb, event, ev->min_len);
+ if (!data)
+ return;
+
+ if (ev->req)
+ ev->func_req(hdev, data, skb, opcode, status, req_complete,
+ req_complete_skb);
+ else
+ ev->func(hdev, data, skb);
+}
+
void hci_event_packet(struct hci_dev *hdev, struct sk_buff *skb)
{
struct hci_event_hdr *hdr = (void *) skb->data;
hci_req_complete_t req_complete = NULL;
hci_req_complete_skb_t req_complete_skb = NULL;
struct sk_buff *orig_skb = NULL;
- u8 status = 0, event = hdr->evt, req_evt = 0;
+ u8 status = 0, event, req_evt = 0;
u16 opcode = HCI_OP_NOP;
- if (!event) {
- bt_dev_warn(hdev, "Received unexpected HCI Event 00000000");
+ if (skb->len < sizeof(*hdr)) {
+ bt_dev_err(hdev, "Malformed HCI Event");
goto done;
}
- if (hdev->sent_cmd && bt_cb(hdev->sent_cmd)->hci.req_event == event) {
- struct hci_command_hdr *cmd_hdr = (void *) hdev->sent_cmd->data;
- opcode = __le16_to_cpu(cmd_hdr->opcode);
- hci_req_cmd_complete(hdev, opcode, status, &req_complete,
- &req_complete_skb);
+ event = hdr->evt;
+ if (!event) {
+ bt_dev_warn(hdev, "Received unexpected HCI Event 0x%2.2x",
+ event);
+ goto done;
+ }
+
+ /* Only match event if command OGF is not for LE */
+ if (hdev->sent_cmd &&
+ hci_opcode_ogf(hci_skb_opcode(hdev->sent_cmd)) != 0x08 &&
+ hci_skb_event(hdev->sent_cmd) == event) {
+ hci_req_cmd_complete(hdev, hci_skb_opcode(hdev->sent_cmd),
+ status, &req_complete, &req_complete_skb);
req_evt = event;
}
@@ -6343,191 +6881,10 @@ void hci_event_packet(struct hci_dev *hdev, struct sk_buff *skb)
/* Store wake reason if we're suspended */
hci_store_wake_reason(hdev, event, skb);
- switch (event) {
- case HCI_EV_INQUIRY_COMPLETE:
- hci_inquiry_complete_evt(hdev, skb);
- break;
+ bt_dev_dbg(hdev, "event 0x%2.2x", event);
- case HCI_EV_INQUIRY_RESULT:
- hci_inquiry_result_evt(hdev, skb);
- break;
-
- case HCI_EV_CONN_COMPLETE:
- hci_conn_complete_evt(hdev, skb);
- break;
-
- case HCI_EV_CONN_REQUEST:
- hci_conn_request_evt(hdev, skb);
- break;
-
- case HCI_EV_DISCONN_COMPLETE:
- hci_disconn_complete_evt(hdev, skb);
- break;
-
- case HCI_EV_AUTH_COMPLETE:
- hci_auth_complete_evt(hdev, skb);
- break;
-
- case HCI_EV_REMOTE_NAME:
- hci_remote_name_evt(hdev, skb);
- break;
-
- case HCI_EV_ENCRYPT_CHANGE:
- hci_encrypt_change_evt(hdev, skb);
- break;
-
- case HCI_EV_CHANGE_LINK_KEY_COMPLETE:
- hci_change_link_key_complete_evt(hdev, skb);
- break;
-
- case HCI_EV_REMOTE_FEATURES:
- hci_remote_features_evt(hdev, skb);
- break;
-
- case HCI_EV_CMD_COMPLETE:
- hci_cmd_complete_evt(hdev, skb, &opcode, &status,
- &req_complete, &req_complete_skb);
- break;
-
- case HCI_EV_CMD_STATUS:
- hci_cmd_status_evt(hdev, skb, &opcode, &status, &req_complete,
- &req_complete_skb);
- break;
-
- case HCI_EV_HARDWARE_ERROR:
- hci_hardware_error_evt(hdev, skb);
- break;
-
- case HCI_EV_ROLE_CHANGE:
- hci_role_change_evt(hdev, skb);
- break;
-
- case HCI_EV_NUM_COMP_PKTS:
- hci_num_comp_pkts_evt(hdev, skb);
- break;
-
- case HCI_EV_MODE_CHANGE:
- hci_mode_change_evt(hdev, skb);
- break;
-
- case HCI_EV_PIN_CODE_REQ:
- hci_pin_code_request_evt(hdev, skb);
- break;
-
- case HCI_EV_LINK_KEY_REQ:
- hci_link_key_request_evt(hdev, skb);
- break;
-
- case HCI_EV_LINK_KEY_NOTIFY:
- hci_link_key_notify_evt(hdev, skb);
- break;
-
- case HCI_EV_CLOCK_OFFSET:
- hci_clock_offset_evt(hdev, skb);
- break;
-
- case HCI_EV_PKT_TYPE_CHANGE:
- hci_pkt_type_change_evt(hdev, skb);
- break;
-
- case HCI_EV_PSCAN_REP_MODE:
- hci_pscan_rep_mode_evt(hdev, skb);
- break;
-
- case HCI_EV_INQUIRY_RESULT_WITH_RSSI:
- hci_inquiry_result_with_rssi_evt(hdev, skb);
- break;
-
- case HCI_EV_REMOTE_EXT_FEATURES:
- hci_remote_ext_features_evt(hdev, skb);
- break;
-
- case HCI_EV_SYNC_CONN_COMPLETE:
- hci_sync_conn_complete_evt(hdev, skb);
- break;
-
- case HCI_EV_EXTENDED_INQUIRY_RESULT:
- hci_extended_inquiry_result_evt(hdev, skb);
- break;
-
- case HCI_EV_KEY_REFRESH_COMPLETE:
- hci_key_refresh_complete_evt(hdev, skb);
- break;
-
- case HCI_EV_IO_CAPA_REQUEST:
- hci_io_capa_request_evt(hdev, skb);
- break;
-
- case HCI_EV_IO_CAPA_REPLY:
- hci_io_capa_reply_evt(hdev, skb);
- break;
-
- case HCI_EV_USER_CONFIRM_REQUEST:
- hci_user_confirm_request_evt(hdev, skb);
- break;
-
- case HCI_EV_USER_PASSKEY_REQUEST:
- hci_user_passkey_request_evt(hdev, skb);
- break;
-
- case HCI_EV_USER_PASSKEY_NOTIFY:
- hci_user_passkey_notify_evt(hdev, skb);
- break;
-
- case HCI_EV_KEYPRESS_NOTIFY:
- hci_keypress_notify_evt(hdev, skb);
- break;
-
- case HCI_EV_SIMPLE_PAIR_COMPLETE:
- hci_simple_pair_complete_evt(hdev, skb);
- break;
-
- case HCI_EV_REMOTE_HOST_FEATURES:
- hci_remote_host_features_evt(hdev, skb);
- break;
-
- case HCI_EV_LE_META:
- hci_le_meta_evt(hdev, skb);
- break;
-
- case HCI_EV_REMOTE_OOB_DATA_REQUEST:
- hci_remote_oob_data_request_evt(hdev, skb);
- break;
-
-#if IS_ENABLED(CONFIG_BT_HS)
- case HCI_EV_CHANNEL_SELECTED:
- hci_chan_selected_evt(hdev, skb);
- break;
-
- case HCI_EV_PHY_LINK_COMPLETE:
- hci_phy_link_complete_evt(hdev, skb);
- break;
-
- case HCI_EV_LOGICAL_LINK_COMPLETE:
- hci_loglink_complete_evt(hdev, skb);
- break;
-
- case HCI_EV_DISCONN_LOGICAL_LINK_COMPLETE:
- hci_disconn_loglink_complete_evt(hdev, skb);
- break;
-
- case HCI_EV_DISCONN_PHY_LINK_COMPLETE:
- hci_disconn_phylink_complete_evt(hdev, skb);
- break;
-#endif
-
- case HCI_EV_NUM_COMP_BLOCKS:
- hci_num_comp_blocks_evt(hdev, skb);
- break;
-
- case HCI_EV_VENDOR:
- msft_vendor_evt(hdev, skb);
- break;
-
- default:
- BT_DBG("%s event 0x%2.2x", hdev->name, event);
- break;
- }
+ hci_event_func(hdev, event, skb, &opcode, &status, &req_complete,
+ &req_complete_skb);
if (req_complete) {
req_complete(hdev, status, opcode);
diff --git a/net/bluetooth/hci_request.c b/net/bluetooth/hci_request.c
index 92611bf..42c8047 100644
--- a/net/bluetooth/hci_request.c
+++ b/net/bluetooth/hci_request.c
@@ -32,10 +32,6 @@
#include "msft.h"
#include "eir.h"
-#define HCI_REQ_DONE 0
-#define HCI_REQ_PEND 1
-#define HCI_REQ_CANCELED 2
-
void hci_req_init(struct hci_request *req, struct hci_dev *hdev)
{
skb_queue_head_init(&req->cmd_q);
@@ -101,8 +97,8 @@ int hci_req_run_skb(struct hci_request *req, hci_req_complete_skb_t complete)
return req_run(req, NULL, complete);
}
-static void hci_req_sync_complete(struct hci_dev *hdev, u8 result, u16 opcode,
- struct sk_buff *skb)
+void hci_req_sync_complete(struct hci_dev *hdev, u8 result, u16 opcode,
+ struct sk_buff *skb)
{
bt_dev_dbg(hdev, "result 0x%2.2x", result);
@@ -115,81 +111,6 @@ static void hci_req_sync_complete(struct hci_dev *hdev, u8 result, u16 opcode,
}
}
-void hci_req_sync_cancel(struct hci_dev *hdev, int err)
-{
- bt_dev_dbg(hdev, "err 0x%2.2x", err);
-
- if (hdev->req_status == HCI_REQ_PEND) {
- hdev->req_result = err;
- hdev->req_status = HCI_REQ_CANCELED;
- wake_up_interruptible(&hdev->req_wait_q);
- }
-}
-
-struct sk_buff *__hci_cmd_sync_ev(struct hci_dev *hdev, u16 opcode, u32 plen,
- const void *param, u8 event, u32 timeout)
-{
- struct hci_request req;
- struct sk_buff *skb;
- int err = 0;
-
- bt_dev_dbg(hdev, "");
-
- hci_req_init(&req, hdev);
-
- hci_req_add_ev(&req, opcode, plen, param, event);
-
- hdev->req_status = HCI_REQ_PEND;
-
- err = hci_req_run_skb(&req, hci_req_sync_complete);
- if (err < 0)
- return ERR_PTR(err);
-
- err = wait_event_interruptible_timeout(hdev->req_wait_q,
- hdev->req_status != HCI_REQ_PEND, timeout);
-
- if (err == -ERESTARTSYS)
- return ERR_PTR(-EINTR);
-
- switch (hdev->req_status) {
- case HCI_REQ_DONE:
- err = -bt_to_errno(hdev->req_result);
- break;
-
- case HCI_REQ_CANCELED:
- err = -hdev->req_result;
- break;
-
- default:
- err = -ETIMEDOUT;
- break;
- }
-
- hdev->req_status = hdev->req_result = 0;
- skb = hdev->req_skb;
- hdev->req_skb = NULL;
-
- bt_dev_dbg(hdev, "end: err %d", err);
-
- if (err < 0) {
- kfree_skb(skb);
- return ERR_PTR(err);
- }
-
- if (!skb)
- return ERR_PTR(-ENODATA);
-
- return skb;
-}
-EXPORT_SYMBOL(__hci_cmd_sync_ev);
-
-struct sk_buff *__hci_cmd_sync(struct hci_dev *hdev, u16 opcode, u32 plen,
- const void *param, u32 timeout)
-{
- return __hci_cmd_sync_ev(hdev, opcode, plen, param, 0, timeout);
-}
-EXPORT_SYMBOL(__hci_cmd_sync);
-
/* Execute request and wait for completion. */
int __hci_req_sync(struct hci_dev *hdev, int (*func)(struct hci_request *req,
unsigned long opt),
@@ -436,82 +357,6 @@ static bool __hci_update_interleaved_scan(struct hci_dev *hdev)
return false;
}
-/* This function controls the background scanning based on hdev->pend_le_conns
- * list. If there are pending LE connection we start the background scanning,
- * otherwise we stop it.
- *
- * This function requires the caller holds hdev->lock.
- */
-static void __hci_update_background_scan(struct hci_request *req)
-{
- struct hci_dev *hdev = req->hdev;
-
- if (!test_bit(HCI_UP, &hdev->flags) ||
- test_bit(HCI_INIT, &hdev->flags) ||
- hci_dev_test_flag(hdev, HCI_SETUP) ||
- hci_dev_test_flag(hdev, HCI_CONFIG) ||
- hci_dev_test_flag(hdev, HCI_AUTO_OFF) ||
- hci_dev_test_flag(hdev, HCI_UNREGISTER))
- return;
-
- /* No point in doing scanning if LE support hasn't been enabled */
- if (!hci_dev_test_flag(hdev, HCI_LE_ENABLED))
- return;
-
- /* If discovery is active don't interfere with it */
- if (hdev->discovery.state != DISCOVERY_STOPPED)
- return;
-
- /* Reset RSSI and UUID filters when starting background scanning
- * since these filters are meant for service discovery only.
- *
- * The Start Discovery and Start Service Discovery operations
- * ensure to set proper values for RSSI threshold and UUID
- * filter list. So it is safe to just reset them here.
- */
- hci_discovery_filter_clear(hdev);
-
- bt_dev_dbg(hdev, "ADV monitoring is %s",
- hci_is_adv_monitoring(hdev) ? "on" : "off");
-
- if (list_empty(&hdev->pend_le_conns) &&
- list_empty(&hdev->pend_le_reports) &&
- !hci_is_adv_monitoring(hdev)) {
- /* If there is no pending LE connections or devices
- * to be scanned for or no ADV monitors, we should stop the
- * background scanning.
- */
-
- /* If controller is not scanning we are done. */
- if (!hci_dev_test_flag(hdev, HCI_LE_SCAN))
- return;
-
- hci_req_add_le_scan_disable(req, false);
-
- bt_dev_dbg(hdev, "stopping background scanning");
- } else {
- /* If there is at least one pending LE connection, we should
- * keep the background scan running.
- */
-
- /* If controller is connecting, we should not start scanning
- * since some controllers are not able to scan and connect at
- * the same time.
- */
- if (hci_lookup_le_connect(hdev))
- return;
-
- /* If controller is currently scanning, we stop it to ensure we
- * don't miss any advertising (due to duplicates filter).
- */
- if (hci_dev_test_flag(hdev, HCI_LE_SCAN))
- hci_req_add_le_scan_disable(req, false);
-
- hci_req_add_le_passive_scan(req);
- bt_dev_dbg(hdev, "starting background scanning");
- }
-}
-
void __hci_req_update_name(struct hci_request *req)
{
struct hci_dev *hdev = req->hdev;
@@ -560,9 +405,6 @@ void hci_req_add_le_scan_disable(struct hci_request *req, bool rpa_le_conn)
return;
}
- if (hdev->suspended)
- set_bit(SUSPEND_SCAN_DISABLE, hdev->suspend_tasks);
-
if (use_ext_scan(hdev)) {
struct hci_cp_le_set_ext_scan_enable cp;
@@ -579,9 +421,7 @@ void hci_req_add_le_scan_disable(struct hci_request *req, bool rpa_le_conn)
}
/* Disable address resolution */
- if (use_ll_privacy(hdev) &&
- hci_dev_test_flag(hdev, HCI_ENABLE_LL_PRIVACY) &&
- hci_dev_test_flag(hdev, HCI_LL_RPA_RESOLUTION) && !rpa_le_conn) {
+ if (hci_dev_test_flag(hdev, HCI_LL_RPA_RESOLUTION) && !rpa_le_conn) {
__u8 enable = 0x00;
hci_req_add(req, HCI_OP_LE_SET_ADDR_RESOLV_ENABLE, 1, &enable);
@@ -600,8 +440,7 @@ static void del_from_accept_list(struct hci_request *req, bdaddr_t *bdaddr,
cp.bdaddr_type);
hci_req_add(req, HCI_OP_LE_DEL_FROM_ACCEPT_LIST, sizeof(cp), &cp);
- if (use_ll_privacy(req->hdev) &&
- hci_dev_test_flag(req->hdev, HCI_ENABLE_LL_PRIVACY)) {
+ if (use_ll_privacy(req->hdev)) {
struct smp_irk *irk;
irk = hci_find_irk_by_addr(req->hdev, bdaddr, bdaddr_type);
@@ -642,8 +481,8 @@ static int add_to_accept_list(struct hci_request *req,
}
/* During suspend, only wakeable devices can be in accept list */
- if (hdev->suspended && !hci_conn_test_flag(HCI_CONN_FLAG_REMOTE_WAKEUP,
- params->current_flags))
+ if (hdev->suspended &&
+ !test_bit(HCI_CONN_FLAG_REMOTE_WAKEUP, params->flags))
return 0;
*num_entries += 1;
@@ -654,8 +493,7 @@ static int add_to_accept_list(struct hci_request *req,
cp.bdaddr_type);
hci_req_add(req, HCI_OP_LE_ADD_TO_ACCEPT_LIST, sizeof(cp), &cp);
- if (use_ll_privacy(hdev) &&
- hci_dev_test_flag(hdev, HCI_ENABLE_LL_PRIVACY)) {
+ if (use_ll_privacy(hdev)) {
struct smp_irk *irk;
irk = hci_find_irk_by_addr(hdev, ¶ms->addr,
@@ -694,8 +532,7 @@ static u8 update_accept_list(struct hci_request *req)
*/
bool allow_rpa = hdev->suspended;
- if (use_ll_privacy(hdev) &&
- hci_dev_test_flag(hdev, HCI_ENABLE_LL_PRIVACY))
+ if (use_ll_privacy(hdev))
allow_rpa = true;
/* Go through the current accept list programmed into the
@@ -784,9 +621,7 @@ static void hci_req_start_scan(struct hci_request *req, u8 type, u16 interval,
return;
}
- if (use_ll_privacy(hdev) &&
- hci_dev_test_flag(hdev, HCI_ENABLE_LL_PRIVACY) &&
- addr_resolv) {
+ if (use_ll_privacy(hdev) && addr_resolv) {
u8 enable = 0x01;
hci_req_add(req, HCI_OP_LE_SET_ADDR_RESOLV_ENABLE, 1, &enable);
@@ -943,8 +778,6 @@ void hci_req_add_le_passive_scan(struct hci_request *req)
if (hdev->suspended) {
window = hdev->le_scan_window_suspend;
interval = hdev->le_scan_int_suspend;
-
- set_bit(SUSPEND_SCAN_ENABLE, hdev->suspend_tasks);
} else if (hci_is_le_conn_scanning(hdev)) {
window = hdev->le_scan_window_connect;
interval = hdev->le_scan_int_connect;
@@ -977,59 +810,6 @@ void hci_req_add_le_passive_scan(struct hci_request *req)
addr_resolv);
}
-static void hci_req_clear_event_filter(struct hci_request *req)
-{
- struct hci_cp_set_event_filter f;
-
- if (!hci_dev_test_flag(req->hdev, HCI_BREDR_ENABLED))
- return;
-
- if (hci_dev_test_flag(req->hdev, HCI_EVENT_FILTER_CONFIGURED)) {
- memset(&f, 0, sizeof(f));
- f.flt_type = HCI_FLT_CLEAR_ALL;
- hci_req_add(req, HCI_OP_SET_EVENT_FLT, 1, &f);
- }
-}
-
-static void hci_req_set_event_filter(struct hci_request *req)
-{
- struct bdaddr_list_with_flags *b;
- struct hci_cp_set_event_filter f;
- struct hci_dev *hdev = req->hdev;
- u8 scan = SCAN_DISABLED;
- bool scanning = test_bit(HCI_PSCAN, &hdev->flags);
-
- if (!hci_dev_test_flag(hdev, HCI_BREDR_ENABLED))
- return;
-
- /* Always clear event filter when starting */
- hci_req_clear_event_filter(req);
-
- list_for_each_entry(b, &hdev->accept_list, list) {
- if (!hci_conn_test_flag(HCI_CONN_FLAG_REMOTE_WAKEUP,
- b->current_flags))
- continue;
-
- memset(&f, 0, sizeof(f));
- bacpy(&f.addr_conn_flt.bdaddr, &b->bdaddr);
- f.flt_type = HCI_FLT_CONN_SETUP;
- f.cond_type = HCI_CONN_SETUP_ALLOW_BDADDR;
- f.addr_conn_flt.auto_accept = HCI_CONN_SETUP_AUTO_ON;
-
- bt_dev_dbg(hdev, "Adding event filters for %pMR", &b->bdaddr);
- hci_req_add(req, HCI_OP_SET_EVENT_FLT, sizeof(f), &f);
- scan = SCAN_PAGE;
- }
-
- if (scan && !scanning) {
- set_bit(SUSPEND_SCAN_ENABLE, hdev->suspend_tasks);
- hci_req_add(req, HCI_OP_WRITE_SCAN_ENABLE, 1, &scan);
- } else if (!scan && scanning) {
- set_bit(SUSPEND_SCAN_DISABLE, hdev->suspend_tasks);
- hci_req_add(req, HCI_OP_WRITE_SCAN_ENABLE, 1, &scan);
- }
-}
-
static void cancel_adv_timeout(struct hci_dev *hdev)
{
if (hdev->adv_instance_timeout) {
@@ -1038,235 +818,6 @@ static void cancel_adv_timeout(struct hci_dev *hdev)
}
}
-/* This function requires the caller holds hdev->lock */
-void __hci_req_pause_adv_instances(struct hci_request *req)
-{
- bt_dev_dbg(req->hdev, "Pausing advertising instances");
-
- /* Call to disable any advertisements active on the controller.
- * This will succeed even if no advertisements are configured.
- */
- __hci_req_disable_advertising(req);
-
- /* If we are using software rotation, pause the loop */
- if (!ext_adv_capable(req->hdev))
- cancel_adv_timeout(req->hdev);
-}
-
-/* This function requires the caller holds hdev->lock */
-static void __hci_req_resume_adv_instances(struct hci_request *req)
-{
- struct adv_info *adv;
-
- bt_dev_dbg(req->hdev, "Resuming advertising instances");
-
- if (ext_adv_capable(req->hdev)) {
- /* Call for each tracked instance to be re-enabled */
- list_for_each_entry(adv, &req->hdev->adv_instances, list) {
- __hci_req_enable_ext_advertising(req,
- adv->instance);
- }
-
- } else {
- /* Schedule for most recent instance to be restarted and begin
- * the software rotation loop
- */
- __hci_req_schedule_adv_instance(req,
- req->hdev->cur_adv_instance,
- true);
- }
-}
-
-/* This function requires the caller holds hdev->lock */
-int hci_req_resume_adv_instances(struct hci_dev *hdev)
-{
- struct hci_request req;
-
- hci_req_init(&req, hdev);
- __hci_req_resume_adv_instances(&req);
-
- return hci_req_run(&req, NULL);
-}
-
-static void suspend_req_complete(struct hci_dev *hdev, u8 status, u16 opcode)
-{
- bt_dev_dbg(hdev, "Request complete opcode=0x%x, status=0x%x", opcode,
- status);
- if (test_bit(SUSPEND_SCAN_ENABLE, hdev->suspend_tasks) ||
- test_bit(SUSPEND_SCAN_DISABLE, hdev->suspend_tasks)) {
- clear_bit(SUSPEND_SCAN_ENABLE, hdev->suspend_tasks);
- clear_bit(SUSPEND_SCAN_DISABLE, hdev->suspend_tasks);
- wake_up(&hdev->suspend_wait_q);
- }
-
- if (test_bit(SUSPEND_SET_ADV_FILTER, hdev->suspend_tasks)) {
- clear_bit(SUSPEND_SET_ADV_FILTER, hdev->suspend_tasks);
- wake_up(&hdev->suspend_wait_q);
- }
-}
-
-static void hci_req_prepare_adv_monitor_suspend(struct hci_request *req,
- bool suspending)
-{
- struct hci_dev *hdev = req->hdev;
-
- switch (hci_get_adv_monitor_offload_ext(hdev)) {
- case HCI_ADV_MONITOR_EXT_MSFT:
- if (suspending)
- msft_suspend(hdev);
- else
- msft_resume(hdev);
- break;
- default:
- return;
- }
-
- /* No need to block when enabling since it's on resume path */
- if (hdev->suspended && suspending)
- set_bit(SUSPEND_SET_ADV_FILTER, hdev->suspend_tasks);
-}
-
-/* Call with hci_dev_lock */
-void hci_req_prepare_suspend(struct hci_dev *hdev, enum suspended_state next)
-{
- int old_state;
- struct hci_conn *conn;
- struct hci_request req;
- u8 page_scan;
- int disconnect_counter;
-
- if (next == hdev->suspend_state) {
- bt_dev_dbg(hdev, "Same state before and after: %d", next);
- goto done;
- }
-
- hdev->suspend_state = next;
- hci_req_init(&req, hdev);
-
- if (next == BT_SUSPEND_DISCONNECT) {
- /* Mark device as suspended */
- hdev->suspended = true;
-
- /* Pause discovery if not already stopped */
- old_state = hdev->discovery.state;
- if (old_state != DISCOVERY_STOPPED) {
- set_bit(SUSPEND_PAUSE_DISCOVERY, hdev->suspend_tasks);
- hci_discovery_set_state(hdev, DISCOVERY_STOPPING);
- queue_work(hdev->req_workqueue, &hdev->discov_update);
- }
-
- hdev->discovery_paused = true;
- hdev->discovery_old_state = old_state;
-
- /* Stop directed advertising */
- old_state = hci_dev_test_flag(hdev, HCI_ADVERTISING);
- if (old_state) {
- set_bit(SUSPEND_PAUSE_ADVERTISING, hdev->suspend_tasks);
- cancel_delayed_work(&hdev->discov_off);
- queue_delayed_work(hdev->req_workqueue,
- &hdev->discov_off, 0);
- }
-
- /* Pause other advertisements */
- if (hdev->adv_instance_cnt)
- __hci_req_pause_adv_instances(&req);
-
- hdev->advertising_paused = true;
- hdev->advertising_old_state = old_state;
-
- /* Disable page scan if enabled */
- if (test_bit(HCI_PSCAN, &hdev->flags)) {
- page_scan = SCAN_DISABLED;
- hci_req_add(&req, HCI_OP_WRITE_SCAN_ENABLE, 1,
- &page_scan);
- set_bit(SUSPEND_SCAN_DISABLE, hdev->suspend_tasks);
- }
-
- /* Disable LE passive scan if enabled */
- if (hci_dev_test_flag(hdev, HCI_LE_SCAN)) {
- cancel_interleave_scan(hdev);
- hci_req_add_le_scan_disable(&req, false);
- }
-
- /* Disable advertisement filters */
- hci_req_prepare_adv_monitor_suspend(&req, true);
-
- /* Prevent disconnects from causing scanning to be re-enabled */
- hdev->scanning_paused = true;
-
- /* Run commands before disconnecting */
- hci_req_run(&req, suspend_req_complete);
-
- disconnect_counter = 0;
- /* Soft disconnect everything (power off) */
- list_for_each_entry(conn, &hdev->conn_hash.list, list) {
- hci_disconnect(conn, HCI_ERROR_REMOTE_POWER_OFF);
- disconnect_counter++;
- }
-
- if (disconnect_counter > 0) {
- bt_dev_dbg(hdev,
- "Had %d disconnects. Will wait on them",
- disconnect_counter);
- set_bit(SUSPEND_DISCONNECTING, hdev->suspend_tasks);
- }
- } else if (next == BT_SUSPEND_CONFIGURE_WAKE) {
- /* Unpause to take care of updating scanning params */
- hdev->scanning_paused = false;
- /* Enable event filter for paired devices */
- hci_req_set_event_filter(&req);
- /* Enable passive scan at lower duty cycle */
- __hci_update_background_scan(&req);
- /* Pause scan changes again. */
- hdev->scanning_paused = true;
- hci_req_run(&req, suspend_req_complete);
- } else {
- hdev->suspended = false;
- hdev->scanning_paused = false;
-
- /* Clear any event filters and restore scan state */
- hci_req_clear_event_filter(&req);
- __hci_req_update_scan(&req);
-
- /* Reset passive/background scanning to normal */
- __hci_update_background_scan(&req);
- /* Enable all of the advertisement filters */
- hci_req_prepare_adv_monitor_suspend(&req, false);
-
- /* Unpause directed advertising */
- hdev->advertising_paused = false;
- if (hdev->advertising_old_state) {
- set_bit(SUSPEND_UNPAUSE_ADVERTISING,
- hdev->suspend_tasks);
- hci_dev_set_flag(hdev, HCI_ADVERTISING);
- queue_work(hdev->req_workqueue,
- &hdev->discoverable_update);
- hdev->advertising_old_state = 0;
- }
-
- /* Resume other advertisements */
- if (hdev->adv_instance_cnt)
- __hci_req_resume_adv_instances(&req);
-
- /* Unpause discovery */
- hdev->discovery_paused = false;
- if (hdev->discovery_old_state != DISCOVERY_STOPPED &&
- hdev->discovery_old_state != DISCOVERY_STOPPING) {
- set_bit(SUSPEND_UNPAUSE_DISCOVERY, hdev->suspend_tasks);
- hci_discovery_set_state(hdev, DISCOVERY_STARTING);
- queue_work(hdev->req_workqueue, &hdev->discov_update);
- }
-
- hci_req_run(&req, suspend_req_complete);
- }
-
- hdev->suspend_state = next;
-
-done:
- clear_bit(SUSPEND_PREPARE_NOTIFIER, hdev->suspend_tasks);
- wake_up(&hdev->suspend_wait_q);
-}
-
static bool adv_cur_instance_is_scannable(struct hci_dev *hdev)
{
return hci_adv_instance_is_scannable(hdev, hdev->cur_adv_instance);
@@ -1548,8 +1099,7 @@ void hci_req_disable_address_resolution(struct hci_dev *hdev)
struct hci_request req;
__u8 enable = 0x00;
- if (!use_ll_privacy(hdev) &&
- !hci_dev_test_flag(hdev, HCI_LL_RPA_RESOLUTION))
+ if (!hci_dev_test_flag(hdev, HCI_LL_RPA_RESOLUTION))
return;
hci_req_init(&req, hdev);
@@ -1692,8 +1242,7 @@ int hci_get_random_address(struct hci_dev *hdev, bool require_privacy,
/* If Controller supports LL Privacy use own address type is
* 0x03
*/
- if (use_ll_privacy(hdev) &&
- hci_dev_test_flag(hdev, HCI_ENABLE_LL_PRIVACY))
+ if (use_ll_privacy(hdev))
*own_addr_type = ADDR_LE_DEV_RANDOM_RESOLVED;
else
*own_addr_type = ADDR_LE_DEV_RANDOM;
@@ -1871,7 +1420,8 @@ int __hci_req_setup_ext_adv_instance(struct hci_request *req, u8 instance)
hci_req_add(req, HCI_OP_LE_SET_EXT_ADV_PARAMS, sizeof(cp), &cp);
- if (own_addr_type == ADDR_LE_DEV_RANDOM &&
+ if ((own_addr_type == ADDR_LE_DEV_RANDOM ||
+ own_addr_type == ADDR_LE_DEV_RANDOM_RESOLVED) &&
bacmp(&random_addr, BDADDR_ANY)) {
struct hci_cp_le_set_adv_set_rand_addr cp;
@@ -2160,8 +1710,7 @@ int hci_update_random_address(struct hci_request *req, bool require_privacy,
/* If Controller supports LL Privacy use own address type is
* 0x03
*/
- if (use_ll_privacy(hdev) &&
- hci_dev_test_flag(hdev, HCI_ENABLE_LL_PRIVACY))
+ if (use_ll_privacy(hdev))
*own_addr_type = ADDR_LE_DEV_RANDOM_RESOLVED;
else
*own_addr_type = ADDR_LE_DEV_RANDOM;
@@ -2301,47 +1850,6 @@ static void scan_update_work(struct work_struct *work)
hci_req_sync(hdev, update_scan, 0, HCI_CMD_TIMEOUT, NULL);
}
-static int connectable_update(struct hci_request *req, unsigned long opt)
-{
- struct hci_dev *hdev = req->hdev;
-
- hci_dev_lock(hdev);
-
- __hci_req_update_scan(req);
-
- /* If BR/EDR is not enabled and we disable advertising as a
- * by-product of disabling connectable, we need to update the
- * advertising flags.
- */
- if (!hci_dev_test_flag(hdev, HCI_BREDR_ENABLED))
- __hci_req_update_adv_data(req, hdev->cur_adv_instance);
-
- /* Update the advertising parameters if necessary */
- if (hci_dev_test_flag(hdev, HCI_ADVERTISING) ||
- !list_empty(&hdev->adv_instances)) {
- if (ext_adv_capable(hdev))
- __hci_req_start_ext_adv(req, hdev->cur_adv_instance);
- else
- __hci_req_enable_advertising(req);
- }
-
- __hci_update_background_scan(req);
-
- hci_dev_unlock(hdev);
-
- return 0;
-}
-
-static void connectable_update_work(struct work_struct *work)
-{
- struct hci_dev *hdev = container_of(work, struct hci_dev,
- connectable_update);
- u8 status;
-
- hci_req_sync(hdev, connectable_update, 0, HCI_CMD_TIMEOUT, &status);
- mgmt_set_connectable_complete(hdev, status);
-}
-
static u8 get_service_classes(struct hci_dev *hdev)
{
struct bt_uuid *uuid;
@@ -2445,16 +1953,6 @@ static int discoverable_update(struct hci_request *req, unsigned long opt)
return 0;
}
-static void discoverable_update_work(struct work_struct *work)
-{
- struct hci_dev *hdev = container_of(work, struct hci_dev,
- discoverable_update);
- u8 status;
-
- hci_req_sync(hdev, discoverable_update, 0, HCI_CMD_TIMEOUT, &status);
- mgmt_set_discoverable_complete(hdev, status);
-}
-
void __hci_abort_conn(struct hci_request *req, struct hci_conn *conn,
u8 reason)
{
@@ -2548,35 +2046,6 @@ int hci_abort_conn(struct hci_conn *conn, u8 reason)
return 0;
}
-static int update_bg_scan(struct hci_request *req, unsigned long opt)
-{
- hci_dev_lock(req->hdev);
- __hci_update_background_scan(req);
- hci_dev_unlock(req->hdev);
- return 0;
-}
-
-static void bg_scan_update(struct work_struct *work)
-{
- struct hci_dev *hdev = container_of(work, struct hci_dev,
- bg_scan_update);
- struct hci_conn *conn;
- u8 status;
- int err;
-
- err = hci_req_sync(hdev, update_bg_scan, 0, HCI_CMD_TIMEOUT, &status);
- if (!err)
- return;
-
- hci_dev_lock(hdev);
-
- conn = hci_conn_hash_lookup_state(hdev, LE_LINK, BT_CONNECT);
- if (conn)
- hci_le_conn_failed(conn, status);
-
- hci_dev_unlock(hdev);
-}
-
static int le_scan_disable(struct hci_request *req, unsigned long opt)
{
hci_req_add_le_scan_disable(req, false);
@@ -3163,10 +2632,7 @@ int __hci_req_hci_power_on(struct hci_dev *hdev)
void hci_request_setup(struct hci_dev *hdev)
{
INIT_WORK(&hdev->discov_update, discov_update);
- INIT_WORK(&hdev->bg_scan_update, bg_scan_update);
INIT_WORK(&hdev->scan_update, scan_update_work);
- INIT_WORK(&hdev->connectable_update, connectable_update_work);
- INIT_WORK(&hdev->discoverable_update, discoverable_update_work);
INIT_DELAYED_WORK(&hdev->discov_off, discov_off);
INIT_DELAYED_WORK(&hdev->le_scan_disable, le_scan_disable_work);
INIT_DELAYED_WORK(&hdev->le_scan_restart, le_scan_restart_work);
@@ -3176,13 +2642,10 @@ void hci_request_setup(struct hci_dev *hdev)
void hci_request_cancel_all(struct hci_dev *hdev)
{
- hci_req_sync_cancel(hdev, ENODEV);
+ __hci_cmd_sync_cancel(hdev, ENODEV);
cancel_work_sync(&hdev->discov_update);
- cancel_work_sync(&hdev->bg_scan_update);
cancel_work_sync(&hdev->scan_update);
- cancel_work_sync(&hdev->connectable_update);
- cancel_work_sync(&hdev->discoverable_update);
cancel_delayed_work_sync(&hdev->discov_off);
cancel_delayed_work_sync(&hdev->le_scan_disable);
cancel_delayed_work_sync(&hdev->le_scan_restart);
diff --git a/net/bluetooth/hci_request.h b/net/bluetooth/hci_request.h
index f31420f..7f8df25 100644
--- a/net/bluetooth/hci_request.h
+++ b/net/bluetooth/hci_request.h
@@ -22,9 +22,17 @@
#include <asm/unaligned.h>
+#define HCI_REQ_DONE 0
+#define HCI_REQ_PEND 1
+#define HCI_REQ_CANCELED 2
+
#define hci_req_sync_lock(hdev) mutex_lock(&hdev->req_lock)
#define hci_req_sync_unlock(hdev) mutex_unlock(&hdev->req_lock)
+#define HCI_REQ_DONE 0
+#define HCI_REQ_PEND 1
+#define HCI_REQ_CANCELED 2
+
struct hci_request {
struct hci_dev *hdev;
struct sk_buff_head cmd_q;
@@ -40,6 +48,8 @@ void hci_req_purge(struct hci_request *req);
bool hci_req_status_pend(struct hci_dev *hdev);
int hci_req_run(struct hci_request *req, hci_req_complete_t complete);
int hci_req_run_skb(struct hci_request *req, hci_req_complete_skb_t complete);
+void hci_req_sync_complete(struct hci_dev *hdev, u8 result, u16 opcode,
+ struct sk_buff *skb);
void hci_req_add(struct hci_request *req, u16 opcode, u32 plen,
const void *param);
void hci_req_add_ev(struct hci_request *req, u16 opcode, u32 plen,
@@ -54,7 +64,6 @@ int hci_req_sync(struct hci_dev *hdev, int (*req)(struct hci_request *req,
int __hci_req_sync(struct hci_dev *hdev, int (*func)(struct hci_request *req,
unsigned long opt),
unsigned long opt, u32 timeout, u8 *hci_status);
-void hci_req_sync_cancel(struct hci_dev *hdev, int err);
struct sk_buff *hci_prepare_cmd(struct hci_dev *hdev, u16 opcode, u32 plen,
const void *param);
@@ -71,8 +80,6 @@ void hci_req_add_le_passive_scan(struct hci_request *req);
void hci_req_prepare_suspend(struct hci_dev *hdev, enum suspended_state next);
void hci_req_disable_address_resolution(struct hci_dev *hdev);
-void __hci_req_pause_adv_instances(struct hci_request *req);
-int hci_req_resume_adv_instances(struct hci_dev *hdev);
void hci_req_reenable_advertising(struct hci_dev *hdev);
void __hci_req_enable_advertising(struct hci_request *req);
void __hci_req_disable_advertising(struct hci_request *req);
@@ -117,10 +124,5 @@ int hci_abort_conn(struct hci_conn *conn, u8 reason);
void __hci_abort_conn(struct hci_request *req, struct hci_conn *conn,
u8 reason);
-static inline void hci_update_background_scan(struct hci_dev *hdev)
-{
- queue_work(hdev->req_workqueue, &hdev->bg_scan_update);
-}
-
void hci_request_setup(struct hci_dev *hdev);
void hci_request_cancel_all(struct hci_dev *hdev);
diff --git a/net/bluetooth/hci_sock.c b/net/bluetooth/hci_sock.c
index d0dad1f..33b3c0f 100644
--- a/net/bluetooth/hci_sock.c
+++ b/net/bluetooth/hci_sock.c
@@ -889,10 +889,6 @@ static int hci_sock_release(struct socket *sock)
}
sock_orphan(sk);
-
- skb_queue_purge(&sk->sk_receive_queue);
- skb_queue_purge(&sk->sk_write_queue);
-
release_sock(sk);
sock_put(sk);
return 0;
@@ -1915,7 +1911,8 @@ static int hci_sock_setsockopt(struct socket *sock, int level, int optname,
sockptr_t optval, unsigned int len)
{
struct sock *sk = sock->sk;
- int err = 0, opt = 0;
+ int err = 0;
+ u16 opt;
BT_DBG("sk %p, opt %d", sk, optname);
@@ -1941,7 +1938,7 @@ static int hci_sock_setsockopt(struct socket *sock, int level, int optname,
goto done;
}
- if (copy_from_sockptr(&opt, optval, sizeof(u16))) {
+ if (copy_from_sockptr(&opt, optval, sizeof(opt))) {
err = -EFAULT;
break;
}
@@ -2058,6 +2055,12 @@ static int hci_sock_getsockopt(struct socket *sock, int level, int optname,
return err;
}
+static void hci_sock_destruct(struct sock *sk)
+{
+ skb_queue_purge(&sk->sk_receive_queue);
+ skb_queue_purge(&sk->sk_write_queue);
+}
+
static const struct proto_ops hci_sock_ops = {
.family = PF_BLUETOOTH,
.owner = THIS_MODULE,
@@ -2111,6 +2114,7 @@ static int hci_sock_create(struct net *net, struct socket *sock, int protocol,
sock->state = SS_UNCONNECTED;
sk->sk_state = BT_OPEN;
+ sk->sk_destruct = hci_sock_destruct;
bt_sock_link(&hci_sk_list, sk);
return 0;
diff --git a/net/bluetooth/hci_sync.c b/net/bluetooth/hci_sync.c
new file mode 100644
index 0000000..0feb68f
--- /dev/null
+++ b/net/bluetooth/hci_sync.c
@@ -0,0 +1,5281 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * BlueZ - Bluetooth protocol stack for Linux
+ *
+ * Copyright (C) 2021 Intel Corporation
+ */
+
+#include <linux/property.h>
+
+#include <net/bluetooth/bluetooth.h>
+#include <net/bluetooth/hci_core.h>
+#include <net/bluetooth/mgmt.h>
+
+#include "hci_request.h"
+#include "hci_debugfs.h"
+#include "smp.h"
+#include "eir.h"
+#include "msft.h"
+#include "aosp.h"
+#include "leds.h"
+
+static void hci_cmd_sync_complete(struct hci_dev *hdev, u8 result, u16 opcode,
+ struct sk_buff *skb)
+{
+ bt_dev_dbg(hdev, "result 0x%2.2x", result);
+
+ if (hdev->req_status != HCI_REQ_PEND)
+ return;
+
+ hdev->req_result = result;
+ hdev->req_status = HCI_REQ_DONE;
+
+ if (skb) {
+ struct sock *sk = hci_skb_sk(skb);
+
+ /* Drop sk reference if set */
+ if (sk)
+ sock_put(sk);
+
+ hdev->req_skb = skb_get(skb);
+ }
+
+ wake_up_interruptible(&hdev->req_wait_q);
+}
+
+static struct sk_buff *hci_cmd_sync_alloc(struct hci_dev *hdev, u16 opcode,
+ u32 plen, const void *param,
+ struct sock *sk)
+{
+ int len = HCI_COMMAND_HDR_SIZE + plen;
+ struct hci_command_hdr *hdr;
+ struct sk_buff *skb;
+
+ skb = bt_skb_alloc(len, GFP_ATOMIC);
+ if (!skb)
+ return NULL;
+
+ hdr = skb_put(skb, HCI_COMMAND_HDR_SIZE);
+ hdr->opcode = cpu_to_le16(opcode);
+ hdr->plen = plen;
+
+ if (plen)
+ skb_put_data(skb, param, plen);
+
+ bt_dev_dbg(hdev, "skb len %d", skb->len);
+
+ hci_skb_pkt_type(skb) = HCI_COMMAND_PKT;
+ hci_skb_opcode(skb) = opcode;
+
+ /* Grab a reference if command needs to be associated with a sock (e.g.
+ * likely mgmt socket that initiated the command).
+ */
+ if (sk) {
+ hci_skb_sk(skb) = sk;
+ sock_hold(sk);
+ }
+
+ return skb;
+}
+
+static void hci_cmd_sync_add(struct hci_request *req, u16 opcode, u32 plen,
+ const void *param, u8 event, struct sock *sk)
+{
+ struct hci_dev *hdev = req->hdev;
+ struct sk_buff *skb;
+
+ bt_dev_dbg(hdev, "opcode 0x%4.4x plen %d", opcode, plen);
+
+ /* If an error occurred during request building, there is no point in
+ * queueing the HCI command. We can simply return.
+ */
+ if (req->err)
+ return;
+
+ skb = hci_cmd_sync_alloc(hdev, opcode, plen, param, sk);
+ if (!skb) {
+ bt_dev_err(hdev, "no memory for command (opcode 0x%4.4x)",
+ opcode);
+ req->err = -ENOMEM;
+ return;
+ }
+
+ if (skb_queue_empty(&req->cmd_q))
+ bt_cb(skb)->hci.req_flags |= HCI_REQ_START;
+
+ hci_skb_event(skb) = event;
+
+ skb_queue_tail(&req->cmd_q, skb);
+}
+
+static int hci_cmd_sync_run(struct hci_request *req)
+{
+ struct hci_dev *hdev = req->hdev;
+ struct sk_buff *skb;
+ unsigned long flags;
+
+ bt_dev_dbg(hdev, "length %u", skb_queue_len(&req->cmd_q));
+
+ /* If an error occurred during request building, remove all HCI
+ * commands queued on the HCI request queue.
+ */
+ if (req->err) {
+ skb_queue_purge(&req->cmd_q);
+ return req->err;
+ }
+
+ /* Do not allow empty requests */
+ if (skb_queue_empty(&req->cmd_q))
+ return -ENODATA;
+
+ skb = skb_peek_tail(&req->cmd_q);
+ bt_cb(skb)->hci.req_complete_skb = hci_cmd_sync_complete;
+ bt_cb(skb)->hci.req_flags |= HCI_REQ_SKB;
+
+ spin_lock_irqsave(&hdev->cmd_q.lock, flags);
+ skb_queue_splice_tail(&req->cmd_q, &hdev->cmd_q);
+ spin_unlock_irqrestore(&hdev->cmd_q.lock, flags);
+
+ queue_work(hdev->workqueue, &hdev->cmd_work);
+
+ return 0;
+}
+
+/* This function requires the caller holds hdev->req_lock. */
+struct sk_buff *__hci_cmd_sync_sk(struct hci_dev *hdev, u16 opcode, u32 plen,
+ const void *param, u8 event, u32 timeout,
+ struct sock *sk)
+{
+ struct hci_request req;
+ struct sk_buff *skb;
+ int err = 0;
+
+ bt_dev_dbg(hdev, "Opcode 0x%4x", opcode);
+
+ hci_req_init(&req, hdev);
+
+ hci_cmd_sync_add(&req, opcode, plen, param, event, sk);
+
+ hdev->req_status = HCI_REQ_PEND;
+
+ err = hci_cmd_sync_run(&req);
+ if (err < 0)
+ return ERR_PTR(err);
+
+ err = wait_event_interruptible_timeout(hdev->req_wait_q,
+ hdev->req_status != HCI_REQ_PEND,
+ timeout);
+
+ if (err == -ERESTARTSYS)
+ return ERR_PTR(-EINTR);
+
+ switch (hdev->req_status) {
+ case HCI_REQ_DONE:
+ err = -bt_to_errno(hdev->req_result);
+ break;
+
+ case HCI_REQ_CANCELED:
+ err = -hdev->req_result;
+ break;
+
+ default:
+ err = -ETIMEDOUT;
+ break;
+ }
+
+ hdev->req_status = 0;
+ hdev->req_result = 0;
+ skb = hdev->req_skb;
+ hdev->req_skb = NULL;
+
+ bt_dev_dbg(hdev, "end: err %d", err);
+
+ if (err < 0) {
+ kfree_skb(skb);
+ return ERR_PTR(err);
+ }
+
+ return skb;
+}
+EXPORT_SYMBOL(__hci_cmd_sync_sk);
+
+/* This function requires the caller holds hdev->req_lock. */
+struct sk_buff *__hci_cmd_sync(struct hci_dev *hdev, u16 opcode, u32 plen,
+ const void *param, u32 timeout)
+{
+ return __hci_cmd_sync_sk(hdev, opcode, plen, param, 0, timeout, NULL);
+}
+EXPORT_SYMBOL(__hci_cmd_sync);
+
+/* Send HCI command and wait for command complete event */
+struct sk_buff *hci_cmd_sync(struct hci_dev *hdev, u16 opcode, u32 plen,
+ const void *param, u32 timeout)
+{
+ struct sk_buff *skb;
+
+ if (!test_bit(HCI_UP, &hdev->flags))
+ return ERR_PTR(-ENETDOWN);
+
+ bt_dev_dbg(hdev, "opcode 0x%4.4x plen %d", opcode, plen);
+
+ hci_req_sync_lock(hdev);
+ skb = __hci_cmd_sync(hdev, opcode, plen, param, timeout);
+ hci_req_sync_unlock(hdev);
+
+ return skb;
+}
+EXPORT_SYMBOL(hci_cmd_sync);
+
+/* This function requires the caller holds hdev->req_lock. */
+struct sk_buff *__hci_cmd_sync_ev(struct hci_dev *hdev, u16 opcode, u32 plen,
+ const void *param, u8 event, u32 timeout)
+{
+ return __hci_cmd_sync_sk(hdev, opcode, plen, param, event, timeout,
+ NULL);
+}
+EXPORT_SYMBOL(__hci_cmd_sync_ev);
+
+/* This function requires the caller holds hdev->req_lock. */
+int __hci_cmd_sync_status_sk(struct hci_dev *hdev, u16 opcode, u32 plen,
+ const void *param, u8 event, u32 timeout,
+ struct sock *sk)
+{
+ struct sk_buff *skb;
+ u8 status;
+
+ skb = __hci_cmd_sync_sk(hdev, opcode, plen, param, event, timeout, sk);
+ if (IS_ERR(skb)) {
+ bt_dev_err(hdev, "Opcode 0x%4x failed: %ld", opcode,
+ PTR_ERR(skb));
+ return PTR_ERR(skb);
+ }
+
+ /* If command return a status event skb will be set to NULL as there are
+ * no parameters, in case of failure IS_ERR(skb) would have be set to
+ * the actual error would be found with PTR_ERR(skb).
+ */
+ if (!skb)
+ return 0;
+
+ status = skb->data[0];
+
+ kfree_skb(skb);
+
+ return status;
+}
+EXPORT_SYMBOL(__hci_cmd_sync_status_sk);
+
+int __hci_cmd_sync_status(struct hci_dev *hdev, u16 opcode, u32 plen,
+ const void *param, u32 timeout)
+{
+ return __hci_cmd_sync_status_sk(hdev, opcode, plen, param, 0, timeout,
+ NULL);
+}
+EXPORT_SYMBOL(__hci_cmd_sync_status);
+
+static void hci_cmd_sync_work(struct work_struct *work)
+{
+ struct hci_dev *hdev = container_of(work, struct hci_dev, cmd_sync_work);
+ struct hci_cmd_sync_work_entry *entry;
+ hci_cmd_sync_work_func_t func;
+ hci_cmd_sync_work_destroy_t destroy;
+ void *data;
+
+ bt_dev_dbg(hdev, "");
+
+ mutex_lock(&hdev->cmd_sync_work_lock);
+ entry = list_first_entry(&hdev->cmd_sync_work_list,
+ struct hci_cmd_sync_work_entry, list);
+ if (entry) {
+ list_del(&entry->list);
+ func = entry->func;
+ data = entry->data;
+ destroy = entry->destroy;
+ kfree(entry);
+ } else {
+ func = NULL;
+ data = NULL;
+ destroy = NULL;
+ }
+ mutex_unlock(&hdev->cmd_sync_work_lock);
+
+ if (func) {
+ int err;
+
+ hci_req_sync_lock(hdev);
+
+ err = func(hdev, data);
+
+ if (destroy)
+ destroy(hdev, data, err);
+
+ hci_req_sync_unlock(hdev);
+ }
+}
+
+static void hci_cmd_sync_cancel_work(struct work_struct *work)
+{
+ struct hci_dev *hdev = container_of(work, struct hci_dev, cmd_sync_cancel_work);
+
+ cancel_delayed_work_sync(&hdev->cmd_timer);
+ cancel_delayed_work_sync(&hdev->ncmd_timer);
+ atomic_set(&hdev->cmd_cnt, 1);
+
+ wake_up_interruptible(&hdev->req_wait_q);
+}
+
+void hci_cmd_sync_init(struct hci_dev *hdev)
+{
+ INIT_WORK(&hdev->cmd_sync_work, hci_cmd_sync_work);
+ INIT_LIST_HEAD(&hdev->cmd_sync_work_list);
+ mutex_init(&hdev->cmd_sync_work_lock);
+
+ INIT_WORK(&hdev->cmd_sync_cancel_work, hci_cmd_sync_cancel_work);
+}
+
+void hci_cmd_sync_clear(struct hci_dev *hdev)
+{
+ struct hci_cmd_sync_work_entry *entry, *tmp;
+
+ cancel_work_sync(&hdev->cmd_sync_work);
+
+ list_for_each_entry_safe(entry, tmp, &hdev->cmd_sync_work_list, list) {
+ if (entry->destroy)
+ entry->destroy(hdev, entry->data, -ECANCELED);
+
+ list_del(&entry->list);
+ kfree(entry);
+ }
+}
+
+void __hci_cmd_sync_cancel(struct hci_dev *hdev, int err)
+{
+ bt_dev_dbg(hdev, "err 0x%2.2x", err);
+
+ if (hdev->req_status == HCI_REQ_PEND) {
+ hdev->req_result = err;
+ hdev->req_status = HCI_REQ_CANCELED;
+
+ cancel_delayed_work_sync(&hdev->cmd_timer);
+ cancel_delayed_work_sync(&hdev->ncmd_timer);
+ atomic_set(&hdev->cmd_cnt, 1);
+
+ wake_up_interruptible(&hdev->req_wait_q);
+ }
+}
+
+void hci_cmd_sync_cancel(struct hci_dev *hdev, int err)
+{
+ bt_dev_dbg(hdev, "err 0x%2.2x", err);
+
+ if (hdev->req_status == HCI_REQ_PEND) {
+ hdev->req_result = err;
+ hdev->req_status = HCI_REQ_CANCELED;
+
+ queue_work(hdev->workqueue, &hdev->cmd_sync_cancel_work);
+ }
+}
+EXPORT_SYMBOL(hci_cmd_sync_cancel);
+
+int hci_cmd_sync_queue(struct hci_dev *hdev, hci_cmd_sync_work_func_t func,
+ void *data, hci_cmd_sync_work_destroy_t destroy)
+{
+ struct hci_cmd_sync_work_entry *entry;
+
+ entry = kmalloc(sizeof(*entry), GFP_KERNEL);
+ if (!entry)
+ return -ENOMEM;
+
+ entry->func = func;
+ entry->data = data;
+ entry->destroy = destroy;
+
+ mutex_lock(&hdev->cmd_sync_work_lock);
+ list_add_tail(&entry->list, &hdev->cmd_sync_work_list);
+ mutex_unlock(&hdev->cmd_sync_work_lock);
+
+ queue_work(hdev->req_workqueue, &hdev->cmd_sync_work);
+
+ return 0;
+}
+EXPORT_SYMBOL(hci_cmd_sync_queue);
+
+int hci_update_eir_sync(struct hci_dev *hdev)
+{
+ struct hci_cp_write_eir cp;
+
+ bt_dev_dbg(hdev, "");
+
+ if (!hdev_is_powered(hdev))
+ return 0;
+
+ if (!lmp_ext_inq_capable(hdev))
+ return 0;
+
+ if (!hci_dev_test_flag(hdev, HCI_SSP_ENABLED))
+ return 0;
+
+ if (hci_dev_test_flag(hdev, HCI_SERVICE_CACHE))
+ return 0;
+
+ memset(&cp, 0, sizeof(cp));
+
+ eir_create(hdev, cp.data);
+
+ if (memcmp(cp.data, hdev->eir, sizeof(cp.data)) == 0)
+ return 0;
+
+ memcpy(hdev->eir, cp.data, sizeof(cp.data));
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_WRITE_EIR, sizeof(cp), &cp,
+ HCI_CMD_TIMEOUT);
+}
+
+static u8 get_service_classes(struct hci_dev *hdev)
+{
+ struct bt_uuid *uuid;
+ u8 val = 0;
+
+ list_for_each_entry(uuid, &hdev->uuids, list)
+ val |= uuid->svc_hint;
+
+ return val;
+}
+
+int hci_update_class_sync(struct hci_dev *hdev)
+{
+ u8 cod[3];
+
+ bt_dev_dbg(hdev, "");
+
+ if (!hdev_is_powered(hdev))
+ return 0;
+
+ if (!hci_dev_test_flag(hdev, HCI_BREDR_ENABLED))
+ return 0;
+
+ if (hci_dev_test_flag(hdev, HCI_SERVICE_CACHE))
+ return 0;
+
+ cod[0] = hdev->minor_class;
+ cod[1] = hdev->major_class;
+ cod[2] = get_service_classes(hdev);
+
+ if (hci_dev_test_flag(hdev, HCI_LIMITED_DISCOVERABLE))
+ cod[1] |= 0x20;
+
+ if (memcmp(cod, hdev->dev_class, 3) == 0)
+ return 0;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_WRITE_CLASS_OF_DEV,
+ sizeof(cod), cod, HCI_CMD_TIMEOUT);
+}
+
+static bool is_advertising_allowed(struct hci_dev *hdev, bool connectable)
+{
+ /* If there is no connection we are OK to advertise. */
+ if (hci_conn_num(hdev, LE_LINK) == 0)
+ return true;
+
+ /* Check le_states if there is any connection in peripheral role. */
+ if (hdev->conn_hash.le_num_peripheral > 0) {
+ /* Peripheral connection state and non connectable mode
+ * bit 20.
+ */
+ if (!connectable && !(hdev->le_states[2] & 0x10))
+ return false;
+
+ /* Peripheral connection state and connectable mode bit 38
+ * and scannable bit 21.
+ */
+ if (connectable && (!(hdev->le_states[4] & 0x40) ||
+ !(hdev->le_states[2] & 0x20)))
+ return false;
+ }
+
+ /* Check le_states if there is any connection in central role. */
+ if (hci_conn_num(hdev, LE_LINK) != hdev->conn_hash.le_num_peripheral) {
+ /* Central connection state and non connectable mode bit 18. */
+ if (!connectable && !(hdev->le_states[2] & 0x02))
+ return false;
+
+ /* Central connection state and connectable mode bit 35 and
+ * scannable 19.
+ */
+ if (connectable && (!(hdev->le_states[4] & 0x08) ||
+ !(hdev->le_states[2] & 0x08)))
+ return false;
+ }
+
+ return true;
+}
+
+static bool adv_use_rpa(struct hci_dev *hdev, uint32_t flags)
+{
+ /* If privacy is not enabled don't use RPA */
+ if (!hci_dev_test_flag(hdev, HCI_PRIVACY))
+ return false;
+
+ /* If basic privacy mode is enabled use RPA */
+ if (!hci_dev_test_flag(hdev, HCI_LIMITED_PRIVACY))
+ return true;
+
+ /* If limited privacy mode is enabled don't use RPA if we're
+ * both discoverable and bondable.
+ */
+ if ((flags & MGMT_ADV_FLAG_DISCOV) &&
+ hci_dev_test_flag(hdev, HCI_BONDABLE))
+ return false;
+
+ /* We're neither bondable nor discoverable in the limited
+ * privacy mode, therefore use RPA.
+ */
+ return true;
+}
+
+static int hci_set_random_addr_sync(struct hci_dev *hdev, bdaddr_t *rpa)
+{
+ /* If we're advertising or initiating an LE connection we can't
+ * go ahead and change the random address at this time. This is
+ * because the eventual initiator address used for the
+ * subsequently created connection will be undefined (some
+ * controllers use the new address and others the one we had
+ * when the operation started).
+ *
+ * In this kind of scenario skip the update and let the random
+ * address be updated at the next cycle.
+ */
+ if (hci_dev_test_flag(hdev, HCI_LE_ADV) ||
+ hci_lookup_le_connect(hdev)) {
+ bt_dev_dbg(hdev, "Deferring random address update");
+ hci_dev_set_flag(hdev, HCI_RPA_EXPIRED);
+ return 0;
+ }
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_LE_SET_RANDOM_ADDR,
+ 6, rpa, HCI_CMD_TIMEOUT);
+}
+
+int hci_update_random_address_sync(struct hci_dev *hdev, bool require_privacy,
+ bool rpa, u8 *own_addr_type)
+{
+ int err;
+
+ /* If privacy is enabled use a resolvable private address. If
+ * current RPA has expired or there is something else than
+ * the current RPA in use, then generate a new one.
+ */
+ if (rpa) {
+ /* If Controller supports LL Privacy use own address type is
+ * 0x03
+ */
+ if (use_ll_privacy(hdev))
+ *own_addr_type = ADDR_LE_DEV_RANDOM_RESOLVED;
+ else
+ *own_addr_type = ADDR_LE_DEV_RANDOM;
+
+ /* Check if RPA is valid */
+ if (rpa_valid(hdev))
+ return 0;
+
+ err = smp_generate_rpa(hdev, hdev->irk, &hdev->rpa);
+ if (err < 0) {
+ bt_dev_err(hdev, "failed to generate new RPA");
+ return err;
+ }
+
+ err = hci_set_random_addr_sync(hdev, &hdev->rpa);
+ if (err)
+ return err;
+
+ return 0;
+ }
+
+ /* In case of required privacy without resolvable private address,
+ * use an non-resolvable private address. This is useful for active
+ * scanning and non-connectable advertising.
+ */
+ if (require_privacy) {
+ bdaddr_t nrpa;
+
+ while (true) {
+ /* The non-resolvable private address is generated
+ * from random six bytes with the two most significant
+ * bits cleared.
+ */
+ get_random_bytes(&nrpa, 6);
+ nrpa.b[5] &= 0x3f;
+
+ /* The non-resolvable private address shall not be
+ * equal to the public address.
+ */
+ if (bacmp(&hdev->bdaddr, &nrpa))
+ break;
+ }
+
+ *own_addr_type = ADDR_LE_DEV_RANDOM;
+
+ return hci_set_random_addr_sync(hdev, &nrpa);
+ }
+
+ /* If forcing static address is in use or there is no public
+ * address use the static address as random address (but skip
+ * the HCI command if the current random address is already the
+ * static one.
+ *
+ * In case BR/EDR has been disabled on a dual-mode controller
+ * and a static address has been configured, then use that
+ * address instead of the public BR/EDR address.
+ */
+ if (hci_dev_test_flag(hdev, HCI_FORCE_STATIC_ADDR) ||
+ !bacmp(&hdev->bdaddr, BDADDR_ANY) ||
+ (!hci_dev_test_flag(hdev, HCI_BREDR_ENABLED) &&
+ bacmp(&hdev->static_addr, BDADDR_ANY))) {
+ *own_addr_type = ADDR_LE_DEV_RANDOM;
+ if (bacmp(&hdev->static_addr, &hdev->random_addr))
+ return hci_set_random_addr_sync(hdev,
+ &hdev->static_addr);
+ return 0;
+ }
+
+ /* Neither privacy nor static address is being used so use a
+ * public address.
+ */
+ *own_addr_type = ADDR_LE_DEV_PUBLIC;
+
+ return 0;
+}
+
+static int hci_disable_ext_adv_instance_sync(struct hci_dev *hdev, u8 instance)
+{
+ struct hci_cp_le_set_ext_adv_enable *cp;
+ struct hci_cp_ext_adv_set *set;
+ u8 data[sizeof(*cp) + sizeof(*set) * 1];
+ u8 size;
+
+ /* If request specifies an instance that doesn't exist, fail */
+ if (instance > 0) {
+ struct adv_info *adv;
+
+ adv = hci_find_adv_instance(hdev, instance);
+ if (!adv)
+ return -EINVAL;
+
+ /* If not enabled there is nothing to do */
+ if (!adv->enabled)
+ return 0;
+ }
+
+ memset(data, 0, sizeof(data));
+
+ cp = (void *)data;
+ set = (void *)cp->data;
+
+ /* Instance 0x00 indicates all advertising instances will be disabled */
+ cp->num_of_sets = !!instance;
+ cp->enable = 0x00;
+
+ set->handle = instance;
+
+ size = sizeof(*cp) + sizeof(*set) * cp->num_of_sets;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_LE_SET_EXT_ADV_ENABLE,
+ size, data, HCI_CMD_TIMEOUT);
+}
+
+static int hci_set_adv_set_random_addr_sync(struct hci_dev *hdev, u8 instance,
+ bdaddr_t *random_addr)
+{
+ struct hci_cp_le_set_adv_set_rand_addr cp;
+ int err;
+
+ if (!instance) {
+ /* Instance 0x00 doesn't have an adv_info, instead it uses
+ * hdev->random_addr to track its address so whenever it needs
+ * to be updated this also set the random address since
+ * hdev->random_addr is shared with scan state machine.
+ */
+ err = hci_set_random_addr_sync(hdev, random_addr);
+ if (err)
+ return err;
+ }
+
+ memset(&cp, 0, sizeof(cp));
+
+ cp.handle = instance;
+ bacpy(&cp.bdaddr, random_addr);
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_LE_SET_ADV_SET_RAND_ADDR,
+ sizeof(cp), &cp, HCI_CMD_TIMEOUT);
+}
+
+int hci_setup_ext_adv_instance_sync(struct hci_dev *hdev, u8 instance)
+{
+ struct hci_cp_le_set_ext_adv_params cp;
+ bool connectable;
+ u32 flags;
+ bdaddr_t random_addr;
+ u8 own_addr_type;
+ int err;
+ struct adv_info *adv;
+ bool secondary_adv;
+
+ if (instance > 0) {
+ adv = hci_find_adv_instance(hdev, instance);
+ if (!adv)
+ return -EINVAL;
+ } else {
+ adv = NULL;
+ }
+
+ /* Updating parameters of an active instance will return a
+ * Command Disallowed error, so we must first disable the
+ * instance if it is active.
+ */
+ if (adv && !adv->pending) {
+ err = hci_disable_ext_adv_instance_sync(hdev, instance);
+ if (err)
+ return err;
+ }
+
+ flags = hci_adv_instance_flags(hdev, instance);
+
+ /* If the "connectable" instance flag was not set, then choose between
+ * ADV_IND and ADV_NONCONN_IND based on the global connectable setting.
+ */
+ connectable = (flags & MGMT_ADV_FLAG_CONNECTABLE) ||
+ mgmt_get_connectable(hdev);
+
+ if (!is_advertising_allowed(hdev, connectable))
+ return -EPERM;
+
+ /* Set require_privacy to true only when non-connectable
+ * advertising is used. In that case it is fine to use a
+ * non-resolvable private address.
+ */
+ err = hci_get_random_address(hdev, !connectable,
+ adv_use_rpa(hdev, flags), adv,
+ &own_addr_type, &random_addr);
+ if (err < 0)
+ return err;
+
+ memset(&cp, 0, sizeof(cp));
+
+ if (adv) {
+ hci_cpu_to_le24(adv->min_interval, cp.min_interval);
+ hci_cpu_to_le24(adv->max_interval, cp.max_interval);
+ cp.tx_power = adv->tx_power;
+ } else {
+ hci_cpu_to_le24(hdev->le_adv_min_interval, cp.min_interval);
+ hci_cpu_to_le24(hdev->le_adv_max_interval, cp.max_interval);
+ cp.tx_power = HCI_ADV_TX_POWER_NO_PREFERENCE;
+ }
+
+ secondary_adv = (flags & MGMT_ADV_FLAG_SEC_MASK);
+
+ if (connectable) {
+ if (secondary_adv)
+ cp.evt_properties = cpu_to_le16(LE_EXT_ADV_CONN_IND);
+ else
+ cp.evt_properties = cpu_to_le16(LE_LEGACY_ADV_IND);
+ } else if (hci_adv_instance_is_scannable(hdev, instance) ||
+ (flags & MGMT_ADV_PARAM_SCAN_RSP)) {
+ if (secondary_adv)
+ cp.evt_properties = cpu_to_le16(LE_EXT_ADV_SCAN_IND);
+ else
+ cp.evt_properties = cpu_to_le16(LE_LEGACY_ADV_SCAN_IND);
+ } else {
+ if (secondary_adv)
+ cp.evt_properties = cpu_to_le16(LE_EXT_ADV_NON_CONN_IND);
+ else
+ cp.evt_properties = cpu_to_le16(LE_LEGACY_NONCONN_IND);
+ }
+
+ /* If Own_Address_Type equals 0x02 or 0x03, the Peer_Address parameter
+ * contains the peer’s Identity Address and the Peer_Address_Type
+ * parameter contains the peer’s Identity Type (i.e., 0x00 or 0x01).
+ * These parameters are used to locate the corresponding local IRK in
+ * the resolving list; this IRK is used to generate their own address
+ * used in the advertisement.
+ */
+ if (own_addr_type == ADDR_LE_DEV_RANDOM_RESOLVED)
+ hci_copy_identity_address(hdev, &cp.peer_addr,
+ &cp.peer_addr_type);
+
+ cp.own_addr_type = own_addr_type;
+ cp.channel_map = hdev->le_adv_channel_map;
+ cp.handle = instance;
+
+ if (flags & MGMT_ADV_FLAG_SEC_2M) {
+ cp.primary_phy = HCI_ADV_PHY_1M;
+ cp.secondary_phy = HCI_ADV_PHY_2M;
+ } else if (flags & MGMT_ADV_FLAG_SEC_CODED) {
+ cp.primary_phy = HCI_ADV_PHY_CODED;
+ cp.secondary_phy = HCI_ADV_PHY_CODED;
+ } else {
+ /* In all other cases use 1M */
+ cp.primary_phy = HCI_ADV_PHY_1M;
+ cp.secondary_phy = HCI_ADV_PHY_1M;
+ }
+
+ err = __hci_cmd_sync_status(hdev, HCI_OP_LE_SET_EXT_ADV_PARAMS,
+ sizeof(cp), &cp, HCI_CMD_TIMEOUT);
+ if (err)
+ return err;
+
+ if ((own_addr_type == ADDR_LE_DEV_RANDOM ||
+ own_addr_type == ADDR_LE_DEV_RANDOM_RESOLVED) &&
+ bacmp(&random_addr, BDADDR_ANY)) {
+ /* Check if random address need to be updated */
+ if (adv) {
+ if (!bacmp(&random_addr, &adv->random_addr))
+ return 0;
+ } else {
+ if (!bacmp(&random_addr, &hdev->random_addr))
+ return 0;
+ }
+
+ return hci_set_adv_set_random_addr_sync(hdev, instance,
+ &random_addr);
+ }
+
+ return 0;
+}
+
+static int hci_set_ext_scan_rsp_data_sync(struct hci_dev *hdev, u8 instance)
+{
+ struct {
+ struct hci_cp_le_set_ext_scan_rsp_data cp;
+ u8 data[HCI_MAX_EXT_AD_LENGTH];
+ } pdu;
+ u8 len;
+
+ memset(&pdu, 0, sizeof(pdu));
+
+ len = eir_create_scan_rsp(hdev, instance, pdu.data);
+
+ if (hdev->scan_rsp_data_len == len &&
+ !memcmp(pdu.data, hdev->scan_rsp_data, len))
+ return 0;
+
+ memcpy(hdev->scan_rsp_data, pdu.data, len);
+ hdev->scan_rsp_data_len = len;
+
+ pdu.cp.handle = instance;
+ pdu.cp.length = len;
+ pdu.cp.operation = LE_SET_ADV_DATA_OP_COMPLETE;
+ pdu.cp.frag_pref = LE_SET_ADV_DATA_NO_FRAG;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_LE_SET_EXT_SCAN_RSP_DATA,
+ sizeof(pdu.cp) + len, &pdu.cp,
+ HCI_CMD_TIMEOUT);
+}
+
+static int __hci_set_scan_rsp_data_sync(struct hci_dev *hdev, u8 instance)
+{
+ struct hci_cp_le_set_scan_rsp_data cp;
+ u8 len;
+
+ memset(&cp, 0, sizeof(cp));
+
+ len = eir_create_scan_rsp(hdev, instance, cp.data);
+
+ if (hdev->scan_rsp_data_len == len &&
+ !memcmp(cp.data, hdev->scan_rsp_data, len))
+ return 0;
+
+ memcpy(hdev->scan_rsp_data, cp.data, sizeof(cp.data));
+ hdev->scan_rsp_data_len = len;
+
+ cp.length = len;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_LE_SET_SCAN_RSP_DATA,
+ sizeof(cp), &cp, HCI_CMD_TIMEOUT);
+}
+
+int hci_update_scan_rsp_data_sync(struct hci_dev *hdev, u8 instance)
+{
+ if (!hci_dev_test_flag(hdev, HCI_LE_ENABLED))
+ return 0;
+
+ if (ext_adv_capable(hdev))
+ return hci_set_ext_scan_rsp_data_sync(hdev, instance);
+
+ return __hci_set_scan_rsp_data_sync(hdev, instance);
+}
+
+int hci_enable_ext_advertising_sync(struct hci_dev *hdev, u8 instance)
+{
+ struct hci_cp_le_set_ext_adv_enable *cp;
+ struct hci_cp_ext_adv_set *set;
+ u8 data[sizeof(*cp) + sizeof(*set) * 1];
+ struct adv_info *adv;
+
+ if (instance > 0) {
+ adv = hci_find_adv_instance(hdev, instance);
+ if (!adv)
+ return -EINVAL;
+ /* If already enabled there is nothing to do */
+ if (adv->enabled)
+ return 0;
+ } else {
+ adv = NULL;
+ }
+
+ cp = (void *)data;
+ set = (void *)cp->data;
+
+ memset(cp, 0, sizeof(*cp));
+
+ cp->enable = 0x01;
+ cp->num_of_sets = 0x01;
+
+ memset(set, 0, sizeof(*set));
+
+ set->handle = instance;
+
+ /* Set duration per instance since controller is responsible for
+ * scheduling it.
+ */
+ if (adv && adv->timeout) {
+ u16 duration = adv->timeout * MSEC_PER_SEC;
+
+ /* Time = N * 10 ms */
+ set->duration = cpu_to_le16(duration / 10);
+ }
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_LE_SET_EXT_ADV_ENABLE,
+ sizeof(*cp) +
+ sizeof(*set) * cp->num_of_sets,
+ data, HCI_CMD_TIMEOUT);
+}
+
+int hci_start_ext_adv_sync(struct hci_dev *hdev, u8 instance)
+{
+ int err;
+
+ err = hci_setup_ext_adv_instance_sync(hdev, instance);
+ if (err)
+ return err;
+
+ err = hci_set_ext_scan_rsp_data_sync(hdev, instance);
+ if (err)
+ return err;
+
+ return hci_enable_ext_advertising_sync(hdev, instance);
+}
+
+static int hci_start_adv_sync(struct hci_dev *hdev, u8 instance)
+{
+ int err;
+
+ if (ext_adv_capable(hdev))
+ return hci_start_ext_adv_sync(hdev, instance);
+
+ err = hci_update_adv_data_sync(hdev, instance);
+ if (err)
+ return err;
+
+ err = hci_update_scan_rsp_data_sync(hdev, instance);
+ if (err)
+ return err;
+
+ return hci_enable_advertising_sync(hdev);
+}
+
+int hci_enable_advertising_sync(struct hci_dev *hdev)
+{
+ struct adv_info *adv_instance;
+ struct hci_cp_le_set_adv_param cp;
+ u8 own_addr_type, enable = 0x01;
+ bool connectable;
+ u16 adv_min_interval, adv_max_interval;
+ u32 flags;
+ u8 status;
+
+ if (ext_adv_capable(hdev))
+ return hci_enable_ext_advertising_sync(hdev,
+ hdev->cur_adv_instance);
+
+ flags = hci_adv_instance_flags(hdev, hdev->cur_adv_instance);
+ adv_instance = hci_find_adv_instance(hdev, hdev->cur_adv_instance);
+
+ /* If the "connectable" instance flag was not set, then choose between
+ * ADV_IND and ADV_NONCONN_IND based on the global connectable setting.
+ */
+ connectable = (flags & MGMT_ADV_FLAG_CONNECTABLE) ||
+ mgmt_get_connectable(hdev);
+
+ if (!is_advertising_allowed(hdev, connectable))
+ return -EINVAL;
+
+ status = hci_disable_advertising_sync(hdev);
+ if (status)
+ return status;
+
+ /* Clear the HCI_LE_ADV bit temporarily so that the
+ * hci_update_random_address knows that it's safe to go ahead
+ * and write a new random address. The flag will be set back on
+ * as soon as the SET_ADV_ENABLE HCI command completes.
+ */
+ hci_dev_clear_flag(hdev, HCI_LE_ADV);
+
+ /* Set require_privacy to true only when non-connectable
+ * advertising is used. In that case it is fine to use a
+ * non-resolvable private address.
+ */
+ status = hci_update_random_address_sync(hdev, !connectable,
+ adv_use_rpa(hdev, flags),
+ &own_addr_type);
+ if (status)
+ return status;
+
+ memset(&cp, 0, sizeof(cp));
+
+ if (adv_instance) {
+ adv_min_interval = adv_instance->min_interval;
+ adv_max_interval = adv_instance->max_interval;
+ } else {
+ adv_min_interval = hdev->le_adv_min_interval;
+ adv_max_interval = hdev->le_adv_max_interval;
+ }
+
+ if (connectable) {
+ cp.type = LE_ADV_IND;
+ } else {
+ if (hci_adv_instance_is_scannable(hdev, hdev->cur_adv_instance))
+ cp.type = LE_ADV_SCAN_IND;
+ else
+ cp.type = LE_ADV_NONCONN_IND;
+
+ if (!hci_dev_test_flag(hdev, HCI_DISCOVERABLE) ||
+ hci_dev_test_flag(hdev, HCI_LIMITED_DISCOVERABLE)) {
+ adv_min_interval = DISCOV_LE_FAST_ADV_INT_MIN;
+ adv_max_interval = DISCOV_LE_FAST_ADV_INT_MAX;
+ }
+ }
+
+ cp.min_interval = cpu_to_le16(adv_min_interval);
+ cp.max_interval = cpu_to_le16(adv_max_interval);
+ cp.own_address_type = own_addr_type;
+ cp.channel_map = hdev->le_adv_channel_map;
+
+ status = __hci_cmd_sync_status(hdev, HCI_OP_LE_SET_ADV_PARAM,
+ sizeof(cp), &cp, HCI_CMD_TIMEOUT);
+ if (status)
+ return status;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_LE_SET_ADV_ENABLE,
+ sizeof(enable), &enable, HCI_CMD_TIMEOUT);
+}
+
+static int enable_advertising_sync(struct hci_dev *hdev, void *data)
+{
+ return hci_enable_advertising_sync(hdev);
+}
+
+int hci_enable_advertising(struct hci_dev *hdev)
+{
+ if (!hci_dev_test_flag(hdev, HCI_ADVERTISING) &&
+ list_empty(&hdev->adv_instances))
+ return 0;
+
+ return hci_cmd_sync_queue(hdev, enable_advertising_sync, NULL, NULL);
+}
+
+int hci_remove_ext_adv_instance_sync(struct hci_dev *hdev, u8 instance,
+ struct sock *sk)
+{
+ int err;
+
+ if (!ext_adv_capable(hdev))
+ return 0;
+
+ err = hci_disable_ext_adv_instance_sync(hdev, instance);
+ if (err)
+ return err;
+
+ /* If request specifies an instance that doesn't exist, fail */
+ if (instance > 0 && !hci_find_adv_instance(hdev, instance))
+ return -EINVAL;
+
+ return __hci_cmd_sync_status_sk(hdev, HCI_OP_LE_REMOVE_ADV_SET,
+ sizeof(instance), &instance, 0,
+ HCI_CMD_TIMEOUT, sk);
+}
+
+static void cancel_adv_timeout(struct hci_dev *hdev)
+{
+ if (hdev->adv_instance_timeout) {
+ hdev->adv_instance_timeout = 0;
+ cancel_delayed_work(&hdev->adv_instance_expire);
+ }
+}
+
+static int hci_set_ext_adv_data_sync(struct hci_dev *hdev, u8 instance)
+{
+ struct {
+ struct hci_cp_le_set_ext_adv_data cp;
+ u8 data[HCI_MAX_EXT_AD_LENGTH];
+ } pdu;
+ u8 len;
+
+ memset(&pdu, 0, sizeof(pdu));
+
+ len = eir_create_adv_data(hdev, instance, pdu.data);
+
+ /* There's nothing to do if the data hasn't changed */
+ if (hdev->adv_data_len == len &&
+ memcmp(pdu.data, hdev->adv_data, len) == 0)
+ return 0;
+
+ memcpy(hdev->adv_data, pdu.data, len);
+ hdev->adv_data_len = len;
+
+ pdu.cp.length = len;
+ pdu.cp.handle = instance;
+ pdu.cp.operation = LE_SET_ADV_DATA_OP_COMPLETE;
+ pdu.cp.frag_pref = LE_SET_ADV_DATA_NO_FRAG;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_LE_SET_EXT_ADV_DATA,
+ sizeof(pdu.cp) + len, &pdu.cp,
+ HCI_CMD_TIMEOUT);
+}
+
+static int hci_set_adv_data_sync(struct hci_dev *hdev, u8 instance)
+{
+ struct hci_cp_le_set_adv_data cp;
+ u8 len;
+
+ memset(&cp, 0, sizeof(cp));
+
+ len = eir_create_adv_data(hdev, instance, cp.data);
+
+ /* There's nothing to do if the data hasn't changed */
+ if (hdev->adv_data_len == len &&
+ memcmp(cp.data, hdev->adv_data, len) == 0)
+ return 0;
+
+ memcpy(hdev->adv_data, cp.data, sizeof(cp.data));
+ hdev->adv_data_len = len;
+
+ cp.length = len;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_LE_SET_ADV_DATA,
+ sizeof(cp), &cp, HCI_CMD_TIMEOUT);
+}
+
+int hci_update_adv_data_sync(struct hci_dev *hdev, u8 instance)
+{
+ if (!hci_dev_test_flag(hdev, HCI_LE_ENABLED))
+ return 0;
+
+ if (ext_adv_capable(hdev))
+ return hci_set_ext_adv_data_sync(hdev, instance);
+
+ return hci_set_adv_data_sync(hdev, instance);
+}
+
+int hci_schedule_adv_instance_sync(struct hci_dev *hdev, u8 instance,
+ bool force)
+{
+ struct adv_info *adv = NULL;
+ u16 timeout;
+
+ if (hci_dev_test_flag(hdev, HCI_ADVERTISING) && !ext_adv_capable(hdev))
+ return -EPERM;
+
+ if (hdev->adv_instance_timeout)
+ return -EBUSY;
+
+ adv = hci_find_adv_instance(hdev, instance);
+ if (!adv)
+ return -ENOENT;
+
+ /* A zero timeout means unlimited advertising. As long as there is
+ * only one instance, duration should be ignored. We still set a timeout
+ * in case further instances are being added later on.
+ *
+ * If the remaining lifetime of the instance is more than the duration
+ * then the timeout corresponds to the duration, otherwise it will be
+ * reduced to the remaining instance lifetime.
+ */
+ if (adv->timeout == 0 || adv->duration <= adv->remaining_time)
+ timeout = adv->duration;
+ else
+ timeout = adv->remaining_time;
+
+ /* The remaining time is being reduced unless the instance is being
+ * advertised without time limit.
+ */
+ if (adv->timeout)
+ adv->remaining_time = adv->remaining_time - timeout;
+
+ /* Only use work for scheduling instances with legacy advertising */
+ if (!ext_adv_capable(hdev)) {
+ hdev->adv_instance_timeout = timeout;
+ queue_delayed_work(hdev->req_workqueue,
+ &hdev->adv_instance_expire,
+ msecs_to_jiffies(timeout * 1000));
+ }
+
+ /* If we're just re-scheduling the same instance again then do not
+ * execute any HCI commands. This happens when a single instance is
+ * being advertised.
+ */
+ if (!force && hdev->cur_adv_instance == instance &&
+ hci_dev_test_flag(hdev, HCI_LE_ADV))
+ return 0;
+
+ hdev->cur_adv_instance = instance;
+
+ return hci_start_adv_sync(hdev, instance);
+}
+
+static int hci_clear_adv_sets_sync(struct hci_dev *hdev, struct sock *sk)
+{
+ int err;
+
+ if (!ext_adv_capable(hdev))
+ return 0;
+
+ /* Disable instance 0x00 to disable all instances */
+ err = hci_disable_ext_adv_instance_sync(hdev, 0x00);
+ if (err)
+ return err;
+
+ return __hci_cmd_sync_status_sk(hdev, HCI_OP_LE_CLEAR_ADV_SETS,
+ 0, NULL, 0, HCI_CMD_TIMEOUT, sk);
+}
+
+static int hci_clear_adv_sync(struct hci_dev *hdev, struct sock *sk, bool force)
+{
+ struct adv_info *adv, *n;
+
+ if (ext_adv_capable(hdev))
+ /* Remove all existing sets */
+ return hci_clear_adv_sets_sync(hdev, sk);
+
+ /* This is safe as long as there is no command send while the lock is
+ * held.
+ */
+ hci_dev_lock(hdev);
+
+ /* Cleanup non-ext instances */
+ list_for_each_entry_safe(adv, n, &hdev->adv_instances, list) {
+ u8 instance = adv->instance;
+ int err;
+
+ if (!(force || adv->timeout))
+ continue;
+
+ err = hci_remove_adv_instance(hdev, instance);
+ if (!err)
+ mgmt_advertising_removed(sk, hdev, instance);
+ }
+
+ hci_dev_unlock(hdev);
+
+ return 0;
+}
+
+static int hci_remove_adv_sync(struct hci_dev *hdev, u8 instance,
+ struct sock *sk)
+{
+ int err;
+
+ /* If we use extended advertising, instance has to be removed first. */
+ if (ext_adv_capable(hdev))
+ return hci_remove_ext_adv_instance_sync(hdev, instance, sk);
+
+ /* This is safe as long as there is no command send while the lock is
+ * held.
+ */
+ hci_dev_lock(hdev);
+
+ err = hci_remove_adv_instance(hdev, instance);
+ if (!err)
+ mgmt_advertising_removed(sk, hdev, instance);
+
+ hci_dev_unlock(hdev);
+
+ return err;
+}
+
+/* For a single instance:
+ * - force == true: The instance will be removed even when its remaining
+ * lifetime is not zero.
+ * - force == false: the instance will be deactivated but kept stored unless
+ * the remaining lifetime is zero.
+ *
+ * For instance == 0x00:
+ * - force == true: All instances will be removed regardless of their timeout
+ * setting.
+ * - force == false: Only instances that have a timeout will be removed.
+ */
+int hci_remove_advertising_sync(struct hci_dev *hdev, struct sock *sk,
+ u8 instance, bool force)
+{
+ struct adv_info *next = NULL;
+ int err;
+
+ /* Cancel any timeout concerning the removed instance(s). */
+ if (!instance || hdev->cur_adv_instance == instance)
+ cancel_adv_timeout(hdev);
+
+ /* Get the next instance to advertise BEFORE we remove
+ * the current one. This can be the same instance again
+ * if there is only one instance.
+ */
+ if (hdev->cur_adv_instance == instance)
+ next = hci_get_next_instance(hdev, instance);
+
+ if (!instance) {
+ err = hci_clear_adv_sync(hdev, sk, force);
+ if (err)
+ return err;
+ } else {
+ struct adv_info *adv = hci_find_adv_instance(hdev, instance);
+
+ if (force || (adv && adv->timeout && !adv->remaining_time)) {
+ /* Don't advertise a removed instance. */
+ if (next && next->instance == instance)
+ next = NULL;
+
+ err = hci_remove_adv_sync(hdev, instance, sk);
+ if (err)
+ return err;
+ }
+ }
+
+ if (!hdev_is_powered(hdev) || hci_dev_test_flag(hdev, HCI_ADVERTISING))
+ return 0;
+
+ if (next && !ext_adv_capable(hdev))
+ hci_schedule_adv_instance_sync(hdev, next->instance, false);
+
+ return 0;
+}
+
+int hci_read_rssi_sync(struct hci_dev *hdev, __le16 handle)
+{
+ struct hci_cp_read_rssi cp;
+
+ cp.handle = handle;
+ return __hci_cmd_sync_status(hdev, HCI_OP_READ_RSSI,
+ sizeof(cp), &cp, HCI_CMD_TIMEOUT);
+}
+
+int hci_read_clock_sync(struct hci_dev *hdev, struct hci_cp_read_clock *cp)
+{
+ return __hci_cmd_sync_status(hdev, HCI_OP_READ_CLOCK,
+ sizeof(*cp), cp, HCI_CMD_TIMEOUT);
+}
+
+int hci_read_tx_power_sync(struct hci_dev *hdev, __le16 handle, u8 type)
+{
+ struct hci_cp_read_tx_power cp;
+
+ cp.handle = handle;
+ cp.type = type;
+ return __hci_cmd_sync_status(hdev, HCI_OP_READ_TX_POWER,
+ sizeof(cp), &cp, HCI_CMD_TIMEOUT);
+}
+
+int hci_disable_advertising_sync(struct hci_dev *hdev)
+{
+ u8 enable = 0x00;
+
+ /* If controller is not advertising we are done. */
+ if (!hci_dev_test_flag(hdev, HCI_LE_ADV))
+ return 0;
+
+ if (ext_adv_capable(hdev))
+ return hci_disable_ext_adv_instance_sync(hdev, 0x00);
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_LE_SET_ADV_ENABLE,
+ sizeof(enable), &enable, HCI_CMD_TIMEOUT);
+}
+
+static int hci_le_set_ext_scan_enable_sync(struct hci_dev *hdev, u8 val,
+ u8 filter_dup)
+{
+ struct hci_cp_le_set_ext_scan_enable cp;
+
+ memset(&cp, 0, sizeof(cp));
+ cp.enable = val;
+ cp.filter_dup = filter_dup;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_LE_SET_EXT_SCAN_ENABLE,
+ sizeof(cp), &cp, HCI_CMD_TIMEOUT);
+}
+
+static int hci_le_set_scan_enable_sync(struct hci_dev *hdev, u8 val,
+ u8 filter_dup)
+{
+ struct hci_cp_le_set_scan_enable cp;
+
+ if (use_ext_scan(hdev))
+ return hci_le_set_ext_scan_enable_sync(hdev, val, filter_dup);
+
+ memset(&cp, 0, sizeof(cp));
+ cp.enable = val;
+ cp.filter_dup = filter_dup;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_LE_SET_SCAN_ENABLE,
+ sizeof(cp), &cp, HCI_CMD_TIMEOUT);
+}
+
+static int hci_le_set_addr_resolution_enable_sync(struct hci_dev *hdev, u8 val)
+{
+ if (!use_ll_privacy(hdev))
+ return 0;
+
+ /* If controller is not/already resolving we are done. */
+ if (val == hci_dev_test_flag(hdev, HCI_LL_RPA_RESOLUTION))
+ return 0;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_LE_SET_ADDR_RESOLV_ENABLE,
+ sizeof(val), &val, HCI_CMD_TIMEOUT);
+}
+
+static int hci_scan_disable_sync(struct hci_dev *hdev)
+{
+ int err;
+
+ /* If controller is not scanning we are done. */
+ if (!hci_dev_test_flag(hdev, HCI_LE_SCAN))
+ return 0;
+
+ if (hdev->scanning_paused) {
+ bt_dev_dbg(hdev, "Scanning is paused for suspend");
+ return 0;
+ }
+
+ err = hci_le_set_scan_enable_sync(hdev, LE_SCAN_DISABLE, 0x00);
+ if (err) {
+ bt_dev_err(hdev, "Unable to disable scanning: %d", err);
+ return err;
+ }
+
+ return err;
+}
+
+static bool scan_use_rpa(struct hci_dev *hdev)
+{
+ return hci_dev_test_flag(hdev, HCI_PRIVACY);
+}
+
+static void hci_start_interleave_scan(struct hci_dev *hdev)
+{
+ hdev->interleave_scan_state = INTERLEAVE_SCAN_NO_FILTER;
+ queue_delayed_work(hdev->req_workqueue,
+ &hdev->interleave_scan, 0);
+}
+
+static bool is_interleave_scanning(struct hci_dev *hdev)
+{
+ return hdev->interleave_scan_state != INTERLEAVE_SCAN_NONE;
+}
+
+static void cancel_interleave_scan(struct hci_dev *hdev)
+{
+ bt_dev_dbg(hdev, "cancelling interleave scan");
+
+ cancel_delayed_work_sync(&hdev->interleave_scan);
+
+ hdev->interleave_scan_state = INTERLEAVE_SCAN_NONE;
+}
+
+/* Return true if interleave_scan wasn't started until exiting this function,
+ * otherwise, return false
+ */
+static bool hci_update_interleaved_scan_sync(struct hci_dev *hdev)
+{
+ /* Do interleaved scan only if all of the following are true:
+ * - There is at least one ADV monitor
+ * - At least one pending LE connection or one device to be scanned for
+ * - Monitor offloading is not supported
+ * If so, we should alternate between allowlist scan and one without
+ * any filters to save power.
+ */
+ bool use_interleaving = hci_is_adv_monitoring(hdev) &&
+ !(list_empty(&hdev->pend_le_conns) &&
+ list_empty(&hdev->pend_le_reports)) &&
+ hci_get_adv_monitor_offload_ext(hdev) ==
+ HCI_ADV_MONITOR_EXT_NONE;
+ bool is_interleaving = is_interleave_scanning(hdev);
+
+ if (use_interleaving && !is_interleaving) {
+ hci_start_interleave_scan(hdev);
+ bt_dev_dbg(hdev, "starting interleave scan");
+ return true;
+ }
+
+ if (!use_interleaving && is_interleaving)
+ cancel_interleave_scan(hdev);
+
+ return false;
+}
+
+/* Removes connection to resolve list if needed.*/
+static int hci_le_del_resolve_list_sync(struct hci_dev *hdev,
+ bdaddr_t *bdaddr, u8 bdaddr_type)
+{
+ struct hci_cp_le_del_from_resolv_list cp;
+ struct bdaddr_list_with_irk *entry;
+
+ if (!use_ll_privacy(hdev))
+ return 0;
+
+ /* Check if the IRK has been programmed */
+ entry = hci_bdaddr_list_lookup_with_irk(&hdev->le_resolv_list, bdaddr,
+ bdaddr_type);
+ if (!entry)
+ return 0;
+
+ cp.bdaddr_type = bdaddr_type;
+ bacpy(&cp.bdaddr, bdaddr);
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_LE_DEL_FROM_RESOLV_LIST,
+ sizeof(cp), &cp, HCI_CMD_TIMEOUT);
+}
+
+static int hci_le_del_accept_list_sync(struct hci_dev *hdev,
+ bdaddr_t *bdaddr, u8 bdaddr_type)
+{
+ struct hci_cp_le_del_from_accept_list cp;
+ int err;
+
+ /* Check if device is on accept list before removing it */
+ if (!hci_bdaddr_list_lookup(&hdev->le_accept_list, bdaddr, bdaddr_type))
+ return 0;
+
+ cp.bdaddr_type = bdaddr_type;
+ bacpy(&cp.bdaddr, bdaddr);
+
+ /* Ignore errors when removing from resolving list as that is likely
+ * that the device was never added.
+ */
+ hci_le_del_resolve_list_sync(hdev, &cp.bdaddr, cp.bdaddr_type);
+
+ err = __hci_cmd_sync_status(hdev, HCI_OP_LE_DEL_FROM_ACCEPT_LIST,
+ sizeof(cp), &cp, HCI_CMD_TIMEOUT);
+ if (err) {
+ bt_dev_err(hdev, "Unable to remove from allow list: %d", err);
+ return err;
+ }
+
+ bt_dev_dbg(hdev, "Remove %pMR (0x%x) from allow list", &cp.bdaddr,
+ cp.bdaddr_type);
+
+ return 0;
+}
+
+/* Adds connection to resolve list if needed.
+ * Setting params to NULL programs local hdev->irk
+ */
+static int hci_le_add_resolve_list_sync(struct hci_dev *hdev,
+ struct hci_conn_params *params)
+{
+ struct hci_cp_le_add_to_resolv_list cp;
+ struct smp_irk *irk;
+ struct bdaddr_list_with_irk *entry;
+
+ if (!use_ll_privacy(hdev))
+ return 0;
+
+ /* Attempt to program local identity address, type and irk if params is
+ * NULL.
+ */
+ if (!params) {
+ if (!hci_dev_test_flag(hdev, HCI_PRIVACY))
+ return 0;
+
+ hci_copy_identity_address(hdev, &cp.bdaddr, &cp.bdaddr_type);
+ memcpy(cp.peer_irk, hdev->irk, 16);
+ goto done;
+ }
+
+ irk = hci_find_irk_by_addr(hdev, ¶ms->addr, params->addr_type);
+ if (!irk)
+ return 0;
+
+ /* Check if the IK has _not_ been programmed yet. */
+ entry = hci_bdaddr_list_lookup_with_irk(&hdev->le_resolv_list,
+ ¶ms->addr,
+ params->addr_type);
+ if (entry)
+ return 0;
+
+ cp.bdaddr_type = params->addr_type;
+ bacpy(&cp.bdaddr, ¶ms->addr);
+ memcpy(cp.peer_irk, irk->val, 16);
+
+done:
+ if (hci_dev_test_flag(hdev, HCI_PRIVACY))
+ memcpy(cp.local_irk, hdev->irk, 16);
+ else
+ memset(cp.local_irk, 0, 16);
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_LE_ADD_TO_RESOLV_LIST,
+ sizeof(cp), &cp, HCI_CMD_TIMEOUT);
+}
+
+/* Set Device Privacy Mode. */
+static int hci_le_set_privacy_mode_sync(struct hci_dev *hdev,
+ struct hci_conn_params *params)
+{
+ struct hci_cp_le_set_privacy_mode cp;
+ struct smp_irk *irk;
+
+ /* If device privacy mode has already been set there is nothing to do */
+ if (params->privacy_mode == HCI_DEVICE_PRIVACY)
+ return 0;
+
+ /* Check if HCI_CONN_FLAG_DEVICE_PRIVACY has been set as it also
+ * indicates that LL Privacy has been enabled and
+ * HCI_OP_LE_SET_PRIVACY_MODE is supported.
+ */
+ if (!test_bit(HCI_CONN_FLAG_DEVICE_PRIVACY, params->flags))
+ return 0;
+
+ irk = hci_find_irk_by_addr(hdev, ¶ms->addr, params->addr_type);
+ if (!irk)
+ return 0;
+
+ memset(&cp, 0, sizeof(cp));
+ cp.bdaddr_type = irk->addr_type;
+ bacpy(&cp.bdaddr, &irk->bdaddr);
+ cp.mode = HCI_DEVICE_PRIVACY;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_LE_SET_PRIVACY_MODE,
+ sizeof(cp), &cp, HCI_CMD_TIMEOUT);
+}
+
+/* Adds connection to allow list if needed, if the device uses RPA (has IRK)
+ * this attempts to program the device in the resolving list as well and
+ * properly set the privacy mode.
+ */
+static int hci_le_add_accept_list_sync(struct hci_dev *hdev,
+ struct hci_conn_params *params,
+ u8 *num_entries)
+{
+ struct hci_cp_le_add_to_accept_list cp;
+ int err;
+
+ /* Select filter policy to accept all advertising */
+ if (*num_entries >= hdev->le_accept_list_size)
+ return -ENOSPC;
+
+ /* Accept list can not be used with RPAs */
+ if (!use_ll_privacy(hdev) &&
+ hci_find_irk_by_addr(hdev, ¶ms->addr, params->addr_type)) {
+ return -EINVAL;
+ }
+
+ /* During suspend, only wakeable devices can be in acceptlist */
+ if (hdev->suspended &&
+ !test_bit(HCI_CONN_FLAG_REMOTE_WAKEUP, params->flags))
+ return 0;
+
+ /* Attempt to program the device in the resolving list first to avoid
+ * having to rollback in case it fails since the resolving list is
+ * dynamic it can probably be smaller than the accept list.
+ */
+ err = hci_le_add_resolve_list_sync(hdev, params);
+ if (err) {
+ bt_dev_err(hdev, "Unable to add to resolve list: %d", err);
+ return err;
+ }
+
+ /* Set Privacy Mode */
+ err = hci_le_set_privacy_mode_sync(hdev, params);
+ if (err) {
+ bt_dev_err(hdev, "Unable to set privacy mode: %d", err);
+ return err;
+ }
+
+ /* Check if already in accept list */
+ if (hci_bdaddr_list_lookup(&hdev->le_accept_list, ¶ms->addr,
+ params->addr_type))
+ return 0;
+
+ *num_entries += 1;
+ cp.bdaddr_type = params->addr_type;
+ bacpy(&cp.bdaddr, ¶ms->addr);
+
+ err = __hci_cmd_sync_status(hdev, HCI_OP_LE_ADD_TO_ACCEPT_LIST,
+ sizeof(cp), &cp, HCI_CMD_TIMEOUT);
+ if (err) {
+ bt_dev_err(hdev, "Unable to add to allow list: %d", err);
+ /* Rollback the device from the resolving list */
+ hci_le_del_resolve_list_sync(hdev, &cp.bdaddr, cp.bdaddr_type);
+ return err;
+ }
+
+ bt_dev_dbg(hdev, "Add %pMR (0x%x) to allow list", &cp.bdaddr,
+ cp.bdaddr_type);
+
+ return 0;
+}
+
+/* This function disables/pause all advertising instances */
+static int hci_pause_advertising_sync(struct hci_dev *hdev)
+{
+ int err;
+ int old_state;
+
+ /* If already been paused there is nothing to do. */
+ if (hdev->advertising_paused)
+ return 0;
+
+ bt_dev_dbg(hdev, "Pausing directed advertising");
+
+ /* Stop directed advertising */
+ old_state = hci_dev_test_flag(hdev, HCI_ADVERTISING);
+ if (old_state) {
+ /* When discoverable timeout triggers, then just make sure
+ * the limited discoverable flag is cleared. Even in the case
+ * of a timeout triggered from general discoverable, it is
+ * safe to unconditionally clear the flag.
+ */
+ hci_dev_clear_flag(hdev, HCI_LIMITED_DISCOVERABLE);
+ hci_dev_clear_flag(hdev, HCI_DISCOVERABLE);
+ hdev->discov_timeout = 0;
+ }
+
+ bt_dev_dbg(hdev, "Pausing advertising instances");
+
+ /* Call to disable any advertisements active on the controller.
+ * This will succeed even if no advertisements are configured.
+ */
+ err = hci_disable_advertising_sync(hdev);
+ if (err)
+ return err;
+
+ /* If we are using software rotation, pause the loop */
+ if (!ext_adv_capable(hdev))
+ cancel_adv_timeout(hdev);
+
+ hdev->advertising_paused = true;
+ hdev->advertising_old_state = old_state;
+
+ return 0;
+}
+
+/* This function enables all user advertising instances */
+static int hci_resume_advertising_sync(struct hci_dev *hdev)
+{
+ struct adv_info *adv, *tmp;
+ int err;
+
+ /* If advertising has not been paused there is nothing to do. */
+ if (!hdev->advertising_paused)
+ return 0;
+
+ /* Resume directed advertising */
+ hdev->advertising_paused = false;
+ if (hdev->advertising_old_state) {
+ hci_dev_set_flag(hdev, HCI_ADVERTISING);
+ hdev->advertising_old_state = 0;
+ }
+
+ bt_dev_dbg(hdev, "Resuming advertising instances");
+
+ if (ext_adv_capable(hdev)) {
+ /* Call for each tracked instance to be re-enabled */
+ list_for_each_entry_safe(adv, tmp, &hdev->adv_instances, list) {
+ err = hci_enable_ext_advertising_sync(hdev,
+ adv->instance);
+ if (!err)
+ continue;
+
+ /* If the instance cannot be resumed remove it */
+ hci_remove_ext_adv_instance_sync(hdev, adv->instance,
+ NULL);
+ }
+ } else {
+ /* Schedule for most recent instance to be restarted and begin
+ * the software rotation loop
+ */
+ err = hci_schedule_adv_instance_sync(hdev,
+ hdev->cur_adv_instance,
+ true);
+ }
+
+ hdev->advertising_paused = false;
+
+ return err;
+}
+
+struct sk_buff *hci_read_local_oob_data_sync(struct hci_dev *hdev,
+ bool extended, struct sock *sk)
+{
+ u16 opcode = extended ? HCI_OP_READ_LOCAL_OOB_EXT_DATA :
+ HCI_OP_READ_LOCAL_OOB_DATA;
+
+ return __hci_cmd_sync_sk(hdev, opcode, 0, NULL, 0, HCI_CMD_TIMEOUT, sk);
+}
+
+/* Device must not be scanning when updating the accept list.
+ *
+ * Update is done using the following sequence:
+ *
+ * use_ll_privacy((Disable Advertising) -> Disable Resolving List) ->
+ * Remove Devices From Accept List ->
+ * (has IRK && use_ll_privacy(Remove Devices From Resolving List))->
+ * Add Devices to Accept List ->
+ * (has IRK && use_ll_privacy(Remove Devices From Resolving List)) ->
+ * use_ll_privacy(Enable Resolving List -> (Enable Advertising)) ->
+ * Enable Scanning
+ *
+ * In case of failure advertising shall be restored to its original state and
+ * return would disable accept list since either accept or resolving list could
+ * not be programmed.
+ *
+ */
+static u8 hci_update_accept_list_sync(struct hci_dev *hdev)
+{
+ struct hci_conn_params *params;
+ struct bdaddr_list *b, *t;
+ u8 num_entries = 0;
+ bool pend_conn, pend_report;
+ int err;
+
+ /* Pause advertising if resolving list can be used as controllers are
+ * cannot accept resolving list modifications while advertising.
+ */
+ if (use_ll_privacy(hdev)) {
+ err = hci_pause_advertising_sync(hdev);
+ if (err) {
+ bt_dev_err(hdev, "pause advertising failed: %d", err);
+ return 0x00;
+ }
+ }
+
+ /* Disable address resolution while reprogramming accept list since
+ * devices that do have an IRK will be programmed in the resolving list
+ * when LL Privacy is enabled.
+ */
+ err = hci_le_set_addr_resolution_enable_sync(hdev, 0x00);
+ if (err) {
+ bt_dev_err(hdev, "Unable to disable LL privacy: %d", err);
+ goto done;
+ }
+
+ /* Go through the current accept list programmed into the
+ * controller one by one and check if that address is still
+ * in the list of pending connections or list of devices to
+ * report. If not present in either list, then remove it from
+ * the controller.
+ */
+ list_for_each_entry_safe(b, t, &hdev->le_accept_list, list) {
+ pend_conn = hci_pend_le_action_lookup(&hdev->pend_le_conns,
+ &b->bdaddr,
+ b->bdaddr_type);
+ pend_report = hci_pend_le_action_lookup(&hdev->pend_le_reports,
+ &b->bdaddr,
+ b->bdaddr_type);
+
+ /* If the device is not likely to connect or report,
+ * remove it from the acceptlist.
+ */
+ if (!pend_conn && !pend_report) {
+ hci_le_del_accept_list_sync(hdev, &b->bdaddr,
+ b->bdaddr_type);
+ continue;
+ }
+
+ num_entries++;
+ }
+
+ /* Since all no longer valid accept list entries have been
+ * removed, walk through the list of pending connections
+ * and ensure that any new device gets programmed into
+ * the controller.
+ *
+ * If the list of the devices is larger than the list of
+ * available accept list entries in the controller, then
+ * just abort and return filer policy value to not use the
+ * accept list.
+ */
+ list_for_each_entry(params, &hdev->pend_le_conns, action) {
+ err = hci_le_add_accept_list_sync(hdev, params, &num_entries);
+ if (err)
+ goto done;
+ }
+
+ /* After adding all new pending connections, walk through
+ * the list of pending reports and also add these to the
+ * accept list if there is still space. Abort if space runs out.
+ */
+ list_for_each_entry(params, &hdev->pend_le_reports, action) {
+ err = hci_le_add_accept_list_sync(hdev, params, &num_entries);
+ if (err)
+ goto done;
+ }
+
+ /* Use the allowlist unless the following conditions are all true:
+ * - We are not currently suspending
+ * - There are 1 or more ADV monitors registered and it's not offloaded
+ * - Interleaved scanning is not currently using the allowlist
+ */
+ if (!idr_is_empty(&hdev->adv_monitors_idr) && !hdev->suspended &&
+ hci_get_adv_monitor_offload_ext(hdev) == HCI_ADV_MONITOR_EXT_NONE &&
+ hdev->interleave_scan_state != INTERLEAVE_SCAN_ALLOWLIST)
+ err = -EINVAL;
+
+done:
+ /* Enable address resolution when LL Privacy is enabled. */
+ err = hci_le_set_addr_resolution_enable_sync(hdev, 0x01);
+ if (err)
+ bt_dev_err(hdev, "Unable to enable LL privacy: %d", err);
+
+ /* Resume advertising if it was paused */
+ if (use_ll_privacy(hdev))
+ hci_resume_advertising_sync(hdev);
+
+ /* Select filter policy to use accept list */
+ return err ? 0x00 : 0x01;
+}
+
+/* Returns true if an le connection is in the scanning state */
+static inline bool hci_is_le_conn_scanning(struct hci_dev *hdev)
+{
+ struct hci_conn_hash *h = &hdev->conn_hash;
+ struct hci_conn *c;
+
+ rcu_read_lock();
+
+ list_for_each_entry_rcu(c, &h->list, list) {
+ if (c->type == LE_LINK && c->state == BT_CONNECT &&
+ test_bit(HCI_CONN_SCANNING, &c->flags)) {
+ rcu_read_unlock();
+ return true;
+ }
+ }
+
+ rcu_read_unlock();
+
+ return false;
+}
+
+static int hci_le_set_ext_scan_param_sync(struct hci_dev *hdev, u8 type,
+ u16 interval, u16 window,
+ u8 own_addr_type, u8 filter_policy)
+{
+ struct hci_cp_le_set_ext_scan_params *cp;
+ struct hci_cp_le_scan_phy_params *phy;
+ u8 data[sizeof(*cp) + sizeof(*phy) * 2];
+ u8 num_phy = 0;
+
+ cp = (void *)data;
+ phy = (void *)cp->data;
+
+ memset(data, 0, sizeof(data));
+
+ cp->own_addr_type = own_addr_type;
+ cp->filter_policy = filter_policy;
+
+ if (scan_1m(hdev) || scan_2m(hdev)) {
+ cp->scanning_phys |= LE_SCAN_PHY_1M;
+
+ phy->type = type;
+ phy->interval = cpu_to_le16(interval);
+ phy->window = cpu_to_le16(window);
+
+ num_phy++;
+ phy++;
+ }
+
+ if (scan_coded(hdev)) {
+ cp->scanning_phys |= LE_SCAN_PHY_CODED;
+
+ phy->type = type;
+ phy->interval = cpu_to_le16(interval);
+ phy->window = cpu_to_le16(window);
+
+ num_phy++;
+ phy++;
+ }
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_LE_SET_EXT_SCAN_PARAMS,
+ sizeof(*cp) + sizeof(*phy) * num_phy,
+ data, HCI_CMD_TIMEOUT);
+}
+
+static int hci_le_set_scan_param_sync(struct hci_dev *hdev, u8 type,
+ u16 interval, u16 window,
+ u8 own_addr_type, u8 filter_policy)
+{
+ struct hci_cp_le_set_scan_param cp;
+
+ if (use_ext_scan(hdev))
+ return hci_le_set_ext_scan_param_sync(hdev, type, interval,
+ window, own_addr_type,
+ filter_policy);
+
+ memset(&cp, 0, sizeof(cp));
+ cp.type = type;
+ cp.interval = cpu_to_le16(interval);
+ cp.window = cpu_to_le16(window);
+ cp.own_address_type = own_addr_type;
+ cp.filter_policy = filter_policy;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_LE_SET_SCAN_PARAM,
+ sizeof(cp), &cp, HCI_CMD_TIMEOUT);
+}
+
+static int hci_start_scan_sync(struct hci_dev *hdev, u8 type, u16 interval,
+ u16 window, u8 own_addr_type, u8 filter_policy,
+ u8 filter_dup)
+{
+ int err;
+
+ if (hdev->scanning_paused) {
+ bt_dev_dbg(hdev, "Scanning is paused for suspend");
+ return 0;
+ }
+
+ err = hci_le_set_scan_param_sync(hdev, type, interval, window,
+ own_addr_type, filter_policy);
+ if (err)
+ return err;
+
+ return hci_le_set_scan_enable_sync(hdev, LE_SCAN_ENABLE, filter_dup);
+}
+
+static int hci_passive_scan_sync(struct hci_dev *hdev)
+{
+ u8 own_addr_type;
+ u8 filter_policy;
+ u16 window, interval;
+ int err;
+
+ if (hdev->scanning_paused) {
+ bt_dev_dbg(hdev, "Scanning is paused for suspend");
+ return 0;
+ }
+
+ err = hci_scan_disable_sync(hdev);
+ if (err) {
+ bt_dev_err(hdev, "disable scanning failed: %d", err);
+ return err;
+ }
+
+ /* Set require_privacy to false since no SCAN_REQ are send
+ * during passive scanning. Not using an non-resolvable address
+ * here is important so that peer devices using direct
+ * advertising with our address will be correctly reported
+ * by the controller.
+ */
+ if (hci_update_random_address_sync(hdev, false, scan_use_rpa(hdev),
+ &own_addr_type))
+ return 0;
+
+ if (hdev->enable_advmon_interleave_scan &&
+ hci_update_interleaved_scan_sync(hdev))
+ return 0;
+
+ bt_dev_dbg(hdev, "interleave state %d", hdev->interleave_scan_state);
+
+ /* Adding or removing entries from the accept list must
+ * happen before enabling scanning. The controller does
+ * not allow accept list modification while scanning.
+ */
+ filter_policy = hci_update_accept_list_sync(hdev);
+
+ /* When the controller is using random resolvable addresses and
+ * with that having LE privacy enabled, then controllers with
+ * Extended Scanner Filter Policies support can now enable support
+ * for handling directed advertising.
+ *
+ * So instead of using filter polices 0x00 (no acceptlist)
+ * and 0x01 (acceptlist enabled) use the new filter policies
+ * 0x02 (no acceptlist) and 0x03 (acceptlist enabled).
+ */
+ if (hci_dev_test_flag(hdev, HCI_PRIVACY) &&
+ (hdev->le_features[0] & HCI_LE_EXT_SCAN_POLICY))
+ filter_policy |= 0x02;
+
+ if (hdev->suspended) {
+ window = hdev->le_scan_window_suspend;
+ interval = hdev->le_scan_int_suspend;
+ } else if (hci_is_le_conn_scanning(hdev)) {
+ window = hdev->le_scan_window_connect;
+ interval = hdev->le_scan_int_connect;
+ } else if (hci_is_adv_monitoring(hdev)) {
+ window = hdev->le_scan_window_adv_monitor;
+ interval = hdev->le_scan_int_adv_monitor;
+ } else {
+ window = hdev->le_scan_window;
+ interval = hdev->le_scan_interval;
+ }
+
+ bt_dev_dbg(hdev, "LE passive scan with acceptlist = %d", filter_policy);
+
+ return hci_start_scan_sync(hdev, LE_SCAN_PASSIVE, interval, window,
+ own_addr_type, filter_policy,
+ LE_SCAN_FILTER_DUP_ENABLE);
+}
+
+/* This function controls the passive scanning based on hdev->pend_le_conns
+ * list. If there are pending LE connection we start the background scanning,
+ * otherwise we stop it in the following sequence:
+ *
+ * If there are devices to scan:
+ *
+ * Disable Scanning -> Update Accept List ->
+ * use_ll_privacy((Disable Advertising) -> Disable Resolving List ->
+ * Update Resolving List -> Enable Resolving List -> (Enable Advertising)) ->
+ * Enable Scanning
+ *
+ * Otherwise:
+ *
+ * Disable Scanning
+ */
+int hci_update_passive_scan_sync(struct hci_dev *hdev)
+{
+ int err;
+
+ if (!test_bit(HCI_UP, &hdev->flags) ||
+ test_bit(HCI_INIT, &hdev->flags) ||
+ hci_dev_test_flag(hdev, HCI_SETUP) ||
+ hci_dev_test_flag(hdev, HCI_CONFIG) ||
+ hci_dev_test_flag(hdev, HCI_AUTO_OFF) ||
+ hci_dev_test_flag(hdev, HCI_UNREGISTER))
+ return 0;
+
+ /* No point in doing scanning if LE support hasn't been enabled */
+ if (!hci_dev_test_flag(hdev, HCI_LE_ENABLED))
+ return 0;
+
+ /* If discovery is active don't interfere with it */
+ if (hdev->discovery.state != DISCOVERY_STOPPED)
+ return 0;
+
+ /* Reset RSSI and UUID filters when starting background scanning
+ * since these filters are meant for service discovery only.
+ *
+ * The Start Discovery and Start Service Discovery operations
+ * ensure to set proper values for RSSI threshold and UUID
+ * filter list. So it is safe to just reset them here.
+ */
+ hci_discovery_filter_clear(hdev);
+
+ bt_dev_dbg(hdev, "ADV monitoring is %s",
+ hci_is_adv_monitoring(hdev) ? "on" : "off");
+
+ if (list_empty(&hdev->pend_le_conns) &&
+ list_empty(&hdev->pend_le_reports) &&
+ !hci_is_adv_monitoring(hdev)) {
+ /* If there is no pending LE connections or devices
+ * to be scanned for or no ADV monitors, we should stop the
+ * background scanning.
+ */
+
+ bt_dev_dbg(hdev, "stopping background scanning");
+
+ err = hci_scan_disable_sync(hdev);
+ if (err)
+ bt_dev_err(hdev, "stop background scanning failed: %d",
+ err);
+ } else {
+ /* If there is at least one pending LE connection, we should
+ * keep the background scan running.
+ */
+
+ /* If controller is connecting, we should not start scanning
+ * since some controllers are not able to scan and connect at
+ * the same time.
+ */
+ if (hci_lookup_le_connect(hdev))
+ return 0;
+
+ bt_dev_dbg(hdev, "start background scanning");
+
+ err = hci_passive_scan_sync(hdev);
+ if (err)
+ bt_dev_err(hdev, "start background scanning failed: %d",
+ err);
+ }
+
+ return err;
+}
+
+static int update_passive_scan_sync(struct hci_dev *hdev, void *data)
+{
+ return hci_update_passive_scan_sync(hdev);
+}
+
+int hci_update_passive_scan(struct hci_dev *hdev)
+{
+ /* Only queue if it would have any effect */
+ if (!test_bit(HCI_UP, &hdev->flags) ||
+ test_bit(HCI_INIT, &hdev->flags) ||
+ hci_dev_test_flag(hdev, HCI_SETUP) ||
+ hci_dev_test_flag(hdev, HCI_CONFIG) ||
+ hci_dev_test_flag(hdev, HCI_AUTO_OFF) ||
+ hci_dev_test_flag(hdev, HCI_UNREGISTER))
+ return 0;
+
+ return hci_cmd_sync_queue(hdev, update_passive_scan_sync, NULL, NULL);
+}
+
+int hci_write_sc_support_sync(struct hci_dev *hdev, u8 val)
+{
+ int err;
+
+ if (!bredr_sc_enabled(hdev) || lmp_host_sc_capable(hdev))
+ return 0;
+
+ err = __hci_cmd_sync_status(hdev, HCI_OP_WRITE_SC_SUPPORT,
+ sizeof(val), &val, HCI_CMD_TIMEOUT);
+
+ if (!err) {
+ if (val) {
+ hdev->features[1][0] |= LMP_HOST_SC;
+ hci_dev_set_flag(hdev, HCI_SC_ENABLED);
+ } else {
+ hdev->features[1][0] &= ~LMP_HOST_SC;
+ hci_dev_clear_flag(hdev, HCI_SC_ENABLED);
+ }
+ }
+
+ return err;
+}
+
+int hci_write_ssp_mode_sync(struct hci_dev *hdev, u8 mode)
+{
+ int err;
+
+ if (!hci_dev_test_flag(hdev, HCI_SSP_ENABLED) ||
+ lmp_host_ssp_capable(hdev))
+ return 0;
+
+ if (!mode && hci_dev_test_flag(hdev, HCI_USE_DEBUG_KEYS)) {
+ __hci_cmd_sync_status(hdev, HCI_OP_WRITE_SSP_DEBUG_MODE,
+ sizeof(mode), &mode, HCI_CMD_TIMEOUT);
+ }
+
+ err = __hci_cmd_sync_status(hdev, HCI_OP_WRITE_SSP_MODE,
+ sizeof(mode), &mode, HCI_CMD_TIMEOUT);
+ if (err)
+ return err;
+
+ return hci_write_sc_support_sync(hdev, 0x01);
+}
+
+int hci_write_le_host_supported_sync(struct hci_dev *hdev, u8 le, u8 simul)
+{
+ struct hci_cp_write_le_host_supported cp;
+
+ if (!hci_dev_test_flag(hdev, HCI_LE_ENABLED) ||
+ !lmp_bredr_capable(hdev))
+ return 0;
+
+ /* Check first if we already have the right host state
+ * (host features set)
+ */
+ if (le == lmp_host_le_capable(hdev) &&
+ simul == lmp_host_le_br_capable(hdev))
+ return 0;
+
+ memset(&cp, 0, sizeof(cp));
+
+ cp.le = le;
+ cp.simul = simul;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_WRITE_LE_HOST_SUPPORTED,
+ sizeof(cp), &cp, HCI_CMD_TIMEOUT);
+}
+
+static int hci_powered_update_adv_sync(struct hci_dev *hdev)
+{
+ struct adv_info *adv, *tmp;
+ int err;
+
+ if (!hci_dev_test_flag(hdev, HCI_LE_ENABLED))
+ return 0;
+
+ /* If RPA Resolution has not been enable yet it means the
+ * resolving list is empty and we should attempt to program the
+ * local IRK in order to support using own_addr_type
+ * ADDR_LE_DEV_RANDOM_RESOLVED (0x03).
+ */
+ if (!hci_dev_test_flag(hdev, HCI_LL_RPA_RESOLUTION)) {
+ hci_le_add_resolve_list_sync(hdev, NULL);
+ hci_le_set_addr_resolution_enable_sync(hdev, 0x01);
+ }
+
+ /* Make sure the controller has a good default for
+ * advertising data. This also applies to the case
+ * where BR/EDR was toggled during the AUTO_OFF phase.
+ */
+ if (hci_dev_test_flag(hdev, HCI_ADVERTISING) ||
+ list_empty(&hdev->adv_instances)) {
+ if (ext_adv_capable(hdev)) {
+ err = hci_setup_ext_adv_instance_sync(hdev, 0x00);
+ if (!err)
+ hci_update_scan_rsp_data_sync(hdev, 0x00);
+ } else {
+ err = hci_update_adv_data_sync(hdev, 0x00);
+ if (!err)
+ hci_update_scan_rsp_data_sync(hdev, 0x00);
+ }
+
+ if (hci_dev_test_flag(hdev, HCI_ADVERTISING))
+ hci_enable_advertising_sync(hdev);
+ }
+
+ /* Call for each tracked instance to be scheduled */
+ list_for_each_entry_safe(adv, tmp, &hdev->adv_instances, list)
+ hci_schedule_adv_instance_sync(hdev, adv->instance, true);
+
+ return 0;
+}
+
+static int hci_write_auth_enable_sync(struct hci_dev *hdev)
+{
+ u8 link_sec;
+
+ link_sec = hci_dev_test_flag(hdev, HCI_LINK_SECURITY);
+ if (link_sec == test_bit(HCI_AUTH, &hdev->flags))
+ return 0;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_WRITE_AUTH_ENABLE,
+ sizeof(link_sec), &link_sec,
+ HCI_CMD_TIMEOUT);
+}
+
+int hci_write_fast_connectable_sync(struct hci_dev *hdev, bool enable)
+{
+ struct hci_cp_write_page_scan_activity cp;
+ u8 type;
+ int err = 0;
+
+ if (!hci_dev_test_flag(hdev, HCI_BREDR_ENABLED))
+ return 0;
+
+ if (hdev->hci_ver < BLUETOOTH_VER_1_2)
+ return 0;
+
+ memset(&cp, 0, sizeof(cp));
+
+ if (enable) {
+ type = PAGE_SCAN_TYPE_INTERLACED;
+
+ /* 160 msec page scan interval */
+ cp.interval = cpu_to_le16(0x0100);
+ } else {
+ type = hdev->def_page_scan_type;
+ cp.interval = cpu_to_le16(hdev->def_page_scan_int);
+ }
+
+ cp.window = cpu_to_le16(hdev->def_page_scan_window);
+
+ if (__cpu_to_le16(hdev->page_scan_interval) != cp.interval ||
+ __cpu_to_le16(hdev->page_scan_window) != cp.window) {
+ err = __hci_cmd_sync_status(hdev,
+ HCI_OP_WRITE_PAGE_SCAN_ACTIVITY,
+ sizeof(cp), &cp, HCI_CMD_TIMEOUT);
+ if (err)
+ return err;
+ }
+
+ if (hdev->page_scan_type != type)
+ err = __hci_cmd_sync_status(hdev,
+ HCI_OP_WRITE_PAGE_SCAN_TYPE,
+ sizeof(type), &type,
+ HCI_CMD_TIMEOUT);
+
+ return err;
+}
+
+static bool disconnected_accept_list_entries(struct hci_dev *hdev)
+{
+ struct bdaddr_list *b;
+
+ list_for_each_entry(b, &hdev->accept_list, list) {
+ struct hci_conn *conn;
+
+ conn = hci_conn_hash_lookup_ba(hdev, ACL_LINK, &b->bdaddr);
+ if (!conn)
+ return true;
+
+ if (conn->state != BT_CONNECTED && conn->state != BT_CONFIG)
+ return true;
+ }
+
+ return false;
+}
+
+static int hci_write_scan_enable_sync(struct hci_dev *hdev, u8 val)
+{
+ return __hci_cmd_sync_status(hdev, HCI_OP_WRITE_SCAN_ENABLE,
+ sizeof(val), &val,
+ HCI_CMD_TIMEOUT);
+}
+
+int hci_update_scan_sync(struct hci_dev *hdev)
+{
+ u8 scan;
+
+ if (!hci_dev_test_flag(hdev, HCI_BREDR_ENABLED))
+ return 0;
+
+ if (!hdev_is_powered(hdev))
+ return 0;
+
+ if (mgmt_powering_down(hdev))
+ return 0;
+
+ if (hdev->scanning_paused)
+ return 0;
+
+ if (hci_dev_test_flag(hdev, HCI_CONNECTABLE) ||
+ disconnected_accept_list_entries(hdev))
+ scan = SCAN_PAGE;
+ else
+ scan = SCAN_DISABLED;
+
+ if (hci_dev_test_flag(hdev, HCI_DISCOVERABLE))
+ scan |= SCAN_INQUIRY;
+
+ if (test_bit(HCI_PSCAN, &hdev->flags) == !!(scan & SCAN_PAGE) &&
+ test_bit(HCI_ISCAN, &hdev->flags) == !!(scan & SCAN_INQUIRY))
+ return 0;
+
+ return hci_write_scan_enable_sync(hdev, scan);
+}
+
+int hci_update_name_sync(struct hci_dev *hdev)
+{
+ struct hci_cp_write_local_name cp;
+
+ memset(&cp, 0, sizeof(cp));
+
+ memcpy(cp.name, hdev->dev_name, sizeof(cp.name));
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_WRITE_LOCAL_NAME,
+ sizeof(cp), &cp,
+ HCI_CMD_TIMEOUT);
+}
+
+/* This function perform powered update HCI command sequence after the HCI init
+ * sequence which end up resetting all states, the sequence is as follows:
+ *
+ * HCI_SSP_ENABLED(Enable SSP)
+ * HCI_LE_ENABLED(Enable LE)
+ * HCI_LE_ENABLED(use_ll_privacy(Add local IRK to Resolving List) ->
+ * Update adv data)
+ * Enable Authentication
+ * lmp_bredr_capable(Set Fast Connectable -> Set Scan Type -> Set Class ->
+ * Set Name -> Set EIR)
+ */
+int hci_powered_update_sync(struct hci_dev *hdev)
+{
+ int err;
+
+ /* Register the available SMP channels (BR/EDR and LE) only when
+ * successfully powering on the controller. This late
+ * registration is required so that LE SMP can clearly decide if
+ * the public address or static address is used.
+ */
+ smp_register(hdev);
+
+ err = hci_write_ssp_mode_sync(hdev, 0x01);
+ if (err)
+ return err;
+
+ err = hci_write_le_host_supported_sync(hdev, 0x01, 0x00);
+ if (err)
+ return err;
+
+ err = hci_powered_update_adv_sync(hdev);
+ if (err)
+ return err;
+
+ err = hci_write_auth_enable_sync(hdev);
+ if (err)
+ return err;
+
+ if (lmp_bredr_capable(hdev)) {
+ if (hci_dev_test_flag(hdev, HCI_FAST_CONNECTABLE))
+ hci_write_fast_connectable_sync(hdev, true);
+ else
+ hci_write_fast_connectable_sync(hdev, false);
+ hci_update_scan_sync(hdev);
+ hci_update_class_sync(hdev);
+ hci_update_name_sync(hdev);
+ hci_update_eir_sync(hdev);
+ }
+
+ return 0;
+}
+
+/**
+ * hci_dev_get_bd_addr_from_property - Get the Bluetooth Device Address
+ * (BD_ADDR) for a HCI device from
+ * a firmware node property.
+ * @hdev: The HCI device
+ *
+ * Search the firmware node for 'local-bd-address'.
+ *
+ * All-zero BD addresses are rejected, because those could be properties
+ * that exist in the firmware tables, but were not updated by the firmware. For
+ * example, the DTS could define 'local-bd-address', with zero BD addresses.
+ */
+static void hci_dev_get_bd_addr_from_property(struct hci_dev *hdev)
+{
+ struct fwnode_handle *fwnode = dev_fwnode(hdev->dev.parent);
+ bdaddr_t ba;
+ int ret;
+
+ ret = fwnode_property_read_u8_array(fwnode, "local-bd-address",
+ (u8 *)&ba, sizeof(ba));
+ if (ret < 0 || !bacmp(&ba, BDADDR_ANY))
+ return;
+
+ bacpy(&hdev->public_addr, &ba);
+}
+
+struct hci_init_stage {
+ int (*func)(struct hci_dev *hdev);
+};
+
+/* Run init stage NULL terminated function table */
+static int hci_init_stage_sync(struct hci_dev *hdev,
+ const struct hci_init_stage *stage)
+{
+ size_t i;
+
+ for (i = 0; stage[i].func; i++) {
+ int err;
+
+ err = stage[i].func(hdev);
+ if (err)
+ return err;
+ }
+
+ return 0;
+}
+
+/* Read Local Version */
+static int hci_read_local_version_sync(struct hci_dev *hdev)
+{
+ return __hci_cmd_sync_status(hdev, HCI_OP_READ_LOCAL_VERSION,
+ 0, NULL, HCI_CMD_TIMEOUT);
+}
+
+/* Read BD Address */
+static int hci_read_bd_addr_sync(struct hci_dev *hdev)
+{
+ return __hci_cmd_sync_status(hdev, HCI_OP_READ_BD_ADDR,
+ 0, NULL, HCI_CMD_TIMEOUT);
+}
+
+#define HCI_INIT(_func) \
+{ \
+ .func = _func, \
+}
+
+static const struct hci_init_stage hci_init0[] = {
+ /* HCI_OP_READ_LOCAL_VERSION */
+ HCI_INIT(hci_read_local_version_sync),
+ /* HCI_OP_READ_BD_ADDR */
+ HCI_INIT(hci_read_bd_addr_sync),
+ {}
+};
+
+int hci_reset_sync(struct hci_dev *hdev)
+{
+ int err;
+
+ set_bit(HCI_RESET, &hdev->flags);
+
+ err = __hci_cmd_sync_status(hdev, HCI_OP_RESET, 0, NULL,
+ HCI_CMD_TIMEOUT);
+ if (err)
+ return err;
+
+ return 0;
+}
+
+static int hci_init0_sync(struct hci_dev *hdev)
+{
+ int err;
+
+ bt_dev_dbg(hdev, "");
+
+ /* Reset */
+ if (!test_bit(HCI_QUIRK_RESET_ON_CLOSE, &hdev->quirks)) {
+ err = hci_reset_sync(hdev);
+ if (err)
+ return err;
+ }
+
+ return hci_init_stage_sync(hdev, hci_init0);
+}
+
+static int hci_unconf_init_sync(struct hci_dev *hdev)
+{
+ int err;
+
+ if (test_bit(HCI_QUIRK_RAW_DEVICE, &hdev->quirks))
+ return 0;
+
+ err = hci_init0_sync(hdev);
+ if (err < 0)
+ return err;
+
+ if (hci_dev_test_flag(hdev, HCI_SETUP))
+ hci_debugfs_create_basic(hdev);
+
+ return 0;
+}
+
+/* Read Local Supported Features. */
+static int hci_read_local_features_sync(struct hci_dev *hdev)
+{
+ /* Not all AMP controllers support this command */
+ if (hdev->dev_type == HCI_AMP && !(hdev->commands[14] & 0x20))
+ return 0;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_READ_LOCAL_FEATURES,
+ 0, NULL, HCI_CMD_TIMEOUT);
+}
+
+/* BR Controller init stage 1 command sequence */
+static const struct hci_init_stage br_init1[] = {
+ /* HCI_OP_READ_LOCAL_FEATURES */
+ HCI_INIT(hci_read_local_features_sync),
+ /* HCI_OP_READ_LOCAL_VERSION */
+ HCI_INIT(hci_read_local_version_sync),
+ /* HCI_OP_READ_BD_ADDR */
+ HCI_INIT(hci_read_bd_addr_sync),
+ {}
+};
+
+/* Read Local Commands */
+static int hci_read_local_cmds_sync(struct hci_dev *hdev)
+{
+ /* All Bluetooth 1.2 and later controllers should support the
+ * HCI command for reading the local supported commands.
+ *
+ * Unfortunately some controllers indicate Bluetooth 1.2 support,
+ * but do not have support for this command. If that is the case,
+ * the driver can quirk the behavior and skip reading the local
+ * supported commands.
+ */
+ if (hdev->hci_ver > BLUETOOTH_VER_1_1 &&
+ !test_bit(HCI_QUIRK_BROKEN_LOCAL_COMMANDS, &hdev->quirks))
+ return __hci_cmd_sync_status(hdev, HCI_OP_READ_LOCAL_COMMANDS,
+ 0, NULL, HCI_CMD_TIMEOUT);
+
+ return 0;
+}
+
+/* Read Local AMP Info */
+static int hci_read_local_amp_info_sync(struct hci_dev *hdev)
+{
+ return __hci_cmd_sync_status(hdev, HCI_OP_READ_LOCAL_AMP_INFO,
+ 0, NULL, HCI_CMD_TIMEOUT);
+}
+
+/* Read Data Blk size */
+static int hci_read_data_block_size_sync(struct hci_dev *hdev)
+{
+ return __hci_cmd_sync_status(hdev, HCI_OP_READ_DATA_BLOCK_SIZE,
+ 0, NULL, HCI_CMD_TIMEOUT);
+}
+
+/* Read Flow Control Mode */
+static int hci_read_flow_control_mode_sync(struct hci_dev *hdev)
+{
+ return __hci_cmd_sync_status(hdev, HCI_OP_READ_FLOW_CONTROL_MODE,
+ 0, NULL, HCI_CMD_TIMEOUT);
+}
+
+/* Read Location Data */
+static int hci_read_location_data_sync(struct hci_dev *hdev)
+{
+ return __hci_cmd_sync_status(hdev, HCI_OP_READ_LOCATION_DATA,
+ 0, NULL, HCI_CMD_TIMEOUT);
+}
+
+/* AMP Controller init stage 1 command sequence */
+static const struct hci_init_stage amp_init1[] = {
+ /* HCI_OP_READ_LOCAL_VERSION */
+ HCI_INIT(hci_read_local_version_sync),
+ /* HCI_OP_READ_LOCAL_COMMANDS */
+ HCI_INIT(hci_read_local_cmds_sync),
+ /* HCI_OP_READ_LOCAL_AMP_INFO */
+ HCI_INIT(hci_read_local_amp_info_sync),
+ /* HCI_OP_READ_DATA_BLOCK_SIZE */
+ HCI_INIT(hci_read_data_block_size_sync),
+ /* HCI_OP_READ_FLOW_CONTROL_MODE */
+ HCI_INIT(hci_read_flow_control_mode_sync),
+ /* HCI_OP_READ_LOCATION_DATA */
+ HCI_INIT(hci_read_location_data_sync),
+};
+
+static int hci_init1_sync(struct hci_dev *hdev)
+{
+ int err;
+
+ bt_dev_dbg(hdev, "");
+
+ /* Reset */
+ if (!test_bit(HCI_QUIRK_RESET_ON_CLOSE, &hdev->quirks)) {
+ err = hci_reset_sync(hdev);
+ if (err)
+ return err;
+ }
+
+ switch (hdev->dev_type) {
+ case HCI_PRIMARY:
+ hdev->flow_ctl_mode = HCI_FLOW_CTL_MODE_PACKET_BASED;
+ return hci_init_stage_sync(hdev, br_init1);
+ case HCI_AMP:
+ hdev->flow_ctl_mode = HCI_FLOW_CTL_MODE_BLOCK_BASED;
+ return hci_init_stage_sync(hdev, amp_init1);
+ default:
+ bt_dev_err(hdev, "Unknown device type %d", hdev->dev_type);
+ break;
+ }
+
+ return 0;
+}
+
+/* AMP Controller init stage 2 command sequence */
+static const struct hci_init_stage amp_init2[] = {
+ /* HCI_OP_READ_LOCAL_FEATURES */
+ HCI_INIT(hci_read_local_features_sync),
+};
+
+/* Read Buffer Size (ACL mtu, max pkt, etc.) */
+static int hci_read_buffer_size_sync(struct hci_dev *hdev)
+{
+ return __hci_cmd_sync_status(hdev, HCI_OP_READ_BUFFER_SIZE,
+ 0, NULL, HCI_CMD_TIMEOUT);
+}
+
+/* Read Class of Device */
+static int hci_read_dev_class_sync(struct hci_dev *hdev)
+{
+ return __hci_cmd_sync_status(hdev, HCI_OP_READ_CLASS_OF_DEV,
+ 0, NULL, HCI_CMD_TIMEOUT);
+}
+
+/* Read Local Name */
+static int hci_read_local_name_sync(struct hci_dev *hdev)
+{
+ return __hci_cmd_sync_status(hdev, HCI_OP_READ_LOCAL_NAME,
+ 0, NULL, HCI_CMD_TIMEOUT);
+}
+
+/* Read Voice Setting */
+static int hci_read_voice_setting_sync(struct hci_dev *hdev)
+{
+ return __hci_cmd_sync_status(hdev, HCI_OP_READ_VOICE_SETTING,
+ 0, NULL, HCI_CMD_TIMEOUT);
+}
+
+/* Read Number of Supported IAC */
+static int hci_read_num_supported_iac_sync(struct hci_dev *hdev)
+{
+ return __hci_cmd_sync_status(hdev, HCI_OP_READ_NUM_SUPPORTED_IAC,
+ 0, NULL, HCI_CMD_TIMEOUT);
+}
+
+/* Read Current IAC LAP */
+static int hci_read_current_iac_lap_sync(struct hci_dev *hdev)
+{
+ return __hci_cmd_sync_status(hdev, HCI_OP_READ_CURRENT_IAC_LAP,
+ 0, NULL, HCI_CMD_TIMEOUT);
+}
+
+static int hci_set_event_filter_sync(struct hci_dev *hdev, u8 flt_type,
+ u8 cond_type, bdaddr_t *bdaddr,
+ u8 auto_accept)
+{
+ struct hci_cp_set_event_filter cp;
+
+ if (!hci_dev_test_flag(hdev, HCI_BREDR_ENABLED))
+ return 0;
+
+ memset(&cp, 0, sizeof(cp));
+ cp.flt_type = flt_type;
+
+ if (flt_type != HCI_FLT_CLEAR_ALL) {
+ cp.cond_type = cond_type;
+ bacpy(&cp.addr_conn_flt.bdaddr, bdaddr);
+ cp.addr_conn_flt.auto_accept = auto_accept;
+ }
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_SET_EVENT_FLT,
+ flt_type == HCI_FLT_CLEAR_ALL ?
+ sizeof(cp.flt_type) : sizeof(cp), &cp,
+ HCI_CMD_TIMEOUT);
+}
+
+static int hci_clear_event_filter_sync(struct hci_dev *hdev)
+{
+ if (!hci_dev_test_flag(hdev, HCI_EVENT_FILTER_CONFIGURED))
+ return 0;
+
+ return hci_set_event_filter_sync(hdev, HCI_FLT_CLEAR_ALL, 0x00,
+ BDADDR_ANY, 0x00);
+}
+
+/* Connection accept timeout ~20 secs */
+static int hci_write_ca_timeout_sync(struct hci_dev *hdev)
+{
+ __le16 param = cpu_to_le16(0x7d00);
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_WRITE_CA_TIMEOUT,
+ sizeof(param), ¶m, HCI_CMD_TIMEOUT);
+}
+
+/* BR Controller init stage 2 command sequence */
+static const struct hci_init_stage br_init2[] = {
+ /* HCI_OP_READ_BUFFER_SIZE */
+ HCI_INIT(hci_read_buffer_size_sync),
+ /* HCI_OP_READ_CLASS_OF_DEV */
+ HCI_INIT(hci_read_dev_class_sync),
+ /* HCI_OP_READ_LOCAL_NAME */
+ HCI_INIT(hci_read_local_name_sync),
+ /* HCI_OP_READ_VOICE_SETTING */
+ HCI_INIT(hci_read_voice_setting_sync),
+ /* HCI_OP_READ_NUM_SUPPORTED_IAC */
+ HCI_INIT(hci_read_num_supported_iac_sync),
+ /* HCI_OP_READ_CURRENT_IAC_LAP */
+ HCI_INIT(hci_read_current_iac_lap_sync),
+ /* HCI_OP_SET_EVENT_FLT */
+ HCI_INIT(hci_clear_event_filter_sync),
+ /* HCI_OP_WRITE_CA_TIMEOUT */
+ HCI_INIT(hci_write_ca_timeout_sync),
+ {}
+};
+
+static int hci_write_ssp_mode_1_sync(struct hci_dev *hdev)
+{
+ u8 mode = 0x01;
+
+ if (!lmp_ssp_capable(hdev) || !hci_dev_test_flag(hdev, HCI_SSP_ENABLED))
+ return 0;
+
+ /* When SSP is available, then the host features page
+ * should also be available as well. However some
+ * controllers list the max_page as 0 as long as SSP
+ * has not been enabled. To achieve proper debugging
+ * output, force the minimum max_page to 1 at least.
+ */
+ hdev->max_page = 0x01;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_WRITE_SSP_MODE,
+ sizeof(mode), &mode, HCI_CMD_TIMEOUT);
+}
+
+static int hci_write_eir_sync(struct hci_dev *hdev)
+{
+ struct hci_cp_write_eir cp;
+
+ if (!lmp_ssp_capable(hdev) || hci_dev_test_flag(hdev, HCI_SSP_ENABLED))
+ return 0;
+
+ memset(hdev->eir, 0, sizeof(hdev->eir));
+ memset(&cp, 0, sizeof(cp));
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_WRITE_EIR, sizeof(cp), &cp,
+ HCI_CMD_TIMEOUT);
+}
+
+static int hci_write_inquiry_mode_sync(struct hci_dev *hdev)
+{
+ u8 mode;
+
+ if (!lmp_inq_rssi_capable(hdev) &&
+ !test_bit(HCI_QUIRK_FIXUP_INQUIRY_MODE, &hdev->quirks))
+ return 0;
+
+ /* If Extended Inquiry Result events are supported, then
+ * they are clearly preferred over Inquiry Result with RSSI
+ * events.
+ */
+ mode = lmp_ext_inq_capable(hdev) ? 0x02 : 0x01;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_WRITE_INQUIRY_MODE,
+ sizeof(mode), &mode, HCI_CMD_TIMEOUT);
+}
+
+static int hci_read_inq_rsp_tx_power_sync(struct hci_dev *hdev)
+{
+ if (!lmp_inq_tx_pwr_capable(hdev))
+ return 0;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_READ_INQ_RSP_TX_POWER,
+ 0, NULL, HCI_CMD_TIMEOUT);
+}
+
+static int hci_read_local_ext_features_sync(struct hci_dev *hdev, u8 page)
+{
+ struct hci_cp_read_local_ext_features cp;
+
+ if (!lmp_ext_feat_capable(hdev))
+ return 0;
+
+ memset(&cp, 0, sizeof(cp));
+ cp.page = page;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_READ_LOCAL_EXT_FEATURES,
+ sizeof(cp), &cp, HCI_CMD_TIMEOUT);
+}
+
+static int hci_read_local_ext_features_1_sync(struct hci_dev *hdev)
+{
+ return hci_read_local_ext_features_sync(hdev, 0x01);
+}
+
+/* HCI Controller init stage 2 command sequence */
+static const struct hci_init_stage hci_init2[] = {
+ /* HCI_OP_READ_LOCAL_COMMANDS */
+ HCI_INIT(hci_read_local_cmds_sync),
+ /* HCI_OP_WRITE_SSP_MODE */
+ HCI_INIT(hci_write_ssp_mode_1_sync),
+ /* HCI_OP_WRITE_EIR */
+ HCI_INIT(hci_write_eir_sync),
+ /* HCI_OP_WRITE_INQUIRY_MODE */
+ HCI_INIT(hci_write_inquiry_mode_sync),
+ /* HCI_OP_READ_INQ_RSP_TX_POWER */
+ HCI_INIT(hci_read_inq_rsp_tx_power_sync),
+ /* HCI_OP_READ_LOCAL_EXT_FEATURES */
+ HCI_INIT(hci_read_local_ext_features_1_sync),
+ /* HCI_OP_WRITE_AUTH_ENABLE */
+ HCI_INIT(hci_write_auth_enable_sync),
+ {}
+};
+
+/* Read LE Buffer Size */
+static int hci_le_read_buffer_size_sync(struct hci_dev *hdev)
+{
+ return __hci_cmd_sync_status(hdev, HCI_OP_LE_READ_BUFFER_SIZE,
+ 0, NULL, HCI_CMD_TIMEOUT);
+}
+
+/* Read LE Local Supported Features */
+static int hci_le_read_local_features_sync(struct hci_dev *hdev)
+{
+ return __hci_cmd_sync_status(hdev, HCI_OP_LE_READ_LOCAL_FEATURES,
+ 0, NULL, HCI_CMD_TIMEOUT);
+}
+
+/* Read LE Supported States */
+static int hci_le_read_supported_states_sync(struct hci_dev *hdev)
+{
+ return __hci_cmd_sync_status(hdev, HCI_OP_LE_READ_SUPPORTED_STATES,
+ 0, NULL, HCI_CMD_TIMEOUT);
+}
+
+/* LE Controller init stage 2 command sequence */
+static const struct hci_init_stage le_init2[] = {
+ /* HCI_OP_LE_READ_BUFFER_SIZE */
+ HCI_INIT(hci_le_read_buffer_size_sync),
+ /* HCI_OP_LE_READ_LOCAL_FEATURES */
+ HCI_INIT(hci_le_read_local_features_sync),
+ /* HCI_OP_LE_READ_SUPPORTED_STATES */
+ HCI_INIT(hci_le_read_supported_states_sync),
+ {}
+};
+
+static int hci_init2_sync(struct hci_dev *hdev)
+{
+ int err;
+
+ bt_dev_dbg(hdev, "");
+
+ if (hdev->dev_type == HCI_AMP)
+ return hci_init_stage_sync(hdev, amp_init2);
+
+ if (lmp_bredr_capable(hdev)) {
+ err = hci_init_stage_sync(hdev, br_init2);
+ if (err)
+ return err;
+ } else {
+ hci_dev_clear_flag(hdev, HCI_BREDR_ENABLED);
+ }
+
+ if (lmp_le_capable(hdev)) {
+ err = hci_init_stage_sync(hdev, le_init2);
+ if (err)
+ return err;
+ /* LE-only controllers have LE implicitly enabled */
+ if (!lmp_bredr_capable(hdev))
+ hci_dev_set_flag(hdev, HCI_LE_ENABLED);
+ }
+
+ return hci_init_stage_sync(hdev, hci_init2);
+}
+
+static int hci_set_event_mask_sync(struct hci_dev *hdev)
+{
+ /* The second byte is 0xff instead of 0x9f (two reserved bits
+ * disabled) since a Broadcom 1.2 dongle doesn't respond to the
+ * command otherwise.
+ */
+ u8 events[8] = { 0xff, 0xff, 0xfb, 0xff, 0x00, 0x00, 0x00, 0x00 };
+
+ /* CSR 1.1 dongles does not accept any bitfield so don't try to set
+ * any event mask for pre 1.2 devices.
+ */
+ if (hdev->hci_ver < BLUETOOTH_VER_1_2)
+ return 0;
+
+ if (lmp_bredr_capable(hdev)) {
+ events[4] |= 0x01; /* Flow Specification Complete */
+
+ /* Don't set Disconnect Complete when suspended as that
+ * would wakeup the host when disconnecting due to
+ * suspend.
+ */
+ if (hdev->suspended)
+ events[0] &= 0xef;
+ } else {
+ /* Use a different default for LE-only devices */
+ memset(events, 0, sizeof(events));
+ events[1] |= 0x20; /* Command Complete */
+ events[1] |= 0x40; /* Command Status */
+ events[1] |= 0x80; /* Hardware Error */
+
+ /* If the controller supports the Disconnect command, enable
+ * the corresponding event. In addition enable packet flow
+ * control related events.
+ */
+ if (hdev->commands[0] & 0x20) {
+ /* Don't set Disconnect Complete when suspended as that
+ * would wakeup the host when disconnecting due to
+ * suspend.
+ */
+ if (!hdev->suspended)
+ events[0] |= 0x10; /* Disconnection Complete */
+ events[2] |= 0x04; /* Number of Completed Packets */
+ events[3] |= 0x02; /* Data Buffer Overflow */
+ }
+
+ /* If the controller supports the Read Remote Version
+ * Information command, enable the corresponding event.
+ */
+ if (hdev->commands[2] & 0x80)
+ events[1] |= 0x08; /* Read Remote Version Information
+ * Complete
+ */
+
+ if (hdev->le_features[0] & HCI_LE_ENCRYPTION) {
+ events[0] |= 0x80; /* Encryption Change */
+ events[5] |= 0x80; /* Encryption Key Refresh Complete */
+ }
+ }
+
+ if (lmp_inq_rssi_capable(hdev) ||
+ test_bit(HCI_QUIRK_FIXUP_INQUIRY_MODE, &hdev->quirks))
+ events[4] |= 0x02; /* Inquiry Result with RSSI */
+
+ if (lmp_ext_feat_capable(hdev))
+ events[4] |= 0x04; /* Read Remote Extended Features Complete */
+
+ if (lmp_esco_capable(hdev)) {
+ events[5] |= 0x08; /* Synchronous Connection Complete */
+ events[5] |= 0x10; /* Synchronous Connection Changed */
+ }
+
+ if (lmp_sniffsubr_capable(hdev))
+ events[5] |= 0x20; /* Sniff Subrating */
+
+ if (lmp_pause_enc_capable(hdev))
+ events[5] |= 0x80; /* Encryption Key Refresh Complete */
+
+ if (lmp_ext_inq_capable(hdev))
+ events[5] |= 0x40; /* Extended Inquiry Result */
+
+ if (lmp_no_flush_capable(hdev))
+ events[7] |= 0x01; /* Enhanced Flush Complete */
+
+ if (lmp_lsto_capable(hdev))
+ events[6] |= 0x80; /* Link Supervision Timeout Changed */
+
+ if (lmp_ssp_capable(hdev)) {
+ events[6] |= 0x01; /* IO Capability Request */
+ events[6] |= 0x02; /* IO Capability Response */
+ events[6] |= 0x04; /* User Confirmation Request */
+ events[6] |= 0x08; /* User Passkey Request */
+ events[6] |= 0x10; /* Remote OOB Data Request */
+ events[6] |= 0x20; /* Simple Pairing Complete */
+ events[7] |= 0x04; /* User Passkey Notification */
+ events[7] |= 0x08; /* Keypress Notification */
+ events[7] |= 0x10; /* Remote Host Supported
+ * Features Notification
+ */
+ }
+
+ if (lmp_le_capable(hdev))
+ events[7] |= 0x20; /* LE Meta-Event */
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_SET_EVENT_MASK,
+ sizeof(events), events, HCI_CMD_TIMEOUT);
+}
+
+static int hci_read_stored_link_key_sync(struct hci_dev *hdev)
+{
+ struct hci_cp_read_stored_link_key cp;
+
+ if (!(hdev->commands[6] & 0x20) ||
+ test_bit(HCI_QUIRK_BROKEN_STORED_LINK_KEY, &hdev->quirks))
+ return 0;
+
+ memset(&cp, 0, sizeof(cp));
+ bacpy(&cp.bdaddr, BDADDR_ANY);
+ cp.read_all = 0x01;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_READ_STORED_LINK_KEY,
+ sizeof(cp), &cp, HCI_CMD_TIMEOUT);
+}
+
+static int hci_setup_link_policy_sync(struct hci_dev *hdev)
+{
+ struct hci_cp_write_def_link_policy cp;
+ u16 link_policy = 0;
+
+ if (!(hdev->commands[5] & 0x10))
+ return 0;
+
+ memset(&cp, 0, sizeof(cp));
+
+ if (lmp_rswitch_capable(hdev))
+ link_policy |= HCI_LP_RSWITCH;
+ if (lmp_hold_capable(hdev))
+ link_policy |= HCI_LP_HOLD;
+ if (lmp_sniff_capable(hdev))
+ link_policy |= HCI_LP_SNIFF;
+ if (lmp_park_capable(hdev))
+ link_policy |= HCI_LP_PARK;
+
+ cp.policy = cpu_to_le16(link_policy);
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_WRITE_DEF_LINK_POLICY,
+ sizeof(cp), &cp, HCI_CMD_TIMEOUT);
+}
+
+static int hci_read_page_scan_activity_sync(struct hci_dev *hdev)
+{
+ if (!(hdev->commands[8] & 0x01))
+ return 0;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_READ_PAGE_SCAN_ACTIVITY,
+ 0, NULL, HCI_CMD_TIMEOUT);
+}
+
+static int hci_read_def_err_data_reporting_sync(struct hci_dev *hdev)
+{
+ if (!(hdev->commands[18] & 0x04) ||
+ test_bit(HCI_QUIRK_BROKEN_ERR_DATA_REPORTING, &hdev->quirks))
+ return 0;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_READ_DEF_ERR_DATA_REPORTING,
+ 0, NULL, HCI_CMD_TIMEOUT);
+}
+
+static int hci_read_page_scan_type_sync(struct hci_dev *hdev)
+{
+ /* Some older Broadcom based Bluetooth 1.2 controllers do not
+ * support the Read Page Scan Type command. Check support for
+ * this command in the bit mask of supported commands.
+ */
+ if (!(hdev->commands[13] & 0x01))
+ return 0;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_READ_PAGE_SCAN_TYPE,
+ 0, NULL, HCI_CMD_TIMEOUT);
+}
+
+/* Read features beyond page 1 if available */
+static int hci_read_local_ext_features_all_sync(struct hci_dev *hdev)
+{
+ u8 page;
+ int err;
+
+ if (!lmp_ext_feat_capable(hdev))
+ return 0;
+
+ for (page = 2; page < HCI_MAX_PAGES && page <= hdev->max_page;
+ page++) {
+ err = hci_read_local_ext_features_sync(hdev, page);
+ if (err)
+ return err;
+ }
+
+ return 0;
+}
+
+/* HCI Controller init stage 3 command sequence */
+static const struct hci_init_stage hci_init3[] = {
+ /* HCI_OP_SET_EVENT_MASK */
+ HCI_INIT(hci_set_event_mask_sync),
+ /* HCI_OP_READ_STORED_LINK_KEY */
+ HCI_INIT(hci_read_stored_link_key_sync),
+ /* HCI_OP_WRITE_DEF_LINK_POLICY */
+ HCI_INIT(hci_setup_link_policy_sync),
+ /* HCI_OP_READ_PAGE_SCAN_ACTIVITY */
+ HCI_INIT(hci_read_page_scan_activity_sync),
+ /* HCI_OP_READ_DEF_ERR_DATA_REPORTING */
+ HCI_INIT(hci_read_def_err_data_reporting_sync),
+ /* HCI_OP_READ_PAGE_SCAN_TYPE */
+ HCI_INIT(hci_read_page_scan_type_sync),
+ /* HCI_OP_READ_LOCAL_EXT_FEATURES */
+ HCI_INIT(hci_read_local_ext_features_all_sync),
+ {}
+};
+
+static int hci_le_set_event_mask_sync(struct hci_dev *hdev)
+{
+ u8 events[8];
+
+ if (!lmp_le_capable(hdev))
+ return 0;
+
+ memset(events, 0, sizeof(events));
+
+ if (hdev->le_features[0] & HCI_LE_ENCRYPTION)
+ events[0] |= 0x10; /* LE Long Term Key Request */
+
+ /* If controller supports the Connection Parameters Request
+ * Link Layer Procedure, enable the corresponding event.
+ */
+ if (hdev->le_features[0] & HCI_LE_CONN_PARAM_REQ_PROC)
+ /* LE Remote Connection Parameter Request */
+ events[0] |= 0x20;
+
+ /* If the controller supports the Data Length Extension
+ * feature, enable the corresponding event.
+ */
+ if (hdev->le_features[0] & HCI_LE_DATA_LEN_EXT)
+ events[0] |= 0x40; /* LE Data Length Change */
+
+ /* If the controller supports LL Privacy feature, enable
+ * the corresponding event.
+ */
+ if (hdev->le_features[0] & HCI_LE_LL_PRIVACY)
+ events[1] |= 0x02; /* LE Enhanced Connection Complete */
+
+ /* If the controller supports Extended Scanner Filter
+ * Policies, enable the corresponding event.
+ */
+ if (hdev->le_features[0] & HCI_LE_EXT_SCAN_POLICY)
+ events[1] |= 0x04; /* LE Direct Advertising Report */
+
+ /* If the controller supports Channel Selection Algorithm #2
+ * feature, enable the corresponding event.
+ */
+ if (hdev->le_features[1] & HCI_LE_CHAN_SEL_ALG2)
+ events[2] |= 0x08; /* LE Channel Selection Algorithm */
+
+ /* If the controller supports the LE Set Scan Enable command,
+ * enable the corresponding advertising report event.
+ */
+ if (hdev->commands[26] & 0x08)
+ events[0] |= 0x02; /* LE Advertising Report */
+
+ /* If the controller supports the LE Create Connection
+ * command, enable the corresponding event.
+ */
+ if (hdev->commands[26] & 0x10)
+ events[0] |= 0x01; /* LE Connection Complete */
+
+ /* If the controller supports the LE Connection Update
+ * command, enable the corresponding event.
+ */
+ if (hdev->commands[27] & 0x04)
+ events[0] |= 0x04; /* LE Connection Update Complete */
+
+ /* If the controller supports the LE Read Remote Used Features
+ * command, enable the corresponding event.
+ */
+ if (hdev->commands[27] & 0x20)
+ /* LE Read Remote Used Features Complete */
+ events[0] |= 0x08;
+
+ /* If the controller supports the LE Read Local P-256
+ * Public Key command, enable the corresponding event.
+ */
+ if (hdev->commands[34] & 0x02)
+ /* LE Read Local P-256 Public Key Complete */
+ events[0] |= 0x80;
+
+ /* If the controller supports the LE Generate DHKey
+ * command, enable the corresponding event.
+ */
+ if (hdev->commands[34] & 0x04)
+ events[1] |= 0x01; /* LE Generate DHKey Complete */
+
+ /* If the controller supports the LE Set Default PHY or
+ * LE Set PHY commands, enable the corresponding event.
+ */
+ if (hdev->commands[35] & (0x20 | 0x40))
+ events[1] |= 0x08; /* LE PHY Update Complete */
+
+ /* If the controller supports LE Set Extended Scan Parameters
+ * and LE Set Extended Scan Enable commands, enable the
+ * corresponding event.
+ */
+ if (use_ext_scan(hdev))
+ events[1] |= 0x10; /* LE Extended Advertising Report */
+
+ /* If the controller supports the LE Extended Advertising
+ * command, enable the corresponding event.
+ */
+ if (ext_adv_capable(hdev))
+ events[2] |= 0x02; /* LE Advertising Set Terminated */
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_LE_SET_EVENT_MASK,
+ sizeof(events), events, HCI_CMD_TIMEOUT);
+}
+
+/* Read LE Advertising Channel TX Power */
+static int hci_le_read_adv_tx_power_sync(struct hci_dev *hdev)
+{
+ if ((hdev->commands[25] & 0x40) && !ext_adv_capable(hdev)) {
+ /* HCI TS spec forbids mixing of legacy and extended
+ * advertising commands wherein READ_ADV_TX_POWER is
+ * also included. So do not call it if extended adv
+ * is supported otherwise controller will return
+ * COMMAND_DISALLOWED for extended commands.
+ */
+ return __hci_cmd_sync_status(hdev,
+ HCI_OP_LE_READ_ADV_TX_POWER,
+ 0, NULL, HCI_CMD_TIMEOUT);
+ }
+
+ return 0;
+}
+
+/* Read LE Min/Max Tx Power*/
+static int hci_le_read_tx_power_sync(struct hci_dev *hdev)
+{
+ if (!(hdev->commands[38] & 0x80) ||
+ test_bit(HCI_QUIRK_BROKEN_READ_TRANSMIT_POWER, &hdev->quirks))
+ return 0;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_LE_READ_TRANSMIT_POWER,
+ 0, NULL, HCI_CMD_TIMEOUT);
+}
+
+/* Read LE Accept List Size */
+static int hci_le_read_accept_list_size_sync(struct hci_dev *hdev)
+{
+ if (!(hdev->commands[26] & 0x40))
+ return 0;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_LE_READ_ACCEPT_LIST_SIZE,
+ 0, NULL, HCI_CMD_TIMEOUT);
+}
+
+/* Clear LE Accept List */
+static int hci_le_clear_accept_list_sync(struct hci_dev *hdev)
+{
+ if (!(hdev->commands[26] & 0x80))
+ return 0;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_LE_CLEAR_ACCEPT_LIST, 0, NULL,
+ HCI_CMD_TIMEOUT);
+}
+
+/* Read LE Resolving List Size */
+static int hci_le_read_resolv_list_size_sync(struct hci_dev *hdev)
+{
+ if (!(hdev->commands[34] & 0x40))
+ return 0;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_LE_READ_RESOLV_LIST_SIZE,
+ 0, NULL, HCI_CMD_TIMEOUT);
+}
+
+/* Clear LE Resolving List */
+static int hci_le_clear_resolv_list_sync(struct hci_dev *hdev)
+{
+ if (!(hdev->commands[34] & 0x20))
+ return 0;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_LE_CLEAR_RESOLV_LIST, 0, NULL,
+ HCI_CMD_TIMEOUT);
+}
+
+/* Set RPA timeout */
+static int hci_le_set_rpa_timeout_sync(struct hci_dev *hdev)
+{
+ __le16 timeout = cpu_to_le16(hdev->rpa_timeout);
+
+ if (!(hdev->commands[35] & 0x04))
+ return 0;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_LE_SET_RPA_TIMEOUT,
+ sizeof(timeout), &timeout,
+ HCI_CMD_TIMEOUT);
+}
+
+/* Read LE Maximum Data Length */
+static int hci_le_read_max_data_len_sync(struct hci_dev *hdev)
+{
+ if (!(hdev->le_features[0] & HCI_LE_DATA_LEN_EXT))
+ return 0;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_LE_READ_MAX_DATA_LEN, 0, NULL,
+ HCI_CMD_TIMEOUT);
+}
+
+/* Read LE Suggested Default Data Length */
+static int hci_le_read_def_data_len_sync(struct hci_dev *hdev)
+{
+ if (!(hdev->le_features[0] & HCI_LE_DATA_LEN_EXT))
+ return 0;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_LE_READ_DEF_DATA_LEN, 0, NULL,
+ HCI_CMD_TIMEOUT);
+}
+
+/* Read LE Number of Supported Advertising Sets */
+static int hci_le_read_num_support_adv_sets_sync(struct hci_dev *hdev)
+{
+ if (!ext_adv_capable(hdev))
+ return 0;
+
+ return __hci_cmd_sync_status(hdev,
+ HCI_OP_LE_READ_NUM_SUPPORTED_ADV_SETS,
+ 0, NULL, HCI_CMD_TIMEOUT);
+}
+
+/* Write LE Host Supported */
+static int hci_set_le_support_sync(struct hci_dev *hdev)
+{
+ struct hci_cp_write_le_host_supported cp;
+
+ /* LE-only devices do not support explicit enablement */
+ if (!lmp_bredr_capable(hdev))
+ return 0;
+
+ memset(&cp, 0, sizeof(cp));
+
+ if (hci_dev_test_flag(hdev, HCI_LE_ENABLED)) {
+ cp.le = 0x01;
+ cp.simul = 0x00;
+ }
+
+ if (cp.le == lmp_host_le_capable(hdev))
+ return 0;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_WRITE_LE_HOST_SUPPORTED,
+ sizeof(cp), &cp, HCI_CMD_TIMEOUT);
+}
+
+/* LE Controller init stage 3 command sequence */
+static const struct hci_init_stage le_init3[] = {
+ /* HCI_OP_LE_SET_EVENT_MASK */
+ HCI_INIT(hci_le_set_event_mask_sync),
+ /* HCI_OP_LE_READ_ADV_TX_POWER */
+ HCI_INIT(hci_le_read_adv_tx_power_sync),
+ /* HCI_OP_LE_READ_TRANSMIT_POWER */
+ HCI_INIT(hci_le_read_tx_power_sync),
+ /* HCI_OP_LE_READ_ACCEPT_LIST_SIZE */
+ HCI_INIT(hci_le_read_accept_list_size_sync),
+ /* HCI_OP_LE_CLEAR_ACCEPT_LIST */
+ HCI_INIT(hci_le_clear_accept_list_sync),
+ /* HCI_OP_LE_READ_RESOLV_LIST_SIZE */
+ HCI_INIT(hci_le_read_resolv_list_size_sync),
+ /* HCI_OP_LE_CLEAR_RESOLV_LIST */
+ HCI_INIT(hci_le_clear_resolv_list_sync),
+ /* HCI_OP_LE_SET_RPA_TIMEOUT */
+ HCI_INIT(hci_le_set_rpa_timeout_sync),
+ /* HCI_OP_LE_READ_MAX_DATA_LEN */
+ HCI_INIT(hci_le_read_max_data_len_sync),
+ /* HCI_OP_LE_READ_DEF_DATA_LEN */
+ HCI_INIT(hci_le_read_def_data_len_sync),
+ /* HCI_OP_LE_READ_NUM_SUPPORTED_ADV_SETS */
+ HCI_INIT(hci_le_read_num_support_adv_sets_sync),
+ /* HCI_OP_WRITE_LE_HOST_SUPPORTED */
+ HCI_INIT(hci_set_le_support_sync),
+ {}
+};
+
+static int hci_init3_sync(struct hci_dev *hdev)
+{
+ int err;
+
+ bt_dev_dbg(hdev, "");
+
+ err = hci_init_stage_sync(hdev, hci_init3);
+ if (err)
+ return err;
+
+ if (lmp_le_capable(hdev))
+ return hci_init_stage_sync(hdev, le_init3);
+
+ return 0;
+}
+
+static int hci_delete_stored_link_key_sync(struct hci_dev *hdev)
+{
+ struct hci_cp_delete_stored_link_key cp;
+
+ /* Some Broadcom based Bluetooth controllers do not support the
+ * Delete Stored Link Key command. They are clearly indicating its
+ * absence in the bit mask of supported commands.
+ *
+ * Check the supported commands and only if the command is marked
+ * as supported send it. If not supported assume that the controller
+ * does not have actual support for stored link keys which makes this
+ * command redundant anyway.
+ *
+ * Some controllers indicate that they support handling deleting
+ * stored link keys, but they don't. The quirk lets a driver
+ * just disable this command.
+ */
+ if (!(hdev->commands[6] & 0x80) ||
+ test_bit(HCI_QUIRK_BROKEN_STORED_LINK_KEY, &hdev->quirks))
+ return 0;
+
+ memset(&cp, 0, sizeof(cp));
+ bacpy(&cp.bdaddr, BDADDR_ANY);
+ cp.delete_all = 0x01;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_DELETE_STORED_LINK_KEY,
+ sizeof(cp), &cp, HCI_CMD_TIMEOUT);
+}
+
+static int hci_set_event_mask_page_2_sync(struct hci_dev *hdev)
+{
+ u8 events[8] = { 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 };
+ bool changed = false;
+
+ /* Set event mask page 2 if the HCI command for it is supported */
+ if (!(hdev->commands[22] & 0x04))
+ return 0;
+
+ /* If Connectionless Peripheral Broadcast central role is supported
+ * enable all necessary events for it.
+ */
+ if (lmp_cpb_central_capable(hdev)) {
+ events[1] |= 0x40; /* Triggered Clock Capture */
+ events[1] |= 0x80; /* Synchronization Train Complete */
+ events[2] |= 0x10; /* Peripheral Page Response Timeout */
+ events[2] |= 0x20; /* CPB Channel Map Change */
+ changed = true;
+ }
+
+ /* If Connectionless Peripheral Broadcast peripheral role is supported
+ * enable all necessary events for it.
+ */
+ if (lmp_cpb_peripheral_capable(hdev)) {
+ events[2] |= 0x01; /* Synchronization Train Received */
+ events[2] |= 0x02; /* CPB Receive */
+ events[2] |= 0x04; /* CPB Timeout */
+ events[2] |= 0x08; /* Truncated Page Complete */
+ changed = true;
+ }
+
+ /* Enable Authenticated Payload Timeout Expired event if supported */
+ if (lmp_ping_capable(hdev) || hdev->le_features[0] & HCI_LE_PING) {
+ events[2] |= 0x80;
+ changed = true;
+ }
+
+ /* Some Broadcom based controllers indicate support for Set Event
+ * Mask Page 2 command, but then actually do not support it. Since
+ * the default value is all bits set to zero, the command is only
+ * required if the event mask has to be changed. In case no change
+ * to the event mask is needed, skip this command.
+ */
+ if (!changed)
+ return 0;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_SET_EVENT_MASK_PAGE_2,
+ sizeof(events), events, HCI_CMD_TIMEOUT);
+}
+
+/* Read local codec list if the HCI command is supported */
+static int hci_read_local_codecs_sync(struct hci_dev *hdev)
+{
+ if (!(hdev->commands[29] & 0x20))
+ return 0;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_READ_LOCAL_CODECS, 0, NULL,
+ HCI_CMD_TIMEOUT);
+}
+
+/* Read local pairing options if the HCI command is supported */
+static int hci_read_local_pairing_opts_sync(struct hci_dev *hdev)
+{
+ if (!(hdev->commands[41] & 0x08))
+ return 0;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_READ_LOCAL_PAIRING_OPTS,
+ 0, NULL, HCI_CMD_TIMEOUT);
+}
+
+/* Get MWS transport configuration if the HCI command is supported */
+static int hci_get_mws_transport_config_sync(struct hci_dev *hdev)
+{
+ if (!(hdev->commands[30] & 0x08))
+ return 0;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_GET_MWS_TRANSPORT_CONFIG,
+ 0, NULL, HCI_CMD_TIMEOUT);
+}
+
+/* Check for Synchronization Train support */
+static int hci_read_sync_train_params_sync(struct hci_dev *hdev)
+{
+ if (!lmp_sync_train_capable(hdev))
+ return 0;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_READ_SYNC_TRAIN_PARAMS,
+ 0, NULL, HCI_CMD_TIMEOUT);
+}
+
+/* Enable Secure Connections if supported and configured */
+static int hci_write_sc_support_1_sync(struct hci_dev *hdev)
+{
+ u8 support = 0x01;
+
+ if (!hci_dev_test_flag(hdev, HCI_SSP_ENABLED) ||
+ !bredr_sc_enabled(hdev))
+ return 0;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_WRITE_SC_SUPPORT,
+ sizeof(support), &support,
+ HCI_CMD_TIMEOUT);
+}
+
+/* Set erroneous data reporting if supported to the wideband speech
+ * setting value
+ */
+static int hci_set_err_data_report_sync(struct hci_dev *hdev)
+{
+ struct hci_cp_write_def_err_data_reporting cp;
+ bool enabled = hci_dev_test_flag(hdev, HCI_WIDEBAND_SPEECH_ENABLED);
+
+ if (!(hdev->commands[18] & 0x08) ||
+ test_bit(HCI_QUIRK_BROKEN_ERR_DATA_REPORTING, &hdev->quirks))
+ return 0;
+
+ if (enabled == hdev->err_data_reporting)
+ return 0;
+
+ memset(&cp, 0, sizeof(cp));
+ cp.err_data_reporting = enabled ? ERR_DATA_REPORTING_ENABLED :
+ ERR_DATA_REPORTING_DISABLED;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_WRITE_DEF_ERR_DATA_REPORTING,
+ sizeof(cp), &cp, HCI_CMD_TIMEOUT);
+}
+
+static const struct hci_init_stage hci_init4[] = {
+ /* HCI_OP_DELETE_STORED_LINK_KEY */
+ HCI_INIT(hci_delete_stored_link_key_sync),
+ /* HCI_OP_SET_EVENT_MASK_PAGE_2 */
+ HCI_INIT(hci_set_event_mask_page_2_sync),
+ /* HCI_OP_READ_LOCAL_CODECS */
+ HCI_INIT(hci_read_local_codecs_sync),
+ /* HCI_OP_READ_LOCAL_PAIRING_OPTS */
+ HCI_INIT(hci_read_local_pairing_opts_sync),
+ /* HCI_OP_GET_MWS_TRANSPORT_CONFIG */
+ HCI_INIT(hci_get_mws_transport_config_sync),
+ /* HCI_OP_READ_SYNC_TRAIN_PARAMS */
+ HCI_INIT(hci_read_sync_train_params_sync),
+ /* HCI_OP_WRITE_SC_SUPPORT */
+ HCI_INIT(hci_write_sc_support_1_sync),
+ /* HCI_OP_WRITE_DEF_ERR_DATA_REPORTING */
+ HCI_INIT(hci_set_err_data_report_sync),
+ {}
+};
+
+/* Set Suggested Default Data Length to maximum if supported */
+static int hci_le_set_write_def_data_len_sync(struct hci_dev *hdev)
+{
+ struct hci_cp_le_write_def_data_len cp;
+
+ if (!(hdev->le_features[0] & HCI_LE_DATA_LEN_EXT))
+ return 0;
+
+ memset(&cp, 0, sizeof(cp));
+ cp.tx_len = cpu_to_le16(hdev->le_max_tx_len);
+ cp.tx_time = cpu_to_le16(hdev->le_max_tx_time);
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_LE_WRITE_DEF_DATA_LEN,
+ sizeof(cp), &cp, HCI_CMD_TIMEOUT);
+}
+
+/* Set Default PHY parameters if command is supported */
+static int hci_le_set_default_phy_sync(struct hci_dev *hdev)
+{
+ struct hci_cp_le_set_default_phy cp;
+
+ if (!(hdev->commands[35] & 0x20))
+ return 0;
+
+ memset(&cp, 0, sizeof(cp));
+ cp.all_phys = 0x00;
+ cp.tx_phys = hdev->le_tx_def_phys;
+ cp.rx_phys = hdev->le_rx_def_phys;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_LE_SET_DEFAULT_PHY,
+ sizeof(cp), &cp, HCI_CMD_TIMEOUT);
+}
+
+static const struct hci_init_stage le_init4[] = {
+ /* HCI_OP_LE_WRITE_DEF_DATA_LEN */
+ HCI_INIT(hci_le_set_write_def_data_len_sync),
+ /* HCI_OP_LE_SET_DEFAULT_PHY */
+ HCI_INIT(hci_le_set_default_phy_sync),
+ {}
+};
+
+static int hci_init4_sync(struct hci_dev *hdev)
+{
+ int err;
+
+ bt_dev_dbg(hdev, "");
+
+ err = hci_init_stage_sync(hdev, hci_init4);
+ if (err)
+ return err;
+
+ if (lmp_le_capable(hdev))
+ return hci_init_stage_sync(hdev, le_init4);
+
+ return 0;
+}
+
+static int hci_init_sync(struct hci_dev *hdev)
+{
+ int err;
+
+ err = hci_init1_sync(hdev);
+ if (err < 0)
+ return err;
+
+ if (hci_dev_test_flag(hdev, HCI_SETUP))
+ hci_debugfs_create_basic(hdev);
+
+ err = hci_init2_sync(hdev);
+ if (err < 0)
+ return err;
+
+ /* HCI_PRIMARY covers both single-mode LE, BR/EDR and dual-mode
+ * BR/EDR/LE type controllers. AMP controllers only need the
+ * first two stages of init.
+ */
+ if (hdev->dev_type != HCI_PRIMARY)
+ return 0;
+
+ err = hci_init3_sync(hdev);
+ if (err < 0)
+ return err;
+
+ err = hci_init4_sync(hdev);
+ if (err < 0)
+ return err;
+
+ /* This function is only called when the controller is actually in
+ * configured state. When the controller is marked as unconfigured,
+ * this initialization procedure is not run.
+ *
+ * It means that it is possible that a controller runs through its
+ * setup phase and then discovers missing settings. If that is the
+ * case, then this function will not be called. It then will only
+ * be called during the config phase.
+ *
+ * So only when in setup phase or config phase, create the debugfs
+ * entries and register the SMP channels.
+ */
+ if (!hci_dev_test_flag(hdev, HCI_SETUP) &&
+ !hci_dev_test_flag(hdev, HCI_CONFIG))
+ return 0;
+
+ hci_debugfs_create_common(hdev);
+
+ if (lmp_bredr_capable(hdev))
+ hci_debugfs_create_bredr(hdev);
+
+ if (lmp_le_capable(hdev))
+ hci_debugfs_create_le(hdev);
+
+ return 0;
+}
+
+int hci_dev_open_sync(struct hci_dev *hdev)
+{
+ int ret = 0;
+
+ bt_dev_dbg(hdev, "");
+
+ if (hci_dev_test_flag(hdev, HCI_UNREGISTER)) {
+ ret = -ENODEV;
+ goto done;
+ }
+
+ if (!hci_dev_test_flag(hdev, HCI_SETUP) &&
+ !hci_dev_test_flag(hdev, HCI_CONFIG)) {
+ /* Check for rfkill but allow the HCI setup stage to
+ * proceed (which in itself doesn't cause any RF activity).
+ */
+ if (hci_dev_test_flag(hdev, HCI_RFKILLED)) {
+ ret = -ERFKILL;
+ goto done;
+ }
+
+ /* Check for valid public address or a configured static
+ * random address, but let the HCI setup proceed to
+ * be able to determine if there is a public address
+ * or not.
+ *
+ * In case of user channel usage, it is not important
+ * if a public address or static random address is
+ * available.
+ *
+ * This check is only valid for BR/EDR controllers
+ * since AMP controllers do not have an address.
+ */
+ if (!hci_dev_test_flag(hdev, HCI_USER_CHANNEL) &&
+ hdev->dev_type == HCI_PRIMARY &&
+ !bacmp(&hdev->bdaddr, BDADDR_ANY) &&
+ !bacmp(&hdev->static_addr, BDADDR_ANY)) {
+ ret = -EADDRNOTAVAIL;
+ goto done;
+ }
+ }
+
+ if (test_bit(HCI_UP, &hdev->flags)) {
+ ret = -EALREADY;
+ goto done;
+ }
+
+ if (hdev->open(hdev)) {
+ ret = -EIO;
+ goto done;
+ }
+
+ set_bit(HCI_RUNNING, &hdev->flags);
+ hci_sock_dev_event(hdev, HCI_DEV_OPEN);
+
+ atomic_set(&hdev->cmd_cnt, 1);
+ set_bit(HCI_INIT, &hdev->flags);
+
+ if (hci_dev_test_flag(hdev, HCI_SETUP) ||
+ test_bit(HCI_QUIRK_NON_PERSISTENT_SETUP, &hdev->quirks)) {
+ bool invalid_bdaddr;
+
+ hci_sock_dev_event(hdev, HCI_DEV_SETUP);
+
+ if (hdev->setup)
+ ret = hdev->setup(hdev);
+
+ /* The transport driver can set the quirk to mark the
+ * BD_ADDR invalid before creating the HCI device or in
+ * its setup callback.
+ */
+ invalid_bdaddr = test_bit(HCI_QUIRK_INVALID_BDADDR,
+ &hdev->quirks);
+
+ if (ret)
+ goto setup_failed;
+
+ if (test_bit(HCI_QUIRK_USE_BDADDR_PROPERTY, &hdev->quirks)) {
+ if (!bacmp(&hdev->public_addr, BDADDR_ANY))
+ hci_dev_get_bd_addr_from_property(hdev);
+
+ if (bacmp(&hdev->public_addr, BDADDR_ANY) &&
+ hdev->set_bdaddr) {
+ ret = hdev->set_bdaddr(hdev,
+ &hdev->public_addr);
+
+ /* If setting of the BD_ADDR from the device
+ * property succeeds, then treat the address
+ * as valid even if the invalid BD_ADDR
+ * quirk indicates otherwise.
+ */
+ if (!ret)
+ invalid_bdaddr = false;
+ }
+ }
+
+setup_failed:
+ /* The transport driver can set these quirks before
+ * creating the HCI device or in its setup callback.
+ *
+ * For the invalid BD_ADDR quirk it is possible that
+ * it becomes a valid address if the bootloader does
+ * provide it (see above).
+ *
+ * In case any of them is set, the controller has to
+ * start up as unconfigured.
+ */
+ if (test_bit(HCI_QUIRK_EXTERNAL_CONFIG, &hdev->quirks) ||
+ invalid_bdaddr)
+ hci_dev_set_flag(hdev, HCI_UNCONFIGURED);
+
+ /* For an unconfigured controller it is required to
+ * read at least the version information provided by
+ * the Read Local Version Information command.
+ *
+ * If the set_bdaddr driver callback is provided, then
+ * also the original Bluetooth public device address
+ * will be read using the Read BD Address command.
+ */
+ if (hci_dev_test_flag(hdev, HCI_UNCONFIGURED))
+ ret = hci_unconf_init_sync(hdev);
+ }
+
+ if (hci_dev_test_flag(hdev, HCI_CONFIG)) {
+ /* If public address change is configured, ensure that
+ * the address gets programmed. If the driver does not
+ * support changing the public address, fail the power
+ * on procedure.
+ */
+ if (bacmp(&hdev->public_addr, BDADDR_ANY) &&
+ hdev->set_bdaddr)
+ ret = hdev->set_bdaddr(hdev, &hdev->public_addr);
+ else
+ ret = -EADDRNOTAVAIL;
+ }
+
+ if (!ret) {
+ if (!hci_dev_test_flag(hdev, HCI_UNCONFIGURED) &&
+ !hci_dev_test_flag(hdev, HCI_USER_CHANNEL)) {
+ ret = hci_init_sync(hdev);
+ if (!ret && hdev->post_init)
+ ret = hdev->post_init(hdev);
+ }
+ }
+
+ /* If the HCI Reset command is clearing all diagnostic settings,
+ * then they need to be reprogrammed after the init procedure
+ * completed.
+ */
+ if (test_bit(HCI_QUIRK_NON_PERSISTENT_DIAG, &hdev->quirks) &&
+ !hci_dev_test_flag(hdev, HCI_USER_CHANNEL) &&
+ hci_dev_test_flag(hdev, HCI_VENDOR_DIAG) && hdev->set_diag)
+ ret = hdev->set_diag(hdev, true);
+
+ if (!hci_dev_test_flag(hdev, HCI_USER_CHANNEL)) {
+ msft_do_open(hdev);
+ aosp_do_open(hdev);
+ }
+
+ clear_bit(HCI_INIT, &hdev->flags);
+
+ if (!ret) {
+ hci_dev_hold(hdev);
+ hci_dev_set_flag(hdev, HCI_RPA_EXPIRED);
+ hci_adv_instances_set_rpa_expired(hdev, true);
+ set_bit(HCI_UP, &hdev->flags);
+ hci_sock_dev_event(hdev, HCI_DEV_UP);
+ hci_leds_update_powered(hdev, true);
+ if (!hci_dev_test_flag(hdev, HCI_SETUP) &&
+ !hci_dev_test_flag(hdev, HCI_CONFIG) &&
+ !hci_dev_test_flag(hdev, HCI_UNCONFIGURED) &&
+ !hci_dev_test_flag(hdev, HCI_USER_CHANNEL) &&
+ hci_dev_test_flag(hdev, HCI_MGMT) &&
+ hdev->dev_type == HCI_PRIMARY) {
+ ret = hci_powered_update_sync(hdev);
+ }
+ } else {
+ /* Init failed, cleanup */
+ flush_work(&hdev->tx_work);
+
+ /* Since hci_rx_work() is possible to awake new cmd_work
+ * it should be flushed first to avoid unexpected call of
+ * hci_cmd_work()
+ */
+ flush_work(&hdev->rx_work);
+ flush_work(&hdev->cmd_work);
+
+ skb_queue_purge(&hdev->cmd_q);
+ skb_queue_purge(&hdev->rx_q);
+
+ if (hdev->flush)
+ hdev->flush(hdev);
+
+ if (hdev->sent_cmd) {
+ kfree_skb(hdev->sent_cmd);
+ hdev->sent_cmd = NULL;
+ }
+
+ clear_bit(HCI_RUNNING, &hdev->flags);
+ hci_sock_dev_event(hdev, HCI_DEV_CLOSE);
+
+ hdev->close(hdev);
+ hdev->flags &= BIT(HCI_RAW);
+ }
+
+done:
+ return ret;
+}
+
+/* This function requires the caller holds hdev->lock */
+static void hci_pend_le_actions_clear(struct hci_dev *hdev)
+{
+ struct hci_conn_params *p;
+
+ list_for_each_entry(p, &hdev->le_conn_params, list) {
+ if (p->conn) {
+ hci_conn_drop(p->conn);
+ hci_conn_put(p->conn);
+ p->conn = NULL;
+ }
+ list_del_init(&p->action);
+ }
+
+ BT_DBG("All LE pending actions cleared");
+}
+
+int hci_dev_close_sync(struct hci_dev *hdev)
+{
+ bool auto_off;
+ int err = 0;
+
+ bt_dev_dbg(hdev, "");
+
+ cancel_delayed_work(&hdev->power_off);
+ cancel_delayed_work(&hdev->ncmd_timer);
+
+ hci_request_cancel_all(hdev);
+
+ if (!hci_dev_test_flag(hdev, HCI_UNREGISTER) &&
+ !hci_dev_test_flag(hdev, HCI_USER_CHANNEL) &&
+ test_bit(HCI_UP, &hdev->flags)) {
+ /* Execute vendor specific shutdown routine */
+ if (hdev->shutdown)
+ err = hdev->shutdown(hdev);
+ }
+
+ if (!test_and_clear_bit(HCI_UP, &hdev->flags)) {
+ cancel_delayed_work_sync(&hdev->cmd_timer);
+ return err;
+ }
+
+ hci_leds_update_powered(hdev, false);
+
+ /* Flush RX and TX works */
+ flush_work(&hdev->tx_work);
+ flush_work(&hdev->rx_work);
+
+ if (hdev->discov_timeout > 0) {
+ hdev->discov_timeout = 0;
+ hci_dev_clear_flag(hdev, HCI_DISCOVERABLE);
+ hci_dev_clear_flag(hdev, HCI_LIMITED_DISCOVERABLE);
+ }
+
+ if (hci_dev_test_and_clear_flag(hdev, HCI_SERVICE_CACHE))
+ cancel_delayed_work(&hdev->service_cache);
+
+ if (hci_dev_test_flag(hdev, HCI_MGMT)) {
+ struct adv_info *adv_instance;
+
+ cancel_delayed_work_sync(&hdev->rpa_expired);
+
+ list_for_each_entry(adv_instance, &hdev->adv_instances, list)
+ cancel_delayed_work_sync(&adv_instance->rpa_expired_cb);
+ }
+
+ /* Avoid potential lockdep warnings from the *_flush() calls by
+ * ensuring the workqueue is empty up front.
+ */
+ drain_workqueue(hdev->workqueue);
+
+ hci_dev_lock(hdev);
+
+ hci_discovery_set_state(hdev, DISCOVERY_STOPPED);
+
+ auto_off = hci_dev_test_and_clear_flag(hdev, HCI_AUTO_OFF);
+
+ if (!auto_off && hdev->dev_type == HCI_PRIMARY &&
+ !hci_dev_test_flag(hdev, HCI_USER_CHANNEL) &&
+ hci_dev_test_flag(hdev, HCI_MGMT))
+ __mgmt_power_off(hdev);
+
+ hci_inquiry_cache_flush(hdev);
+ hci_pend_le_actions_clear(hdev);
+ hci_conn_hash_flush(hdev);
+ hci_dev_unlock(hdev);
+
+ smp_unregister(hdev);
+
+ hci_sock_dev_event(hdev, HCI_DEV_DOWN);
+
+ if (!hci_dev_test_flag(hdev, HCI_USER_CHANNEL)) {
+ aosp_do_close(hdev);
+ msft_do_close(hdev);
+ }
+
+ if (hdev->flush)
+ hdev->flush(hdev);
+
+ /* Reset device */
+ skb_queue_purge(&hdev->cmd_q);
+ atomic_set(&hdev->cmd_cnt, 1);
+ if (test_bit(HCI_QUIRK_RESET_ON_CLOSE, &hdev->quirks) &&
+ !auto_off && !hci_dev_test_flag(hdev, HCI_UNCONFIGURED)) {
+ set_bit(HCI_INIT, &hdev->flags);
+ hci_reset_sync(hdev);
+ clear_bit(HCI_INIT, &hdev->flags);
+ }
+
+ /* flush cmd work */
+ flush_work(&hdev->cmd_work);
+
+ /* Drop queues */
+ skb_queue_purge(&hdev->rx_q);
+ skb_queue_purge(&hdev->cmd_q);
+ skb_queue_purge(&hdev->raw_q);
+
+ /* Drop last sent command */
+ if (hdev->sent_cmd) {
+ cancel_delayed_work_sync(&hdev->cmd_timer);
+ kfree_skb(hdev->sent_cmd);
+ hdev->sent_cmd = NULL;
+ }
+
+ clear_bit(HCI_RUNNING, &hdev->flags);
+ hci_sock_dev_event(hdev, HCI_DEV_CLOSE);
+
+ /* After this point our queues are empty and no tasks are scheduled. */
+ hdev->close(hdev);
+
+ /* Clear flags */
+ hdev->flags &= BIT(HCI_RAW);
+ hci_dev_clear_volatile_flags(hdev);
+
+ /* Controller radio is available but is currently powered down */
+ hdev->amp_status = AMP_STATUS_POWERED_DOWN;
+
+ memset(hdev->eir, 0, sizeof(hdev->eir));
+ memset(hdev->dev_class, 0, sizeof(hdev->dev_class));
+ bacpy(&hdev->random_addr, BDADDR_ANY);
+
+ hci_dev_put(hdev);
+ return err;
+}
+
+/* This function perform power on HCI command sequence as follows:
+ *
+ * If controller is already up (HCI_UP) performs hci_powered_update_sync
+ * sequence otherwise run hci_dev_open_sync which will follow with
+ * hci_powered_update_sync after the init sequence is completed.
+ */
+static int hci_power_on_sync(struct hci_dev *hdev)
+{
+ int err;
+
+ if (test_bit(HCI_UP, &hdev->flags) &&
+ hci_dev_test_flag(hdev, HCI_MGMT) &&
+ hci_dev_test_and_clear_flag(hdev, HCI_AUTO_OFF)) {
+ cancel_delayed_work(&hdev->power_off);
+ return hci_powered_update_sync(hdev);
+ }
+
+ err = hci_dev_open_sync(hdev);
+ if (err < 0)
+ return err;
+
+ /* During the HCI setup phase, a few error conditions are
+ * ignored and they need to be checked now. If they are still
+ * valid, it is important to return the device back off.
+ */
+ if (hci_dev_test_flag(hdev, HCI_RFKILLED) ||
+ hci_dev_test_flag(hdev, HCI_UNCONFIGURED) ||
+ (hdev->dev_type == HCI_PRIMARY &&
+ !bacmp(&hdev->bdaddr, BDADDR_ANY) &&
+ !bacmp(&hdev->static_addr, BDADDR_ANY))) {
+ hci_dev_clear_flag(hdev, HCI_AUTO_OFF);
+ hci_dev_close_sync(hdev);
+ } else if (hci_dev_test_flag(hdev, HCI_AUTO_OFF)) {
+ queue_delayed_work(hdev->req_workqueue, &hdev->power_off,
+ HCI_AUTO_OFF_TIMEOUT);
+ }
+
+ if (hci_dev_test_and_clear_flag(hdev, HCI_SETUP)) {
+ /* For unconfigured devices, set the HCI_RAW flag
+ * so that userspace can easily identify them.
+ */
+ if (hci_dev_test_flag(hdev, HCI_UNCONFIGURED))
+ set_bit(HCI_RAW, &hdev->flags);
+
+ /* For fully configured devices, this will send
+ * the Index Added event. For unconfigured devices,
+ * it will send Unconfigued Index Added event.
+ *
+ * Devices with HCI_QUIRK_RAW_DEVICE are ignored
+ * and no event will be send.
+ */
+ mgmt_index_added(hdev);
+ } else if (hci_dev_test_and_clear_flag(hdev, HCI_CONFIG)) {
+ /* When the controller is now configured, then it
+ * is important to clear the HCI_RAW flag.
+ */
+ if (!hci_dev_test_flag(hdev, HCI_UNCONFIGURED))
+ clear_bit(HCI_RAW, &hdev->flags);
+
+ /* Powering on the controller with HCI_CONFIG set only
+ * happens with the transition from unconfigured to
+ * configured. This will send the Index Added event.
+ */
+ mgmt_index_added(hdev);
+ }
+
+ return 0;
+}
+
+static int hci_remote_name_cancel_sync(struct hci_dev *hdev, bdaddr_t *addr)
+{
+ struct hci_cp_remote_name_req_cancel cp;
+
+ memset(&cp, 0, sizeof(cp));
+ bacpy(&cp.bdaddr, addr);
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_REMOTE_NAME_REQ_CANCEL,
+ sizeof(cp), &cp, HCI_CMD_TIMEOUT);
+}
+
+int hci_stop_discovery_sync(struct hci_dev *hdev)
+{
+ struct discovery_state *d = &hdev->discovery;
+ struct inquiry_entry *e;
+ int err;
+
+ bt_dev_dbg(hdev, "state %u", hdev->discovery.state);
+
+ if (d->state == DISCOVERY_FINDING || d->state == DISCOVERY_STOPPING) {
+ if (test_bit(HCI_INQUIRY, &hdev->flags)) {
+ err = __hci_cmd_sync_status(hdev, HCI_OP_INQUIRY_CANCEL,
+ 0, NULL, HCI_CMD_TIMEOUT);
+ if (err)
+ return err;
+ }
+
+ if (hci_dev_test_flag(hdev, HCI_LE_SCAN)) {
+ cancel_delayed_work(&hdev->le_scan_disable);
+ cancel_delayed_work(&hdev->le_scan_restart);
+
+ err = hci_scan_disable_sync(hdev);
+ if (err)
+ return err;
+ }
+
+ } else {
+ err = hci_scan_disable_sync(hdev);
+ if (err)
+ return err;
+ }
+
+ /* Resume advertising if it was paused */
+ if (use_ll_privacy(hdev))
+ hci_resume_advertising_sync(hdev);
+
+ /* No further actions needed for LE-only discovery */
+ if (d->type == DISCOV_TYPE_LE)
+ return 0;
+
+ if (d->state == DISCOVERY_RESOLVING || d->state == DISCOVERY_STOPPING) {
+ e = hci_inquiry_cache_lookup_resolve(hdev, BDADDR_ANY,
+ NAME_PENDING);
+ if (!e)
+ return 0;
+
+ return hci_remote_name_cancel_sync(hdev, &e->data.bdaddr);
+ }
+
+ return 0;
+}
+
+static int hci_disconnect_phy_link_sync(struct hci_dev *hdev, u16 handle,
+ u8 reason)
+{
+ struct hci_cp_disconn_phy_link cp;
+
+ memset(&cp, 0, sizeof(cp));
+ cp.phy_handle = HCI_PHY_HANDLE(handle);
+ cp.reason = reason;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_DISCONN_PHY_LINK,
+ sizeof(cp), &cp, HCI_CMD_TIMEOUT);
+}
+
+static int hci_disconnect_sync(struct hci_dev *hdev, struct hci_conn *conn,
+ u8 reason)
+{
+ struct hci_cp_disconnect cp;
+
+ if (conn->type == AMP_LINK)
+ return hci_disconnect_phy_link_sync(hdev, conn->handle, reason);
+
+ memset(&cp, 0, sizeof(cp));
+ cp.handle = cpu_to_le16(conn->handle);
+ cp.reason = reason;
+
+ /* Wait for HCI_EV_DISCONN_COMPLETE not HCI_EV_CMD_STATUS when not
+ * suspending.
+ */
+ if (!hdev->suspended)
+ return __hci_cmd_sync_status_sk(hdev, HCI_OP_DISCONNECT,
+ sizeof(cp), &cp,
+ HCI_EV_DISCONN_COMPLETE,
+ HCI_CMD_TIMEOUT, NULL);
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_DISCONNECT, sizeof(cp), &cp,
+ HCI_CMD_TIMEOUT);
+}
+
+static int hci_le_connect_cancel_sync(struct hci_dev *hdev,
+ struct hci_conn *conn)
+{
+ if (test_bit(HCI_CONN_SCANNING, &conn->flags))
+ return 0;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_LE_CREATE_CONN_CANCEL,
+ 6, &conn->dst, HCI_CMD_TIMEOUT);
+}
+
+static int hci_connect_cancel_sync(struct hci_dev *hdev, struct hci_conn *conn)
+{
+ if (conn->type == LE_LINK)
+ return hci_le_connect_cancel_sync(hdev, conn);
+
+ if (hdev->hci_ver < BLUETOOTH_VER_1_2)
+ return 0;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_CREATE_CONN_CANCEL,
+ 6, &conn->dst, HCI_CMD_TIMEOUT);
+}
+
+static int hci_reject_sco_sync(struct hci_dev *hdev, struct hci_conn *conn,
+ u8 reason)
+{
+ struct hci_cp_reject_sync_conn_req cp;
+
+ memset(&cp, 0, sizeof(cp));
+ bacpy(&cp.bdaddr, &conn->dst);
+ cp.reason = reason;
+
+ /* SCO rejection has its own limited set of
+ * allowed error values (0x0D-0x0F).
+ */
+ if (reason < 0x0d || reason > 0x0f)
+ cp.reason = HCI_ERROR_REJ_LIMITED_RESOURCES;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_REJECT_SYNC_CONN_REQ,
+ sizeof(cp), &cp, HCI_CMD_TIMEOUT);
+}
+
+static int hci_reject_conn_sync(struct hci_dev *hdev, struct hci_conn *conn,
+ u8 reason)
+{
+ struct hci_cp_reject_conn_req cp;
+
+ if (conn->type == SCO_LINK || conn->type == ESCO_LINK)
+ return hci_reject_sco_sync(hdev, conn, reason);
+
+ memset(&cp, 0, sizeof(cp));
+ bacpy(&cp.bdaddr, &conn->dst);
+ cp.reason = reason;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_REJECT_CONN_REQ,
+ sizeof(cp), &cp, HCI_CMD_TIMEOUT);
+}
+
+static int hci_abort_conn_sync(struct hci_dev *hdev, struct hci_conn *conn,
+ u8 reason)
+{
+ switch (conn->state) {
+ case BT_CONNECTED:
+ case BT_CONFIG:
+ return hci_disconnect_sync(hdev, conn, reason);
+ case BT_CONNECT:
+ return hci_connect_cancel_sync(hdev, conn);
+ case BT_CONNECT2:
+ return hci_reject_conn_sync(hdev, conn, reason);
+ default:
+ conn->state = BT_CLOSED;
+ break;
+ }
+
+ return 0;
+}
+
+static int hci_disconnect_all_sync(struct hci_dev *hdev, u8 reason)
+{
+ struct hci_conn *conn, *tmp;
+ int err;
+
+ list_for_each_entry_safe(conn, tmp, &hdev->conn_hash.list, list) {
+ err = hci_abort_conn_sync(hdev, conn, reason);
+ if (err)
+ return err;
+ }
+
+ return err;
+}
+
+/* This function perform power off HCI command sequence as follows:
+ *
+ * Clear Advertising
+ * Stop Discovery
+ * Disconnect all connections
+ * hci_dev_close_sync
+ */
+static int hci_power_off_sync(struct hci_dev *hdev)
+{
+ int err;
+
+ /* If controller is already down there is nothing to do */
+ if (!test_bit(HCI_UP, &hdev->flags))
+ return 0;
+
+ if (test_bit(HCI_ISCAN, &hdev->flags) ||
+ test_bit(HCI_PSCAN, &hdev->flags)) {
+ err = hci_write_scan_enable_sync(hdev, 0x00);
+ if (err)
+ return err;
+ }
+
+ err = hci_clear_adv_sync(hdev, NULL, false);
+ if (err)
+ return err;
+
+ err = hci_stop_discovery_sync(hdev);
+ if (err)
+ return err;
+
+ /* Terminated due to Power Off */
+ err = hci_disconnect_all_sync(hdev, HCI_ERROR_REMOTE_POWER_OFF);
+ if (err)
+ return err;
+
+ return hci_dev_close_sync(hdev);
+}
+
+int hci_set_powered_sync(struct hci_dev *hdev, u8 val)
+{
+ if (val)
+ return hci_power_on_sync(hdev);
+
+ return hci_power_off_sync(hdev);
+}
+
+static int hci_write_iac_sync(struct hci_dev *hdev)
+{
+ struct hci_cp_write_current_iac_lap cp;
+
+ if (!hci_dev_test_flag(hdev, HCI_DISCOVERABLE))
+ return 0;
+
+ memset(&cp, 0, sizeof(cp));
+
+ if (hci_dev_test_flag(hdev, HCI_LIMITED_DISCOVERABLE)) {
+ /* Limited discoverable mode */
+ cp.num_iac = min_t(u8, hdev->num_iac, 2);
+ cp.iac_lap[0] = 0x00; /* LIAC */
+ cp.iac_lap[1] = 0x8b;
+ cp.iac_lap[2] = 0x9e;
+ cp.iac_lap[3] = 0x33; /* GIAC */
+ cp.iac_lap[4] = 0x8b;
+ cp.iac_lap[5] = 0x9e;
+ } else {
+ /* General discoverable mode */
+ cp.num_iac = 1;
+ cp.iac_lap[0] = 0x33; /* GIAC */
+ cp.iac_lap[1] = 0x8b;
+ cp.iac_lap[2] = 0x9e;
+ }
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_WRITE_CURRENT_IAC_LAP,
+ (cp.num_iac * 3) + 1, &cp,
+ HCI_CMD_TIMEOUT);
+}
+
+int hci_update_discoverable_sync(struct hci_dev *hdev)
+{
+ int err = 0;
+
+ if (hci_dev_test_flag(hdev, HCI_BREDR_ENABLED)) {
+ err = hci_write_iac_sync(hdev);
+ if (err)
+ return err;
+
+ err = hci_update_scan_sync(hdev);
+ if (err)
+ return err;
+
+ err = hci_update_class_sync(hdev);
+ if (err)
+ return err;
+ }
+
+ /* Advertising instances don't use the global discoverable setting, so
+ * only update AD if advertising was enabled using Set Advertising.
+ */
+ if (hci_dev_test_flag(hdev, HCI_ADVERTISING)) {
+ err = hci_update_adv_data_sync(hdev, 0x00);
+ if (err)
+ return err;
+
+ /* Discoverable mode affects the local advertising
+ * address in limited privacy mode.
+ */
+ if (hci_dev_test_flag(hdev, HCI_LIMITED_PRIVACY)) {
+ if (ext_adv_capable(hdev))
+ err = hci_start_ext_adv_sync(hdev, 0x00);
+ else
+ err = hci_enable_advertising_sync(hdev);
+ }
+ }
+
+ return err;
+}
+
+static int update_discoverable_sync(struct hci_dev *hdev, void *data)
+{
+ return hci_update_discoverable_sync(hdev);
+}
+
+int hci_update_discoverable(struct hci_dev *hdev)
+{
+ /* Only queue if it would have any effect */
+ if (hdev_is_powered(hdev) &&
+ hci_dev_test_flag(hdev, HCI_ADVERTISING) &&
+ hci_dev_test_flag(hdev, HCI_DISCOVERABLE) &&
+ hci_dev_test_flag(hdev, HCI_LIMITED_PRIVACY))
+ return hci_cmd_sync_queue(hdev, update_discoverable_sync, NULL,
+ NULL);
+
+ return 0;
+}
+
+int hci_update_connectable_sync(struct hci_dev *hdev)
+{
+ int err;
+
+ err = hci_update_scan_sync(hdev);
+ if (err)
+ return err;
+
+ /* If BR/EDR is not enabled and we disable advertising as a
+ * by-product of disabling connectable, we need to update the
+ * advertising flags.
+ */
+ if (!hci_dev_test_flag(hdev, HCI_BREDR_ENABLED))
+ err = hci_update_adv_data_sync(hdev, hdev->cur_adv_instance);
+
+ /* Update the advertising parameters if necessary */
+ if (hci_dev_test_flag(hdev, HCI_ADVERTISING) ||
+ !list_empty(&hdev->adv_instances)) {
+ if (ext_adv_capable(hdev))
+ err = hci_start_ext_adv_sync(hdev,
+ hdev->cur_adv_instance);
+ else
+ err = hci_enable_advertising_sync(hdev);
+
+ if (err)
+ return err;
+ }
+
+ return hci_update_passive_scan_sync(hdev);
+}
+
+static int hci_inquiry_sync(struct hci_dev *hdev, u8 length)
+{
+ const u8 giac[3] = { 0x33, 0x8b, 0x9e };
+ const u8 liac[3] = { 0x00, 0x8b, 0x9e };
+ struct hci_cp_inquiry cp;
+
+ bt_dev_dbg(hdev, "");
+
+ if (hci_dev_test_flag(hdev, HCI_INQUIRY))
+ return 0;
+
+ hci_dev_lock(hdev);
+ hci_inquiry_cache_flush(hdev);
+ hci_dev_unlock(hdev);
+
+ memset(&cp, 0, sizeof(cp));
+
+ if (hdev->discovery.limited)
+ memcpy(&cp.lap, liac, sizeof(cp.lap));
+ else
+ memcpy(&cp.lap, giac, sizeof(cp.lap));
+
+ cp.length = length;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_INQUIRY,
+ sizeof(cp), &cp, HCI_CMD_TIMEOUT);
+}
+
+static int hci_active_scan_sync(struct hci_dev *hdev, uint16_t interval)
+{
+ u8 own_addr_type;
+ /* Accept list is not used for discovery */
+ u8 filter_policy = 0x00;
+ /* Default is to enable duplicates filter */
+ u8 filter_dup = LE_SCAN_FILTER_DUP_ENABLE;
+ int err;
+
+ bt_dev_dbg(hdev, "");
+
+ /* If controller is scanning, it means the passive scanning is
+ * running. Thus, we should temporarily stop it in order to set the
+ * discovery scanning parameters.
+ */
+ err = hci_scan_disable_sync(hdev);
+ if (err) {
+ bt_dev_err(hdev, "Unable to disable scanning: %d", err);
+ return err;
+ }
+
+ cancel_interleave_scan(hdev);
+
+ /* Pause advertising since active scanning disables address resolution
+ * which advertising depend on in order to generate its RPAs.
+ */
+ if (use_ll_privacy(hdev)) {
+ err = hci_pause_advertising_sync(hdev);
+ if (err) {
+ bt_dev_err(hdev, "pause advertising failed: %d", err);
+ goto failed;
+ }
+ }
+
+ /* Disable address resolution while doing active scanning since the
+ * accept list shall not be used and all reports shall reach the host
+ * anyway.
+ */
+ err = hci_le_set_addr_resolution_enable_sync(hdev, 0x00);
+ if (err) {
+ bt_dev_err(hdev, "Unable to disable Address Resolution: %d",
+ err);
+ goto failed;
+ }
+
+ /* All active scans will be done with either a resolvable private
+ * address (when privacy feature has been enabled) or non-resolvable
+ * private address.
+ */
+ err = hci_update_random_address_sync(hdev, true, scan_use_rpa(hdev),
+ &own_addr_type);
+ if (err < 0)
+ own_addr_type = ADDR_LE_DEV_PUBLIC;
+
+ if (hci_is_adv_monitoring(hdev)) {
+ /* Duplicate filter should be disabled when some advertisement
+ * monitor is activated, otherwise AdvMon can only receive one
+ * advertisement for one peer(*) during active scanning, and
+ * might report loss to these peers.
+ *
+ * Note that different controllers have different meanings of
+ * |duplicate|. Some of them consider packets with the same
+ * address as duplicate, and others consider packets with the
+ * same address and the same RSSI as duplicate. Although in the
+ * latter case we don't need to disable duplicate filter, but
+ * it is common to have active scanning for a short period of
+ * time, the power impact should be neglectable.
+ */
+ filter_dup = LE_SCAN_FILTER_DUP_DISABLE;
+ }
+
+ err = hci_start_scan_sync(hdev, LE_SCAN_ACTIVE, interval,
+ hdev->le_scan_window_discovery,
+ own_addr_type, filter_policy, filter_dup);
+ if (!err)
+ return err;
+
+failed:
+ /* Resume advertising if it was paused */
+ if (use_ll_privacy(hdev))
+ hci_resume_advertising_sync(hdev);
+
+ /* Resume passive scanning */
+ hci_update_passive_scan_sync(hdev);
+ return err;
+}
+
+static int hci_start_interleaved_discovery_sync(struct hci_dev *hdev)
+{
+ int err;
+
+ bt_dev_dbg(hdev, "");
+
+ err = hci_active_scan_sync(hdev, hdev->le_scan_int_discovery * 2);
+ if (err)
+ return err;
+
+ return hci_inquiry_sync(hdev, DISCOV_BREDR_INQUIRY_LEN);
+}
+
+int hci_start_discovery_sync(struct hci_dev *hdev)
+{
+ unsigned long timeout;
+ int err;
+
+ bt_dev_dbg(hdev, "type %u", hdev->discovery.type);
+
+ switch (hdev->discovery.type) {
+ case DISCOV_TYPE_BREDR:
+ return hci_inquiry_sync(hdev, DISCOV_BREDR_INQUIRY_LEN);
+ case DISCOV_TYPE_INTERLEAVED:
+ /* When running simultaneous discovery, the LE scanning time
+ * should occupy the whole discovery time sine BR/EDR inquiry
+ * and LE scanning are scheduled by the controller.
+ *
+ * For interleaving discovery in comparison, BR/EDR inquiry
+ * and LE scanning are done sequentially with separate
+ * timeouts.
+ */
+ if (test_bit(HCI_QUIRK_SIMULTANEOUS_DISCOVERY,
+ &hdev->quirks)) {
+ timeout = msecs_to_jiffies(DISCOV_LE_TIMEOUT);
+ /* During simultaneous discovery, we double LE scan
+ * interval. We must leave some time for the controller
+ * to do BR/EDR inquiry.
+ */
+ err = hci_start_interleaved_discovery_sync(hdev);
+ break;
+ }
+
+ timeout = msecs_to_jiffies(hdev->discov_interleaved_timeout);
+ err = hci_active_scan_sync(hdev, hdev->le_scan_int_discovery);
+ break;
+ case DISCOV_TYPE_LE:
+ timeout = msecs_to_jiffies(DISCOV_LE_TIMEOUT);
+ err = hci_active_scan_sync(hdev, hdev->le_scan_int_discovery);
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ if (err)
+ return err;
+
+ bt_dev_dbg(hdev, "timeout %u ms", jiffies_to_msecs(timeout));
+
+ /* When service discovery is used and the controller has a
+ * strict duplicate filter, it is important to remember the
+ * start and duration of the scan. This is required for
+ * restarting scanning during the discovery phase.
+ */
+ if (test_bit(HCI_QUIRK_STRICT_DUPLICATE_FILTER, &hdev->quirks) &&
+ hdev->discovery.result_filtering) {
+ hdev->discovery.scan_start = jiffies;
+ hdev->discovery.scan_duration = timeout;
+ }
+
+ queue_delayed_work(hdev->req_workqueue, &hdev->le_scan_disable,
+ timeout);
+ return 0;
+}
+
+static void hci_suspend_monitor_sync(struct hci_dev *hdev)
+{
+ switch (hci_get_adv_monitor_offload_ext(hdev)) {
+ case HCI_ADV_MONITOR_EXT_MSFT:
+ msft_suspend_sync(hdev);
+ break;
+ default:
+ return;
+ }
+}
+
+/* This function disables discovery and mark it as paused */
+static int hci_pause_discovery_sync(struct hci_dev *hdev)
+{
+ int old_state = hdev->discovery.state;
+ int err;
+
+ /* If discovery already stopped/stopping/paused there nothing to do */
+ if (old_state == DISCOVERY_STOPPED || old_state == DISCOVERY_STOPPING ||
+ hdev->discovery_paused)
+ return 0;
+
+ hci_discovery_set_state(hdev, DISCOVERY_STOPPING);
+ err = hci_stop_discovery_sync(hdev);
+ if (err)
+ return err;
+
+ hdev->discovery_paused = true;
+ hdev->discovery_old_state = old_state;
+ hci_discovery_set_state(hdev, DISCOVERY_STOPPED);
+
+ return 0;
+}
+
+static int hci_update_event_filter_sync(struct hci_dev *hdev)
+{
+ struct bdaddr_list_with_flags *b;
+ u8 scan = SCAN_DISABLED;
+ bool scanning = test_bit(HCI_PSCAN, &hdev->flags);
+ int err;
+
+ if (!hci_dev_test_flag(hdev, HCI_BREDR_ENABLED))
+ return 0;
+
+ /* Always clear event filter when starting */
+ hci_clear_event_filter_sync(hdev);
+
+ list_for_each_entry(b, &hdev->accept_list, list) {
+ if (!test_bit(HCI_CONN_FLAG_REMOTE_WAKEUP, b->flags))
+ continue;
+
+ bt_dev_dbg(hdev, "Adding event filters for %pMR", &b->bdaddr);
+
+ err = hci_set_event_filter_sync(hdev, HCI_FLT_CONN_SETUP,
+ HCI_CONN_SETUP_ALLOW_BDADDR,
+ &b->bdaddr,
+ HCI_CONN_SETUP_AUTO_ON);
+ if (err)
+ bt_dev_dbg(hdev, "Failed to set event filter for %pMR",
+ &b->bdaddr);
+ else
+ scan = SCAN_PAGE;
+ }
+
+ if (scan && !scanning)
+ hci_write_scan_enable_sync(hdev, scan);
+ else if (!scan && scanning)
+ hci_write_scan_enable_sync(hdev, scan);
+
+ return 0;
+}
+
+/* This function performs the HCI suspend procedures in the follow order:
+ *
+ * Pause discovery (active scanning/inquiry)
+ * Pause Directed Advertising/Advertising
+ * Disconnect all connections
+ * Set suspend_status to BT_SUSPEND_DISCONNECT if hdev cannot wakeup
+ * otherwise:
+ * Update event mask (only set events that are allowed to wake up the host)
+ * Update event filter (with devices marked with HCI_CONN_FLAG_REMOTE_WAKEUP)
+ * Update passive scanning (lower duty cycle)
+ * Set suspend_status to BT_SUSPEND_CONFIGURE_WAKE
+ */
+int hci_suspend_sync(struct hci_dev *hdev)
+{
+ int err;
+
+ /* If marked as suspended there nothing to do */
+ if (hdev->suspended)
+ return 0;
+
+ /* Mark device as suspended */
+ hdev->suspended = true;
+
+ /* Pause discovery if not already stopped */
+ hci_pause_discovery_sync(hdev);
+
+ /* Pause other advertisements */
+ hci_pause_advertising_sync(hdev);
+
+ /* Disable page scan if enabled */
+ if (test_bit(HCI_PSCAN, &hdev->flags))
+ hci_write_scan_enable_sync(hdev, SCAN_DISABLED);
+
+ /* Suspend monitor filters */
+ hci_suspend_monitor_sync(hdev);
+
+ /* Prevent disconnects from causing scanning to be re-enabled */
+ hdev->scanning_paused = true;
+
+ /* Soft disconnect everything (power off) */
+ err = hci_disconnect_all_sync(hdev, HCI_ERROR_REMOTE_POWER_OFF);
+ if (err) {
+ /* Set state to BT_RUNNING so resume doesn't notify */
+ hdev->suspend_state = BT_RUNNING;
+ hci_resume_sync(hdev);
+ return err;
+ }
+
+ /* Only configure accept list if disconnect succeeded and wake
+ * isn't being prevented.
+ */
+ if (!hdev->wakeup || !hdev->wakeup(hdev)) {
+ hdev->suspend_state = BT_SUSPEND_DISCONNECT;
+ return 0;
+ }
+
+ /* Unpause to take care of updating scanning params */
+ hdev->scanning_paused = false;
+
+ /* Update event mask so only the allowed event can wakeup the host */
+ hci_set_event_mask_sync(hdev);
+
+ /* Enable event filter for paired devices */
+ hci_update_event_filter_sync(hdev);
+
+ /* Update LE passive scan if enabled */
+ hci_update_passive_scan_sync(hdev);
+
+ /* Pause scan changes again. */
+ hdev->scanning_paused = true;
+
+ hdev->suspend_state = BT_SUSPEND_CONFIGURE_WAKE;
+
+ return 0;
+}
+
+/* This function resumes discovery */
+static int hci_resume_discovery_sync(struct hci_dev *hdev)
+{
+ int err;
+
+ /* If discovery not paused there nothing to do */
+ if (!hdev->discovery_paused)
+ return 0;
+
+ hdev->discovery_paused = false;
+
+ hci_discovery_set_state(hdev, DISCOVERY_STARTING);
+
+ err = hci_start_discovery_sync(hdev);
+
+ hci_discovery_set_state(hdev, err ? DISCOVERY_STOPPED :
+ DISCOVERY_FINDING);
+
+ return err;
+}
+
+static void hci_resume_monitor_sync(struct hci_dev *hdev)
+{
+ switch (hci_get_adv_monitor_offload_ext(hdev)) {
+ case HCI_ADV_MONITOR_EXT_MSFT:
+ msft_resume_sync(hdev);
+ break;
+ default:
+ return;
+ }
+}
+
+/* This function performs the HCI suspend procedures in the follow order:
+ *
+ * Restore event mask
+ * Clear event filter
+ * Update passive scanning (normal duty cycle)
+ * Resume Directed Advertising/Advertising
+ * Resume discovery (active scanning/inquiry)
+ */
+int hci_resume_sync(struct hci_dev *hdev)
+{
+ /* If not marked as suspended there nothing to do */
+ if (!hdev->suspended)
+ return 0;
+
+ hdev->suspended = false;
+ hdev->scanning_paused = false;
+
+ /* Restore event mask */
+ hci_set_event_mask_sync(hdev);
+
+ /* Clear any event filters and restore scan state */
+ hci_clear_event_filter_sync(hdev);
+ hci_update_scan_sync(hdev);
+
+ /* Reset passive scanning to normal */
+ hci_update_passive_scan_sync(hdev);
+
+ /* Resume monitor filters */
+ hci_resume_monitor_sync(hdev);
+
+ /* Resume other advertisements */
+ hci_resume_advertising_sync(hdev);
+
+ /* Resume discovery */
+ hci_resume_discovery_sync(hdev);
+
+ return 0;
+}
+
+static bool conn_use_rpa(struct hci_conn *conn)
+{
+ struct hci_dev *hdev = conn->hdev;
+
+ return hci_dev_test_flag(hdev, HCI_PRIVACY);
+}
+
+static int hci_le_ext_directed_advertising_sync(struct hci_dev *hdev,
+ struct hci_conn *conn)
+{
+ struct hci_cp_le_set_ext_adv_params cp;
+ int err;
+ bdaddr_t random_addr;
+ u8 own_addr_type;
+
+ err = hci_update_random_address_sync(hdev, false, conn_use_rpa(conn),
+ &own_addr_type);
+ if (err)
+ return err;
+
+ /* Set require_privacy to false so that the remote device has a
+ * chance of identifying us.
+ */
+ err = hci_get_random_address(hdev, false, conn_use_rpa(conn), NULL,
+ &own_addr_type, &random_addr);
+ if (err)
+ return err;
+
+ memset(&cp, 0, sizeof(cp));
+
+ cp.evt_properties = cpu_to_le16(LE_LEGACY_ADV_DIRECT_IND);
+ cp.own_addr_type = own_addr_type;
+ cp.channel_map = hdev->le_adv_channel_map;
+ cp.tx_power = HCI_TX_POWER_INVALID;
+ cp.primary_phy = HCI_ADV_PHY_1M;
+ cp.secondary_phy = HCI_ADV_PHY_1M;
+ cp.handle = 0x00; /* Use instance 0 for directed adv */
+ cp.own_addr_type = own_addr_type;
+ cp.peer_addr_type = conn->dst_type;
+ bacpy(&cp.peer_addr, &conn->dst);
+
+ /* As per Core Spec 5.2 Vol 2, PART E, Sec 7.8.53, for
+ * advertising_event_property LE_LEGACY_ADV_DIRECT_IND
+ * does not supports advertising data when the advertising set already
+ * contains some, the controller shall return erroc code 'Invalid
+ * HCI Command Parameters(0x12).
+ * So it is required to remove adv set for handle 0x00. since we use
+ * instance 0 for directed adv.
+ */
+ err = hci_remove_ext_adv_instance_sync(hdev, cp.handle, NULL);
+ if (err)
+ return err;
+
+ err = __hci_cmd_sync_status(hdev, HCI_OP_LE_SET_EXT_ADV_PARAMS,
+ sizeof(cp), &cp, HCI_CMD_TIMEOUT);
+ if (err)
+ return err;
+
+ /* Check if random address need to be updated */
+ if (own_addr_type == ADDR_LE_DEV_RANDOM &&
+ bacmp(&random_addr, BDADDR_ANY) &&
+ bacmp(&random_addr, &hdev->random_addr)) {
+ err = hci_set_adv_set_random_addr_sync(hdev, 0x00,
+ &random_addr);
+ if (err)
+ return err;
+ }
+
+ return hci_enable_ext_advertising_sync(hdev, 0x00);
+}
+
+static int hci_le_directed_advertising_sync(struct hci_dev *hdev,
+ struct hci_conn *conn)
+{
+ struct hci_cp_le_set_adv_param cp;
+ u8 status;
+ u8 own_addr_type;
+ u8 enable;
+
+ if (ext_adv_capable(hdev))
+ return hci_le_ext_directed_advertising_sync(hdev, conn);
+
+ /* Clear the HCI_LE_ADV bit temporarily so that the
+ * hci_update_random_address knows that it's safe to go ahead
+ * and write a new random address. The flag will be set back on
+ * as soon as the SET_ADV_ENABLE HCI command completes.
+ */
+ hci_dev_clear_flag(hdev, HCI_LE_ADV);
+
+ /* Set require_privacy to false so that the remote device has a
+ * chance of identifying us.
+ */
+ status = hci_update_random_address_sync(hdev, false, conn_use_rpa(conn),
+ &own_addr_type);
+ if (status)
+ return status;
+
+ memset(&cp, 0, sizeof(cp));
+
+ /* Some controllers might reject command if intervals are not
+ * within range for undirected advertising.
+ * BCM20702A0 is known to be affected by this.
+ */
+ cp.min_interval = cpu_to_le16(0x0020);
+ cp.max_interval = cpu_to_le16(0x0020);
+
+ cp.type = LE_ADV_DIRECT_IND;
+ cp.own_address_type = own_addr_type;
+ cp.direct_addr_type = conn->dst_type;
+ bacpy(&cp.direct_addr, &conn->dst);
+ cp.channel_map = hdev->le_adv_channel_map;
+
+ status = __hci_cmd_sync_status(hdev, HCI_OP_LE_SET_ADV_PARAM,
+ sizeof(cp), &cp, HCI_CMD_TIMEOUT);
+ if (status)
+ return status;
+
+ enable = 0x01;
+
+ return __hci_cmd_sync_status(hdev, HCI_OP_LE_SET_ADV_ENABLE,
+ sizeof(enable), &enable, HCI_CMD_TIMEOUT);
+}
+
+static void set_ext_conn_params(struct hci_conn *conn,
+ struct hci_cp_le_ext_conn_param *p)
+{
+ struct hci_dev *hdev = conn->hdev;
+
+ memset(p, 0, sizeof(*p));
+
+ p->scan_interval = cpu_to_le16(hdev->le_scan_int_connect);
+ p->scan_window = cpu_to_le16(hdev->le_scan_window_connect);
+ p->conn_interval_min = cpu_to_le16(conn->le_conn_min_interval);
+ p->conn_interval_max = cpu_to_le16(conn->le_conn_max_interval);
+ p->conn_latency = cpu_to_le16(conn->le_conn_latency);
+ p->supervision_timeout = cpu_to_le16(conn->le_supv_timeout);
+ p->min_ce_len = cpu_to_le16(0x0000);
+ p->max_ce_len = cpu_to_le16(0x0000);
+}
+
+int hci_le_ext_create_conn_sync(struct hci_dev *hdev, struct hci_conn *conn,
+ u8 own_addr_type)
+{
+ struct hci_cp_le_ext_create_conn *cp;
+ struct hci_cp_le_ext_conn_param *p;
+ u8 data[sizeof(*cp) + sizeof(*p) * 3];
+ u32 plen;
+
+ cp = (void *)data;
+ p = (void *)cp->data;
+
+ memset(cp, 0, sizeof(*cp));
+
+ bacpy(&cp->peer_addr, &conn->dst);
+ cp->peer_addr_type = conn->dst_type;
+ cp->own_addr_type = own_addr_type;
+
+ plen = sizeof(*cp);
+
+ if (scan_1m(hdev)) {
+ cp->phys |= LE_SCAN_PHY_1M;
+ set_ext_conn_params(conn, p);
+
+ p++;
+ plen += sizeof(*p);
+ }
+
+ if (scan_2m(hdev)) {
+ cp->phys |= LE_SCAN_PHY_2M;
+ set_ext_conn_params(conn, p);
+
+ p++;
+ plen += sizeof(*p);
+ }
+
+ if (scan_coded(hdev)) {
+ cp->phys |= LE_SCAN_PHY_CODED;
+ set_ext_conn_params(conn, p);
+
+ plen += sizeof(*p);
+ }
+
+ return __hci_cmd_sync_status_sk(hdev, HCI_OP_LE_EXT_CREATE_CONN,
+ plen, data,
+ HCI_EV_LE_ENHANCED_CONN_COMPLETE,
+ HCI_CMD_TIMEOUT, NULL);
+}
+
+int hci_le_create_conn_sync(struct hci_dev *hdev, struct hci_conn *conn)
+{
+ struct hci_cp_le_create_conn cp;
+ struct hci_conn_params *params;
+ u8 own_addr_type;
+ int err;
+
+ /* If requested to connect as peripheral use directed advertising */
+ if (conn->role == HCI_ROLE_SLAVE) {
+ /* If we're active scanning and simultaneous roles is not
+ * enabled simply reject the attempt.
+ */
+ if (hci_dev_test_flag(hdev, HCI_LE_SCAN) &&
+ hdev->le_scan_type == LE_SCAN_ACTIVE &&
+ !hci_dev_test_flag(hdev, HCI_LE_SIMULTANEOUS_ROLES)) {
+ hci_conn_del(conn);
+ return -EBUSY;
+ }
+
+ /* Pause advertising while doing directed advertising. */
+ hci_pause_advertising_sync(hdev);
+
+ err = hci_le_directed_advertising_sync(hdev, conn);
+ goto done;
+ }
+
+ /* Disable advertising if simultaneous roles is not in use. */
+ if (!hci_dev_test_flag(hdev, HCI_LE_SIMULTANEOUS_ROLES))
+ hci_pause_advertising_sync(hdev);
+
+ params = hci_conn_params_lookup(hdev, &conn->dst, conn->dst_type);
+ if (params) {
+ conn->le_conn_min_interval = params->conn_min_interval;
+ conn->le_conn_max_interval = params->conn_max_interval;
+ conn->le_conn_latency = params->conn_latency;
+ conn->le_supv_timeout = params->supervision_timeout;
+ } else {
+ conn->le_conn_min_interval = hdev->le_conn_min_interval;
+ conn->le_conn_max_interval = hdev->le_conn_max_interval;
+ conn->le_conn_latency = hdev->le_conn_latency;
+ conn->le_supv_timeout = hdev->le_supv_timeout;
+ }
+
+ /* If controller is scanning, we stop it since some controllers are
+ * not able to scan and connect at the same time. Also set the
+ * HCI_LE_SCAN_INTERRUPTED flag so that the command complete
+ * handler for scan disabling knows to set the correct discovery
+ * state.
+ */
+ if (hci_dev_test_flag(hdev, HCI_LE_SCAN)) {
+ hci_scan_disable_sync(hdev);
+ hci_dev_set_flag(hdev, HCI_LE_SCAN_INTERRUPTED);
+ }
+
+ /* Update random address, but set require_privacy to false so
+ * that we never connect with an non-resolvable address.
+ */
+ err = hci_update_random_address_sync(hdev, false, conn_use_rpa(conn),
+ &own_addr_type);
+ if (err)
+ goto done;
+
+ if (use_ext_conn(hdev)) {
+ err = hci_le_ext_create_conn_sync(hdev, conn, own_addr_type);
+ goto done;
+ }
+
+ memset(&cp, 0, sizeof(cp));
+
+ cp.scan_interval = cpu_to_le16(hdev->le_scan_int_connect);
+ cp.scan_window = cpu_to_le16(hdev->le_scan_window_connect);
+
+ bacpy(&cp.peer_addr, &conn->dst);
+ cp.peer_addr_type = conn->dst_type;
+ cp.own_address_type = own_addr_type;
+ cp.conn_interval_min = cpu_to_le16(conn->le_conn_min_interval);
+ cp.conn_interval_max = cpu_to_le16(conn->le_conn_max_interval);
+ cp.conn_latency = cpu_to_le16(conn->le_conn_latency);
+ cp.supervision_timeout = cpu_to_le16(conn->le_supv_timeout);
+ cp.min_ce_len = cpu_to_le16(0x0000);
+ cp.max_ce_len = cpu_to_le16(0x0000);
+
+ err = __hci_cmd_sync_status_sk(hdev, HCI_OP_LE_CREATE_CONN,
+ sizeof(cp), &cp, HCI_EV_LE_CONN_COMPLETE,
+ HCI_CMD_TIMEOUT, NULL);
+
+done:
+ /* Re-enable advertising after the connection attempt is finished. */
+ hci_resume_advertising_sync(hdev);
+ return err;
+}
diff --git a/net/bluetooth/hci_sysfs.c b/net/bluetooth/hci_sysfs.c
index 7827639..4e3e045 100644
--- a/net/bluetooth/hci_sysfs.c
+++ b/net/bluetooth/hci_sysfs.c
@@ -86,6 +86,8 @@ static void bt_host_release(struct device *dev)
if (hci_dev_test_flag(hdev, HCI_UNREGISTER))
hci_release_dev(hdev);
+ else
+ kfree(hdev);
module_put(THIS_MODULE);
}
diff --git a/net/bluetooth/hidp/sock.c b/net/bluetooth/hidp/sock.c
index 595fb3c..369ed92 100644
--- a/net/bluetooth/hidp/sock.c
+++ b/net/bluetooth/hidp/sock.c
@@ -20,6 +20,7 @@
SOFTWARE IS DISCLAIMED.
*/
+#include <linux/compat.h>
#include <linux/export.h>
#include <linux/file.h>
diff --git a/net/bluetooth/l2cap_core.c b/net/bluetooth/l2cap_core.c
index 4f8f375..e817ff0 100644
--- a/net/bluetooth/l2cap_core.c
+++ b/net/bluetooth/l2cap_core.c
@@ -7905,7 +7905,7 @@ int l2cap_chan_connect(struct l2cap_chan *chan, __le16 psm, u16 cid,
hcon = hci_connect_le(hdev, dst, dst_type, false,
chan->sec_level,
HCI_LE_CONN_TIMEOUT,
- HCI_ROLE_SLAVE, NULL);
+ HCI_ROLE_SLAVE);
else
hcon = hci_connect_le_scan(hdev, dst, dst_type,
chan->sec_level,
diff --git a/net/bluetooth/l2cap_sock.c b/net/bluetooth/l2cap_sock.c
index 160c016..ca8f07f 100644
--- a/net/bluetooth/l2cap_sock.c
+++ b/net/bluetooth/l2cap_sock.c
@@ -29,6 +29,7 @@
#include <linux/module.h>
#include <linux/export.h>
+#include <linux/filter.h>
#include <linux/sched/signal.h>
#include <net/bluetooth/bluetooth.h>
@@ -161,7 +162,11 @@ static int l2cap_sock_bind(struct socket *sock, struct sockaddr *addr, int alen)
break;
}
- if (chan->psm && bdaddr_type_is_le(chan->src_type))
+ /* Use L2CAP_MODE_LE_FLOWCTL (CoC) in case of LE address and
+ * L2CAP_MODE_EXT_FLOWCTL (ECRED) has not been set.
+ */
+ if (chan->psm && bdaddr_type_is_le(chan->src_type) &&
+ chan->mode != L2CAP_MODE_EXT_FLOWCTL)
chan->mode = L2CAP_MODE_LE_FLOWCTL;
chan->state = BT_BOUND;
@@ -172,6 +177,21 @@ static int l2cap_sock_bind(struct socket *sock, struct sockaddr *addr, int alen)
return err;
}
+static void l2cap_sock_init_pid(struct sock *sk)
+{
+ struct l2cap_chan *chan = l2cap_pi(sk)->chan;
+
+ /* Only L2CAP_MODE_EXT_FLOWCTL ever need to access the PID in order to
+ * group the channels being requested.
+ */
+ if (chan->mode != L2CAP_MODE_EXT_FLOWCTL)
+ return;
+
+ spin_lock(&sk->sk_peer_lock);
+ sk->sk_peer_pid = get_pid(task_tgid(current));
+ spin_unlock(&sk->sk_peer_lock);
+}
+
static int l2cap_sock_connect(struct socket *sock, struct sockaddr *addr,
int alen, int flags)
{
@@ -240,9 +260,15 @@ static int l2cap_sock_connect(struct socket *sock, struct sockaddr *addr,
return -EINVAL;
}
- if (chan->psm && bdaddr_type_is_le(chan->src_type) && !chan->mode)
+ /* Use L2CAP_MODE_LE_FLOWCTL (CoC) in case of LE address and
+ * L2CAP_MODE_EXT_FLOWCTL (ECRED) has not been set.
+ */
+ if (chan->psm && bdaddr_type_is_le(chan->src_type) &&
+ chan->mode != L2CAP_MODE_EXT_FLOWCTL)
chan->mode = L2CAP_MODE_LE_FLOWCTL;
+ l2cap_sock_init_pid(sk);
+
err = l2cap_chan_connect(chan, la.l2_psm, __le16_to_cpu(la.l2_cid),
&la.l2_bdaddr, la.l2_bdaddr_type);
if (err)
@@ -298,6 +324,8 @@ static int l2cap_sock_listen(struct socket *sock, int backlog)
goto done;
}
+ l2cap_sock_init_pid(sk);
+
sk->sk_max_ack_backlog = backlog;
sk->sk_ack_backlog = 0;
@@ -876,6 +904,8 @@ static int l2cap_sock_setsockopt(struct socket *sock, int level, int optname,
struct l2cap_conn *conn;
int len, err = 0;
u32 opt;
+ u16 mtu;
+ u8 mode;
BT_DBG("sk %p", sk);
@@ -1058,16 +1088,16 @@ static int l2cap_sock_setsockopt(struct socket *sock, int level, int optname,
break;
}
- if (copy_from_sockptr(&opt, optval, sizeof(u16))) {
+ if (copy_from_sockptr(&mtu, optval, sizeof(u16))) {
err = -EFAULT;
break;
}
if (chan->mode == L2CAP_MODE_EXT_FLOWCTL &&
sk->sk_state == BT_CONNECTED)
- err = l2cap_chan_reconfigure(chan, opt);
+ err = l2cap_chan_reconfigure(chan, mtu);
else
- chan->imtu = opt;
+ chan->imtu = mtu;
break;
@@ -1089,14 +1119,14 @@ static int l2cap_sock_setsockopt(struct socket *sock, int level, int optname,
break;
}
- if (copy_from_sockptr(&opt, optval, sizeof(u8))) {
+ if (copy_from_sockptr(&mode, optval, sizeof(u8))) {
err = -EFAULT;
break;
}
- BT_DBG("opt %u", opt);
+ BT_DBG("mode %u", mode);
- err = l2cap_set_mode(chan, opt);
+ err = l2cap_set_mode(chan, mode);
if (err)
break;
diff --git a/net/bluetooth/mgmt.c b/net/bluetooth/mgmt.c
index 3e528360..37087cf 100644
--- a/net/bluetooth/mgmt.c
+++ b/net/bluetooth/mgmt.c
@@ -39,6 +39,7 @@
#include "mgmt_config.h"
#include "msft.h"
#include "eir.h"
+#include "aosp.h"
#define MGMT_VERSION 1
#define MGMT_REVISION 21
@@ -276,10 +277,39 @@ static const u8 mgmt_status_table[] = {
MGMT_STATUS_CONNECT_FAILED, /* MAC Connection Failed */
};
-static u8 mgmt_status(u8 hci_status)
+static u8 mgmt_errno_status(int err)
{
- if (hci_status < ARRAY_SIZE(mgmt_status_table))
- return mgmt_status_table[hci_status];
+ switch (err) {
+ case 0:
+ return MGMT_STATUS_SUCCESS;
+ case -EPERM:
+ return MGMT_STATUS_REJECTED;
+ case -EINVAL:
+ return MGMT_STATUS_INVALID_PARAMS;
+ case -EOPNOTSUPP:
+ return MGMT_STATUS_NOT_SUPPORTED;
+ case -EBUSY:
+ return MGMT_STATUS_BUSY;
+ case -ETIMEDOUT:
+ return MGMT_STATUS_AUTH_FAILED;
+ case -ENOMEM:
+ return MGMT_STATUS_NO_RESOURCES;
+ case -EISCONN:
+ return MGMT_STATUS_ALREADY_CONNECTED;
+ case -ENOTCONN:
+ return MGMT_STATUS_DISCONNECTED;
+ }
+
+ return MGMT_STATUS_FAILED;
+}
+
+static u8 mgmt_status(int err)
+{
+ if (err < 0)
+ return mgmt_errno_status(err);
+
+ if (err < ARRAY_SIZE(mgmt_status_table))
+ return mgmt_status_table[err];
return MGMT_STATUS_FAILED;
}
@@ -305,6 +335,12 @@ static int mgmt_event(u16 event, struct hci_dev *hdev, void *data, u16 len,
HCI_SOCK_TRUSTED, skip_sk);
}
+static int mgmt_event_skb(struct sk_buff *skb, struct sock *skip_sk)
+{
+ return mgmt_send_event_skb(HCI_CHANNEL_CONTROL, skb, HCI_SOCK_TRUSTED,
+ skip_sk);
+}
+
static u8 le_addr_type(u8 mgmt_addr_type)
{
if (mgmt_addr_type == BDADDR_LE_PUBLIC)
@@ -810,12 +846,7 @@ static u32 get_supported_settings(struct hci_dev *hdev)
settings |= MGMT_SETTING_SECURE_CONN;
settings |= MGMT_SETTING_PRIVACY;
settings |= MGMT_SETTING_STATIC_ADDRESS;
-
- /* When the experimental feature for LL Privacy support is
- * enabled, then advertising is no longer supported.
- */
- if (!hci_dev_test_flag(hdev, HCI_ENABLE_LL_PRIVACY))
- settings |= MGMT_SETTING_ADVERTISING;
+ settings |= MGMT_SETTING_ADVERTISING;
}
if (test_bit(HCI_QUIRK_EXTERNAL_CONFIG, &hdev->quirks) ||
@@ -903,13 +934,6 @@ static struct mgmt_pending_cmd *pending_find(u16 opcode, struct hci_dev *hdev)
return mgmt_pending_find(HCI_CHANNEL_CONTROL, opcode, hdev);
}
-static struct mgmt_pending_cmd *pending_find_data(u16 opcode,
- struct hci_dev *hdev,
- const void *data)
-{
- return mgmt_pending_find_data(HCI_CHANNEL_CONTROL, opcode, hdev, data);
-}
-
u8 mgmt_get_adv_discov_flags(struct hci_dev *hdev)
{
struct mgmt_pending_cmd *cmd;
@@ -951,32 +975,41 @@ bool mgmt_get_connectable(struct hci_dev *hdev)
return hci_dev_test_flag(hdev, HCI_CONNECTABLE);
}
+static int service_cache_sync(struct hci_dev *hdev, void *data)
+{
+ hci_update_eir_sync(hdev);
+ hci_update_class_sync(hdev);
+
+ return 0;
+}
+
static void service_cache_off(struct work_struct *work)
{
struct hci_dev *hdev = container_of(work, struct hci_dev,
service_cache.work);
- struct hci_request req;
if (!hci_dev_test_and_clear_flag(hdev, HCI_SERVICE_CACHE))
return;
- hci_req_init(&req, hdev);
+ hci_cmd_sync_queue(hdev, service_cache_sync, NULL, NULL);
+}
- hci_dev_lock(hdev);
-
- __hci_req_update_eir(&req);
- __hci_req_update_class(&req);
-
- hci_dev_unlock(hdev);
-
- hci_req_run(&req, NULL);
+static int rpa_expired_sync(struct hci_dev *hdev, void *data)
+{
+ /* The generation of a new RPA and programming it into the
+ * controller happens in the hci_req_enable_advertising()
+ * function.
+ */
+ if (ext_adv_capable(hdev))
+ return hci_start_ext_adv_sync(hdev, hdev->cur_adv_instance);
+ else
+ return hci_enable_advertising_sync(hdev);
}
static void rpa_expired(struct work_struct *work)
{
struct hci_dev *hdev = container_of(work, struct hci_dev,
rpa_expired.work);
- struct hci_request req;
bt_dev_dbg(hdev, "");
@@ -985,16 +1018,7 @@ static void rpa_expired(struct work_struct *work)
if (!hci_dev_test_flag(hdev, HCI_ADVERTISING))
return;
- /* The generation of a new RPA and programming it into the
- * controller happens in the hci_req_enable_advertising()
- * function.
- */
- hci_req_init(&req, hdev);
- if (ext_adv_capable(hdev))
- __hci_req_start_ext_adv(&req, hdev->cur_adv_instance);
- else
- __hci_req_enable_advertising(&req);
- hci_req_run(&req, NULL);
+ hci_cmd_sync_queue(hdev, rpa_expired_sync, NULL, NULL);
}
static void mgmt_init_hdev(struct sock *sk, struct hci_dev *hdev)
@@ -1131,16 +1155,6 @@ static int send_settings_rsp(struct sock *sk, u16 opcode, struct hci_dev *hdev)
sizeof(settings));
}
-static void clean_up_hci_complete(struct hci_dev *hdev, u8 status, u16 opcode)
-{
- bt_dev_dbg(hdev, "status 0x%02x", status);
-
- if (hci_conn_count(hdev) == 0) {
- cancel_delayed_work(&hdev->power_off);
- queue_work(hdev->req_workqueue, &hdev->power_off.work);
- }
-}
-
void mgmt_advertising_added(struct sock *sk, struct hci_dev *hdev, u8 instance)
{
struct mgmt_ev_advertising_added ev;
@@ -1168,38 +1182,77 @@ static void cancel_adv_timeout(struct hci_dev *hdev)
}
}
-static int clean_up_hci_state(struct hci_dev *hdev)
+/* This function requires the caller holds hdev->lock */
+static void restart_le_actions(struct hci_dev *hdev)
{
- struct hci_request req;
- struct hci_conn *conn;
- bool discov_stopped;
- int err;
+ struct hci_conn_params *p;
- hci_req_init(&req, hdev);
+ list_for_each_entry(p, &hdev->le_conn_params, list) {
+ /* Needed for AUTO_OFF case where might not "really"
+ * have been powered off.
+ */
+ list_del_init(&p->action);
- if (test_bit(HCI_ISCAN, &hdev->flags) ||
- test_bit(HCI_PSCAN, &hdev->flags)) {
- u8 scan = 0x00;
- hci_req_add(&req, HCI_OP_WRITE_SCAN_ENABLE, 1, &scan);
+ switch (p->auto_connect) {
+ case HCI_AUTO_CONN_DIRECT:
+ case HCI_AUTO_CONN_ALWAYS:
+ list_add(&p->action, &hdev->pend_le_conns);
+ break;
+ case HCI_AUTO_CONN_REPORT:
+ list_add(&p->action, &hdev->pend_le_reports);
+ break;
+ default:
+ break;
+ }
+ }
+}
+
+static int new_settings(struct hci_dev *hdev, struct sock *skip)
+{
+ __le32 ev = cpu_to_le32(get_current_settings(hdev));
+
+ return mgmt_limited_event(MGMT_EV_NEW_SETTINGS, hdev, &ev,
+ sizeof(ev), HCI_MGMT_SETTING_EVENTS, skip);
+}
+
+static void mgmt_set_powered_complete(struct hci_dev *hdev, void *data, int err)
+{
+ struct mgmt_pending_cmd *cmd = data;
+ struct mgmt_mode *cp = cmd->param;
+
+ bt_dev_dbg(hdev, "err %d", err);
+
+ if (!err) {
+ if (cp->val) {
+ hci_dev_lock(hdev);
+ restart_le_actions(hdev);
+ hci_update_passive_scan(hdev);
+ hci_dev_unlock(hdev);
+ }
+
+ send_settings_rsp(cmd->sk, cmd->opcode, hdev);
+
+ /* Only call new_setting for power on as power off is deferred
+ * to hdev->power_off work which does call hci_dev_do_close.
+ */
+ if (cp->val)
+ new_settings(hdev, cmd->sk);
+ } else {
+ mgmt_cmd_status(cmd->sk, hdev->id, MGMT_OP_SET_POWERED,
+ mgmt_status(err));
}
- hci_req_clear_adv_instance(hdev, NULL, NULL, 0x00, false);
+ mgmt_pending_free(cmd);
+}
- if (hci_dev_test_flag(hdev, HCI_LE_ADV))
- __hci_req_disable_advertising(&req);
+static int set_powered_sync(struct hci_dev *hdev, void *data)
+{
+ struct mgmt_pending_cmd *cmd = data;
+ struct mgmt_mode *cp = cmd->param;
- discov_stopped = hci_req_stop_discovery(&req);
+ BT_DBG("%s", hdev->name);
- list_for_each_entry(conn, &hdev->conn_hash.list, list) {
- /* 0x15 == Terminated due to Power Off */
- __hci_abort_conn(&req, conn, 0x15);
- }
-
- err = hci_req_run(&req, clean_up_hci_complete);
- if (!err && discov_stopped)
- hci_discovery_set_state(hdev, DISCOVERY_STOPPING);
-
- return err;
+ return hci_set_powered_sync(hdev, cp->val);
}
static int set_powered(struct sock *sk, struct hci_dev *hdev, void *data,
@@ -1228,43 +1281,20 @@ static int set_powered(struct sock *sk, struct hci_dev *hdev, void *data,
goto failed;
}
- cmd = mgmt_pending_add(sk, MGMT_OP_SET_POWERED, hdev, data, len);
+ cmd = mgmt_pending_new(sk, MGMT_OP_SET_POWERED, hdev, data, len);
if (!cmd) {
err = -ENOMEM;
goto failed;
}
- if (cp->val) {
- queue_work(hdev->req_workqueue, &hdev->power_on);
- err = 0;
- } else {
- /* Disconnect connections, stop scans, etc */
- err = clean_up_hci_state(hdev);
- if (!err)
- queue_delayed_work(hdev->req_workqueue, &hdev->power_off,
- HCI_POWER_OFF_TIMEOUT);
-
- /* ENODATA means there were no HCI commands queued */
- if (err == -ENODATA) {
- cancel_delayed_work(&hdev->power_off);
- queue_work(hdev->req_workqueue, &hdev->power_off.work);
- err = 0;
- }
- }
+ err = hci_cmd_sync_queue(hdev, set_powered_sync, cmd,
+ mgmt_set_powered_complete);
failed:
hci_dev_unlock(hdev);
return err;
}
-static int new_settings(struct hci_dev *hdev, struct sock *skip)
-{
- __le32 ev = cpu_to_le32(get_current_settings(hdev));
-
- return mgmt_limited_event(MGMT_EV_NEW_SETTINGS, hdev, &ev,
- sizeof(ev), HCI_MGMT_SETTING_EVENTS, skip);
-}
-
int mgmt_new_settings(struct hci_dev *hdev)
{
return new_settings(hdev, NULL);
@@ -1346,23 +1376,20 @@ static u8 mgmt_le_support(struct hci_dev *hdev)
return MGMT_STATUS_SUCCESS;
}
-void mgmt_set_discoverable_complete(struct hci_dev *hdev, u8 status)
+static void mgmt_set_discoverable_complete(struct hci_dev *hdev, void *data,
+ int err)
{
- struct mgmt_pending_cmd *cmd;
+ struct mgmt_pending_cmd *cmd = data;
- bt_dev_dbg(hdev, "status 0x%02x", status);
+ bt_dev_dbg(hdev, "err %d", err);
hci_dev_lock(hdev);
- cmd = pending_find(MGMT_OP_SET_DISCOVERABLE, hdev);
- if (!cmd)
- goto unlock;
-
- if (status) {
- u8 mgmt_err = mgmt_status(status);
+ if (err) {
+ u8 mgmt_err = mgmt_status(err);
mgmt_cmd_status(cmd->sk, cmd->index, cmd->opcode, mgmt_err);
hci_dev_clear_flag(hdev, HCI_LIMITED_DISCOVERABLE);
- goto remove_cmd;
+ goto done;
}
if (hci_dev_test_flag(hdev, HCI_DISCOVERABLE) &&
@@ -1374,13 +1401,18 @@ void mgmt_set_discoverable_complete(struct hci_dev *hdev, u8 status)
send_settings_rsp(cmd->sk, MGMT_OP_SET_DISCOVERABLE, hdev);
new_settings(hdev, cmd->sk);
-remove_cmd:
- mgmt_pending_remove(cmd);
-
-unlock:
+done:
+ mgmt_pending_free(cmd);
hci_dev_unlock(hdev);
}
+static int set_discoverable_sync(struct hci_dev *hdev, void *data)
+{
+ BT_DBG("%s", hdev->name);
+
+ return hci_update_discoverable_sync(hdev);
+}
+
static int set_discoverable(struct sock *sk, struct hci_dev *hdev, void *data,
u16 len)
{
@@ -1479,7 +1511,7 @@ static int set_discoverable(struct sock *sk, struct hci_dev *hdev, void *data,
goto failed;
}
- cmd = mgmt_pending_add(sk, MGMT_OP_SET_DISCOVERABLE, hdev, data, len);
+ cmd = mgmt_pending_new(sk, MGMT_OP_SET_DISCOVERABLE, hdev, data, len);
if (!cmd) {
err = -ENOMEM;
goto failed;
@@ -1503,39 +1535,34 @@ static int set_discoverable(struct sock *sk, struct hci_dev *hdev, void *data,
else
hci_dev_clear_flag(hdev, HCI_LIMITED_DISCOVERABLE);
- queue_work(hdev->req_workqueue, &hdev->discoverable_update);
- err = 0;
+ err = hci_cmd_sync_queue(hdev, set_discoverable_sync, cmd,
+ mgmt_set_discoverable_complete);
failed:
hci_dev_unlock(hdev);
return err;
}
-void mgmt_set_connectable_complete(struct hci_dev *hdev, u8 status)
+static void mgmt_set_connectable_complete(struct hci_dev *hdev, void *data,
+ int err)
{
- struct mgmt_pending_cmd *cmd;
+ struct mgmt_pending_cmd *cmd = data;
- bt_dev_dbg(hdev, "status 0x%02x", status);
+ bt_dev_dbg(hdev, "err %d", err);
hci_dev_lock(hdev);
- cmd = pending_find(MGMT_OP_SET_CONNECTABLE, hdev);
- if (!cmd)
- goto unlock;
-
- if (status) {
- u8 mgmt_err = mgmt_status(status);
+ if (err) {
+ u8 mgmt_err = mgmt_status(err);
mgmt_cmd_status(cmd->sk, cmd->index, cmd->opcode, mgmt_err);
- goto remove_cmd;
+ goto done;
}
send_settings_rsp(cmd->sk, MGMT_OP_SET_CONNECTABLE, hdev);
new_settings(hdev, cmd->sk);
-remove_cmd:
- mgmt_pending_remove(cmd);
-
-unlock:
+done:
+ mgmt_pending_free(cmd);
hci_dev_unlock(hdev);
}
@@ -1561,13 +1588,20 @@ static int set_connectable_update_settings(struct hci_dev *hdev,
if (changed) {
hci_req_update_scan(hdev);
- hci_update_background_scan(hdev);
+ hci_update_passive_scan(hdev);
return new_settings(hdev, sk);
}
return 0;
}
+static int set_connectable_sync(struct hci_dev *hdev, void *data)
+{
+ BT_DBG("%s", hdev->name);
+
+ return hci_update_connectable_sync(hdev);
+}
+
static int set_connectable(struct sock *sk, struct hci_dev *hdev, void *data,
u16 len)
{
@@ -1600,7 +1634,7 @@ static int set_connectable(struct sock *sk, struct hci_dev *hdev, void *data,
goto failed;
}
- cmd = mgmt_pending_add(sk, MGMT_OP_SET_CONNECTABLE, hdev, data, len);
+ cmd = mgmt_pending_new(sk, MGMT_OP_SET_CONNECTABLE, hdev, data, len);
if (!cmd) {
err = -ENOMEM;
goto failed;
@@ -1617,8 +1651,8 @@ static int set_connectable(struct sock *sk, struct hci_dev *hdev, void *data,
hci_dev_clear_flag(hdev, HCI_CONNECTABLE);
}
- queue_work(hdev->req_workqueue, &hdev->connectable_update);
- err = 0;
+ err = hci_cmd_sync_queue(hdev, set_connectable_sync, cmd,
+ mgmt_set_connectable_complete);
failed:
hci_dev_unlock(hdev);
@@ -1653,12 +1687,7 @@ static int set_bondable(struct sock *sk, struct hci_dev *hdev, void *data,
/* In limited privacy mode the change of bondable mode
* may affect the local advertising address.
*/
- if (hdev_is_powered(hdev) &&
- hci_dev_test_flag(hdev, HCI_ADVERTISING) &&
- hci_dev_test_flag(hdev, HCI_DISCOVERABLE) &&
- hci_dev_test_flag(hdev, HCI_LIMITED_PRIVACY))
- queue_work(hdev->req_workqueue,
- &hdev->discoverable_update);
+ hci_update_discoverable(hdev);
err = new_settings(hdev, sk);
}
@@ -1737,6 +1766,69 @@ static int set_link_security(struct sock *sk, struct hci_dev *hdev, void *data,
return err;
}
+static void set_ssp_complete(struct hci_dev *hdev, void *data, int err)
+{
+ struct cmd_lookup match = { NULL, hdev };
+ struct mgmt_pending_cmd *cmd = data;
+ struct mgmt_mode *cp = cmd->param;
+ u8 enable = cp->val;
+ bool changed;
+
+ if (err) {
+ u8 mgmt_err = mgmt_status(err);
+
+ if (enable && hci_dev_test_and_clear_flag(hdev,
+ HCI_SSP_ENABLED)) {
+ hci_dev_clear_flag(hdev, HCI_HS_ENABLED);
+ new_settings(hdev, NULL);
+ }
+
+ mgmt_pending_foreach(MGMT_OP_SET_SSP, hdev, cmd_status_rsp,
+ &mgmt_err);
+ return;
+ }
+
+ if (enable) {
+ changed = !hci_dev_test_and_set_flag(hdev, HCI_SSP_ENABLED);
+ } else {
+ changed = hci_dev_test_and_clear_flag(hdev, HCI_SSP_ENABLED);
+
+ if (!changed)
+ changed = hci_dev_test_and_clear_flag(hdev,
+ HCI_HS_ENABLED);
+ else
+ hci_dev_clear_flag(hdev, HCI_HS_ENABLED);
+ }
+
+ mgmt_pending_foreach(MGMT_OP_SET_SSP, hdev, settings_rsp, &match);
+
+ if (changed)
+ new_settings(hdev, match.sk);
+
+ if (match.sk)
+ sock_put(match.sk);
+
+ hci_update_eir_sync(hdev);
+}
+
+static int set_ssp_sync(struct hci_dev *hdev, void *data)
+{
+ struct mgmt_pending_cmd *cmd = data;
+ struct mgmt_mode *cp = cmd->param;
+ bool changed = false;
+ int err;
+
+ if (cp->val)
+ changed = !hci_dev_test_and_set_flag(hdev, HCI_SSP_ENABLED);
+
+ err = hci_write_ssp_mode_sync(hdev, cp->val);
+
+ if (!err && changed)
+ hci_dev_clear_flag(hdev, HCI_SSP_ENABLED);
+
+ return err;
+}
+
static int set_ssp(struct sock *sk, struct hci_dev *hdev, void *data, u16 len)
{
struct mgmt_mode *cp = data;
@@ -1798,19 +1890,18 @@ static int set_ssp(struct sock *sk, struct hci_dev *hdev, void *data, u16 len)
}
cmd = mgmt_pending_add(sk, MGMT_OP_SET_SSP, hdev, data, len);
- if (!cmd) {
+ if (!cmd)
err = -ENOMEM;
- goto failed;
- }
+ else
+ err = hci_cmd_sync_queue(hdev, set_ssp_sync, cmd,
+ set_ssp_complete);
- if (!cp->val && hci_dev_test_flag(hdev, HCI_USE_DEBUG_KEYS))
- hci_send_cmd(hdev, HCI_OP_WRITE_SSP_DEBUG_MODE,
- sizeof(cp->val), &cp->val);
-
- err = hci_send_cmd(hdev, HCI_OP_WRITE_SSP_MODE, 1, &cp->val);
if (err < 0) {
- mgmt_pending_remove(cmd);
- goto failed;
+ err = mgmt_cmd_status(sk, hdev->id, MGMT_OP_SET_SSP,
+ MGMT_STATUS_FAILED);
+
+ if (cmd)
+ mgmt_pending_remove(cmd);
}
failed:
@@ -1879,18 +1970,17 @@ static int set_hs(struct sock *sk, struct hci_dev *hdev, void *data, u16 len)
return err;
}
-static void le_enable_complete(struct hci_dev *hdev, u8 status, u16 opcode)
+static void set_le_complete(struct hci_dev *hdev, void *data, int err)
{
struct cmd_lookup match = { NULL, hdev };
+ u8 status = mgmt_status(err);
- hci_dev_lock(hdev);
+ bt_dev_dbg(hdev, "err %d", err);
if (status) {
- u8 mgmt_err = mgmt_status(status);
-
mgmt_pending_foreach(MGMT_OP_SET_LE, hdev, cmd_status_rsp,
- &mgmt_err);
- goto unlock;
+ &status);
+ return;
}
mgmt_pending_foreach(MGMT_OP_SET_LE, hdev, settings_rsp, &match);
@@ -1899,39 +1989,54 @@ static void le_enable_complete(struct hci_dev *hdev, u8 status, u16 opcode)
if (match.sk)
sock_put(match.sk);
+}
+
+static int set_le_sync(struct hci_dev *hdev, void *data)
+{
+ struct mgmt_pending_cmd *cmd = data;
+ struct mgmt_mode *cp = cmd->param;
+ u8 val = !!cp->val;
+ int err;
+
+ if (!val) {
+ if (hci_dev_test_flag(hdev, HCI_LE_ADV))
+ hci_disable_advertising_sync(hdev);
+
+ if (ext_adv_capable(hdev))
+ hci_remove_ext_adv_instance_sync(hdev, 0, cmd->sk);
+ } else {
+ hci_dev_set_flag(hdev, HCI_LE_ENABLED);
+ }
+
+ err = hci_write_le_host_supported_sync(hdev, val, 0);
/* Make sure the controller has a good default for
* advertising data. Restrict the update to when LE
* has actually been enabled. During power on, the
* update in powered_update_hci will take care of it.
*/
- if (hci_dev_test_flag(hdev, HCI_LE_ENABLED)) {
- struct hci_request req;
- hci_req_init(&req, hdev);
+ if (!err && hci_dev_test_flag(hdev, HCI_LE_ENABLED)) {
if (ext_adv_capable(hdev)) {
- int err;
+ int status;
- err = __hci_req_setup_ext_adv_instance(&req, 0x00);
- if (!err)
- __hci_req_update_scan_rsp_data(&req, 0x00);
+ status = hci_setup_ext_adv_instance_sync(hdev, 0x00);
+ if (!status)
+ hci_update_scan_rsp_data_sync(hdev, 0x00);
} else {
- __hci_req_update_adv_data(&req, 0x00);
- __hci_req_update_scan_rsp_data(&req, 0x00);
+ hci_update_adv_data_sync(hdev, 0x00);
+ hci_update_scan_rsp_data_sync(hdev, 0x00);
}
- hci_req_run(&req, NULL);
- hci_update_background_scan(hdev);
+
+ hci_update_passive_scan(hdev);
}
-unlock:
- hci_dev_unlock(hdev);
+ return err;
}
static int set_le(struct sock *sk, struct hci_dev *hdev, void *data, u16 len)
{
struct mgmt_mode *cp = data;
- struct hci_cp_write_le_host_supported hci_cp;
struct mgmt_pending_cmd *cmd;
- struct hci_request req;
int err;
u8 val, enabled;
@@ -2001,33 +2106,20 @@ static int set_le(struct sock *sk, struct hci_dev *hdev, void *data, u16 len)
}
cmd = mgmt_pending_add(sk, MGMT_OP_SET_LE, hdev, data, len);
- if (!cmd) {
+ if (!cmd)
err = -ENOMEM;
- goto unlock;
+ else
+ err = hci_cmd_sync_queue(hdev, set_le_sync, cmd,
+ set_le_complete);
+
+ if (err < 0) {
+ err = mgmt_cmd_status(sk, hdev->id, MGMT_OP_SET_LE,
+ MGMT_STATUS_FAILED);
+
+ if (cmd)
+ mgmt_pending_remove(cmd);
}
- hci_req_init(&req, hdev);
-
- memset(&hci_cp, 0, sizeof(hci_cp));
-
- if (val) {
- hci_cp.le = val;
- hci_cp.simul = 0x00;
- } else {
- if (hci_dev_test_flag(hdev, HCI_LE_ADV))
- __hci_req_disable_advertising(&req);
-
- if (ext_adv_capable(hdev))
- __hci_req_clear_ext_adv_sets(&req);
- }
-
- hci_req_add(&req, HCI_OP_WRITE_LE_HOST_SUPPORTED, sizeof(hci_cp),
- &hci_cp);
-
- err = hci_req_run(&req, le_enable_complete);
- if (err < 0)
- mgmt_pending_remove(cmd);
-
unlock:
hci_dev_unlock(hdev);
return err;
@@ -2075,37 +2167,33 @@ static u8 get_uuid_size(const u8 *uuid)
return 16;
}
-static void mgmt_class_complete(struct hci_dev *hdev, u16 mgmt_op, u8 status)
+static void mgmt_class_complete(struct hci_dev *hdev, void *data, int err)
{
- struct mgmt_pending_cmd *cmd;
+ struct mgmt_pending_cmd *cmd = data;
- hci_dev_lock(hdev);
-
- cmd = pending_find(mgmt_op, hdev);
- if (!cmd)
- goto unlock;
+ bt_dev_dbg(hdev, "err %d", err);
mgmt_cmd_complete(cmd->sk, cmd->index, cmd->opcode,
- mgmt_status(status), hdev->dev_class, 3);
+ mgmt_status(err), hdev->dev_class, 3);
- mgmt_pending_remove(cmd);
-
-unlock:
- hci_dev_unlock(hdev);
+ mgmt_pending_free(cmd);
}
-static void add_uuid_complete(struct hci_dev *hdev, u8 status, u16 opcode)
+static int add_uuid_sync(struct hci_dev *hdev, void *data)
{
- bt_dev_dbg(hdev, "status 0x%02x", status);
+ int err;
- mgmt_class_complete(hdev, MGMT_OP_ADD_UUID, status);
+ err = hci_update_class_sync(hdev);
+ if (err)
+ return err;
+
+ return hci_update_eir_sync(hdev);
}
static int add_uuid(struct sock *sk, struct hci_dev *hdev, void *data, u16 len)
{
struct mgmt_cp_add_uuid *cp = data;
struct mgmt_pending_cmd *cmd;
- struct hci_request req;
struct bt_uuid *uuid;
int err;
@@ -2131,28 +2219,17 @@ static int add_uuid(struct sock *sk, struct hci_dev *hdev, void *data, u16 len)
list_add_tail(&uuid->list, &hdev->uuids);
- hci_req_init(&req, hdev);
-
- __hci_req_update_class(&req);
- __hci_req_update_eir(&req);
-
- err = hci_req_run(&req, add_uuid_complete);
- if (err < 0) {
- if (err != -ENODATA)
- goto failed;
-
- err = mgmt_cmd_complete(sk, hdev->id, MGMT_OP_ADD_UUID, 0,
- hdev->dev_class, 3);
- goto failed;
- }
-
- cmd = mgmt_pending_add(sk, MGMT_OP_ADD_UUID, hdev, data, len);
+ cmd = mgmt_pending_new(sk, MGMT_OP_ADD_UUID, hdev, data, len);
if (!cmd) {
err = -ENOMEM;
goto failed;
}
- err = 0;
+ err = hci_cmd_sync_queue(hdev, add_uuid_sync, cmd, mgmt_class_complete);
+ if (err < 0) {
+ mgmt_pending_free(cmd);
+ goto failed;
+ }
failed:
hci_dev_unlock(hdev);
@@ -2173,11 +2250,15 @@ static bool enable_service_cache(struct hci_dev *hdev)
return false;
}
-static void remove_uuid_complete(struct hci_dev *hdev, u8 status, u16 opcode)
+static int remove_uuid_sync(struct hci_dev *hdev, void *data)
{
- bt_dev_dbg(hdev, "status 0x%02x", status);
+ int err;
- mgmt_class_complete(hdev, MGMT_OP_REMOVE_UUID, status);
+ err = hci_update_class_sync(hdev);
+ if (err)
+ return err;
+
+ return hci_update_eir_sync(hdev);
}
static int remove_uuid(struct sock *sk, struct hci_dev *hdev, void *data,
@@ -2187,7 +2268,6 @@ static int remove_uuid(struct sock *sk, struct hci_dev *hdev, void *data,
struct mgmt_pending_cmd *cmd;
struct bt_uuid *match, *tmp;
u8 bt_uuid_any[] = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 };
- struct hci_request req;
int err, found;
bt_dev_dbg(hdev, "sock %p", sk);
@@ -2231,39 +2311,35 @@ static int remove_uuid(struct sock *sk, struct hci_dev *hdev, void *data,
}
update_class:
- hci_req_init(&req, hdev);
-
- __hci_req_update_class(&req);
- __hci_req_update_eir(&req);
-
- err = hci_req_run(&req, remove_uuid_complete);
- if (err < 0) {
- if (err != -ENODATA)
- goto unlock;
-
- err = mgmt_cmd_complete(sk, hdev->id, MGMT_OP_REMOVE_UUID, 0,
- hdev->dev_class, 3);
- goto unlock;
- }
-
- cmd = mgmt_pending_add(sk, MGMT_OP_REMOVE_UUID, hdev, data, len);
+ cmd = mgmt_pending_new(sk, MGMT_OP_REMOVE_UUID, hdev, data, len);
if (!cmd) {
err = -ENOMEM;
goto unlock;
}
- err = 0;
+ err = hci_cmd_sync_queue(hdev, remove_uuid_sync, cmd,
+ mgmt_class_complete);
+ if (err < 0)
+ mgmt_pending_free(cmd);
unlock:
hci_dev_unlock(hdev);
return err;
}
-static void set_class_complete(struct hci_dev *hdev, u8 status, u16 opcode)
+static int set_class_sync(struct hci_dev *hdev, void *data)
{
- bt_dev_dbg(hdev, "status 0x%02x", status);
+ int err = 0;
- mgmt_class_complete(hdev, MGMT_OP_SET_DEV_CLASS, status);
+ if (hci_dev_test_and_clear_flag(hdev, HCI_SERVICE_CACHE)) {
+ cancel_delayed_work_sync(&hdev->service_cache);
+ err = hci_update_eir_sync(hdev);
+ }
+
+ if (err)
+ return err;
+
+ return hci_update_class_sync(hdev);
}
static int set_dev_class(struct sock *sk, struct hci_dev *hdev, void *data,
@@ -2271,7 +2347,6 @@ static int set_dev_class(struct sock *sk, struct hci_dev *hdev, void *data,
{
struct mgmt_cp_set_dev_class *cp = data;
struct mgmt_pending_cmd *cmd;
- struct hci_request req;
int err;
bt_dev_dbg(hdev, "sock %p", sk);
@@ -2303,34 +2378,16 @@ static int set_dev_class(struct sock *sk, struct hci_dev *hdev, void *data,
goto unlock;
}
- hci_req_init(&req, hdev);
-
- if (hci_dev_test_and_clear_flag(hdev, HCI_SERVICE_CACHE)) {
- hci_dev_unlock(hdev);
- cancel_delayed_work_sync(&hdev->service_cache);
- hci_dev_lock(hdev);
- __hci_req_update_eir(&req);
- }
-
- __hci_req_update_class(&req);
-
- err = hci_req_run(&req, set_class_complete);
- if (err < 0) {
- if (err != -ENODATA)
- goto unlock;
-
- err = mgmt_cmd_complete(sk, hdev->id, MGMT_OP_SET_DEV_CLASS, 0,
- hdev->dev_class, 3);
- goto unlock;
- }
-
- cmd = mgmt_pending_add(sk, MGMT_OP_SET_DEV_CLASS, hdev, data, len);
+ cmd = mgmt_pending_new(sk, MGMT_OP_SET_DEV_CLASS, hdev, data, len);
if (!cmd) {
err = -ENOMEM;
goto unlock;
}
- err = 0;
+ err = hci_cmd_sync_queue(hdev, set_class_sync, cmd,
+ mgmt_class_complete);
+ if (err < 0)
+ mgmt_pending_free(cmd);
unlock:
hci_dev_unlock(hdev);
@@ -3228,65 +3285,70 @@ static int user_passkey_neg_reply(struct sock *sk, struct hci_dev *hdev,
HCI_OP_USER_PASSKEY_NEG_REPLY, 0);
}
-static void adv_expire(struct hci_dev *hdev, u32 flags)
+static int adv_expire_sync(struct hci_dev *hdev, u32 flags)
{
struct adv_info *adv_instance;
- struct hci_request req;
- int err;
adv_instance = hci_find_adv_instance(hdev, hdev->cur_adv_instance);
if (!adv_instance)
- return;
+ return 0;
/* stop if current instance doesn't need to be changed */
if (!(adv_instance->flags & flags))
- return;
+ return 0;
cancel_adv_timeout(hdev);
adv_instance = hci_get_next_instance(hdev, adv_instance->instance);
if (!adv_instance)
- return;
+ return 0;
- hci_req_init(&req, hdev);
- err = __hci_req_schedule_adv_instance(&req, adv_instance->instance,
- true);
- if (err)
- return;
+ hci_schedule_adv_instance_sync(hdev, adv_instance->instance, true);
- hci_req_run(&req, NULL);
+ return 0;
}
-static void set_name_complete(struct hci_dev *hdev, u8 status, u16 opcode)
+static int name_changed_sync(struct hci_dev *hdev, void *data)
{
- struct mgmt_cp_set_local_name *cp;
- struct mgmt_pending_cmd *cmd;
+ return adv_expire_sync(hdev, MGMT_ADV_FLAG_LOCAL_NAME);
+}
- bt_dev_dbg(hdev, "status 0x%02x", status);
+static void set_name_complete(struct hci_dev *hdev, void *data, int err)
+{
+ struct mgmt_pending_cmd *cmd = data;
+ struct mgmt_cp_set_local_name *cp = cmd->param;
+ u8 status = mgmt_status(err);
- hci_dev_lock(hdev);
-
- cmd = pending_find(MGMT_OP_SET_LOCAL_NAME, hdev);
- if (!cmd)
- goto unlock;
-
- cp = cmd->param;
+ bt_dev_dbg(hdev, "err %d", err);
if (status) {
mgmt_cmd_status(cmd->sk, hdev->id, MGMT_OP_SET_LOCAL_NAME,
- mgmt_status(status));
+ status);
} else {
mgmt_cmd_complete(cmd->sk, hdev->id, MGMT_OP_SET_LOCAL_NAME, 0,
cp, sizeof(*cp));
if (hci_dev_test_flag(hdev, HCI_LE_ADV))
- adv_expire(hdev, MGMT_ADV_FLAG_LOCAL_NAME);
+ hci_cmd_sync_queue(hdev, name_changed_sync, NULL, NULL);
}
mgmt_pending_remove(cmd);
+}
-unlock:
- hci_dev_unlock(hdev);
+static int set_name_sync(struct hci_dev *hdev, void *data)
+{
+ if (lmp_bredr_capable(hdev)) {
+ hci_update_name_sync(hdev);
+ hci_update_eir_sync(hdev);
+ }
+
+ /* The name is stored in the scan response data and so
+ * no need to update the advertising data here.
+ */
+ if (lmp_le_capable(hdev) && hci_dev_test_flag(hdev, HCI_ADVERTISING))
+ hci_update_scan_rsp_data_sync(hdev, hdev->cur_adv_instance);
+
+ return 0;
}
static int set_local_name(struct sock *sk, struct hci_dev *hdev, void *data,
@@ -3294,7 +3356,6 @@ static int set_local_name(struct sock *sk, struct hci_dev *hdev, void *data,
{
struct mgmt_cp_set_local_name *cp = data;
struct mgmt_pending_cmd *cmd;
- struct hci_request req;
int err;
bt_dev_dbg(hdev, "sock %p", sk);
@@ -3330,35 +3391,34 @@ static int set_local_name(struct sock *sk, struct hci_dev *hdev, void *data,
}
cmd = mgmt_pending_add(sk, MGMT_OP_SET_LOCAL_NAME, hdev, data, len);
- if (!cmd) {
+ if (!cmd)
err = -ENOMEM;
+ else
+ err = hci_cmd_sync_queue(hdev, set_name_sync, cmd,
+ set_name_complete);
+
+ if (err < 0) {
+ err = mgmt_cmd_status(sk, hdev->id, MGMT_OP_SET_LOCAL_NAME,
+ MGMT_STATUS_FAILED);
+
+ if (cmd)
+ mgmt_pending_remove(cmd);
+
goto failed;
}
memcpy(hdev->dev_name, cp->name, sizeof(hdev->dev_name));
- hci_req_init(&req, hdev);
-
- if (lmp_bredr_capable(hdev)) {
- __hci_req_update_name(&req);
- __hci_req_update_eir(&req);
- }
-
- /* The name is stored in the scan response data and so
- * no need to update the advertising data here.
- */
- if (lmp_le_capable(hdev) && hci_dev_test_flag(hdev, HCI_ADVERTISING))
- __hci_req_update_scan_rsp_data(&req, hdev->cur_adv_instance);
-
- err = hci_req_run(&req, set_name_complete);
- if (err < 0)
- mgmt_pending_remove(cmd);
-
failed:
hci_dev_unlock(hdev);
return err;
}
+static int appearance_changed_sync(struct hci_dev *hdev, void *data)
+{
+ return adv_expire_sync(hdev, MGMT_ADV_FLAG_APPEARANCE);
+}
+
static int set_appearance(struct sock *sk, struct hci_dev *hdev, void *data,
u16 len)
{
@@ -3380,7 +3440,8 @@ static int set_appearance(struct sock *sk, struct hci_dev *hdev, void *data,
hdev->appearance = appearance;
if (hci_dev_test_flag(hdev, HCI_LE_ADV))
- adv_expire(hdev, MGMT_ADV_FLAG_APPEARANCE);
+ hci_cmd_sync_queue(hdev, appearance_changed_sync, NULL,
+ NULL);
ext_info_changed(hdev, sk);
}
@@ -3426,23 +3487,26 @@ int mgmt_phy_configuration_changed(struct hci_dev *hdev, struct sock *skip)
sizeof(ev), skip);
}
-static void set_default_phy_complete(struct hci_dev *hdev, u8 status,
- u16 opcode, struct sk_buff *skb)
+static void set_default_phy_complete(struct hci_dev *hdev, void *data, int err)
{
- struct mgmt_pending_cmd *cmd;
+ struct mgmt_pending_cmd *cmd = data;
+ struct sk_buff *skb = cmd->skb;
+ u8 status = mgmt_status(err);
- bt_dev_dbg(hdev, "status 0x%02x", status);
+ if (!status) {
+ if (!skb)
+ status = MGMT_STATUS_FAILED;
+ else if (IS_ERR(skb))
+ status = mgmt_status(PTR_ERR(skb));
+ else
+ status = mgmt_status(skb->data[0]);
+ }
- hci_dev_lock(hdev);
-
- cmd = pending_find(MGMT_OP_SET_PHY_CONFIGURATION, hdev);
- if (!cmd)
- goto unlock;
+ bt_dev_dbg(hdev, "status %d", status);
if (status) {
mgmt_cmd_status(cmd->sk, hdev->id,
- MGMT_OP_SET_PHY_CONFIGURATION,
- mgmt_status(status));
+ MGMT_OP_SET_PHY_CONFIGURATION, status);
} else {
mgmt_cmd_complete(cmd->sk, hdev->id,
MGMT_OP_SET_PHY_CONFIGURATION, 0,
@@ -3451,19 +3515,56 @@ static void set_default_phy_complete(struct hci_dev *hdev, u8 status,
mgmt_phy_configuration_changed(hdev, cmd->sk);
}
- mgmt_pending_remove(cmd);
+ if (skb && !IS_ERR(skb))
+ kfree_skb(skb);
-unlock:
- hci_dev_unlock(hdev);
+ mgmt_pending_remove(cmd);
+}
+
+static int set_default_phy_sync(struct hci_dev *hdev, void *data)
+{
+ struct mgmt_pending_cmd *cmd = data;
+ struct mgmt_cp_set_phy_configuration *cp = cmd->param;
+ struct hci_cp_le_set_default_phy cp_phy;
+ u32 selected_phys = __le32_to_cpu(cp->selected_phys);
+
+ memset(&cp_phy, 0, sizeof(cp_phy));
+
+ if (!(selected_phys & MGMT_PHY_LE_TX_MASK))
+ cp_phy.all_phys |= 0x01;
+
+ if (!(selected_phys & MGMT_PHY_LE_RX_MASK))
+ cp_phy.all_phys |= 0x02;
+
+ if (selected_phys & MGMT_PHY_LE_1M_TX)
+ cp_phy.tx_phys |= HCI_LE_SET_PHY_1M;
+
+ if (selected_phys & MGMT_PHY_LE_2M_TX)
+ cp_phy.tx_phys |= HCI_LE_SET_PHY_2M;
+
+ if (selected_phys & MGMT_PHY_LE_CODED_TX)
+ cp_phy.tx_phys |= HCI_LE_SET_PHY_CODED;
+
+ if (selected_phys & MGMT_PHY_LE_1M_RX)
+ cp_phy.rx_phys |= HCI_LE_SET_PHY_1M;
+
+ if (selected_phys & MGMT_PHY_LE_2M_RX)
+ cp_phy.rx_phys |= HCI_LE_SET_PHY_2M;
+
+ if (selected_phys & MGMT_PHY_LE_CODED_RX)
+ cp_phy.rx_phys |= HCI_LE_SET_PHY_CODED;
+
+ cmd->skb = __hci_cmd_sync(hdev, HCI_OP_LE_SET_DEFAULT_PHY,
+ sizeof(cp_phy), &cp_phy, HCI_CMD_TIMEOUT);
+
+ return 0;
}
static int set_phy_configuration(struct sock *sk, struct hci_dev *hdev,
void *data, u16 len)
{
struct mgmt_cp_set_phy_configuration *cp = data;
- struct hci_cp_le_set_default_phy cp_phy;
struct mgmt_pending_cmd *cmd;
- struct hci_request req;
u32 selected_phys, configurable_phys, supported_phys, unconfigure_phys;
u16 pkt_type = (HCI_DH1 | HCI_DM1);
bool changed = false;
@@ -3567,45 +3668,21 @@ static int set_phy_configuration(struct sock *sk, struct hci_dev *hdev,
cmd = mgmt_pending_add(sk, MGMT_OP_SET_PHY_CONFIGURATION, hdev, data,
len);
- if (!cmd) {
+ if (!cmd)
err = -ENOMEM;
- goto unlock;
+ else
+ err = hci_cmd_sync_queue(hdev, set_default_phy_sync, cmd,
+ set_default_phy_complete);
+
+ if (err < 0) {
+ err = mgmt_cmd_status(sk, hdev->id,
+ MGMT_OP_SET_PHY_CONFIGURATION,
+ MGMT_STATUS_FAILED);
+
+ if (cmd)
+ mgmt_pending_remove(cmd);
}
- hci_req_init(&req, hdev);
-
- memset(&cp_phy, 0, sizeof(cp_phy));
-
- if (!(selected_phys & MGMT_PHY_LE_TX_MASK))
- cp_phy.all_phys |= 0x01;
-
- if (!(selected_phys & MGMT_PHY_LE_RX_MASK))
- cp_phy.all_phys |= 0x02;
-
- if (selected_phys & MGMT_PHY_LE_1M_TX)
- cp_phy.tx_phys |= HCI_LE_SET_PHY_1M;
-
- if (selected_phys & MGMT_PHY_LE_2M_TX)
- cp_phy.tx_phys |= HCI_LE_SET_PHY_2M;
-
- if (selected_phys & MGMT_PHY_LE_CODED_TX)
- cp_phy.tx_phys |= HCI_LE_SET_PHY_CODED;
-
- if (selected_phys & MGMT_PHY_LE_1M_RX)
- cp_phy.rx_phys |= HCI_LE_SET_PHY_1M;
-
- if (selected_phys & MGMT_PHY_LE_2M_RX)
- cp_phy.rx_phys |= HCI_LE_SET_PHY_2M;
-
- if (selected_phys & MGMT_PHY_LE_CODED_RX)
- cp_phy.rx_phys |= HCI_LE_SET_PHY_CODED;
-
- hci_req_add(&req, HCI_OP_LE_SET_DEFAULT_PHY, sizeof(cp_phy), &cp_phy);
-
- err = hci_req_run_skb(&req, set_default_phy_complete);
- if (err < 0)
- mgmt_pending_remove(cmd);
-
unlock:
hci_dev_unlock(hdev);
@@ -3805,7 +3882,7 @@ static const u8 offload_codecs_uuid[16] = {
};
/* 671b10b5-42c0-4696-9227-eb28d1b049d6 */
-static const u8 simult_central_periph_uuid[16] = {
+static const u8 le_simultaneous_roles_uuid[16] = {
0xd6, 0x49, 0xb0, 0xd1, 0x28, 0xeb, 0x27, 0x92,
0x96, 0x46, 0xc0, 0x42, 0xb5, 0x10, 0x1b, 0x67,
};
@@ -3838,21 +3915,18 @@ static int read_exp_features_info(struct sock *sk, struct hci_dev *hdev,
}
#endif
- if (hdev) {
- if (test_bit(HCI_QUIRK_VALID_LE_STATES, &hdev->quirks) &&
- (hdev->le_states[4] & 0x08) && /* Central */
- (hdev->le_states[4] & 0x40) && /* Peripheral */
- (hdev->le_states[3] & 0x10)) /* Simultaneous */
+ if (hdev && hci_dev_le_state_simultaneous(hdev)) {
+ if (hci_dev_test_flag(hdev, HCI_LE_SIMULTANEOUS_ROLES))
flags = BIT(0);
else
flags = 0;
- memcpy(rp->features[idx].uuid, simult_central_periph_uuid, 16);
+ memcpy(rp->features[idx].uuid, le_simultaneous_roles_uuid, 16);
rp->features[idx].flags = cpu_to_le32(flags);
idx++;
}
- if (hdev && use_ll_privacy(hdev)) {
+ if (hdev && ll_privacy_capable(hdev)) {
if (hci_dev_test_flag(hdev, HCI_ENABLE_LL_PRIVACY))
flags = BIT(0) | BIT(1);
else
@@ -3863,7 +3937,8 @@ static int read_exp_features_info(struct sock *sk, struct hci_dev *hdev,
idx++;
}
- if (hdev && hdev->set_quality_report) {
+ if (hdev && (aosp_has_quality_report(hdev) ||
+ hdev->set_quality_report)) {
if (hci_dev_test_flag(hdev, HCI_QUALITY_REPORT))
flags = BIT(0);
else
@@ -3906,36 +3981,27 @@ static int exp_ll_privacy_feature_changed(bool enabled, struct hci_dev *hdev,
memcpy(ev.uuid, rpa_resolution_uuid, 16);
ev.flags = cpu_to_le32((enabled ? BIT(0) : 0) | BIT(1));
+ if (enabled && privacy_mode_capable(hdev))
+ set_bit(HCI_CONN_FLAG_DEVICE_PRIVACY, hdev->conn_flags);
+ else
+ clear_bit(HCI_CONN_FLAG_DEVICE_PRIVACY, hdev->conn_flags);
+
return mgmt_limited_event(MGMT_EV_EXP_FEATURE_CHANGED, hdev,
&ev, sizeof(ev),
HCI_MGMT_EXP_FEATURE_EVENTS, skip);
}
-#ifdef CONFIG_BT_FEATURE_DEBUG
-static int exp_debug_feature_changed(bool enabled, struct sock *skip)
+static int exp_feature_changed(struct hci_dev *hdev, const u8 *uuid,
+ bool enabled, struct sock *skip)
{
struct mgmt_ev_exp_feature_changed ev;
memset(&ev, 0, sizeof(ev));
- memcpy(ev.uuid, debug_uuid, 16);
+ memcpy(ev.uuid, uuid, 16);
ev.flags = cpu_to_le32(enabled ? BIT(0) : 0);
- return mgmt_limited_event(MGMT_EV_EXP_FEATURE_CHANGED, NULL,
- &ev, sizeof(ev),
- HCI_MGMT_EXP_FEATURE_EVENTS, skip);
-}
-#endif
-
-static int exp_quality_report_feature_changed(bool enabled, struct sock *skip)
-{
- struct mgmt_ev_exp_feature_changed ev;
-
- memset(&ev, 0, sizeof(ev));
- memcpy(ev.uuid, quality_report_uuid, 16);
- ev.flags = cpu_to_le32(enabled ? BIT(0) : 0);
-
- return mgmt_limited_event(MGMT_EV_EXP_FEATURE_CHANGED, NULL,
+ return mgmt_limited_event(MGMT_EV_EXP_FEATURE_CHANGED, hdev,
&ev, sizeof(ev),
HCI_MGMT_EXP_FEATURE_EVENTS, skip);
}
@@ -3962,17 +4028,18 @@ static int set_zero_key_func(struct sock *sk, struct hci_dev *hdev,
bt_dbg_set(false);
if (changed)
- exp_debug_feature_changed(false, sk);
+ exp_feature_changed(NULL, ZERO_KEY, false, sk);
}
#endif
if (hdev && use_ll_privacy(hdev) && !hdev_is_powered(hdev)) {
- bool changed = hci_dev_test_flag(hdev, HCI_ENABLE_LL_PRIVACY);
+ bool changed;
- hci_dev_clear_flag(hdev, HCI_ENABLE_LL_PRIVACY);
-
+ changed = hci_dev_test_and_clear_flag(hdev,
+ HCI_ENABLE_LL_PRIVACY);
if (changed)
- exp_ll_privacy_feature_changed(false, hdev, sk);
+ exp_feature_changed(hdev, rpa_resolution_uuid, false,
+ sk);
}
hci_sock_set_flag(sk, HCI_MGMT_EXP_FEATURE_EVENTS);
@@ -4023,7 +4090,7 @@ static int set_debug_func(struct sock *sk, struct hci_dev *hdev,
&rp, sizeof(rp));
if (changed)
- exp_debug_feature_changed(val, sk);
+ exp_feature_changed(hdev, debug_uuid, val, sk);
return err;
}
@@ -4065,15 +4132,15 @@ static int set_rpa_resolution_func(struct sock *sk, struct hci_dev *hdev,
val = !!cp->param[0];
if (val) {
- changed = !hci_dev_test_flag(hdev, HCI_ENABLE_LL_PRIVACY);
- hci_dev_set_flag(hdev, HCI_ENABLE_LL_PRIVACY);
+ changed = !hci_dev_test_and_set_flag(hdev,
+ HCI_ENABLE_LL_PRIVACY);
hci_dev_clear_flag(hdev, HCI_ADVERTISING);
/* Enable LL privacy + supported settings changed */
flags = BIT(0) | BIT(1);
} else {
- changed = hci_dev_test_flag(hdev, HCI_ENABLE_LL_PRIVACY);
- hci_dev_clear_flag(hdev, HCI_ENABLE_LL_PRIVACY);
+ changed = hci_dev_test_and_clear_flag(hdev,
+ HCI_ENABLE_LL_PRIVACY);
/* Disable LL privacy + supported settings changed */
flags = BIT(1);
@@ -4125,7 +4192,7 @@ static int set_quality_report_func(struct sock *sk, struct hci_dev *hdev,
val = !!cp->param[0];
changed = (val != hci_dev_test_flag(hdev, HCI_QUALITY_REPORT));
- if (!hdev->set_quality_report) {
+ if (!aosp_has_quality_report(hdev) && !hdev->set_quality_report) {
err = mgmt_cmd_status(sk, hdev->id,
MGMT_OP_SET_EXP_FEATURE,
MGMT_STATUS_NOT_SUPPORTED);
@@ -4133,13 +4200,18 @@ static int set_quality_report_func(struct sock *sk, struct hci_dev *hdev,
}
if (changed) {
- err = hdev->set_quality_report(hdev, val);
+ if (hdev->set_quality_report)
+ err = hdev->set_quality_report(hdev, val);
+ else
+ err = aosp_set_quality_report(hdev, val);
+
if (err) {
err = mgmt_cmd_status(sk, hdev->id,
MGMT_OP_SET_EXP_FEATURE,
MGMT_STATUS_FAILED);
goto unlock_quality_report;
}
+
if (val)
hci_dev_set_flag(hdev, HCI_QUALITY_REPORT);
else
@@ -4151,31 +4223,18 @@ static int set_quality_report_func(struct sock *sk, struct hci_dev *hdev,
memcpy(rp.uuid, quality_report_uuid, 16);
rp.flags = cpu_to_le32(val ? BIT(0) : 0);
hci_sock_set_flag(sk, HCI_MGMT_EXP_FEATURE_EVENTS);
- err = mgmt_cmd_complete(sk, hdev->id,
- MGMT_OP_SET_EXP_FEATURE, 0,
+
+ err = mgmt_cmd_complete(sk, hdev->id, MGMT_OP_SET_EXP_FEATURE, 0,
&rp, sizeof(rp));
if (changed)
- exp_quality_report_feature_changed(val, sk);
+ exp_feature_changed(hdev, quality_report_uuid, val, sk);
unlock_quality_report:
hci_req_sync_unlock(hdev);
return err;
}
-static int exp_offload_codec_feature_changed(bool enabled, struct sock *skip)
-{
- struct mgmt_ev_exp_feature_changed ev;
-
- memset(&ev, 0, sizeof(ev));
- memcpy(ev.uuid, offload_codecs_uuid, 16);
- ev.flags = cpu_to_le32(enabled ? BIT(0) : 0);
-
- return mgmt_limited_event(MGMT_EV_EXP_FEATURE_CHANGED, NULL,
- &ev, sizeof(ev),
- HCI_MGMT_EXP_FEATURE_EVENTS, skip);
-}
-
static int set_offload_codec_func(struct sock *sk, struct hci_dev *hdev,
struct mgmt_cp_set_exp_feature *cp,
u16 data_len)
@@ -4229,7 +4288,65 @@ static int set_offload_codec_func(struct sock *sk, struct hci_dev *hdev,
&rp, sizeof(rp));
if (changed)
- exp_offload_codec_feature_changed(val, sk);
+ exp_feature_changed(hdev, offload_codecs_uuid, val, sk);
+
+ return err;
+}
+
+static int set_le_simultaneous_roles_func(struct sock *sk, struct hci_dev *hdev,
+ struct mgmt_cp_set_exp_feature *cp,
+ u16 data_len)
+{
+ bool val, changed;
+ int err;
+ struct mgmt_rp_set_exp_feature rp;
+
+ /* Command requires to use a valid controller index */
+ if (!hdev)
+ return mgmt_cmd_status(sk, MGMT_INDEX_NONE,
+ MGMT_OP_SET_EXP_FEATURE,
+ MGMT_STATUS_INVALID_INDEX);
+
+ /* Parameters are limited to a single octet */
+ if (data_len != MGMT_SET_EXP_FEATURE_SIZE + 1)
+ return mgmt_cmd_status(sk, hdev->id,
+ MGMT_OP_SET_EXP_FEATURE,
+ MGMT_STATUS_INVALID_PARAMS);
+
+ /* Only boolean on/off is supported */
+ if (cp->param[0] != 0x00 && cp->param[0] != 0x01)
+ return mgmt_cmd_status(sk, hdev->id,
+ MGMT_OP_SET_EXP_FEATURE,
+ MGMT_STATUS_INVALID_PARAMS);
+
+ val = !!cp->param[0];
+ changed = (val != hci_dev_test_flag(hdev, HCI_LE_SIMULTANEOUS_ROLES));
+
+ if (!hci_dev_le_state_simultaneous(hdev)) {
+ return mgmt_cmd_status(sk, hdev->id,
+ MGMT_OP_SET_EXP_FEATURE,
+ MGMT_STATUS_NOT_SUPPORTED);
+ }
+
+ if (changed) {
+ if (val)
+ hci_dev_set_flag(hdev, HCI_LE_SIMULTANEOUS_ROLES);
+ else
+ hci_dev_clear_flag(hdev, HCI_LE_SIMULTANEOUS_ROLES);
+ }
+
+ bt_dev_info(hdev, "LE simultaneous roles enable %d changed %d",
+ val, changed);
+
+ memcpy(rp.uuid, le_simultaneous_roles_uuid, 16);
+ rp.flags = cpu_to_le32(val ? BIT(0) : 0);
+ hci_sock_set_flag(sk, HCI_MGMT_EXP_FEATURE_EVENTS);
+ err = mgmt_cmd_complete(sk, hdev->id,
+ MGMT_OP_SET_EXP_FEATURE, 0,
+ &rp, sizeof(rp));
+
+ if (changed)
+ exp_feature_changed(hdev, le_simultaneous_roles_uuid, val, sk);
return err;
}
@@ -4246,6 +4363,7 @@ static const struct mgmt_exp_feature {
EXP_FEAT(rpa_resolution_uuid, set_rpa_resolution_func),
EXP_FEAT(quality_report_uuid, set_quality_report_func),
EXP_FEAT(offload_codecs_uuid, set_offload_codec_func),
+ EXP_FEAT(le_simultaneous_roles_uuid, set_le_simultaneous_roles_func),
/* end with a null feature */
EXP_FEAT(NULL, NULL)
@@ -4269,8 +4387,6 @@ static int set_exp_feature(struct sock *sk, struct hci_dev *hdev,
MGMT_STATUS_NOT_SUPPORTED);
}
-#define SUPPORTED_DEVICE_FLAGS() ((1U << HCI_CONN_FLAG_MAX) - 1)
-
static int get_device_flags(struct sock *sk, struct hci_dev *hdev, void *data,
u16 data_len)
{
@@ -4278,7 +4394,7 @@ static int get_device_flags(struct sock *sk, struct hci_dev *hdev, void *data,
struct mgmt_rp_get_device_flags rp;
struct bdaddr_list_with_flags *br_params;
struct hci_conn_params *params;
- u32 supported_flags = SUPPORTED_DEVICE_FLAGS();
+ u32 supported_flags;
u32 current_flags = 0;
u8 status = MGMT_STATUS_INVALID_PARAMS;
@@ -4287,6 +4403,9 @@ static int get_device_flags(struct sock *sk, struct hci_dev *hdev, void *data,
hci_dev_lock(hdev);
+ bitmap_to_arr32(&supported_flags, hdev->conn_flags,
+ __HCI_CONN_NUM_FLAGS);
+
memset(&rp, 0, sizeof(rp));
if (cp->addr.type == BDADDR_BREDR) {
@@ -4296,7 +4415,8 @@ static int get_device_flags(struct sock *sk, struct hci_dev *hdev, void *data,
if (!br_params)
goto done;
- current_flags = br_params->current_flags;
+ bitmap_to_arr32(¤t_flags, br_params->flags,
+ __HCI_CONN_NUM_FLAGS);
} else {
params = hci_conn_params_lookup(hdev, &cp->addr.bdaddr,
le_addr_type(cp->addr.type));
@@ -4304,7 +4424,8 @@ static int get_device_flags(struct sock *sk, struct hci_dev *hdev, void *data,
if (!params)
goto done;
- current_flags = params->current_flags;
+ bitmap_to_arr32(¤t_flags, params->flags,
+ __HCI_CONN_NUM_FLAGS);
}
bacpy(&rp.addr.bdaddr, &cp->addr.bdaddr);
@@ -4342,13 +4463,16 @@ static int set_device_flags(struct sock *sk, struct hci_dev *hdev, void *data,
struct bdaddr_list_with_flags *br_params;
struct hci_conn_params *params;
u8 status = MGMT_STATUS_INVALID_PARAMS;
- u32 supported_flags = SUPPORTED_DEVICE_FLAGS();
+ u32 supported_flags;
u32 current_flags = __le32_to_cpu(cp->current_flags);
bt_dev_dbg(hdev, "Set device flags %pMR (type 0x%x) = 0x%x",
&cp->addr.bdaddr, cp->addr.type,
__le32_to_cpu(current_flags));
+ bitmap_to_arr32(&supported_flags, hdev->conn_flags,
+ __HCI_CONN_NUM_FLAGS);
+
if ((supported_flags | current_flags) != supported_flags) {
bt_dev_warn(hdev, "Bad flag given (0x%x) vs supported (0x%0x)",
current_flags, supported_flags);
@@ -4363,7 +4487,7 @@ static int set_device_flags(struct sock *sk, struct hci_dev *hdev, void *data,
cp->addr.type);
if (br_params) {
- br_params->current_flags = current_flags;
+ bitmap_from_u64(br_params->flags, current_flags);
status = MGMT_STATUS_SUCCESS;
} else {
bt_dev_warn(hdev, "No such BR/EDR device %pMR (0x%x)",
@@ -4373,8 +4497,15 @@ static int set_device_flags(struct sock *sk, struct hci_dev *hdev, void *data,
params = hci_conn_params_lookup(hdev, &cp->addr.bdaddr,
le_addr_type(cp->addr.type));
if (params) {
- params->current_flags = current_flags;
+ bitmap_from_u64(params->flags, current_flags);
status = MGMT_STATUS_SUCCESS;
+
+ /* Update passive scan if HCI_CONN_FLAG_DEVICE_PRIVACY
+ * has been set.
+ */
+ if (test_bit(HCI_CONN_FLAG_DEVICE_PRIVACY,
+ params->flags))
+ hci_update_passive_scan(hdev);
} else {
bt_dev_warn(hdev, "No such LE device %pMR (0x%x)",
&cp->addr.bdaddr,
@@ -4496,7 +4627,7 @@ int mgmt_add_adv_patterns_monitor_complete(struct hci_dev *hdev, u8 status)
hdev->adv_monitors_cnt++;
if (monitor->state == ADV_MONITOR_STATE_NOT_REGISTERED)
monitor->state = ADV_MONITOR_STATE_REGISTERED;
- hci_update_background_scan(hdev);
+ hci_update_passive_scan(hdev);
}
err = mgmt_cmd_complete(cmd->sk, cmd->index, cmd->opcode,
@@ -4722,7 +4853,7 @@ int mgmt_remove_adv_monitor_complete(struct hci_dev *hdev, u8 status)
rp.monitor_handle = cp->monitor_handle;
if (!status)
- hci_update_background_scan(hdev);
+ hci_update_passive_scan(hdev);
err = mgmt_cmd_complete(cmd->sk, cmd->index, cmd->opcode,
mgmt_status(status), &rp, sizeof(rp));
@@ -4801,28 +4932,33 @@ static int remove_adv_monitor(struct sock *sk, struct hci_dev *hdev,
status);
}
-static void read_local_oob_data_complete(struct hci_dev *hdev, u8 status,
- u16 opcode, struct sk_buff *skb)
+static void read_local_oob_data_complete(struct hci_dev *hdev, void *data, int err)
{
struct mgmt_rp_read_local_oob_data mgmt_rp;
size_t rp_size = sizeof(mgmt_rp);
- struct mgmt_pending_cmd *cmd;
+ struct mgmt_pending_cmd *cmd = data;
+ struct sk_buff *skb = cmd->skb;
+ u8 status = mgmt_status(err);
- bt_dev_dbg(hdev, "status %u", status);
+ if (!status) {
+ if (!skb)
+ status = MGMT_STATUS_FAILED;
+ else if (IS_ERR(skb))
+ status = mgmt_status(PTR_ERR(skb));
+ else
+ status = mgmt_status(skb->data[0]);
+ }
- cmd = pending_find(MGMT_OP_READ_LOCAL_OOB_DATA, hdev);
- if (!cmd)
- return;
+ bt_dev_dbg(hdev, "status %d", status);
- if (status || !skb) {
- mgmt_cmd_status(cmd->sk, hdev->id, MGMT_OP_READ_LOCAL_OOB_DATA,
- status ? mgmt_status(status) : MGMT_STATUS_FAILED);
+ if (status) {
+ mgmt_cmd_status(cmd->sk, hdev->id, MGMT_OP_READ_LOCAL_OOB_DATA, status);
goto remove;
}
memset(&mgmt_rp, 0, sizeof(mgmt_rp));
- if (opcode == HCI_OP_READ_LOCAL_OOB_DATA) {
+ if (!bredr_sc_enabled(hdev)) {
struct hci_rp_read_local_oob_data *rp = (void *) skb->data;
if (skb->len < sizeof(*rp)) {
@@ -4857,14 +4993,31 @@ static void read_local_oob_data_complete(struct hci_dev *hdev, u8 status,
MGMT_STATUS_SUCCESS, &mgmt_rp, rp_size);
remove:
- mgmt_pending_remove(cmd);
+ if (skb && !IS_ERR(skb))
+ kfree_skb(skb);
+
+ mgmt_pending_free(cmd);
+}
+
+static int read_local_oob_data_sync(struct hci_dev *hdev, void *data)
+{
+ struct mgmt_pending_cmd *cmd = data;
+
+ if (bredr_sc_enabled(hdev))
+ cmd->skb = hci_read_local_oob_data_sync(hdev, true, cmd->sk);
+ else
+ cmd->skb = hci_read_local_oob_data_sync(hdev, false, cmd->sk);
+
+ if (IS_ERR(cmd->skb))
+ return PTR_ERR(cmd->skb);
+ else
+ return 0;
}
static int read_local_oob_data(struct sock *sk, struct hci_dev *hdev,
void *data, u16 data_len)
{
struct mgmt_pending_cmd *cmd;
- struct hci_request req;
int err;
bt_dev_dbg(hdev, "sock %p", sk);
@@ -4889,22 +5042,20 @@ static int read_local_oob_data(struct sock *sk, struct hci_dev *hdev,
goto unlock;
}
- cmd = mgmt_pending_add(sk, MGMT_OP_READ_LOCAL_OOB_DATA, hdev, NULL, 0);
- if (!cmd) {
+ cmd = mgmt_pending_new(sk, MGMT_OP_READ_LOCAL_OOB_DATA, hdev, NULL, 0);
+ if (!cmd)
err = -ENOMEM;
- goto unlock;
- }
-
- hci_req_init(&req, hdev);
-
- if (bredr_sc_enabled(hdev))
- hci_req_add(&req, HCI_OP_READ_LOCAL_OOB_EXT_DATA, 0, NULL);
else
- hci_req_add(&req, HCI_OP_READ_LOCAL_OOB_DATA, 0, NULL);
+ err = hci_cmd_sync_queue(hdev, read_local_oob_data_sync, cmd,
+ read_local_oob_data_complete);
- err = hci_req_run_skb(&req, read_local_oob_data_complete);
- if (err < 0)
- mgmt_pending_remove(cmd);
+ if (err < 0) {
+ err = mgmt_cmd_status(sk, hdev->id, MGMT_OP_READ_LOCAL_OOB_DATA,
+ MGMT_STATUS_FAILED);
+
+ if (cmd)
+ mgmt_pending_free(cmd);
+ }
unlock:
hci_dev_unlock(hdev);
@@ -5077,13 +5228,6 @@ void mgmt_start_discovery_complete(struct hci_dev *hdev, u8 status)
}
hci_dev_unlock(hdev);
-
- /* Handle suspend notifier */
- if (test_and_clear_bit(SUSPEND_UNPAUSE_DISCOVERY,
- hdev->suspend_tasks)) {
- bt_dev_dbg(hdev, "Unpaused discovery");
- wake_up(&hdev->suspend_wait_q);
- }
}
static bool discovery_type_is_valid(struct hci_dev *hdev, uint8_t type,
@@ -5113,6 +5257,25 @@ static bool discovery_type_is_valid(struct hci_dev *hdev, uint8_t type,
return true;
}
+static void start_discovery_complete(struct hci_dev *hdev, void *data, int err)
+{
+ struct mgmt_pending_cmd *cmd = data;
+
+ bt_dev_dbg(hdev, "err %d", err);
+
+ mgmt_cmd_complete(cmd->sk, cmd->index, cmd->opcode, mgmt_status(err),
+ cmd->param, 1);
+ mgmt_pending_free(cmd);
+
+ hci_discovery_set_state(hdev, err ? DISCOVERY_STOPPED:
+ DISCOVERY_FINDING);
+}
+
+static int start_discovery_sync(struct hci_dev *hdev, void *data)
+{
+ return hci_start_discovery_sync(hdev);
+}
+
static int start_discovery_internal(struct sock *sk, struct hci_dev *hdev,
u16 op, void *data, u16 len)
{
@@ -5164,17 +5327,20 @@ static int start_discovery_internal(struct sock *sk, struct hci_dev *hdev,
else
hdev->discovery.limited = false;
- cmd = mgmt_pending_add(sk, op, hdev, data, len);
+ cmd = mgmt_pending_new(sk, op, hdev, data, len);
if (!cmd) {
err = -ENOMEM;
goto failed;
}
- cmd->cmd_complete = generic_cmd_complete;
+ err = hci_cmd_sync_queue(hdev, start_discovery_sync, cmd,
+ start_discovery_complete);
+ if (err < 0) {
+ mgmt_pending_free(cmd);
+ goto failed;
+ }
hci_discovery_set_state(hdev, DISCOVERY_STARTING);
- queue_work(hdev->req_workqueue, &hdev->discov_update);
- err = 0;
failed:
hci_dev_unlock(hdev);
@@ -5196,13 +5362,6 @@ static int start_limited_discovery(struct sock *sk, struct hci_dev *hdev,
data, len);
}
-static int service_discovery_cmd_complete(struct mgmt_pending_cmd *cmd,
- u8 status)
-{
- return mgmt_cmd_complete(cmd->sk, cmd->index, cmd->opcode, status,
- cmd->param, 1);
-}
-
static int start_service_discovery(struct sock *sk, struct hci_dev *hdev,
void *data, u16 len)
{
@@ -5271,15 +5430,13 @@ static int start_service_discovery(struct sock *sk, struct hci_dev *hdev,
goto failed;
}
- cmd = mgmt_pending_add(sk, MGMT_OP_START_SERVICE_DISCOVERY,
+ cmd = mgmt_pending_new(sk, MGMT_OP_START_SERVICE_DISCOVERY,
hdev, data, len);
if (!cmd) {
err = -ENOMEM;
goto failed;
}
- cmd->cmd_complete = service_discovery_cmd_complete;
-
/* Clear the discovery filter first to free any previously
* allocated memory for the UUID list.
*/
@@ -5303,9 +5460,14 @@ static int start_service_discovery(struct sock *sk, struct hci_dev *hdev,
}
}
+ err = hci_cmd_sync_queue(hdev, start_discovery_sync, cmd,
+ start_discovery_complete);
+ if (err < 0) {
+ mgmt_pending_free(cmd);
+ goto failed;
+ }
+
hci_discovery_set_state(hdev, DISCOVERY_STARTING);
- queue_work(hdev->req_workqueue, &hdev->discov_update);
- err = 0;
failed:
hci_dev_unlock(hdev);
@@ -5327,12 +5489,25 @@ void mgmt_stop_discovery_complete(struct hci_dev *hdev, u8 status)
}
hci_dev_unlock(hdev);
+}
- /* Handle suspend notifier */
- if (test_and_clear_bit(SUSPEND_PAUSE_DISCOVERY, hdev->suspend_tasks)) {
- bt_dev_dbg(hdev, "Paused discovery");
- wake_up(&hdev->suspend_wait_q);
- }
+static void stop_discovery_complete(struct hci_dev *hdev, void *data, int err)
+{
+ struct mgmt_pending_cmd *cmd = data;
+
+ bt_dev_dbg(hdev, "err %d", err);
+
+ mgmt_cmd_complete(cmd->sk, cmd->index, cmd->opcode, mgmt_status(err),
+ cmd->param, 1);
+ mgmt_pending_free(cmd);
+
+ if (!err)
+ hci_discovery_set_state(hdev, DISCOVERY_STOPPED);
+}
+
+static int stop_discovery_sync(struct hci_dev *hdev, void *data)
+{
+ return hci_stop_discovery_sync(hdev);
}
static int stop_discovery(struct sock *sk, struct hci_dev *hdev, void *data,
@@ -5360,17 +5535,20 @@ static int stop_discovery(struct sock *sk, struct hci_dev *hdev, void *data,
goto unlock;
}
- cmd = mgmt_pending_add(sk, MGMT_OP_STOP_DISCOVERY, hdev, data, len);
+ cmd = mgmt_pending_new(sk, MGMT_OP_STOP_DISCOVERY, hdev, data, len);
if (!cmd) {
err = -ENOMEM;
goto unlock;
}
- cmd->cmd_complete = generic_cmd_complete;
+ err = hci_cmd_sync_queue(hdev, stop_discovery_sync, cmd,
+ stop_discovery_complete);
+ if (err < 0) {
+ mgmt_pending_free(cmd);
+ goto unlock;
+ }
hci_discovery_set_state(hdev, DISCOVERY_STOPPING);
- queue_work(hdev->req_workqueue, &hdev->discov_update);
- err = 0;
unlock:
hci_dev_unlock(hdev);
@@ -5491,11 +5669,15 @@ static int unblock_device(struct sock *sk, struct hci_dev *hdev, void *data,
return err;
}
+static int set_device_id_sync(struct hci_dev *hdev, void *data)
+{
+ return hci_update_eir_sync(hdev);
+}
+
static int set_device_id(struct sock *sk, struct hci_dev *hdev, void *data,
u16 len)
{
struct mgmt_cp_set_device_id *cp = data;
- struct hci_request req;
int err;
__u16 source;
@@ -5517,38 +5699,32 @@ static int set_device_id(struct sock *sk, struct hci_dev *hdev, void *data,
err = mgmt_cmd_complete(sk, hdev->id, MGMT_OP_SET_DEVICE_ID, 0,
NULL, 0);
- hci_req_init(&req, hdev);
- __hci_req_update_eir(&req);
- hci_req_run(&req, NULL);
+ hci_cmd_sync_queue(hdev, set_device_id_sync, NULL, NULL);
hci_dev_unlock(hdev);
return err;
}
-static void enable_advertising_instance(struct hci_dev *hdev, u8 status,
- u16 opcode)
+static void enable_advertising_instance(struct hci_dev *hdev, int err)
{
- bt_dev_dbg(hdev, "status %u", status);
+ if (err)
+ bt_dev_err(hdev, "failed to re-configure advertising %d", err);
+ else
+ bt_dev_dbg(hdev, "status %d", err);
}
-static void set_advertising_complete(struct hci_dev *hdev, u8 status,
- u16 opcode)
+static void set_advertising_complete(struct hci_dev *hdev, void *data, int err)
{
struct cmd_lookup match = { NULL, hdev };
- struct hci_request req;
u8 instance;
struct adv_info *adv_instance;
- int err;
-
- hci_dev_lock(hdev);
+ u8 status = mgmt_status(err);
if (status) {
- u8 mgmt_err = mgmt_status(status);
-
mgmt_pending_foreach(MGMT_OP_SET_ADVERTISING, hdev,
- cmd_status_rsp, &mgmt_err);
- goto unlock;
+ cmd_status_rsp, &status);
+ return;
}
if (hci_dev_test_flag(hdev, HCI_LE_ADV))
@@ -5564,46 +5740,60 @@ static void set_advertising_complete(struct hci_dev *hdev, u8 status,
if (match.sk)
sock_put(match.sk);
- /* Handle suspend notifier */
- if (test_and_clear_bit(SUSPEND_PAUSE_ADVERTISING,
- hdev->suspend_tasks)) {
- bt_dev_dbg(hdev, "Paused advertising");
- wake_up(&hdev->suspend_wait_q);
- } else if (test_and_clear_bit(SUSPEND_UNPAUSE_ADVERTISING,
- hdev->suspend_tasks)) {
- bt_dev_dbg(hdev, "Unpaused advertising");
- wake_up(&hdev->suspend_wait_q);
- }
-
/* If "Set Advertising" was just disabled and instance advertising was
* set up earlier, then re-enable multi-instance advertising.
*/
if (hci_dev_test_flag(hdev, HCI_ADVERTISING) ||
list_empty(&hdev->adv_instances))
- goto unlock;
+ return;
instance = hdev->cur_adv_instance;
if (!instance) {
adv_instance = list_first_entry_or_null(&hdev->adv_instances,
struct adv_info, list);
if (!adv_instance)
- goto unlock;
+ return;
instance = adv_instance->instance;
}
- hci_req_init(&req, hdev);
+ err = hci_schedule_adv_instance_sync(hdev, instance, true);
- err = __hci_req_schedule_adv_instance(&req, instance, true);
+ enable_advertising_instance(hdev, err);
+}
- if (!err)
- err = hci_req_run(&req, enable_advertising_instance);
+static int set_adv_sync(struct hci_dev *hdev, void *data)
+{
+ struct mgmt_pending_cmd *cmd = data;
+ struct mgmt_mode *cp = cmd->param;
+ u8 val = !!cp->val;
- if (err)
- bt_dev_err(hdev, "failed to re-configure advertising");
+ if (cp->val == 0x02)
+ hci_dev_set_flag(hdev, HCI_ADVERTISING_CONNECTABLE);
+ else
+ hci_dev_clear_flag(hdev, HCI_ADVERTISING_CONNECTABLE);
-unlock:
- hci_dev_unlock(hdev);
+ cancel_adv_timeout(hdev);
+
+ if (val) {
+ /* Switch to instance "0" for the Set Advertising setting.
+ * We cannot use update_[adv|scan_rsp]_data() here as the
+ * HCI_ADVERTISING flag is not yet set.
+ */
+ hdev->cur_adv_instance = 0x00;
+
+ if (ext_adv_capable(hdev)) {
+ hci_start_ext_adv_sync(hdev, 0x00);
+ } else {
+ hci_update_adv_data_sync(hdev, 0x00);
+ hci_update_scan_rsp_data_sync(hdev, 0x00);
+ hci_enable_advertising_sync(hdev);
+ }
+ } else {
+ hci_disable_advertising_sync(hdev);
+ }
+
+ return 0;
}
static int set_advertising(struct sock *sk, struct hci_dev *hdev, void *data,
@@ -5611,7 +5801,6 @@ static int set_advertising(struct sock *sk, struct hci_dev *hdev, void *data,
{
struct mgmt_mode *cp = data;
struct mgmt_pending_cmd *cmd;
- struct hci_request req;
u8 val, status;
int err;
@@ -5622,13 +5811,6 @@ static int set_advertising(struct sock *sk, struct hci_dev *hdev, void *data,
return mgmt_cmd_status(sk, hdev->id, MGMT_OP_SET_ADVERTISING,
status);
- /* Enabling the experimental LL Privay support disables support for
- * advertising.
- */
- if (hci_dev_test_flag(hdev, HCI_ENABLE_LL_PRIVACY))
- return mgmt_cmd_status(sk, hdev->id, MGMT_OP_SET_ADVERTISING,
- MGMT_STATUS_NOT_SUPPORTED);
-
if (cp->val != 0x00 && cp->val != 0x01 && cp->val != 0x02)
return mgmt_cmd_status(sk, hdev->id, MGMT_OP_SET_ADVERTISING,
MGMT_STATUS_INVALID_PARAMS);
@@ -5684,40 +5866,13 @@ static int set_advertising(struct sock *sk, struct hci_dev *hdev, void *data,
}
cmd = mgmt_pending_add(sk, MGMT_OP_SET_ADVERTISING, hdev, data, len);
- if (!cmd) {
+ if (!cmd)
err = -ENOMEM;
- goto unlock;
- }
-
- hci_req_init(&req, hdev);
-
- if (cp->val == 0x02)
- hci_dev_set_flag(hdev, HCI_ADVERTISING_CONNECTABLE);
else
- hci_dev_clear_flag(hdev, HCI_ADVERTISING_CONNECTABLE);
+ err = hci_cmd_sync_queue(hdev, set_adv_sync, cmd,
+ set_advertising_complete);
- cancel_adv_timeout(hdev);
-
- if (val) {
- /* Switch to instance "0" for the Set Advertising setting.
- * We cannot use update_[adv|scan_rsp]_data() here as the
- * HCI_ADVERTISING flag is not yet set.
- */
- hdev->cur_adv_instance = 0x00;
-
- if (ext_adv_capable(hdev)) {
- __hci_req_start_ext_adv(&req, 0x00);
- } else {
- __hci_req_update_adv_data(&req, 0x00);
- __hci_req_update_scan_rsp_data(&req, 0x00);
- __hci_req_enable_advertising(&req);
- }
- } else {
- __hci_req_disable_advertising(&req);
- }
-
- err = hci_req_run(&req, set_advertising_complete);
- if (err < 0)
+ if (err < 0 && cmd)
mgmt_pending_remove(cmd);
unlock:
@@ -5810,38 +5965,23 @@ static int set_scan_params(struct sock *sk, struct hci_dev *hdev,
* loaded.
*/
if (hci_dev_test_flag(hdev, HCI_LE_SCAN) &&
- hdev->discovery.state == DISCOVERY_STOPPED) {
- struct hci_request req;
-
- hci_req_init(&req, hdev);
-
- hci_req_add_le_scan_disable(&req, false);
- hci_req_add_le_passive_scan(&req);
-
- hci_req_run(&req, NULL);
- }
+ hdev->discovery.state == DISCOVERY_STOPPED)
+ hci_update_passive_scan(hdev);
hci_dev_unlock(hdev);
return err;
}
-static void fast_connectable_complete(struct hci_dev *hdev, u8 status,
- u16 opcode)
+static void fast_connectable_complete(struct hci_dev *hdev, void *data, int err)
{
- struct mgmt_pending_cmd *cmd;
+ struct mgmt_pending_cmd *cmd = data;
- bt_dev_dbg(hdev, "status 0x%02x", status);
+ bt_dev_dbg(hdev, "err %d", err);
- hci_dev_lock(hdev);
-
- cmd = pending_find(MGMT_OP_SET_FAST_CONNECTABLE, hdev);
- if (!cmd)
- goto unlock;
-
- if (status) {
+ if (err) {
mgmt_cmd_status(cmd->sk, hdev->id, MGMT_OP_SET_FAST_CONNECTABLE,
- mgmt_status(status));
+ mgmt_status(err));
} else {
struct mgmt_mode *cp = cmd->param;
@@ -5854,10 +5994,15 @@ static void fast_connectable_complete(struct hci_dev *hdev, u8 status,
new_settings(hdev, cmd->sk);
}
- mgmt_pending_remove(cmd);
+ mgmt_pending_free(cmd);
+}
-unlock:
- hci_dev_unlock(hdev);
+static int write_fast_connectable_sync(struct hci_dev *hdev, void *data)
+{
+ struct mgmt_pending_cmd *cmd = data;
+ struct mgmt_mode *cp = cmd->param;
+
+ return hci_write_fast_connectable_sync(hdev, cp->val);
}
static int set_fast_connectable(struct sock *sk, struct hci_dev *hdev,
@@ -5865,58 +6010,49 @@ static int set_fast_connectable(struct sock *sk, struct hci_dev *hdev,
{
struct mgmt_mode *cp = data;
struct mgmt_pending_cmd *cmd;
- struct hci_request req;
int err;
bt_dev_dbg(hdev, "sock %p", sk);
if (!hci_dev_test_flag(hdev, HCI_BREDR_ENABLED) ||
hdev->hci_ver < BLUETOOTH_VER_1_2)
- return mgmt_cmd_status(sk, hdev->id, MGMT_OP_SET_FAST_CONNECTABLE,
+ return mgmt_cmd_status(sk, hdev->id,
+ MGMT_OP_SET_FAST_CONNECTABLE,
MGMT_STATUS_NOT_SUPPORTED);
if (cp->val != 0x00 && cp->val != 0x01)
- return mgmt_cmd_status(sk, hdev->id, MGMT_OP_SET_FAST_CONNECTABLE,
+ return mgmt_cmd_status(sk, hdev->id,
+ MGMT_OP_SET_FAST_CONNECTABLE,
MGMT_STATUS_INVALID_PARAMS);
hci_dev_lock(hdev);
- if (pending_find(MGMT_OP_SET_FAST_CONNECTABLE, hdev)) {
- err = mgmt_cmd_status(sk, hdev->id, MGMT_OP_SET_FAST_CONNECTABLE,
- MGMT_STATUS_BUSY);
- goto unlock;
- }
-
if (!!cp->val == hci_dev_test_flag(hdev, HCI_FAST_CONNECTABLE)) {
- err = send_settings_rsp(sk, MGMT_OP_SET_FAST_CONNECTABLE,
- hdev);
+ err = send_settings_rsp(sk, MGMT_OP_SET_FAST_CONNECTABLE, hdev);
goto unlock;
}
if (!hdev_is_powered(hdev)) {
hci_dev_change_flag(hdev, HCI_FAST_CONNECTABLE);
- err = send_settings_rsp(sk, MGMT_OP_SET_FAST_CONNECTABLE,
- hdev);
+ err = send_settings_rsp(sk, MGMT_OP_SET_FAST_CONNECTABLE, hdev);
new_settings(hdev, sk);
goto unlock;
}
- cmd = mgmt_pending_add(sk, MGMT_OP_SET_FAST_CONNECTABLE, hdev,
- data, len);
- if (!cmd) {
+ cmd = mgmt_pending_new(sk, MGMT_OP_SET_FAST_CONNECTABLE, hdev, data,
+ len);
+ if (!cmd)
err = -ENOMEM;
- goto unlock;
- }
+ else
+ err = hci_cmd_sync_queue(hdev, write_fast_connectable_sync, cmd,
+ fast_connectable_complete);
- hci_req_init(&req, hdev);
-
- __hci_req_write_fast_connectable(&req, cp->val);
-
- err = hci_req_run(&req, fast_connectable_complete);
if (err < 0) {
- err = mgmt_cmd_status(sk, hdev->id, MGMT_OP_SET_FAST_CONNECTABLE,
- MGMT_STATUS_FAILED);
- mgmt_pending_remove(cmd);
+ mgmt_cmd_status(sk, hdev->id, MGMT_OP_SET_FAST_CONNECTABLE,
+ MGMT_STATUS_FAILED);
+
+ if (cmd)
+ mgmt_pending_free(cmd);
}
unlock:
@@ -5925,20 +6061,14 @@ static int set_fast_connectable(struct sock *sk, struct hci_dev *hdev,
return err;
}
-static void set_bredr_complete(struct hci_dev *hdev, u8 status, u16 opcode)
+static void set_bredr_complete(struct hci_dev *hdev, void *data, int err)
{
- struct mgmt_pending_cmd *cmd;
+ struct mgmt_pending_cmd *cmd = data;
- bt_dev_dbg(hdev, "status 0x%02x", status);
+ bt_dev_dbg(hdev, "err %d", err);
- hci_dev_lock(hdev);
-
- cmd = pending_find(MGMT_OP_SET_BREDR, hdev);
- if (!cmd)
- goto unlock;
-
- if (status) {
- u8 mgmt_err = mgmt_status(status);
+ if (err) {
+ u8 mgmt_err = mgmt_status(err);
/* We need to restore the flag if related HCI commands
* failed.
@@ -5951,17 +6081,31 @@ static void set_bredr_complete(struct hci_dev *hdev, u8 status, u16 opcode)
new_settings(hdev, cmd->sk);
}
- mgmt_pending_remove(cmd);
+ mgmt_pending_free(cmd);
+}
-unlock:
- hci_dev_unlock(hdev);
+static int set_bredr_sync(struct hci_dev *hdev, void *data)
+{
+ int status;
+
+ status = hci_write_fast_connectable_sync(hdev, false);
+
+ if (!status)
+ status = hci_update_scan_sync(hdev);
+
+ /* Since only the advertising data flags will change, there
+ * is no need to update the scan response data.
+ */
+ if (!status)
+ status = hci_update_adv_data_sync(hdev, hdev->cur_adv_instance);
+
+ return status;
}
static int set_bredr(struct sock *sk, struct hci_dev *hdev, void *data, u16 len)
{
struct mgmt_mode *cp = data;
struct mgmt_pending_cmd *cmd;
- struct hci_request req;
int err;
bt_dev_dbg(hdev, "sock %p", sk);
@@ -6033,15 +6177,19 @@ static int set_bredr(struct sock *sk, struct hci_dev *hdev, void *data, u16 len)
}
}
- if (pending_find(MGMT_OP_SET_BREDR, hdev)) {
- err = mgmt_cmd_status(sk, hdev->id, MGMT_OP_SET_BREDR,
- MGMT_STATUS_BUSY);
- goto unlock;
- }
-
- cmd = mgmt_pending_add(sk, MGMT_OP_SET_BREDR, hdev, data, len);
- if (!cmd) {
+ cmd = mgmt_pending_new(sk, MGMT_OP_SET_BREDR, hdev, data, len);
+ if (!cmd)
err = -ENOMEM;
+ else
+ err = hci_cmd_sync_queue(hdev, set_bredr_sync, cmd,
+ set_bredr_complete);
+
+ if (err < 0) {
+ mgmt_cmd_status(sk, hdev->id, MGMT_OP_SET_BREDR,
+ MGMT_STATUS_FAILED);
+ if (cmd)
+ mgmt_pending_free(cmd);
+
goto unlock;
}
@@ -6050,42 +6198,23 @@ static int set_bredr(struct sock *sk, struct hci_dev *hdev, void *data, u16 len)
*/
hci_dev_set_flag(hdev, HCI_BREDR_ENABLED);
- hci_req_init(&req, hdev);
-
- __hci_req_write_fast_connectable(&req, false);
- __hci_req_update_scan(&req);
-
- /* Since only the advertising data flags will change, there
- * is no need to update the scan response data.
- */
- __hci_req_update_adv_data(&req, hdev->cur_adv_instance);
-
- err = hci_req_run(&req, set_bredr_complete);
- if (err < 0)
- mgmt_pending_remove(cmd);
-
unlock:
hci_dev_unlock(hdev);
return err;
}
-static void sc_enable_complete(struct hci_dev *hdev, u8 status, u16 opcode)
+static void set_secure_conn_complete(struct hci_dev *hdev, void *data, int err)
{
- struct mgmt_pending_cmd *cmd;
+ struct mgmt_pending_cmd *cmd = data;
struct mgmt_mode *cp;
- bt_dev_dbg(hdev, "status %u", status);
+ bt_dev_dbg(hdev, "err %d", err);
- hci_dev_lock(hdev);
+ if (err) {
+ u8 mgmt_err = mgmt_status(err);
- cmd = pending_find(MGMT_OP_SET_SECURE_CONN, hdev);
- if (!cmd)
- goto unlock;
-
- if (status) {
- mgmt_cmd_status(cmd->sk, cmd->index, cmd->opcode,
- mgmt_status(status));
- goto remove;
+ mgmt_cmd_status(cmd->sk, cmd->index, cmd->opcode, mgmt_err);
+ goto done;
}
cp = cmd->param;
@@ -6105,13 +6234,23 @@ static void sc_enable_complete(struct hci_dev *hdev, u8 status, u16 opcode)
break;
}
- send_settings_rsp(cmd->sk, MGMT_OP_SET_SECURE_CONN, hdev);
+ send_settings_rsp(cmd->sk, cmd->opcode, hdev);
new_settings(hdev, cmd->sk);
-remove:
- mgmt_pending_remove(cmd);
-unlock:
- hci_dev_unlock(hdev);
+done:
+ mgmt_pending_free(cmd);
+}
+
+static int set_secure_conn_sync(struct hci_dev *hdev, void *data)
+{
+ struct mgmt_pending_cmd *cmd = data;
+ struct mgmt_mode *cp = cmd->param;
+ u8 val = !!cp->val;
+
+ /* Force write of val */
+ hci_dev_set_flag(hdev, HCI_SC_ENABLED);
+
+ return hci_write_sc_support_sync(hdev, val);
}
static int set_secure_conn(struct sock *sk, struct hci_dev *hdev,
@@ -6119,7 +6258,6 @@ static int set_secure_conn(struct sock *sk, struct hci_dev *hdev,
{
struct mgmt_mode *cp = data;
struct mgmt_pending_cmd *cmd;
- struct hci_request req;
u8 val;
int err;
@@ -6138,7 +6276,7 @@ static int set_secure_conn(struct sock *sk, struct hci_dev *hdev,
if (cp->val != 0x00 && cp->val != 0x01 && cp->val != 0x02)
return mgmt_cmd_status(sk, hdev->id, MGMT_OP_SET_SECURE_CONN,
- MGMT_STATUS_INVALID_PARAMS);
+ MGMT_STATUS_INVALID_PARAMS);
hci_dev_lock(hdev);
@@ -6169,12 +6307,6 @@ static int set_secure_conn(struct sock *sk, struct hci_dev *hdev,
goto failed;
}
- if (pending_find(MGMT_OP_SET_SECURE_CONN, hdev)) {
- err = mgmt_cmd_status(sk, hdev->id, MGMT_OP_SET_SECURE_CONN,
- MGMT_STATUS_BUSY);
- goto failed;
- }
-
val = !!cp->val;
if (val == hci_dev_test_flag(hdev, HCI_SC_ENABLED) &&
@@ -6183,18 +6315,18 @@ static int set_secure_conn(struct sock *sk, struct hci_dev *hdev,
goto failed;
}
- cmd = mgmt_pending_add(sk, MGMT_OP_SET_SECURE_CONN, hdev, data, len);
- if (!cmd) {
+ cmd = mgmt_pending_new(sk, MGMT_OP_SET_SECURE_CONN, hdev, data, len);
+ if (!cmd)
err = -ENOMEM;
- goto failed;
- }
+ else
+ err = hci_cmd_sync_queue(hdev, set_secure_conn_sync, cmd,
+ set_secure_conn_complete);
- hci_req_init(&req, hdev);
- hci_req_add(&req, HCI_OP_WRITE_SC_SUPPORT, 1, &val);
- err = hci_req_run(&req, sc_enable_complete);
if (err < 0) {
- mgmt_pending_remove(cmd);
- goto failed;
+ mgmt_cmd_status(sk, hdev->id, MGMT_OP_SET_SECURE_CONN,
+ MGMT_STATUS_FAILED);
+ if (cmd)
+ mgmt_pending_free(cmd);
}
failed:
@@ -6508,14 +6640,19 @@ static int load_long_term_keys(struct sock *sk, struct hci_dev *hdev,
return err;
}
-static int conn_info_cmd_complete(struct mgmt_pending_cmd *cmd, u8 status)
+static void get_conn_info_complete(struct hci_dev *hdev, void *data, int err)
{
+ struct mgmt_pending_cmd *cmd = data;
struct hci_conn *conn = cmd->user_data;
+ struct mgmt_cp_get_conn_info *cp = cmd->param;
struct mgmt_rp_get_conn_info rp;
- int err;
+ u8 status;
- memcpy(&rp.addr, cmd->param, sizeof(rp.addr));
+ bt_dev_dbg(hdev, "err %d", err);
+ memcpy(&rp.addr, &cp->addr.bdaddr, sizeof(rp.addr));
+
+ status = mgmt_status(err);
if (status == MGMT_STATUS_SUCCESS) {
rp.rssi = conn->rssi;
rp.tx_power = conn->tx_power;
@@ -6526,67 +6663,58 @@ static int conn_info_cmd_complete(struct mgmt_pending_cmd *cmd, u8 status)
rp.max_tx_power = HCI_TX_POWER_INVALID;
}
- err = mgmt_cmd_complete(cmd->sk, cmd->index, MGMT_OP_GET_CONN_INFO,
- status, &rp, sizeof(rp));
+ mgmt_cmd_complete(cmd->sk, cmd->index, MGMT_OP_GET_CONN_INFO, status,
+ &rp, sizeof(rp));
- hci_conn_drop(conn);
- hci_conn_put(conn);
+ if (conn) {
+ hci_conn_drop(conn);
+ hci_conn_put(conn);
+ }
- return err;
+ mgmt_pending_free(cmd);
}
-static void conn_info_refresh_complete(struct hci_dev *hdev, u8 hci_status,
- u16 opcode)
+static int get_conn_info_sync(struct hci_dev *hdev, void *data)
{
- struct hci_cp_read_rssi *cp;
- struct mgmt_pending_cmd *cmd;
+ struct mgmt_pending_cmd *cmd = data;
+ struct mgmt_cp_get_conn_info *cp = cmd->param;
struct hci_conn *conn;
- u16 handle;
- u8 status;
+ int err;
+ __le16 handle;
- bt_dev_dbg(hdev, "status 0x%02x", hci_status);
+ /* Make sure we are still connected */
+ if (cp->addr.type == BDADDR_BREDR)
+ conn = hci_conn_hash_lookup_ba(hdev, ACL_LINK,
+ &cp->addr.bdaddr);
+ else
+ conn = hci_conn_hash_lookup_ba(hdev, LE_LINK, &cp->addr.bdaddr);
- hci_dev_lock(hdev);
+ if (!conn || conn != cmd->user_data || conn->state != BT_CONNECTED) {
+ if (cmd->user_data) {
+ hci_conn_drop(cmd->user_data);
+ hci_conn_put(cmd->user_data);
+ cmd->user_data = NULL;
+ }
+ return MGMT_STATUS_NOT_CONNECTED;
+ }
- /* Commands sent in request are either Read RSSI or Read Transmit Power
- * Level so we check which one was last sent to retrieve connection
- * handle. Both commands have handle as first parameter so it's safe to
- * cast data on the same command struct.
- *
- * First command sent is always Read RSSI and we fail only if it fails.
- * In other case we simply override error to indicate success as we
- * already remembered if TX power value is actually valid.
+ handle = cpu_to_le16(conn->handle);
+
+ /* Refresh RSSI each time */
+ err = hci_read_rssi_sync(hdev, handle);
+
+ /* For LE links TX power does not change thus we don't need to
+ * query for it once value is known.
*/
- cp = hci_sent_cmd_data(hdev, HCI_OP_READ_RSSI);
- if (!cp) {
- cp = hci_sent_cmd_data(hdev, HCI_OP_READ_TX_POWER);
- status = MGMT_STATUS_SUCCESS;
- } else {
- status = mgmt_status(hci_status);
- }
+ if (!err && (!bdaddr_type_is_le(cp->addr.type) ||
+ conn->tx_power == HCI_TX_POWER_INVALID))
+ err = hci_read_tx_power_sync(hdev, handle, 0x00);
- if (!cp) {
- bt_dev_err(hdev, "invalid sent_cmd in conn_info response");
- goto unlock;
- }
+ /* Max TX power needs to be read only once per connection */
+ if (!err && conn->max_tx_power == HCI_TX_POWER_INVALID)
+ err = hci_read_tx_power_sync(hdev, handle, 0x01);
- handle = __le16_to_cpu(cp->handle);
- conn = hci_conn_hash_lookup_handle(hdev, handle);
- if (!conn) {
- bt_dev_err(hdev, "unknown handle (%u) in conn_info response",
- handle);
- goto unlock;
- }
-
- cmd = pending_find_data(MGMT_OP_GET_CONN_INFO, hdev, conn);
- if (!cmd)
- goto unlock;
-
- cmd->cmd_complete(cmd, status);
- mgmt_pending_remove(cmd);
-
-unlock:
- hci_dev_unlock(hdev);
+ return err;
}
static int get_conn_info(struct sock *sk, struct hci_dev *hdev, void *data,
@@ -6631,12 +6759,6 @@ static int get_conn_info(struct sock *sk, struct hci_dev *hdev, void *data,
goto unlock;
}
- if (pending_find_data(MGMT_OP_GET_CONN_INFO, hdev, conn)) {
- err = mgmt_cmd_complete(sk, hdev->id, MGMT_OP_GET_CONN_INFO,
- MGMT_STATUS_BUSY, &rp, sizeof(rp));
- goto unlock;
- }
-
/* To avoid client trying to guess when to poll again for information we
* calculate conn info age as random value between min/max set in hdev.
*/
@@ -6650,49 +6772,28 @@ static int get_conn_info(struct sock *sk, struct hci_dev *hdev, void *data,
if (time_after(jiffies, conn->conn_info_timestamp +
msecs_to_jiffies(conn_info_age)) ||
!conn->conn_info_timestamp) {
- struct hci_request req;
- struct hci_cp_read_tx_power req_txp_cp;
- struct hci_cp_read_rssi req_rssi_cp;
struct mgmt_pending_cmd *cmd;
- hci_req_init(&req, hdev);
- req_rssi_cp.handle = cpu_to_le16(conn->handle);
- hci_req_add(&req, HCI_OP_READ_RSSI, sizeof(req_rssi_cp),
- &req_rssi_cp);
-
- /* For LE links TX power does not change thus we don't need to
- * query for it once value is known.
- */
- if (!bdaddr_type_is_le(cp->addr.type) ||
- conn->tx_power == HCI_TX_POWER_INVALID) {
- req_txp_cp.handle = cpu_to_le16(conn->handle);
- req_txp_cp.type = 0x00;
- hci_req_add(&req, HCI_OP_READ_TX_POWER,
- sizeof(req_txp_cp), &req_txp_cp);
- }
-
- /* Max TX power needs to be read only once per connection */
- if (conn->max_tx_power == HCI_TX_POWER_INVALID) {
- req_txp_cp.handle = cpu_to_le16(conn->handle);
- req_txp_cp.type = 0x01;
- hci_req_add(&req, HCI_OP_READ_TX_POWER,
- sizeof(req_txp_cp), &req_txp_cp);
- }
-
- err = hci_req_run(&req, conn_info_refresh_complete);
- if (err < 0)
- goto unlock;
-
- cmd = mgmt_pending_add(sk, MGMT_OP_GET_CONN_INFO, hdev,
- data, len);
- if (!cmd) {
+ cmd = mgmt_pending_new(sk, MGMT_OP_GET_CONN_INFO, hdev, data,
+ len);
+ if (!cmd)
err = -ENOMEM;
+ else
+ err = hci_cmd_sync_queue(hdev, get_conn_info_sync,
+ cmd, get_conn_info_complete);
+
+ if (err < 0) {
+ mgmt_cmd_complete(sk, hdev->id, MGMT_OP_GET_CONN_INFO,
+ MGMT_STATUS_FAILED, &rp, sizeof(rp));
+
+ if (cmd)
+ mgmt_pending_free(cmd);
+
goto unlock;
}
hci_conn_hold(conn);
cmd->user_data = hci_conn_get(conn);
- cmd->cmd_complete = conn_info_cmd_complete;
conn->conn_info_timestamp = jiffies;
} else {
@@ -6710,82 +6811,76 @@ static int get_conn_info(struct sock *sk, struct hci_dev *hdev, void *data,
return err;
}
-static int clock_info_cmd_complete(struct mgmt_pending_cmd *cmd, u8 status)
+static void get_clock_info_complete(struct hci_dev *hdev, void *data, int err)
{
- struct hci_conn *conn = cmd->user_data;
+ struct mgmt_pending_cmd *cmd = data;
+ struct mgmt_cp_get_clock_info *cp = cmd->param;
struct mgmt_rp_get_clock_info rp;
- struct hci_dev *hdev;
- int err;
+ struct hci_conn *conn = cmd->user_data;
+ u8 status = mgmt_status(err);
+
+ bt_dev_dbg(hdev, "err %d", err);
memset(&rp, 0, sizeof(rp));
- memcpy(&rp.addr, cmd->param, sizeof(rp.addr));
+ bacpy(&rp.addr.bdaddr, &cp->addr.bdaddr);
+ rp.addr.type = cp->addr.type;
- if (status)
+ if (err)
goto complete;
- hdev = hci_dev_get(cmd->index);
- if (hdev) {
- rp.local_clock = cpu_to_le32(hdev->clock);
- hci_dev_put(hdev);
- }
+ rp.local_clock = cpu_to_le32(hdev->clock);
if (conn) {
rp.piconet_clock = cpu_to_le32(conn->clock);
rp.accuracy = cpu_to_le16(conn->clock_accuracy);
+ hci_conn_drop(conn);
+ hci_conn_put(conn);
}
complete:
- err = mgmt_cmd_complete(cmd->sk, cmd->index, cmd->opcode, status, &rp,
- sizeof(rp));
+ mgmt_cmd_complete(cmd->sk, cmd->index, cmd->opcode, status, &rp,
+ sizeof(rp));
+
+ mgmt_pending_free(cmd);
+}
+
+static int get_clock_info_sync(struct hci_dev *hdev, void *data)
+{
+ struct mgmt_pending_cmd *cmd = data;
+ struct mgmt_cp_get_clock_info *cp = cmd->param;
+ struct hci_cp_read_clock hci_cp;
+ struct hci_conn *conn = cmd->user_data;
+ int err;
+
+ memset(&hci_cp, 0, sizeof(hci_cp));
+ err = hci_read_clock_sync(hdev, &hci_cp);
if (conn) {
- hci_conn_drop(conn);
- hci_conn_put(conn);
+ /* Make sure connection still exists */
+ conn = hci_conn_hash_lookup_ba(hdev, ACL_LINK,
+ &cp->addr.bdaddr);
+
+ if (conn && conn == cmd->user_data &&
+ conn->state == BT_CONNECTED) {
+ hci_cp.handle = cpu_to_le16(conn->handle);
+ hci_cp.which = 0x01; /* Piconet clock */
+ err = hci_read_clock_sync(hdev, &hci_cp);
+ } else if (cmd->user_data) {
+ hci_conn_drop(cmd->user_data);
+ hci_conn_put(cmd->user_data);
+ cmd->user_data = NULL;
+ }
}
return err;
}
-static void get_clock_info_complete(struct hci_dev *hdev, u8 status, u16 opcode)
-{
- struct hci_cp_read_clock *hci_cp;
- struct mgmt_pending_cmd *cmd;
- struct hci_conn *conn;
-
- bt_dev_dbg(hdev, "status %u", status);
-
- hci_dev_lock(hdev);
-
- hci_cp = hci_sent_cmd_data(hdev, HCI_OP_READ_CLOCK);
- if (!hci_cp)
- goto unlock;
-
- if (hci_cp->which) {
- u16 handle = __le16_to_cpu(hci_cp->handle);
- conn = hci_conn_hash_lookup_handle(hdev, handle);
- } else {
- conn = NULL;
- }
-
- cmd = pending_find_data(MGMT_OP_GET_CLOCK_INFO, hdev, conn);
- if (!cmd)
- goto unlock;
-
- cmd->cmd_complete(cmd, mgmt_status(status));
- mgmt_pending_remove(cmd);
-
-unlock:
- hci_dev_unlock(hdev);
-}
-
static int get_clock_info(struct sock *sk, struct hci_dev *hdev, void *data,
- u16 len)
+ u16 len)
{
struct mgmt_cp_get_clock_info *cp = data;
struct mgmt_rp_get_clock_info rp;
- struct hci_cp_read_clock hci_cp;
struct mgmt_pending_cmd *cmd;
- struct hci_request req;
struct hci_conn *conn;
int err;
@@ -6823,31 +6918,25 @@ static int get_clock_info(struct sock *sk, struct hci_dev *hdev, void *data,
conn = NULL;
}
- cmd = mgmt_pending_add(sk, MGMT_OP_GET_CLOCK_INFO, hdev, data, len);
- if (!cmd) {
+ cmd = mgmt_pending_new(sk, MGMT_OP_GET_CLOCK_INFO, hdev, data, len);
+ if (!cmd)
err = -ENOMEM;
- goto unlock;
- }
+ else
+ err = hci_cmd_sync_queue(hdev, get_clock_info_sync, cmd,
+ get_clock_info_complete);
- cmd->cmd_complete = clock_info_cmd_complete;
+ if (err < 0) {
+ err = mgmt_cmd_complete(sk, hdev->id, MGMT_OP_GET_CLOCK_INFO,
+ MGMT_STATUS_FAILED, &rp, sizeof(rp));
- hci_req_init(&req, hdev);
+ if (cmd)
+ mgmt_pending_free(cmd);
- memset(&hci_cp, 0, sizeof(hci_cp));
- hci_req_add(&req, HCI_OP_READ_CLOCK, sizeof(hci_cp), &hci_cp);
-
- if (conn) {
+ } else if (conn) {
hci_conn_hold(conn);
cmd->user_data = hci_conn_get(conn);
-
- hci_cp.handle = cpu_to_le16(conn->handle);
- hci_cp.which = 0x01; /* Piconet clock */
- hci_req_add(&req, HCI_OP_READ_CLOCK, sizeof(hci_cp), &hci_cp);
}
- err = hci_req_run(&req, get_clock_info_complete);
- if (err < 0)
- mgmt_pending_remove(cmd);
unlock:
hci_dev_unlock(hdev);
@@ -6928,6 +7017,11 @@ static void device_added(struct sock *sk, struct hci_dev *hdev,
mgmt_event(MGMT_EV_DEVICE_ADDED, hdev, &ev, sizeof(ev), sk);
}
+static int add_device_sync(struct hci_dev *hdev, void *data)
+{
+ return hci_update_passive_scan_sync(hdev);
+}
+
static int add_device(struct sock *sk, struct hci_dev *hdev,
void *data, u16 len)
{
@@ -6936,6 +7030,7 @@ static int add_device(struct sock *sk, struct hci_dev *hdev,
struct hci_conn_params *params;
int err;
u32 current_flags = 0;
+ u32 supported_flags;
bt_dev_dbg(hdev, "sock %p", sk);
@@ -7007,15 +7102,20 @@ static int add_device(struct sock *sk, struct hci_dev *hdev,
params = hci_conn_params_lookup(hdev, &cp->addr.bdaddr,
addr_type);
if (params)
- current_flags = params->current_flags;
+ bitmap_to_arr32(¤t_flags, params->flags,
+ __HCI_CONN_NUM_FLAGS);
}
- hci_update_background_scan(hdev);
+ err = hci_cmd_sync_queue(hdev, add_device_sync, NULL, NULL);
+ if (err < 0)
+ goto unlock;
added:
device_added(sk, hdev, &cp->addr.bdaddr, cp->addr.type, cp->action);
+ bitmap_to_arr32(&supported_flags, hdev->conn_flags,
+ __HCI_CONN_NUM_FLAGS);
device_flags_changed(NULL, hdev, &cp->addr.bdaddr, cp->addr.type,
- SUPPORTED_DEVICE_FLAGS(), current_flags);
+ supported_flags, current_flags);
err = mgmt_cmd_complete(sk, hdev->id, MGMT_OP_ADD_DEVICE,
MGMT_STATUS_SUCCESS, &cp->addr,
@@ -7037,6 +7137,11 @@ static void device_removed(struct sock *sk, struct hci_dev *hdev,
mgmt_event(MGMT_EV_DEVICE_REMOVED, hdev, &ev, sizeof(ev), sk);
}
+static int remove_device_sync(struct hci_dev *hdev, void *data)
+{
+ return hci_update_passive_scan_sync(hdev);
+}
+
static int remove_device(struct sock *sk, struct hci_dev *hdev,
void *data, u16 len)
{
@@ -7116,7 +7221,6 @@ static int remove_device(struct sock *sk, struct hci_dev *hdev,
list_del(¶ms->action);
list_del(¶ms->list);
kfree(params);
- hci_update_background_scan(hdev);
device_removed(sk, hdev, &cp->addr.bdaddr, cp->addr.type);
} else {
@@ -7153,10 +7257,10 @@ static int remove_device(struct sock *sk, struct hci_dev *hdev,
}
bt_dev_dbg(hdev, "All LE connection parameters were removed");
-
- hci_update_background_scan(hdev);
}
+ hci_cmd_sync_queue(hdev, remove_device_sync, NULL, NULL);
+
complete:
err = mgmt_cmd_complete(sk, hdev->id, MGMT_OP_REMOVE_DEVICE,
MGMT_STATUS_SUCCESS, &cp->addr,
@@ -7359,22 +7463,28 @@ static int set_public_address(struct sock *sk, struct hci_dev *hdev,
return err;
}
-static void read_local_oob_ext_data_complete(struct hci_dev *hdev, u8 status,
- u16 opcode, struct sk_buff *skb)
+static void read_local_oob_ext_data_complete(struct hci_dev *hdev, void *data,
+ int err)
{
const struct mgmt_cp_read_local_oob_ext_data *mgmt_cp;
struct mgmt_rp_read_local_oob_ext_data *mgmt_rp;
u8 *h192, *r192, *h256, *r256;
- struct mgmt_pending_cmd *cmd;
+ struct mgmt_pending_cmd *cmd = data;
+ struct sk_buff *skb = cmd->skb;
+ u8 status = mgmt_status(err);
u16 eir_len;
- int err;
+
+ if (!status) {
+ if (!skb)
+ status = MGMT_STATUS_FAILED;
+ else if (IS_ERR(skb))
+ status = mgmt_status(PTR_ERR(skb));
+ else
+ status = mgmt_status(skb->data[0]);
+ }
bt_dev_dbg(hdev, "status %u", status);
- cmd = pending_find(MGMT_OP_READ_LOCAL_OOB_EXT_DATA, hdev);
- if (!cmd)
- return;
-
mgmt_cp = cmd->param;
if (status) {
@@ -7385,7 +7495,7 @@ static void read_local_oob_ext_data_complete(struct hci_dev *hdev, u8 status,
r192 = NULL;
h256 = NULL;
r256 = NULL;
- } else if (opcode == HCI_OP_READ_LOCAL_OOB_DATA) {
+ } else if (!bredr_sc_enabled(hdev)) {
struct hci_rp_read_local_oob_data *rp;
if (skb->len != sizeof(*rp)) {
@@ -7466,6 +7576,9 @@ static void read_local_oob_ext_data_complete(struct hci_dev *hdev, u8 status,
mgmt_rp, sizeof(*mgmt_rp) + eir_len,
HCI_MGMT_OOB_DATA_EVENTS, cmd->sk);
done:
+ if (skb && !IS_ERR(skb))
+ kfree_skb(skb);
+
kfree(mgmt_rp);
mgmt_pending_remove(cmd);
}
@@ -7474,7 +7587,6 @@ static int read_local_ssp_oob_req(struct hci_dev *hdev, struct sock *sk,
struct mgmt_cp_read_local_oob_ext_data *cp)
{
struct mgmt_pending_cmd *cmd;
- struct hci_request req;
int err;
cmd = mgmt_pending_add(sk, MGMT_OP_READ_LOCAL_OOB_EXT_DATA, hdev,
@@ -7482,14 +7594,9 @@ static int read_local_ssp_oob_req(struct hci_dev *hdev, struct sock *sk,
if (!cmd)
return -ENOMEM;
- hci_req_init(&req, hdev);
+ err = hci_cmd_sync_queue(hdev, read_local_oob_data_sync, cmd,
+ read_local_oob_ext_data_complete);
- if (bredr_sc_enabled(hdev))
- hci_req_add(&req, HCI_OP_READ_LOCAL_OOB_EXT_DATA, 0, NULL);
- else
- hci_req_add(&req, HCI_OP_READ_LOCAL_OOB_DATA, 0, NULL);
-
- err = hci_req_run_skb(&req, read_local_oob_ext_data_complete);
if (err < 0) {
mgmt_pending_remove(cmd);
return err;
@@ -7713,13 +7820,6 @@ static int read_adv_features(struct sock *sk, struct hci_dev *hdev,
return mgmt_cmd_status(sk, hdev->id, MGMT_OP_READ_ADV_FEATURES,
MGMT_STATUS_REJECTED);
- /* Enabling the experimental LL Privay support disables support for
- * advertising.
- */
- if (hci_dev_test_flag(hdev, HCI_ENABLE_LL_PRIVACY))
- return mgmt_cmd_status(sk, hdev->id, MGMT_OP_READ_ADV_FEATURES,
- MGMT_STATUS_NOT_SUPPORTED);
-
hci_dev_lock(hdev);
rp_len = sizeof(*rp) + hdev->adv_instance_cnt;
@@ -7876,58 +7976,66 @@ static bool adv_busy(struct hci_dev *hdev)
pending_find(MGMT_OP_ADD_EXT_ADV_DATA, hdev));
}
-static void add_advertising_complete(struct hci_dev *hdev, u8 status,
- u16 opcode)
+static void add_adv_complete(struct hci_dev *hdev, struct sock *sk, u8 instance,
+ int err)
{
- struct mgmt_pending_cmd *cmd;
- struct mgmt_cp_add_advertising *cp;
- struct mgmt_rp_add_advertising rp;
- struct adv_info *adv_instance, *n;
- u8 instance;
+ struct adv_info *adv, *n;
- bt_dev_dbg(hdev, "status %u", status);
+ bt_dev_dbg(hdev, "err %d", err);
hci_dev_lock(hdev);
- cmd = pending_find(MGMT_OP_ADD_ADVERTISING, hdev);
- if (!cmd)
- cmd = pending_find(MGMT_OP_ADD_EXT_ADV_DATA, hdev);
+ list_for_each_entry_safe(adv, n, &hdev->adv_instances, list) {
+ u8 instance;
- list_for_each_entry_safe(adv_instance, n, &hdev->adv_instances, list) {
- if (!adv_instance->pending)
+ if (!adv->pending)
continue;
- if (!status) {
- adv_instance->pending = false;
+ if (!err) {
+ adv->pending = false;
continue;
}
- instance = adv_instance->instance;
+ instance = adv->instance;
if (hdev->cur_adv_instance == instance)
cancel_adv_timeout(hdev);
hci_remove_adv_instance(hdev, instance);
- mgmt_advertising_removed(cmd ? cmd->sk : NULL, hdev, instance);
+ mgmt_advertising_removed(sk, hdev, instance);
}
- if (!cmd)
- goto unlock;
+ hci_dev_unlock(hdev);
+}
- cp = cmd->param;
+static void add_advertising_complete(struct hci_dev *hdev, void *data, int err)
+{
+ struct mgmt_pending_cmd *cmd = data;
+ struct mgmt_cp_add_advertising *cp = cmd->param;
+ struct mgmt_rp_add_advertising rp;
+
+ memset(&rp, 0, sizeof(rp));
+
rp.instance = cp->instance;
- if (status)
+ if (err)
mgmt_cmd_status(cmd->sk, cmd->index, cmd->opcode,
- mgmt_status(status));
+ mgmt_status(err));
else
mgmt_cmd_complete(cmd->sk, cmd->index, cmd->opcode,
- mgmt_status(status), &rp, sizeof(rp));
+ mgmt_status(err), &rp, sizeof(rp));
- mgmt_pending_remove(cmd);
+ add_adv_complete(hdev, cmd->sk, cp->instance, err);
-unlock:
- hci_dev_unlock(hdev);
+ mgmt_pending_free(cmd);
+}
+
+static int add_advertising_sync(struct hci_dev *hdev, void *data)
+{
+ struct mgmt_pending_cmd *cmd = data;
+ struct mgmt_cp_add_advertising *cp = cmd->param;
+
+ return hci_schedule_adv_instance_sync(hdev, cp->instance, true);
}
static int add_advertising(struct sock *sk, struct hci_dev *hdev,
@@ -7943,7 +8051,6 @@ static int add_advertising(struct sock *sk, struct hci_dev *hdev,
struct adv_info *next_instance;
int err;
struct mgmt_pending_cmd *cmd;
- struct hci_request req;
bt_dev_dbg(hdev, "sock %p", sk);
@@ -7952,13 +8059,6 @@ static int add_advertising(struct sock *sk, struct hci_dev *hdev,
return mgmt_cmd_status(sk, hdev->id, MGMT_OP_ADD_ADVERTISING,
status);
- /* Enabling the experimental LL Privay support disables support for
- * advertising.
- */
- if (hci_dev_test_flag(hdev, HCI_ENABLE_LL_PRIVACY))
- return mgmt_cmd_status(sk, hdev->id, MGMT_OP_ADD_ADVERTISING,
- MGMT_STATUS_NOT_SUPPORTED);
-
if (cp->instance < 1 || cp->instance > hdev->le_num_of_adv_sets)
return mgmt_cmd_status(sk, hdev->id, MGMT_OP_ADD_ADVERTISING,
MGMT_STATUS_INVALID_PARAMS);
@@ -8051,25 +8151,19 @@ static int add_advertising(struct sock *sk, struct hci_dev *hdev,
/* We're good to go, update advertising data, parameters, and start
* advertising.
*/
- cmd = mgmt_pending_add(sk, MGMT_OP_ADD_ADVERTISING, hdev, data,
+ cmd = mgmt_pending_new(sk, MGMT_OP_ADD_ADVERTISING, hdev, data,
data_len);
if (!cmd) {
err = -ENOMEM;
goto unlock;
}
- hci_req_init(&req, hdev);
+ cp->instance = schedule_instance;
- err = __hci_req_schedule_adv_instance(&req, schedule_instance, true);
-
- if (!err)
- err = hci_req_run(&req, add_advertising_complete);
-
- if (err < 0) {
- err = mgmt_cmd_status(sk, hdev->id, MGMT_OP_ADD_ADVERTISING,
- MGMT_STATUS_FAILED);
- mgmt_pending_remove(cmd);
- }
+ err = hci_cmd_sync_queue(hdev, add_advertising_sync, cmd,
+ add_advertising_complete);
+ if (err < 0)
+ mgmt_pending_free(cmd);
unlock:
hci_dev_unlock(hdev);
@@ -8077,30 +8171,25 @@ static int add_advertising(struct sock *sk, struct hci_dev *hdev,
return err;
}
-static void add_ext_adv_params_complete(struct hci_dev *hdev, u8 status,
- u16 opcode)
+static void add_ext_adv_params_complete(struct hci_dev *hdev, void *data,
+ int err)
{
- struct mgmt_pending_cmd *cmd;
- struct mgmt_cp_add_ext_adv_params *cp;
+ struct mgmt_pending_cmd *cmd = data;
+ struct mgmt_cp_add_ext_adv_params *cp = cmd->param;
struct mgmt_rp_add_ext_adv_params rp;
- struct adv_info *adv_instance;
+ struct adv_info *adv;
u32 flags;
BT_DBG("%s", hdev->name);
hci_dev_lock(hdev);
- cmd = pending_find(MGMT_OP_ADD_EXT_ADV_PARAMS, hdev);
- if (!cmd)
- goto unlock;
-
- cp = cmd->param;
- adv_instance = hci_find_adv_instance(hdev, cp->instance);
- if (!adv_instance)
+ adv = hci_find_adv_instance(hdev, cp->instance);
+ if (!adv)
goto unlock;
rp.instance = cp->instance;
- rp.tx_power = adv_instance->tx_power;
+ rp.tx_power = adv->tx_power;
/* While we're at it, inform userspace of the available space for this
* advertisement, given the flags that will be used.
@@ -8109,39 +8198,44 @@ static void add_ext_adv_params_complete(struct hci_dev *hdev, u8 status,
rp.max_adv_data_len = tlv_data_max_len(hdev, flags, true);
rp.max_scan_rsp_len = tlv_data_max_len(hdev, flags, false);
- if (status) {
+ if (err) {
/* If this advertisement was previously advertising and we
* failed to update it, we signal that it has been removed and
* delete its structure
*/
- if (!adv_instance->pending)
+ if (!adv->pending)
mgmt_advertising_removed(cmd->sk, hdev, cp->instance);
hci_remove_adv_instance(hdev, cp->instance);
mgmt_cmd_status(cmd->sk, cmd->index, cmd->opcode,
- mgmt_status(status));
-
+ mgmt_status(err));
} else {
mgmt_cmd_complete(cmd->sk, cmd->index, cmd->opcode,
- mgmt_status(status), &rp, sizeof(rp));
+ mgmt_status(err), &rp, sizeof(rp));
}
unlock:
if (cmd)
- mgmt_pending_remove(cmd);
+ mgmt_pending_free(cmd);
hci_dev_unlock(hdev);
}
+static int add_ext_adv_params_sync(struct hci_dev *hdev, void *data)
+{
+ struct mgmt_pending_cmd *cmd = data;
+ struct mgmt_cp_add_ext_adv_params *cp = cmd->param;
+
+ return hci_setup_ext_adv_instance_sync(hdev, cp->instance);
+}
+
static int add_ext_adv_params(struct sock *sk, struct hci_dev *hdev,
void *data, u16 data_len)
{
struct mgmt_cp_add_ext_adv_params *cp = data;
struct mgmt_rp_add_ext_adv_params rp;
struct mgmt_pending_cmd *cmd = NULL;
- struct adv_info *adv_instance;
- struct hci_request req;
u32 flags, min_interval, max_interval;
u16 timeout, duration;
u8 status;
@@ -8223,29 +8317,18 @@ static int add_ext_adv_params(struct sock *sk, struct hci_dev *hdev,
/* Submit request for advertising params if ext adv available */
if (ext_adv_capable(hdev)) {
- hci_req_init(&req, hdev);
- adv_instance = hci_find_adv_instance(hdev, cp->instance);
-
- /* Updating parameters of an active instance will return a
- * Command Disallowed error, so we must first disable the
- * instance if it is active.
- */
- if (!adv_instance->pending)
- __hci_req_disable_ext_adv_instance(&req, cp->instance);
-
- __hci_req_setup_ext_adv_instance(&req, cp->instance);
-
- err = hci_req_run(&req, add_ext_adv_params_complete);
-
- if (!err)
- cmd = mgmt_pending_add(sk, MGMT_OP_ADD_EXT_ADV_PARAMS,
- hdev, data, data_len);
+ cmd = mgmt_pending_new(sk, MGMT_OP_ADD_EXT_ADV_PARAMS, hdev,
+ data, data_len);
if (!cmd) {
err = -ENOMEM;
hci_remove_adv_instance(hdev, cp->instance);
goto unlock;
}
+ err = hci_cmd_sync_queue(hdev, add_ext_adv_params_sync, cmd,
+ add_ext_adv_params_complete);
+ if (err < 0)
+ mgmt_pending_free(cmd);
} else {
rp.instance = cp->instance;
rp.tx_power = HCI_ADV_TX_POWER_NO_PREFERENCE;
@@ -8262,6 +8345,49 @@ static int add_ext_adv_params(struct sock *sk, struct hci_dev *hdev,
return err;
}
+static void add_ext_adv_data_complete(struct hci_dev *hdev, void *data, int err)
+{
+ struct mgmt_pending_cmd *cmd = data;
+ struct mgmt_cp_add_ext_adv_data *cp = cmd->param;
+ struct mgmt_rp_add_advertising rp;
+
+ add_adv_complete(hdev, cmd->sk, cp->instance, err);
+
+ memset(&rp, 0, sizeof(rp));
+
+ rp.instance = cp->instance;
+
+ if (err)
+ mgmt_cmd_status(cmd->sk, cmd->index, cmd->opcode,
+ mgmt_status(err));
+ else
+ mgmt_cmd_complete(cmd->sk, cmd->index, cmd->opcode,
+ mgmt_status(err), &rp, sizeof(rp));
+
+ mgmt_pending_free(cmd);
+}
+
+static int add_ext_adv_data_sync(struct hci_dev *hdev, void *data)
+{
+ struct mgmt_pending_cmd *cmd = data;
+ struct mgmt_cp_add_ext_adv_data *cp = cmd->param;
+ int err;
+
+ if (ext_adv_capable(hdev)) {
+ err = hci_update_adv_data_sync(hdev, cp->instance);
+ if (err)
+ return err;
+
+ err = hci_update_scan_rsp_data_sync(hdev, cp->instance);
+ if (err)
+ return err;
+
+ return hci_enable_ext_advertising_sync(hdev, cp->instance);
+ }
+
+ return hci_schedule_adv_instance_sync(hdev, cp->instance, true);
+}
+
static int add_ext_adv_data(struct sock *sk, struct hci_dev *hdev, void *data,
u16 data_len)
{
@@ -8272,7 +8398,6 @@ static int add_ext_adv_data(struct sock *sk, struct hci_dev *hdev, void *data,
struct adv_info *adv_instance;
int err = 0;
struct mgmt_pending_cmd *cmd;
- struct hci_request req;
BT_DBG("%s", hdev->name);
@@ -8314,78 +8439,52 @@ static int add_ext_adv_data(struct sock *sk, struct hci_dev *hdev, void *data,
cp->data, cp->scan_rsp_len,
cp->data + cp->adv_data_len);
- /* We're good to go, update advertising data, parameters, and start
- * advertising.
- */
-
- hci_req_init(&req, hdev);
-
- hci_req_add(&req, HCI_OP_READ_LOCAL_NAME, 0, NULL);
-
- if (ext_adv_capable(hdev)) {
- __hci_req_update_adv_data(&req, cp->instance);
- __hci_req_update_scan_rsp_data(&req, cp->instance);
- __hci_req_enable_ext_advertising(&req, cp->instance);
-
- } else {
- /* If using software rotation, determine next instance to use */
-
- if (hdev->cur_adv_instance == cp->instance) {
- /* If the currently advertised instance is being changed
- * then cancel the current advertising and schedule the
- * next instance. If there is only one instance then the
- * overridden advertising data will be visible right
- * away
- */
- cancel_adv_timeout(hdev);
-
- next_instance = hci_get_next_instance(hdev,
- cp->instance);
- if (next_instance)
- schedule_instance = next_instance->instance;
- } else if (!hdev->adv_instance_timeout) {
- /* Immediately advertise the new instance if no other
- * instance is currently being advertised.
- */
- schedule_instance = cp->instance;
- }
-
- /* If the HCI_ADVERTISING flag is set or there is no instance to
- * be advertised then we have no HCI communication to make.
- * Simply return.
+ /* If using software rotation, determine next instance to use */
+ if (hdev->cur_adv_instance == cp->instance) {
+ /* If the currently advertised instance is being changed
+ * then cancel the current advertising and schedule the
+ * next instance. If there is only one instance then the
+ * overridden advertising data will be visible right
+ * away
*/
- if (hci_dev_test_flag(hdev, HCI_ADVERTISING) ||
- !schedule_instance) {
- if (adv_instance->pending) {
- mgmt_advertising_added(sk, hdev, cp->instance);
- adv_instance->pending = false;
- }
- rp.instance = cp->instance;
- err = mgmt_cmd_complete(sk, hdev->id,
- MGMT_OP_ADD_EXT_ADV_DATA,
- MGMT_STATUS_SUCCESS, &rp,
- sizeof(rp));
- goto unlock;
- }
+ cancel_adv_timeout(hdev);
- err = __hci_req_schedule_adv_instance(&req, schedule_instance,
- true);
+ next_instance = hci_get_next_instance(hdev, cp->instance);
+ if (next_instance)
+ schedule_instance = next_instance->instance;
+ } else if (!hdev->adv_instance_timeout) {
+ /* Immediately advertise the new instance if no other
+ * instance is currently being advertised.
+ */
+ schedule_instance = cp->instance;
}
- cmd = mgmt_pending_add(sk, MGMT_OP_ADD_EXT_ADV_DATA, hdev, data,
+ /* If the HCI_ADVERTISING flag is set or there is no instance to
+ * be advertised then we have no HCI communication to make.
+ * Simply return.
+ */
+ if (hci_dev_test_flag(hdev, HCI_ADVERTISING) || !schedule_instance) {
+ if (adv_instance->pending) {
+ mgmt_advertising_added(sk, hdev, cp->instance);
+ adv_instance->pending = false;
+ }
+ rp.instance = cp->instance;
+ err = mgmt_cmd_complete(sk, hdev->id, MGMT_OP_ADD_EXT_ADV_DATA,
+ MGMT_STATUS_SUCCESS, &rp, sizeof(rp));
+ goto unlock;
+ }
+
+ cmd = mgmt_pending_new(sk, MGMT_OP_ADD_EXT_ADV_DATA, hdev, data,
data_len);
if (!cmd) {
err = -ENOMEM;
goto clear_new_instance;
}
- if (!err)
- err = hci_req_run(&req, add_advertising_complete);
-
+ err = hci_cmd_sync_queue(hdev, add_ext_adv_data_sync, cmd,
+ add_ext_adv_data_complete);
if (err < 0) {
- err = mgmt_cmd_status(sk, hdev->id, MGMT_OP_ADD_EXT_ADV_DATA,
- MGMT_STATUS_FAILED);
- mgmt_pending_remove(cmd);
+ mgmt_pending_free(cmd);
goto clear_new_instance;
}
@@ -8408,54 +8507,53 @@ static int add_ext_adv_data(struct sock *sk, struct hci_dev *hdev, void *data,
return err;
}
-static void remove_advertising_complete(struct hci_dev *hdev, u8 status,
- u16 opcode)
+static void remove_advertising_complete(struct hci_dev *hdev, void *data,
+ int err)
{
- struct mgmt_pending_cmd *cmd;
- struct mgmt_cp_remove_advertising *cp;
+ struct mgmt_pending_cmd *cmd = data;
+ struct mgmt_cp_remove_advertising *cp = cmd->param;
struct mgmt_rp_remove_advertising rp;
- bt_dev_dbg(hdev, "status %u", status);
+ bt_dev_dbg(hdev, "err %d", err);
- hci_dev_lock(hdev);
-
- /* A failure status here only means that we failed to disable
- * advertising. Otherwise, the advertising instance has been removed,
- * so report success.
- */
- cmd = pending_find(MGMT_OP_REMOVE_ADVERTISING, hdev);
- if (!cmd)
- goto unlock;
-
- cp = cmd->param;
+ memset(&rp, 0, sizeof(rp));
rp.instance = cp->instance;
- mgmt_cmd_complete(cmd->sk, cmd->index, cmd->opcode, MGMT_STATUS_SUCCESS,
- &rp, sizeof(rp));
- mgmt_pending_remove(cmd);
+ if (err)
+ mgmt_cmd_status(cmd->sk, cmd->index, cmd->opcode,
+ mgmt_status(err));
+ else
+ mgmt_cmd_complete(cmd->sk, cmd->index, cmd->opcode,
+ MGMT_STATUS_SUCCESS, &rp, sizeof(rp));
-unlock:
- hci_dev_unlock(hdev);
+ mgmt_pending_free(cmd);
+}
+
+static int remove_advertising_sync(struct hci_dev *hdev, void *data)
+{
+ struct mgmt_pending_cmd *cmd = data;
+ struct mgmt_cp_remove_advertising *cp = cmd->param;
+ int err;
+
+ err = hci_remove_advertising_sync(hdev, cmd->sk, cp->instance, true);
+ if (err)
+ return err;
+
+ if (list_empty(&hdev->adv_instances))
+ err = hci_disable_advertising_sync(hdev);
+
+ return err;
}
static int remove_advertising(struct sock *sk, struct hci_dev *hdev,
void *data, u16 data_len)
{
struct mgmt_cp_remove_advertising *cp = data;
- struct mgmt_rp_remove_advertising rp;
struct mgmt_pending_cmd *cmd;
- struct hci_request req;
int err;
bt_dev_dbg(hdev, "sock %p", sk);
- /* Enabling the experimental LL Privay support disables support for
- * advertising.
- */
- if (hci_dev_test_flag(hdev, HCI_ENABLE_LL_PRIVACY))
- return mgmt_cmd_status(sk, hdev->id, MGMT_OP_REMOVE_ADVERTISING,
- MGMT_STATUS_NOT_SUPPORTED);
-
hci_dev_lock(hdev);
if (cp->instance && !hci_find_adv_instance(hdev, cp->instance)) {
@@ -8479,44 +8577,17 @@ static int remove_advertising(struct sock *sk, struct hci_dev *hdev,
goto unlock;
}
- hci_req_init(&req, hdev);
-
- /* If we use extended advertising, instance is disabled and removed */
- if (ext_adv_capable(hdev)) {
- __hci_req_disable_ext_adv_instance(&req, cp->instance);
- __hci_req_remove_ext_adv_instance(&req, cp->instance);
- }
-
- hci_req_clear_adv_instance(hdev, sk, &req, cp->instance, true);
-
- if (list_empty(&hdev->adv_instances))
- __hci_req_disable_advertising(&req);
-
- /* If no HCI commands have been collected so far or the HCI_ADVERTISING
- * flag is set or the device isn't powered then we have no HCI
- * communication to make. Simply return.
- */
- if (skb_queue_empty(&req.cmd_q) ||
- !hdev_is_powered(hdev) ||
- hci_dev_test_flag(hdev, HCI_ADVERTISING)) {
- hci_req_purge(&req);
- rp.instance = cp->instance;
- err = mgmt_cmd_complete(sk, hdev->id,
- MGMT_OP_REMOVE_ADVERTISING,
- MGMT_STATUS_SUCCESS, &rp, sizeof(rp));
- goto unlock;
- }
-
- cmd = mgmt_pending_add(sk, MGMT_OP_REMOVE_ADVERTISING, hdev, data,
+ cmd = mgmt_pending_new(sk, MGMT_OP_REMOVE_ADVERTISING, hdev, data,
data_len);
if (!cmd) {
err = -ENOMEM;
goto unlock;
}
- err = hci_req_run(&req, remove_advertising_complete);
+ err = hci_cmd_sync_queue(hdev, remove_advertising_sync, cmd,
+ remove_advertising_complete);
if (err < 0)
- mgmt_pending_remove(cmd);
+ mgmt_pending_free(cmd);
unlock:
hci_dev_unlock(hdev);
@@ -8758,31 +8829,6 @@ void mgmt_index_removed(struct hci_dev *hdev)
HCI_MGMT_EXT_INDEX_EVENTS);
}
-/* This function requires the caller holds hdev->lock */
-static void restart_le_actions(struct hci_dev *hdev)
-{
- struct hci_conn_params *p;
-
- list_for_each_entry(p, &hdev->le_conn_params, list) {
- /* Needed for AUTO_OFF case where might not "really"
- * have been powered off.
- */
- list_del_init(&p->action);
-
- switch (p->auto_connect) {
- case HCI_AUTO_CONN_DIRECT:
- case HCI_AUTO_CONN_ALWAYS:
- list_add(&p->action, &hdev->pend_le_conns);
- break;
- case HCI_AUTO_CONN_REPORT:
- list_add(&p->action, &hdev->pend_le_reports);
- break;
- default:
- break;
- }
- }
-}
-
void mgmt_power_on(struct hci_dev *hdev, int err)
{
struct cmd_lookup match = { NULL, hdev };
@@ -8793,7 +8839,7 @@ void mgmt_power_on(struct hci_dev *hdev, int err)
if (!err) {
restart_le_actions(hdev);
- hci_update_background_scan(hdev);
+ hci_update_passive_scan(hdev);
}
mgmt_pending_foreach(MGMT_OP_SET_POWERED, hdev, settings_rsp, &match);
@@ -9008,11 +9054,19 @@ void mgmt_new_conn_param(struct hci_dev *hdev, bdaddr_t *bdaddr,
void mgmt_device_connected(struct hci_dev *hdev, struct hci_conn *conn,
u8 *name, u8 name_len)
{
- char buf[512];
- struct mgmt_ev_device_connected *ev = (void *) buf;
+ struct sk_buff *skb;
+ struct mgmt_ev_device_connected *ev;
u16 eir_len = 0;
u32 flags = 0;
+ if (conn->le_adv_data_len > 0)
+ skb = mgmt_alloc_skb(hdev, MGMT_EV_DEVICE_CONNECTED,
+ conn->le_adv_data_len);
+ else
+ skb = mgmt_alloc_skb(hdev, MGMT_EV_DEVICE_CONNECTED,
+ 2 + name_len + 5);
+
+ ev = skb_put(skb, sizeof(*ev));
bacpy(&ev->addr.bdaddr, &conn->dst);
ev->addr.type = link_to_bdaddr(conn->type, conn->dst_type);
@@ -9026,24 +9080,26 @@ void mgmt_device_connected(struct hci_dev *hdev, struct hci_conn *conn,
* adding any BR/EDR data to the LE adv.
*/
if (conn->le_adv_data_len > 0) {
- memcpy(&ev->eir[eir_len],
- conn->le_adv_data, conn->le_adv_data_len);
+ skb_put_data(skb, conn->le_adv_data, conn->le_adv_data_len);
eir_len = conn->le_adv_data_len;
} else {
- if (name_len > 0)
+ if (name_len > 0) {
eir_len = eir_append_data(ev->eir, 0, EIR_NAME_COMPLETE,
name, name_len);
+ skb_put(skb, eir_len);
+ }
- if (memcmp(conn->dev_class, "\0\0\0", 3) != 0)
+ if (memcmp(conn->dev_class, "\0\0\0", 3) != 0) {
eir_len = eir_append_data(ev->eir, eir_len,
EIR_CLASS_OF_DEV,
conn->dev_class, 3);
+ skb_put(skb, 5);
+ }
}
ev->eir_len = cpu_to_le16(eir_len);
- mgmt_event(MGMT_EV_DEVICE_CONNECTED, hdev, buf,
- sizeof(*ev) + eir_len, NULL);
+ mgmt_event_skb(skb, NULL);
}
static void disconnect_rsp(struct mgmt_pending_cmd *cmd, void *data)
@@ -9349,74 +9405,6 @@ void mgmt_auth_enable_complete(struct hci_dev *hdev, u8 status)
sock_put(match.sk);
}
-static void clear_eir(struct hci_request *req)
-{
- struct hci_dev *hdev = req->hdev;
- struct hci_cp_write_eir cp;
-
- if (!lmp_ext_inq_capable(hdev))
- return;
-
- memset(hdev->eir, 0, sizeof(hdev->eir));
-
- memset(&cp, 0, sizeof(cp));
-
- hci_req_add(req, HCI_OP_WRITE_EIR, sizeof(cp), &cp);
-}
-
-void mgmt_ssp_enable_complete(struct hci_dev *hdev, u8 enable, u8 status)
-{
- struct cmd_lookup match = { NULL, hdev };
- struct hci_request req;
- bool changed = false;
-
- if (status) {
- u8 mgmt_err = mgmt_status(status);
-
- if (enable && hci_dev_test_and_clear_flag(hdev,
- HCI_SSP_ENABLED)) {
- hci_dev_clear_flag(hdev, HCI_HS_ENABLED);
- new_settings(hdev, NULL);
- }
-
- mgmt_pending_foreach(MGMT_OP_SET_SSP, hdev, cmd_status_rsp,
- &mgmt_err);
- return;
- }
-
- if (enable) {
- changed = !hci_dev_test_and_set_flag(hdev, HCI_SSP_ENABLED);
- } else {
- changed = hci_dev_test_and_clear_flag(hdev, HCI_SSP_ENABLED);
- if (!changed)
- changed = hci_dev_test_and_clear_flag(hdev,
- HCI_HS_ENABLED);
- else
- hci_dev_clear_flag(hdev, HCI_HS_ENABLED);
- }
-
- mgmt_pending_foreach(MGMT_OP_SET_SSP, hdev, settings_rsp, &match);
-
- if (changed)
- new_settings(hdev, match.sk);
-
- if (match.sk)
- sock_put(match.sk);
-
- hci_req_init(&req, hdev);
-
- if (hci_dev_test_flag(hdev, HCI_SSP_ENABLED)) {
- if (hci_dev_test_flag(hdev, HCI_USE_DEBUG_KEYS))
- hci_req_add(&req, HCI_OP_WRITE_SSP_DEBUG_MODE,
- sizeof(enable), &enable);
- __hci_req_update_eir(&req);
- } else {
- clear_eir(&req);
- }
-
- hci_req_run(&req, NULL);
-}
-
static void sk_lookup(struct mgmt_pending_cmd *cmd, void *data)
{
struct cmd_lookup *match = data;
@@ -9605,9 +9593,8 @@ void mgmt_device_found(struct hci_dev *hdev, bdaddr_t *bdaddr, u8 link_type,
u8 addr_type, u8 *dev_class, s8 rssi, u32 flags,
u8 *eir, u16 eir_len, u8 *scan_rsp, u8 scan_rsp_len)
{
- char buf[512];
- struct mgmt_ev_device_found *ev = (void *)buf;
- size_t ev_size;
+ struct sk_buff *skb;
+ struct mgmt_ev_device_found *ev;
/* Don't send events for a non-kernel initiated discovery. With
* LE one exception is if we have pend_le_reports > 0 in which
@@ -9642,13 +9629,13 @@ void mgmt_device_found(struct hci_dev *hdev, bdaddr_t *bdaddr, u8 link_type,
}
}
- /* Make sure that the buffer is big enough. The 5 extra bytes
- * are for the potential CoD field.
- */
- if (sizeof(*ev) + eir_len + scan_rsp_len + 5 > sizeof(buf))
+ /* Allocate skb. The 5 extra bytes are for the potential CoD field */
+ skb = mgmt_alloc_skb(hdev, MGMT_EV_DEVICE_FOUND,
+ sizeof(*ev) + eir_len + scan_rsp_len + 5);
+ if (!skb)
return;
- memset(buf, 0, sizeof(buf));
+ ev = skb_put(skb, sizeof(*ev));
/* In case of device discovery with BR/EDR devices (pre 1.2), the
* RSSI value was reported as 0 when not available. This behavior
@@ -9669,44 +9656,57 @@ void mgmt_device_found(struct hci_dev *hdev, bdaddr_t *bdaddr, u8 link_type,
if (eir_len > 0)
/* Copy EIR or advertising data into event */
- memcpy(ev->eir, eir, eir_len);
+ skb_put_data(skb, eir, eir_len);
- if (dev_class && !eir_get_data(ev->eir, eir_len, EIR_CLASS_OF_DEV,
- NULL))
- eir_len = eir_append_data(ev->eir, eir_len, EIR_CLASS_OF_DEV,
- dev_class, 3);
+ if (dev_class && !eir_get_data(eir, eir_len, EIR_CLASS_OF_DEV, NULL)) {
+ u8 eir_cod[5];
+
+ eir_len += eir_append_data(eir_cod, 0, EIR_CLASS_OF_DEV,
+ dev_class, 3);
+ skb_put_data(skb, eir_cod, sizeof(eir_cod));
+ }
if (scan_rsp_len > 0)
/* Append scan response data to event */
- memcpy(ev->eir + eir_len, scan_rsp, scan_rsp_len);
+ skb_put_data(skb, scan_rsp, scan_rsp_len);
ev->eir_len = cpu_to_le16(eir_len + scan_rsp_len);
- ev_size = sizeof(*ev) + eir_len + scan_rsp_len;
- mgmt_event(MGMT_EV_DEVICE_FOUND, hdev, ev, ev_size, NULL);
+ mgmt_event_skb(skb, NULL);
}
void mgmt_remote_name(struct hci_dev *hdev, bdaddr_t *bdaddr, u8 link_type,
u8 addr_type, s8 rssi, u8 *name, u8 name_len)
{
+ struct sk_buff *skb;
struct mgmt_ev_device_found *ev;
- char buf[sizeof(*ev) + HCI_MAX_NAME_LENGTH + 2];
u16 eir_len;
+ u32 flags;
- ev = (struct mgmt_ev_device_found *) buf;
+ if (name_len)
+ skb = mgmt_alloc_skb(hdev, MGMT_EV_DEVICE_FOUND, 2 + name_len);
+ else
+ skb = mgmt_alloc_skb(hdev, MGMT_EV_DEVICE_FOUND, 0);
- memset(buf, 0, sizeof(buf));
-
+ ev = skb_put(skb, sizeof(*ev));
bacpy(&ev->addr.bdaddr, bdaddr);
ev->addr.type = link_to_bdaddr(link_type, addr_type);
ev->rssi = rssi;
- eir_len = eir_append_data(ev->eir, 0, EIR_NAME_COMPLETE, name,
- name_len);
+ if (name) {
+ eir_len = eir_append_data(ev->eir, 0, EIR_NAME_COMPLETE, name,
+ name_len);
+ flags = 0;
+ skb_put(skb, eir_len);
+ } else {
+ eir_len = 0;
+ flags = MGMT_DEV_FOUND_NAME_REQUEST_FAILED;
+ }
ev->eir_len = cpu_to_le16(eir_len);
+ ev->flags = cpu_to_le32(flags);
- mgmt_event(MGMT_EV_DEVICE_FOUND, hdev, ev, sizeof(*ev) + eir_len, NULL);
+ mgmt_event_skb(skb, NULL);
}
void mgmt_discovering(struct hci_dev *hdev, u8 discovering)
diff --git a/net/bluetooth/mgmt_util.c b/net/bluetooth/mgmt_util.c
index 0d0a6d7..edee60b 100644
--- a/net/bluetooth/mgmt_util.c
+++ b/net/bluetooth/mgmt_util.c
@@ -56,40 +56,72 @@ static struct sk_buff *create_monitor_ctrl_event(__le16 index, u32 cookie,
return skb;
}
-int mgmt_send_event(u16 event, struct hci_dev *hdev, unsigned short channel,
- void *data, u16 data_len, int flag, struct sock *skip_sk)
+struct sk_buff *mgmt_alloc_skb(struct hci_dev *hdev, u16 opcode,
+ unsigned int size)
{
struct sk_buff *skb;
- struct mgmt_hdr *hdr;
- skb = alloc_skb(sizeof(*hdr) + data_len, GFP_KERNEL);
+ skb = alloc_skb(sizeof(struct mgmt_hdr) + size, GFP_KERNEL);
if (!skb)
- return -ENOMEM;
+ return skb;
- hdr = skb_put(skb, sizeof(*hdr));
- hdr->opcode = cpu_to_le16(event);
- if (hdev)
- hdr->index = cpu_to_le16(hdev->id);
- else
- hdr->index = cpu_to_le16(MGMT_INDEX_NONE);
- hdr->len = cpu_to_le16(data_len);
+ skb_reserve(skb, sizeof(struct mgmt_hdr));
+ bt_cb(skb)->mgmt.hdev = hdev;
+ bt_cb(skb)->mgmt.opcode = opcode;
- if (data)
- skb_put_data(skb, data, data_len);
+ return skb;
+}
+
+int mgmt_send_event_skb(unsigned short channel, struct sk_buff *skb, int flag,
+ struct sock *skip_sk)
+{
+ struct hci_dev *hdev;
+ struct mgmt_hdr *hdr;
+ int len = skb->len;
+
+ if (!skb)
+ return -EINVAL;
+
+ hdev = bt_cb(skb)->mgmt.hdev;
/* Time stamp */
__net_timestamp(skb);
- hci_send_to_channel(channel, skb, flag, skip_sk);
-
+ /* Send just the data, without headers, to the monitor */
if (channel == HCI_CHANNEL_CONTROL)
- hci_send_monitor_ctrl_event(hdev, event, data, data_len,
+ hci_send_monitor_ctrl_event(hdev, bt_cb(skb)->mgmt.opcode,
+ skb->data, skb->len,
skb_get_ktime(skb), flag, skip_sk);
+ hdr = skb_push(skb, sizeof(*hdr));
+ hdr->opcode = cpu_to_le16(bt_cb(skb)->mgmt.opcode);
+ if (hdev)
+ hdr->index = cpu_to_le16(hdev->id);
+ else
+ hdr->index = cpu_to_le16(MGMT_INDEX_NONE);
+ hdr->len = cpu_to_le16(len);
+
+ hci_send_to_channel(channel, skb, flag, skip_sk);
+
kfree_skb(skb);
return 0;
}
+int mgmt_send_event(u16 event, struct hci_dev *hdev, unsigned short channel,
+ void *data, u16 data_len, int flag, struct sock *skip_sk)
+{
+ struct sk_buff *skb;
+
+ skb = mgmt_alloc_skb(hdev, event, data_len);
+ if (!skb)
+ return -ENOMEM;
+
+ if (data)
+ skb_put_data(skb, data, data_len);
+
+ return mgmt_send_event_skb(channel, skb, flag, skip_sk);
+}
+
int mgmt_cmd_status(struct sock *sk, u16 index, u16 cmd, u8 status)
{
struct sk_buff *skb, *mskb;
@@ -227,7 +259,7 @@ void mgmt_pending_foreach(u16 opcode, struct hci_dev *hdev,
}
}
-struct mgmt_pending_cmd *mgmt_pending_add(struct sock *sk, u16 opcode,
+struct mgmt_pending_cmd *mgmt_pending_new(struct sock *sk, u16 opcode,
struct hci_dev *hdev,
void *data, u16 len)
{
@@ -251,6 +283,19 @@ struct mgmt_pending_cmd *mgmt_pending_add(struct sock *sk, u16 opcode,
cmd->sk = sk;
sock_hold(sk);
+ return cmd;
+}
+
+struct mgmt_pending_cmd *mgmt_pending_add(struct sock *sk, u16 opcode,
+ struct hci_dev *hdev,
+ void *data, u16 len)
+{
+ struct mgmt_pending_cmd *cmd;
+
+ cmd = mgmt_pending_new(sk, opcode, hdev, data, len);
+ if (!cmd)
+ return NULL;
+
list_add(&cmd->list, &hdev->mgmt_pending);
return cmd;
diff --git a/net/bluetooth/mgmt_util.h b/net/bluetooth/mgmt_util.h
index 6559f18..98e4039 100644
--- a/net/bluetooth/mgmt_util.h
+++ b/net/bluetooth/mgmt_util.h
@@ -27,10 +27,15 @@ struct mgmt_pending_cmd {
void *param;
size_t param_len;
struct sock *sk;
+ struct sk_buff *skb;
void *user_data;
int (*cmd_complete)(struct mgmt_pending_cmd *cmd, u8 status);
};
+struct sk_buff *mgmt_alloc_skb(struct hci_dev *hdev, u16 opcode,
+ unsigned int size);
+int mgmt_send_event_skb(unsigned short channel, struct sk_buff *skb, int flag,
+ struct sock *skip_sk);
int mgmt_send_event(u16 event, struct hci_dev *hdev, unsigned short channel,
void *data, u16 data_len, int flag, struct sock *skip_sk);
int mgmt_cmd_status(struct sock *sk, u16 index, u16 cmd, u8 status);
@@ -49,5 +54,8 @@ void mgmt_pending_foreach(u16 opcode, struct hci_dev *hdev,
struct mgmt_pending_cmd *mgmt_pending_add(struct sock *sk, u16 opcode,
struct hci_dev *hdev,
void *data, u16 len);
+struct mgmt_pending_cmd *mgmt_pending_new(struct sock *sk, u16 opcode,
+ struct hci_dev *hdev,
+ void *data, u16 len);
void mgmt_pending_free(struct mgmt_pending_cmd *cmd);
void mgmt_pending_remove(struct mgmt_pending_cmd *cmd);
diff --git a/net/bluetooth/msft.c b/net/bluetooth/msft.c
index 255cffa..6a94363 100644
--- a/net/bluetooth/msft.c
+++ b/net/bluetooth/msft.c
@@ -93,7 +93,7 @@ struct msft_data {
struct list_head handle_map;
__u16 pending_add_handle;
__u16 pending_remove_handle;
- __u8 reregistering;
+ __u8 resuming;
__u8 suspending;
__u8 filter_enabled;
};
@@ -156,7 +156,6 @@ static bool read_supported_features(struct hci_dev *hdev,
return false;
}
-/* This function requires the caller holds hdev->lock */
static void reregister_monitor(struct hci_dev *hdev, int handle)
{
struct adv_monitor *monitor;
@@ -166,9 +165,9 @@ static void reregister_monitor(struct hci_dev *hdev, int handle)
while (1) {
monitor = idr_get_next(&hdev->adv_monitors_idr, &handle);
if (!monitor) {
- /* All monitors have been reregistered */
- msft->reregistering = false;
- hci_update_background_scan(hdev);
+ /* All monitors have been resumed */
+ msft->resuming = false;
+ hci_update_passive_scan(hdev);
return;
}
@@ -185,202 +184,6 @@ static void reregister_monitor(struct hci_dev *hdev, int handle)
}
}
-/* This function requires the caller holds hdev->lock */
-static void remove_monitor_on_suspend(struct hci_dev *hdev, int handle)
-{
- struct adv_monitor *monitor;
- struct msft_data *msft = hdev->msft_data;
- int err;
-
- while (1) {
- monitor = idr_get_next(&hdev->adv_monitors_idr, &handle);
- if (!monitor) {
- /* All monitors have been removed */
- msft->suspending = false;
- hci_update_background_scan(hdev);
- return;
- }
-
- msft->pending_remove_handle = (u16)handle;
- err = __msft_remove_monitor(hdev, monitor, handle);
-
- /* If success, return and wait for monitor removed callback */
- if (!err)
- return;
-
- /* Otherwise free the monitor and keep removing */
- hci_free_adv_monitor(hdev, monitor);
- handle++;
- }
-}
-
-/* This function requires the caller holds hdev->lock */
-void msft_suspend(struct hci_dev *hdev)
-{
- struct msft_data *msft = hdev->msft_data;
-
- if (!msft)
- return;
-
- if (msft_monitor_supported(hdev)) {
- msft->suspending = true;
- /* Quitely remove all monitors on suspend to avoid waking up
- * the system.
- */
- remove_monitor_on_suspend(hdev, 0);
- }
-}
-
-/* This function requires the caller holds hdev->lock */
-void msft_resume(struct hci_dev *hdev)
-{
- struct msft_data *msft = hdev->msft_data;
-
- if (!msft)
- return;
-
- if (msft_monitor_supported(hdev)) {
- msft->reregistering = true;
- /* Monitors are removed on suspend, so we need to add all
- * monitors on resume.
- */
- reregister_monitor(hdev, 0);
- }
-}
-
-void msft_do_open(struct hci_dev *hdev)
-{
- struct msft_data *msft = hdev->msft_data;
-
- if (hdev->msft_opcode == HCI_OP_NOP)
- return;
-
- if (!msft) {
- bt_dev_err(hdev, "MSFT extension not registered");
- return;
- }
-
- bt_dev_dbg(hdev, "Initialize MSFT extension");
-
- /* Reset existing MSFT data before re-reading */
- kfree(msft->evt_prefix);
- msft->evt_prefix = NULL;
- msft->evt_prefix_len = 0;
- msft->features = 0;
-
- if (!read_supported_features(hdev, msft)) {
- hdev->msft_data = NULL;
- kfree(msft);
- return;
- }
-
- if (msft_monitor_supported(hdev)) {
- msft->reregistering = true;
- msft_set_filter_enable(hdev, true);
- /* Monitors get removed on power off, so we need to explicitly
- * tell the controller to re-monitor.
- */
- reregister_monitor(hdev, 0);
- }
-}
-
-void msft_do_close(struct hci_dev *hdev)
-{
- struct msft_data *msft = hdev->msft_data;
- struct msft_monitor_advertisement_handle_data *handle_data, *tmp;
- struct adv_monitor *monitor;
-
- if (!msft)
- return;
-
- bt_dev_dbg(hdev, "Cleanup of MSFT extension");
-
- /* The controller will silently remove all monitors on power off.
- * Therefore, remove handle_data mapping and reset monitor state.
- */
- list_for_each_entry_safe(handle_data, tmp, &msft->handle_map, list) {
- monitor = idr_find(&hdev->adv_monitors_idr,
- handle_data->mgmt_handle);
-
- if (monitor && monitor->state == ADV_MONITOR_STATE_OFFLOADED)
- monitor->state = ADV_MONITOR_STATE_REGISTERED;
-
- list_del(&handle_data->list);
- kfree(handle_data);
- }
-}
-
-void msft_register(struct hci_dev *hdev)
-{
- struct msft_data *msft = NULL;
-
- bt_dev_dbg(hdev, "Register MSFT extension");
-
- msft = kzalloc(sizeof(*msft), GFP_KERNEL);
- if (!msft) {
- bt_dev_err(hdev, "Failed to register MSFT extension");
- return;
- }
-
- INIT_LIST_HEAD(&msft->handle_map);
- hdev->msft_data = msft;
-}
-
-void msft_unregister(struct hci_dev *hdev)
-{
- struct msft_data *msft = hdev->msft_data;
-
- if (!msft)
- return;
-
- bt_dev_dbg(hdev, "Unregister MSFT extension");
-
- hdev->msft_data = NULL;
-
- kfree(msft->evt_prefix);
- kfree(msft);
-}
-
-void msft_vendor_evt(struct hci_dev *hdev, struct sk_buff *skb)
-{
- struct msft_data *msft = hdev->msft_data;
- u8 event;
-
- if (!msft)
- return;
-
- /* When the extension has defined an event prefix, check that it
- * matches, and otherwise just return.
- */
- if (msft->evt_prefix_len > 0) {
- if (skb->len < msft->evt_prefix_len)
- return;
-
- if (memcmp(skb->data, msft->evt_prefix, msft->evt_prefix_len))
- return;
-
- skb_pull(skb, msft->evt_prefix_len);
- }
-
- /* Every event starts at least with an event code and the rest of
- * the data is variable and depends on the event code.
- */
- if (skb->len < 1)
- return;
-
- event = *skb->data;
- skb_pull(skb, 1);
-
- bt_dev_dbg(hdev, "MSFT vendor event %u", event);
-}
-
-__u64 msft_get_features(struct hci_dev *hdev)
-{
- struct msft_data *msft = hdev->msft_data;
-
- return msft ? msft->features : 0;
-}
-
/* is_mgmt = true matches the handle exposed to userspace via mgmt.
* is_mgmt = false matches the handle used by the msft controller.
* This function requires the caller holds hdev->lock
@@ -446,13 +249,9 @@ static void msft_le_monitor_advertisement_cb(struct hci_dev *hdev,
if (status && monitor)
hci_free_adv_monitor(hdev, monitor);
- /* If in restart/reregister sequence, keep registering. */
- if (msft->reregistering)
- reregister_monitor(hdev, msft->pending_add_handle + 1);
-
hci_dev_unlock(hdev);
- if (!msft->reregistering)
+ if (!msft->resuming)
hci_add_adv_patterns_monitor_complete(hdev, status);
}
@@ -499,11 +298,6 @@ static void msft_le_cancel_monitor_advertisement_cb(struct hci_dev *hdev,
kfree(handle_data);
}
- /* If in suspend/remove sequence, keep removing. */
- if (msft->suspending)
- remove_monitor_on_suspend(hdev,
- msft->pending_remove_handle + 1);
-
/* If remove all monitors is required, we need to continue the process
* here because the earlier it was paused when waiting for the
* response from controller.
@@ -526,6 +320,316 @@ static void msft_le_cancel_monitor_advertisement_cb(struct hci_dev *hdev,
hci_remove_adv_monitor_complete(hdev, status);
}
+static int msft_remove_monitor_sync(struct hci_dev *hdev,
+ struct adv_monitor *monitor)
+{
+ struct msft_cp_le_cancel_monitor_advertisement cp;
+ struct msft_monitor_advertisement_handle_data *handle_data;
+ struct sk_buff *skb;
+ u8 status;
+
+ handle_data = msft_find_handle_data(hdev, monitor->handle, true);
+
+ /* If no matched handle, just remove without telling controller */
+ if (!handle_data)
+ return -ENOENT;
+
+ cp.sub_opcode = MSFT_OP_LE_CANCEL_MONITOR_ADVERTISEMENT;
+ cp.handle = handle_data->msft_handle;
+
+ skb = __hci_cmd_sync(hdev, hdev->msft_opcode, sizeof(cp), &cp,
+ HCI_CMD_TIMEOUT);
+ if (IS_ERR(skb))
+ return PTR_ERR(skb);
+
+ status = skb->data[0];
+ skb_pull(skb, 1);
+
+ msft_le_cancel_monitor_advertisement_cb(hdev, status, hdev->msft_opcode,
+ skb);
+
+ return status;
+}
+
+/* This function requires the caller holds hci_req_sync_lock */
+int msft_suspend_sync(struct hci_dev *hdev)
+{
+ struct msft_data *msft = hdev->msft_data;
+ struct adv_monitor *monitor;
+ int handle = 0;
+
+ if (!msft || !msft_monitor_supported(hdev))
+ return 0;
+
+ msft->suspending = true;
+
+ while (1) {
+ monitor = idr_get_next(&hdev->adv_monitors_idr, &handle);
+ if (!monitor)
+ break;
+
+ msft_remove_monitor_sync(hdev, monitor);
+
+ handle++;
+ }
+
+ /* All monitors have been removed */
+ msft->suspending = false;
+
+ return 0;
+}
+
+static bool msft_monitor_rssi_valid(struct adv_monitor *monitor)
+{
+ struct adv_rssi_thresholds *r = &monitor->rssi;
+
+ if (r->high_threshold < MSFT_RSSI_THRESHOLD_VALUE_MIN ||
+ r->high_threshold > MSFT_RSSI_THRESHOLD_VALUE_MAX ||
+ r->low_threshold < MSFT_RSSI_THRESHOLD_VALUE_MIN ||
+ r->low_threshold > MSFT_RSSI_THRESHOLD_VALUE_MAX)
+ return false;
+
+ /* High_threshold_timeout is not supported,
+ * once high_threshold is reached, events are immediately reported.
+ */
+ if (r->high_threshold_timeout != 0)
+ return false;
+
+ if (r->low_threshold_timeout > MSFT_RSSI_LOW_TIMEOUT_MAX)
+ return false;
+
+ /* Sampling period from 0x00 to 0xFF are all allowed */
+ return true;
+}
+
+static bool msft_monitor_pattern_valid(struct adv_monitor *monitor)
+{
+ return msft_monitor_rssi_valid(monitor);
+ /* No additional check needed for pattern-based monitor */
+}
+
+static int msft_add_monitor_sync(struct hci_dev *hdev,
+ struct adv_monitor *monitor)
+{
+ struct msft_cp_le_monitor_advertisement *cp;
+ struct msft_le_monitor_advertisement_pattern_data *pattern_data;
+ struct msft_le_monitor_advertisement_pattern *pattern;
+ struct adv_pattern *entry;
+ size_t total_size = sizeof(*cp) + sizeof(*pattern_data);
+ ptrdiff_t offset = 0;
+ u8 pattern_count = 0;
+ struct sk_buff *skb;
+ u8 status;
+
+ if (!msft_monitor_pattern_valid(monitor))
+ return -EINVAL;
+
+ list_for_each_entry(entry, &monitor->patterns, list) {
+ pattern_count++;
+ total_size += sizeof(*pattern) + entry->length;
+ }
+
+ cp = kmalloc(total_size, GFP_KERNEL);
+ if (!cp)
+ return -ENOMEM;
+
+ cp->sub_opcode = MSFT_OP_LE_MONITOR_ADVERTISEMENT;
+ cp->rssi_high = monitor->rssi.high_threshold;
+ cp->rssi_low = monitor->rssi.low_threshold;
+ cp->rssi_low_interval = (u8)monitor->rssi.low_threshold_timeout;
+ cp->rssi_sampling_period = monitor->rssi.sampling_period;
+
+ cp->cond_type = MSFT_MONITOR_ADVERTISEMENT_TYPE_PATTERN;
+
+ pattern_data = (void *)cp->data;
+ pattern_data->count = pattern_count;
+
+ list_for_each_entry(entry, &monitor->patterns, list) {
+ pattern = (void *)(pattern_data->data + offset);
+ /* the length also includes data_type and offset */
+ pattern->length = entry->length + 2;
+ pattern->data_type = entry->ad_type;
+ pattern->start_byte = entry->offset;
+ memcpy(pattern->pattern, entry->value, entry->length);
+ offset += sizeof(*pattern) + entry->length;
+ }
+
+ skb = __hci_cmd_sync(hdev, hdev->msft_opcode, total_size, cp,
+ HCI_CMD_TIMEOUT);
+ kfree(cp);
+
+ if (IS_ERR(skb))
+ return PTR_ERR(skb);
+
+ status = skb->data[0];
+ skb_pull(skb, 1);
+
+ msft_le_monitor_advertisement_cb(hdev, status, hdev->msft_opcode, skb);
+
+ return status;
+}
+
+/* This function requires the caller holds hci_req_sync_lock */
+int msft_resume_sync(struct hci_dev *hdev)
+{
+ struct msft_data *msft = hdev->msft_data;
+ struct adv_monitor *monitor;
+ int handle = 0;
+
+ if (!msft || !msft_monitor_supported(hdev))
+ return 0;
+
+ msft->resuming = true;
+
+ while (1) {
+ monitor = idr_get_next(&hdev->adv_monitors_idr, &handle);
+ if (!monitor)
+ break;
+
+ msft_add_monitor_sync(hdev, monitor);
+
+ handle++;
+ }
+
+ /* All monitors have been resumed */
+ msft->resuming = false;
+
+ return 0;
+}
+
+void msft_do_open(struct hci_dev *hdev)
+{
+ struct msft_data *msft = hdev->msft_data;
+
+ if (hdev->msft_opcode == HCI_OP_NOP)
+ return;
+
+ if (!msft) {
+ bt_dev_err(hdev, "MSFT extension not registered");
+ return;
+ }
+
+ bt_dev_dbg(hdev, "Initialize MSFT extension");
+
+ /* Reset existing MSFT data before re-reading */
+ kfree(msft->evt_prefix);
+ msft->evt_prefix = NULL;
+ msft->evt_prefix_len = 0;
+ msft->features = 0;
+
+ if (!read_supported_features(hdev, msft)) {
+ hdev->msft_data = NULL;
+ kfree(msft);
+ return;
+ }
+
+ if (msft_monitor_supported(hdev)) {
+ msft->resuming = true;
+ msft_set_filter_enable(hdev, true);
+ /* Monitors get removed on power off, so we need to explicitly
+ * tell the controller to re-monitor.
+ */
+ reregister_monitor(hdev, 0);
+ }
+}
+
+void msft_do_close(struct hci_dev *hdev)
+{
+ struct msft_data *msft = hdev->msft_data;
+ struct msft_monitor_advertisement_handle_data *handle_data, *tmp;
+ struct adv_monitor *monitor;
+
+ if (!msft)
+ return;
+
+ bt_dev_dbg(hdev, "Cleanup of MSFT extension");
+
+ /* The controller will silently remove all monitors on power off.
+ * Therefore, remove handle_data mapping and reset monitor state.
+ */
+ list_for_each_entry_safe(handle_data, tmp, &msft->handle_map, list) {
+ monitor = idr_find(&hdev->adv_monitors_idr,
+ handle_data->mgmt_handle);
+
+ if (monitor && monitor->state == ADV_MONITOR_STATE_OFFLOADED)
+ monitor->state = ADV_MONITOR_STATE_REGISTERED;
+
+ list_del(&handle_data->list);
+ kfree(handle_data);
+ }
+}
+
+void msft_register(struct hci_dev *hdev)
+{
+ struct msft_data *msft = NULL;
+
+ bt_dev_dbg(hdev, "Register MSFT extension");
+
+ msft = kzalloc(sizeof(*msft), GFP_KERNEL);
+ if (!msft) {
+ bt_dev_err(hdev, "Failed to register MSFT extension");
+ return;
+ }
+
+ INIT_LIST_HEAD(&msft->handle_map);
+ hdev->msft_data = msft;
+}
+
+void msft_unregister(struct hci_dev *hdev)
+{
+ struct msft_data *msft = hdev->msft_data;
+
+ if (!msft)
+ return;
+
+ bt_dev_dbg(hdev, "Unregister MSFT extension");
+
+ hdev->msft_data = NULL;
+
+ kfree(msft->evt_prefix);
+ kfree(msft);
+}
+
+void msft_vendor_evt(struct hci_dev *hdev, void *data, struct sk_buff *skb)
+{
+ struct msft_data *msft = hdev->msft_data;
+ u8 event;
+
+ if (!msft)
+ return;
+
+ /* When the extension has defined an event prefix, check that it
+ * matches, and otherwise just return.
+ */
+ if (msft->evt_prefix_len > 0) {
+ if (skb->len < msft->evt_prefix_len)
+ return;
+
+ if (memcmp(skb->data, msft->evt_prefix, msft->evt_prefix_len))
+ return;
+
+ skb_pull(skb, msft->evt_prefix_len);
+ }
+
+ /* Every event starts at least with an event code and the rest of
+ * the data is variable and depends on the event code.
+ */
+ if (skb->len < 1)
+ return;
+
+ event = *skb->data;
+ skb_pull(skb, 1);
+
+ bt_dev_dbg(hdev, "MSFT vendor event %u", event);
+}
+
+__u64 msft_get_features(struct hci_dev *hdev)
+{
+ struct msft_data *msft = hdev->msft_data;
+
+ return msft ? msft->features : 0;
+}
+
static void msft_le_set_advertisement_filter_enable_cb(struct hci_dev *hdev,
u8 status, u16 opcode,
struct sk_buff *skb)
@@ -560,35 +664,6 @@ static void msft_le_set_advertisement_filter_enable_cb(struct hci_dev *hdev,
hci_dev_unlock(hdev);
}
-static bool msft_monitor_rssi_valid(struct adv_monitor *monitor)
-{
- struct adv_rssi_thresholds *r = &monitor->rssi;
-
- if (r->high_threshold < MSFT_RSSI_THRESHOLD_VALUE_MIN ||
- r->high_threshold > MSFT_RSSI_THRESHOLD_VALUE_MAX ||
- r->low_threshold < MSFT_RSSI_THRESHOLD_VALUE_MIN ||
- r->low_threshold > MSFT_RSSI_THRESHOLD_VALUE_MAX)
- return false;
-
- /* High_threshold_timeout is not supported,
- * once high_threshold is reached, events are immediately reported.
- */
- if (r->high_threshold_timeout != 0)
- return false;
-
- if (r->low_threshold_timeout > MSFT_RSSI_LOW_TIMEOUT_MAX)
- return false;
-
- /* Sampling period from 0x00 to 0xFF are all allowed */
- return true;
-}
-
-static bool msft_monitor_pattern_valid(struct adv_monitor *monitor)
-{
- return msft_monitor_rssi_valid(monitor);
- /* No additional check needed for pattern-based monitor */
-}
-
/* This function requires the caller holds hdev->lock */
static int __msft_add_monitor_pattern(struct hci_dev *hdev,
struct adv_monitor *monitor)
@@ -656,7 +731,7 @@ int msft_add_monitor_pattern(struct hci_dev *hdev, struct adv_monitor *monitor)
if (!msft)
return -EOPNOTSUPP;
- if (msft->reregistering || msft->suspending)
+ if (msft->resuming || msft->suspending)
return -EBUSY;
return __msft_add_monitor_pattern(hdev, monitor);
@@ -700,7 +775,7 @@ int msft_remove_monitor(struct hci_dev *hdev, struct adv_monitor *monitor,
if (!msft)
return -EOPNOTSUPP;
- if (msft->reregistering || msft->suspending)
+ if (msft->resuming || msft->suspending)
return -EBUSY;
return __msft_remove_monitor(hdev, monitor, handle);
diff --git a/net/bluetooth/msft.h b/net/bluetooth/msft.h
index 59c6e08..afcaf7d 100644
--- a/net/bluetooth/msft.h
+++ b/net/bluetooth/msft.h
@@ -17,15 +17,15 @@ void msft_register(struct hci_dev *hdev);
void msft_unregister(struct hci_dev *hdev);
void msft_do_open(struct hci_dev *hdev);
void msft_do_close(struct hci_dev *hdev);
-void msft_vendor_evt(struct hci_dev *hdev, struct sk_buff *skb);
+void msft_vendor_evt(struct hci_dev *hdev, void *data, struct sk_buff *skb);
__u64 msft_get_features(struct hci_dev *hdev);
int msft_add_monitor_pattern(struct hci_dev *hdev, struct adv_monitor *monitor);
int msft_remove_monitor(struct hci_dev *hdev, struct adv_monitor *monitor,
u16 handle);
void msft_req_add_set_filter_enable(struct hci_request *req, bool enable);
int msft_set_filter_enable(struct hci_dev *hdev, bool enable);
-void msft_suspend(struct hci_dev *hdev);
-void msft_resume(struct hci_dev *hdev);
+int msft_suspend_sync(struct hci_dev *hdev);
+int msft_resume_sync(struct hci_dev *hdev);
bool msft_curve_validity(struct hci_dev *hdev);
#else
@@ -39,7 +39,8 @@ static inline void msft_register(struct hci_dev *hdev) {}
static inline void msft_unregister(struct hci_dev *hdev) {}
static inline void msft_do_open(struct hci_dev *hdev) {}
static inline void msft_do_close(struct hci_dev *hdev) {}
-static inline void msft_vendor_evt(struct hci_dev *hdev, struct sk_buff *skb) {}
+static inline void msft_vendor_evt(struct hci_dev *hdev, void *data,
+ struct sk_buff *skb) {}
static inline __u64 msft_get_features(struct hci_dev *hdev) { return 0; }
static inline int msft_add_monitor_pattern(struct hci_dev *hdev,
struct adv_monitor *monitor)
@@ -61,8 +62,15 @@ static inline int msft_set_filter_enable(struct hci_dev *hdev, bool enable)
return -EOPNOTSUPP;
}
-static inline void msft_suspend(struct hci_dev *hdev) {}
-static inline void msft_resume(struct hci_dev *hdev) {}
+static inline int msft_suspend_sync(struct hci_dev *hdev)
+{
+ return -EOPNOTSUPP;
+}
+
+static inline int msft_resume_sync(struct hci_dev *hdev)
+{
+ return -EOPNOTSUPP;
+}
static inline bool msft_curve_validity(struct hci_dev *hdev)
{
diff --git a/net/bridge/br_if.c b/net/bridge/br_if.c
index c1183fe..a52ad81 100644
--- a/net/bridge/br_if.c
+++ b/net/bridge/br_if.c
@@ -274,7 +274,7 @@ static void destroy_nbp(struct net_bridge_port *p)
p->br = NULL;
p->dev = NULL;
- dev_put(dev);
+ dev_put_track(dev, &p->dev_tracker);
kobject_put(&p->kobj);
}
@@ -397,10 +397,10 @@ static int find_portno(struct net_bridge *br)
if (!inuse)
return -ENOMEM;
- set_bit(0, inuse); /* zero is reserved */
- list_for_each_entry(p, &br->port_list, list) {
- set_bit(p->port_no, inuse);
- }
+ __set_bit(0, inuse); /* zero is reserved */
+ list_for_each_entry(p, &br->port_list, list)
+ __set_bit(p->port_no, inuse);
+
index = find_first_zero_bit(inuse, BR_MAX_PORTS);
bitmap_free(inuse);
@@ -423,7 +423,7 @@ static struct net_bridge_port *new_nbp(struct net_bridge *br,
return ERR_PTR(-ENOMEM);
p->br = br;
- dev_hold(dev);
+ dev_hold_track(dev, &p->dev_tracker, GFP_KERNEL);
p->dev = dev;
p->path_cost = port_cost(dev);
p->priority = 0x8000 >> BR_PORT_BITS;
@@ -434,7 +434,7 @@ static struct net_bridge_port *new_nbp(struct net_bridge *br,
br_stp_port_timer_init(p);
err = br_multicast_add_port(p);
if (err) {
- dev_put(dev);
+ dev_put_track(dev, &p->dev_tracker);
kfree(p);
p = ERR_PTR(err);
}
@@ -525,8 +525,8 @@ static void br_set_gso_limits(struct net_bridge *br)
gso_max_size = min(gso_max_size, p->dev->gso_max_size);
gso_max_segs = min(gso_max_segs, p->dev->gso_max_segs);
}
- br->dev->gso_max_size = gso_max_size;
- br->dev->gso_max_segs = gso_max_segs;
+ netif_set_gso_max_size(br->dev, gso_max_size);
+ netif_set_gso_max_segs(br->dev, gso_max_segs);
}
/*
diff --git a/net/bridge/br_ioctl.c b/net/bridge/br_ioctl.c
index 891cfcf..f213ed1 100644
--- a/net/bridge/br_ioctl.c
+++ b/net/bridge/br_ioctl.c
@@ -8,6 +8,7 @@
*/
#include <linux/capability.h>
+#include <linux/compat.h>
#include <linux/kernel.h>
#include <linux/if_bridge.h>
#include <linux/netdevice.h>
@@ -102,37 +103,56 @@ static int add_del_if(struct net_bridge *br, int ifindex, int isadd)
return ret;
}
+#define BR_UARGS_MAX 4
+static int br_dev_read_uargs(unsigned long *args, size_t nr_args,
+ void __user **argp, void __user *data)
+{
+ int ret;
+
+ if (nr_args < 2 || nr_args > BR_UARGS_MAX)
+ return -EINVAL;
+
+ if (in_compat_syscall()) {
+ unsigned int cargs[BR_UARGS_MAX];
+ int i;
+
+ ret = copy_from_user(cargs, data, nr_args * sizeof(*cargs));
+ if (ret)
+ goto fault;
+
+ for (i = 0; i < nr_args; ++i)
+ args[i] = cargs[i];
+
+ *argp = compat_ptr(args[1]);
+ } else {
+ ret = copy_from_user(args, data, nr_args * sizeof(*args));
+ if (ret)
+ goto fault;
+ *argp = (void __user *)args[1];
+ }
+
+ return 0;
+fault:
+ return -EFAULT;
+}
+
/*
* Legacy ioctl's through SIOCDEVPRIVATE
* This interface is deprecated because it was too difficult
* to do the translation for 32/64bit ioctl compatibility.
*/
-int br_dev_siocdevprivate(struct net_device *dev, struct ifreq *rq, void __user *data, int cmd)
+int br_dev_siocdevprivate(struct net_device *dev, struct ifreq *rq,
+ void __user *data, int cmd)
{
struct net_bridge *br = netdev_priv(dev);
struct net_bridge_port *p = NULL;
unsigned long args[4];
void __user *argp;
- int ret = -EOPNOTSUPP;
+ int ret;
- if (in_compat_syscall()) {
- unsigned int cargs[4];
-
- if (copy_from_user(cargs, data, sizeof(cargs)))
- return -EFAULT;
-
- args[0] = cargs[0];
- args[1] = cargs[1];
- args[2] = cargs[2];
- args[3] = cargs[3];
-
- argp = compat_ptr(args[1]);
- } else {
- if (copy_from_user(args, data, sizeof(args)))
- return -EFAULT;
-
- argp = (void __user *)args[1];
- }
+ ret = br_dev_read_uargs(args, ARRAY_SIZE(args), &argp, data);
+ if (ret)
+ return ret;
switch (args[0]) {
case BRCTL_ADD_IF:
@@ -301,6 +321,9 @@ int br_dev_siocdevprivate(struct net_device *dev, struct ifreq *rq, void __user
case BRCTL_GET_FDB_ENTRIES:
return get_fdb_entries(br, argp, args[2], args[3]);
+
+ default:
+ ret = -EOPNOTSUPP;
}
if (!ret) {
@@ -313,12 +336,15 @@ int br_dev_siocdevprivate(struct net_device *dev, struct ifreq *rq, void __user
return ret;
}
-static int old_deviceless(struct net *net, void __user *uarg)
+static int old_deviceless(struct net *net, void __user *data)
{
unsigned long args[3];
+ void __user *argp;
+ int ret;
- if (copy_from_user(args, uarg, sizeof(args)))
- return -EFAULT;
+ ret = br_dev_read_uargs(args, ARRAY_SIZE(args), &argp, data);
+ if (ret)
+ return ret;
switch (args[0]) {
case BRCTL_GET_VERSION:
@@ -337,7 +363,7 @@ static int old_deviceless(struct net *net, void __user *uarg)
args[2] = get_bridge_ifindices(net, indices, args[2]);
- ret = copy_to_user((void __user *)args[1], indices,
+ ret = copy_to_user(argp, indices,
array_size(args[2], sizeof(int)))
? -EFAULT : args[2];
@@ -353,7 +379,7 @@ static int old_deviceless(struct net *net, void __user *uarg)
if (!ns_capable(net->user_ns, CAP_NET_ADMIN))
return -EPERM;
- if (copy_from_user(buf, (void __user *)args[1], IFNAMSIZ))
+ if (copy_from_user(buf, argp, IFNAMSIZ))
return -EFAULT;
buf[IFNAMSIZ-1] = 0;
diff --git a/net/bridge/br_multicast.c b/net/bridge/br_multicast.c
index f3d7511..de24098 100644
--- a/net/bridge/br_multicast.c
+++ b/net/bridge/br_multicast.c
@@ -4522,6 +4522,38 @@ int br_multicast_set_mld_version(struct net_bridge_mcast *brmctx,
}
#endif
+void br_multicast_set_query_intvl(struct net_bridge_mcast *brmctx,
+ unsigned long val)
+{
+ unsigned long intvl_jiffies = clock_t_to_jiffies(val);
+
+ if (intvl_jiffies < BR_MULTICAST_QUERY_INTVL_MIN) {
+ br_info(brmctx->br,
+ "trying to set multicast query interval below minimum, setting to %lu (%ums)\n",
+ jiffies_to_clock_t(BR_MULTICAST_QUERY_INTVL_MIN),
+ jiffies_to_msecs(BR_MULTICAST_QUERY_INTVL_MIN));
+ intvl_jiffies = BR_MULTICAST_QUERY_INTVL_MIN;
+ }
+
+ brmctx->multicast_query_interval = intvl_jiffies;
+}
+
+void br_multicast_set_startup_query_intvl(struct net_bridge_mcast *brmctx,
+ unsigned long val)
+{
+ unsigned long intvl_jiffies = clock_t_to_jiffies(val);
+
+ if (intvl_jiffies < BR_MULTICAST_STARTUP_QUERY_INTVL_MIN) {
+ br_info(brmctx->br,
+ "trying to set multicast startup query interval below minimum, setting to %lu (%ums)\n",
+ jiffies_to_clock_t(BR_MULTICAST_STARTUP_QUERY_INTVL_MIN),
+ jiffies_to_msecs(BR_MULTICAST_STARTUP_QUERY_INTVL_MIN));
+ intvl_jiffies = BR_MULTICAST_STARTUP_QUERY_INTVL_MIN;
+ }
+
+ brmctx->multicast_startup_query_interval = intvl_jiffies;
+}
+
/**
* br_multicast_list_adjacent - Returns snooped multicast addresses
* @dev: The bridge port adjacent to which to retrieve addresses
diff --git a/net/bridge/br_netfilter_hooks.c b/net/bridge/br_netfilter_hooks.c
index b5af68c..4fd8826 100644
--- a/net/bridge/br_netfilter_hooks.c
+++ b/net/bridge/br_netfilter_hooks.c
@@ -743,6 +743,9 @@ static int br_nf_dev_queue_xmit(struct net *net, struct sock *sk, struct sk_buff
if (nf_bridge->frag_max_size && nf_bridge->frag_max_size < mtu)
mtu = nf_bridge->frag_max_size;
+ nf_bridge_update_protocol(skb);
+ nf_bridge_push_encap_header(skb);
+
if (skb_is_gso(skb) || skb->len + mtu_reserved <= mtu) {
nf_bridge_info_free(skb);
return br_dev_queue_push_xmit(net, sk, skb);
@@ -760,8 +763,6 @@ static int br_nf_dev_queue_xmit(struct net *net, struct sock *sk, struct sk_buff
IPCB(skb)->frag_max_size = nf_bridge->frag_max_size;
- nf_bridge_update_protocol(skb);
-
data = this_cpu_ptr(&brnf_frag_data_storage);
if (skb_vlan_tag_present(skb)) {
@@ -789,8 +790,6 @@ static int br_nf_dev_queue_xmit(struct net *net, struct sock *sk, struct sk_buff
IP6CB(skb)->frag_max_size = nf_bridge->frag_max_size;
- nf_bridge_update_protocol(skb);
-
data = this_cpu_ptr(&brnf_frag_data_storage);
data->encap_size = nf_bridge_encap_header_len(skb);
data->size = ETH_HLEN + data->encap_size;
diff --git a/net/bridge/br_netlink.c b/net/bridge/br_netlink.c
index 0c8b5f1..2ff83d8 100644
--- a/net/bridge/br_netlink.c
+++ b/net/bridge/br_netlink.c
@@ -1357,7 +1357,7 @@ static int br_changelink(struct net_device *brdev, struct nlattr *tb[],
if (data[IFLA_BR_MCAST_QUERY_INTVL]) {
u64 val = nla_get_u64(data[IFLA_BR_MCAST_QUERY_INTVL]);
- br->multicast_ctx.multicast_query_interval = clock_t_to_jiffies(val);
+ br_multicast_set_query_intvl(&br->multicast_ctx, val);
}
if (data[IFLA_BR_MCAST_QUERY_RESPONSE_INTVL]) {
@@ -1369,7 +1369,7 @@ static int br_changelink(struct net_device *brdev, struct nlattr *tb[],
if (data[IFLA_BR_MCAST_STARTUP_QUERY_INTVL]) {
u64 val = nla_get_u64(data[IFLA_BR_MCAST_STARTUP_QUERY_INTVL]);
- br->multicast_ctx.multicast_startup_query_interval = clock_t_to_jiffies(val);
+ br_multicast_set_startup_query_intvl(&br->multicast_ctx, val);
}
if (data[IFLA_BR_MCAST_STATS_ENABLED]) {
diff --git a/net/bridge/br_private.h b/net/bridge/br_private.h
index c0efd69..2661dda 100644
--- a/net/bridge/br_private.h
+++ b/net/bridge/br_private.h
@@ -28,6 +28,8 @@
#define BR_MAX_PORTS (1<<BR_PORT_BITS)
#define BR_MULTICAST_DEFAULT_HASH_MAX 4096
+#define BR_MULTICAST_QUERY_INTVL_MIN msecs_to_jiffies(1000)
+#define BR_MULTICAST_STARTUP_QUERY_INTVL_MIN BR_MULTICAST_QUERY_INTVL_MIN
#define BR_HWDOM_MAX BITS_PER_LONG
@@ -344,6 +346,7 @@ struct net_bridge_mdb_entry {
struct net_bridge_port {
struct net_bridge *br;
struct net_device *dev;
+ netdevice_tracker dev_tracker;
struct list_head list;
unsigned long flags;
@@ -963,6 +966,10 @@ int br_multicast_dump_querier_state(struct sk_buff *skb,
int nest_attr);
size_t br_multicast_querier_state_size(void);
size_t br_rports_size(const struct net_bridge_mcast *brmctx);
+void br_multicast_set_query_intvl(struct net_bridge_mcast *brmctx,
+ unsigned long val);
+void br_multicast_set_startup_query_intvl(struct net_bridge_mcast *brmctx,
+ unsigned long val);
static inline bool br_group_is_l2(const struct br_ip *group)
{
@@ -1147,9 +1154,9 @@ br_multicast_port_ctx_get_global(const struct net_bridge_mcast_port *pmctx)
static inline bool
br_multicast_ctx_vlan_global_disabled(const struct net_bridge_mcast *brmctx)
{
- return br_opt_get(brmctx->br, BROPT_MCAST_VLAN_SNOOPING_ENABLED) &&
- br_multicast_ctx_is_vlan(brmctx) &&
- !(brmctx->vlan->priv_flags & BR_VLFLAG_GLOBAL_MCAST_ENABLED);
+ return br_multicast_ctx_is_vlan(brmctx) &&
+ (!br_opt_get(brmctx->br, BROPT_MCAST_VLAN_SNOOPING_ENABLED) ||
+ !(brmctx->vlan->priv_flags & BR_VLFLAG_GLOBAL_MCAST_ENABLED));
}
static inline bool
diff --git a/net/bridge/br_sysfs_br.c b/net/bridge/br_sysfs_br.c
index d9a89dd..3f7ca88 100644
--- a/net/bridge/br_sysfs_br.c
+++ b/net/bridge/br_sysfs_br.c
@@ -36,15 +36,14 @@ static ssize_t store_bridge_parm(struct device *d,
struct net_bridge *br = to_bridge(d);
struct netlink_ext_ack extack = {0};
unsigned long val;
- char *endp;
int err;
if (!ns_capable(dev_net(br->dev)->user_ns, CAP_NET_ADMIN))
return -EPERM;
- val = simple_strtoul(buf, &endp, 0);
- if (endp == buf)
- return -EINVAL;
+ err = kstrtoul(buf, 0, &val);
+ if (err != 0)
+ return err;
if (!rtnl_trylock())
return restart_syscall();
@@ -658,7 +657,7 @@ static ssize_t multicast_query_interval_show(struct device *d,
static int set_query_interval(struct net_bridge *br, unsigned long val,
struct netlink_ext_ack *extack)
{
- br->multicast_ctx.multicast_query_interval = clock_t_to_jiffies(val);
+ br_multicast_set_query_intvl(&br->multicast_ctx, val);
return 0;
}
@@ -706,7 +705,7 @@ static ssize_t multicast_startup_query_interval_show(
static int set_startup_query_interval(struct net_bridge *br, unsigned long val,
struct netlink_ext_ack *extack)
{
- br->multicast_ctx.multicast_startup_query_interval = clock_t_to_jiffies(val);
+ br_multicast_set_startup_query_intvl(&br->multicast_ctx, val);
return 0;
}
diff --git a/net/bridge/br_vlan.c b/net/bridge/br_vlan.c
index 49e105e..84ba456 100644
--- a/net/bridge/br_vlan.c
+++ b/net/bridge/br_vlan.c
@@ -1063,7 +1063,7 @@ int __br_vlan_set_default_pvid(struct net_bridge *br, u16 pvid,
if (br_vlan_delete(br, old_pvid))
br_vlan_notify(br, NULL, old_pvid, 0, RTM_DELVLAN);
br_vlan_notify(br, NULL, pvid, 0, RTM_NEWVLAN);
- set_bit(0, changed);
+ __set_bit(0, changed);
}
list_for_each_entry(p, &br->port_list, list) {
@@ -1085,7 +1085,7 @@ int __br_vlan_set_default_pvid(struct net_bridge *br, u16 pvid,
if (nbp_vlan_delete(p, old_pvid))
br_vlan_notify(br, p, old_pvid, 0, RTM_DELVLAN);
br_vlan_notify(p->br, p, pvid, 0, RTM_NEWVLAN);
- set_bit(p->port_no, changed);
+ __set_bit(p->port_no, changed);
}
br->default_pvid = pvid;
diff --git a/net/bridge/br_vlan_options.c b/net/bridge/br_vlan_options.c
index 8ffd4ed..a638297 100644
--- a/net/bridge/br_vlan_options.c
+++ b/net/bridge/br_vlan_options.c
@@ -521,7 +521,7 @@ static int br_vlan_process_global_one_opts(const struct net_bridge *br,
u64 val;
val = nla_get_u64(tb[BRIDGE_VLANDB_GOPTS_MCAST_QUERY_INTVL]);
- v->br_mcast_ctx.multicast_query_interval = clock_t_to_jiffies(val);
+ br_multicast_set_query_intvl(&v->br_mcast_ctx, val);
*changed = true;
}
if (tb[BRIDGE_VLANDB_GOPTS_MCAST_QUERY_RESPONSE_INTVL]) {
@@ -535,7 +535,7 @@ static int br_vlan_process_global_one_opts(const struct net_bridge *br,
u64 val;
val = nla_get_u64(tb[BRIDGE_VLANDB_GOPTS_MCAST_STARTUP_QUERY_INTVL]);
- v->br_mcast_ctx.multicast_startup_query_interval = clock_t_to_jiffies(val);
+ br_multicast_set_startup_query_intvl(&v->br_mcast_ctx, val);
*changed = true;
}
if (tb[BRIDGE_VLANDB_GOPTS_MCAST_QUERIER]) {
diff --git a/net/bridge/netfilter/nft_meta_bridge.c b/net/bridge/netfilter/nft_meta_bridge.c
index 97805ec..c1ef9cc 100644
--- a/net/bridge/netfilter/nft_meta_bridge.c
+++ b/net/bridge/netfilter/nft_meta_bridge.c
@@ -100,6 +100,25 @@ static const struct nft_expr_ops nft_meta_bridge_get_ops = {
.dump = nft_meta_get_dump,
};
+static bool nft_meta_bridge_set_reduce(struct nft_regs_track *track,
+ const struct nft_expr *expr)
+{
+ int i;
+
+ for (i = 0; i < NFT_REG32_NUM; i++) {
+ if (!track->regs[i].selector)
+ continue;
+
+ if (track->regs[i].selector->ops != &nft_meta_bridge_get_ops)
+ continue;
+
+ track->regs[i].selector = NULL;
+ track->regs[i].bitwise = NULL;
+ }
+
+ return false;
+}
+
static const struct nft_expr_ops nft_meta_bridge_set_ops = {
.type = &nft_meta_bridge_type,
.size = NFT_EXPR_SIZE(sizeof(struct nft_meta)),
@@ -107,6 +126,7 @@ static const struct nft_expr_ops nft_meta_bridge_set_ops = {
.init = nft_meta_set_init,
.destroy = nft_meta_set_destroy,
.dump = nft_meta_set_dump,
+ .reduce = nft_meta_bridge_set_reduce,
.validate = nft_meta_set_validate,
};
diff --git a/net/caif/caif_socket.c b/net/caif/caif_socket.c
index e12fd3c..2b8892d5 100644
--- a/net/caif/caif_socket.c
+++ b/net/caif/caif_socket.c
@@ -6,6 +6,7 @@
#define pr_fmt(fmt) KBUILD_MODNAME ":%s(): " fmt, __func__
+#include <linux/filter.h>
#include <linux/fs.h>
#include <linux/init.h>
#include <linux/module.h>
diff --git a/net/caif/cfserl.c b/net/caif/cfserl.c
index 40cd57a..aee11c7 100644
--- a/net/caif/cfserl.c
+++ b/net/caif/cfserl.c
@@ -128,7 +128,6 @@ static int cfserl_receive(struct cflayer *l, struct cfpkt *newpkt)
if (pkt != NULL)
cfpkt_destroy(pkt);
layr->incomplete_frm = NULL;
- expectlen = 0;
spin_unlock(&layr->sync);
return -EPROTO;
}
diff --git a/net/can/isotp.c b/net/can/isotp.c
index df6968b..02cbcb2 100644
--- a/net/can/isotp.c
+++ b/net/can/isotp.c
@@ -119,8 +119,8 @@ enum {
};
struct tpcon {
- int idx;
- int len;
+ unsigned int idx;
+ unsigned int len;
u32 state;
u8 bs;
u8 sn;
diff --git a/net/core/Makefile b/net/core/Makefile
index 4268846..a8e4f73 100644
--- a/net/core/Makefile
+++ b/net/core/Makefile
@@ -11,7 +11,9 @@
obj-y += dev.o dev_addr_lists.o dst.o netevent.o \
neighbour.o rtnetlink.o utils.o link_watch.o filter.o \
sock_diag.o dev_ioctl.o tso.o sock_reuseport.o \
- fib_notifier.o xdp.o flow_offload.o
+ fib_notifier.o xdp.o flow_offload.o gro.o
+
+obj-$(CONFIG_NETDEV_ADDR_LIST_TEST) += dev_addr_lists_test.o
obj-y += net-sysfs.o
obj-$(CONFIG_PAGE_POOL) += page_pool.o
diff --git a/net/core/bpf_sk_storage.c b/net/core/bpf_sk_storage.c
index 68d2cbf..d9c37fd1 100644
--- a/net/core/bpf_sk_storage.c
+++ b/net/core/bpf_sk_storage.c
@@ -13,6 +13,7 @@
#include <net/sock.h>
#include <uapi/linux/sock_diag.h>
#include <uapi/linux/btf.h>
+#include <linux/rcupdate_trace.h>
DEFINE_BPF_STORAGE_CACHE(sk_cache);
@@ -22,7 +23,8 @@ bpf_sk_storage_lookup(struct sock *sk, struct bpf_map *map, bool cacheit_lockit)
struct bpf_local_storage *sk_storage;
struct bpf_local_storage_map *smap;
- sk_storage = rcu_dereference(sk->sk_bpf_storage);
+ sk_storage =
+ rcu_dereference_check(sk->sk_bpf_storage, bpf_rcu_lock_held());
if (!sk_storage)
return NULL;
@@ -258,6 +260,7 @@ BPF_CALL_4(bpf_sk_storage_get, struct bpf_map *, map, struct sock *, sk,
{
struct bpf_local_storage_data *sdata;
+ WARN_ON_ONCE(!bpf_rcu_lock_held());
if (!sk || !sk_fullsock(sk) || flags > BPF_SK_STORAGE_GET_F_CREATE)
return (unsigned long)NULL;
@@ -288,6 +291,7 @@ BPF_CALL_4(bpf_sk_storage_get, struct bpf_map *, map, struct sock *, sk,
BPF_CALL_2(bpf_sk_storage_delete, struct bpf_map *, map, struct sock *, sk)
{
+ WARN_ON_ONCE(!bpf_rcu_lock_held());
if (!sk || !sk_fullsock(sk))
return -EINVAL;
@@ -416,6 +420,7 @@ static bool bpf_sk_storage_tracing_allowed(const struct bpf_prog *prog)
BPF_CALL_4(bpf_sk_storage_get_tracing, struct bpf_map *, map, struct sock *, sk,
void *, value, u64, flags)
{
+ WARN_ON_ONCE(!bpf_rcu_lock_held());
if (in_hardirq() || in_nmi())
return (unsigned long)NULL;
@@ -425,6 +430,7 @@ BPF_CALL_4(bpf_sk_storage_get_tracing, struct bpf_map *, map, struct sock *, sk,
BPF_CALL_2(bpf_sk_storage_delete_tracing, struct bpf_map *, map,
struct sock *, sk)
{
+ WARN_ON_ONCE(!bpf_rcu_lock_held());
if (in_hardirq() || in_nmi())
return -EPERM;
@@ -929,7 +935,7 @@ static struct bpf_iter_reg bpf_sk_storage_map_reg_info = {
{ offsetof(struct bpf_iter__bpf_sk_storage_map, sk),
PTR_TO_BTF_ID_OR_NULL },
{ offsetof(struct bpf_iter__bpf_sk_storage_map, value),
- PTR_TO_RDWR_BUF_OR_NULL },
+ PTR_TO_BUF | PTR_MAYBE_NULL },
},
.seq_info = &iter_seq_info,
};
diff --git a/net/core/dev.c b/net/core/dev.c
index c4708e2..84a0d95 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -153,16 +153,10 @@
#include "net-sysfs.h"
-#define MAX_GRO_SKBS 8
-
-/* This should be increased if a protocol with a bigger head is added. */
-#define GRO_MAX_HEAD (MAX_HEADER + 128)
static DEFINE_SPINLOCK(ptype_lock);
-static DEFINE_SPINLOCK(offload_lock);
struct list_head ptype_base[PTYPE_HASH_SIZE] __read_mostly;
struct list_head ptype_all __read_mostly; /* Taps */
-static struct list_head offload_base __read_mostly;
static int netif_rx_internal(struct sk_buff *skb);
static int call_netdevice_notifiers_info(unsigned long val,
@@ -371,12 +365,12 @@ static void list_netdevice(struct net_device *dev)
ASSERT_RTNL();
- write_lock_bh(&dev_base_lock);
+ write_lock(&dev_base_lock);
list_add_tail_rcu(&dev->dev_list, &net->dev_base_head);
netdev_name_node_add(net, dev->name_node);
hlist_add_head_rcu(&dev->index_hlist,
dev_index_hash(net, dev->ifindex));
- write_unlock_bh(&dev_base_lock);
+ write_unlock(&dev_base_lock);
dev_base_seq_inc(net);
}
@@ -389,11 +383,11 @@ static void unlist_netdevice(struct net_device *dev)
ASSERT_RTNL();
/* Unlink dev from the device chain */
- write_lock_bh(&dev_base_lock);
+ write_lock(&dev_base_lock);
list_del_rcu(&dev->dev_list);
netdev_name_node_del(dev->name_node);
hlist_del_rcu(&dev->index_hlist);
- write_unlock_bh(&dev_base_lock);
+ write_unlock(&dev_base_lock);
dev_base_seq_inc(dev_net(dev));
}
@@ -604,84 +598,6 @@ void dev_remove_pack(struct packet_type *pt)
EXPORT_SYMBOL(dev_remove_pack);
-/**
- * dev_add_offload - register offload handlers
- * @po: protocol offload declaration
- *
- * Add protocol offload handlers to the networking stack. The passed
- * &proto_offload is linked into kernel lists and may not be freed until
- * it has been removed from the kernel lists.
- *
- * This call does not sleep therefore it can not
- * guarantee all CPU's that are in middle of receiving packets
- * will see the new offload handlers (until the next received packet).
- */
-void dev_add_offload(struct packet_offload *po)
-{
- struct packet_offload *elem;
-
- spin_lock(&offload_lock);
- list_for_each_entry(elem, &offload_base, list) {
- if (po->priority < elem->priority)
- break;
- }
- list_add_rcu(&po->list, elem->list.prev);
- spin_unlock(&offload_lock);
-}
-EXPORT_SYMBOL(dev_add_offload);
-
-/**
- * __dev_remove_offload - remove offload handler
- * @po: packet offload declaration
- *
- * Remove a protocol offload handler that was previously added to the
- * kernel offload handlers by dev_add_offload(). The passed &offload_type
- * is removed from the kernel lists and can be freed or reused once this
- * function returns.
- *
- * The packet type might still be in use by receivers
- * and must not be freed until after all the CPU's have gone
- * through a quiescent state.
- */
-static void __dev_remove_offload(struct packet_offload *po)
-{
- struct list_head *head = &offload_base;
- struct packet_offload *po1;
-
- spin_lock(&offload_lock);
-
- list_for_each_entry(po1, head, list) {
- if (po == po1) {
- list_del_rcu(&po->list);
- goto out;
- }
- }
-
- pr_warn("dev_remove_offload: %p not found\n", po);
-out:
- spin_unlock(&offload_lock);
-}
-
-/**
- * dev_remove_offload - remove packet offload handler
- * @po: packet offload declaration
- *
- * Remove a packet offload handler that was previously added to the kernel
- * offload handlers by dev_add_offload(). The passed &offload_type is
- * removed from the kernel lists and can be freed or reused once this
- * function returns.
- *
- * This call sleeps to guarantee that no CPU is looking at the packet
- * type after return.
- */
-void dev_remove_offload(struct packet_offload *po)
-{
- __dev_remove_offload(po);
-
- synchronize_net();
-}
-EXPORT_SYMBOL(dev_remove_offload);
-
/*******************************************************************************
*
* Device Interface Subroutines
@@ -1272,15 +1188,15 @@ int dev_change_name(struct net_device *dev, const char *newname)
netdev_adjacent_rename_links(dev, oldname);
- write_lock_bh(&dev_base_lock);
+ write_lock(&dev_base_lock);
netdev_name_node_del(dev->name_node);
- write_unlock_bh(&dev_base_lock);
+ write_unlock(&dev_base_lock);
synchronize_rcu();
- write_lock_bh(&dev_base_lock);
+ write_lock(&dev_base_lock);
netdev_name_node_add(net, dev->name_node);
- write_unlock_bh(&dev_base_lock);
+ write_unlock(&dev_base_lock);
ret = call_netdevice_notifiers(NETDEV_CHANGENAME, dev);
ret = notifier_to_errno(ret);
@@ -1461,6 +1377,7 @@ static int __dev_open(struct net_device *dev, struct netlink_ext_ack *extack)
int ret;
ASSERT_RTNL();
+ dev_addr_check(dev);
if (!netif_device_present(dev)) {
/* may be detached because parent is runtime-suspended */
@@ -3315,40 +3232,6 @@ __be16 skb_network_protocol(struct sk_buff *skb, int *depth)
return __vlan_get_protocol(skb, type, depth);
}
-/**
- * skb_mac_gso_segment - mac layer segmentation handler.
- * @skb: buffer to segment
- * @features: features for the output path (see dev->features)
- */
-struct sk_buff *skb_mac_gso_segment(struct sk_buff *skb,
- netdev_features_t features)
-{
- struct sk_buff *segs = ERR_PTR(-EPROTONOSUPPORT);
- struct packet_offload *ptype;
- int vlan_depth = skb->mac_len;
- __be16 type = skb_network_protocol(skb, &vlan_depth);
-
- if (unlikely(!type))
- return ERR_PTR(-EINVAL);
-
- __skb_pull(skb, vlan_depth);
-
- rcu_read_lock();
- list_for_each_entry_rcu(ptype, &offload_base, list) {
- if (ptype->type == type && ptype->callbacks.gso_segment) {
- segs = ptype->callbacks.gso_segment(skb, features);
- break;
- }
- }
- rcu_read_unlock();
-
- __skb_push(skb, skb->data - skb_mac_header(skb));
-
- return segs;
-}
-EXPORT_SYMBOL(skb_mac_gso_segment);
-
-
/* openvswitch calls this on rx path, so we need a different check.
*/
static inline bool skb_needs_check(struct sk_buff *skb, bool tx_path)
@@ -3513,7 +3396,7 @@ static netdev_features_t gso_features_check(const struct sk_buff *skb,
{
u16 gso_segs = skb_shinfo(skb)->gso_segs;
- if (gso_segs > dev->gso_max_segs)
+ if (gso_segs > READ_ONCE(dev->gso_max_segs))
return features & ~NETIF_F_GSO_MASK;
if (!skb_shinfo(skb)->gso_type) {
@@ -3836,8 +3719,12 @@ static inline int __dev_xmit_skb(struct sk_buff *skb, struct Qdisc *q,
* separate lock before trying to get qdisc main lock.
* This permits qdisc->running owner to get the lock more
* often and dequeue packets faster.
+ * On PREEMPT_RT it is possible to preempt the qdisc owner during xmit
+ * and then other tasks will only enqueue packets. The packets will be
+ * sent after the qdisc owner is scheduled again. To prevent this
+ * scenario the task always serialize on the lock.
*/
- contended = qdisc_is_running(q);
+ contended = qdisc_is_running(q) || IS_ENABLED(CONFIG_PREEMPT_RT);
if (unlikely(contended))
spin_lock(&q->busylock);
@@ -4323,8 +4210,6 @@ int dev_weight_rx_bias __read_mostly = 1; /* bias for backlog weight */
int dev_weight_tx_bias __read_mostly = 1; /* bias for output_queue quota */
int dev_rx_weight __read_mostly = 64;
int dev_tx_weight __read_mostly = 64;
-/* Maximum number of GRO_NORMAL skbs to batch up for list-RX */
-int gro_normal_batch __read_mostly = 8;
/* Called with irq disabled */
static inline void ____napi_schedule(struct softnet_data *sd,
@@ -4827,7 +4712,7 @@ static u32 netif_receive_generic_xdp(struct sk_buff *skb,
case XDP_PASS:
break;
default:
- bpf_warn_invalid_xdp_action(act);
+ bpf_warn_invalid_xdp_action(skb->dev, xdp_prog, act);
fallthrough;
case XDP_ABORTED:
trace_xdp_exception(skb->dev, xdp_prog, act);
@@ -5014,7 +4899,8 @@ static __latent_entropy void net_tx_action(struct softirq_action *h)
if (likely(get_kfree_skb_cb(skb)->reason == SKB_REASON_CONSUMED))
trace_consume_skb(skb);
else
- trace_kfree_skb(skb, net_tx_action);
+ trace_kfree_skb(skb, net_tx_action,
+ SKB_DROP_REASON_NOT_SPECIFIED);
if (skb->fclone != SKB_FCLONE_UNAVAILABLE)
__kfree_skb(skb);
@@ -5667,7 +5553,7 @@ static int netif_receive_skb_internal(struct sk_buff *skb)
return ret;
}
-static void netif_receive_skb_list_internal(struct list_head *head)
+void netif_receive_skb_list_internal(struct list_head *head)
{
struct sk_buff *skb, *next;
struct list_head sublist;
@@ -5845,550 +5731,6 @@ static void flush_all_backlogs(void)
cpus_read_unlock();
}
-/* Pass the currently batched GRO_NORMAL SKBs up to the stack. */
-static void gro_normal_list(struct napi_struct *napi)
-{
- if (!napi->rx_count)
- return;
- netif_receive_skb_list_internal(&napi->rx_list);
- INIT_LIST_HEAD(&napi->rx_list);
- napi->rx_count = 0;
-}
-
-/* Queue one GRO_NORMAL SKB up for list processing. If batch size exceeded,
- * pass the whole batch up to the stack.
- */
-static void gro_normal_one(struct napi_struct *napi, struct sk_buff *skb, int segs)
-{
- list_add_tail(&skb->list, &napi->rx_list);
- napi->rx_count += segs;
- if (napi->rx_count >= gro_normal_batch)
- gro_normal_list(napi);
-}
-
-static void napi_gro_complete(struct napi_struct *napi, struct sk_buff *skb)
-{
- struct packet_offload *ptype;
- __be16 type = skb->protocol;
- struct list_head *head = &offload_base;
- int err = -ENOENT;
-
- BUILD_BUG_ON(sizeof(struct napi_gro_cb) > sizeof(skb->cb));
-
- if (NAPI_GRO_CB(skb)->count == 1) {
- skb_shinfo(skb)->gso_size = 0;
- goto out;
- }
-
- rcu_read_lock();
- list_for_each_entry_rcu(ptype, head, list) {
- if (ptype->type != type || !ptype->callbacks.gro_complete)
- continue;
-
- err = INDIRECT_CALL_INET(ptype->callbacks.gro_complete,
- ipv6_gro_complete, inet_gro_complete,
- skb, 0);
- break;
- }
- rcu_read_unlock();
-
- if (err) {
- WARN_ON(&ptype->list == head);
- kfree_skb(skb);
- return;
- }
-
-out:
- gro_normal_one(napi, skb, NAPI_GRO_CB(skb)->count);
-}
-
-static void __napi_gro_flush_chain(struct napi_struct *napi, u32 index,
- bool flush_old)
-{
- struct list_head *head = &napi->gro_hash[index].list;
- struct sk_buff *skb, *p;
-
- list_for_each_entry_safe_reverse(skb, p, head, list) {
- if (flush_old && NAPI_GRO_CB(skb)->age == jiffies)
- return;
- skb_list_del_init(skb);
- napi_gro_complete(napi, skb);
- napi->gro_hash[index].count--;
- }
-
- if (!napi->gro_hash[index].count)
- __clear_bit(index, &napi->gro_bitmask);
-}
-
-/* napi->gro_hash[].list contains packets ordered by age.
- * youngest packets at the head of it.
- * Complete skbs in reverse order to reduce latencies.
- */
-void napi_gro_flush(struct napi_struct *napi, bool flush_old)
-{
- unsigned long bitmask = napi->gro_bitmask;
- unsigned int i, base = ~0U;
-
- while ((i = ffs(bitmask)) != 0) {
- bitmask >>= i;
- base += i;
- __napi_gro_flush_chain(napi, base, flush_old);
- }
-}
-EXPORT_SYMBOL(napi_gro_flush);
-
-static void gro_list_prepare(const struct list_head *head,
- const struct sk_buff *skb)
-{
- unsigned int maclen = skb->dev->hard_header_len;
- u32 hash = skb_get_hash_raw(skb);
- struct sk_buff *p;
-
- list_for_each_entry(p, head, list) {
- unsigned long diffs;
-
- NAPI_GRO_CB(p)->flush = 0;
-
- if (hash != skb_get_hash_raw(p)) {
- NAPI_GRO_CB(p)->same_flow = 0;
- continue;
- }
-
- diffs = (unsigned long)p->dev ^ (unsigned long)skb->dev;
- diffs |= skb_vlan_tag_present(p) ^ skb_vlan_tag_present(skb);
- if (skb_vlan_tag_present(p))
- diffs |= skb_vlan_tag_get(p) ^ skb_vlan_tag_get(skb);
- diffs |= skb_metadata_differs(p, skb);
- if (maclen == ETH_HLEN)
- diffs |= compare_ether_header(skb_mac_header(p),
- skb_mac_header(skb));
- else if (!diffs)
- diffs = memcmp(skb_mac_header(p),
- skb_mac_header(skb),
- maclen);
-
- /* in most common scenarions 'slow_gro' is 0
- * otherwise we are already on some slower paths
- * either skip all the infrequent tests altogether or
- * avoid trying too hard to skip each of them individually
- */
- if (!diffs && unlikely(skb->slow_gro | p->slow_gro)) {
-#if IS_ENABLED(CONFIG_SKB_EXTENSIONS) && IS_ENABLED(CONFIG_NET_TC_SKB_EXT)
- struct tc_skb_ext *skb_ext;
- struct tc_skb_ext *p_ext;
-#endif
-
- diffs |= p->sk != skb->sk;
- diffs |= skb_metadata_dst_cmp(p, skb);
- diffs |= skb_get_nfct(p) ^ skb_get_nfct(skb);
-
-#if IS_ENABLED(CONFIG_SKB_EXTENSIONS) && IS_ENABLED(CONFIG_NET_TC_SKB_EXT)
- skb_ext = skb_ext_find(skb, TC_SKB_EXT);
- p_ext = skb_ext_find(p, TC_SKB_EXT);
-
- diffs |= (!!p_ext) ^ (!!skb_ext);
- if (!diffs && unlikely(skb_ext))
- diffs |= p_ext->chain ^ skb_ext->chain;
-#endif
- }
-
- NAPI_GRO_CB(p)->same_flow = !diffs;
- }
-}
-
-static inline void skb_gro_reset_offset(struct sk_buff *skb, u32 nhoff)
-{
- const struct skb_shared_info *pinfo = skb_shinfo(skb);
- const skb_frag_t *frag0 = &pinfo->frags[0];
-
- NAPI_GRO_CB(skb)->data_offset = 0;
- NAPI_GRO_CB(skb)->frag0 = NULL;
- NAPI_GRO_CB(skb)->frag0_len = 0;
-
- if (!skb_headlen(skb) && pinfo->nr_frags &&
- !PageHighMem(skb_frag_page(frag0)) &&
- (!NET_IP_ALIGN || !((skb_frag_off(frag0) + nhoff) & 3))) {
- NAPI_GRO_CB(skb)->frag0 = skb_frag_address(frag0);
- NAPI_GRO_CB(skb)->frag0_len = min_t(unsigned int,
- skb_frag_size(frag0),
- skb->end - skb->tail);
- }
-}
-
-static void gro_pull_from_frag0(struct sk_buff *skb, int grow)
-{
- struct skb_shared_info *pinfo = skb_shinfo(skb);
-
- BUG_ON(skb->end - skb->tail < grow);
-
- memcpy(skb_tail_pointer(skb), NAPI_GRO_CB(skb)->frag0, grow);
-
- skb->data_len -= grow;
- skb->tail += grow;
-
- skb_frag_off_add(&pinfo->frags[0], grow);
- skb_frag_size_sub(&pinfo->frags[0], grow);
-
- if (unlikely(!skb_frag_size(&pinfo->frags[0]))) {
- skb_frag_unref(skb, 0);
- memmove(pinfo->frags, pinfo->frags + 1,
- --pinfo->nr_frags * sizeof(pinfo->frags[0]));
- }
-}
-
-static void gro_flush_oldest(struct napi_struct *napi, struct list_head *head)
-{
- struct sk_buff *oldest;
-
- oldest = list_last_entry(head, struct sk_buff, list);
-
- /* We are called with head length >= MAX_GRO_SKBS, so this is
- * impossible.
- */
- if (WARN_ON_ONCE(!oldest))
- return;
-
- /* Do not adjust napi->gro_hash[].count, caller is adding a new
- * SKB to the chain.
- */
- skb_list_del_init(oldest);
- napi_gro_complete(napi, oldest);
-}
-
-static enum gro_result dev_gro_receive(struct napi_struct *napi, struct sk_buff *skb)
-{
- u32 bucket = skb_get_hash_raw(skb) & (GRO_HASH_BUCKETS - 1);
- struct gro_list *gro_list = &napi->gro_hash[bucket];
- struct list_head *head = &offload_base;
- struct packet_offload *ptype;
- __be16 type = skb->protocol;
- struct sk_buff *pp = NULL;
- enum gro_result ret;
- int same_flow;
- int grow;
-
- if (netif_elide_gro(skb->dev))
- goto normal;
-
- gro_list_prepare(&gro_list->list, skb);
-
- rcu_read_lock();
- list_for_each_entry_rcu(ptype, head, list) {
- if (ptype->type != type || !ptype->callbacks.gro_receive)
- continue;
-
- skb_set_network_header(skb, skb_gro_offset(skb));
- skb_reset_mac_len(skb);
- NAPI_GRO_CB(skb)->same_flow = 0;
- NAPI_GRO_CB(skb)->flush = skb_is_gso(skb) || skb_has_frag_list(skb);
- NAPI_GRO_CB(skb)->free = 0;
- NAPI_GRO_CB(skb)->encap_mark = 0;
- NAPI_GRO_CB(skb)->recursion_counter = 0;
- NAPI_GRO_CB(skb)->is_fou = 0;
- NAPI_GRO_CB(skb)->is_atomic = 1;
- NAPI_GRO_CB(skb)->gro_remcsum_start = 0;
-
- /* Setup for GRO checksum validation */
- switch (skb->ip_summed) {
- case CHECKSUM_COMPLETE:
- NAPI_GRO_CB(skb)->csum = skb->csum;
- NAPI_GRO_CB(skb)->csum_valid = 1;
- NAPI_GRO_CB(skb)->csum_cnt = 0;
- break;
- case CHECKSUM_UNNECESSARY:
- NAPI_GRO_CB(skb)->csum_cnt = skb->csum_level + 1;
- NAPI_GRO_CB(skb)->csum_valid = 0;
- break;
- default:
- NAPI_GRO_CB(skb)->csum_cnt = 0;
- NAPI_GRO_CB(skb)->csum_valid = 0;
- }
-
- pp = INDIRECT_CALL_INET(ptype->callbacks.gro_receive,
- ipv6_gro_receive, inet_gro_receive,
- &gro_list->list, skb);
- break;
- }
- rcu_read_unlock();
-
- if (&ptype->list == head)
- goto normal;
-
- if (PTR_ERR(pp) == -EINPROGRESS) {
- ret = GRO_CONSUMED;
- goto ok;
- }
-
- same_flow = NAPI_GRO_CB(skb)->same_flow;
- ret = NAPI_GRO_CB(skb)->free ? GRO_MERGED_FREE : GRO_MERGED;
-
- if (pp) {
- skb_list_del_init(pp);
- napi_gro_complete(napi, pp);
- gro_list->count--;
- }
-
- if (same_flow)
- goto ok;
-
- if (NAPI_GRO_CB(skb)->flush)
- goto normal;
-
- if (unlikely(gro_list->count >= MAX_GRO_SKBS))
- gro_flush_oldest(napi, &gro_list->list);
- else
- gro_list->count++;
-
- NAPI_GRO_CB(skb)->count = 1;
- NAPI_GRO_CB(skb)->age = jiffies;
- NAPI_GRO_CB(skb)->last = skb;
- skb_shinfo(skb)->gso_size = skb_gro_len(skb);
- list_add(&skb->list, &gro_list->list);
- ret = GRO_HELD;
-
-pull:
- grow = skb_gro_offset(skb) - skb_headlen(skb);
- if (grow > 0)
- gro_pull_from_frag0(skb, grow);
-ok:
- if (gro_list->count) {
- if (!test_bit(bucket, &napi->gro_bitmask))
- __set_bit(bucket, &napi->gro_bitmask);
- } else if (test_bit(bucket, &napi->gro_bitmask)) {
- __clear_bit(bucket, &napi->gro_bitmask);
- }
-
- return ret;
-
-normal:
- ret = GRO_NORMAL;
- goto pull;
-}
-
-struct packet_offload *gro_find_receive_by_type(__be16 type)
-{
- struct list_head *offload_head = &offload_base;
- struct packet_offload *ptype;
-
- list_for_each_entry_rcu(ptype, offload_head, list) {
- if (ptype->type != type || !ptype->callbacks.gro_receive)
- continue;
- return ptype;
- }
- return NULL;
-}
-EXPORT_SYMBOL(gro_find_receive_by_type);
-
-struct packet_offload *gro_find_complete_by_type(__be16 type)
-{
- struct list_head *offload_head = &offload_base;
- struct packet_offload *ptype;
-
- list_for_each_entry_rcu(ptype, offload_head, list) {
- if (ptype->type != type || !ptype->callbacks.gro_complete)
- continue;
- return ptype;
- }
- return NULL;
-}
-EXPORT_SYMBOL(gro_find_complete_by_type);
-
-static gro_result_t napi_skb_finish(struct napi_struct *napi,
- struct sk_buff *skb,
- gro_result_t ret)
-{
- switch (ret) {
- case GRO_NORMAL:
- gro_normal_one(napi, skb, 1);
- break;
-
- case GRO_MERGED_FREE:
- if (NAPI_GRO_CB(skb)->free == NAPI_GRO_FREE_STOLEN_HEAD)
- napi_skb_free_stolen_head(skb);
- else if (skb->fclone != SKB_FCLONE_UNAVAILABLE)
- __kfree_skb(skb);
- else
- __kfree_skb_defer(skb);
- break;
-
- case GRO_HELD:
- case GRO_MERGED:
- case GRO_CONSUMED:
- break;
- }
-
- return ret;
-}
-
-gro_result_t napi_gro_receive(struct napi_struct *napi, struct sk_buff *skb)
-{
- gro_result_t ret;
-
- skb_mark_napi_id(skb, napi);
- trace_napi_gro_receive_entry(skb);
-
- skb_gro_reset_offset(skb, 0);
-
- ret = napi_skb_finish(napi, skb, dev_gro_receive(napi, skb));
- trace_napi_gro_receive_exit(ret);
-
- return ret;
-}
-EXPORT_SYMBOL(napi_gro_receive);
-
-static void napi_reuse_skb(struct napi_struct *napi, struct sk_buff *skb)
-{
- if (unlikely(skb->pfmemalloc)) {
- consume_skb(skb);
- return;
- }
- __skb_pull(skb, skb_headlen(skb));
- /* restore the reserve we had after netdev_alloc_skb_ip_align() */
- skb_reserve(skb, NET_SKB_PAD + NET_IP_ALIGN - skb_headroom(skb));
- __vlan_hwaccel_clear_tag(skb);
- skb->dev = napi->dev;
- skb->skb_iif = 0;
-
- /* eth_type_trans() assumes pkt_type is PACKET_HOST */
- skb->pkt_type = PACKET_HOST;
-
- skb->encapsulation = 0;
- skb_shinfo(skb)->gso_type = 0;
- skb->truesize = SKB_TRUESIZE(skb_end_offset(skb));
- if (unlikely(skb->slow_gro)) {
- skb_orphan(skb);
- skb_ext_reset(skb);
- nf_reset_ct(skb);
- skb->slow_gro = 0;
- }
-
- napi->skb = skb;
-}
-
-struct sk_buff *napi_get_frags(struct napi_struct *napi)
-{
- struct sk_buff *skb = napi->skb;
-
- if (!skb) {
- skb = napi_alloc_skb(napi, GRO_MAX_HEAD);
- if (skb) {
- napi->skb = skb;
- skb_mark_napi_id(skb, napi);
- }
- }
- return skb;
-}
-EXPORT_SYMBOL(napi_get_frags);
-
-static gro_result_t napi_frags_finish(struct napi_struct *napi,
- struct sk_buff *skb,
- gro_result_t ret)
-{
- switch (ret) {
- case GRO_NORMAL:
- case GRO_HELD:
- __skb_push(skb, ETH_HLEN);
- skb->protocol = eth_type_trans(skb, skb->dev);
- if (ret == GRO_NORMAL)
- gro_normal_one(napi, skb, 1);
- break;
-
- case GRO_MERGED_FREE:
- if (NAPI_GRO_CB(skb)->free == NAPI_GRO_FREE_STOLEN_HEAD)
- napi_skb_free_stolen_head(skb);
- else
- napi_reuse_skb(napi, skb);
- break;
-
- case GRO_MERGED:
- case GRO_CONSUMED:
- break;
- }
-
- return ret;
-}
-
-/* Upper GRO stack assumes network header starts at gro_offset=0
- * Drivers could call both napi_gro_frags() and napi_gro_receive()
- * We copy ethernet header into skb->data to have a common layout.
- */
-static struct sk_buff *napi_frags_skb(struct napi_struct *napi)
-{
- struct sk_buff *skb = napi->skb;
- const struct ethhdr *eth;
- unsigned int hlen = sizeof(*eth);
-
- napi->skb = NULL;
-
- skb_reset_mac_header(skb);
- skb_gro_reset_offset(skb, hlen);
-
- if (unlikely(skb_gro_header_hard(skb, hlen))) {
- eth = skb_gro_header_slow(skb, hlen, 0);
- if (unlikely(!eth)) {
- net_warn_ratelimited("%s: dropping impossible skb from %s\n",
- __func__, napi->dev->name);
- napi_reuse_skb(napi, skb);
- return NULL;
- }
- } else {
- eth = (const struct ethhdr *)skb->data;
- gro_pull_from_frag0(skb, hlen);
- NAPI_GRO_CB(skb)->frag0 += hlen;
- NAPI_GRO_CB(skb)->frag0_len -= hlen;
- }
- __skb_pull(skb, hlen);
-
- /*
- * This works because the only protocols we care about don't require
- * special handling.
- * We'll fix it up properly in napi_frags_finish()
- */
- skb->protocol = eth->h_proto;
-
- return skb;
-}
-
-gro_result_t napi_gro_frags(struct napi_struct *napi)
-{
- gro_result_t ret;
- struct sk_buff *skb = napi_frags_skb(napi);
-
- trace_napi_gro_frags_entry(skb);
-
- ret = napi_frags_finish(napi, skb, dev_gro_receive(napi, skb));
- trace_napi_gro_frags_exit(ret);
-
- return ret;
-}
-EXPORT_SYMBOL(napi_gro_frags);
-
-/* Compute the checksum from gro_offset and return the folded value
- * after adding in any pseudo checksum.
- */
-__sum16 __skb_gro_checksum_complete(struct sk_buff *skb)
-{
- __wsum wsum;
- __sum16 sum;
-
- wsum = skb_checksum(skb, skb_gro_offset(skb), skb_gro_len(skb), 0);
-
- /* NAPI_GRO_CB(skb)->csum holds pseudo checksum */
- sum = csum_fold(csum_add(NAPI_GRO_CB(skb)->csum, wsum));
- /* See comments in __skb_checksum_complete(). */
- if (likely(!sum)) {
- if (unlikely(skb->ip_summed == CHECKSUM_COMPLETE) &&
- !skb->csum_complete_sw)
- netdev_rx_csum_fault(skb->dev, skb);
- }
-
- NAPI_GRO_CB(skb)->csum = wsum;
- NAPI_GRO_CB(skb)->csum_valid = 1;
-
- return sum;
-}
-EXPORT_SYMBOL(__skb_gro_checksum_complete);
-
static void net_rps_send_ipi(struct softnet_data *remsd)
{
#ifdef CONFIG_RPS
@@ -7200,6 +6542,7 @@ static __latent_entropy void net_rx_action(struct softirq_action *h)
struct netdev_adjacent {
struct net_device *dev;
+ netdevice_tracker dev_tracker;
/* upper master flag, there can only be one master device per list */
bool master;
@@ -7964,7 +7307,7 @@ static int __netdev_adjacent_dev_insert(struct net_device *dev,
adj->ref_nr = 1;
adj->private = private;
adj->ignore = false;
- dev_hold(adj_dev);
+ dev_hold_track(adj_dev, &adj->dev_tracker, GFP_KERNEL);
pr_debug("Insert adjacency: dev %s adj_dev %s adj->ref_nr %d; dev_hold on %s\n",
dev->name, adj_dev->name, adj->ref_nr, adj_dev->name);
@@ -7993,8 +7336,8 @@ static int __netdev_adjacent_dev_insert(struct net_device *dev,
if (netdev_adjacent_is_neigh_list(dev, adj_dev, dev_list))
netdev_adjacent_sysfs_del(dev, adj_dev->name, dev_list);
free_adj:
+ dev_put_track(adj_dev, &adj->dev_tracker);
kfree(adj);
- dev_put(adj_dev);
return ret;
}
@@ -8035,7 +7378,7 @@ static void __netdev_adjacent_dev_remove(struct net_device *dev,
list_del_rcu(&adj->list);
pr_debug("adjacency: dev_put for %s, because link removed from %s to %s\n",
adj_dev->name, dev->name, adj_dev->name);
- dev_put(adj_dev);
+ dev_put_track(adj_dev, &adj->dev_tracker);
kfree_rcu(adj, rcu);
}
@@ -9224,35 +8567,17 @@ bool netdev_port_same_parent_id(struct net_device *a, struct net_device *b)
EXPORT_SYMBOL(netdev_port_same_parent_id);
/**
- * dev_change_proto_down - update protocol port state information
+ * dev_change_proto_down - set carrier according to proto_down.
+ *
* @dev: device
* @proto_down: new value
- *
- * This info can be used by switch drivers to set the phys state of the
- * port.
*/
int dev_change_proto_down(struct net_device *dev, bool proto_down)
{
- const struct net_device_ops *ops = dev->netdev_ops;
-
- if (!ops->ndo_change_proto_down)
+ if (!(dev->priv_flags & IFF_CHANGE_PROTO_DOWN))
return -EOPNOTSUPP;
if (!netif_device_present(dev))
return -ENODEV;
- return ops->ndo_change_proto_down(dev, proto_down);
-}
-EXPORT_SYMBOL(dev_change_proto_down);
-
-/**
- * dev_change_proto_down_generic - generic implementation for
- * ndo_change_proto_down that sets carrier according to
- * proto_down.
- *
- * @dev: device
- * @proto_down: new value
- */
-int dev_change_proto_down_generic(struct net_device *dev, bool proto_down)
-{
if (proto_down)
netif_carrier_off(dev);
else
@@ -9260,7 +8585,7 @@ int dev_change_proto_down_generic(struct net_device *dev, bool proto_down)
dev->proto_down = proto_down;
return 0;
}
-EXPORT_SYMBOL(dev_change_proto_down_generic);
+EXPORT_SYMBOL(dev_change_proto_down);
/**
* dev_change_proto_down_reason - proto down reason
@@ -10545,6 +9870,7 @@ static void netdev_wait_allrefs(struct net_device *dev)
netdev_unregister_timeout_secs * HZ)) {
pr_emerg("unregister_netdevice: waiting for %s to become free. Usage count = %d\n",
dev->name, refcnt);
+ ref_tracker_dir_print(&dev->refcnt_tracker, 10);
warning_time = jiffies;
}
}
@@ -10835,6 +10161,7 @@ struct net_device *alloc_netdev_mqs(int sizeof_priv, const char *name,
dev = PTR_ALIGN(p, NETDEV_ALIGN);
dev->padded = (char *)dev - (char *)p;
+ ref_tracker_dir_init(&dev->refcnt_tracker, 128);
#ifdef CONFIG_PCPU_DEV_REFCNT
dev->pcpu_refcnt = alloc_percpu(int);
if (!dev->pcpu_refcnt)
@@ -10854,6 +10181,7 @@ struct net_device *alloc_netdev_mqs(int sizeof_priv, const char *name,
dev->gso_max_size = GSO_MAX_SIZE;
dev->gso_max_segs = GSO_MAX_SEGS;
+ dev->gro_max_size = GRO_MAX_SIZE;
dev->upper_level = 1;
dev->lower_level = 1;
#ifdef CONFIG_LOCKDEP
@@ -10951,6 +10279,7 @@ void free_netdev(struct net_device *dev)
list_for_each_entry_safe(p, n, &dev->napi_list, dev_list)
netif_napi_del(p);
+ ref_tracker_dir_exit(&dev->refcnt_tracker);
#ifdef CONFIG_PCPU_DEV_REFCNT
free_percpu(dev->pcpu_refcnt);
dev->pcpu_refcnt = NULL;
@@ -11643,8 +10972,6 @@ static int __init net_dev_init(void)
for (i = 0; i < PTYPE_HASH_SIZE; i++)
INIT_LIST_HEAD(&ptype_base[i]);
- INIT_LIST_HEAD(&offload_base);
-
if (register_pernet_subsys(&netdev_net_ops))
goto out;
diff --git a/net/core/dev_addr_lists.c b/net/core/dev_addr_lists.c
index f0cb383..bead38c 100644
--- a/net/core/dev_addr_lists.c
+++ b/net/core/dev_addr_lists.c
@@ -16,6 +16,35 @@
* General list handling functions
*/
+static int __hw_addr_insert(struct netdev_hw_addr_list *list,
+ struct netdev_hw_addr *new, int addr_len)
+{
+ struct rb_node **ins_point = &list->tree.rb_node, *parent = NULL;
+ struct netdev_hw_addr *ha;
+
+ while (*ins_point) {
+ int diff;
+
+ ha = rb_entry(*ins_point, struct netdev_hw_addr, node);
+ diff = memcmp(new->addr, ha->addr, addr_len);
+ if (diff == 0)
+ diff = memcmp(&new->type, &ha->type, sizeof(new->type));
+
+ parent = *ins_point;
+ if (diff < 0)
+ ins_point = &parent->rb_left;
+ else if (diff > 0)
+ ins_point = &parent->rb_right;
+ else
+ return -EEXIST;
+ }
+
+ rb_link_node_rcu(&new->node, parent, ins_point);
+ rb_insert_color(&new->node, &list->tree);
+
+ return 0;
+}
+
static struct netdev_hw_addr*
__hw_addr_create(const unsigned char *addr, int addr_len,
unsigned char addr_type, bool global, bool sync)
@@ -50,11 +79,6 @@ static int __hw_addr_add_ex(struct netdev_hw_addr_list *list,
if (addr_len > MAX_ADDR_LEN)
return -EINVAL;
- ha = list_first_entry(&list->list, struct netdev_hw_addr, list);
- if (ha && !memcmp(addr, ha->addr, addr_len) &&
- (!addr_type || addr_type == ha->type))
- goto found_it;
-
while (*ins_point) {
int diff;
@@ -69,7 +93,6 @@ static int __hw_addr_add_ex(struct netdev_hw_addr_list *list,
} else if (diff > 0) {
ins_point = &parent->rb_right;
} else {
-found_it:
if (exclusive)
return -EEXIST;
if (global) {
@@ -94,16 +117,8 @@ static int __hw_addr_add_ex(struct netdev_hw_addr_list *list,
if (!ha)
return -ENOMEM;
- /* The first address in dev->dev_addrs is pointed to by dev->dev_addr
- * and mutated freely by device drivers and netdev ops, so if we insert
- * it into the tree we'll end up with an invalid rbtree.
- */
- if (list->count > 0) {
- rb_link_node(&ha->node, parent, ins_point);
- rb_insert_color(&ha->node, &list->tree);
- } else {
- RB_CLEAR_NODE(&ha->node);
- }
+ rb_link_node(&ha->node, parent, ins_point);
+ rb_insert_color(&ha->node, &list->tree);
list_add_tail_rcu(&ha->list, &list->list);
list->count++;
@@ -138,8 +153,7 @@ static int __hw_addr_del_entry(struct netdev_hw_addr_list *list,
if (--ha->refcount)
return 0;
- if (!RB_EMPTY_NODE(&ha->node))
- rb_erase(&ha->node, &list->tree);
+ rb_erase(&ha->node, &list->tree);
list_del_rcu(&ha->list);
kfree_rcu(ha, rcu_head);
@@ -151,18 +165,8 @@ static struct netdev_hw_addr *__hw_addr_lookup(struct netdev_hw_addr_list *list,
const unsigned char *addr, int addr_len,
unsigned char addr_type)
{
- struct netdev_hw_addr *ha;
struct rb_node *node;
- /* The first address isn't inserted into the tree because in the dev->dev_addrs
- * list it's the address pointed to by dev->dev_addr which is freely mutated
- * in place, so we need to check it separately.
- */
- ha = list_first_entry(&list->list, struct netdev_hw_addr, list);
- if (ha && !memcmp(addr, ha->addr, addr_len) &&
- (!addr_type || addr_type == ha->type))
- return ha;
-
node = list->tree.rb_node;
while (node) {
@@ -498,6 +502,21 @@ EXPORT_SYMBOL(__hw_addr_init);
* Device addresses handling functions
*/
+/* Check that netdev->dev_addr is not written to directly as this would
+ * break the rbtree layout. All changes should go thru dev_addr_set() and co.
+ * Remove this check in mid-2024.
+ */
+void dev_addr_check(struct net_device *dev)
+{
+ if (!memcmp(dev->dev_addr, dev->dev_addr_shadow, MAX_ADDR_LEN))
+ return;
+
+ netdev_warn(dev, "Current addr: %*ph\n", MAX_ADDR_LEN, dev->dev_addr);
+ netdev_warn(dev, "Expected addr: %*ph\n",
+ MAX_ADDR_LEN, dev->dev_addr_shadow);
+ netdev_WARN(dev, "Incorrect netdev->dev_addr\n");
+}
+
/**
* dev_addr_flush - Flush device address list
* @dev: device
@@ -509,11 +528,11 @@ EXPORT_SYMBOL(__hw_addr_init);
void dev_addr_flush(struct net_device *dev)
{
/* rtnl_mutex must be held here */
+ dev_addr_check(dev);
__hw_addr_flush(&dev->dev_addrs);
dev->dev_addr = NULL;
}
-EXPORT_SYMBOL(dev_addr_flush);
/**
* dev_addr_init - Init device address list
@@ -547,7 +566,21 @@ int dev_addr_init(struct net_device *dev)
}
return err;
}
-EXPORT_SYMBOL(dev_addr_init);
+
+void dev_addr_mod(struct net_device *dev, unsigned int offset,
+ const void *addr, size_t len)
+{
+ struct netdev_hw_addr *ha;
+
+ dev_addr_check(dev);
+
+ ha = container_of(dev->dev_addr, struct netdev_hw_addr, addr[0]);
+ rb_erase(&ha->node, &dev->dev_addrs.tree);
+ memcpy(&ha->addr[offset], addr, len);
+ memcpy(&dev->dev_addr_shadow[offset], addr, len);
+ WARN_ON(__hw_addr_insert(&dev->dev_addrs, ha, dev->addr_len));
+}
+EXPORT_SYMBOL(dev_addr_mod);
/**
* dev_addr_add - Add a device address
diff --git a/net/core/dev_addr_lists_test.c b/net/core/dev_addr_lists_test.c
new file mode 100644
index 0000000..049cfbc
--- /dev/null
+++ b/net/core/dev_addr_lists_test.c
@@ -0,0 +1,236 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+
+#include <kunit/test.h>
+#include <linux/etherdevice.h>
+#include <linux/netdevice.h>
+#include <linux/rtnetlink.h>
+
+static const struct net_device_ops dummy_netdev_ops = {
+};
+
+struct dev_addr_test_priv {
+ u32 addr_seen;
+};
+
+static int dev_addr_test_sync(struct net_device *netdev, const unsigned char *a)
+{
+ struct dev_addr_test_priv *datp = netdev_priv(netdev);
+
+ if (a[0] < 31 && !memchr_inv(a, a[0], ETH_ALEN))
+ datp->addr_seen |= 1 << a[0];
+ return 0;
+}
+
+static int dev_addr_test_unsync(struct net_device *netdev,
+ const unsigned char *a)
+{
+ struct dev_addr_test_priv *datp = netdev_priv(netdev);
+
+ if (a[0] < 31 && !memchr_inv(a, a[0], ETH_ALEN))
+ datp->addr_seen &= ~(1 << a[0]);
+ return 0;
+}
+
+static int dev_addr_test_init(struct kunit *test)
+{
+ struct dev_addr_test_priv *datp;
+ struct net_device *netdev;
+ int err;
+
+ netdev = alloc_etherdev(sizeof(*datp));
+ KUNIT_ASSERT_TRUE(test, !!netdev);
+
+ test->priv = netdev;
+ netdev->netdev_ops = &dummy_netdev_ops;
+
+ err = register_netdev(netdev);
+ if (err) {
+ free_netdev(netdev);
+ KUNIT_FAIL(test, "Can't register netdev %d", err);
+ }
+
+ rtnl_lock();
+ return 0;
+}
+
+static void dev_addr_test_exit(struct kunit *test)
+{
+ struct net_device *netdev = test->priv;
+
+ rtnl_unlock();
+ unregister_netdev(netdev);
+ free_netdev(netdev);
+}
+
+static void dev_addr_test_basic(struct kunit *test)
+{
+ struct net_device *netdev = test->priv;
+ u8 addr[ETH_ALEN];
+
+ KUNIT_EXPECT_TRUE(test, !!netdev->dev_addr);
+
+ memset(addr, 2, sizeof(addr));
+ eth_hw_addr_set(netdev, addr);
+ KUNIT_EXPECT_EQ(test, 0, memcmp(netdev->dev_addr, addr, sizeof(addr)));
+
+ memset(addr, 3, sizeof(addr));
+ dev_addr_set(netdev, addr);
+ KUNIT_EXPECT_EQ(test, 0, memcmp(netdev->dev_addr, addr, sizeof(addr)));
+}
+
+static void dev_addr_test_sync_one(struct kunit *test)
+{
+ struct net_device *netdev = test->priv;
+ struct dev_addr_test_priv *datp;
+ u8 addr[ETH_ALEN];
+
+ datp = netdev_priv(netdev);
+
+ memset(addr, 1, sizeof(addr));
+ eth_hw_addr_set(netdev, addr);
+
+ __hw_addr_sync_dev(&netdev->dev_addrs, netdev, dev_addr_test_sync,
+ dev_addr_test_unsync);
+ KUNIT_EXPECT_EQ(test, 2, datp->addr_seen);
+
+ memset(addr, 2, sizeof(addr));
+ eth_hw_addr_set(netdev, addr);
+
+ datp->addr_seen = 0;
+ __hw_addr_sync_dev(&netdev->dev_addrs, netdev, dev_addr_test_sync,
+ dev_addr_test_unsync);
+ /* It's not going to sync anything because the main address is
+ * considered synced and we overwrite in place.
+ */
+ KUNIT_EXPECT_EQ(test, 0, datp->addr_seen);
+}
+
+static void dev_addr_test_add_del(struct kunit *test)
+{
+ struct net_device *netdev = test->priv;
+ struct dev_addr_test_priv *datp;
+ u8 addr[ETH_ALEN];
+ int i;
+
+ datp = netdev_priv(netdev);
+
+ for (i = 1; i < 4; i++) {
+ memset(addr, i, sizeof(addr));
+ KUNIT_EXPECT_EQ(test, 0, dev_addr_add(netdev, addr,
+ NETDEV_HW_ADDR_T_LAN));
+ }
+ /* Add 3 again */
+ KUNIT_EXPECT_EQ(test, 0, dev_addr_add(netdev, addr,
+ NETDEV_HW_ADDR_T_LAN));
+
+ __hw_addr_sync_dev(&netdev->dev_addrs, netdev, dev_addr_test_sync,
+ dev_addr_test_unsync);
+ KUNIT_EXPECT_EQ(test, 0xf, datp->addr_seen);
+
+ KUNIT_EXPECT_EQ(test, 0, dev_addr_del(netdev, addr,
+ NETDEV_HW_ADDR_T_LAN));
+
+ __hw_addr_sync_dev(&netdev->dev_addrs, netdev, dev_addr_test_sync,
+ dev_addr_test_unsync);
+ KUNIT_EXPECT_EQ(test, 0xf, datp->addr_seen);
+
+ for (i = 1; i < 4; i++) {
+ memset(addr, i, sizeof(addr));
+ KUNIT_EXPECT_EQ(test, 0, dev_addr_del(netdev, addr,
+ NETDEV_HW_ADDR_T_LAN));
+ }
+
+ __hw_addr_sync_dev(&netdev->dev_addrs, netdev, dev_addr_test_sync,
+ dev_addr_test_unsync);
+ KUNIT_EXPECT_EQ(test, 1, datp->addr_seen);
+}
+
+static void dev_addr_test_del_main(struct kunit *test)
+{
+ struct net_device *netdev = test->priv;
+ u8 addr[ETH_ALEN];
+
+ memset(addr, 1, sizeof(addr));
+ eth_hw_addr_set(netdev, addr);
+
+ KUNIT_EXPECT_EQ(test, -ENOENT, dev_addr_del(netdev, addr,
+ NETDEV_HW_ADDR_T_LAN));
+ KUNIT_EXPECT_EQ(test, 0, dev_addr_add(netdev, addr,
+ NETDEV_HW_ADDR_T_LAN));
+ KUNIT_EXPECT_EQ(test, 0, dev_addr_del(netdev, addr,
+ NETDEV_HW_ADDR_T_LAN));
+ KUNIT_EXPECT_EQ(test, -ENOENT, dev_addr_del(netdev, addr,
+ NETDEV_HW_ADDR_T_LAN));
+}
+
+static void dev_addr_test_add_set(struct kunit *test)
+{
+ struct net_device *netdev = test->priv;
+ struct dev_addr_test_priv *datp;
+ u8 addr[ETH_ALEN];
+ int i;
+
+ datp = netdev_priv(netdev);
+
+ /* There is no external API like dev_addr_add_excl(),
+ * so shuffle the tree a little bit and exploit aliasing.
+ */
+ for (i = 1; i < 16; i++) {
+ memset(addr, i, sizeof(addr));
+ KUNIT_EXPECT_EQ(test, 0, dev_addr_add(netdev, addr,
+ NETDEV_HW_ADDR_T_LAN));
+ }
+
+ memset(addr, i, sizeof(addr));
+ eth_hw_addr_set(netdev, addr);
+ KUNIT_EXPECT_EQ(test, 0, dev_addr_add(netdev, addr,
+ NETDEV_HW_ADDR_T_LAN));
+ memset(addr, 0, sizeof(addr));
+ eth_hw_addr_set(netdev, addr);
+
+ __hw_addr_sync_dev(&netdev->dev_addrs, netdev, dev_addr_test_sync,
+ dev_addr_test_unsync);
+ KUNIT_EXPECT_EQ(test, 0xffff, datp->addr_seen);
+}
+
+static void dev_addr_test_add_excl(struct kunit *test)
+{
+ struct net_device *netdev = test->priv;
+ u8 addr[ETH_ALEN];
+ int i;
+
+ for (i = 0; i < 10; i++) {
+ memset(addr, i, sizeof(addr));
+ KUNIT_EXPECT_EQ(test, 0, dev_uc_add_excl(netdev, addr));
+ }
+ KUNIT_EXPECT_EQ(test, -EEXIST, dev_uc_add_excl(netdev, addr));
+
+ for (i = 0; i < 10; i += 2) {
+ memset(addr, i, sizeof(addr));
+ KUNIT_EXPECT_EQ(test, 0, dev_uc_del(netdev, addr));
+ }
+ for (i = 1; i < 10; i += 2) {
+ memset(addr, i, sizeof(addr));
+ KUNIT_EXPECT_EQ(test, -EEXIST, dev_uc_add_excl(netdev, addr));
+ }
+}
+
+static struct kunit_case dev_addr_test_cases[] = {
+ KUNIT_CASE(dev_addr_test_basic),
+ KUNIT_CASE(dev_addr_test_sync_one),
+ KUNIT_CASE(dev_addr_test_add_del),
+ KUNIT_CASE(dev_addr_test_del_main),
+ KUNIT_CASE(dev_addr_test_add_set),
+ KUNIT_CASE(dev_addr_test_add_excl),
+ {}
+};
+
+static struct kunit_suite dev_addr_test_suite = {
+ .name = "dev-addr-list-test",
+ .test_cases = dev_addr_test_cases,
+ .init = dev_addr_test_init,
+ .exit = dev_addr_test_exit,
+};
+kunit_test_suite(dev_addr_test_suite);
+
+MODULE_LICENSE("GPL");
diff --git a/net/core/dev_ioctl.c b/net/core/dev_ioctl.c
index cbab5fe..1b807d1 100644
--- a/net/core/dev_ioctl.c
+++ b/net/core/dev_ioctl.c
@@ -192,7 +192,7 @@ static int net_hwtstamp_validate(struct ifreq *ifr)
if (copy_from_user(&cfg, ifr->ifr_data, sizeof(cfg)))
return -EFAULT;
- if (cfg.flags) /* reserved for future extensions */
+ if (cfg.flags & ~HWTSTAMP_FLAG_MASK)
return -EINVAL;
tx_type = cfg.tx_type;
@@ -313,6 +313,7 @@ static int dev_ifsioc(struct net *net, struct ifreq *ifr, void __user *data,
int err;
struct net_device *dev = __dev_get_by_name(net, ifr->ifr_name);
const struct net_device_ops *ops;
+ netdevice_tracker dev_tracker;
if (!dev)
return -ENODEV;
@@ -381,10 +382,10 @@ static int dev_ifsioc(struct net *net, struct ifreq *ifr, void __user *data,
return -ENODEV;
if (!netif_is_bridge_master(dev))
return -EOPNOTSUPP;
- dev_hold(dev);
+ dev_hold_track(dev, &dev_tracker, GFP_KERNEL);
rtnl_unlock();
err = br_ioctl_call(net, netdev_priv(dev), cmd, ifr, NULL);
- dev_put(dev);
+ dev_put_track(dev, &dev_tracker);
rtnl_lock();
return err;
diff --git a/net/core/devlink.c b/net/core/devlink.c
index c06c9ba..fcd9f6d 100644
--- a/net/core/devlink.c
+++ b/net/core/devlink.c
@@ -7,6 +7,7 @@
* Copyright (c) 2016 Jiri Pirko <jiri@mellanox.com>
*/
+#include <linux/etherdevice.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/types.h>
@@ -69,6 +70,35 @@ struct devlink {
char priv[] __aligned(NETDEV_ALIGN);
};
+/**
+ * struct devlink_resource - devlink resource
+ * @name: name of the resource
+ * @id: id, per devlink instance
+ * @size: size of the resource
+ * @size_new: updated size of the resource, reload is needed
+ * @size_valid: valid in case the total size of the resource is valid
+ * including its children
+ * @parent: parent resource
+ * @size_params: size parameters
+ * @list: parent list
+ * @resource_list: list of child resources
+ * @occ_get: occupancy getter callback
+ * @occ_get_priv: occupancy getter callback priv
+ */
+struct devlink_resource {
+ const char *name;
+ u64 id;
+ u64 size;
+ u64 size_new;
+ bool size_valid;
+ struct devlink_resource *parent;
+ struct devlink_resource_size_params size_params;
+ struct list_head list;
+ struct list_head resource_list;
+ devlink_resource_occ_get_t *occ_get;
+ void *occ_get_priv;
+};
+
void *devlink_priv(struct devlink *devlink)
{
return &devlink->priv;
@@ -4432,6 +4462,21 @@ static const struct devlink_param devlink_param_generic[] = {
.name = DEVLINK_PARAM_GENERIC_ENABLE_VNET_NAME,
.type = DEVLINK_PARAM_GENERIC_ENABLE_VNET_TYPE,
},
+ {
+ .id = DEVLINK_PARAM_GENERIC_ID_ENABLE_IWARP,
+ .name = DEVLINK_PARAM_GENERIC_ENABLE_IWARP_NAME,
+ .type = DEVLINK_PARAM_GENERIC_ENABLE_IWARP_TYPE,
+ },
+ {
+ .id = DEVLINK_PARAM_GENERIC_ID_IO_EQ_SIZE,
+ .name = DEVLINK_PARAM_GENERIC_IO_EQ_SIZE_NAME,
+ .type = DEVLINK_PARAM_GENERIC_IO_EQ_SIZE_TYPE,
+ },
+ {
+ .id = DEVLINK_PARAM_GENERIC_ID_EVENT_EQ_SIZE,
+ .name = DEVLINK_PARAM_GENERIC_EVENT_EQ_SIZE_NAME,
+ .type = DEVLINK_PARAM_GENERIC_EVENT_EQ_SIZE_TYPE,
+ },
};
static int devlink_param_generic_verify(const struct devlink_param *param)
@@ -8840,8 +8885,6 @@ static const struct genl_small_ops devlink_nl_ops[] = {
GENL_DONT_VALIDATE_DUMP_STRICT,
.dumpit = devlink_nl_cmd_health_reporter_dump_get_dumpit,
.flags = GENL_ADMIN_PERM,
- .internal_flags = DEVLINK_NL_FLAG_NEED_DEVLINK_OR_PORT |
- DEVLINK_NL_FLAG_NO_LOCK,
},
{
.cmd = DEVLINK_CMD_HEALTH_REPORTER_DUMP_CLEAR,
@@ -9905,34 +9948,38 @@ int devlink_resource_register(struct devlink *devlink,
}
EXPORT_SYMBOL_GPL(devlink_resource_register);
+static void devlink_resource_unregister(struct devlink *devlink,
+ struct devlink_resource *resource)
+{
+ struct devlink_resource *tmp, *child_resource;
+
+ list_for_each_entry_safe(child_resource, tmp, &resource->resource_list,
+ list) {
+ devlink_resource_unregister(devlink, child_resource);
+ list_del(&child_resource->list);
+ kfree(child_resource);
+ }
+}
+
/**
* devlink_resources_unregister - free all resources
*
* @devlink: devlink
- * @resource: resource
*/
-void devlink_resources_unregister(struct devlink *devlink,
- struct devlink_resource *resource)
+void devlink_resources_unregister(struct devlink *devlink)
{
struct devlink_resource *tmp, *child_resource;
- struct list_head *resource_list;
- if (resource)
- resource_list = &resource->resource_list;
- else
- resource_list = &devlink->resource_list;
+ mutex_lock(&devlink->lock);
- if (!resource)
- mutex_lock(&devlink->lock);
-
- list_for_each_entry_safe(child_resource, tmp, resource_list, list) {
- devlink_resources_unregister(devlink, child_resource);
+ list_for_each_entry_safe(child_resource, tmp, &devlink->resource_list,
+ list) {
+ devlink_resource_unregister(devlink, child_resource);
list_del(&child_resource->list);
kfree(child_resource);
}
- if (!resource)
- mutex_unlock(&devlink->lock);
+ mutex_unlock(&devlink->lock);
}
EXPORT_SYMBOL_GPL(devlink_resources_unregister);
diff --git a/net/core/drop_monitor.c b/net/core/drop_monitor.c
index 49442ca..7b288a1 100644
--- a/net/core/drop_monitor.c
+++ b/net/core/drop_monitor.c
@@ -110,7 +110,8 @@ static u32 net_dm_queue_len = 1000;
struct net_dm_alert_ops {
void (*kfree_skb_probe)(void *ignore, struct sk_buff *skb,
- void *location);
+ void *location,
+ enum skb_drop_reason reason);
void (*napi_poll_probe)(void *ignore, struct napi_struct *napi,
int work, int budget);
void (*work_item_func)(struct work_struct *work);
@@ -262,7 +263,9 @@ static void trace_drop_common(struct sk_buff *skb, void *location)
spin_unlock_irqrestore(&data->lock, flags);
}
-static void trace_kfree_skb_hit(void *ignore, struct sk_buff *skb, void *location)
+static void trace_kfree_skb_hit(void *ignore, struct sk_buff *skb,
+ void *location,
+ enum skb_drop_reason reason)
{
trace_drop_common(skb, location);
}
@@ -490,7 +493,8 @@ static const struct net_dm_alert_ops net_dm_alert_summary_ops = {
static void net_dm_packet_trace_kfree_skb_hit(void *ignore,
struct sk_buff *skb,
- void *location)
+ void *location,
+ enum skb_drop_reason reason)
{
ktime_t tstamp = ktime_get_real();
struct per_cpu_dm_data *data;
@@ -850,7 +854,7 @@ net_dm_hw_metadata_copy(const struct devlink_trap_metadata *metadata)
}
hw_metadata->input_dev = metadata->input_dev;
- dev_hold(hw_metadata->input_dev);
+ dev_hold_track(hw_metadata->input_dev, &hw_metadata->dev_tracker, GFP_ATOMIC);
return hw_metadata;
@@ -864,9 +868,9 @@ net_dm_hw_metadata_copy(const struct devlink_trap_metadata *metadata)
}
static void
-net_dm_hw_metadata_free(const struct devlink_trap_metadata *hw_metadata)
+net_dm_hw_metadata_free(struct devlink_trap_metadata *hw_metadata)
{
- dev_put(hw_metadata->input_dev);
+ dev_put_track(hw_metadata->input_dev, &hw_metadata->dev_tracker);
kfree(hw_metadata->fa_cookie);
kfree(hw_metadata->trap_name);
kfree(hw_metadata->trap_group_name);
diff --git a/net/core/dst.c b/net/core/dst.c
index 497ef9b..d16c2c9 100644
--- a/net/core/dst.c
+++ b/net/core/dst.c
@@ -49,7 +49,7 @@ void dst_init(struct dst_entry *dst, struct dst_ops *ops,
unsigned short flags)
{
dst->dev = dev;
- dev_hold(dev);
+ dev_hold_track(dev, &dst->dev_tracker, GFP_ATOMIC);
dst->ops = ops;
dst_init_metrics(dst, dst_default_metrics.metrics, true);
dst->expires = 0UL;
@@ -117,7 +117,7 @@ struct dst_entry *dst_destroy(struct dst_entry * dst)
if (dst->ops->destroy)
dst->ops->destroy(dst);
- dev_put(dst->dev);
+ dev_put_track(dst->dev, &dst->dev_tracker);
lwtstate_put(dst->lwtstate);
@@ -159,8 +159,8 @@ void dst_dev_put(struct dst_entry *dst)
dst->input = dst_discard;
dst->output = dst_discard_out;
dst->dev = blackhole_netdev;
- dev_hold(dst->dev);
- dev_put(dev);
+ dev_replace_track(dev, blackhole_netdev, &dst->dev_tracker,
+ GFP_ATOMIC);
}
EXPORT_SYMBOL(dst_dev_put);
diff --git a/net/core/failover.c b/net/core/failover.c
index b5cd3c7..dcaa92a 100644
--- a/net/core/failover.c
+++ b/net/core/failover.c
@@ -252,7 +252,7 @@ struct failover *failover_register(struct net_device *dev,
return ERR_PTR(-ENOMEM);
rcu_assign_pointer(failover->ops, ops);
- dev_hold(dev);
+ dev_hold_track(dev, &failover->dev_tracker, GFP_KERNEL);
dev->priv_flags |= IFF_FAILOVER;
rcu_assign_pointer(failover->failover_dev, dev);
@@ -285,7 +285,7 @@ void failover_unregister(struct failover *failover)
failover_dev->name);
failover_dev->priv_flags &= ~IFF_FAILOVER;
- dev_put(failover_dev);
+ dev_put_track(failover_dev, &failover->dev_tracker);
spin_lock(&failover_lock);
list_del(&failover->list);
diff --git a/net/core/fib_rules.c b/net/core/fib_rules.c
index 1bb567a..7528222 100644
--- a/net/core/fib_rules.c
+++ b/net/core/fib_rules.c
@@ -750,6 +750,27 @@ static int rule_exists(struct fib_rules_ops *ops, struct fib_rule_hdr *frh,
return 0;
}
+static const struct nla_policy fib_rule_policy[FRA_MAX + 1] = {
+ [FRA_UNSPEC] = { .strict_start_type = FRA_DPORT_RANGE + 1 },
+ [FRA_IIFNAME] = { .type = NLA_STRING, .len = IFNAMSIZ - 1 },
+ [FRA_OIFNAME] = { .type = NLA_STRING, .len = IFNAMSIZ - 1 },
+ [FRA_PRIORITY] = { .type = NLA_U32 },
+ [FRA_FWMARK] = { .type = NLA_U32 },
+ [FRA_FLOW] = { .type = NLA_U32 },
+ [FRA_TUN_ID] = { .type = NLA_U64 },
+ [FRA_FWMASK] = { .type = NLA_U32 },
+ [FRA_TABLE] = { .type = NLA_U32 },
+ [FRA_SUPPRESS_PREFIXLEN] = { .type = NLA_U32 },
+ [FRA_SUPPRESS_IFGROUP] = { .type = NLA_U32 },
+ [FRA_GOTO] = { .type = NLA_U32 },
+ [FRA_L3MDEV] = { .type = NLA_U8 },
+ [FRA_UID_RANGE] = { .len = sizeof(struct fib_rule_uid_range) },
+ [FRA_PROTOCOL] = { .type = NLA_U8 },
+ [FRA_IP_PROTO] = { .type = NLA_U8 },
+ [FRA_SPORT_RANGE] = { .len = sizeof(struct fib_rule_port_range) },
+ [FRA_DPORT_RANGE] = { .len = sizeof(struct fib_rule_port_range) }
+};
+
int fib_nl_newrule(struct sk_buff *skb, struct nlmsghdr *nlh,
struct netlink_ext_ack *extack)
{
@@ -774,7 +795,7 @@ int fib_nl_newrule(struct sk_buff *skb, struct nlmsghdr *nlh,
}
err = nlmsg_parse_deprecated(nlh, sizeof(*frh), tb, FRA_MAX,
- ops->policy, extack);
+ fib_rule_policy, extack);
if (err < 0) {
NL_SET_ERR_MSG(extack, "Error parsing msg");
goto errout;
@@ -882,7 +903,7 @@ int fib_nl_delrule(struct sk_buff *skb, struct nlmsghdr *nlh,
}
err = nlmsg_parse_deprecated(nlh, sizeof(*frh), tb, FRA_MAX,
- ops->policy, extack);
+ fib_rule_policy, extack);
if (err < 0) {
NL_SET_ERR_MSG(extack, "Error parsing msg");
goto errout;
diff --git a/net/core/filter.c b/net/core/filter.c
index 6102f09..4603b7c 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -301,7 +301,7 @@ static u32 convert_skb_access(int skb_field, int dst_reg, int src_reg,
break;
case SKF_AD_PKTTYPE:
- *insn++ = BPF_LDX_MEM(BPF_B, dst_reg, src_reg, PKT_TYPE_OFFSET());
+ *insn++ = BPF_LDX_MEM(BPF_B, dst_reg, src_reg, PKT_TYPE_OFFSET);
*insn++ = BPF_ALU32_IMM(BPF_AND, dst_reg, PKT_TYPE_MAX);
#ifdef __BIG_ENDIAN_BITFIELD
*insn++ = BPF_ALU32_IMM(BPF_RSH, dst_reg, 5);
@@ -323,7 +323,7 @@ static u32 convert_skb_access(int skb_field, int dst_reg, int src_reg,
offsetof(struct sk_buff, vlan_tci));
break;
case SKF_AD_VLAN_TAG_PRESENT:
- *insn++ = BPF_LDX_MEM(BPF_B, dst_reg, src_reg, PKT_VLAN_PRESENT_OFFSET());
+ *insn++ = BPF_LDX_MEM(BPF_B, dst_reg, src_reg, PKT_VLAN_PRESENT_OFFSET);
if (PKT_VLAN_PRESENT_BIT)
*insn++ = BPF_ALU32_IMM(BPF_RSH, dst_reg, PKT_VLAN_PRESENT_BIT);
if (PKT_VLAN_PRESENT_BIT < 7)
@@ -1242,10 +1242,9 @@ static struct bpf_prog *bpf_migrate_filter(struct bpf_prog *fp)
int err, new_len, old_len = fp->len;
bool seen_ld_abs = false;
- /* We are free to overwrite insns et al right here as it
- * won't be used at this point in time anymore internally
- * after the migration to the internal BPF instruction
- * representation.
+ /* We are free to overwrite insns et al right here as it won't be used at
+ * this point in time anymore internally after the migration to the eBPF
+ * instruction representation.
*/
BUILD_BUG_ON(sizeof(struct sock_filter) !=
sizeof(struct bpf_insn));
@@ -1336,8 +1335,8 @@ static struct bpf_prog *bpf_prepare_filter(struct bpf_prog *fp,
*/
bpf_jit_compile(fp);
- /* JIT compiler couldn't process this filter, so do the
- * internal BPF translation for the optimized interpreter.
+ /* JIT compiler couldn't process this filter, so do the eBPF translation
+ * for the optimized interpreter.
*/
if (!fp->jited)
fp = bpf_migrate_filter(fp);
@@ -1713,7 +1712,7 @@ static const struct bpf_func_proto bpf_skb_store_bytes_proto = {
.ret_type = RET_INTEGER,
.arg1_type = ARG_PTR_TO_CTX,
.arg2_type = ARG_ANYTHING,
- .arg3_type = ARG_PTR_TO_MEM,
+ .arg3_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg4_type = ARG_CONST_SIZE,
.arg5_type = ARG_ANYTHING,
};
@@ -2018,9 +2017,9 @@ static const struct bpf_func_proto bpf_csum_diff_proto = {
.gpl_only = false,
.pkt_access = true,
.ret_type = RET_INTEGER,
- .arg1_type = ARG_PTR_TO_MEM_OR_NULL,
+ .arg1_type = ARG_PTR_TO_MEM | PTR_MAYBE_NULL | MEM_RDONLY,
.arg2_type = ARG_CONST_SIZE_OR_ZERO,
- .arg3_type = ARG_PTR_TO_MEM_OR_NULL,
+ .arg3_type = ARG_PTR_TO_MEM | PTR_MAYBE_NULL | MEM_RDONLY,
.arg4_type = ARG_CONST_SIZE_OR_ZERO,
.arg5_type = ARG_ANYTHING,
};
@@ -2541,7 +2540,7 @@ static const struct bpf_func_proto bpf_redirect_neigh_proto = {
.gpl_only = false,
.ret_type = RET_INTEGER,
.arg1_type = ARG_ANYTHING,
- .arg2_type = ARG_PTR_TO_MEM_OR_NULL,
+ .arg2_type = ARG_PTR_TO_MEM | PTR_MAYBE_NULL | MEM_RDONLY,
.arg3_type = ARG_CONST_SIZE_OR_ZERO,
.arg4_type = ARG_ANYTHING,
};
@@ -3958,10 +3957,35 @@ u32 xdp_master_redirect(struct xdp_buff *xdp)
}
EXPORT_SYMBOL_GPL(xdp_master_redirect);
-int xdp_do_redirect(struct net_device *dev, struct xdp_buff *xdp,
- struct bpf_prog *xdp_prog)
+static inline int __xdp_do_redirect_xsk(struct bpf_redirect_info *ri,
+ struct net_device *dev,
+ struct xdp_buff *xdp,
+ struct bpf_prog *xdp_prog)
{
- struct bpf_redirect_info *ri = this_cpu_ptr(&bpf_redirect_info);
+ enum bpf_map_type map_type = ri->map_type;
+ void *fwd = ri->tgt_value;
+ u32 map_id = ri->map_id;
+ int err;
+
+ ri->map_id = 0; /* Valid map id idr range: [1,INT_MAX[ */
+ ri->map_type = BPF_MAP_TYPE_UNSPEC;
+
+ err = __xsk_map_redirect(fwd, xdp);
+ if (unlikely(err))
+ goto err;
+
+ _trace_xdp_redirect_map(dev, xdp_prog, fwd, map_type, map_id, ri->tgt_index);
+ return 0;
+err:
+ _trace_xdp_redirect_map_err(dev, xdp_prog, fwd, map_type, map_id, ri->tgt_index, err);
+ return err;
+}
+
+static __always_inline int __xdp_do_redirect_frame(struct bpf_redirect_info *ri,
+ struct net_device *dev,
+ struct xdp_frame *xdpf,
+ struct bpf_prog *xdp_prog)
+{
enum bpf_map_type map_type = ri->map_type;
void *fwd = ri->tgt_value;
u32 map_id = ri->map_id;
@@ -3971,6 +3995,11 @@ int xdp_do_redirect(struct net_device *dev, struct xdp_buff *xdp,
ri->map_id = 0; /* Valid map id idr range: [1,INT_MAX[ */
ri->map_type = BPF_MAP_TYPE_UNSPEC;
+ if (unlikely(!xdpf)) {
+ err = -EOVERFLOW;
+ goto err;
+ }
+
switch (map_type) {
case BPF_MAP_TYPE_DEVMAP:
fallthrough;
@@ -3978,17 +4007,14 @@ int xdp_do_redirect(struct net_device *dev, struct xdp_buff *xdp,
map = READ_ONCE(ri->map);
if (unlikely(map)) {
WRITE_ONCE(ri->map, NULL);
- err = dev_map_enqueue_multi(xdp, dev, map,
+ err = dev_map_enqueue_multi(xdpf, dev, map,
ri->flags & BPF_F_EXCLUDE_INGRESS);
} else {
- err = dev_map_enqueue(fwd, xdp, dev);
+ err = dev_map_enqueue(fwd, xdpf, dev);
}
break;
case BPF_MAP_TYPE_CPUMAP:
- err = cpu_map_enqueue(fwd, xdp, dev);
- break;
- case BPF_MAP_TYPE_XSKMAP:
- err = __xsk_map_redirect(fwd, xdp);
+ err = cpu_map_enqueue(fwd, xdpf, dev);
break;
case BPF_MAP_TYPE_UNSPEC:
if (map_id == INT_MAX) {
@@ -3997,7 +4023,7 @@ int xdp_do_redirect(struct net_device *dev, struct xdp_buff *xdp,
err = -EINVAL;
break;
}
- err = dev_xdp_enqueue(fwd, xdp, dev);
+ err = dev_xdp_enqueue(fwd, xdpf, dev);
break;
}
fallthrough;
@@ -4014,8 +4040,34 @@ int xdp_do_redirect(struct net_device *dev, struct xdp_buff *xdp,
_trace_xdp_redirect_map_err(dev, xdp_prog, fwd, map_type, map_id, ri->tgt_index, err);
return err;
}
+
+int xdp_do_redirect(struct net_device *dev, struct xdp_buff *xdp,
+ struct bpf_prog *xdp_prog)
+{
+ struct bpf_redirect_info *ri = this_cpu_ptr(&bpf_redirect_info);
+ enum bpf_map_type map_type = ri->map_type;
+
+ if (map_type == BPF_MAP_TYPE_XSKMAP)
+ return __xdp_do_redirect_xsk(ri, dev, xdp, xdp_prog);
+
+ return __xdp_do_redirect_frame(ri, dev, xdp_convert_buff_to_frame(xdp),
+ xdp_prog);
+}
EXPORT_SYMBOL_GPL(xdp_do_redirect);
+int xdp_do_redirect_frame(struct net_device *dev, struct xdp_buff *xdp,
+ struct xdp_frame *xdpf, struct bpf_prog *xdp_prog)
+{
+ struct bpf_redirect_info *ri = this_cpu_ptr(&bpf_redirect_info);
+ enum bpf_map_type map_type = ri->map_type;
+
+ if (map_type == BPF_MAP_TYPE_XSKMAP)
+ return __xdp_do_redirect_xsk(ri, dev, xdp, xdp_prog);
+
+ return __xdp_do_redirect_frame(ri, dev, xdpf, xdp_prog);
+}
+EXPORT_SYMBOL_GPL(xdp_do_redirect_frame);
+
static int xdp_do_generic_redirect_map(struct net_device *dev,
struct sk_buff *skb,
struct xdp_buff *xdp,
@@ -4174,7 +4226,7 @@ static const struct bpf_func_proto bpf_skb_event_output_proto = {
.arg1_type = ARG_PTR_TO_CTX,
.arg2_type = ARG_CONST_MAP_PTR,
.arg3_type = ARG_ANYTHING,
- .arg4_type = ARG_PTR_TO_MEM,
+ .arg4_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg5_type = ARG_CONST_SIZE_OR_ZERO,
};
@@ -4188,7 +4240,7 @@ const struct bpf_func_proto bpf_skb_output_proto = {
.arg1_btf_id = &bpf_skb_output_btf_ids[0],
.arg2_type = ARG_CONST_MAP_PTR,
.arg3_type = ARG_ANYTHING,
- .arg4_type = ARG_PTR_TO_MEM,
+ .arg4_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg5_type = ARG_CONST_SIZE_OR_ZERO,
};
@@ -4371,7 +4423,7 @@ static const struct bpf_func_proto bpf_skb_set_tunnel_key_proto = {
.gpl_only = false,
.ret_type = RET_INTEGER,
.arg1_type = ARG_PTR_TO_CTX,
- .arg2_type = ARG_PTR_TO_MEM,
+ .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg3_type = ARG_CONST_SIZE,
.arg4_type = ARG_ANYTHING,
};
@@ -4397,7 +4449,7 @@ static const struct bpf_func_proto bpf_skb_set_tunnel_opt_proto = {
.gpl_only = false,
.ret_type = RET_INTEGER,
.arg1_type = ARG_PTR_TO_CTX,
- .arg2_type = ARG_PTR_TO_MEM,
+ .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg3_type = ARG_CONST_SIZE,
};
@@ -4567,7 +4619,7 @@ static const struct bpf_func_proto bpf_xdp_event_output_proto = {
.arg1_type = ARG_PTR_TO_CTX,
.arg2_type = ARG_CONST_MAP_PTR,
.arg3_type = ARG_ANYTHING,
- .arg4_type = ARG_PTR_TO_MEM,
+ .arg4_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg5_type = ARG_CONST_SIZE_OR_ZERO,
};
@@ -4581,7 +4633,7 @@ const struct bpf_func_proto bpf_xdp_output_proto = {
.arg1_btf_id = &bpf_xdp_output_btf_ids[0],
.arg2_type = ARG_CONST_MAP_PTR,
.arg3_type = ARG_ANYTHING,
- .arg4_type = ARG_PTR_TO_MEM,
+ .arg4_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg5_type = ARG_CONST_SIZE_OR_ZERO,
};
@@ -4742,12 +4794,14 @@ static int _bpf_setsockopt(struct sock *sk, int level, int optname,
switch (optname) {
case SO_RCVBUF:
val = min_t(u32, val, sysctl_rmem_max);
+ val = min_t(int, val, INT_MAX / 2);
sk->sk_userlocks |= SOCK_RCVBUF_LOCK;
WRITE_ONCE(sk->sk_rcvbuf,
max_t(int, val * 2, SOCK_MIN_RCVBUF));
break;
case SO_SNDBUF:
val = min_t(u32, val, sysctl_wmem_max);
+ val = min_t(int, val, INT_MAX / 2);
sk->sk_userlocks |= SOCK_SNDBUF_LOCK;
WRITE_ONCE(sk->sk_sndbuf,
max_t(int, val * 2, SOCK_MIN_SNDBUF));
@@ -4968,6 +5022,12 @@ static int _bpf_getsockopt(struct sock *sk, int level, int optname,
goto err_clear;
switch (optname) {
+ case SO_RCVBUF:
+ *((int *)optval) = sk->sk_rcvbuf;
+ break;
+ case SO_SNDBUF:
+ *((int *)optval) = sk->sk_sndbuf;
+ break;
case SO_MARK:
*((int *)optval) = sk->sk_mark;
break;
@@ -5067,7 +5127,7 @@ const struct bpf_func_proto bpf_sk_setsockopt_proto = {
.arg1_type = ARG_PTR_TO_BTF_ID_SOCK_COMMON,
.arg2_type = ARG_ANYTHING,
.arg3_type = ARG_ANYTHING,
- .arg4_type = ARG_PTR_TO_MEM,
+ .arg4_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg5_type = ARG_CONST_SIZE,
};
@@ -5101,7 +5161,7 @@ static const struct bpf_func_proto bpf_sock_addr_setsockopt_proto = {
.arg1_type = ARG_PTR_TO_CTX,
.arg2_type = ARG_ANYTHING,
.arg3_type = ARG_ANYTHING,
- .arg4_type = ARG_PTR_TO_MEM,
+ .arg4_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg5_type = ARG_CONST_SIZE,
};
@@ -5135,7 +5195,7 @@ static const struct bpf_func_proto bpf_sock_ops_setsockopt_proto = {
.arg1_type = ARG_PTR_TO_CTX,
.arg2_type = ARG_ANYTHING,
.arg3_type = ARG_ANYTHING,
- .arg4_type = ARG_PTR_TO_MEM,
+ .arg4_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg5_type = ARG_CONST_SIZE,
};
@@ -5310,7 +5370,7 @@ static const struct bpf_func_proto bpf_bind_proto = {
.gpl_only = false,
.ret_type = RET_INTEGER,
.arg1_type = ARG_PTR_TO_CTX,
- .arg2_type = ARG_PTR_TO_MEM,
+ .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg3_type = ARG_CONST_SIZE,
};
@@ -5898,7 +5958,7 @@ static const struct bpf_func_proto bpf_lwt_in_push_encap_proto = {
.ret_type = RET_INTEGER,
.arg1_type = ARG_PTR_TO_CTX,
.arg2_type = ARG_ANYTHING,
- .arg3_type = ARG_PTR_TO_MEM,
+ .arg3_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg4_type = ARG_CONST_SIZE
};
@@ -5908,7 +5968,7 @@ static const struct bpf_func_proto bpf_lwt_xmit_push_encap_proto = {
.ret_type = RET_INTEGER,
.arg1_type = ARG_PTR_TO_CTX,
.arg2_type = ARG_ANYTHING,
- .arg3_type = ARG_PTR_TO_MEM,
+ .arg3_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg4_type = ARG_CONST_SIZE
};
@@ -5951,7 +6011,7 @@ static const struct bpf_func_proto bpf_lwt_seg6_store_bytes_proto = {
.ret_type = RET_INTEGER,
.arg1_type = ARG_PTR_TO_CTX,
.arg2_type = ARG_ANYTHING,
- .arg3_type = ARG_PTR_TO_MEM,
+ .arg3_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg4_type = ARG_CONST_SIZE
};
@@ -6039,7 +6099,7 @@ static const struct bpf_func_proto bpf_lwt_seg6_action_proto = {
.ret_type = RET_INTEGER,
.arg1_type = ARG_PTR_TO_CTX,
.arg2_type = ARG_ANYTHING,
- .arg3_type = ARG_PTR_TO_MEM,
+ .arg3_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg4_type = ARG_CONST_SIZE
};
@@ -6264,7 +6324,7 @@ static const struct bpf_func_proto bpf_skc_lookup_tcp_proto = {
.pkt_access = true,
.ret_type = RET_PTR_TO_SOCK_COMMON_OR_NULL,
.arg1_type = ARG_PTR_TO_CTX,
- .arg2_type = ARG_PTR_TO_MEM,
+ .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg3_type = ARG_CONST_SIZE,
.arg4_type = ARG_ANYTHING,
.arg5_type = ARG_ANYTHING,
@@ -6283,7 +6343,7 @@ static const struct bpf_func_proto bpf_sk_lookup_tcp_proto = {
.pkt_access = true,
.ret_type = RET_PTR_TO_SOCKET_OR_NULL,
.arg1_type = ARG_PTR_TO_CTX,
- .arg2_type = ARG_PTR_TO_MEM,
+ .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg3_type = ARG_CONST_SIZE,
.arg4_type = ARG_ANYTHING,
.arg5_type = ARG_ANYTHING,
@@ -6302,7 +6362,7 @@ static const struct bpf_func_proto bpf_sk_lookup_udp_proto = {
.pkt_access = true,
.ret_type = RET_PTR_TO_SOCKET_OR_NULL,
.arg1_type = ARG_PTR_TO_CTX,
- .arg2_type = ARG_PTR_TO_MEM,
+ .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg3_type = ARG_CONST_SIZE,
.arg4_type = ARG_ANYTHING,
.arg5_type = ARG_ANYTHING,
@@ -6339,7 +6399,7 @@ static const struct bpf_func_proto bpf_xdp_sk_lookup_udp_proto = {
.pkt_access = true,
.ret_type = RET_PTR_TO_SOCKET_OR_NULL,
.arg1_type = ARG_PTR_TO_CTX,
- .arg2_type = ARG_PTR_TO_MEM,
+ .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg3_type = ARG_CONST_SIZE,
.arg4_type = ARG_ANYTHING,
.arg5_type = ARG_ANYTHING,
@@ -6362,7 +6422,7 @@ static const struct bpf_func_proto bpf_xdp_skc_lookup_tcp_proto = {
.pkt_access = true,
.ret_type = RET_PTR_TO_SOCK_COMMON_OR_NULL,
.arg1_type = ARG_PTR_TO_CTX,
- .arg2_type = ARG_PTR_TO_MEM,
+ .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg3_type = ARG_CONST_SIZE,
.arg4_type = ARG_ANYTHING,
.arg5_type = ARG_ANYTHING,
@@ -6385,7 +6445,7 @@ static const struct bpf_func_proto bpf_xdp_sk_lookup_tcp_proto = {
.pkt_access = true,
.ret_type = RET_PTR_TO_SOCKET_OR_NULL,
.arg1_type = ARG_PTR_TO_CTX,
- .arg2_type = ARG_PTR_TO_MEM,
+ .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg3_type = ARG_CONST_SIZE,
.arg4_type = ARG_ANYTHING,
.arg5_type = ARG_ANYTHING,
@@ -6404,7 +6464,7 @@ static const struct bpf_func_proto bpf_sock_addr_skc_lookup_tcp_proto = {
.gpl_only = false,
.ret_type = RET_PTR_TO_SOCK_COMMON_OR_NULL,
.arg1_type = ARG_PTR_TO_CTX,
- .arg2_type = ARG_PTR_TO_MEM,
+ .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg3_type = ARG_CONST_SIZE,
.arg4_type = ARG_ANYTHING,
.arg5_type = ARG_ANYTHING,
@@ -6423,7 +6483,7 @@ static const struct bpf_func_proto bpf_sock_addr_sk_lookup_tcp_proto = {
.gpl_only = false,
.ret_type = RET_PTR_TO_SOCKET_OR_NULL,
.arg1_type = ARG_PTR_TO_CTX,
- .arg2_type = ARG_PTR_TO_MEM,
+ .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg3_type = ARG_CONST_SIZE,
.arg4_type = ARG_ANYTHING,
.arg5_type = ARG_ANYTHING,
@@ -6442,7 +6502,7 @@ static const struct bpf_func_proto bpf_sock_addr_sk_lookup_udp_proto = {
.gpl_only = false,
.ret_type = RET_PTR_TO_SOCKET_OR_NULL,
.arg1_type = ARG_PTR_TO_CTX,
- .arg2_type = ARG_PTR_TO_MEM,
+ .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg3_type = ARG_CONST_SIZE,
.arg4_type = ARG_ANYTHING,
.arg5_type = ARG_ANYTHING,
@@ -6755,9 +6815,9 @@ static const struct bpf_func_proto bpf_tcp_check_syncookie_proto = {
.pkt_access = true,
.ret_type = RET_INTEGER,
.arg1_type = ARG_PTR_TO_BTF_ID_SOCK_COMMON,
- .arg2_type = ARG_PTR_TO_MEM,
+ .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg3_type = ARG_CONST_SIZE,
- .arg4_type = ARG_PTR_TO_MEM,
+ .arg4_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg5_type = ARG_CONST_SIZE,
};
@@ -6824,9 +6884,9 @@ static const struct bpf_func_proto bpf_tcp_gen_syncookie_proto = {
.pkt_access = true,
.ret_type = RET_INTEGER,
.arg1_type = ARG_PTR_TO_BTF_ID_SOCK_COMMON,
- .arg2_type = ARG_PTR_TO_MEM,
+ .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg3_type = ARG_CONST_SIZE,
- .arg4_type = ARG_PTR_TO_MEM,
+ .arg4_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg5_type = ARG_CONST_SIZE,
};
@@ -7055,7 +7115,7 @@ static const struct bpf_func_proto bpf_sock_ops_store_hdr_opt_proto = {
.gpl_only = false,
.ret_type = RET_INTEGER,
.arg1_type = ARG_PTR_TO_CTX,
- .arg2_type = ARG_PTR_TO_MEM,
+ .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg3_type = ARG_CONST_SIZE,
.arg4_type = ARG_ANYTHING,
};
@@ -8029,7 +8089,7 @@ static int bpf_unclone_prologue(struct bpf_insn *insn_buf, bool direct_write,
* (Fast-path, otherwise approximation that we might be
* a clone, do the rest in helper.)
*/
- *insn++ = BPF_LDX_MEM(BPF_B, BPF_REG_6, BPF_REG_1, CLONED_OFFSET());
+ *insn++ = BPF_LDX_MEM(BPF_B, BPF_REG_6, BPF_REG_1, CLONED_OFFSET);
*insn++ = BPF_ALU32_IMM(BPF_AND, BPF_REG_6, CLONED_MASK);
*insn++ = BPF_JMP_IMM(BPF_JEQ, BPF_REG_6, 0, 7);
@@ -8181,13 +8241,13 @@ static bool xdp_is_valid_access(int off, int size,
return __is_valid_xdp_access(off, size);
}
-void bpf_warn_invalid_xdp_action(u32 act)
+void bpf_warn_invalid_xdp_action(struct net_device *dev, struct bpf_prog *prog, u32 act)
{
const u32 act_max = XDP_REDIRECT;
- WARN_ONCE(1, "%s XDP return value %u, expect packet loss!\n",
- act > act_max ? "Illegal" : "Driver unsupported",
- act);
+ pr_warn_once("%s XDP return value %u on prog %s (id %d) dev %s, expect packet loss!\n",
+ act > act_max ? "Illegal" : "Driver unsupported",
+ act, prog->aux->name, prog->aux->id, dev ? dev->name : "N/A");
}
EXPORT_SYMBOL_GPL(bpf_warn_invalid_xdp_action);
@@ -8617,7 +8677,7 @@ static u32 bpf_convert_ctx_access(enum bpf_access_type type,
case offsetof(struct __sk_buff, pkt_type):
*target_size = 1;
*insn++ = BPF_LDX_MEM(BPF_B, si->dst_reg, si->src_reg,
- PKT_TYPE_OFFSET());
+ PKT_TYPE_OFFSET);
*insn++ = BPF_ALU32_IMM(BPF_AND, si->dst_reg, PKT_TYPE_MAX);
#ifdef __BIG_ENDIAN_BITFIELD
*insn++ = BPF_ALU32_IMM(BPF_RSH, si->dst_reg, 5);
@@ -8642,7 +8702,7 @@ static u32 bpf_convert_ctx_access(enum bpf_access_type type,
case offsetof(struct __sk_buff, vlan_present):
*target_size = 1;
*insn++ = BPF_LDX_MEM(BPF_B, si->dst_reg, si->src_reg,
- PKT_VLAN_PRESENT_OFFSET());
+ PKT_VLAN_PRESENT_OFFSET);
if (PKT_VLAN_PRESENT_BIT)
*insn++ = BPF_ALU32_IMM(BPF_RSH, si->dst_reg, PKT_VLAN_PRESENT_BIT);
if (PKT_VLAN_PRESENT_BIT < 7)
@@ -10543,6 +10603,7 @@ static bool sk_lookup_is_valid_access(int off, int size,
case bpf_ctx_range_till(struct bpf_sk_lookup, local_ip6[0], local_ip6[3]):
case bpf_ctx_range(struct bpf_sk_lookup, remote_port):
case bpf_ctx_range(struct bpf_sk_lookup, local_port):
+ case bpf_ctx_range(struct bpf_sk_lookup, ingress_ifindex):
bpf_ctx_record_field_size(info, sizeof(__u32));
return bpf_ctx_narrow_access_ok(off, size, sizeof(__u32));
@@ -10632,6 +10693,12 @@ static u32 sk_lookup_convert_ctx_access(enum bpf_access_type type,
bpf_target_off(struct bpf_sk_lookup_kern,
dport, 2, target_size));
break;
+
+ case offsetof(struct bpf_sk_lookup, ingress_ifindex):
+ *insn++ = BPF_LDX_MEM(BPF_W, si->dst_reg, si->src_reg,
+ bpf_target_off(struct bpf_sk_lookup_kern,
+ ingress_ifindex, 4, target_size));
+ break;
}
return insn - insn_buf;
@@ -10656,14 +10723,10 @@ void bpf_prog_change_xdp(struct bpf_prog *prev_prog, struct bpf_prog *prog)
bpf_dispatcher_change_prog(BPF_DISPATCHER_PTR(xdp), prev_prog, prog);
}
-#ifdef CONFIG_DEBUG_INFO_BTF
-BTF_ID_LIST_GLOBAL(btf_sock_ids)
+BTF_ID_LIST_GLOBAL(btf_sock_ids, MAX_BTF_SOCK_TYPE)
#define BTF_SOCK_TYPE(name, type) BTF_ID(struct, type)
BTF_SOCK_TYPE_xxx
#undef BTF_SOCK_TYPE
-#else
-u32 btf_sock_ids[MAX_BTF_SOCK_TYPE];
-#endif
BPF_CALL_1(bpf_skc_to_tcp6_sock, struct sock *, sk)
{
diff --git a/net/core/flow_dissector.c b/net/core/flow_dissector.c
index 1b094c4..15833e1 100644
--- a/net/core/flow_dissector.c
+++ b/net/core/flow_dissector.c
@@ -5,6 +5,7 @@
#include <linux/ip.h>
#include <linux/ipv6.h>
#include <linux/if_vlan.h>
+#include <linux/filter.h>
#include <net/dsa.h>
#include <net/dst_metadata.h>
#include <net/ip.h>
@@ -1461,7 +1462,7 @@ bool __skb_flow_dissect(const struct net *net,
}
EXPORT_SYMBOL(__skb_flow_dissect);
-static siphash_key_t hashrnd __read_mostly;
+static siphash_aligned_key_t hashrnd;
static __always_inline void __flow_hash_secret_init(void)
{
net_get_random_once(&hashrnd, sizeof(hashrnd));
diff --git a/net/core/flow_offload.c b/net/core/flow_offload.c
index 6beaea1..73f68d4 100644
--- a/net/core/flow_offload.c
+++ b/net/core/flow_offload.c
@@ -1,6 +1,7 @@
/* SPDX-License-Identifier: GPL-2.0 */
#include <linux/kernel.h>
#include <linux/slab.h>
+#include <net/act_api.h>
#include <net/flow_offload.h>
#include <linux/rtnetlink.h>
#include <linux/mutex.h>
@@ -27,6 +28,26 @@ struct flow_rule *flow_rule_alloc(unsigned int num_actions)
}
EXPORT_SYMBOL(flow_rule_alloc);
+struct flow_offload_action *offload_action_alloc(unsigned int num_actions)
+{
+ struct flow_offload_action *fl_action;
+ int i;
+
+ fl_action = kzalloc(struct_size(fl_action, action.entries, num_actions),
+ GFP_KERNEL);
+ if (!fl_action)
+ return NULL;
+
+ fl_action->action.num_entries = num_actions;
+ /* Pre-fill each action hw_stats with DONT_CARE.
+ * Caller can override this if it wants stats for a given action.
+ */
+ for (i = 0; i < num_actions; i++)
+ fl_action->action.entries[i].hw_stats = FLOW_ACTION_HW_STATS_DONT_CARE;
+
+ return fl_action;
+}
+
#define FLOW_DISSECTOR_MATCH(__rule, __type, __out) \
const struct flow_match *__m = &(__rule)->match; \
struct flow_dissector *__d = (__m)->dissector; \
@@ -397,6 +418,8 @@ int flow_indr_dev_register(flow_indr_block_bind_cb_t *cb, void *cb_priv)
existing_qdiscs_register(cb, cb_priv);
mutex_unlock(&flow_indr_block_lock);
+ tcf_action_reoffload_cb(cb, cb_priv, true);
+
return 0;
}
EXPORT_SYMBOL(flow_indr_dev_register);
@@ -449,6 +472,7 @@ void flow_indr_dev_unregister(flow_indr_block_bind_cb_t *cb, void *cb_priv,
__flow_block_indr_cleanup(release, cb_priv, &cleanup_list);
mutex_unlock(&flow_indr_block_lock);
+ tcf_action_reoffload_cb(cb, cb_priv, false);
flow_block_indr_notify(&cleanup_list);
kfree(indr_dev);
}
@@ -549,19 +573,25 @@ int flow_indr_dev_setup_offload(struct net_device *dev, struct Qdisc *sch,
void (*cleanup)(struct flow_block_cb *block_cb))
{
struct flow_indr_dev *this;
+ u32 count = 0;
+ int err;
mutex_lock(&flow_indr_block_lock);
+ if (bo) {
+ if (bo->command == FLOW_BLOCK_BIND)
+ indir_dev_add(data, dev, sch, type, cleanup, bo);
+ else if (bo->command == FLOW_BLOCK_UNBIND)
+ indir_dev_remove(data);
+ }
- if (bo->command == FLOW_BLOCK_BIND)
- indir_dev_add(data, dev, sch, type, cleanup, bo);
- else if (bo->command == FLOW_BLOCK_UNBIND)
- indir_dev_remove(data);
-
- list_for_each_entry(this, &flow_block_indr_dev_list, list)
- this->cb(dev, sch, this->cb_priv, type, bo, data, cleanup);
+ list_for_each_entry(this, &flow_block_indr_dev_list, list) {
+ err = this->cb(dev, sch, this->cb_priv, type, bo, data, cleanup);
+ if (!err)
+ count++;
+ }
mutex_unlock(&flow_indr_block_lock);
- return list_empty(&bo->cb_list) ? -EOPNOTSUPP : 0;
+ return (bo && list_empty(&bo->cb_list)) ? -EOPNOTSUPP : count;
}
EXPORT_SYMBOL(flow_indr_dev_setup_offload);
diff --git a/net/core/gro.c b/net/core/gro.c
new file mode 100644
index 0000000..a11b286
--- /dev/null
+++ b/net/core/gro.c
@@ -0,0 +1,770 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+#include <net/gro.h>
+#include <net/dst_metadata.h>
+#include <net/busy_poll.h>
+#include <trace/events/net.h>
+
+#define MAX_GRO_SKBS 8
+
+/* This should be increased if a protocol with a bigger head is added. */
+#define GRO_MAX_HEAD (MAX_HEADER + 128)
+
+static DEFINE_SPINLOCK(offload_lock);
+static struct list_head offload_base __read_mostly = LIST_HEAD_INIT(offload_base);
+/* Maximum number of GRO_NORMAL skbs to batch up for list-RX */
+int gro_normal_batch __read_mostly = 8;
+
+/**
+ * dev_add_offload - register offload handlers
+ * @po: protocol offload declaration
+ *
+ * Add protocol offload handlers to the networking stack. The passed
+ * &proto_offload is linked into kernel lists and may not be freed until
+ * it has been removed from the kernel lists.
+ *
+ * This call does not sleep therefore it can not
+ * guarantee all CPU's that are in middle of receiving packets
+ * will see the new offload handlers (until the next received packet).
+ */
+void dev_add_offload(struct packet_offload *po)
+{
+ struct packet_offload *elem;
+
+ spin_lock(&offload_lock);
+ list_for_each_entry(elem, &offload_base, list) {
+ if (po->priority < elem->priority)
+ break;
+ }
+ list_add_rcu(&po->list, elem->list.prev);
+ spin_unlock(&offload_lock);
+}
+EXPORT_SYMBOL(dev_add_offload);
+
+/**
+ * __dev_remove_offload - remove offload handler
+ * @po: packet offload declaration
+ *
+ * Remove a protocol offload handler that was previously added to the
+ * kernel offload handlers by dev_add_offload(). The passed &offload_type
+ * is removed from the kernel lists and can be freed or reused once this
+ * function returns.
+ *
+ * The packet type might still be in use by receivers
+ * and must not be freed until after all the CPU's have gone
+ * through a quiescent state.
+ */
+static void __dev_remove_offload(struct packet_offload *po)
+{
+ struct list_head *head = &offload_base;
+ struct packet_offload *po1;
+
+ spin_lock(&offload_lock);
+
+ list_for_each_entry(po1, head, list) {
+ if (po == po1) {
+ list_del_rcu(&po->list);
+ goto out;
+ }
+ }
+
+ pr_warn("dev_remove_offload: %p not found\n", po);
+out:
+ spin_unlock(&offload_lock);
+}
+
+/**
+ * dev_remove_offload - remove packet offload handler
+ * @po: packet offload declaration
+ *
+ * Remove a packet offload handler that was previously added to the kernel
+ * offload handlers by dev_add_offload(). The passed &offload_type is
+ * removed from the kernel lists and can be freed or reused once this
+ * function returns.
+ *
+ * This call sleeps to guarantee that no CPU is looking at the packet
+ * type after return.
+ */
+void dev_remove_offload(struct packet_offload *po)
+{
+ __dev_remove_offload(po);
+
+ synchronize_net();
+}
+EXPORT_SYMBOL(dev_remove_offload);
+
+/**
+ * skb_mac_gso_segment - mac layer segmentation handler.
+ * @skb: buffer to segment
+ * @features: features for the output path (see dev->features)
+ */
+struct sk_buff *skb_mac_gso_segment(struct sk_buff *skb,
+ netdev_features_t features)
+{
+ struct sk_buff *segs = ERR_PTR(-EPROTONOSUPPORT);
+ struct packet_offload *ptype;
+ int vlan_depth = skb->mac_len;
+ __be16 type = skb_network_protocol(skb, &vlan_depth);
+
+ if (unlikely(!type))
+ return ERR_PTR(-EINVAL);
+
+ __skb_pull(skb, vlan_depth);
+
+ rcu_read_lock();
+ list_for_each_entry_rcu(ptype, &offload_base, list) {
+ if (ptype->type == type && ptype->callbacks.gso_segment) {
+ segs = ptype->callbacks.gso_segment(skb, features);
+ break;
+ }
+ }
+ rcu_read_unlock();
+
+ __skb_push(skb, skb->data - skb_mac_header(skb));
+
+ return segs;
+}
+EXPORT_SYMBOL(skb_mac_gso_segment);
+
+int skb_gro_receive(struct sk_buff *p, struct sk_buff *skb)
+{
+ struct skb_shared_info *pinfo, *skbinfo = skb_shinfo(skb);
+ unsigned int offset = skb_gro_offset(skb);
+ unsigned int headlen = skb_headlen(skb);
+ unsigned int len = skb_gro_len(skb);
+ unsigned int delta_truesize;
+ unsigned int gro_max_size;
+ unsigned int new_truesize;
+ struct sk_buff *lp;
+
+ /* pairs with WRITE_ONCE() in netif_set_gro_max_size() */
+ gro_max_size = READ_ONCE(p->dev->gro_max_size);
+
+ if (unlikely(p->len + len >= gro_max_size || NAPI_GRO_CB(skb)->flush))
+ return -E2BIG;
+
+ lp = NAPI_GRO_CB(p)->last;
+ pinfo = skb_shinfo(lp);
+
+ if (headlen <= offset) {
+ skb_frag_t *frag;
+ skb_frag_t *frag2;
+ int i = skbinfo->nr_frags;
+ int nr_frags = pinfo->nr_frags + i;
+
+ if (nr_frags > MAX_SKB_FRAGS)
+ goto merge;
+
+ offset -= headlen;
+ pinfo->nr_frags = nr_frags;
+ skbinfo->nr_frags = 0;
+
+ frag = pinfo->frags + nr_frags;
+ frag2 = skbinfo->frags + i;
+ do {
+ *--frag = *--frag2;
+ } while (--i);
+
+ skb_frag_off_add(frag, offset);
+ skb_frag_size_sub(frag, offset);
+
+ /* all fragments truesize : remove (head size + sk_buff) */
+ new_truesize = SKB_TRUESIZE(skb_end_offset(skb));
+ delta_truesize = skb->truesize - new_truesize;
+
+ skb->truesize = new_truesize;
+ skb->len -= skb->data_len;
+ skb->data_len = 0;
+
+ NAPI_GRO_CB(skb)->free = NAPI_GRO_FREE;
+ goto done;
+ } else if (skb->head_frag) {
+ int nr_frags = pinfo->nr_frags;
+ skb_frag_t *frag = pinfo->frags + nr_frags;
+ struct page *page = virt_to_head_page(skb->head);
+ unsigned int first_size = headlen - offset;
+ unsigned int first_offset;
+
+ if (nr_frags + 1 + skbinfo->nr_frags > MAX_SKB_FRAGS)
+ goto merge;
+
+ first_offset = skb->data -
+ (unsigned char *)page_address(page) +
+ offset;
+
+ pinfo->nr_frags = nr_frags + 1 + skbinfo->nr_frags;
+
+ __skb_frag_set_page(frag, page);
+ skb_frag_off_set(frag, first_offset);
+ skb_frag_size_set(frag, first_size);
+
+ memcpy(frag + 1, skbinfo->frags, sizeof(*frag) * skbinfo->nr_frags);
+ /* We dont need to clear skbinfo->nr_frags here */
+
+ new_truesize = SKB_DATA_ALIGN(sizeof(struct sk_buff));
+ delta_truesize = skb->truesize - new_truesize;
+ skb->truesize = new_truesize;
+ NAPI_GRO_CB(skb)->free = NAPI_GRO_FREE_STOLEN_HEAD;
+ goto done;
+ }
+
+merge:
+ /* sk owenrship - if any - completely transferred to the aggregated packet */
+ skb->destructor = NULL;
+ delta_truesize = skb->truesize;
+ if (offset > headlen) {
+ unsigned int eat = offset - headlen;
+
+ skb_frag_off_add(&skbinfo->frags[0], eat);
+ skb_frag_size_sub(&skbinfo->frags[0], eat);
+ skb->data_len -= eat;
+ skb->len -= eat;
+ offset = headlen;
+ }
+
+ __skb_pull(skb, offset);
+
+ if (NAPI_GRO_CB(p)->last == p)
+ skb_shinfo(p)->frag_list = skb;
+ else
+ NAPI_GRO_CB(p)->last->next = skb;
+ NAPI_GRO_CB(p)->last = skb;
+ __skb_header_release(skb);
+ lp = p;
+
+done:
+ NAPI_GRO_CB(p)->count++;
+ p->data_len += len;
+ p->truesize += delta_truesize;
+ p->len += len;
+ if (lp != p) {
+ lp->data_len += len;
+ lp->truesize += delta_truesize;
+ lp->len += len;
+ }
+ NAPI_GRO_CB(skb)->same_flow = 1;
+ return 0;
+}
+
+
+static void napi_gro_complete(struct napi_struct *napi, struct sk_buff *skb)
+{
+ struct packet_offload *ptype;
+ __be16 type = skb->protocol;
+ struct list_head *head = &offload_base;
+ int err = -ENOENT;
+
+ BUILD_BUG_ON(sizeof(struct napi_gro_cb) > sizeof(skb->cb));
+
+ if (NAPI_GRO_CB(skb)->count == 1) {
+ skb_shinfo(skb)->gso_size = 0;
+ goto out;
+ }
+
+ rcu_read_lock();
+ list_for_each_entry_rcu(ptype, head, list) {
+ if (ptype->type != type || !ptype->callbacks.gro_complete)
+ continue;
+
+ err = INDIRECT_CALL_INET(ptype->callbacks.gro_complete,
+ ipv6_gro_complete, inet_gro_complete,
+ skb, 0);
+ break;
+ }
+ rcu_read_unlock();
+
+ if (err) {
+ WARN_ON(&ptype->list == head);
+ kfree_skb(skb);
+ return;
+ }
+
+out:
+ gro_normal_one(napi, skb, NAPI_GRO_CB(skb)->count);
+}
+
+static void __napi_gro_flush_chain(struct napi_struct *napi, u32 index,
+ bool flush_old)
+{
+ struct list_head *head = &napi->gro_hash[index].list;
+ struct sk_buff *skb, *p;
+
+ list_for_each_entry_safe_reverse(skb, p, head, list) {
+ if (flush_old && NAPI_GRO_CB(skb)->age == jiffies)
+ return;
+ skb_list_del_init(skb);
+ napi_gro_complete(napi, skb);
+ napi->gro_hash[index].count--;
+ }
+
+ if (!napi->gro_hash[index].count)
+ __clear_bit(index, &napi->gro_bitmask);
+}
+
+/* napi->gro_hash[].list contains packets ordered by age.
+ * youngest packets at the head of it.
+ * Complete skbs in reverse order to reduce latencies.
+ */
+void napi_gro_flush(struct napi_struct *napi, bool flush_old)
+{
+ unsigned long bitmask = napi->gro_bitmask;
+ unsigned int i, base = ~0U;
+
+ while ((i = ffs(bitmask)) != 0) {
+ bitmask >>= i;
+ base += i;
+ __napi_gro_flush_chain(napi, base, flush_old);
+ }
+}
+EXPORT_SYMBOL(napi_gro_flush);
+
+static void gro_list_prepare(const struct list_head *head,
+ const struct sk_buff *skb)
+{
+ unsigned int maclen = skb->dev->hard_header_len;
+ u32 hash = skb_get_hash_raw(skb);
+ struct sk_buff *p;
+
+ list_for_each_entry(p, head, list) {
+ unsigned long diffs;
+
+ NAPI_GRO_CB(p)->flush = 0;
+
+ if (hash != skb_get_hash_raw(p)) {
+ NAPI_GRO_CB(p)->same_flow = 0;
+ continue;
+ }
+
+ diffs = (unsigned long)p->dev ^ (unsigned long)skb->dev;
+ diffs |= skb_vlan_tag_present(p) ^ skb_vlan_tag_present(skb);
+ if (skb_vlan_tag_present(p))
+ diffs |= skb_vlan_tag_get(p) ^ skb_vlan_tag_get(skb);
+ diffs |= skb_metadata_differs(p, skb);
+ if (maclen == ETH_HLEN)
+ diffs |= compare_ether_header(skb_mac_header(p),
+ skb_mac_header(skb));
+ else if (!diffs)
+ diffs = memcmp(skb_mac_header(p),
+ skb_mac_header(skb),
+ maclen);
+
+ /* in most common scenarions 'slow_gro' is 0
+ * otherwise we are already on some slower paths
+ * either skip all the infrequent tests altogether or
+ * avoid trying too hard to skip each of them individually
+ */
+ if (!diffs && unlikely(skb->slow_gro | p->slow_gro)) {
+#if IS_ENABLED(CONFIG_SKB_EXTENSIONS) && IS_ENABLED(CONFIG_NET_TC_SKB_EXT)
+ struct tc_skb_ext *skb_ext;
+ struct tc_skb_ext *p_ext;
+#endif
+
+ diffs |= p->sk != skb->sk;
+ diffs |= skb_metadata_dst_cmp(p, skb);
+ diffs |= skb_get_nfct(p) ^ skb_get_nfct(skb);
+
+#if IS_ENABLED(CONFIG_SKB_EXTENSIONS) && IS_ENABLED(CONFIG_NET_TC_SKB_EXT)
+ skb_ext = skb_ext_find(skb, TC_SKB_EXT);
+ p_ext = skb_ext_find(p, TC_SKB_EXT);
+
+ diffs |= (!!p_ext) ^ (!!skb_ext);
+ if (!diffs && unlikely(skb_ext))
+ diffs |= p_ext->chain ^ skb_ext->chain;
+#endif
+ }
+
+ NAPI_GRO_CB(p)->same_flow = !diffs;
+ }
+}
+
+static inline void skb_gro_reset_offset(struct sk_buff *skb, u32 nhoff)
+{
+ const struct skb_shared_info *pinfo = skb_shinfo(skb);
+ const skb_frag_t *frag0 = &pinfo->frags[0];
+
+ NAPI_GRO_CB(skb)->data_offset = 0;
+ NAPI_GRO_CB(skb)->frag0 = NULL;
+ NAPI_GRO_CB(skb)->frag0_len = 0;
+
+ if (!skb_headlen(skb) && pinfo->nr_frags &&
+ !PageHighMem(skb_frag_page(frag0)) &&
+ (!NET_IP_ALIGN || !((skb_frag_off(frag0) + nhoff) & 3))) {
+ NAPI_GRO_CB(skb)->frag0 = skb_frag_address(frag0);
+ NAPI_GRO_CB(skb)->frag0_len = min_t(unsigned int,
+ skb_frag_size(frag0),
+ skb->end - skb->tail);
+ }
+}
+
+static void gro_pull_from_frag0(struct sk_buff *skb, int grow)
+{
+ struct skb_shared_info *pinfo = skb_shinfo(skb);
+
+ BUG_ON(skb->end - skb->tail < grow);
+
+ memcpy(skb_tail_pointer(skb), NAPI_GRO_CB(skb)->frag0, grow);
+
+ skb->data_len -= grow;
+ skb->tail += grow;
+
+ skb_frag_off_add(&pinfo->frags[0], grow);
+ skb_frag_size_sub(&pinfo->frags[0], grow);
+
+ if (unlikely(!skb_frag_size(&pinfo->frags[0]))) {
+ skb_frag_unref(skb, 0);
+ memmove(pinfo->frags, pinfo->frags + 1,
+ --pinfo->nr_frags * sizeof(pinfo->frags[0]));
+ }
+}
+
+static void gro_flush_oldest(struct napi_struct *napi, struct list_head *head)
+{
+ struct sk_buff *oldest;
+
+ oldest = list_last_entry(head, struct sk_buff, list);
+
+ /* We are called with head length >= MAX_GRO_SKBS, so this is
+ * impossible.
+ */
+ if (WARN_ON_ONCE(!oldest))
+ return;
+
+ /* Do not adjust napi->gro_hash[].count, caller is adding a new
+ * SKB to the chain.
+ */
+ skb_list_del_init(oldest);
+ napi_gro_complete(napi, oldest);
+}
+
+static enum gro_result dev_gro_receive(struct napi_struct *napi, struct sk_buff *skb)
+{
+ u32 bucket = skb_get_hash_raw(skb) & (GRO_HASH_BUCKETS - 1);
+ struct gro_list *gro_list = &napi->gro_hash[bucket];
+ struct list_head *head = &offload_base;
+ struct packet_offload *ptype;
+ __be16 type = skb->protocol;
+ struct sk_buff *pp = NULL;
+ enum gro_result ret;
+ int same_flow;
+ int grow;
+
+ if (netif_elide_gro(skb->dev))
+ goto normal;
+
+ gro_list_prepare(&gro_list->list, skb);
+
+ rcu_read_lock();
+ list_for_each_entry_rcu(ptype, head, list) {
+ if (ptype->type != type || !ptype->callbacks.gro_receive)
+ continue;
+
+ skb_set_network_header(skb, skb_gro_offset(skb));
+ skb_reset_mac_len(skb);
+ NAPI_GRO_CB(skb)->same_flow = 0;
+ NAPI_GRO_CB(skb)->flush = skb_is_gso(skb) || skb_has_frag_list(skb);
+ NAPI_GRO_CB(skb)->free = 0;
+ NAPI_GRO_CB(skb)->encap_mark = 0;
+ NAPI_GRO_CB(skb)->recursion_counter = 0;
+ NAPI_GRO_CB(skb)->is_fou = 0;
+ NAPI_GRO_CB(skb)->is_atomic = 1;
+ NAPI_GRO_CB(skb)->gro_remcsum_start = 0;
+
+ /* Setup for GRO checksum validation */
+ switch (skb->ip_summed) {
+ case CHECKSUM_COMPLETE:
+ NAPI_GRO_CB(skb)->csum = skb->csum;
+ NAPI_GRO_CB(skb)->csum_valid = 1;
+ NAPI_GRO_CB(skb)->csum_cnt = 0;
+ break;
+ case CHECKSUM_UNNECESSARY:
+ NAPI_GRO_CB(skb)->csum_cnt = skb->csum_level + 1;
+ NAPI_GRO_CB(skb)->csum_valid = 0;
+ break;
+ default:
+ NAPI_GRO_CB(skb)->csum_cnt = 0;
+ NAPI_GRO_CB(skb)->csum_valid = 0;
+ }
+
+ pp = INDIRECT_CALL_INET(ptype->callbacks.gro_receive,
+ ipv6_gro_receive, inet_gro_receive,
+ &gro_list->list, skb);
+ break;
+ }
+ rcu_read_unlock();
+
+ if (&ptype->list == head)
+ goto normal;
+
+ if (PTR_ERR(pp) == -EINPROGRESS) {
+ ret = GRO_CONSUMED;
+ goto ok;
+ }
+
+ same_flow = NAPI_GRO_CB(skb)->same_flow;
+ ret = NAPI_GRO_CB(skb)->free ? GRO_MERGED_FREE : GRO_MERGED;
+
+ if (pp) {
+ skb_list_del_init(pp);
+ napi_gro_complete(napi, pp);
+ gro_list->count--;
+ }
+
+ if (same_flow)
+ goto ok;
+
+ if (NAPI_GRO_CB(skb)->flush)
+ goto normal;
+
+ if (unlikely(gro_list->count >= MAX_GRO_SKBS))
+ gro_flush_oldest(napi, &gro_list->list);
+ else
+ gro_list->count++;
+
+ NAPI_GRO_CB(skb)->count = 1;
+ NAPI_GRO_CB(skb)->age = jiffies;
+ NAPI_GRO_CB(skb)->last = skb;
+ skb_shinfo(skb)->gso_size = skb_gro_len(skb);
+ list_add(&skb->list, &gro_list->list);
+ ret = GRO_HELD;
+
+pull:
+ grow = skb_gro_offset(skb) - skb_headlen(skb);
+ if (grow > 0)
+ gro_pull_from_frag0(skb, grow);
+ok:
+ if (gro_list->count) {
+ if (!test_bit(bucket, &napi->gro_bitmask))
+ __set_bit(bucket, &napi->gro_bitmask);
+ } else if (test_bit(bucket, &napi->gro_bitmask)) {
+ __clear_bit(bucket, &napi->gro_bitmask);
+ }
+
+ return ret;
+
+normal:
+ ret = GRO_NORMAL;
+ goto pull;
+}
+
+struct packet_offload *gro_find_receive_by_type(__be16 type)
+{
+ struct list_head *offload_head = &offload_base;
+ struct packet_offload *ptype;
+
+ list_for_each_entry_rcu(ptype, offload_head, list) {
+ if (ptype->type != type || !ptype->callbacks.gro_receive)
+ continue;
+ return ptype;
+ }
+ return NULL;
+}
+EXPORT_SYMBOL(gro_find_receive_by_type);
+
+struct packet_offload *gro_find_complete_by_type(__be16 type)
+{
+ struct list_head *offload_head = &offload_base;
+ struct packet_offload *ptype;
+
+ list_for_each_entry_rcu(ptype, offload_head, list) {
+ if (ptype->type != type || !ptype->callbacks.gro_complete)
+ continue;
+ return ptype;
+ }
+ return NULL;
+}
+EXPORT_SYMBOL(gro_find_complete_by_type);
+
+static gro_result_t napi_skb_finish(struct napi_struct *napi,
+ struct sk_buff *skb,
+ gro_result_t ret)
+{
+ switch (ret) {
+ case GRO_NORMAL:
+ gro_normal_one(napi, skb, 1);
+ break;
+
+ case GRO_MERGED_FREE:
+ if (NAPI_GRO_CB(skb)->free == NAPI_GRO_FREE_STOLEN_HEAD)
+ napi_skb_free_stolen_head(skb);
+ else if (skb->fclone != SKB_FCLONE_UNAVAILABLE)
+ __kfree_skb(skb);
+ else
+ __kfree_skb_defer(skb);
+ break;
+
+ case GRO_HELD:
+ case GRO_MERGED:
+ case GRO_CONSUMED:
+ break;
+ }
+
+ return ret;
+}
+
+gro_result_t napi_gro_receive(struct napi_struct *napi, struct sk_buff *skb)
+{
+ gro_result_t ret;
+
+ skb_mark_napi_id(skb, napi);
+ trace_napi_gro_receive_entry(skb);
+
+ skb_gro_reset_offset(skb, 0);
+
+ ret = napi_skb_finish(napi, skb, dev_gro_receive(napi, skb));
+ trace_napi_gro_receive_exit(ret);
+
+ return ret;
+}
+EXPORT_SYMBOL(napi_gro_receive);
+
+static void napi_reuse_skb(struct napi_struct *napi, struct sk_buff *skb)
+{
+ if (unlikely(skb->pfmemalloc)) {
+ consume_skb(skb);
+ return;
+ }
+ __skb_pull(skb, skb_headlen(skb));
+ /* restore the reserve we had after netdev_alloc_skb_ip_align() */
+ skb_reserve(skb, NET_SKB_PAD + NET_IP_ALIGN - skb_headroom(skb));
+ __vlan_hwaccel_clear_tag(skb);
+ skb->dev = napi->dev;
+ skb->skb_iif = 0;
+
+ /* eth_type_trans() assumes pkt_type is PACKET_HOST */
+ skb->pkt_type = PACKET_HOST;
+
+ skb->encapsulation = 0;
+ skb_shinfo(skb)->gso_type = 0;
+ skb->truesize = SKB_TRUESIZE(skb_end_offset(skb));
+ if (unlikely(skb->slow_gro)) {
+ skb_orphan(skb);
+ skb_ext_reset(skb);
+ nf_reset_ct(skb);
+ skb->slow_gro = 0;
+ }
+
+ napi->skb = skb;
+}
+
+struct sk_buff *napi_get_frags(struct napi_struct *napi)
+{
+ struct sk_buff *skb = napi->skb;
+
+ if (!skb) {
+ skb = napi_alloc_skb(napi, GRO_MAX_HEAD);
+ if (skb) {
+ napi->skb = skb;
+ skb_mark_napi_id(skb, napi);
+ }
+ }
+ return skb;
+}
+EXPORT_SYMBOL(napi_get_frags);
+
+static gro_result_t napi_frags_finish(struct napi_struct *napi,
+ struct sk_buff *skb,
+ gro_result_t ret)
+{
+ switch (ret) {
+ case GRO_NORMAL:
+ case GRO_HELD:
+ __skb_push(skb, ETH_HLEN);
+ skb->protocol = eth_type_trans(skb, skb->dev);
+ if (ret == GRO_NORMAL)
+ gro_normal_one(napi, skb, 1);
+ break;
+
+ case GRO_MERGED_FREE:
+ if (NAPI_GRO_CB(skb)->free == NAPI_GRO_FREE_STOLEN_HEAD)
+ napi_skb_free_stolen_head(skb);
+ else
+ napi_reuse_skb(napi, skb);
+ break;
+
+ case GRO_MERGED:
+ case GRO_CONSUMED:
+ break;
+ }
+
+ return ret;
+}
+
+/* Upper GRO stack assumes network header starts at gro_offset=0
+ * Drivers could call both napi_gro_frags() and napi_gro_receive()
+ * We copy ethernet header into skb->data to have a common layout.
+ */
+static struct sk_buff *napi_frags_skb(struct napi_struct *napi)
+{
+ struct sk_buff *skb = napi->skb;
+ const struct ethhdr *eth;
+ unsigned int hlen = sizeof(*eth);
+
+ napi->skb = NULL;
+
+ skb_reset_mac_header(skb);
+ skb_gro_reset_offset(skb, hlen);
+
+ if (unlikely(skb_gro_header_hard(skb, hlen))) {
+ eth = skb_gro_header_slow(skb, hlen, 0);
+ if (unlikely(!eth)) {
+ net_warn_ratelimited("%s: dropping impossible skb from %s\n",
+ __func__, napi->dev->name);
+ napi_reuse_skb(napi, skb);
+ return NULL;
+ }
+ } else {
+ eth = (const struct ethhdr *)skb->data;
+ gro_pull_from_frag0(skb, hlen);
+ NAPI_GRO_CB(skb)->frag0 += hlen;
+ NAPI_GRO_CB(skb)->frag0_len -= hlen;
+ }
+ __skb_pull(skb, hlen);
+
+ /*
+ * This works because the only protocols we care about don't require
+ * special handling.
+ * We'll fix it up properly in napi_frags_finish()
+ */
+ skb->protocol = eth->h_proto;
+
+ return skb;
+}
+
+gro_result_t napi_gro_frags(struct napi_struct *napi)
+{
+ gro_result_t ret;
+ struct sk_buff *skb = napi_frags_skb(napi);
+
+ trace_napi_gro_frags_entry(skb);
+
+ ret = napi_frags_finish(napi, skb, dev_gro_receive(napi, skb));
+ trace_napi_gro_frags_exit(ret);
+
+ return ret;
+}
+EXPORT_SYMBOL(napi_gro_frags);
+
+/* Compute the checksum from gro_offset and return the folded value
+ * after adding in any pseudo checksum.
+ */
+__sum16 __skb_gro_checksum_complete(struct sk_buff *skb)
+{
+ __wsum wsum;
+ __sum16 sum;
+
+ wsum = skb_checksum(skb, skb_gro_offset(skb), skb_gro_len(skb), 0);
+
+ /* NAPI_GRO_CB(skb)->csum holds pseudo checksum */
+ sum = csum_fold(csum_add(NAPI_GRO_CB(skb)->csum, wsum));
+ /* See comments in __skb_checksum_complete(). */
+ if (likely(!sum)) {
+ if (unlikely(skb->ip_summed == CHECKSUM_COMPLETE) &&
+ !skb->csum_complete_sw)
+ netdev_rx_csum_fault(skb->dev, skb);
+ }
+
+ NAPI_GRO_CB(skb)->csum = wsum;
+ NAPI_GRO_CB(skb)->csum_valid = 1;
+
+ return sum;
+}
+EXPORT_SYMBOL(__skb_gro_checksum_complete);
diff --git a/net/core/link_watch.c b/net/core/link_watch.c
index 1a45584..b0f5344 100644
--- a/net/core/link_watch.c
+++ b/net/core/link_watch.c
@@ -55,7 +55,7 @@ static void rfc2863_policy(struct net_device *dev)
if (operstate == dev->operstate)
return;
- write_lock_bh(&dev_base_lock);
+ write_lock(&dev_base_lock);
switch(dev->link_mode) {
case IF_LINK_MODE_TESTING:
@@ -74,7 +74,7 @@ static void rfc2863_policy(struct net_device *dev)
dev->operstate = operstate;
- write_unlock_bh(&dev_base_lock);
+ write_unlock(&dev_base_lock);
}
@@ -109,7 +109,7 @@ static void linkwatch_add_event(struct net_device *dev)
spin_lock_irqsave(&lweventlist_lock, flags);
if (list_empty(&dev->link_watch_list)) {
list_add_tail(&dev->link_watch_list, &lweventlist);
- dev_hold(dev);
+ dev_hold_track(dev, &dev->linkwatch_dev_tracker, GFP_ATOMIC);
}
spin_unlock_irqrestore(&lweventlist_lock, flags);
}
@@ -166,6 +166,9 @@ static void linkwatch_do_dev(struct net_device *dev)
netdev_state_change(dev);
}
+ /* Note: our callers are responsible for
+ * calling netdev_tracker_free().
+ */
dev_put(dev);
}
@@ -209,6 +212,10 @@ static void __linkwatch_run_queue(int urgent_only)
list_add_tail(&dev->link_watch_list, &lweventlist);
continue;
}
+ /* We must free netdev tracker under
+ * the spinlock protection.
+ */
+ netdev_tracker_free(dev, &dev->linkwatch_dev_tracker);
spin_unlock_irq(&lweventlist_lock);
linkwatch_do_dev(dev);
do_dev--;
@@ -232,6 +239,10 @@ void linkwatch_forget_dev(struct net_device *dev)
if (!list_empty(&dev->link_watch_list)) {
list_del_init(&dev->link_watch_list);
clean = 1;
+ /* We must release netdev tracker under
+ * the spinlock protection.
+ */
+ netdev_tracker_free(dev, &dev->linkwatch_dev_tracker);
}
spin_unlock_irqrestore(&lweventlist_lock, flags);
if (clean)
diff --git a/net/core/lwt_bpf.c b/net/core/lwt_bpf.c
index 2f7940b..349480e 100644
--- a/net/core/lwt_bpf.c
+++ b/net/core/lwt_bpf.c
@@ -2,6 +2,7 @@
/* Copyright (c) 2016 Thomas Graf <tgraf@tgraf.ch>
*/
+#include <linux/filter.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/skbuff.h>
diff --git a/net/core/lwtunnel.c b/net/core/lwtunnel.c
index 2820aca..9ccd64e 100644
--- a/net/core/lwtunnel.c
+++ b/net/core/lwtunnel.c
@@ -197,6 +197,10 @@ int lwtunnel_valid_encap_type_attr(struct nlattr *attr, int remaining,
nla_entype = nla_find(attrs, attrlen, RTA_ENCAP_TYPE);
if (nla_entype) {
+ if (nla_len(nla_entype) < sizeof(u16)) {
+ NL_SET_ERR_MSG(extack, "Invalid RTA_ENCAP_TYPE");
+ return -EINVAL;
+ }
encap_type = nla_get_u16(nla_entype);
if (lwtunnel_valid_encap_type(encap_type,
diff --git a/net/core/neighbour.c b/net/core/neighbour.c
index dda12fb..213cb7b 100644
--- a/net/core/neighbour.c
+++ b/net/core/neighbour.c
@@ -624,7 +624,7 @@ ___neigh_create(struct neigh_table *tbl, const void *pkey,
memcpy(n->primary_key, pkey, key_len);
n->dev = dev;
- dev_hold(dev);
+ dev_hold_track(dev, &n->dev_tracker, GFP_ATOMIC);
/* Protocol specific setup. */
if (tbl->constructor && (error = tbl->constructor(n)) < 0) {
@@ -770,10 +770,10 @@ struct pneigh_entry * pneigh_lookup(struct neigh_table *tbl,
write_pnet(&n->net, net);
memcpy(n->key, pkey, key_len);
n->dev = dev;
- dev_hold(dev);
+ dev_hold_track(dev, &n->dev_tracker, GFP_KERNEL);
if (tbl->pconstructor && tbl->pconstructor(n)) {
- dev_put(dev);
+ dev_put_track(dev, &n->dev_tracker);
kfree(n);
n = NULL;
goto out;
@@ -805,7 +805,7 @@ int pneigh_delete(struct neigh_table *tbl, struct net *net, const void *pkey,
write_unlock_bh(&tbl->lock);
if (tbl->pdestructor)
tbl->pdestructor(n);
- dev_put(n->dev);
+ dev_put_track(n->dev, &n->dev_tracker);
kfree(n);
return 0;
}
@@ -838,7 +838,7 @@ static int pneigh_ifdown_and_unlock(struct neigh_table *tbl,
n->next = NULL;
if (tbl->pdestructor)
tbl->pdestructor(n);
- dev_put(n->dev);
+ dev_put_track(n->dev, &n->dev_tracker);
kfree(n);
}
return -ENOENT;
@@ -879,7 +879,7 @@ void neigh_destroy(struct neighbour *neigh)
if (dev->netdev_ops->ndo_neigh_destroy)
dev->netdev_ops->ndo_neigh_destroy(dev, neigh);
- dev_put(dev);
+ dev_put_track(dev, &neigh->dev_tracker);
neigh_parms_put(neigh->parms);
neigh_dbg(2, "neigh %p is destroyed\n", neigh);
@@ -1665,13 +1665,13 @@ struct neigh_parms *neigh_parms_alloc(struct net_device *dev,
refcount_set(&p->refcnt, 1);
p->reachable_time =
neigh_rand_reach_time(NEIGH_VAR(p, BASE_REACHABLE_TIME));
- dev_hold(dev);
+ dev_hold_track(dev, &p->dev_tracker, GFP_KERNEL);
p->dev = dev;
write_pnet(&p->net, net);
p->sysctl_table = NULL;
if (ops->ndo_neigh_setup && ops->ndo_neigh_setup(dev, p)) {
- dev_put(dev);
+ dev_put_track(dev, &p->dev_tracker);
kfree(p);
return NULL;
}
@@ -1702,7 +1702,7 @@ void neigh_parms_release(struct neigh_table *tbl, struct neigh_parms *parms)
list_del(&parms->list);
parms->dead = 1;
write_unlock_bh(&tbl->lock);
- dev_put(parms->dev);
+ dev_put_track(parms->dev, &parms->dev_tracker);
call_rcu(&parms->rcu_head, neigh_rcu_free_parms);
}
EXPORT_SYMBOL(neigh_parms_release);
@@ -3770,10 +3770,6 @@ int neigh_sysctl_register(struct net_device *dev, struct neigh_parms *p,
neigh_proc_base_reachable_time;
}
- /* Don't export sysctls to unprivileged users */
- if (neigh_parms_net(p)->user_ns != &init_user_ns)
- t->neigh_vars[0].procname = NULL;
-
switch (neigh_parms_family(p)) {
case AF_INET:
p_name = "ipv4";
diff --git a/net/core/net-sysfs.c b/net/core/net-sysfs.c
index 9c01c64..53ea262 100644
--- a/net/core/net-sysfs.c
+++ b/net/core/net-sysfs.c
@@ -488,14 +488,6 @@ static ssize_t proto_down_store(struct device *dev,
struct device_attribute *attr,
const char *buf, size_t len)
{
- struct net_device *netdev = to_net_dev(dev);
-
- /* The check is also done in change_proto_down; this helps returning
- * early without hitting the trylock/restart in netdev_store.
- */
- if (!netdev->netdev_ops->ndo_change_proto_down)
- return -EOPNOTSUPP;
-
return netdev_store(dev, attr, buf, len, change_proto_down);
}
NETDEVICE_SHOW_RW(proto_down, fmt_dec);
@@ -1012,7 +1004,7 @@ static void rx_queue_release(struct kobject *kobj)
#endif
memset(kobj, 0, sizeof(*kobj));
- dev_put(queue->dev);
+ dev_put_track(queue->dev, &queue->dev_tracker);
}
static const void *rx_queue_namespace(struct kobject *kobj)
@@ -1052,7 +1044,7 @@ static int rx_queue_add_kobject(struct net_device *dev, int index)
/* Kobject_put later will trigger rx_queue_release call which
* decreases dev refcount: Take that reference here
*/
- dev_hold(queue->dev);
+ dev_hold_track(queue->dev, &queue->dev_tracker, GFP_KERNEL);
kobj->kset = dev->queues_kset;
error = kobject_init_and_add(kobj, &rx_queue_ktype, NULL,
@@ -1201,11 +1193,7 @@ static const struct sysfs_ops netdev_queue_sysfs_ops = {
static ssize_t tx_timeout_show(struct netdev_queue *queue, char *buf)
{
- unsigned long trans_timeout;
-
- spin_lock_irq(&queue->_xmit_lock);
- trans_timeout = queue->trans_timeout;
- spin_unlock_irq(&queue->_xmit_lock);
+ unsigned long trans_timeout = atomic_long_read(&queue->trans_timeout);
return sprintf(buf, fmt_ulong, trans_timeout);
}
@@ -1452,7 +1440,7 @@ static ssize_t xps_queue_show(struct net_device *dev, unsigned int index,
for (i = map->len; i--;) {
if (map->queues[i] == index) {
- set_bit(j, mask);
+ __set_bit(j, mask);
break;
}
}
@@ -1619,7 +1607,7 @@ static void netdev_queue_release(struct kobject *kobj)
struct netdev_queue *queue = to_netdev_queue(kobj);
memset(kobj, 0, sizeof(*kobj));
- dev_put(queue->dev);
+ dev_put_track(queue->dev, &queue->dev_tracker);
}
static const void *netdev_queue_namespace(struct kobject *kobj)
@@ -1659,7 +1647,7 @@ static int netdev_queue_add_kobject(struct net_device *dev, int index)
/* Kobject_put later will trigger netdev_queue_release call
* which decreases dev refcount: Take that reference here
*/
- dev_hold(queue->dev);
+ dev_hold_track(queue->dev, &queue->dev_tracker, GFP_KERNEL);
kobj->kset = dev->queues_kset;
error = kobject_init_and_add(kobj, &netdev_queue_ktype, NULL,
@@ -1706,6 +1694,13 @@ netdev_queue_update_kobjects(struct net_device *dev, int old_num, int new_num)
int i;
int error = 0;
+ /* Tx queue kobjects are allowed to be updated when a device is being
+ * unregistered, but solely to remove queues from qdiscs. Any path
+ * adding queues should be fixed.
+ */
+ WARN(dev->reg_state == NETREG_UNREGISTERING && new_num > old_num,
+ "New queues can't be registered after device unregistration.");
+
for (i = old_num; i < new_num; i++) {
error = netdev_queue_add_kobject(dev, i);
if (error) {
@@ -1820,6 +1815,9 @@ static void remove_queue_kobjects(struct net_device *dev)
net_rx_queue_update_kobjects(dev, real_rx, 0);
netdev_queue_update_kobjects(dev, real_tx, 0);
+
+ dev->real_num_rx_queues = 0;
+ dev->real_num_tx_queues = 0;
#ifdef CONFIG_SYSFS
kset_unregister(dev->queues_kset);
#endif
diff --git a/net/core/net_namespace.c b/net/core/net_namespace.c
index 202fa5e..9b7171c 100644
--- a/net/core/net_namespace.c
+++ b/net/core/net_namespace.c
@@ -311,6 +311,8 @@ static __net_init int setup_net(struct net *net, struct user_namespace *user_ns)
LIST_HEAD(net_exit_list);
refcount_set(&net->ns.count, 1);
+ ref_tracker_dir_init(&net->refcnt_tracker, 128);
+
refcount_set(&net->passive, 1);
get_random_bytes(&net->hash_mix, sizeof(u32));
preempt_disable();
@@ -635,6 +637,7 @@ static DECLARE_WORK(net_cleanup_work, cleanup_net);
void __put_net(struct net *net)
{
+ ref_tracker_dir_exit(&net->refcnt_tracker);
/* Cleanup the network namespace in process context */
if (llist_add(&net->cleanup_list, &cleanup_list))
queue_work(netns_wq, &net_cleanup_work);
diff --git a/net/core/netpoll.c b/net/core/netpoll.c
index edfc0f8..db72446 100644
--- a/net/core/netpoll.c
+++ b/net/core/netpoll.c
@@ -776,7 +776,7 @@ int netpoll_setup(struct netpoll *np)
err = __netpoll_setup(np, ndev);
if (err)
goto put;
-
+ netdev_tracker_alloc(ndev, &np->dev_tracker, GFP_KERNEL);
rtnl_unlock();
return 0;
@@ -853,7 +853,7 @@ void netpoll_cleanup(struct netpoll *np)
if (!np->dev)
goto out;
__netpoll_cleanup(np);
- dev_put(np->dev);
+ dev_put_track(np->dev, &np->dev_tracker);
np->dev = NULL;
out:
rtnl_unlock();
diff --git a/net/core/of_net.c b/net/core/of_net.c
index f1a9bf7..95a64c8 100644
--- a/net/core/of_net.c
+++ b/net/core/of_net.c
@@ -61,7 +61,7 @@ static int of_get_mac_addr_nvmem(struct device_node *np, u8 *addr)
{
struct platform_device *pdev = of_find_device_by_node(np);
struct nvmem_cell *cell;
- const void *mac;
+ const void *buf;
size_t len;
int ret;
@@ -78,21 +78,32 @@ static int of_get_mac_addr_nvmem(struct device_node *np, u8 *addr)
if (IS_ERR(cell))
return PTR_ERR(cell);
- mac = nvmem_cell_read(cell, &len);
+ buf = nvmem_cell_read(cell, &len);
nvmem_cell_put(cell);
- if (IS_ERR(mac))
- return PTR_ERR(mac);
+ if (IS_ERR(buf))
+ return PTR_ERR(buf);
- if (len != ETH_ALEN || !is_valid_ether_addr(mac)) {
- kfree(mac);
- return -EINVAL;
+ ret = 0;
+ if (len == ETH_ALEN) {
+ if (is_valid_ether_addr(buf))
+ memcpy(addr, buf, ETH_ALEN);
+ else
+ ret = -EINVAL;
+ } else if (len == 3 * ETH_ALEN - 1) {
+ u8 mac[ETH_ALEN];
+
+ if (mac_pton(buf, mac))
+ memcpy(addr, mac, ETH_ALEN);
+ else
+ ret = -EINVAL;
+ } else {
+ ret = -EINVAL;
}
- memcpy(addr, mac, ETH_ALEN);
- kfree(mac);
+ kfree(buf);
- return 0;
+ return ret;
}
/**
diff --git a/net/core/page_pool.c b/net/core/page_pool.c
index 1a69784..bd62c01 100644
--- a/net/core/page_pool.c
+++ b/net/core/page_pool.c
@@ -130,9 +130,6 @@ static struct page *page_pool_refill_alloc_cache(struct page_pool *pool)
pref_nid = numa_mem_id(); /* will be zero like page_to_nid() */
#endif
- /* Slower-path: Get pages from locked ring queue */
- spin_lock(&r->consumer_lock);
-
/* Refill alloc array, but only if NUMA match */
do {
page = __ptr_ring_consume(r);
@@ -157,7 +154,6 @@ static struct page *page_pool_refill_alloc_cache(struct page_pool *pool)
if (likely(pool->alloc.count > 0))
page = pool->alloc.cache[--pool->alloc.count];
- spin_unlock(&r->consumer_lock);
return page;
}
@@ -217,6 +213,8 @@ static void page_pool_set_pp_info(struct page_pool *pool,
{
page->pp = pool;
page->pp_magic |= PP_SIGNATURE;
+ if (pool->p.init_callback)
+ pool->p.init_callback(page, pool->p.init_arg);
}
static void page_pool_clear_pp_info(struct page *page)
@@ -691,10 +689,12 @@ static void page_pool_release_retry(struct work_struct *wq)
schedule_delayed_work(&pool->release_dw, DEFER_TIME);
}
-void page_pool_use_xdp_mem(struct page_pool *pool, void (*disconnect)(void *))
+void page_pool_use_xdp_mem(struct page_pool *pool, void (*disconnect)(void *),
+ struct xdp_mem_info *mem)
{
refcount_inc(&pool->user_cnt);
pool->disconnect = disconnect;
+ pool->xdp_mem_id = mem->id;
}
void page_pool_destroy(struct page_pool *pool)
diff --git a/net/core/pktgen.c b/net/core/pktgen.c
index a3d74e2..560a5e7 100644
--- a/net/core/pktgen.c
+++ b/net/core/pktgen.c
@@ -410,6 +410,7 @@ struct pktgen_dev {
* device name (not when the inject is
* started as it used to do.)
*/
+ netdevice_tracker dev_tracker;
char odevname[32];
struct flow_state *flows;
unsigned int cflows; /* Concurrent flows (config) */
@@ -2099,7 +2100,7 @@ static int pktgen_setup_dev(const struct pktgen_net *pn,
/* Clean old setups */
if (pkt_dev->odev) {
- dev_put(pkt_dev->odev);
+ dev_put_track(pkt_dev->odev, &pkt_dev->dev_tracker);
pkt_dev->odev = NULL;
}
@@ -2117,6 +2118,7 @@ static int pktgen_setup_dev(const struct pktgen_net *pn,
err = -ENETDOWN;
} else {
pkt_dev->odev = odev;
+ netdev_tracker_alloc(odev, &pkt_dev->dev_tracker, GFP_KERNEL);
return 0;
}
@@ -3805,7 +3807,7 @@ static int pktgen_add_device(struct pktgen_thread *t, const char *ifname)
return add_dev_to_thread(t, pkt_dev);
out2:
- dev_put(pkt_dev->odev);
+ dev_put_track(pkt_dev->odev, &pkt_dev->dev_tracker);
out1:
#ifdef CONFIG_XFRM
free_SAs(pkt_dev);
@@ -3899,7 +3901,7 @@ static int pktgen_remove_device(struct pktgen_thread *t,
/* Dis-associate from the interface */
if (pkt_dev->odev) {
- dev_put(pkt_dev->odev);
+ dev_put_track(pkt_dev->odev, &pkt_dev->dev_tracker);
pkt_dev->odev = NULL;
}
diff --git a/net/core/rtnetlink.c b/net/core/rtnetlink.c
index 2af8aee..e476403 100644
--- a/net/core/rtnetlink.c
+++ b/net/core/rtnetlink.c
@@ -842,9 +842,9 @@ static void set_operstate(struct net_device *dev, unsigned char transition)
}
if (dev->operstate != operstate) {
- write_lock_bh(&dev_base_lock);
+ write_lock(&dev_base_lock);
dev->operstate = operstate;
- write_unlock_bh(&dev_base_lock);
+ write_unlock(&dev_base_lock);
netdev_state_change(dev);
}
}
@@ -1026,6 +1026,7 @@ static noinline size_t if_nlmsg_size(const struct net_device *dev,
+ nla_total_size(4) /* IFLA_NUM_RX_QUEUES */
+ nla_total_size(4) /* IFLA_GSO_MAX_SEGS */
+ nla_total_size(4) /* IFLA_GSO_MAX_SIZE */
+ + nla_total_size(4) /* IFLA_GRO_MAX_SIZE */
+ nla_total_size(1) /* IFLA_OPERSTATE */
+ nla_total_size(1) /* IFLA_LINKMODE */
+ nla_total_size(4) /* IFLA_CARRIER_CHANGES */
@@ -1728,6 +1729,7 @@ static int rtnl_fill_ifinfo(struct sk_buff *skb,
nla_put_u32(skb, IFLA_NUM_TX_QUEUES, dev->num_tx_queues) ||
nla_put_u32(skb, IFLA_GSO_MAX_SEGS, dev->gso_max_segs) ||
nla_put_u32(skb, IFLA_GSO_MAX_SIZE, dev->gso_max_size) ||
+ nla_put_u32(skb, IFLA_GRO_MAX_SIZE, dev->gro_max_size) ||
#ifdef CONFIG_RPS
nla_put_u32(skb, IFLA_NUM_RX_QUEUES, dev->num_rx_queues) ||
#endif
@@ -1880,6 +1882,7 @@ static const struct nla_policy ifla_policy[IFLA_MAX+1] = {
[IFLA_PROTO_DOWN_REASON] = { .type = NLA_NESTED },
[IFLA_NEW_IFINDEX] = NLA_POLICY_MIN(NLA_S32, 1),
[IFLA_PARENT_DEV_NAME] = { .type = NLA_NUL_STRING },
+ [IFLA_GRO_MAX_SIZE] = { .type = NLA_U32 },
};
static const struct nla_policy ifla_info_policy[IFLA_INFO_MAX+1] = {
@@ -2299,6 +2302,14 @@ static int validate_linkmsg(struct net_device *dev, struct nlattr *tb[],
}
}
+ if (tb[IFLA_GRO_MAX_SIZE]) {
+ u32 gro_max_size = nla_get_u32(tb[IFLA_GRO_MAX_SIZE]);
+
+ if (gro_max_size > GRO_MAX_SIZE) {
+ NL_SET_ERR_MSG(extack, "too big gro_max_size");
+ return -EINVAL;
+ }
+ }
return 0;
}
@@ -2539,13 +2550,12 @@ static int do_set_proto_down(struct net_device *dev,
struct netlink_ext_ack *extack)
{
struct nlattr *pdreason[IFLA_PROTO_DOWN_REASON_MAX + 1];
- const struct net_device_ops *ops = dev->netdev_ops;
unsigned long mask = 0;
u32 value;
bool proto_down;
int err;
- if (!ops->ndo_change_proto_down) {
+ if (!(dev->priv_flags & IFF_CHANGE_PROTO_DOWN)) {
NL_SET_ERR_MSG(extack, "Protodown not supported by device");
return -EOPNOTSUPP;
}
@@ -2768,7 +2778,16 @@ static int do_setlink(const struct sk_buff *skb,
}
if (dev->gso_max_segs ^ max_segs) {
- dev->gso_max_segs = max_segs;
+ netif_set_gso_max_segs(dev, max_segs);
+ status |= DO_SETLINK_MODIFIED;
+ }
+ }
+
+ if (tb[IFLA_GRO_MAX_SIZE]) {
+ u32 gro_max_size = nla_get_u32(tb[IFLA_GRO_MAX_SIZE]);
+
+ if (dev->gro_max_size ^ gro_max_size) {
+ netif_set_gro_max_size(dev, gro_max_size);
status |= DO_SETLINK_MODIFIED;
}
}
@@ -2779,11 +2798,11 @@ static int do_setlink(const struct sk_buff *skb,
if (tb[IFLA_LINKMODE]) {
unsigned char value = nla_get_u8(tb[IFLA_LINKMODE]);
- write_lock_bh(&dev_base_lock);
+ write_lock(&dev_base_lock);
if (dev->link_mode ^ value)
status |= DO_SETLINK_NOTIFY;
dev->link_mode = value;
- write_unlock_bh(&dev_base_lock);
+ write_unlock(&dev_base_lock);
}
if (tb[IFLA_VFINFO_LIST]) {
@@ -3222,7 +3241,9 @@ struct net_device *rtnl_create_link(struct net *net, const char *ifname,
if (tb[IFLA_GSO_MAX_SIZE])
netif_set_gso_max_size(dev, nla_get_u32(tb[IFLA_GSO_MAX_SIZE]));
if (tb[IFLA_GSO_MAX_SEGS])
- dev->gso_max_segs = nla_get_u32(tb[IFLA_GSO_MAX_SEGS]);
+ netif_set_gso_max_segs(dev, nla_get_u32(tb[IFLA_GSO_MAX_SEGS]));
+ if (tb[IFLA_GRO_MAX_SIZE])
+ netif_set_gro_max_size(dev, nla_get_u32(tb[IFLA_GRO_MAX_SIZE]));
return dev;
}
diff --git a/net/core/secure_seq.c b/net/core/secure_seq.c
index b5bc680..9b84437 100644
--- a/net/core/secure_seq.c
+++ b/net/core/secure_seq.c
@@ -19,8 +19,8 @@
#include <linux/in6.h>
#include <net/tcp.h>
-static siphash_key_t net_secret __read_mostly;
-static siphash_key_t ts_secret __read_mostly;
+static siphash_aligned_key_t net_secret;
+static siphash_aligned_key_t ts_secret;
static __always_inline void net_secret_init(void)
{
diff --git a/net/core/skbuff.c b/net/core/skbuff.c
index 909db87..0118f0a 100644
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c
@@ -759,21 +759,23 @@ void __kfree_skb(struct sk_buff *skb)
EXPORT_SYMBOL(__kfree_skb);
/**
- * kfree_skb - free an sk_buff
+ * kfree_skb_reason - free an sk_buff with special reason
* @skb: buffer to free
+ * @reason: reason why this skb is dropped
*
* Drop a reference to the buffer and free it if the usage count has
- * hit zero.
+ * hit zero. Meanwhile, pass the drop reason to 'kfree_skb'
+ * tracepoint.
*/
-void kfree_skb(struct sk_buff *skb)
+void kfree_skb_reason(struct sk_buff *skb, enum skb_drop_reason reason)
{
if (!skb_unref(skb))
return;
- trace_kfree_skb(skb, __builtin_return_address(0));
+ trace_kfree_skb(skb, __builtin_return_address(0), reason);
__kfree_skb(skb);
}
-EXPORT_SYMBOL(kfree_skb);
+EXPORT_SYMBOL(kfree_skb_reason);
void kfree_skb_list(struct sk_buff *segs)
{
@@ -992,12 +994,10 @@ void napi_consume_skb(struct sk_buff *skb, int budget)
}
EXPORT_SYMBOL(napi_consume_skb);
-/* Make sure a field is enclosed inside headers_start/headers_end section */
+/* Make sure a field is contained by headers group */
#define CHECK_SKB_FIELD(field) \
- BUILD_BUG_ON(offsetof(struct sk_buff, field) < \
- offsetof(struct sk_buff, headers_start)); \
- BUILD_BUG_ON(offsetof(struct sk_buff, field) > \
- offsetof(struct sk_buff, headers_end)); \
+ BUILD_BUG_ON(offsetof(struct sk_buff, field) != \
+ offsetof(struct sk_buff, headers.field)); \
static void __copy_skb_header(struct sk_buff *new, const struct sk_buff *old)
{
@@ -1009,14 +1009,12 @@ static void __copy_skb_header(struct sk_buff *new, const struct sk_buff *old)
__skb_ext_copy(new, old);
__nf_copy(new, old, false);
- /* Note : this field could be in headers_start/headers_end section
+ /* Note : this field could be in the headers group.
* It is not yet because we do not want to have a 16 bit hole
*/
new->queue_mapping = old->queue_mapping;
- memcpy(&new->headers_start, &old->headers_start,
- offsetof(struct sk_buff, headers_end) -
- offsetof(struct sk_buff, headers_start));
+ memcpy(&new->headers, &old->headers, sizeof(new->headers));
CHECK_SKB_FIELD(protocol);
CHECK_SKB_FIELD(csum);
CHECK_SKB_FIELD(hash);
@@ -2028,6 +2026,30 @@ void *skb_pull(struct sk_buff *skb, unsigned int len)
EXPORT_SYMBOL(skb_pull);
/**
+ * skb_pull_data - remove data from the start of a buffer returning its
+ * original position.
+ * @skb: buffer to use
+ * @len: amount of data to remove
+ *
+ * This function removes data from the start of a buffer, returning
+ * the memory to the headroom. A pointer to the original data in the buffer
+ * is returned after checking if there is enough data to pull. Once the
+ * data has been pulled future pushes will overwrite the old data.
+ */
+void *skb_pull_data(struct sk_buff *skb, size_t len)
+{
+ void *data = skb->data;
+
+ if (skb->len < len)
+ return NULL;
+
+ skb_pull(skb, len);
+
+ return data;
+}
+EXPORT_SYMBOL(skb_pull_data);
+
+/**
* skb_trim - remove end from a buffer
* @skb: buffer to alter
* @len: new length
@@ -3919,32 +3941,6 @@ struct sk_buff *skb_segment_list(struct sk_buff *skb,
}
EXPORT_SYMBOL_GPL(skb_segment_list);
-int skb_gro_receive_list(struct sk_buff *p, struct sk_buff *skb)
-{
- if (unlikely(p->len + skb->len >= 65536))
- return -E2BIG;
-
- if (NAPI_GRO_CB(p)->last == p)
- skb_shinfo(p)->frag_list = skb;
- else
- NAPI_GRO_CB(p)->last->next = skb;
-
- skb_pull(skb, skb_gro_offset(skb));
-
- NAPI_GRO_CB(p)->last = skb;
- NAPI_GRO_CB(p)->count++;
- p->data_len += skb->len;
-
- /* sk owenrship - if any - completely transferred to the aggregated packet */
- skb->destructor = NULL;
- p->truesize += skb->truesize;
- p->len += skb->len;
-
- NAPI_GRO_CB(skb)->same_flow = 1;
-
- return 0;
-}
-
/**
* skb_segment - Perform protocol segmentation on skb.
* @head_skb: buffer to segment
@@ -4297,122 +4293,6 @@ struct sk_buff *skb_segment(struct sk_buff *head_skb,
}
EXPORT_SYMBOL_GPL(skb_segment);
-int skb_gro_receive(struct sk_buff *p, struct sk_buff *skb)
-{
- struct skb_shared_info *pinfo, *skbinfo = skb_shinfo(skb);
- unsigned int offset = skb_gro_offset(skb);
- unsigned int headlen = skb_headlen(skb);
- unsigned int len = skb_gro_len(skb);
- unsigned int delta_truesize;
- unsigned int new_truesize;
- struct sk_buff *lp;
-
- if (unlikely(p->len + len >= 65536 || NAPI_GRO_CB(skb)->flush))
- return -E2BIG;
-
- lp = NAPI_GRO_CB(p)->last;
- pinfo = skb_shinfo(lp);
-
- if (headlen <= offset) {
- skb_frag_t *frag;
- skb_frag_t *frag2;
- int i = skbinfo->nr_frags;
- int nr_frags = pinfo->nr_frags + i;
-
- if (nr_frags > MAX_SKB_FRAGS)
- goto merge;
-
- offset -= headlen;
- pinfo->nr_frags = nr_frags;
- skbinfo->nr_frags = 0;
-
- frag = pinfo->frags + nr_frags;
- frag2 = skbinfo->frags + i;
- do {
- *--frag = *--frag2;
- } while (--i);
-
- skb_frag_off_add(frag, offset);
- skb_frag_size_sub(frag, offset);
-
- /* all fragments truesize : remove (head size + sk_buff) */
- new_truesize = SKB_TRUESIZE(skb_end_offset(skb));
- delta_truesize = skb->truesize - new_truesize;
-
- skb->truesize = new_truesize;
- skb->len -= skb->data_len;
- skb->data_len = 0;
-
- NAPI_GRO_CB(skb)->free = NAPI_GRO_FREE;
- goto done;
- } else if (skb->head_frag) {
- int nr_frags = pinfo->nr_frags;
- skb_frag_t *frag = pinfo->frags + nr_frags;
- struct page *page = virt_to_head_page(skb->head);
- unsigned int first_size = headlen - offset;
- unsigned int first_offset;
-
- if (nr_frags + 1 + skbinfo->nr_frags > MAX_SKB_FRAGS)
- goto merge;
-
- first_offset = skb->data -
- (unsigned char *)page_address(page) +
- offset;
-
- pinfo->nr_frags = nr_frags + 1 + skbinfo->nr_frags;
-
- __skb_frag_set_page(frag, page);
- skb_frag_off_set(frag, first_offset);
- skb_frag_size_set(frag, first_size);
-
- memcpy(frag + 1, skbinfo->frags, sizeof(*frag) * skbinfo->nr_frags);
- /* We dont need to clear skbinfo->nr_frags here */
-
- new_truesize = SKB_DATA_ALIGN(sizeof(struct sk_buff));
- delta_truesize = skb->truesize - new_truesize;
- skb->truesize = new_truesize;
- NAPI_GRO_CB(skb)->free = NAPI_GRO_FREE_STOLEN_HEAD;
- goto done;
- }
-
-merge:
- /* sk owenrship - if any - completely transferred to the aggregated packet */
- skb->destructor = NULL;
- delta_truesize = skb->truesize;
- if (offset > headlen) {
- unsigned int eat = offset - headlen;
-
- skb_frag_off_add(&skbinfo->frags[0], eat);
- skb_frag_size_sub(&skbinfo->frags[0], eat);
- skb->data_len -= eat;
- skb->len -= eat;
- offset = headlen;
- }
-
- __skb_pull(skb, offset);
-
- if (NAPI_GRO_CB(p)->last == p)
- skb_shinfo(p)->frag_list = skb;
- else
- NAPI_GRO_CB(p)->last->next = skb;
- NAPI_GRO_CB(p)->last = skb;
- __skb_header_release(skb);
- lp = p;
-
-done:
- NAPI_GRO_CB(p)->count++;
- p->data_len += len;
- p->truesize += delta_truesize;
- p->len += len;
- if (lp != p) {
- lp->data_len += len;
- lp->truesize += delta_truesize;
- lp->len += len;
- }
- NAPI_GRO_CB(skb)->same_flow = 1;
- return 0;
-}
-
#ifdef CONFIG_SKB_EXTENSIONS
#define SKB_EXT_ALIGN_VALUE 8
#define SKB_EXT_CHUNKSIZEOF(x) (ALIGN((sizeof(x)), SKB_EXT_ALIGN_VALUE) / SKB_EXT_ALIGN_VALUE)
@@ -4849,8 +4729,7 @@ static void __skb_complete_tx_timestamp(struct sk_buff *skb,
serr->header.h4.iif = skb->dev ? skb->dev->ifindex : 0;
if (sk->sk_tsflags & SOF_TIMESTAMPING_OPT_ID) {
serr->ee.ee_data = skb_shinfo(skb)->tskey;
- if (sk->sk_protocol == IPPROTO_TCP &&
- sk->sk_type == SOCK_STREAM)
+ if (sk_is_tcp(sk))
serr->ee.ee_data -= sk->sk_tskey;
}
@@ -4919,8 +4798,7 @@ void __skb_tstamp_tx(struct sk_buff *orig_skb,
if (tsonly) {
#ifdef CONFIG_INET
if ((sk->sk_tsflags & SOF_TIMESTAMPING_OPT_STATS) &&
- sk->sk_protocol == IPPROTO_TCP &&
- sk->sk_type == SOCK_STREAM) {
+ sk_is_tcp(sk)) {
skb = tcp_get_timestamping_opt_stats(sk, orig_skb,
ack_skb);
opt_stats = true;
diff --git a/net/core/sock.c b/net/core/sock.c
index 41e91d0..e21485a 100644
--- a/net/core/sock.c
+++ b/net/core/sock.c
@@ -144,8 +144,6 @@
static DEFINE_MUTEX(proto_list_mutex);
static LIST_HEAD(proto_list);
-static void sock_inuse_add(struct net *net, int val);
-
/**
* sk_ns_capable - General socket capability test
* @sk: Socket to use a capability on or through
@@ -327,7 +325,10 @@ int __sk_backlog_rcv(struct sock *sk, struct sk_buff *skb)
BUG_ON(!sock_flag(sk, SOCK_MEMALLOC));
noreclaim_flag = memalloc_noreclaim_save();
- ret = sk->sk_backlog_rcv(sk, skb);
+ ret = INDIRECT_CALL_INET(sk->sk_backlog_rcv,
+ tcp_v6_do_rcv,
+ tcp_v4_do_rcv,
+ sk, skb);
memalloc_noreclaim_restore(noreclaim_flag);
return ret;
@@ -872,8 +873,7 @@ int sock_set_timestamping(struct sock *sk, int optname,
if (val & SOF_TIMESTAMPING_OPT_ID &&
!(sk->sk_tsflags & SOF_TIMESTAMPING_OPT_ID)) {
- if (sk->sk_protocol == IPPROTO_TCP &&
- sk->sk_type == SOCK_STREAM) {
+ if (sk_is_tcp(sk)) {
if ((1 << sk->sk_state) &
(TCPF_CLOSE | TCPF_LISTEN))
return -EINVAL;
@@ -1135,6 +1135,7 @@ int sock_setsockopt(struct socket *sock, int level, int optname,
case SO_PRIORITY:
if ((val >= 0 && val <= 6) ||
+ ns_capable(sock_net(sk)->user_ns, CAP_NET_RAW) ||
ns_capable(sock_net(sk)->user_ns, CAP_NET_ADMIN))
sk->sk_priority = val;
else
@@ -1280,7 +1281,8 @@ int sock_setsockopt(struct socket *sock, int level, int optname,
clear_bit(SOCK_PASSSEC, &sock->flags);
break;
case SO_MARK:
- if (!ns_capable(sock_net(sk)->user_ns, CAP_NET_ADMIN)) {
+ if (!ns_capable(sock_net(sk)->user_ns, CAP_NET_RAW) &&
+ !ns_capable(sock_net(sk)->user_ns, CAP_NET_ADMIN)) {
ret = -EPERM;
break;
}
@@ -1370,8 +1372,7 @@ int sock_setsockopt(struct socket *sock, int level, int optname,
case SO_ZEROCOPY:
if (sk->sk_family == PF_INET || sk->sk_family == PF_INET6) {
- if (!((sk->sk_type == SOCK_STREAM &&
- sk->sk_protocol == IPPROTO_TCP) ||
+ if (!(sk_is_tcp(sk) ||
(sk->sk_type == SOCK_DGRAM &&
sk->sk_protocol == IPPROTO_UDP)))
ret = -ENOTSUPP;
@@ -1982,7 +1983,7 @@ struct sock *sk_alloc(struct net *net, int family, gfp_t priority,
sock_lock_init(sk);
sk->sk_net_refcnt = kern ? 0 : 1;
if (likely(sk->sk_net_refcnt)) {
- get_net(net);
+ get_net_track(net, &sk->ns_tracker, priority);
sock_inuse_add(net, 1);
}
@@ -2038,7 +2039,7 @@ static void __sk_destruct(struct rcu_head *head)
put_pid(sk->sk_peer_pid);
if (likely(sk->sk_net_refcnt))
- put_net(sock_net(sk));
+ put_net_track(sock_net(sk), &sk->ns_tracker);
sk_prot_free(sk->sk_prot_creator, sk);
}
@@ -2125,7 +2126,7 @@ struct sock *sk_clone_lock(const struct sock *sk, const gfp_t priority)
/* SANITY */
if (likely(newsk->sk_net_refcnt)) {
- get_net(sock_net(newsk));
+ get_net_track(sock_net(newsk), &newsk->ns_tracker, priority);
sock_inuse_add(sock_net(newsk), 1);
}
sk_node_init(&newsk->sk_node);
@@ -2246,17 +2247,22 @@ void sk_setup_caps(struct sock *sk, struct dst_entry *dst)
u32 max_segs = 1;
sk_dst_set(sk, dst);
- sk->sk_route_caps = dst->dev->features | sk->sk_route_forced_caps;
+ sk->sk_route_caps = dst->dev->features;
+ if (sk_is_tcp(sk))
+ sk->sk_route_caps |= NETIF_F_GSO;
if (sk->sk_route_caps & NETIF_F_GSO)
sk->sk_route_caps |= NETIF_F_GSO_SOFTWARE;
- sk->sk_route_caps &= ~sk->sk_route_nocaps;
+ if (unlikely(sk->sk_gso_disabled))
+ sk->sk_route_caps &= ~NETIF_F_GSO_MASK;
if (sk_can_gso(sk)) {
if (dst->header_len && !xfrm_dst_offload_ok(dst)) {
sk->sk_route_caps &= ~NETIF_F_GSO_MASK;
} else {
sk->sk_route_caps |= NETIF_F_SG | NETIF_F_HW_CSUM;
- sk->sk_gso_max_size = dst->dev->gso_max_size;
- max_segs = max_t(u32, dst->dev->gso_max_segs, 1);
+ /* pairs with the WRITE_ONCE() in netif_set_gso_max_size() */
+ sk->sk_gso_max_size = READ_ONCE(dst->dev->gso_max_size);
+ /* pairs with the WRITE_ONCE() in netif_set_gso_max_segs() */
+ max_segs = max_t(u32, READ_ONCE(dst->dev->gso_max_segs), 1);
}
}
sk->sk_gso_max_segs = max_segs;
@@ -3286,7 +3292,7 @@ void lock_sock_nested(struct sock *sk, int subclass)
might_sleep();
spin_lock_bh(&sk->sk_lock.slock);
- if (sk->sk_lock.owned)
+ if (sock_owned_by_user_nocheck(sk))
__lock_sock(sk);
sk->sk_lock.owned = 1;
spin_unlock_bh(&sk->sk_lock.slock);
@@ -3317,7 +3323,7 @@ bool __lock_sock_fast(struct sock *sk) __acquires(&sk->sk_lock.slock)
might_sleep();
spin_lock_bh(&sk->sk_lock.slock);
- if (!sk->sk_lock.owned) {
+ if (!sock_owned_by_user_nocheck(sk)) {
/*
* Fast path return with bottom halves disabled and
* sock::sk_lock.slock held.
@@ -3532,19 +3538,8 @@ void sk_get_meminfo(const struct sock *sk, u32 *mem)
}
#ifdef CONFIG_PROC_FS
-#define PROTO_INUSE_NR 64 /* should be enough for the first time */
-struct prot_inuse {
- int val[PROTO_INUSE_NR];
-};
-
static DECLARE_BITMAP(proto_inuse_idx, PROTO_INUSE_NR);
-void sock_prot_inuse_add(struct net *net, struct proto *prot, int val)
-{
- __this_cpu_add(net->core.prot_inuse->val[prot->inuse_idx], val);
-}
-EXPORT_SYMBOL_GPL(sock_prot_inuse_add);
-
int sock_prot_inuse_get(struct net *net, struct proto *prot)
{
int cpu, idx = prot->inuse_idx;
@@ -3557,17 +3552,12 @@ int sock_prot_inuse_get(struct net *net, struct proto *prot)
}
EXPORT_SYMBOL_GPL(sock_prot_inuse_get);
-static void sock_inuse_add(struct net *net, int val)
-{
- this_cpu_add(*net->core.sock_inuse, val);
-}
-
int sock_inuse_get(struct net *net)
{
int cpu, res = 0;
for_each_possible_cpu(cpu)
- res += *per_cpu_ptr(net->core.sock_inuse, cpu);
+ res += per_cpu_ptr(net->core.prot_inuse, cpu)->all;
return res;
}
@@ -3579,22 +3569,12 @@ static int __net_init sock_inuse_init_net(struct net *net)
net->core.prot_inuse = alloc_percpu(struct prot_inuse);
if (net->core.prot_inuse == NULL)
return -ENOMEM;
-
- net->core.sock_inuse = alloc_percpu(int);
- if (net->core.sock_inuse == NULL)
- goto out;
-
return 0;
-
-out:
- free_percpu(net->core.prot_inuse);
- return -ENOMEM;
}
static void __net_exit sock_inuse_exit_net(struct net *net)
{
free_percpu(net->core.prot_inuse);
- free_percpu(net->core.sock_inuse);
}
static struct pernet_operations net_inuse_ops = {
@@ -3640,9 +3620,6 @@ static inline void release_proto_idx(struct proto *prot)
{
}
-static void sock_inuse_add(struct net *net, int val)
-{
-}
#endif
static void tw_prot_cleanup(struct timewait_sock_ops *twsk_prot)
diff --git a/net/core/sock_diag.c b/net/core/sock_diag.c
index c9c45b9..f7cf74c 100644
--- a/net/core/sock_diag.c
+++ b/net/core/sock_diag.c
@@ -1,5 +1,6 @@
/* License: GPL */
+#include <linux/filter.h>
#include <linux/mutex.h>
#include <linux/socket.h>
#include <linux/skbuff.h>
diff --git a/net/core/sock_map.c b/net/core/sock_map.c
index 4ca4b11..1827669 100644
--- a/net/core/sock_map.c
+++ b/net/core/sock_map.c
@@ -292,15 +292,23 @@ static int sock_map_link(struct bpf_map *map, struct sock *sk)
if (skb_verdict)
psock_set_prog(&psock->progs.skb_verdict, skb_verdict);
+ /* msg_* and stream_* programs references tracked in psock after this
+ * point. Reference dec and cleanup will occur through psock destructor
+ */
ret = sock_map_init_proto(sk, psock);
- if (ret < 0)
- goto out_drop;
+ if (ret < 0) {
+ sk_psock_put(sk, psock);
+ goto out;
+ }
write_lock_bh(&sk->sk_callback_lock);
if (stream_parser && stream_verdict && !psock->saved_data_ready) {
ret = sk_psock_init_strp(sk, psock);
- if (ret)
- goto out_unlock_drop;
+ if (ret) {
+ write_unlock_bh(&sk->sk_callback_lock);
+ sk_psock_put(sk, psock);
+ goto out;
+ }
sk_psock_start_strp(sk, psock);
} else if (!stream_parser && stream_verdict && !psock->saved_data_ready) {
sk_psock_start_verdict(sk,psock);
@@ -309,10 +317,6 @@ static int sock_map_link(struct bpf_map *map, struct sock *sk)
}
write_unlock_bh(&sk->sk_callback_lock);
return 0;
-out_unlock_drop:
- write_unlock_bh(&sk->sk_callback_lock);
-out_drop:
- sk_psock_put(sk, psock);
out_progs:
if (skb_verdict)
bpf_prog_put(skb_verdict);
@@ -325,6 +329,7 @@ static int sock_map_link(struct bpf_map *map, struct sock *sk)
out_put_stream_verdict:
if (stream_verdict)
bpf_prog_put(stream_verdict);
+out:
return ret;
}
@@ -1564,7 +1569,7 @@ static struct bpf_iter_reg sock_map_iter_reg = {
.ctx_arg_info_size = 2,
.ctx_arg_info = {
{ offsetof(struct bpf_iter__sockmap, key),
- PTR_TO_RDONLY_BUF_OR_NULL },
+ PTR_TO_BUF | PTR_MAYBE_NULL | MEM_RDONLY },
{ offsetof(struct bpf_iter__sockmap, sk),
PTR_TO_BTF_ID_OR_NULL },
},
diff --git a/net/core/sysctl_net_core.c b/net/core/sysctl_net_core.c
index 5f88526..7b4d485 100644
--- a/net/core/sysctl_net_core.c
+++ b/net/core/sysctl_net_core.c
@@ -6,6 +6,7 @@
* Added /proc/sys/net/core directory entry (empty =) ). [MS]
*/
+#include <linux/filter.h>
#include <linux/mm.h>
#include <linux/sysctl.h>
#include <linux/module.h>
diff --git a/net/core/xdp.c b/net/core/xdp.c
index 5ddc29f..7aba355 100644
--- a/net/core/xdp.c
+++ b/net/core/xdp.c
@@ -110,20 +110,15 @@ static void mem_allocator_disconnect(void *allocator)
mutex_unlock(&mem_id_lock);
}
-void xdp_rxq_info_unreg_mem_model(struct xdp_rxq_info *xdp_rxq)
+void xdp_unreg_mem_model(struct xdp_mem_info *mem)
{
struct xdp_mem_allocator *xa;
- int type = xdp_rxq->mem.type;
- int id = xdp_rxq->mem.id;
+ int type = mem->type;
+ int id = mem->id;
/* Reset mem info to defaults */
- xdp_rxq->mem.id = 0;
- xdp_rxq->mem.type = 0;
-
- if (xdp_rxq->reg_state != REG_STATE_REGISTERED) {
- WARN(1, "Missing register, driver bug");
- return;
- }
+ mem->id = 0;
+ mem->type = 0;
if (id == 0)
return;
@@ -135,6 +130,17 @@ void xdp_rxq_info_unreg_mem_model(struct xdp_rxq_info *xdp_rxq)
rcu_read_unlock();
}
}
+EXPORT_SYMBOL_GPL(xdp_unreg_mem_model);
+
+void xdp_rxq_info_unreg_mem_model(struct xdp_rxq_info *xdp_rxq)
+{
+ if (xdp_rxq->reg_state != REG_STATE_REGISTERED) {
+ WARN(1, "Missing register, driver bug");
+ return;
+ }
+
+ xdp_unreg_mem_model(&xdp_rxq->mem);
+}
EXPORT_SYMBOL_GPL(xdp_rxq_info_unreg_mem_model);
void xdp_rxq_info_unreg(struct xdp_rxq_info *xdp_rxq)
@@ -159,6 +165,11 @@ static void xdp_rxq_info_init(struct xdp_rxq_info *xdp_rxq)
int xdp_rxq_info_reg(struct xdp_rxq_info *xdp_rxq,
struct net_device *dev, u32 queue_index, unsigned int napi_id)
{
+ if (!dev) {
+ WARN(1, "Missing net_device from driver");
+ return -ENODEV;
+ }
+
if (xdp_rxq->reg_state == REG_STATE_UNUSED) {
WARN(1, "Driver promised not to register this");
return -EINVAL;
@@ -169,11 +180,6 @@ int xdp_rxq_info_reg(struct xdp_rxq_info *xdp_rxq,
xdp_rxq_info_unreg(xdp_rxq);
}
- if (!dev) {
- WARN(1, "Missing net_device from driver");
- return -ENODEV;
- }
-
/* State either UNREGISTERED or NEW */
xdp_rxq_info_init(xdp_rxq);
xdp_rxq->dev = dev;
@@ -259,28 +265,24 @@ static bool __is_supported_mem_type(enum xdp_mem_type type)
return true;
}
-int xdp_rxq_info_reg_mem_model(struct xdp_rxq_info *xdp_rxq,
- enum xdp_mem_type type, void *allocator)
+static struct xdp_mem_allocator *__xdp_reg_mem_model(struct xdp_mem_info *mem,
+ enum xdp_mem_type type,
+ void *allocator)
{
struct xdp_mem_allocator *xdp_alloc;
gfp_t gfp = GFP_KERNEL;
int id, errno, ret;
void *ptr;
- if (xdp_rxq->reg_state != REG_STATE_REGISTERED) {
- WARN(1, "Missing register, driver bug");
- return -EFAULT;
- }
-
if (!__is_supported_mem_type(type))
- return -EOPNOTSUPP;
+ return ERR_PTR(-EOPNOTSUPP);
- xdp_rxq->mem.type = type;
+ mem->type = type;
if (!allocator) {
if (type == MEM_TYPE_PAGE_POOL)
- return -EINVAL; /* Setup time check page_pool req */
- return 0;
+ return ERR_PTR(-EINVAL); /* Setup time check page_pool req */
+ return NULL;
}
/* Delay init of rhashtable to save memory if feature isn't used */
@@ -290,13 +292,13 @@ int xdp_rxq_info_reg_mem_model(struct xdp_rxq_info *xdp_rxq,
mutex_unlock(&mem_id_lock);
if (ret < 0) {
WARN_ON(1);
- return ret;
+ return ERR_PTR(ret);
}
}
xdp_alloc = kzalloc(sizeof(*xdp_alloc), gfp);
if (!xdp_alloc)
- return -ENOMEM;
+ return ERR_PTR(-ENOMEM);
mutex_lock(&mem_id_lock);
id = __mem_id_cyclic_get(gfp);
@@ -304,31 +306,61 @@ int xdp_rxq_info_reg_mem_model(struct xdp_rxq_info *xdp_rxq,
errno = id;
goto err;
}
- xdp_rxq->mem.id = id;
- xdp_alloc->mem = xdp_rxq->mem;
+ mem->id = id;
+ xdp_alloc->mem = *mem;
xdp_alloc->allocator = allocator;
/* Insert allocator into ID lookup table */
ptr = rhashtable_insert_slow(mem_id_ht, &id, &xdp_alloc->node);
if (IS_ERR(ptr)) {
- ida_simple_remove(&mem_id_pool, xdp_rxq->mem.id);
- xdp_rxq->mem.id = 0;
+ ida_simple_remove(&mem_id_pool, mem->id);
+ mem->id = 0;
errno = PTR_ERR(ptr);
goto err;
}
if (type == MEM_TYPE_PAGE_POOL)
- page_pool_use_xdp_mem(allocator, mem_allocator_disconnect);
+ page_pool_use_xdp_mem(allocator, mem_allocator_disconnect, mem);
mutex_unlock(&mem_id_lock);
- trace_mem_connect(xdp_alloc, xdp_rxq);
- return 0;
+ return xdp_alloc;
err:
mutex_unlock(&mem_id_lock);
kfree(xdp_alloc);
- return errno;
+ return ERR_PTR(errno);
}
+
+int xdp_reg_mem_model(struct xdp_mem_info *mem,
+ enum xdp_mem_type type, void *allocator)
+{
+ struct xdp_mem_allocator *xdp_alloc;
+
+ xdp_alloc = __xdp_reg_mem_model(mem, type, allocator);
+ if (IS_ERR(xdp_alloc))
+ return PTR_ERR(xdp_alloc);
+ return 0;
+}
+EXPORT_SYMBOL_GPL(xdp_reg_mem_model);
+
+int xdp_rxq_info_reg_mem_model(struct xdp_rxq_info *xdp_rxq,
+ enum xdp_mem_type type, void *allocator)
+{
+ struct xdp_mem_allocator *xdp_alloc;
+
+ if (xdp_rxq->reg_state != REG_STATE_REGISTERED) {
+ WARN(1, "Missing register, driver bug");
+ return -EFAULT;
+ }
+
+ xdp_alloc = __xdp_reg_mem_model(&xdp_rxq->mem, type, allocator);
+ if (IS_ERR(xdp_alloc))
+ return PTR_ERR(xdp_alloc);
+
+ trace_mem_connect(xdp_alloc, xdp_rxq);
+ return 0;
+}
+
EXPORT_SYMBOL_GPL(xdp_rxq_info_reg_mem_model);
/* XDP RX runs under NAPI protection, and in different delivery error
diff --git a/net/dccp/proto.c b/net/dccp/proto.c
index fc44dad..a976b4d 100644
--- a/net/dccp/proto.c
+++ b/net/dccp/proto.c
@@ -238,17 +238,6 @@ void dccp_destroy_sock(struct sock *sk)
EXPORT_SYMBOL_GPL(dccp_destroy_sock);
-static inline int dccp_listen_start(struct sock *sk, int backlog)
-{
- struct dccp_sock *dp = dccp_sk(sk);
-
- dp->dccps_role = DCCP_ROLE_LISTEN;
- /* do not start to listen if feature negotiation setup fails */
- if (dccp_feat_finalise_settings(dp))
- return -EPROTO;
- return inet_csk_listen_start(sk, backlog);
-}
-
static inline int dccp_need_reset(int state)
{
return state != DCCP_CLOSED && state != DCCP_LISTEN &&
@@ -931,11 +920,17 @@ int inet_dccp_listen(struct socket *sock, int backlog)
* we can only allow the backlog to be adjusted.
*/
if (old_state != DCCP_LISTEN) {
- /*
- * FIXME: here it probably should be sk->sk_prot->listen_start
- * see tcp_listen_start
- */
- err = dccp_listen_start(sk, backlog);
+ struct dccp_sock *dp = dccp_sk(sk);
+
+ dp->dccps_role = DCCP_ROLE_LISTEN;
+
+ /* do not start to listen if feature negotiation setup fails */
+ if (dccp_feat_finalise_settings(dp)) {
+ err = -EPROTO;
+ goto out;
+ }
+
+ err = inet_csk_listen_start(sk);
if (err)
goto out;
}
diff --git a/net/dccp/trace.h b/net/dccp/trace.h
index 5062421..5a43b35 100644
--- a/net/dccp/trace.h
+++ b/net/dccp/trace.h
@@ -60,9 +60,7 @@ TRACE_EVENT(dccp_probe,
__entry->tx_t_ipi = hc->tx_t_ipi;
} else {
__entry->tx_s = 0;
- memset(&__entry->tx_rtt, 0, (void *)&__entry->tx_t_ipi -
- (void *)&__entry->tx_rtt +
- sizeof(__entry->tx_t_ipi));
+ memset_startat(__entry, 0, tx_rtt);
}
),
diff --git a/net/decnet/dn_nsp_in.c b/net/decnet/dn_nsp_in.c
index 7ab788f..c59be5b 100644
--- a/net/decnet/dn_nsp_in.c
+++ b/net/decnet/dn_nsp_in.c
@@ -38,6 +38,7 @@
*******************************************************************************/
#include <linux/errno.h>
+#include <linux/filter.h>
#include <linux/types.h>
#include <linux/socket.h>
#include <linux/in.h>
diff --git a/net/decnet/dn_rules.c b/net/decnet/dn_rules.c
index 4a4e3c1..ee73057 100644
--- a/net/decnet/dn_rules.c
+++ b/net/decnet/dn_rules.c
@@ -101,10 +101,6 @@ static int dn_fib_rule_action(struct fib_rule *rule, struct flowi *flp,
return err;
}
-static const struct nla_policy dn_fib_rule_policy[FRA_MAX+1] = {
- FRA_GENERIC_POLICY,
-};
-
static int dn_fib_rule_match(struct fib_rule *rule, struct flowi *fl, int flags)
{
struct dn_fib_rule *r = (struct dn_fib_rule *)rule;
@@ -235,7 +231,6 @@ static const struct fib_rules_ops __net_initconst dn_fib_rules_ops_template = {
.fill = dn_fib_rule_fill,
.flush_cache = dn_fib_rule_flush_cache,
.nlgroup = RTNLGRP_DECnet_RULE,
- .policy = dn_fib_rule_policy,
.owner = THIS_MODULE,
.fro_net = &init_net,
};
diff --git a/net/dsa/dsa.c b/net/dsa/dsa.c
index ea5169e..d9d0d22 100644
--- a/net/dsa/dsa.c
+++ b/net/dsa/dsa.c
@@ -406,7 +406,7 @@ EXPORT_SYMBOL_GPL(dsa_devlink_resource_register);
void dsa_devlink_resources_unregister(struct dsa_switch *ds)
{
- devlink_resources_unregister(ds->devlink, NULL);
+ devlink_resources_unregister(ds->devlink);
}
EXPORT_SYMBOL_GPL(dsa_devlink_resources_unregister);
diff --git a/net/dsa/dsa2.c b/net/dsa/dsa2.c
index 826957b..3d21521 100644
--- a/net/dsa/dsa2.c
+++ b/net/dsa/dsa2.c
@@ -129,35 +129,52 @@ void dsa_lag_unmap(struct dsa_switch_tree *dst, struct net_device *lag)
}
}
+struct dsa_bridge *dsa_tree_bridge_find(struct dsa_switch_tree *dst,
+ const struct net_device *br)
+{
+ struct dsa_port *dp;
+
+ list_for_each_entry(dp, &dst->ports, list)
+ if (dsa_port_bridge_dev_get(dp) == br)
+ return dp->bridge;
+
+ return NULL;
+}
+
static int dsa_bridge_num_find(const struct net_device *bridge_dev)
{
struct dsa_switch_tree *dst;
- struct dsa_port *dp;
- /* When preparing the offload for a port, it will have a valid
- * dp->bridge_dev pointer but a not yet valid dp->bridge_num.
- * However there might be other ports having the same dp->bridge_dev
- * and a valid dp->bridge_num, so just ignore this port.
- */
- list_for_each_entry(dst, &dsa_tree_list, list)
- list_for_each_entry(dp, &dst->ports, list)
- if (dp->bridge_dev == bridge_dev &&
- dp->bridge_num != -1)
- return dp->bridge_num;
+ list_for_each_entry(dst, &dsa_tree_list, list) {
+ struct dsa_bridge *bridge;
- return -1;
+ bridge = dsa_tree_bridge_find(dst, bridge_dev);
+ if (bridge)
+ return bridge->num;
+ }
+
+ return 0;
}
-int dsa_bridge_num_get(const struct net_device *bridge_dev, int max)
+unsigned int dsa_bridge_num_get(const struct net_device *bridge_dev, int max)
{
- int bridge_num = dsa_bridge_num_find(bridge_dev);
+ unsigned int bridge_num = dsa_bridge_num_find(bridge_dev);
- if (bridge_num < 0) {
- /* First port that offloads TX forwarding for this bridge */
- bridge_num = find_first_zero_bit(&dsa_fwd_offloading_bridges,
- DSA_MAX_NUM_OFFLOADING_BRIDGES);
+ /* Switches without FDB isolation support don't get unique
+ * bridge numbering
+ */
+ if (!max)
+ return 0;
+
+ if (!bridge_num) {
+ /* First port that requests FDB isolation or TX forwarding
+ * offload for this bridge
+ */
+ bridge_num = find_next_zero_bit(&dsa_fwd_offloading_bridges,
+ DSA_MAX_NUM_OFFLOADING_BRIDGES,
+ 1);
if (bridge_num >= max)
- return -1;
+ return 0;
set_bit(bridge_num, &dsa_fwd_offloading_bridges);
}
@@ -165,13 +182,14 @@ int dsa_bridge_num_get(const struct net_device *bridge_dev, int max)
return bridge_num;
}
-void dsa_bridge_num_put(const struct net_device *bridge_dev, int bridge_num)
+void dsa_bridge_num_put(const struct net_device *bridge_dev,
+ unsigned int bridge_num)
{
- /* Check if the bridge is still in use, otherwise it is time
- * to clean it up so we can reuse this bridge_num later.
+ /* Since we refcount bridges, we know that when we call this function
+ * it is no longer in use, so we can just go ahead and remove it from
+ * the bit mask.
*/
- if (dsa_bridge_num_find(bridge_dev) < 0)
- clear_bit(bridge_num, &dsa_fwd_offloading_bridges);
+ clear_bit(bridge_num, &dsa_fwd_offloading_bridges);
}
struct dsa_switch *dsa_switch_find(int tree_index, int sw_index)
@@ -543,6 +561,7 @@ static void dsa_port_teardown(struct dsa_port *dp)
struct devlink_port *dlp = &dp->devlink_port;
struct dsa_switch *ds = dp->ds;
struct dsa_mac_addr *a, *tmp;
+ struct net_device *slave;
if (!dp->setup)
return;
@@ -564,9 +583,11 @@ static void dsa_port_teardown(struct dsa_port *dp)
dsa_port_link_unregister_of(dp);
break;
case DSA_PORT_TYPE_USER:
- if (dp->slave) {
- dsa_slave_destroy(dp->slave);
+ slave = dp->slave;
+
+ if (slave) {
dp->slave = NULL;
+ dsa_slave_destroy(slave);
}
break;
}
@@ -804,7 +825,7 @@ static int dsa_switch_setup_tag_protocol(struct dsa_switch *ds)
int err;
if (tag_ops->proto == dst->default_proto)
- return 0;
+ goto connect;
dsa_switch_for_each_cpu_port(cpu_dp, ds) {
rtnl_lock();
@@ -818,7 +839,30 @@ static int dsa_switch_setup_tag_protocol(struct dsa_switch *ds)
}
}
+connect:
+ if (tag_ops->connect) {
+ err = tag_ops->connect(ds);
+ if (err)
+ return err;
+ }
+
+ if (ds->ops->connect_tag_protocol) {
+ err = ds->ops->connect_tag_protocol(ds, tag_ops->proto);
+ if (err) {
+ dev_err(ds->dev,
+ "Unable to connect to tag protocol \"%s\": %pe\n",
+ tag_ops->name, ERR_PTR(err));
+ goto disconnect;
+ }
+ }
+
return 0;
+
+disconnect:
+ if (tag_ops->disconnect)
+ tag_ops->disconnect(ds);
+
+ return err;
}
static int dsa_switch_setup(struct dsa_switch *ds)
@@ -962,23 +1006,28 @@ static void dsa_tree_teardown_switches(struct dsa_switch_tree *dst)
dsa_switch_teardown(dp->ds);
}
-static int dsa_tree_setup_switches(struct dsa_switch_tree *dst)
+/* Bring shared ports up first, then non-shared ports */
+static int dsa_tree_setup_ports(struct dsa_switch_tree *dst)
{
struct dsa_port *dp;
- int err;
+ int err = 0;
list_for_each_entry(dp, &dst->ports, list) {
- err = dsa_switch_setup(dp->ds);
- if (err)
- goto teardown;
+ if (dsa_port_is_dsa(dp) || dsa_port_is_cpu(dp)) {
+ err = dsa_port_setup(dp);
+ if (err)
+ goto teardown;
+ }
}
list_for_each_entry(dp, &dst->ports, list) {
- err = dsa_port_setup(dp);
- if (err) {
- err = dsa_port_reinit_as_unused(dp);
- if (err)
- goto teardown;
+ if (dsa_port_is_user(dp) || dsa_port_is_unused(dp)) {
+ err = dsa_port_setup(dp);
+ if (err) {
+ err = dsa_port_reinit_as_unused(dp);
+ if (err)
+ goto teardown;
+ }
}
}
@@ -987,7 +1036,21 @@ static int dsa_tree_setup_switches(struct dsa_switch_tree *dst)
teardown:
dsa_tree_teardown_ports(dst);
- dsa_tree_teardown_switches(dst);
+ return err;
+}
+
+static int dsa_tree_setup_switches(struct dsa_switch_tree *dst)
+{
+ struct dsa_port *dp;
+ int err = 0;
+
+ list_for_each_entry(dp, &dst->ports, list) {
+ err = dsa_switch_setup(dp->ds);
+ if (err) {
+ dsa_tree_teardown_switches(dst);
+ break;
+ }
+ }
return err;
}
@@ -997,6 +1060,8 @@ static int dsa_tree_setup_master(struct dsa_switch_tree *dst)
struct dsa_port *dp;
int err;
+ rtnl_lock();
+
list_for_each_entry(dp, &dst->ports, list) {
if (dsa_port_is_cpu(dp)) {
err = dsa_master_setup(dp->master, dp);
@@ -1005,6 +1070,8 @@ static int dsa_tree_setup_master(struct dsa_switch_tree *dst)
}
}
+ rtnl_unlock();
+
return 0;
}
@@ -1012,9 +1079,13 @@ static void dsa_tree_teardown_master(struct dsa_switch_tree *dst)
{
struct dsa_port *dp;
+ rtnl_lock();
+
list_for_each_entry(dp, &dst->ports, list)
if (dsa_port_is_cpu(dp))
dsa_master_teardown(dp->master);
+
+ rtnl_unlock();
}
static int dsa_tree_setup_lags(struct dsa_switch_tree *dst)
@@ -1070,20 +1141,25 @@ static int dsa_tree_setup(struct dsa_switch_tree *dst)
if (err)
goto teardown_switches;
- err = dsa_tree_setup_lags(dst);
+ err = dsa_tree_setup_ports(dst);
if (err)
goto teardown_master;
+ err = dsa_tree_setup_lags(dst);
+ if (err)
+ goto teardown_ports;
+
dst->setup = true;
pr_info("DSA: tree %d setup\n", dst->index);
return 0;
+teardown_ports:
+ dsa_tree_teardown_ports(dst);
teardown_master:
dsa_tree_teardown_master(dst);
teardown_switches:
- dsa_tree_teardown_ports(dst);
dsa_tree_teardown_switches(dst);
teardown_cpu_ports:
dsa_tree_teardown_cpu_ports(dst);
@@ -1100,10 +1176,10 @@ static void dsa_tree_teardown(struct dsa_switch_tree *dst)
dsa_tree_teardown_lags(dst);
- dsa_tree_teardown_master(dst);
-
dsa_tree_teardown_ports(dst);
+ dsa_tree_teardown_master(dst);
+
dsa_tree_teardown_switches(dst);
dsa_tree_teardown_cpu_ports(dst);
@@ -1118,6 +1194,37 @@ static void dsa_tree_teardown(struct dsa_switch_tree *dst)
dst->setup = false;
}
+static int dsa_tree_bind_tag_proto(struct dsa_switch_tree *dst,
+ const struct dsa_device_ops *tag_ops)
+{
+ const struct dsa_device_ops *old_tag_ops = dst->tag_ops;
+ struct dsa_notifier_tag_proto_info info;
+ int err;
+
+ dst->tag_ops = tag_ops;
+
+ /* Notify the switches from this tree about the connection
+ * to the new tagger
+ */
+ info.tag_ops = tag_ops;
+ err = dsa_tree_notify(dst, DSA_NOTIFIER_TAG_PROTO_CONNECT, &info);
+ if (err && err != -EOPNOTSUPP)
+ goto out_disconnect;
+
+ /* Notify the old tagger about the disconnection from this tree */
+ info.tag_ops = old_tag_ops;
+ dsa_tree_notify(dst, DSA_NOTIFIER_TAG_PROTO_DISCONNECT, &info);
+
+ return 0;
+
+out_disconnect:
+ info.tag_ops = tag_ops;
+ dsa_tree_notify(dst, DSA_NOTIFIER_TAG_PROTO_DISCONNECT, &info);
+ dst->tag_ops = old_tag_ops;
+
+ return err;
+}
+
/* Since the dsa/tagging sysfs device attribute is per master, the assumption
* is that all DSA switches within a tree share the same tagger, otherwise
* they would have formed disjoint trees (different "dsa,member" values).
@@ -1150,12 +1257,15 @@ int dsa_tree_change_tag_proto(struct dsa_switch_tree *dst,
goto out_unlock;
}
+ /* Notify the tag protocol change */
info.tag_ops = tag_ops;
err = dsa_tree_notify(dst, DSA_NOTIFIER_TAG_PROTO, &info);
if (err)
- goto out_unwind_tagger;
+ return err;
- dst->tag_ops = tag_ops;
+ err = dsa_tree_bind_tag_proto(dst, tag_ops);
+ if (err)
+ goto out_unwind_tagger;
rtnl_unlock();
@@ -1184,7 +1294,6 @@ static struct dsa_port *dsa_port_touch(struct dsa_switch *ds, int index)
dp->ds = ds;
dp->index = index;
- dp->bridge_num = -1;
INIT_LIST_HEAD(&dp->list);
list_add_tail(&dp->list, &dst->ports);
@@ -1366,7 +1475,7 @@ static int dsa_switch_parse_ports_of(struct dsa_switch *ds,
}
if (reg >= ds->num_ports) {
- dev_err(ds->dev, "port %pOF index %u exceeds num_ports (%zu)\n",
+ dev_err(ds->dev, "port %pOF index %u exceeds num_ports (%u)\n",
port, reg, ds->num_ports);
of_node_put(port);
err = -EINVAL;
diff --git a/net/dsa/dsa_priv.h b/net/dsa/dsa_priv.h
index a5c9bc7..760306f 100644
--- a/net/dsa/dsa_priv.h
+++ b/net/dsa/dsa_priv.h
@@ -8,6 +8,7 @@
#define __DSA_PRIV_H
#include <linux/if_bridge.h>
+#include <linux/if_vlan.h>
#include <linux/phy.h>
#include <linux/netdevice.h>
#include <linux/netpoll.h>
@@ -24,8 +25,6 @@ enum {
DSA_NOTIFIER_FDB_DEL,
DSA_NOTIFIER_HOST_FDB_ADD,
DSA_NOTIFIER_HOST_FDB_DEL,
- DSA_NOTIFIER_HSR_JOIN,
- DSA_NOTIFIER_HSR_LEAVE,
DSA_NOTIFIER_LAG_CHANGE,
DSA_NOTIFIER_LAG_JOIN,
DSA_NOTIFIER_LAG_LEAVE,
@@ -37,10 +36,8 @@ enum {
DSA_NOTIFIER_VLAN_DEL,
DSA_NOTIFIER_MTU,
DSA_NOTIFIER_TAG_PROTO,
- DSA_NOTIFIER_MRP_ADD,
- DSA_NOTIFIER_MRP_DEL,
- DSA_NOTIFIER_MRP_ADD_RING_ROLE,
- DSA_NOTIFIER_MRP_DEL_RING_ROLE,
+ DSA_NOTIFIER_TAG_PROTO_CONNECT,
+ DSA_NOTIFIER_TAG_PROTO_DISCONNECT,
DSA_NOTIFIER_TAG_8021Q_VLAN_ADD,
DSA_NOTIFIER_TAG_8021Q_VLAN_DEL,
};
@@ -52,10 +49,11 @@ struct dsa_notifier_ageing_time_info {
/* DSA_NOTIFIER_BRIDGE_* */
struct dsa_notifier_bridge_info {
- struct net_device *br;
+ struct dsa_bridge bridge;
int tree_index;
int sw_index;
int port;
+ bool tx_fwd_offload;
};
/* DSA_NOTIFIER_FDB_* */
@@ -103,20 +101,6 @@ struct dsa_notifier_tag_proto_info {
const struct dsa_device_ops *tag_ops;
};
-/* DSA_NOTIFIER_MRP_* */
-struct dsa_notifier_mrp_info {
- const struct switchdev_obj_mrp *mrp;
- int sw_index;
- int port;
-};
-
-/* DSA_NOTIFIER_MRP_* */
-struct dsa_notifier_mrp_ring_role_info {
- const struct switchdev_obj_ring_role_mrp *mrp;
- int sw_index;
- int port;
-};
-
/* DSA_NOTIFIER_TAG_8021Q_VLAN_* */
struct dsa_notifier_tag_8021q_vlan_info {
int tree_index;
@@ -139,13 +123,6 @@ struct dsa_switchdev_event_work {
bool host_addr;
};
-/* DSA_NOTIFIER_HSR_* */
-struct dsa_notifier_hsr_info {
- struct net_device *hsr;
- int sw_index;
- int port;
-};
-
struct dsa_slave_priv {
/* Copy of CPU port xmit for faster access in slave transmit hot path */
struct sk_buff * (*xmit)(struct sk_buff *skb,
@@ -258,54 +235,13 @@ int dsa_port_mrp_add_ring_role(const struct dsa_port *dp,
const struct switchdev_obj_ring_role_mrp *mrp);
int dsa_port_mrp_del_ring_role(const struct dsa_port *dp,
const struct switchdev_obj_ring_role_mrp *mrp);
+int dsa_port_phylink_create(struct dsa_port *dp);
int dsa_port_link_register_of(struct dsa_port *dp);
void dsa_port_link_unregister_of(struct dsa_port *dp);
int dsa_port_hsr_join(struct dsa_port *dp, struct net_device *hsr);
void dsa_port_hsr_leave(struct dsa_port *dp, struct net_device *hsr);
int dsa_port_tag_8021q_vlan_add(struct dsa_port *dp, u16 vid, bool broadcast);
void dsa_port_tag_8021q_vlan_del(struct dsa_port *dp, u16 vid, bool broadcast);
-extern const struct phylink_mac_ops dsa_port_phylink_mac_ops;
-
-static inline bool dsa_port_offloads_bridge_port(struct dsa_port *dp,
- const struct net_device *dev)
-{
- return dsa_port_to_bridge_port(dp) == dev;
-}
-
-static inline bool dsa_port_offloads_bridge(struct dsa_port *dp,
- const struct net_device *bridge_dev)
-{
- /* DSA ports connected to a bridge, and event was emitted
- * for the bridge.
- */
- return dp->bridge_dev == bridge_dev;
-}
-
-/* Returns true if any port of this tree offloads the given net_device */
-static inline bool dsa_tree_offloads_bridge_port(struct dsa_switch_tree *dst,
- const struct net_device *dev)
-{
- struct dsa_port *dp;
-
- list_for_each_entry(dp, &dst->ports, list)
- if (dsa_port_offloads_bridge_port(dp, dev))
- return true;
-
- return false;
-}
-
-/* Returns true if any port of this tree offloads the given bridge */
-static inline bool dsa_tree_offloads_bridge(struct dsa_switch_tree *dst,
- const struct net_device *bridge_dev)
-{
- struct dsa_port *dp;
-
- list_for_each_entry(dp, &dst->ports, list)
- if (dsa_port_offloads_bridge(dp, bridge_dev))
- return true;
-
- return false;
-}
/* slave.c */
extern const struct dsa_device_ops notag_netdev_ops;
@@ -345,7 +281,7 @@ dsa_slave_to_master(const struct net_device *dev)
static inline struct sk_buff *dsa_untag_bridge_pvid(struct sk_buff *skb)
{
struct dsa_port *dp = dsa_slave_to_port(skb->dev);
- struct net_device *br = dp->bridge_dev;
+ struct net_device *br = dsa_port_bridge_dev_get(dp);
struct net_device *dev = skb->dev;
struct net_device *upper_dev;
u16 vid, pvid, proto;
@@ -415,7 +351,7 @@ dsa_find_designated_bridge_port_by_vid(struct net_device *master, u16 vid)
if (dp->type != DSA_PORT_TYPE_USER)
continue;
- if (!dp->bridge_dev)
+ if (!dp->bridge)
continue;
if (dp->stp_state != BR_STATE_LEARNING &&
@@ -444,7 +380,7 @@ dsa_find_designated_bridge_port_by_vid(struct net_device *master, u16 vid)
/* If the ingress port offloads the bridge, we mark the frame as autonomously
* forwarded by hardware, so the software bridge doesn't forward in twice, back
* to us, because we already did. However, if we're in fallback mode and we do
- * software bridging, we are not offloading it, therefore the dp->bridge_dev
+ * software bridging, we are not offloading it, therefore the dp->bridge
* pointer is not populated, and flooding needs to be done by software (we are
* effectively operating in standalone ports mode).
*/
@@ -452,7 +388,7 @@ static inline void dsa_default_offload_fwd_mark(struct sk_buff *skb)
{
struct dsa_port *dp = dsa_slave_to_port(skb->dev);
- skb->offload_fwd_mark = !!(dp->bridge_dev);
+ skb->offload_fwd_mark = !!(dp->bridge);
}
/* Helper for removing DSA header tags from packets in the RX path.
@@ -546,8 +482,11 @@ int dsa_tree_change_tag_proto(struct dsa_switch_tree *dst,
struct net_device *master,
const struct dsa_device_ops *tag_ops,
const struct dsa_device_ops *old_tag_ops);
-int dsa_bridge_num_get(const struct net_device *bridge_dev, int max);
-void dsa_bridge_num_put(const struct net_device *bridge_dev, int bridge_num);
+unsigned int dsa_bridge_num_get(const struct net_device *bridge_dev, int max);
+void dsa_bridge_num_put(const struct net_device *bridge_dev,
+ unsigned int bridge_num);
+struct dsa_bridge *dsa_tree_bridge_find(struct dsa_switch_tree *dst,
+ const struct net_device *br);
/* tag_8021q.c */
int dsa_tag_8021q_bridge_join(struct dsa_switch *ds,
diff --git a/net/dsa/master.c b/net/dsa/master.c
index e8e1985..2199104 100644
--- a/net/dsa/master.c
+++ b/net/dsa/master.c
@@ -267,9 +267,9 @@ static void dsa_master_set_promiscuity(struct net_device *dev, int inc)
if (!ops->promisc_on_master)
return;
- rtnl_lock();
+ ASSERT_RTNL();
+
dev_set_promiscuity(dev, inc);
- rtnl_unlock();
}
static ssize_t tagging_show(struct device *d, struct device_attribute *attr,
@@ -330,28 +330,13 @@ static const struct attribute_group dsa_group = {
.attrs = dsa_slave_attrs,
};
-static void dsa_master_reset_mtu(struct net_device *dev)
-{
- int err;
-
- rtnl_lock();
- err = dev_set_mtu(dev, ETH_DATA_LEN);
- if (err)
- netdev_dbg(dev,
- "Unable to reset MTU to exclude DSA overheads\n");
- rtnl_unlock();
-}
-
static struct lock_class_key dsa_master_addr_list_lock_key;
int dsa_master_setup(struct net_device *dev, struct dsa_port *cpu_dp)
{
- const struct dsa_device_ops *tag_ops = cpu_dp->tag_ops;
struct dsa_switch *ds = cpu_dp->ds;
struct device_link *consumer_link;
- int mtu, ret;
-
- mtu = ETH_DATA_LEN + dsa_tag_protocol_overhead(tag_ops);
+ int ret;
/* The DSA master must use SET_NETDEV_DEV for this to work. */
consumer_link = device_link_add(ds->dev, dev->dev.parent,
@@ -361,13 +346,6 @@ int dsa_master_setup(struct net_device *dev, struct dsa_port *cpu_dp)
"Failed to create a device link to DSA switch %s\n",
dev_name(ds->dev));
- rtnl_lock();
- ret = dev_set_mtu(dev, mtu);
- rtnl_unlock();
- if (ret)
- netdev_warn(dev, "error %d setting MTU to %d to include DSA overhead\n",
- ret, mtu);
-
/* If we use a tagging format that doesn't have an ethertype
* field, make sure that all packets from this point on get
* sent to the tag format's receive function.
@@ -405,7 +383,6 @@ void dsa_master_teardown(struct net_device *dev)
sysfs_remove_group(&dev->dev.kobj, &dsa_group);
dsa_netdev_ops_set(dev, NULL);
dsa_master_ethtool_teardown(dev);
- dsa_master_reset_mtu(dev);
dsa_master_set_promiscuity(dev, -1);
dev->dsa_ptr = NULL;
diff --git a/net/dsa/port.c b/net/dsa/port.c
index f6f12ad..bd78192e0 100644
--- a/net/dsa/port.c
+++ b/net/dsa/port.c
@@ -130,7 +130,7 @@ int dsa_port_enable_rt(struct dsa_port *dp, struct phy_device *phy)
return err;
}
- if (!dp->bridge_dev)
+ if (!dp->bridge)
dsa_port_set_state_now(dp, BR_STATE_FORWARDING, false);
if (dp->pl)
@@ -158,7 +158,7 @@ void dsa_port_disable_rt(struct dsa_port *dp)
if (dp->pl)
phylink_stop(dp->pl);
- if (!dp->bridge_dev)
+ if (!dp->bridge)
dsa_port_set_state_now(dp, BR_STATE_DISABLED, false);
if (ds->ops->port_disable)
@@ -221,7 +221,7 @@ static int dsa_port_switchdev_sync_attrs(struct dsa_port *dp,
struct netlink_ext_ack *extack)
{
struct net_device *brport_dev = dsa_port_to_bridge_port(dp);
- struct net_device *br = dp->bridge_dev;
+ struct net_device *br = dsa_port_bridge_dev_get(dp);
int err;
err = dsa_port_inherit_brport_flags(dp, extack);
@@ -270,52 +270,55 @@ static void dsa_port_switchdev_unsync_attrs(struct dsa_port *dp)
*/
}
-static void dsa_port_bridge_tx_fwd_unoffload(struct dsa_port *dp,
- struct net_device *bridge_dev)
-{
- int bridge_num = dp->bridge_num;
- struct dsa_switch *ds = dp->ds;
-
- /* No bridge TX forwarding offload => do nothing */
- if (!ds->ops->port_bridge_tx_fwd_unoffload || dp->bridge_num == -1)
- return;
-
- dp->bridge_num = -1;
-
- dsa_bridge_num_put(bridge_dev, bridge_num);
-
- /* Notify the chips only once the offload has been deactivated, so
- * that they can update their configuration accordingly.
- */
- ds->ops->port_bridge_tx_fwd_unoffload(ds, dp->index, bridge_dev,
- bridge_num);
-}
-
-static bool dsa_port_bridge_tx_fwd_offload(struct dsa_port *dp,
- struct net_device *bridge_dev)
+static int dsa_port_bridge_create(struct dsa_port *dp,
+ struct net_device *br,
+ struct netlink_ext_ack *extack)
{
struct dsa_switch *ds = dp->ds;
- int bridge_num, err;
+ struct dsa_bridge *bridge;
- if (!ds->ops->port_bridge_tx_fwd_offload)
- return false;
-
- bridge_num = dsa_bridge_num_get(bridge_dev,
- ds->num_fwd_offloading_bridges);
- if (bridge_num < 0)
- return false;
-
- dp->bridge_num = bridge_num;
-
- /* Notify the driver */
- err = ds->ops->port_bridge_tx_fwd_offload(ds, dp->index, bridge_dev,
- bridge_num);
- if (err) {
- dsa_port_bridge_tx_fwd_unoffload(dp, bridge_dev);
- return false;
+ bridge = dsa_tree_bridge_find(ds->dst, br);
+ if (bridge) {
+ refcount_inc(&bridge->refcount);
+ dp->bridge = bridge;
+ return 0;
}
- return true;
+ bridge = kzalloc(sizeof(*bridge), GFP_KERNEL);
+ if (!bridge)
+ return -ENOMEM;
+
+ refcount_set(&bridge->refcount, 1);
+
+ bridge->dev = br;
+
+ bridge->num = dsa_bridge_num_get(br, ds->max_num_bridges);
+ if (ds->max_num_bridges && !bridge->num) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "Range of offloadable bridges exceeded");
+ kfree(bridge);
+ return -EOPNOTSUPP;
+ }
+
+ dp->bridge = bridge;
+
+ return 0;
+}
+
+static void dsa_port_bridge_destroy(struct dsa_port *dp,
+ const struct net_device *br)
+{
+ struct dsa_bridge *bridge = dp->bridge;
+
+ dp->bridge = NULL;
+
+ if (!refcount_dec_and_test(&bridge->refcount))
+ return;
+
+ if (bridge->num)
+ dsa_bridge_num_put(br, bridge->num);
+
+ kfree(bridge);
}
int dsa_port_bridge_join(struct dsa_port *dp, struct net_device *br,
@@ -325,30 +328,32 @@ int dsa_port_bridge_join(struct dsa_port *dp, struct net_device *br,
.tree_index = dp->ds->dst->index,
.sw_index = dp->ds->index,
.port = dp->index,
- .br = br,
};
struct net_device *dev = dp->slave;
struct net_device *brport_dev;
- bool tx_fwd_offload;
int err;
/* Here the interface is already bridged. Reflect the current
* configuration so that drivers can program their chips accordingly.
*/
- dp->bridge_dev = br;
+ err = dsa_port_bridge_create(dp, br, extack);
+ if (err)
+ return err;
brport_dev = dsa_port_to_bridge_port(dp);
+ info.bridge = *dp->bridge;
err = dsa_broadcast(DSA_NOTIFIER_BRIDGE_JOIN, &info);
if (err)
goto out_rollback;
- tx_fwd_offload = dsa_port_bridge_tx_fwd_offload(dp, br);
+ /* Drivers which support bridge TX forwarding should set this */
+ dp->bridge->tx_fwd_offload = info.tx_fwd_offload;
err = switchdev_bridge_port_offload(brport_dev, dev, dp,
&dsa_slave_switchdev_notifier,
&dsa_slave_switchdev_blocking_notifier,
- tx_fwd_offload, extack);
+ dp->bridge->tx_fwd_offload, extack);
if (err)
goto out_rollback_unbridge;
@@ -365,7 +370,7 @@ int dsa_port_bridge_join(struct dsa_port *dp, struct net_device *br,
out_rollback_unbridge:
dsa_broadcast(DSA_NOTIFIER_BRIDGE_LEAVE, &info);
out_rollback:
- dp->bridge_dev = NULL;
+ dsa_port_bridge_destroy(dp, br);
return err;
}
@@ -390,16 +395,14 @@ void dsa_port_bridge_leave(struct dsa_port *dp, struct net_device *br)
.tree_index = dp->ds->dst->index,
.sw_index = dp->ds->index,
.port = dp->index,
- .br = br,
+ .bridge = *dp->bridge,
};
int err;
/* Here the port is already unbridged. Reflect the current configuration
* so that drivers can program their chips accordingly.
*/
- dp->bridge_dev = NULL;
-
- dsa_port_bridge_tx_fwd_unoffload(dp, br);
+ dsa_port_bridge_destroy(dp, br);
err = dsa_broadcast(DSA_NOTIFIER_BRIDGE_LEAVE, &info);
if (err)
@@ -477,12 +480,15 @@ int dsa_port_lag_join(struct dsa_port *dp, struct net_device *lag,
void dsa_port_pre_lag_leave(struct dsa_port *dp, struct net_device *lag)
{
- if (dp->bridge_dev)
- dsa_port_pre_bridge_leave(dp, dp->bridge_dev);
+ struct net_device *br = dsa_port_bridge_dev_get(dp);
+
+ if (br)
+ dsa_port_pre_bridge_leave(dp, br);
}
void dsa_port_lag_leave(struct dsa_port *dp, struct net_device *lag)
{
+ struct net_device *br = dsa_port_bridge_dev_get(dp);
struct dsa_notifier_lag_info info = {
.sw_index = dp->ds->index,
.port = dp->index,
@@ -496,8 +502,8 @@ void dsa_port_lag_leave(struct dsa_port *dp, struct net_device *lag)
/* Port might have been part of a LAG that in turn was
* attached to a bridge.
*/
- if (dp->bridge_dev)
- dsa_port_bridge_leave(dp, dp->bridge_dev);
+ if (br)
+ dsa_port_bridge_leave(dp, br);
dp->lag_tx_enabled = false;
dp->lag_dev = NULL;
@@ -526,8 +532,8 @@ static bool dsa_port_can_apply_vlan_filtering(struct dsa_port *dp,
* as long as we have 8021q uppers.
*/
if (vlan_filtering && dsa_port_is_user(dp)) {
+ struct net_device *br = dsa_port_bridge_dev_get(dp);
struct net_device *upper_dev, *slave = dp->slave;
- struct net_device *br = dp->bridge_dev;
struct list_head *iter;
netdev_for_each_upper_dev_rcu(slave, upper_dev, iter) {
@@ -561,17 +567,15 @@ static bool dsa_port_can_apply_vlan_filtering(struct dsa_port *dp,
* different setting than what is being requested.
*/
dsa_switch_for_each_port(other_dp, ds) {
- struct net_device *other_bridge;
+ struct net_device *other_br = dsa_port_bridge_dev_get(other_dp);
- other_bridge = other_dp->bridge_dev;
- if (!other_bridge)
- continue;
/* If it's the same bridge, it also has same
* vlan_filtering setting => no need to check
*/
- if (other_bridge == dp->bridge_dev)
+ if (!other_br || other_br == dsa_port_bridge_dev_get(dp))
continue;
- if (br_vlan_enabled(other_bridge) != vlan_filtering) {
+
+ if (br_vlan_enabled(other_br) != vlan_filtering) {
NL_SET_ERR_MSG_MOD(extack,
"VLAN filtering is a global setting");
return false;
@@ -655,13 +659,13 @@ int dsa_port_vlan_filtering(struct dsa_port *dp, bool vlan_filtering,
*/
bool dsa_port_skip_vlan_configuration(struct dsa_port *dp)
{
+ struct net_device *br = dsa_port_bridge_dev_get(dp);
struct dsa_switch *ds = dp->ds;
- if (!dp->bridge_dev)
+ if (!br)
return false;
- return (!ds->configure_vlan_while_not_filtering &&
- !br_vlan_enabled(dp->bridge_dev));
+ return !ds->configure_vlan_while_not_filtering && !br_vlan_enabled(br);
}
int dsa_port_ageing_time(struct dsa_port *dp, clock_t ageing_clock)
@@ -903,49 +907,45 @@ int dsa_port_vlan_del(struct dsa_port *dp,
int dsa_port_mrp_add(const struct dsa_port *dp,
const struct switchdev_obj_mrp *mrp)
{
- struct dsa_notifier_mrp_info info = {
- .sw_index = dp->ds->index,
- .port = dp->index,
- .mrp = mrp,
- };
+ struct dsa_switch *ds = dp->ds;
- return dsa_port_notify(dp, DSA_NOTIFIER_MRP_ADD, &info);
+ if (!ds->ops->port_mrp_add)
+ return -EOPNOTSUPP;
+
+ return ds->ops->port_mrp_add(ds, dp->index, mrp);
}
int dsa_port_mrp_del(const struct dsa_port *dp,
const struct switchdev_obj_mrp *mrp)
{
- struct dsa_notifier_mrp_info info = {
- .sw_index = dp->ds->index,
- .port = dp->index,
- .mrp = mrp,
- };
+ struct dsa_switch *ds = dp->ds;
- return dsa_port_notify(dp, DSA_NOTIFIER_MRP_DEL, &info);
+ if (!ds->ops->port_mrp_del)
+ return -EOPNOTSUPP;
+
+ return ds->ops->port_mrp_del(ds, dp->index, mrp);
}
int dsa_port_mrp_add_ring_role(const struct dsa_port *dp,
const struct switchdev_obj_ring_role_mrp *mrp)
{
- struct dsa_notifier_mrp_ring_role_info info = {
- .sw_index = dp->ds->index,
- .port = dp->index,
- .mrp = mrp,
- };
+ struct dsa_switch *ds = dp->ds;
- return dsa_port_notify(dp, DSA_NOTIFIER_MRP_ADD_RING_ROLE, &info);
+ if (!ds->ops->port_mrp_add_ring_role)
+ return -EOPNOTSUPP;
+
+ return ds->ops->port_mrp_add_ring_role(ds, dp->index, mrp);
}
int dsa_port_mrp_del_ring_role(const struct dsa_port *dp,
const struct switchdev_obj_ring_role_mrp *mrp)
{
- struct dsa_notifier_mrp_ring_role_info info = {
- .sw_index = dp->ds->index,
- .port = dp->index,
- .mrp = mrp,
- };
+ struct dsa_switch *ds = dp->ds;
- return dsa_port_notify(dp, DSA_NOTIFIER_MRP_DEL_RING_ROLE, &info);
+ if (!ds->ops->port_mrp_del_ring_role)
+ return -EOPNOTSUPP;
+
+ return ds->ops->port_mrp_del_ring_role(ds, dp->index, mrp);
}
void dsa_port_set_tag_protocol(struct dsa_port *cpu_dp,
@@ -981,8 +981,11 @@ static void dsa_port_phylink_validate(struct phylink_config *config,
struct dsa_port *dp = container_of(config, struct dsa_port, pl_config);
struct dsa_switch *ds = dp->ds;
- if (!ds->ops->phylink_validate)
+ if (!ds->ops->phylink_validate) {
+ if (config->mac_capabilities)
+ phylink_generic_validate(config, supported, state);
return;
+ }
ds->ops->phylink_validate(ds, dp->index, supported, state);
}
@@ -1072,7 +1075,7 @@ static void dsa_port_phylink_mac_link_up(struct phylink_config *config,
speed, duplex, tx_pause, rx_pause);
}
-const struct phylink_mac_ops dsa_port_phylink_mac_ops = {
+static const struct phylink_mac_ops dsa_port_phylink_mac_ops = {
.validate = dsa_port_phylink_validate,
.mac_pcs_get_state = dsa_port_phylink_mac_pcs_get_state,
.mac_config = dsa_port_phylink_mac_config,
@@ -1081,6 +1084,36 @@ const struct phylink_mac_ops dsa_port_phylink_mac_ops = {
.mac_link_up = dsa_port_phylink_mac_link_up,
};
+int dsa_port_phylink_create(struct dsa_port *dp)
+{
+ struct dsa_switch *ds = dp->ds;
+ phy_interface_t mode;
+ int err;
+
+ err = of_get_phy_mode(dp->dn, &mode);
+ if (err)
+ mode = PHY_INTERFACE_MODE_NA;
+
+ /* Presence of phylink_mac_link_state or phylink_mac_an_restart is
+ * an indicator of a legacy phylink driver.
+ */
+ if (ds->ops->phylink_mac_link_state ||
+ ds->ops->phylink_mac_an_restart)
+ dp->pl_config.legacy_pre_march2020 = true;
+
+ if (ds->ops->phylink_get_caps)
+ ds->ops->phylink_get_caps(ds, dp->index, &dp->pl_config);
+
+ dp->pl = phylink_create(&dp->pl_config, of_fwnode_handle(dp->dn),
+ mode, &dsa_port_phylink_mac_ops);
+ if (IS_ERR(dp->pl)) {
+ pr_err("error creating PHYLINK: %ld\n", PTR_ERR(dp->pl));
+ return PTR_ERR(dp->pl);
+ }
+
+ return 0;
+}
+
static int dsa_port_setup_phy_of(struct dsa_port *dp, bool enable)
{
struct dsa_switch *ds = dp->ds;
@@ -1157,27 +1190,15 @@ static int dsa_port_phylink_register(struct dsa_port *dp)
{
struct dsa_switch *ds = dp->ds;
struct device_node *port_dn = dp->dn;
- phy_interface_t mode;
int err;
- err = of_get_phy_mode(port_dn, &mode);
- if (err)
- mode = PHY_INTERFACE_MODE_NA;
-
dp->pl_config.dev = ds->dev;
dp->pl_config.type = PHYLINK_DEV;
dp->pl_config.pcs_poll = ds->pcs_poll;
- if (ds->ops->phylink_get_interfaces)
- ds->ops->phylink_get_interfaces(ds, dp->index,
- dp->pl_config.supported_interfaces);
-
- dp->pl = phylink_create(&dp->pl_config, of_fwnode_handle(port_dn),
- mode, &dsa_port_phylink_mac_ops);
- if (IS_ERR(dp->pl)) {
- pr_err("error creating PHYLINK: %ld\n", PTR_ERR(dp->pl));
- return PTR_ERR(dp->pl);
- }
+ err = dsa_port_phylink_create(dp);
+ if (err)
+ return err;
err = phylink_of_phy_connect(dp->pl, port_dn, 0);
if (err && err != -ENODEV) {
@@ -1296,16 +1317,15 @@ EXPORT_SYMBOL_GPL(dsa_port_get_phy_sset_count);
int dsa_port_hsr_join(struct dsa_port *dp, struct net_device *hsr)
{
- struct dsa_notifier_hsr_info info = {
- .sw_index = dp->ds->index,
- .port = dp->index,
- .hsr = hsr,
- };
+ struct dsa_switch *ds = dp->ds;
int err;
+ if (!ds->ops->port_hsr_join)
+ return -EOPNOTSUPP;
+
dp->hsr_dev = hsr;
- err = dsa_port_notify(dp, DSA_NOTIFIER_HSR_JOIN, &info);
+ err = ds->ops->port_hsr_join(ds, dp->index, hsr);
if (err)
dp->hsr_dev = NULL;
@@ -1314,20 +1334,18 @@ int dsa_port_hsr_join(struct dsa_port *dp, struct net_device *hsr)
void dsa_port_hsr_leave(struct dsa_port *dp, struct net_device *hsr)
{
- struct dsa_notifier_hsr_info info = {
- .sw_index = dp->ds->index,
- .port = dp->index,
- .hsr = hsr,
- };
+ struct dsa_switch *ds = dp->ds;
int err;
dp->hsr_dev = NULL;
- err = dsa_port_notify(dp, DSA_NOTIFIER_HSR_LEAVE, &info);
- if (err)
- dev_err(dp->ds->dev,
- "port %d failed to notify DSA_NOTIFIER_HSR_LEAVE: %pe\n",
- dp->index, ERR_PTR(err));
+ if (ds->ops->port_hsr_leave) {
+ err = ds->ops->port_hsr_leave(ds, dp->index, hsr);
+ if (err)
+ dev_err(dp->ds->dev,
+ "port %d failed to leave HSR %s: %pe\n",
+ dp->index, hsr->name, ERR_PTR(err));
+ }
}
int dsa_port_tag_8021q_vlan_add(struct dsa_port *dp, u16 vid, bool broadcast)
diff --git a/net/dsa/slave.c b/net/dsa/slave.c
index ad61f6b..22241af 100644
--- a/net/dsa/slave.c
+++ b/net/dsa/slave.c
@@ -289,14 +289,14 @@ static int dsa_slave_port_attr_set(struct net_device *dev, const void *ctx,
ret = dsa_port_set_state(dp, attr->u.stp_state, true);
break;
case SWITCHDEV_ATTR_ID_BRIDGE_VLAN_FILTERING:
- if (!dsa_port_offloads_bridge(dp, attr->orig_dev))
+ if (!dsa_port_offloads_bridge_dev(dp, attr->orig_dev))
return -EOPNOTSUPP;
ret = dsa_port_vlan_filtering(dp, attr->u.vlan_filtering,
extack);
break;
case SWITCHDEV_ATTR_ID_BRIDGE_AGEING_TIME:
- if (!dsa_port_offloads_bridge(dp, attr->orig_dev))
+ if (!dsa_port_offloads_bridge_dev(dp, attr->orig_dev))
return -EOPNOTSUPP;
ret = dsa_port_ageing_time(dp, attr->u.ageing_time);
@@ -363,7 +363,7 @@ static int dsa_slave_vlan_add(struct net_device *dev,
/* Deny adding a bridge VLAN when there is already an 802.1Q upper with
* the same VID.
*/
- if (br_vlan_enabled(dp->bridge_dev)) {
+ if (br_vlan_enabled(dsa_port_bridge_dev_get(dp))) {
rcu_read_lock();
err = dsa_slave_vlan_check_for_8021q_uppers(dev, &vlan);
rcu_read_unlock();
@@ -409,7 +409,7 @@ static int dsa_slave_port_obj_add(struct net_device *dev, const void *ctx,
err = dsa_port_mdb_add(dp, SWITCHDEV_OBJ_PORT_MDB(obj));
break;
case SWITCHDEV_OBJ_ID_HOST_MDB:
- if (!dsa_port_offloads_bridge(dp, obj->orig_dev))
+ if (!dsa_port_offloads_bridge_dev(dp, obj->orig_dev))
return -EOPNOTSUPP;
err = dsa_port_host_mdb_add(dp, SWITCHDEV_OBJ_PORT_MDB(obj));
@@ -421,13 +421,13 @@ static int dsa_slave_port_obj_add(struct net_device *dev, const void *ctx,
err = dsa_slave_vlan_add(dev, obj, extack);
break;
case SWITCHDEV_OBJ_ID_MRP:
- if (!dsa_port_offloads_bridge(dp, obj->orig_dev))
+ if (!dsa_port_offloads_bridge_dev(dp, obj->orig_dev))
return -EOPNOTSUPP;
err = dsa_port_mrp_add(dp, SWITCHDEV_OBJ_MRP(obj));
break;
case SWITCHDEV_OBJ_ID_RING_ROLE_MRP:
- if (!dsa_port_offloads_bridge(dp, obj->orig_dev))
+ if (!dsa_port_offloads_bridge_dev(dp, obj->orig_dev))
return -EOPNOTSUPP;
err = dsa_port_mrp_add_ring_role(dp,
@@ -483,7 +483,7 @@ static int dsa_slave_port_obj_del(struct net_device *dev, const void *ctx,
err = dsa_port_mdb_del(dp, SWITCHDEV_OBJ_PORT_MDB(obj));
break;
case SWITCHDEV_OBJ_ID_HOST_MDB:
- if (!dsa_port_offloads_bridge(dp, obj->orig_dev))
+ if (!dsa_port_offloads_bridge_dev(dp, obj->orig_dev))
return -EOPNOTSUPP;
err = dsa_port_host_mdb_del(dp, SWITCHDEV_OBJ_PORT_MDB(obj));
@@ -495,13 +495,13 @@ static int dsa_slave_port_obj_del(struct net_device *dev, const void *ctx,
err = dsa_slave_vlan_del(dev, obj);
break;
case SWITCHDEV_OBJ_ID_MRP:
- if (!dsa_port_offloads_bridge(dp, obj->orig_dev))
+ if (!dsa_port_offloads_bridge_dev(dp, obj->orig_dev))
return -EOPNOTSUPP;
err = dsa_port_mrp_del(dp, SWITCHDEV_OBJ_MRP(obj));
break;
case SWITCHDEV_OBJ_ID_RING_ROLE_MRP:
- if (!dsa_port_offloads_bridge(dp, obj->orig_dev))
+ if (!dsa_port_offloads_bridge_dev(dp, obj->orig_dev))
return -EOPNOTSUPP;
err = dsa_port_mrp_del_ring_role(dp,
@@ -1564,7 +1564,7 @@ static void dsa_bridge_mtu_normalization(struct dsa_port *dp)
if (!dp->ds->mtu_enforcement_ingress)
return;
- if (!dp->bridge_dev)
+ if (!dp->bridge)
return;
INIT_LIST_HEAD(&hw_port_list);
@@ -1580,7 +1580,7 @@ static void dsa_bridge_mtu_normalization(struct dsa_port *dp)
if (other_dp->type != DSA_PORT_TYPE_USER)
continue;
- if (other_dp->bridge_dev != dp->bridge_dev)
+ if (!dsa_port_bridge_same(dp, other_dp))
continue;
if (!other_dp->ds->mtu_enforcement_ingress)
@@ -1851,14 +1851,9 @@ static int dsa_slave_phy_setup(struct net_device *slave_dev)
struct dsa_port *dp = dsa_slave_to_port(slave_dev);
struct device_node *port_dn = dp->dn;
struct dsa_switch *ds = dp->ds;
- phy_interface_t mode;
u32 phy_flags = 0;
int ret;
- ret = of_get_phy_mode(port_dn, &mode);
- if (ret)
- mode = PHY_INTERFACE_MODE_NA;
-
dp->pl_config.dev = &slave_dev->dev;
dp->pl_config.type = PHYLINK_NETDEV;
@@ -1871,17 +1866,9 @@ static int dsa_slave_phy_setup(struct net_device *slave_dev)
dp->pl_config.poll_fixed_state = true;
}
- if (ds->ops->phylink_get_interfaces)
- ds->ops->phylink_get_interfaces(ds, dp->index,
- dp->pl_config.supported_interfaces);
-
- dp->pl = phylink_create(&dp->pl_config, of_fwnode_handle(port_dn), mode,
- &dsa_port_phylink_mac_ops);
- if (IS_ERR(dp->pl)) {
- netdev_err(slave_dev,
- "error creating PHYLINK: %ld\n", PTR_ERR(dp->pl));
- return PTR_ERR(dp->pl);
- }
+ ret = dsa_port_phylink_create(dp);
+ if (ret)
+ return ret;
if (ds->ops->get_phy_flags)
phy_flags = ds->ops->get_phy_flags(ds, dp->index);
@@ -2024,13 +2011,6 @@ int dsa_slave_create(struct dsa_port *port)
port->slave = slave_dev;
dsa_slave_setup_tagger(slave_dev);
- rtnl_lock();
- ret = dsa_slave_change_mtu(slave_dev, ETH_DATA_LEN);
- rtnl_unlock();
- if (ret && ret != -EOPNOTSUPP)
- dev_warn(ds->dev, "nonfatal error %d setting MTU to %d on port %d\n",
- ret, ETH_DATA_LEN, port->index);
-
netif_carrier_off(slave_dev);
ret = dsa_slave_phy_setup(slave_dev);
@@ -2043,6 +2023,11 @@ int dsa_slave_create(struct dsa_port *port)
rtnl_lock();
+ ret = dsa_slave_change_mtu(slave_dev, ETH_DATA_LEN);
+ if (ret && ret != -EOPNOTSUPP)
+ dev_warn(ds->dev, "nonfatal error %d setting MTU to %d on port %d\n",
+ ret, ETH_DATA_LEN, port->index);
+
ret = register_netdevice(slave_dev);
if (ret) {
netdev_err(master, "error %d registering interface %s\n",
@@ -2233,7 +2218,7 @@ dsa_prevent_bridging_8021q_upper(struct net_device *dev,
struct netdev_notifier_changeupper_info *info)
{
struct netlink_ext_ack *ext_ack;
- struct net_device *slave;
+ struct net_device *slave, *br;
struct dsa_port *dp;
ext_ack = netdev_notifier_info_to_extack(&info->info);
@@ -2246,11 +2231,12 @@ dsa_prevent_bridging_8021q_upper(struct net_device *dev,
return NOTIFY_DONE;
dp = dsa_slave_to_port(slave);
- if (!dp->bridge_dev)
+ br = dsa_port_bridge_dev_get(dp);
+ if (!br)
return NOTIFY_DONE;
/* Deny enslaving a VLAN device into a VLAN-aware bridge */
- if (br_vlan_enabled(dp->bridge_dev) &&
+ if (br_vlan_enabled(br) &&
netif_is_bridge_master(info->upper_dev) && info->linking) {
NL_SET_ERR_MSG_MOD(ext_ack,
"Cannot enslave VLAN device into VLAN aware bridge");
@@ -2265,7 +2251,7 @@ dsa_slave_check_8021q_upper(struct net_device *dev,
struct netdev_notifier_changeupper_info *info)
{
struct dsa_port *dp = dsa_slave_to_port(dev);
- struct net_device *br = dp->bridge_dev;
+ struct net_device *br = dsa_port_bridge_dev_get(dp);
struct bridge_vlan_info br_info;
struct netlink_ext_ack *extack;
int err = NOTIFY_DONE;
@@ -2462,7 +2448,7 @@ static bool dsa_foreign_dev_check(const struct net_device *dev,
struct dsa_switch_tree *dst = dp->ds->dst;
if (netif_is_bridge_master(foreign_dev))
- return !dsa_tree_offloads_bridge(dst, foreign_dev);
+ return !dsa_tree_offloads_bridge_dev(dst, foreign_dev);
if (netif_is_bridge_port(foreign_dev))
return !dsa_tree_offloads_bridge_port(dst, foreign_dev);
diff --git a/net/dsa/switch.c b/net/dsa/switch.c
index bb155a1..e3c7d26 100644
--- a/net/dsa/switch.c
+++ b/net/dsa/switch.c
@@ -95,7 +95,8 @@ static int dsa_switch_bridge_join(struct dsa_switch *ds,
if (!ds->ops->port_bridge_join)
return -EOPNOTSUPP;
- err = ds->ops->port_bridge_join(ds, info->port, info->br);
+ err = ds->ops->port_bridge_join(ds, info->port, info->bridge,
+ &info->tx_fwd_offload);
if (err)
return err;
}
@@ -104,7 +105,7 @@ static int dsa_switch_bridge_join(struct dsa_switch *ds,
ds->ops->crosschip_bridge_join) {
err = ds->ops->crosschip_bridge_join(ds, info->tree_index,
info->sw_index,
- info->port, info->br);
+ info->port, info->bridge);
if (err)
return err;
}
@@ -124,19 +125,20 @@ static int dsa_switch_bridge_leave(struct dsa_switch *ds,
if (dst->index == info->tree_index && ds->index == info->sw_index &&
ds->ops->port_bridge_leave)
- ds->ops->port_bridge_leave(ds, info->port, info->br);
+ ds->ops->port_bridge_leave(ds, info->port, info->bridge);
if ((dst->index != info->tree_index || ds->index != info->sw_index) &&
ds->ops->crosschip_bridge_leave)
ds->ops->crosschip_bridge_leave(ds, info->tree_index,
info->sw_index, info->port,
- info->br);
+ info->bridge);
- if (ds->needs_standalone_vlan_filtering && !br_vlan_enabled(info->br)) {
+ if (ds->needs_standalone_vlan_filtering &&
+ !br_vlan_enabled(info->bridge.dev)) {
change_vlan_filtering = true;
vlan_filtering = true;
} else if (!ds->needs_standalone_vlan_filtering &&
- br_vlan_enabled(info->br)) {
+ br_vlan_enabled(info->bridge.dev)) {
change_vlan_filtering = true;
vlan_filtering = false;
}
@@ -151,11 +153,9 @@ static int dsa_switch_bridge_leave(struct dsa_switch *ds,
*/
if (change_vlan_filtering && ds->vlan_filtering_is_global) {
dsa_switch_for_each_port(dp, ds) {
- struct net_device *bridge_dev;
+ struct net_device *br = dsa_port_bridge_dev_get(dp);
- bridge_dev = dp->bridge_dev;
-
- if (bridge_dev && br_vlan_enabled(bridge_dev)) {
+ if (br && br_vlan_enabled(br)) {
change_vlan_filtering = false;
break;
}
@@ -437,24 +437,6 @@ static int dsa_switch_fdb_del(struct dsa_switch *ds,
return dsa_port_do_fdb_del(dp, info->addr, info->vid);
}
-static int dsa_switch_hsr_join(struct dsa_switch *ds,
- struct dsa_notifier_hsr_info *info)
-{
- if (ds->index == info->sw_index && ds->ops->port_hsr_join)
- return ds->ops->port_hsr_join(ds, info->port, info->hsr);
-
- return -EOPNOTSUPP;
-}
-
-static int dsa_switch_hsr_leave(struct dsa_switch *ds,
- struct dsa_notifier_hsr_info *info)
-{
- if (ds->index == info->sw_index && ds->ops->port_hsr_leave)
- return ds->ops->port_hsr_leave(ds, info->port, info->hsr);
-
- return -EOPNOTSUPP;
-}
-
static int dsa_switch_lag_change(struct dsa_switch *ds,
struct dsa_notifier_lag_info *info)
{
@@ -647,55 +629,57 @@ static int dsa_switch_change_tag_proto(struct dsa_switch *ds,
return 0;
}
-static int dsa_switch_mrp_add(struct dsa_switch *ds,
- struct dsa_notifier_mrp_info *info)
+/* We use the same cross-chip notifiers to inform both the tagger side, as well
+ * as the switch side, of connection and disconnection events.
+ * Since ds->tagger_data is owned by the tagger, it isn't a hard error if the
+ * switch side doesn't support connecting to this tagger, and therefore, the
+ * fact that we don't disconnect the tagger side doesn't constitute a memory
+ * leak: the tagger will still operate with persistent per-switch memory, just
+ * with the switch side unconnected to it. What does constitute a hard error is
+ * when the switch side supports connecting but fails.
+ */
+static int
+dsa_switch_connect_tag_proto(struct dsa_switch *ds,
+ struct dsa_notifier_tag_proto_info *info)
{
- if (!ds->ops->port_mrp_add)
+ const struct dsa_device_ops *tag_ops = info->tag_ops;
+ int err;
+
+ /* Notify the new tagger about the connection to this switch */
+ if (tag_ops->connect) {
+ err = tag_ops->connect(ds);
+ if (err)
+ return err;
+ }
+
+ if (!ds->ops->connect_tag_protocol)
return -EOPNOTSUPP;
- if (ds->index == info->sw_index)
- return ds->ops->port_mrp_add(ds, info->port, info->mrp);
-
- return 0;
-}
-
-static int dsa_switch_mrp_del(struct dsa_switch *ds,
- struct dsa_notifier_mrp_info *info)
-{
- if (!ds->ops->port_mrp_del)
- return -EOPNOTSUPP;
-
- if (ds->index == info->sw_index)
- return ds->ops->port_mrp_del(ds, info->port, info->mrp);
+ /* Notify the switch about the connection to the new tagger */
+ err = ds->ops->connect_tag_protocol(ds, tag_ops->proto);
+ if (err) {
+ /* Revert the new tagger's connection to this tree */
+ if (tag_ops->disconnect)
+ tag_ops->disconnect(ds);
+ return err;
+ }
return 0;
}
static int
-dsa_switch_mrp_add_ring_role(struct dsa_switch *ds,
- struct dsa_notifier_mrp_ring_role_info *info)
+dsa_switch_disconnect_tag_proto(struct dsa_switch *ds,
+ struct dsa_notifier_tag_proto_info *info)
{
- if (!ds->ops->port_mrp_add)
- return -EOPNOTSUPP;
+ const struct dsa_device_ops *tag_ops = info->tag_ops;
- if (ds->index == info->sw_index)
- return ds->ops->port_mrp_add_ring_role(ds, info->port,
- info->mrp);
+ /* Notify the tagger about the disconnection from this switch */
+ if (tag_ops->disconnect && ds->tagger_data)
+ tag_ops->disconnect(ds);
- return 0;
-}
-
-static int
-dsa_switch_mrp_del_ring_role(struct dsa_switch *ds,
- struct dsa_notifier_mrp_ring_role_info *info)
-{
- if (!ds->ops->port_mrp_del)
- return -EOPNOTSUPP;
-
- if (ds->index == info->sw_index)
- return ds->ops->port_mrp_del_ring_role(ds, info->port,
- info->mrp);
-
+ /* No need to notify the switch, since it shouldn't have any
+ * resources to tear down
+ */
return 0;
}
@@ -727,12 +711,6 @@ static int dsa_switch_event(struct notifier_block *nb,
case DSA_NOTIFIER_HOST_FDB_DEL:
err = dsa_switch_host_fdb_del(ds, info);
break;
- case DSA_NOTIFIER_HSR_JOIN:
- err = dsa_switch_hsr_join(ds, info);
- break;
- case DSA_NOTIFIER_HSR_LEAVE:
- err = dsa_switch_hsr_leave(ds, info);
- break;
case DSA_NOTIFIER_LAG_CHANGE:
err = dsa_switch_lag_change(ds, info);
break;
@@ -766,17 +744,11 @@ static int dsa_switch_event(struct notifier_block *nb,
case DSA_NOTIFIER_TAG_PROTO:
err = dsa_switch_change_tag_proto(ds, info);
break;
- case DSA_NOTIFIER_MRP_ADD:
- err = dsa_switch_mrp_add(ds, info);
+ case DSA_NOTIFIER_TAG_PROTO_CONNECT:
+ err = dsa_switch_connect_tag_proto(ds, info);
break;
- case DSA_NOTIFIER_MRP_DEL:
- err = dsa_switch_mrp_del(ds, info);
- break;
- case DSA_NOTIFIER_MRP_ADD_RING_ROLE:
- err = dsa_switch_mrp_add_ring_role(ds, info);
- break;
- case DSA_NOTIFIER_MRP_DEL_RING_ROLE:
- err = dsa_switch_mrp_del_ring_role(ds, info);
+ case DSA_NOTIFIER_TAG_PROTO_DISCONNECT:
+ err = dsa_switch_disconnect_tag_proto(ds, info);
break;
case DSA_NOTIFIER_TAG_8021Q_VLAN_ADD:
err = dsa_switch_tag_8021q_vlan_add(ds, info);
diff --git a/net/dsa/tag_8021q.c b/net/dsa/tag_8021q.c
index 72cac2c..27712a8 100644
--- a/net/dsa/tag_8021q.c
+++ b/net/dsa/tag_8021q.c
@@ -67,10 +67,12 @@
#define DSA_8021Q_PORT(x) (((x) << DSA_8021Q_PORT_SHIFT) & \
DSA_8021Q_PORT_MASK)
-u16 dsa_8021q_bridge_tx_fwd_offload_vid(int bridge_num)
+u16 dsa_8021q_bridge_tx_fwd_offload_vid(unsigned int bridge_num)
{
- /* The VBID value of 0 is reserved for precise TX */
- return DSA_8021Q_DIR_TX | DSA_8021Q_VBID(bridge_num + 1);
+ /* The VBID value of 0 is reserved for precise TX, but it is also
+ * reserved/invalid for the bridge_num, so all is well.
+ */
+ return DSA_8021Q_DIR_TX | DSA_8021Q_VBID(bridge_num);
}
EXPORT_SYMBOL_GPL(dsa_8021q_bridge_tx_fwd_offload_vid);
@@ -335,7 +337,7 @@ dsa_port_tag_8021q_bridge_match(struct dsa_port *dp,
return false;
if (dsa_port_is_user(dp))
- return dp->bridge_dev == info->br;
+ return dsa_port_offloads_bridge(dp, &info->bridge);
return false;
}
@@ -408,10 +410,9 @@ int dsa_tag_8021q_bridge_leave(struct dsa_switch *ds,
}
int dsa_tag_8021q_bridge_tx_fwd_offload(struct dsa_switch *ds, int port,
- struct net_device *br,
- int bridge_num)
+ struct dsa_bridge bridge)
{
- u16 tx_vid = dsa_8021q_bridge_tx_fwd_offload_vid(bridge_num);
+ u16 tx_vid = dsa_8021q_bridge_tx_fwd_offload_vid(bridge.num);
return dsa_port_tag_8021q_vlan_add(dsa_to_port(ds, port), tx_vid,
true);
@@ -419,10 +420,9 @@ int dsa_tag_8021q_bridge_tx_fwd_offload(struct dsa_switch *ds, int port,
EXPORT_SYMBOL_GPL(dsa_tag_8021q_bridge_tx_fwd_offload);
void dsa_tag_8021q_bridge_tx_fwd_unoffload(struct dsa_switch *ds, int port,
- struct net_device *br,
- int bridge_num)
+ struct dsa_bridge bridge)
{
- u16 tx_vid = dsa_8021q_bridge_tx_fwd_offload_vid(bridge_num);
+ u16 tx_vid = dsa_8021q_bridge_tx_fwd_offload_vid(bridge.num);
dsa_port_tag_8021q_vlan_del(dsa_to_port(ds, port), tx_vid, true);
}
diff --git a/net/dsa/tag_dsa.c b/net/dsa/tag_dsa.c
index b3da4b2..8abf39d 100644
--- a/net/dsa/tag_dsa.c
+++ b/net/dsa/tag_dsa.c
@@ -132,6 +132,7 @@ static struct sk_buff *dsa_xmit_ll(struct sk_buff *skb, struct net_device *dev,
u8 *dsa_header;
if (skb->offload_fwd_mark) {
+ unsigned int bridge_num = dsa_port_bridge_num_get(dp);
struct dsa_switch_tree *dst = dp->ds->dst;
cmd = DSA_CMD_FORWARD;
@@ -140,7 +141,7 @@ static struct sk_buff *dsa_xmit_ll(struct sk_buff *skb, struct net_device *dev,
* packets on behalf of a virtual switch device with an index
* past the physical switches.
*/
- tag_dev = dst->last_switch + 1 + dp->bridge_num;
+ tag_dev = dst->last_switch + bridge_num;
tag_port = 0;
} else {
cmd = DSA_CMD_FROM_CPU;
@@ -165,7 +166,7 @@ static struct sk_buff *dsa_xmit_ll(struct sk_buff *skb, struct net_device *dev,
dsa_header[2] &= ~0x10;
}
} else {
- struct net_device *br = dp->bridge_dev;
+ struct net_device *br = dsa_port_bridge_dev_get(dp);
u16 vid;
vid = br ? MV88E6XXX_VID_BRIDGED : MV88E6XXX_VID_STANDALONE;
diff --git a/net/dsa/tag_ocelot.c b/net/dsa/tag_ocelot.c
index 4ed74d5..0d81f17 100644
--- a/net/dsa/tag_ocelot.c
+++ b/net/dsa/tag_ocelot.c
@@ -12,7 +12,7 @@
static void ocelot_xmit_get_vlan_info(struct sk_buff *skb, struct dsa_port *dp,
u64 *vlan_tci, u64 *tag_type)
{
- struct net_device *br = READ_ONCE(dp->bridge_dev);
+ struct net_device *br = dsa_port_bridge_dev_get(dp);
struct vlan_ethhdr *hdr;
u16 proto, tci;
diff --git a/net/dsa/tag_ocelot_8021q.c b/net/dsa/tag_ocelot_8021q.c
index a1919ea..68982b2 100644
--- a/net/dsa/tag_ocelot_8021q.c
+++ b/net/dsa/tag_ocelot_8021q.c
@@ -12,25 +12,39 @@
#include <linux/dsa/ocelot.h>
#include "dsa_priv.h"
+struct ocelot_8021q_tagger_private {
+ struct ocelot_8021q_tagger_data data; /* Must be first */
+ struct kthread_worker *xmit_worker;
+};
+
static struct sk_buff *ocelot_defer_xmit(struct dsa_port *dp,
struct sk_buff *skb)
{
+ struct ocelot_8021q_tagger_private *priv = dp->ds->tagger_data;
+ struct ocelot_8021q_tagger_data *data = &priv->data;
+ void (*xmit_work_fn)(struct kthread_work *work);
struct felix_deferred_xmit_work *xmit_work;
- struct felix_port *felix_port = dp->priv;
+ struct kthread_worker *xmit_worker;
+
+ xmit_work_fn = data->xmit_work_fn;
+ xmit_worker = priv->xmit_worker;
+
+ if (!xmit_work_fn || !xmit_worker)
+ return NULL;
xmit_work = kzalloc(sizeof(*xmit_work), GFP_ATOMIC);
if (!xmit_work)
return NULL;
/* Calls felix_port_deferred_xmit in felix.c */
- kthread_init_work(&xmit_work->work, felix_port->xmit_work_fn);
+ kthread_init_work(&xmit_work->work, xmit_work_fn);
/* Increase refcount so the kfree_skb in dsa_slave_xmit
* won't really free the packet.
*/
xmit_work->dp = dp;
xmit_work->skb = skb_get(skb);
- kthread_queue_work(felix_port->xmit_worker, &xmit_work->work);
+ kthread_queue_work(xmit_worker, &xmit_work->work);
return NULL;
}
@@ -67,11 +81,43 @@ static struct sk_buff *ocelot_rcv(struct sk_buff *skb,
return skb;
}
+static void ocelot_disconnect(struct dsa_switch *ds)
+{
+ struct ocelot_8021q_tagger_private *priv = ds->tagger_data;
+
+ kthread_destroy_worker(priv->xmit_worker);
+ kfree(priv);
+ ds->tagger_data = NULL;
+}
+
+static int ocelot_connect(struct dsa_switch *ds)
+{
+ struct ocelot_8021q_tagger_private *priv;
+ int err;
+
+ priv = kzalloc(sizeof(*priv), GFP_KERNEL);
+ if (!priv)
+ return -ENOMEM;
+
+ priv->xmit_worker = kthread_create_worker(0, "felix_xmit");
+ if (IS_ERR(priv->xmit_worker)) {
+ err = PTR_ERR(priv->xmit_worker);
+ kfree(priv);
+ return err;
+ }
+
+ ds->tagger_data = priv;
+
+ return 0;
+}
+
static const struct dsa_device_ops ocelot_8021q_netdev_ops = {
.name = "ocelot-8021q",
.proto = DSA_TAG_PROTO_OCELOT_8021Q,
.xmit = ocelot_xmit,
.rcv = ocelot_rcv,
+ .connect = ocelot_connect,
+ .disconnect = ocelot_disconnect,
.needed_headroom = VLAN_HLEN,
.promisc_on_master = true,
};
diff --git a/net/dsa/tag_sja1105.c b/net/dsa/tag_sja1105.c
index 262c883..72d5e0e 100644
--- a/net/dsa/tag_sja1105.c
+++ b/net/dsa/tag_sja1105.c
@@ -4,7 +4,6 @@
#include <linux/if_vlan.h>
#include <linux/dsa/sja1105.h>
#include <linux/dsa/8021q.h>
-#include <linux/skbuff.h>
#include <linux/packing.h>
#include "dsa_priv.h"
@@ -54,11 +53,25 @@
#define SJA1110_TX_TRAILER_LEN 4
#define SJA1110_MAX_PADDING_LEN 15
-enum sja1110_meta_tstamp {
- SJA1110_META_TSTAMP_TX = 0,
- SJA1110_META_TSTAMP_RX = 1,
+#define SJA1105_HWTS_RX_EN 0
+
+struct sja1105_tagger_private {
+ struct sja1105_tagger_data data; /* Must be first */
+ unsigned long state;
+ /* Protects concurrent access to the meta state machine
+ * from taggers running on multiple ports on SMP systems
+ */
+ spinlock_t meta_lock;
+ struct sk_buff *stampable_skb;
+ struct kthread_worker *xmit_worker;
};
+static struct sja1105_tagger_private *
+sja1105_tagger_private(struct dsa_switch *ds)
+{
+ return ds->tagger_data;
+}
+
/* Similar to is_link_local_ether_addr(hdr->h_dest) but also covers PTP */
static inline bool sja1105_is_link_local(const struct sk_buff *skb)
{
@@ -125,16 +138,30 @@ static inline bool sja1105_is_meta_frame(const struct sk_buff *skb)
static struct sk_buff *sja1105_defer_xmit(struct dsa_port *dp,
struct sk_buff *skb)
{
- struct sja1105_port *sp = dp->priv;
+ struct sja1105_tagger_data *tagger_data = sja1105_tagger_data(dp->ds);
+ struct sja1105_tagger_private *priv = sja1105_tagger_private(dp->ds);
+ void (*xmit_work_fn)(struct kthread_work *work);
+ struct sja1105_deferred_xmit_work *xmit_work;
+ struct kthread_worker *xmit_worker;
- if (!dsa_port_is_sja1105(dp))
- return skb;
+ xmit_work_fn = tagger_data->xmit_work_fn;
+ xmit_worker = priv->xmit_worker;
+ if (!xmit_work_fn || !xmit_worker)
+ return NULL;
+
+ xmit_work = kzalloc(sizeof(*xmit_work), GFP_ATOMIC);
+ if (!xmit_work)
+ return NULL;
+
+ kthread_init_work(&xmit_work->work, xmit_work_fn);
/* Increase refcount so the kfree_skb in dsa_slave_xmit
* won't really free the packet.
*/
- skb_queue_tail(&sp->xmit_queue, skb_get(skb));
- kthread_queue_work(sp->xmit_worker, &sp->xmit_work);
+ xmit_work->dp = dp;
+ xmit_work->skb = skb_get(skb);
+
+ kthread_queue_work(xmit_worker, &xmit_work->work);
return NULL;
}
@@ -159,14 +186,16 @@ static u16 sja1105_xmit_tpid(struct dsa_port *dp)
* need to find it.
*/
dsa_switch_for_each_port(other_dp, ds) {
- if (!other_dp->bridge_dev)
+ struct net_device *br = dsa_port_bridge_dev_get(other_dp);
+
+ if (!br)
continue;
/* Error is returned only if CONFIG_BRIDGE_VLAN_FILTERING,
* which seems pointless to handle, as our port cannot become
* VLAN-aware in that case.
*/
- br_vlan_get_proto(other_dp->bridge_dev, &proto);
+ br_vlan_get_proto(br, &proto);
return proto;
}
@@ -180,7 +209,8 @@ static struct sk_buff *sja1105_imprecise_xmit(struct sk_buff *skb,
struct net_device *netdev)
{
struct dsa_port *dp = dsa_slave_to_port(netdev);
- struct net_device *br = dp->bridge_dev;
+ unsigned int bridge_num = dsa_port_bridge_num_get(dp);
+ struct net_device *br = dsa_port_bridge_dev_get(dp);
u16 tx_vid;
/* If the port is under a VLAN-aware bridge, just slide the
@@ -196,7 +226,7 @@ static struct sk_buff *sja1105_imprecise_xmit(struct sk_buff *skb,
* TX VLAN that targets the bridge's entire broadcast domain,
* instead of just the specific port.
*/
- tx_vid = dsa_8021q_bridge_tx_fwd_offload_vid(dp->bridge_num);
+ tx_vid = dsa_8021q_bridge_tx_fwd_offload_vid(bridge_num);
return dsa_8021q_xmit(skb, netdev, sja1105_xmit_tpid(dp), tx_vid);
}
@@ -352,32 +382,32 @@ static struct sk_buff
*/
if (is_link_local) {
struct dsa_port *dp = dsa_slave_to_port(skb->dev);
- struct sja1105_port *sp = dp->priv;
+ struct sja1105_tagger_private *priv;
+ struct dsa_switch *ds = dp->ds;
- if (unlikely(!dsa_port_is_sja1105(dp)))
- return skb;
+ priv = sja1105_tagger_private(ds);
- if (!test_bit(SJA1105_HWTS_RX_EN, &sp->data->state))
+ if (!test_bit(SJA1105_HWTS_RX_EN, &priv->state))
/* Do normal processing. */
return skb;
- spin_lock(&sp->data->meta_lock);
+ spin_lock(&priv->meta_lock);
/* Was this a link-local frame instead of the meta
* that we were expecting?
*/
- if (sp->data->stampable_skb) {
- dev_err_ratelimited(dp->ds->dev,
+ if (priv->stampable_skb) {
+ dev_err_ratelimited(ds->dev,
"Expected meta frame, is %12llx "
"in the DSA master multicast filter?\n",
SJA1105_META_DMAC);
- kfree_skb(sp->data->stampable_skb);
+ kfree_skb(priv->stampable_skb);
}
/* Hold a reference to avoid dsa_switch_rcv
* from freeing the skb.
*/
- sp->data->stampable_skb = skb_get(skb);
- spin_unlock(&sp->data->meta_lock);
+ priv->stampable_skb = skb_get(skb);
+ spin_unlock(&priv->meta_lock);
/* Tell DSA we got nothing */
return NULL;
@@ -390,37 +420,37 @@ static struct sk_buff
*/
} else if (is_meta) {
struct dsa_port *dp = dsa_slave_to_port(skb->dev);
- struct sja1105_port *sp = dp->priv;
+ struct sja1105_tagger_private *priv;
+ struct dsa_switch *ds = dp->ds;
struct sk_buff *stampable_skb;
- if (unlikely(!dsa_port_is_sja1105(dp)))
- return skb;
+ priv = sja1105_tagger_private(ds);
/* Drop the meta frame if we're not in the right state
* to process it.
*/
- if (!test_bit(SJA1105_HWTS_RX_EN, &sp->data->state))
+ if (!test_bit(SJA1105_HWTS_RX_EN, &priv->state))
return NULL;
- spin_lock(&sp->data->meta_lock);
+ spin_lock(&priv->meta_lock);
- stampable_skb = sp->data->stampable_skb;
- sp->data->stampable_skb = NULL;
+ stampable_skb = priv->stampable_skb;
+ priv->stampable_skb = NULL;
/* Was this a meta frame instead of the link-local
* that we were expecting?
*/
if (!stampable_skb) {
- dev_err_ratelimited(dp->ds->dev,
+ dev_err_ratelimited(ds->dev,
"Unexpected meta frame\n");
- spin_unlock(&sp->data->meta_lock);
+ spin_unlock(&priv->meta_lock);
return NULL;
}
if (stampable_skb->dev != skb->dev) {
- dev_err_ratelimited(dp->ds->dev,
+ dev_err_ratelimited(ds->dev,
"Meta frame on wrong port\n");
- spin_unlock(&sp->data->meta_lock);
+ spin_unlock(&priv->meta_lock);
return NULL;
}
@@ -431,12 +461,36 @@ static struct sk_buff
skb = stampable_skb;
sja1105_transfer_meta(skb, meta);
- spin_unlock(&sp->data->meta_lock);
+ spin_unlock(&priv->meta_lock);
}
return skb;
}
+static bool sja1105_rxtstamp_get_state(struct dsa_switch *ds)
+{
+ struct sja1105_tagger_private *priv = sja1105_tagger_private(ds);
+
+ return test_bit(SJA1105_HWTS_RX_EN, &priv->state);
+}
+
+static void sja1105_rxtstamp_set_state(struct dsa_switch *ds, bool on)
+{
+ struct sja1105_tagger_private *priv = sja1105_tagger_private(ds);
+
+ if (on)
+ set_bit(SJA1105_HWTS_RX_EN, &priv->state);
+ else
+ clear_bit(SJA1105_HWTS_RX_EN, &priv->state);
+
+ /* Initialize the meta state machine to a known state */
+ if (!priv->stampable_skb)
+ return;
+
+ kfree_skb(priv->stampable_skb);
+ priv->stampable_skb = NULL;
+}
+
static bool sja1105_skb_has_tag_8021q(const struct sk_buff *skb)
{
u16 tpid = ntohs(eth_hdr(skb)->h_proto);
@@ -523,48 +577,12 @@ static struct sk_buff *sja1105_rcv(struct sk_buff *skb,
is_meta);
}
-static void sja1110_process_meta_tstamp(struct dsa_switch *ds, int port,
- u8 ts_id, enum sja1110_meta_tstamp dir,
- u64 tstamp)
-{
- struct sk_buff *skb, *skb_tmp, *skb_match = NULL;
- struct dsa_port *dp = dsa_to_port(ds, port);
- struct skb_shared_hwtstamps shwt = {0};
- struct sja1105_port *sp = dp->priv;
-
- if (!dsa_port_is_sja1105(dp))
- return;
-
- /* We don't care about RX timestamps on the CPU port */
- if (dir == SJA1110_META_TSTAMP_RX)
- return;
-
- spin_lock(&sp->data->skb_txtstamp_queue.lock);
-
- skb_queue_walk_safe(&sp->data->skb_txtstamp_queue, skb, skb_tmp) {
- if (SJA1105_SKB_CB(skb)->ts_id != ts_id)
- continue;
-
- __skb_unlink(skb, &sp->data->skb_txtstamp_queue);
- skb_match = skb;
-
- break;
- }
-
- spin_unlock(&sp->data->skb_txtstamp_queue.lock);
-
- if (WARN_ON(!skb_match))
- return;
-
- shwt.hwtstamp = ns_to_ktime(sja1105_ticks_to_ns(tstamp));
- skb_complete_tx_timestamp(skb_match, &shwt);
-}
-
static struct sk_buff *sja1110_rcv_meta(struct sk_buff *skb, u16 rx_header)
{
u8 *buf = dsa_etype_header_pos_rx(skb) + SJA1110_HEADER_LEN;
int switch_id = SJA1110_RX_HEADER_SWITCH_ID(rx_header);
int n_ts = SJA1110_RX_HEADER_N_TS(rx_header);
+ struct sja1105_tagger_data *tagger_data;
struct net_device *master = skb->dev;
struct dsa_port *cpu_dp;
struct dsa_switch *ds;
@@ -578,6 +596,10 @@ static struct sk_buff *sja1110_rcv_meta(struct sk_buff *skb, u16 rx_header)
return NULL;
}
+ tagger_data = sja1105_tagger_data(ds);
+ if (!tagger_data->meta_tstamp_handler)
+ return NULL;
+
for (i = 0; i <= n_ts; i++) {
u8 ts_id, source_port, dir;
u64 tstamp;
@@ -587,8 +609,8 @@ static struct sk_buff *sja1110_rcv_meta(struct sk_buff *skb, u16 rx_header)
dir = (buf[1] & BIT(3)) >> 3;
tstamp = be64_to_cpu(*(__be64 *)(buf + 2));
- sja1110_process_meta_tstamp(ds, source_port, ts_id, dir,
- tstamp);
+ tagger_data->meta_tstamp_handler(ds, source_port, ts_id, dir,
+ tstamp);
buf += SJA1110_META_TSTAMP_SIZE;
}
@@ -719,11 +741,53 @@ static void sja1110_flow_dissect(const struct sk_buff *skb, __be16 *proto,
*proto = ((__be16 *)skb->data)[(VLAN_HLEN / 2) - 1];
}
+static void sja1105_disconnect(struct dsa_switch *ds)
+{
+ struct sja1105_tagger_private *priv = ds->tagger_data;
+
+ kthread_destroy_worker(priv->xmit_worker);
+ kfree(priv);
+ ds->tagger_data = NULL;
+}
+
+static int sja1105_connect(struct dsa_switch *ds)
+{
+ struct sja1105_tagger_data *tagger_data;
+ struct sja1105_tagger_private *priv;
+ struct kthread_worker *xmit_worker;
+ int err;
+
+ priv = kzalloc(sizeof(*priv), GFP_KERNEL);
+ if (!priv)
+ return -ENOMEM;
+
+ spin_lock_init(&priv->meta_lock);
+
+ xmit_worker = kthread_create_worker(0, "dsa%d:%d_xmit",
+ ds->dst->index, ds->index);
+ if (IS_ERR(xmit_worker)) {
+ err = PTR_ERR(xmit_worker);
+ kfree(priv);
+ return err;
+ }
+
+ priv->xmit_worker = xmit_worker;
+ /* Export functions for switch driver use */
+ tagger_data = &priv->data;
+ tagger_data->rxtstamp_get_state = sja1105_rxtstamp_get_state;
+ tagger_data->rxtstamp_set_state = sja1105_rxtstamp_set_state;
+ ds->tagger_data = priv;
+
+ return 0;
+}
+
static const struct dsa_device_ops sja1105_netdev_ops = {
.name = "sja1105",
.proto = DSA_TAG_PROTO_SJA1105,
.xmit = sja1105_xmit,
.rcv = sja1105_rcv,
+ .connect = sja1105_connect,
+ .disconnect = sja1105_disconnect,
.needed_headroom = VLAN_HLEN,
.flow_dissect = sja1105_flow_dissect,
.promisc_on_master = true,
@@ -737,6 +801,8 @@ static const struct dsa_device_ops sja1110_netdev_ops = {
.proto = DSA_TAG_PROTO_SJA1110,
.xmit = sja1110_xmit,
.rcv = sja1110_rcv,
+ .connect = sja1105_connect,
+ .disconnect = sja1105_disconnect,
.flow_dissect = sja1110_flow_dissect,
.needed_headroom = SJA1110_HEADER_LEN + VLAN_HLEN,
.needed_tailroom = SJA1110_RX_TRAILER_LEN + SJA1110_MAX_PADDING_LEN,
diff --git a/net/ethernet/eth.c b/net/ethernet/eth.c
index c7d9e08..ebcc812 100644
--- a/net/ethernet/eth.c
+++ b/net/ethernet/eth.c
@@ -436,11 +436,10 @@ struct sk_buff *eth_gro_receive(struct list_head *head, struct sk_buff *skb)
type = eh->h_proto;
- rcu_read_lock();
ptype = gro_find_receive_by_type(type);
if (ptype == NULL) {
flush = 1;
- goto out_unlock;
+ goto out;
}
skb_gro_pull(skb, sizeof(*eh));
@@ -450,8 +449,6 @@ struct sk_buff *eth_gro_receive(struct list_head *head, struct sk_buff *skb)
ipv6_gro_receive, inet_gro_receive,
head, skb);
-out_unlock:
- rcu_read_unlock();
out:
skb_gro_flush_final(skb, pp, flush);
@@ -469,14 +466,12 @@ int eth_gro_complete(struct sk_buff *skb, int nhoff)
if (skb->encapsulation)
skb_set_inner_mac_header(skb, nhoff);
- rcu_read_lock();
ptype = gro_find_complete_by_type(type);
if (ptype != NULL)
err = INDIRECT_CALL_INET(ptype->callbacks.gro_complete,
ipv6_gro_complete, inet_gro_complete,
skb, nhoff + sizeof(*eh));
- rcu_read_unlock();
return err;
}
EXPORT_SYMBOL(eth_gro_complete);
diff --git a/net/ethtool/cabletest.c b/net/ethtool/cabletest.c
index 63560bb..920aac0 100644
--- a/net/ethtool/cabletest.c
+++ b/net/ethtool/cabletest.c
@@ -96,7 +96,7 @@ int ethnl_act_cable_test(struct sk_buff *skb, struct genl_info *info)
out_rtnl:
rtnl_unlock();
out_dev_put:
- dev_put(dev);
+ ethnl_parse_header_dev_put(&req_info);
return ret;
}
@@ -353,7 +353,7 @@ int ethnl_act_cable_test_tdr(struct sk_buff *skb, struct genl_info *info)
out_rtnl:
rtnl_unlock();
out_dev_put:
- dev_put(dev);
+ ethnl_parse_header_dev_put(&req_info);
return ret;
}
diff --git a/net/ethtool/channels.c b/net/ethtool/channels.c
index 6a070dc..4031588 100644
--- a/net/ethtool/channels.c
+++ b/net/ethtool/channels.c
@@ -219,6 +219,6 @@ int ethnl_set_channels(struct sk_buff *skb, struct genl_info *info)
out_rtnl:
rtnl_unlock();
out_dev:
- dev_put(dev);
+ ethnl_parse_header_dev_put(&req_info);
return ret;
}
diff --git a/net/ethtool/coalesce.c b/net/ethtool/coalesce.c
index 46776ea..487bdf3 100644
--- a/net/ethtool/coalesce.c
+++ b/net/ethtool/coalesce.c
@@ -336,6 +336,6 @@ int ethnl_set_coalesce(struct sk_buff *skb, struct genl_info *info)
out_rtnl:
rtnl_unlock();
out_dev:
- dev_put(dev);
+ ethnl_parse_header_dev_put(&req_info);
return ret;
}
diff --git a/net/ethtool/common.c b/net/ethtool/common.c
index c63e073..0c52100 100644
--- a/net/ethtool/common.c
+++ b/net/ethtool/common.c
@@ -89,6 +89,7 @@ tunable_strings[__ETHTOOL_TUNABLE_COUNT][ETH_GSTRING_LEN] = {
[ETHTOOL_RX_COPYBREAK] = "rx-copybreak",
[ETHTOOL_TX_COPYBREAK] = "tx-copybreak",
[ETHTOOL_PFC_PREVENTION_TOUT] = "pfc-prevention-tout",
+ [ETHTOOL_TX_COPYBREAK_BUF_SIZE] = "tx-copybreak-buf-size",
};
const char
diff --git a/net/ethtool/debug.c b/net/ethtool/debug.c
index f99912d..d73888c 100644
--- a/net/ethtool/debug.c
+++ b/net/ethtool/debug.c
@@ -123,6 +123,6 @@ int ethnl_set_debug(struct sk_buff *skb, struct genl_info *info)
out_rtnl:
rtnl_unlock();
out_dev:
- dev_put(dev);
+ ethnl_parse_header_dev_put(&req_info);
return ret;
}
diff --git a/net/ethtool/eee.c b/net/ethtool/eee.c
index e10bfcc..45c42b2 100644
--- a/net/ethtool/eee.c
+++ b/net/ethtool/eee.c
@@ -185,6 +185,6 @@ int ethnl_set_eee(struct sk_buff *skb, struct genl_info *info)
out_rtnl:
rtnl_unlock();
out_dev:
- dev_put(dev);
+ ethnl_parse_header_dev_put(&req_info);
return ret;
}
diff --git a/net/ethtool/features.c b/net/ethtool/features.c
index 1c9f4df..55d449a 100644
--- a/net/ethtool/features.c
+++ b/net/ethtool/features.c
@@ -136,7 +136,6 @@ static void ethnl_features_to_bitmap(unsigned long *dest, netdev_features_t val)
const unsigned int words = BITS_TO_LONGS(NETDEV_FEATURE_COUNT);
unsigned int i;
- bitmap_zero(dest, NETDEV_FEATURE_COUNT);
for (i = 0; i < words; i++)
dest[i] = (unsigned long)(val >> (i * BITS_PER_LONG));
}
@@ -284,6 +283,6 @@ int ethnl_set_features(struct sk_buff *skb, struct genl_info *info)
out_rtnl:
rtnl_unlock();
- dev_put(dev);
+ ethnl_parse_header_dev_put(&req_info);
return ret;
}
diff --git a/net/ethtool/fec.c b/net/ethtool/fec.c
index 8738daf..9f5a134 100644
--- a/net/ethtool/fec.c
+++ b/net/ethtool/fec.c
@@ -305,6 +305,6 @@ int ethnl_set_fec(struct sk_buff *skb, struct genl_info *info)
out_rtnl:
rtnl_unlock();
out_dev:
- dev_put(dev);
+ ethnl_parse_header_dev_put(&req_info);
return ret;
}
diff --git a/net/ethtool/ioctl.c b/net/ethtool/ioctl.c
index 20bcf86..326e14e 100644
--- a/net/ethtool/ioctl.c
+++ b/net/ethtool/ioctl.c
@@ -8,6 +8,7 @@
*/
#include <linux/compat.h>
+#include <linux/etherdevice.h>
#include <linux/module.h>
#include <linux/types.h>
#include <linux/capability.h>
@@ -734,6 +735,9 @@ ethtool_get_drvinfo(struct net_device *dev, struct ethtool_devlink_compat *rsp)
sizeof(rsp->info.bus_info));
strlcpy(rsp->info.driver, dev->dev.parent->driver->name,
sizeof(rsp->info.driver));
+ } else if (dev->rtnl_link_ops) {
+ strlcpy(rsp->info.driver, dev->rtnl_link_ops->kind,
+ sizeof(rsp->info.driver));
} else {
return -EOPNOTSUPP;
}
@@ -1743,11 +1747,13 @@ static noinline_for_stack int ethtool_set_coalesce(struct net_device *dev,
static int ethtool_get_ringparam(struct net_device *dev, void __user *useraddr)
{
struct ethtool_ringparam ringparam = { .cmd = ETHTOOL_GRINGPARAM };
+ struct kernel_ethtool_ringparam kernel_ringparam = {};
if (!dev->ethtool_ops->get_ringparam)
return -EOPNOTSUPP;
- dev->ethtool_ops->get_ringparam(dev, &ringparam);
+ dev->ethtool_ops->get_ringparam(dev, &ringparam,
+ &kernel_ringparam, NULL);
if (copy_to_user(useraddr, &ringparam, sizeof(ringparam)))
return -EFAULT;
@@ -1757,6 +1763,7 @@ static int ethtool_get_ringparam(struct net_device *dev, void __user *useraddr)
static int ethtool_set_ringparam(struct net_device *dev, void __user *useraddr)
{
struct ethtool_ringparam ringparam, max = { .cmd = ETHTOOL_GRINGPARAM };
+ struct kernel_ethtool_ringparam kernel_ringparam;
int ret;
if (!dev->ethtool_ops->set_ringparam || !dev->ethtool_ops->get_ringparam)
@@ -1765,7 +1772,7 @@ static int ethtool_set_ringparam(struct net_device *dev, void __user *useraddr)
if (copy_from_user(&ringparam, useraddr, sizeof(ringparam)))
return -EFAULT;
- dev->ethtool_ops->get_ringparam(dev, &max);
+ dev->ethtool_ops->get_ringparam(dev, &max, &kernel_ringparam, NULL);
/* ensure new ring parameters are within the maximums */
if (ringparam.rx_pending > max.rx_max_pending ||
@@ -1774,7 +1781,8 @@ static int ethtool_set_ringparam(struct net_device *dev, void __user *useraddr)
ringparam.tx_pending > max.tx_max_pending)
return -EINVAL;
- ret = dev->ethtool_ops->set_ringparam(dev, &ringparam);
+ ret = dev->ethtool_ops->set_ringparam(dev, &ringparam,
+ &kernel_ringparam, NULL);
if (!ret)
ethtool_notify(dev, ETHTOOL_MSG_RINGS_NTF, NULL);
return ret;
@@ -1982,6 +1990,7 @@ static int ethtool_phys_id(struct net_device *dev, void __user *useraddr)
struct ethtool_value id;
static bool busy;
const struct ethtool_ops *ops = dev->ethtool_ops;
+ netdevice_tracker dev_tracker;
int rc;
if (!ops->set_phys_id)
@@ -2001,7 +2010,7 @@ static int ethtool_phys_id(struct net_device *dev, void __user *useraddr)
* removal of the device.
*/
busy = true;
- dev_hold(dev);
+ dev_hold_track(dev, &dev_tracker, GFP_KERNEL);
rtnl_unlock();
if (rc == 0) {
@@ -2025,7 +2034,7 @@ static int ethtool_phys_id(struct net_device *dev, void __user *useraddr)
}
rtnl_lock();
- dev_put(dev);
+ dev_put_track(dev, &dev_tracker);
busy = false;
(void) ops->set_phys_id(dev, ETHTOOL_ID_INACTIVE);
@@ -2087,9 +2096,9 @@ static int ethtool_get_phy_stats(struct net_device *dev, void __user *useraddr)
if (!phydev && (!ops->get_ethtool_phy_stats || !ops->get_sset_count))
return -EOPNOTSUPP;
- if (dev->phydev && !ops->get_ethtool_phy_stats &&
+ if (phydev && !ops->get_ethtool_phy_stats &&
phy_ops && phy_ops->get_sset_count)
- n_stats = phy_ops->get_sset_count(dev->phydev);
+ n_stats = phy_ops->get_sset_count(phydev);
else
n_stats = ops->get_sset_count(dev, ETH_SS_PHY_STATS);
if (n_stats < 0)
@@ -2108,9 +2117,9 @@ static int ethtool_get_phy_stats(struct net_device *dev, void __user *useraddr)
if (!data)
return -ENOMEM;
- if (dev->phydev && !ops->get_ethtool_phy_stats &&
+ if (phydev && !ops->get_ethtool_phy_stats &&
phy_ops && phy_ops->get_stats) {
- ret = phy_ops->get_stats(dev->phydev, &stats, data);
+ ret = phy_ops->get_stats(phydev, &stats, data);
if (ret < 0)
goto out;
} else {
@@ -2396,6 +2405,7 @@ static int ethtool_tunable_valid(const struct ethtool_tunable *tuna)
switch (tuna->id) {
case ETHTOOL_RX_COPYBREAK:
case ETHTOOL_TX_COPYBREAK:
+ case ETHTOOL_TX_COPYBREAK_BUF_SIZE:
if (tuna->len != sizeof(u32) ||
tuna->type_id != ETHTOOL_TUNABLE_U32)
return -EINVAL;
diff --git a/net/ethtool/linkinfo.c b/net/ethtool/linkinfo.c
index b918398..efa0f7f 100644
--- a/net/ethtool/linkinfo.c
+++ b/net/ethtool/linkinfo.c
@@ -149,6 +149,6 @@ int ethnl_set_linkinfo(struct sk_buff *skb, struct genl_info *info)
out_rtnl:
rtnl_unlock();
out_dev:
- dev_put(dev);
+ ethnl_parse_header_dev_put(&req_info);
return ret;
}
diff --git a/net/ethtool/linkmodes.c b/net/ethtool/linkmodes.c
index f9eda59..99b29b4 100644
--- a/net/ethtool/linkmodes.c
+++ b/net/ethtool/linkmodes.c
@@ -358,6 +358,6 @@ int ethnl_set_linkmodes(struct sk_buff *skb, struct genl_info *info)
out_rtnl:
rtnl_unlock();
out_dev:
- dev_put(dev);
+ ethnl_parse_header_dev_put(&req_info);
return ret;
}
diff --git a/net/ethtool/module.c b/net/ethtool/module.c
index bc2cef1..898ed43 100644
--- a/net/ethtool/module.c
+++ b/net/ethtool/module.c
@@ -175,6 +175,6 @@ int ethnl_set_module(struct sk_buff *skb, struct genl_info *info)
ethnl_ops_complete(dev);
out_rtnl:
rtnl_unlock();
- dev_put(dev);
+ ethnl_parse_header_dev_put(&req_info);
return ret;
}
diff --git a/net/ethtool/netlink.c b/net/ethtool/netlink.c
index 96f4180..5fe8f4a 100644
--- a/net/ethtool/netlink.c
+++ b/net/ethtool/netlink.c
@@ -142,6 +142,8 @@ int ethnl_parse_header_dev_get(struct ethnl_req_info *req_info,
}
req_info->dev = dev;
+ if (dev)
+ netdev_tracker_alloc(dev, &req_info->dev_tracker, GFP_KERNEL);
req_info->flags = flags;
return 0;
}
@@ -400,7 +402,7 @@ static int ethnl_default_doit(struct sk_buff *skb, struct genl_info *info)
ops->cleanup_data(reply_data);
genlmsg_end(rskb, reply_payload);
- dev_put(req_info->dev);
+ dev_put_track(req_info->dev, &req_info->dev_tracker);
kfree(reply_data);
kfree(req_info);
return genlmsg_reply(rskb, info);
@@ -412,7 +414,7 @@ static int ethnl_default_doit(struct sk_buff *skb, struct genl_info *info)
if (ops->cleanup_data)
ops->cleanup_data(reply_data);
err_dev:
- dev_put(req_info->dev);
+ dev_put_track(req_info->dev, &req_info->dev_tracker);
kfree(reply_data);
kfree(req_info);
return ret;
@@ -548,7 +550,7 @@ static int ethnl_default_start(struct netlink_callback *cb)
* same parser as for non-dump (doit) requests is used, it
* would take reference to the device if it finds one
*/
- dev_put(req_info->dev);
+ dev_put_track(req_info->dev, &req_info->dev_tracker);
req_info->dev = NULL;
}
if (ret < 0)
@@ -635,7 +637,6 @@ static void ethnl_default_notify(struct net_device *dev, unsigned int cmd,
if (ret < 0)
goto err_cleanup;
reply_len = ret + ethnl_reply_header_size();
- ret = -ENOMEM;
skb = genlmsg_new(reply_len, GFP_KERNEL);
if (!skb)
goto err_cleanup;
diff --git a/net/ethtool/netlink.h b/net/ethtool/netlink.h
index 836ee71..75856db 100644
--- a/net/ethtool/netlink.h
+++ b/net/ethtool/netlink.h
@@ -222,6 +222,7 @@ static inline unsigned int ethnl_reply_header_size(void)
/**
* struct ethnl_req_info - base type of request information for GET requests
* @dev: network device the request is for (may be null)
+ * @dev_tracker: refcount tracker for @dev reference
* @flags: request flags common for all request types
*
* This is a common base for request specific structures holding data from
@@ -230,9 +231,15 @@ static inline unsigned int ethnl_reply_header_size(void)
*/
struct ethnl_req_info {
struct net_device *dev;
+ netdevice_tracker dev_tracker;
u32 flags;
};
+static inline void ethnl_parse_header_dev_put(struct ethnl_req_info *req_info)
+{
+ dev_put_track(req_info->dev, &req_info->dev_tracker);
+}
+
/**
* struct ethnl_reply_data - base type of reply data for GET requests
* @dev: device for current reply message; in single shot requests it is
@@ -356,7 +363,7 @@ extern const struct nla_policy ethnl_features_set_policy[ETHTOOL_A_FEATURES_WANT
extern const struct nla_policy ethnl_privflags_get_policy[ETHTOOL_A_PRIVFLAGS_HEADER + 1];
extern const struct nla_policy ethnl_privflags_set_policy[ETHTOOL_A_PRIVFLAGS_FLAGS + 1];
extern const struct nla_policy ethnl_rings_get_policy[ETHTOOL_A_RINGS_HEADER + 1];
-extern const struct nla_policy ethnl_rings_set_policy[ETHTOOL_A_RINGS_TX + 1];
+extern const struct nla_policy ethnl_rings_set_policy[ETHTOOL_A_RINGS_RX_BUF_LEN + 1];
extern const struct nla_policy ethnl_channels_get_policy[ETHTOOL_A_CHANNELS_HEADER + 1];
extern const struct nla_policy ethnl_channels_set_policy[ETHTOOL_A_CHANNELS_COMBINED_COUNT + 1];
extern const struct nla_policy ethnl_coalesce_get_policy[ETHTOOL_A_COALESCE_HEADER + 1];
diff --git a/net/ethtool/pause.c b/net/ethtool/pause.c
index ee1e580..a8c113d 100644
--- a/net/ethtool/pause.c
+++ b/net/ethtool/pause.c
@@ -181,6 +181,6 @@ int ethnl_set_pause(struct sk_buff *skb, struct genl_info *info)
out_rtnl:
rtnl_unlock();
out_dev:
- dev_put(dev);
+ ethnl_parse_header_dev_put(&req_info);
return ret;
}
diff --git a/net/ethtool/privflags.c b/net/ethtool/privflags.c
index fc9f3be..4c7bfa8 100644
--- a/net/ethtool/privflags.c
+++ b/net/ethtool/privflags.c
@@ -196,6 +196,6 @@ int ethnl_set_privflags(struct sk_buff *skb, struct genl_info *info)
out_rtnl:
rtnl_unlock();
out_dev:
- dev_put(dev);
+ ethnl_parse_header_dev_put(&req_info);
return ret;
}
diff --git a/net/ethtool/rings.c b/net/ethtool/rings.c
index 4e09781..c1d5f5e 100644
--- a/net/ethtool/rings.c
+++ b/net/ethtool/rings.c
@@ -10,6 +10,7 @@ struct rings_req_info {
struct rings_reply_data {
struct ethnl_reply_data base;
struct ethtool_ringparam ringparam;
+ struct kernel_ethtool_ringparam kernel_ringparam;
};
#define RINGS_REPDATA(__reply_base) \
@@ -25,6 +26,7 @@ static int rings_prepare_data(const struct ethnl_req_info *req_base,
struct genl_info *info)
{
struct rings_reply_data *data = RINGS_REPDATA(reply_base);
+ struct netlink_ext_ack *extack = info ? info->extack : NULL;
struct net_device *dev = reply_base->dev;
int ret;
@@ -33,7 +35,8 @@ static int rings_prepare_data(const struct ethnl_req_info *req_base,
ret = ethnl_ops_begin(dev);
if (ret < 0)
return ret;
- dev->ethtool_ops->get_ringparam(dev, &data->ringparam);
+ dev->ethtool_ops->get_ringparam(dev, &data->ringparam,
+ &data->kernel_ringparam, extack);
ethnl_ops_complete(dev);
return 0;
@@ -49,7 +52,8 @@ static int rings_reply_size(const struct ethnl_req_info *req_base,
nla_total_size(sizeof(u32)) + /* _RINGS_RX */
nla_total_size(sizeof(u32)) + /* _RINGS_RX_MINI */
nla_total_size(sizeof(u32)) + /* _RINGS_RX_JUMBO */
- nla_total_size(sizeof(u32)); /* _RINGS_TX */
+ nla_total_size(sizeof(u32)) + /* _RINGS_TX */
+ nla_total_size(sizeof(u32)); /* _RINGS_RX_BUF_LEN */
}
static int rings_fill_reply(struct sk_buff *skb,
@@ -57,6 +61,7 @@ static int rings_fill_reply(struct sk_buff *skb,
const struct ethnl_reply_data *reply_base)
{
const struct rings_reply_data *data = RINGS_REPDATA(reply_base);
+ const struct kernel_ethtool_ringparam *kernel_ringparam = &data->kernel_ringparam;
const struct ethtool_ringparam *ringparam = &data->ringparam;
if ((ringparam->rx_max_pending &&
@@ -78,7 +83,10 @@ static int rings_fill_reply(struct sk_buff *skb,
(nla_put_u32(skb, ETHTOOL_A_RINGS_TX_MAX,
ringparam->tx_max_pending) ||
nla_put_u32(skb, ETHTOOL_A_RINGS_TX,
- ringparam->tx_pending))))
+ ringparam->tx_pending))) ||
+ (kernel_ringparam->rx_buf_len &&
+ (nla_put_u32(skb, ETHTOOL_A_RINGS_RX_BUF_LEN,
+ kernel_ringparam->rx_buf_len))))
return -EMSGSIZE;
return 0;
@@ -105,10 +113,12 @@ const struct nla_policy ethnl_rings_set_policy[] = {
[ETHTOOL_A_RINGS_RX_MINI] = { .type = NLA_U32 },
[ETHTOOL_A_RINGS_RX_JUMBO] = { .type = NLA_U32 },
[ETHTOOL_A_RINGS_TX] = { .type = NLA_U32 },
+ [ETHTOOL_A_RINGS_RX_BUF_LEN] = NLA_POLICY_MIN(NLA_U32, 1),
};
int ethnl_set_rings(struct sk_buff *skb, struct genl_info *info)
{
+ struct kernel_ethtool_ringparam kernel_ringparam = {};
struct ethtool_ringparam ringparam = {};
struct ethnl_req_info req_info = {};
struct nlattr **tb = info->attrs;
@@ -134,7 +144,7 @@ int ethnl_set_rings(struct sk_buff *skb, struct genl_info *info)
ret = ethnl_ops_begin(dev);
if (ret < 0)
goto out_rtnl;
- ops->get_ringparam(dev, &ringparam);
+ ops->get_ringparam(dev, &ringparam, &kernel_ringparam, info->extack);
ethnl_update_u32(&ringparam.rx_pending, tb[ETHTOOL_A_RINGS_RX], &mod);
ethnl_update_u32(&ringparam.rx_mini_pending,
@@ -142,6 +152,8 @@ int ethnl_set_rings(struct sk_buff *skb, struct genl_info *info)
ethnl_update_u32(&ringparam.rx_jumbo_pending,
tb[ETHTOOL_A_RINGS_RX_JUMBO], &mod);
ethnl_update_u32(&ringparam.tx_pending, tb[ETHTOOL_A_RINGS_TX], &mod);
+ ethnl_update_u32(&kernel_ringparam.rx_buf_len,
+ tb[ETHTOOL_A_RINGS_RX_BUF_LEN], &mod);
ret = 0;
if (!mod)
goto out_ops;
@@ -164,7 +176,17 @@ int ethnl_set_rings(struct sk_buff *skb, struct genl_info *info)
goto out_ops;
}
- ret = dev->ethtool_ops->set_ringparam(dev, &ringparam);
+ if (kernel_ringparam.rx_buf_len != 0 &&
+ !(ops->supported_ring_params & ETHTOOL_RING_USE_RX_BUF_LEN)) {
+ ret = -EOPNOTSUPP;
+ NL_SET_ERR_MSG_ATTR(info->extack,
+ tb[ETHTOOL_A_RINGS_RX_BUF_LEN],
+ "setting rx buf len not supported");
+ goto out_ops;
+ }
+
+ ret = dev->ethtool_ops->set_ringparam(dev, &ringparam,
+ &kernel_ringparam, info->extack);
if (ret < 0)
goto out_ops;
ethtool_notify(dev, ETHTOOL_MSG_RINGS_NTF, NULL);
@@ -174,6 +196,6 @@ int ethnl_set_rings(struct sk_buff *skb, struct genl_info *info)
out_rtnl:
rtnl_unlock();
out_dev:
- dev_put(dev);
+ ethnl_parse_header_dev_put(&req_info);
return ret;
}
diff --git a/net/ethtool/stats.c b/net/ethtool/stats.c
index ec07f57..a20e0a2 100644
--- a/net/ethtool/stats.c
+++ b/net/ethtool/stats.c
@@ -14,10 +14,12 @@ struct stats_req_info {
struct stats_reply_data {
struct ethnl_reply_data base;
- struct ethtool_eth_phy_stats phy_stats;
- struct ethtool_eth_mac_stats mac_stats;
- struct ethtool_eth_ctrl_stats ctrl_stats;
- struct ethtool_rmon_stats rmon_stats;
+ struct_group(stats,
+ struct ethtool_eth_phy_stats phy_stats;
+ struct ethtool_eth_mac_stats mac_stats;
+ struct ethtool_eth_ctrl_stats ctrl_stats;
+ struct ethtool_rmon_stats rmon_stats;
+ );
const struct ethtool_rmon_hist_range *rmon_ranges;
};
@@ -117,10 +119,7 @@ static int stats_prepare_data(const struct ethnl_req_info *req_base,
/* Mark all stats as unset (see ETHTOOL_STAT_NOT_SET) to prevent them
* from being reported to user space in case driver did not set them.
*/
- memset(&data->phy_stats, 0xff, sizeof(data->phy_stats));
- memset(&data->mac_stats, 0xff, sizeof(data->mac_stats));
- memset(&data->ctrl_stats, 0xff, sizeof(data->ctrl_stats));
- memset(&data->rmon_stats, 0xff, sizeof(data->rmon_stats));
+ memset(&data->stats, 0xff, sizeof(data->stats));
if (test_bit(ETHTOOL_STATS_ETH_PHY, req_info->stat_mask) &&
dev->ethtool_ops->get_eth_phy_stats)
diff --git a/net/ethtool/tunnels.c b/net/ethtool/tunnels.c
index e7f2ee0..efde335 100644
--- a/net/ethtool/tunnels.c
+++ b/net/ethtool/tunnels.c
@@ -195,7 +195,7 @@ int ethnl_tunnel_info_doit(struct sk_buff *skb, struct genl_info *info)
if (ret)
goto err_free_msg;
rtnl_unlock();
- dev_put(req_info.dev);
+ ethnl_parse_header_dev_put(&req_info);
genlmsg_end(rskb, reply_payload);
return genlmsg_reply(rskb, info);
@@ -204,7 +204,7 @@ int ethnl_tunnel_info_doit(struct sk_buff *skb, struct genl_info *info)
nlmsg_free(rskb);
err_unlock_rtnl:
rtnl_unlock();
- dev_put(req_info.dev);
+ ethnl_parse_header_dev_put(&req_info);
return ret;
}
@@ -230,7 +230,7 @@ int ethnl_tunnel_info_start(struct netlink_callback *cb)
sock_net(cb->skb->sk), cb->extack,
false);
if (ctx->req_info.dev) {
- dev_put(ctx->req_info.dev);
+ ethnl_parse_header_dev_put(&ctx->req_info);
ctx->req_info.dev = NULL;
}
diff --git a/net/ethtool/wol.c b/net/ethtool/wol.c
index ada7df2..88f435e 100644
--- a/net/ethtool/wol.c
+++ b/net/ethtool/wol.c
@@ -165,6 +165,6 @@ int ethnl_set_wol(struct sk_buff *skb, struct genl_info *info)
out_rtnl:
rtnl_unlock();
out_dev:
- dev_put(dev);
+ ethnl_parse_header_dev_put(&req_info);
return ret;
}
diff --git a/net/hsr/hsr_device.c b/net/hsr/hsr_device.c
index 737e4f1..e57fdad 100644
--- a/net/hsr/hsr_device.c
+++ b/net/hsr/hsr_device.c
@@ -30,13 +30,13 @@ static bool is_slave_up(struct net_device *dev)
static void __hsr_set_operstate(struct net_device *dev, int transition)
{
- write_lock_bh(&dev_base_lock);
+ write_lock(&dev_base_lock);
if (dev->operstate != transition) {
dev->operstate = transition;
- write_unlock_bh(&dev_base_lock);
+ write_unlock(&dev_base_lock);
netdev_state_change(dev);
} else {
- write_unlock_bh(&dev_base_lock);
+ write_unlock(&dev_base_lock);
}
}
diff --git a/net/ieee802154/socket.c b/net/ieee802154/socket.c
index 7bb9ef3..3b2366a 100644
--- a/net/ieee802154/socket.c
+++ b/net/ieee802154/socket.c
@@ -174,8 +174,8 @@ static int raw_hash(struct sock *sk)
{
write_lock_bh(&raw_lock);
sk_add_node(sk, &raw_head);
- sock_prot_inuse_add(sock_net(sk), sk->sk_prot, 1);
write_unlock_bh(&raw_lock);
+ sock_prot_inuse_add(sock_net(sk), sk->sk_prot, 1);
return 0;
}
@@ -453,8 +453,8 @@ static int dgram_hash(struct sock *sk)
{
write_lock_bh(&dgram_lock);
sk_add_node(sk, &dgram_head);
- sock_prot_inuse_add(sock_net(sk), sk->sk_prot, 1);
write_unlock_bh(&dgram_lock);
+ sock_prot_inuse_add(sock_net(sk), sk->sk_prot, 1);
return 0;
}
diff --git a/net/ipv4/af_inet.c b/net/ipv4/af_inet.c
index 6b59565..9c465ba 100644
--- a/net/ipv4/af_inet.c
+++ b/net/ipv4/af_inet.c
@@ -99,6 +99,7 @@
#include <net/route.h>
#include <net/ip_fib.h>
#include <net/inet_connection_sock.h>
+#include <net/gro.h>
#include <net/tcp.h>
#include <net/udp.h>
#include <net/udplite.h>
@@ -224,7 +225,7 @@ int inet_listen(struct socket *sock, int backlog)
tcp_fastopen_init_key_once(sock_net(sk));
}
- err = inet_csk_listen_start(sk, backlog);
+ err = inet_csk_listen_start(sk);
if (err)
goto out;
tcp_call_bpf(sk, BPF_SOCK_OPS_TCP_LISTEN_CB, 0, NULL);
@@ -488,11 +489,8 @@ int __inet_bind(struct sock *sk, struct sockaddr *uaddr, int addr_len,
* is temporarily down)
*/
err = -EADDRNOTAVAIL;
- if (!inet_can_nonlocal_bind(net, inet) &&
- addr->sin_addr.s_addr != htonl(INADDR_ANY) &&
- chk_addr_ret != RTN_LOCAL &&
- chk_addr_ret != RTN_MULTICAST &&
- chk_addr_ret != RTN_BROADCAST)
+ if (!inet_addr_valid_or_nonlocal(net, inet, addr->sin_addr.s_addr,
+ chk_addr_ret))
goto out;
snum = ntohs(addr->sin_port);
@@ -533,6 +531,8 @@ int __inet_bind(struct sock *sk, struct sockaddr *uaddr, int addr_len,
err = BPF_CGROUP_RUN_PROG_INET4_POST_BIND(sk);
if (err) {
inet->inet_saddr = inet->inet_rcv_saddr = 0;
+ if (sk->sk_prot->put_port)
+ sk->sk_prot->put_port(sk);
goto out_release_sock;
}
}
@@ -1454,19 +1454,18 @@ struct sk_buff *inet_gro_receive(struct list_head *head, struct sk_buff *skb)
proto = iph->protocol;
- rcu_read_lock();
ops = rcu_dereference(inet_offloads[proto]);
if (!ops || !ops->callbacks.gro_receive)
- goto out_unlock;
+ goto out;
if (*(u8 *)iph != 0x45)
- goto out_unlock;
+ goto out;
if (ip_is_fragment(iph))
- goto out_unlock;
+ goto out;
if (unlikely(ip_fast_csum((u8 *)iph, 5)))
- goto out_unlock;
+ goto out;
id = ntohl(*(__be32 *)&iph->id);
flush = (u16)((ntohl(*(__be32 *)iph) ^ skb_gro_len(skb)) | (id & ~IP_DF));
@@ -1543,9 +1542,6 @@ struct sk_buff *inet_gro_receive(struct list_head *head, struct sk_buff *skb)
pp = indirect_call_gro_receive(tcp4_gro_receive, udp4_gro_receive,
ops->callbacks.gro_receive, head, skb);
-out_unlock:
- rcu_read_unlock();
-
out:
skb_gro_flush_final(skb, pp, flush);
@@ -1618,10 +1614,9 @@ int inet_gro_complete(struct sk_buff *skb, int nhoff)
csum_replace2(&iph->check, iph->tot_len, newlen);
iph->tot_len = newlen;
- rcu_read_lock();
ops = rcu_dereference(inet_offloads[proto]);
if (WARN_ON(!ops || !ops->callbacks.gro_complete))
- goto out_unlock;
+ goto out;
/* Only need to add sizeof(*iph) to get to the next hdr below
* because any hdr with option will have been flushed in
@@ -1631,9 +1626,7 @@ int inet_gro_complete(struct sk_buff *skb, int nhoff)
tcp4_gro_complete, udp4_gro_complete,
skb, nhoff + sizeof(*iph));
-out_unlock:
- rcu_read_unlock();
-
+out:
return err;
}
@@ -1994,6 +1987,10 @@ static int __init inet_init(void)
ip_init();
+ /* Initialise per-cpu ipv4 mibs */
+ if (init_ipv4_mibs())
+ panic("%s: Cannot init ipv4 mibs\n", __func__);
+
/* Setup TCP slab cache for open requests. */
tcp_init();
@@ -2024,12 +2021,6 @@ static int __init inet_init(void)
if (init_inet_pernet_ops())
pr_crit("%s: Cannot init ipv4 inet pernet ops\n", __func__);
- /*
- * Initialise per-cpu ipv4 mibs
- */
-
- if (init_ipv4_mibs())
- pr_crit("%s: Cannot init ipv4 mibs\n", __func__);
ipv4_proc_init();
diff --git a/net/ipv4/arp.c b/net/ipv4/arp.c
index 857a144b1..4db0325 100644
--- a/net/ipv4/arp.c
+++ b/net/ipv4/arp.c
@@ -1299,21 +1299,6 @@ static struct packet_type arp_packet_type __read_mostly = {
.func = arp_rcv,
};
-static int arp_proc_init(void);
-
-void __init arp_init(void)
-{
- neigh_table_init(NEIGH_ARP_TABLE, &arp_tbl);
-
- dev_add_pack(&arp_packet_type);
- arp_proc_init();
-#ifdef CONFIG_SYSCTL
- neigh_sysctl_register(NULL, &arp_tbl.parms, NULL);
-#endif
- register_netdevice_notifier(&arp_netdev_notifier);
-}
-
-#ifdef CONFIG_PROC_FS
#if IS_ENABLED(CONFIG_AX25)
/* ------------------------------------------------------------------------ */
@@ -1451,16 +1436,14 @@ static struct pernet_operations arp_net_ops = {
.exit = arp_net_exit,
};
-static int __init arp_proc_init(void)
+void __init arp_init(void)
{
- return register_pernet_subsys(&arp_net_ops);
+ neigh_table_init(NEIGH_ARP_TABLE, &arp_tbl);
+
+ dev_add_pack(&arp_packet_type);
+ register_pernet_subsys(&arp_net_ops);
+#ifdef CONFIG_SYSCTL
+ neigh_sysctl_register(NULL, &arp_tbl.parms, NULL);
+#endif
+ register_netdevice_notifier(&arp_netdev_notifier);
}
-
-#else /* CONFIG_PROC_FS */
-
-static int __init arp_proc_init(void)
-{
- return 0;
-}
-
-#endif /* CONFIG_PROC_FS */
diff --git a/net/ipv4/bpf_tcp_ca.c b/net/ipv4/bpf_tcp_ca.c
index 4bb9401..de610cb 100644
--- a/net/ipv4/bpf_tcp_ca.c
+++ b/net/ipv4/bpf_tcp_ca.c
@@ -169,7 +169,7 @@ static u32 prog_ops_moff(const struct bpf_prog *prog)
t = bpf_tcp_congestion_ops.type;
m = &btf_type_member(t)[midx];
- return btf_member_bit_offset(t, m) / 8;
+ return __btf_member_bit_offset(t, m) / 8;
}
static const struct bpf_func_proto *
@@ -246,7 +246,7 @@ static int bpf_tcp_ca_init_member(const struct btf_type *t,
utcp_ca = (const struct tcp_congestion_ops *)udata;
tcp_ca = (struct tcp_congestion_ops *)kdata;
- moff = btf_member_bit_offset(t, member) / 8;
+ moff = __btf_member_bit_offset(t, member) / 8;
switch (moff) {
case offsetof(struct tcp_congestion_ops, flags):
if (utcp_ca->flags & ~TCP_CONG_MASK)
@@ -276,7 +276,7 @@ static int bpf_tcp_ca_init_member(const struct btf_type *t,
static int bpf_tcp_ca_check_member(const struct btf_type *t,
const struct btf_member *member)
{
- if (is_unsupported(btf_member_bit_offset(t, member) / 8))
+ if (is_unsupported(__btf_member_bit_offset(t, member) / 8))
return -ENOTSUPP;
return 0;
}
diff --git a/net/ipv4/devinet.c b/net/ipv4/devinet.c
index 323e622..fba2bff 100644
--- a/net/ipv4/devinet.c
+++ b/net/ipv4/devinet.c
@@ -243,7 +243,7 @@ void in_dev_finish_destroy(struct in_device *idev)
#ifdef NET_REFCNT_DEBUG
pr_debug("%s: %p=%s\n", __func__, idev, dev ? dev->name : "NIL");
#endif
- dev_put(dev);
+ dev_put_track(dev, &idev->dev_tracker);
if (!idev->dead)
pr_err("Freeing alive in_device %p\n", idev);
else
@@ -271,7 +271,7 @@ static struct in_device *inetdev_init(struct net_device *dev)
if (IPV4_DEVCONF(in_dev->cnf, FORWARDING))
dev_disable_lro(dev);
/* Reference in_dev->dev */
- dev_hold(dev);
+ dev_hold_track(dev, &in_dev->dev_tracker, GFP_KERNEL);
/* Account for reference dev->ip_ptr (below) */
refcount_set(&in_dev->refcnt, 1);
diff --git a/net/ipv4/esp4_offload.c b/net/ipv4/esp4_offload.c
index 8e4e9aa..d87f02a 100644
--- a/net/ipv4/esp4_offload.c
+++ b/net/ipv4/esp4_offload.c
@@ -16,6 +16,7 @@
#include <crypto/authenc.h>
#include <linux/err.h>
#include <linux/module.h>
+#include <net/gro.h>
#include <net/ip.h>
#include <net/xfrm.h>
#include <net/esp.h>
diff --git a/net/ipv4/fib_rules.c b/net/ipv4/fib_rules.c
index d279cb8..e0b6c8b 100644
--- a/net/ipv4/fib_rules.c
+++ b/net/ipv4/fib_rules.c
@@ -216,11 +216,6 @@ static struct fib_table *fib_empty_table(struct net *net)
return NULL;
}
-static const struct nla_policy fib4_rule_policy[FRA_MAX+1] = {
- FRA_GENERIC_POLICY,
- [FRA_FLOW] = { .type = NLA_U32 },
-};
-
static int fib4_rule_configure(struct fib_rule *rule, struct sk_buff *skb,
struct fib_rule_hdr *frh,
struct nlattr **tb,
@@ -386,7 +381,6 @@ static const struct fib_rules_ops __net_initconst fib4_rules_ops_template = {
.nlmsg_payload = fib4_rule_nlmsg_payload,
.flush_cache = fib4_rule_flush_cache,
.nlgroup = RTNLGRP_IPV4_RULE,
- .policy = fib4_rule_policy,
.owner = THIS_MODULE,
};
diff --git a/net/ipv4/fib_semantics.c b/net/ipv4/fib_semantics.c
index fde7797..828de17 100644
--- a/net/ipv4/fib_semantics.c
+++ b/net/ipv4/fib_semantics.c
@@ -208,7 +208,7 @@ static void rt_fibinfo_free_cpus(struct rtable __rcu * __percpu *rtp)
void fib_nh_common_release(struct fib_nh_common *nhc)
{
- dev_put(nhc->nhc_dev);
+ dev_put_track(nhc->nhc_dev, &nhc->nhc_dev_tracker);
lwtstate_put(nhc->nhc_lwtstate);
rt_fibinfo_free_cpus(nhc->nhc_pcpu_rth_output);
rt_fibinfo_free(&nhc->nhc_rth_input);
@@ -662,6 +662,19 @@ static int fib_count_nexthops(struct rtnexthop *rtnh, int remaining,
return nhs;
}
+static int fib_gw_from_attr(__be32 *gw, struct nlattr *nla,
+ struct netlink_ext_ack *extack)
+{
+ if (nla_len(nla) < sizeof(*gw)) {
+ NL_SET_ERR_MSG(extack, "Invalid IPv4 address in RTA_GATEWAY");
+ return -EINVAL;
+ }
+
+ *gw = nla_get_in_addr(nla);
+
+ return 0;
+}
+
/* only called when fib_nh is integrated into fib_info */
static int fib_get_nhs(struct fib_info *fi, struct rtnexthop *rtnh,
int remaining, struct fib_config *cfg,
@@ -704,7 +717,11 @@ static int fib_get_nhs(struct fib_info *fi, struct rtnexthop *rtnh,
return -EINVAL;
}
if (nla) {
- fib_cfg.fc_gw4 = nla_get_in_addr(nla);
+ ret = fib_gw_from_attr(&fib_cfg.fc_gw4, nla,
+ extack);
+ if (ret)
+ goto errout;
+
if (fib_cfg.fc_gw4)
fib_cfg.fc_gw_family = AF_INET;
} else if (nlav) {
@@ -714,10 +731,18 @@ static int fib_get_nhs(struct fib_info *fi, struct rtnexthop *rtnh,
}
nla = nla_find(attrs, attrlen, RTA_FLOW);
- if (nla)
+ if (nla) {
+ if (nla_len(nla) < sizeof(u32)) {
+ NL_SET_ERR_MSG(extack, "Invalid RTA_FLOW");
+ return -EINVAL;
+ }
fib_cfg.fc_flow = nla_get_u32(nla);
+ }
fib_cfg.fc_encap = nla_find(attrs, attrlen, RTA_ENCAP);
+ /* RTA_ENCAP_TYPE length checked in
+ * lwtunnel_valid_encap_type_attr
+ */
nla = nla_find(attrs, attrlen, RTA_ENCAP_TYPE);
if (nla)
fib_cfg.fc_encap_type = nla_get_u16(nla);
@@ -902,6 +927,7 @@ int fib_nh_match(struct net *net, struct fib_config *cfg, struct fib_info *fi,
attrlen = rtnh_attrlen(rtnh);
if (attrlen > 0) {
struct nlattr *nla, *nlav, *attrs = rtnh_attrs(rtnh);
+ int err;
nla = nla_find(attrs, attrlen, RTA_GATEWAY);
nlav = nla_find(attrs, attrlen, RTA_VIA);
@@ -912,12 +938,17 @@ int fib_nh_match(struct net *net, struct fib_config *cfg, struct fib_info *fi,
}
if (nla) {
+ __be32 gw;
+
+ err = fib_gw_from_attr(&gw, nla, extack);
+ if (err)
+ return err;
+
if (nh->fib_nh_gw_family != AF_INET ||
- nla_get_in_addr(nla) != nh->fib_nh_gw4)
+ gw != nh->fib_nh_gw4)
return 1;
} else if (nlav) {
struct fib_config cfg2;
- int err;
err = fib_gw_from_via(&cfg2, nlav, extack);
if (err)
@@ -940,8 +971,14 @@ int fib_nh_match(struct net *net, struct fib_config *cfg, struct fib_info *fi,
#ifdef CONFIG_IP_ROUTE_CLASSID
nla = nla_find(attrs, attrlen, RTA_FLOW);
- if (nla && nla_get_u32(nla) != nh->nh_tclassid)
- return 1;
+ if (nla) {
+ if (nla_len(nla) < sizeof(u32)) {
+ NL_SET_ERR_MSG(extack, "Invalid RTA_FLOW");
+ return -EINVAL;
+ }
+ if (nla_get_u32(nla) != nh->nh_tclassid)
+ return 1;
+ }
#endif
}
@@ -1006,7 +1043,7 @@ static int fib_check_nh_v6_gw(struct net *net, struct fib_nh *nh,
err = ipv6_stub->fib6_nh_init(net, &fib6_nh, &cfg, GFP_KERNEL, extack);
if (!err) {
nh->fib_nh_dev = fib6_nh.fib_nh_dev;
- dev_hold(nh->fib_nh_dev);
+ dev_hold_track(nh->fib_nh_dev, &nh->fib_nh_dev_tracker, GFP_KERNEL);
nh->fib_nh_oif = nh->fib_nh_dev->ifindex;
nh->fib_nh_scope = RT_SCOPE_LINK;
@@ -1090,7 +1127,7 @@ static int fib_check_nh_v4_gw(struct net *net, struct fib_nh *nh, u32 table,
if (!netif_carrier_ok(dev))
nh->fib_nh_flags |= RTNH_F_LINKDOWN;
nh->fib_nh_dev = dev;
- dev_hold(dev);
+ dev_hold_track(dev, &nh->fib_nh_dev_tracker, GFP_ATOMIC);
nh->fib_nh_scope = RT_SCOPE_LINK;
return 0;
}
@@ -1144,7 +1181,7 @@ static int fib_check_nh_v4_gw(struct net *net, struct fib_nh *nh, u32 table,
"No egress device for nexthop gateway");
goto out;
}
- dev_hold(dev);
+ dev_hold_track(dev, &nh->fib_nh_dev_tracker, GFP_ATOMIC);
if (!netif_carrier_ok(dev))
nh->fib_nh_flags |= RTNH_F_LINKDOWN;
err = (dev->flags & IFF_UP) ? 0 : -ENETDOWN;
@@ -1178,7 +1215,7 @@ static int fib_check_nh_nongw(struct net *net, struct fib_nh *nh,
}
nh->fib_nh_dev = in_dev->dev;
- dev_hold(nh->fib_nh_dev);
+ dev_hold_track(nh->fib_nh_dev, &nh->fib_nh_dev_tracker, GFP_ATOMIC);
nh->fib_nh_scope = RT_SCOPE_HOST;
if (!netif_carrier_ok(nh->fib_nh_dev))
nh->fib_nh_flags |= RTNH_F_LINKDOWN;
@@ -1508,6 +1545,8 @@ struct fib_info *fib_create_info(struct fib_config *cfg,
err = -ENODEV;
if (!nh->fib_nh_dev)
goto failure;
+ netdev_tracker_alloc(nh->fib_nh_dev, &nh->fib_nh_dev_tracker,
+ GFP_KERNEL);
} else {
int linkdown = 0;
diff --git a/net/ipv4/fou.c b/net/ipv4/fou.c
index 8fcbc62..0d085cc 100644
--- a/net/ipv4/fou.c
+++ b/net/ipv4/fou.c
@@ -9,6 +9,7 @@
#include <linux/types.h>
#include <linux/kernel.h>
#include <net/genetlink.h>
+#include <net/gro.h>
#include <net/gue.h>
#include <net/fou.h>
#include <net/ip.h>
@@ -246,17 +247,14 @@ static struct sk_buff *fou_gro_receive(struct sock *sk,
/* Flag this frame as already having an outer encap header */
NAPI_GRO_CB(skb)->is_fou = 1;
- rcu_read_lock();
offloads = NAPI_GRO_CB(skb)->is_ipv6 ? inet6_offloads : inet_offloads;
ops = rcu_dereference(offloads[proto]);
if (!ops || !ops->callbacks.gro_receive)
- goto out_unlock;
+ goto out;
pp = call_gro_receive(ops->callbacks.gro_receive, head, skb);
-out_unlock:
- rcu_read_unlock();
-
+out:
return pp;
}
@@ -268,19 +266,16 @@ static int fou_gro_complete(struct sock *sk, struct sk_buff *skb,
const struct net_offload *ops;
int err = -ENOSYS;
- rcu_read_lock();
offloads = NAPI_GRO_CB(skb)->is_ipv6 ? inet6_offloads : inet_offloads;
ops = rcu_dereference(offloads[proto]);
if (WARN_ON(!ops || !ops->callbacks.gro_complete))
- goto out_unlock;
+ goto out;
err = ops->callbacks.gro_complete(skb, nhoff);
skb_set_inner_mac_header(skb, nhoff);
-out_unlock:
- rcu_read_unlock();
-
+out:
return err;
}
@@ -438,17 +433,14 @@ static struct sk_buff *gue_gro_receive(struct sock *sk,
/* Flag this frame as already having an outer encap header */
NAPI_GRO_CB(skb)->is_fou = 1;
- rcu_read_lock();
offloads = NAPI_GRO_CB(skb)->is_ipv6 ? inet6_offloads : inet_offloads;
ops = rcu_dereference(offloads[proto]);
if (WARN_ON_ONCE(!ops || !ops->callbacks.gro_receive))
- goto out_unlock;
+ goto out;
pp = call_gro_receive(ops->callbacks.gro_receive, head, skb);
flush = 0;
-out_unlock:
- rcu_read_unlock();
out:
skb_gro_flush_final_remcsum(skb, pp, flush, &grc);
@@ -485,18 +477,16 @@ static int gue_gro_complete(struct sock *sk, struct sk_buff *skb, int nhoff)
return err;
}
- rcu_read_lock();
offloads = NAPI_GRO_CB(skb)->is_ipv6 ? inet6_offloads : inet_offloads;
ops = rcu_dereference(offloads[proto]);
if (WARN_ON(!ops || !ops->callbacks.gro_complete))
- goto out_unlock;
+ goto out;
err = ops->callbacks.gro_complete(skb, nhoff + guehlen);
skb_set_inner_mac_header(skb, nhoff + guehlen);
-out_unlock:
- rcu_read_unlock();
+out:
return err;
}
diff --git a/net/ipv4/gre_offload.c b/net/ipv4/gre_offload.c
index 1121a9d..07073fa 100644
--- a/net/ipv4/gre_offload.c
+++ b/net/ipv4/gre_offload.c
@@ -10,6 +10,7 @@
#include <linux/init.h>
#include <net/protocol.h>
#include <net/gre.h>
+#include <net/gro.h>
static struct sk_buff *gre_gso_segment(struct sk_buff *skb,
netdev_features_t features)
@@ -162,10 +163,9 @@ static struct sk_buff *gre_gro_receive(struct list_head *head,
type = greh->protocol;
- rcu_read_lock();
ptype = gro_find_receive_by_type(type);
if (!ptype)
- goto out_unlock;
+ goto out;
grehlen = GRE_HEADER_SECTION;
@@ -179,13 +179,13 @@ static struct sk_buff *gre_gro_receive(struct list_head *head,
if (skb_gro_header_hard(skb, hlen)) {
greh = skb_gro_header_slow(skb, hlen, off);
if (unlikely(!greh))
- goto out_unlock;
+ goto out;
}
/* Don't bother verifying checksum if we're going to flush anyway. */
if ((greh->flags & GRE_CSUM) && !NAPI_GRO_CB(skb)->flush) {
if (skb_gro_checksum_simple_validate(skb))
- goto out_unlock;
+ goto out;
skb_gro_checksum_try_convert(skb, IPPROTO_GRE,
null_compute_pseudo);
@@ -229,8 +229,6 @@ static struct sk_buff *gre_gro_receive(struct list_head *head,
pp = call_gro_receive(ptype->callbacks.gro_receive, head, skb);
flush = 0;
-out_unlock:
- rcu_read_unlock();
out:
skb_gro_flush_final(skb, pp, flush);
@@ -255,13 +253,10 @@ static int gre_gro_complete(struct sk_buff *skb, int nhoff)
if (greh->flags & GRE_CSUM)
grehlen += GRE_HEADER_SECTION;
- rcu_read_lock();
ptype = gro_find_complete_by_type(type);
if (ptype)
err = ptype->callbacks.gro_complete(skb, nhoff + grehlen);
- rcu_read_unlock();
-
skb_set_inner_mac_header(skb, nhoff + grehlen);
return err;
diff --git a/net/ipv4/igmp.c b/net/ipv4/igmp.c
index d2e2b3d..2ad3c7b 100644
--- a/net/ipv4/igmp.c
+++ b/net/ipv4/igmp.c
@@ -2558,7 +2558,6 @@ int ip_mc_msfget(struct sock *sk, struct ip_msfilter *msf,
msf->imsf_fmode = pmc->sfmode;
psl = rtnl_dereference(pmc->sflist);
if (!psl) {
- len = 0;
count = 0;
} else {
count = psl->sl_count;
diff --git a/net/ipv4/inet_connection_sock.c b/net/ipv4/inet_connection_sock.c
index 62a67fd..fc2a985 100644
--- a/net/ipv4/inet_connection_sock.c
+++ b/net/ipv4/inet_connection_sock.c
@@ -1035,7 +1035,7 @@ void inet_csk_prepare_forced_close(struct sock *sk)
}
EXPORT_SYMBOL(inet_csk_prepare_forced_close);
-int inet_csk_listen_start(struct sock *sk, int backlog)
+int inet_csk_listen_start(struct sock *sk)
{
struct inet_connection_sock *icsk = inet_csk(sk);
struct inet_sock *inet = inet_sk(sk);
diff --git a/net/ipv4/inet_hashtables.c b/net/ipv4/inet_hashtables.c
index 7573726..30ab717f 100644
--- a/net/ipv4/inet_hashtables.c
+++ b/net/ipv4/inet_hashtables.c
@@ -307,7 +307,7 @@ static inline struct sock *inet_lookup_run_bpf(struct net *net,
struct inet_hashinfo *hashinfo,
struct sk_buff *skb, int doff,
__be32 saddr, __be16 sport,
- __be32 daddr, u16 hnum)
+ __be32 daddr, u16 hnum, const int dif)
{
struct sock *sk, *reuse_sk;
bool no_reuseport;
@@ -315,8 +315,8 @@ static inline struct sock *inet_lookup_run_bpf(struct net *net,
if (hashinfo != &tcp_hashinfo)
return NULL; /* only TCP is supported */
- no_reuseport = bpf_sk_lookup_run_v4(net, IPPROTO_TCP,
- saddr, sport, daddr, hnum, &sk);
+ no_reuseport = bpf_sk_lookup_run_v4(net, IPPROTO_TCP, saddr, sport,
+ daddr, hnum, dif, &sk);
if (no_reuseport || IS_ERR_OR_NULL(sk))
return sk;
@@ -340,7 +340,7 @@ struct sock *__inet_lookup_listener(struct net *net,
/* Lookup redirect from BPF */
if (static_branch_unlikely(&bpf_sk_lookup_enabled)) {
result = inet_lookup_run_bpf(net, hashinfo, skb, doff,
- saddr, sport, daddr, hnum);
+ saddr, sport, daddr, hnum, dif);
if (result)
goto done;
}
diff --git a/net/ipv4/ip_output.c b/net/ipv4/ip_output.c
index 9bca57e..57c1d84 100644
--- a/net/ipv4/ip_output.c
+++ b/net/ipv4/ip_output.c
@@ -672,7 +672,6 @@ struct sk_buff *ip_frag_next(struct sk_buff *skb, struct ip_frag_state *state)
struct sk_buff *skb2;
struct iphdr *iph;
- len = state->left;
/* IF: it doesn't fit, use 'mtu' - the data space left */
if (len > state->mtu)
len = state->mtu;
diff --git a/net/ipv4/ip_sockglue.c b/net/ipv4/ip_sockglue.c
index 38d29b1..445a9ec 100644
--- a/net/ipv4/ip_sockglue.c
+++ b/net/ipv4/ip_sockglue.c
@@ -576,7 +576,7 @@ int ip_recv_error(struct sock *sk, struct msghdr *msg, int len, int *addr_len)
return err;
}
-static void __ip_sock_set_tos(struct sock *sk, int val)
+void __ip_sock_set_tos(struct sock *sk, int val)
{
if (sk->sk_type == SOCK_STREAM) {
val &= ~INET_ECN_MASK;
diff --git a/net/ipv4/ipmr.c b/net/ipv4/ipmr.c
index 2dda856..0727461 100644
--- a/net/ipv4/ipmr.c
+++ b/net/ipv4/ipmr.c
@@ -195,10 +195,6 @@ static int ipmr_rule_match(struct fib_rule *rule, struct flowi *fl, int flags)
return 1;
}
-static const struct nla_policy ipmr_rule_policy[FRA_MAX + 1] = {
- FRA_GENERIC_POLICY,
-};
-
static int ipmr_rule_configure(struct fib_rule *rule, struct sk_buff *skb,
struct fib_rule_hdr *frh, struct nlattr **tb,
struct netlink_ext_ack *extack)
@@ -231,7 +227,6 @@ static const struct fib_rules_ops __net_initconst ipmr_rules_ops_template = {
.compare = ipmr_rule_compare,
.fill = ipmr_rule_fill,
.nlgroup = RTNLGRP_IPV4_RULE,
- .policy = ipmr_rule_policy,
.owner = THIS_MODULE,
};
@@ -696,7 +691,7 @@ static int vif_delete(struct mr_table *mrt, int vifi, int notify,
if (v->flags & (VIFF_TUNNEL | VIFF_REGISTER) && !notify)
unregister_netdevice_queue(dev, head);
- dev_put(dev);
+ dev_put_track(dev, &v->dev_tracker);
return 0;
}
@@ -896,6 +891,7 @@ static int vif_add(struct net *net, struct mr_table *mrt,
/* And finish update writing critical data */
write_lock_bh(&mrt_lock);
v->dev = dev;
+ netdev_tracker_alloc(dev, &v->dev_tracker, GFP_ATOMIC);
if (v->flags & VIFF_REGISTER)
mrt->mroute_reg_vif_num = vifi;
if (vifi+1 > mrt->maxvif)
diff --git a/net/ipv4/netfilter/Kconfig b/net/ipv4/netfilter/Kconfig
index 63cb953..67087f9 100644
--- a/net/ipv4/netfilter/Kconfig
+++ b/net/ipv4/netfilter/Kconfig
@@ -59,12 +59,8 @@
endif # NF_TABLES
config NF_FLOW_TABLE_IPV4
- tristate "Netfilter flow table IPv4 module"
- depends on NF_FLOW_TABLE
- help
- This option adds the flow table IPv4 support.
-
- To compile it as a module, choose M here.
+ tristate
+ select NF_FLOW_TABLE_INET
config NF_DUP_IPV4
tristate "Netfilter IPv4 packet duplication to alternate destination"
diff --git a/net/ipv4/netfilter/Makefile b/net/ipv4/netfilter/Makefile
index f38fb13..93bad11 100644
--- a/net/ipv4/netfilter/Makefile
+++ b/net/ipv4/netfilter/Makefile
@@ -24,9 +24,6 @@
obj-$(CONFIG_NFT_FIB_IPV4) += nft_fib_ipv4.o
obj-$(CONFIG_NFT_DUP_IPV4) += nft_dup_ipv4.o
-# flow table support
-obj-$(CONFIG_NF_FLOW_TABLE_IPV4) += nf_flow_table_ipv4.o
-
# generic IP tables
obj-$(CONFIG_IP_NF_IPTABLES) += ip_tables.o
diff --git a/net/ipv4/netfilter/ipt_CLUSTERIP.c b/net/ipv4/netfilter/ipt_CLUSTERIP.c
index 8fd1aba..b518f20 100644
--- a/net/ipv4/netfilter/ipt_CLUSTERIP.c
+++ b/net/ipv4/netfilter/ipt_CLUSTERIP.c
@@ -520,8 +520,11 @@ static int clusterip_tg_check(const struct xt_tgchk_param *par)
if (IS_ERR(config))
return PTR_ERR(config);
}
- } else if (memcmp(&config->clustermac, &cipinfo->clustermac, ETH_ALEN))
+ } else if (memcmp(&config->clustermac, &cipinfo->clustermac, ETH_ALEN)) {
+ clusterip_config_entry_put(config);
+ clusterip_config_put(config);
return -EINVAL;
+ }
ret = nf_ct_netns_get(par->net, par->family);
if (ret < 0) {
diff --git a/net/ipv4/netfilter/nf_flow_table_ipv4.c b/net/ipv4/netfilter/nf_flow_table_ipv4.c
index aba65fe..e69de29 100644
--- a/net/ipv4/netfilter/nf_flow_table_ipv4.c
+++ b/net/ipv4/netfilter/nf_flow_table_ipv4.c
@@ -1,37 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-only
-#include <linux/kernel.h>
-#include <linux/init.h>
-#include <linux/module.h>
-#include <linux/netfilter.h>
-#include <net/netfilter/nf_flow_table.h>
-#include <net/netfilter/nf_tables.h>
-
-static struct nf_flowtable_type flowtable_ipv4 = {
- .family = NFPROTO_IPV4,
- .init = nf_flow_table_init,
- .setup = nf_flow_table_offload_setup,
- .action = nf_flow_rule_route_ipv4,
- .free = nf_flow_table_free,
- .hook = nf_flow_offload_ip_hook,
- .owner = THIS_MODULE,
-};
-
-static int __init nf_flow_ipv4_module_init(void)
-{
- nft_register_flowtable_type(&flowtable_ipv4);
-
- return 0;
-}
-
-static void __exit nf_flow_ipv4_module_exit(void)
-{
- nft_unregister_flowtable_type(&flowtable_ipv4);
-}
-
-module_init(nf_flow_ipv4_module_init);
-module_exit(nf_flow_ipv4_module_exit);
-
-MODULE_LICENSE("GPL");
-MODULE_AUTHOR("Pablo Neira Ayuso <pablo@netfilter.org>");
-MODULE_ALIAS_NF_FLOWTABLE(AF_INET);
-MODULE_DESCRIPTION("Netfilter flow table support");
diff --git a/net/ipv4/nexthop.c b/net/ipv4/nexthop.c
index 5dbd4b5..eeafecc 100644
--- a/net/ipv4/nexthop.c
+++ b/net/ipv4/nexthop.c
@@ -8,6 +8,7 @@
#include <linux/nexthop.h>
#include <linux/rtnetlink.h>
#include <linux/slab.h>
+#include <linux/vmalloc.h>
#include <net/arp.h>
#include <net/ipv6_stubs.h>
#include <net/lwtunnel.h>
@@ -1918,9 +1919,6 @@ static void nh_rt_cache_flush(struct net *net, struct nexthop *nh,
if (!replaced_nh->is_group)
return;
- /* new dsts must use only the new nexthop group */
- synchronize_net();
-
nhg = rtnl_dereference(replaced_nh->nh_grp);
for (i = 0; i < nhg->num_nh; i++) {
struct nh_grp_entry *nhge = &nhg->nh_entries[i];
@@ -2002,9 +2000,10 @@ static int replace_nexthop_grp(struct net *net, struct nexthop *old,
rcu_assign_pointer(old->nh_grp, newg);
+ /* Make sure concurrent readers are not using 'oldg' anymore. */
+ synchronize_net();
+
if (newg->resilient) {
- /* Make sure concurrent readers are not using 'oldg' anymore. */
- synchronize_net();
rcu_assign_pointer(oldg->res_table, tmp_table);
rcu_assign_pointer(oldg->spare->res_table, tmp_table);
}
diff --git a/net/ipv4/ping.c b/net/ipv4/ping.c
index 1e44a43..0e56df3 100644
--- a/net/ipv4/ping.c
+++ b/net/ipv4/ping.c
@@ -311,15 +311,11 @@ static int ping_check_bind_addr(struct sock *sk, struct inet_sock *isk,
pr_debug("ping_check_bind_addr(sk=%p,addr=%pI4,port=%d)\n",
sk, &addr->sin_addr.s_addr, ntohs(addr->sin_port));
- if (addr->sin_addr.s_addr == htonl(INADDR_ANY))
- chk_addr_ret = RTN_LOCAL;
- else
- chk_addr_ret = inet_addr_type(net, addr->sin_addr.s_addr);
+ chk_addr_ret = inet_addr_type(net, addr->sin_addr.s_addr);
- if ((!inet_can_nonlocal_bind(net, isk) &&
- chk_addr_ret != RTN_LOCAL) ||
- chk_addr_ret == RTN_MULTICAST ||
- chk_addr_ret == RTN_BROADCAST)
+ if (!inet_addr_valid_or_nonlocal(net, inet_sk(sk),
+ addr->sin_addr.s_addr,
+ chk_addr_ret))
return -EADDRNOTAVAIL;
#if IS_ENABLED(CONFIG_IPV6)
@@ -998,6 +994,7 @@ struct proto ping_prot = {
.hash = ping_hash,
.unhash = ping_unhash,
.get_port = ping_get_port,
+ .put_port = ping_unhash,
.obj_size = sizeof(struct inet_sock),
};
EXPORT_SYMBOL(ping_prot);
diff --git a/net/ipv4/raw.c b/net/ipv4/raw.c
index bb446e6..a53f256 100644
--- a/net/ipv4/raw.c
+++ b/net/ipv4/raw.c
@@ -99,8 +99,8 @@ int raw_hash_sk(struct sock *sk)
write_lock_bh(&h->lock);
sk_add_node(sk, head);
- sock_prot_inuse_add(sock_net(sk), sk->sk_prot, 1);
write_unlock_bh(&h->lock);
+ sock_prot_inuse_add(sock_net(sk), sk->sk_prot, 1);
return 0;
}
@@ -717,6 +717,7 @@ static int raw_bind(struct sock *sk, struct sockaddr *uaddr, int addr_len)
{
struct inet_sock *inet = inet_sk(sk);
struct sockaddr_in *addr = (struct sockaddr_in *) uaddr;
+ struct net *net = sock_net(sk);
u32 tb_id = RT_TABLE_LOCAL;
int ret = -EINVAL;
int chk_addr_ret;
@@ -725,16 +726,16 @@ static int raw_bind(struct sock *sk, struct sockaddr *uaddr, int addr_len)
goto out;
if (sk->sk_bound_dev_if)
- tb_id = l3mdev_fib_table_by_index(sock_net(sk),
- sk->sk_bound_dev_if) ? : tb_id;
+ tb_id = l3mdev_fib_table_by_index(net,
+ sk->sk_bound_dev_if) ? : tb_id;
- chk_addr_ret = inet_addr_type_table(sock_net(sk), addr->sin_addr.s_addr,
- tb_id);
+ chk_addr_ret = inet_addr_type_table(net, addr->sin_addr.s_addr, tb_id);
ret = -EADDRNOTAVAIL;
- if (addr->sin_addr.s_addr && chk_addr_ret != RTN_LOCAL &&
- chk_addr_ret != RTN_MULTICAST && chk_addr_ret != RTN_BROADCAST)
+ if (!inet_addr_valid_or_nonlocal(net, inet, addr->sin_addr.s_addr,
+ chk_addr_ret))
goto out;
+
inet->inet_rcv_saddr = inet->inet_saddr = addr->sin_addr.s_addr;
if (chk_addr_ret == RTN_MULTICAST || chk_addr_ret == RTN_BROADCAST)
inet->inet_saddr = 0; /* Use device */
diff --git a/net/ipv4/route.c b/net/ipv4/route.c
index 0b4103b..ff6f91c 100644
--- a/net/ipv4/route.c
+++ b/net/ipv4/route.c
@@ -110,14 +110,15 @@
#define RT_GC_TIMEOUT (300*HZ)
+#define DEFAULT_MIN_PMTU (512 + 20 + 20)
+#define DEFAULT_MTU_EXPIRES (10 * 60 * HZ)
+
static int ip_rt_max_size;
static int ip_rt_redirect_number __read_mostly = 9;
static int ip_rt_redirect_load __read_mostly = HZ / 50;
static int ip_rt_redirect_silence __read_mostly = ((HZ / 50) << (9 + 1));
static int ip_rt_error_cost __read_mostly = HZ;
static int ip_rt_error_burst __read_mostly = 5 * HZ;
-static int ip_rt_mtu_expires __read_mostly = 10 * 60 * HZ;
-static u32 ip_rt_min_pmtu __read_mostly = 512 + 20 + 20;
static int ip_rt_min_advmss __read_mostly = 256;
static int ip_rt_gc_timeout __read_mostly = RT_GC_TIMEOUT;
@@ -602,7 +603,7 @@ static void fnhe_remove_oldest(struct fnhe_hash_bucket *hash)
static u32 fnhe_hashfun(__be32 daddr)
{
- static siphash_key_t fnhe_hash_key __read_mostly;
+ static siphash_aligned_key_t fnhe_hash_key;
u64 hval;
net_get_random_once(&fnhe_hash_key, sizeof(fnhe_hash_key));
@@ -1018,13 +1019,13 @@ static void __ip_rt_update_pmtu(struct rtable *rt, struct flowi4 *fl4, u32 mtu)
if (old_mtu < mtu)
return;
- if (mtu < ip_rt_min_pmtu) {
+ if (mtu < net->ipv4.ip_rt_min_pmtu) {
lock = true;
- mtu = min(old_mtu, ip_rt_min_pmtu);
+ mtu = min(old_mtu, net->ipv4.ip_rt_min_pmtu);
}
if (rt->rt_pmtu == mtu && !lock &&
- time_before(jiffies, dst->expires - ip_rt_mtu_expires / 2))
+ time_before(jiffies, dst->expires - net->ipv4.ip_rt_mtu_expires / 2))
return;
rcu_read_lock();
@@ -1034,7 +1035,7 @@ static void __ip_rt_update_pmtu(struct rtable *rt, struct flowi4 *fl4, u32 mtu)
fib_select_path(net, &res, fl4, NULL);
nhc = FIB_RES_NHC(res);
update_or_create_fnhe(nhc, fl4->daddr, 0, mtu, lock,
- jiffies + ip_rt_mtu_expires);
+ jiffies + net->ipv4.ip_rt_mtu_expires);
}
rcu_read_unlock();
}
@@ -1531,8 +1532,9 @@ void rt_flush_dev(struct net_device *dev)
if (rt->dst.dev != dev)
continue;
rt->dst.dev = blackhole_netdev;
- dev_hold(rt->dst.dev);
- dev_put(dev);
+ dev_replace_track(dev, blackhole_netdev,
+ &rt->dst.dev_tracker,
+ GFP_ATOMIC);
}
spin_unlock_bh(&ul->lock);
}
@@ -2819,7 +2821,7 @@ struct dst_entry *ipv4_blackhole_route(struct net *net, struct dst_entry *dst_or
new->output = dst_discard_out;
new->dev = net->loopback_dev;
- dev_hold(new->dev);
+ dev_hold_track(new->dev, &new->dev_tracker, GFP_ATOMIC);
rt->rt_is_input = ort->rt_is_input;
rt->rt_iif = ort->rt_iif;
@@ -3534,21 +3536,6 @@ static struct ctl_table ipv4_route_table[] = {
.proc_handler = proc_dointvec,
},
{
- .procname = "mtu_expires",
- .data = &ip_rt_mtu_expires,
- .maxlen = sizeof(int),
- .mode = 0644,
- .proc_handler = proc_dointvec_jiffies,
- },
- {
- .procname = "min_pmtu",
- .data = &ip_rt_min_pmtu,
- .maxlen = sizeof(int),
- .mode = 0644,
- .proc_handler = proc_dointvec_minmax,
- .extra1 = &ip_min_valid_pmtu,
- },
- {
.procname = "min_adv_mss",
.data = &ip_rt_min_advmss,
.maxlen = sizeof(int),
@@ -3560,13 +3547,28 @@ static struct ctl_table ipv4_route_table[] = {
static const char ipv4_route_flush_procname[] = "flush";
-static struct ctl_table ipv4_route_flush_table[] = {
+static struct ctl_table ipv4_route_netns_table[] = {
{
.procname = ipv4_route_flush_procname,
.maxlen = sizeof(int),
.mode = 0200,
.proc_handler = ipv4_sysctl_rtcache_flush,
},
+ {
+ .procname = "min_pmtu",
+ .data = &init_net.ipv4.ip_rt_min_pmtu,
+ .maxlen = sizeof(int),
+ .mode = 0644,
+ .proc_handler = proc_dointvec_minmax,
+ .extra1 = &ip_min_valid_pmtu,
+ },
+ {
+ .procname = "mtu_expires",
+ .data = &init_net.ipv4.ip_rt_mtu_expires,
+ .maxlen = sizeof(int),
+ .mode = 0644,
+ .proc_handler = proc_dointvec_jiffies,
+ },
{ },
};
@@ -3574,9 +3576,11 @@ static __net_init int sysctl_route_net_init(struct net *net)
{
struct ctl_table *tbl;
- tbl = ipv4_route_flush_table;
+ tbl = ipv4_route_netns_table;
if (!net_eq(net, &init_net)) {
- tbl = kmemdup(tbl, sizeof(ipv4_route_flush_table), GFP_KERNEL);
+ int i;
+
+ tbl = kmemdup(tbl, sizeof(ipv4_route_netns_table), GFP_KERNEL);
if (!tbl)
goto err_dup;
@@ -3585,6 +3589,12 @@ static __net_init int sysctl_route_net_init(struct net *net)
if (tbl[0].procname != ipv4_route_flush_procname)
tbl[0].procname = NULL;
}
+
+ /* Update the variables to point into the current struct net
+ * except for the first element flush
+ */
+ for (i = 1; i < ARRAY_SIZE(ipv4_route_netns_table) - 1; i++)
+ tbl[i].data += (void *)net - (void *)&init_net;
}
tbl[0].extra1 = net;
@@ -3594,7 +3604,7 @@ static __net_init int sysctl_route_net_init(struct net *net)
return 0;
err_reg:
- if (tbl != ipv4_route_flush_table)
+ if (tbl != ipv4_route_netns_table)
kfree(tbl);
err_dup:
return -ENOMEM;
@@ -3606,7 +3616,7 @@ static __net_exit void sysctl_route_net_exit(struct net *net)
tbl = net->ipv4.route_hdr->ctl_table_arg;
unregister_net_sysctl_table(net->ipv4.route_hdr);
- BUG_ON(tbl == ipv4_route_flush_table);
+ BUG_ON(tbl == ipv4_route_netns_table);
kfree(tbl);
}
@@ -3616,6 +3626,18 @@ static __net_initdata struct pernet_operations sysctl_route_ops = {
};
#endif
+static __net_init int netns_ip_rt_init(struct net *net)
+{
+ /* Set default value for namespaceified sysctls */
+ net->ipv4.ip_rt_min_pmtu = DEFAULT_MIN_PMTU;
+ net->ipv4.ip_rt_mtu_expires = DEFAULT_MTU_EXPIRES;
+ return 0;
+}
+
+static struct pernet_operations __net_initdata ip_rt_ops = {
+ .init = netns_ip_rt_init,
+};
+
static __net_init int rt_genid_init(struct net *net)
{
atomic_set(&net->ipv4.rt_genid, 0);
@@ -3721,6 +3743,7 @@ int __init ip_rt_init(void)
#ifdef CONFIG_SYSCTL
register_pernet_subsys(&sysctl_route_ops);
#endif
+ register_pernet_subsys(&ip_rt_ops);
register_pernet_subsys(&rt_genid_ops);
register_pernet_subsys(&ipv4_inetpeer_ops);
return 0;
diff --git a/net/ipv4/syncookies.c b/net/ipv4/syncookies.c
index 8696dc34..2cb3b85 100644
--- a/net/ipv4/syncookies.c
+++ b/net/ipv4/syncookies.c
@@ -14,7 +14,7 @@
#include <net/tcp.h>
#include <net/route.h>
-static siphash_key_t syncookie_secret[2] __read_mostly;
+static siphash_aligned_key_t syncookie_secret[2];
#define COOKIEBITS 24 /* Upper bits store count */
#define COOKIEMASK (((__u32)1 << COOKIEBITS) - 1)
diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index 2bb28bf..3b75836 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -292,7 +292,7 @@ EXPORT_PER_CPU_SYMBOL_GPL(tcp_orphan_count);
long sysctl_tcp_mem[3] __read_mostly;
EXPORT_SYMBOL(sysctl_tcp_mem);
-atomic_long_t tcp_memory_allocated; /* Current allocated memory. */
+atomic_long_t tcp_memory_allocated ____cacheline_aligned_in_smp; /* Current allocated memory. */
EXPORT_SYMBOL(tcp_memory_allocated);
#if IS_ENABLED(CONFIG_SMC)
@@ -303,7 +303,7 @@ EXPORT_SYMBOL(tcp_have_smc);
/*
* Current number of TCP sockets.
*/
-struct percpu_counter tcp_sockets_allocated;
+struct percpu_counter tcp_sockets_allocated ____cacheline_aligned_in_smp;
EXPORT_SYMBOL(tcp_sockets_allocated);
/*
@@ -456,7 +456,6 @@ void tcp_init_sock(struct sock *sk)
WRITE_ONCE(sk->sk_rcvbuf, sock_net(sk)->ipv4.sysctl_tcp_rmem[1]);
sk_sockets_allocated_inc(sk);
- sk->sk_route_forced_caps = NETIF_F_GSO;
}
EXPORT_SYMBOL(tcp_init_sock);
@@ -546,10 +545,11 @@ __poll_t tcp_poll(struct file *file, struct socket *sock, poll_table *wait)
if (state != TCP_SYN_SENT &&
(state != TCP_SYN_RECV || rcu_access_pointer(tp->fastopen_rsk))) {
int target = sock_rcvlowat(sk, 0, INT_MAX);
+ u16 urg_data = READ_ONCE(tp->urg_data);
- if (READ_ONCE(tp->urg_seq) == READ_ONCE(tp->copied_seq) &&
- !sock_flag(sk, SOCK_URGINLINE) &&
- tp->urg_data)
+ if (unlikely(urg_data) &&
+ READ_ONCE(tp->urg_seq) == READ_ONCE(tp->copied_seq) &&
+ !sock_flag(sk, SOCK_URGINLINE))
target++;
if (tcp_stream_is_readable(sk, target))
@@ -574,7 +574,7 @@ __poll_t tcp_poll(struct file *file, struct socket *sock, poll_table *wait)
} else
mask |= EPOLLOUT | EPOLLWRNORM;
- if (tp->urg_data & TCP_URG_VALID)
+ if (urg_data & TCP_URG_VALID)
mask |= EPOLLPRI;
} else if (state == TCP_SYN_SENT && inet_sk(sk)->defer_connect) {
/* Active TCP fastopen socket with defer_connect
@@ -608,7 +608,7 @@ int tcp_ioctl(struct sock *sk, int cmd, unsigned long arg)
unlock_sock_fast(sk, slow);
break;
case SIOCATMARK:
- answ = tp->urg_data &&
+ answ = READ_ONCE(tp->urg_data) &&
READ_ONCE(tp->urg_seq) == READ_ONCE(tp->copied_seq);
break;
case SIOCOUTQ:
@@ -1466,7 +1466,7 @@ static int tcp_recv_urg(struct sock *sk, struct msghdr *msg, int len, int flags)
char c = tp->urg_data;
if (!(flags & MSG_PEEK))
- tp->urg_data = TCP_URG_READ;
+ WRITE_ONCE(tp->urg_data, TCP_URG_READ);
/* Read urgent data. */
msg->msg_flags |= MSG_OOB;
@@ -1580,6 +1580,36 @@ void tcp_cleanup_rbuf(struct sock *sk, int copied)
tcp_send_ack(sk);
}
+void __sk_defer_free_flush(struct sock *sk)
+{
+ struct llist_node *head;
+ struct sk_buff *skb, *n;
+
+ head = llist_del_all(&sk->defer_list);
+ llist_for_each_entry_safe(skb, n, head, ll_node) {
+ prefetch(n);
+ skb_mark_not_on_list(skb);
+ __kfree_skb(skb);
+ }
+}
+EXPORT_SYMBOL(__sk_defer_free_flush);
+
+static void tcp_eat_recv_skb(struct sock *sk, struct sk_buff *skb)
+{
+ __skb_unlink(skb, &sk->sk_receive_queue);
+ if (likely(skb->destructor == sock_rfree)) {
+ sock_rfree(skb);
+ skb->destructor = NULL;
+ skb->sk = NULL;
+ if (!skb_queue_empty(&sk->sk_receive_queue) ||
+ !llist_empty(&sk->defer_list)) {
+ llist_add(&skb->ll_node, &sk->defer_list);
+ return;
+ }
+ }
+ __kfree_skb(skb);
+}
+
static struct sk_buff *tcp_recv_skb(struct sock *sk, u32 seq, u32 *off)
{
struct sk_buff *skb;
@@ -1599,7 +1629,7 @@ static struct sk_buff *tcp_recv_skb(struct sock *sk, u32 seq, u32 *off)
* splitted a fat GRO packet, while we released socket lock
* in skb_splice_bits()
*/
- sk_eat_skb(sk, skb);
+ tcp_eat_recv_skb(sk, skb);
}
return NULL;
}
@@ -1633,7 +1663,7 @@ int tcp_read_sock(struct sock *sk, read_descriptor_t *desc,
len = skb->len - offset;
/* Stop reading if we hit a patch of urgent data */
- if (tp->urg_data) {
+ if (unlikely(tp->urg_data)) {
u32 urg_offset = tp->urg_seq - seq;
if (urg_offset < len)
len = urg_offset;
@@ -1665,11 +1695,11 @@ int tcp_read_sock(struct sock *sk, read_descriptor_t *desc,
continue;
}
if (TCP_SKB_CB(skb)->tcp_flags & TCPHDR_FIN) {
- sk_eat_skb(sk, skb);
+ tcp_eat_recv_skb(sk, skb);
++seq;
break;
}
- sk_eat_skb(sk, skb);
+ tcp_eat_recv_skb(sk, skb);
if (!desc->count)
break;
WRITE_ONCE(tp->copied_seq, seq);
@@ -2329,7 +2359,7 @@ static int tcp_recvmsg_locked(struct sock *sk, struct msghdr *msg, size_t len,
u32 offset;
/* Are we at urgent data? Stop if we have read anything or have SIGURG pending. */
- if (tp->urg_data && tp->urg_seq == *seq) {
+ if (unlikely(tp->urg_data) && tp->urg_seq == *seq) {
if (copied)
break;
if (signal_pending(current)) {
@@ -2372,10 +2402,10 @@ static int tcp_recvmsg_locked(struct sock *sk, struct msghdr *msg, size_t len,
break;
if (copied) {
- if (sk->sk_err ||
+ if (!timeo ||
+ sk->sk_err ||
sk->sk_state == TCP_CLOSE ||
(sk->sk_shutdown & RCV_SHUTDOWN) ||
- !timeo ||
signal_pending(current))
break;
} else {
@@ -2409,13 +2439,12 @@ static int tcp_recvmsg_locked(struct sock *sk, struct msghdr *msg, size_t len,
}
}
- tcp_cleanup_rbuf(sk, copied);
-
if (copied >= target) {
/* Do not sleep, just process backlog. */
- release_sock(sk);
- lock_sock(sk);
+ __sk_flush_backlog(sk);
} else {
+ tcp_cleanup_rbuf(sk, copied);
+ sk_defer_free_flush(sk);
sk_wait_data(sk, &timeo, last);
}
@@ -2435,7 +2464,7 @@ static int tcp_recvmsg_locked(struct sock *sk, struct msghdr *msg, size_t len,
used = len;
/* Do we have urgent data here? */
- if (tp->urg_data) {
+ if (unlikely(tp->urg_data)) {
u32 urg_offset = tp->urg_seq - *seq;
if (urg_offset < used) {
if (!urg_offset) {
@@ -2469,8 +2498,8 @@ static int tcp_recvmsg_locked(struct sock *sk, struct msghdr *msg, size_t len,
tcp_rcv_space_adjust(sk);
skip_copy:
- if (tp->urg_data && after(tp->copied_seq, tp->urg_seq)) {
- tp->urg_data = 0;
+ if (unlikely(tp->urg_data) && after(tp->copied_seq, tp->urg_seq)) {
+ WRITE_ONCE(tp->urg_data, 0);
tcp_fast_path_check(sk);
}
@@ -2485,14 +2514,14 @@ static int tcp_recvmsg_locked(struct sock *sk, struct msghdr *msg, size_t len,
if (TCP_SKB_CB(skb)->tcp_flags & TCPHDR_FIN)
goto found_fin_ok;
if (!(flags & MSG_PEEK))
- sk_eat_skb(sk, skb);
+ tcp_eat_recv_skb(sk, skb);
continue;
found_fin_ok:
/* Process the FIN. */
WRITE_ONCE(*seq, *seq + 1);
if (!(flags & MSG_PEEK))
- sk_eat_skb(sk, skb);
+ tcp_eat_recv_skb(sk, skb);
break;
} while (len > 0);
@@ -2534,6 +2563,7 @@ int tcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len, int nonblock,
ret = tcp_recvmsg_locked(sk, msg, len, nonblock, flags, &tss,
&cmsg_flags);
release_sock(sk);
+ sk_defer_free_flush(sk);
if (cmsg_flags && ret >= 0) {
if (cmsg_flags & TCP_CMSG_TS)
@@ -2964,7 +2994,7 @@ int tcp_disconnect(struct sock *sk, int flags)
tcp_clear_xmit_timers(sk);
__skb_queue_purge(&sk->sk_receive_queue);
WRITE_ONCE(tp->copied_seq, tp->rcv_nxt);
- tp->urg_data = 0;
+ WRITE_ONCE(tp->urg_data, 0);
tcp_write_queue_purge(sk);
tcp_fastopen_active_disable_ofo_check(sk);
skb_rbtree_purge(&tp->out_of_order_queue);
@@ -3058,7 +3088,7 @@ int tcp_disconnect(struct sock *sk, int flags)
sk->sk_frag.page = NULL;
sk->sk_frag.offset = 0;
}
-
+ sk_defer_free_flush(sk);
sk_error_report(sk);
return 0;
}
@@ -3176,7 +3206,7 @@ static void tcp_enable_tx_delay(void)
* TCP_CORK can be set together with TCP_NODELAY and it is stronger than
* TCP_NODELAY.
*/
-static void __tcp_sock_set_cork(struct sock *sk, bool on)
+void __tcp_sock_set_cork(struct sock *sk, bool on)
{
struct tcp_sock *tp = tcp_sk(sk);
@@ -3204,7 +3234,7 @@ EXPORT_SYMBOL(tcp_sock_set_cork);
* However, when TCP_NODELAY is set we make an explicit push, which overrides
* even TCP_CORK for currently queued segments.
*/
-static void __tcp_sock_set_nodelay(struct sock *sk, bool on)
+void __tcp_sock_set_nodelay(struct sock *sk, bool on)
{
if (on) {
tcp_sk(sk)->nonagle |= TCP_NAGLE_OFF|TCP_NAGLE_PUSH;
@@ -3773,10 +3803,12 @@ void tcp_get_info(struct sock *sk, struct tcp_info *info)
tcp_get_info_chrono_stats(tp, info);
info->tcpi_segs_out = tp->segs_out;
- info->tcpi_segs_in = tp->segs_in;
+
+ /* segs_in and data_segs_in can be updated from tcp_segs_in() from BH */
+ info->tcpi_segs_in = READ_ONCE(tp->segs_in);
+ info->tcpi_data_segs_in = READ_ONCE(tp->data_segs_in);
info->tcpi_min_rtt = tcp_min_rtt(tp);
- info->tcpi_data_segs_in = tp->data_segs_in;
info->tcpi_data_segs_out = tp->data_segs_out;
info->tcpi_delivery_rate_app_limited = tp->rate_app_limited ? 1 : 0;
@@ -4185,6 +4217,7 @@ static int do_tcp_getsockopt(struct sock *sk, int level,
err = BPF_CGROUP_RUN_PROG_GETSOCKOPT_KERN(sk, level, optname,
&zc, &len, err);
release_sock(sk);
+ sk_defer_free_flush(sk);
if (len >= offsetofend(struct tcp_zerocopy_receive, msg_flags))
goto zerocopy_rcv_cmsg;
switch (len) {
diff --git a/net/ipv4/tcp_bpf.c b/net/ipv4/tcp_bpf.c
index f70aa09..9b9b020 100644
--- a/net/ipv4/tcp_bpf.c
+++ b/net/ipv4/tcp_bpf.c
@@ -196,12 +196,39 @@ static int tcp_bpf_recvmsg_parser(struct sock *sk,
long timeo;
int data;
+ if (sock_flag(sk, SOCK_DONE))
+ goto out;
+
+ if (sk->sk_err) {
+ copied = sock_error(sk);
+ goto out;
+ }
+
+ if (sk->sk_shutdown & RCV_SHUTDOWN)
+ goto out;
+
+ if (sk->sk_state == TCP_CLOSE) {
+ copied = -ENOTCONN;
+ goto out;
+ }
+
timeo = sock_rcvtimeo(sk, nonblock);
+ if (!timeo) {
+ copied = -EAGAIN;
+ goto out;
+ }
+
+ if (signal_pending(current)) {
+ copied = sock_intr_errno(timeo);
+ goto out;
+ }
+
data = tcp_msg_wait_data(sk, psock, timeo);
if (data && !sk_psock_queue_empty(psock))
goto msg_bytes_ready;
copied = -EAGAIN;
}
+out:
release_sock(sk);
sk_psock_put(sk, psock);
return copied;
diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
index 0ce4684..dc49a3d 100644
--- a/net/ipv4/tcp_input.c
+++ b/net/ipv4/tcp_input.c
@@ -3601,7 +3601,7 @@ bool tcp_oow_rate_limited(struct net *net, const struct sk_buff *skb,
}
/* RFC 5961 7 [ACK Throttling] */
-static void tcp_send_challenge_ack(struct sock *sk, const struct sk_buff *skb)
+static void tcp_send_challenge_ack(struct sock *sk)
{
/* unprotected vars, we dont care of overwrites */
static u32 challenge_timestamp;
@@ -3763,7 +3763,7 @@ static int tcp_ack(struct sock *sk, const struct sk_buff *skb, int flag)
/* RFC 5961 5.2 [Blind Data Injection Attack].[Mitigation] */
if (before(ack, prior_snd_una - tp->max_window)) {
if (!(flag & FLAG_NO_CHALLENGE_ACK))
- tcp_send_challenge_ack(sk, skb);
+ tcp_send_challenge_ack(sk);
return -1;
}
goto old_ack;
@@ -5591,7 +5591,7 @@ static void tcp_check_urg(struct sock *sk, const struct tcphdr *th)
}
}
- tp->urg_data = TCP_URG_NOTYET;
+ WRITE_ONCE(tp->urg_data, TCP_URG_NOTYET);
WRITE_ONCE(tp->urg_seq, ptr);
/* Disable header prediction. */
@@ -5604,11 +5604,11 @@ static void tcp_urg(struct sock *sk, struct sk_buff *skb, const struct tcphdr *t
struct tcp_sock *tp = tcp_sk(sk);
/* Check if we get a new urgent pointer - normally not. */
- if (th->urg)
+ if (unlikely(th->urg))
tcp_check_urg(sk, th);
/* Do we wait for any urgent data? - normally not... */
- if (tp->urg_data == TCP_URG_NOTYET) {
+ if (unlikely(tp->urg_data == TCP_URG_NOTYET)) {
u32 ptr = tp->urg_seq - ntohl(th->seq) + (th->doff * 4) -
th->syn;
@@ -5617,7 +5617,7 @@ static void tcp_urg(struct sock *sk, struct sk_buff *skb, const struct tcphdr *t
u8 tmp;
if (skb_copy_bits(skb, ptr, &tmp, 1))
BUG();
- tp->urg_data = TCP_URG_VALID | tmp;
+ WRITE_ONCE(tp->urg_data, TCP_URG_VALID | tmp);
if (!sock_flag(sk, SOCK_DEAD))
sk->sk_data_ready(sk);
}
@@ -5726,7 +5726,7 @@ static bool tcp_validate_incoming(struct sock *sk, struct sk_buff *skb,
if (tp->syn_fastopen && !tp->data_segs_in &&
sk->sk_state == TCP_ESTABLISHED)
tcp_fastopen_active_disable(sk);
- tcp_send_challenge_ack(sk, skb);
+ tcp_send_challenge_ack(sk);
}
goto discard;
}
@@ -5741,7 +5741,7 @@ static bool tcp_validate_incoming(struct sock *sk, struct sk_buff *skb,
if (syn_inerr)
TCP_INC_STATS(sock_net(sk), TCP_MIB_INERRS);
NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPSYNCHALLENGE);
- tcp_send_challenge_ack(sk, skb);
+ tcp_send_challenge_ack(sk);
goto discard;
}
@@ -6456,7 +6456,7 @@ int tcp_rcv_state_process(struct sock *sk, struct sk_buff *skb)
if (!acceptable) {
if (sk->sk_state == TCP_SYN_RECV)
return 1; /* send one RST */
- tcp_send_challenge_ack(sk, skb);
+ tcp_send_challenge_ack(sk);
goto discard;
}
switch (sk->sk_state) {
diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c
index 084df22..b3f34e3 100644
--- a/net/ipv4/tcp_ipv4.c
+++ b/net/ipv4/tcp_ipv4.c
@@ -1182,7 +1182,7 @@ int tcp_md5_do_add(struct sock *sk, const union tcp_md5_addr *addr,
if (!md5sig)
return -ENOMEM;
- sk_nocaps_add(sk, NETIF_F_GSO_MASK);
+ sk_gso_disable(sk);
INIT_HLIST_HEAD(&md5sig->head);
rcu_assign_pointer(tp->md5sig_info, md5sig);
}
@@ -1620,7 +1620,7 @@ struct sock *tcp_v4_syn_recv_sock(const struct sock *sk, struct sk_buff *skb,
*/
tcp_md5_do_add(newsk, addr, AF_INET, 32, l3index, key->flags,
key->key, key->keylen, GFP_ATOMIC);
- sk_nocaps_add(newsk, NETIF_F_GSO_MASK);
+ sk_gso_disable(newsk);
}
#endif
@@ -1803,8 +1803,7 @@ int tcp_v4_early_demux(struct sk_buff *skb)
bool tcp_add_backlog(struct sock *sk, struct sk_buff *skb)
{
- u32 limit = READ_ONCE(sk->sk_rcvbuf) + READ_ONCE(sk->sk_sndbuf);
- u32 tail_gso_size, tail_gso_segs;
+ u32 limit, tail_gso_size, tail_gso_segs;
struct skb_shared_info *shinfo;
const struct tcphdr *th;
struct tcphdr *thtail;
@@ -1912,7 +1911,7 @@ bool tcp_add_backlog(struct sock *sk, struct sk_buff *skb)
* to reduce memory overhead, so add a little headroom here.
* Few sockets backlog are possibly concurrently non empty.
*/
- limit += 64*1024;
+ limit = READ_ONCE(sk->sk_rcvbuf) + READ_ONCE(sk->sk_sndbuf) + 64*1024;
if (unlikely(sk_add_backlog(sk, skb, limit))) {
bh_unlock_sock(sk);
@@ -1972,8 +1971,10 @@ int tcp_v4_rcv(struct sk_buff *skb)
const struct tcphdr *th;
bool refcounted;
struct sock *sk;
+ int drop_reason;
int ret;
+ drop_reason = SKB_DROP_REASON_NOT_SPECIFIED;
if (skb->pkt_type != PACKET_HOST)
goto discard_it;
@@ -1985,8 +1986,10 @@ int tcp_v4_rcv(struct sk_buff *skb)
th = (const struct tcphdr *)skb->data;
- if (unlikely(th->doff < sizeof(struct tcphdr) / 4))
+ if (unlikely(th->doff < sizeof(struct tcphdr) / 4)) {
+ drop_reason = SKB_DROP_REASON_PKT_TOO_SMALL;
goto bad_packet;
+ }
if (!pskb_may_pull(skb, th->doff * 4))
goto discard_it;
@@ -2091,8 +2094,10 @@ int tcp_v4_rcv(struct sk_buff *skb)
nf_reset_ct(skb);
- if (tcp_filter(sk, skb))
+ if (tcp_filter(sk, skb)) {
+ drop_reason = SKB_DROP_REASON_TCP_FILTER;
goto discard_and_relse;
+ }
th = (const struct tcphdr *)skb->data;
iph = ip_hdr(skb);
tcp_v4_fill_cb(skb, iph, th);
@@ -2106,6 +2111,7 @@ int tcp_v4_rcv(struct sk_buff *skb)
sk_incoming_cpu_update(sk);
+ sk_defer_free_flush(sk);
bh_lock_sock_nested(sk);
tcp_segs_in(tcp_sk(sk), skb);
ret = 0;
@@ -2124,6 +2130,7 @@ int tcp_v4_rcv(struct sk_buff *skb)
return ret;
no_tcp_socket:
+ drop_reason = SKB_DROP_REASON_NO_SOCKET;
if (!xfrm4_policy_check(NULL, XFRM_POLICY_IN, skb))
goto discard_it;
@@ -2131,6 +2138,7 @@ int tcp_v4_rcv(struct sk_buff *skb)
if (tcp_checksum_complete(skb)) {
csum_error:
+ drop_reason = SKB_DROP_REASON_TCP_CSUM;
trace_tcp_bad_csum(skb);
__TCP_INC_STATS(net, TCP_MIB_CSUMERRORS);
bad_packet:
@@ -2141,7 +2149,7 @@ int tcp_v4_rcv(struct sk_buff *skb)
discard_it:
/* Discard frame. */
- kfree_skb(skb);
+ kfree_skb_reason(skb, drop_reason);
return 0;
discard_and_relse:
@@ -3076,6 +3084,7 @@ struct proto tcp_prot = {
.hash = inet_hash,
.unhash = inet_unhash,
.get_port = inet_csk_get_port,
+ .put_port = inet_put_port,
#ifdef CONFIG_BPF_SYSCALL
.psock_update_sk_prot = tcp_bpf_update_proto,
#endif
diff --git a/net/ipv4/tcp_offload.c b/net/ipv4/tcp_offload.c
index fc61cd3..30abde8 100644
--- a/net/ipv4/tcp_offload.c
+++ b/net/ipv4/tcp_offload.c
@@ -8,6 +8,7 @@
#include <linux/indirect_call_wrapper.h>
#include <linux/skbuff.h>
+#include <net/gro.h>
#include <net/tcp.h>
#include <net/protocol.h>
diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c
index 2e6e5a7..5079832 100644
--- a/net/ipv4/tcp_output.c
+++ b/net/ipv4/tcp_output.c
@@ -1359,7 +1359,7 @@ static int __tcp_transmit_skb(struct sock *sk, struct sk_buff *skb,
#ifdef CONFIG_TCP_MD5SIG
/* Calculate the MD5 hash, as we have all we need now */
if (md5) {
- sk_nocaps_add(sk, NETIF_F_GSO_MASK);
+ sk_gso_disable(sk);
tp->af_specific->calc_md5_hash(opts.hash_location,
md5, sk, skb);
}
diff --git a/net/ipv4/udp.c b/net/ipv4/udp.c
index 15c6b45..464590e 100644
--- a/net/ipv4/udp.c
+++ b/net/ipv4/udp.c
@@ -74,6 +74,7 @@
#define pr_fmt(fmt) "UDP: " fmt
+#include <linux/bpf-cgroup.h>
#include <linux/uaccess.h>
#include <asm/ioctls.h>
#include <linux/memblock.h>
@@ -122,7 +123,7 @@ EXPORT_SYMBOL(udp_table);
long sysctl_udp_mem[3] __read_mostly;
EXPORT_SYMBOL(sysctl_udp_mem);
-atomic_long_t udp_memory_allocated;
+atomic_long_t udp_memory_allocated ____cacheline_aligned_in_smp;
EXPORT_SYMBOL(udp_memory_allocated);
#define MAX_UDP_PORTS 65536
@@ -459,7 +460,7 @@ static struct sock *udp4_lookup_run_bpf(struct net *net,
struct udp_table *udptable,
struct sk_buff *skb,
__be32 saddr, __be16 sport,
- __be32 daddr, u16 hnum)
+ __be32 daddr, u16 hnum, const int dif)
{
struct sock *sk, *reuse_sk;
bool no_reuseport;
@@ -467,8 +468,8 @@ static struct sock *udp4_lookup_run_bpf(struct net *net,
if (udptable != &udp_table)
return NULL; /* only UDP is supported */
- no_reuseport = bpf_sk_lookup_run_v4(net, IPPROTO_UDP,
- saddr, sport, daddr, hnum, &sk);
+ no_reuseport = bpf_sk_lookup_run_v4(net, IPPROTO_UDP, saddr, sport,
+ daddr, hnum, dif, &sk);
if (no_reuseport || IS_ERR_OR_NULL(sk))
return sk;
@@ -504,7 +505,7 @@ struct sock *__udp4_lib_lookup(struct net *net, __be32 saddr,
/* Lookup redirect from BPF */
if (static_branch_unlikely(&bpf_sk_lookup_enabled)) {
sk = udp4_lookup_run_bpf(net, udptable, skb,
- saddr, sport, daddr, hnum);
+ saddr, sport, daddr, hnum, dif);
if (sk) {
result = sk;
goto done;
@@ -2410,6 +2411,9 @@ int __udp4_lib_rcv(struct sk_buff *skb, struct udp_table *udptable,
__be32 saddr, daddr;
struct net *net = dev_net(skb->dev);
bool refcounted;
+ int drop_reason;
+
+ drop_reason = SKB_DROP_REASON_NOT_SPECIFIED;
/*
* Validate the packet.
@@ -2465,6 +2469,7 @@ int __udp4_lib_rcv(struct sk_buff *skb, struct udp_table *udptable,
if (udp_lib_checksum_complete(skb))
goto csum_error;
+ drop_reason = SKB_DROP_REASON_NO_SOCKET;
__UDP_INC_STATS(net, UDP_MIB_NOPORTS, proto == IPPROTO_UDPLITE);
icmp_send(skb, ICMP_DEST_UNREACH, ICMP_PORT_UNREACH, 0);
@@ -2472,10 +2477,11 @@ int __udp4_lib_rcv(struct sk_buff *skb, struct udp_table *udptable,
* Hmm. We got an UDP packet to a port to which we
* don't wanna listen. Ignore it.
*/
- kfree_skb(skb);
+ kfree_skb_reason(skb, drop_reason);
return 0;
short_packet:
+ drop_reason = SKB_DROP_REASON_PKT_TOO_SMALL;
net_dbg_ratelimited("UDP%s: short packet: From %pI4:%u %d/%d to %pI4:%u\n",
proto == IPPROTO_UDPLITE ? "Lite" : "",
&saddr, ntohs(uh->source),
@@ -2488,6 +2494,7 @@ int __udp4_lib_rcv(struct sk_buff *skb, struct udp_table *udptable,
* RFC1122: OK. Discards the bad packet silently (as far as
* the network is concerned, anyway) as per 4.1.3.4 (MUST).
*/
+ drop_reason = SKB_DROP_REASON_UDP_CSUM;
net_dbg_ratelimited("UDP%s: bad checksum. From %pI4:%u to %pI4:%u ulen %d\n",
proto == IPPROTO_UDPLITE ? "Lite" : "",
&saddr, ntohs(uh->source), &daddr, ntohs(uh->dest),
@@ -2495,7 +2502,7 @@ int __udp4_lib_rcv(struct sk_buff *skb, struct udp_table *udptable,
__UDP_INC_STATS(net, UDP_MIB_CSUMERRORS, proto == IPPROTO_UDPLITE);
drop:
__UDP_INC_STATS(net, UDP_MIB_INERRORS, proto == IPPROTO_UDPLITE);
- kfree_skb(skb);
+ kfree_skb_reason(skb, drop_reason);
return 0;
}
@@ -2926,6 +2933,7 @@ struct proto udp_prot = {
.unhash = udp_lib_unhash,
.rehash = udp_v4_rehash,
.get_port = udp_v4_get_port,
+ .put_port = udp_lib_unhash,
#ifdef CONFIG_BPF_SYSCALL
.psock_update_sk_prot = udp_bpf_update_proto,
#endif
@@ -3075,7 +3083,7 @@ int udp4_seq_show(struct seq_file *seq, void *v)
{
seq_setwidth(seq, 127);
if (v == SEQ_START_TOKEN)
- seq_puts(seq, " sl local_address rem_address st tx_queue "
+ seq_puts(seq, " sl local_address rem_address st tx_queue "
"rx_queue tr tm->when retrnsmt uid timeout "
"inode ref pointer drops");
else {
diff --git a/net/ipv4/udp_offload.c b/net/ipv4/udp_offload.c
index 86d32a1..6d1a4be 100644
--- a/net/ipv4/udp_offload.c
+++ b/net/ipv4/udp_offload.c
@@ -7,6 +7,7 @@
*/
#include <linux/skbuff.h>
+#include <net/gro.h>
#include <net/udp.h>
#include <net/protocol.h>
#include <net/inet_common.h>
@@ -424,6 +425,33 @@ static struct sk_buff *udp4_ufo_fragment(struct sk_buff *skb,
return segs;
}
+static int skb_gro_receive_list(struct sk_buff *p, struct sk_buff *skb)
+{
+ if (unlikely(p->len + skb->len >= 65536))
+ return -E2BIG;
+
+ if (NAPI_GRO_CB(p)->last == p)
+ skb_shinfo(p)->frag_list = skb;
+ else
+ NAPI_GRO_CB(p)->last->next = skb;
+
+ skb_pull(skb, skb_gro_offset(skb));
+
+ NAPI_GRO_CB(p)->last = skb;
+ NAPI_GRO_CB(p)->count++;
+ p->data_len += skb->len;
+
+ /* sk owenrship - if any - completely transferred to the aggregated packet */
+ skb->destructor = NULL;
+ p->truesize += skb->truesize;
+ p->len += skb->len;
+
+ NAPI_GRO_CB(skb)->same_flow = 1;
+
+ return 0;
+}
+
+
#define UDP_GRO_CNT_MAX 64
static struct sk_buff *udp_gro_receive_segment(struct list_head *head,
struct sk_buff *skb)
@@ -600,13 +628,11 @@ struct sk_buff *udp4_gro_receive(struct list_head *head, struct sk_buff *skb)
inet_gro_compute_pseudo);
skip:
NAPI_GRO_CB(skb)->is_ipv6 = 0;
- rcu_read_lock();
if (static_branch_unlikely(&udp_encap_needed_key))
sk = udp4_gro_lookup_skb(skb, uh->source, uh->dest);
pp = udp_gro_receive(head, skb, uh, sk);
- rcu_read_unlock();
return pp;
flush:
@@ -641,7 +667,6 @@ int udp_gro_complete(struct sk_buff *skb, int nhoff,
uh->len = newlen;
- rcu_read_lock();
sk = INDIRECT_CALL_INET(lookup, udp6_lib_lookup_skb,
udp4_lib_lookup_skb, skb, uh->source, uh->dest);
if (sk && udp_sk(sk)->gro_complete) {
@@ -662,7 +687,6 @@ int udp_gro_complete(struct sk_buff *skb, int nhoff,
} else {
err = udp_gro_complete_segment(skb);
}
- rcu_read_unlock();
if (skb->remcsum_offload)
skb_shinfo(skb)->gso_type |= SKB_GSO_TUNNEL_REMCSUM;
diff --git a/net/ipv4/xfrm4_policy.c b/net/ipv4/xfrm4_policy.c
index 9ebd547..9e83bcb 100644
--- a/net/ipv4/xfrm4_policy.c
+++ b/net/ipv4/xfrm4_policy.c
@@ -77,7 +77,7 @@ static int xfrm4_fill_dst(struct xfrm_dst *xdst, struct net_device *dev,
xdst->u.rt.rt_iif = fl4->flowi4_iif;
xdst->u.dst.dev = dev;
- dev_hold(dev);
+ dev_hold_track(dev, &xdst->u.dst.dev_tracker, GFP_ATOMIC);
/* Sheit... I remember I did this right. Apparently,
* it was magically lost, so this code needs audit */
diff --git a/net/ipv6/addrconf.c b/net/ipv6/addrconf.c
index 3445f80..3eee177 100644
--- a/net/ipv6/addrconf.c
+++ b/net/ipv6/addrconf.c
@@ -405,13 +405,13 @@ static struct inet6_dev *ipv6_add_dev(struct net_device *dev)
if (ndev->cnf.forwarding)
dev_disable_lro(dev);
/* We refer to the device */
- dev_hold(dev);
+ dev_hold_track(dev, &ndev->dev_tracker, GFP_KERNEL);
if (snmp6_alloc_dev(ndev) < 0) {
netdev_dbg(dev, "%s: cannot allocate memory for statistics\n",
__func__);
neigh_parms_release(&nd_tbl, ndev->nd_parms);
- dev_put(dev);
+ dev_put_track(dev, &ndev->dev_tracker);
kfree(ndev);
return ERR_PTR(err);
}
diff --git a/net/ipv6/addrconf_core.c b/net/ipv6/addrconf_core.c
index 1d4054b..881d147 100644
--- a/net/ipv6/addrconf_core.c
+++ b/net/ipv6/addrconf_core.c
@@ -263,7 +263,7 @@ void in6_dev_finish_destroy(struct inet6_dev *idev)
#ifdef NET_REFCNT_DEBUG
pr_debug("%s: %s\n", __func__, dev ? dev->name : "NIL");
#endif
- dev_put(dev);
+ dev_put_track(dev, &idev->dev_tracker);
if (!idev->dead) {
pr_warn("Freeing alive inet6 device %p\n", idev);
return;
diff --git a/net/ipv6/af_inet6.c b/net/ipv6/af_inet6.c
index dab4a04..8fe7900f 100644
--- a/net/ipv6/af_inet6.c
+++ b/net/ipv6/af_inet6.c
@@ -337,11 +337,8 @@ static int __inet6_bind(struct sock *sk, struct sockaddr *uaddr, int addr_len,
chk_addr_ret = inet_addr_type_dev_table(net, dev, v4addr);
rcu_read_unlock();
- if (!inet_can_nonlocal_bind(net, inet) &&
- v4addr != htonl(INADDR_ANY) &&
- chk_addr_ret != RTN_LOCAL &&
- chk_addr_ret != RTN_MULTICAST &&
- chk_addr_ret != RTN_BROADCAST) {
+ if (!inet_addr_valid_or_nonlocal(net, inet, v4addr,
+ chk_addr_ret)) {
err = -EADDRNOTAVAIL;
goto out;
}
@@ -416,6 +413,8 @@ static int __inet6_bind(struct sock *sk, struct sockaddr *uaddr, int addr_len,
if (err) {
sk->sk_ipv6only = saved_ipv6only;
inet_reset_saddr(sk);
+ if (sk->sk_prot->put_port)
+ sk->sk_prot->put_port(sk);
goto out;
}
}
diff --git a/net/ipv6/ah6.c b/net/ipv6/ah6.c
index 828e625..b5995c1 100644
--- a/net/ipv6/ah6.c
+++ b/net/ipv6/ah6.c
@@ -175,7 +175,6 @@ static void ipv6_rearrange_destopt(struct ipv6hdr *iph, struct ipv6_opt_hdr *des
* See 11.3.2 of RFC 3775 for details.
*/
if (opt[off] == IPV6_TLV_HAO) {
- struct in6_addr final_addr;
struct ipv6_destopt_hao *hao;
hao = (struct ipv6_destopt_hao *)&opt[off];
@@ -184,9 +183,7 @@ static void ipv6_rearrange_destopt(struct ipv6hdr *iph, struct ipv6_opt_hdr *des
hao->length);
goto bad;
}
- final_addr = hao->addr;
- hao->addr = iph->saddr;
- iph->saddr = final_addr;
+ swap(hao->addr, iph->saddr);
}
break;
}
diff --git a/net/ipv6/esp6.c b/net/ipv6/esp6.c
index f0bac6f..8bb2c40 100644
--- a/net/ipv6/esp6.c
+++ b/net/ipv6/esp6.c
@@ -114,7 +114,6 @@ static inline struct scatterlist *esp_req_sg(struct crypto_aead *aead,
static void esp_ssg_unref(struct xfrm_state *x, void *tmp)
{
- struct esp_output_extra *extra = esp_tmp_extra(tmp);
struct crypto_aead *aead = x->data;
int extralen = 0;
u8 *iv;
@@ -122,7 +121,7 @@ static void esp_ssg_unref(struct xfrm_state *x, void *tmp)
struct scatterlist *sg;
if (x->props.flags & XFRM_STATE_ESN)
- extralen += sizeof(*extra);
+ extralen += sizeof(struct esp_output_extra);
iv = esp_tmp_iv(aead, tmp, extralen);
req = esp_tmp_req(aead, iv);
diff --git a/net/ipv6/esp6_offload.c b/net/ipv6/esp6_offload.c
index a349d47..ba5e81c 100644
--- a/net/ipv6/esp6_offload.c
+++ b/net/ipv6/esp6_offload.c
@@ -16,6 +16,7 @@
#include <crypto/authenc.h>
#include <linux/err.h>
#include <linux/module.h>
+#include <net/gro.h>
#include <net/ip.h>
#include <net/xfrm.h>
#include <net/esp.h>
diff --git a/net/ipv6/exthdrs.c b/net/ipv6/exthdrs.c
index 38ece3b..77e34ae 100644
--- a/net/ipv6/exthdrs.c
+++ b/net/ipv6/exthdrs.c
@@ -686,7 +686,6 @@ static int ipv6_rthdr_rcv(struct sk_buff *skb)
struct net *net = dev_net(skb->dev);
int accept_source_route = net->ipv6.devconf_all->accept_source_route;
- idev = __in6_dev_get(skb->dev);
if (idev && accept_source_route > idev->cnf.accept_source_route)
accept_source_route = idev->cnf.accept_source_route;
diff --git a/net/ipv6/fib6_rules.c b/net/ipv6/fib6_rules.c
index dcedfe2..ec029c8 100644
--- a/net/ipv6/fib6_rules.c
+++ b/net/ipv6/fib6_rules.c
@@ -340,10 +340,6 @@ INDIRECT_CALLABLE_SCOPE int fib6_rule_match(struct fib_rule *rule,
return 1;
}
-static const struct nla_policy fib6_rule_policy[FRA_MAX+1] = {
- FRA_GENERIC_POLICY,
-};
-
static int fib6_rule_configure(struct fib_rule *rule, struct sk_buff *skb,
struct fib_rule_hdr *frh,
struct nlattr **tb,
@@ -459,7 +455,6 @@ static const struct fib_rules_ops __net_initconst fib6_rules_ops_template = {
.fill = fib6_rule_fill,
.nlmsg_payload = fib6_rule_nlmsg_payload,
.nlgroup = RTNLGRP_IPV6_RULE,
- .policy = fib6_rule_policy,
.owner = THIS_MODULE,
.fro_net = &init_net,
};
diff --git a/net/ipv6/icmp.c b/net/ipv6/icmp.c
index a7c31ab..96c5cc0 100644
--- a/net/ipv6/icmp.c
+++ b/net/ipv6/icmp.c
@@ -57,6 +57,7 @@
#include <net/protocol.h>
#include <net/raw.h>
#include <net/rawv6.h>
+#include <net/seg6.h>
#include <net/transp_v6.h>
#include <net/ip6_route.h>
#include <net/addrconf.h>
@@ -820,6 +821,7 @@ static void icmpv6_echo_reply(struct sk_buff *skb)
void icmpv6_notify(struct sk_buff *skb, u8 type, u8 code, __be32 info)
{
+ struct inet6_skb_parm *opt = IP6CB(skb);
const struct inet6_protocol *ipprot;
int inner_offset;
__be16 frag_off;
@@ -829,6 +831,8 @@ void icmpv6_notify(struct sk_buff *skb, u8 type, u8 code, __be32 info)
if (!pskb_may_pull(skb, sizeof(struct ipv6hdr)))
goto out;
+ seg6_icmp_srh(skb, opt);
+
nexthdr = ((struct ipv6hdr *)skb->data)->nexthdr;
if (ipv6_ext_hdr(nexthdr)) {
/* now skip over extension headers */
@@ -853,7 +857,7 @@ void icmpv6_notify(struct sk_buff *skb, u8 type, u8 code, __be32 info)
ipprot = rcu_dereference(inet6_protos[nexthdr]);
if (ipprot && ipprot->err_handler)
- ipprot->err_handler(skb, NULL, type, code, inner_offset, info);
+ ipprot->err_handler(skb, opt, type, code, inner_offset, info);
raw6_icmp_error(skb, nexthdr, type, code, inner_offset, info);
return;
diff --git a/net/ipv6/inet6_hashtables.c b/net/ipv6/inet6_hashtables.c
index 67c9114..4514444 100644
--- a/net/ipv6/inet6_hashtables.c
+++ b/net/ipv6/inet6_hashtables.c
@@ -165,7 +165,7 @@ static inline struct sock *inet6_lookup_run_bpf(struct net *net,
const struct in6_addr *saddr,
const __be16 sport,
const struct in6_addr *daddr,
- const u16 hnum)
+ const u16 hnum, const int dif)
{
struct sock *sk, *reuse_sk;
bool no_reuseport;
@@ -173,8 +173,8 @@ static inline struct sock *inet6_lookup_run_bpf(struct net *net,
if (hashinfo != &tcp_hashinfo)
return NULL; /* only TCP is supported */
- no_reuseport = bpf_sk_lookup_run_v6(net, IPPROTO_TCP,
- saddr, sport, daddr, hnum, &sk);
+ no_reuseport = bpf_sk_lookup_run_v6(net, IPPROTO_TCP, saddr, sport,
+ daddr, hnum, dif, &sk);
if (no_reuseport || IS_ERR_OR_NULL(sk))
return sk;
@@ -198,7 +198,7 @@ struct sock *inet6_lookup_listener(struct net *net,
/* Lookup redirect from BPF */
if (static_branch_unlikely(&bpf_sk_lookup_enabled)) {
result = inet6_lookup_run_bpf(net, hashinfo, skb, doff,
- saddr, sport, daddr, hnum);
+ saddr, sport, daddr, hnum, dif);
if (result)
goto done;
}
diff --git a/net/ipv6/ioam6.c b/net/ipv6/ioam6.c
index 122a3d4..e159eb4 100644
--- a/net/ipv6/ioam6.c
+++ b/net/ipv6/ioam6.c
@@ -13,10 +13,12 @@
#include <linux/ioam6.h>
#include <linux/ioam6_genl.h>
#include <linux/rhashtable.h>
+#include <linux/netdevice.h>
#include <net/addrconf.h>
#include <net/genetlink.h>
#include <net/ioam6.h>
+#include <net/sch_generic.h>
static void ioam6_ns_release(struct ioam6_namespace *ns)
{
@@ -717,7 +719,19 @@ static void __ioam6_fill_trace_data(struct sk_buff *skb,
/* queue depth */
if (trace->type.bit6) {
- *(__be32 *)data = cpu_to_be32(IOAM6_U32_UNAVAILABLE);
+ struct netdev_queue *queue;
+ struct Qdisc *qdisc;
+ __u32 qlen, backlog;
+
+ if (skb_dst(skb)->dev->flags & IFF_LOOPBACK) {
+ *(__be32 *)data = cpu_to_be32(IOAM6_U32_UNAVAILABLE);
+ } else {
+ queue = skb_get_tx_queue(skb_dst(skb)->dev, skb);
+ qdisc = rcu_dereference(queue->qdisc);
+ qdisc_qstats_qlen_backlog(qdisc, &qlen, &backlog);
+
+ *(__be32 *)data = cpu_to_be32(backlog);
+ }
data += sizeof(__be32);
}
diff --git a/net/ipv6/ip6_fib.c b/net/ipv6/ip6_fib.c
index 0371d2c..463c37d 100644
--- a/net/ipv6/ip6_fib.c
+++ b/net/ipv6/ip6_fib.c
@@ -15,6 +15,7 @@
#define pr_fmt(fmt) "IPv6: " fmt
+#include <linux/bpf.h>
#include <linux/errno.h>
#include <linux/types.h>
#include <linux/net.h>
diff --git a/net/ipv6/ip6_gre.c b/net/ipv6/ip6_gre.c
index d831d243..8753e9c 100644
--- a/net/ipv6/ip6_gre.c
+++ b/net/ipv6/ip6_gre.c
@@ -403,7 +403,7 @@ static void ip6erspan_tunnel_uninit(struct net_device *dev)
ip6erspan_tunnel_unlink_md(ign, t);
ip6gre_tunnel_unlink(ign, t);
dst_cache_reset(&t->dst_cache);
- dev_put(dev);
+ dev_put_track(dev, &t->dev_tracker);
}
static void ip6gre_tunnel_uninit(struct net_device *dev)
@@ -416,7 +416,7 @@ static void ip6gre_tunnel_uninit(struct net_device *dev)
if (ign->fb_tunnel_dev == dev)
WRITE_ONCE(ign->fb_tunnel_dev, NULL);
dst_cache_reset(&t->dst_cache);
- dev_put(dev);
+ dev_put_track(dev, &t->dev_tracker);
}
@@ -755,6 +755,7 @@ static netdev_tx_t __gre6_xmit(struct sk_buff *skb,
fl6->daddr = key->u.ipv6.dst;
fl6->flowlabel = key->label;
fl6->flowi6_uid = sock_net_uid(dev_net(dev), NULL);
+ fl6->fl6_gre_key = tunnel_id_to_key32(key->tun_id);
dsfield = key->tos;
flags = key->tun_flags &
@@ -990,6 +991,7 @@ static netdev_tx_t ip6erspan_tunnel_xmit(struct sk_buff *skb,
fl6.daddr = key->u.ipv6.dst;
fl6.flowlabel = key->label;
fl6.flowi6_uid = sock_net_uid(dev_net(dev), NULL);
+ fl6.fl6_gre_key = tunnel_id_to_key32(key->tun_id);
dsfield = key->tos;
if (!(tun_info->key.tun_flags & TUNNEL_ERSPAN_OPT))
@@ -1098,6 +1100,7 @@ static void ip6gre_tnl_link_config_common(struct ip6_tnl *t)
fl6->flowi6_oif = p->link;
fl6->flowlabel = 0;
fl6->flowi6_proto = IPPROTO_GRE;
+ fl6->fl6_gre_key = t->parms.o_key;
if (!(p->flags&IP6_TNL_F_USE_ORIG_TCLASS))
fl6->flowlabel |= IPV6_TCLASS_MASK & p->flowinfo;
@@ -1496,7 +1499,7 @@ static int ip6gre_tunnel_init_common(struct net_device *dev)
}
ip6gre_tnl_init_features(dev);
- dev_hold(dev);
+ dev_hold_track(dev, &tunnel->dev_tracker, GFP_KERNEL);
return 0;
cleanup_dst_cache_init:
@@ -1544,7 +1547,7 @@ static void ip6gre_fb_tunnel_init(struct net_device *dev)
static struct inet6_protocol ip6gre_protocol __read_mostly = {
.handler = gre_rcv,
.err_handler = ip6gre_err,
- .flags = INET6_PROTO_NOPOLICY|INET6_PROTO_FINAL,
+ .flags = INET6_PROTO_FINAL,
};
static void ip6gre_destroy_tunnels(struct net *net, struct list_head *head)
@@ -1888,7 +1891,7 @@ static int ip6erspan_tap_init(struct net_device *dev)
dev->priv_flags |= IFF_LIVE_ADDR_CHANGE;
ip6erspan_tnl_link_config(tunnel, 1);
- dev_hold(dev);
+ dev_hold_track(dev, &tunnel->dev_tracker, GFP_KERNEL);
return 0;
cleanup_dst_cache_init:
diff --git a/net/ipv6/ip6_offload.c b/net/ipv6/ip6_offload.c
index 1cbd49d..b29e9ba 100644
--- a/net/ipv6/ip6_offload.c
+++ b/net/ipv6/ip6_offload.c
@@ -208,7 +208,6 @@ INDIRECT_CALLABLE_SCOPE struct sk_buff *ipv6_gro_receive(struct list_head *head,
flush += ntohs(iph->payload_len) != skb_gro_len(skb);
- rcu_read_lock();
proto = iph->nexthdr;
ops = rcu_dereference(inet6_offloads[proto]);
if (!ops || !ops->callbacks.gro_receive) {
@@ -221,7 +220,7 @@ INDIRECT_CALLABLE_SCOPE struct sk_buff *ipv6_gro_receive(struct list_head *head,
ops = rcu_dereference(inet6_offloads[proto]);
if (!ops || !ops->callbacks.gro_receive)
- goto out_unlock;
+ goto out;
iph = ipv6_hdr(skb);
}
@@ -279,9 +278,6 @@ INDIRECT_CALLABLE_SCOPE struct sk_buff *ipv6_gro_receive(struct list_head *head,
pp = indirect_call_gro_receive_l4(tcp6_gro_receive, udp6_gro_receive,
ops->callbacks.gro_receive, head, skb);
-out_unlock:
- rcu_read_unlock();
-
out:
skb_gro_flush_final(skb, pp, flush);
@@ -331,18 +327,14 @@ INDIRECT_CALLABLE_SCOPE int ipv6_gro_complete(struct sk_buff *skb, int nhoff)
iph->payload_len = htons(skb->len - nhoff - sizeof(*iph));
- rcu_read_lock();
-
nhoff += sizeof(*iph) + ipv6_exthdrs_len(iph, &ops);
if (WARN_ON(!ops || !ops->callbacks.gro_complete))
- goto out_unlock;
+ goto out;
err = INDIRECT_CALL_L4(ops->callbacks.gro_complete, tcp6_gro_complete,
udp6_gro_complete, skb, nhoff);
-out_unlock:
- rcu_read_unlock();
-
+out:
return err;
}
diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c
index ff4e83e..2995f8d 100644
--- a/net/ipv6/ip6_output.c
+++ b/net/ipv6/ip6_output.c
@@ -977,7 +977,7 @@ int ip6_fragment(struct net *net, struct sock *sk, struct sk_buff *skb,
fail_toobig:
if (skb->sk && dst_allfrag(skb_dst(skb)))
- sk_nocaps_add(skb->sk, NETIF_F_GSO_MASK);
+ sk_gso_disable(skb->sk);
icmpv6_send(skb, ICMPV6_PKT_TOOBIG, 0, mtu);
err = -EMSGSIZE;
diff --git a/net/ipv6/ip6_tunnel.c b/net/ipv6/ip6_tunnel.c
index 484aca4..fe786df 100644
--- a/net/ipv6/ip6_tunnel.c
+++ b/net/ipv6/ip6_tunnel.c
@@ -383,7 +383,7 @@ ip6_tnl_dev_uninit(struct net_device *dev)
else
ip6_tnl_unlink(ip6n, t);
dst_cache_reset(&t->dst_cache);
- dev_put(dev);
+ dev_put_track(dev, &t->dev_tracker);
}
/**
@@ -1883,7 +1883,7 @@ ip6_tnl_dev_init_gen(struct net_device *dev)
dev->min_mtu = ETH_MIN_MTU;
dev->max_mtu = IP6_MAX_MTU - dev->hard_header_len;
- dev_hold(dev);
+ dev_hold_track(dev, &t->dev_tracker, GFP_KERNEL);
return 0;
destroy_dst:
diff --git a/net/ipv6/ip6_vti.c b/net/ipv6/ip6_vti.c
index 527e9ea..3a434d7 100644
--- a/net/ipv6/ip6_vti.c
+++ b/net/ipv6/ip6_vti.c
@@ -293,7 +293,7 @@ static void vti6_dev_uninit(struct net_device *dev)
RCU_INIT_POINTER(ip6n->tnls_wc[0], NULL);
else
vti6_tnl_unlink(ip6n, t);
- dev_put(dev);
+ dev_put_track(dev, &t->dev_tracker);
}
static int vti6_input_proto(struct sk_buff *skb, int nexthdr, __be32 spi,
@@ -808,6 +808,8 @@ vti6_siocdevprivate(struct net_device *dev, struct ifreq *ifr, void __user *data
struct net *net = dev_net(dev);
struct vti6_net *ip6n = net_generic(net, vti6_net_id);
+ memset(&p1, 0, sizeof(p1));
+
switch (cmd) {
case SIOCGETTUNNEL:
if (dev == ip6n->fb_tnl_dev) {
@@ -934,7 +936,7 @@ static inline int vti6_dev_init_gen(struct net_device *dev)
dev->tstats = netdev_alloc_pcpu_stats(struct pcpu_sw_netstats);
if (!dev->tstats)
return -ENOMEM;
- dev_hold(dev);
+ dev_hold_track(dev, &t->dev_tracker, GFP_KERNEL);
return 0;
}
diff --git a/net/ipv6/ip6mr.c b/net/ipv6/ip6mr.c
index 36ed9ef..7cf73e6 100644
--- a/net/ipv6/ip6mr.c
+++ b/net/ipv6/ip6mr.c
@@ -182,10 +182,6 @@ static int ip6mr_rule_match(struct fib_rule *rule, struct flowi *flp, int flags)
return 1;
}
-static const struct nla_policy ip6mr_rule_policy[FRA_MAX + 1] = {
- FRA_GENERIC_POLICY,
-};
-
static int ip6mr_rule_configure(struct fib_rule *rule, struct sk_buff *skb,
struct fib_rule_hdr *frh, struct nlattr **tb,
struct netlink_ext_ack *extack)
@@ -218,7 +214,6 @@ static const struct fib_rules_ops __net_initconst ip6mr_rules_ops_template = {
.compare = ip6mr_rule_compare,
.fill = ip6mr_rule_fill,
.nlgroup = RTNLGRP_IPV6_RULE,
- .policy = ip6mr_rule_policy,
.owner = THIS_MODULE,
};
@@ -746,7 +741,7 @@ static int mif6_delete(struct mr_table *mrt, int vifi, int notify,
if ((v->flags & MIFF_REGISTER) && !notify)
unregister_netdevice_queue(dev, head);
- dev_put(dev);
+ dev_put_track(dev, &v->dev_tracker);
return 0;
}
@@ -919,6 +914,7 @@ static int mif6_add(struct net *net, struct mr_table *mrt,
/* And finish update writing critical data */
write_lock_bh(&mrt_lock);
v->dev = dev;
+ netdev_tracker_alloc(dev, &v->dev_tracker, GFP_ATOMIC);
#ifdef CONFIG_IPV6_PIMSM_V2
if (v->flags & MIFF_REGISTER)
mrt->mroute_reg_vif_num = vifi;
diff --git a/net/ipv6/ipv6_sockglue.c b/net/ipv6/ipv6_sockglue.c
index 41efca8..a733803 100644
--- a/net/ipv6/ipv6_sockglue.c
+++ b/net/ipv6/ipv6_sockglue.c
@@ -471,10 +471,10 @@ static int do_ipv6_setsockopt(struct sock *sk, int level, int optname,
if (sk->sk_protocol == IPPROTO_TCP) {
struct inet_connection_sock *icsk = inet_csk(sk);
- local_bh_disable();
+
sock_prot_inuse_add(net, sk->sk_prot, -1);
sock_prot_inuse_add(net, &tcp_prot, 1);
- local_bh_enable();
+
sk->sk_prot = &tcp_prot;
icsk->icsk_af_ops = &ipv4_specific;
sk->sk_socket->ops = &inet_stream_ops;
@@ -485,10 +485,10 @@ static int do_ipv6_setsockopt(struct sock *sk, int level, int optname,
if (sk->sk_protocol == IPPROTO_UDPLITE)
prot = &udplite_prot;
- local_bh_disable();
+
sock_prot_inuse_add(net, sk->sk_prot, -1);
sock_prot_inuse_add(net, prot, 1);
- local_bh_enable();
+
sk->sk_prot = prot;
sk->sk_socket->ops = &inet_dgram_ops;
sk->sk_family = PF_INET;
@@ -599,7 +599,14 @@ static int do_ipv6_setsockopt(struct sock *sk, int level, int optname,
/* RFC 3542, 6.5: default traffic class of 0x0 */
if (val == -1)
val = 0;
- np->tclass = val;
+ if (sk->sk_type == SOCK_STREAM) {
+ val &= ~INET_ECN_MASK;
+ val |= np->tclass & INET_ECN_MASK;
+ }
+ if (np->tclass != val) {
+ np->tclass = val;
+ sk_dst_reset(sk);
+ }
retv = 0;
break;
diff --git a/net/ipv6/netfilter/Kconfig b/net/ipv6/netfilter/Kconfig
index f22233e..97d3d1b 100644
--- a/net/ipv6/netfilter/Kconfig
+++ b/net/ipv6/netfilter/Kconfig
@@ -48,12 +48,8 @@
endif # NF_TABLES
config NF_FLOW_TABLE_IPV6
- tristate "Netfilter flow table IPv6 module"
- depends on NF_FLOW_TABLE
- help
- This option adds the flow table IPv6 support.
-
- To compile it as a module, choose M here.
+ tristate
+ select NF_FLOW_TABLE_INET
config NF_DUP_IPV6
tristate "Netfilter IPv6 packet duplication to alternate destination"
diff --git a/net/ipv6/netfilter/nf_flow_table_ipv6.c b/net/ipv6/netfilter/nf_flow_table_ipv6.c
index 667b8af..e69de29 100644
--- a/net/ipv6/netfilter/nf_flow_table_ipv6.c
+++ b/net/ipv6/netfilter/nf_flow_table_ipv6.c
@@ -1,38 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-only
-#include <linux/kernel.h>
-#include <linux/init.h>
-#include <linux/module.h>
-#include <linux/netfilter.h>
-#include <linux/rhashtable.h>
-#include <net/netfilter/nf_flow_table.h>
-#include <net/netfilter/nf_tables.h>
-
-static struct nf_flowtable_type flowtable_ipv6 = {
- .family = NFPROTO_IPV6,
- .init = nf_flow_table_init,
- .setup = nf_flow_table_offload_setup,
- .action = nf_flow_rule_route_ipv6,
- .free = nf_flow_table_free,
- .hook = nf_flow_offload_ipv6_hook,
- .owner = THIS_MODULE,
-};
-
-static int __init nf_flow_ipv6_module_init(void)
-{
- nft_register_flowtable_type(&flowtable_ipv6);
-
- return 0;
-}
-
-static void __exit nf_flow_ipv6_module_exit(void)
-{
- nft_unregister_flowtable_type(&flowtable_ipv6);
-}
-
-module_init(nf_flow_ipv6_module_init);
-module_exit(nf_flow_ipv6_module_exit);
-
-MODULE_LICENSE("GPL");
-MODULE_AUTHOR("Pablo Neira Ayuso <pablo@netfilter.org>");
-MODULE_ALIAS_NF_FLOWTABLE(AF_INET6);
-MODULE_DESCRIPTION("Netfilter flow table IPv6 module");
diff --git a/net/ipv6/ping.c b/net/ipv6/ping.c
index 6ac88fe..9256f6b 100644
--- a/net/ipv6/ping.c
+++ b/net/ipv6/ping.c
@@ -177,6 +177,7 @@ struct proto pingv6_prot = {
.hash = ping_hash,
.unhash = ping_unhash,
.get_port = ping_get_port,
+ .put_port = ping_unhash,
.obj_size = sizeof(struct raw6_sock),
};
EXPORT_SYMBOL_GPL(pingv6_prot);
diff --git a/net/ipv6/raw.c b/net/ipv6/raw.c
index 60f1e4f..c51d5ce 100644
--- a/net/ipv6/raw.c
+++ b/net/ipv6/raw.c
@@ -1020,6 +1020,9 @@ static int do_rawv6_setsockopt(struct sock *sk, int level, int optname,
struct raw6_sock *rp = raw6_sk(sk);
int val;
+ if (optlen < sizeof(val))
+ return -EINVAL;
+
if (copy_from_sockptr(&val, optval, sizeof(val)))
return -EFAULT;
diff --git a/net/ipv6/route.c b/net/ipv6/route.c
index 42d60c7..e6de942 100644
--- a/net/ipv6/route.c
+++ b/net/ipv6/route.c
@@ -182,8 +182,9 @@ static void rt6_uncached_list_flush_dev(struct net *net, struct net_device *dev)
if (rt_dev == dev) {
rt->dst.dev = blackhole_netdev;
- dev_hold(rt->dst.dev);
- dev_put(rt_dev);
+ dev_replace_track(rt_dev, blackhole_netdev,
+ &rt->dst.dev_tracker,
+ GFP_ATOMIC);
}
}
spin_unlock_bh(&ul->lock);
@@ -328,9 +329,7 @@ static const struct rt6_info ip6_blk_hole_entry_template = {
static void rt6_info_init(struct rt6_info *rt)
{
- struct dst_entry *dst = &rt->dst;
-
- memset(dst + 1, 0, sizeof(*rt) - sizeof(*dst));
+ memset_after(rt, 0, dst);
INIT_LIST_HEAD(&rt->rt6i_uncached);
}
@@ -594,6 +593,7 @@ struct __rt6_probe_work {
struct work_struct work;
struct in6_addr target;
struct net_device *dev;
+ netdevice_tracker dev_tracker;
};
static void rt6_probe_deferred(struct work_struct *w)
@@ -604,7 +604,7 @@ static void rt6_probe_deferred(struct work_struct *w)
addrconf_addr_solict_mult(&work->target, &mcaddr);
ndisc_send_ns(work->dev, &work->target, &mcaddr, NULL, 0);
- dev_put(work->dev);
+ dev_put_track(work->dev, &work->dev_tracker);
kfree(work);
}
@@ -658,7 +658,7 @@ static void rt6_probe(struct fib6_nh *fib6_nh)
} else {
INIT_WORK(&work->work, rt6_probe_deferred);
work->target = *nh_gw;
- dev_hold(dev);
+ dev_hold_track(dev, &work->dev_tracker, GFP_ATOMIC);
work->dev = dev;
schedule_work(&work->work);
}
@@ -1485,7 +1485,7 @@ static void rt6_exception_remove_oldest(struct rt6_exception_bucket *bucket)
static u32 rt6_exception_hash(const struct in6_addr *dst,
const struct in6_addr *src)
{
- static siphash_key_t rt6_exception_key __read_mostly;
+ static siphash_aligned_key_t rt6_exception_key;
struct {
struct in6_addr dst;
struct in6_addr src;
@@ -3628,6 +3628,8 @@ int fib6_nh_init(struct net *net, struct fib6_nh *fib6_nh,
}
fib6_nh->fib_nh_dev = dev;
+ netdev_tracker_alloc(dev, &fib6_nh->fib_nh_dev_tracker, gfp_flags);
+
fib6_nh->fib_nh_oif = dev->ifindex;
err = 0;
out:
@@ -3658,24 +3660,8 @@ void fib6_nh_release(struct fib6_nh *fib6_nh)
rcu_read_unlock();
- if (fib6_nh->rt6i_pcpu) {
- int cpu;
-
- for_each_possible_cpu(cpu) {
- struct rt6_info **ppcpu_rt;
- struct rt6_info *pcpu_rt;
-
- ppcpu_rt = per_cpu_ptr(fib6_nh->rt6i_pcpu, cpu);
- pcpu_rt = *ppcpu_rt;
- if (pcpu_rt) {
- dst_dev_put(&pcpu_rt->dst);
- dst_release(&pcpu_rt->dst);
- *ppcpu_rt = NULL;
- }
- }
-
- free_percpu(fib6_nh->rt6i_pcpu);
- }
+ fib6_nh_release_dsts(fib6_nh);
+ free_percpu(fib6_nh->rt6i_pcpu);
fib_nh_common_release(&fib6_nh->nh_common);
}
@@ -5224,6 +5210,19 @@ static bool ip6_route_mpath_should_notify(const struct fib6_info *rt)
return should_notify;
}
+static int fib6_gw_from_attr(struct in6_addr *gw, struct nlattr *nla,
+ struct netlink_ext_ack *extack)
+{
+ if (nla_len(nla) < sizeof(*gw)) {
+ NL_SET_ERR_MSG(extack, "Invalid IPv6 address in RTA_GATEWAY");
+ return -EINVAL;
+ }
+
+ *gw = nla_get_in6_addr(nla);
+
+ return 0;
+}
+
static int ip6_route_multipath_add(struct fib6_config *cfg,
struct netlink_ext_ack *extack)
{
@@ -5264,10 +5263,18 @@ static int ip6_route_multipath_add(struct fib6_config *cfg,
nla = nla_find(attrs, attrlen, RTA_GATEWAY);
if (nla) {
- r_cfg.fc_gateway = nla_get_in6_addr(nla);
+ err = fib6_gw_from_attr(&r_cfg.fc_gateway, nla,
+ extack);
+ if (err)
+ goto cleanup;
+
r_cfg.fc_flags |= RTF_GATEWAY;
}
r_cfg.fc_encap = nla_find(attrs, attrlen, RTA_ENCAP);
+
+ /* RTA_ENCAP_TYPE length checked in
+ * lwtunnel_valid_encap_type_attr
+ */
nla = nla_find(attrs, attrlen, RTA_ENCAP_TYPE);
if (nla)
r_cfg.fc_encap_type = nla_get_u16(nla);
@@ -5434,7 +5441,13 @@ static int ip6_route_multipath_del(struct fib6_config *cfg,
nla = nla_find(attrs, attrlen, RTA_GATEWAY);
if (nla) {
- nla_memcpy(&r_cfg.fc_gateway, nla, 16);
+ err = fib6_gw_from_attr(&r_cfg.fc_gateway, nla,
+ extack);
+ if (err) {
+ last_err = err;
+ goto next_rtnh;
+ }
+
r_cfg.fc_flags |= RTF_GATEWAY;
}
}
@@ -5442,6 +5455,7 @@ static int ip6_route_multipath_del(struct fib6_config *cfg,
if (err)
last_err = err;
+next_rtnh:
rtnh = rtnh_next(rtnh, &remaining);
}
diff --git a/net/ipv6/seg6.c b/net/ipv6/seg6.c
index a8b5784..73aaabf 100644
--- a/net/ipv6/seg6.c
+++ b/net/ipv6/seg6.c
@@ -75,6 +75,65 @@ bool seg6_validate_srh(struct ipv6_sr_hdr *srh, int len, bool reduced)
return true;
}
+struct ipv6_sr_hdr *seg6_get_srh(struct sk_buff *skb, int flags)
+{
+ struct ipv6_sr_hdr *srh;
+ int len, srhoff = 0;
+
+ if (ipv6_find_hdr(skb, &srhoff, IPPROTO_ROUTING, NULL, &flags) < 0)
+ return NULL;
+
+ if (!pskb_may_pull(skb, srhoff + sizeof(*srh)))
+ return NULL;
+
+ srh = (struct ipv6_sr_hdr *)(skb->data + srhoff);
+
+ len = (srh->hdrlen + 1) << 3;
+
+ if (!pskb_may_pull(skb, srhoff + len))
+ return NULL;
+
+ /* note that pskb_may_pull may change pointers in header;
+ * for this reason it is necessary to reload them when needed.
+ */
+ srh = (struct ipv6_sr_hdr *)(skb->data + srhoff);
+
+ if (!seg6_validate_srh(srh, len, true))
+ return NULL;
+
+ return srh;
+}
+
+/* Determine if an ICMP invoking packet contains a segment routing
+ * header. If it does, extract the offset to the true destination
+ * address, which is in the first segment address.
+ */
+void seg6_icmp_srh(struct sk_buff *skb, struct inet6_skb_parm *opt)
+{
+ __u16 network_header = skb->network_header;
+ struct ipv6_sr_hdr *srh;
+
+ /* Update network header to point to the invoking packet
+ * inside the ICMP packet, so we can use the seg6_get_srh()
+ * helper.
+ */
+ skb_reset_network_header(skb);
+
+ srh = seg6_get_srh(skb, 0);
+ if (!srh)
+ goto out;
+
+ if (srh->type != IPV6_SRCRT_TYPE_4)
+ goto out;
+
+ opt->flags |= IP6SKB_SEG6;
+ opt->srhoff = (unsigned char *)srh - skb->data;
+
+out:
+ /* Restore the network header back to the ICMP packet */
+ skb->network_header = network_header;
+}
+
static struct genl_family seg6_genl_family;
static const struct nla_policy seg6_genl_policy[SEG6_ATTR_MAX + 1] = {
diff --git a/net/ipv6/seg6_local.c b/net/ipv6/seg6_local.c
index 2dc40b3..9fbe243 100644
--- a/net/ipv6/seg6_local.c
+++ b/net/ipv6/seg6_local.c
@@ -7,6 +7,7 @@
* eBPF support: Mathieu Xhonneux <m.xhonneux@gmail.com>
*/
+#include <linux/filter.h>
#include <linux/types.h>
#include <linux/skbuff.h>
#include <linux/net.h>
@@ -150,40 +151,11 @@ static struct seg6_local_lwt *seg6_local_lwtunnel(struct lwtunnel_state *lwt)
return (struct seg6_local_lwt *)lwt->data;
}
-static struct ipv6_sr_hdr *get_srh(struct sk_buff *skb, int flags)
-{
- struct ipv6_sr_hdr *srh;
- int len, srhoff = 0;
-
- if (ipv6_find_hdr(skb, &srhoff, IPPROTO_ROUTING, NULL, &flags) < 0)
- return NULL;
-
- if (!pskb_may_pull(skb, srhoff + sizeof(*srh)))
- return NULL;
-
- srh = (struct ipv6_sr_hdr *)(skb->data + srhoff);
-
- len = (srh->hdrlen + 1) << 3;
-
- if (!pskb_may_pull(skb, srhoff + len))
- return NULL;
-
- /* note that pskb_may_pull may change pointers in header;
- * for this reason it is necessary to reload them when needed.
- */
- srh = (struct ipv6_sr_hdr *)(skb->data + srhoff);
-
- if (!seg6_validate_srh(srh, len, true))
- return NULL;
-
- return srh;
-}
-
static struct ipv6_sr_hdr *get_and_validate_srh(struct sk_buff *skb)
{
struct ipv6_sr_hdr *srh;
- srh = get_srh(skb, IP6_FH_F_SKIP_RH);
+ srh = seg6_get_srh(skb, IP6_FH_F_SKIP_RH);
if (!srh)
return NULL;
@@ -200,7 +172,7 @@ static bool decap_and_validate(struct sk_buff *skb, int proto)
struct ipv6_sr_hdr *srh;
unsigned int off = 0;
- srh = get_srh(skb, 0);
+ srh = seg6_get_srh(skb, 0);
if (srh && srh->segments_left > 0)
return false;
diff --git a/net/ipv6/sit.c b/net/ipv6/sit.c
index 8a3618a..a618dce 100644
--- a/net/ipv6/sit.c
+++ b/net/ipv6/sit.c
@@ -521,7 +521,7 @@ static void ipip6_tunnel_uninit(struct net_device *dev)
ipip6_tunnel_del_prl(tunnel, NULL);
}
dst_cache_reset(&tunnel->dst_cache);
- dev_put(dev);
+ dev_put_track(dev, &tunnel->dev_tracker);
}
static int ipip6_err(struct sk_buff *skb, u32 info)
@@ -1463,7 +1463,7 @@ static int ipip6_tunnel_init(struct net_device *dev)
dev->tstats = NULL;
return err;
}
- dev_hold(dev);
+ dev_hold_track(dev, &tunnel->dev_tracker, GFP_KERNEL);
return 0;
}
diff --git a/net/ipv6/syncookies.c b/net/ipv6/syncookies.c
index e8cfb9e..d1b61d0 100644
--- a/net/ipv6/syncookies.c
+++ b/net/ipv6/syncookies.c
@@ -20,7 +20,7 @@
#define COOKIEBITS 24 /* Upper bits store count */
#define COOKIEMASK (((__u32)1 << COOKIEBITS) - 1)
-static siphash_key_t syncookie6_secret[2] __read_mostly;
+static siphash_aligned_key_t syncookie6_secret[2];
/* RFC 2460, Section 8.3:
* [ipv6 tcp] MSS must be computed as the maximum packet size minus 60 [..]
diff --git a/net/ipv6/tcp_ipv6.c b/net/ipv6/tcp_ipv6.c
index 680e648..075ee8a 100644
--- a/net/ipv6/tcp_ipv6.c
+++ b/net/ipv6/tcp_ipv6.c
@@ -72,7 +72,7 @@ static void tcp_v6_send_reset(const struct sock *sk, struct sk_buff *skb);
static void tcp_v6_reqsk_send_ack(const struct sock *sk, struct sk_buff *skb,
struct request_sock *req);
-static int tcp_v6_do_rcv(struct sock *sk, struct sk_buff *skb);
+INDIRECT_CALLABLE_SCOPE int tcp_v6_do_rcv(struct sock *sk, struct sk_buff *skb);
static const struct inet_connection_sock_af_ops ipv6_mapped;
const struct inet_connection_sock_af_ops ipv6_specific;
@@ -1466,7 +1466,8 @@ INDIRECT_CALLABLE_DECLARE(struct dst_entry *ipv4_dst_check(struct dst_entry *,
* This is because we cannot sleep with the original spinlock
* held.
*/
-static int tcp_v6_do_rcv(struct sock *sk, struct sk_buff *skb)
+INDIRECT_CALLABLE_SCOPE
+int tcp_v6_do_rcv(struct sock *sk, struct sk_buff *skb)
{
struct ipv6_pinfo *np = tcp_inet6_sk(sk);
struct sk_buff *opt_skb = NULL;
@@ -1760,6 +1761,7 @@ INDIRECT_CALLABLE_SCOPE int tcp_v6_rcv(struct sk_buff *skb)
sk_incoming_cpu_update(sk);
+ sk_defer_free_flush(sk);
bh_lock_sock_nested(sk);
tcp_segs_in(tcp_sk(sk), skb);
ret = 0;
@@ -1896,9 +1898,7 @@ static struct timewait_sock_ops tcp6_timewait_sock_ops = {
INDIRECT_CALLABLE_SCOPE void tcp_v6_send_check(struct sock *sk, struct sk_buff *skb)
{
- struct ipv6_pinfo *np = inet6_sk(sk);
-
- __tcp_v6_send_check(skb, &np->saddr, &sk->sk_v6_daddr);
+ __tcp_v6_send_check(skb, &sk->sk_v6_rcv_saddr, &sk->sk_v6_daddr);
}
const struct inet_connection_sock_af_ops ipv6_specific = {
@@ -2181,6 +2181,7 @@ struct proto tcpv6_prot = {
.hash = inet6_hash,
.unhash = inet_unhash,
.get_port = inet_csk_get_port,
+ .put_port = inet_put_port,
#ifdef CONFIG_BPF_SYSCALL
.psock_update_sk_prot = tcp_bpf_update_proto,
#endif
diff --git a/net/ipv6/tcpv6_offload.c b/net/ipv6/tcpv6_offload.c
index 1796856..39db5a2 100644
--- a/net/ipv6/tcpv6_offload.c
+++ b/net/ipv6/tcpv6_offload.c
@@ -7,6 +7,7 @@
*/
#include <linux/indirect_call_wrapper.h>
#include <linux/skbuff.h>
+#include <net/gro.h>
#include <net/protocol.h>
#include <net/tcp.h>
#include <net/ip6_checksum.h>
diff --git a/net/ipv6/udp.c b/net/ipv6/udp.c
index a2caca6..528b81e 100644
--- a/net/ipv6/udp.c
+++ b/net/ipv6/udp.c
@@ -17,6 +17,7 @@
* YOSHIFUJI Hideaki @USAGI: convert /proc/net/udp6 to seq_file.
*/
+#include <linux/bpf-cgroup.h>
#include <linux/errno.h>
#include <linux/types.h>
#include <linux/socket.h>
@@ -40,6 +41,7 @@
#include <net/transp_v6.h>
#include <net/ip6_route.h>
#include <net/raw.h>
+#include <net/seg6.h>
#include <net/tcp_states.h>
#include <net/ip6_checksum.h>
#include <net/ip6_tunnel.h>
@@ -195,7 +197,7 @@ static inline struct sock *udp6_lookup_run_bpf(struct net *net,
const struct in6_addr *saddr,
__be16 sport,
const struct in6_addr *daddr,
- u16 hnum)
+ u16 hnum, const int dif)
{
struct sock *sk, *reuse_sk;
bool no_reuseport;
@@ -203,8 +205,8 @@ static inline struct sock *udp6_lookup_run_bpf(struct net *net,
if (udptable != &udp_table)
return NULL; /* only UDP is supported */
- no_reuseport = bpf_sk_lookup_run_v6(net, IPPROTO_UDP,
- saddr, sport, daddr, hnum, &sk);
+ no_reuseport = bpf_sk_lookup_run_v6(net, IPPROTO_UDP, saddr, sport,
+ daddr, hnum, dif, &sk);
if (no_reuseport || IS_ERR_OR_NULL(sk))
return sk;
@@ -240,7 +242,7 @@ struct sock *__udp6_lib_lookup(struct net *net,
/* Lookup redirect from BPF */
if (static_branch_unlikely(&bpf_sk_lookup_enabled)) {
sk = udp6_lookup_run_bpf(net, udptable, skb,
- saddr, sport, daddr, hnum);
+ saddr, sport, daddr, hnum, dif);
if (sk) {
result = sk;
goto done;
@@ -561,7 +563,7 @@ int __udp6_lib_err(struct sk_buff *skb, struct inet6_skb_parm *opt,
struct ipv6_pinfo *np;
const struct ipv6hdr *hdr = (const struct ipv6hdr *)skb->data;
const struct in6_addr *saddr = &hdr->saddr;
- const struct in6_addr *daddr = &hdr->daddr;
+ const struct in6_addr *daddr = seg6_get_daddr(skb, opt) ? : &hdr->daddr;
struct udphdr *uh = (struct udphdr *)(skb->data+offset);
bool tunnel = false;
struct sock *sk;
@@ -1204,7 +1206,7 @@ static int udp_v6_send_skb(struct sk_buff *skb, struct flowi6 *fl6,
kfree_skb(skb);
return -EINVAL;
}
- if (skb->len > cork->gso_size * UDP_MAX_SEGMENTS) {
+ if (datalen > cork->gso_size * UDP_MAX_SEGMENTS) {
kfree_skb(skb);
return -EINVAL;
}
@@ -1731,6 +1733,7 @@ struct proto udpv6_prot = {
.unhash = udp_lib_unhash,
.rehash = udp_v6_rehash,
.get_port = udp_v6_get_port,
+ .put_port = udp_lib_unhash,
#ifdef CONFIG_BPF_SYSCALL
.psock_update_sk_prot = udp_bpf_update_proto,
#endif
diff --git a/net/ipv6/udp_offload.c b/net/ipv6/udp_offload.c
index b3d9ed9..7720d04 100644
--- a/net/ipv6/udp_offload.c
+++ b/net/ipv6/udp_offload.c
@@ -13,6 +13,7 @@
#include <net/udp.h>
#include <net/ip6_checksum.h>
#include "ip6_offload.h"
+#include <net/gro.h>
static struct sk_buff *udp6_ufo_fragment(struct sk_buff *skb,
netdev_features_t features)
@@ -144,13 +145,11 @@ struct sk_buff *udp6_gro_receive(struct list_head *head, struct sk_buff *skb)
skip:
NAPI_GRO_CB(skb)->is_ipv6 = 1;
- rcu_read_lock();
if (static_branch_unlikely(&udpv6_encap_needed_key))
sk = udp6_gro_lookup_skb(skb, uh->source, uh->dest);
pp = udp_gro_receive(head, skb, uh, sk);
- rcu_read_unlock();
return pp;
flush:
diff --git a/net/ipv6/xfrm6_policy.c b/net/ipv6/xfrm6_policy.c
index af7a4b8..fad687e 100644
--- a/net/ipv6/xfrm6_policy.c
+++ b/net/ipv6/xfrm6_policy.c
@@ -74,11 +74,11 @@ static int xfrm6_fill_dst(struct xfrm_dst *xdst, struct net_device *dev,
struct rt6_info *rt = (struct rt6_info *)xdst->route;
xdst->u.dst.dev = dev;
- dev_hold(dev);
+ dev_hold_track(dev, &xdst->u.dst.dev_tracker, GFP_ATOMIC);
xdst->u.rt6.rt6i_idev = in6_dev_get(dev);
if (!xdst->u.rt6.rt6i_idev) {
- dev_put(dev);
+ dev_put_track(dev, &xdst->u.dst.dev_tracker);
return -ENODEV;
}
diff --git a/net/iucv/af_iucv.c b/net/iucv/af_iucv.c
index 18316ee..a1760ad 100644
--- a/net/iucv/af_iucv.c
+++ b/net/iucv/af_iucv.c
@@ -13,6 +13,7 @@
#define KMSG_COMPONENT "af_iucv"
#define pr_fmt(fmt) KMSG_COMPONENT ": " fmt
+#include <linux/filter.h>
#include <linux/module.h>
#include <linux/netdevice.h>
#include <linux/types.h>
@@ -142,7 +143,7 @@ static inline size_t iucv_msg_length(struct iucv_message *msg)
* iucv_sock_in_state() - check for specific states
* @sk: sock structure
* @state: first iucv sk state
- * @state: second iucv sk state
+ * @state2: second iucv sk state
*
* Returns true if the socket in either in the first or second state.
*/
@@ -172,7 +173,7 @@ static inline int iucv_below_msglim(struct sock *sk)
(atomic_read(&iucv->pendings) <= 0));
}
-/**
+/*
* iucv_sock_wake_msglim() - Wake up thread waiting on msg limit
*/
static void iucv_sock_wake_msglim(struct sock *sk)
@@ -187,7 +188,7 @@ static void iucv_sock_wake_msglim(struct sock *sk)
rcu_read_unlock();
}
-/**
+/*
* afiucv_hs_send() - send a message through HiperSockets transport
*/
static int afiucv_hs_send(struct iucv_message *imsg, struct sock *sock,
@@ -473,7 +474,7 @@ static struct sock *iucv_sock_alloc(struct socket *sock, int proto, gfp_t prio,
atomic_set(&iucv->msg_recv, 0);
iucv->path = NULL;
iucv->sk_txnotify = afiucv_hs_callback_txnotify;
- memset(&iucv->src_user_id , 0, 32);
+ memset(&iucv->init, 0, sizeof(iucv->init));
if (pr_iucv)
iucv->transport = AF_IUCV_TRANS_IUCV;
else
@@ -1831,9 +1832,9 @@ static void afiucv_swap_src_dest(struct sk_buff *skb)
memset(skb->data, 0, ETH_HLEN);
}
-/**
+/*
* afiucv_hs_callback_syn - react on received SYN
- **/
+ */
static int afiucv_hs_callback_syn(struct sock *sk, struct sk_buff *skb)
{
struct af_iucv_trans_hdr *trans_hdr = iucv_trans_hdr(skb);
@@ -1896,9 +1897,9 @@ static int afiucv_hs_callback_syn(struct sock *sk, struct sk_buff *skb)
return NET_RX_SUCCESS;
}
-/**
+/*
* afiucv_hs_callback_synack() - react on received SYN-ACK
- **/
+ */
static int afiucv_hs_callback_synack(struct sock *sk, struct sk_buff *skb)
{
struct iucv_sock *iucv = iucv_sk(sk);
@@ -1917,9 +1918,9 @@ static int afiucv_hs_callback_synack(struct sock *sk, struct sk_buff *skb)
return NET_RX_SUCCESS;
}
-/**
+/*
* afiucv_hs_callback_synfin() - react on received SYN_FIN
- **/
+ */
static int afiucv_hs_callback_synfin(struct sock *sk, struct sk_buff *skb)
{
struct iucv_sock *iucv = iucv_sk(sk);
@@ -1937,9 +1938,9 @@ static int afiucv_hs_callback_synfin(struct sock *sk, struct sk_buff *skb)
return NET_RX_SUCCESS;
}
-/**
+/*
* afiucv_hs_callback_fin() - react on received FIN
- **/
+ */
static int afiucv_hs_callback_fin(struct sock *sk, struct sk_buff *skb)
{
struct iucv_sock *iucv = iucv_sk(sk);
@@ -1960,9 +1961,9 @@ static int afiucv_hs_callback_fin(struct sock *sk, struct sk_buff *skb)
return NET_RX_SUCCESS;
}
-/**
+/*
* afiucv_hs_callback_win() - react on received WIN
- **/
+ */
static int afiucv_hs_callback_win(struct sock *sk, struct sk_buff *skb)
{
struct iucv_sock *iucv = iucv_sk(sk);
@@ -1978,9 +1979,9 @@ static int afiucv_hs_callback_win(struct sock *sk, struct sk_buff *skb)
return NET_RX_SUCCESS;
}
-/**
+/*
* afiucv_hs_callback_rx() - react on received data
- **/
+ */
static int afiucv_hs_callback_rx(struct sock *sk, struct sk_buff *skb)
{
struct iucv_sock *iucv = iucv_sk(sk);
@@ -2022,11 +2023,11 @@ static int afiucv_hs_callback_rx(struct sock *sk, struct sk_buff *skb)
return NET_RX_SUCCESS;
}
-/**
+/*
* afiucv_hs_rcv() - base function for arriving data through HiperSockets
* transport
* called from netif RX softirq
- **/
+ */
static int afiucv_hs_rcv(struct sk_buff *skb, struct net_device *dev,
struct packet_type *pt, struct net_device *orig_dev)
{
@@ -2128,10 +2129,10 @@ static int afiucv_hs_rcv(struct sk_buff *skb, struct net_device *dev,
return err;
}
-/**
+/*
* afiucv_hs_callback_txnotify() - handle send notifications from HiperSockets
* transport
- **/
+ */
static void afiucv_hs_callback_txnotify(struct sock *sk, enum iucv_tx_notify n)
{
struct iucv_sock *iucv = iucv_sk(sk);
diff --git a/net/iucv/iucv.c b/net/iucv/iucv.c
index f3343a8..8f4d49a 100644
--- a/net/iucv/iucv.c
+++ b/net/iucv/iucv.c
@@ -276,8 +276,8 @@ static union iucv_param *iucv_param[NR_CPUS];
static union iucv_param *iucv_param_irq[NR_CPUS];
/**
- * iucv_call_b2f0
- * @code: identifier of IUCV call to CP.
+ * __iucv_call_b2f0
+ * @command: identifier of IUCV call to CP.
* @parm: pointer to a struct iucv_parm block
*
* Calls CP to execute IUCV commands.
@@ -309,7 +309,7 @@ static inline int iucv_call_b2f0(int command, union iucv_param *parm)
return ccode == 1 ? parm->ctrl.iprcode : ccode;
}
-/**
+/*
* iucv_query_maxconn
*
* Determines the maximum number of connections that may be established.
@@ -493,8 +493,8 @@ static void iucv_retrieve_cpu(void *data)
cpumask_clear_cpu(cpu, &iucv_buffer_cpumask);
}
-/**
- * iucv_setmask_smp
+/*
+ * iucv_setmask_mp
*
* Allow iucv interrupts on all cpus.
*/
@@ -512,7 +512,7 @@ static void iucv_setmask_mp(void)
cpus_read_unlock();
}
-/**
+/*
* iucv_setmask_up
*
* Allow iucv interrupts on a single cpu.
@@ -529,7 +529,7 @@ static void iucv_setmask_up(void)
smp_call_function_single(cpu, iucv_block_cpu, NULL, 1);
}
-/**
+/*
* iucv_enable
*
* This function makes iucv ready for use. It allocates the pathid
@@ -564,7 +564,7 @@ static int iucv_enable(void)
return rc;
}
-/**
+/*
* iucv_disable
*
* This function shuts down iucv. It disables iucv interrupts, retrieves
@@ -1347,8 +1347,9 @@ EXPORT_SYMBOL(iucv_message_send);
* @srccls: source class of message
* @buffer: address of send buffer or address of struct iucv_array
* @size: length of send buffer
- * @ansbuf: address of answer buffer or address of struct iucv_array
+ * @answer: address of answer buffer or address of struct iucv_array
* @asize: size of reply buffer
+ * @residual: ignored
*
* This function transmits data to another application. Data to be
* transmitted is in a buffer. The receiver of the send is expected to
@@ -1400,13 +1401,6 @@ int iucv_message_send2way(struct iucv_path *path, struct iucv_message *msg,
}
EXPORT_SYMBOL(iucv_message_send2way);
-/**
- * iucv_path_pending
- * @data: Pointer to external interrupt buffer
- *
- * Process connection pending work item. Called from tasklet while holding
- * iucv_table_lock.
- */
struct iucv_path_pending {
u16 ippathid;
u8 ipflags1;
@@ -1420,6 +1414,13 @@ struct iucv_path_pending {
u8 res4[3];
} __packed;
+/**
+ * iucv_path_pending
+ * @data: Pointer to external interrupt buffer
+ *
+ * Process connection pending work item. Called from tasklet while holding
+ * iucv_table_lock.
+ */
static void iucv_path_pending(struct iucv_irq_data *data)
{
struct iucv_path_pending *ipp = (void *) data;
@@ -1461,13 +1462,6 @@ static void iucv_path_pending(struct iucv_irq_data *data)
iucv_sever_pathid(ipp->ippathid, error);
}
-/**
- * iucv_path_complete
- * @data: Pointer to external interrupt buffer
- *
- * Process connection complete work item. Called from tasklet while holding
- * iucv_table_lock.
- */
struct iucv_path_complete {
u16 ippathid;
u8 ipflags1;
@@ -1481,6 +1475,13 @@ struct iucv_path_complete {
u8 res4[3];
} __packed;
+/**
+ * iucv_path_complete
+ * @data: Pointer to external interrupt buffer
+ *
+ * Process connection complete work item. Called from tasklet while holding
+ * iucv_table_lock.
+ */
static void iucv_path_complete(struct iucv_irq_data *data)
{
struct iucv_path_complete *ipc = (void *) data;
@@ -1492,13 +1493,6 @@ static void iucv_path_complete(struct iucv_irq_data *data)
path->handler->path_complete(path, ipc->ipuser);
}
-/**
- * iucv_path_severed
- * @data: Pointer to external interrupt buffer
- *
- * Process connection severed work item. Called from tasklet while holding
- * iucv_table_lock.
- */
struct iucv_path_severed {
u16 ippathid;
u8 res1;
@@ -1511,6 +1505,13 @@ struct iucv_path_severed {
u8 res5[3];
} __packed;
+/**
+ * iucv_path_severed
+ * @data: Pointer to external interrupt buffer
+ *
+ * Process connection severed work item. Called from tasklet while holding
+ * iucv_table_lock.
+ */
static void iucv_path_severed(struct iucv_irq_data *data)
{
struct iucv_path_severed *ips = (void *) data;
@@ -1528,13 +1529,6 @@ static void iucv_path_severed(struct iucv_irq_data *data)
}
}
-/**
- * iucv_path_quiesced
- * @data: Pointer to external interrupt buffer
- *
- * Process connection quiesced work item. Called from tasklet while holding
- * iucv_table_lock.
- */
struct iucv_path_quiesced {
u16 ippathid;
u8 res1;
@@ -1547,6 +1541,13 @@ struct iucv_path_quiesced {
u8 res5[3];
} __packed;
+/**
+ * iucv_path_quiesced
+ * @data: Pointer to external interrupt buffer
+ *
+ * Process connection quiesced work item. Called from tasklet while holding
+ * iucv_table_lock.
+ */
static void iucv_path_quiesced(struct iucv_irq_data *data)
{
struct iucv_path_quiesced *ipq = (void *) data;
@@ -1556,13 +1557,6 @@ static void iucv_path_quiesced(struct iucv_irq_data *data)
path->handler->path_quiesced(path, ipq->ipuser);
}
-/**
- * iucv_path_resumed
- * @data: Pointer to external interrupt buffer
- *
- * Process connection resumed work item. Called from tasklet while holding
- * iucv_table_lock.
- */
struct iucv_path_resumed {
u16 ippathid;
u8 res1;
@@ -1575,6 +1569,13 @@ struct iucv_path_resumed {
u8 res5[3];
} __packed;
+/**
+ * iucv_path_resumed
+ * @data: Pointer to external interrupt buffer
+ *
+ * Process connection resumed work item. Called from tasklet while holding
+ * iucv_table_lock.
+ */
static void iucv_path_resumed(struct iucv_irq_data *data)
{
struct iucv_path_resumed *ipr = (void *) data;
@@ -1584,13 +1585,6 @@ static void iucv_path_resumed(struct iucv_irq_data *data)
path->handler->path_resumed(path, ipr->ipuser);
}
-/**
- * iucv_message_complete
- * @data: Pointer to external interrupt buffer
- *
- * Process message complete work item. Called from tasklet while holding
- * iucv_table_lock.
- */
struct iucv_message_complete {
u16 ippathid;
u8 ipflags1;
@@ -1606,6 +1600,13 @@ struct iucv_message_complete {
u8 res2[3];
} __packed;
+/**
+ * iucv_message_complete
+ * @data: Pointer to external interrupt buffer
+ *
+ * Process message complete work item. Called from tasklet while holding
+ * iucv_table_lock.
+ */
static void iucv_message_complete(struct iucv_irq_data *data)
{
struct iucv_message_complete *imc = (void *) data;
@@ -1624,13 +1625,6 @@ static void iucv_message_complete(struct iucv_irq_data *data)
}
}
-/**
- * iucv_message_pending
- * @data: Pointer to external interrupt buffer
- *
- * Process message pending work item. Called from tasklet while holding
- * iucv_table_lock.
- */
struct iucv_message_pending {
u16 ippathid;
u8 ipflags1;
@@ -1653,6 +1647,13 @@ struct iucv_message_pending {
u8 res2[3];
} __packed;
+/**
+ * iucv_message_pending
+ * @data: Pointer to external interrupt buffer
+ *
+ * Process message pending work item. Called from tasklet while holding
+ * iucv_table_lock.
+ */
static void iucv_message_pending(struct iucv_irq_data *data)
{
struct iucv_message_pending *imp = (void *) data;
@@ -1673,7 +1674,7 @@ static void iucv_message_pending(struct iucv_irq_data *data)
}
}
-/**
+/*
* iucv_tasklet_fn:
*
* This tasklet loops over the queue of irq buffers created by
@@ -1717,7 +1718,7 @@ static void iucv_tasklet_fn(unsigned long ignored)
spin_unlock(&iucv_table_lock);
}
-/**
+/*
* iucv_work_fn:
*
* This work function loops over the queue of path pending irq blocks
@@ -1748,9 +1749,8 @@ static void iucv_work_fn(struct work_struct *work)
spin_unlock_bh(&iucv_table_lock);
}
-/**
+/*
* iucv_external_interrupt
- * @code: irq code
*
* Handles external interrupts coming in from CP.
* Places the interrupt buffer on a queue and schedules iucv_tasklet_fn().
diff --git a/net/kcm/kcmsock.c b/net/kcm/kcmsock.c
index 11a715d..71899e5 100644
--- a/net/kcm/kcmsock.c
+++ b/net/kcm/kcmsock.c
@@ -9,6 +9,7 @@
#include <linux/errno.h>
#include <linux/errqueue.h>
#include <linux/file.h>
+#include <linux/filter.h>
#include <linux/in.h>
#include <linux/kernel.h>
#include <linux/module.h>
diff --git a/net/l2tp/l2tp_core.c b/net/l2tp/l2tp_core.c
index 93271a2..7499c51 100644
--- a/net/l2tp/l2tp_core.c
+++ b/net/l2tp/l2tp_core.c
@@ -250,15 +250,15 @@ struct l2tp_session *l2tp_tunnel_get_session(struct l2tp_tunnel *tunnel,
session_list = l2tp_session_id_hash(tunnel, session_id);
- read_lock_bh(&tunnel->hlist_lock);
- hlist_for_each_entry(session, session_list, hlist)
+ rcu_read_lock_bh();
+ hlist_for_each_entry_rcu(session, session_list, hlist)
if (session->session_id == session_id) {
l2tp_session_inc_refcount(session);
- read_unlock_bh(&tunnel->hlist_lock);
+ rcu_read_unlock_bh();
return session;
}
- read_unlock_bh(&tunnel->hlist_lock);
+ rcu_read_unlock_bh();
return NULL;
}
@@ -291,18 +291,18 @@ struct l2tp_session *l2tp_session_get_nth(struct l2tp_tunnel *tunnel, int nth)
struct l2tp_session *session;
int count = 0;
- read_lock_bh(&tunnel->hlist_lock);
+ rcu_read_lock_bh();
for (hash = 0; hash < L2TP_HASH_SIZE; hash++) {
- hlist_for_each_entry(session, &tunnel->session_hlist[hash], hlist) {
+ hlist_for_each_entry_rcu(session, &tunnel->session_hlist[hash], hlist) {
if (++count > nth) {
l2tp_session_inc_refcount(session);
- read_unlock_bh(&tunnel->hlist_lock);
+ rcu_read_unlock_bh();
return session;
}
}
}
- read_unlock_bh(&tunnel->hlist_lock);
+ rcu_read_unlock_bh();
return NULL;
}
@@ -347,7 +347,7 @@ int l2tp_session_register(struct l2tp_session *session,
head = l2tp_session_id_hash(tunnel, session->session_id);
- write_lock_bh(&tunnel->hlist_lock);
+ spin_lock_bh(&tunnel->hlist_lock);
if (!tunnel->acpt_newsess) {
err = -ENODEV;
goto err_tlock;
@@ -384,8 +384,8 @@ int l2tp_session_register(struct l2tp_session *session,
l2tp_tunnel_inc_refcount(tunnel);
}
- hlist_add_head(&session->hlist, head);
- write_unlock_bh(&tunnel->hlist_lock);
+ hlist_add_head_rcu(&session->hlist, head);
+ spin_unlock_bh(&tunnel->hlist_lock);
trace_register_session(session);
@@ -394,7 +394,7 @@ int l2tp_session_register(struct l2tp_session *session,
err_tlock_pnlock:
spin_unlock_bh(&pn->l2tp_session_hlist_lock);
err_tlock:
- write_unlock_bh(&tunnel->hlist_lock);
+ spin_unlock_bh(&tunnel->hlist_lock);
return err;
}
@@ -1170,9 +1170,9 @@ static void l2tp_session_unhash(struct l2tp_session *session)
/* Remove the session from core hashes */
if (tunnel) {
/* Remove from the per-tunnel hash */
- write_lock_bh(&tunnel->hlist_lock);
- hlist_del_init(&session->hlist);
- write_unlock_bh(&tunnel->hlist_lock);
+ spin_lock_bh(&tunnel->hlist_lock);
+ hlist_del_init_rcu(&session->hlist);
+ spin_unlock_bh(&tunnel->hlist_lock);
/* For L2TPv3 we have a per-net hash: remove from there, too */
if (tunnel->version != L2TP_HDR_VER_2) {
@@ -1181,8 +1181,9 @@ static void l2tp_session_unhash(struct l2tp_session *session)
spin_lock_bh(&pn->l2tp_session_hlist_lock);
hlist_del_init_rcu(&session->global_hlist);
spin_unlock_bh(&pn->l2tp_session_hlist_lock);
- synchronize_rcu();
}
+
+ synchronize_rcu();
}
}
@@ -1190,22 +1191,19 @@ static void l2tp_session_unhash(struct l2tp_session *session)
*/
static void l2tp_tunnel_closeall(struct l2tp_tunnel *tunnel)
{
- int hash;
- struct hlist_node *walk;
- struct hlist_node *tmp;
struct l2tp_session *session;
+ int hash;
- write_lock_bh(&tunnel->hlist_lock);
+ spin_lock_bh(&tunnel->hlist_lock);
tunnel->acpt_newsess = false;
for (hash = 0; hash < L2TP_HASH_SIZE; hash++) {
again:
- hlist_for_each_safe(walk, tmp, &tunnel->session_hlist[hash]) {
- session = hlist_entry(walk, struct l2tp_session, hlist);
- hlist_del_init(&session->hlist);
+ hlist_for_each_entry_rcu(session, &tunnel->session_hlist[hash], hlist) {
+ hlist_del_init_rcu(&session->hlist);
- write_unlock_bh(&tunnel->hlist_lock);
+ spin_unlock_bh(&tunnel->hlist_lock);
l2tp_session_delete(session);
- write_lock_bh(&tunnel->hlist_lock);
+ spin_lock_bh(&tunnel->hlist_lock);
/* Now restart from the beginning of this hash
* chain. We always remove a session from the
@@ -1215,7 +1213,7 @@ static void l2tp_tunnel_closeall(struct l2tp_tunnel *tunnel)
goto again;
}
}
- write_unlock_bh(&tunnel->hlist_lock);
+ spin_unlock_bh(&tunnel->hlist_lock);
}
/* Tunnel socket destroy hook for UDP encapsulation */
@@ -1408,7 +1406,7 @@ int l2tp_tunnel_create(int fd, int version, u32 tunnel_id, u32 peer_tunnel_id,
tunnel->magic = L2TP_TUNNEL_MAGIC;
sprintf(&tunnel->name[0], "tunl %u", tunnel_id);
- rwlock_init(&tunnel->hlist_lock);
+ spin_lock_init(&tunnel->hlist_lock);
tunnel->acpt_newsess = true;
tunnel->encap = encap;
diff --git a/net/l2tp/l2tp_core.h b/net/l2tp/l2tp_core.h
index 98ea98e..a88e070 100644
--- a/net/l2tp/l2tp_core.h
+++ b/net/l2tp/l2tp_core.h
@@ -160,7 +160,7 @@ struct l2tp_tunnel {
unsigned long dead;
struct rcu_head rcu;
- rwlock_t hlist_lock; /* protect session_hlist */
+ spinlock_t hlist_lock; /* write-protection for session_hlist */
bool acpt_newsess; /* indicates whether this tunnel accepts
* new sessions. Protected by hlist_lock.
*/
diff --git a/net/l2tp/l2tp_debugfs.c b/net/l2tp/l2tp_debugfs.c
index bca75be..9d1aafe7 100644
--- a/net/l2tp/l2tp_debugfs.c
+++ b/net/l2tp/l2tp_debugfs.c
@@ -32,7 +32,8 @@
static struct dentry *rootdir;
struct l2tp_dfs_seq_data {
- struct net *net;
+ struct net *net;
+ netns_tracker ns_tracker;
int tunnel_idx; /* current tunnel */
int session_idx; /* index of session within current tunnel */
struct l2tp_tunnel *tunnel;
@@ -120,24 +121,21 @@ static void l2tp_dfs_seq_stop(struct seq_file *p, void *v)
static void l2tp_dfs_seq_tunnel_show(struct seq_file *m, void *v)
{
struct l2tp_tunnel *tunnel = v;
+ struct l2tp_session *session;
int session_count = 0;
int hash;
- struct hlist_node *walk;
- struct hlist_node *tmp;
- read_lock_bh(&tunnel->hlist_lock);
+ rcu_read_lock_bh();
for (hash = 0; hash < L2TP_HASH_SIZE; hash++) {
- hlist_for_each_safe(walk, tmp, &tunnel->session_hlist[hash]) {
- struct l2tp_session *session;
-
- session = hlist_entry(walk, struct l2tp_session, hlist);
+ hlist_for_each_entry_rcu(session, &tunnel->session_hlist[hash], hlist) {
+ /* Session ID of zero is a dummy/reserved value used by pppol2tp */
if (session->session_id == 0)
continue;
session_count++;
}
}
- read_unlock_bh(&tunnel->hlist_lock);
+ rcu_read_unlock_bh();
seq_printf(m, "\nTUNNEL %u peer %u", tunnel->tunnel_id, tunnel->peer_tunnel_id);
if (tunnel->sock) {
@@ -284,7 +282,7 @@ static int l2tp_dfs_seq_open(struct inode *inode, struct file *file)
rc = PTR_ERR(pd->net);
goto err_free_pd;
}
-
+ netns_tracker_alloc(pd->net, &pd->ns_tracker, GFP_KERNEL);
rc = seq_open(file, &l2tp_dfs_seq_ops);
if (rc)
goto err_free_net;
@@ -296,7 +294,7 @@ static int l2tp_dfs_seq_open(struct inode *inode, struct file *file)
return rc;
err_free_net:
- put_net(pd->net);
+ put_net_track(pd->net, &pd->ns_tracker);
err_free_pd:
kfree(pd);
goto out;
@@ -310,7 +308,7 @@ static int l2tp_dfs_seq_release(struct inode *inode, struct file *file)
seq = file->private_data;
pd = seq->private;
if (pd->net)
- put_net(pd->net);
+ put_net_track(pd->net, &pd->ns_tracker);
kfree(pd);
seq_release(inode, file);
diff --git a/net/llc/af_llc.c b/net/llc/af_llc.c
index 3086f4a..26c00eb 100644
--- a/net/llc/af_llc.c
+++ b/net/llc/af_llc.c
@@ -224,7 +224,7 @@ static int llc_ui_release(struct socket *sock)
} else {
release_sock(sk);
}
- dev_put(llc->dev);
+ dev_put_track(llc->dev, &llc->dev_tracker);
sock_put(sk);
llc_sk_free(sk);
out:
@@ -295,6 +295,7 @@ static int llc_ui_autobind(struct socket *sock, struct sockaddr_llc *addr)
llc->dev = dev_getfirstbyhwtype(&init_net, addr->sllc_arphrd);
if (!llc->dev)
goto out;
+ netdev_tracker_alloc(llc->dev, &llc->dev_tracker, GFP_KERNEL);
rc = -EUSERS;
llc->laddr.lsap = llc_ui_autoport();
if (!llc->laddr.lsap)
@@ -362,7 +363,7 @@ static int llc_ui_bind(struct socket *sock, struct sockaddr *uaddr, int addrlen)
} else
llc->dev = dev_getbyhwaddr_rcu(&init_net, addr->sllc_arphrd,
addr->sllc_mac);
- dev_hold(llc->dev);
+ dev_hold_track(llc->dev, &llc->dev_tracker, GFP_ATOMIC);
rcu_read_unlock();
if (!llc->dev)
goto out;
diff --git a/net/llc/llc_proc.c b/net/llc/llc_proc.c
index 0ff490a..07e9abb 100644
--- a/net/llc/llc_proc.c
+++ b/net/llc/llc_proc.c
@@ -195,7 +195,7 @@ static int llc_seq_core_show(struct seq_file *seq, void *v)
timer_pending(&llc->pf_cycle_timer.timer),
timer_pending(&llc->rej_sent_timer.timer),
timer_pending(&llc->busy_state_timer.timer),
- !!sk->sk_backlog.tail, !!sk->sk_lock.owned);
+ !!sk->sk_backlog.tail, sock_owned_by_user_nocheck(sk));
out:
return 0;
}
diff --git a/net/mac80211/cfg.c b/net/mac80211/cfg.c
index 2d0dd69..87a2080 100644
--- a/net/mac80211/cfg.c
+++ b/net/mac80211/cfg.c
@@ -5,7 +5,7 @@
* Copyright 2006-2010 Johannes Berg <johannes@sipsolutions.net>
* Copyright 2013-2015 Intel Mobile Communications GmbH
* Copyright (C) 2015-2017 Intel Deutschland GmbH
- * Copyright (C) 2018-2020 Intel Corporation
+ * Copyright (C) 2018-2021 Intel Corporation
*/
#include <linux/ieee80211.h>
@@ -3201,6 +3201,18 @@ void ieee80211_csa_finish(struct ieee80211_vif *vif)
}
EXPORT_SYMBOL(ieee80211_csa_finish);
+void ieee80211_channel_switch_disconnect(struct ieee80211_vif *vif, bool block_tx)
+{
+ struct ieee80211_sub_if_data *sdata = vif_to_sdata(vif);
+ struct ieee80211_if_managed *ifmgd = &sdata->u.mgd;
+ struct ieee80211_local *local = sdata->local;
+
+ sdata->csa_block_tx = block_tx;
+ sdata_info(sdata, "channel switch failed, disconnecting\n");
+ ieee80211_queue_work(&local->hw, &ifmgd->csa_connection_drop_work);
+}
+EXPORT_SYMBOL(ieee80211_channel_switch_disconnect);
+
static int ieee80211_set_after_csa_beacon(struct ieee80211_sub_if_data *sdata,
u32 *changed)
{
@@ -4271,6 +4283,21 @@ ieee80211_color_change_bss_config_notify(struct ieee80211_sub_if_data *sdata,
changed |= BSS_CHANGED_HE_BSS_COLOR;
ieee80211_bss_info_change_notify(sdata, changed);
+
+ if (!sdata->vif.bss_conf.nontransmitted && sdata->vif.mbssid_tx_vif) {
+ struct ieee80211_sub_if_data *child;
+
+ mutex_lock(&sdata->local->iflist_mtx);
+ list_for_each_entry(child, &sdata->local->interfaces, list) {
+ if (child != sdata && child->vif.mbssid_tx_vif == &sdata->vif) {
+ child->vif.bss_conf.he_bss_color.color = color;
+ child->vif.bss_conf.he_bss_color.enabled = enable;
+ ieee80211_bss_info_change_notify(child,
+ BSS_CHANGED_HE_BSS_COLOR);
+ }
+ }
+ mutex_unlock(&sdata->local->iflist_mtx);
+ }
}
static int ieee80211_color_change_finalize(struct ieee80211_sub_if_data *sdata)
@@ -4355,6 +4382,9 @@ ieee80211_color_change(struct wiphy *wiphy, struct net_device *dev,
sdata_assert_lock(sdata);
+ if (sdata->vif.bss_conf.nontransmitted)
+ return -EINVAL;
+
mutex_lock(&local->mtx);
/* don't allow another color change if one is already active or if csa
@@ -4386,6 +4416,18 @@ ieee80211_color_change(struct wiphy *wiphy, struct net_device *dev,
return err;
}
+static int
+ieee80211_set_radar_background(struct wiphy *wiphy,
+ struct cfg80211_chan_def *chandef)
+{
+ struct ieee80211_local *local = wiphy_priv(wiphy);
+
+ if (!local->ops->set_radar_background)
+ return -EOPNOTSUPP;
+
+ return local->ops->set_radar_background(&local->hw, chandef);
+}
+
const struct cfg80211_ops mac80211_config_ops = {
.add_virtual_intf = ieee80211_add_iface,
.del_virtual_intf = ieee80211_del_iface,
@@ -4490,4 +4532,5 @@ const struct cfg80211_ops mac80211_config_ops = {
.reset_tid_config = ieee80211_reset_tid_config,
.set_sar_specs = ieee80211_set_sar_specs,
.color_change = ieee80211_color_change,
+ .set_radar_background = ieee80211_set_radar_background,
};
diff --git a/net/mac80211/debugfs_sta.c b/net/mac80211/debugfs_sta.c
index 481f01b..9479f27 100644
--- a/net/mac80211/debugfs_sta.c
+++ b/net/mac80211/debugfs_sta.c
@@ -936,14 +936,15 @@ static ssize_t sta_he_capa_read(struct file *file, char __user *userbuf,
PFLAG(PHY, 9, RX_FULL_BW_SU_USING_MU_WITH_NON_COMP_SIGB,
"RX-FULL-BW-SU-USING-MU-WITH-NON-COMP-SIGB");
- switch (cap[9] & IEEE80211_HE_PHY_CAP9_NOMIMAL_PKT_PADDING_MASK) {
- case IEEE80211_HE_PHY_CAP9_NOMIMAL_PKT_PADDING_0US:
+ switch (u8_get_bits(cap[9],
+ IEEE80211_HE_PHY_CAP9_NOMINAL_PKT_PADDING_MASK)) {
+ case IEEE80211_HE_PHY_CAP9_NOMINAL_PKT_PADDING_0US:
PRINT("NOMINAL-PACKET-PADDING-0US");
break;
- case IEEE80211_HE_PHY_CAP9_NOMIMAL_PKT_PADDING_8US:
+ case IEEE80211_HE_PHY_CAP9_NOMINAL_PKT_PADDING_8US:
PRINT("NOMINAL-PACKET-PADDING-8US");
break;
- case IEEE80211_HE_PHY_CAP9_NOMIMAL_PKT_PADDING_16US:
+ case IEEE80211_HE_PHY_CAP9_NOMINAL_PKT_PADDING_16US:
PRINT("NOMINAL-PACKET-PADDING-16US");
break;
}
diff --git a/net/mac80211/driver-ops.h b/net/mac80211/driver-ops.h
index c336267..4e2fc1a 100644
--- a/net/mac80211/driver-ops.h
+++ b/net/mac80211/driver-ops.h
@@ -1486,4 +1486,26 @@ static inline void drv_twt_teardown_request(struct ieee80211_local *local,
trace_drv_return_void(local);
}
+static inline int drv_net_fill_forward_path(struct ieee80211_local *local,
+ struct ieee80211_sub_if_data *sdata,
+ struct ieee80211_sta *sta,
+ struct net_device_path_ctx *ctx,
+ struct net_device_path *path)
+{
+ int ret = -EOPNOTSUPP;
+
+ sdata = get_bss_sdata(sdata);
+ if (!check_sdata_in_driver(sdata))
+ return -EIO;
+
+ trace_drv_net_fill_forward_path(local, sdata, sta);
+ if (local->ops->net_fill_forward_path)
+ ret = local->ops->net_fill_forward_path(&local->hw,
+ &sdata->vif, sta,
+ ctx, path);
+ trace_drv_return_int(local, ret);
+
+ return ret;
+}
+
#endif /* __MAC80211_DRIVER_OPS */
diff --git a/net/mac80211/ethtool.c b/net/mac80211/ethtool.c
index 99a2e30..b2253df 100644
--- a/net/mac80211/ethtool.c
+++ b/net/mac80211/ethtool.c
@@ -14,7 +14,9 @@
#include "driver-ops.h"
static int ieee80211_set_ringparam(struct net_device *dev,
- struct ethtool_ringparam *rp)
+ struct ethtool_ringparam *rp,
+ struct kernel_ethtool_ringparam *kernel_rp,
+ struct netlink_ext_ack *extack)
{
struct ieee80211_local *local = wiphy_priv(dev->ieee80211_ptr->wiphy);
@@ -25,7 +27,9 @@ static int ieee80211_set_ringparam(struct net_device *dev,
}
static void ieee80211_get_ringparam(struct net_device *dev,
- struct ethtool_ringparam *rp)
+ struct ethtool_ringparam *rp,
+ struct kernel_ethtool_ringparam *kernel_rp,
+ struct netlink_ext_ack *extack)
{
struct ieee80211_local *local = wiphy_priv(dev->ieee80211_ptr->wiphy);
diff --git a/net/mac80211/ieee80211_i.h b/net/mac80211/ieee80211_i.h
index 5666bbb..330ea62 100644
--- a/net/mac80211/ieee80211_i.h
+++ b/net/mac80211/ieee80211_i.h
@@ -647,6 +647,26 @@ struct mesh_csa_settings {
struct cfg80211_csa_settings settings;
};
+/**
+ * struct mesh_table
+ *
+ * @known_gates: list of known mesh gates and their mpaths by the station. The
+ * gate's mpath may or may not be resolved and active.
+ * @gates_lock: protects updates to known_gates
+ * @rhead: the rhashtable containing struct mesh_paths, keyed by dest addr
+ * @walk_head: linked list containing all mesh_path objects
+ * @walk_lock: lock protecting walk_head
+ * @entries: number of entries in the table
+ */
+struct mesh_table {
+ struct hlist_head known_gates;
+ spinlock_t gates_lock;
+ struct rhashtable rhead;
+ struct hlist_head walk_head;
+ spinlock_t walk_lock;
+ atomic_t entries; /* Up to MAX_MESH_NEIGHBOURS */
+};
+
struct ieee80211_if_mesh {
struct timer_list housekeeping_timer;
struct timer_list mesh_path_timer;
@@ -721,8 +741,8 @@ struct ieee80211_if_mesh {
/* offset from skb->data while building IE */
int meshconf_offset;
- struct mesh_table *mesh_paths;
- struct mesh_table *mpp_paths; /* Store paths for MPP&MAP */
+ struct mesh_table mesh_paths;
+ struct mesh_table mpp_paths; /* Store paths for MPP&MAP */
int mesh_paths_generation;
int mpp_paths_generation;
};
@@ -1463,7 +1483,7 @@ struct ieee80211_local {
};
static inline struct ieee80211_sub_if_data *
-IEEE80211_DEV_TO_SUB_IF(struct net_device *dev)
+IEEE80211_DEV_TO_SUB_IF(const struct net_device *dev)
{
return netdev_priv(dev);
}
diff --git a/net/mac80211/iface.c b/net/mac80211/iface.c
index 20aa5cc..4153147 100644
--- a/net/mac80211/iface.c
+++ b/net/mac80211/iface.c
@@ -789,6 +789,64 @@ static const struct net_device_ops ieee80211_monitorif_ops = {
.ndo_get_stats64 = ieee80211_get_stats64,
};
+static int ieee80211_netdev_fill_forward_path(struct net_device_path_ctx *ctx,
+ struct net_device_path *path)
+{
+ struct ieee80211_sub_if_data *sdata;
+ struct ieee80211_local *local;
+ struct sta_info *sta;
+ int ret = -ENOENT;
+
+ sdata = IEEE80211_DEV_TO_SUB_IF(ctx->dev);
+ local = sdata->local;
+
+ if (!local->ops->net_fill_forward_path)
+ return -EOPNOTSUPP;
+
+ rcu_read_lock();
+ switch (sdata->vif.type) {
+ case NL80211_IFTYPE_AP_VLAN:
+ sta = rcu_dereference(sdata->u.vlan.sta);
+ if (sta)
+ break;
+ if (sdata->wdev.use_4addr)
+ goto out;
+ if (is_multicast_ether_addr(ctx->daddr))
+ goto out;
+ sta = sta_info_get_bss(sdata, ctx->daddr);
+ break;
+ case NL80211_IFTYPE_AP:
+ if (is_multicast_ether_addr(ctx->daddr))
+ goto out;
+ sta = sta_info_get(sdata, ctx->daddr);
+ break;
+ case NL80211_IFTYPE_STATION:
+ if (sdata->wdev.wiphy->flags & WIPHY_FLAG_SUPPORTS_TDLS) {
+ sta = sta_info_get(sdata, ctx->daddr);
+ if (sta && test_sta_flag(sta, WLAN_STA_TDLS_PEER)) {
+ if (!test_sta_flag(sta, WLAN_STA_TDLS_PEER_AUTH))
+ goto out;
+
+ break;
+ }
+ }
+
+ sta = sta_info_get(sdata, sdata->u.mgd.bssid);
+ break;
+ default:
+ goto out;
+ }
+
+ if (!sta)
+ goto out;
+
+ ret = drv_net_fill_forward_path(local, sdata, &sta->sta, ctx, path);
+out:
+ rcu_read_unlock();
+
+ return ret;
+}
+
static const struct net_device_ops ieee80211_dataif_8023_ops = {
.ndo_open = ieee80211_open,
.ndo_stop = ieee80211_stop,
@@ -798,6 +856,7 @@ static const struct net_device_ops ieee80211_dataif_8023_ops = {
.ndo_set_mac_address = ieee80211_change_mac,
.ndo_select_queue = ieee80211_netdev_select_queue,
.ndo_get_stats64 = ieee80211_get_stats64,
+ .ndo_fill_forward_path = ieee80211_netdev_fill_forward_path,
};
static bool ieee80211_iftype_supports_hdr_offload(enum nl80211_iftype iftype)
diff --git a/net/mac80211/main.c b/net/mac80211/main.c
index 45fb517..5311c3c 100644
--- a/net/mac80211/main.c
+++ b/net/mac80211/main.c
@@ -1131,17 +1131,14 @@ int ieee80211_register_hw(struct ieee80211_hw *hw)
local->scan_ies_len +=
2 + sizeof(struct ieee80211_vht_cap);
- /* HE cap element is variable in size - set len to allow max size */
/*
- * TODO: 1 is added at the end of the calculation to accommodate for
- * the temporary placing of the HE capabilities IE under EXT.
- * Remove it once it is placed in the final place.
- */
- if (supp_he)
+ * HE cap element is variable in size - set len to allow max size */
+ if (supp_he) {
local->scan_ies_len +=
- 2 + sizeof(struct ieee80211_he_cap_elem) +
+ 3 + sizeof(struct ieee80211_he_cap_elem) +
sizeof(struct ieee80211_he_mcs_nss_supp) +
- IEEE80211_HE_PPE_THRES_MAX_LEN + 1;
+ IEEE80211_HE_PPE_THRES_MAX_LEN;
+ }
if (!local->ops->hw_scan) {
/* For hw_scan, driver needs to set these up. */
diff --git a/net/mac80211/mesh.h b/net/mac80211/mesh.h
index 77080b4..b2b717a 100644
--- a/net/mac80211/mesh.h
+++ b/net/mac80211/mesh.h
@@ -127,26 +127,6 @@ struct mesh_path {
u32 path_change_count;
};
-/**
- * struct mesh_table
- *
- * @known_gates: list of known mesh gates and their mpaths by the station. The
- * gate's mpath may or may not be resolved and active.
- * @gates_lock: protects updates to known_gates
- * @rhead: the rhashtable containing struct mesh_paths, keyed by dest addr
- * @walk_head: linked list containing all mesh_path objects
- * @walk_lock: lock protecting walk_head
- * @entries: number of entries in the table
- */
-struct mesh_table {
- struct hlist_head known_gates;
- spinlock_t gates_lock;
- struct rhashtable rhead;
- struct hlist_head walk_head;
- spinlock_t walk_lock;
- atomic_t entries; /* Up to MAX_MESH_NEIGHBOURS */
-};
-
/* Recent multicast cache */
/* RMC_BUCKETS must be a power of 2, maximum 256 */
#define RMC_BUCKETS 256
@@ -308,7 +288,7 @@ int mesh_path_error_tx(struct ieee80211_sub_if_data *sdata,
void mesh_path_assign_nexthop(struct mesh_path *mpath, struct sta_info *sta);
void mesh_path_flush_pending(struct mesh_path *mpath);
void mesh_path_tx_pending(struct mesh_path *mpath);
-int mesh_pathtbl_init(struct ieee80211_sub_if_data *sdata);
+void mesh_pathtbl_init(struct ieee80211_sub_if_data *sdata);
void mesh_pathtbl_unregister(struct ieee80211_sub_if_data *sdata);
int mesh_path_del(struct ieee80211_sub_if_data *sdata, const u8 *addr);
void mesh_path_timer(struct timer_list *t);
diff --git a/net/mac80211/mesh_pathtbl.c b/net/mac80211/mesh_pathtbl.c
index 7cab1cf..acc1c29 100644
--- a/net/mac80211/mesh_pathtbl.c
+++ b/net/mac80211/mesh_pathtbl.c
@@ -47,32 +47,24 @@ static void mesh_path_rht_free(void *ptr, void *tblptr)
mesh_path_free_rcu(tbl, mpath);
}
-static struct mesh_table *mesh_table_alloc(void)
+static void mesh_table_init(struct mesh_table *tbl)
{
- struct mesh_table *newtbl;
+ INIT_HLIST_HEAD(&tbl->known_gates);
+ INIT_HLIST_HEAD(&tbl->walk_head);
+ atomic_set(&tbl->entries, 0);
+ spin_lock_init(&tbl->gates_lock);
+ spin_lock_init(&tbl->walk_lock);
- newtbl = kmalloc(sizeof(struct mesh_table), GFP_ATOMIC);
- if (!newtbl)
- return NULL;
-
- INIT_HLIST_HEAD(&newtbl->known_gates);
- INIT_HLIST_HEAD(&newtbl->walk_head);
- atomic_set(&newtbl->entries, 0);
- spin_lock_init(&newtbl->gates_lock);
- spin_lock_init(&newtbl->walk_lock);
- if (rhashtable_init(&newtbl->rhead, &mesh_rht_params)) {
- kfree(newtbl);
- return NULL;
- }
-
- return newtbl;
+ /* rhashtable_init() may fail only in case of wrong
+ * mesh_rht_params
+ */
+ WARN_ON(rhashtable_init(&tbl->rhead, &mesh_rht_params));
}
static void mesh_table_free(struct mesh_table *tbl)
{
rhashtable_free_and_destroy(&tbl->rhead,
mesh_path_rht_free, tbl);
- kfree(tbl);
}
/**
@@ -238,13 +230,13 @@ static struct mesh_path *mpath_lookup(struct mesh_table *tbl, const u8 *dst,
struct mesh_path *
mesh_path_lookup(struct ieee80211_sub_if_data *sdata, const u8 *dst)
{
- return mpath_lookup(sdata->u.mesh.mesh_paths, dst, sdata);
+ return mpath_lookup(&sdata->u.mesh.mesh_paths, dst, sdata);
}
struct mesh_path *
mpp_path_lookup(struct ieee80211_sub_if_data *sdata, const u8 *dst)
{
- return mpath_lookup(sdata->u.mesh.mpp_paths, dst, sdata);
+ return mpath_lookup(&sdata->u.mesh.mpp_paths, dst, sdata);
}
static struct mesh_path *
@@ -281,7 +273,7 @@ __mesh_path_lookup_by_idx(struct mesh_table *tbl, int idx)
struct mesh_path *
mesh_path_lookup_by_idx(struct ieee80211_sub_if_data *sdata, int idx)
{
- return __mesh_path_lookup_by_idx(sdata->u.mesh.mesh_paths, idx);
+ return __mesh_path_lookup_by_idx(&sdata->u.mesh.mesh_paths, idx);
}
/**
@@ -296,7 +288,7 @@ mesh_path_lookup_by_idx(struct ieee80211_sub_if_data *sdata, int idx)
struct mesh_path *
mpp_path_lookup_by_idx(struct ieee80211_sub_if_data *sdata, int idx)
{
- return __mesh_path_lookup_by_idx(sdata->u.mesh.mpp_paths, idx);
+ return __mesh_path_lookup_by_idx(&sdata->u.mesh.mpp_paths, idx);
}
/**
@@ -309,7 +301,7 @@ int mesh_path_add_gate(struct mesh_path *mpath)
int err;
rcu_read_lock();
- tbl = mpath->sdata->u.mesh.mesh_paths;
+ tbl = &mpath->sdata->u.mesh.mesh_paths;
spin_lock_bh(&mpath->state_lock);
if (mpath->is_gate) {
@@ -418,7 +410,7 @@ struct mesh_path *mesh_path_add(struct ieee80211_sub_if_data *sdata,
if (!new_mpath)
return ERR_PTR(-ENOMEM);
- tbl = sdata->u.mesh.mesh_paths;
+ tbl = &sdata->u.mesh.mesh_paths;
spin_lock_bh(&tbl->walk_lock);
mpath = rhashtable_lookup_get_insert_fast(&tbl->rhead,
&new_mpath->rhash,
@@ -460,7 +452,7 @@ int mpp_path_add(struct ieee80211_sub_if_data *sdata,
return -ENOMEM;
memcpy(new_mpath->mpp, mpp, ETH_ALEN);
- tbl = sdata->u.mesh.mpp_paths;
+ tbl = &sdata->u.mesh.mpp_paths;
spin_lock_bh(&tbl->walk_lock);
ret = rhashtable_lookup_insert_fast(&tbl->rhead,
@@ -489,7 +481,7 @@ int mpp_path_add(struct ieee80211_sub_if_data *sdata,
void mesh_plink_broken(struct sta_info *sta)
{
struct ieee80211_sub_if_data *sdata = sta->sdata;
- struct mesh_table *tbl = sdata->u.mesh.mesh_paths;
+ struct mesh_table *tbl = &sdata->u.mesh.mesh_paths;
static const u8 bcast[ETH_ALEN] = {0xff, 0xff, 0xff, 0xff, 0xff, 0xff};
struct mesh_path *mpath;
@@ -548,7 +540,7 @@ static void __mesh_path_del(struct mesh_table *tbl, struct mesh_path *mpath)
void mesh_path_flush_by_nexthop(struct sta_info *sta)
{
struct ieee80211_sub_if_data *sdata = sta->sdata;
- struct mesh_table *tbl = sdata->u.mesh.mesh_paths;
+ struct mesh_table *tbl = &sdata->u.mesh.mesh_paths;
struct mesh_path *mpath;
struct hlist_node *n;
@@ -563,7 +555,7 @@ void mesh_path_flush_by_nexthop(struct sta_info *sta)
static void mpp_flush_by_proxy(struct ieee80211_sub_if_data *sdata,
const u8 *proxy)
{
- struct mesh_table *tbl = sdata->u.mesh.mpp_paths;
+ struct mesh_table *tbl = &sdata->u.mesh.mpp_paths;
struct mesh_path *mpath;
struct hlist_node *n;
@@ -597,8 +589,8 @@ static void table_flush_by_iface(struct mesh_table *tbl)
*/
void mesh_path_flush_by_iface(struct ieee80211_sub_if_data *sdata)
{
- table_flush_by_iface(sdata->u.mesh.mesh_paths);
- table_flush_by_iface(sdata->u.mesh.mpp_paths);
+ table_flush_by_iface(&sdata->u.mesh.mesh_paths);
+ table_flush_by_iface(&sdata->u.mesh.mpp_paths);
}
/**
@@ -644,7 +636,7 @@ int mesh_path_del(struct ieee80211_sub_if_data *sdata, const u8 *addr)
/* flush relevant mpp entries first */
mpp_flush_by_proxy(sdata, addr);
- err = table_path_del(sdata->u.mesh.mesh_paths, sdata, addr);
+ err = table_path_del(&sdata->u.mesh.mesh_paths, sdata, addr);
sdata->u.mesh.mesh_paths_generation++;
return err;
}
@@ -682,7 +674,7 @@ int mesh_path_send_to_gates(struct mesh_path *mpath)
struct mesh_path *gate;
bool copy = false;
- tbl = sdata->u.mesh.mesh_paths;
+ tbl = &sdata->u.mesh.mesh_paths;
rcu_read_lock();
hlist_for_each_entry_rcu(gate, &tbl->known_gates, gate_list) {
@@ -762,29 +754,10 @@ void mesh_path_fix_nexthop(struct mesh_path *mpath, struct sta_info *next_hop)
mesh_path_tx_pending(mpath);
}
-int mesh_pathtbl_init(struct ieee80211_sub_if_data *sdata)
+void mesh_pathtbl_init(struct ieee80211_sub_if_data *sdata)
{
- struct mesh_table *tbl_path, *tbl_mpp;
- int ret;
-
- tbl_path = mesh_table_alloc();
- if (!tbl_path)
- return -ENOMEM;
-
- tbl_mpp = mesh_table_alloc();
- if (!tbl_mpp) {
- ret = -ENOMEM;
- goto free_path;
- }
-
- sdata->u.mesh.mesh_paths = tbl_path;
- sdata->u.mesh.mpp_paths = tbl_mpp;
-
- return 0;
-
-free_path:
- mesh_table_free(tbl_path);
- return ret;
+ mesh_table_init(&sdata->u.mesh.mesh_paths);
+ mesh_table_init(&sdata->u.mesh.mpp_paths);
}
static
@@ -806,12 +779,12 @@ void mesh_path_tbl_expire(struct ieee80211_sub_if_data *sdata,
void mesh_path_expire(struct ieee80211_sub_if_data *sdata)
{
- mesh_path_tbl_expire(sdata, sdata->u.mesh.mesh_paths);
- mesh_path_tbl_expire(sdata, sdata->u.mesh.mpp_paths);
+ mesh_path_tbl_expire(sdata, &sdata->u.mesh.mesh_paths);
+ mesh_path_tbl_expire(sdata, &sdata->u.mesh.mpp_paths);
}
void mesh_pathtbl_unregister(struct ieee80211_sub_if_data *sdata)
{
- mesh_table_free(sdata->u.mesh.mesh_paths);
- mesh_table_free(sdata->u.mesh.mpp_paths);
+ mesh_table_free(&sdata->u.mesh.mesh_paths);
+ mesh_table_free(&sdata->u.mesh.mpp_paths);
}
diff --git a/net/mac80211/mlme.c b/net/mac80211/mlme.c
index 37f7d97..1eeabdf 100644
--- a/net/mac80211/mlme.c
+++ b/net/mac80211/mlme.c
@@ -164,12 +164,15 @@ ieee80211_determine_chantype(struct ieee80211_sub_if_data *sdata,
chandef->freq1_offset = channel->freq_offset;
if (channel->band == NL80211_BAND_6GHZ) {
- if (!ieee80211_chandef_he_6ghz_oper(sdata, he_oper, chandef))
+ if (!ieee80211_chandef_he_6ghz_oper(sdata, he_oper, chandef)) {
+ mlme_dbg(sdata,
+ "bad 6 GHz operation, disabling HT/VHT/HE\n");
ret = IEEE80211_STA_DISABLE_HT |
IEEE80211_STA_DISABLE_VHT |
IEEE80211_STA_DISABLE_HE;
- else
+ } else {
ret = 0;
+ }
vht_chandef = *chandef;
goto out;
} else if (sband->band == NL80211_BAND_S1GHZ) {
@@ -190,6 +193,7 @@ ieee80211_determine_chantype(struct ieee80211_sub_if_data *sdata,
ieee80211_apply_htcap_overrides(sdata, &sta_ht_cap);
if (!ht_oper || !sta_ht_cap.ht_supported) {
+ mlme_dbg(sdata, "HT operation missing / HT not supported\n");
ret = IEEE80211_STA_DISABLE_HT |
IEEE80211_STA_DISABLE_VHT |
IEEE80211_STA_DISABLE_HE;
@@ -223,6 +227,7 @@ ieee80211_determine_chantype(struct ieee80211_sub_if_data *sdata,
if (sta_ht_cap.cap & IEEE80211_HT_CAP_SUP_WIDTH_20_40) {
ieee80211_chandef_ht_oper(ht_oper, chandef);
} else {
+ mlme_dbg(sdata, "40 MHz not supported\n");
/* 40 MHz (and 80 MHz) must be supported for VHT */
ret = IEEE80211_STA_DISABLE_VHT;
/* also mark 40 MHz disabled */
@@ -231,6 +236,7 @@ ieee80211_determine_chantype(struct ieee80211_sub_if_data *sdata,
}
if (!vht_oper || !sband->vht_cap.vht_supported) {
+ mlme_dbg(sdata, "VHT operation missing / VHT not supported\n");
ret = IEEE80211_STA_DISABLE_VHT;
goto out;
}
@@ -253,7 +259,7 @@ ieee80211_determine_chantype(struct ieee80211_sub_if_data *sdata,
&vht_chandef)) {
if (!(ifmgd->flags & IEEE80211_STA_DISABLE_HE))
sdata_info(sdata,
- "HE AP VHT information is invalid, disable HE\n");
+ "HE AP VHT information is invalid, disabling HE\n");
ret = IEEE80211_STA_DISABLE_HE;
goto out;
}
@@ -263,7 +269,7 @@ ieee80211_determine_chantype(struct ieee80211_sub_if_data *sdata,
&vht_chandef)) {
if (!(ifmgd->flags & IEEE80211_STA_DISABLE_VHT))
sdata_info(sdata,
- "AP VHT information is invalid, disable VHT\n");
+ "AP VHT information is invalid, disabling VHT\n");
ret = IEEE80211_STA_DISABLE_VHT;
goto out;
}
@@ -271,7 +277,7 @@ ieee80211_determine_chantype(struct ieee80211_sub_if_data *sdata,
if (!cfg80211_chandef_valid(&vht_chandef)) {
if (!(ifmgd->flags & IEEE80211_STA_DISABLE_VHT))
sdata_info(sdata,
- "AP VHT information is invalid, disable VHT\n");
+ "AP VHT information is invalid, disabling VHT\n");
ret = IEEE80211_STA_DISABLE_VHT;
goto out;
}
@@ -284,7 +290,7 @@ ieee80211_determine_chantype(struct ieee80211_sub_if_data *sdata,
if (!cfg80211_chandef_compatible(chandef, &vht_chandef)) {
if (!(ifmgd->flags & IEEE80211_STA_DISABLE_VHT))
sdata_info(sdata,
- "AP VHT information doesn't match HT, disable VHT\n");
+ "AP VHT information doesn't match HT, disabling VHT\n");
ret = IEEE80211_STA_DISABLE_VHT;
goto out;
}
@@ -649,10 +655,6 @@ static void ieee80211_add_he_ie(struct ieee80211_sub_if_data *sdata,
if (!he_cap || !reg_cap)
return;
- /*
- * TODO: the 1 added is because this temporarily is under the EXTENSION
- * IE. Get rid of it when it moves.
- */
he_cap_size =
2 + 1 + sizeof(he_cap->he_cap_elem) +
ieee80211_he_mcs_nss_size(&he_cap->he_cap_elem) +
@@ -3741,6 +3743,10 @@ static void ieee80211_rx_mgmt_assoc_resp(struct ieee80211_sub_if_data *sdata,
elems->timeout_int &&
elems->timeout_int->type == WLAN_TIMEOUT_ASSOC_COMEBACK) {
u32 tu, ms;
+
+ cfg80211_assoc_comeback(sdata->dev, assoc_data->bss,
+ le32_to_cpu(elems->timeout_int->value));
+
tu = le32_to_cpu(elems->timeout_int->value);
ms = tu * 1024 / 1000;
sdata_info(sdata,
@@ -4900,7 +4906,7 @@ static u8 ieee80211_ht_vht_rx_chains(struct ieee80211_sub_if_data *sdata,
struct cfg80211_bss *cbss)
{
struct ieee80211_if_managed *ifmgd = &sdata->u.mgd;
- const u8 *ht_cap_ie, *vht_cap_ie;
+ const struct element *ht_cap_elem, *vht_cap_elem;
const struct ieee80211_ht_cap *ht_cap;
const struct ieee80211_vht_cap *vht_cap;
u8 chains = 1;
@@ -4908,9 +4914,9 @@ static u8 ieee80211_ht_vht_rx_chains(struct ieee80211_sub_if_data *sdata,
if (ifmgd->flags & IEEE80211_STA_DISABLE_HT)
return chains;
- ht_cap_ie = ieee80211_bss_get_ie(cbss, WLAN_EID_HT_CAPABILITY);
- if (ht_cap_ie && ht_cap_ie[1] >= sizeof(*ht_cap)) {
- ht_cap = (void *)(ht_cap_ie + 2);
+ ht_cap_elem = ieee80211_bss_get_elem(cbss, WLAN_EID_HT_CAPABILITY);
+ if (ht_cap_elem && ht_cap_elem->datalen >= sizeof(*ht_cap)) {
+ ht_cap = (void *)ht_cap_elem->data;
chains = ieee80211_mcs_to_chains(&ht_cap->mcs);
/*
* TODO: use "Tx Maximum Number Spatial Streams Supported" and
@@ -4921,12 +4927,12 @@ static u8 ieee80211_ht_vht_rx_chains(struct ieee80211_sub_if_data *sdata,
if (ifmgd->flags & IEEE80211_STA_DISABLE_VHT)
return chains;
- vht_cap_ie = ieee80211_bss_get_ie(cbss, WLAN_EID_VHT_CAPABILITY);
- if (vht_cap_ie && vht_cap_ie[1] >= sizeof(*vht_cap)) {
+ vht_cap_elem = ieee80211_bss_get_elem(cbss, WLAN_EID_VHT_CAPABILITY);
+ if (vht_cap_elem && vht_cap_elem->datalen >= sizeof(*vht_cap)) {
u8 nss;
u16 tx_mcs_map;
- vht_cap = (void *)(vht_cap_ie + 2);
+ vht_cap = (void *)vht_cap_elem->data;
tx_mcs_map = le16_to_cpu(vht_cap->supp_mcs.tx_mcs_map);
for (nss = 8; nss > 0; nss--) {
if (((tx_mcs_map >> (2 * (nss - 1))) & 3) !=
@@ -5043,19 +5049,23 @@ static int ieee80211_prep_channel(struct ieee80211_sub_if_data *sdata,
/* disable HT/VHT/HE if we don't support them */
if (!sband->ht_cap.ht_supported && !is_6ghz) {
+ mlme_dbg(sdata, "HT not supported, disabling HT/VHT/HE\n");
ifmgd->flags |= IEEE80211_STA_DISABLE_HT;
ifmgd->flags |= IEEE80211_STA_DISABLE_VHT;
ifmgd->flags |= IEEE80211_STA_DISABLE_HE;
}
if (!sband->vht_cap.vht_supported && is_5ghz) {
+ mlme_dbg(sdata, "VHT not supported, disabling VHT/HE\n");
ifmgd->flags |= IEEE80211_STA_DISABLE_VHT;
ifmgd->flags |= IEEE80211_STA_DISABLE_HE;
}
if (!ieee80211_get_he_iftype_cap(sband,
- ieee80211_vif_type_p2p(&sdata->vif)))
+ ieee80211_vif_type_p2p(&sdata->vif))) {
+ mlme_dbg(sdata, "HE not supported, disabling it\n");
ifmgd->flags |= IEEE80211_STA_DISABLE_HE;
+ }
if (!(ifmgd->flags & IEEE80211_STA_DISABLE_HT) && !is_6ghz) {
ht_oper = elems->ht_operation;
@@ -5079,6 +5089,8 @@ static int ieee80211_prep_channel(struct ieee80211_sub_if_data *sdata,
}
if (!elems->vht_cap_elem) {
+ sdata_info(sdata,
+ "bad VHT capabilities, disabling VHT\n");
ifmgd->flags |= IEEE80211_STA_DISABLE_VHT;
vht_oper = NULL;
}
@@ -5126,8 +5138,10 @@ static int ieee80211_prep_channel(struct ieee80211_sub_if_data *sdata,
break;
}
- if (!have_80mhz)
+ if (!have_80mhz) {
+ sdata_info(sdata, "80 MHz not supported, disabling VHT\n");
ifmgd->flags |= IEEE80211_STA_DISABLE_VHT;
+ }
if (sband->band == NL80211_BAND_S1GHZ) {
s1g_oper = elems->s1g_oper;
@@ -5265,7 +5279,7 @@ static int ieee80211_prep_connection(struct ieee80211_sub_if_data *sdata,
*/
if (new_sta) {
u32 rates = 0, basic_rates = 0;
- bool have_higher_than_11mbit;
+ bool have_higher_than_11mbit = false;
int min_rate = INT_MAX, min_rate_index = -1;
const struct cfg80211_bss_ies *ies;
int shift = ieee80211_vif_get_shift(&sdata->vif);
@@ -5691,12 +5705,14 @@ int ieee80211_mgd_assoc(struct ieee80211_sub_if_data *sdata,
else if (!is_6ghz)
ifmgd->flags |= IEEE80211_STA_DISABLE_HT;
vht_elem = ieee80211_bss_get_elem(req->bss, WLAN_EID_VHT_CAPABILITY);
- if (vht_elem && vht_elem->datalen >= sizeof(struct ieee80211_vht_cap))
+ if (vht_elem && vht_elem->datalen >= sizeof(struct ieee80211_vht_cap)) {
memcpy(&assoc_data->ap_vht_cap, vht_elem->data,
sizeof(struct ieee80211_vht_cap));
- else if (is_5ghz)
+ } else if (is_5ghz) {
+ sdata_info(sdata, "VHT capa missing/short, disabling VHT/HE\n");
ifmgd->flags |= IEEE80211_STA_DISABLE_VHT |
IEEE80211_STA_DISABLE_HE;
+ }
rcu_read_unlock();
if (WARN((sdata->vif.driver_flags & IEEE80211_VIF_SUPPORTS_UAPSD) &&
@@ -5770,16 +5786,21 @@ int ieee80211_mgd_assoc(struct ieee80211_sub_if_data *sdata,
}
if (req->flags & ASSOC_REQ_DISABLE_HT) {
+ mlme_dbg(sdata, "HT disabled by flag, disabling HT/VHT/HE\n");
ifmgd->flags |= IEEE80211_STA_DISABLE_HT;
ifmgd->flags |= IEEE80211_STA_DISABLE_VHT;
ifmgd->flags |= IEEE80211_STA_DISABLE_HE;
}
- if (req->flags & ASSOC_REQ_DISABLE_VHT)
+ if (req->flags & ASSOC_REQ_DISABLE_VHT) {
+ mlme_dbg(sdata, "VHT disabled by flag, disabling VHT\n");
ifmgd->flags |= IEEE80211_STA_DISABLE_VHT;
+ }
- if (req->flags & ASSOC_REQ_DISABLE_HE)
+ if (req->flags & ASSOC_REQ_DISABLE_HE) {
+ mlme_dbg(sdata, "HE disabled by flag, disabling VHT\n");
ifmgd->flags |= IEEE80211_STA_DISABLE_HE;
+ }
err = ieee80211_prep_connection(sdata, req->bss, true, override);
if (err)
diff --git a/net/mac80211/rc80211_minstrel_ht.c b/net/mac80211/rc80211_minstrel_ht.c
index 72b44d4..9c3b7fc 100644
--- a/net/mac80211/rc80211_minstrel_ht.c
+++ b/net/mac80211/rc80211_minstrel_ht.c
@@ -18,8 +18,6 @@
#define AVG_AMPDU_SIZE 16
#define AVG_PKT_SIZE 1200
-#define SAMPLE_SWITCH_THR 100
-
/* Number of bits for an average sized packet */
#define MCS_NBITS ((AVG_PKT_SIZE * AVG_AMPDU_SIZE) << 3)
diff --git a/net/mac80211/rx.c b/net/mac80211/rx.c
index 0544563..93680af 100644
--- a/net/mac80211/rx.c
+++ b/net/mac80211/rx.c
@@ -465,7 +465,12 @@ ieee80211_add_rx_radiotap_header(struct ieee80211_local *local,
unsigned int stbc;
rthdr->it_present |= cpu_to_le32(BIT(IEEE80211_RADIOTAP_MCS));
- *pos++ = local->hw.radiotap_mcs_details;
+ *pos = local->hw.radiotap_mcs_details;
+ if (status->enc_flags & RX_ENC_FLAG_HT_GF)
+ *pos |= IEEE80211_RADIOTAP_MCS_HAVE_FMT;
+ if (status->enc_flags & RX_ENC_FLAG_LDPC)
+ *pos |= IEEE80211_RADIOTAP_MCS_HAVE_FEC;
+ pos++;
*pos = 0;
if (status->enc_flags & RX_ENC_FLAG_SHORT_GI)
*pos |= IEEE80211_RADIOTAP_MCS_SGI;
@@ -4924,7 +4929,7 @@ void ieee80211_rx_list(struct ieee80211_hw *hw, struct ieee80211_sta *pubsta,
goto drop;
break;
case RX_ENC_VHT:
- if (WARN_ONCE(status->rate_idx > 9 ||
+ if (WARN_ONCE(status->rate_idx > 11 ||
!status->nss ||
status->nss > 8,
"Rate marked as a VHT rate but data is invalid: MCS: %d, NSS: %d\n",
diff --git a/net/mac80211/trace.h b/net/mac80211/trace.h
index 9e8381b..d91498f 100644
--- a/net/mac80211/trace.h
+++ b/net/mac80211/trace.h
@@ -2892,6 +2892,13 @@ TRACE_EVENT(drv_twt_teardown_request,
)
);
+DEFINE_EVENT(sta_event, drv_net_fill_forward_path,
+ TP_PROTO(struct ieee80211_local *local,
+ struct ieee80211_sub_if_data *sdata,
+ struct ieee80211_sta *sta),
+ TP_ARGS(local, sdata, sta)
+);
+
#endif /* !__MAC80211_DRIVER_TRACE || TRACE_HEADER_MULTI_READ */
#undef TRACE_INCLUDE_PATH
diff --git a/net/mac80211/tx.c b/net/mac80211/tx.c
index 86a54df..6d054fe 100644
--- a/net/mac80211/tx.c
+++ b/net/mac80211/tx.c
@@ -3821,7 +3821,7 @@ struct ieee80211_txq *ieee80211_next_txq(struct ieee80211_hw *hw, u8 ac)
{
struct ieee80211_local *local = hw_to_local(hw);
struct airtime_sched_info *air_sched;
- u64 now = ktime_get_boottime_ns();
+ u64 now = ktime_get_coarse_boottime_ns();
struct ieee80211_txq *ret = NULL;
struct airtime_info *air_info;
struct txq_info *txqi = NULL;
@@ -3948,7 +3948,7 @@ void ieee80211_update_airtime_weight(struct ieee80211_local *local,
u64 weight_sum = 0;
if (unlikely(!now))
- now = ktime_get_boottime_ns();
+ now = ktime_get_coarse_boottime_ns();
lockdep_assert_held(&air_sched->lock);
@@ -3974,7 +3974,7 @@ void ieee80211_schedule_txq(struct ieee80211_hw *hw,
struct ieee80211_local *local = hw_to_local(hw);
struct txq_info *txqi = to_txq_info(txq);
struct airtime_sched_info *air_sched;
- u64 now = ktime_get_boottime_ns();
+ u64 now = ktime_get_coarse_boottime_ns();
struct airtime_info *air_info;
u8 ac = txq->ac;
bool was_active;
@@ -4032,7 +4032,7 @@ static void __ieee80211_unschedule_txq(struct ieee80211_hw *hw,
if (!purge)
airtime_set_active(air_sched, air_info,
- ktime_get_boottime_ns());
+ ktime_get_coarse_boottime_ns());
rb_erase_cached(&txqi->schedule_order,
&air_sched->active_txqs);
@@ -4120,7 +4120,7 @@ bool ieee80211_txq_may_transmit(struct ieee80211_hw *hw,
if (RB_EMPTY_NODE(&txqi->schedule_order))
goto out;
- now = ktime_get_boottime_ns();
+ now = ktime_get_coarse_boottime_ns();
/* Like in ieee80211_next_txq(), make sure the first station in the
* scheduling order is eligible for transmission to avoid starvation.
diff --git a/net/mac80211/util.c b/net/mac80211/util.c
index 0e4e195..f71b042 100644
--- a/net/mac80211/util.c
+++ b/net/mac80211/util.c
@@ -862,6 +862,19 @@ static void __iterate_stations(struct ieee80211_local *local,
}
}
+void ieee80211_iterate_stations(struct ieee80211_hw *hw,
+ void (*iterator)(void *data,
+ struct ieee80211_sta *sta),
+ void *data)
+{
+ struct ieee80211_local *local = hw_to_local(hw);
+
+ mutex_lock(&local->sta_mtx);
+ __iterate_stations(local, iterator, data);
+ mutex_unlock(&local->sta_mtx);
+}
+EXPORT_SYMBOL_GPL(ieee80211_iterate_stations);
+
void ieee80211_iterate_stations_atomic(struct ieee80211_hw *hw,
void (*iterator)(void *data,
struct ieee80211_sta *sta),
diff --git a/net/mac80211/wpa.c b/net/mac80211/wpa.c
index 4eed23e..7ed0d26 100644
--- a/net/mac80211/wpa.c
+++ b/net/mac80211/wpa.c
@@ -449,7 +449,6 @@ static int ccmp_encrypt_skb(struct ieee80211_tx_data *tx, struct sk_buff *skb,
(info->control.hw_key->flags & IEEE80211_KEY_FLAG_PUT_IV_SPACE))
return 0;
- hdr = (struct ieee80211_hdr *) pos;
pos += hdrlen;
pn64 = atomic64_inc_return(&key->conf.tx_pn);
@@ -686,7 +685,6 @@ static int gcmp_encrypt_skb(struct ieee80211_tx_data *tx, struct sk_buff *skb)
(info->control.hw_key->flags & IEEE80211_KEY_FLAG_PUT_IV_SPACE))
return 0;
- hdr = (struct ieee80211_hdr *)pos;
pos += hdrlen;
pn64 = atomic64_inc_return(&key->conf.tx_pn);
@@ -881,8 +879,6 @@ ieee80211_crypto_cs_decrypt(struct ieee80211_rx_data *rx)
if (skb_linearize(rx->skb))
return RX_DROP_UNUSABLE;
- hdr = (struct ieee80211_hdr *)rx->skb->data;
-
rx_pn = key->u.gen.rx_pn[qos_tid];
skb_pn = rx->skb->data + hdrlen + cs->pn_off;
diff --git a/net/mctp/af_mctp.c b/net/mctp/af_mctp.c
index 871cf62..c921de6 100644
--- a/net/mctp/af_mctp.c
+++ b/net/mctp/af_mctp.c
@@ -405,8 +405,7 @@ static void mctp_sk_unhash(struct sock *sk)
trace_mctp_key_release(key, MCTP_TRACE_KEY_CLOSED);
spin_lock(&key->lock);
- if (key->reasm_head)
- kfree_skb(key->reasm_head);
+ kfree_skb(key->reasm_head);
key->reasm_head = NULL;
key->reasm_dead = true;
key->valid = false;
diff --git a/net/mctp/device.c b/net/mctp/device.c
index 8799ee7..ef2755f 100644
--- a/net/mctp/device.c
+++ b/net/mctp/device.c
@@ -35,14 +35,24 @@ struct mctp_dev *mctp_dev_get_rtnl(const struct net_device *dev)
return rtnl_dereference(dev->mctp_ptr);
}
-static int mctp_fill_addrinfo(struct sk_buff *skb, struct netlink_callback *cb,
- struct mctp_dev *mdev, mctp_eid_t eid)
+static int mctp_addrinfo_size(void)
+{
+ return NLMSG_ALIGN(sizeof(struct ifaddrmsg))
+ + nla_total_size(1) // IFA_LOCAL
+ + nla_total_size(1) // IFA_ADDRESS
+ ;
+}
+
+/* flag should be NLM_F_MULTI for dump calls */
+static int mctp_fill_addrinfo(struct sk_buff *skb,
+ struct mctp_dev *mdev, mctp_eid_t eid,
+ int msg_type, u32 portid, u32 seq, int flag)
{
struct ifaddrmsg *hdr;
struct nlmsghdr *nlh;
- nlh = nlmsg_put(skb, NETLINK_CB(cb->skb).portid, cb->nlh->nlmsg_seq,
- RTM_NEWADDR, sizeof(*hdr), NLM_F_MULTI);
+ nlh = nlmsg_put(skb, portid, seq,
+ msg_type, sizeof(*hdr), flag);
if (!nlh)
return -EMSGSIZE;
@@ -72,10 +82,14 @@ static int mctp_dump_dev_addrinfo(struct mctp_dev *mdev, struct sk_buff *skb,
struct netlink_callback *cb)
{
struct mctp_dump_cb *mcb = (void *)cb->ctx;
+ u32 portid, seq;
int rc = 0;
+ portid = NETLINK_CB(cb->skb).portid;
+ seq = cb->nlh->nlmsg_seq;
for (; mcb->a_idx < mdev->num_addrs; mcb->a_idx++) {
- rc = mctp_fill_addrinfo(skb, cb, mdev, mdev->addrs[mcb->a_idx]);
+ rc = mctp_fill_addrinfo(skb, mdev, mdev->addrs[mcb->a_idx],
+ RTM_NEWADDR, portid, seq, NLM_F_MULTI);
if (rc < 0)
break;
}
@@ -127,6 +141,32 @@ static int mctp_dump_addrinfo(struct sk_buff *skb, struct netlink_callback *cb)
return skb->len;
}
+static void mctp_addr_notify(struct mctp_dev *mdev, mctp_eid_t eid, int msg_type,
+ struct sk_buff *req_skb, struct nlmsghdr *req_nlh)
+{
+ u32 portid = NETLINK_CB(req_skb).portid;
+ struct net *net = dev_net(mdev->dev);
+ struct sk_buff *skb;
+ int rc = -ENOBUFS;
+
+ skb = nlmsg_new(mctp_addrinfo_size(), GFP_KERNEL);
+ if (!skb)
+ goto out;
+
+ rc = mctp_fill_addrinfo(skb, mdev, eid, msg_type,
+ portid, req_nlh->nlmsg_seq, 0);
+ if (rc < 0) {
+ WARN_ON_ONCE(rc == -EMSGSIZE);
+ goto out;
+ }
+
+ rtnl_notify(skb, net, portid, RTNLGRP_MCTP_IFADDR, req_nlh, GFP_KERNEL);
+ return;
+out:
+ kfree_skb(skb);
+ rtnl_set_sk_err(net, RTNLGRP_MCTP_IFADDR, rc);
+}
+
static const struct nla_policy ifa_mctp_policy[IFA_MAX + 1] = {
[IFA_ADDRESS] = { .type = NLA_U8 },
[IFA_LOCAL] = { .type = NLA_U8 },
@@ -189,6 +229,7 @@ static int mctp_rtm_newaddr(struct sk_buff *skb, struct nlmsghdr *nlh,
kfree(tmp_addrs);
+ mctp_addr_notify(mdev, addr->s_addr, RTM_NEWADDR, skb, nlh);
mctp_route_add_local(mdev, addr->s_addr);
return 0;
@@ -244,6 +285,8 @@ static int mctp_rtm_deladdr(struct sk_buff *skb, struct nlmsghdr *nlh,
mdev->num_addrs--;
spin_unlock_irqrestore(&mdev->addrs_lock, flags);
+ mctp_addr_notify(mdev, addr->s_addr, RTM_DELADDR, skb, nlh);
+
return 0;
}
diff --git a/net/mctp/neigh.c b/net/mctp/neigh.c
index 5cc0421..6ad3e33 100644
--- a/net/mctp/neigh.c
+++ b/net/mctp/neigh.c
@@ -85,8 +85,8 @@ void mctp_neigh_remove_dev(struct mctp_dev *mdev)
mutex_unlock(&net->mctp.neigh_lock);
}
-// TODO: add a "source" flag so netlink can only delete static neighbours?
-static int mctp_neigh_remove(struct mctp_dev *mdev, mctp_eid_t eid)
+static int mctp_neigh_remove(struct mctp_dev *mdev, mctp_eid_t eid,
+ enum mctp_neigh_source source)
{
struct net *net = dev_net(mdev->dev);
struct mctp_neigh *neigh, *tmp;
@@ -94,7 +94,8 @@ static int mctp_neigh_remove(struct mctp_dev *mdev, mctp_eid_t eid)
mutex_lock(&net->mctp.neigh_lock);
list_for_each_entry_safe(neigh, tmp, &net->mctp.neighbours, list) {
- if (neigh->dev == mdev && neigh->eid == eid) {
+ if (neigh->dev == mdev && neigh->eid == eid &&
+ neigh->source == source) {
list_del_rcu(&neigh->list);
/* TODO: immediate RTM_DELNEIGH */
call_rcu(&neigh->rcu, __mctp_neigh_free);
@@ -202,7 +203,7 @@ static int mctp_rtm_delneigh(struct sk_buff *skb, struct nlmsghdr *nlh,
if (!mdev)
return -ENODEV;
- return mctp_neigh_remove(mdev, eid);
+ return mctp_neigh_remove(mdev, eid, MCTP_NEIGH_STATIC);
}
static int mctp_fill_neigh(struct sk_buff *skb, u32 portid, u32 seq, int event,
diff --git a/net/mctp/route.c b/net/mctp/route.c
index cdf09c2..8d9f4ff 100644
--- a/net/mctp/route.c
+++ b/net/mctp/route.c
@@ -231,9 +231,7 @@ static void __mctp_key_unlock_drop(struct mctp_sk_key *key, struct net *net,
/* and one for the local reference */
mctp_key_unref(key);
- if (skb)
- kfree_skb(skb);
-
+ kfree_skb(skb);
}
#ifdef CONFIG_MCTP_FLOWS
@@ -892,8 +890,7 @@ int mctp_local_output(struct sock *sk, struct mctp_route *rt,
if (!ext_rt)
mctp_route_release(rt);
- if (dev)
- dev_put(dev);
+ dev_put(dev);
return rc;
diff --git a/net/mctp/test/route-test.c b/net/mctp/test/route-test.c
index 36fac3d..86ad15a 100644
--- a/net/mctp/test/route-test.c
+++ b/net/mctp/test/route-test.c
@@ -150,11 +150,6 @@ static void mctp_test_fragment(struct kunit *test)
rt = mctp_test_create_route(&init_net, NULL, 10, mtu);
KUNIT_ASSERT_TRUE(test, rt);
- /* The refcount would usually be incremented as part of a route lookup,
- * but we're setting the route directly here.
- */
- refcount_inc(&rt->rt.refs);
-
rc = mctp_do_fragment_route(&rt->rt, skb, mtu, MCTP_TAG_OWNER);
KUNIT_EXPECT_FALSE(test, rc);
diff --git a/net/mpls/af_mpls.c b/net/mpls/af_mpls.c
index 0c7bde1..48f75a5 100644
--- a/net/mpls/af_mpls.c
+++ b/net/mpls/af_mpls.c
@@ -230,8 +230,8 @@ static struct mpls_nh *mpls_get_nexthop(struct mpls_route *rt, u8 index)
* Since those fields can change at any moment, use READ_ONCE to
* access both.
*/
-static struct mpls_nh *mpls_select_multipath(struct mpls_route *rt,
- struct sk_buff *skb)
+static const struct mpls_nh *mpls_select_multipath(struct mpls_route *rt,
+ struct sk_buff *skb)
{
u32 hash = 0;
int nh_index = 0;
@@ -343,8 +343,8 @@ static int mpls_forward(struct sk_buff *skb, struct net_device *dev,
{
struct net *net = dev_net(dev);
struct mpls_shim_hdr *hdr;
+ const struct mpls_nh *nh;
struct mpls_route *rt;
- struct mpls_nh *nh;
struct mpls_entry_decoded dec;
struct net_device *out_dev;
struct mpls_dev *out_mdev;
@@ -2360,12 +2360,12 @@ static int mpls_getroute(struct sk_buff *in_skb, struct nlmsghdr *in_nlh,
u32 labels[MAX_NEW_LABELS];
struct mpls_shim_hdr *hdr;
unsigned int hdr_size = 0;
+ const struct mpls_nh *nh;
struct net_device *dev;
struct mpls_route *rt;
struct rtmsg *rtm, *r;
struct nlmsghdr *nlh;
struct sk_buff *skb;
- struct mpls_nh *nh;
u8 n_labels;
int err;
diff --git a/net/mpls/internal.h b/net/mpls/internal.h
index 893df00..b9f492d 100644
--- a/net/mpls/internal.h
+++ b/net/mpls/internal.h
@@ -158,17 +158,16 @@ struct mpls_route { /* next hop label forwarding entry */
};
#define for_nexthops(rt) { \
- int nhsel; struct mpls_nh *nh; u8 *__nh; \
- for (nhsel = 0, nh = (rt)->rt_nh, __nh = (u8 *)((rt)->rt_nh); \
+ int nhsel; const struct mpls_nh *nh; \
+ for (nhsel = 0, nh = (rt)->rt_nh; \
nhsel < (rt)->rt_nhn; \
- __nh += rt->rt_nh_size, nh = (struct mpls_nh *)__nh, nhsel++)
+ nh = (void *)nh + (rt)->rt_nh_size, nhsel++)
#define change_nexthops(rt) { \
- int nhsel; struct mpls_nh *nh; u8 *__nh; \
- for (nhsel = 0, nh = (struct mpls_nh *)((rt)->rt_nh), \
- __nh = (u8 *)((rt)->rt_nh); \
+ int nhsel; struct mpls_nh *nh; \
+ for (nhsel = 0, nh = (rt)->rt_nh; \
nhsel < (rt)->rt_nhn; \
- __nh += rt->rt_nh_size, nh = (struct mpls_nh *)__nh, nhsel++)
+ nh = (void *)nh + (rt)->rt_nh_size, nhsel++)
#define endfor_nexthops(rt) }
diff --git a/net/mptcp/options.c b/net/mptcp/options.c
index fe98e4f..645dd98 100644
--- a/net/mptcp/options.c
+++ b/net/mptcp/options.c
@@ -768,6 +768,28 @@ static noinline bool mptcp_established_options_rst(struct sock *sk, struct sk_bu
return true;
}
+static bool mptcp_established_options_fastclose(struct sock *sk,
+ unsigned int *size,
+ unsigned int remaining,
+ struct mptcp_out_options *opts)
+{
+ struct mptcp_subflow_context *subflow = mptcp_subflow_ctx(sk);
+ struct mptcp_sock *msk = mptcp_sk(subflow->conn);
+
+ if (likely(!subflow->send_fastclose))
+ return false;
+
+ if (remaining < TCPOLEN_MPTCP_FASTCLOSE)
+ return false;
+
+ *size = TCPOLEN_MPTCP_FASTCLOSE;
+ opts->suboptions |= OPTION_MPTCP_FASTCLOSE;
+ opts->rcvr_key = msk->remote_key;
+
+ pr_debug("FASTCLOSE key=%llu", opts->rcvr_key);
+ return true;
+}
+
static bool mptcp_established_options_mp_fail(struct sock *sk,
unsigned int *size,
unsigned int remaining,
@@ -806,10 +828,12 @@ bool mptcp_established_options(struct sock *sk, struct sk_buff *skb,
return false;
if (unlikely(skb && TCP_SKB_CB(skb)->tcp_flags & TCPHDR_RST)) {
- if (mptcp_established_options_mp_fail(sk, &opt_size, remaining, opts)) {
+ if (mptcp_established_options_fastclose(sk, &opt_size, remaining, opts) ||
+ mptcp_established_options_mp_fail(sk, &opt_size, remaining, opts)) {
*size += opt_size;
remaining -= opt_size;
}
+ /* MP_RST can be used with MP_FASTCLOSE and MP_FAIL if there is room */
if (mptcp_established_options_rst(sk, skb, &opt_size, remaining, opts)) {
*size += opt_size;
remaining -= opt_size;
@@ -821,10 +845,13 @@ bool mptcp_established_options(struct sock *sk, struct sk_buff *skb,
if (mptcp_established_options_mp(sk, skb, snd_data_fin, &opt_size, remaining, opts))
ret = true;
else if (mptcp_established_options_dss(sk, skb, snd_data_fin, &opt_size, remaining, opts)) {
+ unsigned int mp_fail_size;
+
ret = true;
- if (mptcp_established_options_mp_fail(sk, &opt_size, remaining, opts)) {
- *size += opt_size;
- remaining -= opt_size;
+ if (mptcp_established_options_mp_fail(sk, &mp_fail_size,
+ remaining - opt_size, opts)) {
+ *size += opt_size + mp_fail_size;
+ remaining -= opt_size - mp_fail_size;
return true;
}
}
@@ -1209,7 +1236,7 @@ static void mptcp_set_rwin(const struct tcp_sock *tp)
WRITE_ONCE(msk->rcv_wnd_sent, ack_seq);
}
-static u16 __mptcp_make_csum(u64 data_seq, u32 subflow_seq, u16 data_len, __sum16 sum)
+u16 __mptcp_make_csum(u64 data_seq, u32 subflow_seq, u16 data_len, __wsum sum)
{
struct csum_pseudo_header header;
__wsum csum;
@@ -1224,14 +1251,14 @@ static u16 __mptcp_make_csum(u64 data_seq, u32 subflow_seq, u16 data_len, __sum1
header.data_len = htons(data_len);
header.csum = 0;
- csum = csum_partial(&header, sizeof(header), ~csum_unfold(sum));
+ csum = csum_partial(&header, sizeof(header), sum);
return (__force u16)csum_fold(csum);
}
static u16 mptcp_make_csum(const struct mptcp_ext *mpext)
{
return __mptcp_make_csum(mpext->data_seq, mpext->subflow_seq, mpext->data_len,
- mpext->csum);
+ ~csum_unfold(mpext->csum));
}
void mptcp_write_options(__be32 *ptr, const struct tcp_sock *tp,
@@ -1251,17 +1278,8 @@ void mptcp_write_options(__be32 *ptr, const struct tcp_sock *tp,
ptr += 2;
}
- /* RST is mutually exclusive with everything else */
- if (unlikely(OPTION_MPTCP_RST & opts->suboptions)) {
- *ptr++ = mptcp_option(MPTCPOPT_RST,
- TCPOLEN_MPTCP_RST,
- opts->reset_transient,
- opts->reset_reason);
- return;
- }
-
- /* DSS, MPC, MPJ and ADD_ADDR are mutually exclusive, see
- * mptcp_established_options*()
+ /* DSS, MPC, MPJ, ADD_ADDR, FASTCLOSE and RST are mutually exclusive,
+ * see mptcp_established_options*()
*/
if (likely(OPTION_MPTCP_DSS & opts->suboptions)) {
struct mptcp_ext *mpext = &opts->ext_copy;
@@ -1316,6 +1334,7 @@ void mptcp_write_options(__be32 *ptr, const struct tcp_sock *tp,
put_unaligned_be32(mpext->data_len << 16 |
TCPOPT_NOP << 8 | TCPOPT_NOP, ptr);
}
+ ptr += 1;
}
} else if (OPTIONS_MPTCP_MPC & opts->suboptions) {
u8 len, flag = MPTCP_CAP_HMAC_SHA256;
@@ -1361,7 +1380,7 @@ void mptcp_write_options(__be32 *ptr, const struct tcp_sock *tp,
__mptcp_make_csum(opts->data_seq,
opts->subflow_seq,
opts->data_len,
- opts->csum), ptr);
+ ~csum_unfold(opts->csum)), ptr);
} else {
put_unaligned_be32(opts->data_len << 16 |
TCPOPT_NOP << 8 | TCPOPT_NOP, ptr);
@@ -1370,27 +1389,29 @@ void mptcp_write_options(__be32 *ptr, const struct tcp_sock *tp,
/* MPC is additionally mutually exclusive with MP_PRIO */
goto mp_capable_done;
- } else if (OPTION_MPTCP_MPJ_SYN & opts->suboptions) {
- *ptr++ = mptcp_option(MPTCPOPT_MP_JOIN,
- TCPOLEN_MPTCP_MPJ_SYN,
- opts->backup, opts->join_id);
- put_unaligned_be32(opts->token, ptr);
- ptr += 1;
- put_unaligned_be32(opts->nonce, ptr);
- ptr += 1;
- } else if (OPTION_MPTCP_MPJ_SYNACK & opts->suboptions) {
- *ptr++ = mptcp_option(MPTCPOPT_MP_JOIN,
- TCPOLEN_MPTCP_MPJ_SYNACK,
- opts->backup, opts->join_id);
- put_unaligned_be64(opts->thmac, ptr);
- ptr += 2;
- put_unaligned_be32(opts->nonce, ptr);
- ptr += 1;
- } else if (OPTION_MPTCP_MPJ_ACK & opts->suboptions) {
- *ptr++ = mptcp_option(MPTCPOPT_MP_JOIN,
- TCPOLEN_MPTCP_MPJ_ACK, 0, 0);
- memcpy(ptr, opts->hmac, MPTCPOPT_HMAC_LEN);
- ptr += 5;
+ } else if (OPTIONS_MPTCP_MPJ & opts->suboptions) {
+ if (OPTION_MPTCP_MPJ_SYN & opts->suboptions) {
+ *ptr++ = mptcp_option(MPTCPOPT_MP_JOIN,
+ TCPOLEN_MPTCP_MPJ_SYN,
+ opts->backup, opts->join_id);
+ put_unaligned_be32(opts->token, ptr);
+ ptr += 1;
+ put_unaligned_be32(opts->nonce, ptr);
+ ptr += 1;
+ } else if (OPTION_MPTCP_MPJ_SYNACK & opts->suboptions) {
+ *ptr++ = mptcp_option(MPTCPOPT_MP_JOIN,
+ TCPOLEN_MPTCP_MPJ_SYNACK,
+ opts->backup, opts->join_id);
+ put_unaligned_be64(opts->thmac, ptr);
+ ptr += 2;
+ put_unaligned_be32(opts->nonce, ptr);
+ ptr += 1;
+ } else {
+ *ptr++ = mptcp_option(MPTCPOPT_MP_JOIN,
+ TCPOLEN_MPTCP_MPJ_ACK, 0, 0);
+ memcpy(ptr, opts->hmac, MPTCPOPT_HMAC_LEN);
+ ptr += 5;
+ }
} else if (OPTION_MPTCP_ADD_ADDR & opts->suboptions) {
u8 len = TCPOLEN_MPTCP_ADD_ADDR_BASE;
u8 echo = MPTCP_ADDR_ECHO;
@@ -1447,6 +1468,24 @@ void mptcp_write_options(__be32 *ptr, const struct tcp_sock *tp,
ptr += 1;
}
}
+ } else if (unlikely(OPTION_MPTCP_FASTCLOSE & opts->suboptions)) {
+ /* FASTCLOSE is mutually exclusive with others except RST */
+ *ptr++ = mptcp_option(MPTCPOPT_MP_FASTCLOSE,
+ TCPOLEN_MPTCP_FASTCLOSE,
+ 0, 0);
+ put_unaligned_be64(opts->rcvr_key, ptr);
+ ptr += 2;
+
+ if (OPTION_MPTCP_RST & opts->suboptions)
+ goto mp_rst;
+ return;
+ } else if (unlikely(OPTION_MPTCP_RST & opts->suboptions)) {
+mp_rst:
+ *ptr++ = mptcp_option(MPTCPOPT_RST,
+ TCPOLEN_MPTCP_RST,
+ opts->reset_transient,
+ opts->reset_reason);
+ return;
}
if (OPTION_MPTCP_PRIO & opts->suboptions) {
diff --git a/net/mptcp/pm.c b/net/mptcp/pm.c
index 6ab386f..696b2c4 100644
--- a/net/mptcp/pm.c
+++ b/net/mptcp/pm.c
@@ -172,9 +172,28 @@ void mptcp_pm_subflow_established(struct mptcp_sock *msk)
spin_unlock_bh(&pm->lock);
}
-void mptcp_pm_subflow_closed(struct mptcp_sock *msk, u8 id)
+void mptcp_pm_subflow_check_next(struct mptcp_sock *msk, const struct sock *ssk,
+ const struct mptcp_subflow_context *subflow)
{
- pr_debug("msk=%p", msk);
+ struct mptcp_pm_data *pm = &msk->pm;
+ bool update_subflows;
+
+ update_subflows = (ssk->sk_state == TCP_CLOSE) &&
+ (subflow->request_join || subflow->mp_join);
+ if (!READ_ONCE(pm->work_pending) && !update_subflows)
+ return;
+
+ spin_lock_bh(&pm->lock);
+ if (update_subflows)
+ pm->subflows--;
+
+ /* Even if this subflow is not really established, tell the PM to try
+ * to pick the next ones, if possible.
+ */
+ if (mptcp_pm_nl_check_work_pending(msk))
+ mptcp_pm_schedule_work(msk, MPTCP_PM_SUBFLOW_ESTABLISHED);
+
+ spin_unlock_bh(&pm->lock);
}
void mptcp_pm_add_addr_received(struct mptcp_sock *msk,
@@ -356,7 +375,7 @@ void mptcp_pm_subflow_chk_stale(const struct mptcp_sock *msk, struct sock *ssk)
}
}
-void mptcp_pm_data_init(struct mptcp_sock *msk)
+void mptcp_pm_data_reset(struct mptcp_sock *msk)
{
msk->pm.add_addr_signaled = 0;
msk->pm.add_addr_accepted = 0;
@@ -370,13 +389,18 @@ void mptcp_pm_data_init(struct mptcp_sock *msk)
WRITE_ONCE(msk->pm.accept_subflow, false);
WRITE_ONCE(msk->pm.remote_deny_join_id0, false);
msk->pm.status = 0;
-
- spin_lock_init(&msk->pm.lock);
- INIT_LIST_HEAD(&msk->pm.anno_list);
+ bitmap_fill(msk->pm.id_avail_bitmap, MPTCP_PM_MAX_ADDR_ID + 1);
mptcp_pm_nl_data_init(msk);
}
+void mptcp_pm_data_init(struct mptcp_sock *msk)
+{
+ spin_lock_init(&msk->pm.lock);
+ INIT_LIST_HEAD(&msk->pm.anno_list);
+ mptcp_pm_data_reset(msk);
+}
+
void __init mptcp_pm_init(void)
{
mptcp_pm_nl_init();
diff --git a/net/mptcp/pm_netlink.c b/net/mptcp/pm_netlink.c
index f523051..75af1f7 100644
--- a/net/mptcp/pm_netlink.c
+++ b/net/mptcp/pm_netlink.c
@@ -38,9 +38,6 @@ struct mptcp_pm_add_entry {
u8 retrans_times;
};
-#define MAX_ADDR_ID 255
-#define BITMAP_SZ DIV_ROUND_UP(MAX_ADDR_ID + 1, BITS_PER_LONG)
-
struct pm_nl_pernet {
/* protects pernet updates */
spinlock_t lock;
@@ -52,14 +49,14 @@ struct pm_nl_pernet {
unsigned int local_addr_max;
unsigned int subflows_max;
unsigned int next_id;
- unsigned long id_bitmap[BITMAP_SZ];
+ DECLARE_BITMAP(id_bitmap, MPTCP_PM_MAX_ADDR_ID + 1);
};
#define MPTCP_PM_ADDR_MAX 8
#define ADD_ADDR_RETRANS_MAX 3
static bool addresses_equal(const struct mptcp_addr_info *a,
- struct mptcp_addr_info *b, bool use_port)
+ const struct mptcp_addr_info *b, bool use_port)
{
bool addr_equals = false;
@@ -168,11 +165,13 @@ select_local_address(const struct pm_nl_pernet *pernet,
msk_owned_by_me(msk);
rcu_read_lock();
- __mptcp_flush_join_list(msk);
list_for_each_entry_rcu(entry, &pernet->local_addr_list, list) {
if (!(entry->flags & MPTCP_PM_ADDR_FLAG_SUBFLOW))
continue;
+ if (!test_bit(entry->addr.id, msk->pm.id_avail_bitmap))
+ continue;
+
if (entry->addr.family != sk->sk_family) {
#if IS_ENABLED(CONFIG_MPTCP_IPV6)
if ((entry->addr.family == AF_INET &&
@@ -183,23 +182,17 @@ select_local_address(const struct pm_nl_pernet *pernet,
continue;
}
- /* avoid any address already in use by subflows and
- * pending join
- */
- if (!lookup_subflow_by_saddr(&msk->conn_list, &entry->addr)) {
- ret = entry;
- break;
- }
+ ret = entry;
+ break;
}
rcu_read_unlock();
return ret;
}
static struct mptcp_pm_addr_entry *
-select_signal_address(struct pm_nl_pernet *pernet, unsigned int pos)
+select_signal_address(struct pm_nl_pernet *pernet, struct mptcp_sock *msk)
{
struct mptcp_pm_addr_entry *entry, *ret = NULL;
- int i = 0;
rcu_read_lock();
/* do not keep any additional per socket state, just signal
@@ -208,12 +201,14 @@ select_signal_address(struct pm_nl_pernet *pernet, unsigned int pos)
* can lead to additional addresses not being announced.
*/
list_for_each_entry_rcu(entry, &pernet->local_addr_list, list) {
+ if (!test_bit(entry->addr.id, msk->pm.id_avail_bitmap))
+ continue;
+
if (!(entry->flags & MPTCP_PM_ADDR_FLAG_SIGNAL))
continue;
- if (i++ == pos) {
- ret = entry;
- break;
- }
+
+ ret = entry;
+ break;
}
rcu_read_unlock();
return ret;
@@ -255,12 +250,17 @@ unsigned int mptcp_pm_get_local_addr_max(struct mptcp_sock *msk)
}
EXPORT_SYMBOL_GPL(mptcp_pm_get_local_addr_max);
-static void check_work_pending(struct mptcp_sock *msk)
+bool mptcp_pm_nl_check_work_pending(struct mptcp_sock *msk)
{
- if (msk->pm.add_addr_signaled == mptcp_pm_get_add_addr_signal_max(msk) &&
- (msk->pm.local_addr_used == mptcp_pm_get_local_addr_max(msk) ||
- msk->pm.subflows == mptcp_pm_get_subflows_max(msk)))
+ struct pm_nl_pernet *pernet = net_generic(sock_net((struct sock *)msk), pm_nl_pernet_id);
+
+ if (msk->pm.subflows == mptcp_pm_get_subflows_max(msk) ||
+ (find_next_and_bit(pernet->id_bitmap, msk->pm.id_avail_bitmap,
+ MPTCP_PM_MAX_ADDR_ID + 1, 0) == MPTCP_PM_MAX_ADDR_ID + 1)) {
WRITE_ONCE(msk->pm.work_pending, false);
+ return false;
+ }
+ return true;
}
struct mptcp_pm_add_entry *
@@ -429,6 +429,7 @@ static bool lookup_address_in_vec(struct mptcp_addr_info *addrs, unsigned int nr
static unsigned int fill_remote_addresses_vec(struct mptcp_sock *msk, bool fullmesh,
struct mptcp_addr_info *addrs)
{
+ bool deny_id0 = READ_ONCE(msk->pm.remote_deny_join_id0);
struct sock *sk = (struct sock *)msk, *ssk;
struct mptcp_subflow_context *subflow;
struct mptcp_addr_info remote = { 0 };
@@ -436,22 +437,28 @@ static unsigned int fill_remote_addresses_vec(struct mptcp_sock *msk, bool fullm
int i = 0;
subflows_max = mptcp_pm_get_subflows_max(msk);
+ remote_address((struct sock_common *)sk, &remote);
/* Non-fullmesh endpoint, fill in the single entry
* corresponding to the primary MPC subflow remote address
*/
if (!fullmesh) {
- remote_address((struct sock_common *)sk, &remote);
+ if (deny_id0)
+ return 0;
+
msk->pm.subflows++;
addrs[i++] = remote;
} else {
mptcp_for_each_subflow(msk, subflow) {
ssk = mptcp_subflow_tcp_sock(subflow);
- remote_address((struct sock_common *)ssk, &remote);
- if (!lookup_address_in_vec(addrs, i, &remote) &&
+ remote_address((struct sock_common *)ssk, &addrs[i]);
+ if (deny_id0 && addresses_equal(&addrs[i], &remote, false))
+ continue;
+
+ if (!lookup_address_in_vec(addrs, i, &addrs[i]) &&
msk->pm.subflows < subflows_max) {
msk->pm.subflows++;
- addrs[i++] = remote;
+ i++;
}
}
}
@@ -459,6 +466,35 @@ static unsigned int fill_remote_addresses_vec(struct mptcp_sock *msk, bool fullm
return i;
}
+static struct mptcp_pm_addr_entry *
+__lookup_addr_by_id(struct pm_nl_pernet *pernet, unsigned int id)
+{
+ struct mptcp_pm_addr_entry *entry;
+
+ list_for_each_entry(entry, &pernet->local_addr_list, list) {
+ if (entry->addr.id == id)
+ return entry;
+ }
+ return NULL;
+}
+
+static int
+lookup_id_by_addr(struct pm_nl_pernet *pernet, const struct mptcp_addr_info *addr)
+{
+ struct mptcp_pm_addr_entry *entry;
+ int ret = -1;
+
+ rcu_read_lock();
+ list_for_each_entry(entry, &pernet->local_addr_list, list) {
+ if (addresses_equal(&entry->addr, addr, entry->addr.port)) {
+ ret = entry->addr.id;
+ break;
+ }
+ }
+ rcu_read_unlock();
+ return ret;
+}
+
static void mptcp_pm_create_subflow_or_signal_addr(struct mptcp_sock *msk)
{
struct sock *sk = (struct sock *)msk;
@@ -474,6 +510,19 @@ static void mptcp_pm_create_subflow_or_signal_addr(struct mptcp_sock *msk)
local_addr_max = mptcp_pm_get_local_addr_max(msk);
subflows_max = mptcp_pm_get_subflows_max(msk);
+ /* do lazy endpoint usage accounting for the MPC subflows */
+ if (unlikely(!(msk->pm.status & BIT(MPTCP_PM_MPC_ENDPOINT_ACCOUNTED))) && msk->first) {
+ struct mptcp_addr_info mpc_addr;
+ int mpc_id;
+
+ local_address((struct sock_common *)msk->first, &mpc_addr);
+ mpc_id = lookup_id_by_addr(pernet, &mpc_addr);
+ if (mpc_id >= 0)
+ __clear_bit(mpc_id, msk->pm.id_avail_bitmap);
+
+ msk->pm.status |= BIT(MPTCP_PM_MPC_ENDPOINT_ACCOUNTED);
+ }
+
pr_debug("local %d:%d signal %d:%d subflows %d:%d\n",
msk->pm.local_addr_used, local_addr_max,
msk->pm.add_addr_signaled, add_addr_signal_max,
@@ -481,47 +530,41 @@ static void mptcp_pm_create_subflow_or_signal_addr(struct mptcp_sock *msk)
/* check first for announce */
if (msk->pm.add_addr_signaled < add_addr_signal_max) {
- local = select_signal_address(pernet,
- msk->pm.add_addr_signaled);
+ local = select_signal_address(pernet, msk);
if (local) {
if (mptcp_pm_alloc_anno_list(msk, local)) {
+ __clear_bit(local->addr.id, msk->pm.id_avail_bitmap);
msk->pm.add_addr_signaled++;
mptcp_pm_announce_addr(msk, &local->addr, false);
mptcp_pm_nl_addr_send_ack(msk);
}
- } else {
- /* pick failed, avoid fourther attempts later */
- msk->pm.local_addr_used = add_addr_signal_max;
}
-
- check_work_pending(msk);
}
/* check if should create a new subflow */
- if (msk->pm.local_addr_used < local_addr_max &&
- msk->pm.subflows < subflows_max &&
- !READ_ONCE(msk->pm.remote_deny_join_id0)) {
+ while (msk->pm.local_addr_used < local_addr_max &&
+ msk->pm.subflows < subflows_max) {
+ struct mptcp_addr_info addrs[MPTCP_PM_ADDR_MAX];
+ bool fullmesh;
+ int i, nr;
+
local = select_local_address(pernet, msk);
- if (local) {
- bool fullmesh = !!(local->flags & MPTCP_PM_ADDR_FLAG_FULLMESH);
- struct mptcp_addr_info addrs[MPTCP_PM_ADDR_MAX];
- int i, nr;
+ if (!local)
+ break;
- msk->pm.local_addr_used++;
- check_work_pending(msk);
- nr = fill_remote_addresses_vec(msk, fullmesh, addrs);
- spin_unlock_bh(&msk->pm.lock);
- for (i = 0; i < nr; i++)
- __mptcp_subflow_connect(sk, &local->addr, &addrs[i]);
- spin_lock_bh(&msk->pm.lock);
- return;
- }
+ fullmesh = !!(local->flags & MPTCP_PM_ADDR_FLAG_FULLMESH);
- /* lookup failed, avoid fourther attempts later */
- msk->pm.local_addr_used = local_addr_max;
- check_work_pending(msk);
+ msk->pm.local_addr_used++;
+ nr = fill_remote_addresses_vec(msk, fullmesh, addrs);
+ if (nr)
+ __clear_bit(local->addr.id, msk->pm.id_avail_bitmap);
+ spin_unlock_bh(&msk->pm.lock);
+ for (i = 0; i < nr; i++)
+ __mptcp_subflow_connect(sk, &local->addr, &addrs[i]);
+ spin_lock_bh(&msk->pm.lock);
}
+ mptcp_pm_nl_check_work_pending(msk);
}
static void mptcp_pm_nl_fully_established(struct mptcp_sock *msk)
@@ -551,7 +594,6 @@ static unsigned int fill_local_addresses_vec(struct mptcp_sock *msk,
subflows_max = mptcp_pm_get_subflows_max(msk);
rcu_read_lock();
- __mptcp_flush_join_list(msk);
list_for_each_entry_rcu(entry, &pernet->local_addr_list, list) {
if (!(entry->flags & MPTCP_PM_ADDR_FLAG_FULLMESH))
continue;
@@ -640,7 +682,6 @@ void mptcp_pm_nl_addr_send_ack(struct mptcp_sock *msk)
!mptcp_pm_should_rm_signal(msk))
return;
- __mptcp_flush_join_list(msk);
subflow = list_first_entry_or_null(&msk->conn_list, typeof(*subflow), node);
if (subflow) {
struct sock *ssk = mptcp_subflow_tcp_sock(subflow);
@@ -710,6 +751,8 @@ static void mptcp_pm_nl_rm_addr_or_subflow(struct mptcp_sock *msk,
return;
for (i = 0; i < rm_list->nr; i++) {
+ bool removed = false;
+
list_for_each_entry_safe(subflow, tmp, &msk->conn_list, node) {
struct sock *ssk = mptcp_subflow_tcp_sock(subflow);
int how = RCV_SHUTDOWN | SEND_SHUTDOWN;
@@ -726,18 +769,24 @@ static void mptcp_pm_nl_rm_addr_or_subflow(struct mptcp_sock *msk,
i, rm_list->ids[i], subflow->local_id, subflow->remote_id);
spin_unlock_bh(&msk->pm.lock);
mptcp_subflow_shutdown(sk, ssk, how);
+
+ /* the following takes care of updating the subflows counter */
mptcp_close_ssk(sk, ssk, subflow);
spin_lock_bh(&msk->pm.lock);
- if (rm_type == MPTCP_MIB_RMADDR) {
- msk->pm.add_addr_accepted--;
- WRITE_ONCE(msk->pm.accept_addr, true);
- } else if (rm_type == MPTCP_MIB_RMSUBFLOW) {
- msk->pm.local_addr_used--;
- }
- msk->pm.subflows--;
+ removed = true;
__MPTCP_INC_STATS(sock_net(sk), rm_type);
}
+ __set_bit(rm_list->ids[1], msk->pm.id_avail_bitmap);
+ if (!removed)
+ continue;
+
+ if (rm_type == MPTCP_MIB_RMADDR) {
+ msk->pm.add_addr_accepted--;
+ WRITE_ONCE(msk->pm.accept_addr, true);
+ } else if (rm_type == MPTCP_MIB_RMSUBFLOW) {
+ msk->pm.local_addr_used--;
+ }
}
}
@@ -758,6 +807,9 @@ void mptcp_pm_nl_work(struct mptcp_sock *msk)
msk_owned_by_me(msk);
+ if (!(pm->status & MPTCP_PM_WORK_MASK))
+ return;
+
spin_lock_bh(&msk->pm.lock);
pr_debug("msk=%p status=%x", msk, pm->status);
@@ -803,7 +855,7 @@ static int mptcp_pm_nl_append_new_local_addr(struct pm_nl_pernet *pernet,
/* to keep the code simple, don't do IDR-like allocation for address ID,
* just bail when we exceed limits
*/
- if (pernet->next_id == MAX_ADDR_ID)
+ if (pernet->next_id == MPTCP_PM_MAX_ADDR_ID)
pernet->next_id = 1;
if (pernet->addrs >= MPTCP_PM_ADDR_MAX)
goto out;
@@ -823,16 +875,15 @@ static int mptcp_pm_nl_append_new_local_addr(struct pm_nl_pernet *pernet,
if (!entry->addr.id) {
find_next:
entry->addr.id = find_next_zero_bit(pernet->id_bitmap,
- MAX_ADDR_ID + 1,
+ MPTCP_PM_MAX_ADDR_ID + 1,
pernet->next_id);
- if ((!entry->addr.id || entry->addr.id > MAX_ADDR_ID) &&
- pernet->next_id != 1) {
+ if (!entry->addr.id && pernet->next_id != 1) {
pernet->next_id = 1;
goto find_next;
}
}
- if (!entry->addr.id || entry->addr.id > MAX_ADDR_ID)
+ if (!entry->addr.id)
goto out;
__set_bit(entry->addr.id, pernet->id_bitmap);
@@ -1191,18 +1242,6 @@ static int mptcp_nl_cmd_add_addr(struct sk_buff *skb, struct genl_info *info)
return 0;
}
-static struct mptcp_pm_addr_entry *
-__lookup_addr_by_id(struct pm_nl_pernet *pernet, unsigned int id)
-{
- struct mptcp_pm_addr_entry *entry;
-
- list_for_each_entry(entry, &pernet->local_addr_list, list) {
- if (entry->addr.id == id)
- return entry;
- }
- return NULL;
-}
-
int mptcp_pm_get_flags_and_ifindex_by_id(struct net *net, unsigned int id,
u8 *flags, int *ifindex)
{
@@ -1461,7 +1500,7 @@ static int mptcp_nl_cmd_flush_addrs(struct sk_buff *skb, struct genl_info *info)
list_splice_init(&pernet->local_addr_list, &free_list);
__reset_counters(pernet);
pernet->next_id = 1;
- bitmap_zero(pernet->id_bitmap, MAX_ADDR_ID + 1);
+ bitmap_zero(pernet->id_bitmap, MPTCP_PM_MAX_ADDR_ID + 1);
spin_unlock_bh(&pernet->lock);
mptcp_nl_remove_addrs_list(sock_net(skb->sk), &free_list);
synchronize_rcu();
@@ -1571,7 +1610,7 @@ static int mptcp_nl_cmd_dump_addrs(struct sk_buff *msg,
pernet = net_generic(net, pm_nl_pernet_id);
spin_lock_bh(&pernet->lock);
- for (i = id; i < MAX_ADDR_ID + 1; i++) {
+ for (i = id; i < MPTCP_PM_MAX_ADDR_ID + 1; i++) {
if (test_bit(i, pernet->id_bitmap)) {
entry = __lookup_addr_by_id(pernet, i);
if (!entry)
@@ -1705,22 +1744,28 @@ static int mptcp_nl_addr_backup(struct net *net,
static int mptcp_nl_cmd_set_flags(struct sk_buff *skb, struct genl_info *info)
{
+ struct mptcp_pm_addr_entry addr = { .addr = { .family = AF_UNSPEC }, }, *entry;
struct nlattr *attr = info->attrs[MPTCP_PM_ATTR_ADDR];
struct pm_nl_pernet *pernet = genl_info_pm_nl(info);
- struct mptcp_pm_addr_entry addr, *entry;
struct net *net = sock_net(skb->sk);
- u8 bkup = 0;
+ u8 bkup = 0, lookup_by_id = 0;
int ret;
- ret = mptcp_pm_parse_addr(attr, info, true, &addr);
+ ret = mptcp_pm_parse_addr(attr, info, false, &addr);
if (ret < 0)
return ret;
if (addr.flags & MPTCP_PM_ADDR_FLAG_BACKUP)
bkup = 1;
+ if (addr.addr.family == AF_UNSPEC) {
+ lookup_by_id = 1;
+ if (!addr.addr.id)
+ return -EOPNOTSUPP;
+ }
list_for_each_entry(entry, &pernet->local_addr_list, list) {
- if (addresses_equal(&entry->addr, &addr.addr, true)) {
+ if ((!lookup_by_id && addresses_equal(&entry->addr, &addr.addr, true)) ||
+ (lookup_by_id && entry->addr.id == addr.addr.id)) {
mptcp_nl_addr_backup(net, &entry->addr, bkup);
if (bkup)
diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c
index 54613f5..f60f01b 100644
--- a/net/mptcp/protocol.c
+++ b/net/mptcp/protocol.c
@@ -22,6 +22,7 @@
#endif
#include <net/mptcp.h>
#include <net/xfrm.h>
+#include <asm/ioctls.h>
#include "protocol.h"
#include "mib.h"
@@ -46,9 +47,10 @@ struct mptcp_skb_cb {
enum {
MPTCP_CMSG_TS = BIT(0),
+ MPTCP_CMSG_INQ = BIT(1),
};
-static struct percpu_counter mptcp_sockets_allocated;
+static struct percpu_counter mptcp_sockets_allocated ____cacheline_aligned_in_smp;
static void __mptcp_destroy_sock(struct sock *sk);
static void __mptcp_check_send_data_fin(struct sock *sk);
@@ -738,6 +740,7 @@ static bool __mptcp_ofo_queue(struct mptcp_sock *msk)
MPTCP_SKB_CB(skb)->map_seq, msk->ack_seq,
delta);
MPTCP_SKB_CB(skb)->offset += delta;
+ MPTCP_SKB_CB(skb)->map_seq += delta;
__skb_queue_tail(&sk->sk_receive_queue, skb);
}
msk->ack_seq = end_seq;
@@ -760,7 +763,7 @@ static bool move_skbs_to_msk(struct mptcp_sock *msk, struct sock *ssk)
if (!sock_owned_by_user(sk))
__mptcp_error_report(sk);
else
- set_bit(MPTCP_ERROR_REPORT, &msk->flags);
+ __set_bit(MPTCP_ERROR_REPORT, &msk->cb_flags);
}
/* If the moves have caught up with the DATA_FIN sequence number
@@ -805,47 +808,38 @@ void mptcp_data_ready(struct sock *sk, struct sock *ssk)
mptcp_data_unlock(sk);
}
-static bool mptcp_do_flush_join_list(struct mptcp_sock *msk)
+static bool __mptcp_finish_join(struct mptcp_sock *msk, struct sock *ssk)
{
- struct mptcp_subflow_context *subflow;
- bool ret = false;
+ struct sock *sk = (struct sock *)msk;
- if (likely(list_empty(&msk->join_list)))
+ if (sk->sk_state != TCP_ESTABLISHED)
return false;
- spin_lock_bh(&msk->join_list_lock);
- list_for_each_entry(subflow, &msk->join_list, node) {
- u32 sseq = READ_ONCE(subflow->setsockopt_seq);
+ /* attach to msk socket only after we are sure we will deal with it
+ * at close time
+ */
+ if (sk->sk_socket && !ssk->sk_socket)
+ mptcp_sock_graft(ssk, sk->sk_socket);
- mptcp_propagate_sndbuf((struct sock *)msk, mptcp_subflow_tcp_sock(subflow));
- if (READ_ONCE(msk->setsockopt_seq) != sseq)
- ret = true;
+ mptcp_propagate_sndbuf((struct sock *)msk, ssk);
+ mptcp_sockopt_sync_locked(msk, ssk);
+ return true;
+}
+
+static void __mptcp_flush_join_list(struct sock *sk)
+{
+ struct mptcp_subflow_context *tmp, *subflow;
+ struct mptcp_sock *msk = mptcp_sk(sk);
+
+ list_for_each_entry_safe(subflow, tmp, &msk->join_list, node) {
+ struct sock *ssk = mptcp_subflow_tcp_sock(subflow);
+ bool slow = lock_sock_fast(ssk);
+
+ list_move_tail(&subflow->node, &msk->conn_list);
+ if (!__mptcp_finish_join(msk, ssk))
+ mptcp_subflow_reset(ssk);
+ unlock_sock_fast(ssk, slow);
}
- list_splice_tail_init(&msk->join_list, &msk->conn_list);
- spin_unlock_bh(&msk->join_list_lock);
-
- return ret;
-}
-
-void __mptcp_flush_join_list(struct mptcp_sock *msk)
-{
- if (likely(!mptcp_do_flush_join_list(msk)))
- return;
-
- if (!test_and_set_bit(MPTCP_WORK_SYNC_SETSOCKOPT, &msk->flags))
- mptcp_schedule_work((struct sock *)msk);
-}
-
-static void mptcp_flush_join_list(struct mptcp_sock *msk)
-{
- bool sync_needed = test_and_clear_bit(MPTCP_WORK_SYNC_SETSOCKOPT, &msk->flags);
-
- might_sleep();
-
- if (!mptcp_do_flush_join_list(msk) && !sync_needed)
- return;
-
- mptcp_sockopt_sync_all(msk);
}
static bool mptcp_timer_pending(struct sock *sk)
@@ -972,7 +966,9 @@ static void __mptcp_mem_reclaim_partial(struct sock *sk)
lockdep_assert_held_once(&sk->sk_lock.slock);
- __mptcp_rmem_reclaim(sk, reclaimable - 1);
+ if (reclaimable > SK_MEM_QUANTUM)
+ __mptcp_rmem_reclaim(sk, reclaimable - 1);
+
sk_mem_reclaim_partial(sk);
}
@@ -1369,7 +1365,7 @@ static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk,
struct subflow_send_info {
struct sock *ssk;
- u64 ratio;
+ u64 linger_time;
};
void mptcp_subflow_set_active(struct mptcp_subflow_context *subflow)
@@ -1394,20 +1390,24 @@ bool mptcp_subflow_active(struct mptcp_subflow_context *subflow)
return __mptcp_subflow_active(subflow);
}
+#define SSK_MODE_ACTIVE 0
+#define SSK_MODE_BACKUP 1
+#define SSK_MODE_MAX 2
+
/* implement the mptcp packet scheduler;
* returns the subflow that will transmit the next DSS
* additionally updates the rtx timeout
*/
static struct sock *mptcp_subflow_get_send(struct mptcp_sock *msk)
{
- struct subflow_send_info send_info[2];
+ struct subflow_send_info send_info[SSK_MODE_MAX];
struct mptcp_subflow_context *subflow;
struct sock *sk = (struct sock *)msk;
+ u32 pace, burst, wmem;
int i, nr_active = 0;
struct sock *ssk;
+ u64 linger_time;
long tout = 0;
- u64 ratio;
- u32 pace;
sock_owned_by_me(sk);
@@ -1426,10 +1426,11 @@ static struct sock *mptcp_subflow_get_send(struct mptcp_sock *msk)
}
/* pick the subflow with the lower wmem/wspace ratio */
- for (i = 0; i < 2; ++i) {
+ for (i = 0; i < SSK_MODE_MAX; ++i) {
send_info[i].ssk = NULL;
- send_info[i].ratio = -1;
+ send_info[i].linger_time = -1;
}
+
mptcp_for_each_subflow(msk, subflow) {
trace_mptcp_subflow_get_send(subflow);
ssk = mptcp_subflow_tcp_sock(subflow);
@@ -1438,34 +1439,51 @@ static struct sock *mptcp_subflow_get_send(struct mptcp_sock *msk)
tout = max(tout, mptcp_timeout_from_subflow(subflow));
nr_active += !subflow->backup;
- if (!sk_stream_memory_free(subflow->tcp_sock) || !tcp_sk(ssk)->snd_wnd)
- continue;
+ pace = subflow->avg_pacing_rate;
+ if (unlikely(!pace)) {
+ /* init pacing rate from socket */
+ subflow->avg_pacing_rate = READ_ONCE(ssk->sk_pacing_rate);
+ pace = subflow->avg_pacing_rate;
+ if (!pace)
+ continue;
+ }
- pace = READ_ONCE(ssk->sk_pacing_rate);
- if (!pace)
- continue;
-
- ratio = div_u64((u64)READ_ONCE(ssk->sk_wmem_queued) << 32,
- pace);
- if (ratio < send_info[subflow->backup].ratio) {
+ linger_time = div_u64((u64)READ_ONCE(ssk->sk_wmem_queued) << 32, pace);
+ if (linger_time < send_info[subflow->backup].linger_time) {
send_info[subflow->backup].ssk = ssk;
- send_info[subflow->backup].ratio = ratio;
+ send_info[subflow->backup].linger_time = linger_time;
}
}
__mptcp_set_timeout(sk, tout);
/* pick the best backup if no other subflow is active */
if (!nr_active)
- send_info[0].ssk = send_info[1].ssk;
+ send_info[SSK_MODE_ACTIVE].ssk = send_info[SSK_MODE_BACKUP].ssk;
- if (send_info[0].ssk) {
- msk->last_snd = send_info[0].ssk;
- msk->snd_burst = min_t(int, MPTCP_SEND_BURST_SIZE,
- tcp_sk(msk->last_snd)->snd_wnd);
- return msk->last_snd;
- }
+ /* According to the blest algorithm, to avoid HoL blocking for the
+ * faster flow, we need to:
+ * - estimate the faster flow linger time
+ * - use the above to estimate the amount of byte transferred
+ * by the faster flow
+ * - check that the amount of queued data is greter than the above,
+ * otherwise do not use the picked, slower, subflow
+ * We select the subflow with the shorter estimated time to flush
+ * the queued mem, which basically ensure the above. We just need
+ * to check that subflow has a non empty cwin.
+ */
+ ssk = send_info[SSK_MODE_ACTIVE].ssk;
+ if (!ssk || !sk_stream_memory_free(ssk) || !tcp_sk(ssk)->snd_wnd)
+ return NULL;
- return NULL;
+ burst = min_t(int, MPTCP_SEND_BURST_SIZE, tcp_sk(ssk)->snd_wnd);
+ wmem = READ_ONCE(ssk->sk_wmem_queued);
+ subflow = mptcp_subflow_ctx(ssk);
+ subflow->avg_pacing_rate = div_u64((u64)subflow->avg_pacing_rate * wmem +
+ READ_ONCE(ssk->sk_pacing_rate) * burst,
+ burst + wmem);
+ msk->last_snd = ssk;
+ msk->snd_burst = burst;
+ return ssk;
}
static void mptcp_push_release(struct sock *ssk, struct mptcp_sendmsg_info *info)
@@ -1499,11 +1517,10 @@ static void mptcp_update_post_push(struct mptcp_sock *msk,
msk->snd_nxt = snd_nxt_new;
}
-static void mptcp_check_and_set_pending(struct sock *sk)
+void mptcp_check_and_set_pending(struct sock *sk)
{
- if (mptcp_send_head(sk) &&
- !test_bit(MPTCP_PUSH_PENDING, &mptcp_sk(sk)->flags))
- set_bit(MPTCP_PUSH_PENDING, &mptcp_sk(sk)->flags);
+ if (mptcp_send_head(sk))
+ mptcp_sk(sk)->push_pending |= BIT(MPTCP_PUSH_PENDING);
}
void __mptcp_push_pending(struct sock *sk, unsigned int flags)
@@ -1524,7 +1541,6 @@ void __mptcp_push_pending(struct sock *sk, unsigned int flags)
int ret = 0;
prev_ssk = ssk;
- __mptcp_flush_join_list(msk);
ssk = mptcp_subflow_get_send(msk);
/* First check. If the ssk has changed since
@@ -1784,8 +1800,10 @@ static int __mptcp_recvmsg_mskq(struct mptcp_sock *msk,
copied += count;
if (count < data_len) {
- if (!(flags & MSG_PEEK))
+ if (!(flags & MSG_PEEK)) {
MPTCP_SKB_CB(skb)->offset += count;
+ MPTCP_SKB_CB(skb)->map_seq += count;
+ }
break;
}
@@ -1927,7 +1945,6 @@ static bool __mptcp_move_skbs(struct mptcp_sock *msk)
unsigned int moved = 0;
bool ret, done;
- mptcp_flush_join_list(msk);
do {
struct sock *ssk = mptcp_subflow_recv_lookup(msk);
bool slowpath;
@@ -1965,6 +1982,27 @@ static bool __mptcp_move_skbs(struct mptcp_sock *msk)
return !skb_queue_empty(&msk->receive_queue);
}
+static unsigned int mptcp_inq_hint(const struct sock *sk)
+{
+ const struct mptcp_sock *msk = mptcp_sk(sk);
+ const struct sk_buff *skb;
+
+ skb = skb_peek(&msk->receive_queue);
+ if (skb) {
+ u64 hint_val = msk->ack_seq - MPTCP_SKB_CB(skb)->map_seq;
+
+ if (hint_val >= INT_MAX)
+ return INT_MAX;
+
+ return (unsigned int)hint_val;
+ }
+
+ if (sk->sk_state == TCP_CLOSE || (sk->sk_shutdown & RCV_SHUTDOWN))
+ return 1;
+
+ return 0;
+}
+
static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len,
int nonblock, int flags, int *addr_len)
{
@@ -1989,6 +2027,9 @@ static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len,
len = min_t(size_t, len, INT_MAX);
target = sock_rcvlowat(sk, flags & MSG_WAITALL, len);
+ if (unlikely(msk->recvmsg_inq))
+ cmsg_flags = MPTCP_CMSG_INQ;
+
while (copied < len) {
int bytes_read;
@@ -2062,6 +2103,12 @@ static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len,
if (cmsg_flags && copied >= 0) {
if (cmsg_flags & MPTCP_CMSG_TS)
tcp_recv_timestamp(msg, sk, &tss);
+
+ if (cmsg_flags & MPTCP_CMSG_INQ) {
+ unsigned int inq = mptcp_inq_hint(sk);
+
+ put_cmsg(msg, SOL_TCP, TCP_CM_INQ, sizeof(inq), &inq);
+ }
}
pr_debug("msk=%p rx queue empty=%d:%d copied=%d",
@@ -2088,7 +2135,7 @@ static void mptcp_retransmit_timer(struct timer_list *t)
mptcp_schedule_work(sk);
} else {
/* delegate our work to tcp_release_cb() */
- set_bit(MPTCP_RETRANSMIT, &msk->flags);
+ __set_bit(MPTCP_RETRANSMIT, &msk->cb_flags);
}
bh_unlock_sock(sk);
sock_put(sk);
@@ -2196,6 +2243,10 @@ bool __mptcp_retransmit_pending_data(struct sock *sk)
return true;
}
+/* flags for __mptcp_close_ssk() */
+#define MPTCP_CF_PUSH BIT(1)
+#define MPTCP_CF_FASTCLOSE BIT(2)
+
/* subflow sockets can be either outgoing (connect) or incoming
* (accept).
*
@@ -2205,22 +2256,37 @@ bool __mptcp_retransmit_pending_data(struct sock *sk)
* parent socket.
*/
static void __mptcp_close_ssk(struct sock *sk, struct sock *ssk,
- struct mptcp_subflow_context *subflow)
+ struct mptcp_subflow_context *subflow,
+ unsigned int flags)
{
struct mptcp_sock *msk = mptcp_sk(sk);
- bool need_push;
+ bool need_push, dispose_it;
- list_del(&subflow->node);
+ dispose_it = !msk->subflow || ssk != msk->subflow->sk;
+ if (dispose_it)
+ list_del(&subflow->node);
lock_sock_nested(ssk, SINGLE_DEPTH_NESTING);
+ if (flags & MPTCP_CF_FASTCLOSE)
+ subflow->send_fastclose = 1;
+
+ need_push = (flags & MPTCP_CF_PUSH) && __mptcp_retransmit_pending_data(sk);
+ if (!dispose_it) {
+ tcp_disconnect(ssk, 0);
+ msk->subflow->state = SS_UNCONNECTED;
+ mptcp_subflow_ctx_reset(subflow);
+ release_sock(ssk);
+
+ goto out;
+ }
+
/* if we are invoked by the msk cleanup code, the subflow is
* already orphaned
*/
if (ssk->sk_socket)
sock_orphan(ssk);
- need_push = __mptcp_retransmit_pending_data(sk);
subflow->disposable = 1;
/* if ssk hit tcp_done(), tcp_cleanup_ulp() cleared the related ops
@@ -2240,14 +2306,12 @@ static void __mptcp_close_ssk(struct sock *sk, struct sock *ssk,
sock_put(ssk);
- if (ssk == msk->last_snd)
- msk->last_snd = NULL;
-
if (ssk == msk->first)
msk->first = NULL;
- if (msk->subflow && ssk == msk->subflow->sk)
- mptcp_dispose_initial_subflow(msk);
+out:
+ if (ssk == msk->last_snd)
+ msk->last_snd = NULL;
if (need_push)
__mptcp_push_pending(sk, 0);
@@ -2258,7 +2322,13 @@ void mptcp_close_ssk(struct sock *sk, struct sock *ssk,
{
if (sk->sk_state == TCP_ESTABLISHED)
mptcp_event(MPTCP_EVENT_SUB_CLOSED, mptcp_sk(sk), ssk, GFP_KERNEL);
- __mptcp_close_ssk(sk, ssk, subflow);
+
+ /* subflow aborted before reaching the fully_established status
+ * attempt the creation of the next subflow
+ */
+ mptcp_pm_subflow_check_next(mptcp_sk(sk), ssk, subflow);
+
+ __mptcp_close_ssk(sk, ssk, subflow, MPTCP_CF_PUSH);
}
static unsigned int mptcp_sync_mss(struct sock *sk, u32 pmtu)
@@ -2410,12 +2480,10 @@ static void mptcp_worker(struct work_struct *work)
goto unlock;
mptcp_check_data_fin_ack(sk);
- mptcp_flush_join_list(msk);
mptcp_check_fastclose(msk);
- if (msk->pm.status)
- mptcp_pm_nl_work(msk);
+ mptcp_pm_nl_work(msk);
if (test_and_clear_bit(MPTCP_WORK_EOF, &msk->flags))
mptcp_check_for_eof(msk);
@@ -2449,8 +2517,6 @@ static int __mptcp_init_sock(struct sock *sk)
{
struct mptcp_sock *msk = mptcp_sk(sk);
- spin_lock_init(&msk->join_list_lock);
-
INIT_LIST_HEAD(&msk->conn_list);
INIT_LIST_HEAD(&msk->join_list);
INIT_LIST_HEAD(&msk->rtx_queue);
@@ -2476,9 +2542,20 @@ static int __mptcp_init_sock(struct sock *sk)
return 0;
}
-static int mptcp_init_sock(struct sock *sk)
+static void mptcp_ca_reset(struct sock *sk)
{
struct inet_connection_sock *icsk = inet_csk(sk);
+
+ tcp_assign_congestion_control(sk);
+ strcpy(mptcp_sk(sk)->ca_name, icsk->icsk_ca_ops->name);
+
+ /* no need to keep a reference to the ops, the name will suffice */
+ tcp_cleanup_congestion_control(sk);
+ icsk->icsk_ca_ops = NULL;
+}
+
+static int mptcp_init_sock(struct sock *sk)
+{
struct net *net = sock_net(sk);
int ret;
@@ -2499,12 +2576,7 @@ static int mptcp_init_sock(struct sock *sk)
/* fetch the ca name; do it outside __mptcp_init_sock(), so that clone will
* propagate the correct value
*/
- tcp_assign_congestion_control(sk);
- strcpy(mptcp_sk(sk)->ca_name, icsk->icsk_ca_ops->name);
-
- /* no need to keep a reference to the ops, the name will suffice */
- tcp_cleanup_congestion_control(sk);
- icsk->icsk_ca_ops = NULL;
+ mptcp_ca_reset(sk);
sk_sockets_allocated_inc(sk);
sk->sk_rcvbuf = sock_net(sk)->ipv4.sysctl_tcp_rmem[1];
@@ -2609,6 +2681,7 @@ static void __mptcp_check_send_data_fin(struct sock *sk)
* state now
*/
if (__mptcp_check_fallback(msk)) {
+ WRITE_ONCE(msk->snd_una, msk->write_seq);
if ((1 << sk->sk_state) & (TCPF_CLOSING | TCPF_LAST_ACK)) {
inet_sk_state_store(sk, TCP_CLOSE);
mptcp_close_wake_up(sk);
@@ -2617,7 +2690,6 @@ static void __mptcp_check_send_data_fin(struct sock *sk)
}
}
- mptcp_flush_join_list(msk);
mptcp_for_each_subflow(msk, subflow) {
struct sock *tcp_sk = mptcp_subflow_tcp_sock(subflow);
@@ -2650,21 +2722,20 @@ static void __mptcp_destroy_sock(struct sock *sk)
might_sleep();
- /* be sure to always acquire the join list lock, to sync vs
- * mptcp_finish_join().
- */
- spin_lock_bh(&msk->join_list_lock);
- list_splice_tail_init(&msk->join_list, &msk->conn_list);
- spin_unlock_bh(&msk->join_list_lock);
+ /* join list will be eventually flushed (with rst) at sock lock release time*/
list_splice_init(&msk->conn_list, &conn_list);
sk_stop_timer(sk, &msk->sk.icsk_retransmit_timer);
sk_stop_timer(sk, &sk->sk_timer);
msk->pm.status = 0;
+ /* clears msk->subflow, allowing the following loop to close
+ * even the initial subflow
+ */
+ mptcp_dispose_initial_subflow(msk);
list_for_each_entry_safe(subflow, tmp, &conn_list, node) {
struct sock *ssk = mptcp_subflow_tcp_sock(subflow);
- __mptcp_close_ssk(sk, ssk, subflow);
+ __mptcp_close_ssk(sk, ssk, subflow, 0);
}
sk->sk_prot->destroy(sk);
@@ -2675,7 +2746,6 @@ static void __mptcp_destroy_sock(struct sock *sk)
xfrm_sk_free_policy(sk);
sk_refcnt_debug_release(sk);
- mptcp_dispose_initial_subflow(msk);
sock_put(sk);
}
@@ -2711,6 +2781,9 @@ static void mptcp_close(struct sock *sk, long timeout)
sock_hold(sk);
pr_debug("msk=%p state=%d", sk, sk->sk_state);
+ if (mptcp_sk(sk)->token)
+ mptcp_event(MPTCP_EVENT_CLOSED, mptcp_sk(sk), NULL, GFP_KERNEL);
+
if (sk->sk_state == TCP_CLOSE) {
__mptcp_destroy_sock(sk);
do_cancel_work = true;
@@ -2721,9 +2794,6 @@ static void mptcp_close(struct sock *sk, long timeout)
if (do_cancel_work)
mptcp_cancel_work(sk);
- if (mptcp_sk(sk)->token)
- mptcp_event(MPTCP_EVENT_CLOSED, mptcp_sk(sk), NULL, GFP_KERNEL);
-
sock_put(sk);
}
@@ -2755,15 +2825,38 @@ static int mptcp_disconnect(struct sock *sk, int flags)
struct mptcp_subflow_context *subflow;
struct mptcp_sock *msk = mptcp_sk(sk);
- mptcp_do_flush_join_list(msk);
+ inet_sk_state_store(sk, TCP_CLOSE);
mptcp_for_each_subflow(msk, subflow) {
struct sock *ssk = mptcp_subflow_tcp_sock(subflow);
- lock_sock(ssk);
- tcp_disconnect(ssk, flags);
- release_sock(ssk);
+ __mptcp_close_ssk(sk, ssk, subflow, MPTCP_CF_FASTCLOSE);
}
+
+ sk_stop_timer(sk, &msk->sk.icsk_retransmit_timer);
+ sk_stop_timer(sk, &sk->sk_timer);
+
+ if (mptcp_sk(sk)->token)
+ mptcp_event(MPTCP_EVENT_CLOSED, mptcp_sk(sk), NULL, GFP_KERNEL);
+
+ mptcp_destroy_common(msk);
+ msk->last_snd = NULL;
+ WRITE_ONCE(msk->flags, 0);
+ msk->cb_flags = 0;
+ msk->push_pending = 0;
+ msk->recovery = false;
+ msk->can_ack = false;
+ msk->fully_established = false;
+ msk->rcv_data_fin = false;
+ msk->snd_data_fin_enable = false;
+ msk->rcv_fastclose = false;
+ msk->use_64bit_ack = false;
+ WRITE_ONCE(msk->csum_enabled, mptcp_is_checksum_enabled(sock_net(sk)));
+ mptcp_pm_data_reset(msk);
+ mptcp_ca_reset(sk);
+
+ sk->sk_shutdown = 0;
+ sk_error_report(sk);
return 0;
}
@@ -2903,9 +2996,11 @@ void mptcp_destroy_common(struct mptcp_sock *msk)
__mptcp_clear_xmit(sk);
/* move to sk_receive_queue, sk_stream_kill_queues will purge it */
+ mptcp_data_lock(sk);
skb_queue_splice_tail_init(&msk->receive_queue, &sk->sk_receive_queue);
__skb_queue_purge(&sk->sk_receive_queue);
skb_rbtree_purge(&msk->out_of_order_queue);
+ mptcp_data_unlock(sk);
/* move all the rx fwd alloc into the sk_mem_reclaim_final in
* inet_sock_destruct() will dispose it
@@ -2929,7 +3024,7 @@ void __mptcp_data_acked(struct sock *sk)
if (!sock_owned_by_user(sk))
__mptcp_clean_una(sk);
else
- set_bit(MPTCP_CLEAN_UNA, &mptcp_sk(sk)->flags);
+ __set_bit(MPTCP_CLEAN_UNA, &mptcp_sk(sk)->cb_flags);
if (mptcp_pending_data_fin_ack(sk))
mptcp_schedule_work(sk);
@@ -2948,20 +3043,23 @@ void __mptcp_check_push(struct sock *sk, struct sock *ssk)
else if (xmit_ssk)
mptcp_subflow_delegate(mptcp_subflow_ctx(xmit_ssk), MPTCP_DELEGATE_SEND);
} else {
- set_bit(MPTCP_PUSH_PENDING, &mptcp_sk(sk)->flags);
+ __set_bit(MPTCP_PUSH_PENDING, &mptcp_sk(sk)->cb_flags);
}
}
+#define MPTCP_FLAGS_PROCESS_CTX_NEED (BIT(MPTCP_PUSH_PENDING) | \
+ BIT(MPTCP_RETRANSMIT) | \
+ BIT(MPTCP_FLUSH_JOIN_LIST))
+
/* processes deferred events and flush wmem */
static void mptcp_release_cb(struct sock *sk)
+ __must_hold(&sk->sk_lock.slock)
{
- for (;;) {
- unsigned long flags = 0;
+ struct mptcp_sock *msk = mptcp_sk(sk);
- if (test_and_clear_bit(MPTCP_PUSH_PENDING, &mptcp_sk(sk)->flags))
- flags |= BIT(MPTCP_PUSH_PENDING);
- if (test_and_clear_bit(MPTCP_RETRANSMIT, &mptcp_sk(sk)->flags))
- flags |= BIT(MPTCP_RETRANSMIT);
+ for (;;) {
+ unsigned long flags = (msk->cb_flags & MPTCP_FLAGS_PROCESS_CTX_NEED) |
+ msk->push_pending;
if (!flags)
break;
@@ -2972,8 +3070,11 @@ static void mptcp_release_cb(struct sock *sk)
* datapath acquires the msk socket spinlock while helding
* the subflow socket lock
*/
-
+ msk->push_pending = 0;
+ msk->cb_flags &= ~flags;
spin_unlock_bh(&sk->sk_lock.slock);
+ if (flags & BIT(MPTCP_FLUSH_JOIN_LIST))
+ __mptcp_flush_join_list(sk);
if (flags & BIT(MPTCP_PUSH_PENDING))
__mptcp_push_pending(sk, 0);
if (flags & BIT(MPTCP_RETRANSMIT))
@@ -2986,11 +3087,11 @@ static void mptcp_release_cb(struct sock *sk)
/* be sure to set the current sk state before tacking actions
* depending on sk_state
*/
- if (test_and_clear_bit(MPTCP_CONNECTED, &mptcp_sk(sk)->flags))
+ if (__test_and_clear_bit(MPTCP_CONNECTED, &msk->cb_flags))
__mptcp_set_connected(sk);
- if (test_and_clear_bit(MPTCP_CLEAN_UNA, &mptcp_sk(sk)->flags))
+ if (__test_and_clear_bit(MPTCP_CLEAN_UNA, &msk->cb_flags))
__mptcp_clean_una_wakeup(sk);
- if (test_and_clear_bit(MPTCP_ERROR_REPORT, &mptcp_sk(sk)->flags))
+ if (__test_and_clear_bit(MPTCP_ERROR_REPORT, &msk->cb_flags))
__mptcp_error_report(sk);
__mptcp_update_rmem(sk);
@@ -3032,7 +3133,7 @@ void mptcp_subflow_process_delegated(struct sock *ssk)
if (!sock_owned_by_user(sk))
__mptcp_subflow_push_pending(sk, ssk);
else
- set_bit(MPTCP_PUSH_PENDING, &mptcp_sk(sk)->flags);
+ __set_bit(MPTCP_PUSH_PENDING, &mptcp_sk(sk)->cb_flags);
mptcp_data_unlock(sk);
mptcp_subflow_delegated_done(subflow, MPTCP_DELEGATE_SEND);
}
@@ -3118,8 +3219,7 @@ bool mptcp_finish_join(struct sock *ssk)
struct mptcp_subflow_context *subflow = mptcp_subflow_ctx(ssk);
struct mptcp_sock *msk = mptcp_sk(subflow->conn);
struct sock *parent = (void *)msk;
- struct socket *parent_sock;
- bool ret;
+ bool ret = true;
pr_debug("msk=%p, subflow=%p", msk, subflow);
@@ -3132,35 +3232,38 @@ bool mptcp_finish_join(struct sock *ssk)
if (!msk->pm.server_side)
goto out;
- if (!mptcp_pm_allow_new_subflow(msk)) {
- subflow->reset_reason = MPTCP_RST_EPROHIBIT;
- return false;
- }
+ if (!mptcp_pm_allow_new_subflow(msk))
+ goto err_prohibited;
- /* active connections are already on conn_list, and we can't acquire
- * msk lock here.
- * use the join list lock as synchronization point and double-check
- * msk status to avoid racing with __mptcp_destroy_sock()
+ if (WARN_ON_ONCE(!list_empty(&subflow->node)))
+ goto err_prohibited;
+
+ /* active connections are already on conn_list.
+ * If we can't acquire msk socket lock here, let the release callback
+ * handle it
*/
- spin_lock_bh(&msk->join_list_lock);
- ret = inet_sk_state_load(parent) == TCP_ESTABLISHED;
- if (ret && !WARN_ON_ONCE(!list_empty(&subflow->node))) {
- list_add_tail(&subflow->node, &msk->join_list);
+ mptcp_data_lock(parent);
+ if (!sock_owned_by_user(parent)) {
+ ret = __mptcp_finish_join(msk, ssk);
+ if (ret) {
+ sock_hold(ssk);
+ list_add_tail(&subflow->node, &msk->conn_list);
+ }
+ } else {
sock_hold(ssk);
+ list_add_tail(&subflow->node, &msk->join_list);
+ __set_bit(MPTCP_FLUSH_JOIN_LIST, &msk->cb_flags);
}
- spin_unlock_bh(&msk->join_list_lock);
+ mptcp_data_unlock(parent);
+
if (!ret) {
+err_prohibited:
subflow->reset_reason = MPTCP_RST_EPROHIBIT;
return false;
}
- /* attach to msk socket only after we are sure he will deal with us
- * at close time
- */
- parent_sock = READ_ONCE(parent->sk_socket);
- if (parent_sock && !ssk->sk_socket)
- mptcp_sock_graft(ssk, parent_sock);
subflow->map_seq = READ_ONCE(msk->ack_seq);
+
out:
mptcp_event(MPTCP_EVENT_SUB_ESTABLISHED, msk, ssk, GFP_ATOMIC);
return true;
@@ -3179,6 +3282,57 @@ static int mptcp_forward_alloc_get(const struct sock *sk)
return sk->sk_forward_alloc + mptcp_sk(sk)->rmem_fwd_alloc;
}
+static int mptcp_ioctl_outq(const struct mptcp_sock *msk, u64 v)
+{
+ const struct sock *sk = (void *)msk;
+ u64 delta;
+
+ if (sk->sk_state == TCP_LISTEN)
+ return -EINVAL;
+
+ if ((1 << sk->sk_state) & (TCPF_SYN_SENT | TCPF_SYN_RECV))
+ return 0;
+
+ delta = msk->write_seq - v;
+ if (delta > INT_MAX)
+ delta = INT_MAX;
+
+ return (int)delta;
+}
+
+static int mptcp_ioctl(struct sock *sk, int cmd, unsigned long arg)
+{
+ struct mptcp_sock *msk = mptcp_sk(sk);
+ bool slow;
+ int answ;
+
+ switch (cmd) {
+ case SIOCINQ:
+ if (sk->sk_state == TCP_LISTEN)
+ return -EINVAL;
+
+ lock_sock(sk);
+ __mptcp_move_skbs(msk);
+ answ = mptcp_inq_hint(sk);
+ release_sock(sk);
+ break;
+ case SIOCOUTQ:
+ slow = lock_sock_fast(sk);
+ answ = mptcp_ioctl_outq(msk, READ_ONCE(msk->snd_una));
+ unlock_sock_fast(sk, slow);
+ break;
+ case SIOCOUTQNSD:
+ slow = lock_sock_fast(sk);
+ answ = mptcp_ioctl_outq(msk, msk->snd_nxt);
+ unlock_sock_fast(sk, slow);
+ break;
+ default:
+ return -ENOIOCTLCMD;
+ }
+
+ return put_user(answ, (int __user *)arg);
+}
+
static struct proto mptcp_prot = {
.name = "MPTCP",
.owner = THIS_MODULE,
@@ -3191,6 +3345,7 @@ static struct proto mptcp_prot = {
.shutdown = mptcp_shutdown,
.destroy = mptcp_destroy,
.sendmsg = mptcp_sendmsg,
+ .ioctl = mptcp_ioctl,
.recvmsg = mptcp_recvmsg,
.release_cb = mptcp_release_cb,
.hash = mptcp_hash,
@@ -3243,9 +3398,20 @@ static int mptcp_stream_connect(struct socket *sock, struct sockaddr *uaddr,
struct mptcp_sock *msk = mptcp_sk(sock->sk);
struct mptcp_subflow_context *subflow;
struct socket *ssock;
- int err;
+ int err = -EINVAL;
lock_sock(sock->sk);
+ if (uaddr) {
+ if (addr_len < sizeof(uaddr->sa_family))
+ goto unlock;
+
+ if (uaddr->sa_family == AF_UNSPEC) {
+ err = mptcp_disconnect(sock->sk, flags);
+ sock->state = err ? SS_DISCONNECTING : SS_UNCONNECTED;
+ goto unlock;
+ }
+ }
+
if (sock->state != SS_UNCONNECTED && msk->subflow) {
/* pending connection or invalid state, let existing subflow
* cope with that
@@ -3255,10 +3421,8 @@ static int mptcp_stream_connect(struct socket *sock, struct sockaddr *uaddr,
}
ssock = __mptcp_nmpc_socket(msk);
- if (!ssock) {
- err = -EINVAL;
+ if (!ssock)
goto unlock;
- }
mptcp_token_destroy(msk);
inet_sk_state_store(sock->sk, TCP_SYN_SENT);
@@ -3332,17 +3496,9 @@ static int mptcp_stream_accept(struct socket *sock, struct socket *newsock,
pr_debug("msk=%p", msk);
- lock_sock(sock->sk);
- if (sock->sk->sk_state != TCP_LISTEN)
- goto unlock_fail;
-
ssock = __mptcp_nmpc_socket(msk);
if (!ssock)
- goto unlock_fail;
-
- clear_bit(MPTCP_DATA_READY, &msk->flags);
- sock_hold(ssock->sk);
- release_sock(sock->sk);
+ return -EINVAL;
err = ssock->ops->accept(sock, newsock, flags, kern);
if (err == 0 && !mptcp_is_tcpsk(newsock->sk)) {
@@ -3372,7 +3528,6 @@ static int mptcp_stream_accept(struct socket *sock, struct socket *newsock,
/* set ssk->sk_socket of accept()ed flows to mptcp socket.
* This is needed so NOSPACE flag can be set from tcp stack.
*/
- mptcp_flush_join_list(msk);
mptcp_for_each_subflow(msk, subflow) {
struct sock *ssk = mptcp_subflow_tcp_sock(subflow);
@@ -3382,14 +3537,7 @@ static int mptcp_stream_accept(struct socket *sock, struct socket *newsock,
release_sock(newsk);
}
- if (inet_csk_listen_poll(ssock->sk))
- set_bit(MPTCP_DATA_READY, &msk->flags);
- sock_put(ssock->sk);
return err;
-
-unlock_fail:
- release_sock(sock->sk);
- return -EINVAL;
}
static __poll_t mptcp_check_readable(struct mptcp_sock *msk)
@@ -3435,8 +3583,12 @@ static __poll_t mptcp_poll(struct file *file, struct socket *sock,
state = inet_sk_state_load(sk);
pr_debug("msk=%p state=%d flags=%lx", msk, state, msk->flags);
- if (state == TCP_LISTEN)
- return test_bit(MPTCP_DATA_READY, &msk->flags) ? EPOLLIN | EPOLLRDNORM : 0;
+ if (state == TCP_LISTEN) {
+ if (WARN_ON_ONCE(!msk->subflow || !msk->subflow->sk))
+ return 0;
+
+ return inet_csk_listen_poll(msk->subflow->sk);
+ }
if (state != TCP_SYN_SENT && state != TCP_SYN_RECV) {
mask |= mptcp_check_readable(msk);
diff --git a/net/mptcp/protocol.h b/net/mptcp/protocol.h
index d87cc04..0e6b42c 100644
--- a/net/mptcp/protocol.h
+++ b/net/mptcp/protocol.h
@@ -110,19 +110,20 @@
/* MPTCP TCPRST flags */
#define MPTCP_RST_TRANSIENT BIT(0)
-/* MPTCP socket flags */
-#define MPTCP_DATA_READY 0
+/* MPTCP socket atomic flags */
#define MPTCP_NOSPACE 1
#define MPTCP_WORK_RTX 2
#define MPTCP_WORK_EOF 3
#define MPTCP_FALLBACK_DONE 4
#define MPTCP_WORK_CLOSE_SUBFLOW 5
-#define MPTCP_PUSH_PENDING 6
-#define MPTCP_CLEAN_UNA 7
-#define MPTCP_ERROR_REPORT 8
-#define MPTCP_RETRANSMIT 9
-#define MPTCP_WORK_SYNC_SETSOCKOPT 10
-#define MPTCP_CONNECTED 11
+
+/* MPTCP socket release cb flags */
+#define MPTCP_PUSH_PENDING 1
+#define MPTCP_CLEAN_UNA 2
+#define MPTCP_ERROR_REPORT 3
+#define MPTCP_RETRANSMIT 4
+#define MPTCP_FLUSH_JOIN_LIST 5
+#define MPTCP_CONNECTED 6
static inline bool before64(__u64 seq1, __u64 seq2)
{
@@ -174,16 +175,25 @@ enum mptcp_pm_status {
MPTCP_PM_ADD_ADDR_SEND_ACK,
MPTCP_PM_RM_ADDR_RECEIVED,
MPTCP_PM_ESTABLISHED,
- MPTCP_PM_ALREADY_ESTABLISHED, /* persistent status, set after ESTABLISHED event */
MPTCP_PM_SUBFLOW_ESTABLISHED,
+ MPTCP_PM_ALREADY_ESTABLISHED, /* persistent status, set after ESTABLISHED event */
+ MPTCP_PM_MPC_ENDPOINT_ACCOUNTED /* persistent status, set after MPC local address is
+ * accounted int id_avail_bitmap
+ */
};
+/* Status bits below MPTCP_PM_ALREADY_ESTABLISHED need pm worker actions */
+#define MPTCP_PM_WORK_MASK ((1 << MPTCP_PM_ALREADY_ESTABLISHED) - 1)
+
enum mptcp_addr_signal_status {
MPTCP_ADD_ADDR_SIGNAL,
MPTCP_ADD_ADDR_ECHO,
MPTCP_RM_ADDR_SIGNAL,
};
+/* max value of mptcp_addr_info.id */
+#define MPTCP_PM_MAX_ADDR_ID U8_MAX
+
struct mptcp_pm_data {
struct mptcp_addr_info local;
struct mptcp_addr_info remote;
@@ -202,6 +212,7 @@ struct mptcp_pm_data {
u8 local_addr_used;
u8 subflows;
u8 status;
+ DECLARE_BITMAP(id_avail_bitmap, MPTCP_PM_MAX_ADDR_ID + 1);
struct mptcp_rm_list rm_list_tx;
struct mptcp_rm_list rm_list_rx;
};
@@ -241,6 +252,8 @@ struct mptcp_sock {
u32 token;
int rmem_released;
unsigned long flags;
+ unsigned long cb_flags;
+ unsigned long push_pending;
bool recovery; /* closing subflow write queue reinjected */
bool can_ack;
bool fully_established;
@@ -249,7 +262,9 @@ struct mptcp_sock {
bool rcv_fastclose;
bool use_64bit_ack; /* Set when we received a 64-bit DSN */
bool csum_enabled;
- spinlock_t join_list_lock;
+ u8 recvmsg_inq:1,
+ cork:1,
+ nodelay:1;
struct work_struct work;
struct sk_buff *ooo_last_skb;
struct rb_root out_of_order_queue;
@@ -392,6 +407,10 @@ DECLARE_PER_CPU(struct mptcp_delegated_action, mptcp_delegated_actions);
/* MPTCP subflow context */
struct mptcp_subflow_context {
struct list_head node;/* conn_list of subflows */
+
+ char reset_start[0];
+
+ unsigned long avg_pacing_rate; /* protected by msk socket lock */
u64 local_key;
u64 remote_key;
u64 idsn;
@@ -419,6 +438,7 @@ struct mptcp_subflow_context {
backup : 1,
send_mp_prio : 1,
send_mp_fail : 1,
+ send_fastclose : 1,
rx_eof : 1,
can_ack : 1, /* only after processing the remote a key */
disposable : 1, /* ctx can be free at ulp release time */
@@ -437,6 +457,9 @@ struct mptcp_subflow_context {
u8 stale_count;
long delegated_status;
+
+ char reset_end[0];
+
struct list_head delegated_node; /* link into delegated_action, protected by local BH */
u32 setsockopt_seq;
@@ -468,6 +491,13 @@ mptcp_subflow_tcp_sock(const struct mptcp_subflow_context *subflow)
return subflow->tcp_sock;
}
+static inline void
+mptcp_subflow_ctx_reset(struct mptcp_subflow_context *subflow)
+{
+ memset(subflow->reset_start, 0, subflow->reset_end - subflow->reset_start);
+ subflow->request_mptcp = 1;
+}
+
static inline u64
mptcp_subflow_get_map_offset(const struct mptcp_subflow_context *subflow)
{
@@ -482,15 +512,6 @@ mptcp_subflow_get_mapped_dsn(const struct mptcp_subflow_context *subflow)
return subflow->map_seq + mptcp_subflow_get_map_offset(subflow);
}
-static inline void mptcp_add_pending_subflow(struct mptcp_sock *msk,
- struct mptcp_subflow_context *subflow)
-{
- sock_hold(mptcp_subflow_tcp_sock(subflow));
- spin_lock_bh(&msk->join_list_lock);
- list_add_tail(&subflow->node, &msk->join_list);
- spin_unlock_bh(&msk->join_list_lock);
-}
-
void mptcp_subflow_process_delegated(struct sock *ssk);
static inline void mptcp_subflow_delegate(struct mptcp_subflow_context *subflow, int action)
@@ -554,6 +575,7 @@ unsigned int mptcp_stale_loss_cnt(const struct net *net);
void mptcp_subflow_fully_established(struct mptcp_subflow_context *subflow,
struct mptcp_options_received *mp_opt);
bool __mptcp_retransmit_pending_data(struct sock *sk);
+void mptcp_check_and_set_pending(struct sock *sk);
void __mptcp_push_pending(struct sock *sk, unsigned int flags);
bool mptcp_subflow_data_available(struct sock *sk);
void __init mptcp_subflow_init(void);
@@ -654,7 +676,6 @@ void __mptcp_data_acked(struct sock *sk);
void __mptcp_error_report(struct sock *sk);
void mptcp_subflow_eof(struct sock *sk);
bool mptcp_update_rcv_data_fin(struct mptcp_sock *msk, u64 data_fin_seq, bool use_64bit);
-void __mptcp_flush_join_list(struct mptcp_sock *msk);
static inline bool mptcp_data_fin_enabled(const struct mptcp_sock *msk)
{
return READ_ONCE(msk->snd_data_fin_enable) &&
@@ -704,9 +725,11 @@ void mptcp_token_destroy(struct mptcp_sock *msk);
void mptcp_crypto_key_sha(u64 key, u32 *token, u64 *idsn);
void mptcp_crypto_hmac_sha(u64 key1, u64 key2, u8 *msg, int len, void *hmac);
+u16 __mptcp_make_csum(u64 data_seq, u32 subflow_seq, u16 data_len, __wsum sum);
void __init mptcp_pm_init(void);
void mptcp_pm_data_init(struct mptcp_sock *msk);
+void mptcp_pm_data_reset(struct mptcp_sock *msk);
void mptcp_pm_subflow_chk_stale(const struct mptcp_sock *msk, struct sock *ssk);
void mptcp_pm_nl_subflow_chk_stale(const struct mptcp_sock *msk, struct sock *ssk);
void mptcp_pm_new_connection(struct mptcp_sock *msk, const struct sock *ssk, int server_side);
@@ -714,7 +737,9 @@ void mptcp_pm_fully_established(struct mptcp_sock *msk, const struct sock *ssk,
bool mptcp_pm_allow_new_subflow(struct mptcp_sock *msk);
void mptcp_pm_connection_closed(struct mptcp_sock *msk);
void mptcp_pm_subflow_established(struct mptcp_sock *msk);
-void mptcp_pm_subflow_closed(struct mptcp_sock *msk, u8 id);
+bool mptcp_pm_nl_check_work_pending(struct mptcp_sock *msk);
+void mptcp_pm_subflow_check_next(struct mptcp_sock *msk, const struct sock *ssk,
+ const struct mptcp_subflow_context *subflow);
void mptcp_pm_add_addr_received(struct mptcp_sock *msk,
const struct mptcp_addr_info *addr);
void mptcp_pm_add_addr_echoed(struct mptcp_sock *msk,
@@ -811,7 +836,7 @@ unsigned int mptcp_pm_get_subflows_max(struct mptcp_sock *msk);
unsigned int mptcp_pm_get_local_addr_max(struct mptcp_sock *msk);
void mptcp_sockopt_sync(struct mptcp_sock *msk, struct sock *ssk);
-void mptcp_sockopt_sync_all(struct mptcp_sock *msk);
+void mptcp_sockopt_sync_locked(struct mptcp_sock *msk, struct sock *ssk);
static inline struct mptcp_ext *mptcp_get_ext(const struct sk_buff *skb)
{
diff --git a/net/mptcp/sockopt.c b/net/mptcp/sockopt.c
index f8efd47..dacf3ce 100644
--- a/net/mptcp/sockopt.c
+++ b/net/mptcp/sockopt.c
@@ -390,6 +390,8 @@ static int mptcp_setsockopt_v6(struct mptcp_sock *msk, int optname,
switch (optname) {
case IPV6_V6ONLY:
+ case IPV6_TRANSPARENT:
+ case IPV6_FREEBIND:
lock_sock(sk);
ssock = __mptcp_nmpc_socket(msk);
if (!ssock) {
@@ -398,8 +400,24 @@ static int mptcp_setsockopt_v6(struct mptcp_sock *msk, int optname,
}
ret = tcp_setsockopt(ssock->sk, SOL_IPV6, optname, optval, optlen);
- if (ret == 0)
+ if (ret != 0) {
+ release_sock(sk);
+ return ret;
+ }
+
+ sockopt_seq_inc(msk);
+
+ switch (optname) {
+ case IPV6_V6ONLY:
sk->sk_ipv6only = ssock->sk->sk_ipv6only;
+ break;
+ case IPV6_TRANSPARENT:
+ inet_sk(sk)->transparent = inet_sk(ssock->sk)->transparent;
+ break;
+ case IPV6_FREEBIND:
+ inet_sk(sk)->freebind = inet_sk(ssock->sk)->freebind;
+ break;
+ }
release_sock(sk);
break;
@@ -538,6 +556,7 @@ static bool mptcp_supported_sockopt(int level, int optname)
case TCP_TIMESTAMP:
case TCP_NOTSENT_LOWAT:
case TCP_TX_DELAY:
+ case TCP_INQ:
return true;
}
@@ -549,7 +568,6 @@ static bool mptcp_supported_sockopt(int level, int optname)
/* TCP_FASTOPEN_KEY, TCP_FASTOPEN TCP_FASTOPEN_CONNECT, TCP_FASTOPEN_NO_COOKIE,
* are not supported fastopen is currently unsupported
*/
- /* TCP_INQ is currently unsupported, needs some recvmsg work */
}
return false;
}
@@ -597,14 +615,171 @@ static int mptcp_setsockopt_sol_tcp_congestion(struct mptcp_sock *msk, sockptr_t
return ret;
}
+static int mptcp_setsockopt_sol_tcp_cork(struct mptcp_sock *msk, sockptr_t optval,
+ unsigned int optlen)
+{
+ struct mptcp_subflow_context *subflow;
+ struct sock *sk = (struct sock *)msk;
+ int val;
+
+ if (optlen < sizeof(int))
+ return -EINVAL;
+
+ if (copy_from_sockptr(&val, optval, sizeof(val)))
+ return -EFAULT;
+
+ lock_sock(sk);
+ sockopt_seq_inc(msk);
+ msk->cork = !!val;
+ mptcp_for_each_subflow(msk, subflow) {
+ struct sock *ssk = mptcp_subflow_tcp_sock(subflow);
+
+ lock_sock(ssk);
+ __tcp_sock_set_cork(ssk, !!val);
+ release_sock(ssk);
+ }
+ if (!val)
+ mptcp_check_and_set_pending(sk);
+ release_sock(sk);
+
+ return 0;
+}
+
+static int mptcp_setsockopt_sol_tcp_nodelay(struct mptcp_sock *msk, sockptr_t optval,
+ unsigned int optlen)
+{
+ struct mptcp_subflow_context *subflow;
+ struct sock *sk = (struct sock *)msk;
+ int val;
+
+ if (optlen < sizeof(int))
+ return -EINVAL;
+
+ if (copy_from_sockptr(&val, optval, sizeof(val)))
+ return -EFAULT;
+
+ lock_sock(sk);
+ sockopt_seq_inc(msk);
+ msk->nodelay = !!val;
+ mptcp_for_each_subflow(msk, subflow) {
+ struct sock *ssk = mptcp_subflow_tcp_sock(subflow);
+
+ lock_sock(ssk);
+ __tcp_sock_set_nodelay(ssk, !!val);
+ release_sock(ssk);
+ }
+ if (val)
+ mptcp_check_and_set_pending(sk);
+ release_sock(sk);
+
+ return 0;
+}
+
+static int mptcp_setsockopt_sol_ip_set_transparent(struct mptcp_sock *msk, int optname,
+ sockptr_t optval, unsigned int optlen)
+{
+ struct sock *sk = (struct sock *)msk;
+ struct inet_sock *issk;
+ struct socket *ssock;
+ int err;
+
+ err = ip_setsockopt(sk, SOL_IP, optname, optval, optlen);
+ if (err != 0)
+ return err;
+
+ lock_sock(sk);
+
+ ssock = __mptcp_nmpc_socket(msk);
+ if (!ssock) {
+ release_sock(sk);
+ return -EINVAL;
+ }
+
+ issk = inet_sk(ssock->sk);
+
+ switch (optname) {
+ case IP_FREEBIND:
+ issk->freebind = inet_sk(sk)->freebind;
+ break;
+ case IP_TRANSPARENT:
+ issk->transparent = inet_sk(sk)->transparent;
+ break;
+ default:
+ release_sock(sk);
+ WARN_ON_ONCE(1);
+ return -EOPNOTSUPP;
+ }
+
+ sockopt_seq_inc(msk);
+ release_sock(sk);
+ return 0;
+}
+
+static int mptcp_setsockopt_v4_set_tos(struct mptcp_sock *msk, int optname,
+ sockptr_t optval, unsigned int optlen)
+{
+ struct mptcp_subflow_context *subflow;
+ struct sock *sk = (struct sock *)msk;
+ int err, val;
+
+ err = ip_setsockopt(sk, SOL_IP, optname, optval, optlen);
+
+ if (err != 0)
+ return err;
+
+ lock_sock(sk);
+ sockopt_seq_inc(msk);
+ val = inet_sk(sk)->tos;
+ mptcp_for_each_subflow(msk, subflow) {
+ struct sock *ssk = mptcp_subflow_tcp_sock(subflow);
+
+ __ip_sock_set_tos(ssk, val);
+ }
+ release_sock(sk);
+
+ return err;
+}
+
+static int mptcp_setsockopt_v4(struct mptcp_sock *msk, int optname,
+ sockptr_t optval, unsigned int optlen)
+{
+ switch (optname) {
+ case IP_FREEBIND:
+ case IP_TRANSPARENT:
+ return mptcp_setsockopt_sol_ip_set_transparent(msk, optname, optval, optlen);
+ case IP_TOS:
+ return mptcp_setsockopt_v4_set_tos(msk, optname, optval, optlen);
+ }
+
+ return -EOPNOTSUPP;
+}
+
static int mptcp_setsockopt_sol_tcp(struct mptcp_sock *msk, int optname,
sockptr_t optval, unsigned int optlen)
{
+ struct sock *sk = (void *)msk;
+ int ret, val;
+
switch (optname) {
+ case TCP_INQ:
+ ret = mptcp_get_int_option(msk, optval, optlen, &val);
+ if (ret)
+ return ret;
+ if (val < 0 || val > 1)
+ return -EINVAL;
+
+ lock_sock(sk);
+ msk->recvmsg_inq = !!val;
+ release_sock(sk);
+ return 0;
case TCP_ULP:
return -EOPNOTSUPP;
case TCP_CONGESTION:
return mptcp_setsockopt_sol_tcp_congestion(msk, optval, optlen);
+ case TCP_CORK:
+ return mptcp_setsockopt_sol_tcp_cork(msk, optval, optlen);
+ case TCP_NODELAY:
+ return mptcp_setsockopt_sol_tcp_nodelay(msk, optval, optlen);
}
return -EOPNOTSUPP;
@@ -636,6 +811,9 @@ int mptcp_setsockopt(struct sock *sk, int level, int optname,
if (ssk)
return tcp_setsockopt(ssk, level, optname, optval, optlen);
+ if (level == SOL_IP)
+ return mptcp_setsockopt_v4(msk, optname, optval, optlen);
+
if (level == SOL_IPV6)
return mptcp_setsockopt_v6(msk, optname, optval, optlen);
@@ -931,6 +1109,35 @@ static int mptcp_getsockopt_subflow_addrs(struct mptcp_sock *msk, char __user *o
return 0;
}
+static int mptcp_put_int_option(struct mptcp_sock *msk, char __user *optval,
+ int __user *optlen, int val)
+{
+ int len;
+
+ if (get_user(len, optlen))
+ return -EFAULT;
+ if (len < 0)
+ return -EINVAL;
+
+ if (len < sizeof(int) && len > 0 && val >= 0 && val <= 255) {
+ unsigned char ucval = (unsigned char)val;
+
+ len = 1;
+ if (put_user(len, optlen))
+ return -EFAULT;
+ if (copy_to_user(optval, &ucval, 1))
+ return -EFAULT;
+ } else {
+ len = min_t(unsigned int, len, sizeof(int));
+ if (put_user(len, optlen))
+ return -EFAULT;
+ if (copy_to_user(optval, &val, len))
+ return -EFAULT;
+ }
+
+ return 0;
+}
+
static int mptcp_getsockopt_sol_tcp(struct mptcp_sock *msk, int optname,
char __user *optval, int __user *optlen)
{
@@ -941,10 +1148,29 @@ static int mptcp_getsockopt_sol_tcp(struct mptcp_sock *msk, int optname,
case TCP_CC_INFO:
return mptcp_getsockopt_first_sf_only(msk, SOL_TCP, optname,
optval, optlen);
+ case TCP_INQ:
+ return mptcp_put_int_option(msk, optval, optlen, msk->recvmsg_inq);
+ case TCP_CORK:
+ return mptcp_put_int_option(msk, optval, optlen, msk->cork);
+ case TCP_NODELAY:
+ return mptcp_put_int_option(msk, optval, optlen, msk->nodelay);
}
return -EOPNOTSUPP;
}
+static int mptcp_getsockopt_v4(struct mptcp_sock *msk, int optname,
+ char __user *optval, int __user *optlen)
+{
+ struct sock *sk = (void *)msk;
+
+ switch (optname) {
+ case IP_TOS:
+ return mptcp_put_int_option(msk, optval, optlen, inet_sk(sk)->tos);
+ }
+
+ return -EOPNOTSUPP;
+}
+
static int mptcp_getsockopt_sol_mptcp(struct mptcp_sock *msk, int optname,
char __user *optval, int __user *optlen)
{
@@ -980,6 +1206,8 @@ int mptcp_getsockopt(struct sock *sk, int level, int optname,
if (ssk)
return tcp_getsockopt(ssk, level, optname, optval, option);
+ if (level == SOL_IP)
+ return mptcp_getsockopt_v4(msk, optname, optval, option);
if (level == SOL_TCP)
return mptcp_getsockopt_sol_tcp(msk, optname, optval, option);
if (level == SOL_MPTCP)
@@ -1002,6 +1230,7 @@ static void sync_socket_options(struct mptcp_sock *msk, struct sock *ssk)
ssk->sk_priority = sk->sk_priority;
ssk->sk_bound_dev_if = sk->sk_bound_dev_if;
ssk->sk_incoming_cpu = sk->sk_incoming_cpu;
+ __ip_sock_set_tos(ssk, inet_sk(sk)->tos);
if (sk->sk_userlocks & tx_rx_locks) {
ssk->sk_userlocks |= sk->sk_userlocks & tx_rx_locks;
@@ -1027,6 +1256,11 @@ static void sync_socket_options(struct mptcp_sock *msk, struct sock *ssk)
if (inet_csk(sk)->icsk_ca_ops != inet_csk(ssk)->icsk_ca_ops)
tcp_set_congestion_control(ssk, msk->ca_name, false, true);
+ __tcp_sock_set_cork(ssk, !!msk->cork);
+ __tcp_sock_set_nodelay(ssk, !!msk->nodelay);
+
+ inet_sk(ssk)->transparent = inet_sk(sk)->transparent;
+ inet_sk(ssk)->freebind = inet_sk(sk)->freebind;
}
static void __mptcp_sockopt_sync(struct mptcp_sock *msk, struct sock *ssk)
@@ -1051,27 +1285,15 @@ void mptcp_sockopt_sync(struct mptcp_sock *msk, struct sock *ssk)
}
}
-void mptcp_sockopt_sync_all(struct mptcp_sock *msk)
+void mptcp_sockopt_sync_locked(struct mptcp_sock *msk, struct sock *ssk)
{
- struct mptcp_subflow_context *subflow;
- struct sock *sk = (struct sock *)msk;
- u32 seq;
+ struct mptcp_subflow_context *subflow = mptcp_subflow_ctx(ssk);
- seq = sockopt_seq_reset(sk);
+ msk_owned_by_me(msk);
- mptcp_for_each_subflow(msk, subflow) {
- struct sock *ssk = mptcp_subflow_tcp_sock(subflow);
- u32 sseq = READ_ONCE(subflow->setsockopt_seq);
+ if (READ_ONCE(subflow->setsockopt_seq) != msk->setsockopt_seq) {
+ sync_socket_options(msk, ssk);
- if (sseq != msk->setsockopt_seq) {
- __mptcp_sockopt_sync(msk, ssk);
- WRITE_ONCE(subflow->setsockopt_seq, seq);
- } else if (sseq != seq) {
- WRITE_ONCE(subflow->setsockopt_seq, seq);
- }
-
- cond_resched();
+ subflow->setsockopt_seq = msk->setsockopt_seq;
}
-
- msk->setsockopt_seq = seq;
}
diff --git a/net/mptcp/subflow.c b/net/mptcp/subflow.c
index 6172f38..bea47a1 100644
--- a/net/mptcp/subflow.c
+++ b/net/mptcp/subflow.c
@@ -388,7 +388,7 @@ static void mptcp_set_connected(struct sock *sk)
if (!sock_owned_by_user(sk))
__mptcp_set_connected(sk);
else
- set_bit(MPTCP_CONNECTED, &mptcp_sk(sk)->flags);
+ __set_bit(MPTCP_CONNECTED, &mptcp_sk(sk)->cb_flags);
mptcp_data_unlock(sk);
}
@@ -845,9 +845,8 @@ static enum mapping_status validate_data_csum(struct sock *ssk, struct sk_buff *
bool csum_reqd)
{
struct mptcp_subflow_context *subflow = mptcp_subflow_ctx(ssk);
- struct csum_pseudo_header header;
u32 offset, seq, delta;
- __wsum csum;
+ u16 csum;
int len;
if (!csum_reqd)
@@ -908,13 +907,11 @@ static enum mapping_status validate_data_csum(struct sock *ssk, struct sk_buff *
* while the pseudo header requires the original DSS data len,
* including that
*/
- header.data_seq = cpu_to_be64(subflow->map_seq);
- header.subflow_seq = htonl(subflow->map_subflow_seq);
- header.data_len = htons(subflow->map_data_len + subflow->map_data_fin);
- header.csum = 0;
-
- csum = csum_partial(&header, sizeof(header), subflow->map_data_csum);
- if (unlikely(csum_fold(csum))) {
+ csum = __mptcp_make_csum(subflow->map_seq,
+ subflow->map_subflow_seq,
+ subflow->map_data_len + subflow->map_data_fin,
+ subflow->map_data_csum);
+ if (unlikely(csum)) {
MPTCP_INC_STATS(sock_net(ssk), MPTCP_MIB_DATACSUMERR);
subflow->send_mp_fail = 1;
MPTCP_INC_STATS(sock_net(ssk), MPTCP_MIB_MPFAILTX);
@@ -1274,7 +1271,7 @@ static void subflow_error_report(struct sock *ssk)
if (!sock_owned_by_user(sk))
__mptcp_error_report(sk);
else
- set_bit(MPTCP_ERROR_REPORT, &mptcp_sk(sk)->flags);
+ __set_bit(MPTCP_ERROR_REPORT, &mptcp_sk(sk)->cb_flags);
mptcp_data_unlock(sk);
}
@@ -1293,7 +1290,6 @@ static void subflow_data_ready(struct sock *sk)
if (reqsk_queue_empty(&inet_csk(sk)->icsk_accept_queue))
return;
- set_bit(MPTCP_DATA_READY, &msk->flags);
parent->sk_data_ready(parent);
return;
}
@@ -1425,6 +1421,8 @@ int __mptcp_subflow_connect(struct sock *sk, const struct mptcp_addr_info *loc,
if (addr.ss_family == AF_INET6)
addrlen = sizeof(struct sockaddr_in6);
#endif
+ mptcp_sockopt_sync(msk, ssk);
+
ssk->sk_bound_dev_if = ifindex;
err = kernel_bind(sf, (struct sockaddr *)&addr, addrlen);
if (err)
@@ -1440,8 +1438,8 @@ int __mptcp_subflow_connect(struct sock *sk, const struct mptcp_addr_info *loc,
subflow->request_bkup = !!(flags & MPTCP_PM_ADDR_FLAG_BACKUP);
mptcp_info2sockaddr(remote, &addr, ssk->sk_family);
- mptcp_add_pending_subflow(msk, subflow);
- mptcp_sockopt_sync(msk, ssk);
+ sock_hold(ssk);
+ list_add_tail(&subflow->node, &msk->conn_list);
err = kernel_connect(sf, (struct sockaddr *)&addr, addrlen, O_NONBLOCK);
if (err && err != -EINPROGRESS)
goto failed_unlink;
@@ -1452,9 +1450,7 @@ int __mptcp_subflow_connect(struct sock *sk, const struct mptcp_addr_info *loc,
return err;
failed_unlink:
- spin_lock_bh(&msk->join_list_lock);
list_del(&subflow->node);
- spin_unlock_bh(&msk->join_list_lock);
sock_put(mptcp_subflow_tcp_sock(subflow));
failed:
@@ -1533,10 +1529,8 @@ int mptcp_subflow_create_socket(struct sock *sk, struct socket **new_sock)
* needs it.
*/
sf->sk->sk_net_refcnt = 1;
- get_net(net);
-#ifdef CONFIG_PROC_FS
- this_cpu_add(*net->core.sock_inuse, 1);
-#endif
+ get_net_track(net, &sf->sk->ns_tracker, GFP_KERNEL);
+ sock_inuse_add(net, 1);
err = tcp_set_ulp(sf->sk, "mptcp");
release_sock(sf->sk);
diff --git a/net/mptcp/token.c b/net/mptcp/token.c
index e581b34..f52ee7b 100644
--- a/net/mptcp/token.c
+++ b/net/mptcp/token.c
@@ -384,6 +384,7 @@ void mptcp_token_destroy(struct mptcp_sock *msk)
bucket->chain_len--;
}
spin_unlock_bh(&bucket->lock);
+ WRITE_ONCE(msk->token, 0);
}
void __init mptcp_token_init(void)
diff --git a/net/ncsi/ncsi-netlink.c b/net/ncsi/ncsi-netlink.c
index bb5f165..c189b4c 100644
--- a/net/ncsi/ncsi-netlink.c
+++ b/net/ncsi/ncsi-netlink.c
@@ -112,7 +112,11 @@ static int ncsi_write_package_info(struct sk_buff *skb,
pnest = nla_nest_start_noflag(skb, NCSI_PKG_ATTR);
if (!pnest)
return -ENOMEM;
- nla_put_u32(skb, NCSI_PKG_ATTR_ID, np->id);
+ rc = nla_put_u32(skb, NCSI_PKG_ATTR_ID, np->id);
+ if (rc) {
+ nla_nest_cancel(skb, pnest);
+ return rc;
+ }
if ((0x1 << np->id) == ndp->package_whitelist)
nla_put_flag(skb, NCSI_PKG_ATTR_FORCED);
cnest = nla_nest_start_noflag(skb, NCSI_PKG_ATTR_CHANNEL_LIST);
diff --git a/net/netfilter/Kconfig b/net/netfilter/Kconfig
index 3646fc1..ddc54b6 100644
--- a/net/netfilter/Kconfig
+++ b/net/netfilter/Kconfig
@@ -515,12 +515,6 @@
This option adds the "flow_offload" expression that you can use to
choose what flows are placed into the hardware.
-config NFT_COUNTER
- tristate "Netfilter nf_tables counter module"
- help
- This option adds the "counter" expression that you can use to
- include packet and byte counters in a rule.
-
config NFT_CONNLIMIT
tristate "Netfilter nf_tables connlimit module"
depends on NF_CONNTRACK
diff --git a/net/netfilter/Makefile b/net/netfilter/Makefile
index aab20e5..a135b1a 100644
--- a/net/netfilter/Makefile
+++ b/net/netfilter/Makefile
@@ -75,7 +75,7 @@
nf_tables_trace.o nft_immediate.o nft_cmp.o nft_range.o \
nft_bitwise.o nft_byteorder.o nft_payload.o nft_lookup.o \
nft_dynset.o nft_meta.o nft_rt.o nft_exthdr.o nft_last.o \
- nft_chain_route.o nf_tables_offload.o \
+ nft_counter.o nft_chain_route.o nf_tables_offload.o \
nft_set_hash.o nft_set_bitmap.o nft_set_rbtree.o \
nft_set_pipapo.o
@@ -100,7 +100,6 @@
obj-$(CONFIG_NFT_REJECT_INET) += nft_reject_inet.o
obj-$(CONFIG_NFT_REJECT_NETDEV) += nft_reject_netdev.o
obj-$(CONFIG_NFT_TUNNEL) += nft_tunnel.o
-obj-$(CONFIG_NFT_COUNTER) += nft_counter.o
obj-$(CONFIG_NFT_LOG) += nft_log.o
obj-$(CONFIG_NFT_MASQ) += nft_masq.o
obj-$(CONFIG_NFT_REDIR) += nft_redir.o
diff --git a/net/netfilter/core.c b/net/netfilter/core.c
index 6dec9cd..354cb47 100644
--- a/net/netfilter/core.c
+++ b/net/netfilter/core.c
@@ -666,32 +666,29 @@ EXPORT_SYMBOL(nf_hook_slow_list);
/* This needs to be compiled in any case to avoid dependencies between the
* nfnetlink_queue code and nf_conntrack.
*/
-struct nfnl_ct_hook __rcu *nfnl_ct_hook __read_mostly;
+const struct nfnl_ct_hook __rcu *nfnl_ct_hook __read_mostly;
EXPORT_SYMBOL_GPL(nfnl_ct_hook);
-struct nf_ct_hook __rcu *nf_ct_hook __read_mostly;
+const struct nf_ct_hook __rcu *nf_ct_hook __read_mostly;
EXPORT_SYMBOL_GPL(nf_ct_hook);
#if IS_ENABLED(CONFIG_NF_CONNTRACK)
-/* This does not belong here, but locally generated errors need it if connection
- tracking in use: without this, connection may not be in hash table, and hence
- manufactured ICMP or RST packets will not be associated with it. */
-void (*ip_ct_attach)(struct sk_buff *, const struct sk_buff *)
- __rcu __read_mostly;
-EXPORT_SYMBOL(ip_ct_attach);
-
-struct nf_nat_hook __rcu *nf_nat_hook __read_mostly;
+const struct nf_nat_hook __rcu *nf_nat_hook __read_mostly;
EXPORT_SYMBOL_GPL(nf_nat_hook);
+/* This does not belong here, but locally generated errors need it if connection
+ * tracking in use: without this, connection may not be in hash table, and hence
+ * manufactured ICMP or RST packets will not be associated with it.
+ */
void nf_ct_attach(struct sk_buff *new, const struct sk_buff *skb)
{
- void (*attach)(struct sk_buff *, const struct sk_buff *);
+ const struct nf_ct_hook *ct_hook;
if (skb->_nfct) {
rcu_read_lock();
- attach = rcu_dereference(ip_ct_attach);
- if (attach)
- attach(new, skb);
+ ct_hook = rcu_dereference(nf_ct_hook);
+ if (ct_hook)
+ ct_hook->attach(new, skb);
rcu_read_unlock();
}
}
@@ -699,7 +696,7 @@ EXPORT_SYMBOL(nf_ct_attach);
void nf_conntrack_destroy(struct nf_conntrack *nfct)
{
- struct nf_ct_hook *ct_hook;
+ const struct nf_ct_hook *ct_hook;
rcu_read_lock();
ct_hook = rcu_dereference(nf_ct_hook);
@@ -712,7 +709,7 @@ EXPORT_SYMBOL(nf_conntrack_destroy);
bool nf_ct_get_tuple_skb(struct nf_conntrack_tuple *dst_tuple,
const struct sk_buff *skb)
{
- struct nf_ct_hook *ct_hook;
+ const struct nf_ct_hook *ct_hook;
bool ret = false;
rcu_read_lock();
diff --git a/net/netfilter/ipvs/ip_vs_ctl.c b/net/netfilter/ipvs/ip_vs_ctl.c
index 39c523b..7f64532 100644
--- a/net/netfilter/ipvs/ip_vs_ctl.c
+++ b/net/netfilter/ipvs/ip_vs_ctl.c
@@ -960,8 +960,7 @@ __ip_vs_update_dest(struct ip_vs_service *svc, struct ip_vs_dest *dest,
* Create a destination for the given service
*/
static int
-ip_vs_new_dest(struct ip_vs_service *svc, struct ip_vs_dest_user_kern *udest,
- struct ip_vs_dest **dest_p)
+ip_vs_new_dest(struct ip_vs_service *svc, struct ip_vs_dest_user_kern *udest)
{
struct ip_vs_dest *dest;
unsigned int atype, i;
@@ -1021,8 +1020,6 @@ ip_vs_new_dest(struct ip_vs_service *svc, struct ip_vs_dest_user_kern *udest,
spin_lock_init(&dest->stats.lock);
__ip_vs_update_dest(svc, dest, udest, 1);
- *dest_p = dest;
-
LeaveFunction(2);
return 0;
@@ -1096,7 +1093,7 @@ ip_vs_add_dest(struct ip_vs_service *svc, struct ip_vs_dest_user_kern *udest)
/*
* Allocate and initialize the dest structure
*/
- ret = ip_vs_new_dest(svc, udest, &dest);
+ ret = ip_vs_new_dest(svc, udest);
}
LeaveFunction(2);
diff --git a/net/netfilter/nf_conntrack_core.c b/net/netfilter/nf_conntrack_core.c
index 4712a90..894a325 100644
--- a/net/netfilter/nf_conntrack_core.c
+++ b/net/netfilter/nf_conntrack_core.c
@@ -47,6 +47,7 @@
#include <net/netfilter/nf_conntrack_timeout.h>
#include <net/netfilter/nf_conntrack_labels.h>
#include <net/netfilter/nf_conntrack_synproxy.h>
+#include <net/netfilter/nf_conntrack_act_ct.h>
#include <net/netfilter/nf_nat.h>
#include <net/netfilter/nf_nat_helper.h>
#include <net/netns/hash.h>
@@ -189,7 +190,7 @@ EXPORT_SYMBOL_GPL(nf_conntrack_htable_size);
unsigned int nf_conntrack_max __read_mostly;
EXPORT_SYMBOL_GPL(nf_conntrack_max);
seqcount_spinlock_t nf_conntrack_generation __read_mostly;
-static siphash_key_t nf_conntrack_hash_rnd __read_mostly;
+static siphash_aligned_key_t nf_conntrack_hash_rnd;
static u32 hash_conntrack_raw(const struct nf_conntrack_tuple *tuple,
unsigned int zoneid,
@@ -482,7 +483,7 @@ EXPORT_SYMBOL_GPL(nf_ct_invert_tuple);
*/
u32 nf_ct_get_id(const struct nf_conn *ct)
{
- static __read_mostly siphash_key_t ct_id_seed;
+ static siphash_aligned_key_t ct_id_seed;
unsigned long a, b, c, d;
net_get_random_once(&ct_id_seed, sizeof(ct_id_seed));
@@ -558,7 +559,7 @@ static void nf_ct_del_from_dying_or_unconfirmed_list(struct nf_conn *ct)
#define NFCT_ALIGN(len) (((len) + NFCT_INFOMASK) & ~NFCT_INFOMASK)
-/* Released via destroy_conntrack() */
+/* Released via nf_ct_destroy() */
struct nf_conn *nf_ct_tmpl_alloc(struct net *net,
const struct nf_conntrack_zone *zone,
gfp_t flags)
@@ -585,7 +586,7 @@ struct nf_conn *nf_ct_tmpl_alloc(struct net *net,
tmpl->status = IPS_TEMPLATE;
write_pnet(&tmpl->ct_net, net);
nf_ct_zone_add(tmpl, zone);
- atomic_set(&tmpl->ct_general.use, 0);
+ refcount_set(&tmpl->ct_general.use, 1);
return tmpl;
}
@@ -612,13 +613,12 @@ static void destroy_gre_conntrack(struct nf_conn *ct)
#endif
}
-static void
-destroy_conntrack(struct nf_conntrack *nfct)
+void nf_ct_destroy(struct nf_conntrack *nfct)
{
struct nf_conn *ct = (struct nf_conn *)nfct;
- pr_debug("destroy_conntrack(%p)\n", ct);
- WARN_ON(atomic_read(&nfct->use) != 0);
+ pr_debug("%s(%p)\n", __func__, ct);
+ WARN_ON(refcount_read(&nfct->use) != 0);
if (unlikely(nf_ct_is_template(ct))) {
nf_ct_tmpl_free(ct);
@@ -643,9 +643,10 @@ destroy_conntrack(struct nf_conntrack *nfct)
if (ct->master)
nf_ct_put(ct->master);
- pr_debug("destroy_conntrack: returning ct=%p to slab\n", ct);
+ pr_debug("%s: returning ct=%p to slab\n", __func__, ct);
nf_conntrack_free(ct);
}
+EXPORT_SYMBOL(nf_ct_destroy);
static void nf_ct_delete_from_lists(struct nf_conn *ct)
{
@@ -742,7 +743,7 @@ nf_ct_match(const struct nf_conn *ct1, const struct nf_conn *ct2)
/* caller must hold rcu readlock and none of the nf_conntrack_locks */
static void nf_ct_gc_expired(struct nf_conn *ct)
{
- if (!atomic_inc_not_zero(&ct->ct_general.use))
+ if (!refcount_inc_not_zero(&ct->ct_general.use))
return;
if (nf_ct_should_gc(ct))
@@ -810,7 +811,7 @@ __nf_conntrack_find_get(struct net *net, const struct nf_conntrack_zone *zone,
* in, try to obtain a reference and re-check tuple
*/
ct = nf_ct_tuplehash_to_ctrack(h);
- if (likely(atomic_inc_not_zero(&ct->ct_general.use))) {
+ if (likely(refcount_inc_not_zero(&ct->ct_general.use))) {
if (likely(nf_ct_key_equal(h, tuple, zone, net)))
goto found;
@@ -907,7 +908,7 @@ nf_conntrack_hash_check_insert(struct nf_conn *ct)
smp_wmb();
/* The caller holds a reference to this object */
- atomic_set(&ct->ct_general.use, 2);
+ refcount_set(&ct->ct_general.use, 2);
__nf_conntrack_hash_insert(ct, hash, reply_hash);
nf_conntrack_double_unlock(hash, reply_hash);
NF_CT_STAT_INC(net, insert);
@@ -958,7 +959,7 @@ static void __nf_conntrack_insert_prepare(struct nf_conn *ct)
{
struct nf_conn_tstamp *tstamp;
- atomic_inc(&ct->ct_general.use);
+ refcount_inc(&ct->ct_general.use);
ct->status |= IPS_CONFIRMED;
/* set conntrack timestamp, if enabled. */
@@ -989,7 +990,7 @@ static int __nf_ct_resolve_clash(struct sk_buff *skb,
nf_ct_acct_merge(ct, ctinfo, loser_ct);
nf_ct_add_to_dying_list(loser_ct);
- nf_conntrack_put(&loser_ct->ct_general);
+ nf_ct_put(loser_ct);
nf_ct_set(skb, ct, ctinfo);
NF_CT_STAT_INC(net, clash_resolve);
@@ -1351,7 +1352,7 @@ static unsigned int early_drop_list(struct net *net,
nf_ct_is_dying(tmp))
continue;
- if (!atomic_inc_not_zero(&tmp->ct_general.use))
+ if (!refcount_inc_not_zero(&tmp->ct_general.use))
continue;
/* kill only if still in same netns -- might have moved due to
@@ -1469,7 +1470,7 @@ static void gc_worker(struct work_struct *work)
continue;
/* need to take reference to avoid possible races */
- if (!atomic_inc_not_zero(&tmp->ct_general.use))
+ if (!refcount_inc_not_zero(&tmp->ct_general.use))
continue;
if (gc_worker_skip_ct(tmp)) {
@@ -1562,16 +1563,14 @@ __nf_conntrack_alloc(struct net *net,
ct->status = 0;
WRITE_ONCE(ct->timeout, 0);
write_pnet(&ct->ct_net, net);
- memset(&ct->__nfct_init_offset, 0,
- offsetof(struct nf_conn, proto) -
- offsetof(struct nf_conn, __nfct_init_offset));
+ memset_after(ct, 0, __nfct_init_offset);
nf_ct_zone_add(ct, zone);
/* Because we use RCU lookups, we set ct_general.use to zero before
* this is inserted in any list.
*/
- atomic_set(&ct->ct_general.use, 0);
+ refcount_set(&ct->ct_general.use, 0);
return ct;
out:
atomic_dec(&cnet->count);
@@ -1596,7 +1595,7 @@ void nf_conntrack_free(struct nf_conn *ct)
/* A freed object has refcnt == 0, that's
* the golden rule for SLAB_TYPESAFE_BY_RCU
*/
- WARN_ON(atomic_read(&ct->ct_general.use) != 0);
+ WARN_ON(refcount_read(&ct->ct_general.use) != 0);
nf_ct_ext_destroy(ct);
kmem_cache_free(nf_conntrack_cachep, ct);
@@ -1688,8 +1687,8 @@ init_conntrack(struct net *net, struct nf_conn *tmpl,
if (!exp)
__nf_ct_try_assign_helper(ct, tmpl, GFP_ATOMIC);
- /* Now it is inserted into the unconfirmed list, bump refcount */
- nf_conntrack_get(&ct->ct_general);
+ /* Now it is inserted into the unconfirmed list, set refcount to 1. */
+ refcount_set(&ct->ct_general.use, 1);
nf_ct_add_to_unconfirmed_list(ct);
local_bh_enable();
@@ -1749,6 +1748,9 @@ resolve_normal_ct(struct nf_conn *tmpl,
return 0;
if (IS_ERR(h))
return PTR_ERR(h);
+
+ ct = nf_ct_tuplehash_to_ctrack(h);
+ ct->local_origin = state->hook == NF_INET_LOCAL_OUT;
}
ct = nf_ct_tuplehash_to_ctrack(h);
@@ -1920,7 +1922,7 @@ nf_conntrack_in(struct sk_buff *skb, const struct nf_hook_state *state)
/* Invalid: inverse of the return code tells
* the netfilter core what to do */
pr_debug("nf_conntrack_in: Can't track with proto module\n");
- nf_conntrack_put(&ct->ct_general);
+ nf_ct_put(ct);
skb->_nfct = 0;
NF_CT_STAT_INC_ATOMIC(state->net, invalid);
if (ret == -NF_DROP)
@@ -2084,9 +2086,9 @@ static int __nf_conntrack_update(struct net *net, struct sk_buff *skb,
struct nf_conn *ct,
enum ip_conntrack_info ctinfo)
{
+ const struct nf_nat_hook *nat_hook;
struct nf_conntrack_tuple_hash *h;
struct nf_conntrack_tuple tuple;
- struct nf_nat_hook *nat_hook;
unsigned int status;
int dataoff;
u16 l3num;
@@ -2299,7 +2301,7 @@ get_next_corpse(int (*iter)(struct nf_conn *i, void *data),
return NULL;
found:
- atomic_inc(&ct->ct_general.use);
+ refcount_inc(&ct->ct_general.use);
spin_unlock(lockp);
local_bh_enable();
return ct;
@@ -2454,7 +2456,6 @@ static int kill_all(struct nf_conn *i, void *data)
void nf_conntrack_cleanup_start(void)
{
conntrack_gc_work.exiting = true;
- RCU_INIT_POINTER(ip_ct_attach, NULL);
}
void nf_conntrack_cleanup_end(void)
@@ -2590,7 +2591,6 @@ int nf_conntrack_hash_resize(unsigned int hashsize)
hlist_nulls_add_head_rcu(&h->hnnode, &hash[bucket]);
}
}
- old_size = nf_conntrack_htable_size;
old_hash = nf_conntrack_hash;
nf_conntrack_hash = hash;
@@ -2629,7 +2629,7 @@ int nf_conntrack_set_hashsize(const char *val, const struct kernel_param *kp)
static __always_inline unsigned int total_extension_size(void)
{
/* remember to add new extensions below */
- BUILD_BUG_ON(NF_CT_EXT_NUM > 9);
+ BUILD_BUG_ON(NF_CT_EXT_NUM > 10);
return sizeof(struct nf_ct_ext) +
sizeof(struct nf_conn_help)
@@ -2653,6 +2653,9 @@ static __always_inline unsigned int total_extension_size(void)
#if IS_ENABLED(CONFIG_NETFILTER_SYNPROXY)
+ sizeof(struct nf_conn_synproxy)
#endif
+#if IS_ENABLED(CONFIG_NET_ACT_CT)
+ + sizeof(struct nf_conn_act_ct_ext)
+#endif
;
};
@@ -2770,16 +2773,15 @@ int nf_conntrack_init_start(void)
return ret;
}
-static struct nf_ct_hook nf_conntrack_hook = {
+static const struct nf_ct_hook nf_conntrack_hook = {
.update = nf_conntrack_update,
- .destroy = destroy_conntrack,
+ .destroy = nf_ct_destroy,
.get_tuple_skb = nf_conntrack_get_tuple_skb,
+ .attach = nf_conntrack_attach,
};
void nf_conntrack_init_end(void)
{
- /* For use by REJECT target */
- RCU_INIT_POINTER(ip_ct_attach, nf_conntrack_attach);
RCU_INIT_POINTER(nf_ct_hook, &nf_conntrack_hook);
}
diff --git a/net/netfilter/nf_conntrack_expect.c b/net/netfilter/nf_conntrack_expect.c
index f562eee..96948e9 100644
--- a/net/netfilter/nf_conntrack_expect.c
+++ b/net/netfilter/nf_conntrack_expect.c
@@ -41,7 +41,7 @@ EXPORT_SYMBOL_GPL(nf_ct_expect_hash);
unsigned int nf_ct_expect_max __read_mostly;
static struct kmem_cache *nf_ct_expect_cachep __read_mostly;
-static siphash_key_t nf_ct_expect_hashrnd __read_mostly;
+static siphash_aligned_key_t nf_ct_expect_hashrnd;
/* nf_conntrack_expect helper functions */
void nf_ct_unlink_expect_report(struct nf_conntrack_expect *exp,
@@ -203,12 +203,12 @@ nf_ct_find_expectation(struct net *net,
* about to invoke ->destroy(), or nf_ct_delete() via timeout
* or early_drop().
*
- * The atomic_inc_not_zero() check tells: If that fails, we
+ * The refcount_inc_not_zero() check tells: If that fails, we
* know that the ct is being destroyed. If it succeeds, we
* can be sure the ct cannot disappear underneath.
*/
if (unlikely(nf_ct_is_dying(exp->master) ||
- !atomic_inc_not_zero(&exp->master->ct_general.use)))
+ !refcount_inc_not_zero(&exp->master->ct_general.use)))
return NULL;
if (exp->flags & NF_CT_EXPECT_PERMANENT) {
diff --git a/net/netfilter/nf_conntrack_netlink.c b/net/netfilter/nf_conntrack_netlink.c
index ec4164c..ac43837 100644
--- a/net/netfilter/nf_conntrack_netlink.c
+++ b/net/netfilter/nf_conntrack_netlink.c
@@ -508,7 +508,7 @@ static int ctnetlink_dump_id(struct sk_buff *skb, const struct nf_conn *ct)
static int ctnetlink_dump_use(struct sk_buff *skb, const struct nf_conn *ct)
{
- if (nla_put_be32(skb, CTA_USE, htonl(atomic_read(&ct->ct_general.use))))
+ if (nla_put_be32(skb, CTA_USE, htonl(refcount_read(&ct->ct_general.use))))
goto nla_put_failure;
return 0;
@@ -1198,7 +1198,7 @@ ctnetlink_dump_table(struct sk_buff *skb, struct netlink_callback *cb)
ct = nf_ct_tuplehash_to_ctrack(h);
if (nf_ct_is_expired(ct)) {
if (i < ARRAY_SIZE(nf_ct_evict) &&
- atomic_inc_not_zero(&ct->ct_general.use))
+ refcount_inc_not_zero(&ct->ct_general.use))
nf_ct_evict[i++] = ct;
continue;
}
@@ -1747,9 +1747,9 @@ ctnetlink_dump_list(struct sk_buff *skb, struct netlink_callback *cb, bool dying
res = ctnetlink_fill_info(skb, NETLINK_CB(cb->skb).portid,
cb->nlh->nlmsg_seq,
NFNL_MSG_TYPE(cb->nlh->nlmsg_type),
- ct, dying ? true : false, 0);
+ ct, dying, 0);
if (res < 0) {
- if (!atomic_inc_not_zero(&ct->ct_general.use))
+ if (!refcount_inc_not_zero(&ct->ct_general.use))
continue;
cb->args[0] = cpu;
cb->args[1] = (unsigned long)ct;
@@ -1820,7 +1820,7 @@ ctnetlink_parse_nat_setup(struct nf_conn *ct,
const struct nlattr *attr)
__must_hold(RCU)
{
- struct nf_nat_hook *nat_hook;
+ const struct nf_nat_hook *nat_hook;
int err;
nat_hook = rcu_dereference(nf_nat_hook);
@@ -2922,7 +2922,7 @@ static void ctnetlink_glue_seqadj(struct sk_buff *skb, struct nf_conn *ct,
nf_ct_tcp_seqadj_set(skb, ct, ctinfo, diff);
}
-static struct nfnl_ct_hook ctnetlink_glue_hook = {
+static const struct nfnl_ct_hook ctnetlink_glue_hook = {
.build_size = ctnetlink_glue_build_size,
.build = ctnetlink_glue_build,
.parse = ctnetlink_glue_parse,
@@ -2996,7 +2996,7 @@ static const union nf_inet_addr any_addr;
static __be32 nf_expect_get_id(const struct nf_conntrack_expect *exp)
{
- static __read_mostly siphash_key_t exp_id_seed;
+ static siphash_aligned_key_t exp_id_seed;
unsigned long a, b, c, d;
net_get_random_once(&exp_id_seed, sizeof(exp_id_seed));
diff --git a/net/netfilter/nf_conntrack_standalone.c b/net/netfilter/nf_conntrack_standalone.c
index 80f675d..3e1afd1 100644
--- a/net/netfilter/nf_conntrack_standalone.c
+++ b/net/netfilter/nf_conntrack_standalone.c
@@ -303,7 +303,7 @@ static int ct_seq_show(struct seq_file *s, void *v)
int ret = 0;
WARN_ON(!ct);
- if (unlikely(!atomic_inc_not_zero(&ct->ct_general.use)))
+ if (unlikely(!refcount_inc_not_zero(&ct->ct_general.use)))
return 0;
if (nf_ct_should_gc(ct)) {
@@ -370,7 +370,7 @@ static int ct_seq_show(struct seq_file *s, void *v)
ct_show_zone(s, ct, NF_CT_DEFAULT_ZONE_DIR);
ct_show_delta_time(s, ct);
- seq_printf(s, "use=%u\n", atomic_read(&ct->ct_general.use));
+ seq_printf(s, "use=%u\n", refcount_read(&ct->ct_general.use));
if (seq_has_overflowed(s))
goto release;
diff --git a/net/netfilter/nf_flow_table_core.c b/net/netfilter/nf_flow_table_core.c
index ed37bb9..b90eca7 100644
--- a/net/netfilter/nf_flow_table_core.c
+++ b/net/netfilter/nf_flow_table_core.c
@@ -48,7 +48,7 @@ struct flow_offload *flow_offload_alloc(struct nf_conn *ct)
struct flow_offload *flow;
if (unlikely(nf_ct_is_dying(ct) ||
- !atomic_inc_not_zero(&ct->ct_general.use)))
+ !refcount_inc_not_zero(&ct->ct_general.use)))
return NULL;
flow = kzalloc(sizeof(*flow), GFP_ATOMIC);
diff --git a/net/netfilter/nf_flow_table_inet.c b/net/netfilter/nf_flow_table_inet.c
index bc4126d..5c57ade 100644
--- a/net/netfilter/nf_flow_table_inet.c
+++ b/net/netfilter/nf_flow_table_inet.c
@@ -54,8 +54,30 @@ static struct nf_flowtable_type flowtable_inet = {
.owner = THIS_MODULE,
};
+static struct nf_flowtable_type flowtable_ipv4 = {
+ .family = NFPROTO_IPV4,
+ .init = nf_flow_table_init,
+ .setup = nf_flow_table_offload_setup,
+ .action = nf_flow_rule_route_ipv4,
+ .free = nf_flow_table_free,
+ .hook = nf_flow_offload_ip_hook,
+ .owner = THIS_MODULE,
+};
+
+static struct nf_flowtable_type flowtable_ipv6 = {
+ .family = NFPROTO_IPV6,
+ .init = nf_flow_table_init,
+ .setup = nf_flow_table_offload_setup,
+ .action = nf_flow_rule_route_ipv6,
+ .free = nf_flow_table_free,
+ .hook = nf_flow_offload_ipv6_hook,
+ .owner = THIS_MODULE,
+};
+
static int __init nf_flow_inet_module_init(void)
{
+ nft_register_flowtable_type(&flowtable_ipv4);
+ nft_register_flowtable_type(&flowtable_ipv6);
nft_register_flowtable_type(&flowtable_inet);
return 0;
@@ -64,6 +86,8 @@ static int __init nf_flow_inet_module_init(void)
static void __exit nf_flow_inet_module_exit(void)
{
nft_unregister_flowtable_type(&flowtable_inet);
+ nft_unregister_flowtable_type(&flowtable_ipv6);
+ nft_unregister_flowtable_type(&flowtable_ipv4);
}
module_init(nf_flow_inet_module_init);
@@ -71,5 +95,7 @@ module_exit(nf_flow_inet_module_exit);
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Pablo Neira Ayuso <pablo@netfilter.org>");
+MODULE_ALIAS_NF_FLOWTABLE(AF_INET);
+MODULE_ALIAS_NF_FLOWTABLE(AF_INET6);
MODULE_ALIAS_NF_FLOWTABLE(1); /* NFPROTO_INET */
MODULE_DESCRIPTION("Netfilter flow table mixed IPv4/IPv6 module");
diff --git a/net/netfilter/nf_nat_core.c b/net/netfilter/nf_nat_core.c
index 4d50d51..2d06a66 100644
--- a/net/netfilter/nf_nat_core.c
+++ b/net/netfilter/nf_nat_core.c
@@ -34,7 +34,7 @@ static unsigned int nat_net_id __read_mostly;
static struct hlist_head *nf_nat_bysource __read_mostly;
static unsigned int nf_nat_htable_size __read_mostly;
-static siphash_key_t nf_nat_hash_rnd __read_mostly;
+static siphash_aligned_key_t nf_nat_hash_rnd;
struct nf_nat_lookup_hook_priv {
struct nf_hook_entries __rcu *entries;
@@ -494,6 +494,38 @@ static void nf_nat_l4proto_unique_tuple(struct nf_conntrack_tuple *tuple,
goto another_round;
}
+static bool tuple_force_port_remap(const struct nf_conntrack_tuple *tuple)
+{
+ u16 sp, dp;
+
+ switch (tuple->dst.protonum) {
+ case IPPROTO_TCP:
+ sp = ntohs(tuple->src.u.tcp.port);
+ dp = ntohs(tuple->dst.u.tcp.port);
+ break;
+ case IPPROTO_UDP:
+ case IPPROTO_UDPLITE:
+ sp = ntohs(tuple->src.u.udp.port);
+ dp = ntohs(tuple->dst.u.udp.port);
+ break;
+ default:
+ return false;
+ }
+
+ /* IANA: System port range: 1-1023,
+ * user port range: 1024-49151,
+ * private port range: 49152-65535.
+ *
+ * Linux default ephemeral port range is 32768-60999.
+ *
+ * Enforce port remapping if sport is significantly lower
+ * than dport to prevent NAT port shadowing, i.e.
+ * accidental match of 'new' inbound connection vs.
+ * existing outbound one.
+ */
+ return sp < 16384 && dp >= 32768;
+}
+
/* Manipulate the tuple into the range given. For NF_INET_POST_ROUTING,
* we change the source to map into the range. For NF_INET_PRE_ROUTING
* and NF_INET_LOCAL_OUT, we change the destination to map into the
@@ -507,11 +539,17 @@ get_unique_tuple(struct nf_conntrack_tuple *tuple,
struct nf_conn *ct,
enum nf_nat_manip_type maniptype)
{
+ bool random_port = range->flags & NF_NAT_RANGE_PROTO_RANDOM_ALL;
const struct nf_conntrack_zone *zone;
struct net *net = nf_ct_net(ct);
zone = nf_ct_zone(ct);
+ if (maniptype == NF_NAT_MANIP_SRC &&
+ !random_port &&
+ !ct->local_origin)
+ random_port = tuple_force_port_remap(orig_tuple);
+
/* 1) If this srcip/proto/src-proto-part is currently mapped,
* and that same mapping gives a unique tuple within the given
* range, use that.
@@ -520,8 +558,7 @@ get_unique_tuple(struct nf_conntrack_tuple *tuple,
* So far, we don't do local source mappings, so multiple
* manips not an issue.
*/
- if (maniptype == NF_NAT_MANIP_SRC &&
- !(range->flags & NF_NAT_RANGE_PROTO_RANDOM_ALL)) {
+ if (maniptype == NF_NAT_MANIP_SRC && !random_port) {
/* try the original tuple first */
if (in_range(orig_tuple, range)) {
if (!nf_nat_used_tuple(orig_tuple, ct)) {
@@ -545,7 +582,7 @@ get_unique_tuple(struct nf_conntrack_tuple *tuple,
*/
/* Only bother mapping if it's not already in range and unique */
- if (!(range->flags & NF_NAT_RANGE_PROTO_RANDOM_ALL)) {
+ if (!random_port) {
if (range->flags & NF_NAT_RANGE_PROTO_SPECIFIED) {
if (!(range->flags & NF_NAT_RANGE_PROTO_OFFSET) &&
l4proto_in_range(tuple, maniptype,
@@ -1130,7 +1167,7 @@ static struct pernet_operations nat_net_ops = {
.size = sizeof(struct nat_net),
};
-static struct nf_nat_hook nat_hook = {
+static const struct nf_nat_hook nat_hook = {
.parse_nat_setup = nfnetlink_parse_nat_setup,
#ifdef CONFIG_XFRM
.decode_session = __nf_nat_decode_session,
diff --git a/net/netfilter/nf_nat_masquerade.c b/net/netfilter/nf_nat_masquerade.c
index acd73f7..e32fac3 100644
--- a/net/netfilter/nf_nat_masquerade.c
+++ b/net/netfilter/nf_nat_masquerade.c
@@ -12,6 +12,7 @@
struct masq_dev_work {
struct work_struct work;
struct net *net;
+ netns_tracker ns_tracker;
union nf_inet_addr addr;
int ifindex;
int (*iter)(struct nf_conn *i, void *data);
@@ -82,7 +83,7 @@ static void iterate_cleanup_work(struct work_struct *work)
nf_ct_iterate_cleanup_net(w->net, w->iter, (void *)w, 0, 0);
- put_net(w->net);
+ put_net_track(w->net, &w->ns_tracker);
kfree(w);
atomic_dec(&masq_worker_count);
module_put(THIS_MODULE);
@@ -119,6 +120,7 @@ static void nf_nat_masq_schedule(struct net *net, union nf_inet_addr *addr,
INIT_WORK(&w->work, iterate_cleanup_work);
w->ifindex = ifindex;
w->net = net;
+ netns_tracker_alloc(net, &w->ns_tracker, gfp_flags);
w->iter = iter;
if (addr)
w->addr = *addr;
diff --git a/net/netfilter/nf_synproxy_core.c b/net/netfilter/nf_synproxy_core.c
index 3d6d494..2dfc5da 100644
--- a/net/netfilter/nf_synproxy_core.c
+++ b/net/netfilter/nf_synproxy_core.c
@@ -349,7 +349,6 @@ static int __net_init synproxy_net_init(struct net *net)
goto err2;
__set_bit(IPS_CONFIRMED_BIT, &ct->status);
- nf_conntrack_get(&ct->ct_general);
snet->tmpl = ct;
snet->stats = alloc_percpu(struct synproxy_stats);
diff --git a/net/netfilter/nf_tables_api.c b/net/netfilter/nf_tables_api.c
index c207728..77938b10 100644
--- a/net/netfilter/nf_tables_api.c
+++ b/net/netfilter/nf_tables_api.c
@@ -1747,16 +1747,16 @@ static void nft_chain_stats_replace(struct nft_trans *trans)
static void nf_tables_chain_free_chain_rules(struct nft_chain *chain)
{
- struct nft_rule **g0 = rcu_dereference_raw(chain->rules_gen_0);
- struct nft_rule **g1 = rcu_dereference_raw(chain->rules_gen_1);
+ struct nft_rule_blob *g0 = rcu_dereference_raw(chain->blob_gen_0);
+ struct nft_rule_blob *g1 = rcu_dereference_raw(chain->blob_gen_1);
if (g0 != g1)
kvfree(g1);
kvfree(g0);
/* should be NULL either via abort or via successful commit */
- WARN_ON_ONCE(chain->rules_next);
- kvfree(chain->rules_next);
+ WARN_ON_ONCE(chain->blob_next);
+ kvfree(chain->blob_next);
}
void nf_tables_chain_destroy(struct nft_ctx *ctx)
@@ -2002,23 +2002,39 @@ static void nft_chain_release_hook(struct nft_chain_hook *hook)
struct nft_rules_old {
struct rcu_head h;
- struct nft_rule **start;
+ struct nft_rule_blob *blob;
};
-static struct nft_rule **nf_tables_chain_alloc_rules(const struct nft_chain *chain,
- unsigned int alloc)
+static void nft_last_rule(struct nft_rule_blob *blob, const void *ptr)
{
- if (alloc > INT_MAX)
+ struct nft_rule_dp *prule;
+
+ prule = (struct nft_rule_dp *)ptr;
+ prule->is_last = 1;
+ ptr += offsetof(struct nft_rule_dp, data);
+ /* blob size does not include the trailer rule */
+}
+
+static struct nft_rule_blob *nf_tables_chain_alloc_rules(unsigned int size)
+{
+ struct nft_rule_blob *blob;
+
+ /* size must include room for the last rule */
+ if (size < offsetof(struct nft_rule_dp, data))
return NULL;
- alloc += 1; /* NULL, ends rules */
- if (sizeof(struct nft_rule *) > INT_MAX / alloc)
+ size += sizeof(struct nft_rule_blob) + sizeof(struct nft_rules_old);
+ if (size > INT_MAX)
return NULL;
- alloc *= sizeof(struct nft_rule *);
- alloc += sizeof(struct nft_rules_old);
+ blob = kvmalloc(size, GFP_KERNEL);
+ if (!blob)
+ return NULL;
- return kvmalloc(alloc, GFP_KERNEL);
+ blob->size = 0;
+ nft_last_rule(blob, blob->data);
+
+ return blob;
}
static void nft_basechain_hook_init(struct nf_hook_ops *ops, u8 family,
@@ -2091,9 +2107,10 @@ static int nf_tables_addchain(struct nft_ctx *ctx, u8 family, u8 genmask,
struct nft_stats __percpu *stats;
struct net *net = ctx->net;
char name[NFT_NAME_MAXLEN];
+ struct nft_rule_blob *blob;
struct nft_trans *trans;
struct nft_chain *chain;
- struct nft_rule **rules;
+ unsigned int data_size;
int err;
if (table->use == UINT_MAX)
@@ -2178,15 +2195,15 @@ static int nf_tables_addchain(struct nft_ctx *ctx, u8 family, u8 genmask,
chain->udlen = nla_len(nla[NFTA_CHAIN_USERDATA]);
}
- rules = nf_tables_chain_alloc_rules(chain, 0);
- if (!rules) {
+ data_size = offsetof(struct nft_rule_dp, data); /* last rule */
+ blob = nf_tables_chain_alloc_rules(data_size);
+ if (!blob) {
err = -ENOMEM;
goto err_destroy_chain;
}
- *rules = NULL;
- rcu_assign_pointer(chain->rules_gen_0, rules);
- rcu_assign_pointer(chain->rules_gen_1, rules);
+ RCU_INIT_POINTER(chain->blob_gen_0, blob);
+ RCU_INIT_POINTER(chain->blob_gen_1, blob);
err = nf_tables_register_hook(net, table, chain);
if (err < 0)
@@ -8241,32 +8258,84 @@ EXPORT_SYMBOL_GPL(nf_tables_trans_destroy_flush_work);
static int nf_tables_commit_chain_prepare(struct net *net, struct nft_chain *chain)
{
+ const struct nft_expr *expr, *last;
+ struct nft_regs_track track = {};
+ unsigned int size, data_size;
+ void *data, *data_boundary;
+ struct nft_rule_dp *prule;
struct nft_rule *rule;
- unsigned int alloc = 0;
int i;
/* already handled or inactive chain? */
- if (chain->rules_next || !nft_is_active_next(net, chain))
+ if (chain->blob_next || !nft_is_active_next(net, chain))
return 0;
rule = list_entry(&chain->rules, struct nft_rule, list);
i = 0;
+ data_size = 0;
list_for_each_entry_continue(rule, &chain->rules, list) {
- if (nft_is_active_next(net, rule))
- alloc++;
+ if (nft_is_active_next(net, rule)) {
+ data_size += sizeof(*prule) + rule->dlen;
+ if (data_size > INT_MAX)
+ return -ENOMEM;
+ }
}
+ data_size += offsetof(struct nft_rule_dp, data); /* last rule */
- chain->rules_next = nf_tables_chain_alloc_rules(chain, alloc);
- if (!chain->rules_next)
+ chain->blob_next = nf_tables_chain_alloc_rules(data_size);
+ if (!chain->blob_next)
return -ENOMEM;
+ data = (void *)chain->blob_next->data;
+ data_boundary = data + data_size;
+ size = 0;
+
list_for_each_entry_continue(rule, &chain->rules, list) {
- if (nft_is_active_next(net, rule))
- chain->rules_next[i++] = rule;
+ if (!nft_is_active_next(net, rule))
+ continue;
+
+ prule = (struct nft_rule_dp *)data;
+ data += offsetof(struct nft_rule_dp, data);
+ if (WARN_ON_ONCE(data > data_boundary))
+ return -ENOMEM;
+
+ size = 0;
+ track.last = last;
+ nft_rule_for_each_expr(expr, last, rule) {
+ track.cur = expr;
+
+ if (expr->ops->reduce &&
+ expr->ops->reduce(&track, expr)) {
+ expr = track.cur;
+ continue;
+ }
+
+ if (WARN_ON_ONCE(data + expr->ops->size > data_boundary))
+ return -ENOMEM;
+
+ memcpy(data + size, expr, expr->ops->size);
+ size += expr->ops->size;
+ }
+ if (WARN_ON_ONCE(size >= 1 << 12))
+ return -ENOMEM;
+
+ prule->handle = rule->handle;
+ prule->dlen = size;
+ prule->is_last = 0;
+
+ data += size;
+ size = 0;
+ chain->blob_next->size += (unsigned long)(data - (void *)prule);
}
- chain->rules_next[i] = NULL;
+ prule = (struct nft_rule_dp *)data;
+ data += offsetof(struct nft_rule_dp, data);
+ if (WARN_ON_ONCE(data > data_boundary))
+ return -ENOMEM;
+
+ nft_last_rule(chain->blob_next, prule);
+
return 0;
}
@@ -8280,8 +8349,8 @@ static void nf_tables_commit_chain_prepare_cancel(struct net *net)
if (trans->msg_type == NFT_MSG_NEWRULE ||
trans->msg_type == NFT_MSG_DELRULE) {
- kvfree(chain->rules_next);
- chain->rules_next = NULL;
+ kvfree(chain->blob_next);
+ chain->blob_next = NULL;
}
}
}
@@ -8290,38 +8359,34 @@ static void __nf_tables_commit_chain_free_rules_old(struct rcu_head *h)
{
struct nft_rules_old *o = container_of(h, struct nft_rules_old, h);
- kvfree(o->start);
+ kvfree(o->blob);
}
-static void nf_tables_commit_chain_free_rules_old(struct nft_rule **rules)
+static void nf_tables_commit_chain_free_rules_old(struct nft_rule_blob *blob)
{
- struct nft_rule **r = rules;
struct nft_rules_old *old;
- while (*r)
- r++;
-
- r++; /* rcu_head is after end marker */
- old = (void *) r;
- old->start = rules;
+ /* rcu_head is after end marker */
+ old = (void *)blob + sizeof(*blob) + blob->size;
+ old->blob = blob;
call_rcu(&old->h, __nf_tables_commit_chain_free_rules_old);
}
static void nf_tables_commit_chain(struct net *net, struct nft_chain *chain)
{
- struct nft_rule **g0, **g1;
+ struct nft_rule_blob *g0, *g1;
bool next_genbit;
next_genbit = nft_gencursor_next(net);
- g0 = rcu_dereference_protected(chain->rules_gen_0,
+ g0 = rcu_dereference_protected(chain->blob_gen_0,
lockdep_commit_lock_is_held(net));
- g1 = rcu_dereference_protected(chain->rules_gen_1,
+ g1 = rcu_dereference_protected(chain->blob_gen_1,
lockdep_commit_lock_is_held(net));
/* No changes to this chain? */
- if (chain->rules_next == NULL) {
+ if (chain->blob_next == NULL) {
/* chain had no change in last or next generation */
if (g0 == g1)
return;
@@ -8330,10 +8395,10 @@ static void nf_tables_commit_chain(struct net *net, struct nft_chain *chain)
* one uses same rules as current generation.
*/
if (next_genbit) {
- rcu_assign_pointer(chain->rules_gen_1, g0);
+ rcu_assign_pointer(chain->blob_gen_1, g0);
nf_tables_commit_chain_free_rules_old(g1);
} else {
- rcu_assign_pointer(chain->rules_gen_0, g1);
+ rcu_assign_pointer(chain->blob_gen_0, g1);
nf_tables_commit_chain_free_rules_old(g0);
}
@@ -8341,11 +8406,11 @@ static void nf_tables_commit_chain(struct net *net, struct nft_chain *chain)
}
if (next_genbit)
- rcu_assign_pointer(chain->rules_gen_1, chain->rules_next);
+ rcu_assign_pointer(chain->blob_gen_1, chain->blob_next);
else
- rcu_assign_pointer(chain->rules_gen_0, chain->rules_next);
+ rcu_assign_pointer(chain->blob_gen_0, chain->blob_next);
- chain->rules_next = NULL;
+ chain->blob_next = NULL;
if (g0 == g1)
return;
diff --git a/net/netfilter/nf_tables_core.c b/net/netfilter/nf_tables_core.c
index adc3480..36e73f9 100644
--- a/net/netfilter/nf_tables_core.c
+++ b/net/netfilter/nf_tables_core.c
@@ -38,7 +38,7 @@ static noinline void __nft_trace_packet(struct nft_traceinfo *info,
static inline void nft_trace_packet(struct nft_traceinfo *info,
const struct nft_chain *chain,
- const struct nft_rule *rule,
+ const struct nft_rule_dp *rule,
enum nft_trace_types type)
{
if (static_branch_unlikely(&nft_trace_enabled)) {
@@ -67,6 +67,36 @@ static void nft_cmp_fast_eval(const struct nft_expr *expr,
regs->verdict.code = NFT_BREAK;
}
+static noinline void __nft_trace_verdict(struct nft_traceinfo *info,
+ const struct nft_chain *chain,
+ const struct nft_regs *regs)
+{
+ enum nft_trace_types type;
+
+ switch (regs->verdict.code) {
+ case NFT_CONTINUE:
+ case NFT_RETURN:
+ type = NFT_TRACETYPE_RETURN;
+ break;
+ default:
+ type = NFT_TRACETYPE_RULE;
+ break;
+ }
+
+ __nft_trace_packet(info, chain, type);
+}
+
+static inline void nft_trace_verdict(struct nft_traceinfo *info,
+ const struct nft_chain *chain,
+ const struct nft_rule_dp *rule,
+ const struct nft_regs *regs)
+{
+ if (static_branch_unlikely(&nft_trace_enabled)) {
+ info->rule = rule;
+ __nft_trace_verdict(info, chain, regs);
+ }
+}
+
static bool nft_payload_fast_eval(const struct nft_expr *expr,
struct nft_regs *regs,
const struct nft_pktinfo *pkt)
@@ -110,7 +140,6 @@ static noinline void nft_update_chain_stats(const struct nft_chain *chain,
base_chain = nft_base_chain(chain);
- rcu_read_lock();
pstats = READ_ONCE(base_chain->stats);
if (pstats) {
local_bh_disable();
@@ -121,12 +150,12 @@ static noinline void nft_update_chain_stats(const struct nft_chain *chain,
u64_stats_update_end(&stats->syncp);
local_bh_enable();
}
- rcu_read_unlock();
}
struct nft_jumpstack {
- const struct nft_chain *chain;
- struct nft_rule *const *rules;
+ const struct nft_chain *chain;
+ const struct nft_rule_dp *rule;
+ const struct nft_rule_dp *last_rule;
};
static void expr_call_ops_eval(const struct nft_expr *expr,
@@ -141,6 +170,7 @@ static void expr_call_ops_eval(const struct nft_expr *expr,
X(e, nft_payload_eval);
X(e, nft_cmp_eval);
+ X(e, nft_counter_eval);
X(e, nft_meta_get_eval);
X(e, nft_lookup_eval);
X(e, nft_range_eval);
@@ -154,18 +184,28 @@ static void expr_call_ops_eval(const struct nft_expr *expr,
expr->ops->eval(expr, regs, pkt);
}
+#define nft_rule_expr_first(rule) (struct nft_expr *)&rule->data[0]
+#define nft_rule_expr_next(expr) ((void *)expr) + expr->ops->size
+#define nft_rule_expr_last(rule) (struct nft_expr *)&rule->data[rule->dlen]
+#define nft_rule_next(rule) (void *)rule + sizeof(*rule) + rule->dlen
+
+#define nft_rule_dp_for_each_expr(expr, last, rule) \
+ for ((expr) = nft_rule_expr_first(rule), (last) = nft_rule_expr_last(rule); \
+ (expr) != (last); \
+ (expr) = nft_rule_expr_next(expr))
+
unsigned int
nft_do_chain(struct nft_pktinfo *pkt, void *priv)
{
const struct nft_chain *chain = priv, *basechain = chain;
+ const struct nft_rule_dp *rule, *last_rule;
const struct net *net = nft_net(pkt);
- struct nft_rule *const *rules;
- const struct nft_rule *rule;
const struct nft_expr *expr, *last;
struct nft_regs regs;
unsigned int stackptr = 0;
struct nft_jumpstack jumpstack[NFT_JUMP_STACK_SIZE];
bool genbit = READ_ONCE(net->nft.gencursor);
+ struct nft_rule_blob *blob;
struct nft_traceinfo info;
info.trace = false;
@@ -173,16 +213,16 @@ nft_do_chain(struct nft_pktinfo *pkt, void *priv)
nft_trace_init(&info, pkt, ®s.verdict, basechain);
do_chain:
if (genbit)
- rules = rcu_dereference(chain->rules_gen_1);
+ blob = rcu_dereference(chain->blob_gen_1);
else
- rules = rcu_dereference(chain->rules_gen_0);
+ blob = rcu_dereference(chain->blob_gen_0);
+ rule = (struct nft_rule_dp *)blob->data;
+ last_rule = (void *)blob->data + blob->size;
next_rule:
- rule = *rules;
regs.verdict.code = NFT_CONTINUE;
- for (; *rules ; rules++) {
- rule = *rules;
- nft_rule_for_each_expr(expr, last, rule) {
+ for (; rule < last_rule; rule = nft_rule_next(rule)) {
+ nft_rule_dp_for_each_expr(expr, last, rule) {
if (expr->ops == &nft_cmp_fast_ops)
nft_cmp_fast_eval(expr, ®s);
else if (expr->ops == &nft_bitwise_fast_ops)
@@ -207,13 +247,13 @@ nft_do_chain(struct nft_pktinfo *pkt, void *priv)
break;
}
+ nft_trace_verdict(&info, chain, rule, ®s);
+
switch (regs.verdict.code & NF_VERDICT_MASK) {
case NF_ACCEPT:
case NF_DROP:
case NF_QUEUE:
case NF_STOLEN:
- nft_trace_packet(&info, chain, rule,
- NFT_TRACETYPE_RULE);
return regs.verdict.code;
}
@@ -222,28 +262,25 @@ nft_do_chain(struct nft_pktinfo *pkt, void *priv)
if (WARN_ON_ONCE(stackptr >= NFT_JUMP_STACK_SIZE))
return NF_DROP;
jumpstack[stackptr].chain = chain;
- jumpstack[stackptr].rules = rules + 1;
+ jumpstack[stackptr].rule = nft_rule_next(rule);
+ jumpstack[stackptr].last_rule = last_rule;
stackptr++;
fallthrough;
case NFT_GOTO:
- nft_trace_packet(&info, chain, rule,
- NFT_TRACETYPE_RULE);
-
chain = regs.verdict.chain;
goto do_chain;
case NFT_CONTINUE:
case NFT_RETURN:
- nft_trace_packet(&info, chain, rule,
- NFT_TRACETYPE_RETURN);
break;
default:
- WARN_ON(1);
+ WARN_ON_ONCE(1);
}
if (stackptr > 0) {
stackptr--;
chain = jumpstack[stackptr].chain;
- rules = jumpstack[stackptr].rules;
+ rule = jumpstack[stackptr].rule;
+ last_rule = jumpstack[stackptr].last_rule;
goto next_rule;
}
@@ -269,18 +306,22 @@ static struct nft_expr_type *nft_basic_types[] = {
&nft_rt_type,
&nft_exthdr_type,
&nft_last_type,
+ &nft_counter_type,
};
static struct nft_object_type *nft_basic_objects[] = {
#ifdef CONFIG_NETWORK_SECMARK
&nft_secmark_obj_type,
#endif
+ &nft_counter_obj_type,
};
int __init nf_tables_core_module_init(void)
{
int err, i, j = 0;
+ nft_counter_init_seqcount();
+
for (i = 0; i < ARRAY_SIZE(nft_basic_objects); i++) {
err = nft_register_obj(nft_basic_objects[i]);
if (err)
diff --git a/net/netfilter/nf_tables_trace.c b/net/netfilter/nf_tables_trace.c
index 84a7dea..5041725 100644
--- a/net/netfilter/nf_tables_trace.c
+++ b/net/netfilter/nf_tables_trace.c
@@ -142,7 +142,7 @@ static int nf_trace_fill_pkt_info(struct sk_buff *nlskb,
static int nf_trace_fill_rule_info(struct sk_buff *nlskb,
const struct nft_traceinfo *info)
{
- if (!info->rule)
+ if (!info->rule || info->rule->is_last)
return 0;
/* a continue verdict with ->type == RETURN means that this is
diff --git a/net/netfilter/nfnetlink_hook.c b/net/netfilter/nfnetlink_hook.c
index d5c719c..71e29ad 100644
--- a/net/netfilter/nfnetlink_hook.c
+++ b/net/netfilter/nfnetlink_hook.c
@@ -6,6 +6,7 @@
*/
#include <linux/module.h>
+#include <linux/kallsyms.h>
#include <linux/kernel.h>
#include <linux/types.h>
#include <linux/skbuff.h>
diff --git a/net/netfilter/nfnetlink_log.c b/net/netfilter/nfnetlink_log.c
index 7f83f96..ae9c075 100644
--- a/net/netfilter/nfnetlink_log.c
+++ b/net/netfilter/nfnetlink_log.c
@@ -66,6 +66,7 @@ struct nfulnl_instance {
struct sk_buff *skb; /* pre-allocatd skb */
struct timer_list timer;
struct net *net;
+ netns_tracker ns_tracker;
struct user_namespace *peer_user_ns; /* User namespace of the peer process */
u32 peer_portid; /* PORTID of the peer process */
@@ -140,7 +141,7 @@ static void nfulnl_instance_free_rcu(struct rcu_head *head)
struct nfulnl_instance *inst =
container_of(head, struct nfulnl_instance, rcu);
- put_net(inst->net);
+ put_net_track(inst->net, &inst->ns_tracker);
kfree(inst);
module_put(THIS_MODULE);
}
@@ -187,7 +188,7 @@ instance_create(struct net *net, u_int16_t group_num,
timer_setup(&inst->timer, nfulnl_timer, 0);
- inst->net = get_net(net);
+ inst->net = get_net_track(net, &inst->ns_tracker, GFP_ATOMIC);
inst->peer_user_ns = user_ns;
inst->peer_portid = portid;
inst->group_num = group_num;
diff --git a/net/netfilter/nfnetlink_queue.c b/net/netfilter/nfnetlink_queue.c
index f0b9e21..ea2d9c2 100644
--- a/net/netfilter/nfnetlink_queue.c
+++ b/net/netfilter/nfnetlink_queue.c
@@ -225,7 +225,7 @@ find_dequeue_entry(struct nfqnl_instance *queue, unsigned int id)
static void nfqnl_reinject(struct nf_queue_entry *entry, unsigned int verdict)
{
- struct nf_ct_hook *ct_hook;
+ const struct nf_ct_hook *ct_hook;
int err;
if (verdict == NF_ACCEPT ||
@@ -388,7 +388,7 @@ nfqnl_build_packet_message(struct net *net, struct nfqnl_instance *queue,
struct net_device *outdev;
struct nf_conn *ct = NULL;
enum ip_conntrack_info ctinfo = 0;
- struct nfnl_ct_hook *nfnl_ct;
+ const struct nfnl_ct_hook *nfnl_ct;
bool csum_verify;
char *secdata = NULL;
u32 seclen = 0;
@@ -1104,7 +1104,7 @@ static int nfqnl_recv_verdict_batch(struct sk_buff *skb,
return 0;
}
-static struct nf_conn *nfqnl_ct_parse(struct nfnl_ct_hook *nfnl_ct,
+static struct nf_conn *nfqnl_ct_parse(const struct nfnl_ct_hook *nfnl_ct,
const struct nlmsghdr *nlh,
const struct nlattr * const nfqa[],
struct nf_queue_entry *entry,
@@ -1171,11 +1171,11 @@ static int nfqnl_recv_verdict(struct sk_buff *skb, const struct nfnl_info *info,
{
struct nfnl_queue_net *q = nfnl_queue_pernet(info->net);
u_int16_t queue_num = ntohs(info->nfmsg->res_id);
+ const struct nfnl_ct_hook *nfnl_ct;
struct nfqnl_msg_verdict_hdr *vhdr;
enum ip_conntrack_info ctinfo;
struct nfqnl_instance *queue;
struct nf_queue_entry *entry;
- struct nfnl_ct_hook *nfnl_ct;
struct nf_conn *ct = NULL;
unsigned int verdict;
int err;
@@ -1528,15 +1528,9 @@ static void __net_exit nfnl_queue_net_exit(struct net *net)
WARN_ON_ONCE(!hlist_empty(&q->instance_table[i]));
}
-static void nfnl_queue_net_exit_batch(struct list_head *net_exit_list)
-{
- synchronize_rcu();
-}
-
static struct pernet_operations nfnl_queue_net_ops = {
.init = nfnl_queue_net_init,
.exit = nfnl_queue_net_exit,
- .exit_batch = nfnl_queue_net_exit_batch,
.id = &nfnl_queue_net_id,
.size = sizeof(struct nfnl_queue_net),
};
diff --git a/net/netfilter/nft_bitwise.c b/net/netfilter/nft_bitwise.c
index 47b0dba..7b727d3 100644
--- a/net/netfilter/nft_bitwise.c
+++ b/net/netfilter/nft_bitwise.c
@@ -278,12 +278,52 @@ static int nft_bitwise_offload(struct nft_offload_ctx *ctx,
return 0;
}
+static bool nft_bitwise_reduce(struct nft_regs_track *track,
+ const struct nft_expr *expr)
+{
+ const struct nft_bitwise *priv = nft_expr_priv(expr);
+ const struct nft_bitwise *bitwise;
+
+ if (!track->regs[priv->sreg].selector)
+ return false;
+
+ bitwise = nft_expr_priv(expr);
+ if (track->regs[priv->sreg].selector == track->regs[priv->dreg].selector &&
+ track->regs[priv->dreg].bitwise &&
+ track->regs[priv->dreg].bitwise->ops == expr->ops &&
+ priv->sreg == bitwise->sreg &&
+ priv->dreg == bitwise->dreg &&
+ priv->op == bitwise->op &&
+ priv->len == bitwise->len &&
+ !memcmp(&priv->mask, &bitwise->mask, sizeof(priv->mask)) &&
+ !memcmp(&priv->xor, &bitwise->xor, sizeof(priv->xor)) &&
+ !memcmp(&priv->data, &bitwise->data, sizeof(priv->data))) {
+ track->cur = expr;
+ return true;
+ }
+
+ if (track->regs[priv->sreg].bitwise) {
+ track->regs[priv->dreg].selector = NULL;
+ track->regs[priv->dreg].bitwise = NULL;
+ return false;
+ }
+
+ if (priv->sreg != priv->dreg) {
+ track->regs[priv->dreg].selector =
+ track->regs[priv->sreg].selector;
+ }
+ track->regs[priv->dreg].bitwise = expr;
+
+ return false;
+}
+
static const struct nft_expr_ops nft_bitwise_ops = {
.type = &nft_bitwise_type,
.size = NFT_EXPR_SIZE(sizeof(struct nft_bitwise)),
.eval = nft_bitwise_eval,
.init = nft_bitwise_init,
.dump = nft_bitwise_dump,
+ .reduce = nft_bitwise_reduce,
.offload = nft_bitwise_offload,
};
@@ -385,12 +425,49 @@ static int nft_bitwise_fast_offload(struct nft_offload_ctx *ctx,
return 0;
}
+static bool nft_bitwise_fast_reduce(struct nft_regs_track *track,
+ const struct nft_expr *expr)
+{
+ const struct nft_bitwise_fast_expr *priv = nft_expr_priv(expr);
+ const struct nft_bitwise_fast_expr *bitwise;
+
+ if (!track->regs[priv->sreg].selector)
+ return false;
+
+ bitwise = nft_expr_priv(expr);
+ if (track->regs[priv->sreg].selector == track->regs[priv->dreg].selector &&
+ track->regs[priv->dreg].bitwise &&
+ track->regs[priv->dreg].bitwise->ops == expr->ops &&
+ priv->sreg == bitwise->sreg &&
+ priv->dreg == bitwise->dreg &&
+ priv->mask == bitwise->mask &&
+ priv->xor == bitwise->xor) {
+ track->cur = expr;
+ return true;
+ }
+
+ if (track->regs[priv->sreg].bitwise) {
+ track->regs[priv->dreg].selector = NULL;
+ track->regs[priv->dreg].bitwise = NULL;
+ return false;
+ }
+
+ if (priv->sreg != priv->dreg) {
+ track->regs[priv->dreg].selector =
+ track->regs[priv->sreg].selector;
+ }
+ track->regs[priv->dreg].bitwise = expr;
+
+ return false;
+}
+
const struct nft_expr_ops nft_bitwise_fast_ops = {
.type = &nft_bitwise_type,
.size = NFT_EXPR_SIZE(sizeof(struct nft_bitwise_fast_expr)),
.eval = NULL, /* inlined */
.init = nft_bitwise_fast_init,
.dump = nft_bitwise_fast_dump,
+ .reduce = nft_bitwise_fast_reduce,
.offload = nft_bitwise_fast_offload,
};
@@ -427,3 +504,21 @@ struct nft_expr_type nft_bitwise_type __read_mostly = {
.maxattr = NFTA_BITWISE_MAX,
.owner = THIS_MODULE,
};
+
+bool nft_expr_reduce_bitwise(struct nft_regs_track *track,
+ const struct nft_expr *expr)
+{
+ const struct nft_expr *last = track->last;
+ const struct nft_expr *next;
+
+ if (expr == last)
+ return false;
+
+ next = nft_expr_next(expr);
+ if (next->ops == &nft_bitwise_ops)
+ return nft_bitwise_reduce(track, next);
+ else if (next->ops == &nft_bitwise_fast_ops)
+ return nft_bitwise_fast_reduce(track, next);
+
+ return false;
+}
diff --git a/net/netfilter/nft_connlimit.c b/net/netfilter/nft_connlimit.c
index 7d0761f..58dcafe 100644
--- a/net/netfilter/nft_connlimit.c
+++ b/net/netfilter/nft_connlimit.c
@@ -14,7 +14,7 @@
#include <net/netfilter/nf_conntrack_zones.h>
struct nft_connlimit {
- struct nf_conncount_list list;
+ struct nf_conncount_list *list;
u32 limit;
bool invert;
};
@@ -43,12 +43,12 @@ static inline void nft_connlimit_do_eval(struct nft_connlimit *priv,
return;
}
- if (nf_conncount_add(nft_net(pkt), &priv->list, tuple_ptr, zone)) {
+ if (nf_conncount_add(nft_net(pkt), priv->list, tuple_ptr, zone)) {
regs->verdict.code = NF_DROP;
return;
}
- count = priv->list.count;
+ count = priv->list->count;
if ((count > priv->limit) ^ priv->invert) {
regs->verdict.code = NFT_BREAK;
@@ -76,7 +76,11 @@ static int nft_connlimit_do_init(const struct nft_ctx *ctx,
invert = true;
}
- nf_conncount_list_init(&priv->list);
+ priv->list = kmalloc(sizeof(*priv->list), GFP_KERNEL);
+ if (!priv->list)
+ return -ENOMEM;
+
+ nf_conncount_list_init(priv->list);
priv->limit = limit;
priv->invert = invert;
@@ -87,7 +91,8 @@ static void nft_connlimit_do_destroy(const struct nft_ctx *ctx,
struct nft_connlimit *priv)
{
nf_ct_netns_put(ctx->net, ctx->family);
- nf_conncount_cache_free(&priv->list);
+ nf_conncount_cache_free(priv->list);
+ kfree(priv->list);
}
static int nft_connlimit_do_dump(struct sk_buff *skb,
@@ -200,7 +205,11 @@ static int nft_connlimit_clone(struct nft_expr *dst, const struct nft_expr *src)
struct nft_connlimit *priv_dst = nft_expr_priv(dst);
struct nft_connlimit *priv_src = nft_expr_priv(src);
- nf_conncount_list_init(&priv_dst->list);
+ priv_dst->list = kmalloc(sizeof(*priv_dst->list), GFP_ATOMIC);
+ if (priv_dst->list)
+ return -ENOMEM;
+
+ nf_conncount_list_init(priv_dst->list);
priv_dst->limit = priv_src->limit;
priv_dst->invert = priv_src->invert;
@@ -212,7 +221,8 @@ static void nft_connlimit_destroy_clone(const struct nft_ctx *ctx,
{
struct nft_connlimit *priv = nft_expr_priv(expr);
- nf_conncount_cache_free(&priv->list);
+ nf_conncount_cache_free(priv->list);
+ kfree(priv->list);
}
static bool nft_connlimit_gc(struct net *net, const struct nft_expr *expr)
@@ -221,7 +231,7 @@ static bool nft_connlimit_gc(struct net *net, const struct nft_expr *expr)
bool ret;
local_bh_disable();
- ret = nf_conncount_gc_list(net, &priv->list);
+ ret = nf_conncount_gc_list(net, priv->list);
local_bh_enable();
return ret;
diff --git a/net/netfilter/nft_counter.c b/net/netfilter/nft_counter.c
index 8edd3b3..f179e8c 100644
--- a/net/netfilter/nft_counter.c
+++ b/net/netfilter/nft_counter.c
@@ -13,6 +13,7 @@
#include <linux/netfilter.h>
#include <linux/netfilter/nf_tables.h>
#include <net/netfilter/nf_tables.h>
+#include <net/netfilter/nf_tables_core.h>
#include <net/netfilter/nf_tables_offload.h>
struct nft_counter {
@@ -174,7 +175,7 @@ static const struct nla_policy nft_counter_policy[NFTA_COUNTER_MAX + 1] = {
[NFTA_COUNTER_BYTES] = { .type = NLA_U64 },
};
-static struct nft_object_type nft_counter_obj_type;
+struct nft_object_type nft_counter_obj_type;
static const struct nft_object_ops nft_counter_obj_ops = {
.type = &nft_counter_obj_type,
.size = sizeof(struct nft_counter_percpu_priv),
@@ -184,7 +185,7 @@ static const struct nft_object_ops nft_counter_obj_ops = {
.dump = nft_counter_obj_dump,
};
-static struct nft_object_type nft_counter_obj_type __read_mostly = {
+struct nft_object_type nft_counter_obj_type __read_mostly = {
.type = NFT_OBJECT_COUNTER,
.ops = &nft_counter_obj_ops,
.maxattr = NFTA_COUNTER_MAX,
@@ -192,9 +193,8 @@ static struct nft_object_type nft_counter_obj_type __read_mostly = {
.owner = THIS_MODULE,
};
-static void nft_counter_eval(const struct nft_expr *expr,
- struct nft_regs *regs,
- const struct nft_pktinfo *pkt)
+void nft_counter_eval(const struct nft_expr *expr, struct nft_regs *regs,
+ const struct nft_pktinfo *pkt)
{
struct nft_counter_percpu_priv *priv = nft_expr_priv(expr);
@@ -275,7 +275,15 @@ static void nft_counter_offload_stats(struct nft_expr *expr,
preempt_enable();
}
-static struct nft_expr_type nft_counter_type;
+void nft_counter_init_seqcount(void)
+{
+ int cpu;
+
+ for_each_possible_cpu(cpu)
+ seqcount_init(per_cpu_ptr(&nft_counter_seq, cpu));
+}
+
+struct nft_expr_type nft_counter_type;
static const struct nft_expr_ops nft_counter_ops = {
.type = &nft_counter_type,
.size = NFT_EXPR_SIZE(sizeof(struct nft_counter_percpu_priv)),
@@ -289,7 +297,7 @@ static const struct nft_expr_ops nft_counter_ops = {
.offload_stats = nft_counter_offload_stats,
};
-static struct nft_expr_type nft_counter_type __read_mostly = {
+struct nft_expr_type nft_counter_type __read_mostly = {
.name = "counter",
.ops = &nft_counter_ops,
.policy = nft_counter_policy,
@@ -297,39 +305,3 @@ static struct nft_expr_type nft_counter_type __read_mostly = {
.flags = NFT_EXPR_STATEFUL,
.owner = THIS_MODULE,
};
-
-static int __init nft_counter_module_init(void)
-{
- int cpu, err;
-
- for_each_possible_cpu(cpu)
- seqcount_init(per_cpu_ptr(&nft_counter_seq, cpu));
-
- err = nft_register_obj(&nft_counter_obj_type);
- if (err < 0)
- return err;
-
- err = nft_register_expr(&nft_counter_type);
- if (err < 0)
- goto err1;
-
- return 0;
-err1:
- nft_unregister_obj(&nft_counter_obj_type);
- return err;
-}
-
-static void __exit nft_counter_module_exit(void)
-{
- nft_unregister_expr(&nft_counter_type);
- nft_unregister_obj(&nft_counter_obj_type);
-}
-
-module_init(nft_counter_module_init);
-module_exit(nft_counter_module_exit);
-
-MODULE_LICENSE("GPL");
-MODULE_AUTHOR("Patrick McHardy <kaber@trash.net>");
-MODULE_ALIAS_NFT_EXPR("counter");
-MODULE_ALIAS_NFT_OBJ(NFT_OBJECT_COUNTER);
-MODULE_DESCRIPTION("nftables counter rule support");
diff --git a/net/netfilter/nft_ct.c b/net/netfilter/nft_ct.c
index 99b1de1..518d96c 100644
--- a/net/netfilter/nft_ct.c
+++ b/net/netfilter/nft_ct.c
@@ -259,7 +259,7 @@ static void nft_ct_set_zone_eval(const struct nft_expr *expr,
ct = this_cpu_read(nft_ct_pcpu_template);
- if (likely(atomic_read(&ct->ct_general.use) == 1)) {
+ if (likely(refcount_read(&ct->ct_general.use) == 1)) {
nf_ct_zone_add(ct, &zone);
} else {
/* previous skb got queued to userspace */
@@ -270,7 +270,6 @@ static void nft_ct_set_zone_eval(const struct nft_expr *expr,
}
}
- atomic_inc(&ct->ct_general.use);
nf_ct_set(skb, ct, IP_CT_NEW);
}
#endif
@@ -375,7 +374,6 @@ static bool nft_ct_tmpl_alloc_pcpu(void)
return false;
}
- atomic_set(&tmp->ct_general.use, 1);
per_cpu(nft_ct_pcpu_template, cpu) = tmp;
}
diff --git a/net/netfilter/nft_fwd_netdev.c b/net/netfilter/nft_fwd_netdev.c
index cd59afd..fa9301ca 100644
--- a/net/netfilter/nft_fwd_netdev.c
+++ b/net/netfilter/nft_fwd_netdev.c
@@ -27,9 +27,11 @@ static void nft_fwd_netdev_eval(const struct nft_expr *expr,
{
struct nft_fwd_netdev *priv = nft_expr_priv(expr);
int oif = regs->data[priv->sreg_dev];
+ struct sk_buff *skb = pkt->skb;
/* This is used by ifb only. */
- skb_set_redirected(pkt->skb, true);
+ skb->skb_iif = skb->dev->ifindex;
+ skb_set_redirected(skb, nft_hook(pkt) == NF_NETDEV_INGRESS);
nf_fwd_netdev_egress(pkt, oif);
regs->verdict.code = NF_STOLEN;
@@ -198,7 +200,8 @@ static int nft_fwd_validate(const struct nft_ctx *ctx,
const struct nft_expr *expr,
const struct nft_data **data)
{
- return nft_chain_validate_hooks(ctx->chain, (1 << NF_NETDEV_INGRESS));
+ return nft_chain_validate_hooks(ctx->chain, (1 << NF_NETDEV_INGRESS) |
+ (1 << NF_NETDEV_EGRESS));
}
static struct nft_expr_type nft_fwd_netdev_type;
diff --git a/net/netfilter/nft_last.c b/net/netfilter/nft_last.c
index 304e33c..5ee33d0 100644
--- a/net/netfilter/nft_last.c
+++ b/net/netfilter/nft_last.c
@@ -8,9 +8,13 @@
#include <net/netfilter/nf_tables_core.h>
#include <net/netfilter/nf_tables.h>
+struct nft_last {
+ unsigned long jiffies;
+ unsigned int set;
+};
+
struct nft_last_priv {
- unsigned long last_jiffies;
- unsigned int last_set;
+ struct nft_last *last;
};
static const struct nla_policy nft_last_policy[NFTA_LAST_MAX + 1] = {
@@ -22,47 +26,55 @@ static int nft_last_init(const struct nft_ctx *ctx, const struct nft_expr *expr,
const struct nlattr * const tb[])
{
struct nft_last_priv *priv = nft_expr_priv(expr);
+ struct nft_last *last;
u64 last_jiffies;
- u32 last_set = 0;
int err;
- if (tb[NFTA_LAST_SET]) {
- last_set = ntohl(nla_get_be32(tb[NFTA_LAST_SET]));
- if (last_set == 1)
- priv->last_set = 1;
- }
+ last = kzalloc(sizeof(*last), GFP_KERNEL);
+ if (!last)
+ return -ENOMEM;
- if (last_set && tb[NFTA_LAST_MSECS]) {
+ if (tb[NFTA_LAST_SET])
+ last->set = ntohl(nla_get_be32(tb[NFTA_LAST_SET]));
+
+ if (last->set && tb[NFTA_LAST_MSECS]) {
err = nf_msecs_to_jiffies64(tb[NFTA_LAST_MSECS], &last_jiffies);
if (err < 0)
- return err;
+ goto err;
- priv->last_jiffies = jiffies - (unsigned long)last_jiffies;
+ last->jiffies = jiffies - (unsigned long)last_jiffies;
}
+ priv->last = last;
return 0;
+err:
+ kfree(last);
+
+ return err;
}
static void nft_last_eval(const struct nft_expr *expr,
struct nft_regs *regs, const struct nft_pktinfo *pkt)
{
struct nft_last_priv *priv = nft_expr_priv(expr);
+ struct nft_last *last = priv->last;
- if (READ_ONCE(priv->last_jiffies) != jiffies)
- WRITE_ONCE(priv->last_jiffies, jiffies);
- if (READ_ONCE(priv->last_set) == 0)
- WRITE_ONCE(priv->last_set, 1);
+ if (READ_ONCE(last->jiffies) != jiffies)
+ WRITE_ONCE(last->jiffies, jiffies);
+ if (READ_ONCE(last->set) == 0)
+ WRITE_ONCE(last->set, 1);
}
static int nft_last_dump(struct sk_buff *skb, const struct nft_expr *expr)
{
struct nft_last_priv *priv = nft_expr_priv(expr);
- unsigned long last_jiffies = READ_ONCE(priv->last_jiffies);
- u32 last_set = READ_ONCE(priv->last_set);
+ struct nft_last *last = priv->last;
+ unsigned long last_jiffies = READ_ONCE(last->jiffies);
+ u32 last_set = READ_ONCE(last->set);
__be64 msecs;
if (time_before(jiffies, last_jiffies)) {
- WRITE_ONCE(priv->last_set, 0);
+ WRITE_ONCE(last->set, 0);
last_set = 0;
}
@@ -81,11 +93,32 @@ static int nft_last_dump(struct sk_buff *skb, const struct nft_expr *expr)
return -1;
}
+static void nft_last_destroy(const struct nft_ctx *ctx,
+ const struct nft_expr *expr)
+{
+ struct nft_last_priv *priv = nft_expr_priv(expr);
+
+ kfree(priv->last);
+}
+
+static int nft_last_clone(struct nft_expr *dst, const struct nft_expr *src)
+{
+ struct nft_last_priv *priv_dst = nft_expr_priv(dst);
+
+ priv_dst->last = kzalloc(sizeof(*priv_dst->last), GFP_ATOMIC);
+ if (priv_dst->last)
+ return -ENOMEM;
+
+ return 0;
+}
+
static const struct nft_expr_ops nft_last_ops = {
.type = &nft_last_type,
.size = NFT_EXPR_SIZE(sizeof(struct nft_last_priv)),
.eval = nft_last_eval,
.init = nft_last_init,
+ .destroy = nft_last_destroy,
+ .clone = nft_last_clone,
.dump = nft_last_dump,
};
diff --git a/net/netfilter/nft_limit.c b/net/netfilter/nft_limit.c
index 82ec27b..f04be5b 100644
--- a/net/netfilter/nft_limit.c
+++ b/net/netfilter/nft_limit.c
@@ -18,6 +18,10 @@ struct nft_limit {
spinlock_t lock;
u64 last;
u64 tokens;
+};
+
+struct nft_limit_priv {
+ struct nft_limit *limit;
u64 tokens_max;
u64 rate;
u64 nsecs;
@@ -25,33 +29,33 @@ struct nft_limit {
bool invert;
};
-static inline bool nft_limit_eval(struct nft_limit *limit, u64 cost)
+static inline bool nft_limit_eval(struct nft_limit_priv *priv, u64 cost)
{
u64 now, tokens;
s64 delta;
- spin_lock_bh(&limit->lock);
+ spin_lock_bh(&priv->limit->lock);
now = ktime_get_ns();
- tokens = limit->tokens + now - limit->last;
- if (tokens > limit->tokens_max)
- tokens = limit->tokens_max;
+ tokens = priv->limit->tokens + now - priv->limit->last;
+ if (tokens > priv->tokens_max)
+ tokens = priv->tokens_max;
- limit->last = now;
+ priv->limit->last = now;
delta = tokens - cost;
if (delta >= 0) {
- limit->tokens = delta;
- spin_unlock_bh(&limit->lock);
- return limit->invert;
+ priv->limit->tokens = delta;
+ spin_unlock_bh(&priv->limit->lock);
+ return priv->invert;
}
- limit->tokens = tokens;
- spin_unlock_bh(&limit->lock);
- return !limit->invert;
+ priv->limit->tokens = tokens;
+ spin_unlock_bh(&priv->limit->lock);
+ return !priv->invert;
}
/* Use same default as in iptables. */
#define NFT_LIMIT_PKT_BURST_DEFAULT 5
-static int nft_limit_init(struct nft_limit *limit,
+static int nft_limit_init(struct nft_limit_priv *priv,
const struct nlattr * const tb[], bool pkts)
{
u64 unit, tokens;
@@ -60,58 +64,62 @@ static int nft_limit_init(struct nft_limit *limit,
tb[NFTA_LIMIT_UNIT] == NULL)
return -EINVAL;
- limit->rate = be64_to_cpu(nla_get_be64(tb[NFTA_LIMIT_RATE]));
+ priv->rate = be64_to_cpu(nla_get_be64(tb[NFTA_LIMIT_RATE]));
unit = be64_to_cpu(nla_get_be64(tb[NFTA_LIMIT_UNIT]));
- limit->nsecs = unit * NSEC_PER_SEC;
- if (limit->rate == 0 || limit->nsecs < unit)
+ priv->nsecs = unit * NSEC_PER_SEC;
+ if (priv->rate == 0 || priv->nsecs < unit)
return -EOVERFLOW;
if (tb[NFTA_LIMIT_BURST])
- limit->burst = ntohl(nla_get_be32(tb[NFTA_LIMIT_BURST]));
+ priv->burst = ntohl(nla_get_be32(tb[NFTA_LIMIT_BURST]));
- if (pkts && limit->burst == 0)
- limit->burst = NFT_LIMIT_PKT_BURST_DEFAULT;
+ if (pkts && priv->burst == 0)
+ priv->burst = NFT_LIMIT_PKT_BURST_DEFAULT;
- if (limit->rate + limit->burst < limit->rate)
+ if (priv->rate + priv->burst < priv->rate)
return -EOVERFLOW;
if (pkts) {
- tokens = div64_u64(limit->nsecs, limit->rate) * limit->burst;
+ tokens = div64_u64(priv->nsecs, priv->rate) * priv->burst;
} else {
/* The token bucket size limits the number of tokens can be
* accumulated. tokens_max specifies the bucket size.
* tokens_max = unit * (rate + burst) / rate.
*/
- tokens = div64_u64(limit->nsecs * (limit->rate + limit->burst),
- limit->rate);
+ tokens = div64_u64(priv->nsecs * (priv->rate + priv->burst),
+ priv->rate);
}
- limit->tokens = tokens;
- limit->tokens_max = limit->tokens;
+ priv->limit = kmalloc(sizeof(*priv->limit), GFP_KERNEL);
+ if (!priv->limit)
+ return -ENOMEM;
+
+ priv->limit->tokens = tokens;
+ priv->tokens_max = priv->limit->tokens;
if (tb[NFTA_LIMIT_FLAGS]) {
u32 flags = ntohl(nla_get_be32(tb[NFTA_LIMIT_FLAGS]));
if (flags & NFT_LIMIT_F_INV)
- limit->invert = true;
+ priv->invert = true;
}
- limit->last = ktime_get_ns();
- spin_lock_init(&limit->lock);
+ priv->limit->last = ktime_get_ns();
+ spin_lock_init(&priv->limit->lock);
return 0;
}
-static int nft_limit_dump(struct sk_buff *skb, const struct nft_limit *limit,
+static int nft_limit_dump(struct sk_buff *skb, const struct nft_limit_priv *priv,
enum nft_limit_type type)
{
- u32 flags = limit->invert ? NFT_LIMIT_F_INV : 0;
- u64 secs = div_u64(limit->nsecs, NSEC_PER_SEC);
+ u32 flags = priv->invert ? NFT_LIMIT_F_INV : 0;
+ u64 secs = div_u64(priv->nsecs, NSEC_PER_SEC);
- if (nla_put_be64(skb, NFTA_LIMIT_RATE, cpu_to_be64(limit->rate),
+ if (nla_put_be64(skb, NFTA_LIMIT_RATE, cpu_to_be64(priv->rate),
NFTA_LIMIT_PAD) ||
nla_put_be64(skb, NFTA_LIMIT_UNIT, cpu_to_be64(secs),
NFTA_LIMIT_PAD) ||
- nla_put_be32(skb, NFTA_LIMIT_BURST, htonl(limit->burst)) ||
+ nla_put_be32(skb, NFTA_LIMIT_BURST, htonl(priv->burst)) ||
nla_put_be32(skb, NFTA_LIMIT_TYPE, htonl(type)) ||
nla_put_be32(skb, NFTA_LIMIT_FLAGS, htonl(flags)))
goto nla_put_failure;
@@ -121,8 +129,34 @@ static int nft_limit_dump(struct sk_buff *skb, const struct nft_limit *limit,
return -1;
}
-struct nft_limit_pkts {
- struct nft_limit limit;
+static void nft_limit_destroy(const struct nft_ctx *ctx,
+ const struct nft_limit_priv *priv)
+{
+ kfree(priv->limit);
+}
+
+static int nft_limit_clone(struct nft_limit_priv *priv_dst,
+ const struct nft_limit_priv *priv_src)
+{
+ priv_dst->tokens_max = priv_src->tokens_max;
+ priv_dst->rate = priv_src->rate;
+ priv_dst->nsecs = priv_src->nsecs;
+ priv_dst->burst = priv_src->burst;
+ priv_dst->invert = priv_src->invert;
+
+ priv_dst->limit = kmalloc(sizeof(*priv_dst->limit), GFP_ATOMIC);
+ if (priv_dst->limit)
+ return -ENOMEM;
+
+ spin_lock_init(&priv_dst->limit->lock);
+ priv_dst->limit->tokens = priv_src->tokens_max;
+ priv_dst->limit->last = ktime_get_ns();
+
+ return 0;
+}
+
+struct nft_limit_priv_pkts {
+ struct nft_limit_priv limit;
u64 cost;
};
@@ -130,7 +164,7 @@ static void nft_limit_pkts_eval(const struct nft_expr *expr,
struct nft_regs *regs,
const struct nft_pktinfo *pkt)
{
- struct nft_limit_pkts *priv = nft_expr_priv(expr);
+ struct nft_limit_priv_pkts *priv = nft_expr_priv(expr);
if (nft_limit_eval(&priv->limit, priv->cost))
regs->verdict.code = NFT_BREAK;
@@ -148,7 +182,7 @@ static int nft_limit_pkts_init(const struct nft_ctx *ctx,
const struct nft_expr *expr,
const struct nlattr * const tb[])
{
- struct nft_limit_pkts *priv = nft_expr_priv(expr);
+ struct nft_limit_priv_pkts *priv = nft_expr_priv(expr);
int err;
err = nft_limit_init(&priv->limit, tb, true);
@@ -161,17 +195,35 @@ static int nft_limit_pkts_init(const struct nft_ctx *ctx,
static int nft_limit_pkts_dump(struct sk_buff *skb, const struct nft_expr *expr)
{
- const struct nft_limit_pkts *priv = nft_expr_priv(expr);
+ const struct nft_limit_priv_pkts *priv = nft_expr_priv(expr);
return nft_limit_dump(skb, &priv->limit, NFT_LIMIT_PKTS);
}
+static void nft_limit_pkts_destroy(const struct nft_ctx *ctx,
+ const struct nft_expr *expr)
+{
+ const struct nft_limit_priv_pkts *priv = nft_expr_priv(expr);
+
+ nft_limit_destroy(ctx, &priv->limit);
+}
+
+static int nft_limit_pkts_clone(struct nft_expr *dst, const struct nft_expr *src)
+{
+ struct nft_limit_priv_pkts *priv_dst = nft_expr_priv(dst);
+ struct nft_limit_priv_pkts *priv_src = nft_expr_priv(src);
+
+ return nft_limit_clone(&priv_dst->limit, &priv_src->limit);
+}
+
static struct nft_expr_type nft_limit_type;
static const struct nft_expr_ops nft_limit_pkts_ops = {
.type = &nft_limit_type,
- .size = NFT_EXPR_SIZE(sizeof(struct nft_limit_pkts)),
+ .size = NFT_EXPR_SIZE(sizeof(struct nft_limit_priv_pkts)),
.eval = nft_limit_pkts_eval,
.init = nft_limit_pkts_init,
+ .destroy = nft_limit_pkts_destroy,
+ .clone = nft_limit_pkts_clone,
.dump = nft_limit_pkts_dump,
};
@@ -179,7 +231,7 @@ static void nft_limit_bytes_eval(const struct nft_expr *expr,
struct nft_regs *regs,
const struct nft_pktinfo *pkt)
{
- struct nft_limit *priv = nft_expr_priv(expr);
+ struct nft_limit_priv *priv = nft_expr_priv(expr);
u64 cost = div64_u64(priv->nsecs * pkt->skb->len, priv->rate);
if (nft_limit_eval(priv, cost))
@@ -190,7 +242,7 @@ static int nft_limit_bytes_init(const struct nft_ctx *ctx,
const struct nft_expr *expr,
const struct nlattr * const tb[])
{
- struct nft_limit *priv = nft_expr_priv(expr);
+ struct nft_limit_priv *priv = nft_expr_priv(expr);
return nft_limit_init(priv, tb, false);
}
@@ -198,17 +250,35 @@ static int nft_limit_bytes_init(const struct nft_ctx *ctx,
static int nft_limit_bytes_dump(struct sk_buff *skb,
const struct nft_expr *expr)
{
- const struct nft_limit *priv = nft_expr_priv(expr);
+ const struct nft_limit_priv *priv = nft_expr_priv(expr);
return nft_limit_dump(skb, priv, NFT_LIMIT_PKT_BYTES);
}
+static void nft_limit_bytes_destroy(const struct nft_ctx *ctx,
+ const struct nft_expr *expr)
+{
+ const struct nft_limit_priv *priv = nft_expr_priv(expr);
+
+ nft_limit_destroy(ctx, priv);
+}
+
+static int nft_limit_bytes_clone(struct nft_expr *dst, const struct nft_expr *src)
+{
+ struct nft_limit_priv *priv_dst = nft_expr_priv(dst);
+ struct nft_limit_priv *priv_src = nft_expr_priv(src);
+
+ return nft_limit_clone(priv_dst, priv_src);
+}
+
static const struct nft_expr_ops nft_limit_bytes_ops = {
.type = &nft_limit_type,
- .size = NFT_EXPR_SIZE(sizeof(struct nft_limit)),
+ .size = NFT_EXPR_SIZE(sizeof(struct nft_limit_priv)),
.eval = nft_limit_bytes_eval,
.init = nft_limit_bytes_init,
.dump = nft_limit_bytes_dump,
+ .clone = nft_limit_bytes_clone,
+ .destroy = nft_limit_bytes_destroy,
};
static const struct nft_expr_ops *
@@ -240,7 +310,7 @@ static void nft_limit_obj_pkts_eval(struct nft_object *obj,
struct nft_regs *regs,
const struct nft_pktinfo *pkt)
{
- struct nft_limit_pkts *priv = nft_obj_data(obj);
+ struct nft_limit_priv_pkts *priv = nft_obj_data(obj);
if (nft_limit_eval(&priv->limit, priv->cost))
regs->verdict.code = NFT_BREAK;
@@ -250,7 +320,7 @@ static int nft_limit_obj_pkts_init(const struct nft_ctx *ctx,
const struct nlattr * const tb[],
struct nft_object *obj)
{
- struct nft_limit_pkts *priv = nft_obj_data(obj);
+ struct nft_limit_priv_pkts *priv = nft_obj_data(obj);
int err;
err = nft_limit_init(&priv->limit, tb, true);
@@ -265,7 +335,7 @@ static int nft_limit_obj_pkts_dump(struct sk_buff *skb,
struct nft_object *obj,
bool reset)
{
- const struct nft_limit_pkts *priv = nft_obj_data(obj);
+ const struct nft_limit_priv_pkts *priv = nft_obj_data(obj);
return nft_limit_dump(skb, &priv->limit, NFT_LIMIT_PKTS);
}
@@ -273,7 +343,7 @@ static int nft_limit_obj_pkts_dump(struct sk_buff *skb,
static struct nft_object_type nft_limit_obj_type;
static const struct nft_object_ops nft_limit_obj_pkts_ops = {
.type = &nft_limit_obj_type,
- .size = NFT_EXPR_SIZE(sizeof(struct nft_limit_pkts)),
+ .size = NFT_EXPR_SIZE(sizeof(struct nft_limit_priv_pkts)),
.init = nft_limit_obj_pkts_init,
.eval = nft_limit_obj_pkts_eval,
.dump = nft_limit_obj_pkts_dump,
@@ -283,7 +353,7 @@ static void nft_limit_obj_bytes_eval(struct nft_object *obj,
struct nft_regs *regs,
const struct nft_pktinfo *pkt)
{
- struct nft_limit *priv = nft_obj_data(obj);
+ struct nft_limit_priv *priv = nft_obj_data(obj);
u64 cost = div64_u64(priv->nsecs * pkt->skb->len, priv->rate);
if (nft_limit_eval(priv, cost))
@@ -294,7 +364,7 @@ static int nft_limit_obj_bytes_init(const struct nft_ctx *ctx,
const struct nlattr * const tb[],
struct nft_object *obj)
{
- struct nft_limit *priv = nft_obj_data(obj);
+ struct nft_limit_priv *priv = nft_obj_data(obj);
return nft_limit_init(priv, tb, false);
}
@@ -303,7 +373,7 @@ static int nft_limit_obj_bytes_dump(struct sk_buff *skb,
struct nft_object *obj,
bool reset)
{
- const struct nft_limit *priv = nft_obj_data(obj);
+ const struct nft_limit_priv *priv = nft_obj_data(obj);
return nft_limit_dump(skb, priv, NFT_LIMIT_PKT_BYTES);
}
@@ -311,7 +381,7 @@ static int nft_limit_obj_bytes_dump(struct sk_buff *skb,
static struct nft_object_type nft_limit_obj_type;
static const struct nft_object_ops nft_limit_obj_bytes_ops = {
.type = &nft_limit_obj_type,
- .size = sizeof(struct nft_limit),
+ .size = sizeof(struct nft_limit_priv),
.init = nft_limit_obj_bytes_init,
.eval = nft_limit_obj_bytes_eval,
.dump = nft_limit_obj_bytes_dump,
diff --git a/net/netfilter/nft_meta.c b/net/netfilter/nft_meta.c
index fe91ff5..5ab4df5 100644
--- a/net/netfilter/nft_meta.c
+++ b/net/netfilter/nft_meta.c
@@ -750,16 +750,63 @@ static int nft_meta_get_offload(struct nft_offload_ctx *ctx,
return 0;
}
+static bool nft_meta_get_reduce(struct nft_regs_track *track,
+ const struct nft_expr *expr)
+{
+ const struct nft_meta *priv = nft_expr_priv(expr);
+ const struct nft_meta *meta;
+
+ if (!track->regs[priv->dreg].selector ||
+ track->regs[priv->dreg].selector->ops != expr->ops) {
+ track->regs[priv->dreg].selector = expr;
+ track->regs[priv->dreg].bitwise = NULL;
+ return false;
+ }
+
+ meta = nft_expr_priv(track->regs[priv->dreg].selector);
+ if (priv->key != meta->key ||
+ priv->dreg != meta->dreg) {
+ track->regs[priv->dreg].selector = expr;
+ track->regs[priv->dreg].bitwise = NULL;
+ return false;
+ }
+
+ if (!track->regs[priv->dreg].bitwise)
+ return true;
+
+ return nft_expr_reduce_bitwise(track, expr);
+}
+
static const struct nft_expr_ops nft_meta_get_ops = {
.type = &nft_meta_type,
.size = NFT_EXPR_SIZE(sizeof(struct nft_meta)),
.eval = nft_meta_get_eval,
.init = nft_meta_get_init,
.dump = nft_meta_get_dump,
+ .reduce = nft_meta_get_reduce,
.validate = nft_meta_get_validate,
.offload = nft_meta_get_offload,
};
+static bool nft_meta_set_reduce(struct nft_regs_track *track,
+ const struct nft_expr *expr)
+{
+ int i;
+
+ for (i = 0; i < NFT_REG32_NUM; i++) {
+ if (!track->regs[i].selector)
+ continue;
+
+ if (track->regs[i].selector->ops != &nft_meta_get_ops)
+ continue;
+
+ track->regs[i].selector = NULL;
+ track->regs[i].bitwise = NULL;
+ }
+
+ return false;
+}
+
static const struct nft_expr_ops nft_meta_set_ops = {
.type = &nft_meta_type,
.size = NFT_EXPR_SIZE(sizeof(struct nft_meta)),
@@ -767,6 +814,7 @@ static const struct nft_expr_ops nft_meta_set_ops = {
.init = nft_meta_set_init,
.destroy = nft_meta_set_destroy,
.dump = nft_meta_set_dump,
+ .reduce = nft_meta_set_reduce,
.validate = nft_meta_set_validate,
};
diff --git a/net/netfilter/nft_numgen.c b/net/netfilter/nft_numgen.c
index 722cac1..1d378ef 100644
--- a/net/netfilter/nft_numgen.c
+++ b/net/netfilter/nft_numgen.c
@@ -18,7 +18,7 @@ static DEFINE_PER_CPU(struct rnd_state, nft_numgen_prandom_state);
struct nft_ng_inc {
u8 dreg;
u32 modulus;
- atomic_t counter;
+ atomic_t *counter;
u32 offset;
};
@@ -27,9 +27,9 @@ static u32 nft_ng_inc_gen(struct nft_ng_inc *priv)
u32 nval, oval;
do {
- oval = atomic_read(&priv->counter);
+ oval = atomic_read(priv->counter);
nval = (oval + 1 < priv->modulus) ? oval + 1 : 0;
- } while (atomic_cmpxchg(&priv->counter, oval, nval) != oval);
+ } while (atomic_cmpxchg(priv->counter, oval, nval) != oval);
return nval + priv->offset;
}
@@ -55,6 +55,7 @@ static int nft_ng_inc_init(const struct nft_ctx *ctx,
const struct nlattr * const tb[])
{
struct nft_ng_inc *priv = nft_expr_priv(expr);
+ int err;
if (tb[NFTA_NG_OFFSET])
priv->offset = ntohl(nla_get_be32(tb[NFTA_NG_OFFSET]));
@@ -66,10 +67,22 @@ static int nft_ng_inc_init(const struct nft_ctx *ctx,
if (priv->offset + priv->modulus - 1 < priv->offset)
return -EOVERFLOW;
- atomic_set(&priv->counter, priv->modulus - 1);
+ priv->counter = kmalloc(sizeof(*priv->counter), GFP_KERNEL);
+ if (!priv->counter)
+ return -ENOMEM;
- return nft_parse_register_store(ctx, tb[NFTA_NG_DREG], &priv->dreg,
- NULL, NFT_DATA_VALUE, sizeof(u32));
+ atomic_set(priv->counter, priv->modulus - 1);
+
+ err = nft_parse_register_store(ctx, tb[NFTA_NG_DREG], &priv->dreg,
+ NULL, NFT_DATA_VALUE, sizeof(u32));
+ if (err < 0)
+ goto err;
+
+ return 0;
+err:
+ kfree(priv->counter);
+
+ return err;
}
static int nft_ng_dump(struct sk_buff *skb, enum nft_registers dreg,
@@ -98,6 +111,14 @@ static int nft_ng_inc_dump(struct sk_buff *skb, const struct nft_expr *expr)
priv->offset);
}
+static void nft_ng_inc_destroy(const struct nft_ctx *ctx,
+ const struct nft_expr *expr)
+{
+ const struct nft_ng_inc *priv = nft_expr_priv(expr);
+
+ kfree(priv->counter);
+}
+
struct nft_ng_random {
u8 dreg;
u32 modulus;
@@ -157,6 +178,7 @@ static const struct nft_expr_ops nft_ng_inc_ops = {
.size = NFT_EXPR_SIZE(sizeof(struct nft_ng_inc)),
.eval = nft_ng_inc_eval,
.init = nft_ng_inc_init,
+ .destroy = nft_ng_inc_destroy,
.dump = nft_ng_inc_dump,
};
diff --git a/net/netfilter/nft_payload.c b/net/netfilter/nft_payload.c
index bd68993..940fed9 100644
--- a/net/netfilter/nft_payload.c
+++ b/net/netfilter/nft_payload.c
@@ -157,7 +157,8 @@ void nft_payload_eval(const struct nft_expr *expr,
goto err;
break;
default:
- BUG();
+ WARN_ON_ONCE(1);
+ goto err;
}
offset += priv->offset;
@@ -209,6 +210,34 @@ static int nft_payload_dump(struct sk_buff *skb, const struct nft_expr *expr)
return -1;
}
+static bool nft_payload_reduce(struct nft_regs_track *track,
+ const struct nft_expr *expr)
+{
+ const struct nft_payload *priv = nft_expr_priv(expr);
+ const struct nft_payload *payload;
+
+ if (!track->regs[priv->dreg].selector ||
+ track->regs[priv->dreg].selector->ops != expr->ops) {
+ track->regs[priv->dreg].selector = expr;
+ track->regs[priv->dreg].bitwise = NULL;
+ return false;
+ }
+
+ payload = nft_expr_priv(track->regs[priv->dreg].selector);
+ if (priv->base != payload->base ||
+ priv->offset != payload->offset ||
+ priv->len != payload->len) {
+ track->regs[priv->dreg].selector = expr;
+ track->regs[priv->dreg].bitwise = NULL;
+ return false;
+ }
+
+ if (!track->regs[priv->dreg].bitwise)
+ return true;
+
+ return nft_expr_reduce_bitwise(track, expr);
+}
+
static bool nft_payload_offload_mask(struct nft_offload_reg *reg,
u32 priv_len, u32 field_len)
{
@@ -512,6 +541,7 @@ static const struct nft_expr_ops nft_payload_ops = {
.eval = nft_payload_eval,
.init = nft_payload_init,
.dump = nft_payload_dump,
+ .reduce = nft_payload_reduce,
.offload = nft_payload_offload,
};
@@ -521,6 +551,7 @@ const struct nft_expr_ops nft_payload_fast_ops = {
.eval = nft_payload_eval,
.init = nft_payload_init,
.dump = nft_payload_dump,
+ .reduce = nft_payload_reduce,
.offload = nft_payload_offload,
};
@@ -546,6 +577,9 @@ static int nft_payload_l4csum_offset(const struct nft_pktinfo *pkt,
struct sk_buff *skb,
unsigned int *l4csum_offset)
{
+ if (pkt->fragoff)
+ return -1;
+
switch (pkt->tprot) {
case IPPROTO_TCP:
*l4csum_offset = offsetof(struct tcphdr, check);
@@ -664,7 +698,8 @@ static void nft_payload_set_eval(const struct nft_expr *expr,
goto err;
break;
default:
- BUG();
+ WARN_ON_ONCE(1);
+ goto err;
}
csum_offset = offset + priv->csum_offset;
@@ -766,12 +801,33 @@ static int nft_payload_set_dump(struct sk_buff *skb, const struct nft_expr *expr
return -1;
}
+static bool nft_payload_set_reduce(struct nft_regs_track *track,
+ const struct nft_expr *expr)
+{
+ int i;
+
+ for (i = 0; i < NFT_REG32_NUM; i++) {
+ if (!track->regs[i].selector)
+ continue;
+
+ if (track->regs[i].selector->ops != &nft_payload_ops &&
+ track->regs[i].selector->ops != &nft_payload_fast_ops)
+ continue;
+
+ track->regs[i].selector = NULL;
+ track->regs[i].bitwise = NULL;
+ }
+
+ return false;
+}
+
static const struct nft_expr_ops nft_payload_set_ops = {
.type = &nft_payload_type,
.size = NFT_EXPR_SIZE(sizeof(struct nft_payload_set)),
.eval = nft_payload_set_eval,
.init = nft_payload_set_init,
.dump = nft_payload_set_dump,
+ .reduce = nft_payload_set_reduce,
};
static const struct nft_expr_ops *
diff --git a/net/netfilter/nft_quota.c b/net/netfilter/nft_quota.c
index c4d1389..0484aef 100644
--- a/net/netfilter/nft_quota.c
+++ b/net/netfilter/nft_quota.c
@@ -15,13 +15,13 @@
struct nft_quota {
atomic64_t quota;
unsigned long flags;
- atomic64_t consumed;
+ atomic64_t *consumed;
};
static inline bool nft_overquota(struct nft_quota *priv,
const struct sk_buff *skb)
{
- return atomic64_add_return(skb->len, &priv->consumed) >=
+ return atomic64_add_return(skb->len, priv->consumed) >=
atomic64_read(&priv->quota);
}
@@ -90,13 +90,23 @@ static int nft_quota_do_init(const struct nlattr * const tb[],
return -EOPNOTSUPP;
}
+ priv->consumed = kmalloc(sizeof(*priv->consumed), GFP_KERNEL);
+ if (!priv->consumed)
+ return -ENOMEM;
+
atomic64_set(&priv->quota, quota);
priv->flags = flags;
- atomic64_set(&priv->consumed, consumed);
+ atomic64_set(priv->consumed, consumed);
return 0;
}
+static void nft_quota_do_destroy(const struct nft_ctx *ctx,
+ struct nft_quota *priv)
+{
+ kfree(priv->consumed);
+}
+
static int nft_quota_obj_init(const struct nft_ctx *ctx,
const struct nlattr * const tb[],
struct nft_object *obj)
@@ -128,7 +138,7 @@ static int nft_quota_do_dump(struct sk_buff *skb, struct nft_quota *priv,
* that we see, don't go over the quota boundary in what we send to
* userspace.
*/
- consumed = atomic64_read(&priv->consumed);
+ consumed = atomic64_read(priv->consumed);
quota = atomic64_read(&priv->quota);
if (consumed >= quota) {
consumed_cap = quota;
@@ -145,7 +155,7 @@ static int nft_quota_do_dump(struct sk_buff *skb, struct nft_quota *priv,
goto nla_put_failure;
if (reset) {
- atomic64_sub(consumed, &priv->consumed);
+ atomic64_sub(consumed, priv->consumed);
clear_bit(NFT_QUOTA_DEPLETED_BIT, &priv->flags);
}
return 0;
@@ -162,11 +172,20 @@ static int nft_quota_obj_dump(struct sk_buff *skb, struct nft_object *obj,
return nft_quota_do_dump(skb, priv, reset);
}
+static void nft_quota_obj_destroy(const struct nft_ctx *ctx,
+ struct nft_object *obj)
+{
+ struct nft_quota *priv = nft_obj_data(obj);
+
+ return nft_quota_do_destroy(ctx, priv);
+}
+
static struct nft_object_type nft_quota_obj_type;
static const struct nft_object_ops nft_quota_obj_ops = {
.type = &nft_quota_obj_type,
.size = sizeof(struct nft_quota),
.init = nft_quota_obj_init,
+ .destroy = nft_quota_obj_destroy,
.eval = nft_quota_obj_eval,
.dump = nft_quota_obj_dump,
.update = nft_quota_obj_update,
@@ -205,12 +224,35 @@ static int nft_quota_dump(struct sk_buff *skb, const struct nft_expr *expr)
return nft_quota_do_dump(skb, priv, false);
}
+static void nft_quota_destroy(const struct nft_ctx *ctx,
+ const struct nft_expr *expr)
+{
+ struct nft_quota *priv = nft_expr_priv(expr);
+
+ return nft_quota_do_destroy(ctx, priv);
+}
+
+static int nft_quota_clone(struct nft_expr *dst, const struct nft_expr *src)
+{
+ struct nft_quota *priv_dst = nft_expr_priv(dst);
+
+ priv_dst->consumed = kmalloc(sizeof(*priv_dst->consumed), GFP_ATOMIC);
+ if (priv_dst->consumed)
+ return -ENOMEM;
+
+ atomic64_set(priv_dst->consumed, 0);
+
+ return 0;
+}
+
static struct nft_expr_type nft_quota_type;
static const struct nft_expr_ops nft_quota_ops = {
.type = &nft_quota_type,
.size = NFT_EXPR_SIZE(sizeof(struct nft_quota)),
.eval = nft_quota_eval,
.init = nft_quota_init,
+ .destroy = nft_quota_destroy,
+ .clone = nft_quota_clone,
.dump = nft_quota_dump,
};
diff --git a/net/netfilter/nft_reject_netdev.c b/net/netfilter/nft_reject_netdev.c
index d89f687..61cd8c4 100644
--- a/net/netfilter/nft_reject_netdev.c
+++ b/net/netfilter/nft_reject_netdev.c
@@ -4,6 +4,7 @@
* Copyright (c) 2020 Jose M. Guisado <guigom@riseup.net>
*/
+#include <linux/etherdevice.h>
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/module.h>
diff --git a/net/netfilter/nft_set_pipapo.c b/net/netfilter/nft_set_pipapo.c
index dce866d..2c8051d 100644
--- a/net/netfilter/nft_set_pipapo.c
+++ b/net/netfilter/nft_set_pipapo.c
@@ -1290,6 +1290,11 @@ static struct nft_pipapo_match *pipapo_clone(struct nft_pipapo_match *old)
if (!new->scratch_aligned)
goto out_scratch;
#endif
+ for_each_possible_cpu(i)
+ *per_cpu_ptr(new->scratch, i) = NULL;
+
+ if (pipapo_realloc_scratch(new, old->bsize_max))
+ goto out_scratch_realloc;
rcu_head_init(&new->rcu);
@@ -1334,6 +1339,9 @@ static struct nft_pipapo_match *pipapo_clone(struct nft_pipapo_match *old)
kvfree(dst->lt);
dst--;
}
+out_scratch_realloc:
+ for_each_possible_cpu(i)
+ kfree(*per_cpu_ptr(new->scratch, i));
#ifdef NFT_PIPAPO_ALIGN
free_percpu(new->scratch_aligned);
#endif
diff --git a/net/netfilter/nft_set_pipapo_avx2.c b/net/netfilter/nft_set_pipapo_avx2.c
index 6f4116e..52e0d02 100644
--- a/net/netfilter/nft_set_pipapo_avx2.c
+++ b/net/netfilter/nft_set_pipapo_avx2.c
@@ -1048,11 +1048,9 @@ static int nft_pipapo_avx2_lookup_slow(unsigned long *map, unsigned long *fill,
struct nft_pipapo_field *f, int offset,
const u8 *pkt, bool first, bool last)
{
- unsigned long *lt = f->lt, bsize = f->bsize;
+ unsigned long bsize = f->bsize;
int i, ret = -1, b;
- lt += offset * NFT_PIPAPO_LONGS_PER_M256;
-
if (first)
memset(map, 0xff, bsize * sizeof(*map));
diff --git a/net/netfilter/xt_CT.c b/net/netfilter/xt_CT.c
index 0a913ce..267757b 100644
--- a/net/netfilter/xt_CT.c
+++ b/net/netfilter/xt_CT.c
@@ -24,7 +24,7 @@ static inline int xt_ct_target(struct sk_buff *skb, struct nf_conn *ct)
return XT_CONTINUE;
if (ct) {
- atomic_inc(&ct->ct_general.use);
+ refcount_inc(&ct->ct_general.use);
nf_ct_set(skb, ct, IP_CT_NEW);
} else {
nf_ct_set(skb, ct, IP_CT_UNTRACKED);
@@ -201,7 +201,6 @@ static int xt_ct_tg_check(const struct xt_tgchk_param *par,
goto err4;
}
__set_bit(IPS_CONFIRMED_BIT, &ct->status);
- nf_conntrack_get(&ct->ct_general);
out:
info->ct = ct;
return 0;
diff --git a/net/netlink/af_netlink.c b/net/netlink/af_netlink.c
index 9eba2e6..7b34403 100644
--- a/net/netlink/af_netlink.c
+++ b/net/netlink/af_netlink.c
@@ -20,8 +20,10 @@
#include <linux/module.h>
+#include <linux/bpf.h>
#include <linux/capability.h>
#include <linux/kernel.h>
+#include <linux/filter.h>
#include <linux/init.h>
#include <linux/signal.h>
#include <linux/sched.h>
@@ -707,9 +709,7 @@ static int netlink_create(struct net *net, struct socket *sock, int protocol,
if (err < 0)
goto out_module;
- local_bh_disable();
sock_prot_inuse_add(net, &netlink_proto, 1);
- local_bh_enable();
nlk = nlk_sk(sock->sk);
nlk->module = module;
@@ -809,9 +809,7 @@ static int netlink_release(struct socket *sock)
netlink_table_ungrab();
}
- local_bh_disable();
sock_prot_inuse_add(sock_net(sk), &netlink_proto, -1);
- local_bh_enable();
call_rcu(&nlk->rcu, deferred_put_nlk_sk);
return 0;
}
diff --git a/net/netrom/af_netrom.c b/net/netrom/af_netrom.c
index 775064c..fa9dc2b 100644
--- a/net/netrom/af_netrom.c
+++ b/net/netrom/af_netrom.c
@@ -298,7 +298,7 @@ static int nr_setsockopt(struct socket *sock, int level, int optname,
{
struct sock *sk = sock->sk;
struct nr_sock *nr = nr_sk(sk);
- unsigned long opt;
+ unsigned int opt;
if (level != SOL_NETROM)
return -ENOPROTOOPT;
@@ -306,18 +306,18 @@ static int nr_setsockopt(struct socket *sock, int level, int optname,
if (optlen < sizeof(unsigned int))
return -EINVAL;
- if (copy_from_sockptr(&opt, optval, sizeof(unsigned int)))
+ if (copy_from_sockptr(&opt, optval, sizeof(opt)))
return -EFAULT;
switch (optname) {
case NETROM_T1:
- if (opt < 1 || opt > ULONG_MAX / HZ)
+ if (opt < 1 || opt > UINT_MAX / HZ)
return -EINVAL;
nr->t1 = opt * HZ;
return 0;
case NETROM_T2:
- if (opt < 1 || opt > ULONG_MAX / HZ)
+ if (opt < 1 || opt > UINT_MAX / HZ)
return -EINVAL;
nr->t2 = opt * HZ;
return 0;
@@ -329,13 +329,13 @@ static int nr_setsockopt(struct socket *sock, int level, int optname,
return 0;
case NETROM_T4:
- if (opt < 1 || opt > ULONG_MAX / HZ)
+ if (opt < 1 || opt > UINT_MAX / HZ)
return -EINVAL;
nr->t4 = opt * HZ;
return 0;
case NETROM_IDLE:
- if (opt > ULONG_MAX / (60 * HZ))
+ if (opt > UINT_MAX / (60 * HZ))
return -EINVAL;
nr->idle = opt * 60 * HZ;
return 0;
diff --git a/net/openvswitch/conntrack.c b/net/openvswitch/conntrack.c
index 1b5eae5..c07afff 100644
--- a/net/openvswitch/conntrack.c
+++ b/net/openvswitch/conntrack.c
@@ -25,6 +25,8 @@
#include <net/netfilter/nf_nat.h>
#endif
+#include <net/netfilter/nf_conntrack_act_ct.h>
+
#include "datapath.h"
#include "conntrack.h"
#include "flow.h"
@@ -574,7 +576,7 @@ ovs_ct_expect_find(struct net *net, const struct nf_conntrack_zone *zone,
struct nf_conn *ct = nf_ct_tuplehash_to_ctrack(h);
nf_ct_delete(ct, 0, 0);
- nf_conntrack_put(&ct->ct_general);
+ nf_ct_put(ct);
}
}
@@ -723,7 +725,7 @@ static bool skb_nfct_cached(struct net *net,
if (nf_ct_is_confirmed(ct))
nf_ct_delete(ct, 0, 0);
- nf_conntrack_put(&ct->ct_general);
+ nf_ct_put(ct);
nf_ct_set(skb, NULL, 0);
return false;
}
@@ -967,7 +969,8 @@ static int __ovs_ct_lookup(struct net *net, struct sw_flow_key *key,
/* Associate skb with specified zone. */
if (tmpl) {
- nf_conntrack_put(skb_nfct(skb));
+ ct = nf_ct_get(skb, &ctinfo);
+ nf_ct_put(ct);
nf_conntrack_get(&tmpl->ct_general);
nf_ct_set(skb, tmpl, IP_CT_NEW);
}
@@ -1045,6 +1048,8 @@ static int __ovs_ct_lookup(struct net *net, struct sw_flow_key *key,
*/
nf_ct_set_tcp_be_liberal(ct);
}
+
+ nf_conn_act_ct_ext_fill(skb, ct, ctinfo);
}
return 0;
@@ -1245,6 +1250,8 @@ static int ovs_ct_commit(struct net *net, struct sw_flow_key *key,
&info->labels.mask);
if (err)
return err;
+
+ nf_conn_act_ct_ext_add(ct);
} else if (IS_ENABLED(CONFIG_NF_CONNTRACK_LABELS) &&
labels_nonzero(&info->labels.mask)) {
err = ovs_ct_set_labels(ct, key, &info->labels.value,
@@ -1328,7 +1335,12 @@ int ovs_ct_execute(struct net *net, struct sk_buff *skb,
int ovs_ct_clear(struct sk_buff *skb, struct sw_flow_key *key)
{
- nf_conntrack_put(skb_nfct(skb));
+ enum ip_conntrack_info ctinfo;
+ struct nf_conn *ct;
+
+ ct = nf_ct_get(skb, &ctinfo);
+
+ nf_ct_put(ct);
nf_ct_set(skb, NULL, IP_CT_UNTRACKED);
ovs_ct_fill_key(skb, key, false);
@@ -1716,7 +1728,6 @@ int ovs_ct_copy_action(struct net *net, const struct nlattr *attr,
goto err_free_ct;
__set_bit(IPS_CONFIRMED_BIT, &ct_info.ct->status);
- nf_conntrack_get(&ct_info.ct->ct_general);
return 0;
err_free_ct:
__ovs_ct_free_action(&ct_info);
diff --git a/net/openvswitch/flow.c b/net/openvswitch/flow.c
index 6d262d9..02096f2 100644
--- a/net/openvswitch/flow.c
+++ b/net/openvswitch/flow.c
@@ -859,7 +859,7 @@ int ovs_flow_key_extract(const struct ip_tunnel_info *tun_info,
#if IS_ENABLED(CONFIG_NET_TC_SKB_EXT)
struct tc_skb_ext *tc_ext;
#endif
- bool post_ct = false;
+ bool post_ct = false, post_ct_snat = false, post_ct_dnat = false;
int res, err;
u16 zone = 0;
@@ -900,6 +900,8 @@ int ovs_flow_key_extract(const struct ip_tunnel_info *tun_info,
key->recirc_id = tc_ext ? tc_ext->chain : 0;
OVS_CB(skb)->mru = tc_ext ? tc_ext->mru : 0;
post_ct = tc_ext ? tc_ext->post_ct : false;
+ post_ct_snat = post_ct ? tc_ext->post_ct_snat : false;
+ post_ct_dnat = post_ct ? tc_ext->post_ct_dnat : false;
zone = post_ct ? tc_ext->zone : 0;
} else {
key->recirc_id = 0;
@@ -911,8 +913,16 @@ int ovs_flow_key_extract(const struct ip_tunnel_info *tun_info,
err = key_extract(skb, key);
if (!err) {
ovs_ct_fill_key(skb, key, post_ct); /* Must be after key_extract(). */
- if (post_ct && !skb_get_nfct(skb))
- key->ct_zone = zone;
+ if (post_ct) {
+ if (!skb_get_nfct(skb)) {
+ key->ct_zone = zone;
+ } else {
+ if (!post_ct_dnat)
+ key->ct_state &= ~OVS_CS_F_DST_NAT;
+ if (!post_ct_snat)
+ key->ct_state &= ~OVS_CS_F_SRC_NAT;
+ }
+ }
}
return err;
}
diff --git a/net/openvswitch/vport-netdev.c b/net/openvswitch/vport-netdev.c
index 8e1a88f..b498dac 100644
--- a/net/openvswitch/vport-netdev.c
+++ b/net/openvswitch/vport-netdev.c
@@ -82,7 +82,7 @@ struct vport *ovs_netdev_link(struct vport *vport, const char *name)
err = -ENODEV;
goto error_free_vport;
}
-
+ netdev_tracker_alloc(vport->dev, &vport->dev_tracker, GFP_KERNEL);
if (vport->dev->flags & IFF_LOOPBACK ||
(vport->dev->type != ARPHRD_ETHER &&
vport->dev->type != ARPHRD_NONE) ||
@@ -115,7 +115,7 @@ struct vport *ovs_netdev_link(struct vport *vport, const char *name)
error_unlock:
rtnl_unlock();
error_put:
- dev_put(vport->dev);
+ dev_put_track(vport->dev, &vport->dev_tracker);
error_free_vport:
ovs_vport_free(vport);
return ERR_PTR(err);
@@ -137,8 +137,7 @@ static void vport_netdev_free(struct rcu_head *rcu)
{
struct vport *vport = container_of(rcu, struct vport, rcu);
- if (vport->dev)
- dev_put(vport->dev);
+ dev_put_track(vport->dev, &vport->dev_tracker);
ovs_vport_free(vport);
}
@@ -174,7 +173,7 @@ void ovs_netdev_tunnel_destroy(struct vport *vport)
*/
if (vport->dev->reg_state == NETREG_REGISTERED)
rtnl_delete_link(vport->dev);
- dev_put(vport->dev);
+ dev_put_track(vport->dev, &vport->dev_tracker);
vport->dev = NULL;
rtnl_unlock();
diff --git a/net/openvswitch/vport.h b/net/openvswitch/vport.h
index 8a930ca..9de5030 100644
--- a/net/openvswitch/vport.h
+++ b/net/openvswitch/vport.h
@@ -58,6 +58,7 @@ struct vport_portids {
/**
* struct vport - one port within a datapath
* @dev: Pointer to net_device.
+ * @dev_tracker: refcount tracker for @dev reference
* @dp: Datapath to which this port belongs.
* @upcall_portids: RCU protected 'struct vport_portids'.
* @port_no: Index into @dp's @ports array.
@@ -69,6 +70,7 @@ struct vport_portids {
*/
struct vport {
struct net_device *dev;
+ netdevice_tracker dev_tracker;
struct datapath *dp;
struct vport_portids __rcu *upcall_portids;
u16 port_no;
diff --git a/net/packet/af_packet.c b/net/packet/af_packet.c
index 76c2dca..5bd409a 100644
--- a/net/packet/af_packet.c
+++ b/net/packet/af_packet.c
@@ -49,6 +49,7 @@
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
#include <linux/ethtool.h>
+#include <linux/filter.h>
#include <linux/types.h>
#include <linux/mm.h>
#include <linux/capability.h>
@@ -3102,16 +3103,14 @@ static int packet_release(struct socket *sock)
sk_del_node_init_rcu(sk);
mutex_unlock(&net->packet.sklist_lock);
- preempt_disable();
sock_prot_inuse_add(net, sk->sk_prot, -1);
- preempt_enable();
spin_lock(&po->bind_lock);
unregister_prot_hook(sk, false);
packet_cached_dev_reset(po);
if (po->prot_hook.dev) {
- dev_put(po->prot_hook.dev);
+ dev_put_track(po->prot_hook.dev, &po->prot_hook.dev_tracker);
po->prot_hook.dev = NULL;
}
spin_unlock(&po->bind_lock);
@@ -3163,12 +3162,10 @@ static int packet_do_bind(struct sock *sk, const char *name, int ifindex,
__be16 proto)
{
struct packet_sock *po = pkt_sk(sk);
- struct net_device *dev_curr;
- __be16 proto_curr;
- bool need_rehook;
struct net_device *dev = NULL;
- int ret = 0;
bool unlisted = false;
+ bool need_rehook;
+ int ret = 0;
lock_sock(sk);
spin_lock(&po->bind_lock);
@@ -3193,14 +3190,10 @@ static int packet_do_bind(struct sock *sk, const char *name, int ifindex,
}
}
- dev_hold(dev);
-
- proto_curr = po->prot_hook.type;
- dev_curr = po->prot_hook.dev;
-
- need_rehook = proto_curr != proto || dev_curr != dev;
+ need_rehook = po->prot_hook.type != proto || po->prot_hook.dev != dev;
if (need_rehook) {
+ dev_hold(dev);
if (po->running) {
rcu_read_unlock();
/* prevents packet_notifier() from calling
@@ -3209,7 +3202,6 @@ static int packet_do_bind(struct sock *sk, const char *name, int ifindex,
WRITE_ONCE(po->num, 0);
__unregister_prot_hook(sk, true);
rcu_read_lock();
- dev_curr = po->prot_hook.dev;
if (dev)
unlisted = !dev_get_by_index_rcu(sock_net(sk),
dev->ifindex);
@@ -3219,18 +3211,21 @@ static int packet_do_bind(struct sock *sk, const char *name, int ifindex,
WRITE_ONCE(po->num, proto);
po->prot_hook.type = proto;
+ dev_put_track(po->prot_hook.dev, &po->prot_hook.dev_tracker);
+
if (unlikely(unlisted)) {
- dev_put(dev);
po->prot_hook.dev = NULL;
WRITE_ONCE(po->ifindex, -1);
packet_cached_dev_reset(po);
} else {
+ dev_hold_track(dev, &po->prot_hook.dev_tracker,
+ GFP_ATOMIC);
po->prot_hook.dev = dev;
WRITE_ONCE(po->ifindex, dev ? dev->ifindex : 0);
packet_cached_dev_assign(po, dev);
}
+ dev_put(dev);
}
- dev_put(dev_curr);
if (proto == 0 || !need_rehook)
goto out_unlock;
@@ -3368,9 +3363,7 @@ static int packet_create(struct net *net, struct socket *sock, int protocol,
sk_add_node_tail_rcu(sk, &net->packet.sklist);
mutex_unlock(&net->packet.sklist_lock);
- preempt_disable();
sock_prot_inuse_add(net, &packet_proto, 1);
- preempt_enable();
return 0;
out2:
@@ -4142,7 +4135,8 @@ static int packet_notifier(struct notifier_block *this,
if (msg == NETDEV_UNREGISTER) {
packet_cached_dev_reset(po);
WRITE_ONCE(po->ifindex, -1);
- dev_put(po->prot_hook.dev);
+ dev_put_track(po->prot_hook.dev,
+ &po->prot_hook.dev_tracker);
po->prot_hook.dev = NULL;
}
spin_unlock(&po->bind_lock);
diff --git a/net/rds/send.c b/net/rds/send.c
index 5344439..0c55040 100644
--- a/net/rds/send.c
+++ b/net/rds/send.c
@@ -272,7 +272,7 @@ int rds_send_xmit(struct rds_conn_path *cp)
/* Unfortunately, the way Infiniband deals with
* RDMA to a bad MR key is by moving the entire
- * queue pair to error state. We cold possibly
+ * queue pair to error state. We could possibly
* recover from that, but right now we drop the
* connection.
* Therefore, we never retransmit messages with RDMA ops.
diff --git a/net/rfkill/core.c b/net/rfkill/core.c
index ac15a94..5b1927d 100644
--- a/net/rfkill/core.c
+++ b/net/rfkill/core.c
@@ -946,6 +946,18 @@ bool rfkill_blocked(struct rfkill *rfkill)
}
EXPORT_SYMBOL(rfkill_blocked);
+bool rfkill_soft_blocked(struct rfkill *rfkill)
+{
+ unsigned long flags;
+ u32 state;
+
+ spin_lock_irqsave(&rfkill->lock, flags);
+ state = rfkill->state;
+ spin_unlock_irqrestore(&rfkill->lock, flags);
+
+ return !!(state & RFKILL_BLOCK_SW);
+}
+EXPORT_SYMBOL(rfkill_soft_blocked);
struct rfkill * __must_check rfkill_alloc(const char *name,
struct device *parent,
diff --git a/net/rose/rose_in.c b/net/rose/rose_in.c
index 6af786d..4d67f36 100644
--- a/net/rose/rose_in.c
+++ b/net/rose/rose_in.c
@@ -9,6 +9,7 @@
* diagrams as the code is not obvious and probably very easy to break.
*/
#include <linux/errno.h>
+#include <linux/filter.h>
#include <linux/types.h>
#include <linux/socket.h>
#include <linux/in.h>
diff --git a/net/sched/act_api.c b/net/sched/act_api.c
index 3258da3..32563ce 100644
--- a/net/sched/act_api.c
+++ b/net/sched/act_api.c
@@ -19,8 +19,10 @@
#include <net/sock.h>
#include <net/sch_generic.h>
#include <net/pkt_cls.h>
+#include <net/tc_act/tc_pedit.h>
#include <net/act_api.h>
#include <net/netlink.h>
+#include <net/flow_offload.h>
#ifdef CONFIG_INET
DEFINE_STATIC_KEY_FALSE(tcf_frag_xmit_count);
@@ -129,8 +131,244 @@ static void free_tcf(struct tc_action *p)
kfree(p);
}
+static void offload_action_hw_count_set(struct tc_action *act,
+ u32 hw_count)
+{
+ act->in_hw_count = hw_count;
+}
+
+static void offload_action_hw_count_inc(struct tc_action *act,
+ u32 hw_count)
+{
+ act->in_hw_count += hw_count;
+}
+
+static void offload_action_hw_count_dec(struct tc_action *act,
+ u32 hw_count)
+{
+ act->in_hw_count = act->in_hw_count > hw_count ?
+ act->in_hw_count - hw_count : 0;
+}
+
+static unsigned int tcf_offload_act_num_actions_single(struct tc_action *act)
+{
+ if (is_tcf_pedit(act))
+ return tcf_pedit_nkeys(act);
+ else
+ return 1;
+}
+
+static bool tc_act_skip_hw(u32 flags)
+{
+ return (flags & TCA_ACT_FLAGS_SKIP_HW) ? true : false;
+}
+
+static bool tc_act_skip_sw(u32 flags)
+{
+ return (flags & TCA_ACT_FLAGS_SKIP_SW) ? true : false;
+}
+
+static bool tc_act_in_hw(struct tc_action *act)
+{
+ return !!act->in_hw_count;
+}
+
+/* SKIP_HW and SKIP_SW are mutually exclusive flags. */
+static bool tc_act_flags_valid(u32 flags)
+{
+ flags &= TCA_ACT_FLAGS_SKIP_HW | TCA_ACT_FLAGS_SKIP_SW;
+
+ return flags ^ (TCA_ACT_FLAGS_SKIP_HW | TCA_ACT_FLAGS_SKIP_SW);
+}
+
+static int offload_action_init(struct flow_offload_action *fl_action,
+ struct tc_action *act,
+ enum offload_act_command cmd,
+ struct netlink_ext_ack *extack)
+{
+ int err;
+
+ fl_action->extack = extack;
+ fl_action->command = cmd;
+ fl_action->index = act->tcfa_index;
+
+ if (act->ops->offload_act_setup) {
+ spin_lock_bh(&act->tcfa_lock);
+ err = act->ops->offload_act_setup(act, fl_action, NULL,
+ false);
+ spin_unlock_bh(&act->tcfa_lock);
+ return err;
+ }
+
+ return -EOPNOTSUPP;
+}
+
+static int tcf_action_offload_cmd_ex(struct flow_offload_action *fl_act,
+ u32 *hw_count)
+{
+ int err;
+
+ err = flow_indr_dev_setup_offload(NULL, NULL, TC_SETUP_ACT,
+ fl_act, NULL, NULL);
+ if (err < 0)
+ return err;
+
+ if (hw_count)
+ *hw_count = err;
+
+ return 0;
+}
+
+static int tcf_action_offload_cmd_cb_ex(struct flow_offload_action *fl_act,
+ u32 *hw_count,
+ flow_indr_block_bind_cb_t *cb,
+ void *cb_priv)
+{
+ int err;
+
+ err = cb(NULL, NULL, cb_priv, TC_SETUP_ACT, NULL, fl_act, NULL);
+ if (err < 0)
+ return err;
+
+ if (hw_count)
+ *hw_count = 1;
+
+ return 0;
+}
+
+static int tcf_action_offload_cmd(struct flow_offload_action *fl_act,
+ u32 *hw_count,
+ flow_indr_block_bind_cb_t *cb,
+ void *cb_priv)
+{
+ return cb ? tcf_action_offload_cmd_cb_ex(fl_act, hw_count,
+ cb, cb_priv) :
+ tcf_action_offload_cmd_ex(fl_act, hw_count);
+}
+
+static int tcf_action_offload_add_ex(struct tc_action *action,
+ struct netlink_ext_ack *extack,
+ flow_indr_block_bind_cb_t *cb,
+ void *cb_priv)
+{
+ bool skip_sw = tc_act_skip_sw(action->tcfa_flags);
+ struct tc_action *actions[TCA_ACT_MAX_PRIO] = {
+ [0] = action,
+ };
+ struct flow_offload_action *fl_action;
+ u32 in_hw_count = 0;
+ int num, err = 0;
+
+ if (tc_act_skip_hw(action->tcfa_flags))
+ return 0;
+
+ num = tcf_offload_act_num_actions_single(action);
+ fl_action = offload_action_alloc(num);
+ if (!fl_action)
+ return -ENOMEM;
+
+ err = offload_action_init(fl_action, action, FLOW_ACT_REPLACE, extack);
+ if (err)
+ goto fl_err;
+
+ err = tc_setup_action(&fl_action->action, actions);
+ if (err) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "Failed to setup tc actions for offload\n");
+ goto fl_err;
+ }
+
+ err = tcf_action_offload_cmd(fl_action, &in_hw_count, cb, cb_priv);
+ if (!err)
+ cb ? offload_action_hw_count_inc(action, in_hw_count) :
+ offload_action_hw_count_set(action, in_hw_count);
+
+ if (skip_sw && !tc_act_in_hw(action))
+ err = -EINVAL;
+
+ tc_cleanup_offload_action(&fl_action->action);
+
+fl_err:
+ kfree(fl_action);
+
+ return err;
+}
+
+/* offload the tc action after it is inserted */
+static int tcf_action_offload_add(struct tc_action *action,
+ struct netlink_ext_ack *extack)
+{
+ return tcf_action_offload_add_ex(action, extack, NULL, NULL);
+}
+
+int tcf_action_update_hw_stats(struct tc_action *action)
+{
+ struct flow_offload_action fl_act = {};
+ int err;
+
+ if (!tc_act_in_hw(action))
+ return -EOPNOTSUPP;
+
+ err = offload_action_init(&fl_act, action, FLOW_ACT_STATS, NULL);
+ if (err)
+ return err;
+
+ err = tcf_action_offload_cmd(&fl_act, NULL, NULL, NULL);
+ if (!err) {
+ preempt_disable();
+ tcf_action_stats_update(action, fl_act.stats.bytes,
+ fl_act.stats.pkts,
+ fl_act.stats.drops,
+ fl_act.stats.lastused,
+ true);
+ preempt_enable();
+ action->used_hw_stats = fl_act.stats.used_hw_stats;
+ action->used_hw_stats_valid = true;
+ } else {
+ return -EOPNOTSUPP;
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL(tcf_action_update_hw_stats);
+
+static int tcf_action_offload_del_ex(struct tc_action *action,
+ flow_indr_block_bind_cb_t *cb,
+ void *cb_priv)
+{
+ struct flow_offload_action fl_act = {};
+ u32 in_hw_count = 0;
+ int err = 0;
+
+ if (!tc_act_in_hw(action))
+ return 0;
+
+ err = offload_action_init(&fl_act, action, FLOW_ACT_DESTROY, NULL);
+ if (err)
+ return err;
+
+ err = tcf_action_offload_cmd(&fl_act, &in_hw_count, cb, cb_priv);
+ if (err < 0)
+ return err;
+
+ if (!cb && action->in_hw_count != in_hw_count)
+ return -EINVAL;
+
+ /* do not need to update hw state when deleting action */
+ if (cb && in_hw_count)
+ offload_action_hw_count_dec(action, in_hw_count);
+
+ return 0;
+}
+
+static int tcf_action_offload_del(struct tc_action *action)
+{
+ return tcf_action_offload_del_ex(action, NULL, NULL);
+}
+
static void tcf_action_cleanup(struct tc_action *p)
{
+ tcf_action_offload_del(p);
if (p->ops->cleanup)
p->ops->cleanup(p);
@@ -497,7 +735,7 @@ int tcf_idr_create(struct tc_action_net *tn, u32 index, struct nlattr *est,
p->tcfa_tm.install = jiffies;
p->tcfa_tm.lastuse = jiffies;
p->tcfa_tm.firstuse = 0;
- p->tcfa_flags = flags & TCA_ACT_FLAGS_USER_MASK;
+ p->tcfa_flags = flags;
if (est) {
err = gen_new_estimator(&p->tcfa_bstats, p->cpu_bstats,
&p->tcfa_rate_est,
@@ -622,6 +860,59 @@ EXPORT_SYMBOL(tcf_idrinfo_destroy);
static LIST_HEAD(act_base);
static DEFINE_RWLOCK(act_mod_lock);
+/* since act ops id is stored in pernet subsystem list,
+ * then there is no way to walk through only all the action
+ * subsystem, so we keep tc action pernet ops id for
+ * reoffload to walk through.
+ */
+static LIST_HEAD(act_pernet_id_list);
+static DEFINE_MUTEX(act_id_mutex);
+struct tc_act_pernet_id {
+ struct list_head list;
+ unsigned int id;
+};
+
+static int tcf_pernet_add_id_list(unsigned int id)
+{
+ struct tc_act_pernet_id *id_ptr;
+ int ret = 0;
+
+ mutex_lock(&act_id_mutex);
+ list_for_each_entry(id_ptr, &act_pernet_id_list, list) {
+ if (id_ptr->id == id) {
+ ret = -EEXIST;
+ goto err_out;
+ }
+ }
+
+ id_ptr = kzalloc(sizeof(*id_ptr), GFP_KERNEL);
+ if (!id_ptr) {
+ ret = -ENOMEM;
+ goto err_out;
+ }
+ id_ptr->id = id;
+
+ list_add_tail(&id_ptr->list, &act_pernet_id_list);
+
+err_out:
+ mutex_unlock(&act_id_mutex);
+ return ret;
+}
+
+static void tcf_pernet_del_id_list(unsigned int id)
+{
+ struct tc_act_pernet_id *id_ptr;
+
+ mutex_lock(&act_id_mutex);
+ list_for_each_entry(id_ptr, &act_pernet_id_list, list) {
+ if (id_ptr->id == id) {
+ list_del(&id_ptr->list);
+ kfree(id_ptr);
+ break;
+ }
+ }
+ mutex_unlock(&act_id_mutex);
+}
int tcf_register_action(struct tc_action_ops *act,
struct pernet_operations *ops)
@@ -640,18 +931,31 @@ int tcf_register_action(struct tc_action_ops *act,
if (ret)
return ret;
+ if (ops->id) {
+ ret = tcf_pernet_add_id_list(*ops->id);
+ if (ret)
+ goto err_id;
+ }
+
write_lock(&act_mod_lock);
list_for_each_entry(a, &act_base, head) {
if (act->id == a->id || (strcmp(act->kind, a->kind) == 0)) {
- write_unlock(&act_mod_lock);
- unregister_pernet_subsys(ops);
- return -EEXIST;
+ ret = -EEXIST;
+ goto err_out;
}
}
list_add_tail(&act->head, &act_base);
write_unlock(&act_mod_lock);
return 0;
+
+err_out:
+ write_unlock(&act_mod_lock);
+ if (ops->id)
+ tcf_pernet_del_id_list(*ops->id);
+err_id:
+ unregister_pernet_subsys(ops);
+ return ret;
}
EXPORT_SYMBOL(tcf_register_action);
@@ -670,8 +974,11 @@ int tcf_unregister_action(struct tc_action_ops *act,
}
}
write_unlock(&act_mod_lock);
- if (!err)
+ if (!err) {
unregister_pernet_subsys(ops);
+ if (ops->id)
+ tcf_pernet_del_id_list(*ops->id);
+ }
return err;
}
EXPORT_SYMBOL(tcf_unregister_action);
@@ -735,6 +1042,9 @@ int tcf_action_exec(struct sk_buff *skb, struct tc_action **actions,
jmp_prgcnt -= 1;
continue;
}
+
+ if (tc_act_skip_sw(a->tcfa_flags))
+ continue;
repeat:
ret = a->ops->act(skb, a, res);
if (ret == TC_ACT_REPEAT)
@@ -821,6 +1131,7 @@ tcf_action_dump_1(struct sk_buff *skb, struct tc_action *a, int bind, int ref)
int err = -EINVAL;
unsigned char *b = skb_tail_pointer(skb);
struct nlattr *nest;
+ u32 flags;
if (tcf_action_dump_terse(skb, a, false))
goto nla_put_failure;
@@ -835,9 +1146,13 @@ tcf_action_dump_1(struct sk_buff *skb, struct tc_action *a, int bind, int ref)
a->used_hw_stats, TCA_ACT_HW_STATS_ANY))
goto nla_put_failure;
- if (a->tcfa_flags &&
+ flags = a->tcfa_flags & TCA_ACT_FLAGS_USER_MASK;
+ if (flags &&
nla_put_bitfield32(skb, TCA_ACT_FLAGS,
- a->tcfa_flags, a->tcfa_flags))
+ flags, flags))
+ goto nla_put_failure;
+
+ if (nla_put_u32(skb, TCA_ACT_IN_HW_COUNT, a->in_hw_count))
goto nla_put_failure;
nest = nla_nest_start_noflag(skb, TCA_OPTIONS);
@@ -919,7 +1234,9 @@ static const struct nla_policy tcf_action_policy[TCA_ACT_MAX + 1] = {
[TCA_ACT_COOKIE] = { .type = NLA_BINARY,
.len = TC_COOKIE_MAX_SIZE },
[TCA_ACT_OPTIONS] = { .type = NLA_NESTED },
- [TCA_ACT_FLAGS] = NLA_POLICY_BITFIELD32(TCA_ACT_FLAGS_NO_PERCPU_STATS),
+ [TCA_ACT_FLAGS] = NLA_POLICY_BITFIELD32(TCA_ACT_FLAGS_NO_PERCPU_STATS |
+ TCA_ACT_FLAGS_SKIP_HW |
+ TCA_ACT_FLAGS_SKIP_SW),
[TCA_ACT_HW_STATS] = NLA_POLICY_BITFIELD32(TCA_ACT_HW_STATS_ANY),
};
@@ -1032,8 +1349,13 @@ struct tc_action *tcf_action_init_1(struct net *net, struct tcf_proto *tp,
}
}
hw_stats = tcf_action_hw_stats_get(tb[TCA_ACT_HW_STATS]);
- if (tb[TCA_ACT_FLAGS])
+ if (tb[TCA_ACT_FLAGS]) {
userflags = nla_get_bitfield32(tb[TCA_ACT_FLAGS]);
+ if (!tc_act_flags_valid(userflags.value)) {
+ err = -EINVAL;
+ goto err_out;
+ }
+ }
err = a_o->init(net, tb[TCA_ACT_OPTIONS], est, &a, tp,
userflags.value | flags, extack);
@@ -1061,11 +1383,17 @@ struct tc_action *tcf_action_init_1(struct net *net, struct tcf_proto *tp,
return ERR_PTR(err);
}
+static bool tc_act_bind(u32 flags)
+{
+ return !!(flags & TCA_ACT_FLAGS_BIND);
+}
+
/* Returns numbers of initialized actions or negative error. */
int tcf_action_init(struct net *net, struct tcf_proto *tp, struct nlattr *nla,
struct nlattr *est, struct tc_action *actions[],
- int init_res[], size_t *attr_size, u32 flags,
+ int init_res[], size_t *attr_size,
+ u32 flags, u32 fl_flags,
struct netlink_ext_ack *extack)
{
struct tc_action_ops *ops[TCA_ACT_MAX_PRIO] = {};
@@ -1103,6 +1431,22 @@ int tcf_action_init(struct net *net, struct tcf_proto *tp, struct nlattr *nla,
sz += tcf_action_fill_size(act);
/* Start from index 0 */
actions[i - 1] = act;
+ if (tc_act_bind(flags)) {
+ bool skip_sw = tc_skip_sw(fl_flags);
+ bool skip_hw = tc_skip_hw(fl_flags);
+
+ if (tc_act_bind(act->tcfa_flags))
+ continue;
+ if (skip_sw != tc_act_skip_sw(act->tcfa_flags) ||
+ skip_hw != tc_act_skip_hw(act->tcfa_flags)) {
+ err = -EINVAL;
+ goto err;
+ }
+ } else {
+ err = tcf_action_offload_add(act, extack);
+ if (tc_act_skip_sw(act->tcfa_flags) && err)
+ goto err;
+ }
}
/* We have to commit them all together, because if any error happened in
@@ -1154,6 +1498,9 @@ int tcf_action_copy_stats(struct sk_buff *skb, struct tc_action *p,
if (p == NULL)
goto errout;
+ /* update hw stats for this action */
+ tcf_action_update_hw_stats(p);
+
/* compat_mode being true specifies a call that is supposed
* to add additional backward compatibility statistic TLVs.
*/
@@ -1396,6 +1743,96 @@ static int tcf_action_delete(struct net *net, struct tc_action *actions[])
}
static int
+tcf_reoffload_del_notify(struct net *net, struct tc_action *action)
+{
+ size_t attr_size = tcf_action_fill_size(action);
+ struct tc_action *actions[TCA_ACT_MAX_PRIO] = {
+ [0] = action,
+ };
+ const struct tc_action_ops *ops = action->ops;
+ struct sk_buff *skb;
+ int ret;
+
+ skb = alloc_skb(attr_size <= NLMSG_GOODSIZE ? NLMSG_GOODSIZE : attr_size,
+ GFP_KERNEL);
+ if (!skb)
+ return -ENOBUFS;
+
+ if (tca_get_fill(skb, actions, 0, 0, 0, RTM_DELACTION, 0, 1) <= 0) {
+ kfree_skb(skb);
+ return -EINVAL;
+ }
+
+ ret = tcf_idr_release_unsafe(action);
+ if (ret == ACT_P_DELETED) {
+ module_put(ops->owner);
+ ret = rtnetlink_send(skb, net, 0, RTNLGRP_TC, 0);
+ } else {
+ kfree_skb(skb);
+ }
+
+ return ret;
+}
+
+int tcf_action_reoffload_cb(flow_indr_block_bind_cb_t *cb,
+ void *cb_priv, bool add)
+{
+ struct tc_act_pernet_id *id_ptr;
+ struct tcf_idrinfo *idrinfo;
+ struct tc_action_net *tn;
+ struct tc_action *p;
+ unsigned int act_id;
+ unsigned long tmp;
+ unsigned long id;
+ struct idr *idr;
+ struct net *net;
+ int ret;
+
+ if (!cb)
+ return -EINVAL;
+
+ down_read(&net_rwsem);
+ mutex_lock(&act_id_mutex);
+
+ for_each_net(net) {
+ list_for_each_entry(id_ptr, &act_pernet_id_list, list) {
+ act_id = id_ptr->id;
+ tn = net_generic(net, act_id);
+ if (!tn)
+ continue;
+ idrinfo = tn->idrinfo;
+ if (!idrinfo)
+ continue;
+
+ mutex_lock(&idrinfo->lock);
+ idr = &idrinfo->action_idr;
+ idr_for_each_entry_ul(idr, p, tmp, id) {
+ if (IS_ERR(p) || tc_act_bind(p->tcfa_flags))
+ continue;
+ if (add) {
+ tcf_action_offload_add_ex(p, NULL, cb,
+ cb_priv);
+ continue;
+ }
+
+ /* cb unregister to update hw count */
+ ret = tcf_action_offload_del_ex(p, cb, cb_priv);
+ if (ret < 0)
+ continue;
+ if (tc_act_skip_sw(p->tcfa_flags) &&
+ !tc_act_in_hw(p))
+ tcf_reoffload_del_notify(net, p);
+ }
+ mutex_unlock(&idrinfo->lock);
+ }
+ }
+ mutex_unlock(&act_id_mutex);
+ up_read(&net_rwsem);
+
+ return 0;
+}
+
+static int
tcf_del_notify(struct net *net, struct nlmsghdr *n, struct tc_action *actions[],
u32 portid, size_t attr_size, struct netlink_ext_ack *extack)
{
@@ -1508,7 +1945,7 @@ static int tcf_action_add(struct net *net, struct nlattr *nla,
for (loop = 0; loop < 10; loop++) {
ret = tcf_action_init(net, NULL, nla, NULL, actions, init_res,
- &attr_size, flags, extack);
+ &attr_size, flags, 0, extack);
if (ret != -EAGAIN)
break;
}
diff --git a/net/sched/act_bpf.c b/net/sched/act_bpf.c
index f2bf896..a77d890 100644
--- a/net/sched/act_bpf.c
+++ b/net/sched/act_bpf.c
@@ -305,7 +305,7 @@ static int tcf_bpf_init(struct net *net, struct nlattr *nla,
ret = tcf_idr_check_alloc(tn, &index, act, bind);
if (!ret) {
ret = tcf_idr_create(tn, index, est, act,
- &act_bpf_ops, bind, true, 0);
+ &act_bpf_ops, bind, true, flags);
if (ret < 0) {
tcf_idr_cleanup(tn, index);
return ret;
diff --git a/net/sched/act_connmark.c b/net/sched/act_connmark.c
index 94e78ac..09e2aaf 100644
--- a/net/sched/act_connmark.c
+++ b/net/sched/act_connmark.c
@@ -124,7 +124,7 @@ static int tcf_connmark_init(struct net *net, struct nlattr *nla,
ret = tcf_idr_check_alloc(tn, &index, a, bind);
if (!ret) {
ret = tcf_idr_create(tn, index, est, a,
- &act_connmark_ops, bind, false, 0);
+ &act_connmark_ops, bind, false, flags);
if (ret) {
tcf_idr_cleanup(tn, index);
return ret;
diff --git a/net/sched/act_csum.c b/net/sched/act_csum.c
index a15ec95..e0f515b 100644
--- a/net/sched/act_csum.c
+++ b/net/sched/act_csum.c
@@ -695,6 +695,24 @@ static size_t tcf_csum_get_fill_size(const struct tc_action *act)
return nla_total_size(sizeof(struct tc_csum));
}
+static int tcf_csum_offload_act_setup(struct tc_action *act, void *entry_data,
+ u32 *index_inc, bool bind)
+{
+ if (bind) {
+ struct flow_action_entry *entry = entry_data;
+
+ entry->id = FLOW_ACTION_CSUM;
+ entry->csum_flags = tcf_csum_update_flags(act);
+ *index_inc = 1;
+ } else {
+ struct flow_offload_action *fl_action = entry_data;
+
+ fl_action->id = FLOW_ACTION_CSUM;
+ }
+
+ return 0;
+}
+
static struct tc_action_ops act_csum_ops = {
.kind = "csum",
.id = TCA_ID_CSUM,
@@ -706,6 +724,7 @@ static struct tc_action_ops act_csum_ops = {
.walk = tcf_csum_walker,
.lookup = tcf_csum_search,
.get_fill_size = tcf_csum_get_fill_size,
+ .offload_act_setup = tcf_csum_offload_act_setup,
.size = sizeof(struct tcf_csum),
};
diff --git a/net/sched/act_ct.c b/net/sched/act_ct.c
index ab35914..f99247f 100644
--- a/net/sched/act_ct.c
+++ b/net/sched/act_ct.c
@@ -32,6 +32,7 @@
#include <net/netfilter/nf_conntrack_helper.h>
#include <net/netfilter/nf_conntrack_acct.h>
#include <net/netfilter/ipv6/nf_defrag_ipv6.h>
+#include <net/netfilter/nf_conntrack_act_ct.h>
#include <uapi/linux/netfilter/nf_nat.h>
static struct workqueue_struct *act_ct_wq;
@@ -56,6 +57,12 @@ static const struct rhashtable_params zones_params = {
.automatic_shrinking = true,
};
+static struct nf_ct_ext_type act_ct_extend __read_mostly = {
+ .len = sizeof(struct nf_conn_act_ct_ext),
+ .align = __alignof__(struct nf_conn_act_ct_ext),
+ .id = NF_CT_EXT_ACT_CT,
+};
+
static struct flow_action_entry *
tcf_ct_flow_table_flow_action_get_next(struct flow_action *flow_action)
{
@@ -358,6 +365,7 @@ static void tcf_ct_flow_table_add(struct tcf_ct_flow_table *ct_ft,
struct nf_conn *ct,
bool tcp)
{
+ struct nf_conn_act_ct_ext *act_ct_ext;
struct flow_offload *entry;
int err;
@@ -375,6 +383,14 @@ static void tcf_ct_flow_table_add(struct tcf_ct_flow_table *ct_ft,
ct->proto.tcp.seen[1].flags |= IP_CT_TCP_FLAG_BE_LIBERAL;
}
+ act_ct_ext = nf_conn_act_ct_ext_find(ct);
+ if (act_ct_ext) {
+ entry->tuplehash[FLOW_OFFLOAD_DIR_ORIGINAL].tuple.iifidx =
+ act_ct_ext->ifindex[IP_CT_DIR_ORIGINAL];
+ entry->tuplehash[FLOW_OFFLOAD_DIR_REPLY].tuple.iifidx =
+ act_ct_ext->ifindex[IP_CT_DIR_REPLY];
+ }
+
err = flow_offload_add(&ct_ft->nf_ft, entry);
if (err)
goto err_add;
@@ -393,7 +409,8 @@ static void tcf_ct_flow_table_process_conn(struct tcf_ct_flow_table *ct_ft,
{
bool tcp = false;
- if (ctinfo != IP_CT_ESTABLISHED && ctinfo != IP_CT_ESTABLISHED_REPLY)
+ if ((ctinfo != IP_CT_ESTABLISHED && ctinfo != IP_CT_ESTABLISHED_REPLY) ||
+ !test_bit(IPS_ASSURED_BIT, &ct->status))
return;
switch (nf_ct_protonum(ct)) {
@@ -597,7 +614,7 @@ static bool tcf_ct_skb_nfct_cached(struct net *net, struct sk_buff *skb,
if (nf_ct_is_confirmed(ct))
nf_ct_kill(ct);
- nf_conntrack_put(&ct->ct_general);
+ nf_ct_put(ct);
nf_ct_set(skb, NULL, IP_CT_UNTRACKED);
return false;
@@ -762,7 +779,7 @@ static void tcf_ct_params_free(struct rcu_head *head)
tcf_ct_flow_table_put(params);
if (params->tmpl)
- nf_conntrack_put(¶ms->tmpl->ct_general);
+ nf_ct_put(params->tmpl);
kfree(params);
}
@@ -839,6 +856,12 @@ static int ct_nat_execute(struct sk_buff *skb, struct nf_conn *ct,
}
err = nf_nat_packet(ct, ctinfo, hooknum, skb);
+ if (err == NF_ACCEPT) {
+ if (maniptype == NF_NAT_MANIP_SRC)
+ tc_skb_cb(skb)->post_ct_snat = 1;
+ if (maniptype == NF_NAT_MANIP_DST)
+ tc_skb_cb(skb)->post_ct_dnat = 1;
+ }
out:
return err;
}
@@ -966,7 +989,7 @@ static int tcf_ct_act(struct sk_buff *skb, const struct tc_action *a,
tc_skb_cb(skb)->post_ct = false;
ct = nf_ct_get(skb, &ctinfo);
if (ct) {
- nf_conntrack_put(&ct->ct_general);
+ nf_ct_put(ct);
nf_ct_set(skb, NULL, IP_CT_UNTRACKED);
}
@@ -1026,6 +1049,7 @@ static int tcf_ct_act(struct sk_buff *skb, const struct tc_action *a,
if (!ct)
goto out_push;
nf_ct_deliver_cached_events(ct);
+ nf_conn_act_ct_ext_fill(skb, ct, ctinfo);
err = tcf_ct_act_nat(skb, ct, ctinfo, p->ct_action, &p->range, commit);
if (err != NF_ACCEPT)
@@ -1035,6 +1059,9 @@ static int tcf_ct_act(struct sk_buff *skb, const struct tc_action *a,
tcf_ct_act_set_mark(ct, p->mark, p->mark_mask);
tcf_ct_act_set_labels(ct, p->labels, p->labels_mask);
+ if (!nf_ct_is_confirmed(ct))
+ nf_conn_act_ct_ext_add(ct);
+
/* This will take care of sending queued events
* even if the connection is already confirmed.
*/
@@ -1228,7 +1255,6 @@ static int tcf_ct_fill_params(struct net *net,
return -ENOMEM;
}
__set_bit(IPS_CONFIRMED_BIT, &tmpl->status);
- nf_conntrack_get(&tmpl->ct_general);
p->tmpl = tmpl;
return 0;
@@ -1493,6 +1519,26 @@ static void tcf_stats_update(struct tc_action *a, u64 bytes, u64 packets,
c->tcf_tm.lastuse = max_t(u64, c->tcf_tm.lastuse, lastuse);
}
+static int tcf_ct_offload_act_setup(struct tc_action *act, void *entry_data,
+ u32 *index_inc, bool bind)
+{
+ if (bind) {
+ struct flow_action_entry *entry = entry_data;
+
+ entry->id = FLOW_ACTION_CT;
+ entry->ct.action = tcf_ct_action(act);
+ entry->ct.zone = tcf_ct_zone(act);
+ entry->ct.flow_table = tcf_ct_ft(act);
+ *index_inc = 1;
+ } else {
+ struct flow_offload_action *fl_action = entry_data;
+
+ fl_action->id = FLOW_ACTION_CT;
+ }
+
+ return 0;
+}
+
static struct tc_action_ops act_ct_ops = {
.kind = "ct",
.id = TCA_ID_CT,
@@ -1504,6 +1550,7 @@ static struct tc_action_ops act_ct_ops = {
.walk = tcf_ct_walker,
.lookup = tcf_ct_search,
.stats_update = tcf_stats_update,
+ .offload_act_setup = tcf_ct_offload_act_setup,
.size = sizeof(struct tcf_ct),
};
@@ -1561,10 +1608,16 @@ static int __init ct_init_module(void)
if (err)
goto err_register;
+ err = nf_ct_extend_register(&act_ct_extend);
+ if (err)
+ goto err_register_extend;
+
static_branch_inc(&tcf_frag_xmit_count);
return 0;
+err_register_extend:
+ tcf_unregister_action(&act_ct_ops, &ct_net_ops);
err_register:
tcf_ct_flow_tables_uninit();
err_tbl_init:
@@ -1575,6 +1628,7 @@ static int __init ct_init_module(void)
static void __exit ct_cleanup_module(void)
{
static_branch_dec(&tcf_frag_xmit_count);
+ nf_ct_extend_unregister(&act_ct_extend);
tcf_unregister_action(&act_ct_ops, &ct_net_ops);
tcf_ct_flow_tables_uninit();
destroy_workqueue(act_ct_wq);
diff --git a/net/sched/act_ctinfo.c b/net/sched/act_ctinfo.c
index 549374a..0281e45 100644
--- a/net/sched/act_ctinfo.c
+++ b/net/sched/act_ctinfo.c
@@ -212,7 +212,7 @@ static int tcf_ctinfo_init(struct net *net, struct nlattr *nla,
err = tcf_idr_check_alloc(tn, &index, a, bind);
if (!err) {
ret = tcf_idr_create(tn, index, est, a,
- &act_ctinfo_ops, bind, false, 0);
+ &act_ctinfo_ops, bind, false, flags);
if (ret) {
tcf_idr_cleanup(tn, index);
return ret;
diff --git a/net/sched/act_gact.c b/net/sched/act_gact.c
index d8dce17..bde6a6c 100644
--- a/net/sched/act_gact.c
+++ b/net/sched/act_gact.c
@@ -252,6 +252,43 @@ static size_t tcf_gact_get_fill_size(const struct tc_action *act)
return sz;
}
+static int tcf_gact_offload_act_setup(struct tc_action *act, void *entry_data,
+ u32 *index_inc, bool bind)
+{
+ if (bind) {
+ struct flow_action_entry *entry = entry_data;
+
+ if (is_tcf_gact_ok(act)) {
+ entry->id = FLOW_ACTION_ACCEPT;
+ } else if (is_tcf_gact_shot(act)) {
+ entry->id = FLOW_ACTION_DROP;
+ } else if (is_tcf_gact_trap(act)) {
+ entry->id = FLOW_ACTION_TRAP;
+ } else if (is_tcf_gact_goto_chain(act)) {
+ entry->id = FLOW_ACTION_GOTO;
+ entry->chain_index = tcf_gact_goto_chain_index(act);
+ } else {
+ return -EOPNOTSUPP;
+ }
+ *index_inc = 1;
+ } else {
+ struct flow_offload_action *fl_action = entry_data;
+
+ if (is_tcf_gact_ok(act))
+ fl_action->id = FLOW_ACTION_ACCEPT;
+ else if (is_tcf_gact_shot(act))
+ fl_action->id = FLOW_ACTION_DROP;
+ else if (is_tcf_gact_trap(act))
+ fl_action->id = FLOW_ACTION_TRAP;
+ else if (is_tcf_gact_goto_chain(act))
+ fl_action->id = FLOW_ACTION_GOTO;
+ else
+ return -EOPNOTSUPP;
+ }
+
+ return 0;
+}
+
static struct tc_action_ops act_gact_ops = {
.kind = "gact",
.id = TCA_ID_GACT,
@@ -263,6 +300,7 @@ static struct tc_action_ops act_gact_ops = {
.walk = tcf_gact_walker,
.lookup = tcf_gact_search,
.get_fill_size = tcf_gact_get_fill_size,
+ .offload_act_setup = tcf_gact_offload_act_setup,
.size = sizeof(struct tcf_gact),
};
diff --git a/net/sched/act_gate.c b/net/sched/act_gate.c
index 7df72a41..d56e738 100644
--- a/net/sched/act_gate.c
+++ b/net/sched/act_gate.c
@@ -357,7 +357,7 @@ static int tcf_gate_init(struct net *net, struct nlattr *nla,
if (!err) {
ret = tcf_idr_create(tn, index, est, a,
- &act_gate_ops, bind, false, 0);
+ &act_gate_ops, bind, false, flags);
if (ret) {
tcf_idr_cleanup(tn, index);
return ret;
@@ -597,6 +597,54 @@ static size_t tcf_gate_get_fill_size(const struct tc_action *act)
return nla_total_size(sizeof(struct tc_gate));
}
+static void tcf_gate_entry_destructor(void *priv)
+{
+ struct action_gate_entry *oe = priv;
+
+ kfree(oe);
+}
+
+static int tcf_gate_get_entries(struct flow_action_entry *entry,
+ const struct tc_action *act)
+{
+ entry->gate.entries = tcf_gate_get_list(act);
+
+ if (!entry->gate.entries)
+ return -EINVAL;
+
+ entry->destructor = tcf_gate_entry_destructor;
+ entry->destructor_priv = entry->gate.entries;
+
+ return 0;
+}
+
+static int tcf_gate_offload_act_setup(struct tc_action *act, void *entry_data,
+ u32 *index_inc, bool bind)
+{
+ int err;
+
+ if (bind) {
+ struct flow_action_entry *entry = entry_data;
+
+ entry->id = FLOW_ACTION_GATE;
+ entry->gate.prio = tcf_gate_prio(act);
+ entry->gate.basetime = tcf_gate_basetime(act);
+ entry->gate.cycletime = tcf_gate_cycletime(act);
+ entry->gate.cycletimeext = tcf_gate_cycletimeext(act);
+ entry->gate.num_entries = tcf_gate_num_entries(act);
+ err = tcf_gate_get_entries(entry, act);
+ if (err)
+ return err;
+ *index_inc = 1;
+ } else {
+ struct flow_offload_action *fl_action = entry_data;
+
+ fl_action->id = FLOW_ACTION_GATE;
+ }
+
+ return 0;
+}
+
static struct tc_action_ops act_gate_ops = {
.kind = "gate",
.id = TCA_ID_GATE,
@@ -609,6 +657,7 @@ static struct tc_action_ops act_gate_ops = {
.stats_update = tcf_gate_stats_update,
.get_fill_size = tcf_gate_get_fill_size,
.lookup = tcf_gate_search,
+ .offload_act_setup = tcf_gate_offload_act_setup,
.size = sizeof(struct tcf_gate),
};
diff --git a/net/sched/act_ife.c b/net/sched/act_ife.c
index b757f90..41ba55e 100644
--- a/net/sched/act_ife.c
+++ b/net/sched/act_ife.c
@@ -553,7 +553,7 @@ static int tcf_ife_init(struct net *net, struct nlattr *nla,
if (!exists) {
ret = tcf_idr_create(tn, index, est, a, &act_ife_ops,
- bind, true, 0);
+ bind, true, flags);
if (ret) {
tcf_idr_cleanup(tn, index);
kfree(p);
diff --git a/net/sched/act_ipt.c b/net/sched/act_ipt.c
index 265b144..2f3d507 100644
--- a/net/sched/act_ipt.c
+++ b/net/sched/act_ipt.c
@@ -145,7 +145,7 @@ static int __tcf_ipt_init(struct net *net, unsigned int id, struct nlattr *nla,
if (!exists) {
ret = tcf_idr_create(tn, index, est, a, ops, bind,
- false, 0);
+ false, flags);
if (ret) {
tcf_idr_cleanup(tn, index);
return ret;
diff --git a/net/sched/act_mirred.c b/net/sched/act_mirred.c
index efc963a..39acd1d 100644
--- a/net/sched/act_mirred.c
+++ b/net/sched/act_mirred.c
@@ -79,7 +79,7 @@ static void tcf_mirred_release(struct tc_action *a)
/* last reference to action, no need to lock */
dev = rcu_dereference_protected(m->tcfm_dev, 1);
- dev_put(dev);
+ dev_put_track(dev, &m->tcfm_dev_tracker);
}
static const struct nla_policy mirred_policy[TCA_MIRRED_MAX + 1] = {
@@ -101,7 +101,6 @@ static int tcf_mirred_init(struct net *net, struct nlattr *nla,
bool mac_header_xmit = false;
struct tc_mirred *parm;
struct tcf_mirred *m;
- struct net_device *dev;
bool exists = false;
int ret, err;
u32 index;
@@ -171,16 +170,19 @@ static int tcf_mirred_init(struct net *net, struct nlattr *nla,
spin_lock_bh(&m->tcf_lock);
if (parm->ifindex) {
- dev = dev_get_by_index(net, parm->ifindex);
- if (!dev) {
+ struct net_device *odev, *ndev;
+
+ ndev = dev_get_by_index(net, parm->ifindex);
+ if (!ndev) {
spin_unlock_bh(&m->tcf_lock);
err = -ENODEV;
goto put_chain;
}
- mac_header_xmit = dev_is_mac_header_xmit(dev);
- dev = rcu_replace_pointer(m->tcfm_dev, dev,
+ mac_header_xmit = dev_is_mac_header_xmit(ndev);
+ odev = rcu_replace_pointer(m->tcfm_dev, ndev,
lockdep_is_held(&m->tcf_lock));
- dev_put(dev);
+ dev_put_track(odev, &m->tcfm_dev_tracker);
+ netdev_tracker_alloc(ndev, &m->tcfm_dev_tracker, GFP_ATOMIC);
m->tcfm_mac_header_xmit = mac_header_xmit;
}
goto_ch = tcf_action_set_ctrlact(*a, parm->action, goto_ch);
@@ -400,7 +402,7 @@ static int mirred_device_event(struct notifier_block *unused,
list_for_each_entry(m, &mirred_list, tcfm_list) {
spin_lock_bh(&m->tcf_lock);
if (tcf_mirred_dev_dereference(m) == dev) {
- dev_put(dev);
+ dev_put_track(dev, &m->tcfm_dev_tracker);
/* Note : no rcu grace period necessary, as
* net_device are already rcu protected.
*/
@@ -448,6 +450,55 @@ static size_t tcf_mirred_get_fill_size(const struct tc_action *act)
return nla_total_size(sizeof(struct tc_mirred));
}
+static void tcf_offload_mirred_get_dev(struct flow_action_entry *entry,
+ const struct tc_action *act)
+{
+ entry->dev = act->ops->get_dev(act, &entry->destructor);
+ if (!entry->dev)
+ return;
+ entry->destructor_priv = entry->dev;
+}
+
+static int tcf_mirred_offload_act_setup(struct tc_action *act, void *entry_data,
+ u32 *index_inc, bool bind)
+{
+ if (bind) {
+ struct flow_action_entry *entry = entry_data;
+
+ if (is_tcf_mirred_egress_redirect(act)) {
+ entry->id = FLOW_ACTION_REDIRECT;
+ tcf_offload_mirred_get_dev(entry, act);
+ } else if (is_tcf_mirred_egress_mirror(act)) {
+ entry->id = FLOW_ACTION_MIRRED;
+ tcf_offload_mirred_get_dev(entry, act);
+ } else if (is_tcf_mirred_ingress_redirect(act)) {
+ entry->id = FLOW_ACTION_REDIRECT_INGRESS;
+ tcf_offload_mirred_get_dev(entry, act);
+ } else if (is_tcf_mirred_ingress_mirror(act)) {
+ entry->id = FLOW_ACTION_MIRRED_INGRESS;
+ tcf_offload_mirred_get_dev(entry, act);
+ } else {
+ return -EOPNOTSUPP;
+ }
+ *index_inc = 1;
+ } else {
+ struct flow_offload_action *fl_action = entry_data;
+
+ if (is_tcf_mirred_egress_redirect(act))
+ fl_action->id = FLOW_ACTION_REDIRECT;
+ else if (is_tcf_mirred_egress_mirror(act))
+ fl_action->id = FLOW_ACTION_MIRRED;
+ else if (is_tcf_mirred_ingress_redirect(act))
+ fl_action->id = FLOW_ACTION_REDIRECT_INGRESS;
+ else if (is_tcf_mirred_ingress_mirror(act))
+ fl_action->id = FLOW_ACTION_MIRRED_INGRESS;
+ else
+ return -EOPNOTSUPP;
+ }
+
+ return 0;
+}
+
static struct tc_action_ops act_mirred_ops = {
.kind = "mirred",
.id = TCA_ID_MIRRED,
@@ -460,6 +511,7 @@ static struct tc_action_ops act_mirred_ops = {
.walk = tcf_mirred_walker,
.lookup = tcf_mirred_search,
.get_fill_size = tcf_mirred_get_fill_size,
+ .offload_act_setup = tcf_mirred_offload_act_setup,
.size = sizeof(struct tcf_mirred),
.get_dev = tcf_mirred_get_dev,
};
diff --git a/net/sched/act_mpls.c b/net/sched/act_mpls.c
index 8faa4c5..b9ff345 100644
--- a/net/sched/act_mpls.c
+++ b/net/sched/act_mpls.c
@@ -248,7 +248,7 @@ static int tcf_mpls_init(struct net *net, struct nlattr *nla,
if (!exists) {
ret = tcf_idr_create(tn, index, est, a,
- &act_mpls_ops, bind, true, 0);
+ &act_mpls_ops, bind, true, flags);
if (ret) {
tcf_idr_cleanup(tn, index);
return ret;
@@ -384,6 +384,57 @@ static int tcf_mpls_search(struct net *net, struct tc_action **a, u32 index)
return tcf_idr_search(tn, a, index);
}
+static int tcf_mpls_offload_act_setup(struct tc_action *act, void *entry_data,
+ u32 *index_inc, bool bind)
+{
+ if (bind) {
+ struct flow_action_entry *entry = entry_data;
+
+ switch (tcf_mpls_action(act)) {
+ case TCA_MPLS_ACT_PUSH:
+ entry->id = FLOW_ACTION_MPLS_PUSH;
+ entry->mpls_push.proto = tcf_mpls_proto(act);
+ entry->mpls_push.label = tcf_mpls_label(act);
+ entry->mpls_push.tc = tcf_mpls_tc(act);
+ entry->mpls_push.bos = tcf_mpls_bos(act);
+ entry->mpls_push.ttl = tcf_mpls_ttl(act);
+ break;
+ case TCA_MPLS_ACT_POP:
+ entry->id = FLOW_ACTION_MPLS_POP;
+ entry->mpls_pop.proto = tcf_mpls_proto(act);
+ break;
+ case TCA_MPLS_ACT_MODIFY:
+ entry->id = FLOW_ACTION_MPLS_MANGLE;
+ entry->mpls_mangle.label = tcf_mpls_label(act);
+ entry->mpls_mangle.tc = tcf_mpls_tc(act);
+ entry->mpls_mangle.bos = tcf_mpls_bos(act);
+ entry->mpls_mangle.ttl = tcf_mpls_ttl(act);
+ break;
+ default:
+ return -EOPNOTSUPP;
+ }
+ *index_inc = 1;
+ } else {
+ struct flow_offload_action *fl_action = entry_data;
+
+ switch (tcf_mpls_action(act)) {
+ case TCA_MPLS_ACT_PUSH:
+ fl_action->id = FLOW_ACTION_MPLS_PUSH;
+ break;
+ case TCA_MPLS_ACT_POP:
+ fl_action->id = FLOW_ACTION_MPLS_POP;
+ break;
+ case TCA_MPLS_ACT_MODIFY:
+ fl_action->id = FLOW_ACTION_MPLS_MANGLE;
+ break;
+ default:
+ return -EOPNOTSUPP;
+ }
+ }
+
+ return 0;
+}
+
static struct tc_action_ops act_mpls_ops = {
.kind = "mpls",
.id = TCA_ID_MPLS,
@@ -394,6 +445,7 @@ static struct tc_action_ops act_mpls_ops = {
.cleanup = tcf_mpls_cleanup,
.walk = tcf_mpls_walker,
.lookup = tcf_mpls_search,
+ .offload_act_setup = tcf_mpls_offload_act_setup,
.size = sizeof(struct tcf_mpls),
};
diff --git a/net/sched/act_nat.c b/net/sched/act_nat.c
index 7dd6b58..2a39b37 100644
--- a/net/sched/act_nat.c
+++ b/net/sched/act_nat.c
@@ -61,7 +61,7 @@ static int tcf_nat_init(struct net *net, struct nlattr *nla, struct nlattr *est,
err = tcf_idr_check_alloc(tn, &index, a, bind);
if (!err) {
ret = tcf_idr_create(tn, index, est, a,
- &act_nat_ops, bind, false, 0);
+ &act_nat_ops, bind, false, flags);
if (ret) {
tcf_idr_cleanup(tn, index);
return ret;
diff --git a/net/sched/act_pedit.c b/net/sched/act_pedit.c
index c6c862c..31fcd27 100644
--- a/net/sched/act_pedit.c
+++ b/net/sched/act_pedit.c
@@ -189,7 +189,7 @@ static int tcf_pedit_init(struct net *net, struct nlattr *nla,
err = tcf_idr_check_alloc(tn, &index, a, bind);
if (!err) {
ret = tcf_idr_create(tn, index, est, a,
- &act_pedit_ops, bind, false, 0);
+ &act_pedit_ops, bind, false, flags);
if (ret) {
tcf_idr_cleanup(tn, index);
goto out_free;
@@ -487,6 +487,39 @@ static int tcf_pedit_search(struct net *net, struct tc_action **a, u32 index)
return tcf_idr_search(tn, a, index);
}
+static int tcf_pedit_offload_act_setup(struct tc_action *act, void *entry_data,
+ u32 *index_inc, bool bind)
+{
+ if (bind) {
+ struct flow_action_entry *entry = entry_data;
+ int k;
+
+ for (k = 0; k < tcf_pedit_nkeys(act); k++) {
+ switch (tcf_pedit_cmd(act, k)) {
+ case TCA_PEDIT_KEY_EX_CMD_SET:
+ entry->id = FLOW_ACTION_MANGLE;
+ break;
+ case TCA_PEDIT_KEY_EX_CMD_ADD:
+ entry->id = FLOW_ACTION_ADD;
+ break;
+ default:
+ return -EOPNOTSUPP;
+ }
+ entry->mangle.htype = tcf_pedit_htype(act, k);
+ entry->mangle.mask = tcf_pedit_mask(act, k);
+ entry->mangle.val = tcf_pedit_val(act, k);
+ entry->mangle.offset = tcf_pedit_offset(act, k);
+ entry->hw_stats = tc_act_hw_stats(act->hw_stats);
+ entry++;
+ }
+ *index_inc = k;
+ } else {
+ return -EOPNOTSUPP;
+ }
+
+ return 0;
+}
+
static struct tc_action_ops act_pedit_ops = {
.kind = "pedit",
.id = TCA_ID_PEDIT,
@@ -498,6 +531,7 @@ static struct tc_action_ops act_pedit_ops = {
.init = tcf_pedit_init,
.walk = tcf_pedit_walker,
.lookup = tcf_pedit_search,
+ .offload_act_setup = tcf_pedit_offload_act_setup,
.size = sizeof(struct tcf_pedit),
};
diff --git a/net/sched/act_police.c b/net/sched/act_police.c
index 9e77ba8..0923aa2 100644
--- a/net/sched/act_police.c
+++ b/net/sched/act_police.c
@@ -90,7 +90,7 @@ static int tcf_police_init(struct net *net, struct nlattr *nla,
if (!exists) {
ret = tcf_idr_create(tn, index, NULL, a,
- &act_police_ops, bind, true, 0);
+ &act_police_ops, bind, true, flags);
if (ret) {
tcf_idr_cleanup(tn, index);
return ret;
@@ -405,6 +405,30 @@ static int tcf_police_search(struct net *net, struct tc_action **a, u32 index)
return tcf_idr_search(tn, a, index);
}
+static int tcf_police_offload_act_setup(struct tc_action *act, void *entry_data,
+ u32 *index_inc, bool bind)
+{
+ if (bind) {
+ struct flow_action_entry *entry = entry_data;
+
+ entry->id = FLOW_ACTION_POLICE;
+ entry->police.burst = tcf_police_burst(act);
+ entry->police.rate_bytes_ps =
+ tcf_police_rate_bytes_ps(act);
+ entry->police.burst_pkt = tcf_police_burst_pkt(act);
+ entry->police.rate_pkt_ps =
+ tcf_police_rate_pkt_ps(act);
+ entry->police.mtu = tcf_police_tcfp_mtu(act);
+ *index_inc = 1;
+ } else {
+ struct flow_offload_action *fl_action = entry_data;
+
+ fl_action->id = FLOW_ACTION_POLICE;
+ }
+
+ return 0;
+}
+
MODULE_AUTHOR("Alexey Kuznetsov");
MODULE_DESCRIPTION("Policing actions");
MODULE_LICENSE("GPL");
@@ -420,6 +444,7 @@ static struct tc_action_ops act_police_ops = {
.walk = tcf_police_walker,
.lookup = tcf_police_search,
.cleanup = tcf_police_cleanup,
+ .offload_act_setup = tcf_police_offload_act_setup,
.size = sizeof(struct tcf_police),
};
diff --git a/net/sched/act_sample.c b/net/sched/act_sample.c
index ce859b0e..9a22cdd 100644
--- a/net/sched/act_sample.c
+++ b/net/sched/act_sample.c
@@ -70,7 +70,7 @@ static int tcf_sample_init(struct net *net, struct nlattr *nla,
if (!exists) {
ret = tcf_idr_create(tn, index, est, a,
- &act_sample_ops, bind, true, 0);
+ &act_sample_ops, bind, true, flags);
if (ret) {
tcf_idr_cleanup(tn, index);
return ret;
@@ -282,6 +282,35 @@ tcf_sample_get_group(const struct tc_action *a,
return group;
}
+static void tcf_offload_sample_get_group(struct flow_action_entry *entry,
+ const struct tc_action *act)
+{
+ entry->sample.psample_group =
+ act->ops->get_psample_group(act, &entry->destructor);
+ entry->destructor_priv = entry->sample.psample_group;
+}
+
+static int tcf_sample_offload_act_setup(struct tc_action *act, void *entry_data,
+ u32 *index_inc, bool bind)
+{
+ if (bind) {
+ struct flow_action_entry *entry = entry_data;
+
+ entry->id = FLOW_ACTION_SAMPLE;
+ entry->sample.trunc_size = tcf_sample_trunc_size(act);
+ entry->sample.truncate = tcf_sample_truncate(act);
+ entry->sample.rate = tcf_sample_rate(act);
+ tcf_offload_sample_get_group(entry, act);
+ *index_inc = 1;
+ } else {
+ struct flow_offload_action *fl_action = entry_data;
+
+ fl_action->id = FLOW_ACTION_SAMPLE;
+ }
+
+ return 0;
+}
+
static struct tc_action_ops act_sample_ops = {
.kind = "sample",
.id = TCA_ID_SAMPLE,
@@ -294,6 +323,7 @@ static struct tc_action_ops act_sample_ops = {
.walk = tcf_sample_walker,
.lookup = tcf_sample_search,
.get_psample_group = tcf_sample_get_group,
+ .offload_act_setup = tcf_sample_offload_act_setup,
.size = sizeof(struct tcf_sample),
};
diff --git a/net/sched/act_simple.c b/net/sched/act_simple.c
index e617ab4..8c1d60b 100644
--- a/net/sched/act_simple.c
+++ b/net/sched/act_simple.c
@@ -129,7 +129,7 @@ static int tcf_simp_init(struct net *net, struct nlattr *nla,
if (!exists) {
ret = tcf_idr_create(tn, index, est, a,
- &act_simp_ops, bind, false, 0);
+ &act_simp_ops, bind, false, flags);
if (ret) {
tcf_idr_cleanup(tn, index);
return ret;
diff --git a/net/sched/act_skbedit.c b/net/sched/act_skbedit.c
index d30ecbf..ceba11b 100644
--- a/net/sched/act_skbedit.c
+++ b/net/sched/act_skbedit.c
@@ -176,7 +176,7 @@ static int tcf_skbedit_init(struct net *net, struct nlattr *nla,
if (!exists) {
ret = tcf_idr_create(tn, index, est, a,
- &act_skbedit_ops, bind, true, 0);
+ &act_skbedit_ops, bind, true, act_flags);
if (ret) {
tcf_idr_cleanup(tn, index);
return ret;
@@ -327,6 +327,41 @@ static size_t tcf_skbedit_get_fill_size(const struct tc_action *act)
+ nla_total_size_64bit(sizeof(u64)); /* TCA_SKBEDIT_FLAGS */
}
+static int tcf_skbedit_offload_act_setup(struct tc_action *act, void *entry_data,
+ u32 *index_inc, bool bind)
+{
+ if (bind) {
+ struct flow_action_entry *entry = entry_data;
+
+ if (is_tcf_skbedit_mark(act)) {
+ entry->id = FLOW_ACTION_MARK;
+ entry->mark = tcf_skbedit_mark(act);
+ } else if (is_tcf_skbedit_ptype(act)) {
+ entry->id = FLOW_ACTION_PTYPE;
+ entry->ptype = tcf_skbedit_ptype(act);
+ } else if (is_tcf_skbedit_priority(act)) {
+ entry->id = FLOW_ACTION_PRIORITY;
+ entry->priority = tcf_skbedit_priority(act);
+ } else {
+ return -EOPNOTSUPP;
+ }
+ *index_inc = 1;
+ } else {
+ struct flow_offload_action *fl_action = entry_data;
+
+ if (is_tcf_skbedit_mark(act))
+ fl_action->id = FLOW_ACTION_MARK;
+ else if (is_tcf_skbedit_ptype(act))
+ fl_action->id = FLOW_ACTION_PTYPE;
+ else if (is_tcf_skbedit_priority(act))
+ fl_action->id = FLOW_ACTION_PRIORITY;
+ else
+ return -EOPNOTSUPP;
+ }
+
+ return 0;
+}
+
static struct tc_action_ops act_skbedit_ops = {
.kind = "skbedit",
.id = TCA_ID_SKBEDIT,
@@ -339,6 +374,7 @@ static struct tc_action_ops act_skbedit_ops = {
.walk = tcf_skbedit_walker,
.get_fill_size = tcf_skbedit_get_fill_size,
.lookup = tcf_skbedit_search,
+ .offload_act_setup = tcf_skbedit_offload_act_setup,
.size = sizeof(struct tcf_skbedit),
};
diff --git a/net/sched/act_skbmod.c b/net/sched/act_skbmod.c
index 9b6b52c..2083612 100644
--- a/net/sched/act_skbmod.c
+++ b/net/sched/act_skbmod.c
@@ -168,7 +168,7 @@ static int tcf_skbmod_init(struct net *net, struct nlattr *nla,
if (!exists) {
ret = tcf_idr_create(tn, index, est, a,
- &act_skbmod_ops, bind, true, 0);
+ &act_skbmod_ops, bind, true, flags);
if (ret) {
tcf_idr_cleanup(tn, index);
return ret;
diff --git a/net/sched/act_tunnel_key.c b/net/sched/act_tunnel_key.c
index d9cd174..23aba03 100644
--- a/net/sched/act_tunnel_key.c
+++ b/net/sched/act_tunnel_key.c
@@ -787,6 +787,59 @@ static int tunnel_key_search(struct net *net, struct tc_action **a, u32 index)
return tcf_idr_search(tn, a, index);
}
+static void tcf_tunnel_encap_put_tunnel(void *priv)
+{
+ struct ip_tunnel_info *tunnel = priv;
+
+ kfree(tunnel);
+}
+
+static int tcf_tunnel_encap_get_tunnel(struct flow_action_entry *entry,
+ const struct tc_action *act)
+{
+ entry->tunnel = tcf_tunnel_info_copy(act);
+ if (!entry->tunnel)
+ return -ENOMEM;
+ entry->destructor = tcf_tunnel_encap_put_tunnel;
+ entry->destructor_priv = entry->tunnel;
+ return 0;
+}
+
+static int tcf_tunnel_key_offload_act_setup(struct tc_action *act,
+ void *entry_data,
+ u32 *index_inc,
+ bool bind)
+{
+ int err;
+
+ if (bind) {
+ struct flow_action_entry *entry = entry_data;
+
+ if (is_tcf_tunnel_set(act)) {
+ entry->id = FLOW_ACTION_TUNNEL_ENCAP;
+ err = tcf_tunnel_encap_get_tunnel(entry, act);
+ if (err)
+ return err;
+ } else if (is_tcf_tunnel_release(act)) {
+ entry->id = FLOW_ACTION_TUNNEL_DECAP;
+ } else {
+ return -EOPNOTSUPP;
+ }
+ *index_inc = 1;
+ } else {
+ struct flow_offload_action *fl_action = entry_data;
+
+ if (is_tcf_tunnel_set(act))
+ fl_action->id = FLOW_ACTION_TUNNEL_ENCAP;
+ else if (is_tcf_tunnel_release(act))
+ fl_action->id = FLOW_ACTION_TUNNEL_DECAP;
+ else
+ return -EOPNOTSUPP;
+ }
+
+ return 0;
+}
+
static struct tc_action_ops act_tunnel_key_ops = {
.kind = "tunnel_key",
.id = TCA_ID_TUNNEL_KEY,
@@ -797,6 +850,7 @@ static struct tc_action_ops act_tunnel_key_ops = {
.cleanup = tunnel_key_release,
.walk = tunnel_key_walker,
.lookup = tunnel_key_search,
+ .offload_act_setup = tcf_tunnel_key_offload_act_setup,
.size = sizeof(struct tcf_tunnel_key),
};
diff --git a/net/sched/act_vlan.c b/net/sched/act_vlan.c
index e4dc5a5..756e2dc 100644
--- a/net/sched/act_vlan.c
+++ b/net/sched/act_vlan.c
@@ -368,6 +368,53 @@ static size_t tcf_vlan_get_fill_size(const struct tc_action *act)
+ nla_total_size(sizeof(u8)); /* TCA_VLAN_PUSH_VLAN_PRIORITY */
}
+static int tcf_vlan_offload_act_setup(struct tc_action *act, void *entry_data,
+ u32 *index_inc, bool bind)
+{
+ if (bind) {
+ struct flow_action_entry *entry = entry_data;
+
+ switch (tcf_vlan_action(act)) {
+ case TCA_VLAN_ACT_PUSH:
+ entry->id = FLOW_ACTION_VLAN_PUSH;
+ entry->vlan.vid = tcf_vlan_push_vid(act);
+ entry->vlan.proto = tcf_vlan_push_proto(act);
+ entry->vlan.prio = tcf_vlan_push_prio(act);
+ break;
+ case TCA_VLAN_ACT_POP:
+ entry->id = FLOW_ACTION_VLAN_POP;
+ break;
+ case TCA_VLAN_ACT_MODIFY:
+ entry->id = FLOW_ACTION_VLAN_MANGLE;
+ entry->vlan.vid = tcf_vlan_push_vid(act);
+ entry->vlan.proto = tcf_vlan_push_proto(act);
+ entry->vlan.prio = tcf_vlan_push_prio(act);
+ break;
+ default:
+ return -EOPNOTSUPP;
+ }
+ *index_inc = 1;
+ } else {
+ struct flow_offload_action *fl_action = entry_data;
+
+ switch (tcf_vlan_action(act)) {
+ case TCA_VLAN_ACT_PUSH:
+ fl_action->id = FLOW_ACTION_VLAN_PUSH;
+ break;
+ case TCA_VLAN_ACT_POP:
+ fl_action->id = FLOW_ACTION_VLAN_POP;
+ break;
+ case TCA_VLAN_ACT_MODIFY:
+ fl_action->id = FLOW_ACTION_VLAN_MANGLE;
+ break;
+ default:
+ return -EOPNOTSUPP;
+ }
+ }
+
+ return 0;
+}
+
static struct tc_action_ops act_vlan_ops = {
.kind = "vlan",
.id = TCA_ID_VLAN,
@@ -380,6 +427,7 @@ static struct tc_action_ops act_vlan_ops = {
.stats_update = tcf_vlan_stats_update,
.get_fill_size = tcf_vlan_get_fill_size,
.lookup = tcf_vlan_search,
+ .offload_act_setup = tcf_vlan_offload_act_setup,
.size = sizeof(struct tcf_vlan),
};
diff --git a/net/sched/cls_api.c b/net/sched/cls_api.c
index 35c74bd..d4e27c6 100644
--- a/net/sched/cls_api.c
+++ b/net/sched/cls_api.c
@@ -1625,6 +1625,8 @@ int tcf_classify(struct sk_buff *skb,
ext->chain = last_executed_chain;
ext->mru = cb->mru;
ext->post_ct = cb->post_ct;
+ ext->post_ct_snat = cb->post_ct_snat;
+ ext->post_ct_dnat = cb->post_ct_dnat;
ext->zone = cb->zone;
}
@@ -3028,9 +3030,9 @@ void tcf_exts_destroy(struct tcf_exts *exts)
}
EXPORT_SYMBOL(tcf_exts_destroy);
-int tcf_exts_validate(struct net *net, struct tcf_proto *tp, struct nlattr **tb,
- struct nlattr *rate_tlv, struct tcf_exts *exts,
- u32 flags, struct netlink_ext_ack *extack)
+int tcf_exts_validate_ex(struct net *net, struct tcf_proto *tp, struct nlattr **tb,
+ struct nlattr *rate_tlv, struct tcf_exts *exts,
+ u32 flags, u32 fl_flags, struct netlink_ext_ack *extack)
{
#ifdef CONFIG_NET_CLS_ACT
{
@@ -3064,7 +3066,8 @@ int tcf_exts_validate(struct net *net, struct tcf_proto *tp, struct nlattr **tb,
flags |= TCA_ACT_FLAGS_BIND;
err = tcf_action_init(net, tp, tb[exts->action],
rate_tlv, exts->actions, init_res,
- &attr_size, flags, extack);
+ &attr_size, flags, fl_flags,
+ extack);
if (err < 0)
return err;
exts->nr_actions = err;
@@ -3080,6 +3083,15 @@ int tcf_exts_validate(struct net *net, struct tcf_proto *tp, struct nlattr **tb,
return 0;
}
+EXPORT_SYMBOL(tcf_exts_validate_ex);
+
+int tcf_exts_validate(struct net *net, struct tcf_proto *tp, struct nlattr **tb,
+ struct nlattr *rate_tlv, struct tcf_exts *exts,
+ u32 flags, struct netlink_ext_ack *extack)
+{
+ return tcf_exts_validate_ex(net, tp, tb, rate_tlv, exts,
+ flags, 0, extack);
+}
EXPORT_SYMBOL(tcf_exts_validate);
void tcf_exts_change(struct tcf_exts *dst, struct tcf_exts *src)
@@ -3323,7 +3335,7 @@ int tc_setup_cb_add(struct tcf_block *block, struct tcf_proto *tp,
up_read(&block->cb_lock);
if (take_rtnl)
rtnl_unlock();
- return ok_count < 0 ? ok_count : 0;
+ return min(ok_count, 0);
}
EXPORT_SYMBOL(tc_setup_cb_add);
@@ -3379,7 +3391,7 @@ int tc_setup_cb_replace(struct tcf_block *block, struct tcf_proto *tp,
up_read(&block->cb_lock);
if (take_rtnl)
rtnl_unlock();
- return ok_count < 0 ? ok_count : 0;
+ return min(ok_count, 0);
}
EXPORT_SYMBOL(tc_setup_cb_replace);
@@ -3417,7 +3429,7 @@ int tc_setup_cb_destroy(struct tcf_block *block, struct tcf_proto *tp,
up_read(&block->cb_lock);
if (take_rtnl)
rtnl_unlock();
- return ok_count < 0 ? ok_count : 0;
+ return min(ok_count, 0);
}
EXPORT_SYMBOL(tc_setup_cb_destroy);
@@ -3464,7 +3476,7 @@ static void tcf_act_put_cookie(struct flow_action_entry *entry)
flow_action_cookie_destroy(entry->cookie);
}
-void tc_cleanup_flow_action(struct flow_action *flow_action)
+void tc_cleanup_offload_action(struct flow_action *flow_action)
{
struct flow_action_entry *entry;
int i;
@@ -3475,93 +3487,37 @@ void tc_cleanup_flow_action(struct flow_action *flow_action)
entry->destructor(entry->destructor_priv);
}
}
-EXPORT_SYMBOL(tc_cleanup_flow_action);
+EXPORT_SYMBOL(tc_cleanup_offload_action);
-static void tcf_mirred_get_dev(struct flow_action_entry *entry,
- const struct tc_action *act)
+static int tc_setup_offload_act(struct tc_action *act,
+ struct flow_action_entry *entry,
+ u32 *index_inc)
{
#ifdef CONFIG_NET_CLS_ACT
- entry->dev = act->ops->get_dev(act, &entry->destructor);
- if (!entry->dev)
- return;
- entry->destructor_priv = entry->dev;
+ if (act->ops->offload_act_setup)
+ return act->ops->offload_act_setup(act, entry, index_inc, true);
+ else
+ return -EOPNOTSUPP;
+#else
+ return 0;
#endif
}
-static void tcf_tunnel_encap_put_tunnel(void *priv)
+int tc_setup_action(struct flow_action *flow_action,
+ struct tc_action *actions[])
{
- struct ip_tunnel_info *tunnel = priv;
-
- kfree(tunnel);
-}
-
-static int tcf_tunnel_encap_get_tunnel(struct flow_action_entry *entry,
- const struct tc_action *act)
-{
- entry->tunnel = tcf_tunnel_info_copy(act);
- if (!entry->tunnel)
- return -ENOMEM;
- entry->destructor = tcf_tunnel_encap_put_tunnel;
- entry->destructor_priv = entry->tunnel;
- return 0;
-}
-
-static void tcf_sample_get_group(struct flow_action_entry *entry,
- const struct tc_action *act)
-{
-#ifdef CONFIG_NET_CLS_ACT
- entry->sample.psample_group =
- act->ops->get_psample_group(act, &entry->destructor);
- entry->destructor_priv = entry->sample.psample_group;
-#endif
-}
-
-static void tcf_gate_entry_destructor(void *priv)
-{
- struct action_gate_entry *oe = priv;
-
- kfree(oe);
-}
-
-static int tcf_gate_get_entries(struct flow_action_entry *entry,
- const struct tc_action *act)
-{
- entry->gate.entries = tcf_gate_get_list(act);
-
- if (!entry->gate.entries)
- return -EINVAL;
-
- entry->destructor = tcf_gate_entry_destructor;
- entry->destructor_priv = entry->gate.entries;
-
- return 0;
-}
-
-static enum flow_action_hw_stats tc_act_hw_stats(u8 hw_stats)
-{
- if (WARN_ON_ONCE(hw_stats > TCA_ACT_HW_STATS_ANY))
- return FLOW_ACTION_HW_STATS_DONT_CARE;
- else if (!hw_stats)
- return FLOW_ACTION_HW_STATS_DISABLED;
-
- return hw_stats;
-}
-
-int tc_setup_flow_action(struct flow_action *flow_action,
- const struct tcf_exts *exts)
-{
+ int i, j, index, err = 0;
struct tc_action *act;
- int i, j, k, err = 0;
BUILD_BUG_ON(TCA_ACT_HW_STATS_ANY != FLOW_ACTION_HW_STATS_ANY);
BUILD_BUG_ON(TCA_ACT_HW_STATS_IMMEDIATE != FLOW_ACTION_HW_STATS_IMMEDIATE);
BUILD_BUG_ON(TCA_ACT_HW_STATS_DELAYED != FLOW_ACTION_HW_STATS_DELAYED);
- if (!exts)
+ if (!actions)
return 0;
j = 0;
- tcf_exts_for_each_action(i, act, exts) {
+ tcf_act_for_each_action(i, act, actions) {
struct flow_action_entry *entry;
entry = &flow_action->entries[j];
@@ -3571,165 +3527,39 @@ int tc_setup_flow_action(struct flow_action *flow_action,
goto err_out_locked;
entry->hw_stats = tc_act_hw_stats(act->hw_stats);
-
- if (is_tcf_gact_ok(act)) {
- entry->id = FLOW_ACTION_ACCEPT;
- } else if (is_tcf_gact_shot(act)) {
- entry->id = FLOW_ACTION_DROP;
- } else if (is_tcf_gact_trap(act)) {
- entry->id = FLOW_ACTION_TRAP;
- } else if (is_tcf_gact_goto_chain(act)) {
- entry->id = FLOW_ACTION_GOTO;
- entry->chain_index = tcf_gact_goto_chain_index(act);
- } else if (is_tcf_mirred_egress_redirect(act)) {
- entry->id = FLOW_ACTION_REDIRECT;
- tcf_mirred_get_dev(entry, act);
- } else if (is_tcf_mirred_egress_mirror(act)) {
- entry->id = FLOW_ACTION_MIRRED;
- tcf_mirred_get_dev(entry, act);
- } else if (is_tcf_mirred_ingress_redirect(act)) {
- entry->id = FLOW_ACTION_REDIRECT_INGRESS;
- tcf_mirred_get_dev(entry, act);
- } else if (is_tcf_mirred_ingress_mirror(act)) {
- entry->id = FLOW_ACTION_MIRRED_INGRESS;
- tcf_mirred_get_dev(entry, act);
- } else if (is_tcf_vlan(act)) {
- switch (tcf_vlan_action(act)) {
- case TCA_VLAN_ACT_PUSH:
- entry->id = FLOW_ACTION_VLAN_PUSH;
- entry->vlan.vid = tcf_vlan_push_vid(act);
- entry->vlan.proto = tcf_vlan_push_proto(act);
- entry->vlan.prio = tcf_vlan_push_prio(act);
- break;
- case TCA_VLAN_ACT_POP:
- entry->id = FLOW_ACTION_VLAN_POP;
- break;
- case TCA_VLAN_ACT_MODIFY:
- entry->id = FLOW_ACTION_VLAN_MANGLE;
- entry->vlan.vid = tcf_vlan_push_vid(act);
- entry->vlan.proto = tcf_vlan_push_proto(act);
- entry->vlan.prio = tcf_vlan_push_prio(act);
- break;
- default:
- err = -EOPNOTSUPP;
- goto err_out_locked;
- }
- } else if (is_tcf_tunnel_set(act)) {
- entry->id = FLOW_ACTION_TUNNEL_ENCAP;
- err = tcf_tunnel_encap_get_tunnel(entry, act);
- if (err)
- goto err_out_locked;
- } else if (is_tcf_tunnel_release(act)) {
- entry->id = FLOW_ACTION_TUNNEL_DECAP;
- } else if (is_tcf_pedit(act)) {
- for (k = 0; k < tcf_pedit_nkeys(act); k++) {
- switch (tcf_pedit_cmd(act, k)) {
- case TCA_PEDIT_KEY_EX_CMD_SET:
- entry->id = FLOW_ACTION_MANGLE;
- break;
- case TCA_PEDIT_KEY_EX_CMD_ADD:
- entry->id = FLOW_ACTION_ADD;
- break;
- default:
- err = -EOPNOTSUPP;
- goto err_out_locked;
- }
- entry->mangle.htype = tcf_pedit_htype(act, k);
- entry->mangle.mask = tcf_pedit_mask(act, k);
- entry->mangle.val = tcf_pedit_val(act, k);
- entry->mangle.offset = tcf_pedit_offset(act, k);
- entry->hw_stats = tc_act_hw_stats(act->hw_stats);
- entry = &flow_action->entries[++j];
- }
- } else if (is_tcf_csum(act)) {
- entry->id = FLOW_ACTION_CSUM;
- entry->csum_flags = tcf_csum_update_flags(act);
- } else if (is_tcf_skbedit_mark(act)) {
- entry->id = FLOW_ACTION_MARK;
- entry->mark = tcf_skbedit_mark(act);
- } else if (is_tcf_sample(act)) {
- entry->id = FLOW_ACTION_SAMPLE;
- entry->sample.trunc_size = tcf_sample_trunc_size(act);
- entry->sample.truncate = tcf_sample_truncate(act);
- entry->sample.rate = tcf_sample_rate(act);
- tcf_sample_get_group(entry, act);
- } else if (is_tcf_police(act)) {
- entry->id = FLOW_ACTION_POLICE;
- entry->police.burst = tcf_police_burst(act);
- entry->police.rate_bytes_ps =
- tcf_police_rate_bytes_ps(act);
- entry->police.burst_pkt = tcf_police_burst_pkt(act);
- entry->police.rate_pkt_ps =
- tcf_police_rate_pkt_ps(act);
- entry->police.mtu = tcf_police_tcfp_mtu(act);
- entry->police.index = act->tcfa_index;
- } else if (is_tcf_ct(act)) {
- entry->id = FLOW_ACTION_CT;
- entry->ct.action = tcf_ct_action(act);
- entry->ct.zone = tcf_ct_zone(act);
- entry->ct.flow_table = tcf_ct_ft(act);
- } else if (is_tcf_mpls(act)) {
- switch (tcf_mpls_action(act)) {
- case TCA_MPLS_ACT_PUSH:
- entry->id = FLOW_ACTION_MPLS_PUSH;
- entry->mpls_push.proto = tcf_mpls_proto(act);
- entry->mpls_push.label = tcf_mpls_label(act);
- entry->mpls_push.tc = tcf_mpls_tc(act);
- entry->mpls_push.bos = tcf_mpls_bos(act);
- entry->mpls_push.ttl = tcf_mpls_ttl(act);
- break;
- case TCA_MPLS_ACT_POP:
- entry->id = FLOW_ACTION_MPLS_POP;
- entry->mpls_pop.proto = tcf_mpls_proto(act);
- break;
- case TCA_MPLS_ACT_MODIFY:
- entry->id = FLOW_ACTION_MPLS_MANGLE;
- entry->mpls_mangle.label = tcf_mpls_label(act);
- entry->mpls_mangle.tc = tcf_mpls_tc(act);
- entry->mpls_mangle.bos = tcf_mpls_bos(act);
- entry->mpls_mangle.ttl = tcf_mpls_ttl(act);
- break;
- default:
- err = -EOPNOTSUPP;
- goto err_out_locked;
- }
- } else if (is_tcf_skbedit_ptype(act)) {
- entry->id = FLOW_ACTION_PTYPE;
- entry->ptype = tcf_skbedit_ptype(act);
- } else if (is_tcf_skbedit_priority(act)) {
- entry->id = FLOW_ACTION_PRIORITY;
- entry->priority = tcf_skbedit_priority(act);
- } else if (is_tcf_gate(act)) {
- entry->id = FLOW_ACTION_GATE;
- entry->gate.index = tcf_gate_index(act);
- entry->gate.prio = tcf_gate_prio(act);
- entry->gate.basetime = tcf_gate_basetime(act);
- entry->gate.cycletime = tcf_gate_cycletime(act);
- entry->gate.cycletimeext = tcf_gate_cycletimeext(act);
- entry->gate.num_entries = tcf_gate_num_entries(act);
- err = tcf_gate_get_entries(entry, act);
- if (err)
- goto err_out_locked;
- } else {
- err = -EOPNOTSUPP;
+ entry->hw_index = act->tcfa_index;
+ index = 0;
+ err = tc_setup_offload_act(act, entry, &index);
+ if (!err)
+ j += index;
+ else
goto err_out_locked;
- }
spin_unlock_bh(&act->tcfa_lock);
-
- if (!is_tcf_pedit(act))
- j++;
}
err_out:
if (err)
- tc_cleanup_flow_action(flow_action);
+ tc_cleanup_offload_action(flow_action);
return err;
err_out_locked:
spin_unlock_bh(&act->tcfa_lock);
goto err_out;
}
-EXPORT_SYMBOL(tc_setup_flow_action);
+
+int tc_setup_offload_action(struct flow_action *flow_action,
+ const struct tcf_exts *exts)
+{
+#ifdef CONFIG_NET_CLS_ACT
+ if (!exts)
+ return 0;
+
+ return tc_setup_action(flow_action, exts->actions);
+#else
+ return 0;
+#endif
+}
+EXPORT_SYMBOL(tc_setup_offload_action);
unsigned int tcf_exts_num_actions(struct tcf_exts *exts)
{
diff --git a/net/sched/cls_flower.c b/net/sched/cls_flower.c
index ef54ed3..1a9b1f1 100644
--- a/net/sched/cls_flower.c
+++ b/net/sched/cls_flower.c
@@ -463,7 +463,7 @@ static int fl_hw_replace_filter(struct tcf_proto *tp,
cls_flower.rule->match.key = &f->mkey;
cls_flower.classid = f->res.classid;
- err = tc_setup_flow_action(&cls_flower.rule->action, &f->exts);
+ err = tc_setup_offload_action(&cls_flower.rule->action, &f->exts);
if (err) {
kfree(cls_flower.rule);
if (skip_sw) {
@@ -475,7 +475,7 @@ static int fl_hw_replace_filter(struct tcf_proto *tp,
err = tc_setup_cb_add(block, tp, TC_SETUP_CLSFLOWER, &cls_flower,
skip_sw, &f->flags, &f->in_hw_count, rtnl_held);
- tc_cleanup_flow_action(&cls_flower.rule->action);
+ tc_cleanup_offload_action(&cls_flower.rule->action);
kfree(cls_flower.rule);
if (err) {
@@ -503,12 +503,12 @@ static void fl_hw_update_stats(struct tcf_proto *tp, struct cls_fl_filter *f,
tc_setup_cb_call(block, TC_SETUP_CLSFLOWER, &cls_flower, false,
rtnl_held);
- tcf_exts_stats_update(&f->exts, cls_flower.stats.bytes,
- cls_flower.stats.pkts,
- cls_flower.stats.drops,
- cls_flower.stats.lastused,
- cls_flower.stats.used_hw_stats,
- cls_flower.stats.used_hw_stats_valid);
+ tcf_exts_hw_stats_update(&f->exts, cls_flower.stats.bytes,
+ cls_flower.stats.pkts,
+ cls_flower.stats.drops,
+ cls_flower.stats.lastused,
+ cls_flower.stats.used_hw_stats,
+ cls_flower.stats.used_hw_stats_valid);
}
static void __fl_put(struct cls_fl_filter *f)
@@ -1919,12 +1919,14 @@ static int fl_set_parms(struct net *net, struct tcf_proto *tp,
struct cls_fl_filter *f, struct fl_flow_mask *mask,
unsigned long base, struct nlattr **tb,
struct nlattr *est,
- struct fl_flow_tmplt *tmplt, u32 flags,
+ struct fl_flow_tmplt *tmplt,
+ u32 flags, u32 fl_flags,
struct netlink_ext_ack *extack)
{
int err;
- err = tcf_exts_validate(net, tp, tb, est, &f->exts, flags, extack);
+ err = tcf_exts_validate_ex(net, tp, tb, est, &f->exts, flags,
+ fl_flags, extack);
if (err < 0)
return err;
@@ -2038,7 +2040,8 @@ static int fl_change(struct net *net, struct sk_buff *in_skb,
}
err = fl_set_parms(net, tp, fnew, mask, base, tb, tca[TCA_RATE],
- tp->chain->tmplt_priv, flags, extack);
+ tp->chain->tmplt_priv, flags, fnew->flags,
+ extack);
if (err)
goto errout;
@@ -2268,7 +2271,7 @@ static int fl_reoffload(struct tcf_proto *tp, bool add, flow_setup_cb_t *cb,
cls_flower.rule->match.mask = &f->mask->key;
cls_flower.rule->match.key = &f->mkey;
- err = tc_setup_flow_action(&cls_flower.rule->action, &f->exts);
+ err = tc_setup_offload_action(&cls_flower.rule->action, &f->exts);
if (err) {
kfree(cls_flower.rule);
if (tc_skip_sw(f->flags)) {
@@ -2285,7 +2288,7 @@ static int fl_reoffload(struct tcf_proto *tp, bool add, flow_setup_cb_t *cb,
TC_SETUP_CLSFLOWER, &cls_flower,
cb_priv, &f->flags,
&f->in_hw_count);
- tc_cleanup_flow_action(&cls_flower.rule->action);
+ tc_cleanup_offload_action(&cls_flower.rule->action);
kfree(cls_flower.rule);
if (err) {
diff --git a/net/sched/cls_matchall.c b/net/sched/cls_matchall.c
index 24f0046..ca5670f 100644
--- a/net/sched/cls_matchall.c
+++ b/net/sched/cls_matchall.c
@@ -97,7 +97,7 @@ static int mall_replace_hw_filter(struct tcf_proto *tp,
cls_mall.command = TC_CLSMATCHALL_REPLACE;
cls_mall.cookie = cookie;
- err = tc_setup_flow_action(&cls_mall.rule->action, &head->exts);
+ err = tc_setup_offload_action(&cls_mall.rule->action, &head->exts);
if (err) {
kfree(cls_mall.rule);
mall_destroy_hw_filter(tp, head, cookie, NULL);
@@ -111,7 +111,7 @@ static int mall_replace_hw_filter(struct tcf_proto *tp,
err = tc_setup_cb_add(block, tp, TC_SETUP_CLSMATCHALL, &cls_mall,
skip_sw, &head->flags, &head->in_hw_count, true);
- tc_cleanup_flow_action(&cls_mall.rule->action);
+ tc_cleanup_offload_action(&cls_mall.rule->action);
kfree(cls_mall.rule);
if (err) {
@@ -163,12 +163,13 @@ static const struct nla_policy mall_policy[TCA_MATCHALL_MAX + 1] = {
static int mall_set_parms(struct net *net, struct tcf_proto *tp,
struct cls_mall_head *head,
unsigned long base, struct nlattr **tb,
- struct nlattr *est, u32 flags,
+ struct nlattr *est, u32 flags, u32 fl_flags,
struct netlink_ext_ack *extack)
{
int err;
- err = tcf_exts_validate(net, tp, tb, est, &head->exts, flags, extack);
+ err = tcf_exts_validate_ex(net, tp, tb, est, &head->exts, flags,
+ fl_flags, extack);
if (err < 0)
return err;
@@ -226,8 +227,8 @@ static int mall_change(struct net *net, struct sk_buff *in_skb,
goto err_alloc_percpu;
}
- err = mall_set_parms(net, tp, new, base, tb, tca[TCA_RATE], flags,
- extack);
+ err = mall_set_parms(net, tp, new, base, tb, tca[TCA_RATE],
+ flags, new->flags, extack);
if (err)
goto err_set_parms;
@@ -301,7 +302,7 @@ static int mall_reoffload(struct tcf_proto *tp, bool add, flow_setup_cb_t *cb,
TC_CLSMATCHALL_REPLACE : TC_CLSMATCHALL_DESTROY;
cls_mall.cookie = (unsigned long)head;
- err = tc_setup_flow_action(&cls_mall.rule->action, &head->exts);
+ err = tc_setup_offload_action(&cls_mall.rule->action, &head->exts);
if (err) {
kfree(cls_mall.rule);
if (add && tc_skip_sw(head->flags)) {
@@ -314,7 +315,7 @@ static int mall_reoffload(struct tcf_proto *tp, bool add, flow_setup_cb_t *cb,
err = tc_setup_cb_reoffload(block, tp, add, cb, TC_SETUP_CLSMATCHALL,
&cls_mall, cb_priv, &head->flags,
&head->in_hw_count);
- tc_cleanup_flow_action(&cls_mall.rule->action);
+ tc_cleanup_offload_action(&cls_mall.rule->action);
kfree(cls_mall.rule);
if (err)
@@ -336,11 +337,11 @@ static void mall_stats_hw_filter(struct tcf_proto *tp,
tc_setup_cb_call(block, TC_SETUP_CLSMATCHALL, &cls_mall, false, true);
- tcf_exts_stats_update(&head->exts, cls_mall.stats.bytes,
- cls_mall.stats.pkts, cls_mall.stats.drops,
- cls_mall.stats.lastused,
- cls_mall.stats.used_hw_stats,
- cls_mall.stats.used_hw_stats_valid);
+ tcf_exts_hw_stats_update(&head->exts, cls_mall.stats.bytes,
+ cls_mall.stats.pkts, cls_mall.stats.drops,
+ cls_mall.stats.lastused,
+ cls_mall.stats.used_hw_stats,
+ cls_mall.stats.used_hw_stats_valid);
}
static int mall_dump(struct net *net, struct tcf_proto *tp, void *fh,
diff --git a/net/sched/cls_u32.c b/net/sched/cls_u32.c
index 4272814..cf56492 100644
--- a/net/sched/cls_u32.c
+++ b/net/sched/cls_u32.c
@@ -709,12 +709,13 @@ static const struct nla_policy u32_policy[TCA_U32_MAX + 1] = {
static int u32_set_parms(struct net *net, struct tcf_proto *tp,
unsigned long base,
struct tc_u_knode *n, struct nlattr **tb,
- struct nlattr *est, u32 flags,
+ struct nlattr *est, u32 flags, u32 fl_flags,
struct netlink_ext_ack *extack)
{
int err;
- err = tcf_exts_validate(net, tp, tb, est, &n->exts, flags, extack);
+ err = tcf_exts_validate_ex(net, tp, tb, est, &n->exts, flags,
+ fl_flags, extack);
if (err < 0)
return err;
@@ -895,7 +896,8 @@ static int u32_change(struct net *net, struct sk_buff *in_skb,
return -ENOMEM;
err = u32_set_parms(net, tp, base, new, tb,
- tca[TCA_RATE], flags, extack);
+ tca[TCA_RATE], flags, new->flags,
+ extack);
if (err) {
u32_destroy_key(new, false);
@@ -1060,8 +1062,8 @@ static int u32_change(struct net *net, struct sk_buff *in_skb,
}
#endif
- err = u32_set_parms(net, tp, base, n, tb, tca[TCA_RATE], flags,
- extack);
+ err = u32_set_parms(net, tp, base, n, tb, tca[TCA_RATE],
+ flags, n->flags, extack);
if (err == 0) {
struct tc_u_knode __rcu **ins;
struct tc_u_knode *pins;
diff --git a/net/sched/sch_api.c b/net/sched/sch_api.c
index efcd0b5..c9c6f49 100644
--- a/net/sched/sch_api.c
+++ b/net/sched/sch_api.c
@@ -1292,7 +1292,7 @@ static struct Qdisc *qdisc_create(struct net_device *dev,
if (ops->destroy)
ops->destroy(sch);
err_out3:
- dev_put(dev);
+ dev_put_track(dev, &sch->dev_tracker);
qdisc_free(sch);
err_out2:
module_put(ops->owner);
diff --git a/net/sched/sch_cake.c b/net/sched/sch_cake.c
index 857aaeb..a43a58a 100644
--- a/net/sched/sch_cake.c
+++ b/net/sched/sch_cake.c
@@ -2342,9 +2342,7 @@ static int cake_config_precedence(struct Qdisc *sch)
/* List of known Diffserv codepoints:
*
- * Least Effort (CS1, LE)
- * Best Effort (CS0)
- * Max Reliability & LLT "Lo" (TOS1)
+ * Default Forwarding (DF/CS0) - Best Effort
* Max Throughput (TOS2)
* Min Delay (TOS4)
* LLT "La" (TOS5)
@@ -2352,6 +2350,7 @@ static int cake_config_precedence(struct Qdisc *sch)
* Assured Forwarding 2 (AF2x) - x3
* Assured Forwarding 3 (AF3x) - x3
* Assured Forwarding 4 (AF4x) - x3
+ * Precedence Class 1 (CS1)
* Precedence Class 2 (CS2)
* Precedence Class 3 (CS3)
* Precedence Class 4 (CS4)
@@ -2360,8 +2359,9 @@ static int cake_config_precedence(struct Qdisc *sch)
* Precedence Class 7 (CS7)
* Voice Admit (VA)
* Expedited Forwarding (EF)
-
- * Total 25 codepoints.
+ * Lower Effort (LE)
+ *
+ * Total 26 codepoints.
*/
/* List of traffic classes in RFC 4594, updated by RFC 8622:
@@ -2375,12 +2375,12 @@ static int cake_config_precedence(struct Qdisc *sch)
* Realtime Interactive (CS4) - eg. games
* Multimedia Streaming (AF3x) - eg. YouTube, NetFlix, Twitch
* Broadcast Video (CS3)
- * Low Latency Data (AF2x,TOS4) - eg. database
- * Ops, Admin, Management (CS2,TOS1) - eg. ssh
- * Standard Service (CS0 & unrecognised codepoints)
- * High Throughput Data (AF1x,TOS2) - eg. web traffic
- * Low Priority Data (CS1,LE) - eg. BitTorrent
-
+ * Low-Latency Data (AF2x,TOS4) - eg. database
+ * Ops, Admin, Management (CS2) - eg. ssh
+ * Standard Service (DF & unrecognised codepoints)
+ * High-Throughput Data (AF1x,TOS2) - eg. web traffic
+ * Low-Priority Data (LE,CS1) - eg. BitTorrent
+ *
* Total 12 traffic classes.
*/
@@ -2390,12 +2390,12 @@ static int cake_config_diffserv8(struct Qdisc *sch)
*
* Network Control (CS6, CS7)
* Minimum Latency (EF, VA, CS5, CS4)
- * Interactive Shell (CS2, TOS1)
+ * Interactive Shell (CS2)
* Low Latency Transactions (AF2x, TOS4)
* Video Streaming (AF4x, AF3x, CS3)
- * Bog Standard (CS0 etc.)
- * High Throughput (AF1x, TOS2)
- * Background Traffic (CS1, LE)
+ * Bog Standard (DF etc.)
+ * High Throughput (AF1x, TOS2, CS1)
+ * Background Traffic (LE)
*
* Total 8 traffic classes.
*/
@@ -2437,9 +2437,9 @@ static int cake_config_diffserv4(struct Qdisc *sch)
/* Further pruned list of traffic classes for four-class system:
*
* Latency Sensitive (CS7, CS6, EF, VA, CS5, CS4)
- * Streaming Media (AF4x, AF3x, CS3, AF2x, TOS4, CS2, TOS1)
- * Best Effort (CS0, AF1x, TOS2, and those not specified)
- * Background Traffic (CS1, LE)
+ * Streaming Media (AF4x, AF3x, CS3, AF2x, TOS4, CS2)
+ * Best Effort (DF, AF1x, TOS2, and those not specified)
+ * Background Traffic (LE, CS1)
*
* Total 4 traffic classes.
*/
@@ -2477,9 +2477,9 @@ static int cake_config_diffserv4(struct Qdisc *sch)
static int cake_config_diffserv3(struct Qdisc *sch)
{
/* Simplified Diffserv structure with 3 tins.
- * Low Priority (CS1, LE)
+ * Latency Sensitive (CS7, CS6, EF, VA, TOS4)
* Best Effort
- * Latency Sensitive (TOS4, VA, EF, CS6, CS7)
+ * Low Priority (LE, CS1)
*/
struct cake_sched_data *q = qdisc_priv(sch);
u32 mtu = psched_mtu(qdisc_dev(sch));
diff --git a/net/sched/sch_frag.c b/net/sched/sch_frag.c
index 5ded4c8..a9bd0a2 100644
--- a/net/sched/sch_frag.c
+++ b/net/sched/sch_frag.c
@@ -1,4 +1,5 @@
// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+#include <linux/if_vlan.h>
#include <net/netlink.h>
#include <net/sch_generic.h>
#include <net/pkt_sched.h>
diff --git a/net/sched/sch_generic.c b/net/sched/sch_generic.c
index 3b0f620..b07bd1c 100644
--- a/net/sched/sch_generic.c
+++ b/net/sched/sch_generic.c
@@ -434,9 +434,9 @@ unsigned long dev_trans_start(struct net_device *dev)
dev = vlan_dev_real_dev(dev);
else if (netif_is_macvlan(dev))
dev = macvlan_dev_real_dev(dev);
- res = netdev_get_tx_queue(dev, 0)->trans_start;
+ res = READ_ONCE(netdev_get_tx_queue(dev, 0)->trans_start);
for (i = 1; i < dev->num_tx_queues; i++) {
- val = netdev_get_tx_queue(dev, i)->trans_start;
+ val = READ_ONCE(netdev_get_tx_queue(dev, i)->trans_start);
if (val && time_after(val, res))
res = val;
}
@@ -445,11 +445,63 @@ unsigned long dev_trans_start(struct net_device *dev)
}
EXPORT_SYMBOL(dev_trans_start);
+static void netif_freeze_queues(struct net_device *dev)
+{
+ unsigned int i;
+ int cpu;
+
+ cpu = smp_processor_id();
+ for (i = 0; i < dev->num_tx_queues; i++) {
+ struct netdev_queue *txq = netdev_get_tx_queue(dev, i);
+
+ /* We are the only thread of execution doing a
+ * freeze, but we have to grab the _xmit_lock in
+ * order to synchronize with threads which are in
+ * the ->hard_start_xmit() handler and already
+ * checked the frozen bit.
+ */
+ __netif_tx_lock(txq, cpu);
+ set_bit(__QUEUE_STATE_FROZEN, &txq->state);
+ __netif_tx_unlock(txq);
+ }
+}
+
+void netif_tx_lock(struct net_device *dev)
+{
+ spin_lock(&dev->tx_global_lock);
+ netif_freeze_queues(dev);
+}
+EXPORT_SYMBOL(netif_tx_lock);
+
+static void netif_unfreeze_queues(struct net_device *dev)
+{
+ unsigned int i;
+
+ for (i = 0; i < dev->num_tx_queues; i++) {
+ struct netdev_queue *txq = netdev_get_tx_queue(dev, i);
+
+ /* No need to grab the _xmit_lock here. If the
+ * queue is not stopped for another reason, we
+ * force a schedule.
+ */
+ clear_bit(__QUEUE_STATE_FROZEN, &txq->state);
+ netif_schedule_queue(txq);
+ }
+}
+
+void netif_tx_unlock(struct net_device *dev)
+{
+ netif_unfreeze_queues(dev);
+ spin_unlock(&dev->tx_global_lock);
+}
+EXPORT_SYMBOL(netif_tx_unlock);
+
static void dev_watchdog(struct timer_list *t)
{
struct net_device *dev = from_timer(dev, t, watchdog_timer);
+ bool release = true;
- netif_tx_lock(dev);
+ spin_lock(&dev->tx_global_lock);
if (!qdisc_tx_is_noop(dev)) {
if (netif_device_present(dev) &&
netif_running(dev) &&
@@ -462,31 +514,34 @@ static void dev_watchdog(struct timer_list *t)
struct netdev_queue *txq;
txq = netdev_get_tx_queue(dev, i);
- trans_start = txq->trans_start;
+ trans_start = READ_ONCE(txq->trans_start);
if (netif_xmit_stopped(txq) &&
time_after(jiffies, (trans_start +
dev->watchdog_timeo))) {
some_queue_timedout = 1;
- txq->trans_timeout++;
+ atomic_long_inc(&txq->trans_timeout);
break;
}
}
- if (some_queue_timedout) {
+ if (unlikely(some_queue_timedout)) {
trace_net_dev_xmit_timeout(dev, i);
WARN_ONCE(1, KERN_INFO "NETDEV WATCHDOG: %s (%s): transmit queue %u timed out\n",
dev->name, netdev_drivername(dev), i);
+ netif_freeze_queues(dev);
dev->netdev_ops->ndo_tx_timeout(dev, i);
+ netif_unfreeze_queues(dev);
}
if (!mod_timer(&dev->watchdog_timer,
round_jiffies(jiffies +
dev->watchdog_timeo)))
- dev_hold(dev);
+ release = false;
}
}
- netif_tx_unlock(dev);
+ spin_unlock(&dev->tx_global_lock);
- dev_put(dev);
+ if (release)
+ dev_put_track(dev, &dev->watchdog_dev_tracker);
}
void __netdev_watchdog_up(struct net_device *dev)
@@ -496,7 +551,7 @@ void __netdev_watchdog_up(struct net_device *dev)
dev->watchdog_timeo = 5*HZ;
if (!mod_timer(&dev->watchdog_timer,
round_jiffies(jiffies + dev->watchdog_timeo)))
- dev_hold(dev);
+ dev_hold_track(dev, &dev->watchdog_dev_tracker, GFP_ATOMIC);
}
}
EXPORT_SYMBOL_GPL(__netdev_watchdog_up);
@@ -510,7 +565,7 @@ static void dev_watchdog_down(struct net_device *dev)
{
netif_tx_lock_bh(dev);
if (del_timer(&dev->watchdog_timer))
- dev_put(dev);
+ dev_put_track(dev, &dev->watchdog_dev_tracker);
netif_tx_unlock_bh(dev);
}
@@ -920,7 +975,7 @@ struct Qdisc *qdisc_alloc(struct netdev_queue *dev_queue,
sch->enqueue = ops->enqueue;
sch->dequeue = ops->dequeue;
sch->dev_queue = dev_queue;
- dev_hold(dev);
+ dev_hold_track(dev, &sch->dev_tracker, GFP_KERNEL);
refcount_set(&sch->refcnt, 1);
return sch;
@@ -1020,7 +1075,7 @@ static void qdisc_destroy(struct Qdisc *qdisc)
ops->destroy(qdisc);
module_put(ops->owner);
- dev_put(qdisc_dev(qdisc));
+ dev_put_track(qdisc_dev(qdisc), &qdisc->dev_tracker);
trace_qdisc_destroy(qdisc);
@@ -1148,7 +1203,7 @@ static void transition_one_qdisc(struct net_device *dev,
rcu_assign_pointer(dev_queue->qdisc, new_qdisc);
if (need_watchdog_p) {
- dev_queue->trans_start = 0;
+ WRITE_ONCE(dev_queue->trans_start, 0);
*need_watchdog_p = 1;
}
}
diff --git a/net/sched/sch_netem.c b/net/sched/sch_netem.c
index ecbb10d..ed4ccef 100644
--- a/net/sched/sch_netem.c
+++ b/net/sched/sch_netem.c
@@ -208,17 +208,17 @@ static bool loss_4state(struct netem_sched_data *q)
* next state and if the next packet has to be transmitted or lost.
* The four states correspond to:
* TX_IN_GAP_PERIOD => successfully transmitted packets within a gap period
- * LOST_IN_BURST_PERIOD => isolated losses within a gap period
- * LOST_IN_GAP_PERIOD => lost packets within a burst period
- * TX_IN_GAP_PERIOD => successfully transmitted packets within a burst period
+ * LOST_IN_GAP_PERIOD => isolated losses within a gap period
+ * LOST_IN_BURST_PERIOD => lost packets within a burst period
+ * TX_IN_BURST_PERIOD => successfully transmitted packets within a burst period
*/
switch (clg->state) {
case TX_IN_GAP_PERIOD:
if (rnd < clg->a4) {
- clg->state = LOST_IN_BURST_PERIOD;
+ clg->state = LOST_IN_GAP_PERIOD;
return true;
} else if (clg->a4 < rnd && rnd < clg->a1 + clg->a4) {
- clg->state = LOST_IN_GAP_PERIOD;
+ clg->state = LOST_IN_BURST_PERIOD;
return true;
} else if (clg->a1 + clg->a4 < rnd) {
clg->state = TX_IN_GAP_PERIOD;
@@ -227,24 +227,24 @@ static bool loss_4state(struct netem_sched_data *q)
break;
case TX_IN_BURST_PERIOD:
if (rnd < clg->a5) {
- clg->state = LOST_IN_GAP_PERIOD;
+ clg->state = LOST_IN_BURST_PERIOD;
return true;
} else {
clg->state = TX_IN_BURST_PERIOD;
}
break;
- case LOST_IN_GAP_PERIOD:
+ case LOST_IN_BURST_PERIOD:
if (rnd < clg->a3)
clg->state = TX_IN_BURST_PERIOD;
else if (clg->a3 < rnd && rnd < clg->a2 + clg->a3) {
clg->state = TX_IN_GAP_PERIOD;
} else if (clg->a2 + clg->a3 < rnd) {
- clg->state = LOST_IN_GAP_PERIOD;
+ clg->state = LOST_IN_BURST_PERIOD;
return true;
}
break;
- case LOST_IN_BURST_PERIOD:
+ case LOST_IN_GAP_PERIOD:
clg->state = TX_IN_GAP_PERIOD;
break;
}
diff --git a/net/sched/sch_qfq.c b/net/sched/sch_qfq.c
index 0b7f9ba..d4ce58c 100644
--- a/net/sched/sch_qfq.c
+++ b/net/sched/sch_qfq.c
@@ -1421,10 +1421,8 @@ static int qfq_init_qdisc(struct Qdisc *sch, struct nlattr *opt,
if (err < 0)
return err;
- if (qdisc_dev(sch)->tx_queue_len + 1 > QFQ_MAX_AGG_CLASSES)
- max_classes = QFQ_MAX_AGG_CLASSES;
- else
- max_classes = qdisc_dev(sch)->tx_queue_len + 1;
+ max_classes = min_t(u64, (u64)qdisc_dev(sch)->tx_queue_len + 1,
+ QFQ_MAX_AGG_CLASSES);
/* max_cl_shift = floor(log_2(max_classes)) */
max_cl_shift = __fls(max_classes);
q->max_agg_classes = 1<<max_cl_shift;
diff --git a/net/sctp/diag.c b/net/sctp/diag.c
index 760b367..034e2c7 100644
--- a/net/sctp/diag.c
+++ b/net/sctp/diag.c
@@ -245,54 +245,49 @@ static size_t inet_assoc_attr_size(struct sctp_association *asoc)
+ 64;
}
-static int sctp_tsp_dump_one(struct sctp_transport *tsp, void *p)
+static int sctp_sock_dump_one(struct sctp_endpoint *ep, struct sctp_transport *tsp, void *p)
{
struct sctp_association *assoc = tsp->asoc;
- struct sock *sk = tsp->asoc->base.sk;
struct sctp_comm_param *commp = p;
- struct sk_buff *in_skb = commp->skb;
+ struct sock *sk = ep->base.sk;
const struct inet_diag_req_v2 *req = commp->r;
- const struct nlmsghdr *nlh = commp->nlh;
- struct net *net = sock_net(in_skb->sk);
+ struct sk_buff *skb = commp->skb;
struct sk_buff *rep;
int err;
err = sock_diag_check_cookie(sk, req->id.idiag_cookie);
if (err)
- goto out;
+ return err;
- err = -ENOMEM;
rep = nlmsg_new(inet_assoc_attr_size(assoc), GFP_KERNEL);
if (!rep)
- goto out;
+ return -ENOMEM;
lock_sock(sk);
- if (sk != assoc->base.sk) {
- release_sock(sk);
- sk = assoc->base.sk;
- lock_sock(sk);
- }
- err = inet_sctp_diag_fill(sk, assoc, rep, req,
- sk_user_ns(NETLINK_CB(in_skb).sk),
- NETLINK_CB(in_skb).portid,
- nlh->nlmsg_seq, 0, nlh,
- commp->net_admin);
- release_sock(sk);
- if (err < 0) {
- WARN_ON(err == -EMSGSIZE);
- kfree_skb(rep);
+ if (ep != assoc->ep) {
+ err = -EAGAIN;
goto out;
}
- err = nlmsg_unicast(net->diag_nlsk, rep, NETLINK_CB(in_skb).portid);
+ err = inet_sctp_diag_fill(sk, assoc, rep, req, sk_user_ns(NETLINK_CB(skb).sk),
+ NETLINK_CB(skb).portid, commp->nlh->nlmsg_seq, 0,
+ commp->nlh, commp->net_admin);
+ if (err < 0) {
+ WARN_ON(err == -EMSGSIZE);
+ goto out;
+ }
+ release_sock(sk);
+
+ return nlmsg_unicast(sock_net(skb->sk)->diag_nlsk, rep, NETLINK_CB(skb).portid);
out:
+ release_sock(sk);
+ kfree_skb(rep);
return err;
}
-static int sctp_sock_dump(struct sctp_transport *tsp, void *p)
+static int sctp_sock_dump(struct sctp_endpoint *ep, struct sctp_transport *tsp, void *p)
{
- struct sctp_endpoint *ep = tsp->asoc->ep;
struct sctp_comm_param *commp = p;
struct sock *sk = ep->base.sk;
struct sk_buff *skb = commp->skb;
@@ -302,6 +297,8 @@ static int sctp_sock_dump(struct sctp_transport *tsp, void *p)
int err = 0;
lock_sock(sk);
+ if (ep != tsp->asoc->ep)
+ goto release;
list_for_each_entry(assoc, &ep->asocs, asocs) {
if (cb->args[4] < cb->args[1])
goto next;
@@ -344,9 +341,8 @@ static int sctp_sock_dump(struct sctp_transport *tsp, void *p)
return err;
}
-static int sctp_sock_filter(struct sctp_transport *tsp, void *p)
+static int sctp_sock_filter(struct sctp_endpoint *ep, struct sctp_transport *tsp, void *p)
{
- struct sctp_endpoint *ep = tsp->asoc->ep;
struct sctp_comm_param *commp = p;
struct sock *sk = ep->base.sk;
const struct inet_diag_req_v2 *r = commp->r;
@@ -429,15 +425,15 @@ static void sctp_diag_get_info(struct sock *sk, struct inet_diag_msg *r,
static int sctp_diag_dump_one(struct netlink_callback *cb,
const struct inet_diag_req_v2 *req)
{
- struct sk_buff *in_skb = cb->skb;
- struct net *net = sock_net(in_skb->sk);
+ struct sk_buff *skb = cb->skb;
+ struct net *net = sock_net(skb->sk);
const struct nlmsghdr *nlh = cb->nlh;
union sctp_addr laddr, paddr;
struct sctp_comm_param commp = {
- .skb = in_skb,
+ .skb = skb,
.r = req,
.nlh = nlh,
- .net_admin = netlink_net_capable(in_skb, CAP_NET_ADMIN),
+ .net_admin = netlink_net_capable(skb, CAP_NET_ADMIN),
};
if (req->sdiag_family == AF_INET) {
@@ -460,7 +456,7 @@ static int sctp_diag_dump_one(struct netlink_callback *cb,
paddr.v6.sin6_family = AF_INET6;
}
- return sctp_transport_lookup_process(sctp_tsp_dump_one,
+ return sctp_transport_lookup_process(sctp_sock_dump_one,
net, &laddr, &paddr, &commp);
}
@@ -505,8 +501,8 @@ static void sctp_diag_dump(struct sk_buff *skb, struct netlink_callback *cb,
if (!(idiag_states & ~(TCPF_LISTEN | TCPF_CLOSE)))
goto done;
- sctp_for_each_transport(sctp_sock_filter, sctp_sock_dump,
- net, &pos, &commp);
+ sctp_transport_traverse_process(sctp_sock_filter, sctp_sock_dump,
+ net, &pos, &commp);
cb->args[2] = pos;
done:
diff --git a/net/sctp/endpointola.c b/net/sctp/endpointola.c
index 48c9c2c..efffde7 100644
--- a/net/sctp/endpointola.c
+++ b/net/sctp/endpointola.c
@@ -184,6 +184,18 @@ void sctp_endpoint_free(struct sctp_endpoint *ep)
}
/* Final destructor for endpoint. */
+static void sctp_endpoint_destroy_rcu(struct rcu_head *head)
+{
+ struct sctp_endpoint *ep = container_of(head, struct sctp_endpoint, rcu);
+ struct sock *sk = ep->base.sk;
+
+ sctp_sk(sk)->ep = NULL;
+ sock_put(sk);
+
+ kfree(ep);
+ SCTP_DBG_OBJCNT_DEC(ep);
+}
+
static void sctp_endpoint_destroy(struct sctp_endpoint *ep)
{
struct sock *sk;
@@ -213,18 +225,13 @@ static void sctp_endpoint_destroy(struct sctp_endpoint *ep)
if (sctp_sk(sk)->bind_hash)
sctp_put_port(sk);
- sctp_sk(sk)->ep = NULL;
- /* Give up our hold on the sock */
- sock_put(sk);
-
- kfree(ep);
- SCTP_DBG_OBJCNT_DEC(ep);
+ call_rcu(&ep->rcu, sctp_endpoint_destroy_rcu);
}
/* Hold a reference to an endpoint. */
-void sctp_endpoint_hold(struct sctp_endpoint *ep)
+int sctp_endpoint_hold(struct sctp_endpoint *ep)
{
- refcount_inc(&ep->base.refcnt);
+ return refcount_inc_not_zero(&ep->base.refcnt);
}
/* Release a reference to an endpoint and clean up if there are
diff --git a/net/sctp/input.c b/net/sctp/input.c
index 1f17860..90e12ba 100644
--- a/net/sctp/input.c
+++ b/net/sctp/input.c
@@ -746,23 +746,21 @@ static int __sctp_hash_endpoint(struct sctp_endpoint *ep)
struct sock *sk = ep->base.sk;
struct net *net = sock_net(sk);
struct sctp_hashbucket *head;
- struct sctp_ep_common *epb;
- epb = &ep->base;
- epb->hashent = sctp_ep_hashfn(net, epb->bind_addr.port);
- head = &sctp_ep_hashtable[epb->hashent];
+ ep->hashent = sctp_ep_hashfn(net, ep->base.bind_addr.port);
+ head = &sctp_ep_hashtable[ep->hashent];
if (sk->sk_reuseport) {
bool any = sctp_is_ep_boundall(sk);
- struct sctp_ep_common *epb2;
+ struct sctp_endpoint *ep2;
struct list_head *list;
int cnt = 0, err = 1;
list_for_each(list, &ep->base.bind_addr.address_list)
cnt++;
- sctp_for_each_hentry(epb2, &head->chain) {
- struct sock *sk2 = epb2->sk;
+ sctp_for_each_hentry(ep2, &head->chain) {
+ struct sock *sk2 = ep2->base.sk;
if (!net_eq(sock_net(sk2), net) || sk2 == sk ||
!uid_eq(sock_i_uid(sk2), sock_i_uid(sk)) ||
@@ -789,7 +787,7 @@ static int __sctp_hash_endpoint(struct sctp_endpoint *ep)
}
write_lock(&head->lock);
- hlist_add_head(&epb->node, &head->chain);
+ hlist_add_head(&ep->node, &head->chain);
write_unlock(&head->lock);
return 0;
}
@@ -811,19 +809,16 @@ static void __sctp_unhash_endpoint(struct sctp_endpoint *ep)
{
struct sock *sk = ep->base.sk;
struct sctp_hashbucket *head;
- struct sctp_ep_common *epb;
- epb = &ep->base;
+ ep->hashent = sctp_ep_hashfn(sock_net(sk), ep->base.bind_addr.port);
- epb->hashent = sctp_ep_hashfn(sock_net(sk), epb->bind_addr.port);
-
- head = &sctp_ep_hashtable[epb->hashent];
+ head = &sctp_ep_hashtable[ep->hashent];
if (rcu_access_pointer(sk->sk_reuseport_cb))
reuseport_detach_sock(sk);
write_lock(&head->lock);
- hlist_del_init(&epb->node);
+ hlist_del_init(&ep->node);
write_unlock(&head->lock);
}
@@ -856,7 +851,6 @@ static struct sctp_endpoint *__sctp_rcv_lookup_endpoint(
const union sctp_addr *paddr)
{
struct sctp_hashbucket *head;
- struct sctp_ep_common *epb;
struct sctp_endpoint *ep;
struct sock *sk;
__be16 lport;
@@ -866,8 +860,7 @@ static struct sctp_endpoint *__sctp_rcv_lookup_endpoint(
hash = sctp_ep_hashfn(net, ntohs(lport));
head = &sctp_ep_hashtable[hash];
read_lock(&head->lock);
- sctp_for_each_hentry(epb, &head->chain) {
- ep = sctp_ep(epb);
+ sctp_for_each_hentry(ep, &head->chain) {
if (sctp_endpoint_is_match(ep, net, laddr))
goto hit;
}
diff --git a/net/sctp/output.c b/net/sctp/output.c
index cdfdbd3..72fe666 100644
--- a/net/sctp/output.c
+++ b/net/sctp/output.c
@@ -134,7 +134,7 @@ void sctp_packet_config(struct sctp_packet *packet, __u32 vtag,
dst_hold(tp->dst);
sk_setup_caps(sk, tp->dst);
}
- packet->max_size = sk_can_gso(sk) ? tp->dst->dev->gso_max_size
+ packet->max_size = sk_can_gso(sk) ? READ_ONCE(tp->dst->dev->gso_max_size)
: asoc->pathmtu;
rcu_read_unlock();
}
diff --git a/net/sctp/outqueue.c b/net/sctp/outqueue.c
index ff47091..a18609f 100644
--- a/net/sctp/outqueue.c
+++ b/net/sctp/outqueue.c
@@ -547,6 +547,9 @@ void sctp_retransmit(struct sctp_outq *q, struct sctp_transport *transport,
sctp_assoc_update_retran_path(transport->asoc);
transport->asoc->rtx_data_chunks +=
transport->asoc->unack_data;
+ if (transport->pl.state == SCTP_PL_COMPLETE &&
+ transport->asoc->unack_data)
+ sctp_transport_reset_probe_timer(transport);
break;
case SCTP_RTXR_FAST_RTX:
SCTP_INC_STATS(net, SCTP_MIB_FAST_RETRANSMITS);
diff --git a/net/sctp/proc.c b/net/sctp/proc.c
index 982a87b..f13d6a3 100644
--- a/net/sctp/proc.c
+++ b/net/sctp/proc.c
@@ -161,7 +161,6 @@ static void *sctp_eps_seq_next(struct seq_file *seq, void *v, loff_t *pos)
static int sctp_eps_seq_show(struct seq_file *seq, void *v)
{
struct sctp_hashbucket *head;
- struct sctp_ep_common *epb;
struct sctp_endpoint *ep;
struct sock *sk;
int hash = *(loff_t *)v;
@@ -171,18 +170,17 @@ static int sctp_eps_seq_show(struct seq_file *seq, void *v)
head = &sctp_ep_hashtable[hash];
read_lock_bh(&head->lock);
- sctp_for_each_hentry(epb, &head->chain) {
- ep = sctp_ep(epb);
- sk = epb->sk;
+ sctp_for_each_hentry(ep, &head->chain) {
+ sk = ep->base.sk;
if (!net_eq(sock_net(sk), seq_file_net(seq)))
continue;
seq_printf(seq, "%8pK %8pK %-3d %-3d %-4d %-5d %5u %5lu ", ep, sk,
sctp_sk(sk)->type, sk->sk_state, hash,
- epb->bind_addr.port,
+ ep->base.bind_addr.port,
from_kuid_munged(seq_user_ns(seq), sock_i_uid(sk)),
sock_i_ino(sk));
- sctp_seq_dump_local_addrs(seq, epb);
+ sctp_seq_dump_local_addrs(seq, &ep->base);
seq_printf(seq, "\n");
}
read_unlock_bh(&head->lock);
diff --git a/net/sctp/sm_statefuns.c b/net/sctp/sm_statefuns.c
index 354c1c4d..cc544a9 100644
--- a/net/sctp/sm_statefuns.c
+++ b/net/sctp/sm_statefuns.c
@@ -1124,12 +1124,11 @@ enum sctp_disposition sctp_sf_send_probe(struct net *net,
if (!sctp_transport_pl_enabled(transport))
return SCTP_DISPOSITION_CONSUME;
- if (sctp_transport_pl_send(transport)) {
- reply = sctp_make_heartbeat(asoc, transport, transport->pl.probe_size);
- if (!reply)
- return SCTP_DISPOSITION_NOMEM;
- sctp_add_cmd_sf(commands, SCTP_CMD_REPLY, SCTP_CHUNK(reply));
- }
+ sctp_transport_pl_send(transport);
+ reply = sctp_make_heartbeat(asoc, transport, transport->pl.probe_size);
+ if (!reply)
+ return SCTP_DISPOSITION_NOMEM;
+ sctp_add_cmd_sf(commands, SCTP_CMD_REPLY, SCTP_CHUNK(reply));
sctp_add_cmd_sf(commands, SCTP_CMD_PROBE_TIMER_UPDATE,
SCTP_TRANSPORT(transport));
diff --git a/net/sctp/socket.c b/net/sctp/socket.c
index 3339125..3e1a960 100644
--- a/net/sctp/socket.c
+++ b/net/sctp/socket.c
@@ -5068,12 +5068,9 @@ static int sctp_init_sock(struct sock *sk)
SCTP_DBG_OBJCNT_INC(sock);
- local_bh_disable();
sk_sockets_allocated_inc(sk);
sock_prot_inuse_add(net, sk->sk_prot, 1);
- local_bh_enable();
-
return 0;
}
@@ -5099,10 +5096,8 @@ static void sctp_destroy_sock(struct sock *sk)
list_del(&sp->auto_asconf_list);
}
sctp_endpoint_free(sp->ep);
- local_bh_disable();
sk_sockets_allocated_dec(sk);
sock_prot_inuse_add(sock_net(sk), sk->sk_prot, -1);
- local_bh_enable();
}
/* Triggered when there are no references on the socket anymore */
@@ -5299,14 +5294,14 @@ int sctp_for_each_endpoint(int (*cb)(struct sctp_endpoint *, void *),
void *p) {
int err = 0;
int hash = 0;
- struct sctp_ep_common *epb;
+ struct sctp_endpoint *ep;
struct sctp_hashbucket *head;
for (head = sctp_ep_hashtable; hash < sctp_ep_hashsize;
hash++, head++) {
read_lock_bh(&head->lock);
- sctp_for_each_hentry(epb, &head->chain) {
- err = cb(sctp_ep(epb), p);
+ sctp_for_each_hentry(ep, &head->chain) {
+ err = cb(ep, p);
if (err)
break;
}
@@ -5317,32 +5312,41 @@ int sctp_for_each_endpoint(int (*cb)(struct sctp_endpoint *, void *),
}
EXPORT_SYMBOL_GPL(sctp_for_each_endpoint);
-int sctp_transport_lookup_process(int (*cb)(struct sctp_transport *, void *),
- struct net *net,
+int sctp_transport_lookup_process(sctp_callback_t cb, struct net *net,
const union sctp_addr *laddr,
const union sctp_addr *paddr, void *p)
{
struct sctp_transport *transport;
- int err;
+ struct sctp_endpoint *ep;
+ int err = -ENOENT;
rcu_read_lock();
transport = sctp_addrs_lookup_transport(net, laddr, paddr);
+ if (!transport) {
+ rcu_read_unlock();
+ return err;
+ }
+ ep = transport->asoc->ep;
+ if (!sctp_endpoint_hold(ep)) { /* asoc can be peeled off */
+ sctp_transport_put(transport);
+ rcu_read_unlock();
+ return err;
+ }
rcu_read_unlock();
- if (!transport)
- return -ENOENT;
- err = cb(transport, p);
+ err = cb(ep, transport, p);
+ sctp_endpoint_put(ep);
sctp_transport_put(transport);
-
return err;
}
EXPORT_SYMBOL_GPL(sctp_transport_lookup_process);
-int sctp_for_each_transport(int (*cb)(struct sctp_transport *, void *),
- int (*cb_done)(struct sctp_transport *, void *),
- struct net *net, int *pos, void *p) {
+int sctp_transport_traverse_process(sctp_callback_t cb, sctp_callback_t cb_done,
+ struct net *net, int *pos, void *p)
+{
struct rhashtable_iter hti;
struct sctp_transport *tsp;
+ struct sctp_endpoint *ep;
int ret;
again:
@@ -5351,26 +5355,32 @@ int sctp_for_each_transport(int (*cb)(struct sctp_transport *, void *),
tsp = sctp_transport_get_idx(net, &hti, *pos + 1);
for (; !IS_ERR_OR_NULL(tsp); tsp = sctp_transport_get_next(net, &hti)) {
- ret = cb(tsp, p);
- if (ret)
- break;
+ ep = tsp->asoc->ep;
+ if (sctp_endpoint_hold(ep)) { /* asoc can be peeled off */
+ ret = cb(ep, tsp, p);
+ if (ret)
+ break;
+ sctp_endpoint_put(ep);
+ }
(*pos)++;
sctp_transport_put(tsp);
}
sctp_transport_walk_stop(&hti);
if (ret) {
- if (cb_done && !cb_done(tsp, p)) {
+ if (cb_done && !cb_done(ep, tsp, p)) {
(*pos)++;
+ sctp_endpoint_put(ep);
sctp_transport_put(tsp);
goto again;
}
+ sctp_endpoint_put(ep);
sctp_transport_put(tsp);
}
return ret;
}
-EXPORT_SYMBOL_GPL(sctp_for_each_transport);
+EXPORT_SYMBOL_GPL(sctp_transport_traverse_process);
/* 7.2.1 Association Status (SCTP_STATUS)
diff --git a/net/sctp/transport.c b/net/sctp/transport.c
index 133f171..f8fd987 100644
--- a/net/sctp/transport.c
+++ b/net/sctp/transport.c
@@ -213,13 +213,18 @@ void sctp_transport_reset_reconf_timer(struct sctp_transport *transport)
void sctp_transport_reset_probe_timer(struct sctp_transport *transport)
{
- if (timer_pending(&transport->probe_timer))
- return;
if (!mod_timer(&transport->probe_timer,
jiffies + transport->probe_interval))
sctp_transport_hold(transport);
}
+void sctp_transport_reset_raise_timer(struct sctp_transport *transport)
+{
+ if (!mod_timer(&transport->probe_timer,
+ jiffies + transport->probe_interval * 30))
+ sctp_transport_hold(transport);
+}
+
/* This transport has been assigned to an association.
* Initialize fields from the association or from the sock itself.
* Register the reference count in the association.
@@ -258,12 +263,11 @@ void sctp_transport_pmtu(struct sctp_transport *transport, struct sock *sk)
sctp_transport_pl_update(transport);
}
-bool sctp_transport_pl_send(struct sctp_transport *t)
+void sctp_transport_pl_send(struct sctp_transport *t)
{
if (t->pl.probe_count < SCTP_MAX_PROBES)
goto out;
- t->pl.last_rtx_chunks = t->asoc->rtx_data_chunks;
t->pl.probe_count = 0;
if (t->pl.state == SCTP_PL_BASE) {
if (t->pl.probe_size == SCTP_BASE_PLPMTU) { /* BASE_PLPMTU Confirmation Failed */
@@ -298,17 +302,9 @@ bool sctp_transport_pl_send(struct sctp_transport *t)
}
out:
- if (t->pl.state == SCTP_PL_COMPLETE && t->pl.raise_count < 30 &&
- !t->pl.probe_count && t->pl.last_rtx_chunks == t->asoc->rtx_data_chunks) {
- t->pl.raise_count++;
- return false;
- }
-
pr_debug("%s: PLPMTUD: transport: %p, state: %d, pmtu: %d, size: %d, high: %d\n",
__func__, t, t->pl.state, t->pl.pmtu, t->pl.probe_size, t->pl.probe_high);
-
t->pl.probe_count++;
- return true;
}
bool sctp_transport_pl_recv(struct sctp_transport *t)
@@ -316,7 +312,6 @@ bool sctp_transport_pl_recv(struct sctp_transport *t)
pr_debug("%s: PLPMTUD: transport: %p, state: %d, pmtu: %d, size: %d, high: %d\n",
__func__, t, t->pl.state, t->pl.pmtu, t->pl.probe_size, t->pl.probe_high);
- t->pl.last_rtx_chunks = t->asoc->rtx_data_chunks;
t->pl.pmtu = t->pl.probe_size;
t->pl.probe_count = 0;
if (t->pl.state == SCTP_PL_BASE) {
@@ -338,14 +333,14 @@ bool sctp_transport_pl_recv(struct sctp_transport *t)
t->pl.probe_size += SCTP_PL_MIN_STEP;
if (t->pl.probe_size >= t->pl.probe_high) {
t->pl.probe_high = 0;
- t->pl.raise_count = 0;
t->pl.state = SCTP_PL_COMPLETE; /* Search -> Search Complete */
t->pl.probe_size = t->pl.pmtu;
t->pathmtu = t->pl.pmtu + sctp_transport_pl_hlen(t);
sctp_assoc_sync_pmtu(t->asoc);
+ sctp_transport_reset_raise_timer(t);
}
- } else if (t->pl.state == SCTP_PL_COMPLETE && t->pl.raise_count == 30) {
+ } else if (t->pl.state == SCTP_PL_COMPLETE) {
/* Raise probe_size again after 30 * interval in Search Complete */
t->pl.state = SCTP_PL_SEARCH; /* Search Complete -> Search */
t->pl.probe_size += SCTP_PL_MIN_STEP;
@@ -393,6 +388,7 @@ static bool sctp_transport_pl_toobig(struct sctp_transport *t, u32 pmtu)
t->pl.probe_high = 0;
t->pl.pmtu = SCTP_BASE_PLPMTU;
t->pathmtu = t->pl.pmtu + sctp_transport_pl_hlen(t);
+ sctp_transport_reset_probe_timer(t);
return true;
}
}
diff --git a/net/smc/af_smc.c b/net/smc/af_smc.c
index 1c9289f..aa3bcaa 100644
--- a/net/smc/af_smc.c
+++ b/net/smc/af_smc.c
@@ -89,8 +89,8 @@ int smc_hash_sk(struct sock *sk)
write_lock_bh(&h->lock);
sk_add_node(sk, head);
- sock_prot_inuse_add(sock_net(sk), sk->sk_prot, 1);
write_unlock_bh(&h->lock);
+ sock_prot_inuse_add(sock_net(sk), sk->sk_prot, 1);
return 0;
}
@@ -632,10 +632,12 @@ static int smc_connect_decline_fallback(struct smc_sock *smc, int reason_code,
static void smc_conn_abort(struct smc_sock *smc, int local_first)
{
+ struct smc_connection *conn = &smc->conn;
+ struct smc_link_group *lgr = conn->lgr;
+
+ smc_conn_free(conn);
if (local_first)
- smc_lgr_cleanup_early(&smc->conn);
- else
- smc_conn_free(&smc->conn);
+ smc_lgr_cleanup_early(lgr);
}
/* check if there is a rdma device available for this connection. */
@@ -2700,8 +2702,8 @@ static const struct proto_ops smc_sock_ops = {
.splice_read = smc_splice_read,
};
-static int smc_create(struct net *net, struct socket *sock, int protocol,
- int kern)
+static int __smc_create(struct net *net, struct socket *sock, int protocol,
+ int kern, struct socket *clcsock)
{
int family = (protocol == SMCPROTO_SMC6) ? PF_INET6 : PF_INET;
struct smc_sock *smc;
@@ -2726,12 +2728,19 @@ static int smc_create(struct net *net, struct socket *sock, int protocol,
smc = smc_sk(sk);
smc->use_fallback = false; /* assume rdma capability first */
smc->fallback_rsn = 0;
- rc = sock_create_kern(net, family, SOCK_STREAM, IPPROTO_TCP,
- &smc->clcsock);
- if (rc) {
- sk_common_release(sk);
- goto out;
+
+ rc = 0;
+ if (!clcsock) {
+ rc = sock_create_kern(net, family, SOCK_STREAM, IPPROTO_TCP,
+ &smc->clcsock);
+ if (rc) {
+ sk_common_release(sk);
+ goto out;
+ }
+ } else {
+ smc->clcsock = clcsock;
}
+
smc->sk.sk_sndbuf = max(smc->clcsock->sk->sk_sndbuf, SMC_BUF_MIN_SIZE);
smc->sk.sk_rcvbuf = max(smc->clcsock->sk->sk_rcvbuf, SMC_BUF_MIN_SIZE);
@@ -2739,12 +2748,76 @@ static int smc_create(struct net *net, struct socket *sock, int protocol,
return rc;
}
+static int smc_create(struct net *net, struct socket *sock, int protocol,
+ int kern)
+{
+ return __smc_create(net, sock, protocol, kern, NULL);
+}
+
static const struct net_proto_family smc_sock_family_ops = {
.family = PF_SMC,
.owner = THIS_MODULE,
.create = smc_create,
};
+static int smc_ulp_init(struct sock *sk)
+{
+ struct socket *tcp = sk->sk_socket;
+ struct net *net = sock_net(sk);
+ struct socket *smcsock;
+ int protocol, ret;
+
+ /* only TCP can be replaced */
+ if (tcp->type != SOCK_STREAM || sk->sk_protocol != IPPROTO_TCP ||
+ (sk->sk_family != AF_INET && sk->sk_family != AF_INET6))
+ return -ESOCKTNOSUPPORT;
+ /* don't handle wq now */
+ if (tcp->state != SS_UNCONNECTED || !tcp->file || tcp->wq.fasync_list)
+ return -ENOTCONN;
+
+ if (sk->sk_family == AF_INET)
+ protocol = SMCPROTO_SMC;
+ else
+ protocol = SMCPROTO_SMC6;
+
+ smcsock = sock_alloc();
+ if (!smcsock)
+ return -ENFILE;
+
+ smcsock->type = SOCK_STREAM;
+ __module_get(THIS_MODULE); /* tried in __tcp_ulp_find_autoload */
+ ret = __smc_create(net, smcsock, protocol, 1, tcp);
+ if (ret) {
+ sock_release(smcsock); /* module_put() which ops won't be NULL */
+ return ret;
+ }
+
+ /* replace tcp socket to smc */
+ smcsock->file = tcp->file;
+ smcsock->file->private_data = smcsock;
+ smcsock->file->f_inode = SOCK_INODE(smcsock); /* replace inode when sock_close */
+ smcsock->file->f_path.dentry->d_inode = SOCK_INODE(smcsock); /* dput() in __fput */
+ tcp->file = NULL;
+
+ return ret;
+}
+
+static void smc_ulp_clone(const struct request_sock *req, struct sock *newsk,
+ const gfp_t priority)
+{
+ struct inet_connection_sock *icsk = inet_csk(newsk);
+
+ /* don't inherit ulp ops to child when listen */
+ icsk->icsk_ulp_ops = NULL;
+}
+
+static struct tcp_ulp_ops smc_ulp_ops __read_mostly = {
+ .name = "smc",
+ .owner = THIS_MODULE,
+ .init = smc_ulp_init,
+ .clone = smc_ulp_clone,
+};
+
unsigned int smc_net_id;
static __net_init int smc_net_init(struct net *net)
@@ -2855,6 +2928,12 @@ static int __init smc_init(void)
goto out_sock;
}
+ rc = tcp_register_ulp(&smc_ulp_ops);
+ if (rc) {
+ pr_err("%s: tcp_ulp_register fails with %d\n", __func__, rc);
+ goto out_sock;
+ }
+
static_branch_enable(&tcp_have_smc);
return 0;
@@ -2883,6 +2962,7 @@ static int __init smc_init(void)
static void __exit smc_exit(void)
{
static_branch_disable(&tcp_have_smc);
+ tcp_unregister_ulp(&smc_ulp_ops);
sock_unregister(PF_SMC);
smc_core_exit();
smc_ib_unregister_client();
@@ -2905,3 +2985,4 @@ MODULE_AUTHOR("Ursula Braun <ubraun@linux.vnet.ibm.com>");
MODULE_DESCRIPTION("smc socket address family");
MODULE_LICENSE("GPL");
MODULE_ALIAS_NETPROTO(PF_SMC);
+MODULE_ALIAS_TCP_ULP("smc");
diff --git a/net/smc/smc.h b/net/smc/smc.h
index f4286ca..1a4fc1c 100644
--- a/net/smc/smc.h
+++ b/net/smc/smc.h
@@ -180,6 +180,11 @@ struct smc_connection {
u16 tx_cdc_seq; /* sequence # for CDC send */
u16 tx_cdc_seq_fin; /* sequence # - tx completed */
spinlock_t send_lock; /* protect wr_sends */
+ atomic_t cdc_pend_tx_wr; /* number of pending tx CDC wqe
+ * - inc when post wqe,
+ * - dec on polled tx cqe
+ */
+ wait_queue_head_t cdc_pend_tx_wq; /* wakeup on no cdc_pend_tx_wr*/
struct delayed_work tx_work; /* retry of smc_cdc_msg_send */
u32 tx_off; /* base offset in peer rmb */
diff --git a/net/smc/smc_cdc.c b/net/smc/smc_cdc.c
index 99acd33..84c8a43 100644
--- a/net/smc/smc_cdc.c
+++ b/net/smc/smc_cdc.c
@@ -31,10 +31,6 @@ static void smc_cdc_tx_handler(struct smc_wr_tx_pend_priv *pnd_snd,
struct smc_sock *smc;
int diff;
- if (!conn)
- /* already dismissed */
- return;
-
smc = container_of(conn, struct smc_sock, conn);
bh_lock_sock(&smc->sk);
if (!wc_status) {
@@ -51,6 +47,12 @@ static void smc_cdc_tx_handler(struct smc_wr_tx_pend_priv *pnd_snd,
conn);
conn->tx_cdc_seq_fin = cdcpend->ctrl_seq;
}
+
+ if (atomic_dec_and_test(&conn->cdc_pend_tx_wr) &&
+ unlikely(wq_has_sleeper(&conn->cdc_pend_tx_wq)))
+ wake_up(&conn->cdc_pend_tx_wq);
+ WARN_ON(atomic_read(&conn->cdc_pend_tx_wr) < 0);
+
smc_tx_sndbuf_nonfull(smc);
bh_unlock_sock(&smc->sk);
}
@@ -107,6 +109,10 @@ int smc_cdc_msg_send(struct smc_connection *conn,
conn->tx_cdc_seq++;
conn->local_tx_ctrl.seqno = conn->tx_cdc_seq;
smc_host_msg_to_cdc((struct smc_cdc_msg *)wr_buf, conn, &cfed);
+
+ atomic_inc(&conn->cdc_pend_tx_wr);
+ smp_mb__after_atomic(); /* Make sure cdc_pend_tx_wr added before post */
+
rc = smc_wr_tx_send(link, (struct smc_wr_tx_pend_priv *)pend);
if (!rc) {
smc_curs_copy(&conn->rx_curs_confirmed, &cfed, conn);
@@ -114,6 +120,7 @@ int smc_cdc_msg_send(struct smc_connection *conn,
} else {
conn->tx_cdc_seq--;
conn->local_tx_ctrl.seqno = conn->tx_cdc_seq;
+ atomic_dec(&conn->cdc_pend_tx_wr);
}
return rc;
@@ -136,7 +143,18 @@ int smcr_cdc_msg_send_validation(struct smc_connection *conn,
peer->token = htonl(local->token);
peer->prod_flags.failover_validation = 1;
+ /* We need to set pend->conn here to make sure smc_cdc_tx_handler()
+ * can handle properly
+ */
+ smc_cdc_add_pending_send(conn, pend);
+
+ atomic_inc(&conn->cdc_pend_tx_wr);
+ smp_mb__after_atomic(); /* Make sure cdc_pend_tx_wr added before post */
+
rc = smc_wr_tx_send(link, (struct smc_wr_tx_pend_priv *)pend);
+ if (unlikely(rc))
+ atomic_dec(&conn->cdc_pend_tx_wr);
+
return rc;
}
@@ -193,31 +211,9 @@ int smc_cdc_get_slot_and_msg_send(struct smc_connection *conn)
return rc;
}
-static bool smc_cdc_tx_filter(struct smc_wr_tx_pend_priv *tx_pend,
- unsigned long data)
+void smc_cdc_wait_pend_tx_wr(struct smc_connection *conn)
{
- struct smc_connection *conn = (struct smc_connection *)data;
- struct smc_cdc_tx_pend *cdc_pend =
- (struct smc_cdc_tx_pend *)tx_pend;
-
- return cdc_pend->conn == conn;
-}
-
-static void smc_cdc_tx_dismisser(struct smc_wr_tx_pend_priv *tx_pend)
-{
- struct smc_cdc_tx_pend *cdc_pend =
- (struct smc_cdc_tx_pend *)tx_pend;
-
- cdc_pend->conn = NULL;
-}
-
-void smc_cdc_tx_dismiss_slots(struct smc_connection *conn)
-{
- struct smc_link *link = conn->lnk;
-
- smc_wr_tx_dismiss_slots(link, SMC_CDC_MSG_TYPE,
- smc_cdc_tx_filter, smc_cdc_tx_dismisser,
- (unsigned long)conn);
+ wait_event(conn->cdc_pend_tx_wq, !atomic_read(&conn->cdc_pend_tx_wr));
}
/* Send a SMC-D CDC header.
diff --git a/net/smc/smc_cdc.h b/net/smc/smc_cdc.h
index 0a0a89a..696cc11 100644
--- a/net/smc/smc_cdc.h
+++ b/net/smc/smc_cdc.h
@@ -291,7 +291,7 @@ int smc_cdc_get_free_slot(struct smc_connection *conn,
struct smc_wr_buf **wr_buf,
struct smc_rdma_wr **wr_rdma_buf,
struct smc_cdc_tx_pend **pend);
-void smc_cdc_tx_dismiss_slots(struct smc_connection *conn);
+void smc_cdc_wait_pend_tx_wr(struct smc_connection *conn);
int smc_cdc_msg_send(struct smc_connection *conn, struct smc_wr_buf *wr_buf,
struct smc_cdc_tx_pend *pend);
int smc_cdc_get_slot_and_msg_send(struct smc_connection *conn);
diff --git a/net/smc/smc_clc.c b/net/smc/smc_clc.c
index 8409ab7..6be95a2 100644
--- a/net/smc/smc_clc.c
+++ b/net/smc/smc_clc.c
@@ -1021,7 +1021,6 @@ static int smc_clc_send_confirm_accept(struct smc_sock *smc,
struct smc_link *link = conn->lnk;
/* SMC-R specific settings */
- link = conn->lnk;
memcpy(clc->hdr.eyecatcher, SMC_EYECATCHER,
sizeof(SMC_EYECATCHER));
clc->hdr.typev1 = SMC_TYPE_R;
diff --git a/net/smc/smc_core.c b/net/smc/smc_core.c
index 387d28b..8935ef4 100644
--- a/net/smc/smc_core.c
+++ b/net/smc/smc_core.c
@@ -171,8 +171,10 @@ static int smc_lgr_register_conn(struct smc_connection *conn, bool first)
if (!conn->lgr->is_smcd) {
rc = smcr_lgr_conn_assign_link(conn, first);
- if (rc)
+ if (rc) {
+ conn->lgr = NULL;
return rc;
+ }
}
/* find a new alert_token_local value not yet used by some connection
* in this link group
@@ -348,6 +350,9 @@ static int smc_nl_fill_lgr(struct smc_link_group *lgr,
goto errattr;
if (nla_put_u8(skb, SMC_NLA_LGR_R_VLAN_ID, lgr->vlan_id))
goto errattr;
+ if (nla_put_u64_64bit(skb, SMC_NLA_LGR_R_NET_COOKIE,
+ lgr->net->net_cookie, SMC_NLA_LGR_R_PAD))
+ goto errattr;
memcpy(smc_target, lgr->pnet_id, SMC_MAX_PNETID_LEN);
smc_target[SMC_MAX_PNETID_LEN] = 0;
if (nla_put_string(skb, SMC_NLA_LGR_R_PNETID, smc_target))
@@ -622,15 +627,13 @@ int smcd_nl_get_lgr(struct sk_buff *skb, struct netlink_callback *cb)
return skb->len;
}
-void smc_lgr_cleanup_early(struct smc_connection *conn)
+void smc_lgr_cleanup_early(struct smc_link_group *lgr)
{
- struct smc_link_group *lgr = conn->lgr;
spinlock_t *lgr_lock;
if (!lgr)
return;
- smc_conn_free(conn);
smc_lgr_list_head(lgr, &lgr_lock);
spin_lock_bh(lgr_lock);
/* do not use this link group for new connections */
@@ -647,7 +650,7 @@ static void smcr_lgr_link_deactivate_all(struct smc_link_group *lgr)
for (i = 0; i < SMC_LINKS_PER_LGR_MAX; i++) {
struct smc_link *lnk = &lgr->lnk[i];
- if (smc_link_usable(lnk))
+ if (smc_link_sendable(lnk))
lnk->state = SMC_LNK_INACTIVE;
}
wake_up_all(&lgr->llc_msg_waiter);
@@ -897,6 +900,7 @@ static int smc_lgr_create(struct smc_sock *smc, struct smc_init_info *ini)
smc_wr_free_lgr_mem(lgr);
goto free_wq;
}
+ lgr->net = smc_ib_net(lnk->smcibdev);
lgr_list = &smc_lgr_list.list;
lgr_lock = &smc_lgr_list.lock;
atomic_inc(&lgr_cnt);
@@ -1101,18 +1105,24 @@ static void smcr_buf_unuse(struct smc_buf_desc *rmb_desc,
smc_buf_free(lgr, true, rmb_desc);
} else {
rmb_desc->used = 0;
+ memset(rmb_desc->cpu_addr, 0, rmb_desc->len);
}
}
static void smc_buf_unuse(struct smc_connection *conn,
struct smc_link_group *lgr)
{
- if (conn->sndbuf_desc)
+ if (conn->sndbuf_desc) {
conn->sndbuf_desc->used = 0;
- if (conn->rmb_desc && lgr->is_smcd)
+ memset(conn->sndbuf_desc->cpu_addr, 0, conn->sndbuf_desc->len);
+ }
+ if (conn->rmb_desc && lgr->is_smcd) {
conn->rmb_desc->used = 0;
- else if (conn->rmb_desc)
+ memset(conn->rmb_desc->cpu_addr, 0, conn->rmb_desc->len +
+ sizeof(struct smcd_cdc_msg));
+ } else if (conn->rmb_desc) {
smcr_buf_unuse(conn->rmb_desc, lgr);
+ }
}
/* remove a finished connection from its link group */
@@ -1127,7 +1137,7 @@ void smc_conn_free(struct smc_connection *conn)
smc_ism_unset_conn(conn);
tasklet_kill(&conn->rx_tsklet);
} else {
- smc_cdc_tx_dismiss_slots(conn);
+ smc_cdc_wait_pend_tx_wr(conn);
if (current_work() != &conn->abort_work)
cancel_work_sync(&conn->abort_work);
}
@@ -1204,7 +1214,7 @@ void smcr_link_clear(struct smc_link *lnk, bool log)
smc_llc_link_clear(lnk, log);
smcr_buf_unmap_lgr(lnk);
smcr_rtoken_clear_link(lnk);
- smc_ib_modify_qp_reset(lnk);
+ smc_ib_modify_qp_error(lnk);
smc_wr_free_link(lnk);
smc_ib_destroy_queue_pair(lnk);
smc_ib_dealloc_protection_domain(lnk);
@@ -1336,7 +1346,7 @@ static void smc_conn_kill(struct smc_connection *conn, bool soft)
else
tasklet_unlock_wait(&conn->rx_tsklet);
} else {
- smc_cdc_tx_dismiss_slots(conn);
+ smc_cdc_wait_pend_tx_wr(conn);
}
smc_lgr_unregister_conn(conn);
smc_close_active_abort(smc);
@@ -1459,11 +1469,16 @@ void smc_smcd_terminate_all(struct smcd_dev *smcd)
/* Called when an SMCR device is removed or the smc module is unloaded.
* If smcibdev is given, all SMCR link groups using this device are terminated.
* If smcibdev is NULL, all SMCR link groups are terminated.
+ *
+ * We must wait here for QPs been destroyed before we destroy the CQs,
+ * or we won't received any CQEs and cdc_pend_tx_wr cannot reach 0 thus
+ * smc_sock cannot be released.
*/
void smc_smcr_terminate_all(struct smc_ib_device *smcibdev)
{
struct smc_link_group *lgr, *lg;
LIST_HEAD(lgr_free_list);
+ LIST_HEAD(lgr_linkdown_list);
int i;
spin_lock_bh(&smc_lgr_list.lock);
@@ -1475,7 +1490,7 @@ void smc_smcr_terminate_all(struct smc_ib_device *smcibdev)
list_for_each_entry_safe(lgr, lg, &smc_lgr_list.list, list) {
for (i = 0; i < SMC_LINKS_PER_LGR_MAX; i++) {
if (lgr->lnk[i].smcibdev == smcibdev)
- smcr_link_down_cond_sched(&lgr->lnk[i]);
+ list_move_tail(&lgr->list, &lgr_linkdown_list);
}
}
}
@@ -1487,6 +1502,16 @@ void smc_smcr_terminate_all(struct smc_ib_device *smcibdev)
__smc_lgr_terminate(lgr, false);
}
+ list_for_each_entry_safe(lgr, lg, &lgr_linkdown_list, list) {
+ for (i = 0; i < SMC_LINKS_PER_LGR_MAX; i++) {
+ if (lgr->lnk[i].smcibdev == smcibdev) {
+ mutex_lock(&lgr->llc_conf_mutex);
+ smcr_link_down_cond(&lgr->lnk[i]);
+ mutex_unlock(&lgr->llc_conf_mutex);
+ }
+ }
+ }
+
if (smcibdev) {
if (atomic_read(&smcibdev->lnk_cnt))
wait_event(smcibdev->lnks_deleted,
@@ -1527,9 +1552,9 @@ void smcr_lgr_set_type(struct smc_link_group *lgr, enum smc_lgr_type new_type)
lgr_type = "ASYMMETRIC_LOCAL";
break;
}
- pr_warn_ratelimited("smc: SMC-R lg %*phN state changed: "
+ pr_warn_ratelimited("smc: SMC-R lg %*phN net %llu state changed: "
"%s, pnetid %.16s\n", SMC_LGR_ID_SIZE, &lgr->id,
- lgr_type, lgr->pnet_id);
+ lgr->net->net_cookie, lgr_type, lgr->pnet_id);
}
/* set new lgr type and tag a link as asymmetric */
@@ -1564,7 +1589,8 @@ void smcr_port_add(struct smc_ib_device *smcibdev, u8 ibport)
if (strncmp(smcibdev->pnetid[ibport - 1], lgr->pnet_id,
SMC_MAX_PNETID_LEN) ||
lgr->type == SMC_LGR_SYMMETRIC ||
- lgr->type == SMC_LGR_ASYMMETRIC_PEER)
+ lgr->type == SMC_LGR_ASYMMETRIC_PEER ||
+ !rdma_dev_access_netns(smcibdev->ibdev, lgr->net))
continue;
/* trigger local add link processing */
@@ -1586,7 +1612,6 @@ static void smcr_link_down(struct smc_link *lnk)
if (!lgr || lnk->state == SMC_LNK_UNUSED || list_empty(&lgr->list))
return;
- smc_ib_modify_qp_reset(lnk);
to_lnk = smc_switch_conns(lgr, lnk, true);
if (!to_lnk) { /* no backup link available */
smcr_link_clear(lnk, true);
@@ -1723,8 +1748,10 @@ static bool smcr_lgr_match(struct smc_link_group *lgr, u8 smcr_version,
u8 peer_systemid[],
u8 peer_gid[],
u8 peer_mac_v1[],
- enum smc_lgr_role role, u32 clcqpn)
+ enum smc_lgr_role role, u32 clcqpn,
+ struct net *net)
{
+ struct smc_link *lnk;
int i;
if (memcmp(lgr->peer_systemid, peer_systemid, SMC_SYSTEMID_LEN) ||
@@ -1732,12 +1759,17 @@ static bool smcr_lgr_match(struct smc_link_group *lgr, u8 smcr_version,
return false;
for (i = 0; i < SMC_LINKS_PER_LGR_MAX; i++) {
- if (!smc_link_active(&lgr->lnk[i]))
+ lnk = &lgr->lnk[i];
+
+ if (!smc_link_active(lnk))
continue;
- if ((lgr->role == SMC_SERV || lgr->lnk[i].peer_qpn == clcqpn) &&
- !memcmp(lgr->lnk[i].peer_gid, peer_gid, SMC_GID_SIZE) &&
+ /* use verbs API to check netns, instead of lgr->net */
+ if (!rdma_dev_access_netns(lnk->smcibdev->ibdev, net))
+ return false;
+ if ((lgr->role == SMC_SERV || lnk->peer_qpn == clcqpn) &&
+ !memcmp(lnk->peer_gid, peer_gid, SMC_GID_SIZE) &&
(smcr_version == SMC_V2 ||
- !memcmp(lgr->lnk[i].peer_mac, peer_mac_v1, ETH_ALEN)))
+ !memcmp(lnk->peer_mac, peer_mac_v1, ETH_ALEN)))
return true;
}
return false;
@@ -1753,6 +1785,7 @@ static bool smcd_lgr_match(struct smc_link_group *lgr,
int smc_conn_create(struct smc_sock *smc, struct smc_init_info *ini)
{
struct smc_connection *conn = &smc->conn;
+ struct net *net = sock_net(&smc->sk);
struct list_head *lgr_list;
struct smc_link_group *lgr;
enum smc_lgr_role role;
@@ -1779,7 +1812,7 @@ int smc_conn_create(struct smc_sock *smc, struct smc_init_info *ini)
smcr_lgr_match(lgr, ini->smcr_version,
ini->peer_systemid,
ini->peer_gid, ini->peer_mac, role,
- ini->ib_clcqpn)) &&
+ ini->ib_clcqpn, net)) &&
!lgr->sync_err &&
(ini->smcd_version == SMC_V2 ||
lgr->vlan_id == ini->vlan_id) &&
@@ -1818,12 +1851,15 @@ int smc_conn_create(struct smc_sock *smc, struct smc_init_info *ini)
write_lock_bh(&lgr->conns_lock);
rc = smc_lgr_register_conn(conn, true);
write_unlock_bh(&lgr->conns_lock);
- if (rc)
+ if (rc) {
+ smc_lgr_cleanup_early(lgr);
goto out;
+ }
}
conn->local_tx_ctrl.common.type = SMC_CDC_MSG_TYPE;
conn->local_tx_ctrl.len = SMC_WR_TX_SIZE;
conn->urg_state = SMC_URG_READ;
+ init_waitqueue_head(&conn->cdc_pend_tx_wq);
INIT_WORK(&smc->conn.abort_work, smc_conn_abort_work);
if (ini->is_smcd) {
conn->rx_off = sizeof(struct smcd_cdc_msg);
@@ -2148,7 +2184,6 @@ static int __smc_buf_create(struct smc_sock *smc, bool is_smcd, bool is_rmb)
if (buf_desc) {
SMC_STAT_RMB_SIZE(smc, is_smcd, is_rmb, bufsize);
SMC_STAT_BUF_REUSE(smc, is_smcd, is_rmb);
- memset(buf_desc->cpu_addr, 0, bufsize);
break; /* found reusable slot */
}
diff --git a/net/smc/smc_core.h b/net/smc/smc_core.h
index 59cef3b..521c64a 100644
--- a/net/smc/smc_core.h
+++ b/net/smc/smc_core.h
@@ -306,6 +306,8 @@ struct smc_link_group {
u8 nexthop_mac[ETH_ALEN];
u8 uses_gateway;
__be32 saddr;
+ /* net namespace */
+ struct net *net;
};
struct { /* SMC-D */
u64 peer_gid;
@@ -407,7 +409,13 @@ static inline struct smc_connection *smc_lgr_find_conn(
return res;
}
-/* returns true if the specified link is usable */
+/*
+ * Returns true if the specified link is usable.
+ *
+ * usable means the link is ready to receive RDMA messages, map memory
+ * on the link, etc. This doesn't ensure we are able to send RDMA messages
+ * on this link, if sending RDMA messages is needed, use smc_link_sendable()
+ */
static inline bool smc_link_usable(struct smc_link *lnk)
{
if (lnk->state == SMC_LNK_UNUSED || lnk->state == SMC_LNK_INACTIVE)
@@ -415,6 +423,21 @@ static inline bool smc_link_usable(struct smc_link *lnk)
return true;
}
+/*
+ * Returns true if the specified link is ready to receive AND send RDMA
+ * messages.
+ *
+ * For the client side in first contact, the underlying QP may still in
+ * RESET or RTR when the link state is ACTIVATING, checks in smc_link_usable()
+ * is not strong enough. For those places that need to send any CDC or LLC
+ * messages, use smc_link_sendable(), otherwise, use smc_link_usable() instead
+ */
+static inline bool smc_link_sendable(struct smc_link *lnk)
+{
+ return smc_link_usable(lnk) &&
+ lnk->qp_attr.cur_qp_state == IB_QPS_RTS;
+}
+
static inline bool smc_link_active(struct smc_link *lnk)
{
return lnk->state == SMC_LNK_ACTIVE;
@@ -462,7 +485,7 @@ static inline void smc_set_pci_values(struct pci_dev *pci_dev,
struct smc_sock;
struct smc_clc_msg_accept_confirm;
-void smc_lgr_cleanup_early(struct smc_connection *conn);
+void smc_lgr_cleanup_early(struct smc_link_group *lgr);
void smc_lgr_terminate_sched(struct smc_link_group *lgr);
void smcr_port_add(struct smc_ib_device *smcibdev, u8 ibport);
void smcr_port_err(struct smc_ib_device *smcibdev, u8 ibport);
diff --git a/net/smc/smc_diag.c b/net/smc/smc_diag.c
index c952986a..7c8dad2 100644
--- a/net/smc/smc_diag.c
+++ b/net/smc/smc_diag.c
@@ -145,19 +145,21 @@ static int __smc_diag_dump(struct sock *sk, struct sk_buff *skb,
if (smc->conn.lgr && !smc->conn.lgr->is_smcd &&
(req->diag_ext & (1 << (SMC_DIAG_LGRINFO - 1))) &&
!list_empty(&smc->conn.lgr->list)) {
+ struct smc_link *link = smc->conn.lnk;
+ struct net *net = read_pnet(&link->smcibdev->ibdev->coredev.rdma_net);
+
struct smc_diag_lgrinfo linfo = {
.role = smc->conn.lgr->role,
- .lnk[0].ibport = smc->conn.lnk->ibport,
- .lnk[0].link_id = smc->conn.lnk->link_id,
+ .lnk[0].ibport = link->ibport,
+ .lnk[0].link_id = link->link_id,
+ .lnk[0].net_cookie = net->net_cookie,
};
memcpy(linfo.lnk[0].ibname,
smc->conn.lgr->lnk[0].smcibdev->ibdev->name,
- sizeof(smc->conn.lnk->smcibdev->ibdev->name));
- smc_gid_be16_convert(linfo.lnk[0].gid,
- smc->conn.lnk->gid);
- smc_gid_be16_convert(linfo.lnk[0].peer_gid,
- smc->conn.lnk->peer_gid);
+ sizeof(link->smcibdev->ibdev->name));
+ smc_gid_be16_convert(linfo.lnk[0].gid, link->gid);
+ smc_gid_be16_convert(linfo.lnk[0].peer_gid, link->peer_gid);
if (nla_put(skb, SMC_DIAG_LGRINFO, sizeof(linfo), &linfo) < 0)
goto errout;
diff --git a/net/smc/smc_ib.c b/net/smc/smc_ib.c
index d93055ec..a3e2d3b 100644
--- a/net/smc/smc_ib.c
+++ b/net/smc/smc_ib.c
@@ -12,6 +12,8 @@
* Author(s): Ursula Braun <ubraun@linux.vnet.ibm.com>
*/
+#include <linux/etherdevice.h>
+#include <linux/if_vlan.h>
#include <linux/random.h>
#include <linux/workqueue.h>
#include <linux/scatterlist.h>
@@ -109,12 +111,12 @@ int smc_ib_modify_qp_rts(struct smc_link *lnk)
IB_QP_MAX_QP_RD_ATOMIC);
}
-int smc_ib_modify_qp_reset(struct smc_link *lnk)
+int smc_ib_modify_qp_error(struct smc_link *lnk)
{
struct ib_qp_attr qp_attr;
memset(&qp_attr, 0, sizeof(qp_attr));
- qp_attr.qp_state = IB_QPS_RESET;
+ qp_attr.qp_state = IB_QPS_ERR;
return ib_modify_qp(lnk->roce_qp, &qp_attr, IB_QP_STATE);
}
diff --git a/net/smc/smc_ib.h b/net/smc/smc_ib.h
index 0758593..5d8b49c 100644
--- a/net/smc/smc_ib.h
+++ b/net/smc/smc_ib.h
@@ -69,6 +69,13 @@ static inline __be32 smc_ib_gid_to_ipv4(u8 gid[SMC_GID_SIZE])
return cpu_to_be32(INADDR_NONE);
}
+static inline struct net *smc_ib_net(struct smc_ib_device *smcibdev)
+{
+ if (smcibdev && smcibdev->ibdev)
+ return read_pnet(&smcibdev->ibdev->coredev.rdma_net);
+ return NULL;
+}
+
struct smc_init_info_smcrv2;
struct smc_buf_desc;
struct smc_link;
@@ -90,6 +97,7 @@ int smc_ib_create_queue_pair(struct smc_link *lnk);
int smc_ib_ready_link(struct smc_link *lnk);
int smc_ib_modify_qp_rts(struct smc_link *lnk);
int smc_ib_modify_qp_reset(struct smc_link *lnk);
+int smc_ib_modify_qp_error(struct smc_link *lnk);
long smc_ib_setup_per_ibdev(struct smc_ib_device *smcibdev);
int smc_ib_get_memory_region(struct ib_pd *pd, int access_flags,
struct smc_buf_desc *buf_slot, u8 link_idx);
diff --git a/net/smc/smc_ism.c b/net/smc/smc_ism.c
index fd28cc4..a2084ec 100644
--- a/net/smc/smc_ism.c
+++ b/net/smc/smc_ism.c
@@ -6,6 +6,7 @@
* Copyright IBM Corp. 2018
*/
+#include <linux/if_vlan.h>
#include <linux/spinlock.h>
#include <linux/mutex.h>
#include <linux/slab.h>
diff --git a/net/smc/smc_llc.c b/net/smc/smc_llc.c
index b102680..c4d057b 100644
--- a/net/smc/smc_llc.c
+++ b/net/smc/smc_llc.c
@@ -242,9 +242,10 @@ static void smc_llc_flow_parallel(struct smc_link_group *lgr, u8 flow_type,
}
/* drop parallel or already-in-progress llc requests */
if (flow_type != msg_type)
- pr_warn_once("smc: SMC-R lg %*phN dropped parallel "
+ pr_warn_once("smc: SMC-R lg %*phN net %llu dropped parallel "
"LLC msg: msg %d flow %d role %d\n",
SMC_LGR_ID_SIZE, &lgr->id,
+ lgr->net->net_cookie,
qentry->msg.raw.hdr.common.type,
flow_type, lgr->role);
kfree(qentry);
@@ -359,9 +360,10 @@ struct smc_llc_qentry *smc_llc_wait(struct smc_link_group *lgr,
smc_llc_flow_qentry_clr(flow));
return NULL;
}
- pr_warn_once("smc: SMC-R lg %*phN dropped unexpected LLC msg: "
+ pr_warn_once("smc: SMC-R lg %*phN net %llu dropped unexpected LLC msg: "
"msg %d exp %d flow %d role %d flags %x\n",
- SMC_LGR_ID_SIZE, &lgr->id, rcv_msg, exp_msg,
+ SMC_LGR_ID_SIZE, &lgr->id, lgr->net->net_cookie,
+ rcv_msg, exp_msg,
flow->type, lgr->role,
flow->qentry->msg.raw.hdr.flags);
smc_llc_flow_qentry_del(flow);
@@ -1630,7 +1632,7 @@ void smc_llc_send_link_delete_all(struct smc_link_group *lgr, bool ord, u32 rsn)
delllc.reason = htonl(rsn);
for (i = 0; i < SMC_LINKS_PER_LGR_MAX; i++) {
- if (!smc_link_usable(&lgr->lnk[i]))
+ if (!smc_link_sendable(&lgr->lnk[i]))
continue;
if (!smc_llc_send_message_wait(&lgr->lnk[i], &delllc))
break;
@@ -1816,8 +1818,9 @@ static void smc_llc_rmt_delete_rkey(struct smc_link_group *lgr)
static void smc_llc_protocol_violation(struct smc_link_group *lgr, u8 type)
{
- pr_warn_ratelimited("smc: SMC-R lg %*phN LLC protocol violation: "
- "llc_type %d\n", SMC_LGR_ID_SIZE, &lgr->id, type);
+ pr_warn_ratelimited("smc: SMC-R lg %*phN net %llu LLC protocol violation: "
+ "llc_type %d\n", SMC_LGR_ID_SIZE, &lgr->id,
+ lgr->net->net_cookie, type);
smc_llc_set_termination_rsn(lgr, SMC_LLC_DEL_PROT_VIOL);
smc_lgr_terminate_sched(lgr);
}
@@ -2146,9 +2149,10 @@ int smc_llc_link_init(struct smc_link *link)
void smc_llc_link_active(struct smc_link *link)
{
- pr_warn_ratelimited("smc: SMC-R lg %*phN link added: id %*phN, "
+ pr_warn_ratelimited("smc: SMC-R lg %*phN net %llu link added: id %*phN, "
"peerid %*phN, ibdev %s, ibport %d\n",
SMC_LGR_ID_SIZE, &link->lgr->id,
+ link->lgr->net->net_cookie,
SMC_LGR_ID_SIZE, &link->link_uid,
SMC_LGR_ID_SIZE, &link->peer_link_uid,
link->smcibdev->ibdev->name, link->ibport);
@@ -2164,9 +2168,10 @@ void smc_llc_link_active(struct smc_link *link)
void smc_llc_link_clear(struct smc_link *link, bool log)
{
if (log)
- pr_warn_ratelimited("smc: SMC-R lg %*phN link removed: id %*phN"
+ pr_warn_ratelimited("smc: SMC-R lg %*phN net %llu link removed: id %*phN"
", peerid %*phN, ibdev %s, ibport %d\n",
SMC_LGR_ID_SIZE, &link->lgr->id,
+ link->lgr->net->net_cookie,
SMC_LGR_ID_SIZE, &link->link_uid,
SMC_LGR_ID_SIZE, &link->peer_link_uid,
link->smcibdev->ibdev->name, link->ibport);
diff --git a/net/smc/smc_pnet.c b/net/smc/smc_pnet.c
index 67e9d9f..db9825c0 100644
--- a/net/smc/smc_pnet.c
+++ b/net/smc/smc_pnet.c
@@ -64,6 +64,7 @@ struct smc_pnetentry {
struct {
char eth_name[IFNAMSIZ + 1];
struct net_device *ndev;
+ netdevice_tracker dev_tracker;
};
struct {
char ib_name[IB_DEVICE_NAME_MAX + 1];
@@ -119,7 +120,7 @@ static int smc_pnet_remove_by_pnetid(struct net *net, char *pnet_name)
smc_pnet_match(pnetelem->pnet_name, pnet_name)) {
list_del(&pnetelem->list);
if (pnetelem->type == SMC_PNET_ETH && pnetelem->ndev) {
- dev_put(pnetelem->ndev);
+ dev_put_track(pnetelem->ndev, &pnetelem->dev_tracker);
pr_warn_ratelimited("smc: net device %s "
"erased user defined "
"pnetid %.16s\n",
@@ -195,7 +196,7 @@ static int smc_pnet_add_by_ndev(struct net_device *ndev)
list_for_each_entry_safe(pnetelem, tmp_pe, &pnettable->pnetlist, list) {
if (pnetelem->type == SMC_PNET_ETH && !pnetelem->ndev &&
!strncmp(pnetelem->eth_name, ndev->name, IFNAMSIZ)) {
- dev_hold(ndev);
+ dev_hold_track(ndev, &pnetelem->dev_tracker, GFP_ATOMIC);
pnetelem->ndev = ndev;
rc = 0;
pr_warn_ratelimited("smc: adding net device %s with "
@@ -226,7 +227,7 @@ static int smc_pnet_remove_by_ndev(struct net_device *ndev)
write_lock(&pnettable->lock);
list_for_each_entry_safe(pnetelem, tmp_pe, &pnettable->pnetlist, list) {
if (pnetelem->type == SMC_PNET_ETH && pnetelem->ndev == ndev) {
- dev_put(pnetelem->ndev);
+ dev_put_track(pnetelem->ndev, &pnetelem->dev_tracker);
pnetelem->ndev = NULL;
rc = 0;
pr_warn_ratelimited("smc: removing net device %s with "
@@ -368,7 +369,7 @@ static int smc_pnet_add_eth(struct smc_pnettable *pnettable, struct net *net,
memcpy(new_pe->pnet_name, pnet_name, SMC_MAX_PNETID_LEN);
strncpy(new_pe->eth_name, eth_name, IFNAMSIZ);
new_pe->ndev = ndev;
-
+ netdev_tracker_alloc(ndev, &new_pe->dev_tracker, GFP_KERNEL);
rc = -EEXIST;
new_netdev = true;
write_lock(&pnettable->lock);
@@ -976,14 +977,16 @@ static int smc_pnet_determine_gid(struct smc_ib_device *ibdev, int i,
/* find a roce device for the given pnetid */
static void _smc_pnet_find_roce_by_pnetid(u8 *pnet_id,
struct smc_init_info *ini,
- struct smc_ib_device *known_dev)
+ struct smc_ib_device *known_dev,
+ struct net *net)
{
struct smc_ib_device *ibdev;
int i;
mutex_lock(&smc_ib_devices.mutex);
list_for_each_entry(ibdev, &smc_ib_devices.list, list) {
- if (ibdev == known_dev)
+ if (ibdev == known_dev ||
+ !rdma_dev_access_netns(ibdev->ibdev, net))
continue;
for (i = 1; i <= SMC_MAX_PORTS; i++) {
if (!rdma_is_port_valid(ibdev->ibdev, i))
@@ -1000,12 +1003,14 @@ static void _smc_pnet_find_roce_by_pnetid(u8 *pnet_id,
mutex_unlock(&smc_ib_devices.mutex);
}
-/* find alternate roce device with same pnet_id and vlan_id */
+/* find alternate roce device with same pnet_id, vlan_id and net namespace */
void smc_pnet_find_alt_roce(struct smc_link_group *lgr,
struct smc_init_info *ini,
struct smc_ib_device *known_dev)
{
- _smc_pnet_find_roce_by_pnetid(lgr->pnet_id, ini, known_dev);
+ struct net *net = lgr->net;
+
+ _smc_pnet_find_roce_by_pnetid(lgr->pnet_id, ini, known_dev, net);
}
/* if handshake network device belongs to a roce device, return its
@@ -1014,6 +1019,7 @@ void smc_pnet_find_alt_roce(struct smc_link_group *lgr,
static void smc_pnet_find_rdma_dev(struct net_device *netdev,
struct smc_init_info *ini)
{
+ struct net *net = dev_net(netdev);
struct smc_ib_device *ibdev;
mutex_lock(&smc_ib_devices.mutex);
@@ -1021,6 +1027,10 @@ static void smc_pnet_find_rdma_dev(struct net_device *netdev,
struct net_device *ndev;
int i;
+ /* check rdma net namespace */
+ if (!rdma_dev_access_netns(ibdev->ibdev, net))
+ continue;
+
for (i = 1; i <= SMC_MAX_PORTS; i++) {
if (!rdma_is_port_valid(ibdev->ibdev, i))
continue;
@@ -1051,15 +1061,17 @@ static void smc_pnet_find_roce_by_pnetid(struct net_device *ndev,
struct smc_init_info *ini)
{
u8 ndev_pnetid[SMC_MAX_PNETID_LEN];
+ struct net *net;
ndev = pnet_find_base_ndev(ndev);
+ net = dev_net(ndev);
if (smc_pnetid_by_dev_port(ndev->dev.parent, ndev->dev_port,
ndev_pnetid) &&
smc_pnet_find_ndev_pnetid_by_table(ndev, ndev_pnetid)) {
smc_pnet_find_rdma_dev(ndev, ini);
return; /* pnetid could not be determined */
}
- _smc_pnet_find_roce_by_pnetid(ndev_pnetid, ini, NULL);
+ _smc_pnet_find_roce_by_pnetid(ndev_pnetid, ini, NULL, net);
}
static void smc_pnet_find_ism_by_pnetid(struct net_device *ndev,
diff --git a/net/smc/smc_tracepoint.h b/net/smc/smc_tracepoint.h
index ec17f29..9fc5e58 100644
--- a/net/smc/smc_tracepoint.h
+++ b/net/smc/smc_tracepoint.h
@@ -22,6 +22,7 @@ TRACE_EVENT(smc_switch_to_fallback,
TP_STRUCT__entry(
__field(const void *, sk)
__field(const void *, clcsk)
+ __field(u64, net_cookie)
__field(int, fallback_rsn)
),
@@ -31,11 +32,13 @@ TRACE_EVENT(smc_switch_to_fallback,
__entry->sk = sk;
__entry->clcsk = clcsk;
+ __entry->net_cookie = sock_net(sk)->net_cookie;
__entry->fallback_rsn = fallback_rsn;
),
- TP_printk("sk=%p clcsk=%p fallback_rsn=%d",
- __entry->sk, __entry->clcsk, __entry->fallback_rsn)
+ TP_printk("sk=%p clcsk=%p net=%llu fallback_rsn=%d",
+ __entry->sk, __entry->clcsk,
+ __entry->net_cookie, __entry->fallback_rsn)
);
DECLARE_EVENT_CLASS(smc_msg_event,
@@ -46,19 +49,23 @@ DECLARE_EVENT_CLASS(smc_msg_event,
TP_STRUCT__entry(
__field(const void *, smc)
+ __field(u64, net_cookie)
__field(size_t, len)
__string(name, smc->conn.lnk->ibname)
),
TP_fast_assign(
+ const struct sock *sk = &smc->sk;
+
__entry->smc = smc;
+ __entry->net_cookie = sock_net(sk)->net_cookie;
__entry->len = len;
__assign_str(name, smc->conn.lnk->ibname);
),
- TP_printk("smc=%p len=%zu dev=%s",
- __entry->smc, __entry->len,
- __get_str(name))
+ TP_printk("smc=%p net=%llu len=%zu dev=%s",
+ __entry->smc, __entry->net_cookie,
+ __entry->len, __get_str(name))
);
DEFINE_EVENT(smc_msg_event, smc_tx_sendmsg,
@@ -84,6 +91,7 @@ TRACE_EVENT(smcr_link_down,
TP_STRUCT__entry(
__field(const void *, lnk)
__field(const void *, lgr)
+ __field(u64, net_cookie)
__field(int, state)
__string(name, lnk->ibname)
__field(void *, location)
@@ -94,13 +102,14 @@ TRACE_EVENT(smcr_link_down,
__entry->lnk = lnk;
__entry->lgr = lgr;
+ __entry->net_cookie = lgr->net->net_cookie;
__entry->state = lnk->state;
__assign_str(name, lnk->ibname);
__entry->location = location;
),
- TP_printk("lnk=%p lgr=%p state=%d dev=%s location=%pS",
- __entry->lnk, __entry->lgr,
+ TP_printk("lnk=%p lgr=%p net=%llu state=%d dev=%s location=%pS",
+ __entry->lnk, __entry->lgr, __entry->net_cookie,
__entry->state, __get_str(name),
__entry->location)
);
diff --git a/net/smc/smc_wr.c b/net/smc/smc_wr.c
index 600ab58..24be1d0 100644
--- a/net/smc/smc_wr.c
+++ b/net/smc/smc_wr.c
@@ -54,21 +54,13 @@ struct smc_wr_tx_pend { /* control data for a pending send request */
/* returns true if at least one tx work request is pending on the given link */
static inline bool smc_wr_is_tx_pend(struct smc_link *link)
{
- if (find_first_bit(link->wr_tx_mask, link->wr_tx_cnt) !=
- link->wr_tx_cnt) {
- return true;
- }
- return false;
+ return !bitmap_empty(link->wr_tx_mask, link->wr_tx_cnt);
}
/* wait till all pending tx work requests on the given link are completed */
-int smc_wr_tx_wait_no_pending_sends(struct smc_link *link)
+void smc_wr_tx_wait_no_pending_sends(struct smc_link *link)
{
- if (wait_event_timeout(link->wr_tx_wait, !smc_wr_is_tx_pend(link),
- SMC_WR_TX_WAIT_PENDING_TIME))
- return 0;
- else /* timeout */
- return -EPIPE;
+ wait_event(link->wr_tx_wait, !smc_wr_is_tx_pend(link));
}
static inline int smc_wr_tx_find_pending_index(struct smc_link *link, u64 wr_id)
@@ -87,7 +79,6 @@ static inline void smc_wr_tx_process_cqe(struct ib_wc *wc)
struct smc_wr_tx_pend pnd_snd;
struct smc_link *link;
u32 pnd_snd_idx;
- int i;
link = wc->qp->qp_context;
@@ -128,14 +119,6 @@ static inline void smc_wr_tx_process_cqe(struct ib_wc *wc)
}
if (wc->status) {
- for_each_set_bit(i, link->wr_tx_mask, link->wr_tx_cnt) {
- /* clear full struct smc_wr_tx_pend including .priv */
- memset(&link->wr_tx_pends[i], 0,
- sizeof(link->wr_tx_pends[i]));
- memset(&link->wr_tx_bufs[i], 0,
- sizeof(link->wr_tx_bufs[i]));
- clear_bit(i, link->wr_tx_mask);
- }
if (link->lgr->smc_version == SMC_V2) {
memset(link->wr_tx_v2_pend, 0,
sizeof(*link->wr_tx_v2_pend));
@@ -188,7 +171,7 @@ void smc_wr_tx_cq_handler(struct ib_cq *ib_cq, void *cq_context)
static inline int smc_wr_tx_get_free_slot_index(struct smc_link *link, u32 *idx)
{
*idx = link->wr_tx_cnt;
- if (!smc_link_usable(link))
+ if (!smc_link_sendable(link))
return -ENOLINK;
for_each_clear_bit(*idx, link->wr_tx_mask, link->wr_tx_cnt) {
if (!test_and_set_bit(*idx, link->wr_tx_mask))
@@ -231,7 +214,7 @@ int smc_wr_tx_get_free_slot(struct smc_link *link,
} else {
rc = wait_event_interruptible_timeout(
link->wr_tx_wait,
- !smc_link_usable(link) ||
+ !smc_link_sendable(link) ||
lgr->terminating ||
(smc_wr_tx_get_free_slot_index(link, &idx) != -EBUSY),
SMC_WR_TX_WAIT_FREE_SLOT_TIME);
@@ -358,18 +341,20 @@ int smc_wr_tx_send_wait(struct smc_link *link, struct smc_wr_tx_pend_priv *priv,
unsigned long timeout)
{
struct smc_wr_tx_pend *pend;
+ u32 pnd_idx;
int rc;
pend = container_of(priv, struct smc_wr_tx_pend, priv);
pend->compl_requested = 1;
- init_completion(&link->wr_tx_compl[pend->idx]);
+ pnd_idx = pend->idx;
+ init_completion(&link->wr_tx_compl[pnd_idx]);
rc = smc_wr_tx_send(link, priv);
if (rc)
return rc;
/* wait for completion by smc_wr_tx_process_cqe() */
rc = wait_for_completion_interruptible_timeout(
- &link->wr_tx_compl[pend->idx], timeout);
+ &link->wr_tx_compl[pnd_idx], timeout);
if (rc <= 0)
rc = -ENODATA;
if (rc > 0)
@@ -419,25 +404,6 @@ int smc_wr_reg_send(struct smc_link *link, struct ib_mr *mr)
return rc;
}
-void smc_wr_tx_dismiss_slots(struct smc_link *link, u8 wr_tx_hdr_type,
- smc_wr_tx_filter filter,
- smc_wr_tx_dismisser dismisser,
- unsigned long data)
-{
- struct smc_wr_tx_pend_priv *tx_pend;
- struct smc_wr_rx_hdr *wr_tx;
- int i;
-
- for_each_set_bit(i, link->wr_tx_mask, link->wr_tx_cnt) {
- wr_tx = (struct smc_wr_rx_hdr *)&link->wr_tx_bufs[i];
- if (wr_tx->type != wr_tx_hdr_type)
- continue;
- tx_pend = &link->wr_tx_pends[i].priv;
- if (filter(tx_pend, data))
- dismisser(tx_pend);
- }
-}
-
/****************************** receive queue ********************************/
int smc_wr_rx_register_handler(struct smc_wr_rx_handler *handler)
@@ -673,10 +639,7 @@ void smc_wr_free_link(struct smc_link *lnk)
smc_wr_wakeup_reg_wait(lnk);
smc_wr_wakeup_tx_wait(lnk);
- if (smc_wr_tx_wait_no_pending_sends(lnk))
- memset(lnk->wr_tx_mask, 0,
- BITS_TO_LONGS(SMC_WR_BUF_CNT) *
- sizeof(*lnk->wr_tx_mask));
+ smc_wr_tx_wait_no_pending_sends(lnk);
wait_event(lnk->wr_reg_wait, (!atomic_read(&lnk->wr_reg_refcnt)));
wait_event(lnk->wr_tx_wait, (!atomic_read(&lnk->wr_tx_refcnt)));
@@ -729,7 +692,7 @@ void smc_wr_free_link_mem(struct smc_link *lnk)
lnk->wr_tx_compl = NULL;
kfree(lnk->wr_tx_pends);
lnk->wr_tx_pends = NULL;
- kfree(lnk->wr_tx_mask);
+ bitmap_free(lnk->wr_tx_mask);
lnk->wr_tx_mask = NULL;
kfree(lnk->wr_tx_sges);
lnk->wr_tx_sges = NULL;
@@ -805,9 +768,7 @@ int smc_wr_alloc_link_mem(struct smc_link *link)
GFP_KERNEL);
if (!link->wr_rx_sges)
goto no_mem_wr_tx_sges;
- link->wr_tx_mask = kcalloc(BITS_TO_LONGS(SMC_WR_BUF_CNT),
- sizeof(*link->wr_tx_mask),
- GFP_KERNEL);
+ link->wr_tx_mask = bitmap_zalloc(SMC_WR_BUF_CNT, GFP_KERNEL);
if (!link->wr_tx_mask)
goto no_mem_wr_rx_sges;
link->wr_tx_pends = kcalloc(SMC_WR_BUF_CNT,
@@ -920,8 +881,7 @@ int smc_wr_create_link(struct smc_link *lnk)
goto dma_unmap;
}
smc_wr_init_sge(lnk);
- memset(lnk->wr_tx_mask, 0,
- BITS_TO_LONGS(SMC_WR_BUF_CNT) * sizeof(*lnk->wr_tx_mask));
+ bitmap_zero(lnk->wr_tx_mask, SMC_WR_BUF_CNT);
init_waitqueue_head(&lnk->wr_tx_wait);
atomic_set(&lnk->wr_tx_refcnt, 0);
init_waitqueue_head(&lnk->wr_reg_wait);
diff --git a/net/smc/smc_wr.h b/net/smc/smc_wr.h
index f353311..47512cc 100644
--- a/net/smc/smc_wr.h
+++ b/net/smc/smc_wr.h
@@ -22,7 +22,6 @@
#define SMC_WR_BUF_CNT 16 /* # of ctrl buffers per link */
#define SMC_WR_TX_WAIT_FREE_SLOT_TIME (10 * HZ)
-#define SMC_WR_TX_WAIT_PENDING_TIME (5 * HZ)
#define SMC_WR_TX_SIZE 44 /* actual size of wr_send data (<=SMC_WR_BUF_SIZE) */
@@ -62,7 +61,7 @@ static inline void smc_wr_tx_set_wr_id(atomic_long_t *wr_tx_id, long val)
static inline bool smc_wr_tx_link_hold(struct smc_link *link)
{
- if (!smc_link_usable(link))
+ if (!smc_link_sendable(link))
return false;
atomic_inc(&link->wr_tx_refcnt);
return true;
@@ -130,7 +129,7 @@ void smc_wr_tx_dismiss_slots(struct smc_link *lnk, u8 wr_rx_hdr_type,
smc_wr_tx_filter filter,
smc_wr_tx_dismisser dismisser,
unsigned long data);
-int smc_wr_tx_wait_no_pending_sends(struct smc_link *link);
+void smc_wr_tx_wait_no_pending_sends(struct smc_link *link);
int smc_wr_rx_register_handler(struct smc_wr_rx_handler *handler);
int smc_wr_rx_post_init(struct smc_link *link);
diff --git a/net/socket.c b/net/socket.c
index 7f64a6e..50cf757 100644
--- a/net/socket.c
+++ b/net/socket.c
@@ -52,6 +52,7 @@
* Based upon Swansea University Computer Society NET3.039
*/
+#include <linux/bpf-cgroup.h>
#include <linux/ethtool.h>
#include <linux/mm.h>
#include <linux/socket.h>
@@ -829,6 +830,7 @@ void __sock_recv_timestamp(struct msghdr *msg, struct sock *sk,
int empty = 1, false_tstamp = 0;
struct skb_shared_hwtstamps *shhwtstamps =
skb_hwtstamps(skb);
+ ktime_t hwtstamp;
/* Race occurred between timestamp enabling and packet
receiving. Fill in the current time for now. */
@@ -877,10 +879,12 @@ void __sock_recv_timestamp(struct msghdr *msg, struct sock *sk,
(sk->sk_tsflags & SOF_TIMESTAMPING_RAW_HARDWARE) &&
!skb_is_swtx_tstamp(skb, false_tstamp)) {
if (sk->sk_tsflags & SOF_TIMESTAMPING_BIND_PHC)
- ptp_convert_timestamp(shhwtstamps, sk->sk_bind_phc);
+ hwtstamp = ptp_convert_timestamp(shhwtstamps,
+ sk->sk_bind_phc);
+ else
+ hwtstamp = shhwtstamps->hwtstamp;
- if (ktime_to_timespec64_cond(shhwtstamps->hwtstamp,
- tss.ts + 2)) {
+ if (ktime_to_timespec64_cond(hwtstamp, tss.ts + 2)) {
empty = 0;
if ((sk->sk_tsflags & SOF_TIMESTAMPING_OPT_PKTINFO) &&
@@ -1946,7 +1950,7 @@ int __sys_getsockname(int fd, struct sockaddr __user *usockaddr,
err = sock->ops->getname(sock, (struct sockaddr *)&address, 0);
if (err < 0)
goto out_put;
- /* "err" is actually length in this case */
+ /* "err" is actually length in this case */
err = move_addr_to_user(&address, err, usockaddr, usockaddr_len);
out_put:
@@ -3233,21 +3237,6 @@ static int compat_ifr_data_ioctl(struct net *net, unsigned int cmd,
return dev_ioctl(net, cmd, &ifreq, data, NULL);
}
-/* Since old style bridge ioctl's endup using SIOCDEVPRIVATE
- * for some operations; this forces use of the newer bridge-utils that
- * use compatible ioctls
- */
-static int old_bridge_ioctl(compat_ulong_t __user *argp)
-{
- compat_ulong_t tmp;
-
- if (get_user(tmp, argp))
- return -EFAULT;
- if (tmp == BRCTL_GET_VERSION)
- return BRCTL_VERSION + 1;
- return -EINVAL;
-}
-
static int compat_sock_ioctl_trans(struct file *file, struct socket *sock,
unsigned int cmd, unsigned long arg)
{
@@ -3259,9 +3248,6 @@ static int compat_sock_ioctl_trans(struct file *file, struct socket *sock,
return sock_ioctl(file, cmd, (unsigned long)argp);
switch (cmd) {
- case SIOCSIFBR:
- case SIOCGIFBR:
- return old_bridge_ioctl(argp);
case SIOCWANDEV:
return compat_siocwandev(net, argp);
case SIOCGSTAMP_OLD:
@@ -3290,6 +3276,8 @@ static int compat_sock_ioctl_trans(struct file *file, struct socket *sock,
case SIOCGSTAMP_NEW:
case SIOCGSTAMPNS_NEW:
case SIOCGIFCONF:
+ case SIOCSIFBR:
+ case SIOCGIFBR:
return sock_ioctl(file, cmd, arg);
case SIOCGIFFLAGS:
diff --git a/net/switchdev/switchdev.c b/net/switchdev/switchdev.c
index 8346047..b625652 100644
--- a/net/switchdev/switchdev.c
+++ b/net/switchdev/switchdev.c
@@ -28,6 +28,7 @@ typedef void switchdev_deferred_func_t(struct net_device *dev,
struct switchdev_deferred_item {
struct list_head list;
struct net_device *dev;
+ netdevice_tracker dev_tracker;
switchdev_deferred_func_t *func;
unsigned long data[];
};
@@ -63,7 +64,7 @@ void switchdev_deferred_process(void)
while ((dfitem = switchdev_deferred_dequeue())) {
dfitem->func(dfitem->dev, dfitem->data);
- dev_put(dfitem->dev);
+ dev_put_track(dfitem->dev, &dfitem->dev_tracker);
kfree(dfitem);
}
}
@@ -90,7 +91,7 @@ static int switchdev_deferred_enqueue(struct net_device *dev,
dfitem->dev = dev;
dfitem->func = func;
memcpy(dfitem->data, data, data_len);
- dev_hold(dev);
+ dev_hold_track(dev, &dfitem->dev_tracker, GFP_ATOMIC);
spin_lock_bh(&deferred_lock);
list_add_tail(&dfitem->list, &deferred);
spin_unlock_bh(&deferred_lock);
diff --git a/net/tipc/bearer.c b/net/tipc/bearer.c
index 60bc74b..473a790 100644
--- a/net/tipc/bearer.c
+++ b/net/tipc/bearer.c
@@ -787,7 +787,7 @@ int tipc_attach_loopback(struct net *net)
if (!dev)
return -ENODEV;
- dev_hold(dev);
+ dev_hold_track(dev, &tn->loopback_pt.dev_tracker, GFP_KERNEL);
tn->loopback_pt.dev = dev;
tn->loopback_pt.type = htons(ETH_P_TIPC);
tn->loopback_pt.func = tipc_loopback_rcv_pkt;
@@ -800,7 +800,7 @@ void tipc_detach_loopback(struct net *net)
struct tipc_net *tn = tipc_net(net);
dev_remove_pack(&tn->loopback_pt);
- dev_put(net->loopback_dev);
+ dev_put_track(net->loopback_dev, &tn->loopback_pt.dev_tracker);
}
/* Caller should hold rtnl_lock to protect the bearer */
diff --git a/net/tipc/crypto.c b/net/tipc/crypto.c
index d293614..9325479 100644
--- a/net/tipc/crypto.c
+++ b/net/tipc/crypto.c
@@ -761,21 +761,10 @@ static int tipc_aead_encrypt(struct tipc_aead *aead, struct sk_buff *skb,
skb_tailroom(skb), tailen);
}
- if (unlikely(!skb_cloned(skb) && tailen <= skb_tailroom(skb))) {
- nsg = 1;
- trailer = skb;
- } else {
- /* TODO: We could avoid skb_cow_data() if skb has no frag_list
- * e.g. by skb_fill_page_desc() to add another page to the skb
- * with the wanted tailen... However, page skbs look not often,
- * so take it easy now!
- * Cloned skbs e.g. from link_xmit() seems no choice though :(
- */
- nsg = skb_cow_data(skb, tailen, &trailer);
- if (unlikely(nsg < 0)) {
- pr_err("TX: skb_cow_data() returned %d\n", nsg);
- return nsg;
- }
+ nsg = skb_cow_data(skb, tailen, &trailer);
+ if (unlikely(nsg < 0)) {
+ pr_err("TX: skb_cow_data() returned %d\n", nsg);
+ return nsg;
}
pskb_put(skb, trailer, tailen);
diff --git a/net/tipc/link.c b/net/tipc/link.c
index 09ae844..8d9e09f 100644
--- a/net/tipc/link.c
+++ b/net/tipc/link.c
@@ -1298,7 +1298,8 @@ static bool tipc_data_input(struct tipc_link *l, struct sk_buff *skb,
return false;
#ifdef CONFIG_TIPC_CRYPTO
case MSG_CRYPTO:
- if (TIPC_SKB_CB(skb)->decrypted) {
+ if (sysctl_tipc_key_exchange_enabled &&
+ TIPC_SKB_CB(skb)->decrypted) {
tipc_crypto_msg_rcv(l->net, skb);
return true;
}
diff --git a/net/tipc/socket.c b/net/tipc/socket.c
index ad570c2..3e63c83 100644
--- a/net/tipc/socket.c
+++ b/net/tipc/socket.c
@@ -1461,6 +1461,8 @@ static int __tipc_sendmsg(struct socket *sock, struct msghdr *m, size_t dlen)
msg_set_syn(hdr, 1);
}
+ memset(&skaddr, 0, sizeof(skaddr));
+
/* Determine destination */
if (atype == TIPC_SERVICE_RANGE) {
return tipc_sendmcast(sock, ua, m, dlen, timeout);
diff --git a/net/tls/tls_sw.c b/net/tls/tls_sw.c
index dfe623a..95e774f 100644
--- a/net/tls/tls_sw.c
+++ b/net/tls/tls_sw.c
@@ -1990,6 +1990,7 @@ int tls_sw_recvmsg(struct sock *sk,
end:
release_sock(sk);
+ sk_defer_free_flush(sk);
if (psock)
sk_psock_put(sk, psock);
return copied ? : err;
@@ -2328,10 +2329,6 @@ int tls_set_sw_offload(struct sock *sk, struct tls_context *ctx, int tx)
struct tls_context *tls_ctx = tls_get_ctx(sk);
struct tls_prot_info *prot = &tls_ctx->prot_info;
struct tls_crypto_info *crypto_info;
- struct tls12_crypto_info_aes_gcm_128 *gcm_128_info;
- struct tls12_crypto_info_aes_gcm_256 *gcm_256_info;
- struct tls12_crypto_info_aes_ccm_128 *ccm_128_info;
- struct tls12_crypto_info_chacha20_poly1305 *chacha20_poly1305_info;
struct tls_sw_context_tx *sw_ctx_tx = NULL;
struct tls_sw_context_rx *sw_ctx_rx = NULL;
struct cipher_context *cctx;
@@ -2394,15 +2391,15 @@ int tls_set_sw_offload(struct sock *sk, struct tls_context *ctx, int tx)
switch (crypto_info->cipher_type) {
case TLS_CIPHER_AES_GCM_128: {
+ struct tls12_crypto_info_aes_gcm_128 *gcm_128_info;
+
+ gcm_128_info = (void *)crypto_info;
nonce_size = TLS_CIPHER_AES_GCM_128_IV_SIZE;
tag_size = TLS_CIPHER_AES_GCM_128_TAG_SIZE;
iv_size = TLS_CIPHER_AES_GCM_128_IV_SIZE;
- iv = ((struct tls12_crypto_info_aes_gcm_128 *)crypto_info)->iv;
+ iv = gcm_128_info->iv;
rec_seq_size = TLS_CIPHER_AES_GCM_128_REC_SEQ_SIZE;
- rec_seq =
- ((struct tls12_crypto_info_aes_gcm_128 *)crypto_info)->rec_seq;
- gcm_128_info =
- (struct tls12_crypto_info_aes_gcm_128 *)crypto_info;
+ rec_seq = gcm_128_info->rec_seq;
keysize = TLS_CIPHER_AES_GCM_128_KEY_SIZE;
key = gcm_128_info->key;
salt = gcm_128_info->salt;
@@ -2411,15 +2408,15 @@ int tls_set_sw_offload(struct sock *sk, struct tls_context *ctx, int tx)
break;
}
case TLS_CIPHER_AES_GCM_256: {
+ struct tls12_crypto_info_aes_gcm_256 *gcm_256_info;
+
+ gcm_256_info = (void *)crypto_info;
nonce_size = TLS_CIPHER_AES_GCM_256_IV_SIZE;
tag_size = TLS_CIPHER_AES_GCM_256_TAG_SIZE;
iv_size = TLS_CIPHER_AES_GCM_256_IV_SIZE;
- iv = ((struct tls12_crypto_info_aes_gcm_256 *)crypto_info)->iv;
+ iv = gcm_256_info->iv;
rec_seq_size = TLS_CIPHER_AES_GCM_256_REC_SEQ_SIZE;
- rec_seq =
- ((struct tls12_crypto_info_aes_gcm_256 *)crypto_info)->rec_seq;
- gcm_256_info =
- (struct tls12_crypto_info_aes_gcm_256 *)crypto_info;
+ rec_seq = gcm_256_info->rec_seq;
keysize = TLS_CIPHER_AES_GCM_256_KEY_SIZE;
key = gcm_256_info->key;
salt = gcm_256_info->salt;
@@ -2428,15 +2425,15 @@ int tls_set_sw_offload(struct sock *sk, struct tls_context *ctx, int tx)
break;
}
case TLS_CIPHER_AES_CCM_128: {
+ struct tls12_crypto_info_aes_ccm_128 *ccm_128_info;
+
+ ccm_128_info = (void *)crypto_info;
nonce_size = TLS_CIPHER_AES_CCM_128_IV_SIZE;
tag_size = TLS_CIPHER_AES_CCM_128_TAG_SIZE;
iv_size = TLS_CIPHER_AES_CCM_128_IV_SIZE;
- iv = ((struct tls12_crypto_info_aes_ccm_128 *)crypto_info)->iv;
+ iv = ccm_128_info->iv;
rec_seq_size = TLS_CIPHER_AES_CCM_128_REC_SEQ_SIZE;
- rec_seq =
- ((struct tls12_crypto_info_aes_ccm_128 *)crypto_info)->rec_seq;
- ccm_128_info =
- (struct tls12_crypto_info_aes_ccm_128 *)crypto_info;
+ rec_seq = ccm_128_info->rec_seq;
keysize = TLS_CIPHER_AES_CCM_128_KEY_SIZE;
key = ccm_128_info->key;
salt = ccm_128_info->salt;
@@ -2445,6 +2442,8 @@ int tls_set_sw_offload(struct sock *sk, struct tls_context *ctx, int tx)
break;
}
case TLS_CIPHER_CHACHA20_POLY1305: {
+ struct tls12_crypto_info_chacha20_poly1305 *chacha20_poly1305_info;
+
chacha20_poly1305_info = (void *)crypto_info;
nonce_size = 0;
tag_size = TLS_CIPHER_CHACHA20_POLY1305_TAG_SIZE;
diff --git a/net/unix/af_unix.c b/net/unix/af_unix.c
index b0bfc78..c195698 100644
--- a/net/unix/af_unix.c
+++ b/net/unix/af_unix.c
@@ -89,6 +89,7 @@
#include <linux/socket.h>
#include <linux/un.h>
#include <linux/fcntl.h>
+#include <linux/filter.h>
#include <linux/termios.h>
#include <linux/sockios.h>
#include <linux/net.h>
@@ -117,24 +118,64 @@
#include "scm.h"
+spinlock_t unix_table_locks[2 * UNIX_HASH_SIZE];
+EXPORT_SYMBOL_GPL(unix_table_locks);
struct hlist_head unix_socket_table[2 * UNIX_HASH_SIZE];
EXPORT_SYMBOL_GPL(unix_socket_table);
-DEFINE_SPINLOCK(unix_table_lock);
-EXPORT_SYMBOL_GPL(unix_table_lock);
static atomic_long_t unix_nr_socks;
+/* SMP locking strategy:
+ * hash table is protected with spinlock unix_table_locks
+ * each socket state is protected by separate spin lock.
+ */
-static struct hlist_head *unix_sockets_unbound(void *addr)
+static unsigned int unix_unbound_hash(struct sock *sk)
{
- unsigned long hash = (unsigned long)addr;
+ unsigned long hash = (unsigned long)sk;
hash ^= hash >> 16;
hash ^= hash >> 8;
- hash %= UNIX_HASH_SIZE;
- return &unix_socket_table[UNIX_HASH_SIZE + hash];
+ hash ^= sk->sk_type;
+
+ return UNIX_HASH_SIZE + (hash & (UNIX_HASH_SIZE - 1));
}
-#define UNIX_ABSTRACT(sk) (unix_sk(sk)->addr->hash < UNIX_HASH_SIZE)
+static unsigned int unix_bsd_hash(struct inode *i)
+{
+ return i->i_ino & (UNIX_HASH_SIZE - 1);
+}
+
+static unsigned int unix_abstract_hash(struct sockaddr_un *sunaddr,
+ int addr_len, int type)
+{
+ __wsum csum = csum_partial(sunaddr, addr_len, 0);
+ unsigned int hash;
+
+ hash = (__force unsigned int)csum_fold(csum);
+ hash ^= hash >> 8;
+ hash ^= type;
+
+ return hash & (UNIX_HASH_SIZE - 1);
+}
+
+static void unix_table_double_lock(unsigned int hash1, unsigned int hash2)
+{
+ /* hash1 and hash2 is never the same because
+ * one is between 0 and UNIX_HASH_SIZE - 1, and
+ * another is between UNIX_HASH_SIZE and UNIX_HASH_SIZE * 2.
+ */
+ if (hash1 > hash2)
+ swap(hash1, hash2);
+
+ spin_lock(&unix_table_locks[hash1]);
+ spin_lock_nested(&unix_table_locks[hash2], SINGLE_DEPTH_NESTING);
+}
+
+static void unix_table_double_unlock(unsigned int hash1, unsigned int hash2)
+{
+ spin_unlock(&unix_table_locks[hash1]);
+ spin_unlock(&unix_table_locks[hash2]);
+}
#ifdef CONFIG_SECURITY_NETWORK
static void unix_get_secdata(struct scm_cookie *scm, struct sk_buff *skb)
@@ -164,20 +205,6 @@ static inline bool unix_secdata_eq(struct scm_cookie *scm, struct sk_buff *skb)
}
#endif /* CONFIG_SECURITY_NETWORK */
-/*
- * SMP locking strategy:
- * hash table is protected with spinlock unix_table_lock
- * each socket state is protected by separate spin lock.
- */
-
-static inline unsigned int unix_hash_fold(__wsum n)
-{
- unsigned int hash = (__force unsigned int)csum_fold(n);
-
- hash ^= hash>>8;
- return hash&(UNIX_HASH_SIZE-1);
-}
-
#define unix_peer(sk) (unix_sk(sk)->peer)
static inline int unix_our_peer(struct sock *sk, struct sock *osk)
@@ -214,6 +241,22 @@ struct sock *unix_peer_get(struct sock *s)
}
EXPORT_SYMBOL_GPL(unix_peer_get);
+static struct unix_address *unix_create_addr(struct sockaddr_un *sunaddr,
+ int addr_len)
+{
+ struct unix_address *addr;
+
+ addr = kmalloc(sizeof(*addr) + addr_len, GFP_KERNEL);
+ if (!addr)
+ return NULL;
+
+ refcount_set(&addr->refcnt, 1);
+ addr->len = addr_len;
+ memcpy(addr->name, sunaddr, addr_len);
+
+ return addr;
+}
+
static inline void unix_release_addr(struct unix_address *addr)
{
if (refcount_dec_and_test(&addr->refcnt))
@@ -227,29 +270,29 @@ static inline void unix_release_addr(struct unix_address *addr)
* - if started by zero, it is abstract name.
*/
-static int unix_mkname(struct sockaddr_un *sunaddr, int len, unsigned int *hashp)
+static int unix_validate_addr(struct sockaddr_un *sunaddr, int addr_len)
{
- *hashp = 0;
-
- if (len <= sizeof(short) || len > sizeof(*sunaddr))
+ if (addr_len <= offsetof(struct sockaddr_un, sun_path) ||
+ addr_len > sizeof(*sunaddr))
return -EINVAL;
- if (!sunaddr || sunaddr->sun_family != AF_UNIX)
- return -EINVAL;
- if (sunaddr->sun_path[0]) {
- /*
- * This may look like an off by one error but it is a bit more
- * subtle. 108 is the longest valid AF_UNIX path for a binding.
- * sun_path[108] doesn't as such exist. However in kernel space
- * we are guaranteed that it is a valid memory location in our
- * kernel address buffer.
- */
- ((char *)sunaddr)[len] = 0;
- len = strlen(sunaddr->sun_path)+1+sizeof(short);
- return len;
- }
- *hashp = unix_hash_fold(csum_partial(sunaddr, len, 0));
- return len;
+ if (sunaddr->sun_family != AF_UNIX)
+ return -EINVAL;
+
+ return 0;
+}
+
+static void unix_mkname_bsd(struct sockaddr_un *sunaddr, int addr_len)
+{
+ /* This may look like an off by one error but it is a bit more
+ * subtle. 108 is the longest valid AF_UNIX path for a binding.
+ * sun_path[108] doesn't as such exist. However in kernel space
+ * we are guaranteed that it is a valid memory location in our
+ * kernel address buffer because syscall functions always pass
+ * a pointer of struct sockaddr_storage which has a bigger buffer
+ * than 108.
+ */
+ ((char *)sunaddr)[addr_len] = 0;
}
static void __unix_remove_socket(struct sock *sk)
@@ -257,32 +300,34 @@ static void __unix_remove_socket(struct sock *sk)
sk_del_node_init(sk);
}
-static void __unix_insert_socket(struct hlist_head *list, struct sock *sk)
+static void __unix_insert_socket(struct sock *sk)
{
WARN_ON(!sk_unhashed(sk));
- sk_add_node(sk, list);
+ sk_add_node(sk, &unix_socket_table[sk->sk_hash]);
}
-static void __unix_set_addr(struct sock *sk, struct unix_address *addr,
- unsigned hash)
+static void __unix_set_addr_hash(struct sock *sk, struct unix_address *addr,
+ unsigned int hash)
{
__unix_remove_socket(sk);
smp_store_release(&unix_sk(sk)->addr, addr);
- __unix_insert_socket(&unix_socket_table[hash], sk);
+
+ sk->sk_hash = hash;
+ __unix_insert_socket(sk);
}
-static inline void unix_remove_socket(struct sock *sk)
+static void unix_remove_socket(struct sock *sk)
{
- spin_lock(&unix_table_lock);
+ spin_lock(&unix_table_locks[sk->sk_hash]);
__unix_remove_socket(sk);
- spin_unlock(&unix_table_lock);
+ spin_unlock(&unix_table_locks[sk->sk_hash]);
}
-static inline void unix_insert_socket(struct hlist_head *list, struct sock *sk)
+static void unix_insert_unbound_socket(struct sock *sk)
{
- spin_lock(&unix_table_lock);
- __unix_insert_socket(list, sk);
- spin_unlock(&unix_table_lock);
+ spin_lock(&unix_table_locks[sk->sk_hash]);
+ __unix_insert_socket(sk);
+ spin_unlock(&unix_table_locks[sk->sk_hash]);
}
static struct sock *__unix_find_socket_byname(struct net *net,
@@ -310,32 +355,31 @@ static inline struct sock *unix_find_socket_byname(struct net *net,
{
struct sock *s;
- spin_lock(&unix_table_lock);
+ spin_lock(&unix_table_locks[hash]);
s = __unix_find_socket_byname(net, sunname, len, hash);
if (s)
sock_hold(s);
- spin_unlock(&unix_table_lock);
+ spin_unlock(&unix_table_locks[hash]);
return s;
}
static struct sock *unix_find_socket_byinode(struct inode *i)
{
+ unsigned int hash = unix_bsd_hash(i);
struct sock *s;
- spin_lock(&unix_table_lock);
- sk_for_each(s,
- &unix_socket_table[i->i_ino & (UNIX_HASH_SIZE - 1)]) {
+ spin_lock(&unix_table_locks[hash]);
+ sk_for_each(s, &unix_socket_table[hash]) {
struct dentry *dentry = unix_sk(s)->path.dentry;
if (dentry && d_backing_inode(dentry) == i) {
sock_hold(s);
- goto found;
+ spin_unlock(&unix_table_locks[hash]);
+ return s;
}
}
- s = NULL;
-found:
- spin_unlock(&unix_table_lock);
- return s;
+ spin_unlock(&unix_table_locks[hash]);
+ return NULL;
}
/* Support code for asymmetrically connected dgram sockets
@@ -522,9 +566,7 @@ static void unix_sock_destructor(struct sock *sk)
unix_release_addr(u->addr);
atomic_long_dec(&unix_nr_socks);
- local_bh_disable();
sock_prot_inuse_add(sock_net(sk), sk->sk_prot, -1);
- local_bh_enable();
#ifdef UNIX_REFCNT_DEBUG
pr_debug("UNIX %p is destroyed, %ld are still alive.\n", sk,
atomic_long_read(&unix_nr_socks));
@@ -872,6 +914,7 @@ static struct sock *unix_create1(struct net *net, struct socket *sock, int kern,
sock_init_data(sock, sk);
+ sk->sk_hash = unix_unbound_hash(sk);
sk->sk_allocation = GFP_KERNEL_ACCOUNT;
sk->sk_write_space = unix_write_space;
sk->sk_max_ack_backlog = net->unx.sysctl_max_dgram_qlen;
@@ -887,11 +930,9 @@ static struct sock *unix_create1(struct net *net, struct socket *sock, int kern,
init_waitqueue_head(&u->peer_wait);
init_waitqueue_func_entry(&u->peer_wake, unix_dgram_peer_wake_relay);
memset(&u->scm_stat, 0, sizeof(struct scm_stat));
- unix_insert_socket(unix_sockets_unbound(sk), sk);
+ unix_insert_unbound_socket(sk);
- local_bh_disable();
sock_prot_inuse_add(sock_net(sk), sk->sk_prot, 1);
- local_bh_enable();
return sk;
@@ -952,15 +993,90 @@ static int unix_release(struct socket *sock)
return 0;
}
-static int unix_autobind(struct socket *sock)
+static struct sock *unix_find_bsd(struct net *net, struct sockaddr_un *sunaddr,
+ int addr_len, int type)
{
- struct sock *sk = sock->sk;
- struct net *net = sock_net(sk);
- struct unix_sock *u = unix_sk(sk);
- static u32 ordernum = 1;
- struct unix_address *addr;
+ struct inode *inode;
+ struct path path;
+ struct sock *sk;
int err;
- unsigned int retries = 0;
+
+ unix_mkname_bsd(sunaddr, addr_len);
+ err = kern_path(sunaddr->sun_path, LOOKUP_FOLLOW, &path);
+ if (err)
+ goto fail;
+
+ err = path_permission(&path, MAY_WRITE);
+ if (err)
+ goto path_put;
+
+ err = -ECONNREFUSED;
+ inode = d_backing_inode(path.dentry);
+ if (!S_ISSOCK(inode->i_mode))
+ goto path_put;
+
+ sk = unix_find_socket_byinode(inode);
+ if (!sk)
+ goto path_put;
+
+ err = -EPROTOTYPE;
+ if (sk->sk_type == type)
+ touch_atime(&path);
+ else
+ goto sock_put;
+
+ path_put(&path);
+
+ return sk;
+
+sock_put:
+ sock_put(sk);
+path_put:
+ path_put(&path);
+fail:
+ return ERR_PTR(err);
+}
+
+static struct sock *unix_find_abstract(struct net *net,
+ struct sockaddr_un *sunaddr,
+ int addr_len, int type)
+{
+ unsigned int hash = unix_abstract_hash(sunaddr, addr_len, type);
+ struct dentry *dentry;
+ struct sock *sk;
+
+ sk = unix_find_socket_byname(net, sunaddr, addr_len, hash);
+ if (!sk)
+ return ERR_PTR(-ECONNREFUSED);
+
+ dentry = unix_sk(sk)->path.dentry;
+ if (dentry)
+ touch_atime(&unix_sk(sk)->path);
+
+ return sk;
+}
+
+static struct sock *unix_find_other(struct net *net,
+ struct sockaddr_un *sunaddr,
+ int addr_len, int type)
+{
+ struct sock *sk;
+
+ if (sunaddr->sun_path[0])
+ sk = unix_find_bsd(net, sunaddr, addr_len, type);
+ else
+ sk = unix_find_abstract(net, sunaddr, addr_len, type);
+
+ return sk;
+}
+
+static int unix_autobind(struct sock *sk)
+{
+ unsigned int new_hash, old_hash = sk->sk_hash;
+ struct unix_sock *u = unix_sk(sk);
+ struct unix_address *addr;
+ u32 lastnum, ordernum;
+ int err;
err = mutex_lock_interruptible(&u->bindlock);
if (err)
@@ -970,141 +1086,103 @@ static int unix_autobind(struct socket *sock)
goto out;
err = -ENOMEM;
- addr = kzalloc(sizeof(*addr) + sizeof(short) + 16, GFP_KERNEL);
+ addr = kzalloc(sizeof(*addr) +
+ offsetof(struct sockaddr_un, sun_path) + 16, GFP_KERNEL);
if (!addr)
goto out;
+ addr->len = offsetof(struct sockaddr_un, sun_path) + 6;
addr->name->sun_family = AF_UNIX;
refcount_set(&addr->refcnt, 1);
+ ordernum = prandom_u32();
+ lastnum = ordernum & 0xFFFFF;
retry:
- addr->len = sprintf(addr->name->sun_path+1, "%05x", ordernum) + 1 + sizeof(short);
- addr->hash = unix_hash_fold(csum_partial(addr->name, addr->len, 0));
- addr->hash ^= sk->sk_type;
+ ordernum = (ordernum + 1) & 0xFFFFF;
+ sprintf(addr->name->sun_path + 1, "%05x", ordernum);
- spin_lock(&unix_table_lock);
- ordernum = (ordernum+1)&0xFFFFF;
+ new_hash = unix_abstract_hash(addr->name, addr->len, sk->sk_type);
+ unix_table_double_lock(old_hash, new_hash);
- if (__unix_find_socket_byname(net, addr->name, addr->len, addr->hash)) {
- spin_unlock(&unix_table_lock);
- /*
- * __unix_find_socket_byname() may take long time if many names
+ if (__unix_find_socket_byname(sock_net(sk), addr->name, addr->len,
+ new_hash)) {
+ unix_table_double_unlock(old_hash, new_hash);
+
+ /* __unix_find_socket_byname() may take long time if many names
* are already in use.
*/
cond_resched();
- /* Give up if all names seems to be in use. */
- if (retries++ == 0xFFFFF) {
+
+ if (ordernum == lastnum) {
+ /* Give up if all names seems to be in use. */
err = -ENOSPC;
- kfree(addr);
+ unix_release_addr(addr);
goto out;
}
+
goto retry;
}
- __unix_set_addr(sk, addr, addr->hash);
- spin_unlock(&unix_table_lock);
+ __unix_set_addr_hash(sk, addr, new_hash);
+ unix_table_double_unlock(old_hash, new_hash);
err = 0;
out: mutex_unlock(&u->bindlock);
return err;
}
-static struct sock *unix_find_other(struct net *net,
- struct sockaddr_un *sunname, int len,
- int type, unsigned int hash, int *error)
+static int unix_bind_bsd(struct sock *sk, struct sockaddr_un *sunaddr,
+ int addr_len)
{
- struct sock *u;
- struct path path;
- int err = 0;
-
- if (sunname->sun_path[0]) {
- struct inode *inode;
- err = kern_path(sunname->sun_path, LOOKUP_FOLLOW, &path);
- if (err)
- goto fail;
- inode = d_backing_inode(path.dentry);
- err = path_permission(&path, MAY_WRITE);
- if (err)
- goto put_fail;
-
- err = -ECONNREFUSED;
- if (!S_ISSOCK(inode->i_mode))
- goto put_fail;
- u = unix_find_socket_byinode(inode);
- if (!u)
- goto put_fail;
-
- if (u->sk_type == type)
- touch_atime(&path);
-
- path_put(&path);
-
- err = -EPROTOTYPE;
- if (u->sk_type != type) {
- sock_put(u);
- goto fail;
- }
- } else {
- err = -ECONNREFUSED;
- u = unix_find_socket_byname(net, sunname, len, type ^ hash);
- if (u) {
- struct dentry *dentry;
- dentry = unix_sk(u)->path.dentry;
- if (dentry)
- touch_atime(&unix_sk(u)->path);
- } else
- goto fail;
- }
- return u;
-
-put_fail:
- path_put(&path);
-fail:
- *error = err;
- return NULL;
-}
-
-static int unix_bind_bsd(struct sock *sk, struct unix_address *addr)
-{
- struct unix_sock *u = unix_sk(sk);
umode_t mode = S_IFSOCK |
(SOCK_INODE(sk->sk_socket)->i_mode & ~current_umask());
+ unsigned int new_hash, old_hash = sk->sk_hash;
+ struct unix_sock *u = unix_sk(sk);
struct user_namespace *ns; // barf...
- struct path parent;
+ struct unix_address *addr;
struct dentry *dentry;
- unsigned int hash;
+ struct path parent;
int err;
+ unix_mkname_bsd(sunaddr, addr_len);
+ addr_len = strlen(sunaddr->sun_path) +
+ offsetof(struct sockaddr_un, sun_path) + 1;
+
+ addr = unix_create_addr(sunaddr, addr_len);
+ if (!addr)
+ return -ENOMEM;
+
/*
* Get the parent directory, calculate the hash for last
* component.
*/
dentry = kern_path_create(AT_FDCWD, addr->name->sun_path, &parent, 0);
- if (IS_ERR(dentry))
- return PTR_ERR(dentry);
- ns = mnt_user_ns(parent.mnt);
+ if (IS_ERR(dentry)) {
+ err = PTR_ERR(dentry);
+ goto out;
+ }
/*
* All right, let's create it.
*/
+ ns = mnt_user_ns(parent.mnt);
err = security_path_mknod(&parent, dentry, mode, 0);
if (!err)
err = vfs_mknod(ns, d_inode(parent.dentry), dentry, mode, 0);
if (err)
- goto out;
+ goto out_path;
err = mutex_lock_interruptible(&u->bindlock);
if (err)
goto out_unlink;
if (u->addr)
goto out_unlock;
- addr->hash = UNIX_HASH_SIZE;
- hash = d_backing_inode(dentry)->i_ino & (UNIX_HASH_SIZE - 1);
- spin_lock(&unix_table_lock);
+ new_hash = unix_bsd_hash(d_backing_inode(dentry));
+ unix_table_double_lock(old_hash, new_hash);
u->path.mnt = mntget(parent.mnt);
u->path.dentry = dget(dentry);
- __unix_set_addr(sk, addr, hash);
- spin_unlock(&unix_table_lock);
+ __unix_set_addr_hash(sk, addr, new_hash);
+ unix_table_double_unlock(old_hash, new_hash);
mutex_unlock(&u->bindlock);
done_path_create(&parent, dentry);
return 0;
@@ -1115,74 +1193,76 @@ static int unix_bind_bsd(struct sock *sk, struct unix_address *addr)
out_unlink:
/* failed after successful mknod? unlink what we'd created... */
vfs_unlink(ns, d_inode(parent.dentry), dentry, NULL);
-out:
+out_path:
done_path_create(&parent, dentry);
- return err;
+out:
+ unix_release_addr(addr);
+ return err == -EEXIST ? -EADDRINUSE : err;
}
-static int unix_bind_abstract(struct sock *sk, struct unix_address *addr)
+static int unix_bind_abstract(struct sock *sk, struct sockaddr_un *sunaddr,
+ int addr_len)
{
+ unsigned int new_hash, old_hash = sk->sk_hash;
struct unix_sock *u = unix_sk(sk);
+ struct unix_address *addr;
int err;
+ addr = unix_create_addr(sunaddr, addr_len);
+ if (!addr)
+ return -ENOMEM;
+
err = mutex_lock_interruptible(&u->bindlock);
if (err)
- return err;
+ goto out;
if (u->addr) {
- mutex_unlock(&u->bindlock);
- return -EINVAL;
+ err = -EINVAL;
+ goto out_mutex;
}
- spin_lock(&unix_table_lock);
+ new_hash = unix_abstract_hash(addr->name, addr->len, sk->sk_type);
+ unix_table_double_lock(old_hash, new_hash);
+
if (__unix_find_socket_byname(sock_net(sk), addr->name, addr->len,
- addr->hash)) {
- spin_unlock(&unix_table_lock);
- mutex_unlock(&u->bindlock);
- return -EADDRINUSE;
- }
- __unix_set_addr(sk, addr, addr->hash);
- spin_unlock(&unix_table_lock);
+ new_hash))
+ goto out_spin;
+
+ __unix_set_addr_hash(sk, addr, new_hash);
+ unix_table_double_unlock(old_hash, new_hash);
mutex_unlock(&u->bindlock);
return 0;
+
+out_spin:
+ unix_table_double_unlock(old_hash, new_hash);
+ err = -EADDRINUSE;
+out_mutex:
+ mutex_unlock(&u->bindlock);
+out:
+ unix_release_addr(addr);
+ return err;
}
static int unix_bind(struct socket *sock, struct sockaddr *uaddr, int addr_len)
{
- struct sock *sk = sock->sk;
struct sockaddr_un *sunaddr = (struct sockaddr_un *)uaddr;
- char *sun_path = sunaddr->sun_path;
+ struct sock *sk = sock->sk;
int err;
- unsigned int hash;
- struct unix_address *addr;
- if (addr_len < offsetofend(struct sockaddr_un, sun_family) ||
- sunaddr->sun_family != AF_UNIX)
- return -EINVAL;
+ if (addr_len == offsetof(struct sockaddr_un, sun_path) &&
+ sunaddr->sun_family == AF_UNIX)
+ return unix_autobind(sk);
- if (addr_len == sizeof(short))
- return unix_autobind(sock);
-
- err = unix_mkname(sunaddr, addr_len, &hash);
- if (err < 0)
- return err;
- addr_len = err;
- addr = kmalloc(sizeof(*addr)+addr_len, GFP_KERNEL);
- if (!addr)
- return -ENOMEM;
-
- memcpy(addr->name, sunaddr, addr_len);
- addr->len = addr_len;
- addr->hash = hash ^ sk->sk_type;
- refcount_set(&addr->refcnt, 1);
-
- if (sun_path[0])
- err = unix_bind_bsd(sk, addr);
- else
- err = unix_bind_abstract(sk, addr);
+ err = unix_validate_addr(sunaddr, addr_len);
if (err)
- unix_release_addr(addr);
- return err == -EEXIST ? -EADDRINUSE : err;
+ return err;
+
+ if (sunaddr->sun_path[0])
+ err = unix_bind_bsd(sk, sunaddr, addr_len);
+ else
+ err = unix_bind_abstract(sk, sunaddr, addr_len);
+
+ return err;
}
static void unix_state_double_lock(struct sock *sk1, struct sock *sk2)
@@ -1217,7 +1297,6 @@ static int unix_dgram_connect(struct socket *sock, struct sockaddr *addr,
struct net *net = sock_net(sk);
struct sockaddr_un *sunaddr = (struct sockaddr_un *)addr;
struct sock *other;
- unsigned int hash;
int err;
err = -EINVAL;
@@ -1225,19 +1304,23 @@ static int unix_dgram_connect(struct socket *sock, struct sockaddr *addr,
goto out;
if (addr->sa_family != AF_UNSPEC) {
- err = unix_mkname(sunaddr, alen, &hash);
- if (err < 0)
+ err = unix_validate_addr(sunaddr, alen);
+ if (err)
goto out;
- alen = err;
if (test_bit(SOCK_PASSCRED, &sock->flags) &&
- !unix_sk(sk)->addr && (err = unix_autobind(sock)) != 0)
- goto out;
+ !unix_sk(sk)->addr) {
+ err = unix_autobind(sk);
+ if (err)
+ goto out;
+ }
restart:
- other = unix_find_other(net, sunaddr, alen, sock->type, hash, &err);
- if (!other)
+ other = unix_find_other(net, sunaddr, alen, sock->type);
+ if (IS_ERR(other)) {
+ err = PTR_ERR(other);
goto out;
+ }
unix_state_double_lock(sk, other);
@@ -1327,19 +1410,19 @@ static int unix_stream_connect(struct socket *sock, struct sockaddr *uaddr,
struct sock *newsk = NULL;
struct sock *other = NULL;
struct sk_buff *skb = NULL;
- unsigned int hash;
int st;
int err;
long timeo;
- err = unix_mkname(sunaddr, addr_len, &hash);
- if (err < 0)
+ err = unix_validate_addr(sunaddr, addr_len);
+ if (err)
goto out;
- addr_len = err;
- if (test_bit(SOCK_PASSCRED, &sock->flags) && !u->addr &&
- (err = unix_autobind(sock)) != 0)
- goto out;
+ if (test_bit(SOCK_PASSCRED, &sock->flags) && !u->addr) {
+ err = unix_autobind(sk);
+ if (err)
+ goto out;
+ }
timeo = sock_sndtimeo(sk, flags & O_NONBLOCK);
@@ -1365,9 +1448,12 @@ static int unix_stream_connect(struct socket *sock, struct sockaddr *uaddr,
restart:
/* Find listening sock. */
- other = unix_find_other(net, sunaddr, addr_len, sk->sk_type, hash, &err);
- if (!other)
+ other = unix_find_other(net, sunaddr, addr_len, sk->sk_type);
+ if (IS_ERR(other)) {
+ err = PTR_ERR(other);
+ other = NULL;
goto out;
+ }
/* Latch state of peer */
unix_state_lock(other);
@@ -1455,9 +1541,9 @@ static int unix_stream_connect(struct socket *sock, struct sockaddr *uaddr,
*
* The contents of *(otheru->addr) and otheru->path
* are seen fully set up here, since we have found
- * otheru in hash under unix_table_lock. Insertion
+ * otheru in hash under unix_table_locks. Insertion
* into the hash chain we'd found it in had been done
- * in an earlier critical area protected by unix_table_lock,
+ * in an earlier critical area protected by unix_table_locks,
* the same one where we'd set *(otheru->addr) contents,
* as well as otheru->path and otheru->addr itself.
*
@@ -1604,7 +1690,7 @@ static int unix_getname(struct socket *sock, struct sockaddr *uaddr, int peer)
if (!addr) {
sunaddr->sun_family = AF_UNIX;
sunaddr->sun_path[0] = 0;
- err = sizeof(short);
+ err = offsetof(struct sockaddr_un, sun_path);
} else {
err = addr->len;
memcpy(sunaddr, addr->name, addr->len);
@@ -1760,9 +1846,7 @@ static int unix_dgram_sendmsg(struct socket *sock, struct msghdr *msg,
struct unix_sock *u = unix_sk(sk);
DECLARE_SOCKADDR(struct sockaddr_un *, sunaddr, msg->msg_name);
struct sock *other = NULL;
- int namelen = 0; /* fake GCC */
int err;
- unsigned int hash;
struct sk_buff *skb;
long timeo;
struct scm_cookie scm;
@@ -1779,10 +1863,9 @@ static int unix_dgram_sendmsg(struct socket *sock, struct msghdr *msg,
goto out;
if (msg->msg_namelen) {
- err = unix_mkname(sunaddr, msg->msg_namelen, &hash);
- if (err < 0)
+ err = unix_validate_addr(sunaddr, msg->msg_namelen);
+ if (err)
goto out;
- namelen = err;
} else {
sunaddr = NULL;
err = -ENOTCONN;
@@ -1791,9 +1874,11 @@ static int unix_dgram_sendmsg(struct socket *sock, struct msghdr *msg,
goto out;
}
- if (test_bit(SOCK_PASSCRED, &sock->flags) && !u->addr
- && (err = unix_autobind(sock)) != 0)
- goto out;
+ if (test_bit(SOCK_PASSCRED, &sock->flags) && !u->addr) {
+ err = unix_autobind(sk);
+ if (err)
+ goto out;
+ }
err = -EMSGSIZE;
if (len > sk->sk_sndbuf - 32)
@@ -1833,10 +1918,13 @@ static int unix_dgram_sendmsg(struct socket *sock, struct msghdr *msg,
if (sunaddr == NULL)
goto out_free;
- other = unix_find_other(net, sunaddr, namelen, sk->sk_type,
- hash, &err);
- if (other == NULL)
+ other = unix_find_other(net, sunaddr, msg->msg_namelen,
+ sk->sk_type);
+ if (IS_ERR(other)) {
+ err = PTR_ERR(other);
+ other = NULL;
goto out_free;
+ }
}
if (sk_filter(other, skb) < 0) {
@@ -3132,7 +3220,7 @@ static __poll_t unix_dgram_poll(struct file *file, struct socket *sock,
#define BUCKET_SPACE (BITS_PER_LONG - (UNIX_HASH_BITS + 1) - 1)
#define get_bucket(x) ((x) >> BUCKET_SPACE)
-#define get_offset(x) ((x) & ((1L << BUCKET_SPACE) - 1))
+#define get_offset(x) ((x) & ((1UL << BUCKET_SPACE) - 1))
#define set_bucket_offset(b, o) ((b) << BUCKET_SPACE | (o))
static struct sock *unix_from_bucket(struct seq_file *seq, loff_t *pos)
@@ -3156,7 +3244,7 @@ static struct sock *unix_next_socket(struct seq_file *seq,
struct sock *sk,
loff_t *pos)
{
- unsigned long bucket;
+ unsigned long bucket = get_bucket(*pos);
while (sk > (struct sock *)SEQ_START_TOKEN) {
sk = sk_next(sk);
@@ -3167,12 +3255,13 @@ static struct sock *unix_next_socket(struct seq_file *seq,
}
do {
+ spin_lock(&unix_table_locks[bucket]);
sk = unix_from_bucket(seq, pos);
if (sk)
return sk;
next_bucket:
- bucket = get_bucket(*pos) + 1;
+ spin_unlock(&unix_table_locks[bucket++]);
*pos = set_bucket_offset(bucket, 1);
} while (bucket < ARRAY_SIZE(unix_socket_table));
@@ -3180,10 +3269,7 @@ static struct sock *unix_next_socket(struct seq_file *seq,
}
static void *unix_seq_start(struct seq_file *seq, loff_t *pos)
- __acquires(unix_table_lock)
{
- spin_lock(&unix_table_lock);
-
if (!*pos)
return SEQ_START_TOKEN;
@@ -3200,9 +3286,11 @@ static void *unix_seq_next(struct seq_file *seq, void *v, loff_t *pos)
}
static void unix_seq_stop(struct seq_file *seq, void *v)
- __releases(unix_table_lock)
{
- spin_unlock(&unix_table_lock);
+ struct sock *sk = v;
+
+ if (sk)
+ spin_unlock(&unix_table_locks[sk->sk_hash]);
}
static int unix_seq_show(struct seq_file *seq, void *v)
@@ -3227,15 +3315,16 @@ static int unix_seq_show(struct seq_file *seq, void *v)
(s->sk_state == TCP_ESTABLISHED ? SS_CONNECTING : SS_DISCONNECTING),
sock_i_ino(s));
- if (u->addr) { // under unix_table_lock here
+ if (u->addr) { // under unix_table_locks here
int i, len;
seq_putc(seq, ' ');
i = 0;
- len = u->addr->len - sizeof(short);
- if (!UNIX_ABSTRACT(s))
+ len = u->addr->len -
+ offsetof(struct sockaddr_un, sun_path);
+ if (u->addr->name->sun_path[0]) {
len--;
- else {
+ } else {
seq_putc(seq, '@');
i++;
}
@@ -3385,10 +3474,13 @@ static void __init bpf_iter_register(void)
static int __init af_unix_init(void)
{
- int rc = -1;
+ int i, rc = -1;
BUILD_BUG_ON(sizeof(struct unix_skb_parms) > sizeof_field(struct sk_buff, cb));
+ for (i = 0; i < 2 * UNIX_HASH_SIZE; i++)
+ spin_lock_init(&unix_table_locks[i]);
+
rc = proto_register(&unix_dgram_proto, 1);
if (rc != 0) {
pr_crit("%s: Cannot create unix_sock SLAB cache!\n", __func__);
diff --git a/net/unix/diag.c b/net/unix/diag.c
index 7e7d7f4..bb0b5ea 100644
--- a/net/unix/diag.c
+++ b/net/unix/diag.c
@@ -13,13 +13,14 @@
static int sk_diag_dump_name(struct sock *sk, struct sk_buff *nlskb)
{
- /* might or might not have unix_table_lock */
+ /* might or might not have unix_table_locks */
struct unix_address *addr = smp_load_acquire(&unix_sk(sk)->addr);
if (!addr)
return 0;
- return nla_put(nlskb, UNIX_DIAG_NAME, addr->len - sizeof(short),
+ return nla_put(nlskb, UNIX_DIAG_NAME,
+ addr->len - offsetof(struct sockaddr_un, sun_path),
addr->name->sun_path);
}
@@ -203,13 +204,13 @@ static int unix_diag_dump(struct sk_buff *skb, struct netlink_callback *cb)
s_slot = cb->args[0];
num = s_num = cb->args[1];
- spin_lock(&unix_table_lock);
for (slot = s_slot;
slot < ARRAY_SIZE(unix_socket_table);
s_num = 0, slot++) {
struct sock *sk;
num = 0;
+ spin_lock(&unix_table_locks[slot]);
sk_for_each(sk, &unix_socket_table[slot]) {
if (!net_eq(sock_net(sk), net))
continue;
@@ -220,14 +221,16 @@ static int unix_diag_dump(struct sk_buff *skb, struct netlink_callback *cb)
if (sk_diag_dump(sk, skb, req,
NETLINK_CB(cb->skb).portid,
cb->nlh->nlmsg_seq,
- NLM_F_MULTI) < 0)
+ NLM_F_MULTI) < 0) {
+ spin_unlock(&unix_table_locks[slot]);
goto done;
+ }
next:
num++;
}
+ spin_unlock(&unix_table_locks[slot]);
}
done:
- spin_unlock(&unix_table_lock);
cb->args[0] = slot;
cb->args[1] = num;
@@ -236,21 +239,19 @@ static int unix_diag_dump(struct sk_buff *skb, struct netlink_callback *cb)
static struct sock *unix_lookup_by_ino(unsigned int ino)
{
- int i;
struct sock *sk;
+ int i;
- spin_lock(&unix_table_lock);
for (i = 0; i < ARRAY_SIZE(unix_socket_table); i++) {
+ spin_lock(&unix_table_locks[i]);
sk_for_each(sk, &unix_socket_table[i])
if (ino == sock_i_ino(sk)) {
sock_hold(sk);
- spin_unlock(&unix_table_lock);
-
+ spin_unlock(&unix_table_locks[i]);
return sk;
}
+ spin_unlock(&unix_table_locks[i]);
}
-
- spin_unlock(&unix_table_lock);
return NULL;
}
diff --git a/net/unix/sysctl_net_unix.c b/net/unix/sysctl_net_unix.c
index c09bea8..01d44e2 100644
--- a/net/unix/sysctl_net_unix.c
+++ b/net/unix/sysctl_net_unix.c
@@ -30,10 +30,6 @@ int __net_init unix_sysctl_register(struct net *net)
if (table == NULL)
goto err_alloc;
- /* Don't export sysctls to unprivileged users */
- if (net->user_ns != &init_user_ns)
- table[0].procname = NULL;
-
table[0].data = &net->unx.sysctl_max_dgram_qlen;
net->unx.ctl = register_net_sysctl(net, "net/unix", table);
if (net->unx.ctl == NULL)
diff --git a/net/vmw_vsock/af_vsock.c b/net/vmw_vsock/af_vsock.c
index ed0df83..3235261 100644
--- a/net/vmw_vsock/af_vsock.c
+++ b/net/vmw_vsock/af_vsock.c
@@ -85,6 +85,7 @@
* TCP_LISTEN - listening
*/
+#include <linux/compat.h>
#include <linux/types.h>
#include <linux/bitops.h>
#include <linux/cred.h>
diff --git a/net/vmw_vsock/hyperv_transport.c b/net/vmw_vsock/hyperv_transport.c
index 19189cf..e111e13 100644
--- a/net/vmw_vsock/hyperv_transport.c
+++ b/net/vmw_vsock/hyperv_transport.c
@@ -225,14 +225,20 @@ static size_t hvs_channel_writable_bytes(struct vmbus_channel *chan)
return round_down(ret, 8);
}
+static int __hvs_send_data(struct vmbus_channel *chan,
+ struct vmpipe_proto_header *hdr,
+ size_t to_write)
+{
+ hdr->pkt_type = 1;
+ hdr->data_size = to_write;
+ return vmbus_sendpacket(chan, hdr, sizeof(*hdr) + to_write,
+ 0, VM_PKT_DATA_INBAND, 0);
+}
+
static int hvs_send_data(struct vmbus_channel *chan,
struct hvs_send_buf *send_buf, size_t to_write)
{
- send_buf->hdr.pkt_type = 1;
- send_buf->hdr.data_size = to_write;
- return vmbus_sendpacket(chan, &send_buf->hdr,
- sizeof(send_buf->hdr) + to_write,
- 0, VM_PKT_DATA_INBAND, 0);
+ return __hvs_send_data(chan, &send_buf->hdr, to_write);
}
static void hvs_channel_cb(void *ctx)
@@ -468,7 +474,7 @@ static void hvs_shutdown_lock_held(struct hvsock *hvs, int mode)
return;
/* It can't fail: see hvs_channel_writable_bytes(). */
- (void)hvs_send_data(hvs->chan, (struct hvs_send_buf *)&hdr, 0);
+ (void)__hvs_send_data(hvs->chan, &hdr, 0);
hvs->fin_sent = true;
}
diff --git a/net/wireless/chan.c b/net/wireless/chan.c
index 869c43d..eb82205 100644
--- a/net/wireless/chan.c
+++ b/net/wireless/chan.c
@@ -245,19 +245,7 @@ bool cfg80211_chandef_valid(const struct cfg80211_chan_def *chandef)
oper_freq - MHZ_TO_KHZ(oper_width) / 2)
return false;
break;
- case NL80211_CHAN_WIDTH_40:
- if (chandef->center_freq1 != control_freq + 10 &&
- chandef->center_freq1 != control_freq - 10)
- return false;
- if (chandef->center_freq2)
- return false;
- break;
case NL80211_CHAN_WIDTH_80P80:
- if (chandef->center_freq1 != control_freq + 30 &&
- chandef->center_freq1 != control_freq + 10 &&
- chandef->center_freq1 != control_freq - 10 &&
- chandef->center_freq1 != control_freq - 30)
- return false;
if (!chandef->center_freq2)
return false;
/* adjacent is not allowed -- that's a 160 MHz channel */
@@ -265,28 +253,42 @@ bool cfg80211_chandef_valid(const struct cfg80211_chan_def *chandef)
chandef->center_freq2 - chandef->center_freq1 == 80)
return false;
break;
- case NL80211_CHAN_WIDTH_80:
- if (chandef->center_freq1 != control_freq + 30 &&
- chandef->center_freq1 != control_freq + 10 &&
- chandef->center_freq1 != control_freq - 10 &&
- chandef->center_freq1 != control_freq - 30)
- return false;
+ default:
if (chandef->center_freq2)
return false;
break;
+ }
+
+ switch (chandef->width) {
+ case NL80211_CHAN_WIDTH_5:
+ case NL80211_CHAN_WIDTH_10:
+ case NL80211_CHAN_WIDTH_20:
+ case NL80211_CHAN_WIDTH_20_NOHT:
+ case NL80211_CHAN_WIDTH_1:
+ case NL80211_CHAN_WIDTH_2:
+ case NL80211_CHAN_WIDTH_4:
+ case NL80211_CHAN_WIDTH_8:
+ case NL80211_CHAN_WIDTH_16:
+ /* all checked above */
+ break;
case NL80211_CHAN_WIDTH_160:
- if (chandef->center_freq1 != control_freq + 70 &&
- chandef->center_freq1 != control_freq + 50 &&
- chandef->center_freq1 != control_freq + 30 &&
- chandef->center_freq1 != control_freq + 10 &&
- chandef->center_freq1 != control_freq - 10 &&
- chandef->center_freq1 != control_freq - 30 &&
- chandef->center_freq1 != control_freq - 50 &&
- chandef->center_freq1 != control_freq - 70)
- return false;
- if (chandef->center_freq2)
- return false;
- break;
+ if (chandef->center_freq1 == control_freq + 70 ||
+ chandef->center_freq1 == control_freq + 50 ||
+ chandef->center_freq1 == control_freq - 50 ||
+ chandef->center_freq1 == control_freq - 70)
+ break;
+ fallthrough;
+ case NL80211_CHAN_WIDTH_80P80:
+ case NL80211_CHAN_WIDTH_80:
+ if (chandef->center_freq1 == control_freq + 30 ||
+ chandef->center_freq1 == control_freq - 30)
+ break;
+ fallthrough;
+ case NL80211_CHAN_WIDTH_40:
+ if (chandef->center_freq1 == control_freq + 10 ||
+ chandef->center_freq1 == control_freq - 10)
+ break;
+ fallthrough;
default:
return false;
}
@@ -712,6 +714,19 @@ static bool cfg80211_is_wiphy_oper_chan(struct wiphy *wiphy,
return false;
}
+static bool
+cfg80211_offchan_chain_is_active(struct cfg80211_registered_device *rdev,
+ struct ieee80211_channel *channel)
+{
+ if (!rdev->background_radar_wdev)
+ return false;
+
+ if (!cfg80211_chandef_valid(&rdev->background_radar_chandef))
+ return false;
+
+ return cfg80211_is_sub_chan(&rdev->background_radar_chandef, channel);
+}
+
bool cfg80211_any_wiphy_oper_chan(struct wiphy *wiphy,
struct ieee80211_channel *chan)
{
@@ -728,6 +743,9 @@ bool cfg80211_any_wiphy_oper_chan(struct wiphy *wiphy,
if (cfg80211_is_wiphy_oper_chan(&rdev->wiphy, chan))
return true;
+
+ if (cfg80211_offchan_chain_is_active(rdev, chan))
+ return true;
}
return false;
diff --git a/net/wireless/core.c b/net/wireless/core.c
index eb297e1..3a54c8e 100644
--- a/net/wireless/core.c
+++ b/net/wireless/core.c
@@ -545,6 +545,10 @@ struct wiphy *wiphy_new_nm(const struct cfg80211_ops *ops, int sizeof_priv,
INIT_WORK(&rdev->rfkill_block, cfg80211_rfkill_block_work);
INIT_WORK(&rdev->conn_work, cfg80211_conn_work);
INIT_WORK(&rdev->event_work, cfg80211_event_work);
+ INIT_WORK(&rdev->background_cac_abort_wk,
+ cfg80211_background_cac_abort_wk);
+ INIT_DELAYED_WORK(&rdev->background_cac_done_wk,
+ cfg80211_background_cac_done_wk);
init_waitqueue_head(&rdev->dev_wait);
@@ -733,6 +737,7 @@ int wiphy_register(struct wiphy *wiphy)
if (wiphy->interface_modes & ~(BIT(NL80211_IFTYPE_STATION) |
BIT(NL80211_IFTYPE_P2P_CLIENT) |
BIT(NL80211_IFTYPE_AP) |
+ BIT(NL80211_IFTYPE_MESH_POINT) |
BIT(NL80211_IFTYPE_P2P_GO) |
BIT(NL80211_IFTYPE_ADHOC) |
BIT(NL80211_IFTYPE_P2P_DEVICE) |
@@ -1054,11 +1059,13 @@ void wiphy_unregister(struct wiphy *wiphy)
cancel_work_sync(&rdev->conn_work);
flush_work(&rdev->event_work);
cancel_delayed_work_sync(&rdev->dfs_update_channels_wk);
+ cancel_delayed_work_sync(&rdev->background_cac_done_wk);
flush_work(&rdev->destroy_work);
flush_work(&rdev->sched_scan_stop_wk);
flush_work(&rdev->propagate_radar_detect_wk);
flush_work(&rdev->propagate_cac_done_wk);
flush_work(&rdev->mgmt_registrations_update_wk);
+ flush_work(&rdev->background_cac_abort_wk);
#ifdef CONFIG_PM
if (rdev->wiphy.wowlan_config && rdev->ops->set_wakeup)
@@ -1207,6 +1214,8 @@ void __cfg80211_leave(struct cfg80211_registered_device *rdev,
cfg80211_pmsr_wdev_down(wdev);
+ cfg80211_stop_background_radar_detection(wdev);
+
switch (wdev->iftype) {
case NL80211_IFTYPE_ADHOC:
__cfg80211_leave_ibss(rdev, dev, true);
diff --git a/net/wireless/core.h b/net/wireless/core.h
index 1720abf..3a7dbd6 100644
--- a/net/wireless/core.h
+++ b/net/wireless/core.h
@@ -84,6 +84,11 @@ struct cfg80211_registered_device {
struct delayed_work dfs_update_channels_wk;
+ struct wireless_dev *background_radar_wdev;
+ struct cfg80211_chan_def background_radar_chandef;
+ struct delayed_work background_cac_done_wk;
+ struct work_struct background_cac_abort_wk;
+
/* netlink port which started critical protocol (0 means not started) */
u32 crit_proto_nlportid;
@@ -491,6 +496,17 @@ cfg80211_chandef_dfs_cac_time(struct wiphy *wiphy,
void cfg80211_sched_dfs_chan_update(struct cfg80211_registered_device *rdev);
+int
+cfg80211_start_background_radar_detection(struct cfg80211_registered_device *rdev,
+ struct wireless_dev *wdev,
+ struct cfg80211_chan_def *chandef);
+
+void cfg80211_stop_background_radar_detection(struct wireless_dev *wdev);
+
+void cfg80211_background_cac_done_wk(struct work_struct *work);
+
+void cfg80211_background_cac_abort_wk(struct work_struct *work);
+
bool cfg80211_any_wiphy_oper_chan(struct wiphy *wiphy,
struct ieee80211_channel *chan);
diff --git a/net/wireless/mlme.c b/net/wireless/mlme.c
index 783acd2..c8155a4 100644
--- a/net/wireless/mlme.c
+++ b/net/wireless/mlme.c
@@ -905,13 +905,13 @@ void cfg80211_dfs_channels_update_work(struct work_struct *work)
}
-void cfg80211_radar_event(struct wiphy *wiphy,
- struct cfg80211_chan_def *chandef,
- gfp_t gfp)
+void __cfg80211_radar_event(struct wiphy *wiphy,
+ struct cfg80211_chan_def *chandef,
+ bool offchan, gfp_t gfp)
{
struct cfg80211_registered_device *rdev = wiphy_to_rdev(wiphy);
- trace_cfg80211_radar_event(wiphy, chandef);
+ trace_cfg80211_radar_event(wiphy, chandef, offchan);
/* only set the chandef supplied channel to unavailable, in
* case the radar is detected on only one of multiple channels
@@ -919,6 +919,9 @@ void cfg80211_radar_event(struct wiphy *wiphy,
*/
cfg80211_set_dfs_state(wiphy, chandef, NL80211_DFS_UNAVAILABLE);
+ if (offchan)
+ queue_work(cfg80211_wq, &rdev->background_cac_abort_wk);
+
cfg80211_sched_dfs_chan_update(rdev);
nl80211_radar_notify(rdev, chandef, NL80211_RADAR_DETECTED, NULL, gfp);
@@ -926,7 +929,7 @@ void cfg80211_radar_event(struct wiphy *wiphy,
memcpy(&rdev->radar_chandef, chandef, sizeof(struct cfg80211_chan_def));
queue_work(cfg80211_wq, &rdev->propagate_radar_detect_wk);
}
-EXPORT_SYMBOL(cfg80211_radar_event);
+EXPORT_SYMBOL(__cfg80211_radar_event);
void cfg80211_cac_event(struct net_device *netdev,
const struct cfg80211_chan_def *chandef,
@@ -970,3 +973,143 @@ void cfg80211_cac_event(struct net_device *netdev,
nl80211_radar_notify(rdev, chandef, event, netdev, gfp);
}
EXPORT_SYMBOL(cfg80211_cac_event);
+
+static void
+__cfg80211_background_cac_event(struct cfg80211_registered_device *rdev,
+ struct wireless_dev *wdev,
+ const struct cfg80211_chan_def *chandef,
+ enum nl80211_radar_event event)
+{
+ struct wiphy *wiphy = &rdev->wiphy;
+ struct net_device *netdev;
+
+ lockdep_assert_wiphy(&rdev->wiphy);
+
+ if (!cfg80211_chandef_valid(chandef))
+ return;
+
+ if (!rdev->background_radar_wdev)
+ return;
+
+ switch (event) {
+ case NL80211_RADAR_CAC_FINISHED:
+ cfg80211_set_dfs_state(wiphy, chandef, NL80211_DFS_AVAILABLE);
+ memcpy(&rdev->cac_done_chandef, chandef, sizeof(*chandef));
+ queue_work(cfg80211_wq, &rdev->propagate_cac_done_wk);
+ cfg80211_sched_dfs_chan_update(rdev);
+ wdev = rdev->background_radar_wdev;
+ break;
+ case NL80211_RADAR_CAC_ABORTED:
+ if (!cancel_delayed_work(&rdev->background_cac_done_wk))
+ return;
+ wdev = rdev->background_radar_wdev;
+ break;
+ case NL80211_RADAR_CAC_STARTED:
+ break;
+ default:
+ return;
+ }
+
+ netdev = wdev ? wdev->netdev : NULL;
+ nl80211_radar_notify(rdev, chandef, event, netdev, GFP_KERNEL);
+}
+
+static void
+cfg80211_background_cac_event(struct cfg80211_registered_device *rdev,
+ const struct cfg80211_chan_def *chandef,
+ enum nl80211_radar_event event)
+{
+ wiphy_lock(&rdev->wiphy);
+ __cfg80211_background_cac_event(rdev, rdev->background_radar_wdev,
+ chandef, event);
+ wiphy_unlock(&rdev->wiphy);
+}
+
+void cfg80211_background_cac_done_wk(struct work_struct *work)
+{
+ struct delayed_work *delayed_work = to_delayed_work(work);
+ struct cfg80211_registered_device *rdev;
+
+ rdev = container_of(delayed_work, struct cfg80211_registered_device,
+ background_cac_done_wk);
+ cfg80211_background_cac_event(rdev, &rdev->background_radar_chandef,
+ NL80211_RADAR_CAC_FINISHED);
+}
+
+void cfg80211_background_cac_abort_wk(struct work_struct *work)
+{
+ struct cfg80211_registered_device *rdev;
+
+ rdev = container_of(work, struct cfg80211_registered_device,
+ background_cac_abort_wk);
+ cfg80211_background_cac_event(rdev, &rdev->background_radar_chandef,
+ NL80211_RADAR_CAC_ABORTED);
+}
+
+void cfg80211_background_cac_abort(struct wiphy *wiphy)
+{
+ struct cfg80211_registered_device *rdev = wiphy_to_rdev(wiphy);
+
+ queue_work(cfg80211_wq, &rdev->background_cac_abort_wk);
+}
+EXPORT_SYMBOL(cfg80211_background_cac_abort);
+
+int
+cfg80211_start_background_radar_detection(struct cfg80211_registered_device *rdev,
+ struct wireless_dev *wdev,
+ struct cfg80211_chan_def *chandef)
+{
+ unsigned int cac_time_ms;
+ int err;
+
+ lockdep_assert_wiphy(&rdev->wiphy);
+
+ if (!wiphy_ext_feature_isset(&rdev->wiphy,
+ NL80211_EXT_FEATURE_RADAR_BACKGROUND))
+ return -EOPNOTSUPP;
+
+ /* Offchannel chain already locked by another wdev */
+ if (rdev->background_radar_wdev && rdev->background_radar_wdev != wdev)
+ return -EBUSY;
+
+ /* CAC already in progress on the offchannel chain */
+ if (rdev->background_radar_wdev == wdev &&
+ delayed_work_pending(&rdev->background_cac_done_wk))
+ return -EBUSY;
+
+ err = rdev_set_radar_background(rdev, chandef);
+ if (err)
+ return err;
+
+ cac_time_ms = cfg80211_chandef_dfs_cac_time(&rdev->wiphy, chandef);
+ if (!cac_time_ms)
+ cac_time_ms = IEEE80211_DFS_MIN_CAC_TIME_MS;
+
+ rdev->background_radar_chandef = *chandef;
+ rdev->background_radar_wdev = wdev; /* Get offchain ownership */
+
+ __cfg80211_background_cac_event(rdev, wdev, chandef,
+ NL80211_RADAR_CAC_STARTED);
+ queue_delayed_work(cfg80211_wq, &rdev->background_cac_done_wk,
+ msecs_to_jiffies(cac_time_ms));
+
+ return 0;
+}
+
+void cfg80211_stop_background_radar_detection(struct wireless_dev *wdev)
+{
+ struct wiphy *wiphy = wdev->wiphy;
+ struct cfg80211_registered_device *rdev = wiphy_to_rdev(wiphy);
+
+ lockdep_assert_wiphy(wiphy);
+
+ if (wdev != rdev->background_radar_wdev)
+ return;
+
+ rdev_set_radar_background(rdev, NULL);
+ rdev->background_radar_wdev = NULL; /* Release offchain ownership */
+
+ __cfg80211_background_cac_event(rdev, wdev,
+ &rdev->background_radar_chandef,
+ NL80211_RADAR_CAC_ABORTED);
+}
diff --git a/net/wireless/nl80211.c b/net/wireless/nl80211.c
index a27b3b5..578bff9 100644
--- a/net/wireless/nl80211.c
+++ b/net/wireless/nl80211.c
@@ -776,6 +776,8 @@ static const struct nla_policy nl80211_policy[NUM_NL80211_ATTR] = {
[NL80211_ATTR_MBSSID_CONFIG] =
NLA_POLICY_NESTED(nl80211_mbssid_config_policy),
[NL80211_ATTR_MBSSID_ELEMS] = { .type = NLA_NESTED },
+ [NL80211_ATTR_RADAR_BACKGROUND] = { .type = NLA_FLAG },
+ [NL80211_ATTR_AP_SETTINGS_FLAGS] = { .type = NLA_U32 },
};
/* policy for the key attributes */
@@ -3669,14 +3671,16 @@ static int nl80211_send_iface(struct sk_buff *msg, u32 portid, u32 seq, int flag
case NL80211_IFTYPE_STATION:
case NL80211_IFTYPE_P2P_CLIENT:
case NL80211_IFTYPE_ADHOC: {
- const u8 *ssid_ie;
+ const struct element *ssid_elem;
+
if (!wdev->current_bss)
break;
rcu_read_lock();
- ssid_ie = ieee80211_bss_get_ie(&wdev->current_bss->pub,
- WLAN_EID_SSID);
- if (ssid_ie &&
- nla_put(msg, NL80211_ATTR_SSID, ssid_ie[1], ssid_ie + 2))
+ ssid_elem = ieee80211_bss_get_elem(&wdev->current_bss->pub,
+ WLAN_EID_SSID);
+ if (ssid_elem &&
+ nla_put(msg, NL80211_ATTR_SSID, ssid_elem->datalen,
+ ssid_elem->data))
goto nla_put_failure_rcu_locked;
rcu_read_unlock();
break;
@@ -5711,8 +5715,11 @@ static int nl80211_start_ap(struct sk_buff *skb, struct genl_info *info)
nl80211_calculate_ap_params(params);
- if (info->attrs[NL80211_ATTR_EXTERNAL_AUTH_SUPPORT])
- params->flags |= AP_SETTINGS_EXTERNAL_AUTH_SUPPORT;
+ if (info->attrs[NL80211_ATTR_AP_SETTINGS_FLAGS])
+ params->flags = nla_get_u32(
+ info->attrs[NL80211_ATTR_AP_SETTINGS_FLAGS]);
+ else if (info->attrs[NL80211_ATTR_EXTERNAL_AUTH_SUPPORT])
+ params->flags |= NL80211_AP_SETTINGS_EXTERNAL_AUTH_SUPPORT;
wdev_lock(wdev);
err = rdev_start_ap(rdev, dev, params);
@@ -9274,38 +9281,60 @@ static int nl80211_start_radar_detection(struct sk_buff *skb,
struct cfg80211_chan_def chandef;
enum nl80211_dfs_regions dfs_region;
unsigned int cac_time_ms;
- int err;
+ int err = -EINVAL;
+
+ flush_delayed_work(&rdev->dfs_update_channels_wk);
+
+ wiphy_lock(wiphy);
dfs_region = reg_get_dfs_region(wiphy);
if (dfs_region == NL80211_DFS_UNSET)
- return -EINVAL;
+ goto unlock;
err = nl80211_parse_chandef(rdev, info, &chandef);
if (err)
- return err;
-
- if (netif_carrier_ok(dev))
- return -EBUSY;
-
- if (wdev->cac_started)
- return -EBUSY;
+ goto unlock;
err = cfg80211_chandef_dfs_required(wiphy, &chandef, wdev->iftype);
if (err < 0)
- return err;
+ goto unlock;
- if (err == 0)
- return -EINVAL;
+ if (err == 0) {
+ err = -EINVAL;
+ goto unlock;
+ }
- if (!cfg80211_chandef_dfs_usable(wiphy, &chandef))
- return -EINVAL;
+ if (!cfg80211_chandef_dfs_usable(wiphy, &chandef)) {
+ err = -EINVAL;
+ goto unlock;
+ }
+
+ if (nla_get_flag(info->attrs[NL80211_ATTR_RADAR_BACKGROUND])) {
+ err = cfg80211_start_background_radar_detection(rdev, wdev,
+ &chandef);
+ goto unlock;
+ }
+
+ if (netif_carrier_ok(dev)) {
+ err = -EBUSY;
+ goto unlock;
+ }
+
+ if (wdev->cac_started) {
+ err = -EBUSY;
+ goto unlock;
+ }
/* CAC start is offloaded to HW and can't be started manually */
- if (wiphy_ext_feature_isset(wiphy, NL80211_EXT_FEATURE_DFS_OFFLOAD))
- return -EOPNOTSUPP;
+ if (wiphy_ext_feature_isset(wiphy, NL80211_EXT_FEATURE_DFS_OFFLOAD)) {
+ err = -EOPNOTSUPP;
+ goto unlock;
+ }
- if (!rdev->ops->start_radar_detection)
- return -EOPNOTSUPP;
+ if (!rdev->ops->start_radar_detection) {
+ err = -EOPNOTSUPP;
+ goto unlock;
+ }
cac_time_ms = cfg80211_chandef_dfs_cac_time(&rdev->wiphy, &chandef);
if (WARN_ON(!cac_time_ms))
@@ -9318,6 +9347,9 @@ static int nl80211_start_radar_detection(struct sk_buff *skb,
wdev->cac_start_time = jiffies;
wdev->cac_time_ms = cac_time_ms;
}
+unlock:
+ wiphy_unlock(wiphy);
+
return err;
}
@@ -15954,7 +15986,8 @@ static const struct genl_small_ops nl80211_small_ops[] = {
.validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP,
.doit = nl80211_start_radar_detection,
.flags = GENL_UNS_ADMIN_PERM,
- .internal_flags = NL80211_FLAG_NEED_NETDEV_UP,
+ .internal_flags = NL80211_FLAG_NEED_NETDEV_UP |
+ NL80211_FLAG_NO_WIPHY_MTX,
},
{
.cmd = NL80211_CMD_GET_PROTOCOL_FEATURES,
@@ -17035,6 +17068,44 @@ static void nl80211_send_remain_on_chan_event(
nlmsg_free(msg);
}
+void cfg80211_assoc_comeback(struct net_device *netdev,
+ struct cfg80211_bss *bss, u32 timeout)
+{
+ struct wireless_dev *wdev = netdev->ieee80211_ptr;
+ struct wiphy *wiphy = wdev->wiphy;
+ struct cfg80211_registered_device *rdev = wiphy_to_rdev(wiphy);
+ struct sk_buff *msg;
+ void *hdr;
+
+ trace_cfg80211_assoc_comeback(wdev, bss->bssid, timeout);
+
+ msg = nlmsg_new(NLMSG_DEFAULT_SIZE, GFP_KERNEL);
+ if (!msg)
+ return;
+
+ hdr = nl80211hdr_put(msg, 0, 0, 0, NL80211_CMD_ASSOC_COMEBACK);
+ if (!hdr) {
+ nlmsg_free(msg);
+ return;
+ }
+
+ if (nla_put_u32(msg, NL80211_ATTR_WIPHY, rdev->wiphy_idx) ||
+ nla_put_u32(msg, NL80211_ATTR_IFINDEX, netdev->ifindex) ||
+ nla_put(msg, NL80211_ATTR_MAC, ETH_ALEN, bss->bssid) ||
+ nla_put_u32(msg, NL80211_ATTR_TIMEOUT, timeout))
+ goto nla_put_failure;
+
+ genlmsg_end(msg, hdr);
+
+ genlmsg_multicast_netns(&nl80211_fam, wiphy_net(&rdev->wiphy), msg, 0,
+ NL80211_MCGRP_MLME, GFP_KERNEL);
+ return;
+
+ nla_put_failure:
+ nlmsg_free(msg);
+}
+EXPORT_SYMBOL(cfg80211_assoc_comeback);
+
void cfg80211_ready_on_channel(struct wireless_dev *wdev, u64 cookie,
struct ieee80211_channel *chan,
unsigned int duration, gfp_t gfp)
diff --git a/net/wireless/rdev-ops.h b/net/wireless/rdev-ops.h
index cc1efec..439bcf5 100644
--- a/net/wireless/rdev-ops.h
+++ b/net/wireless/rdev-ops.h
@@ -1395,4 +1395,21 @@ rdev_set_fils_aad(struct cfg80211_registered_device *rdev,
return ret;
}
+static inline int
+rdev_set_radar_background(struct cfg80211_registered_device *rdev,
+ struct cfg80211_chan_def *chandef)
+{
+ struct wiphy *wiphy = &rdev->wiphy;
+ int ret;
+
+ if (!rdev->ops->set_radar_background)
+ return -EOPNOTSUPP;
+
+ trace_rdev_set_radar_background(wiphy, chandef);
+ ret = rdev->ops->set_radar_background(wiphy, chandef);
+ trace_rdev_return_int(wiphy, ret);
+
+ return ret;
+}
+
#endif /* __CFG80211_RDEV_OPS */
diff --git a/net/wireless/reg.c b/net/wireless/reg.c
index f8f01a3..ec25924 100644
--- a/net/wireless/reg.c
+++ b/net/wireless/reg.c
@@ -2371,6 +2371,7 @@ static bool reg_wdev_chan_valid(struct wiphy *wiphy, struct wireless_dev *wdev)
switch (iftype) {
case NL80211_IFTYPE_AP:
case NL80211_IFTYPE_P2P_GO:
+ case NL80211_IFTYPE_MESH_POINT:
if (!wdev->beacon_interval)
goto wdev_inactive_unlock;
chandef = wdev->chandef;
@@ -2409,6 +2410,7 @@ static bool reg_wdev_chan_valid(struct wiphy *wiphy, struct wireless_dev *wdev)
case NL80211_IFTYPE_AP:
case NL80211_IFTYPE_P2P_GO:
case NL80211_IFTYPE_ADHOC:
+ case NL80211_IFTYPE_MESH_POINT:
wiphy_lock(wiphy);
ret = cfg80211_reg_can_beacon_relax(wiphy, &chandef, iftype);
wiphy_unlock(wiphy);
diff --git a/net/wireless/scan.c b/net/wireless/scan.c
index 22e92be..b888522 100644
--- a/net/wireless/scan.c
+++ b/net/wireless/scan.c
@@ -406,22 +406,20 @@ static int
cfg80211_add_nontrans_list(struct cfg80211_bss *trans_bss,
struct cfg80211_bss *nontrans_bss)
{
- const u8 *ssid;
- size_t ssid_len;
+ const struct element *ssid_elem;
struct cfg80211_bss *bss = NULL;
rcu_read_lock();
- ssid = ieee80211_bss_get_ie(nontrans_bss, WLAN_EID_SSID);
- if (!ssid) {
+ ssid_elem = ieee80211_bss_get_elem(nontrans_bss, WLAN_EID_SSID);
+ if (!ssid_elem) {
rcu_read_unlock();
return -EINVAL;
}
- ssid_len = ssid[1];
- ssid = ssid + 2;
/* check if nontrans_bss is in the list */
list_for_each_entry(bss, &trans_bss->nontrans_list, nontrans_list) {
- if (is_bss(bss, nontrans_bss->bssid, ssid, ssid_len)) {
+ if (is_bss(bss, nontrans_bss->bssid, ssid_elem->data,
+ ssid_elem->datalen)) {
rcu_read_unlock();
return 0;
}
@@ -1795,33 +1793,52 @@ cfg80211_bss_update(struct cfg80211_registered_device *rdev,
}
int cfg80211_get_ies_channel_number(const u8 *ie, size_t ielen,
- enum nl80211_band band)
+ enum nl80211_band band,
+ enum cfg80211_bss_frame_type ftype)
{
- const u8 *tmp;
- int channel_number = -1;
+ const struct element *tmp;
- if (band == NL80211_BAND_S1GHZ) {
- tmp = cfg80211_find_ie(WLAN_EID_S1G_OPERATION, ie, ielen);
- if (tmp && tmp[1] >= sizeof(struct ieee80211_s1g_oper_ie)) {
- struct ieee80211_s1g_oper_ie *s1gop = (void *)(tmp + 2);
+ if (band == NL80211_BAND_6GHZ) {
+ struct ieee80211_he_operation *he_oper;
- channel_number = s1gop->primary_ch;
+ tmp = cfg80211_find_ext_elem(WLAN_EID_EXT_HE_OPERATION, ie,
+ ielen);
+ if (tmp && tmp->datalen >= sizeof(*he_oper) &&
+ tmp->datalen >= ieee80211_he_oper_size(&tmp->data[1])) {
+ const struct ieee80211_he_6ghz_oper *he_6ghz_oper;
+
+ he_oper = (void *)&tmp->data[1];
+
+ he_6ghz_oper = ieee80211_he_6ghz_oper(he_oper);
+ if (!he_6ghz_oper)
+ return -1;
+
+ if (ftype != CFG80211_BSS_FTYPE_BEACON ||
+ he_6ghz_oper->control & IEEE80211_HE_6GHZ_OPER_CTRL_DUP_BEACON)
+ return he_6ghz_oper->primary;
+ }
+ } else if (band == NL80211_BAND_S1GHZ) {
+ tmp = cfg80211_find_elem(WLAN_EID_S1G_OPERATION, ie, ielen);
+ if (tmp && tmp->datalen >= sizeof(struct ieee80211_s1g_oper_ie)) {
+ struct ieee80211_s1g_oper_ie *s1gop = (void *)tmp->data;
+
+ return s1gop->primary_ch;
}
} else {
- tmp = cfg80211_find_ie(WLAN_EID_DS_PARAMS, ie, ielen);
- if (tmp && tmp[1] == 1) {
- channel_number = tmp[2];
- } else {
- tmp = cfg80211_find_ie(WLAN_EID_HT_OPERATION, ie, ielen);
- if (tmp && tmp[1] >= sizeof(struct ieee80211_ht_operation)) {
- struct ieee80211_ht_operation *htop = (void *)(tmp + 2);
+ tmp = cfg80211_find_elem(WLAN_EID_DS_PARAMS, ie, ielen);
+ if (tmp && tmp->datalen == 1)
+ return tmp->data[0];
- channel_number = htop->primary_chan;
- }
+ tmp = cfg80211_find_elem(WLAN_EID_HT_OPERATION, ie, ielen);
+ if (tmp &&
+ tmp->datalen >= sizeof(struct ieee80211_ht_operation)) {
+ struct ieee80211_ht_operation *htop = (void *)tmp->data;
+
+ return htop->primary_chan;
}
}
- return channel_number;
+ return -1;
}
EXPORT_SYMBOL(cfg80211_get_ies_channel_number);
@@ -1831,18 +1848,20 @@ EXPORT_SYMBOL(cfg80211_get_ies_channel_number);
* from neighboring channels and the Beacon frames use the DSSS Parameter Set
* element to indicate the current (transmitting) channel, but this might also
* be needed on other bands if RX frequency does not match with the actual
- * operating channel of a BSS.
+ * operating channel of a BSS, or if the AP reports a different primary channel.
*/
static struct ieee80211_channel *
cfg80211_get_bss_channel(struct wiphy *wiphy, const u8 *ie, size_t ielen,
struct ieee80211_channel *channel,
- enum nl80211_bss_scan_width scan_width)
+ enum nl80211_bss_scan_width scan_width,
+ enum cfg80211_bss_frame_type ftype)
{
u32 freq;
int channel_number;
struct ieee80211_channel *alt_channel;
- channel_number = cfg80211_get_ies_channel_number(ie, ielen, channel->band);
+ channel_number = cfg80211_get_ies_channel_number(ie, ielen,
+ channel->band, ftype);
if (channel_number < 0) {
/* No channel information in frame payload */
@@ -1850,6 +1869,16 @@ cfg80211_get_bss_channel(struct wiphy *wiphy, const u8 *ie, size_t ielen,
}
freq = ieee80211_channel_to_freq_khz(channel_number, channel->band);
+
+ /*
+ * In 6GHz, duplicated beacon indication is relevant for
+ * beacons only.
+ */
+ if (channel->band == NL80211_BAND_6GHZ &&
+ (freq == channel->center_freq ||
+ abs(freq - channel->center_freq) > 80))
+ return channel;
+
alt_channel = ieee80211_get_channel_khz(wiphy, freq);
if (!alt_channel) {
if (channel->band == NL80211_BAND_2GHZ) {
@@ -1911,7 +1940,7 @@ cfg80211_inform_single_bss_data(struct wiphy *wiphy,
return NULL;
channel = cfg80211_get_bss_channel(wiphy, ie, ielen, data->chan,
- data->scan_width);
+ data->scan_width, ftype);
if (!channel)
return NULL;
@@ -2234,7 +2263,8 @@ cfg80211_update_notlisted_nontrans(struct wiphy *wiphy,
struct ieee80211_mgmt *mgmt, size_t len)
{
u8 *ie, *new_ie, *pos;
- const u8 *nontrans_ssid, *trans_ssid, *mbssid;
+ const struct element *nontrans_ssid;
+ const u8 *trans_ssid, *mbssid;
size_t ielen = len - offsetof(struct ieee80211_mgmt,
u.probe_resp.variable);
size_t new_ie_len;
@@ -2261,11 +2291,11 @@ cfg80211_update_notlisted_nontrans(struct wiphy *wiphy,
return;
new_ie_len -= mbssid[1];
- nontrans_ssid = ieee80211_bss_get_ie(nontrans_bss, WLAN_EID_SSID);
+ nontrans_ssid = ieee80211_bss_get_elem(nontrans_bss, WLAN_EID_SSID);
if (!nontrans_ssid)
return;
- new_ie_len += nontrans_ssid[1];
+ new_ie_len += nontrans_ssid->datalen;
/* generate new ie for nontrans BSS
* 1. replace SSID with nontrans BSS' SSID
@@ -2282,7 +2312,7 @@ cfg80211_update_notlisted_nontrans(struct wiphy *wiphy,
pos = new_ie;
/* copy the nontransmitted SSID */
- cpy_len = nontrans_ssid[1] + 2;
+ cpy_len = nontrans_ssid->datalen + 2;
memcpy(pos, nontrans_ssid, cpy_len);
pos += cpy_len;
/* copy the IEs between SSID and MBSSID */
@@ -2333,6 +2363,7 @@ cfg80211_inform_single_bss_frame_data(struct wiphy *wiphy,
size_t ielen, min_hdr_len = offsetof(struct ieee80211_mgmt,
u.probe_resp.variable);
int bss_type;
+ enum cfg80211_bss_frame_type ftype;
BUILD_BUG_ON(offsetof(struct ieee80211_mgmt, u.probe_resp.variable) !=
offsetof(struct ieee80211_mgmt, u.beacon.variable));
@@ -2369,8 +2400,16 @@ cfg80211_inform_single_bss_frame_data(struct wiphy *wiphy,
variable = ext->u.s1g_beacon.variable;
}
+ if (ieee80211_is_beacon(mgmt->frame_control))
+ ftype = CFG80211_BSS_FTYPE_BEACON;
+ else if (ieee80211_is_probe_resp(mgmt->frame_control))
+ ftype = CFG80211_BSS_FTYPE_PRESP;
+ else
+ ftype = CFG80211_BSS_FTYPE_UNKNOWN;
+
channel = cfg80211_get_bss_channel(wiphy, variable,
- ielen, data->chan, data->scan_width);
+ ielen, data->chan, data->scan_width,
+ ftype);
if (!channel)
return NULL;
@@ -2687,7 +2726,7 @@ int cfg80211_wext_siwscan(struct net_device *dev,
struct cfg80211_registered_device *rdev;
struct wiphy *wiphy;
struct iw_scan_req *wreq = NULL;
- struct cfg80211_scan_request *creq = NULL;
+ struct cfg80211_scan_request *creq;
int i, err, n_channels = 0;
enum nl80211_band band;
@@ -2702,10 +2741,8 @@ int cfg80211_wext_siwscan(struct net_device *dev,
if (IS_ERR(rdev))
return PTR_ERR(rdev);
- if (rdev->scan_req || rdev->scan_msg) {
- err = -EBUSY;
- goto out;
- }
+ if (rdev->scan_req || rdev->scan_msg)
+ return -EBUSY;
wiphy = &rdev->wiphy;
@@ -2718,10 +2755,8 @@ int cfg80211_wext_siwscan(struct net_device *dev,
creq = kzalloc(sizeof(*creq) + sizeof(struct cfg80211_ssid) +
n_channels * sizeof(void *),
GFP_ATOMIC);
- if (!creq) {
- err = -ENOMEM;
- goto out;
- }
+ if (!creq)
+ return -ENOMEM;
creq->wiphy = wiphy;
creq->wdev = dev->ieee80211_ptr;
diff --git a/net/wireless/sme.c b/net/wireless/sme.c
index 08a70b4..ff4d48fc 100644
--- a/net/wireless/sme.c
+++ b/net/wireless/sme.c
@@ -680,7 +680,9 @@ void __cfg80211_connect_result(struct net_device *dev,
bool wextev)
{
struct wireless_dev *wdev = dev->ieee80211_ptr;
- const u8 *country_ie;
+ const struct element *country_elem;
+ const u8 *country_data;
+ u8 country_datalen;
#ifdef CONFIG_CFG80211_WEXT
union iwreq_data wrqu;
#endif
@@ -762,26 +764,22 @@ void __cfg80211_connect_result(struct net_device *dev,
cfg80211_upload_connect_keys(wdev);
rcu_read_lock();
- country_ie = ieee80211_bss_get_ie(cr->bss, WLAN_EID_COUNTRY);
- if (!country_ie) {
+ country_elem = ieee80211_bss_get_elem(cr->bss, WLAN_EID_COUNTRY);
+ if (!country_elem) {
rcu_read_unlock();
return;
}
- country_ie = kmemdup(country_ie, 2 + country_ie[1], GFP_ATOMIC);
+ country_datalen = country_elem->datalen;
+ country_data = kmemdup(country_elem->data, country_datalen, GFP_ATOMIC);
rcu_read_unlock();
- if (!country_ie)
+ if (!country_data)
return;
- /*
- * ieee80211_bss_get_ie() ensures we can access:
- * - country_ie + 2, the start of the country ie data, and
- * - and country_ie[1] which is the IE length
- */
regulatory_hint_country_ie(wdev->wiphy, cr->bss->channel->band,
- country_ie + 2, country_ie[1]);
- kfree(country_ie);
+ country_data, country_datalen);
+ kfree(country_data);
}
/* Consumes bss object one way or another */
diff --git a/net/wireless/trace.h b/net/wireless/trace.h
index ad6c16a..228079d 100644
--- a/net/wireless/trace.h
+++ b/net/wireless/trace.h
@@ -3053,18 +3053,21 @@ TRACE_EVENT(cfg80211_ch_switch_started_notify,
);
TRACE_EVENT(cfg80211_radar_event,
- TP_PROTO(struct wiphy *wiphy, struct cfg80211_chan_def *chandef),
- TP_ARGS(wiphy, chandef),
+ TP_PROTO(struct wiphy *wiphy, struct cfg80211_chan_def *chandef,
+ bool offchan),
+ TP_ARGS(wiphy, chandef, offchan),
TP_STRUCT__entry(
WIPHY_ENTRY
CHAN_DEF_ENTRY
+ __field(bool, offchan)
),
TP_fast_assign(
WIPHY_ASSIGN;
CHAN_DEF_ASSIGN(chandef);
+ __entry->offchan = offchan;
),
- TP_printk(WIPHY_PR_FMT ", " CHAN_DEF_PR_FMT,
- WIPHY_PR_ARG, CHAN_DEF_PR_ARG)
+ TP_printk(WIPHY_PR_FMT ", " CHAN_DEF_PR_FMT ", offchan %d",
+ WIPHY_PR_ARG, CHAN_DEF_PR_ARG, __entry->offchan)
);
TRACE_EVENT(cfg80211_cac_event,
@@ -3674,6 +3677,42 @@ TRACE_EVENT(cfg80211_bss_color_notify,
__entry->color_bitmap)
);
+TRACE_EVENT(rdev_set_radar_background,
+ TP_PROTO(struct wiphy *wiphy, struct cfg80211_chan_def *chandef),
+
+ TP_ARGS(wiphy, chandef),
+
+ TP_STRUCT__entry(
+ WIPHY_ENTRY
+ CHAN_DEF_ENTRY
+ ),
+
+ TP_fast_assign(
+ WIPHY_ASSIGN;
+ CHAN_DEF_ASSIGN(chandef)
+ ),
+
+ TP_printk(WIPHY_PR_FMT ", " CHAN_DEF_PR_FMT,
+ WIPHY_PR_ARG, CHAN_DEF_PR_ARG)
+);
+
+TRACE_EVENT(cfg80211_assoc_comeback,
+ TP_PROTO(struct wireless_dev *wdev, const u8 *bssid, u32 timeout),
+ TP_ARGS(wdev, bssid, timeout),
+ TP_STRUCT__entry(
+ WDEV_ENTRY
+ MAC_ENTRY(bssid)
+ __field(u32, timeout)
+ ),
+ TP_fast_assign(
+ WDEV_ASSIGN;
+ MAC_ASSIGN(bssid, bssid);
+ __entry->timeout = timeout;
+ ),
+ TP_printk(WDEV_PR_FMT ", " MAC_PR_FMT ", timeout: %u TUs",
+ WDEV_PR_ARG, MAC_PR_ARG(bssid), __entry->timeout)
+);
+
#endif /* !__RDEV_OPS_TRACE || TRACE_HEADER_MULTI_READ */
#undef TRACE_INCLUDE_PATH
diff --git a/net/wireless/wext-sme.c b/net/wireless/wext-sme.c
index 193a18a..cd09a90 100644
--- a/net/wireless/wext-sme.c
+++ b/net/wireless/wext-sme.c
@@ -212,18 +212,18 @@ int cfg80211_mgd_wext_giwessid(struct net_device *dev,
wdev_lock(wdev);
if (wdev->current_bss) {
- const u8 *ie;
+ const struct element *ssid_elem;
rcu_read_lock();
- ie = ieee80211_bss_get_ie(&wdev->current_bss->pub,
- WLAN_EID_SSID);
- if (ie) {
+ ssid_elem = ieee80211_bss_get_elem(&wdev->current_bss->pub,
+ WLAN_EID_SSID);
+ if (ssid_elem) {
data->flags = 1;
- data->length = ie[1];
+ data->length = ssid_elem->datalen;
if (data->length > IW_ESSID_MAX_SIZE)
ret = -EINVAL;
else
- memcpy(ssid, ie + 2, data->length);
+ memcpy(ssid, ssid_elem->data, data->length);
}
rcu_read_unlock();
} else if (wdev->wext.connect.ssid && wdev->wext.connect.ssid_len) {
diff --git a/net/x25/x25_in.c b/net/x25/x25_in.c
index e1c4197..b981a48 100644
--- a/net/x25/x25_in.c
+++ b/net/x25/x25_in.c
@@ -41,7 +41,7 @@ static int x25_queue_rx_frame(struct sock *sk, struct sk_buff *skb, int more)
return 0;
}
- if (!more && x25->fraglen > 0) { /* End of fragment */
+ if (x25->fraglen > 0) { /* End of fragment */
int len = x25->fraglen + skb->len;
if ((skbn = alloc_skb(len, GFP_ATOMIC)) == NULL){
diff --git a/net/xdp/xsk.c b/net/xdp/xsk.c
index 7a466ea..28ef3f4 100644
--- a/net/xdp/xsk.c
+++ b/net/xdp/xsk.c
@@ -677,6 +677,8 @@ static __poll_t xsk_poll(struct file *file, struct socket *sock,
struct xdp_sock *xs = xdp_sk(sk);
struct xsk_buff_pool *pool;
+ sock_poll_wait(file, sock, wait);
+
if (unlikely(!xsk_is_bound(xs)))
return mask;
@@ -688,8 +690,6 @@ static __poll_t xsk_poll(struct file *file, struct socket *sock,
else
/* Poll needs to drive Tx also in copy mode */
__xsk_sendmsg(sk);
- } else {
- sock_poll_wait(file, sock, wait);
}
if (xs->rx && !xskq_prod_is_empty(xs->rx))
@@ -794,9 +794,7 @@ static int xsk_release(struct socket *sock)
sk_del_node_init_rcu(sk);
mutex_unlock(&net->xdp.lock);
- local_bh_disable();
sock_prot_inuse_add(net, sk->sk_prot, -1);
- local_bh_enable();
xsk_delete_from_maps(xs);
mutex_lock(&xs->mutex);
@@ -1396,9 +1394,7 @@ static int xsk_create(struct net *net, struct socket *sock, int protocol,
sk_add_node_rcu(sk, &net->xdp.list);
mutex_unlock(&net->xdp.lock);
- local_bh_disable();
sock_prot_inuse_add(net, &xsk_proto, 1);
- local_bh_enable();
return 0;
}
diff --git a/net/xdp/xsk_buff_pool.c b/net/xdp/xsk_buff_pool.c
index bc4ad48..fd39bb6 100644
--- a/net/xdp/xsk_buff_pool.c
+++ b/net/xdp/xsk_buff_pool.c
@@ -83,6 +83,7 @@ struct xsk_buff_pool *xp_create_and_assign_umem(struct xdp_sock *xs,
xskb = &pool->heads[i];
xskb->pool = pool;
xskb->xdp.frame_sz = umem->chunk_size - umem->headroom;
+ INIT_LIST_HEAD(&xskb->free_list_node);
if (pool->unaligned)
pool->free_heads[i] = xskb;
else
diff --git a/net/xdp/xskmap.c b/net/xdp/xskmap.c
index 2e48d0e..65b53fb 100644
--- a/net/xdp/xskmap.c
+++ b/net/xdp/xskmap.c
@@ -4,6 +4,7 @@
*/
#include <linux/bpf.h>
+#include <linux/filter.h>
#include <linux/capability.h>
#include <net/xdp_sock.h>
#include <linux/slab.h>
diff --git a/net/xfrm/xfrm_algo.c b/net/xfrm/xfrm_algo.c
index 4dae3ab..094734f 100644
--- a/net/xfrm/xfrm_algo.c
+++ b/net/xfrm/xfrm_algo.c
@@ -341,6 +341,26 @@ static struct xfrm_algo_desc aalg_list[] = {
.pfkey_supported = 0,
},
+{
+ .name = "hmac(sm3)",
+ .compat = "sm3",
+
+ .uinfo = {
+ .auth = {
+ .icv_truncbits = 256,
+ .icv_fullbits = 256,
+ }
+ },
+
+ .pfkey_supported = 1,
+
+ .desc = {
+ .sadb_alg_id = SADB_X_AALG_SM3_256HMAC,
+ .sadb_alg_ivlen = 0,
+ .sadb_alg_minbits = 256,
+ .sadb_alg_maxbits = 256
+ }
+},
};
static struct xfrm_algo_desc ealg_list[] = {
@@ -552,6 +572,27 @@ static struct xfrm_algo_desc ealg_list[] = {
.sadb_alg_maxbits = 288
}
},
+{
+ .name = "cbc(sm4)",
+ .compat = "sm4",
+
+ .uinfo = {
+ .encr = {
+ .geniv = "echainiv",
+ .blockbits = 128,
+ .defkeybits = 128,
+ }
+ },
+
+ .pfkey_supported = 1,
+
+ .desc = {
+ .sadb_alg_id = SADB_X_EALG_SM4CBC,
+ .sadb_alg_ivlen = 16,
+ .sadb_alg_minbits = 128,
+ .sadb_alg_maxbits = 256
+ }
+},
};
static struct xfrm_algo_desc calg_list[] = {
diff --git a/net/xfrm/xfrm_compat.c b/net/xfrm/xfrm_compat.c
index 2bf2693..a0f62fa 100644
--- a/net/xfrm/xfrm_compat.c
+++ b/net/xfrm/xfrm_compat.c
@@ -127,6 +127,7 @@ static const struct nla_policy compat_policy[XFRMA_MAX+1] = {
[XFRMA_SET_MARK] = { .type = NLA_U32 },
[XFRMA_SET_MARK_MASK] = { .type = NLA_U32 },
[XFRMA_IF_ID] = { .type = NLA_U32 },
+ [XFRMA_MTIMER_THRESH] = { .type = NLA_U32 },
};
static struct nlmsghdr *xfrm_nlmsg_put_compat(struct sk_buff *skb,
@@ -274,9 +275,10 @@ static int xfrm_xlate64_attr(struct sk_buff *dst, const struct nlattr *src)
case XFRMA_SET_MARK:
case XFRMA_SET_MARK_MASK:
case XFRMA_IF_ID:
+ case XFRMA_MTIMER_THRESH:
return xfrm_nla_cpy(dst, src, nla_len(src));
default:
- BUILD_BUG_ON(XFRMA_MAX != XFRMA_IF_ID);
+ BUILD_BUG_ON(XFRMA_MAX != XFRMA_MTIMER_THRESH);
pr_warn_once("unsupported nla_type %d\n", src->nla_type);
return -EOPNOTSUPP;
}
@@ -431,7 +433,7 @@ static int xfrm_xlate32_attr(void *dst, const struct nlattr *nla,
int err;
if (type > XFRMA_MAX) {
- BUILD_BUG_ON(XFRMA_MAX != XFRMA_IF_ID);
+ BUILD_BUG_ON(XFRMA_MAX != XFRMA_MTIMER_THRESH);
NL_SET_ERR_MSG(extack, "Bad attribute");
return -EOPNOTSUPP;
}
diff --git a/net/xfrm/xfrm_device.c b/net/xfrm/xfrm_device.c
index e843b0d..3fa0664 100644
--- a/net/xfrm/xfrm_device.c
+++ b/net/xfrm/xfrm_device.c
@@ -259,6 +259,7 @@ int xfrm_dev_state_add(struct net *net, struct xfrm_state *x,
}
xso->dev = dev;
+ netdev_tracker_alloc(dev, &xso->dev_tracker, GFP_ATOMIC);
xso->real_dev = dev;
xso->num_exthdrs = 1;
xso->flags = xuo->flags;
@@ -269,7 +270,7 @@ int xfrm_dev_state_add(struct net *net, struct xfrm_state *x,
xso->flags = 0;
xso->dev = NULL;
xso->real_dev = NULL;
- dev_put(dev);
+ dev_put_track(dev, &xso->dev_tracker);
if (err != -EOPNOTSUPP)
return err;
diff --git a/net/xfrm/xfrm_input.c b/net/xfrm/xfrm_input.c
index 70a8c36..144238a 100644
--- a/net/xfrm/xfrm_input.c
+++ b/net/xfrm/xfrm_input.c
@@ -669,6 +669,7 @@ int xfrm_input(struct sk_buff *skb, int nexthdr, __be32 spi, int encap_type)
x->curlft.bytes += skb->len;
x->curlft.packets++;
+ x->curlft.use_time = ktime_get_real_seconds();
spin_unlock(&x->lock);
diff --git a/net/xfrm/xfrm_interface.c b/net/xfrm/xfrm_interface.c
index 41de46b..57448fc 100644
--- a/net/xfrm/xfrm_interface.c
+++ b/net/xfrm/xfrm_interface.c
@@ -637,11 +637,16 @@ static int xfrmi_newlink(struct net *src_net, struct net_device *dev,
struct netlink_ext_ack *extack)
{
struct net *net = dev_net(dev);
- struct xfrm_if_parms p;
+ struct xfrm_if_parms p = {};
struct xfrm_if *xi;
int err;
xfrmi_netlink_parms(data, &p);
+ if (!p.if_id) {
+ NL_SET_ERR_MSG(extack, "if_id must be non zero");
+ return -EINVAL;
+ }
+
xi = xfrmi_locate(net, &p);
if (xi)
return -EEXIST;
@@ -666,7 +671,12 @@ static int xfrmi_changelink(struct net_device *dev, struct nlattr *tb[],
{
struct xfrm_if *xi = netdev_priv(dev);
struct net *net = xi->net;
- struct xfrm_if_parms p;
+ struct xfrm_if_parms p = {};
+
+ if (!p.if_id) {
+ NL_SET_ERR_MSG(extack, "if_id must be non zero");
+ return -EINVAL;
+ }
xfrmi_netlink_parms(data, &p);
xi = xfrmi_locate(net, &p);
diff --git a/net/xfrm/xfrm_output.c b/net/xfrm/xfrm_output.c
index 229544b..d4935b3 100644
--- a/net/xfrm/xfrm_output.c
+++ b/net/xfrm/xfrm_output.c
@@ -533,6 +533,7 @@ static int xfrm_output_one(struct sk_buff *skb, int err)
x->curlft.bytes += skb->len;
x->curlft.packets++;
+ x->curlft.use_time = ktime_get_real_seconds();
spin_unlock_bh(&x->lock);
@@ -647,10 +648,12 @@ static int xfrm_output_gso(struct net *net, struct sock *sk, struct sk_buff *skb
* This requires hardware to know the inner packet type to calculate
* the inner header checksum. Save inner ip protocol here to avoid
* traversing the packet in the vendor's xmit code.
- * If the encap type is IPIP, just save skb->inner_ipproto. Otherwise,
- * get the ip protocol from the IP header.
+ * For IPsec tunnel mode save the ip protocol from the IP header of the
+ * plain text packet. Otherwise If the encap type is IPIP, just save
+ * skb->inner_ipproto in any other case get the ip protocol from the IP
+ * header.
*/
-static void xfrm_get_inner_ipproto(struct sk_buff *skb)
+static void xfrm_get_inner_ipproto(struct sk_buff *skb, struct xfrm_state *x)
{
struct xfrm_offload *xo = xfrm_offload(skb);
const struct ethhdr *eth;
@@ -658,6 +661,25 @@ static void xfrm_get_inner_ipproto(struct sk_buff *skb)
if (!xo)
return;
+ if (x->outer_mode.encap == XFRM_MODE_TUNNEL) {
+ switch (x->outer_mode.family) {
+ case AF_INET:
+ xo->inner_ipproto = ip_hdr(skb)->protocol;
+ break;
+ case AF_INET6:
+ xo->inner_ipproto = ipv6_hdr(skb)->nexthdr;
+ break;
+ default:
+ break;
+ }
+
+ return;
+ }
+
+ /* non-Tunnel Mode */
+ if (!skb->encapsulation)
+ return;
+
if (skb->inner_protocol_type == ENCAP_TYPE_IPPROTO) {
xo->inner_ipproto = skb->inner_ipproto;
return;
@@ -712,8 +734,7 @@ int xfrm_output(struct sock *sk, struct sk_buff *skb)
sp->xvec[sp->len++] = x;
xfrm_state_hold(x);
- if (skb->encapsulation)
- xfrm_get_inner_ipproto(skb);
+ xfrm_get_inner_ipproto(skb, x);
skb->encapsulation = 1;
if (skb_is_gso(skb)) {
diff --git a/net/xfrm/xfrm_policy.c b/net/xfrm/xfrm_policy.c
index 1a06585..dccb8f3 100644
--- a/net/xfrm/xfrm_policy.c
+++ b/net/xfrm/xfrm_policy.c
@@ -33,6 +33,7 @@
#include <net/flow.h>
#include <net/xfrm.h>
#include <net/ip.h>
+#include <net/gre.h>
#if IS_ENABLED(CONFIG_IPV6_MIP6)
#include <net/mip6.h>
#endif
@@ -2680,7 +2681,7 @@ static int xfrm_expand_policies(const struct flowi *fl, u16 family,
*num_xfrms = pols[0]->xfrm_nr;
#ifdef CONFIG_XFRM_SUB_POLICY
- if (pols[0] && pols[0]->action == XFRM_POLICY_ALLOW &&
+ if (pols[0]->action == XFRM_POLICY_ALLOW &&
pols[0]->type != XFRM_POLICY_TYPE_MAIN) {
pols[1] = xfrm_policy_lookup_bytype(xp_net(pols[0]),
XFRM_POLICY_TYPE_MAIN,
@@ -3392,7 +3393,6 @@ decode_session6(struct sk_buff *skb, struct flowi *fl, bool reverse)
case NEXTHDR_DEST:
offset += ipv6_optlen(exthdr);
nexthdr = exthdr->nexthdr;
- exthdr = (struct ipv6_opt_hdr *)(nh + offset);
break;
case IPPROTO_UDP:
case IPPROTO_UDPLITE:
@@ -3422,6 +3422,26 @@ decode_session6(struct sk_buff *skb, struct flowi *fl, bool reverse)
}
fl6->flowi6_proto = nexthdr;
return;
+ case IPPROTO_GRE:
+ if (!onlyproto &&
+ (nh + offset + 12 < skb->data ||
+ pskb_may_pull(skb, nh + offset + 12 - skb->data))) {
+ struct gre_base_hdr *gre_hdr;
+ __be32 *gre_key;
+
+ nh = skb_network_header(skb);
+ gre_hdr = (struct gre_base_hdr *)(nh + offset);
+ gre_key = (__be32 *)(gre_hdr + 1);
+
+ if (gre_hdr->flags & GRE_KEY) {
+ if (gre_hdr->flags & GRE_CSUM)
+ gre_key++;
+ fl6->fl6_gre_key = *gre_key;
+ }
+ }
+ fl6->flowi6_proto = nexthdr;
+ return;
+
#if IS_ENABLED(CONFIG_IPV6_MIP6)
case IPPROTO_MH:
offset += ipv6_optlen(exthdr);
diff --git a/net/xfrm/xfrm_state.c b/net/xfrm/xfrm_state.c
index a2f4001..ca6bee1 100644
--- a/net/xfrm/xfrm_state.c
+++ b/net/xfrm/xfrm_state.c
@@ -14,6 +14,7 @@
*
*/
+#include <linux/compat.h>
#include <linux/workqueue.h>
#include <net/xfrm.h>
#include <linux/pfkeyv2.h>
@@ -1593,6 +1594,9 @@ static struct xfrm_state *xfrm_state_clone(struct xfrm_state *orig,
x->km.seq = orig->km.seq;
x->replay = orig->replay;
x->preplay = orig->preplay;
+ x->mapping_maxage = orig->mapping_maxage;
+ x->new_mapping = 0;
+ x->new_mapping_sport = 0;
return x;
@@ -2242,7 +2246,7 @@ int km_query(struct xfrm_state *x, struct xfrm_tmpl *t, struct xfrm_policy *pol)
}
EXPORT_SYMBOL(km_query);
-int km_new_mapping(struct xfrm_state *x, xfrm_address_t *ipaddr, __be16 sport)
+static int __km_new_mapping(struct xfrm_state *x, xfrm_address_t *ipaddr, __be16 sport)
{
int err = -EINVAL;
struct xfrm_mgr *km;
@@ -2257,6 +2261,24 @@ int km_new_mapping(struct xfrm_state *x, xfrm_address_t *ipaddr, __be16 sport)
rcu_read_unlock();
return err;
}
+
+int km_new_mapping(struct xfrm_state *x, xfrm_address_t *ipaddr, __be16 sport)
+{
+ int ret = 0;
+
+ if (x->mapping_maxage) {
+ if ((jiffies / HZ - x->new_mapping) > x->mapping_maxage ||
+ x->new_mapping_sport != sport) {
+ x->new_mapping_sport = sport;
+ x->new_mapping = jiffies / HZ;
+ ret = __km_new_mapping(x, ipaddr, sport);
+ }
+ } else {
+ ret = __km_new_mapping(x, ipaddr, sport);
+ }
+
+ return ret;
+}
EXPORT_SYMBOL(km_new_mapping);
void km_policy_expired(struct xfrm_policy *pol, int dir, int hard, u32 portid)
diff --git a/net/xfrm/xfrm_user.c b/net/xfrm/xfrm_user.c
index 7c36cc1..8cd6c81 100644
--- a/net/xfrm/xfrm_user.c
+++ b/net/xfrm/xfrm_user.c
@@ -11,6 +11,7 @@
*
*/
+#include <linux/compat.h>
#include <linux/crypto.h>
#include <linux/module.h>
#include <linux/kernel.h>
@@ -282,6 +283,10 @@ static int verify_newsa_info(struct xfrm_usersa_info *p,
err = 0;
+ if (attrs[XFRMA_MTIMER_THRESH])
+ if (!attrs[XFRMA_ENCAP])
+ err = -EINVAL;
+
out:
return err;
}
@@ -521,6 +526,7 @@ static void xfrm_update_ae_params(struct xfrm_state *x, struct nlattr **attrs,
struct nlattr *lt = attrs[XFRMA_LTIME_VAL];
struct nlattr *et = attrs[XFRMA_ETIMER_THRESH];
struct nlattr *rt = attrs[XFRMA_REPLAY_THRESH];
+ struct nlattr *mt = attrs[XFRMA_MTIMER_THRESH];
if (re) {
struct xfrm_replay_state_esn *replay_esn;
@@ -552,6 +558,9 @@ static void xfrm_update_ae_params(struct xfrm_state *x, struct nlattr **attrs,
if (rt)
x->replay_maxdiff = nla_get_u32(rt);
+
+ if (mt)
+ x->mapping_maxage = nla_get_u32(mt);
}
static void xfrm_smark_init(struct nlattr **attrs, struct xfrm_mark *m)
@@ -621,8 +630,13 @@ static struct xfrm_state *xfrm_state_construct(struct net *net,
xfrm_smark_init(attrs, &x->props.smark);
- if (attrs[XFRMA_IF_ID])
+ if (attrs[XFRMA_IF_ID]) {
x->if_id = nla_get_u32(attrs[XFRMA_IF_ID]);
+ if (!x->if_id) {
+ err = -EINVAL;
+ goto error;
+ }
+ }
err = __xfrm_init_state(x, false, attrs[XFRMA_OFFLOAD_DEV]);
if (err)
@@ -1024,8 +1038,13 @@ static int copy_to_user_state_extra(struct xfrm_state *x,
if (ret)
goto out;
}
- if (x->security)
+ if (x->security) {
ret = copy_sec_ctx(x->security, skb);
+ if (ret)
+ goto out;
+ }
+ if (x->mapping_maxage)
+ ret = nla_put_u32(skb, XFRMA_MTIMER_THRESH, x->mapping_maxage);
out:
return ret;
}
@@ -1413,8 +1432,13 @@ static int xfrm_alloc_userspi(struct sk_buff *skb, struct nlmsghdr *nlh,
mark = xfrm_mark_get(attrs, &m);
- if (attrs[XFRMA_IF_ID])
+ if (attrs[XFRMA_IF_ID]) {
if_id = nla_get_u32(attrs[XFRMA_IF_ID]);
+ if (!if_id) {
+ err = -EINVAL;
+ goto out_noput;
+ }
+ }
if (p->info.seq) {
x = xfrm_find_acq_byseq(net, mark, p->info.seq);
@@ -1727,8 +1751,13 @@ static struct xfrm_policy *xfrm_policy_construct(struct net *net, struct xfrm_us
xfrm_mark_get(attrs, &xp->mark);
- if (attrs[XFRMA_IF_ID])
+ if (attrs[XFRMA_IF_ID]) {
xp->if_id = nla_get_u32(attrs[XFRMA_IF_ID]);
+ if (!xp->if_id) {
+ err = -EINVAL;
+ goto error;
+ }
+ }
return xp;
error:
@@ -3058,7 +3087,7 @@ static inline unsigned int xfrm_sa_len(struct xfrm_state *x)
if (x->props.extra_flags)
l += nla_total_size(sizeof(x->props.extra_flags));
if (x->xso.dev)
- l += nla_total_size(sizeof(x->xso));
+ l += nla_total_size(sizeof(struct xfrm_user_offload));
if (x->props.smark.v | x->props.smark.m) {
l += nla_total_size(sizeof(x->props.smark.v));
l += nla_total_size(sizeof(x->props.smark.m));
@@ -3069,6 +3098,9 @@ static inline unsigned int xfrm_sa_len(struct xfrm_state *x)
/* Must count x->lastused as it may become non-zero behind our back. */
l += nla_total_size_64bit(sizeof(u64));
+ if (x->mapping_maxage)
+ l += nla_total_size(sizeof(x->mapping_maxage));
+
return l;
}
diff --git a/samples/bpf/Makefile b/samples/bpf/Makefile
index a886dff..3863884 100644
--- a/samples/bpf/Makefile
+++ b/samples/bpf/Makefile
@@ -215,6 +215,11 @@
endif
TPROGS_LDLIBS += $(LIBBPF) -lelf -lz
+TPROGLDLIBS_xdp_monitor += -lm
+TPROGLDLIBS_xdp_redirect += -lm
+TPROGLDLIBS_xdp_redirect_cpu += -lm
+TPROGLDLIBS_xdp_redirect_map += -lm
+TPROGLDLIBS_xdp_redirect_map_multi += -lm
TPROGLDLIBS_tracex4 += -lrt
TPROGLDLIBS_trace_output += -lrt
TPROGLDLIBS_map_perf_test += -lrt
@@ -328,7 +333,7 @@
$(src)/*.c: verify_target_bpf $(LIBBPF)
libbpf_hdrs: $(LIBBPF)
-$(obj)/$(TRACE_HELPERS): | libbpf_hdrs
+$(obj)/$(TRACE_HELPERS) $(obj)/$(CGROUP_HELPERS) $(obj)/$(XDP_SAMPLE): | libbpf_hdrs
.PHONY: libbpf_hdrs
@@ -343,6 +348,17 @@
$(obj)/hbm.o: $(src)/hbm.h
$(obj)/hbm_edt_kern.o: $(src)/hbm.h $(src)/hbm_kern.h
+# Override includes for xdp_sample_user.o because $(srctree)/usr/include in
+# TPROGS_CFLAGS causes conflicts
+XDP_SAMPLE_CFLAGS += -Wall -O2 \
+ -I$(src)/../../tools/include \
+ -I$(src)/../../tools/include/uapi \
+ -I$(LIBBPF_INCLUDE) \
+ -I$(src)/../../tools/testing/selftests/bpf
+
+$(obj)/$(XDP_SAMPLE): TPROGS_CFLAGS = $(XDP_SAMPLE_CFLAGS)
+$(obj)/$(XDP_SAMPLE): $(src)/xdp_sample_user.h $(src)/xdp_sample_shared.h
+
-include $(BPF_SAMPLES_PATH)/Makefile.target
VMLINUX_BTF_PATHS ?= $(abspath $(if $(O),$(O)/vmlinux)) \
diff --git a/samples/bpf/Makefile.target b/samples/bpf/Makefile.target
index 5a368af..7621f55 100644
--- a/samples/bpf/Makefile.target
+++ b/samples/bpf/Makefile.target
@@ -73,14 +73,3 @@
cmd_tprog-cobjs = $(CC) $(tprogc_flags) -c -o $@ $<
$(tprog-cobjs): $(obj)/%.o: $(src)/%.c FORCE
$(call if_changed_dep,tprog-cobjs)
-
-# Override includes for xdp_sample_user.o because $(srctree)/usr/include in
-# TPROGS_CFLAGS causes conflicts
-XDP_SAMPLE_CFLAGS += -Wall -O2 -lm \
- -I./tools/include \
- -I./tools/include/uapi \
- -I./tools/lib \
- -I./tools/testing/selftests/bpf
-$(obj)/xdp_sample_user.o: $(src)/xdp_sample_user.c \
- $(src)/xdp_sample_user.h $(src)/xdp_sample_shared.h
- $(CC) $(XDP_SAMPLE_CFLAGS) -c -o $@ $<
diff --git a/samples/bpf/cookie_uid_helper_example.c b/samples/bpf/cookie_uid_helper_example.c
index 5495880..f0df3dd 100644
--- a/samples/bpf/cookie_uid_helper_example.c
+++ b/samples/bpf/cookie_uid_helper_example.c
@@ -67,8 +67,8 @@ static bool test_finish;
static void maps_create(void)
{
- map_fd = bpf_create_map(BPF_MAP_TYPE_HASH, sizeof(uint32_t),
- sizeof(struct stats), 100, 0);
+ map_fd = bpf_map_create(BPF_MAP_TYPE_HASH, NULL, sizeof(uint32_t),
+ sizeof(struct stats), 100, NULL);
if (map_fd < 0)
error(1, errno, "map create failed!\n");
}
@@ -157,9 +157,13 @@ static void prog_load(void)
offsetof(struct __sk_buff, len)),
BPF_EXIT_INSN(),
};
- prog_fd = bpf_load_program(BPF_PROG_TYPE_SOCKET_FILTER, prog,
- ARRAY_SIZE(prog), "GPL", 0,
- log_buf, sizeof(log_buf));
+ LIBBPF_OPTS(bpf_prog_load_opts, opts,
+ .log_buf = log_buf,
+ .log_size = sizeof(log_buf),
+ );
+
+ prog_fd = bpf_prog_load(BPF_PROG_TYPE_SOCKET_FILTER, NULL, "GPL",
+ prog, ARRAY_SIZE(prog), &opts);
if (prog_fd < 0)
error(1, errno, "failed to load prog\n%s\n", log_buf);
}
diff --git a/samples/bpf/fds_example.c b/samples/bpf/fds_example.c
index 59f45fe..16dbf49 100644
--- a/samples/bpf/fds_example.c
+++ b/samples/bpf/fds_example.c
@@ -46,12 +46,6 @@ static void usage(void)
printf(" -h Display this help.\n");
}
-static int bpf_map_create(void)
-{
- return bpf_create_map(BPF_MAP_TYPE_ARRAY, sizeof(uint32_t),
- sizeof(uint32_t), 1024, 0);
-}
-
static int bpf_prog_create(const char *object)
{
static struct bpf_insn insns[] = {
@@ -60,16 +54,22 @@ static int bpf_prog_create(const char *object)
};
size_t insns_cnt = sizeof(insns) / sizeof(struct bpf_insn);
struct bpf_object *obj;
- int prog_fd;
+ int err;
if (object) {
- assert(!bpf_prog_load(object, BPF_PROG_TYPE_UNSPEC,
- &obj, &prog_fd));
- return prog_fd;
+ obj = bpf_object__open_file(object, NULL);
+ assert(!libbpf_get_error(obj));
+ err = bpf_object__load(obj);
+ assert(!err);
+ return bpf_program__fd(bpf_object__next_program(obj, NULL));
} else {
- return bpf_load_program(BPF_PROG_TYPE_SOCKET_FILTER,
- insns, insns_cnt, "GPL", 0,
- bpf_log_buf, BPF_LOG_BUF_SIZE);
+ LIBBPF_OPTS(bpf_prog_load_opts, opts,
+ .log_buf = bpf_log_buf,
+ .log_size = BPF_LOG_BUF_SIZE,
+ );
+
+ return bpf_prog_load(BPF_PROG_TYPE_SOCKET_FILTER, NULL, "GPL",
+ insns, insns_cnt, &opts);
}
}
@@ -79,7 +79,8 @@ static int bpf_do_map(const char *file, uint32_t flags, uint32_t key,
int fd, ret;
if (flags & BPF_F_PIN) {
- fd = bpf_map_create();
+ fd = bpf_map_create(BPF_MAP_TYPE_ARRAY, NULL, sizeof(uint32_t),
+ sizeof(uint32_t), 1024, NULL);
printf("bpf: map fd:%d (%s)\n", fd, strerror(errno));
assert(fd > 0);
diff --git a/samples/bpf/hbm.c b/samples/bpf/hbm.c
index b0c18ef..1fe5bca 100644
--- a/samples/bpf/hbm.c
+++ b/samples/bpf/hbm.c
@@ -120,6 +120,9 @@ static void do_error(char *msg, bool errno_flag)
static int prog_load(char *prog)
{
+ struct bpf_program *pos;
+ const char *sec_name;
+
obj = bpf_object__open_file(prog, NULL);
if (libbpf_get_error(obj)) {
printf("ERROR: opening BPF object file failed\n");
@@ -132,7 +135,13 @@ static int prog_load(char *prog)
goto err;
}
- bpf_prog = bpf_object__find_program_by_title(obj, "cgroup_skb/egress");
+ bpf_object__for_each_program(pos, obj) {
+ sec_name = bpf_program__section_name(pos);
+ if (sec_name && !strcmp(sec_name, "cgroup_skb/egress")) {
+ bpf_prog = pos;
+ break;
+ }
+ }
if (!bpf_prog) {
printf("ERROR: finding a prog in obj file failed\n");
goto err;
diff --git a/samples/bpf/lwt_len_hist_kern.c b/samples/bpf/lwt_len_hist_kern.c
index 9ed63e1..1fa14c5 100644
--- a/samples/bpf/lwt_len_hist_kern.c
+++ b/samples/bpf/lwt_len_hist_kern.c
@@ -16,13 +16,6 @@
#include <uapi/linux/in.h>
#include <bpf/bpf_helpers.h>
-# define printk(fmt, ...) \
- ({ \
- char ____fmt[] = fmt; \
- bpf_trace_printk(____fmt, sizeof(____fmt), \
- ##__VA_ARGS__); \
- })
-
struct bpf_elf_map {
__u32 type;
__u32 size_key;
diff --git a/samples/bpf/map_perf_test_user.c b/samples/bpf/map_perf_test_user.c
index 9db9492..319fd31 100644
--- a/samples/bpf/map_perf_test_user.c
+++ b/samples/bpf/map_perf_test_user.c
@@ -134,19 +134,22 @@ static void do_test_lru(enum test_type test, int cpu)
*/
int outer_fd = map_fd[array_of_lru_hashs_idx];
unsigned int mycpu, mynode;
+ LIBBPF_OPTS(bpf_map_create_opts, opts,
+ .map_flags = BPF_F_NUMA_NODE,
+ );
assert(cpu < MAX_NR_CPUS);
ret = syscall(__NR_getcpu, &mycpu, &mynode, NULL);
assert(!ret);
+ opts.numa_node = mynode;
inner_lru_map_fds[cpu] =
- bpf_create_map_node(BPF_MAP_TYPE_LRU_HASH,
- test_map_names[INNER_LRU_HASH_PREALLOC],
- sizeof(uint32_t),
- sizeof(long),
- inner_lru_hash_size, 0,
- mynode);
+ bpf_map_create(BPF_MAP_TYPE_LRU_HASH,
+ test_map_names[INNER_LRU_HASH_PREALLOC],
+ sizeof(uint32_t),
+ sizeof(long),
+ inner_lru_hash_size, &opts);
if (inner_lru_map_fds[cpu] == -1) {
printf("cannot create BPF_MAP_TYPE_LRU_HASH %s(%d)\n",
strerror(errno), errno);
diff --git a/samples/bpf/sock_example.c b/samples/bpf/sock_example.c
index 23d1930..a88f695 100644
--- a/samples/bpf/sock_example.c
+++ b/samples/bpf/sock_example.c
@@ -37,8 +37,8 @@ static int test_sock(void)
int sock = -1, map_fd, prog_fd, i, key;
long long value = 0, tcp_cnt, udp_cnt, icmp_cnt;
- map_fd = bpf_create_map(BPF_MAP_TYPE_ARRAY, sizeof(key), sizeof(value),
- 256, 0);
+ map_fd = bpf_map_create(BPF_MAP_TYPE_ARRAY, NULL, sizeof(key), sizeof(value),
+ 256, NULL);
if (map_fd < 0) {
printf("failed to create map '%s'\n", strerror(errno));
goto cleanup;
@@ -59,9 +59,13 @@ static int test_sock(void)
BPF_EXIT_INSN(),
};
size_t insns_cnt = sizeof(prog) / sizeof(struct bpf_insn);
+ LIBBPF_OPTS(bpf_prog_load_opts, opts,
+ .log_buf = bpf_log_buf,
+ .log_size = BPF_LOG_BUF_SIZE,
+ );
- prog_fd = bpf_load_program(BPF_PROG_TYPE_SOCKET_FILTER, prog, insns_cnt,
- "GPL", 0, bpf_log_buf, BPF_LOG_BUF_SIZE);
+ prog_fd = bpf_prog_load(BPF_PROG_TYPE_SOCKET_FILTER, NULL, "GPL",
+ prog, insns_cnt, &opts);
if (prog_fd < 0) {
printf("failed to load prog '%s'\n", strerror(errno));
goto cleanup;
diff --git a/samples/bpf/sockex1_user.c b/samples/bpf/sockex1_user.c
index 3c83722..9e8d39e 100644
--- a/samples/bpf/sockex1_user.c
+++ b/samples/bpf/sockex1_user.c
@@ -11,17 +11,26 @@
int main(int ac, char **argv)
{
struct bpf_object *obj;
+ struct bpf_program *prog;
int map_fd, prog_fd;
char filename[256];
- int i, sock;
+ int i, sock, err;
FILE *f;
snprintf(filename, sizeof(filename), "%s_kern.o", argv[0]);
- if (bpf_prog_load(filename, BPF_PROG_TYPE_SOCKET_FILTER,
- &obj, &prog_fd))
+ obj = bpf_object__open_file(filename, NULL);
+ if (libbpf_get_error(obj))
return 1;
+ prog = bpf_object__next_program(obj, NULL);
+ bpf_program__set_type(prog, BPF_PROG_TYPE_SOCKET_FILTER);
+
+ err = bpf_object__load(obj);
+ if (err)
+ return 1;
+
+ prog_fd = bpf_program__fd(prog);
map_fd = bpf_object__find_map_fd_by_name(obj, "my_map");
sock = open_raw_sock("lo");
diff --git a/samples/bpf/sockex2_user.c b/samples/bpf/sockex2_user.c
index bafa567b..6a3fd36 100644
--- a/samples/bpf/sockex2_user.c
+++ b/samples/bpf/sockex2_user.c
@@ -16,18 +16,26 @@ struct pair {
int main(int ac, char **argv)
{
+ struct bpf_program *prog;
struct bpf_object *obj;
int map_fd, prog_fd;
char filename[256];
- int i, sock;
+ int i, sock, err;
FILE *f;
snprintf(filename, sizeof(filename), "%s_kern.o", argv[0]);
-
- if (bpf_prog_load(filename, BPF_PROG_TYPE_SOCKET_FILTER,
- &obj, &prog_fd))
+ obj = bpf_object__open_file(filename, NULL);
+ if (libbpf_get_error(obj))
return 1;
+ prog = bpf_object__next_program(obj, NULL);
+ bpf_program__set_type(prog, BPF_PROG_TYPE_SOCKET_FILTER);
+
+ err = bpf_object__load(obj);
+ if (err)
+ return 1;
+
+ prog_fd = bpf_program__fd(prog);
map_fd = bpf_object__find_map_fd_by_name(obj, "hash_map");
sock = open_raw_sock("lo");
diff --git a/samples/bpf/test_cgrp2_array_pin.c b/samples/bpf/test_cgrp2_array_pin.c
index 6d564aa..05e88aa 100644
--- a/samples/bpf/test_cgrp2_array_pin.c
+++ b/samples/bpf/test_cgrp2_array_pin.c
@@ -64,9 +64,9 @@ int main(int argc, char **argv)
}
if (create_array) {
- array_fd = bpf_create_map(BPF_MAP_TYPE_CGROUP_ARRAY,
+ array_fd = bpf_map_create(BPF_MAP_TYPE_CGROUP_ARRAY, NULL,
sizeof(uint32_t), sizeof(uint32_t),
- 1, 0);
+ 1, NULL);
if (array_fd < 0) {
fprintf(stderr,
"bpf_create_map(BPF_MAP_TYPE_CGROUP_ARRAY,...): %s(%d)\n",
diff --git a/samples/bpf/test_cgrp2_attach.c b/samples/bpf/test_cgrp2_attach.c
index 390ff38..6d90874 100644
--- a/samples/bpf/test_cgrp2_attach.c
+++ b/samples/bpf/test_cgrp2_attach.c
@@ -71,10 +71,13 @@ static int prog_load(int map_fd, int verdict)
BPF_EXIT_INSN(),
};
size_t insns_cnt = sizeof(prog) / sizeof(struct bpf_insn);
+ LIBBPF_OPTS(bpf_prog_load_opts, opts,
+ .log_buf = bpf_log_buf,
+ .log_size = BPF_LOG_BUF_SIZE,
+ );
- return bpf_load_program(BPF_PROG_TYPE_CGROUP_SKB,
- prog, insns_cnt, "GPL", 0,
- bpf_log_buf, BPF_LOG_BUF_SIZE);
+ return bpf_prog_load(BPF_PROG_TYPE_CGROUP_SKB, NULL, "GPL",
+ prog, insns_cnt, &opts);
}
static int usage(const char *argv0)
@@ -90,9 +93,9 @@ static int attach_filter(int cg_fd, int type, int verdict)
int prog_fd, map_fd, ret, key;
long long pkt_cnt, byte_cnt;
- map_fd = bpf_create_map(BPF_MAP_TYPE_ARRAY,
+ map_fd = bpf_map_create(BPF_MAP_TYPE_ARRAY, NULL,
sizeof(key), sizeof(byte_cnt),
- 256, 0);
+ 256, NULL);
if (map_fd < 0) {
printf("Failed to create map: '%s'\n", strerror(errno));
return EXIT_FAILURE;
diff --git a/samples/bpf/test_cgrp2_sock.c b/samples/bpf/test_cgrp2_sock.c
index b0811da..a0811df 100644
--- a/samples/bpf/test_cgrp2_sock.c
+++ b/samples/bpf/test_cgrp2_sock.c
@@ -70,6 +70,10 @@ static int prog_load(__u32 idx, __u32 mark, __u32 prio)
BPF_MOV64_IMM(BPF_REG_2, offsetof(struct bpf_sock, priority)),
BPF_STX_MEM(BPF_W, BPF_REG_1, BPF_REG_3, offsetof(struct bpf_sock, priority)),
};
+ LIBBPF_OPTS(bpf_prog_load_opts, opts,
+ .log_buf = bpf_log_buf,
+ .log_size = BPF_LOG_BUF_SIZE,
+ );
struct bpf_insn *prog;
size_t insns_cnt;
@@ -115,8 +119,8 @@ static int prog_load(__u32 idx, __u32 mark, __u32 prio)
insns_cnt /= sizeof(struct bpf_insn);
- ret = bpf_load_program(BPF_PROG_TYPE_CGROUP_SOCK, prog, insns_cnt,
- "GPL", 0, bpf_log_buf, BPF_LOG_BUF_SIZE);
+ ret = bpf_prog_load(BPF_PROG_TYPE_CGROUP_SOCK, NULL, "GPL",
+ prog, insns_cnt, &opts);
free(prog);
diff --git a/samples/bpf/test_lru_dist.c b/samples/bpf/test_lru_dist.c
index c92c5c0..75e8778 100644
--- a/samples/bpf/test_lru_dist.c
+++ b/samples/bpf/test_lru_dist.c
@@ -105,10 +105,10 @@ struct pfect_lru {
static void pfect_lru_init(struct pfect_lru *lru, unsigned int lru_size,
unsigned int nr_possible_elems)
{
- lru->map_fd = bpf_create_map(BPF_MAP_TYPE_HASH,
+ lru->map_fd = bpf_map_create(BPF_MAP_TYPE_HASH, NULL,
sizeof(unsigned long long),
sizeof(struct pfect_lru_node *),
- nr_possible_elems, 0);
+ nr_possible_elems, NULL);
assert(lru->map_fd != -1);
lru->free_nodes = malloc(lru_size * sizeof(struct pfect_lru_node));
@@ -207,10 +207,13 @@ static unsigned int read_keys(const char *dist_file,
static int create_map(int map_type, int map_flags, unsigned int size)
{
+ LIBBPF_OPTS(bpf_map_create_opts, opts,
+ .map_flags = map_flags,
+ );
int map_fd;
- map_fd = bpf_create_map(map_type, sizeof(unsigned long long),
- sizeof(unsigned long long), size, map_flags);
+ map_fd = bpf_map_create(map_type, NULL, sizeof(unsigned long long),
+ sizeof(unsigned long long), size, &opts);
if (map_fd == -1)
perror("bpf_create_map");
diff --git a/samples/bpf/trace_output_user.c b/samples/bpf/trace_output_user.c
index 364b987..371732f 100644
--- a/samples/bpf/trace_output_user.c
+++ b/samples/bpf/trace_output_user.c
@@ -43,7 +43,6 @@ static void print_bpf_output(void *ctx, int cpu, void *data, __u32 size)
int main(int argc, char **argv)
{
- struct perf_buffer_opts pb_opts = {};
struct bpf_link *link = NULL;
struct bpf_program *prog;
struct perf_buffer *pb;
@@ -84,8 +83,7 @@ int main(int argc, char **argv)
goto cleanup;
}
- pb_opts.sample_cb = print_bpf_output;
- pb = perf_buffer__new(map_fd, 8, &pb_opts);
+ pb = perf_buffer__new(map_fd, 8, print_bpf_output, NULL, NULL, NULL);
ret = libbpf_get_error(pb);
if (ret) {
printf("failed to setup perf_buffer: %d\n", ret);
diff --git a/samples/bpf/xdp_fwd_user.c b/samples/bpf/xdp_fwd_user.c
index 0006126..4ad8967 100644
--- a/samples/bpf/xdp_fwd_user.c
+++ b/samples/bpf/xdp_fwd_user.c
@@ -79,7 +79,9 @@ int main(int argc, char **argv)
.prog_type = BPF_PROG_TYPE_XDP,
};
const char *prog_name = "xdp_fwd";
- struct bpf_program *prog;
+ struct bpf_program *prog = NULL;
+ struct bpf_program *pos;
+ const char *sec_name;
int prog_fd, map_fd = -1;
char filename[PATH_MAX];
struct bpf_object *obj;
@@ -134,7 +136,13 @@ int main(int argc, char **argv)
return 1;
}
- prog = bpf_object__find_program_by_title(obj, prog_name);
+ bpf_object__for_each_program(pos, obj) {
+ sec_name = bpf_program__section_name(pos);
+ if (sec_name && !strcmp(sec_name, prog_name)) {
+ prog = pos;
+ break;
+ }
+ }
prog_fd = bpf_program__fd(prog);
if (prog_fd < 0) {
printf("program not found: %s\n", strerror(prog_fd));
diff --git a/samples/bpf/xdp_redirect_cpu.bpf.c b/samples/bpf/xdp_redirect_cpu.bpf.c
index f10fe3c..25e3a40 100644
--- a/samples/bpf/xdp_redirect_cpu.bpf.c
+++ b/samples/bpf/xdp_redirect_cpu.bpf.c
@@ -100,7 +100,6 @@ u16 get_dest_port_ipv4_udp(struct xdp_md *ctx, u64 nh_off)
void *data = (void *)(long)ctx->data;
struct iphdr *iph = data + nh_off;
struct udphdr *udph;
- u16 dport;
if (iph + 1 > data_end)
return 0;
@@ -111,8 +110,7 @@ u16 get_dest_port_ipv4_udp(struct xdp_md *ctx, u64 nh_off)
if (udph + 1 > data_end)
return 0;
- dport = bpf_ntohs(udph->dest);
- return dport;
+ return bpf_ntohs(udph->dest);
}
static __always_inline
diff --git a/samples/bpf/xdp_sample_pkts_user.c b/samples/bpf/xdp_sample_pkts_user.c
index f4382cc..587eacb 100644
--- a/samples/bpf/xdp_sample_pkts_user.c
+++ b/samples/bpf/xdp_sample_pkts_user.c
@@ -110,12 +110,9 @@ static void usage(const char *prog)
int main(int argc, char **argv)
{
- struct bpf_prog_load_attr prog_load_attr = {
- .prog_type = BPF_PROG_TYPE_XDP,
- };
- struct perf_buffer_opts pb_opts = {};
const char *optstr = "FS";
int prog_fd, map_fd, opt;
+ struct bpf_program *prog;
struct bpf_object *obj;
struct bpf_map *map;
char filename[256];
@@ -144,15 +141,19 @@ int main(int argc, char **argv)
}
snprintf(filename, sizeof(filename), "%s_kern.o", argv[0]);
- prog_load_attr.file = filename;
- if (bpf_prog_load_xattr(&prog_load_attr, &obj, &prog_fd))
+ obj = bpf_object__open_file(filename, NULL);
+ if (libbpf_get_error(obj))
return 1;
- if (!prog_fd) {
- printf("bpf_prog_load_xattr: %s\n", strerror(errno));
+ prog = bpf_object__next_program(obj, NULL);
+ bpf_program__set_type(prog, BPF_PROG_TYPE_XDP);
+
+ err = bpf_object__load(obj);
+ if (err)
return 1;
- }
+
+ prog_fd = bpf_program__fd(prog);
map = bpf_object__next_map(obj, NULL);
if (!map) {
@@ -181,8 +182,7 @@ int main(int argc, char **argv)
return 1;
}
- pb_opts.sample_cb = print_bpf_output;
- pb = perf_buffer__new(map_fd, 8, &pb_opts);
+ pb = perf_buffer__new(map_fd, 8, print_bpf_output, NULL, NULL, NULL);
err = libbpf_get_error(pb);
if (err) {
perror("perf_buffer setup failed");
diff --git a/samples/bpf/xdp_sample_user.h b/samples/bpf/xdp_sample_user.h
index d97465f..5f44b87 100644
--- a/samples/bpf/xdp_sample_user.h
+++ b/samples/bpf/xdp_sample_user.h
@@ -45,7 +45,9 @@ const char *get_driver_name(int ifindex);
int get_mac_addr(int ifindex, void *mac_addr);
#pragma GCC diagnostic push
+#ifndef __clang__
#pragma GCC diagnostic ignored "-Wstringop-truncation"
+#endif
__attribute__((unused))
static inline char *safe_strncpy(char *dst, const char *src, size_t size)
{
diff --git a/samples/bpf/xdpsock_ctrl_proc.c b/samples/bpf/xdpsock_ctrl_proc.c
index 384e62e..cc44087 100644
--- a/samples/bpf/xdpsock_ctrl_proc.c
+++ b/samples/bpf/xdpsock_ctrl_proc.c
@@ -15,6 +15,9 @@
#include <bpf/xsk.h>
#include "xdpsock.h"
+/* libbpf APIs for AF_XDP are deprecated starting from v0.7 */
+#pragma GCC diagnostic ignored "-Wdeprecated-declarations"
+
static const char *opt_if = "";
static struct option long_options[] = {
diff --git a/samples/bpf/xdpsock_user.c b/samples/bpf/xdpsock_user.c
index 49d7a6a..aa50864 100644
--- a/samples/bpf/xdpsock_user.c
+++ b/samples/bpf/xdpsock_user.c
@@ -14,6 +14,7 @@
#include <arpa/inet.h>
#include <locale.h>
#include <net/ethernet.h>
+#include <netinet/ether.h>
#include <net/if.h>
#include <poll.h>
#include <pthread.h>
@@ -30,12 +31,16 @@
#include <sys/un.h>
#include <time.h>
#include <unistd.h>
+#include <sched.h>
#include <bpf/libbpf.h>
#include <bpf/xsk.h>
#include <bpf/bpf.h>
#include "xdpsock.h"
+/* libbpf APIs for AF_XDP are deprecated starting from v0.7 */
+#pragma GCC diagnostic ignored "-Wdeprecated-declarations"
+
#ifndef SOL_XDP
#define SOL_XDP 283
#endif
@@ -53,12 +58,27 @@
#define DEBUG_HEXDUMP 0
+#define VLAN_PRIO_MASK 0xe000 /* Priority Code Point */
+#define VLAN_PRIO_SHIFT 13
+#define VLAN_VID_MASK 0x0fff /* VLAN Identifier */
+#define VLAN_VID__DEFAULT 1
+#define VLAN_PRI__DEFAULT 0
+
+#define NSEC_PER_SEC 1000000000UL
+#define NSEC_PER_USEC 1000
+
+#define SCHED_PRI__DEFAULT 0
+
typedef __u64 u64;
typedef __u32 u32;
typedef __u16 u16;
typedef __u8 u8;
static unsigned long prev_time;
+static long tx_cycle_diff_min;
+static long tx_cycle_diff_max;
+static double tx_cycle_diff_ave;
+static long tx_cycle_cnt;
enum benchmark_type {
BENCH_RXDROP = 0,
@@ -78,14 +98,23 @@ static u32 opt_batch_size = 64;
static int opt_pkt_count;
static u16 opt_pkt_size = MIN_PKT_SIZE;
static u32 opt_pkt_fill_pattern = 0x12345678;
+static bool opt_vlan_tag;
+static u16 opt_pkt_vlan_id = VLAN_VID__DEFAULT;
+static u16 opt_pkt_vlan_pri = VLAN_PRI__DEFAULT;
+static struct ether_addr opt_txdmac = {{ 0x3c, 0xfd, 0xfe,
+ 0x9e, 0x7f, 0x71 }};
+static struct ether_addr opt_txsmac = {{ 0xec, 0xb1, 0xd7,
+ 0x98, 0x3a, 0xc0 }};
static bool opt_extra_stats;
static bool opt_quiet;
static bool opt_app_stats;
static const char *opt_irq_str = "";
static u32 irq_no;
static int irqs_at_init = -1;
+static u32 sequence;
static int opt_poll;
static int opt_interval = 1;
+static int opt_retries = 3;
static u32 opt_xdp_bind_flags = XDP_USE_NEED_WAKEUP;
static u32 opt_umem_flags;
static int opt_unaligned_chunks;
@@ -97,6 +126,27 @@ static u32 opt_num_xsks = 1;
static u32 prog_id;
static bool opt_busy_poll;
static bool opt_reduced_cap;
+static clockid_t opt_clock = CLOCK_MONOTONIC;
+static unsigned long opt_tx_cycle_ns;
+static int opt_schpolicy = SCHED_OTHER;
+static int opt_schprio = SCHED_PRI__DEFAULT;
+static bool opt_tstamp;
+
+struct vlan_ethhdr {
+ unsigned char h_dest[6];
+ unsigned char h_source[6];
+ __be16 h_vlan_proto;
+ __be16 h_vlan_TCI;
+ __be16 h_vlan_encapsulated_proto;
+};
+
+#define PKTGEN_MAGIC 0xbe9be955
+struct pktgen_hdr {
+ __be32 pgh_magic;
+ __be32 seq_num;
+ __be32 tv_sec;
+ __be32 tv_usec;
+};
struct xsk_ring_stats {
unsigned long rx_npkts;
@@ -153,15 +203,63 @@ struct xsk_socket_info {
u32 outstanding_tx;
};
+static const struct clockid_map {
+ const char *name;
+ clockid_t clockid;
+} clockids_map[] = {
+ { "REALTIME", CLOCK_REALTIME },
+ { "TAI", CLOCK_TAI },
+ { "BOOTTIME", CLOCK_BOOTTIME },
+ { "MONOTONIC", CLOCK_MONOTONIC },
+ { NULL }
+};
+
+static const struct sched_map {
+ const char *name;
+ int policy;
+} schmap[] = {
+ { "OTHER", SCHED_OTHER },
+ { "FIFO", SCHED_FIFO },
+ { NULL }
+};
+
static int num_socks;
struct xsk_socket_info *xsks[MAX_SOCKS];
int sock;
+static int get_clockid(clockid_t *id, const char *name)
+{
+ const struct clockid_map *clk;
+
+ for (clk = clockids_map; clk->name; clk++) {
+ if (strcasecmp(clk->name, name) == 0) {
+ *id = clk->clockid;
+ return 0;
+ }
+ }
+
+ return -1;
+}
+
+static int get_schpolicy(int *policy, const char *name)
+{
+ const struct sched_map *sch;
+
+ for (sch = schmap; sch->name; sch++) {
+ if (strcasecmp(sch->name, name) == 0) {
+ *policy = sch->policy;
+ return 0;
+ }
+ }
+
+ return -1;
+}
+
static unsigned long get_nsecs(void)
{
struct timespec ts;
- clock_gettime(CLOCK_MONOTONIC, &ts);
+ clock_gettime(opt_clock, &ts);
return ts.tv_sec * 1000000000UL + ts.tv_nsec;
}
@@ -254,6 +352,15 @@ static void dump_app_stats(long dt)
xsks[i]->app_stats.prev_tx_wakeup_sendtos = xsks[i]->app_stats.tx_wakeup_sendtos;
xsks[i]->app_stats.prev_opt_polls = xsks[i]->app_stats.opt_polls;
}
+
+ if (opt_tx_cycle_ns) {
+ printf("\n%-18s %-10s %-10s %-10s %-10s %-10s\n",
+ "", "period", "min", "ave", "max", "cycle");
+ printf("%-18s %-10lu %-10lu %-10lu %-10lu %-10lu\n",
+ "Cyclic TX", opt_tx_cycle_ns, tx_cycle_diff_min,
+ (long)(tx_cycle_diff_ave / tx_cycle_cnt),
+ tx_cycle_diff_max, tx_cycle_cnt);
+ }
}
static bool get_interrupt_number(void)
@@ -737,29 +844,69 @@ static inline u16 udp_csum(u32 saddr, u32 daddr, u32 len,
#define ETH_FCS_SIZE 4
-#define PKT_HDR_SIZE (sizeof(struct ethhdr) + sizeof(struct iphdr) + \
- sizeof(struct udphdr))
+#define ETH_HDR_SIZE (opt_vlan_tag ? sizeof(struct vlan_ethhdr) : \
+ sizeof(struct ethhdr))
+#define PKTGEN_HDR_SIZE (opt_tstamp ? sizeof(struct pktgen_hdr) : 0)
+#define PKT_HDR_SIZE (ETH_HDR_SIZE + sizeof(struct iphdr) + \
+ sizeof(struct udphdr) + PKTGEN_HDR_SIZE)
+#define PKTGEN_HDR_OFFSET (ETH_HDR_SIZE + sizeof(struct iphdr) + \
+ sizeof(struct udphdr))
+#define PKTGEN_SIZE_MIN (PKTGEN_HDR_OFFSET + sizeof(struct pktgen_hdr) + \
+ ETH_FCS_SIZE)
#define PKT_SIZE (opt_pkt_size - ETH_FCS_SIZE)
-#define IP_PKT_SIZE (PKT_SIZE - sizeof(struct ethhdr))
+#define IP_PKT_SIZE (PKT_SIZE - ETH_HDR_SIZE)
#define UDP_PKT_SIZE (IP_PKT_SIZE - sizeof(struct iphdr))
-#define UDP_PKT_DATA_SIZE (UDP_PKT_SIZE - sizeof(struct udphdr))
+#define UDP_PKT_DATA_SIZE (UDP_PKT_SIZE - \
+ (sizeof(struct udphdr) + PKTGEN_HDR_SIZE))
static u8 pkt_data[XSK_UMEM__DEFAULT_FRAME_SIZE];
static void gen_eth_hdr_data(void)
{
- struct udphdr *udp_hdr = (struct udphdr *)(pkt_data +
- sizeof(struct ethhdr) +
- sizeof(struct iphdr));
- struct iphdr *ip_hdr = (struct iphdr *)(pkt_data +
- sizeof(struct ethhdr));
- struct ethhdr *eth_hdr = (struct ethhdr *)pkt_data;
+ struct pktgen_hdr *pktgen_hdr;
+ struct udphdr *udp_hdr;
+ struct iphdr *ip_hdr;
- /* ethernet header */
- memcpy(eth_hdr->h_dest, "\x3c\xfd\xfe\x9e\x7f\x71", ETH_ALEN);
- memcpy(eth_hdr->h_source, "\xec\xb1\xd7\x98\x3a\xc0", ETH_ALEN);
- eth_hdr->h_proto = htons(ETH_P_IP);
+ if (opt_vlan_tag) {
+ struct vlan_ethhdr *veth_hdr = (struct vlan_ethhdr *)pkt_data;
+ u16 vlan_tci = 0;
+
+ udp_hdr = (struct udphdr *)(pkt_data +
+ sizeof(struct vlan_ethhdr) +
+ sizeof(struct iphdr));
+ ip_hdr = (struct iphdr *)(pkt_data +
+ sizeof(struct vlan_ethhdr));
+ pktgen_hdr = (struct pktgen_hdr *)(pkt_data +
+ sizeof(struct vlan_ethhdr) +
+ sizeof(struct iphdr) +
+ sizeof(struct udphdr));
+ /* ethernet & VLAN header */
+ memcpy(veth_hdr->h_dest, &opt_txdmac, ETH_ALEN);
+ memcpy(veth_hdr->h_source, &opt_txsmac, ETH_ALEN);
+ veth_hdr->h_vlan_proto = htons(ETH_P_8021Q);
+ vlan_tci = opt_pkt_vlan_id & VLAN_VID_MASK;
+ vlan_tci |= (opt_pkt_vlan_pri << VLAN_PRIO_SHIFT) & VLAN_PRIO_MASK;
+ veth_hdr->h_vlan_TCI = htons(vlan_tci);
+ veth_hdr->h_vlan_encapsulated_proto = htons(ETH_P_IP);
+ } else {
+ struct ethhdr *eth_hdr = (struct ethhdr *)pkt_data;
+
+ udp_hdr = (struct udphdr *)(pkt_data +
+ sizeof(struct ethhdr) +
+ sizeof(struct iphdr));
+ ip_hdr = (struct iphdr *)(pkt_data +
+ sizeof(struct ethhdr));
+ pktgen_hdr = (struct pktgen_hdr *)(pkt_data +
+ sizeof(struct ethhdr) +
+ sizeof(struct iphdr) +
+ sizeof(struct udphdr));
+ /* ethernet header */
+ memcpy(eth_hdr->h_dest, &opt_txdmac, ETH_ALEN);
+ memcpy(eth_hdr->h_source, &opt_txsmac, ETH_ALEN);
+ eth_hdr->h_proto = htons(ETH_P_IP);
+ }
+
/* IP header */
ip_hdr->version = IPVERSION;
@@ -782,6 +929,9 @@ static void gen_eth_hdr_data(void)
udp_hdr->dest = htons(0x1000);
udp_hdr->len = htons(UDP_PKT_SIZE);
+ if (opt_tstamp)
+ pktgen_hdr->pgh_magic = htonl(PKTGEN_MAGIC);
+
/* UDP data */
memset32_htonl(pkt_data + PKT_HDR_SIZE, opt_pkt_fill_pattern,
UDP_PKT_DATA_SIZE);
@@ -905,6 +1055,7 @@ static struct option long_options[] = {
{"xdp-skb", no_argument, 0, 'S'},
{"xdp-native", no_argument, 0, 'N'},
{"interval", required_argument, 0, 'n'},
+ {"retries", required_argument, 0, 'O'},
{"zero-copy", no_argument, 0, 'z'},
{"copy", no_argument, 0, 'c'},
{"frame-size", required_argument, 0, 'f'},
@@ -913,10 +1064,20 @@ static struct option long_options[] = {
{"shared-umem", no_argument, 0, 'M'},
{"force", no_argument, 0, 'F'},
{"duration", required_argument, 0, 'd'},
+ {"clock", required_argument, 0, 'w'},
{"batch-size", required_argument, 0, 'b'},
{"tx-pkt-count", required_argument, 0, 'C'},
{"tx-pkt-size", required_argument, 0, 's'},
{"tx-pkt-pattern", required_argument, 0, 'P'},
+ {"tx-vlan", no_argument, 0, 'V'},
+ {"tx-vlan-id", required_argument, 0, 'J'},
+ {"tx-vlan-pri", required_argument, 0, 'K'},
+ {"tx-dmac", required_argument, 0, 'G'},
+ {"tx-smac", required_argument, 0, 'H'},
+ {"tx-cycle", required_argument, 0, 'T'},
+ {"tstamp", no_argument, 0, 'y'},
+ {"policy", required_argument, 0, 'W'},
+ {"schpri", required_argument, 0, 'U'},
{"extra-stats", no_argument, 0, 'x'},
{"quiet", no_argument, 0, 'Q'},
{"app-stats", no_argument, 0, 'a'},
@@ -940,6 +1101,7 @@ static void usage(const char *prog)
" -S, --xdp-skb=n Use XDP skb-mod\n"
" -N, --xdp-native=n Enforce XDP native mode\n"
" -n, --interval=n Specify statistics update interval (default 1 sec).\n"
+ " -O, --retries=n Specify time-out retries (1s interval) attempt (default 3).\n"
" -z, --zero-copy Force zero-copy mode.\n"
" -c, --copy Force copy mode.\n"
" -m, --no-need-wakeup Turn off use of driver need wakeup flag.\n"
@@ -949,6 +1111,7 @@ static void usage(const char *prog)
" -F, --force Force loading the XDP prog\n"
" -d, --duration=n Duration in secs to run command.\n"
" Default: forever.\n"
+ " -w, --clock=CLOCK Clock NAME (default MONOTONIC).\n"
" -b, --batch-size=n Batch size for sending or receiving\n"
" packets. Default: %d\n"
" -C, --tx-pkt-count=n Number of packets to send.\n"
@@ -957,6 +1120,15 @@ static void usage(const char *prog)
" (Default: %d bytes)\n"
" Min size: %d, Max size %d.\n"
" -P, --tx-pkt-pattern=nPacket fill pattern. Default: 0x%x\n"
+ " -V, --tx-vlan Send VLAN tagged packets (For -t|--txonly)\n"
+ " -J, --tx-vlan-id=n Tx VLAN ID [1-4095]. Default: %d (For -V|--tx-vlan)\n"
+ " -K, --tx-vlan-pri=n Tx VLAN Priority [0-7]. Default: %d (For -V|--tx-vlan)\n"
+ " -G, --tx-dmac=<MAC> Dest MAC addr of TX frame in aa:bb:cc:dd:ee:ff format (For -V|--tx-vlan)\n"
+ " -H, --tx-smac=<MAC> Src MAC addr of TX frame in aa:bb:cc:dd:ee:ff format (For -V|--tx-vlan)\n"
+ " -T, --tx-cycle=n Tx cycle time in micro-seconds (For -t|--txonly).\n"
+ " -y, --tstamp Add time-stamp to packet (For -t|--txonly).\n"
+ " -W, --policy=POLICY Schedule policy. Default: SCHED_OTHER\n"
+ " -U, --schpri=n Schedule priority. Default: %d\n"
" -x, --extra-stats Display extra statistics.\n"
" -Q, --quiet Do not display any stats.\n"
" -a, --app-stats Display application (syscall) statistics.\n"
@@ -966,7 +1138,9 @@ static void usage(const char *prog)
"\n";
fprintf(stderr, str, prog, XSK_UMEM__DEFAULT_FRAME_SIZE,
opt_batch_size, MIN_PKT_SIZE, MIN_PKT_SIZE,
- XSK_UMEM__DEFAULT_FRAME_SIZE, opt_pkt_fill_pattern);
+ XSK_UMEM__DEFAULT_FRAME_SIZE, opt_pkt_fill_pattern,
+ VLAN_VID__DEFAULT, VLAN_PRI__DEFAULT,
+ SCHED_PRI__DEFAULT);
exit(EXIT_FAILURE);
}
@@ -978,7 +1152,8 @@ static void parse_command_line(int argc, char **argv)
opterr = 0;
for (;;) {
- c = getopt_long(argc, argv, "Frtli:q:pSNn:czf:muMd:b:C:s:P:xQaI:BR",
+ c = getopt_long(argc, argv,
+ "Frtli:q:pSNn:w:O:czf:muMd:b:C:s:P:VJ:K:G:H:T:yW:U:xQaI:BR",
long_options, &option_index);
if (c == -1)
break;
@@ -1012,6 +1187,17 @@ static void parse_command_line(int argc, char **argv)
case 'n':
opt_interval = atoi(optarg);
break;
+ case 'w':
+ if (get_clockid(&opt_clock, optarg)) {
+ fprintf(stderr,
+ "ERROR: Invalid clock %s. Default to CLOCK_MONOTONIC.\n",
+ optarg);
+ opt_clock = CLOCK_MONOTONIC;
+ }
+ break;
+ case 'O':
+ opt_retries = atoi(optarg);
+ break;
case 'z':
opt_xdp_bind_flags |= XDP_ZEROCOPY;
break;
@@ -1059,6 +1245,49 @@ static void parse_command_line(int argc, char **argv)
case 'P':
opt_pkt_fill_pattern = strtol(optarg, NULL, 16);
break;
+ case 'V':
+ opt_vlan_tag = true;
+ break;
+ case 'J':
+ opt_pkt_vlan_id = atoi(optarg);
+ break;
+ case 'K':
+ opt_pkt_vlan_pri = atoi(optarg);
+ break;
+ case 'G':
+ if (!ether_aton_r(optarg,
+ (struct ether_addr *)&opt_txdmac)) {
+ fprintf(stderr, "Invalid dmac address:%s\n",
+ optarg);
+ usage(basename(argv[0]));
+ }
+ break;
+ case 'H':
+ if (!ether_aton_r(optarg,
+ (struct ether_addr *)&opt_txsmac)) {
+ fprintf(stderr, "Invalid smac address:%s\n",
+ optarg);
+ usage(basename(argv[0]));
+ }
+ break;
+ case 'T':
+ opt_tx_cycle_ns = atoi(optarg);
+ opt_tx_cycle_ns *= NSEC_PER_USEC;
+ break;
+ case 'y':
+ opt_tstamp = 1;
+ break;
+ case 'W':
+ if (get_schpolicy(&opt_schpolicy, optarg)) {
+ fprintf(stderr,
+ "ERROR: Invalid policy %s. Default to SCHED_OTHER.\n",
+ optarg);
+ opt_schpolicy = SCHED_OTHER;
+ }
+ break;
+ case 'U':
+ opt_schprio = atoi(optarg);
+ break;
case 'x':
opt_extra_stats = 1;
break;
@@ -1264,16 +1493,22 @@ static void rx_drop_all(void)
}
}
-static void tx_only(struct xsk_socket_info *xsk, u32 *frame_nb, int batch_size)
+static int tx_only(struct xsk_socket_info *xsk, u32 *frame_nb,
+ int batch_size, unsigned long tx_ns)
{
- u32 idx;
+ u32 idx, tv_sec, tv_usec;
unsigned int i;
while (xsk_ring_prod__reserve(&xsk->tx, batch_size, &idx) <
batch_size) {
complete_tx_only(xsk, batch_size);
if (benchmark_done)
- return;
+ return 0;
+ }
+
+ if (opt_tstamp) {
+ tv_sec = (u32)(tx_ns / NSEC_PER_SEC);
+ tv_usec = (u32)((tx_ns % NSEC_PER_SEC) / 1000);
}
for (i = 0; i < batch_size; i++) {
@@ -1281,6 +1516,21 @@ static void tx_only(struct xsk_socket_info *xsk, u32 *frame_nb, int batch_size)
idx + i);
tx_desc->addr = (*frame_nb + i) * opt_xsk_frame_size;
tx_desc->len = PKT_SIZE;
+
+ if (opt_tstamp) {
+ struct pktgen_hdr *pktgen_hdr;
+ u64 addr = tx_desc->addr;
+ char *pkt;
+
+ pkt = xsk_umem__get_data(xsk->umem->buffer, addr);
+ pktgen_hdr = (struct pktgen_hdr *)(pkt + PKTGEN_HDR_OFFSET);
+
+ pktgen_hdr->seq_num = htonl(sequence++);
+ pktgen_hdr->tv_sec = htonl(tv_sec);
+ pktgen_hdr->tv_usec = htonl(tv_usec);
+
+ hex_dump(pkt, PKT_SIZE, addr);
+ }
}
xsk_ring_prod__submit(&xsk->tx, batch_size);
@@ -1289,6 +1539,8 @@ static void tx_only(struct xsk_socket_info *xsk, u32 *frame_nb, int batch_size)
*frame_nb += batch_size;
*frame_nb %= NUM_FRAMES;
complete_tx_only(xsk, batch_size);
+
+ return batch_size;
}
static inline int get_batch_size(int pkt_cnt)
@@ -1315,23 +1567,48 @@ static void complete_tx_only_all(void)
pending = !!xsks[i]->outstanding_tx;
}
}
- } while (pending);
+ sleep(1);
+ } while (pending && opt_retries-- > 0);
}
static void tx_only_all(void)
{
struct pollfd fds[MAX_SOCKS] = {};
u32 frame_nb[MAX_SOCKS] = {};
+ unsigned long next_tx_ns = 0;
int pkt_cnt = 0;
int i, ret;
+ if (opt_poll && opt_tx_cycle_ns) {
+ fprintf(stderr,
+ "Error: --poll and --tx-cycles are both set\n");
+ return;
+ }
+
for (i = 0; i < num_socks; i++) {
fds[0].fd = xsk_socket__fd(xsks[i]->xsk);
fds[0].events = POLLOUT;
}
+ if (opt_tx_cycle_ns) {
+ /* Align Tx time to micro-second boundary */
+ next_tx_ns = (get_nsecs() / NSEC_PER_USEC + 1) *
+ NSEC_PER_USEC;
+ next_tx_ns += opt_tx_cycle_ns;
+
+ /* Initialize periodic Tx scheduling variance */
+ tx_cycle_diff_min = 1000000000;
+ tx_cycle_diff_max = 0;
+ tx_cycle_diff_ave = 0.0;
+ }
+
while ((opt_pkt_count && pkt_cnt < opt_pkt_count) || !opt_pkt_count) {
int batch_size = get_batch_size(pkt_cnt);
+ unsigned long tx_ns = 0;
+ struct timespec next;
+ int tx_cnt = 0;
+ long diff;
+ int err;
if (opt_poll) {
for (i = 0; i < num_socks; i++)
@@ -1344,13 +1621,43 @@ static void tx_only_all(void)
continue;
}
- for (i = 0; i < num_socks; i++)
- tx_only(xsks[i], &frame_nb[i], batch_size);
+ if (opt_tx_cycle_ns) {
+ next.tv_sec = next_tx_ns / NSEC_PER_SEC;
+ next.tv_nsec = next_tx_ns % NSEC_PER_SEC;
+ err = clock_nanosleep(opt_clock, TIMER_ABSTIME, &next, NULL);
+ if (err) {
+ if (err != EINTR)
+ fprintf(stderr,
+ "clock_nanosleep failed. Err:%d errno:%d\n",
+ err, errno);
+ break;
+ }
- pkt_cnt += batch_size;
+ /* Measure periodic Tx scheduling variance */
+ tx_ns = get_nsecs();
+ diff = tx_ns - next_tx_ns;
+ if (diff < tx_cycle_diff_min)
+ tx_cycle_diff_min = diff;
+
+ if (diff > tx_cycle_diff_max)
+ tx_cycle_diff_max = diff;
+
+ tx_cycle_diff_ave += (double)diff;
+ tx_cycle_cnt++;
+ } else if (opt_tstamp) {
+ tx_ns = get_nsecs();
+ }
+
+ for (i = 0; i < num_socks; i++)
+ tx_cnt += tx_only(xsks[i], &frame_nb[i], batch_size, tx_ns);
+
+ pkt_cnt += tx_cnt;
if (benchmark_done)
break;
+
+ if (opt_tx_cycle_ns)
+ next_tx_ns += opt_tx_cycle_ns;
}
if (opt_pkt_count)
@@ -1581,6 +1888,7 @@ int main(int argc, char **argv)
struct __user_cap_data_struct data[2] = { { 0 } };
struct rlimit r = {RLIM_INFINITY, RLIM_INFINITY};
bool rx = false, tx = false;
+ struct sched_param schparam;
struct xsk_umem_info *umem;
struct bpf_object *obj;
int xsks_map_fd = 0;
@@ -1643,6 +1951,9 @@ int main(int argc, char **argv)
apply_setsockopt(xsks[i]);
if (opt_bench == BENCH_TXONLY) {
+ if (opt_tstamp && opt_pkt_size < PKTGEN_SIZE_MIN)
+ opt_pkt_size = PKTGEN_SIZE_MIN;
+
gen_eth_hdr_data();
for (i = 0; i < NUM_FRAMES; i++)
@@ -1682,6 +1993,16 @@ int main(int argc, char **argv)
prev_time = get_nsecs();
start_time = prev_time;
+ /* Configure sched priority for better wake-up accuracy */
+ memset(&schparam, 0, sizeof(schparam));
+ schparam.sched_priority = opt_schprio;
+ ret = sched_setscheduler(0, opt_schpolicy, &schparam);
+ if (ret) {
+ fprintf(stderr, "Error(%d) in setting priority(%d): %s\n",
+ errno, opt_schprio, strerror(errno));
+ goto out;
+ }
+
if (opt_bench == BENCH_RXDROP)
rx_drop_all();
else if (opt_bench == BENCH_TXONLY)
@@ -1689,6 +2010,7 @@ int main(int argc, char **argv)
else
l2fwd_all();
+out:
benchmark_done = true;
if (!opt_quiet)
diff --git a/samples/bpf/xsk_fwd.c b/samples/bpf/xsk_fwd.c
index 1cd97c8..52e7c4f 100644
--- a/samples/bpf/xsk_fwd.c
+++ b/samples/bpf/xsk_fwd.c
@@ -27,6 +27,9 @@
#include <bpf/xsk.h>
#include <bpf/bpf.h>
+/* libbpf APIs for AF_XDP are deprecated starting from v0.7 */
+#pragma GCC diagnostic ignored "-Wdeprecated-declarations"
+
#define ARRAY_SIZE(x) (sizeof(x) / sizeof((x)[0]))
typedef __u64 u64;
diff --git a/samples/ftrace/ftrace-direct-modify.c b/samples/ftrace/ftrace-direct-modify.c
index 690e4a9..2877cb0 100644
--- a/samples/ftrace/ftrace-direct-modify.c
+++ b/samples/ftrace/ftrace-direct-modify.c
@@ -4,6 +4,9 @@
#include <linux/ftrace.h>
#include <asm/asm-offsets.h>
+extern void my_direct_func1(void);
+extern void my_direct_func2(void);
+
void my_direct_func1(void)
{
trace_printk("my direct func1\n");
diff --git a/samples/ftrace/ftrace-direct-multi-modify.c b/samples/ftrace/ftrace-direct-multi-modify.c
index 91bc42a..6f43a39 100644
--- a/samples/ftrace/ftrace-direct-multi-modify.c
+++ b/samples/ftrace/ftrace-direct-multi-modify.c
@@ -4,6 +4,9 @@
#include <linux/ftrace.h>
#include <asm/asm-offsets.h>
+extern void my_direct_func1(unsigned long ip);
+extern void my_direct_func2(unsigned long ip);
+
void my_direct_func1(unsigned long ip)
{
trace_printk("my direct func1 ip %lx\n", ip);
diff --git a/samples/ftrace/ftrace-direct-too.c b/samples/ftrace/ftrace-direct-too.c
index 6e0de72..b97e5ed 100644
--- a/samples/ftrace/ftrace-direct-too.c
+++ b/samples/ftrace/ftrace-direct-too.c
@@ -5,6 +5,9 @@
#include <linux/ftrace.h>
#include <asm/asm-offsets.h>
+extern void my_direct_func(struct vm_area_struct *vma,
+ unsigned long address, unsigned int flags);
+
void my_direct_func(struct vm_area_struct *vma,
unsigned long address, unsigned int flags)
{
diff --git a/samples/ftrace/ftrace-direct.c b/samples/ftrace/ftrace-direct.c
index a30aa42..c918b13 100644
--- a/samples/ftrace/ftrace-direct.c
+++ b/samples/ftrace/ftrace-direct.c
@@ -5,6 +5,8 @@
#include <linux/ftrace.h>
#include <asm/asm-offsets.h>
+extern void my_direct_func(struct task_struct *p);
+
void my_direct_func(struct task_struct *p)
{
trace_printk("waking up %s-%d\n", p->comm, p->pid);
diff --git a/scripts/recordmcount.pl b/scripts/recordmcount.pl
index 52a000b..3ccb2c7 100755
--- a/scripts/recordmcount.pl
+++ b/scripts/recordmcount.pl
@@ -219,7 +219,7 @@
} elsif ($arch eq "s390" && $bits == 64) {
if ($cc =~ /-DCC_USING_HOTPATCH/) {
- $mcount_regex = "^\\s*([0-9a-fA-F]+):\\s*c0 04 00 00 00 00\\s*(bcrl\\s*0,|jgnop\\s*)[0-9a-f]+ <([^\+]*)>\$";
+ $mcount_regex = "^\\s*([0-9a-fA-F]+):\\s*c0 04 00 00 00 00\\s*(brcl\\s*0,|jgnop\\s*)[0-9a-f]+ <([^\+]*)>\$";
$mcount_adjust = 0;
}
$alignment = 8;
diff --git a/security/device_cgroup.c b/security/device_cgroup.c
index 04375df..842889f 100644
--- a/security/device_cgroup.c
+++ b/security/device_cgroup.c
@@ -5,6 +5,7 @@
* Copyright 2007 IBM Corp
*/
+#include <linux/bpf-cgroup.h>
#include <linux/device_cgroup.h>
#include <linux/cgroup.h>
#include <linux/ctype.h>
diff --git a/security/selinux/hooks.c b/security/selinux/hooks.c
index 1afc06f..dde4ecc 100644
--- a/security/selinux/hooks.c
+++ b/security/selinux/hooks.c
@@ -5785,7 +5785,7 @@ static unsigned int selinux_ip_postroute_compat(struct sk_buff *skb,
struct sk_security_struct *sksec;
struct common_audit_data ad;
struct lsm_network_audit net = {0,};
- u8 proto;
+ u8 proto = 0;
sk = skb_to_full_sk(skb);
if (sk == NULL)
diff --git a/tools/bpf/bpftool/.gitignore b/tools/bpf/bpftool/.gitignore
index 05ce444..a736f64 100644
--- a/tools/bpf/bpftool/.gitignore
+++ b/tools/bpf/bpftool/.gitignore
@@ -1,4 +1,4 @@
-# SPDX-License-Identifier: GPL-2.0-only
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
*.d
/bootstrap/
/bpftool
diff --git a/tools/bpf/bpftool/Documentation/Makefile b/tools/bpf/bpftool/Documentation/Makefile
index c494879..ac8487d 100644
--- a/tools/bpf/bpftool/Documentation/Makefile
+++ b/tools/bpf/bpftool/Documentation/Makefile
@@ -1,6 +1,5 @@
-# SPDX-License-Identifier: GPL-2.0-only
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
include ../../../scripts/Makefile.include
-include ../../../scripts/utilities.mak
INSTALL ?= install
RM ?= rm -f
@@ -25,7 +24,7 @@
man8: $(DOC_MAN8)
RST2MAN_DEP := $(shell command -v rst2man 2>/dev/null)
-RST2MAN_OPTS += --verbose
+RST2MAN_OPTS += --verbose --strip-comments
list_pages = $(sort $(basename $(filter-out $(1),$(MAN8_RST))))
see_also = $(subst " ",, \
diff --git a/tools/bpf/bpftool/Documentation/bpftool-btf.rst b/tools/bpf/bpftool/Documentation/bpftool-btf.rst
index 88b28aa..342716f 100644
--- a/tools/bpf/bpftool/Documentation/bpftool-btf.rst
+++ b/tools/bpf/bpftool/Documentation/bpftool-btf.rst
@@ -1,3 +1,5 @@
+.. SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+
================
bpftool-btf
================
@@ -7,13 +9,14 @@
:Manual section: 8
+.. include:: substitutions.rst
+
SYNOPSIS
========
**bpftool** [*OPTIONS*] **btf** *COMMAND*
- *OPTIONS* := { { **-j** | **--json** } [{ **-p** | **--pretty** }] | {**-d** | **--debug** } |
- { **-B** | **--base-btf** } }
+ *OPTIONS* := { |COMMON_OPTIONS| | { **-B** | **--base-btf** } }
*COMMANDS* := { **dump** | **help** }
diff --git a/tools/bpf/bpftool/Documentation/bpftool-cgroup.rst b/tools/bpf/bpftool/Documentation/bpftool-cgroup.rst
index 3e4395e..a17e9aa 100644
--- a/tools/bpf/bpftool/Documentation/bpftool-cgroup.rst
+++ b/tools/bpf/bpftool/Documentation/bpftool-cgroup.rst
@@ -1,3 +1,5 @@
+.. SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+
================
bpftool-cgroup
================
@@ -7,13 +9,14 @@
:Manual section: 8
+.. include:: substitutions.rst
+
SYNOPSIS
========
**bpftool** [*OPTIONS*] **cgroup** *COMMAND*
- *OPTIONS* := { { **-j** | **--json** } [{ **-p** | **--pretty** }] | { **-d** | **--debug** } |
- { **-f** | **--bpffs** } }
+ *OPTIONS* := { |COMMON_OPTIONS| | { **-f** | **--bpffs** } }
*COMMANDS* :=
{ **show** | **list** | **tree** | **attach** | **detach** | **help** }
@@ -30,9 +33,9 @@
| *PROG* := { **id** *PROG_ID* | **pinned** *FILE* | **tag** *PROG_TAG* }
| *ATTACH_TYPE* := { **ingress** | **egress** | **sock_create** | **sock_ops** | **device** |
| **bind4** | **bind6** | **post_bind4** | **post_bind6** | **connect4** | **connect6** |
-| **getpeername4** | **getpeername6** | **getsockname4** | **getsockname6** | **sendmsg4** |
-| **sendmsg6** | **recvmsg4** | **recvmsg6** | **sysctl** | **getsockopt** | **setsockopt** |
-| **sock_release** }
+| **getpeername4** | **getpeername6** | **getsockname4** | **getsockname6** | **sendmsg4** |
+| **sendmsg6** | **recvmsg4** | **recvmsg6** | **sysctl** | **getsockopt** | **setsockopt** |
+| **sock_release** }
| *ATTACH_FLAGS* := { **multi** | **override** }
DESCRIPTION
@@ -98,9 +101,9 @@
**sendmsg6** call to sendto(2), sendmsg(2), sendmmsg(2) for an
unconnected udp6 socket (since 4.18);
**recvmsg4** call to recvfrom(2), recvmsg(2), recvmmsg(2) for
- an unconnected udp4 socket (since 5.2);
+ an unconnected udp4 socket (since 5.2);
**recvmsg6** call to recvfrom(2), recvmsg(2), recvmmsg(2) for
- an unconnected udp6 socket (since 5.2);
+ an unconnected udp6 socket (since 5.2);
**sysctl** sysctl access (since 5.2);
**getsockopt** call to getsockopt (since 5.3);
**setsockopt** call to setsockopt (since 5.3);
diff --git a/tools/bpf/bpftool/Documentation/bpftool-feature.rst b/tools/bpf/bpftool/Documentation/bpftool-feature.rst
index ab9f57ee..4ce9a77 100644
--- a/tools/bpf/bpftool/Documentation/bpftool-feature.rst
+++ b/tools/bpf/bpftool/Documentation/bpftool-feature.rst
@@ -1,3 +1,5 @@
+.. SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+
===============
bpftool-feature
===============
@@ -7,12 +9,14 @@
:Manual section: 8
+.. include:: substitutions.rst
+
SYNOPSIS
========
**bpftool** [*OPTIONS*] **feature** *COMMAND*
- *OPTIONS* := { { **-j** | **--json** } [{ **-p** | **--pretty** }] | { **-d** | **--debug** } }
+ *OPTIONS* := { |COMMON_OPTIONS| }
*COMMANDS* := { **probe** | **help** }
diff --git a/tools/bpf/bpftool/Documentation/bpftool-gen.rst b/tools/bpf/bpftool/Documentation/bpftool-gen.rst
index 2ef2f2d..bc27638 100644
--- a/tools/bpf/bpftool/Documentation/bpftool-gen.rst
+++ b/tools/bpf/bpftool/Documentation/bpftool-gen.rst
@@ -1,3 +1,5 @@
+.. SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+
================
bpftool-gen
================
@@ -7,13 +9,14 @@
:Manual section: 8
+.. include:: substitutions.rst
+
SYNOPSIS
========
**bpftool** [*OPTIONS*] **gen** *COMMAND*
- *OPTIONS* := { { **-j** | **--json** } [{ **-p** | **--pretty** }] | { **-d** | **--debug** } |
- { **-L** | **--use-loader** } }
+ *OPTIONS* := { |COMMON_OPTIONS| | { **-L** | **--use-loader** } }
*COMMAND* := { **object** | **skeleton** | **help** }
diff --git a/tools/bpf/bpftool/Documentation/bpftool-iter.rst b/tools/bpf/bpftool/Documentation/bpftool-iter.rst
index 471f363..84839d4 100644
--- a/tools/bpf/bpftool/Documentation/bpftool-iter.rst
+++ b/tools/bpf/bpftool/Documentation/bpftool-iter.rst
@@ -1,3 +1,5 @@
+.. SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+
============
bpftool-iter
============
@@ -7,12 +9,14 @@
:Manual section: 8
+.. include:: substitutions.rst
+
SYNOPSIS
========
**bpftool** [*OPTIONS*] **iter** *COMMAND*
- *OPTIONS* := { { **-j** | **--json** } [{ **-p** | **--pretty** }] | { **-d** | **--debug** } }
+ *OPTIONS* := { |COMMON_OPTIONS| }
*COMMANDS* := { **pin** | **help** }
diff --git a/tools/bpf/bpftool/Documentation/bpftool-link.rst b/tools/bpf/bpftool/Documentation/bpftool-link.rst
index 0de90f0..52a4eee 100644
--- a/tools/bpf/bpftool/Documentation/bpftool-link.rst
+++ b/tools/bpf/bpftool/Documentation/bpftool-link.rst
@@ -1,3 +1,5 @@
+.. SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+
================
bpftool-link
================
@@ -7,13 +9,14 @@
:Manual section: 8
+.. include:: substitutions.rst
+
SYNOPSIS
========
**bpftool** [*OPTIONS*] **link** *COMMAND*
- *OPTIONS* := { { **-j** | **--json** } [{ **-p** | **--pretty** }] | { **-d** | **--debug** } |
- { **-f** | **--bpffs** } | { **-n** | **--nomount** } }
+ *OPTIONS* := { |COMMON_OPTIONS| | { **-f** | **--bpffs** } | { **-n** | **--nomount** } }
*COMMANDS* := { **show** | **list** | **pin** | **help** }
diff --git a/tools/bpf/bpftool/Documentation/bpftool-map.rst b/tools/bpf/bpftool/Documentation/bpftool-map.rst
index d0c4abe..7c188a5 100644
--- a/tools/bpf/bpftool/Documentation/bpftool-map.rst
+++ b/tools/bpf/bpftool/Documentation/bpftool-map.rst
@@ -1,3 +1,5 @@
+.. SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+
================
bpftool-map
================
@@ -7,17 +9,18 @@
:Manual section: 8
+.. include:: substitutions.rst
+
SYNOPSIS
========
**bpftool** [*OPTIONS*] **map** *COMMAND*
- *OPTIONS* := { { **-j** | **--json** } [{ **-p** | **--pretty** }] | { **-d** | **--debug** } |
- { **-f** | **--bpffs** } | { **-n** | **--nomount** } }
+ *OPTIONS* := { |COMMON_OPTIONS| | { **-f** | **--bpffs** } | { **-n** | **--nomount** } }
*COMMANDS* :=
- { **show** | **list** | **create** | **dump** | **update** | **lookup** | **getnext**
- | **delete** | **pin** | **help** }
+ { **show** | **list** | **create** | **dump** | **update** | **lookup** | **getnext** |
+ **delete** | **pin** | **help** }
MAP COMMANDS
=============
@@ -52,7 +55,7 @@
| | **devmap** | **devmap_hash** | **sockmap** | **cpumap** | **xskmap** | **sockhash**
| | **cgroup_storage** | **reuseport_sockarray** | **percpu_cgroup_storage**
| | **queue** | **stack** | **sk_storage** | **struct_ops** | **ringbuf** | **inode_storage**
- | **task_storage** }
+| | **task_storage** | **bloom_filter** }
DESCRIPTION
===========
diff --git a/tools/bpf/bpftool/Documentation/bpftool-net.rst b/tools/bpf/bpftool/Documentation/bpftool-net.rst
index 1ae0375..f4e0a51 100644
--- a/tools/bpf/bpftool/Documentation/bpftool-net.rst
+++ b/tools/bpf/bpftool/Documentation/bpftool-net.rst
@@ -1,3 +1,5 @@
+.. SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+
================
bpftool-net
================
@@ -7,12 +9,14 @@
:Manual section: 8
+.. include:: substitutions.rst
+
SYNOPSIS
========
**bpftool** [*OPTIONS*] **net** *COMMAND*
- *OPTIONS* := { { **-j** | **--json** } [{ **-p** | **--pretty** }] | { **-d** | **--debug** } }
+ *OPTIONS* := { |COMMON_OPTIONS| }
*COMMANDS* :=
{ **show** | **list** | **attach** | **detach** | **help** }
@@ -31,44 +35,44 @@
DESCRIPTION
===========
**bpftool net { show | list }** [ **dev** *NAME* ]
- List bpf program attachments in the kernel networking subsystem.
+ List bpf program attachments in the kernel networking subsystem.
- Currently, only device driver xdp attachments and tc filter
- classification/action attachments are implemented, i.e., for
- program types **BPF_PROG_TYPE_SCHED_CLS**,
- **BPF_PROG_TYPE_SCHED_ACT** and **BPF_PROG_TYPE_XDP**.
- For programs attached to a particular cgroup, e.g.,
- **BPF_PROG_TYPE_CGROUP_SKB**, **BPF_PROG_TYPE_CGROUP_SOCK**,
- **BPF_PROG_TYPE_SOCK_OPS** and **BPF_PROG_TYPE_CGROUP_SOCK_ADDR**,
- users can use **bpftool cgroup** to dump cgroup attachments.
- For sk_{filter, skb, msg, reuseport} and lwt/seg6
- bpf programs, users should consult other tools, e.g., iproute2.
+ Currently, only device driver xdp attachments and tc filter
+ classification/action attachments are implemented, i.e., for
+ program types **BPF_PROG_TYPE_SCHED_CLS**,
+ **BPF_PROG_TYPE_SCHED_ACT** and **BPF_PROG_TYPE_XDP**.
+ For programs attached to a particular cgroup, e.g.,
+ **BPF_PROG_TYPE_CGROUP_SKB**, **BPF_PROG_TYPE_CGROUP_SOCK**,
+ **BPF_PROG_TYPE_SOCK_OPS** and **BPF_PROG_TYPE_CGROUP_SOCK_ADDR**,
+ users can use **bpftool cgroup** to dump cgroup attachments.
+ For sk_{filter, skb, msg, reuseport} and lwt/seg6
+ bpf programs, users should consult other tools, e.g., iproute2.
- The current output will start with all xdp program attachments, followed by
- all tc class/qdisc bpf program attachments. Both xdp programs and
- tc programs are ordered based on ifindex number. If multiple bpf
- programs attached to the same networking device through **tc filter**,
- the order will be first all bpf programs attached to tc classes, then
- all bpf programs attached to non clsact qdiscs, and finally all
- bpf programs attached to root and clsact qdisc.
+ The current output will start with all xdp program attachments, followed by
+ all tc class/qdisc bpf program attachments. Both xdp programs and
+ tc programs are ordered based on ifindex number. If multiple bpf
+ programs attached to the same networking device through **tc filter**,
+ the order will be first all bpf programs attached to tc classes, then
+ all bpf programs attached to non clsact qdiscs, and finally all
+ bpf programs attached to root and clsact qdisc.
**bpftool** **net attach** *ATTACH_TYPE* *PROG* **dev** *NAME* [ **overwrite** ]
- Attach bpf program *PROG* to network interface *NAME* with
- type specified by *ATTACH_TYPE*. Previously attached bpf program
- can be replaced by the command used with **overwrite** option.
- Currently, only XDP-related modes are supported for *ATTACH_TYPE*.
+ Attach bpf program *PROG* to network interface *NAME* with
+ type specified by *ATTACH_TYPE*. Previously attached bpf program
+ can be replaced by the command used with **overwrite** option.
+ Currently, only XDP-related modes are supported for *ATTACH_TYPE*.
- *ATTACH_TYPE* can be of:
- **xdp** - try native XDP and fallback to generic XDP if NIC driver does not support it;
- **xdpgeneric** - Generic XDP. runs at generic XDP hook when packet already enters receive path as skb;
- **xdpdrv** - Native XDP. runs earliest point in driver's receive path;
- **xdpoffload** - Offload XDP. runs directly on NIC on each packet reception;
+ *ATTACH_TYPE* can be of:
+ **xdp** - try native XDP and fallback to generic XDP if NIC driver does not support it;
+ **xdpgeneric** - Generic XDP. runs at generic XDP hook when packet already enters receive path as skb;
+ **xdpdrv** - Native XDP. runs earliest point in driver's receive path;
+ **xdpoffload** - Offload XDP. runs directly on NIC on each packet reception;
**bpftool** **net detach** *ATTACH_TYPE* **dev** *NAME*
- Detach bpf program attached to network interface *NAME* with
- type specified by *ATTACH_TYPE*. To detach bpf program, same
- *ATTACH_TYPE* previously used for attach must be specified.
- Currently, only XDP-related modes are supported for *ATTACH_TYPE*.
+ Detach bpf program attached to network interface *NAME* with
+ type specified by *ATTACH_TYPE*. To detach bpf program, same
+ *ATTACH_TYPE* previously used for attach must be specified.
+ Currently, only XDP-related modes are supported for *ATTACH_TYPE*.
**bpftool net help**
Print short help message.
diff --git a/tools/bpf/bpftool/Documentation/bpftool-perf.rst b/tools/bpf/bpftool/Documentation/bpftool-perf.rst
index ce52798..5fea633 100644
--- a/tools/bpf/bpftool/Documentation/bpftool-perf.rst
+++ b/tools/bpf/bpftool/Documentation/bpftool-perf.rst
@@ -1,3 +1,5 @@
+.. SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+
================
bpftool-perf
================
@@ -7,12 +9,14 @@
:Manual section: 8
+.. include:: substitutions.rst
+
SYNOPSIS
========
**bpftool** [*OPTIONS*] **perf** *COMMAND*
- *OPTIONS* := { { **-j** | **--json** } [{ **-p** | **--pretty** }] | { **-d** | **--debug** } }
+ *OPTIONS* := { |COMMON_OPTIONS| }
*COMMANDS* :=
{ **show** | **list** | **help** }
diff --git a/tools/bpf/bpftool/Documentation/bpftool-prog.rst b/tools/bpf/bpftool/Documentation/bpftool-prog.rst
index 91608cb..a2e9359 100644
--- a/tools/bpf/bpftool/Documentation/bpftool-prog.rst
+++ b/tools/bpf/bpftool/Documentation/bpftool-prog.rst
@@ -1,3 +1,5 @@
+.. SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+
================
bpftool-prog
================
@@ -7,18 +9,20 @@
:Manual section: 8
+.. include:: substitutions.rst
+
SYNOPSIS
========
**bpftool** [*OPTIONS*] **prog** *COMMAND*
- *OPTIONS* := { { **-j** | **--json** } [{ **-p** | **--pretty** }] | { **-d** | **--debug** } |
- { **-f** | **--bpffs** } | { **-m** | **--mapcompat** } | { **-n** | **--nomount** } |
- { **-L** | **--use-loader** } }
+ *OPTIONS* := { |COMMON_OPTIONS| |
+ { **-f** | **--bpffs** } | { **-m** | **--mapcompat** } | { **-n** | **--nomount** } |
+ { **-L** | **--use-loader** } }
*COMMANDS* :=
- { **show** | **list** | **dump xlated** | **dump jited** | **pin** | **load**
- | **loadall** | **help** }
+ { **show** | **list** | **dump xlated** | **dump jited** | **pin** | **load** |
+ **loadall** | **help** }
PROG COMMANDS
=============
diff --git a/tools/bpf/bpftool/Documentation/bpftool-struct_ops.rst b/tools/bpf/bpftool/Documentation/bpftool-struct_ops.rst
index 02afc0f..ee53a12 100644
--- a/tools/bpf/bpftool/Documentation/bpftool-struct_ops.rst
+++ b/tools/bpf/bpftool/Documentation/bpftool-struct_ops.rst
@@ -1,3 +1,5 @@
+.. SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+
==================
bpftool-struct_ops
==================
@@ -7,12 +9,14 @@
:Manual section: 8
+.. include:: substitutions.rst
+
SYNOPSIS
========
**bpftool** [*OPTIONS*] **struct_ops** *COMMAND*
- *OPTIONS* := { { **-j** | **--json** } [{ **-p** | **--pretty** }] | { **-d** | **--debug** } }
+ *OPTIONS* := { |COMMON_OPTIONS| }
*COMMANDS* :=
{ **show** | **list** | **dump** | **register** | **unregister** | **help** }
diff --git a/tools/bpf/bpftool/Documentation/bpftool.rst b/tools/bpf/bpftool/Documentation/bpftool.rst
index bb23f55..7084dd9 100644
--- a/tools/bpf/bpftool/Documentation/bpftool.rst
+++ b/tools/bpf/bpftool/Documentation/bpftool.rst
@@ -1,3 +1,5 @@
+.. SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+
================
BPFTOOL
================
@@ -7,6 +9,8 @@
:Manual section: 8
+.. include:: substitutions.rst
+
SYNOPSIS
========
@@ -18,15 +22,14 @@
*OBJECT* := { **map** | **program** | **cgroup** | **perf** | **net** | **feature** }
- *OPTIONS* := { { **-V** | **--version** } |
- { **-j** | **--json** } [{ **-p** | **--pretty** }] | { **-d** | **--debug** } }
+ *OPTIONS* := { { **-V** | **--version** } | |COMMON_OPTIONS| }
*MAP-COMMANDS* :=
{ **show** | **list** | **create** | **dump** | **update** | **lookup** | **getnext** |
- **delete** | **pin** | **event_pipe** | **help** }
+ **delete** | **pin** | **event_pipe** | **help** }
*PROG-COMMANDS* := { **show** | **list** | **dump jited** | **dump xlated** | **pin** |
- **load** | **attach** | **detach** | **help** }
+ **load** | **attach** | **detach** | **help** }
*CGROUP-COMMANDS* := { **show** | **list** | **attach** | **detach** | **help** }
diff --git a/tools/bpf/bpftool/Documentation/common_options.rst b/tools/bpf/bpftool/Documentation/common_options.rst
index 05d06c7..908487b 100644
--- a/tools/bpf/bpftool/Documentation/common_options.rst
+++ b/tools/bpf/bpftool/Documentation/common_options.rst
@@ -1,3 +1,5 @@
+.. SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+
-h, --help
Print short help message (similar to **bpftool help**).
@@ -20,3 +22,12 @@
Print all logs available, even debug-level information. This includes
logs from libbpf as well as from the verifier, when attempting to
load programs.
+
+-l, --legacy
+ Use legacy libbpf mode which has more relaxed BPF program
+ requirements. By default, bpftool has more strict requirements
+ about section names, changes pinning logic and doesn't support
+ some of the older non-BTF map declarations.
+
+ See https://github.com/libbpf/libbpf/wiki/Libbpf:-the-road-to-v1.0
+ for details.
diff --git a/tools/bpf/bpftool/Documentation/substitutions.rst b/tools/bpf/bpftool/Documentation/substitutions.rst
new file mode 100644
index 0000000..ccf1ffa
--- /dev/null
+++ b/tools/bpf/bpftool/Documentation/substitutions.rst
@@ -0,0 +1,3 @@
+.. SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+
+.. |COMMON_OPTIONS| replace:: { **-j** | **--json** } [{ **-p** | **--pretty** }] | { **-d** | **--debug** } | { **-l** | **--legacy** }
diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
index 7cfba11..83369f5 100644
--- a/tools/bpf/bpftool/Makefile
+++ b/tools/bpf/bpftool/Makefile
@@ -1,6 +1,5 @@
-# SPDX-License-Identifier: GPL-2.0-only
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
include ../../scripts/Makefile.include
-include ../../scripts/utilities.mak
ifeq ($(srctree),)
srctree := $(patsubst %/,%,$(dir $(CURDIR)))
@@ -58,7 +57,7 @@
$(LIBBPF_BOOTSTRAP): $(wildcard $(BPF_DIR)/*.[ch] $(BPF_DIR)/Makefile) | $(LIBBPF_BOOTSTRAP_OUTPUT)
$(Q)$(MAKE) -C $(BPF_DIR) OUTPUT=$(LIBBPF_BOOTSTRAP_OUTPUT) \
DESTDIR=$(LIBBPF_BOOTSTRAP_DESTDIR) prefix= \
- ARCH= CC=$(HOSTCC) LD=$(HOSTLD) $@ install_headers
+ ARCH= CROSS_COMPILE= CC=$(HOSTCC) LD=$(HOSTLD) $@ install_headers
$(LIBBPF_BOOTSTRAP_INTERNAL_HDRS): $(LIBBPF_BOOTSTRAP_HDRS_DIR)/%.h: $(BPF_DIR)/%.h | $(LIBBPF_BOOTSTRAP_HDRS_DIR)
$(call QUIET_INSTALL, $@)
@@ -153,6 +152,9 @@
SRCS += $(BFD_SRCS)
endif
+HOST_CFLAGS = $(subst -I$(LIBBPF_INCLUDE),-I$(LIBBPF_BOOTSTRAP_INCLUDE),\
+ $(subst $(CLANG_CROSS_FLAGS),,$(CFLAGS)))
+
BPFTOOL_BOOTSTRAP := $(BOOTSTRAP_OUTPUT)bpftool
BOOTSTRAP_OBJS = $(addprefix $(BOOTSTRAP_OUTPUT),main.o common.o json_writer.o gen.o btf.o xlated_dumper.o btf_dumper.o disasm.o)
@@ -187,7 +189,8 @@
-I$(if $(OUTPUT),$(OUTPUT),.) \
-I$(srctree)/tools/include/uapi/ \
-I$(LIBBPF_BOOTSTRAP_INCLUDE) \
- -g -O2 -Wall -target bpf -c $< -o $@ && $(LLVM_STRIP) -g $@
+ -g -O2 -Wall -target bpf -c $< -o $@
+ $(Q)$(LLVM_STRIP) -g $@
$(OUTPUT)%.skel.h: $(OUTPUT)%.bpf.o $(BPFTOOL_BOOTSTRAP)
$(QUIET_GEN)$(BPFTOOL_BOOTSTRAP) gen skeleton $< > $@
@@ -202,10 +205,10 @@
CFLAGS += $(if $(BUILD_BPF_SKELS),,-DBPFTOOL_WITHOUT_SKELETONS)
$(BOOTSTRAP_OUTPUT)disasm.o: $(srctree)/kernel/bpf/disasm.c
- $(QUIET_CC)$(HOSTCC) $(CFLAGS) -c -MMD -o $@ $<
+ $(QUIET_CC)$(HOSTCC) $(HOST_CFLAGS) -c -MMD $< -o $@
$(OUTPUT)disasm.o: $(srctree)/kernel/bpf/disasm.c
- $(QUIET_CC)$(CC) $(CFLAGS) -c -MMD -o $@ $<
+ $(QUIET_CC)$(CC) $(CFLAGS) -c -MMD $< -o $@
$(OUTPUT)feature.o:
ifneq ($(feature-zlib), 1)
@@ -213,19 +216,16 @@
endif
$(BPFTOOL_BOOTSTRAP): $(BOOTSTRAP_OBJS) $(LIBBPF_BOOTSTRAP)
- $(QUIET_LINK)$(HOSTCC) $(CFLAGS) $(LDFLAGS) -o $@ $(BOOTSTRAP_OBJS) \
- $(LIBS_BOOTSTRAP)
+ $(QUIET_LINK)$(HOSTCC) $(HOST_CFLAGS) $(LDFLAGS) $(BOOTSTRAP_OBJS) $(LIBS_BOOTSTRAP) -o $@
$(OUTPUT)bpftool: $(OBJS) $(LIBBPF)
- $(QUIET_LINK)$(CC) $(CFLAGS) $(LDFLAGS) -o $@ $(OBJS) $(LIBS)
+ $(QUIET_LINK)$(CC) $(CFLAGS) $(LDFLAGS) $(OBJS) $(LIBS) -o $@
$(BOOTSTRAP_OUTPUT)%.o: %.c $(LIBBPF_BOOTSTRAP_INTERNAL_HDRS) | $(BOOTSTRAP_OUTPUT)
- $(QUIET_CC)$(HOSTCC) \
- $(subst -I$(LIBBPF_INCLUDE),-I$(LIBBPF_BOOTSTRAP_INCLUDE),$(CFLAGS)) \
- -c -MMD -o $@ $<
+ $(QUIET_CC)$(HOSTCC) $(HOST_CFLAGS) -c -MMD $< -o $@
$(OUTPUT)%.o: %.c
- $(QUIET_CC)$(CC) $(CFLAGS) -c -MMD -o $@ $<
+ $(QUIET_CC)$(CC) $(CFLAGS) -c -MMD $< -o $@
feature-detect-clean:
$(call QUIET_CLEAN, feature-detect)
diff --git a/tools/bpf/bpftool/bash-completion/bpftool b/tools/bpf/bpftool/bash-completion/bpftool
index 88e2bcf..493753a 100644
--- a/tools/bpf/bpftool/bash-completion/bpftool
+++ b/tools/bpf/bpftool/bash-completion/bpftool
@@ -261,7 +261,7 @@
# Deal with options
if [[ ${words[cword]} == -* ]]; then
local c='--version --json --pretty --bpffs --mapcompat --debug \
- --use-loader --base-btf'
+ --use-loader --base-btf --legacy'
COMPREPLY=( $( compgen -W "$c" -- "$cur" ) )
return 0
fi
@@ -710,7 +710,8 @@
hash_of_maps devmap devmap_hash sockmap cpumap \
xskmap sockhash cgroup_storage reuseport_sockarray \
percpu_cgroup_storage queue stack sk_storage \
- struct_ops inode_storage task_storage ringbuf'
+ struct_ops ringbuf inode_storage task_storage \
+ bloom_filter'
COMPREPLY=( $( compgen -W "$BPFTOOL_MAP_CREATE_TYPES" -- "$cur" ) )
return 0
;;
diff --git a/tools/bpf/bpftool/btf.c b/tools/bpf/bpftool/btf.c
index 015d275..5983312 100644
--- a/tools/bpf/bpftool/btf.c
+++ b/tools/bpf/bpftool/btf.c
@@ -39,6 +39,7 @@ static const char * const btf_kind_str[NR_BTF_KINDS] = {
[BTF_KIND_DATASEC] = "DATASEC",
[BTF_KIND_FLOAT] = "FLOAT",
[BTF_KIND_DECL_TAG] = "DECL_TAG",
+ [BTF_KIND_TYPE_TAG] = "TYPE_TAG",
};
struct btf_attach_point {
@@ -142,6 +143,7 @@ static int dump_btf_type(const struct btf *btf, __u32 id,
case BTF_KIND_VOLATILE:
case BTF_KIND_RESTRICT:
case BTF_KIND_TYPEDEF:
+ case BTF_KIND_TYPE_TAG:
if (json_output)
jsonw_uint_field(w, "type_id", t->type);
else
@@ -418,9 +420,10 @@ static int dump_btf_c(const struct btf *btf,
struct btf_dump *d;
int err = 0, i;
- d = btf_dump__new(btf, NULL, NULL, btf_dump_printf);
- if (IS_ERR(d))
- return PTR_ERR(d);
+ d = btf_dump__new(btf, btf_dump_printf, NULL, NULL);
+ err = libbpf_get_error(d);
+ if (err)
+ return err;
printf("#ifndef __VMLINUX_H__\n");
printf("#define __VMLINUX_H__\n");
@@ -547,8 +550,8 @@ static int do_dump(int argc, char **argv)
}
btf = btf__parse_split(*argv, base ?: base_btf);
- if (IS_ERR(btf)) {
- err = -PTR_ERR(btf);
+ err = libbpf_get_error(btf);
+ if (err) {
btf = NULL;
p_err("failed to load BTF from %s: %s",
*argv, strerror(err));
diff --git a/tools/bpf/bpftool/btf_dumper.c b/tools/bpf/bpftool/btf_dumper.c
index 9c25286..f5dddf8 100644
--- a/tools/bpf/bpftool/btf_dumper.c
+++ b/tools/bpf/bpftool/btf_dumper.c
@@ -32,14 +32,16 @@ static int dump_prog_id_as_func_ptr(const struct btf_dumper *d,
const struct btf_type *func_proto,
__u32 prog_id)
{
- struct bpf_prog_info_linear *prog_info = NULL;
const struct btf_type *func_type;
+ int prog_fd = -1, func_sig_len;
+ struct bpf_prog_info info = {};
+ __u32 info_len = sizeof(info);
const char *prog_name = NULL;
- struct bpf_func_info *finfo;
struct btf *prog_btf = NULL;
- struct bpf_prog_info *info;
- int prog_fd, func_sig_len;
+ struct bpf_func_info finfo;
+ __u32 finfo_rec_size;
char prog_str[1024];
+ int err;
/* Get the ptr's func_proto */
func_sig_len = btf_dump_func(d->btf, prog_str, func_proto, NULL, 0,
@@ -52,25 +54,30 @@ static int dump_prog_id_as_func_ptr(const struct btf_dumper *d,
/* Get the bpf_prog's name. Obtain from func_info. */
prog_fd = bpf_prog_get_fd_by_id(prog_id);
- if (prog_fd == -1)
+ if (prog_fd < 0)
goto print;
- prog_info = bpf_program__get_prog_info_linear(prog_fd,
- 1UL << BPF_PROG_INFO_FUNC_INFO);
- close(prog_fd);
- if (IS_ERR(prog_info)) {
- prog_info = NULL;
+ err = bpf_obj_get_info_by_fd(prog_fd, &info, &info_len);
+ if (err)
goto print;
- }
- info = &prog_info->info;
- if (!info->btf_id || !info->nr_func_info)
+ if (!info.btf_id || !info.nr_func_info)
goto print;
- prog_btf = btf__load_from_kernel_by_id(info->btf_id);
+
+ finfo_rec_size = info.func_info_rec_size;
+ memset(&info, 0, sizeof(info));
+ info.nr_func_info = 1;
+ info.func_info_rec_size = finfo_rec_size;
+ info.func_info = ptr_to_u64(&finfo);
+
+ err = bpf_obj_get_info_by_fd(prog_fd, &info, &info_len);
+ if (err)
+ goto print;
+
+ prog_btf = btf__load_from_kernel_by_id(info.btf_id);
if (libbpf_get_error(prog_btf))
goto print;
- finfo = u64_to_ptr(info->func_info);
- func_type = btf__type_by_id(prog_btf, finfo->type_id);
+ func_type = btf__type_by_id(prog_btf, finfo.type_id);
if (!func_type || !btf_is_func(func_type))
goto print;
@@ -92,7 +99,8 @@ static int dump_prog_id_as_func_ptr(const struct btf_dumper *d,
prog_str[sizeof(prog_str) - 1] = '\0';
jsonw_string(d->jw, prog_str);
btf__free(prog_btf);
- free(prog_info);
+ if (prog_fd >= 0)
+ close(prog_fd);
return 0;
}
diff --git a/tools/bpf/bpftool/common.c b/tools/bpf/bpftool/common.c
index 511eccd..fa8eb81 100644
--- a/tools/bpf/bpftool/common.c
+++ b/tools/bpf/bpftool/common.c
@@ -74,6 +74,7 @@ const char * const attach_type_name[__MAX_BPF_ATTACH_TYPE] = {
[BPF_XDP] = "xdp",
[BPF_SK_REUSEPORT_SELECT] = "sk_skb_reuseport_select",
[BPF_SK_REUSEPORT_SELECT_OR_MIGRATE] = "sk_skb_reuseport_select_or_migrate",
+ [BPF_PERF_EVENT] = "perf_event",
};
void p_err(const char *fmt, ...)
diff --git a/tools/bpf/bpftool/feature.c b/tools/bpf/bpftool/feature.c
index ade4457..e999159 100644
--- a/tools/bpf/bpftool/feature.c
+++ b/tools/bpf/bpftool/feature.c
@@ -467,7 +467,7 @@ static bool probe_bpf_syscall(const char *define_prefix)
{
bool res;
- bpf_load_program(BPF_PROG_TYPE_UNSPEC, NULL, 0, NULL, 0, NULL, 0);
+ bpf_prog_load(BPF_PROG_TYPE_UNSPEC, NULL, NULL, NULL, 0, NULL);
res = (errno != ENOSYS);
print_bool_feature("have_bpf_syscall",
@@ -643,15 +643,111 @@ probe_helpers_for_progtype(enum bpf_prog_type prog_type, bool supported_type,
}
static void
-probe_large_insn_limit(const char *define_prefix, __u32 ifindex)
+probe_misc_feature(struct bpf_insn *insns, size_t len,
+ const char *define_prefix, __u32 ifindex,
+ const char *feat_name, const char *plain_name,
+ const char *define_name)
{
+ LIBBPF_OPTS(bpf_prog_load_opts, opts,
+ .prog_ifindex = ifindex,
+ );
bool res;
+ int fd;
- res = bpf_probe_large_insn_limit(ifindex);
- print_bool_feature("have_large_insn_limit",
+ errno = 0;
+ fd = bpf_prog_load(BPF_PROG_TYPE_SOCKET_FILTER, NULL, "GPL",
+ insns, len, &opts);
+ res = fd >= 0 || !errno;
+
+ if (fd >= 0)
+ close(fd);
+
+ print_bool_feature(feat_name, plain_name, define_name, res,
+ define_prefix);
+}
+
+/*
+ * Probe for availability of kernel commit (5.3):
+ *
+ * c04c0d2b968a ("bpf: increase complexity limit and maximum program size")
+ */
+static void probe_large_insn_limit(const char *define_prefix, __u32 ifindex)
+{
+ struct bpf_insn insns[BPF_MAXINSNS + 1];
+ int i;
+
+ for (i = 0; i < BPF_MAXINSNS; i++)
+ insns[i] = BPF_MOV64_IMM(BPF_REG_0, 1);
+ insns[BPF_MAXINSNS] = BPF_EXIT_INSN();
+
+ probe_misc_feature(insns, ARRAY_SIZE(insns),
+ define_prefix, ifindex,
+ "have_large_insn_limit",
"Large program size limit",
- "LARGE_INSN_LIMIT",
- res, define_prefix);
+ "LARGE_INSN_LIMIT");
+}
+
+/*
+ * Probe for bounded loop support introduced in commit 2589726d12a1
+ * ("bpf: introduce bounded loops").
+ */
+static void
+probe_bounded_loops(const char *define_prefix, __u32 ifindex)
+{
+ struct bpf_insn insns[4] = {
+ BPF_MOV64_IMM(BPF_REG_0, 10),
+ BPF_ALU64_IMM(BPF_SUB, BPF_REG_0, 1),
+ BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, -2),
+ BPF_EXIT_INSN()
+ };
+
+ probe_misc_feature(insns, ARRAY_SIZE(insns),
+ define_prefix, ifindex,
+ "have_bounded_loops",
+ "Bounded loop support",
+ "BOUNDED_LOOPS");
+}
+
+/*
+ * Probe for the v2 instruction set extension introduced in commit 92b31a9af73b
+ * ("bpf: add BPF_J{LT,LE,SLT,SLE} instructions").
+ */
+static void
+probe_v2_isa_extension(const char *define_prefix, __u32 ifindex)
+{
+ struct bpf_insn insns[4] = {
+ BPF_MOV64_IMM(BPF_REG_0, 0),
+ BPF_JMP_IMM(BPF_JLT, BPF_REG_0, 0, 1),
+ BPF_MOV64_IMM(BPF_REG_0, 1),
+ BPF_EXIT_INSN()
+ };
+
+ probe_misc_feature(insns, ARRAY_SIZE(insns),
+ define_prefix, ifindex,
+ "have_v2_isa_extension",
+ "ISA extension v2",
+ "V2_ISA_EXTENSION");
+}
+
+/*
+ * Probe for the v3 instruction set extension introduced in commit 092ed0968bb6
+ * ("bpf: verifier support JMP32").
+ */
+static void
+probe_v3_isa_extension(const char *define_prefix, __u32 ifindex)
+{
+ struct bpf_insn insns[4] = {
+ BPF_MOV64_IMM(BPF_REG_0, 0),
+ BPF_JMP32_IMM(BPF_JLT, BPF_REG_0, 0, 1),
+ BPF_MOV64_IMM(BPF_REG_0, 1),
+ BPF_EXIT_INSN()
+ };
+
+ probe_misc_feature(insns, ARRAY_SIZE(insns),
+ define_prefix, ifindex,
+ "have_v3_isa_extension",
+ "ISA extension v3",
+ "V3_ISA_EXTENSION");
}
static void
@@ -768,6 +864,9 @@ static void section_misc(const char *define_prefix, __u32 ifindex)
"/*** eBPF misc features ***/",
define_prefix);
probe_large_insn_limit(define_prefix, ifindex);
+ probe_bounded_loops(define_prefix, ifindex);
+ probe_v2_isa_extension(define_prefix, ifindex);
+ probe_v3_isa_extension(define_prefix, ifindex);
print_end_section();
}
diff --git a/tools/bpf/bpftool/gen.c b/tools/bpf/bpftool/gen.c
index 5c18351..b4695df 100644
--- a/tools/bpf/bpftool/gen.c
+++ b/tools/bpf/bpftool/gen.c
@@ -218,9 +218,10 @@ static int codegen_datasecs(struct bpf_object *obj, const char *obj_name)
char sec_ident[256], map_ident[256];
int i, err = 0;
- d = btf_dump__new(btf, NULL, NULL, codegen_btf_dump_printf);
- if (IS_ERR(d))
- return PTR_ERR(d);
+ d = btf_dump__new(btf, codegen_btf_dump_printf, NULL, NULL);
+ err = libbpf_get_error(d);
+ if (err)
+ return err;
bpf_object__for_each_map(map, obj) {
/* only generate definitions for memory-mapped internal maps */
@@ -485,7 +486,6 @@ static void codegen_destroy(struct bpf_object *obj, const char *obj_name)
static int gen_trace(struct bpf_object *obj, const char *obj_name, const char *header_guard)
{
- struct bpf_object_load_attr load_attr = {};
DECLARE_LIBBPF_OPTS(gen_loader_opts, opts);
struct bpf_map *map;
char ident[256];
@@ -495,12 +495,7 @@ static int gen_trace(struct bpf_object *obj, const char *obj_name, const char *h
if (err)
return err;
- load_attr.obj = obj;
- if (verifier_logs)
- /* log_level1 + log_level2 + stats, but not stable UAPI */
- load_attr.log_level = 1 + 2 + 4;
-
- err = bpf_object__load_xattr(&load_attr);
+ err = bpf_object__load(obj);
if (err) {
p_err("failed to load object file");
goto out;
@@ -718,11 +713,15 @@ static int do_skeleton(int argc, char **argv)
if (obj_name[0] == '\0')
get_obj_name(obj_name, file);
opts.object_name = obj_name;
+ if (verifier_logs)
+ /* log_level1 + log_level2 + stats, but not stable UAPI */
+ opts.kernel_log_level = 1 + 2 + 4;
obj = bpf_object__open_mem(obj_data, file_sz, &opts);
- if (IS_ERR(obj)) {
+ err = libbpf_get_error(obj);
+ if (err) {
char err_buf[256];
- libbpf_strerror(PTR_ERR(obj), err_buf, sizeof(err_buf));
+ libbpf_strerror(err, err_buf, sizeof(err_buf));
p_err("failed to open BPF object file: %s", err_buf);
obj = NULL;
goto out;
diff --git a/tools/bpf/bpftool/iter.c b/tools/bpf/bpftool/iter.c
index 6c0de64..f88fdc8 100644
--- a/tools/bpf/bpftool/iter.c
+++ b/tools/bpf/bpftool/iter.c
@@ -46,7 +46,8 @@ static int do_pin(int argc, char **argv)
}
obj = bpf_object__open(objfile);
- if (IS_ERR(obj)) {
+ err = libbpf_get_error(obj);
+ if (err) {
p_err("can't open objfile %s", objfile);
goto close_map_fd;
}
@@ -64,8 +65,8 @@ static int do_pin(int argc, char **argv)
}
link = bpf_program__attach_iter(prog, &iter_opts);
- if (IS_ERR(link)) {
- err = PTR_ERR(link);
+ err = libbpf_get_error(link);
+ if (err) {
p_err("attach_iter failed for program %s",
bpf_program__name(prog));
goto close_obj;
diff --git a/tools/bpf/bpftool/main.c b/tools/bpf/bpftool/main.c
index 28237d7..020e91a 100644
--- a/tools/bpf/bpftool/main.c
+++ b/tools/bpf/bpftool/main.c
@@ -31,6 +31,7 @@ bool block_mount;
bool verifier_logs;
bool relaxed_maps;
bool use_loader;
+bool legacy_libbpf;
struct btf *base_btf;
struct hashmap *refs_table;
@@ -92,6 +93,7 @@ static int do_version(int argc, char **argv)
jsonw_name(json_wtr, "features");
jsonw_start_object(json_wtr); /* features */
jsonw_bool_field(json_wtr, "libbfd", has_libbfd);
+ jsonw_bool_field(json_wtr, "libbpf_strict", !legacy_libbpf);
jsonw_bool_field(json_wtr, "skeletons", has_skeletons);
jsonw_end_object(json_wtr); /* features */
@@ -105,6 +107,10 @@ static int do_version(int argc, char **argv)
printf(" libbfd");
nb_features++;
}
+ if (!legacy_libbpf) {
+ printf("%s libbpf_strict", nb_features++ ? "," : "");
+ nb_features++;
+ }
if (has_skeletons)
printf("%s skeletons", nb_features++ ? "," : "");
printf("\n");
@@ -396,10 +402,14 @@ int main(int argc, char **argv)
{ "debug", no_argument, NULL, 'd' },
{ "use-loader", no_argument, NULL, 'L' },
{ "base-btf", required_argument, NULL, 'B' },
+ { "legacy", no_argument, NULL, 'l' },
{ 0 }
};
+ bool version_requested = false;
int opt, ret;
+ setlinebuf(stdout);
+
last_do_help = do_help;
pretty_output = false;
json_output = false;
@@ -408,11 +418,12 @@ int main(int argc, char **argv)
bin_name = argv[0];
opterr = 0;
- while ((opt = getopt_long(argc, argv, "VhpjfLmndB:",
+ while ((opt = getopt_long(argc, argv, "VhpjfLmndB:l",
options, NULL)) >= 0) {
switch (opt) {
case 'V':
- return do_version(argc, argv);
+ version_requested = true;
+ break;
case 'h':
return do_help(argc, argv);
case 'p':
@@ -454,6 +465,9 @@ int main(int argc, char **argv)
case 'L':
use_loader = true;
break;
+ case 'l':
+ legacy_libbpf = true;
+ break;
default:
p_err("unrecognized option '%s'", argv[optind - 1]);
if (json_output)
@@ -463,11 +477,20 @@ int main(int argc, char **argv)
}
}
+ if (!legacy_libbpf) {
+ ret = libbpf_set_strict_mode(LIBBPF_STRICT_ALL);
+ if (ret)
+ p_err("failed to enable libbpf strict mode: %d", ret);
+ }
+
argc -= optind;
argv += optind;
if (argc < 0)
usage();
+ if (version_requested)
+ return do_version(argc, argv);
+
ret = cmd_select(cmds, argc, argv, do_help);
if (json_output)
diff --git a/tools/bpf/bpftool/main.h b/tools/bpf/bpftool/main.h
index 383835c..8d76d93 100644
--- a/tools/bpf/bpftool/main.h
+++ b/tools/bpf/bpftool/main.h
@@ -57,7 +57,7 @@ static inline void *u64_to_ptr(__u64 ptr)
#define HELP_SPEC_PROGRAM \
"PROG := { id PROG_ID | pinned FILE | tag PROG_TAG | name PROG_NAME }"
#define HELP_SPEC_OPTIONS \
- "OPTIONS := { {-j|--json} [{-p|--pretty}] | {-d|--debug}"
+ "OPTIONS := { {-j|--json} [{-p|--pretty}] | {-d|--debug} | {-l|--legacy}"
#define HELP_SPEC_MAP \
"MAP := { id MAP_ID | pinned FILE | name MAP_NAME }"
#define HELP_SPEC_LINK \
@@ -90,6 +90,7 @@ extern bool block_mount;
extern bool verifier_logs;
extern bool relaxed_maps;
extern bool use_loader;
+extern bool legacy_libbpf;
extern struct btf *base_btf;
extern struct hashmap *refs_table;
diff --git a/tools/bpf/bpftool/map.c b/tools/bpf/bpftool/map.c
index cae1f11..cc530a2 100644
--- a/tools/bpf/bpftool/map.c
+++ b/tools/bpf/bpftool/map.c
@@ -53,6 +53,7 @@ const char * const map_type_name[] = {
[BPF_MAP_TYPE_RINGBUF] = "ringbuf",
[BPF_MAP_TYPE_INODE_STORAGE] = "inode_storage",
[BPF_MAP_TYPE_TASK_STORAGE] = "task_storage",
+ [BPF_MAP_TYPE_BLOOM_FILTER] = "bloom_filter",
};
const size_t map_type_name_size = ARRAY_SIZE(map_type_name);
@@ -811,7 +812,7 @@ static struct btf *get_map_kv_btf(const struct bpf_map_info *info)
if (info->btf_vmlinux_value_type_id) {
if (!btf_vmlinux) {
btf_vmlinux = libbpf_find_kernel_btf();
- if (IS_ERR(btf_vmlinux))
+ if (libbpf_get_error(btf_vmlinux))
p_err("failed to get kernel btf");
}
return btf_vmlinux;
@@ -831,13 +832,13 @@ static struct btf *get_map_kv_btf(const struct bpf_map_info *info)
static void free_map_kv_btf(struct btf *btf)
{
- if (!IS_ERR(btf) && btf != btf_vmlinux)
+ if (!libbpf_get_error(btf) && btf != btf_vmlinux)
btf__free(btf);
}
static void free_btf_vmlinux(void)
{
- if (!IS_ERR(btf_vmlinux))
+ if (!libbpf_get_error(btf_vmlinux))
btf__free(btf_vmlinux);
}
@@ -862,8 +863,8 @@ map_dump(int fd, struct bpf_map_info *info, json_writer_t *wtr,
if (wtr) {
btf = get_map_kv_btf(info);
- if (IS_ERR(btf)) {
- err = PTR_ERR(btf);
+ err = libbpf_get_error(btf);
+ if (err) {
goto exit_free;
}
@@ -1260,7 +1261,10 @@ static int do_pin(int argc, char **argv)
static int do_create(int argc, char **argv)
{
- struct bpf_create_map_attr attr = { NULL, };
+ LIBBPF_OPTS(bpf_map_create_opts, attr);
+ enum bpf_map_type map_type = BPF_MAP_TYPE_UNSPEC;
+ __u32 key_size = 0, value_size = 0, max_entries = 0;
+ const char *map_name = NULL;
const char *pinfile;
int err = -1, fd;
@@ -1275,30 +1279,30 @@ static int do_create(int argc, char **argv)
if (is_prefix(*argv, "type")) {
NEXT_ARG();
- if (attr.map_type) {
+ if (map_type) {
p_err("map type already specified");
goto exit;
}
- attr.map_type = map_type_from_str(*argv);
- if ((int)attr.map_type < 0) {
+ map_type = map_type_from_str(*argv);
+ if ((int)map_type < 0) {
p_err("unrecognized map type: %s", *argv);
goto exit;
}
NEXT_ARG();
} else if (is_prefix(*argv, "name")) {
NEXT_ARG();
- attr.name = GET_ARG();
+ map_name = GET_ARG();
} else if (is_prefix(*argv, "key")) {
- if (parse_u32_arg(&argc, &argv, &attr.key_size,
+ if (parse_u32_arg(&argc, &argv, &key_size,
"key size"))
goto exit;
} else if (is_prefix(*argv, "value")) {
- if (parse_u32_arg(&argc, &argv, &attr.value_size,
+ if (parse_u32_arg(&argc, &argv, &value_size,
"value size"))
goto exit;
} else if (is_prefix(*argv, "entries")) {
- if (parse_u32_arg(&argc, &argv, &attr.max_entries,
+ if (parse_u32_arg(&argc, &argv, &max_entries,
"max entries"))
goto exit;
} else if (is_prefix(*argv, "flags")) {
@@ -1339,14 +1343,14 @@ static int do_create(int argc, char **argv)
}
}
- if (!attr.name) {
+ if (!map_name) {
p_err("map name not specified");
goto exit;
}
set_max_rlimit();
- fd = bpf_create_map_xattr(&attr);
+ fd = bpf_map_create(map_type, map_name, key_size, value_size, max_entries, &attr);
if (fd < 0) {
p_err("map create failed: %s", strerror(errno));
goto exit;
@@ -1477,7 +1481,7 @@ static int do_help(int argc, char **argv)
" devmap | devmap_hash | sockmap | cpumap | xskmap | sockhash |\n"
" cgroup_storage | reuseport_sockarray | percpu_cgroup_storage |\n"
" queue | stack | sk_storage | struct_ops | ringbuf | inode_storage |\n"
- " task_storage }\n"
+ " task_storage | bloom_filter }\n"
" " HELP_SPEC_OPTIONS " |\n"
" {-f|--bpffs} | {-n|--nomount} }\n"
"",
diff --git a/tools/bpf/bpftool/map_perf_ring.c b/tools/bpf/bpftool/map_perf_ring.c
index b98ea70..6b0c410 100644
--- a/tools/bpf/bpftool/map_perf_ring.c
+++ b/tools/bpf/bpftool/map_perf_ring.c
@@ -124,7 +124,7 @@ int do_event_pipe(int argc, char **argv)
.wakeup_events = 1,
};
struct bpf_map_info map_info = {};
- struct perf_buffer_raw_opts opts = {};
+ LIBBPF_OPTS(perf_buffer_raw_opts, opts);
struct event_pipe_ctx ctx = {
.all_cpus = true,
.cpu = -1,
@@ -190,14 +190,11 @@ int do_event_pipe(int argc, char **argv)
ctx.idx = 0;
}
- opts.attr = &perf_attr;
- opts.event_cb = print_bpf_output;
- opts.ctx = &ctx;
opts.cpu_cnt = ctx.all_cpus ? 0 : 1;
opts.cpus = &ctx.cpu;
opts.map_keys = &ctx.idx;
-
- pb = perf_buffer__new_raw(map_fd, MMAP_PAGE_CNT, &opts);
+ pb = perf_buffer__new_raw(map_fd, MMAP_PAGE_CNT, &perf_attr,
+ print_bpf_output, &ctx, &opts);
err = libbpf_get_error(pb);
if (err) {
p_err("failed to create perf buffer: %s (%d)",
diff --git a/tools/bpf/bpftool/prog.c b/tools/bpf/bpftool/prog.c
index 515d229..2a21d50 100644
--- a/tools/bpf/bpftool/prog.c
+++ b/tools/bpf/bpftool/prog.c
@@ -100,6 +100,76 @@ static enum bpf_attach_type parse_attach_type(const char *str)
return __MAX_BPF_ATTACH_TYPE;
}
+static int prep_prog_info(struct bpf_prog_info *const info, enum dump_mode mode,
+ void **info_data, size_t *const info_data_sz)
+{
+ struct bpf_prog_info holder = {};
+ size_t needed = 0;
+ void *ptr;
+
+ if (mode == DUMP_JITED) {
+ holder.jited_prog_len = info->jited_prog_len;
+ needed += info->jited_prog_len;
+ } else {
+ holder.xlated_prog_len = info->xlated_prog_len;
+ needed += info->xlated_prog_len;
+ }
+
+ holder.nr_jited_ksyms = info->nr_jited_ksyms;
+ needed += info->nr_jited_ksyms * sizeof(__u64);
+
+ holder.nr_jited_func_lens = info->nr_jited_func_lens;
+ needed += info->nr_jited_func_lens * sizeof(__u32);
+
+ holder.nr_func_info = info->nr_func_info;
+ holder.func_info_rec_size = info->func_info_rec_size;
+ needed += info->nr_func_info * info->func_info_rec_size;
+
+ holder.nr_line_info = info->nr_line_info;
+ holder.line_info_rec_size = info->line_info_rec_size;
+ needed += info->nr_line_info * info->line_info_rec_size;
+
+ holder.nr_jited_line_info = info->nr_jited_line_info;
+ holder.jited_line_info_rec_size = info->jited_line_info_rec_size;
+ needed += info->nr_jited_line_info * info->jited_line_info_rec_size;
+
+ if (needed > *info_data_sz) {
+ ptr = realloc(*info_data, needed);
+ if (!ptr)
+ return -1;
+
+ *info_data = ptr;
+ *info_data_sz = needed;
+ }
+ ptr = *info_data;
+
+ if (mode == DUMP_JITED) {
+ holder.jited_prog_insns = ptr_to_u64(ptr);
+ ptr += holder.jited_prog_len;
+ } else {
+ holder.xlated_prog_insns = ptr_to_u64(ptr);
+ ptr += holder.xlated_prog_len;
+ }
+
+ holder.jited_ksyms = ptr_to_u64(ptr);
+ ptr += holder.nr_jited_ksyms * sizeof(__u64);
+
+ holder.jited_func_lens = ptr_to_u64(ptr);
+ ptr += holder.nr_jited_func_lens * sizeof(__u32);
+
+ holder.func_info = ptr_to_u64(ptr);
+ ptr += holder.nr_func_info * holder.func_info_rec_size;
+
+ holder.line_info = ptr_to_u64(ptr);
+ ptr += holder.nr_line_info * holder.line_info_rec_size;
+
+ holder.jited_line_info = ptr_to_u64(ptr);
+ ptr += holder.nr_jited_line_info * holder.jited_line_info_rec_size;
+
+ *info = holder;
+ return 0;
+}
+
static void print_boot_time(__u64 nsecs, char *buf, unsigned int size)
{
struct timespec real_time_ts, boot_time_ts;
@@ -639,8 +709,8 @@ prog_dump(struct bpf_prog_info *info, enum dump_mode mode,
char func_sig[1024];
unsigned char *buf;
__u32 member_len;
+ int fd, err = -1;
ssize_t n;
- int fd;
if (mode == DUMP_JITED) {
if (info->jited_prog_len == 0 || !info->jited_prog_insns) {
@@ -679,7 +749,7 @@ prog_dump(struct bpf_prog_info *info, enum dump_mode mode,
if (fd < 0) {
p_err("can't open file %s: %s", filepath,
strerror(errno));
- return -1;
+ goto exit_free;
}
n = write(fd, buf, member_len);
@@ -687,7 +757,7 @@ prog_dump(struct bpf_prog_info *info, enum dump_mode mode,
if (n != (ssize_t)member_len) {
p_err("error writing output file: %s",
n < 0 ? strerror(errno) : "short write");
- return -1;
+ goto exit_free;
}
if (json_output)
@@ -701,7 +771,7 @@ prog_dump(struct bpf_prog_info *info, enum dump_mode mode,
info->netns_ino,
&disasm_opt);
if (!name)
- return -1;
+ goto exit_free;
}
if (info->nr_jited_func_lens && info->jited_func_lens) {
@@ -796,23 +866,28 @@ prog_dump(struct bpf_prog_info *info, enum dump_mode mode,
kernel_syms_destroy(&dd);
}
- btf__free(btf);
+ err = 0;
- return 0;
+exit_free:
+ btf__free(btf);
+ bpf_prog_linfo__free(prog_linfo);
+ return err;
}
static int do_dump(int argc, char **argv)
{
- struct bpf_prog_info_linear *info_linear;
+ struct bpf_prog_info info;
+ __u32 info_len = sizeof(info);
+ size_t info_data_sz = 0;
+ void *info_data = NULL;
char *filepath = NULL;
bool opcodes = false;
bool visual = false;
enum dump_mode mode;
bool linum = false;
- int *fds = NULL;
int nb_fds, i = 0;
+ int *fds = NULL;
int err = -1;
- __u64 arrays;
if (is_prefix(*argv, "jited")) {
if (disasm_init())
@@ -872,43 +947,44 @@ static int do_dump(int argc, char **argv)
goto exit_close;
}
- if (mode == DUMP_JITED)
- arrays = 1UL << BPF_PROG_INFO_JITED_INSNS;
- else
- arrays = 1UL << BPF_PROG_INFO_XLATED_INSNS;
-
- arrays |= 1UL << BPF_PROG_INFO_JITED_KSYMS;
- arrays |= 1UL << BPF_PROG_INFO_JITED_FUNC_LENS;
- arrays |= 1UL << BPF_PROG_INFO_FUNC_INFO;
- arrays |= 1UL << BPF_PROG_INFO_LINE_INFO;
- arrays |= 1UL << BPF_PROG_INFO_JITED_LINE_INFO;
-
if (json_output && nb_fds > 1)
jsonw_start_array(json_wtr); /* root array */
for (i = 0; i < nb_fds; i++) {
- info_linear = bpf_program__get_prog_info_linear(fds[i], arrays);
- if (IS_ERR_OR_NULL(info_linear)) {
+ memset(&info, 0, sizeof(info));
+
+ err = bpf_obj_get_info_by_fd(fds[i], &info, &info_len);
+ if (err) {
+ p_err("can't get prog info: %s", strerror(errno));
+ break;
+ }
+
+ err = prep_prog_info(&info, mode, &info_data, &info_data_sz);
+ if (err) {
+ p_err("can't grow prog info_data");
+ break;
+ }
+
+ err = bpf_obj_get_info_by_fd(fds[i], &info, &info_len);
+ if (err) {
p_err("can't get prog info: %s", strerror(errno));
break;
}
if (json_output && nb_fds > 1) {
jsonw_start_object(json_wtr); /* prog object */
- print_prog_header_json(&info_linear->info);
+ print_prog_header_json(&info);
jsonw_name(json_wtr, "insns");
} else if (nb_fds > 1) {
- print_prog_header_plain(&info_linear->info);
+ print_prog_header_plain(&info);
}
- err = prog_dump(&info_linear->info, mode, filepath, opcodes,
- visual, linum);
+ err = prog_dump(&info, mode, filepath, opcodes, visual, linum);
if (json_output && nb_fds > 1)
jsonw_end_object(json_wtr); /* prog object */
else if (i != nb_fds - 1 && nb_fds > 1)
printf("\n");
- free(info_linear);
if (err)
break;
close(fds[i]);
@@ -920,6 +996,7 @@ static int do_dump(int argc, char **argv)
for (; i < nb_fds; i++)
close(fds[i]);
exit_free:
+ free(info_data);
free(fds);
return err;
}
@@ -1387,7 +1464,6 @@ static int load_with_options(int argc, char **argv, bool first_prog_only)
DECLARE_LIBBPF_OPTS(bpf_object_open_opts, open_opts,
.relaxed_maps = relaxed_maps,
);
- struct bpf_object_load_attr load_attr = { 0 };
enum bpf_attach_type expected_attach_type;
struct map_replace *map_replace = NULL;
struct bpf_program *prog = NULL, *pos;
@@ -1409,8 +1485,6 @@ static int load_with_options(int argc, char **argv, bool first_prog_only)
while (argc) {
if (is_prefix(*argv, "type")) {
- char *type;
-
NEXT_ARG();
if (common_prog_type != BPF_PROG_TYPE_UNSPEC) {
@@ -1420,21 +1494,26 @@ static int load_with_options(int argc, char **argv, bool first_prog_only)
if (!REQ_ARGS(1))
goto err_free_reuse_maps;
- /* Put a '/' at the end of type to appease libbpf */
- type = malloc(strlen(*argv) + 2);
- if (!type) {
- p_err("mem alloc failed");
- goto err_free_reuse_maps;
- }
- *type = 0;
- strcat(type, *argv);
- strcat(type, "/");
+ err = libbpf_prog_type_by_name(*argv, &common_prog_type,
+ &expected_attach_type);
+ if (err < 0) {
+ /* Put a '/' at the end of type to appease libbpf */
+ char *type = malloc(strlen(*argv) + 2);
- err = get_prog_type_by_name(type, &common_prog_type,
- &expected_attach_type);
- free(type);
- if (err < 0)
- goto err_free_reuse_maps;
+ if (!type) {
+ p_err("mem alloc failed");
+ goto err_free_reuse_maps;
+ }
+ *type = 0;
+ strcat(type, *argv);
+ strcat(type, "/");
+
+ err = get_prog_type_by_name(type, &common_prog_type,
+ &expected_attach_type);
+ free(type);
+ if (err < 0)
+ goto err_free_reuse_maps;
+ }
NEXT_ARG();
} else if (is_prefix(*argv, "map")) {
@@ -1518,6 +1597,10 @@ static int load_with_options(int argc, char **argv, bool first_prog_only)
set_max_rlimit();
+ if (verifier_logs)
+ /* log_level1 + log_level2 + stats, but not stable UAPI */
+ open_opts.kernel_log_level = 1 + 2 + 4;
+
obj = bpf_object__open_file(file, &open_opts);
if (libbpf_get_error(obj)) {
p_err("failed to open object file");
@@ -1572,7 +1655,7 @@ static int load_with_options(int argc, char **argv, bool first_prog_only)
j = 0;
idx = 0;
bpf_object__for_each_map(map, obj) {
- if (!bpf_map__is_offload_neutral(map))
+ if (bpf_map__type(map) != BPF_MAP_TYPE_PERF_EVENT_ARRAY)
bpf_map__set_ifindex(map, ifindex);
if (j < old_map_fds && idx == map_replace[j].idx) {
@@ -1597,12 +1680,7 @@ static int load_with_options(int argc, char **argv, bool first_prog_only)
goto err_close_obj;
}
- load_attr.obj = obj;
- if (verifier_logs)
- /* log_level1 + log_level2 + stats, but not stable UAPI */
- load_attr.log_level = 1 + 2 + 4;
-
- err = bpf_object__load_xattr(&load_attr);
+ err = bpf_object__load(obj);
if (err) {
p_err("failed to load object file");
goto err_close_obj;
@@ -1657,6 +1735,11 @@ static int load_with_options(int argc, char **argv, bool first_prog_only)
else
bpf_object__unpin_programs(obj, pinfile);
err_close_obj:
+ if (!legacy_libbpf) {
+ p_info("Warning: bpftool is now running in libbpf strict mode and has more stringent requirements about BPF programs.\n"
+ "If it used to work for this object file but now doesn't, see --legacy option for more details.\n");
+ }
+
bpf_object__close(obj);
err_free_reuse_maps:
for (i = 0; i < old_map_fds; i++)
@@ -1689,17 +1772,19 @@ static int try_loader(struct gen_loader_opts *gen)
sizeof(struct bpf_prog_desc));
int log_buf_sz = (1u << 24) - 1;
int err, fds_before, fd_delta;
- char *log_buf;
+ char *log_buf = NULL;
ctx = alloca(ctx_sz);
memset(ctx, 0, ctx_sz);
ctx->sz = ctx_sz;
- ctx->log_level = 1;
- ctx->log_size = log_buf_sz;
- log_buf = malloc(log_buf_sz);
- if (!log_buf)
- return -ENOMEM;
- ctx->log_buf = (long) log_buf;
+ if (verifier_logs) {
+ ctx->log_level = 1 + 2 + 4;
+ ctx->log_size = log_buf_sz;
+ log_buf = malloc(log_buf_sz);
+ if (!log_buf)
+ return -ENOMEM;
+ ctx->log_buf = (long) log_buf;
+ }
opts.ctx = ctx;
opts.data = gen->data;
opts.data_sz = gen->data_sz;
@@ -1708,9 +1793,9 @@ static int try_loader(struct gen_loader_opts *gen)
fds_before = count_open_fds();
err = bpf_load_and_run(&opts);
fd_delta = count_open_fds() - fds_before;
- if (err < 0) {
+ if (err < 0 || verifier_logs) {
fprintf(stderr, "err %d\n%s\n%s", err, opts.errstr, log_buf);
- if (fd_delta)
+ if (fd_delta && err < 0)
fprintf(stderr, "loader prog leaked %d FDs\n",
fd_delta);
}
@@ -1722,7 +1807,6 @@ static int do_loader(int argc, char **argv)
{
DECLARE_LIBBPF_OPTS(bpf_object_open_opts, open_opts);
DECLARE_LIBBPF_OPTS(gen_loader_opts, gen);
- struct bpf_object_load_attr load_attr = {};
struct bpf_object *obj;
const char *file;
int err = 0;
@@ -1731,6 +1815,10 @@ static int do_loader(int argc, char **argv)
return -1;
file = GET_ARG();
+ if (verifier_logs)
+ /* log_level1 + log_level2 + stats, but not stable UAPI */
+ open_opts.kernel_log_level = 1 + 2 + 4;
+
obj = bpf_object__open_file(file, &open_opts);
if (libbpf_get_error(obj)) {
p_err("failed to open object file");
@@ -1741,12 +1829,7 @@ static int do_loader(int argc, char **argv)
if (err)
goto err_close_obj;
- load_attr.obj = obj;
- if (verifier_logs)
- /* log_level1 + log_level2 + stats, but not stable UAPI */
- load_attr.log_level = 1 + 2 + 4;
-
- err = bpf_object__load_xattr(&load_attr);
+ err = bpf_object__load(obj);
if (err) {
p_err("failed to load object file");
goto err_close_obj;
@@ -2016,41 +2099,58 @@ static void profile_print_readings(void)
static char *profile_target_name(int tgt_fd)
{
- struct bpf_prog_info_linear *info_linear;
- struct bpf_func_info *func_info;
+ struct bpf_func_info func_info;
+ struct bpf_prog_info info = {};
+ __u32 info_len = sizeof(info);
const struct btf_type *t;
+ __u32 func_info_rec_size;
struct btf *btf = NULL;
char *name = NULL;
+ int err;
- info_linear = bpf_program__get_prog_info_linear(
- tgt_fd, 1UL << BPF_PROG_INFO_FUNC_INFO);
- if (IS_ERR_OR_NULL(info_linear)) {
- p_err("failed to get info_linear for prog FD %d", tgt_fd);
- return NULL;
+ err = bpf_obj_get_info_by_fd(tgt_fd, &info, &info_len);
+ if (err) {
+ p_err("failed to bpf_obj_get_info_by_fd for prog FD %d", tgt_fd);
+ goto out;
}
- if (info_linear->info.btf_id == 0) {
+ if (info.btf_id == 0) {
p_err("prog FD %d doesn't have valid btf", tgt_fd);
goto out;
}
- btf = btf__load_from_kernel_by_id(info_linear->info.btf_id);
+ func_info_rec_size = info.func_info_rec_size;
+ if (info.nr_func_info == 0) {
+ p_err("bpf_obj_get_info_by_fd for prog FD %d found 0 func_info", tgt_fd);
+ goto out;
+ }
+
+ memset(&info, 0, sizeof(info));
+ info.nr_func_info = 1;
+ info.func_info_rec_size = func_info_rec_size;
+ info.func_info = ptr_to_u64(&func_info);
+
+ err = bpf_obj_get_info_by_fd(tgt_fd, &info, &info_len);
+ if (err) {
+ p_err("failed to get func_info for prog FD %d", tgt_fd);
+ goto out;
+ }
+
+ btf = btf__load_from_kernel_by_id(info.btf_id);
if (libbpf_get_error(btf)) {
p_err("failed to load btf for prog FD %d", tgt_fd);
goto out;
}
- func_info = u64_to_ptr(info_linear->info.func_info);
- t = btf__type_by_id(btf, func_info[0].type_id);
+ t = btf__type_by_id(btf, func_info.type_id);
if (!t) {
p_err("btf %d doesn't have type %d",
- info_linear->info.btf_id, func_info[0].type_id);
+ info.btf_id, func_info.type_id);
goto out;
}
name = strdup(btf__name_by_offset(btf, t->name_off));
out:
btf__free(btf);
- free(info_linear);
return name;
}
diff --git a/tools/bpf/bpftool/struct_ops.c b/tools/bpf/bpftool/struct_ops.c
index ab2d229..2f693b0 100644
--- a/tools/bpf/bpftool/struct_ops.c
+++ b/tools/bpf/bpftool/struct_ops.c
@@ -32,7 +32,7 @@ static const struct btf *get_btf_vmlinux(void)
return btf_vmlinux;
btf_vmlinux = libbpf_find_kernel_btf();
- if (IS_ERR(btf_vmlinux))
+ if (libbpf_get_error(btf_vmlinux))
p_err("struct_ops requires kernel CONFIG_DEBUG_INFO_BTF=y");
return btf_vmlinux;
@@ -45,7 +45,7 @@ static const char *get_kern_struct_ops_name(const struct bpf_map_info *info)
const char *st_ops_name;
kern_btf = get_btf_vmlinux();
- if (IS_ERR(kern_btf))
+ if (libbpf_get_error(kern_btf))
return "<btf_vmlinux_not_found>";
t = btf__type_by_id(kern_btf, info->btf_vmlinux_value_type_id);
@@ -63,7 +63,7 @@ static __s32 get_map_info_type_id(void)
return map_info_type_id;
kern_btf = get_btf_vmlinux();
- if (IS_ERR(kern_btf)) {
+ if (libbpf_get_error(kern_btf)) {
map_info_type_id = PTR_ERR(kern_btf);
return map_info_type_id;
}
@@ -252,7 +252,7 @@ static struct res do_one_id(const char *id_str, work_func func, void *data,
}
fd = bpf_map_get_fd_by_id(id);
- if (fd == -1) {
+ if (fd < 0) {
p_err("can't get map by id (%lu): %s", id, strerror(errno));
res.nr_errs++;
return res;
@@ -415,7 +415,7 @@ static int do_dump(int argc, char **argv)
}
kern_btf = get_btf_vmlinux();
- if (IS_ERR(kern_btf))
+ if (libbpf_get_error(kern_btf))
return -1;
if (!json_output) {
@@ -479,7 +479,7 @@ static int do_unregister(int argc, char **argv)
static int do_register(int argc, char **argv)
{
- struct bpf_object_load_attr load_attr = {};
+ LIBBPF_OPTS(bpf_object_open_opts, open_opts);
const struct bpf_map_def *def;
struct bpf_map_info info = {};
__u32 info_len = sizeof(info);
@@ -494,18 +494,17 @@ static int do_register(int argc, char **argv)
file = GET_ARG();
- obj = bpf_object__open(file);
- if (IS_ERR_OR_NULL(obj))
+ if (verifier_logs)
+ /* log_level1 + log_level2 + stats, but not stable UAPI */
+ open_opts.kernel_log_level = 1 + 2 + 4;
+
+ obj = bpf_object__open_file(file, &open_opts);
+ if (libbpf_get_error(obj))
return -1;
set_max_rlimit();
- load_attr.obj = obj;
- if (verifier_logs)
- /* log_level1 + log_level2 + stats, but not stable UAPI */
- load_attr.log_level = 1 + 2 + 4;
-
- if (bpf_object__load_xattr(&load_attr)) {
+ if (bpf_object__load(obj)) {
bpf_object__close(obj);
return -1;
}
@@ -516,7 +515,7 @@ static int do_register(int argc, char **argv)
continue;
link = bpf_map__attach_struct_ops(map);
- if (IS_ERR(link)) {
+ if (libbpf_get_error(link)) {
p_err("can't register struct_ops %s: %s",
bpf_map__name(map),
strerror(-PTR_ERR(link)));
@@ -596,7 +595,7 @@ int do_struct_ops(int argc, char **argv)
err = cmd_select(cmds, argc, argv, do_help);
- if (!IS_ERR(btf_vmlinux))
+ if (!libbpf_get_error(btf_vmlinux))
btf__free(btf_vmlinux);
return err;
diff --git a/tools/bpf/resolve_btfids/Makefile b/tools/bpf/resolve_btfids/Makefile
index 751643f..9ddeca9 100644
--- a/tools/bpf/resolve_btfids/Makefile
+++ b/tools/bpf/resolve_btfids/Makefile
@@ -19,6 +19,7 @@
LD = $(HOSTLD)
ARCH = $(HOSTARCH)
RM ?= rm
+CROSS_COMPILE =
OUTPUT ?= $(srctree)/tools/bpf/resolve_btfids/
diff --git a/tools/bpf/resolve_btfids/main.c b/tools/bpf/resolve_btfids/main.c
index 73409e2..5d26f3c 100644
--- a/tools/bpf/resolve_btfids/main.c
+++ b/tools/bpf/resolve_btfids/main.c
@@ -168,7 +168,7 @@ static struct btf_id *btf_id__find(struct rb_root *root, const char *name)
return NULL;
}
-static struct btf_id*
+static struct btf_id *
btf_id__add(struct rb_root *root, char *name, bool unique)
{
struct rb_node **p = &root->rb_node;
@@ -732,7 +732,8 @@ int main(int argc, const char **argv)
if (obj.efile.idlist_shndx == -1 ||
obj.efile.symbols_shndx == -1) {
pr_debug("Cannot find .BTF_ids or symbols sections, nothing to do\n");
- return 0;
+ err = 0;
+ goto out;
}
if (symbols_collect(&obj))
diff --git a/tools/bpf/runqslower/Makefile b/tools/bpf/runqslower/Makefile
index 8791d0e..da6de16 100644
--- a/tools/bpf/runqslower/Makefile
+++ b/tools/bpf/runqslower/Makefile
@@ -12,7 +12,7 @@
BPF_DESTDIR := $(BPFOBJ_OUTPUT)
BPF_INCLUDE := $(BPF_DESTDIR)/include
INCLUDES := -I$(OUTPUT) -I$(BPF_INCLUDE) -I$(abspath ../../include/uapi)
-CFLAGS := -g -Wall
+CFLAGS := -g -Wall $(CLANG_CROSS_FLAGS)
# Try to detect best kernel BTF source
KERNEL_REL := $(shell uname -r)
@@ -88,4 +88,4 @@
$(DEFAULT_BPFTOOL): $(BPFOBJ) | $(BPFTOOL_OUTPUT)
$(Q)$(MAKE) $(submake_extras) -C ../bpftool OUTPUT=$(BPFTOOL_OUTPUT) \
- CC=$(HOSTCC) LD=$(HOSTLD)
+ ARCH= CROSS_COMPILE= CC=$(HOSTCC) LD=$(HOSTLD)
diff --git a/tools/bpf/runqslower/runqslower.c b/tools/bpf/runqslower/runqslower.c
index d897158..2414cc7 100644
--- a/tools/bpf/runqslower/runqslower.c
+++ b/tools/bpf/runqslower/runqslower.c
@@ -123,7 +123,6 @@ int main(int argc, char **argv)
.parser = parse_arg,
.doc = argp_program_doc,
};
- struct perf_buffer_opts pb_opts;
struct perf_buffer *pb = NULL;
struct runqslower_bpf *obj;
int err;
@@ -165,9 +164,8 @@ int main(int argc, char **argv)
printf("Tracing run queue latency higher than %llu us\n", env.min_us);
printf("%-8s %-16s %-6s %14s\n", "TIME", "COMM", "PID", "LAT(us)");
- pb_opts.sample_cb = handle_event;
- pb_opts.lost_cb = handle_lost_events;
- pb = perf_buffer__new(bpf_map__fd(obj->maps.events), 64, &pb_opts);
+ pb = perf_buffer__new(bpf_map__fd(obj->maps.events), 64,
+ handle_event, handle_lost_events, NULL, NULL);
err = libbpf_get_error(pb);
if (err) {
pb = NULL;
diff --git a/tools/build/feature/test-bpf.c b/tools/build/feature/test-bpf.c
index 82070ea..727d22e 100644
--- a/tools/build/feature/test-bpf.c
+++ b/tools/build/feature/test-bpf.c
@@ -14,6 +14,12 @@
# define __NR_bpf 349
# elif defined(__s390__)
# define __NR_bpf 351
+# elif defined(__mips__) && defined(_ABIO32)
+# define __NR_bpf 4355
+# elif defined(__mips__) && defined(_ABIN32)
+# define __NR_bpf 6319
+# elif defined(__mips__) && defined(_ABI64)
+# define __NR_bpf 5315
# else
# error __NR_bpf not defined. libbpf does not support your arch.
# endif
diff --git a/tools/include/linux/kernel.h b/tools/include/linux/kernel.h
index 3e8df50..9701e83 100644
--- a/tools/include/linux/kernel.h
+++ b/tools/include/linux/kernel.h
@@ -92,7 +92,9 @@ int vscnprintf(char *buf, size_t size, const char *fmt, va_list args);
int scnprintf(char * buf, size_t size, const char * fmt, ...);
int scnprintf_pad(char * buf, size_t size, const char * fmt, ...);
+#ifndef ARRAY_SIZE
#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]) + __must_be_array(arr))
+#endif
#define current_gfp_context(k) 0
#define synchronize_rcu()
diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
index ba5af15..b0383d3 100644
--- a/tools/include/uapi/linux/bpf.h
+++ b/tools/include/uapi/linux/bpf.h
@@ -1342,8 +1342,10 @@ union bpf_attr {
/* or valid module BTF object fd or 0 to attach to vmlinux */
__u32 attach_btf_obj_fd;
};
- __u32 :32; /* pad */
+ __u32 core_relo_cnt; /* number of bpf_core_relo */
__aligned_u64 fd_array; /* array of FDs */
+ __aligned_u64 core_relos;
+ __u32 core_relo_rec_size; /* sizeof(struct bpf_core_relo) */
};
struct { /* anonymous struct used by BPF_OBJ_* commands */
@@ -1744,7 +1746,7 @@ union bpf_attr {
* if the maximum number of tail calls has been reached for this
* chain of programs. This limit is defined in the kernel by the
* macro **MAX_TAIL_CALL_CNT** (not accessible to user space),
- * which is currently set to 32.
+ * which is currently set to 33.
* Return
* 0 on success, or a negative error in case of failure.
*
@@ -4938,6 +4940,84 @@ union bpf_attr {
* **-ENOENT** if symbol is not found.
*
* **-EPERM** if caller does not have permission to obtain kernel address.
+ *
+ * long bpf_find_vma(struct task_struct *task, u64 addr, void *callback_fn, void *callback_ctx, u64 flags)
+ * Description
+ * Find vma of *task* that contains *addr*, call *callback_fn*
+ * function with *task*, *vma*, and *callback_ctx*.
+ * The *callback_fn* should be a static function and
+ * the *callback_ctx* should be a pointer to the stack.
+ * The *flags* is used to control certain aspects of the helper.
+ * Currently, the *flags* must be 0.
+ *
+ * The expected callback signature is
+ *
+ * long (\*callback_fn)(struct task_struct \*task, struct vm_area_struct \*vma, void \*callback_ctx);
+ *
+ * Return
+ * 0 on success.
+ * **-ENOENT** if *task->mm* is NULL, or no vma contains *addr*.
+ * **-EBUSY** if failed to try lock mmap_lock.
+ * **-EINVAL** for invalid **flags**.
+ *
+ * long bpf_loop(u32 nr_loops, void *callback_fn, void *callback_ctx, u64 flags)
+ * Description
+ * For **nr_loops**, call **callback_fn** function
+ * with **callback_ctx** as the context parameter.
+ * The **callback_fn** should be a static function and
+ * the **callback_ctx** should be a pointer to the stack.
+ * The **flags** is used to control certain aspects of the helper.
+ * Currently, the **flags** must be 0. Currently, nr_loops is
+ * limited to 1 << 23 (~8 million) loops.
+ *
+ * long (\*callback_fn)(u32 index, void \*ctx);
+ *
+ * where **index** is the current index in the loop. The index
+ * is zero-indexed.
+ *
+ * If **callback_fn** returns 0, the helper will continue to the next
+ * loop. If return value is 1, the helper will skip the rest of
+ * the loops and return. Other return values are not used now,
+ * and will be rejected by the verifier.
+ *
+ * Return
+ * The number of loops performed, **-EINVAL** for invalid **flags**,
+ * **-E2BIG** if **nr_loops** exceeds the maximum number of loops.
+ *
+ * long bpf_strncmp(const char *s1, u32 s1_sz, const char *s2)
+ * Description
+ * Do strncmp() between **s1** and **s2**. **s1** doesn't need
+ * to be null-terminated and **s1_sz** is the maximum storage
+ * size of **s1**. **s2** must be a read-only string.
+ * Return
+ * An integer less than, equal to, or greater than zero
+ * if the first **s1_sz** bytes of **s1** is found to be
+ * less than, to match, or be greater than **s2**.
+ *
+ * long bpf_get_func_arg(void *ctx, u32 n, u64 *value)
+ * Description
+ * Get **n**-th argument (zero based) of the traced function (for tracing programs)
+ * returned in **value**.
+ *
+ * Return
+ * 0 on success.
+ * **-EINVAL** if n >= arguments count of traced function.
+ *
+ * long bpf_get_func_ret(void *ctx, u64 *value)
+ * Description
+ * Get return value of the traced function (for tracing programs)
+ * in **value**.
+ *
+ * Return
+ * 0 on success.
+ * **-EOPNOTSUPP** for tracing programs other than BPF_TRACE_FEXIT or BPF_MODIFY_RETURN.
+ *
+ * long bpf_get_func_arg_cnt(void *ctx)
+ * Description
+ * Get number of arguments of the traced function (for tracing programs).
+ *
+ * Return
+ * The number of arguments of the traced function.
*/
#define __BPF_FUNC_MAPPER(FN) \
FN(unspec), \
@@ -5120,6 +5200,12 @@ union bpf_attr {
FN(trace_vprintk), \
FN(skc_to_unix_sock), \
FN(kallsyms_lookup_name), \
+ FN(find_vma), \
+ FN(loop), \
+ FN(strncmp), \
+ FN(get_func_arg), \
+ FN(get_func_ret), \
+ FN(get_func_arg_cnt), \
/* */
/* integer value in 'imm' field of BPF_CALL instruction selects which helper
@@ -6296,6 +6382,7 @@ struct bpf_sk_lookup {
__u32 local_ip4; /* Network byte order */
__u32 local_ip6[4]; /* Network byte order */
__u32 local_port; /* Host byte order */
+ __u32 ingress_ifindex; /* The arriving interface. Determined by inet_iif. */
};
/*
@@ -6328,4 +6415,78 @@ enum {
BTF_F_ZERO = (1ULL << 3),
};
+/* bpf_core_relo_kind encodes which aspect of captured field/type/enum value
+ * has to be adjusted by relocations. It is emitted by llvm and passed to
+ * libbpf and later to the kernel.
+ */
+enum bpf_core_relo_kind {
+ BPF_CORE_FIELD_BYTE_OFFSET = 0, /* field byte offset */
+ BPF_CORE_FIELD_BYTE_SIZE = 1, /* field size in bytes */
+ BPF_CORE_FIELD_EXISTS = 2, /* field existence in target kernel */
+ BPF_CORE_FIELD_SIGNED = 3, /* field signedness (0 - unsigned, 1 - signed) */
+ BPF_CORE_FIELD_LSHIFT_U64 = 4, /* bitfield-specific left bitshift */
+ BPF_CORE_FIELD_RSHIFT_U64 = 5, /* bitfield-specific right bitshift */
+ BPF_CORE_TYPE_ID_LOCAL = 6, /* type ID in local BPF object */
+ BPF_CORE_TYPE_ID_TARGET = 7, /* type ID in target kernel */
+ BPF_CORE_TYPE_EXISTS = 8, /* type existence in target kernel */
+ BPF_CORE_TYPE_SIZE = 9, /* type size in bytes */
+ BPF_CORE_ENUMVAL_EXISTS = 10, /* enum value existence in target kernel */
+ BPF_CORE_ENUMVAL_VALUE = 11, /* enum value integer value */
+};
+
+/*
+ * "struct bpf_core_relo" is used to pass relocation data form LLVM to libbpf
+ * and from libbpf to the kernel.
+ *
+ * CO-RE relocation captures the following data:
+ * - insn_off - instruction offset (in bytes) within a BPF program that needs
+ * its insn->imm field to be relocated with actual field info;
+ * - type_id - BTF type ID of the "root" (containing) entity of a relocatable
+ * type or field;
+ * - access_str_off - offset into corresponding .BTF string section. String
+ * interpretation depends on specific relocation kind:
+ * - for field-based relocations, string encodes an accessed field using
+ * a sequence of field and array indices, separated by colon (:). It's
+ * conceptually very close to LLVM's getelementptr ([0]) instruction's
+ * arguments for identifying offset to a field.
+ * - for type-based relocations, strings is expected to be just "0";
+ * - for enum value-based relocations, string contains an index of enum
+ * value within its enum type;
+ * - kind - one of enum bpf_core_relo_kind;
+ *
+ * Example:
+ * struct sample {
+ * int a;
+ * struct {
+ * int b[10];
+ * };
+ * };
+ *
+ * struct sample *s = ...;
+ * int *x = &s->a; // encoded as "0:0" (a is field #0)
+ * int *y = &s->b[5]; // encoded as "0:1:0:5" (anon struct is field #1,
+ * // b is field #0 inside anon struct, accessing elem #5)
+ * int *z = &s[10]->b; // encoded as "10:1" (ptr is used as an array)
+ *
+ * type_id for all relocs in this example will capture BTF type id of
+ * `struct sample`.
+ *
+ * Such relocation is emitted when using __builtin_preserve_access_index()
+ * Clang built-in, passing expression that captures field address, e.g.:
+ *
+ * bpf_probe_read(&dst, sizeof(dst),
+ * __builtin_preserve_access_index(&src->a.b.c));
+ *
+ * In this case Clang will emit field relocation recording necessary data to
+ * be able to find offset of embedded `a.b.c` field within `src` struct.
+ *
+ * [0] https://llvm.org/docs/LangRef.html#getelementptr-instruction
+ */
+struct bpf_core_relo {
+ __u32 insn_off;
+ __u32 type_id;
+ __u32 access_str_off;
+ enum bpf_core_relo_kind kind;
+};
+
#endif /* _UAPI__LINUX_BPF_H__ */
diff --git a/tools/include/uapi/linux/btf.h b/tools/include/uapi/linux/btf.h
index deb12f7..b0d8fea 100644
--- a/tools/include/uapi/linux/btf.h
+++ b/tools/include/uapi/linux/btf.h
@@ -43,7 +43,7 @@ struct btf_type {
* "size" tells the size of the type it is describing.
*
* "type" is used by PTR, TYPEDEF, VOLATILE, CONST, RESTRICT,
- * FUNC, FUNC_PROTO, VAR and DECL_TAG.
+ * FUNC, FUNC_PROTO, VAR, DECL_TAG and TYPE_TAG.
* "type" is a type_id referring to another type.
*/
union {
@@ -75,6 +75,7 @@ enum {
BTF_KIND_DATASEC = 15, /* Section */
BTF_KIND_FLOAT = 16, /* Floating point */
BTF_KIND_DECL_TAG = 17, /* Decl Tag */
+ BTF_KIND_TYPE_TAG = 18, /* Type Tag */
NR_BTF_KINDS,
BTF_KIND_MAX = NR_BTF_KINDS - 1,
diff --git a/tools/include/uapi/linux/if_link.h b/tools/include/uapi/linux/if_link.h
index eebd389..6218f93 100644
--- a/tools/include/uapi/linux/if_link.h
+++ b/tools/include/uapi/linux/if_link.h
@@ -347,6 +347,7 @@ enum {
*/
IFLA_PARENT_DEV_NAME,
IFLA_PARENT_DEV_BUS_NAME,
+ IFLA_GRO_MAX_SIZE,
__IFLA_MAX
};
@@ -858,6 +859,7 @@ enum {
IFLA_BOND_TLB_DYNAMIC_LB,
IFLA_BOND_PEER_NOTIF_DELAY,
IFLA_BOND_AD_LACP_ACTIVE,
+ IFLA_BOND_MISSED_MAX,
__IFLA_BOND_MAX,
};
diff --git a/tools/lib/bpf/Makefile b/tools/lib/bpf/Makefile
index b393b5e..f947b61 100644
--- a/tools/lib/bpf/Makefile
+++ b/tools/lib/bpf/Makefile
@@ -84,11 +84,13 @@
endif
# Append required CFLAGS
+override CFLAGS += -std=gnu89
override CFLAGS += $(EXTRA_WARNINGS) -Wno-switch-enum
override CFLAGS += -Werror -Wall
override CFLAGS += $(INCLUDES)
override CFLAGS += -fvisibility=hidden
override CFLAGS += -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64
+override CFLAGS += $(CLANG_CROSS_FLAGS)
# flags specific for shared library
SHLIB_FLAGS := -DSHARED -fPIC
@@ -161,7 +163,7 @@
$(OUTPUT)libbpf.so: $(OUTPUT)libbpf.so.$(LIBBPF_VERSION)
$(OUTPUT)libbpf.so.$(LIBBPF_VERSION): $(BPF_IN_SHARED) $(VERSION_SCRIPT)
- $(QUIET_LINK)$(CC) $(LDFLAGS) \
+ $(QUIET_LINK)$(CC) $(CFLAGS) $(LDFLAGS) \
--shared -Wl,-soname,libbpf.so.$(LIBBPF_MAJOR_VERSION) \
-Wl,--version-script=$(VERSION_SCRIPT) $< -lelf -lz -o $@
@ln -sf $(@F) $(OUTPUT)libbpf.so
diff --git a/tools/lib/bpf/bpf.c b/tools/lib/bpf/bpf.c
index 7257012..550b4cb 100644
--- a/tools/lib/bpf/bpf.c
+++ b/tools/lib/bpf/bpf.c
@@ -28,6 +28,9 @@
#include <asm/unistd.h>
#include <errno.h>
#include <linux/bpf.h>
+#include <linux/filter.h>
+#include <limits.h>
+#include <sys/resource.h>
#include "bpf.h"
#include "libbpf.h"
#include "libbpf_internal.h"
@@ -49,6 +52,12 @@
# define __NR_bpf 351
# elif defined(__arc__)
# define __NR_bpf 280
+# elif defined(__mips__) && defined(_ABIO32)
+# define __NR_bpf 4355
+# elif defined(__mips__) && defined(_ABIN32)
+# define __NR_bpf 6319
+# elif defined(__mips__) && defined(_ABI64)
+# define __NR_bpf 5315
# else
# error __NR_bpf not defined. libbpf does not support your arch.
# endif
@@ -74,158 +83,208 @@ static inline int sys_bpf_fd(enum bpf_cmd cmd, union bpf_attr *attr,
return ensure_good_fd(fd);
}
-static inline int sys_bpf_prog_load(union bpf_attr *attr, unsigned int size)
+#define PROG_LOAD_ATTEMPTS 5
+
+static inline int sys_bpf_prog_load(union bpf_attr *attr, unsigned int size, int attempts)
{
- int retries = 5;
int fd;
do {
fd = sys_bpf_fd(BPF_PROG_LOAD, attr, size);
- } while (fd < 0 && errno == EAGAIN && retries-- > 0);
+ } while (fd < 0 && errno == EAGAIN && --attempts > 0);
return fd;
}
-int libbpf__bpf_create_map_xattr(const struct bpf_create_map_params *create_attr)
+/* Probe whether kernel switched from memlock-based (RLIMIT_MEMLOCK) to
+ * memcg-based memory accounting for BPF maps and progs. This was done in [0].
+ * We use the support for bpf_ktime_get_coarse_ns() helper, which was added in
+ * the same 5.11 Linux release ([1]), to detect memcg-based accounting for BPF.
+ *
+ * [0] https://lore.kernel.org/bpf/20201201215900.3569844-1-guro@fb.com/
+ * [1] d05512618056 ("bpf: Add bpf_ktime_get_coarse_ns helper")
+ */
+int probe_memcg_account(void)
{
+ const size_t prog_load_attr_sz = offsetofend(union bpf_attr, attach_btf_obj_fd);
+ struct bpf_insn insns[] = {
+ BPF_EMIT_CALL(BPF_FUNC_ktime_get_coarse_ns),
+ BPF_EXIT_INSN(),
+ };
+ size_t insn_cnt = sizeof(insns) / sizeof(insns[0]);
+ union bpf_attr attr;
+ int prog_fd;
+
+ /* attempt loading freplace trying to use custom BTF */
+ memset(&attr, 0, prog_load_attr_sz);
+ attr.prog_type = BPF_PROG_TYPE_SOCKET_FILTER;
+ attr.insns = ptr_to_u64(insns);
+ attr.insn_cnt = insn_cnt;
+ attr.license = ptr_to_u64("GPL");
+
+ prog_fd = sys_bpf_fd(BPF_PROG_LOAD, &attr, prog_load_attr_sz);
+ if (prog_fd >= 0) {
+ close(prog_fd);
+ return 1;
+ }
+ return 0;
+}
+
+static bool memlock_bumped;
+static rlim_t memlock_rlim = RLIM_INFINITY;
+
+int libbpf_set_memlock_rlim(size_t memlock_bytes)
+{
+ if (memlock_bumped)
+ return libbpf_err(-EBUSY);
+
+ memlock_rlim = memlock_bytes;
+ return 0;
+}
+
+int bump_rlimit_memlock(void)
+{
+ struct rlimit rlim;
+
+ /* this the default in libbpf 1.0, but for now user has to opt-in explicitly */
+ if (!(libbpf_mode & LIBBPF_STRICT_AUTO_RLIMIT_MEMLOCK))
+ return 0;
+
+ /* if kernel supports memcg-based accounting, skip bumping RLIMIT_MEMLOCK */
+ if (memlock_bumped || kernel_supports(NULL, FEAT_MEMCG_ACCOUNT))
+ return 0;
+
+ memlock_bumped = true;
+
+ /* zero memlock_rlim_max disables auto-bumping RLIMIT_MEMLOCK */
+ if (memlock_rlim == 0)
+ return 0;
+
+ rlim.rlim_cur = rlim.rlim_max = memlock_rlim;
+ if (setrlimit(RLIMIT_MEMLOCK, &rlim))
+ return -errno;
+
+ return 0;
+}
+
+int bpf_map_create(enum bpf_map_type map_type,
+ const char *map_name,
+ __u32 key_size,
+ __u32 value_size,
+ __u32 max_entries,
+ const struct bpf_map_create_opts *opts)
+{
+ const size_t attr_sz = offsetofend(union bpf_attr, map_extra);
union bpf_attr attr;
int fd;
- memset(&attr, '\0', sizeof(attr));
+ bump_rlimit_memlock();
- attr.map_type = create_attr->map_type;
- attr.key_size = create_attr->key_size;
- attr.value_size = create_attr->value_size;
- attr.max_entries = create_attr->max_entries;
- attr.map_flags = create_attr->map_flags;
- if (create_attr->name)
- memcpy(attr.map_name, create_attr->name,
- min(strlen(create_attr->name), BPF_OBJ_NAME_LEN - 1));
- attr.numa_node = create_attr->numa_node;
- attr.btf_fd = create_attr->btf_fd;
- attr.btf_key_type_id = create_attr->btf_key_type_id;
- attr.btf_value_type_id = create_attr->btf_value_type_id;
- attr.map_ifindex = create_attr->map_ifindex;
- if (attr.map_type == BPF_MAP_TYPE_STRUCT_OPS)
- attr.btf_vmlinux_value_type_id =
- create_attr->btf_vmlinux_value_type_id;
- else
- attr.inner_map_fd = create_attr->inner_map_fd;
- attr.map_extra = create_attr->map_extra;
+ memset(&attr, 0, attr_sz);
- fd = sys_bpf_fd(BPF_MAP_CREATE, &attr, sizeof(attr));
+ if (!OPTS_VALID(opts, bpf_map_create_opts))
+ return libbpf_err(-EINVAL);
+
+ attr.map_type = map_type;
+ if (map_name)
+ libbpf_strlcpy(attr.map_name, map_name, sizeof(attr.map_name));
+ attr.key_size = key_size;
+ attr.value_size = value_size;
+ attr.max_entries = max_entries;
+
+ attr.btf_fd = OPTS_GET(opts, btf_fd, 0);
+ attr.btf_key_type_id = OPTS_GET(opts, btf_key_type_id, 0);
+ attr.btf_value_type_id = OPTS_GET(opts, btf_value_type_id, 0);
+ attr.btf_vmlinux_value_type_id = OPTS_GET(opts, btf_vmlinux_value_type_id, 0);
+
+ attr.inner_map_fd = OPTS_GET(opts, inner_map_fd, 0);
+ attr.map_flags = OPTS_GET(opts, map_flags, 0);
+ attr.map_extra = OPTS_GET(opts, map_extra, 0);
+ attr.numa_node = OPTS_GET(opts, numa_node, 0);
+ attr.map_ifindex = OPTS_GET(opts, map_ifindex, 0);
+
+ fd = sys_bpf_fd(BPF_MAP_CREATE, &attr, attr_sz);
return libbpf_err_errno(fd);
}
int bpf_create_map_xattr(const struct bpf_create_map_attr *create_attr)
{
- struct bpf_create_map_params p = {};
+ LIBBPF_OPTS(bpf_map_create_opts, p);
- p.map_type = create_attr->map_type;
- p.key_size = create_attr->key_size;
- p.value_size = create_attr->value_size;
- p.max_entries = create_attr->max_entries;
p.map_flags = create_attr->map_flags;
- p.name = create_attr->name;
p.numa_node = create_attr->numa_node;
p.btf_fd = create_attr->btf_fd;
p.btf_key_type_id = create_attr->btf_key_type_id;
p.btf_value_type_id = create_attr->btf_value_type_id;
p.map_ifindex = create_attr->map_ifindex;
- if (p.map_type == BPF_MAP_TYPE_STRUCT_OPS)
- p.btf_vmlinux_value_type_id =
- create_attr->btf_vmlinux_value_type_id;
+ if (create_attr->map_type == BPF_MAP_TYPE_STRUCT_OPS)
+ p.btf_vmlinux_value_type_id = create_attr->btf_vmlinux_value_type_id;
else
p.inner_map_fd = create_attr->inner_map_fd;
- return libbpf__bpf_create_map_xattr(&p);
+ return bpf_map_create(create_attr->map_type, create_attr->name,
+ create_attr->key_size, create_attr->value_size,
+ create_attr->max_entries, &p);
}
int bpf_create_map_node(enum bpf_map_type map_type, const char *name,
int key_size, int value_size, int max_entries,
__u32 map_flags, int node)
{
- struct bpf_create_map_attr map_attr = {};
+ LIBBPF_OPTS(bpf_map_create_opts, opts);
- map_attr.name = name;
- map_attr.map_type = map_type;
- map_attr.map_flags = map_flags;
- map_attr.key_size = key_size;
- map_attr.value_size = value_size;
- map_attr.max_entries = max_entries;
+ opts.map_flags = map_flags;
if (node >= 0) {
- map_attr.numa_node = node;
- map_attr.map_flags |= BPF_F_NUMA_NODE;
+ opts.numa_node = node;
+ opts.map_flags |= BPF_F_NUMA_NODE;
}
- return bpf_create_map_xattr(&map_attr);
+ return bpf_map_create(map_type, name, key_size, value_size, max_entries, &opts);
}
int bpf_create_map(enum bpf_map_type map_type, int key_size,
int value_size, int max_entries, __u32 map_flags)
{
- struct bpf_create_map_attr map_attr = {};
+ LIBBPF_OPTS(bpf_map_create_opts, opts, .map_flags = map_flags);
- map_attr.map_type = map_type;
- map_attr.map_flags = map_flags;
- map_attr.key_size = key_size;
- map_attr.value_size = value_size;
- map_attr.max_entries = max_entries;
-
- return bpf_create_map_xattr(&map_attr);
+ return bpf_map_create(map_type, NULL, key_size, value_size, max_entries, &opts);
}
int bpf_create_map_name(enum bpf_map_type map_type, const char *name,
int key_size, int value_size, int max_entries,
__u32 map_flags)
{
- struct bpf_create_map_attr map_attr = {};
+ LIBBPF_OPTS(bpf_map_create_opts, opts, .map_flags = map_flags);
- map_attr.name = name;
- map_attr.map_type = map_type;
- map_attr.map_flags = map_flags;
- map_attr.key_size = key_size;
- map_attr.value_size = value_size;
- map_attr.max_entries = max_entries;
-
- return bpf_create_map_xattr(&map_attr);
+ return bpf_map_create(map_type, name, key_size, value_size, max_entries, &opts);
}
int bpf_create_map_in_map_node(enum bpf_map_type map_type, const char *name,
int key_size, int inner_map_fd, int max_entries,
__u32 map_flags, int node)
{
- union bpf_attr attr;
- int fd;
+ LIBBPF_OPTS(bpf_map_create_opts, opts);
- memset(&attr, '\0', sizeof(attr));
-
- attr.map_type = map_type;
- attr.key_size = key_size;
- attr.value_size = 4;
- attr.inner_map_fd = inner_map_fd;
- attr.max_entries = max_entries;
- attr.map_flags = map_flags;
- if (name)
- memcpy(attr.map_name, name,
- min(strlen(name), BPF_OBJ_NAME_LEN - 1));
-
+ opts.inner_map_fd = inner_map_fd;
+ opts.map_flags = map_flags;
if (node >= 0) {
- attr.map_flags |= BPF_F_NUMA_NODE;
- attr.numa_node = node;
+ opts.map_flags |= BPF_F_NUMA_NODE;
+ opts.numa_node = node;
}
- fd = sys_bpf_fd(BPF_MAP_CREATE, &attr, sizeof(attr));
- return libbpf_err_errno(fd);
+ return bpf_map_create(map_type, name, key_size, 4, max_entries, &opts);
}
int bpf_create_map_in_map(enum bpf_map_type map_type, const char *name,
int key_size, int inner_map_fd, int max_entries,
__u32 map_flags)
{
- return bpf_create_map_in_map_node(map_type, name, key_size,
- inner_map_fd, max_entries, map_flags,
- -1);
+ LIBBPF_OPTS(bpf_map_create_opts, opts,
+ .inner_map_fd = inner_map_fd,
+ .map_flags = map_flags,
+ );
+
+ return bpf_map_create(map_type, name, key_size, 4, max_entries, &opts);
}
static void *
@@ -253,58 +312,95 @@ alloc_zero_tailing_info(const void *orecord, __u32 cnt,
return info;
}
-int libbpf__bpf_prog_load(const struct bpf_prog_load_params *load_attr)
+DEFAULT_VERSION(bpf_prog_load_v0_6_0, bpf_prog_load, LIBBPF_0.6.0)
+int bpf_prog_load_v0_6_0(enum bpf_prog_type prog_type,
+ const char *prog_name, const char *license,
+ const struct bpf_insn *insns, size_t insn_cnt,
+ const struct bpf_prog_load_opts *opts)
{
void *finfo = NULL, *linfo = NULL;
+ const char *func_info, *line_info;
+ __u32 log_size, log_level, attach_prog_fd, attach_btf_obj_fd;
+ __u32 func_info_rec_size, line_info_rec_size;
+ int fd, attempts;
union bpf_attr attr;
- int fd;
+ char *log_buf;
- if (!load_attr->log_buf != !load_attr->log_buf_sz)
+ bump_rlimit_memlock();
+
+ if (!OPTS_VALID(opts, bpf_prog_load_opts))
return libbpf_err(-EINVAL);
- if (load_attr->log_level > (4 | 2 | 1) || (load_attr->log_level && !load_attr->log_buf))
+ attempts = OPTS_GET(opts, attempts, 0);
+ if (attempts < 0)
return libbpf_err(-EINVAL);
+ if (attempts == 0)
+ attempts = PROG_LOAD_ATTEMPTS;
memset(&attr, 0, sizeof(attr));
- attr.prog_type = load_attr->prog_type;
- attr.expected_attach_type = load_attr->expected_attach_type;
- if (load_attr->attach_prog_fd)
- attr.attach_prog_fd = load_attr->attach_prog_fd;
+ attr.prog_type = prog_type;
+ attr.expected_attach_type = OPTS_GET(opts, expected_attach_type, 0);
+
+ attr.prog_btf_fd = OPTS_GET(opts, prog_btf_fd, 0);
+ attr.prog_flags = OPTS_GET(opts, prog_flags, 0);
+ attr.prog_ifindex = OPTS_GET(opts, prog_ifindex, 0);
+ attr.kern_version = OPTS_GET(opts, kern_version, 0);
+
+ if (prog_name)
+ libbpf_strlcpy(attr.prog_name, prog_name, sizeof(attr.prog_name));
+ attr.license = ptr_to_u64(license);
+
+ if (insn_cnt > UINT_MAX)
+ return libbpf_err(-E2BIG);
+
+ attr.insns = ptr_to_u64(insns);
+ attr.insn_cnt = (__u32)insn_cnt;
+
+ attach_prog_fd = OPTS_GET(opts, attach_prog_fd, 0);
+ attach_btf_obj_fd = OPTS_GET(opts, attach_btf_obj_fd, 0);
+
+ if (attach_prog_fd && attach_btf_obj_fd)
+ return libbpf_err(-EINVAL);
+
+ attr.attach_btf_id = OPTS_GET(opts, attach_btf_id, 0);
+ if (attach_prog_fd)
+ attr.attach_prog_fd = attach_prog_fd;
else
- attr.attach_btf_obj_fd = load_attr->attach_btf_obj_fd;
- attr.attach_btf_id = load_attr->attach_btf_id;
+ attr.attach_btf_obj_fd = attach_btf_obj_fd;
- attr.prog_ifindex = load_attr->prog_ifindex;
- attr.kern_version = load_attr->kern_version;
+ log_buf = OPTS_GET(opts, log_buf, NULL);
+ log_size = OPTS_GET(opts, log_size, 0);
+ log_level = OPTS_GET(opts, log_level, 0);
- attr.insn_cnt = (__u32)load_attr->insn_cnt;
- attr.insns = ptr_to_u64(load_attr->insns);
- attr.license = ptr_to_u64(load_attr->license);
+ if (!!log_buf != !!log_size)
+ return libbpf_err(-EINVAL);
+ if (log_level > (4 | 2 | 1))
+ return libbpf_err(-EINVAL);
+ if (log_level && !log_buf)
+ return libbpf_err(-EINVAL);
- attr.log_level = load_attr->log_level;
- if (attr.log_level) {
- attr.log_buf = ptr_to_u64(load_attr->log_buf);
- attr.log_size = load_attr->log_buf_sz;
+ func_info_rec_size = OPTS_GET(opts, func_info_rec_size, 0);
+ func_info = OPTS_GET(opts, func_info, NULL);
+ attr.func_info_rec_size = func_info_rec_size;
+ attr.func_info = ptr_to_u64(func_info);
+ attr.func_info_cnt = OPTS_GET(opts, func_info_cnt, 0);
+
+ line_info_rec_size = OPTS_GET(opts, line_info_rec_size, 0);
+ line_info = OPTS_GET(opts, line_info, NULL);
+ attr.line_info_rec_size = line_info_rec_size;
+ attr.line_info = ptr_to_u64(line_info);
+ attr.line_info_cnt = OPTS_GET(opts, line_info_cnt, 0);
+
+ attr.fd_array = ptr_to_u64(OPTS_GET(opts, fd_array, NULL));
+
+ if (log_level) {
+ attr.log_buf = ptr_to_u64(log_buf);
+ attr.log_size = log_size;
+ attr.log_level = log_level;
}
- attr.prog_btf_fd = load_attr->prog_btf_fd;
- attr.prog_flags = load_attr->prog_flags;
-
- attr.func_info_rec_size = load_attr->func_info_rec_size;
- attr.func_info_cnt = load_attr->func_info_cnt;
- attr.func_info = ptr_to_u64(load_attr->func_info);
-
- attr.line_info_rec_size = load_attr->line_info_rec_size;
- attr.line_info_cnt = load_attr->line_info_cnt;
- attr.line_info = ptr_to_u64(load_attr->line_info);
- attr.fd_array = ptr_to_u64(load_attr->fd_array);
-
- if (load_attr->name)
- memcpy(attr.prog_name, load_attr->name,
- min(strlen(load_attr->name), (size_t)BPF_OBJ_NAME_LEN - 1));
-
- fd = sys_bpf_prog_load(&attr, sizeof(attr));
+ fd = sys_bpf_prog_load(&attr, sizeof(attr), attempts);
if (fd >= 0)
return fd;
@@ -314,11 +410,11 @@ int libbpf__bpf_prog_load(const struct bpf_prog_load_params *load_attr)
*/
while (errno == E2BIG && (!finfo || !linfo)) {
if (!finfo && attr.func_info_cnt &&
- attr.func_info_rec_size < load_attr->func_info_rec_size) {
+ attr.func_info_rec_size < func_info_rec_size) {
/* try with corrected func info records */
- finfo = alloc_zero_tailing_info(load_attr->func_info,
- load_attr->func_info_cnt,
- load_attr->func_info_rec_size,
+ finfo = alloc_zero_tailing_info(func_info,
+ attr.func_info_cnt,
+ func_info_rec_size,
attr.func_info_rec_size);
if (!finfo) {
errno = E2BIG;
@@ -326,13 +422,12 @@ int libbpf__bpf_prog_load(const struct bpf_prog_load_params *load_attr)
}
attr.func_info = ptr_to_u64(finfo);
- attr.func_info_rec_size = load_attr->func_info_rec_size;
+ attr.func_info_rec_size = func_info_rec_size;
} else if (!linfo && attr.line_info_cnt &&
- attr.line_info_rec_size <
- load_attr->line_info_rec_size) {
- linfo = alloc_zero_tailing_info(load_attr->line_info,
- load_attr->line_info_cnt,
- load_attr->line_info_rec_size,
+ attr.line_info_rec_size < line_info_rec_size) {
+ linfo = alloc_zero_tailing_info(line_info,
+ attr.line_info_cnt,
+ line_info_rec_size,
attr.line_info_rec_size);
if (!linfo) {
errno = E2BIG;
@@ -340,26 +435,27 @@ int libbpf__bpf_prog_load(const struct bpf_prog_load_params *load_attr)
}
attr.line_info = ptr_to_u64(linfo);
- attr.line_info_rec_size = load_attr->line_info_rec_size;
+ attr.line_info_rec_size = line_info_rec_size;
} else {
break;
}
- fd = sys_bpf_prog_load(&attr, sizeof(attr));
+ fd = sys_bpf_prog_load(&attr, sizeof(attr), attempts);
if (fd >= 0)
goto done;
}
- if (load_attr->log_level || !load_attr->log_buf)
- goto done;
+ if (log_level == 0 && log_buf) {
+ /* log_level == 0 with non-NULL log_buf requires retrying on error
+ * with log_level == 1 and log_buf/log_buf_size set, to get details of
+ * failure
+ */
+ attr.log_buf = ptr_to_u64(log_buf);
+ attr.log_size = log_size;
+ attr.log_level = 1;
- /* Try again with log */
- attr.log_buf = ptr_to_u64(load_attr->log_buf);
- attr.log_size = load_attr->log_buf_sz;
- attr.log_level = 1;
- load_attr->log_buf[0] = 0;
-
- fd = sys_bpf_prog_load(&attr, sizeof(attr));
+ fd = sys_bpf_prog_load(&attr, sizeof(attr), attempts);
+ }
done:
/* free() doesn't affect errno, so we don't need to restore it */
free(finfo);
@@ -367,17 +463,20 @@ int libbpf__bpf_prog_load(const struct bpf_prog_load_params *load_attr)
return libbpf_err_errno(fd);
}
+__attribute__((alias("bpf_load_program_xattr2")))
int bpf_load_program_xattr(const struct bpf_load_program_attr *load_attr,
- char *log_buf, size_t log_buf_sz)
+ char *log_buf, size_t log_buf_sz);
+
+static int bpf_load_program_xattr2(const struct bpf_load_program_attr *load_attr,
+ char *log_buf, size_t log_buf_sz)
{
- struct bpf_prog_load_params p = {};
+ LIBBPF_OPTS(bpf_prog_load_opts, p);
if (!load_attr || !log_buf != !log_buf_sz)
return libbpf_err(-EINVAL);
- p.prog_type = load_attr->prog_type;
p.expected_attach_type = load_attr->expected_attach_type;
- switch (p.prog_type) {
+ switch (load_attr->prog_type) {
case BPF_PROG_TYPE_STRUCT_OPS:
case BPF_PROG_TYPE_LSM:
p.attach_btf_id = load_attr->attach_btf_id;
@@ -391,12 +490,9 @@ int bpf_load_program_xattr(const struct bpf_load_program_attr *load_attr,
p.prog_ifindex = load_attr->prog_ifindex;
p.kern_version = load_attr->kern_version;
}
- p.insn_cnt = load_attr->insns_cnt;
- p.insns = load_attr->insns;
- p.license = load_attr->license;
p.log_level = load_attr->log_level;
p.log_buf = log_buf;
- p.log_buf_sz = log_buf_sz;
+ p.log_size = log_buf_sz;
p.prog_btf_fd = load_attr->prog_btf_fd;
p.func_info_rec_size = load_attr->func_info_rec_size;
p.func_info_cnt = load_attr->func_info_cnt;
@@ -404,10 +500,10 @@ int bpf_load_program_xattr(const struct bpf_load_program_attr *load_attr,
p.line_info_rec_size = load_attr->line_info_rec_size;
p.line_info_cnt = load_attr->line_info_cnt;
p.line_info = load_attr->line_info;
- p.name = load_attr->name;
p.prog_flags = load_attr->prog_flags;
- return libbpf__bpf_prog_load(&p);
+ return bpf_prog_load(load_attr->prog_type, load_attr->name, load_attr->license,
+ load_attr->insns, load_attr->insns_cnt, &p);
}
int bpf_load_program(enum bpf_prog_type type, const struct bpf_insn *insns,
@@ -426,7 +522,7 @@ int bpf_load_program(enum bpf_prog_type type, const struct bpf_insn *insns,
load_attr.license = license;
load_attr.kern_version = kern_version;
- return bpf_load_program_xattr(&load_attr, log_buf, log_buf_sz);
+ return bpf_load_program_xattr2(&load_attr, log_buf, log_buf_sz);
}
int bpf_verify_program(enum bpf_prog_type type, const struct bpf_insn *insns,
@@ -437,6 +533,8 @@ int bpf_verify_program(enum bpf_prog_type type, const struct bpf_insn *insns,
union bpf_attr attr;
int fd;
+ bump_rlimit_memlock();
+
memset(&attr, 0, sizeof(attr));
attr.prog_type = type;
attr.insn_cnt = (__u32)insns_cnt;
@@ -449,7 +547,7 @@ int bpf_verify_program(enum bpf_prog_type type, const struct bpf_insn *insns,
attr.kern_version = kern_version;
attr.prog_flags = prog_flags;
- fd = sys_bpf_prog_load(&attr, sizeof(attr));
+ fd = sys_bpf_prog_load(&attr, sizeof(attr), PROG_LOAD_ATTEMPTS);
return libbpf_err_errno(fd);
}
@@ -593,11 +691,11 @@ static int bpf_map_batch_common(int cmd, int fd, void *in_batch,
return libbpf_err_errno(ret);
}
-int bpf_map_delete_batch(int fd, void *keys, __u32 *count,
+int bpf_map_delete_batch(int fd, const void *keys, __u32 *count,
const struct bpf_map_batch_opts *opts)
{
return bpf_map_batch_common(BPF_MAP_DELETE_BATCH, fd, NULL,
- NULL, keys, NULL, count, opts);
+ NULL, (void *)keys, NULL, count, opts);
}
int bpf_map_lookup_batch(int fd, void *in_batch, void *out_batch, void *keys,
@@ -617,11 +715,11 @@ int bpf_map_lookup_and_delete_batch(int fd, void *in_batch, void *out_batch,
count, opts);
}
-int bpf_map_update_batch(int fd, void *keys, void *values, __u32 *count,
+int bpf_map_update_batch(int fd, const void *keys, const void *values, __u32 *count,
const struct bpf_map_batch_opts *opts)
{
return bpf_map_batch_common(BPF_MAP_UPDATE_BATCH, fd, NULL, NULL,
- keys, values, count, opts);
+ (void *)keys, (void *)values, count, opts);
}
int bpf_obj_pin(int fd, const char *pathname)
@@ -1028,24 +1126,67 @@ int bpf_raw_tracepoint_open(const char *name, int prog_fd)
return libbpf_err_errno(fd);
}
-int bpf_load_btf(const void *btf, __u32 btf_size, char *log_buf, __u32 log_buf_size,
- bool do_log)
+int bpf_btf_load(const void *btf_data, size_t btf_size, const struct bpf_btf_load_opts *opts)
{
- union bpf_attr attr = {};
+ const size_t attr_sz = offsetofend(union bpf_attr, btf_log_level);
+ union bpf_attr attr;
+ char *log_buf;
+ size_t log_size;
+ __u32 log_level;
int fd;
- attr.btf = ptr_to_u64(btf);
+ bump_rlimit_memlock();
+
+ memset(&attr, 0, attr_sz);
+
+ if (!OPTS_VALID(opts, bpf_btf_load_opts))
+ return libbpf_err(-EINVAL);
+
+ log_buf = OPTS_GET(opts, log_buf, NULL);
+ log_size = OPTS_GET(opts, log_size, 0);
+ log_level = OPTS_GET(opts, log_level, 0);
+
+ if (log_size > UINT_MAX)
+ return libbpf_err(-EINVAL);
+ if (log_size && !log_buf)
+ return libbpf_err(-EINVAL);
+
+ attr.btf = ptr_to_u64(btf_data);
attr.btf_size = btf_size;
+ /* log_level == 0 and log_buf != NULL means "try loading without
+ * log_buf, but retry with log_buf and log_level=1 on error", which is
+ * consistent across low-level and high-level BTF and program loading
+ * APIs within libbpf and provides a sensible behavior in practice
+ */
+ if (log_level) {
+ attr.btf_log_buf = ptr_to_u64(log_buf);
+ attr.btf_log_size = (__u32)log_size;
+ attr.btf_log_level = log_level;
+ }
+
+ fd = sys_bpf_fd(BPF_BTF_LOAD, &attr, attr_sz);
+ if (fd < 0 && log_buf && log_level == 0) {
+ attr.btf_log_buf = ptr_to_u64(log_buf);
+ attr.btf_log_size = (__u32)log_size;
+ attr.btf_log_level = 1;
+ fd = sys_bpf_fd(BPF_BTF_LOAD, &attr, attr_sz);
+ }
+ return libbpf_err_errno(fd);
+}
+
+int bpf_load_btf(const void *btf, __u32 btf_size, char *log_buf, __u32 log_buf_size, bool do_log)
+{
+ LIBBPF_OPTS(bpf_btf_load_opts, opts);
+ int fd;
retry:
if (do_log && log_buf && log_buf_size) {
- attr.btf_log_level = 1;
- attr.btf_log_size = log_buf_size;
- attr.btf_log_buf = ptr_to_u64(log_buf);
+ opts.log_buf = log_buf;
+ opts.log_size = log_buf_size;
+ opts.log_level = 1;
}
- fd = sys_bpf_fd(BPF_BTF_LOAD, &attr, sizeof(attr));
-
+ fd = bpf_btf_load(btf, btf_size, &opts);
if (fd < 0 && !do_log && log_buf && log_buf_size) {
do_log = true;
goto retry;
diff --git a/tools/lib/bpf/bpf.h b/tools/lib/bpf/bpf.h
index 6fffb3c..14e0d97 100644
--- a/tools/lib/bpf/bpf.h
+++ b/tools/lib/bpf/bpf.h
@@ -29,11 +29,38 @@
#include <stdint.h>
#include "libbpf_common.h"
+#include "libbpf_legacy.h"
#ifdef __cplusplus
extern "C" {
#endif
+int libbpf_set_memlock_rlim(size_t memlock_bytes);
+
+struct bpf_map_create_opts {
+ size_t sz; /* size of this struct for forward/backward compatibility */
+
+ __u32 btf_fd;
+ __u32 btf_key_type_id;
+ __u32 btf_value_type_id;
+ __u32 btf_vmlinux_value_type_id;
+
+ __u32 inner_map_fd;
+ __u32 map_flags;
+ __u64 map_extra;
+
+ __u32 numa_node;
+ __u32 map_ifindex;
+};
+#define bpf_map_create_opts__last_field map_ifindex
+
+LIBBPF_API int bpf_map_create(enum bpf_map_type map_type,
+ const char *map_name,
+ __u32 key_size,
+ __u32 value_size,
+ __u32 max_entries,
+ const struct bpf_map_create_opts *opts);
+
struct bpf_create_map_attr {
const char *name;
enum bpf_map_type map_type;
@@ -52,25 +79,95 @@ struct bpf_create_map_attr {
};
};
-LIBBPF_API int
-bpf_create_map_xattr(const struct bpf_create_map_attr *create_attr);
+LIBBPF_DEPRECATED_SINCE(0, 7, "use bpf_map_create() instead")
+LIBBPF_API int bpf_create_map_xattr(const struct bpf_create_map_attr *create_attr);
+LIBBPF_DEPRECATED_SINCE(0, 7, "use bpf_map_create() instead")
LIBBPF_API int bpf_create_map_node(enum bpf_map_type map_type, const char *name,
int key_size, int value_size,
int max_entries, __u32 map_flags, int node);
+LIBBPF_DEPRECATED_SINCE(0, 7, "use bpf_map_create() instead")
LIBBPF_API int bpf_create_map_name(enum bpf_map_type map_type, const char *name,
int key_size, int value_size,
int max_entries, __u32 map_flags);
+LIBBPF_DEPRECATED_SINCE(0, 7, "use bpf_map_create() instead")
LIBBPF_API int bpf_create_map(enum bpf_map_type map_type, int key_size,
int value_size, int max_entries, __u32 map_flags);
+LIBBPF_DEPRECATED_SINCE(0, 7, "use bpf_map_create() instead")
LIBBPF_API int bpf_create_map_in_map_node(enum bpf_map_type map_type,
const char *name, int key_size,
int inner_map_fd, int max_entries,
__u32 map_flags, int node);
+LIBBPF_DEPRECATED_SINCE(0, 7, "use bpf_map_create() instead")
LIBBPF_API int bpf_create_map_in_map(enum bpf_map_type map_type,
const char *name, int key_size,
int inner_map_fd, int max_entries,
__u32 map_flags);
+struct bpf_prog_load_opts {
+ size_t sz; /* size of this struct for forward/backward compatibility */
+
+ /* libbpf can retry BPF_PROG_LOAD command if bpf() syscall returns
+ * -EAGAIN. This field determines how many attempts libbpf has to
+ * make. If not specified, libbpf will use default value of 5.
+ */
+ int attempts;
+
+ enum bpf_attach_type expected_attach_type;
+ __u32 prog_btf_fd;
+ __u32 prog_flags;
+ __u32 prog_ifindex;
+ __u32 kern_version;
+
+ __u32 attach_btf_id;
+ __u32 attach_prog_fd;
+ __u32 attach_btf_obj_fd;
+
+ const int *fd_array;
+
+ /* .BTF.ext func info data */
+ const void *func_info;
+ __u32 func_info_cnt;
+ __u32 func_info_rec_size;
+
+ /* .BTF.ext line info data */
+ const void *line_info;
+ __u32 line_info_cnt;
+ __u32 line_info_rec_size;
+
+ /* verifier log options */
+ __u32 log_level;
+ __u32 log_size;
+ char *log_buf;
+};
+#define bpf_prog_load_opts__last_field log_buf
+
+LIBBPF_API int bpf_prog_load(enum bpf_prog_type prog_type,
+ const char *prog_name, const char *license,
+ const struct bpf_insn *insns, size_t insn_cnt,
+ const struct bpf_prog_load_opts *opts);
+/* this "specialization" should go away in libbpf 1.0 */
+LIBBPF_API int bpf_prog_load_v0_6_0(enum bpf_prog_type prog_type,
+ const char *prog_name, const char *license,
+ const struct bpf_insn *insns, size_t insn_cnt,
+ const struct bpf_prog_load_opts *opts);
+
+/* This is an elaborate way to not conflict with deprecated bpf_prog_load()
+ * API, defined in libbpf.h. Once we hit libbpf 1.0, all this will be gone.
+ * With this approach, if someone is calling bpf_prog_load() with
+ * 4 arguments, they will use the deprecated API, which keeps backwards
+ * compatibility (both source code and binary). If bpf_prog_load() is called
+ * with 6 arguments, though, it gets redirected to __bpf_prog_load.
+ * So looking forward to libbpf 1.0 when this hack will be gone and
+ * __bpf_prog_load() will be called just bpf_prog_load().
+ */
+#ifndef bpf_prog_load
+#define bpf_prog_load(...) ___libbpf_overload(___bpf_prog_load, __VA_ARGS__)
+#define ___bpf_prog_load4(file, type, pobj, prog_fd) \
+ bpf_prog_load_deprecated(file, type, pobj, prog_fd)
+#define ___bpf_prog_load6(prog_type, prog_name, license, insns, insn_cnt, opts) \
+ bpf_prog_load(prog_type, prog_name, license, insns, insn_cnt, opts)
+#endif /* bpf_prog_load */
+
struct bpf_load_program_attr {
enum bpf_prog_type prog_type;
enum bpf_attach_type expected_attach_type;
@@ -100,15 +197,18 @@ struct bpf_load_program_attr {
/* Flags to direct loading requirements */
#define MAPS_RELAX_COMPAT 0x01
-/* Recommend log buffer size */
+/* Recommended log buffer size */
#define BPF_LOG_BUF_SIZE (UINT32_MAX >> 8) /* verifier maximum in kernels <= 5.1 */
-LIBBPF_API int
-bpf_load_program_xattr(const struct bpf_load_program_attr *load_attr,
- char *log_buf, size_t log_buf_sz);
+
+LIBBPF_DEPRECATED_SINCE(0, 7, "use bpf_prog_load() instead")
+LIBBPF_API int bpf_load_program_xattr(const struct bpf_load_program_attr *load_attr,
+ char *log_buf, size_t log_buf_sz);
+LIBBPF_DEPRECATED_SINCE(0, 7, "use bpf_prog_load() instead")
LIBBPF_API int bpf_load_program(enum bpf_prog_type type,
const struct bpf_insn *insns, size_t insns_cnt,
const char *license, __u32 kern_version,
char *log_buf, size_t log_buf_sz);
+LIBBPF_DEPRECATED_SINCE(0, 7, "use bpf_prog_load() instead")
LIBBPF_API int bpf_verify_program(enum bpf_prog_type type,
const struct bpf_insn *insns,
size_t insns_cnt, __u32 prog_flags,
@@ -116,6 +216,23 @@ LIBBPF_API int bpf_verify_program(enum bpf_prog_type type,
char *log_buf, size_t log_buf_sz,
int log_level);
+struct bpf_btf_load_opts {
+ size_t sz; /* size of this struct for forward/backward compatibility */
+
+ /* kernel log options */
+ char *log_buf;
+ __u32 log_level;
+ __u32 log_size;
+};
+#define bpf_btf_load_opts__last_field log_size
+
+LIBBPF_API int bpf_btf_load(const void *btf_data, size_t btf_size,
+ const struct bpf_btf_load_opts *opts);
+
+LIBBPF_DEPRECATED_SINCE(0, 8, "use bpf_btf_load() instead")
+LIBBPF_API int bpf_load_btf(const void *btf, __u32 btf_size, char *log_buf,
+ __u32 log_buf_size, bool do_log);
+
LIBBPF_API int bpf_map_update_elem(int fd, const void *key, const void *value,
__u64 flags);
@@ -137,17 +254,128 @@ struct bpf_map_batch_opts {
};
#define bpf_map_batch_opts__last_field flags
-LIBBPF_API int bpf_map_delete_batch(int fd, void *keys,
+
+/**
+ * @brief **bpf_map_delete_batch()** allows for batch deletion of multiple
+ * elements in a BPF map.
+ *
+ * @param fd BPF map file descriptor
+ * @param keys pointer to an array of *count* keys
+ * @param count input and output parameter; on input **count** represents the
+ * number of elements in the map to delete in batch;
+ * on output if a non-EFAULT error is returned, **count** represents the number of deleted
+ * elements if the output **count** value is not equal to the input **count** value
+ * If EFAULT is returned, **count** should not be trusted to be correct.
+ * @param opts options for configuring the way the batch deletion works
+ * @return 0, on success; negative error code, otherwise (errno is also set to
+ * the error code)
+ */
+LIBBPF_API int bpf_map_delete_batch(int fd, const void *keys,
__u32 *count,
const struct bpf_map_batch_opts *opts);
+
+/**
+ * @brief **bpf_map_lookup_batch()** allows for batch lookup of BPF map elements.
+ *
+ * The parameter *in_batch* is the address of the first element in the batch to read.
+ * *out_batch* is an output parameter that should be passed as *in_batch* to subsequent
+ * calls to **bpf_map_lookup_batch()**. NULL can be passed for *in_batch* to indicate
+ * that the batched lookup starts from the beginning of the map.
+ *
+ * The *keys* and *values* are output parameters which must point to memory large enough to
+ * hold *count* items based on the key and value size of the map *map_fd*. The *keys*
+ * buffer must be of *key_size* * *count*. The *values* buffer must be of
+ * *value_size* * *count*.
+ *
+ * @param fd BPF map file descriptor
+ * @param in_batch address of the first element in batch to read, can pass NULL to
+ * indicate that the batched lookup starts from the beginning of the map.
+ * @param out_batch output parameter that should be passed to next call as *in_batch*
+ * @param keys pointer to an array large enough for *count* keys
+ * @param values pointer to an array large enough for *count* values
+ * @param count input and output parameter; on input it's the number of elements
+ * in the map to read in batch; on output it's the number of elements that were
+ * successfully read.
+ * If a non-EFAULT error is returned, count will be set as the number of elements
+ * that were read before the error occurred.
+ * If EFAULT is returned, **count** should not be trusted to be correct.
+ * @param opts options for configuring the way the batch lookup works
+ * @return 0, on success; negative error code, otherwise (errno is also set to
+ * the error code)
+ */
LIBBPF_API int bpf_map_lookup_batch(int fd, void *in_batch, void *out_batch,
void *keys, void *values, __u32 *count,
const struct bpf_map_batch_opts *opts);
+
+/**
+ * @brief **bpf_map_lookup_and_delete_batch()** allows for batch lookup and deletion
+ * of BPF map elements where each element is deleted after being retrieved.
+ *
+ * @param fd BPF map file descriptor
+ * @param in_batch address of the first element in batch to read, can pass NULL to
+ * get address of the first element in *out_batch*
+ * @param out_batch output parameter that should be passed to next call as *in_batch*
+ * @param keys pointer to an array of *count* keys
+ * @param values pointer to an array large enough for *count* values
+ * @param count input and output parameter; on input it's the number of elements
+ * in the map to read and delete in batch; on output it represents the number of
+ * elements that were successfully read and deleted
+ * If a non-**EFAULT** error code is returned and if the output **count** value
+ * is not equal to the input **count** value, up to **count** elements may
+ * have been deleted.
+ * if **EFAULT** is returned up to *count* elements may have been deleted without
+ * being returned via the *keys* and *values* output parameters.
+ * @param opts options for configuring the way the batch lookup and delete works
+ * @return 0, on success; negative error code, otherwise (errno is also set to
+ * the error code)
+ */
LIBBPF_API int bpf_map_lookup_and_delete_batch(int fd, void *in_batch,
void *out_batch, void *keys,
void *values, __u32 *count,
const struct bpf_map_batch_opts *opts);
-LIBBPF_API int bpf_map_update_batch(int fd, void *keys, void *values,
+
+/**
+ * @brief **bpf_map_update_batch()** updates multiple elements in a map
+ * by specifying keys and their corresponding values.
+ *
+ * The *keys* and *values* parameters must point to memory large enough
+ * to hold *count* items based on the key and value size of the map.
+ *
+ * The *opts* parameter can be used to control how *bpf_map_update_batch()*
+ * should handle keys that either do or do not already exist in the map.
+ * In particular the *flags* parameter of *bpf_map_batch_opts* can be
+ * one of the following:
+ *
+ * Note that *count* is an input and output parameter, where on output it
+ * represents how many elements were successfully updated. Also note that if
+ * **EFAULT** then *count* should not be trusted to be correct.
+ *
+ * **BPF_ANY**
+ * Create new elements or update existing.
+ *
+ * **BPF_NOEXIST**
+ * Create new elements only if they do not exist.
+ *
+ * **BPF_EXIST**
+ * Update existing elements.
+ *
+ * **BPF_F_LOCK**
+ * Update spin_lock-ed map elements. This must be
+ * specified if the map value contains a spinlock.
+ *
+ * @param fd BPF map file descriptor
+ * @param keys pointer to an array of *count* keys
+ * @param values pointer to an array of *count* values
+ * @param count input and output parameter; on input it's the number of elements
+ * in the map to update in batch; on output if a non-EFAULT error is returned,
+ * **count** represents the number of updated elements if the output **count**
+ * value is not equal to the input **count** value.
+ * If EFAULT is returned, **count** should not be trusted to be correct.
+ * @param opts options for configuring the way the batch update works
+ * @return 0, on success; negative error code, otherwise (errno is also set to
+ * the error code)
+ */
+LIBBPF_API int bpf_map_update_batch(int fd, const void *keys, const void *values,
__u32 *count,
const struct bpf_map_batch_opts *opts);
@@ -243,8 +471,6 @@ LIBBPF_API int bpf_prog_query(int target_fd, enum bpf_attach_type type,
__u32 query_flags, __u32 *attach_flags,
__u32 *prog_ids, __u32 *prog_cnt);
LIBBPF_API int bpf_raw_tracepoint_open(const char *name, int prog_fd);
-LIBBPF_API int bpf_load_btf(const void *btf, __u32 btf_size, char *log_buf,
- __u32 log_buf_size, bool do_log);
LIBBPF_API int bpf_task_fd_query(int pid, int fd, __u32 flags, char *buf,
__u32 *buf_len, __u32 *prog_id, __u32 *fd_type,
__u64 *probe_offset, __u64 *probe_addr);
diff --git a/tools/lib/bpf/bpf_gen_internal.h b/tools/lib/bpf/bpf_gen_internal.h
index 6f3df00..2233089 100644
--- a/tools/lib/bpf/bpf_gen_internal.h
+++ b/tools/lib/bpf/bpf_gen_internal.h
@@ -3,6 +3,8 @@
#ifndef __BPF_GEN_INTERNAL_H
#define __BPF_GEN_INTERNAL_H
+#include "bpf.h"
+
struct ksym_relo_desc {
const char *name;
int kind;
@@ -37,6 +39,8 @@ struct bpf_gen {
int error;
struct ksym_relo_desc *relos;
int relo_cnt;
+ struct bpf_core_relo *core_relos;
+ int core_relo_cnt;
char attach_target[128];
int attach_kind;
struct ksym_desc *ksyms;
@@ -49,13 +53,20 @@ void bpf_gen__init(struct bpf_gen *gen, int log_level, int nr_progs, int nr_maps
int bpf_gen__finish(struct bpf_gen *gen, int nr_progs, int nr_maps);
void bpf_gen__free(struct bpf_gen *gen);
void bpf_gen__load_btf(struct bpf_gen *gen, const void *raw_data, __u32 raw_size);
-void bpf_gen__map_create(struct bpf_gen *gen, struct bpf_create_map_params *map_attr, int map_idx);
-struct bpf_prog_load_params;
-void bpf_gen__prog_load(struct bpf_gen *gen, struct bpf_prog_load_params *load_attr, int prog_idx);
+void bpf_gen__map_create(struct bpf_gen *gen,
+ enum bpf_map_type map_type, const char *map_name,
+ __u32 key_size, __u32 value_size, __u32 max_entries,
+ struct bpf_map_create_opts *map_attr, int map_idx);
+void bpf_gen__prog_load(struct bpf_gen *gen,
+ enum bpf_prog_type prog_type, const char *prog_name,
+ const char *license, struct bpf_insn *insns, size_t insn_cnt,
+ struct bpf_prog_load_opts *load_attr, int prog_idx);
void bpf_gen__map_update_elem(struct bpf_gen *gen, int map_idx, void *value, __u32 value_size);
void bpf_gen__map_freeze(struct bpf_gen *gen, int map_idx);
void bpf_gen__record_attach_target(struct bpf_gen *gen, const char *name, enum bpf_attach_type type);
void bpf_gen__record_extern(struct bpf_gen *gen, const char *name, bool is_weak,
bool is_typeless, int kind, int insn_idx);
+void bpf_gen__record_relo_core(struct bpf_gen *gen, const struct bpf_core_relo *core_relo);
+void bpf_gen__populate_outer_map(struct bpf_gen *gen, int outer_map_idx, int key, int inner_map_idx);
#endif
diff --git a/tools/lib/bpf/bpf_tracing.h b/tools/lib/bpf/bpf_tracing.h
index db05a59..90f56b0 100644
--- a/tools/lib/bpf/bpf_tracing.h
+++ b/tools/lib/bpf/bpf_tracing.h
@@ -66,277 +66,204 @@
#if defined(__KERNEL__) || defined(__VMLINUX_H__)
-#define PT_REGS_PARM1(x) ((x)->di)
-#define PT_REGS_PARM2(x) ((x)->si)
-#define PT_REGS_PARM3(x) ((x)->dx)
-#define PT_REGS_PARM4(x) ((x)->cx)
-#define PT_REGS_PARM5(x) ((x)->r8)
-#define PT_REGS_RET(x) ((x)->sp)
-#define PT_REGS_FP(x) ((x)->bp)
-#define PT_REGS_RC(x) ((x)->ax)
-#define PT_REGS_SP(x) ((x)->sp)
-#define PT_REGS_IP(x) ((x)->ip)
-
-#define PT_REGS_PARM1_CORE(x) BPF_CORE_READ((x), di)
-#define PT_REGS_PARM2_CORE(x) BPF_CORE_READ((x), si)
-#define PT_REGS_PARM3_CORE(x) BPF_CORE_READ((x), dx)
-#define PT_REGS_PARM4_CORE(x) BPF_CORE_READ((x), cx)
-#define PT_REGS_PARM5_CORE(x) BPF_CORE_READ((x), r8)
-#define PT_REGS_RET_CORE(x) BPF_CORE_READ((x), sp)
-#define PT_REGS_FP_CORE(x) BPF_CORE_READ((x), bp)
-#define PT_REGS_RC_CORE(x) BPF_CORE_READ((x), ax)
-#define PT_REGS_SP_CORE(x) BPF_CORE_READ((x), sp)
-#define PT_REGS_IP_CORE(x) BPF_CORE_READ((x), ip)
+#define __PT_PARM1_REG di
+#define __PT_PARM2_REG si
+#define __PT_PARM3_REG dx
+#define __PT_PARM4_REG cx
+#define __PT_PARM5_REG r8
+#define __PT_RET_REG sp
+#define __PT_FP_REG bp
+#define __PT_RC_REG ax
+#define __PT_SP_REG sp
+#define __PT_IP_REG ip
#else
#ifdef __i386__
+
+#define __PT_PARM1_REG eax
+#define __PT_PARM2_REG edx
+#define __PT_PARM3_REG ecx
/* i386 kernel is built with -mregparm=3 */
-#define PT_REGS_PARM1(x) ((x)->eax)
-#define PT_REGS_PARM2(x) ((x)->edx)
-#define PT_REGS_PARM3(x) ((x)->ecx)
-#define PT_REGS_PARM4(x) 0
-#define PT_REGS_PARM5(x) 0
-#define PT_REGS_RET(x) ((x)->esp)
-#define PT_REGS_FP(x) ((x)->ebp)
-#define PT_REGS_RC(x) ((x)->eax)
-#define PT_REGS_SP(x) ((x)->esp)
-#define PT_REGS_IP(x) ((x)->eip)
+#define __PT_PARM4_REG __unsupported__
+#define __PT_PARM5_REG __unsupported__
+#define __PT_RET_REG esp
+#define __PT_FP_REG ebp
+#define __PT_RC_REG eax
+#define __PT_SP_REG esp
+#define __PT_IP_REG eip
-#define PT_REGS_PARM1_CORE(x) BPF_CORE_READ((x), eax)
-#define PT_REGS_PARM2_CORE(x) BPF_CORE_READ((x), edx)
-#define PT_REGS_PARM3_CORE(x) BPF_CORE_READ((x), ecx)
-#define PT_REGS_PARM4_CORE(x) 0
-#define PT_REGS_PARM5_CORE(x) 0
-#define PT_REGS_RET_CORE(x) BPF_CORE_READ((x), esp)
-#define PT_REGS_FP_CORE(x) BPF_CORE_READ((x), ebp)
-#define PT_REGS_RC_CORE(x) BPF_CORE_READ((x), eax)
-#define PT_REGS_SP_CORE(x) BPF_CORE_READ((x), esp)
-#define PT_REGS_IP_CORE(x) BPF_CORE_READ((x), eip)
+#else /* __i386__ */
-#else
+#define __PT_PARM1_REG rdi
+#define __PT_PARM2_REG rsi
+#define __PT_PARM3_REG rdx
+#define __PT_PARM4_REG rcx
+#define __PT_PARM5_REG r8
+#define __PT_RET_REG rsp
+#define __PT_FP_REG rbp
+#define __PT_RC_REG rax
+#define __PT_SP_REG rsp
+#define __PT_IP_REG rip
-#define PT_REGS_PARM1(x) ((x)->rdi)
-#define PT_REGS_PARM2(x) ((x)->rsi)
-#define PT_REGS_PARM3(x) ((x)->rdx)
-#define PT_REGS_PARM4(x) ((x)->rcx)
-#define PT_REGS_PARM5(x) ((x)->r8)
-#define PT_REGS_RET(x) ((x)->rsp)
-#define PT_REGS_FP(x) ((x)->rbp)
-#define PT_REGS_RC(x) ((x)->rax)
-#define PT_REGS_SP(x) ((x)->rsp)
-#define PT_REGS_IP(x) ((x)->rip)
+#endif /* __i386__ */
-#define PT_REGS_PARM1_CORE(x) BPF_CORE_READ((x), rdi)
-#define PT_REGS_PARM2_CORE(x) BPF_CORE_READ((x), rsi)
-#define PT_REGS_PARM3_CORE(x) BPF_CORE_READ((x), rdx)
-#define PT_REGS_PARM4_CORE(x) BPF_CORE_READ((x), rcx)
-#define PT_REGS_PARM5_CORE(x) BPF_CORE_READ((x), r8)
-#define PT_REGS_RET_CORE(x) BPF_CORE_READ((x), rsp)
-#define PT_REGS_FP_CORE(x) BPF_CORE_READ((x), rbp)
-#define PT_REGS_RC_CORE(x) BPF_CORE_READ((x), rax)
-#define PT_REGS_SP_CORE(x) BPF_CORE_READ((x), rsp)
-#define PT_REGS_IP_CORE(x) BPF_CORE_READ((x), rip)
-
-#endif
-#endif
+#endif /* __KERNEL__ || __VMLINUX_H__ */
#elif defined(bpf_target_s390)
/* s390 provides user_pt_regs instead of struct pt_regs to userspace */
-struct pt_regs;
-#define PT_REGS_S390 const volatile user_pt_regs
-#define PT_REGS_PARM1(x) (((PT_REGS_S390 *)(x))->gprs[2])
-#define PT_REGS_PARM2(x) (((PT_REGS_S390 *)(x))->gprs[3])
-#define PT_REGS_PARM3(x) (((PT_REGS_S390 *)(x))->gprs[4])
-#define PT_REGS_PARM4(x) (((PT_REGS_S390 *)(x))->gprs[5])
-#define PT_REGS_PARM5(x) (((PT_REGS_S390 *)(x))->gprs[6])
-#define PT_REGS_RET(x) (((PT_REGS_S390 *)(x))->gprs[14])
-/* Works only with CONFIG_FRAME_POINTER */
-#define PT_REGS_FP(x) (((PT_REGS_S390 *)(x))->gprs[11])
-#define PT_REGS_RC(x) (((PT_REGS_S390 *)(x))->gprs[2])
-#define PT_REGS_SP(x) (((PT_REGS_S390 *)(x))->gprs[15])
-#define PT_REGS_IP(x) (((PT_REGS_S390 *)(x))->psw.addr)
-
-#define PT_REGS_PARM1_CORE(x) BPF_CORE_READ((PT_REGS_S390 *)(x), gprs[2])
-#define PT_REGS_PARM2_CORE(x) BPF_CORE_READ((PT_REGS_S390 *)(x), gprs[3])
-#define PT_REGS_PARM3_CORE(x) BPF_CORE_READ((PT_REGS_S390 *)(x), gprs[4])
-#define PT_REGS_PARM4_CORE(x) BPF_CORE_READ((PT_REGS_S390 *)(x), gprs[5])
-#define PT_REGS_PARM5_CORE(x) BPF_CORE_READ((PT_REGS_S390 *)(x), gprs[6])
-#define PT_REGS_RET_CORE(x) BPF_CORE_READ((PT_REGS_S390 *)(x), gprs[14])
-#define PT_REGS_FP_CORE(x) BPF_CORE_READ((PT_REGS_S390 *)(x), gprs[11])
-#define PT_REGS_RC_CORE(x) BPF_CORE_READ((PT_REGS_S390 *)(x), gprs[2])
-#define PT_REGS_SP_CORE(x) BPF_CORE_READ((PT_REGS_S390 *)(x), gprs[15])
-#define PT_REGS_IP_CORE(x) BPF_CORE_READ((PT_REGS_S390 *)(x), psw.addr)
+#define __PT_REGS_CAST(x) ((const user_pt_regs *)(x))
+#define __PT_PARM1_REG gprs[2]
+#define __PT_PARM2_REG gprs[3]
+#define __PT_PARM3_REG gprs[4]
+#define __PT_PARM4_REG gprs[5]
+#define __PT_PARM5_REG gprs[6]
+#define __PT_RET_REG grps[14]
+#define __PT_FP_REG gprs[11] /* Works only with CONFIG_FRAME_POINTER */
+#define __PT_RC_REG gprs[2]
+#define __PT_SP_REG gprs[15]
+#define __PT_IP_REG psw.addr
#elif defined(bpf_target_arm)
-#define PT_REGS_PARM1(x) ((x)->uregs[0])
-#define PT_REGS_PARM2(x) ((x)->uregs[1])
-#define PT_REGS_PARM3(x) ((x)->uregs[2])
-#define PT_REGS_PARM4(x) ((x)->uregs[3])
-#define PT_REGS_PARM5(x) ((x)->uregs[4])
-#define PT_REGS_RET(x) ((x)->uregs[14])
-#define PT_REGS_FP(x) ((x)->uregs[11]) /* Works only with CONFIG_FRAME_POINTER */
-#define PT_REGS_RC(x) ((x)->uregs[0])
-#define PT_REGS_SP(x) ((x)->uregs[13])
-#define PT_REGS_IP(x) ((x)->uregs[12])
-
-#define PT_REGS_PARM1_CORE(x) BPF_CORE_READ((x), uregs[0])
-#define PT_REGS_PARM2_CORE(x) BPF_CORE_READ((x), uregs[1])
-#define PT_REGS_PARM3_CORE(x) BPF_CORE_READ((x), uregs[2])
-#define PT_REGS_PARM4_CORE(x) BPF_CORE_READ((x), uregs[3])
-#define PT_REGS_PARM5_CORE(x) BPF_CORE_READ((x), uregs[4])
-#define PT_REGS_RET_CORE(x) BPF_CORE_READ((x), uregs[14])
-#define PT_REGS_FP_CORE(x) BPF_CORE_READ((x), uregs[11])
-#define PT_REGS_RC_CORE(x) BPF_CORE_READ((x), uregs[0])
-#define PT_REGS_SP_CORE(x) BPF_CORE_READ((x), uregs[13])
-#define PT_REGS_IP_CORE(x) BPF_CORE_READ((x), uregs[12])
+#define __PT_PARM1_REG uregs[0]
+#define __PT_PARM2_REG uregs[1]
+#define __PT_PARM3_REG uregs[2]
+#define __PT_PARM4_REG uregs[3]
+#define __PT_PARM5_REG uregs[4]
+#define __PT_RET_REG uregs[14]
+#define __PT_FP_REG uregs[11] /* Works only with CONFIG_FRAME_POINTER */
+#define __PT_RC_REG uregs[0]
+#define __PT_SP_REG uregs[13]
+#define __PT_IP_REG uregs[12]
#elif defined(bpf_target_arm64)
/* arm64 provides struct user_pt_regs instead of struct pt_regs to userspace */
-struct pt_regs;
-#define PT_REGS_ARM64 const volatile struct user_pt_regs
-#define PT_REGS_PARM1(x) (((PT_REGS_ARM64 *)(x))->regs[0])
-#define PT_REGS_PARM2(x) (((PT_REGS_ARM64 *)(x))->regs[1])
-#define PT_REGS_PARM3(x) (((PT_REGS_ARM64 *)(x))->regs[2])
-#define PT_REGS_PARM4(x) (((PT_REGS_ARM64 *)(x))->regs[3])
-#define PT_REGS_PARM5(x) (((PT_REGS_ARM64 *)(x))->regs[4])
-#define PT_REGS_RET(x) (((PT_REGS_ARM64 *)(x))->regs[30])
-/* Works only with CONFIG_FRAME_POINTER */
-#define PT_REGS_FP(x) (((PT_REGS_ARM64 *)(x))->regs[29])
-#define PT_REGS_RC(x) (((PT_REGS_ARM64 *)(x))->regs[0])
-#define PT_REGS_SP(x) (((PT_REGS_ARM64 *)(x))->sp)
-#define PT_REGS_IP(x) (((PT_REGS_ARM64 *)(x))->pc)
-
-#define PT_REGS_PARM1_CORE(x) BPF_CORE_READ((PT_REGS_ARM64 *)(x), regs[0])
-#define PT_REGS_PARM2_CORE(x) BPF_CORE_READ((PT_REGS_ARM64 *)(x), regs[1])
-#define PT_REGS_PARM3_CORE(x) BPF_CORE_READ((PT_REGS_ARM64 *)(x), regs[2])
-#define PT_REGS_PARM4_CORE(x) BPF_CORE_READ((PT_REGS_ARM64 *)(x), regs[3])
-#define PT_REGS_PARM5_CORE(x) BPF_CORE_READ((PT_REGS_ARM64 *)(x), regs[4])
-#define PT_REGS_RET_CORE(x) BPF_CORE_READ((PT_REGS_ARM64 *)(x), regs[30])
-#define PT_REGS_FP_CORE(x) BPF_CORE_READ((PT_REGS_ARM64 *)(x), regs[29])
-#define PT_REGS_RC_CORE(x) BPF_CORE_READ((PT_REGS_ARM64 *)(x), regs[0])
-#define PT_REGS_SP_CORE(x) BPF_CORE_READ((PT_REGS_ARM64 *)(x), sp)
-#define PT_REGS_IP_CORE(x) BPF_CORE_READ((PT_REGS_ARM64 *)(x), pc)
+#define __PT_REGS_CAST(x) ((const struct user_pt_regs *)(x))
+#define __PT_PARM1_REG regs[0]
+#define __PT_PARM2_REG regs[1]
+#define __PT_PARM3_REG regs[2]
+#define __PT_PARM4_REG regs[3]
+#define __PT_PARM5_REG regs[4]
+#define __PT_RET_REG regs[30]
+#define __PT_FP_REG regs[29] /* Works only with CONFIG_FRAME_POINTER */
+#define __PT_RC_REG regs[0]
+#define __PT_SP_REG sp
+#define __PT_IP_REG pc
#elif defined(bpf_target_mips)
-#define PT_REGS_PARM1(x) ((x)->regs[4])
-#define PT_REGS_PARM2(x) ((x)->regs[5])
-#define PT_REGS_PARM3(x) ((x)->regs[6])
-#define PT_REGS_PARM4(x) ((x)->regs[7])
-#define PT_REGS_PARM5(x) ((x)->regs[8])
-#define PT_REGS_RET(x) ((x)->regs[31])
-#define PT_REGS_FP(x) ((x)->regs[30]) /* Works only with CONFIG_FRAME_POINTER */
-#define PT_REGS_RC(x) ((x)->regs[2])
-#define PT_REGS_SP(x) ((x)->regs[29])
-#define PT_REGS_IP(x) ((x)->cp0_epc)
-
-#define PT_REGS_PARM1_CORE(x) BPF_CORE_READ((x), regs[4])
-#define PT_REGS_PARM2_CORE(x) BPF_CORE_READ((x), regs[5])
-#define PT_REGS_PARM3_CORE(x) BPF_CORE_READ((x), regs[6])
-#define PT_REGS_PARM4_CORE(x) BPF_CORE_READ((x), regs[7])
-#define PT_REGS_PARM5_CORE(x) BPF_CORE_READ((x), regs[8])
-#define PT_REGS_RET_CORE(x) BPF_CORE_READ((x), regs[31])
-#define PT_REGS_FP_CORE(x) BPF_CORE_READ((x), regs[30])
-#define PT_REGS_RC_CORE(x) BPF_CORE_READ((x), regs[2])
-#define PT_REGS_SP_CORE(x) BPF_CORE_READ((x), regs[29])
-#define PT_REGS_IP_CORE(x) BPF_CORE_READ((x), cp0_epc)
+#define __PT_PARM1_REG regs[4]
+#define __PT_PARM2_REG regs[5]
+#define __PT_PARM3_REG regs[6]
+#define __PT_PARM4_REG regs[7]
+#define __PT_PARM5_REG regs[8]
+#define __PT_RET_REG regs[31]
+#define __PT_FP_REG regs[30] /* Works only with CONFIG_FRAME_POINTER */
+#define __PT_RC_REG regs[2]
+#define __PT_SP_REG regs[29]
+#define __PT_IP_REG cp0_epc
#elif defined(bpf_target_powerpc)
-#define PT_REGS_PARM1(x) ((x)->gpr[3])
-#define PT_REGS_PARM2(x) ((x)->gpr[4])
-#define PT_REGS_PARM3(x) ((x)->gpr[5])
-#define PT_REGS_PARM4(x) ((x)->gpr[6])
-#define PT_REGS_PARM5(x) ((x)->gpr[7])
-#define PT_REGS_RC(x) ((x)->gpr[3])
-#define PT_REGS_SP(x) ((x)->sp)
-#define PT_REGS_IP(x) ((x)->nip)
-
-#define PT_REGS_PARM1_CORE(x) BPF_CORE_READ((x), gpr[3])
-#define PT_REGS_PARM2_CORE(x) BPF_CORE_READ((x), gpr[4])
-#define PT_REGS_PARM3_CORE(x) BPF_CORE_READ((x), gpr[5])
-#define PT_REGS_PARM4_CORE(x) BPF_CORE_READ((x), gpr[6])
-#define PT_REGS_PARM5_CORE(x) BPF_CORE_READ((x), gpr[7])
-#define PT_REGS_RC_CORE(x) BPF_CORE_READ((x), gpr[3])
-#define PT_REGS_SP_CORE(x) BPF_CORE_READ((x), sp)
-#define PT_REGS_IP_CORE(x) BPF_CORE_READ((x), nip)
+#define __PT_PARM1_REG gpr[3]
+#define __PT_PARM2_REG gpr[4]
+#define __PT_PARM3_REG gpr[5]
+#define __PT_PARM4_REG gpr[6]
+#define __PT_PARM5_REG gpr[7]
+#define __PT_RET_REG regs[31]
+#define __PT_FP_REG __unsupported__
+#define __PT_RC_REG gpr[3]
+#define __PT_SP_REG sp
+#define __PT_IP_REG nip
#elif defined(bpf_target_sparc)
-#define PT_REGS_PARM1(x) ((x)->u_regs[UREG_I0])
-#define PT_REGS_PARM2(x) ((x)->u_regs[UREG_I1])
-#define PT_REGS_PARM3(x) ((x)->u_regs[UREG_I2])
-#define PT_REGS_PARM4(x) ((x)->u_regs[UREG_I3])
-#define PT_REGS_PARM5(x) ((x)->u_regs[UREG_I4])
-#define PT_REGS_RET(x) ((x)->u_regs[UREG_I7])
-#define PT_REGS_RC(x) ((x)->u_regs[UREG_I0])
-#define PT_REGS_SP(x) ((x)->u_regs[UREG_FP])
-
-#define PT_REGS_PARM1_CORE(x) BPF_CORE_READ((x), u_regs[UREG_I0])
-#define PT_REGS_PARM2_CORE(x) BPF_CORE_READ((x), u_regs[UREG_I1])
-#define PT_REGS_PARM3_CORE(x) BPF_CORE_READ((x), u_regs[UREG_I2])
-#define PT_REGS_PARM4_CORE(x) BPF_CORE_READ((x), u_regs[UREG_I3])
-#define PT_REGS_PARM5_CORE(x) BPF_CORE_READ((x), u_regs[UREG_I4])
-#define PT_REGS_RET_CORE(x) BPF_CORE_READ((x), u_regs[UREG_I7])
-#define PT_REGS_RC_CORE(x) BPF_CORE_READ((x), u_regs[UREG_I0])
-#define PT_REGS_SP_CORE(x) BPF_CORE_READ((x), u_regs[UREG_FP])
-
+#define __PT_PARM1_REG u_regs[UREG_I0]
+#define __PT_PARM2_REG u_regs[UREG_I1]
+#define __PT_PARM3_REG u_regs[UREG_I2]
+#define __PT_PARM4_REG u_regs[UREG_I3]
+#define __PT_PARM5_REG u_regs[UREG_I4]
+#define __PT_RET_REG u_regs[UREG_I7]
+#define __PT_FP_REG __unsupported__
+#define __PT_RC_REG u_regs[UREG_I0]
+#define __PT_SP_REG u_regs[UREG_FP]
/* Should this also be a bpf_target check for the sparc case? */
#if defined(__arch64__)
-#define PT_REGS_IP(x) ((x)->tpc)
-#define PT_REGS_IP_CORE(x) BPF_CORE_READ((x), tpc)
+#define __PT_IP_REG tpc
#else
-#define PT_REGS_IP(x) ((x)->pc)
-#define PT_REGS_IP_CORE(x) BPF_CORE_READ((x), pc)
+#define __PT_IP_REG pc
#endif
#elif defined(bpf_target_riscv)
-struct pt_regs;
-#define PT_REGS_RV const volatile struct user_regs_struct
-#define PT_REGS_PARM1(x) (((PT_REGS_RV *)(x))->a0)
-#define PT_REGS_PARM2(x) (((PT_REGS_RV *)(x))->a1)
-#define PT_REGS_PARM3(x) (((PT_REGS_RV *)(x))->a2)
-#define PT_REGS_PARM4(x) (((PT_REGS_RV *)(x))->a3)
-#define PT_REGS_PARM5(x) (((PT_REGS_RV *)(x))->a4)
-#define PT_REGS_RET(x) (((PT_REGS_RV *)(x))->ra)
-#define PT_REGS_FP(x) (((PT_REGS_RV *)(x))->s5)
-#define PT_REGS_RC(x) (((PT_REGS_RV *)(x))->a5)
-#define PT_REGS_SP(x) (((PT_REGS_RV *)(x))->sp)
-#define PT_REGS_IP(x) (((PT_REGS_RV *)(x))->epc)
-
-#define PT_REGS_PARM1_CORE(x) BPF_CORE_READ((PT_REGS_RV *)(x), a0)
-#define PT_REGS_PARM2_CORE(x) BPF_CORE_READ((PT_REGS_RV *)(x), a1)
-#define PT_REGS_PARM3_CORE(x) BPF_CORE_READ((PT_REGS_RV *)(x), a2)
-#define PT_REGS_PARM4_CORE(x) BPF_CORE_READ((PT_REGS_RV *)(x), a3)
-#define PT_REGS_PARM5_CORE(x) BPF_CORE_READ((PT_REGS_RV *)(x), a4)
-#define PT_REGS_RET_CORE(x) BPF_CORE_READ((PT_REGS_RV *)(x), ra)
-#define PT_REGS_FP_CORE(x) BPF_CORE_READ((PT_REGS_RV *)(x), fp)
-#define PT_REGS_RC_CORE(x) BPF_CORE_READ((PT_REGS_RV *)(x), a5)
-#define PT_REGS_SP_CORE(x) BPF_CORE_READ((PT_REGS_RV *)(x), sp)
-#define PT_REGS_IP_CORE(x) BPF_CORE_READ((PT_REGS_RV *)(x), epc)
+#define __PT_REGS_CAST(x) ((const struct user_regs_struct *)(x))
+#define __PT_PARM1_REG a0
+#define __PT_PARM2_REG a1
+#define __PT_PARM3_REG a2
+#define __PT_PARM4_REG a3
+#define __PT_PARM5_REG a4
+#define __PT_RET_REG ra
+#define __PT_FP_REG fp
+#define __PT_RC_REG a5
+#define __PT_SP_REG sp
+#define __PT_IP_REG epc
#endif
+#if defined(bpf_target_defined)
+
+struct pt_regs;
+
+/* allow some architecutres to override `struct pt_regs` */
+#ifndef __PT_REGS_CAST
+#define __PT_REGS_CAST(x) (x)
+#endif
+
+#define PT_REGS_PARM1(x) (__PT_REGS_CAST(x)->__PT_PARM1_REG)
+#define PT_REGS_PARM2(x) (__PT_REGS_CAST(x)->__PT_PARM2_REG)
+#define PT_REGS_PARM3(x) (__PT_REGS_CAST(x)->__PT_PARM3_REG)
+#define PT_REGS_PARM4(x) (__PT_REGS_CAST(x)->__PT_PARM4_REG)
+#define PT_REGS_PARM5(x) (__PT_REGS_CAST(x)->__PT_PARM5_REG)
+#define PT_REGS_RET(x) (__PT_REGS_CAST(x)->__PT_RET_REG)
+#define PT_REGS_FP(x) (__PT_REGS_CAST(x)->__PT_FP_REG)
+#define PT_REGS_RC(x) (__PT_REGS_CAST(x)->__PT_RC_REG)
+#define PT_REGS_SP(x) (__PT_REGS_CAST(x)->__PT_SP_REG)
+#define PT_REGS_IP(x) (__PT_REGS_CAST(x)->__PT_IP_REG)
+
+#define PT_REGS_PARM1_CORE(x) BPF_CORE_READ(__PT_REGS_CAST(x), __PT_PARM1_REG)
+#define PT_REGS_PARM2_CORE(x) BPF_CORE_READ(__PT_REGS_CAST(x), __PT_PARM2_REG)
+#define PT_REGS_PARM3_CORE(x) BPF_CORE_READ(__PT_REGS_CAST(x), __PT_PARM3_REG)
+#define PT_REGS_PARM4_CORE(x) BPF_CORE_READ(__PT_REGS_CAST(x), __PT_PARM4_REG)
+#define PT_REGS_PARM5_CORE(x) BPF_CORE_READ(__PT_REGS_CAST(x), __PT_PARM5_REG)
+#define PT_REGS_RET_CORE(x) BPF_CORE_READ(__PT_REGS_CAST(x), __PT_RET_REG)
+#define PT_REGS_FP_CORE(x) BPF_CORE_READ(__PT_REGS_CAST(x), __PT_FP_REG)
+#define PT_REGS_RC_CORE(x) BPF_CORE_READ(__PT_REGS_CAST(x), __PT_RC_REG)
+#define PT_REGS_SP_CORE(x) BPF_CORE_READ(__PT_REGS_CAST(x), __PT_SP_REG)
+#define PT_REGS_IP_CORE(x) BPF_CORE_READ(__PT_REGS_CAST(x), __PT_IP_REG)
+
#if defined(bpf_target_powerpc)
+
#define BPF_KPROBE_READ_RET_IP(ip, ctx) ({ (ip) = (ctx)->link; })
#define BPF_KRETPROBE_READ_RET_IP BPF_KPROBE_READ_RET_IP
+
#elif defined(bpf_target_sparc)
+
#define BPF_KPROBE_READ_RET_IP(ip, ctx) ({ (ip) = PT_REGS_RET(ctx); })
#define BPF_KRETPROBE_READ_RET_IP BPF_KPROBE_READ_RET_IP
-#elif defined(bpf_target_defined)
+
+#else
+
#define BPF_KPROBE_READ_RET_IP(ip, ctx) \
({ bpf_probe_read_kernel(&(ip), sizeof(ip), (void *)PT_REGS_RET(ctx)); })
#define BPF_KRETPROBE_READ_RET_IP(ip, ctx) \
- ({ bpf_probe_read_kernel(&(ip), sizeof(ip), \
- (void *)(PT_REGS_FP(ctx) + sizeof(ip))); })
+ ({ bpf_probe_read_kernel(&(ip), sizeof(ip), (void *)(PT_REGS_FP(ctx) + sizeof(ip))); })
+
#endif
-#if !defined(bpf_target_defined)
+#else /* defined(bpf_target_defined) */
#define PT_REGS_PARM1(x) ({ _Pragma(__BPF_TARGET_MISSING); 0l; })
#define PT_REGS_PARM2(x) ({ _Pragma(__BPF_TARGET_MISSING); 0l; })
@@ -363,7 +290,7 @@ struct pt_regs;
#define BPF_KPROBE_READ_RET_IP(ip, ctx) ({ _Pragma(__BPF_TARGET_MISSING); 0l; })
#define BPF_KRETPROBE_READ_RET_IP(ip, ctx) ({ _Pragma(__BPF_TARGET_MISSING); 0l; })
-#endif /* !defined(bpf_target_defined) */
+#endif /* defined(bpf_target_defined) */
#ifndef ___bpf_concat
#define ___bpf_concat(a, b) a ## b
@@ -375,25 +302,23 @@ struct pt_regs;
#define ___bpf_nth(_, _1, _2, _3, _4, _5, _6, _7, _8, _9, _a, _b, _c, N, ...) N
#endif
#ifndef ___bpf_narg
-#define ___bpf_narg(...) \
- ___bpf_nth(_, ##__VA_ARGS__, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0)
+#define ___bpf_narg(...) ___bpf_nth(_, ##__VA_ARGS__, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0)
#endif
-#define ___bpf_ctx_cast0() ctx
-#define ___bpf_ctx_cast1(x) ___bpf_ctx_cast0(), (void *)ctx[0]
-#define ___bpf_ctx_cast2(x, args...) ___bpf_ctx_cast1(args), (void *)ctx[1]
-#define ___bpf_ctx_cast3(x, args...) ___bpf_ctx_cast2(args), (void *)ctx[2]
-#define ___bpf_ctx_cast4(x, args...) ___bpf_ctx_cast3(args), (void *)ctx[3]
-#define ___bpf_ctx_cast5(x, args...) ___bpf_ctx_cast4(args), (void *)ctx[4]
-#define ___bpf_ctx_cast6(x, args...) ___bpf_ctx_cast5(args), (void *)ctx[5]
-#define ___bpf_ctx_cast7(x, args...) ___bpf_ctx_cast6(args), (void *)ctx[6]
-#define ___bpf_ctx_cast8(x, args...) ___bpf_ctx_cast7(args), (void *)ctx[7]
-#define ___bpf_ctx_cast9(x, args...) ___bpf_ctx_cast8(args), (void *)ctx[8]
+#define ___bpf_ctx_cast0() ctx
+#define ___bpf_ctx_cast1(x) ___bpf_ctx_cast0(), (void *)ctx[0]
+#define ___bpf_ctx_cast2(x, args...) ___bpf_ctx_cast1(args), (void *)ctx[1]
+#define ___bpf_ctx_cast3(x, args...) ___bpf_ctx_cast2(args), (void *)ctx[2]
+#define ___bpf_ctx_cast4(x, args...) ___bpf_ctx_cast3(args), (void *)ctx[3]
+#define ___bpf_ctx_cast5(x, args...) ___bpf_ctx_cast4(args), (void *)ctx[4]
+#define ___bpf_ctx_cast6(x, args...) ___bpf_ctx_cast5(args), (void *)ctx[5]
+#define ___bpf_ctx_cast7(x, args...) ___bpf_ctx_cast6(args), (void *)ctx[6]
+#define ___bpf_ctx_cast8(x, args...) ___bpf_ctx_cast7(args), (void *)ctx[7]
+#define ___bpf_ctx_cast9(x, args...) ___bpf_ctx_cast8(args), (void *)ctx[8]
#define ___bpf_ctx_cast10(x, args...) ___bpf_ctx_cast9(args), (void *)ctx[9]
#define ___bpf_ctx_cast11(x, args...) ___bpf_ctx_cast10(args), (void *)ctx[10]
#define ___bpf_ctx_cast12(x, args...) ___bpf_ctx_cast11(args), (void *)ctx[11]
-#define ___bpf_ctx_cast(args...) \
- ___bpf_apply(___bpf_ctx_cast, ___bpf_narg(args))(args)
+#define ___bpf_ctx_cast(args...) ___bpf_apply(___bpf_ctx_cast, ___bpf_narg(args))(args)
/*
* BPF_PROG is a convenience wrapper for generic tp_btf/fentry/fexit and
@@ -426,19 +351,13 @@ ____##name(unsigned long long *ctx, ##args)
struct pt_regs;
-#define ___bpf_kprobe_args0() ctx
-#define ___bpf_kprobe_args1(x) \
- ___bpf_kprobe_args0(), (void *)PT_REGS_PARM1(ctx)
-#define ___bpf_kprobe_args2(x, args...) \
- ___bpf_kprobe_args1(args), (void *)PT_REGS_PARM2(ctx)
-#define ___bpf_kprobe_args3(x, args...) \
- ___bpf_kprobe_args2(args), (void *)PT_REGS_PARM3(ctx)
-#define ___bpf_kprobe_args4(x, args...) \
- ___bpf_kprobe_args3(args), (void *)PT_REGS_PARM4(ctx)
-#define ___bpf_kprobe_args5(x, args...) \
- ___bpf_kprobe_args4(args), (void *)PT_REGS_PARM5(ctx)
-#define ___bpf_kprobe_args(args...) \
- ___bpf_apply(___bpf_kprobe_args, ___bpf_narg(args))(args)
+#define ___bpf_kprobe_args0() ctx
+#define ___bpf_kprobe_args1(x) ___bpf_kprobe_args0(), (void *)PT_REGS_PARM1(ctx)
+#define ___bpf_kprobe_args2(x, args...) ___bpf_kprobe_args1(args), (void *)PT_REGS_PARM2(ctx)
+#define ___bpf_kprobe_args3(x, args...) ___bpf_kprobe_args2(args), (void *)PT_REGS_PARM3(ctx)
+#define ___bpf_kprobe_args4(x, args...) ___bpf_kprobe_args3(args), (void *)PT_REGS_PARM4(ctx)
+#define ___bpf_kprobe_args5(x, args...) ___bpf_kprobe_args4(args), (void *)PT_REGS_PARM5(ctx)
+#define ___bpf_kprobe_args(args...) ___bpf_apply(___bpf_kprobe_args, ___bpf_narg(args))(args)
/*
* BPF_KPROBE serves the same purpose for kprobes as BPF_PROG for
@@ -464,11 +383,9 @@ typeof(name(0)) name(struct pt_regs *ctx) \
static __attribute__((always_inline)) typeof(name(0)) \
____##name(struct pt_regs *ctx, ##args)
-#define ___bpf_kretprobe_args0() ctx
-#define ___bpf_kretprobe_args1(x) \
- ___bpf_kretprobe_args0(), (void *)PT_REGS_RC(ctx)
-#define ___bpf_kretprobe_args(args...) \
- ___bpf_apply(___bpf_kretprobe_args, ___bpf_narg(args))(args)
+#define ___bpf_kretprobe_args0() ctx
+#define ___bpf_kretprobe_args1(x) ___bpf_kretprobe_args0(), (void *)PT_REGS_RC(ctx)
+#define ___bpf_kretprobe_args(args...) ___bpf_apply(___bpf_kretprobe_args, ___bpf_narg(args))(args)
/*
* BPF_KRETPROBE is similar to BPF_KPROBE, except, it only provides optional
diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
index 7e4c558..9aa19c8 100644
--- a/tools/lib/bpf/btf.c
+++ b/tools/lib/bpf/btf.c
@@ -299,6 +299,7 @@ static int btf_type_size(const struct btf_type *t)
case BTF_KIND_TYPEDEF:
case BTF_KIND_FUNC:
case BTF_KIND_FLOAT:
+ case BTF_KIND_TYPE_TAG:
return base_size;
case BTF_KIND_INT:
return base_size + sizeof(__u32);
@@ -349,6 +350,7 @@ static int btf_bswap_type_rest(struct btf_type *t)
case BTF_KIND_TYPEDEF:
case BTF_KIND_FUNC:
case BTF_KIND_FLOAT:
+ case BTF_KIND_TYPE_TAG:
return 0;
case BTF_KIND_INT:
*(__u32 *)(t + 1) = bswap_32(*(__u32 *)(t + 1));
@@ -452,7 +454,7 @@ const struct btf *btf__base_btf(const struct btf *btf)
}
/* internal helper returning non-const pointer to a type */
-struct btf_type *btf_type_by_id(struct btf *btf, __u32 type_id)
+struct btf_type *btf_type_by_id(const struct btf *btf, __u32 type_id)
{
if (type_id == 0)
return &btf_void;
@@ -608,6 +610,7 @@ __s64 btf__resolve_size(const struct btf *btf, __u32 type_id)
case BTF_KIND_RESTRICT:
case BTF_KIND_VAR:
case BTF_KIND_DECL_TAG:
+ case BTF_KIND_TYPE_TAG:
type_id = t->type;
break;
case BTF_KIND_ARRAY:
@@ -649,6 +652,7 @@ int btf__align_of(const struct btf *btf, __u32 id)
case BTF_KIND_VOLATILE:
case BTF_KIND_CONST:
case BTF_KIND_RESTRICT:
+ case BTF_KIND_TYPE_TAG:
return btf__align_of(btf, t->type);
case BTF_KIND_ARRAY:
return btf__align_of(btf, btf_array(t)->type);
@@ -1120,54 +1124,86 @@ struct btf *btf__parse_split(const char *path, struct btf *base_btf)
static void *btf_get_raw_data(const struct btf *btf, __u32 *size, bool swap_endian);
-int btf__load_into_kernel(struct btf *btf)
+int btf_load_into_kernel(struct btf *btf, char *log_buf, size_t log_sz, __u32 log_level)
{
- __u32 log_buf_size = 0, raw_size;
- char *log_buf = NULL;
+ LIBBPF_OPTS(bpf_btf_load_opts, opts);
+ __u32 buf_sz = 0, raw_size;
+ char *buf = NULL, *tmp;
void *raw_data;
int err = 0;
if (btf->fd >= 0)
return libbpf_err(-EEXIST);
+ if (log_sz && !log_buf)
+ return libbpf_err(-EINVAL);
-retry_load:
- if (log_buf_size) {
- log_buf = malloc(log_buf_size);
- if (!log_buf)
- return libbpf_err(-ENOMEM);
-
- *log_buf = 0;
- }
-
+ /* cache native raw data representation */
raw_data = btf_get_raw_data(btf, &raw_size, false);
if (!raw_data) {
err = -ENOMEM;
goto done;
}
- /* cache native raw data representation */
btf->raw_size = raw_size;
btf->raw_data = raw_data;
- btf->fd = bpf_load_btf(raw_data, raw_size, log_buf, log_buf_size, false);
- if (btf->fd < 0) {
- if (!log_buf || errno == ENOSPC) {
- log_buf_size = max((__u32)BPF_LOG_BUF_SIZE,
- log_buf_size << 1);
- free(log_buf);
- goto retry_load;
+retry_load:
+ /* if log_level is 0, we won't provide log_buf/log_size to the kernel,
+ * initially. Only if BTF loading fails, we bump log_level to 1 and
+ * retry, using either auto-allocated or custom log_buf. This way
+ * non-NULL custom log_buf provides a buffer just in case, but hopes
+ * for successful load and no need for log_buf.
+ */
+ if (log_level) {
+ /* if caller didn't provide custom log_buf, we'll keep
+ * allocating our own progressively bigger buffers for BTF
+ * verification log
+ */
+ if (!log_buf) {
+ buf_sz = max((__u32)BPF_LOG_BUF_SIZE, buf_sz * 2);
+ tmp = realloc(buf, buf_sz);
+ if (!tmp) {
+ err = -ENOMEM;
+ goto done;
+ }
+ buf = tmp;
+ buf[0] = '\0';
}
+ opts.log_buf = log_buf ? log_buf : buf;
+ opts.log_size = log_buf ? log_sz : buf_sz;
+ opts.log_level = log_level;
+ }
+
+ btf->fd = bpf_btf_load(raw_data, raw_size, &opts);
+ if (btf->fd < 0) {
+ /* time to turn on verbose mode and try again */
+ if (log_level == 0) {
+ log_level = 1;
+ goto retry_load;
+ }
+ /* only retry if caller didn't provide custom log_buf, but
+ * make sure we can never overflow buf_sz
+ */
+ if (!log_buf && errno == ENOSPC && buf_sz <= UINT_MAX / 2)
+ goto retry_load;
+
err = -errno;
- pr_warn("Error loading BTF: %s(%d)\n", strerror(errno), errno);
- if (*log_buf)
- pr_warn("%s\n", log_buf);
- goto done;
+ pr_warn("BTF loading error: %d\n", err);
+ /* don't print out contents of custom log_buf */
+ if (!log_buf && buf[0])
+ pr_warn("-- BEGIN BTF LOAD LOG ---\n%s\n-- END BTF LOAD LOG --\n", buf);
}
done:
- free(log_buf);
+ free(buf);
return libbpf_err(err);
}
+
+int btf__load_into_kernel(struct btf *btf)
+{
+ return btf_load_into_kernel(btf, NULL, 0, 0);
+}
+
int btf__load(struct btf *) __attribute__((alias("btf__load_into_kernel")));
int btf__fd(const struct btf *btf)
@@ -2236,6 +2272,22 @@ int btf__add_restrict(struct btf *btf, int ref_type_id)
}
/*
+ * Append new BTF_KIND_TYPE_TAG type with:
+ * - *value*, non-empty/non-NULL tag value;
+ * - *ref_type_id* - referenced type ID, it might not exist yet;
+ * Returns:
+ * - >0, type ID of newly added BTF type;
+ * - <0, on error.
+ */
+int btf__add_type_tag(struct btf *btf, const char *value, int ref_type_id)
+{
+ if (!value|| !value[0])
+ return libbpf_err(-EINVAL);
+
+ return btf_add_ref_kind(btf, BTF_KIND_TYPE_TAG, value, ref_type_id);
+}
+
+/*
* Append new BTF_KIND_FUNC type with:
* - *name*, non-empty/non-NULL name;
* - *proto_type_id* - FUNC_PROTO's type ID, it might not exist yet;
@@ -2711,15 +2763,11 @@ void btf_ext__free(struct btf_ext *btf_ext)
free(btf_ext);
}
-struct btf_ext *btf_ext__new(__u8 *data, __u32 size)
+struct btf_ext *btf_ext__new(const __u8 *data, __u32 size)
{
struct btf_ext *btf_ext;
int err;
- err = btf_ext_parse_hdr(data, size);
- if (err)
- return libbpf_err_ptr(err);
-
btf_ext = calloc(1, sizeof(struct btf_ext));
if (!btf_ext)
return libbpf_err_ptr(-ENOMEM);
@@ -2732,6 +2780,10 @@ struct btf_ext *btf_ext__new(__u8 *data, __u32 size)
}
memcpy(btf_ext->data, data, size);
+ err = btf_ext_parse_hdr(btf_ext->data, size);
+ if (err)
+ goto done;
+
if (btf_ext->hdr->hdr_len < offsetofend(struct btf_ext_header, line_info_len)) {
err = -EINVAL;
goto done;
@@ -2846,8 +2898,7 @@ __u32 btf_ext__line_info_rec_size(const struct btf_ext *btf_ext)
struct btf_dedup;
-static struct btf_dedup *btf_dedup_new(struct btf *btf, struct btf_ext *btf_ext,
- const struct btf_dedup_opts *opts);
+static struct btf_dedup *btf_dedup_new(struct btf *btf, const struct btf_dedup_opts *opts);
static void btf_dedup_free(struct btf_dedup *d);
static int btf_dedup_prep(struct btf_dedup *d);
static int btf_dedup_strings(struct btf_dedup *d);
@@ -2994,12 +3045,17 @@ static int btf_dedup_remap_types(struct btf_dedup *d);
* deduplicating structs/unions is described in greater details in comments for
* `btf_dedup_is_equiv` function.
*/
-int btf__dedup(struct btf *btf, struct btf_ext *btf_ext,
- const struct btf_dedup_opts *opts)
+
+DEFAULT_VERSION(btf__dedup_v0_6_0, btf__dedup, LIBBPF_0.6.0)
+int btf__dedup_v0_6_0(struct btf *btf, const struct btf_dedup_opts *opts)
{
- struct btf_dedup *d = btf_dedup_new(btf, btf_ext, opts);
+ struct btf_dedup *d;
int err;
+ if (!OPTS_VALID(opts, btf_dedup_opts))
+ return libbpf_err(-EINVAL);
+
+ d = btf_dedup_new(btf, opts);
if (IS_ERR(d)) {
pr_debug("btf_dedup_new failed: %ld", PTR_ERR(d));
return libbpf_err(-EINVAL);
@@ -3051,6 +3107,19 @@ int btf__dedup(struct btf *btf, struct btf_ext *btf_ext,
return libbpf_err(err);
}
+COMPAT_VERSION(btf__dedup_deprecated, btf__dedup, LIBBPF_0.0.2)
+int btf__dedup_deprecated(struct btf *btf, struct btf_ext *btf_ext, const void *unused_opts)
+{
+ LIBBPF_OPTS(btf_dedup_opts, opts, .btf_ext = btf_ext);
+
+ if (unused_opts) {
+ pr_warn("please use new version of btf__dedup() that supports options\n");
+ return libbpf_err(-ENOTSUP);
+ }
+
+ return btf__dedup(btf, &opts);
+}
+
#define BTF_UNPROCESSED_ID ((__u32)-1)
#define BTF_IN_PROGRESS_ID ((__u32)-2)
@@ -3163,8 +3232,7 @@ static bool btf_dedup_equal_fn(const void *k1, const void *k2, void *ctx)
return k1 == k2;
}
-static struct btf_dedup *btf_dedup_new(struct btf *btf, struct btf_ext *btf_ext,
- const struct btf_dedup_opts *opts)
+static struct btf_dedup *btf_dedup_new(struct btf *btf, const struct btf_dedup_opts *opts)
{
struct btf_dedup *d = calloc(1, sizeof(struct btf_dedup));
hashmap_hash_fn hash_fn = btf_dedup_identity_hash_fn;
@@ -3173,13 +3241,11 @@ static struct btf_dedup *btf_dedup_new(struct btf *btf, struct btf_ext *btf_ext,
if (!d)
return ERR_PTR(-ENOMEM);
- d->opts.dont_resolve_fwds = opts && opts->dont_resolve_fwds;
- /* dedup_table_size is now used only to force collisions in tests */
- if (opts && opts->dedup_table_size == 1)
+ if (OPTS_GET(opts, force_collisions, false))
hash_fn = btf_dedup_collision_hash_fn;
d->btf = btf;
- d->btf_ext = btf_ext;
+ d->btf_ext = OPTS_GET(opts, btf_ext, NULL);
d->dedup_table = hashmap__new(hash_fn, btf_dedup_equal_fn, NULL);
if (IS_ERR(d->dedup_table)) {
@@ -3443,8 +3509,8 @@ static long btf_hash_struct(struct btf_type *t)
}
/*
- * Check structural compatibility of two FUNC_PROTOs, ignoring referenced type
- * IDs. This check is performed during type graph equivalence check and
+ * Check structural compatibility of two STRUCTs/UNIONs, ignoring referenced
+ * type IDs. This check is performed during type graph equivalence check and
* referenced types equivalence is checked separately.
*/
static bool btf_shallow_equal_struct(struct btf_type *t1, struct btf_type *t2)
@@ -3625,6 +3691,7 @@ static int btf_dedup_prep(struct btf_dedup *d)
case BTF_KIND_TYPEDEF:
case BTF_KIND_FUNC:
case BTF_KIND_FLOAT:
+ case BTF_KIND_TYPE_TAG:
h = btf_hash_common(t);
break;
case BTF_KIND_INT:
@@ -3685,6 +3752,7 @@ static int btf_dedup_prim_type(struct btf_dedup *d, __u32 type_id)
case BTF_KIND_VAR:
case BTF_KIND_DATASEC:
case BTF_KIND_DECL_TAG:
+ case BTF_KIND_TYPE_TAG:
return 0;
case BTF_KIND_INT:
@@ -3708,8 +3776,6 @@ static int btf_dedup_prim_type(struct btf_dedup *d, __u32 type_id)
new_id = cand_id;
break;
}
- if (d->opts.dont_resolve_fwds)
- continue;
if (btf_compat_enum(t, cand)) {
if (btf_is_enum_fwd(t)) {
/* resolve fwd to full enum */
@@ -3817,6 +3883,31 @@ static int btf_dedup_identical_arrays(struct btf_dedup *d, __u32 id1, __u32 id2)
return btf_equal_array(t1, t2);
}
+/* Check if given two types are identical STRUCT/UNION definitions */
+static bool btf_dedup_identical_structs(struct btf_dedup *d, __u32 id1, __u32 id2)
+{
+ const struct btf_member *m1, *m2;
+ struct btf_type *t1, *t2;
+ int n, i;
+
+ t1 = btf_type_by_id(d->btf, id1);
+ t2 = btf_type_by_id(d->btf, id2);
+
+ if (!btf_is_composite(t1) || btf_kind(t1) != btf_kind(t2))
+ return false;
+
+ if (!btf_shallow_equal_struct(t1, t2))
+ return false;
+
+ m1 = btf_members(t1);
+ m2 = btf_members(t2);
+ for (i = 0, n = btf_vlen(t1); i < n; i++, m1++, m2++) {
+ if (m1->type != m2->type)
+ return false;
+ }
+ return true;
+}
+
/*
* Check equivalence of BTF type graph formed by candidate struct/union (we'll
* call it "candidate graph" in this description for brevity) to a type graph
@@ -3928,6 +4019,8 @@ static int btf_dedup_is_equiv(struct btf_dedup *d, __u32 cand_id,
hypot_type_id = d->hypot_map[canon_id];
if (hypot_type_id <= BTF_MAX_NR_TYPES) {
+ if (hypot_type_id == cand_id)
+ return 1;
/* In some cases compiler will generate different DWARF types
* for *identical* array type definitions and use them for
* different fields within the *same* struct. This breaks type
@@ -3936,8 +4029,18 @@ static int btf_dedup_is_equiv(struct btf_dedup *d, __u32 cand_id,
* types within a single CU. So work around that by explicitly
* allowing identical array types here.
*/
- return hypot_type_id == cand_id ||
- btf_dedup_identical_arrays(d, hypot_type_id, cand_id);
+ if (btf_dedup_identical_arrays(d, hypot_type_id, cand_id))
+ return 1;
+ /* It turns out that similar situation can happen with
+ * struct/union sometimes, sigh... Handle the case where
+ * structs/unions are exactly the same, down to the referenced
+ * type IDs. Anything more complicated (e.g., if referenced
+ * types are different, but equivalent) is *way more*
+ * complicated and requires a many-to-many equivalence mapping.
+ */
+ if (btf_dedup_identical_structs(d, hypot_type_id, cand_id))
+ return 1;
+ return 0;
}
if (btf_dedup_hypot_map_add(d, canon_id, cand_id))
@@ -3952,8 +4055,7 @@ static int btf_dedup_is_equiv(struct btf_dedup *d, __u32 cand_id,
return 0;
/* FWD <--> STRUCT/UNION equivalence check, if enabled */
- if (!d->opts.dont_resolve_fwds
- && (cand_kind == BTF_KIND_FWD || canon_kind == BTF_KIND_FWD)
+ if ((cand_kind == BTF_KIND_FWD || canon_kind == BTF_KIND_FWD)
&& cand_kind != canon_kind) {
__u16 real_kind;
__u16 fwd_kind;
@@ -3979,10 +4081,7 @@ static int btf_dedup_is_equiv(struct btf_dedup *d, __u32 cand_id,
return btf_equal_int_tag(cand_type, canon_type);
case BTF_KIND_ENUM:
- if (d->opts.dont_resolve_fwds)
- return btf_equal_enum(cand_type, canon_type);
- else
- return btf_compat_enum(cand_type, canon_type);
+ return btf_compat_enum(cand_type, canon_type);
case BTF_KIND_FWD:
case BTF_KIND_FLOAT:
@@ -3994,6 +4093,7 @@ static int btf_dedup_is_equiv(struct btf_dedup *d, __u32 cand_id,
case BTF_KIND_PTR:
case BTF_KIND_TYPEDEF:
case BTF_KIND_FUNC:
+ case BTF_KIND_TYPE_TAG:
if (cand_type->info != canon_type->info)
return 0;
return btf_dedup_is_equiv(d, cand_type->type, canon_type->type);
@@ -4289,6 +4389,7 @@ static int btf_dedup_ref_type(struct btf_dedup *d, __u32 type_id)
case BTF_KIND_PTR:
case BTF_KIND_TYPEDEF:
case BTF_KIND_FUNC:
+ case BTF_KIND_TYPE_TAG:
ref_type_id = btf_dedup_ref_type(d, t->type);
if (ref_type_id < 0)
return ref_type_id;
@@ -4595,6 +4696,7 @@ int btf_type_visit_type_ids(struct btf_type *t, type_id_visit_fn visit, void *ct
case BTF_KIND_FUNC:
case BTF_KIND_VAR:
case BTF_KIND_DECL_TAG:
+ case BTF_KIND_TYPE_TAG:
return visit(&t->type, ctx);
case BTF_KIND_ARRAY: {
diff --git a/tools/lib/bpf/btf.h b/tools/lib/bpf/btf.h
index bc005ba..061839f 100644
--- a/tools/lib/bpf/btf.h
+++ b/tools/lib/bpf/btf.h
@@ -157,7 +157,7 @@ LIBBPF_API int btf__get_map_kv_tids(const struct btf *btf, const char *map_name,
__u32 expected_value_size,
__u32 *key_type_id, __u32 *value_type_id);
-LIBBPF_API struct btf_ext *btf_ext__new(__u8 *data, __u32 size);
+LIBBPF_API struct btf_ext *btf_ext__new(const __u8 *data, __u32 size);
LIBBPF_API void btf_ext__free(struct btf_ext *btf_ext);
LIBBPF_API const void *btf_ext__get_raw_data(const struct btf_ext *btf_ext,
__u32 *size);
@@ -227,6 +227,7 @@ LIBBPF_API int btf__add_typedef(struct btf *btf, const char *name, int ref_type_
LIBBPF_API int btf__add_volatile(struct btf *btf, int ref_type_id);
LIBBPF_API int btf__add_const(struct btf *btf, int ref_type_id);
LIBBPF_API int btf__add_restrict(struct btf *btf, int ref_type_id);
+LIBBPF_API int btf__add_type_tag(struct btf *btf, const char *value, int ref_type_id);
/* func and func_proto construction APIs */
LIBBPF_API int btf__add_func(struct btf *btf, const char *name,
@@ -245,25 +246,86 @@ LIBBPF_API int btf__add_decl_tag(struct btf *btf, const char *value, int ref_typ
int component_idx);
struct btf_dedup_opts {
- unsigned int dedup_table_size;
- bool dont_resolve_fwds;
+ size_t sz;
+ /* optional .BTF.ext info to dedup along the main BTF info */
+ struct btf_ext *btf_ext;
+ /* force hash collisions (used for testing) */
+ bool force_collisions;
+ size_t :0;
};
+#define btf_dedup_opts__last_field force_collisions
-LIBBPF_API int btf__dedup(struct btf *btf, struct btf_ext *btf_ext,
- const struct btf_dedup_opts *opts);
+LIBBPF_API int btf__dedup(struct btf *btf, const struct btf_dedup_opts *opts);
+
+LIBBPF_API int btf__dedup_v0_6_0(struct btf *btf, const struct btf_dedup_opts *opts);
+
+LIBBPF_DEPRECATED_SINCE(0, 7, "use btf__dedup() instead")
+LIBBPF_API int btf__dedup_deprecated(struct btf *btf, struct btf_ext *btf_ext, const void *opts);
+#define btf__dedup(...) ___libbpf_overload(___btf_dedup, __VA_ARGS__)
+#define ___btf_dedup3(btf, btf_ext, opts) btf__dedup_deprecated(btf, btf_ext, opts)
+#define ___btf_dedup2(btf, opts) btf__dedup(btf, opts)
struct btf_dump;
struct btf_dump_opts {
- void *ctx;
+ union {
+ size_t sz;
+ void *ctx; /* DEPRECATED: will be gone in v1.0 */
+ };
};
typedef void (*btf_dump_printf_fn_t)(void *ctx, const char *fmt, va_list args);
LIBBPF_API struct btf_dump *btf_dump__new(const struct btf *btf,
- const struct btf_ext *btf_ext,
- const struct btf_dump_opts *opts,
- btf_dump_printf_fn_t printf_fn);
+ btf_dump_printf_fn_t printf_fn,
+ void *ctx,
+ const struct btf_dump_opts *opts);
+
+LIBBPF_API struct btf_dump *btf_dump__new_v0_6_0(const struct btf *btf,
+ btf_dump_printf_fn_t printf_fn,
+ void *ctx,
+ const struct btf_dump_opts *opts);
+
+LIBBPF_API struct btf_dump *btf_dump__new_deprecated(const struct btf *btf,
+ const struct btf_ext *btf_ext,
+ const struct btf_dump_opts *opts,
+ btf_dump_printf_fn_t printf_fn);
+
+/* Choose either btf_dump__new() or btf_dump__new_deprecated() based on the
+ * type of 4th argument. If it's btf_dump's print callback, use deprecated
+ * API; otherwise, choose the new btf_dump__new(). ___libbpf_override()
+ * doesn't work here because both variants have 4 input arguments.
+ *
+ * (void *) casts are necessary to avoid compilation warnings about type
+ * mismatches, because even though __builtin_choose_expr() only ever evaluates
+ * one side the other side still has to satisfy type constraints (this is
+ * compiler implementation limitation which might be lifted eventually,
+ * according to the documentation). So passing struct btf_ext in place of
+ * btf_dump_printf_fn_t would be generating compilation warning. Casting to
+ * void * avoids this issue.
+ *
+ * Also, two type compatibility checks for a function and function pointer are
+ * required because passing function reference into btf_dump__new() as
+ * btf_dump__new(..., my_callback, ...) and as btf_dump__new(...,
+ * &my_callback, ...) (not explicit ampersand in the latter case) actually
+ * differs as far as __builtin_types_compatible_p() is concerned. Thus two
+ * checks are combined to detect callback argument.
+ *
+ * The rest works just like in case of ___libbpf_override() usage with symbol
+ * versioning.
+ *
+ * C++ compilers don't support __builtin_types_compatible_p(), so at least
+ * don't screw up compilation for them and let C++ users pick btf_dump__new
+ * vs btf_dump__new_deprecated explicitly.
+ */
+#ifndef __cplusplus
+#define btf_dump__new(a1, a2, a3, a4) __builtin_choose_expr( \
+ __builtin_types_compatible_p(typeof(a4), btf_dump_printf_fn_t) || \
+ __builtin_types_compatible_p(typeof(a4), void(void *, const char *, va_list)), \
+ btf_dump__new_deprecated((void *)a1, (void *)a2, (void *)a3, (void *)a4), \
+ btf_dump__new((void *)a1, (void *)a2, (void *)a3, (void *)a4))
+#endif
+
LIBBPF_API void btf_dump__free(struct btf_dump *d);
LIBBPF_API int btf_dump__dump_type(struct btf_dump *d, __u32 id);
@@ -403,7 +465,8 @@ static inline bool btf_is_mod(const struct btf_type *t)
return kind == BTF_KIND_VOLATILE ||
kind == BTF_KIND_CONST ||
- kind == BTF_KIND_RESTRICT;
+ kind == BTF_KIND_RESTRICT ||
+ kind == BTF_KIND_TYPE_TAG;
}
static inline bool btf_is_func(const struct btf_type *t)
@@ -436,6 +499,11 @@ static inline bool btf_is_decl_tag(const struct btf_type *t)
return btf_kind(t) == BTF_KIND_DECL_TAG;
}
+static inline bool btf_is_type_tag(const struct btf_type *t)
+{
+ return btf_kind(t) == BTF_KIND_TYPE_TAG;
+}
+
static inline __u8 btf_int_encoding(const struct btf_type *t)
{
return BTF_INT_ENCODING(*(__u32 *)(t + 1));
diff --git a/tools/lib/bpf/btf_dump.c b/tools/lib/bpf/btf_dump.c
index 17db62b..b9a3260c 100644
--- a/tools/lib/bpf/btf_dump.c
+++ b/tools/lib/bpf/btf_dump.c
@@ -77,9 +77,8 @@ struct btf_dump_data {
struct btf_dump {
const struct btf *btf;
- const struct btf_ext *btf_ext;
btf_dump_printf_fn_t printf_fn;
- struct btf_dump_opts opts;
+ void *cb_ctx;
int ptr_sz;
bool strip_mods;
bool skip_anon_defs;
@@ -138,29 +137,32 @@ static void btf_dump_printf(const struct btf_dump *d, const char *fmt, ...)
va_list args;
va_start(args, fmt);
- d->printf_fn(d->opts.ctx, fmt, args);
+ d->printf_fn(d->cb_ctx, fmt, args);
va_end(args);
}
static int btf_dump_mark_referenced(struct btf_dump *d);
static int btf_dump_resize(struct btf_dump *d);
-struct btf_dump *btf_dump__new(const struct btf *btf,
- const struct btf_ext *btf_ext,
- const struct btf_dump_opts *opts,
- btf_dump_printf_fn_t printf_fn)
+DEFAULT_VERSION(btf_dump__new_v0_6_0, btf_dump__new, LIBBPF_0.6.0)
+struct btf_dump *btf_dump__new_v0_6_0(const struct btf *btf,
+ btf_dump_printf_fn_t printf_fn,
+ void *ctx,
+ const struct btf_dump_opts *opts)
{
struct btf_dump *d;
int err;
+ if (!printf_fn)
+ return libbpf_err_ptr(-EINVAL);
+
d = calloc(1, sizeof(struct btf_dump));
if (!d)
return libbpf_err_ptr(-ENOMEM);
d->btf = btf;
- d->btf_ext = btf_ext;
d->printf_fn = printf_fn;
- d->opts.ctx = opts ? opts->ctx : NULL;
+ d->cb_ctx = ctx;
d->ptr_sz = btf__pointer_size(btf) ? : sizeof(void *);
d->type_names = hashmap__new(str_hash_fn, str_equal_fn, NULL);
@@ -186,6 +188,17 @@ struct btf_dump *btf_dump__new(const struct btf *btf,
return libbpf_err_ptr(err);
}
+COMPAT_VERSION(btf_dump__new_deprecated, btf_dump__new, LIBBPF_0.0.4)
+struct btf_dump *btf_dump__new_deprecated(const struct btf *btf,
+ const struct btf_ext *btf_ext,
+ const struct btf_dump_opts *opts,
+ btf_dump_printf_fn_t printf_fn)
+{
+ if (!printf_fn)
+ return libbpf_err_ptr(-EINVAL);
+ return btf_dump__new_v0_6_0(btf, printf_fn, opts ? opts->ctx : NULL, opts);
+}
+
static int btf_dump_resize(struct btf_dump *d)
{
int err, last_id = btf__type_cnt(d->btf) - 1;
@@ -317,6 +330,7 @@ static int btf_dump_mark_referenced(struct btf_dump *d)
case BTF_KIND_FUNC:
case BTF_KIND_VAR:
case BTF_KIND_DECL_TAG:
+ case BTF_KIND_TYPE_TAG:
d->type_states[t->type].referenced = 1;
break;
@@ -560,6 +574,7 @@ static int btf_dump_order_type(struct btf_dump *d, __u32 id, bool through_ptr)
case BTF_KIND_VOLATILE:
case BTF_KIND_CONST:
case BTF_KIND_RESTRICT:
+ case BTF_KIND_TYPE_TAG:
return btf_dump_order_type(d, t->type, through_ptr);
case BTF_KIND_FUNC_PROTO: {
@@ -734,6 +749,7 @@ static void btf_dump_emit_type(struct btf_dump *d, __u32 id, __u32 cont_id)
case BTF_KIND_VOLATILE:
case BTF_KIND_CONST:
case BTF_KIND_RESTRICT:
+ case BTF_KIND_TYPE_TAG:
btf_dump_emit_type(d, t->type, cont_id);
break;
case BTF_KIND_ARRAY:
@@ -1154,6 +1170,7 @@ static void btf_dump_emit_type_decl(struct btf_dump *d, __u32 id,
case BTF_KIND_CONST:
case BTF_KIND_RESTRICT:
case BTF_KIND_FUNC_PROTO:
+ case BTF_KIND_TYPE_TAG:
id = t->type;
break;
case BTF_KIND_ARRAY:
@@ -1322,6 +1339,11 @@ static void btf_dump_emit_type_chain(struct btf_dump *d,
case BTF_KIND_RESTRICT:
btf_dump_printf(d, " restrict");
break;
+ case BTF_KIND_TYPE_TAG:
+ btf_dump_emit_mods(d, decls);
+ name = btf_name_of(d, t->name_off);
+ btf_dump_printf(d, " __attribute__((btf_type_tag(\"%s\")))", name);
+ break;
case BTF_KIND_ARRAY: {
const struct btf_array *a = btf_array(t);
const struct btf_type *next_t;
@@ -2194,7 +2216,7 @@ static int btf_dump_dump_type_data(struct btf_dump *d,
__u8 bits_offset,
__u8 bit_sz)
{
- int size, err;
+ int size, err = 0;
size = btf_dump_type_data_check_overflow(d, t, id, data, bits_offset);
if (size < 0)
@@ -2299,8 +2321,8 @@ int btf_dump__dump_type_data(struct btf_dump *d, __u32 id,
if (!opts->indent_str)
d->typed_dump->indent_str[0] = '\t';
else
- strncat(d->typed_dump->indent_str, opts->indent_str,
- sizeof(d->typed_dump->indent_str) - 1);
+ libbpf_strlcpy(d->typed_dump->indent_str, opts->indent_str,
+ sizeof(d->typed_dump->indent_str));
d->typed_dump->compact = OPTS_GET(opts, compact, false);
d->typed_dump->skip_names = OPTS_GET(opts, skip_names, false);
diff --git a/tools/lib/bpf/gen_loader.c b/tools/lib/bpf/gen_loader.c
index 9934851..8ecef108 100644
--- a/tools/lib/bpf/gen_loader.c
+++ b/tools/lib/bpf/gen_loader.c
@@ -371,8 +371,9 @@ int bpf_gen__finish(struct bpf_gen *gen, int nr_progs, int nr_maps)
{
int i;
- if (nr_progs != gen->nr_progs || nr_maps != gen->nr_maps) {
- pr_warn("progs/maps mismatch\n");
+ if (nr_progs < gen->nr_progs || nr_maps != gen->nr_maps) {
+ pr_warn("nr_progs %d/%d nr_maps %d/%d mismatch\n",
+ nr_progs, gen->nr_progs, nr_maps, gen->nr_maps);
gen->error = -EFAULT;
return gen->error;
}
@@ -445,47 +446,32 @@ void bpf_gen__load_btf(struct bpf_gen *gen, const void *btf_raw_data,
}
void bpf_gen__map_create(struct bpf_gen *gen,
- struct bpf_create_map_params *map_attr, int map_idx)
+ enum bpf_map_type map_type,
+ const char *map_name,
+ __u32 key_size, __u32 value_size, __u32 max_entries,
+ struct bpf_map_create_opts *map_attr, int map_idx)
{
- int attr_size = offsetofend(union bpf_attr, btf_vmlinux_value_type_id);
+ int attr_size = offsetofend(union bpf_attr, map_extra);
bool close_inner_map_fd = false;
int map_create_attr, idx;
union bpf_attr attr;
memset(&attr, 0, attr_size);
- attr.map_type = map_attr->map_type;
- attr.key_size = map_attr->key_size;
- attr.value_size = map_attr->value_size;
+ attr.map_type = map_type;
+ attr.key_size = key_size;
+ attr.value_size = value_size;
attr.map_flags = map_attr->map_flags;
attr.map_extra = map_attr->map_extra;
- memcpy(attr.map_name, map_attr->name,
- min((unsigned)strlen(map_attr->name), BPF_OBJ_NAME_LEN - 1));
+ if (map_name)
+ libbpf_strlcpy(attr.map_name, map_name, sizeof(attr.map_name));
attr.numa_node = map_attr->numa_node;
attr.map_ifindex = map_attr->map_ifindex;
- attr.max_entries = map_attr->max_entries;
- switch (attr.map_type) {
- case BPF_MAP_TYPE_PERF_EVENT_ARRAY:
- case BPF_MAP_TYPE_CGROUP_ARRAY:
- case BPF_MAP_TYPE_STACK_TRACE:
- case BPF_MAP_TYPE_ARRAY_OF_MAPS:
- case BPF_MAP_TYPE_HASH_OF_MAPS:
- case BPF_MAP_TYPE_DEVMAP:
- case BPF_MAP_TYPE_DEVMAP_HASH:
- case BPF_MAP_TYPE_CPUMAP:
- case BPF_MAP_TYPE_XSKMAP:
- case BPF_MAP_TYPE_SOCKMAP:
- case BPF_MAP_TYPE_SOCKHASH:
- case BPF_MAP_TYPE_QUEUE:
- case BPF_MAP_TYPE_STACK:
- case BPF_MAP_TYPE_RINGBUF:
- break;
- default:
- attr.btf_key_type_id = map_attr->btf_key_type_id;
- attr.btf_value_type_id = map_attr->btf_value_type_id;
- }
+ attr.max_entries = max_entries;
+ attr.btf_key_type_id = map_attr->btf_key_type_id;
+ attr.btf_value_type_id = map_attr->btf_value_type_id;
pr_debug("gen: map_create: %s idx %d type %d value_type_id %d\n",
- attr.map_name, map_idx, map_attr->map_type, attr.btf_value_type_id);
+ attr.map_name, map_idx, map_type, attr.btf_value_type_id);
map_create_attr = add_data(gen, &attr, attr_size);
if (attr.btf_value_type_id)
@@ -512,7 +498,7 @@ void bpf_gen__map_create(struct bpf_gen *gen,
/* emit MAP_CREATE command */
emit_sys_bpf(gen, BPF_MAP_CREATE, map_create_attr, attr_size);
debug_ret(gen, "map_create %s idx %d type %d value_size %d value_btf_id %d",
- attr.map_name, map_idx, map_attr->map_type, attr.value_size,
+ attr.map_name, map_idx, map_type, value_size,
attr.btf_value_type_id);
emit_check_err(gen);
/* remember map_fd in the stack, if successful */
@@ -597,8 +583,9 @@ void bpf_gen__record_extern(struct bpf_gen *gen, const char *name, bool is_weak,
static struct ksym_desc *get_ksym_desc(struct bpf_gen *gen, struct ksym_relo_desc *relo)
{
struct ksym_desc *kdesc;
+ int i;
- for (int i = 0; i < gen->nr_ksyms; i++) {
+ for (i = 0; i < gen->nr_ksyms; i++) {
if (!strcmp(gen->ksyms[i].name, relo->name)) {
gen->ksyms[i].ref++;
return &gen->ksyms[i];
@@ -700,27 +687,29 @@ static void emit_relo_kfunc_btf(struct bpf_gen *gen, struct ksym_relo_desc *relo
return;
}
kdesc->off = btf_fd_idx;
- /* set a default value for imm */
+ /* jump to success case */
+ emit(gen, BPF_JMP_IMM(BPF_JSGE, BPF_REG_7, 0, 3));
+ /* set value for imm, off as 0 */
emit(gen, BPF_ST_MEM(BPF_W, BPF_REG_8, offsetof(struct bpf_insn, imm), 0));
- /* skip success case store if ret < 0 */
- emit(gen, BPF_JMP_IMM(BPF_JSLT, BPF_REG_7, 0, 1));
+ emit(gen, BPF_ST_MEM(BPF_H, BPF_REG_8, offsetof(struct bpf_insn, off), 0));
+ /* skip success case for ret < 0 */
+ emit(gen, BPF_JMP_IMM(BPF_JA, 0, 0, 10));
/* store btf_id into insn[insn_idx].imm */
emit(gen, BPF_STX_MEM(BPF_W, BPF_REG_8, BPF_REG_7, offsetof(struct bpf_insn, imm)));
+ /* obtain fd in BPF_REG_9 */
+ emit(gen, BPF_MOV64_REG(BPF_REG_9, BPF_REG_7));
+ emit(gen, BPF_ALU64_IMM(BPF_RSH, BPF_REG_9, 32));
+ /* jump to fd_array store if fd denotes module BTF */
+ emit(gen, BPF_JMP_IMM(BPF_JNE, BPF_REG_9, 0, 2));
+ /* set the default value for off */
+ emit(gen, BPF_ST_MEM(BPF_H, BPF_REG_8, offsetof(struct bpf_insn, off), 0));
+ /* skip BTF fd store for vmlinux BTF */
+ emit(gen, BPF_JMP_IMM(BPF_JA, 0, 0, 4));
/* load fd_array slot pointer */
emit2(gen, BPF_LD_IMM64_RAW_FULL(BPF_REG_0, BPF_PSEUDO_MAP_IDX_VALUE,
0, 0, 0, blob_fd_array_off(gen, btf_fd_idx)));
- /* skip store of BTF fd if ret < 0 */
- emit(gen, BPF_JMP_IMM(BPF_JSLT, BPF_REG_7, 0, 3));
/* store BTF fd in slot */
- emit(gen, BPF_MOV64_REG(BPF_REG_9, BPF_REG_7));
- emit(gen, BPF_ALU64_IMM(BPF_RSH, BPF_REG_9, 32));
emit(gen, BPF_STX_MEM(BPF_W, BPF_REG_0, BPF_REG_9, 0));
- /* set a default value for off */
- emit(gen, BPF_ST_MEM(BPF_H, BPF_REG_8, offsetof(struct bpf_insn, off), 0));
- /* skip insn->off store if ret < 0 */
- emit(gen, BPF_JMP_IMM(BPF_JSLT, BPF_REG_7, 0, 2));
- /* skip if vmlinux BTF */
- emit(gen, BPF_JMP_IMM(BPF_JEQ, BPF_REG_9, 0, 1));
/* store index into insn[insn_idx].off */
emit(gen, BPF_ST_MEM(BPF_H, BPF_REG_8, offsetof(struct bpf_insn, off), btf_fd_idx));
log:
@@ -819,9 +808,8 @@ static void emit_relo_ksym_btf(struct bpf_gen *gen, struct ksym_relo_desc *relo,
kdesc->insn + offsetof(struct bpf_insn, imm));
move_blob2blob(gen, insn + sizeof(struct bpf_insn) + offsetof(struct bpf_insn, imm), 4,
kdesc->insn + sizeof(struct bpf_insn) + offsetof(struct bpf_insn, imm));
- emit(gen, BPF_LDX_MEM(BPF_W, BPF_REG_9, BPF_REG_8, offsetof(struct bpf_insn, imm)));
- /* jump over src_reg adjustment if imm is not 0 */
- emit(gen, BPF_JMP_IMM(BPF_JNE, BPF_REG_9, 0, 3));
+ /* jump over src_reg adjustment if imm is not 0, reuse BPF_REG_0 from move_blob2blob */
+ emit(gen, BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 3));
goto clear_src_reg;
}
/* remember insn offset, so we can copy BTF ID and FD later */
@@ -829,17 +817,20 @@ static void emit_relo_ksym_btf(struct bpf_gen *gen, struct ksym_relo_desc *relo,
emit_bpf_find_by_name_kind(gen, relo);
if (!relo->is_weak)
emit_check_err(gen);
- /* set default values as 0 */
+ /* jump to success case */
+ emit(gen, BPF_JMP_IMM(BPF_JSGE, BPF_REG_7, 0, 3));
+ /* set values for insn[insn_idx].imm, insn[insn_idx + 1].imm as 0 */
emit(gen, BPF_ST_MEM(BPF_W, BPF_REG_8, offsetof(struct bpf_insn, imm), 0));
emit(gen, BPF_ST_MEM(BPF_W, BPF_REG_8, sizeof(struct bpf_insn) + offsetof(struct bpf_insn, imm), 0));
- /* skip success case stores if ret < 0 */
- emit(gen, BPF_JMP_IMM(BPF_JSLT, BPF_REG_7, 0, 4));
+ /* skip success case for ret < 0 */
+ emit(gen, BPF_JMP_IMM(BPF_JA, 0, 0, 4));
/* store btf_id into insn[insn_idx].imm */
emit(gen, BPF_STX_MEM(BPF_W, BPF_REG_8, BPF_REG_7, offsetof(struct bpf_insn, imm)));
/* store btf_obj_fd into insn[insn_idx + 1].imm */
emit(gen, BPF_ALU64_IMM(BPF_RSH, BPF_REG_7, 32));
emit(gen, BPF_STX_MEM(BPF_W, BPF_REG_8, BPF_REG_7,
sizeof(struct bpf_insn) + offsetof(struct bpf_insn, imm)));
+ /* skip src_reg adjustment */
emit(gen, BPF_JMP_IMM(BPF_JSGE, BPF_REG_7, 0, 3));
clear_src_reg:
/* clear bpf_object__relocate_data's src_reg assignment, otherwise we get a verifier failure */
@@ -851,6 +842,22 @@ static void emit_relo_ksym_btf(struct bpf_gen *gen, struct ksym_relo_desc *relo,
emit_ksym_relo_log(gen, relo, kdesc->ref);
}
+void bpf_gen__record_relo_core(struct bpf_gen *gen,
+ const struct bpf_core_relo *core_relo)
+{
+ struct bpf_core_relo *relos;
+
+ relos = libbpf_reallocarray(gen->core_relos, gen->core_relo_cnt + 1, sizeof(*relos));
+ if (!relos) {
+ gen->error = -ENOMEM;
+ return;
+ }
+ gen->core_relos = relos;
+ relos += gen->core_relo_cnt;
+ memcpy(relos, core_relo, sizeof(*relos));
+ gen->core_relo_cnt++;
+}
+
static void emit_relo(struct bpf_gen *gen, struct ksym_relo_desc *relo, int insns)
{
int insn;
@@ -883,6 +890,15 @@ static void emit_relos(struct bpf_gen *gen, int insns)
emit_relo(gen, gen->relos + i, insns);
}
+static void cleanup_core_relo(struct bpf_gen *gen)
+{
+ if (!gen->core_relo_cnt)
+ return;
+ free(gen->core_relos);
+ gen->core_relo_cnt = 0;
+ gen->core_relos = NULL;
+}
+
static void cleanup_relos(struct bpf_gen *gen, int insns)
{
int i, insn;
@@ -910,30 +926,32 @@ static void cleanup_relos(struct bpf_gen *gen, int insns)
gen->relo_cnt = 0;
gen->relos = NULL;
}
+ cleanup_core_relo(gen);
}
void bpf_gen__prog_load(struct bpf_gen *gen,
- struct bpf_prog_load_params *load_attr, int prog_idx)
+ enum bpf_prog_type prog_type, const char *prog_name,
+ const char *license, struct bpf_insn *insns, size_t insn_cnt,
+ struct bpf_prog_load_opts *load_attr, int prog_idx)
{
- int attr_size = offsetofend(union bpf_attr, fd_array);
- int prog_load_attr, license, insns, func_info, line_info;
+ int prog_load_attr, license_off, insns_off, func_info, line_info, core_relos;
+ int attr_size = offsetofend(union bpf_attr, core_relo_rec_size);
union bpf_attr attr;
memset(&attr, 0, attr_size);
- pr_debug("gen: prog_load: type %d insns_cnt %zd\n",
- load_attr->prog_type, load_attr->insn_cnt);
+ pr_debug("gen: prog_load: type %d insns_cnt %zd progi_idx %d\n",
+ prog_type, insn_cnt, prog_idx);
/* add license string to blob of bytes */
- license = add_data(gen, load_attr->license, strlen(load_attr->license) + 1);
+ license_off = add_data(gen, license, strlen(license) + 1);
/* add insns to blob of bytes */
- insns = add_data(gen, load_attr->insns,
- load_attr->insn_cnt * sizeof(struct bpf_insn));
+ insns_off = add_data(gen, insns, insn_cnt * sizeof(struct bpf_insn));
- attr.prog_type = load_attr->prog_type;
+ attr.prog_type = prog_type;
attr.expected_attach_type = load_attr->expected_attach_type;
attr.attach_btf_id = load_attr->attach_btf_id;
attr.prog_ifindex = load_attr->prog_ifindex;
attr.kern_version = 0;
- attr.insn_cnt = (__u32)load_attr->insn_cnt;
+ attr.insn_cnt = (__u32)insn_cnt;
attr.prog_flags = load_attr->prog_flags;
attr.func_info_rec_size = load_attr->func_info_rec_size;
@@ -946,15 +964,19 @@ void bpf_gen__prog_load(struct bpf_gen *gen,
line_info = add_data(gen, load_attr->line_info,
attr.line_info_cnt * attr.line_info_rec_size);
- memcpy(attr.prog_name, load_attr->name,
- min((unsigned)strlen(load_attr->name), BPF_OBJ_NAME_LEN - 1));
+ attr.core_relo_rec_size = sizeof(struct bpf_core_relo);
+ attr.core_relo_cnt = gen->core_relo_cnt;
+ core_relos = add_data(gen, gen->core_relos,
+ attr.core_relo_cnt * attr.core_relo_rec_size);
+
+ libbpf_strlcpy(attr.prog_name, prog_name, sizeof(attr.prog_name));
prog_load_attr = add_data(gen, &attr, attr_size);
/* populate union bpf_attr with a pointer to license */
- emit_rel_store(gen, attr_field(prog_load_attr, license), license);
+ emit_rel_store(gen, attr_field(prog_load_attr, license), license_off);
/* populate union bpf_attr with a pointer to instructions */
- emit_rel_store(gen, attr_field(prog_load_attr, insns), insns);
+ emit_rel_store(gen, attr_field(prog_load_attr, insns), insns_off);
/* populate union bpf_attr with a pointer to func_info */
emit_rel_store(gen, attr_field(prog_load_attr, func_info), func_info);
@@ -962,6 +984,9 @@ void bpf_gen__prog_load(struct bpf_gen *gen,
/* populate union bpf_attr with a pointer to line_info */
emit_rel_store(gen, attr_field(prog_load_attr, line_info), line_info);
+ /* populate union bpf_attr with a pointer to core_relos */
+ emit_rel_store(gen, attr_field(prog_load_attr, core_relos), core_relos);
+
/* populate union bpf_attr fd_array with a pointer to data where map_fds are saved */
emit_rel_store(gen, attr_field(prog_load_attr, fd_array), gen->fd_array);
@@ -986,15 +1011,17 @@ void bpf_gen__prog_load(struct bpf_gen *gen,
emit(gen, BPF_STX_MEM(BPF_W, BPF_REG_0, BPF_REG_7,
offsetof(union bpf_attr, attach_btf_obj_fd)));
}
- emit_relos(gen, insns);
+ emit_relos(gen, insns_off);
/* emit PROG_LOAD command */
emit_sys_bpf(gen, BPF_PROG_LOAD, prog_load_attr, attr_size);
debug_ret(gen, "prog_load %s insn_cnt %d", attr.prog_name, attr.insn_cnt);
/* successful or not, close btf module FDs used in extern ksyms and attach_btf_obj_fd */
- cleanup_relos(gen, insns);
- if (gen->attach_kind)
+ cleanup_relos(gen, insns_off);
+ if (gen->attach_kind) {
emit_sys_close_blob(gen,
attr_field(prog_load_attr, attach_btf_obj_fd));
+ gen->attach_kind = 0;
+ }
emit_check_err(gen);
/* remember prog_fd in the stack, if successful */
emit(gen, BPF_STX_MEM(BPF_W, BPF_REG_10, BPF_REG_7,
@@ -1040,6 +1067,33 @@ void bpf_gen__map_update_elem(struct bpf_gen *gen, int map_idx, void *pvalue,
emit_check_err(gen);
}
+void bpf_gen__populate_outer_map(struct bpf_gen *gen, int outer_map_idx, int slot,
+ int inner_map_idx)
+{
+ int attr_size = offsetofend(union bpf_attr, flags);
+ int map_update_attr, key;
+ union bpf_attr attr;
+
+ memset(&attr, 0, attr_size);
+ pr_debug("gen: populate_outer_map: outer %d key %d inner %d\n",
+ outer_map_idx, slot, inner_map_idx);
+
+ key = add_data(gen, &slot, sizeof(slot));
+
+ map_update_attr = add_data(gen, &attr, attr_size);
+ move_blob2blob(gen, attr_field(map_update_attr, map_fd), 4,
+ blob_fd_array_off(gen, outer_map_idx));
+ emit_rel_store(gen, attr_field(map_update_attr, key), key);
+ emit_rel_store(gen, attr_field(map_update_attr, value),
+ blob_fd_array_off(gen, inner_map_idx));
+
+ /* emit MAP_UPDATE_ELEM command */
+ emit_sys_bpf(gen, BPF_MAP_UPDATE_ELEM, map_update_attr, attr_size);
+ debug_ret(gen, "populate_outer_map outer %d key %d inner %d",
+ outer_map_idx, slot, inner_map_idx);
+ emit_check_err(gen);
+}
+
void bpf_gen__map_freeze(struct bpf_gen *gen, int map_idx)
{
int attr_size = offsetofend(union bpf_attr, map_fd);
diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index 7c74342..7f10dd5 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -168,39 +168,24 @@ int libbpf_set_strict_mode(enum libbpf_strict_mode mode)
return 0;
}
-enum kern_feature_id {
- /* v4.14: kernel support for program & map names. */
- FEAT_PROG_NAME,
- /* v5.2: kernel support for global data sections. */
- FEAT_GLOBAL_DATA,
- /* BTF support */
- FEAT_BTF,
- /* BTF_KIND_FUNC and BTF_KIND_FUNC_PROTO support */
- FEAT_BTF_FUNC,
- /* BTF_KIND_VAR and BTF_KIND_DATASEC support */
- FEAT_BTF_DATASEC,
- /* BTF_FUNC_GLOBAL is supported */
- FEAT_BTF_GLOBAL_FUNC,
- /* BPF_F_MMAPABLE is supported for arrays */
- FEAT_ARRAY_MMAP,
- /* kernel support for expected_attach_type in BPF_PROG_LOAD */
- FEAT_EXP_ATTACH_TYPE,
- /* bpf_probe_read_{kernel,user}[_str] helpers */
- FEAT_PROBE_READ_KERN,
- /* BPF_PROG_BIND_MAP is supported */
- FEAT_PROG_BIND_MAP,
- /* Kernel support for module BTFs */
- FEAT_MODULE_BTF,
- /* BTF_KIND_FLOAT support */
- FEAT_BTF_FLOAT,
- /* BPF perf link support */
- FEAT_PERF_LINK,
- /* BTF_KIND_DECL_TAG support */
- FEAT_BTF_DECL_TAG,
- __FEAT_CNT,
-};
+__u32 libbpf_major_version(void)
+{
+ return LIBBPF_MAJOR_VERSION;
+}
-static bool kernel_supports(const struct bpf_object *obj, enum kern_feature_id feat_id);
+__u32 libbpf_minor_version(void)
+{
+ return LIBBPF_MINOR_VERSION;
+}
+
+const char *libbpf_version_string(void)
+{
+#define __S(X) #X
+#define _S(X) __S(X)
+ return "v" _S(LIBBPF_MAJOR_VERSION) "." _S(LIBBPF_MINOR_VERSION);
+#undef _S
+#undef __S
+}
enum reloc_type {
RELO_LD64,
@@ -209,19 +194,25 @@ enum reloc_type {
RELO_EXTERN_VAR,
RELO_EXTERN_FUNC,
RELO_SUBPROG_ADDR,
+ RELO_CORE,
};
struct reloc_desc {
enum reloc_type type;
int insn_idx;
- int map_idx;
- int sym_off;
+ union {
+ const struct bpf_core_relo *core_relo; /* used when type == RELO_CORE */
+ struct {
+ int map_idx;
+ int sym_off;
+ };
+ };
};
struct bpf_sec_def;
typedef int (*init_fn_t)(struct bpf_program *prog, long cookie);
-typedef int (*preload_fn_t)(struct bpf_program *prog, struct bpf_prog_load_params *attr, long cookie);
+typedef int (*preload_fn_t)(struct bpf_program *prog, struct bpf_prog_load_opts *opts, long cookie);
typedef struct bpf_link *(*attach_fn_t)(const struct bpf_program *prog, long cookie);
/* stored as sec_def->cookie for all libbpf-supported SEC()s */
@@ -304,7 +295,11 @@ struct bpf_program {
struct reloc_desc *reloc_desc;
int nr_reloc;
- int log_level;
+
+ /* BPF verifier log settings */
+ char *log_buf;
+ size_t log_size;
+ __u32 log_level;
struct {
int nr;
@@ -400,6 +395,7 @@ struct bpf_map {
char *pin_path;
bool pinned;
bool reused;
+ bool skipped;
__u64 map_extra;
};
@@ -546,6 +542,11 @@ struct bpf_object {
size_t btf_module_cnt;
size_t btf_module_cap;
+ /* optional log settings passed to BPF_BTF_LOAD and BPF_PROG_LOAD commands */
+ char *log_buf;
+ size_t log_size;
+ __u32 log_level;
+
void *priv;
bpf_object_clear_priv_t clear_priv;
@@ -681,6 +682,9 @@ bpf_object__init_prog(struct bpf_object *obj, struct bpf_program *prog,
prog->instances.fds = NULL;
prog->instances.nr = -1;
+ /* inherit object's log_level */
+ prog->log_level = obj->log_level;
+
prog->sec_name = strdup(sec_name);
if (!prog->sec_name)
goto errout;
@@ -791,11 +795,36 @@ bpf_object__add_programs(struct bpf_object *obj, Elf_Data *sec_data,
return 0;
}
-static __u32 get_kernel_version(void)
+__u32 get_kernel_version(void)
{
+ /* On Ubuntu LINUX_VERSION_CODE doesn't correspond to info.release,
+ * but Ubuntu provides /proc/version_signature file, as described at
+ * https://ubuntu.com/kernel, with an example contents below, which we
+ * can use to get a proper LINUX_VERSION_CODE.
+ *
+ * Ubuntu 5.4.0-12.15-generic 5.4.8
+ *
+ * In the above, 5.4.8 is what kernel is actually expecting, while
+ * uname() call will return 5.4.0 in info.release.
+ */
+ const char *ubuntu_kver_file = "/proc/version_signature";
__u32 major, minor, patch;
struct utsname info;
+ if (access(ubuntu_kver_file, R_OK) == 0) {
+ FILE *f;
+
+ f = fopen(ubuntu_kver_file, "r");
+ if (f) {
+ if (fscanf(f, "%*s %*s %d.%d.%d\n", &major, &minor, &patch) == 3) {
+ fclose(f);
+ return KERNEL_VERSION(major, minor, patch);
+ }
+ fclose(f);
+ }
+ /* something went wrong, fall back to uname() approach */
+ }
+
uname(&info);
if (sscanf(info.release, "%u.%u.%u", &major, &minor, &patch) != 3)
return 0;
@@ -1161,12 +1190,10 @@ static struct bpf_object *bpf_object__new(const char *path,
strcpy(obj->path, path);
if (obj_name) {
- strncpy(obj->name, obj_name, sizeof(obj->name) - 1);
- obj->name[sizeof(obj->name) - 1] = 0;
+ libbpf_strlcpy(obj->name, obj_name, sizeof(obj->name));
} else {
/* Using basename() GNU version which doesn't modify arg. */
- strncpy(obj->name, basename((void *)path),
- sizeof(obj->name) - 1);
+ libbpf_strlcpy(obj->name, basename((void *)path), sizeof(obj->name));
end = strchr(obj->name, '.');
if (end)
*end = 0;
@@ -1318,7 +1345,10 @@ static int bpf_object__check_endianness(struct bpf_object *obj)
static int
bpf_object__init_license(struct bpf_object *obj, void *data, size_t size)
{
- memcpy(obj->license, data, min(size, sizeof(obj->license) - 1));
+ /* libbpf_strlcpy() only copies first N - 1 bytes, so size + 1 won't
+ * go over allowed ELF data section buffer
+ */
+ libbpf_strlcpy(obj->license, data, min(size + 1, sizeof(obj->license)));
pr_debug("license of %s is %s\n", obj->path, obj->license);
return 0;
}
@@ -2076,6 +2106,7 @@ static const char *__btf_kind_str(__u16 kind)
case BTF_KIND_DATASEC: return "datasec";
case BTF_KIND_FLOAT: return "float";
case BTF_KIND_DECL_TAG: return "decl_tag";
+ case BTF_KIND_TYPE_TAG: return "type_tag";
default: return "unknown";
}
}
@@ -2255,6 +2286,9 @@ int parse_btf_map_def(const char *map_name, struct btf *btf,
map_def->parts |= MAP_DEF_VALUE_SIZE | MAP_DEF_VALUE_TYPE;
}
else if (strcmp(name, "values") == 0) {
+ bool is_map_in_map = bpf_map_type__is_map_in_map(map_def->map_type);
+ bool is_prog_array = map_def->map_type == BPF_MAP_TYPE_PROG_ARRAY;
+ const char *desc = is_map_in_map ? "map-in-map inner" : "prog-array value";
char inner_map_name[128];
int err;
@@ -2268,8 +2302,8 @@ int parse_btf_map_def(const char *map_name, struct btf *btf,
map_name, name);
return -EINVAL;
}
- if (!bpf_map_type__is_map_in_map(map_def->map_type)) {
- pr_warn("map '%s': should be map-in-map.\n",
+ if (!is_map_in_map && !is_prog_array) {
+ pr_warn("map '%s': should be map-in-map or prog-array.\n",
map_name);
return -ENOTSUP;
}
@@ -2281,22 +2315,30 @@ int parse_btf_map_def(const char *map_name, struct btf *btf,
map_def->value_size = 4;
t = btf__type_by_id(btf, m->type);
if (!t) {
- pr_warn("map '%s': map-in-map inner type [%d] not found.\n",
- map_name, m->type);
+ pr_warn("map '%s': %s type [%d] not found.\n",
+ map_name, desc, m->type);
return -EINVAL;
}
if (!btf_is_array(t) || btf_array(t)->nelems) {
- pr_warn("map '%s': map-in-map inner spec is not a zero-sized array.\n",
- map_name);
+ pr_warn("map '%s': %s spec is not a zero-sized array.\n",
+ map_name, desc);
return -EINVAL;
}
t = skip_mods_and_typedefs(btf, btf_array(t)->type, NULL);
if (!btf_is_ptr(t)) {
- pr_warn("map '%s': map-in-map inner def is of unexpected kind %s.\n",
- map_name, btf_kind_str(t));
+ pr_warn("map '%s': %s def is of unexpected kind %s.\n",
+ map_name, desc, btf_kind_str(t));
return -EINVAL;
}
t = skip_mods_and_typedefs(btf, t->type, NULL);
+ if (is_prog_array) {
+ if (!btf_is_func_proto(t)) {
+ pr_warn("map '%s': prog-array value def is of unexpected kind %s.\n",
+ map_name, btf_kind_str(t));
+ return -EINVAL;
+ }
+ continue;
+ }
if (!btf_is_struct(t)) {
pr_warn("map '%s': map-in-map inner def is of unexpected kind %s.\n",
map_name, btf_kind_str(t));
@@ -2588,8 +2630,10 @@ static bool btf_needs_sanitization(struct bpf_object *obj)
bool has_float = kernel_supports(obj, FEAT_BTF_FLOAT);
bool has_func = kernel_supports(obj, FEAT_BTF_FUNC);
bool has_decl_tag = kernel_supports(obj, FEAT_BTF_DECL_TAG);
+ bool has_type_tag = kernel_supports(obj, FEAT_BTF_TYPE_TAG);
- return !has_func || !has_datasec || !has_func_global || !has_float || !has_decl_tag;
+ return !has_func || !has_datasec || !has_func_global || !has_float ||
+ !has_decl_tag || !has_type_tag;
}
static void bpf_object__sanitize_btf(struct bpf_object *obj, struct btf *btf)
@@ -2599,6 +2643,7 @@ static void bpf_object__sanitize_btf(struct bpf_object *obj, struct btf *btf)
bool has_float = kernel_supports(obj, FEAT_BTF_FLOAT);
bool has_func = kernel_supports(obj, FEAT_BTF_FUNC);
bool has_decl_tag = kernel_supports(obj, FEAT_BTF_DECL_TAG);
+ bool has_type_tag = kernel_supports(obj, FEAT_BTF_TYPE_TAG);
struct btf_type *t;
int i, j, vlen;
@@ -2657,6 +2702,10 @@ static void bpf_object__sanitize_btf(struct bpf_object *obj, struct btf *btf)
*/
t->name_off = 0;
t->info = BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 0);
+ } else if (!has_type_tag && btf_is_type_tag(t)) {
+ /* replace TYPE_TAG with a CONST */
+ t->name_off = 0;
+ t->info = BTF_INFO_ENC(BTF_KIND_CONST, 0, 0);
}
}
}
@@ -2752,13 +2801,12 @@ static int btf_fixup_datasec(struct bpf_object *obj, struct btf *btf,
for (i = 0, vsi = btf_var_secinfos(t); i < vars; i++, vsi++) {
t_var = btf__type_by_id(btf, vsi->type);
- var = btf_var(t_var);
-
- if (!btf_is_var(t_var)) {
+ if (!t_var || !btf_is_var(t_var)) {
pr_debug("Non-VAR type seen in section %s\n", name);
return -EINVAL;
}
+ var = btf_var(t_var);
if (var->linkage == BTF_VAR_STATIC)
continue;
@@ -2972,7 +3020,9 @@ static int bpf_object__sanitize_and_load_btf(struct bpf_object *obj)
*/
btf__set_fd(kern_btf, 0);
} else {
- err = btf__load_into_kernel(kern_btf);
+ /* currently BPF_BTF_LOAD only supports log_level 1 */
+ err = btf_load_into_kernel(kern_btf, obj->log_buf, obj->log_size,
+ obj->log_level ? 1 : 0);
}
if (sanitize) {
if (!err) {
@@ -3191,11 +3241,11 @@ static int bpf_object__elf_collect(struct bpf_object *obj)
Elf_Scn *scn;
Elf64_Shdr *sh;
- /* ELF section indices are 1-based, so allocate +1 element to keep
- * indexing simple. Also include 0th invalid section into sec_cnt for
- * simpler and more traditional iteration logic.
+ /* ELF section indices are 0-based, but sec #0 is special "invalid"
+ * section. e_shnum does include sec #0, so e_shnum is the necessary
+ * size of an array to keep all the sections.
*/
- obj->efile.sec_cnt = 1 + obj->efile.ehdr->e_shnum;
+ obj->efile.sec_cnt = obj->efile.ehdr->e_shnum;
obj->efile.secs = calloc(obj->efile.sec_cnt, sizeof(*obj->efile.secs));
if (!obj->efile.secs)
return -ENOMEM;
@@ -3271,8 +3321,12 @@ static int bpf_object__elf_collect(struct bpf_object *obj)
} else if (strcmp(name, MAPS_ELF_SEC) == 0) {
obj->efile.btf_maps_shndx = idx;
} else if (strcmp(name, BTF_ELF_SEC) == 0) {
+ if (sh->sh_type != SHT_PROGBITS)
+ return -LIBBPF_ERRNO__FORMAT;
btf_data = data;
} else if (strcmp(name, BTF_EXT_ELF_SEC) == 0) {
+ if (sh->sh_type != SHT_PROGBITS)
+ return -LIBBPF_ERRNO__FORMAT;
btf_ext_data = data;
} else if (sh->sh_type == SHT_SYMTAB) {
/* already processed during the first pass above */
@@ -3303,6 +3357,10 @@ static int bpf_object__elf_collect(struct bpf_object *obj)
} else if (sh->sh_type == SHT_REL) {
int targ_sec_idx = sh->sh_info; /* points to other section */
+ if (sh->sh_entsize != sizeof(Elf64_Rel) ||
+ targ_sec_idx >= obj->efile.sec_cnt)
+ return -LIBBPF_ERRNO__FORMAT;
+
/* Only do relo for section with exec instructions */
if (!section_have_execinstr(obj, targ_sec_idx) &&
strcmp(name, ".rel" STRUCT_OPS_SEC) &&
@@ -3333,7 +3391,8 @@ static int bpf_object__elf_collect(struct bpf_object *obj)
/* sort BPF programs by section name and in-section instruction offset
* for faster search */
- qsort(obj->programs, obj->nr_programs, sizeof(*obj->programs), cmp_progs);
+ if (obj->nr_programs)
+ qsort(obj->programs, obj->nr_programs, sizeof(*obj->programs), cmp_progs);
return bpf_object__init_btf(obj, btf_data, btf_ext_data);
}
@@ -3555,7 +3614,7 @@ static int bpf_object__collect_externs(struct bpf_object *obj)
scn = elf_sec_by_idx(obj, obj->efile.symbols_shndx);
sh = elf_sec_hdr(obj, scn);
- if (!sh)
+ if (!sh || sh->sh_entsize != sizeof(Elf64_Sym))
return -LIBBPF_ERRNO__FORMAT;
dummy_var_btf_id = add_dummy_ksym_var(obj->btf);
@@ -4022,7 +4081,7 @@ static int
bpf_object__collect_prog_relos(struct bpf_object *obj, Elf64_Shdr *shdr, Elf_Data *data)
{
const char *relo_sec_name, *sec_name;
- size_t sec_idx = shdr->sh_info;
+ size_t sec_idx = shdr->sh_info, sym_idx;
struct bpf_program *prog;
struct reloc_desc *relos;
int err, i, nrels;
@@ -4033,6 +4092,9 @@ bpf_object__collect_prog_relos(struct bpf_object *obj, Elf64_Shdr *shdr, Elf_Dat
Elf64_Sym *sym;
Elf64_Rel *rel;
+ if (sec_idx >= obj->efile.sec_cnt)
+ return -EINVAL;
+
scn = elf_sec_by_idx(obj, sec_idx);
scn_data = elf_sec_data(obj, scn);
@@ -4052,16 +4114,23 @@ bpf_object__collect_prog_relos(struct bpf_object *obj, Elf64_Shdr *shdr, Elf_Dat
return -LIBBPF_ERRNO__FORMAT;
}
- sym = elf_sym_by_idx(obj, ELF64_R_SYM(rel->r_info));
+ sym_idx = ELF64_R_SYM(rel->r_info);
+ sym = elf_sym_by_idx(obj, sym_idx);
if (!sym) {
- pr_warn("sec '%s': symbol 0x%zx not found for relo #%d\n",
- relo_sec_name, (size_t)ELF64_R_SYM(rel->r_info), i);
+ pr_warn("sec '%s': symbol #%zu not found for relo #%d\n",
+ relo_sec_name, sym_idx, i);
+ return -LIBBPF_ERRNO__FORMAT;
+ }
+
+ if (sym->st_shndx >= obj->efile.sec_cnt) {
+ pr_warn("sec '%s': corrupted symbol #%zu pointing to invalid section #%zu for relo #%d\n",
+ relo_sec_name, sym_idx, (size_t)sym->st_shndx, i);
return -LIBBPF_ERRNO__FORMAT;
}
if (rel->r_offset % BPF_INSN_SZ || rel->r_offset >= scn_data->d_size) {
pr_warn("sec '%s': invalid offset 0x%zx for relo #%d\n",
- relo_sec_name, (size_t)ELF64_R_SYM(rel->r_info), i);
+ relo_sec_name, (size_t)rel->r_offset, i);
return -LIBBPF_ERRNO__FORMAT;
}
@@ -4265,30 +4334,24 @@ int bpf_map__resize(struct bpf_map *map, __u32 max_entries)
static int
bpf_object__probe_loading(struct bpf_object *obj)
{
- struct bpf_load_program_attr attr;
char *cp, errmsg[STRERR_BUFSIZE];
struct bpf_insn insns[] = {
BPF_MOV64_IMM(BPF_REG_0, 0),
BPF_EXIT_INSN(),
};
- int ret;
+ int ret, insn_cnt = ARRAY_SIZE(insns);
if (obj->gen_loader)
return 0;
+ ret = bump_rlimit_memlock();
+ if (ret)
+ pr_warn("Failed to bump RLIMIT_MEMLOCK (err = %d), you might need to do it explicitly!\n", ret);
+
/* make sure basic loading works */
-
- memset(&attr, 0, sizeof(attr));
- attr.prog_type = BPF_PROG_TYPE_SOCKET_FILTER;
- attr.insns = insns;
- attr.insns_cnt = ARRAY_SIZE(insns);
- attr.license = "GPL";
-
- ret = bpf_load_program_xattr(&attr, NULL, 0);
- if (ret < 0) {
- attr.prog_type = BPF_PROG_TYPE_TRACEPOINT;
- ret = bpf_load_program_xattr(&attr, NULL, 0);
- }
+ ret = bpf_prog_load(BPF_PROG_TYPE_SOCKET_FILTER, NULL, "GPL", insns, insn_cnt, NULL);
+ if (ret < 0)
+ ret = bpf_prog_load(BPF_PROG_TYPE_TRACEPOINT, NULL, "GPL", insns, insn_cnt, NULL);
if (ret < 0) {
ret = errno;
cp = libbpf_strerror_r(ret, errmsg, sizeof(errmsg));
@@ -4312,29 +4375,19 @@ static int probe_fd(int fd)
static int probe_kern_prog_name(void)
{
- struct bpf_load_program_attr attr;
struct bpf_insn insns[] = {
BPF_MOV64_IMM(BPF_REG_0, 0),
BPF_EXIT_INSN(),
};
- int ret;
+ int ret, insn_cnt = ARRAY_SIZE(insns);
/* make sure loading with name works */
-
- memset(&attr, 0, sizeof(attr));
- attr.prog_type = BPF_PROG_TYPE_SOCKET_FILTER;
- attr.insns = insns;
- attr.insns_cnt = ARRAY_SIZE(insns);
- attr.license = "GPL";
- attr.name = "test";
- ret = bpf_load_program_xattr(&attr, NULL, 0);
+ ret = bpf_prog_load(BPF_PROG_TYPE_SOCKET_FILTER, "test", "GPL", insns, insn_cnt, NULL);
return probe_fd(ret);
}
static int probe_kern_global_data(void)
{
- struct bpf_load_program_attr prg_attr;
- struct bpf_create_map_attr map_attr;
char *cp, errmsg[STRERR_BUFSIZE];
struct bpf_insn insns[] = {
BPF_LD_MAP_VALUE(BPF_REG_1, 0, 16),
@@ -4342,15 +4395,9 @@ static int probe_kern_global_data(void)
BPF_MOV64_IMM(BPF_REG_0, 0),
BPF_EXIT_INSN(),
};
- int ret, map;
+ int ret, map, insn_cnt = ARRAY_SIZE(insns);
- memset(&map_attr, 0, sizeof(map_attr));
- map_attr.map_type = BPF_MAP_TYPE_ARRAY;
- map_attr.key_size = sizeof(int);
- map_attr.value_size = 32;
- map_attr.max_entries = 1;
-
- map = bpf_create_map_xattr(&map_attr);
+ map = bpf_map_create(BPF_MAP_TYPE_ARRAY, NULL, sizeof(int), 32, 1, NULL);
if (map < 0) {
ret = -errno;
cp = libbpf_strerror_r(ret, errmsg, sizeof(errmsg));
@@ -4361,13 +4408,7 @@ static int probe_kern_global_data(void)
insns[0].imm = map;
- memset(&prg_attr, 0, sizeof(prg_attr));
- prg_attr.prog_type = BPF_PROG_TYPE_SOCKET_FILTER;
- prg_attr.insns = insns;
- prg_attr.insns_cnt = ARRAY_SIZE(insns);
- prg_attr.license = "GPL";
-
- ret = bpf_load_program_xattr(&prg_attr, NULL, 0);
+ ret = bpf_prog_load(BPF_PROG_TYPE_SOCKET_FILTER, NULL, "GPL", insns, insn_cnt, NULL);
close(map);
return probe_fd(ret);
}
@@ -4468,45 +4509,51 @@ static int probe_kern_btf_decl_tag(void)
strs, sizeof(strs)));
}
-static int probe_kern_array_mmap(void)
+static int probe_kern_btf_type_tag(void)
{
- struct bpf_create_map_attr attr = {
- .map_type = BPF_MAP_TYPE_ARRAY,
- .map_flags = BPF_F_MMAPABLE,
- .key_size = sizeof(int),
- .value_size = sizeof(int),
- .max_entries = 1,
+ static const char strs[] = "\0tag";
+ __u32 types[] = {
+ /* int */
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */
+ /* attr */
+ BTF_TYPE_TYPE_TAG_ENC(1, 1), /* [2] */
+ /* ptr */
+ BTF_TYPE_ENC(0, BTF_INFO_ENC(BTF_KIND_PTR, 0, 0), 2), /* [3] */
};
- return probe_fd(bpf_create_map_xattr(&attr));
+ return probe_fd(libbpf__load_raw_btf((char *)types, sizeof(types),
+ strs, sizeof(strs)));
+}
+
+static int probe_kern_array_mmap(void)
+{
+ LIBBPF_OPTS(bpf_map_create_opts, opts, .map_flags = BPF_F_MMAPABLE);
+ int fd;
+
+ fd = bpf_map_create(BPF_MAP_TYPE_ARRAY, NULL, sizeof(int), sizeof(int), 1, &opts);
+ return probe_fd(fd);
}
static int probe_kern_exp_attach_type(void)
{
- struct bpf_load_program_attr attr;
+ LIBBPF_OPTS(bpf_prog_load_opts, opts, .expected_attach_type = BPF_CGROUP_INET_SOCK_CREATE);
struct bpf_insn insns[] = {
BPF_MOV64_IMM(BPF_REG_0, 0),
BPF_EXIT_INSN(),
};
+ int fd, insn_cnt = ARRAY_SIZE(insns);
- memset(&attr, 0, sizeof(attr));
/* use any valid combination of program type and (optional)
* non-zero expected attach type (i.e., not a BPF_CGROUP_INET_INGRESS)
* to see if kernel supports expected_attach_type field for
* BPF_PROG_LOAD command
*/
- attr.prog_type = BPF_PROG_TYPE_CGROUP_SOCK;
- attr.expected_attach_type = BPF_CGROUP_INET_SOCK_CREATE;
- attr.insns = insns;
- attr.insns_cnt = ARRAY_SIZE(insns);
- attr.license = "GPL";
-
- return probe_fd(bpf_load_program_xattr(&attr, NULL, 0));
+ fd = bpf_prog_load(BPF_PROG_TYPE_CGROUP_SOCK, NULL, "GPL", insns, insn_cnt, &opts);
+ return probe_fd(fd);
}
static int probe_kern_probe_read_kernel(void)
{
- struct bpf_load_program_attr attr;
struct bpf_insn insns[] = {
BPF_MOV64_REG(BPF_REG_1, BPF_REG_10), /* r1 = r10 (fp) */
BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, -8), /* r1 += -8 */
@@ -4515,34 +4562,22 @@ static int probe_kern_probe_read_kernel(void)
BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_probe_read_kernel),
BPF_EXIT_INSN(),
};
+ int fd, insn_cnt = ARRAY_SIZE(insns);
- memset(&attr, 0, sizeof(attr));
- attr.prog_type = BPF_PROG_TYPE_KPROBE;
- attr.insns = insns;
- attr.insns_cnt = ARRAY_SIZE(insns);
- attr.license = "GPL";
-
- return probe_fd(bpf_load_program_xattr(&attr, NULL, 0));
+ fd = bpf_prog_load(BPF_PROG_TYPE_KPROBE, NULL, "GPL", insns, insn_cnt, NULL);
+ return probe_fd(fd);
}
static int probe_prog_bind_map(void)
{
- struct bpf_load_program_attr prg_attr;
- struct bpf_create_map_attr map_attr;
char *cp, errmsg[STRERR_BUFSIZE];
struct bpf_insn insns[] = {
BPF_MOV64_IMM(BPF_REG_0, 0),
BPF_EXIT_INSN(),
};
- int ret, map, prog;
+ int ret, map, prog, insn_cnt = ARRAY_SIZE(insns);
- memset(&map_attr, 0, sizeof(map_attr));
- map_attr.map_type = BPF_MAP_TYPE_ARRAY;
- map_attr.key_size = sizeof(int);
- map_attr.value_size = 32;
- map_attr.max_entries = 1;
-
- map = bpf_create_map_xattr(&map_attr);
+ map = bpf_map_create(BPF_MAP_TYPE_ARRAY, NULL, sizeof(int), 32, 1, NULL);
if (map < 0) {
ret = -errno;
cp = libbpf_strerror_r(ret, errmsg, sizeof(errmsg));
@@ -4551,13 +4586,7 @@ static int probe_prog_bind_map(void)
return ret;
}
- memset(&prg_attr, 0, sizeof(prg_attr));
- prg_attr.prog_type = BPF_PROG_TYPE_SOCKET_FILTER;
- prg_attr.insns = insns;
- prg_attr.insns_cnt = ARRAY_SIZE(insns);
- prg_attr.license = "GPL";
-
- prog = bpf_load_program_xattr(&prg_attr, NULL, 0);
+ prog = bpf_prog_load(BPF_PROG_TYPE_SOCKET_FILTER, NULL, "GPL", insns, insn_cnt, NULL);
if (prog < 0) {
close(map);
return 0;
@@ -4602,19 +4631,14 @@ static int probe_module_btf(void)
static int probe_perf_link(void)
{
- struct bpf_load_program_attr attr;
struct bpf_insn insns[] = {
BPF_MOV64_IMM(BPF_REG_0, 0),
BPF_EXIT_INSN(),
};
int prog_fd, link_fd, err;
- memset(&attr, 0, sizeof(attr));
- attr.prog_type = BPF_PROG_TYPE_TRACEPOINT;
- attr.insns = insns;
- attr.insns_cnt = ARRAY_SIZE(insns);
- attr.license = "GPL";
- prog_fd = bpf_load_program_xattr(&attr, NULL, 0);
+ prog_fd = bpf_prog_load(BPF_PROG_TYPE_TRACEPOINT, NULL, "GPL",
+ insns, ARRAY_SIZE(insns), NULL);
if (prog_fd < 0)
return -errno;
@@ -4687,14 +4711,20 @@ static struct kern_feature_desc {
[FEAT_BTF_DECL_TAG] = {
"BTF_KIND_DECL_TAG support", probe_kern_btf_decl_tag,
},
+ [FEAT_BTF_TYPE_TAG] = {
+ "BTF_KIND_TYPE_TAG support", probe_kern_btf_type_tag,
+ },
+ [FEAT_MEMCG_ACCOUNT] = {
+ "memcg-based memory accounting", probe_memcg_account,
+ },
};
-static bool kernel_supports(const struct bpf_object *obj, enum kern_feature_id feat_id)
+bool kernel_supports(const struct bpf_object *obj, enum kern_feature_id feat_id)
{
struct kern_feature_desc *feat = &feature_probes[feat_id];
int ret;
- if (obj->gen_loader)
+ if (obj && obj->gen_loader)
/* To generate loader program assume the latest kernel
* to avoid doing extra prog_load, map_create syscalls.
*/
@@ -4821,19 +4851,16 @@ static void bpf_map__destroy(struct bpf_map *map);
static int bpf_object__create_map(struct bpf_object *obj, struct bpf_map *map, bool is_inner)
{
- struct bpf_create_map_params create_attr;
+ LIBBPF_OPTS(bpf_map_create_opts, create_attr);
struct bpf_map_def *def = &map->def;
+ const char *map_name = NULL;
+ __u32 max_entries;
int err = 0;
- memset(&create_attr, 0, sizeof(create_attr));
-
if (kernel_supports(obj, FEAT_PROG_NAME))
- create_attr.name = map->name;
+ map_name = map->name;
create_attr.map_ifindex = map->map_ifindex;
- create_attr.map_type = def->type;
create_attr.map_flags = def->map_flags;
- create_attr.key_size = def->key_size;
- create_attr.value_size = def->value_size;
create_attr.numa_node = map->numa_node;
create_attr.map_extra = map->map_extra;
@@ -4847,18 +4874,14 @@ static int bpf_object__create_map(struct bpf_object *obj, struct bpf_map *map, b
return nr_cpus;
}
pr_debug("map '%s': setting size to %d\n", map->name, nr_cpus);
- create_attr.max_entries = nr_cpus;
+ max_entries = nr_cpus;
} else {
- create_attr.max_entries = def->max_entries;
+ max_entries = def->max_entries;
}
if (bpf_map__is_struct_ops(map))
- create_attr.btf_vmlinux_value_type_id =
- map->btf_vmlinux_value_type_id;
+ create_attr.btf_vmlinux_value_type_id = map->btf_vmlinux_value_type_id;
- create_attr.btf_fd = 0;
- create_attr.btf_key_type_id = 0;
- create_attr.btf_value_type_id = 0;
if (obj->btf && btf__fd(obj->btf) >= 0 && !bpf_map_find_btf_info(obj, map)) {
create_attr.btf_fd = btf__fd(obj->btf);
create_attr.btf_key_type_id = map->btf_key_type_id;
@@ -4904,13 +4927,17 @@ static int bpf_object__create_map(struct bpf_object *obj, struct bpf_map *map, b
}
if (obj->gen_loader) {
- bpf_gen__map_create(obj->gen_loader, &create_attr, is_inner ? -1 : map - obj->maps);
+ bpf_gen__map_create(obj->gen_loader, def->type, map_name,
+ def->key_size, def->value_size, max_entries,
+ &create_attr, is_inner ? -1 : map - obj->maps);
/* Pretend to have valid FD to pass various fd >= 0 checks.
* This fd == 0 will not be used with any syscall and will be reset to -1 eventually.
*/
map->fd = 0;
} else {
- map->fd = libbpf__bpf_create_map_xattr(&create_attr);
+ map->fd = bpf_map_create(def->type, map_name,
+ def->key_size, def->value_size,
+ max_entries, &create_attr);
}
if (map->fd < 0 && (create_attr.btf_key_type_id ||
create_attr.btf_value_type_id)) {
@@ -4925,7 +4952,9 @@ static int bpf_object__create_map(struct bpf_object *obj, struct bpf_map *map, b
create_attr.btf_value_type_id = 0;
map->btf_key_type_id = 0;
map->btf_value_type_id = 0;
- map->fd = libbpf__bpf_create_map_xattr(&create_attr);
+ map->fd = bpf_map_create(def->type, map_name,
+ def->key_size, def->value_size,
+ max_entries, &create_attr);
}
err = map->fd < 0 ? -errno : 0;
@@ -4940,7 +4969,7 @@ static int bpf_object__create_map(struct bpf_object *obj, struct bpf_map *map, b
return err;
}
-static int init_map_slots(struct bpf_object *obj, struct bpf_map *map)
+static int init_map_in_map_slots(struct bpf_object *obj, struct bpf_map *map)
{
const struct bpf_map *targ_map;
unsigned int i;
@@ -4952,18 +4981,18 @@ static int init_map_slots(struct bpf_object *obj, struct bpf_map *map)
targ_map = map->init_slots[i];
fd = bpf_map__fd(targ_map);
+
if (obj->gen_loader) {
- pr_warn("// TODO map_update_elem: idx %td key %d value==map_idx %td\n",
- map - obj->maps, i, targ_map - obj->maps);
- return -ENOTSUP;
+ bpf_gen__populate_outer_map(obj->gen_loader,
+ map - obj->maps, i,
+ targ_map - obj->maps);
} else {
err = bpf_map_update_elem(map->fd, &i, &fd, 0);
}
if (err) {
err = -errno;
pr_warn("map '%s': failed to initialize slot [%d] to map '%s' fd=%d: %d\n",
- map->name, i, targ_map->name,
- fd, err);
+ map->name, i, targ_map->name, fd, err);
return err;
}
pr_debug("map '%s': slot [%d] set to map '%s' fd=%d\n",
@@ -4976,6 +5005,59 @@ static int init_map_slots(struct bpf_object *obj, struct bpf_map *map)
return 0;
}
+static int init_prog_array_slots(struct bpf_object *obj, struct bpf_map *map)
+{
+ const struct bpf_program *targ_prog;
+ unsigned int i;
+ int fd, err;
+
+ if (obj->gen_loader)
+ return -ENOTSUP;
+
+ for (i = 0; i < map->init_slots_sz; i++) {
+ if (!map->init_slots[i])
+ continue;
+
+ targ_prog = map->init_slots[i];
+ fd = bpf_program__fd(targ_prog);
+
+ err = bpf_map_update_elem(map->fd, &i, &fd, 0);
+ if (err) {
+ err = -errno;
+ pr_warn("map '%s': failed to initialize slot [%d] to prog '%s' fd=%d: %d\n",
+ map->name, i, targ_prog->name, fd, err);
+ return err;
+ }
+ pr_debug("map '%s': slot [%d] set to prog '%s' fd=%d\n",
+ map->name, i, targ_prog->name, fd);
+ }
+
+ zfree(&map->init_slots);
+ map->init_slots_sz = 0;
+
+ return 0;
+}
+
+static int bpf_object_init_prog_arrays(struct bpf_object *obj)
+{
+ struct bpf_map *map;
+ int i, err;
+
+ for (i = 0; i < obj->nr_maps; i++) {
+ map = &obj->maps[i];
+
+ if (!map->init_slots_sz || map->def.type != BPF_MAP_TYPE_PROG_ARRAY)
+ continue;
+
+ err = init_prog_array_slots(obj, map);
+ if (err < 0) {
+ zclose(map->fd);
+ return err;
+ }
+ }
+ return 0;
+}
+
static int
bpf_object__create_maps(struct bpf_object *obj)
{
@@ -4988,6 +5070,26 @@ bpf_object__create_maps(struct bpf_object *obj)
for (i = 0; i < obj->nr_maps; i++) {
map = &obj->maps[i];
+ /* To support old kernels, we skip creating global data maps
+ * (.rodata, .data, .kconfig, etc); later on, during program
+ * loading, if we detect that at least one of the to-be-loaded
+ * programs is referencing any global data map, we'll error
+ * out with program name and relocation index logged.
+ * This approach allows to accommodate Clang emitting
+ * unnecessary .rodata.str1.1 sections for string literals,
+ * but also it allows to have CO-RE applications that use
+ * global variables in some of BPF programs, but not others.
+ * If those global variable-using programs are not loaded at
+ * runtime due to bpf_program__set_autoload(prog, false),
+ * bpf_object loading will succeed just fine even on old
+ * kernels.
+ */
+ if (bpf_map__is_internal(map) &&
+ !kernel_supports(obj, FEAT_GLOBAL_DATA)) {
+ map->skipped = true;
+ continue;
+ }
+
retried = false;
retry:
if (map->pin_path) {
@@ -5024,8 +5126,8 @@ bpf_object__create_maps(struct bpf_object *obj)
}
}
- if (map->init_slots_sz) {
- err = init_map_slots(obj, map);
+ if (map->init_slots_sz && map->def.type != BPF_MAP_TYPE_PROG_ARRAY) {
+ err = init_map_in_map_slots(obj, map);
if (err < 0) {
zclose(map->fd);
goto err_out;
@@ -5097,15 +5199,18 @@ static int bpf_core_add_cands(struct bpf_core_cand *local_cand,
struct bpf_core_cand_list *cands)
{
struct bpf_core_cand *new_cands, *cand;
- const struct btf_type *t;
- const char *targ_name;
+ const struct btf_type *t, *local_t;
+ const char *targ_name, *local_name;
size_t targ_essent_len;
int n, i;
+ local_t = btf__type_by_id(local_cand->btf, local_cand->id);
+ local_name = btf__str_by_offset(local_cand->btf, local_t->name_off);
+
n = btf__type_cnt(targ_btf);
for (i = targ_start_id; i < n; i++) {
t = btf__type_by_id(targ_btf, i);
- if (btf_kind(t) != btf_kind(local_cand->t))
+ if (btf_kind(t) != btf_kind(local_t))
continue;
targ_name = btf__name_by_offset(targ_btf, t->name_off);
@@ -5116,12 +5221,12 @@ static int bpf_core_add_cands(struct bpf_core_cand *local_cand,
if (targ_essent_len != local_essent_len)
continue;
- if (strncmp(local_cand->name, targ_name, local_essent_len) != 0)
+ if (strncmp(local_name, targ_name, local_essent_len) != 0)
continue;
pr_debug("CO-RE relocating [%d] %s %s: found target candidate [%d] %s %s in [%s]\n",
- local_cand->id, btf_kind_str(local_cand->t),
- local_cand->name, i, btf_kind_str(t), targ_name,
+ local_cand->id, btf_kind_str(local_t),
+ local_name, i, btf_kind_str(t), targ_name,
targ_btf_name);
new_cands = libbpf_reallocarray(cands->cands, cands->len + 1,
sizeof(*cands->cands));
@@ -5130,8 +5235,6 @@ static int bpf_core_add_cands(struct bpf_core_cand *local_cand,
cand = &new_cands[cands->len];
cand->btf = targ_btf;
- cand->t = t;
- cand->name = targ_name;
cand->id = i;
cands->cands = new_cands;
@@ -5238,18 +5341,21 @@ bpf_core_find_cands(struct bpf_object *obj, const struct btf *local_btf, __u32 l
struct bpf_core_cand local_cand = {};
struct bpf_core_cand_list *cands;
const struct btf *main_btf;
+ const struct btf_type *local_t;
+ const char *local_name;
size_t local_essent_len;
int err, i;
local_cand.btf = local_btf;
- local_cand.t = btf__type_by_id(local_btf, local_type_id);
- if (!local_cand.t)
+ local_cand.id = local_type_id;
+ local_t = btf__type_by_id(local_btf, local_type_id);
+ if (!local_t)
return ERR_PTR(-EINVAL);
- local_cand.name = btf__name_by_offset(local_btf, local_cand.t->name_off);
- if (str_is_empty(local_cand.name))
+ local_name = btf__name_by_offset(local_btf, local_t->name_off);
+ if (str_is_empty(local_name))
return ERR_PTR(-EINVAL);
- local_essent_len = bpf_core_essential_name_len(local_cand.name);
+ local_essent_len = bpf_core_essential_name_len(local_name);
cands = calloc(1, sizeof(*cands));
if (!cands)
@@ -5399,12 +5505,31 @@ static void *u32_as_hash_key(__u32 x)
return (void *)(uintptr_t)x;
}
+static int record_relo_core(struct bpf_program *prog,
+ const struct bpf_core_relo *core_relo, int insn_idx)
+{
+ struct reloc_desc *relos, *relo;
+
+ relos = libbpf_reallocarray(prog->reloc_desc,
+ prog->nr_reloc + 1, sizeof(*relos));
+ if (!relos)
+ return -ENOMEM;
+ relo = &relos[prog->nr_reloc];
+ relo->type = RELO_CORE;
+ relo->insn_idx = insn_idx;
+ relo->core_relo = core_relo;
+ prog->reloc_desc = relos;
+ prog->nr_reloc++;
+ return 0;
+}
+
static int bpf_core_apply_relo(struct bpf_program *prog,
const struct bpf_core_relo *relo,
int relo_idx,
const struct btf *local_btf,
struct hashmap *cand_cache)
{
+ struct bpf_core_spec specs_scratch[3] = {};
const void *type_key = u32_as_hash_key(relo->type_id);
struct bpf_core_cand_list *cands = NULL;
const char *prog_name = prog->name;
@@ -5435,13 +5560,15 @@ static int bpf_core_apply_relo(struct bpf_program *prog,
return -EINVAL;
if (prog->obj->gen_loader) {
- pr_warn("// TODO core_relo: prog %td insn[%d] %s kind %d\n",
+ const char *spec_str = btf__name_by_offset(local_btf, relo->access_str_off);
+
+ pr_debug("record_relo_core: prog %td insn[%d] %s %s %s final insn_idx %d\n",
prog - prog->obj->programs, relo->insn_off / 8,
- local_name, relo->kind);
- return -ENOTSUP;
+ btf_kind_str(local_type), local_name, spec_str, insn_idx);
+ return record_relo_core(prog, relo, insn_idx);
}
- if (relo->kind != BPF_TYPE_ID_LOCAL &&
+ if (relo->kind != BPF_CORE_TYPE_ID_LOCAL &&
!hashmap__find(cand_cache, type_key, (void **)&cands)) {
cands = bpf_core_find_cands(prog->obj, local_btf, local_id);
if (IS_ERR(cands)) {
@@ -5457,7 +5584,8 @@ static int bpf_core_apply_relo(struct bpf_program *prog,
}
}
- return bpf_core_apply_relo_insn(prog_name, insn, insn_idx, relo, relo_idx, local_btf, cands);
+ return bpf_core_apply_relo_insn(prog_name, insn, insn_idx, relo,
+ relo_idx, local_btf, cands, specs_scratch);
}
static int
@@ -5587,6 +5715,13 @@ bpf_object__relocate_data(struct bpf_object *obj, struct bpf_program *prog)
insn[0].src_reg = BPF_PSEUDO_MAP_IDX_VALUE;
insn[0].imm = relo->map_idx;
} else {
+ const struct bpf_map *map = &obj->maps[relo->map_idx];
+
+ if (map->skipped) {
+ pr_warn("prog '%s': relo #%d: kernel doesn't support global data\n",
+ prog->name, i);
+ return -ENOTSUP;
+ }
insn[0].src_reg = BPF_PSEUDO_MAP_VALUE;
insn[0].imm = obj->maps[relo->map_idx].fd;
}
@@ -5635,6 +5770,9 @@ bpf_object__relocate_data(struct bpf_object *obj, struct bpf_program *prog)
case RELO_CALL:
/* handled already */
break;
+ case RELO_CORE:
+ /* will be handled by bpf_program_record_relos() */
+ break;
default:
pr_warn("prog '%s': relo #%d: bad relo type %d\n",
prog->name, i, relo->type);
@@ -5798,6 +5936,8 @@ static int cmp_relo_by_insn_idx(const void *key, const void *elem)
static struct reloc_desc *find_prog_insn_relo(const struct bpf_program *prog, size_t insn_idx)
{
+ if (!prog->nr_reloc)
+ return NULL;
return bsearch(&insn_idx, prog->reloc_desc, prog->nr_reloc,
sizeof(*prog->reloc_desc), cmp_relo_by_insn_idx);
}
@@ -5813,8 +5953,9 @@ static int append_subprog_relos(struct bpf_program *main_prog, struct bpf_progra
relos = libbpf_reallocarray(main_prog->reloc_desc, new_cnt, sizeof(*relos));
if (!relos)
return -ENOMEM;
- memcpy(relos + main_prog->nr_reloc, subprog->reloc_desc,
- sizeof(*relos) * subprog->nr_reloc);
+ if (subprog->nr_reloc)
+ memcpy(relos + main_prog->nr_reloc, subprog->reloc_desc,
+ sizeof(*relos) * subprog->nr_reloc);
for (i = main_prog->nr_reloc; i < new_cnt; i++)
relos[i].insn_idx += subprog->sub_insn_off;
@@ -6072,6 +6213,35 @@ bpf_object__free_relocs(struct bpf_object *obj)
}
}
+static int cmp_relocs(const void *_a, const void *_b)
+{
+ const struct reloc_desc *a = _a;
+ const struct reloc_desc *b = _b;
+
+ if (a->insn_idx != b->insn_idx)
+ return a->insn_idx < b->insn_idx ? -1 : 1;
+
+ /* no two relocations should have the same insn_idx, but ... */
+ if (a->type != b->type)
+ return a->type < b->type ? -1 : 1;
+
+ return 0;
+}
+
+static void bpf_object__sort_relos(struct bpf_object *obj)
+{
+ int i;
+
+ for (i = 0; i < obj->nr_programs; i++) {
+ struct bpf_program *p = &obj->programs[i];
+
+ if (!p->nr_reloc)
+ continue;
+
+ qsort(p->reloc_desc, p->nr_reloc, sizeof(*p->reloc_desc), cmp_relocs);
+ }
+}
+
static int
bpf_object__relocate(struct bpf_object *obj, const char *targ_btf_path)
{
@@ -6086,6 +6256,8 @@ bpf_object__relocate(struct bpf_object *obj, const char *targ_btf_path)
err);
return err;
}
+ if (obj->gen_loader)
+ bpf_object__sort_relos(obj);
}
/* Before relocating calls pre-process relocations and mark
@@ -6121,6 +6293,8 @@ bpf_object__relocate(struct bpf_object *obj, const char *targ_btf_path)
*/
if (prog_is_subprog(obj, prog))
continue;
+ if (!prog->load)
+ continue;
err = bpf_object__relocate_calls(obj, prog);
if (err) {
@@ -6134,6 +6308,8 @@ bpf_object__relocate(struct bpf_object *obj, const char *targ_btf_path)
prog = &obj->programs[i];
if (prog_is_subprog(obj, prog))
continue;
+ if (!prog->load)
+ continue;
err = bpf_object__relocate_data(obj, prog);
if (err) {
pr_warn("prog '%s': failed to relocate data references: %d\n",
@@ -6156,9 +6332,11 @@ static int bpf_object__collect_map_relos(struct bpf_object *obj,
int i, j, nrels, new_sz;
const struct btf_var_secinfo *vi = NULL;
const struct btf_type *sec, *var, *def;
- struct bpf_map *map = NULL, *targ_map;
+ struct bpf_map *map = NULL, *targ_map = NULL;
+ struct bpf_program *targ_prog = NULL;
+ bool is_prog_array, is_map_in_map;
const struct btf_member *member;
- const char *name, *mname;
+ const char *name, *mname, *type;
unsigned int moff;
Elf64_Sym *sym;
Elf64_Rel *rel;
@@ -6185,11 +6363,6 @@ static int bpf_object__collect_map_relos(struct bpf_object *obj,
return -LIBBPF_ERRNO__FORMAT;
}
name = elf_sym_str(obj, sym->st_name) ?: "<?>";
- if (sym->st_shndx != obj->efile.btf_maps_shndx) {
- pr_warn(".maps relo #%d: '%s' isn't a BTF-defined map\n",
- i, name);
- return -LIBBPF_ERRNO__RELOC;
- }
pr_debug(".maps relo #%d: for %zd value %zd rel->r_offset %zu name %d ('%s')\n",
i, (ssize_t)(rel->r_info >> 32), (size_t)sym->st_value,
@@ -6211,19 +6384,45 @@ static int bpf_object__collect_map_relos(struct bpf_object *obj,
return -EINVAL;
}
- if (!bpf_map_type__is_map_in_map(map->def.type))
- return -EINVAL;
- if (map->def.type == BPF_MAP_TYPE_HASH_OF_MAPS &&
- map->def.key_size != sizeof(int)) {
- pr_warn(".maps relo #%d: hash-of-maps '%s' should have key size %zu.\n",
- i, map->name, sizeof(int));
+ is_map_in_map = bpf_map_type__is_map_in_map(map->def.type);
+ is_prog_array = map->def.type == BPF_MAP_TYPE_PROG_ARRAY;
+ type = is_map_in_map ? "map" : "prog";
+ if (is_map_in_map) {
+ if (sym->st_shndx != obj->efile.btf_maps_shndx) {
+ pr_warn(".maps relo #%d: '%s' isn't a BTF-defined map\n",
+ i, name);
+ return -LIBBPF_ERRNO__RELOC;
+ }
+ if (map->def.type == BPF_MAP_TYPE_HASH_OF_MAPS &&
+ map->def.key_size != sizeof(int)) {
+ pr_warn(".maps relo #%d: hash-of-maps '%s' should have key size %zu.\n",
+ i, map->name, sizeof(int));
+ return -EINVAL;
+ }
+ targ_map = bpf_object__find_map_by_name(obj, name);
+ if (!targ_map) {
+ pr_warn(".maps relo #%d: '%s' isn't a valid map reference\n",
+ i, name);
+ return -ESRCH;
+ }
+ } else if (is_prog_array) {
+ targ_prog = bpf_object__find_program_by_name(obj, name);
+ if (!targ_prog) {
+ pr_warn(".maps relo #%d: '%s' isn't a valid program reference\n",
+ i, name);
+ return -ESRCH;
+ }
+ if (targ_prog->sec_idx != sym->st_shndx ||
+ targ_prog->sec_insn_off * 8 != sym->st_value ||
+ prog_is_subprog(obj, targ_prog)) {
+ pr_warn(".maps relo #%d: '%s' isn't an entry-point program\n",
+ i, name);
+ return -LIBBPF_ERRNO__RELOC;
+ }
+ } else {
return -EINVAL;
}
- targ_map = bpf_object__find_map_by_name(obj, name);
- if (!targ_map)
- return -ESRCH;
-
var = btf__type_by_id(obj->btf, vi->type);
def = skip_mods_and_typedefs(obj->btf, var->type, NULL);
if (btf_vlen(def) == 0)
@@ -6254,30 +6453,15 @@ static int bpf_object__collect_map_relos(struct bpf_object *obj,
(new_sz - map->init_slots_sz) * host_ptr_sz);
map->init_slots_sz = new_sz;
}
- map->init_slots[moff] = targ_map;
+ map->init_slots[moff] = is_map_in_map ? (void *)targ_map : (void *)targ_prog;
- pr_debug(".maps relo #%d: map '%s' slot [%d] points to map '%s'\n",
- i, map->name, moff, name);
+ pr_debug(".maps relo #%d: map '%s' slot [%d] points to %s '%s'\n",
+ i, map->name, moff, type, name);
}
return 0;
}
-static int cmp_relocs(const void *_a, const void *_b)
-{
- const struct reloc_desc *a = _a;
- const struct reloc_desc *b = _b;
-
- if (a->insn_idx != b->insn_idx)
- return a->insn_idx < b->insn_idx ? -1 : 1;
-
- /* no two relocations should have the same insn_idx, but ... */
- if (a->type != b->type)
- return a->type < b->type ? -1 : 1;
-
- return 0;
-}
-
static int bpf_object__collect_relos(struct bpf_object *obj)
{
int i, err;
@@ -6310,14 +6494,7 @@ static int bpf_object__collect_relos(struct bpf_object *obj)
return err;
}
- for (i = 0; i < obj->nr_programs; i++) {
- struct bpf_program *p = &obj->programs[i];
-
- if (!p->nr_reloc)
- continue;
-
- qsort(p->reloc_desc, p->nr_reloc, sizeof(*p->reloc_desc), cmp_relocs);
- }
+ bpf_object__sort_relos(obj);
return 0;
}
@@ -6374,16 +6551,16 @@ static int libbpf_find_attach_btf_id(struct bpf_program *prog, const char *attac
/* this is called as prog->sec_def->preload_fn for libbpf-supported sec_defs */
static int libbpf_preload_prog(struct bpf_program *prog,
- struct bpf_prog_load_params *attr, long cookie)
+ struct bpf_prog_load_opts *opts, long cookie)
{
enum sec_def_flags def = cookie;
/* old kernels might not support specifying expected_attach_type */
if ((def & SEC_EXP_ATTACH_OPT) && !kernel_supports(prog->obj, FEAT_EXP_ATTACH_TYPE))
- attr->expected_attach_type = 0;
+ opts->expected_attach_type = 0;
if (def & SEC_SLEEPABLE)
- attr->prog_flags |= BPF_F_SLEEPABLE;
+ opts->prog_flags |= BPF_F_SLEEPABLE;
if ((prog->type == BPF_PROG_TYPE_TRACING ||
prog->type == BPF_PROG_TYPE_LSM ||
@@ -6402,25 +6579,28 @@ static int libbpf_preload_prog(struct bpf_program *prog,
/* but by now libbpf common logic is not utilizing
* prog->atach_btf_obj_fd/prog->attach_btf_id anymore because
- * this callback is called after attrs were populated by
- * libbpf, so this callback has to update attr explicitly here
+ * this callback is called after opts were populated by
+ * libbpf, so this callback has to update opts explicitly here
*/
- attr->attach_btf_obj_fd = btf_obj_fd;
- attr->attach_btf_id = btf_type_id;
+ opts->attach_btf_obj_fd = btf_obj_fd;
+ opts->attach_btf_id = btf_type_id;
}
return 0;
}
-static int
-load_program(struct bpf_program *prog, struct bpf_insn *insns, int insns_cnt,
- char *license, __u32 kern_version, int *pfd)
+static int bpf_object_load_prog_instance(struct bpf_object *obj, struct bpf_program *prog,
+ struct bpf_insn *insns, int insns_cnt,
+ const char *license, __u32 kern_version,
+ int *prog_fd)
{
- struct bpf_prog_load_params load_attr = {};
- struct bpf_object *obj = prog->obj;
+ LIBBPF_OPTS(bpf_prog_load_opts, load_attr);
+ const char *prog_name = NULL;
char *cp, errmsg[STRERR_BUFSIZE];
size_t log_buf_size = 0;
- char *log_buf = NULL;
+ char *log_buf = NULL, *tmp;
int btf_fd, ret, err;
+ bool own_log_buf = true;
+ __u32 log_level = prog->log_level;
if (prog->type == BPF_PROG_TYPE_UNSPEC) {
/*
@@ -6435,14 +6615,9 @@ load_program(struct bpf_program *prog, struct bpf_insn *insns, int insns_cnt,
if (!insns || !insns_cnt)
return -EINVAL;
- load_attr.prog_type = prog->type;
load_attr.expected_attach_type = prog->expected_attach_type;
if (kernel_supports(obj, FEAT_PROG_NAME))
- load_attr.name = prog->name;
- load_attr.insns = insns;
- load_attr.insn_cnt = insns_cnt;
- load_attr.license = license;
- load_attr.attach_btf_id = prog->attach_btf_id;
+ prog_name = prog->name;
load_attr.attach_prog_fd = prog->attach_prog_fd;
load_attr.attach_btf_obj_fd = prog->attach_btf_obj_fd;
load_attr.attach_btf_id = prog->attach_btf_id;
@@ -6460,7 +6635,7 @@ load_program(struct bpf_program *prog, struct bpf_insn *insns, int insns_cnt,
load_attr.line_info_rec_size = prog->line_info_rec_size;
load_attr.line_info_cnt = prog->line_info_cnt;
}
- load_attr.log_level = prog->log_level;
+ load_attr.log_level = log_level;
load_attr.prog_flags = prog->prog_flags;
load_attr.fd_array = obj->fd_array;
@@ -6475,27 +6650,51 @@ load_program(struct bpf_program *prog, struct bpf_insn *insns, int insns_cnt,
}
if (obj->gen_loader) {
- bpf_gen__prog_load(obj->gen_loader, &load_attr,
+ bpf_gen__prog_load(obj->gen_loader, prog->type, prog->name,
+ license, insns, insns_cnt, &load_attr,
prog - obj->programs);
- *pfd = -1;
+ *prog_fd = -1;
return 0;
}
-retry_load:
- if (log_buf_size) {
- log_buf = malloc(log_buf_size);
- if (!log_buf)
- return -ENOMEM;
- *log_buf = 0;
+retry_load:
+ /* if log_level is zero, we don't request logs initiallly even if
+ * custom log_buf is specified; if the program load fails, then we'll
+ * bump log_level to 1 and use either custom log_buf or we'll allocate
+ * our own and retry the load to get details on what failed
+ */
+ if (log_level) {
+ if (prog->log_buf) {
+ log_buf = prog->log_buf;
+ log_buf_size = prog->log_size;
+ own_log_buf = false;
+ } else if (obj->log_buf) {
+ log_buf = obj->log_buf;
+ log_buf_size = obj->log_size;
+ own_log_buf = false;
+ } else {
+ log_buf_size = max((size_t)BPF_LOG_BUF_SIZE, log_buf_size * 2);
+ tmp = realloc(log_buf, log_buf_size);
+ if (!tmp) {
+ ret = -ENOMEM;
+ goto out;
+ }
+ log_buf = tmp;
+ log_buf[0] = '\0';
+ own_log_buf = true;
+ }
}
load_attr.log_buf = log_buf;
- load_attr.log_buf_sz = log_buf_size;
- ret = libbpf__bpf_prog_load(&load_attr);
+ load_attr.log_size = log_buf_size;
+ load_attr.log_level = log_level;
+ ret = bpf_prog_load(prog->type, prog_name, license, insns, insns_cnt, &load_attr);
if (ret >= 0) {
- if (log_buf && load_attr.log_level)
- pr_debug("verifier log:\n%s", log_buf);
+ if (log_level && own_log_buf) {
+ pr_debug("prog '%s': -- BEGIN PROG LOAD LOG --\n%s-- END PROG LOAD LOG --\n",
+ prog->name, log_buf);
+ }
if (obj->has_rodata && kernel_supports(obj, FEAT_PROG_BIND_MAP)) {
struct bpf_map *map;
@@ -6508,61 +6707,53 @@ load_program(struct bpf_program *prog, struct bpf_insn *insns, int insns_cnt,
if (bpf_prog_bind_map(ret, bpf_map__fd(map), NULL)) {
cp = libbpf_strerror_r(errno, errmsg, sizeof(errmsg));
- pr_warn("prog '%s': failed to bind .rodata map: %s\n",
- prog->name, cp);
+ pr_warn("prog '%s': failed to bind map '%s': %s\n",
+ prog->name, map->real_name, cp);
/* Don't fail hard if can't bind rodata. */
}
}
}
- *pfd = ret;
+ *prog_fd = ret;
ret = 0;
goto out;
}
- if (!log_buf || errno == ENOSPC) {
- log_buf_size = max((size_t)BPF_LOG_BUF_SIZE,
- log_buf_size << 1);
-
- free(log_buf);
+ if (log_level == 0) {
+ log_level = 1;
goto retry_load;
}
- ret = errno ? -errno : -LIBBPF_ERRNO__LOAD;
+ /* On ENOSPC, increase log buffer size and retry, unless custom
+ * log_buf is specified.
+ * Be careful to not overflow u32, though. Kernel's log buf size limit
+ * isn't part of UAPI so it can always be bumped to full 4GB. So don't
+ * multiply by 2 unless we are sure we'll fit within 32 bits.
+ * Currently, we'll get -EINVAL when we reach (UINT_MAX >> 2).
+ */
+ if (own_log_buf && errno == ENOSPC && log_buf_size <= UINT_MAX / 2)
+ goto retry_load;
+
+ ret = -errno;
cp = libbpf_strerror_r(errno, errmsg, sizeof(errmsg));
- pr_warn("load bpf program failed: %s\n", cp);
+ pr_warn("prog '%s': BPF program load failed: %s\n", prog->name, cp);
pr_perm_msg(ret);
- if (log_buf && log_buf[0] != '\0') {
- ret = -LIBBPF_ERRNO__VERIFY;
- pr_warn("-- BEGIN DUMP LOG ---\n");
- pr_warn("\n%s\n", log_buf);
- pr_warn("-- END LOG --\n");
- } else if (load_attr.insn_cnt >= BPF_MAXINSNS) {
- pr_warn("Program too large (%zu insns), at most %d insns\n",
- load_attr.insn_cnt, BPF_MAXINSNS);
- ret = -LIBBPF_ERRNO__PROG2BIG;
- } else if (load_attr.prog_type != BPF_PROG_TYPE_KPROBE) {
- /* Wrong program type? */
- int fd;
-
- load_attr.prog_type = BPF_PROG_TYPE_KPROBE;
- load_attr.expected_attach_type = 0;
- load_attr.log_buf = NULL;
- load_attr.log_buf_sz = 0;
- fd = libbpf__bpf_prog_load(&load_attr);
- if (fd >= 0) {
- close(fd);
- ret = -LIBBPF_ERRNO__PROGTYPE;
- goto out;
- }
+ if (own_log_buf && log_buf && log_buf[0] != '\0') {
+ pr_warn("prog '%s': -- BEGIN PROG LOAD LOG --\n%s-- END PROG LOAD LOG --\n",
+ prog->name, log_buf);
+ }
+ if (insns_cnt >= BPF_MAXINSNS) {
+ pr_warn("prog '%s': program too large (%d insns), at most %d insns\n",
+ prog->name, insns_cnt, BPF_MAXINSNS);
}
out:
- free(log_buf);
+ if (own_log_buf)
+ free(log_buf);
return ret;
}
-static int bpf_program__record_externs(struct bpf_program *prog)
+static int bpf_program_record_relos(struct bpf_program *prog)
{
struct bpf_object *obj = prog->obj;
int i;
@@ -6584,6 +6775,17 @@ static int bpf_program__record_externs(struct bpf_program *prog)
ext->is_weak, false, BTF_KIND_FUNC,
relo->insn_idx);
break;
+ case RELO_CORE: {
+ struct bpf_core_relo cr = {
+ .insn_off = relo->insn_idx * 8,
+ .type_id = relo->core_relo->type_id,
+ .access_str_off = relo->core_relo->access_str_off,
+ .kind = relo->core_relo->kind,
+ };
+
+ bpf_gen__record_relo_core(obj->gen_loader, &cr);
+ break;
+ }
default:
continue;
}
@@ -6591,11 +6793,12 @@ static int bpf_program__record_externs(struct bpf_program *prog)
return 0;
}
-int bpf_program__load(struct bpf_program *prog, char *license, __u32 kern_ver)
+static int bpf_object_load_prog(struct bpf_object *obj, struct bpf_program *prog,
+ const char *license, __u32 kern_ver)
{
int err = 0, fd, i;
- if (prog->obj->loaded) {
+ if (obj->loaded) {
pr_warn("prog '%s': can't load after object was loaded\n", prog->name);
return libbpf_err(-EINVAL);
}
@@ -6621,10 +6824,11 @@ int bpf_program__load(struct bpf_program *prog, char *license, __u32 kern_ver)
pr_warn("prog '%s': inconsistent nr(%d) != 1\n",
prog->name, prog->instances.nr);
}
- if (prog->obj->gen_loader)
- bpf_program__record_externs(prog);
- err = load_program(prog, prog->insns, prog->insns_cnt,
- license, kern_ver, &fd);
+ if (obj->gen_loader)
+ bpf_program_record_relos(prog);
+ err = bpf_object_load_prog_instance(obj, prog,
+ prog->insns, prog->insns_cnt,
+ license, kern_ver, &fd);
if (!err)
prog->instances.fds[0] = fd;
goto out;
@@ -6652,8 +6856,9 @@ int bpf_program__load(struct bpf_program *prog, char *license, __u32 kern_ver)
continue;
}
- err = load_program(prog, result.new_insn_ptr,
- result.new_insn_cnt, license, kern_ver, &fd);
+ err = bpf_object_load_prog_instance(obj, prog,
+ result.new_insn_ptr, result.new_insn_cnt,
+ license, kern_ver, &fd);
if (err) {
pr_warn("Loading the %dth instance of program '%s' failed\n",
i, prog->name);
@@ -6670,6 +6875,11 @@ int bpf_program__load(struct bpf_program *prog, char *license, __u32 kern_ver)
return libbpf_err(err);
}
+int bpf_program__load(struct bpf_program *prog, const char *license, __u32 kern_ver)
+{
+ return bpf_object_load_prog(prog->obj, prog, license, kern_ver);
+}
+
static int
bpf_object__load_progs(struct bpf_object *obj, int log_level)
{
@@ -6693,7 +6903,7 @@ bpf_object__load_progs(struct bpf_object *obj, int log_level)
continue;
}
prog->log_level |= log_level;
- err = bpf_program__load(prog, obj->license, obj->kern_version);
+ err = bpf_object_load_prog(obj, prog, obj->license, obj->kern_version);
if (err)
return err;
}
@@ -6744,14 +6954,16 @@ static int bpf_object_init_progs(struct bpf_object *obj, const struct bpf_object
return 0;
}
-static struct bpf_object *
-__bpf_object__open(const char *path, const void *obj_buf, size_t obj_buf_sz,
- const struct bpf_object_open_opts *opts)
+static struct bpf_object *bpf_object_open(const char *path, const void *obj_buf, size_t obj_buf_sz,
+ const struct bpf_object_open_opts *opts)
{
const char *obj_name, *kconfig, *btf_tmp_path;
struct bpf_object *obj;
char tmp_name[64];
int err;
+ char *log_buf;
+ size_t log_size;
+ __u32 log_level;
if (elf_version(EV_CURRENT) == EV_NONE) {
pr_warn("failed to init libelf for %s\n",
@@ -6774,10 +6986,22 @@ __bpf_object__open(const char *path, const void *obj_buf, size_t obj_buf_sz,
pr_debug("loading object '%s' from buffer\n", obj_name);
}
+ log_buf = OPTS_GET(opts, kernel_log_buf, NULL);
+ log_size = OPTS_GET(opts, kernel_log_size, 0);
+ log_level = OPTS_GET(opts, kernel_log_level, 0);
+ if (log_size > UINT_MAX)
+ return ERR_PTR(-EINVAL);
+ if (log_size && !log_buf)
+ return ERR_PTR(-EINVAL);
+
obj = bpf_object__new(path, obj_buf, obj_buf_sz, obj_name);
if (IS_ERR(obj))
return obj;
+ obj->log_buf = log_buf;
+ obj->log_size = log_size;
+ obj->log_level = log_level;
+
btf_tmp_path = OPTS_GET(opts, btf_custom_path, NULL);
if (btf_tmp_path) {
if (strlen(btf_tmp_path) >= PATH_MAX) {
@@ -6831,7 +7055,7 @@ __bpf_object__open_xattr(struct bpf_object_open_attr *attr, int flags)
return NULL;
pr_debug("loading %s\n", attr->file);
- return __bpf_object__open(attr->file, NULL, 0, &opts);
+ return bpf_object_open(attr->file, NULL, 0, &opts);
}
struct bpf_object *bpf_object__open_xattr(struct bpf_object_open_attr *attr)
@@ -6857,7 +7081,7 @@ bpf_object__open_file(const char *path, const struct bpf_object_open_opts *opts)
pr_debug("loading %s\n", path);
- return libbpf_ptr(__bpf_object__open(path, NULL, 0, opts));
+ return libbpf_ptr(bpf_object_open(path, NULL, 0, opts));
}
struct bpf_object *
@@ -6867,7 +7091,7 @@ bpf_object__open_mem(const void *obj_buf, size_t obj_buf_sz,
if (!obj_buf || obj_buf_sz == 0)
return libbpf_err_ptr(-EINVAL);
- return libbpf_ptr(__bpf_object__open(NULL, obj_buf, obj_buf_sz, opts));
+ return libbpf_ptr(bpf_object_open(NULL, obj_buf, obj_buf_sz, opts));
}
struct bpf_object *
@@ -6884,7 +7108,7 @@ bpf_object__open_buffer(const void *obj_buf, size_t obj_buf_sz,
if (!obj_buf || obj_buf_sz == 0)
return errno = EINVAL, NULL;
- return libbpf_ptr(__bpf_object__open(NULL, obj_buf, obj_buf_sz, &opts));
+ return libbpf_ptr(bpf_object_open(NULL, obj_buf, obj_buf_sz, &opts));
}
static int bpf_object_unload(struct bpf_object *obj)
@@ -6915,10 +7139,6 @@ static int bpf_object__sanitize_maps(struct bpf_object *obj)
bpf_object__for_each_map(m, obj) {
if (!bpf_map__is_internal(m))
continue;
- if (!kernel_supports(obj, FEAT_GLOBAL_DATA)) {
- pr_warn("kernel doesn't support global data\n");
- return -ENOTSUP;
- }
if (!kernel_supports(obj, FEAT_ARRAY_MMAP))
m->def.map_flags ^= BPF_F_MMAPABLE;
}
@@ -7241,14 +7461,10 @@ static int bpf_object__resolve_externs(struct bpf_object *obj,
return 0;
}
-int bpf_object__load_xattr(struct bpf_object_load_attr *attr)
+static int bpf_object_load(struct bpf_object *obj, int extra_log_level, const char *target_btf_path)
{
- struct bpf_object *obj;
int err, i;
- if (!attr)
- return libbpf_err(-EINVAL);
- obj = attr->obj;
if (!obj)
return libbpf_err(-EINVAL);
@@ -7258,7 +7474,7 @@ int bpf_object__load_xattr(struct bpf_object_load_attr *attr)
}
if (obj->gen_loader)
- bpf_gen__init(obj->gen_loader, attr->log_level, obj->nr_programs, obj->nr_maps);
+ bpf_gen__init(obj->gen_loader, extra_log_level, obj->nr_programs, obj->nr_maps);
err = bpf_object__probe_loading(obj);
err = err ? : bpf_object__load_vmlinux_btf(obj, false);
@@ -7267,8 +7483,9 @@ int bpf_object__load_xattr(struct bpf_object_load_attr *attr)
err = err ? : bpf_object__sanitize_maps(obj);
err = err ? : bpf_object__init_kern_struct_ops_maps(obj);
err = err ? : bpf_object__create_maps(obj);
- err = err ? : bpf_object__relocate(obj, obj->btf_custom_path ? : attr->target_btf_path);
- err = err ? : bpf_object__load_progs(obj, attr->log_level);
+ err = err ? : bpf_object__relocate(obj, obj->btf_custom_path ? : target_btf_path);
+ err = err ? : bpf_object__load_progs(obj, extra_log_level);
+ err = err ? : bpf_object_init_prog_arrays(obj);
if (obj->gen_loader) {
/* reset FDs */
@@ -7312,13 +7529,14 @@ int bpf_object__load_xattr(struct bpf_object_load_attr *attr)
return libbpf_err(err);
}
+int bpf_object__load_xattr(struct bpf_object_load_attr *attr)
+{
+ return bpf_object_load(attr->obj, attr->log_level, attr->target_btf_path);
+}
+
int bpf_object__load(struct bpf_object *obj)
{
- struct bpf_object_load_attr attr = {
- .obj = obj,
- };
-
- return bpf_object__load_xattr(&attr);
+ return bpf_object_load(obj, 0, NULL);
}
static int make_parent_dir(const char *path)
@@ -7707,6 +7925,9 @@ int bpf_object__pin_maps(struct bpf_object *obj, const char *path)
char *pin_path = NULL;
char buf[PATH_MAX];
+ if (map->skipped)
+ continue;
+
if (path) {
int len;
@@ -7733,7 +7954,7 @@ int bpf_object__pin_maps(struct bpf_object *obj, const char *path)
return 0;
err_unpin_maps:
- while ((map = bpf_map__prev(map, obj))) {
+ while ((map = bpf_object__prev_map(obj, map))) {
if (!map->pin_path)
continue;
@@ -7813,7 +8034,7 @@ int bpf_object__pin_programs(struct bpf_object *obj, const char *path)
return 0;
err_unpin_programs:
- while ((prog = bpf_program__prev(prog, obj))) {
+ while ((prog = bpf_object__prev_program(obj, prog))) {
char buf[PATH_MAX];
int len;
@@ -8154,9 +8375,11 @@ int bpf_program__set_autoload(struct bpf_program *prog, bool autoload)
return 0;
}
+static int bpf_program_nth_fd(const struct bpf_program *prog, int n);
+
int bpf_program__fd(const struct bpf_program *prog)
{
- return bpf_program__nth_fd(prog, 0);
+ return bpf_program_nth_fd(prog, 0);
}
size_t bpf_program__size(const struct bpf_program *prog)
@@ -8202,7 +8425,10 @@ int bpf_program__set_prep(struct bpf_program *prog, int nr_instances,
return 0;
}
-int bpf_program__nth_fd(const struct bpf_program *prog, int n)
+__attribute__((alias("bpf_program_nth_fd")))
+int bpf_program__nth_fd(const struct bpf_program *prog, int n);
+
+static int bpf_program_nth_fd(const struct bpf_program *prog, int n)
{
int fd;
@@ -8281,6 +8507,54 @@ void bpf_program__set_expected_attach_type(struct bpf_program *prog,
prog->expected_attach_type = type;
}
+__u32 bpf_program__flags(const struct bpf_program *prog)
+{
+ return prog->prog_flags;
+}
+
+int bpf_program__set_flags(struct bpf_program *prog, __u32 flags)
+{
+ if (prog->obj->loaded)
+ return libbpf_err(-EBUSY);
+
+ prog->prog_flags = flags;
+ return 0;
+}
+
+__u32 bpf_program__log_level(const struct bpf_program *prog)
+{
+ return prog->log_level;
+}
+
+int bpf_program__set_log_level(struct bpf_program *prog, __u32 log_level)
+{
+ if (prog->obj->loaded)
+ return libbpf_err(-EBUSY);
+
+ prog->log_level = log_level;
+ return 0;
+}
+
+const char *bpf_program__log_buf(const struct bpf_program *prog, size_t *log_size)
+{
+ *log_size = prog->log_size;
+ return prog->log_buf;
+}
+
+int bpf_program__set_log_buf(struct bpf_program *prog, char *log_buf, size_t log_size)
+{
+ if (log_size && !log_buf)
+ return -EINVAL;
+ if (prog->log_size > UINT_MAX)
+ return -EINVAL;
+ if (prog->obj->loaded)
+ return -EBUSY;
+
+ prog->log_buf = log_buf;
+ prog->log_size = log_size;
+ return 0;
+}
+
#define SEC_DEF(sec_pfx, ptype, atype, flags, ...) { \
.sec = sec_pfx, \
.prog_type = BPF_PROG_TYPE_##ptype, \
@@ -9028,7 +9302,10 @@ int bpf_map__set_inner_map_fd(struct bpf_map *map, int fd)
pr_warn("error: inner_map_fd already specified\n");
return libbpf_err(-EINVAL);
}
- zfree(&map->inner_map);
+ if (map->inner_map) {
+ bpf_map__destroy(map->inner_map);
+ zfree(&map->inner_map);
+ }
map->inner_map_fd = fd;
return 0;
}
@@ -9145,21 +9422,12 @@ long libbpf_get_error(const void *ptr)
return -errno;
}
-int bpf_prog_load(const char *file, enum bpf_prog_type type,
- struct bpf_object **pobj, int *prog_fd)
-{
- struct bpf_prog_load_attr attr;
-
- memset(&attr, 0, sizeof(struct bpf_prog_load_attr));
- attr.file = file;
- attr.prog_type = type;
- attr.expected_attach_type = 0;
-
- return bpf_prog_load_xattr(&attr, pobj, prog_fd);
-}
-
+__attribute__((alias("bpf_prog_load_xattr2")))
int bpf_prog_load_xattr(const struct bpf_prog_load_attr *attr,
- struct bpf_object **pobj, int *prog_fd)
+ struct bpf_object **pobj, int *prog_fd);
+
+static int bpf_prog_load_xattr2(const struct bpf_prog_load_attr *attr,
+ struct bpf_object **pobj, int *prog_fd)
{
struct bpf_object_open_attr open_attr = {};
struct bpf_program *prog, *first_prog = NULL;
@@ -9230,6 +9498,20 @@ int bpf_prog_load_xattr(const struct bpf_prog_load_attr *attr,
return 0;
}
+COMPAT_VERSION(bpf_prog_load_deprecated, bpf_prog_load, LIBBPF_0.0.1)
+int bpf_prog_load_deprecated(const char *file, enum bpf_prog_type type,
+ struct bpf_object **pobj, int *prog_fd)
+{
+ struct bpf_prog_load_attr attr;
+
+ memset(&attr, 0, sizeof(struct bpf_prog_load_attr));
+ attr.file = file;
+ attr.prog_type = type;
+ attr.expected_attach_type = 0;
+
+ return bpf_prog_load_xattr2(&attr, pobj, prog_fd);
+}
+
struct bpf_link {
int (*detach)(struct bpf_link *link);
void (*dealloc)(struct bpf_link *link);
@@ -9634,7 +9916,10 @@ static int append_to_file(const char *file, const char *fmt, ...)
static void gen_kprobe_legacy_event_name(char *buf, size_t buf_sz,
const char *kfunc_name, size_t offset)
{
- snprintf(buf, buf_sz, "libbpf_%u_%s_0x%zx", getpid(), kfunc_name, offset);
+ static int index = 0;
+
+ snprintf(buf, buf_sz, "libbpf_%u_%s_0x%zx_%d", getpid(), kfunc_name, offset,
+ __sync_fetch_and_add(&index, 1));
}
static int add_kprobe_event_legacy(const char *probe_name, bool retprobe,
@@ -9735,7 +10020,7 @@ bpf_program__attach_kprobe_opts(const struct bpf_program *prog,
gen_kprobe_legacy_event_name(probe_name, sizeof(probe_name),
func_name, offset);
- legacy_probe = strdup(func_name);
+ legacy_probe = strdup(probe_name);
if (!legacy_probe)
return libbpf_err_ptr(-ENOMEM);
@@ -10394,10 +10679,10 @@ struct bpf_link *bpf_map__attach_struct_ops(const struct bpf_map *map)
return link;
}
-enum bpf_perf_event_ret
-bpf_perf_event_read_simple(void *mmap_mem, size_t mmap_size, size_t page_size,
- void **copy_mem, size_t *copy_size,
- bpf_perf_event_print_t fn, void *private_data)
+static enum bpf_perf_event_ret
+perf_event_read_simple(void *mmap_mem, size_t mmap_size, size_t page_size,
+ void **copy_mem, size_t *copy_size,
+ bpf_perf_event_print_t fn, void *private_data)
{
struct perf_event_mmap_page *header = mmap_mem;
__u64 data_head = ring_buffer_read_head(header);
@@ -10442,6 +10727,12 @@ bpf_perf_event_read_simple(void *mmap_mem, size_t mmap_size, size_t page_size,
return libbpf_err(ret);
}
+__attribute__((alias("perf_event_read_simple")))
+enum bpf_perf_event_ret
+bpf_perf_event_read_simple(void *mmap_mem, size_t mmap_size, size_t page_size,
+ void **copy_mem, size_t *copy_size,
+ bpf_perf_event_print_t fn, void *private_data);
+
struct perf_buffer;
struct perf_buffer_params {
@@ -10575,11 +10866,18 @@ perf_buffer__open_cpu_buf(struct perf_buffer *pb, struct perf_event_attr *attr,
static struct perf_buffer *__perf_buffer__new(int map_fd, size_t page_cnt,
struct perf_buffer_params *p);
-struct perf_buffer *perf_buffer__new(int map_fd, size_t page_cnt,
- const struct perf_buffer_opts *opts)
+DEFAULT_VERSION(perf_buffer__new_v0_6_0, perf_buffer__new, LIBBPF_0.6.0)
+struct perf_buffer *perf_buffer__new_v0_6_0(int map_fd, size_t page_cnt,
+ perf_buffer_sample_fn sample_cb,
+ perf_buffer_lost_fn lost_cb,
+ void *ctx,
+ const struct perf_buffer_opts *opts)
{
struct perf_buffer_params p = {};
- struct perf_event_attr attr = { 0, };
+ struct perf_event_attr attr = {};
+
+ if (!OPTS_VALID(opts, perf_buffer_opts))
+ return libbpf_err_ptr(-EINVAL);
attr.config = PERF_COUNT_SW_BPF_OUTPUT;
attr.type = PERF_TYPE_SOFTWARE;
@@ -10588,29 +10886,62 @@ struct perf_buffer *perf_buffer__new(int map_fd, size_t page_cnt,
attr.wakeup_events = 1;
p.attr = &attr;
- p.sample_cb = opts ? opts->sample_cb : NULL;
- p.lost_cb = opts ? opts->lost_cb : NULL;
- p.ctx = opts ? opts->ctx : NULL;
+ p.sample_cb = sample_cb;
+ p.lost_cb = lost_cb;
+ p.ctx = ctx;
return libbpf_ptr(__perf_buffer__new(map_fd, page_cnt, &p));
}
-struct perf_buffer *
-perf_buffer__new_raw(int map_fd, size_t page_cnt,
- const struct perf_buffer_raw_opts *opts)
+COMPAT_VERSION(perf_buffer__new_deprecated, perf_buffer__new, LIBBPF_0.0.4)
+struct perf_buffer *perf_buffer__new_deprecated(int map_fd, size_t page_cnt,
+ const struct perf_buffer_opts *opts)
+{
+ return perf_buffer__new_v0_6_0(map_fd, page_cnt,
+ opts ? opts->sample_cb : NULL,
+ opts ? opts->lost_cb : NULL,
+ opts ? opts->ctx : NULL,
+ NULL);
+}
+
+DEFAULT_VERSION(perf_buffer__new_raw_v0_6_0, perf_buffer__new_raw, LIBBPF_0.6.0)
+struct perf_buffer *perf_buffer__new_raw_v0_6_0(int map_fd, size_t page_cnt,
+ struct perf_event_attr *attr,
+ perf_buffer_event_fn event_cb, void *ctx,
+ const struct perf_buffer_raw_opts *opts)
{
struct perf_buffer_params p = {};
- p.attr = opts->attr;
- p.event_cb = opts->event_cb;
- p.ctx = opts->ctx;
- p.cpu_cnt = opts->cpu_cnt;
- p.cpus = opts->cpus;
- p.map_keys = opts->map_keys;
+ if (page_cnt == 0 || !attr)
+ return libbpf_err_ptr(-EINVAL);
+
+ if (!OPTS_VALID(opts, perf_buffer_raw_opts))
+ return libbpf_err_ptr(-EINVAL);
+
+ p.attr = attr;
+ p.event_cb = event_cb;
+ p.ctx = ctx;
+ p.cpu_cnt = OPTS_GET(opts, cpu_cnt, 0);
+ p.cpus = OPTS_GET(opts, cpus, NULL);
+ p.map_keys = OPTS_GET(opts, map_keys, NULL);
return libbpf_ptr(__perf_buffer__new(map_fd, page_cnt, &p));
}
+COMPAT_VERSION(perf_buffer__new_raw_deprecated, perf_buffer__new_raw, LIBBPF_0.0.4)
+struct perf_buffer *perf_buffer__new_raw_deprecated(int map_fd, size_t page_cnt,
+ const struct perf_buffer_raw_opts *opts)
+{
+ LIBBPF_OPTS(perf_buffer_raw_opts, inner_opts,
+ .cpu_cnt = opts->cpu_cnt,
+ .cpus = opts->cpus,
+ .map_keys = opts->map_keys,
+ );
+
+ return perf_buffer__new_raw_v0_6_0(map_fd, page_cnt, opts->attr,
+ opts->event_cb, opts->ctx, &inner_opts);
+}
+
static struct perf_buffer *__perf_buffer__new(int map_fd, size_t page_cnt,
struct perf_buffer_params *p)
{
@@ -10810,10 +11141,10 @@ static int perf_buffer__process_records(struct perf_buffer *pb,
{
enum bpf_perf_event_ret ret;
- ret = bpf_perf_event_read_simple(cpu_buf->base, pb->mmap_size,
- pb->page_size, &cpu_buf->buf,
- &cpu_buf->buf_size,
- perf_buffer__process_record, cpu_buf);
+ ret = perf_event_read_simple(cpu_buf->base, pb->mmap_size,
+ pb->page_size, &cpu_buf->buf,
+ &cpu_buf->buf_size,
+ perf_buffer__process_record, cpu_buf);
if (ret != LIBBPF_PERF_EVENT_CONT)
return ret;
return 0;
diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h
index 9de0f29..8b9bc5e 100644
--- a/tools/lib/bpf/libbpf.h
+++ b/tools/lib/bpf/libbpf.h
@@ -24,6 +24,10 @@
extern "C" {
#endif
+LIBBPF_API __u32 libbpf_major_version(void);
+LIBBPF_API __u32 libbpf_minor_version(void);
+LIBBPF_API const char *libbpf_version_string(void);
+
enum libbpf_errno {
__LIBBPF_ERRNO__START = 4000,
@@ -104,12 +108,73 @@ struct bpf_object_open_opts {
* struct_ops, etc) will need actual kernel BTF at /sys/kernel/btf/vmlinux.
*/
const char *btf_custom_path;
+ /* Pointer to a buffer for storing kernel logs for applicable BPF
+ * commands. Valid kernel_log_size has to be specified as well and are
+ * passed-through to bpf() syscall. Keep in mind that kernel might
+ * fail operation with -ENOSPC error if provided buffer is too small
+ * to contain entire log output.
+ * See the comment below for kernel_log_level for interaction between
+ * log_buf and log_level settings.
+ *
+ * If specified, this log buffer will be passed for:
+ * - each BPF progral load (BPF_PROG_LOAD) attempt, unless overriden
+ * with bpf_program__set_log() on per-program level, to get
+ * BPF verifier log output.
+ * - during BPF object's BTF load into kernel (BPF_BTF_LOAD) to get
+ * BTF sanity checking log.
+ *
+ * Each BPF command (BPF_BTF_LOAD or BPF_PROG_LOAD) will overwrite
+ * previous contents, so if you need more fine-grained control, set
+ * per-program buffer with bpf_program__set_log_buf() to preserve each
+ * individual program's verification log. Keep using kernel_log_buf
+ * for BTF verification log, if necessary.
+ */
+ char *kernel_log_buf;
+ size_t kernel_log_size;
+ /*
+ * Log level can be set independently from log buffer. Log_level=0
+ * means that libbpf will attempt loading BTF or program without any
+ * logging requested, but will retry with either its own or custom log
+ * buffer, if provided, and log_level=1 on any error.
+ * And vice versa, setting log_level>0 will request BTF or prog
+ * loading with verbose log from the first attempt (and as such also
+ * for successfully loaded BTF or program), and the actual log buffer
+ * could be either libbpf's own auto-allocated log buffer, if
+ * kernel_log_buffer is NULL, or user-provided custom kernel_log_buf.
+ * If user didn't provide custom log buffer, libbpf will emit captured
+ * logs through its print callback.
+ */
+ __u32 kernel_log_level;
+
+ size_t :0;
};
-#define bpf_object_open_opts__last_field btf_custom_path
+#define bpf_object_open_opts__last_field kernel_log_level
LIBBPF_API struct bpf_object *bpf_object__open(const char *path);
+
+/**
+ * @brief **bpf_object__open_file()** creates a bpf_object by opening
+ * the BPF ELF object file pointed to by the passed path and loading it
+ * into memory.
+ * @param path BPF object file path
+ * @param opts options for how to load the bpf object, this parameter is
+ * optional and can be set to NULL
+ * @return pointer to the new bpf_object; or NULL is returned on error,
+ * error code is stored in errno
+ */
LIBBPF_API struct bpf_object *
bpf_object__open_file(const char *path, const struct bpf_object_open_opts *opts);
+
+/**
+ * @brief **bpf_object__open_mem()** creates a bpf_object by reading
+ * the BPF objects raw bytes from a memory buffer containing a valid
+ * BPF ELF object file.
+ * @param obj_buf pointer to the buffer containing ELF file bytes
+ * @param obj_buf_sz number of bytes in the buffer
+ * @param opts options for how to load the bpf object
+ * @return pointer to the new bpf_object; or NULL is returned on error,
+ * error code is stored in errno
+ */
LIBBPF_API struct bpf_object *
bpf_object__open_mem(const void *obj_buf, size_t obj_buf_sz,
const struct bpf_object_open_opts *opts);
@@ -149,6 +214,7 @@ struct bpf_object_load_attr {
/* Load/unload object into/from kernel */
LIBBPF_API int bpf_object__load(struct bpf_object *obj);
+LIBBPF_DEPRECATED_SINCE(0, 8, "use bpf_object__load() instead")
LIBBPF_API int bpf_object__load_xattr(struct bpf_object_load_attr *attr);
LIBBPF_DEPRECATED_SINCE(0, 6, "bpf_object__unload() is deprecated, use bpf_object__close() instead")
LIBBPF_API int bpf_object__unload(struct bpf_object *obj);
@@ -161,6 +227,7 @@ struct btf;
LIBBPF_API struct btf *bpf_object__btf(const struct bpf_object *obj);
LIBBPF_API int bpf_object__btf_fd(const struct bpf_object *obj);
+LIBBPF_DEPRECATED_SINCE(0, 7, "use bpf_object__find_program_by_name() instead")
LIBBPF_API struct bpf_program *
bpf_object__find_program_by_title(const struct bpf_object *obj,
const char *title);
@@ -262,8 +329,8 @@ LIBBPF_API const struct bpf_insn *bpf_program__insns(const struct bpf_program *p
*/
LIBBPF_API size_t bpf_program__insn_cnt(const struct bpf_program *prog);
-LIBBPF_API int bpf_program__load(struct bpf_program *prog, char *license,
- __u32 kern_version);
+LIBBPF_DEPRECATED_SINCE(0, 6, "use bpf_object__load() instead")
+LIBBPF_API int bpf_program__load(struct bpf_program *prog, const char *license, __u32 kern_version);
LIBBPF_API int bpf_program__fd(const struct bpf_program *prog);
LIBBPF_DEPRECATED_SINCE(0, 7, "multi-instance bpf_program support is deprecated")
LIBBPF_API int bpf_program__pin_instance(struct bpf_program *prog,
@@ -273,7 +340,31 @@ LIBBPF_DEPRECATED_SINCE(0, 7, "multi-instance bpf_program support is deprecated"
LIBBPF_API int bpf_program__unpin_instance(struct bpf_program *prog,
const char *path,
int instance);
+
+/**
+ * @brief **bpf_program__pin()** pins the BPF program to a file
+ * in the BPF FS specified by a path. This increments the programs
+ * reference count, allowing it to stay loaded after the process
+ * which loaded it has exited.
+ *
+ * @param prog BPF program to pin, must already be loaded
+ * @param path file path in a BPF file system
+ * @return 0, on success; negative error code, otherwise
+ */
LIBBPF_API int bpf_program__pin(struct bpf_program *prog, const char *path);
+
+/**
+ * @brief **bpf_program__unpin()** unpins the BPF program from a file
+ * in the BPFFS specified by a path. This decrements the programs
+ * reference count.
+ *
+ * The file pinning the BPF program can also be unlinked by a different
+ * process in which case this function will return an error.
+ *
+ * @param prog BPF program to unpin
+ * @param path file path to the pin in a BPF file system
+ * @return 0, on success; negative error code, otherwise
+ */
LIBBPF_API int bpf_program__unpin(struct bpf_program *prog, const char *path);
LIBBPF_API void bpf_program__unload(struct bpf_program *prog);
@@ -344,10 +435,41 @@ struct bpf_uprobe_opts {
};
#define bpf_uprobe_opts__last_field retprobe
+/**
+ * @brief **bpf_program__attach_uprobe()** attaches a BPF program
+ * to the userspace function which is found by binary path and
+ * offset. You can optionally specify a particular proccess to attach
+ * to. You can also optionally attach the program to the function
+ * exit instead of entry.
+ *
+ * @param prog BPF program to attach
+ * @param retprobe Attach to function exit
+ * @param pid Process ID to attach the uprobe to, 0 for self (own process),
+ * -1 for all processes
+ * @param binary_path Path to binary that contains the function symbol
+ * @param func_offset Offset within the binary of the function symbol
+ * @return Reference to the newly created BPF link; or NULL is returned on error,
+ * error code is stored in errno
+ */
LIBBPF_API struct bpf_link *
bpf_program__attach_uprobe(const struct bpf_program *prog, bool retprobe,
pid_t pid, const char *binary_path,
size_t func_offset);
+
+/**
+ * @brief **bpf_program__attach_uprobe_opts()** is just like
+ * bpf_program__attach_uprobe() except with a options struct
+ * for various configurations.
+ *
+ * @param prog BPF program to attach
+ * @param pid Process ID to attach the uprobe to, 0 for self (own process),
+ * -1 for all processes
+ * @param binary_path Path to binary that contains the function symbol
+ * @param func_offset Offset within the binary of the function symbol
+ * @param opts Options for altering program attachment
+ * @return Reference to the newly created BPF link; or NULL is returned on error,
+ * error code is stored in errno
+ */
LIBBPF_API struct bpf_link *
bpf_program__attach_uprobe_opts(const struct bpf_program *prog, pid_t pid,
const char *binary_path, size_t func_offset,
@@ -431,7 +553,6 @@ bpf_program__attach_iter(const struct bpf_program *prog,
* one instance. In this case bpf_program__fd(prog) is equal to
* bpf_program__nth_fd(prog, 0).
*/
-LIBBPF_DEPRECATED_SINCE(0, 7, "use bpf_program__insns() for getting bpf_program instructions")
struct bpf_prog_prep_result {
/*
* If not NULL, load new instruction array.
@@ -494,6 +615,18 @@ LIBBPF_API void
bpf_program__set_expected_attach_type(struct bpf_program *prog,
enum bpf_attach_type type);
+LIBBPF_API __u32 bpf_program__flags(const struct bpf_program *prog);
+LIBBPF_API int bpf_program__set_flags(struct bpf_program *prog, __u32 flags);
+
+/* Per-program log level and log buffer getters/setters.
+ * See bpf_object_open_opts comments regarding log_level and log_buf
+ * interactions.
+ */
+LIBBPF_API __u32 bpf_program__log_level(const struct bpf_program *prog);
+LIBBPF_API int bpf_program__set_log_level(struct bpf_program *prog, __u32 log_level);
+LIBBPF_API const char *bpf_program__log_buf(const struct bpf_program *prog, size_t *log_size);
+LIBBPF_API int bpf_program__set_log_buf(struct bpf_program *prog, char *log_buf, size_t log_size);
+
LIBBPF_API int
bpf_program__set_attach_target(struct bpf_program *prog, int attach_prog_fd,
const char *attach_func_name);
@@ -544,7 +677,8 @@ bpf_object__find_map_fd_by_name(const struct bpf_object *obj, const char *name);
* Get bpf_map through the offset of corresponding struct bpf_map_def
* in the BPF object file.
*/
-LIBBPF_API struct bpf_map *
+LIBBPF_API LIBBPF_DEPRECATED_SINCE(0, 8, "use bpf_object__find_map_by_name() instead")
+struct bpf_map *
bpf_object__find_map_by_offset(struct bpf_object *obj, size_t offset);
LIBBPF_API LIBBPF_DEPRECATED_SINCE(0, 7, "use bpf_object__next_map() instead")
@@ -611,6 +745,7 @@ LIBBPF_API void *bpf_map__priv(const struct bpf_map *map);
LIBBPF_API int bpf_map__set_initial_value(struct bpf_map *map,
const void *data, size_t size);
LIBBPF_API const void *bpf_map__initial_value(struct bpf_map *map, size_t *psize);
+LIBBPF_DEPRECATED_SINCE(0, 8, "use bpf_map__type() instead")
LIBBPF_API bool bpf_map__is_offload_neutral(const struct bpf_map *map);
/**
@@ -674,10 +809,12 @@ struct bpf_prog_load_attr {
int prog_flags;
};
+LIBBPF_DEPRECATED_SINCE(0, 8, "use bpf_object__open() and bpf_object__load() instead")
LIBBPF_API int bpf_prog_load_xattr(const struct bpf_prog_load_attr *attr,
struct bpf_object **pobj, int *prog_fd);
-LIBBPF_API int bpf_prog_load(const char *file, enum bpf_prog_type type,
- struct bpf_object **pobj, int *prog_fd);
+LIBBPF_DEPRECATED_SINCE(0, 7, "use bpf_object__open() and bpf_object__load() instead")
+LIBBPF_API int bpf_prog_load_deprecated(const char *file, enum bpf_prog_type type,
+ struct bpf_object **pobj, int *prog_fd);
/* XDP related API */
struct xdp_link_info {
@@ -775,18 +912,52 @@ typedef void (*perf_buffer_lost_fn)(void *ctx, int cpu, __u64 cnt);
/* common use perf buffer options */
struct perf_buffer_opts {
- /* if specified, sample_cb is called for each sample */
- perf_buffer_sample_fn sample_cb;
- /* if specified, lost_cb is called for each batch of lost samples */
- perf_buffer_lost_fn lost_cb;
- /* ctx is provided to sample_cb and lost_cb */
- void *ctx;
+ union {
+ size_t sz;
+ struct { /* DEPRECATED: will be removed in v1.0 */
+ /* if specified, sample_cb is called for each sample */
+ perf_buffer_sample_fn sample_cb;
+ /* if specified, lost_cb is called for each batch of lost samples */
+ perf_buffer_lost_fn lost_cb;
+ /* ctx is provided to sample_cb and lost_cb */
+ void *ctx;
+ };
+ };
};
+#define perf_buffer_opts__last_field sz
+/**
+ * @brief **perf_buffer__new()** creates BPF perfbuf manager for a specified
+ * BPF_PERF_EVENT_ARRAY map
+ * @param map_fd FD of BPF_PERF_EVENT_ARRAY BPF map that will be used by BPF
+ * code to send data over to user-space
+ * @param page_cnt number of memory pages allocated for each per-CPU buffer
+ * @param sample_cb function called on each received data record
+ * @param lost_cb function called when record loss has occurred
+ * @param ctx user-provided extra context passed into *sample_cb* and *lost_cb*
+ * @return a new instance of struct perf_buffer on success, NULL on error with
+ * *errno* containing an error code
+ */
LIBBPF_API struct perf_buffer *
perf_buffer__new(int map_fd, size_t page_cnt,
+ perf_buffer_sample_fn sample_cb, perf_buffer_lost_fn lost_cb, void *ctx,
const struct perf_buffer_opts *opts);
+LIBBPF_API struct perf_buffer *
+perf_buffer__new_v0_6_0(int map_fd, size_t page_cnt,
+ perf_buffer_sample_fn sample_cb, perf_buffer_lost_fn lost_cb, void *ctx,
+ const struct perf_buffer_opts *opts);
+
+LIBBPF_API LIBBPF_DEPRECATED_SINCE(0, 7, "use new variant of perf_buffer__new() instead")
+struct perf_buffer *perf_buffer__new_deprecated(int map_fd, size_t page_cnt,
+ const struct perf_buffer_opts *opts);
+
+#define perf_buffer__new(...) ___libbpf_overload(___perf_buffer_new, __VA_ARGS__)
+#define ___perf_buffer_new6(map_fd, page_cnt, sample_cb, lost_cb, ctx, opts) \
+ perf_buffer__new(map_fd, page_cnt, sample_cb, lost_cb, ctx, opts)
+#define ___perf_buffer_new3(map_fd, page_cnt, opts) \
+ perf_buffer__new_deprecated(map_fd, page_cnt, opts)
+
enum bpf_perf_event_ret {
LIBBPF_PERF_EVENT_DONE = 0,
LIBBPF_PERF_EVENT_ERROR = -1,
@@ -800,12 +971,21 @@ typedef enum bpf_perf_event_ret
/* raw perf buffer options, giving most power and control */
struct perf_buffer_raw_opts {
- /* perf event attrs passed directly into perf_event_open() */
- struct perf_event_attr *attr;
- /* raw event callback */
- perf_buffer_event_fn event_cb;
- /* ctx is provided to event_cb */
- void *ctx;
+ union {
+ struct {
+ size_t sz;
+ long :0;
+ long :0;
+ };
+ struct { /* DEPRECATED: will be removed in v1.0 */
+ /* perf event attrs passed directly into perf_event_open() */
+ struct perf_event_attr *attr;
+ /* raw event callback */
+ perf_buffer_event_fn event_cb;
+ /* ctx is provided to event_cb */
+ void *ctx;
+ };
+ };
/* if cpu_cnt == 0, open all on all possible CPUs (up to the number of
* max_entries of given PERF_EVENT_ARRAY map)
*/
@@ -815,11 +995,28 @@ struct perf_buffer_raw_opts {
/* if cpu_cnt > 0, map_keys specify map keys to set per-CPU FDs for */
int *map_keys;
};
+#define perf_buffer_raw_opts__last_field map_keys
LIBBPF_API struct perf_buffer *
-perf_buffer__new_raw(int map_fd, size_t page_cnt,
+perf_buffer__new_raw(int map_fd, size_t page_cnt, struct perf_event_attr *attr,
+ perf_buffer_event_fn event_cb, void *ctx,
const struct perf_buffer_raw_opts *opts);
+LIBBPF_API struct perf_buffer *
+perf_buffer__new_raw_v0_6_0(int map_fd, size_t page_cnt, struct perf_event_attr *attr,
+ perf_buffer_event_fn event_cb, void *ctx,
+ const struct perf_buffer_raw_opts *opts);
+
+LIBBPF_API LIBBPF_DEPRECATED_SINCE(0, 7, "use new variant of perf_buffer__new_raw() instead")
+struct perf_buffer *perf_buffer__new_raw_deprecated(int map_fd, size_t page_cnt,
+ const struct perf_buffer_raw_opts *opts);
+
+#define perf_buffer__new_raw(...) ___libbpf_overload(___perf_buffer_new_raw, __VA_ARGS__)
+#define ___perf_buffer_new_raw6(map_fd, page_cnt, attr, event_cb, ctx, opts) \
+ perf_buffer__new_raw(map_fd, page_cnt, attr, event_cb, ctx, opts)
+#define ___perf_buffer_new_raw3(map_fd, page_cnt, opts) \
+ perf_buffer__new_raw_deprecated(map_fd, page_cnt, opts)
+
LIBBPF_API void perf_buffer__free(struct perf_buffer *pb);
LIBBPF_API int perf_buffer__epoll_fd(const struct perf_buffer *pb);
LIBBPF_API int perf_buffer__poll(struct perf_buffer *pb, int timeout_ms);
@@ -831,6 +1028,7 @@ LIBBPF_API int perf_buffer__buffer_fd(const struct perf_buffer *pb, size_t buf_i
typedef enum bpf_perf_event_ret
(*bpf_perf_event_print_t)(struct perf_event_header *hdr,
void *private_data);
+LIBBPF_DEPRECATED_SINCE(0, 8, "use perf_buffer__poll() or perf_buffer__consume() instead")
LIBBPF_API enum bpf_perf_event_ret
bpf_perf_event_read_simple(void *mmap_mem, size_t mmap_size, size_t page_size,
void **copy_mem, size_t *copy_size,
@@ -857,13 +1055,57 @@ bpf_prog_linfo__lfind(const struct bpf_prog_linfo *prog_linfo,
* user, causing subsequent probes to fail. In this case, the caller may want
* to adjust that limit with setrlimit().
*/
-LIBBPF_API bool bpf_probe_prog_type(enum bpf_prog_type prog_type,
- __u32 ifindex);
+LIBBPF_DEPRECATED_SINCE(0, 8, "use libbpf_probe_bpf_prog_type() instead")
+LIBBPF_API bool bpf_probe_prog_type(enum bpf_prog_type prog_type, __u32 ifindex);
+LIBBPF_DEPRECATED_SINCE(0, 8, "use libbpf_probe_bpf_map_type() instead")
LIBBPF_API bool bpf_probe_map_type(enum bpf_map_type map_type, __u32 ifindex);
-LIBBPF_API bool bpf_probe_helper(enum bpf_func_id id,
- enum bpf_prog_type prog_type, __u32 ifindex);
+LIBBPF_DEPRECATED_SINCE(0, 8, "use libbpf_probe_bpf_helper() instead")
+LIBBPF_API bool bpf_probe_helper(enum bpf_func_id id, enum bpf_prog_type prog_type, __u32 ifindex);
+LIBBPF_DEPRECATED_SINCE(0, 8, "implement your own or use bpftool for feature detection")
LIBBPF_API bool bpf_probe_large_insn_limit(__u32 ifindex);
+/**
+ * @brief **libbpf_probe_bpf_prog_type()** detects if host kernel supports
+ * BPF programs of a given type.
+ * @param prog_type BPF program type to detect kernel support for
+ * @param opts reserved for future extensibility, should be NULL
+ * @return 1, if given program type is supported; 0, if given program type is
+ * not supported; negative error code if feature detection failed or can't be
+ * performed
+ *
+ * Make sure the process has required set of CAP_* permissions (or runs as
+ * root) when performing feature checking.
+ */
+LIBBPF_API int libbpf_probe_bpf_prog_type(enum bpf_prog_type prog_type, const void *opts);
+/**
+ * @brief **libbpf_probe_bpf_map_type()** detects if host kernel supports
+ * BPF maps of a given type.
+ * @param map_type BPF map type to detect kernel support for
+ * @param opts reserved for future extensibility, should be NULL
+ * @return 1, if given map type is supported; 0, if given map type is
+ * not supported; negative error code if feature detection failed or can't be
+ * performed
+ *
+ * Make sure the process has required set of CAP_* permissions (or runs as
+ * root) when performing feature checking.
+ */
+LIBBPF_API int libbpf_probe_bpf_map_type(enum bpf_map_type map_type, const void *opts);
+/**
+ * @brief **libbpf_probe_bpf_helper()** detects if host kernel supports the
+ * use of a given BPF helper from specified BPF program type.
+ * @param prog_type BPF program type used to check the support of BPF helper
+ * @param helper_id BPF helper ID (enum bpf_func_id) to check support for
+ * @param opts reserved for future extensibility, should be NULL
+ * @return 1, if given combination of program type and helper is supported; 0,
+ * if the combination is not supported; negative error code if feature
+ * detection for provided input arguments failed or can't be performed
+ *
+ * Make sure the process has required set of CAP_* permissions (or runs as
+ * root) when performing feature checking.
+ */
+LIBBPF_API int libbpf_probe_bpf_helper(enum bpf_prog_type prog_type,
+ enum bpf_func_id helper_id, const void *opts);
+
/*
* Get bpf_prog_info in continuous memory
*
@@ -918,12 +1160,15 @@ struct bpf_prog_info_linear {
__u8 data[];
};
+LIBBPF_DEPRECATED_SINCE(0, 6, "use a custom linear prog_info wrapper")
LIBBPF_API struct bpf_prog_info_linear *
bpf_program__get_prog_info_linear(int fd, __u64 arrays);
+LIBBPF_DEPRECATED_SINCE(0, 6, "use a custom linear prog_info wrapper")
LIBBPF_API void
bpf_program__bpil_addr_to_offs(struct bpf_prog_info_linear *info_linear);
+LIBBPF_DEPRECATED_SINCE(0, 6, "use a custom linear prog_info wrapper")
LIBBPF_API void
bpf_program__bpil_offs_to_addr(struct bpf_prog_info_linear *info_linear);
@@ -965,11 +1210,11 @@ struct bpf_object_skeleton {
struct bpf_object **obj;
int map_cnt;
- int map_skel_sz; /* sizeof(struct bpf_skeleton_map) */
+ int map_skel_sz; /* sizeof(struct bpf_map_skeleton) */
struct bpf_map_skeleton *maps;
int prog_cnt;
- int prog_skel_sz; /* sizeof(struct bpf_skeleton_prog) */
+ int prog_skel_sz; /* sizeof(struct bpf_prog_skeleton) */
struct bpf_prog_skeleton *progs;
};
diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map
index 43580eb..5297839 100644
--- a/tools/lib/bpf/libbpf.map
+++ b/tools/lib/bpf/libbpf.map
@@ -391,14 +391,44 @@
global:
bpf_map__map_extra;
bpf_map__set_map_extra;
+ bpf_map_create;
bpf_object__next_map;
bpf_object__next_program;
bpf_object__prev_map;
bpf_object__prev_program;
+ bpf_prog_load_deprecated;
+ bpf_prog_load;
+ bpf_program__flags;
bpf_program__insn_cnt;
bpf_program__insns;
+ bpf_program__set_flags;
btf__add_btf;
btf__add_decl_tag;
+ btf__add_type_tag;
+ btf__dedup;
+ btf__dedup_deprecated;
btf__raw_data;
btf__type_cnt;
+ btf_dump__new;
+ btf_dump__new_deprecated;
+ libbpf_major_version;
+ libbpf_minor_version;
+ libbpf_version_string;
+ perf_buffer__new;
+ perf_buffer__new_deprecated;
+ perf_buffer__new_raw;
+ perf_buffer__new_raw_deprecated;
} LIBBPF_0.5.0;
+
+LIBBPF_0.7.0 {
+ global:
+ bpf_btf_load;
+ bpf_program__log_buf;
+ bpf_program__log_level;
+ bpf_program__set_log_buf;
+ bpf_program__set_log_level;
+ libbpf_probe_bpf_helper;
+ libbpf_probe_bpf_map_type;
+ libbpf_probe_bpf_prog_type;
+ libbpf_set_memlock_rlim_max;
+};
diff --git a/tools/lib/bpf/libbpf_common.h b/tools/lib/bpf/libbpf_common.h
index aaa1efbf..000e377 100644
--- a/tools/lib/bpf/libbpf_common.h
+++ b/tools/lib/bpf/libbpf_common.h
@@ -40,6 +40,23 @@
#else
#define __LIBBPF_MARK_DEPRECATED_0_7(X)
#endif
+#if __LIBBPF_CURRENT_VERSION_GEQ(0, 8)
+#define __LIBBPF_MARK_DEPRECATED_0_8(X) X
+#else
+#define __LIBBPF_MARK_DEPRECATED_0_8(X)
+#endif
+
+/* This set of internal macros allows to do "function overloading" based on
+ * number of arguments provided by used in backwards-compatible way during the
+ * transition to libbpf 1.0
+ * It's ugly but necessary evil that will be cleaned up when we get to 1.0.
+ * See bpf_prog_load() overload for example.
+ */
+#define ___libbpf_cat(A, B) A ## B
+#define ___libbpf_select(NAME, NUM) ___libbpf_cat(NAME, NUM)
+#define ___libbpf_nth(_1, _2, _3, _4, _5, _6, N, ...) N
+#define ___libbpf_cnt(...) ___libbpf_nth(__VA_ARGS__, 6, 5, 4, 3, 2, 1)
+#define ___libbpf_overload(NAME, ...) ___libbpf_select(NAME, ___libbpf_cnt(__VA_ARGS__))(__VA_ARGS__)
/* Helper macro to declare and initialize libbpf options struct
*
@@ -54,7 +71,7 @@
* including any extra padding, it with memset() and then assigns initial
* values provided by users in struct initializer-syntax as varargs.
*/
-#define DECLARE_LIBBPF_OPTS(TYPE, NAME, ...) \
+#define LIBBPF_OPTS(TYPE, NAME, ...) \
struct TYPE NAME = ({ \
memset(&NAME, 0, sizeof(struct TYPE)); \
(struct TYPE) { \
diff --git a/tools/lib/bpf/libbpf_internal.h b/tools/lib/bpf/libbpf_internal.h
index aeb79e3..1565679 100644
--- a/tools/lib/bpf/libbpf_internal.h
+++ b/tools/lib/bpf/libbpf_internal.h
@@ -73,6 +73,8 @@
BTF_TYPE_ENC(name, BTF_INFO_ENC(BTF_KIND_FLOAT, 0, 0), sz)
#define BTF_TYPE_DECL_TAG_ENC(value, type, component_idx) \
BTF_TYPE_ENC(value, BTF_INFO_ENC(BTF_KIND_DECL_TAG, 0, 0), type), (component_idx)
+#define BTF_TYPE_TYPE_TAG_ENC(value, type) \
+ BTF_TYPE_ENC(value, BTF_INFO_ENC(BTF_KIND_TYPE_TAG, 0, 0), type)
#ifndef likely
#define likely(x) __builtin_expect(!!(x), 1)
@@ -167,10 +169,31 @@ static inline void *libbpf_reallocarray(void *ptr, size_t nmemb, size_t size)
return realloc(ptr, total);
}
+/* Copy up to sz - 1 bytes from zero-terminated src string and ensure that dst
+ * is zero-terminated string no matter what (unless sz == 0, in which case
+ * it's a no-op). It's conceptually close to FreeBSD's strlcpy(), but differs
+ * in what is returned. Given this is internal helper, it's trivial to extend
+ * this, when necessary. Use this instead of strncpy inside libbpf source code.
+ */
+static inline void libbpf_strlcpy(char *dst, const char *src, size_t sz)
+{
+ size_t i;
+
+ if (sz == 0)
+ return;
+
+ sz--;
+ for (i = 0; i < sz && src[i]; i++)
+ dst[i] = src[i];
+ dst[i] = '\0';
+}
+
+__u32 get_kernel_version(void);
+
struct btf;
struct btf_type;
-struct btf_type *btf_type_by_id(struct btf *btf, __u32 type_id);
+struct btf_type *btf_type_by_id(const struct btf *btf, __u32 type_id);
const char *btf_kind_str(const struct btf_type *t);
const struct btf_type *skip_mods_and_typedefs(const struct btf *btf, __u32 id, __u32 *res_id);
@@ -270,63 +293,51 @@ static inline bool libbpf_validate_opts(const char *opts,
(opts)->sz - __off); \
})
+enum kern_feature_id {
+ /* v4.14: kernel support for program & map names. */
+ FEAT_PROG_NAME,
+ /* v5.2: kernel support for global data sections. */
+ FEAT_GLOBAL_DATA,
+ /* BTF support */
+ FEAT_BTF,
+ /* BTF_KIND_FUNC and BTF_KIND_FUNC_PROTO support */
+ FEAT_BTF_FUNC,
+ /* BTF_KIND_VAR and BTF_KIND_DATASEC support */
+ FEAT_BTF_DATASEC,
+ /* BTF_FUNC_GLOBAL is supported */
+ FEAT_BTF_GLOBAL_FUNC,
+ /* BPF_F_MMAPABLE is supported for arrays */
+ FEAT_ARRAY_MMAP,
+ /* kernel support for expected_attach_type in BPF_PROG_LOAD */
+ FEAT_EXP_ATTACH_TYPE,
+ /* bpf_probe_read_{kernel,user}[_str] helpers */
+ FEAT_PROBE_READ_KERN,
+ /* BPF_PROG_BIND_MAP is supported */
+ FEAT_PROG_BIND_MAP,
+ /* Kernel support for module BTFs */
+ FEAT_MODULE_BTF,
+ /* BTF_KIND_FLOAT support */
+ FEAT_BTF_FLOAT,
+ /* BPF perf link support */
+ FEAT_PERF_LINK,
+ /* BTF_KIND_DECL_TAG support */
+ FEAT_BTF_DECL_TAG,
+ /* BTF_KIND_TYPE_TAG support */
+ FEAT_BTF_TYPE_TAG,
+ /* memcg-based accounting for BPF maps and progs */
+ FEAT_MEMCG_ACCOUNT,
+ __FEAT_CNT,
+};
+
+int probe_memcg_account(void);
+bool kernel_supports(const struct bpf_object *obj, enum kern_feature_id feat_id);
+int bump_rlimit_memlock(void);
int parse_cpu_mask_str(const char *s, bool **mask, int *mask_sz);
int parse_cpu_mask_file(const char *fcpu, bool **mask, int *mask_sz);
int libbpf__load_raw_btf(const char *raw_types, size_t types_len,
const char *str_sec, size_t str_len);
-
-struct bpf_prog_load_params {
- enum bpf_prog_type prog_type;
- enum bpf_attach_type expected_attach_type;
- const char *name;
- const struct bpf_insn *insns;
- size_t insn_cnt;
- const char *license;
- __u32 kern_version;
- __u32 attach_prog_fd;
- __u32 attach_btf_obj_fd;
- __u32 attach_btf_id;
- __u32 prog_ifindex;
- __u32 prog_btf_fd;
- __u32 prog_flags;
-
- __u32 func_info_rec_size;
- const void *func_info;
- __u32 func_info_cnt;
-
- __u32 line_info_rec_size;
- const void *line_info;
- __u32 line_info_cnt;
-
- __u32 log_level;
- char *log_buf;
- size_t log_buf_sz;
- int *fd_array;
-};
-
-int libbpf__bpf_prog_load(const struct bpf_prog_load_params *load_attr);
-
-struct bpf_create_map_params {
- const char *name;
- enum bpf_map_type map_type;
- __u32 map_flags;
- __u32 key_size;
- __u32 value_size;
- __u32 max_entries;
- __u32 numa_node;
- __u32 btf_fd;
- __u32 btf_key_type_id;
- __u32 btf_value_type_id;
- __u32 map_ifindex;
- union {
- __u32 inner_map_fd;
- __u32 btf_vmlinux_value_type_id;
- };
- __u64 map_extra;
-};
-
-int libbpf__bpf_create_map_xattr(const struct bpf_create_map_params *create_attr);
+int btf_load_into_kernel(struct btf *btf, char *log_buf, size_t log_sz, __u32 log_level);
struct btf *btf_get_from_fd(int btf_fd, struct btf *base_btf);
void btf_get_kernel_prefix_kind(enum bpf_attach_type attach_type,
diff --git a/tools/lib/bpf/libbpf_legacy.h b/tools/lib/bpf/libbpf_legacy.h
index 5ba5c9b..79131f76 100644
--- a/tools/lib/bpf/libbpf_legacy.h
+++ b/tools/lib/bpf/libbpf_legacy.h
@@ -45,7 +45,6 @@ enum libbpf_strict_mode {
* (positive) error code.
*/
LIBBPF_STRICT_DIRECT_ERRS = 0x02,
-
/*
* Enforce strict BPF program section (SEC()) names.
* E.g., while prefiously SEC("xdp_whatever") or SEC("perf_event_blah") were
@@ -63,12 +62,24 @@ enum libbpf_strict_mode {
* Clients can maintain it on their own if it is valuable for them.
*/
LIBBPF_STRICT_NO_OBJECT_LIST = 0x08,
+ /*
+ * Automatically bump RLIMIT_MEMLOCK using setrlimit() before the
+ * first BPF program or map creation operation. This is done only if
+ * kernel is too old to support memcg-based memory accounting for BPF
+ * subsystem. By default, RLIMIT_MEMLOCK limit is set to RLIM_INFINITY,
+ * but it can be overriden with libbpf_set_memlock_rlim_max() API.
+ * Note that libbpf_set_memlock_rlim_max() needs to be called before
+ * the very first bpf_prog_load(), bpf_map_create() or bpf_object__load()
+ * operation.
+ */
+ LIBBPF_STRICT_AUTO_RLIMIT_MEMLOCK = 0x10,
__LIBBPF_STRICT_LAST,
};
LIBBPF_API int libbpf_set_strict_mode(enum libbpf_strict_mode mode);
+#define DECLARE_LIBBPF_OPTS LIBBPF_OPTS
#ifdef __cplusplus
} /* extern "C" */
diff --git a/tools/lib/bpf/libbpf_probes.c b/tools/lib/bpf/libbpf_probes.c
index 68f2dbf..97b06ce 100644
--- a/tools/lib/bpf/libbpf_probes.c
+++ b/tools/lib/bpf/libbpf_probes.c
@@ -48,41 +48,65 @@ static int get_vendor_id(int ifindex)
return strtol(buf, NULL, 0);
}
-static int get_kernel_version(void)
+static int probe_prog_load(enum bpf_prog_type prog_type,
+ const struct bpf_insn *insns, size_t insns_cnt,
+ char *log_buf, size_t log_buf_sz,
+ __u32 ifindex)
{
- int version, subversion, patchlevel;
- struct utsname utsn;
-
- /* Return 0 on failure, and attempt to probe with empty kversion */
- if (uname(&utsn))
- return 0;
-
- if (sscanf(utsn.release, "%d.%d.%d",
- &version, &subversion, &patchlevel) != 3)
- return 0;
-
- return (version << 16) + (subversion << 8) + patchlevel;
-}
-
-static void
-probe_load(enum bpf_prog_type prog_type, const struct bpf_insn *insns,
- size_t insns_cnt, char *buf, size_t buf_len, __u32 ifindex)
-{
- struct bpf_load_program_attr xattr = {};
- int fd;
+ LIBBPF_OPTS(bpf_prog_load_opts, opts,
+ .log_buf = log_buf,
+ .log_size = log_buf_sz,
+ .log_level = log_buf ? 1 : 0,
+ .prog_ifindex = ifindex,
+ );
+ int fd, err, exp_err = 0;
+ const char *exp_msg = NULL;
+ char buf[4096];
switch (prog_type) {
case BPF_PROG_TYPE_CGROUP_SOCK_ADDR:
- xattr.expected_attach_type = BPF_CGROUP_INET4_CONNECT;
+ opts.expected_attach_type = BPF_CGROUP_INET4_CONNECT;
break;
case BPF_PROG_TYPE_CGROUP_SOCKOPT:
- xattr.expected_attach_type = BPF_CGROUP_GETSOCKOPT;
+ opts.expected_attach_type = BPF_CGROUP_GETSOCKOPT;
break;
case BPF_PROG_TYPE_SK_LOOKUP:
- xattr.expected_attach_type = BPF_SK_LOOKUP;
+ opts.expected_attach_type = BPF_SK_LOOKUP;
break;
case BPF_PROG_TYPE_KPROBE:
- xattr.kern_version = get_kernel_version();
+ opts.kern_version = get_kernel_version();
+ break;
+ case BPF_PROG_TYPE_LIRC_MODE2:
+ opts.expected_attach_type = BPF_LIRC_MODE2;
+ break;
+ case BPF_PROG_TYPE_TRACING:
+ case BPF_PROG_TYPE_LSM:
+ opts.log_buf = buf;
+ opts.log_size = sizeof(buf);
+ opts.log_level = 1;
+ if (prog_type == BPF_PROG_TYPE_TRACING)
+ opts.expected_attach_type = BPF_TRACE_FENTRY;
+ else
+ opts.expected_attach_type = BPF_MODIFY_RETURN;
+ opts.attach_btf_id = 1;
+
+ exp_err = -EINVAL;
+ exp_msg = "attach_btf_id 1 is not a function";
+ break;
+ case BPF_PROG_TYPE_EXT:
+ opts.log_buf = buf;
+ opts.log_size = sizeof(buf);
+ opts.log_level = 1;
+ opts.attach_btf_id = 1;
+
+ exp_err = -EINVAL;
+ exp_msg = "Cannot replace kernel functions";
+ break;
+ case BPF_PROG_TYPE_SYSCALL:
+ opts.prog_flags = BPF_F_SLEEPABLE;
+ break;
+ case BPF_PROG_TYPE_STRUCT_OPS:
+ exp_err = -524; /* -ENOTSUPP */
break;
case BPF_PROG_TYPE_UNSPEC:
case BPF_PROG_TYPE_SOCKET_FILTER:
@@ -103,27 +127,42 @@ probe_load(enum bpf_prog_type prog_type, const struct bpf_insn *insns,
case BPF_PROG_TYPE_RAW_TRACEPOINT:
case BPF_PROG_TYPE_RAW_TRACEPOINT_WRITABLE:
case BPF_PROG_TYPE_LWT_SEG6LOCAL:
- case BPF_PROG_TYPE_LIRC_MODE2:
case BPF_PROG_TYPE_SK_REUSEPORT:
case BPF_PROG_TYPE_FLOW_DISSECTOR:
case BPF_PROG_TYPE_CGROUP_SYSCTL:
- case BPF_PROG_TYPE_TRACING:
- case BPF_PROG_TYPE_STRUCT_OPS:
- case BPF_PROG_TYPE_EXT:
- case BPF_PROG_TYPE_LSM:
- default:
break;
+ default:
+ return -EOPNOTSUPP;
}
- xattr.prog_type = prog_type;
- xattr.insns = insns;
- xattr.insns_cnt = insns_cnt;
- xattr.license = "GPL";
- xattr.prog_ifindex = ifindex;
-
- fd = bpf_load_program_xattr(&xattr, buf, buf_len);
+ fd = bpf_prog_load(prog_type, NULL, "GPL", insns, insns_cnt, &opts);
+ err = -errno;
if (fd >= 0)
close(fd);
+ if (exp_err) {
+ if (fd >= 0 || err != exp_err)
+ return 0;
+ if (exp_msg && !strstr(buf, exp_msg))
+ return 0;
+ return 1;
+ }
+ return fd >= 0 ? 1 : 0;
+}
+
+int libbpf_probe_bpf_prog_type(enum bpf_prog_type prog_type, const void *opts)
+{
+ struct bpf_insn insns[] = {
+ BPF_MOV64_IMM(BPF_REG_0, 0),
+ BPF_EXIT_INSN()
+ };
+ const size_t insn_cnt = ARRAY_SIZE(insns);
+ int ret;
+
+ if (opts)
+ return libbpf_err(-EINVAL);
+
+ ret = probe_prog_load(prog_type, insns, insn_cnt, NULL, 0, 0);
+ return libbpf_err(ret);
}
bool bpf_probe_prog_type(enum bpf_prog_type prog_type, __u32 ifindex)
@@ -133,12 +172,16 @@ bool bpf_probe_prog_type(enum bpf_prog_type prog_type, __u32 ifindex)
BPF_EXIT_INSN()
};
+ /* prefer libbpf_probe_bpf_prog_type() unless offload is requested */
+ if (ifindex == 0)
+ return libbpf_probe_bpf_prog_type(prog_type, NULL) == 1;
+
if (ifindex && prog_type == BPF_PROG_TYPE_SCHED_CLS)
/* nfp returns -EINVAL on exit(0) with TC offload */
insns[0].imm = 2;
errno = 0;
- probe_load(prog_type, insns, ARRAY_SIZE(insns), NULL, 0, ifindex);
+ probe_prog_load(prog_type, insns, ARRAY_SIZE(insns), NULL, 0, ifindex);
return errno != EINVAL && errno != EOPNOTSUPP;
}
@@ -166,7 +209,7 @@ int libbpf__load_raw_btf(const char *raw_types, size_t types_len,
memcpy(raw_btf + hdr.hdr_len, raw_types, hdr.type_len);
memcpy(raw_btf + hdr.hdr_len + hdr.type_len, str_sec, hdr.str_len);
- btf_fd = bpf_load_btf(raw_btf, btf_len, NULL, 0, false);
+ btf_fd = bpf_btf_load(raw_btf, btf_len, NULL);
free(raw_btf);
return btf_fd;
@@ -199,17 +242,18 @@ static int load_local_storage_btf(void)
strs, sizeof(strs));
}
-bool bpf_probe_map_type(enum bpf_map_type map_type, __u32 ifindex)
+static int probe_map_create(enum bpf_map_type map_type, __u32 ifindex)
{
- int key_size, value_size, max_entries, map_flags;
+ LIBBPF_OPTS(bpf_map_create_opts, opts);
+ int key_size, value_size, max_entries;
__u32 btf_key_type_id = 0, btf_value_type_id = 0;
- struct bpf_create_map_attr attr = {};
- int fd = -1, btf_fd = -1, fd_inner;
+ int fd = -1, btf_fd = -1, fd_inner = -1, exp_err = 0, err;
+
+ opts.map_ifindex = ifindex;
key_size = sizeof(__u32);
value_size = sizeof(__u32);
max_entries = 1;
- map_flags = 0;
switch (map_type) {
case BPF_MAP_TYPE_STACK_TRACE:
@@ -218,7 +262,7 @@ bool bpf_probe_map_type(enum bpf_map_type map_type, __u32 ifindex)
case BPF_MAP_TYPE_LPM_TRIE:
key_size = sizeof(__u64);
value_size = sizeof(__u64);
- map_flags = BPF_F_NO_PREALLOC;
+ opts.map_flags = BPF_F_NO_PREALLOC;
break;
case BPF_MAP_TYPE_CGROUP_STORAGE:
case BPF_MAP_TYPE_PERCPU_CGROUP_STORAGE:
@@ -237,17 +281,25 @@ bool bpf_probe_map_type(enum bpf_map_type map_type, __u32 ifindex)
btf_value_type_id = 3;
value_size = 8;
max_entries = 0;
- map_flags = BPF_F_NO_PREALLOC;
+ opts.map_flags = BPF_F_NO_PREALLOC;
btf_fd = load_local_storage_btf();
if (btf_fd < 0)
- return false;
+ return btf_fd;
break;
case BPF_MAP_TYPE_RINGBUF:
key_size = 0;
value_size = 0;
max_entries = 4096;
break;
- case BPF_MAP_TYPE_UNSPEC:
+ case BPF_MAP_TYPE_STRUCT_OPS:
+ /* we'll get -ENOTSUPP for invalid BTF type ID for struct_ops */
+ opts.btf_vmlinux_value_type_id = 1;
+ exp_err = -524; /* -ENOTSUPP */
+ break;
+ case BPF_MAP_TYPE_BLOOM_FILTER:
+ key_size = 0;
+ max_entries = 1;
+ break;
case BPF_MAP_TYPE_HASH:
case BPF_MAP_TYPE_ARRAY:
case BPF_MAP_TYPE_PROG_ARRAY:
@@ -266,9 +318,10 @@ bool bpf_probe_map_type(enum bpf_map_type map_type, __u32 ifindex)
case BPF_MAP_TYPE_XSKMAP:
case BPF_MAP_TYPE_SOCKHASH:
case BPF_MAP_TYPE_REUSEPORT_SOCKARRAY:
- case BPF_MAP_TYPE_STRUCT_OPS:
- default:
break;
+ case BPF_MAP_TYPE_UNSPEC:
+ default:
+ return -EOPNOTSUPP;
}
if (map_type == BPF_MAP_TYPE_ARRAY_OF_MAPS ||
@@ -277,37 +330,102 @@ bool bpf_probe_map_type(enum bpf_map_type map_type, __u32 ifindex)
* map-in-map for offload
*/
if (ifindex)
- return false;
+ goto cleanup;
- fd_inner = bpf_create_map(BPF_MAP_TYPE_HASH,
- sizeof(__u32), sizeof(__u32), 1, 0);
+ fd_inner = bpf_map_create(BPF_MAP_TYPE_HASH, NULL,
+ sizeof(__u32), sizeof(__u32), 1, NULL);
if (fd_inner < 0)
- return false;
- fd = bpf_create_map_in_map(map_type, NULL, sizeof(__u32),
- fd_inner, 1, 0);
- close(fd_inner);
- } else {
- /* Note: No other restriction on map type probes for offload */
- attr.map_type = map_type;
- attr.key_size = key_size;
- attr.value_size = value_size;
- attr.max_entries = max_entries;
- attr.map_flags = map_flags;
- attr.map_ifindex = ifindex;
- if (btf_fd >= 0) {
- attr.btf_fd = btf_fd;
- attr.btf_key_type_id = btf_key_type_id;
- attr.btf_value_type_id = btf_value_type_id;
- }
+ goto cleanup;
- fd = bpf_create_map_xattr(&attr);
+ opts.inner_map_fd = fd_inner;
}
+
+ if (btf_fd >= 0) {
+ opts.btf_fd = btf_fd;
+ opts.btf_key_type_id = btf_key_type_id;
+ opts.btf_value_type_id = btf_value_type_id;
+ }
+
+ fd = bpf_map_create(map_type, NULL, key_size, value_size, max_entries, &opts);
+ err = -errno;
+
+cleanup:
if (fd >= 0)
close(fd);
+ if (fd_inner >= 0)
+ close(fd_inner);
if (btf_fd >= 0)
close(btf_fd);
- return fd >= 0;
+ if (exp_err)
+ return fd < 0 && err == exp_err ? 1 : 0;
+ else
+ return fd >= 0 ? 1 : 0;
+}
+
+int libbpf_probe_bpf_map_type(enum bpf_map_type map_type, const void *opts)
+{
+ int ret;
+
+ if (opts)
+ return libbpf_err(-EINVAL);
+
+ ret = probe_map_create(map_type, 0);
+ return libbpf_err(ret);
+}
+
+bool bpf_probe_map_type(enum bpf_map_type map_type, __u32 ifindex)
+{
+ return probe_map_create(map_type, ifindex) == 1;
+}
+
+int libbpf_probe_bpf_helper(enum bpf_prog_type prog_type, enum bpf_func_id helper_id,
+ const void *opts)
+{
+ struct bpf_insn insns[] = {
+ BPF_EMIT_CALL((__u32)helper_id),
+ BPF_EXIT_INSN(),
+ };
+ const size_t insn_cnt = ARRAY_SIZE(insns);
+ char buf[4096];
+ int ret;
+
+ if (opts)
+ return libbpf_err(-EINVAL);
+
+ /* we can't successfully load all prog types to check for BPF helper
+ * support, so bail out with -EOPNOTSUPP error
+ */
+ switch (prog_type) {
+ case BPF_PROG_TYPE_TRACING:
+ case BPF_PROG_TYPE_EXT:
+ case BPF_PROG_TYPE_LSM:
+ case BPF_PROG_TYPE_STRUCT_OPS:
+ return -EOPNOTSUPP;
+ default:
+ break;
+ }
+
+ buf[0] = '\0';
+ ret = probe_prog_load(prog_type, insns, insn_cnt, buf, sizeof(buf), 0);
+ if (ret < 0)
+ return libbpf_err(ret);
+
+ /* If BPF verifier doesn't recognize BPF helper ID (enum bpf_func_id)
+ * at all, it will emit something like "invalid func unknown#181".
+ * If BPF verifier recognizes BPF helper but it's not supported for
+ * given BPF program type, it will emit "unknown func bpf_sys_bpf#166".
+ * In both cases, provided combination of BPF program type and BPF
+ * helper is not supported by the kernel.
+ * In all other cases, probe_prog_load() above will either succeed (e.g.,
+ * because BPF helper happens to accept no input arguments or it
+ * accepts one input argument and initial PTR_TO_CTX is fine for
+ * that), or we'll get some more specific BPF verifier error about
+ * some unsatisfied conditions.
+ */
+ if (ret == 0 && (strstr(buf, "invalid func ") || strstr(buf, "unknown func ")))
+ return 0;
+ return 1; /* assume supported */
}
bool bpf_probe_helper(enum bpf_func_id id, enum bpf_prog_type prog_type,
@@ -320,8 +438,7 @@ bool bpf_probe_helper(enum bpf_func_id id, enum bpf_prog_type prog_type,
char buf[4096] = {};
bool res;
- probe_load(prog_type, insns, ARRAY_SIZE(insns), buf, sizeof(buf),
- ifindex);
+ probe_prog_load(prog_type, insns, ARRAY_SIZE(insns), buf, sizeof(buf), ifindex);
res = !grep(buf, "invalid func ") && !grep(buf, "unknown func ");
if (ifindex) {
@@ -353,8 +470,8 @@ bool bpf_probe_large_insn_limit(__u32 ifindex)
insns[BPF_MAXINSNS] = BPF_EXIT_INSN();
errno = 0;
- probe_load(BPF_PROG_TYPE_SCHED_CLS, insns, ARRAY_SIZE(insns), NULL, 0,
- ifindex);
+ probe_prog_load(BPF_PROG_TYPE_SCHED_CLS, insns, ARRAY_SIZE(insns), NULL, 0,
+ ifindex);
return errno != E2BIG && errno != EINVAL;
}
diff --git a/tools/lib/bpf/libbpf_version.h b/tools/lib/bpf/libbpf_version.h
index dd56d76..0fefefc 100644
--- a/tools/lib/bpf/libbpf_version.h
+++ b/tools/lib/bpf/libbpf_version.h
@@ -4,6 +4,6 @@
#define __LIBBPF_VERSION_H
#define LIBBPF_MAJOR_VERSION 0
-#define LIBBPF_MINOR_VERSION 6
+#define LIBBPF_MINOR_VERSION 7
#endif /* __LIBBPF_VERSION_H */
diff --git a/tools/lib/bpf/linker.c b/tools/lib/bpf/linker.c
index f677dcc..9aa016f 100644
--- a/tools/lib/bpf/linker.c
+++ b/tools/lib/bpf/linker.c
@@ -210,6 +210,7 @@ void bpf_linker__free(struct bpf_linker *linker)
}
free(linker->secs);
+ free(linker->glob_syms);
free(linker);
}
@@ -1999,7 +2000,7 @@ static int linker_append_elf_sym(struct bpf_linker *linker, struct src_obj *obj,
static int linker_append_elf_relos(struct bpf_linker *linker, struct src_obj *obj)
{
struct src_sec *src_symtab = &obj->secs[obj->symtab_sec_idx];
- struct dst_sec *dst_symtab = &linker->secs[linker->symtab_sec_idx];
+ struct dst_sec *dst_symtab;
int i, err;
for (i = 1; i < obj->sec_cnt; i++) {
@@ -2032,6 +2033,9 @@ static int linker_append_elf_relos(struct bpf_linker *linker, struct src_obj *ob
return -1;
}
+ /* add_dst_sec() above could have invalidated linker->secs */
+ dst_symtab = &linker->secs[linker->symtab_sec_idx];
+
/* shdr->sh_link points to SYMTAB */
dst_sec->shdr->sh_link = linker->symtab_sec_idx;
@@ -2650,6 +2654,7 @@ static int emit_elf_data_sec(struct bpf_linker *linker, const char *sec_name,
static int finalize_btf(struct bpf_linker *linker)
{
+ LIBBPF_OPTS(btf_dedup_opts, opts);
struct btf *btf = linker->btf;
const void *raw_data;
int i, j, id, err;
@@ -2686,7 +2691,8 @@ static int finalize_btf(struct bpf_linker *linker)
return err;
}
- err = btf__dedup(linker->btf, linker->btf_ext, NULL);
+ opts.btf_ext = linker->btf_ext;
+ err = btf__dedup(linker->btf, &opts);
if (err) {
pr_warn("BTF dedup failed: %d\n", err);
return err;
diff --git a/tools/lib/bpf/relo_core.c b/tools/lib/bpf/relo_core.c
index b5b8956..910865e 100644
--- a/tools/lib/bpf/relo_core.c
+++ b/tools/lib/bpf/relo_core.c
@@ -1,6 +1,60 @@
// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
/* Copyright (c) 2019 Facebook */
+#ifdef __KERNEL__
+#include <linux/bpf.h>
+#include <linux/btf.h>
+#include <linux/string.h>
+#include <linux/bpf_verifier.h>
+#include "relo_core.h"
+
+static const char *btf_kind_str(const struct btf_type *t)
+{
+ return btf_type_str(t);
+}
+
+static bool is_ldimm64_insn(struct bpf_insn *insn)
+{
+ return insn->code == (BPF_LD | BPF_IMM | BPF_DW);
+}
+
+static const struct btf_type *
+skip_mods_and_typedefs(const struct btf *btf, u32 id, u32 *res_id)
+{
+ return btf_type_skip_modifiers(btf, id, res_id);
+}
+
+static const char *btf__name_by_offset(const struct btf *btf, u32 offset)
+{
+ return btf_name_by_offset(btf, offset);
+}
+
+static s64 btf__resolve_size(const struct btf *btf, u32 type_id)
+{
+ const struct btf_type *t;
+ int size;
+
+ t = btf_type_by_id(btf, type_id);
+ t = btf_resolve_size(btf, t, &size);
+ if (IS_ERR(t))
+ return PTR_ERR(t);
+ return size;
+}
+
+enum libbpf_print_level {
+ LIBBPF_WARN,
+ LIBBPF_INFO,
+ LIBBPF_DEBUG,
+};
+
+#undef pr_warn
+#undef pr_info
+#undef pr_debug
+#define pr_warn(fmt, log, ...) bpf_log((void *)log, fmt, "", ##__VA_ARGS__)
+#define pr_info(fmt, log, ...) bpf_log((void *)log, fmt, "", ##__VA_ARGS__)
+#define pr_debug(fmt, log, ...) bpf_log((void *)log, fmt, "", ##__VA_ARGS__)
+#define libbpf_print(level, fmt, ...) bpf_log((void *)prog_name, fmt, ##__VA_ARGS__)
+#else
#include <stdio.h>
#include <string.h>
#include <errno.h>
@@ -12,33 +66,7 @@
#include "btf.h"
#include "str_error.h"
#include "libbpf_internal.h"
-
-#define BPF_CORE_SPEC_MAX_LEN 64
-
-/* represents BPF CO-RE field or array element accessor */
-struct bpf_core_accessor {
- __u32 type_id; /* struct/union type or array element type */
- __u32 idx; /* field index or array index */
- const char *name; /* field name or NULL for array accessor */
-};
-
-struct bpf_core_spec {
- const struct btf *btf;
- /* high-level spec: named fields and array indices only */
- struct bpf_core_accessor spec[BPF_CORE_SPEC_MAX_LEN];
- /* original unresolved (no skip_mods_or_typedefs) root type ID */
- __u32 root_type_id;
- /* CO-RE relocation kind */
- enum bpf_core_relo_kind relo_kind;
- /* high-level spec length */
- int len;
- /* raw, low-level spec: 1-to-1 with accessor spec string */
- int raw_spec[BPF_CORE_SPEC_MAX_LEN];
- /* raw spec length */
- int raw_len;
- /* field bit offset represented by spec */
- __u32 bit_offset;
-};
+#endif
static bool is_flex_arr(const struct btf *btf,
const struct bpf_core_accessor *acc,
@@ -51,25 +79,25 @@ static bool is_flex_arr(const struct btf *btf,
return false;
/* has to be the last member of enclosing struct */
- t = btf__type_by_id(btf, acc->type_id);
+ t = btf_type_by_id(btf, acc->type_id);
return acc->idx == btf_vlen(t) - 1;
}
static const char *core_relo_kind_str(enum bpf_core_relo_kind kind)
{
switch (kind) {
- case BPF_FIELD_BYTE_OFFSET: return "byte_off";
- case BPF_FIELD_BYTE_SIZE: return "byte_sz";
- case BPF_FIELD_EXISTS: return "field_exists";
- case BPF_FIELD_SIGNED: return "signed";
- case BPF_FIELD_LSHIFT_U64: return "lshift_u64";
- case BPF_FIELD_RSHIFT_U64: return "rshift_u64";
- case BPF_TYPE_ID_LOCAL: return "local_type_id";
- case BPF_TYPE_ID_TARGET: return "target_type_id";
- case BPF_TYPE_EXISTS: return "type_exists";
- case BPF_TYPE_SIZE: return "type_size";
- case BPF_ENUMVAL_EXISTS: return "enumval_exists";
- case BPF_ENUMVAL_VALUE: return "enumval_value";
+ case BPF_CORE_FIELD_BYTE_OFFSET: return "byte_off";
+ case BPF_CORE_FIELD_BYTE_SIZE: return "byte_sz";
+ case BPF_CORE_FIELD_EXISTS: return "field_exists";
+ case BPF_CORE_FIELD_SIGNED: return "signed";
+ case BPF_CORE_FIELD_LSHIFT_U64: return "lshift_u64";
+ case BPF_CORE_FIELD_RSHIFT_U64: return "rshift_u64";
+ case BPF_CORE_TYPE_ID_LOCAL: return "local_type_id";
+ case BPF_CORE_TYPE_ID_TARGET: return "target_type_id";
+ case BPF_CORE_TYPE_EXISTS: return "type_exists";
+ case BPF_CORE_TYPE_SIZE: return "type_size";
+ case BPF_CORE_ENUMVAL_EXISTS: return "enumval_exists";
+ case BPF_CORE_ENUMVAL_VALUE: return "enumval_value";
default: return "unknown";
}
}
@@ -77,12 +105,12 @@ static const char *core_relo_kind_str(enum bpf_core_relo_kind kind)
static bool core_relo_is_field_based(enum bpf_core_relo_kind kind)
{
switch (kind) {
- case BPF_FIELD_BYTE_OFFSET:
- case BPF_FIELD_BYTE_SIZE:
- case BPF_FIELD_EXISTS:
- case BPF_FIELD_SIGNED:
- case BPF_FIELD_LSHIFT_U64:
- case BPF_FIELD_RSHIFT_U64:
+ case BPF_CORE_FIELD_BYTE_OFFSET:
+ case BPF_CORE_FIELD_BYTE_SIZE:
+ case BPF_CORE_FIELD_EXISTS:
+ case BPF_CORE_FIELD_SIGNED:
+ case BPF_CORE_FIELD_LSHIFT_U64:
+ case BPF_CORE_FIELD_RSHIFT_U64:
return true;
default:
return false;
@@ -92,10 +120,10 @@ static bool core_relo_is_field_based(enum bpf_core_relo_kind kind)
static bool core_relo_is_type_based(enum bpf_core_relo_kind kind)
{
switch (kind) {
- case BPF_TYPE_ID_LOCAL:
- case BPF_TYPE_ID_TARGET:
- case BPF_TYPE_EXISTS:
- case BPF_TYPE_SIZE:
+ case BPF_CORE_TYPE_ID_LOCAL:
+ case BPF_CORE_TYPE_ID_TARGET:
+ case BPF_CORE_TYPE_EXISTS:
+ case BPF_CORE_TYPE_SIZE:
return true;
default:
return false;
@@ -105,8 +133,8 @@ static bool core_relo_is_type_based(enum bpf_core_relo_kind kind)
static bool core_relo_is_enumval_based(enum bpf_core_relo_kind kind)
{
switch (kind) {
- case BPF_ENUMVAL_EXISTS:
- case BPF_ENUMVAL_VALUE:
+ case BPF_CORE_ENUMVAL_EXISTS:
+ case BPF_CORE_ENUMVAL_VALUE:
return true;
default:
return false;
@@ -150,7 +178,7 @@ static bool core_relo_is_enumval_based(enum bpf_core_relo_kind kind)
* Enum value-based relocations (ENUMVAL_EXISTS/ENUMVAL_VALUE) use access
* string to specify enumerator's value index that need to be relocated.
*/
-static int bpf_core_parse_spec(const struct btf *btf,
+static int bpf_core_parse_spec(const char *prog_name, const struct btf *btf,
__u32 type_id,
const char *spec_str,
enum bpf_core_relo_kind relo_kind,
@@ -272,8 +300,8 @@ static int bpf_core_parse_spec(const struct btf *btf,
return sz;
spec->bit_offset += access_idx * sz * 8;
} else {
- pr_warn("relo for [%u] %s (at idx %d) captures type [%d] of unexpected kind %s\n",
- type_id, spec_str, i, id, btf_kind_str(t));
+ pr_warn("prog '%s': relo for [%u] %s (at idx %d) captures type [%d] of unexpected kind %s\n",
+ prog_name, type_id, spec_str, i, id, btf_kind_str(t));
return -EINVAL;
}
}
@@ -346,8 +374,6 @@ static int bpf_core_fields_are_compat(const struct btf *local_btf,
targ_id = btf_array(targ_type)->type;
goto recur;
default:
- pr_warn("unexpected kind %d relocated, local [%d], target [%d]\n",
- btf_kind(local_type), local_id, targ_id);
return 0;
}
}
@@ -388,7 +414,7 @@ static int bpf_core_match_member(const struct btf *local_btf,
return 0;
local_id = local_acc->type_id;
- local_type = btf__type_by_id(local_btf, local_id);
+ local_type = btf_type_by_id(local_btf, local_id);
local_member = btf_members(local_type) + local_acc->idx;
local_name = btf__name_by_offset(local_btf, local_member->name_off);
@@ -571,7 +597,7 @@ static int bpf_core_calc_field_relo(const char *prog_name,
*field_sz = 0;
- if (relo->kind == BPF_FIELD_EXISTS) {
+ if (relo->kind == BPF_CORE_FIELD_EXISTS) {
*val = spec ? 1 : 0;
return 0;
}
@@ -580,11 +606,11 @@ static int bpf_core_calc_field_relo(const char *prog_name,
return -EUCLEAN; /* request instruction poisoning */
acc = &spec->spec[spec->len - 1];
- t = btf__type_by_id(spec->btf, acc->type_id);
+ t = btf_type_by_id(spec->btf, acc->type_id);
/* a[n] accessor needs special handling */
if (!acc->name) {
- if (relo->kind == BPF_FIELD_BYTE_OFFSET) {
+ if (relo->kind == BPF_CORE_FIELD_BYTE_OFFSET) {
*val = spec->bit_offset / 8;
/* remember field size for load/store mem size */
sz = btf__resolve_size(spec->btf, acc->type_id);
@@ -592,7 +618,7 @@ static int bpf_core_calc_field_relo(const char *prog_name,
return -EINVAL;
*field_sz = sz;
*type_id = acc->type_id;
- } else if (relo->kind == BPF_FIELD_BYTE_SIZE) {
+ } else if (relo->kind == BPF_CORE_FIELD_BYTE_SIZE) {
sz = btf__resolve_size(spec->btf, acc->type_id);
if (sz < 0)
return -EINVAL;
@@ -644,36 +670,36 @@ static int bpf_core_calc_field_relo(const char *prog_name,
*validate = !bitfield;
switch (relo->kind) {
- case BPF_FIELD_BYTE_OFFSET:
+ case BPF_CORE_FIELD_BYTE_OFFSET:
*val = byte_off;
if (!bitfield) {
*field_sz = byte_sz;
*type_id = field_type_id;
}
break;
- case BPF_FIELD_BYTE_SIZE:
+ case BPF_CORE_FIELD_BYTE_SIZE:
*val = byte_sz;
break;
- case BPF_FIELD_SIGNED:
+ case BPF_CORE_FIELD_SIGNED:
/* enums will be assumed unsigned */
*val = btf_is_enum(mt) ||
(btf_int_encoding(mt) & BTF_INT_SIGNED);
if (validate)
*validate = true; /* signedness is never ambiguous */
break;
- case BPF_FIELD_LSHIFT_U64:
+ case BPF_CORE_FIELD_LSHIFT_U64:
#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
*val = 64 - (bit_off + bit_sz - byte_off * 8);
#else
*val = (8 - byte_sz) * 8 + (bit_off - byte_off * 8);
#endif
break;
- case BPF_FIELD_RSHIFT_U64:
+ case BPF_CORE_FIELD_RSHIFT_U64:
*val = 64 - bit_sz;
if (validate)
*validate = true; /* right shift is never ambiguous */
break;
- case BPF_FIELD_EXISTS:
+ case BPF_CORE_FIELD_EXISTS:
default:
return -EOPNOTSUPP;
}
@@ -683,10 +709,14 @@ static int bpf_core_calc_field_relo(const char *prog_name,
static int bpf_core_calc_type_relo(const struct bpf_core_relo *relo,
const struct bpf_core_spec *spec,
- __u32 *val)
+ __u32 *val, bool *validate)
{
__s64 sz;
+ /* by default, always check expected value in bpf_insn */
+ if (validate)
+ *validate = true;
+
/* type-based relos return zero when target type is not found */
if (!spec) {
*val = 0;
@@ -694,20 +724,25 @@ static int bpf_core_calc_type_relo(const struct bpf_core_relo *relo,
}
switch (relo->kind) {
- case BPF_TYPE_ID_TARGET:
+ case BPF_CORE_TYPE_ID_TARGET:
*val = spec->root_type_id;
+ /* type ID, embedded in bpf_insn, might change during linking,
+ * so enforcing it is pointless
+ */
+ if (validate)
+ *validate = false;
break;
- case BPF_TYPE_EXISTS:
+ case BPF_CORE_TYPE_EXISTS:
*val = 1;
break;
- case BPF_TYPE_SIZE:
+ case BPF_CORE_TYPE_SIZE:
sz = btf__resolve_size(spec->btf, spec->root_type_id);
if (sz < 0)
return -EINVAL;
*val = sz;
break;
- case BPF_TYPE_ID_LOCAL:
- /* BPF_TYPE_ID_LOCAL is handled specially and shouldn't get here */
+ case BPF_CORE_TYPE_ID_LOCAL:
+ /* BPF_CORE_TYPE_ID_LOCAL is handled specially and shouldn't get here */
default:
return -EOPNOTSUPP;
}
@@ -723,13 +758,13 @@ static int bpf_core_calc_enumval_relo(const struct bpf_core_relo *relo,
const struct btf_enum *e;
switch (relo->kind) {
- case BPF_ENUMVAL_EXISTS:
+ case BPF_CORE_ENUMVAL_EXISTS:
*val = spec ? 1 : 0;
break;
- case BPF_ENUMVAL_VALUE:
+ case BPF_CORE_ENUMVAL_VALUE:
if (!spec)
return -EUCLEAN; /* request instruction poisoning */
- t = btf__type_by_id(spec->btf, spec->spec[0].type_id);
+ t = btf_type_by_id(spec->btf, spec->spec[0].type_id);
e = btf_enum(t) + spec->spec[0].idx;
*val = e->val;
break;
@@ -805,8 +840,8 @@ static int bpf_core_calc_relo(const char *prog_name,
if (res->orig_sz != res->new_sz) {
const struct btf_type *orig_t, *new_t;
- orig_t = btf__type_by_id(local_spec->btf, res->orig_type_id);
- new_t = btf__type_by_id(targ_spec->btf, res->new_type_id);
+ orig_t = btf_type_by_id(local_spec->btf, res->orig_type_id);
+ new_t = btf_type_by_id(targ_spec->btf, res->new_type_id);
/* There are two use cases in which it's safe to
* adjust load/store's mem size:
@@ -835,8 +870,8 @@ static int bpf_core_calc_relo(const char *prog_name,
res->fail_memsz_adjust = true;
}
} else if (core_relo_is_type_based(relo->kind)) {
- err = bpf_core_calc_type_relo(relo, local_spec, &res->orig_val);
- err = err ?: bpf_core_calc_type_relo(relo, targ_spec, &res->new_val);
+ err = bpf_core_calc_type_relo(relo, local_spec, &res->orig_val, &res->validate);
+ err = err ?: bpf_core_calc_type_relo(relo, targ_spec, &res->new_val, NULL);
} else if (core_relo_is_enumval_based(relo->kind)) {
err = bpf_core_calc_enumval_relo(relo, local_spec, &res->orig_val);
err = err ?: bpf_core_calc_enumval_relo(relo, targ_spec, &res->new_val);
@@ -1045,7 +1080,7 @@ static int bpf_core_patch_insn(const char *prog_name, struct bpf_insn *insn,
* [<type-id>] (<type-name>) + <raw-spec> => <offset>@<spec>,
* where <spec> is a C-syntax view of recorded field access, e.g.: x.a[3].b
*/
-static void bpf_core_dump_spec(int level, const struct bpf_core_spec *spec)
+static void bpf_core_dump_spec(const char *prog_name, int level, const struct bpf_core_spec *spec)
{
const struct btf_type *t;
const struct btf_enum *e;
@@ -1054,7 +1089,7 @@ static void bpf_core_dump_spec(int level, const struct bpf_core_spec *spec)
int i;
type_id = spec->root_type_id;
- t = btf__type_by_id(spec->btf, type_id);
+ t = btf_type_by_id(spec->btf, type_id);
s = btf__name_by_offset(spec->btf, t->name_off);
libbpf_print(level, "[%u] %s %s", type_id, btf_kind_str(t), str_is_empty(s) ? "<anon>" : s);
@@ -1147,9 +1182,12 @@ int bpf_core_apply_relo_insn(const char *prog_name, struct bpf_insn *insn,
const struct bpf_core_relo *relo,
int relo_idx,
const struct btf *local_btf,
- struct bpf_core_cand_list *cands)
+ struct bpf_core_cand_list *cands,
+ struct bpf_core_spec *specs_scratch)
{
- struct bpf_core_spec local_spec, cand_spec, targ_spec = {};
+ struct bpf_core_spec *local_spec = &specs_scratch[0];
+ struct bpf_core_spec *cand_spec = &specs_scratch[1];
+ struct bpf_core_spec *targ_spec = &specs_scratch[2];
struct bpf_core_relo_res cand_res, targ_res;
const struct btf_type *local_type;
const char *local_name;
@@ -1158,10 +1196,7 @@ int bpf_core_apply_relo_insn(const char *prog_name, struct bpf_insn *insn,
int i, j, err;
local_id = relo->type_id;
- local_type = btf__type_by_id(local_btf, local_id);
- if (!local_type)
- return -EINVAL;
-
+ local_type = btf_type_by_id(local_btf, local_id);
local_name = btf__name_by_offset(local_btf, local_type->name_off);
if (!local_name)
return -EINVAL;
@@ -1170,7 +1205,8 @@ int bpf_core_apply_relo_insn(const char *prog_name, struct bpf_insn *insn,
if (str_is_empty(spec_str))
return -EINVAL;
- err = bpf_core_parse_spec(local_btf, local_id, spec_str, relo->kind, &local_spec);
+ err = bpf_core_parse_spec(prog_name, local_btf, local_id, spec_str,
+ relo->kind, local_spec);
if (err) {
pr_warn("prog '%s': relo #%d: parsing [%d] %s %s + %s failed: %d\n",
prog_name, relo_idx, local_id, btf_kind_str(local_type),
@@ -1181,15 +1217,17 @@ int bpf_core_apply_relo_insn(const char *prog_name, struct bpf_insn *insn,
pr_debug("prog '%s': relo #%d: kind <%s> (%d), spec is ", prog_name,
relo_idx, core_relo_kind_str(relo->kind), relo->kind);
- bpf_core_dump_spec(LIBBPF_DEBUG, &local_spec);
+ bpf_core_dump_spec(prog_name, LIBBPF_DEBUG, local_spec);
libbpf_print(LIBBPF_DEBUG, "\n");
/* TYPE_ID_LOCAL relo is special and doesn't need candidate search */
- if (relo->kind == BPF_TYPE_ID_LOCAL) {
- targ_res.validate = true;
+ if (relo->kind == BPF_CORE_TYPE_ID_LOCAL) {
+ /* bpf_insn's imm value could get out of sync during linking */
+ memset(&targ_res, 0, sizeof(targ_res));
+ targ_res.validate = false;
targ_res.poison = false;
- targ_res.orig_val = local_spec.root_type_id;
- targ_res.new_val = local_spec.root_type_id;
+ targ_res.orig_val = local_spec->root_type_id;
+ targ_res.new_val = local_spec->root_type_id;
goto patch_insn;
}
@@ -1200,40 +1238,39 @@ int bpf_core_apply_relo_insn(const char *prog_name, struct bpf_insn *insn,
return -EOPNOTSUPP;
}
-
for (i = 0, j = 0; i < cands->len; i++) {
- err = bpf_core_spec_match(&local_spec, cands->cands[i].btf,
- cands->cands[i].id, &cand_spec);
+ err = bpf_core_spec_match(local_spec, cands->cands[i].btf,
+ cands->cands[i].id, cand_spec);
if (err < 0) {
pr_warn("prog '%s': relo #%d: error matching candidate #%d ",
prog_name, relo_idx, i);
- bpf_core_dump_spec(LIBBPF_WARN, &cand_spec);
+ bpf_core_dump_spec(prog_name, LIBBPF_WARN, cand_spec);
libbpf_print(LIBBPF_WARN, ": %d\n", err);
return err;
}
pr_debug("prog '%s': relo #%d: %s candidate #%d ", prog_name,
relo_idx, err == 0 ? "non-matching" : "matching", i);
- bpf_core_dump_spec(LIBBPF_DEBUG, &cand_spec);
+ bpf_core_dump_spec(prog_name, LIBBPF_DEBUG, cand_spec);
libbpf_print(LIBBPF_DEBUG, "\n");
if (err == 0)
continue;
- err = bpf_core_calc_relo(prog_name, relo, relo_idx, &local_spec, &cand_spec, &cand_res);
+ err = bpf_core_calc_relo(prog_name, relo, relo_idx, local_spec, cand_spec, &cand_res);
if (err)
return err;
if (j == 0) {
targ_res = cand_res;
- targ_spec = cand_spec;
- } else if (cand_spec.bit_offset != targ_spec.bit_offset) {
+ *targ_spec = *cand_spec;
+ } else if (cand_spec->bit_offset != targ_spec->bit_offset) {
/* if there are many field relo candidates, they
* should all resolve to the same bit offset
*/
pr_warn("prog '%s': relo #%d: field offset ambiguity: %u != %u\n",
- prog_name, relo_idx, cand_spec.bit_offset,
- targ_spec.bit_offset);
+ prog_name, relo_idx, cand_spec->bit_offset,
+ targ_spec->bit_offset);
return -EINVAL;
} else if (cand_res.poison != targ_res.poison || cand_res.new_val != targ_res.new_val) {
/* all candidates should result in the same relocation
@@ -1251,7 +1288,7 @@ int bpf_core_apply_relo_insn(const char *prog_name, struct bpf_insn *insn,
}
/*
- * For BPF_FIELD_EXISTS relo or when used BPF program has field
+ * For BPF_CORE_FIELD_EXISTS relo or when used BPF program has field
* existence checks or kernel version/config checks, it's expected
* that we might not find any candidates. In this case, if field
* wasn't found in any candidate, the list of candidates shouldn't
@@ -1277,7 +1314,7 @@ int bpf_core_apply_relo_insn(const char *prog_name, struct bpf_insn *insn,
prog_name, relo_idx);
/* calculate single target relo result explicitly */
- err = bpf_core_calc_relo(prog_name, relo, relo_idx, &local_spec, NULL, &targ_res);
+ err = bpf_core_calc_relo(prog_name, relo, relo_idx, local_spec, NULL, &targ_res);
if (err)
return err;
}
diff --git a/tools/lib/bpf/relo_core.h b/tools/lib/bpf/relo_core.h
index 3b9f8f1..1779981 100644
--- a/tools/lib/bpf/relo_core.h
+++ b/tools/lib/bpf/relo_core.h
@@ -4,81 +4,10 @@
#ifndef __RELO_CORE_H
#define __RELO_CORE_H
-/* bpf_core_relo_kind encodes which aspect of captured field/type/enum value
- * has to be adjusted by relocations.
- */
-enum bpf_core_relo_kind {
- BPF_FIELD_BYTE_OFFSET = 0, /* field byte offset */
- BPF_FIELD_BYTE_SIZE = 1, /* field size in bytes */
- BPF_FIELD_EXISTS = 2, /* field existence in target kernel */
- BPF_FIELD_SIGNED = 3, /* field signedness (0 - unsigned, 1 - signed) */
- BPF_FIELD_LSHIFT_U64 = 4, /* bitfield-specific left bitshift */
- BPF_FIELD_RSHIFT_U64 = 5, /* bitfield-specific right bitshift */
- BPF_TYPE_ID_LOCAL = 6, /* type ID in local BPF object */
- BPF_TYPE_ID_TARGET = 7, /* type ID in target kernel */
- BPF_TYPE_EXISTS = 8, /* type existence in target kernel */
- BPF_TYPE_SIZE = 9, /* type size in bytes */
- BPF_ENUMVAL_EXISTS = 10, /* enum value existence in target kernel */
- BPF_ENUMVAL_VALUE = 11, /* enum value integer value */
-};
-
-/* The minimum bpf_core_relo checked by the loader
- *
- * CO-RE relocation captures the following data:
- * - insn_off - instruction offset (in bytes) within a BPF program that needs
- * its insn->imm field to be relocated with actual field info;
- * - type_id - BTF type ID of the "root" (containing) entity of a relocatable
- * type or field;
- * - access_str_off - offset into corresponding .BTF string section. String
- * interpretation depends on specific relocation kind:
- * - for field-based relocations, string encodes an accessed field using
- * a sequence of field and array indices, separated by colon (:). It's
- * conceptually very close to LLVM's getelementptr ([0]) instruction's
- * arguments for identifying offset to a field.
- * - for type-based relocations, strings is expected to be just "0";
- * - for enum value-based relocations, string contains an index of enum
- * value within its enum type;
- *
- * Example to provide a better feel.
- *
- * struct sample {
- * int a;
- * struct {
- * int b[10];
- * };
- * };
- *
- * struct sample *s = ...;
- * int x = &s->a; // encoded as "0:0" (a is field #0)
- * int y = &s->b[5]; // encoded as "0:1:0:5" (anon struct is field #1,
- * // b is field #0 inside anon struct, accessing elem #5)
- * int z = &s[10]->b; // encoded as "10:1" (ptr is used as an array)
- *
- * type_id for all relocs in this example will capture BTF type id of
- * `struct sample`.
- *
- * Such relocation is emitted when using __builtin_preserve_access_index()
- * Clang built-in, passing expression that captures field address, e.g.:
- *
- * bpf_probe_read(&dst, sizeof(dst),
- * __builtin_preserve_access_index(&src->a.b.c));
- *
- * In this case Clang will emit field relocation recording necessary data to
- * be able to find offset of embedded `a.b.c` field within `src` struct.
- *
- * [0] https://llvm.org/docs/LangRef.html#getelementptr-instruction
- */
-struct bpf_core_relo {
- __u32 insn_off;
- __u32 type_id;
- __u32 access_str_off;
- enum bpf_core_relo_kind kind;
-};
+#include <linux/bpf.h>
struct bpf_core_cand {
const struct btf *btf;
- const struct btf_type *t;
- const char *name;
__u32 id;
};
@@ -88,11 +17,39 @@ struct bpf_core_cand_list {
int len;
};
+#define BPF_CORE_SPEC_MAX_LEN 64
+
+/* represents BPF CO-RE field or array element accessor */
+struct bpf_core_accessor {
+ __u32 type_id; /* struct/union type or array element type */
+ __u32 idx; /* field index or array index */
+ const char *name; /* field name or NULL for array accessor */
+};
+
+struct bpf_core_spec {
+ const struct btf *btf;
+ /* high-level spec: named fields and array indices only */
+ struct bpf_core_accessor spec[BPF_CORE_SPEC_MAX_LEN];
+ /* original unresolved (no skip_mods_or_typedefs) root type ID */
+ __u32 root_type_id;
+ /* CO-RE relocation kind */
+ enum bpf_core_relo_kind relo_kind;
+ /* high-level spec length */
+ int len;
+ /* raw, low-level spec: 1-to-1 with accessor spec string */
+ int raw_spec[BPF_CORE_SPEC_MAX_LEN];
+ /* raw spec length */
+ int raw_len;
+ /* field bit offset represented by spec */
+ __u32 bit_offset;
+};
+
int bpf_core_apply_relo_insn(const char *prog_name,
struct bpf_insn *insn, int insn_idx,
const struct bpf_core_relo *relo, int relo_idx,
const struct btf *local_btf,
- struct bpf_core_cand_list *cands);
+ struct bpf_core_cand_list *cands,
+ struct bpf_core_spec *specs_scratch);
int bpf_core_types_are_compat(const struct btf *local_btf, __u32 local_id,
const struct btf *targ_btf, __u32 targ_id);
diff --git a/tools/lib/bpf/skel_internal.h b/tools/lib/bpf/skel_internal.h
index 9cf6670..0b84d8e 100644
--- a/tools/lib/bpf/skel_internal.h
+++ b/tools/lib/bpf/skel_internal.h
@@ -7,6 +7,16 @@
#include <sys/syscall.h>
#include <sys/mman.h>
+#ifndef __NR_bpf
+# if defined(__mips__) && defined(_ABIO32)
+# define __NR_bpf 4355
+# elif defined(__mips__) && defined(_ABIN32)
+# define __NR_bpf 6319
+# elif defined(__mips__) && defined(_ABI64)
+# define __NR_bpf 5315
+# endif
+#endif
+
/* This file is a base header for auto-generated *.lskel.h files.
* Its contents will change and may become part of auto-generation in the future.
*
@@ -65,8 +75,7 @@ static inline int bpf_load_and_run(struct bpf_load_and_run_opts *opts)
int map_fd = -1, prog_fd = -1, key = 0, err;
union bpf_attr attr;
- map_fd = bpf_create_map_name(BPF_MAP_TYPE_ARRAY, "__loader.map", 4,
- opts->data_sz, 1, 0);
+ map_fd = bpf_map_create(BPF_MAP_TYPE_ARRAY, "__loader.map", 4, opts->data_sz, 1, NULL);
if (map_fd < 0) {
opts->errstr = "failed to create loader map";
err = -errno;
diff --git a/tools/lib/bpf/xsk.c b/tools/lib/bpf/xsk.c
index 81f8fbc..edafe56 100644
--- a/tools/lib/bpf/xsk.c
+++ b/tools/lib/bpf/xsk.c
@@ -35,6 +35,11 @@
#include "libbpf_internal.h"
#include "xsk.h"
+/* entire xsk.h and xsk.c is going away in libbpf 1.0, so ignore all internal
+ * uses of deprecated APIs
+ */
+#pragma GCC diagnostic ignored "-Wdeprecated-declarations"
+
#ifndef SOL_XDP
#define SOL_XDP 283
#endif
@@ -364,8 +369,6 @@ int xsk_umem__create_v0_0_2(struct xsk_umem **umem_ptr, void *umem_area,
static enum xsk_prog get_xsk_prog(void)
{
enum xsk_prog detected = XSK_PROG_FALLBACK;
- struct bpf_load_program_attr prog_attr;
- struct bpf_create_map_attr map_attr;
__u32 size_out, retval, duration;
char data_in = 0, data_out;
struct bpf_insn insns[] = {
@@ -375,27 +378,15 @@ static enum xsk_prog get_xsk_prog(void)
BPF_EMIT_CALL(BPF_FUNC_redirect_map),
BPF_EXIT_INSN(),
};
- int prog_fd, map_fd, ret;
+ int prog_fd, map_fd, ret, insn_cnt = ARRAY_SIZE(insns);
- memset(&map_attr, 0, sizeof(map_attr));
- map_attr.map_type = BPF_MAP_TYPE_XSKMAP;
- map_attr.key_size = sizeof(int);
- map_attr.value_size = sizeof(int);
- map_attr.max_entries = 1;
-
- map_fd = bpf_create_map_xattr(&map_attr);
+ map_fd = bpf_map_create(BPF_MAP_TYPE_XSKMAP, NULL, sizeof(int), sizeof(int), 1, NULL);
if (map_fd < 0)
return detected;
insns[0].imm = map_fd;
- memset(&prog_attr, 0, sizeof(prog_attr));
- prog_attr.prog_type = BPF_PROG_TYPE_XDP;
- prog_attr.insns = insns;
- prog_attr.insns_cnt = ARRAY_SIZE(insns);
- prog_attr.license = "GPL";
-
- prog_fd = bpf_load_program_xattr(&prog_attr, NULL, 0);
+ prog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", insns, insn_cnt, NULL);
if (prog_fd < 0) {
close(map_fd);
return detected;
@@ -495,10 +486,13 @@ static int xsk_load_xdp_prog(struct xsk_socket *xsk)
};
struct bpf_insn *progs[] = {prog, prog_redirect_flags};
enum xsk_prog option = get_xsk_prog();
+ LIBBPF_OPTS(bpf_prog_load_opts, opts,
+ .log_buf = log_buf,
+ .log_size = log_buf_size,
+ );
- prog_fd = bpf_load_program(BPF_PROG_TYPE_XDP, progs[option], insns_cnt[option],
- "LGPL-2.1 or BSD-2-Clause", 0, log_buf,
- log_buf_size);
+ prog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "LGPL-2.1 or BSD-2-Clause",
+ progs[option], insns_cnt[option], &opts);
if (prog_fd < 0) {
pr_warn("BPF log buffer:\n%s", log_buf);
return prog_fd;
@@ -554,8 +548,7 @@ static int xsk_get_max_queues(struct xsk_socket *xsk)
return -errno;
ifr.ifr_data = (void *)&channels;
- memcpy(ifr.ifr_name, ctx->ifname, IFNAMSIZ - 1);
- ifr.ifr_name[IFNAMSIZ - 1] = '\0';
+ libbpf_strlcpy(ifr.ifr_name, ctx->ifname, IFNAMSIZ);
err = ioctl(fd, SIOCETHTOOL, &ifr);
if (err && errno != EOPNOTSUPP) {
ret = -errno;
@@ -590,8 +583,8 @@ static int xsk_create_bpf_maps(struct xsk_socket *xsk)
if (max_queues < 0)
return max_queues;
- fd = bpf_create_map_name(BPF_MAP_TYPE_XSKMAP, "xsks_map",
- sizeof(int), sizeof(int), max_queues, 0);
+ fd = bpf_map_create(BPF_MAP_TYPE_XSKMAP, "xsks_map",
+ sizeof(int), sizeof(int), max_queues, NULL);
if (fd < 0)
return fd;
@@ -725,14 +718,12 @@ static int xsk_link_lookup(int ifindex, __u32 *prog_id, int *link_fd)
static bool xsk_probe_bpf_link(void)
{
- DECLARE_LIBBPF_OPTS(bpf_link_create_opts, opts,
- .flags = XDP_FLAGS_SKB_MODE);
- struct bpf_load_program_attr prog_attr;
+ LIBBPF_OPTS(bpf_link_create_opts, opts, .flags = XDP_FLAGS_SKB_MODE);
struct bpf_insn insns[2] = {
BPF_MOV64_IMM(BPF_REG_0, XDP_PASS),
BPF_EXIT_INSN()
};
- int prog_fd, link_fd = -1;
+ int prog_fd, link_fd = -1, insn_cnt = ARRAY_SIZE(insns);
int ifindex_lo = 1;
bool ret = false;
int err;
@@ -744,13 +735,7 @@ static bool xsk_probe_bpf_link(void)
if (link_fd >= 0)
return true;
- memset(&prog_attr, 0, sizeof(prog_attr));
- prog_attr.prog_type = BPF_PROG_TYPE_XDP;
- prog_attr.insns = insns;
- prog_attr.insns_cnt = ARRAY_SIZE(insns);
- prog_attr.license = "GPL";
-
- prog_fd = bpf_load_program_xattr(&prog_attr, NULL, 0);
+ prog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", insns, insn_cnt, NULL);
if (prog_fd < 0)
return ret;
@@ -782,8 +767,7 @@ static int xsk_create_xsk_struct(int ifindex, struct xsk_socket *xsk)
}
ctx->ifindex = ifindex;
- memcpy(ctx->ifname, ifname, IFNAMSIZ -1);
- ctx->ifname[IFNAMSIZ - 1] = 0;
+ libbpf_strlcpy(ctx->ifname, ifname, IFNAMSIZ);
xsk->ctx = ctx;
xsk->ctx->has_bpf_link = xsk_probe_bpf_link();
@@ -965,8 +949,7 @@ static struct xsk_ctx *xsk_create_ctx(struct xsk_socket *xsk,
ctx->refcount = 1;
ctx->umem = umem;
ctx->queue_id = queue_id;
- memcpy(ctx->ifname, ifname, IFNAMSIZ - 1);
- ctx->ifname[IFNAMSIZ - 1] = '\0';
+ libbpf_strlcpy(ctx->ifname, ifname, IFNAMSIZ);
ctx->fill = fill;
ctx->comp = comp;
diff --git a/tools/perf/builtin-script.c b/tools/perf/builtin-script.c
index 9434367..c82b033 100644
--- a/tools/perf/builtin-script.c
+++ b/tools/perf/builtin-script.c
@@ -2473,7 +2473,7 @@ static int process_switch_event(struct perf_tool *tool,
if (perf_event__process_switch(tool, event, sample, machine) < 0)
return -1;
- if (scripting_ops && scripting_ops->process_switch)
+ if (scripting_ops && scripting_ops->process_switch && !filter_cpu(sample))
scripting_ops->process_switch(event, sample, machine);
if (!script->show_switch_events)
diff --git a/tools/perf/builtin-trace.c b/tools/perf/builtin-trace.c
index 0b52e08..df9fc00 100644
--- a/tools/perf/builtin-trace.c
+++ b/tools/perf/builtin-trace.c
@@ -3257,10 +3257,21 @@ static void trace__set_bpf_map_syscalls(struct trace *trace)
static struct bpf_program *trace__find_bpf_program_by_title(struct trace *trace, const char *name)
{
+ struct bpf_program *pos, *prog = NULL;
+ const char *sec_name;
+
if (trace->bpf_obj == NULL)
return NULL;
- return bpf_object__find_program_by_title(trace->bpf_obj, name);
+ bpf_object__for_each_program(pos, trace->bpf_obj) {
+ sec_name = bpf_program__section_name(pos);
+ if (sec_name && !strcmp(sec_name, name)) {
+ prog = pos;
+ break;
+ }
+ }
+
+ return prog;
}
static struct bpf_program *trace__find_syscall_bpf_prog(struct trace *trace, struct syscall *sc,
@@ -3925,6 +3936,7 @@ static int trace__run(struct trace *trace, int argc, const char **argv)
bool draining = false;
trace->live = true;
+ signal(SIGCHLD, sig_handler);
if (!trace->raw_augmented_syscalls) {
if (trace->trace_syscalls && trace__add_syscall_newtp(trace))
@@ -4873,7 +4885,6 @@ int cmd_trace(int argc, const char **argv)
signal(SIGSEGV, sighandler_dump_stack);
signal(SIGFPE, sighandler_dump_stack);
- signal(SIGCHLD, sig_handler);
signal(SIGINT, sig_handler);
trace.evlist = evlist__new();
diff --git a/tools/perf/scripts/python/intel-pt-events.py b/tools/perf/scripts/python/intel-pt-events.py
index 1d3a189..66452a8 100644
--- a/tools/perf/scripts/python/intel-pt-events.py
+++ b/tools/perf/scripts/python/intel-pt-events.py
@@ -32,8 +32,7 @@
except:
broken_pipe_exception = IOError
-glb_switch_str = None
-glb_switch_printed = True
+glb_switch_str = {}
glb_insn = False
glb_disassembler = None
glb_src = False
@@ -70,6 +69,7 @@
ap = argparse.ArgumentParser(usage = "", add_help = False)
ap.add_argument("--insn-trace", action='store_true')
ap.add_argument("--src-trace", action='store_true')
+ ap.add_argument("--all-switch-events", action='store_true')
global glb_args
global glb_insn
global glb_src
@@ -256,10 +256,6 @@
print(start_str, src_str)
def do_process_event(param_dict):
- global glb_switch_printed
- if not glb_switch_printed:
- print(glb_switch_str)
- glb_switch_printed = True
event_attr = param_dict["attr"]
sample = param_dict["sample"]
raw_buf = param_dict["raw_buf"]
@@ -274,6 +270,11 @@
dso = get_optional(param_dict, "dso")
symbol = get_optional(param_dict, "symbol")
+ cpu = sample["cpu"]
+ if cpu in glb_switch_str:
+ print(glb_switch_str[cpu])
+ del glb_switch_str[cpu]
+
if name[0:12] == "instructions":
if glb_src:
print_srccode(comm, param_dict, sample, symbol, dso, True)
@@ -336,8 +337,6 @@
sys.exit(1)
def context_switch(ts, cpu, pid, tid, np_pid, np_tid, machine_pid, out, out_preempt, *x):
- global glb_switch_printed
- global glb_switch_str
if out:
out_str = "Switch out "
else:
@@ -350,6 +349,10 @@
machine_str = ""
else:
machine_str = "machine PID %d" % machine_pid
- glb_switch_str = "%16s %5d/%-5d [%03u] %9u.%09u %5d/%-5d %s %s" % \
+ switch_str = "%16s %5d/%-5d [%03u] %9u.%09u %5d/%-5d %s %s" % \
(out_str, pid, tid, cpu, ts / 1000000000, ts %1000000000, np_pid, np_tid, machine_str, preempt_str)
- glb_switch_printed = False
+ if glb_args.all_switch_events:
+ print(switch_str);
+ else:
+ global glb_switch_str
+ glb_switch_str[cpu] = switch_str
diff --git a/tools/perf/tests/bpf.c b/tools/perf/tests/bpf.c
index 329f77f..5734905 100644
--- a/tools/perf/tests/bpf.c
+++ b/tools/perf/tests/bpf.c
@@ -296,9 +296,13 @@ static int check_env(void)
return err;
}
+/* temporarily disable libbpf deprecation warnings */
+#pragma GCC diagnostic push
+#pragma GCC diagnostic ignored "-Wdeprecated-declarations"
err = bpf_load_program(BPF_PROG_TYPE_KPROBE, insns,
ARRAY_SIZE(insns),
license, kver_int, NULL, 0);
+#pragma GCC diagnostic pop
if (err < 0) {
pr_err("Missing basic BPF support, skip this test: %s\n",
strerror(errno));
diff --git a/tools/perf/ui/tui/setup.c b/tools/perf/ui/tui/setup.c
index e9bfe85..b1be59b 100644
--- a/tools/perf/ui/tui/setup.c
+++ b/tools/perf/ui/tui/setup.c
@@ -170,9 +170,11 @@ void ui__exit(bool wait_for_ok)
"Press any key...", 0);
SLtt_set_cursor_visibility(1);
- SLsmg_refresh();
- SLsmg_reset_smg();
+ if (!pthread_mutex_trylock(&ui__lock)) {
+ SLsmg_refresh();
+ SLsmg_reset_smg();
+ pthread_mutex_unlock(&ui__lock);
+ }
SLang_reset_tty();
-
perf_error__unregister(&perf_tui_eops);
}
diff --git a/tools/perf/util/bpf-loader.c b/tools/perf/util/bpf-loader.c
index fbb3c40..528aeb0 100644
--- a/tools/perf/util/bpf-loader.c
+++ b/tools/perf/util/bpf-loader.c
@@ -29,6 +29,9 @@
#include <internal/xyarray.h>
+/* temporarily disable libbpf deprecation warnings */
+#pragma GCC diagnostic ignored "-Wdeprecated-declarations"
+
static int libbpf_perf_print(enum libbpf_print_level level __attribute__((unused)),
const char *fmt, va_list args)
{
diff --git a/tools/perf/util/bpf_counter.c b/tools/perf/util/bpf_counter.c
index c17d4a4..5a97fd7 100644
--- a/tools/perf/util/bpf_counter.c
+++ b/tools/perf/util/bpf_counter.c
@@ -307,6 +307,20 @@ static bool bperf_attr_map_compatible(int attr_map_fd)
(map_info.value_size == sizeof(struct perf_event_attr_map_entry));
}
+int __weak
+bpf_map_create(enum bpf_map_type map_type,
+ const char *map_name __maybe_unused,
+ __u32 key_size,
+ __u32 value_size,
+ __u32 max_entries,
+ const struct bpf_map_create_opts *opts __maybe_unused)
+{
+#pragma GCC diagnostic push
+#pragma GCC diagnostic ignored "-Wdeprecated-declarations"
+ return bpf_create_map(map_type, key_size, value_size, max_entries, 0);
+#pragma GCC diagnostic pop
+}
+
static int bperf_lock_attr_map(struct target *target)
{
char path[PATH_MAX];
@@ -320,10 +334,10 @@ static int bperf_lock_attr_map(struct target *target)
}
if (access(path, F_OK)) {
- map_fd = bpf_create_map(BPF_MAP_TYPE_HASH,
+ map_fd = bpf_map_create(BPF_MAP_TYPE_HASH, NULL,
sizeof(struct perf_event_attr),
sizeof(struct perf_event_attr_map_entry),
- ATTR_MAP_SIZE, 0);
+ ATTR_MAP_SIZE, NULL);
if (map_fd < 0)
return -1;
diff --git a/tools/perf/util/debug.c b/tools/perf/util/debug.c
index c7a9fa0..2c06abf 100644
--- a/tools/perf/util/debug.c
+++ b/tools/perf/util/debug.c
@@ -24,16 +24,6 @@
#include "util/parse-sublevel-options.h"
#include <linux/ctype.h>
-#include <traceevent/event-parse.h>
-
-#define MAKE_LIBTRACEEVENT_VERSION(a, b, c) ((a)*255*255+(b)*255+(c))
-#ifndef LIBTRACEEVENT_VERSION
-/*
- * If LIBTRACEEVENT_VERSION wasn't computed then set to version 1.1.0 that ships
- * with the Linux kernel tools.
- */
-#define LIBTRACEEVENT_VERSION MAKE_LIBTRACEEVENT_VERSION(1, 1, 0)
-#endif
int verbose;
int debug_peo_args;
@@ -238,15 +228,6 @@ int perf_debug_option(const char *str)
/* Allow only verbose value in range (0, 10), otherwise set 0. */
verbose = (verbose < 0) || (verbose > 10) ? 0 : verbose;
-#if MAKE_LIBTRACEEVENT_VERSION(1, 3, 0) <= LIBTRACEEVENT_VERSION
- if (verbose == 1)
- tep_set_loglevel(TEP_LOG_INFO);
- else if (verbose == 2)
- tep_set_loglevel(TEP_LOG_DEBUG);
- else if (verbose >= 3)
- tep_set_loglevel(TEP_LOG_ALL);
-#endif
-
return 0;
}
diff --git a/tools/perf/util/expr.c b/tools/perf/util/expr.c
index 2546010..666b59b 100644
--- a/tools/perf/util/expr.c
+++ b/tools/perf/util/expr.c
@@ -66,7 +66,12 @@ static bool key_equal(const void *key1, const void *key2,
struct hashmap *ids__new(void)
{
- return hashmap__new(key_hash, key_equal, NULL);
+ struct hashmap *hash;
+
+ hash = hashmap__new(key_hash, key_equal, NULL);
+ if (IS_ERR(hash))
+ return NULL;
+ return hash;
}
void ids__free(struct hashmap *ids)
diff --git a/tools/perf/util/intel-pt.c b/tools/perf/util/intel-pt.c
index 10c3187..e8613cb 100644
--- a/tools/perf/util/intel-pt.c
+++ b/tools/perf/util/intel-pt.c
@@ -3625,6 +3625,7 @@ static int intel_pt_parse_vm_tm_corr_arg(struct intel_pt *pt, char **args)
*args = p;
return 0;
}
+ p += 1;
while (1) {
vmcs = strtoull(p, &p, 0);
if (errno)
diff --git a/tools/perf/util/pmu.c b/tools/perf/util/pmu.c
index 6ae5840..8dfbba1 100644
--- a/tools/perf/util/pmu.c
+++ b/tools/perf/util/pmu.c
@@ -1659,6 +1659,21 @@ bool is_pmu_core(const char *name)
return !strcmp(name, "cpu") || is_arm_pmu_core(name);
}
+static bool pmu_alias_is_duplicate(struct sevent *alias_a,
+ struct sevent *alias_b)
+{
+ /* Different names -> never duplicates */
+ if (strcmp(alias_a->name, alias_b->name))
+ return false;
+
+ /* Don't remove duplicates for hybrid PMUs */
+ if (perf_pmu__is_hybrid(alias_a->pmu) &&
+ perf_pmu__is_hybrid(alias_b->pmu))
+ return false;
+
+ return true;
+}
+
void print_pmu_events(const char *event_glob, bool name_only, bool quiet_flag,
bool long_desc, bool details_flag, bool deprecated,
const char *pmu_name)
@@ -1744,12 +1759,8 @@ void print_pmu_events(const char *event_glob, bool name_only, bool quiet_flag,
qsort(aliases, len, sizeof(struct sevent), cmp_sevent);
for (j = 0; j < len; j++) {
/* Skip duplicates */
- if (j > 0 && !strcmp(aliases[j].name, aliases[j - 1].name)) {
- if (!aliases[j].pmu || !aliases[j - 1].pmu ||
- !strcmp(aliases[j].pmu, aliases[j - 1].pmu)) {
- continue;
- }
- }
+ if (j > 0 && pmu_alias_is_duplicate(&aliases[j], &aliases[j - 1]))
+ continue;
if (name_only) {
printf("%s ", aliases[j].name);
diff --git a/tools/scripts/Makefile.include b/tools/scripts/Makefile.include
index 071312f..b0be5f4 100644
--- a/tools/scripts/Makefile.include
+++ b/tools/scripts/Makefile.include
@@ -87,7 +87,18 @@
ifeq ($(CC_NO_CLANG), 1)
EXTRA_WARNINGS += -Wstrict-aliasing=3
-endif
+
+else ifneq ($(CROSS_COMPILE),)
+CLANG_CROSS_FLAGS := --target=$(notdir $(CROSS_COMPILE:%-=%))
+GCC_TOOLCHAIN_DIR := $(dir $(shell which $(CROSS_COMPILE)gcc))
+ifneq ($(GCC_TOOLCHAIN_DIR),)
+CLANG_CROSS_FLAGS += --prefix=$(GCC_TOOLCHAIN_DIR)$(notdir $(CROSS_COMPILE))
+CLANG_CROSS_FLAGS += --sysroot=$(shell $(CROSS_COMPILE)gcc -print-sysroot)
+CLANG_CROSS_FLAGS += --gcc-toolchain=$(realpath $(GCC_TOOLCHAIN_DIR)/..)
+endif # GCC_TOOLCHAIN_DIR
+CFLAGS += $(CLANG_CROSS_FLAGS)
+AFLAGS += $(CLANG_CROSS_FLAGS)
+endif # CROSS_COMPILE
# Hack to avoid type-punned warnings on old systems such as RHEL5:
# We should be changing CFLAGS and checking gcc version, but this
diff --git a/tools/testing/kunit/kunit.py b/tools/testing/kunit/kunit.py
index 68e6f46..7a706f9 100755
--- a/tools/testing/kunit/kunit.py
+++ b/tools/testing/kunit/kunit.py
@@ -15,38 +15,57 @@
assert sys.version_info >= (3, 7), "Python version is too old"
-from collections import namedtuple
+from dataclasses import dataclass
from enum import Enum, auto
-from typing import Iterable, Sequence, List
+from typing import Any, Iterable, Sequence, List, Optional
import kunit_json
import kunit_kernel
import kunit_parser
-KunitResult = namedtuple('KunitResult', ['status','result','elapsed_time'])
-
-KunitConfigRequest = namedtuple('KunitConfigRequest',
- ['build_dir', 'make_options'])
-KunitBuildRequest = namedtuple('KunitBuildRequest',
- ['jobs', 'build_dir', 'alltests',
- 'make_options'])
-KunitExecRequest = namedtuple('KunitExecRequest',
- ['timeout', 'build_dir', 'alltests',
- 'filter_glob', 'kernel_args', 'run_isolated'])
-KunitParseRequest = namedtuple('KunitParseRequest',
- ['raw_output', 'build_dir', 'json'])
-KunitRequest = namedtuple('KunitRequest', ['raw_output','timeout', 'jobs',
- 'build_dir', 'alltests', 'filter_glob',
- 'kernel_args', 'run_isolated', 'json', 'make_options'])
-
-KernelDirectoryPath = sys.argv[0].split('tools/testing/kunit/')[0]
-
class KunitStatus(Enum):
SUCCESS = auto()
CONFIG_FAILURE = auto()
BUILD_FAILURE = auto()
TEST_FAILURE = auto()
+@dataclass
+class KunitResult:
+ status: KunitStatus
+ result: Any
+ elapsed_time: float
+
+@dataclass
+class KunitConfigRequest:
+ build_dir: str
+ make_options: Optional[List[str]]
+
+@dataclass
+class KunitBuildRequest(KunitConfigRequest):
+ jobs: int
+ alltests: bool
+
+@dataclass
+class KunitParseRequest:
+ raw_output: Optional[str]
+ build_dir: str
+ json: Optional[str]
+
+@dataclass
+class KunitExecRequest(KunitParseRequest):
+ timeout: int
+ alltests: bool
+ filter_glob: str
+ kernel_args: Optional[List[str]]
+ run_isolated: Optional[str]
+
+@dataclass
+class KunitRequest(KunitExecRequest, KunitBuildRequest):
+ pass
+
+
+KernelDirectoryPath = sys.argv[0].split('tools/testing/kunit/')[0]
+
def get_kernel_root_path() -> str:
path = sys.argv[0] if not __file__ else __file__
parts = os.path.realpath(path).split('tools/testing/kunit')
@@ -91,6 +110,14 @@
'built kernel successfully',
build_end - build_start)
+def config_and_build_tests(linux: kunit_kernel.LinuxSourceTree,
+ request: KunitBuildRequest) -> KunitResult:
+ config_result = config_tests(linux, request)
+ if config_result.status != KunitStatus.SUCCESS:
+ return config_result
+
+ return build_tests(linux, request)
+
def _list_tests(linux: kunit_kernel.LinuxSourceTree, request: KunitExecRequest) -> List[str]:
args = ['kunit.action=list']
if request.kernel_args:
@@ -121,8 +148,7 @@
-def exec_tests(linux: kunit_kernel.LinuxSourceTree, request: KunitExecRequest,
- parse_request: KunitParseRequest) -> KunitResult:
+def exec_tests(linux: kunit_kernel.LinuxSourceTree, request: KunitExecRequest) -> KunitResult:
filter_globs = [request.filter_glob]
if request.run_isolated:
tests = _list_tests(linux, request)
@@ -147,17 +173,23 @@
filter_glob=filter_glob,
build_dir=request.build_dir)
- result = parse_tests(parse_request, run_result)
+ result = parse_tests(request, run_result)
# run_kernel() doesn't block on the kernel exiting.
# That only happens after we get the last line of output from `run_result`.
# So exec_time here actually contains parsing + execution time, which is fine.
test_end = time.time()
exec_time += test_end - test_start
- test_counts.add_subtest_counts(result.result.test.counts)
+ test_counts.add_subtest_counts(result.result.counts)
+
+ if len(filter_globs) == 1 and test_counts.crashed > 0:
+ bd = request.build_dir
+ print('The kernel seems to have crashed; you can decode the stack traces with:')
+ print('$ scripts/decode_stacktrace.sh {}/vmlinux {} < {} | tee {}/decoded.log | {} parse'.format(
+ bd, bd, kunit_kernel.get_outfile_path(bd), bd, sys.argv[0]))
kunit_status = _map_to_overall_status(test_counts.get_status())
- return KunitResult(status=kunit_status, result=result.result, elapsed_time=exec_time)
+ return KunitResult(status=kunit_status, result=result, elapsed_time=exec_time)
def _map_to_overall_status(test_status: kunit_parser.TestStatus) -> KunitStatus:
if test_status in (kunit_parser.TestStatus.SUCCESS, kunit_parser.TestStatus.SKIPPED):
@@ -168,14 +200,12 @@
def parse_tests(request: KunitParseRequest, input_data: Iterable[str]) -> KunitResult:
parse_start = time.time()
- test_result = kunit_parser.TestResult(kunit_parser.TestStatus.SUCCESS,
- kunit_parser.Test(),
- 'Tests not Parsed.')
+ test_result = kunit_parser.Test()
if request.raw_output:
# Treat unparsed results as one passing test.
- test_result.test.status = kunit_parser.TestStatus.SUCCESS
- test_result.test.counts.passed = 1
+ test_result.status = kunit_parser.TestStatus.SUCCESS
+ test_result.counts.passed = 1
output: Iterable[str] = input_data
if request.raw_output == 'all':
@@ -193,7 +223,7 @@
if request.json:
json_obj = kunit_json.get_json_result(
- test_result=test_result,
+ test=test_result,
def_config='kunit_defconfig',
build_dir=request.build_dir,
json_path=request.json)
@@ -211,27 +241,15 @@
request: KunitRequest) -> KunitResult:
run_start = time.time()
- config_request = KunitConfigRequest(request.build_dir,
- request.make_options)
- config_result = config_tests(linux, config_request)
+ config_result = config_tests(linux, request)
if config_result.status != KunitStatus.SUCCESS:
return config_result
- build_request = KunitBuildRequest(request.jobs, request.build_dir,
- request.alltests,
- request.make_options)
- build_result = build_tests(linux, build_request)
+ build_result = build_tests(linux, request)
if build_result.status != KunitStatus.SUCCESS:
return build_result
- exec_request = KunitExecRequest(request.timeout, request.build_dir,
- request.alltests, request.filter_glob,
- request.kernel_args, request.run_isolated)
- parse_request = KunitParseRequest(request.raw_output,
- request.build_dir,
- request.json)
-
- exec_result = exec_tests(linux, exec_request, parse_request)
+ exec_result = exec_tests(linux, request)
run_end = time.time()
@@ -264,6 +282,9 @@
return f'{arg}={pseudo_bool_flag_defaults[arg]}'
return list(map(massage_arg, argv))
+def get_default_jobs() -> int:
+ return len(os.sched_getaffinity(0))
+
def add_common_opts(parser) -> None:
parser.add_argument('--build_dir',
help='As in the make command, it specifies the build '
@@ -280,6 +301,10 @@
' If given a directory, (e.g. lib/kunit), "/.kunitconfig" '
'will get automatically appended.',
metavar='kunitconfig')
+ parser.add_argument('--kconfig_add',
+ help='Additional Kconfig options to append to the '
+ '.kunitconfig, e.g. CONFIG_KASAN=y. Can be repeated.',
+ action='append')
parser.add_argument('--arch',
help=('Specifies the architecture to run tests under. '
@@ -310,7 +335,7 @@
parser.add_argument('--jobs',
help='As in the make command, "Specifies the number of '
'jobs (commands) to run simultaneously."',
- type=int, default=8, metavar='jobs')
+ type=int, default=get_default_jobs(), metavar='jobs')
def add_exec_opts(parser) -> None:
parser.add_argument('--timeout',
@@ -398,20 +423,21 @@
if not linux:
linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir,
kunitconfig_path=cli_args.kunitconfig,
+ kconfig_add=cli_args.kconfig_add,
arch=cli_args.arch,
cross_compile=cli_args.cross_compile,
qemu_config_path=cli_args.qemu_config)
- request = KunitRequest(cli_args.raw_output,
- cli_args.timeout,
- cli_args.jobs,
- cli_args.build_dir,
- cli_args.alltests,
- cli_args.filter_glob,
- cli_args.kernel_args,
- cli_args.run_isolated,
- cli_args.json,
- cli_args.make_options)
+ request = KunitRequest(build_dir=cli_args.build_dir,
+ make_options=cli_args.make_options,
+ jobs=cli_args.jobs,
+ alltests=cli_args.alltests,
+ raw_output=cli_args.raw_output,
+ json=cli_args.json,
+ timeout=cli_args.timeout,
+ filter_glob=cli_args.filter_glob,
+ kernel_args=cli_args.kernel_args,
+ run_isolated=cli_args.run_isolated)
result = run_tests(linux, request)
if result.status != KunitStatus.SUCCESS:
sys.exit(1)
@@ -423,12 +449,13 @@
if not linux:
linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir,
kunitconfig_path=cli_args.kunitconfig,
+ kconfig_add=cli_args.kconfig_add,
arch=cli_args.arch,
cross_compile=cli_args.cross_compile,
qemu_config_path=cli_args.qemu_config)
- request = KunitConfigRequest(cli_args.build_dir,
- cli_args.make_options)
+ request = KunitConfigRequest(build_dir=cli_args.build_dir,
+ make_options=cli_args.make_options)
result = config_tests(linux, request)
kunit_parser.print_with_timestamp((
'Elapsed time: %.3fs\n') % (
@@ -439,15 +466,16 @@
if not linux:
linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir,
kunitconfig_path=cli_args.kunitconfig,
+ kconfig_add=cli_args.kconfig_add,
arch=cli_args.arch,
cross_compile=cli_args.cross_compile,
qemu_config_path=cli_args.qemu_config)
- request = KunitBuildRequest(cli_args.jobs,
- cli_args.build_dir,
- cli_args.alltests,
- cli_args.make_options)
- result = build_tests(linux, request)
+ request = KunitBuildRequest(build_dir=cli_args.build_dir,
+ make_options=cli_args.make_options,
+ jobs=cli_args.jobs,
+ alltests=cli_args.alltests)
+ result = config_and_build_tests(linux, request)
kunit_parser.print_with_timestamp((
'Elapsed time: %.3fs\n') % (
result.elapsed_time))
@@ -457,20 +485,20 @@
if not linux:
linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir,
kunitconfig_path=cli_args.kunitconfig,
+ kconfig_add=cli_args.kconfig_add,
arch=cli_args.arch,
cross_compile=cli_args.cross_compile,
qemu_config_path=cli_args.qemu_config)
- exec_request = KunitExecRequest(cli_args.timeout,
- cli_args.build_dir,
- cli_args.alltests,
- cli_args.filter_glob,
- cli_args.kernel_args,
- cli_args.run_isolated)
- parse_request = KunitParseRequest(cli_args.raw_output,
- cli_args.build_dir,
- cli_args.json)
- result = exec_tests(linux, exec_request, parse_request)
+ exec_request = KunitExecRequest(raw_output=cli_args.raw_output,
+ build_dir=cli_args.build_dir,
+ json=cli_args.json,
+ timeout=cli_args.timeout,
+ alltests=cli_args.alltests,
+ filter_glob=cli_args.filter_glob,
+ kernel_args=cli_args.kernel_args,
+ run_isolated=cli_args.run_isolated)
+ result = exec_tests(linux, exec_request)
kunit_parser.print_with_timestamp((
'Elapsed time: %.3fs\n') % (result.elapsed_time))
if result.status != KunitStatus.SUCCESS:
@@ -482,9 +510,9 @@
else:
with open(cli_args.file, 'r', errors='backslashreplace') as f:
kunit_output = f.read().splitlines()
- request = KunitParseRequest(cli_args.raw_output,
- None,
- cli_args.json)
+ request = KunitParseRequest(raw_output=cli_args.raw_output,
+ build_dir='',
+ json=cli_args.json)
result = parse_tests(request, kunit_output)
if result.status != KunitStatus.SUCCESS:
sys.exit(1)
diff --git a/tools/testing/kunit/kunit_config.py b/tools/testing/kunit/kunit_config.py
index c77c7d2..6773545 100644
--- a/tools/testing/kunit/kunit_config.py
+++ b/tools/testing/kunit/kunit_config.py
@@ -62,33 +62,34 @@
for entry in self.entries():
f.write(str(entry) + '\n')
- def parse_from_string(self, blob: str) -> None:
- """Parses a string containing KconfigEntrys and populates this Kconfig."""
- self._entries = []
- is_not_set_matcher = re.compile(CONFIG_IS_NOT_SET_PATTERN)
- config_matcher = re.compile(CONFIG_PATTERN)
- for line in blob.split('\n'):
- line = line.strip()
- if not line:
- continue
+def parse_file(path: str) -> Kconfig:
+ with open(path, 'r') as f:
+ return parse_from_string(f.read())
- match = config_matcher.match(line)
- if match:
- entry = KconfigEntry(match.group(1), match.group(2))
- self.add_entry(entry)
- continue
+def parse_from_string(blob: str) -> Kconfig:
+ """Parses a string containing Kconfig entries."""
+ kconfig = Kconfig()
+ is_not_set_matcher = re.compile(CONFIG_IS_NOT_SET_PATTERN)
+ config_matcher = re.compile(CONFIG_PATTERN)
+ for line in blob.split('\n'):
+ line = line.strip()
+ if not line:
+ continue
- empty_match = is_not_set_matcher.match(line)
- if empty_match:
- entry = KconfigEntry(empty_match.group(1), 'n')
- self.add_entry(entry)
- continue
+ match = config_matcher.match(line)
+ if match:
+ entry = KconfigEntry(match.group(1), match.group(2))
+ kconfig.add_entry(entry)
+ continue
- if line[0] == '#':
- continue
- else:
- raise KconfigParseError('Failed to parse: ' + line)
+ empty_match = is_not_set_matcher.match(line)
+ if empty_match:
+ entry = KconfigEntry(empty_match.group(1), 'n')
+ kconfig.add_entry(entry)
+ continue
- def read_from_file(self, path: str) -> None:
- with open(path, 'r') as f:
- self.parse_from_string(f.read())
+ if line[0] == '#':
+ continue
+ else:
+ raise KconfigParseError('Failed to parse: ' + line)
+ return kconfig
diff --git a/tools/testing/kunit/kunit_json.py b/tools/testing/kunit/kunit_json.py
index 746bec7..6862671 100644
--- a/tools/testing/kunit/kunit_json.py
+++ b/tools/testing/kunit/kunit_json.py
@@ -11,7 +11,7 @@
import kunit_parser
-from kunit_parser import Test, TestResult, TestStatus
+from kunit_parser import Test, TestStatus
from typing import Any, Dict, Optional
JsonObj = Dict[str, Any]
@@ -30,6 +30,8 @@
test_case = {"name": subtest.name, "status": "FAIL"}
if subtest.status == TestStatus.SUCCESS:
test_case["status"] = "PASS"
+ elif subtest.status == TestStatus.SKIPPED:
+ test_case["status"] = "SKIP"
elif subtest.status == TestStatus.TEST_CRASHED:
test_case["status"] = "ERROR"
test_cases.append(test_case)
@@ -48,9 +50,9 @@
}
return test_group
-def get_json_result(test_result: TestResult, def_config: str,
+def get_json_result(test: Test, def_config: str,
build_dir: Optional[str], json_path: str) -> str:
- test_group = _get_group_json(test_result.test, def_config, build_dir)
+ test_group = _get_group_json(test, def_config, build_dir)
test_group["name"] = "KUnit Test Group"
json_obj = json.dumps(test_group, indent=4)
if json_path != 'stdout':
diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py
index 6609556..44bbe54 100644
--- a/tools/testing/kunit/kunit_kernel.py
+++ b/tools/testing/kunit/kunit_kernel.py
@@ -21,6 +21,7 @@
KCONFIG_PATH = '.config'
KUNITCONFIG_PATH = '.kunitconfig'
+OLD_KUNITCONFIG_PATH = 'last_used_kunitconfig'
DEFAULT_KUNITCONFIG_PATH = 'tools/testing/kunit/configs/default.config'
BROKEN_ALLCONFIG_PATH = 'tools/testing/kunit/configs/broken_on_uml.config'
OUTFILE_PATH = 'test.log'
@@ -116,8 +117,7 @@
self._extra_qemu_params = qemu_arch_params.extra_qemu_params
def make_arch_qemuconfig(self, base_kunitconfig: kunit_config.Kconfig) -> None:
- kconfig = kunit_config.Kconfig()
- kconfig.parse_from_string(self._kconfig)
+ kconfig = kunit_config.parse_from_string(self._kconfig)
base_kunitconfig.merge_in_entries(kconfig)
def start(self, params: List[str], build_dir: str) -> subprocess.Popen:
@@ -180,6 +180,9 @@
def get_kunitconfig_path(build_dir) -> str:
return get_file_path(build_dir, KUNITCONFIG_PATH)
+def get_old_kunitconfig_path(build_dir) -> str:
+ return get_file_path(build_dir, OLD_KUNITCONFIG_PATH)
+
def get_outfile_path(build_dir) -> str:
return get_file_path(build_dir, OUTFILE_PATH)
@@ -206,6 +209,7 @@
# exists as a file.
module_path = '.' + os.path.join(os.path.basename(QEMU_CONFIGS_DIR), os.path.basename(config_path))
spec = importlib.util.spec_from_file_location(module_path, config_path)
+ assert spec is not None
config = importlib.util.module_from_spec(spec)
# See https://github.com/python/typeshed/pull/2626 for context.
assert isinstance(spec.loader, importlib.abc.Loader)
@@ -225,6 +229,7 @@
build_dir: str,
load_config=True,
kunitconfig_path='',
+ kconfig_add: Optional[List[str]]=None,
arch=None,
cross_compile=None,
qemu_config_path=None) -> None:
@@ -249,8 +254,11 @@
if not os.path.exists(kunitconfig_path):
shutil.copyfile(DEFAULT_KUNITCONFIG_PATH, kunitconfig_path)
- self._kconfig = kunit_config.Kconfig()
- self._kconfig.read_from_file(kunitconfig_path)
+ self._kconfig = kunit_config.parse_file(kunitconfig_path)
+ if kconfig_add:
+ kconfig = kunit_config.parse_from_string('\n'.join(kconfig_add))
+ self._kconfig.merge_in_entries(kconfig)
+
def clean(self) -> bool:
try:
@@ -262,17 +270,18 @@
def validate_config(self, build_dir) -> bool:
kconfig_path = get_kconfig_path(build_dir)
- validated_kconfig = kunit_config.Kconfig()
- validated_kconfig.read_from_file(kconfig_path)
- if not self._kconfig.is_subset_of(validated_kconfig):
- invalid = self._kconfig.entries() - validated_kconfig.entries()
- message = 'Provided Kconfig is not contained in validated .config. Following fields found in kunitconfig, ' \
- 'but not in .config: %s' % (
- ', '.join([str(e) for e in invalid])
- )
- logging.error(message)
- return False
- return True
+ validated_kconfig = kunit_config.parse_file(kconfig_path)
+ if self._kconfig.is_subset_of(validated_kconfig):
+ return True
+ invalid = self._kconfig.entries() - validated_kconfig.entries()
+ message = 'Not all Kconfig options selected in kunitconfig were in the generated .config.\n' \
+ 'This is probably due to unsatisfied dependencies.\n' \
+ 'Missing: ' + ', '.join([str(e) for e in invalid])
+ if self._arch == 'um':
+ message += '\nNote: many Kconfig options aren\'t available on UML. You can try running ' \
+ 'on a different architecture with something like "--arch=x86_64".'
+ logging.error(message)
+ return False
def build_config(self, build_dir, make_options) -> bool:
kconfig_path = get_kconfig_path(build_dir)
@@ -285,25 +294,38 @@
except ConfigError as e:
logging.error(e)
return False
- return self.validate_config(build_dir)
+ if not self.validate_config(build_dir):
+ return False
+
+ old_path = get_old_kunitconfig_path(build_dir)
+ if os.path.exists(old_path):
+ os.remove(old_path) # write_to_file appends to the file
+ self._kconfig.write_to_file(old_path)
+ return True
+
+ def _kunitconfig_changed(self, build_dir: str) -> bool:
+ old_path = get_old_kunitconfig_path(build_dir)
+ if not os.path.exists(old_path):
+ return True
+
+ old_kconfig = kunit_config.parse_file(old_path)
+ return old_kconfig.entries() != self._kconfig.entries()
def build_reconfig(self, build_dir, make_options) -> bool:
"""Creates a new .config if it is not a subset of the .kunitconfig."""
kconfig_path = get_kconfig_path(build_dir)
- if os.path.exists(kconfig_path):
- existing_kconfig = kunit_config.Kconfig()
- existing_kconfig.read_from_file(kconfig_path)
- self._ops.make_arch_qemuconfig(self._kconfig)
- if not self._kconfig.is_subset_of(existing_kconfig):
- print('Regenerating .config ...')
- os.remove(kconfig_path)
- return self.build_config(build_dir, make_options)
- else:
- return True
- else:
+ if not os.path.exists(kconfig_path):
print('Generating .config ...')
return self.build_config(build_dir, make_options)
+ existing_kconfig = kunit_config.parse_file(kconfig_path)
+ self._ops.make_arch_qemuconfig(self._kconfig)
+ if self._kconfig.is_subset_of(existing_kconfig) and not self._kunitconfig_changed(build_dir):
+ return True
+ print('Regenerating .config ...')
+ os.remove(kconfig_path)
+ return self.build_config(build_dir, make_options)
+
def build_kernel(self, alltests, jobs, build_dir, make_options) -> bool:
try:
if alltests:
diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py
index 3355196..05ff334 100644
--- a/tools/testing/kunit/kunit_parser.py
+++ b/tools/testing/kunit/kunit_parser.py
@@ -12,14 +12,11 @@
from __future__ import annotations
import re
-from collections import namedtuple
-from datetime import datetime
+import datetime
from enum import Enum, auto
from functools import reduce
from typing import Iterable, Iterator, List, Optional, Tuple
-TestResult = namedtuple('TestResult', ['status','test','log'])
-
class Test(object):
"""
A class to represent a test parsed from KTAP results. All KTAP
@@ -168,42 +165,51 @@
class LineStream:
"""
A class to represent the lines of kernel output.
- Provides a peek()/pop() interface over an iterator of
+ Provides a lazy peek()/pop() interface over an iterator of
(line#, text).
"""
_lines: Iterator[Tuple[int, str]]
_next: Tuple[int, str]
+ _need_next: bool
_done: bool
def __init__(self, lines: Iterator[Tuple[int, str]]):
"""Creates a new LineStream that wraps the given iterator."""
self._lines = lines
self._done = False
+ self._need_next = True
self._next = (0, '')
- self._get_next()
def _get_next(self) -> None:
- """Advances the LineSteam to the next line."""
+ """Advances the LineSteam to the next line, if necessary."""
+ if not self._need_next:
+ return
try:
self._next = next(self._lines)
except StopIteration:
self._done = True
+ finally:
+ self._need_next = False
def peek(self) -> str:
"""Returns the current line, without advancing the LineStream.
"""
+ self._get_next()
return self._next[1]
def pop(self) -> str:
"""Returns the current line and advances the LineStream to
the next line.
"""
- n = self._next
- self._get_next()
- return n[1]
+ s = self.peek()
+ if self._done:
+ raise ValueError(f'LineStream: going past EOF, last line was {s}')
+ self._need_next = True
+ return s
def __bool__(self) -> bool:
"""Returns True if stream has more lines."""
+ self._get_next()
return not self._done
# Only used by kunit_tool_test.py.
@@ -216,6 +222,7 @@
def line_number(self) -> int:
"""Returns the line number of the current line."""
+ self._get_next()
return self._next[0]
# Parsing helper methods:
@@ -340,8 +347,8 @@
"""
Parses test plan line and stores the expected number of subtests in
test object. Reports an error if expected count is 0.
- Returns False and reports missing test plan error if fails to parse
- test plan.
+ Returns False and sets expected_count to None if there is no valid test
+ plan.
Accepted format:
- '1..[number of subtests]'
@@ -356,14 +363,10 @@
match = TEST_PLAN.match(lines.peek())
if not match:
test.expected_count = None
- test.add_error('missing plan line!')
return False
test.log.append(lines.pop())
expected_count = int(match.group(1))
test.expected_count = expected_count
- if expected_count == 0:
- test.status = TestStatus.NO_TESTS
- test.add_error('0 tests run!')
return True
TEST_RESULT = re.compile(r'^(ok|not ok) ([0-9]+) (- )?([^#]*)( # .*)?$')
@@ -514,7 +517,7 @@
def print_with_timestamp(message: str) -> None:
"""Prints message with timestamp at beginning."""
- print('[%s] %s' % (datetime.now().strftime('%H:%M:%S'), message))
+ print('[%s] %s' % (datetime.datetime.now().strftime('%H:%M:%S'), message))
def format_test_divider(message: str, len_message: int) -> str:
"""
@@ -590,6 +593,8 @@
return (green('[PASSED] ') + test.name)
elif test.status == TestStatus.SKIPPED:
return (yellow('[SKIPPED] ') + test.name)
+ elif test.status == TestStatus.NO_TESTS:
+ return (yellow('[NO TESTS RUN] ') + test.name)
elif test.status == TestStatus.TEST_CRASHED:
print_log(test.log)
return (red('[CRASHED] ') + test.name)
@@ -732,6 +737,7 @@
# test plan
test.name = "main"
parse_test_plan(lines, test)
+ parent_test = True
else:
# If KTAP/TAP header is not found, test must be subtest
# header or test result line so parse attempt to parser
@@ -745,7 +751,7 @@
expected_count = test.expected_count
subtests = []
test_num = 1
- while expected_count is None or test_num <= expected_count:
+ while parent_test and (expected_count is None or test_num <= expected_count):
# Loop to parse any subtests.
# Break after parsing expected number of tests or
# if expected number of tests is unknown break when test
@@ -780,9 +786,15 @@
parse_test_result(lines, test, expected_num)
else:
test.add_error('missing subtest result line!')
+
+ # Check for there being no tests
+ if parent_test and len(subtests) == 0:
+ test.status = TestStatus.NO_TESTS
+ test.add_error('0 tests run!')
+
# Add statuses to TestCounts attribute in Test object
bubble_up_test_results(test)
- if parent_test:
+ if parent_test and not main:
# If test has subtests and is not the main test object, print
# footer.
print_test_footer(test)
@@ -790,7 +802,7 @@
print_test_result(test)
return test
-def parse_run_tests(kernel_output: Iterable[str]) -> TestResult:
+def parse_run_tests(kernel_output: Iterable[str]) -> Test:
"""
Using kernel output, extract KTAP lines, parse the lines for test
results and print condensed test results and summary line .
@@ -799,8 +811,7 @@
kernel_output - Iterable object contains lines of kernel output
Return:
- TestResult - Tuple containg status of main test object, main test
- object with all subtests, and log of all KTAP lines.
+ Test - the main test object with all subtests.
"""
print_with_timestamp(DIVIDER)
lines = extract_tap_lines(kernel_output)
@@ -814,4 +825,4 @@
test.status = test.counts.get_status()
print_with_timestamp(DIVIDER)
print_summary_line(test)
- return TestResult(test.status, test, lines)
+ return test
diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py
index 9c41267..352369d 100755
--- a/tools/testing/kunit/kunit_tool_test.py
+++ b/tools/testing/kunit/kunit_tool_test.py
@@ -13,9 +13,10 @@
import itertools
import json
+import os
import signal
import subprocess
-import os
+from typing import Iterable
import kunit_config
import kunit_parser
@@ -50,10 +51,9 @@
self.assertFalse(kconfig1.is_subset_of(kconfig0))
def test_read_from_file(self):
- kconfig = kunit_config.Kconfig()
kconfig_path = test_data_path('test_read_from_file.kconfig')
- kconfig.read_from_file(kconfig_path)
+ kconfig = kunit_config.parse_file(kconfig_path)
expected_kconfig = kunit_config.Kconfig()
expected_kconfig.add_entry(
@@ -86,8 +86,7 @@
expected_kconfig.write_to_file(kconfig_path)
- actual_kconfig = kunit_config.Kconfig()
- actual_kconfig.read_from_file(kconfig_path)
+ actual_kconfig = kunit_config.parse_file(kconfig_path)
self.assertEqual(actual_kconfig.entries(),
expected_kconfig.entries())
@@ -179,7 +178,7 @@
with open(empty_log) as file:
result = kunit_parser.parse_run_tests(
kunit_parser.extract_tap_lines(file.readlines()))
- self.assertEqual(0, len(result.test.subtests))
+ self.assertEqual(0, len(result.subtests))
self.assertEqual(
kunit_parser.TestStatus.FAILURE_TO_PARSE_TESTS,
result.status)
@@ -191,7 +190,10 @@
result = kunit_parser.parse_run_tests(
kunit_parser.extract_tap_lines(
file.readlines()))
- self.assertEqual(2, result.test.counts.errors)
+ # A missing test plan is not an error.
+ self.assertEqual(0, result.counts.errors)
+ # All tests should be accounted for.
+ self.assertEqual(10, result.counts.total())
self.assertEqual(
kunit_parser.TestStatus.SUCCESS,
result.status)
@@ -201,11 +203,23 @@
with open(header_log) as file:
result = kunit_parser.parse_run_tests(
kunit_parser.extract_tap_lines(file.readlines()))
- self.assertEqual(0, len(result.test.subtests))
+ self.assertEqual(0, len(result.subtests))
self.assertEqual(
kunit_parser.TestStatus.NO_TESTS,
result.status)
+ def test_no_tests_no_plan(self):
+ no_plan_log = test_data_path('test_is_test_passed-no_tests_no_plan.log')
+ with open(no_plan_log) as file:
+ result = kunit_parser.parse_run_tests(
+ kunit_parser.extract_tap_lines(file.readlines()))
+ self.assertEqual(0, len(result.subtests[0].subtests[0].subtests))
+ self.assertEqual(
+ kunit_parser.TestStatus.NO_TESTS,
+ result.subtests[0].subtests[0].status)
+ self.assertEqual(1, result.counts.errors)
+
+
def test_no_kunit_output(self):
crash_log = test_data_path('test_insufficient_memory.log')
print_mock = mock.patch('builtins.print').start()
@@ -214,7 +228,7 @@
kunit_parser.extract_tap_lines(file.readlines()))
print_mock.assert_any_call(StrContains('invalid KTAP input!'))
print_mock.stop()
- self.assertEqual(0, len(result.test.subtests))
+ self.assertEqual(0, len(result.subtests))
def test_crashed_test(self):
crashed_log = test_data_path('test_is_test_passed-crash.log')
@@ -255,10 +269,10 @@
result.status)
self.assertEqual(
"sysctl_test",
- result.test.subtests[0].name)
+ result.subtests[0].name)
self.assertEqual(
"example",
- result.test.subtests[1].name)
+ result.subtests[1].name)
file.close()
@@ -269,7 +283,7 @@
self.assertEqual(
kunit_parser.TestStatus.SUCCESS,
result.status)
- self.assertEqual('kunit-resource-test', result.test.subtests[0].name)
+ self.assertEqual('kunit-resource-test', result.subtests[0].name)
def test_ignores_multiple_prefixes(self):
prefix_log = test_data_path('test_multiple_prefixes.log')
@@ -278,7 +292,7 @@
self.assertEqual(
kunit_parser.TestStatus.SUCCESS,
result.status)
- self.assertEqual('kunit-resource-test', result.test.subtests[0].name)
+ self.assertEqual('kunit-resource-test', result.subtests[0].name)
def test_prefix_mixed_kernel_output(self):
mixed_prefix_log = test_data_path('test_interrupted_tap_output.log')
@@ -287,7 +301,7 @@
self.assertEqual(
kunit_parser.TestStatus.SUCCESS,
result.status)
- self.assertEqual('kunit-resource-test', result.test.subtests[0].name)
+ self.assertEqual('kunit-resource-test', result.subtests[0].name)
def test_prefix_poundsign(self):
pound_log = test_data_path('test_pound_sign.log')
@@ -296,7 +310,7 @@
self.assertEqual(
kunit_parser.TestStatus.SUCCESS,
result.status)
- self.assertEqual('kunit-resource-test', result.test.subtests[0].name)
+ self.assertEqual('kunit-resource-test', result.subtests[0].name)
def test_kernel_panic_end(self):
panic_log = test_data_path('test_kernel_panic_interrupt.log')
@@ -305,7 +319,7 @@
self.assertEqual(
kunit_parser.TestStatus.TEST_CRASHED,
result.status)
- self.assertEqual('kunit-resource-test', result.test.subtests[0].name)
+ self.assertEqual('kunit-resource-test', result.subtests[0].name)
def test_pound_no_prefix(self):
pound_log = test_data_path('test_pound_no_prefix.log')
@@ -314,7 +328,46 @@
self.assertEqual(
kunit_parser.TestStatus.SUCCESS,
result.status)
- self.assertEqual('kunit-resource-test', result.test.subtests[0].name)
+ self.assertEqual('kunit-resource-test', result.subtests[0].name)
+
+def line_stream_from_strs(strs: Iterable[str]) -> kunit_parser.LineStream:
+ return kunit_parser.LineStream(enumerate(strs, start=1))
+
+class LineStreamTest(unittest.TestCase):
+
+ def test_basic(self):
+ stream = line_stream_from_strs(['hello', 'world'])
+
+ self.assertTrue(stream, msg='Should be more input')
+ self.assertEqual(stream.line_number(), 1)
+ self.assertEqual(stream.peek(), 'hello')
+ self.assertEqual(stream.pop(), 'hello')
+
+ self.assertTrue(stream, msg='Should be more input')
+ self.assertEqual(stream.line_number(), 2)
+ self.assertEqual(stream.peek(), 'world')
+ self.assertEqual(stream.pop(), 'world')
+
+ self.assertFalse(stream, msg='Should be no more input')
+ with self.assertRaisesRegex(ValueError, 'LineStream: going past EOF'):
+ stream.pop()
+
+ def test_is_lazy(self):
+ called_times = 0
+ def generator():
+ nonlocal called_times
+ for i in range(1,5):
+ called_times += 1
+ yield called_times, str(called_times)
+
+ stream = kunit_parser.LineStream(generator())
+ self.assertEqual(called_times, 0)
+
+ self.assertEqual(stream.pop(), '1')
+ self.assertEqual(called_times, 1)
+
+ self.assertEqual(stream.pop(), '2')
+ self.assertEqual(called_times, 2)
class LinuxSourceTreeTest(unittest.TestCase):
@@ -336,6 +389,10 @@
pass
kunit_kernel.LinuxSourceTree('', kunitconfig_path=dir)
+ def test_kconfig_add(self):
+ tree = kunit_kernel.LinuxSourceTree('', kconfig_add=['CONFIG_NOT_REAL=y'])
+ self.assertIn(kunit_config.KconfigEntry('NOT_REAL', 'y'), tree._kconfig.entries())
+
def test_invalid_arch(self):
with self.assertRaisesRegex(kunit_kernel.ConfigError, 'not a valid arch, options are.*x86_64'):
kunit_kernel.LinuxSourceTree('', arch='invalid')
@@ -356,6 +413,51 @@
with open(kunit_kernel.get_outfile_path(build_dir), 'rt') as outfile:
self.assertEqual(outfile.read(), 'hi\nbye\n', msg='Missing some output')
+ def test_build_reconfig_no_config(self):
+ with tempfile.TemporaryDirectory('') as build_dir:
+ with open(kunit_kernel.get_kunitconfig_path(build_dir), 'w') as f:
+ f.write('CONFIG_KUNIT=y')
+
+ tree = kunit_kernel.LinuxSourceTree(build_dir)
+ mock_build_config = mock.patch.object(tree, 'build_config').start()
+
+ # Should generate the .config
+ self.assertTrue(tree.build_reconfig(build_dir, make_options=[]))
+ mock_build_config.assert_called_once_with(build_dir, [])
+
+ def test_build_reconfig_existing_config(self):
+ with tempfile.TemporaryDirectory('') as build_dir:
+ # Existing .config is a superset, should not touch it
+ with open(kunit_kernel.get_kunitconfig_path(build_dir), 'w') as f:
+ f.write('CONFIG_KUNIT=y')
+ with open(kunit_kernel.get_old_kunitconfig_path(build_dir), 'w') as f:
+ f.write('CONFIG_KUNIT=y')
+ with open(kunit_kernel.get_kconfig_path(build_dir), 'w') as f:
+ f.write('CONFIG_KUNIT=y\nCONFIG_KUNIT_TEST=y')
+
+ tree = kunit_kernel.LinuxSourceTree(build_dir)
+ mock_build_config = mock.patch.object(tree, 'build_config').start()
+
+ self.assertTrue(tree.build_reconfig(build_dir, make_options=[]))
+ self.assertEqual(mock_build_config.call_count, 0)
+
+ def test_build_reconfig_remove_option(self):
+ with tempfile.TemporaryDirectory('') as build_dir:
+ # We removed CONFIG_KUNIT_TEST=y from our .kunitconfig...
+ with open(kunit_kernel.get_kunitconfig_path(build_dir), 'w') as f:
+ f.write('CONFIG_KUNIT=y')
+ with open(kunit_kernel.get_old_kunitconfig_path(build_dir), 'w') as f:
+ f.write('CONFIG_KUNIT=y\nCONFIG_KUNIT_TEST=y')
+ with open(kunit_kernel.get_kconfig_path(build_dir), 'w') as f:
+ f.write('CONFIG_KUNIT=y\nCONFIG_KUNIT_TEST=y')
+
+ tree = kunit_kernel.LinuxSourceTree(build_dir)
+ mock_build_config = mock.patch.object(tree, 'build_config').start()
+
+ # ... so we should trigger a call to build_config()
+ self.assertTrue(tree.build_reconfig(build_dir, make_options=[]))
+ mock_build_config.assert_called_once_with(build_dir, [])
+
# TODO: add more test cases.
@@ -365,7 +467,7 @@
with open(test_data_path(log_file)) as file:
test_result = kunit_parser.parse_run_tests(file)
json_obj = kunit_json.get_json_result(
- test_result=test_result,
+ test=test_result,
def_config='kunit_defconfig',
build_dir=None,
json_path='stdout')
@@ -383,6 +485,12 @@
{'name': 'example_simple_test', 'status': 'ERROR'},
result["sub_groups"][1]["test_cases"][0])
+ def test_skipped_test_json(self):
+ result = self._json_for('test_skip_tests.log')
+ self.assertEqual(
+ {'name': 'example_skip_test', 'status': 'SKIP'},
+ result["sub_groups"][1]["test_cases"][1])
+
def test_no_tests_json(self):
result = self._json_for('test_is_test_passed-no_tests_run_with_header.log')
self.assertEqual(0, len(result['sub_groups']))
@@ -418,8 +526,8 @@
def test_build_passes_args_pass(self):
kunit.main(['build'], self.linux_source_mock)
- self.assertEqual(self.linux_source_mock.build_reconfig.call_count, 0)
- self.linux_source_mock.build_kernel.assert_called_once_with(False, 8, '.kunit', None)
+ self.assertEqual(self.linux_source_mock.build_reconfig.call_count, 1)
+ self.linux_source_mock.build_kernel.assert_called_once_with(False, kunit.get_default_jobs(), '.kunit', None)
self.assertEqual(self.linux_source_mock.run_kernel.call_count, 0)
def test_exec_passes_args_pass(self):
@@ -525,8 +633,9 @@
def test_build_builddir(self):
build_dir = '.kunit'
+ jobs = kunit.get_default_jobs()
kunit.main(['build', '--build_dir', build_dir], self.linux_source_mock)
- self.linux_source_mock.build_kernel.assert_called_once_with(False, 8, build_dir, None)
+ self.linux_source_mock.build_kernel.assert_called_once_with(False, jobs, build_dir, None)
def test_exec_builddir(self):
build_dir = '.kunit'
@@ -542,6 +651,7 @@
# Just verify that we parsed and initialized it correctly here.
mock_linux_init.assert_called_once_with('.kunit',
kunitconfig_path='mykunitconfig',
+ kconfig_add=None,
arch='um',
cross_compile=None,
qemu_config_path=None)
@@ -553,6 +663,19 @@
# Just verify that we parsed and initialized it correctly here.
mock_linux_init.assert_called_once_with('.kunit',
kunitconfig_path='mykunitconfig',
+ kconfig_add=None,
+ arch='um',
+ cross_compile=None,
+ qemu_config_path=None)
+
+ @mock.patch.object(kunit_kernel, 'LinuxSourceTree')
+ def test_run_kconfig_add(self, mock_linux_init):
+ mock_linux_init.return_value = self.linux_source_mock
+ kunit.main(['run', '--kconfig_add=CONFIG_KASAN=y', '--kconfig_add=CONFIG_KCSAN=y'])
+ # Just verify that we parsed and initialized it correctly here.
+ mock_linux_init.assert_called_once_with('.kunit',
+ kunitconfig_path=None,
+ kconfig_add=['CONFIG_KASAN=y', 'CONFIG_KCSAN=y'],
arch='um',
cross_compile=None,
qemu_config_path=None)
@@ -569,7 +692,7 @@
self.linux_source_mock.run_kernel.return_value = ['TAP version 14', 'init: random output'] + want
got = kunit._list_tests(self.linux_source_mock,
- kunit.KunitExecRequest(300, '.kunit', False, 'suite*', None, 'suite'))
+ kunit.KunitExecRequest(None, '.kunit', None, 300, False, 'suite*', None, 'suite'))
self.assertEqual(got, want)
# Should respect the user's filter glob when listing tests.
@@ -584,7 +707,7 @@
# Should respect the user's filter glob when listing tests.
mock_tests.assert_called_once_with(mock.ANY,
- kunit.KunitExecRequest(300, '.kunit', False, 'suite*.test*', None, 'suite'))
+ kunit.KunitExecRequest(None, '.kunit', None, 300, False, 'suite*.test*', None, 'suite'))
self.linux_source_mock.run_kernel.assert_has_calls([
mock.call(args=None, build_dir='.kunit', filter_glob='suite.test*', timeout=300),
mock.call(args=None, build_dir='.kunit', filter_glob='suite2.test*', timeout=300),
@@ -597,7 +720,7 @@
# Should respect the user's filter glob when listing tests.
mock_tests.assert_called_once_with(mock.ANY,
- kunit.KunitExecRequest(300, '.kunit', False, 'suite*', None, 'test'))
+ kunit.KunitExecRequest(None, '.kunit', None, 300, False, 'suite*', None, 'test'))
self.linux_source_mock.run_kernel.assert_has_calls([
mock.call(args=None, build_dir='.kunit', filter_glob='suite.test1', timeout=300),
mock.call(args=None, build_dir='.kunit', filter_glob='suite.test2', timeout=300),
diff --git a/tools/testing/kunit/run_checks.py b/tools/testing/kunit/run_checks.py
new file mode 100755
index 0000000..4f32133
--- /dev/null
+++ b/tools/testing/kunit/run_checks.py
@@ -0,0 +1,81 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+#
+# This file runs some basic checks to verify kunit works.
+# It is only of interest if you're making changes to KUnit itself.
+#
+# Copyright (C) 2021, Google LLC.
+# Author: Daniel Latypov <dlatypov@google.com.com>
+
+from concurrent import futures
+import datetime
+import os
+import shutil
+import subprocess
+import sys
+import textwrap
+from typing import Dict, List, Sequence, Tuple
+
+ABS_TOOL_PATH = os.path.abspath(os.path.dirname(__file__))
+TIMEOUT = datetime.timedelta(minutes=5).total_seconds()
+
+commands: Dict[str, Sequence[str]] = {
+ 'kunit_tool_test.py': ['./kunit_tool_test.py'],
+ 'kunit smoke test': ['./kunit.py', 'run', '--kunitconfig=lib/kunit', '--build_dir=kunit_run_checks'],
+ 'pytype': ['/bin/sh', '-c', 'pytype *.py'],
+ 'mypy': ['/bin/sh', '-c', 'mypy *.py'],
+}
+
+# The user might not have mypy or pytype installed, skip them if so.
+# Note: you can install both via `$ pip install mypy pytype`
+necessary_deps : Dict[str, str] = {
+ 'pytype': 'pytype',
+ 'mypy': 'mypy',
+}
+
+def main(argv: Sequence[str]) -> None:
+ if argv:
+ raise RuntimeError('This script takes no arguments')
+
+ future_to_name: Dict[futures.Future, str] = {}
+ executor = futures.ThreadPoolExecutor(max_workers=len(commands))
+ for name, argv in commands.items():
+ if name in necessary_deps and shutil.which(necessary_deps[name]) is None:
+ print(f'{name}: SKIPPED, {necessary_deps[name]} not in $PATH')
+ continue
+ f = executor.submit(run_cmd, argv)
+ future_to_name[f] = name
+
+ has_failures = False
+ print(f'Waiting on {len(future_to_name)} checks ({", ".join(future_to_name.values())})...')
+ for f in futures.as_completed(future_to_name.keys()):
+ name = future_to_name[f]
+ ex = f.exception()
+ if not ex:
+ print(f'{name}: PASSED')
+ continue
+
+ has_failures = True
+ if isinstance(ex, subprocess.TimeoutExpired):
+ print(f'{name}: TIMED OUT')
+ elif isinstance(ex, subprocess.CalledProcessError):
+ print(f'{name}: FAILED')
+ else:
+ print('{name}: unexpected exception: {ex}')
+ continue
+
+ output = ex.output
+ if output:
+ print(textwrap.indent(output.decode(), '> '))
+ executor.shutdown()
+
+ if has_failures:
+ sys.exit(1)
+
+
+def run_cmd(argv: Sequence[str]):
+ subprocess.check_output(argv, stderr=subprocess.STDOUT, cwd=ABS_TOOL_PATH, timeout=TIMEOUT)
+
+
+if __name__ == '__main__':
+ main(sys.argv[1:])
diff --git a/tools/testing/kunit/test_data/test_is_test_passed-no_tests_no_plan.log b/tools/testing/kunit/test_data/test_is_test_passed-no_tests_no_plan.log
new file mode 100644
index 0000000..dd873c9
--- /dev/null
+++ b/tools/testing/kunit/test_data/test_is_test_passed-no_tests_no_plan.log
@@ -0,0 +1,7 @@
+TAP version 14
+1..1
+ # Subtest: suite
+ 1..1
+ # Subtest: case
+ ok 1 - case # SKIP
+ok 1 - suite
diff --git a/tools/testing/selftests/arm64/Makefile b/tools/testing/selftests/arm64/Makefile
index ced910f..1e8d9a8 100644
--- a/tools/testing/selftests/arm64/Makefile
+++ b/tools/testing/selftests/arm64/Makefile
@@ -4,7 +4,7 @@
ARCH ?= $(shell uname -m 2>/dev/null || echo not)
ifneq (,$(filter $(ARCH),aarch64 arm64))
-ARM64_SUBTARGETS ?= tags signal pauth fp mte bti
+ARM64_SUBTARGETS ?= tags signal pauth fp mte bti abi
else
ARM64_SUBTARGETS :=
endif
diff --git a/tools/testing/selftests/arm64/abi/.gitignore b/tools/testing/selftests/arm64/abi/.gitignore
new file mode 100644
index 0000000..b79cf58
--- /dev/null
+++ b/tools/testing/selftests/arm64/abi/.gitignore
@@ -0,0 +1 @@
+syscall-abi
diff --git a/tools/testing/selftests/arm64/abi/Makefile b/tools/testing/selftests/arm64/abi/Makefile
new file mode 100644
index 0000000..96eba97
--- /dev/null
+++ b/tools/testing/selftests/arm64/abi/Makefile
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2021 ARM Limited
+
+TEST_GEN_PROGS := syscall-abi
+
+include ../../lib.mk
+
+$(OUTPUT)/syscall-abi: syscall-abi.c syscall-abi-asm.S
diff --git a/tools/testing/selftests/arm64/abi/syscall-abi-asm.S b/tools/testing/selftests/arm64/abi/syscall-abi-asm.S
new file mode 100644
index 0000000..983467c
--- /dev/null
+++ b/tools/testing/selftests/arm64/abi/syscall-abi-asm.S
@@ -0,0 +1,240 @@
+// SPDX-License-Identifier: GPL-2.0-only
+// Copyright (C) 2021 ARM Limited.
+//
+// Assembly portion of the syscall ABI test
+
+//
+// Load values from memory into registers, invoke a syscall and save the
+// register values back to memory for later checking. The syscall to be
+// invoked is configured in x8 of the input GPR data.
+//
+// x0: SVE VL, 0 for FP only
+//
+// GPRs: gpr_in, gpr_out
+// FPRs: fpr_in, fpr_out
+// Zn: z_in, z_out
+// Pn: p_in, p_out
+// FFR: ffr_in, ffr_out
+
+.arch_extension sve
+
+.globl do_syscall
+do_syscall:
+ // Store callee saved registers x19-x29 (80 bytes) plus x0 and x1
+ stp x29, x30, [sp, #-112]!
+ mov x29, sp
+ stp x0, x1, [sp, #16]
+ stp x19, x20, [sp, #32]
+ stp x21, x22, [sp, #48]
+ stp x23, x24, [sp, #64]
+ stp x25, x26, [sp, #80]
+ stp x27, x28, [sp, #96]
+
+ // Load GPRs x8-x28, and save our SP/FP for later comparison
+ ldr x2, =gpr_in
+ add x2, x2, #64
+ ldp x8, x9, [x2], #16
+ ldp x10, x11, [x2], #16
+ ldp x12, x13, [x2], #16
+ ldp x14, x15, [x2], #16
+ ldp x16, x17, [x2], #16
+ ldp x18, x19, [x2], #16
+ ldp x20, x21, [x2], #16
+ ldp x22, x23, [x2], #16
+ ldp x24, x25, [x2], #16
+ ldp x26, x27, [x2], #16
+ ldr x28, [x2], #8
+ str x29, [x2], #8 // FP
+ str x30, [x2], #8 // LR
+
+ // Load FPRs if we're not doing SVE
+ cbnz x0, 1f
+ ldr x2, =fpr_in
+ ldp q0, q1, [x2]
+ ldp q2, q3, [x2, #16 * 2]
+ ldp q4, q5, [x2, #16 * 4]
+ ldp q6, q7, [x2, #16 * 6]
+ ldp q8, q9, [x2, #16 * 8]
+ ldp q10, q11, [x2, #16 * 10]
+ ldp q12, q13, [x2, #16 * 12]
+ ldp q14, q15, [x2, #16 * 14]
+ ldp q16, q17, [x2, #16 * 16]
+ ldp q18, q19, [x2, #16 * 18]
+ ldp q20, q21, [x2, #16 * 20]
+ ldp q22, q23, [x2, #16 * 22]
+ ldp q24, q25, [x2, #16 * 24]
+ ldp q26, q27, [x2, #16 * 26]
+ ldp q28, q29, [x2, #16 * 28]
+ ldp q30, q31, [x2, #16 * 30]
+1:
+
+ // Load the SVE registers if we're doing SVE
+ cbz x0, 1f
+
+ ldr x2, =z_in
+ ldr z0, [x2, #0, MUL VL]
+ ldr z1, [x2, #1, MUL VL]
+ ldr z2, [x2, #2, MUL VL]
+ ldr z3, [x2, #3, MUL VL]
+ ldr z4, [x2, #4, MUL VL]
+ ldr z5, [x2, #5, MUL VL]
+ ldr z6, [x2, #6, MUL VL]
+ ldr z7, [x2, #7, MUL VL]
+ ldr z8, [x2, #8, MUL VL]
+ ldr z9, [x2, #9, MUL VL]
+ ldr z10, [x2, #10, MUL VL]
+ ldr z11, [x2, #11, MUL VL]
+ ldr z12, [x2, #12, MUL VL]
+ ldr z13, [x2, #13, MUL VL]
+ ldr z14, [x2, #14, MUL VL]
+ ldr z15, [x2, #15, MUL VL]
+ ldr z16, [x2, #16, MUL VL]
+ ldr z17, [x2, #17, MUL VL]
+ ldr z18, [x2, #18, MUL VL]
+ ldr z19, [x2, #19, MUL VL]
+ ldr z20, [x2, #20, MUL VL]
+ ldr z21, [x2, #21, MUL VL]
+ ldr z22, [x2, #22, MUL VL]
+ ldr z23, [x2, #23, MUL VL]
+ ldr z24, [x2, #24, MUL VL]
+ ldr z25, [x2, #25, MUL VL]
+ ldr z26, [x2, #26, MUL VL]
+ ldr z27, [x2, #27, MUL VL]
+ ldr z28, [x2, #28, MUL VL]
+ ldr z29, [x2, #29, MUL VL]
+ ldr z30, [x2, #30, MUL VL]
+ ldr z31, [x2, #31, MUL VL]
+
+ ldr x2, =ffr_in
+ ldr p0, [x2, #0]
+ wrffr p0.b
+
+ ldr x2, =p_in
+ ldr p0, [x2, #0, MUL VL]
+ ldr p1, [x2, #1, MUL VL]
+ ldr p2, [x2, #2, MUL VL]
+ ldr p3, [x2, #3, MUL VL]
+ ldr p4, [x2, #4, MUL VL]
+ ldr p5, [x2, #5, MUL VL]
+ ldr p6, [x2, #6, MUL VL]
+ ldr p7, [x2, #7, MUL VL]
+ ldr p8, [x2, #8, MUL VL]
+ ldr p9, [x2, #9, MUL VL]
+ ldr p10, [x2, #10, MUL VL]
+ ldr p11, [x2, #11, MUL VL]
+ ldr p12, [x2, #12, MUL VL]
+ ldr p13, [x2, #13, MUL VL]
+ ldr p14, [x2, #14, MUL VL]
+ ldr p15, [x2, #15, MUL VL]
+1:
+
+ // Do the syscall
+ svc #0
+
+ // Save GPRs x8-x30
+ ldr x2, =gpr_out
+ add x2, x2, #64
+ stp x8, x9, [x2], #16
+ stp x10, x11, [x2], #16
+ stp x12, x13, [x2], #16
+ stp x14, x15, [x2], #16
+ stp x16, x17, [x2], #16
+ stp x18, x19, [x2], #16
+ stp x20, x21, [x2], #16
+ stp x22, x23, [x2], #16
+ stp x24, x25, [x2], #16
+ stp x26, x27, [x2], #16
+ stp x28, x29, [x2], #16
+ str x30, [x2]
+
+ // Restore x0 and x1 for feature checks
+ ldp x0, x1, [sp, #16]
+
+ // Save FPSIMD state
+ ldr x2, =fpr_out
+ stp q0, q1, [x2]
+ stp q2, q3, [x2, #16 * 2]
+ stp q4, q5, [x2, #16 * 4]
+ stp q6, q7, [x2, #16 * 6]
+ stp q8, q9, [x2, #16 * 8]
+ stp q10, q11, [x2, #16 * 10]
+ stp q12, q13, [x2, #16 * 12]
+ stp q14, q15, [x2, #16 * 14]
+ stp q16, q17, [x2, #16 * 16]
+ stp q18, q19, [x2, #16 * 18]
+ stp q20, q21, [x2, #16 * 20]
+ stp q22, q23, [x2, #16 * 22]
+ stp q24, q25, [x2, #16 * 24]
+ stp q26, q27, [x2, #16 * 26]
+ stp q28, q29, [x2, #16 * 28]
+ stp q30, q31, [x2, #16 * 30]
+
+ // Save the SVE state if we have some
+ cbz x0, 1f
+
+ ldr x2, =z_out
+ str z0, [x2, #0, MUL VL]
+ str z1, [x2, #1, MUL VL]
+ str z2, [x2, #2, MUL VL]
+ str z3, [x2, #3, MUL VL]
+ str z4, [x2, #4, MUL VL]
+ str z5, [x2, #5, MUL VL]
+ str z6, [x2, #6, MUL VL]
+ str z7, [x2, #7, MUL VL]
+ str z8, [x2, #8, MUL VL]
+ str z9, [x2, #9, MUL VL]
+ str z10, [x2, #10, MUL VL]
+ str z11, [x2, #11, MUL VL]
+ str z12, [x2, #12, MUL VL]
+ str z13, [x2, #13, MUL VL]
+ str z14, [x2, #14, MUL VL]
+ str z15, [x2, #15, MUL VL]
+ str z16, [x2, #16, MUL VL]
+ str z17, [x2, #17, MUL VL]
+ str z18, [x2, #18, MUL VL]
+ str z19, [x2, #19, MUL VL]
+ str z20, [x2, #20, MUL VL]
+ str z21, [x2, #21, MUL VL]
+ str z22, [x2, #22, MUL VL]
+ str z23, [x2, #23, MUL VL]
+ str z24, [x2, #24, MUL VL]
+ str z25, [x2, #25, MUL VL]
+ str z26, [x2, #26, MUL VL]
+ str z27, [x2, #27, MUL VL]
+ str z28, [x2, #28, MUL VL]
+ str z29, [x2, #29, MUL VL]
+ str z30, [x2, #30, MUL VL]
+ str z31, [x2, #31, MUL VL]
+
+ ldr x2, =p_out
+ str p0, [x2, #0, MUL VL]
+ str p1, [x2, #1, MUL VL]
+ str p2, [x2, #2, MUL VL]
+ str p3, [x2, #3, MUL VL]
+ str p4, [x2, #4, MUL VL]
+ str p5, [x2, #5, MUL VL]
+ str p6, [x2, #6, MUL VL]
+ str p7, [x2, #7, MUL VL]
+ str p8, [x2, #8, MUL VL]
+ str p9, [x2, #9, MUL VL]
+ str p10, [x2, #10, MUL VL]
+ str p11, [x2, #11, MUL VL]
+ str p12, [x2, #12, MUL VL]
+ str p13, [x2, #13, MUL VL]
+ str p14, [x2, #14, MUL VL]
+ str p15, [x2, #15, MUL VL]
+
+ ldr x2, =ffr_out
+ rdffr p0.b
+ str p0, [x2, #0]
+1:
+
+ // Restore callee saved registers x19-x30
+ ldp x19, x20, [sp, #32]
+ ldp x21, x22, [sp, #48]
+ ldp x23, x24, [sp, #64]
+ ldp x25, x26, [sp, #80]
+ ldp x27, x28, [sp, #96]
+ ldp x29, x30, [sp], #112
+
+ ret
diff --git a/tools/testing/selftests/arm64/abi/syscall-abi.c b/tools/testing/selftests/arm64/abi/syscall-abi.c
new file mode 100644
index 0000000..d8eeeaf
--- /dev/null
+++ b/tools/testing/selftests/arm64/abi/syscall-abi.c
@@ -0,0 +1,318 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2021 ARM Limited.
+ */
+
+#include <errno.h>
+#include <stdbool.h>
+#include <stddef.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+#include <sys/auxv.h>
+#include <sys/prctl.h>
+#include <asm/hwcap.h>
+#include <asm/sigcontext.h>
+#include <asm/unistd.h>
+
+#include "../../kselftest.h"
+
+#define ARRAY_SIZE(a) (sizeof(a) / sizeof(a[0]))
+#define NUM_VL ((SVE_VQ_MAX - SVE_VQ_MIN) + 1)
+
+extern void do_syscall(int sve_vl);
+
+static void fill_random(void *buf, size_t size)
+{
+ int i;
+ uint32_t *lbuf = buf;
+
+ /* random() returns a 32 bit number regardless of the size of long */
+ for (i = 0; i < size / sizeof(uint32_t); i++)
+ lbuf[i] = random();
+}
+
+/*
+ * We also repeat the test for several syscalls to try to expose different
+ * behaviour.
+ */
+static struct syscall_cfg {
+ int syscall_nr;
+ const char *name;
+} syscalls[] = {
+ { __NR_getpid, "getpid()" },
+ { __NR_sched_yield, "sched_yield()" },
+};
+
+#define NUM_GPR 31
+uint64_t gpr_in[NUM_GPR];
+uint64_t gpr_out[NUM_GPR];
+
+static void setup_gpr(struct syscall_cfg *cfg, int sve_vl)
+{
+ fill_random(gpr_in, sizeof(gpr_in));
+ gpr_in[8] = cfg->syscall_nr;
+ memset(gpr_out, 0, sizeof(gpr_out));
+}
+
+static int check_gpr(struct syscall_cfg *cfg, int sve_vl)
+{
+ int errors = 0;
+ int i;
+
+ /*
+ * GPR x0-x7 may be clobbered, and all others should be preserved.
+ */
+ for (i = 9; i < ARRAY_SIZE(gpr_in); i++) {
+ if (gpr_in[i] != gpr_out[i]) {
+ ksft_print_msg("%s SVE VL %d mismatch in GPR %d: %llx != %llx\n",
+ cfg->name, sve_vl, i,
+ gpr_in[i], gpr_out[i]);
+ errors++;
+ }
+ }
+
+ return errors;
+}
+
+#define NUM_FPR 32
+uint64_t fpr_in[NUM_FPR * 2];
+uint64_t fpr_out[NUM_FPR * 2];
+
+static void setup_fpr(struct syscall_cfg *cfg, int sve_vl)
+{
+ fill_random(fpr_in, sizeof(fpr_in));
+ memset(fpr_out, 0, sizeof(fpr_out));
+}
+
+static int check_fpr(struct syscall_cfg *cfg, int sve_vl)
+{
+ int errors = 0;
+ int i;
+
+ if (!sve_vl) {
+ for (i = 0; i < ARRAY_SIZE(fpr_in); i++) {
+ if (fpr_in[i] != fpr_out[i]) {
+ ksft_print_msg("%s Q%d/%d mismatch %llx != %llx\n",
+ cfg->name,
+ i / 2, i % 2,
+ fpr_in[i], fpr_out[i]);
+ errors++;
+ }
+ }
+ }
+
+ return errors;
+}
+
+static uint8_t z_zero[__SVE_ZREG_SIZE(SVE_VQ_MAX)];
+uint8_t z_in[SVE_NUM_PREGS * __SVE_ZREG_SIZE(SVE_VQ_MAX)];
+uint8_t z_out[SVE_NUM_PREGS * __SVE_ZREG_SIZE(SVE_VQ_MAX)];
+
+static void setup_z(struct syscall_cfg *cfg, int sve_vl)
+{
+ fill_random(z_in, sizeof(z_in));
+ fill_random(z_out, sizeof(z_out));
+}
+
+static int check_z(struct syscall_cfg *cfg, int sve_vl)
+{
+ size_t reg_size = sve_vl;
+ int errors = 0;
+ int i;
+
+ if (!sve_vl)
+ return 0;
+
+ /*
+ * After a syscall the low 128 bits of the Z registers should
+ * be preserved and the rest be zeroed or preserved.
+ */
+ for (i = 0; i < SVE_NUM_ZREGS; i++) {
+ void *in = &z_in[reg_size * i];
+ void *out = &z_out[reg_size * i];
+
+ if (memcmp(in, out, SVE_VQ_BYTES) != 0) {
+ ksft_print_msg("%s SVE VL %d Z%d low 128 bits changed\n",
+ cfg->name, sve_vl, i);
+ errors++;
+ }
+ }
+
+ return errors;
+}
+
+uint8_t p_in[SVE_NUM_PREGS * __SVE_PREG_SIZE(SVE_VQ_MAX)];
+uint8_t p_out[SVE_NUM_PREGS * __SVE_PREG_SIZE(SVE_VQ_MAX)];
+
+static void setup_p(struct syscall_cfg *cfg, int sve_vl)
+{
+ fill_random(p_in, sizeof(p_in));
+ fill_random(p_out, sizeof(p_out));
+}
+
+static int check_p(struct syscall_cfg *cfg, int sve_vl)
+{
+ size_t reg_size = sve_vq_from_vl(sve_vl) * 2; /* 1 bit per VL byte */
+
+ int errors = 0;
+ int i;
+
+ if (!sve_vl)
+ return 0;
+
+ /* After a syscall the P registers should be preserved or zeroed */
+ for (i = 0; i < SVE_NUM_PREGS * reg_size; i++)
+ if (p_out[i] && (p_in[i] != p_out[i]))
+ errors++;
+ if (errors)
+ ksft_print_msg("%s SVE VL %d predicate registers non-zero\n",
+ cfg->name, sve_vl);
+
+ return errors;
+}
+
+uint8_t ffr_in[__SVE_PREG_SIZE(SVE_VQ_MAX)];
+uint8_t ffr_out[__SVE_PREG_SIZE(SVE_VQ_MAX)];
+
+static void setup_ffr(struct syscall_cfg *cfg, int sve_vl)
+{
+ /*
+ * It is only valid to set a contiguous set of bits starting
+ * at 0. For now since we're expecting this to be cleared by
+ * a syscall just set all bits.
+ */
+ memset(ffr_in, 0xff, sizeof(ffr_in));
+ fill_random(ffr_out, sizeof(ffr_out));
+}
+
+static int check_ffr(struct syscall_cfg *cfg, int sve_vl)
+{
+ size_t reg_size = sve_vq_from_vl(sve_vl) * 2; /* 1 bit per VL byte */
+ int errors = 0;
+ int i;
+
+ if (!sve_vl)
+ return 0;
+
+ /* After a syscall the P registers should be preserved or zeroed */
+ for (i = 0; i < reg_size; i++)
+ if (ffr_out[i] && (ffr_in[i] != ffr_out[i]))
+ errors++;
+ if (errors)
+ ksft_print_msg("%s SVE VL %d FFR non-zero\n",
+ cfg->name, sve_vl);
+
+ return errors;
+}
+
+typedef void (*setup_fn)(struct syscall_cfg *cfg, int sve_vl);
+typedef int (*check_fn)(struct syscall_cfg *cfg, int sve_vl);
+
+/*
+ * Each set of registers has a setup function which is called before
+ * the syscall to fill values in a global variable for loading by the
+ * test code and a check function which validates that the results are
+ * as expected. Vector lengths are passed everywhere, a vector length
+ * of 0 should be treated as do not test.
+ */
+static struct {
+ setup_fn setup;
+ check_fn check;
+} regset[] = {
+ { setup_gpr, check_gpr },
+ { setup_fpr, check_fpr },
+ { setup_z, check_z },
+ { setup_p, check_p },
+ { setup_ffr, check_ffr },
+};
+
+static bool do_test(struct syscall_cfg *cfg, int sve_vl)
+{
+ int errors = 0;
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(regset); i++)
+ regset[i].setup(cfg, sve_vl);
+
+ do_syscall(sve_vl);
+
+ for (i = 0; i < ARRAY_SIZE(regset); i++)
+ errors += regset[i].check(cfg, sve_vl);
+
+ return errors == 0;
+}
+
+static void test_one_syscall(struct syscall_cfg *cfg)
+{
+ int sve_vq, sve_vl;
+
+ /* FPSIMD only case */
+ ksft_test_result(do_test(cfg, 0),
+ "%s FPSIMD\n", cfg->name);
+
+ if (!(getauxval(AT_HWCAP) & HWCAP_SVE))
+ return;
+
+ for (sve_vq = SVE_VQ_MAX; sve_vq > 0; --sve_vq) {
+ sve_vl = prctl(PR_SVE_SET_VL, sve_vq * 16);
+ if (sve_vl == -1)
+ ksft_exit_fail_msg("PR_SVE_SET_VL failed: %s (%d)\n",
+ strerror(errno), errno);
+
+ sve_vl &= PR_SVE_VL_LEN_MASK;
+
+ if (sve_vq != sve_vq_from_vl(sve_vl))
+ sve_vq = sve_vq_from_vl(sve_vl);
+
+ ksft_test_result(do_test(cfg, sve_vl),
+ "%s SVE VL %d\n", cfg->name, sve_vl);
+ }
+}
+
+int sve_count_vls(void)
+{
+ unsigned int vq;
+ int vl_count = 0;
+ int vl;
+
+ if (!(getauxval(AT_HWCAP) & HWCAP_SVE))
+ return 0;
+
+ /*
+ * Enumerate up to SVE_VQ_MAX vector lengths
+ */
+ for (vq = SVE_VQ_MAX; vq > 0; --vq) {
+ vl = prctl(PR_SVE_SET_VL, vq * 16);
+ if (vl == -1)
+ ksft_exit_fail_msg("PR_SVE_SET_VL failed: %s (%d)\n",
+ strerror(errno), errno);
+
+ vl &= PR_SVE_VL_LEN_MASK;
+
+ if (vq != sve_vq_from_vl(vl))
+ vq = sve_vq_from_vl(vl);
+
+ vl_count++;
+ }
+
+ return vl_count;
+}
+
+int main(void)
+{
+ int i;
+
+ srandom(getpid());
+
+ ksft_print_header();
+ ksft_set_plan(ARRAY_SIZE(syscalls) * (sve_count_vls() + 1));
+
+ for (i = 0; i < ARRAY_SIZE(syscalls); i++)
+ test_one_syscall(&syscalls[i]);
+
+ ksft_print_cnts();
+
+ return 0;
+}
diff --git a/tools/testing/selftests/arm64/fp/.gitignore b/tools/testing/selftests/arm64/fp/.gitignore
index b673959..c50d863 100644
--- a/tools/testing/selftests/arm64/fp/.gitignore
+++ b/tools/testing/selftests/arm64/fp/.gitignore
@@ -1,3 +1,4 @@
+fp-pidbench
fpsimd-test
rdvl-sve
sve-probe-vls
diff --git a/tools/testing/selftests/arm64/fp/Makefile b/tools/testing/selftests/arm64/fp/Makefile
index ba1488c..95f0b87 100644
--- a/tools/testing/selftests/arm64/fp/Makefile
+++ b/tools/testing/selftests/arm64/fp/Makefile
@@ -2,13 +2,15 @@
CFLAGS += -I../../../../../usr/include/
TEST_GEN_PROGS := sve-ptrace sve-probe-vls vec-syscfg
-TEST_PROGS_EXTENDED := fpsimd-test fpsimd-stress \
+TEST_PROGS_EXTENDED := fp-pidbench fpsimd-test fpsimd-stress \
rdvl-sve \
sve-test sve-stress \
vlset
all: $(TEST_GEN_PROGS) $(TEST_PROGS_EXTENDED)
+fp-pidbench: fp-pidbench.S asm-utils.o
+ $(CC) -nostdlib $^ -o $@
fpsimd-test: fpsimd-test.o asm-utils.o
$(CC) -nostdlib $^ -o $@
rdvl-sve: rdvl-sve.o rdvl.o
diff --git a/tools/testing/selftests/arm64/fp/fp-pidbench.S b/tools/testing/selftests/arm64/fp/fp-pidbench.S
new file mode 100644
index 0000000..16a4363
--- /dev/null
+++ b/tools/testing/selftests/arm64/fp/fp-pidbench.S
@@ -0,0 +1,71 @@
+// SPDX-License-Identifier: GPL-2.0-only
+// Copyright (C) 2021 ARM Limited.
+// Original author: Mark Brown <broonie@kernel.org>
+//
+// Trivial syscall overhead benchmark.
+//
+// This is implemented in asm to ensure that we don't have any issues with
+// system libraries using instructions that disrupt the test.
+
+#include <asm/unistd.h>
+#include "assembler.h"
+
+.arch_extension sve
+
+.macro test_loop per_loop
+ mov x10, x20
+ mov x8, #__NR_getpid
+ mrs x11, CNTVCT_EL0
+1:
+ \per_loop
+ svc #0
+ sub x10, x10, #1
+ cbnz x10, 1b
+
+ mrs x12, CNTVCT_EL0
+ sub x0, x12, x11
+ bl putdec
+ puts "\n"
+.endm
+
+// Main program entry point
+.globl _start
+function _start
+_start:
+ puts "Iterations per test: "
+ mov x20, #10000
+ lsl x20, x20, #8
+ mov x0, x20
+ bl putdec
+ puts "\n"
+
+ // Test having never used SVE
+ puts "No SVE: "
+ test_loop
+
+ // Check for SVE support - should use hwcap but that's hard in asm
+ mrs x0, ID_AA64PFR0_EL1
+ ubfx x0, x0, #32, #4
+ cbnz x0, 1f
+ puts "System does not support SVE\n"
+ b out
+1:
+
+ // Execute a SVE instruction
+ puts "SVE VL: "
+ rdvl x0, #8
+ bl putdec
+ puts "\n"
+
+ puts "SVE used once: "
+ test_loop
+
+ // Use SVE per syscall
+ puts "SVE used per syscall: "
+ test_loop "rdvl x0, #8"
+
+ // And we're done
+out:
+ mov x0, #0
+ mov x8, #__NR_exit
+ svc #0
diff --git a/tools/testing/selftests/arm64/fp/sve-ptrace.c b/tools/testing/selftests/arm64/fp/sve-ptrace.c
index c4417bc..af798b9 100644
--- a/tools/testing/selftests/arm64/fp/sve-ptrace.c
+++ b/tools/testing/selftests/arm64/fp/sve-ptrace.c
@@ -21,16 +21,37 @@
#include "../../kselftest.h"
-#define VL_TESTS (((SVE_VQ_MAX - SVE_VQ_MIN) + 1) * 3)
-#define FPSIMD_TESTS 5
-
-#define EXPECTED_TESTS (VL_TESTS + FPSIMD_TESTS)
+#define ARRAY_SIZE(a) (sizeof(a) / sizeof(a[0]))
/* <linux/elf.h> and <sys/auxv.h> don't like each other, so: */
#ifndef NT_ARM_SVE
#define NT_ARM_SVE 0x405
#endif
+struct vec_type {
+ const char *name;
+ unsigned long hwcap_type;
+ unsigned long hwcap;
+ int regset;
+ int prctl_set;
+};
+
+static const struct vec_type vec_types[] = {
+ {
+ .name = "SVE",
+ .hwcap_type = AT_HWCAP,
+ .hwcap = HWCAP_SVE,
+ .regset = NT_ARM_SVE,
+ .prctl_set = PR_SVE_SET_VL,
+ },
+};
+
+#define VL_TESTS (((SVE_VQ_MAX - SVE_VQ_MIN) + 1) * 3)
+#define FLAG_TESTS 2
+#define FPSIMD_TESTS 3
+
+#define EXPECTED_TESTS ((VL_TESTS + FLAG_TESTS + FPSIMD_TESTS) * ARRAY_SIZE(vec_types))
+
static void fill_buf(char *buf, size_t size)
{
int i;
@@ -59,7 +80,8 @@ static int get_fpsimd(pid_t pid, struct user_fpsimd_state *fpsimd)
return ptrace(PTRACE_GETREGSET, pid, NT_PRFPREG, &iov);
}
-static struct user_sve_header *get_sve(pid_t pid, void **buf, size_t *size)
+static struct user_sve_header *get_sve(pid_t pid, const struct vec_type *type,
+ void **buf, size_t *size)
{
struct user_sve_header *sve;
void *p;
@@ -80,7 +102,7 @@ static struct user_sve_header *get_sve(pid_t pid, void **buf, size_t *size)
iov.iov_base = *buf;
iov.iov_len = sz;
- if (ptrace(PTRACE_GETREGSET, pid, NT_ARM_SVE, &iov))
+ if (ptrace(PTRACE_GETREGSET, pid, type->regset, &iov))
goto error;
sve = *buf;
@@ -96,17 +118,18 @@ static struct user_sve_header *get_sve(pid_t pid, void **buf, size_t *size)
return NULL;
}
-static int set_sve(pid_t pid, const struct user_sve_header *sve)
+static int set_sve(pid_t pid, const struct vec_type *type,
+ const struct user_sve_header *sve)
{
struct iovec iov;
iov.iov_base = (void *)sve;
iov.iov_len = sve->size;
- return ptrace(PTRACE_SETREGSET, pid, NT_ARM_SVE, &iov);
+ return ptrace(PTRACE_SETREGSET, pid, type->regset, &iov);
}
/* Validate setting and getting the inherit flag */
-static void ptrace_set_get_inherit(pid_t child)
+static void ptrace_set_get_inherit(pid_t child, const struct vec_type *type)
{
struct user_sve_header sve;
struct user_sve_header *new_sve = NULL;
@@ -118,9 +141,10 @@ static void ptrace_set_get_inherit(pid_t child)
sve.size = sizeof(sve);
sve.vl = sve_vl_from_vq(SVE_VQ_MIN);
sve.flags = SVE_PT_VL_INHERIT;
- ret = set_sve(child, &sve);
+ ret = set_sve(child, type, &sve);
if (ret != 0) {
- ksft_test_result_fail("Failed to set SVE_PT_VL_INHERIT\n");
+ ksft_test_result_fail("Failed to set %s SVE_PT_VL_INHERIT\n",
+ type->name);
return;
}
@@ -128,35 +152,39 @@ static void ptrace_set_get_inherit(pid_t child)
* Read back the new register state and verify that we have
* set the flags we expected.
*/
- if (!get_sve(child, (void **)&new_sve, &new_sve_size)) {
- ksft_test_result_fail("Failed to read SVE flags\n");
+ if (!get_sve(child, type, (void **)&new_sve, &new_sve_size)) {
+ ksft_test_result_fail("Failed to read %s SVE flags\n",
+ type->name);
return;
}
ksft_test_result(new_sve->flags & SVE_PT_VL_INHERIT,
- "SVE_PT_VL_INHERIT set\n");
+ "%s SVE_PT_VL_INHERIT set\n", type->name);
/* Now clear */
sve.flags &= ~SVE_PT_VL_INHERIT;
- ret = set_sve(child, &sve);
+ ret = set_sve(child, type, &sve);
if (ret != 0) {
- ksft_test_result_fail("Failed to clear SVE_PT_VL_INHERIT\n");
+ ksft_test_result_fail("Failed to clear %s SVE_PT_VL_INHERIT\n",
+ type->name);
return;
}
- if (!get_sve(child, (void **)&new_sve, &new_sve_size)) {
- ksft_test_result_fail("Failed to read SVE flags\n");
+ if (!get_sve(child, type, (void **)&new_sve, &new_sve_size)) {
+ ksft_test_result_fail("Failed to read %s SVE flags\n",
+ type->name);
return;
}
ksft_test_result(!(new_sve->flags & SVE_PT_VL_INHERIT),
- "SVE_PT_VL_INHERIT cleared\n");
+ "%s SVE_PT_VL_INHERIT cleared\n", type->name);
free(new_sve);
}
/* Validate attempting to set the specfied VL via ptrace */
-static void ptrace_set_get_vl(pid_t child, unsigned int vl, bool *supported)
+static void ptrace_set_get_vl(pid_t child, const struct vec_type *type,
+ unsigned int vl, bool *supported)
{
struct user_sve_header sve;
struct user_sve_header *new_sve = NULL;
@@ -166,10 +194,10 @@ static void ptrace_set_get_vl(pid_t child, unsigned int vl, bool *supported)
*supported = false;
/* Check if the VL is supported in this process */
- prctl_vl = prctl(PR_SVE_SET_VL, vl);
+ prctl_vl = prctl(type->prctl_set, vl);
if (prctl_vl == -1)
- ksft_exit_fail_msg("prctl(PR_SVE_SET_VL) failed: %s (%d)\n",
- strerror(errno), errno);
+ ksft_exit_fail_msg("prctl(PR_%s_SET_VL) failed: %s (%d)\n",
+ type->name, strerror(errno), errno);
/* If the VL is not supported then a supported VL will be returned */
*supported = (prctl_vl == vl);
@@ -178,9 +206,10 @@ static void ptrace_set_get_vl(pid_t child, unsigned int vl, bool *supported)
memset(&sve, 0, sizeof(sve));
sve.size = sizeof(sve);
sve.vl = vl;
- ret = set_sve(child, &sve);
+ ret = set_sve(child, type, &sve);
if (ret != 0) {
- ksft_test_result_fail("Failed to set VL %u\n", vl);
+ ksft_test_result_fail("Failed to set %s VL %u\n",
+ type->name, vl);
return;
}
@@ -188,12 +217,14 @@ static void ptrace_set_get_vl(pid_t child, unsigned int vl, bool *supported)
* Read back the new register state and verify that we have the
* same VL that we got from prctl() on ourselves.
*/
- if (!get_sve(child, (void **)&new_sve, &new_sve_size)) {
- ksft_test_result_fail("Failed to read VL %u\n", vl);
+ if (!get_sve(child, type, (void **)&new_sve, &new_sve_size)) {
+ ksft_test_result_fail("Failed to read %s VL %u\n",
+ type->name, vl);
return;
}
- ksft_test_result(new_sve->vl = prctl_vl, "Set VL %u\n", vl);
+ ksft_test_result(new_sve->vl = prctl_vl, "Set %s VL %u\n",
+ type->name, vl);
free(new_sve);
}
@@ -209,7 +240,7 @@ static void check_u32(unsigned int vl, const char *reg,
}
/* Access the FPSIMD registers via the SVE regset */
-static void ptrace_sve_fpsimd(pid_t child)
+static void ptrace_sve_fpsimd(pid_t child, const struct vec_type *type)
{
void *svebuf = NULL;
size_t svebufsz = 0;
@@ -219,17 +250,18 @@ static void ptrace_sve_fpsimd(pid_t child)
unsigned char *p;
/* New process should start with FPSIMD registers only */
- sve = get_sve(child, &svebuf, &svebufsz);
+ sve = get_sve(child, type, &svebuf, &svebufsz);
if (!sve) {
- ksft_test_result_fail("get_sve: %s\n", strerror(errno));
+ ksft_test_result_fail("get_sve(%s): %s\n",
+ type->name, strerror(errno));
return;
} else {
- ksft_test_result_pass("get_sve(FPSIMD)\n");
+ ksft_test_result_pass("get_sve(%s FPSIMD)\n", type->name);
}
ksft_test_result((sve->flags & SVE_PT_REGS_MASK) == SVE_PT_REGS_FPSIMD,
- "Set FPSIMD registers\n");
+ "Set FPSIMD registers via %s\n", type->name);
if ((sve->flags & SVE_PT_REGS_MASK) != SVE_PT_REGS_FPSIMD)
goto out;
@@ -243,9 +275,9 @@ static void ptrace_sve_fpsimd(pid_t child)
p[j] = j;
}
- if (set_sve(child, sve)) {
- ksft_test_result_fail("set_sve(FPSIMD): %s\n",
- strerror(errno));
+ if (set_sve(child, type, sve)) {
+ ksft_test_result_fail("set_sve(%s FPSIMD): %s\n",
+ type->name, strerror(errno));
goto out;
}
@@ -257,16 +289,20 @@ static void ptrace_sve_fpsimd(pid_t child)
goto out;
}
if (memcmp(fpsimd, &new_fpsimd, sizeof(*fpsimd)) == 0)
- ksft_test_result_pass("get_fpsimd() gave same state\n");
+ ksft_test_result_pass("%s get_fpsimd() gave same state\n",
+ type->name);
else
- ksft_test_result_fail("get_fpsimd() gave different state\n");
+ ksft_test_result_fail("%s get_fpsimd() gave different state\n",
+ type->name);
out:
free(svebuf);
}
/* Validate attempting to set SVE data and read SVE data */
-static void ptrace_set_sve_get_sve_data(pid_t child, unsigned int vl)
+static void ptrace_set_sve_get_sve_data(pid_t child,
+ const struct vec_type *type,
+ unsigned int vl)
{
void *write_buf;
void *read_buf = NULL;
@@ -281,8 +317,8 @@ static void ptrace_set_sve_get_sve_data(pid_t child, unsigned int vl)
data_size = SVE_PT_SVE_OFFSET + SVE_PT_SVE_SIZE(vq, SVE_PT_REGS_SVE);
write_buf = malloc(data_size);
if (!write_buf) {
- ksft_test_result_fail("Error allocating %d byte buffer for VL %u\n",
- data_size, vl);
+ ksft_test_result_fail("Error allocating %d byte buffer for %s VL %u\n",
+ data_size, type->name, vl);
return;
}
write_sve = write_buf;
@@ -306,23 +342,26 @@ static void ptrace_set_sve_get_sve_data(pid_t child, unsigned int vl)
/* TODO: Generate a valid FFR pattern */
- ret = set_sve(child, write_sve);
+ ret = set_sve(child, type, write_sve);
if (ret != 0) {
- ksft_test_result_fail("Failed to set VL %u data\n", vl);
+ ksft_test_result_fail("Failed to set %s VL %u data\n",
+ type->name, vl);
goto out;
}
/* Read the data back */
- if (!get_sve(child, (void **)&read_buf, &read_sve_size)) {
- ksft_test_result_fail("Failed to read VL %u data\n", vl);
+ if (!get_sve(child, type, (void **)&read_buf, &read_sve_size)) {
+ ksft_test_result_fail("Failed to read %s VL %u data\n",
+ type->name, vl);
goto out;
}
read_sve = read_buf;
/* We might read more data if there's extensions we don't know */
if (read_sve->size < write_sve->size) {
- ksft_test_result_fail("Wrote %d bytes, only read %d\n",
- write_sve->size, read_sve->size);
+ ksft_test_result_fail("%s wrote %d bytes, only read %d\n",
+ type->name, write_sve->size,
+ read_sve->size);
goto out_read;
}
@@ -349,7 +388,8 @@ static void ptrace_set_sve_get_sve_data(pid_t child, unsigned int vl)
check_u32(vl, "FPCR", write_buf + SVE_PT_SVE_FPCR_OFFSET(vq),
read_buf + SVE_PT_SVE_FPCR_OFFSET(vq), &errors);
- ksft_test_result(errors == 0, "Set and get SVE data for VL %u\n", vl);
+ ksft_test_result(errors == 0, "Set and get %s data for VL %u\n",
+ type->name, vl);
out_read:
free(read_buf);
@@ -358,7 +398,9 @@ static void ptrace_set_sve_get_sve_data(pid_t child, unsigned int vl)
}
/* Validate attempting to set SVE data and read SVE data */
-static void ptrace_set_sve_get_fpsimd_data(pid_t child, unsigned int vl)
+static void ptrace_set_sve_get_fpsimd_data(pid_t child,
+ const struct vec_type *type,
+ unsigned int vl)
{
void *write_buf;
struct user_sve_header *write_sve;
@@ -376,8 +418,8 @@ static void ptrace_set_sve_get_fpsimd_data(pid_t child, unsigned int vl)
data_size = SVE_PT_SVE_OFFSET + SVE_PT_SVE_SIZE(vq, SVE_PT_REGS_SVE);
write_buf = malloc(data_size);
if (!write_buf) {
- ksft_test_result_fail("Error allocating %d byte buffer for VL %u\n",
- data_size, vl);
+ ksft_test_result_fail("Error allocating %d byte buffer for %s VL %u\n",
+ data_size, type->name, vl);
return;
}
write_sve = write_buf;
@@ -395,16 +437,17 @@ static void ptrace_set_sve_get_fpsimd_data(pid_t child, unsigned int vl)
fill_buf(write_buf + SVE_PT_SVE_FPSR_OFFSET(vq), SVE_PT_SVE_FPSR_SIZE);
fill_buf(write_buf + SVE_PT_SVE_FPCR_OFFSET(vq), SVE_PT_SVE_FPCR_SIZE);
- ret = set_sve(child, write_sve);
+ ret = set_sve(child, type, write_sve);
if (ret != 0) {
- ksft_test_result_fail("Failed to set VL %u data\n", vl);
+ ksft_test_result_fail("Failed to set %s VL %u data\n",
+ type->name, vl);
goto out;
}
/* Read the data back */
if (get_fpsimd(child, &fpsimd_state)) {
- ksft_test_result_fail("Failed to read VL %u FPSIMD data\n",
- vl);
+ ksft_test_result_fail("Failed to read %s VL %u FPSIMD data\n",
+ type->name, vl);
goto out;
}
@@ -419,7 +462,8 @@ static void ptrace_set_sve_get_fpsimd_data(pid_t child, unsigned int vl)
sizeof(tmp));
if (tmp != fpsimd_state.vregs[i]) {
- printf("# Mismatch in FPSIMD for VL %u Z%d\n", vl, i);
+ printf("# Mismatch in FPSIMD for %s VL %u Z%d\n",
+ type->name, vl, i);
errors++;
}
}
@@ -429,8 +473,8 @@ static void ptrace_set_sve_get_fpsimd_data(pid_t child, unsigned int vl)
check_u32(vl, "FPCR", write_buf + SVE_PT_SVE_FPCR_OFFSET(vq),
&fpsimd_state.fpcr, &errors);
- ksft_test_result(errors == 0, "Set and get FPSIMD data for VL %u\n",
- vl);
+ ksft_test_result(errors == 0, "Set and get FPSIMD data for %s VL %u\n",
+ type->name, vl);
out:
free(write_buf);
@@ -440,7 +484,7 @@ static int do_parent(pid_t child)
{
int ret = EXIT_FAILURE;
pid_t pid;
- int status;
+ int status, i;
siginfo_t si;
unsigned int vq, vl;
bool vl_supported;
@@ -499,26 +543,47 @@ static int do_parent(pid_t child)
}
}
- /* FPSIMD via SVE regset */
- ptrace_sve_fpsimd(child);
-
- /* prctl() flags */
- ptrace_set_get_inherit(child);
-
- /* Step through every possible VQ */
- for (vq = SVE_VQ_MIN; vq <= SVE_VQ_MAX; vq++) {
- vl = sve_vl_from_vq(vq);
-
- /* First, try to set this vector length */
- ptrace_set_get_vl(child, vl, &vl_supported);
-
- /* If the VL is supported validate data set/get */
- if (vl_supported) {
- ptrace_set_sve_get_sve_data(child, vl);
- ptrace_set_sve_get_fpsimd_data(child, vl);
+ for (i = 0; i < ARRAY_SIZE(vec_types); i++) {
+ /* FPSIMD via SVE regset */
+ if (getauxval(vec_types[i].hwcap_type) & vec_types[i].hwcap) {
+ ptrace_sve_fpsimd(child, &vec_types[i]);
} else {
- ksft_test_result_skip("set SVE get SVE for VL %d\n", vl);
- ksft_test_result_skip("set SVE get FPSIMD for VL %d\n", vl);
+ ksft_test_result_skip("%s FPSIMD get via SVE\n",
+ vec_types[i].name);
+ ksft_test_result_skip("%s FPSIMD set via SVE\n",
+ vec_types[i].name);
+ ksft_test_result_skip("%s set read via FPSIMD\n",
+ vec_types[i].name);
+ }
+
+ /* prctl() flags */
+ ptrace_set_get_inherit(child, &vec_types[i]);
+
+ /* Step through every possible VQ */
+ for (vq = SVE_VQ_MIN; vq <= SVE_VQ_MAX; vq++) {
+ vl = sve_vl_from_vq(vq);
+
+ /* First, try to set this vector length */
+ if (getauxval(vec_types[i].hwcap_type) &
+ vec_types[i].hwcap) {
+ ptrace_set_get_vl(child, &vec_types[i], vl,
+ &vl_supported);
+ } else {
+ ksft_test_result_skip("%s get/set VL %d\n",
+ vec_types[i].name, vl);
+ vl_supported = false;
+ }
+
+ /* If the VL is supported validate data set/get */
+ if (vl_supported) {
+ ptrace_set_sve_get_sve_data(child, &vec_types[i], vl);
+ ptrace_set_sve_get_fpsimd_data(child, &vec_types[i], vl);
+ } else {
+ ksft_test_result_skip("%s set SVE get SVE for VL %d\n",
+ vec_types[i].name, vl);
+ ksft_test_result_skip("%s set SVE get FPSIMD for VL %d\n",
+ vec_types[i].name, vl);
+ }
}
}
diff --git a/tools/testing/selftests/arm64/fp/vec-syscfg.c b/tools/testing/selftests/arm64/fp/vec-syscfg.c
index 272b888..c906588 100644
--- a/tools/testing/selftests/arm64/fp/vec-syscfg.c
+++ b/tools/testing/selftests/arm64/fp/vec-syscfg.c
@@ -21,8 +21,6 @@
#include "../../kselftest.h"
#include "rdvl.h"
-#define ARRAY_SIZE(a) (sizeof(a) / sizeof(a[0]))
-
#define ARCH_MIN_VL SVE_VL_MIN
struct vec_data {
diff --git a/tools/testing/selftests/arm64/signal/test_signals_utils.c b/tools/testing/selftests/arm64/signal/test_signals_utils.c
index 22722ab..2f8c23a 100644
--- a/tools/testing/selftests/arm64/signal/test_signals_utils.c
+++ b/tools/testing/selftests/arm64/signal/test_signals_utils.c
@@ -310,14 +310,12 @@ int test_setup(struct tdescr *td)
int test_run(struct tdescr *td)
{
- if (td->sig_trig) {
- if (td->trigger)
- return td->trigger(td);
- else
- return default_trigger(td);
- } else {
+ if (td->trigger)
+ return td->trigger(td);
+ else if (td->sig_trig)
+ return default_trigger(td);
+ else
return td->run(td, NULL, NULL);
- }
}
void test_result(struct tdescr *td)
diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index 62fafbe..42ffc24 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -23,9 +23,8 @@
SAN_CFLAGS ?=
CFLAGS += -g -O0 -rdynamic -Wall $(GENFLAGS) $(SAN_CFLAGS) \
-I$(CURDIR) -I$(INCLUDE_DIR) -I$(GENDIR) -I$(LIBDIR) \
- -I$(TOOLSINCDIR) -I$(APIDIR) -I$(OUTPUT) \
- -Dbpf_prog_load=bpf_prog_test_load \
- -Dbpf_load_program=bpf_test_load_program
+ -I$(TOOLSINCDIR) -I$(APIDIR) -I$(OUTPUT)
+LDFLAGS += $(SAN_CFLAGS)
LDLIBS += -lcap -lelf -lz -lrt -lpthread
# Silence some warnings when compiled with clang
@@ -46,10 +45,8 @@
TEST_GEN_PROGS += test_progs-bpf_gcc
endif
-TEST_GEN_FILES = test_lwt_ip_encap.o \
- test_tc_edt.o
-TEST_FILES = xsk_prereqs.sh \
- $(wildcard progs/btf_dump_test_case_*.c)
+TEST_GEN_FILES = test_lwt_ip_encap.o test_tc_edt.o
+TEST_FILES = xsk_prereqs.sh $(wildcard progs/btf_dump_test_case_*.c)
# Order correspond to 'make run_tests' order
TEST_PROGS := test_kmod.sh \
@@ -108,7 +105,10 @@
OVERRIDE_TARGETS := 1
override define CLEAN
$(call msg,CLEAN)
- $(Q)$(RM) -r $(TEST_GEN_PROGS) $(TEST_GEN_PROGS_EXTENDED) $(TEST_GEN_FILES) $(EXTRA_CLEAN)
+ $(Q)$(RM) -r $(TEST_GEN_PROGS)
+ $(Q)$(RM) -r $(TEST_GEN_PROGS_EXTENDED)
+ $(Q)$(RM) -r $(TEST_GEN_FILES)
+ $(Q)$(RM) -r $(EXTRA_CLEAN)
$(Q)$(MAKE) -C bpf_testmod clean
$(Q)$(MAKE) docs-clean
endef
@@ -170,7 +170,7 @@
$(OUTPUT)/urandom_read: urandom_read.c
$(call msg,BINARY,,$@)
- $(Q)$(CC) $(LDFLAGS) -o $@ $< $(LDLIBS) -Wl,--build-id=sha1
+ $(Q)$(CC) $(CFLAGS) $(LDFLAGS) $< $(LDLIBS) -Wl,--build-id=sha1 -o $@
$(OUTPUT)/bpf_testmod.ko: $(VMLINUX_BTF) $(wildcard bpf_testmod/Makefile bpf_testmod/*.[ch])
$(call msg,MOD,,$@)
@@ -178,10 +178,6 @@
$(Q)$(MAKE) $(submake_extras) -C bpf_testmod
$(Q)cp bpf_testmod/bpf_testmod.ko $@
-$(OUTPUT)/test_stub.o: test_stub.c $(BPFOBJ)
- $(call msg,CC,,$@)
- $(Q)$(CC) -c $(CFLAGS) -o $@ $<
-
DEFAULT_BPFTOOL := $(HOST_SCRATCH_DIR)/sbin/bpftool
$(OUTPUT)/runqslower: $(BPFOBJ) | $(DEFAULT_BPFTOOL) $(RUNQSLOWER_OUTPUT)
@@ -194,24 +190,34 @@
TEST_GEN_PROGS_EXTENDED += $(DEFAULT_BPFTOOL)
-$(TEST_GEN_PROGS) $(TEST_GEN_PROGS_EXTENDED): $(OUTPUT)/test_stub.o $(BPFOBJ)
+$(TEST_GEN_PROGS) $(TEST_GEN_PROGS_EXTENDED): $(BPFOBJ)
-$(OUTPUT)/test_dev_cgroup: cgroup_helpers.c
-$(OUTPUT)/test_skb_cgroup_id_user: cgroup_helpers.c
-$(OUTPUT)/test_sock: cgroup_helpers.c
-$(OUTPUT)/test_sock_addr: cgroup_helpers.c
-$(OUTPUT)/test_sockmap: cgroup_helpers.c
-$(OUTPUT)/test_tcpnotify_user: cgroup_helpers.c trace_helpers.c
-$(OUTPUT)/get_cgroup_id_user: cgroup_helpers.c
-$(OUTPUT)/test_cgroup_storage: cgroup_helpers.c
-$(OUTPUT)/test_sock_fields: cgroup_helpers.c
-$(OUTPUT)/test_sysctl: cgroup_helpers.c
+CGROUP_HELPERS := $(OUTPUT)/cgroup_helpers.o
+TESTING_HELPERS := $(OUTPUT)/testing_helpers.o
+TRACE_HELPERS := $(OUTPUT)/trace_helpers.o
+
+$(OUTPUT)/test_dev_cgroup: $(CGROUP_HELPERS) $(TESTING_HELPERS)
+$(OUTPUT)/test_skb_cgroup_id_user: $(CGROUP_HELPERS) $(TESTING_HELPERS)
+$(OUTPUT)/test_sock: $(CGROUP_HELPERS) $(TESTING_HELPERS)
+$(OUTPUT)/test_sock_addr: $(CGROUP_HELPERS) $(TESTING_HELPERS)
+$(OUTPUT)/test_sockmap: $(CGROUP_HELPERS) $(TESTING_HELPERS)
+$(OUTPUT)/test_tcpnotify_user: $(CGROUP_HELPERS) $(TESTING_HELPERS) $(TRACE_HELPERS)
+$(OUTPUT)/get_cgroup_id_user: $(CGROUP_HELPERS) $(TESTING_HELPERS)
+$(OUTPUT)/test_cgroup_storage: $(CGROUP_HELPERS) $(TESTING_HELPERS)
+$(OUTPUT)/test_sock_fields: $(CGROUP_HELPERS) $(TESTING_HELPERS)
+$(OUTPUT)/test_sysctl: $(CGROUP_HELPERS) $(TESTING_HELPERS)
+$(OUTPUT)/test_tag: $(TESTING_HELPERS)
+$(OUTPUT)/test_lirc_mode2_user: $(TESTING_HELPERS)
+$(OUTPUT)/xdping: $(TESTING_HELPERS)
+$(OUTPUT)/flow_dissector_load: $(TESTING_HELPERS)
+$(OUTPUT)/test_maps: $(TESTING_HELPERS)
+$(OUTPUT)/test_verifier: $(TESTING_HELPERS)
BPFTOOL ?= $(DEFAULT_BPFTOOL)
$(DEFAULT_BPFTOOL): $(wildcard $(BPFTOOLDIR)/*.[ch] $(BPFTOOLDIR)/Makefile) \
$(HOST_BPFOBJ) | $(HOST_BUILD_DIR)/bpftool
$(Q)$(MAKE) $(submake_extras) -C $(BPFTOOLDIR) \
- CC=$(HOSTCC) LD=$(HOSTLD) \
+ ARCH= CROSS_COMPILE= CC=$(HOSTCC) LD=$(HOSTLD) \
EXTRA_CFLAGS='-g -O0' \
OUTPUT=$(HOST_BUILD_DIR)/bpftool/ \
LIBBPF_OUTPUT=$(HOST_BUILD_DIR)/libbpf/ \
@@ -231,18 +237,18 @@
prefix= OUTPUT=$(OUTPUT)/ DESTDIR=$(OUTPUT)/ $@
$(BPFOBJ): $(wildcard $(BPFDIR)/*.[ch] $(BPFDIR)/Makefile) \
- ../../../include/uapi/linux/bpf.h \
+ $(APIDIR)/linux/bpf.h \
| $(BUILD_DIR)/libbpf
$(Q)$(MAKE) $(submake_extras) -C $(BPFDIR) OUTPUT=$(BUILD_DIR)/libbpf/ \
EXTRA_CFLAGS='-g -O0' \
DESTDIR=$(SCRATCH_DIR) prefix= all install_headers
ifneq ($(BPFOBJ),$(HOST_BPFOBJ))
-$(HOST_BPFOBJ): $(wildcard $(BPFDIR)/*.[ch] $(BPFDIR)/Makefile) \
- ../../../include/uapi/linux/bpf.h \
- | $(HOST_BUILD_DIR)/libbpf
+$(HOST_BPFOBJ): $(wildcard $(BPFDIR)/*.[ch] $(BPFDIR)/Makefile) \
+ $(APIDIR)/linux/bpf.h \
+ | $(HOST_BUILD_DIR)/libbpf
$(Q)$(MAKE) $(submake_extras) -C $(BPFDIR) \
- EXTRA_CFLAGS='-g -O0' \
+ EXTRA_CFLAGS='-g -O0' ARCH= CROSS_COMPILE= \
OUTPUT=$(HOST_BUILD_DIR)/libbpf/ CC=$(HOSTCC) LD=$(HOSTLD) \
DESTDIR=$(HOST_SCRATCH_DIR)/ prefix= all install_headers
endif
@@ -304,12 +310,12 @@
# $3 - CFLAGS
define CLANG_BPF_BUILD_RULE
$(call msg,CLNG-BPF,$(TRUNNER_BINARY),$2)
- $(Q)$(CLANG) $3 -O2 -target bpf -c $1 -o $2 -mcpu=v3
+ $(Q)$(CLANG) $3 -O2 -target bpf -c $1 -mcpu=v3 -o $2
endef
# Similar to CLANG_BPF_BUILD_RULE, but with disabled alu32
define CLANG_NOALU32_BPF_BUILD_RULE
$(call msg,CLNG-BPF,$(TRUNNER_BINARY),$2)
- $(Q)$(CLANG) $3 -O2 -target bpf -c $1 -o $2 -mcpu=v2
+ $(Q)$(CLANG) $3 -O2 -target bpf -c $1 -mcpu=v2 -o $2
endef
# Build BPF object using GCC
define GCC_BPF_BUILD_RULE
@@ -323,9 +329,10 @@
linked_vars.skel.h linked_maps.skel.h
LSKELS := kfunc_call_test.c fentry_test.c fexit_test.c fexit_sleep.c \
- test_ringbuf.c atomics.c trace_printk.c trace_vprintk.c
+ test_ringbuf.c atomics.c trace_printk.c trace_vprintk.c \
+ map_ptr_kern.c core_kern.c
# Generate both light skeleton and libbpf skeleton for these
-LSKELS_EXTRA := test_ksyms_module.c test_ksyms_weak.c
+LSKELS_EXTRA := test_ksyms_module.c test_ksyms_weak.c kfunc_call_test_subprog.c
SKEL_BLACKLIST += $$(LSKELS)
test_static_linked.skel.h-deps := test_static_linked1.o test_static_linked2.o
@@ -471,13 +478,12 @@
TRUNNER_BPF_PROGS_DIR := progs
TRUNNER_EXTRA_SOURCES := test_progs.c cgroup_helpers.c trace_helpers.c \
network_helpers.c testing_helpers.c \
- btf_helpers.c flow_dissector_load.h
+ btf_helpers.c flow_dissector_load.h
TRUNNER_EXTRA_FILES := $(OUTPUT)/urandom_read $(OUTPUT)/bpf_testmod.ko \
ima_setup.sh \
$(wildcard progs/btf_dump_test_case_*.c)
TRUNNER_BPF_BUILD_RULE := CLANG_BPF_BUILD_RULE
-TRUNNER_BPF_CFLAGS := $(BPF_CFLAGS) $(CLANG_CFLAGS)
-TRUNNER_BPF_CFLAGS += -DENABLE_ATOMICS_TESTS
+TRUNNER_BPF_CFLAGS := $(BPF_CFLAGS) $(CLANG_CFLAGS) -DENABLE_ATOMICS_TESTS
$(eval $(call DEFINE_TEST_RUNNER,test_progs))
# Define test_progs-no_alu32 test runner.
@@ -530,16 +536,22 @@
$(OUTPUT)/bench_ringbufs.o: $(OUTPUT)/ringbuf_bench.skel.h \
$(OUTPUT)/perfbuf_bench.skel.h
$(OUTPUT)/bench_bloom_filter_map.o: $(OUTPUT)/bloom_filter_bench.skel.h
+$(OUTPUT)/bench_bpf_loop.o: $(OUTPUT)/bpf_loop_bench.skel.h
+$(OUTPUT)/bench_strncmp.o: $(OUTPUT)/strncmp_bench.skel.h
$(OUTPUT)/bench.o: bench.h testing_helpers.h $(BPFOBJ)
$(OUTPUT)/bench: LDLIBS += -lm
-$(OUTPUT)/bench: $(OUTPUT)/bench.o $(OUTPUT)/testing_helpers.o \
+$(OUTPUT)/bench: $(OUTPUT)/bench.o \
+ $(TESTING_HELPERS) \
+ $(TRACE_HELPERS) \
$(OUTPUT)/bench_count.o \
$(OUTPUT)/bench_rename.o \
$(OUTPUT)/bench_trigger.o \
$(OUTPUT)/bench_ringbufs.o \
- $(OUTPUT)/bench_bloom_filter_map.o
+ $(OUTPUT)/bench_bloom_filter_map.o \
+ $(OUTPUT)/bench_bpf_loop.o \
+ $(OUTPUT)/bench_strncmp.o
$(call msg,BINARY,,$@)
- $(Q)$(CC) $(LDFLAGS) -o $@ $(filter %.a %.o,$^) $(LDLIBS)
+ $(Q)$(CC) $(CFLAGS) $(LDFLAGS) $(filter %.a %.o,$^) $(LDLIBS) -o $@
EXTRA_CLEAN := $(TEST_CUSTOM_PROGS) $(SCRATCH_DIR) $(HOST_SCRATCH_DIR) \
prog_tests/tests.h map_tests/tests.h verifier/tests.h \
diff --git a/tools/testing/selftests/bpf/README.rst b/tools/testing/selftests/bpf/README.rst
index 5e287e4..42ef250 100644
--- a/tools/testing/selftests/bpf/README.rst
+++ b/tools/testing/selftests/bpf/README.rst
@@ -204,16 +204,17 @@
btf_tag test and Clang version
==============================
-The btf_tag selftest require LLVM support to recognize the btf_decl_tag attribute.
-It was introduced in `Clang 14`__.
+The btf_tag selftest requires LLVM support to recognize the btf_decl_tag and
+btf_type_tag attributes. They are introduced in `Clang 14` [0_, 1_].
-Without it, the btf_tag selftest will be skipped and you will observe:
+Without them, the btf_tag selftest will be skipped and you will observe:
.. code-block:: console
#<test_num> btf_tag:SKIP
-__ https://reviews.llvm.org/D111588
+.. _0: https://reviews.llvm.org/D111588
+.. _1: https://reviews.llvm.org/D111199
Clang dependencies for static linking tests
===========================================
diff --git a/tools/testing/selftests/bpf/bench.c b/tools/testing/selftests/bpf/bench.c
index cc4722f..f973320 100644
--- a/tools/testing/selftests/bpf/bench.c
+++ b/tools/testing/selftests/bpf/bench.c
@@ -29,26 +29,10 @@ static int libbpf_print_fn(enum libbpf_print_level level,
return vfprintf(stderr, format, args);
}
-static int bump_memlock_rlimit(void)
+void setup_libbpf(void)
{
- struct rlimit rlim_new = {
- .rlim_cur = RLIM_INFINITY,
- .rlim_max = RLIM_INFINITY,
- };
-
- return setrlimit(RLIMIT_MEMLOCK, &rlim_new);
-}
-
-void setup_libbpf()
-{
- int err;
-
libbpf_set_strict_mode(LIBBPF_STRICT_ALL);
libbpf_set_print(libbpf_print_fn);
-
- err = bump_memlock_rlimit();
- if (err)
- fprintf(stderr, "failed to increase RLIMIT_MEMLOCK: %d", err);
}
void false_hits_report_progress(int iter, struct bench_res *res, long delta_ns)
@@ -134,6 +118,39 @@ void hits_drops_report_final(struct bench_res res[], int res_cnt)
total_ops_mean, total_ops_stddev);
}
+void ops_report_progress(int iter, struct bench_res *res, long delta_ns)
+{
+ double hits_per_sec, hits_per_prod;
+
+ hits_per_sec = res->hits / 1000000.0 / (delta_ns / 1000000000.0);
+ hits_per_prod = hits_per_sec / env.producer_cnt;
+
+ printf("Iter %3d (%7.3lfus): ", iter, (delta_ns - 1000000000) / 1000.0);
+
+ printf("hits %8.3lfM/s (%7.3lfM/prod)\n", hits_per_sec, hits_per_prod);
+}
+
+void ops_report_final(struct bench_res res[], int res_cnt)
+{
+ double hits_mean = 0.0, hits_stddev = 0.0;
+ int i;
+
+ for (i = 0; i < res_cnt; i++)
+ hits_mean += res[i].hits / 1000000.0 / (0.0 + res_cnt);
+
+ if (res_cnt > 1) {
+ for (i = 0; i < res_cnt; i++)
+ hits_stddev += (hits_mean - res[i].hits / 1000000.0) *
+ (hits_mean - res[i].hits / 1000000.0) /
+ (res_cnt - 1.0);
+
+ hits_stddev = sqrt(hits_stddev);
+ }
+ printf("Summary: throughput %8.3lf \u00B1 %5.3lf M ops/s (%7.3lfM ops/prod), ",
+ hits_mean, hits_stddev, hits_mean / env.producer_cnt);
+ printf("latency %8.3lf ns/op\n", 1000.0 / hits_mean * env.producer_cnt);
+}
+
const char *argp_program_version = "benchmark";
const char *argp_program_bug_address = "<bpf@vger.kernel.org>";
const char argp_program_doc[] =
@@ -171,10 +188,14 @@ static const struct argp_option opts[] = {
extern struct argp bench_ringbufs_argp;
extern struct argp bench_bloom_map_argp;
+extern struct argp bench_bpf_loop_argp;
+extern struct argp bench_strncmp_argp;
static const struct argp_child bench_parsers[] = {
{ &bench_ringbufs_argp, 0, "Ring buffers benchmark", 0 },
{ &bench_bloom_map_argp, 0, "Bloom filter map benchmark", 0 },
+ { &bench_bpf_loop_argp, 0, "bpf_loop helper benchmark", 0 },
+ { &bench_strncmp_argp, 0, "bpf_strncmp helper benchmark", 0 },
{},
};
@@ -359,6 +380,11 @@ extern const struct bench bench_trig_kprobe;
extern const struct bench bench_trig_fentry;
extern const struct bench bench_trig_fentry_sleep;
extern const struct bench bench_trig_fmodret;
+extern const struct bench bench_trig_uprobe_base;
+extern const struct bench bench_trig_uprobe_with_nop;
+extern const struct bench bench_trig_uretprobe_with_nop;
+extern const struct bench bench_trig_uprobe_without_nop;
+extern const struct bench bench_trig_uretprobe_without_nop;
extern const struct bench bench_rb_libbpf;
extern const struct bench bench_rb_custom;
extern const struct bench bench_pb_libbpf;
@@ -368,6 +394,9 @@ extern const struct bench bench_bloom_update;
extern const struct bench bench_bloom_false_positive;
extern const struct bench bench_hashmap_without_bloom;
extern const struct bench bench_hashmap_with_bloom;
+extern const struct bench bench_bpf_loop;
+extern const struct bench bench_strncmp_no_helper;
+extern const struct bench bench_strncmp_helper;
static const struct bench *benchs[] = {
&bench_count_global,
@@ -385,6 +414,11 @@ static const struct bench *benchs[] = {
&bench_trig_fentry,
&bench_trig_fentry_sleep,
&bench_trig_fmodret,
+ &bench_trig_uprobe_base,
+ &bench_trig_uprobe_with_nop,
+ &bench_trig_uretprobe_with_nop,
+ &bench_trig_uprobe_without_nop,
+ &bench_trig_uretprobe_without_nop,
&bench_rb_libbpf,
&bench_rb_custom,
&bench_pb_libbpf,
@@ -394,6 +428,9 @@ static const struct bench *benchs[] = {
&bench_bloom_false_positive,
&bench_hashmap_without_bloom,
&bench_hashmap_with_bloom,
+ &bench_bpf_loop,
+ &bench_strncmp_no_helper,
+ &bench_strncmp_helper,
};
static void setup_benchmark()
diff --git a/tools/testing/selftests/bpf/bench.h b/tools/testing/selftests/bpf/bench.h
index 624c6b1..fb3e213 100644
--- a/tools/testing/selftests/bpf/bench.h
+++ b/tools/testing/selftests/bpf/bench.h
@@ -38,8 +38,8 @@ struct bench_res {
struct bench {
const char *name;
- void (*validate)();
- void (*setup)();
+ void (*validate)(void);
+ void (*setup)(void);
void *(*producer_thread)(void *ctx);
void *(*consumer_thread)(void *ctx);
void (*measure)(struct bench_res* res);
@@ -54,13 +54,16 @@ struct counter {
extern struct env env;
extern const struct bench *bench;
-void setup_libbpf();
+void setup_libbpf(void);
void hits_drops_report_progress(int iter, struct bench_res *res, long delta_ns);
void hits_drops_report_final(struct bench_res res[], int res_cnt);
void false_hits_report_progress(int iter, struct bench_res *res, long delta_ns);
void false_hits_report_final(struct bench_res res[], int res_cnt);
+void ops_report_progress(int iter, struct bench_res *res, long delta_ns);
+void ops_report_final(struct bench_res res[], int res_cnt);
-static inline __u64 get_time_ns() {
+static inline __u64 get_time_ns(void)
+{
struct timespec t;
clock_gettime(CLOCK_MONOTONIC, &t);
diff --git a/tools/testing/selftests/bpf/benchs/bench_bloom_filter_map.c b/tools/testing/selftests/bpf/benchs/bench_bloom_filter_map.c
index 6eeeed2..5bcb8a8 100644
--- a/tools/testing/selftests/bpf/benchs/bench_bloom_filter_map.c
+++ b/tools/testing/selftests/bpf/benchs/bench_bloom_filter_map.c
@@ -63,29 +63,34 @@ static const struct argp_option opts[] = {
static error_t parse_arg(int key, char *arg, struct argp_state *state)
{
+ long ret;
+
switch (key) {
case ARG_NR_ENTRIES:
- args.nr_entries = strtol(arg, NULL, 10);
- if (args.nr_entries == 0) {
+ ret = strtol(arg, NULL, 10);
+ if (ret < 1 || ret > UINT_MAX) {
fprintf(stderr, "Invalid nr_entries count.");
argp_usage(state);
}
+ args.nr_entries = ret;
break;
case ARG_NR_HASH_FUNCS:
- args.nr_hash_funcs = strtol(arg, NULL, 10);
- if (args.nr_hash_funcs == 0 || args.nr_hash_funcs > 15) {
+ ret = strtol(arg, NULL, 10);
+ if (ret < 1 || ret > 15) {
fprintf(stderr,
"The bloom filter must use 1 to 15 hash functions.");
argp_usage(state);
}
+ args.nr_hash_funcs = ret;
break;
case ARG_VALUE_SIZE:
- args.value_size = strtol(arg, NULL, 10);
- if (args.value_size < 2 || args.value_size > 256) {
+ ret = strtol(arg, NULL, 10);
+ if (ret < 2 || ret > 256) {
fprintf(stderr,
"Invalid value size. Must be between 2 and 256 bytes");
argp_usage(state);
}
+ args.value_size = ret;
break;
default:
return ARGP_ERR_UNKNOWN;
diff --git a/tools/testing/selftests/bpf/benchs/bench_bpf_loop.c b/tools/testing/selftests/bpf/benchs/bench_bpf_loop.c
new file mode 100644
index 0000000..d0a6572
--- /dev/null
+++ b/tools/testing/selftests/bpf/benchs/bench_bpf_loop.c
@@ -0,0 +1,105 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2021 Facebook */
+
+#include <argp.h>
+#include "bench.h"
+#include "bpf_loop_bench.skel.h"
+
+/* BPF triggering benchmarks */
+static struct ctx {
+ struct bpf_loop_bench *skel;
+} ctx;
+
+static struct {
+ __u32 nr_loops;
+} args = {
+ .nr_loops = 10,
+};
+
+enum {
+ ARG_NR_LOOPS = 4000,
+};
+
+static const struct argp_option opts[] = {
+ { "nr_loops", ARG_NR_LOOPS, "nr_loops", 0,
+ "Set number of loops for the bpf_loop helper"},
+ {},
+};
+
+static error_t parse_arg(int key, char *arg, struct argp_state *state)
+{
+ switch (key) {
+ case ARG_NR_LOOPS:
+ args.nr_loops = strtol(arg, NULL, 10);
+ break;
+ default:
+ return ARGP_ERR_UNKNOWN;
+ }
+
+ return 0;
+}
+
+/* exported into benchmark runner */
+const struct argp bench_bpf_loop_argp = {
+ .options = opts,
+ .parser = parse_arg,
+};
+
+static void validate(void)
+{
+ if (env.consumer_cnt != 1) {
+ fprintf(stderr, "benchmark doesn't support multi-consumer!\n");
+ exit(1);
+ }
+}
+
+static void *producer(void *input)
+{
+ while (true)
+ /* trigger the bpf program */
+ syscall(__NR_getpgid);
+
+ return NULL;
+}
+
+static void *consumer(void *input)
+{
+ return NULL;
+}
+
+static void measure(struct bench_res *res)
+{
+ res->hits = atomic_swap(&ctx.skel->bss->hits, 0);
+}
+
+static void setup(void)
+{
+ struct bpf_link *link;
+
+ setup_libbpf();
+
+ ctx.skel = bpf_loop_bench__open_and_load();
+ if (!ctx.skel) {
+ fprintf(stderr, "failed to open skeleton\n");
+ exit(1);
+ }
+
+ link = bpf_program__attach(ctx.skel->progs.benchmark);
+ if (!link) {
+ fprintf(stderr, "failed to attach program!\n");
+ exit(1);
+ }
+
+ ctx.skel->bss->nr_loops = args.nr_loops;
+}
+
+const struct bench bench_bpf_loop = {
+ .name = "bpf-loop",
+ .validate = validate,
+ .setup = setup,
+ .producer_thread = producer,
+ .consumer_thread = consumer,
+ .measure = measure,
+ .report_progress = ops_report_progress,
+ .report_final = ops_report_final,
+};
diff --git a/tools/testing/selftests/bpf/benchs/bench_count.c b/tools/testing/selftests/bpf/benchs/bench_count.c
index befba7a..078972c 100644
--- a/tools/testing/selftests/bpf/benchs/bench_count.c
+++ b/tools/testing/selftests/bpf/benchs/bench_count.c
@@ -36,7 +36,7 @@ static struct count_local_ctx {
struct counter *hits;
} count_local_ctx;
-static void count_local_setup()
+static void count_local_setup(void)
{
struct count_local_ctx *ctx = &count_local_ctx;
diff --git a/tools/testing/selftests/bpf/benchs/bench_rename.c b/tools/testing/selftests/bpf/benchs/bench_rename.c
index c7ec114..3c203b6 100644
--- a/tools/testing/selftests/bpf/benchs/bench_rename.c
+++ b/tools/testing/selftests/bpf/benchs/bench_rename.c
@@ -11,7 +11,7 @@ static struct ctx {
int fd;
} ctx;
-static void validate()
+static void validate(void)
{
if (env.producer_cnt != 1) {
fprintf(stderr, "benchmark doesn't support multi-producer!\n");
@@ -43,7 +43,7 @@ static void measure(struct bench_res *res)
res->hits = atomic_swap(&ctx.hits.value, 0);
}
-static void setup_ctx()
+static void setup_ctx(void)
{
setup_libbpf();
@@ -71,36 +71,36 @@ static void attach_bpf(struct bpf_program *prog)
}
}
-static void setup_base()
+static void setup_base(void)
{
setup_ctx();
}
-static void setup_kprobe()
+static void setup_kprobe(void)
{
setup_ctx();
attach_bpf(ctx.skel->progs.prog1);
}
-static void setup_kretprobe()
+static void setup_kretprobe(void)
{
setup_ctx();
attach_bpf(ctx.skel->progs.prog2);
}
-static void setup_rawtp()
+static void setup_rawtp(void)
{
setup_ctx();
attach_bpf(ctx.skel->progs.prog3);
}
-static void setup_fentry()
+static void setup_fentry(void)
{
setup_ctx();
attach_bpf(ctx.skel->progs.prog4);
}
-static void setup_fexit()
+static void setup_fexit(void)
{
setup_ctx();
attach_bpf(ctx.skel->progs.prog5);
diff --git a/tools/testing/selftests/bpf/benchs/bench_ringbufs.c b/tools/testing/selftests/bpf/benchs/bench_ringbufs.c
index d167bff..da8593b 100644
--- a/tools/testing/selftests/bpf/benchs/bench_ringbufs.c
+++ b/tools/testing/selftests/bpf/benchs/bench_ringbufs.c
@@ -88,12 +88,12 @@ const struct argp bench_ringbufs_argp = {
static struct counter buf_hits;
-static inline void bufs_trigger_batch()
+static inline void bufs_trigger_batch(void)
{
(void)syscall(__NR_getpgid);
}
-static void bufs_validate()
+static void bufs_validate(void)
{
if (env.consumer_cnt != 1) {
fprintf(stderr, "rb-libbpf benchmark doesn't support multi-consumer!\n");
@@ -132,7 +132,7 @@ static void ringbuf_libbpf_measure(struct bench_res *res)
res->drops = atomic_swap(&ctx->skel->bss->dropped, 0);
}
-static struct ringbuf_bench *ringbuf_setup_skeleton()
+static struct ringbuf_bench *ringbuf_setup_skeleton(void)
{
struct ringbuf_bench *skel;
@@ -167,7 +167,7 @@ static int buf_process_sample(void *ctx, void *data, size_t len)
return 0;
}
-static void ringbuf_libbpf_setup()
+static void ringbuf_libbpf_setup(void)
{
struct ringbuf_libbpf_ctx *ctx = &ringbuf_libbpf_ctx;
struct bpf_link *link;
@@ -223,7 +223,7 @@ static void ringbuf_custom_measure(struct bench_res *res)
res->drops = atomic_swap(&ctx->skel->bss->dropped, 0);
}
-static void ringbuf_custom_setup()
+static void ringbuf_custom_setup(void)
{
struct ringbuf_custom_ctx *ctx = &ringbuf_custom_ctx;
const size_t page_size = getpagesize();
@@ -352,7 +352,7 @@ static void perfbuf_measure(struct bench_res *res)
res->drops = atomic_swap(&ctx->skel->bss->dropped, 0);
}
-static struct perfbuf_bench *perfbuf_setup_skeleton()
+static struct perfbuf_bench *perfbuf_setup_skeleton(void)
{
struct perfbuf_bench *skel;
@@ -390,15 +390,10 @@ perfbuf_process_sample_raw(void *input_ctx, int cpu,
return LIBBPF_PERF_EVENT_CONT;
}
-static void perfbuf_libbpf_setup()
+static void perfbuf_libbpf_setup(void)
{
struct perfbuf_libbpf_ctx *ctx = &perfbuf_libbpf_ctx;
struct perf_event_attr attr;
- struct perf_buffer_raw_opts pb_opts = {
- .event_cb = perfbuf_process_sample_raw,
- .ctx = (void *)(long)0,
- .attr = &attr,
- };
struct bpf_link *link;
ctx->skel = perfbuf_setup_skeleton();
@@ -423,7 +418,8 @@ static void perfbuf_libbpf_setup()
}
ctx->perfbuf = perf_buffer__new_raw(bpf_map__fd(ctx->skel->maps.perfbuf),
- args.perfbuf_sz, &pb_opts);
+ args.perfbuf_sz, &attr,
+ perfbuf_process_sample_raw, NULL, NULL);
if (!ctx->perfbuf) {
fprintf(stderr, "failed to create perfbuf\n");
exit(1);
diff --git a/tools/testing/selftests/bpf/benchs/bench_strncmp.c b/tools/testing/selftests/bpf/benchs/bench_strncmp.c
new file mode 100644
index 0000000..494b591
--- /dev/null
+++ b/tools/testing/selftests/bpf/benchs/bench_strncmp.c
@@ -0,0 +1,161 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2021. Huawei Technologies Co., Ltd */
+#include <argp.h>
+#include "bench.h"
+#include "strncmp_bench.skel.h"
+
+static struct strncmp_ctx {
+ struct strncmp_bench *skel;
+} ctx;
+
+static struct strncmp_args {
+ u32 cmp_str_len;
+} args = {
+ .cmp_str_len = 32,
+};
+
+enum {
+ ARG_CMP_STR_LEN = 5000,
+};
+
+static const struct argp_option opts[] = {
+ { "cmp-str-len", ARG_CMP_STR_LEN, "CMP_STR_LEN", 0,
+ "Set the length of compared string" },
+ {},
+};
+
+static error_t strncmp_parse_arg(int key, char *arg, struct argp_state *state)
+{
+ switch (key) {
+ case ARG_CMP_STR_LEN:
+ args.cmp_str_len = strtoul(arg, NULL, 10);
+ if (!args.cmp_str_len ||
+ args.cmp_str_len >= sizeof(ctx.skel->bss->str)) {
+ fprintf(stderr, "Invalid cmp str len (limit %zu)\n",
+ sizeof(ctx.skel->bss->str));
+ argp_usage(state);
+ }
+ break;
+ default:
+ return ARGP_ERR_UNKNOWN;
+ }
+
+ return 0;
+}
+
+const struct argp bench_strncmp_argp = {
+ .options = opts,
+ .parser = strncmp_parse_arg,
+};
+
+static void strncmp_validate(void)
+{
+ if (env.consumer_cnt != 1) {
+ fprintf(stderr, "strncmp benchmark doesn't support multi-consumer!\n");
+ exit(1);
+ }
+}
+
+static void strncmp_setup(void)
+{
+ int err;
+ char *target;
+ size_t i, sz;
+
+ sz = sizeof(ctx.skel->rodata->target);
+ if (!sz || sz < sizeof(ctx.skel->bss->str)) {
+ fprintf(stderr, "invalid string size (target %zu, src %zu)\n",
+ sz, sizeof(ctx.skel->bss->str));
+ exit(1);
+ }
+
+ setup_libbpf();
+
+ ctx.skel = strncmp_bench__open();
+ if (!ctx.skel) {
+ fprintf(stderr, "failed to open skeleton\n");
+ exit(1);
+ }
+
+ srandom(time(NULL));
+ target = ctx.skel->rodata->target;
+ for (i = 0; i < sz - 1; i++)
+ target[i] = '1' + random() % 9;
+ target[sz - 1] = '\0';
+
+ ctx.skel->rodata->cmp_str_len = args.cmp_str_len;
+
+ memcpy(ctx.skel->bss->str, target, args.cmp_str_len);
+ ctx.skel->bss->str[args.cmp_str_len] = '\0';
+ /* Make bss->str < rodata->target */
+ ctx.skel->bss->str[args.cmp_str_len - 1] -= 1;
+
+ err = strncmp_bench__load(ctx.skel);
+ if (err) {
+ fprintf(stderr, "failed to load skeleton\n");
+ strncmp_bench__destroy(ctx.skel);
+ exit(1);
+ }
+}
+
+static void strncmp_attach_prog(struct bpf_program *prog)
+{
+ struct bpf_link *link;
+
+ link = bpf_program__attach(prog);
+ if (!link) {
+ fprintf(stderr, "failed to attach program!\n");
+ exit(1);
+ }
+}
+
+static void strncmp_no_helper_setup(void)
+{
+ strncmp_setup();
+ strncmp_attach_prog(ctx.skel->progs.strncmp_no_helper);
+}
+
+static void strncmp_helper_setup(void)
+{
+ strncmp_setup();
+ strncmp_attach_prog(ctx.skel->progs.strncmp_helper);
+}
+
+static void *strncmp_producer(void *ctx)
+{
+ while (true)
+ (void)syscall(__NR_getpgid);
+ return NULL;
+}
+
+static void *strncmp_consumer(void *ctx)
+{
+ return NULL;
+}
+
+static void strncmp_measure(struct bench_res *res)
+{
+ res->hits = atomic_swap(&ctx.skel->bss->hits, 0);
+}
+
+const struct bench bench_strncmp_no_helper = {
+ .name = "strncmp-no-helper",
+ .validate = strncmp_validate,
+ .setup = strncmp_no_helper_setup,
+ .producer_thread = strncmp_producer,
+ .consumer_thread = strncmp_consumer,
+ .measure = strncmp_measure,
+ .report_progress = hits_drops_report_progress,
+ .report_final = hits_drops_report_final,
+};
+
+const struct bench bench_strncmp_helper = {
+ .name = "strncmp-helper",
+ .validate = strncmp_validate,
+ .setup = strncmp_helper_setup,
+ .producer_thread = strncmp_producer,
+ .consumer_thread = strncmp_consumer,
+ .measure = strncmp_measure,
+ .report_progress = hits_drops_report_progress,
+ .report_final = hits_drops_report_final,
+};
diff --git a/tools/testing/selftests/bpf/benchs/bench_trigger.c b/tools/testing/selftests/bpf/benchs/bench_trigger.c
index f41a491..7f957c5 100644
--- a/tools/testing/selftests/bpf/benchs/bench_trigger.c
+++ b/tools/testing/selftests/bpf/benchs/bench_trigger.c
@@ -2,6 +2,7 @@
/* Copyright (c) 2020 Facebook */
#include "bench.h"
#include "trigger_bench.skel.h"
+#include "trace_helpers.h"
/* BPF triggering benchmarks */
static struct trigger_ctx {
@@ -10,7 +11,7 @@ static struct trigger_ctx {
static struct counter base_hits;
-static void trigger_validate()
+static void trigger_validate(void)
{
if (env.consumer_cnt != 1) {
fprintf(stderr, "benchmark doesn't support multi-consumer!\n");
@@ -44,7 +45,7 @@ static void trigger_measure(struct bench_res *res)
res->hits = atomic_swap(&ctx.skel->bss->hits, 0);
}
-static void setup_ctx()
+static void setup_ctx(void)
{
setup_libbpf();
@@ -66,37 +67,37 @@ static void attach_bpf(struct bpf_program *prog)
}
}
-static void trigger_tp_setup()
+static void trigger_tp_setup(void)
{
setup_ctx();
attach_bpf(ctx.skel->progs.bench_trigger_tp);
}
-static void trigger_rawtp_setup()
+static void trigger_rawtp_setup(void)
{
setup_ctx();
attach_bpf(ctx.skel->progs.bench_trigger_raw_tp);
}
-static void trigger_kprobe_setup()
+static void trigger_kprobe_setup(void)
{
setup_ctx();
attach_bpf(ctx.skel->progs.bench_trigger_kprobe);
}
-static void trigger_fentry_setup()
+static void trigger_fentry_setup(void)
{
setup_ctx();
attach_bpf(ctx.skel->progs.bench_trigger_fentry);
}
-static void trigger_fentry_sleep_setup()
+static void trigger_fentry_sleep_setup(void)
{
setup_ctx();
attach_bpf(ctx.skel->progs.bench_trigger_fentry_sleep);
}
-static void trigger_fmodret_setup()
+static void trigger_fmodret_setup(void)
{
setup_ctx();
attach_bpf(ctx.skel->progs.bench_trigger_fmodret);
@@ -107,6 +108,101 @@ static void *trigger_consumer(void *input)
return NULL;
}
+/* make sure call is not inlined and not avoided by compiler, so __weak and
+ * inline asm volatile in the body of the function
+ *
+ * There is a performance difference between uprobing at nop location vs other
+ * instructions. So use two different targets, one of which starts with nop
+ * and another doesn't.
+ *
+ * GCC doesn't generate stack setup preample for these functions due to them
+ * having no input arguments and doing nothing in the body.
+ */
+__weak void uprobe_target_with_nop(void)
+{
+ asm volatile ("nop");
+}
+
+__weak void uprobe_target_without_nop(void)
+{
+ asm volatile ("");
+}
+
+static void *uprobe_base_producer(void *input)
+{
+ while (true) {
+ uprobe_target_with_nop();
+ atomic_inc(&base_hits.value);
+ }
+ return NULL;
+}
+
+static void *uprobe_producer_with_nop(void *input)
+{
+ while (true)
+ uprobe_target_with_nop();
+ return NULL;
+}
+
+static void *uprobe_producer_without_nop(void *input)
+{
+ while (true)
+ uprobe_target_without_nop();
+ return NULL;
+}
+
+static void usetup(bool use_retprobe, bool use_nop)
+{
+ size_t uprobe_offset;
+ ssize_t base_addr;
+ struct bpf_link *link;
+
+ setup_libbpf();
+
+ ctx.skel = trigger_bench__open_and_load();
+ if (!ctx.skel) {
+ fprintf(stderr, "failed to open skeleton\n");
+ exit(1);
+ }
+
+ base_addr = get_base_addr();
+ if (use_nop)
+ uprobe_offset = get_uprobe_offset(&uprobe_target_with_nop, base_addr);
+ else
+ uprobe_offset = get_uprobe_offset(&uprobe_target_without_nop, base_addr);
+
+ link = bpf_program__attach_uprobe(ctx.skel->progs.bench_trigger_uprobe,
+ use_retprobe,
+ -1 /* all PIDs */,
+ "/proc/self/exe",
+ uprobe_offset);
+ if (!link) {
+ fprintf(stderr, "failed to attach uprobe!\n");
+ exit(1);
+ }
+ ctx.skel->links.bench_trigger_uprobe = link;
+}
+
+static void uprobe_setup_with_nop(void)
+{
+ usetup(false, true);
+}
+
+static void uretprobe_setup_with_nop(void)
+{
+ usetup(true, true);
+}
+
+static void uprobe_setup_without_nop(void)
+{
+ usetup(false, false);
+}
+
+static void uretprobe_setup_without_nop(void)
+{
+ usetup(true, false);
+}
+
const struct bench bench_trig_base = {
.name = "trig-base",
.validate = trigger_validate,
@@ -182,3 +278,53 @@ const struct bench bench_trig_fmodret = {
.report_progress = hits_drops_report_progress,
.report_final = hits_drops_report_final,
};
+
+const struct bench bench_trig_uprobe_base = {
+ .name = "trig-uprobe-base",
+ .setup = NULL, /* no uprobe/uretprobe is attached */
+ .producer_thread = uprobe_base_producer,
+ .consumer_thread = trigger_consumer,
+ .measure = trigger_base_measure,
+ .report_progress = hits_drops_report_progress,
+ .report_final = hits_drops_report_final,
+};
+
+const struct bench bench_trig_uprobe_with_nop = {
+ .name = "trig-uprobe-with-nop",
+ .setup = uprobe_setup_with_nop,
+ .producer_thread = uprobe_producer_with_nop,
+ .consumer_thread = trigger_consumer,
+ .measure = trigger_measure,
+ .report_progress = hits_drops_report_progress,
+ .report_final = hits_drops_report_final,
+};
+
+const struct bench bench_trig_uretprobe_with_nop = {
+ .name = "trig-uretprobe-with-nop",
+ .setup = uretprobe_setup_with_nop,
+ .producer_thread = uprobe_producer_with_nop,
+ .consumer_thread = trigger_consumer,
+ .measure = trigger_measure,
+ .report_progress = hits_drops_report_progress,
+ .report_final = hits_drops_report_final,
+};
+
+const struct bench bench_trig_uprobe_without_nop = {
+ .name = "trig-uprobe-without-nop",
+ .setup = uprobe_setup_without_nop,
+ .producer_thread = uprobe_producer_without_nop,
+ .consumer_thread = trigger_consumer,
+ .measure = trigger_measure,
+ .report_progress = hits_drops_report_progress,
+ .report_final = hits_drops_report_final,
+};
+
+const struct bench bench_trig_uretprobe_without_nop = {
+ .name = "trig-uretprobe-without-nop",
+ .setup = uretprobe_setup_without_nop,
+ .producer_thread = uprobe_producer_without_nop,
+ .consumer_thread = trigger_consumer,
+ .measure = trigger_measure,
+ .report_progress = hits_drops_report_progress,
+ .report_final = hits_drops_report_final,
+};
diff --git a/tools/testing/selftests/bpf/benchs/run_bench_bpf_loop.sh b/tools/testing/selftests/bpf/benchs/run_bench_bpf_loop.sh
new file mode 100755
index 0000000..d4f5f73
--- /dev/null
+++ b/tools/testing/selftests/bpf/benchs/run_bench_bpf_loop.sh
@@ -0,0 +1,15 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+source ./benchs/run_common.sh
+
+set -eufo pipefail
+
+for t in 1 4 8 12 16; do
+for i in 10 100 500 1000 5000 10000 50000 100000 500000 1000000; do
+subtitle "nr_loops: $i, nr_threads: $t"
+ summarize_ops "bpf_loop: " \
+ "$($RUN_BENCH -p $t --nr_loops $i bpf-loop)"
+ printf "\n"
+done
+done
diff --git a/tools/testing/selftests/bpf/benchs/run_bench_strncmp.sh b/tools/testing/selftests/bpf/benchs/run_bench_strncmp.sh
new file mode 100755
index 0000000..1426972
--- /dev/null
+++ b/tools/testing/selftests/bpf/benchs/run_bench_strncmp.sh
@@ -0,0 +1,12 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+source ./benchs/run_common.sh
+
+set -eufo pipefail
+
+for s in 1 8 64 512 2048 4095; do
+ for b in no-helper helper; do
+ summarize ${b}-${s} "$($RUN_BENCH --cmp-str-len=$s strncmp-${b})"
+ done
+done
diff --git a/tools/testing/selftests/bpf/benchs/run_common.sh b/tools/testing/selftests/bpf/benchs/run_common.sh
index 9a16be7..6c5e602 100644
--- a/tools/testing/selftests/bpf/benchs/run_common.sh
+++ b/tools/testing/selftests/bpf/benchs/run_common.sh
@@ -33,6 +33,14 @@
echo "$*" | sed -E "s/.*Percentage\s=\s+([0-9]+\.[0-9]+).*/\1/"
}
+function ops()
+{
+ echo -n "throughput: "
+ echo -n "$*" | sed -E "s/.*throughput\s+([0-9]+\.[0-9]+ ± [0-9]+\.[0-9]+\sM\sops\/s).*/\1/"
+ echo -n -e ", latency: "
+ echo "$*" | sed -E "s/.*latency\s+([0-9]+\.[0-9]+\sns\/op).*/\1/"
+}
+
function total()
{
echo "$*" | sed -E "s/.*total operations\s+([0-9]+\.[0-9]+ ± [0-9]+\.[0-9]+M\/s).*/\1/"
@@ -52,6 +60,13 @@
printf "%-20s %s%%\n" "$bench" "$(percentage $summary)"
}
+function summarize_ops()
+{
+ bench="$1"
+ summary=$(echo $2 | tail -n1)
+ printf "%-20s %s\n" "$bench" "$(ops $summary)"
+}
+
function summarize_total()
{
bench="$1"
diff --git a/tools/testing/selftests/bpf/btf_helpers.c b/tools/testing/selftests/bpf/btf_helpers.c
index b5b6b01..b5941d5 100644
--- a/tools/testing/selftests/bpf/btf_helpers.c
+++ b/tools/testing/selftests/bpf/btf_helpers.c
@@ -25,11 +25,12 @@ static const char * const btf_kind_str_mapping[] = {
[BTF_KIND_DATASEC] = "DATASEC",
[BTF_KIND_FLOAT] = "FLOAT",
[BTF_KIND_DECL_TAG] = "DECL_TAG",
+ [BTF_KIND_TYPE_TAG] = "TYPE_TAG",
};
static const char *btf_kind_str(__u16 kind)
{
- if (kind > BTF_KIND_DECL_TAG)
+ if (kind > BTF_KIND_TYPE_TAG)
return "UNKNOWN";
return btf_kind_str_mapping[kind];
}
@@ -109,6 +110,7 @@ int fprintf_btf_type_raw(FILE *out, const struct btf *btf, __u32 id)
case BTF_KIND_VOLATILE:
case BTF_KIND_RESTRICT:
case BTF_KIND_TYPEDEF:
+ case BTF_KIND_TYPE_TAG:
fprintf(out, " type_id=%u", t->type);
break;
case BTF_KIND_ARRAY: {
@@ -238,7 +240,6 @@ const char *btf_type_c_dump(const struct btf *btf)
static char buf[16 * 1024];
FILE *buf_file;
struct btf_dump *d = NULL;
- struct btf_dump_opts opts = {};
int err, i;
buf_file = fmemopen(buf, sizeof(buf) - 1, "w");
@@ -247,22 +248,26 @@ const char *btf_type_c_dump(const struct btf *btf)
return NULL;
}
- opts.ctx = buf_file;
- d = btf_dump__new(btf, NULL, &opts, btf_dump_printf);
+ d = btf_dump__new(btf, btf_dump_printf, buf_file, NULL);
if (libbpf_get_error(d)) {
fprintf(stderr, "Failed to create btf_dump instance: %ld\n", libbpf_get_error(d));
- return NULL;
+ goto err_out;
}
for (i = 1; i < btf__type_cnt(btf); i++) {
err = btf_dump__dump_type(d, i);
if (err) {
fprintf(stderr, "Failed to dump type [%d]: %d\n", i, err);
- return NULL;
+ goto err_out;
}
}
+ btf_dump__free(d);
fflush(buf_file);
fclose(buf_file);
return buf;
+err_out:
+ btf_dump__free(d);
+ fclose(buf_file);
+ return NULL;
}
diff --git a/tools/testing/selftests/bpf/config b/tools/testing/selftests/bpf/config
index 5192305..f628713 100644
--- a/tools/testing/selftests/bpf/config
+++ b/tools/testing/selftests/bpf/config
@@ -38,7 +38,9 @@
CONFIG_BPF_JIT=y
CONFIG_BPF_LSM=y
CONFIG_SECURITY=y
+CONFIG_RC_CORE=y
CONFIG_LIRC=y
+CONFIG_BPF_LIRC_MODE2=y
CONFIG_IMA=y
CONFIG_SECURITYFS=y
CONFIG_IMA_WRITE_POLICY=y
diff --git a/tools/testing/selftests/bpf/flow_dissector_load.h b/tools/testing/selftests/bpf/flow_dissector_load.h
index 9d0acc2..f40b585 100644
--- a/tools/testing/selftests/bpf/flow_dissector_load.h
+++ b/tools/testing/selftests/bpf/flow_dissector_load.h
@@ -4,6 +4,7 @@
#include <bpf/bpf.h>
#include <bpf/libbpf.h>
+#include "testing_helpers.h"
static inline int bpf_flow_load(struct bpf_object **obj,
const char *path,
@@ -18,7 +19,7 @@ static inline int bpf_flow_load(struct bpf_object **obj,
int prog_array_fd;
int ret, fd, i;
- ret = bpf_prog_load(path, BPF_PROG_TYPE_FLOW_DISSECTOR, obj,
+ ret = bpf_prog_test_load(path, BPF_PROG_TYPE_FLOW_DISSECTOR, obj,
prog_fd);
if (ret)
return ret;
diff --git a/tools/testing/selftests/bpf/get_cgroup_id_user.c b/tools/testing/selftests/bpf/get_cgroup_id_user.c
index 99628e1..3a7b82b 100644
--- a/tools/testing/selftests/bpf/get_cgroup_id_user.c
+++ b/tools/testing/selftests/bpf/get_cgroup_id_user.c
@@ -19,6 +19,7 @@
#include <bpf/libbpf.h>
#include "cgroup_helpers.h"
+#include "testing_helpers.h"
#include "bpf_rlimit.h"
#define CHECK(condition, tag, format...) ({ \
@@ -66,8 +67,8 @@ int main(int argc, char **argv)
if (CHECK(cgroup_fd < 0, "cgroup_setup_and_join", "err %d errno %d\n", cgroup_fd, errno))
return 1;
- err = bpf_prog_load(file, BPF_PROG_TYPE_TRACEPOINT, &obj, &prog_fd);
- if (CHECK(err, "bpf_prog_load", "err %d errno %d\n", err, errno))
+ err = bpf_prog_test_load(file, BPF_PROG_TYPE_TRACEPOINT, &obj, &prog_fd);
+ if (CHECK(err, "bpf_prog_test_load", "err %d errno %d\n", err, errno))
goto cleanup_cgroup_env;
cgidmap_fd = bpf_find_map(__func__, obj, "cg_ids");
diff --git a/tools/testing/selftests/bpf/map_tests/array_map_batch_ops.c b/tools/testing/selftests/bpf/map_tests/array_map_batch_ops.c
index f4d870d..78c7649 100644
--- a/tools/testing/selftests/bpf/map_tests/array_map_batch_ops.c
+++ b/tools/testing/selftests/bpf/map_tests/array_map_batch_ops.c
@@ -68,13 +68,6 @@ static void map_batch_verify(int *visited, __u32 max_entries, int *keys,
static void __test_map_lookup_and_update_batch(bool is_pcpu)
{
- struct bpf_create_map_attr xattr = {
- .name = "array_map",
- .map_type = is_pcpu ? BPF_MAP_TYPE_PERCPU_ARRAY :
- BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(int),
- .value_size = sizeof(__s64),
- };
int map_fd, *keys, *visited;
__u32 count, total, total_success;
const __u32 max_entries = 10;
@@ -86,10 +79,10 @@ static void __test_map_lookup_and_update_batch(bool is_pcpu)
.flags = 0,
);
- xattr.max_entries = max_entries;
- map_fd = bpf_create_map_xattr(&xattr);
+ map_fd = bpf_map_create(is_pcpu ? BPF_MAP_TYPE_PERCPU_ARRAY : BPF_MAP_TYPE_ARRAY,
+ "array_map", sizeof(int), sizeof(__s64), max_entries, NULL);
CHECK(map_fd == -1,
- "bpf_create_map_xattr()", "error:%s\n", strerror(errno));
+ "bpf_map_create()", "error:%s\n", strerror(errno));
value_size = sizeof(__s64);
if (is_pcpu)
diff --git a/tools/testing/selftests/bpf/map_tests/htab_map_batch_ops.c b/tools/testing/selftests/bpf/map_tests/htab_map_batch_ops.c
index 976bf41..f807d53 100644
--- a/tools/testing/selftests/bpf/map_tests/htab_map_batch_ops.c
+++ b/tools/testing/selftests/bpf/map_tests/htab_map_batch_ops.c
@@ -83,22 +83,15 @@ void __test_map_lookup_and_delete_batch(bool is_pcpu)
int err, step, value_size;
bool nospace_err;
void *values;
- struct bpf_create_map_attr xattr = {
- .name = "hash_map",
- .map_type = is_pcpu ? BPF_MAP_TYPE_PERCPU_HASH :
- BPF_MAP_TYPE_HASH,
- .key_size = sizeof(int),
- .value_size = sizeof(int),
- };
DECLARE_LIBBPF_OPTS(bpf_map_batch_opts, opts,
.elem_flags = 0,
.flags = 0,
);
- xattr.max_entries = max_entries;
- map_fd = bpf_create_map_xattr(&xattr);
+ map_fd = bpf_map_create(is_pcpu ? BPF_MAP_TYPE_PERCPU_HASH : BPF_MAP_TYPE_HASH,
+ "hash_map", sizeof(int), sizeof(int), max_entries, NULL);
CHECK(map_fd == -1,
- "bpf_create_map_xattr()", "error:%s\n", strerror(errno));
+ "bpf_map_create()", "error:%s\n", strerror(errno));
value_size = is_pcpu ? sizeof(value) : sizeof(int);
keys = malloc(max_entries * sizeof(int));
diff --git a/tools/testing/selftests/bpf/map_tests/lpm_trie_map_batch_ops.c b/tools/testing/selftests/bpf/map_tests/lpm_trie_map_batch_ops.c
index 2e986e5..87d07b5 100644
--- a/tools/testing/selftests/bpf/map_tests/lpm_trie_map_batch_ops.c
+++ b/tools/testing/selftests/bpf/map_tests/lpm_trie_map_batch_ops.c
@@ -64,13 +64,7 @@ static void map_batch_verify(int *visited, __u32 max_entries,
void test_lpm_trie_map_batch_ops(void)
{
- struct bpf_create_map_attr xattr = {
- .name = "lpm_trie_map",
- .map_type = BPF_MAP_TYPE_LPM_TRIE,
- .key_size = sizeof(struct test_lpm_key),
- .value_size = sizeof(int),
- .map_flags = BPF_F_NO_PREALLOC,
- };
+ LIBBPF_OPTS(bpf_map_create_opts, create_opts, .map_flags = BPF_F_NO_PREALLOC);
struct test_lpm_key *keys, key;
int map_fd, *values, *visited;
__u32 step, count, total, total_success;
@@ -82,9 +76,10 @@ void test_lpm_trie_map_batch_ops(void)
.flags = 0,
);
- xattr.max_entries = max_entries;
- map_fd = bpf_create_map_xattr(&xattr);
- CHECK(map_fd == -1, "bpf_create_map_xattr()", "error:%s\n",
+ map_fd = bpf_map_create(BPF_MAP_TYPE_LPM_TRIE, "lpm_trie_map",
+ sizeof(struct test_lpm_key), sizeof(int),
+ max_entries, &create_opts);
+ CHECK(map_fd == -1, "bpf_map_create()", "error:%s\n",
strerror(errno));
keys = malloc(max_entries * sizeof(struct test_lpm_key));
diff --git a/tools/testing/selftests/bpf/map_tests/sk_storage_map.c b/tools/testing/selftests/bpf/map_tests/sk_storage_map.c
index e569edc..099eb4d 100644
--- a/tools/testing/selftests/bpf/map_tests/sk_storage_map.c
+++ b/tools/testing/selftests/bpf/map_tests/sk_storage_map.c
@@ -19,16 +19,12 @@
#include <test_btf.h>
#include <test_maps.h>
-static struct bpf_create_map_attr xattr = {
- .name = "sk_storage_map",
- .map_type = BPF_MAP_TYPE_SK_STORAGE,
- .map_flags = BPF_F_NO_PREALLOC,
- .max_entries = 0,
- .key_size = 4,
- .value_size = 8,
+static struct bpf_map_create_opts map_opts = {
+ .sz = sizeof(map_opts),
.btf_key_type_id = 1,
.btf_value_type_id = 3,
.btf_fd = -1,
+ .map_flags = BPF_F_NO_PREALLOC,
};
static unsigned int nr_sk_threads_done;
@@ -140,7 +136,7 @@ static int load_btf(void)
memcpy(raw_btf + sizeof(btf_hdr) + sizeof(btf_raw_types),
btf_str_sec, sizeof(btf_str_sec));
- return bpf_load_btf(raw_btf, sizeof(raw_btf), 0, 0, 0);
+ return bpf_btf_load(raw_btf, sizeof(raw_btf), NULL);
}
static int create_sk_storage_map(void)
@@ -150,13 +146,13 @@ static int create_sk_storage_map(void)
btf_fd = load_btf();
CHECK(btf_fd == -1, "bpf_load_btf", "btf_fd:%d errno:%d\n",
btf_fd, errno);
- xattr.btf_fd = btf_fd;
+ map_opts.btf_fd = btf_fd;
- map_fd = bpf_create_map_xattr(&xattr);
- xattr.btf_fd = -1;
+ map_fd = bpf_map_create(BPF_MAP_TYPE_SK_STORAGE, "sk_storage_map", 4, 8, 0, &map_opts);
+ map_opts.btf_fd = -1;
close(btf_fd);
CHECK(map_fd == -1,
- "bpf_create_map_xattr()", "errno:%d\n", errno);
+ "bpf_map_create()", "errno:%d\n", errno);
return map_fd;
}
@@ -463,20 +459,20 @@ static void test_sk_storage_map_basic(void)
int cnt;
int lock;
} value = { .cnt = 0xeB9f, .lock = 0, }, lookup_value;
- struct bpf_create_map_attr bad_xattr;
+ struct bpf_map_create_opts bad_xattr;
int btf_fd, map_fd, sk_fd, err;
btf_fd = load_btf();
CHECK(btf_fd == -1, "bpf_load_btf", "btf_fd:%d errno:%d\n",
btf_fd, errno);
- xattr.btf_fd = btf_fd;
+ map_opts.btf_fd = btf_fd;
sk_fd = socket(AF_INET6, SOCK_STREAM, 0);
CHECK(sk_fd == -1, "socket()", "sk_fd:%d errno:%d\n",
sk_fd, errno);
- map_fd = bpf_create_map_xattr(&xattr);
- CHECK(map_fd == -1, "bpf_create_map_xattr(good_xattr)",
+ map_fd = bpf_map_create(BPF_MAP_TYPE_SK_STORAGE, "sk_storage_map", 4, 8, 0, &map_opts);
+ CHECK(map_fd == -1, "bpf_map_create(good_xattr)",
"map_fd:%d errno:%d\n", map_fd, errno);
/* Add new elem */
@@ -560,31 +556,29 @@ static void test_sk_storage_map_basic(void)
CHECK(!err || errno != ENOENT, "bpf_map_delete_elem()",
"err:%d errno:%d\n", err, errno);
- memcpy(&bad_xattr, &xattr, sizeof(xattr));
+ memcpy(&bad_xattr, &map_opts, sizeof(map_opts));
bad_xattr.btf_key_type_id = 0;
- err = bpf_create_map_xattr(&bad_xattr);
- CHECK(!err || errno != EINVAL, "bap_create_map_xattr(bad_xattr)",
+ err = bpf_map_create(BPF_MAP_TYPE_SK_STORAGE, "sk_storage_map", 4, 8, 0, &bad_xattr);
+ CHECK(!err || errno != EINVAL, "bpf_map_create(bad_xattr)",
"err:%d errno:%d\n", err, errno);
- memcpy(&bad_xattr, &xattr, sizeof(xattr));
+ memcpy(&bad_xattr, &map_opts, sizeof(map_opts));
bad_xattr.btf_key_type_id = 3;
- err = bpf_create_map_xattr(&bad_xattr);
- CHECK(!err || errno != EINVAL, "bap_create_map_xattr(bad_xattr)",
+ err = bpf_map_create(BPF_MAP_TYPE_SK_STORAGE, "sk_storage_map", 4, 8, 0, &bad_xattr);
+ CHECK(!err || errno != EINVAL, "bpf_map_create(bad_xattr)",
"err:%d errno:%d\n", err, errno);
- memcpy(&bad_xattr, &xattr, sizeof(xattr));
- bad_xattr.max_entries = 1;
- err = bpf_create_map_xattr(&bad_xattr);
- CHECK(!err || errno != EINVAL, "bap_create_map_xattr(bad_xattr)",
+ err = bpf_map_create(BPF_MAP_TYPE_SK_STORAGE, "sk_storage_map", 4, 8, 1, &map_opts);
+ CHECK(!err || errno != EINVAL, "bpf_map_create(bad_xattr)",
"err:%d errno:%d\n", err, errno);
- memcpy(&bad_xattr, &xattr, sizeof(xattr));
+ memcpy(&bad_xattr, &map_opts, sizeof(map_opts));
bad_xattr.map_flags = 0;
- err = bpf_create_map_xattr(&bad_xattr);
+ err = bpf_map_create(BPF_MAP_TYPE_SK_STORAGE, "sk_storage_map", 4, 8, 0, &bad_xattr);
CHECK(!err || errno != EINVAL, "bap_create_map_xattr(bad_xattr)",
"err:%d errno:%d\n", err, errno);
- xattr.btf_fd = -1;
+ map_opts.btf_fd = -1;
close(btf_fd);
close(map_fd);
close(sk_fd);
diff --git a/tools/testing/selftests/bpf/prog_tests/align.c b/tools/testing/selftests/bpf/prog_tests/align.c
index 5861446..0ee29e1 100644
--- a/tools/testing/selftests/bpf/prog_tests/align.c
+++ b/tools/testing/selftests/bpf/prog_tests/align.c
@@ -39,13 +39,13 @@ static struct bpf_align_test tests[] = {
},
.prog_type = BPF_PROG_TYPE_SCHED_CLS,
.matches = {
- {1, "R1=ctx(id=0,off=0,imm=0)"},
- {1, "R10=fp0"},
- {1, "R3_w=inv2"},
- {2, "R3_w=inv4"},
- {3, "R3_w=inv8"},
- {4, "R3_w=inv16"},
- {5, "R3_w=inv32"},
+ {0, "R1=ctx(id=0,off=0,imm=0)"},
+ {0, "R10=fp0"},
+ {0, "R3_w=inv2"},
+ {1, "R3_w=inv4"},
+ {2, "R3_w=inv8"},
+ {3, "R3_w=inv16"},
+ {4, "R3_w=inv32"},
},
},
{
@@ -67,19 +67,19 @@ static struct bpf_align_test tests[] = {
},
.prog_type = BPF_PROG_TYPE_SCHED_CLS,
.matches = {
- {1, "R1=ctx(id=0,off=0,imm=0)"},
- {1, "R10=fp0"},
- {1, "R3_w=inv1"},
- {2, "R3_w=inv2"},
- {3, "R3_w=inv4"},
- {4, "R3_w=inv8"},
- {5, "R3_w=inv16"},
- {6, "R3_w=inv1"},
- {7, "R4_w=inv32"},
- {8, "R4_w=inv16"},
- {9, "R4_w=inv8"},
- {10, "R4_w=inv4"},
- {11, "R4_w=inv2"},
+ {0, "R1=ctx(id=0,off=0,imm=0)"},
+ {0, "R10=fp0"},
+ {0, "R3_w=inv1"},
+ {1, "R3_w=inv2"},
+ {2, "R3_w=inv4"},
+ {3, "R3_w=inv8"},
+ {4, "R3_w=inv16"},
+ {5, "R3_w=inv1"},
+ {6, "R4_w=inv32"},
+ {7, "R4_w=inv16"},
+ {8, "R4_w=inv8"},
+ {9, "R4_w=inv4"},
+ {10, "R4_w=inv2"},
},
},
{
@@ -96,14 +96,14 @@ static struct bpf_align_test tests[] = {
},
.prog_type = BPF_PROG_TYPE_SCHED_CLS,
.matches = {
- {1, "R1=ctx(id=0,off=0,imm=0)"},
- {1, "R10=fp0"},
- {1, "R3_w=inv4"},
- {2, "R3_w=inv8"},
- {3, "R3_w=inv10"},
- {4, "R4_w=inv8"},
- {5, "R4_w=inv12"},
- {6, "R4_w=inv14"},
+ {0, "R1=ctx(id=0,off=0,imm=0)"},
+ {0, "R10=fp0"},
+ {0, "R3_w=inv4"},
+ {1, "R3_w=inv8"},
+ {2, "R3_w=inv10"},
+ {3, "R4_w=inv8"},
+ {4, "R4_w=inv12"},
+ {5, "R4_w=inv14"},
},
},
{
@@ -118,12 +118,12 @@ static struct bpf_align_test tests[] = {
},
.prog_type = BPF_PROG_TYPE_SCHED_CLS,
.matches = {
- {1, "R1=ctx(id=0,off=0,imm=0)"},
- {1, "R10=fp0"},
+ {0, "R1=ctx(id=0,off=0,imm=0)"},
+ {0, "R10=fp0"},
+ {0, "R3_w=inv7"},
{1, "R3_w=inv7"},
- {2, "R3_w=inv7"},
- {3, "R3_w=inv14"},
- {4, "R3_w=inv56"},
+ {2, "R3_w=inv14"},
+ {3, "R3_w=inv56"},
},
},
@@ -161,19 +161,19 @@ static struct bpf_align_test tests[] = {
},
.prog_type = BPF_PROG_TYPE_SCHED_CLS,
.matches = {
- {7, "R0_w=pkt(id=0,off=8,r=8,imm=0)"},
- {7, "R3_w=inv(id=0,umax_value=255,var_off=(0x0; 0xff))"},
- {8, "R3_w=inv(id=0,umax_value=510,var_off=(0x0; 0x1fe))"},
- {9, "R3_w=inv(id=0,umax_value=1020,var_off=(0x0; 0x3fc))"},
- {10, "R3_w=inv(id=0,umax_value=2040,var_off=(0x0; 0x7f8))"},
- {11, "R3_w=inv(id=0,umax_value=4080,var_off=(0x0; 0xff0))"},
- {18, "R3=pkt_end(id=0,off=0,imm=0)"},
- {18, "R4_w=inv(id=0,umax_value=255,var_off=(0x0; 0xff))"},
- {19, "R4_w=inv(id=0,umax_value=8160,var_off=(0x0; 0x1fe0))"},
- {20, "R4_w=inv(id=0,umax_value=4080,var_off=(0x0; 0xff0))"},
- {21, "R4_w=inv(id=0,umax_value=2040,var_off=(0x0; 0x7f8))"},
- {22, "R4_w=inv(id=0,umax_value=1020,var_off=(0x0; 0x3fc))"},
- {23, "R4_w=inv(id=0,umax_value=510,var_off=(0x0; 0x1fe))"},
+ {6, "R0_w=pkt(id=0,off=8,r=8,imm=0)"},
+ {6, "R3_w=inv(id=0,umax_value=255,var_off=(0x0; 0xff))"},
+ {7, "R3_w=inv(id=0,umax_value=510,var_off=(0x0; 0x1fe))"},
+ {8, "R3_w=inv(id=0,umax_value=1020,var_off=(0x0; 0x3fc))"},
+ {9, "R3_w=inv(id=0,umax_value=2040,var_off=(0x0; 0x7f8))"},
+ {10, "R3_w=inv(id=0,umax_value=4080,var_off=(0x0; 0xff0))"},
+ {12, "R3_w=pkt_end(id=0,off=0,imm=0)"},
+ {17, "R4_w=inv(id=0,umax_value=255,var_off=(0x0; 0xff))"},
+ {18, "R4_w=inv(id=0,umax_value=8160,var_off=(0x0; 0x1fe0))"},
+ {19, "R4_w=inv(id=0,umax_value=4080,var_off=(0x0; 0xff0))"},
+ {20, "R4_w=inv(id=0,umax_value=2040,var_off=(0x0; 0x7f8))"},
+ {21, "R4_w=inv(id=0,umax_value=1020,var_off=(0x0; 0x3fc))"},
+ {22, "R4_w=inv(id=0,umax_value=510,var_off=(0x0; 0x1fe))"},
},
},
{
@@ -194,16 +194,16 @@ static struct bpf_align_test tests[] = {
},
.prog_type = BPF_PROG_TYPE_SCHED_CLS,
.matches = {
- {7, "R3_w=inv(id=0,umax_value=255,var_off=(0x0; 0xff))"},
- {8, "R4_w=inv(id=1,umax_value=255,var_off=(0x0; 0xff))"},
- {9, "R4_w=inv(id=0,umax_value=255,var_off=(0x0; 0xff))"},
- {10, "R4_w=inv(id=1,umax_value=255,var_off=(0x0; 0xff))"},
- {11, "R4_w=inv(id=0,umax_value=510,var_off=(0x0; 0x1fe))"},
- {12, "R4_w=inv(id=1,umax_value=255,var_off=(0x0; 0xff))"},
- {13, "R4_w=inv(id=0,umax_value=1020,var_off=(0x0; 0x3fc))"},
- {14, "R4_w=inv(id=1,umax_value=255,var_off=(0x0; 0xff))"},
- {15, "R4_w=inv(id=0,umax_value=2040,var_off=(0x0; 0x7f8))"},
- {16, "R4_w=inv(id=0,umax_value=4080,var_off=(0x0; 0xff0))"},
+ {6, "R3_w=inv(id=0,umax_value=255,var_off=(0x0; 0xff))"},
+ {7, "R4_w=inv(id=1,umax_value=255,var_off=(0x0; 0xff))"},
+ {8, "R4_w=inv(id=0,umax_value=255,var_off=(0x0; 0xff))"},
+ {9, "R4_w=inv(id=1,umax_value=255,var_off=(0x0; 0xff))"},
+ {10, "R4_w=inv(id=0,umax_value=510,var_off=(0x0; 0x1fe))"},
+ {11, "R4_w=inv(id=1,umax_value=255,var_off=(0x0; 0xff))"},
+ {12, "R4_w=inv(id=0,umax_value=1020,var_off=(0x0; 0x3fc))"},
+ {13, "R4_w=inv(id=1,umax_value=255,var_off=(0x0; 0xff))"},
+ {14, "R4_w=inv(id=0,umax_value=2040,var_off=(0x0; 0x7f8))"},
+ {15, "R4_w=inv(id=0,umax_value=4080,var_off=(0x0; 0xff0))"},
},
},
{
@@ -234,14 +234,14 @@ static struct bpf_align_test tests[] = {
},
.prog_type = BPF_PROG_TYPE_SCHED_CLS,
.matches = {
- {4, "R5_w=pkt(id=0,off=0,r=0,imm=0)"},
- {5, "R5_w=pkt(id=0,off=14,r=0,imm=0)"},
- {6, "R4_w=pkt(id=0,off=14,r=0,imm=0)"},
- {10, "R2=pkt(id=0,off=0,r=18,imm=0)"},
+ {2, "R5_w=pkt(id=0,off=0,r=0,imm=0)"},
+ {4, "R5_w=pkt(id=0,off=14,r=0,imm=0)"},
+ {5, "R4_w=pkt(id=0,off=14,r=0,imm=0)"},
+ {9, "R2=pkt(id=0,off=0,r=18,imm=0)"},
{10, "R5=pkt(id=0,off=14,r=18,imm=0)"},
{10, "R4_w=inv(id=0,umax_value=255,var_off=(0x0; 0xff))"},
+ {13, "R4_w=inv(id=0,umax_value=65535,var_off=(0x0; 0xffff))"},
{14, "R4_w=inv(id=0,umax_value=65535,var_off=(0x0; 0xffff))"},
- {15, "R4_w=inv(id=0,umax_value=65535,var_off=(0x0; 0xffff))"},
},
},
{
@@ -296,8 +296,8 @@ static struct bpf_align_test tests[] = {
/* Calculated offset in R6 has unknown value, but known
* alignment of 4.
*/
- {8, "R2_w=pkt(id=0,off=0,r=8,imm=0)"},
- {8, "R6_w=inv(id=0,umax_value=1020,var_off=(0x0; 0x3fc))"},
+ {6, "R2_w=pkt(id=0,off=0,r=8,imm=0)"},
+ {7, "R6_w=inv(id=0,umax_value=1020,var_off=(0x0; 0x3fc))"},
/* Offset is added to packet pointer R5, resulting in
* known fixed offset, and variable offset from R6.
*/
@@ -313,11 +313,11 @@ static struct bpf_align_test tests[] = {
/* Variable offset is added to R5 packet pointer,
* resulting in auxiliary alignment of 4.
*/
- {18, "R5_w=pkt(id=2,off=0,r=0,umax_value=1020,var_off=(0x0; 0x3fc))"},
+ {17, "R5_w=pkt(id=2,off=0,r=0,umax_value=1020,var_off=(0x0; 0x3fc))"},
/* Constant offset is added to R5, resulting in
* reg->off of 14.
*/
- {19, "R5_w=pkt(id=2,off=14,r=0,umax_value=1020,var_off=(0x0; 0x3fc))"},
+ {18, "R5_w=pkt(id=2,off=14,r=0,umax_value=1020,var_off=(0x0; 0x3fc))"},
/* At the time the word size load is performed from R5,
* its total fixed offset is NET_IP_ALIGN + reg->off
* (14) which is 16. Then the variable offset is 4-byte
@@ -329,18 +329,18 @@ static struct bpf_align_test tests[] = {
/* Constant offset is added to R5 packet pointer,
* resulting in reg->off value of 14.
*/
- {26, "R5_w=pkt(id=0,off=14,r=8"},
+ {25, "R5_w=pkt(id=0,off=14,r=8"},
/* Variable offset is added to R5, resulting in a
* variable offset of (4n).
*/
- {27, "R5_w=pkt(id=3,off=14,r=0,umax_value=1020,var_off=(0x0; 0x3fc))"},
+ {26, "R5_w=pkt(id=3,off=14,r=0,umax_value=1020,var_off=(0x0; 0x3fc))"},
/* Constant is added to R5 again, setting reg->off to 18. */
- {28, "R5_w=pkt(id=3,off=18,r=0,umax_value=1020,var_off=(0x0; 0x3fc))"},
+ {27, "R5_w=pkt(id=3,off=18,r=0,umax_value=1020,var_off=(0x0; 0x3fc))"},
/* And once more we add a variable; resulting var_off
* is still (4n), fixed offset is not changed.
* Also, we create a new reg->id.
*/
- {29, "R5_w=pkt(id=4,off=18,r=0,umax_value=2040,var_off=(0x0; 0x7fc)"},
+ {28, "R5_w=pkt(id=4,off=18,r=0,umax_value=2040,var_off=(0x0; 0x7fc)"},
/* At the time the word size load is performed from R5,
* its total fixed offset is NET_IP_ALIGN + reg->off (18)
* which is 20. Then the variable offset is (4n), so
@@ -386,13 +386,13 @@ static struct bpf_align_test tests[] = {
/* Calculated offset in R6 has unknown value, but known
* alignment of 4.
*/
- {8, "R2_w=pkt(id=0,off=0,r=8,imm=0)"},
- {8, "R6_w=inv(id=0,umax_value=1020,var_off=(0x0; 0x3fc))"},
+ {6, "R2_w=pkt(id=0,off=0,r=8,imm=0)"},
+ {7, "R6_w=inv(id=0,umax_value=1020,var_off=(0x0; 0x3fc))"},
/* Adding 14 makes R6 be (4n+2) */
- {9, "R6_w=inv(id=0,umin_value=14,umax_value=1034,var_off=(0x2; 0x7fc))"},
+ {8, "R6_w=inv(id=0,umin_value=14,umax_value=1034,var_off=(0x2; 0x7fc))"},
/* Packet pointer has (4n+2) offset */
{11, "R5_w=pkt(id=1,off=0,r=0,umin_value=14,umax_value=1034,var_off=(0x2; 0x7fc)"},
- {13, "R4=pkt(id=1,off=4,r=0,umin_value=14,umax_value=1034,var_off=(0x2; 0x7fc)"},
+ {12, "R4=pkt(id=1,off=4,r=0,umin_value=14,umax_value=1034,var_off=(0x2; 0x7fc)"},
/* At the time the word size load is performed from R5,
* its total fixed offset is NET_IP_ALIGN + reg->off (0)
* which is 2. Then the variable offset is (4n+2), so
@@ -403,12 +403,12 @@ static struct bpf_align_test tests[] = {
/* Newly read value in R6 was shifted left by 2, so has
* known alignment of 4.
*/
- {18, "R6_w=inv(id=0,umax_value=1020,var_off=(0x0; 0x3fc))"},
+ {17, "R6_w=inv(id=0,umax_value=1020,var_off=(0x0; 0x3fc))"},
/* Added (4n) to packet pointer's (4n+2) var_off, giving
* another (4n+2).
*/
{19, "R5_w=pkt(id=2,off=0,r=0,umin_value=14,umax_value=2054,var_off=(0x2; 0xffc)"},
- {21, "R4=pkt(id=2,off=4,r=0,umin_value=14,umax_value=2054,var_off=(0x2; 0xffc)"},
+ {20, "R4=pkt(id=2,off=4,r=0,umin_value=14,umax_value=2054,var_off=(0x2; 0xffc)"},
/* At the time the word size load is performed from R5,
* its total fixed offset is NET_IP_ALIGN + reg->off (0)
* which is 2. Then the variable offset is (4n+2), so
@@ -448,18 +448,18 @@ static struct bpf_align_test tests[] = {
.prog_type = BPF_PROG_TYPE_SCHED_CLS,
.result = REJECT,
.matches = {
- {4, "R5_w=pkt_end(id=0,off=0,imm=0)"},
+ {3, "R5_w=pkt_end(id=0,off=0,imm=0)"},
/* (ptr - ptr) << 2 == unknown, (4n) */
- {6, "R5_w=inv(id=0,smax_value=9223372036854775804,umax_value=18446744073709551612,var_off=(0x0; 0xfffffffffffffffc)"},
+ {5, "R5_w=inv(id=0,smax_value=9223372036854775804,umax_value=18446744073709551612,var_off=(0x0; 0xfffffffffffffffc)"},
/* (4n) + 14 == (4n+2). We blow our bounds, because
* the add could overflow.
*/
- {7, "R5_w=inv(id=0,smin_value=-9223372036854775806,smax_value=9223372036854775806,umin_value=2,umax_value=18446744073709551614,var_off=(0x2; 0xfffffffffffffffc)"},
+ {6, "R5_w=inv(id=0,smin_value=-9223372036854775806,smax_value=9223372036854775806,umin_value=2,umax_value=18446744073709551614,var_off=(0x2; 0xfffffffffffffffc)"},
/* Checked s>=0 */
{9, "R5=inv(id=0,umin_value=2,umax_value=9223372036854775806,var_off=(0x2; 0x7ffffffffffffffc)"},
/* packet pointer + nonnegative (4n+2) */
{11, "R6_w=pkt(id=1,off=0,r=0,umin_value=2,umax_value=9223372036854775806,var_off=(0x2; 0x7ffffffffffffffc)"},
- {13, "R4_w=pkt(id=1,off=4,r=0,umin_value=2,umax_value=9223372036854775806,var_off=(0x2; 0x7ffffffffffffffc)"},
+ {12, "R4_w=pkt(id=1,off=4,r=0,umin_value=2,umax_value=9223372036854775806,var_off=(0x2; 0x7ffffffffffffffc)"},
/* NET_IP_ALIGN + (4n+2) == (4n), alignment is fine.
* We checked the bounds, but it might have been able
* to overflow if the packet pointer started in the
@@ -502,14 +502,14 @@ static struct bpf_align_test tests[] = {
/* Calculated offset in R6 has unknown value, but known
* alignment of 4.
*/
- {7, "R2_w=pkt(id=0,off=0,r=8,imm=0)"},
- {9, "R6_w=inv(id=0,umax_value=1020,var_off=(0x0; 0x3fc))"},
+ {6, "R2_w=pkt(id=0,off=0,r=8,imm=0)"},
+ {8, "R6_w=inv(id=0,umax_value=1020,var_off=(0x0; 0x3fc))"},
/* Adding 14 makes R6 be (4n+2) */
- {10, "R6_w=inv(id=0,umin_value=14,umax_value=1034,var_off=(0x2; 0x7fc))"},
+ {9, "R6_w=inv(id=0,umin_value=14,umax_value=1034,var_off=(0x2; 0x7fc))"},
/* New unknown value in R7 is (4n) */
- {11, "R7_w=inv(id=0,umax_value=1020,var_off=(0x0; 0x3fc))"},
+ {10, "R7_w=inv(id=0,umax_value=1020,var_off=(0x0; 0x3fc))"},
/* Subtracting it from R6 blows our unsigned bounds */
- {12, "R6=inv(id=0,smin_value=-1006,smax_value=1034,umin_value=2,umax_value=18446744073709551614,var_off=(0x2; 0xfffffffffffffffc)"},
+ {11, "R6=inv(id=0,smin_value=-1006,smax_value=1034,umin_value=2,umax_value=18446744073709551614,var_off=(0x2; 0xfffffffffffffffc)"},
/* Checked s>= 0 */
{14, "R6=inv(id=0,umin_value=2,umax_value=1034,var_off=(0x2; 0x7fc))"},
/* At the time the word size load is performed from R5,
@@ -556,14 +556,14 @@ static struct bpf_align_test tests[] = {
/* Calculated offset in R6 has unknown value, but known
* alignment of 4.
*/
- {7, "R2_w=pkt(id=0,off=0,r=8,imm=0)"},
- {10, "R6_w=inv(id=0,umax_value=60,var_off=(0x0; 0x3c))"},
+ {6, "R2_w=pkt(id=0,off=0,r=8,imm=0)"},
+ {9, "R6_w=inv(id=0,umax_value=60,var_off=(0x0; 0x3c))"},
/* Adding 14 makes R6 be (4n+2) */
- {11, "R6_w=inv(id=0,umin_value=14,umax_value=74,var_off=(0x2; 0x7c))"},
+ {10, "R6_w=inv(id=0,umin_value=14,umax_value=74,var_off=(0x2; 0x7c))"},
/* Subtracting from packet pointer overflows ubounds */
{13, "R5_w=pkt(id=2,off=0,r=8,umin_value=18446744073709551542,umax_value=18446744073709551602,var_off=(0xffffffffffffff82; 0x7c)"},
/* New unknown value in R7 is (4n), >= 76 */
- {15, "R7_w=inv(id=0,umin_value=76,umax_value=1096,var_off=(0x0; 0x7fc))"},
+ {14, "R7_w=inv(id=0,umin_value=76,umax_value=1096,var_off=(0x0; 0x7fc))"},
/* Adding it to packet pointer gives nice bounds again */
{16, "R5_w=pkt(id=3,off=0,r=0,umin_value=2,umax_value=1082,var_off=(0x2; 0xfffffffc)"},
/* At the time the word size load is performed from R5,
@@ -594,6 +594,12 @@ static int do_test_single(struct bpf_align_test *test)
struct bpf_insn *prog = test->insns;
int prog_type = test->prog_type;
char bpf_vlog_copy[32768];
+ LIBBPF_OPTS(bpf_prog_load_opts, opts,
+ .prog_flags = BPF_F_STRICT_ALIGNMENT,
+ .log_buf = bpf_vlog,
+ .log_size = sizeof(bpf_vlog),
+ .log_level = 2,
+ );
const char *line_ptr;
int cur_line = -1;
int prog_len, i;
@@ -601,9 +607,8 @@ static int do_test_single(struct bpf_align_test *test)
int ret;
prog_len = probe_filter_length(prog);
- fd_prog = bpf_verify_program(prog_type ? : BPF_PROG_TYPE_SOCKET_FILTER,
- prog, prog_len, BPF_F_STRICT_ALIGNMENT,
- "GPL", 0, bpf_vlog, sizeof(bpf_vlog), 2);
+ fd_prog = bpf_prog_load(prog_type ? : BPF_PROG_TYPE_SOCKET_FILTER, NULL, "GPL",
+ prog, prog_len, &opts);
if (fd_prog < 0 && test->result != REJECT) {
printf("Failed to load program.\n");
printf("%s", bpf_vlog);
@@ -620,12 +625,15 @@ static int do_test_single(struct bpf_align_test *test)
line_ptr = strtok(bpf_vlog_copy, "\n");
for (i = 0; i < MAX_MATCHES; i++) {
struct bpf_reg_match m = test->matches[i];
+ int tmp;
if (!m.match)
break;
while (line_ptr) {
cur_line = -1;
sscanf(line_ptr, "%u: ", &cur_line);
+ if (cur_line == -1)
+ sscanf(line_ptr, "from %u to %u: ", &tmp, &cur_line);
if (cur_line == m.line)
break;
line_ptr = strtok(NULL, "\n");
@@ -637,7 +645,19 @@ static int do_test_single(struct bpf_align_test *test)
printf("%s", bpf_vlog);
break;
}
+ /* Check the next line as well in case the previous line
+ * did not have a corresponding bpf insn. Example:
+ * func#0 @0
+ * 0: R1=ctx(id=0,off=0,imm=0) R10=fp0
+ * 0: (b7) r3 = 2 ; R3_w=inv2
+ */
if (!strstr(line_ptr, m.match)) {
+ cur_line = -1;
+ line_ptr = strtok(NULL, "\n");
+ sscanf(line_ptr, "%u: ", &cur_line);
+ }
+ if (cur_line != m.line || !line_ptr ||
+ !strstr(line_ptr, m.match)) {
printf("Failed to find match %u: %s\n",
m.line, m.match);
ret = 1;
diff --git a/tools/testing/selftests/bpf/prog_tests/atomics.c b/tools/testing/selftests/bpf/prog_tests/atomics.c
index 0f95252..86b7d5d 100644
--- a/tools/testing/selftests/bpf/prog_tests/atomics.c
+++ b/tools/testing/selftests/bpf/prog_tests/atomics.c
@@ -167,7 +167,7 @@ static void test_cmpxchg(struct atomics_lskel *skel)
prog_fd = skel->progs.cmpxchg.prog_fd;
err = bpf_prog_test_run(prog_fd, 1, NULL, 0,
NULL, NULL, &retval, &duration);
- if (CHECK(err || retval, "test_run add",
+ if (CHECK(err || retval, "test_run cmpxchg",
"err %d errno %d retval %d duration %d\n", err, errno, retval, duration))
goto cleanup;
@@ -196,7 +196,7 @@ static void test_xchg(struct atomics_lskel *skel)
prog_fd = skel->progs.xchg.prog_fd;
err = bpf_prog_test_run(prog_fd, 1, NULL, 0,
NULL, NULL, &retval, &duration);
- if (CHECK(err || retval, "test_run add",
+ if (CHECK(err || retval, "test_run xchg",
"err %d errno %d retval %d duration %d\n", err, errno, retval, duration))
goto cleanup;
diff --git a/tools/testing/selftests/bpf/prog_tests/bloom_filter_map.c b/tools/testing/selftests/bpf/prog_tests/bloom_filter_map.c
index be73e3d..d2d9e96 100644
--- a/tools/testing/selftests/bpf/prog_tests/bloom_filter_map.c
+++ b/tools/testing/selftests/bpf/prog_tests/bloom_filter_map.c
@@ -7,32 +7,33 @@
static void test_fail_cases(void)
{
+ LIBBPF_OPTS(bpf_map_create_opts, opts);
__u32 value;
int fd, err;
/* Invalid key size */
- fd = bpf_create_map(BPF_MAP_TYPE_BLOOM_FILTER, 4, sizeof(value), 100, 0);
- if (!ASSERT_LT(fd, 0, "bpf_create_map bloom filter invalid key size"))
+ fd = bpf_map_create(BPF_MAP_TYPE_BLOOM_FILTER, NULL, 4, sizeof(value), 100, NULL);
+ if (!ASSERT_LT(fd, 0, "bpf_map_create bloom filter invalid key size"))
close(fd);
/* Invalid value size */
- fd = bpf_create_map(BPF_MAP_TYPE_BLOOM_FILTER, 0, 0, 100, 0);
- if (!ASSERT_LT(fd, 0, "bpf_create_map bloom filter invalid value size 0"))
+ fd = bpf_map_create(BPF_MAP_TYPE_BLOOM_FILTER, NULL, 0, 0, 100, NULL);
+ if (!ASSERT_LT(fd, 0, "bpf_map_create bloom filter invalid value size 0"))
close(fd);
/* Invalid max entries size */
- fd = bpf_create_map(BPF_MAP_TYPE_BLOOM_FILTER, 0, sizeof(value), 0, 0);
- if (!ASSERT_LT(fd, 0, "bpf_create_map bloom filter invalid max entries size"))
+ fd = bpf_map_create(BPF_MAP_TYPE_BLOOM_FILTER, NULL, 0, sizeof(value), 0, NULL);
+ if (!ASSERT_LT(fd, 0, "bpf_map_create bloom filter invalid max entries size"))
close(fd);
/* Bloom filter maps do not support BPF_F_NO_PREALLOC */
- fd = bpf_create_map(BPF_MAP_TYPE_BLOOM_FILTER, 0, sizeof(value), 100,
- BPF_F_NO_PREALLOC);
- if (!ASSERT_LT(fd, 0, "bpf_create_map bloom filter invalid flags"))
+ opts.map_flags = BPF_F_NO_PREALLOC;
+ fd = bpf_map_create(BPF_MAP_TYPE_BLOOM_FILTER, NULL, 0, sizeof(value), 100, &opts);
+ if (!ASSERT_LT(fd, 0, "bpf_map_create bloom filter invalid flags"))
close(fd);
- fd = bpf_create_map(BPF_MAP_TYPE_BLOOM_FILTER, 0, sizeof(value), 100, 0);
- if (!ASSERT_GE(fd, 0, "bpf_create_map bloom filter"))
+ fd = bpf_map_create(BPF_MAP_TYPE_BLOOM_FILTER, NULL, 0, sizeof(value), 100, NULL);
+ if (!ASSERT_GE(fd, 0, "bpf_map_create bloom filter"))
return;
/* Test invalid flags */
@@ -56,13 +57,14 @@ static void test_fail_cases(void)
static void test_success_cases(void)
{
+ LIBBPF_OPTS(bpf_map_create_opts, opts);
char value[11];
int fd, err;
/* Create a map */
- fd = bpf_create_map(BPF_MAP_TYPE_BLOOM_FILTER, 0, sizeof(value), 100,
- BPF_F_ZERO_SEED | BPF_F_NUMA_NODE);
- if (!ASSERT_GE(fd, 0, "bpf_create_map bloom filter success case"))
+ opts.map_flags = BPF_F_ZERO_SEED | BPF_F_NUMA_NODE;
+ fd = bpf_map_create(BPF_MAP_TYPE_BLOOM_FILTER, NULL, 0, sizeof(value), 100, &opts);
+ if (!ASSERT_GE(fd, 0, "bpf_map_create bloom filter success case"))
return;
/* Add a value to the bloom filter */
@@ -100,9 +102,9 @@ static void test_inner_map(struct bloom_filter_map *skel, const __u32 *rand_vals
struct bpf_link *link;
/* Create a bloom filter map that will be used as the inner map */
- inner_map_fd = bpf_create_map(BPF_MAP_TYPE_BLOOM_FILTER, 0, sizeof(*rand_vals),
- nr_rand_vals, 0);
- if (!ASSERT_GE(inner_map_fd, 0, "bpf_create_map bloom filter inner map"))
+ inner_map_fd = bpf_map_create(BPF_MAP_TYPE_BLOOM_FILTER, NULL, 0, sizeof(*rand_vals),
+ nr_rand_vals, NULL);
+ if (!ASSERT_GE(inner_map_fd, 0, "bpf_map_create bloom filter inner map"))
return;
for (i = 0; i < nr_rand_vals; i++) {
diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_iter.c b/tools/testing/selftests/bpf/prog_tests/bpf_iter.c
index 9454331..b84f859 100644
--- a/tools/testing/selftests/bpf/prog_tests/bpf_iter.c
+++ b/tools/testing/selftests/bpf/prog_tests/bpf_iter.c
@@ -469,12 +469,12 @@ static void test_overflow(bool test_e2big_overflow, bool ret1)
* fills seq_file buffer and then the other will trigger
* overflow and needs restart.
*/
- map1_fd = bpf_create_map(BPF_MAP_TYPE_ARRAY, 4, 8, 1, 0);
- if (CHECK(map1_fd < 0, "bpf_create_map",
+ map1_fd = bpf_map_create(BPF_MAP_TYPE_ARRAY, NULL, 4, 8, 1, NULL);
+ if (CHECK(map1_fd < 0, "bpf_map_create",
"map_creation failed: %s\n", strerror(errno)))
goto out;
- map2_fd = bpf_create_map(BPF_MAP_TYPE_ARRAY, 4, 8, 1, 0);
- if (CHECK(map2_fd < 0, "bpf_create_map",
+ map2_fd = bpf_map_create(BPF_MAP_TYPE_ARRAY, NULL, 4, 8, 1, NULL);
+ if (CHECK(map2_fd < 0, "bpf_map_create",
"map_creation failed: %s\n", strerror(errno)))
goto free_map1;
@@ -699,14 +699,13 @@ static void test_bpf_percpu_hash_map(void)
char buf[64];
void *val;
- val = malloc(8 * bpf_num_possible_cpus());
-
skel = bpf_iter_bpf_percpu_hash_map__open();
if (CHECK(!skel, "bpf_iter_bpf_percpu_hash_map__open",
"skeleton open failed\n"))
return;
skel->rodata->num_cpus = bpf_num_possible_cpus();
+ val = malloc(8 * bpf_num_possible_cpus());
err = bpf_iter_bpf_percpu_hash_map__load(skel);
if (CHECK(!skel, "bpf_iter_bpf_percpu_hash_map__load",
@@ -770,6 +769,7 @@ static void test_bpf_percpu_hash_map(void)
bpf_link__destroy(link);
out:
bpf_iter_bpf_percpu_hash_map__destroy(skel);
+ free(val);
}
static void test_bpf_array_map(void)
@@ -870,14 +870,13 @@ static void test_bpf_percpu_array_map(void)
void *val;
int len;
- val = malloc(8 * bpf_num_possible_cpus());
-
skel = bpf_iter_bpf_percpu_array_map__open();
if (CHECK(!skel, "bpf_iter_bpf_percpu_array_map__open",
"skeleton open failed\n"))
return;
skel->rodata->num_cpus = bpf_num_possible_cpus();
+ val = malloc(8 * bpf_num_possible_cpus());
err = bpf_iter_bpf_percpu_array_map__load(skel);
if (CHECK(!skel, "bpf_iter_bpf_percpu_array_map__load",
@@ -933,6 +932,7 @@ static void test_bpf_percpu_array_map(void)
bpf_link__destroy(link);
out:
bpf_iter_bpf_percpu_array_map__destroy(skel);
+ free(val);
}
/* An iterator program deletes all local storage in a map. */
@@ -1206,13 +1206,14 @@ static void test_task_vma(void)
goto out;
/* Read CMP_BUFFER_SIZE (1kB) from bpf_iter. Read in small chunks
- * to trigger seq_file corner cases. The expected output is much
- * longer than 1kB, so the while loop will terminate.
+ * to trigger seq_file corner cases.
*/
len = 0;
while (len < CMP_BUFFER_SIZE) {
err = read_fd_into_buffer(iter_fd, task_vma_output + len,
min(read_size, CMP_BUFFER_SIZE - len));
+ if (!err)
+ break;
if (CHECK(err < 0, "read_iter_fd", "read_iter_fd failed\n"))
goto out;
len += err;
diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_loop.c b/tools/testing/selftests/bpf/prog_tests/bpf_loop.c
new file mode 100644
index 0000000..380d7a2
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/bpf_loop.c
@@ -0,0 +1,145 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2021 Facebook */
+
+#include <test_progs.h>
+#include <network_helpers.h>
+#include "bpf_loop.skel.h"
+
+static void check_nr_loops(struct bpf_loop *skel)
+{
+ struct bpf_link *link;
+
+ link = bpf_program__attach(skel->progs.test_prog);
+ if (!ASSERT_OK_PTR(link, "link"))
+ return;
+
+ /* test 0 loops */
+ skel->bss->nr_loops = 0;
+
+ usleep(1);
+
+ ASSERT_EQ(skel->bss->nr_loops_returned, skel->bss->nr_loops,
+ "0 loops");
+
+ /* test 500 loops */
+ skel->bss->nr_loops = 500;
+
+ usleep(1);
+
+ ASSERT_EQ(skel->bss->nr_loops_returned, skel->bss->nr_loops,
+ "500 loops");
+ ASSERT_EQ(skel->bss->g_output, (500 * 499) / 2, "g_output");
+
+ /* test exceeding the max limit */
+ skel->bss->nr_loops = -1;
+
+ usleep(1);
+
+ ASSERT_EQ(skel->bss->err, -E2BIG, "over max limit");
+
+ bpf_link__destroy(link);
+}
+
+static void check_callback_fn_stop(struct bpf_loop *skel)
+{
+ struct bpf_link *link;
+
+ link = bpf_program__attach(skel->progs.test_prog);
+ if (!ASSERT_OK_PTR(link, "link"))
+ return;
+
+ /* testing that loop is stopped when callback_fn returns 1 */
+ skel->bss->nr_loops = 400;
+ skel->data->stop_index = 50;
+
+ usleep(1);
+
+ ASSERT_EQ(skel->bss->nr_loops_returned, skel->data->stop_index + 1,
+ "nr_loops_returned");
+ ASSERT_EQ(skel->bss->g_output, (50 * 49) / 2,
+ "g_output");
+
+ bpf_link__destroy(link);
+}
+
+static void check_null_callback_ctx(struct bpf_loop *skel)
+{
+ struct bpf_link *link;
+
+ /* check that user is able to pass in a null callback_ctx */
+ link = bpf_program__attach(skel->progs.prog_null_ctx);
+ if (!ASSERT_OK_PTR(link, "link"))
+ return;
+
+ skel->bss->nr_loops = 10;
+
+ usleep(1);
+
+ ASSERT_EQ(skel->bss->nr_loops_returned, skel->bss->nr_loops,
+ "nr_loops_returned");
+
+ bpf_link__destroy(link);
+}
+
+static void check_invalid_flags(struct bpf_loop *skel)
+{
+ struct bpf_link *link;
+
+ /* check that passing in non-zero flags returns -EINVAL */
+ link = bpf_program__attach(skel->progs.prog_invalid_flags);
+ if (!ASSERT_OK_PTR(link, "link"))
+ return;
+
+ usleep(1);
+
+ ASSERT_EQ(skel->bss->err, -EINVAL, "err");
+
+ bpf_link__destroy(link);
+}
+
+static void check_nested_calls(struct bpf_loop *skel)
+{
+ __u32 nr_loops = 100, nested_callback_nr_loops = 4;
+ struct bpf_link *link;
+
+ /* check that nested calls are supported */
+ link = bpf_program__attach(skel->progs.prog_nested_calls);
+ if (!ASSERT_OK_PTR(link, "link"))
+ return;
+
+ skel->bss->nr_loops = nr_loops;
+ skel->bss->nested_callback_nr_loops = nested_callback_nr_loops;
+
+ usleep(1);
+
+ ASSERT_EQ(skel->bss->nr_loops_returned, nr_loops * nested_callback_nr_loops
+ * nested_callback_nr_loops, "nr_loops_returned");
+ ASSERT_EQ(skel->bss->g_output, (4 * 3) / 2 * nested_callback_nr_loops
+ * nr_loops, "g_output");
+
+ bpf_link__destroy(link);
+}
+
+void test_bpf_loop(void)
+{
+ struct bpf_loop *skel;
+
+ skel = bpf_loop__open_and_load();
+ if (!ASSERT_OK_PTR(skel, "bpf_loop__open_and_load"))
+ return;
+
+ skel->bss->pid = getpid();
+
+ if (test__start_subtest("check_nr_loops"))
+ check_nr_loops(skel);
+ if (test__start_subtest("check_callback_fn_stop"))
+ check_callback_fn_stop(skel);
+ if (test__start_subtest("check_null_callback_ctx"))
+ check_null_callback_ctx(skel);
+ if (test__start_subtest("check_invalid_flags"))
+ check_invalid_flags(skel);
+ if (test__start_subtest("check_nested_calls"))
+ check_nested_calls(skel);
+
+ bpf_loop__destroy(skel);
+}
diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_obj_id.c b/tools/testing/selftests/bpf/prog_tests/bpf_obj_id.c
index eb8eeeb..dbe56fa 100644
--- a/tools/testing/selftests/bpf/prog_tests/bpf_obj_id.c
+++ b/tools/testing/selftests/bpf/prog_tests/bpf_obj_id.c
@@ -48,7 +48,7 @@ void serial_test_bpf_obj_id(void)
bzero(zeros, sizeof(zeros));
for (i = 0; i < nr_iters; i++) {
now = time(NULL);
- err = bpf_prog_load(file, BPF_PROG_TYPE_RAW_TRACEPOINT,
+ err = bpf_prog_test_load(file, BPF_PROG_TYPE_RAW_TRACEPOINT,
&objs[i], &prog_fds[i]);
/* test_obj_id.o is a dumb prog. It should never fail
* to load.
@@ -65,8 +65,8 @@ void serial_test_bpf_obj_id(void)
if (CHECK_FAIL(err))
goto done;
- prog = bpf_object__find_program_by_title(objs[i],
- "raw_tp/sys_enter");
+ prog = bpf_object__find_program_by_name(objs[i],
+ "test_obj_id");
if (CHECK_FAIL(!prog))
goto done;
links[i] = bpf_program__attach(prog);
diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_tcp_ca.c b/tools/testing/selftests/bpf/prog_tests/bpf_tcp_ca.c
index 94e03df..8f7a1ce 100644
--- a/tools/testing/selftests/bpf/prog_tests/bpf_tcp_ca.c
+++ b/tools/testing/selftests/bpf/prog_tests/bpf_tcp_ca.c
@@ -217,21 +217,22 @@ static bool found;
static int libbpf_debug_print(enum libbpf_print_level level,
const char *format, va_list args)
{
- char *log_buf;
+ const char *prog_name, *log_buf;
if (level != LIBBPF_WARN ||
- strcmp(format, "libbpf: \n%s\n")) {
+ !strstr(format, "-- BEGIN PROG LOAD LOG --")) {
vprintf(format, args);
return 0;
}
+ prog_name = va_arg(args, char *);
log_buf = va_arg(args, char *);
if (!log_buf)
goto out;
if (err_str && strstr(log_buf, err_str) != NULL)
found = true;
out:
- printf(format, log_buf);
+ printf(format, prog_name, log_buf);
return 0;
}
diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_verif_scale.c b/tools/testing/selftests/bpf/prog_tests/bpf_verif_scale.c
index 27f5d8e..ff6cce9 100644
--- a/tools/testing/selftests/bpf/prog_tests/bpf_verif_scale.c
+++ b/tools/testing/selftests/bpf/prog_tests/bpf_verif_scale.c
@@ -19,16 +19,28 @@ extern int extra_prog_load_log_flags;
static int check_load(const char *file, enum bpf_prog_type type)
{
- struct bpf_prog_load_attr attr;
struct bpf_object *obj = NULL;
- int err, prog_fd;
+ struct bpf_program *prog;
+ int err;
- memset(&attr, 0, sizeof(struct bpf_prog_load_attr));
- attr.file = file;
- attr.prog_type = type;
- attr.log_level = 4 | extra_prog_load_log_flags;
- attr.prog_flags = BPF_F_TEST_RND_HI32;
- err = bpf_prog_load_xattr(&attr, &obj, &prog_fd);
+ obj = bpf_object__open_file(file, NULL);
+ err = libbpf_get_error(obj);
+ if (err)
+ return err;
+
+ prog = bpf_object__next_program(obj, NULL);
+ if (!prog) {
+ err = -ENOENT;
+ goto err_out;
+ }
+
+ bpf_program__set_type(prog, type);
+ bpf_program__set_flags(prog, BPF_F_TEST_RND_HI32);
+ bpf_program__set_log_level(prog, 4 | extra_prog_load_log_flags);
+
+ err = bpf_object__load(obj);
+
+err_out:
bpf_object__close(obj);
return err;
}
@@ -115,6 +127,12 @@ void test_verif_scale_pyperf600()
scale_test("pyperf600.o", BPF_PROG_TYPE_RAW_TRACEPOINT, false);
}
+void test_verif_scale_pyperf600_bpf_loop(void)
+{
+ /* use the bpf_loop helper*/
+ scale_test("pyperf600_bpf_loop.o", BPF_PROG_TYPE_RAW_TRACEPOINT, false);
+}
+
void test_verif_scale_pyperf600_nounroll()
{
/* no unroll at all.
@@ -165,6 +183,12 @@ void test_verif_scale_strobemeta()
scale_test("strobemeta.o", BPF_PROG_TYPE_RAW_TRACEPOINT, false);
}
+void test_verif_scale_strobemeta_bpf_loop(void)
+{
+ /* use the bpf_loop helper*/
+ scale_test("strobemeta_bpf_loop.o", BPF_PROG_TYPE_RAW_TRACEPOINT, false);
+}
+
void test_verif_scale_strobemeta_nounroll1()
{
/* no unroll, tiny loops */
diff --git a/tools/testing/selftests/bpf/prog_tests/btf.c b/tools/testing/selftests/bpf/prog_tests/btf.c
index ac596cb..8ba53ac 100644
--- a/tools/testing/selftests/bpf/prog_tests/btf.c
+++ b/tools/testing/selftests/bpf/prog_tests/btf.c
@@ -22,7 +22,6 @@
#include <bpf/libbpf.h>
#include <bpf/btf.h>
-#include "bpf_rlimit.h"
#include "bpf_util.h"
#include "../test_btf.h"
#include "test_progs.h"
@@ -3939,6 +3938,23 @@ static struct btf_raw_test raw_tests[] = {
.btf_load_err = true,
.err_str = "Invalid component_idx",
},
+{
+ .descr = "type_tag test #1",
+ .raw_types = {
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */
+ BTF_TYPE_TAG_ENC(NAME_TBD, 1), /* [2] */
+ BTF_PTR_ENC(2), /* [3] */
+ BTF_END_RAW,
+ },
+ BTF_STR_SEC("\0tag"),
+ .map_type = BPF_MAP_TYPE_ARRAY,
+ .map_name = "tag_type_check_btf",
+ .key_size = sizeof(int),
+ .value_size = 4,
+ .key_type_id = 1,
+ .value_type_id = 1,
+ .max_entries = 1,
+},
}; /* struct btf_raw_test raw_tests[] */
@@ -4046,20 +4062,40 @@ static void *btf_raw_create(const struct btf_header *hdr,
next_str_idx < strs_cnt ? strs_idx[next_str_idx] : NULL;
done:
+ free(strs_idx);
if (err) {
- if (raw_btf)
- free(raw_btf);
- if (strs_idx)
- free(strs_idx);
+ free(raw_btf);
return NULL;
}
return raw_btf;
}
+static int load_raw_btf(const void *raw_data, size_t raw_size)
+{
+ LIBBPF_OPTS(bpf_btf_load_opts, opts);
+ int btf_fd;
+
+ if (always_log) {
+ opts.log_buf = btf_log_buf,
+ opts.log_size = BTF_LOG_BUF_SIZE,
+ opts.log_level = 1;
+ }
+
+ btf_fd = bpf_btf_load(raw_data, raw_size, &opts);
+ if (btf_fd < 0 && !always_log) {
+ opts.log_buf = btf_log_buf,
+ opts.log_size = BTF_LOG_BUF_SIZE,
+ opts.log_level = 1;
+ btf_fd = bpf_btf_load(raw_data, raw_size, &opts);
+ }
+
+ return btf_fd;
+}
+
static void do_test_raw(unsigned int test_num)
{
struct btf_raw_test *test = &raw_tests[test_num - 1];
- struct bpf_create_map_attr create_attr = {};
+ LIBBPF_OPTS(bpf_map_create_opts, opts);
int map_fd = -1, btf_fd = -1;
unsigned int raw_btf_size;
struct btf_header *hdr;
@@ -4085,16 +4121,14 @@ static void do_test_raw(unsigned int test_num)
hdr->str_len = (int)hdr->str_len + test->str_len_delta;
*btf_log_buf = '\0';
- btf_fd = bpf_load_btf(raw_btf, raw_btf_size,
- btf_log_buf, BTF_LOG_BUF_SIZE,
- always_log);
+ btf_fd = load_raw_btf(raw_btf, raw_btf_size);
free(raw_btf);
err = ((btf_fd < 0) != test->btf_load_err);
if (CHECK(err, "btf_fd:%d test->btf_load_err:%u",
btf_fd, test->btf_load_err) ||
CHECK(test->err_str && !strstr(btf_log_buf, test->err_str),
- "expected err_str:%s", test->err_str)) {
+ "expected err_str:%s\n", test->err_str)) {
err = -1;
goto done;
}
@@ -4102,16 +4136,11 @@ static void do_test_raw(unsigned int test_num)
if (err || btf_fd < 0)
goto done;
- create_attr.name = test->map_name;
- create_attr.map_type = test->map_type;
- create_attr.key_size = test->key_size;
- create_attr.value_size = test->value_size;
- create_attr.max_entries = test->max_entries;
- create_attr.btf_fd = btf_fd;
- create_attr.btf_key_type_id = test->key_type_id;
- create_attr.btf_value_type_id = test->value_type_id;
-
- map_fd = bpf_create_map_xattr(&create_attr);
+ opts.btf_fd = btf_fd;
+ opts.btf_key_type_id = test->key_type_id;
+ opts.btf_value_type_id = test->value_type_id;
+ map_fd = bpf_map_create(test->map_type, test->map_name,
+ test->key_size, test->value_size, test->max_entries, &opts);
err = ((map_fd < 0) != test->map_create_err);
CHECK(err, "map_fd:%d test->map_create_err:%u",
@@ -4217,9 +4246,7 @@ static int test_big_btf_info(unsigned int test_num)
goto done;
}
- btf_fd = bpf_load_btf(raw_btf, raw_btf_size,
- btf_log_buf, BTF_LOG_BUF_SIZE,
- always_log);
+ btf_fd = load_raw_btf(raw_btf, raw_btf_size);
if (CHECK(btf_fd < 0, "errno:%d", errno)) {
err = -1;
goto done;
@@ -4275,7 +4302,7 @@ static int test_big_btf_info(unsigned int test_num)
static int test_btf_id(unsigned int test_num)
{
const struct btf_get_info_test *test = &get_info_tests[test_num - 1];
- struct bpf_create_map_attr create_attr = {};
+ LIBBPF_OPTS(bpf_map_create_opts, opts);
uint8_t *raw_btf = NULL, *user_btf[2] = {};
int btf_fd[2] = {-1, -1}, map_fd = -1;
struct bpf_map_info map_info = {};
@@ -4305,9 +4332,7 @@ static int test_btf_id(unsigned int test_num)
info[i].btf_size = raw_btf_size;
}
- btf_fd[0] = bpf_load_btf(raw_btf, raw_btf_size,
- btf_log_buf, BTF_LOG_BUF_SIZE,
- always_log);
+ btf_fd[0] = load_raw_btf(raw_btf, raw_btf_size);
if (CHECK(btf_fd[0] < 0, "errno:%d", errno)) {
err = -1;
goto done;
@@ -4340,16 +4365,11 @@ static int test_btf_id(unsigned int test_num)
}
/* Test btf members in struct bpf_map_info */
- create_attr.name = "test_btf_id";
- create_attr.map_type = BPF_MAP_TYPE_ARRAY;
- create_attr.key_size = sizeof(int);
- create_attr.value_size = sizeof(unsigned int);
- create_attr.max_entries = 4;
- create_attr.btf_fd = btf_fd[0];
- create_attr.btf_key_type_id = 1;
- create_attr.btf_value_type_id = 2;
-
- map_fd = bpf_create_map_xattr(&create_attr);
+ opts.btf_fd = btf_fd[0];
+ opts.btf_key_type_id = 1;
+ opts.btf_value_type_id = 2;
+ map_fd = bpf_map_create(BPF_MAP_TYPE_ARRAY, "test_btf_id",
+ sizeof(int), sizeof(int), 4, &opts);
if (CHECK(map_fd < 0, "errno:%d", errno)) {
err = -1;
goto done;
@@ -4442,9 +4462,7 @@ static void do_test_get_info(unsigned int test_num)
goto done;
}
- btf_fd = bpf_load_btf(raw_btf, raw_btf_size,
- btf_log_buf, BTF_LOG_BUF_SIZE,
- always_log);
+ btf_fd = load_raw_btf(raw_btf, raw_btf_size);
if (CHECK(btf_fd <= 0, "errno:%d", errno)) {
err = -1;
goto done;
@@ -5138,7 +5156,7 @@ static void do_test_pprint(int test_num)
{
const struct btf_raw_test *test = &pprint_test_template[test_num];
enum pprint_mapv_kind_t mapv_kind = test->mapv_kind;
- struct bpf_create_map_attr create_attr = {};
+ LIBBPF_OPTS(bpf_map_create_opts, opts);
bool ordered_map, lossless_map, percpu_map;
int err, ret, num_cpus, rounded_value_size;
unsigned int key, nr_read_elems;
@@ -5164,26 +5182,19 @@ static void do_test_pprint(int test_num)
return;
*btf_log_buf = '\0';
- btf_fd = bpf_load_btf(raw_btf, raw_btf_size,
- btf_log_buf, BTF_LOG_BUF_SIZE,
- always_log);
+ btf_fd = load_raw_btf(raw_btf, raw_btf_size);
free(raw_btf);
- if (CHECK(btf_fd < 0, "errno:%d", errno)) {
+ if (CHECK(btf_fd < 0, "errno:%d\n", errno)) {
err = -1;
goto done;
}
- create_attr.name = test->map_name;
- create_attr.map_type = test->map_type;
- create_attr.key_size = test->key_size;
- create_attr.value_size = test->value_size;
- create_attr.max_entries = test->max_entries;
- create_attr.btf_fd = btf_fd;
- create_attr.btf_key_type_id = test->key_type_id;
- create_attr.btf_value_type_id = test->value_type_id;
-
- map_fd = bpf_create_map_xattr(&create_attr);
+ opts.btf_fd = btf_fd;
+ opts.btf_key_type_id = test->key_type_id;
+ opts.btf_value_type_id = test->value_type_id;
+ map_fd = bpf_map_create(test->map_type, test->map_name,
+ test->key_size, test->value_size, test->max_entries, &opts);
if (CHECK(map_fd < 0, "errno:%d", errno)) {
err = -1;
goto done;
@@ -6538,9 +6549,7 @@ static void do_test_info_raw(unsigned int test_num)
return;
*btf_log_buf = '\0';
- btf_fd = bpf_load_btf(raw_btf, raw_btf_size,
- btf_log_buf, BTF_LOG_BUF_SIZE,
- always_log);
+ btf_fd = load_raw_btf(raw_btf, raw_btf_size);
free(raw_btf);
if (CHECK(btf_fd < 0, "invalid btf_fd errno:%d", errno)) {
@@ -6629,7 +6638,7 @@ struct btf_dedup_test {
struct btf_dedup_opts opts;
};
-const struct btf_dedup_test dedup_tests[] = {
+static struct btf_dedup_test dedup_tests[] = {
{
.descr = "dedup: unused strings filtering",
@@ -6649,9 +6658,6 @@ const struct btf_dedup_test dedup_tests[] = {
},
BTF_STR_SEC("\0int\0long"),
},
- .opts = {
- .dont_resolve_fwds = false,
- },
},
{
.descr = "dedup: strings deduplication",
@@ -6674,9 +6680,6 @@ const struct btf_dedup_test dedup_tests[] = {
},
BTF_STR_SEC("\0int\0long int"),
},
- .opts = {
- .dont_resolve_fwds = false,
- },
},
{
.descr = "dedup: struct example #1",
@@ -6757,9 +6760,6 @@ const struct btf_dedup_test dedup_tests[] = {
},
BTF_STR_SEC("\0a\0b\0c\0d\0int\0float\0next\0s"),
},
- .opts = {
- .dont_resolve_fwds = false,
- },
},
{
.descr = "dedup: struct <-> fwd resolution w/ hash collision",
@@ -6802,8 +6802,7 @@ const struct btf_dedup_test dedup_tests[] = {
BTF_STR_SEC("\0s\0x"),
},
.opts = {
- .dont_resolve_fwds = false,
- .dedup_table_size = 1, /* force hash collisions */
+ .force_collisions = true, /* force hash collisions */
},
},
{
@@ -6849,8 +6848,7 @@ const struct btf_dedup_test dedup_tests[] = {
BTF_STR_SEC("\0s\0x"),
},
.opts = {
- .dont_resolve_fwds = false,
- .dedup_table_size = 1, /* force hash collisions */
+ .force_collisions = true, /* force hash collisions */
},
},
{
@@ -6874,15 +6872,16 @@ const struct btf_dedup_test dedup_tests[] = {
BTF_RESTRICT_ENC(8), /* [11] restrict */
BTF_FUNC_PROTO_ENC(1, 2), /* [12] func_proto */
BTF_FUNC_PROTO_ARG_ENC(NAME_TBD, 1),
- BTF_FUNC_PROTO_ARG_ENC(NAME_TBD, 8),
+ BTF_FUNC_PROTO_ARG_ENC(NAME_TBD, 18),
BTF_FUNC_ENC(NAME_TBD, 12), /* [13] func */
BTF_TYPE_FLOAT_ENC(NAME_TBD, 2), /* [14] float */
BTF_DECL_TAG_ENC(NAME_TBD, 13, -1), /* [15] decl_tag */
BTF_DECL_TAG_ENC(NAME_TBD, 13, 1), /* [16] decl_tag */
BTF_DECL_TAG_ENC(NAME_TBD, 7, -1), /* [17] decl_tag */
+ BTF_TYPE_TAG_ENC(NAME_TBD, 8), /* [18] type_tag */
BTF_END_RAW,
},
- BTF_STR_SEC("\0A\0B\0C\0D\0E\0F\0G\0H\0I\0J\0K\0L\0M\0N\0O\0P\0Q"),
+ BTF_STR_SEC("\0A\0B\0C\0D\0E\0F\0G\0H\0I\0J\0K\0L\0M\0N\0O\0P\0Q\0R"),
},
.expect = {
.raw_types = {
@@ -6903,18 +6902,16 @@ const struct btf_dedup_test dedup_tests[] = {
BTF_RESTRICT_ENC(8), /* [11] restrict */
BTF_FUNC_PROTO_ENC(1, 2), /* [12] func_proto */
BTF_FUNC_PROTO_ARG_ENC(NAME_TBD, 1),
- BTF_FUNC_PROTO_ARG_ENC(NAME_TBD, 8),
+ BTF_FUNC_PROTO_ARG_ENC(NAME_TBD, 18),
BTF_FUNC_ENC(NAME_TBD, 12), /* [13] func */
BTF_TYPE_FLOAT_ENC(NAME_TBD, 2), /* [14] float */
BTF_DECL_TAG_ENC(NAME_TBD, 13, -1), /* [15] decl_tag */
BTF_DECL_TAG_ENC(NAME_TBD, 13, 1), /* [16] decl_tag */
BTF_DECL_TAG_ENC(NAME_TBD, 7, -1), /* [17] decl_tag */
+ BTF_TYPE_TAG_ENC(NAME_TBD, 8), /* [18] type_tag */
BTF_END_RAW,
},
- BTF_STR_SEC("\0A\0B\0C\0D\0E\0F\0G\0H\0I\0J\0K\0L\0M\0N\0O\0P\0Q"),
- },
- .opts = {
- .dont_resolve_fwds = false,
+ BTF_STR_SEC("\0A\0B\0C\0D\0E\0F\0G\0H\0I\0J\0K\0L\0M\0N\0O\0P\0Q\0R"),
},
},
{
@@ -6967,9 +6964,6 @@ const struct btf_dedup_test dedup_tests[] = {
},
BTF_STR_SEC("\0int\0some other int\0float"),
},
- .opts = {
- .dont_resolve_fwds = false,
- },
},
{
.descr = "dedup: enum fwd resolution",
@@ -7011,9 +7005,6 @@ const struct btf_dedup_test dedup_tests[] = {
},
BTF_STR_SEC("\0e1\0e1_val\0e2\0e2_val"),
},
- .opts = {
- .dont_resolve_fwds = false,
- },
},
{
.descr = "dedup: datasec and vars pass-through",
@@ -7056,8 +7047,7 @@ const struct btf_dedup_test dedup_tests[] = {
BTF_STR_SEC("\0.bss\0t"),
},
.opts = {
- .dont_resolve_fwds = false,
- .dedup_table_size = 1
+ .force_collisions = true
},
},
{
@@ -7101,9 +7091,6 @@ const struct btf_dedup_test dedup_tests[] = {
},
BTF_STR_SEC("\0t\0a1\0a2\0f\0tag"),
},
- .opts = {
- .dont_resolve_fwds = false,
- },
},
{
.descr = "dedup: func/func_param tags",
@@ -7154,9 +7141,6 @@ const struct btf_dedup_test dedup_tests[] = {
},
BTF_STR_SEC("\0a1\0a2\0f\0tag1\0tag2\0tag3"),
},
- .opts = {
- .dont_resolve_fwds = false,
- },
},
{
.descr = "dedup: struct/struct_member tags",
@@ -7202,9 +7186,6 @@ const struct btf_dedup_test dedup_tests[] = {
},
BTF_STR_SEC("\0t\0m1\0m2\0tag1\0tag2\0tag3"),
},
- .opts = {
- .dont_resolve_fwds = false,
- },
},
{
.descr = "dedup: typedef tags",
@@ -7235,8 +7216,160 @@ const struct btf_dedup_test dedup_tests[] = {
},
BTF_STR_SEC("\0t\0tag1\0tag2\0tag3"),
},
- .opts = {
- .dont_resolve_fwds = false,
+},
+{
+ .descr = "dedup: btf_type_tag #1",
+ .input = {
+ .raw_types = {
+ /* ptr -> tag2 -> tag1 -> int */
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */
+ BTF_TYPE_TAG_ENC(NAME_NTH(1), 1), /* [2] */
+ BTF_TYPE_TAG_ENC(NAME_NTH(2), 2), /* [3] */
+ BTF_PTR_ENC(3), /* [4] */
+ /* ptr -> tag2 -> tag1 -> int */
+ BTF_TYPE_TAG_ENC(NAME_NTH(1), 1), /* [5] */
+ BTF_TYPE_TAG_ENC(NAME_NTH(2), 5), /* [6] */
+ BTF_PTR_ENC(6), /* [7] */
+ /* ptr -> tag1 -> int */
+ BTF_TYPE_TAG_ENC(NAME_NTH(1), 1), /* [8] */
+ BTF_PTR_ENC(8), /* [9] */
+ BTF_END_RAW,
+ },
+ BTF_STR_SEC("\0tag1\0tag2"),
+ },
+ .expect = {
+ .raw_types = {
+ /* ptr -> tag2 -> tag1 -> int */
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */
+ BTF_TYPE_TAG_ENC(NAME_NTH(1), 1), /* [2] */
+ BTF_TYPE_TAG_ENC(NAME_NTH(2), 2), /* [3] */
+ BTF_PTR_ENC(3), /* [4] */
+ /* ptr -> tag1 -> int */
+ BTF_PTR_ENC(2), /* [5] */
+ BTF_END_RAW,
+ },
+ BTF_STR_SEC("\0tag1\0tag2"),
+ },
+},
+{
+ .descr = "dedup: btf_type_tag #2",
+ .input = {
+ .raw_types = {
+ /* ptr -> tag2 -> tag1 -> int */
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */
+ BTF_TYPE_TAG_ENC(NAME_NTH(1), 1), /* [2] */
+ BTF_TYPE_TAG_ENC(NAME_NTH(2), 2), /* [3] */
+ BTF_PTR_ENC(3), /* [4] */
+ /* ptr -> tag2 -> int */
+ BTF_TYPE_TAG_ENC(NAME_NTH(2), 1), /* [5] */
+ BTF_PTR_ENC(5), /* [6] */
+ BTF_END_RAW,
+ },
+ BTF_STR_SEC("\0tag1\0tag2"),
+ },
+ .expect = {
+ .raw_types = {
+ /* ptr -> tag2 -> tag1 -> int */
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */
+ BTF_TYPE_TAG_ENC(NAME_NTH(1), 1), /* [2] */
+ BTF_TYPE_TAG_ENC(NAME_NTH(2), 2), /* [3] */
+ BTF_PTR_ENC(3), /* [4] */
+ /* ptr -> tag2 -> int */
+ BTF_TYPE_TAG_ENC(NAME_NTH(2), 1), /* [5] */
+ BTF_PTR_ENC(5), /* [6] */
+ BTF_END_RAW,
+ },
+ BTF_STR_SEC("\0tag1\0tag2"),
+ },
+},
+{
+ .descr = "dedup: btf_type_tag #3",
+ .input = {
+ .raw_types = {
+ /* ptr -> tag2 -> tag1 -> int */
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */
+ BTF_TYPE_TAG_ENC(NAME_NTH(1), 1), /* [2] */
+ BTF_TYPE_TAG_ENC(NAME_NTH(2), 2), /* [3] */
+ BTF_PTR_ENC(3), /* [4] */
+ /* ptr -> tag1 -> tag2 -> int */
+ BTF_TYPE_TAG_ENC(NAME_NTH(2), 1), /* [5] */
+ BTF_TYPE_TAG_ENC(NAME_NTH(1), 5), /* [6] */
+ BTF_PTR_ENC(6), /* [7] */
+ BTF_END_RAW,
+ },
+ BTF_STR_SEC("\0tag1\0tag2"),
+ },
+ .expect = {
+ .raw_types = {
+ /* ptr -> tag2 -> tag1 -> int */
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */
+ BTF_TYPE_TAG_ENC(NAME_NTH(1), 1), /* [2] */
+ BTF_TYPE_TAG_ENC(NAME_NTH(2), 2), /* [3] */
+ BTF_PTR_ENC(3), /* [4] */
+ /* ptr -> tag1 -> tag2 -> int */
+ BTF_TYPE_TAG_ENC(NAME_NTH(2), 1), /* [5] */
+ BTF_TYPE_TAG_ENC(NAME_NTH(1), 5), /* [6] */
+ BTF_PTR_ENC(6), /* [7] */
+ BTF_END_RAW,
+ },
+ BTF_STR_SEC("\0tag1\0tag2"),
+ },
+},
+{
+ .descr = "dedup: btf_type_tag #4",
+ .input = {
+ .raw_types = {
+ /* ptr -> tag1 -> int */
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */
+ BTF_TYPE_TAG_ENC(NAME_NTH(1), 1), /* [2] */
+ BTF_PTR_ENC(2), /* [3] */
+ /* ptr -> tag1 -> long */
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 64, 8), /* [4] */
+ BTF_TYPE_TAG_ENC(NAME_NTH(1), 4), /* [5] */
+ BTF_PTR_ENC(5), /* [6] */
+ BTF_END_RAW,
+ },
+ BTF_STR_SEC("\0tag1"),
+ },
+ .expect = {
+ .raw_types = {
+ /* ptr -> tag1 -> int */
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */
+ BTF_TYPE_TAG_ENC(NAME_NTH(1), 1), /* [2] */
+ BTF_PTR_ENC(2), /* [3] */
+ /* ptr -> tag1 -> long */
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 64, 8), /* [4] */
+ BTF_TYPE_TAG_ENC(NAME_NTH(1), 4), /* [5] */
+ BTF_PTR_ENC(5), /* [6] */
+ BTF_END_RAW,
+ },
+ BTF_STR_SEC("\0tag1"),
+ },
+},
+{
+ .descr = "dedup: btf_type_tag #5, struct",
+ .input = {
+ .raw_types = {
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */
+ BTF_TYPE_TAG_ENC(NAME_NTH(1), 1), /* [2] */
+ BTF_TYPE_ENC(NAME_NTH(2), BTF_INFO_ENC(BTF_KIND_STRUCT, 1, 1), 4), /* [3] */
+ BTF_MEMBER_ENC(NAME_NTH(3), 2, BTF_MEMBER_OFFSET(0, 0)),
+ BTF_TYPE_TAG_ENC(NAME_NTH(1), 1), /* [4] */
+ BTF_TYPE_ENC(NAME_NTH(2), BTF_INFO_ENC(BTF_KIND_STRUCT, 1, 1), 4), /* [5] */
+ BTF_MEMBER_ENC(NAME_NTH(3), 4, BTF_MEMBER_OFFSET(0, 0)),
+ BTF_END_RAW,
+ },
+ BTF_STR_SEC("\0tag1\0t\0m"),
+ },
+ .expect = {
+ .raw_types = {
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */
+ BTF_TYPE_TAG_ENC(NAME_NTH(1), 1), /* [2] */
+ BTF_TYPE_ENC(NAME_NTH(2), BTF_INFO_ENC(BTF_KIND_STRUCT, 1, 1), 4), /* [3] */
+ BTF_MEMBER_ENC(NAME_NTH(3), 2, BTF_MEMBER_OFFSET(0, 0)),
+ BTF_END_RAW,
+ },
+ BTF_STR_SEC("\0tag1\0t\0m"),
},
},
@@ -7257,6 +7390,7 @@ static int btf_type_size(const struct btf_type *t)
case BTF_KIND_TYPEDEF:
case BTF_KIND_FUNC:
case BTF_KIND_FLOAT:
+ case BTF_KIND_TYPE_TAG:
return base_size;
case BTF_KIND_INT:
return base_size + sizeof(__u32);
@@ -7295,7 +7429,7 @@ static void dump_btf_strings(const char *strs, __u32 len)
static void do_test_dedup(unsigned int test_num)
{
- const struct btf_dedup_test *test = &dedup_tests[test_num - 1];
+ struct btf_dedup_test *test = &dedup_tests[test_num - 1];
__u32 test_nr_types, expect_nr_types, test_btf_size, expect_btf_size;
const struct btf_header *test_hdr, *expect_hdr;
struct btf *test_btf = NULL, *expect_btf = NULL;
@@ -7339,7 +7473,8 @@ static void do_test_dedup(unsigned int test_num)
goto done;
}
- err = btf__dedup(test_btf, NULL, &test->opts);
+ test->opts.sz = sizeof(test->opts);
+ err = btf__dedup(test_btf, &test->opts);
if (CHECK(err, "btf_dedup failed errno:%d", err)) {
err = -1;
goto done;
diff --git a/tools/testing/selftests/bpf/prog_tests/btf_dedup_split.c b/tools/testing/selftests/bpf/prog_tests/btf_dedup_split.c
index 64554fd..90aac43 100644
--- a/tools/testing/selftests/bpf/prog_tests/btf_dedup_split.c
+++ b/tools/testing/selftests/bpf/prog_tests/btf_dedup_split.c
@@ -92,7 +92,7 @@ struct s2 {\n\
int *f3;\n\
};\n\n", "c_dump");
- err = btf__dedup(btf2, NULL, NULL);
+ err = btf__dedup(btf2, NULL);
if (!ASSERT_OK(err, "btf_dedup"))
goto cleanup;
@@ -186,7 +186,7 @@ static void test_split_fwd_resolve() {
"\t'f1' type_id=7 bits_offset=0\n"
"\t'f2' type_id=9 bits_offset=64");
- err = btf__dedup(btf2, NULL, NULL);
+ err = btf__dedup(btf2, NULL);
if (!ASSERT_OK(err, "btf_dedup"))
goto cleanup;
@@ -283,7 +283,7 @@ static void test_split_struct_duped() {
"[13] STRUCT 's3' size=8 vlen=1\n"
"\t'f1' type_id=12 bits_offset=0");
- err = btf__dedup(btf2, NULL, NULL);
+ err = btf__dedup(btf2, NULL);
if (!ASSERT_OK(err, "btf_dedup"))
goto cleanup;
@@ -314,6 +314,117 @@ static void test_split_struct_duped() {
btf__free(btf1);
}
+static void btf_add_dup_struct_in_cu(struct btf *btf, int start_id)
+{
+#define ID(n) (start_id + n)
+ btf__set_pointer_size(btf, 8); /* enforce 64-bit arch */
+
+ btf__add_int(btf, "int", 4, BTF_INT_SIGNED); /* [1] int */
+
+ btf__add_struct(btf, "s", 8); /* [2] struct s { */
+ btf__add_field(btf, "a", ID(3), 0, 0); /* struct anon a; */
+ btf__add_field(btf, "b", ID(4), 0, 0); /* struct anon b; */
+ /* } */
+
+ btf__add_struct(btf, "(anon)", 8); /* [3] struct anon { */
+ btf__add_field(btf, "f1", ID(1), 0, 0); /* int f1; */
+ btf__add_field(btf, "f2", ID(1), 32, 0); /* int f2; */
+ /* } */
+
+ btf__add_struct(btf, "(anon)", 8); /* [4] struct anon { */
+ btf__add_field(btf, "f1", ID(1), 0, 0); /* int f1; */
+ btf__add_field(btf, "f2", ID(1), 32, 0); /* int f2; */
+ /* } */
+#undef ID
+}
+
+static void test_split_dup_struct_in_cu()
+{
+ struct btf *btf1, *btf2 = NULL;
+ int err;
+
+ /* generate the base data.. */
+ btf1 = btf__new_empty();
+ if (!ASSERT_OK_PTR(btf1, "empty_main_btf"))
+ return;
+
+ btf_add_dup_struct_in_cu(btf1, 0);
+
+ VALIDATE_RAW_BTF(
+ btf1,
+ "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED",
+ "[2] STRUCT 's' size=8 vlen=2\n"
+ "\t'a' type_id=3 bits_offset=0\n"
+ "\t'b' type_id=4 bits_offset=0",
+ "[3] STRUCT '(anon)' size=8 vlen=2\n"
+ "\t'f1' type_id=1 bits_offset=0\n"
+ "\t'f2' type_id=1 bits_offset=32",
+ "[4] STRUCT '(anon)' size=8 vlen=2\n"
+ "\t'f1' type_id=1 bits_offset=0\n"
+ "\t'f2' type_id=1 bits_offset=32");
+
+ /* ..dedup them... */
+ err = btf__dedup(btf1, NULL);
+ if (!ASSERT_OK(err, "btf_dedup"))
+ goto cleanup;
+
+ VALIDATE_RAW_BTF(
+ btf1,
+ "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED",
+ "[2] STRUCT 's' size=8 vlen=2\n"
+ "\t'a' type_id=3 bits_offset=0\n"
+ "\t'b' type_id=3 bits_offset=0",
+ "[3] STRUCT '(anon)' size=8 vlen=2\n"
+ "\t'f1' type_id=1 bits_offset=0\n"
+ "\t'f2' type_id=1 bits_offset=32");
+
+ /* and add the same data on top of it */
+ btf2 = btf__new_empty_split(btf1);
+ if (!ASSERT_OK_PTR(btf2, "empty_split_btf"))
+ goto cleanup;
+
+ btf_add_dup_struct_in_cu(btf2, 3);
+
+ VALIDATE_RAW_BTF(
+ btf2,
+ "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED",
+ "[2] STRUCT 's' size=8 vlen=2\n"
+ "\t'a' type_id=3 bits_offset=0\n"
+ "\t'b' type_id=3 bits_offset=0",
+ "[3] STRUCT '(anon)' size=8 vlen=2\n"
+ "\t'f1' type_id=1 bits_offset=0\n"
+ "\t'f2' type_id=1 bits_offset=32",
+ "[4] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED",
+ "[5] STRUCT 's' size=8 vlen=2\n"
+ "\t'a' type_id=6 bits_offset=0\n"
+ "\t'b' type_id=7 bits_offset=0",
+ "[6] STRUCT '(anon)' size=8 vlen=2\n"
+ "\t'f1' type_id=4 bits_offset=0\n"
+ "\t'f2' type_id=4 bits_offset=32",
+ "[7] STRUCT '(anon)' size=8 vlen=2\n"
+ "\t'f1' type_id=4 bits_offset=0\n"
+ "\t'f2' type_id=4 bits_offset=32");
+
+ err = btf__dedup(btf2, NULL);
+ if (!ASSERT_OK(err, "btf_dedup"))
+ goto cleanup;
+
+ /* after dedup it should match the original data */
+ VALIDATE_RAW_BTF(
+ btf2,
+ "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED",
+ "[2] STRUCT 's' size=8 vlen=2\n"
+ "\t'a' type_id=3 bits_offset=0\n"
+ "\t'b' type_id=3 bits_offset=0",
+ "[3] STRUCT '(anon)' size=8 vlen=2\n"
+ "\t'f1' type_id=1 bits_offset=0\n"
+ "\t'f2' type_id=1 bits_offset=32");
+
+cleanup:
+ btf__free(btf2);
+ btf__free(btf1);
+}
+
void test_btf_dedup_split()
{
if (test__start_subtest("split_simple"))
@@ -322,4 +433,6 @@ void test_btf_dedup_split()
test_split_struct_duped();
if (test__start_subtest("split_fwd_resolve"))
test_split_fwd_resolve();
+ if (test__start_subtest("split_dup_struct_in_cu"))
+ test_split_dup_struct_in_cu();
}
diff --git a/tools/testing/selftests/bpf/prog_tests/btf_dump.c b/tools/testing/selftests/bpf/prog_tests/btf_dump.c
index aa76360..9e26903 100644
--- a/tools/testing/selftests/bpf/prog_tests/btf_dump.c
+++ b/tools/testing/selftests/bpf/prog_tests/btf_dump.c
@@ -13,25 +13,23 @@ static struct btf_dump_test_case {
const char *name;
const char *file;
bool known_ptr_sz;
- struct btf_dump_opts opts;
} btf_dump_test_cases[] = {
- {"btf_dump: syntax", "btf_dump_test_case_syntax", true, {}},
- {"btf_dump: ordering", "btf_dump_test_case_ordering", false, {}},
- {"btf_dump: padding", "btf_dump_test_case_padding", true, {}},
- {"btf_dump: packing", "btf_dump_test_case_packing", true, {}},
- {"btf_dump: bitfields", "btf_dump_test_case_bitfields", true, {}},
- {"btf_dump: multidim", "btf_dump_test_case_multidim", false, {}},
- {"btf_dump: namespacing", "btf_dump_test_case_namespacing", false, {}},
+ {"btf_dump: syntax", "btf_dump_test_case_syntax", true},
+ {"btf_dump: ordering", "btf_dump_test_case_ordering", false},
+ {"btf_dump: padding", "btf_dump_test_case_padding", true},
+ {"btf_dump: packing", "btf_dump_test_case_packing", true},
+ {"btf_dump: bitfields", "btf_dump_test_case_bitfields", true},
+ {"btf_dump: multidim", "btf_dump_test_case_multidim", false},
+ {"btf_dump: namespacing", "btf_dump_test_case_namespacing", false},
};
-static int btf_dump_all_types(const struct btf *btf,
- const struct btf_dump_opts *opts)
+static int btf_dump_all_types(const struct btf *btf, void *ctx)
{
size_t type_cnt = btf__type_cnt(btf);
struct btf_dump *d;
int err = 0, id;
- d = btf_dump__new(btf, NULL, opts, btf_dump_printf);
+ d = btf_dump__new(btf, btf_dump_printf, ctx, NULL);
err = libbpf_get_error(d);
if (err)
return err;
@@ -88,8 +86,7 @@ static int test_btf_dump_case(int n, struct btf_dump_test_case *t)
goto done;
}
- t->opts.ctx = f;
- err = btf_dump_all_types(btf, &t->opts);
+ err = btf_dump_all_types(btf, f);
fclose(f);
close(fd);
if (CHECK(err, "btf_dump", "failure during C dumping: %d\n", err)) {
@@ -137,7 +134,6 @@ static void test_btf_dump_incremental(void)
{
struct btf *btf = NULL;
struct btf_dump *d = NULL;
- struct btf_dump_opts opts;
int id, err, i;
dump_buf_file = open_memstream(&dump_buf, &dump_buf_sz);
@@ -146,8 +142,7 @@ static void test_btf_dump_incremental(void)
btf = btf__new_empty();
if (!ASSERT_OK_PTR(btf, "new_empty"))
goto err_out;
- opts.ctx = dump_buf_file;
- d = btf_dump__new(btf, NULL, &opts, btf_dump_printf);
+ d = btf_dump__new(btf, btf_dump_printf, dump_buf_file, NULL);
if (!ASSERT_OK(libbpf_get_error(d), "btf_dump__new"))
goto err_out;
@@ -328,7 +323,7 @@ static void test_btf_dump_int_data(struct btf *btf, struct btf_dump *d,
char *str)
{
#ifdef __SIZEOF_INT128__
- __int128 i = 0xffffffffffffffff;
+ unsigned __int128 i = 0xffffffffffffffff;
/* this dance is required because we cannot directly initialize
* a 128-bit value to anything larger than a 64-bit value.
@@ -761,7 +756,7 @@ static void test_btf_dump_struct_data(struct btf *btf, struct btf_dump *d,
/* overflow bpf_sock_ops struct with final element nonzero/zero.
* Regardless of the value of the final field, we don't have all the
* data we need to display it, so we should trigger an overflow.
- * In other words oveflow checking should trump "is field zero?"
+ * In other words overflow checking should trump "is field zero?"
* checks because if we've overflowed, it shouldn't matter what the
* field is - we can't trust its value so shouldn't display it.
*/
@@ -814,26 +809,28 @@ static void test_btf_datasec(struct btf *btf, struct btf_dump *d, char *str,
static void test_btf_dump_datasec_data(char *str)
{
- struct btf *btf = btf__parse("xdping_kern.o", NULL);
- struct btf_dump_opts opts = { .ctx = str };
+ struct btf *btf;
char license[4] = "GPL";
struct btf_dump *d;
+ btf = btf__parse("xdping_kern.o", NULL);
if (!ASSERT_OK_PTR(btf, "xdping_kern.o BTF not found"))
return;
- d = btf_dump__new(btf, NULL, &opts, btf_dump_snprintf);
+ d = btf_dump__new(btf, btf_dump_snprintf, str, NULL);
if (!ASSERT_OK_PTR(d, "could not create BTF dump"))
- return;
+ goto out;
test_btf_datasec(btf, d, str, "license",
"SEC(\"license\") char[4] _license = (char[4])['G','P','L',];",
license, sizeof(license));
+out:
+ btf_dump__free(d);
+ btf__free(btf);
}
void test_btf_dump() {
char str[STRSIZE];
- struct btf_dump_opts opts = { .ctx = str };
struct btf_dump *d;
struct btf *btf;
int i;
@@ -853,7 +850,7 @@ void test_btf_dump() {
if (!ASSERT_OK_PTR(btf, "no kernel BTF found"))
return;
- d = btf_dump__new(btf, NULL, &opts, btf_dump_snprintf);
+ d = btf_dump__new(btf, btf_dump_snprintf, str, NULL);
if (!ASSERT_OK_PTR(d, "could not create BTF dump"))
return;
diff --git a/tools/testing/selftests/bpf/prog_tests/btf_split.c b/tools/testing/selftests/bpf/prog_tests/btf_split.c
index b1ffe61..eef1158 100644
--- a/tools/testing/selftests/bpf/prog_tests/btf_split.c
+++ b/tools/testing/selftests/bpf/prog_tests/btf_split.c
@@ -13,7 +13,6 @@ static void btf_dump_printf(void *ctx, const char *fmt, va_list args)
}
void test_btf_split() {
- struct btf_dump_opts opts;
struct btf_dump *d = NULL;
const struct btf_type *t;
struct btf *btf1, *btf2;
@@ -68,8 +67,7 @@ void test_btf_split() {
dump_buf_file = open_memstream(&dump_buf, &dump_buf_sz);
if (!ASSERT_OK_PTR(dump_buf_file, "dump_memstream"))
return;
- opts.ctx = dump_buf_file;
- d = btf_dump__new(btf2, NULL, &opts, btf_dump_printf);
+ d = btf_dump__new(btf2, btf_dump_printf, dump_buf_file, NULL);
if (!ASSERT_OK_PTR(d, "btf_dump__new"))
goto cleanup;
for (i = 1; i < btf__type_cnt(btf2); i++) {
diff --git a/tools/testing/selftests/bpf/prog_tests/btf_tag.c b/tools/testing/selftests/bpf/prog_tests/btf_tag.c
index 91821f4..88d63e2 100644
--- a/tools/testing/selftests/bpf/prog_tests/btf_tag.c
+++ b/tools/testing/selftests/bpf/prog_tests/btf_tag.c
@@ -1,20 +1,50 @@
// SPDX-License-Identifier: GPL-2.0
/* Copyright (c) 2021 Facebook */
#include <test_progs.h>
-#include "tag.skel.h"
+#include "btf_decl_tag.skel.h"
-void test_btf_tag(void)
+/* struct btf_type_tag_test is referenced in btf_type_tag.skel.h */
+struct btf_type_tag_test {
+ int **p;
+};
+#include "btf_type_tag.skel.h"
+
+static void test_btf_decl_tag(void)
{
- struct tag *skel;
+ struct btf_decl_tag *skel;
- skel = tag__open_and_load();
- if (!ASSERT_OK_PTR(skel, "btf_tag"))
+ skel = btf_decl_tag__open_and_load();
+ if (!ASSERT_OK_PTR(skel, "btf_decl_tag"))
return;
if (skel->rodata->skip_tests) {
- printf("%s:SKIP: btf_tag attribute not supported", __func__);
+ printf("%s:SKIP: btf_decl_tag attribute not supported", __func__);
test__skip();
}
- tag__destroy(skel);
+ btf_decl_tag__destroy(skel);
+}
+
+static void test_btf_type_tag(void)
+{
+ struct btf_type_tag *skel;
+
+ skel = btf_type_tag__open_and_load();
+ if (!ASSERT_OK_PTR(skel, "btf_type_tag"))
+ return;
+
+ if (skel->rodata->skip_tests) {
+ printf("%s:SKIP: btf_type_tag attribute not supported", __func__);
+ test__skip();
+ }
+
+ btf_type_tag__destroy(skel);
+}
+
+void test_btf_tag(void)
+{
+ if (test__start_subtest("btf_decl_tag"))
+ test_btf_decl_tag();
+ if (test__start_subtest("btf_type_tag"))
+ test_btf_type_tag();
}
diff --git a/tools/testing/selftests/bpf/prog_tests/btf_write.c b/tools/testing/selftests/bpf/prog_tests/btf_write.c
index b912eeb..addf99c 100644
--- a/tools/testing/selftests/bpf/prog_tests/btf_write.c
+++ b/tools/testing/selftests/bpf/prog_tests/btf_write.c
@@ -297,6 +297,16 @@ static void gen_btf(struct btf *btf)
ASSERT_EQ(btf_decl_tag(t)->component_idx, 1, "tag_component_idx");
ASSERT_STREQ(btf_type_raw_dump(btf, 19),
"[19] DECL_TAG 'tag2' type_id=14 component_idx=1", "raw_dump");
+
+ /* TYPE_TAG */
+ id = btf__add_type_tag(btf, "tag1", 1);
+ ASSERT_EQ(id, 20, "tag_id");
+ t = btf__type_by_id(btf, 20);
+ ASSERT_STREQ(btf__str_by_offset(btf, t->name_off), "tag1", "tag_value");
+ ASSERT_EQ(btf_kind(t), BTF_KIND_TYPE_TAG, "tag_kind");
+ ASSERT_EQ(t->type, 1, "tag_type");
+ ASSERT_STREQ(btf_type_raw_dump(btf, 20),
+ "[20] TYPE_TAG 'tag1' type_id=1", "raw_dump");
}
static void test_btf_add()
@@ -337,7 +347,8 @@ static void test_btf_add()
"[17] DATASEC 'datasec1' size=12 vlen=1\n"
"\ttype_id=1 offset=4 size=8",
"[18] DECL_TAG 'tag1' type_id=16 component_idx=-1",
- "[19] DECL_TAG 'tag2' type_id=14 component_idx=1");
+ "[19] DECL_TAG 'tag2' type_id=14 component_idx=1",
+ "[20] TYPE_TAG 'tag1' type_id=1");
btf__free(btf);
}
@@ -359,7 +370,7 @@ static void test_btf_add_btf()
gen_btf(btf2);
id = btf__add_btf(btf1, btf2);
- if (!ASSERT_EQ(id, 20, "id"))
+ if (!ASSERT_EQ(id, 21, "id"))
goto cleanup;
VALIDATE_RAW_BTF(
@@ -391,35 +402,37 @@ static void test_btf_add_btf()
"\ttype_id=1 offset=4 size=8",
"[18] DECL_TAG 'tag1' type_id=16 component_idx=-1",
"[19] DECL_TAG 'tag2' type_id=14 component_idx=1",
+ "[20] TYPE_TAG 'tag1' type_id=1",
/* types appended from the second BTF */
- "[20] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED",
- "[21] PTR '(anon)' type_id=20",
- "[22] CONST '(anon)' type_id=24",
- "[23] VOLATILE '(anon)' type_id=22",
- "[24] RESTRICT '(anon)' type_id=23",
- "[25] ARRAY '(anon)' type_id=21 index_type_id=20 nr_elems=10",
- "[26] STRUCT 's1' size=8 vlen=2\n"
- "\t'f1' type_id=20 bits_offset=0\n"
- "\t'f2' type_id=20 bits_offset=32 bitfield_size=16",
- "[27] UNION 'u1' size=8 vlen=1\n"
- "\t'f1' type_id=20 bits_offset=0 bitfield_size=16",
- "[28] ENUM 'e1' size=4 vlen=2\n"
+ "[21] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED",
+ "[22] PTR '(anon)' type_id=21",
+ "[23] CONST '(anon)' type_id=25",
+ "[24] VOLATILE '(anon)' type_id=23",
+ "[25] RESTRICT '(anon)' type_id=24",
+ "[26] ARRAY '(anon)' type_id=22 index_type_id=21 nr_elems=10",
+ "[27] STRUCT 's1' size=8 vlen=2\n"
+ "\t'f1' type_id=21 bits_offset=0\n"
+ "\t'f2' type_id=21 bits_offset=32 bitfield_size=16",
+ "[28] UNION 'u1' size=8 vlen=1\n"
+ "\t'f1' type_id=21 bits_offset=0 bitfield_size=16",
+ "[29] ENUM 'e1' size=4 vlen=2\n"
"\t'v1' val=1\n"
"\t'v2' val=2",
- "[29] FWD 'struct_fwd' fwd_kind=struct",
- "[30] FWD 'union_fwd' fwd_kind=union",
- "[31] ENUM 'enum_fwd' size=4 vlen=0",
- "[32] TYPEDEF 'typedef1' type_id=20",
- "[33] FUNC 'func1' type_id=34 linkage=global",
- "[34] FUNC_PROTO '(anon)' ret_type_id=20 vlen=2\n"
- "\t'p1' type_id=20\n"
- "\t'p2' type_id=21",
- "[35] VAR 'var1' type_id=20, linkage=global-alloc",
- "[36] DATASEC 'datasec1' size=12 vlen=1\n"
- "\ttype_id=20 offset=4 size=8",
- "[37] DECL_TAG 'tag1' type_id=35 component_idx=-1",
- "[38] DECL_TAG 'tag2' type_id=33 component_idx=1");
+ "[30] FWD 'struct_fwd' fwd_kind=struct",
+ "[31] FWD 'union_fwd' fwd_kind=union",
+ "[32] ENUM 'enum_fwd' size=4 vlen=0",
+ "[33] TYPEDEF 'typedef1' type_id=21",
+ "[34] FUNC 'func1' type_id=35 linkage=global",
+ "[35] FUNC_PROTO '(anon)' ret_type_id=21 vlen=2\n"
+ "\t'p1' type_id=21\n"
+ "\t'p2' type_id=22",
+ "[36] VAR 'var1' type_id=21, linkage=global-alloc",
+ "[37] DATASEC 'datasec1' size=12 vlen=1\n"
+ "\ttype_id=21 offset=4 size=8",
+ "[38] DECL_TAG 'tag1' type_id=36 component_idx=-1",
+ "[39] DECL_TAG 'tag2' type_id=34 component_idx=1",
+ "[40] TYPE_TAG 'tag1' type_id=21");
cleanup:
btf__free(btf1);
diff --git a/tools/testing/selftests/bpf/prog_tests/cgroup_attach_autodetach.c b/tools/testing/selftests/bpf/prog_tests/cgroup_attach_autodetach.c
index 5de485c..858916d 100644
--- a/tools/testing/selftests/bpf/prog_tests/cgroup_attach_autodetach.c
+++ b/tools/testing/selftests/bpf/prog_tests/cgroup_attach_autodetach.c
@@ -16,7 +16,7 @@ static int prog_load(void)
};
size_t insns_cnt = sizeof(prog) / sizeof(struct bpf_insn);
- return bpf_load_program(BPF_PROG_TYPE_CGROUP_SKB,
+ return bpf_test_load_program(BPF_PROG_TYPE_CGROUP_SKB,
prog, insns_cnt, "GPL", 0,
bpf_log_buf, BPF_LOG_BUF_SIZE);
}
diff --git a/tools/testing/selftests/bpf/prog_tests/cgroup_attach_multi.c b/tools/testing/selftests/bpf/prog_tests/cgroup_attach_multi.c
index 731bea8..d3e8f72 100644
--- a/tools/testing/selftests/bpf/prog_tests/cgroup_attach_multi.c
+++ b/tools/testing/selftests/bpf/prog_tests/cgroup_attach_multi.c
@@ -15,22 +15,22 @@ static int prog_load_cnt(int verdict, int val)
int cgroup_storage_fd, percpu_cgroup_storage_fd;
if (map_fd < 0)
- map_fd = bpf_create_map(BPF_MAP_TYPE_ARRAY, 4, 8, 1, 0);
+ map_fd = bpf_map_create(BPF_MAP_TYPE_ARRAY, NULL, 4, 8, 1, NULL);
if (map_fd < 0) {
printf("failed to create map '%s'\n", strerror(errno));
return -1;
}
- cgroup_storage_fd = bpf_create_map(BPF_MAP_TYPE_CGROUP_STORAGE,
- sizeof(struct bpf_cgroup_storage_key), 8, 0, 0);
+ cgroup_storage_fd = bpf_map_create(BPF_MAP_TYPE_CGROUP_STORAGE, NULL,
+ sizeof(struct bpf_cgroup_storage_key), 8, 0, NULL);
if (cgroup_storage_fd < 0) {
printf("failed to create map '%s'\n", strerror(errno));
return -1;
}
- percpu_cgroup_storage_fd = bpf_create_map(
- BPF_MAP_TYPE_PERCPU_CGROUP_STORAGE,
- sizeof(struct bpf_cgroup_storage_key), 8, 0, 0);
+ percpu_cgroup_storage_fd = bpf_map_create(
+ BPF_MAP_TYPE_PERCPU_CGROUP_STORAGE, NULL,
+ sizeof(struct bpf_cgroup_storage_key), 8, 0, NULL);
if (percpu_cgroup_storage_fd < 0) {
printf("failed to create map '%s'\n", strerror(errno));
return -1;
@@ -66,7 +66,7 @@ static int prog_load_cnt(int verdict, int val)
size_t insns_cnt = sizeof(prog) / sizeof(struct bpf_insn);
int ret;
- ret = bpf_load_program(BPF_PROG_TYPE_CGROUP_SKB,
+ ret = bpf_test_load_program(BPF_PROG_TYPE_CGROUP_SKB,
prog, insns_cnt, "GPL", 0,
bpf_log_buf, BPF_LOG_BUF_SIZE);
diff --git a/tools/testing/selftests/bpf/prog_tests/cgroup_attach_override.c b/tools/testing/selftests/bpf/prog_tests/cgroup_attach_override.c
index 10d3c33..356547e 100644
--- a/tools/testing/selftests/bpf/prog_tests/cgroup_attach_override.c
+++ b/tools/testing/selftests/bpf/prog_tests/cgroup_attach_override.c
@@ -18,7 +18,7 @@ static int prog_load(int verdict)
};
size_t insns_cnt = sizeof(prog) / sizeof(struct bpf_insn);
- return bpf_load_program(BPF_PROG_TYPE_CGROUP_SKB,
+ return bpf_test_load_program(BPF_PROG_TYPE_CGROUP_SKB,
prog, insns_cnt, "GPL", 0,
bpf_log_buf, BPF_LOG_BUF_SIZE);
}
diff --git a/tools/testing/selftests/bpf/prog_tests/connect_force_port.c b/tools/testing/selftests/bpf/prog_tests/connect_force_port.c
index 9229db2..9c4325f 100644
--- a/tools/testing/selftests/bpf/prog_tests/connect_force_port.c
+++ b/tools/testing/selftests/bpf/prog_tests/connect_force_port.c
@@ -51,24 +51,25 @@ static int run_test(int cgroup_fd, int server_fd, int family, int type)
bool v4 = family == AF_INET;
__u16 expected_local_port = v4 ? 22222 : 22223;
__u16 expected_peer_port = 60000;
- struct bpf_prog_load_attr attr = {
- .file = v4 ? "./connect_force_port4.o" :
- "./connect_force_port6.o",
- };
struct bpf_program *prog;
struct bpf_object *obj;
- int xlate_fd, fd, err;
+ const char *obj_file = v4 ? "connect_force_port4.o" : "connect_force_port6.o";
+ int fd, err;
__u32 duration = 0;
- err = bpf_prog_load_xattr(&attr, &obj, &xlate_fd);
- if (err) {
- log_err("Failed to load BPF object");
+ obj = bpf_object__open_file(obj_file, NULL);
+ if (!ASSERT_OK_PTR(obj, "bpf_obj_open"))
return -1;
+
+ err = bpf_object__load(obj);
+ if (!ASSERT_OK(err, "bpf_obj_load")) {
+ err = -EIO;
+ goto close_bpf_object;
}
- prog = bpf_object__find_program_by_title(obj, v4 ?
- "cgroup/connect4" :
- "cgroup/connect6");
+ prog = bpf_object__find_program_by_name(obj, v4 ?
+ "connect4" :
+ "connect6");
if (CHECK(!prog, "find_prog", "connect prog not found\n")) {
err = -EIO;
goto close_bpf_object;
@@ -82,9 +83,9 @@ static int run_test(int cgroup_fd, int server_fd, int family, int type)
goto close_bpf_object;
}
- prog = bpf_object__find_program_by_title(obj, v4 ?
- "cgroup/getpeername4" :
- "cgroup/getpeername6");
+ prog = bpf_object__find_program_by_name(obj, v4 ?
+ "getpeername4" :
+ "getpeername6");
if (CHECK(!prog, "find_prog", "getpeername prog not found\n")) {
err = -EIO;
goto close_bpf_object;
@@ -98,9 +99,9 @@ static int run_test(int cgroup_fd, int server_fd, int family, int type)
goto close_bpf_object;
}
- prog = bpf_object__find_program_by_title(obj, v4 ?
- "cgroup/getsockname4" :
- "cgroup/getsockname6");
+ prog = bpf_object__find_program_by_name(obj, v4 ?
+ "getsockname4" :
+ "getsockname6");
if (CHECK(!prog, "find_prog", "getsockname prog not found\n")) {
err = -EIO;
goto close_bpf_object;
diff --git a/tools/testing/selftests/bpf/prog_tests/core_kern.c b/tools/testing/selftests/bpf/prog_tests/core_kern.c
new file mode 100644
index 0000000..561c518
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/core_kern.c
@@ -0,0 +1,14 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2021 Facebook */
+
+#include "test_progs.h"
+#include "core_kern.lskel.h"
+
+void test_core_kern_lskel(void)
+{
+ struct core_kern_lskel *skel;
+
+ skel = core_kern_lskel__open_and_load();
+ ASSERT_OK_PTR(skel, "open_and_load");
+ core_kern_lskel__destroy(skel);
+}
diff --git a/tools/testing/selftests/bpf/prog_tests/core_reloc.c b/tools/testing/selftests/bpf/prog_tests/core_reloc.c
index 55ec85b..b8bdd1c 100644
--- a/tools/testing/selftests/bpf/prog_tests/core_reloc.c
+++ b/tools/testing/selftests/bpf/prog_tests/core_reloc.c
@@ -10,7 +10,7 @@ static int duration = 0;
#define STRUCT_TO_CHAR_PTR(struct_name) (const char *)&(struct struct_name)
-#define MODULES_CASE(name, sec_name, tp_name) { \
+#define MODULES_CASE(name, pg_name, tp_name) { \
.case_name = name, \
.bpf_obj_file = "test_core_reloc_module.o", \
.btf_src_file = NULL, /* find in kernel module BTFs */ \
@@ -28,7 +28,7 @@ static int duration = 0;
.comm_len = sizeof("test_progs"), \
}, \
.output_len = sizeof(struct core_reloc_module_output), \
- .prog_sec_name = sec_name, \
+ .prog_name = pg_name, \
.raw_tp_name = tp_name, \
.trigger = __trigger_module_test_read, \
.needs_testmod = true, \
@@ -43,7 +43,9 @@ static int duration = 0;
#define FLAVORS_CASE_COMMON(name) \
.case_name = #name, \
.bpf_obj_file = "test_core_reloc_flavors.o", \
- .btf_src_file = "btf__core_reloc_" #name ".o" \
+ .btf_src_file = "btf__core_reloc_" #name ".o", \
+ .raw_tp_name = "sys_enter", \
+ .prog_name = "test_core_flavors" \
#define FLAVORS_CASE(name) { \
FLAVORS_CASE_COMMON(name), \
@@ -66,7 +68,9 @@ static int duration = 0;
#define NESTING_CASE_COMMON(name) \
.case_name = #name, \
.bpf_obj_file = "test_core_reloc_nesting.o", \
- .btf_src_file = "btf__core_reloc_" #name ".o"
+ .btf_src_file = "btf__core_reloc_" #name ".o", \
+ .raw_tp_name = "sys_enter", \
+ .prog_name = "test_core_nesting" \
#define NESTING_CASE(name) { \
NESTING_CASE_COMMON(name), \
@@ -91,7 +95,9 @@ static int duration = 0;
#define ARRAYS_CASE_COMMON(name) \
.case_name = #name, \
.bpf_obj_file = "test_core_reloc_arrays.o", \
- .btf_src_file = "btf__core_reloc_" #name ".o"
+ .btf_src_file = "btf__core_reloc_" #name ".o", \
+ .raw_tp_name = "sys_enter", \
+ .prog_name = "test_core_arrays" \
#define ARRAYS_CASE(name) { \
ARRAYS_CASE_COMMON(name), \
@@ -123,7 +129,9 @@ static int duration = 0;
#define PRIMITIVES_CASE_COMMON(name) \
.case_name = #name, \
.bpf_obj_file = "test_core_reloc_primitives.o", \
- .btf_src_file = "btf__core_reloc_" #name ".o"
+ .btf_src_file = "btf__core_reloc_" #name ".o", \
+ .raw_tp_name = "sys_enter", \
+ .prog_name = "test_core_primitives" \
#define PRIMITIVES_CASE(name) { \
PRIMITIVES_CASE_COMMON(name), \
@@ -158,6 +166,8 @@ static int duration = 0;
.e = 5, .f = 6, .g = 7, .h = 8, \
}, \
.output_len = sizeof(struct core_reloc_mods_output), \
+ .raw_tp_name = "sys_enter", \
+ .prog_name = "test_core_mods", \
}
#define PTR_AS_ARR_CASE(name) { \
@@ -174,6 +184,8 @@ static int duration = 0;
.a = 3, \
}, \
.output_len = sizeof(struct core_reloc_ptr_as_arr), \
+ .raw_tp_name = "sys_enter", \
+ .prog_name = "test_core_ptr_as_arr", \
}
#define INTS_DATA(struct_name) STRUCT_TO_CHAR_PTR(struct_name) { \
@@ -190,7 +202,9 @@ static int duration = 0;
#define INTS_CASE_COMMON(name) \
.case_name = #name, \
.bpf_obj_file = "test_core_reloc_ints.o", \
- .btf_src_file = "btf__core_reloc_" #name ".o"
+ .btf_src_file = "btf__core_reloc_" #name ".o", \
+ .raw_tp_name = "sys_enter", \
+ .prog_name = "test_core_ints"
#define INTS_CASE(name) { \
INTS_CASE_COMMON(name), \
@@ -208,7 +222,9 @@ static int duration = 0;
#define FIELD_EXISTS_CASE_COMMON(name) \
.case_name = #name, \
.bpf_obj_file = "test_core_reloc_existence.o", \
- .btf_src_file = "btf__core_reloc_" #name ".o" \
+ .btf_src_file = "btf__core_reloc_" #name ".o", \
+ .raw_tp_name = "sys_enter", \
+ .prog_name = "test_core_existence"
#define BITFIELDS_CASE_COMMON(objfile, test_name_prefix, name) \
.case_name = test_name_prefix#name, \
@@ -223,6 +239,8 @@ static int duration = 0;
.output = STRUCT_TO_CHAR_PTR(core_reloc_bitfields_output) \
__VA_ARGS__, \
.output_len = sizeof(struct core_reloc_bitfields_output), \
+ .raw_tp_name = "sys_enter", \
+ .prog_name = "test_core_bitfields", \
}, { \
BITFIELDS_CASE_COMMON("test_core_reloc_bitfields_direct.o", \
"direct:", name), \
@@ -231,7 +249,7 @@ static int duration = 0;
.output = STRUCT_TO_CHAR_PTR(core_reloc_bitfields_output) \
__VA_ARGS__, \
.output_len = sizeof(struct core_reloc_bitfields_output), \
- .prog_sec_name = "tp_btf/sys_enter", \
+ .prog_name = "test_core_bitfields_direct", \
}
@@ -239,17 +257,21 @@ static int duration = 0;
BITFIELDS_CASE_COMMON("test_core_reloc_bitfields_probed.o", \
"probed:", name), \
.fails = true, \
+ .raw_tp_name = "sys_enter", \
+ .prog_name = "test_core_bitfields", \
}, { \
BITFIELDS_CASE_COMMON("test_core_reloc_bitfields_direct.o", \
"direct:", name), \
- .prog_sec_name = "tp_btf/sys_enter", \
.fails = true, \
+ .prog_name = "test_core_bitfields_direct", \
}
#define SIZE_CASE_COMMON(name) \
.case_name = #name, \
.bpf_obj_file = "test_core_reloc_size.o", \
- .btf_src_file = "btf__core_reloc_" #name ".o"
+ .btf_src_file = "btf__core_reloc_" #name ".o", \
+ .raw_tp_name = "sys_enter", \
+ .prog_name = "test_core_size"
#define SIZE_OUTPUT_DATA(type) \
STRUCT_TO_CHAR_PTR(core_reloc_size_output) { \
@@ -277,8 +299,10 @@ static int duration = 0;
#define TYPE_BASED_CASE_COMMON(name) \
.case_name = #name, \
- .bpf_obj_file = "test_core_reloc_type_based.o", \
- .btf_src_file = "btf__core_reloc_" #name ".o" \
+ .bpf_obj_file = "test_core_reloc_type_based.o", \
+ .btf_src_file = "btf__core_reloc_" #name ".o", \
+ .raw_tp_name = "sys_enter", \
+ .prog_name = "test_core_type_based"
#define TYPE_BASED_CASE(name, ...) { \
TYPE_BASED_CASE_COMMON(name), \
@@ -295,7 +319,9 @@ static int duration = 0;
#define TYPE_ID_CASE_COMMON(name) \
.case_name = #name, \
.bpf_obj_file = "test_core_reloc_type_id.o", \
- .btf_src_file = "btf__core_reloc_" #name ".o" \
+ .btf_src_file = "btf__core_reloc_" #name ".o", \
+ .raw_tp_name = "sys_enter", \
+ .prog_name = "test_core_type_id"
#define TYPE_ID_CASE(name, setup_fn) { \
TYPE_ID_CASE_COMMON(name), \
@@ -312,7 +338,9 @@ static int duration = 0;
#define ENUMVAL_CASE_COMMON(name) \
.case_name = #name, \
.bpf_obj_file = "test_core_reloc_enumval.o", \
- .btf_src_file = "btf__core_reloc_" #name ".o" \
+ .btf_src_file = "btf__core_reloc_" #name ".o", \
+ .raw_tp_name = "sys_enter", \
+ .prog_name = "test_core_enumval"
#define ENUMVAL_CASE(name, ...) { \
ENUMVAL_CASE_COMMON(name), \
@@ -342,7 +370,7 @@ struct core_reloc_test_case {
bool fails;
bool needs_testmod;
bool relaxed_core_relocs;
- const char *prog_sec_name;
+ const char *prog_name;
const char *raw_tp_name;
setup_test_fn setup;
trigger_test_fn trigger;
@@ -433,7 +461,7 @@ static int setup_type_id_case_local(struct core_reloc_test_case *test)
static int setup_type_id_case_success(struct core_reloc_test_case *test) {
struct core_reloc_type_id_output *exp = (void *)test->output;
- struct btf *targ_btf = btf__parse(test->btf_src_file, NULL);
+ struct btf *targ_btf;
int err;
err = setup_type_id_case_local(test);
@@ -497,11 +525,13 @@ static struct core_reloc_test_case test_cases[] = {
.comm_len = sizeof("test_progs"),
},
.output_len = sizeof(struct core_reloc_kernel_output),
+ .raw_tp_name = "sys_enter",
+ .prog_name = "test_core_kernel",
},
/* validate we can find kernel module BTF types for relocs/attach */
- MODULES_CASE("module_probed", "raw_tp/bpf_testmod_test_read", "bpf_testmod_test_read"),
- MODULES_CASE("module_direct", "tp_btf/bpf_testmod_test_read", NULL),
+ MODULES_CASE("module_probed", "test_core_module_probed", "bpf_testmod_test_read"),
+ MODULES_CASE("module_direct", "test_core_module_direct", NULL),
/* validate BPF program can use multiple flavors to match against
* single target BTF type
@@ -580,6 +610,8 @@ static struct core_reloc_test_case test_cases[] = {
.c = 0, /* BUG in clang, should be 3 */
},
.output_len = sizeof(struct core_reloc_misc_output),
+ .raw_tp_name = "sys_enter",
+ .prog_name = "test_core_misc",
},
/* validate field existence checks */
@@ -848,14 +880,9 @@ void test_core_reloc(void)
if (!ASSERT_OK_PTR(obj, "obj_open"))
goto cleanup;
- probe_name = "raw_tracepoint/sys_enter";
- tp_name = "sys_enter";
- if (test_case->prog_sec_name) {
- probe_name = test_case->prog_sec_name;
- tp_name = test_case->raw_tp_name; /* NULL for tp_btf */
- }
-
- prog = bpf_object__find_program_by_title(obj, probe_name);
+ probe_name = test_case->prog_name;
+ tp_name = test_case->raw_tp_name; /* NULL for tp_btf */
+ prog = bpf_object__find_program_by_name(obj, probe_name);
if (CHECK(!prog, "find_probe",
"prog '%s' not found\n", probe_name))
goto cleanup;
@@ -881,7 +908,8 @@ void test_core_reloc(void)
data = mmap_data;
memset(mmap_data, 0, sizeof(*data));
- memcpy(data->in, test_case->input, test_case->input_len);
+ if (test_case->input_len)
+ memcpy(data->in, test_case->input, test_case->input_len);
data->my_pid_tgid = my_pid_tgid;
link = bpf_program__attach_raw_tracepoint(prog, tp_name);
diff --git a/tools/testing/selftests/bpf/prog_tests/d_path.c b/tools/testing/selftests/bpf/prog_tests/d_path.c
index 0a577a2..32fc5b3 100644
--- a/tools/testing/selftests/bpf/prog_tests/d_path.c
+++ b/tools/testing/selftests/bpf/prog_tests/d_path.c
@@ -9,6 +9,7 @@
#define MAX_FILES 7
#include "test_d_path.skel.h"
+#include "test_d_path_check_rdonly_mem.skel.h"
static int duration;
@@ -99,7 +100,7 @@ static int trigger_fstat_events(pid_t pid)
return ret;
}
-void test_d_path(void)
+static void test_d_path_basic(void)
{
struct test_d_path__bss *bss;
struct test_d_path *skel;
@@ -155,3 +156,22 @@ void test_d_path(void)
cleanup:
test_d_path__destroy(skel);
}
+
+static void test_d_path_check_rdonly_mem(void)
+{
+ struct test_d_path_check_rdonly_mem *skel;
+
+ skel = test_d_path_check_rdonly_mem__open_and_load();
+ ASSERT_ERR_PTR(skel, "unexpected_load_overwriting_rdonly_mem");
+
+ test_d_path_check_rdonly_mem__destroy(skel);
+}
+
+void test_d_path(void)
+{
+ if (test__start_subtest("basic"))
+ test_d_path_basic();
+
+ if (test__start_subtest("check_rdonly_mem"))
+ test_d_path_check_rdonly_mem();
+}
diff --git a/tools/testing/selftests/bpf/prog_tests/exhandler.c b/tools/testing/selftests/bpf/prog_tests/exhandler.c
new file mode 100644
index 0000000..118bb18
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/exhandler.c
@@ -0,0 +1,43 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2021, Oracle and/or its affiliates. */
+
+#include <test_progs.h>
+
+/* Test that verifies exception handling is working. fork()
+ * triggers task_newtask tracepoint; that new task will have a
+ * NULL pointer task_works, and the associated task->task_works->func
+ * should not be NULL if task_works itself is non-NULL.
+ *
+ * So to verify exception handling we want to see a NULL task_works
+ * and task_works->func; if we see this we can conclude that the
+ * exception handler ran when we attempted to dereference task->task_works
+ * and zeroed the destination register.
+ */
+#include "exhandler_kern.skel.h"
+
+void test_exhandler(void)
+{
+ int err = 0, duration = 0, status;
+ struct exhandler_kern *skel;
+ pid_t cpid;
+
+ skel = exhandler_kern__open_and_load();
+ if (CHECK(!skel, "skel_load", "skeleton failed: %d\n", err))
+ goto cleanup;
+
+ skel->bss->test_pid = getpid();
+
+ err = exhandler_kern__attach(skel);
+ if (!ASSERT_OK(err, "attach"))
+ goto cleanup;
+ cpid = fork();
+ if (!ASSERT_GT(cpid, -1, "fork failed"))
+ goto cleanup;
+ if (cpid == 0)
+ _exit(0);
+ waitpid(cpid, &status, 0);
+
+ ASSERT_NEQ(skel->bss->exception_triggered, 0, "verify exceptions occurred");
+cleanup:
+ exhandler_kern__destroy(skel);
+}
diff --git a/tools/testing/selftests/bpf/prog_tests/fexit_bpf2bpf.c b/tools/testing/selftests/bpf/prog_tests/fexit_bpf2bpf.c
index 9cff14a..c52f99f 100644
--- a/tools/testing/selftests/bpf/prog_tests/fexit_bpf2bpf.c
+++ b/tools/testing/selftests/bpf/prog_tests/fexit_bpf2bpf.c
@@ -65,7 +65,7 @@ static void test_fexit_bpf2bpf_common(const char *obj_file,
int err, tgt_fd, i;
struct btf *btf;
- err = bpf_prog_load(target_obj_file, BPF_PROG_TYPE_UNSPEC,
+ err = bpf_prog_test_load(target_obj_file, BPF_PROG_TYPE_UNSPEC,
&tgt_obj, &tgt_fd);
if (!ASSERT_OK(err, "tgt_prog_load"))
return;
@@ -101,6 +101,8 @@ static void test_fexit_bpf2bpf_common(const char *obj_file,
for (i = 0; i < prog_cnt; i++) {
struct bpf_link_info link_info;
+ struct bpf_program *pos;
+ const char *pos_sec_name;
char *tgt_name;
__s32 btf_id;
@@ -109,7 +111,14 @@ static void test_fexit_bpf2bpf_common(const char *obj_file,
goto close_prog;
btf_id = btf__find_by_name_kind(btf, tgt_name + 1, BTF_KIND_FUNC);
- prog[i] = bpf_object__find_program_by_title(obj, prog_name[i]);
+ prog[i] = NULL;
+ bpf_object__for_each_program(pos, obj) {
+ pos_sec_name = bpf_program__section_name(pos);
+ if (pos_sec_name && !strcmp(pos_sec_name, prog_name[i])) {
+ prog[i] = pos;
+ break;
+ }
+ }
if (!ASSERT_OK_PTR(prog[i], prog_name[i]))
goto close_prog;
@@ -211,8 +220,8 @@ static void test_func_replace_verify(void)
static int test_second_attach(struct bpf_object *obj)
{
- const char *prog_name = "freplace/get_constant";
- const char *tgt_name = prog_name + 9; /* cut off freplace/ */
+ const char *prog_name = "security_new_get_constant";
+ const char *tgt_name = "get_constant";
const char *tgt_obj_file = "./test_pkt_access.o";
struct bpf_program *prog = NULL;
struct bpf_object *tgt_obj;
@@ -220,11 +229,11 @@ static int test_second_attach(struct bpf_object *obj)
struct bpf_link *link;
int err = 0, tgt_fd;
- prog = bpf_object__find_program_by_title(obj, prog_name);
+ prog = bpf_object__find_program_by_name(obj, prog_name);
if (CHECK(!prog, "find_prog", "prog %s not found\n", prog_name))
return -ENOENT;
- err = bpf_prog_load(tgt_obj_file, BPF_PROG_TYPE_UNSPEC,
+ err = bpf_prog_test_load(tgt_obj_file, BPF_PROG_TYPE_UNSPEC,
&tgt_obj, &tgt_fd);
if (CHECK(err, "second_prog_load", "file %s err %d errno %d\n",
tgt_obj_file, err, errno))
@@ -274,7 +283,7 @@ static void test_fmod_ret_freplace(void)
__u32 duration = 0;
int err, pkt_fd, attach_prog_fd;
- err = bpf_prog_load(tgt_name, BPF_PROG_TYPE_UNSPEC,
+ err = bpf_prog_test_load(tgt_name, BPF_PROG_TYPE_UNSPEC,
&pkt_obj, &pkt_fd);
/* the target prog should load fine */
if (CHECK(err, "tgt_prog_load", "file %s err %d errno %d\n",
@@ -341,7 +350,7 @@ static void test_obj_load_failure_common(const char *obj_file,
int err, pkt_fd;
__u32 duration = 0;
- err = bpf_prog_load(target_obj_file, BPF_PROG_TYPE_UNSPEC,
+ err = bpf_prog_test_load(target_obj_file, BPF_PROG_TYPE_UNSPEC,
&pkt_obj, &pkt_fd);
/* the target prog should load fine */
if (CHECK(err, "tgt_prog_load", "file %s err %d errno %d\n",
diff --git a/tools/testing/selftests/bpf/prog_tests/fexit_stress.c b/tools/testing/selftests/bpf/prog_tests/fexit_stress.c
index 7c9b62e..e4cede6 100644
--- a/tools/testing/selftests/bpf/prog_tests/fexit_stress.c
+++ b/tools/testing/selftests/bpf/prog_tests/fexit_stress.c
@@ -20,34 +20,33 @@ void test_fexit_stress(void)
BPF_EXIT_INSN(),
};
- struct bpf_load_program_attr load_attr = {
- .prog_type = BPF_PROG_TYPE_TRACING,
- .license = "GPL",
- .insns = trace_program,
- .insns_cnt = sizeof(trace_program) / sizeof(struct bpf_insn),
+ LIBBPF_OPTS(bpf_prog_load_opts, trace_opts,
.expected_attach_type = BPF_TRACE_FEXIT,
- };
+ .log_buf = error,
+ .log_size = sizeof(error),
+ );
const struct bpf_insn skb_program[] = {
BPF_MOV64_IMM(BPF_REG_0, 0),
BPF_EXIT_INSN(),
};
- struct bpf_load_program_attr skb_load_attr = {
- .prog_type = BPF_PROG_TYPE_SOCKET_FILTER,
- .license = "GPL",
- .insns = skb_program,
- .insns_cnt = sizeof(skb_program) / sizeof(struct bpf_insn),
- };
+ LIBBPF_OPTS(bpf_prog_load_opts, skb_opts,
+ .log_buf = error,
+ .log_size = sizeof(error),
+ );
err = libbpf_find_vmlinux_btf_id("bpf_fentry_test1",
- load_attr.expected_attach_type);
+ trace_opts.expected_attach_type);
if (CHECK(err <= 0, "find_vmlinux_btf_id", "failed: %d\n", err))
goto out;
- load_attr.attach_btf_id = err;
+ trace_opts.attach_btf_id = err;
for (i = 0; i < CNT; i++) {
- fexit_fd[i] = bpf_load_program_xattr(&load_attr, error, sizeof(error));
+ fexit_fd[i] = bpf_prog_load(BPF_PROG_TYPE_TRACING, NULL, "GPL",
+ trace_program,
+ sizeof(trace_program) / sizeof(struct bpf_insn),
+ &trace_opts);
if (CHECK(fexit_fd[i] < 0, "fexit loaded",
"failed: %d errno %d\n", fexit_fd[i], errno))
goto out;
@@ -57,7 +56,9 @@ void test_fexit_stress(void)
goto out;
}
- filter_fd = bpf_load_program_xattr(&skb_load_attr, error, sizeof(error));
+ filter_fd = bpf_prog_load(BPF_PROG_TYPE_SOCKET_FILTER, NULL, "GPL",
+ skb_program, sizeof(skb_program) / sizeof(struct bpf_insn),
+ &skb_opts);
if (CHECK(filter_fd < 0, "test_program_loaded", "failed: %d errno %d\n",
filter_fd, errno))
goto out;
diff --git a/tools/testing/selftests/bpf/prog_tests/find_vma.c b/tools/testing/selftests/bpf/prog_tests/find_vma.c
new file mode 100644
index 0000000..b74b3c0
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/find_vma.c
@@ -0,0 +1,117 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2021 Facebook */
+#include <test_progs.h>
+#include <sys/types.h>
+#include <unistd.h>
+#include "find_vma.skel.h"
+#include "find_vma_fail1.skel.h"
+#include "find_vma_fail2.skel.h"
+
+static void test_and_reset_skel(struct find_vma *skel, int expected_find_zero_ret)
+{
+ ASSERT_EQ(skel->bss->found_vm_exec, 1, "found_vm_exec");
+ ASSERT_EQ(skel->data->find_addr_ret, 0, "find_addr_ret");
+ ASSERT_EQ(skel->data->find_zero_ret, expected_find_zero_ret, "find_zero_ret");
+ ASSERT_OK_PTR(strstr(skel->bss->d_iname, "test_progs"), "find_test_progs");
+
+ skel->bss->found_vm_exec = 0;
+ skel->data->find_addr_ret = -1;
+ skel->data->find_zero_ret = -1;
+ skel->bss->d_iname[0] = 0;
+}
+
+static int open_pe(void)
+{
+ struct perf_event_attr attr = {0};
+ int pfd;
+
+ /* create perf event */
+ attr.size = sizeof(attr);
+ attr.type = PERF_TYPE_HARDWARE;
+ attr.config = PERF_COUNT_HW_CPU_CYCLES;
+ attr.freq = 1;
+ attr.sample_freq = 4000;
+ pfd = syscall(__NR_perf_event_open, &attr, 0, -1, -1, PERF_FLAG_FD_CLOEXEC);
+
+ return pfd >= 0 ? pfd : -errno;
+}
+
+static void test_find_vma_pe(struct find_vma *skel)
+{
+ struct bpf_link *link = NULL;
+ volatile int j = 0;
+ int pfd, i;
+
+ pfd = open_pe();
+ if (pfd < 0) {
+ if (pfd == -ENOENT || pfd == -EOPNOTSUPP) {
+ printf("%s:SKIP:no PERF_COUNT_HW_CPU_CYCLES\n", __func__);
+ test__skip();
+ goto cleanup;
+ }
+ if (!ASSERT_GE(pfd, 0, "perf_event_open"))
+ goto cleanup;
+ }
+
+ link = bpf_program__attach_perf_event(skel->progs.handle_pe, pfd);
+ if (!ASSERT_OK_PTR(link, "attach_perf_event"))
+ goto cleanup;
+
+ for (i = 0; i < 1000000; ++i)
+ ++j;
+
+ test_and_reset_skel(skel, -EBUSY /* in nmi, irq_work is busy */);
+cleanup:
+ bpf_link__destroy(link);
+ close(pfd);
+}
+
+static void test_find_vma_kprobe(struct find_vma *skel)
+{
+ int err;
+
+ err = find_vma__attach(skel);
+ if (!ASSERT_OK(err, "get_branch_snapshot__attach"))
+ return;
+
+ getpgid(skel->bss->target_pid);
+ test_and_reset_skel(skel, -ENOENT /* could not find vma for ptr 0 */);
+}
+
+static void test_illegal_write_vma(void)
+{
+ struct find_vma_fail1 *skel;
+
+ skel = find_vma_fail1__open_and_load();
+ if (!ASSERT_ERR_PTR(skel, "find_vma_fail1__open_and_load"))
+ find_vma_fail1__destroy(skel);
+}
+
+static void test_illegal_write_task(void)
+{
+ struct find_vma_fail2 *skel;
+
+ skel = find_vma_fail2__open_and_load();
+ if (!ASSERT_ERR_PTR(skel, "find_vma_fail2__open_and_load"))
+ find_vma_fail2__destroy(skel);
+}
+
+void serial_test_find_vma(void)
+{
+ struct find_vma *skel;
+
+ skel = find_vma__open_and_load();
+ if (!ASSERT_OK_PTR(skel, "find_vma__open_and_load"))
+ return;
+
+ skel->bss->target_pid = getpid();
+ skel->bss->addr = (__u64)(uintptr_t)test_find_vma_pe;
+
+ test_find_vma_pe(skel);
+ usleep(100000); /* allow the irq_work to finish */
+ test_find_vma_kprobe(skel);
+
+ find_vma__destroy(skel);
+ test_illegal_write_vma();
+ test_illegal_write_task();
+}
diff --git a/tools/testing/selftests/bpf/prog_tests/flow_dissector_load_bytes.c b/tools/testing/selftests/bpf/prog_tests/flow_dissector_load_bytes.c
index 6093728..93ac3f2 100644
--- a/tools/testing/selftests/bpf/prog_tests/flow_dissector_load_bytes.c
+++ b/tools/testing/selftests/bpf/prog_tests/flow_dissector_load_bytes.c
@@ -30,7 +30,7 @@ void serial_test_flow_dissector_load_bytes(void)
/* make sure bpf_skb_load_bytes is not allowed from skb-less context
*/
- fd = bpf_load_program(BPF_PROG_TYPE_FLOW_DISSECTOR, prog,
+ fd = bpf_test_load_program(BPF_PROG_TYPE_FLOW_DISSECTOR, prog,
ARRAY_SIZE(prog), "GPL", 0, NULL, 0);
CHECK(fd < 0,
"flow_dissector-bpf_skb_load_bytes-load",
diff --git a/tools/testing/selftests/bpf/prog_tests/flow_dissector_reattach.c b/tools/testing/selftests/bpf/prog_tests/flow_dissector_reattach.c
index f0c6c22..7c79462 100644
--- a/tools/testing/selftests/bpf/prog_tests/flow_dissector_reattach.c
+++ b/tools/testing/selftests/bpf/prog_tests/flow_dissector_reattach.c
@@ -47,9 +47,9 @@ static int load_prog(enum bpf_prog_type type)
};
int fd;
- fd = bpf_load_program(type, prog, ARRAY_SIZE(prog), "GPL", 0, NULL, 0);
+ fd = bpf_test_load_program(type, prog, ARRAY_SIZE(prog), "GPL", 0, NULL, 0);
if (CHECK_FAIL(fd < 0))
- perror("bpf_load_program");
+ perror("bpf_test_load_program");
return fd;
}
diff --git a/tools/testing/selftests/bpf/prog_tests/get_func_args_test.c b/tools/testing/selftests/bpf/prog_tests/get_func_args_test.c
new file mode 100644
index 0000000..85c4271
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/get_func_args_test.c
@@ -0,0 +1,44 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <test_progs.h>
+#include "get_func_args_test.skel.h"
+
+void test_get_func_args_test(void)
+{
+ struct get_func_args_test *skel = NULL;
+ __u32 duration = 0, retval;
+ int err, prog_fd;
+
+ skel = get_func_args_test__open_and_load();
+ if (!ASSERT_OK_PTR(skel, "get_func_args_test__open_and_load"))
+ return;
+
+ err = get_func_args_test__attach(skel);
+ if (!ASSERT_OK(err, "get_func_args_test__attach"))
+ goto cleanup;
+
+ /* This runs bpf_fentry_test* functions and triggers
+ * fentry/fexit programs.
+ */
+ prog_fd = bpf_program__fd(skel->progs.test1);
+ err = bpf_prog_test_run(prog_fd, 1, NULL, 0,
+ NULL, NULL, &retval, &duration);
+ ASSERT_OK(err, "test_run");
+ ASSERT_EQ(retval, 0, "test_run");
+
+ /* This runs bpf_modify_return_test function and triggers
+ * fmod_ret_test and fexit_test programs.
+ */
+ prog_fd = bpf_program__fd(skel->progs.fmod_ret_test);
+ err = bpf_prog_test_run(prog_fd, 1, NULL, 0,
+ NULL, NULL, &retval, &duration);
+ ASSERT_OK(err, "test_run");
+ ASSERT_EQ(retval, 1234, "test_run");
+
+ ASSERT_EQ(skel->bss->test1_result, 1, "test1_result");
+ ASSERT_EQ(skel->bss->test2_result, 1, "test2_result");
+ ASSERT_EQ(skel->bss->test3_result, 1, "test3_result");
+ ASSERT_EQ(skel->bss->test4_result, 1, "test4_result");
+
+cleanup:
+ get_func_args_test__destroy(skel);
+}
diff --git a/tools/testing/selftests/bpf/prog_tests/get_stack_raw_tp.c b/tools/testing/selftests/bpf/prog_tests/get_stack_raw_tp.c
index 522237a..e834a01 100644
--- a/tools/testing/selftests/bpf/prog_tests/get_stack_raw_tp.c
+++ b/tools/testing/selftests/bpf/prog_tests/get_stack_raw_tp.c
@@ -24,13 +24,19 @@ static void get_stack_print_output(void *ctx, int cpu, void *data, __u32 size)
{
bool good_kern_stack = false, good_user_stack = false;
const char *nonjit_func = "___bpf_prog_run";
- struct get_stack_trace_t *e = data;
+ /* perfbuf-submitted data is 4-byte aligned, but we need 8-byte
+ * alignment, so copy data into a local variable, for simplicity
+ */
+ struct get_stack_trace_t e;
int i, num_stack;
static __u64 cnt;
struct ksym *ks;
cnt++;
+ memset(&e, 0, sizeof(e));
+ memcpy(&e, data, size <= sizeof(e) ? size : sizeof(e));
+
if (size < sizeof(struct get_stack_trace_t)) {
__u64 *raw_data = data;
bool found = false;
@@ -57,19 +63,19 @@ static void get_stack_print_output(void *ctx, int cpu, void *data, __u32 size)
good_user_stack = true;
}
} else {
- num_stack = e->kern_stack_size / sizeof(__u64);
+ num_stack = e.kern_stack_size / sizeof(__u64);
if (env.jit_enabled) {
good_kern_stack = num_stack > 0;
} else {
for (i = 0; i < num_stack; i++) {
- ks = ksym_search(e->kern_stack[i]);
+ ks = ksym_search(e.kern_stack[i]);
if (ks && (strcmp(ks->name, nonjit_func) == 0)) {
good_kern_stack = true;
break;
}
}
}
- if (e->user_stack_size > 0 && e->user_stack_buildid_size > 0)
+ if (e.user_stack_size > 0 && e.user_stack_buildid_size > 0)
good_user_stack = true;
}
@@ -83,9 +89,8 @@ void test_get_stack_raw_tp(void)
{
const char *file = "./test_get_stack_rawtp.o";
const char *file_err = "./test_get_stack_rawtp_err.o";
- const char *prog_name = "raw_tracepoint/sys_enter";
+ const char *prog_name = "bpf_prog1";
int i, err, prog_fd, exp_cnt = MAX_CNT_RAWTP;
- struct perf_buffer_opts pb_opts = {};
struct perf_buffer *pb = NULL;
struct bpf_link *link = NULL;
struct timespec tv = {0, 10};
@@ -94,15 +99,15 @@ void test_get_stack_raw_tp(void)
struct bpf_map *map;
cpu_set_t cpu_set;
- err = bpf_prog_load(file_err, BPF_PROG_TYPE_RAW_TRACEPOINT, &obj, &prog_fd);
+ err = bpf_prog_test_load(file_err, BPF_PROG_TYPE_RAW_TRACEPOINT, &obj, &prog_fd);
if (CHECK(err >= 0, "prog_load raw tp", "err %d errno %d\n", err, errno))
return;
- err = bpf_prog_load(file, BPF_PROG_TYPE_RAW_TRACEPOINT, &obj, &prog_fd);
+ err = bpf_prog_test_load(file, BPF_PROG_TYPE_RAW_TRACEPOINT, &obj, &prog_fd);
if (CHECK(err, "prog_load raw tp", "err %d errno %d\n", err, errno))
return;
- prog = bpf_object__find_program_by_title(obj, prog_name);
+ prog = bpf_object__find_program_by_name(obj, prog_name);
if (CHECK(!prog, "find_probe", "prog '%s' not found\n", prog_name))
goto close_prog;
@@ -124,8 +129,8 @@ void test_get_stack_raw_tp(void)
if (!ASSERT_OK_PTR(link, "attach_raw_tp"))
goto close_prog;
- pb_opts.sample_cb = get_stack_print_output;
- pb = perf_buffer__new(bpf_map__fd(map), 8, &pb_opts);
+ pb = perf_buffer__new(bpf_map__fd(map), 8, get_stack_print_output,
+ NULL, NULL, NULL);
if (!ASSERT_OK_PTR(pb, "perf_buf__new"))
goto close_prog;
diff --git a/tools/testing/selftests/bpf/prog_tests/global_data.c b/tools/testing/selftests/bpf/prog_tests/global_data.c
index afd8639..9da131b 100644
--- a/tools/testing/selftests/bpf/prog_tests/global_data.c
+++ b/tools/testing/selftests/bpf/prog_tests/global_data.c
@@ -136,7 +136,7 @@ void test_global_data(void)
struct bpf_object *obj;
int err, prog_fd;
- err = bpf_prog_load(file, BPF_PROG_TYPE_SCHED_CLS, &obj, &prog_fd);
+ err = bpf_prog_test_load(file, BPF_PROG_TYPE_SCHED_CLS, &obj, &prog_fd);
if (CHECK(err, "load program", "error %d loading %s\n", err, file))
return;
diff --git a/tools/testing/selftests/bpf/prog_tests/global_func_args.c b/tools/testing/selftests/bpf/prog_tests/global_func_args.c
index 8bcc286..93a2439 100644
--- a/tools/testing/selftests/bpf/prog_tests/global_func_args.c
+++ b/tools/testing/selftests/bpf/prog_tests/global_func_args.c
@@ -44,7 +44,7 @@ void test_global_func_args(void)
struct bpf_object *obj;
int err, prog_fd;
- err = bpf_prog_load(file, BPF_PROG_TYPE_CGROUP_SKB, &obj, &prog_fd);
+ err = bpf_prog_test_load(file, BPF_PROG_TYPE_CGROUP_SKB, &obj, &prog_fd);
if (CHECK(err, "load program", "error %d loading %s\n", err, file))
return;
diff --git a/tools/testing/selftests/bpf/prog_tests/kfree_skb.c b/tools/testing/selftests/bpf/prog_tests/kfree_skb.c
index 01e51d1..ce10d2f 100644
--- a/tools/testing/selftests/bpf/prog_tests/kfree_skb.c
+++ b/tools/testing/selftests/bpf/prog_tests/kfree_skb.c
@@ -1,6 +1,7 @@
// SPDX-License-Identifier: GPL-2.0
#include <test_progs.h>
#include <network_helpers.h>
+#include "kfree_skb.skel.h"
struct meta {
int ifindex;
@@ -58,63 +59,43 @@ void serial_test_kfree_skb(void)
.ctx_in = &skb,
.ctx_size_in = sizeof(skb),
};
- struct bpf_prog_load_attr attr = {
- .file = "./kfree_skb.o",
- };
-
- struct bpf_link *link = NULL, *link_fentry = NULL, *link_fexit = NULL;
- struct bpf_map *perf_buf_map, *global_data;
- struct bpf_program *prog, *fentry, *fexit;
- struct bpf_object *obj, *obj2 = NULL;
- struct perf_buffer_opts pb_opts = {};
+ struct kfree_skb *skel = NULL;
+ struct bpf_link *link;
+ struct bpf_object *obj;
struct perf_buffer *pb = NULL;
- int err, kfree_skb_fd;
+ int err;
bool passed = false;
__u32 duration = 0;
const int zero = 0;
bool test_ok[2];
- err = bpf_prog_load("./test_pkt_access.o", BPF_PROG_TYPE_SCHED_CLS,
+ err = bpf_prog_test_load("./test_pkt_access.o", BPF_PROG_TYPE_SCHED_CLS,
&obj, &tattr.prog_fd);
if (CHECK(err, "prog_load sched cls", "err %d errno %d\n", err, errno))
return;
- err = bpf_prog_load_xattr(&attr, &obj2, &kfree_skb_fd);
- if (CHECK(err, "prog_load raw tp", "err %d errno %d\n", err, errno))
+ skel = kfree_skb__open_and_load();
+ if (!ASSERT_OK_PTR(skel, "kfree_skb_skel"))
goto close_prog;
- prog = bpf_object__find_program_by_title(obj2, "tp_btf/kfree_skb");
- if (CHECK(!prog, "find_prog", "prog kfree_skb not found\n"))
- goto close_prog;
- fentry = bpf_object__find_program_by_title(obj2, "fentry/eth_type_trans");
- if (CHECK(!fentry, "find_prog", "prog eth_type_trans not found\n"))
- goto close_prog;
- fexit = bpf_object__find_program_by_title(obj2, "fexit/eth_type_trans");
- if (CHECK(!fexit, "find_prog", "prog eth_type_trans not found\n"))
- goto close_prog;
-
- global_data = bpf_object__find_map_by_name(obj2, ".bss");
- if (CHECK(!global_data, "find global data", "not found\n"))
- goto close_prog;
-
- link = bpf_program__attach_raw_tracepoint(prog, NULL);
+ link = bpf_program__attach_raw_tracepoint(skel->progs.trace_kfree_skb, NULL);
if (!ASSERT_OK_PTR(link, "attach_raw_tp"))
goto close_prog;
- link_fentry = bpf_program__attach_trace(fentry);
- if (!ASSERT_OK_PTR(link_fentry, "attach fentry"))
- goto close_prog;
- link_fexit = bpf_program__attach_trace(fexit);
- if (!ASSERT_OK_PTR(link_fexit, "attach fexit"))
- goto close_prog;
+ skel->links.trace_kfree_skb = link;
- perf_buf_map = bpf_object__find_map_by_name(obj2, "perf_buf_map");
- if (CHECK(!perf_buf_map, "find_perf_buf_map", "not found\n"))
+ link = bpf_program__attach_trace(skel->progs.fentry_eth_type_trans);
+ if (!ASSERT_OK_PTR(link, "attach fentry"))
goto close_prog;
+ skel->links.fentry_eth_type_trans = link;
+
+ link = bpf_program__attach_trace(skel->progs.fexit_eth_type_trans);
+ if (!ASSERT_OK_PTR(link, "attach fexit"))
+ goto close_prog;
+ skel->links.fexit_eth_type_trans = link;
/* set up perf buffer */
- pb_opts.sample_cb = on_sample;
- pb_opts.ctx = &passed;
- pb = perf_buffer__new(bpf_map__fd(perf_buf_map), 1, &pb_opts);
+ pb = perf_buffer__new(bpf_map__fd(skel->maps.perf_buf_map), 1,
+ on_sample, NULL, &passed, NULL);
if (!ASSERT_OK_PTR(pb, "perf_buf__new"))
goto close_prog;
@@ -135,7 +116,7 @@ void serial_test_kfree_skb(void)
*/
ASSERT_TRUE(passed, "passed");
- err = bpf_map_lookup_elem(bpf_map__fd(global_data), &zero, test_ok);
+ err = bpf_map_lookup_elem(bpf_map__fd(skel->maps.bss), &zero, test_ok);
if (CHECK(err, "get_result",
"failed to get output data: %d\n", err))
goto close_prog;
@@ -143,9 +124,6 @@ void serial_test_kfree_skb(void)
CHECK_FAIL(!test_ok[0] || !test_ok[1]);
close_prog:
perf_buffer__free(pb);
- bpf_link__destroy(link);
- bpf_link__destroy(link_fentry);
- bpf_link__destroy(link_fexit);
bpf_object__close(obj);
- bpf_object__close(obj2);
+ kfree_skb__destroy(skel);
}
diff --git a/tools/testing/selftests/bpf/prog_tests/kfunc_call.c b/tools/testing/selftests/bpf/prog_tests/kfunc_call.c
index 5c9c017..7d7445c 100644
--- a/tools/testing/selftests/bpf/prog_tests/kfunc_call.c
+++ b/tools/testing/selftests/bpf/prog_tests/kfunc_call.c
@@ -4,6 +4,7 @@
#include <network_helpers.h>
#include "kfunc_call_test.lskel.h"
#include "kfunc_call_test_subprog.skel.h"
+#include "kfunc_call_test_subprog.lskel.h"
static void test_main(void)
{
@@ -49,6 +50,26 @@ static void test_subprog(void)
kfunc_call_test_subprog__destroy(skel);
}
+static void test_subprog_lskel(void)
+{
+ struct kfunc_call_test_subprog_lskel *skel;
+ int prog_fd, retval, err;
+
+ skel = kfunc_call_test_subprog_lskel__open_and_load();
+ if (!ASSERT_OK_PTR(skel, "skel"))
+ return;
+
+ prog_fd = skel->progs.kfunc_call_test1.prog_fd;
+ err = bpf_prog_test_run(prog_fd, 1, &pkt_v4, sizeof(pkt_v4),
+ NULL, NULL, (__u32 *)&retval, NULL);
+ ASSERT_OK(err, "bpf_prog_test_run(test1)");
+ ASSERT_EQ(retval, 10, "test1-retval");
+ ASSERT_NEQ(skel->data->active_res, -1, "active_res");
+ ASSERT_EQ(skel->data->sk_state_res, BPF_TCP_CLOSE, "sk_state_res");
+
+ kfunc_call_test_subprog_lskel__destroy(skel);
+}
+
void test_kfunc_call(void)
{
if (test__start_subtest("main"))
@@ -56,4 +77,7 @@ void test_kfunc_call(void)
if (test__start_subtest("subprog"))
test_subprog();
+
+ if (test__start_subtest("subprog_lskel"))
+ test_subprog_lskel();
}
diff --git a/tools/testing/selftests/bpf/prog_tests/ksyms_btf.c b/tools/testing/selftests/bpf/prog_tests/ksyms_btf.c
index 79f6bd1..f6933b0 100644
--- a/tools/testing/selftests/bpf/prog_tests/ksyms_btf.c
+++ b/tools/testing/selftests/bpf/prog_tests/ksyms_btf.c
@@ -8,6 +8,7 @@
#include "test_ksyms_btf_null_check.skel.h"
#include "test_ksyms_weak.skel.h"
#include "test_ksyms_weak.lskel.h"
+#include "test_ksyms_btf_write_check.skel.h"
static int duration;
@@ -137,6 +138,16 @@ static void test_weak_syms_lskel(void)
test_ksyms_weak_lskel__destroy(skel);
}
+static void test_write_check(void)
+{
+ struct test_ksyms_btf_write_check *skel;
+
+ skel = test_ksyms_btf_write_check__open_and_load();
+ ASSERT_ERR_PTR(skel, "unexpected load of a prog writing to ksym memory\n");
+
+ test_ksyms_btf_write_check__destroy(skel);
+}
+
void test_ksyms_btf(void)
{
int percpu_datasec;
@@ -167,4 +178,7 @@ void test_ksyms_btf(void)
if (test__start_subtest("weak_ksyms_lskel"))
test_weak_syms_lskel();
+
+ if (test__start_subtest("write_check"))
+ test_write_check();
}
diff --git a/tools/testing/selftests/bpf/prog_tests/l4lb_all.c b/tools/testing/selftests/bpf/prog_tests/l4lb_all.c
index 8073105..540ef28 100644
--- a/tools/testing/selftests/bpf/prog_tests/l4lb_all.c
+++ b/tools/testing/selftests/bpf/prog_tests/l4lb_all.c
@@ -30,7 +30,7 @@ static void test_l4lb(const char *file)
char buf[128];
u32 *magic = (u32 *)buf;
- err = bpf_prog_load(file, BPF_PROG_TYPE_SCHED_CLS, &obj, &prog_fd);
+ err = bpf_prog_test_load(file, BPF_PROG_TYPE_SCHED_CLS, &obj, &prog_fd);
if (CHECK_FAIL(err))
return;
diff --git a/tools/testing/selftests/bpf/prog_tests/legacy_printk.c b/tools/testing/selftests/bpf/prog_tests/legacy_printk.c
new file mode 100644
index 0000000..ec6e45f
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/legacy_printk.c
@@ -0,0 +1,65 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2021 Facebook */
+#include <test_progs.h>
+#include "test_legacy_printk.skel.h"
+
+static int execute_one_variant(bool legacy)
+{
+ struct test_legacy_printk *skel;
+ int err, zero = 0, my_pid = getpid(), res, map_fd;
+
+ skel = test_legacy_printk__open();
+ if (!ASSERT_OK_PTR(skel, "skel_open"))
+ return -errno;
+
+ bpf_program__set_autoload(skel->progs.handle_legacy, legacy);
+ bpf_program__set_autoload(skel->progs.handle_modern, !legacy);
+
+ err = test_legacy_printk__load(skel);
+ /* no ASSERT_OK, we expect one of two variants can fail here */
+ if (err)
+ goto err_out;
+
+ if (legacy) {
+ map_fd = bpf_map__fd(skel->maps.my_pid_map);
+ err = bpf_map_update_elem(map_fd, &zero, &my_pid, BPF_ANY);
+ if (!ASSERT_OK(err, "my_pid_map_update"))
+ goto err_out;
+ err = bpf_map_lookup_elem(map_fd, &zero, &res);
+ } else {
+ skel->bss->my_pid_var = my_pid;
+ }
+
+ err = test_legacy_printk__attach(skel);
+ if (!ASSERT_OK(err, "skel_attach"))
+ goto err_out;
+
+ usleep(1); /* trigger */
+
+ if (legacy) {
+ map_fd = bpf_map__fd(skel->maps.res_map);
+ err = bpf_map_lookup_elem(map_fd, &zero, &res);
+ if (!ASSERT_OK(err, "res_map_lookup"))
+ goto err_out;
+ } else {
+ res = skel->bss->res_var;
+ }
+
+ if (!ASSERT_GT(res, 0, "res")) {
+ err = -EINVAL;
+ goto err_out;
+ }
+
+err_out:
+ test_legacy_printk__destroy(skel);
+ return err;
+}
+
+void test_legacy_printk(void)
+{
+ /* legacy variant should work everywhere */
+ ASSERT_OK(execute_one_variant(true /* legacy */), "legacy_case");
+
+ /* execute modern variant, can fail the load on old kernels */
+ execute_one_variant(false);
+}
diff --git a/tools/testing/selftests/bpf/prog_tests/libbpf_probes.c b/tools/testing/selftests/bpf/prog_tests/libbpf_probes.c
new file mode 100644
index 0000000..9f766dd
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/libbpf_probes.c
@@ -0,0 +1,124 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (c) 2021 Facebook */
+
+#include <test_progs.h>
+#include <bpf/btf.h>
+
+void test_libbpf_probe_prog_types(void)
+{
+ struct btf *btf;
+ const struct btf_type *t;
+ const struct btf_enum *e;
+ int i, n, id;
+
+ btf = btf__parse("/sys/kernel/btf/vmlinux", NULL);
+ if (!ASSERT_OK_PTR(btf, "btf_parse"))
+ return;
+
+ /* find enum bpf_prog_type and enumerate each value */
+ id = btf__find_by_name_kind(btf, "bpf_prog_type", BTF_KIND_ENUM);
+ if (!ASSERT_GT(id, 0, "bpf_prog_type_id"))
+ goto cleanup;
+ t = btf__type_by_id(btf, id);
+ if (!ASSERT_OK_PTR(t, "bpf_prog_type_enum"))
+ goto cleanup;
+
+ for (e = btf_enum(t), i = 0, n = btf_vlen(t); i < n; e++, i++) {
+ const char *prog_type_name = btf__str_by_offset(btf, e->name_off);
+ enum bpf_prog_type prog_type = (enum bpf_prog_type)e->val;
+ int res;
+
+ if (prog_type == BPF_PROG_TYPE_UNSPEC)
+ continue;
+
+ if (!test__start_subtest(prog_type_name))
+ continue;
+
+ res = libbpf_probe_bpf_prog_type(prog_type, NULL);
+ ASSERT_EQ(res, 1, prog_type_name);
+ }
+
+cleanup:
+ btf__free(btf);
+}
+
+void test_libbpf_probe_map_types(void)
+{
+ struct btf *btf;
+ const struct btf_type *t;
+ const struct btf_enum *e;
+ int i, n, id;
+
+ btf = btf__parse("/sys/kernel/btf/vmlinux", NULL);
+ if (!ASSERT_OK_PTR(btf, "btf_parse"))
+ return;
+
+ /* find enum bpf_map_type and enumerate each value */
+ id = btf__find_by_name_kind(btf, "bpf_map_type", BTF_KIND_ENUM);
+ if (!ASSERT_GT(id, 0, "bpf_map_type_id"))
+ goto cleanup;
+ t = btf__type_by_id(btf, id);
+ if (!ASSERT_OK_PTR(t, "bpf_map_type_enum"))
+ goto cleanup;
+
+ for (e = btf_enum(t), i = 0, n = btf_vlen(t); i < n; e++, i++) {
+ const char *map_type_name = btf__str_by_offset(btf, e->name_off);
+ enum bpf_map_type map_type = (enum bpf_map_type)e->val;
+ int res;
+
+ if (map_type == BPF_MAP_TYPE_UNSPEC)
+ continue;
+
+ if (!test__start_subtest(map_type_name))
+ continue;
+
+ res = libbpf_probe_bpf_map_type(map_type, NULL);
+ ASSERT_EQ(res, 1, map_type_name);
+ }
+
+cleanup:
+ btf__free(btf);
+}
+
+void test_libbpf_probe_helpers(void)
+{
+#define CASE(prog, helper, supp) { \
+ .prog_type_name = "BPF_PROG_TYPE_" # prog, \
+ .helper_name = "bpf_" # helper, \
+ .prog_type = BPF_PROG_TYPE_ ## prog, \
+ .helper_id = BPF_FUNC_ ## helper, \
+ .supported = supp, \
+}
+ const struct case_def {
+ const char *prog_type_name;
+ const char *helper_name;
+ enum bpf_prog_type prog_type;
+ enum bpf_func_id helper_id;
+ bool supported;
+ } cases[] = {
+ CASE(KPROBE, unspec, false),
+ CASE(KPROBE, map_lookup_elem, true),
+ CASE(KPROBE, loop, true),
+
+ CASE(KPROBE, ktime_get_coarse_ns, false),
+ CASE(SOCKET_FILTER, ktime_get_coarse_ns, true),
+
+ CASE(KPROBE, sys_bpf, false),
+ CASE(SYSCALL, sys_bpf, true),
+ };
+ size_t case_cnt = ARRAY_SIZE(cases), i;
+ char buf[128];
+
+ for (i = 0; i < case_cnt; i++) {
+ const struct case_def *d = &cases[i];
+ int res;
+
+ snprintf(buf, sizeof(buf), "%s+%s", d->prog_type_name, d->helper_name);
+
+ if (!test__start_subtest(buf))
+ continue;
+
+ res = libbpf_probe_bpf_helper(d->prog_type, d->helper_id, NULL);
+ ASSERT_EQ(res, d->supported, buf);
+ }
+}
diff --git a/tools/testing/selftests/bpf/prog_tests/load_bytes_relative.c b/tools/testing/selftests/bpf/prog_tests/load_bytes_relative.c
index 5a2a689..4e0b2ec 100644
--- a/tools/testing/selftests/bpf/prog_tests/load_bytes_relative.c
+++ b/tools/testing/selftests/bpf/prog_tests/load_bytes_relative.c
@@ -27,7 +27,7 @@ void test_load_bytes_relative(void)
if (CHECK_FAIL(server_fd < 0))
goto close_cgroup_fd;
- err = bpf_prog_load("./load_bytes_relative.o", BPF_PROG_TYPE_CGROUP_SKB,
+ err = bpf_prog_test_load("./load_bytes_relative.o", BPF_PROG_TYPE_CGROUP_SKB,
&obj, &prog_fd);
if (CHECK_FAIL(err))
goto close_server_fd;
diff --git a/tools/testing/selftests/bpf/prog_tests/log_buf.c b/tools/testing/selftests/bpf/prog_tests/log_buf.c
new file mode 100644
index 0000000..e469b02
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/log_buf.c
@@ -0,0 +1,276 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2021 Facebook */
+
+#include <test_progs.h>
+#include <bpf/btf.h>
+
+#include "test_log_buf.skel.h"
+
+static size_t libbpf_log_pos;
+static char libbpf_log_buf[1024 * 1024];
+static bool libbpf_log_error;
+
+static int libbpf_print_cb(enum libbpf_print_level level, const char *fmt, va_list args)
+{
+ int emitted_cnt;
+ size_t left_cnt;
+
+ left_cnt = sizeof(libbpf_log_buf) - libbpf_log_pos;
+ emitted_cnt = vsnprintf(libbpf_log_buf + libbpf_log_pos, left_cnt, fmt, args);
+
+ if (emitted_cnt < 0 || emitted_cnt + 1 > left_cnt) {
+ libbpf_log_error = true;
+ return 0;
+ }
+
+ libbpf_log_pos += emitted_cnt;
+ return 0;
+}
+
+static void obj_load_log_buf(void)
+{
+ libbpf_print_fn_t old_print_cb = libbpf_set_print(libbpf_print_cb);
+ LIBBPF_OPTS(bpf_object_open_opts, opts);
+ const size_t log_buf_sz = 1024 * 1024;
+ struct test_log_buf* skel;
+ char *obj_log_buf, *good_log_buf, *bad_log_buf;
+ int err;
+
+ obj_log_buf = malloc(3 * log_buf_sz);
+ if (!ASSERT_OK_PTR(obj_log_buf, "obj_log_buf"))
+ return;
+
+ good_log_buf = obj_log_buf + log_buf_sz;
+ bad_log_buf = obj_log_buf + 2 * log_buf_sz;
+ obj_log_buf[0] = good_log_buf[0] = bad_log_buf[0] = '\0';
+
+ opts.kernel_log_buf = obj_log_buf;
+ opts.kernel_log_size = log_buf_sz;
+ opts.kernel_log_level = 4; /* for BTF this will turn into 1 */
+
+ /* In the first round every prog has its own log_buf, so libbpf logs
+ * don't have program failure logs
+ */
+ skel = test_log_buf__open_opts(&opts);
+ if (!ASSERT_OK_PTR(skel, "skel_open"))
+ goto cleanup;
+
+ /* set very verbose level for good_prog so we always get detailed logs */
+ bpf_program__set_log_buf(skel->progs.good_prog, good_log_buf, log_buf_sz);
+ bpf_program__set_log_level(skel->progs.good_prog, 2);
+
+ bpf_program__set_log_buf(skel->progs.bad_prog, bad_log_buf, log_buf_sz);
+ /* log_level 0 with custom log_buf means that verbose logs are not
+ * requested if program load is successful, but libbpf should retry
+ * with log_level 1 on error and put program's verbose load log into
+ * custom log_buf
+ */
+ bpf_program__set_log_level(skel->progs.bad_prog, 0);
+
+ err = test_log_buf__load(skel);
+ if (!ASSERT_ERR(err, "unexpected_load_success"))
+ goto cleanup;
+
+ ASSERT_FALSE(libbpf_log_error, "libbpf_log_error");
+
+ /* there should be no prog loading log because we specified per-prog log buf */
+ ASSERT_NULL(strstr(libbpf_log_buf, "-- BEGIN PROG LOAD LOG --"), "unexp_libbpf_log");
+ ASSERT_OK_PTR(strstr(libbpf_log_buf, "prog 'bad_prog': BPF program load failed"),
+ "libbpf_log_not_empty");
+ ASSERT_OK_PTR(strstr(obj_log_buf, "DATASEC license"), "obj_log_not_empty");
+ ASSERT_OK_PTR(strstr(good_log_buf, "0: R1=ctx(id=0,off=0,imm=0) R10=fp0"),
+ "good_log_verbose");
+ ASSERT_OK_PTR(strstr(bad_log_buf, "invalid access to map value, value_size=16 off=16000 size=4"),
+ "bad_log_not_empty");
+
+ if (env.verbosity > VERBOSE_NONE) {
+ printf("LIBBPF LOG: \n=================\n%s=================\n", libbpf_log_buf);
+ printf("OBJ LOG: \n=================\n%s=================\n", obj_log_buf);
+ printf("GOOD_PROG LOG:\n=================\n%s=================\n", good_log_buf);
+ printf("BAD_PROG LOG:\n=================\n%s=================\n", bad_log_buf);
+ }
+
+ /* reset everything */
+ test_log_buf__destroy(skel);
+ obj_log_buf[0] = good_log_buf[0] = bad_log_buf[0] = '\0';
+ libbpf_log_buf[0] = '\0';
+ libbpf_log_pos = 0;
+ libbpf_log_error = false;
+
+ /* In the second round we let bad_prog's failure be logged through print callback */
+ opts.kernel_log_buf = NULL; /* let everything through into print callback */
+ opts.kernel_log_size = 0;
+ opts.kernel_log_level = 1;
+
+ skel = test_log_buf__open_opts(&opts);
+ if (!ASSERT_OK_PTR(skel, "skel_open"))
+ goto cleanup;
+
+ /* set normal verbose level for good_prog to check log_level is taken into account */
+ bpf_program__set_log_buf(skel->progs.good_prog, good_log_buf, log_buf_sz);
+ bpf_program__set_log_level(skel->progs.good_prog, 1);
+
+ err = test_log_buf__load(skel);
+ if (!ASSERT_ERR(err, "unexpected_load_success"))
+ goto cleanup;
+
+ ASSERT_FALSE(libbpf_log_error, "libbpf_log_error");
+
+ /* this time prog loading error should be logged through print callback */
+ ASSERT_OK_PTR(strstr(libbpf_log_buf, "libbpf: prog 'bad_prog': -- BEGIN PROG LOAD LOG --"),
+ "libbpf_log_correct");
+ ASSERT_STREQ(obj_log_buf, "", "obj_log__empty");
+ ASSERT_STREQ(good_log_buf, "processed 4 insns (limit 1000000) max_states_per_insn 0 total_states 0 peak_states 0 mark_read 0\n",
+ "good_log_ok");
+ ASSERT_STREQ(bad_log_buf, "", "bad_log_empty");
+
+ if (env.verbosity > VERBOSE_NONE) {
+ printf("LIBBPF LOG: \n=================\n%s=================\n", libbpf_log_buf);
+ printf("OBJ LOG: \n=================\n%s=================\n", obj_log_buf);
+ printf("GOOD_PROG LOG:\n=================\n%s=================\n", good_log_buf);
+ printf("BAD_PROG LOG:\n=================\n%s=================\n", bad_log_buf);
+ }
+
+cleanup:
+ free(obj_log_buf);
+ test_log_buf__destroy(skel);
+ libbpf_set_print(old_print_cb);
+}
+
+static void bpf_prog_load_log_buf(void)
+{
+ const struct bpf_insn good_prog_insns[] = {
+ BPF_MOV64_IMM(BPF_REG_0, 0),
+ BPF_EXIT_INSN(),
+ };
+ const size_t good_prog_insn_cnt = sizeof(good_prog_insns) / sizeof(struct bpf_insn);
+ const struct bpf_insn bad_prog_insns[] = {
+ BPF_EXIT_INSN(),
+ };
+ size_t bad_prog_insn_cnt = sizeof(bad_prog_insns) / sizeof(struct bpf_insn);
+ LIBBPF_OPTS(bpf_prog_load_opts, opts);
+ const size_t log_buf_sz = 1024 * 1024;
+ char *log_buf;
+ int fd = -1;
+
+ log_buf = malloc(log_buf_sz);
+ if (!ASSERT_OK_PTR(log_buf, "log_buf_alloc"))
+ return;
+ opts.log_buf = log_buf;
+ opts.log_size = log_buf_sz;
+
+ /* with log_level == 0 log_buf shoud stay empty for good prog */
+ log_buf[0] = '\0';
+ opts.log_level = 0;
+ fd = bpf_prog_load(BPF_PROG_TYPE_SOCKET_FILTER, "good_prog", "GPL",
+ good_prog_insns, good_prog_insn_cnt, &opts);
+ ASSERT_STREQ(log_buf, "", "good_log_0");
+ ASSERT_GE(fd, 0, "good_fd1");
+ if (fd >= 0)
+ close(fd);
+ fd = -1;
+
+ /* log_level == 2 should always fill log_buf, even for good prog */
+ log_buf[0] = '\0';
+ opts.log_level = 2;
+ fd = bpf_prog_load(BPF_PROG_TYPE_SOCKET_FILTER, "good_prog", "GPL",
+ good_prog_insns, good_prog_insn_cnt, &opts);
+ ASSERT_OK_PTR(strstr(log_buf, "0: R1=ctx(id=0,off=0,imm=0) R10=fp0"), "good_log_2");
+ ASSERT_GE(fd, 0, "good_fd2");
+ if (fd >= 0)
+ close(fd);
+ fd = -1;
+
+ /* log_level == 0 should fill log_buf for bad prog */
+ log_buf[0] = '\0';
+ opts.log_level = 0;
+ fd = bpf_prog_load(BPF_PROG_TYPE_SOCKET_FILTER, "bad_prog", "GPL",
+ bad_prog_insns, bad_prog_insn_cnt, &opts);
+ ASSERT_OK_PTR(strstr(log_buf, "R0 !read_ok"), "bad_log_0");
+ ASSERT_LT(fd, 0, "bad_fd");
+ if (fd >= 0)
+ close(fd);
+ fd = -1;
+
+ free(log_buf);
+}
+
+static void bpf_btf_load_log_buf(void)
+{
+ LIBBPF_OPTS(bpf_btf_load_opts, opts);
+ const size_t log_buf_sz = 1024 * 1024;
+ const void *raw_btf_data;
+ __u32 raw_btf_size;
+ struct btf *btf;
+ char *log_buf;
+ int fd = -1;
+
+ btf = btf__new_empty();
+ if (!ASSERT_OK_PTR(btf, "empty_btf"))
+ return;
+
+ ASSERT_GT(btf__add_int(btf, "int", 4, 0), 0, "int_type");
+
+ raw_btf_data = btf__raw_data(btf, &raw_btf_size);
+ if (!ASSERT_OK_PTR(raw_btf_data, "raw_btf_data_good"))
+ goto cleanup;
+
+ log_buf = malloc(log_buf_sz);
+ if (!ASSERT_OK_PTR(log_buf, "log_buf_alloc"))
+ goto cleanup;
+ opts.log_buf = log_buf;
+ opts.log_size = log_buf_sz;
+
+ /* with log_level == 0 log_buf shoud stay empty for good BTF */
+ log_buf[0] = '\0';
+ opts.log_level = 0;
+ fd = bpf_btf_load(raw_btf_data, raw_btf_size, &opts);
+ ASSERT_STREQ(log_buf, "", "good_log_0");
+ ASSERT_GE(fd, 0, "good_fd1");
+ if (fd >= 0)
+ close(fd);
+ fd = -1;
+
+ /* log_level == 2 should always fill log_buf, even for good BTF */
+ log_buf[0] = '\0';
+ opts.log_level = 2;
+ fd = bpf_btf_load(raw_btf_data, raw_btf_size, &opts);
+ printf("LOG_BUF: %s\n", log_buf);
+ ASSERT_OK_PTR(strstr(log_buf, "magic: 0xeb9f"), "good_log_2");
+ ASSERT_GE(fd, 0, "good_fd2");
+ if (fd >= 0)
+ close(fd);
+ fd = -1;
+
+ /* make BTF bad, add pointer pointing to non-existing type */
+ ASSERT_GT(btf__add_ptr(btf, 100), 0, "bad_ptr_type");
+
+ raw_btf_data = btf__raw_data(btf, &raw_btf_size);
+ if (!ASSERT_OK_PTR(raw_btf_data, "raw_btf_data_bad"))
+ goto cleanup;
+
+ /* log_level == 0 should fill log_buf for bad BTF */
+ log_buf[0] = '\0';
+ opts.log_level = 0;
+ fd = bpf_btf_load(raw_btf_data, raw_btf_size, &opts);
+ printf("LOG_BUF: %s\n", log_buf);
+ ASSERT_OK_PTR(strstr(log_buf, "[2] PTR (anon) type_id=100 Invalid type_id"), "bad_log_0");
+ ASSERT_LT(fd, 0, "bad_fd");
+ if (fd >= 0)
+ close(fd);
+ fd = -1;
+
+cleanup:
+ free(log_buf);
+ btf__free(btf);
+}
+
+void test_log_buf(void)
+{
+ if (test__start_subtest("obj_load_log_buf"))
+ obj_load_log_buf();
+ if (test__start_subtest("bpf_prog_load_log_buf"))
+ bpf_prog_load_log_buf();
+ if (test__start_subtest("bpf_btf_load_log_buf"))
+ bpf_btf_load_log_buf();
+}
diff --git a/tools/testing/selftests/bpf/prog_tests/map_lock.c b/tools/testing/selftests/bpf/prog_tests/map_lock.c
index ce17b1e..23d19e9c 100644
--- a/tools/testing/selftests/bpf/prog_tests/map_lock.c
+++ b/tools/testing/selftests/bpf/prog_tests/map_lock.c
@@ -53,9 +53,9 @@ void test_map_lock(void)
int err = 0, key = 0, i;
void *ret;
- err = bpf_prog_load(file, BPF_PROG_TYPE_CGROUP_SKB, &obj, &prog_fd);
+ err = bpf_prog_test_load(file, BPF_PROG_TYPE_CGROUP_SKB, &obj, &prog_fd);
if (CHECK_FAIL(err)) {
- printf("test_map_lock:bpf_prog_load errno %d\n", errno);
+ printf("test_map_lock:bpf_prog_test_load errno %d\n", errno);
goto close_prog;
}
map_fd[0] = bpf_find_map(__func__, obj, "hash_map");
diff --git a/tools/testing/selftests/bpf/prog_tests/map_ptr.c b/tools/testing/selftests/bpf/prog_tests/map_ptr.c
index 4972f92..2737255 100644
--- a/tools/testing/selftests/bpf/prog_tests/map_ptr.c
+++ b/tools/testing/selftests/bpf/prog_tests/map_ptr.c
@@ -4,31 +4,29 @@
#include <test_progs.h>
#include <network_helpers.h>
-#include "map_ptr_kern.skel.h"
+#include "map_ptr_kern.lskel.h"
void test_map_ptr(void)
{
- struct map_ptr_kern *skel;
+ struct map_ptr_kern_lskel *skel;
__u32 duration = 0, retval;
char buf[128];
int err;
int page_size = getpagesize();
- skel = map_ptr_kern__open();
+ skel = map_ptr_kern_lskel__open();
if (!ASSERT_OK_PTR(skel, "skel_open"))
return;
- err = bpf_map__set_max_entries(skel->maps.m_ringbuf, page_size);
- if (!ASSERT_OK(err, "bpf_map__set_max_entries"))
- goto cleanup;
+ skel->maps.m_ringbuf.max_entries = page_size;
- err = map_ptr_kern__load(skel);
+ err = map_ptr_kern_lskel__load(skel);
if (!ASSERT_OK(err, "skel_load"))
goto cleanup;
skel->bss->page_size = page_size;
- err = bpf_prog_test_run(bpf_program__fd(skel->progs.cg_skb), 1, &pkt_v4,
+ err = bpf_prog_test_run(skel->progs.cg_skb.prog_fd, 1, &pkt_v4,
sizeof(pkt_v4), buf, NULL, &retval, NULL);
if (CHECK(err, "test_run", "err=%d errno=%d\n", err, errno))
@@ -39,5 +37,5 @@ void test_map_ptr(void)
goto cleanup;
cleanup:
- map_ptr_kern__destroy(skel);
+ map_ptr_kern_lskel__destroy(skel);
}
diff --git a/tools/testing/selftests/bpf/prog_tests/migrate_reuseport.c b/tools/testing/selftests/bpf/prog_tests/migrate_reuseport.c
index 7589c03..eb2feaa 100644
--- a/tools/testing/selftests/bpf/prog_tests/migrate_reuseport.c
+++ b/tools/testing/selftests/bpf/prog_tests/migrate_reuseport.c
@@ -204,8 +204,8 @@ static int pass_ack(struct migrate_reuseport_test_case *test_case)
{
int err;
- err = bpf_link__detach(test_case->link);
- if (!ASSERT_OK(err, "bpf_link__detach"))
+ err = bpf_link__destroy(test_case->link);
+ if (!ASSERT_OK(err, "bpf_link__destroy"))
return -1;
test_case->link = NULL;
diff --git a/tools/testing/selftests/bpf/prog_tests/perf_buffer.c b/tools/testing/selftests/bpf/prog_tests/perf_buffer.c
index 4e32f35..5fc2b3a 100644
--- a/tools/testing/selftests/bpf/prog_tests/perf_buffer.c
+++ b/tools/testing/selftests/bpf/prog_tests/perf_buffer.c
@@ -47,7 +47,6 @@ void serial_test_perf_buffer(void)
{
int err, on_len, nr_on_cpus = 0, nr_cpus, i, j;
int zero = 0, my_pid = getpid();
- struct perf_buffer_opts pb_opts = {};
struct test_perf_buffer *skel;
cpu_set_t cpu_seen;
struct perf_buffer *pb;
@@ -82,9 +81,8 @@ void serial_test_perf_buffer(void)
goto out_close;
/* set up perf buffer */
- pb_opts.sample_cb = on_sample;
- pb_opts.ctx = &cpu_seen;
- pb = perf_buffer__new(bpf_map__fd(skel->maps.perf_buf_map), 1, &pb_opts);
+ pb = perf_buffer__new(bpf_map__fd(skel->maps.perf_buf_map), 1,
+ on_sample, NULL, &cpu_seen, NULL);
if (!ASSERT_OK_PTR(pb, "perf_buf__new"))
goto out_close;
diff --git a/tools/testing/selftests/bpf/prog_tests/pinning.c b/tools/testing/selftests/bpf/prog_tests/pinning.c
index d4b953a..31c09ba 100644
--- a/tools/testing/selftests/bpf/prog_tests/pinning.c
+++ b/tools/testing/selftests/bpf/prog_tests/pinning.c
@@ -241,8 +241,8 @@ void test_pinning(void)
goto out;
}
- map_fd = bpf_create_map(BPF_MAP_TYPE_ARRAY, sizeof(__u32),
- sizeof(__u64), 1, 0);
+ map_fd = bpf_map_create(BPF_MAP_TYPE_ARRAY, NULL, sizeof(__u32),
+ sizeof(__u64), 1, NULL);
if (CHECK(map_fd < 0, "create pinmap manually", "fd %d\n", map_fd))
goto out;
diff --git a/tools/testing/selftests/bpf/prog_tests/pkt_access.c b/tools/testing/selftests/bpf/prog_tests/pkt_access.c
index 44b514f..6628710 100644
--- a/tools/testing/selftests/bpf/prog_tests/pkt_access.c
+++ b/tools/testing/selftests/bpf/prog_tests/pkt_access.c
@@ -9,7 +9,7 @@ void test_pkt_access(void)
__u32 duration, retval;
int err, prog_fd;
- err = bpf_prog_load(file, BPF_PROG_TYPE_SCHED_CLS, &obj, &prog_fd);
+ err = bpf_prog_test_load(file, BPF_PROG_TYPE_SCHED_CLS, &obj, &prog_fd);
if (CHECK_FAIL(err))
return;
diff --git a/tools/testing/selftests/bpf/prog_tests/pkt_md_access.c b/tools/testing/selftests/bpf/prog_tests/pkt_md_access.c
index 939015c..c9d2d6a 100644
--- a/tools/testing/selftests/bpf/prog_tests/pkt_md_access.c
+++ b/tools/testing/selftests/bpf/prog_tests/pkt_md_access.c
@@ -9,7 +9,7 @@ void test_pkt_md_access(void)
__u32 duration, retval;
int err, prog_fd;
- err = bpf_prog_load(file, BPF_PROG_TYPE_SCHED_CLS, &obj, &prog_fd);
+ err = bpf_prog_test_load(file, BPF_PROG_TYPE_SCHED_CLS, &obj, &prog_fd);
if (CHECK_FAIL(err))
return;
diff --git a/tools/testing/selftests/bpf/prog_tests/prog_array_init.c b/tools/testing/selftests/bpf/prog_tests/prog_array_init.c
new file mode 100644
index 0000000..fc46576
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/prog_array_init.c
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (c) 2021 Hengqi Chen */
+
+#include <test_progs.h>
+#include "test_prog_array_init.skel.h"
+
+void test_prog_array_init(void)
+{
+ struct test_prog_array_init *skel;
+ int err;
+
+ skel = test_prog_array_init__open();
+ if (!ASSERT_OK_PTR(skel, "could not open BPF object"))
+ return;
+
+ skel->rodata->my_pid = getpid();
+
+ err = test_prog_array_init__load(skel);
+ if (!ASSERT_OK(err, "could not load BPF object"))
+ goto cleanup;
+
+ skel->links.entry = bpf_program__attach_raw_tracepoint(skel->progs.entry, "sys_enter");
+ if (!ASSERT_OK_PTR(skel->links.entry, "could not attach BPF program"))
+ goto cleanup;
+
+ usleep(1);
+
+ ASSERT_EQ(skel->bss->value, 42, "unexpected value");
+
+cleanup:
+ test_prog_array_init__destroy(skel);
+}
diff --git a/tools/testing/selftests/bpf/prog_tests/queue_stack_map.c b/tools/testing/selftests/bpf/prog_tests/queue_stack_map.c
index f47e7b1..b9822f9 100644
--- a/tools/testing/selftests/bpf/prog_tests/queue_stack_map.c
+++ b/tools/testing/selftests/bpf/prog_tests/queue_stack_map.c
@@ -14,7 +14,7 @@ static void test_queue_stack_map_by_type(int type)
int i, err, prog_fd, map_in_fd, map_out_fd;
char file[32], buf[128];
struct bpf_object *obj;
- struct iphdr *iph = (void *)buf + sizeof(struct ethhdr);
+ struct iphdr iph;
/* Fill test values to be used */
for (i = 0; i < MAP_SIZE; i++)
@@ -27,7 +27,7 @@ static void test_queue_stack_map_by_type(int type)
else
return;
- err = bpf_prog_load(file, BPF_PROG_TYPE_SCHED_CLS, &obj, &prog_fd);
+ err = bpf_prog_test_load(file, BPF_PROG_TYPE_SCHED_CLS, &obj, &prog_fd);
if (CHECK_FAIL(err))
return;
@@ -60,15 +60,17 @@ static void test_queue_stack_map_by_type(int type)
err = bpf_prog_test_run(prog_fd, 1, &pkt_v4, sizeof(pkt_v4),
buf, &size, &retval, &duration);
- if (err || retval || size != sizeof(pkt_v4) ||
- iph->daddr != val)
+ if (err || retval || size != sizeof(pkt_v4))
+ break;
+ memcpy(&iph, buf + sizeof(struct ethhdr), sizeof(iph));
+ if (iph.daddr != val)
break;
}
- CHECK(err || retval || size != sizeof(pkt_v4) || iph->daddr != val,
+ CHECK(err || retval || size != sizeof(pkt_v4) || iph.daddr != val,
"bpf_map_pop_elem",
"err %d errno %d retval %d size %d iph->daddr %u\n",
- err, errno, retval, size, iph->daddr);
+ err, errno, retval, size, iph.daddr);
/* Queue is empty, program should return TC_ACT_SHOT */
err = bpf_prog_test_run(prog_fd, 1, &pkt_v4, sizeof(pkt_v4),
diff --git a/tools/testing/selftests/bpf/prog_tests/raw_tp_writable_reject_nbd_invalid.c b/tools/testing/selftests/bpf/prog_tests/raw_tp_writable_reject_nbd_invalid.c
index 9807336..e2f1445 100644
--- a/tools/testing/selftests/bpf/prog_tests/raw_tp_writable_reject_nbd_invalid.c
+++ b/tools/testing/selftests/bpf/prog_tests/raw_tp_writable_reject_nbd_invalid.c
@@ -18,15 +18,15 @@ void test_raw_tp_writable_reject_nbd_invalid(void)
BPF_EXIT_INSN(),
};
- struct bpf_load_program_attr load_attr = {
- .prog_type = BPF_PROG_TYPE_RAW_TRACEPOINT_WRITABLE,
- .license = "GPL v2",
- .insns = program,
- .insns_cnt = sizeof(program) / sizeof(struct bpf_insn),
+ LIBBPF_OPTS(bpf_prog_load_opts, opts,
.log_level = 2,
- };
+ .log_buf = error,
+ .log_size = sizeof(error),
+ );
- bpf_fd = bpf_load_program_xattr(&load_attr, error, sizeof(error));
+ bpf_fd = bpf_prog_load(BPF_PROG_TYPE_RAW_TRACEPOINT_WRITABLE, NULL, "GPL v2",
+ program, sizeof(program) / sizeof(struct bpf_insn),
+ &opts);
if (CHECK(bpf_fd < 0, "bpf_raw_tracepoint_writable load",
"failed: %d errno %d\n", bpf_fd, errno))
return;
diff --git a/tools/testing/selftests/bpf/prog_tests/raw_tp_writable_test_run.c b/tools/testing/selftests/bpf/prog_tests/raw_tp_writable_test_run.c
index ddefa11..239bacca 100644
--- a/tools/testing/selftests/bpf/prog_tests/raw_tp_writable_test_run.c
+++ b/tools/testing/selftests/bpf/prog_tests/raw_tp_writable_test_run.c
@@ -17,15 +17,15 @@ void serial_test_raw_tp_writable_test_run(void)
BPF_EXIT_INSN(),
};
- struct bpf_load_program_attr load_attr = {
- .prog_type = BPF_PROG_TYPE_RAW_TRACEPOINT_WRITABLE,
- .license = "GPL v2",
- .insns = trace_program,
- .insns_cnt = sizeof(trace_program) / sizeof(struct bpf_insn),
+ LIBBPF_OPTS(bpf_prog_load_opts, trace_opts,
.log_level = 2,
- };
+ .log_buf = error,
+ .log_size = sizeof(error),
+ );
- int bpf_fd = bpf_load_program_xattr(&load_attr, error, sizeof(error));
+ int bpf_fd = bpf_prog_load(BPF_PROG_TYPE_RAW_TRACEPOINT_WRITABLE, NULL, "GPL v2",
+ trace_program, sizeof(trace_program) / sizeof(struct bpf_insn),
+ &trace_opts);
if (CHECK(bpf_fd < 0, "bpf_raw_tracepoint_writable loaded",
"failed: %d errno %d\n", bpf_fd, errno))
return;
@@ -35,15 +35,14 @@ void serial_test_raw_tp_writable_test_run(void)
BPF_EXIT_INSN(),
};
- struct bpf_load_program_attr skb_load_attr = {
- .prog_type = BPF_PROG_TYPE_SOCKET_FILTER,
- .license = "GPL v2",
- .insns = skb_program,
- .insns_cnt = sizeof(skb_program) / sizeof(struct bpf_insn),
- };
+ LIBBPF_OPTS(bpf_prog_load_opts, skb_opts,
+ .log_buf = error,
+ .log_size = sizeof(error),
+ );
- int filter_fd =
- bpf_load_program_xattr(&skb_load_attr, error, sizeof(error));
+ int filter_fd = bpf_prog_load(BPF_PROG_TYPE_SOCKET_FILTER, NULL, "GPL v2",
+ skb_program, sizeof(skb_program) / sizeof(struct bpf_insn),
+ &skb_opts);
if (CHECK(filter_fd < 0, "test_program_loaded", "failed: %d errno %d\n",
filter_fd, errno))
goto out_bpffd;
diff --git a/tools/testing/selftests/bpf/prog_tests/ringbuf_multi.c b/tools/testing/selftests/bpf/prog_tests/ringbuf_multi.c
index 167cd8a..e945195 100644
--- a/tools/testing/selftests/bpf/prog_tests/ringbuf_multi.c
+++ b/tools/testing/selftests/bpf/prog_tests/ringbuf_multi.c
@@ -62,8 +62,8 @@ void test_ringbuf_multi(void)
if (CHECK(err != 0, "bpf_map__set_max_entries", "bpf_map__set_max_entries failed\n"))
goto cleanup;
- proto_fd = bpf_create_map(BPF_MAP_TYPE_RINGBUF, 0, 0, page_size, 0);
- if (CHECK(proto_fd < 0, "bpf_create_map", "bpf_create_map failed\n"))
+ proto_fd = bpf_map_create(BPF_MAP_TYPE_RINGBUF, NULL, 0, 0, page_size, NULL);
+ if (CHECK(proto_fd < 0, "bpf_map_create", "bpf_map_create failed\n"))
goto cleanup;
err = bpf_map__set_inner_map_fd(skel->maps.ringbuf_hash, proto_fd);
diff --git a/tools/testing/selftests/bpf/prog_tests/select_reuseport.c b/tools/testing/selftests/bpf/prog_tests/select_reuseport.c
index 3cfc910..1cbd8cd 100644
--- a/tools/testing/selftests/bpf/prog_tests/select_reuseport.c
+++ b/tools/testing/selftests/bpf/prog_tests/select_reuseport.c
@@ -18,7 +18,6 @@
#include <netinet/in.h>
#include <bpf/bpf.h>
#include <bpf/libbpf.h>
-#include "bpf_rlimit.h"
#include "bpf_util.h"
#include "test_progs.h"
@@ -66,29 +65,20 @@ static union sa46 {
static int create_maps(enum bpf_map_type inner_type)
{
- struct bpf_create_map_attr attr = {};
+ LIBBPF_OPTS(bpf_map_create_opts, opts);
inner_map_type = inner_type;
/* Creating reuseport_array */
- attr.name = "reuseport_array";
- attr.map_type = inner_type;
- attr.key_size = sizeof(__u32);
- attr.value_size = sizeof(__u32);
- attr.max_entries = REUSEPORT_ARRAY_SIZE;
-
- reuseport_array = bpf_create_map_xattr(&attr);
+ reuseport_array = bpf_map_create(inner_type, "reuseport_array",
+ sizeof(__u32), sizeof(__u32), REUSEPORT_ARRAY_SIZE, NULL);
RET_ERR(reuseport_array < 0, "creating reuseport_array",
"reuseport_array:%d errno:%d\n", reuseport_array, errno);
/* Creating outer_map */
- attr.name = "outer_map";
- attr.map_type = BPF_MAP_TYPE_ARRAY_OF_MAPS;
- attr.key_size = sizeof(__u32);
- attr.value_size = sizeof(__u32);
- attr.max_entries = 1;
- attr.inner_map_fd = reuseport_array;
- outer_map = bpf_create_map_xattr(&attr);
+ opts.inner_map_fd = reuseport_array;
+ outer_map = bpf_map_create(BPF_MAP_TYPE_ARRAY_OF_MAPS, "outer_map",
+ sizeof(__u32), sizeof(__u32), 1, &opts);
RET_ERR(outer_map < 0, "creating outer_map",
"outer_map:%d errno:%d\n", outer_map, errno);
diff --git a/tools/testing/selftests/bpf/prog_tests/signal_pending.c b/tools/testing/selftests/bpf/prog_tests/signal_pending.c
index fdfdcff..aecfe66 100644
--- a/tools/testing/selftests/bpf/prog_tests/signal_pending.c
+++ b/tools/testing/selftests/bpf/prog_tests/signal_pending.c
@@ -22,7 +22,7 @@ static void test_signal_pending_by_type(enum bpf_prog_type prog_type)
prog[i] = BPF_ALU64_IMM(BPF_MOV, BPF_REG_0, 0);
prog[ARRAY_SIZE(prog) - 1] = BPF_EXIT_INSN();
- prog_fd = bpf_load_program(prog_type, prog, ARRAY_SIZE(prog),
+ prog_fd = bpf_test_load_program(prog_type, prog, ARRAY_SIZE(prog),
"GPL", 0, NULL, 0);
CHECK(prog_fd < 0, "test-run", "errno %d\n", errno);
diff --git a/tools/testing/selftests/bpf/prog_tests/sk_lookup.c b/tools/testing/selftests/bpf/prog_tests/sk_lookup.c
index 6db07401b..597d046 100644
--- a/tools/testing/selftests/bpf/prog_tests/sk_lookup.c
+++ b/tools/testing/selftests/bpf/prog_tests/sk_lookup.c
@@ -30,7 +30,6 @@
#include <bpf/bpf.h>
#include "test_progs.h"
-#include "bpf_rlimit.h"
#include "bpf_util.h"
#include "cgroup_helpers.h"
#include "network_helpers.h"
@@ -937,6 +936,37 @@ static void test_drop_on_lookup(struct test_sk_lookup *skel)
.connect_to = { EXT_IP6, EXT_PORT },
.listen_at = { EXT_IP6, INT_PORT },
},
+ /* The program will drop on success, meaning that the ifindex
+ * was 1.
+ */
+ {
+ .desc = "TCP IPv4 drop on valid ifindex",
+ .lookup_prog = skel->progs.check_ifindex,
+ .sotype = SOCK_STREAM,
+ .connect_to = { EXT_IP4, EXT_PORT },
+ .listen_at = { EXT_IP4, EXT_PORT },
+ },
+ {
+ .desc = "TCP IPv6 drop on valid ifindex",
+ .lookup_prog = skel->progs.check_ifindex,
+ .sotype = SOCK_STREAM,
+ .connect_to = { EXT_IP6, EXT_PORT },
+ .listen_at = { EXT_IP6, EXT_PORT },
+ },
+ {
+ .desc = "UDP IPv4 drop on valid ifindex",
+ .lookup_prog = skel->progs.check_ifindex,
+ .sotype = SOCK_DGRAM,
+ .connect_to = { EXT_IP4, EXT_PORT },
+ .listen_at = { EXT_IP4, EXT_PORT },
+ },
+ {
+ .desc = "UDP IPv6 drop on valid ifindex",
+ .lookup_prog = skel->progs.check_ifindex,
+ .sotype = SOCK_DGRAM,
+ .connect_to = { EXT_IP6, EXT_PORT },
+ .listen_at = { EXT_IP6, EXT_PORT },
+ },
};
const struct test *t;
diff --git a/tools/testing/selftests/bpf/prog_tests/skb_ctx.c b/tools/testing/selftests/bpf/prog_tests/skb_ctx.c
index c437e6ba..b5319ba 100644
--- a/tools/testing/selftests/bpf/prog_tests/skb_ctx.c
+++ b/tools/testing/selftests/bpf/prog_tests/skb_ctx.c
@@ -32,7 +32,7 @@ void test_skb_ctx(void)
int err;
int i;
- err = bpf_prog_load("./test_skb_ctx.o", BPF_PROG_TYPE_SCHED_CLS, &obj,
+ err = bpf_prog_test_load("./test_skb_ctx.o", BPF_PROG_TYPE_SCHED_CLS, &obj,
&tattr.prog_fd);
if (CHECK_ATTR(err, "load", "err %d errno %d\n", err, errno))
return;
@@ -111,4 +111,6 @@ void test_skb_ctx(void)
"ctx_out_mark",
"skb->mark == %u, expected %d\n",
skb.mark, 10);
+
+ bpf_object__close(obj);
}
diff --git a/tools/testing/selftests/bpf/prog_tests/skb_helpers.c b/tools/testing/selftests/bpf/prog_tests/skb_helpers.c
index f302ad8..6f802a1 100644
--- a/tools/testing/selftests/bpf/prog_tests/skb_helpers.c
+++ b/tools/testing/selftests/bpf/prog_tests/skb_helpers.c
@@ -20,7 +20,7 @@ void test_skb_helpers(void)
struct bpf_object *obj;
int err;
- err = bpf_prog_load("./test_skb_helpers.o", BPF_PROG_TYPE_SCHED_CLS, &obj,
+ err = bpf_prog_test_load("./test_skb_helpers.o", BPF_PROG_TYPE_SCHED_CLS, &obj,
&tattr.prog_fd);
if (CHECK_ATTR(err, "load", "err %d errno %d\n", err, errno))
return;
diff --git a/tools/testing/selftests/bpf/prog_tests/sock_fields.c b/tools/testing/selftests/bpf/prog_tests/sock_fields.c
index fae40db..9fc040e 100644
--- a/tools/testing/selftests/bpf/prog_tests/sock_fields.c
+++ b/tools/testing/selftests/bpf/prog_tests/sock_fields.c
@@ -15,7 +15,6 @@
#include "network_helpers.h"
#include "cgroup_helpers.h"
#include "test_progs.h"
-#include "bpf_rlimit.h"
#include "test_sock_fields.skel.h"
enum bpf_linum_array_idx {
diff --git a/tools/testing/selftests/bpf/prog_tests/sockmap_basic.c b/tools/testing/selftests/bpf/prog_tests/sockmap_basic.c
index 1352ec1..85db0f4 100644
--- a/tools/testing/selftests/bpf/prog_tests/sockmap_basic.c
+++ b/tools/testing/selftests/bpf/prog_tests/sockmap_basic.c
@@ -91,9 +91,9 @@ static void test_sockmap_create_update_free(enum bpf_map_type map_type)
if (CHECK_FAIL(s < 0))
return;
- map = bpf_create_map(map_type, sizeof(int), sizeof(int), 1, 0);
+ map = bpf_map_create(map_type, NULL, sizeof(int), sizeof(int), 1, NULL);
if (CHECK_FAIL(map < 0)) {
- perror("bpf_create_map");
+ perror("bpf_cmap_create");
goto out;
}
diff --git a/tools/testing/selftests/bpf/prog_tests/sockmap_ktls.c b/tools/testing/selftests/bpf/prog_tests/sockmap_ktls.c
index 7a0d64f..af293ea 100644
--- a/tools/testing/selftests/bpf/prog_tests/sockmap_ktls.c
+++ b/tools/testing/selftests/bpf/prog_tests/sockmap_ktls.c
@@ -97,7 +97,7 @@ static void run_tests(int family, enum bpf_map_type map_type)
char test_name[MAX_TEST_NAME];
int map;
- map = bpf_create_map(map_type, sizeof(int), sizeof(int), 1, 0);
+ map = bpf_map_create(map_type, NULL, sizeof(int), sizeof(int), 1, NULL);
if (CHECK_FAIL(map < 0)) {
perror("bpf_map_create");
return;
diff --git a/tools/testing/selftests/bpf/prog_tests/sockmap_listen.c b/tools/testing/selftests/bpf/prog_tests/sockmap_listen.c
index 2a9cb95..7e21bfa 100644
--- a/tools/testing/selftests/bpf/prog_tests/sockmap_listen.c
+++ b/tools/testing/selftests/bpf/prog_tests/sockmap_listen.c
@@ -502,8 +502,8 @@ static void test_lookup_32_bit_value(int family, int sotype, int mapfd)
if (s < 0)
return;
- mapfd = bpf_create_map(BPF_MAP_TYPE_SOCKMAP, sizeof(key),
- sizeof(value32), 1, 0);
+ mapfd = bpf_map_create(BPF_MAP_TYPE_SOCKMAP, NULL, sizeof(key),
+ sizeof(value32), 1, NULL);
if (mapfd < 0) {
FAIL_ERRNO("map_create");
goto close;
diff --git a/tools/testing/selftests/bpf/prog_tests/sockopt.c b/tools/testing/selftests/bpf/prog_tests/sockopt.c
index 3e8517a8..cd09f4c 100644
--- a/tools/testing/selftests/bpf/prog_tests/sockopt.c
+++ b/tools/testing/selftests/bpf/prog_tests/sockopt.c
@@ -852,22 +852,21 @@ static struct sockopt_test {
static int load_prog(const struct bpf_insn *insns,
enum bpf_attach_type expected_attach_type)
{
- struct bpf_load_program_attr attr = {
- .prog_type = BPF_PROG_TYPE_CGROUP_SOCKOPT,
+ LIBBPF_OPTS(bpf_prog_load_opts, opts,
.expected_attach_type = expected_attach_type,
- .insns = insns,
- .license = "GPL",
.log_level = 2,
- };
- int fd;
+ .log_buf = bpf_log_buf,
+ .log_size = sizeof(bpf_log_buf),
+ );
+ int fd, insns_cnt = 0;
for (;
- insns[attr.insns_cnt].code != (BPF_JMP | BPF_EXIT);
- attr.insns_cnt++) {
+ insns[insns_cnt].code != (BPF_JMP | BPF_EXIT);
+ insns_cnt++) {
}
- attr.insns_cnt++;
+ insns_cnt++;
- fd = bpf_load_program_xattr(&attr, bpf_log_buf, sizeof(bpf_log_buf));
+ fd = bpf_prog_load(BPF_PROG_TYPE_CGROUP_SOCKOPT, NULL, "GPL", insns, insns_cnt, &opts);
if (verbose && fd < 0)
fprintf(stderr, "%s\n", bpf_log_buf);
diff --git a/tools/testing/selftests/bpf/prog_tests/sockopt_inherit.c b/tools/testing/selftests/bpf/prog_tests/sockopt_inherit.c
index 86f9768..8ed78a9 100644
--- a/tools/testing/selftests/bpf/prog_tests/sockopt_inherit.c
+++ b/tools/testing/selftests/bpf/prog_tests/sockopt_inherit.c
@@ -136,7 +136,8 @@ static int start_server(void)
return fd;
}
-static int prog_attach(struct bpf_object *obj, int cgroup_fd, const char *title)
+static int prog_attach(struct bpf_object *obj, int cgroup_fd, const char *title,
+ const char *prog_name)
{
enum bpf_attach_type attach_type;
enum bpf_prog_type prog_type;
@@ -145,20 +146,20 @@ static int prog_attach(struct bpf_object *obj, int cgroup_fd, const char *title)
err = libbpf_prog_type_by_name(title, &prog_type, &attach_type);
if (err) {
- log_err("Failed to deduct types for %s BPF program", title);
+ log_err("Failed to deduct types for %s BPF program", prog_name);
return -1;
}
- prog = bpf_object__find_program_by_title(obj, title);
+ prog = bpf_object__find_program_by_name(obj, prog_name);
if (!prog) {
- log_err("Failed to find %s BPF program", title);
+ log_err("Failed to find %s BPF program", prog_name);
return -1;
}
err = bpf_prog_attach(bpf_program__fd(prog), cgroup_fd,
attach_type, 0);
if (err) {
- log_err("Failed to attach %s BPF program", title);
+ log_err("Failed to attach %s BPF program", prog_name);
return -1;
}
@@ -167,25 +168,25 @@ static int prog_attach(struct bpf_object *obj, int cgroup_fd, const char *title)
static void run_test(int cgroup_fd)
{
- struct bpf_prog_load_attr attr = {
- .file = "./sockopt_inherit.o",
- };
int server_fd = -1, client_fd;
struct bpf_object *obj;
void *server_err;
pthread_t tid;
- int ignored;
int err;
- err = bpf_prog_load_xattr(&attr, &obj, &ignored);
- if (CHECK_FAIL(err))
+ obj = bpf_object__open_file("sockopt_inherit.o", NULL);
+ if (!ASSERT_OK_PTR(obj, "obj_open"))
return;
- err = prog_attach(obj, cgroup_fd, "cgroup/getsockopt");
+ err = bpf_object__load(obj);
+ if (!ASSERT_OK(err, "obj_load"))
+ goto close_bpf_object;
+
+ err = prog_attach(obj, cgroup_fd, "cgroup/getsockopt", "_getsockopt");
if (CHECK_FAIL(err))
goto close_bpf_object;
- err = prog_attach(obj, cgroup_fd, "cgroup/setsockopt");
+ err = prog_attach(obj, cgroup_fd, "cgroup/setsockopt", "_setsockopt");
if (CHECK_FAIL(err))
goto close_bpf_object;
diff --git a/tools/testing/selftests/bpf/prog_tests/sockopt_multi.c b/tools/testing/selftests/bpf/prog_tests/sockopt_multi.c
index bc34f77..abce12d 100644
--- a/tools/testing/selftests/bpf/prog_tests/sockopt_multi.c
+++ b/tools/testing/selftests/bpf/prog_tests/sockopt_multi.c
@@ -297,14 +297,10 @@ static int run_setsockopt_test(struct bpf_object *obj, int cg_parent,
void test_sockopt_multi(void)
{
- struct bpf_prog_load_attr attr = {
- .file = "./sockopt_multi.o",
- };
int cg_parent = -1, cg_child = -1;
struct bpf_object *obj = NULL;
int sock_fd = -1;
int err = -1;
- int ignored;
cg_parent = test__join_cgroup("/parent");
if (CHECK_FAIL(cg_parent < 0))
@@ -314,8 +310,12 @@ void test_sockopt_multi(void)
if (CHECK_FAIL(cg_child < 0))
goto out;
- err = bpf_prog_load_xattr(&attr, &obj, &ignored);
- if (CHECK_FAIL(err))
+ obj = bpf_object__open_file("sockopt_multi.o", NULL);
+ if (!ASSERT_OK_PTR(obj, "obj_load"))
+ goto out;
+
+ err = bpf_object__load(obj);
+ if (!ASSERT_OK(err, "obj_load"))
goto out;
sock_fd = socket(AF_INET, SOCK_STREAM, 0);
diff --git a/tools/testing/selftests/bpf/prog_tests/spinlock.c b/tools/testing/selftests/bpf/prog_tests/spinlock.c
index 7577a77..6307f5d 100644
--- a/tools/testing/selftests/bpf/prog_tests/spinlock.c
+++ b/tools/testing/selftests/bpf/prog_tests/spinlock.c
@@ -24,9 +24,9 @@ void test_spinlock(void)
int err = 0, i;
void *ret;
- err = bpf_prog_load(file, BPF_PROG_TYPE_CGROUP_SKB, &obj, &prog_fd);
+ err = bpf_prog_test_load(file, BPF_PROG_TYPE_CGROUP_SKB, &obj, &prog_fd);
if (CHECK_FAIL(err)) {
- printf("test_spin_lock:bpf_prog_load errno %d\n", errno);
+ printf("test_spin_lock:bpf_prog_test_load errno %d\n", errno);
goto close_prog;
}
for (i = 0; i < 4; i++)
diff --git a/tools/testing/selftests/bpf/prog_tests/stacktrace_map.c b/tools/testing/selftests/bpf/prog_tests/stacktrace_map.c
index 04b476b..313f0a6 100644
--- a/tools/testing/selftests/bpf/prog_tests/stacktrace_map.c
+++ b/tools/testing/selftests/bpf/prog_tests/stacktrace_map.c
@@ -4,7 +4,7 @@
void test_stacktrace_map(void)
{
int control_map_fd, stackid_hmap_fd, stackmap_fd, stack_amap_fd;
- const char *prog_name = "tracepoint/sched/sched_switch";
+ const char *prog_name = "oncpu";
int err, prog_fd, stack_trace_len;
const char *file = "./test_stacktrace_map.o";
__u32 key, val, duration = 0;
@@ -12,11 +12,11 @@ void test_stacktrace_map(void)
struct bpf_object *obj;
struct bpf_link *link;
- err = bpf_prog_load(file, BPF_PROG_TYPE_TRACEPOINT, &obj, &prog_fd);
+ err = bpf_prog_test_load(file, BPF_PROG_TYPE_TRACEPOINT, &obj, &prog_fd);
if (CHECK(err, "prog_load", "err %d errno %d\n", err, errno))
return;
- prog = bpf_object__find_program_by_title(obj, prog_name);
+ prog = bpf_object__find_program_by_name(obj, prog_name);
if (CHECK(!prog, "find_prog", "prog '%s' not found\n", prog_name))
goto close_prog;
diff --git a/tools/testing/selftests/bpf/prog_tests/stacktrace_map_raw_tp.c b/tools/testing/selftests/bpf/prog_tests/stacktrace_map_raw_tp.c
index 4fd30bb..1cb8dd3 100644
--- a/tools/testing/selftests/bpf/prog_tests/stacktrace_map_raw_tp.c
+++ b/tools/testing/selftests/bpf/prog_tests/stacktrace_map_raw_tp.c
@@ -3,7 +3,7 @@
void test_stacktrace_map_raw_tp(void)
{
- const char *prog_name = "tracepoint/sched/sched_switch";
+ const char *prog_name = "oncpu";
int control_map_fd, stackid_hmap_fd, stackmap_fd;
const char *file = "./test_stacktrace_map.o";
__u32 key, val, duration = 0;
@@ -12,11 +12,11 @@ void test_stacktrace_map_raw_tp(void)
struct bpf_object *obj;
struct bpf_link *link = NULL;
- err = bpf_prog_load(file, BPF_PROG_TYPE_RAW_TRACEPOINT, &obj, &prog_fd);
+ err = bpf_prog_test_load(file, BPF_PROG_TYPE_RAW_TRACEPOINT, &obj, &prog_fd);
if (CHECK(err, "prog_load raw tp", "err %d errno %d\n", err, errno))
return;
- prog = bpf_object__find_program_by_title(obj, prog_name);
+ prog = bpf_object__find_program_by_name(obj, prog_name);
if (CHECK(!prog, "find_prog", "prog '%s' not found\n", prog_name))
goto close_prog;
diff --git a/tools/testing/selftests/bpf/prog_tests/tailcalls.c b/tools/testing/selftests/bpf/prog_tests/tailcalls.c
index 9825f1f..5dc0f42 100644
--- a/tools/testing/selftests/bpf/prog_tests/tailcalls.c
+++ b/tools/testing/selftests/bpf/prog_tests/tailcalls.c
@@ -16,7 +16,7 @@ static void test_tailcall_1(void)
char prog_name[32];
char buff[128] = {};
- err = bpf_prog_load("tailcall1.o", BPF_PROG_TYPE_SCHED_CLS, &obj,
+ err = bpf_prog_test_load("tailcall1.o", BPF_PROG_TYPE_SCHED_CLS, &obj,
&prog_fd);
if (CHECK_FAIL(err))
return;
@@ -154,7 +154,7 @@ static void test_tailcall_2(void)
char prog_name[32];
char buff[128] = {};
- err = bpf_prog_load("tailcall2.o", BPF_PROG_TYPE_SCHED_CLS, &obj,
+ err = bpf_prog_test_load("tailcall2.o", BPF_PROG_TYPE_SCHED_CLS, &obj,
&prog_fd);
if (CHECK_FAIL(err))
return;
@@ -228,7 +228,7 @@ static void test_tailcall_count(const char *which)
__u32 retval, duration;
char buff[128] = {};
- err = bpf_prog_load(which, BPF_PROG_TYPE_SCHED_CLS, &obj,
+ err = bpf_prog_test_load(which, BPF_PROG_TYPE_SCHED_CLS, &obj,
&prog_fd);
if (CHECK_FAIL(err))
return;
@@ -324,7 +324,7 @@ static void test_tailcall_4(void)
char buff[128] = {};
char prog_name[32];
- err = bpf_prog_load("tailcall4.o", BPF_PROG_TYPE_SCHED_CLS, &obj,
+ err = bpf_prog_test_load("tailcall4.o", BPF_PROG_TYPE_SCHED_CLS, &obj,
&prog_fd);
if (CHECK_FAIL(err))
return;
@@ -412,7 +412,7 @@ static void test_tailcall_5(void)
char buff[128] = {};
char prog_name[32];
- err = bpf_prog_load("tailcall5.o", BPF_PROG_TYPE_SCHED_CLS, &obj,
+ err = bpf_prog_test_load("tailcall5.o", BPF_PROG_TYPE_SCHED_CLS, &obj,
&prog_fd);
if (CHECK_FAIL(err))
return;
@@ -498,7 +498,7 @@ static void test_tailcall_bpf2bpf_1(void)
__u32 retval, duration;
char prog_name[32];
- err = bpf_prog_load("tailcall_bpf2bpf1.o", BPF_PROG_TYPE_SCHED_CLS,
+ err = bpf_prog_test_load("tailcall_bpf2bpf1.o", BPF_PROG_TYPE_SCHED_CLS,
&obj, &prog_fd);
if (CHECK_FAIL(err))
return;
@@ -582,7 +582,7 @@ static void test_tailcall_bpf2bpf_2(void)
__u32 retval, duration;
char buff[128] = {};
- err = bpf_prog_load("tailcall_bpf2bpf2.o", BPF_PROG_TYPE_SCHED_CLS,
+ err = bpf_prog_test_load("tailcall_bpf2bpf2.o", BPF_PROG_TYPE_SCHED_CLS,
&obj, &prog_fd);
if (CHECK_FAIL(err))
return;
@@ -660,7 +660,7 @@ static void test_tailcall_bpf2bpf_3(void)
__u32 retval, duration;
char prog_name[32];
- err = bpf_prog_load("tailcall_bpf2bpf3.o", BPF_PROG_TYPE_SCHED_CLS,
+ err = bpf_prog_test_load("tailcall_bpf2bpf3.o", BPF_PROG_TYPE_SCHED_CLS,
&obj, &prog_fd);
if (CHECK_FAIL(err))
return;
@@ -757,7 +757,7 @@ static void test_tailcall_bpf2bpf_4(bool noise)
__u32 retval, duration;
char prog_name[32];
- err = bpf_prog_load("tailcall_bpf2bpf4.o", BPF_PROG_TYPE_SCHED_CLS,
+ err = bpf_prog_test_load("tailcall_bpf2bpf4.o", BPF_PROG_TYPE_SCHED_CLS,
&obj, &prog_fd);
if (CHECK_FAIL(err))
return;
diff --git a/tools/testing/selftests/bpf/prog_tests/task_fd_query_rawtp.c b/tools/testing/selftests/bpf/prog_tests/task_fd_query_rawtp.c
index 1bdc1d8..17947c9 100644
--- a/tools/testing/selftests/bpf/prog_tests/task_fd_query_rawtp.c
+++ b/tools/testing/selftests/bpf/prog_tests/task_fd_query_rawtp.c
@@ -11,7 +11,7 @@ void test_task_fd_query_rawtp(void)
__u32 duration = 0;
char buf[256];
- err = bpf_prog_load(file, BPF_PROG_TYPE_RAW_TRACEPOINT, &obj, &prog_fd);
+ err = bpf_prog_test_load(file, BPF_PROG_TYPE_RAW_TRACEPOINT, &obj, &prog_fd);
if (CHECK(err, "prog_load raw tp", "err %d errno %d\n", err, errno))
return;
diff --git a/tools/testing/selftests/bpf/prog_tests/task_fd_query_tp.c b/tools/testing/selftests/bpf/prog_tests/task_fd_query_tp.c
index 3f131b8..c2a98a7 100644
--- a/tools/testing/selftests/bpf/prog_tests/task_fd_query_tp.c
+++ b/tools/testing/selftests/bpf/prog_tests/task_fd_query_tp.c
@@ -13,8 +13,8 @@ static void test_task_fd_query_tp_core(const char *probe_name,
__u32 duration = 0;
char buf[256];
- err = bpf_prog_load(file, BPF_PROG_TYPE_TRACEPOINT, &obj, &prog_fd);
- if (CHECK(err, "bpf_prog_load", "err %d errno %d\n", err, errno))
+ err = bpf_prog_test_load(file, BPF_PROG_TYPE_TRACEPOINT, &obj, &prog_fd);
+ if (CHECK(err, "bpf_prog_test_load", "err %d errno %d\n", err, errno))
goto close_prog;
snprintf(buf, sizeof(buf),
diff --git a/tools/testing/selftests/bpf/prog_tests/tc_redirect.c b/tools/testing/selftests/bpf/prog_tests/tc_redirect.c
index 4b18b73..c2426df 100644
--- a/tools/testing/selftests/bpf/prog_tests/tc_redirect.c
+++ b/tools/testing/selftests/bpf/prog_tests/tc_redirect.c
@@ -105,6 +105,13 @@ static int setns_by_fd(int nsfd)
if (!ASSERT_OK(err, "unshare"))
return err;
+ /* Make our /sys mount private, so the following umount won't
+ * trigger the global umount in case it's shared.
+ */
+ err = mount("none", "/sys", NULL, MS_PRIVATE, NULL);
+ if (!ASSERT_OK(err, "remount private /sys"))
+ return err;
+
err = umount2("/sys", MNT_DETACH);
if (!ASSERT_OK(err, "umount2 /sys"))
return err;
diff --git a/tools/testing/selftests/bpf/prog_tests/tcp_estats.c b/tools/testing/selftests/bpf/prog_tests/tcp_estats.c
index 594307d..11bf755 100644
--- a/tools/testing/selftests/bpf/prog_tests/tcp_estats.c
+++ b/tools/testing/selftests/bpf/prog_tests/tcp_estats.c
@@ -8,7 +8,7 @@ void test_tcp_estats(void)
struct bpf_object *obj;
__u32 duration = 0;
- err = bpf_prog_load(file, BPF_PROG_TYPE_TRACEPOINT, &obj, &prog_fd);
+ err = bpf_prog_test_load(file, BPF_PROG_TYPE_TRACEPOINT, &obj, &prog_fd);
CHECK(err, "", "err %d errno %d\n", err, errno);
if (err)
return;
diff --git a/tools/testing/selftests/bpf/prog_tests/tcp_rtt.c b/tools/testing/selftests/bpf/prog_tests/tcp_rtt.c
index 265b4fe..96ff2c2 100644
--- a/tools/testing/selftests/bpf/prog_tests/tcp_rtt.c
+++ b/tools/testing/selftests/bpf/prog_tests/tcp_rtt.c
@@ -2,6 +2,7 @@
#include <test_progs.h>
#include "cgroup_helpers.h"
#include "network_helpers.h"
+#include "tcp_rtt.skel.h"
struct tcp_rtt_storage {
__u32 invoked;
@@ -91,26 +92,18 @@ static int verify_sk(int map_fd, int client_fd, const char *msg, __u32 invoked,
static int run_test(int cgroup_fd, int server_fd)
{
- struct bpf_prog_load_attr attr = {
- .prog_type = BPF_PROG_TYPE_SOCK_OPS,
- .file = "./tcp_rtt.o",
- .expected_attach_type = BPF_CGROUP_SOCK_OPS,
- };
- struct bpf_object *obj;
- struct bpf_map *map;
+ struct tcp_rtt *skel;
int client_fd;
int prog_fd;
int map_fd;
int err;
- err = bpf_prog_load_xattr(&attr, &obj, &prog_fd);
- if (err) {
- log_err("Failed to load BPF object");
+ skel = tcp_rtt__open_and_load();
+ if (!ASSERT_OK_PTR(skel, "skel_open_load"))
return -1;
- }
- map = bpf_object__next_map(obj, NULL);
- map_fd = bpf_map__fd(map);
+ map_fd = bpf_map__fd(skel->maps.socket_storage_map);
+ prog_fd = bpf_program__fd(skel->progs._sockops);
err = bpf_prog_attach(prog_fd, cgroup_fd, BPF_CGROUP_SOCK_OPS, 0);
if (err) {
@@ -149,7 +142,7 @@ static int run_test(int cgroup_fd, int server_fd)
close(client_fd);
close_bpf_object:
- bpf_object__close(obj);
+ tcp_rtt__destroy(skel);
return err;
}
diff --git a/tools/testing/selftests/bpf/prog_tests/test_bpffs.c b/tools/testing/selftests/bpf/prog_tests/test_bpffs.c
index d29ebfe..214d9f4 100644
--- a/tools/testing/selftests/bpf/prog_tests/test_bpffs.c
+++ b/tools/testing/selftests/bpf/prog_tests/test_bpffs.c
@@ -19,11 +19,13 @@ static int read_iter(char *file)
fd = open(file, 0);
if (fd < 0)
return -1;
- while ((len = read(fd, buf, sizeof(buf))) > 0)
+ while ((len = read(fd, buf, sizeof(buf))) > 0) {
+ buf[sizeof(buf) - 1] = '\0';
if (strstr(buf, "iter")) {
close(fd);
return 0;
}
+ }
close(fd);
return -1;
}
@@ -80,7 +82,7 @@ static int fn(void)
if (!ASSERT_OK(err, "creating " TDIR "/fs1/b"))
goto out;
- map = bpf_create_map(BPF_MAP_TYPE_ARRAY, 4, 4, 1, 0);
+ map = bpf_map_create(BPF_MAP_TYPE_ARRAY, NULL, 4, 4, 1, NULL);
if (!ASSERT_GT(map, 0, "create_map(ARRAY)"))
goto out;
err = bpf_obj_pin(map, TDIR "/fs1/c");
diff --git a/tools/testing/selftests/bpf/prog_tests/test_global_funcs.c b/tools/testing/selftests/bpf/prog_tests/test_global_funcs.c
index 7e13129..509e21d 100644
--- a/tools/testing/selftests/bpf/prog_tests/test_global_funcs.c
+++ b/tools/testing/selftests/bpf/prog_tests/test_global_funcs.c
@@ -30,17 +30,29 @@ extern int extra_prog_load_log_flags;
static int check_load(const char *file)
{
- struct bpf_prog_load_attr attr;
struct bpf_object *obj = NULL;
- int err, prog_fd;
+ struct bpf_program *prog;
+ int err;
- memset(&attr, 0, sizeof(struct bpf_prog_load_attr));
- attr.file = file;
- attr.prog_type = BPF_PROG_TYPE_UNSPEC;
- attr.log_level = extra_prog_load_log_flags;
- attr.prog_flags = BPF_F_TEST_RND_HI32;
found = false;
- err = bpf_prog_load_xattr(&attr, &obj, &prog_fd);
+
+ obj = bpf_object__open_file(file, NULL);
+ err = libbpf_get_error(obj);
+ if (err)
+ return err;
+
+ prog = bpf_object__next_program(obj, NULL);
+ if (!prog) {
+ err = -ENOENT;
+ goto err_out;
+ }
+
+ bpf_program__set_flags(prog, BPF_F_TEST_RND_HI32);
+ bpf_program__set_log_level(prog, extra_prog_load_log_flags);
+
+ err = bpf_object__load(obj);
+
+err_out:
bpf_object__close(obj);
return err;
}
diff --git a/tools/testing/selftests/bpf/prog_tests/test_local_storage.c b/tools/testing/selftests/bpf/prog_tests/test_local_storage.c
index d2c16ea..26ac26a 100644
--- a/tools/testing/selftests/bpf/prog_tests/test_local_storage.c
+++ b/tools/testing/selftests/bpf/prog_tests/test_local_storage.c
@@ -28,10 +28,6 @@ static unsigned int duration;
struct storage {
void *inode;
unsigned int value;
- /* Lock ensures that spin locked versions of local stoage operations
- * also work, most operations in this tests are still single threaded
- */
- struct bpf_spin_lock lock;
};
/* Fork and exec the provided rm binary and return the exit code of the
@@ -66,27 +62,24 @@ static int run_self_unlink(int *monitored_pid, const char *rm_path)
static bool check_syscall_operations(int map_fd, int obj_fd)
{
- struct storage val = { .value = TEST_STORAGE_VALUE, .lock = { 0 } },
- lookup_val = { .value = 0, .lock = { 0 } };
+ struct storage val = { .value = TEST_STORAGE_VALUE },
+ lookup_val = { .value = 0 };
int err;
/* Looking up an existing element should fail initially */
- err = bpf_map_lookup_elem_flags(map_fd, &obj_fd, &lookup_val,
- BPF_F_LOCK);
+ err = bpf_map_lookup_elem_flags(map_fd, &obj_fd, &lookup_val, 0);
if (CHECK(!err || errno != ENOENT, "bpf_map_lookup_elem",
"err:%d errno:%d\n", err, errno))
return false;
/* Create a new element */
- err = bpf_map_update_elem(map_fd, &obj_fd, &val,
- BPF_NOEXIST | BPF_F_LOCK);
+ err = bpf_map_update_elem(map_fd, &obj_fd, &val, BPF_NOEXIST);
if (CHECK(err < 0, "bpf_map_update_elem", "err:%d errno:%d\n", err,
errno))
return false;
/* Lookup the newly created element */
- err = bpf_map_lookup_elem_flags(map_fd, &obj_fd, &lookup_val,
- BPF_F_LOCK);
+ err = bpf_map_lookup_elem_flags(map_fd, &obj_fd, &lookup_val, 0);
if (CHECK(err < 0, "bpf_map_lookup_elem", "err:%d errno:%d", err,
errno))
return false;
@@ -102,8 +95,7 @@ static bool check_syscall_operations(int map_fd, int obj_fd)
return false;
/* The lookup should fail, now that the element has been deleted */
- err = bpf_map_lookup_elem_flags(map_fd, &obj_fd, &lookup_val,
- BPF_F_LOCK);
+ err = bpf_map_lookup_elem_flags(map_fd, &obj_fd, &lookup_val, 0);
if (CHECK(!err || errno != ENOENT, "bpf_map_lookup_elem",
"err:%d errno:%d\n", err, errno))
return false;
diff --git a/tools/testing/selftests/bpf/prog_tests/test_overhead.c b/tools/testing/selftests/bpf/prog_tests/test_overhead.c
index 123c68c..05acb37 100644
--- a/tools/testing/selftests/bpf/prog_tests/test_overhead.c
+++ b/tools/testing/selftests/bpf/prog_tests/test_overhead.c
@@ -56,11 +56,11 @@ static void setaffinity(void)
void test_test_overhead(void)
{
- const char *kprobe_name = "kprobe/__set_task_comm";
- const char *kretprobe_name = "kretprobe/__set_task_comm";
- const char *raw_tp_name = "raw_tp/task_rename";
- const char *fentry_name = "fentry/__set_task_comm";
- const char *fexit_name = "fexit/__set_task_comm";
+ const char *kprobe_name = "prog1";
+ const char *kretprobe_name = "prog2";
+ const char *raw_tp_name = "prog3";
+ const char *fentry_name = "prog4";
+ const char *fexit_name = "prog5";
const char *kprobe_func = "__set_task_comm";
struct bpf_program *kprobe_prog, *kretprobe_prog, *raw_tp_prog;
struct bpf_program *fentry_prog, *fexit_prog;
@@ -76,23 +76,23 @@ void test_test_overhead(void)
if (!ASSERT_OK_PTR(obj, "obj_open_file"))
return;
- kprobe_prog = bpf_object__find_program_by_title(obj, kprobe_name);
+ kprobe_prog = bpf_object__find_program_by_name(obj, kprobe_name);
if (CHECK(!kprobe_prog, "find_probe",
"prog '%s' not found\n", kprobe_name))
goto cleanup;
- kretprobe_prog = bpf_object__find_program_by_title(obj, kretprobe_name);
+ kretprobe_prog = bpf_object__find_program_by_name(obj, kretprobe_name);
if (CHECK(!kretprobe_prog, "find_probe",
"prog '%s' not found\n", kretprobe_name))
goto cleanup;
- raw_tp_prog = bpf_object__find_program_by_title(obj, raw_tp_name);
+ raw_tp_prog = bpf_object__find_program_by_name(obj, raw_tp_name);
if (CHECK(!raw_tp_prog, "find_probe",
"prog '%s' not found\n", raw_tp_name))
goto cleanup;
- fentry_prog = bpf_object__find_program_by_title(obj, fentry_name);
+ fentry_prog = bpf_object__find_program_by_name(obj, fentry_name);
if (CHECK(!fentry_prog, "find_probe",
"prog '%s' not found\n", fentry_name))
goto cleanup;
- fexit_prog = bpf_object__find_program_by_title(obj, fexit_name);
+ fexit_prog = bpf_object__find_program_by_name(obj, fexit_name);
if (CHECK(!fexit_prog, "find_probe",
"prog '%s' not found\n", fexit_name))
goto cleanup;
diff --git a/tools/testing/selftests/bpf/prog_tests/test_strncmp.c b/tools/testing/selftests/bpf/prog_tests/test_strncmp.c
new file mode 100644
index 0000000..b57a300
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/test_strncmp.c
@@ -0,0 +1,167 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2021. Huawei Technologies Co., Ltd */
+#include <test_progs.h>
+#include "strncmp_test.skel.h"
+
+static int trigger_strncmp(const struct strncmp_test *skel)
+{
+ int cmp;
+
+ usleep(1);
+
+ cmp = skel->bss->cmp_ret;
+ if (cmp > 0)
+ return 1;
+ if (cmp < 0)
+ return -1;
+ return 0;
+}
+
+/*
+ * Compare str and target after making str[i] != target[i].
+ * When exp is -1, make str[i] < target[i] and delta = -1.
+ */
+static void strncmp_full_str_cmp(struct strncmp_test *skel, const char *name,
+ int exp)
+{
+ size_t nr = sizeof(skel->bss->str);
+ char *str = skel->bss->str;
+ int delta = exp;
+ int got;
+ size_t i;
+
+ memcpy(str, skel->rodata->target, nr);
+ for (i = 0; i < nr - 1; i++) {
+ str[i] += delta;
+
+ got = trigger_strncmp(skel);
+ ASSERT_EQ(got, exp, name);
+
+ str[i] -= delta;
+ }
+}
+
+static void test_strncmp_ret(void)
+{
+ struct strncmp_test *skel;
+ struct bpf_program *prog;
+ int err, got;
+
+ skel = strncmp_test__open();
+ if (!ASSERT_OK_PTR(skel, "strncmp_test open"))
+ return;
+
+ bpf_object__for_each_program(prog, skel->obj)
+ bpf_program__set_autoload(prog, false);
+
+ bpf_program__set_autoload(skel->progs.do_strncmp, true);
+
+ err = strncmp_test__load(skel);
+ if (!ASSERT_EQ(err, 0, "strncmp_test load"))
+ goto out;
+
+ err = strncmp_test__attach(skel);
+ if (!ASSERT_EQ(err, 0, "strncmp_test attach"))
+ goto out;
+
+ skel->bss->target_pid = getpid();
+
+ /* Empty str */
+ skel->bss->str[0] = '\0';
+ got = trigger_strncmp(skel);
+ ASSERT_EQ(got, -1, "strncmp: empty str");
+
+ /* Same string */
+ memcpy(skel->bss->str, skel->rodata->target, sizeof(skel->bss->str));
+ got = trigger_strncmp(skel);
+ ASSERT_EQ(got, 0, "strncmp: same str");
+
+ /* Not-null-termainted string */
+ memcpy(skel->bss->str, skel->rodata->target, sizeof(skel->bss->str));
+ skel->bss->str[sizeof(skel->bss->str) - 1] = 'A';
+ got = trigger_strncmp(skel);
+ ASSERT_EQ(got, 1, "strncmp: not-null-term str");
+
+ strncmp_full_str_cmp(skel, "strncmp: less than", -1);
+ strncmp_full_str_cmp(skel, "strncmp: greater than", 1);
+out:
+ strncmp_test__destroy(skel);
+}
+
+static void test_strncmp_bad_not_const_str_size(void)
+{
+ struct strncmp_test *skel;
+ struct bpf_program *prog;
+ int err;
+
+ skel = strncmp_test__open();
+ if (!ASSERT_OK_PTR(skel, "strncmp_test open"))
+ return;
+
+ bpf_object__for_each_program(prog, skel->obj)
+ bpf_program__set_autoload(prog, false);
+
+ bpf_program__set_autoload(skel->progs.strncmp_bad_not_const_str_size,
+ true);
+
+ err = strncmp_test__load(skel);
+ ASSERT_ERR(err, "strncmp_test load bad_not_const_str_size");
+
+ strncmp_test__destroy(skel);
+}
+
+static void test_strncmp_bad_writable_target(void)
+{
+ struct strncmp_test *skel;
+ struct bpf_program *prog;
+ int err;
+
+ skel = strncmp_test__open();
+ if (!ASSERT_OK_PTR(skel, "strncmp_test open"))
+ return;
+
+ bpf_object__for_each_program(prog, skel->obj)
+ bpf_program__set_autoload(prog, false);
+
+ bpf_program__set_autoload(skel->progs.strncmp_bad_writable_target,
+ true);
+
+ err = strncmp_test__load(skel);
+ ASSERT_ERR(err, "strncmp_test load bad_writable_target");
+
+ strncmp_test__destroy(skel);
+}
+
+static void test_strncmp_bad_not_null_term_target(void)
+{
+ struct strncmp_test *skel;
+ struct bpf_program *prog;
+ int err;
+
+ skel = strncmp_test__open();
+ if (!ASSERT_OK_PTR(skel, "strncmp_test open"))
+ return;
+
+ bpf_object__for_each_program(prog, skel->obj)
+ bpf_program__set_autoload(prog, false);
+
+ bpf_program__set_autoload(skel->progs.strncmp_bad_not_null_term_target,
+ true);
+
+ err = strncmp_test__load(skel);
+ ASSERT_ERR(err, "strncmp_test load bad_not_null_term_target");
+
+ strncmp_test__destroy(skel);
+}
+
+void test_test_strncmp(void)
+{
+ if (test__start_subtest("strncmp_ret"))
+ test_strncmp_ret();
+ if (test__start_subtest("strncmp_bad_not_const_str_size"))
+ test_strncmp_bad_not_const_str_size();
+ if (test__start_subtest("strncmp_bad_writable_target"))
+ test_strncmp_bad_writable_target();
+ if (test__start_subtest("strncmp_bad_not_null_term_target"))
+ test_strncmp_bad_not_null_term_target();
+}
diff --git a/tools/testing/selftests/bpf/prog_tests/tp_attach_query.c b/tools/testing/selftests/bpf/prog_tests/tp_attach_query.c
index 8652d0a..39e7929 100644
--- a/tools/testing/selftests/bpf/prog_tests/tp_attach_query.c
+++ b/tools/testing/selftests/bpf/prog_tests/tp_attach_query.c
@@ -35,7 +35,7 @@ void serial_test_tp_attach_query(void)
query = malloc(sizeof(*query) + sizeof(__u32) * num_progs);
for (i = 0; i < num_progs; i++) {
- err = bpf_prog_load(file, BPF_PROG_TYPE_TRACEPOINT, &obj[i],
+ err = bpf_prog_test_load(file, BPF_PROG_TYPE_TRACEPOINT, &obj[i],
&prog_fd[i]);
if (CHECK(err, "prog_load", "err %d errno %d\n", err, errno))
goto cleanup1;
diff --git a/tools/testing/selftests/bpf/prog_tests/trampoline_count.c b/tools/testing/selftests/bpf/prog_tests/trampoline_count.c
index fc14667..9c795ee 100644
--- a/tools/testing/selftests/bpf/prog_tests/trampoline_count.c
+++ b/tools/testing/selftests/bpf/prog_tests/trampoline_count.c
@@ -35,7 +35,7 @@ static struct bpf_link *load(struct bpf_object *obj, const char *name)
struct bpf_program *prog;
int duration = 0;
- prog = bpf_object__find_program_by_title(obj, name);
+ prog = bpf_object__find_program_by_name(obj, name);
if (CHECK(!prog, "find_probe", "prog '%s' not found\n", name))
return ERR_PTR(-EINVAL);
return bpf_program__attach_trace(prog);
@@ -44,8 +44,8 @@ static struct bpf_link *load(struct bpf_object *obj, const char *name)
/* TODO: use different target function to run in concurrent mode */
void serial_test_trampoline_count(void)
{
- const char *fentry_name = "fentry/__set_task_comm";
- const char *fexit_name = "fexit/__set_task_comm";
+ const char *fentry_name = "prog1";
+ const char *fexit_name = "prog2";
const char *object = "test_trampoline_count.o";
struct inst inst[MAX_TRAMP_PROGS] = {};
int err, i = 0, duration = 0;
diff --git a/tools/testing/selftests/bpf/prog_tests/xdp.c b/tools/testing/selftests/bpf/prog_tests/xdp.c
index 48921ff..ac65456 100644
--- a/tools/testing/selftests/bpf/prog_tests/xdp.c
+++ b/tools/testing/selftests/bpf/prog_tests/xdp.c
@@ -11,12 +11,12 @@ void test_xdp(void)
const char *file = "./test_xdp.o";
struct bpf_object *obj;
char buf[128];
- struct ipv6hdr *iph6 = (void *)buf + sizeof(struct ethhdr);
- struct iphdr *iph = (void *)buf + sizeof(struct ethhdr);
+ struct ipv6hdr iph6;
+ struct iphdr iph;
__u32 duration, retval, size;
int err, prog_fd, map_fd;
- err = bpf_prog_load(file, BPF_PROG_TYPE_XDP, &obj, &prog_fd);
+ err = bpf_prog_test_load(file, BPF_PROG_TYPE_XDP, &obj, &prog_fd);
if (CHECK_FAIL(err))
return;
@@ -28,16 +28,17 @@ void test_xdp(void)
err = bpf_prog_test_run(prog_fd, 1, &pkt_v4, sizeof(pkt_v4),
buf, &size, &retval, &duration);
-
+ memcpy(&iph, buf + sizeof(struct ethhdr), sizeof(iph));
CHECK(err || retval != XDP_TX || size != 74 ||
- iph->protocol != IPPROTO_IPIP, "ipv4",
+ iph.protocol != IPPROTO_IPIP, "ipv4",
"err %d errno %d retval %d size %d\n",
err, errno, retval, size);
err = bpf_prog_test_run(prog_fd, 1, &pkt_v6, sizeof(pkt_v6),
buf, &size, &retval, &duration);
+ memcpy(&iph6, buf + sizeof(struct ethhdr), sizeof(iph6));
CHECK(err || retval != XDP_TX || size != 114 ||
- iph6->nexthdr != IPPROTO_IPV6, "ipv6",
+ iph6.nexthdr != IPPROTO_IPV6, "ipv6",
"err %d errno %d retval %d size %d\n",
err, errno, retval, size);
out:
diff --git a/tools/testing/selftests/bpf/prog_tests/xdp_adjust_tail.c b/tools/testing/selftests/bpf/prog_tests/xdp_adjust_tail.c
index f529e3c..3f5a17c 100644
--- a/tools/testing/selftests/bpf/prog_tests/xdp_adjust_tail.c
+++ b/tools/testing/selftests/bpf/prog_tests/xdp_adjust_tail.c
@@ -10,7 +10,7 @@ static void test_xdp_adjust_tail_shrink(void)
int err, prog_fd;
char buf[128];
- err = bpf_prog_load(file, BPF_PROG_TYPE_XDP, &obj, &prog_fd);
+ err = bpf_prog_test_load(file, BPF_PROG_TYPE_XDP, &obj, &prog_fd);
if (CHECK_FAIL(err))
return;
@@ -38,7 +38,7 @@ static void test_xdp_adjust_tail_grow(void)
__u32 duration, retval, size, expect_sz;
int err, prog_fd;
- err = bpf_prog_load(file, BPF_PROG_TYPE_XDP, &obj, &prog_fd);
+ err = bpf_prog_test_load(file, BPF_PROG_TYPE_XDP, &obj, &prog_fd);
if (CHECK_FAIL(err))
return;
@@ -75,7 +75,7 @@ static void test_xdp_adjust_tail_grow2(void)
.data_size_out = 0, /* Per test */
};
- err = bpf_prog_load(file, BPF_PROG_TYPE_XDP, &obj, &tattr.prog_fd);
+ err = bpf_prog_test_load(file, BPF_PROG_TYPE_XDP, &obj, &tattr.prog_fd);
if (CHECK_ATTR(err, "load", "err %d errno %d\n", err, errno))
return;
diff --git a/tools/testing/selftests/bpf/prog_tests/xdp_attach.c b/tools/testing/selftests/bpf/prog_tests/xdp_attach.c
index 4c40572..c6fa390 100644
--- a/tools/testing/selftests/bpf/prog_tests/xdp_attach.c
+++ b/tools/testing/selftests/bpf/prog_tests/xdp_attach.c
@@ -16,7 +16,7 @@ void serial_test_xdp_attach(void)
len = sizeof(info);
- err = bpf_prog_load(file, BPF_PROG_TYPE_XDP, &obj1, &fd1);
+ err = bpf_prog_test_load(file, BPF_PROG_TYPE_XDP, &obj1, &fd1);
if (CHECK_FAIL(err))
return;
err = bpf_obj_get_info_by_fd(fd1, &info, &len);
@@ -24,7 +24,7 @@ void serial_test_xdp_attach(void)
goto out_1;
id1 = info.id;
- err = bpf_prog_load(file, BPF_PROG_TYPE_XDP, &obj2, &fd2);
+ err = bpf_prog_test_load(file, BPF_PROG_TYPE_XDP, &obj2, &fd2);
if (CHECK_FAIL(err))
goto out_1;
@@ -34,7 +34,7 @@ void serial_test_xdp_attach(void)
goto out_2;
id2 = info.id;
- err = bpf_prog_load(file, BPF_PROG_TYPE_XDP, &obj3, &fd3);
+ err = bpf_prog_test_load(file, BPF_PROG_TYPE_XDP, &obj3, &fd3);
if (CHECK_FAIL(err))
goto out_2;
diff --git a/tools/testing/selftests/bpf/prog_tests/xdp_bonding.c b/tools/testing/selftests/bpf/prog_tests/xdp_bonding.c
index faa22b8..5e3a26b 100644
--- a/tools/testing/selftests/bpf/prog_tests/xdp_bonding.c
+++ b/tools/testing/selftests/bpf/prog_tests/xdp_bonding.c
@@ -218,9 +218,9 @@ static int send_udp_packets(int vary_dst_ip)
.h_dest = BOND2_MAC,
.h_proto = htons(ETH_P_IP),
};
- uint8_t buf[128] = {};
- struct iphdr *iph = (struct iphdr *)(buf + sizeof(eh));
- struct udphdr *uh = (struct udphdr *)(buf + sizeof(eh) + sizeof(*iph));
+ struct iphdr iph = {};
+ struct udphdr uh = {};
+ uint8_t buf[128];
int i, s = -1;
int ifindex;
@@ -232,17 +232,16 @@ static int send_udp_packets(int vary_dst_ip)
if (!ASSERT_GT(ifindex, 0, "get bond1 ifindex"))
goto err;
- memcpy(buf, &eh, sizeof(eh));
- iph->ihl = 5;
- iph->version = 4;
- iph->tos = 16;
- iph->id = 1;
- iph->ttl = 64;
- iph->protocol = IPPROTO_UDP;
- iph->saddr = 1;
- iph->daddr = 2;
- iph->tot_len = htons(sizeof(buf) - ETH_HLEN);
- iph->check = 0;
+ iph.ihl = 5;
+ iph.version = 4;
+ iph.tos = 16;
+ iph.id = 1;
+ iph.ttl = 64;
+ iph.protocol = IPPROTO_UDP;
+ iph.saddr = 1;
+ iph.daddr = 2;
+ iph.tot_len = htons(sizeof(buf) - ETH_HLEN);
+ iph.check = 0;
for (i = 1; i <= NPACKETS; i++) {
int n;
@@ -253,10 +252,15 @@ static int send_udp_packets(int vary_dst_ip)
};
/* vary the UDP destination port for even distribution with roundrobin/xor modes */
- uh->dest++;
+ uh.dest++;
if (vary_dst_ip)
- iph->daddr++;
+ iph.daddr++;
+
+ /* construct a packet */
+ memcpy(buf, &eh, sizeof(eh));
+ memcpy(buf + sizeof(eh), &iph, sizeof(iph));
+ memcpy(buf + sizeof(eh) + sizeof(iph), &uh, sizeof(uh));
n = sendto(s, buf, sizeof(buf), 0, (struct sockaddr *)&saddr_ll, sizeof(saddr_ll));
if (!ASSERT_EQ(n, sizeof(buf), "sendto"))
diff --git a/tools/testing/selftests/bpf/prog_tests/xdp_bpf2bpf.c b/tools/testing/selftests/bpf/prog_tests/xdp_bpf2bpf.c
index 3bd5904..c98a897 100644
--- a/tools/testing/selftests/bpf/prog_tests/xdp_bpf2bpf.c
+++ b/tools/testing/selftests/bpf/prog_tests/xdp_bpf2bpf.c
@@ -42,14 +42,13 @@ void test_xdp_bpf2bpf(void)
char buf[128];
int err, pkt_fd, map_fd;
bool passed = false;
- struct iphdr *iph = (void *)buf + sizeof(struct ethhdr);
+ struct iphdr iph;
struct iptnl_info value4 = {.family = AF_INET};
struct test_xdp *pkt_skel = NULL;
struct test_xdp_bpf2bpf *ftrace_skel = NULL;
struct vip key4 = {.protocol = 6, .family = AF_INET};
struct bpf_program *prog;
struct perf_buffer *pb = NULL;
- struct perf_buffer_opts pb_opts = {};
/* Load XDP program to introspect */
pkt_skel = test_xdp__open_and_load();
@@ -86,19 +85,17 @@ void test_xdp_bpf2bpf(void)
goto out;
/* Set up perf buffer */
- pb_opts.sample_cb = on_sample;
- pb_opts.ctx = &passed;
- pb = perf_buffer__new(bpf_map__fd(ftrace_skel->maps.perf_buf_map),
- 1, &pb_opts);
+ pb = perf_buffer__new(bpf_map__fd(ftrace_skel->maps.perf_buf_map), 1,
+ on_sample, NULL, &passed, NULL);
if (!ASSERT_OK_PTR(pb, "perf_buf__new"))
goto out;
/* Run test program */
err = bpf_prog_test_run(pkt_fd, 1, &pkt_v4, sizeof(pkt_v4),
buf, &size, &retval, &duration);
-
+ memcpy(&iph, buf + sizeof(struct ethhdr), sizeof(iph));
if (CHECK(err || retval != XDP_TX || size != 74 ||
- iph->protocol != IPPROTO_IPIP, "ipv4",
+ iph.protocol != IPPROTO_IPIP, "ipv4",
"err %d errno %d retval %d size %d\n",
err, errno, retval, size))
goto out;
diff --git a/tools/testing/selftests/bpf/prog_tests/xdp_info.c b/tools/testing/selftests/bpf/prog_tests/xdp_info.c
index 4e2a4fd..abe48e8 100644
--- a/tools/testing/selftests/bpf/prog_tests/xdp_info.c
+++ b/tools/testing/selftests/bpf/prog_tests/xdp_info.c
@@ -29,7 +29,7 @@ void serial_test_xdp_info(void)
/* Setup prog */
- err = bpf_prog_load(file, BPF_PROG_TYPE_XDP, &obj, &prog_fd);
+ err = bpf_prog_test_load(file, BPF_PROG_TYPE_XDP, &obj, &prog_fd);
if (CHECK_FAIL(err))
return;
diff --git a/tools/testing/selftests/bpf/prog_tests/xdp_perf.c b/tools/testing/selftests/bpf/prog_tests/xdp_perf.c
index 7185bee..15a3900 100644
--- a/tools/testing/selftests/bpf/prog_tests/xdp_perf.c
+++ b/tools/testing/selftests/bpf/prog_tests/xdp_perf.c
@@ -9,7 +9,7 @@ void test_xdp_perf(void)
char in[128], out[128];
int err, prog_fd;
- err = bpf_prog_load(file, BPF_PROG_TYPE_XDP, &obj, &prog_fd);
+ err = bpf_prog_test_load(file, BPF_PROG_TYPE_XDP, &obj, &prog_fd);
if (CHECK_FAIL(err))
return;
diff --git a/tools/testing/selftests/bpf/progs/bpf_iter_unix.c b/tools/testing/selftests/bpf/progs/bpf_iter_unix.c
index 9442390..c21e3f5 100644
--- a/tools/testing/selftests/bpf/progs/bpf_iter_unix.c
+++ b/tools/testing/selftests/bpf/progs/bpf_iter_unix.c
@@ -49,7 +49,7 @@ int dump_unix(struct bpf_iter__unix *ctx)
sock_i_ino(sk));
if (unix_sk->addr) {
- if (!UNIX_ABSTRACT(unix_sk)) {
+ if (unix_sk->addr->name->sun_path[0]) {
BPF_SEQ_PRINTF(seq, " %s", unix_sk->addr->name->sun_path);
} else {
/* The name of the abstract UNIX domain socket starts
diff --git a/tools/testing/selftests/bpf/progs/bpf_loop.c b/tools/testing/selftests/bpf/progs/bpf_loop.c
new file mode 100644
index 0000000..12349e4
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/bpf_loop.c
@@ -0,0 +1,112 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2021 Facebook */
+
+#include "vmlinux.h"
+#include <bpf/bpf_helpers.h>
+
+char _license[] SEC("license") = "GPL";
+
+struct callback_ctx {
+ int output;
+};
+
+/* These should be set by the user program */
+u32 nested_callback_nr_loops;
+u32 stop_index = -1;
+u32 nr_loops;
+int pid;
+
+/* Making these global variables so that the userspace program
+ * can verify the output through the skeleton
+ */
+int nr_loops_returned;
+int g_output;
+int err;
+
+static int callback(__u32 index, void *data)
+{
+ struct callback_ctx *ctx = data;
+
+ if (index >= stop_index)
+ return 1;
+
+ ctx->output += index;
+
+ return 0;
+}
+
+static int empty_callback(__u32 index, void *data)
+{
+ return 0;
+}
+
+static int nested_callback2(__u32 index, void *data)
+{
+ nr_loops_returned += bpf_loop(nested_callback_nr_loops, callback, data, 0);
+
+ return 0;
+}
+
+static int nested_callback1(__u32 index, void *data)
+{
+ bpf_loop(nested_callback_nr_loops, nested_callback2, data, 0);
+ return 0;
+}
+
+SEC("fentry/__x64_sys_nanosleep")
+int test_prog(void *ctx)
+{
+ struct callback_ctx data = {};
+
+ if (bpf_get_current_pid_tgid() >> 32 != pid)
+ return 0;
+
+ nr_loops_returned = bpf_loop(nr_loops, callback, &data, 0);
+
+ if (nr_loops_returned < 0)
+ err = nr_loops_returned;
+ else
+ g_output = data.output;
+
+ return 0;
+}
+
+SEC("fentry/__x64_sys_nanosleep")
+int prog_null_ctx(void *ctx)
+{
+ if (bpf_get_current_pid_tgid() >> 32 != pid)
+ return 0;
+
+ nr_loops_returned = bpf_loop(nr_loops, empty_callback, NULL, 0);
+
+ return 0;
+}
+
+SEC("fentry/__x64_sys_nanosleep")
+int prog_invalid_flags(void *ctx)
+{
+ struct callback_ctx data = {};
+
+ if (bpf_get_current_pid_tgid() >> 32 != pid)
+ return 0;
+
+ err = bpf_loop(nr_loops, callback, &data, 1);
+
+ return 0;
+}
+
+SEC("fentry/__x64_sys_nanosleep")
+int prog_nested_calls(void *ctx)
+{
+ struct callback_ctx data = {};
+
+ if (bpf_get_current_pid_tgid() >> 32 != pid)
+ return 0;
+
+ nr_loops_returned = 0;
+ bpf_loop(nr_loops, nested_callback1, &data, 0);
+
+ g_output = data.output;
+
+ return 0;
+}
diff --git a/tools/testing/selftests/bpf/progs/bpf_loop_bench.c b/tools/testing/selftests/bpf/progs/bpf_loop_bench.c
new file mode 100644
index 0000000..9dafdc2
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/bpf_loop_bench.c
@@ -0,0 +1,26 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2021 Facebook */
+
+#include "vmlinux.h"
+#include <bpf/bpf_helpers.h>
+
+char _license[] SEC("license") = "GPL";
+
+u32 nr_loops;
+long hits;
+
+static int empty_callback(__u32 index, void *data)
+{
+ return 0;
+}
+
+SEC("fentry/__x64_sys_getpgid")
+int benchmark(void *ctx)
+{
+ for (int i = 0; i < 1000; i++) {
+ bpf_loop(nr_loops, empty_callback, NULL, 0);
+
+ __sync_add_and_fetch(&hits, nr_loops);
+ }
+ return 0;
+}
diff --git a/tools/testing/selftests/bpf/progs/bpf_tracing_net.h b/tools/testing/selftests/bpf/progs/bpf_tracing_net.h
index eef5646..e0f4260 100644
--- a/tools/testing/selftests/bpf/progs/bpf_tracing_net.h
+++ b/tools/testing/selftests/bpf/progs/bpf_tracing_net.h
@@ -6,8 +6,6 @@
#define AF_INET6 10
#define __SO_ACCEPTCON (1 << 16)
-#define UNIX_HASH_SIZE 256
-#define UNIX_ABSTRACT(unix_sk) (unix_sk->addr->hash < UNIX_HASH_SIZE)
#define SOL_TCP 6
#define TCP_CONGESTION 13
diff --git a/tools/testing/selftests/bpf/progs/btf_decl_tag.c b/tools/testing/selftests/bpf/progs/btf_decl_tag.c
new file mode 100644
index 0000000..c88ccc5
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/btf_decl_tag.c
@@ -0,0 +1,50 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2021 Facebook */
+#include "vmlinux.h"
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+
+#if __has_attribute(btf_decl_tag)
+#define __tag1 __attribute__((btf_decl_tag("tag1")))
+#define __tag2 __attribute__((btf_decl_tag("tag2")))
+volatile const bool skip_tests __tag1 __tag2 = false;
+#else
+#define __tag1
+#define __tag2
+volatile const bool skip_tests = true;
+#endif
+
+struct key_t {
+ int a;
+ int b __tag1 __tag2;
+ int c;
+} __tag1 __tag2;
+
+typedef struct {
+ int a;
+ int b;
+} value_t __tag1 __tag2;
+
+struct {
+ __uint(type, BPF_MAP_TYPE_HASH);
+ __uint(max_entries, 3);
+ __type(key, struct key_t);
+ __type(value, value_t);
+} hashmap1 SEC(".maps");
+
+
+static __noinline int foo(int x __tag1 __tag2) __tag1 __tag2
+{
+ struct key_t key;
+ value_t val = {};
+
+ key.a = key.b = key.c = x;
+ bpf_map_update_elem(&hashmap1, &key, &val, 0);
+ return 0;
+}
+
+SEC("fentry/bpf_fentry_test1")
+int BPF_PROG(sub, int x)
+{
+ return foo(x);
+}
diff --git a/tools/testing/selftests/bpf/progs/btf_type_tag.c b/tools/testing/selftests/bpf/progs/btf_type_tag.c
new file mode 100644
index 0000000..1d488da
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/btf_type_tag.c
@@ -0,0 +1,25 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2021 Facebook */
+#include "vmlinux.h"
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+
+#if __has_attribute(btf_type_tag)
+#define __tag1 __attribute__((btf_type_tag("tag1")))
+#define __tag2 __attribute__((btf_type_tag("tag2")))
+volatile const bool skip_tests = false;
+#else
+#define __tag1
+#define __tag2
+volatile const bool skip_tests = true;
+#endif
+
+struct btf_type_tag_test {
+ int __tag1 * __tag1 __tag2 *p;
+} g;
+
+SEC("fentry/bpf_fentry_test1")
+int BPF_PROG(sub, int x)
+{
+ return 0;
+}
diff --git a/tools/testing/selftests/bpf/progs/core_kern.c b/tools/testing/selftests/bpf/progs/core_kern.c
new file mode 100644
index 0000000..13499cc
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/core_kern.c
@@ -0,0 +1,104 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2021 Facebook */
+#include "vmlinux.h"
+
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+#include <bpf/bpf_core_read.h>
+
+#define ATTR __always_inline
+#include "test_jhash.h"
+
+struct {
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __type(key, u32);
+ __type(value, u32);
+ __uint(max_entries, 256);
+} array1 SEC(".maps");
+
+struct {
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __type(key, u32);
+ __type(value, u32);
+ __uint(max_entries, 256);
+} array2 SEC(".maps");
+
+static __noinline int randmap(int v, const struct net_device *dev)
+{
+ struct bpf_map *map = (struct bpf_map *)&array1;
+ int key = bpf_get_prandom_u32() & 0xff;
+ int *val;
+
+ if (bpf_get_prandom_u32() & 1)
+ map = (struct bpf_map *)&array2;
+
+ val = bpf_map_lookup_elem(map, &key);
+ if (val)
+ *val = bpf_get_prandom_u32() + v + dev->mtu;
+
+ return 0;
+}
+
+SEC("tp_btf/xdp_devmap_xmit")
+int BPF_PROG(tp_xdp_devmap_xmit_multi, const struct net_device
+ *from_dev, const struct net_device *to_dev, int sent, int drops,
+ int err)
+{
+ return randmap(from_dev->ifindex, from_dev);
+}
+
+SEC("fentry/eth_type_trans")
+int BPF_PROG(fentry_eth_type_trans, struct sk_buff *skb,
+ struct net_device *dev, unsigned short protocol)
+{
+ return randmap(dev->ifindex + skb->len, dev);
+}
+
+SEC("fexit/eth_type_trans")
+int BPF_PROG(fexit_eth_type_trans, struct sk_buff *skb,
+ struct net_device *dev, unsigned short protocol)
+{
+ return randmap(dev->ifindex + skb->len, dev);
+}
+
+volatile const int never;
+
+struct __sk_bUfF /* it will not exist in vmlinux */ {
+ int len;
+} __attribute__((preserve_access_index));
+
+struct bpf_testmod_test_read_ctx /* it exists in bpf_testmod */ {
+ size_t len;
+} __attribute__((preserve_access_index));
+
+SEC("tc")
+int balancer_ingress(struct __sk_buff *ctx)
+{
+ void *data_end = (void *)(long)ctx->data_end;
+ void *data = (void *)(long)ctx->data;
+ void *ptr;
+ int ret = 0, nh_off, i = 0;
+
+ nh_off = 14;
+
+ /* pragma unroll doesn't work on large loops */
+#define C do { \
+ ptr = data + i; \
+ if (ptr + nh_off > data_end) \
+ break; \
+ ctx->tc_index = jhash(ptr, nh_off, ctx->cb[0] + i++); \
+ if (never) { \
+ /* below is a dead code with unresolvable CO-RE relo */ \
+ i += ((struct __sk_bUfF *)ctx)->len; \
+ /* this CO-RE relo may or may not resolve
+ * depending on whether bpf_testmod is loaded.
+ */ \
+ i += ((struct bpf_testmod_test_read_ctx *)ctx)->len; \
+ } \
+ } while (0);
+#define C30 C;C;C;C;C;C;C;C;C;C;C;C;C;C;C;C;C;C;C;C;C;C;C;C;C;C;C;C;C;C;
+ C30;C30;C30; /* 90 calls */
+ return 0;
+}
+
+char LICENSE[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/progs/exhandler_kern.c b/tools/testing/selftests/bpf/progs/exhandler_kern.c
new file mode 100644
index 0000000..f5ca142
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/exhandler_kern.c
@@ -0,0 +1,43 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2021, Oracle and/or its affiliates. */
+
+#include "vmlinux.h"
+
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+#include <bpf/bpf_core_read.h>
+
+char _license[] SEC("license") = "GPL";
+
+unsigned int exception_triggered;
+int test_pid;
+
+/* TRACE_EVENT(task_newtask,
+ * TP_PROTO(struct task_struct *p, u64 clone_flags)
+ */
+SEC("tp_btf/task_newtask")
+int BPF_PROG(trace_task_newtask, struct task_struct *task, u64 clone_flags)
+{
+ int pid = bpf_get_current_pid_tgid() >> 32;
+ struct callback_head *work;
+ void *func;
+
+ if (test_pid != pid)
+ return 0;
+
+ /* To verify we hit an exception we dereference task->task_works->func.
+ * If task work has been added,
+ * - task->task_works is non-NULL; and
+ * - task->task_works->func is non-NULL also (the callback function
+ * must be specified for the task work.
+ *
+ * However, for a newly-created task, task->task_works is NULLed,
+ * so we know the exception handler triggered if task_works is
+ * NULL and func is NULL.
+ */
+ work = task->task_works;
+ func = work->func;
+ if (!work && !func)
+ exception_triggered++;
+ return 0;
+}
diff --git a/tools/testing/selftests/bpf/progs/fexit_bpf2bpf.c b/tools/testing/selftests/bpf/progs/fexit_bpf2bpf.c
index 49a84a3..48cd14b4 100644
--- a/tools/testing/selftests/bpf/progs/fexit_bpf2bpf.c
+++ b/tools/testing/selftests/bpf/progs/fexit_bpf2bpf.c
@@ -73,7 +73,7 @@ int test_subprog2(struct args_subprog2 *ctx)
__builtin_preserve_access_index(&skb->len));
ret = ctx->ret;
- /* bpf_prog_load() loads "test_pkt_access.o" with BPF_F_TEST_RND_HI32
+ /* bpf_prog_test_load() loads "test_pkt_access.o" with BPF_F_TEST_RND_HI32
* which randomizes upper 32 bits after BPF_ALU32 insns.
* Hence after 'w0 <<= 1' upper bits of $rax are random.
* That is expected and correct. Trim them.
diff --git a/tools/testing/selftests/bpf/progs/find_vma.c b/tools/testing/selftests/bpf/progs/find_vma.c
new file mode 100644
index 0000000..38034fb
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/find_vma.c
@@ -0,0 +1,69 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2021 Facebook */
+#include "vmlinux.h"
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+
+char _license[] SEC("license") = "GPL";
+
+struct callback_ctx {
+ int dummy;
+};
+
+#define VM_EXEC 0x00000004
+#define DNAME_INLINE_LEN 32
+
+pid_t target_pid = 0;
+char d_iname[DNAME_INLINE_LEN] = {0};
+__u32 found_vm_exec = 0;
+__u64 addr = 0;
+int find_zero_ret = -1;
+int find_addr_ret = -1;
+
+static long check_vma(struct task_struct *task, struct vm_area_struct *vma,
+ struct callback_ctx *data)
+{
+ if (vma->vm_file)
+ bpf_probe_read_kernel_str(d_iname, DNAME_INLINE_LEN - 1,
+ vma->vm_file->f_path.dentry->d_iname);
+
+ /* check for VM_EXEC */
+ if (vma->vm_flags & VM_EXEC)
+ found_vm_exec = 1;
+
+ return 0;
+}
+
+SEC("raw_tp/sys_enter")
+int handle_getpid(void)
+{
+ struct task_struct *task = bpf_get_current_task_btf();
+ struct callback_ctx data = {};
+
+ if (task->pid != target_pid)
+ return 0;
+
+ find_addr_ret = bpf_find_vma(task, addr, check_vma, &data, 0);
+
+ /* this should return -ENOENT */
+ find_zero_ret = bpf_find_vma(task, 0, check_vma, &data, 0);
+ return 0;
+}
+
+SEC("perf_event")
+int handle_pe(void)
+{
+ struct task_struct *task = bpf_get_current_task_btf();
+ struct callback_ctx data = {};
+
+ if (task->pid != target_pid)
+ return 0;
+
+ find_addr_ret = bpf_find_vma(task, addr, check_vma, &data, 0);
+
+ /* In NMI, this should return -EBUSY, as the previous call is using
+ * the irq_work.
+ */
+ find_zero_ret = bpf_find_vma(task, 0, check_vma, &data, 0);
+ return 0;
+}
diff --git a/tools/testing/selftests/bpf/progs/find_vma_fail1.c b/tools/testing/selftests/bpf/progs/find_vma_fail1.c
new file mode 100644
index 0000000..b3b326b
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/find_vma_fail1.c
@@ -0,0 +1,29 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2021 Facebook */
+#include "vmlinux.h"
+#include <bpf/bpf_helpers.h>
+
+char _license[] SEC("license") = "GPL";
+
+struct callback_ctx {
+ int dummy;
+};
+
+static long write_vma(struct task_struct *task, struct vm_area_struct *vma,
+ struct callback_ctx *data)
+{
+ /* writing to vma, which is illegal */
+ vma->vm_flags |= 0x55;
+
+ return 0;
+}
+
+SEC("raw_tp/sys_enter")
+int handle_getpid(void)
+{
+ struct task_struct *task = bpf_get_current_task_btf();
+ struct callback_ctx data = {};
+
+ bpf_find_vma(task, 0, write_vma, &data, 0);
+ return 0;
+}
diff --git a/tools/testing/selftests/bpf/progs/find_vma_fail2.c b/tools/testing/selftests/bpf/progs/find_vma_fail2.c
new file mode 100644
index 0000000..9bcf320
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/find_vma_fail2.c
@@ -0,0 +1,29 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2021 Facebook */
+#include "vmlinux.h"
+#include <bpf/bpf_helpers.h>
+
+char _license[] SEC("license") = "GPL";
+
+struct callback_ctx {
+ int dummy;
+};
+
+static long write_task(struct task_struct *task, struct vm_area_struct *vma,
+ struct callback_ctx *data)
+{
+ /* writing to task, which is illegal */
+ task->mm = NULL;
+
+ return 0;
+}
+
+SEC("raw_tp/sys_enter")
+int handle_getpid(void)
+{
+ struct task_struct *task = bpf_get_current_task_btf();
+ struct callback_ctx data = {};
+
+ bpf_find_vma(task, 0, write_task, &data, 0);
+ return 0;
+}
diff --git a/tools/testing/selftests/bpf/progs/get_func_args_test.c b/tools/testing/selftests/bpf/progs/get_func_args_test.c
new file mode 100644
index 0000000..e0f34a5
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/get_func_args_test.c
@@ -0,0 +1,123 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/bpf.h>
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+#include <errno.h>
+
+char _license[] SEC("license") = "GPL";
+
+__u64 test1_result = 0;
+SEC("fentry/bpf_fentry_test1")
+int BPF_PROG(test1)
+{
+ __u64 cnt = bpf_get_func_arg_cnt(ctx);
+ __u64 a = 0, z = 0, ret = 0;
+ __s64 err;
+
+ test1_result = cnt == 1;
+
+ /* valid arguments */
+ err = bpf_get_func_arg(ctx, 0, &a);
+
+ /* We need to cast access to traced function argument values with
+ * proper type cast, because trampoline uses type specific instruction
+ * to save it, like for 'int a' with 32-bit mov like:
+ *
+ * mov %edi,-0x8(%rbp)
+ *
+ * so the upper 4 bytes are not zeroed.
+ */
+ test1_result &= err == 0 && ((int) a == 1);
+
+ /* not valid argument */
+ err = bpf_get_func_arg(ctx, 1, &z);
+ test1_result &= err == -EINVAL;
+
+ /* return value fails in fentry */
+ err = bpf_get_func_ret(ctx, &ret);
+ test1_result &= err == -EOPNOTSUPP;
+ return 0;
+}
+
+__u64 test2_result = 0;
+SEC("fexit/bpf_fentry_test2")
+int BPF_PROG(test2)
+{
+ __u64 cnt = bpf_get_func_arg_cnt(ctx);
+ __u64 a = 0, b = 0, z = 0, ret = 0;
+ __s64 err;
+
+ test2_result = cnt == 2;
+
+ /* valid arguments */
+ err = bpf_get_func_arg(ctx, 0, &a);
+ test2_result &= err == 0 && (int) a == 2;
+
+ err = bpf_get_func_arg(ctx, 1, &b);
+ test2_result &= err == 0 && b == 3;
+
+ /* not valid argument */
+ err = bpf_get_func_arg(ctx, 2, &z);
+ test2_result &= err == -EINVAL;
+
+ /* return value */
+ err = bpf_get_func_ret(ctx, &ret);
+ test2_result &= err == 0 && ret == 5;
+ return 0;
+}
+
+__u64 test3_result = 0;
+SEC("fmod_ret/bpf_modify_return_test")
+int BPF_PROG(fmod_ret_test, int _a, int *_b, int _ret)
+{
+ __u64 cnt = bpf_get_func_arg_cnt(ctx);
+ __u64 a = 0, b = 0, z = 0, ret = 0;
+ __s64 err;
+
+ test3_result = cnt == 2;
+
+ /* valid arguments */
+ err = bpf_get_func_arg(ctx, 0, &a);
+ test3_result &= err == 0 && ((int) a == 1);
+
+ err = bpf_get_func_arg(ctx, 1, &b);
+ test3_result &= err == 0 && ((int *) b == _b);
+
+ /* not valid argument */
+ err = bpf_get_func_arg(ctx, 2, &z);
+ test3_result &= err == -EINVAL;
+
+ /* return value */
+ err = bpf_get_func_ret(ctx, &ret);
+ test3_result &= err == 0 && ret == 0;
+
+ /* change return value, it's checked in fexit_test program */
+ return 1234;
+}
+
+__u64 test4_result = 0;
+SEC("fexit/bpf_modify_return_test")
+int BPF_PROG(fexit_test, int _a, int *_b, int _ret)
+{
+ __u64 cnt = bpf_get_func_arg_cnt(ctx);
+ __u64 a = 0, b = 0, z = 0, ret = 0;
+ __s64 err;
+
+ test4_result = cnt == 2;
+
+ /* valid arguments */
+ err = bpf_get_func_arg(ctx, 0, &a);
+ test4_result &= err == 0 && ((int) a == 1);
+
+ err = bpf_get_func_arg(ctx, 1, &b);
+ test4_result &= err == 0 && ((int *) b == _b);
+
+ /* not valid argument */
+ err = bpf_get_func_arg(ctx, 2, &z);
+ test4_result &= err == -EINVAL;
+
+ /* return value */
+ err = bpf_get_func_ret(ctx, &ret);
+ test4_result &= err == 0 && ret == 1234;
+ return 0;
+}
diff --git a/tools/testing/selftests/bpf/progs/local_storage.c b/tools/testing/selftests/bpf/progs/local_storage.c
index 95868bc..9b1f9b7 100644
--- a/tools/testing/selftests/bpf/progs/local_storage.c
+++ b/tools/testing/selftests/bpf/progs/local_storage.c
@@ -20,7 +20,6 @@ int sk_storage_result = -1;
struct local_storage {
struct inode *exec_inode;
__u32 value;
- struct bpf_spin_lock lock;
};
struct {
@@ -58,9 +57,7 @@ int BPF_PROG(unlink_hook, struct inode *dir, struct dentry *victim)
bpf_get_current_task_btf(), 0, 0);
if (storage) {
/* Don't let an executable delete itself */
- bpf_spin_lock(&storage->lock);
is_self_unlink = storage->exec_inode == victim->d_inode;
- bpf_spin_unlock(&storage->lock);
if (is_self_unlink)
return -EPERM;
}
@@ -68,7 +65,7 @@ int BPF_PROG(unlink_hook, struct inode *dir, struct dentry *victim)
return 0;
}
-SEC("lsm/inode_rename")
+SEC("lsm.s/inode_rename")
int BPF_PROG(inode_rename, struct inode *old_dir, struct dentry *old_dentry,
struct inode *new_dir, struct dentry *new_dentry,
unsigned int flags)
@@ -89,10 +86,8 @@ int BPF_PROG(inode_rename, struct inode *old_dir, struct dentry *old_dentry,
if (!storage)
return 0;
- bpf_spin_lock(&storage->lock);
if (storage->value != DUMMY_STORAGE_VALUE)
inode_storage_result = -1;
- bpf_spin_unlock(&storage->lock);
err = bpf_inode_storage_delete(&inode_storage_map, old_dentry->d_inode);
if (!err)
@@ -101,7 +96,7 @@ int BPF_PROG(inode_rename, struct inode *old_dir, struct dentry *old_dentry,
return 0;
}
-SEC("lsm/socket_bind")
+SEC("lsm.s/socket_bind")
int BPF_PROG(socket_bind, struct socket *sock, struct sockaddr *address,
int addrlen)
{
@@ -117,10 +112,8 @@ int BPF_PROG(socket_bind, struct socket *sock, struct sockaddr *address,
if (!storage)
return 0;
- bpf_spin_lock(&storage->lock);
if (storage->value != DUMMY_STORAGE_VALUE)
sk_storage_result = -1;
- bpf_spin_unlock(&storage->lock);
err = bpf_sk_storage_delete(&sk_storage_map, sock->sk);
if (!err)
@@ -129,7 +122,7 @@ int BPF_PROG(socket_bind, struct socket *sock, struct sockaddr *address,
return 0;
}
-SEC("lsm/socket_post_create")
+SEC("lsm.s/socket_post_create")
int BPF_PROG(socket_post_create, struct socket *sock, int family, int type,
int protocol, int kern)
{
@@ -144,9 +137,7 @@ int BPF_PROG(socket_post_create, struct socket *sock, int family, int type,
if (!storage)
return 0;
- bpf_spin_lock(&storage->lock);
storage->value = DUMMY_STORAGE_VALUE;
- bpf_spin_unlock(&storage->lock);
return 0;
}
@@ -154,7 +145,7 @@ int BPF_PROG(socket_post_create, struct socket *sock, int family, int type,
/* This uses the local storage to remember the inode of the binary that a
* process was originally executing.
*/
-SEC("lsm/bprm_committed_creds")
+SEC("lsm.s/bprm_committed_creds")
void BPF_PROG(exec, struct linux_binprm *bprm)
{
__u32 pid = bpf_get_current_pid_tgid() >> 32;
@@ -166,18 +157,13 @@ void BPF_PROG(exec, struct linux_binprm *bprm)
storage = bpf_task_storage_get(&task_storage_map,
bpf_get_current_task_btf(), 0,
BPF_LOCAL_STORAGE_GET_F_CREATE);
- if (storage) {
- bpf_spin_lock(&storage->lock);
+ if (storage)
storage->exec_inode = bprm->file->f_inode;
- bpf_spin_unlock(&storage->lock);
- }
storage = bpf_inode_storage_get(&inode_storage_map, bprm->file->f_inode,
0, BPF_LOCAL_STORAGE_GET_F_CREATE);
if (!storage)
return;
- bpf_spin_lock(&storage->lock);
storage->value = DUMMY_STORAGE_VALUE;
- bpf_spin_unlock(&storage->lock);
}
diff --git a/tools/testing/selftests/bpf/progs/loop3.c b/tools/testing/selftests/bpf/progs/loop3.c
index 76e93b3..717dab1 100644
--- a/tools/testing/selftests/bpf/progs/loop3.c
+++ b/tools/testing/selftests/bpf/progs/loop3.c
@@ -12,9 +12,9 @@
char _license[] SEC("license") = "GPL";
SEC("raw_tracepoint/consume_skb")
-int while_true(volatile struct pt_regs* ctx)
+int while_true(struct pt_regs *ctx)
{
- __u64 i = 0, sum = 0;
+ volatile __u64 i = 0, sum = 0;
do {
i++;
sum += PT_REGS_RC(ctx);
diff --git a/tools/testing/selftests/bpf/progs/map_ptr_kern.c b/tools/testing/selftests/bpf/progs/map_ptr_kern.c
index b1b711d..b64df94 100644
--- a/tools/testing/selftests/bpf/progs/map_ptr_kern.c
+++ b/tools/testing/selftests/bpf/progs/map_ptr_kern.c
@@ -334,9 +334,11 @@ static inline int check_lpm_trie(void)
return 1;
}
+#define INNER_MAX_ENTRIES 1234
+
struct inner_map {
__uint(type, BPF_MAP_TYPE_ARRAY);
- __uint(max_entries, 1);
+ __uint(max_entries, INNER_MAX_ENTRIES);
__type(key, __u32);
__type(value, __u32);
} inner_map SEC(".maps");
@@ -348,7 +350,7 @@ struct {
__type(value, __u32);
__array(values, struct {
__uint(type, BPF_MAP_TYPE_ARRAY);
- __uint(max_entries, 1);
+ __uint(max_entries, INNER_MAX_ENTRIES);
__type(key, __u32);
__type(value, __u32);
});
@@ -360,8 +362,13 @@ static inline int check_array_of_maps(void)
{
struct bpf_array *array_of_maps = (struct bpf_array *)&m_array_of_maps;
struct bpf_map *map = (struct bpf_map *)&m_array_of_maps;
+ struct bpf_array *inner_map;
+ int key = 0;
VERIFY(check_default(&array_of_maps->map, map));
+ inner_map = bpf_map_lookup_elem(array_of_maps, &key);
+ VERIFY(inner_map != 0);
+ VERIFY(inner_map->map.max_entries == INNER_MAX_ENTRIES);
return 1;
}
@@ -382,8 +389,13 @@ static inline int check_hash_of_maps(void)
{
struct bpf_htab *hash_of_maps = (struct bpf_htab *)&m_hash_of_maps;
struct bpf_map *map = (struct bpf_map *)&m_hash_of_maps;
+ struct bpf_htab *inner_map;
+ int key = 2;
VERIFY(check_default(&hash_of_maps->map, map));
+ inner_map = bpf_map_lookup_elem(hash_of_maps, &key);
+ VERIFY(inner_map != 0);
+ VERIFY(inner_map->map.max_entries == INNER_MAX_ENTRIES);
return 1;
}
diff --git a/tools/testing/selftests/bpf/progs/pyperf.h b/tools/testing/selftests/bpf/progs/pyperf.h
index 2fb7ada..1ed2888 100644
--- a/tools/testing/selftests/bpf/progs/pyperf.h
+++ b/tools/testing/selftests/bpf/progs/pyperf.h
@@ -159,6 +159,59 @@ struct {
__uint(value_size, sizeof(long long) * 127);
} stackmap SEC(".maps");
+#ifdef USE_BPF_LOOP
+struct process_frame_ctx {
+ int cur_cpu;
+ int32_t *symbol_counter;
+ void *frame_ptr;
+ FrameData *frame;
+ PidData *pidData;
+ Symbol *sym;
+ Event *event;
+ bool done;
+};
+
+#define barrier_var(var) asm volatile("" : "=r"(var) : "0"(var))
+
+static int process_frame_callback(__u32 i, struct process_frame_ctx *ctx)
+{
+ int zero = 0;
+ void *frame_ptr = ctx->frame_ptr;
+ PidData *pidData = ctx->pidData;
+ FrameData *frame = ctx->frame;
+ int32_t *symbol_counter = ctx->symbol_counter;
+ int cur_cpu = ctx->cur_cpu;
+ Event *event = ctx->event;
+ Symbol *sym = ctx->sym;
+
+ if (frame_ptr && get_frame_data(frame_ptr, pidData, frame, sym)) {
+ int32_t new_symbol_id = *symbol_counter * 64 + cur_cpu;
+ int32_t *symbol_id = bpf_map_lookup_elem(&symbolmap, sym);
+
+ if (!symbol_id) {
+ bpf_map_update_elem(&symbolmap, sym, &zero, 0);
+ symbol_id = bpf_map_lookup_elem(&symbolmap, sym);
+ if (!symbol_id) {
+ ctx->done = true;
+ return 1;
+ }
+ }
+ if (*symbol_id == new_symbol_id)
+ (*symbol_counter)++;
+
+ barrier_var(i);
+ if (i >= STACK_MAX_LEN)
+ return 1;
+
+ event->stack[i] = *symbol_id;
+
+ event->stack_len = i + 1;
+ frame_ptr = frame->f_back;
+ }
+ return 0;
+}
+#endif /* USE_BPF_LOOP */
+
#ifdef GLOBAL_FUNC
__noinline
#elif defined(SUBPROGS)
@@ -228,11 +281,26 @@ int __on_event(struct bpf_raw_tracepoint_args *ctx)
int32_t* symbol_counter = bpf_map_lookup_elem(&symbolmap, &sym);
if (symbol_counter == NULL)
return 0;
+#ifdef USE_BPF_LOOP
+ struct process_frame_ctx ctx = {
+ .cur_cpu = cur_cpu,
+ .symbol_counter = symbol_counter,
+ .frame_ptr = frame_ptr,
+ .frame = &frame,
+ .pidData = pidData,
+ .sym = &sym,
+ .event = event,
+ };
+
+ bpf_loop(STACK_MAX_LEN, process_frame_callback, &ctx, 0);
+ if (ctx.done)
+ return 0;
+#else
#ifdef NO_UNROLL
#pragma clang loop unroll(disable)
#else
#pragma clang loop unroll(full)
-#endif
+#endif /* NO_UNROLL */
/* Unwind python stack */
for (int i = 0; i < STACK_MAX_LEN; ++i) {
if (frame_ptr && get_frame_data(frame_ptr, pidData, &frame, &sym)) {
@@ -251,6 +319,7 @@ int __on_event(struct bpf_raw_tracepoint_args *ctx)
frame_ptr = frame.f_back;
}
}
+#endif /* USE_BPF_LOOP */
event->stack_complete = frame_ptr == NULL;
} else {
event->stack_complete = 1;
diff --git a/tools/testing/selftests/bpf/progs/pyperf600_bpf_loop.c b/tools/testing/selftests/bpf/progs/pyperf600_bpf_loop.c
new file mode 100644
index 0000000..5c2059d
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/pyperf600_bpf_loop.c
@@ -0,0 +1,6 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2021 Facebook */
+
+#define STACK_MAX_LEN 600
+#define USE_BPF_LOOP
+#include "pyperf.h"
diff --git a/tools/testing/selftests/bpf/progs/strncmp_bench.c b/tools/testing/selftests/bpf/progs/strncmp_bench.c
new file mode 100644
index 0000000..18373a7
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/strncmp_bench.c
@@ -0,0 +1,50 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2021. Huawei Technologies Co., Ltd */
+#include <linux/types.h>
+#include <linux/bpf.h>
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+
+#define STRNCMP_STR_SZ 4096
+
+/* Will be updated by benchmark before program loading */
+const volatile unsigned int cmp_str_len = 1;
+const char target[STRNCMP_STR_SZ];
+
+long hits = 0;
+char str[STRNCMP_STR_SZ];
+
+char _license[] SEC("license") = "GPL";
+
+static __always_inline int local_strncmp(const char *s1, unsigned int sz,
+ const char *s2)
+{
+ int ret = 0;
+ unsigned int i;
+
+ for (i = 0; i < sz; i++) {
+ /* E.g. 0xff > 0x31 */
+ ret = (unsigned char)s1[i] - (unsigned char)s2[i];
+ if (ret || !s1[i])
+ break;
+ }
+
+ return ret;
+}
+
+SEC("tp/syscalls/sys_enter_getpgid")
+int strncmp_no_helper(void *ctx)
+{
+ if (local_strncmp(str, cmp_str_len + 1, target) < 0)
+ __sync_add_and_fetch(&hits, 1);
+ return 0;
+}
+
+SEC("tp/syscalls/sys_enter_getpgid")
+int strncmp_helper(void *ctx)
+{
+ if (bpf_strncmp(str, cmp_str_len + 1, target) < 0)
+ __sync_add_and_fetch(&hits, 1);
+ return 0;
+}
+
diff --git a/tools/testing/selftests/bpf/progs/strncmp_test.c b/tools/testing/selftests/bpf/progs/strncmp_test.c
new file mode 100644
index 0000000..900d930
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/strncmp_test.c
@@ -0,0 +1,54 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2021. Huawei Technologies Co., Ltd */
+#include <stdbool.h>
+#include <linux/types.h>
+#include <linux/bpf.h>
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+
+#define STRNCMP_STR_SZ 8
+
+const char target[STRNCMP_STR_SZ] = "EEEEEEE";
+char str[STRNCMP_STR_SZ];
+int cmp_ret = 0;
+int target_pid = 0;
+
+const char no_str_target[STRNCMP_STR_SZ] = "12345678";
+char writable_target[STRNCMP_STR_SZ];
+unsigned int no_const_str_size = STRNCMP_STR_SZ;
+
+char _license[] SEC("license") = "GPL";
+
+SEC("tp/syscalls/sys_enter_nanosleep")
+int do_strncmp(void *ctx)
+{
+ if ((bpf_get_current_pid_tgid() >> 32) != target_pid)
+ return 0;
+
+ cmp_ret = bpf_strncmp(str, STRNCMP_STR_SZ, target);
+ return 0;
+}
+
+SEC("tp/syscalls/sys_enter_nanosleep")
+int strncmp_bad_not_const_str_size(void *ctx)
+{
+ /* The value of string size is not const, so will fail */
+ cmp_ret = bpf_strncmp(str, no_const_str_size, target);
+ return 0;
+}
+
+SEC("tp/syscalls/sys_enter_nanosleep")
+int strncmp_bad_writable_target(void *ctx)
+{
+ /* Compared target is not read-only, so will fail */
+ cmp_ret = bpf_strncmp(str, STRNCMP_STR_SZ, writable_target);
+ return 0;
+}
+
+SEC("tp/syscalls/sys_enter_nanosleep")
+int strncmp_bad_not_null_term_target(void *ctx)
+{
+ /* Compared target is not null-terminated, so will fail */
+ cmp_ret = bpf_strncmp(str, STRNCMP_STR_SZ, no_str_target);
+ return 0;
+}
diff --git a/tools/testing/selftests/bpf/progs/strobemeta.h b/tools/testing/selftests/bpf/progs/strobemeta.h
index 60c93ae..7537185 100644
--- a/tools/testing/selftests/bpf/progs/strobemeta.h
+++ b/tools/testing/selftests/bpf/progs/strobemeta.h
@@ -445,6 +445,48 @@ static __always_inline void *read_map_var(struct strobemeta_cfg *cfg,
return payload;
}
+#ifdef USE_BPF_LOOP
+enum read_type {
+ READ_INT_VAR,
+ READ_MAP_VAR,
+ READ_STR_VAR,
+};
+
+struct read_var_ctx {
+ struct strobemeta_payload *data;
+ void *tls_base;
+ struct strobemeta_cfg *cfg;
+ void *payload;
+ /* value gets mutated */
+ struct strobe_value_generic *value;
+ enum read_type type;
+};
+
+static int read_var_callback(__u32 index, struct read_var_ctx *ctx)
+{
+ switch (ctx->type) {
+ case READ_INT_VAR:
+ if (index >= STROBE_MAX_INTS)
+ return 1;
+ read_int_var(ctx->cfg, index, ctx->tls_base, ctx->value, ctx->data);
+ break;
+ case READ_MAP_VAR:
+ if (index >= STROBE_MAX_MAPS)
+ return 1;
+ ctx->payload = read_map_var(ctx->cfg, index, ctx->tls_base,
+ ctx->value, ctx->data, ctx->payload);
+ break;
+ case READ_STR_VAR:
+ if (index >= STROBE_MAX_STRS)
+ return 1;
+ ctx->payload += read_str_var(ctx->cfg, index, ctx->tls_base,
+ ctx->value, ctx->data, ctx->payload);
+ break;
+ }
+ return 0;
+}
+#endif /* USE_BPF_LOOP */
+
/*
* read_strobe_meta returns NULL, if no metadata was read; otherwise returns
* pointer to *right after* payload ends
@@ -475,11 +517,36 @@ static void *read_strobe_meta(struct task_struct *task,
*/
tls_base = (void *)task;
+#ifdef USE_BPF_LOOP
+ struct read_var_ctx ctx = {
+ .cfg = cfg,
+ .tls_base = tls_base,
+ .value = &value,
+ .data = data,
+ .payload = payload,
+ };
+ int err;
+
+ ctx.type = READ_INT_VAR;
+ err = bpf_loop(STROBE_MAX_INTS, read_var_callback, &ctx, 0);
+ if (err != STROBE_MAX_INTS)
+ return NULL;
+
+ ctx.type = READ_STR_VAR;
+ err = bpf_loop(STROBE_MAX_STRS, read_var_callback, &ctx, 0);
+ if (err != STROBE_MAX_STRS)
+ return NULL;
+
+ ctx.type = READ_MAP_VAR;
+ err = bpf_loop(STROBE_MAX_MAPS, read_var_callback, &ctx, 0);
+ if (err != STROBE_MAX_MAPS)
+ return NULL;
+#else
#ifdef NO_UNROLL
#pragma clang loop unroll(disable)
#else
#pragma unroll
-#endif
+#endif /* NO_UNROLL */
for (int i = 0; i < STROBE_MAX_INTS; ++i) {
read_int_var(cfg, i, tls_base, &value, data);
}
@@ -487,7 +554,7 @@ static void *read_strobe_meta(struct task_struct *task,
#pragma clang loop unroll(disable)
#else
#pragma unroll
-#endif
+#endif /* NO_UNROLL */
for (int i = 0; i < STROBE_MAX_STRS; ++i) {
payload += read_str_var(cfg, i, tls_base, &value, data, payload);
}
@@ -495,10 +562,12 @@ static void *read_strobe_meta(struct task_struct *task,
#pragma clang loop unroll(disable)
#else
#pragma unroll
-#endif
+#endif /* NO_UNROLL */
for (int i = 0; i < STROBE_MAX_MAPS; ++i) {
payload = read_map_var(cfg, i, tls_base, &value, data, payload);
}
+#endif /* USE_BPF_LOOP */
+
/*
* return pointer right after end of payload, so it's possible to
* calculate exact amount of useful data that needs to be sent
diff --git a/tools/testing/selftests/bpf/progs/strobemeta_bpf_loop.c b/tools/testing/selftests/bpf/progs/strobemeta_bpf_loop.c
new file mode 100644
index 0000000..d18b992
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/strobemeta_bpf_loop.c
@@ -0,0 +1,9 @@
+// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
+/* Copyright (c) 2021 Facebook */
+
+#define STROBE_MAX_INTS 2
+#define STROBE_MAX_STRS 25
+#define STROBE_MAX_MAPS 100
+#define STROBE_MAX_MAP_ENTRIES 20
+#define USE_BPF_LOOP
+#include "strobemeta.h"
diff --git a/tools/testing/selftests/bpf/progs/tag.c b/tools/testing/selftests/bpf/progs/tag.c
deleted file mode 100644
index 1792f4e..0000000
--- a/tools/testing/selftests/bpf/progs/tag.c
+++ /dev/null
@@ -1,54 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/* Copyright (c) 2021 Facebook */
-#include "vmlinux.h"
-#include <bpf/bpf_helpers.h>
-#include <bpf/bpf_tracing.h>
-
-#ifndef __has_attribute
-#define __has_attribute(x) 0
-#endif
-
-#if __has_attribute(btf_decl_tag)
-#define __tag1 __attribute__((btf_decl_tag("tag1")))
-#define __tag2 __attribute__((btf_decl_tag("tag2")))
-volatile const bool skip_tests __tag1 __tag2 = false;
-#else
-#define __tag1
-#define __tag2
-volatile const bool skip_tests = true;
-#endif
-
-struct key_t {
- int a;
- int b __tag1 __tag2;
- int c;
-} __tag1 __tag2;
-
-typedef struct {
- int a;
- int b;
-} value_t __tag1 __tag2;
-
-struct {
- __uint(type, BPF_MAP_TYPE_HASH);
- __uint(max_entries, 3);
- __type(key, struct key_t);
- __type(value, value_t);
-} hashmap1 SEC(".maps");
-
-
-static __noinline int foo(int x __tag1 __tag2) __tag1 __tag2
-{
- struct key_t key;
- value_t val = {};
-
- key.a = key.b = key.c = x;
- bpf_map_update_elem(&hashmap1, &key, &val, 0);
- return 0;
-}
-
-SEC("fentry/bpf_fentry_test1")
-int BPF_PROG(sub, int x)
-{
- return foo(x);
-}
diff --git a/tools/testing/selftests/bpf/progs/test_d_path_check_rdonly_mem.c b/tools/testing/selftests/bpf/progs/test_d_path_check_rdonly_mem.c
new file mode 100644
index 0000000..27c27cf
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/test_d_path_check_rdonly_mem.c
@@ -0,0 +1,28 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2022 Google */
+
+#include "vmlinux.h"
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+
+extern const int bpf_prog_active __ksym;
+
+SEC("fentry/security_inode_getattr")
+int BPF_PROG(d_path_check_rdonly_mem, struct path *path, struct kstat *stat,
+ __u32 request_mask, unsigned int query_flags)
+{
+ void *active;
+ __u32 cpu;
+
+ cpu = bpf_get_smp_processor_id();
+ active = (void *)bpf_per_cpu_ptr(&bpf_prog_active, cpu);
+ if (active) {
+ /* FAIL here! 'active' points to readonly memory. bpf helpers
+ * that update its arguments can not write into it.
+ */
+ bpf_d_path(path, active, sizeof(int));
+ }
+ return 0;
+}
+
+char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/progs/test_ksyms_btf_write_check.c b/tools/testing/selftests/bpf/progs/test_ksyms_btf_write_check.c
new file mode 100644
index 0000000..2180c41cd
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/test_ksyms_btf_write_check.c
@@ -0,0 +1,29 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2021 Google */
+
+#include "vmlinux.h"
+
+#include <bpf/bpf_helpers.h>
+
+extern const int bpf_prog_active __ksym; /* int type global var. */
+
+SEC("raw_tp/sys_enter")
+int handler(const void *ctx)
+{
+ int *active;
+ __u32 cpu;
+
+ cpu = bpf_get_smp_processor_id();
+ active = (int *)bpf_per_cpu_ptr(&bpf_prog_active, cpu);
+ if (active) {
+ /* Kernel memory obtained from bpf_{per,this}_cpu_ptr
+ * is read-only, should _not_ pass verification.
+ */
+ /* WRITE_ONCE */
+ *(volatile int *)active = -1;
+ }
+
+ return 0;
+}
+
+char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/progs/test_ksyms_weak.c b/tools/testing/selftests/bpf/progs/test_ksyms_weak.c
index 8eadbd4..5f8379a 100644
--- a/tools/testing/selftests/bpf/progs/test_ksyms_weak.c
+++ b/tools/testing/selftests/bpf/progs/test_ksyms_weak.c
@@ -38,7 +38,7 @@ int pass_handler(const void *ctx)
/* tests existing symbols. */
rq = (struct rq *)bpf_per_cpu_ptr(&runqueues, 0);
if (rq)
- out__existing_typed = 0;
+ out__existing_typed = rq->cpu;
out__existing_typeless = (__u64)&bpf_prog_active;
/* tests non-existent symbols. */
diff --git a/tools/testing/selftests/bpf/progs/test_l4lb.c b/tools/testing/selftests/bpf/progs/test_l4lb.c
index 04fee08..c26057e 100644
--- a/tools/testing/selftests/bpf/progs/test_l4lb.c
+++ b/tools/testing/selftests/bpf/progs/test_l4lb.c
@@ -448,7 +448,7 @@ static __always_inline int process_packet(void *data, __u64 off, void *data_end,
return bpf_redirect(ifindex, 0);
}
-SEC("l4lb-demo")
+SEC("tc")
int balancer_ingress(struct __sk_buff *ctx)
{
void *data_end = (void *)(long)ctx->data_end;
diff --git a/tools/testing/selftests/bpf/progs/test_l4lb_noinline.c b/tools/testing/selftests/bpf/progs/test_l4lb_noinline.c
index b9e2753..19e4d20 100644
--- a/tools/testing/selftests/bpf/progs/test_l4lb_noinline.c
+++ b/tools/testing/selftests/bpf/progs/test_l4lb_noinline.c
@@ -447,7 +447,7 @@ static __noinline int process_packet(void *data, __u64 off, void *data_end,
return bpf_redirect(ifindex, 0);
}
-SEC("l4lb-demo")
+SEC("tc")
int balancer_ingress(struct __sk_buff *ctx)
{
void *data_end = (void *)(long)ctx->data_end;
diff --git a/tools/testing/selftests/bpf/progs/test_legacy_printk.c b/tools/testing/selftests/bpf/progs/test_legacy_printk.c
new file mode 100644
index 0000000..64c2d9c
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/test_legacy_printk.c
@@ -0,0 +1,73 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2021 Facebook */
+
+#include <linux/bpf.h>
+#define BPF_NO_GLOBAL_DATA
+#include <bpf/bpf_helpers.h>
+
+char LICENSE[] SEC("license") = "GPL";
+
+struct {
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __type(key, int);
+ __type(value, int);
+ __uint(max_entries, 1);
+} my_pid_map SEC(".maps");
+
+struct {
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __type(key, int);
+ __type(value, int);
+ __uint(max_entries, 1);
+} res_map SEC(".maps");
+
+volatile int my_pid_var = 0;
+volatile int res_var = 0;
+
+SEC("tp/raw_syscalls/sys_enter")
+int handle_legacy(void *ctx)
+{
+ int zero = 0, *my_pid, cur_pid, *my_res;
+
+ my_pid = bpf_map_lookup_elem(&my_pid_map, &zero);
+ if (!my_pid)
+ return 1;
+
+ cur_pid = bpf_get_current_pid_tgid() >> 32;
+ if (cur_pid != *my_pid)
+ return 1;
+
+ my_res = bpf_map_lookup_elem(&res_map, &zero);
+ if (!my_res)
+ return 1;
+
+ if (*my_res == 0)
+ /* use bpf_printk() in combination with BPF_NO_GLOBAL_DATA to
+ * force .rodata.str1.1 section that previously caused
+ * problems on old kernels due to libbpf always tried to
+ * create a global data map for it
+ */
+ bpf_printk("Legacy-case bpf_printk test, pid %d\n", cur_pid);
+ *my_res = 1;
+
+ return *my_res;
+}
+
+SEC("tp/raw_syscalls/sys_enter")
+int handle_modern(void *ctx)
+{
+ int zero = 0, cur_pid;
+
+ cur_pid = bpf_get_current_pid_tgid() >> 32;
+ if (cur_pid != my_pid_var)
+ return 1;
+
+ if (res_var == 0)
+ /* we need bpf_printk() to validate libbpf logic around unused
+ * global maps and legacy kernels; see comment in handle_legacy()
+ */
+ bpf_printk("Modern-case bpf_printk test, pid %d\n", cur_pid);
+ res_var = 1;
+
+ return res_var;
+}
diff --git a/tools/testing/selftests/bpf/progs/test_log_buf.c b/tools/testing/selftests/bpf/progs/test_log_buf.c
new file mode 100644
index 0000000..199f459
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/test_log_buf.c
@@ -0,0 +1,24 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2021 Facebook */
+
+#include <linux/bpf.h>
+#include <bpf/bpf_helpers.h>
+
+int a[4];
+const volatile int off = 4000;
+
+SEC("raw_tp/sys_enter")
+int good_prog(const void *ctx)
+{
+ a[0] = (int)(long)ctx;
+ return a[1];
+}
+
+SEC("raw_tp/sys_enter")
+int bad_prog(const void *ctx)
+{
+ /* out of bounds access */
+ return a[off];
+}
+
+char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/progs/test_map_lock.c b/tools/testing/selftests/bpf/progs/test_map_lock.c
index b5c07ae..acf073d 100644
--- a/tools/testing/selftests/bpf/progs/test_map_lock.c
+++ b/tools/testing/selftests/bpf/progs/test_map_lock.c
@@ -30,7 +30,7 @@ struct {
__type(value, struct array_elem);
} array_map SEC(".maps");
-SEC("map_lock_demo")
+SEC("cgroup/skb")
int bpf_map_lock_test(struct __sk_buff *skb)
{
struct hmap_elem zero = {}, *val;
diff --git a/tools/testing/selftests/bpf/progs/test_prog_array_init.c b/tools/testing/selftests/bpf/progs/test_prog_array_init.c
new file mode 100644
index 0000000..2cd1383
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/test_prog_array_init.c
@@ -0,0 +1,39 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (c) 2021 Hengqi Chen */
+
+#include "vmlinux.h"
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+
+const volatile pid_t my_pid = 0;
+int value = 0;
+
+SEC("raw_tp/sys_enter")
+int tailcall_1(void *ctx)
+{
+ value = 42;
+ return 0;
+}
+
+struct {
+ __uint(type, BPF_MAP_TYPE_PROG_ARRAY);
+ __uint(max_entries, 2);
+ __uint(key_size, sizeof(__u32));
+ __array(values, int (void *));
+} prog_array_init SEC(".maps") = {
+ .values = {
+ [1] = (void *)&tailcall_1,
+ },
+};
+
+SEC("raw_tp/sys_enter")
+int entry(void *ctx)
+{
+ pid_t pid = bpf_get_current_pid_tgid() >> 32;
+
+ if (pid != my_pid)
+ return 0;
+
+ bpf_tail_call(ctx, &prog_array_init, 1);
+ return 0;
+}
diff --git a/tools/testing/selftests/bpf/progs/test_queue_stack_map.h b/tools/testing/selftests/bpf/progs/test_queue_stack_map.h
index 0fcd3ff..648e8ca 100644
--- a/tools/testing/selftests/bpf/progs/test_queue_stack_map.h
+++ b/tools/testing/selftests/bpf/progs/test_queue_stack_map.h
@@ -24,7 +24,7 @@ struct {
__uint(value_size, sizeof(__u32));
} map_out SEC(".maps");
-SEC("test")
+SEC("tc")
int _test(struct __sk_buff *skb)
{
void *data_end = (void *)(long)skb->data_end;
diff --git a/tools/testing/selftests/bpf/progs/test_sk_lookup.c b/tools/testing/selftests/bpf/progs/test_sk_lookup.c
index 19d2465..83b0aaa 100644
--- a/tools/testing/selftests/bpf/progs/test_sk_lookup.c
+++ b/tools/testing/selftests/bpf/progs/test_sk_lookup.c
@@ -84,6 +84,14 @@ int lookup_drop(struct bpf_sk_lookup *ctx)
return SK_DROP;
}
+SEC("sk_lookup")
+int check_ifindex(struct bpf_sk_lookup *ctx)
+{
+ if (ctx->ingress_ifindex == 1)
+ return SK_DROP;
+ return SK_PASS;
+}
+
SEC("sk_reuseport")
int reuseport_pass(struct sk_reuseport_md *ctx)
{
diff --git a/tools/testing/selftests/bpf/progs/test_sk_storage_tracing.c b/tools/testing/selftests/bpf/progs/test_sk_storage_tracing.c
index 8e94e5c..6dc1f28 100644
--- a/tools/testing/selftests/bpf/progs/test_sk_storage_tracing.c
+++ b/tools/testing/selftests/bpf/progs/test_sk_storage_tracing.c
@@ -68,7 +68,7 @@ static void set_task_info(struct sock *sk)
}
SEC("fentry/inet_csk_listen_start")
-int BPF_PROG(trace_inet_csk_listen_start, struct sock *sk, int backlog)
+int BPF_PROG(trace_inet_csk_listen_start, struct sock *sk)
{
set_task_info(sk);
diff --git a/tools/testing/selftests/bpf/progs/test_skb_ctx.c b/tools/testing/selftests/bpf/progs/test_skb_ctx.c
index 1d61b36..c482110 100644
--- a/tools/testing/selftests/bpf/progs/test_skb_ctx.c
+++ b/tools/testing/selftests/bpf/progs/test_skb_ctx.c
@@ -5,7 +5,7 @@
char _license[] SEC("license") = "GPL";
-SEC("skb_ctx")
+SEC("tc")
int process(struct __sk_buff *skb)
{
#pragma clang loop unroll(full)
diff --git a/tools/testing/selftests/bpf/progs/test_skc_to_unix_sock.c b/tools/testing/selftests/bpf/progs/test_skc_to_unix_sock.c
index a408ec9..eacda9f 100644
--- a/tools/testing/selftests/bpf/progs/test_skc_to_unix_sock.c
+++ b/tools/testing/selftests/bpf/progs/test_skc_to_unix_sock.c
@@ -23,7 +23,7 @@ int BPF_PROG(unix_listen, struct socket *sock, int backlog)
if (!unix_sk)
return 0;
- if (!UNIX_ABSTRACT(unix_sk))
+ if (unix_sk->addr->name->sun_path[0])
return 0;
len = unix_sk->addr->len - sizeof(short);
diff --git a/tools/testing/selftests/bpf/progs/test_spin_lock.c b/tools/testing/selftests/bpf/progs/test_spin_lock.c
index 0d31a3b..7e88309 100644
--- a/tools/testing/selftests/bpf/progs/test_spin_lock.c
+++ b/tools/testing/selftests/bpf/progs/test_spin_lock.c
@@ -45,7 +45,7 @@ struct {
#define CREDIT_PER_NS(delta, rate) (((delta) * rate) >> 20)
-SEC("spin_lock_demo")
+SEC("tc")
int bpf_sping_lock_test(struct __sk_buff *skb)
{
volatile int credit = 0, max_credit = 100, pkt_len = 64;
diff --git a/tools/testing/selftests/bpf/progs/test_tcp_estats.c b/tools/testing/selftests/bpf/progs/test_tcp_estats.c
index 2c5c602..e2ae049 100644
--- a/tools/testing/selftests/bpf/progs/test_tcp_estats.c
+++ b/tools/testing/selftests/bpf/progs/test_tcp_estats.c
@@ -244,7 +244,7 @@ static __always_inline void send_basic_event(struct sock *sk,
bpf_map_update_elem(&ev_record_map, &key, &ev, BPF_ANY);
}
-SEC("dummy_tracepoint")
+SEC("tp/dummy/tracepoint")
int _dummy_tracepoint(struct dummy_tracepoint_args *arg)
{
if (!arg->sock)
diff --git a/tools/testing/selftests/bpf/progs/test_verif_scale2.c b/tools/testing/selftests/bpf/progs/test_verif_scale2.c
index f024154..f90ffca 100644
--- a/tools/testing/selftests/bpf/progs/test_verif_scale2.c
+++ b/tools/testing/selftests/bpf/progs/test_verif_scale2.c
@@ -1,11 +1,11 @@
// SPDX-License-Identifier: GPL-2.0
// Copyright (c) 2019 Facebook
-#include <linux/bpf.h>
+#include "vmlinux.h"
#include <bpf/bpf_helpers.h>
#define ATTR __always_inline
#include "test_jhash.h"
-SEC("scale90_inline")
+SEC("tc")
int balancer_ingress(struct __sk_buff *ctx)
{
void *data_end = (void *)(long)ctx->data_end;
diff --git a/tools/testing/selftests/bpf/progs/trigger_bench.c b/tools/testing/selftests/bpf/progs/trigger_bench.c
index 9a4d095..2098f3f 100644
--- a/tools/testing/selftests/bpf/progs/trigger_bench.c
+++ b/tools/testing/selftests/bpf/progs/trigger_bench.c
@@ -52,3 +52,10 @@ int bench_trigger_fmodret(void *ctx)
__sync_add_and_fetch(&hits, 1);
return -22;
}
+
+SEC("uprobe/self/uprobe_target")
+int bench_trigger_uprobe(void *ctx)
+{
+ __sync_add_and_fetch(&hits, 1);
+ return 0;
+}
diff --git a/tools/testing/selftests/bpf/test_bpftool_synctypes.py b/tools/testing/selftests/bpf/test_bpftool_synctypes.py
index be54b73..6bf21e4 100755
--- a/tools/testing/selftests/bpf/test_bpftool_synctypes.py
+++ b/tools/testing/selftests/bpf/test_bpftool_synctypes.py
@@ -9,7 +9,15 @@
LINUX_ROOT = os.path.abspath(os.path.join(__file__,
os.pardir, os.pardir, os.pardir, os.pardir, os.pardir))
-BPFTOOL_DIR = os.path.join(LINUX_ROOT, 'tools/bpf/bpftool')
+BPFTOOL_DIR = os.getenv('BPFTOOL_DIR',
+ os.path.join(LINUX_ROOT, 'tools/bpf/bpftool'))
+BPFTOOL_BASHCOMP_DIR = os.getenv('BPFTOOL_BASHCOMP_DIR',
+ os.path.join(BPFTOOL_DIR, 'bash-completion'))
+BPFTOOL_DOC_DIR = os.getenv('BPFTOOL_DOC_DIR',
+ os.path.join(BPFTOOL_DIR, 'Documentation'))
+INCLUDE_DIR = os.getenv('INCLUDE_DIR',
+ os.path.join(LINUX_ROOT, 'tools/include'))
+
retval = 0
class BlockParser(object):
@@ -242,12 +250,6 @@
end_marker = re.compile('}\\\\n')
return self.__get_description_list(start_marker, pattern, end_marker)
- def default_options(self):
- """
- Return the default options contained in HELP_SPEC_OPTIONS
- """
- return { '-j', '--json', '-p', '--pretty', '-d', '--debug' }
-
def get_bashcomp_list(self, block_name):
"""
Search for and parse a list of type names from a variable in bash
@@ -274,7 +276,56 @@
defined in children classes.
"""
def get_options(self):
- return self.default_options().union(self.get_help_list_macro('HELP_SPEC_OPTIONS'))
+ return self.get_help_list_macro('HELP_SPEC_OPTIONS')
+
+class MainHeaderFileExtractor(SourceFileExtractor):
+ """
+ An extractor for bpftool's main.h
+ """
+ filename = os.path.join(BPFTOOL_DIR, 'main.h')
+
+ def get_common_options(self):
+ """
+ Parse the list of common options in main.h (options that apply to all
+ commands), which looks to the lists of options in other source files
+ but has different start and end markers:
+
+ "OPTIONS := { {-j|--json} [{-p|--pretty}] | {-d|--debug} | {-l|--legacy}"
+
+ Return a set containing all options, such as:
+
+ {'-p', '-d', '--legacy', '--pretty', '--debug', '--json', '-l', '-j'}
+ """
+ start_marker = re.compile(f'"OPTIONS :=')
+ pattern = re.compile('([\w-]+) ?(?:\||}[ }\]"])')
+ end_marker = re.compile('#define')
+
+ parser = InlineListParser(self.reader)
+ parser.search_block(start_marker)
+ return parser.parse(pattern, end_marker)
+
+class ManSubstitutionsExtractor(SourceFileExtractor):
+ """
+ An extractor for substitutions.rst
+ """
+ filename = os.path.join(BPFTOOL_DOC_DIR, 'substitutions.rst')
+
+ def get_common_options(self):
+ """
+ Parse the list of common options in substitutions.rst (options that
+ apply to all commands).
+
+ Return a set containing all options, such as:
+
+ {'-p', '-d', '--legacy', '--pretty', '--debug', '--json', '-l', '-j'}
+ """
+ start_marker = re.compile('\|COMMON_OPTIONS\| replace:: {')
+ pattern = re.compile('\*\*([\w/-]+)\*\*')
+ end_marker = re.compile('}$')
+
+ parser = InlineListParser(self.reader)
+ parser.search_block(start_marker)
+ return parser.parse(pattern, end_marker)
class ProgFileExtractor(SourceFileExtractor):
"""
@@ -350,7 +401,7 @@
"""
An extractor for the UAPI BPF header.
"""
- filename = os.path.join(LINUX_ROOT, 'tools/include/uapi/linux/bpf.h')
+ filename = os.path.join(INCLUDE_DIR, 'uapi/linux/bpf.h')
def get_prog_types(self):
return self.get_enum('bpf_prog_type')
@@ -374,7 +425,7 @@
"""
An extractor for bpftool-prog.rst.
"""
- filename = os.path.join(BPFTOOL_DIR, 'Documentation/bpftool-prog.rst')
+ filename = os.path.join(BPFTOOL_DOC_DIR, 'bpftool-prog.rst')
def get_attach_types(self):
return self.get_rst_list('ATTACH_TYPE')
@@ -383,7 +434,7 @@
"""
An extractor for bpftool-map.rst.
"""
- filename = os.path.join(BPFTOOL_DIR, 'Documentation/bpftool-map.rst')
+ filename = os.path.join(BPFTOOL_DOC_DIR, 'bpftool-map.rst')
def get_map_types(self):
return self.get_rst_list('TYPE')
@@ -392,7 +443,7 @@
"""
An extractor for bpftool-cgroup.rst.
"""
- filename = os.path.join(BPFTOOL_DIR, 'Documentation/bpftool-cgroup.rst')
+ filename = os.path.join(BPFTOOL_DOC_DIR, 'bpftool-cgroup.rst')
def get_attach_types(self):
return self.get_rst_list('ATTACH_TYPE')
@@ -411,7 +462,7 @@
"""
An extractor for bpftool's bash completion file.
"""
- filename = os.path.join(BPFTOOL_DIR, 'bash-completion/bpftool')
+ filename = os.path.join(BPFTOOL_BASHCOMP_DIR, 'bpftool')
def get_prog_attach_types(self):
return self.get_bashcomp_list('BPFTOOL_PROG_ATTACH_TYPES')
@@ -562,7 +613,7 @@
help_cmd_options = source_info.get_options()
source_info.close()
- man_cmd_info = ManGenericExtractor(os.path.join('Documentation', 'bpftool-' + cmd + '.rst'))
+ man_cmd_info = ManGenericExtractor(os.path.join(BPFTOOL_DOC_DIR, 'bpftool-' + cmd + '.rst'))
man_cmd_options = man_cmd_info.get_options()
man_cmd_info.close()
@@ -573,13 +624,26 @@
help_main_options = source_main_info.get_options()
source_main_info.close()
- man_main_info = ManGenericExtractor(os.path.join('Documentation', 'bpftool.rst'))
+ man_main_info = ManGenericExtractor(os.path.join(BPFTOOL_DOC_DIR, 'bpftool.rst'))
man_main_options = man_main_info.get_options()
man_main_info.close()
verify(help_main_options, man_main_options,
f'Comparing {source_main_info.filename} (do_help() OPTIONS) and {man_main_info.filename} (OPTIONS):')
+ # Compare common options (options that apply to all commands)
+
+ main_hdr_info = MainHeaderFileExtractor()
+ source_common_options = main_hdr_info.get_common_options()
+ main_hdr_info.close()
+
+ man_substitutions = ManSubstitutionsExtractor()
+ man_common_options = man_substitutions.get_common_options()
+ man_substitutions.close()
+
+ verify(source_common_options, man_common_options,
+ f'Comparing common options from {main_hdr_info.filename} (HELP_SPEC_OPTIONS) and {man_substitutions.filename}:')
+
sys.exit(retval)
if __name__ == "__main__":
diff --git a/tools/testing/selftests/bpf/test_btf.h b/tools/testing/selftests/bpf/test_btf.h
index 32c7a57..128989b 100644
--- a/tools/testing/selftests/bpf/test_btf.h
+++ b/tools/testing/selftests/bpf/test_btf.h
@@ -72,4 +72,7 @@
#define BTF_DECL_TAG_ENC(value, type, component_idx) \
BTF_TYPE_ENC(value, BTF_INFO_ENC(BTF_KIND_DECL_TAG, 0, 0), type), (component_idx)
+#define BTF_TYPE_TAG_ENC(value, type) \
+ BTF_TYPE_ENC(value, BTF_INFO_ENC(BTF_KIND_TYPE_TAG, 0, 0), type)
+
#endif /* _TEST_BTF_H */
diff --git a/tools/testing/selftests/bpf/test_cgroup_storage.c b/tools/testing/selftests/bpf/test_cgroup_storage.c
index 0cda61d..5b8314c 100644
--- a/tools/testing/selftests/bpf/test_cgroup_storage.c
+++ b/tools/testing/selftests/bpf/test_cgroup_storage.c
@@ -8,6 +8,7 @@
#include "bpf_rlimit.h"
#include "cgroup_helpers.h"
+#include "testing_helpers.h"
char bpf_log_buf[BPF_LOG_BUF_SIZE];
@@ -50,15 +51,15 @@ int main(int argc, char **argv)
goto err;
}
- map_fd = bpf_create_map(BPF_MAP_TYPE_CGROUP_STORAGE, sizeof(key),
- sizeof(value), 0, 0);
+ map_fd = bpf_map_create(BPF_MAP_TYPE_CGROUP_STORAGE, NULL, sizeof(key),
+ sizeof(value), 0, NULL);
if (map_fd < 0) {
printf("Failed to create map: %s\n", strerror(errno));
goto out;
}
- percpu_map_fd = bpf_create_map(BPF_MAP_TYPE_PERCPU_CGROUP_STORAGE,
- sizeof(key), sizeof(value), 0, 0);
+ percpu_map_fd = bpf_map_create(BPF_MAP_TYPE_PERCPU_CGROUP_STORAGE, NULL,
+ sizeof(key), sizeof(value), 0, NULL);
if (percpu_map_fd < 0) {
printf("Failed to create map: %s\n", strerror(errno));
goto out;
@@ -66,7 +67,7 @@ int main(int argc, char **argv)
prog[0].imm = percpu_map_fd;
prog[7].imm = map_fd;
- prog_fd = bpf_load_program(BPF_PROG_TYPE_CGROUP_SKB,
+ prog_fd = bpf_test_load_program(BPF_PROG_TYPE_CGROUP_SKB,
prog, insns_cnt, "GPL", 0,
bpf_log_buf, BPF_LOG_BUF_SIZE);
if (prog_fd < 0) {
diff --git a/tools/testing/selftests/bpf/test_cpp.cpp b/tools/testing/selftests/bpf/test_cpp.cpp
index a8d2e9a..e00201d 100644
--- a/tools/testing/selftests/bpf/test_cpp.cpp
+++ b/tools/testing/selftests/bpf/test_cpp.cpp
@@ -7,9 +7,15 @@
/* do nothing, just make sure we can link successfully */
+static void dump_printf(void *ctx, const char *fmt, va_list args)
+{
+}
+
int main(int argc, char *argv[])
{
+ struct btf_dump_opts opts = { };
struct test_core_extern *skel;
+ struct btf *btf;
/* libbpf.h */
libbpf_set_print(NULL);
@@ -18,7 +24,8 @@
bpf_prog_get_fd_by_id(0);
/* btf.h */
- btf__new(NULL, 0);
+ btf = btf__new(NULL, 0);
+ btf_dump__new(btf, dump_printf, nullptr, &opts);
/* BPF skeleton */
skel = test_core_extern__open_and_load();
diff --git a/tools/testing/selftests/bpf/test_dev_cgroup.c b/tools/testing/selftests/bpf/test_dev_cgroup.c
index 804dddd..c299d34 100644
--- a/tools/testing/selftests/bpf/test_dev_cgroup.c
+++ b/tools/testing/selftests/bpf/test_dev_cgroup.c
@@ -14,6 +14,7 @@
#include <bpf/libbpf.h>
#include "cgroup_helpers.h"
+#include "testing_helpers.h"
#include "bpf_rlimit.h"
#define DEV_CGROUP_PROG "./dev_cgroup.o"
@@ -27,7 +28,7 @@ int main(int argc, char **argv)
int prog_fd, cgroup_fd;
__u32 prog_cnt;
- if (bpf_prog_load(DEV_CGROUP_PROG, BPF_PROG_TYPE_CGROUP_DEVICE,
+ if (bpf_prog_test_load(DEV_CGROUP_PROG, BPF_PROG_TYPE_CGROUP_DEVICE,
&obj, &prog_fd)) {
printf("Failed to load DEV_CGROUP program\n");
goto out;
diff --git a/tools/testing/selftests/bpf/test_lirc_mode2_user.c b/tools/testing/selftests/bpf/test_lirc_mode2_user.c
index fb5fd68..ebf68dc 100644
--- a/tools/testing/selftests/bpf/test_lirc_mode2_user.c
+++ b/tools/testing/selftests/bpf/test_lirc_mode2_user.c
@@ -45,6 +45,8 @@
#include <bpf/bpf.h>
#include <bpf/libbpf.h>
+#include "testing_helpers.h"
+
int main(int argc, char **argv)
{
struct bpf_object *obj;
@@ -58,8 +60,8 @@ int main(int argc, char **argv)
return 2;
}
- ret = bpf_prog_load("test_lirc_mode2_kern.o",
- BPF_PROG_TYPE_LIRC_MODE2, &obj, &progfd);
+ ret = bpf_prog_test_load("test_lirc_mode2_kern.o",
+ BPF_PROG_TYPE_LIRC_MODE2, &obj, &progfd);
if (ret) {
printf("Failed to load bpf program\n");
return 1;
diff --git a/tools/testing/selftests/bpf/test_lpm_map.c b/tools/testing/selftests/bpf/test_lpm_map.c
index 006be39..baa3e3e 100644
--- a/tools/testing/selftests/bpf/test_lpm_map.c
+++ b/tools/testing/selftests/bpf/test_lpm_map.c
@@ -208,6 +208,7 @@ static void test_lpm_order(void)
static void test_lpm_map(int keysize)
{
+ LIBBPF_OPTS(bpf_map_create_opts, opts, .map_flags = BPF_F_NO_PREALLOC);
size_t i, j, n_matches, n_matches_after_delete, n_nodes, n_lookups;
struct tlpm_node *t, *list = NULL;
struct bpf_lpm_trie_key *key;
@@ -233,11 +234,11 @@ static void test_lpm_map(int keysize)
key = alloca(sizeof(*key) + keysize);
memset(key, 0, sizeof(*key) + keysize);
- map = bpf_create_map(BPF_MAP_TYPE_LPM_TRIE,
+ map = bpf_map_create(BPF_MAP_TYPE_LPM_TRIE, NULL,
sizeof(*key) + keysize,
keysize + 1,
4096,
- BPF_F_NO_PREALLOC);
+ &opts);
assert(map >= 0);
for (i = 0; i < n_nodes; ++i) {
@@ -329,6 +330,7 @@ static void test_lpm_map(int keysize)
static void test_lpm_ipaddr(void)
{
+ LIBBPF_OPTS(bpf_map_create_opts, opts, .map_flags = BPF_F_NO_PREALLOC);
struct bpf_lpm_trie_key *key_ipv4;
struct bpf_lpm_trie_key *key_ipv6;
size_t key_size_ipv4;
@@ -342,14 +344,14 @@ static void test_lpm_ipaddr(void)
key_ipv4 = alloca(key_size_ipv4);
key_ipv6 = alloca(key_size_ipv6);
- map_fd_ipv4 = bpf_create_map(BPF_MAP_TYPE_LPM_TRIE,
+ map_fd_ipv4 = bpf_map_create(BPF_MAP_TYPE_LPM_TRIE, NULL,
key_size_ipv4, sizeof(value),
- 100, BPF_F_NO_PREALLOC);
+ 100, &opts);
assert(map_fd_ipv4 >= 0);
- map_fd_ipv6 = bpf_create_map(BPF_MAP_TYPE_LPM_TRIE,
+ map_fd_ipv6 = bpf_map_create(BPF_MAP_TYPE_LPM_TRIE, NULL,
key_size_ipv6, sizeof(value),
- 100, BPF_F_NO_PREALLOC);
+ 100, &opts);
assert(map_fd_ipv6 >= 0);
/* Fill data some IPv4 and IPv6 address ranges */
@@ -423,6 +425,7 @@ static void test_lpm_ipaddr(void)
static void test_lpm_delete(void)
{
+ LIBBPF_OPTS(bpf_map_create_opts, opts, .map_flags = BPF_F_NO_PREALLOC);
struct bpf_lpm_trie_key *key;
size_t key_size;
int map_fd;
@@ -431,9 +434,9 @@ static void test_lpm_delete(void)
key_size = sizeof(*key) + sizeof(__u32);
key = alloca(key_size);
- map_fd = bpf_create_map(BPF_MAP_TYPE_LPM_TRIE,
+ map_fd = bpf_map_create(BPF_MAP_TYPE_LPM_TRIE, NULL,
key_size, sizeof(value),
- 100, BPF_F_NO_PREALLOC);
+ 100, &opts);
assert(map_fd >= 0);
/* Add nodes:
@@ -535,6 +538,7 @@ static void test_lpm_delete(void)
static void test_lpm_get_next_key(void)
{
+ LIBBPF_OPTS(bpf_map_create_opts, opts, .map_flags = BPF_F_NO_PREALLOC);
struct bpf_lpm_trie_key *key_p, *next_key_p;
size_t key_size;
__u32 value = 0;
@@ -544,8 +548,7 @@ static void test_lpm_get_next_key(void)
key_p = alloca(key_size);
next_key_p = alloca(key_size);
- map_fd = bpf_create_map(BPF_MAP_TYPE_LPM_TRIE, key_size, sizeof(value),
- 100, BPF_F_NO_PREALLOC);
+ map_fd = bpf_map_create(BPF_MAP_TYPE_LPM_TRIE, NULL, key_size, sizeof(value), 100, &opts);
assert(map_fd >= 0);
/* empty tree. get_next_key should return ENOENT */
@@ -753,6 +756,7 @@ static void setup_lpm_mt_test_info(struct lpm_mt_test_info *info, int map_fd)
static void test_lpm_multi_thread(void)
{
+ LIBBPF_OPTS(bpf_map_create_opts, opts, .map_flags = BPF_F_NO_PREALLOC);
struct lpm_mt_test_info info[4];
size_t key_size, value_size;
pthread_t thread_id[4];
@@ -762,8 +766,7 @@ static void test_lpm_multi_thread(void)
/* create a trie */
value_size = sizeof(__u32);
key_size = sizeof(struct bpf_lpm_trie_key) + value_size;
- map_fd = bpf_create_map(BPF_MAP_TYPE_LPM_TRIE, key_size, value_size,
- 100, BPF_F_NO_PREALLOC);
+ map_fd = bpf_map_create(BPF_MAP_TYPE_LPM_TRIE, NULL, key_size, value_size, 100, &opts);
/* create 4 threads to test update, delete, lookup and get_next_key */
setup_lpm_mt_test_info(&info[0], map_fd);
diff --git a/tools/testing/selftests/bpf/test_lru_map.c b/tools/testing/selftests/bpf/test_lru_map.c
index 7e9049f..b9f1bbb 100644
--- a/tools/testing/selftests/bpf/test_lru_map.c
+++ b/tools/testing/selftests/bpf/test_lru_map.c
@@ -28,13 +28,14 @@ static int nr_cpus;
static int create_map(int map_type, int map_flags, unsigned int size)
{
+ LIBBPF_OPTS(bpf_map_create_opts, opts, .map_flags = map_flags);
int map_fd;
- map_fd = bpf_create_map(map_type, sizeof(unsigned long long),
- sizeof(unsigned long long), size, map_flags);
+ map_fd = bpf_map_create(map_type, NULL, sizeof(unsigned long long),
+ sizeof(unsigned long long), size, &opts);
if (map_fd == -1)
- perror("bpf_create_map");
+ perror("bpf_map_create");
return map_fd;
}
@@ -42,8 +43,6 @@ static int create_map(int map_type, int map_flags, unsigned int size)
static int bpf_map_lookup_elem_with_ref_bit(int fd, unsigned long long key,
void *value)
{
- struct bpf_load_program_attr prog;
- struct bpf_create_map_attr map;
struct bpf_insn insns[] = {
BPF_LD_MAP_VALUE(BPF_REG_9, 0, 0),
BPF_LD_MAP_FD(BPF_REG_1, fd),
@@ -64,25 +63,13 @@ static int bpf_map_lookup_elem_with_ref_bit(int fd, unsigned long long key,
int mfd, pfd, ret, zero = 0;
__u32 retval = 0;
- memset(&map, 0, sizeof(map));
- map.map_type = BPF_MAP_TYPE_ARRAY;
- map.key_size = sizeof(int);
- map.value_size = sizeof(unsigned long long);
- map.max_entries = 1;
-
- mfd = bpf_create_map_xattr(&map);
+ mfd = bpf_map_create(BPF_MAP_TYPE_ARRAY, NULL, sizeof(int), sizeof(__u64), 1, NULL);
if (mfd < 0)
return -1;
insns[0].imm = mfd;
- memset(&prog, 0, sizeof(prog));
- prog.prog_type = BPF_PROG_TYPE_SCHED_CLS;
- prog.insns = insns;
- prog.insns_cnt = ARRAY_SIZE(insns);
- prog.license = "GPL";
-
- pfd = bpf_load_program_xattr(&prog, NULL, 0);
+ pfd = bpf_prog_load(BPF_PROG_TYPE_SCHED_CLS, NULL, "GPL", insns, ARRAY_SIZE(insns), NULL);
if (pfd < 0) {
close(mfd);
return -1;
diff --git a/tools/testing/selftests/bpf/test_maps.c b/tools/testing/selftests/bpf/test_maps.c
index c7a36a9..50f7e74 100644
--- a/tools/testing/selftests/bpf/test_maps.c
+++ b/tools/testing/selftests/bpf/test_maps.c
@@ -23,8 +23,8 @@
#include <bpf/libbpf.h>
#include "bpf_util.h"
-#include "bpf_rlimit.h"
#include "test_maps.h"
+#include "testing_helpers.h"
#ifndef ENOTSUPP
#define ENOTSUPP 524
@@ -32,15 +32,14 @@
static int skips;
-static int map_flags;
+static struct bpf_map_create_opts map_opts = { .sz = sizeof(map_opts) };
static void test_hashmap(unsigned int task, void *data)
{
long long key, next_key, first_key, value;
int fd;
- fd = bpf_create_map(BPF_MAP_TYPE_HASH, sizeof(key), sizeof(value),
- 2, map_flags);
+ fd = bpf_map_create(BPF_MAP_TYPE_HASH, NULL, sizeof(key), sizeof(value), 2, &map_opts);
if (fd < 0) {
printf("Failed to create hashmap '%s'!\n", strerror(errno));
exit(1);
@@ -137,8 +136,7 @@ static void test_hashmap_sizes(unsigned int task, void *data)
for (i = 1; i <= 512; i <<= 1)
for (j = 1; j <= 1 << 18; j <<= 1) {
- fd = bpf_create_map(BPF_MAP_TYPE_HASH, i, j,
- 2, map_flags);
+ fd = bpf_map_create(BPF_MAP_TYPE_HASH, NULL, i, j, 2, &map_opts);
if (fd < 0) {
if (errno == ENOMEM)
return;
@@ -159,8 +157,8 @@ static void test_hashmap_percpu(unsigned int task, void *data)
int expected_key_mask = 0;
int fd, i;
- fd = bpf_create_map(BPF_MAP_TYPE_PERCPU_HASH, sizeof(key),
- sizeof(bpf_percpu(value, 0)), 2, map_flags);
+ fd = bpf_map_create(BPF_MAP_TYPE_PERCPU_HASH, NULL, sizeof(key),
+ sizeof(bpf_percpu(value, 0)), 2, &map_opts);
if (fd < 0) {
printf("Failed to create hashmap '%s'!\n", strerror(errno));
exit(1);
@@ -271,11 +269,11 @@ static int helper_fill_hashmap(int max_entries)
int i, fd, ret;
long long key, value;
- fd = bpf_create_map(BPF_MAP_TYPE_HASH, sizeof(key), sizeof(value),
- max_entries, map_flags);
+ fd = bpf_map_create(BPF_MAP_TYPE_HASH, NULL, sizeof(key), sizeof(value),
+ max_entries, &map_opts);
CHECK(fd < 0,
"failed to create hashmap",
- "err: %s, flags: 0x%x\n", strerror(errno), map_flags);
+ "err: %s, flags: 0x%x\n", strerror(errno), map_opts.map_flags);
for (i = 0; i < max_entries; i++) {
key = i; value = key;
@@ -331,8 +329,8 @@ static void test_hashmap_zero_seed(void)
int i, first, second, old_flags;
long long key, next_first, next_second;
- old_flags = map_flags;
- map_flags |= BPF_F_ZERO_SEED;
+ old_flags = map_opts.map_flags;
+ map_opts.map_flags |= BPF_F_ZERO_SEED;
first = helper_fill_hashmap(3);
second = helper_fill_hashmap(3);
@@ -354,7 +352,7 @@ static void test_hashmap_zero_seed(void)
key = next_first;
}
- map_flags = old_flags;
+ map_opts.map_flags = old_flags;
close(first);
close(second);
}
@@ -364,8 +362,7 @@ static void test_arraymap(unsigned int task, void *data)
int key, next_key, fd;
long long value;
- fd = bpf_create_map(BPF_MAP_TYPE_ARRAY, sizeof(key), sizeof(value),
- 2, 0);
+ fd = bpf_map_create(BPF_MAP_TYPE_ARRAY, NULL, sizeof(key), sizeof(value), 2, NULL);
if (fd < 0) {
printf("Failed to create arraymap '%s'!\n", strerror(errno));
exit(1);
@@ -420,8 +417,8 @@ static void test_arraymap_percpu(unsigned int task, void *data)
BPF_DECLARE_PERCPU(long, values);
int key, next_key, fd, i;
- fd = bpf_create_map(BPF_MAP_TYPE_PERCPU_ARRAY, sizeof(key),
- sizeof(bpf_percpu(values, 0)), 2, 0);
+ fd = bpf_map_create(BPF_MAP_TYPE_PERCPU_ARRAY, NULL, sizeof(key),
+ sizeof(bpf_percpu(values, 0)), 2, NULL);
if (fd < 0) {
printf("Failed to create arraymap '%s'!\n", strerror(errno));
exit(1);
@@ -483,8 +480,8 @@ static void test_arraymap_percpu_many_keys(void)
unsigned int nr_keys = 2000;
int key, fd, i;
- fd = bpf_create_map(BPF_MAP_TYPE_PERCPU_ARRAY, sizeof(key),
- sizeof(bpf_percpu(values, 0)), nr_keys, 0);
+ fd = bpf_map_create(BPF_MAP_TYPE_PERCPU_ARRAY, NULL, sizeof(key),
+ sizeof(bpf_percpu(values, 0)), nr_keys, NULL);
if (fd < 0) {
printf("Failed to create per-cpu arraymap '%s'!\n",
strerror(errno));
@@ -515,8 +512,7 @@ static void test_devmap(unsigned int task, void *data)
int fd;
__u32 key, value;
- fd = bpf_create_map(BPF_MAP_TYPE_DEVMAP, sizeof(key), sizeof(value),
- 2, 0);
+ fd = bpf_map_create(BPF_MAP_TYPE_DEVMAP, NULL, sizeof(key), sizeof(value), 2, NULL);
if (fd < 0) {
printf("Failed to create devmap '%s'!\n", strerror(errno));
exit(1);
@@ -530,8 +526,7 @@ static void test_devmap_hash(unsigned int task, void *data)
int fd;
__u32 key, value;
- fd = bpf_create_map(BPF_MAP_TYPE_DEVMAP_HASH, sizeof(key), sizeof(value),
- 2, 0);
+ fd = bpf_map_create(BPF_MAP_TYPE_DEVMAP_HASH, NULL, sizeof(key), sizeof(value), 2, NULL);
if (fd < 0) {
printf("Failed to create devmap_hash '%s'!\n", strerror(errno));
exit(1);
@@ -551,14 +546,12 @@ static void test_queuemap(unsigned int task, void *data)
vals[i] = rand();
/* Invalid key size */
- fd = bpf_create_map(BPF_MAP_TYPE_QUEUE, 4, sizeof(val), MAP_SIZE,
- map_flags);
+ fd = bpf_map_create(BPF_MAP_TYPE_QUEUE, NULL, 4, sizeof(val), MAP_SIZE, &map_opts);
assert(fd < 0 && errno == EINVAL);
- fd = bpf_create_map(BPF_MAP_TYPE_QUEUE, 0, sizeof(val), MAP_SIZE,
- map_flags);
+ fd = bpf_map_create(BPF_MAP_TYPE_QUEUE, NULL, 0, sizeof(val), MAP_SIZE, &map_opts);
/* Queue map does not support BPF_F_NO_PREALLOC */
- if (map_flags & BPF_F_NO_PREALLOC) {
+ if (map_opts.map_flags & BPF_F_NO_PREALLOC) {
assert(fd < 0 && errno == EINVAL);
return;
}
@@ -609,14 +602,12 @@ static void test_stackmap(unsigned int task, void *data)
vals[i] = rand();
/* Invalid key size */
- fd = bpf_create_map(BPF_MAP_TYPE_STACK, 4, sizeof(val), MAP_SIZE,
- map_flags);
+ fd = bpf_map_create(BPF_MAP_TYPE_STACK, NULL, 4, sizeof(val), MAP_SIZE, &map_opts);
assert(fd < 0 && errno == EINVAL);
- fd = bpf_create_map(BPF_MAP_TYPE_STACK, 0, sizeof(val), MAP_SIZE,
- map_flags);
+ fd = bpf_map_create(BPF_MAP_TYPE_STACK, NULL, 0, sizeof(val), MAP_SIZE, &map_opts);
/* Stack map does not support BPF_F_NO_PREALLOC */
- if (map_flags & BPF_F_NO_PREALLOC) {
+ if (map_opts.map_flags & BPF_F_NO_PREALLOC) {
assert(fd < 0 && errno == EINVAL);
return;
}
@@ -743,9 +734,9 @@ static void test_sockmap(unsigned int tasks, void *data)
}
/* Test sockmap with connected sockets */
- fd = bpf_create_map(BPF_MAP_TYPE_SOCKMAP,
+ fd = bpf_map_create(BPF_MAP_TYPE_SOCKMAP, NULL,
sizeof(key), sizeof(value),
- 6, 0);
+ 6, NULL);
if (fd < 0) {
if (!bpf_probe_map_type(BPF_MAP_TYPE_SOCKMAP, 0)) {
printf("%s SKIP (unsupported map type BPF_MAP_TYPE_SOCKMAP)\n",
@@ -830,21 +821,21 @@ static void test_sockmap(unsigned int tasks, void *data)
}
/* Load SK_SKB program and Attach */
- err = bpf_prog_load(SOCKMAP_PARSE_PROG,
+ err = bpf_prog_test_load(SOCKMAP_PARSE_PROG,
BPF_PROG_TYPE_SK_SKB, &obj, &parse_prog);
if (err) {
printf("Failed to load SK_SKB parse prog\n");
goto out_sockmap;
}
- err = bpf_prog_load(SOCKMAP_TCP_MSG_PROG,
+ err = bpf_prog_test_load(SOCKMAP_TCP_MSG_PROG,
BPF_PROG_TYPE_SK_MSG, &obj, &msg_prog);
if (err) {
printf("Failed to load SK_SKB msg prog\n");
goto out_sockmap;
}
- err = bpf_prog_load(SOCKMAP_VERDICT_PROG,
+ err = bpf_prog_test_load(SOCKMAP_VERDICT_PROG,
BPF_PROG_TYPE_SK_SKB, &obj, &verdict_prog);
if (err) {
printf("Failed to load SK_SKB verdict prog\n");
@@ -1167,8 +1158,7 @@ static void test_map_in_map(void)
obj = bpf_object__open(MAPINMAP_PROG);
- fd = bpf_create_map(BPF_MAP_TYPE_HASH, sizeof(int), sizeof(int),
- 2, 0);
+ fd = bpf_map_create(BPF_MAP_TYPE_HASH, NULL, sizeof(int), sizeof(int), 2, NULL);
if (fd < 0) {
printf("Failed to create hashmap '%s'!\n", strerror(errno));
exit(1);
@@ -1314,8 +1304,8 @@ static void test_map_large(void)
} key;
int fd, i, value;
- fd = bpf_create_map(BPF_MAP_TYPE_HASH, sizeof(key), sizeof(value),
- MAP_SIZE, map_flags);
+ fd = bpf_map_create(BPF_MAP_TYPE_HASH, NULL, sizeof(key), sizeof(value),
+ MAP_SIZE, &map_opts);
if (fd < 0) {
printf("Failed to create large map '%s'!\n", strerror(errno));
exit(1);
@@ -1468,8 +1458,8 @@ static void test_map_parallel(void)
int i, fd, key = 0, value = 0;
int data[2];
- fd = bpf_create_map(BPF_MAP_TYPE_HASH, sizeof(key), sizeof(value),
- MAP_SIZE, map_flags);
+ fd = bpf_map_create(BPF_MAP_TYPE_HASH, NULL, sizeof(key), sizeof(value),
+ MAP_SIZE, &map_opts);
if (fd < 0) {
printf("Failed to create map for parallel test '%s'!\n",
strerror(errno));
@@ -1517,9 +1507,13 @@ static void test_map_parallel(void)
static void test_map_rdonly(void)
{
int fd, key = 0, value = 0;
+ __u32 old_flags;
- fd = bpf_create_map(BPF_MAP_TYPE_HASH, sizeof(key), sizeof(value),
- MAP_SIZE, map_flags | BPF_F_RDONLY);
+ old_flags = map_opts.map_flags;
+ map_opts.map_flags |= BPF_F_RDONLY;
+ fd = bpf_map_create(BPF_MAP_TYPE_HASH, NULL, sizeof(key), sizeof(value),
+ MAP_SIZE, &map_opts);
+ map_opts.map_flags = old_flags;
if (fd < 0) {
printf("Failed to create map for read only test '%s'!\n",
strerror(errno));
@@ -1542,9 +1536,13 @@ static void test_map_rdonly(void)
static void test_map_wronly_hash(void)
{
int fd, key = 0, value = 0;
+ __u32 old_flags;
- fd = bpf_create_map(BPF_MAP_TYPE_HASH, sizeof(key), sizeof(value),
- MAP_SIZE, map_flags | BPF_F_WRONLY);
+ old_flags = map_opts.map_flags;
+ map_opts.map_flags |= BPF_F_WRONLY;
+ fd = bpf_map_create(BPF_MAP_TYPE_HASH, NULL, sizeof(key), sizeof(value),
+ MAP_SIZE, &map_opts);
+ map_opts.map_flags = old_flags;
if (fd < 0) {
printf("Failed to create map for write only test '%s'!\n",
strerror(errno));
@@ -1566,13 +1564,17 @@ static void test_map_wronly_hash(void)
static void test_map_wronly_stack_or_queue(enum bpf_map_type map_type)
{
int fd, value = 0;
+ __u32 old_flags;
+
assert(map_type == BPF_MAP_TYPE_QUEUE ||
map_type == BPF_MAP_TYPE_STACK);
- fd = bpf_create_map(map_type, 0, sizeof(value), MAP_SIZE,
- map_flags | BPF_F_WRONLY);
+ old_flags = map_opts.map_flags;
+ map_opts.map_flags |= BPF_F_WRONLY;
+ fd = bpf_map_create(map_type, NULL, 0, sizeof(value), MAP_SIZE, &map_opts);
+ map_opts.map_flags = old_flags;
/* Stack/Queue maps do not support BPF_F_NO_PREALLOC */
- if (map_flags & BPF_F_NO_PREALLOC) {
+ if (map_opts.map_flags & BPF_F_NO_PREALLOC) {
assert(fd < 0 && errno == EINVAL);
return;
}
@@ -1699,8 +1701,8 @@ static void test_reuseport_array(void)
__u32 fds_idx = 0;
int fd;
- map_fd = bpf_create_map(BPF_MAP_TYPE_REUSEPORT_SOCKARRAY,
- sizeof(__u32), sizeof(__u64), array_size, 0);
+ map_fd = bpf_map_create(BPF_MAP_TYPE_REUSEPORT_SOCKARRAY, NULL,
+ sizeof(__u32), sizeof(__u64), array_size, NULL);
CHECK(map_fd < 0, "reuseport array create",
"map_fd:%d, errno:%d\n", map_fd, errno);
@@ -1836,8 +1838,8 @@ static void test_reuseport_array(void)
close(map_fd);
/* Test 32 bit fd */
- map_fd = bpf_create_map(BPF_MAP_TYPE_REUSEPORT_SOCKARRAY,
- sizeof(__u32), sizeof(__u32), array_size, 0);
+ map_fd = bpf_map_create(BPF_MAP_TYPE_REUSEPORT_SOCKARRAY, NULL,
+ sizeof(__u32), sizeof(__u32), array_size, NULL);
CHECK(map_fd < 0, "reuseport array create",
"map_fd:%d, errno:%d\n", map_fd, errno);
prepare_reuseport_grp(SOCK_STREAM, map_fd, sizeof(__u32), &fd64,
@@ -1895,10 +1897,10 @@ int main(void)
libbpf_set_strict_mode(LIBBPF_STRICT_ALL);
- map_flags = 0;
+ map_opts.map_flags = 0;
run_all_tests();
- map_flags = BPF_F_NO_PREALLOC;
+ map_opts.map_flags = BPF_F_NO_PREALLOC;
run_all_tests();
#define DEFINE_TEST(name) test_##name();
diff --git a/tools/testing/selftests/bpf/test_progs.c b/tools/testing/selftests/bpf/test_progs.c
index c65986b..2ecb73a 100644
--- a/tools/testing/selftests/bpf/test_progs.c
+++ b/tools/testing/selftests/bpf/test_progs.c
@@ -4,7 +4,6 @@
#define _GNU_SOURCE
#include "test_progs.h"
#include "cgroup_helpers.h"
-#include "bpf_rlimit.h"
#include <argp.h>
#include <pthread.h>
#include <sched.h>
@@ -473,11 +472,11 @@ static struct prog_test_def prog_test_defs[] = {
#include <prog_tests/tests.h>
#undef DEFINE_TEST
};
-const int prog_test_cnt = ARRAY_SIZE(prog_test_defs);
+static const int prog_test_cnt = ARRAY_SIZE(prog_test_defs);
const char *argp_program_version = "test_progs 0.1";
const char *argp_program_bug_address = "<bpf@vger.kernel.org>";
-const char argp_program_doc[] = "BPF selftests test runner";
+static const char argp_program_doc[] = "BPF selftests test runner";
enum ARG_KEYS {
ARG_TEST_NUM = 'n',
@@ -939,7 +938,7 @@ static void *dispatch_thread(void *ctx)
{
struct dispatch_data *data = ctx;
int sock_fd;
- FILE *log_fd = NULL;
+ FILE *log_fp = NULL;
sock_fd = data->sock_fd;
@@ -1002,8 +1001,8 @@ static void *dispatch_thread(void *ctx)
/* collect all logs */
if (msg_test_done.test_done.have_log) {
- log_fd = open_memstream(&result->log_buf, &result->log_cnt);
- if (!log_fd)
+ log_fp = open_memstream(&result->log_buf, &result->log_cnt);
+ if (!log_fp)
goto error;
while (true) {
@@ -1014,12 +1013,12 @@ static void *dispatch_thread(void *ctx)
if (msg_log.type != MSG_TEST_LOG)
goto error;
- fprintf(log_fd, "%s", msg_log.test_log.log_buf);
+ fprintf(log_fp, "%s", msg_log.test_log.log_buf);
if (msg_log.test_log.is_last)
break;
}
- fclose(log_fd);
- log_fd = NULL;
+ fclose(log_fp);
+ log_fp = NULL;
}
/* output log */
{
@@ -1045,8 +1044,8 @@ static void *dispatch_thread(void *ctx)
if (env.debug)
fprintf(stderr, "[%d]: Protocol/IO error: %s.\n", data->worker_id, strerror(errno));
- if (log_fd)
- fclose(log_fd);
+ if (log_fp)
+ fclose(log_fp);
done:
{
struct msg msg_exit;
@@ -1198,11 +1197,11 @@ static int server_main(void)
env.sub_succ_cnt += result->sub_succ_cnt;
}
+ print_all_error_logs();
+
fprintf(stdout, "Summary: %d/%d PASSED, %d SKIPPED, %d FAILED\n",
env.succ_cnt, env.sub_succ_cnt, env.skip_cnt, env.fail_cnt);
- print_all_error_logs();
-
/* reap all workers */
for (i = 0; i < env.workers; i++) {
int wstatus, pid;
@@ -1342,7 +1341,6 @@ int main(int argc, char **argv)
/* Use libbpf 1.0 API mode */
libbpf_set_strict_mode(LIBBPF_STRICT_ALL);
-
libbpf_set_print(libbpf_print_fn);
srand(time(NULL));
@@ -1484,11 +1482,11 @@ int main(int argc, char **argv)
if (env.list_test_names)
goto out;
+ print_all_error_logs();
+
fprintf(stdout, "Summary: %d/%d PASSED, %d SKIPPED, %d FAILED\n",
env.succ_cnt, env.sub_succ_cnt, env.skip_cnt, env.fail_cnt);
- print_all_error_logs();
-
close(env.saved_netns_fd);
out:
if (!env.list_test_names && env.has_testmod)
diff --git a/tools/testing/selftests/bpf/test_sock.c b/tools/testing/selftests/bpf/test_sock.c
index 9613f753..fe10f81 100644
--- a/tools/testing/selftests/bpf/test_sock.c
+++ b/tools/testing/selftests/bpf/test_sock.c
@@ -35,18 +35,21 @@ struct sock_test {
/* Endpoint to bind() to */
const char *ip;
unsigned short port;
+ unsigned short port_retry;
/* Expected test result */
enum {
LOAD_REJECT,
ATTACH_REJECT,
BIND_REJECT,
SUCCESS,
+ RETRY_SUCCESS,
+ RETRY_REJECT
} result;
};
static struct sock_test tests[] = {
{
- "bind4 load with invalid access: src_ip6",
+ .descr = "bind4 load with invalid access: src_ip6",
.insns = {
BPF_MOV64_REG(BPF_REG_6, BPF_REG_1),
BPF_LDX_MEM(BPF_W, BPF_REG_7, BPF_REG_6,
@@ -54,16 +57,12 @@ static struct sock_test tests[] = {
BPF_MOV64_IMM(BPF_REG_0, 1),
BPF_EXIT_INSN(),
},
- BPF_CGROUP_INET4_POST_BIND,
- BPF_CGROUP_INET4_POST_BIND,
- 0,
- 0,
- NULL,
- 0,
- LOAD_REJECT,
+ .expected_attach_type = BPF_CGROUP_INET4_POST_BIND,
+ .attach_type = BPF_CGROUP_INET4_POST_BIND,
+ .result = LOAD_REJECT,
},
{
- "bind4 load with invalid access: mark",
+ .descr = "bind4 load with invalid access: mark",
.insns = {
BPF_MOV64_REG(BPF_REG_6, BPF_REG_1),
BPF_LDX_MEM(BPF_W, BPF_REG_7, BPF_REG_6,
@@ -71,16 +70,12 @@ static struct sock_test tests[] = {
BPF_MOV64_IMM(BPF_REG_0, 1),
BPF_EXIT_INSN(),
},
- BPF_CGROUP_INET4_POST_BIND,
- BPF_CGROUP_INET4_POST_BIND,
- 0,
- 0,
- NULL,
- 0,
- LOAD_REJECT,
+ .expected_attach_type = BPF_CGROUP_INET4_POST_BIND,
+ .attach_type = BPF_CGROUP_INET4_POST_BIND,
+ .result = LOAD_REJECT,
},
{
- "bind6 load with invalid access: src_ip4",
+ .descr = "bind6 load with invalid access: src_ip4",
.insns = {
BPF_MOV64_REG(BPF_REG_6, BPF_REG_1),
BPF_LDX_MEM(BPF_W, BPF_REG_7, BPF_REG_6,
@@ -88,16 +83,12 @@ static struct sock_test tests[] = {
BPF_MOV64_IMM(BPF_REG_0, 1),
BPF_EXIT_INSN(),
},
- BPF_CGROUP_INET6_POST_BIND,
- BPF_CGROUP_INET6_POST_BIND,
- 0,
- 0,
- NULL,
- 0,
- LOAD_REJECT,
+ .expected_attach_type = BPF_CGROUP_INET6_POST_BIND,
+ .attach_type = BPF_CGROUP_INET6_POST_BIND,
+ .result = LOAD_REJECT,
},
{
- "sock_create load with invalid access: src_port",
+ .descr = "sock_create load with invalid access: src_port",
.insns = {
BPF_MOV64_REG(BPF_REG_6, BPF_REG_1),
BPF_LDX_MEM(BPF_W, BPF_REG_7, BPF_REG_6,
@@ -105,128 +96,106 @@ static struct sock_test tests[] = {
BPF_MOV64_IMM(BPF_REG_0, 1),
BPF_EXIT_INSN(),
},
- BPF_CGROUP_INET_SOCK_CREATE,
- BPF_CGROUP_INET_SOCK_CREATE,
- 0,
- 0,
- NULL,
- 0,
- LOAD_REJECT,
+ .expected_attach_type = BPF_CGROUP_INET_SOCK_CREATE,
+ .attach_type = BPF_CGROUP_INET_SOCK_CREATE,
+ .result = LOAD_REJECT,
},
{
- "sock_create load w/o expected_attach_type (compat mode)",
+ .descr = "sock_create load w/o expected_attach_type (compat mode)",
.insns = {
BPF_MOV64_IMM(BPF_REG_0, 1),
BPF_EXIT_INSN(),
},
- 0,
- BPF_CGROUP_INET_SOCK_CREATE,
- AF_INET,
- SOCK_STREAM,
- "127.0.0.1",
- 8097,
- SUCCESS,
+ .expected_attach_type = 0,
+ .attach_type = BPF_CGROUP_INET_SOCK_CREATE,
+ .domain = AF_INET,
+ .type = SOCK_STREAM,
+ .ip = "127.0.0.1",
+ .port = 8097,
+ .result = SUCCESS,
},
{
- "sock_create load w/ expected_attach_type",
+ .descr = "sock_create load w/ expected_attach_type",
.insns = {
BPF_MOV64_IMM(BPF_REG_0, 1),
BPF_EXIT_INSN(),
},
- BPF_CGROUP_INET_SOCK_CREATE,
- BPF_CGROUP_INET_SOCK_CREATE,
- AF_INET,
- SOCK_STREAM,
- "127.0.0.1",
- 8097,
- SUCCESS,
+ .expected_attach_type = BPF_CGROUP_INET_SOCK_CREATE,
+ .attach_type = BPF_CGROUP_INET_SOCK_CREATE,
+ .domain = AF_INET,
+ .type = SOCK_STREAM,
+ .ip = "127.0.0.1",
+ .port = 8097,
+ .result = SUCCESS,
},
{
- "attach type mismatch bind4 vs bind6",
+ .descr = "attach type mismatch bind4 vs bind6",
.insns = {
BPF_MOV64_IMM(BPF_REG_0, 1),
BPF_EXIT_INSN(),
},
- BPF_CGROUP_INET4_POST_BIND,
- BPF_CGROUP_INET6_POST_BIND,
- 0,
- 0,
- NULL,
- 0,
- ATTACH_REJECT,
+ .expected_attach_type = BPF_CGROUP_INET4_POST_BIND,
+ .attach_type = BPF_CGROUP_INET6_POST_BIND,
+ .result = ATTACH_REJECT,
},
{
- "attach type mismatch bind6 vs bind4",
+ .descr = "attach type mismatch bind6 vs bind4",
.insns = {
BPF_MOV64_IMM(BPF_REG_0, 1),
BPF_EXIT_INSN(),
},
- BPF_CGROUP_INET6_POST_BIND,
- BPF_CGROUP_INET4_POST_BIND,
- 0,
- 0,
- NULL,
- 0,
- ATTACH_REJECT,
+ .expected_attach_type = BPF_CGROUP_INET6_POST_BIND,
+ .attach_type = BPF_CGROUP_INET4_POST_BIND,
+ .result = ATTACH_REJECT,
},
{
- "attach type mismatch default vs bind4",
+ .descr = "attach type mismatch default vs bind4",
.insns = {
BPF_MOV64_IMM(BPF_REG_0, 1),
BPF_EXIT_INSN(),
},
- 0,
- BPF_CGROUP_INET4_POST_BIND,
- 0,
- 0,
- NULL,
- 0,
- ATTACH_REJECT,
+ .expected_attach_type = 0,
+ .attach_type = BPF_CGROUP_INET4_POST_BIND,
+ .result = ATTACH_REJECT,
},
{
- "attach type mismatch bind6 vs sock_create",
+ .descr = "attach type mismatch bind6 vs sock_create",
.insns = {
BPF_MOV64_IMM(BPF_REG_0, 1),
BPF_EXIT_INSN(),
},
- BPF_CGROUP_INET6_POST_BIND,
- BPF_CGROUP_INET_SOCK_CREATE,
- 0,
- 0,
- NULL,
- 0,
- ATTACH_REJECT,
+ .expected_attach_type = BPF_CGROUP_INET6_POST_BIND,
+ .attach_type = BPF_CGROUP_INET_SOCK_CREATE,
+ .result = ATTACH_REJECT,
},
{
- "bind4 reject all",
+ .descr = "bind4 reject all",
.insns = {
BPF_MOV64_IMM(BPF_REG_0, 0),
BPF_EXIT_INSN(),
},
- BPF_CGROUP_INET4_POST_BIND,
- BPF_CGROUP_INET4_POST_BIND,
- AF_INET,
- SOCK_STREAM,
- "0.0.0.0",
- 0,
- BIND_REJECT,
+ .expected_attach_type = BPF_CGROUP_INET4_POST_BIND,
+ .attach_type = BPF_CGROUP_INET4_POST_BIND,
+ .domain = AF_INET,
+ .type = SOCK_STREAM,
+ .ip = "0.0.0.0",
+ .result = BIND_REJECT,
},
{
- "bind6 reject all",
+ .descr = "bind6 reject all",
.insns = {
BPF_MOV64_IMM(BPF_REG_0, 0),
BPF_EXIT_INSN(),
},
- BPF_CGROUP_INET6_POST_BIND,
- BPF_CGROUP_INET6_POST_BIND,
- AF_INET6,
- SOCK_STREAM,
- "::",
- 0,
- BIND_REJECT,
+ .expected_attach_type = BPF_CGROUP_INET6_POST_BIND,
+ .attach_type = BPF_CGROUP_INET6_POST_BIND,
+ .domain = AF_INET6,
+ .type = SOCK_STREAM,
+ .ip = "::",
+ .result = BIND_REJECT,
},
{
- "bind6 deny specific IP & port",
+ .descr = "bind6 deny specific IP & port",
.insns = {
BPF_MOV64_REG(BPF_REG_6, BPF_REG_1),
@@ -247,16 +216,16 @@ static struct sock_test tests[] = {
BPF_MOV64_IMM(BPF_REG_0, 1),
BPF_EXIT_INSN(),
},
- BPF_CGROUP_INET6_POST_BIND,
- BPF_CGROUP_INET6_POST_BIND,
- AF_INET6,
- SOCK_STREAM,
- "::1",
- 8193,
- BIND_REJECT,
+ .expected_attach_type = BPF_CGROUP_INET6_POST_BIND,
+ .attach_type = BPF_CGROUP_INET6_POST_BIND,
+ .domain = AF_INET6,
+ .type = SOCK_STREAM,
+ .ip = "::1",
+ .port = 8193,
+ .result = BIND_REJECT,
},
{
- "bind4 allow specific IP & port",
+ .descr = "bind4 allow specific IP & port",
.insns = {
BPF_MOV64_REG(BPF_REG_6, BPF_REG_1),
@@ -277,41 +246,132 @@ static struct sock_test tests[] = {
BPF_MOV64_IMM(BPF_REG_0, 0),
BPF_EXIT_INSN(),
},
- BPF_CGROUP_INET4_POST_BIND,
- BPF_CGROUP_INET4_POST_BIND,
- AF_INET,
- SOCK_STREAM,
- "127.0.0.1",
- 4098,
- SUCCESS,
+ .expected_attach_type = BPF_CGROUP_INET4_POST_BIND,
+ .attach_type = BPF_CGROUP_INET4_POST_BIND,
+ .domain = AF_INET,
+ .type = SOCK_STREAM,
+ .ip = "127.0.0.1",
+ .port = 4098,
+ .result = SUCCESS,
},
{
- "bind4 allow all",
+ .descr = "bind4 deny specific IP & port of TCP, and retry",
+ .insns = {
+ BPF_MOV64_REG(BPF_REG_6, BPF_REG_1),
+
+ /* if (ip == expected && port == expected) */
+ BPF_LDX_MEM(BPF_W, BPF_REG_7, BPF_REG_6,
+ offsetof(struct bpf_sock, src_ip4)),
+ BPF_JMP_IMM(BPF_JNE, BPF_REG_7,
+ __bpf_constant_ntohl(0x7F000001), 4),
+ BPF_LDX_MEM(BPF_W, BPF_REG_7, BPF_REG_6,
+ offsetof(struct bpf_sock, src_port)),
+ BPF_JMP_IMM(BPF_JNE, BPF_REG_7, 0x1002, 2),
+
+ /* return DENY; */
+ BPF_MOV64_IMM(BPF_REG_0, 0),
+ BPF_JMP_A(1),
+
+ /* else return ALLOW; */
+ BPF_MOV64_IMM(BPF_REG_0, 1),
+ BPF_EXIT_INSN(),
+ },
+ .expected_attach_type = BPF_CGROUP_INET4_POST_BIND,
+ .attach_type = BPF_CGROUP_INET4_POST_BIND,
+ .domain = AF_INET,
+ .type = SOCK_STREAM,
+ .ip = "127.0.0.1",
+ .port = 4098,
+ .port_retry = 5000,
+ .result = RETRY_SUCCESS,
+ },
+ {
+ .descr = "bind4 deny specific IP & port of UDP, and retry",
+ .insns = {
+ BPF_MOV64_REG(BPF_REG_6, BPF_REG_1),
+
+ /* if (ip == expected && port == expected) */
+ BPF_LDX_MEM(BPF_W, BPF_REG_7, BPF_REG_6,
+ offsetof(struct bpf_sock, src_ip4)),
+ BPF_JMP_IMM(BPF_JNE, BPF_REG_7,
+ __bpf_constant_ntohl(0x7F000001), 4),
+ BPF_LDX_MEM(BPF_W, BPF_REG_7, BPF_REG_6,
+ offsetof(struct bpf_sock, src_port)),
+ BPF_JMP_IMM(BPF_JNE, BPF_REG_7, 0x1002, 2),
+
+ /* return DENY; */
+ BPF_MOV64_IMM(BPF_REG_0, 0),
+ BPF_JMP_A(1),
+
+ /* else return ALLOW; */
+ BPF_MOV64_IMM(BPF_REG_0, 1),
+ BPF_EXIT_INSN(),
+ },
+ .expected_attach_type = BPF_CGROUP_INET4_POST_BIND,
+ .attach_type = BPF_CGROUP_INET4_POST_BIND,
+ .domain = AF_INET,
+ .type = SOCK_DGRAM,
+ .ip = "127.0.0.1",
+ .port = 4098,
+ .port_retry = 5000,
+ .result = RETRY_SUCCESS,
+ },
+ {
+ .descr = "bind6 deny specific IP & port, and retry",
+ .insns = {
+ BPF_MOV64_REG(BPF_REG_6, BPF_REG_1),
+
+ /* if (ip == expected && port == expected) */
+ BPF_LDX_MEM(BPF_W, BPF_REG_7, BPF_REG_6,
+ offsetof(struct bpf_sock, src_ip6[3])),
+ BPF_JMP_IMM(BPF_JNE, BPF_REG_7,
+ __bpf_constant_ntohl(0x00000001), 4),
+ BPF_LDX_MEM(BPF_W, BPF_REG_7, BPF_REG_6,
+ offsetof(struct bpf_sock, src_port)),
+ BPF_JMP_IMM(BPF_JNE, BPF_REG_7, 0x2001, 2),
+
+ /* return DENY; */
+ BPF_MOV64_IMM(BPF_REG_0, 0),
+ BPF_JMP_A(1),
+
+ /* else return ALLOW; */
+ BPF_MOV64_IMM(BPF_REG_0, 1),
+ BPF_EXIT_INSN(),
+ },
+ .expected_attach_type = BPF_CGROUP_INET6_POST_BIND,
+ .attach_type = BPF_CGROUP_INET6_POST_BIND,
+ .domain = AF_INET6,
+ .type = SOCK_STREAM,
+ .ip = "::1",
+ .port = 8193,
+ .port_retry = 9000,
+ .result = RETRY_SUCCESS,
+ },
+ {
+ .descr = "bind4 allow all",
.insns = {
BPF_MOV64_IMM(BPF_REG_0, 1),
BPF_EXIT_INSN(),
},
- BPF_CGROUP_INET4_POST_BIND,
- BPF_CGROUP_INET4_POST_BIND,
- AF_INET,
- SOCK_STREAM,
- "0.0.0.0",
- 0,
- SUCCESS,
+ .expected_attach_type = BPF_CGROUP_INET4_POST_BIND,
+ .attach_type = BPF_CGROUP_INET4_POST_BIND,
+ .domain = AF_INET,
+ .type = SOCK_STREAM,
+ .ip = "0.0.0.0",
+ .result = SUCCESS,
},
{
- "bind6 allow all",
+ .descr = "bind6 allow all",
.insns = {
BPF_MOV64_IMM(BPF_REG_0, 1),
BPF_EXIT_INSN(),
},
- BPF_CGROUP_INET6_POST_BIND,
- BPF_CGROUP_INET6_POST_BIND,
- AF_INET6,
- SOCK_STREAM,
- "::",
- 0,
- SUCCESS,
+ .expected_attach_type = BPF_CGROUP_INET6_POST_BIND,
+ .attach_type = BPF_CGROUP_INET6_POST_BIND,
+ .domain = AF_INET6,
+ .type = SOCK_STREAM,
+ .ip = "::",
+ .result = SUCCESS,
},
};
@@ -328,18 +388,17 @@ static size_t probe_prog_length(const struct bpf_insn *fp)
static int load_sock_prog(const struct bpf_insn *prog,
enum bpf_attach_type attach_type)
{
- struct bpf_load_program_attr attr;
- int ret;
+ LIBBPF_OPTS(bpf_prog_load_opts, opts);
+ int ret, insn_cnt;
- memset(&attr, 0, sizeof(struct bpf_load_program_attr));
- attr.prog_type = BPF_PROG_TYPE_CGROUP_SOCK;
- attr.expected_attach_type = attach_type;
- attr.insns = prog;
- attr.insns_cnt = probe_prog_length(attr.insns);
- attr.license = "GPL";
- attr.log_level = 2;
+ insn_cnt = probe_prog_length(prog);
- ret = bpf_load_program_xattr(&attr, bpf_log_buf, BPF_LOG_BUF_SIZE);
+ opts.expected_attach_type = attach_type;
+ opts.log_buf = bpf_log_buf;
+ opts.log_size = BPF_LOG_BUF_SIZE;
+ opts.log_level = 2;
+
+ ret = bpf_prog_load(BPF_PROG_TYPE_CGROUP_SOCK, NULL, "GPL", prog, insn_cnt, &opts);
if (verbose && ret < 0)
fprintf(stderr, "%s\n", bpf_log_buf);
@@ -352,14 +411,15 @@ static int attach_sock_prog(int cgfd, int progfd,
return bpf_prog_attach(progfd, cgfd, attach_type, BPF_F_ALLOW_OVERRIDE);
}
-static int bind_sock(int domain, int type, const char *ip, unsigned short port)
+static int bind_sock(int domain, int type, const char *ip,
+ unsigned short port, unsigned short port_retry)
{
struct sockaddr_storage addr;
struct sockaddr_in6 *addr6;
struct sockaddr_in *addr4;
int sockfd = -1;
socklen_t len;
- int err = 0;
+ int res = SUCCESS;
sockfd = socket(domain, type, 0);
if (sockfd < 0)
@@ -385,21 +445,44 @@ static int bind_sock(int domain, int type, const char *ip, unsigned short port)
goto err;
}
- if (bind(sockfd, (const struct sockaddr *)&addr, len) == -1)
- goto err;
+ if (bind(sockfd, (const struct sockaddr *)&addr, len) == -1) {
+ /* sys_bind() may fail for different reasons, errno has to be
+ * checked to confirm that BPF program rejected it.
+ */
+ if (errno != EPERM)
+ goto err;
+ if (port_retry)
+ goto retry;
+ res = BIND_REJECT;
+ goto out;
+ }
goto out;
+retry:
+ if (domain == AF_INET)
+ addr4->sin_port = htons(port_retry);
+ else
+ addr6->sin6_port = htons(port_retry);
+ if (bind(sockfd, (const struct sockaddr *)&addr, len) == -1) {
+ if (errno != EPERM)
+ goto err;
+ res = RETRY_REJECT;
+ } else {
+ res = RETRY_SUCCESS;
+ }
+ goto out;
err:
- err = -1;
+ res = -1;
out:
close(sockfd);
- return err;
+ return res;
}
static int run_test_case(int cgfd, const struct sock_test *test)
{
int progfd = -1;
int err = 0;
+ int res;
printf("Test case: %s .. ", test->descr);
progfd = load_sock_prog(test->insns, test->expected_attach_type);
@@ -417,21 +500,11 @@ static int run_test_case(int cgfd, const struct sock_test *test)
goto err;
}
- if (bind_sock(test->domain, test->type, test->ip, test->port) == -1) {
- /* sys_bind() may fail for different reasons, errno has to be
- * checked to confirm that BPF program rejected it.
- */
- if (test->result == BIND_REJECT && errno == EPERM)
- goto out;
- else
- goto err;
- }
+ res = bind_sock(test->domain, test->type, test->ip, test->port,
+ test->port_retry);
+ if (res > 0 && test->result == res)
+ goto out;
-
- if (test->result != SUCCESS)
- goto err;
-
- goto out;
err:
err = -1;
out:
diff --git a/tools/testing/selftests/bpf/test_sock_addr.c b/tools/testing/selftests/bpf/test_sock_addr.c
index aa3f185..f0c8d05b 100644
--- a/tools/testing/selftests/bpf/test_sock_addr.c
+++ b/tools/testing/selftests/bpf/test_sock_addr.c
@@ -645,17 +645,14 @@ static int mk_sockaddr(int domain, const char *ip, unsigned short port,
static int load_insns(const struct sock_addr_test *test,
const struct bpf_insn *insns, size_t insns_cnt)
{
- struct bpf_load_program_attr load_attr;
+ LIBBPF_OPTS(bpf_prog_load_opts, opts);
int ret;
- memset(&load_attr, 0, sizeof(struct bpf_load_program_attr));
- load_attr.prog_type = BPF_PROG_TYPE_CGROUP_SOCK_ADDR;
- load_attr.expected_attach_type = test->expected_attach_type;
- load_attr.insns = insns;
- load_attr.insns_cnt = insns_cnt;
- load_attr.license = "GPL";
+ opts.expected_attach_type = test->expected_attach_type;
+ opts.log_buf = bpf_log_buf;
+ opts.log_size = BPF_LOG_BUF_SIZE;
- ret = bpf_load_program_xattr(&load_attr, bpf_log_buf, BPF_LOG_BUF_SIZE);
+ ret = bpf_prog_load(BPF_PROG_TYPE_CGROUP_SOCK_ADDR, NULL, "GPL", insns, insns_cnt, &opts);
if (ret < 0 && test->expected_result != LOAD_REJECT) {
log_err(">>> Loading program error.\n"
">>> Verifier output:\n%s\n-------\n", bpf_log_buf);
@@ -666,23 +663,36 @@ static int load_insns(const struct sock_addr_test *test,
static int load_path(const struct sock_addr_test *test, const char *path)
{
- struct bpf_prog_load_attr attr;
struct bpf_object *obj;
- int prog_fd;
+ struct bpf_program *prog;
+ int err;
- memset(&attr, 0, sizeof(struct bpf_prog_load_attr));
- attr.file = path;
- attr.prog_type = BPF_PROG_TYPE_CGROUP_SOCK_ADDR;
- attr.expected_attach_type = test->expected_attach_type;
- attr.prog_flags = BPF_F_TEST_RND_HI32;
-
- if (bpf_prog_load_xattr(&attr, &obj, &prog_fd)) {
- if (test->expected_result != LOAD_REJECT)
- log_err(">>> Loading program (%s) error.\n", path);
+ obj = bpf_object__open_file(path, NULL);
+ err = libbpf_get_error(obj);
+ if (err) {
+ log_err(">>> Opening BPF object (%s) error.\n", path);
return -1;
}
- return prog_fd;
+ prog = bpf_object__next_program(obj, NULL);
+ if (!prog)
+ goto err_out;
+
+ bpf_program__set_type(prog, BPF_PROG_TYPE_CGROUP_SOCK_ADDR);
+ bpf_program__set_expected_attach_type(prog, test->expected_attach_type);
+ bpf_program__set_flags(prog, BPF_F_TEST_RND_HI32);
+
+ err = bpf_object__load(obj);
+ if (err) {
+ if (test->expected_result != LOAD_REJECT)
+ log_err(">>> Loading program (%s) error.\n", path);
+ goto err_out;
+ }
+
+ return bpf_program__fd(prog);
+err_out:
+ bpf_object__close(obj);
+ return -1;
}
static int bind4_prog_load(const struct sock_addr_test *test)
diff --git a/tools/testing/selftests/bpf/test_stub.c b/tools/testing/selftests/bpf/test_stub.c
deleted file mode 100644
index 47e1327..0000000
--- a/tools/testing/selftests/bpf/test_stub.c
+++ /dev/null
@@ -1,44 +0,0 @@
-// SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
-/* Copyright (C) 2019 Netronome Systems, Inc. */
-
-#include <bpf/bpf.h>
-#include <bpf/libbpf.h>
-#include <string.h>
-
-int extra_prog_load_log_flags = 0;
-
-int bpf_prog_test_load(const char *file, enum bpf_prog_type type,
- struct bpf_object **pobj, int *prog_fd)
-{
- struct bpf_prog_load_attr attr;
-
- memset(&attr, 0, sizeof(struct bpf_prog_load_attr));
- attr.file = file;
- attr.prog_type = type;
- attr.expected_attach_type = 0;
- attr.prog_flags = BPF_F_TEST_RND_HI32;
- attr.log_level = extra_prog_load_log_flags;
-
- return bpf_prog_load_xattr(&attr, pobj, prog_fd);
-}
-
-int bpf_test_load_program(enum bpf_prog_type type, const struct bpf_insn *insns,
- size_t insns_cnt, const char *license,
- __u32 kern_version, char *log_buf,
- size_t log_buf_sz)
-{
- struct bpf_load_program_attr load_attr;
-
- memset(&load_attr, 0, sizeof(struct bpf_load_program_attr));
- load_attr.prog_type = type;
- load_attr.expected_attach_type = 0;
- load_attr.name = NULL;
- load_attr.insns = insns;
- load_attr.insns_cnt = insns_cnt;
- load_attr.license = license;
- load_attr.kern_version = kern_version;
- load_attr.prog_flags = BPF_F_TEST_RND_HI32;
- load_attr.log_level = extra_prog_load_log_flags;
-
- return bpf_load_program_xattr(&load_attr, log_buf, log_buf_sz);
-}
diff --git a/tools/testing/selftests/bpf/test_sysctl.c b/tools/testing/selftests/bpf/test_sysctl.c
index a3bb6d3..4f6cf83 100644
--- a/tools/testing/selftests/bpf/test_sysctl.c
+++ b/tools/testing/selftests/bpf/test_sysctl.c
@@ -17,6 +17,7 @@
#include "bpf_rlimit.h"
#include "bpf_util.h"
#include "cgroup_helpers.h"
+#include "testing_helpers.h"
#define CG_PATH "/foo"
#define MAX_INSNS 512
@@ -1435,14 +1436,10 @@ static int load_sysctl_prog_insns(struct sysctl_test *test,
const char *sysctl_path)
{
struct bpf_insn *prog = test->insns;
- struct bpf_load_program_attr attr;
- int ret;
+ LIBBPF_OPTS(bpf_prog_load_opts, opts);
+ int ret, insn_cnt;
- memset(&attr, 0, sizeof(struct bpf_load_program_attr));
- attr.prog_type = BPF_PROG_TYPE_CGROUP_SYSCTL;
- attr.insns = prog;
- attr.insns_cnt = probe_prog_length(attr.insns);
- attr.license = "GPL";
+ insn_cnt = probe_prog_length(prog);
if (test->fixup_value_insn) {
char buf[128];
@@ -1465,7 +1462,10 @@ static int load_sysctl_prog_insns(struct sysctl_test *test,
return -1;
}
- ret = bpf_load_program_xattr(&attr, bpf_log_buf, BPF_LOG_BUF_SIZE);
+ opts.log_buf = bpf_log_buf;
+ opts.log_size = BPF_LOG_BUF_SIZE;
+
+ ret = bpf_prog_load(BPF_PROG_TYPE_CGROUP_SYSCTL, NULL, "GPL", prog, insn_cnt, &opts);
if (ret < 0 && test->result != LOAD_REJECT) {
log_err(">>> Loading program error.\n"
">>> Verifier output:\n%s\n-------\n", bpf_log_buf);
@@ -1476,15 +1476,10 @@ static int load_sysctl_prog_insns(struct sysctl_test *test,
static int load_sysctl_prog_file(struct sysctl_test *test)
{
- struct bpf_prog_load_attr attr;
struct bpf_object *obj;
int prog_fd;
- memset(&attr, 0, sizeof(struct bpf_prog_load_attr));
- attr.file = test->prog_file;
- attr.prog_type = BPF_PROG_TYPE_CGROUP_SYSCTL;
-
- if (bpf_prog_load_xattr(&attr, &obj, &prog_fd)) {
+ if (bpf_prog_test_load(test->prog_file, BPF_PROG_TYPE_CGROUP_SYSCTL, &obj, &prog_fd)) {
if (test->result != LOAD_REJECT)
log_err(">>> Loading program (%s) error.\n",
test->prog_file);
diff --git a/tools/testing/selftests/bpf/test_tag.c b/tools/testing/selftests/bpf/test_tag.c
index 6272c784..0851c42 100644
--- a/tools/testing/selftests/bpf/test_tag.c
+++ b/tools/testing/selftests/bpf/test_tag.c
@@ -21,6 +21,7 @@
#include "../../../include/linux/filter.h"
#include "bpf_rlimit.h"
+#include "testing_helpers.h"
static struct bpf_insn prog[BPF_MAXINSNS];
@@ -57,7 +58,7 @@ static int bpf_try_load_prog(int insns, int fd_map,
int fd_prog;
bpf_filler(insns, fd_map);
- fd_prog = bpf_load_program(BPF_PROG_TYPE_SCHED_CLS, prog, insns, "", 0,
+ fd_prog = bpf_test_load_program(BPF_PROG_TYPE_SCHED_CLS, prog, insns, "", 0,
NULL, 0);
assert(fd_prog > 0);
if (fd_map > 0)
@@ -184,11 +185,12 @@ static void do_test(uint32_t *tests, int start_insns, int fd_map,
int main(void)
{
+ LIBBPF_OPTS(bpf_map_create_opts, opts, .map_flags = BPF_F_NO_PREALLOC);
uint32_t tests = 0;
int i, fd_map;
- fd_map = bpf_create_map(BPF_MAP_TYPE_HASH, sizeof(int),
- sizeof(int), 1, BPF_F_NO_PREALLOC);
+ fd_map = bpf_map_create(BPF_MAP_TYPE_HASH, NULL, sizeof(int),
+ sizeof(int), 1, &opts);
assert(fd_map > 0);
for (i = 0; i < 5; i++) {
diff --git a/tools/testing/selftests/bpf/test_tcpnotify_user.c b/tools/testing/selftests/bpf/test_tcpnotify_user.c
index 4a39304..4c51147 100644
--- a/tools/testing/selftests/bpf/test_tcpnotify_user.c
+++ b/tools/testing/selftests/bpf/test_tcpnotify_user.c
@@ -25,6 +25,7 @@
#include "test_tcpnotify.h"
#include "trace_helpers.h"
+#include "testing_helpers.h"
#define SOCKET_BUFFER_SIZE (getpagesize() < 8192L ? getpagesize() : 8192L)
@@ -71,7 +72,6 @@ int main(int argc, char **argv)
{
const char *file = "test_tcpnotify_kern.o";
struct bpf_map *perf_map, *global_map;
- struct perf_buffer_opts pb_opts = {};
struct tcpnotify_globals g = {0};
struct perf_buffer *pb = NULL;
const char *cg_path = "/foo";
@@ -92,7 +92,7 @@ int main(int argc, char **argv)
if (cg_fd < 0)
goto err;
- if (bpf_prog_load(file, BPF_PROG_TYPE_SOCK_OPS, &obj, &prog_fd)) {
+ if (bpf_prog_test_load(file, BPF_PROG_TYPE_SOCK_OPS, &obj, &prog_fd)) {
printf("FAILED: load_bpf_file failed for: %s\n", file);
goto err;
}
@@ -116,8 +116,7 @@ int main(int argc, char **argv)
return -1;
}
- pb_opts.sample_cb = dummyfn;
- pb = perf_buffer__new(bpf_map__fd(perf_map), 8, &pb_opts);
+ pb = perf_buffer__new(bpf_map__fd(perf_map), 8, dummyfn, NULL, NULL, NULL);
if (!pb)
goto err;
diff --git a/tools/testing/selftests/bpf/test_verifier.c b/tools/testing/selftests/bpf/test_verifier.c
index d3bf83d..76cd903 100644
--- a/tools/testing/selftests/bpf/test_verifier.c
+++ b/tools/testing/selftests/bpf/test_verifier.c
@@ -41,7 +41,6 @@
# define CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS 1
# endif
#endif
-#include "bpf_rlimit.h"
#include "bpf_rand.h"
#include "bpf_util.h"
#include "test_btf.h"
@@ -462,11 +461,11 @@ static int __create_map(uint32_t type, uint32_t size_key,
uint32_t size_value, uint32_t max_elem,
uint32_t extra_flags)
{
+ LIBBPF_OPTS(bpf_map_create_opts, opts);
int fd;
- fd = bpf_create_map(type, size_key, size_value, max_elem,
- (type == BPF_MAP_TYPE_HASH ?
- BPF_F_NO_PREALLOC : 0) | extra_flags);
+ opts.map_flags = (type == BPF_MAP_TYPE_HASH ? BPF_F_NO_PREALLOC : 0) | extra_flags;
+ fd = bpf_map_create(type, NULL, size_key, size_value, max_elem, &opts);
if (fd < 0) {
if (skip_unsupported_map(type))
return -1;
@@ -499,8 +498,7 @@ static int create_prog_dummy_simple(enum bpf_prog_type prog_type, int ret)
BPF_EXIT_INSN(),
};
- return bpf_load_program(prog_type, prog,
- ARRAY_SIZE(prog), "GPL", 0, NULL, 0);
+ return bpf_prog_load(prog_type, NULL, "GPL", prog, ARRAY_SIZE(prog), NULL);
}
static int create_prog_dummy_loop(enum bpf_prog_type prog_type, int mfd,
@@ -515,8 +513,7 @@ static int create_prog_dummy_loop(enum bpf_prog_type prog_type, int mfd,
BPF_EXIT_INSN(),
};
- return bpf_load_program(prog_type, prog,
- ARRAY_SIZE(prog), "GPL", 0, NULL, 0);
+ return bpf_prog_load(prog_type, NULL, "GPL", prog, ARRAY_SIZE(prog), NULL);
}
static int create_prog_array(enum bpf_prog_type prog_type, uint32_t max_elem,
@@ -524,8 +521,8 @@ static int create_prog_array(enum bpf_prog_type prog_type, uint32_t max_elem,
{
int mfd, p1fd, p2fd, p3fd;
- mfd = bpf_create_map(BPF_MAP_TYPE_PROG_ARRAY, sizeof(int),
- sizeof(int), max_elem, 0);
+ mfd = bpf_map_create(BPF_MAP_TYPE_PROG_ARRAY, NULL, sizeof(int),
+ sizeof(int), max_elem, NULL);
if (mfd < 0) {
if (skip_unsupported_map(BPF_MAP_TYPE_PROG_ARRAY))
return -1;
@@ -555,10 +552,11 @@ static int create_prog_array(enum bpf_prog_type prog_type, uint32_t max_elem,
static int create_map_in_map(void)
{
+ LIBBPF_OPTS(bpf_map_create_opts, opts);
int inner_map_fd, outer_map_fd;
- inner_map_fd = bpf_create_map(BPF_MAP_TYPE_ARRAY, sizeof(int),
- sizeof(int), 1, 0);
+ inner_map_fd = bpf_map_create(BPF_MAP_TYPE_ARRAY, NULL, sizeof(int),
+ sizeof(int), 1, NULL);
if (inner_map_fd < 0) {
if (skip_unsupported_map(BPF_MAP_TYPE_ARRAY))
return -1;
@@ -566,8 +564,9 @@ static int create_map_in_map(void)
return inner_map_fd;
}
- outer_map_fd = bpf_create_map_in_map(BPF_MAP_TYPE_ARRAY_OF_MAPS, NULL,
- sizeof(int), inner_map_fd, 1, 0);
+ opts.inner_map_fd = inner_map_fd;
+ outer_map_fd = bpf_map_create(BPF_MAP_TYPE_ARRAY_OF_MAPS, NULL,
+ sizeof(int), sizeof(int), 1, &opts);
if (outer_map_fd < 0) {
if (skip_unsupported_map(BPF_MAP_TYPE_ARRAY_OF_MAPS))
return -1;
@@ -586,8 +585,8 @@ static int create_cgroup_storage(bool percpu)
BPF_MAP_TYPE_CGROUP_STORAGE;
int fd;
- fd = bpf_create_map(type, sizeof(struct bpf_cgroup_storage_key),
- TEST_DATA_LEN, 0, 0);
+ fd = bpf_map_create(type, NULL, sizeof(struct bpf_cgroup_storage_key),
+ TEST_DATA_LEN, 0, NULL);
if (fd < 0) {
if (skip_unsupported_map(type))
return -1;
@@ -654,7 +653,7 @@ static int load_btf(void)
memcpy(ptr, btf_str_sec, hdr.str_len);
ptr += hdr.str_len;
- btf_fd = bpf_load_btf(raw_btf, ptr - raw_btf, 0, 0, 0);
+ btf_fd = bpf_btf_load(raw_btf, ptr - raw_btf, NULL);
free(raw_btf);
if (btf_fd < 0)
return -1;
@@ -663,22 +662,17 @@ static int load_btf(void)
static int create_map_spin_lock(void)
{
- struct bpf_create_map_attr attr = {
- .name = "test_map",
- .map_type = BPF_MAP_TYPE_ARRAY,
- .key_size = 4,
- .value_size = 8,
- .max_entries = 1,
+ LIBBPF_OPTS(bpf_map_create_opts, opts,
.btf_key_type_id = 1,
.btf_value_type_id = 3,
- };
+ );
int fd, btf_fd;
btf_fd = load_btf();
if (btf_fd < 0)
return -1;
- attr.btf_fd = btf_fd;
- fd = bpf_create_map_xattr(&attr);
+ opts.btf_fd = btf_fd;
+ fd = bpf_map_create(BPF_MAP_TYPE_ARRAY, "test_map", 4, 8, 1, &opts);
if (fd < 0)
printf("Failed to create map with spin_lock\n");
return fd;
@@ -686,24 +680,19 @@ static int create_map_spin_lock(void)
static int create_sk_storage_map(void)
{
- struct bpf_create_map_attr attr = {
- .name = "test_map",
- .map_type = BPF_MAP_TYPE_SK_STORAGE,
- .key_size = 4,
- .value_size = 8,
- .max_entries = 0,
+ LIBBPF_OPTS(bpf_map_create_opts, opts,
.map_flags = BPF_F_NO_PREALLOC,
.btf_key_type_id = 1,
.btf_value_type_id = 3,
- };
+ );
int fd, btf_fd;
btf_fd = load_btf();
if (btf_fd < 0)
return -1;
- attr.btf_fd = btf_fd;
- fd = bpf_create_map_xattr(&attr);
- close(attr.btf_fd);
+ opts.btf_fd = btf_fd;
+ fd = bpf_map_create(BPF_MAP_TYPE_SK_STORAGE, "test_map", 4, 8, 0, &opts);
+ close(opts.btf_fd);
if (fd < 0)
printf("Failed to create sk_storage_map\n");
return fd;
@@ -711,22 +700,18 @@ static int create_sk_storage_map(void)
static int create_map_timer(void)
{
- struct bpf_create_map_attr attr = {
- .name = "test_map",
- .map_type = BPF_MAP_TYPE_ARRAY,
- .key_size = 4,
- .value_size = 16,
- .max_entries = 1,
+ LIBBPF_OPTS(bpf_map_create_opts, opts,
.btf_key_type_id = 1,
.btf_value_type_id = 5,
- };
+ );
int fd, btf_fd;
btf_fd = load_btf();
if (btf_fd < 0)
return -1;
- attr.btf_fd = btf_fd;
- fd = bpf_create_map_xattr(&attr);
+
+ opts.btf_fd = btf_fd;
+ fd = bpf_map_create(BPF_MAP_TYPE_ARRAY, "test_map", 4, 16, 1, &opts);
if (fd < 0)
printf("Failed to create map with timer\n");
return fd;
@@ -1089,7 +1074,7 @@ static void do_test_single(struct bpf_test *test, bool unpriv,
int fd_prog, expected_ret, alignment_prevented_execution;
int prog_len, prog_type = test->prog_type;
struct bpf_insn *prog = test->insns;
- struct bpf_load_program_attr attr;
+ LIBBPF_OPTS(bpf_prog_load_opts, opts);
int run_errs, run_successes;
int map_fds[MAX_NR_MAPS];
const char *expected_err;
@@ -1129,32 +1114,34 @@ static void do_test_single(struct bpf_test *test, bool unpriv,
test->result_unpriv : test->result;
expected_err = unpriv && test->errstr_unpriv ?
test->errstr_unpriv : test->errstr;
- memset(&attr, 0, sizeof(attr));
- attr.prog_type = prog_type;
- attr.expected_attach_type = test->expected_attach_type;
- attr.insns = prog;
- attr.insns_cnt = prog_len;
- attr.license = "GPL";
+
+ opts.expected_attach_type = test->expected_attach_type;
if (verbose)
- attr.log_level = 1;
+ opts.log_level = 1;
else if (expected_ret == VERBOSE_ACCEPT)
- attr.log_level = 2;
+ opts.log_level = 2;
else
- attr.log_level = 4;
- attr.prog_flags = pflags;
+ opts.log_level = 4;
+ opts.prog_flags = pflags;
if (prog_type == BPF_PROG_TYPE_TRACING && test->kfunc) {
- attr.attach_btf_id = libbpf_find_vmlinux_btf_id(test->kfunc,
- attr.expected_attach_type);
- if (attr.attach_btf_id < 0) {
+ int attach_btf_id;
+
+ attach_btf_id = libbpf_find_vmlinux_btf_id(test->kfunc,
+ opts.expected_attach_type);
+ if (attach_btf_id < 0) {
printf("FAIL\nFailed to find BTF ID for '%s'!\n",
test->kfunc);
(*errors)++;
return;
}
+
+ opts.attach_btf_id = attach_btf_id;
}
- fd_prog = bpf_load_program_xattr(&attr, bpf_vlog, sizeof(bpf_vlog));
+ opts.log_buf = bpf_vlog;
+ opts.log_size = sizeof(bpf_vlog);
+ fd_prog = bpf_prog_load(prog_type, NULL, "GPL", prog, prog_len, &opts);
saved_errno = errno;
/* BPF_PROG_TYPE_TRACING requires more setup and
@@ -1407,6 +1394,9 @@ int main(int argc, char **argv)
return EXIT_FAILURE;
}
+ /* Use libbpf 1.0 API mode */
+ libbpf_set_strict_mode(LIBBPF_STRICT_ALL);
+
bpf_semi_rand_init();
return do_test(unpriv, from, to);
}
diff --git a/tools/testing/selftests/bpf/testing_helpers.c b/tools/testing/selftests/bpf/testing_helpers.c
index 800d503..795b679 100644
--- a/tools/testing/selftests/bpf/testing_helpers.c
+++ b/tools/testing/selftests/bpf/testing_helpers.c
@@ -1,7 +1,11 @@
// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
+/* Copyright (C) 2019 Netronome Systems, Inc. */
/* Copyright (C) 2020 Facebook, Inc. */
#include <stdlib.h>
+#include <string.h>
#include <errno.h>
+#include <bpf/bpf.h>
+#include <bpf/libbpf.h>
#include "testing_helpers.h"
int parse_num_list(const char *s, bool **num_set, int *num_set_len)
@@ -78,3 +82,61 @@ __u32 link_info_prog_id(const struct bpf_link *link, struct bpf_link_info *info)
}
return info->prog_id;
}
+
+int extra_prog_load_log_flags = 0;
+
+int bpf_prog_test_load(const char *file, enum bpf_prog_type type,
+ struct bpf_object **pobj, int *prog_fd)
+{
+ LIBBPF_OPTS(bpf_object_open_opts, opts,
+ .kernel_log_level = extra_prog_load_log_flags,
+ );
+ struct bpf_object *obj;
+ struct bpf_program *prog;
+ __u32 flags;
+ int err;
+
+ obj = bpf_object__open_file(file, &opts);
+ if (!obj)
+ return -errno;
+
+ prog = bpf_object__next_program(obj, NULL);
+ if (!prog) {
+ err = -ENOENT;
+ goto err_out;
+ }
+
+ if (type != BPF_PROG_TYPE_UNSPEC)
+ bpf_program__set_type(prog, type);
+
+ flags = bpf_program__flags(prog) | BPF_F_TEST_RND_HI32;
+ bpf_program__set_flags(prog, flags);
+
+ err = bpf_object__load(obj);
+ if (err)
+ goto err_out;
+
+ *pobj = obj;
+ *prog_fd = bpf_program__fd(prog);
+
+ return 0;
+err_out:
+ bpf_object__close(obj);
+ return err;
+}
+
+int bpf_test_load_program(enum bpf_prog_type type, const struct bpf_insn *insns,
+ size_t insns_cnt, const char *license,
+ __u32 kern_version, char *log_buf,
+ size_t log_buf_sz)
+{
+ LIBBPF_OPTS(bpf_prog_load_opts, opts,
+ .kern_version = kern_version,
+ .prog_flags = BPF_F_TEST_RND_HI32,
+ .log_level = extra_prog_load_log_flags,
+ .log_buf = log_buf,
+ .log_size = log_buf_sz,
+ );
+
+ return bpf_prog_load(type, NULL, license, insns, insns_cnt, &opts);
+}
diff --git a/tools/testing/selftests/bpf/testing_helpers.h b/tools/testing/selftests/bpf/testing_helpers.h
index d4f8e74..f46ebc4 100644
--- a/tools/testing/selftests/bpf/testing_helpers.h
+++ b/tools/testing/selftests/bpf/testing_helpers.h
@@ -6,3 +6,9 @@
int parse_num_list(const char *s, bool **set, int *set_len);
__u32 link_info_prog_id(const struct bpf_link *link, struct bpf_link_info *info);
+int bpf_prog_test_load(const char *file, enum bpf_prog_type type,
+ struct bpf_object **pobj, int *prog_fd);
+int bpf_test_load_program(enum bpf_prog_type type, const struct bpf_insn *insns,
+ size_t insns_cnt, const char *license,
+ __u32 kern_version, char *log_buf,
+ size_t log_buf_sz);
diff --git a/tools/testing/selftests/bpf/verifier/btf_ctx_access.c b/tools/testing/selftests/bpf/verifier/btf_ctx_access.c
new file mode 100644
index 0000000..6340db6
--- /dev/null
+++ b/tools/testing/selftests/bpf/verifier/btf_ctx_access.c
@@ -0,0 +1,12 @@
+{
+ "btf_ctx_access accept",
+ .insns = {
+ BPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_1, 8), /* load 2nd argument value (int pointer) */
+ BPF_MOV64_IMM(BPF_REG_0, 0),
+ BPF_EXIT_INSN(),
+ },
+ .result = ACCEPT,
+ .prog_type = BPF_PROG_TYPE_TRACING,
+ .expected_attach_type = BPF_TRACE_FENTRY,
+ .kfunc = "bpf_modify_return_test",
+},
diff --git a/tools/testing/selftests/bpf/verifier/ctx_sk_lookup.c b/tools/testing/selftests/bpf/verifier/ctx_sk_lookup.c
index d78627b..a2b006e 100644
--- a/tools/testing/selftests/bpf/verifier/ctx_sk_lookup.c
+++ b/tools/testing/selftests/bpf/verifier/ctx_sk_lookup.c
@@ -229,6 +229,24 @@
BPF_LDX_MEM(BPF_W, BPF_REG_0, BPF_REG_1,
offsetof(struct bpf_sk_lookup, local_port)),
+ /* 1-byte read from ingress_ifindex field */
+ BPF_LDX_MEM(BPF_B, BPF_REG_0, BPF_REG_1,
+ offsetof(struct bpf_sk_lookup, ingress_ifindex)),
+ BPF_LDX_MEM(BPF_B, BPF_REG_0, BPF_REG_1,
+ offsetof(struct bpf_sk_lookup, ingress_ifindex) + 1),
+ BPF_LDX_MEM(BPF_B, BPF_REG_0, BPF_REG_1,
+ offsetof(struct bpf_sk_lookup, ingress_ifindex) + 2),
+ BPF_LDX_MEM(BPF_B, BPF_REG_0, BPF_REG_1,
+ offsetof(struct bpf_sk_lookup, ingress_ifindex) + 3),
+ /* 2-byte read from ingress_ifindex field */
+ BPF_LDX_MEM(BPF_H, BPF_REG_0, BPF_REG_1,
+ offsetof(struct bpf_sk_lookup, ingress_ifindex)),
+ BPF_LDX_MEM(BPF_H, BPF_REG_0, BPF_REG_1,
+ offsetof(struct bpf_sk_lookup, ingress_ifindex) + 2),
+ /* 4-byte read from ingress_ifindex field */
+ BPF_LDX_MEM(BPF_W, BPF_REG_0, BPF_REG_1,
+ offsetof(struct bpf_sk_lookup, ingress_ifindex)),
+
/* 8-byte read from sk field */
BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_1,
offsetof(struct bpf_sk_lookup, sk)),
@@ -351,6 +369,20 @@
.expected_attach_type = BPF_SK_LOOKUP,
.flags = F_NEEDS_EFFICIENT_UNALIGNED_ACCESS,
},
+{
+ "invalid 8-byte read from bpf_sk_lookup ingress_ifindex field",
+ .insns = {
+ BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_1,
+ offsetof(struct bpf_sk_lookup, ingress_ifindex)),
+ BPF_MOV32_IMM(BPF_REG_0, 0),
+ BPF_EXIT_INSN(),
+ },
+ .errstr = "invalid bpf_context access",
+ .result = REJECT,
+ .prog_type = BPF_PROG_TYPE_SK_LOOKUP,
+ .expected_attach_type = BPF_SK_LOOKUP,
+ .flags = F_NEEDS_EFFICIENT_UNALIGNED_ACCESS,
+},
/* invalid 1,2,4-byte reads from 8-byte fields in bpf_sk_lookup */
{
"invalid 4-byte read from bpf_sk_lookup sk field",
diff --git a/tools/testing/selftests/bpf/verifier/spill_fill.c b/tools/testing/selftests/bpf/verifier/spill_fill.c
index 6c90714..1a8eb96 100644
--- a/tools/testing/selftests/bpf/verifier/spill_fill.c
+++ b/tools/testing/selftests/bpf/verifier/spill_fill.c
@@ -59,6 +59,34 @@
.result_unpriv = ACCEPT,
},
{
+ "check with invalid reg offset 0",
+ .insns = {
+ /* reserve 8 byte ringbuf memory */
+ BPF_ST_MEM(BPF_DW, BPF_REG_10, -8, 0),
+ BPF_LD_MAP_FD(BPF_REG_1, 0),
+ BPF_MOV64_IMM(BPF_REG_2, 8),
+ BPF_MOV64_IMM(BPF_REG_3, 0),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_ringbuf_reserve),
+ /* store a pointer to the reserved memory in R6 */
+ BPF_MOV64_REG(BPF_REG_6, BPF_REG_0),
+ /* add invalid offset to memory or NULL */
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_0, 1),
+ /* check whether the reservation was successful */
+ BPF_JMP_IMM(BPF_JEQ, BPF_REG_0, 0, 4),
+ /* should not be able to access *(R7) = 0 */
+ BPF_ST_MEM(BPF_W, BPF_REG_6, 0, 0),
+ /* submit the reserved ringbuf memory */
+ BPF_MOV64_REG(BPF_REG_1, BPF_REG_6),
+ BPF_MOV64_IMM(BPF_REG_2, 0),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_ringbuf_submit),
+ BPF_MOV64_IMM(BPF_REG_0, 0),
+ BPF_EXIT_INSN(),
+ },
+ .fixup_map_ringbuf = { 1 },
+ .result = REJECT,
+ .errstr = "R0 pointer arithmetic on mem_or_null prohibited",
+},
+{
"check corrupted spill/fill",
.insns = {
/* spill R1(ctx) into stack */
diff --git a/tools/testing/selftests/bpf/verifier/value_ptr_arith.c b/tools/testing/selftests/bpf/verifier/value_ptr_arith.c
index 4d347bc..359f3e8 100644
--- a/tools/testing/selftests/bpf/verifier/value_ptr_arith.c
+++ b/tools/testing/selftests/bpf/verifier/value_ptr_arith.c
@@ -1078,7 +1078,7 @@
.errstr_unpriv = "R0 pointer -= pointer prohibited",
},
{
- "map access: trying to leak tained dst reg",
+ "map access: trying to leak tainted dst reg",
.insns = {
BPF_MOV64_IMM(BPF_REG_0, 0),
BPF_ST_MEM(BPF_DW, BPF_REG_10, -8, 0),
diff --git a/tools/testing/selftests/bpf/vmtest.sh b/tools/testing/selftests/bpf/vmtest.sh
index 0271987..b3afd43 100755
--- a/tools/testing/selftests/bpf/vmtest.sh
+++ b/tools/testing/selftests/bpf/vmtest.sh
@@ -4,18 +4,35 @@
set -u
set -e
-# This script currently only works for x86_64, as
-# it is based on the VM image used by the BPF CI which is
-# x86_64.
-QEMU_BINARY="${QEMU_BINARY:="qemu-system-x86_64"}"
-X86_BZIMAGE="arch/x86/boot/bzImage"
+# This script currently only works for x86_64 and s390x, as
+# it is based on the VM image used by the BPF CI, which is
+# available only for these architectures.
+ARCH="$(uname -m)"
+case "${ARCH}" in
+s390x)
+ QEMU_BINARY=qemu-system-s390x
+ QEMU_CONSOLE="ttyS1"
+ QEMU_FLAGS=(-smp 2)
+ BZIMAGE="arch/s390/boot/compressed/vmlinux"
+ ;;
+x86_64)
+ QEMU_BINARY=qemu-system-x86_64
+ QEMU_CONSOLE="ttyS0,115200"
+ QEMU_FLAGS=(-cpu host -smp 8)
+ BZIMAGE="arch/x86/boot/bzImage"
+ ;;
+*)
+ echo "Unsupported architecture"
+ exit 1
+ ;;
+esac
DEFAULT_COMMAND="./test_progs"
MOUNT_DIR="mnt"
ROOTFS_IMAGE="root.img"
OUTPUT_DIR="$HOME/.bpf_selftests"
-KCONFIG_URL="https://raw.githubusercontent.com/libbpf/libbpf/master/travis-ci/vmtest/configs/latest.config"
-KCONFIG_API_URL="https://api.github.com/repos/libbpf/libbpf/contents/travis-ci/vmtest/configs/latest.config"
-INDEX_URL="https://raw.githubusercontent.com/libbpf/libbpf/master/travis-ci/vmtest/configs/INDEX"
+KCONFIG_URL="https://raw.githubusercontent.com/libbpf/libbpf/master/travis-ci/vmtest/configs/config-latest.${ARCH}"
+KCONFIG_API_URL="https://api.github.com/repos/libbpf/libbpf/contents/travis-ci/vmtest/configs/config-latest.${ARCH}"
+INDEX_URL="https://raw.githubusercontent.com/libbpf/ci/master/INDEX"
NUM_COMPILE_JOBS="$(nproc)"
LOG_FILE_BASE="$(date +"bpf_selftests.%Y-%m-%d_%H-%M-%S")"
LOG_FILE="${LOG_FILE_BASE}.log"
@@ -85,7 +102,7 @@
{
{
for file in "${!URLS[@]}"; do
- if [[ $file =~ ^libbpf-vmtest-rootfs-(.*)\.tar\.zst$ ]]; then
+ if [[ $file =~ ^"${ARCH}"/libbpf-vmtest-rootfs-(.*)\.tar\.zst$ ]]; then
echo "${BASH_REMATCH[1]}"
fi
done
@@ -102,7 +119,7 @@
exit 1
fi
- download "libbpf-vmtest-rootfs-$rootfsversion.tar.zst" |
+ download "${ARCH}/libbpf-vmtest-rootfs-$rootfsversion.tar.zst" |
zstd -d | sudo tar -C "$dir" -x
}
@@ -224,13 +241,12 @@
-nodefaults \
-display none \
-serial mon:stdio \
- -cpu host \
+ "${qemu_flags[@]}" \
-enable-kvm \
- -smp 8 \
-m 4G \
-drive file="${rootfs_img}",format=raw,index=1,media=disk,if=virtio,cache=none \
-kernel "${kernel_bzimage}" \
- -append "root=/dev/vda rw console=ttyS0,115200"
+ -append "root=/dev/vda rw console=${QEMU_CONSOLE}"
}
copy_logs()
@@ -282,7 +298,7 @@
local kernel_checkout=$(realpath "${script_dir}"/../../../../)
# By default the script searches for the kernel in the checkout directory but
# it also obeys environment variables O= and KBUILD_OUTPUT=
- local kernel_bzimage="${kernel_checkout}/${X86_BZIMAGE}"
+ local kernel_bzimage="${kernel_checkout}/${BZIMAGE}"
local command="${DEFAULT_COMMAND}"
local update_image="no"
local exit_command="poweroff -f"
@@ -337,13 +353,13 @@
if is_rel_path "${O}"; then
O="$(realpath "${PWD}/${O}")"
fi
- kernel_bzimage="${O}/${X86_BZIMAGE}"
+ kernel_bzimage="${O}/${BZIMAGE}"
make_command="${make_command} O=${O}"
elif [[ "${KBUILD_OUTPUT:=""}" != "" ]]; then
if is_rel_path "${KBUILD_OUTPUT}"; then
KBUILD_OUTPUT="$(realpath "${PWD}/${KBUILD_OUTPUT}")"
fi
- kernel_bzimage="${KBUILD_OUTPUT}/${X86_BZIMAGE}"
+ kernel_bzimage="${KBUILD_OUTPUT}/${BZIMAGE}"
make_command="${make_command} KBUILD_OUTPUT=${KBUILD_OUTPUT}"
fi
diff --git a/tools/testing/selftests/bpf/xdp_redirect_multi.c b/tools/testing/selftests/bpf/xdp_redirect_multi.c
index f5ffba3..51c8224 100644
--- a/tools/testing/selftests/bpf/xdp_redirect_multi.c
+++ b/tools/testing/selftests/bpf/xdp_redirect_multi.c
@@ -85,10 +85,7 @@ int main(int argc, char **argv)
{
int prog_fd, group_all, mac_map;
struct bpf_program *ingress_prog, *egress_prog;
- struct bpf_prog_load_attr prog_load_attr = {
- .prog_type = BPF_PROG_TYPE_UNSPEC,
- };
- int i, ret, opt, egress_prog_fd = 0;
+ int i, err, ret, opt, egress_prog_fd = 0;
struct bpf_devmap_val devmap_val;
bool attach_egress_prog = false;
unsigned char mac_addr[6];
@@ -147,10 +144,14 @@ int main(int argc, char **argv)
printf("\n");
snprintf(filename, sizeof(filename), "%s_kern.o", argv[0]);
- prog_load_attr.file = filename;
-
- if (bpf_prog_load_xattr(&prog_load_attr, &obj, &prog_fd))
+ obj = bpf_object__open_file(filename, NULL);
+ err = libbpf_get_error(obj);
+ if (err)
goto err_out;
+ err = bpf_object__load(obj);
+ if (err)
+ goto err_out;
+ prog_fd = bpf_program__fd(bpf_object__next_program(obj, NULL));
if (attach_egress_prog)
group_all = bpf_object__find_map_fd_by_name(obj, "map_egress");
diff --git a/tools/testing/selftests/bpf/xdping.c b/tools/testing/selftests/bpf/xdping.c
index 30f1263..baa870a 100644
--- a/tools/testing/selftests/bpf/xdping.c
+++ b/tools/testing/selftests/bpf/xdping.c
@@ -22,6 +22,7 @@
#include "bpf/libbpf.h"
#include "xdping.h"
+#include "testing_helpers.h"
static int ifindex;
static __u32 xdp_flags = XDP_FLAGS_UPDATE_IF_NOEXIST;
@@ -173,7 +174,7 @@ int main(int argc, char **argv)
snprintf(filename, sizeof(filename), "%s_kern.o", argv[0]);
- if (bpf_prog_load(filename, BPF_PROG_TYPE_XDP, &obj, &prog_fd)) {
+ if (bpf_prog_test_load(filename, BPF_PROG_TYPE_XDP, &obj, &prog_fd)) {
fprintf(stderr, "load of %s failed\n", filename);
return 1;
}
diff --git a/tools/testing/selftests/bpf/xdpxceiver.c b/tools/testing/selftests/bpf/xdpxceiver.c
index 6c7cf8a..0a5d23d 100644
--- a/tools/testing/selftests/bpf/xdpxceiver.c
+++ b/tools/testing/selftests/bpf/xdpxceiver.c
@@ -100,6 +100,12 @@
#include "xdpxceiver.h"
#include "../kselftest.h"
+/* AF_XDP APIs were moved into libxdp and marked as deprecated in libbpf.
+ * Until xdpxceiver is either moved or re-writed into libxdp, suppress
+ * deprecation warnings in this file
+ */
+#pragma GCC diagnostic ignored "-Wdeprecated-declarations"
+
static const char *MAC1 = "\x00\x0A\x56\x9E\xEE\x62";
static const char *MAC2 = "\x00\x0A\x56\x9E\xEE\x61";
static const char *IP1 = "192.168.100.162";
@@ -744,7 +750,6 @@ static void receive_pkts(struct pkt_stream *pkt_stream, struct xsk_socket_info *
struct pkt *pkt = pkt_stream_get_next_rx_pkt(pkt_stream);
struct xsk_umem_info *umem = xsk->umem;
u32 idx_rx = 0, idx_fq = 0, rcvd, i;
- u32 total = 0;
int ret;
while (pkt) {
@@ -799,7 +804,6 @@ static void receive_pkts(struct pkt_stream *pkt_stream, struct xsk_socket_info *
pthread_mutex_lock(&pacing_mutex);
pkts_in_flight -= rcvd;
- total += rcvd;
if (pkts_in_flight < umem->num_frames)
pthread_cond_signal(&pacing_cond);
pthread_mutex_unlock(&pacing_mutex);
@@ -1219,7 +1223,7 @@ static bool hugepages_present(struct ifobject *ifobject)
void *bufs;
bufs = mmap(NULL, mmap_sz, PROT_READ | PROT_WRITE,
- MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE | MAP_HUGETLB, -1, 0);
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_HUGETLB, -1, 0);
if (bufs == MAP_FAILED)
return false;
@@ -1366,6 +1370,10 @@ static void run_pkt_test(struct test_spec *test, enum test_mode mode, enum test_
testapp_invalid_desc(test);
break;
case TEST_TYPE_UNALIGNED_INV_DESC:
+ if (!hugepages_present(test->ifobj_tx)) {
+ ksft_test_result_skip("No 2M huge pages present.\n");
+ return;
+ }
test_spec_set_name(test, "UNALIGNED_INV_DESC");
test->ifobj_tx->umem->unaligned_mode = true;
test->ifobj_rx->umem->unaligned_mode = true;
diff --git a/tools/testing/selftests/cgroup/Makefile b/tools/testing/selftests/cgroup/Makefile
index 59e2224..745fe25 100644
--- a/tools/testing/selftests/cgroup/Makefile
+++ b/tools/testing/selftests/cgroup/Makefile
@@ -11,10 +11,12 @@
TEST_GEN_PROGS += test_freezer
TEST_GEN_PROGS += test_kill
+LOCAL_HDRS += $(selfdir)/clone3/clone3_selftests.h $(selfdir)/pidfd/pidfd.h
+
include ../lib.mk
-$(OUTPUT)/test_memcontrol: cgroup_util.c ../clone3/clone3_selftests.h
-$(OUTPUT)/test_kmem: cgroup_util.c ../clone3/clone3_selftests.h
-$(OUTPUT)/test_core: cgroup_util.c ../clone3/clone3_selftests.h
-$(OUTPUT)/test_freezer: cgroup_util.c ../clone3/clone3_selftests.h
-$(OUTPUT)/test_kill: cgroup_util.c ../clone3/clone3_selftests.h ../pidfd/pidfd.h
+$(OUTPUT)/test_memcontrol: cgroup_util.c
+$(OUTPUT)/test_kmem: cgroup_util.c
+$(OUTPUT)/test_core: cgroup_util.c
+$(OUTPUT)/test_freezer: cgroup_util.c
+$(OUTPUT)/test_kill: cgroup_util.c
diff --git a/tools/testing/selftests/cgroup/cgroup_util.c b/tools/testing/selftests/cgroup/cgroup_util.c
index 623cec0..0cf7e90 100644
--- a/tools/testing/selftests/cgroup/cgroup_util.c
+++ b/tools/testing/selftests/cgroup/cgroup_util.c
@@ -221,7 +221,7 @@ int cg_find_unified_root(char *root, size_t len)
int cg_create(const char *cgroup)
{
- return mkdir(cgroup, 0644);
+ return mkdir(cgroup, 0755);
}
int cg_wait_for_proc_count(const char *cgroup, int count)
diff --git a/tools/testing/selftests/cgroup/cgroup_util.h b/tools/testing/selftests/cgroup/cgroup_util.h
index 82e59cd..4f66d10 100644
--- a/tools/testing/selftests/cgroup/cgroup_util.h
+++ b/tools/testing/selftests/cgroup/cgroup_util.h
@@ -2,9 +2,9 @@
#include <stdbool.h>
#include <stdlib.h>
-#define PAGE_SIZE 4096
+#include "../kselftest.h"
-#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]))
+#define PAGE_SIZE 4096
#define MB(x) (x << 20)
diff --git a/tools/testing/selftests/cgroup/test_core.c b/tools/testing/selftests/cgroup/test_core.c
index 3df648c..6001235 100644
--- a/tools/testing/selftests/cgroup/test_core.c
+++ b/tools/testing/selftests/cgroup/test_core.c
@@ -1,11 +1,14 @@
/* SPDX-License-Identifier: GPL-2.0 */
+#define _GNU_SOURCE
#include <linux/limits.h>
+#include <linux/sched.h>
#include <sys/types.h>
#include <sys/mman.h>
#include <sys/wait.h>
#include <unistd.h>
#include <fcntl.h>
+#include <sched.h>
#include <stdio.h>
#include <errno.h>
#include <signal.h>
@@ -674,6 +677,166 @@ static int test_cgcore_thread_migration(const char *root)
return ret;
}
+/*
+ * cgroup migration permission check should be performed based on the
+ * credentials at the time of open instead of write.
+ */
+static int test_cgcore_lesser_euid_open(const char *root)
+{
+ const uid_t test_euid = 65534; /* usually nobody, any !root is fine */
+ int ret = KSFT_FAIL;
+ char *cg_test_a = NULL, *cg_test_b = NULL;
+ char *cg_test_a_procs = NULL, *cg_test_b_procs = NULL;
+ int cg_test_b_procs_fd = -1;
+ uid_t saved_uid;
+
+ cg_test_a = cg_name(root, "cg_test_a");
+ cg_test_b = cg_name(root, "cg_test_b");
+
+ if (!cg_test_a || !cg_test_b)
+ goto cleanup;
+
+ cg_test_a_procs = cg_name(cg_test_a, "cgroup.procs");
+ cg_test_b_procs = cg_name(cg_test_b, "cgroup.procs");
+
+ if (!cg_test_a_procs || !cg_test_b_procs)
+ goto cleanup;
+
+ if (cg_create(cg_test_a) || cg_create(cg_test_b))
+ goto cleanup;
+
+ if (cg_enter_current(cg_test_a))
+ goto cleanup;
+
+ if (chown(cg_test_a_procs, test_euid, -1) ||
+ chown(cg_test_b_procs, test_euid, -1))
+ goto cleanup;
+
+ saved_uid = geteuid();
+ if (seteuid(test_euid))
+ goto cleanup;
+
+ cg_test_b_procs_fd = open(cg_test_b_procs, O_RDWR);
+
+ if (seteuid(saved_uid))
+ goto cleanup;
+
+ if (cg_test_b_procs_fd < 0)
+ goto cleanup;
+
+ if (write(cg_test_b_procs_fd, "0", 1) >= 0 || errno != EACCES)
+ goto cleanup;
+
+ ret = KSFT_PASS;
+
+cleanup:
+ cg_enter_current(root);
+ if (cg_test_b_procs_fd >= 0)
+ close(cg_test_b_procs_fd);
+ if (cg_test_b)
+ cg_destroy(cg_test_b);
+ if (cg_test_a)
+ cg_destroy(cg_test_a);
+ free(cg_test_b_procs);
+ free(cg_test_a_procs);
+ free(cg_test_b);
+ free(cg_test_a);
+ return ret;
+}
+
+struct lesser_ns_open_thread_arg {
+ const char *path;
+ int fd;
+ int err;
+};
+
+static int lesser_ns_open_thread_fn(void *arg)
+{
+ struct lesser_ns_open_thread_arg *targ = arg;
+
+ targ->fd = open(targ->path, O_RDWR);
+ targ->err = errno;
+ return 0;
+}
+
+/*
+ * cgroup migration permission check should be performed based on the cgroup
+ * namespace at the time of open instead of write.
+ */
+static int test_cgcore_lesser_ns_open(const char *root)
+{
+ static char stack[65536];
+ const uid_t test_euid = 65534; /* usually nobody, any !root is fine */
+ int ret = KSFT_FAIL;
+ char *cg_test_a = NULL, *cg_test_b = NULL;
+ char *cg_test_a_procs = NULL, *cg_test_b_procs = NULL;
+ int cg_test_b_procs_fd = -1;
+ struct lesser_ns_open_thread_arg targ = { .fd = -1 };
+ pid_t pid;
+ int status;
+
+ cg_test_a = cg_name(root, "cg_test_a");
+ cg_test_b = cg_name(root, "cg_test_b");
+
+ if (!cg_test_a || !cg_test_b)
+ goto cleanup;
+
+ cg_test_a_procs = cg_name(cg_test_a, "cgroup.procs");
+ cg_test_b_procs = cg_name(cg_test_b, "cgroup.procs");
+
+ if (!cg_test_a_procs || !cg_test_b_procs)
+ goto cleanup;
+
+ if (cg_create(cg_test_a) || cg_create(cg_test_b))
+ goto cleanup;
+
+ if (cg_enter_current(cg_test_b))
+ goto cleanup;
+
+ if (chown(cg_test_a_procs, test_euid, -1) ||
+ chown(cg_test_b_procs, test_euid, -1))
+ goto cleanup;
+
+ targ.path = cg_test_b_procs;
+ pid = clone(lesser_ns_open_thread_fn, stack + sizeof(stack),
+ CLONE_NEWCGROUP | CLONE_FILES | CLONE_VM | SIGCHLD,
+ &targ);
+ if (pid < 0)
+ goto cleanup;
+
+ if (waitpid(pid, &status, 0) < 0)
+ goto cleanup;
+
+ if (!WIFEXITED(status))
+ goto cleanup;
+
+ cg_test_b_procs_fd = targ.fd;
+ if (cg_test_b_procs_fd < 0)
+ goto cleanup;
+
+ if (cg_enter_current(cg_test_a))
+ goto cleanup;
+
+ if ((status = write(cg_test_b_procs_fd, "0", 1)) >= 0 || errno != ENOENT)
+ goto cleanup;
+
+ ret = KSFT_PASS;
+
+cleanup:
+ cg_enter_current(root);
+ if (cg_test_b_procs_fd >= 0)
+ close(cg_test_b_procs_fd);
+ if (cg_test_b)
+ cg_destroy(cg_test_b);
+ if (cg_test_a)
+ cg_destroy(cg_test_a);
+ free(cg_test_b_procs);
+ free(cg_test_a_procs);
+ free(cg_test_b);
+ free(cg_test_a);
+ return ret;
+}
+
#define T(x) { x, #x }
struct corecg_test {
int (*fn)(const char *root);
@@ -689,6 +852,8 @@ struct corecg_test {
T(test_cgcore_proc_migration),
T(test_cgcore_thread_migration),
T(test_cgcore_destroy),
+ T(test_cgcore_lesser_euid_open),
+ T(test_cgcore_lesser_ns_open),
};
#undef T
diff --git a/tools/testing/selftests/clone3/clone3.c b/tools/testing/selftests/clone3/clone3.c
index 42be3b9..076cf43 100644
--- a/tools/testing/selftests/clone3/clone3.c
+++ b/tools/testing/selftests/clone3/clone3.c
@@ -52,6 +52,12 @@ static int call_clone3(uint64_t flags, size_t size, enum test_mode test_mode)
size = sizeof(struct __clone_args);
switch (test_mode) {
+ case CLONE3_ARGS_NO_TEST:
+ /*
+ * Uses default 'flags' and 'SIGCHLD'
+ * assignment.
+ */
+ break;
case CLONE3_ARGS_ALL_0:
args.flags = 0;
args.exit_signal = 0;
diff --git a/tools/testing/selftests/core/close_range_test.c b/tools/testing/selftests/core/close_range_test.c
index aa7d13d..7492399 100644
--- a/tools/testing/selftests/core/close_range_test.c
+++ b/tools/testing/selftests/core/close_range_test.c
@@ -50,10 +50,6 @@ static inline int sys_close_range(unsigned int fd, unsigned int max_fd,
return syscall(__NR_close_range, fd, max_fd, flags);
}
-#ifndef ARRAY_SIZE
-#define ARRAY_SIZE(x) (sizeof(x) / sizeof((x)[0]))
-#endif
-
TEST(core_close_range)
{
int i, ret;
diff --git a/tools/testing/selftests/drivers/net/mlxsw/devlink_trap_tunnel_vxlan.sh b/tools/testing/selftests/drivers/net/mlxsw/devlink_trap_tunnel_vxlan.sh
index 10e0f3d..5f6eb96 100755
--- a/tools/testing/selftests/drivers/net/mlxsw/devlink_trap_tunnel_vxlan.sh
+++ b/tools/testing/selftests/drivers/net/mlxsw/devlink_trap_tunnel_vxlan.sh
@@ -217,9 +217,11 @@
dest_mac=$(mac_get $h1)
p=$(:
)"08:"$( : VXLAN flags
- )"01:00:00:"$( : VXLAN reserved
+ )"00:00:00:"$( : VXLAN reserved
)"00:03:e8:"$( : VXLAN VNI : 1000
)"00:"$( : VXLAN reserved
+ )"$dest_mac:"$( : ETH daddr
+ )"00:00:00:00:00:00:"$( : ETH saddr
)
echo $p
}
@@ -263,7 +265,8 @@
corrupted_packet_test "Decap error: Reserved bits in use" \
"reserved_bits_payload_get"
- corrupted_packet_test "Decap error: No L2 header" "short_payload_get"
+ corrupted_packet_test "Decap error: Too short inner packet" \
+ "short_payload_get"
}
mc_smac_payload_get()
diff --git a/tools/testing/selftests/drivers/net/mlxsw/devlink_trap_tunnel_vxlan_ipv6.sh b/tools/testing/selftests/drivers/net/mlxsw/devlink_trap_tunnel_vxlan_ipv6.sh
new file mode 100755
index 0000000..f6c16cb
--- /dev/null
+++ b/tools/testing/selftests/drivers/net/mlxsw/devlink_trap_tunnel_vxlan_ipv6.sh
@@ -0,0 +1,342 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# Test devlink-trap tunnel drops and exceptions functionality over mlxsw.
+# Check all traps to make sure they are triggered under the right
+# conditions.
+
+# +------------------------+
+# | H1 (vrf) |
+# | + $h1 |
+# | | 2001:db8:1::1/64 |
+# +----|-------------------+
+# |
+# +----|----------------------------------------------------------------------+
+# | SW | |
+# | +--|--------------------------------------------------------------------+ |
+# | | + $swp1 BR1 (802.1d) | |
+# | | | |
+# | | + vx1 (vxlan) | |
+# | | local 2001:db8:3::1 | |
+# | | id 1000 dstport $VXPORT | |
+# | +-----------------------------------------------------------------------+ |
+# | |
+# | + $rp1 |
+# | | 2001:db8:3::1/64 |
+# +----|----------------------------------------------------------------------+
+# |
+# +----|--------------------------------------------------------+
+# | | VRF2 |
+# | + $rp2 |
+# | 2001:db8:3::2/64 |
+# | |
+# +-------------------------------------------------------------+
+
+lib_dir=$(dirname $0)/../../../net/forwarding
+
+ALL_TESTS="
+ decap_error_test
+ overlay_smac_is_mc_test
+"
+
+NUM_NETIFS=4
+source $lib_dir/lib.sh
+source $lib_dir/tc_common.sh
+source $lib_dir/devlink_lib.sh
+
+: ${VXPORT:=4789}
+export VXPORT
+
+h1_create()
+{
+ simple_if_init $h1 2001:db8:1::1/64
+}
+
+h1_destroy()
+{
+ simple_if_fini $h1 2001:db8:1::1/64
+}
+
+switch_create()
+{
+ ip link add name br1 type bridge vlan_filtering 0 mcast_snooping 0
+ # Make sure the bridge uses the MAC address of the local port and not
+ # that of the VxLAN's device.
+ ip link set dev br1 address $(mac_get $swp1)
+ ip link set dev br1 up
+
+ tc qdisc add dev $swp1 clsact
+ ip link set dev $swp1 master br1
+ ip link set dev $swp1 up
+
+ ip link add name vx1 type vxlan id 1000 local 2001:db8:3::1 \
+ dstport "$VXPORT" nolearning udp6zerocsumrx udp6zerocsumtx \
+ tos inherit ttl 100
+ ip link set dev vx1 master br1
+ ip link set dev vx1 up
+
+ ip link set dev $rp1 up
+ ip address add dev $rp1 2001:db8:3::1/64
+}
+
+switch_destroy()
+{
+ ip address del dev $rp1 2001:db8:3::1/64
+ ip link set dev $rp1 down
+
+ ip link set dev vx1 down
+ ip link set dev vx1 nomaster
+ ip link del dev vx1
+
+ ip link set dev $swp1 down
+ ip link set dev $swp1 nomaster
+ tc qdisc del dev $swp1 clsact
+
+ ip link set dev br1 down
+ ip link del dev br1
+}
+
+vrf2_create()
+{
+ simple_if_init $rp2 2001:db8:3::2/64
+}
+
+vrf2_destroy()
+{
+ simple_if_fini $rp2 2001:db8:3::2/64
+}
+
+setup_prepare()
+{
+ h1=${NETIFS[p1]}
+ swp1=${NETIFS[p2]}
+
+ rp1=${NETIFS[p3]}
+ rp2=${NETIFS[p4]}
+
+ vrf_prepare
+ forwarding_enable
+ h1_create
+ switch_create
+ vrf2_create
+}
+
+cleanup()
+{
+ pre_cleanup
+
+ vrf2_destroy
+ switch_destroy
+ h1_destroy
+ forwarding_restore
+ vrf_cleanup
+}
+
+ecn_payload_get()
+{
+ local dest_mac=$(mac_get $h1)
+ local saddr="20:01:0d:b8:00:01:00:00:00:00:00:00:00:00:00:03"
+ local daddr="20:01:0d:b8:00:01:00:00:00:00:00:00:00:00:00:01"
+ p=$(:
+ )"08:"$( : VXLAN flags
+ )"00:00:00:"$( : VXLAN reserved
+ )"00:03:e8:"$( : VXLAN VNI : 1000
+ )"00:"$( : VXLAN reserved
+ )"$dest_mac:"$( : ETH daddr
+ )"00:00:00:00:00:00:"$( : ETH saddr
+ )"86:dd:"$( : ETH type
+ )"6"$( : IP version
+ )"0:0"$( : Traffic class
+ )"0:00:00:"$( : Flow label
+ )"00:08:"$( : Payload length
+ )"3a:"$( : Next header
+ )"04:"$( : Hop limit
+ )"$saddr:"$( : IP saddr
+ )"$daddr:"$( : IP daddr
+ )"80:"$( : ICMPv6.type
+ )"00:"$( : ICMPv6.code
+ )"00:"$( : ICMPv6.checksum
+ )
+ echo $p
+}
+
+ecn_decap_test()
+{
+ local trap_name="decap_error"
+ local desc=$1; shift
+ local ecn_desc=$1; shift
+ local outer_tos=$1; shift
+ local mz_pid
+
+ RET=0
+
+ tc filter add dev $swp1 egress protocol ipv6 pref 1 handle 101 \
+ flower src_ip 2001:db8:1::3 dst_ip 2001:db8:1::1 action pass
+
+ rp1_mac=$(mac_get $rp1)
+ payload=$(ecn_payload_get)
+
+ ip vrf exec v$rp2 $MZ -6 $rp2 -c 0 -d 1msec -b $rp1_mac \
+ -B 2001:db8:3::1 -t udp \
+ sp=12345,dp=$VXPORT,tos=$outer_tos,p=$payload -q &
+ mz_pid=$!
+
+ devlink_trap_exception_test $trap_name
+
+ tc_check_packets "dev $swp1 egress" 101 0
+ check_err $? "Packets were not dropped"
+
+ log_test "$desc: Inner ECN is not ECT and outer is $ecn_desc"
+
+ kill $mz_pid && wait $mz_pid &> /dev/null
+ tc filter del dev $swp1 egress protocol ipv6 pref 1 handle 101 flower
+}
+
+reserved_bits_payload_get()
+{
+ local dest_mac=$(mac_get $h1)
+ local saddr="20:01:0d:b8:00:01:00:00:00:00:00:00:00:00:00:03"
+ local daddr="20:01:0d:b8:00:01:00:00:00:00:00:00:00:00:00:01"
+ p=$(:
+ )"08:"$( : VXLAN flags
+ )"01:00:00:"$( : VXLAN reserved
+ )"00:03:e8:"$( : VXLAN VNI : 1000
+ )"00:"$( : VXLAN reserved
+ )"$dest_mac:"$( : ETH daddr
+ )"00:00:00:00:00:00:"$( : ETH saddr
+ )"86:dd:"$( : ETH type
+ )"6"$( : IP version
+ )"0:0"$( : Traffic class
+ )"0:00:00:"$( : Flow label
+ )"00:08:"$( : Payload length
+ )"3a:"$( : Next header
+ )"04:"$( : Hop limit
+ )"$saddr:"$( : IP saddr
+ )"$daddr:"$( : IP daddr
+ )"80:"$( : ICMPv6.type
+ )"00:"$( : ICMPv6.code
+ )"00:"$( : ICMPv6.checksum
+ )
+ echo $p
+}
+
+short_payload_get()
+{
+ dest_mac=$(mac_get $h1)
+ p=$(:
+ )"08:"$( : VXLAN flags
+ )"00:00:00:"$( : VXLAN reserved
+ )"00:03:e8:"$( : VXLAN VNI : 1000
+ )"00:"$( : VXLAN reserved
+ )"$dest_mac:"$( : ETH daddr
+ )"00:00:00:00:00:00:"$( : ETH saddr
+ )
+ echo $p
+}
+
+corrupted_packet_test()
+{
+ local trap_name="decap_error"
+ local desc=$1; shift
+ local payload_get=$1; shift
+ local mz_pid
+
+ RET=0
+
+ # In case of too short packet, there is no any inner packet,
+ # so the matching will always succeed
+ tc filter add dev $swp1 egress protocol ipv6 pref 1 handle 101 \
+ flower skip_hw src_ip 2001:db8:3::1 dst_ip 2001:db8:1::1 \
+ action pass
+
+ rp1_mac=$(mac_get $rp1)
+ payload=$($payload_get)
+ ip vrf exec v$rp2 $MZ -6 $rp2 -c 0 -d 1msec -b $rp1_mac \
+ -B 2001:db8:3::1 -t udp sp=12345,dp=$VXPORT,p=$payload -q &
+ mz_pid=$!
+
+ devlink_trap_exception_test $trap_name
+
+ tc_check_packets "dev $swp1 egress" 101 0
+ check_err $? "Packets were not dropped"
+
+ log_test "$desc"
+
+ kill $mz_pid && wait $mz_pid &> /dev/null
+ tc filter del dev $swp1 egress protocol ipv6 pref 1 handle 101 flower
+}
+
+decap_error_test()
+{
+ ecn_decap_test "Decap error" "ECT(1)" 01
+ ecn_decap_test "Decap error" "ECT(0)" 02
+ ecn_decap_test "Decap error" "CE" 03
+
+ corrupted_packet_test "Decap error: Reserved bits in use" \
+ "reserved_bits_payload_get"
+ corrupted_packet_test "Decap error: Too short inner packet" \
+ "short_payload_get"
+}
+
+mc_smac_payload_get()
+{
+ local dest_mac=$(mac_get $h1)
+ local source_mac="01:02:03:04:05:06"
+ local saddr="20:01:0d:b8:00:01:00:00:00:00:00:00:00:00:00:03"
+ local daddr="20:01:0d:b8:00:01:00:00:00:00:00:00:00:00:00:01"
+ p=$(:
+ )"08:"$( : VXLAN flags
+ )"00:00:00:"$( : VXLAN reserved
+ )"00:03:e8:"$( : VXLAN VNI : 1000
+ )"00:"$( : VXLAN reserved
+ )"$dest_mac:"$( : ETH daddr
+ )"$source_mac:"$( : ETH saddr
+ )"86:dd:"$( : ETH type
+ )"6"$( : IP version
+ )"0:0"$( : Traffic class
+ )"0:00:00:"$( : Flow label
+ )"00:08:"$( : Payload length
+ )"3a:"$( : Next header
+ )"04:"$( : Hop limit
+ )"$saddr:"$( : IP saddr
+ )"$daddr:"$( : IP daddr
+ )"80:"$( : ICMPv6.type
+ )"00:"$( : ICMPv6.code
+ )"00:"$( : ICMPv6.checksum
+ )
+ echo $p
+}
+
+overlay_smac_is_mc_test()
+{
+ local trap_name="overlay_smac_is_mc"
+ local mz_pid
+
+ RET=0
+
+ # The matching will be checked on devlink_trap_drop_test()
+ # and the filter will be removed on devlink_trap_drop_cleanup()
+ tc filter add dev $swp1 egress protocol ipv6 pref 1 handle 101 \
+ flower src_mac 01:02:03:04:05:06 action pass
+
+ rp1_mac=$(mac_get $rp1)
+ payload=$(mc_smac_payload_get)
+
+ ip vrf exec v$rp2 $MZ -6 $rp2 -c 0 -d 1msec -b $rp1_mac \
+ -B 2001:db8:3::1 -t udp sp=12345,dp=$VXPORT,p=$payload -q &
+ mz_pid=$!
+
+ devlink_trap_drop_test $trap_name $swp1 101
+
+ log_test "Overlay source MAC is multicast"
+
+ devlink_trap_drop_cleanup $mz_pid $swp1 "ipv6" 1 101
+}
+
+trap cleanup EXIT
+
+setup_prepare
+setup_wait
+tests_run
+
+exit $EXIT_STATUS
diff --git a/tools/testing/selftests/drivers/net/mlxsw/spectrum-2/vxlan_flooding_ipv6.sh b/tools/testing/selftests/drivers/net/mlxsw/spectrum-2/vxlan_flooding_ipv6.sh
new file mode 100755
index 0000000..429f7ee
--- /dev/null
+++ b/tools/testing/selftests/drivers/net/mlxsw/spectrum-2/vxlan_flooding_ipv6.sh
@@ -0,0 +1,322 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# Test VxLAN flooding. The device stores flood records in a singly linked list
+# where each record stores up to four IPv6 addresses of remote VTEPs. The test
+# verifies that packets are correctly flooded in various cases such as deletion
+# of a record in the middle of the list.
+#
+# +-----------------------+
+# | H1 (vrf) |
+# | + $h1 |
+# | | 2001:db8:1::1/64 |
+# +----|------------------+
+# |
+# +----|----------------------------------------------------------------------+
+# | SW | |
+# | +--|--------------------------------------------------------------------+ |
+# | | + $swp1 BR0 (802.1d) | |
+# | | | |
+# | | + vxlan0 (vxlan) | |
+# | | local 2001:db8:2::1 | |
+# | | remote 2001:db8:2::{2..17} | |
+# | | id 10 dstport 4789 | |
+# | +-----------------------------------------------------------------------+ |
+# | |
+# | 2001:db8:2::0/64 via 2001:db8:3::2 |
+# | |
+# | + $rp1 |
+# | | 2001:db8:3::1/64 |
+# +----|----------------------------------------------------------------------+
+# |
+# +----|--------------------------------------------------------+
+# | | R2 (vrf) |
+# | + $rp2 |
+# | 2001:db8:3::2/64 |
+# | |
+# +-------------------------------------------------------------+
+
+lib_dir=$(dirname $0)/../../../../net/forwarding
+
+ALL_TESTS="flooding_test"
+NUM_NETIFS=4
+source $lib_dir/tc_common.sh
+source $lib_dir/lib.sh
+
+h1_create()
+{
+ simple_if_init $h1 2001:db8:1::1/64
+}
+
+h1_destroy()
+{
+ simple_if_fini $h1 2001:db8:1::1/64
+}
+
+switch_create()
+{
+ # Make sure the bridge uses the MAC address of the local port and
+ # not that of the VxLAN's device
+ ip link add dev br0 type bridge mcast_snooping 0
+ ip link set dev br0 address $(mac_get $swp1)
+
+ ip link add name vxlan0 type vxlan id 10 nolearning \
+ udp6zerocsumrx udp6zerocsumtx ttl 20 tos inherit \
+ local 2001:db8:2::1 dstport 4789
+
+ ip address add 2001:db8:2::1/128 dev lo
+
+ ip link set dev $swp1 master br0
+ ip link set dev vxlan0 master br0
+
+ ip link set dev br0 up
+ ip link set dev $swp1 up
+ ip link set dev vxlan0 up
+}
+
+switch_destroy()
+{
+ ip link set dev vxlan0 down
+ ip link set dev $swp1 down
+ ip link set dev br0 down
+
+ ip link set dev vxlan0 nomaster
+ ip link set dev $swp1 nomaster
+
+ ip address del 2001:db8:2::1/128 dev lo
+
+ ip link del dev vxlan0
+
+ ip link del dev br0
+}
+
+router1_create()
+{
+ # This router is in the default VRF, where the VxLAN device is
+ # performing the L3 lookup
+ ip link set dev $rp1 up
+ ip address add 2001:db8:3::1/64 dev $rp1
+ ip route add 2001:db8:2::0/64 via 2001:db8:3::2
+}
+
+router1_destroy()
+{
+ ip route del 2001:db8:2::0/64 via 2001:db8:3::2
+ ip address del 2001:db8:3::1/64 dev $rp1
+ ip link set dev $rp1 down
+}
+
+router2_create()
+{
+ # This router is not in the default VRF, so use simple_if_init()
+ simple_if_init $rp2 2001:db8:3::2/64
+}
+
+router2_destroy()
+{
+ simple_if_fini $rp2 2001:db8:3::2/64
+}
+
+setup_prepare()
+{
+ h1=${NETIFS[p1]}
+ swp1=${NETIFS[p2]}
+
+ rp1=${NETIFS[p3]}
+ rp2=${NETIFS[p4]}
+
+ vrf_prepare
+
+ h1_create
+
+ switch_create
+
+ router1_create
+ router2_create
+
+ forwarding_enable
+}
+
+cleanup()
+{
+ pre_cleanup
+
+ forwarding_restore
+
+ router2_destroy
+ router1_destroy
+
+ switch_destroy
+
+ h1_destroy
+
+ vrf_cleanup
+}
+
+flooding_remotes_add()
+{
+ local num_remotes=$1
+ local lsb
+ local i
+
+ for i in $(eval echo {1..$num_remotes}); do
+ lsb=$((i + 1))
+
+ bridge fdb append 00:00:00:00:00:00 dev vxlan0 self \
+ dst 2001:db8:2::$lsb
+ done
+}
+
+flooding_filters_add()
+{
+ local num_remotes=$1
+ local lsb
+ local i
+
+ tc qdisc add dev $rp2 clsact
+
+ for i in $(eval echo {1..$num_remotes}); do
+ lsb=$((i + 1))
+
+ tc filter add dev $rp2 ingress protocol ipv6 pref $i handle $i \
+ flower ip_proto udp dst_ip 2001:db8:2::$lsb \
+ dst_port 4789 skip_sw action drop
+ done
+}
+
+flooding_filters_del()
+{
+ local num_remotes=$1
+ local i
+
+ for i in $(eval echo {1..$num_remotes}); do
+ tc filter del dev $rp2 ingress protocol ipv6 pref $i \
+ handle $i flower
+ done
+
+ tc qdisc del dev $rp2 clsact
+}
+
+flooding_check_packets()
+{
+ local packets=("$@")
+ local num_remotes=${#packets[@]}
+ local i
+
+ for i in $(eval echo {1..$num_remotes}); do
+ tc_check_packets "dev $rp2 ingress" $i ${packets[i - 1]}
+ check_err $? "remote $i - did not get expected number of packets"
+ done
+}
+
+flooding_test()
+{
+ # Use 16 remote VTEPs that will be stored in 4 records. The array
+ # 'packets' will store how many packets are expected to be received
+ # by each remote VTEP at each stage of the test
+ declare -a packets=(1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1)
+ local num_remotes=16
+
+ RET=0
+
+ # Add FDB entries for remote VTEPs and corresponding tc filters on the
+ # ingress of the nexthop router. These filters will count how many
+ # packets were flooded to each remote VTEP
+ flooding_remotes_add $num_remotes
+ flooding_filters_add $num_remotes
+
+ # Send one packet and make sure it is flooded to all the remote VTEPs
+ $MZ $h1 -q -p 64 -b de:ad:be:ef:13:37 -t ip -c 1
+ flooding_check_packets "${packets[@]}"
+ log_test "flood after 1 packet"
+
+ # Delete the third record which corresponds to VTEPs with LSB 10..13
+ # and check that packet is flooded correctly when we remove a record
+ # from the middle of the list
+ RET=0
+
+ packets=(2 2 2 2 2 2 2 2 1 1 1 1 2 2 2 2)
+ bridge fdb del 00:00:00:00:00:00 dev vxlan0 self dst 2001:db8:2::10
+ bridge fdb del 00:00:00:00:00:00 dev vxlan0 self dst 2001:db8:2::11
+ bridge fdb del 00:00:00:00:00:00 dev vxlan0 self dst 2001:db8:2::12
+ bridge fdb del 00:00:00:00:00:00 dev vxlan0 self dst 2001:db8:2::13
+
+ $MZ $h1 -q -p 64 -b de:ad:be:ef:13:37 -t ip -c 1
+ flooding_check_packets "${packets[@]}"
+ log_test "flood after 2 packets"
+
+ # Delete the first record and make sure the packet is flooded correctly
+ RET=0
+
+ packets=(2 2 2 2 3 3 3 3 1 1 1 1 3 3 3 3)
+ bridge fdb del 00:00:00:00:00:00 dev vxlan0 self dst 2001:db8:2::2
+ bridge fdb del 00:00:00:00:00:00 dev vxlan0 self dst 2001:db8:2::3
+ bridge fdb del 00:00:00:00:00:00 dev vxlan0 self dst 2001:db8:2::4
+ bridge fdb del 00:00:00:00:00:00 dev vxlan0 self dst 2001:db8:2::5
+
+ $MZ $h1 -q -p 64 -b de:ad:be:ef:13:37 -t ip -c 1
+ flooding_check_packets "${packets[@]}"
+ log_test "flood after 3 packets"
+
+ # Delete the last record and make sure the packet is flooded correctly
+ RET=0
+
+ packets=(2 2 2 2 4 4 4 4 1 1 1 1 3 3 3 3)
+ bridge fdb del 00:00:00:00:00:00 dev vxlan0 self dst 2001:db8:2::14
+ bridge fdb del 00:00:00:00:00:00 dev vxlan0 self dst 2001:db8:2::15
+ bridge fdb del 00:00:00:00:00:00 dev vxlan0 self dst 2001:db8:2::16
+ bridge fdb del 00:00:00:00:00:00 dev vxlan0 self dst 2001:db8:2::17
+
+ $MZ -6 $h1 -q -p 64 -b de:ad:be:ef:13:37 -t ip -c 1
+ flooding_check_packets "${packets[@]}"
+ log_test "flood after 4 packets"
+
+ # Delete the last record, one entry at a time and make sure single
+ # entries are correctly removed
+ RET=0
+
+ packets=(2 2 2 2 4 5 5 5 1 1 1 1 3 3 3 3)
+ bridge fdb del 00:00:00:00:00:00 dev vxlan0 self dst 2001:db8:2::6
+
+ $MZ -6 $h1 -q -p 64 -b de:ad:be:ef:13:37 -t ip -c 1
+ flooding_check_packets "${packets[@]}"
+ log_test "flood after 5 packets"
+
+ RET=0
+
+ packets=(2 2 2 2 4 5 6 6 1 1 1 1 3 3 3 3)
+ bridge fdb del 00:00:00:00:00:00 dev vxlan0 self dst 2001:db8:2::7
+
+ $MZ -6 $h1 -q -p 64 -b de:ad:be:ef:13:37 -t ip -c 1
+ flooding_check_packets "${packets[@]}"
+ log_test "flood after 6 packets"
+
+ RET=0
+
+ packets=(2 2 2 2 4 5 6 7 1 1 1 1 3 3 3 3)
+ bridge fdb del 00:00:00:00:00:00 dev vxlan0 self dst 2001:db8:2::8
+
+ $MZ -6 $h1 -q -p 64 -b de:ad:be:ef:13:37 -t ip -c 1
+ flooding_check_packets "${packets[@]}"
+ log_test "flood after 7 packets"
+
+ RET=0
+
+ packets=(2 2 2 2 4 5 6 7 1 1 1 1 3 3 3 3)
+ bridge fdb del 00:00:00:00:00:00 dev vxlan0 self dst 2001:db8:2::9
+
+ $MZ -6 $h1 -q -p 64 -b de:ad:be:ef:13:37 -t ip -c 1
+ flooding_check_packets "${packets[@]}"
+ log_test "flood after 8 packets"
+
+ flooding_filters_del $num_remotes
+}
+
+trap cleanup EXIT
+
+setup_prepare
+setup_wait
+
+tests_run
+
+exit $EXIT_STATUS
diff --git a/tools/testing/selftests/drivers/net/mlxsw/spectrum/vxlan_flooding_ipv6.sh b/tools/testing/selftests/drivers/net/mlxsw/spectrum/vxlan_flooding_ipv6.sh
new file mode 100755
index 0000000..d8fd875
--- /dev/null
+++ b/tools/testing/selftests/drivers/net/mlxsw/spectrum/vxlan_flooding_ipv6.sh
@@ -0,0 +1,334 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# Test VxLAN flooding. The device stores flood records in a singly linked list
+# where each record stores up to five IPv6 addresses of remote VTEPs. The test
+# verifies that packets are correctly flooded in various cases such as deletion
+# of a record in the middle of the list.
+#
+# +-----------------------+
+# | H1 (vrf) |
+# | + $h1 |
+# | | 2001:db8:1::1/64 |
+# +----|------------------+
+# |
+# +----|----------------------------------------------------------------------+
+# | SW | |
+# | +--|--------------------------------------------------------------------+ |
+# | | + $swp1 BR0 (802.1d) | |
+# | | | |
+# | | + vxlan0 (vxlan) | |
+# | | local 2001:db8:2::1 | |
+# | | remote 2001:db8:2::{2..21} | |
+# | | id 10 dstport 4789 | |
+# | +-----------------------------------------------------------------------+ |
+# | |
+# | 2001:db8:2::0/64 via 2001:db8:3::2 |
+# | |
+# | + $rp1 |
+# | | 2001:db8:3::1/64 |
+# +----|----------------------------------------------------------------------+
+# |
+# +----|--------------------------------------------------------+
+# | | R2 (vrf) |
+# | + $rp2 |
+# | 2001:db8:3::2/64 |
+# | |
+# +-------------------------------------------------------------+
+
+lib_dir=$(dirname $0)/../../../../net/forwarding
+
+ALL_TESTS="flooding_test"
+NUM_NETIFS=4
+source $lib_dir/tc_common.sh
+source $lib_dir/lib.sh
+
+h1_create()
+{
+ simple_if_init $h1 2001:db8:1::1/64
+}
+
+h1_destroy()
+{
+ simple_if_fini $h1 2001:db8:1::1/64
+}
+
+switch_create()
+{
+ # Make sure the bridge uses the MAC address of the local port and
+ # not that of the VxLAN's device
+ ip link add dev br0 type bridge mcast_snooping 0
+ ip link set dev br0 address $(mac_get $swp1)
+
+ ip link add name vxlan0 type vxlan id 10 nolearning \
+ udp6zerocsumrx udp6zerocsumtx ttl 20 tos inherit \
+ local 2001:db8:2::1 dstport 4789
+
+ ip address add 2001:db8:2::1/128 dev lo
+
+ ip link set dev $swp1 master br0
+ ip link set dev vxlan0 master br0
+
+ ip link set dev br0 up
+ ip link set dev $swp1 up
+ ip link set dev vxlan0 up
+}
+
+switch_destroy()
+{
+ ip link set dev vxlan0 down
+ ip link set dev $swp1 down
+ ip link set dev br0 down
+
+ ip link set dev vxlan0 nomaster
+ ip link set dev $swp1 nomaster
+
+ ip address del 2001:db8:2::1/128 dev lo
+
+ ip link del dev vxlan0
+
+ ip link del dev br0
+}
+
+router1_create()
+{
+ # This router is in the default VRF, where the VxLAN device is
+ # performing the L3 lookup
+ ip link set dev $rp1 up
+ ip address add 2001:db8:3::1/64 dev $rp1
+ ip route add 2001:db8:2::0/64 via 2001:db8:3::2
+}
+
+router1_destroy()
+{
+ ip route del 2001:db8:2::0/64 via 2001:db8:3::2
+ ip address del 2001:db8:3::1/64 dev $rp1
+ ip link set dev $rp1 down
+}
+
+router2_create()
+{
+ # This router is not in the default VRF, so use simple_if_init()
+ simple_if_init $rp2 2001:db8:3::2/64
+}
+
+router2_destroy()
+{
+ simple_if_fini $rp2 2001:db8:3::2/64
+}
+
+setup_prepare()
+{
+ h1=${NETIFS[p1]}
+ swp1=${NETIFS[p2]}
+
+ rp1=${NETIFS[p3]}
+ rp2=${NETIFS[p4]}
+
+ vrf_prepare
+
+ h1_create
+
+ switch_create
+
+ router1_create
+ router2_create
+
+ forwarding_enable
+}
+
+cleanup()
+{
+ pre_cleanup
+
+ forwarding_restore
+
+ router2_destroy
+ router1_destroy
+
+ switch_destroy
+
+ h1_destroy
+
+ vrf_cleanup
+}
+
+flooding_remotes_add()
+{
+ local num_remotes=$1
+ local lsb
+ local i
+
+ for i in $(eval echo {1..$num_remotes}); do
+ lsb=$((i + 1))
+
+ bridge fdb append 00:00:00:00:00:00 dev vxlan0 self \
+ dst 2001:db8:2::$lsb
+ done
+}
+
+flooding_filters_add()
+{
+ local num_remotes=$1
+ local lsb
+ local i
+
+ tc qdisc add dev $rp2 clsact
+
+ for i in $(eval echo {1..$num_remotes}); do
+ lsb=$((i + 1))
+
+ tc filter add dev $rp2 ingress protocol ipv6 pref $i handle $i \
+ flower ip_proto udp dst_ip 2001:db8:2::$lsb \
+ dst_port 4789 skip_sw action drop
+ done
+}
+
+flooding_filters_del()
+{
+ local num_remotes=$1
+ local i
+
+ for i in $(eval echo {1..$num_remotes}); do
+ tc filter del dev $rp2 ingress protocol ipv6 pref $i \
+ handle $i flower
+ done
+
+ tc qdisc del dev $rp2 clsact
+}
+
+flooding_check_packets()
+{
+ local packets=("$@")
+ local num_remotes=${#packets[@]}
+ local i
+
+ for i in $(eval echo {1..$num_remotes}); do
+ tc_check_packets "dev $rp2 ingress" $i ${packets[i - 1]}
+ check_err $? "remote $i - did not get expected number of packets"
+ done
+}
+
+flooding_test()
+{
+ # Use 20 remote VTEPs that will be stored in 4 records. The array
+ # 'packets' will store how many packets are expected to be received
+ # by each remote VTEP at each stage of the test
+ declare -a packets=(1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1)
+ local num_remotes=20
+
+ RET=0
+
+ # Add FDB entries for remote VTEPs and corresponding tc filters on the
+ # ingress of the nexthop router. These filters will count how many
+ # packets were flooded to each remote VTEP
+ flooding_remotes_add $num_remotes
+ flooding_filters_add $num_remotes
+
+ # Send one packet and make sure it is flooded to all the remote VTEPs
+ $MZ -6 $h1 -q -p 64 -b de:ad:be:ef:13:37 -t ip -c 1
+ flooding_check_packets "${packets[@]}"
+ log_test "flood after 1 packet"
+
+ # Delete the third record which corresponds to VTEPs with LSB 12..16
+ # and check that packet is flooded correctly when we remove a record
+ # from the middle of the list
+ RET=0
+
+ packets=(2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 2 2 2 2 2)
+ bridge fdb del 00:00:00:00:00:00 dev vxlan0 self dst 2001:db8:2::12
+ bridge fdb del 00:00:00:00:00:00 dev vxlan0 self dst 2001:db8:2::13
+ bridge fdb del 00:00:00:00:00:00 dev vxlan0 self dst 2001:db8:2::14
+ bridge fdb del 00:00:00:00:00:00 dev vxlan0 self dst 2001:db8:2::15
+ bridge fdb del 00:00:00:00:00:00 dev vxlan0 self dst 2001:db8:2::16
+
+ $MZ -6 $h1 -q -p 64 -b de:ad:be:ef:13:37 -t ip -c 1
+ flooding_check_packets "${packets[@]}"
+ log_test "flood after 2 packets"
+
+ # Delete the first record and make sure the packet is flooded correctly
+ RET=0
+
+ packets=(2 2 2 2 2 3 3 3 3 3 1 1 1 1 1 3 3 3 3 3)
+ bridge fdb del 00:00:00:00:00:00 dev vxlan0 self dst 2001:db8:2::2
+ bridge fdb del 00:00:00:00:00:00 dev vxlan0 self dst 2001:db8:2::3
+ bridge fdb del 00:00:00:00:00:00 dev vxlan0 self dst 2001:db8:2::4
+ bridge fdb del 00:00:00:00:00:00 dev vxlan0 self dst 2001:db8:2::5
+ bridge fdb del 00:00:00:00:00:00 dev vxlan0 self dst 2001:db8:2::6
+
+ $MZ -6 $h1 -q -p 64 -b de:ad:be:ef:13:37 -t ip -c 1
+ flooding_check_packets "${packets[@]}"
+ log_test "flood after 3 packets"
+
+ # Delete the last record and make sure the packet is flooded correctly
+ RET=0
+
+ packets=(2 2 2 2 2 4 4 4 4 4 1 1 1 1 1 3 3 3 3 3)
+ bridge fdb del 00:00:00:00:00:00 dev vxlan0 self dst 2001:db8:2::17
+ bridge fdb del 00:00:00:00:00:00 dev vxlan0 self dst 2001:db8:2::18
+ bridge fdb del 00:00:00:00:00:00 dev vxlan0 self dst 2001:db8:2::19
+ bridge fdb del 00:00:00:00:00:00 dev vxlan0 self dst 2001:db8:2::20
+ bridge fdb del 00:00:00:00:00:00 dev vxlan0 self dst 2001:db8:2::21
+
+ $MZ -6 $h1 -q -p 64 -b de:ad:be:ef:13:37 -t ip -c 1
+ flooding_check_packets "${packets[@]}"
+ log_test "flood after 4 packets"
+
+ # Delete the last record, one entry at a time and make sure single
+ # entries are correctly removed
+ RET=0
+
+ packets=(2 2 2 2 2 4 5 5 5 5 1 1 1 1 1 3 3 3 3 3)
+ bridge fdb del 00:00:00:00:00:00 dev vxlan0 self dst 2001:db8:2::7
+
+ $MZ -6 $h1 -q -p 64 -b de:ad:be:ef:13:37 -t ip -c 1
+ flooding_check_packets "${packets[@]}"
+ log_test "flood after 5 packets"
+
+ RET=0
+
+ packets=(2 2 2 2 2 4 5 6 6 6 1 1 1 1 1 3 3 3 3 3)
+ bridge fdb del 00:00:00:00:00:00 dev vxlan0 self dst 2001:db8:2::8
+
+ $MZ -6 $h1 -q -p 64 -b de:ad:be:ef:13:37 -t ip -c 1
+ flooding_check_packets "${packets[@]}"
+ log_test "flood after 6 packets"
+
+ RET=0
+
+ packets=(2 2 2 2 2 4 5 6 7 7 1 1 1 1 1 3 3 3 3 3)
+ bridge fdb del 00:00:00:00:00:00 dev vxlan0 self dst 2001:db8:2::9
+
+ $MZ -6 $h1 -q -p 64 -b de:ad:be:ef:13:37 -t ip -c 1
+ flooding_check_packets "${packets[@]}"
+ log_test "flood after 7 packets"
+
+ RET=0
+
+ packets=(2 2 2 2 2 4 5 6 7 8 1 1 1 1 1 3 3 3 3 3)
+ bridge fdb del 00:00:00:00:00:00 dev vxlan0 self dst 2001:db8:2::10
+
+ $MZ -6 $h1 -q -p 64 -b de:ad:be:ef:13:37 -t ip -c 1
+ flooding_check_packets "${packets[@]}"
+ log_test "flood after 8 packets"
+
+ RET=0
+
+ packets=(2 2 2 2 2 4 5 6 7 8 1 1 1 1 1 3 3 3 3 3)
+ bridge fdb del 00:00:00:00:00:00 dev vxlan0 self dst 2001:db8:2::11
+
+ $MZ -6 $h1 -q -p 64 -b de:ad:be:ef:13:37 -t ip -c 1
+ flooding_check_packets "${packets[@]}"
+ log_test "flood after 9 packets"
+
+ flooding_filters_del $num_remotes
+}
+
+trap cleanup EXIT
+
+setup_prepare
+setup_wait
+
+tests_run
+
+exit $EXIT_STATUS
diff --git a/tools/testing/selftests/drivers/net/mlxsw/vxlan.sh b/tools/testing/selftests/drivers/net/mlxsw/vxlan.sh
index 729a86c..99a332b 100755
--- a/tools/testing/selftests/drivers/net/mlxsw/vxlan.sh
+++ b/tools/testing/selftests/drivers/net/mlxsw/vxlan.sh
@@ -4,10 +4,35 @@
# Test various aspects of VxLAN offloading which are specific to mlxsw, such
# as sanitization of invalid configurations and offload indication.
-lib_dir=$(dirname $0)/../../../net/forwarding
+: ${ADDR_FAMILY:=ipv4}
+export ADDR_FAMILY
-ALL_TESTS="sanitization_test offload_indication_test \
- sanitization_vlan_aware_test offload_indication_vlan_aware_test"
+: ${LOCAL_IP_1:=198.51.100.1}
+export LOCAL_IP_1
+
+: ${LOCAL_IP_2:=198.51.100.2}
+export LOCAL_IP_2
+
+: ${PREFIX_LEN:=32}
+export PREFIX_LEN
+
+: ${UDPCSUM_FLAFS:=noudpcsum}
+export UDPCSUM_FLAFS
+
+: ${MC_IP:=239.0.0.1}
+export MC_IP
+
+: ${IP_FLAG:=""}
+export IP_FLAG
+
+: ${ALL_TESTS:="
+ sanitization_test
+ offload_indication_test
+ sanitization_vlan_aware_test
+ offload_indication_vlan_aware_test
+"}
+
+lib_dir=$(dirname $0)/../../../net/forwarding
NUM_NETIFS=2
: ${TIMEOUT:=20000} # ms
source $lib_dir/lib.sh
@@ -63,8 +88,8 @@
ip link add dev br0 type bridge mcast_snooping 0
- ip link add name vxlan0 up type vxlan id 10 nolearning noudpcsum \
- ttl 20 tos inherit local 198.51.100.1 dstport 4789
+ ip link add name vxlan0 up type vxlan id 10 nolearning $UDPCSUM_FLAFS \
+ ttl 20 tos inherit local $LOCAL_IP_1 dstport 4789
sanitization_single_dev_test_pass
@@ -80,8 +105,8 @@
ip link add dev br0 type bridge mcast_snooping 0 vlan_filtering 1
- ip link add name vxlan0 up type vxlan id 10 nolearning noudpcsum \
- ttl 20 tos inherit local 198.51.100.1 dstport 4789
+ ip link add name vxlan0 up type vxlan id 10 nolearning $UDPCSUM_FLAFS \
+ ttl 20 tos inherit local $LOCAL_IP_1 dstport 4789
sanitization_single_dev_test_pass
@@ -97,8 +122,8 @@
ip link add dev br0 type bridge
- ip link add name vxlan0 up type vxlan id 10 nolearning noudpcsum \
- ttl 20 tos inherit local 198.51.100.1 dstport 4789
+ ip link add name vxlan0 up type vxlan id 10 nolearning $UDPCSUM_FLAFS \
+ ttl 20 tos inherit local $LOCAL_IP_1 dstport 4789
sanitization_single_dev_test_fail
@@ -115,9 +140,9 @@
ip link add dev br0 type bridge mcast_snooping 0
ip link add name dummy1 up type dummy
- ip link add name vxlan0 up type vxlan id 10 nolearning noudpcsum \
- ttl 20 tos inherit local 198.51.100.1 dstport 4789 \
- dev dummy1 group 239.0.0.1
+ ip link add name vxlan0 up type vxlan id 10 nolearning $UDPCSUM_FLAFS \
+ ttl 20 tos inherit local $LOCAL_IP_1 dstport 4789 \
+ dev dummy1 group $MC_IP
sanitization_single_dev_test_fail
@@ -134,7 +159,7 @@
ip link add dev br0 type bridge mcast_snooping 0
- ip link add name vxlan0 up type vxlan id 10 nolearning noudpcsum \
+ ip link add name vxlan0 up type vxlan id 10 nolearning $UDPCSUM_FLAFS \
ttl 20 tos inherit dstport 4789
sanitization_single_dev_test_fail
@@ -145,31 +170,14 @@
log_test "vxlan device with no local ip"
}
-sanitization_single_dev_local_ipv6_test()
+sanitization_single_dev_learning_enabled_ipv4_test()
{
RET=0
ip link add dev br0 type bridge mcast_snooping 0
- ip link add name vxlan0 up type vxlan id 10 nolearning noudpcsum \
- ttl 20 tos inherit local 2001:db8::1 dstport 4789
-
- sanitization_single_dev_test_fail
-
- ip link del dev vxlan0
- ip link del dev br0
-
- log_test "vxlan device with local ipv6 address"
-}
-
-sanitization_single_dev_learning_enabled_test()
-{
- RET=0
-
- ip link add dev br0 type bridge mcast_snooping 0
-
- ip link add name vxlan0 up type vxlan id 10 learning noudpcsum \
- ttl 20 tos inherit local 198.51.100.1 dstport 4789
+ ip link add name vxlan0 up type vxlan id 10 learning $UDPCSUM_FLAFS \
+ ttl 20 tos inherit local $LOCAL_IP_1 dstport 4789
sanitization_single_dev_test_pass
@@ -186,8 +194,8 @@
ip link add dev br0 type bridge mcast_snooping 0
ip link add name dummy1 up type dummy
- ip link add name vxlan0 up type vxlan id 10 nolearning noudpcsum \
- ttl 20 tos inherit local 198.51.100.1 dstport 4789 dev dummy1
+ ip link add name vxlan0 up type vxlan id 10 nolearning $UDPCSUM_FLAFS \
+ ttl 20 tos inherit local $LOCAL_IP_1 dstport 4789 dev dummy1
sanitization_single_dev_test_fail
@@ -204,8 +212,8 @@
ip link add dev br0 type bridge mcast_snooping 0
- ip link add name vxlan0 up type vxlan id 10 nolearning noudpcsum \
- ttl 20 tos inherit local 198.51.100.1 dstport 4789 \
+ ip link add name vxlan0 up type vxlan id 10 nolearning $UDPCSUM_FLAFS \
+ ttl 20 tos inherit local $LOCAL_IP_1 dstport 4789 \
srcport 4000 5000
sanitization_single_dev_test_fail
@@ -222,8 +230,8 @@
ip link add dev br0 type bridge mcast_snooping 0
- ip link add name vxlan0 up type vxlan id 10 nolearning noudpcsum \
- ttl 20 tos 20 local 198.51.100.1 dstport 4789
+ ip link add name vxlan0 up type vxlan id 10 nolearning $UDPCSUM_FLAFS \
+ ttl 20 tos 20 local $LOCAL_IP_1 dstport 4789
sanitization_single_dev_test_fail
@@ -239,8 +247,8 @@
ip link add dev br0 type bridge mcast_snooping 0
- ip link add name vxlan0 up type vxlan id 10 nolearning noudpcsum \
- ttl inherit tos inherit local 198.51.100.1 dstport 4789
+ ip link add name vxlan0 up type vxlan id 10 nolearning $UDPCSUM_FLAFS \
+ ttl inherit tos inherit local $LOCAL_IP_1 dstport 4789
sanitization_single_dev_test_fail
@@ -250,14 +258,14 @@
log_test "vxlan device with inherit ttl"
}
-sanitization_single_dev_udp_checksum_test()
+sanitization_single_dev_udp_checksum_ipv4_test()
{
RET=0
ip link add dev br0 type bridge mcast_snooping 0
ip link add name vxlan0 up type vxlan id 10 nolearning udpcsum \
- ttl 20 tos inherit local 198.51.100.1 dstport 4789
+ ttl 20 tos inherit local $LOCAL_IP_1 dstport 4789
sanitization_single_dev_test_fail
@@ -276,13 +284,12 @@
sanitization_single_dev_mcast_enabled_test
sanitization_single_dev_mcast_group_test
sanitization_single_dev_no_local_ip_test
- sanitization_single_dev_local_ipv6_test
- sanitization_single_dev_learning_enabled_test
+ sanitization_single_dev_learning_enabled_"$ADDR_FAMILY"_test
sanitization_single_dev_local_interface_test
sanitization_single_dev_port_range_test
sanitization_single_dev_tos_static_test
sanitization_single_dev_ttl_inherit_test
- sanitization_single_dev_udp_checksum_test
+ sanitization_single_dev_udp_checksum_"$ADDR_FAMILY"_test
}
sanitization_multi_devs_test_pass()
@@ -334,10 +341,10 @@
ip link add dev br0 type bridge mcast_snooping 0
ip link add dev br1 type bridge mcast_snooping 0
- ip link add name vxlan0 up type vxlan id 10 nolearning noudpcsum \
- ttl 20 tos inherit local 198.51.100.1 dstport 4789
- ip link add name vxlan1 up type vxlan id 20 nolearning noudpcsum \
- ttl 20 tos inherit local 198.51.100.1 dstport 4789
+ ip link add name vxlan0 up type vxlan id 10 nolearning $UDPCSUM_FLAFS \
+ ttl 20 tos inherit local $LOCAL_IP_1 dstport 4789
+ ip link add name vxlan1 up type vxlan id 20 nolearning $UDPCSUM_FLAFS \
+ ttl 20 tos inherit local $LOCAL_IP_1 dstport 4789
sanitization_multi_devs_test_pass
@@ -356,10 +363,10 @@
ip link add dev br0 type bridge mcast_snooping 0
ip link add dev br1 type bridge mcast_snooping 0
- ip link add name vxlan0 up type vxlan id 10 nolearning noudpcsum \
- ttl 20 tos inherit local 198.51.100.1 dstport 4789
- ip link add name vxlan1 up type vxlan id 20 nolearning noudpcsum \
- ttl 40 tos inherit local 198.51.100.1 dstport 4789
+ ip link add name vxlan0 up type vxlan id 10 nolearning $UDPCSUM_FLAFS \
+ ttl 20 tos inherit local $LOCAL_IP_1 dstport 4789
+ ip link add name vxlan1 up type vxlan id 20 nolearning $UDPCSUM_FLAFS \
+ ttl 40 tos inherit local $LOCAL_IP_1 dstport 4789
sanitization_multi_devs_test_fail
@@ -378,10 +385,10 @@
ip link add dev br0 type bridge mcast_snooping 0
ip link add dev br1 type bridge mcast_snooping 0
- ip link add name vxlan0 up type vxlan id 10 nolearning noudpcsum \
- ttl 20 tos inherit local 198.51.100.1 dstport 4789
- ip link add name vxlan1 up type vxlan id 20 nolearning noudpcsum \
- ttl 20 tos inherit local 198.51.100.1 dstport 5789
+ ip link add name vxlan0 up type vxlan id 10 nolearning $UDPCSUM_FLAFS \
+ ttl 20 tos inherit local $LOCAL_IP_1 dstport 4789
+ ip link add name vxlan1 up type vxlan id 20 nolearning $UDPCSUM_FLAFS \
+ ttl 20 tos inherit local $LOCAL_IP_1 dstport 5789
sanitization_multi_devs_test_fail
@@ -400,10 +407,10 @@
ip link add dev br0 type bridge mcast_snooping 0
ip link add dev br1 type bridge mcast_snooping 0
- ip link add name vxlan0 up type vxlan id 10 nolearning noudpcsum \
- ttl 20 tos inherit local 198.51.100.1 dstport 4789
- ip link add name vxlan1 up type vxlan id 20 nolearning noudpcsum \
- ttl 20 tos inherit local 198.51.100.2 dstport 4789
+ ip link add name vxlan0 up type vxlan id 10 nolearning $UDPCSUM_FLAFS \
+ ttl 20 tos inherit local $LOCAL_IP_1 dstport 4789
+ ip link add name vxlan1 up type vxlan id 20 nolearning $UDPCSUM_FLAFS \
+ ttl 20 tos inherit local $LOCAL_IP_2 dstport 4789
sanitization_multi_devs_test_fail
@@ -443,12 +450,12 @@
ip link set dev $swp1 master br0
ip link set dev $swp2 master br1
- ip address add 198.51.100.1/32 dev lo
+ ip address add $LOCAL_IP_1/$PREFIX_LEN dev lo
ip link add name vxlan0 up master br0 type vxlan id 10 nolearning \
- noudpcsum ttl 20 tos inherit local 198.51.100.1 dstport 4789
+ $UDPCSUM_FLAFS ttl 20 tos inherit local $LOCAL_IP_1 dstport 4789
ip link add name vxlan1 up master br1 type vxlan id 20 nolearning \
- noudpcsum ttl 20 tos inherit local 198.51.100.1 dstport 4789
+ $UDPCSUM_FLAFS ttl 20 tos inherit local $LOCAL_IP_1 dstport 4789
}
offload_indication_setup_destroy()
@@ -456,7 +463,7 @@
ip link del dev vxlan1
ip link del dev vxlan0
- ip address del 198.51.100.1/32 dev lo
+ ip address del $LOCAL_IP_1/$PREFIX_LEN dev lo
ip link set dev $swp2 nomaster
ip link set dev $swp1 nomaster
@@ -469,7 +476,7 @@
{
RET=0
- bridge fdb append 00:00:00:00:00:00 dev vxlan0 self dst 198.51.100.2
+ bridge fdb append 00:00:00:00:00:00 dev vxlan0 self dst $LOCAL_IP_2
busywait "$TIMEOUT" wait_for_offload grep_bridge_fdb 00:00:00:00:00:00 \
bridge fdb show brport vxlan0
@@ -485,7 +492,7 @@
RET=0
bridge fdb add de:ad:be:ef:13:37 dev vxlan0 self master static \
- dst 198.51.100.2
+ dst $LOCAL_IP_2
busywait "$TIMEOUT" wait_for_offload grep_bridge_fdb \
de:ad:be:ef:13:37 self bridge fdb show brport vxlan0
@@ -536,7 +543,7 @@
# marked as offloaded in both drivers
RET=0
- bridge fdb add de:ad:be:ef:13:37 dev vxlan0 self dst 198.51.100.2
+ bridge fdb add de:ad:be:ef:13:37 dev vxlan0 self dst $LOCAL_IP_2
busywait "$TIMEOUT" wait_for_offload grep_bridge_fdb \
de:ad:be:ef:13:37 self bridge fdb show brport vxlan0
check_err $?
@@ -560,17 +567,17 @@
RET=0
busywait "$TIMEOUT" wait_for_offload \
- ip route show table local 198.51.100.1
+ ip $IP_FLAG route show table local $LOCAL_IP_1
check_err $?
ip link set dev vxlan0 down
busywait "$TIMEOUT" wait_for_offload \
- ip route show table local 198.51.100.1
+ ip $IP_FLAG route show table local $LOCAL_IP_1
check_err $?
ip link set dev vxlan1 down
busywait "$TIMEOUT" not wait_for_offload \
- ip route show table local 198.51.100.1
+ ip $IP_FLAG route show table local $LOCAL_IP_1
check_err $?
log_test "vxlan decap route - vxlan device down"
@@ -579,26 +586,26 @@
ip link set dev vxlan1 up
busywait "$TIMEOUT" wait_for_offload \
- ip route show table local 198.51.100.1
+ ip $IP_FLAG route show table local $LOCAL_IP_1
check_err $?
ip link set dev vxlan0 up
busywait "$TIMEOUT" wait_for_offload \
- ip route show table local 198.51.100.1
+ ip $IP_FLAG route show table local $LOCAL_IP_1
check_err $?
log_test "vxlan decap route - vxlan device up"
RET=0
- ip address delete 198.51.100.1/32 dev lo
+ ip address delete $LOCAL_IP_1/$PREFIX_LEN dev lo
busywait "$TIMEOUT" not wait_for_offload \
- ip route show table local 198.51.100.1
+ ip $IP_FLAG route show table local $LOCAL_IP_1
check_err $?
- ip address add 198.51.100.1/32 dev lo
+ ip address add $LOCAL_IP_1/$PREFIX_LEN dev lo
busywait "$TIMEOUT" wait_for_offload \
- ip route show table local 198.51.100.1
+ ip $IP_FLAG route show table local $LOCAL_IP_1
check_err $?
log_test "vxlan decap route - add local route"
@@ -607,18 +614,18 @@
ip link set dev $swp1 nomaster
busywait "$TIMEOUT" wait_for_offload \
- ip route show table local 198.51.100.1
+ ip $IP_FLAG route show table local $LOCAL_IP_1
check_err $?
ip link set dev $swp2 nomaster
busywait "$TIMEOUT" not wait_for_offload \
- ip route show table local 198.51.100.1
+ ip $IP_FLAG route show table local $LOCAL_IP_1
check_err $?
ip link set dev $swp1 master br0
ip link set dev $swp2 master br1
busywait "$TIMEOUT" wait_for_offload \
- ip route show table local 198.51.100.1
+ ip $IP_FLAG route show table local $LOCAL_IP_1
check_err $?
log_test "vxlan decap route - local ports enslavement"
@@ -627,12 +634,12 @@
ip link del dev br0
busywait "$TIMEOUT" wait_for_offload \
- ip route show table local 198.51.100.1
+ ip $IP_FLAG route show table local $LOCAL_IP_1
check_err $?
ip link del dev br1
busywait "$TIMEOUT" not wait_for_offload \
- ip route show table local 198.51.100.1
+ ip $IP_FLAG route show table local $LOCAL_IP_1
check_err $?
log_test "vxlan decap route - bridge device deletion"
@@ -646,25 +653,25 @@
ip link set dev vxlan0 master br0
ip link set dev vxlan1 master br1
busywait "$TIMEOUT" wait_for_offload \
- ip route show table local 198.51.100.1
+ ip $IP_FLAG route show table local $LOCAL_IP_1
check_err $?
ip link del dev vxlan0
busywait "$TIMEOUT" wait_for_offload \
- ip route show table local 198.51.100.1
+ ip $IP_FLAG route show table local $LOCAL_IP_1
check_err $?
ip link del dev vxlan1
busywait "$TIMEOUT" not wait_for_offload \
- ip route show table local 198.51.100.1
+ ip $IP_FLAG route show table local $LOCAL_IP_1
check_err $?
log_test "vxlan decap route - vxlan device deletion"
ip link add name vxlan0 up master br0 type vxlan id 10 nolearning \
- noudpcsum ttl 20 tos inherit local 198.51.100.1 dstport 4789
+ $UDPCSUM_FLAFS ttl 20 tos inherit local $LOCAL_IP_1 dstport 4789
ip link add name vxlan1 up master br1 type vxlan id 20 nolearning \
- noudpcsum ttl 20 tos inherit local 198.51.100.1 dstport 4789
+ $UDPCSUM_FLAFS ttl 20 tos inherit local $LOCAL_IP_1 dstport 4789
}
check_fdb_offloaded()
@@ -721,10 +728,10 @@
local mac=00:11:22:33:44:55
local zmac=00:00:00:00:00:00
- bridge fdb append $zmac dev vxlan0 self dst 198.51.100.2
+ bridge fdb append $zmac dev vxlan0 self dst $LOCAL_IP_2
ip link set dev vxlan0 master br0
- bridge fdb add dev vxlan0 $mac self master static dst 198.51.100.2
+ bridge fdb add dev vxlan0 $mac self master static dst $LOCAL_IP_2
RET=0
check_vxlan_fdb_not_offloaded
@@ -774,8 +781,8 @@
offload_indication_join_vxlan_first()
{
ip link add dev br0 up type bridge mcast_snooping 0
- ip link add name vxlan0 up type vxlan id 10 nolearning noudpcsum \
- ttl 20 tos inherit local 198.51.100.1 dstport 4789
+ ip link add name vxlan0 up type vxlan id 10 nolearning $UDPCSUM_FLAFS \
+ ttl 20 tos inherit local $LOCAL_IP_1 dstport 4789
__offload_indication_join_vxlan_first
@@ -789,7 +796,7 @@
RET=0
- bridge fdb append $zmac dev vxlan0 self dst 198.51.100.2
+ bridge fdb append $zmac dev vxlan0 self dst $LOCAL_IP_2
ip link set dev $swp1 master br0
@@ -809,8 +816,8 @@
offload_indication_join_vxlan_last()
{
ip link add dev br0 up type bridge mcast_snooping 0
- ip link add name vxlan0 up type vxlan id 10 nolearning noudpcsum \
- ttl 20 tos inherit local 198.51.100.1 dstport 4789
+ ip link add name vxlan0 up type vxlan id 10 nolearning $UDPCSUM_FLAFS \
+ ttl 20 tos inherit local $LOCAL_IP_1 dstport 4789
__offload_indication_join_vxlan_last
@@ -837,10 +844,10 @@
ip link add dev br0 type bridge mcast_snooping 0 vlan_filtering 1
ip link add name vxlan10 up master br0 type vxlan id 10 nolearning \
- noudpcsum ttl 20 tos inherit local 198.51.100.1 dstport 4789
+ $UDPCSUM_FLAFS ttl 20 tos inherit local $LOCAL_IP_1 dstport 4789
ip link add name vxlan20 up master br0 type vxlan id 20 nolearning \
- noudpcsum ttl 20 tos inherit local 198.51.100.1 dstport 4789
+ $UDPCSUM_FLAFS ttl 20 tos inherit local $LOCAL_IP_1 dstport 4789
# Test that when each VNI is mapped to a different VLAN we can enslave
# a port to the bridge
@@ -884,20 +891,20 @@
# Use the offload indication of the local route to ensure the VXLAN
# configuration was correctly rollbacked.
- ip address add 198.51.100.1/32 dev lo
+ ip address add $LOCAL_IP_1/$PREFIX_LEN dev lo
ip link set dev vxlan10 type vxlan ttl 10
ip link set dev $swp1 master br0 &> /dev/null
check_fail $?
busywait "$TIMEOUT" not wait_for_offload \
- ip route show table local 198.51.100.1
+ ip $IP_FLAG route show table local $LOCAL_IP_1
check_err $?
log_test "vlan-aware - failed enslavement to bridge due to conflict"
ip link set dev vxlan10 type vxlan ttl 20
- ip address del 198.51.100.1/32 dev lo
+ ip address del $LOCAL_IP_1/$PREFIX_LEN dev lo
ip link del dev vxlan20
ip link del dev vxlan10
@@ -916,12 +923,12 @@
bridge vlan add vid 10 dev $swp1
bridge vlan add vid 20 dev $swp1
- ip address add 198.51.100.1/32 dev lo
+ ip address add $LOCAL_IP_1/$PREFIX_LEN dev lo
ip link add name vxlan10 up master br0 type vxlan id 10 nolearning \
- noudpcsum ttl 20 tos inherit local 198.51.100.1 dstport 4789
+ $UDPCSUM_FLAFS ttl 20 tos inherit local $LOCAL_IP_1 dstport 4789
ip link add name vxlan20 up master br0 type vxlan id 20 nolearning \
- noudpcsum ttl 20 tos inherit local 198.51.100.1 dstport 4789
+ $UDPCSUM_FLAFS ttl 20 tos inherit local $LOCAL_IP_1 dstport 4789
bridge vlan add vid 10 dev vxlan10 pvid untagged
bridge vlan add vid 20 dev vxlan20 pvid untagged
@@ -935,7 +942,7 @@
ip link del dev vxlan20
ip link del dev vxlan10
- ip address del 198.51.100.1/32 dev lo
+ ip address del $LOCAL_IP_1/$PREFIX_LEN dev lo
bridge vlan del vid 20 dev $swp1
bridge vlan del vid 10 dev $swp1
@@ -952,7 +959,7 @@
log_info "vxlan entry offload indication - vlan-aware"
bridge fdb add de:ad:be:ef:13:37 dev vxlan10 self master static \
- dst 198.51.100.2 vlan 10
+ dst $LOCAL_IP_2 vlan 10
busywait "$TIMEOUT" wait_for_offload grep_bridge_fdb \
de:ad:be:ef:13:37 self bridge fdb show brport vxlan10
@@ -1003,7 +1010,7 @@
# marked as offloaded in both drivers
RET=0
- bridge fdb add de:ad:be:ef:13:37 dev vxlan10 self dst 198.51.100.2
+ bridge fdb add de:ad:be:ef:13:37 dev vxlan10 self dst $LOCAL_IP_2
busywait "$TIMEOUT" wait_for_offload grep_bridge_fdb \
de:ad:be:ef:13:37 self bridge fdb show brport vxlan10
check_err $?
@@ -1021,7 +1028,7 @@
RET=0
busywait "$TIMEOUT" wait_for_offload \
- ip route show table local 198.51.100.1
+ ip $IP_FLAG route show table local $LOCAL_IP_1
check_err $?
# Toggle PVID flag on one VxLAN device and make sure route is still
@@ -1029,7 +1036,7 @@
bridge vlan add vid 10 dev vxlan10 untagged
busywait "$TIMEOUT" wait_for_offload \
- ip route show table local 198.51.100.1
+ ip $IP_FLAG route show table local $LOCAL_IP_1
check_err $?
# Toggle PVID flag on second VxLAN device and make sure route is no
@@ -1037,14 +1044,15 @@
bridge vlan add vid 20 dev vxlan20 untagged
busywait "$TIMEOUT" not wait_for_offload \
- ip route show table local 198.51.100.1
+ ip $IP_FLAG route show table local $LOCAL_IP_1
check_err $?
# Toggle PVID flag back and make sure route is marked as offloaded
bridge vlan add vid 10 dev vxlan10 pvid untagged
bridge vlan add vid 20 dev vxlan20 pvid untagged
- busywait "$TIMEOUT" wait_for_offload ip route show table local 198.51.100.1
+ busywait "$TIMEOUT" wait_for_offload ip $IP_FLAG route show table local \
+ $LOCAL_IP_1
check_err $?
log_test "vxlan decap route - vni map/unmap"
@@ -1054,8 +1062,8 @@
{
ip link add dev br0 up type bridge mcast_snooping 0 \
vlan_filtering 1 vlan_default_pvid 1
- ip link add name vxlan0 up type vxlan id 10 nolearning noudpcsum \
- ttl 20 tos inherit local 198.51.100.1 dstport 4789
+ ip link add name vxlan0 up type vxlan id 10 nolearning $UDPCSUM_FLAFS \
+ ttl 20 tos inherit local $LOCAL_IP_1 dstport 4789
__offload_indication_join_vxlan_first 1
@@ -1067,8 +1075,8 @@
{
ip link add dev br0 up type bridge mcast_snooping 0 \
vlan_filtering 1 vlan_default_pvid 1
- ip link add name vxlan0 up type vxlan id 10 nolearning noudpcsum \
- ttl 20 tos inherit local 198.51.100.1 dstport 4789
+ ip link add name vxlan0 up type vxlan id 10 nolearning $UDPCSUM_FLAFS \
+ ttl 20 tos inherit local $LOCAL_IP_1 dstport 4789
__offload_indication_join_vxlan_last
@@ -1085,14 +1093,14 @@
sysctl_set net.ipv6.conf.default.disable_ipv6 1
ip link add dev br0 up type bridge mcast_snooping 0 \
vlan_filtering 1 vlan_default_pvid 0
- ip link add name vxlan0 up type vxlan id 10 nolearning noudpcsum \
- ttl 20 tos inherit local 198.51.100.1 dstport 4789
+ ip link add name vxlan0 up type vxlan id 10 nolearning $UDPCSUM_FLAFS \
+ ttl 20 tos inherit local $LOCAL_IP_1 dstport 4789
ip link set dev $swp1 master br0
# The test will use the offload indication on the FDB entry to
# understand if the tunnel is offloaded or not
- bridge fdb append $zmac dev vxlan0 self dst 192.0.2.1
+ bridge fdb append $zmac dev vxlan0 self dst $LOCAL_IP_2
ip link set dev vxlan0 master br0
bridge vlan add dev vxlan0 vid 10 pvid untagged
diff --git a/tools/testing/selftests/drivers/net/mlxsw/vxlan_fdb_veto.sh b/tools/testing/selftests/drivers/net/mlxsw/vxlan_fdb_veto.sh
index 749ba3c..38148f5 100755
--- a/tools/testing/selftests/drivers/net/mlxsw/vxlan_fdb_veto.sh
+++ b/tools/testing/selftests/drivers/net/mlxsw/vxlan_fdb_veto.sh
@@ -4,6 +4,21 @@
# Test vetoing of FDB entries that mlxsw can not offload. This exercises several
# different veto vectors to test various rollback scenarios in the vxlan driver.
+: ${LOCAL_IP:=198.51.100.1}
+export LOCAL_IP
+
+: ${REMOTE_IP_1:=198.51.100.2}
+export REMOTE_IP_1
+
+: ${REMOTE_IP_2:=198.51.100.3}
+export REMOTE_IP_2
+
+: ${UDPCSUM_FLAFS:=noudpcsum}
+export UDPCSUM_FLAFS
+
+: ${MC_IP:=224.0.0.1}
+export MC_IP
+
lib_dir=$(dirname $0)/../../../net/forwarding
ALL_TESTS="
@@ -26,8 +41,8 @@
ip link set dev $swp1 master br0
ip link set dev $swp2 up
- ip link add name vxlan0 up type vxlan id 10 nolearning noudpcsum \
- ttl 20 tos inherit local 198.51.100.1 dstport 4789
+ ip link add name vxlan0 up type vxlan id 10 nolearning $UDPCSUM_FLAFS \
+ ttl 20 tos inherit local $LOCAL_IP dstport 4789
ip link set dev vxlan0 master br0
}
@@ -50,11 +65,11 @@
RET=0
bridge fdb add 01:02:03:04:05:06 dev vxlan0 self static \
- dst 198.51.100.2 2>/dev/null
+ dst $REMOTE_IP_1 2>/dev/null
check_fail $? "multicast MAC not rejected"
bridge fdb add 01:02:03:04:05:06 dev vxlan0 self static \
- dst 198.51.100.2 2>&1 >/dev/null | grep -q mlxsw_spectrum
+ dst $REMOTE_IP_1 2>&1 >/dev/null | grep -q mlxsw_spectrum
check_err $? "multicast MAC rejected without extack"
log_test "vxlan FDB veto - create"
@@ -65,15 +80,15 @@
RET=0
bridge fdb add 00:01:02:03:04:05 dev vxlan0 self static \
- dst 198.51.100.2
+ dst $REMOTE_IP_1
check_err $? "valid FDB rejected"
bridge fdb replace 00:01:02:03:04:05 dev vxlan0 self static \
- dst 198.51.100.2 port 1234 2>/dev/null
+ dst $REMOTE_IP_1 port 1234 2>/dev/null
check_fail $? "FDB with an explicit port not rejected"
bridge fdb replace 00:01:02:03:04:05 dev vxlan0 self static \
- dst 198.51.100.2 port 1234 2>&1 >/dev/null \
+ dst $REMOTE_IP_1 port 1234 2>&1 >/dev/null \
| grep -q mlxsw_spectrum
check_err $? "FDB with an explicit port rejected without extack"
@@ -85,15 +100,15 @@
RET=0
bridge fdb add 00:00:00:00:00:00 dev vxlan0 self static \
- dst 198.51.100.2
+ dst $REMOTE_IP_1
check_err $? "valid FDB rejected"
bridge fdb append 00:00:00:00:00:00 dev vxlan0 self static \
- dst 198.51.100.3 port 1234 2>/dev/null
+ dst $REMOTE_IP_2 port 1234 2>/dev/null
check_fail $? "FDB with an explicit port not rejected"
bridge fdb append 00:00:00:00:00:00 dev vxlan0 self static \
- dst 198.51.100.3 port 1234 2>&1 >/dev/null \
+ dst $REMOTE_IP_2 port 1234 2>&1 >/dev/null \
| grep -q mlxsw_spectrum
check_err $? "FDB with an explicit port rejected without extack"
@@ -105,11 +120,11 @@
RET=0
ip link set dev vxlan0 type vxlan \
- group 224.0.0.1 dev lo 2>/dev/null
+ group $MC_IP dev lo 2>/dev/null
check_fail $? "FDB with a multicast IP not rejected"
ip link set dev vxlan0 type vxlan \
- group 224.0.0.1 dev lo 2>&1 >/dev/null \
+ group $MC_IP dev lo 2>&1 >/dev/null \
| grep -q mlxsw_spectrum
check_err $? "FDB with a multicast IP rejected without extack"
diff --git a/tools/testing/selftests/drivers/net/mlxsw/vxlan_fdb_veto_ipv6.sh b/tools/testing/selftests/drivers/net/mlxsw/vxlan_fdb_veto_ipv6.sh
new file mode 100755
index 0000000..66c87aa
--- /dev/null
+++ b/tools/testing/selftests/drivers/net/mlxsw/vxlan_fdb_veto_ipv6.sh
@@ -0,0 +1,12 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+# A wrapper to run VXLAN test for IPv6.
+
+LOCAL_IP=2001:db8:1::1
+REMOTE_IP_1=2001:db8:2::1
+REMOTE_IP_2=2001:db8:3::1
+UDPCSUM_FLAFS="udp6zerocsumrx udp6zerocsumtx"
+MC_IP=FF02::2
+
+source vxlan_fdb_veto.sh
diff --git a/tools/testing/selftests/drivers/net/mlxsw/vxlan_ipv6.sh b/tools/testing/selftests/drivers/net/mlxsw/vxlan_ipv6.sh
new file mode 100755
index 0000000..f2ea016
--- /dev/null
+++ b/tools/testing/selftests/drivers/net/mlxsw/vxlan_ipv6.sh
@@ -0,0 +1,65 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+# A wrapper to run VXLAN test for IPv6.
+
+ADDR_FAMILY=ipv6
+LOCAL_IP_1=2001:db8:1::1
+LOCAL_IP_2=2001:db8:1::2
+PREFIX_LEN=128
+UDPCSUM_FLAFS="udp6zerocsumrx udp6zerocsumtx"
+MC_IP=FF02::2
+IP_FLAG="-6"
+
+ALL_TESTS="
+ sanitization_test
+ offload_indication_test
+ sanitization_vlan_aware_test
+ offload_indication_vlan_aware_test
+"
+
+sanitization_single_dev_learning_enabled_ipv6_test()
+{
+ RET=0
+
+ ip link add dev br0 type bridge mcast_snooping 0
+
+ ip link add name vxlan0 up type vxlan id 10 learning $UDPCSUM_FLAFS \
+ ttl 20 tos inherit local $LOCAL_IP_1 dstport 4789
+
+ sanitization_single_dev_test_fail
+
+ ip link del dev vxlan0
+ ip link del dev br0
+
+ log_test "vxlan device with learning enabled"
+}
+
+sanitization_single_dev_udp_checksum_ipv6_test()
+{
+ RET=0
+
+ ip link add dev br0 type bridge mcast_snooping 0
+
+ ip link add name vxlan0 up type vxlan id 10 nolearning \
+ noudp6zerocsumrx udp6zerocsumtx ttl 20 tos inherit \
+ local $LOCAL_IP_1 dstport 4789
+
+ sanitization_single_dev_test_fail
+ log_test "vxlan device without zero udp checksum at RX"
+
+ ip link del dev vxlan0
+
+ ip link add name vxlan0 up type vxlan id 10 nolearning \
+ udp6zerocsumrx noudp6zerocsumtx ttl 20 tos inherit \
+ local $LOCAL_IP_1 dstport 4789
+
+ sanitization_single_dev_test_fail
+ log_test "vxlan device without zero udp checksum at TX"
+
+ ip link del dev vxlan0
+ ip link del dev br0
+
+}
+
+source vxlan.sh
diff --git a/tools/testing/selftests/ftrace/test.d/kprobe/profile.tc b/tools/testing/selftests/ftrace/test.d/kprobe/profile.tc
index 98166fa..34fb89b 100644
--- a/tools/testing/selftests/ftrace/test.d/kprobe/profile.tc
+++ b/tools/testing/selftests/ftrace/test.d/kprobe/profile.tc
@@ -1,6 +1,6 @@
#!/bin/sh
# SPDX-License-Identifier: GPL-2.0
-# description: Kprobe dynamic event - adding and removing
+# description: Kprobe profile
# requires: kprobe_events
! grep -q 'myevent' kprobe_profile
diff --git a/tools/testing/selftests/ir/ir_loopback.c b/tools/testing/selftests/ir/ir_loopback.c
index af7f9c7..06256c9 100644
--- a/tools/testing/selftests/ir/ir_loopback.c
+++ b/tools/testing/selftests/ir/ir_loopback.c
@@ -26,7 +26,6 @@
#include "../kselftest.h"
#define TEST_SCANCODES 10
-#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]))
#define SYSFS_PATH_MAX 256
#define DNAME_PATH_MAX 256
diff --git a/tools/testing/selftests/kselftest.h b/tools/testing/selftests/kselftest.h
index 8d50483..f118098 100644
--- a/tools/testing/selftests/kselftest.h
+++ b/tools/testing/selftests/kselftest.h
@@ -48,6 +48,10 @@
#include <stdarg.h>
#include <stdio.h>
+#ifndef ARRAY_SIZE
+#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]))
+#endif
+
/* define kselftest exit codes */
#define KSFT_PASS 0
#define KSFT_FAIL 1
diff --git a/tools/testing/selftests/kselftest_harness.h b/tools/testing/selftests/kselftest_harness.h
index ae0f0f3..471eaa7 100644
--- a/tools/testing/selftests/kselftest_harness.h
+++ b/tools/testing/selftests/kselftest_harness.h
@@ -671,7 +671,9 @@
#define EXPECT_STRNE(expected, seen) \
__EXPECT_STR(expected, seen, !=, 0)
+#ifndef ARRAY_SIZE
#define ARRAY_SIZE(a) (sizeof(a) / sizeof(a[0]))
+#endif
/* Support an optional handler after and ASSERT_* or EXPECT_*. The approach is
* not thread-safe, but it should be fine in most sane test scenarios.
@@ -969,7 +971,7 @@ void __run_test(struct __fixture_metadata *f,
t->passed = 1;
t->skip = 0;
t->trigger = 0;
- t->step = 0;
+ t->step = 1;
t->no_print = 0;
memset(t->results->reason, 0, sizeof(t->results->reason));
diff --git a/tools/testing/selftests/landlock/common.h b/tools/testing/selftests/landlock/common.h
index 20e2a92..183b7e8 100644
--- a/tools/testing/selftests/landlock/common.h
+++ b/tools/testing/selftests/landlock/common.h
@@ -17,10 +17,6 @@
#include "../kselftest_harness.h"
-#ifndef ARRAY_SIZE
-#define ARRAY_SIZE(x) (sizeof(x) / sizeof((x)[0]))
-#endif
-
/*
* TEST_F_FORK() is useful when a test drop privileges but the corresponding
* FIXTURE_TEARDOWN() requires them (e.g. to remove files from a directory
diff --git a/tools/testing/selftests/lib.mk b/tools/testing/selftests/lib.mk
index fe7ee2b..a40add3 100644
--- a/tools/testing/selftests/lib.mk
+++ b/tools/testing/selftests/lib.mk
@@ -141,7 +141,7 @@
# Selftest makefiles can override those targets by setting
# OVERRIDE_TARGETS = 1.
ifeq ($(OVERRIDE_TARGETS),)
-LOCAL_HDRS := $(selfdir)/kselftest_harness.h $(selfdir)/kselftest.h
+LOCAL_HDRS += $(selfdir)/kselftest_harness.h $(selfdir)/kselftest.h
$(OUTPUT)/%:%.c $(LOCAL_HDRS)
$(LINK.c) $(filter-out $(LOCAL_HDRS),$^) $(LDLIBS) -o $@
diff --git a/tools/testing/selftests/mount/unprivileged-remount-test.c b/tools/testing/selftests/mount/unprivileged-remount-test.c
index 584dc6b..d291705 100644
--- a/tools/testing/selftests/mount/unprivileged-remount-test.c
+++ b/tools/testing/selftests/mount/unprivileged-remount-test.c
@@ -204,7 +204,7 @@ bool test_unpriv_remount(const char *fstype, const char *mount_options,
if (!WIFEXITED(status)) {
die("child did not terminate cleanly\n");
}
- return WEXITSTATUS(status) == EXIT_SUCCESS ? true : false;
+ return WEXITSTATUS(status) == EXIT_SUCCESS;
}
create_and_enter_userns();
@@ -282,7 +282,7 @@ static bool test_priv_mount_unpriv_remount(void)
if (!WIFEXITED(status)) {
die("child did not terminate cleanly\n");
}
- return WEXITSTATUS(status) == EXIT_SUCCESS ? true : false;
+ return WEXITSTATUS(status) == EXIT_SUCCESS;
}
orig_mnt_flags = read_mnt_flags(orig_path);
diff --git a/tools/testing/selftests/move_mount_set_group/move_mount_set_group_test.c b/tools/testing/selftests/move_mount_set_group/move_mount_set_group_test.c
index 860198f..50ed5d4 100644
--- a/tools/testing/selftests/move_mount_set_group/move_mount_set_group_test.c
+++ b/tools/testing/selftests/move_mount_set_group/move_mount_set_group_test.c
@@ -191,7 +191,7 @@ static bool is_shared_mount(const char *path)
#define SET_GROUP_FROM "/tmp/move_mount_set_group_supported_from"
#define SET_GROUP_TO "/tmp/move_mount_set_group_supported_to"
-static int move_mount_set_group_supported(void)
+static bool move_mount_set_group_supported(void)
{
int ret;
@@ -222,7 +222,7 @@ static int move_mount_set_group_supported(void)
AT_FDCWD, SET_GROUP_TO, MOVE_MOUNT_SET_GROUP);
umount2("/tmp", MNT_DETACH);
- return ret < 0 ? false : true;
+ return ret >= 0;
}
FIXTURE(move_mount_set_group) {
@@ -232,7 +232,7 @@ FIXTURE(move_mount_set_group) {
FIXTURE_SETUP(move_mount_set_group)
{
- int ret;
+ bool ret;
ASSERT_EQ(prepare_unpriv_mountns(), 0);
@@ -254,7 +254,7 @@ FIXTURE_SETUP(move_mount_set_group)
FIXTURE_TEARDOWN(move_mount_set_group)
{
- int ret;
+ bool ret;
ret = move_mount_set_group_supported();
ASSERT_GE(ret, 0);
@@ -348,7 +348,7 @@ TEST_F(move_mount_set_group, complex_sharing_copying)
.shared = false,
};
pid_t pid;
- int ret;
+ bool ret;
ret = move_mount_set_group_supported();
ASSERT_GE(ret, 0);
diff --git a/tools/testing/selftests/net/amt.sh b/tools/testing/selftests/net/amt.sh
old mode 100644
new mode 100755
diff --git a/tools/testing/selftests/net/fcnal-test.sh b/tools/testing/selftests/net/fcnal-test.sh
index ad2982b..412d852 100755
--- a/tools/testing/selftests/net/fcnal-test.sh
+++ b/tools/testing/selftests/net/fcnal-test.sh
@@ -66,6 +66,10 @@
NSA_LO_IP6=2001:db8:2::1
NSB_LO_IP6=2001:db8:2::2
+# non-local addresses for freebind tests
+NL_IP=172.17.1.1
+NL_IP6=2001:db8:4::1
+
MD5_PW=abc123
MD5_WRONG_PW=abc1234
@@ -316,6 +320,9 @@
${NSB_LO_IP6}) echo "ns-B loopback IPv6";;
${NSB_LINKIP6}|${NSB_LINKIP6}%*) echo "ns-B IPv6 LLA";;
+ ${NL_IP}) echo "nonlocal IP";;
+ ${NL_IP6}) echo "nonlocal IPv6";;
+
${VRF_IP}) echo "VRF IP";;
${VRF_IP6}) echo "VRF IPv6";;
@@ -1780,6 +1787,14 @@
done
#
+ # raw socket with nonlocal bind
+ #
+ a=${NL_IP}
+ log_start
+ run_cmd nettest -s -R -P icmp -f -l ${a} -I ${NSA_DEV} -b
+ log_test_addr ${a} $? 0 "Raw socket bind to nonlocal address after device bind"
+
+ #
# tcp sockets
#
a=${NSA_IP}
@@ -1829,6 +1844,14 @@
log_test_addr ${a} $? 1 "Raw socket bind to out of scope address after VRF bind"
#
+ # raw socket with nonlocal bind
+ #
+ a=${NL_IP}
+ log_start
+ run_cmd nettest -s -R -P icmp -f -l ${a} -I ${VRF} -b
+ log_test_addr ${a} $? 0 "Raw socket bind to nonlocal address after VRF bind"
+
+ #
# tcp sockets
#
for a in ${NSA_IP} ${VRF_IP}
@@ -1978,6 +2001,7 @@
a=${NSA_IP}
log_start
+
run_cmd nettest ${varg} -s &
sleep 1
run_cmd nettest ${varg} -d ${NSA_DEV} -r ${a} &
@@ -3418,6 +3442,14 @@
done
#
+ # raw socket with nonlocal bind
+ #
+ a=${NL_IP6}
+ log_start
+ run_cmd nettest -6 -s -R -P icmp -f -l ${a} -I ${NSA_DEV} -b
+ log_test_addr ${a} $? 0 "Raw socket bind to nonlocal address"
+
+ #
# tcp sockets
#
a=${NSA_IP6}
@@ -3462,6 +3494,14 @@
log_test_addr ${a} $? 1 "Raw socket bind to invalid local address after vrf bind"
#
+ # raw socket with nonlocal bind
+ #
+ a=${NL_IP6}
+ log_start
+ run_cmd nettest -6 -s -R -P icmp -f -l ${a} -I ${VRF} -b
+ log_test_addr ${a} $? 0 "Raw socket bind to nonlocal address after VRF bind"
+
+ #
# tcp sockets
#
# address on enslaved device is valid for the VRF or device in a VRF
@@ -4091,8 +4131,6 @@
# setup namespaces and config, but do not run any tests
setup) setup; exit 0;;
vrf_setup) setup "yes"; exit 0;;
-
- help) echo "Test names: $TESTS"; exit 0;;
esac
done
diff --git a/tools/testing/selftests/net/forwarding/bridge_vlan_mcast.sh b/tools/testing/selftests/net/forwarding/bridge_vlan_mcast.sh
new file mode 100755
index 0000000..8748d1b
--- /dev/null
+++ b/tools/testing/selftests/net/forwarding/bridge_vlan_mcast.sh
@@ -0,0 +1,543 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+ALL_TESTS="vlmc_control_test vlmc_querier_test vlmc_igmp_mld_version_test \
+ vlmc_last_member_test vlmc_startup_query_test vlmc_membership_test \
+ vlmc_querier_intvl_test vlmc_query_intvl_test vlmc_query_response_intvl_test \
+ vlmc_router_port_test vlmc_filtering_test"
+NUM_NETIFS=4
+CHECK_TC="yes"
+TEST_GROUP="239.10.10.10"
+
+source lib.sh
+
+h1_create()
+{
+ simple_if_init $h1 192.0.2.1/24 2001:db8:1::1/64
+ ip link add l $h1 $h1.10 up type vlan id 10
+}
+
+h1_destroy()
+{
+ ip link del $h1.10
+ simple_if_fini $h1 192.0.2.1/24 2001:db8:1::1/64
+}
+
+h2_create()
+{
+ simple_if_init $h2 192.0.2.2/24 2001:db8:1::2/64
+ ip link add l $h2 $h2.10 up type vlan id 10
+}
+
+h2_destroy()
+{
+ ip link del $h2.10
+ simple_if_fini $h2 192.0.2.2/24 2001:db8:1::2/64
+}
+
+switch_create()
+{
+ ip link add dev br0 type bridge mcast_snooping 1 mcast_querier 1 vlan_filtering 1
+
+ ip link set dev $swp1 master br0
+ ip link set dev $swp2 master br0
+
+ ip link set dev br0 up
+ ip link set dev $swp1 up
+ ip link set dev $swp2 up
+
+ tc qdisc add dev $swp1 clsact
+ tc qdisc add dev $swp2 clsact
+
+ bridge vlan add vid 10-11 dev $swp1 master
+ bridge vlan add vid 10-11 dev $swp2 master
+
+ ip link set dev br0 type bridge mcast_vlan_snooping 1
+ check_err $? "Could not enable global vlan multicast snooping"
+ log_test "Vlan multicast snooping enable"
+}
+
+switch_destroy()
+{
+ ip link set dev $swp2 down
+ ip link set dev $swp1 down
+
+ ip link del dev br0
+}
+
+setup_prepare()
+{
+ h1=${NETIFS[p1]}
+ swp1=${NETIFS[p2]}
+
+ swp2=${NETIFS[p3]}
+ h2=${NETIFS[p4]}
+
+ vrf_prepare
+
+ h1_create
+ h2_create
+
+ switch_create
+}
+
+cleanup()
+{
+ pre_cleanup
+
+ switch_destroy
+
+ h2_destroy
+ h1_destroy
+
+ vrf_cleanup
+}
+
+vlmc_v2join_test()
+{
+ local expect=$1
+
+ RET=0
+ ip address add dev $h2.10 $TEST_GROUP/32 autojoin
+ check_err $? "Could not join $TEST_GROUP"
+
+ sleep 5
+ bridge -j mdb show dev br0 |
+ jq -e ".[].mdb[] | select(.grp == \"$TEST_GROUP\" and .vid == 10)" &>/dev/null
+ if [ $expect -eq 0 ]; then
+ check_err $? "IGMPv2 report didn't create mdb entry for $TEST_GROUP"
+ else
+ check_fail $? "IGMPv2 report shouldn't have created mdb entry for $TEST_GROUP"
+ fi
+
+ # check if we need to cleanup
+ if [ $RET -eq 0 ]; then
+ ip address del dev $h2.10 $TEST_GROUP/32 2>&1 1>/dev/null
+ sleep 5
+ bridge -j mdb show dev br0 |
+ jq -e ".[].mdb[] | select(.grp == \"$TEST_GROUP\" and \
+ .vid == 10)" &>/dev/null
+ check_fail $? "IGMPv2 leave didn't remove mdb entry for $TEST_GROUP"
+ fi
+}
+
+vlmc_control_test()
+{
+ RET=0
+ local goutput=`bridge -j vlan global show`
+ echo -n $goutput |
+ jq -e ".[].vlans[] | select(.vlan == 10)" &>/dev/null
+ check_err $? "Could not find vlan 10's global options"
+ log_test "Vlan global options existence"
+
+ RET=0
+ echo -n $goutput |
+ jq -e ".[].vlans[] | select(.vlan == 10 and .mcast_snooping == 1) " &>/dev/null
+ check_err $? "Wrong default mcast_snooping global option value"
+ log_test "Vlan mcast_snooping global option default value"
+
+ RET=0
+ vlmc_v2join_test 0
+ bridge vlan global set vid 10 dev br0 mcast_snooping 0
+ check_err $? "Could not disable multicast snooping in vlan 10"
+ vlmc_v2join_test 1
+ log_test "Vlan 10 multicast snooping control"
+}
+
+# setup for general query counting
+vlmc_query_cnt_xstats()
+{
+ local type=$1
+ local version=$2
+ local dev=$3
+
+ ip -j link xstats type bridge_slave dev $dev | \
+ jq -e ".[].multicast.${type}_queries.tx_v${version}"
+}
+
+vlmc_query_cnt_setup()
+{
+ local type=$1
+ local dev=$2
+
+ if [[ $type == "igmp" ]]; then
+ tc filter add dev $dev egress pref 10 prot 802.1Q \
+ flower vlan_id 10 vlan_ethtype ipv4 dst_ip 224.0.0.1 ip_proto 2 \
+ action pass
+ else
+ tc filter add dev $dev egress pref 10 prot 802.1Q \
+ flower vlan_id 10 vlan_ethtype ipv6 dst_ip ff02::1 ip_proto icmpv6 \
+ action pass
+ fi
+
+ ip link set dev br0 type bridge mcast_stats_enabled 1
+}
+
+vlmc_query_cnt_cleanup()
+{
+ local dev=$1
+
+ ip link set dev br0 type bridge mcast_stats_enabled 0
+ tc filter del dev $dev egress pref 10
+}
+
+vlmc_check_query()
+{
+ local type=$1
+ local version=$2
+ local dev=$3
+ local expect=$4
+ local time=$5
+ local ret=0
+
+ vlmc_query_cnt_setup $type $dev
+
+ local pre_tx_xstats=$(vlmc_query_cnt_xstats $type $version $dev)
+ bridge vlan global set vid 10 dev br0 mcast_snooping 1 mcast_querier 1
+ ret=$?
+ if [[ $ret -eq 0 ]]; then
+ sleep $time
+
+ local tcstats=$(tc_rule_stats_get $dev 10 egress)
+ local post_tx_xstats=$(vlmc_query_cnt_xstats $type $version $dev)
+
+ if [[ $tcstats != $expect || \
+ $(($post_tx_xstats-$pre_tx_xstats)) != $expect || \
+ $tcstats != $(($post_tx_xstats-$pre_tx_xstats)) ]]; then
+ ret=1
+ fi
+ fi
+
+ bridge vlan global set vid 10 dev br0 mcast_snooping 1 mcast_querier 0
+ vlmc_query_cnt_cleanup $dev
+
+ return $ret
+}
+
+vlmc_querier_test()
+{
+ RET=0
+ local goutput=`bridge -j vlan global show`
+ echo -n $goutput |
+ jq -e ".[].vlans[] | select(.vlan == 10)" &>/dev/null
+ check_err $? "Could not find vlan 10's global options"
+
+ echo -n $goutput |
+ jq -e ".[].vlans[] | select(.vlan == 10 and .mcast_querier == 0) " &>/dev/null
+ check_err $? "Wrong default mcast_querier global vlan option value"
+ log_test "Vlan mcast_querier global option default value"
+
+ RET=0
+ bridge vlan global set vid 10 dev br0 mcast_snooping 1 mcast_querier 1
+ check_err $? "Could not enable querier in vlan 10"
+ log_test "Vlan 10 multicast querier enable"
+ bridge vlan global set vid 10 dev br0 mcast_snooping 1 mcast_querier 0
+
+ RET=0
+ vlmc_check_query igmp 2 $swp1 1 1
+ check_err $? "No vlan tagged IGMPv2 general query packets sent"
+ log_test "Vlan 10 tagged IGMPv2 general query sent"
+
+ RET=0
+ vlmc_check_query mld 1 $swp1 1 1
+ check_err $? "No vlan tagged MLD general query packets sent"
+ log_test "Vlan 10 tagged MLD general query sent"
+}
+
+vlmc_igmp_mld_version_test()
+{
+ RET=0
+ local goutput=`bridge -j vlan global show`
+ echo -n $goutput |
+ jq -e ".[].vlans[] | select(.vlan == 10)" &>/dev/null
+ check_err $? "Could not find vlan 10's global options"
+
+ echo -n $goutput |
+ jq -e ".[].vlans[] | select(.vlan == 10 and .mcast_igmp_version == 2) " &>/dev/null
+ check_err $? "Wrong default mcast_igmp_version global vlan option value"
+ log_test "Vlan mcast_igmp_version global option default value"
+
+ RET=0
+ echo -n $goutput |
+ jq -e ".[].vlans[] | select(.vlan == 10 and .mcast_mld_version == 1) " &>/dev/null
+ check_err $? "Wrong default mcast_mld_version global vlan option value"
+ log_test "Vlan mcast_mld_version global option default value"
+
+ RET=0
+ bridge vlan global set vid 10 dev br0 mcast_snooping 1 mcast_igmp_version 3
+ check_err $? "Could not set mcast_igmp_version in vlan 10"
+ log_test "Vlan 10 mcast_igmp_version option changed to 3"
+
+ RET=0
+ vlmc_check_query igmp 3 $swp1 1 1
+ check_err $? "No vlan tagged IGMPv3 general query packets sent"
+ log_test "Vlan 10 tagged IGMPv3 general query sent"
+
+ RET=0
+ bridge vlan global set vid 10 dev br0 mcast_snooping 1 mcast_mld_version 2
+ check_err $? "Could not set mcast_mld_version in vlan 10"
+ log_test "Vlan 10 mcast_mld_version option changed to 2"
+
+ RET=0
+ vlmc_check_query mld 2 $swp1 1 1
+ check_err $? "No vlan tagged MLDv2 general query packets sent"
+ log_test "Vlan 10 tagged MLDv2 general query sent"
+
+ bridge vlan global set vid 10 dev br0 mcast_snooping 1 mcast_igmp_version 2
+ bridge vlan global set vid 10 dev br0 mcast_snooping 1 mcast_mld_version 1
+}
+
+vlmc_last_member_test()
+{
+ RET=0
+ local goutput=`bridge -j vlan global show`
+ echo -n $goutput |
+ jq -e ".[].vlans[] | select(.vlan == 10)" &>/dev/null
+ check_err $? "Could not find vlan 10's global options"
+
+ echo -n $goutput |
+ jq -e ".[].vlans[] | select(.vlan == 10 and \
+ .mcast_last_member_count == 2) " &>/dev/null
+ check_err $? "Wrong default mcast_last_member_count global vlan option value"
+ log_test "Vlan mcast_last_member_count global option default value"
+
+ RET=0
+ echo -n $goutput |
+ jq -e ".[].vlans[] | select(.vlan == 10 and \
+ .mcast_last_member_interval == 100) " &>/dev/null
+ check_err $? "Wrong default mcast_last_member_interval global vlan option value"
+ log_test "Vlan mcast_last_member_interval global option default value"
+
+ RET=0
+ bridge vlan global set vid 10 dev br0 mcast_snooping 1 mcast_last_member_count 3
+ check_err $? "Could not set mcast_last_member_count in vlan 10"
+ log_test "Vlan 10 mcast_last_member_count option changed to 3"
+ bridge vlan global set vid 10 dev br0 mcast_snooping 1 mcast_last_member_count 2
+
+ RET=0
+ bridge vlan global set vid 10 dev br0 mcast_snooping 1 mcast_last_member_interval 200
+ check_err $? "Could not set mcast_last_member_interval in vlan 10"
+ log_test "Vlan 10 mcast_last_member_interval option changed to 200"
+ bridge vlan global set vid 10 dev br0 mcast_snooping 1 mcast_last_member_interval 100
+}
+
+vlmc_startup_query_test()
+{
+ RET=0
+ local goutput=`bridge -j vlan global show`
+ echo -n $goutput |
+ jq -e ".[].vlans[] | select(.vlan == 10)" &>/dev/null
+ check_err $? "Could not find vlan 10's global options"
+
+ echo -n $goutput |
+ jq -e ".[].vlans[] | select(.vlan == 10 and \
+ .mcast_startup_query_interval == 3125) " &>/dev/null
+ check_err $? "Wrong default mcast_startup_query_interval global vlan option value"
+ log_test "Vlan mcast_startup_query_interval global option default value"
+
+ RET=0
+ echo -n $goutput |
+ jq -e ".[].vlans[] | select(.vlan == 10 and \
+ .mcast_startup_query_count == 2) " &>/dev/null
+ check_err $? "Wrong default mcast_startup_query_count global vlan option value"
+ log_test "Vlan mcast_startup_query_count global option default value"
+
+ RET=0
+ bridge vlan global set vid 10 dev br0 mcast_snooping 1 mcast_startup_query_interval 100
+ check_err $? "Could not set mcast_startup_query_interval in vlan 10"
+ vlmc_check_query igmp 2 $swp1 2 3
+ check_err $? "Wrong number of tagged IGMPv2 general queries sent"
+ log_test "Vlan 10 mcast_startup_query_interval option changed to 100"
+
+ RET=0
+ bridge vlan global set vid 10 dev br0 mcast_snooping 1 mcast_startup_query_count 3
+ check_err $? "Could not set mcast_startup_query_count in vlan 10"
+ vlmc_check_query igmp 2 $swp1 3 4
+ check_err $? "Wrong number of tagged IGMPv2 general queries sent"
+ log_test "Vlan 10 mcast_startup_query_count option changed to 3"
+
+ bridge vlan global set vid 10 dev br0 mcast_snooping 1 mcast_startup_query_interval 3125
+ bridge vlan global set vid 10 dev br0 mcast_snooping 1 mcast_startup_query_count 2
+}
+
+vlmc_membership_test()
+{
+ RET=0
+ local goutput=`bridge -j vlan global show`
+ echo -n $goutput |
+ jq -e ".[].vlans[] | select(.vlan == 10)" &>/dev/null
+ check_err $? "Could not find vlan 10's global options"
+
+ echo -n $goutput |
+ jq -e ".[].vlans[] | select(.vlan == 10 and \
+ .mcast_membership_interval == 26000) " &>/dev/null
+ check_err $? "Wrong default mcast_membership_interval global vlan option value"
+ log_test "Vlan mcast_membership_interval global option default value"
+
+ RET=0
+ bridge vlan global set vid 10 dev br0 mcast_snooping 1 mcast_membership_interval 200
+ check_err $? "Could not set mcast_membership_interval in vlan 10"
+ log_test "Vlan 10 mcast_membership_interval option changed to 200"
+
+ RET=0
+ vlmc_v2join_test 1
+ log_test "Vlan 10 mcast_membership_interval mdb entry expire"
+
+ bridge vlan global set vid 10 dev br0 mcast_snooping 1 mcast_membership_interval 26000
+}
+
+vlmc_querier_intvl_test()
+{
+ RET=0
+ local goutput=`bridge -j vlan global show`
+ echo -n $goutput |
+ jq -e ".[].vlans[] | select(.vlan == 10)" &>/dev/null
+ check_err $? "Could not find vlan 10's global options"
+
+ echo -n $goutput |
+ jq -e ".[].vlans[] | select(.vlan == 10 and \
+ .mcast_querier_interval == 25500) " &>/dev/null
+ check_err $? "Wrong default mcast_querier_interval global vlan option value"
+ log_test "Vlan mcast_querier_interval global option default value"
+
+ RET=0
+ bridge vlan global set vid 10 dev br0 mcast_snooping 1 mcast_querier_interval 100
+ check_err $? "Could not set mcast_querier_interval in vlan 10"
+ log_test "Vlan 10 mcast_querier_interval option changed to 100"
+
+ RET=0
+ ip link add dev br1 type bridge mcast_snooping 1 mcast_querier 1 vlan_filtering 1 \
+ mcast_vlan_snooping 1
+ bridge vlan add vid 10 dev br1 self pvid untagged
+ ip link set dev $h1 master br1
+ ip link set dev br1 up
+ bridge vlan add vid 10 dev $h1 master
+ bridge vlan global set vid 10 dev br1 mcast_snooping 1 mcast_querier 1
+ sleep 2
+ ip link del dev br1
+ ip addr replace 2001:db8:1::1/64 dev $h1
+ vlmc_check_query igmp 2 $swp1 1 1
+ check_err $? "Wrong number of IGMPv2 general queries after querier interval"
+ log_test "Vlan 10 mcast_querier_interval expire after outside query"
+
+ bridge vlan global set vid 10 dev br0 mcast_snooping 1 mcast_querier_interval 25500
+}
+
+vlmc_query_intvl_test()
+{
+ RET=0
+ local goutput=`bridge -j vlan global show`
+ echo -n $goutput |
+ jq -e ".[].vlans[] | select(.vlan == 10)" &>/dev/null
+ check_err $? "Could not find vlan 10's global options"
+
+ echo -n $goutput |
+ jq -e ".[].vlans[] | select(.vlan == 10 and \
+ .mcast_query_interval == 12500) " &>/dev/null
+ check_err $? "Wrong default mcast_query_interval global vlan option value"
+ log_test "Vlan mcast_query_interval global option default value"
+
+ RET=0
+ bridge vlan global set vid 10 dev br0 mcast_snooping 1 mcast_startup_query_count 0
+ bridge vlan global set vid 10 dev br0 mcast_snooping 1 mcast_query_interval 200
+ check_err $? "Could not set mcast_query_interval in vlan 10"
+ # 1 is sent immediately, then 2 more in the next 5 seconds
+ vlmc_check_query igmp 2 $swp1 3 5
+ check_err $? "Wrong number of tagged IGMPv2 general queries sent"
+ log_test "Vlan 10 mcast_query_interval option changed to 200"
+
+ bridge vlan global set vid 10 dev br0 mcast_snooping 1 mcast_startup_query_count 2
+ bridge vlan global set vid 10 dev br0 mcast_snooping 1 mcast_query_interval 12500
+}
+
+vlmc_query_response_intvl_test()
+{
+ RET=0
+ local goutput=`bridge -j vlan global show`
+ echo -n $goutput |
+ jq -e ".[].vlans[] | select(.vlan == 10)" &>/dev/null
+ check_err $? "Could not find vlan 10's global options"
+
+ echo -n $goutput |
+ jq -e ".[].vlans[] | select(.vlan == 10 and \
+ .mcast_query_response_interval == 1000) " &>/dev/null
+ check_err $? "Wrong default mcast_query_response_interval global vlan option value"
+ log_test "Vlan mcast_query_response_interval global option default value"
+
+ RET=0
+ bridge vlan global set vid 10 dev br0 mcast_snooping 1 mcast_query_response_interval 200
+ check_err $? "Could not set mcast_query_response_interval in vlan 10"
+ log_test "Vlan 10 mcast_query_response_interval option changed to 200"
+
+ bridge vlan global set vid 10 dev br0 mcast_snooping 1 mcast_query_response_interval 1000
+}
+
+vlmc_router_port_test()
+{
+ RET=0
+ local goutput=`bridge -j -d vlan show`
+ echo -n $goutput |
+ jq -e ".[] | select(.ifname == \"$swp1\" and \
+ .vlans[].vlan == 10)" &>/dev/null
+ check_err $? "Could not find port vlan 10's options"
+
+ echo -n $goutput |
+ jq -e ".[] | select(.ifname == \"$swp1\" and \
+ .vlans[].vlan == 10 and \
+ .vlans[].mcast_router == 1)" &>/dev/null
+ check_err $? "Wrong default port mcast_router option value"
+ log_test "Port vlan 10 option mcast_router default value"
+
+ RET=0
+ bridge vlan set vid 10 dev $swp1 mcast_router 2
+ check_err $? "Could not set port vlan 10's mcast_router option"
+ log_test "Port vlan 10 mcast_router option changed to 2"
+
+ RET=0
+ tc filter add dev $swp1 egress pref 10 prot 802.1Q \
+ flower vlan_id 10 vlan_ethtype ipv4 dst_ip 239.1.1.1 ip_proto udp action pass
+ tc filter add dev $swp2 egress pref 10 prot 802.1Q \
+ flower vlan_id 10 vlan_ethtype ipv4 dst_ip 239.1.1.1 ip_proto udp action pass
+ bridge vlan set vid 10 dev $swp2 mcast_router 0
+ # we need to enable querier and disable query response interval to
+ # make sure packets are flooded only to router ports
+ bridge vlan global set vid 10 dev br0 mcast_snooping 1 mcast_querier 1 \
+ mcast_query_response_interval 0
+ bridge vlan add vid 10 dev br0 self
+ sleep 1
+ mausezahn br0 -Q 10 -c 10 -p 128 -b 01:00:5e:01:01:01 -B 239.1.1.1 \
+ -t udp "dp=1024" &>/dev/null
+ local swp1_tcstats=$(tc_rule_stats_get $swp1 10 egress)
+ if [[ $swp1_tcstats != 10 ]]; then
+ check_err 1 "Wrong number of vlan 10 multicast packets flooded"
+ fi
+ local swp2_tcstats=$(tc_rule_stats_get $swp2 10 egress)
+ check_err $swp2_tcstats "Vlan 10 multicast packets flooded to non-router port"
+ log_test "Flood unknown vlan multicast packets to router port only"
+
+ tc filter del dev $swp2 egress pref 10
+ tc filter del dev $swp1 egress pref 10
+ bridge vlan del vid 10 dev br0 self
+ bridge vlan global set vid 10 dev br0 mcast_snooping 1 mcast_query_response_interval 1000
+ bridge vlan set vid 10 dev $swp2 mcast_router 1
+ bridge vlan set vid 10 dev $swp1 mcast_router 1
+}
+
+vlmc_filtering_test()
+{
+ RET=0
+ ip link set dev br0 type bridge vlan_filtering 0
+ ip -j -d link show dev br0 | \
+ jq -e "select(.[0].linkinfo.info_data.mcast_vlan_snooping == 1)" &>/dev/null
+ check_fail $? "Vlan filtering is disabled but multicast vlan snooping is still enabled"
+ log_test "Disable multicast vlan snooping when vlan filtering is disabled"
+}
+
+trap cleanup EXIT
+
+setup_prepare
+setup_wait
+
+tests_run
+
+exit $EXIT_STATUS
diff --git a/tools/testing/selftests/net/forwarding/lib.sh b/tools/testing/selftests/net/forwarding/lib.sh
index dfd827b..7da783d 100644
--- a/tools/testing/selftests/net/forwarding/lib.sh
+++ b/tools/testing/selftests/net/forwarding/lib.sh
@@ -20,6 +20,7 @@
NETIF_CREATE=${NETIF_CREATE:=yes}
MCD=${MCD:=smcrouted}
MC_CLI=${MC_CLI:=smcroutectl}
+PING_COUNT=${PING_COUNT:=10}
PING_TIMEOUT=${PING_TIMEOUT:=5}
WAIT_TIMEOUT=${WAIT_TIMEOUT:=20}
INTERFACE_TIMEOUT=${INTERFACE_TIMEOUT:=600}
@@ -1111,7 +1112,8 @@
vrf_name=$(master_name_get $if_name)
ip vrf exec $vrf_name \
- $PING $args $dip -c 10 -i 0.1 -w $PING_TIMEOUT &> /dev/null
+ $PING $args $dip -c $PING_COUNT -i 0.1 \
+ -w $PING_TIMEOUT &> /dev/null
}
ping_test()
@@ -1132,7 +1134,8 @@
vrf_name=$(master_name_get $if_name)
ip vrf exec $vrf_name \
- $PING6 $args $dip -c 10 -i 0.1 -w $PING_TIMEOUT &> /dev/null
+ $PING6 $args $dip -c $PING_COUNT -i 0.1 \
+ -w $PING_TIMEOUT &> /dev/null
}
ping6_test()
diff --git a/tools/testing/selftests/net/forwarding/q_in_vni_ipv6.sh b/tools/testing/selftests/net/forwarding/q_in_vni_ipv6.sh
new file mode 100755
index 0000000..0548b2b
--- /dev/null
+++ b/tools/testing/selftests/net/forwarding/q_in_vni_ipv6.sh
@@ -0,0 +1,347 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+# +-----------------------+ +------------------------+
+# | H1 (vrf) | | H2 (vrf) |
+# | + $h1.10 | | + $h2.10 |
+# | | 2001:db8:1::1/64 | | | 2001:db8:1::2/64 |
+# | | | | | |
+# | | + $h1.20 | | | + $h2.20 |
+# | \ | 2001:db8:2::1/64 | | \ | 2001:db8:2::2/64 |
+# | \| | | \| |
+# | + $h1 | | + $h2 |
+# +----|------------------+ +----|-------------------+
+# | |
+# +----|--------------------------------------------------|-------------------+
+# | SW | | |
+# | +--|--------------------------------------------------|-----------------+ |
+# | | + $swp1 BR1 (802.1ad) + $swp2 | |
+# | | vid 100 pvid untagged vid 100 pvid | |
+# | | untagged | |
+# | | + vx100 (vxlan) | |
+# | | local 2001:db8:3::1 | |
+# | | remote 2001:db8:4::1 2001:db8:5::1 | |
+# | | id 1000 dstport $VXPORT | |
+# | | vid 100 pvid untagged | |
+# | +-----------------------------------------------------------------------+ |
+# | |
+# | 2001:db8:4::0/64 via 2001:db8:3::2 |
+# | 2001:db8:5::0/64 via 2001:db8:3::2 |
+# | |
+# | + $rp1 |
+# | | 2001:db8:3::1/64 |
+# +----|----------------------------------------------------------------------+
+# |
+# +----|----------------------------------------------------------+
+# | | VRP2 (vrf) |
+# | + $rp2 |
+# | 2001:db8:3::2/64 |
+# | | (maybe) HW
+# =============================================================================
+# | | (likely) SW
+# | + v1 (veth) + v3 (veth) |
+# | | 2001:db8:4::2/64 | 2001:db8:5::2/64 |
+# +----|---------------------------------------|------------------+
+# | |
+# +----|--------------------------------+ +----|-------------------------------+
+# | + v2 (veth) NS1 (netns) | | + v4 (veth) NS2 (netns) |
+# | 2001:db8:4::1/64 | | 2001:db8:5::1/64 |
+# | | | |
+# | 2001:db8:3::0/64 via 2001:db8:4::2 | | 2001:db8:3::0/64 via 2001:db8:5::2 |
+# | 2001:db8:5::1/128 via 2001:db8:4::2 | | 2001:db8:4::1/128 via |
+# | | | 2001:db8:5::2 |
+# | +-------------------------------+ | | +-------------------------------+ |
+# | | BR2 (802.1ad) | | | | BR2 (802.1ad) | |
+# | | + vx100 (vxlan) | | | | + vx100 (vxlan) | |
+# | | local 2001:db8:4::1 | | | | local 2001:db8:5::1 | |
+# | | remote 2001:db8:3::1 | | | | remote 2001:db8:3::1 | |
+# | | remote 2001:db8:5::1 | | | | remote 2001:db8:4::1 | |
+# | | id 1000 dstport $VXPORT | | | | id 1000 dstport $VXPORT | |
+# | | vid 100 pvid untagged | | | | vid 100 pvid untagged | |
+# | | | | | | | |
+# | | + w1 (veth) | | | | + w1 (veth) | |
+# | | | vid 100 pvid untagged | | | | | vid 100 pvid untagged | |
+# | +--|----------------------------+ | | +--|----------------------------+ |
+# | | | | | |
+# | +--|----------------------------+ | | +--|----------------------------+ |
+# | | | VW2 (vrf) | | | | | VW2 (vrf) | |
+# | | + w2 (veth) | | | | + w2 (veth) | |
+# | | |\ | | | | |\ | |
+# | | | + w2.10 | | | | | + w2.10 | |
+# | | | 2001:db8:1::3/64 | | | | | 2001:db8:1::4/64 | |
+# | | | | | | | | | |
+# | | + w2.20 | | | | + w2.20 | |
+# | | 2001:db8:2::3/64 | | | | 2001:db8:2::4/64 | |
+# | +-------------------------------+ | | +-------------------------------+ |
+# +-------------------------------------+ +------------------------------------+
+
+: ${VXPORT:=4789}
+export VXPORT
+
+: ${ALL_TESTS:="
+ ping_ipv6
+ "}
+
+NUM_NETIFS=6
+source lib.sh
+
+h1_create()
+{
+ simple_if_init $h1
+ tc qdisc add dev $h1 clsact
+ vlan_create $h1 10 v$h1 2001:db8:1::1/64
+ vlan_create $h1 20 v$h1 2001:db8:2::1/64
+}
+
+h1_destroy()
+{
+ vlan_destroy $h1 20
+ vlan_destroy $h1 10
+ tc qdisc del dev $h1 clsact
+ simple_if_fini $h1
+}
+
+h2_create()
+{
+ simple_if_init $h2
+ tc qdisc add dev $h2 clsact
+ vlan_create $h2 10 v$h2 2001:db8:1::2/64
+ vlan_create $h2 20 v$h2 2001:db8:2::2/64
+}
+
+h2_destroy()
+{
+ vlan_destroy $h2 20
+ vlan_destroy $h2 10
+ tc qdisc del dev $h2 clsact
+ simple_if_fini $h2
+}
+
+rp1_set_addr()
+{
+ ip address add dev $rp1 2001:db8:3::1/64
+
+ ip route add 2001:db8:4::0/64 nexthop via 2001:db8:3::2
+ ip route add 2001:db8:5::0/64 nexthop via 2001:db8:3::2
+}
+
+rp1_unset_addr()
+{
+ ip route del 2001:db8:5::0/64 nexthop via 2001:db8:3::2
+ ip route del 2001:db8:4::0/64 nexthop via 2001:db8:3::2
+
+ ip address del dev $rp1 2001:db8:3::1/64
+}
+
+switch_create()
+{
+ ip link add name br1 type bridge vlan_filtering 1 vlan_protocol 802.1ad \
+ vlan_default_pvid 0 mcast_snooping 0
+ # Make sure the bridge uses the MAC address of the local port and not
+ # that of the VxLAN's device.
+ ip link set dev br1 address $(mac_get $swp1)
+ ip link set dev br1 up
+
+ ip link set dev $rp1 up
+ rp1_set_addr
+
+ ip link add name vx100 type vxlan id 1000 \
+ local 2001:db8:3::1 dstport "$VXPORT" \
+ nolearning udp6zerocsumrx udp6zerocsumtx tos inherit ttl 100
+ ip link set dev vx100 up
+
+ ip link set dev vx100 master br1
+ bridge vlan add vid 100 dev vx100 pvid untagged
+
+ ip link set dev $swp1 master br1
+ ip link set dev $swp1 up
+ bridge vlan add vid 100 dev $swp1 pvid untagged
+
+ ip link set dev $swp2 master br1
+ ip link set dev $swp2 up
+ bridge vlan add vid 100 dev $swp2 pvid untagged
+
+ bridge fdb append dev vx100 00:00:00:00:00:00 dst 2001:db8:4::1 self
+ bridge fdb append dev vx100 00:00:00:00:00:00 dst 2001:db8:5::1 self
+}
+
+switch_destroy()
+{
+ bridge fdb del dev vx100 00:00:00:00:00:00 dst 2001:db8:5::1 self
+ bridge fdb del dev vx100 00:00:00:00:00:00 dst 2001:db8:4::1 self
+
+ bridge vlan del vid 100 dev $swp2
+ ip link set dev $swp2 down
+ ip link set dev $swp2 nomaster
+
+ bridge vlan del vid 100 dev $swp1
+ ip link set dev $swp1 down
+ ip link set dev $swp1 nomaster
+
+ ip link set dev vx100 nomaster
+ ip link set dev vx100 down
+ ip link del dev vx100
+
+ rp1_unset_addr
+ ip link set dev $rp1 down
+
+ ip link set dev br1 down
+ ip link del dev br1
+}
+
+vrp2_create()
+{
+ simple_if_init $rp2 2001:db8:3::2/64
+ __simple_if_init v1 v$rp2 2001:db8:4::2/64
+ __simple_if_init v3 v$rp2 2001:db8:5::2/64
+ tc qdisc add dev v1 clsact
+}
+
+vrp2_destroy()
+{
+ tc qdisc del dev v1 clsact
+ __simple_if_fini v3 2001:db8:5::2/64
+ __simple_if_fini v1 2001:db8:4::2/64
+ simple_if_fini $rp2 2001:db8:3::2/64
+}
+
+ns_init_common()
+{
+ local in_if=$1; shift
+ local in_addr=$1; shift
+ local other_in_addr=$1; shift
+ local nh_addr=$1; shift
+ local host_addr1=$1; shift
+ local host_addr2=$1; shift
+
+ ip link set dev $in_if up
+ ip address add dev $in_if $in_addr/64
+ tc qdisc add dev $in_if clsact
+
+ ip link add name br2 type bridge vlan_filtering 1 vlan_protocol 802.1ad \
+ vlan_default_pvid 0
+ ip link set dev br2 up
+
+ ip link add name w1 type veth peer name w2
+
+ ip link set dev w1 master br2
+ ip link set dev w1 up
+ bridge vlan add vid 100 dev w1 pvid untagged
+
+ ip link add name vx100 type vxlan id 1000 local $in_addr \
+ dstport "$VXPORT" udp6zerocsumrx
+ ip link set dev vx100 up
+ bridge fdb append dev vx100 00:00:00:00:00:00 dst 2001:db8:3::1 self
+ bridge fdb append dev vx100 00:00:00:00:00:00 dst $other_in_addr self
+
+ ip link set dev vx100 master br2
+ tc qdisc add dev vx100 clsact
+
+ bridge vlan add vid 100 dev vx100 pvid untagged
+
+ simple_if_init w2
+ vlan_create w2 10 vw2 $host_addr1/64
+ vlan_create w2 20 vw2 $host_addr2/64
+
+ ip route add 2001:db8:3::0/64 nexthop via $nh_addr
+ ip route add $other_in_addr/128 nexthop via $nh_addr
+}
+export -f ns_init_common
+
+ns1_create()
+{
+ ip netns add ns1
+ ip link set dev v2 netns ns1
+ in_ns ns1 \
+ ns_init_common v2 2001:db8:4::1 2001:db8:5::1 2001:db8:4::2 \
+ 2001:db8:1::3 2001:db8:2::3
+}
+
+ns1_destroy()
+{
+ ip netns exec ns1 ip link set dev v2 netns 1
+ ip netns del ns1
+}
+
+ns2_create()
+{
+ ip netns add ns2
+ ip link set dev v4 netns ns2
+ in_ns ns2 \
+ ns_init_common v4 2001:db8:5::1 2001:db8:4::1 2001:db8:5::2 \
+ 2001:db8:1::4 2001:db8:2::4
+}
+
+ns2_destroy()
+{
+ ip netns exec ns2 ip link set dev v4 netns 1
+ ip netns del ns2
+}
+
+setup_prepare()
+{
+ h1=${NETIFS[p1]}
+ swp1=${NETIFS[p2]}
+
+ swp2=${NETIFS[p3]}
+ h2=${NETIFS[p4]}
+
+ rp1=${NETIFS[p5]}
+ rp2=${NETIFS[p6]}
+
+ vrf_prepare
+ forwarding_enable
+
+ h1_create
+ h2_create
+ switch_create
+
+ ip link add name v1 type veth peer name v2
+ ip link add name v3 type veth peer name v4
+ vrp2_create
+ ns1_create
+ ns2_create
+
+ r1_mac=$(in_ns ns1 mac_get w2)
+ r2_mac=$(in_ns ns2 mac_get w2)
+ h2_mac=$(mac_get $h2)
+}
+
+cleanup()
+{
+ pre_cleanup
+
+ ns2_destroy
+ ns1_destroy
+ vrp2_destroy
+ ip link del dev v3
+ ip link del dev v1
+
+ switch_destroy
+ h2_destroy
+ h1_destroy
+
+ forwarding_restore
+ vrf_cleanup
+}
+
+ping_ipv6()
+{
+ ping6_test $h1 2001:db8:1::2 ": local->local"
+ ping6_test $h1 2001:db8:1::3 ": local->remote 1"
+ ping6_test $h1 2001:db8:1::4 ": local->remote 2"
+}
+
+test_all()
+{
+ echo "Running tests with UDP port $VXPORT"
+ tests_run
+}
+
+trap cleanup EXIT
+
+setup_prepare
+setup_wait
+test_all
+
+exit $EXIT_STATUS
diff --git a/tools/testing/selftests/net/forwarding/vxlan_asymmetric_ipv6.sh b/tools/testing/selftests/net/forwarding/vxlan_asymmetric_ipv6.sh
new file mode 100755
index 0000000..f493009
--- /dev/null
+++ b/tools/testing/selftests/net/forwarding/vxlan_asymmetric_ipv6.sh
@@ -0,0 +1,504 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+# +--------------------------------+ +-----------------------------+
+# | vrf-h1 | | vrf-h2 |
+# | + $h1 | | + $h2 |
+# | | 2001:db8:1::1/64 | | | 2001:db8:2::1/64 |
+# | | default via 2001:db8:1::3 | | | default via 2001:db8:2::3 |
+# +----|---------------------------+ +-|---------------------------+
+# | |
+# +----|------------------------------------------|---------------------------+
+# | SW | | |
+# | +--|------------------------------------------|-------------------------+ |
+# | | + $swp1 br1 + $swp2 | |
+# | | vid 10 pvid untagged vid 20 pvid untagged | |
+# | | | |
+# | | + vx10 + vx20 | |
+# | | local 2001:db8:3::1 local 2001:db8:3::1 | |
+# | | remote 2001:db8:3::2 remote 2001:db8:3::2 | |
+# | | id 1000 id 2000 | |
+# | | dstport 4789 dstport 4789 | |
+# | | vid 10 pvid untagged vid 20 pvid untagged | |
+# | | | |
+# | +-----------------------------------+-----------------------------------+ |
+# | | |
+# | +-----------------------------------|-----------------------------------+ |
+# | | | | |
+# | | +--------------------------------+--------------------------------+ | |
+# | | | | | |
+# | | + vlan10 vlan20 + | |
+# | | | 2001:db8:1::2/64 2001:db8:2::2/64 | | |
+# | | | | | |
+# | | + vlan10-v (macvlan) vlan20-v (macvlan) + | |
+# | | 2001:db8:1::3/64 2001:db8:2::3/64 | |
+# | | 00:00:5e:00:01:01 00:00:5e:00:01:01 | |
+# | | vrf-green | |
+# | +-----------------------------------------------------------------------+ |
+# | |
+# | + $rp1 +lo |
+# | | 2001:db8:4::1/64 2001:db8:3::1/128 |
+# +----|----------------------------------------------------------------------+
+# |
+# +----|--------------------------------------------------------+
+# | | vrf-spine |
+# | + $rp2 |
+# | 2001:db8:4::2/64 |
+# | | (maybe) HW
+# =============================================================================
+# | | (likely) SW
+# | |
+# | + v1 (veth) |
+# | | 2001:db8:5::2/64 |
+# +----|--------------------------------------------------------+
+# |
+# +----|----------------------------------------------------------------------+
+# | + v2 (veth) +lo NS1 (netns) |
+# | 2001:db8:5::1/64 2001:db8:3::2/128 |
+# | |
+# | +-----------------------------------------------------------------------+ |
+# | | vrf-green | |
+# | | + vlan10-v (macvlan) vlan20-v (macvlan) + | |
+# | | | 2001:db8:1::3/64 2001:db8:2::3/64 | | |
+# | | | 00:00:5e:00:01:01 00:00:5e:00:01:01 | | |
+# | | | | | |
+# | | + vlan10 vlan20 + | |
+# | | | 2001:db8:1::3/64 2001:db8:2::3/64 | | |
+# | | | | | |
+# | | +--------------------------------+--------------------------------+ | |
+# | | | | |
+# | +-----------------------------------|-----------------------------------+ |
+# | | |
+# | +-----------------------------------+-----------------------------------+ |
+# | | | |
+# | | + vx10 + vx20 | |
+# | | local 2001:db8:3::2 local 2001:db8:3::2 | |
+# | | remote 2001:db8:3::1 remote 2001:db8:3::1 | |
+# | | id 1000 id 2000 | |
+# | | dstport 4789 dstport 4789 | |
+# | | vid 10 pvid untagged vid 20 pvid untagged | |
+# | | | |
+# | | + w1 (veth) + w3 (veth) | |
+# | | | vid 10 pvid untagged br1 | vid 20 pvid untagged | |
+# | +--|------------------------------------------|-------------------------+ |
+# | | | |
+# | | | |
+# | +--|----------------------+ +--|-------------------------+ |
+# | | | vrf-h1 | | | vrf-h2 | |
+# | | + w2 (veth) | | + w4 (veth) | |
+# | | 2001:db8:1::4/64 | | 2001:db8:2::4/64 | |
+# | | default via | | default via | |
+# | | 2001:db8:1::3/64 | | 2001:db8:2::3/64 | |
+# | +-------------------------+ +----------------------------+ |
+# +---------------------------------------------------------------------------+
+
+ALL_TESTS="
+ ping_ipv6
+ arp_decap
+"
+NUM_NETIFS=6
+source lib.sh
+
+require_command $ARPING
+
+hx_create()
+{
+ local vrf_name=$1; shift
+ local if_name=$1; shift
+ local ip_addr=$1; shift
+ local gw_ip=$1; shift
+
+ vrf_create $vrf_name
+ ip link set dev $if_name master $vrf_name
+ ip link set dev $vrf_name up
+ ip link set dev $if_name up
+
+ ip address add $ip_addr/64 dev $if_name
+ ip neigh replace $gw_ip lladdr 00:00:5e:00:01:01 nud permanent \
+ dev $if_name
+ ip route add default vrf $vrf_name nexthop via $gw_ip
+}
+export -f hx_create
+
+hx_destroy()
+{
+ local vrf_name=$1; shift
+ local if_name=$1; shift
+ local ip_addr=$1; shift
+ local gw_ip=$1; shift
+
+ ip route del default vrf $vrf_name nexthop via $gw_ip
+ ip neigh del $gw_ip dev $if_name
+ ip address del $ip_addr/64 dev $if_name
+
+ ip link set dev $if_name down
+ vrf_destroy $vrf_name
+}
+
+h1_create()
+{
+ hx_create "vrf-h1" $h1 2001:db8:1::1 2001:db8:1::3
+}
+
+h1_destroy()
+{
+ hx_destroy "vrf-h1" $h1 2001:db8:1::1 2001:db8:1::3
+}
+
+h2_create()
+{
+ hx_create "vrf-h2" $h2 2001:db8:2::1 2001:db8:2::3
+}
+
+h2_destroy()
+{
+ hx_destroy "vrf-h2" $h2 2001:db8:2::1 2001:db8:2::3
+}
+
+switch_create()
+{
+ ip link add name br1 type bridge vlan_filtering 1 vlan_default_pvid 0 \
+ mcast_snooping 0
+ # Make sure the bridge uses the MAC address of the local port and not
+ # that of the VxLAN's device.
+ ip link set dev br1 address $(mac_get $swp1)
+ ip link set dev br1 up
+
+ ip link set dev $rp1 up
+ ip address add dev $rp1 2001:db8:4::1/64
+ ip route add 2001:db8:3::2/128 nexthop via 2001:db8:4::2
+
+ ip link add name vx10 type vxlan id 1000 \
+ local 2001:db8:3::1 remote 2001:db8:3::2 dstport 4789 \
+ nolearning udp6zerocsumrx udp6zerocsumtx tos inherit ttl 100
+ ip link set dev vx10 up
+
+ ip link set dev vx10 master br1
+ bridge vlan add vid 10 dev vx10 pvid untagged
+
+ ip link add name vx20 type vxlan id 2000 \
+ local 2001:db8:3::1 remote 2001:db8:3::2 dstport 4789 \
+ nolearning udp6zerocsumrx udp6zerocsumtx tos inherit ttl 100
+ ip link set dev vx20 up
+
+ ip link set dev vx20 master br1
+ bridge vlan add vid 20 dev vx20 pvid untagged
+
+ ip link set dev $swp1 master br1
+ ip link set dev $swp1 up
+ bridge vlan add vid 10 dev $swp1 pvid untagged
+
+ ip link set dev $swp2 master br1
+ ip link set dev $swp2 up
+ bridge vlan add vid 20 dev $swp2 pvid untagged
+
+ ip address add 2001:db8:3::1/128 dev lo
+
+ # Create SVIs
+ vrf_create "vrf-green"
+ ip link set dev vrf-green up
+
+ ip link add link br1 name vlan10 up master vrf-green type vlan id 10
+ ip address add 2001:db8:1::2/64 dev vlan10
+ ip link add link vlan10 name vlan10-v up master vrf-green \
+ address 00:00:5e:00:01:01 type macvlan mode private
+ ip address add 2001:db8:1::3/64 dev vlan10-v
+
+ ip link add link br1 name vlan20 up master vrf-green type vlan id 20
+ ip address add 2001:db8:2::2/64 dev vlan20
+ ip link add link vlan20 name vlan20-v up master vrf-green \
+ address 00:00:5e:00:01:01 type macvlan mode private
+ ip address add 2001:db8:2::3/64 dev vlan20-v
+
+ bridge vlan add vid 10 dev br1 self
+ bridge vlan add vid 20 dev br1 self
+
+ bridge fdb add 00:00:5e:00:01:01 dev br1 self local vlan 10
+ bridge fdb add 00:00:5e:00:01:01 dev br1 self local vlan 20
+
+}
+
+switch_destroy()
+{
+ bridge fdb del 00:00:5e:00:01:01 dev br1 self local vlan 20
+ bridge fdb del 00:00:5e:00:01:01 dev br1 self local vlan 10
+
+ bridge vlan del vid 20 dev br1 self
+ bridge vlan del vid 10 dev br1 self
+
+ ip link del dev vlan20
+
+ ip link del dev vlan10
+
+ vrf_destroy "vrf-green"
+
+ ip address del 2001:db8:3::1/128 dev lo
+
+ bridge vlan del vid 20 dev $swp2
+ ip link set dev $swp2 down
+ ip link set dev $swp2 nomaster
+
+ bridge vlan del vid 10 dev $swp1
+ ip link set dev $swp1 down
+ ip link set dev $swp1 nomaster
+
+ bridge vlan del vid 20 dev vx20
+ ip link set dev vx20 nomaster
+
+ ip link set dev vx20 down
+ ip link del dev vx20
+
+ bridge vlan del vid 10 dev vx10
+ ip link set dev vx10 nomaster
+
+ ip link set dev vx10 down
+ ip link del dev vx10
+
+ ip route del 2001:db8:3::2 nexthop via 2001:db8:4::2
+ ip address del dev $rp1 2001:db8:4::1/64
+ ip link set dev $rp1 down
+
+ ip link set dev br1 down
+ ip link del dev br1
+}
+
+spine_create()
+{
+ vrf_create "vrf-spine"
+ ip link set dev $rp2 master vrf-spine
+ ip link set dev v1 master vrf-spine
+ ip link set dev vrf-spine up
+ ip link set dev $rp2 up
+ ip link set dev v1 up
+
+ ip address add 2001:db8:4::2/64 dev $rp2
+ ip address add 2001:db8:5::2/64 dev v1
+
+ ip route add 2001:db8:3::1/128 vrf vrf-spine nexthop via \
+ 2001:db8:4::1
+ ip route add 2001:db8:3::2/128 vrf vrf-spine nexthop via \
+ 2001:db8:5::1
+}
+
+spine_destroy()
+{
+ ip route del 2001:db8:3::2/128 vrf vrf-spine nexthop via \
+ 2001:db8:5::1
+ ip route del 2001:db8:3::1/128 vrf vrf-spine nexthop via \
+ 2001:db8:4::1
+
+ ip address del 2001:db8:5::2/64 dev v1
+ ip address del 2001:db8:4::2/64 dev $rp2
+
+ ip link set dev v1 down
+ ip link set dev $rp2 down
+ vrf_destroy "vrf-spine"
+}
+
+ns_h1_create()
+{
+ hx_create "vrf-h1" w2 2001:db8:1::4 2001:db8:1::3
+}
+export -f ns_h1_create
+
+ns_h2_create()
+{
+ hx_create "vrf-h2" w4 2001:db8:2::4 2001:db8:2::3
+}
+export -f ns_h2_create
+
+ns_switch_create()
+{
+ ip link add name br1 type bridge vlan_filtering 1 vlan_default_pvid 0 \
+ mcast_snooping 0
+ ip link set dev br1 up
+
+ ip link set dev v2 up
+ ip address add dev v2 2001:db8:5::1/64
+ ip route add 2001:db8:3::1 nexthop via 2001:db8:5::2
+
+ ip link add name vx10 type vxlan id 1000 \
+ local 2001:db8:3::2 remote 2001:db8:3::1 dstport 4789 \
+ nolearning udp6zerocsumrx udp6zerocsumtx tos inherit ttl 100
+ ip link set dev vx10 up
+
+ ip link set dev vx10 master br1
+ bridge vlan add vid 10 dev vx10 pvid untagged
+
+ ip link add name vx20 type vxlan id 2000 \
+ local 2001:db8:3::2 remote 2001:db8:3::1 dstport 4789 \
+ nolearning udp6zerocsumrx udp6zerocsumtx tos inherit ttl 100
+ ip link set dev vx20 up
+
+ ip link set dev vx20 master br1
+ bridge vlan add vid 20 dev vx20 pvid untagged
+
+ ip link set dev w1 master br1
+ ip link set dev w1 up
+ bridge vlan add vid 10 dev w1 pvid untagged
+
+ ip link set dev w3 master br1
+ ip link set dev w3 up
+ bridge vlan add vid 20 dev w3 pvid untagged
+
+ ip address add 2001:db8:3::2/128 dev lo
+
+ # Create SVIs
+ vrf_create "vrf-green"
+ ip link set dev vrf-green up
+
+ ip link add link br1 name vlan10 up master vrf-green type vlan id 10
+ ip address add 2001:db8:1::3/64 dev vlan10
+ ip link add link vlan10 name vlan10-v up master vrf-green \
+ address 00:00:5e:00:01:01 type macvlan mode private
+ ip address add 2001:db8:1::3/64 dev vlan10-v
+
+ ip link add link br1 name vlan20 up master vrf-green type vlan id 20
+ ip address add 2001:db8:2::3/64 dev vlan20
+ ip link add link vlan20 name vlan20-v up master vrf-green \
+ address 00:00:5e:00:01:01 type macvlan mode private
+ ip address add 2001:db8:2::3/64 dev vlan20-v
+
+ bridge vlan add vid 10 dev br1 self
+ bridge vlan add vid 20 dev br1 self
+
+ bridge fdb add 00:00:5e:00:01:01 dev br1 self local vlan 10
+ bridge fdb add 00:00:5e:00:01:01 dev br1 self local vlan 20
+}
+export -f ns_switch_create
+
+ns_init()
+{
+ ip link add name w1 type veth peer name w2
+ ip link add name w3 type veth peer name w4
+
+ ip link set dev lo up
+
+ ns_h1_create
+ ns_h2_create
+ ns_switch_create
+}
+export -f ns_init
+
+ns1_create()
+{
+ ip netns add ns1
+ ip link set dev v2 netns ns1
+ in_ns ns1 ns_init
+}
+
+ns1_destroy()
+{
+ ip netns exec ns1 ip link set dev v2 netns 1
+ ip netns del ns1
+}
+
+macs_populate()
+{
+ local mac1=$1; shift
+ local mac2=$1; shift
+ local ip1=$1; shift
+ local ip2=$1; shift
+ local dst=$1; shift
+
+ bridge fdb add $mac1 dev vx10 self master extern_learn static \
+ dst $dst vlan 10
+ bridge fdb add $mac2 dev vx20 self master extern_learn static \
+ dst $dst vlan 20
+
+ ip neigh add $ip1 lladdr $mac1 nud noarp dev vlan10 \
+ extern_learn
+ ip neigh add $ip2 lladdr $mac2 nud noarp dev vlan20 \
+ extern_learn
+}
+export -f macs_populate
+
+macs_initialize()
+{
+ local h1_ns_mac=$(in_ns ns1 mac_get w2)
+ local h2_ns_mac=$(in_ns ns1 mac_get w4)
+ local h1_mac=$(mac_get $h1)
+ local h2_mac=$(mac_get $h2)
+
+ macs_populate $h1_ns_mac $h2_ns_mac 2001:db8:1::4 2001:db8:2::4 \
+ 2001:db8:3::2
+ in_ns ns1 macs_populate $h1_mac $h2_mac 2001:db8:1::1 2001:db8:2::1 \
+ 2001:db8:3::1
+}
+
+setup_prepare()
+{
+ h1=${NETIFS[p1]}
+ swp1=${NETIFS[p2]}
+
+ swp2=${NETIFS[p3]}
+ h2=${NETIFS[p4]}
+
+ rp1=${NETIFS[p5]}
+ rp2=${NETIFS[p6]}
+
+ vrf_prepare
+ forwarding_enable
+
+ h1_create
+ h2_create
+ switch_create
+
+ ip link add name v1 type veth peer name v2
+ spine_create
+ ns1_create
+ in_ns ns1 forwarding_enable
+
+ macs_initialize
+}
+
+cleanup()
+{
+ pre_cleanup
+
+ ns1_destroy
+ spine_destroy
+ ip link del dev v1
+
+ switch_destroy
+ h2_destroy
+ h1_destroy
+
+ forwarding_restore
+ vrf_cleanup
+}
+
+ping_ipv6()
+{
+ ping6_test $h1 2001:db8:2::1 ": local->local vid 10->vid 20"
+ ping6_test $h1 2001:db8:1::4 ": local->remote vid 10->vid 10"
+ ping6_test $h2 2001:db8:2::4 ": local->remote vid 20->vid 20"
+ ping6_test $h1 2001:db8:2::4 ": local->remote vid 10->vid 20"
+ ping6_test $h2 2001:db8:1::4 ": local->remote vid 20->vid 10"
+}
+
+arp_decap()
+{
+ # Repeat the ping tests, but without populating the neighbours. This
+ # makes sure we correctly decapsulate ARP packets
+ log_info "deleting neighbours from vlan interfaces"
+
+ ip neigh del 2001:db8:1::4 dev vlan10
+ ip neigh del 2001:db8:2::4 dev vlan20
+
+ ping_ipv6
+
+ ip neigh replace 2001:db8:1::4 lladdr $(in_ns ns1 mac_get w2) \
+ nud noarp dev vlan10 extern_learn
+ ip neigh replace 2001:db8:2::4 lladdr $(in_ns ns1 mac_get w4) \
+ nud noarp dev vlan20 extern_learn
+}
+
+trap cleanup EXIT
+
+setup_prepare
+setup_wait
+
+tests_run
+
+exit $EXIT_STATUS
diff --git a/tools/testing/selftests/net/forwarding/vxlan_bridge_1d_ipv6.sh b/tools/testing/selftests/net/forwarding/vxlan_bridge_1d_ipv6.sh
new file mode 100755
index 0000000..ac97f07
--- /dev/null
+++ b/tools/testing/selftests/net/forwarding/vxlan_bridge_1d_ipv6.sh
@@ -0,0 +1,804 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+# +-----------------------+ +------------------------+
+# | H1 (vrf) | | H2 (vrf) |
+# | + $h1 | | + $h2 |
+# | | 192.0.2.1/28 | | | 192.0.2.2/28 |
+# | | 2001:db8:1::1/64 | | | 2001:db8:1::2/64 |
+# +----|------------------+ +----|-------------------+
+# | |
+# +----|--------------------------------------------------|-------------------+
+# | SW | | |
+# | +--|--------------------------------------------------|-----------------+ |
+# | | + $swp1 BR1 (802.1d) + $swp2 | |
+# | | | |
+# | | + vx1 (vxlan) | |
+# | | local 2001:db8:3::1 | |
+# | | remote 2001:db8:4::1 2001:db8:5::1 | |
+# | | id 1000 dstport $VXPORT | |
+# | +-----------------------------------------------------------------------+ |
+# | |
+# | 2001:db8:4::0/64 via 2001:db8:3::2 |
+# | 2001:db8:5::0/64 via 2001:db8:3::2 |
+# | |
+# | + $rp1 |
+# | | 2001:db8:3::1/64 |
+# +----|----------------------------------------------------------------------+
+# |
+# +----|----------------------------------------------------------+
+# | | VRP2 (vrf) |
+# | + $rp2 |
+# | 2001:db8:3::2/64 |
+# | | (maybe) HW
+# =============================================================================
+# | | (likely) SW
+# | + v1 (veth) + v3 (veth) |
+# | | 2001:db8:4::2/64 | 2001:db8:5::2/64 |
+# +----|---------------------------------------|------------------+
+# | |
+# +----|--------------------------------+ +----|-------------------------------+
+# | + v2 (veth) NS1 (netns) | | + v4 (veth) NS2 (netns) |
+# | 2001:db8:4::1/64 | | 2001:db8:5::1/64 |
+# | | | |
+# | 2001:db8:3::0/64 via 2001:db8:4::2 | | 2001:db8:3::0/64 via 2001:db8:5::2 |
+# | 2001:db8:5::1/128 via 2001:db8:4::2 | | 2001:db8:4::1/128 via |
+# | | | 2001:db8:5::2 |
+# | | | |
+# | +-------------------------------+ | | +-------------------------------+ |
+# | | BR2 (802.1d) | | | | BR2 (802.1d) | |
+# | | + vx2 (vxlan) | | | | + vx2 (vxlan) | |
+# | | local 2001:db8:4::1 | | | | local 2001:db8:5::1 | |
+# | | remote 2001:db8:3::1 | | | | remote 2001:db8:3::1 | |
+# | | remote 2001:db8:5::1 | | | | remote 2001:db8:4::1 | |
+# | | id 1000 dstport $VXPORT | | | | id 1000 dstport $VXPORT | |
+# | | | | | | | |
+# | | + w1 (veth) | | | | + w1 (veth) | |
+# | +--|----------------------------+ | | +--|----------------------------+ |
+# | | | | | |
+# | +--|----------------------------+ | | +--|----------------------------+ |
+# | | + w2 (veth) VW2 (vrf) | | | | + w2 (veth) VW2 (vrf) | |
+# | | 192.0.2.3/28 | | | | 192.0.2.4/28 | |
+# | | 2001:db8:1::3/64 | | | | 2001:db8:1::4/64 | |
+# | +-------------------------------+ | | +-------------------------------+ |
+# +-------------------------------------+ +------------------------------------+
+
+: ${VXPORT:=4789}
+export VXPORT
+
+: ${ALL_TESTS:="
+ ping_ipv4
+ ping_ipv6
+ test_flood
+ test_unicast
+ test_ttl
+ test_tos
+ test_ecn_encap
+ test_ecn_decap
+ reapply_config
+ ping_ipv4
+ ping_ipv6
+ test_flood
+ test_unicast
+"}
+
+NUM_NETIFS=6
+source lib.sh
+source tc_common.sh
+
+h1_create()
+{
+ simple_if_init $h1 192.0.2.1/28 2001:db8:1::1/64
+ tc qdisc add dev $h1 clsact
+}
+
+h1_destroy()
+{
+ tc qdisc del dev $h1 clsact
+ simple_if_fini $h1 192.0.2.1/28 2001:db8:1::1/64
+}
+
+h2_create()
+{
+ simple_if_init $h2 192.0.2.2/28 2001:db8:1::2/64
+ tc qdisc add dev $h2 clsact
+}
+
+h2_destroy()
+{
+ tc qdisc del dev $h2 clsact
+ simple_if_fini $h2 192.0.2.2/28 2001:db8:1::2/64
+}
+
+rp1_set_addr()
+{
+ ip address add dev $rp1 2001:db8:3::1/64
+
+ ip route add 2001:db8:4::0/64 nexthop via 2001:db8:3::2
+ ip route add 2001:db8:5::0/64 nexthop via 2001:db8:3::2
+}
+
+rp1_unset_addr()
+{
+ ip route del 2001:db8:5::0/64 nexthop via 2001:db8:3::2
+ ip route del 2001:db8:4::0/64 nexthop via 2001:db8:3::2
+
+ ip address del dev $rp1 2001:db8:3::1/64
+}
+
+switch_create()
+{
+ ip link add name br1 type bridge vlan_filtering 0 mcast_snooping 0
+ # Make sure the bridge uses the MAC address of the local port and not
+ # that of the VxLAN's device.
+ ip link set dev br1 address $(mac_get $swp1)
+ ip link set dev br1 up
+
+ ip link set dev $rp1 up
+ rp1_set_addr
+ tc qdisc add dev $rp1 clsact
+
+ ip link add name vx1 type vxlan id 1000 local 2001:db8:3::1 \
+ dstport "$VXPORT" nolearning udp6zerocsumrx udp6zerocsumtx \
+ tos inherit ttl 100
+ ip link set dev vx1 up
+
+ ip link set dev vx1 master br1
+ ip link set dev $swp1 master br1
+ ip link set dev $swp1 up
+ tc qdisc add dev $swp1 clsact
+
+ ip link set dev $swp2 master br1
+ ip link set dev $swp2 up
+
+ bridge fdb append dev vx1 00:00:00:00:00:00 dst 2001:db8:4::1 self
+ bridge fdb append dev vx1 00:00:00:00:00:00 dst 2001:db8:5::1 self
+}
+
+switch_destroy()
+{
+ bridge fdb del dev vx1 00:00:00:00:00:00 dst 2001:db8:5::1 self
+ bridge fdb del dev vx1 00:00:00:00:00:00 dst 2001:db8:4::1 self
+
+ ip link set dev $swp2 down
+ ip link set dev $swp2 nomaster
+
+ tc qdisc del dev $swp1 clsact
+ ip link set dev $swp1 down
+ ip link set dev $swp1 nomaster
+
+ ip link set dev vx1 nomaster
+ ip link set dev vx1 down
+ ip link del dev vx1
+
+ tc qdisc del dev $rp1 clsact
+ rp1_unset_addr
+ ip link set dev $rp1 down
+
+ ip link set dev br1 down
+ ip link del dev br1
+}
+
+vrp2_create()
+{
+ simple_if_init $rp2 2001:db8:3::2/64
+ __simple_if_init v1 v$rp2 2001:db8:4::2/64
+ __simple_if_init v3 v$rp2 2001:db8:5::2/64
+ tc qdisc add dev v1 clsact
+}
+
+vrp2_destroy()
+{
+ tc qdisc del dev v1 clsact
+ __simple_if_fini v3 2001:db8:5::2/64
+ __simple_if_fini v1 2001:db8:4::2/64
+ simple_if_fini $rp2 2001:db8:3::2/64
+}
+
+ns_init_common()
+{
+ local in_if=$1; shift
+ local in_addr=$1; shift
+ local other_in_addr=$1; shift
+ local nh_addr=$1; shift
+ local host_addr_ipv4=$1; shift
+ local host_addr_ipv6=$1; shift
+
+ ip link set dev $in_if up
+ ip address add dev $in_if $in_addr/64
+ tc qdisc add dev $in_if clsact
+
+ ip link add name br2 type bridge vlan_filtering 0
+ ip link set dev br2 up
+
+ ip link add name w1 type veth peer name w2
+
+ ip link set dev w1 master br2
+ ip link set dev w1 up
+
+ ip link add name vx2 type vxlan id 1000 local $in_addr \
+ dstport "$VXPORT" udp6zerocsumrx
+ ip link set dev vx2 up
+ bridge fdb append dev vx2 00:00:00:00:00:00 dst 2001:db8:3::1 self
+ bridge fdb append dev vx2 00:00:00:00:00:00 dst $other_in_addr self
+
+ ip link set dev vx2 master br2
+ tc qdisc add dev vx2 clsact
+
+ simple_if_init w2 $host_addr_ipv4/28 $host_addr_ipv6/64
+
+ ip route add 2001:db8:3::0/64 nexthop via $nh_addr
+ ip route add $other_in_addr/128 nexthop via $nh_addr
+}
+export -f ns_init_common
+
+ns1_create()
+{
+ ip netns add ns1
+ ip link set dev v2 netns ns1
+ in_ns ns1 \
+ ns_init_common v2 2001:db8:4::1 2001:db8:5::1 2001:db8:4::2 \
+ 192.0.2.3 2001:db8:1::3
+}
+
+ns1_destroy()
+{
+ ip netns exec ns1 ip link set dev v2 netns 1
+ ip netns del ns1
+}
+
+ns2_create()
+{
+ ip netns add ns2
+ ip link set dev v4 netns ns2
+ in_ns ns2 \
+ ns_init_common v4 2001:db8:5::1 2001:db8:4::1 2001:db8:5::2 \
+ 192.0.2.4 2001:db8:1::4
+}
+
+ns2_destroy()
+{
+ ip netns exec ns2 ip link set dev v4 netns 1
+ ip netns del ns2
+}
+
+setup_prepare()
+{
+ h1=${NETIFS[p1]}
+ swp1=${NETIFS[p2]}
+
+ swp2=${NETIFS[p3]}
+ h2=${NETIFS[p4]}
+
+ rp1=${NETIFS[p5]}
+ rp2=${NETIFS[p6]}
+
+ vrf_prepare
+ forwarding_enable
+
+ h1_create
+ h2_create
+ switch_create
+
+ ip link add name v1 type veth peer name v2
+ ip link add name v3 type veth peer name v4
+ vrp2_create
+ ns1_create
+ ns2_create
+
+ r1_mac=$(in_ns ns1 mac_get w2)
+ r2_mac=$(in_ns ns2 mac_get w2)
+ h2_mac=$(mac_get $h2)
+}
+
+cleanup()
+{
+ pre_cleanup
+
+ ns2_destroy
+ ns1_destroy
+ vrp2_destroy
+ ip link del dev v3
+ ip link del dev v1
+
+ switch_destroy
+ h2_destroy
+ h1_destroy
+
+ forwarding_restore
+ vrf_cleanup
+}
+
+# For the first round of tests, vx1 is the first device to get
+# attached to the bridge, and at that point the local IP is already
+# configured. Try the other scenario of attaching the devices to a an
+# already-offloaded bridge, and only then assign the local IP.
+reapply_config()
+{
+ log_info "Reapplying configuration"
+
+ bridge fdb del dev vx1 00:00:00:00:00:00 dst 2001:db8:5::1 self
+ bridge fdb del dev vx1 00:00:00:00:00:00 dst 2001:db8:4::1 self
+ ip link set dev vx1 nomaster
+ rp1_unset_addr
+ sleep 5
+
+ ip link set dev vx1 master br1
+ bridge fdb append dev vx1 00:00:00:00:00:00 dst 2001:db8:4::1 self
+ bridge fdb append dev vx1 00:00:00:00:00:00 dst 2001:db8:5::1 self
+ sleep 1
+ rp1_set_addr
+ sleep 5
+}
+
+__ping_ipv4()
+{
+ local vxlan_local_ip=$1; shift
+ local vxlan_remote_ip=$1; shift
+ local src_ip=$1; shift
+ local dst_ip=$1; shift
+ local dev=$1; shift
+ local info=$1; shift
+
+ RET=0
+
+ tc filter add dev $rp1 egress protocol ipv6 pref 1 handle 101 \
+ flower ip_proto udp src_ip $vxlan_local_ip \
+ dst_ip $vxlan_remote_ip dst_port $VXPORT $TC_FLAG action pass
+ # Match ICMP-reply packets after decapsulation, so source IP is
+ # destination IP of the ping and destination IP is source IP of the
+ # ping.
+ tc filter add dev $swp1 egress protocol ip pref 1 handle 101 \
+ flower src_ip $dst_ip dst_ip $src_ip \
+ $TC_FLAG action pass
+
+ # Send 100 packets and verify that at least 100 packets hit the rule,
+ # to overcome ARP noise.
+ PING_COUNT=100 PING_TIMEOUT=11 ping_do $dev $dst_ip
+ check_err $? "Ping failed"
+
+ tc_check_at_least_x_packets "dev $rp1 egress" 101 10 100
+ check_err $? "Encapsulated packets did not go through router"
+
+ tc_check_at_least_x_packets "dev $swp1 egress" 101 10 100
+ check_err $? "Decapsulated packets did not go through switch"
+
+ log_test "ping: $info"
+
+ tc filter del dev $swp1 egress
+ tc filter del dev $rp1 egress
+}
+
+ping_ipv4()
+{
+ RET=0
+
+ local local_sw_ip=2001:db8:3::1
+ local remote_ns1_ip=2001:db8:4::1
+ local remote_ns2_ip=2001:db8:5::1
+ local h1_ip=192.0.2.1
+ local w2_ns1_ip=192.0.2.3
+ local w2_ns2_ip=192.0.2.4
+
+ ping_test $h1 192.0.2.2 ": local->local"
+
+ __ping_ipv4 $local_sw_ip $remote_ns1_ip $h1_ip $w2_ns1_ip $h1 \
+ "local->remote 1"
+ __ping_ipv4 $local_sw_ip $remote_ns2_ip $h1_ip $w2_ns2_ip $h1 \
+ "local->remote 2"
+}
+
+__ping_ipv6()
+{
+ local vxlan_local_ip=$1; shift
+ local vxlan_remote_ip=$1; shift
+ local src_ip=$1; shift
+ local dst_ip=$1; shift
+ local dev=$1; shift
+ local info=$1; shift
+
+ RET=0
+
+ tc filter add dev $rp1 egress protocol ipv6 pref 1 handle 101 \
+ flower ip_proto udp src_ip $vxlan_local_ip \
+ dst_ip $vxlan_remote_ip dst_port $VXPORT $TC_FLAG action pass
+ # Match ICMP-reply packets after decapsulation, so source IP is
+ # destination IP of the ping and destination IP is source IP of the
+ # ping.
+ tc filter add dev $swp1 egress protocol ipv6 pref 1 handle 101 \
+ flower src_ip $dst_ip dst_ip $src_ip $TC_FLAG action pass
+
+ # Send 100 packets and verify that at least 100 packets hit the rule,
+ # to overcome neighbor discovery noise.
+ PING_COUNT=100 PING_TIMEOUT=11 ping6_do $dev $dst_ip
+ check_err $? "Ping failed"
+
+ tc_check_at_least_x_packets "dev $rp1 egress" 101 100
+ check_err $? "Encapsulated packets did not go through router"
+
+ tc_check_at_least_x_packets "dev $swp1 egress" 101 100
+ check_err $? "Decapsulated packets did not go through switch"
+
+ log_test "ping6: $info"
+
+ tc filter del dev $swp1 egress
+ tc filter del dev $rp1 egress
+}
+
+ping_ipv6()
+{
+ RET=0
+
+ local local_sw_ip=2001:db8:3::1
+ local remote_ns1_ip=2001:db8:4::1
+ local remote_ns2_ip=2001:db8:5::1
+ local h1_ip=2001:db8:1::1
+ local w2_ns1_ip=2001:db8:1::3
+ local w2_ns2_ip=2001:db8:1::4
+
+ ping6_test $h1 2001:db8:1::2 ": local->local"
+
+ __ping_ipv6 $local_sw_ip $remote_ns1_ip $h1_ip $w2_ns1_ip $h1 \
+ "local->remote 1"
+ __ping_ipv6 $local_sw_ip $remote_ns2_ip $h1_ip $w2_ns2_ip $h1 \
+ "local->remote 2"
+}
+
+maybe_in_ns()
+{
+ echo ${1:+in_ns} $1
+}
+
+__flood_counter_add_del()
+{
+ local add_del=$1; shift
+ local dst_ip=$1; shift
+ local dev=$1; shift
+ local ns=$1; shift
+
+ # Putting the ICMP capture both to HW and to SW will end up
+ # double-counting the packets that are trapped to slow path, such as for
+ # the unicast test. Adding either skip_hw or skip_sw fixes this problem,
+ # but with skip_hw, the flooded packets are not counted at all, because
+ # those are dropped due to MAC address mismatch; and skip_sw is a no-go
+ # for veth-based topologies.
+ #
+ # So try to install with skip_sw and fall back to skip_sw if that fails.
+
+ $(maybe_in_ns $ns) tc filter $add_del dev "$dev" ingress \
+ proto ipv6 pref 100 flower dst_ip $dst_ip ip_proto \
+ icmpv6 skip_sw action pass 2>/dev/null || \
+ $(maybe_in_ns $ns) tc filter $add_del dev "$dev" ingress \
+ proto ipv6 pref 100 flower dst_ip $dst_ip ip_proto \
+ icmpv6 skip_hw action pass
+}
+
+flood_counter_install()
+{
+ __flood_counter_add_del add "$@"
+}
+
+flood_counter_uninstall()
+{
+ __flood_counter_add_del del "$@"
+}
+
+flood_fetch_stat()
+{
+ local dev=$1; shift
+ local ns=$1; shift
+
+ $(maybe_in_ns $ns) tc_rule_stats_get $dev 100 ingress
+}
+
+flood_fetch_stats()
+{
+ local counters=("${@}")
+ local counter
+
+ for counter in "${counters[@]}"; do
+ flood_fetch_stat $counter
+ done
+}
+
+vxlan_flood_test()
+{
+ local mac=$1; shift
+ local dst=$1; shift
+ local -a expects=("${@}")
+
+ local -a counters=($h2 "vx2 ns1" "vx2 ns2")
+ local counter
+ local key
+
+ for counter in "${counters[@]}"; do
+ flood_counter_install $dst $counter
+ done
+
+ local -a t0s=($(flood_fetch_stats "${counters[@]}"))
+ $MZ -6 $h1 -c 10 -d 100msec -p 64 -b $mac -B $dst -t icmp6 type=128 -q
+ sleep 1
+ local -a t1s=($(flood_fetch_stats "${counters[@]}"))
+
+ for key in ${!t0s[@]}; do
+ local delta=$((t1s[$key] - t0s[$key]))
+ local expect=${expects[$key]}
+
+ ((expect == delta))
+ check_err $? "${counters[$key]}: Expected to capture $expect packets, got $delta."
+ done
+
+ for counter in "${counters[@]}"; do
+ flood_counter_uninstall $dst $counter
+ done
+}
+
+__test_flood()
+{
+ local mac=$1; shift
+ local dst=$1; shift
+ local what=$1; shift
+
+ RET=0
+
+ vxlan_flood_test $mac $dst 10 10 10
+
+ log_test "VXLAN: $what"
+}
+
+test_flood()
+{
+ __test_flood de:ad:be:ef:13:37 2001:db8:1::100 "flood"
+}
+
+vxlan_fdb_add_del()
+{
+ local add_del=$1; shift
+ local mac=$1; shift
+ local dev=$1; shift
+ local dst=$1; shift
+
+ bridge fdb $add_del dev $dev $mac self static permanent \
+ ${dst:+dst} $dst 2>/dev/null
+ bridge fdb $add_del dev $dev $mac master static 2>/dev/null
+}
+
+__test_unicast()
+{
+ local mac=$1; shift
+ local dst=$1; shift
+ local hit_idx=$1; shift
+ local what=$1; shift
+
+ RET=0
+
+ local -a expects=(0 0 0)
+ expects[$hit_idx]=10
+
+ vxlan_flood_test $mac $dst "${expects[@]}"
+
+ log_test "VXLAN: $what"
+}
+
+test_unicast()
+{
+ local -a targets=("$h2_mac $h2"
+ "$r1_mac vx1 2001:db8:4::1"
+ "$r2_mac vx1 2001:db8:5::1")
+ local target
+
+ for target in "${targets[@]}"; do
+ vxlan_fdb_add_del add $target
+ done
+
+ __test_unicast $h2_mac 2001:db8:1::2 0 "local MAC unicast"
+ __test_unicast $r1_mac 2001:db8:1::3 1 "remote MAC 1 unicast"
+ __test_unicast $r2_mac 2001:db8:1::4 2 "remote MAC 2 unicast"
+
+ for target in "${targets[@]}"; do
+ vxlan_fdb_add_del del $target
+ done
+}
+
+vxlan_ping_test()
+{
+ local ping_dev=$1; shift
+ local ping_dip=$1; shift
+ local ping_args=$1; shift
+ local capture_dev=$1; shift
+ local capture_dir=$1; shift
+ local capture_pref=$1; shift
+ local expect=$1; shift
+
+ local t0=$(tc_rule_stats_get $capture_dev $capture_pref $capture_dir)
+ ping6_do $ping_dev $ping_dip "$ping_args"
+ local t1=$(tc_rule_stats_get $capture_dev $capture_pref $capture_dir)
+ local delta=$((t1 - t0))
+
+ # Tolerate a couple stray extra packets.
+ ((expect <= delta && delta <= expect + 2))
+ check_err $? "$capture_dev: Expected to capture $expect packets, got $delta."
+}
+
+test_ttl()
+{
+ RET=0
+
+ tc filter add dev v1 egress pref 77 protocol ipv6 \
+ flower ip_ttl 99 action pass
+ vxlan_ping_test $h1 2001:db8:1::3 "" v1 egress 77 10
+ tc filter del dev v1 egress pref 77 protocol ipv6
+
+ log_test "VXLAN: envelope TTL"
+}
+
+test_tos()
+{
+ RET=0
+
+ tc filter add dev v1 egress pref 77 protocol ipv6 \
+ flower ip_tos 0x14 action pass
+ vxlan_ping_test $h1 2001:db8:1::3 "-Q 0x14" v1 egress 77 10
+ vxlan_ping_test $h1 2001:db8:1::3 "-Q 0x18" v1 egress 77 0
+ tc filter del dev v1 egress pref 77 protocol ipv6
+
+ log_test "VXLAN: envelope TOS inheritance"
+}
+
+__test_ecn_encap()
+{
+ local q=$1; shift
+ local tos=$1; shift
+
+ RET=0
+
+ tc filter add dev v1 egress pref 77 protocol ipv6 \
+ flower ip_tos $tos action pass
+ sleep 1
+ vxlan_ping_test $h1 2001:db8:1::3 "-Q $q" v1 egress 77 10
+ tc filter del dev v1 egress pref 77 protocol ipv6
+
+ log_test "VXLAN: ECN encap: $q->$tos"
+}
+
+test_ecn_encap()
+{
+ # In accordance with INET_ECN_encapsulate()
+ __test_ecn_encap 0x00 0x00
+ __test_ecn_encap 0x01 0x01
+ __test_ecn_encap 0x02 0x02
+ __test_ecn_encap 0x03 0x02
+}
+
+vxlan_encapped_ping_do()
+{
+ local count=$1; shift
+ local dev=$1; shift
+ local next_hop_mac=$1; shift
+ local dest_ip=$1; shift
+ local dest_mac=$1; shift
+ local inner_tos=$1; shift
+ local outer_tos=$1; shift
+ local saddr="20:01:0d:b8:00:01:00:00:00:00:00:00:00:00:00:03"
+ local daddr="20:01:0d:b8:00:01:00:00:00:00:00:00:00:00:00:01"
+
+ $MZ -6 $dev -c $count -d 100msec -q \
+ -b $next_hop_mac -B $dest_ip \
+ -t udp tos=$outer_tos,sp=23456,dp=$VXPORT,p=$(:
+ )"08:"$( : VXLAN flags
+ )"00:00:00:"$( : VXLAN reserved
+ )"00:03:e8:"$( : VXLAN VNI
+ )"00:"$( : VXLAN reserved
+ )"$dest_mac:"$( : ETH daddr
+ )"$(mac_get w2):"$( : ETH saddr
+ )"86:dd:"$( : ETH type
+ )"6"$( : IP version
+ )"$inner_tos"$( : Traffic class
+ )"0:00:00:"$( : Flow label
+ )"00:08:"$( : Payload length
+ )"3a:"$( : Next header
+ )"04:"$( : Hop limit
+ )"$saddr:"$( : IP saddr
+ )"$daddr:"$( : IP daddr
+ )"80:"$( : ICMPv6.type
+ )"00:"$( : ICMPv6.code
+ )"00:"$( : ICMPv6.checksum
+ )
+}
+export -f vxlan_encapped_ping_do
+
+vxlan_encapped_ping_test()
+{
+ local ping_dev=$1; shift
+ local nh_dev=$1; shift
+ local ping_dip=$1; shift
+ local inner_tos=$1; shift
+ local outer_tos=$1; shift
+ local stat_get=$1; shift
+ local expect=$1; shift
+
+ local t0=$($stat_get)
+
+ in_ns ns1 \
+ vxlan_encapped_ping_do 10 $ping_dev $(mac_get $nh_dev) \
+ $ping_dip $(mac_get $h1) \
+ $inner_tos $outer_tos
+ sleep 1
+ local t1=$($stat_get)
+ local delta=$((t1 - t0))
+
+ # Tolerate a couple stray extra packets.
+ ((expect <= delta && delta <= expect + 2))
+ check_err $? "Expected to capture $expect packets, got $delta."
+}
+export -f vxlan_encapped_ping_test
+
+__test_ecn_decap()
+{
+ local orig_inner_tos=$1; shift
+ local orig_outer_tos=$1; shift
+ local decapped_tos=$1; shift
+
+ RET=0
+
+ tc filter add dev $h1 ingress pref 77 protocol ipv6 \
+ flower src_ip 2001:db8:1::3 dst_ip 2001:db8:1::1 \
+ ip_tos $decapped_tos action drop
+ sleep 1
+ vxlan_encapped_ping_test v2 v1 2001:db8:3::1 \
+ $orig_inner_tos $orig_outer_tos \
+ "tc_rule_stats_get $h1 77 ingress" 10
+ tc filter del dev $h1 ingress pref 77
+
+ log_test "VXLAN: ECN decap: $orig_outer_tos/$orig_inner_tos->$decapped_tos"
+}
+
+test_ecn_decap_error()
+{
+ local orig_inner_tos="0:0"
+ local orig_outer_tos=03
+
+ RET=0
+
+ vxlan_encapped_ping_test v2 v1 2001:db8:3::1 \
+ $orig_inner_tos $orig_outer_tos \
+ "link_stats_rx_errors_get vx1" 10
+
+ log_test "VXLAN: ECN decap: $orig_outer_tos/$orig_inner_tos->error"
+}
+
+test_ecn_decap()
+{
+ # In accordance with INET_ECN_decapsulate()
+ __test_ecn_decap "0:0" 00 0x00
+ __test_ecn_decap "0:0" 01 0x00
+ __test_ecn_decap "0:0" 02 0x00
+ # 00 03 is tested in test_ecn_decap_error()
+ __test_ecn_decap "0:1" 00 0x01
+ __test_ecn_decap "0:1" 01 0x01
+ __test_ecn_decap "0:1" 02 0x01
+ __test_ecn_decap "0:1" 03 0x03
+ __test_ecn_decap "0:2" 00 0x02
+ __test_ecn_decap "0:2" 01 0x01
+ __test_ecn_decap "0:2" 02 0x02
+ __test_ecn_decap "0:2" 03 0x03
+ __test_ecn_decap "0:3" 00 0x03
+ __test_ecn_decap "0:3" 01 0x03
+ __test_ecn_decap "0:3" 02 0x03
+ __test_ecn_decap "0:3" 03 0x03
+ test_ecn_decap_error
+}
+
+test_all()
+{
+ log_info "Running tests with UDP port $VXPORT"
+ tests_run
+}
+
+trap cleanup EXIT
+
+setup_prepare
+setup_wait
+test_all
+
+exit $EXIT_STATUS
diff --git a/tools/testing/selftests/net/forwarding/vxlan_bridge_1d_port_8472_ipv6.sh b/tools/testing/selftests/net/forwarding/vxlan_bridge_1d_port_8472_ipv6.sh
new file mode 100755
index 0000000..0054031
--- /dev/null
+++ b/tools/testing/selftests/net/forwarding/vxlan_bridge_1d_port_8472_ipv6.sh
@@ -0,0 +1,11 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+# A wrapper to run VXLAN tests with an unusual port number.
+
+VXPORT=8472
+ALL_TESTS="
+ ping_ipv4
+ ping_ipv6
+"
+source vxlan_bridge_1d_ipv6.sh
diff --git a/tools/testing/selftests/net/forwarding/vxlan_bridge_1q.sh b/tools/testing/selftests/net/forwarding/vxlan_bridge_1q.sh
index a578972..a596bbf 100755
--- a/tools/testing/selftests/net/forwarding/vxlan_bridge_1q.sh
+++ b/tools/testing/selftests/net/forwarding/vxlan_bridge_1q.sh
@@ -680,26 +680,6 @@
log_test "VXLAN: flood after vlan re-add"
}
-vxlan_ping_test()
-{
- local ping_dev=$1; shift
- local ping_dip=$1; shift
- local ping_args=$1; shift
- local capture_dev=$1; shift
- local capture_dir=$1; shift
- local capture_pref=$1; shift
- local expect=$1; shift
-
- local t0=$(tc_rule_stats_get $capture_dev $capture_pref $capture_dir)
- ping_do $ping_dev $ping_dip "$ping_args"
- local t1=$(tc_rule_stats_get $capture_dev $capture_pref $capture_dir)
- local delta=$((t1 - t0))
-
- # Tolerate a couple stray extra packets.
- ((expect <= delta && delta <= expect + 2))
- check_err $? "$capture_dev: Expected to capture $expect packets, got $delta."
-}
-
__test_learning()
{
local -a expects=(0 0 0 0 0)
diff --git a/tools/testing/selftests/net/forwarding/vxlan_bridge_1q_ipv6.sh b/tools/testing/selftests/net/forwarding/vxlan_bridge_1q_ipv6.sh
new file mode 100755
index 0000000..d880df8
--- /dev/null
+++ b/tools/testing/selftests/net/forwarding/vxlan_bridge_1q_ipv6.sh
@@ -0,0 +1,837 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+# +-----------------------+ +------------------------+
+# | H1 (vrf) | | H2 (vrf) |
+# | + $h1.10 | | + $h2.10 |
+# | | 192.0.2.1/28 | | | 192.0.2.2/28 |
+# | | 2001:db8:1::1/64 | | | 2001:db8:1::2/64 |
+# | | | | | |
+# | | + $h1.20 | | | + $h2.20 |
+# | \ | 198.51.100.1/24 | | \ | 198.51.100.2/24 |
+# | \ | 2001:db8:2::1/64 | | \ | 2001:db8:2::2/64 |
+# | \| | | \| |
+# | + $h1 | | + $h2 |
+# +----|------------------+ +----|-------------------+
+# | |
+# +----|--------------------------------------------------|-------------------+
+# | SW | | |
+# | +--|--------------------------------------------------|-----------------+ |
+# | | + $swp1 BR1 (802.1q) + $swp2 | |
+# | | vid 10 vid 10 | |
+# | | vid 20 vid 20 | |
+# | | | |
+# | | + vx10 (vxlan) + vx20 (vxlan) | |
+# | | local: local: | |
+# | | 2001:db8:3::1 2001:db8:3::1 | |
+# | | remote: remote: | |
+# | | 2001:db8:4::1 2001:db8:5::1 2001:db8:4::1 2001:db8:5::1 | |
+# | | id 1000 dstport $VXPORT id 2000 dstport $VXPORT | |
+# | | vid 10 pvid untagged vid 20 pvid untagged | |
+# | +-----------------------------------------------------------------------+ |
+# | |
+# | 2001:db8:4::0/64 via 2001:db8:3::2 |
+# | 2001:db8:5::0/64 via 2001:db8:3::2 |
+# | |
+# | + $rp1 |
+# | | 2001:db8:3::1/64 |
+# +----|----------------------------------------------------------------------+
+# |
+# +----|----------------------------------------------------------+
+# | | VRP2 (vrf) |
+# | + $rp2 |
+# | 2001:db8:3::2/64 |
+# | | (maybe) HW
+# =============================================================================
+# | | (likely) SW
+# | + v1 (veth) + v3 (veth) |
+# | | 2001:db8:4::2/64 | 2001:db8:5::2/64 |
+# +----|---------------------------------------|------------------+
+# | |
+# +----|--------------------------------+ +----|-------------------------------+
+# | + v2 (veth) NS1 (netns) | | + v4 (veth) NS2 (netns) |
+# | 2001:db8:4::1/64 | | 2001:db8:5::1/64 |
+# | | | |
+# | 2001:db8:3::0/64 via 2001:db8:4::2 | | 2001:db8:3::0/64 via 2001:db8:5::2 |
+# | 2001:db8:5::1/128 via 2001:db8:4::2 | | 2001:db8:4::1/128 via |
+# | | | 2001:db8:5::2 |
+# | | | |
+# | +-------------------------------+ | | +-------------------------------+ |
+# | | BR2 (802.1q) | | | | BR2 (802.1q) | |
+# | | + vx10 (vxlan) | | | | + vx10 (vxlan) | |
+# | | local 2001:db8:4::1 | | | | local 2001:db8:5::1 | |
+# | | remote 2001:db8:3::1 | | | | remote 2001:db8:3::1 | |
+# | | remote 2001:db8:5::1 | | | | remote 2001:db8:4::1 | |
+# | | id 1000 dstport $VXPORT | | | | id 1000 dstport $VXPORT | |
+# | | vid 10 pvid untagged | | | | vid 10 pvid untagged | |
+# | | | | | | | |
+# | | + vx20 (vxlan) | | | | + vx20 (vxlan) | |
+# | | local 2001:db8:4::1 | | | | local 2001:db8:5::1 | |
+# | | remote 2001:db8:3::1 | | | | remote 2001:db8:3::1 | |
+# | | remote 2001:db8:5::1 | | | | remote 2001:db8:4::1 | |
+# | | id 2000 dstport $VXPORT | | | | id 2000 dstport $VXPORT | |
+# | | vid 20 pvid untagged | | | | vid 20 pvid untagged | |
+# | | | | | | | |
+# | | + w1 (veth) | | | | + w1 (veth) | |
+# | | | vid 10 | | | | | vid 10 | |
+# | | | vid 20 | | | | | vid 20 | |
+# | +--|----------------------------+ | | +--|----------------------------+ |
+# | | | | | |
+# | +--|----------------------------+ | | +--|----------------------------+ |
+# | | + w2 (veth) VW2 (vrf) | | | | + w2 (veth) VW2 (vrf) | |
+# | | |\ | | | | |\ | |
+# | | | + w2.10 | | | | | + w2.10 | |
+# | | | 192.0.2.3/28 | | | | | 192.0.2.4/28 | |
+# | | | 2001:db8:1::3/64 | | | | | 2001:db8:1::4/64 | |
+# | | | | | | | | | |
+# | | + w2.20 | | | | + w2.20 | |
+# | | 198.51.100.3/24 | | | | 198.51.100.4/24 | |
+# | | 2001:db8:2::3/64 | | | | 2001:db8:2::4/64 | |
+# | +-------------------------------+ | | +-------------------------------+ |
+# +-------------------------------------+ +------------------------------------+
+
+: ${VXPORT:=4789}
+export VXPORT
+
+: ${ALL_TESTS:="
+ ping_ipv4
+ ping_ipv6
+ test_flood
+ test_unicast
+ reapply_config
+ ping_ipv4
+ ping_ipv6
+ test_flood
+ test_unicast
+ test_pvid
+ ping_ipv4
+ ping_ipv6
+ test_flood
+ test_pvid
+"}
+
+NUM_NETIFS=6
+source lib.sh
+source tc_common.sh
+
+h1_create()
+{
+ simple_if_init $h1
+ tc qdisc add dev $h1 clsact
+ vlan_create $h1 10 v$h1 192.0.2.1/28 2001:db8:1::1/64
+ vlan_create $h1 20 v$h1 198.51.100.1/24 2001:db8:2::1/64
+}
+
+h1_destroy()
+{
+ vlan_destroy $h1 20
+ vlan_destroy $h1 10
+ tc qdisc del dev $h1 clsact
+ simple_if_fini $h1
+}
+
+h2_create()
+{
+ simple_if_init $h2
+ tc qdisc add dev $h2 clsact
+ vlan_create $h2 10 v$h2 192.0.2.2/28 2001:db8:1::2/64
+ vlan_create $h2 20 v$h2 198.51.100.2/24 2001:db8:2::2/64
+}
+
+h2_destroy()
+{
+ vlan_destroy $h2 20
+ vlan_destroy $h2 10
+ tc qdisc del dev $h2 clsact
+ simple_if_fini $h2
+}
+
+rp1_set_addr()
+{
+ ip address add dev $rp1 2001:db8:3::1/64
+
+ ip route add 2001:db8:4::0/64 nexthop via 2001:db8:3::2
+ ip route add 2001:db8:5::0/64 nexthop via 2001:db8:3::2
+}
+
+rp1_unset_addr()
+{
+ ip route del 2001:db8:5::0/64 nexthop via 2001:db8:3::2
+ ip route del 2001:db8:4::0/64 nexthop via 2001:db8:3::2
+
+ ip address del dev $rp1 2001:db8:3::1/64
+}
+
+switch_create()
+{
+ ip link add name br1 type bridge vlan_filtering 1 vlan_default_pvid 0 \
+ mcast_snooping 0
+ # Make sure the bridge uses the MAC address of the local port and not
+ # that of the VxLAN's device.
+ ip link set dev br1 address $(mac_get $swp1)
+ ip link set dev br1 up
+
+ ip link set dev $rp1 up
+ rp1_set_addr
+ tc qdisc add dev $rp1 clsact
+
+ ip link add name vx10 type vxlan id 1000 local 2001:db8:3::1 \
+ dstport "$VXPORT" nolearning udp6zerocsumrx udp6zerocsumtx \
+ tos inherit ttl 100
+ ip link set dev vx10 up
+
+ ip link set dev vx10 master br1
+ bridge vlan add vid 10 dev vx10 pvid untagged
+
+ ip link add name vx20 type vxlan id 2000 local 2001:db8:3::1 \
+ dstport "$VXPORT" nolearning udp6zerocsumrx udp6zerocsumtx \
+ tos inherit ttl 100
+ ip link set dev vx20 up
+
+ ip link set dev vx20 master br1
+ bridge vlan add vid 20 dev vx20 pvid untagged
+
+ ip link set dev $swp1 master br1
+ ip link set dev $swp1 up
+ tc qdisc add dev $swp1 clsact
+ bridge vlan add vid 10 dev $swp1
+ bridge vlan add vid 20 dev $swp1
+
+ ip link set dev $swp2 master br1
+ ip link set dev $swp2 up
+ bridge vlan add vid 10 dev $swp2
+ bridge vlan add vid 20 dev $swp2
+
+ bridge fdb append dev vx10 00:00:00:00:00:00 dst 2001:db8:4::1 self
+ bridge fdb append dev vx10 00:00:00:00:00:00 dst 2001:db8:5::1 self
+
+ bridge fdb append dev vx20 00:00:00:00:00:00 dst 2001:db8:4::1 self
+ bridge fdb append dev vx20 00:00:00:00:00:00 dst 2001:db8:5::1 self
+}
+
+switch_destroy()
+{
+ bridge fdb del dev vx20 00:00:00:00:00:00 dst 2001:db8:5::1 self
+ bridge fdb del dev vx20 00:00:00:00:00:00 dst 2001:db8:4::1 self
+
+ bridge fdb del dev vx10 00:00:00:00:00:00 dst 2001:db8:5::1 self
+ bridge fdb del dev vx10 00:00:00:00:00:00 dst 2001:db8:4::1 self
+
+ bridge vlan del vid 20 dev $swp2
+ bridge vlan del vid 10 dev $swp2
+ ip link set dev $swp2 down
+ ip link set dev $swp2 nomaster
+
+ bridge vlan del vid 20 dev $swp1
+ bridge vlan del vid 10 dev $swp1
+ tc qdisc del dev $swp1 clsact
+ ip link set dev $swp1 down
+ ip link set dev $swp1 nomaster
+
+ bridge vlan del vid 20 dev vx20
+ ip link set dev vx20 nomaster
+
+ ip link set dev vx20 down
+ ip link del dev vx20
+
+ bridge vlan del vid 10 dev vx10
+ ip link set dev vx10 nomaster
+
+ ip link set dev vx10 down
+ ip link del dev vx10
+
+ tc qdisc del dev $rp1 clsact
+ rp1_unset_addr
+ ip link set dev $rp1 down
+
+ ip link set dev br1 down
+ ip link del dev br1
+}
+
+vrp2_create()
+{
+ simple_if_init $rp2 2001:db8:3::2/64
+ __simple_if_init v1 v$rp2 2001:db8:4::2/64
+ __simple_if_init v3 v$rp2 2001:db8:5::2/64
+ tc qdisc add dev v1 clsact
+}
+
+vrp2_destroy()
+{
+ tc qdisc del dev v1 clsact
+ __simple_if_fini v3 2001:db8:5::2/64
+ __simple_if_fini v1 2001:db8:4::2/64
+ simple_if_fini $rp2 2001:db8:3::2/64
+}
+
+ns_init_common()
+{
+ local in_if=$1; shift
+ local in_addr=$1; shift
+ local other_in_addr=$1; shift
+ local nh_addr=$1; shift
+ local host_addr1_ipv4=$1; shift
+ local host_addr1_ipv6=$1; shift
+ local host_addr2_ipv4=$1; shift
+ local host_addr2_ipv6=$1; shift
+
+ ip link set dev $in_if up
+ ip address add dev $in_if $in_addr/64
+ tc qdisc add dev $in_if clsact
+
+ ip link add name br2 type bridge vlan_filtering 1 vlan_default_pvid 0
+ ip link set dev br2 up
+
+ ip link add name w1 type veth peer name w2
+
+ ip link set dev w1 master br2
+ ip link set dev w1 up
+
+ bridge vlan add vid 10 dev w1
+ bridge vlan add vid 20 dev w1
+
+ ip link add name vx10 type vxlan id 1000 local $in_addr \
+ dstport "$VXPORT" udp6zerocsumrx
+ ip link set dev vx10 up
+ bridge fdb append dev vx10 00:00:00:00:00:00 dst 2001:db8:3::1 self
+ bridge fdb append dev vx10 00:00:00:00:00:00 dst $other_in_addr self
+
+ ip link set dev vx10 master br2
+ tc qdisc add dev vx10 clsact
+
+ bridge vlan add vid 10 dev vx10 pvid untagged
+
+ ip link add name vx20 type vxlan id 2000 local $in_addr \
+ dstport "$VXPORT" udp6zerocsumrx
+ ip link set dev vx20 up
+ bridge fdb append dev vx20 00:00:00:00:00:00 dst 2001:db8:3::1 self
+ bridge fdb append dev vx20 00:00:00:00:00:00 dst $other_in_addr self
+
+ ip link set dev vx20 master br2
+ tc qdisc add dev vx20 clsact
+
+ bridge vlan add vid 20 dev vx20 pvid untagged
+
+ simple_if_init w2
+ vlan_create w2 10 vw2 $host_addr1_ipv4/28 $host_addr1_ipv6/64
+ vlan_create w2 20 vw2 $host_addr2_ipv4/24 $host_addr2_ipv6/64
+
+ ip route add 2001:db8:3::0/64 nexthop via $nh_addr
+ ip route add $other_in_addr/128 nexthop via $nh_addr
+}
+export -f ns_init_common
+
+ns1_create()
+{
+ ip netns add ns1
+ ip link set dev v2 netns ns1
+ in_ns ns1 \
+ ns_init_common v2 2001:db8:4::1 2001:db8:5::1 2001:db8:4::2 \
+ 192.0.2.3 2001:db8:1::3 198.51.100.3 2001:db8:2::3
+}
+
+ns1_destroy()
+{
+ ip netns exec ns1 ip link set dev v2 netns 1
+ ip netns del ns1
+}
+
+ns2_create()
+{
+ ip netns add ns2
+ ip link set dev v4 netns ns2
+ in_ns ns2 \
+ ns_init_common v4 2001:db8:5::1 2001:db8:4::1 2001:db8:5::2 \
+ 192.0.2.4 2001:db8:1::4 198.51.100.4 2001:db8:2::4
+}
+
+ns2_destroy()
+{
+ ip netns exec ns2 ip link set dev v4 netns 1
+ ip netns del ns2
+}
+
+setup_prepare()
+{
+ h1=${NETIFS[p1]}
+ swp1=${NETIFS[p2]}
+
+ swp2=${NETIFS[p3]}
+ h2=${NETIFS[p4]}
+
+ rp1=${NETIFS[p5]}
+ rp2=${NETIFS[p6]}
+
+ vrf_prepare
+ forwarding_enable
+
+ h1_create
+ h2_create
+ switch_create
+
+ ip link add name v1 type veth peer name v2
+ ip link add name v3 type veth peer name v4
+ vrp2_create
+ ns1_create
+ ns2_create
+
+ r1_mac=$(in_ns ns1 mac_get w2)
+ r2_mac=$(in_ns ns2 mac_get w2)
+ h2_mac=$(mac_get $h2)
+}
+
+cleanup()
+{
+ pre_cleanup
+
+ ns2_destroy
+ ns1_destroy
+ vrp2_destroy
+ ip link del dev v3
+ ip link del dev v1
+
+ switch_destroy
+ h2_destroy
+ h1_destroy
+
+ forwarding_restore
+ vrf_cleanup
+}
+
+# For the first round of tests, vx10 and vx20 were the first devices to get
+# attached to the bridge, and at that point the local IP is already
+# configured. Try the other scenario of attaching these devices to a bridge
+# that already has local ports members, and only then assign the local IP.
+reapply_config()
+{
+ log_info "Reapplying configuration"
+
+ bridge fdb del dev vx20 00:00:00:00:00:00 dst 2001:db8:5::1 self
+ bridge fdb del dev vx20 00:00:00:00:00:00 dst 2001:db8:4::1 self
+
+ bridge fdb del dev vx10 00:00:00:00:00:00 dst 2001:db8:5::1 self
+ bridge fdb del dev vx10 00:00:00:00:00:00 dst 2001:db8:4::1 self
+
+ ip link set dev vx20 nomaster
+ ip link set dev vx10 nomaster
+
+ rp1_unset_addr
+ sleep 5
+
+ ip link set dev vx10 master br1
+ bridge vlan add vid 10 dev vx10 pvid untagged
+
+ ip link set dev vx20 master br1
+ bridge vlan add vid 20 dev vx20 pvid untagged
+
+ bridge fdb append dev vx10 00:00:00:00:00:00 dst 2001:db8:4::1 self
+ bridge fdb append dev vx10 00:00:00:00:00:00 dst 2001:db8:5::1 self
+
+ bridge fdb append dev vx20 00:00:00:00:00:00 dst 2001:db8:4::1 self
+ bridge fdb append dev vx20 00:00:00:00:00:00 dst 2001:db8:5::1 self
+
+ rp1_set_addr
+ sleep 5
+}
+
+__ping_ipv4()
+{
+ local vxlan_local_ip=$1; shift
+ local vxlan_remote_ip=$1; shift
+ local src_ip=$1; shift
+ local dst_ip=$1; shift
+ local dev=$1; shift
+ local info=$1; shift
+
+ RET=0
+
+ tc filter add dev $rp1 egress protocol ipv6 pref 1 handle 101 \
+ flower ip_proto udp src_ip $vxlan_local_ip \
+ dst_ip $vxlan_remote_ip dst_port $VXPORT $TC_FLAG action pass
+ # Match ICMP-reply packets after decapsulation, so source IP is
+ # destination IP of the ping and destination IP is source IP of the
+ # ping.
+ tc filter add dev $swp1 egress protocol 802.1q pref 1 handle 101 \
+ flower vlan_ethtype ipv4 src_ip $dst_ip dst_ip $src_ip \
+ $TC_FLAG action pass
+
+ # Send 100 packets and verify that at least 100 packets hit the rule,
+ # to overcome ARP noise.
+ PING_COUNT=100 PING_TIMEOUT=11 ping_do $dev $dst_ip
+ check_err $? "Ping failed"
+
+ tc_check_at_least_x_packets "dev $rp1 egress" 101 10 100
+ check_err $? "Encapsulated packets did not go through router"
+
+ tc_check_at_least_x_packets "dev $swp1 egress" 101 10 100
+ check_err $? "Decapsulated packets did not go through switch"
+
+ log_test "ping: $info"
+
+ tc filter del dev $swp1 egress
+ tc filter del dev $rp1 egress
+}
+
+ping_ipv4()
+{
+ RET=0
+
+ local local_sw_ip=2001:db8:3::1
+ local remote_ns1_ip=2001:db8:4::1
+ local remote_ns2_ip=2001:db8:5::1
+ local h1_10_ip=192.0.2.1
+ local h1_20_ip=198.51.100.1
+ local w2_10_ns1_ip=192.0.2.3
+ local w2_10_ns2_ip=192.0.2.4
+ local w2_20_ns1_ip=198.51.100.3
+ local w2_20_ns2_ip=198.51.100.4
+
+ ping_test $h1.10 192.0.2.2 ": local->local vid 10"
+ ping_test $h1.20 198.51.100.2 ": local->local vid 20"
+
+ __ping_ipv4 $local_sw_ip $remote_ns1_ip $h1_10_ip $w2_10_ns1_ip $h1.10 \
+ "local->remote 1 vid 10"
+ __ping_ipv4 $local_sw_ip $remote_ns2_ip $h1_10_ip $w2_10_ns2_ip $h1.10 \
+ "local->remote 2 vid 10"
+ __ping_ipv4 $local_sw_ip $remote_ns1_ip $h1_20_ip $w2_20_ns1_ip $h1.20 \
+ "local->remote 1 vid 20"
+ __ping_ipv4 $local_sw_ip $remote_ns2_ip $h1_20_ip $w2_20_ns2_ip $h1.20 \
+ "local->remote 2 vid 20"
+}
+
+__ping_ipv6()
+{
+ local vxlan_local_ip=$1; shift
+ local vxlan_remote_ip=$1; shift
+ local src_ip=$1; shift
+ local dst_ip=$1; shift
+ local dev=$1; shift
+ local info=$1; shift
+
+ RET=0
+
+ tc filter add dev $rp1 egress protocol ipv6 pref 1 handle 101 \
+ flower ip_proto udp src_ip $vxlan_local_ip \
+ dst_ip $vxlan_remote_ip dst_port $VXPORT $TC_FLAG action pass
+ # Match ICMP-reply packets after decapsulation, so source IP is
+ # destination IP of the ping and destination IP is source IP of the
+ # ping.
+ tc filter add dev $swp1 egress protocol 802.1q pref 1 handle 101 \
+ flower vlan_ethtype ipv6 src_ip $dst_ip dst_ip $src_ip \
+ $TC_FLAG action pass
+
+ # Send 100 packets and verify that at least 100 packets hit the rule,
+ # to overcome neighbor discovery noise.
+ PING_COUNT=100 PING_TIMEOUT=11 ping6_do $dev $dst_ip
+ check_err $? "Ping failed"
+
+ tc_check_at_least_x_packets "dev $rp1 egress" 101 100
+ check_err $? "Encapsulated packets did not go through router"
+
+ tc_check_at_least_x_packets "dev $swp1 egress" 101 100
+ check_err $? "Decapsulated packets did not go through switch"
+
+ log_test "ping6: $info"
+
+ tc filter del dev $swp1 egress
+ tc filter del dev $rp1 egress
+}
+
+ping_ipv6()
+{
+ RET=0
+
+ local local_sw_ip=2001:db8:3::1
+ local remote_ns1_ip=2001:db8:4::1
+ local remote_ns2_ip=2001:db8:5::1
+ local h1_10_ip=2001:db8:1::1
+ local h1_20_ip=2001:db8:2::1
+ local w2_10_ns1_ip=2001:db8:1::3
+ local w2_10_ns2_ip=2001:db8:1::4
+ local w2_20_ns1_ip=2001:db8:2::3
+ local w2_20_ns2_ip=2001:db8:2::4
+
+ ping6_test $h1.10 2001:db8:1::2 ": local->local vid 10"
+ ping6_test $h1.20 2001:db8:2::2 ": local->local vid 20"
+
+ __ping_ipv6 $local_sw_ip $remote_ns1_ip $h1_10_ip $w2_10_ns1_ip $h1.10 \
+ "local->remote 1 vid 10"
+ __ping_ipv6 $local_sw_ip $remote_ns2_ip $h1_10_ip $w2_10_ns2_ip $h1.10 \
+ "local->remote 2 vid 10"
+ __ping_ipv6 $local_sw_ip $remote_ns1_ip $h1_20_ip $w2_20_ns1_ip $h1.20 \
+ "local->remote 1 vid 20"
+ __ping_ipv6 $local_sw_ip $remote_ns2_ip $h1_20_ip $w2_20_ns2_ip $h1.20 \
+ "local->remote 2 vid 20"
+}
+
+maybe_in_ns()
+{
+ echo ${1:+in_ns} $1
+}
+
+__flood_counter_add_del()
+{
+ local add_del=$1; shift
+ local dst_ip=$1; shift
+ local dev=$1; shift
+ local ns=$1; shift
+
+ # Putting the ICMP capture both to HW and to SW will end up
+ # double-counting the packets that are trapped to slow path, such as for
+ # the unicast test. Adding either skip_hw or skip_sw fixes this problem,
+ # but with skip_hw, the flooded packets are not counted at all, because
+ # those are dropped due to MAC address mismatch; and skip_sw is a no-go
+ # for veth-based topologies.
+ #
+ # So try to install with skip_sw and fall back to skip_sw if that fails.
+
+ $(maybe_in_ns $ns) tc filter $add_del dev "$dev" ingress \
+ proto ipv6 pref 100 flower dst_ip $dst_ip ip_proto \
+ icmpv6 skip_sw action pass 2>/dev/null || \
+ $(maybe_in_ns $ns) tc filter $add_del dev "$dev" ingress \
+ proto ipv6 pref 100 flower dst_ip $dst_ip ip_proto \
+ icmpv6 skip_hw action pass
+}
+
+flood_counter_install()
+{
+ __flood_counter_add_del add "$@"
+}
+
+flood_counter_uninstall()
+{
+ __flood_counter_add_del del "$@"
+}
+
+flood_fetch_stat()
+{
+ local dev=$1; shift
+ local ns=$1; shift
+
+ $(maybe_in_ns $ns) tc_rule_stats_get $dev 100 ingress
+}
+
+flood_fetch_stats()
+{
+ local counters=("${@}")
+ local counter
+
+ for counter in "${counters[@]}"; do
+ flood_fetch_stat $counter
+ done
+}
+
+vxlan_flood_test()
+{
+ local mac=$1; shift
+ local dst=$1; shift
+ local vid=$1; shift
+ local -a expects=("${@}")
+
+ local -a counters=($h2 "vx10 ns1" "vx20 ns1" "vx10 ns2" "vx20 ns2")
+ local counter
+ local key
+
+ # Packets reach the local host tagged whereas they reach the VxLAN
+ # devices untagged. In order to be able to use the same filter for
+ # all counters, make sure the packets also reach the local host
+ # untagged
+ bridge vlan add vid $vid dev $swp2 untagged
+ for counter in "${counters[@]}"; do
+ flood_counter_install $dst $counter
+ done
+
+ local -a t0s=($(flood_fetch_stats "${counters[@]}"))
+ $MZ -6 $h1 -Q $vid -c 10 -d 100msec -p 64 -b $mac -B $dst -t icmp6 type=128 -q
+ sleep 1
+ local -a t1s=($(flood_fetch_stats "${counters[@]}"))
+
+ for key in ${!t0s[@]}; do
+ local delta=$((t1s[$key] - t0s[$key]))
+ local expect=${expects[$key]}
+
+ ((expect == delta))
+ check_err $? "${counters[$key]}: Expected to capture $expect packets, got $delta."
+ done
+
+ for counter in "${counters[@]}"; do
+ flood_counter_uninstall $dst $counter
+ done
+ bridge vlan add vid $vid dev $swp2
+}
+
+__test_flood()
+{
+ local mac=$1; shift
+ local dst=$1; shift
+ local vid=$1; shift
+ local what=$1; shift
+ local -a expects=("${@}")
+
+ RET=0
+
+ vxlan_flood_test $mac $dst $vid "${expects[@]}"
+
+ log_test "VXLAN: $what"
+}
+
+test_flood()
+{
+ __test_flood de:ad:be:ef:13:37 2001:db8:1::100 10 "flood vlan 10" \
+ 10 10 0 10 0
+ __test_flood ca:fe:be:ef:13:37 2001:db8:2::100 20 "flood vlan 20" \
+ 10 0 10 0 10
+}
+
+vxlan_fdb_add_del()
+{
+ local add_del=$1; shift
+ local vid=$1; shift
+ local mac=$1; shift
+ local dev=$1; shift
+ local dst=$1; shift
+
+ bridge fdb $add_del dev $dev $mac self static permanent \
+ ${dst:+dst} $dst 2>/dev/null
+ bridge fdb $add_del dev $dev $mac master static vlan $vid 2>/dev/null
+}
+
+__test_unicast()
+{
+ local mac=$1; shift
+ local dst=$1; shift
+ local hit_idx=$1; shift
+ local vid=$1; shift
+ local what=$1; shift
+
+ RET=0
+
+ local -a expects=(0 0 0 0 0)
+ expects[$hit_idx]=10
+
+ vxlan_flood_test $mac $dst $vid "${expects[@]}"
+
+ log_test "VXLAN: $what"
+}
+
+test_unicast()
+{
+ local -a targets=("$h2_mac $h2"
+ "$r1_mac vx10 2001:db8:4::1"
+ "$r2_mac vx10 2001:db8:5::1")
+ local target
+
+ log_info "unicast vlan 10"
+
+ for target in "${targets[@]}"; do
+ vxlan_fdb_add_del add 10 $target
+ done
+
+ __test_unicast $h2_mac 2001:db8:1::2 0 10 "local MAC unicast"
+ __test_unicast $r1_mac 2001:db8:1::3 1 10 "remote MAC 1 unicast"
+ __test_unicast $r2_mac 2001:db8:1::4 3 10 "remote MAC 2 unicast"
+
+ for target in "${targets[@]}"; do
+ vxlan_fdb_add_del del 10 $target
+ done
+
+ log_info "unicast vlan 20"
+
+ targets=("$h2_mac $h2" "$r1_mac vx20 2001:db8:4::1" \
+ "$r2_mac vx20 2001:db8:5::1")
+
+ for target in "${targets[@]}"; do
+ vxlan_fdb_add_del add 20 $target
+ done
+
+ __test_unicast $h2_mac 2001:db8:2::2 0 20 "local MAC unicast"
+ __test_unicast $r1_mac 2001:db8:2::3 2 20 "remote MAC 1 unicast"
+ __test_unicast $r2_mac 2001:db8:2::4 4 20 "remote MAC 2 unicast"
+
+ for target in "${targets[@]}"; do
+ vxlan_fdb_add_del del 20 $target
+ done
+}
+
+test_pvid()
+{
+ local -a expects=(0 0 0 0 0)
+ local mac=de:ad:be:ef:13:37
+ local dst=2001:db8:1::100
+ local vid=10
+
+ # Check that flooding works
+ RET=0
+
+ expects[0]=10; expects[1]=10; expects[3]=10
+ vxlan_flood_test $mac $dst $vid "${expects[@]}"
+
+ log_test "VXLAN: flood before pvid off"
+
+ # Toggle PVID off and test that flood to remote hosts does not work
+ RET=0
+
+ bridge vlan add vid 10 dev vx10
+
+ expects[0]=10; expects[1]=0; expects[3]=0
+ vxlan_flood_test $mac $dst $vid "${expects[@]}"
+
+ log_test "VXLAN: flood after pvid off"
+
+ # Toggle PVID on and test that flood to remote hosts does work
+ RET=0
+
+ bridge vlan add vid 10 dev vx10 pvid untagged
+
+ expects[0]=10; expects[1]=10; expects[3]=10
+ vxlan_flood_test $mac $dst $vid "${expects[@]}"
+
+ log_test "VXLAN: flood after pvid on"
+
+ # Add a new VLAN and test that it does not affect flooding
+ RET=0
+
+ bridge vlan add vid 30 dev vx10
+
+ expects[0]=10; expects[1]=10; expects[3]=10
+ vxlan_flood_test $mac $dst $vid "${expects[@]}"
+
+ bridge vlan del vid 30 dev vx10
+
+ log_test "VXLAN: flood after vlan add"
+
+ # Remove currently mapped VLAN and test that flood to remote hosts does
+ # not work
+ RET=0
+
+ bridge vlan del vid 10 dev vx10
+
+ expects[0]=10; expects[1]=0; expects[3]=0
+ vxlan_flood_test $mac $dst $vid "${expects[@]}"
+
+ log_test "VXLAN: flood after vlan delete"
+
+ # Re-add the VLAN and test that flood to remote hosts does work
+ RET=0
+
+ bridge vlan add vid 10 dev vx10 pvid untagged
+
+ expects[0]=10; expects[1]=10; expects[3]=10
+ vxlan_flood_test $mac $dst $vid "${expects[@]}"
+
+ log_test "VXLAN: flood after vlan re-add"
+}
+
+test_all()
+{
+ log_info "Running tests with UDP port $VXPORT"
+ tests_run
+}
+
+trap cleanup EXIT
+
+setup_prepare
+setup_wait
+test_all
+
+exit $EXIT_STATUS
diff --git a/tools/testing/selftests/net/forwarding/vxlan_bridge_1q_port_8472_ipv6.sh b/tools/testing/selftests/net/forwarding/vxlan_bridge_1q_port_8472_ipv6.sh
new file mode 100755
index 0000000..344f43cc
--- /dev/null
+++ b/tools/testing/selftests/net/forwarding/vxlan_bridge_1q_port_8472_ipv6.sh
@@ -0,0 +1,11 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+# A wrapper to run VXLAN tests with an unusual port number.
+
+VXPORT=8472
+ALL_TESTS="
+ ping_ipv4
+ ping_ipv6
+"
+source vxlan_bridge_1q_ipv6.sh
diff --git a/tools/testing/selftests/net/forwarding/vxlan_symmetric_ipv6.sh b/tools/testing/selftests/net/forwarding/vxlan_symmetric_ipv6.sh
new file mode 100755
index 0000000..9046334
--- /dev/null
+++ b/tools/testing/selftests/net/forwarding/vxlan_symmetric_ipv6.sh
@@ -0,0 +1,563 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+
+# +--------------------------------+ +-----------------------------+
+# | vrf-h1 | | vrf-h2 |
+# | + $h1 | | + $h2 |
+# | | 2001:db8:1::1/64 | | | 2001:db8:2::1/64 |
+# | | default via 2001:db8:1::3 | | | default via 2001:db8:2::3 |
+# +----|---------------------------+ +-|---------------------------+
+# | |
+# +----|------------------------------------------|---------------------------+
+# | SW | | |
+# | +--|------------------------------------------|-------------------------+ |
+# | | + $swp1 br1 + $swp2 | |
+# | | vid 10 pvid untagged vid 20 pvid untagged | |
+# | | | |
+# | | + vx10 + vx20 | |
+# | | local 2001:db8:3::1 local 2001:db8:3::1 | |
+# | | remote 2001:db8:3::2 remote 2001:db8:3::2 | |
+# | | id 1010 id 1020 | |
+# | | dstport 4789 dstport 4789 | |
+# | | vid 10 pvid untagged vid 20 pvid untagged | |
+# | | | |
+# | | + vx4001 | |
+# | | local 2001:db8:3::1 | |
+# | | remote 2001:db8:3::2 | |
+# | | id 104001 | |
+# | | dstport 4789 | |
+# | | vid 4001 pvid untagged | |
+# | | | |
+# | +-----------------------------------+-----------------------------------+ |
+# | | |
+# | +-----------------------------------|-----------------------------------+ |
+# | | | | |
+# | | +--------------------------------+--------------------------------+ | |
+# | | | | | | |
+# | | + vlan10 | vlan20 + | |
+# | | | 2001:db8:1::2/64 | 2001:db8:2::2/64 | | |
+# | | | | | | |
+# | | + vlan10-v (macvlan) + vlan20-v (macvlan) + | |
+# | | 2001:db8:1::3/64 vlan4001 2001:db8:2::3/64 | |
+# | | 00:00:5e:00:01:01 00:00:5e:00:01:01 | |
+# | | vrf-green | |
+# | +-----------------------------------------------------------------------+ |
+# | |
+# | + $rp1 +lo |
+# | | 2001:db8:4::1/64 2001:db8:3::1 |
+# +----|----------------------------------------------------------------------+
+# |
+# +----|--------------------------------------------------------+
+# | | vrf-spine |
+# | + $rp2 |
+# | 2001:db8:4::2/64 |
+# | | (maybe) HW
+# =============================================================================
+# | | (likely) SW
+# | |
+# | + v1 (veth) |
+# | | 2001:db8:5::2/64 |
+# +----|--------------------------------------------------------+
+# |
+# +----|----------------------------------------------------------------------+
+# | + v2 (veth) +lo NS1 (netns) |
+# | 2001:db8:5::1/64 2001:db8:3::2/128 |
+# | |
+# | +-----------------------------------------------------------------------+ |
+# | | vrf-green | |
+# | | + vlan10-v (macvlan) vlan20-v (macvlan) + | |
+# | | | 2001:db8:1::3/64 2001:db8:2::3/64 | | |
+# | | | 00:00:5e:00:01:01 00:00:5e:00:01:01 | | |
+# | | | vlan4001 | | |
+# | | + vlan10 + vlan20 + | |
+# | | | 2001:db8:1::3/64 | 2001:db8:2::3/64 | | |
+# | | | | | | |
+# | | +--------------------------------+--------------------------------+ | |
+# | | | | |
+# | +-----------------------------------|-----------------------------------+ |
+# | | |
+# | +-----------------------------------+-----------------------------------+ |
+# | | | |
+# | | + vx10 + vx20 | |
+# | | local 2001:db8:3::2 local 2001:db8:3::2 | |
+# | | remote 2001:db8:3::1 remote 2001:db8:3::1 | |
+# | | id 1010 id 1020 | |
+# | | dstport 4789 dstport 4789 | |
+# | | vid 10 pvid untagged vid 20 pvid untagged | |
+# | | | |
+# | | + vx4001 | |
+# | | local 2001:db8:3::2 | |
+# | | remote 2001:db8:3::1 | |
+# | | id 104001 | |
+# | | dstport 4789 | |
+# | | vid 4001 pvid untagged | |
+# | | | |
+# | | + w1 (veth) + w3 (veth) | |
+# | | | vid 10 pvid untagged br1 | vid 20 pvid untagged | |
+# | +--|------------------------------------------|-------------------------+ |
+# | | | |
+# | | | |
+# | +--|----------------------+ +--|-------------------------+ |
+# | | | vrf-h1 | | | vrf-h2 | |
+# | | + w2 (veth) | | + w4 (veth) | |
+# | | 2001:db8:1::4/64 | | 2001:db8:2::4/64 | |
+# | | default via | | default via | |
+# | | 2001:db8:1::3/64 | | 2001:db8:2::3/64 | |
+# | +-------------------------+ +----------------------------+ |
+# +---------------------------------------------------------------------------+
+
+ALL_TESTS="
+ ping_ipv6
+"
+NUM_NETIFS=6
+source lib.sh
+
+hx_create()
+{
+ local vrf_name=$1; shift
+ local if_name=$1; shift
+ local ip_addr=$1; shift
+ local gw_ip=$1; shift
+
+ vrf_create $vrf_name
+ ip link set dev $if_name master $vrf_name
+ ip link set dev $vrf_name up
+ ip link set dev $if_name up
+
+ ip address add $ip_addr/64 dev $if_name
+ ip neigh replace $gw_ip lladdr 00:00:5e:00:01:01 nud permanent \
+ dev $if_name
+ ip route add default vrf $vrf_name nexthop via $gw_ip
+}
+export -f hx_create
+
+hx_destroy()
+{
+ local vrf_name=$1; shift
+ local if_name=$1; shift
+ local ip_addr=$1; shift
+ local gw_ip=$1; shift
+
+ ip route del default vrf $vrf_name nexthop via $gw_ip
+ ip neigh del $gw_ip dev $if_name
+ ip address del $ip_addr/64 dev $if_name
+
+ ip link set dev $if_name down
+ vrf_destroy $vrf_name
+}
+
+h1_create()
+{
+ hx_create "vrf-h1" $h1 2001:db8:1::1 2001:db8:1::3
+}
+
+h1_destroy()
+{
+ hx_destroy "vrf-h1" $h1 2001:db8:1::1 2001:db8:1::3
+}
+
+h2_create()
+{
+ hx_create "vrf-h2" $h2 2001:db8:2::1 2001:db8:2::3
+}
+
+h2_destroy()
+{
+ hx_destroy "vrf-h2" $h2 2001:db8:2::1 2001:db8:2::3
+}
+
+switch_create()
+{
+ ip link add name br1 type bridge vlan_filtering 1 vlan_default_pvid 0 \
+ mcast_snooping 0
+ # Make sure the bridge uses the MAC address of the local port and not
+ # that of the VxLAN's device.
+ ip link set dev br1 address $(mac_get $swp1)
+ ip link set dev br1 up
+
+ ip link set dev $rp1 up
+ ip address add dev $rp1 2001:db8:4::1/64
+ ip route add 2001:db8:3::2/128 nexthop via 2001:db8:4::2
+
+ ip link add name vx10 type vxlan id 1010 \
+ local 2001:db8:3::1 remote 2001:db8:3::2 dstport 4789 \
+ nolearning udp6zerocsumrx udp6zerocsumtx tos inherit ttl 100
+ ip link set dev vx10 up
+
+ ip link set dev vx10 master br1
+ bridge vlan add vid 10 dev vx10 pvid untagged
+
+ ip link add name vx20 type vxlan id 1020 \
+ local 2001:db8:3::1 remote 2001:db8:3::2 dstport 4789 \
+ nolearning udp6zerocsumrx udp6zerocsumtx tos inherit ttl 100
+ ip link set dev vx20 up
+
+ ip link set dev vx20 master br1
+ bridge vlan add vid 20 dev vx20 pvid untagged
+
+ ip link set dev $swp1 master br1
+ ip link set dev $swp1 up
+
+ ip link set dev $swp2 master br1
+ ip link set dev $swp2 up
+
+ ip link add name vx4001 type vxlan id 104001 \
+ local 2001:db8:3::1 dstport 4789 \
+ nolearning udp6zerocsumrx udp6zerocsumtx tos inherit ttl 100
+ ip link set dev vx4001 up
+
+ ip link set dev vx4001 master br1
+ bridge vlan add vid 4001 dev vx4001 pvid untagged
+
+ ip address add 2001:db8:3::1/128 dev lo
+
+ # Create SVIs
+ vrf_create "vrf-green"
+ ip link set dev vrf-green up
+
+ ip link add link br1 name vlan10 up master vrf-green type vlan id 10
+ ip address add 2001:db8:1::2/64 dev vlan10
+ ip link add link vlan10 name vlan10-v up master vrf-green \
+ address 00:00:5e:00:01:01 type macvlan mode private
+ ip address add 2001:db8:1::3/64 dev vlan10-v
+
+ ip link add link br1 name vlan20 up master vrf-green type vlan id 20
+ ip address add 2001:db8:2::2/64 dev vlan20
+ ip link add link vlan20 name vlan20-v up master vrf-green \
+ address 00:00:5e:00:01:01 type macvlan mode private
+ ip address add 2001:db8:2::3/64 dev vlan20-v
+
+ ip link add link br1 name vlan4001 up master vrf-green \
+ type vlan id 4001
+
+ bridge vlan add vid 10 dev br1 self
+ bridge vlan add vid 20 dev br1 self
+ bridge vlan add vid 4001 dev br1 self
+
+ bridge fdb add 00:00:5e:00:01:01 dev br1 self local vlan 10
+ bridge fdb add 00:00:5e:00:01:01 dev br1 self local vlan 20
+
+ bridge vlan add vid 10 dev $swp1 pvid untagged
+ bridge vlan add vid 20 dev $swp2 pvid untagged
+}
+
+switch_destroy()
+{
+ bridge vlan del vid 20 dev br1 self
+ bridge vlan del vid 10 dev br1 self
+
+ bridge fdb del 00:00:5e:00:01:01 dev br1 self local vlan 20
+ bridge fdb del 00:00:5e:00:01:01 dev br1 self local vlan 10
+
+ bridge vlan del vid 4001 dev br1 self
+ ip link del dev vlan4001
+
+ ip link del dev vlan20
+
+ ip link del dev vlan10
+
+ vrf_destroy "vrf-green"
+
+ ip address del 2001:db8:3::1/128 dev lo
+
+ bridge vlan del vid 20 dev $swp2
+ ip link set dev $swp2 down
+ ip link set dev $swp2 nomaster
+
+ bridge vlan del vid 10 dev $swp1
+ ip link set dev $swp1 down
+ ip link set dev $swp1 nomaster
+
+ bridge vlan del vid 4001 dev vx4001
+ ip link set dev vx4001 nomaster
+
+ ip link set dev vx4001 down
+ ip link del dev vx4001
+
+ bridge vlan del vid 20 dev vx20
+ ip link set dev vx20 nomaster
+
+ ip link set dev vx20 down
+ ip link del dev vx20
+
+ bridge vlan del vid 10 dev vx10
+ ip link set dev vx10 nomaster
+
+ ip link set dev vx10 down
+ ip link del dev vx10
+
+ ip route del 2001:db8:3::2 nexthop via 2001:db8:4::2
+ ip address del dev $rp1 2001:db8:4::1/64
+ ip link set dev $rp1 down
+
+ ip link set dev br1 down
+ ip link del dev br1
+}
+
+spine_create()
+{
+ vrf_create "vrf-spine"
+ ip link set dev $rp2 master vrf-spine
+ ip link set dev v1 master vrf-spine
+ ip link set dev vrf-spine up
+ ip link set dev $rp2 up
+ ip link set dev v1 up
+
+ ip address add 2001:db8:4::2/64 dev $rp2
+ ip address add 2001:db8:5::2/64 dev v1
+
+ ip route add 2001:db8:3::1/128 vrf vrf-spine nexthop via \
+ 2001:db8:4::1
+ ip route add 2001:db8:3::2/128 vrf vrf-spine nexthop via \
+ 2001:db8:5::1
+}
+
+spine_destroy()
+{
+ ip route del 2001:db8:3::2/128 vrf vrf-spine nexthop via \
+ 2001:db8:5::1
+ ip route del 2001:db8:3::1/128 vrf vrf-spine nexthop via \
+ 2001:db8:4::1
+
+ ip address del 2001:db8:5::2/64 dev v1
+ ip address del 2001:db8:4::2/64 dev $rp2
+
+ ip link set dev v1 down
+ ip link set dev $rp2 down
+ vrf_destroy "vrf-spine"
+}
+
+ns_h1_create()
+{
+ hx_create "vrf-h1" w2 2001:db8:1::4 2001:db8:1::3
+}
+export -f ns_h1_create
+
+ns_h2_create()
+{
+ hx_create "vrf-h2" w4 2001:db8:2::4 2001:db8:2::3
+}
+export -f ns_h2_create
+
+ns_switch_create()
+{
+ ip link add name br1 type bridge vlan_filtering 1 vlan_default_pvid 0 \
+ mcast_snooping 0
+ ip link set dev br1 up
+
+ ip link set dev v2 up
+ ip address add dev v2 2001:db8:5::1/64
+ ip route add 2001:db8:3::1 nexthop via 2001:db8:5::2
+
+ ip link add name vx10 type vxlan id 1010 \
+ local 2001:db8:3::2 remote 2001:db8:3::1 dstport 4789 \
+ nolearning udp6zerocsumrx udp6zerocsumtx tos inherit ttl 100
+ ip link set dev vx10 up
+
+ ip link set dev vx10 master br1
+ bridge vlan add vid 10 dev vx10 pvid untagged
+
+ ip link add name vx20 type vxlan id 1020 \
+ local 2001:db8:3::2 remote 2001:db8:3::1 dstport 4789 \
+ nolearning udp6zerocsumrx udp6zerocsumtx tos inherit ttl 100
+ ip link set dev vx20 up
+
+ ip link set dev vx20 master br1
+ bridge vlan add vid 20 dev vx20 pvid untagged
+
+ ip link add name vx4001 type vxlan id 104001 \
+ local 2001:db8:3::2 dstport 4789 \
+ nolearning udp6zerocsumrx udp6zerocsumtx tos inherit ttl 100
+ ip link set dev vx4001 up
+
+ ip link set dev vx4001 master br1
+ bridge vlan add vid 4001 dev vx4001 pvid untagged
+
+ ip link set dev w1 master br1
+ ip link set dev w1 up
+ bridge vlan add vid 10 dev w1 pvid untagged
+
+ ip link set dev w3 master br1
+ ip link set dev w3 up
+ bridge vlan add vid 20 dev w3 pvid untagged
+
+ ip address add 2001:db8:3::2/128 dev lo
+
+ # Create SVIs
+ vrf_create "vrf-green"
+ ip link set dev vrf-green up
+
+ ip link add link br1 name vlan10 up master vrf-green type vlan id 10
+ ip address add 2001:db8:1::3/64 dev vlan10
+ ip link add link vlan10 name vlan10-v up master vrf-green \
+ address 00:00:5e:00:01:01 type macvlan mode private
+ ip address add 2001:db8:1::3/64 dev vlan10-v
+
+ ip link add link br1 name vlan20 up master vrf-green type vlan id 20
+ ip address add 2001:db8:2::3/64 dev vlan20
+ ip link add link vlan20 name vlan20-v up master vrf-green \
+ address 00:00:5e:00:01:01 type macvlan mode private
+ ip address add 2001:db8:2::3/64 dev vlan20-v
+
+ ip link add link br1 name vlan4001 up master vrf-green \
+ type vlan id 4001
+
+ bridge vlan add vid 10 dev br1 self
+ bridge vlan add vid 20 dev br1 self
+ bridge vlan add vid 4001 dev br1 self
+
+ bridge fdb add 00:00:5e:00:01:01 dev br1 self local vlan 10
+ bridge fdb add 00:00:5e:00:01:01 dev br1 self local vlan 20
+}
+export -f ns_switch_create
+
+ns_init()
+{
+ ip link add name w1 type veth peer name w2
+ ip link add name w3 type veth peer name w4
+
+ ip link set dev lo up
+
+ ns_h1_create
+ ns_h2_create
+ ns_switch_create
+}
+export -f ns_init
+
+ns1_create()
+{
+ ip netns add ns1
+ ip link set dev v2 netns ns1
+ in_ns ns1 ns_init
+}
+
+ns1_destroy()
+{
+ ip netns exec ns1 ip link set dev v2 netns 1
+ ip netns del ns1
+}
+
+__l2_vni_init()
+{
+ local mac1=$1; shift
+ local mac2=$1; shift
+ local ip1=$1; shift
+ local ip2=$1; shift
+ local dst=$1; shift
+
+ bridge fdb add $mac1 dev vx10 self master extern_learn static \
+ dst $dst vlan 10
+ bridge fdb add $mac2 dev vx20 self master extern_learn static \
+ dst $dst vlan 20
+
+ ip neigh add $ip1 lladdr $mac1 nud noarp dev vlan10 \
+ extern_learn
+ ip neigh add $ip2 lladdr $mac2 nud noarp dev vlan20 \
+ extern_learn
+}
+export -f __l2_vni_init
+
+l2_vni_init()
+{
+ local h1_ns_mac=$(in_ns ns1 mac_get w2)
+ local h2_ns_mac=$(in_ns ns1 mac_get w4)
+ local h1_mac=$(mac_get $h1)
+ local h2_mac=$(mac_get $h2)
+
+ __l2_vni_init $h1_ns_mac $h2_ns_mac 2001:db8:1::4 2001:db8:2::4 \
+ 2001:db8:3::2
+ in_ns ns1 __l2_vni_init $h1_mac $h2_mac 2001:db8:1::1 2001:db8:2::1 \
+ 2001:db8:3::1
+}
+
+__l3_vni_init()
+{
+ local mac=$1; shift
+ local vtep_ip=$1; shift
+ local host1_ip=$1; shift
+ local host2_ip=$1; shift
+
+ bridge fdb add $mac dev vx4001 self master extern_learn static \
+ dst $vtep_ip vlan 4001
+
+ ip neigh add $vtep_ip lladdr $mac nud noarp dev vlan4001 extern_learn
+
+ ip route add $host1_ip/128 vrf vrf-green nexthop via $vtep_ip \
+ dev vlan4001 onlink
+ ip route add $host2_ip/128 vrf vrf-green nexthop via $vtep_ip \
+ dev vlan4001 onlink
+}
+export -f __l3_vni_init
+
+l3_vni_init()
+{
+ local vlan4001_ns_mac=$(in_ns ns1 mac_get vlan4001)
+ local vlan4001_mac=$(mac_get vlan4001)
+
+ __l3_vni_init $vlan4001_ns_mac 2001:db8:3::2 2001:db8:1::4 \
+ 2001:db8:2::4
+ in_ns ns1 __l3_vni_init $vlan4001_mac 2001:db8:3::1 2001:db8:1::1 \
+ 2001:db8:2::1
+}
+
+setup_prepare()
+{
+ h1=${NETIFS[p1]}
+ swp1=${NETIFS[p2]}
+
+ swp2=${NETIFS[p3]}
+ h2=${NETIFS[p4]}
+
+ rp1=${NETIFS[p5]}
+ rp2=${NETIFS[p6]}
+
+ vrf_prepare
+ forwarding_enable
+
+ h1_create
+ h2_create
+ switch_create
+
+ ip link add name v1 type veth peer name v2
+ spine_create
+ ns1_create
+ in_ns ns1 forwarding_enable
+
+ l2_vni_init
+ l3_vni_init
+}
+
+cleanup()
+{
+ pre_cleanup
+
+ ns1_destroy
+ spine_destroy
+ ip link del dev v1
+
+ switch_destroy
+ h2_destroy
+ h1_destroy
+
+ forwarding_restore
+ vrf_cleanup
+}
+
+ping_ipv6()
+{
+ ping6_test $h1 2001:db8:2::1 ": local->local vid 10->vid 20"
+ ping6_test $h1 2001:db8:1::4 ": local->remote vid 10->vid 10"
+ ping6_test $h2 2001:db8:2::4 ": local->remote vid 20->vid 20"
+ ping6_test $h1 2001:db8:2::4 ": local->remote vid 10->vid 20"
+ ping6_test $h2 2001:db8:1::4 ": local->remote vid 20->vid 10"
+}
+
+trap cleanup EXIT
+
+setup_prepare
+setup_wait
+
+tests_run
+
+exit $EXIT_STATUS
diff --git a/tools/testing/selftests/net/gro.c b/tools/testing/selftests/net/gro.c
index cf37ce8..30024d0 100644
--- a/tools/testing/selftests/net/gro.c
+++ b/tools/testing/selftests/net/gro.c
@@ -57,17 +57,14 @@
#include <string.h>
#include <unistd.h>
+#include "../kselftest.h"
+
#define DPORT 8000
#define SPORT 1500
#define PAYLOAD_LEN 100
-#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]))
#define NUM_PACKETS 4
#define START_SEQ 100
#define START_ACK 100
-#define SIP6 "fdaa::2"
-#define DIP6 "fdaa::1"
-#define SIP4 "192.168.1.200"
-#define DIP4 "192.168.1.100"
#define ETH_P_NONE 0
#define TOTAL_HDR_LEN (ETH_HLEN + sizeof(struct ipv6hdr) + sizeof(struct tcphdr))
#define MSS (4096 - sizeof(struct tcphdr) - sizeof(struct ipv6hdr))
@@ -75,6 +72,10 @@
#define NUM_LARGE_PKT (MAX_PAYLOAD / MSS)
#define MAX_HDR_LEN (ETH_HLEN + sizeof(struct ipv6hdr) + sizeof(struct tcphdr))
+static const char *addr6_src = "fdaa::2";
+static const char *addr6_dst = "fdaa::1";
+static const char *addr4_src = "192.168.1.200";
+static const char *addr4_dst = "192.168.1.100";
static int proto = -1;
static uint8_t src_mac[ETH_ALEN], dst_mac[ETH_ALEN];
static char *testname = "data";
@@ -178,18 +179,18 @@ static uint16_t tcp_checksum(void *buf, int payload_len)
uint32_t sum = 0;
if (proto == PF_INET6) {
- if (inet_pton(AF_INET6, SIP6, &ph6.saddr) != 1)
+ if (inet_pton(AF_INET6, addr6_src, &ph6.saddr) != 1)
error(1, errno, "inet_pton6 source ip pseudo");
- if (inet_pton(AF_INET6, DIP6, &ph6.daddr) != 1)
+ if (inet_pton(AF_INET6, addr6_dst, &ph6.daddr) != 1)
error(1, errno, "inet_pton6 dest ip pseudo");
ph6.protocol = htons(IPPROTO_TCP);
ph6.payload_len = htons(sizeof(struct tcphdr) + payload_len);
sum = checksum_nofold(&ph6, sizeof(ph6), 0);
} else if (proto == PF_INET) {
- if (inet_pton(AF_INET, SIP4, &ph4.saddr) != 1)
+ if (inet_pton(AF_INET, addr4_src, &ph4.saddr) != 1)
error(1, errno, "inet_pton source ip pseudo");
- if (inet_pton(AF_INET, DIP4, &ph4.daddr) != 1)
+ if (inet_pton(AF_INET, addr4_dst, &ph4.daddr) != 1)
error(1, errno, "inet_pton dest ip pseudo");
ph4.protocol = htons(IPPROTO_TCP);
ph4.payload_len = htons(sizeof(struct tcphdr) + payload_len);
@@ -229,9 +230,9 @@ static void fill_networklayer(void *buf, int payload_len)
ip6h->payload_len = htons(sizeof(struct tcphdr) + payload_len);
ip6h->nexthdr = IPPROTO_TCP;
ip6h->hop_limit = 8;
- if (inet_pton(AF_INET6, SIP6, &ip6h->saddr) != 1)
+ if (inet_pton(AF_INET6, addr6_src, &ip6h->saddr) != 1)
error(1, errno, "inet_pton source ip6");
- if (inet_pton(AF_INET6, DIP6, &ip6h->daddr) != 1)
+ if (inet_pton(AF_INET6, addr6_dst, &ip6h->daddr) != 1)
error(1, errno, "inet_pton dest ip6");
} else if (proto == PF_INET) {
memset(iph, 0, sizeof(*iph));
@@ -243,9 +244,9 @@ static void fill_networklayer(void *buf, int payload_len)
iph->tot_len = htons(sizeof(struct tcphdr) +
payload_len + sizeof(struct iphdr));
iph->frag_off = htons(0x4000); /* DF = 1, MF = 0 */
- if (inet_pton(AF_INET, SIP4, &iph->saddr) != 1)
+ if (inet_pton(AF_INET, addr4_src, &iph->saddr) != 1)
error(1, errno, "inet_pton source ip");
- if (inet_pton(AF_INET, DIP4, &iph->daddr) != 1)
+ if (inet_pton(AF_INET, addr4_dst, &iph->daddr) != 1)
error(1, errno, "inet_pton dest ip");
iph->check = checksum_fold(buf, sizeof(struct iphdr), 0);
}
@@ -731,7 +732,7 @@ static void set_timeout(int fd)
{
struct timeval timeout;
- timeout.tv_sec = 120;
+ timeout.tv_sec = 3;
timeout.tv_usec = 0;
if (setsockopt(fd, SOL_SOCKET, SO_RCVTIMEO, (char *)&timeout,
sizeof(timeout)) < 0)
@@ -1023,11 +1024,13 @@ static void gro_receiver(void)
static void parse_args(int argc, char **argv)
{
static const struct option opts[] = {
+ { "daddr", required_argument, NULL, 'd' },
{ "dmac", required_argument, NULL, 'D' },
{ "iface", required_argument, NULL, 'i' },
{ "ipv4", no_argument, NULL, '4' },
{ "ipv6", no_argument, NULL, '6' },
{ "rx", no_argument, NULL, 'r' },
+ { "saddr", required_argument, NULL, 's' },
{ "smac", required_argument, NULL, 'S' },
{ "test", required_argument, NULL, 't' },
{ "verbose", no_argument, NULL, 'v' },
@@ -1035,7 +1038,7 @@ static void parse_args(int argc, char **argv)
};
int c;
- while ((c = getopt_long(argc, argv, "46D:i:rS:t:v", opts, NULL)) != -1) {
+ while ((c = getopt_long(argc, argv, "46d:D:i:rs:S:t:v", opts, NULL)) != -1) {
switch (c) {
case '4':
proto = PF_INET;
@@ -1045,6 +1048,9 @@ static void parse_args(int argc, char **argv)
proto = PF_INET6;
ethhdr_proto = htons(ETH_P_IPV6);
break;
+ case 'd':
+ addr4_dst = addr6_dst = optarg;
+ break;
case 'D':
dmac = optarg;
break;
@@ -1054,6 +1060,9 @@ static void parse_args(int argc, char **argv)
case 'r':
tx_socket = false;
break;
+ case 's':
+ addr4_src = addr6_src = optarg;
+ break;
case 'S':
smac = optarg;
break;
@@ -1091,5 +1100,7 @@ int main(int argc, char **argv)
gro_sender();
else
gro_receiver();
+
+ fprintf(stderr, "Gro::%s test passed.\n", testname);
return 0;
}
diff --git a/tools/testing/selftests/net/ipsec.c b/tools/testing/selftests/net/ipsec.c
index 3d7dde2..cc10c10 100644
--- a/tools/testing/selftests/net/ipsec.c
+++ b/tools/testing/selftests/net/ipsec.c
@@ -41,7 +41,6 @@
#define pr_err(fmt, ...) printk(fmt ": %m", ##__VA_ARGS__)
-#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]))
#define BUILD_BUG_ON(condition) ((void)sizeof(char[1 - 2*!!(condition)]))
#define IPV4_STR_SZ 16 /* xxx.xxx.xxx.xxx is longest + \0 */
diff --git a/tools/testing/selftests/net/mptcp/.gitignore b/tools/testing/selftests/net/mptcp/.gitignore
index 7569d89..49daae7 100644
--- a/tools/testing/selftests/net/mptcp/.gitignore
+++ b/tools/testing/selftests/net/mptcp/.gitignore
@@ -1,5 +1,6 @@
# SPDX-License-Identifier: GPL-2.0-only
mptcp_connect
+mptcp_inq
mptcp_sockopt
pm_nl_ctl
*.pcap
diff --git a/tools/testing/selftests/net/mptcp/Makefile b/tools/testing/selftests/net/mptcp/Makefile
index bbf4e44..0356c45 100644
--- a/tools/testing/selftests/net/mptcp/Makefile
+++ b/tools/testing/selftests/net/mptcp/Makefile
@@ -8,7 +8,7 @@
TEST_PROGS := mptcp_connect.sh pm_netlink.sh mptcp_join.sh diag.sh \
simult_flows.sh mptcp_sockopt.sh
-TEST_GEN_FILES = mptcp_connect pm_nl_ctl mptcp_sockopt
+TEST_GEN_FILES = mptcp_connect pm_nl_ctl mptcp_sockopt mptcp_inq
TEST_FILES := settings
diff --git a/tools/testing/selftests/net/mptcp/config b/tools/testing/selftests/net/mptcp/config
index 0faaccd..d36b7da 100644
--- a/tools/testing/selftests/net/mptcp/config
+++ b/tools/testing/selftests/net/mptcp/config
@@ -9,9 +9,13 @@
CONFIG_NETFILTER_ADVANCED=y
CONFIG_NETFILTER_NETLINK=m
CONFIG_NF_TABLES=m
-CONFIG_NFT_COUNTER=m
CONFIG_NFT_COMPAT=m
CONFIG_NETFILTER_XTABLES=m
CONFIG_NETFILTER_XT_MATCH_BPF=m
-CONFIG_NF_TABLES_IPV4=y
-CONFIG_NF_TABLES_IPV6=y
+CONFIG_NF_TABLES_INET=y
+CONFIG_NFT_TPROXY=m
+CONFIG_NFT_SOCKET=m
+CONFIG_IP_ADVANCED_ROUTER=y
+CONFIG_IP_MULTIPLE_TABLES=y
+CONFIG_IP_NF_TARGET_REJECT=m
+CONFIG_IPV6_MULTIPLE_TABLES=y
diff --git a/tools/testing/selftests/net/mptcp/mptcp_connect.c b/tools/testing/selftests/net/mptcp/mptcp_connect.c
index 95e81d5..8628aa6 100644
--- a/tools/testing/selftests/net/mptcp/mptcp_connect.c
+++ b/tools/testing/selftests/net/mptcp/mptcp_connect.c
@@ -16,6 +16,7 @@
#include <unistd.h>
#include <time.h>
+#include <sys/ioctl.h>
#include <sys/poll.h>
#include <sys/sendfile.h>
#include <sys/stat.h>
@@ -28,6 +29,7 @@
#include <linux/tcp.h>
#include <linux/time_types.h>
+#include <linux/sockios.h>
extern int optind;
@@ -59,7 +61,6 @@ static enum cfg_peek cfg_peek = CFG_NONE_PEEK;
static const char *cfg_host;
static const char *cfg_port = "12000";
static int cfg_sock_proto = IPPROTO_MPTCP;
-static bool tcpulp_audit;
static int pf = AF_INET;
static int cfg_sndbuf;
static int cfg_rcvbuf;
@@ -69,32 +70,56 @@ static unsigned int cfg_time;
static unsigned int cfg_do_w;
static int cfg_wait;
static uint32_t cfg_mark;
+static char *cfg_input;
+static int cfg_repeat = 1;
struct cfg_cmsg_types {
unsigned int cmsg_enabled:1;
unsigned int timestampns:1;
+ unsigned int tcp_inq:1;
};
+struct cfg_sockopt_types {
+ unsigned int transparent:1;
+};
+
+struct tcp_inq_state {
+ unsigned int last;
+ bool expect_eof;
+};
+
+static struct tcp_inq_state tcp_inq;
+
static struct cfg_cmsg_types cfg_cmsg_types;
+static struct cfg_sockopt_types cfg_sockopt_types;
static void die_usage(void)
{
- fprintf(stderr, "Usage: mptcp_connect [-6] [-u] [-s MPTCP|TCP] [-p port] [-m mode]"
- "[-l] [-w sec] [-t num] [-T num] connect_address\n");
+ fprintf(stderr, "Usage: mptcp_connect [-6] [-c cmsg] [-i file] [-I num] [-j] [-l] "
+ "[-m mode] [-M mark] [-o option] [-p port] [-P mode] [-j] [-l] [-r num] "
+ "[-s MPTCP|TCP] [-S num] [-r num] [-t num] [-T num] [-u] [-w sec] connect_address\n");
fprintf(stderr, "\t-6 use ipv6\n");
- fprintf(stderr, "\t-t num -- set poll timeout to num\n");
- fprintf(stderr, "\t-T num -- set expected runtime to num ms\n");
- fprintf(stderr, "\t-S num -- set SO_SNDBUF to num\n");
- fprintf(stderr, "\t-R num -- set SO_RCVBUF to num\n");
- fprintf(stderr, "\t-p num -- use port num\n");
- fprintf(stderr, "\t-s [MPTCP|TCP] -- use mptcp(default) or tcp sockets\n");
+ fprintf(stderr, "\t-c cmsg -- test cmsg type <cmsg>\n");
+ fprintf(stderr, "\t-i file -- read the data to send from the given file instead of stdin");
+ fprintf(stderr, "\t-I num -- repeat the transfer 'num' times. In listen mode accepts num "
+ "incoming connections, in client mode, disconnect and reconnect to the server\n");
+ fprintf(stderr, "\t-j -- add additional sleep at connection start and tear down "
+ "-- for MPJ tests\n");
+ fprintf(stderr, "\t-l -- listens mode, accepts incoming connection\n");
fprintf(stderr, "\t-m [poll|mmap|sendfile] -- use poll(default)/mmap+write/sendfile\n");
fprintf(stderr, "\t-M mark -- set socket packet mark\n");
- fprintf(stderr, "\t-u -- check mptcp ulp\n");
- fprintf(stderr, "\t-w num -- wait num sec before closing the socket\n");
- fprintf(stderr, "\t-c cmsg -- test cmsg type <cmsg>\n");
+ fprintf(stderr, "\t-o option -- test sockopt <option>\n");
+ fprintf(stderr, "\t-p num -- use port num\n");
fprintf(stderr,
"\t-P [saveWithPeek|saveAfterPeek] -- save data with/after MSG_PEEK form tcp socket\n");
+ fprintf(stderr, "\t-t num -- set poll timeout to num\n");
+ fprintf(stderr, "\t-T num -- set expected runtime to num ms\n");
+ fprintf(stderr, "\t-r num -- enable slow mode, limiting each write to num bytes "
+ "-- for remove addr tests\n");
+ fprintf(stderr, "\t-R num -- set SO_RCVBUF to num\n");
+ fprintf(stderr, "\t-s [MPTCP|TCP] -- use mptcp(default) or tcp sockets\n");
+ fprintf(stderr, "\t-S num -- set SO_SNDBUF to num\n");
+ fprintf(stderr, "\t-w num -- wait num sec before closing the socket\n");
exit(1);
}
@@ -185,6 +210,58 @@ static void set_mark(int fd, uint32_t mark)
}
}
+static void set_transparent(int fd, int pf)
+{
+ int one = 1;
+
+ switch (pf) {
+ case AF_INET:
+ if (-1 == setsockopt(fd, SOL_IP, IP_TRANSPARENT, &one, sizeof(one)))
+ perror("IP_TRANSPARENT");
+ break;
+ case AF_INET6:
+ if (-1 == setsockopt(fd, IPPROTO_IPV6, IPV6_TRANSPARENT, &one, sizeof(one)))
+ perror("IPV6_TRANSPARENT");
+ break;
+ }
+}
+
+static int do_ulp_so(int sock, const char *name)
+{
+ return setsockopt(sock, IPPROTO_TCP, TCP_ULP, name, strlen(name));
+}
+
+#define X(m) xerror("%s:%u: %s: failed for proto %d at line %u", __FILE__, __LINE__, (m), proto, line)
+static void sock_test_tcpulp(int sock, int proto, unsigned int line)
+{
+ socklen_t buflen = 8;
+ char buf[8] = "";
+ int ret = getsockopt(sock, IPPROTO_TCP, TCP_ULP, buf, &buflen);
+
+ if (ret != 0)
+ X("getsockopt");
+
+ if (buflen > 0) {
+ if (strcmp(buf, "mptcp") != 0)
+ xerror("unexpected ULP '%s' for proto %d at line %u", buf, proto, line);
+ ret = do_ulp_so(sock, "tls");
+ if (ret == 0)
+ X("setsockopt");
+ } else if (proto == IPPROTO_MPTCP) {
+ ret = do_ulp_so(sock, "tls");
+ if (ret != -1)
+ X("setsockopt");
+ }
+
+ ret = do_ulp_so(sock, "mptcp");
+ if (ret != -1)
+ X("setsockopt");
+
+#undef X
+}
+
+#define SOCK_TEST_TCPULP(s, p) sock_test_tcpulp((s), (p), __LINE__)
+
static int sock_listen_mptcp(const char * const listenaddr,
const char * const port)
{
@@ -208,10 +285,15 @@ static int sock_listen_mptcp(const char * const listenaddr,
if (sock < 0)
continue;
+ SOCK_TEST_TCPULP(sock, cfg_sock_proto);
+
if (-1 == setsockopt(sock, SOL_SOCKET, SO_REUSEADDR, &one,
sizeof(one)))
perror("setsockopt");
+ if (cfg_sockopt_types.transparent)
+ set_transparent(sock, pf);
+
if (bind(sock, a->ai_addr, a->ai_addrlen) == 0)
break; /* success */
@@ -227,54 +309,22 @@ static int sock_listen_mptcp(const char * const listenaddr,
return sock;
}
+ SOCK_TEST_TCPULP(sock, cfg_sock_proto);
+
if (listen(sock, 20)) {
perror("listen");
close(sock);
return -1;
}
+ SOCK_TEST_TCPULP(sock, cfg_sock_proto);
+
return sock;
}
-static bool sock_test_tcpulp(const char * const remoteaddr,
- const char * const port)
-{
- struct addrinfo hints = {
- .ai_protocol = IPPROTO_TCP,
- .ai_socktype = SOCK_STREAM,
- };
- struct addrinfo *a, *addr;
- int sock = -1, ret = 0;
- bool test_pass = false;
-
- hints.ai_family = AF_INET;
-
- xgetaddrinfo(remoteaddr, port, &hints, &addr);
- for (a = addr; a; a = a->ai_next) {
- sock = socket(a->ai_family, a->ai_socktype, IPPROTO_TCP);
- if (sock < 0) {
- perror("socket");
- continue;
- }
- ret = setsockopt(sock, IPPROTO_TCP, TCP_ULP, "mptcp",
- sizeof("mptcp"));
- if (ret == -1 && errno == EOPNOTSUPP)
- test_pass = true;
- close(sock);
-
- if (test_pass)
- break;
- if (!ret)
- fprintf(stderr,
- "setsockopt(TCP_ULP) returned 0\n");
- else
- perror("setsockopt(TCP_ULP)");
- }
- return test_pass;
-}
-
static int sock_connect_mptcp(const char * const remoteaddr,
- const char * const port, int proto)
+ const char * const port, int proto,
+ struct addrinfo **peer)
{
struct addrinfo hints = {
.ai_protocol = IPPROTO_TCP,
@@ -293,11 +343,15 @@ static int sock_connect_mptcp(const char * const remoteaddr,
continue;
}
+ SOCK_TEST_TCPULP(sock, proto);
+
if (cfg_mark)
set_mark(sock, cfg_mark);
- if (connect(sock, a->ai_addr, a->ai_addrlen) == 0)
+ if (connect(sock, a->ai_addr, a->ai_addrlen) == 0) {
+ *peer = a;
break; /* success */
+ }
perror("connect()");
close(sock);
@@ -305,6 +359,8 @@ static int sock_connect_mptcp(const char * const remoteaddr,
}
freeaddrinfo(addr);
+ if (sock != -1)
+ SOCK_TEST_TCPULP(sock, proto);
return sock;
}
@@ -364,7 +420,9 @@ static size_t do_write(const int fd, char *buf, const size_t len)
static void process_cmsg(struct msghdr *msgh)
{
struct __kernel_timespec ts;
+ bool inq_found = false;
bool ts_found = false;
+ unsigned int inq = 0;
struct cmsghdr *cmsg;
for (cmsg = CMSG_FIRSTHDR(msgh); cmsg ; cmsg = CMSG_NXTHDR(msgh, cmsg)) {
@@ -373,12 +431,27 @@ static void process_cmsg(struct msghdr *msgh)
ts_found = true;
continue;
}
+ if (cmsg->cmsg_level == IPPROTO_TCP && cmsg->cmsg_type == TCP_CM_INQ) {
+ memcpy(&inq, CMSG_DATA(cmsg), sizeof(inq));
+ inq_found = true;
+ continue;
+ }
+
}
if (cfg_cmsg_types.timestampns) {
if (!ts_found)
xerror("TIMESTAMPNS not present\n");
}
+
+ if (cfg_cmsg_types.tcp_inq) {
+ if (!inq_found)
+ xerror("TCP_INQ not present\n");
+
+ if (inq > 1024)
+ xerror("tcp_inq %u is larger than one kbyte\n", inq);
+ tcp_inq.last = inq;
+ }
}
static ssize_t do_recvmsg_cmsg(const int fd, char *buf, const size_t len)
@@ -395,10 +468,23 @@ static ssize_t do_recvmsg_cmsg(const int fd, char *buf, const size_t len)
.msg_controllen = sizeof(msg_buf),
};
int flags = 0;
+ unsigned int last_hint = tcp_inq.last;
int ret = recvmsg(fd, &msg, flags);
- if (ret <= 0)
+ if (ret <= 0) {
+ if (ret == 0 && tcp_inq.expect_eof)
+ return ret;
+
+ if (ret == 0 && cfg_cmsg_types.tcp_inq)
+ if (last_hint != 1 && last_hint != 0)
+ xerror("EOF but last tcp_inq hint was %u\n", last_hint);
+
return ret;
+ }
+
+ if (tcp_inq.expect_eof)
+ xerror("expected EOF, last_hint %u, now %u\n",
+ last_hint, tcp_inq.last);
if (msg.msg_controllen && !cfg_cmsg_types.cmsg_enabled)
xerror("got %lu bytes of cmsg data, expected 0\n",
@@ -410,6 +496,19 @@ static ssize_t do_recvmsg_cmsg(const int fd, char *buf, const size_t len)
if (msg.msg_controllen)
process_cmsg(&msg);
+ if (cfg_cmsg_types.tcp_inq) {
+ if ((size_t)ret < len && last_hint > (unsigned int)ret) {
+ if (ret + 1 != (int)last_hint) {
+ int next = read(fd, msg_buf, sizeof(msg_buf));
+
+ xerror("read %u of %u, last_hint was %u tcp_inq hint now %u next_read returned %d/%m\n",
+ ret, (unsigned int)len, last_hint, tcp_inq.last, next);
+ } else {
+ tcp_inq.expect_eof = true;
+ }
+ }
+ }
+
return ret;
}
@@ -441,14 +540,17 @@ static ssize_t do_rnd_read(const int fd, char *buf, const size_t len)
return ret;
}
-static void set_nonblock(int fd)
+static void set_nonblock(int fd, bool nonblock)
{
int flags = fcntl(fd, F_GETFL);
if (flags == -1)
return;
- fcntl(fd, F_SETFL, flags | O_NONBLOCK);
+ if (nonblock)
+ fcntl(fd, F_SETFL, flags | O_NONBLOCK);
+ else
+ fcntl(fd, F_SETFL, flags & ~O_NONBLOCK);
}
static int copyfd_io_poll(int infd, int peerfd, int outfd, bool *in_closed_after_out)
@@ -460,7 +562,7 @@ static int copyfd_io_poll(int infd, int peerfd, int outfd, bool *in_closed_after
unsigned int woff = 0, wlen = 0;
char wbuf[8192];
- set_nonblock(peerfd);
+ set_nonblock(peerfd, true);
for (;;) {
char rbuf[8192];
@@ -555,7 +657,6 @@ static int copyfd_io_poll(int infd, int peerfd, int outfd, bool *in_closed_after
if (cfg_remove)
usleep(cfg_wait);
- close(peerfd);
return 0;
}
@@ -697,7 +798,7 @@ static int copyfd_io_sendfile(int infd, int peerfd, int outfd,
return err;
}
-static int copyfd_io(int infd, int peerfd, int outfd)
+static int copyfd_io(int infd, int peerfd, int outfd, bool close_peerfd)
{
bool in_closed_after_out = false;
struct timespec start, end;
@@ -736,6 +837,9 @@ static int copyfd_io(int infd, int peerfd, int outfd)
if (ret)
return ret;
+ if (close_peerfd)
+ close(peerfd);
+
if (cfg_time) {
unsigned int delta_ms;
@@ -847,7 +951,7 @@ static void maybe_close(int fd)
{
unsigned int r = rand();
- if (!(cfg_join || cfg_remove) && (r & 1))
+ if (!(cfg_join || cfg_remove || cfg_repeat > 1) && (r & 1))
close(fd);
}
@@ -857,7 +961,9 @@ int main_loop_s(int listensock)
struct pollfd polls;
socklen_t salen;
int remotesock;
+ int fd = 0;
+again:
polls.fd = listensock;
polls.events = POLLIN;
@@ -878,12 +984,27 @@ int main_loop_s(int listensock)
check_sockaddr(pf, &ss, salen);
check_getpeername(remotesock, &ss, salen);
- return copyfd_io(0, remotesock, 1);
+ if (cfg_input) {
+ fd = open(cfg_input, O_RDONLY);
+ if (fd < 0)
+ xerror("can't open %s: %d", cfg_input, errno);
+ }
+
+ SOCK_TEST_TCPULP(remotesock, 0);
+
+ copyfd_io(fd, remotesock, 1, true);
+ } else {
+ perror("accept");
+ return 1;
}
- perror("accept");
+ if (--cfg_repeat > 0) {
+ if (cfg_input)
+ close(fd);
+ goto again;
+ }
- return 1;
+ return 0;
}
static void init_rng(void)
@@ -919,6 +1040,8 @@ static void apply_cmsg_types(int fd, const struct cfg_cmsg_types *cmsg)
if (cmsg->timestampns)
xsetsockopt(fd, SOL_SOCKET, SO_TIMESTAMPNS_NEW, &on, sizeof(on));
+ if (cmsg->tcp_inq)
+ xsetsockopt(fd, IPPROTO_TCP, TCP_INQ, &on, sizeof(on));
}
static void parse_cmsg_types(const char *type)
@@ -940,21 +1063,81 @@ static void parse_cmsg_types(const char *type)
return;
}
+ if (strncmp(type, "TCPINQ", len) == 0) {
+ cfg_cmsg_types.tcp_inq = 1;
+ return;
+ }
+
fprintf(stderr, "Unrecognized cmsg option %s\n", type);
exit(1);
}
+static void parse_setsock_options(const char *name)
+{
+ char *next = strchr(name, ',');
+ unsigned int len = 0;
+
+ if (next) {
+ parse_setsock_options(next + 1);
+ len = next - name;
+ } else {
+ len = strlen(name);
+ }
+
+ if (strncmp(name, "TRANSPARENT", len) == 0) {
+ cfg_sockopt_types.transparent = 1;
+ return;
+ }
+
+ fprintf(stderr, "Unrecognized setsockopt option %s\n", name);
+ exit(1);
+}
+
+void xdisconnect(int fd, int addrlen)
+{
+ struct sockaddr_storage empty;
+ int msec_sleep = 10;
+ int queued = 1;
+ int i;
+
+ shutdown(fd, SHUT_WR);
+
+ /* while until the pending data is completely flushed, the later
+ * disconnect will bypass/ignore/drop any pending data.
+ */
+ for (i = 0; ; i += msec_sleep) {
+ if (ioctl(fd, SIOCOUTQ, &queued) < 0)
+ xerror("can't query out socket queue: %d", errno);
+
+ if (!queued)
+ break;
+
+ if (i > poll_timeout)
+ xerror("timeout while waiting for spool to complete");
+ usleep(msec_sleep * 1000);
+ }
+
+ memset(&empty, 0, sizeof(empty));
+ empty.ss_family = AF_UNSPEC;
+ if (connect(fd, (struct sockaddr *)&empty, addrlen) < 0)
+ xerror("can't disconnect: %d", errno);
+}
+
int main_loop(void)
{
- int fd;
+ int fd, ret, fd_in = 0;
+ struct addrinfo *peer;
/* listener is ready. */
- fd = sock_connect_mptcp(cfg_host, cfg_port, cfg_sock_proto);
+ fd = sock_connect_mptcp(cfg_host, cfg_port, cfg_sock_proto, &peer);
if (fd < 0)
return 2;
+again:
check_getpeername_connect(fd);
+ SOCK_TEST_TCPULP(fd, cfg_sock_proto);
+
if (cfg_rcvbuf)
set_rcvbuf(fd, cfg_rcvbuf);
if (cfg_sndbuf)
@@ -962,7 +1145,31 @@ int main_loop(void)
if (cfg_cmsg_types.cmsg_enabled)
apply_cmsg_types(fd, &cfg_cmsg_types);
- return copyfd_io(0, fd, 1);
+ if (cfg_input) {
+ fd_in = open(cfg_input, O_RDONLY);
+ if (fd < 0)
+ xerror("can't open %s:%d", cfg_input, errno);
+ }
+
+ /* close the client socket open only if we are not going to reconnect */
+ ret = copyfd_io(fd_in, fd, 1, cfg_repeat == 1);
+ if (ret)
+ return ret;
+
+ if (--cfg_repeat > 0) {
+ xdisconnect(fd, peer->ai_addrlen);
+
+ /* the socket could be unblocking at this point, we need the
+ * connect to be blocking
+ */
+ set_nonblock(fd, false);
+ if (connect(fd, peer->ai_addr, peer->ai_addrlen))
+ xerror("can't reconnect: %d", errno);
+ if (cfg_input)
+ close(fd_in);
+ goto again;
+ }
+ return 0;
}
int parse_proto(const char *proto)
@@ -1047,7 +1254,7 @@ static void parse_opts(int argc, char **argv)
{
int c;
- while ((c = getopt(argc, argv, "6jr:lp:s:hut:T:m:S:R:w:M:P:c:")) != -1) {
+ while ((c = getopt(argc, argv, "6c:hi:I:jlm:M:o:p:P:r:R:s:S:t:T:w:")) != -1) {
switch (c) {
case 'j':
cfg_join = true;
@@ -1061,6 +1268,12 @@ static void parse_opts(int argc, char **argv)
if (cfg_do_w <= 0)
cfg_do_w = 50;
break;
+ case 'i':
+ cfg_input = optarg;
+ break;
+ case 'I':
+ cfg_repeat = atoi(optarg);
+ break;
case 'l':
listen_mode = true;
break;
@@ -1073,9 +1286,6 @@ static void parse_opts(int argc, char **argv)
case 'h':
die_usage();
break;
- case 'u':
- tcpulp_audit = true;
- break;
case '6':
pf = AF_INET6;
break;
@@ -1108,6 +1318,9 @@ static void parse_opts(int argc, char **argv)
case 'c':
parse_cmsg_types(optarg);
break;
+ case 'o':
+ parse_setsock_options(optarg);
+ break;
}
}
@@ -1126,9 +1339,6 @@ int main(int argc, char *argv[])
signal(SIGUSR1, handle_signal);
parse_opts(argc, argv);
- if (tcpulp_audit)
- return sock_test_tcpulp(cfg_host, cfg_port) ? 0 : 1;
-
if (listen_mode) {
int fd = sock_listen_mptcp(cfg_host, cfg_port);
diff --git a/tools/testing/selftests/net/mptcp/mptcp_connect.sh b/tools/testing/selftests/net/mptcp/mptcp_connect.sh
index 559173a..cb5809b 100755
--- a/tools/testing/selftests/net/mptcp/mptcp_connect.sh
+++ b/tools/testing/selftests/net/mptcp/mptcp_connect.sh
@@ -7,6 +7,7 @@
ret=0
sin=""
sout=""
+cin_disconnect=""
cin=""
cout=""
ksft_skip=4
@@ -24,6 +25,7 @@
do_tcp=0
checksum=false
filesize=0
+connect_per_transfer=1
if [ $tc_loss -eq 100 ];then
tc_loss=1%
@@ -127,6 +129,7 @@
cleanup()
{
+ rm -f "$cin_disconnect" "$cout_disconnect"
rm -f "$cin" "$cout"
rm -f "$sin" "$sout"
rm -f "$capout"
@@ -149,6 +152,8 @@
cin=$(mktemp)
cout=$(mktemp)
capout=$(mktemp)
+cin_disconnect="$cin".disconnect
+cout_disconnect="$cout".disconnect
trap cleanup EXIT
for i in "$ns1" "$ns2" "$ns3" "$ns4";do
@@ -296,24 +301,6 @@
return 0
}
-check_mptcp_ulp_setsockopt()
-{
- local t retval
- t="ns_ulp-$sech-$(mktemp -u XXXXXX)"
-
- ip netns add ${t} || exit $ksft_skip
- if ! ip netns exec ${t} ./mptcp_connect -u -p 10000 -s TCP 127.0.0.1 2>&1; then
- printf "setsockopt(..., TCP_ULP, \"mptcp\", ...) allowed\t[ FAIL ]\n"
- retval=1
- ret=$retval
- else
- printf "setsockopt(..., TCP_ULP, \"mptcp\", ...) blocked\t[ OK ]\n"
- retval=0
- fi
- ip netns del ${t}
- return $retval
-}
-
# $1: IP address
is_v6()
{
@@ -518,8 +505,8 @@
cookies=${cookies##*=}
if [ ${cl_proto} = "MPTCP" ] && [ ${srv_proto} = "MPTCP" ]; then
- expect_synrx=$((stat_synrx_last_l+1))
- expect_ackrx=$((stat_ackrx_last_l+1))
+ expect_synrx=$((stat_synrx_last_l+$connect_per_transfer))
+ expect_ackrx=$((stat_ackrx_last_l+$connect_per_transfer))
fi
if [ ${stat_synrx_now_l} -lt ${expect_synrx} ]; then
@@ -671,6 +658,82 @@
run_tests_lo $1 $2 $3 0
}
+run_test_transparent()
+{
+ local connect_addr="$1"
+ local msg="$2"
+
+ local connector_ns="$ns1"
+ local listener_ns="$ns2"
+ local lret=0
+ local r6flag=""
+
+ # skip if we don't want v6
+ if ! $ipv6 && is_v6 "${connect_addr}"; then
+ return 0
+ fi
+
+ip netns exec "$listener_ns" nft -f /dev/stdin <<"EOF"
+flush ruleset
+table inet mangle {
+ chain divert {
+ type filter hook prerouting priority -150;
+
+ meta l4proto tcp socket transparent 1 meta mark set 1 accept
+ tcp dport 20000 tproxy to :20000 meta mark set 1 accept
+ }
+}
+EOF
+ if [ $? -ne 0 ]; then
+ echo "SKIP: $msg, could not load nft ruleset"
+ return
+ fi
+
+ local local_addr
+ if is_v6 "${connect_addr}"; then
+ local_addr="::"
+ r6flag="-6"
+ else
+ local_addr="0.0.0.0"
+ fi
+
+ ip -net "$listener_ns" $r6flag rule add fwmark 1 lookup 100
+ if [ $? -ne 0 ]; then
+ ip netns exec "$listener_ns" nft flush ruleset
+ echo "SKIP: $msg, ip $r6flag rule failed"
+ return
+ fi
+
+ ip -net "$listener_ns" route add local $local_addr/0 dev lo table 100
+ if [ $? -ne 0 ]; then
+ ip netns exec "$listener_ns" nft flush ruleset
+ ip -net "$listener_ns" $r6flag rule del fwmark 1 lookup 100
+ echo "SKIP: $msg, ip route add local $local_addr failed"
+ return
+ fi
+
+ echo "INFO: test $msg"
+
+ TEST_COUNT=10000
+ local extra_args="-o TRANSPARENT"
+ do_transfer ${listener_ns} ${connector_ns} MPTCP MPTCP \
+ ${connect_addr} ${local_addr} "${extra_args}"
+ lret=$?
+
+ ip netns exec "$listener_ns" nft flush ruleset
+ ip -net "$listener_ns" $r6flag rule del fwmark 1 lookup 100
+ ip -net "$listener_ns" route del local $local_addr/0 dev lo table 100
+
+ if [ $lret -ne 0 ]; then
+ echo "FAIL: $msg, mptcp connection error" 1>&2
+ ret=$lret
+ return 1
+ fi
+
+ echo "PASS: $msg"
+ return 0
+}
+
run_tests_peekmode()
{
local peekmode="$1"
@@ -680,6 +743,33 @@
run_tests_lo "$ns1" "$ns1" dead:beef:1::1 1 "-P ${peekmode}"
}
+run_tests_disconnect()
+{
+ local peekmode="$1"
+ local old_cin=$cin
+ local old_sin=$sin
+
+ cat $cin $cin $cin > "$cin".disconnect
+
+ # force do_transfer to cope with the multiple tranmissions
+ sin="$cin.disconnect"
+ sin_disconnect=$old_sin
+ cin="$cin.disconnect"
+ cin_disconnect="$old_cin"
+ connect_per_transfer=3
+
+ echo "INFO: disconnect"
+ run_tests_lo "$ns1" "$ns1" 10.0.1.1 1 "-I 3 -i $old_cin"
+ run_tests_lo "$ns1" "$ns1" dead:beef:1::1 1 "-I 3 -i $old_cin"
+
+ # restore previous status
+ cout=$old_cout
+ cout_disconnect="$cout".disconnect
+ cin=$old_cin
+ cin_disconnect="$cin".disconnect
+ connect_per_transfer=1
+}
+
display_time()
{
time_end=$(date +%s)
@@ -704,8 +794,6 @@
check_mptcp_disabled
-check_mptcp_ulp_setsockopt
-
stop_if_error "The kernel configuration is not valid for MPTCP"
echo "INFO: validating network environment with pings"
@@ -794,5 +882,12 @@
run_tests_peekmode "saveAfterPeek"
stop_if_error "Tests with peek mode have failed"
+# connect to ns4 ip address, ns2 should intercept/proxy
+run_test_transparent 10.0.3.1 "tproxy ipv4"
+run_test_transparent dead:beef:3::1 "tproxy ipv6"
+stop_if_error "Tests with tproxy have failed"
+
+run_tests_disconnect
+
display_time
exit $ret
diff --git a/tools/testing/selftests/net/mptcp/mptcp_inq.c b/tools/testing/selftests/net/mptcp/mptcp_inq.c
new file mode 100644
index 0000000..29f75e2a
--- /dev/null
+++ b/tools/testing/selftests/net/mptcp/mptcp_inq.c
@@ -0,0 +1,602 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#define _GNU_SOURCE
+
+#include <assert.h>
+#include <errno.h>
+#include <fcntl.h>
+#include <limits.h>
+#include <string.h>
+#include <stdarg.h>
+#include <stdbool.h>
+#include <stdint.h>
+#include <inttypes.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <strings.h>
+#include <unistd.h>
+#include <time.h>
+
+#include <sys/ioctl.h>
+#include <sys/socket.h>
+#include <sys/types.h>
+#include <sys/wait.h>
+
+#include <netdb.h>
+#include <netinet/in.h>
+
+#include <linux/tcp.h>
+#include <linux/sockios.h>
+
+#ifndef IPPROTO_MPTCP
+#define IPPROTO_MPTCP 262
+#endif
+#ifndef SOL_MPTCP
+#define SOL_MPTCP 284
+#endif
+
+static int pf = AF_INET;
+static int proto_tx = IPPROTO_MPTCP;
+static int proto_rx = IPPROTO_MPTCP;
+
+static void die_perror(const char *msg)
+{
+ perror(msg);
+ exit(1);
+}
+
+static void die_usage(int r)
+{
+ fprintf(stderr, "Usage: mptcp_inq [-6] [ -t tcp|mptcp ] [ -r tcp|mptcp]\n");
+ exit(r);
+}
+
+static void xerror(const char *fmt, ...)
+{
+ va_list ap;
+
+ va_start(ap, fmt);
+ vfprintf(stderr, fmt, ap);
+ va_end(ap);
+ fputc('\n', stderr);
+ exit(1);
+}
+
+static const char *getxinfo_strerr(int err)
+{
+ if (err == EAI_SYSTEM)
+ return strerror(errno);
+
+ return gai_strerror(err);
+}
+
+static void xgetaddrinfo(const char *node, const char *service,
+ const struct addrinfo *hints,
+ struct addrinfo **res)
+{
+ int err = getaddrinfo(node, service, hints, res);
+
+ if (err) {
+ const char *errstr = getxinfo_strerr(err);
+
+ fprintf(stderr, "Fatal: getaddrinfo(%s:%s): %s\n",
+ node ? node : "", service ? service : "", errstr);
+ exit(1);
+ }
+}
+
+static int sock_listen_mptcp(const char * const listenaddr,
+ const char * const port)
+{
+ int sock;
+ struct addrinfo hints = {
+ .ai_protocol = IPPROTO_TCP,
+ .ai_socktype = SOCK_STREAM,
+ .ai_flags = AI_PASSIVE | AI_NUMERICHOST
+ };
+
+ hints.ai_family = pf;
+
+ struct addrinfo *a, *addr;
+ int one = 1;
+
+ xgetaddrinfo(listenaddr, port, &hints, &addr);
+ hints.ai_family = pf;
+
+ for (a = addr; a; a = a->ai_next) {
+ sock = socket(a->ai_family, a->ai_socktype, proto_rx);
+ if (sock < 0)
+ continue;
+
+ if (-1 == setsockopt(sock, SOL_SOCKET, SO_REUSEADDR, &one,
+ sizeof(one)))
+ perror("setsockopt");
+
+ if (bind(sock, a->ai_addr, a->ai_addrlen) == 0)
+ break; /* success */
+
+ perror("bind");
+ close(sock);
+ sock = -1;
+ }
+
+ freeaddrinfo(addr);
+
+ if (sock < 0)
+ xerror("could not create listen socket");
+
+ if (listen(sock, 20))
+ die_perror("listen");
+
+ return sock;
+}
+
+static int sock_connect_mptcp(const char * const remoteaddr,
+ const char * const port, int proto)
+{
+ struct addrinfo hints = {
+ .ai_protocol = IPPROTO_TCP,
+ .ai_socktype = SOCK_STREAM,
+ };
+ struct addrinfo *a, *addr;
+ int sock = -1;
+
+ hints.ai_family = pf;
+
+ xgetaddrinfo(remoteaddr, port, &hints, &addr);
+ for (a = addr; a; a = a->ai_next) {
+ sock = socket(a->ai_family, a->ai_socktype, proto);
+ if (sock < 0)
+ continue;
+
+ if (connect(sock, a->ai_addr, a->ai_addrlen) == 0)
+ break; /* success */
+
+ die_perror("connect");
+ }
+
+ if (sock < 0)
+ xerror("could not create connect socket");
+
+ freeaddrinfo(addr);
+ return sock;
+}
+
+static int protostr_to_num(const char *s)
+{
+ if (strcasecmp(s, "tcp") == 0)
+ return IPPROTO_TCP;
+ if (strcasecmp(s, "mptcp") == 0)
+ return IPPROTO_MPTCP;
+
+ die_usage(1);
+ return 0;
+}
+
+static void parse_opts(int argc, char **argv)
+{
+ int c;
+
+ while ((c = getopt(argc, argv, "h6t:r:")) != -1) {
+ switch (c) {
+ case 'h':
+ die_usage(0);
+ break;
+ case '6':
+ pf = AF_INET6;
+ break;
+ case 't':
+ proto_tx = protostr_to_num(optarg);
+ break;
+ case 'r':
+ proto_rx = protostr_to_num(optarg);
+ break;
+ default:
+ die_usage(1);
+ break;
+ }
+ }
+}
+
+/* wait up to timeout milliseconds */
+static void wait_for_ack(int fd, int timeout, size_t total)
+{
+ int i;
+
+ for (i = 0; i < timeout; i++) {
+ int nsd, ret, queued = -1;
+ struct timespec req;
+
+ ret = ioctl(fd, TIOCOUTQ, &queued);
+ if (ret < 0)
+ die_perror("TIOCOUTQ");
+
+ ret = ioctl(fd, SIOCOUTQNSD, &nsd);
+ if (ret < 0)
+ die_perror("SIOCOUTQNSD");
+
+ if ((size_t)queued > total)
+ xerror("TIOCOUTQ %u, but only %zu expected\n", queued, total);
+ assert(nsd <= queued);
+
+ if (queued == 0)
+ return;
+
+ /* wait for peer to ack rx of all data */
+ req.tv_sec = 0;
+ req.tv_nsec = 1 * 1000 * 1000ul; /* 1ms */
+ nanosleep(&req, NULL);
+ }
+
+ xerror("still tx data queued after %u ms\n", timeout);
+}
+
+static void connect_one_server(int fd, int unixfd)
+{
+ size_t len, i, total, sent;
+ char buf[4096], buf2[4096];
+ ssize_t ret;
+
+ len = rand() % (sizeof(buf) - 1);
+
+ if (len < 128)
+ len = 128;
+
+ for (i = 0; i < len ; i++) {
+ buf[i] = rand() % 26;
+ buf[i] += 'A';
+ }
+
+ buf[i] = '\n';
+
+ /* un-block server */
+ ret = read(unixfd, buf2, 4);
+ assert(ret == 4);
+
+ assert(strncmp(buf2, "xmit", 4) == 0);
+
+ ret = write(unixfd, &len, sizeof(len));
+ assert(ret == (ssize_t)sizeof(len));
+
+ ret = write(fd, buf, len);
+ if (ret < 0)
+ die_perror("write");
+
+ if (ret != (ssize_t)len)
+ xerror("short write");
+
+ ret = read(unixfd, buf2, 4);
+ assert(strncmp(buf2, "huge", 4) == 0);
+
+ total = rand() % (16 * 1024 * 1024);
+ total += (1 * 1024 * 1024);
+ sent = total;
+
+ ret = write(unixfd, &total, sizeof(total));
+ assert(ret == (ssize_t)sizeof(total));
+
+ wait_for_ack(fd, 5000, len);
+
+ while (total > 0) {
+ if (total > sizeof(buf))
+ len = sizeof(buf);
+ else
+ len = total;
+
+ ret = write(fd, buf, len);
+ if (ret < 0)
+ die_perror("write");
+ total -= ret;
+
+ /* we don't have to care about buf content, only
+ * number of total bytes sent
+ */
+ }
+
+ ret = read(unixfd, buf2, 4);
+ assert(ret == 4);
+ assert(strncmp(buf2, "shut", 4) == 0);
+
+ wait_for_ack(fd, 5000, sent);
+
+ ret = write(fd, buf, 1);
+ assert(ret == 1);
+ close(fd);
+ ret = write(unixfd, "closed", 6);
+ assert(ret == 6);
+
+ close(unixfd);
+}
+
+static void get_tcp_inq(struct msghdr *msgh, unsigned int *inqv)
+{
+ struct cmsghdr *cmsg;
+
+ for (cmsg = CMSG_FIRSTHDR(msgh); cmsg ; cmsg = CMSG_NXTHDR(msgh, cmsg)) {
+ if (cmsg->cmsg_level == IPPROTO_TCP && cmsg->cmsg_type == TCP_CM_INQ) {
+ memcpy(inqv, CMSG_DATA(cmsg), sizeof(*inqv));
+ return;
+ }
+ }
+
+ xerror("could not find TCP_CM_INQ cmsg type");
+}
+
+static void process_one_client(int fd, int unixfd)
+{
+ unsigned int tcp_inq;
+ size_t expect_len;
+ char msg_buf[4096];
+ char buf[4096];
+ char tmp[16];
+ struct iovec iov = {
+ .iov_base = buf,
+ .iov_len = 1,
+ };
+ struct msghdr msg = {
+ .msg_iov = &iov,
+ .msg_iovlen = 1,
+ .msg_control = msg_buf,
+ .msg_controllen = sizeof(msg_buf),
+ };
+ ssize_t ret, tot;
+
+ ret = write(unixfd, "xmit", 4);
+ assert(ret == 4);
+
+ ret = read(unixfd, &expect_len, sizeof(expect_len));
+ assert(ret == (ssize_t)sizeof(expect_len));
+
+ if (expect_len > sizeof(buf))
+ xerror("expect len %zu exceeds buffer size", expect_len);
+
+ for (;;) {
+ struct timespec req;
+ unsigned int queued;
+
+ ret = ioctl(fd, FIONREAD, &queued);
+ if (ret < 0)
+ die_perror("FIONREAD");
+ if (queued > expect_len)
+ xerror("FIONREAD returned %u, but only %zu expected\n",
+ queued, expect_len);
+ if (queued == expect_len)
+ break;
+
+ req.tv_sec = 0;
+ req.tv_nsec = 1000 * 1000ul;
+ nanosleep(&req, NULL);
+ }
+
+ /* read one byte, expect cmsg to return expected - 1 */
+ ret = recvmsg(fd, &msg, 0);
+ if (ret < 0)
+ die_perror("recvmsg");
+
+ if (msg.msg_controllen == 0)
+ xerror("msg_controllen is 0");
+
+ get_tcp_inq(&msg, &tcp_inq);
+
+ assert((size_t)tcp_inq == (expect_len - 1));
+
+ iov.iov_len = sizeof(buf);
+ ret = recvmsg(fd, &msg, 0);
+ if (ret < 0)
+ die_perror("recvmsg");
+
+ /* should have gotten exact remainder of all pending data */
+ assert(ret == (ssize_t)tcp_inq);
+
+ /* should be 0, all drained */
+ get_tcp_inq(&msg, &tcp_inq);
+ assert(tcp_inq == 0);
+
+ /* request a large swath of data. */
+ ret = write(unixfd, "huge", 4);
+ assert(ret == 4);
+
+ ret = read(unixfd, &expect_len, sizeof(expect_len));
+ assert(ret == (ssize_t)sizeof(expect_len));
+
+ /* peer should send us a few mb of data */
+ if (expect_len <= sizeof(buf))
+ xerror("expect len %zu too small\n", expect_len);
+
+ tot = 0;
+ do {
+ iov.iov_len = sizeof(buf);
+ ret = recvmsg(fd, &msg, 0);
+ if (ret < 0)
+ die_perror("recvmsg");
+
+ tot += ret;
+
+ get_tcp_inq(&msg, &tcp_inq);
+
+ if (tcp_inq > expect_len - tot)
+ xerror("inq %d, remaining %d total_len %d\n",
+ tcp_inq, expect_len - tot, (int)expect_len);
+
+ assert(tcp_inq <= expect_len - tot);
+ } while ((size_t)tot < expect_len);
+
+ ret = write(unixfd, "shut", 4);
+ assert(ret == 4);
+
+ /* wait for hangup. Should have received one more byte of data. */
+ ret = read(unixfd, tmp, sizeof(tmp));
+ assert(ret == 6);
+ assert(strncmp(tmp, "closed", 6) == 0);
+
+ sleep(1);
+
+ iov.iov_len = 1;
+ ret = recvmsg(fd, &msg, 0);
+ if (ret < 0)
+ die_perror("recvmsg");
+ assert(ret == 1);
+
+ get_tcp_inq(&msg, &tcp_inq);
+
+ /* tcp_inq should be 1 due to received fin. */
+ assert(tcp_inq == 1);
+
+ iov.iov_len = 1;
+ ret = recvmsg(fd, &msg, 0);
+ if (ret < 0)
+ die_perror("recvmsg");
+
+ /* expect EOF */
+ assert(ret == 0);
+ get_tcp_inq(&msg, &tcp_inq);
+ assert(tcp_inq == 1);
+
+ close(fd);
+}
+
+static int xaccept(int s)
+{
+ int fd = accept(s, NULL, 0);
+
+ if (fd < 0)
+ die_perror("accept");
+
+ return fd;
+}
+
+static int server(int unixfd)
+{
+ int fd = -1, r, on = 1;
+
+ switch (pf) {
+ case AF_INET:
+ fd = sock_listen_mptcp("127.0.0.1", "15432");
+ break;
+ case AF_INET6:
+ fd = sock_listen_mptcp("::1", "15432");
+ break;
+ default:
+ xerror("Unknown pf %d\n", pf);
+ break;
+ }
+
+ r = write(unixfd, "conn", 4);
+ assert(r == 4);
+
+ alarm(15);
+ r = xaccept(fd);
+
+ if (-1 == setsockopt(r, IPPROTO_TCP, TCP_INQ, &on, sizeof(on)))
+ die_perror("setsockopt");
+
+ process_one_client(r, unixfd);
+
+ return 0;
+}
+
+static int client(int unixfd)
+{
+ int fd = -1;
+
+ alarm(15);
+
+ switch (pf) {
+ case AF_INET:
+ fd = sock_connect_mptcp("127.0.0.1", "15432", proto_tx);
+ break;
+ case AF_INET6:
+ fd = sock_connect_mptcp("::1", "15432", proto_tx);
+ break;
+ default:
+ xerror("Unknown pf %d\n", pf);
+ }
+
+ connect_one_server(fd, unixfd);
+
+ return 0;
+}
+
+static void init_rng(void)
+{
+ int fd = open("/dev/urandom", O_RDONLY);
+ unsigned int foo;
+
+ if (fd > 0) {
+ int ret = read(fd, &foo, sizeof(foo));
+
+ if (ret < 0)
+ srand(fd + foo);
+ close(fd);
+ }
+
+ srand(foo);
+}
+
+static pid_t xfork(void)
+{
+ pid_t p = fork();
+
+ if (p < 0)
+ die_perror("fork");
+ else if (p == 0)
+ init_rng();
+
+ return p;
+}
+
+static int rcheck(int wstatus, const char *what)
+{
+ if (WIFEXITED(wstatus)) {
+ if (WEXITSTATUS(wstatus) == 0)
+ return 0;
+ fprintf(stderr, "%s exited, status=%d\n", what, WEXITSTATUS(wstatus));
+ return WEXITSTATUS(wstatus);
+ } else if (WIFSIGNALED(wstatus)) {
+ xerror("%s killed by signal %d\n", what, WTERMSIG(wstatus));
+ } else if (WIFSTOPPED(wstatus)) {
+ xerror("%s stopped by signal %d\n", what, WSTOPSIG(wstatus));
+ }
+
+ return 111;
+}
+
+int main(int argc, char *argv[])
+{
+ int e1, e2, wstatus;
+ pid_t s, c, ret;
+ int unixfds[2];
+
+ parse_opts(argc, argv);
+
+ e1 = socketpair(AF_UNIX, SOCK_DGRAM, 0, unixfds);
+ if (e1 < 0)
+ die_perror("pipe");
+
+ s = xfork();
+ if (s == 0)
+ return server(unixfds[1]);
+
+ close(unixfds[1]);
+
+ /* wait until server bound a socket */
+ e1 = read(unixfds[0], &e1, 4);
+ assert(e1 == 4);
+
+ c = xfork();
+ if (c == 0)
+ return client(unixfds[0]);
+
+ close(unixfds[0]);
+
+ ret = waitpid(s, &wstatus, 0);
+ if (ret == -1)
+ die_perror("waitpid");
+ e1 = rcheck(wstatus, "server");
+ ret = waitpid(c, &wstatus, 0);
+ if (ret == -1)
+ die_perror("waitpid");
+ e2 = rcheck(wstatus, "client");
+
+ return e1 ? e1 : e2;
+}
diff --git a/tools/testing/selftests/net/mptcp/mptcp_join.sh b/tools/testing/selftests/net/mptcp/mptcp_join.sh
index 7ef639a..27d0eb9af 100755
--- a/tools/testing/selftests/net/mptcp/mptcp_join.sh
+++ b/tools/testing/selftests/net/mptcp/mptcp_join.sh
@@ -238,6 +238,45 @@
[ -z "${1##*:*}" ]
}
+# $1: ns, $2: port
+wait_local_port_listen()
+{
+ local listener_ns="${1}"
+ local port="${2}"
+
+ local port_hex i
+
+ port_hex="$(printf "%04X" "${port}")"
+ for i in $(seq 10); do
+ ip netns exec "${listener_ns}" cat /proc/net/tcp* | \
+ awk "BEGIN {rc=1} {if (\$2 ~ /:${port_hex}\$/ && \$4 ~ /0A/) {rc=0; exit}} END {exit rc}" &&
+ break
+ sleep 0.1
+ done
+}
+
+rm_addr_count()
+{
+ ns=${1}
+
+ ip netns exec ${ns} nstat -as | grep MPTcpExtRmAddr | awk '{print $2}'
+}
+
+# $1: ns, $2: old rm_addr counter in $ns
+wait_rm_addr()
+{
+ local ns="${1}"
+ local old_cnt="${2}"
+ local cnt
+ local i
+
+ for i in $(seq 10); do
+ cnt=$(rm_addr_count ${ns})
+ [ "$cnt" = "${old_cnt}" ] || break
+ sleep 0.1
+ done
+}
+
do_transfer()
{
listener_ns="$1"
@@ -307,7 +346,7 @@
fi
spid=$!
- sleep 1
+ wait_local_port_listen "${listener_ns}" "${port}"
if [ "$test_link_fail" -eq 0 ];then
timeout ${timeout_test} \
@@ -324,10 +363,13 @@
fi
cpid=$!
+ # let the mptcp subflow be established in background before
+ # do endpoint manipulation
+ [ $addr_nr_ns1 = "0" -a $addr_nr_ns2 = "0" ] || sleep 1
+
if [ $addr_nr_ns1 -gt 0 ]; then
let add_nr_ns1=addr_nr_ns1
counter=2
- sleep 1
while [ $add_nr_ns1 -gt 0 ]; do
local addr
if is_v6 "${connect_addr}"; then
@@ -339,7 +381,6 @@
let counter+=1
let add_nr_ns1-=1
done
- sleep 1
elif [ $addr_nr_ns1 -lt 0 ]; then
let rm_nr_ns1=-addr_nr_ns1
if [ $rm_nr_ns1 -lt 8 ]; then
@@ -347,22 +388,19 @@
pos=1
dump=(`ip netns exec ${listener_ns} ./pm_nl_ctl dump`)
if [ ${#dump[@]} -gt 0 ]; then
- sleep 1
-
while [ $counter -le $rm_nr_ns1 ]
do
id=${dump[$pos]}
+ rm_addr=$(rm_addr_count ${connector_ns})
ip netns exec ${listener_ns} ./pm_nl_ctl del $id
- sleep 1
+ wait_rm_addr ${connector_ns} ${rm_addr}
let counter+=1
let pos+=5
done
fi
elif [ $rm_nr_ns1 -eq 8 ]; then
- sleep 1
ip netns exec ${listener_ns} ./pm_nl_ctl flush
elif [ $rm_nr_ns1 -eq 9 ]; then
- sleep 1
ip netns exec ${listener_ns} ./pm_nl_ctl del 0 ${connect_addr}
fi
fi
@@ -373,10 +411,13 @@
addr_nr_ns2=${addr_nr_ns2:9}
fi
+ # if newly added endpoints must be deleted, give the background msk
+ # some time to created them
+ [ $addr_nr_ns1 -gt 0 -a $addr_nr_ns2 -lt 0 ] && sleep 1
+
if [ $addr_nr_ns2 -gt 0 ]; then
let add_nr_ns2=addr_nr_ns2
counter=3
- sleep 1
while [ $add_nr_ns2 -gt 0 ]; do
local addr
if is_v6 "${connect_addr}"; then
@@ -388,7 +429,6 @@
let counter+=1
let add_nr_ns2-=1
done
- sleep 1
elif [ $addr_nr_ns2 -lt 0 ]; then
let rm_nr_ns2=-addr_nr_ns2
if [ $rm_nr_ns2 -lt 8 ]; then
@@ -396,19 +436,18 @@
pos=1
dump=(`ip netns exec ${connector_ns} ./pm_nl_ctl dump`)
if [ ${#dump[@]} -gt 0 ]; then
- sleep 1
-
while [ $counter -le $rm_nr_ns2 ]
do
+ # rm_addr are serialized, allow the previous one to complete
id=${dump[$pos]}
+ rm_addr=$(rm_addr_count ${listener_ns})
ip netns exec ${connector_ns} ./pm_nl_ctl del $id
- sleep 1
+ wait_rm_addr ${listener_ns} ${rm_addr}
let counter+=1
let pos+=5
done
fi
elif [ $rm_nr_ns2 -eq 8 ]; then
- sleep 1
ip netns exec ${connector_ns} ./pm_nl_ctl flush
elif [ $rm_nr_ns2 -eq 9 ]; then
local addr
@@ -417,7 +456,6 @@
else
addr="10.0.1.2"
fi
- sleep 1
ip netns exec ${connector_ns} ./pm_nl_ctl del 0 $addr
fi
fi
@@ -539,6 +577,14 @@
lret=$?
}
+dump_stats()
+{
+ echo Server ns stats
+ ip netns exec $ns1 nstat -as | grep Tcp
+ echo Client ns stats
+ ip netns exec $ns2 nstat -as | grep Tcp
+}
+
chk_csum_nr()
{
local msg=${1:-""}
@@ -570,12 +616,7 @@
else
echo "[ ok ]"
fi
- if [ "${dump_stats}" = 1 ]; then
- echo Server ns stats
- ip netns exec $ns1 nstat -as | grep MPTcp
- echo Client ns stats
- ip netns exec $ns2 nstat -as | grep MPTcp
- fi
+ [ "${dump_stats}" = 1 ] && dump_stats
}
chk_fail_nr()
@@ -607,12 +648,7 @@
echo "[ ok ]"
fi
- if [ "${dump_stats}" = 1 ]; then
- echo Server ns stats
- ip netns exec $ns1 nstat -as | grep MPTcp
- echo Client ns stats
- ip netns exec $ns2 nstat -as | grep MPTcp
- fi
+ [ "${dump_stats}" = 1 ] && dump_stats
}
chk_join_nr()
@@ -656,12 +692,7 @@
else
echo "[ ok ]"
fi
- if [ "${dump_stats}" = 1 ]; then
- echo Server ns stats
- ip netns exec $ns1 nstat -as | grep MPTcp
- echo Client ns stats
- ip netns exec $ns2 nstat -as | grep MPTcp
- fi
+ [ "${dump_stats}" = 1 ] && dump_stats
if [ $checksum -eq 1 ]; then
chk_csum_nr
chk_fail_nr 0 0
@@ -823,12 +854,7 @@
echo ""
fi
- if [ "${dump_stats}" = 1 ]; then
- echo Server ns stats
- ip netns exec $ns1 nstat -as | grep MPTcp
- echo Client ns stats
- ip netns exec $ns2 nstat -as | grep MPTcp
- fi
+ [ "${dump_stats}" = 1 ] && dump_stats
}
chk_rm_nr()
@@ -871,12 +897,7 @@
echo "[ ok ]"
fi
- if [ "${dump_stats}" = 1 ]; then
- echo Server ns stats
- ip netns exec $ns1 nstat -as | grep MPTcp
- echo Client ns stats
- ip netns exec $ns2 nstat -as | grep MPTcp
- fi
+ [ "${dump_stats}" = 1 ] && dump_stats
}
chk_prio_nr()
@@ -908,12 +929,7 @@
echo "[ ok ]"
fi
- if [ "${dump_stats}" = 1 ]; then
- echo Server ns stats
- ip netns exec $ns1 nstat -as | grep MPTcp
- echo Client ns stats
- ip netns exec $ns2 nstat -as | grep MPTcp
- fi
+ [ "${dump_stats}" = 1 ] && dump_stats
}
chk_link_usage()
@@ -937,6 +953,22 @@
fi
}
+wait_for_tw()
+{
+ local timeout_ms=$((timeout_poll * 1000))
+ local time=0
+ local ns=$1
+
+ while [ $time -lt $timeout_ms ]; do
+ local cnt=$(ip netns exec $ns ss -t state time-wait |wc -l)
+
+ [ "$cnt" = 1 ] && return 1
+ time=$((time + 100))
+ sleep 0.1
+ done
+ return 1
+}
+
subflows_tests()
{
reset
@@ -994,6 +1026,61 @@
chk_join_nr "single subflow, dev" 1 1 1
}
+subflows_error_tests()
+{
+ # If a single subflow is configured, and matches the MPC src
+ # address, no additional subflow should be created
+ reset
+ ip netns exec $ns1 ./pm_nl_ctl limits 0 1
+ ip netns exec $ns2 ./pm_nl_ctl limits 0 1
+ ip netns exec $ns2 ./pm_nl_ctl add 10.0.1.2 flags subflow
+ run_tests $ns1 $ns2 10.0.1.1 0 0 0 slow
+ chk_join_nr "no MPC reuse with single endpoint" 0 0 0
+
+ # multiple subflows, with subflow creation error
+ reset
+ ip netns exec $ns1 ./pm_nl_ctl limits 0 2
+ ip netns exec $ns2 ./pm_nl_ctl limits 0 2
+ ip netns exec $ns2 ./pm_nl_ctl add 10.0.3.2 flags subflow
+ ip netns exec $ns2 ./pm_nl_ctl add 10.0.2.2 flags subflow
+ ip netns exec $ns1 iptables -A INPUT -s 10.0.3.2 -p tcp -j REJECT
+ run_tests $ns1 $ns2 10.0.1.1 0 0 0 slow
+ chk_join_nr "multi subflows, with failing subflow" 1 1 1
+
+ # multiple subflows, with subflow timeout on MPJ
+ reset
+ ip netns exec $ns1 ./pm_nl_ctl limits 0 2
+ ip netns exec $ns2 ./pm_nl_ctl limits 0 2
+ ip netns exec $ns2 ./pm_nl_ctl add 10.0.3.2 flags subflow
+ ip netns exec $ns2 ./pm_nl_ctl add 10.0.2.2 flags subflow
+ ip netns exec $ns1 iptables -A INPUT -s 10.0.3.2 -p tcp -j DROP
+ run_tests $ns1 $ns2 10.0.1.1 0 0 0 slow
+ chk_join_nr "multi subflows, with subflow timeout" 1 1 1
+
+ # multiple subflows, check that the endpoint corresponding to
+ # closed subflow (due to reset) is not reused if additional
+ # subflows are added later
+ reset
+ ip netns exec $ns1 ./pm_nl_ctl limits 0 1
+ ip netns exec $ns2 ./pm_nl_ctl limits 0 1
+ ip netns exec $ns2 ./pm_nl_ctl add 10.0.3.2 flags subflow
+ ip netns exec $ns1 iptables -A INPUT -s 10.0.3.2 -p tcp -j REJECT
+ run_tests $ns1 $ns2 10.0.1.1 0 0 0 slow &
+
+ # updates in the child shell do not have any effect here, we
+ # need to bump the test counter for the above case
+ TEST_COUNT=$((TEST_COUNT+1))
+
+ # mpj subflow will be in TW after the reset
+ wait_for_tw $ns2
+ ip netns exec $ns2 ./pm_nl_ctl add 10.0.2.2 flags subflow
+ wait
+
+ # additional subflow could be created only if the PM select
+ # the later endpoint, skipping the already used one
+ chk_join_nr "multi subflows, fair usage on close" 1 1 1
+}
+
signal_address_tests()
{
# add_address, unused
@@ -1071,7 +1158,10 @@
ip netns exec $ns2 ./pm_nl_ctl add 10.0.3.2 flags signal
ip netns exec $ns2 ./pm_nl_ctl add 10.0.4.2 flags signal
run_tests $ns1 $ns2 10.0.1.1
- chk_add_nr 4 4
+
+ # the server will not signal the address terminating
+ # the MPC subflow
+ chk_add_nr 3 3
}
link_failure_tests()
@@ -1577,7 +1667,7 @@
ip netns exec $ns2 ./pm_nl_ctl limits 1 3
ip netns exec $ns2 ./pm_nl_ctl add 10.0.3.2 flags subflow
ip netns exec $ns2 ./pm_nl_ctl add 10.0.4.2 flags subflow
- run_tests $ns1 $ns2 10.0.1.1 0 -8 -8 slow
+ run_tests $ns1 $ns2 10.0.1.1 0 -8 -2 slow
chk_join_nr "flush subflows and signal with port" 3 3 3
chk_add_nr 1 1
chk_rm_nr 2 2
@@ -1802,6 +1892,7 @@
all_tests()
{
subflows_tests
+ subflows_error_tests
signal_address_tests
link_failure_tests
add_addr_timeout_tests
@@ -1821,6 +1912,7 @@
{
echo "mptcp_join usage:"
echo " -f subflows_tests"
+ echo " -e subflows_error_tests"
echo " -s signal_address_tests"
echo " -l link_failure_tests"
echo " -t add_addr_timeout_tests"
@@ -1869,11 +1961,14 @@
exit $ret
fi
-while getopts 'fsltra64bpkdmchCS' opt; do
+while getopts 'fesltra64bpkdmchCS' opt; do
case $opt in
f)
subflows_tests
;;
+ e)
+ subflows_error_tests
+ ;;
s)
signal_address_tests
;;
diff --git a/tools/testing/selftests/net/mptcp/mptcp_sockopt.c b/tools/testing/selftests/net/mptcp/mptcp_sockopt.c
index 417b11c..ac9a4d9 100644
--- a/tools/testing/selftests/net/mptcp/mptcp_sockopt.c
+++ b/tools/testing/selftests/net/mptcp/mptcp_sockopt.c
@@ -4,6 +4,7 @@
#include <assert.h>
#include <errno.h>
+#include <fcntl.h>
#include <limits.h>
#include <string.h>
#include <stdarg.h>
@@ -13,6 +14,7 @@
#include <stdio.h>
#include <stdlib.h>
#include <strings.h>
+#include <time.h>
#include <unistd.h>
#include <sys/socket.h>
@@ -594,6 +596,44 @@ static int server(int pipefd)
return 0;
}
+static void test_ip_tos_sockopt(int fd)
+{
+ uint8_t tos_in, tos_out;
+ socklen_t s;
+ int r;
+
+ tos_in = rand() & 0xfc;
+ r = setsockopt(fd, SOL_IP, IP_TOS, &tos_in, sizeof(tos_out));
+ if (r != 0)
+ die_perror("setsockopt IP_TOS");
+
+ tos_out = 0;
+ s = sizeof(tos_out);
+ r = getsockopt(fd, SOL_IP, IP_TOS, &tos_out, &s);
+ if (r != 0)
+ die_perror("getsockopt IP_TOS");
+
+ if (tos_in != tos_out)
+ xerror("tos %x != %x socklen_t %d\n", tos_in, tos_out, s);
+
+ if (s != 1)
+ xerror("tos should be 1 byte");
+
+ s = 0;
+ r = getsockopt(fd, SOL_IP, IP_TOS, &tos_out, &s);
+ if (r != 0)
+ die_perror("getsockopt IP_TOS 0");
+ if (s != 0)
+ xerror("expect socklen_t == 0");
+
+ s = -1;
+ r = getsockopt(fd, SOL_IP, IP_TOS, &tos_out, &s);
+ if (r != -1 && errno != EINVAL)
+ die_perror("getsockopt IP_TOS did not indicate -EINVAL");
+ if (s != -1)
+ xerror("expect socklen_t == -1");
+}
+
static int client(int pipefd)
{
int fd = -1;
@@ -611,6 +651,8 @@ static int client(int pipefd)
xerror("Unknown pf %d\n", pf);
}
+ test_ip_tos_sockopt(fd);
+
connect_one_server(fd, pipefd);
return 0;
@@ -642,6 +684,25 @@ static int rcheck(int wstatus, const char *what)
return 111;
}
+static void init_rng(void)
+{
+ int fd = open("/dev/urandom", O_RDONLY);
+
+ if (fd >= 0) {
+ unsigned int foo;
+ ssize_t ret;
+
+ /* can't fail */
+ ret = read(fd, &foo, sizeof(foo));
+ assert(ret == sizeof(foo));
+
+ close(fd);
+ srand(foo);
+ } else {
+ srand(time(NULL));
+ }
+}
+
int main(int argc, char *argv[])
{
int e1, e2, wstatus;
@@ -650,6 +711,8 @@ int main(int argc, char *argv[])
parse_opts(argc, argv);
+ init_rng();
+
e1 = pipe(pipefds);
if (e1 < 0)
die_perror("pipe");
diff --git a/tools/testing/selftests/net/mptcp/mptcp_sockopt.sh b/tools/testing/selftests/net/mptcp/mptcp_sockopt.sh
index 41de643..0879da9 100755
--- a/tools/testing/selftests/net/mptcp/mptcp_sockopt.sh
+++ b/tools/testing/selftests/net/mptcp/mptcp_sockopt.sh
@@ -178,7 +178,7 @@
timeout ${timeout_test} \
ip netns exec ${listener_ns} \
- $mptcp_connect -t ${timeout_poll} -l -M 1 -p $port -s ${srv_proto} -c TIMESTAMPNS \
+ $mptcp_connect -t ${timeout_poll} -l -M 1 -p $port -s ${srv_proto} -c TIMESTAMPNS,TCPINQ \
${local_addr} < "$sin" > "$sout" &
spid=$!
@@ -186,7 +186,7 @@
timeout ${timeout_test} \
ip netns exec ${connector_ns} \
- $mptcp_connect -t ${timeout_poll} -M 2 -p $port -s ${cl_proto} -c TIMESTAMPNS \
+ $mptcp_connect -t ${timeout_poll} -M 2 -p $port -s ${cl_proto} -c TIMESTAMPNS,TCPINQ \
$connect_addr < "$cin" > "$cout" &
cpid=$!
@@ -279,6 +279,45 @@
fi
}
+do_tcpinq_test()
+{
+ ip netns exec "$ns1" ./mptcp_inq "$@"
+ lret=$?
+ if [ $lret -ne 0 ];then
+ ret=$lret
+ echo "FAIL: mptcp_inq $@" 1>&2
+ return $lret
+ fi
+
+ echo "PASS: TCP_INQ cmsg/ioctl $@"
+ return $lret
+}
+
+do_tcpinq_tests()
+{
+ local lret=0
+
+ ip netns exec "$ns1" iptables -F
+ ip netns exec "$ns1" ip6tables -F
+
+ for args in "-t tcp" "-r tcp"; do
+ do_tcpinq_test $args
+ lret=$?
+ if [ $lret -ne 0 ] ; then
+ return $lret
+ fi
+ do_tcpinq_test -6 $args
+ lret=$?
+ if [ $lret -ne 0 ] ; then
+ return $lret
+ fi
+ done
+
+ do_tcpinq_test -r tcp -t tcp
+
+ return $?
+}
+
sin=$(mktemp)
sout=$(mktemp)
cin=$(mktemp)
@@ -300,4 +339,5 @@
echo "PASS: SOL_MPTCP getsockopt has expected information"
fi
+do_tcpinq_tests
exit $ret
diff --git a/tools/testing/selftests/net/nettest.c b/tools/testing/selftests/net/nettest.c
index b599003..d9a6fd2 100644
--- a/tools/testing/selftests/net/nettest.c
+++ b/tools/testing/selftests/net/nettest.c
@@ -85,6 +85,7 @@ struct sock_args {
int version; /* AF_INET/AF_INET6 */
int use_setsockopt;
+ int use_freebind;
int use_cmsg;
const char *dev;
const char *server_dev;
@@ -514,6 +515,29 @@ static int set_membership(int sd, uint32_t grp, uint32_t addr, int ifindex)
return 0;
}
+static int set_freebind(int sd, int version)
+{
+ unsigned int one = 1;
+ int rc = 0;
+
+ switch (version) {
+ case AF_INET:
+ if (setsockopt(sd, SOL_IP, IP_FREEBIND, &one, sizeof(one))) {
+ log_err_errno("setsockopt(IP_FREEBIND)");
+ rc = -1;
+ }
+ break;
+ case AF_INET6:
+ if (setsockopt(sd, SOL_IPV6, IPV6_FREEBIND, &one, sizeof(one))) {
+ log_err_errno("setsockopt(IPV6_FREEBIND");
+ rc = -1;
+ }
+ break;
+ }
+
+ return rc;
+}
+
static int set_broadcast(int sd)
{
unsigned int one = 1;
@@ -1419,6 +1443,9 @@ static int lsock_init(struct sock_args *args)
set_unicast_if(sd, args->ifindex, args->version))
goto err;
+ if (args->use_freebind && set_freebind(sd, args->version))
+ goto err;
+
if (bind_socket(sd, args))
goto err;
@@ -1827,7 +1854,7 @@ static int ipc_parent(int cpid, int fd, struct sock_args *args)
return client_status;
}
-#define GETOPT_STR "sr:l:c:p:t:g:P:DRn:M:X:m:d:I:BN:O:SCi6xL:0:1:2:3:Fbq"
+#define GETOPT_STR "sr:l:c:p:t:g:P:DRn:M:X:m:d:I:BN:O:SCi6xL:0:1:2:3:Fbqf"
#define OPT_FORCE_BIND_KEY_IFINDEX 1001
#define OPT_NO_BIND_KEY_IFINDEX 1002
@@ -1864,6 +1891,7 @@ static void print_usage(char *prog)
" -I dev bind socket to given device name - server mode\n"
" -S use setsockopt (IP_UNICAST_IF or IP_MULTICAST_IF)\n"
" to set device binding\n"
+ " -f bind socket with the IP[V6]_FREEBIND option\n"
" -C use cmsg and IP_PKTINFO to specify device binding\n"
"\n"
" -L len send random message of given length\n"
@@ -1999,6 +2027,9 @@ int main(int argc, char *argv[])
case 'S':
args.use_setsockopt = 1;
break;
+ case 'f':
+ args.use_freebind = 1;
+ break;
case 'C':
args.use_cmsg = 1;
break;
diff --git a/tools/testing/selftests/net/reuseport_bpf.c b/tools/testing/selftests/net/reuseport_bpf.c
index b527710..072d709c 100644
--- a/tools/testing/selftests/net/reuseport_bpf.c
+++ b/tools/testing/selftests/net/reuseport_bpf.c
@@ -24,9 +24,7 @@
#include <sys/resource.h>
#include <unistd.h>
-#ifndef ARRAY_SIZE
-#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]))
-#endif
+#include "../kselftest.h"
struct test_params {
int recv_family;
diff --git a/tools/testing/selftests/net/rxtimestamp.c b/tools/testing/selftests/net/rxtimestamp.c
index e4613ce..9eb4257 100644
--- a/tools/testing/selftests/net/rxtimestamp.c
+++ b/tools/testing/selftests/net/rxtimestamp.c
@@ -18,7 +18,7 @@
#include <linux/net_tstamp.h>
#include <linux/errqueue.h>
-#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]))
+#include "../kselftest.h"
struct options {
int so_timestamp;
diff --git a/tools/testing/selftests/net/socket.c b/tools/testing/selftests/net/socket.c
index afca1ead..db1aeb8 100644
--- a/tools/testing/selftests/net/socket.c
+++ b/tools/testing/selftests/net/socket.c
@@ -7,6 +7,8 @@
#include <sys/socket.h>
#include <netinet/in.h>
+#include "../kselftest.h"
+
struct socket_testcase {
int domain;
int type;
@@ -31,7 +33,6 @@ static struct socket_testcase tests[] = {
{ AF_INET, SOCK_STREAM, IPPROTO_UDP, -EPROTONOSUPPORT, 1 },
};
-#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]))
#define ERR_STRING_SZ 64
static int run_tests(void)
diff --git a/tools/testing/selftests/net/tcp_fastopen_backup_key.c b/tools/testing/selftests/net/tcp_fastopen_backup_key.c
index 9c55ec4..c1cb0c7 100644
--- a/tools/testing/selftests/net/tcp_fastopen_backup_key.c
+++ b/tools/testing/selftests/net/tcp_fastopen_backup_key.c
@@ -26,6 +26,8 @@
#include <fcntl.h>
#include <time.h>
+#include "../kselftest.h"
+
#ifndef TCP_FASTOPEN_KEY
#define TCP_FASTOPEN_KEY 33
#endif
@@ -34,10 +36,6 @@
#define PROC_FASTOPEN_KEY "/proc/sys/net/ipv4/tcp_fastopen_key"
#define KEY_LENGTH 16
-#ifndef ARRAY_SIZE
-#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]))
-#endif
-
static bool do_ipv6;
static bool do_sockopt;
static bool do_rotate;
diff --git a/tools/testing/selftests/net/udpgro_fwd.sh b/tools/testing/selftests/net/udpgro_fwd.sh
index 7f26591..6f05e06 100755
--- a/tools/testing/selftests/net/udpgro_fwd.sh
+++ b/tools/testing/selftests/net/udpgro_fwd.sh
@@ -132,7 +132,7 @@
local rcv=`ip netns exec $NS_DST $ipt"-save" -c | grep 'dport 8000' | \
sed -e 's/\[//' -e 's/:.*//'`
if [ $rcv != $pkts ]; then
- echo " fail - received $rvs packets, expected $pkts"
+ echo " fail - received $rcv packets, expected $pkts"
ret=1
return
fi
@@ -185,6 +185,7 @@
IPT=iptables
SUFFIX=24
VXDEV=vxlan
+ PING=ping
if [ $family = 6 ]; then
BM_NET=$BM_NET_V6
@@ -192,6 +193,8 @@
SUFFIX="64 nodad"
VXDEV=vxlan6
IPT=ip6tables
+ # Use ping6 on systems where ping doesn't handle IPv6
+ ping -w 1 -c 1 ::1 > /dev/null 2>&1 || PING="ping6"
fi
echo "IPv$family"
@@ -237,7 +240,7 @@
# load arp cache before running the test to reduce the amount of
# stray traffic on top of the UDP tunnel
- ip netns exec $NS_SRC ping -q -c 1 $OL_NET$DST_NAT >/dev/null
+ ip netns exec $NS_SRC $PING -q -c 1 $OL_NET$DST_NAT >/dev/null
run_test "GRO fwd over UDP tunnel" $OL_NET$DST_NAT 1 1 $OL_NET$DST
cleanup
diff --git a/tools/testing/selftests/net/udpgso.c b/tools/testing/selftests/net/udpgso.c
index c66da6ff..7badaf2 100644
--- a/tools/testing/selftests/net/udpgso.c
+++ b/tools/testing/selftests/net/udpgso.c
@@ -156,13 +156,13 @@ struct testcase testcases_v4[] = {
},
{
/* send max number of min sized segments */
- .tlen = UDP_MAX_SEGMENTS - CONST_HDRLEN_V4,
+ .tlen = UDP_MAX_SEGMENTS,
.gso_len = 1,
- .r_num_mss = UDP_MAX_SEGMENTS - CONST_HDRLEN_V4,
+ .r_num_mss = UDP_MAX_SEGMENTS,
},
{
/* send max number + 1 of min sized segments: fail */
- .tlen = UDP_MAX_SEGMENTS - CONST_HDRLEN_V4 + 1,
+ .tlen = UDP_MAX_SEGMENTS + 1,
.gso_len = 1,
.tfail = true,
},
@@ -259,13 +259,13 @@ struct testcase testcases_v6[] = {
},
{
/* send max number of min sized segments */
- .tlen = UDP_MAX_SEGMENTS - CONST_HDRLEN_V6,
+ .tlen = UDP_MAX_SEGMENTS,
.gso_len = 1,
- .r_num_mss = UDP_MAX_SEGMENTS - CONST_HDRLEN_V6,
+ .r_num_mss = UDP_MAX_SEGMENTS,
},
{
/* send max number + 1 of min sized segments: fail */
- .tlen = UDP_MAX_SEGMENTS - CONST_HDRLEN_V6 + 1,
+ .tlen = UDP_MAX_SEGMENTS + 1,
.gso_len = 1,
.tfail = true,
},
diff --git a/tools/testing/selftests/net/udpgso_bench_tx.c b/tools/testing/selftests/net/udpgso_bench_tx.c
index 17512a4..f1fdaa2 100644
--- a/tools/testing/selftests/net/udpgso_bench_tx.c
+++ b/tools/testing/selftests/net/udpgso_bench_tx.c
@@ -419,6 +419,7 @@ static void usage(const char *filepath)
static void parse_opts(int argc, char **argv)
{
+ const char *bind_addr = NULL;
int max_len, hdrlen;
int c;
@@ -446,7 +447,7 @@ static void parse_opts(int argc, char **argv)
cfg_cpu = strtol(optarg, NULL, 0);
break;
case 'D':
- setup_sockaddr(cfg_family, optarg, &cfg_dst_addr);
+ bind_addr = optarg;
break;
case 'l':
cfg_runtime_ms = strtoul(optarg, NULL, 10) * 1000;
@@ -492,6 +493,11 @@ static void parse_opts(int argc, char **argv)
}
}
+ if (!bind_addr)
+ bind_addr = cfg_family == PF_INET6 ? "::" : "0.0.0.0";
+
+ setup_sockaddr(cfg_family, bind_addr, &cfg_dst_addr);
+
if (optind != argc)
usage(argv[0]);
diff --git a/tools/testing/selftests/netfilter/ipip-conntrack-mtu.sh b/tools/testing/selftests/netfilter/ipip-conntrack-mtu.sh
index 4a6f5c3..eb9553e4 100755
--- a/tools/testing/selftests/netfilter/ipip-conntrack-mtu.sh
+++ b/tools/testing/selftests/netfilter/ipip-conntrack-mtu.sh
@@ -41,7 +41,7 @@
checktool "iptables --version" "run test without iptables"
checktool "ip -Version" "run test without ip tool"
-checktool "which nc" "run test without nc (netcat)"
+checktool "which socat" "run test without socat"
checktool "ip netns add ${r_a}" "create net namespace"
for n in ${r_b} ${r_w} ${c_a} ${c_b};do
@@ -60,11 +60,12 @@
test_path() {
msg="$1"
- ip netns exec ${c_b} nc -n -w 3 -q 3 -u -l -p 5000 > ${rx} < /dev/null &
+ ip netns exec ${c_b} socat -t 3 - udp4-listen:5000,reuseaddr > ${rx} < /dev/null &
sleep 1
for i in 1 2 3; do
- head -c1400 /dev/zero | tr "\000" "a" | ip netns exec ${c_a} nc -n -w 1 -u 192.168.20.2 5000
+ head -c1400 /dev/zero | tr "\000" "a" | \
+ ip netns exec ${c_a} socat -t 1 -u STDIN UDP:192.168.20.2:5000
done
wait
@@ -189,7 +190,7 @@
#---------------------
#Now we send a 1400 bytes UDP packet from Client A to Client B:
-# clienta:~# head -c1400 /dev/zero | tr "\000" "a" | nc -u 192.168.20.2 5000
+# clienta:~# head -c1400 /dev/zero | tr "\000" "a" | socat -u STDIN UDP:192.168.20.2:5000
test_path "without"
# The IPv4 stack on Client A already knows the PMTU to Client B, so the
diff --git a/tools/testing/selftests/netfilter/nf_nat_edemux.sh b/tools/testing/selftests/netfilter/nf_nat_edemux.sh
index cfee3b6..1092bbc 100755
--- a/tools/testing/selftests/netfilter/nf_nat_edemux.sh
+++ b/tools/testing/selftests/netfilter/nf_nat_edemux.sh
@@ -76,23 +76,23 @@
sleep 1
# add a persistent connection from the other namespace
-ip netns exec $ns2 nc -q 10 -w 10 192.168.1.1 5201 > /dev/null &
+ip netns exec $ns2 socat -t 10 - TCP:192.168.1.1:5201 > /dev/null &
sleep 1
# ip daddr:dport will be rewritten to 192.168.1.1 5201
# NAT must reallocate source port 10000 because
# 192.168.1.2:10000 -> 192.168.1.1:5201 is already in use
-echo test | ip netns exec $ns2 nc -w 3 -q 3 10.96.0.1 443 >/dev/null
+echo test | ip netns exec $ns2 socat -t 3 -u STDIN TCP:10.96.0.1:443 >/dev/null
ret=$?
kill $iperfs
-# Check nc can connect to 10.96.0.1:443 (aka 192.168.1.1:5201).
+# Check socat can connect to 10.96.0.1:443 (aka 192.168.1.1:5201).
if [ $ret -eq 0 ]; then
- echo "PASS: nc can connect via NAT'd address"
+ echo "PASS: socat can connect via NAT'd address"
else
- echo "FAIL: nc cannot connect via NAT'd address"
+ echo "FAIL: socat cannot connect via NAT'd address"
exit 1
fi
diff --git a/tools/testing/selftests/netfilter/nft_nat.sh b/tools/testing/selftests/netfilter/nft_nat.sh
index d88867d..349a319 100755
--- a/tools/testing/selftests/netfilter/nft_nat.sh
+++ b/tools/testing/selftests/netfilter/nft_nat.sh
@@ -880,8 +880,9 @@
return $ksft_skip
fi
- # test default behaviour. Packet from ns1 to ns0 is redirected to ns2.
- test_port_shadow "default" "CLIENT"
+ # test default behaviour. Packet from ns1 to ns0 is not redirected
+ # due to automatic port translation.
+ test_port_shadow "default" "ROUTER"
# test packet filter based mitigation: prevent forwarding of
# packets claiming to come from the service port.
diff --git a/tools/testing/selftests/ptp/testptp.c b/tools/testing/selftests/ptp/testptp.c
index f7911aa..c0f6a06 100644
--- a/tools/testing/selftests/ptp/testptp.c
+++ b/tools/testing/selftests/ptp/testptp.c
@@ -354,6 +354,18 @@ int main(int argc, char *argv[])
}
}
+ if (pin_index >= 0) {
+ memset(&desc, 0, sizeof(desc));
+ desc.index = pin_index;
+ desc.func = pin_func;
+ desc.chan = index;
+ if (ioctl(fd, PTP_PIN_SETFUNC, &desc)) {
+ perror("PTP_PIN_SETFUNC");
+ } else {
+ puts("set pin function okay");
+ }
+ }
+
if (extts) {
memset(&extts_request, 0, sizeof(extts_request));
extts_request.index = index;
@@ -444,18 +456,6 @@ int main(int argc, char *argv[])
}
}
- if (pin_index >= 0) {
- memset(&desc, 0, sizeof(desc));
- desc.index = pin_index;
- desc.func = pin_func;
- desc.chan = index;
- if (ioctl(fd, PTP_PIN_SETFUNC, &desc)) {
- perror("PTP_PIN_SETFUNC");
- } else {
- puts("set pin function okay");
- }
- }
-
if (pps != -1) {
int enable = pps ? 1 : 0;
if (ioctl(fd, PTP_ENABLE_PPS, enable)) {
diff --git a/tools/testing/selftests/rseq/basic_percpu_ops_test.c b/tools/testing/selftests/rseq/basic_percpu_ops_test.c
index eb3f6db..b953a52 100644
--- a/tools/testing/selftests/rseq/basic_percpu_ops_test.c
+++ b/tools/testing/selftests/rseq/basic_percpu_ops_test.c
@@ -9,10 +9,9 @@
#include <string.h>
#include <stddef.h>
+#include "../kselftest.h"
#include "rseq.h"
-#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]))
-
struct percpu_lock_entry {
intptr_t v;
} __attribute__((aligned(128)));
diff --git a/tools/testing/selftests/rseq/rseq.c b/tools/testing/selftests/rseq/rseq.c
index 7159eb7..fb440df 100644
--- a/tools/testing/selftests/rseq/rseq.c
+++ b/tools/testing/selftests/rseq/rseq.c
@@ -27,10 +27,9 @@
#include <signal.h>
#include <limits.h>
+#include "../kselftest.h"
#include "rseq.h"
-#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]))
-
__thread volatile struct rseq __rseq_abi = {
.cpu_id = RSEQ_CPU_ID_UNINITIALIZED,
};
diff --git a/tools/testing/selftests/seccomp/seccomp_benchmark.c b/tools/testing/selftests/seccomp/seccomp_benchmark.c
index 6e5102a..5b5c9d5 100644
--- a/tools/testing/selftests/seccomp/seccomp_benchmark.c
+++ b/tools/testing/selftests/seccomp/seccomp_benchmark.c
@@ -18,7 +18,7 @@
#include <sys/syscall.h>
#include <sys/types.h>
-#define ARRAY_SIZE(a) (sizeof(a) / sizeof(a[0]))
+#include "../kselftest.h"
unsigned long long timing(clockid_t clk_id, unsigned long long samples)
{
diff --git a/tools/testing/selftests/seccomp/seccomp_bpf.c b/tools/testing/selftests/seccomp/seccomp_bpf.c
index d425688..9d126d7 100644
--- a/tools/testing/selftests/seccomp/seccomp_bpf.c
+++ b/tools/testing/selftests/seccomp/seccomp_bpf.c
@@ -1487,7 +1487,7 @@ TEST_F(precedence, log_is_fifth_in_any_order)
#define PTRACE_EVENT_SECCOMP 7
#endif
-#define IS_SECCOMP_EVENT(status) ((status >> 16) == PTRACE_EVENT_SECCOMP)
+#define PTRACE_EVENT_MASK(status) ((status) >> 16)
bool tracer_running;
void tracer_stop(int sig)
{
@@ -1539,12 +1539,22 @@ void start_tracer(struct __test_metadata *_metadata, int fd, pid_t tracee,
if (wait(&status) != tracee)
continue;
- if (WIFSIGNALED(status) || WIFEXITED(status))
- /* Child is dead. Time to go. */
- return;
- /* Check if this is a seccomp event. */
- ASSERT_EQ(!ptrace_syscall, IS_SECCOMP_EVENT(status));
+ if (WIFSIGNALED(status)) {
+ /* Child caught a fatal signal. */
+ return;
+ }
+ if (WIFEXITED(status)) {
+ /* Child exited with code. */
+ return;
+ }
+
+ /* Check if we got an expected event. */
+ ASSERT_EQ(WIFCONTINUED(status), false);
+ ASSERT_EQ(WIFSTOPPED(status), true);
+ ASSERT_EQ(WSTOPSIG(status) & SIGTRAP, SIGTRAP) {
+ TH_LOG("Unexpected WSTOPSIG: %d", WSTOPSIG(status));
+ }
tracer_func(_metadata, tracee, status, args);
@@ -1961,6 +1971,11 @@ void tracer_seccomp(struct __test_metadata *_metadata, pid_t tracee,
int ret;
unsigned long msg;
+ EXPECT_EQ(PTRACE_EVENT_MASK(status), PTRACE_EVENT_SECCOMP) {
+ TH_LOG("Unexpected ptrace event: %d", PTRACE_EVENT_MASK(status));
+ return;
+ }
+
/* Make sure we got the right message. */
ret = ptrace(PTRACE_GETEVENTMSG, tracee, NULL, &msg);
EXPECT_EQ(0, ret);
@@ -2011,6 +2026,11 @@ void tracer_ptrace(struct __test_metadata *_metadata, pid_t tracee,
long *syscall_nr = NULL, *syscall_ret = NULL;
FIXTURE_DATA(TRACE_syscall) *self = args;
+ EXPECT_EQ(WSTOPSIG(status) & 0x80, 0x80) {
+ TH_LOG("Unexpected WSTOPSIG: %d", WSTOPSIG(status));
+ return;
+ }
+
/*
* The traditional way to tell PTRACE_SYSCALL entry/exit
* is by counting.
@@ -2128,6 +2148,7 @@ FIXTURE_SETUP(TRACE_syscall)
ret = prctl(PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0);
ASSERT_EQ(0, ret);
+ /* Do not install seccomp rewrite filters, as we'll use ptrace instead. */
if (variant->use_ptrace)
return;
@@ -2186,6 +2207,29 @@ TEST_F(TRACE_syscall, syscall_faked)
EXPECT_SYSCALL_RETURN(45000, syscall(__NR_gettid));
}
+TEST_F_SIGNAL(TRACE_syscall, kill_immediate, SIGSYS)
+{
+ struct sock_filter filter[] = {
+ BPF_STMT(BPF_LD|BPF_W|BPF_ABS,
+ offsetof(struct seccomp_data, nr)),
+ BPF_JUMP(BPF_JMP|BPF_JEQ|BPF_K, __NR_mknodat, 0, 1),
+ BPF_STMT(BPF_RET|BPF_K, SECCOMP_RET_KILL_THREAD),
+ BPF_STMT(BPF_RET|BPF_K, SECCOMP_RET_ALLOW),
+ };
+ struct sock_fprog prog = {
+ .len = (unsigned short)ARRAY_SIZE(filter),
+ .filter = filter,
+ };
+ long ret;
+
+ /* Install "kill on mknodat" filter. */
+ ret = prctl(PR_SET_SECCOMP, SECCOMP_MODE_FILTER, &prog, 0, 0);
+ ASSERT_EQ(0, ret);
+
+ /* This should immediately die with SIGSYS, regardless of tracer. */
+ EXPECT_EQ(-1, syscall(__NR_mknodat, -1, NULL, 0, 0));
+}
+
TEST_F(TRACE_syscall, skip_after)
{
struct sock_filter filter[] = {
@@ -4087,7 +4131,7 @@ TEST(user_notification_addfd)
* lowest available fd to be assigned here.
*/
EXPECT_EQ(fd, nextfd++);
- EXPECT_EQ(filecmp(getpid(), pid, memfd, fd), 0);
+ ASSERT_EQ(filecmp(getpid(), pid, memfd, fd), 0);
/*
* This sets the ID of the ADD FD to the last request plus 1. The
diff --git a/tools/testing/selftests/sgx/Makefile b/tools/testing/selftests/sgx/Makefile
index 7f12d55..2956584 100644
--- a/tools/testing/selftests/sgx/Makefile
+++ b/tools/testing/selftests/sgx/Makefile
@@ -45,7 +45,7 @@
$(CC) $(HOST_CFLAGS) -c $< -o $@
$(OUTPUT)/test_encl.elf: test_encl.lds test_encl.c test_encl_bootstrap.S
- $(CC) $(ENCL_CFLAGS) -T $^ -o $@
+ $(CC) $(ENCL_CFLAGS) -T $^ -o $@ -Wl,--build-id=none
EXTRA_CLEAN := \
$(OUTPUT)/test_encl.elf \
diff --git a/tools/testing/selftests/sgx/defines.h b/tools/testing/selftests/sgx/defines.h
index f88562a..02d7757 100644
--- a/tools/testing/selftests/sgx/defines.h
+++ b/tools/testing/selftests/sgx/defines.h
@@ -19,13 +19,38 @@
#include "../../../../arch/x86/include/uapi/asm/sgx.h"
enum encl_op_type {
- ENCL_OP_PUT,
- ENCL_OP_GET,
+ ENCL_OP_PUT_TO_BUFFER,
+ ENCL_OP_GET_FROM_BUFFER,
+ ENCL_OP_PUT_TO_ADDRESS,
+ ENCL_OP_GET_FROM_ADDRESS,
+ ENCL_OP_NOP,
+ ENCL_OP_MAX,
};
-struct encl_op {
+struct encl_op_header {
uint64_t type;
- uint64_t buffer;
+};
+
+struct encl_op_put_to_buf {
+ struct encl_op_header header;
+ uint64_t value;
+};
+
+struct encl_op_get_from_buf {
+ struct encl_op_header header;
+ uint64_t value;
+};
+
+struct encl_op_put_to_addr {
+ struct encl_op_header header;
+ uint64_t value;
+ uint64_t addr;
+};
+
+struct encl_op_get_from_addr {
+ struct encl_op_header header;
+ uint64_t value;
+ uint64_t addr;
};
#endif /* DEFINES_H */
diff --git a/tools/testing/selftests/sgx/load.c b/tools/testing/selftests/sgx/load.c
index 3ebe5d1..9d4322c 100644
--- a/tools/testing/selftests/sgx/load.c
+++ b/tools/testing/selftests/sgx/load.c
@@ -21,6 +21,8 @@
void encl_delete(struct encl *encl)
{
+ struct encl_segment *heap_seg = &encl->segment_tbl[encl->nr_segments - 1];
+
if (encl->encl_base)
munmap((void *)encl->encl_base, encl->encl_size);
@@ -30,6 +32,8 @@ void encl_delete(struct encl *encl)
if (encl->fd)
close(encl->fd);
+ munmap(heap_seg->src, heap_seg->size);
+
if (encl->segment_tbl)
free(encl->segment_tbl);
@@ -107,11 +111,14 @@ static bool encl_ioc_add_pages(struct encl *encl, struct encl_segment *seg)
memset(&secinfo, 0, sizeof(secinfo));
secinfo.flags = seg->flags;
- ioc.src = (uint64_t)encl->src + seg->offset;
+ ioc.src = (uint64_t)seg->src;
ioc.offset = seg->offset;
ioc.length = seg->size;
ioc.secinfo = (unsigned long)&secinfo;
- ioc.flags = SGX_PAGE_MEASURE;
+ if (seg->measure)
+ ioc.flags = SGX_PAGE_MEASURE;
+ else
+ ioc.flags = 0;
rc = ioctl(encl->fd, SGX_IOC_ENCLAVE_ADD_PAGES, &ioc);
if (rc < 0) {
@@ -122,11 +129,10 @@ static bool encl_ioc_add_pages(struct encl *encl, struct encl_segment *seg)
return true;
}
-
-
-bool encl_load(const char *path, struct encl *encl)
+bool encl_load(const char *path, struct encl *encl, unsigned long heap_size)
{
const char device_path[] = "/dev/sgx_enclave";
+ struct encl_segment *seg;
Elf64_Phdr *phdr_tbl;
off_t src_offset;
Elf64_Ehdr *ehdr;
@@ -178,6 +184,8 @@ bool encl_load(const char *path, struct encl *encl)
ehdr = encl->bin;
phdr_tbl = encl->bin + ehdr->e_phoff;
+ encl->nr_segments = 1; /* one for the heap */
+
for (i = 0; i < ehdr->e_phnum; i++) {
Elf64_Phdr *phdr = &phdr_tbl[i];
@@ -193,7 +201,6 @@ bool encl_load(const char *path, struct encl *encl)
for (i = 0, j = 0; i < ehdr->e_phnum; i++) {
Elf64_Phdr *phdr = &phdr_tbl[i];
unsigned int flags = phdr->p_flags;
- struct encl_segment *seg;
if (phdr->p_type != PT_LOAD)
continue;
@@ -216,6 +223,7 @@ bool encl_load(const char *path, struct encl *encl)
if (j == 0) {
src_offset = phdr->p_offset & PAGE_MASK;
+ encl->src = encl->bin + src_offset;
seg->prot = PROT_READ | PROT_WRITE;
seg->flags = SGX_PAGE_TYPE_TCS << 8;
@@ -228,15 +236,27 @@ bool encl_load(const char *path, struct encl *encl)
seg->offset = (phdr->p_offset & PAGE_MASK) - src_offset;
seg->size = (phdr->p_filesz + PAGE_SIZE - 1) & PAGE_MASK;
+ seg->src = encl->src + seg->offset;
+ seg->measure = true;
j++;
}
- assert(j == encl->nr_segments);
+ assert(j == encl->nr_segments - 1);
- encl->src = encl->bin + src_offset;
- encl->src_size = encl->segment_tbl[j - 1].offset +
- encl->segment_tbl[j - 1].size;
+ seg = &encl->segment_tbl[j];
+ seg->offset = encl->segment_tbl[j - 1].offset + encl->segment_tbl[j - 1].size;
+ seg->size = heap_size;
+ seg->src = mmap(NULL, heap_size, PROT_READ | PROT_WRITE,
+ MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
+ seg->prot = PROT_READ | PROT_WRITE;
+ seg->flags = (SGX_PAGE_TYPE_REG << 8) | seg->prot;
+ seg->measure = false;
+
+ if (seg->src == MAP_FAILED)
+ goto err;
+
+ encl->src_size = encl->segment_tbl[j].offset + encl->segment_tbl[j].size;
for (encl->encl_size = 4096; encl->encl_size < encl->src_size; )
encl->encl_size <<= 1;
diff --git a/tools/testing/selftests/sgx/main.c b/tools/testing/selftests/sgx/main.c
index e252015..370c499 100644
--- a/tools/testing/selftests/sgx/main.c
+++ b/tools/testing/selftests/sgx/main.c
@@ -1,6 +1,7 @@
// SPDX-License-Identifier: GPL-2.0
/* Copyright(c) 2016-20 Intel Corporation. */
+#include <cpuid.h>
#include <elf.h>
#include <errno.h>
#include <fcntl.h>
@@ -21,6 +22,7 @@
#include "main.h"
static const uint64_t MAGIC = 0x1122334455667788ULL;
+static const uint64_t MAGIC2 = 0x8877665544332211ULL;
vdso_sgx_enter_enclave_t vdso_sgx_enter_enclave;
struct vdso_symtab {
@@ -107,12 +109,32 @@ static Elf64_Sym *vdso_symtab_get(struct vdso_symtab *symtab, const char *name)
return NULL;
}
+/*
+ * Return the offset in the enclave where the data segment can be found.
+ * The first RW segment loaded is the TCS, skip that to get info on the
+ * data segment.
+ */
+static off_t encl_get_data_offset(struct encl *encl)
+{
+ int i;
+
+ for (i = 1; i < encl->nr_segments; i++) {
+ struct encl_segment *seg = &encl->segment_tbl[i];
+
+ if (seg->prot == (PROT_READ | PROT_WRITE))
+ return seg->offset;
+ }
+
+ return -1;
+}
+
FIXTURE(enclave) {
struct encl encl;
struct sgx_enclave_run run;
};
-FIXTURE_SETUP(enclave)
+static bool setup_test_encl(unsigned long heap_size, struct encl *encl,
+ struct __test_metadata *_metadata)
{
Elf64_Sym *sgx_enter_enclave_sym = NULL;
struct vdso_symtab symtab;
@@ -122,31 +144,25 @@ FIXTURE_SETUP(enclave)
unsigned int i;
void *addr;
- if (!encl_load("test_encl.elf", &self->encl)) {
- encl_delete(&self->encl);
- ksft_exit_skip("cannot load enclaves\n");
+ if (!encl_load("test_encl.elf", encl, heap_size)) {
+ encl_delete(encl);
+ TH_LOG("Failed to load the test enclave.\n");
}
- for (i = 0; i < self->encl.nr_segments; i++) {
- seg = &self->encl.segment_tbl[i];
-
- TH_LOG("0x%016lx 0x%016lx 0x%02x", seg->offset, seg->size, seg->prot);
- }
-
- if (!encl_measure(&self->encl))
+ if (!encl_measure(encl))
goto err;
- if (!encl_build(&self->encl))
+ if (!encl_build(encl))
goto err;
/*
* An enclave consumer only must do this.
*/
- for (i = 0; i < self->encl.nr_segments; i++) {
- struct encl_segment *seg = &self->encl.segment_tbl[i];
+ for (i = 0; i < encl->nr_segments; i++) {
+ struct encl_segment *seg = &encl->segment_tbl[i];
- addr = mmap((void *)self->encl.encl_base + seg->offset, seg->size,
- seg->prot, MAP_SHARED | MAP_FIXED, self->encl.fd, 0);
+ addr = mmap((void *)encl->encl_base + seg->offset, seg->size,
+ seg->prot, MAP_SHARED | MAP_FIXED, encl->fd, 0);
EXPECT_NE(addr, MAP_FAILED);
if (addr == MAP_FAILED)
goto err;
@@ -166,8 +182,16 @@ FIXTURE_SETUP(enclave)
vdso_sgx_enter_enclave = addr + sgx_enter_enclave_sym->st_value;
- memset(&self->run, 0, sizeof(self->run));
- self->run.tcs = self->encl.encl_base;
+ return true;
+
+err:
+ encl_delete(encl);
+
+ for (i = 0; i < encl->nr_segments; i++) {
+ seg = &encl->segment_tbl[i];
+
+ TH_LOG("0x%016lx 0x%016lx 0x%02x", seg->offset, seg->size, seg->prot);
+ }
maps_file = fopen("/proc/self/maps", "r");
if (maps_file != NULL) {
@@ -181,11 +205,13 @@ FIXTURE_SETUP(enclave)
fclose(maps_file);
}
-err:
- if (!sgx_enter_enclave_sym)
- encl_delete(&self->encl);
+ TH_LOG("Failed to initialize the test enclave.\n");
- ASSERT_NE(sgx_enter_enclave_sym, NULL);
+ return false;
+}
+
+FIXTURE_SETUP(enclave)
+{
}
FIXTURE_TEARDOWN(enclave)
@@ -215,44 +241,130 @@ FIXTURE_TEARDOWN(enclave)
TEST_F(enclave, unclobbered_vdso)
{
- struct encl_op op;
+ struct encl_op_get_from_buf get_op;
+ struct encl_op_put_to_buf put_op;
- op.type = ENCL_OP_PUT;
- op.buffer = MAGIC;
+ ASSERT_TRUE(setup_test_encl(ENCL_HEAP_SIZE_DEFAULT, &self->encl, _metadata));
- EXPECT_EQ(ENCL_CALL(&op, &self->run, false), 0);
+ memset(&self->run, 0, sizeof(self->run));
+ self->run.tcs = self->encl.encl_base;
+
+ put_op.header.type = ENCL_OP_PUT_TO_BUFFER;
+ put_op.value = MAGIC;
+
+ EXPECT_EQ(ENCL_CALL(&put_op, &self->run, false), 0);
EXPECT_EEXIT(&self->run);
EXPECT_EQ(self->run.user_data, 0);
- op.type = ENCL_OP_GET;
- op.buffer = 0;
+ get_op.header.type = ENCL_OP_GET_FROM_BUFFER;
+ get_op.value = 0;
- EXPECT_EQ(ENCL_CALL(&op, &self->run, false), 0);
+ EXPECT_EQ(ENCL_CALL(&get_op, &self->run, false), 0);
- EXPECT_EQ(op.buffer, MAGIC);
+ EXPECT_EQ(get_op.value, MAGIC);
EXPECT_EEXIT(&self->run);
EXPECT_EQ(self->run.user_data, 0);
}
-TEST_F(enclave, clobbered_vdso)
+/*
+ * A section metric is concatenated in a way that @low bits 12-31 define the
+ * bits 12-31 of the metric and @high bits 0-19 define the bits 32-51 of the
+ * metric.
+ */
+static unsigned long sgx_calc_section_metric(unsigned int low,
+ unsigned int high)
{
- struct encl_op op;
+ return (low & GENMASK_ULL(31, 12)) +
+ ((high & GENMASK_ULL(19, 0)) << 32);
+}
- op.type = ENCL_OP_PUT;
- op.buffer = MAGIC;
+/*
+ * Sum total available physical SGX memory across all EPC sections
+ *
+ * Return: total available physical SGX memory available on system
+ */
+static unsigned long get_total_epc_mem(void)
+{
+ unsigned int eax, ebx, ecx, edx;
+ unsigned long total_size = 0;
+ unsigned int type;
+ int section = 0;
- EXPECT_EQ(ENCL_CALL(&op, &self->run, true), 0);
+ while (true) {
+ __cpuid_count(SGX_CPUID, section + SGX_CPUID_EPC, eax, ebx, ecx, edx);
+
+ type = eax & SGX_CPUID_EPC_MASK;
+ if (type == SGX_CPUID_EPC_INVALID)
+ break;
+
+ if (type != SGX_CPUID_EPC_SECTION)
+ break;
+
+ total_size += sgx_calc_section_metric(ecx, edx);
+
+ section++;
+ }
+
+ return total_size;
+}
+
+TEST_F(enclave, unclobbered_vdso_oversubscribed)
+{
+ struct encl_op_get_from_buf get_op;
+ struct encl_op_put_to_buf put_op;
+ unsigned long total_mem;
+
+ total_mem = get_total_epc_mem();
+ ASSERT_NE(total_mem, 0);
+ ASSERT_TRUE(setup_test_encl(total_mem, &self->encl, _metadata));
+
+ memset(&self->run, 0, sizeof(self->run));
+ self->run.tcs = self->encl.encl_base;
+
+ put_op.header.type = ENCL_OP_PUT_TO_BUFFER;
+ put_op.value = MAGIC;
+
+ EXPECT_EQ(ENCL_CALL(&put_op, &self->run, false), 0);
EXPECT_EEXIT(&self->run);
EXPECT_EQ(self->run.user_data, 0);
- op.type = ENCL_OP_GET;
- op.buffer = 0;
+ get_op.header.type = ENCL_OP_GET_FROM_BUFFER;
+ get_op.value = 0;
- EXPECT_EQ(ENCL_CALL(&op, &self->run, true), 0);
+ EXPECT_EQ(ENCL_CALL(&get_op, &self->run, false), 0);
- EXPECT_EQ(op.buffer, MAGIC);
+ EXPECT_EQ(get_op.value, MAGIC);
+ EXPECT_EEXIT(&self->run);
+ EXPECT_EQ(self->run.user_data, 0);
+
+}
+
+TEST_F(enclave, clobbered_vdso)
+{
+ struct encl_op_get_from_buf get_op;
+ struct encl_op_put_to_buf put_op;
+
+ ASSERT_TRUE(setup_test_encl(ENCL_HEAP_SIZE_DEFAULT, &self->encl, _metadata));
+
+ memset(&self->run, 0, sizeof(self->run));
+ self->run.tcs = self->encl.encl_base;
+
+ put_op.header.type = ENCL_OP_PUT_TO_BUFFER;
+ put_op.value = MAGIC;
+
+ EXPECT_EQ(ENCL_CALL(&put_op, &self->run, true), 0);
+
+ EXPECT_EEXIT(&self->run);
+ EXPECT_EQ(self->run.user_data, 0);
+
+ get_op.header.type = ENCL_OP_GET_FROM_BUFFER;
+ get_op.value = 0;
+
+ EXPECT_EQ(ENCL_CALL(&get_op, &self->run, true), 0);
+
+ EXPECT_EQ(get_op.value, MAGIC);
EXPECT_EEXIT(&self->run);
EXPECT_EQ(self->run.user_data, 0);
}
@@ -267,27 +379,179 @@ static int test_handler(long rdi, long rsi, long rdx, long ursp, long r8, long r
TEST_F(enclave, clobbered_vdso_and_user_function)
{
- struct encl_op op;
+ struct encl_op_get_from_buf get_op;
+ struct encl_op_put_to_buf put_op;
+
+ ASSERT_TRUE(setup_test_encl(ENCL_HEAP_SIZE_DEFAULT, &self->encl, _metadata));
+
+ memset(&self->run, 0, sizeof(self->run));
+ self->run.tcs = self->encl.encl_base;
self->run.user_handler = (__u64)test_handler;
self->run.user_data = 0xdeadbeef;
- op.type = ENCL_OP_PUT;
- op.buffer = MAGIC;
+ put_op.header.type = ENCL_OP_PUT_TO_BUFFER;
+ put_op.value = MAGIC;
- EXPECT_EQ(ENCL_CALL(&op, &self->run, true), 0);
+ EXPECT_EQ(ENCL_CALL(&put_op, &self->run, true), 0);
EXPECT_EEXIT(&self->run);
EXPECT_EQ(self->run.user_data, 0);
- op.type = ENCL_OP_GET;
- op.buffer = 0;
+ get_op.header.type = ENCL_OP_GET_FROM_BUFFER;
+ get_op.value = 0;
+
+ EXPECT_EQ(ENCL_CALL(&get_op, &self->run, true), 0);
+
+ EXPECT_EQ(get_op.value, MAGIC);
+ EXPECT_EEXIT(&self->run);
+ EXPECT_EQ(self->run.user_data, 0);
+}
+
+/*
+ * Sanity check that it is possible to enter either of the two hardcoded TCS
+ */
+TEST_F(enclave, tcs_entry)
+{
+ struct encl_op_header op;
+
+ ASSERT_TRUE(setup_test_encl(ENCL_HEAP_SIZE_DEFAULT, &self->encl, _metadata));
+
+ memset(&self->run, 0, sizeof(self->run));
+ self->run.tcs = self->encl.encl_base;
+
+ op.type = ENCL_OP_NOP;
EXPECT_EQ(ENCL_CALL(&op, &self->run, true), 0);
- EXPECT_EQ(op.buffer, MAGIC);
EXPECT_EEXIT(&self->run);
- EXPECT_EQ(self->run.user_data, 0);
+ EXPECT_EQ(self->run.exception_vector, 0);
+ EXPECT_EQ(self->run.exception_error_code, 0);
+ EXPECT_EQ(self->run.exception_addr, 0);
+
+ /* Move to the next TCS. */
+ self->run.tcs = self->encl.encl_base + PAGE_SIZE;
+
+ EXPECT_EQ(ENCL_CALL(&op, &self->run, true), 0);
+
+ EXPECT_EEXIT(&self->run);
+ EXPECT_EQ(self->run.exception_vector, 0);
+ EXPECT_EQ(self->run.exception_error_code, 0);
+ EXPECT_EQ(self->run.exception_addr, 0);
+}
+
+/*
+ * Second page of .data segment is used to test changing PTE permissions.
+ * This spans the local encl_buffer within the test enclave.
+ *
+ * 1) Start with a sanity check: a value is written to the target page within
+ * the enclave and read back to ensure target page can be written to.
+ * 2) Change PTE permissions (RW -> RO) of target page within enclave.
+ * 3) Repeat (1) - this time expecting a regular #PF communicated via the
+ * vDSO.
+ * 4) Change PTE permissions of target page within enclave back to be RW.
+ * 5) Repeat (1) by resuming enclave, now expected to be possible to write to
+ * and read from target page within enclave.
+ */
+TEST_F(enclave, pte_permissions)
+{
+ struct encl_op_get_from_addr get_addr_op;
+ struct encl_op_put_to_addr put_addr_op;
+ unsigned long data_start;
+ int ret;
+
+ ASSERT_TRUE(setup_test_encl(ENCL_HEAP_SIZE_DEFAULT, &self->encl, _metadata));
+
+ memset(&self->run, 0, sizeof(self->run));
+ self->run.tcs = self->encl.encl_base;
+
+ data_start = self->encl.encl_base +
+ encl_get_data_offset(&self->encl) +
+ PAGE_SIZE;
+
+ /*
+ * Sanity check to ensure it is possible to write to page that will
+ * have its permissions manipulated.
+ */
+
+ /* Write MAGIC to page */
+ put_addr_op.value = MAGIC;
+ put_addr_op.addr = data_start;
+ put_addr_op.header.type = ENCL_OP_PUT_TO_ADDRESS;
+
+ EXPECT_EQ(ENCL_CALL(&put_addr_op, &self->run, true), 0);
+
+ EXPECT_EEXIT(&self->run);
+ EXPECT_EQ(self->run.exception_vector, 0);
+ EXPECT_EQ(self->run.exception_error_code, 0);
+ EXPECT_EQ(self->run.exception_addr, 0);
+
+ /*
+ * Read memory that was just written to, confirming that it is the
+ * value previously written (MAGIC).
+ */
+ get_addr_op.value = 0;
+ get_addr_op.addr = data_start;
+ get_addr_op.header.type = ENCL_OP_GET_FROM_ADDRESS;
+
+ EXPECT_EQ(ENCL_CALL(&get_addr_op, &self->run, true), 0);
+
+ EXPECT_EQ(get_addr_op.value, MAGIC);
+ EXPECT_EEXIT(&self->run);
+ EXPECT_EQ(self->run.exception_vector, 0);
+ EXPECT_EQ(self->run.exception_error_code, 0);
+ EXPECT_EQ(self->run.exception_addr, 0);
+
+ /* Change PTE permissions of target page within the enclave */
+ ret = mprotect((void *)data_start, PAGE_SIZE, PROT_READ);
+ if (ret)
+ perror("mprotect");
+
+ /*
+ * PTE permissions of target page changed to read-only, EPCM
+ * permissions unchanged (EPCM permissions are RW), attempt to
+ * write to the page, expecting a regular #PF.
+ */
+
+ put_addr_op.value = MAGIC2;
+
+ EXPECT_EQ(ENCL_CALL(&put_addr_op, &self->run, true), 0);
+
+ EXPECT_EQ(self->run.exception_vector, 14);
+ EXPECT_EQ(self->run.exception_error_code, 0x7);
+ EXPECT_EQ(self->run.exception_addr, data_start);
+
+ self->run.exception_vector = 0;
+ self->run.exception_error_code = 0;
+ self->run.exception_addr = 0;
+
+ /*
+ * Change PTE permissions back to enable enclave to write to the
+ * target page and resume enclave - do not expect any exceptions this
+ * time.
+ */
+ ret = mprotect((void *)data_start, PAGE_SIZE, PROT_READ | PROT_WRITE);
+ if (ret)
+ perror("mprotect");
+
+ EXPECT_EQ(vdso_sgx_enter_enclave((unsigned long)&put_addr_op, 0,
+ 0, ERESUME, 0, 0, &self->run),
+ 0);
+
+ EXPECT_EEXIT(&self->run);
+ EXPECT_EQ(self->run.exception_vector, 0);
+ EXPECT_EQ(self->run.exception_error_code, 0);
+ EXPECT_EQ(self->run.exception_addr, 0);
+
+ get_addr_op.value = 0;
+
+ EXPECT_EQ(ENCL_CALL(&get_addr_op, &self->run, true), 0);
+
+ EXPECT_EQ(get_addr_op.value, MAGIC2);
+ EXPECT_EEXIT(&self->run);
+ EXPECT_EQ(self->run.exception_vector, 0);
+ EXPECT_EQ(self->run.exception_error_code, 0);
+ EXPECT_EQ(self->run.exception_addr, 0);
}
TEST_HARNESS_MAIN
diff --git a/tools/testing/selftests/sgx/main.h b/tools/testing/selftests/sgx/main.h
index 68672fd..b45c52e 100644
--- a/tools/testing/selftests/sgx/main.h
+++ b/tools/testing/selftests/sgx/main.h
@@ -6,11 +6,15 @@
#ifndef MAIN_H
#define MAIN_H
+#define ENCL_HEAP_SIZE_DEFAULT 4096
+
struct encl_segment {
+ void *src;
off_t offset;
size_t size;
unsigned int prot;
unsigned int flags;
+ bool measure;
};
struct encl {
@@ -31,7 +35,7 @@ extern unsigned char sign_key[];
extern unsigned char sign_key_end[];
void encl_delete(struct encl *ctx);
-bool encl_load(const char *path, struct encl *encl);
+bool encl_load(const char *path, struct encl *encl, unsigned long heap_size);
bool encl_measure(struct encl *encl);
bool encl_build(struct encl *encl);
diff --git a/tools/testing/selftests/sgx/sigstruct.c b/tools/testing/selftests/sgx/sigstruct.c
index 92bbc5a..50c5ab1 100644
--- a/tools/testing/selftests/sgx/sigstruct.c
+++ b/tools/testing/selftests/sgx/sigstruct.c
@@ -289,15 +289,17 @@ static bool mrenclave_eextend(EVP_MD_CTX *ctx, uint64_t offset,
static bool mrenclave_segment(EVP_MD_CTX *ctx, struct encl *encl,
struct encl_segment *seg)
{
- uint64_t end = seg->offset + seg->size;
+ uint64_t end = seg->size;
uint64_t offset;
- for (offset = seg->offset; offset < end; offset += PAGE_SIZE) {
- if (!mrenclave_eadd(ctx, offset, seg->flags))
+ for (offset = 0; offset < end; offset += PAGE_SIZE) {
+ if (!mrenclave_eadd(ctx, seg->offset + offset, seg->flags))
return false;
- if (!mrenclave_eextend(ctx, offset, encl->src + offset))
- return false;
+ if (seg->measure) {
+ if (!mrenclave_eextend(ctx, seg->offset + offset, seg->src + offset))
+ return false;
+ }
}
return true;
diff --git a/tools/testing/selftests/sgx/test_encl.c b/tools/testing/selftests/sgx/test_encl.c
index 734ea52..4fca01c 100644
--- a/tools/testing/selftests/sgx/test_encl.c
+++ b/tools/testing/selftests/sgx/test_encl.c
@@ -4,6 +4,11 @@
#include <stddef.h>
#include "defines.h"
+/*
+ * Data buffer spanning two pages that will be placed first in .data
+ * segment. Even if not used internally the second page is needed by
+ * external test manipulating page permissions.
+ */
static uint8_t encl_buffer[8192] = { 1 };
static void *memcpy(void *dest, const void *src, size_t n)
@@ -16,20 +21,51 @@ static void *memcpy(void *dest, const void *src, size_t n)
return dest;
}
+static void do_encl_op_put_to_buf(void *op)
+{
+ struct encl_op_put_to_buf *op2 = op;
+
+ memcpy(&encl_buffer[0], &op2->value, 8);
+}
+
+static void do_encl_op_get_from_buf(void *op)
+{
+ struct encl_op_get_from_buf *op2 = op;
+
+ memcpy(&op2->value, &encl_buffer[0], 8);
+}
+
+static void do_encl_op_put_to_addr(void *_op)
+{
+ struct encl_op_put_to_addr *op = _op;
+
+ memcpy((void *)op->addr, &op->value, 8);
+}
+
+static void do_encl_op_get_from_addr(void *_op)
+{
+ struct encl_op_get_from_addr *op = _op;
+
+ memcpy(&op->value, (void *)op->addr, 8);
+}
+
+static void do_encl_op_nop(void *_op)
+{
+
+}
+
void encl_body(void *rdi, void *rsi)
{
- struct encl_op *op = (struct encl_op *)rdi;
+ const void (*encl_op_array[ENCL_OP_MAX])(void *) = {
+ do_encl_op_put_to_buf,
+ do_encl_op_get_from_buf,
+ do_encl_op_put_to_addr,
+ do_encl_op_get_from_addr,
+ do_encl_op_nop,
+ };
- switch (op->type) {
- case ENCL_OP_PUT:
- memcpy(&encl_buffer[0], &op->buffer, 8);
- break;
+ struct encl_op_header *op = (struct encl_op_header *)rdi;
- case ENCL_OP_GET:
- memcpy(&op->buffer, &encl_buffer[0], 8);
- break;
-
- default:
- break;
- }
+ if (op->type < ENCL_OP_MAX)
+ (*encl_op_array[op->type])(op);
}
diff --git a/tools/testing/selftests/sgx/test_encl_bootstrap.S b/tools/testing/selftests/sgx/test_encl_bootstrap.S
index 5d5680d..82fb0df 100644
--- a/tools/testing/selftests/sgx/test_encl_bootstrap.S
+++ b/tools/testing/selftests/sgx/test_encl_bootstrap.S
@@ -12,7 +12,7 @@
.fill 1, 8, 0 # STATE (set by CPU)
.fill 1, 8, 0 # FLAGS
- .quad encl_ssa # OSSA
+ .quad encl_ssa_tcs1 # OSSA
.fill 1, 4, 0 # CSSA (set by CPU)
.fill 1, 4, 1 # NSSA
.quad encl_entry # OENTRY
@@ -23,10 +23,10 @@
.fill 1, 4, 0xFFFFFFFF # GSLIMIT
.fill 4024, 1, 0 # Reserved
- # Identical to the previous TCS.
+ # TCS2
.fill 1, 8, 0 # STATE (set by CPU)
.fill 1, 8, 0 # FLAGS
- .quad encl_ssa # OSSA
+ .quad encl_ssa_tcs2 # OSSA
.fill 1, 4, 0 # CSSA (set by CPU)
.fill 1, 4, 1 # NSSA
.quad encl_entry # OENTRY
@@ -40,8 +40,9 @@
.text
encl_entry:
- # RBX contains the base address for TCS, which is also the first address
- # inside the enclave. By adding the value of le_stack_end to it, we get
+ # RBX contains the base address for TCS, which is the first address
+ # inside the enclave for TCS #1 and one page into the enclave for
+ # TCS #2. By adding the value of encl_stack to it, we get
# the absolute address for the stack.
lea (encl_stack)(%rbx), %rax
xchg %rsp, %rax
@@ -81,9 +82,15 @@
.section ".data", "aw"
-encl_ssa:
+encl_ssa_tcs1:
+ .space 4096
+encl_ssa_tcs2:
.space 4096
.balign 4096
- .space 8192
+ # Stack of TCS #1
+ .space 4096
encl_stack:
+ .balign 4096
+ # Stack of TCS #2
+ .space 4096
diff --git a/tools/testing/selftests/sparc64/drivers/adi-test.c b/tools/testing/selftests/sparc64/drivers/adi-test.c
index 95d93c6..84e5d9f 100644
--- a/tools/testing/selftests/sparc64/drivers/adi-test.c
+++ b/tools/testing/selftests/sparc64/drivers/adi-test.c
@@ -24,10 +24,6 @@
#define DEBUG_LEVEL_4_BIT (0x0008)
#define DEBUG_TIMING_BIT (0x1000)
-#ifndef ARRAY_SIZE
-# define ARRAY_SIZE(x) (sizeof(x) / sizeof((x)[0]))
-#endif
-
/* bit mask of enabled bits to print */
#define DEBUG 0x0001
diff --git a/tools/testing/selftests/tc-testing/tc-tests/actions/police.json b/tools/testing/selftests/tc-testing/tc-tests/actions/police.json
index 8e45792..b7205a0 100644
--- a/tools/testing/selftests/tc-testing/tc-tests/actions/police.json
+++ b/tools/testing/selftests/tc-testing/tc-tests/actions/police.json
@@ -812,5 +812,29 @@
"teardown": [
"$TC actions flush action police"
]
+ },
+ {
+ "id": "7d64",
+ "name": "Add police action with skip_hw option",
+ "category": [
+ "actions",
+ "police"
+ ],
+ "setup": [
+ [
+ "$TC actions flush action police",
+ 0,
+ 1,
+ 255
+ ]
+ ],
+ "cmdUnderTest": "$TC actions add action police rate 1kbit burst 10k index 100 skip_hw",
+ "expExitCode": "0",
+ "verifyCmd": "$TC actions ls action police | grep skip_hw",
+ "matchPattern": "skip_hw",
+ "matchCount": "1",
+ "teardown": [
+ "$TC actions flush action police"
+ ]
}
]
diff --git a/tools/testing/selftests/tc-testing/tc-tests/filters/matchall.json b/tools/testing/selftests/tc-testing/tc-tests/filters/matchall.json
index 5179987..2df6801 100644
--- a/tools/testing/selftests/tc-testing/tc-tests/filters/matchall.json
+++ b/tools/testing/selftests/tc-testing/tc-tests/filters/matchall.json
@@ -387,5 +387,77 @@
"$TC qdisc del dev $DUMMY ingress",
"$IP link del dev $DUMMY type dummy"
]
+ },
+ {
+ "id": "3329",
+ "name": "Validate flags of the matchall filter with skip_sw and police action with skip_hw",
+ "category": [
+ "filter",
+ "matchall"
+ ],
+ "setup": [
+ "$IP link add dev $DUMMY type dummy || /bin/true",
+ "$TC qdisc add dev $DUMMY ingress",
+ "$TC actions flush action police",
+ "$TC actions add action police rate 1mbit burst 100k index 199 skip_hw"
+ ],
+ "cmdUnderTest": "$TC filter add dev $DUMMY parent ffff: handle 0x1 prio 1 protocol ipv4 matchall skip_sw action police index 199",
+ "expExitCode": "2",
+ "verifyCmd": "$TC filter get dev $DUMMY parent ffff: handle 1 prio 1 protocol ipv4 matchall",
+ "matchPattern": "^filter parent ffff: protocol ip pref 1 matchall.*handle 0x1.*",
+ "matchCount": "0",
+ "teardown": [
+ "$TC qdisc del dev $DUMMY ingress",
+ "$IP link del dev $DUMMY type dummy",
+ "$TC actions del action police index 199"
+ ]
+ },
+ {
+ "id": "0eeb",
+ "name": "Validate flags of the matchall filter with skip_hw and police action",
+ "category": [
+ "filter",
+ "matchall"
+ ],
+ "setup": [
+ "$IP link add dev $DUMMY type dummy || /bin/true",
+ "$TC qdisc add dev $DUMMY ingress",
+ "$TC actions flush action police",
+ "$TC actions add action police rate 1mbit burst 100k index 199"
+ ],
+ "cmdUnderTest": "$TC filter add dev $DUMMY parent ffff: handle 0x1 prio 1 protocol ipv4 matchall skip_hw action police index 199",
+ "expExitCode": "2",
+ "verifyCmd": "$TC filter get dev $DUMMY parent ffff: handle 1 prio 1 protocol ipv4 matchall",
+ "matchPattern": "^filter parent ffff: protocol ip pref 1 matchall.*handle 0x1.*",
+ "matchCount": "0",
+ "teardown": [
+ "$TC qdisc del dev $DUMMY ingress",
+ "$IP link del dev $DUMMY type dummy",
+ "$TC actions del action police index 199"
+ ]
+ },
+ {
+ "id": "eee4",
+ "name": "Validate flags of the matchall filter with skip_sw and police action",
+ "category": [
+ "filter",
+ "matchall"
+ ],
+ "setup": [
+ "$IP link add dev $DUMMY type dummy || /bin/true",
+ "$TC qdisc add dev $DUMMY ingress",
+ "$TC actions flush action police",
+ "$TC actions add action police rate 1mbit burst 100k index 199"
+ ],
+ "cmdUnderTest": "$TC filter add dev $DUMMY parent ffff: handle 0x1 prio 1 protocol ipv4 matchall skip_sw action police index 199",
+ "expExitCode": "2",
+ "verifyCmd": "$TC filter get dev $DUMMY parent ffff: handle 1 prio 1 protocol ipv4 matchall",
+ "matchPattern": "^filter parent ffff: protocol ip pref 1 matchall.*handle 0x1.*",
+ "matchCount": "0",
+ "teardown": [
+ "$TC qdisc del dev $DUMMY ingress",
+ "$IP link del dev $DUMMY type dummy",
+ "$TC actions del action police index 199"
+ ]
}
]
diff --git a/tools/testing/selftests/timens/procfs.c b/tools/testing/selftests/timens/procfs.c
index f251915..1833ca9 100644
--- a/tools/testing/selftests/timens/procfs.c
+++ b/tools/testing/selftests/timens/procfs.c
@@ -24,8 +24,6 @@
#define DAY_IN_SEC (60*60*24)
#define TEN_DAYS_IN_SEC (10*DAY_IN_SEC)
-#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]))
-
static int child_ns, parent_ns;
static int switch_ns(int fd)
diff --git a/tools/testing/selftests/timens/timens.c b/tools/testing/selftests/timens/timens.c
index 52b6a11..3872207 100644
--- a/tools/testing/selftests/timens/timens.c
+++ b/tools/testing/selftests/timens/timens.c
@@ -22,8 +22,6 @@
#define DAY_IN_SEC (60*60*24)
#define TEN_DAYS_IN_SEC (10*DAY_IN_SEC)
-#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]))
-
struct test_clock {
clockid_t id;
char *name;
diff --git a/tools/testing/selftests/timers/alarmtimer-suspend.c b/tools/testing/selftests/timers/alarmtimer-suspend.c
index 4da09db..54da4b08 100644
--- a/tools/testing/selftests/timers/alarmtimer-suspend.c
+++ b/tools/testing/selftests/timers/alarmtimer-suspend.c
@@ -79,7 +79,7 @@ char *clockstring(int clockid)
return "CLOCK_BOOTTIME_ALARM";
case CLOCK_TAI:
return "CLOCK_TAI";
- };
+ }
return "UNKNOWN_CLOCKID";
}
diff --git a/tools/testing/selftests/timers/inconsistency-check.c b/tools/testing/selftests/timers/inconsistency-check.c
index 022d3ff..e6756d9 100644
--- a/tools/testing/selftests/timers/inconsistency-check.c
+++ b/tools/testing/selftests/timers/inconsistency-check.c
@@ -72,7 +72,7 @@ char *clockstring(int clockid)
return "CLOCK_BOOTTIME_ALARM";
case CLOCK_TAI:
return "CLOCK_TAI";
- };
+ }
return "UNKNOWN_CLOCKID";
}
diff --git a/tools/testing/selftests/vm/mremap_test.c b/tools/testing/selftests/vm/mremap_test.c
index 0624d1b..7c0b061 100644
--- a/tools/testing/selftests/vm/mremap_test.c
+++ b/tools/testing/selftests/vm/mremap_test.c
@@ -20,7 +20,6 @@
#define VALIDATION_DEFAULT_THRESHOLD 4 /* 4MB */
#define VALIDATION_NO_THRESHOLD 0 /* Verify the entire region */
-#define ARRAY_SIZE(x) (sizeof(x) / sizeof((x)[0]))
#define MIN(X, Y) ((X) < (Y) ? (X) : (Y))
struct config {
diff --git a/tools/testing/selftests/vm/pkey-helpers.h b/tools/testing/selftests/vm/pkey-helpers.h
index 622a858..92f3be3 100644
--- a/tools/testing/selftests/vm/pkey-helpers.h
+++ b/tools/testing/selftests/vm/pkey-helpers.h
@@ -13,6 +13,8 @@
#include <ucontext.h>
#include <sys/mman.h>
+#include "../kselftest.h"
+
/* Define some kernel-like types */
#define u8 __u8
#define u16 __u16
@@ -175,7 +177,6 @@ static inline void __pkey_write_allow(int pkey, int do_allow_write)
dprintf4("pkey_reg now: %016llx\n", read_pkey_reg());
}
-#define ARRAY_SIZE(x) (sizeof(x) / sizeof(*(x)))
#define ALIGN_UP(x, align_to) (((x) + ((align_to)-1)) & ~((align_to)-1))
#define ALIGN_DOWN(x, align_to) ((x) & ~((align_to)-1))
#define ALIGN_PTR_UP(p, ptr_align_to) \
diff --git a/tools/testing/selftests/vm/userfaultfd.c b/tools/testing/selftests/vm/userfaultfd.c
index 8a09057..9354a5e 100644
--- a/tools/testing/selftests/vm/userfaultfd.c
+++ b/tools/testing/selftests/vm/userfaultfd.c
@@ -87,7 +87,7 @@ static bool test_uffdio_minor = false;
static bool map_shared;
static int shm_fd;
-static int huge_fd;
+static int huge_fd = -1; /* only used for hugetlb_shared test */
static char *huge_fd_off0;
static unsigned long long *count_verify;
static int uffd = -1;
@@ -223,6 +223,9 @@ static void noop_alias_mapping(__u64 *start, size_t len, unsigned long offset)
static void hugetlb_release_pages(char *rel_area)
{
+ if (huge_fd == -1)
+ return;
+
if (fallocate(huge_fd, FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE,
rel_area == huge_fd_off0 ? 0 : nr_pages * page_size,
nr_pages * page_size))
@@ -235,16 +238,17 @@ static void hugetlb_allocate_area(void **alloc_area)
char **alloc_area_alias;
*alloc_area = mmap(NULL, nr_pages * page_size, PROT_READ | PROT_WRITE,
- (map_shared ? MAP_SHARED : MAP_PRIVATE) |
- MAP_HUGETLB,
- huge_fd, *alloc_area == area_src ? 0 :
- nr_pages * page_size);
+ map_shared ? MAP_SHARED :
+ MAP_PRIVATE | MAP_HUGETLB |
+ (*alloc_area == area_src ? 0 : MAP_NORESERVE),
+ huge_fd,
+ *alloc_area == area_src ? 0 : nr_pages * page_size);
if (*alloc_area == MAP_FAILED)
err("mmap of hugetlbfs file failed");
if (map_shared) {
area_alias = mmap(NULL, nr_pages * page_size, PROT_READ | PROT_WRITE,
- MAP_SHARED | MAP_HUGETLB,
+ MAP_SHARED,
huge_fd, *alloc_area == area_src ? 0 :
nr_pages * page_size);
if (area_alias == MAP_FAILED)
diff --git a/tools/testing/selftests/vm/va_128TBswitch.c b/tools/testing/selftests/vm/va_128TBswitch.c
index 83acdff..da6ec3b 100644
--- a/tools/testing/selftests/vm/va_128TBswitch.c
+++ b/tools/testing/selftests/vm/va_128TBswitch.c
@@ -9,7 +9,7 @@
#include <sys/mman.h>
#include <string.h>
-#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]))
+#include "../kselftest.h"
#ifdef __powerpc64__
#define PAGE_SIZE (64 << 10)
